2023-09-10T00:18:14.789 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-10T00:18:14.790 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526 --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} --name pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526/orig.config.yaml 2023-09-10T00:18:14.850 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-10T00:18:14.940 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526 branch: wip-batrick-testing-20230909.145638 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} email: pdonnell@redhat.com first_in_suite: false job_id: '7392526' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-batrick-testing-20230909.145638 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 9670cca5bc8e17916865dc84939edfc10b1f2c00 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 9670cca5bc8e17916865dc84939edfc10b1f2c00 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20230909.145638 sha1: 9670cca5bc8e17916865dc84939edfc10b1f2c00 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5964 sha1: 9670cca5bc8e17916865dc84939edfc10b1f2c00 sleep_before_teardown: 0 subset: 225/256 suite: fs suite_branch: wip-batrick-testing-20230909.145638 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9670cca5bc8e17916865dc84939edfc10b1f2c00 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 500 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-09_19:00:20 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1394 2023-09-10T00:18:14.940 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa; will attempt to use it 2023-09-10T00:18:14.941 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks 2023-09-10T00:18:14.941 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-10T00:18:14.943 INFO:teuthology.task.internal:Checking packages... 2023-09-10T00:18:14.966 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '9670cca5bc8e17916865dc84939edfc10b1f2c00' 2023-09-10T00:18:14.966 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-10T00:18:14.967 INFO:teuthology.packaging:ref: None 2023-09-10T00:18:14.967 INFO:teuthology.packaging:tag: None 2023-09-10T00:18:14.967 INFO:teuthology.packaging:branch: wip-batrick-testing-20230909.145638 2023-09-10T00:18:14.968 INFO:teuthology.packaging:sha1: 9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-10T00:18:14.968 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-batrick-testing-20230909.145638 2023-09-10T00:18:15.202 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6025.g9670cca5 2023-09-10T00:18:15.203 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-10T00:18:15.210 INFO:teuthology.task.internal:no buildpackages task found 2023-09-10T00:18:15.211 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-10T00:18:15.222 INFO:teuthology.task.internal:Saving configuration 2023-09-10T00:18:15.243 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-10T00:18:15.272 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-10T00:18:15.299 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-10 00:13:07.006153', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-10T00:18:15.322 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-10 00:13:07.008015', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-10T00:18:15.343 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-10 00:13:07.009325', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-10T00:18:15.344 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-10T00:18:15.350 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-09-10T00:18:15.350 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-09-10T00:18:15.350 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-09-10T00:18:15.351 INFO:teuthology.run_tasks:Running task console_log... 2023-09-10T00:18:15.452 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5a28534d30>, signals=[15]) 2023-09-10T00:18:15.453 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-10T00:18:15.460 INFO:teuthology.task.internal:Opening connections... 2023-09-10T00:18:15.460 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2023-09-10T00:18:15.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-10T00:18:15.531 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2023-09-10T00:18:15.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-10T00:18:15.599 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2023-09-10T00:18:15.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-10T00:18:15.666 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-10T00:18:15.673 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2023-09-10T00:18:15.698 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2023-09-10T00:18:15.699 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2023-09-10T00:18:15.758 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="Red Hat Enterprise Linux" 2023-09-10T00:18:15.758 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="8.6 (Ootpa)" 2023-09-10T00:18:15.758 INFO:teuthology.orchestra.run.smithi044.stdout:ID="rhel" 2023-09-10T00:18:15.759 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="fedora" 2023-09-10T00:18:15.759 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="8.6" 2023-09-10T00:18:15.759 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el8" 2023-09-10T00:18:15.759 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-10T00:18:15.760 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2023-09-10T00:18:15.760 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-10T00:18:15.760 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://www.redhat.com/" 2023-09-10T00:18:15.760 INFO:teuthology.orchestra.run.smithi044.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-10T00:18:15.761 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-10T00:18:15.761 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:18:15.761 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-10T00:18:15.761 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-10T00:18:15.761 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-10T00:18:15.762 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-10T00:18:15.762 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2023-09-10T00:18:15.784 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2023-09-10T00:18:15.804 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2023-09-10T00:18:15.804 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2023-09-10T00:18:15.863 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="Red Hat Enterprise Linux" 2023-09-10T00:18:15.864 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="8.6 (Ootpa)" 2023-09-10T00:18:15.864 INFO:teuthology.orchestra.run.smithi099.stdout:ID="rhel" 2023-09-10T00:18:15.864 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="fedora" 2023-09-10T00:18:15.864 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="8.6" 2023-09-10T00:18:15.865 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el8" 2023-09-10T00:18:15.865 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-10T00:18:15.865 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2023-09-10T00:18:15.865 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-10T00:18:15.866 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://www.redhat.com/" 2023-09-10T00:18:15.866 INFO:teuthology.orchestra.run.smithi099.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-10T00:18:15.866 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-10T00:18:15.866 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-10T00:18:15.866 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-10T00:18:15.867 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-10T00:18:15.867 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-10T00:18:15.867 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-10T00:18:15.867 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2023-09-10T00:18:15.892 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2023-09-10T00:18:15.912 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2023-09-10T00:18:15.913 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2023-09-10T00:18:15.971 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="Red Hat Enterprise Linux" 2023-09-10T00:18:15.971 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="8.6 (Ootpa)" 2023-09-10T00:18:15.972 INFO:teuthology.orchestra.run.smithi131.stdout:ID="rhel" 2023-09-10T00:18:15.972 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="fedora" 2023-09-10T00:18:15.972 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="8.6" 2023-09-10T00:18:15.972 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el8" 2023-09-10T00:18:15.972 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-10T00:18:15.973 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2023-09-10T00:18:15.973 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-10T00:18:15.973 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://www.redhat.com/" 2023-09-10T00:18:15.973 INFO:teuthology.orchestra.run.smithi131.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-10T00:18:15.973 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-10T00:18:15.974 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-10T00:18:15.974 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-10T00:18:15.974 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-10T00:18:15.974 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-10T00:18:15.974 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-10T00:18:15.975 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2023-09-10T00:18:16.000 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-10T00:18:16.011 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-10T00:18:16.019 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-10T00:18:16.019 DEBUG:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2023-09-10T00:18:16.022 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2023-09-10T00:18:16.024 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2023-09-10T00:18:16.042 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-10T00:18:16.049 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-10T00:18:16.050 DEBUG:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2023-09-10T00:18:16.081 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2023-09-10T00:18:16.084 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2023-09-10T00:18:16.109 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-10T00:18:16.186 INFO:teuthology.run_tasks:Running task kernel... 2023-09-10T00:18:16.199 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-09-10T00:18:16.200 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-09-10T00:18:16.200 DEBUG:teuthology.orchestra.run.smithi044:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-10T00:18:16.201 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-10T00:18:16.202 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-10T00:18:16.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-10T00:18:16.220 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2023-09-10T00:18:16.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-10T00:18:16.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-10T00:18:16.223 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2023-09-10T00:18:16.223 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2023-09-10T00:18:16.280 INFO:teuthology.orchestra.run.smithi131.stdout:4.18.0-372.9.1.el8.x86_64 2023-09-10T00:18:16.280 INFO:teuthology.task.kernel:Running kernel on smithi131: 4.18.0-372.9.1.el8.x86_64 2023-09-10T00:18:16.281 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum install -y kernel 2023-09-10T00:18:16.282 INFO:teuthology.orchestra.run.smithi099.stdout:4.18.0-372.9.1.el8.x86_64 2023-09-10T00:18:16.283 INFO:teuthology.task.kernel:Running kernel on smithi099: 4.18.0-372.9.1.el8.x86_64 2023-09-10T00:18:16.283 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum install -y kernel 2023-09-10T00:18:16.284 INFO:teuthology.orchestra.run.smithi044.stdout:4.18.0-372.9.1.el8.x86_64 2023-09-10T00:18:16.284 INFO:teuthology.task.kernel:Running kernel on smithi044: 4.18.0-372.9.1.el8.x86_64 2023-09-10T00:18:16.285 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum install -y kernel 2023-09-10T00:18:17.538 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2023-09-10T00:18:17.538 INFO:teuthology.orchestra.run.smithi044.stdout:Unable to read consumer identity 2023-09-10T00:18:17.542 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2023-09-10T00:18:17.542 INFO:teuthology.orchestra.run.smithi131.stdout:Unable to read consumer identity 2023-09-10T00:18:17.595 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-09-10T00:18:17.595 INFO:teuthology.orchestra.run.smithi099.stdout:Unable to read consumer identity 2023-09-10T00:18:18.911 INFO:teuthology.orchestra.run.smithi044.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-09-10T00:18:18.970 INFO:teuthology.orchestra.run.smithi131.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-09-10T00:18:19.043 INFO:teuthology.orchestra.run.smithi099.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-09-10T00:18:19.726 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 24 MB/s | 16 MB 00:00 2023-09-10T00:18:20.303 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-09-10T00:18:20.996 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 7.9 MB/s | 16 MB 00:02 2023-09-10T00:18:23.160 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 637 kB/s | 24 kB 00:00 2023-09-10T00:18:23.854 INFO:teuthology.orchestra.run.smithi099.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-09-10T00:18:23.899 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-10T00:18:23.900 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2023-09-10T00:18:23.900 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-10T00:18:23.978 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 449 kB/s | 24 kB 00:00 2023-09-10T00:18:24.036 DEBUG:teuthology.orchestra.run.smithi099:> echo no | sudo yum reinstall kernel || true 2023-09-10T00:18:24.365 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-09-10T00:18:24.365 INFO:teuthology.orchestra.run.smithi099.stdout:Unable to read consumer identity 2023-09-10T00:18:24.489 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 374 kB/s | 24 kB 00:00 2023-09-10T00:18:24.624 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:01 ago on Sun 10 Sep 2023 12:18:23 AM UTC. 2023-09-10T00:18:24.664 INFO:teuthology.orchestra.run.smithi099.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-09-10T00:18:24.698 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-09-10T00:18:24.699 INFO:teuthology.orchestra.run.smithi099.stderr:Error: No packages marked for reinstall. 2023-09-10T00:18:24.744 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum reinstall -y kernel || true 2023-09-10T00:18:24.753 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2023-09-10T00:18:24.754 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2023-09-10T00:18:24.755 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2023-09-10T00:18:24.903 DEBUG:teuthology.orchestra.run.smithi044:> echo no | sudo yum reinstall kernel || true 2023-09-10T00:18:25.072 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-09-10T00:18:25.072 INFO:teuthology.orchestra.run.smithi099.stdout:Unable to read consumer identity 2023-09-10T00:18:25.207 INFO:teuthology.orchestra.run.smithi131.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-09-10T00:18:25.247 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2023-09-10T00:18:25.248 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2023-09-10T00:18:25.248 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2023-09-10T00:18:25.252 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2023-09-10T00:18:25.252 INFO:teuthology.orchestra.run.smithi044.stdout:Unable to read consumer identity 2023-09-10T00:18:25.324 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:02 ago on Sun 10 Sep 2023 12:18:23 AM UTC. 2023-09-10T00:18:25.363 INFO:teuthology.orchestra.run.smithi099.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-09-10T00:18:25.380 DEBUG:teuthology.orchestra.run.smithi131:> echo no | sudo yum reinstall kernel || true 2023-09-10T00:18:25.399 INFO:teuthology.orchestra.run.smithi099.stderr:Error: No packages marked for reinstall. 2023-09-10T00:18:25.448 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q kernel | sort -rV | head -n 1 2023-09-10T00:18:25.524 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:02 ago on Sun 10 Sep 2023 12:18:23 AM UTC. 2023-09-10T00:18:25.527 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-09-10T00:18:25.528 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-09-10T00:18:25.528 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-09-10T00:18:25.528 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-09-10T00:18:25.529 DEBUG:teuthology.orchestra.run.smithi099:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-10T00:18:25.555 INFO:teuthology.orchestra.run.smithi099.stdout:ttyS1 2023-09-10T00:18:25.567 DEBUG:teuthology.parallel:result is None 2023-09-10T00:18:25.570 INFO:teuthology.orchestra.run.smithi044.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-09-10T00:18:25.604 INFO:teuthology.orchestra.run.smithi044.stderr:Error: No packages marked for reinstall. 2023-09-10T00:18:25.652 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum reinstall -y kernel || true 2023-09-10T00:18:25.712 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2023-09-10T00:18:25.712 INFO:teuthology.orchestra.run.smithi131.stdout:Unable to read consumer identity 2023-09-10T00:18:25.970 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:01 ago on Sun 10 Sep 2023 12:18:24 AM UTC. 2023-09-10T00:18:26.005 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2023-09-10T00:18:26.005 INFO:teuthology.orchestra.run.smithi044.stdout:Unable to read consumer identity 2023-09-10T00:18:26.020 INFO:teuthology.orchestra.run.smithi131.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-09-10T00:18:26.059 INFO:teuthology.orchestra.run.smithi131.stderr:Error: No packages marked for reinstall. 2023-09-10T00:18:26.106 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum reinstall -y kernel || true 2023-09-10T00:18:26.270 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:03 ago on Sun 10 Sep 2023 12:18:23 AM UTC. 2023-09-10T00:18:26.313 INFO:teuthology.orchestra.run.smithi044.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-09-10T00:18:26.344 INFO:teuthology.orchestra.run.smithi044.stderr:Error: No packages marked for reinstall. 2023-09-10T00:18:26.391 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q kernel | sort -rV | head -n 1 2023-09-10T00:18:26.435 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2023-09-10T00:18:26.436 INFO:teuthology.orchestra.run.smithi131.stdout:Unable to read consumer identity 2023-09-10T00:18:26.473 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-09-10T00:18:26.474 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-09-10T00:18:26.474 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-09-10T00:18:26.474 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-09-10T00:18:26.474 DEBUG:teuthology.orchestra.run.smithi044:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-10T00:18:26.507 INFO:teuthology.orchestra.run.smithi044.stdout:ttyS1 2023-09-10T00:18:26.522 DEBUG:teuthology.parallel:result is None 2023-09-10T00:18:26.689 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:02 ago on Sun 10 Sep 2023 12:18:24 AM UTC. 2023-09-10T00:18:26.729 INFO:teuthology.orchestra.run.smithi131.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-09-10T00:18:26.755 INFO:teuthology.orchestra.run.smithi131.stderr:Error: No packages marked for reinstall. 2023-09-10T00:18:26.802 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q kernel | sort -rV | head -n 1 2023-09-10T00:18:26.883 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-09-10T00:18:26.884 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-09-10T00:18:26.884 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-09-10T00:18:26.884 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2023-09-10T00:18:26.884 DEBUG:teuthology.orchestra.run.smithi131:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-10T00:18:26.915 INFO:teuthology.orchestra.run.smithi131.stdout:ttyS1 2023-09-10T00:18:26.928 DEBUG:teuthology.parallel:result is None 2023-09-10T00:18:26.928 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-10T00:18:26.940 INFO:teuthology.task.internal:Creating test directory... 2023-09-10T00:18:26.941 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-10T00:18:26.943 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-10T00:18:26.946 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-10T00:18:26.984 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-10T00:18:26.990 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-10T00:18:26.999 INFO:teuthology.task.internal:Creating archive directory... 2023-09-10T00:18:26.999 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-10T00:18:27.001 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-10T00:18:27.005 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-10T00:18:27.047 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-10T00:18:27.053 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-10T00:18:27.053 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-10T00:18:27.060 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-10T00:18:27.065 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-10T00:18:27.087 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-10T00:18:27.098 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-10T00:18:27.107 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-10T00:18:27.114 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-10T00:18:27.118 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-10T00:18:27.124 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-10T00:18:27.126 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-10T00:18:27.140 INFO:teuthology.task.internal:Configuring sudo... 2023-09-10T00:18:27.141 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-10T00:18:27.143 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-10T00:18:27.162 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-10T00:18:27.192 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-10T00:18:27.201 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-10T00:18:27.201 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-10T00:18:27.212 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-10T00:18:27.230 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-10T00:18:27.249 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-10T00:18:27.311 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-10T00:18:27.373 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-10T00:18:27.415 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-10T00:18:27.459 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:18:27.460 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-10T00:18:27.550 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-10T00:18:27.597 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-10T00:18:27.655 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-10T00:18:27.742 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-10T00:18:27.786 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-10T00:18:27.787 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-10T00:18:27.857 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-10T00:18:27.901 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-10T00:18:27.950 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-10T00:18:28.028 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-10T00:18:28.078 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-10T00:18:28.079 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-10T00:18:28.147 DEBUG:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2023-09-10T00:18:28.149 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2023-09-10T00:18:28.152 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2023-09-10T00:18:28.212 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-10T00:18:28.212 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-10T00:18:28.238 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-10T00:18:28.620 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-10T00:18:28.630 INFO:teuthology.task.internal:Starting timer... 2023-09-10T00:18:28.631 INFO:teuthology.run_tasks:Running task pcp... 2023-09-10T00:18:28.645 INFO:teuthology.run_tasks:Running task selinux... 2023-09-10T00:18:28.660 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-09-10T00:18:28.661 DEBUG:teuthology.orchestra.run.smithi044:> sudo service auditd rotate 2023-09-10T00:18:28.731 INFO:teuthology.orchestra.run.smithi044.stdout:Rotating logs: 2023-09-10T00:18:28.733 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2023-09-10T00:18:28.802 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2023-09-10T00:18:28.804 DEBUG:teuthology.orchestra.run.smithi131:> sudo service auditd rotate 2023-09-10T00:18:28.875 INFO:teuthology.orchestra.run.smithi131.stdout:Rotating logs: 2023-09-10T00:18:28.877 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-10T00:18:28.877 DEBUG:teuthology.orchestra.run.smithi044:> /usr/sbin/getenforce 2023-09-10T00:18:28.901 INFO:teuthology.orchestra.run.smithi044.stdout:Permissive 2023-09-10T00:18:28.901 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2023-09-10T00:18:28.923 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2023-09-10T00:18:28.923 DEBUG:teuthology.orchestra.run.smithi131:> /usr/sbin/getenforce 2023-09-10T00:18:28.952 INFO:teuthology.orchestra.run.smithi131.stdout:Permissive 2023-09-10T00:18:28.953 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi044.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi131.front.sepia.ceph.com': 'permissive'} 2023-09-10T00:18:28.953 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-10T00:18:28.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-10T00:18:28.982 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-10T00:18:29.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-10T00:18:29.009 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-10T00:18:29.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-10T00:18:29.036 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-10T00:18:29.037 DEBUG:teuthology.orchestra.run.smithi044:> sudo /usr/sbin/setenforce permissive 2023-09-10T00:18:29.066 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2023-09-10T00:18:29.095 DEBUG:teuthology.orchestra.run.smithi131:> sudo /usr/sbin/setenforce permissive 2023-09-10T00:18:29.122 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-10T00:18:29.134 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-09-10T00:18:29.140 INFO:teuthology.repo_utils:Fetching main from origin 2023-09-10T00:18:29.253 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-10T00:18:29.269 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-10T00:18:29.270 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi044.front.sepia.ceph.com,smithi099.front.sepia.ceph.com,smithi131.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-10T00:27:40.389 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi044.front.sepia.ceph.com'), Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi131.front.sepia.ceph.com')] 2023-09-10T00:27:40.390 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2023-09-10T00:27:40.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-10T00:27:40.460 DEBUG:teuthology.orchestra.run.smithi044:> true 2023-09-10T00:27:40.527 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2023-09-10T00:27:40.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-09-10T00:27:40.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-10T00:27:40.597 DEBUG:teuthology.orchestra.run.smithi099:> true 2023-09-10T00:27:40.663 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-09-10T00:27:40.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-09-10T00:27:40.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-10T00:27:40.733 DEBUG:teuthology.orchestra.run.smithi131:> true 2023-09-10T00:27:40.800 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-09-10T00:27:40.800 INFO:teuthology.run_tasks:Running task clock... 2023-09-10T00:27:40.810 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-10T00:27:40.811 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-10T00:27:40.811 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-10T00:27:40.814 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-10T00:27:40.815 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-10T00:27:40.817 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-10T00:27:40.817 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-10T00:27:40.847 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-10T00:27:40.848 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-10T00:27:40.863 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-10T00:27:40.864 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-10T00:27:40.873 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-10T00:27:40.888 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-10T00:27:40.931 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: ntpd: command not found 2023-09-10T00:27:40.939 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2023-09-10T00:27:40.944 INFO:teuthology.orchestra.run.smithi044.stdout:506 Cannot talk to daemon 2023-09-10T00:27:40.950 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2023-09-10T00:27:40.956 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: ntpd: command not found 2023-09-10T00:27:40.960 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-10T00:27:40.966 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-10T00:27:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:506 Cannot talk to daemon 2023-09-10T00:27:40.975 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-10T00:27:40.979 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-10T00:27:40.982 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-10T00:27:40.996 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-10T00:27:41.052 INFO:teuthology.orchestra.run.smithi044.stderr:bash: ntpq: command not found 2023-09-10T00:27:41.055 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-10T00:27:41.055 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2023-09-10T00:27:41.055 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-10T00:27:41.056 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-10T00:27:41.056 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-10T00:27:41.056 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-10T00:27:41.057 INFO:teuthology.orchestra.run.smithi099.stderr:bash: ntpq: command not found 2023-09-10T00:27:41.058 INFO:teuthology.orchestra.run.smithi131.stderr:bash: ntpq: command not found 2023-09-10T00:27:41.059 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-10T00:27:41.059 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2023-09-10T00:27:41.059 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-10T00:27:41.059 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-10T00:27:41.060 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-10T00:27:41.060 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-10T00:27:41.061 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-10T00:27:41.061 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2023-09-10T00:27:41.062 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-10T00:27:41.062 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-10T00:27:41.062 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-10T00:27:41.062 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-10T00:27:41.063 INFO:teuthology.run_tasks:Running task pexec... 2023-09-10T00:27:41.072 INFO:teuthology.task.pexec:Executing custom commands... 2023-09-10T00:27:41.072 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi044.front.sepia.ceph.com 2023-09-10T00:27:41.073 DEBUG:teuthology.orchestra.run.smithi044:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-10T00:27:41.073 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi099.front.sepia.ceph.com 2023-09-10T00:27:41.073 DEBUG:teuthology.orchestra.run.smithi099:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-10T00:27:41.074 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi131.front.sepia.ceph.com 2023-09-10T00:27:41.074 DEBUG:teuthology.orchestra.run.smithi131:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-10T00:27:41.471 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2023-09-10T00:27:41.472 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-09-10T00:27:41.489 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2023-09-10T00:27:42.281 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:53 ago on Sun 10 Sep 2023 12:25:48 AM UTC. 2023-09-10T00:27:42.288 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:53 ago on Sun 10 Sep 2023 12:25:48 AM UTC. 2023-09-10T00:27:42.332 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:36 ago on Sun 10 Sep 2023 12:26:05 AM UTC. 2023-09-10T00:27:43.994 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-10T00:27:43.995 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-10T00:27:43.995 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2023-09-10T00:27:43.995 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-10T00:27:43.995 INFO:teuthology.orchestra.run.smithi099.stdout:Resetting modules: 2023-09-10T00:27:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: container-tools 2023-09-10T00:27:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-10T00:27:43.996 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-10T00:27:43.996 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-09-10T00:27:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-10T00:27:44.108 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2023-09-10T00:27:44.110 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2023-09-10T00:27:44.110 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2023-09-10T00:27:44.110 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2023-09-10T00:27:44.110 INFO:teuthology.orchestra.run.smithi131.stdout:Resetting modules: 2023-09-10T00:27:44.110 INFO:teuthology.orchestra.run.smithi131.stdout: container-tools 2023-09-10T00:27:44.111 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-10T00:27:44.111 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2023-09-10T00:27:44.111 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2023-09-10T00:27:44.111 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-10T00:27:44.342 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2023-09-10T00:27:44.343 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2023-09-10T00:27:44.344 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2023-09-10T00:27:44.344 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2023-09-10T00:27:44.344 INFO:teuthology.orchestra.run.smithi044.stdout:Resetting modules: 2023-09-10T00:27:44.344 INFO:teuthology.orchestra.run.smithi044.stdout: container-tools 2023-09-10T00:27:44.344 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:27:44.344 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2023-09-10T00:27:44.345 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2023-09-10T00:27:44.345 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:27:44.957 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:56 ago on Sun 10 Sep 2023 12:25:48 AM UTC. 2023-09-10T00:27:45.067 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:56 ago on Sun 10 Sep 2023 12:25:48 AM UTC. 2023-09-10T00:27:45.345 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:39 ago on Sun 10 Sep 2023 12:26:05 AM UTC. 2023-09-10T00:27:47.407 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-10T00:27:47.857 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2023-09-10T00:27:47.870 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-09-10T00:27:48.118 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2023-09-10T00:27:48.335 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2023-09-10T00:27:48.600 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2023-09-10T00:27:48.683 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:00 ago on Sun 10 Sep 2023 12:25:48 AM UTC. 2023-09-10T00:27:49.150 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:02:00 ago on Sun 10 Sep 2023 12:25:48 AM UTC. 2023-09-10T00:27:49.455 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:44 ago on Sun 10 Sep 2023 12:26:05 AM UTC. 2023-09-10T00:27:50.394 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-10T00:27:50.487 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================================================ 2023-09-10T00:27:50.488 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-09-10T00:27:50.488 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================================================ 2023-09-10T00:27:50.488 INFO:teuthology.orchestra.run.smithi099.stdout:Installing group/module packages: 2023-09-10T00:27:50.488 INFO:teuthology.orchestra.run.smithi099.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-10T00:27:50.488 INFO:teuthology.orchestra.run.smithi099.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-10T00:27:50.489 INFO:teuthology.orchestra.run.smithi099.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-10T00:27:50.489 INFO:teuthology.orchestra.run.smithi099.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-10T00:27:50.489 INFO:teuthology.orchestra.run.smithi099.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-10T00:27:50.489 INFO:teuthology.orchestra.run.smithi099.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-10T00:27:50.489 INFO:teuthology.orchestra.run.smithi099.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-10T00:27:50.489 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-09-10T00:27:50.490 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-10T00:27:50.490 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-10T00:27:50.490 INFO:teuthology.orchestra.run.smithi099.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-10T00:27:50.490 INFO:teuthology.orchestra.run.smithi099.stdout:Installing module profiles: 2023-09-10T00:27:50.490 INFO:teuthology.orchestra.run.smithi099.stdout: container-tools/common 2023-09-10T00:27:50.491 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-10T00:27:50.491 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling module streams: 2023-09-10T00:27:50.491 INFO:teuthology.orchestra.run.smithi099.stdout: container-tools rhel8 2023-09-10T00:27:50.491 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-10T00:27:50.491 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-10T00:27:50.492 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================================================ 2023-09-10T00:27:50.492 INFO:teuthology.orchestra.run.smithi099.stdout:Install 10 Packages 2023-09-10T00:27:50.492 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-10T00:27:50.493 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 18 M 2023-09-10T00:27:50.493 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 65 M 2023-09-10T00:27:50.493 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-10T00:27:50.575 INFO:teuthology.orchestra.run.smithi099.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 482 kB/s | 41 kB 00:00 2023-09-10T00:27:50.600 INFO:teuthology.orchestra.run.smithi099.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 226 kB/s | 25 kB 00:00 2023-09-10T00:27:50.625 INFO:teuthology.orchestra.run.smithi099.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 985 kB/s | 49 kB 00:00 2023-09-10T00:27:50.650 INFO:teuthology.orchestra.run.smithi099.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 585 kB/s | 94 kB 00:00 2023-09-10T00:27:50.717 INFO:teuthology.orchestra.run.smithi099.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.4 MB/s | 524 kB 00:00 2023-09-10T00:27:50.743 INFO:teuthology.orchestra.run.smithi099.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.6 MB/s | 149 kB 00:00 2023-09-10T00:27:50.768 INFO:teuthology.orchestra.run.smithi099.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-09-10T00:27:50.863 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2023-09-10T00:27:50.868 INFO:teuthology.orchestra.run.smithi099.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-09-10T00:27:50.957 INFO:teuthology.orchestra.run.smithi131.stdout:============================================================================================================ 2023-09-10T00:27:50.958 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2023-09-10T00:27:50.958 INFO:teuthology.orchestra.run.smithi131.stdout:============================================================================================================ 2023-09-10T00:27:50.958 INFO:teuthology.orchestra.run.smithi131.stdout:Installing group/module packages: 2023-09-10T00:27:50.958 INFO:teuthology.orchestra.run.smithi131.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-10T00:27:50.958 INFO:teuthology.orchestra.run.smithi131.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-10T00:27:50.959 INFO:teuthology.orchestra.run.smithi131.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-10T00:27:50.959 INFO:teuthology.orchestra.run.smithi131.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-10T00:27:50.959 INFO:teuthology.orchestra.run.smithi131.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-10T00:27:50.959 INFO:teuthology.orchestra.run.smithi131.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-10T00:27:50.959 INFO:teuthology.orchestra.run.smithi131.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-10T00:27:50.960 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2023-09-10T00:27:50.960 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-10T00:27:50.960 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-10T00:27:50.960 INFO:teuthology.orchestra.run.smithi131.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-10T00:27:50.960 INFO:teuthology.orchestra.run.smithi131.stdout:Installing module profiles: 2023-09-10T00:27:50.960 INFO:teuthology.orchestra.run.smithi131.stdout: container-tools/common 2023-09-10T00:27:50.961 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-10T00:27:50.961 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling module streams: 2023-09-10T00:27:50.961 INFO:teuthology.orchestra.run.smithi131.stdout: container-tools rhel8 2023-09-10T00:27:50.961 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-10T00:27:50.961 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2023-09-10T00:27:50.961 INFO:teuthology.orchestra.run.smithi131.stdout:============================================================================================================ 2023-09-10T00:27:50.962 INFO:teuthology.orchestra.run.smithi131.stdout:Install 10 Packages 2023-09-10T00:27:50.962 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-10T00:27:50.962 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 18 M 2023-09-10T00:27:50.963 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 65 M 2023-09-10T00:27:50.963 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2023-09-10T00:27:50.968 INFO:teuthology.orchestra.run.smithi099.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 31 MB/s | 7.0 MB 00:00 2023-09-10T00:27:51.042 INFO:teuthology.orchestra.run.smithi131.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.1 MB/s | 94 kB 00:00 2023-09-10T00:27:51.067 INFO:teuthology.orchestra.run.smithi131.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 233 kB/s | 25 kB 00:00 2023-09-10T00:27:51.069 INFO:teuthology.orchestra.run.smithi099.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-09-10T00:27:51.069 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-10T00:27:51.070 INFO:teuthology.orchestra.run.smithi099.stdout:Total 31 MB/s | 18 MB 00:00 2023-09-10T00:27:51.092 INFO:teuthology.orchestra.run.smithi131.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 310 kB/s | 41 kB 00:00 2023-09-10T00:27:51.117 INFO:teuthology.orchestra.run.smithi131.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 654 kB/s | 49 kB 00:00 2023-09-10T00:27:51.151 INFO:teuthology.orchestra.run.smithi131.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-09-10T00:27:51.176 INFO:teuthology.orchestra.run.smithi131.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-09-10T00:27:51.201 INFO:teuthology.orchestra.run.smithi131.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-09-10T00:27:51.271 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-10T00:27:51.290 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-10T00:27:51.290 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-10T00:27:51.302 INFO:teuthology.orchestra.run.smithi131.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-09-10T00:27:51.410 INFO:teuthology.orchestra.run.smithi131.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 30 MB/s | 7.0 MB 00:00 2023-09-10T00:27:51.433 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2023-09-10T00:27:51.471 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-10T00:27:51.475 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-10T00:27:51.493 INFO:teuthology.orchestra.run.smithi131.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 20 MB/s | 8.0 MB 00:00 2023-09-10T00:27:51.494 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2023-09-10T00:27:51.494 INFO:teuthology.orchestra.run.smithi131.stdout:Total 34 MB/s | 18 MB 00:00 2023-09-10T00:27:51.533 INFO:teuthology.orchestra.run.smithi044.stdout:============================================================================================================ 2023-09-10T00:27:51.533 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2023-09-10T00:27:51.533 INFO:teuthology.orchestra.run.smithi044.stdout:============================================================================================================ 2023-09-10T00:27:51.534 INFO:teuthology.orchestra.run.smithi044.stdout:Installing group/module packages: 2023-09-10T00:27:51.534 INFO:teuthology.orchestra.run.smithi044.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-10T00:27:51.534 INFO:teuthology.orchestra.run.smithi044.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-10T00:27:51.534 INFO:teuthology.orchestra.run.smithi044.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-10T00:27:51.534 INFO:teuthology.orchestra.run.smithi044.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-10T00:27:51.535 INFO:teuthology.orchestra.run.smithi044.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-10T00:27:51.535 INFO:teuthology.orchestra.run.smithi044.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-10T00:27:51.535 INFO:teuthology.orchestra.run.smithi044.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-10T00:27:51.535 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2023-09-10T00:27:51.535 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-10T00:27:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-10T00:27:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-10T00:27:51.536 INFO:teuthology.orchestra.run.smithi044.stdout:Installing module profiles: 2023-09-10T00:27:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: container-tools/common 2023-09-10T00:27:51.536 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:27:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:Enabling module streams: 2023-09-10T00:27:51.537 INFO:teuthology.orchestra.run.smithi044.stdout: container-tools rhel8 2023-09-10T00:27:51.537 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:27:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2023-09-10T00:27:51.538 INFO:teuthology.orchestra.run.smithi044.stdout:============================================================================================================ 2023-09-10T00:27:51.538 INFO:teuthology.orchestra.run.smithi044.stdout:Install 10 Packages 2023-09-10T00:27:51.538 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:27:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 18 M 2023-09-10T00:27:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 65 M 2023-09-10T00:27:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2023-09-10T00:27:51.611 INFO:teuthology.orchestra.run.smithi044.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.2 MB/s | 94 kB 00:00 2023-09-10T00:27:51.636 INFO:teuthology.orchestra.run.smithi044.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 248 kB/s | 25 kB 00:00 2023-09-10T00:27:51.653 INFO:teuthology.orchestra.run.smithi044.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 348 kB/s | 41 kB 00:00 2023-09-10T00:27:51.679 INFO:teuthology.orchestra.run.smithi044.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 734 kB/s | 49 kB 00:00 2023-09-10T00:27:51.694 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2023-09-10T00:27:51.712 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2023-09-10T00:27:51.713 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2023-09-10T00:27:51.735 INFO:teuthology.orchestra.run.smithi044.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.2 MB/s | 524 kB 00:00 2023-09-10T00:27:51.773 INFO:teuthology.orchestra.run.smithi044.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.6 MB/s | 149 kB 00:00 2023-09-10T00:27:51.796 INFO:teuthology.orchestra.run.smithi044.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.4 MB/s | 213 kB 00:00 2023-09-10T00:27:51.893 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2023-09-10T00:27:51.897 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2023-09-10T00:27:51.905 INFO:teuthology.orchestra.run.smithi044.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 20 MB/s | 2.2 MB 00:00 2023-09-10T00:27:52.010 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-10T00:27:52.030 INFO:teuthology.orchestra.run.smithi044.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 27 MB/s | 7.0 MB 00:00 2023-09-10T00:27:52.122 INFO:teuthology.orchestra.run.smithi044.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 17 MB/s | 8.0 MB 00:00 2023-09-10T00:27:52.122 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2023-09-10T00:27:52.123 INFO:teuthology.orchestra.run.smithi044.stdout:Total 31 MB/s | 18 MB 00:00 2023-09-10T00:27:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-10T00:27:52.329 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2023-09-10T00:27:52.350 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2023-09-10T00:27:52.350 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2023-09-10T00:27:52.396 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-10T00:27:52.437 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2023-09-10T00:27:52.546 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2023-09-10T00:27:52.550 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2023-09-10T00:27:52.599 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-10T00:27:52.632 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-10T00:27:52.816 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-10T00:27:52.837 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-10T00:27:53.031 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-10T00:27:53.183 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-10T00:27:53.232 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2023-09-10T00:27:53.243 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-10T00:27:53.433 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-10T00:27:53.624 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-10T00:27:53.656 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-10T00:27:53.879 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-10T00:27:53.998 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-10T00:27:54.128 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-10T00:27:54.403 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-10T00:27:54.530 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-10T00:27:54.909 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-10T00:27:55.128 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-10T00:27:55.302 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-10T00:27:55.337 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-10T00:27:55.378 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-10T00:27:55.514 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-10T00:27:55.567 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-10T00:27:55.752 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-10T00:27:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-10T00:27:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-10T00:27:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-10T00:27:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-10T00:27:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-10T00:27:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-10T00:27:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-10T00:27:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-10T00:27:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-10T00:27:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-10T00:27:55.988 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-10T00:27:56.256 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-10T00:27:56.256 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-10T00:27:56.256 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-10T00:27:56.257 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-10T00:27:56.257 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-10T00:27:56.257 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-10T00:27:56.257 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-10T00:27:56.257 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-10T00:27:56.258 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-10T00:27:56.258 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-10T00:27:56.307 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-10T00:27:56.615 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-10T00:27:56.928 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-10T00:27:56.928 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:08 ago on Sun 10 Sep 2023 12:25:48 AM UTC. 2023-09-10T00:27:56.972 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-10T00:27:57.173 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-10T00:27:57.365 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-10T00:27:57.366 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:02:08 ago on Sun 10 Sep 2023 12:25:48 AM UTC. 2023-09-10T00:27:57.448 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-10T00:27:57.449 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-10T00:27:57.449 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-10T00:27:57.449 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-10T00:27:57.449 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-10T00:27:57.450 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-10T00:27:57.450 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-10T00:27:57.450 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-10T00:27:57.450 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-10T00:27:57.450 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-10T00:27:58.661 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-10T00:27:58.661 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:53 ago on Sun 10 Sep 2023 12:26:05 AM UTC. 2023-09-10T00:27:59.550 INFO:teuthology.orchestra.run.smithi099.stdout:Installed products updated. 2023-09-10T00:27:59.753 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-10T00:27:59.753 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-10T00:27:59.754 INFO:teuthology.orchestra.run.smithi099.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-10T00:27:59.754 INFO:teuthology.orchestra.run.smithi099.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-10T00:27:59.754 INFO:teuthology.orchestra.run.smithi099.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-10T00:27:59.754 INFO:teuthology.orchestra.run.smithi099.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-10T00:27:59.755 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-10T00:27:59.755 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-10T00:27:59.755 INFO:teuthology.orchestra.run.smithi099.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-10T00:27:59.755 INFO:teuthology.orchestra.run.smithi099.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-10T00:27:59.755 INFO:teuthology.orchestra.run.smithi099.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-10T00:27:59.756 INFO:teuthology.orchestra.run.smithi099.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-10T00:27:59.756 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-10T00:27:59.756 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-10T00:27:59.896 DEBUG:teuthology.parallel:result is None 2023-09-10T00:27:59.923 INFO:teuthology.orchestra.run.smithi131.stdout:Installed products updated. 2023-09-10T00:28:00.117 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-10T00:28:00.117 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2023-09-10T00:28:00.117 INFO:teuthology.orchestra.run.smithi131.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-10T00:28:00.117 INFO:teuthology.orchestra.run.smithi131.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-10T00:28:00.118 INFO:teuthology.orchestra.run.smithi131.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-10T00:28:00.118 INFO:teuthology.orchestra.run.smithi131.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-10T00:28:00.118 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-10T00:28:00.118 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-10T00:28:00.119 INFO:teuthology.orchestra.run.smithi131.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-10T00:28:00.119 INFO:teuthology.orchestra.run.smithi131.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-10T00:28:00.119 INFO:teuthology.orchestra.run.smithi131.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-10T00:28:00.119 INFO:teuthology.orchestra.run.smithi131.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-10T00:28:00.119 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-10T00:28:00.120 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2023-09-10T00:28:00.265 DEBUG:teuthology.parallel:result is None 2023-09-10T00:28:01.401 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2023-09-10T00:28:01.627 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:28:01.627 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2023-09-10T00:28:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-10T00:28:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-10T00:28:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-10T00:28:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-10T00:28:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-10T00:28:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-10T00:28:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-10T00:28:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-10T00:28:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-10T00:28:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-10T00:28:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:28:01.630 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2023-09-10T00:28:01.780 DEBUG:teuthology.parallel:result is None 2023-09-10T00:28:01.780 INFO:teuthology.run_tasks:Running task install... 2023-09-10T00:28:01.793 DEBUG:teuthology.task.install:project ceph 2023-09-10T00:28:01.793 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9670cca5bc8e17916865dc84939edfc10b1f2c00'}} 2023-09-10T00:28:01.794 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9670cca5bc8e17916865dc84939edfc10b1f2c00'} 2023-09-10T00:28:01.794 INFO:teuthology.task.install:Using flavor: default 2023-09-10T00:28:01.800 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-10T00:28:01.800 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-09-10T00:28:01.801 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9670cca5bc8e17916865dc84939edfc10b1f2c00', 'tag': None, 'wait_for_package': False} 2023-09-10T00:28:01.801 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-10T00:28:01.803 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9670cca5bc8e17916865dc84939edfc10b1f2c00', 'tag': None, 'wait_for_package': False} 2023-09-10T00:28:01.803 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-10T00:28:01.805 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9670cca5bc8e17916865dc84939edfc10b1f2c00', 'tag': None, 'wait_for_package': False} 2023-09-10T00:28:01.805 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-10T00:28:01.946 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/ 2023-09-10T00:28:01.946 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6025.g9670cca5 2023-09-10T00:28:01.951 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/ 2023-09-10T00:28:01.951 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6025.g9670cca5 2023-09-10T00:28:02.020 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/ 2023-09-10T00:28:02.021 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6025.g9670cca5 2023-09-10T00:28:02.150 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-10T00:28:02.151 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:28:02.151 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-10T00:28:02.181 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-10T00:28:02.181 DEBUG:teuthology.orchestra.run.smithi044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9670cca5bc8e17916865dc84939edfc10b1f2c00/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-10T00:28:02.210 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-10T00:28:02.210 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-10T00:28:02.210 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-10T00:28:02.243 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-10T00:28:02.243 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9670cca5bc8e17916865dc84939edfc10b1f2c00/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-10T00:28:02.250 DEBUG:teuthology.orchestra.run.smithi044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-10T00:28:02.259 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-10T00:28:02.259 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-10T00:28:02.259 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-10T00:28:02.291 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-10T00:28:02.292 DEBUG:teuthology.orchestra.run.smithi131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9670cca5bc8e17916865dc84939edfc10b1f2c00/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-10T00:28:02.315 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-10T00:28:02.329 DEBUG:teuthology.orchestra.run.smithi044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-10T00:28:02.364 DEBUG:teuthology.orchestra.run.smithi131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-10T00:28:02.396 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-10T00:28:02.397 INFO:teuthology.orchestra.run.smithi044.stdout:check_obsoletes = 1 2023-09-10T00:28:02.398 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum clean all 2023-09-10T00:28:02.445 DEBUG:teuthology.orchestra.run.smithi131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-10T00:28:02.464 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2023-09-10T00:28:02.465 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2023-09-10T00:28:02.513 INFO:teuthology.orchestra.run.smithi131.stdout:check_obsoletes = 1 2023-09-10T00:28:02.514 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum clean all 2023-09-10T00:28:02.791 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2023-09-10T00:28:02.837 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-09-10T00:28:02.888 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2023-09-10T00:28:03.068 INFO:teuthology.orchestra.run.smithi044.stdout:44 files removed 2023-09-10T00:28:03.106 INFO:teuthology.orchestra.run.smithi099.stdout:44 files removed 2023-09-10T00:28:03.115 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-10T00:28:03.152 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-10T00:28:03.165 INFO:teuthology.orchestra.run.smithi131.stdout:44 files removed 2023-09-10T00:28:03.212 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-10T00:28:03.466 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2023-09-10T00:28:03.484 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-09-10T00:28:03.545 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2023-09-10T00:28:04.459 INFO:teuthology.orchestra.run.smithi044.stdout:ceph packages for x86_64 136 kB/s | 81 kB 00:00 2023-09-10T00:28:04.480 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 135 kB/s | 81 kB 00:00 2023-09-10T00:28:04.590 INFO:teuthology.orchestra.run.smithi131.stdout:ceph packages for x86_64 130 kB/s | 81 kB 00:00 2023-09-10T00:28:05.089 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 16 kB/s | 9.4 kB 00:00 2023-09-10T00:28:05.101 INFO:teuthology.orchestra.run.smithi044.stdout:ceph noarch packages 16 kB/s | 9.4 kB 00:00 2023-09-10T00:28:05.355 INFO:teuthology.orchestra.run.smithi131.stdout:ceph noarch packages 13 kB/s | 9.4 kB 00:00 2023-09-10T00:28:05.443 INFO:teuthology.orchestra.run.smithi044.stdout:ceph source packages 4.7 kB/s | 1.5 kB 00:00 2023-09-10T00:28:05.456 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 4.3 kB/s | 1.5 kB 00:00 2023-09-10T00:28:05.590 INFO:teuthology.orchestra.run.smithi099.stdout:Copr repo for python3-asyncssh owned by ceph 31 kB/s | 3.5 kB 00:00 2023-09-10T00:28:05.610 INFO:teuthology.orchestra.run.smithi044.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-09-10T00:28:05.742 INFO:teuthology.orchestra.run.smithi131.stdout:ceph source packages 4.1 kB/s | 1.5 kB 00:00 2023-09-10T00:28:05.884 INFO:teuthology.orchestra.run.smithi131.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-09-10T00:28:06.810 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2023-09-10T00:28:07.752 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 8.6 MB/s | 16 MB 00:01 2023-09-10T00:28:08.522 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 5.5 MB/s | 16 MB 00:02 2023-09-10T00:28:10.349 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 418 kB/s | 24 kB 00:00 2023-09-10T00:28:11.260 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 462 kB/s | 24 kB 00:00 2023-09-10T00:28:11.293 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 54 MB/s | 47 MB 00:00 2023-09-10T00:28:12.170 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 53 MB 00:00 2023-09-10T00:28:12.195 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 406 kB/s | 24 kB 00:00 2023-09-10T00:28:13.172 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 53 MB/s | 47 MB 00:00 2023-09-10T00:28:19.202 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 60 MB/s | 53 MB 00:00 2023-09-10T00:28:20.303 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 56 MB/s | 47 MB 00:00 2023-09-10T00:28:21.806 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-09-10T00:28:30.366 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-10T00:28:30.367 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-10T00:28:30.381 INFO:teuthology.orchestra.run.smithi099.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-10T00:28:30.382 INFO:teuthology.orchestra.run.smithi099.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-10T00:28:30.383 INFO:teuthology.orchestra.run.smithi099.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-10T00:28:30.384 INFO:teuthology.orchestra.run.smithi099.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-10T00:28:30.386 INFO:teuthology.orchestra.run.smithi099.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-10T00:28:30.389 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-10T00:28:30.390 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-10T00:28:30.521 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-09-10T00:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout:=========================================================================================================================== 2023-09-10T00:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-09-10T00:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout:=========================================================================================================================== 2023-09-10T00:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-09-10T00:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-10T00:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-10T00:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 6.4 k 2023-09-10T00:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 5.4 M 2023-09-10T00:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 907 k 2023-09-10T00:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 167 k 2023-09-10T00:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 1.6 M 2023-09-10T00:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 132 k 2023-09-10T00:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 1.7 M 2023-09-10T00:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 7.4 M 2023-09-10T00:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 47 k 2023-09-10T00:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 14 M 2023-09-10T00:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 49 M 2023-09-10T00:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 261 k 2023-09-10T00:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 176 k 2023-09-10T00:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 260 k 2023-09-10T00:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 18 k 2023-09-10T00:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-10T00:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-10T00:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-10T00:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 31 k 2023-09-10T00:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 827 k 2023-09-10T00:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 139 k 2023-09-10T00:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-10T00:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-10T00:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-10T00:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-10T00:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-10T00:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-10T00:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 197 k 2023-09-10T00:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 372 k 2023-09-10T00:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 364 k 2023-09-10T00:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 107 k 2023-09-10T00:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 94 k 2023-09-10T00:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.6 M 2023-09-10T00:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 179 k 2023-09-10T00:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-10T00:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2023-09-10T00:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-10T00:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.7 M 2023-09-10T00:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.7 M 2023-09-10T00:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-09-10T00:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-10T00:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 22 M 2023-09-10T00:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 23 k 2023-09-10T00:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 2.3 M 2023-09-10T00:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 242 k 2023-09-10T00:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 4.6 M 2023-09-10T00:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 18 M 2023-09-10T00:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 15 k 2023-09-10T00:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 24 k 2023-09-10T00:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-10T00:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-10T00:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-10T00:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-10T00:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-10T00:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 184 k 2023-09-10T00:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-10T00:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-10T00:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-10T00:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-10T00:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-10T00:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-10T00:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 519 k 2023-09-10T00:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-10T00:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 6.8 M 2023-09-10T00:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-10T00:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-10T00:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-10T00:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-10T00:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-10T00:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-10T00:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-10T00:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-10T00:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-10T00:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-10T00:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-10T00:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-10T00:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-10T00:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-10T00:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-10T00:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-10T00:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 45 k 2023-09-10T00:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 126 k 2023-09-10T00:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-10T00:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-10T00:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-10T00:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-10T00:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-10T00:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-10T00:28:30.552 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-10T00:28:30.552 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-10T00:28:30.552 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-10T00:28:30.552 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-10T00:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-10T00:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-10T00:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-10T00:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-10T00:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-10T00:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-10T00:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-10T00:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-10T00:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-10T00:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-10T00:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-10T00:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-10T00:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-10T00:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-10T00:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-10T00:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-10T00:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-10T00:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-10T00:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-10T00:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-10T00:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-10T00:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-10T00:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-10T00:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-10T00:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-10T00:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-10T00:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-10T00:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-10T00:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-10T00:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-10T00:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-10T00:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-10T00:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-10T00:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-10T00:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-10T00:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-10T00:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-10T00:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-10T00:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-10T00:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling module streams: 2023-09-10T00:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql 10 2023-09-10T00:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-10T00:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-09-10T00:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout:=========================================================================================================================== 2023-09-10T00:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout:Install 128 Packages 2023-09-10T00:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 3 Packages 2023-09-10T00:28:30.562 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-10T00:28:30.562 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 208 M 2023-09-10T00:28:30.562 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-09-10T00:28:30.885 INFO:teuthology.orchestra.run.smithi099.stdout:(1/131): ceph-18.0.0-6025.g9670cca5.el8.x86_64. 34 kB/s | 6.4 kB 00:00 2023-09-10T00:28:31.219 INFO:teuthology.orchestra.run.smithi099.stdout:(2/131): ceph-fuse-18.0.0-6025.g9670cca5.el8.x8 2.7 MB/s | 907 kB 00:00 2023-09-10T00:28:31.229 INFO:teuthology.orchestra.run.smithi131.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-10T00:28:31.230 INFO:teuthology.orchestra.run.smithi131.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-10T00:28:31.245 INFO:teuthology.orchestra.run.smithi131.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-10T00:28:31.246 INFO:teuthology.orchestra.run.smithi131.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-10T00:28:31.247 INFO:teuthology.orchestra.run.smithi131.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-10T00:28:31.249 INFO:teuthology.orchestra.run.smithi131.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-10T00:28:31.251 INFO:teuthology.orchestra.run.smithi131.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-10T00:28:31.254 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-10T00:28:31.256 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-10T00:28:31.302 INFO:teuthology.orchestra.run.smithi099.stdout:(3/131): ceph-immutable-object-cache-18.0.0-602 2.0 MB/s | 167 kB 00:00 2023-09-10T00:28:31.388 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2023-09-10T00:28:31.400 INFO:teuthology.orchestra.run.smithi131.stdout:=========================================================================================================================== 2023-09-10T00:28:31.401 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2023-09-10T00:28:31.401 INFO:teuthology.orchestra.run.smithi131.stdout:=========================================================================================================================== 2023-09-10T00:28:31.401 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2023-09-10T00:28:31.401 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-10T00:28:31.401 INFO:teuthology.orchestra.run.smithi131.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-10T00:28:31.402 INFO:teuthology.orchestra.run.smithi131.stdout: ceph x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 6.4 k 2023-09-10T00:28:31.402 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 5.4 M 2023-09-10T00:28:31.402 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 907 k 2023-09-10T00:28:31.402 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 167 k 2023-09-10T00:28:31.403 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 1.6 M 2023-09-10T00:28:31.403 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 132 k 2023-09-10T00:28:31.403 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 1.7 M 2023-09-10T00:28:31.403 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 7.4 M 2023-09-10T00:28:31.403 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 47 k 2023-09-10T00:28:31.403 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 14 M 2023-09-10T00:28:31.404 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 49 M 2023-09-10T00:28:31.404 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 261 k 2023-09-10T00:28:31.404 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 176 k 2023-09-10T00:28:31.404 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 260 k 2023-09-10T00:28:31.404 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 18 k 2023-09-10T00:28:31.405 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-10T00:28:31.405 INFO:teuthology.orchestra.run.smithi131.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-10T00:28:31.405 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-10T00:28:31.405 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 31 k 2023-09-10T00:28:31.405 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 827 k 2023-09-10T00:28:31.406 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 139 k 2023-09-10T00:28:31.406 INFO:teuthology.orchestra.run.smithi131.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-10T00:28:31.406 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-10T00:28:31.406 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-10T00:28:31.406 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-10T00:28:31.407 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-10T00:28:31.407 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-10T00:28:31.407 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 197 k 2023-09-10T00:28:31.407 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 372 k 2023-09-10T00:28:31.407 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 364 k 2023-09-10T00:28:31.408 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 107 k 2023-09-10T00:28:31.408 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 94 k 2023-09-10T00:28:31.408 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.6 M 2023-09-10T00:28:31.408 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 179 k 2023-09-10T00:28:31.408 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-10T00:28:31.409 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrading: 2023-09-10T00:28:31.409 INFO:teuthology.orchestra.run.smithi131.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-10T00:28:31.409 INFO:teuthology.orchestra.run.smithi131.stdout: librados2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.7 M 2023-09-10T00:28:31.409 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.7 M 2023-09-10T00:28:31.409 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2023-09-10T00:28:31.409 INFO:teuthology.orchestra.run.smithi131.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-10T00:28:31.410 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 22 M 2023-09-10T00:28:31.410 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 23 k 2023-09-10T00:28:31.410 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 2.3 M 2023-09-10T00:28:31.410 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 242 k 2023-09-10T00:28:31.410 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 4.6 M 2023-09-10T00:28:31.411 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 18 M 2023-09-10T00:28:31.411 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 15 k 2023-09-10T00:28:31.411 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 24 k 2023-09-10T00:28:31.411 INFO:teuthology.orchestra.run.smithi131.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-10T00:28:31.411 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-10T00:28:31.412 INFO:teuthology.orchestra.run.smithi131.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-10T00:28:31.412 INFO:teuthology.orchestra.run.smithi131.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-10T00:28:31.412 INFO:teuthology.orchestra.run.smithi131.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-10T00:28:31.412 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 184 k 2023-09-10T00:28:31.412 INFO:teuthology.orchestra.run.smithi131.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-10T00:28:31.413 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-10T00:28:31.413 INFO:teuthology.orchestra.run.smithi131.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-10T00:28:31.413 INFO:teuthology.orchestra.run.smithi131.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-10T00:28:31.413 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-10T00:28:31.413 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-10T00:28:31.414 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 519 k 2023-09-10T00:28:31.414 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-10T00:28:31.414 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 6.8 M 2023-09-10T00:28:31.414 INFO:teuthology.orchestra.run.smithi131.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-10T00:28:31.414 INFO:teuthology.orchestra.run.smithi131.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-10T00:28:31.415 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-10T00:28:31.415 INFO:teuthology.orchestra.run.smithi131.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-10T00:28:31.415 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-10T00:28:31.415 INFO:teuthology.orchestra.run.smithi131.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-10T00:28:31.415 INFO:teuthology.orchestra.run.smithi131.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-10T00:28:31.415 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-10T00:28:31.416 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-10T00:28:31.416 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-10T00:28:31.416 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-10T00:28:31.416 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-10T00:28:31.416 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-10T00:28:31.417 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-10T00:28:31.417 INFO:teuthology.orchestra.run.smithi131.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-10T00:28:31.417 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-10T00:28:31.417 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 45 k 2023-09-10T00:28:31.417 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 126 k 2023-09-10T00:28:31.418 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-10T00:28:31.418 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-10T00:28:31.418 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-10T00:28:31.418 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-10T00:28:31.418 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-10T00:28:31.419 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-10T00:28:31.419 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-10T00:28:31.419 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-10T00:28:31.419 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-10T00:28:31.419 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-10T00:28:31.419 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-10T00:28:31.420 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-10T00:28:31.420 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-10T00:28:31.420 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-10T00:28:31.420 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-10T00:28:31.420 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-10T00:28:31.420 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-10T00:28:31.421 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-10T00:28:31.421 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-10T00:28:31.421 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-10T00:28:31.421 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-10T00:28:31.421 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-10T00:28:31.422 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-10T00:28:31.422 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-10T00:28:31.422 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-10T00:28:31.422 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-10T00:28:31.422 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-10T00:28:31.423 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-10T00:28:31.423 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-10T00:28:31.423 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-10T00:28:31.423 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-10T00:28:31.423 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-10T00:28:31.424 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-10T00:28:31.424 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-10T00:28:31.424 INFO:teuthology.orchestra.run.smithi131.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-10T00:28:31.424 INFO:teuthology.orchestra.run.smithi131.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-10T00:28:31.424 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-10T00:28:31.424 INFO:teuthology.orchestra.run.smithi131.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-10T00:28:31.425 INFO:teuthology.orchestra.run.smithi131.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-10T00:28:31.425 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-10T00:28:31.425 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-10T00:28:31.425 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-10T00:28:31.426 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-10T00:28:31.426 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-10T00:28:31.426 INFO:teuthology.orchestra.run.smithi131.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-10T00:28:31.426 INFO:teuthology.orchestra.run.smithi131.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-10T00:28:31.426 INFO:teuthology.orchestra.run.smithi131.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-10T00:28:31.427 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-10T00:28:31.427 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-10T00:28:31.427 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling module streams: 2023-09-10T00:28:31.427 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql 10 2023-09-10T00:28:31.427 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-10T00:28:31.427 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2023-09-10T00:28:31.428 INFO:teuthology.orchestra.run.smithi131.stdout:=========================================================================================================================== 2023-09-10T00:28:31.428 INFO:teuthology.orchestra.run.smithi131.stdout:Install 128 Packages 2023-09-10T00:28:31.428 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrade 3 Packages 2023-09-10T00:28:31.428 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-10T00:28:31.429 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 208 M 2023-09-10T00:28:31.429 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2023-09-10T00:28:31.836 INFO:teuthology.orchestra.run.smithi099.stdout:(4/131): ceph-base-18.0.0-6025.g9670cca5.el8.x8 4.7 MB/s | 5.4 MB 00:01 2023-09-10T00:28:31.895 INFO:teuthology.orchestra.run.smithi099.stdout:(5/131): ceph-mds-18.0.0-6025.g9670cca5.el8.x86 4.0 MB/s | 2.3 MB 00:00 2023-09-10T00:28:31.947 INFO:teuthology.orchestra.run.smithi131.stdout:(1/131): ceph-18.0.0-6025.g9670cca5.el8.x86_64. 30 kB/s | 6.4 kB 00:00 2023-09-10T00:28:32.136 INFO:teuthology.orchestra.run.smithi099.stdout:(6/131): ceph-mgr-18.0.0-6025.g9670cca5.el8.x86 5.4 MB/s | 1.6 MB 00:00 2023-09-10T00:28:32.553 INFO:teuthology.orchestra.run.smithi099.stdout:(7/131): ceph-common-18.0.0-6025.g9670cca5.el8. 12 MB/s | 22 MB 00:01 2023-09-10T00:28:32.754 INFO:teuthology.orchestra.run.smithi099.stdout:(8/131): ceph-mon-18.0.0-6025.g9670cca5.el8.x86 5.4 MB/s | 4.6 MB 00:00 2023-09-10T00:28:32.773 INFO:teuthology.orchestra.run.smithi131.stdout:(2/131): ceph-base-18.0.0-6025.g9670cca5.el8.x8 5.2 MB/s | 5.4 MB 00:01 2023-09-10T00:28:32.804 INFO:teuthology.orchestra.run.smithi099.stdout:(9/131): ceph-selinux-18.0.0-6025.g9670cca5.el8 486 kB/s | 24 kB 00:00 2023-09-10T00:28:32.833 INFO:teuthology.orchestra.run.smithi131.stdout:(3/131): ceph-fuse-18.0.0-6025.g9670cca5.el8.x8 1.0 MB/s | 907 kB 00:00 2023-09-10T00:28:32.865 INFO:teuthology.orchestra.run.smithi131.stdout:(4/131): ceph-immutable-object-cache-18.0.0-602 1.8 MB/s | 167 kB 00:00 2023-09-10T00:28:33.174 INFO:teuthology.orchestra.run.smithi131.stdout:(5/131): ceph-mgr-18.0.0-6025.g9670cca5.el8.x86 5.2 MB/s | 1.6 MB 00:00 2023-09-10T00:28:33.567 INFO:teuthology.orchestra.run.smithi044.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-10T00:28:33.568 INFO:teuthology.orchestra.run.smithi044.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-10T00:28:33.583 INFO:teuthology.orchestra.run.smithi044.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-10T00:28:33.584 INFO:teuthology.orchestra.run.smithi044.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-10T00:28:33.586 INFO:teuthology.orchestra.run.smithi044.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-10T00:28:33.587 INFO:teuthology.orchestra.run.smithi044.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-10T00:28:33.589 INFO:teuthology.orchestra.run.smithi044.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-10T00:28:33.592 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-10T00:28:33.593 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-10T00:28:33.736 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2023-09-10T00:28:33.749 INFO:teuthology.orchestra.run.smithi044.stdout:=========================================================================================================================== 2023-09-10T00:28:33.749 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2023-09-10T00:28:33.749 INFO:teuthology.orchestra.run.smithi044.stdout:=========================================================================================================================== 2023-09-10T00:28:33.750 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2023-09-10T00:28:33.750 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-10T00:28:33.750 INFO:teuthology.orchestra.run.smithi044.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-10T00:28:33.750 INFO:teuthology.orchestra.run.smithi044.stdout: ceph x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 6.4 k 2023-09-10T00:28:33.750 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 5.4 M 2023-09-10T00:28:33.751 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 907 k 2023-09-10T00:28:33.751 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 167 k 2023-09-10T00:28:33.751 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 1.6 M 2023-09-10T00:28:33.751 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 132 k 2023-09-10T00:28:33.751 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 1.7 M 2023-09-10T00:28:33.752 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 7.4 M 2023-09-10T00:28:33.752 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 47 k 2023-09-10T00:28:33.752 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 14 M 2023-09-10T00:28:33.752 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 49 M 2023-09-10T00:28:33.752 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 261 k 2023-09-10T00:28:33.753 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 176 k 2023-09-10T00:28:33.753 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 260 k 2023-09-10T00:28:33.753 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 18 k 2023-09-10T00:28:33.753 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-10T00:28:33.753 INFO:teuthology.orchestra.run.smithi044.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-10T00:28:33.754 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-10T00:28:33.754 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 31 k 2023-09-10T00:28:33.754 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 827 k 2023-09-10T00:28:33.754 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 139 k 2023-09-10T00:28:33.754 INFO:teuthology.orchestra.run.smithi044.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-10T00:28:33.755 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-10T00:28:33.755 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-10T00:28:33.755 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-10T00:28:33.755 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-10T00:28:33.756 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-10T00:28:33.756 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 197 k 2023-09-10T00:28:33.756 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 372 k 2023-09-10T00:28:33.756 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 364 k 2023-09-10T00:28:33.756 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 107 k 2023-09-10T00:28:33.756 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 94 k 2023-09-10T00:28:33.757 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.6 M 2023-09-10T00:28:33.757 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 179 k 2023-09-10T00:28:33.757 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-10T00:28:33.757 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2023-09-10T00:28:33.757 INFO:teuthology.orchestra.run.smithi044.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-10T00:28:33.758 INFO:teuthology.orchestra.run.smithi044.stdout: librados2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.7 M 2023-09-10T00:28:33.758 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.7 M 2023-09-10T00:28:33.758 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2023-09-10T00:28:33.758 INFO:teuthology.orchestra.run.smithi044.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-10T00:28:33.759 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 22 M 2023-09-10T00:28:33.759 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 23 k 2023-09-10T00:28:33.759 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 2.3 M 2023-09-10T00:28:33.759 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 242 k 2023-09-10T00:28:33.759 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 4.6 M 2023-09-10T00:28:33.760 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 18 M 2023-09-10T00:28:33.760 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 15 k 2023-09-10T00:28:33.760 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 24 k 2023-09-10T00:28:33.760 INFO:teuthology.orchestra.run.smithi044.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-10T00:28:33.760 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-10T00:28:33.761 INFO:teuthology.orchestra.run.smithi044.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-10T00:28:33.761 INFO:teuthology.orchestra.run.smithi044.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-10T00:28:33.761 INFO:teuthology.orchestra.run.smithi044.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-10T00:28:33.761 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 184 k 2023-09-10T00:28:33.761 INFO:teuthology.orchestra.run.smithi044.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-10T00:28:33.761 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-10T00:28:33.762 INFO:teuthology.orchestra.run.smithi044.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-10T00:28:33.762 INFO:teuthology.orchestra.run.smithi044.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-10T00:28:33.762 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-10T00:28:33.762 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-10T00:28:33.762 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 519 k 2023-09-10T00:28:33.763 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-10T00:28:33.763 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 6.8 M 2023-09-10T00:28:33.763 INFO:teuthology.orchestra.run.smithi044.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-10T00:28:33.763 INFO:teuthology.orchestra.run.smithi044.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-10T00:28:33.763 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-10T00:28:33.764 INFO:teuthology.orchestra.run.smithi044.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-10T00:28:33.764 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-10T00:28:33.764 INFO:teuthology.orchestra.run.smithi044.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-10T00:28:33.764 INFO:teuthology.orchestra.run.smithi044.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-10T00:28:33.765 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-10T00:28:33.765 INFO:teuthology.orchestra.run.smithi044.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-10T00:28:33.765 INFO:teuthology.orchestra.run.smithi044.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-10T00:28:33.765 INFO:teuthology.orchestra.run.smithi044.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-10T00:28:33.765 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-10T00:28:33.765 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-10T00:28:33.766 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-10T00:28:33.766 INFO:teuthology.orchestra.run.smithi044.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-10T00:28:33.766 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-10T00:28:33.766 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 45 k 2023-09-10T00:28:33.766 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 126 k 2023-09-10T00:28:33.767 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-10T00:28:33.767 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-10T00:28:33.767 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-10T00:28:33.767 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-10T00:28:33.767 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-10T00:28:33.768 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-10T00:28:33.768 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-10T00:28:33.768 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-10T00:28:33.768 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-10T00:28:33.768 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-10T00:28:33.768 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-10T00:28:33.769 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-10T00:28:33.769 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-10T00:28:33.769 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-10T00:28:33.769 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-10T00:28:33.769 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-10T00:28:33.769 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-10T00:28:33.770 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-10T00:28:33.770 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-10T00:28:33.770 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-10T00:28:33.770 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-10T00:28:33.770 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-10T00:28:33.771 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-10T00:28:33.771 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-10T00:28:33.771 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-10T00:28:33.771 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-10T00:28:33.771 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-10T00:28:33.772 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-10T00:28:33.772 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-10T00:28:33.772 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-10T00:28:33.772 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-10T00:28:33.772 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-10T00:28:33.772 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-10T00:28:33.773 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-10T00:28:33.773 INFO:teuthology.orchestra.run.smithi044.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-10T00:28:33.773 INFO:teuthology.orchestra.run.smithi044.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-10T00:28:33.773 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-10T00:28:33.773 INFO:teuthology.orchestra.run.smithi044.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-10T00:28:33.773 INFO:teuthology.orchestra.run.smithi044.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-10T00:28:33.774 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-10T00:28:33.774 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-10T00:28:33.774 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-10T00:28:33.774 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-10T00:28:33.774 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-10T00:28:33.775 INFO:teuthology.orchestra.run.smithi044.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-10T00:28:33.775 INFO:teuthology.orchestra.run.smithi044.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-10T00:28:33.775 INFO:teuthology.orchestra.run.smithi044.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-10T00:28:33.775 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-10T00:28:33.775 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-10T00:28:33.776 INFO:teuthology.orchestra.run.smithi044.stdout:Enabling module streams: 2023-09-10T00:28:33.776 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql 10 2023-09-10T00:28:33.776 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:28:33.776 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2023-09-10T00:28:33.776 INFO:teuthology.orchestra.run.smithi044.stdout:=========================================================================================================================== 2023-09-10T00:28:33.777 INFO:teuthology.orchestra.run.smithi044.stdout:Install 128 Packages 2023-09-10T00:28:33.777 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 3 Packages 2023-09-10T00:28:33.777 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:28:33.778 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 208 M 2023-09-10T00:28:33.778 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2023-09-10T00:28:34.082 INFO:teuthology.orchestra.run.smithi131.stdout:(6/131): ceph-mon-18.0.0-6025.g9670cca5.el8.x86 5.1 MB/s | 4.6 MB 00:00 2023-09-10T00:28:34.132 INFO:teuthology.orchestra.run.smithi044.stdout:(1/131): ceph-18.0.0-6025.g9670cca5.el8.x86_64. 21 kB/s | 6.4 kB 00:00 2023-09-10T00:28:34.262 INFO:teuthology.orchestra.run.smithi099.stdout:(10/131): ceph-radosgw-18.0.0-6025.g9670cca5.el 8.0 MB/s | 14 MB 00:01 2023-09-10T00:28:34.363 INFO:teuthology.orchestra.run.smithi099.stdout:(11/131): cephfs-mirror-18.0.0-6025.g9670cca5.e 2.5 MB/s | 260 kB 00:00 2023-09-10T00:28:34.463 INFO:teuthology.orchestra.run.smithi099.stdout:(12/131): libcephfs-devel-18.0.0-6025.g9670cca5 311 kB/s | 31 kB 00:00 2023-09-10T00:28:34.733 INFO:teuthology.orchestra.run.smithi044.stdout:(2/131): ceph-fuse-18.0.0-6025.g9670cca5.el8.x8 1.5 MB/s | 907 kB 00:00 2023-09-10T00:28:34.816 INFO:teuthology.orchestra.run.smithi131.stdout:(7/131): ceph-mds-18.0.0-6025.g9670cca5.el8.x86 1.2 MB/s | 2.3 MB 00:01 2023-09-10T00:28:34.855 INFO:teuthology.orchestra.run.smithi099.stdout:(13/131): libcephfs2-18.0.0-6025.g9670cca5.el8. 2.1 MB/s | 827 kB 00:00 2023-09-10T00:28:34.891 INFO:teuthology.orchestra.run.smithi044.stdout:(3/131): ceph-immutable-object-cache-18.0.0-602 1.0 MB/s | 167 kB 00:00 2023-09-10T00:28:34.955 INFO:teuthology.orchestra.run.smithi099.stdout:(14/131): libcephsqlite-18.0.0-6025.g9670cca5.e 1.8 MB/s | 184 kB 00:00 2023-09-10T00:28:35.064 INFO:teuthology.orchestra.run.smithi099.stdout:(15/131): librados-devel-18.0.0-6025.g9670cca5. 1.3 MB/s | 139 kB 00:00 2023-09-10T00:28:35.422 INFO:teuthology.orchestra.run.smithi099.stdout:(16/131): libradosstriper1-18.0.0-6025.g9670cca 1.4 MB/s | 519 kB 00:00 2023-09-10T00:28:35.698 INFO:teuthology.orchestra.run.smithi099.stdout:(17/131): ceph-osd-18.0.0-6025.g9670cca5.el8.x8 5.0 MB/s | 18 MB 00:03 2023-09-10T00:28:35.798 INFO:teuthology.orchestra.run.smithi099.stdout:(18/131): python3-ceph-argparse-18.0.0-6025.g96 449 kB/s | 45 kB 00:00 2023-09-10T00:28:35.849 INFO:teuthology.orchestra.run.smithi044.stdout:(4/131): ceph-mds-18.0.0-6025.g9670cca5.el8.x86 2.4 MB/s | 2.3 MB 00:00 2023-09-10T00:28:35.907 INFO:teuthology.orchestra.run.smithi099.stdout:(19/131): python3-ceph-common-18.0.0-6025.g9670 1.1 MB/s | 126 kB 00:00 2023-09-10T00:28:36.015 INFO:teuthology.orchestra.run.smithi099.stdout:(20/131): python3-cephfs-18.0.0-6025.g9670cca5. 1.8 MB/s | 197 kB 00:00 2023-09-10T00:28:36.141 INFO:teuthology.orchestra.run.smithi099.stdout:(21/131): python3-rados-18.0.0-6025.g9670cca5.e 2.9 MB/s | 372 kB 00:00 2023-09-10T00:28:36.258 INFO:teuthology.orchestra.run.smithi099.stdout:(22/131): python3-rbd-18.0.0-6025.g9670cca5.el8 3.1 MB/s | 364 kB 00:00 2023-09-10T00:28:36.275 INFO:teuthology.orchestra.run.smithi044.stdout:(5/131): ceph-base-18.0.0-6025.g9670cca5.el8.x8 2.2 MB/s | 5.4 MB 00:02 2023-09-10T00:28:36.333 INFO:teuthology.orchestra.run.smithi099.stdout:(23/131): python3-rgw-18.0.0-6025.g9670cca5.el8 1.4 MB/s | 107 kB 00:00 2023-09-10T00:28:36.433 INFO:teuthology.orchestra.run.smithi099.stdout:(24/131): rbd-fuse-18.0.0-6025.g9670cca5.el8.x8 937 kB/s | 94 kB 00:00 2023-09-10T00:28:36.492 INFO:teuthology.orchestra.run.smithi044.stdout:(6/131): ceph-mgr-18.0.0-6025.g9670cca5.el8.x86 2.5 MB/s | 1.6 MB 00:00 2023-09-10T00:28:37.075 INFO:teuthology.orchestra.run.smithi099.stdout:(25/131): librgw2-18.0.0-6025.g9670cca5.el8.x86 4.1 MB/s | 6.8 MB 00:01 2023-09-10T00:28:37.201 INFO:teuthology.orchestra.run.smithi099.stdout:(26/131): rbd-nbd-18.0.0-6025.g9670cca5.el8.x86 1.4 MB/s | 179 kB 00:00 2023-09-10T00:28:37.293 INFO:teuthology.orchestra.run.smithi099.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6025.g 252 kB/s | 23 kB 00:00 2023-09-10T00:28:37.351 INFO:teuthology.orchestra.run.smithi099.stdout:(28/131): rbd-mirror-18.0.0-6025.g9670cca5.el8. 3.9 MB/s | 3.6 MB 00:00 2023-09-10T00:28:37.385 INFO:teuthology.orchestra.run.smithi099.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6025.g9670cca 1.4 MB/s | 132 kB 00:00 2023-09-10T00:28:37.800 INFO:teuthology.orchestra.run.smithi044.stdout:(7/131): ceph-mon-18.0.0-6025.g9670cca5.el8.x86 3.0 MB/s | 4.6 MB 00:01 2023-09-10T00:28:37.852 INFO:teuthology.orchestra.run.smithi099.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6025.g9670c 3.5 MB/s | 1.7 MB 00:00 2023-09-10T00:28:37.952 INFO:teuthology.orchestra.run.smithi099.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6025.g96 2.4 MB/s | 242 kB 00:00 2023-09-10T00:28:38.036 INFO:teuthology.orchestra.run.smithi099.stdout:(32/131): ceph-mgr-rook-18.0.0-6025.g9670cca5.e 570 kB/s | 47 kB 00:00 2023-09-10T00:28:38.119 INFO:teuthology.orchestra.run.smithi099.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6025.g9 174 kB/s | 15 kB 00:00 2023-09-10T00:28:38.228 INFO:teuthology.orchestra.run.smithi099.stdout:(34/131): ceph-volume-18.0.0-6025.g9670cca5.el8 2.4 MB/s | 261 kB 00:00 2023-09-10T00:28:38.312 INFO:teuthology.orchestra.run.smithi099.stdout:(35/131): cephadm-18.0.0-6025.g9670cca5.el8.noa 2.1 MB/s | 176 kB 00:00 2023-09-10T00:28:38.357 INFO:teuthology.orchestra.run.smithi131.stdout:(8/131): ceph-osd-18.0.0-6025.g9670cca5.el8.x86 4.2 MB/s | 18 MB 00:04 2023-09-10T00:28:38.387 INFO:teuthology.orchestra.run.smithi099.stdout:(36/131): cephfs-top-18.0.0-6025.g9670cca5.el8. 240 kB/s | 18 kB 00:00 2023-09-10T00:28:38.433 INFO:teuthology.orchestra.run.smithi131.stdout:(9/131): ceph-selinux-18.0.0-6025.g9670cca5.el8 326 kB/s | 24 kB 00:00 2023-09-10T00:28:38.962 INFO:teuthology.orchestra.run.smithi099.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 533 kB/s | 306 kB 00:00 2023-09-10T00:28:39.063 INFO:teuthology.orchestra.run.smithi099.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 589 kB/s | 59 kB 00:00 2023-09-10T00:28:39.169 INFO:teuthology.orchestra.run.smithi099.stdout:(39/131): ceph-mgr-diskprediction-local-18.0.0- 4.1 MB/s | 7.4 MB 00:01 2023-09-10T00:28:39.197 INFO:teuthology.orchestra.run.smithi099.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 562 kB/s | 75 kB 00:00 2023-09-10T00:28:39.322 INFO:teuthology.orchestra.run.smithi099.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 349 kB/s | 44 kB 00:00 2023-09-10T00:28:39.530 INFO:teuthology.orchestra.run.smithi099.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 889 kB/s | 185 kB 00:00 2023-09-10T00:28:39.622 INFO:teuthology.orchestra.run.smithi099.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 360 kB/s | 33 kB 00:00 2023-09-10T00:28:39.698 INFO:teuthology.orchestra.run.smithi099.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 217 kB/s | 16 kB 00:00 2023-09-10T00:28:39.798 INFO:teuthology.orchestra.run.smithi099.stdout:(45/131): python3-asyncssh-2.7.0-2.el8.noarch.r 757 kB/s | 476 kB 00:00 2023-09-10T00:28:39.823 INFO:teuthology.orchestra.run.smithi099.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-09-10T00:28:39.907 INFO:teuthology.orchestra.run.smithi099.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2023-09-10T00:28:39.933 INFO:teuthology.orchestra.run.smithi099.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 747 kB/s | 81 kB 00:00 2023-09-10T00:28:39.974 INFO:teuthology.orchestra.run.smithi099.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2023-09-10T00:28:40.008 INFO:teuthology.orchestra.run.smithi099.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 234 kB/s | 18 kB 00:00 2023-09-10T00:28:40.084 INFO:teuthology.orchestra.run.smithi099.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 649 kB/s | 49 kB 00:00 2023-09-10T00:28:40.134 INFO:teuthology.orchestra.run.smithi099.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 7.3 MB/s | 1.2 MB 00:00 2023-09-10T00:28:40.184 INFO:teuthology.orchestra.run.smithi099.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 585 kB/s | 59 kB 00:00 2023-09-10T00:28:40.210 INFO:teuthology.orchestra.run.smithi099.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 801 kB/s | 60 kB 00:00 2023-09-10T00:28:40.275 INFO:teuthology.orchestra.run.smithi131.stdout:(10/131): ceph-common-18.0.0-6025.g9670cca5.el8 2.6 MB/s | 22 MB 00:08 2023-09-10T00:28:40.285 INFO:teuthology.orchestra.run.smithi099.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 208 kB/s | 16 kB 00:00 2023-09-10T00:28:40.310 INFO:teuthology.orchestra.run.smithi099.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2023-09-10T00:28:40.352 INFO:teuthology.orchestra.run.smithi099.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 504 kB/s | 34 kB 00:00 2023-09-10T00:28:40.408 INFO:teuthology.orchestra.run.smithi131.stdout:(11/131): cephfs-mirror-18.0.0-6025.g9670cca5.e 1.9 MB/s | 260 kB 00:00 2023-09-10T00:28:40.467 INFO:teuthology.orchestra.run.smithi131.stdout:(12/131): libcephfs-devel-18.0.0-6025.g9670cca5 535 kB/s | 31 kB 00:00 2023-09-10T00:28:40.591 INFO:teuthology.orchestra.run.smithi099.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 700 kB/s | 196 kB 00:00 2023-09-10T00:28:40.709 INFO:teuthology.orchestra.run.smithi044.stdout:(8/131): ceph-osd-18.0.0-6025.g9670cca5.el8.x86 4.2 MB/s | 18 MB 00:04 2023-09-10T00:28:40.709 INFO:teuthology.orchestra.run.smithi131.stdout:(13/131): libcephfs2-18.0.0-6025.g9670cca5.el8. 3.3 MB/s | 827 kB 00:00 2023-09-10T00:28:40.767 INFO:teuthology.orchestra.run.smithi044.stdout:(9/131): ceph-selinux-18.0.0-6025.g9670cca5.el8 419 kB/s | 24 kB 00:00 2023-09-10T00:28:40.776 INFO:teuthology.orchestra.run.smithi099.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 151 kB/s | 64 kB 00:00 2023-09-10T00:28:40.792 INFO:teuthology.orchestra.run.smithi131.stdout:(14/131): libcephsqlite-18.0.0-6025.g9670cca5.e 2.2 MB/s | 184 kB 00:00 2023-09-10T00:28:40.803 INFO:teuthology.orchestra.run.smithi099.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 88 kB/s | 19 kB 00:00 2023-09-10T00:28:40.845 INFO:teuthology.orchestra.run.smithi099.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 348 kB/s | 24 kB 00:00 2023-09-10T00:28:40.876 INFO:teuthology.orchestra.run.smithi131.stdout:(15/131): librados-devel-18.0.0-6025.g9670cca5. 1.6 MB/s | 139 kB 00:00 2023-09-10T00:28:40.879 INFO:teuthology.orchestra.run.smithi099.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 389 kB/s | 29 kB 00:00 2023-09-10T00:28:40.913 INFO:teuthology.orchestra.run.smithi099.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 411 kB/s | 27 kB 00:00 2023-09-10T00:28:40.980 INFO:teuthology.orchestra.run.smithi099.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-09-10T00:28:41.005 INFO:teuthology.orchestra.run.smithi099.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2023-09-10T00:28:41.047 INFO:teuthology.orchestra.run.smithi099.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 916 kB/s | 61 kB 00:00 2023-09-10T00:28:41.068 INFO:teuthology.orchestra.run.smithi131.stdout:(16/131): libradosstriper1-18.0.0-6025.g9670cca 2.6 MB/s | 519 kB 00:00 2023-09-10T00:28:41.081 INFO:teuthology.orchestra.run.smithi099.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-09-10T00:28:41.114 INFO:teuthology.orchestra.run.smithi099.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 342 kB/s | 23 kB 00:00 2023-09-10T00:28:41.139 INFO:teuthology.orchestra.run.smithi099.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-09-10T00:28:41.207 INFO:teuthology.orchestra.run.smithi099.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 14 MB/s | 1.7 MB 00:00 2023-09-10T00:28:41.265 INFO:teuthology.orchestra.run.smithi099.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-09-10T00:28:41.326 INFO:teuthology.orchestra.run.smithi044.stdout:(10/131): ceph-radosgw-18.0.0-6025.g9670cca5.el 3.9 MB/s | 14 MB 00:03 2023-09-10T00:28:41.434 INFO:teuthology.orchestra.run.smithi044.stdout:(11/131): cephfs-mirror-18.0.0-6025.g9670cca5.e 2.4 MB/s | 260 kB 00:00 2023-09-10T00:28:41.535 INFO:teuthology.orchestra.run.smithi044.stdout:(12/131): libcephfs-devel-18.0.0-6025.g9670cca5 310 kB/s | 31 kB 00:00 2023-09-10T00:28:41.707 INFO:teuthology.orchestra.run.smithi099.stdout:(72/131): ceph-test-18.0.0-6025.g9670cca5.el8.x 5.5 MB/s | 49 MB 00:08 2023-09-10T00:28:41.757 INFO:teuthology.orchestra.run.smithi099.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 110 kB/s | 54 kB 00:00 2023-09-10T00:28:41.760 INFO:teuthology.orchestra.run.smithi044.stdout:(13/131): libcephfs2-18.0.0-6025.g9670cca5.el8. 3.6 MB/s | 827 kB 00:00 2023-09-10T00:28:41.808 INFO:teuthology.orchestra.run.smithi099.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.5 MB/s | 259 kB 00:00 2023-09-10T00:28:41.833 INFO:teuthology.orchestra.run.smithi099.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.5 MB/s | 323 kB 00:00 2023-09-10T00:28:41.835 INFO:teuthology.orchestra.run.smithi044.stdout:(14/131): libcephsqlite-18.0.0-6025.g9670cca5.e 2.4 MB/s | 184 kB 00:00 2023-09-10T00:28:41.858 INFO:teuthology.orchestra.run.smithi099.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 771 kB/s | 39 kB 00:00 2023-09-10T00:28:41.892 INFO:teuthology.orchestra.run.smithi099.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 11 MB/s | 688 kB 00:00 2023-09-10T00:28:41.911 INFO:teuthology.orchestra.run.smithi044.stdout:(15/131): librados-devel-18.0.0-6025.g9670cca5. 1.8 MB/s | 139 kB 00:00 2023-09-10T00:28:41.926 INFO:teuthology.orchestra.run.smithi099.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.8 MB/s | 538 kB 00:00 2023-09-10T00:28:41.951 INFO:teuthology.orchestra.run.smithi099.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-09-10T00:28:41.976 INFO:teuthology.orchestra.run.smithi099.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-09-10T00:28:42.027 INFO:teuthology.orchestra.run.smithi099.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 9.3 MB/s | 713 kB 00:00 2023-09-10T00:28:42.068 INFO:teuthology.orchestra.run.smithi099.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 641 kB/s | 27 kB 00:00 2023-09-10T00:28:42.105 INFO:teuthology.orchestra.run.smithi044.stdout:(16/131): libradosstriper1-18.0.0-6025.g9670cca 2.6 MB/s | 519 kB 00:00 2023-09-10T00:28:42.119 INFO:teuthology.orchestra.run.smithi099.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-09-10T00:28:42.161 INFO:teuthology.orchestra.run.smithi099.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-09-10T00:28:42.270 INFO:teuthology.orchestra.run.smithi099.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2023-09-10T00:28:42.320 INFO:teuthology.orchestra.run.smithi099.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-09-10T00:28:42.353 INFO:teuthology.orchestra.run.smithi044.stdout:(17/131): ceph-common-18.0.0-6025.g9670cca5.el8 2.6 MB/s | 22 MB 00:08 2023-09-10T00:28:42.420 INFO:teuthology.orchestra.run.smithi044.stdout:(18/131): python3-ceph-argparse-18.0.0-6025.g96 673 kB/s | 45 kB 00:00 2023-09-10T00:28:42.462 INFO:teuthology.orchestra.run.smithi099.stdout:(87/131): python3-scipy-1.0.0-21.module+el8.5.0 10 MB/s | 14 MB 00:01 2023-09-10T00:28:42.487 INFO:teuthology.orchestra.run.smithi099.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.3 MB/s | 225 kB 00:00 2023-09-10T00:28:42.512 INFO:teuthology.orchestra.run.smithi099.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-09-10T00:28:42.538 INFO:teuthology.orchestra.run.smithi099.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-09-10T00:28:42.545 INFO:teuthology.orchestra.run.smithi044.stdout:(19/131): python3-ceph-common-18.0.0-6025.g9670 1.0 MB/s | 126 kB 00:00 2023-09-10T00:28:42.563 INFO:teuthology.orchestra.run.smithi099.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-09-10T00:28:42.585 INFO:teuthology.orchestra.run.smithi131.stdout:(17/131): librgw2-18.0.0-6025.g9670cca5.el8.x86 4.5 MB/s | 6.8 MB 00:01 2023-09-10T00:28:42.588 INFO:teuthology.orchestra.run.smithi099.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-09-10T00:28:42.652 INFO:teuthology.orchestra.run.smithi131.stdout:(18/131): python3-ceph-argparse-18.0.0-6025.g96 673 kB/s | 45 kB 00:00 2023-09-10T00:28:42.655 INFO:teuthology.orchestra.run.smithi099.stdout:(93/131): python3-numpy-1.14.3-10.el8.x86_64.rp 7.6 MB/s | 3.7 MB 00:00 2023-09-10T00:28:42.687 INFO:teuthology.orchestra.run.smithi044.stdout:(20/131): python3-cephfs-18.0.0-6025.g9670cca5. 1.4 MB/s | 197 kB 00:00 2023-09-10T00:28:42.697 INFO:teuthology.orchestra.run.smithi099.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 5.2 MB/s | 709 kB 00:00 2023-09-10T00:28:42.719 INFO:teuthology.orchestra.run.smithi131.stdout:(19/131): python3-ceph-common-18.0.0-6025.g9670 1.8 MB/s | 126 kB 00:00 2023-09-10T00:28:42.722 INFO:teuthology.orchestra.run.smithi099.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2023-09-10T00:28:42.789 INFO:teuthology.orchestra.run.smithi099.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.0 MB/s | 202 kB 00:00 2023-09-10T00:28:42.831 INFO:teuthology.orchestra.run.smithi099.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-09-10T00:28:42.837 INFO:teuthology.orchestra.run.smithi044.stdout:(21/131): python3-rados-18.0.0-6025.g9670cca5.e 2.4 MB/s | 372 kB 00:00 2023-09-10T00:28:42.861 INFO:teuthology.orchestra.run.smithi131.stdout:(20/131): ceph-radosgw-18.0.0-6025.g9670cca5.el 1.7 MB/s | 14 MB 00:08 2023-09-10T00:28:42.886 INFO:teuthology.orchestra.run.smithi131.stdout:(21/131): python3-cephfs-18.0.0-6025.g9670cca5. 1.2 MB/s | 197 kB 00:00 2023-09-10T00:28:42.932 INFO:teuthology.orchestra.run.smithi099.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 14 MB/s | 4.9 MB 00:00 2023-09-10T00:28:42.954 INFO:teuthology.orchestra.run.smithi044.stdout:(22/131): python3-rbd-18.0.0-6025.g9670cca5.el8 3.1 MB/s | 364 kB 00:00 2023-09-10T00:28:43.007 INFO:teuthology.orchestra.run.smithi099.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-09-10T00:28:43.011 INFO:teuthology.orchestra.run.smithi131.stdout:(22/131): python3-rbd-18.0.0-6025.g9670cca5.el8 2.8 MB/s | 364 kB 00:00 2023-09-10T00:28:43.037 INFO:teuthology.orchestra.run.smithi131.stdout:(23/131): python3-rados-18.0.0-6025.g9670cca5.e 2.1 MB/s | 372 kB 00:00 2023-09-10T00:28:43.049 INFO:teuthology.orchestra.run.smithi099.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-09-10T00:28:43.079 INFO:teuthology.orchestra.run.smithi131.stdout:(24/131): python3-rgw-18.0.0-6025.g9670cca5.el8 1.6 MB/s | 107 kB 00:00 2023-09-10T00:28:43.082 INFO:teuthology.orchestra.run.smithi099.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-09-10T00:28:43.104 INFO:teuthology.orchestra.run.smithi131.stdout:(25/131): rbd-fuse-18.0.0-6025.g9670cca5.el8.x8 1.4 MB/s | 94 kB 00:00 2023-09-10T00:28:43.113 INFO:teuthology.orchestra.run.smithi044.stdout:(23/131): python3-rgw-18.0.0-6025.g9670cca5.el8 673 kB/s | 107 kB 00:00 2023-09-10T00:28:43.125 INFO:teuthology.orchestra.run.smithi099.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-09-10T00:28:43.150 INFO:teuthology.orchestra.run.smithi099.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-09-10T00:28:43.167 INFO:teuthology.orchestra.run.smithi099.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-09-10T00:28:43.192 INFO:teuthology.orchestra.run.smithi099.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.1 MB/s | 220 kB 00:00 2023-09-10T00:28:43.209 INFO:teuthology.orchestra.run.smithi099.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 1.1 MB/s | 48 kB 00:00 2023-09-10T00:28:43.221 INFO:teuthology.orchestra.run.smithi131.stdout:(26/131): rbd-nbd-18.0.0-6025.g9670cca5.el8.x86 1.5 MB/s | 179 kB 00:00 2023-09-10T00:28:43.253 INFO:teuthology.orchestra.run.smithi099.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.4 MB/s | 87 kB 00:00 2023-09-10T00:28:43.276 INFO:teuthology.orchestra.run.smithi044.stdout:(24/131): rbd-fuse-18.0.0-6025.g9670cca5.el8.x8 575 kB/s | 94 kB 00:00 2023-09-10T00:28:43.277 INFO:teuthology.orchestra.run.smithi099.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 636 kB/s | 43 kB 00:00 2023-09-10T00:28:43.302 INFO:teuthology.orchestra.run.smithi099.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.2 MB/s | 109 kB 00:00 2023-09-10T00:28:43.315 INFO:teuthology.orchestra.run.smithi131.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6025.g 246 kB/s | 23 kB 00:00 2023-09-10T00:28:43.327 INFO:teuthology.orchestra.run.smithi099.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-09-10T00:28:43.394 INFO:teuthology.orchestra.run.smithi099.stdout:(111/131): postgresql-server-10.21-2.module+el8 9.0 MB/s | 5.1 MB 00:00 2023-09-10T00:28:43.399 INFO:teuthology.orchestra.run.smithi131.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6025.g9670cca 1.5 MB/s | 132 kB 00:00 2023-09-10T00:28:43.419 INFO:teuthology.orchestra.run.smithi099.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 203 kB/s | 19 kB 00:00 2023-09-10T00:28:43.453 INFO:teuthology.orchestra.run.smithi099.stdout:(113/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.6 MB/s | 560 kB 00:00 2023-09-10T00:28:43.478 INFO:teuthology.orchestra.run.smithi099.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.8 MB/s | 155 kB 00:00 2023-09-10T00:28:43.504 INFO:teuthology.orchestra.run.smithi099.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 357 kB/s | 18 kB 00:00 2023-09-10T00:28:43.546 INFO:teuthology.orchestra.run.smithi099.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.3 MB/s | 559 kB 00:00 2023-09-10T00:28:43.571 INFO:teuthology.orchestra.run.smithi099.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.4 MB/s | 605 kB 00:00 2023-09-10T00:28:43.596 INFO:teuthology.orchestra.run.smithi099.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-09-10T00:28:43.622 INFO:teuthology.orchestra.run.smithi099.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-09-10T00:28:43.631 INFO:teuthology.orchestra.run.smithi044.stdout:(25/131): librgw2-18.0.0-6025.g9670cca5.el8.x86 4.5 MB/s | 6.8 MB 00:01 2023-09-10T00:28:43.647 INFO:teuthology.orchestra.run.smithi099.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-09-10T00:28:43.672 INFO:teuthology.orchestra.run.smithi099.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-09-10T00:28:43.697 INFO:teuthology.orchestra.run.smithi099.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-09-10T00:28:43.714 INFO:teuthology.orchestra.run.smithi044.stdout:(26/131): rbd-nbd-18.0.0-6025.g9670cca5.el8.x86 2.1 MB/s | 179 kB 00:00 2023-09-10T00:28:43.723 INFO:teuthology.orchestra.run.smithi099.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-09-10T00:28:43.773 INFO:teuthology.orchestra.run.smithi099.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 512 kB/s | 39 kB 00:00 2023-09-10T00:28:43.790 INFO:teuthology.orchestra.run.smithi044.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6025.g 308 kB/s | 23 kB 00:00 2023-09-10T00:28:43.815 INFO:teuthology.orchestra.run.smithi099.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 6.8 MB/s | 644 kB 00:00 2023-09-10T00:28:43.841 INFO:teuthology.orchestra.run.smithi099.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-09-10T00:28:43.866 INFO:teuthology.orchestra.run.smithi099.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-09-10T00:28:43.896 INFO:teuthology.orchestra.run.smithi044.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6025.g9670cca 1.2 MB/s | 132 kB 00:00 2023-09-10T00:28:43.916 INFO:teuthology.orchestra.run.smithi099.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.3 MB/s | 2.3 MB 00:00 2023-09-10T00:28:43.941 INFO:teuthology.orchestra.run.smithi131.stdout:(29/131): rbd-mirror-18.0.0-6025.g9670cca5.el8. 4.2 MB/s | 3.6 MB 00:00 2023-09-10T00:28:43.975 INFO:teuthology.orchestra.run.smithi099.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-09-10T00:28:44.091 INFO:teuthology.orchestra.run.smithi131.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6025.g9670c 2.5 MB/s | 1.7 MB 00:00 2023-09-10T00:28:44.275 INFO:teuthology.orchestra.run.smithi131.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6025.g96 1.3 MB/s | 242 kB 00:00 2023-09-10T00:28:44.357 INFO:teuthology.orchestra.run.smithi044.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6025.g9670c 3.8 MB/s | 1.7 MB 00:00 2023-09-10T00:28:44.358 INFO:teuthology.orchestra.run.smithi131.stdout:(32/131): ceph-mgr-rook-18.0.0-6025.g9670cca5.e 569 kB/s | 47 kB 00:00 2023-09-10T00:28:44.415 INFO:teuthology.orchestra.run.smithi044.stdout:(30/131): rbd-mirror-18.0.0-6025.g9670cca5.el8. 3.1 MB/s | 3.6 MB 00:01 2023-09-10T00:28:44.442 INFO:teuthology.orchestra.run.smithi131.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6025.g9 174 kB/s | 15 kB 00:00 2023-09-10T00:28:44.582 INFO:teuthology.orchestra.run.smithi044.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6025.g96 1.4 MB/s | 242 kB 00:00 2023-09-10T00:28:44.609 INFO:teuthology.orchestra.run.smithi131.stdout:(34/131): ceph-volume-18.0.0-6025.g9670cca5.el8 1.5 MB/s | 261 kB 00:00 2023-09-10T00:28:44.683 INFO:teuthology.orchestra.run.smithi044.stdout:(32/131): ceph-mgr-rook-18.0.0-6025.g9670cca5.e 474 kB/s | 47 kB 00:00 2023-09-10T00:28:44.775 INFO:teuthology.orchestra.run.smithi044.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6025.g9 158 kB/s | 15 kB 00:00 2023-09-10T00:28:44.800 INFO:teuthology.orchestra.run.smithi099.stdout:(130/131): librados2-18.0.0-6025.g9670cca5.el8. 3.9 MB/s | 3.7 MB 00:00 2023-09-10T00:28:44.883 INFO:teuthology.orchestra.run.smithi044.stdout:(34/131): ceph-volume-18.0.0-6025.g9670cca5.el8 2.4 MB/s | 261 kB 00:00 2023-09-10T00:28:44.910 INFO:teuthology.orchestra.run.smithi099.stdout:(131/131): librbd1-18.0.0-6025.g9670cca5.el8.x8 3.5 MB/s | 3.7 MB 00:01 2023-09-10T00:28:44.914 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-09-10T00:28:44.914 INFO:teuthology.orchestra.run.smithi099.stdout:Total 14 MB/s | 208 MB 00:14 2023-09-10T00:28:44.958 INFO:teuthology.orchestra.run.smithi044.stdout:(35/131): cephadm-18.0.0-6025.g9670cca5.el8.noa 2.3 MB/s | 176 kB 00:00 2023-09-10T00:28:45.051 INFO:teuthology.orchestra.run.smithi044.stdout:(36/131): cephfs-top-18.0.0-6025.g9670cca5.el8. 196 kB/s | 18 kB 00:00 2023-09-10T00:28:45.126 INFO:teuthology.orchestra.run.smithi131.stdout:(35/131): ceph-test-18.0.0-6025.g9670cca5.el8.x 7.3 MB/s | 49 MB 00:06 2023-09-10T00:28:45.151 INFO:teuthology.orchestra.run.smithi131.stdout:(36/131): cephadm-18.0.0-6025.g9670cca5.el8.noa 325 kB/s | 176 kB 00:00 2023-09-10T00:28:45.177 INFO:teuthology.orchestra.run.smithi131.stdout:(37/131): cephfs-top-18.0.0-6025.g9670cca5.el8. 359 kB/s | 18 kB 00:00 2023-09-10T00:28:45.335 INFO:teuthology.orchestra.run.smithi131.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 372 kB/s | 59 kB 00:00 2023-09-10T00:28:45.436 INFO:teuthology.orchestra.run.smithi131.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2023-09-10T00:28:45.461 INFO:teuthology.orchestra.run.smithi131.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 599 kB/s | 75 kB 00:00 2023-09-10T00:28:45.526 INFO:teuthology.orchestra.run.smithi044.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 645 kB/s | 306 kB 00:00 2023-09-10T00:28:45.528 INFO:teuthology.orchestra.run.smithi131.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 654 kB/s | 44 kB 00:00 2023-09-10T00:28:45.553 INFO:teuthology.orchestra.run.smithi131.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.0 MB/s | 476 kB 00:00 2023-09-10T00:28:45.612 INFO:teuthology.orchestra.run.smithi131.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 566 kB/s | 33 kB 00:00 2023-09-10T00:28:45.643 INFO:teuthology.orchestra.run.smithi044.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 505 kB/s | 59 kB 00:00 2023-09-10T00:28:45.670 INFO:teuthology.orchestra.run.smithi131.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 280 kB/s | 16 kB 00:00 2023-09-10T00:28:45.729 INFO:teuthology.orchestra.run.smithi131.stdout:(45/131): python3-beautifulsoup4-4.6.3-2.el8.1. 922 kB/s | 185 kB 00:00 2023-09-10T00:28:45.739 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-09-10T00:28:45.754 INFO:teuthology.orchestra.run.smithi131.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2023-09-10T00:28:45.760 INFO:teuthology.orchestra.run.smithi044.stdout:(39/131): ceph-mgr-diskprediction-local-18.0.0- 5.3 MB/s | 7.4 MB 00:01 2023-09-10T00:28:45.777 INFO:teuthology.orchestra.run.smithi044.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 561 kB/s | 75 kB 00:00 2023-09-10T00:28:45.813 INFO:teuthology.orchestra.run.smithi131.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-09-10T00:28:45.859 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-09-10T00:28:45.860 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-09-10T00:28:45.877 INFO:teuthology.orchestra.run.smithi044.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 436 kB/s | 44 kB 00:00 2023-09-10T00:28:45.905 INFO:teuthology.orchestra.run.smithi131.stdout:(48/131): ceph-mgr-diskprediction-local-18.0.0- 3.8 MB/s | 7.4 MB 00:01 2023-09-10T00:28:45.922 INFO:teuthology.orchestra.run.smithi131.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2023-09-10T00:28:45.947 INFO:teuthology.orchestra.run.smithi131.stdout:(50/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.7 MB/s | 384 kB 00:00 2023-09-10T00:28:45.964 INFO:teuthology.orchestra.run.smithi131.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 299 kB/s | 18 kB 00:00 2023-09-10T00:28:46.023 INFO:teuthology.orchestra.run.smithi131.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 645 kB/s | 49 kB 00:00 2023-09-10T00:28:46.040 INFO:teuthology.orchestra.run.smithi131.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 777 kB/s | 59 kB 00:00 2023-09-10T00:28:46.052 INFO:teuthology.orchestra.run.smithi044.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.0 MB/s | 185 kB 00:00 2023-09-10T00:28:46.074 INFO:teuthology.orchestra.run.smithi131.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-09-10T00:28:46.107 INFO:teuthology.orchestra.run.smithi131.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2023-09-10T00:28:46.124 INFO:teuthology.orchestra.run.smithi131.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 310 kB/s | 16 kB 00:00 2023-09-10T00:28:46.158 INFO:teuthology.orchestra.run.smithi131.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 669 kB/s | 34 kB 00:00 2023-09-10T00:28:46.161 INFO:teuthology.orchestra.run.smithi044.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 304 kB/s | 33 kB 00:00 2023-09-10T00:28:46.184 INFO:teuthology.orchestra.run.smithi131.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.3 MB/s | 196 kB 00:00 2023-09-10T00:28:46.217 INFO:teuthology.orchestra.run.smithi131.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2023-09-10T00:28:46.242 INFO:teuthology.orchestra.run.smithi131.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 320 kB/s | 19 kB 00:00 2023-09-10T00:28:46.276 INFO:teuthology.orchestra.run.smithi131.stdout:(61/131): python3-kubernetes-11.0.0-6.el8.noarc 3.3 MB/s | 1.2 MB 00:00 2023-09-10T00:28:46.278 INFO:teuthology.orchestra.run.smithi044.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 140 kB/s | 16 kB 00:00 2023-09-10T00:28:46.311 INFO:teuthology.orchestra.run.smithi044.stdout:(45/131): python3-asyncssh-2.7.0-2.el8.noarch.r 863 kB/s | 476 kB 00:00 2023-09-10T00:28:46.326 INFO:teuthology.orchestra.run.smithi131.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 219 kB/s | 24 kB 00:00 2023-09-10T00:28:46.352 INFO:teuthology.orchestra.run.smithi131.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 268 kB/s | 29 kB 00:00 2023-09-10T00:28:46.377 INFO:teuthology.orchestra.run.smithi131.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2023-09-10T00:28:46.387 INFO:teuthology.orchestra.run.smithi044.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.6 MB/s | 173 kB 00:00 2023-09-10T00:28:46.419 INFO:teuthology.orchestra.run.smithi131.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 3.7 MB/s | 255 kB 00:00 2023-09-10T00:28:46.429 INFO:teuthology.orchestra.run.smithi044.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.2 MB/s | 384 kB 00:00 2023-09-10T00:28:46.444 INFO:teuthology.orchestra.run.smithi131.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2023-09-10T00:28:46.469 INFO:teuthology.orchestra.run.smithi131.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 665 kB/s | 61 kB 00:00 2023-09-10T00:28:46.494 INFO:teuthology.orchestra.run.smithi131.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-09-10T00:28:46.512 INFO:teuthology.orchestra.run.smithi044.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 649 kB/s | 81 kB 00:00 2023-09-10T00:28:46.520 INFO:teuthology.orchestra.run.smithi131.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 303 kB/s | 23 kB 00:00 2023-09-10T00:28:46.538 INFO:teuthology.orchestra.run.smithi044.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2023-09-10T00:28:46.545 INFO:teuthology.orchestra.run.smithi131.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-09-10T00:28:46.628 INFO:teuthology.orchestra.run.smithi131.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 434 kB/s | 47 kB 00:00 2023-09-10T00:28:46.638 INFO:teuthology.orchestra.run.smithi044.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 140 kB/s | 18 kB 00:00 2023-09-10T00:28:46.663 INFO:teuthology.orchestra.run.smithi131.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.8 MB/s | 200 kB 00:00 2023-09-10T00:28:46.705 INFO:teuthology.orchestra.run.smithi131.stdout:(73/131): thrift-0.13.0-2.el8.x86_64.rpm 7.4 MB/s | 1.7 MB 00:00 2023-09-10T00:28:46.730 INFO:teuthology.orchestra.run.smithi131.stdout:(74/131): python3-jwt-1.6.1-2.el8.noarch.rpm 420 kB/s | 43 kB 00:00 2023-09-10T00:28:46.763 INFO:teuthology.orchestra.run.smithi131.stdout:(75/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.1 MB/s | 220 kB 00:00 2023-09-10T00:28:46.780 INFO:teuthology.orchestra.run.smithi044.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 344 kB/s | 49 kB 00:00 2023-09-10T00:28:46.789 INFO:teuthology.orchestra.run.smithi131.stdout:(76/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 575 kB/s | 48 kB 00:00 2023-09-10T00:28:46.814 INFO:teuthology.orchestra.run.smithi131.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.1 MB/s | 87 kB 00:00 2023-09-10T00:28:46.822 INFO:teuthology.orchestra.run.smithi044.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 4.1 MB/s | 1.2 MB 00:00 2023-09-10T00:28:46.839 INFO:teuthology.orchestra.run.smithi131.stdout:(78/131): python3-requests-oauthlib-1.0.0-1.el8 564 kB/s | 43 kB 00:00 2023-09-10T00:28:46.865 INFO:teuthology.orchestra.run.smithi131.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-09-10T00:28:46.890 INFO:teuthology.orchestra.run.smithi131.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.9 MB/s | 229 kB 00:00 2023-09-10T00:28:46.949 INFO:teuthology.orchestra.run.smithi131.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 224 kB/s | 19 kB 00:00 2023-09-10T00:28:46.974 INFO:teuthology.orchestra.run.smithi131.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.8 MB/s | 155 kB 00:00 2023-09-10T00:28:46.976 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-09-10T00:28:46.980 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-09-10T00:28:47.016 INFO:teuthology.orchestra.run.smithi131.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.1 MB/s | 560 kB 00:00 2023-09-10T00:28:47.041 INFO:teuthology.orchestra.run.smithi131.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 267 kB/s | 18 kB 00:00 2023-09-10T00:28:47.075 INFO:teuthology.orchestra.run.smithi131.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.4 MB/s | 559 kB 00:00 2023-09-10T00:28:47.100 INFO:teuthology.orchestra.run.smithi131.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-09-10T00:28:47.126 INFO:teuthology.orchestra.run.smithi131.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-09-10T00:28:47.151 INFO:teuthology.orchestra.run.smithi131.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-09-10T00:28:47.176 INFO:teuthology.orchestra.run.smithi131.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-09-10T00:28:47.229 INFO:teuthology.orchestra.run.smithi131.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 774 kB/s | 60 kB 00:00 2023-09-10T00:28:47.260 INFO:teuthology.orchestra.run.smithi131.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.2 MB/s | 98 kB 00:00 2023-09-10T00:28:47.264 INFO:teuthology.orchestra.run.smithi044.stdout:(53/131): ceph-test-18.0.0-6025.g9670cca5.el8.x 7.5 MB/s | 49 MB 00:06 2023-09-10T00:28:47.277 INFO:teuthology.orchestra.run.smithi131.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.2 MB/s | 58 kB 00:00 2023-09-10T00:28:47.281 INFO:teuthology.orchestra.run.smithi044.stdout:(54/131): python3-more-itertools-7.2.0-3.el8.no 117 kB/s | 59 kB 00:00 2023-09-10T00:28:47.294 INFO:teuthology.orchestra.run.smithi131.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 1.1 MB/s | 39 kB 00:00 2023-09-10T00:28:47.298 INFO:teuthology.orchestra.run.smithi044.stdout:(55/131): python3-natsort-7.1.1-2.el8.noarch.rp 127 kB/s | 60 kB 00:00 2023-09-10T00:28:47.336 INFO:teuthology.orchestra.run.smithi131.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-09-10T00:28:47.361 INFO:teuthology.orchestra.run.smithi131.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-09-10T00:28:47.386 INFO:teuthology.orchestra.run.smithi131.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-09-10T00:28:47.390 INFO:teuthology.orchestra.run.smithi044.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2023-09-10T00:28:47.407 INFO:teuthology.orchestra.run.smithi044.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 311 kB/s | 34 kB 00:00 2023-09-10T00:28:47.445 INFO:teuthology.orchestra.run.smithi131.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.3 MB/s | 2.3 MB 00:00 2023-09-10T00:28:47.471 INFO:teuthology.orchestra.run.smithi131.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-09-10T00:28:47.524 INFO:teuthology.orchestra.run.smithi044.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 551 kB/s | 64 kB 00:00 2023-09-10T00:28:47.529 INFO:teuthology.orchestra.run.smithi131.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 649 kB/s | 54 kB 00:00 2023-09-10T00:28:47.554 INFO:teuthology.orchestra.run.smithi131.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.0 MB/s | 259 kB 00:00 2023-09-10T00:28:47.566 INFO:teuthology.orchestra.run.smithi044.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.1 MB/s | 196 kB 00:00 2023-09-10T00:28:47.588 INFO:teuthology.orchestra.run.smithi131.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-09-10T00:28:47.624 INFO:teuthology.orchestra.run.smithi044.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 186 kB/s | 19 kB 00:00 2023-09-10T00:28:47.666 INFO:teuthology.orchestra.run.smithi044.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2023-09-10T00:28:47.671 INFO:teuthology.orchestra.run.smithi131.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 332 kB/s | 39 kB 00:00 2023-09-10T00:28:47.705 INFO:teuthology.orchestra.run.smithi131.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 5.8 MB/s | 688 kB 00:00 2023-09-10T00:28:47.733 INFO:teuthology.orchestra.run.smithi044.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 269 kB/s | 29 kB 00:00 2023-09-10T00:28:47.747 INFO:teuthology.orchestra.run.smithi131.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-09-10T00:28:47.773 INFO:teuthology.orchestra.run.smithi131.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-09-10T00:28:47.783 INFO:teuthology.orchestra.run.smithi044.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 235 kB/s | 27 kB 00:00 2023-09-10T00:28:47.817 INFO:teuthology.orchestra.run.smithi044.stdout:(64/131): python3-pecan-1.3.2-9.el8.noarch.rpm 512 kB/s | 283 kB 00:00 2023-09-10T00:28:47.829 INFO:teuthology.orchestra.run.smithi131.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.3 MB/s | 187 kB 00:00 2023-09-10T00:28:47.874 INFO:teuthology.orchestra.run.smithi131.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 6.9 MB/s | 713 kB 00:00 2023-09-10T00:28:47.876 INFO:teuthology.orchestra.run.smithi044.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 1.8 MB/s | 255 kB 00:00 2023-09-10T00:28:47.907 INFO:teuthology.orchestra.run.smithi131.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 799 kB/s | 27 kB 00:00 2023-09-10T00:28:47.943 INFO:teuthology.orchestra.run.smithi044.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.5 MB/s | 251 kB 00:00 2023-09-10T00:28:47.949 INFO:teuthology.orchestra.run.smithi131.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-09-10T00:28:47.968 INFO:teuthology.orchestra.run.smithi044.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 407 kB/s | 61 kB 00:00 2023-09-10T00:28:47.983 INFO:teuthology.orchestra.run.smithi131.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-09-10T00:28:48.018 INFO:teuthology.orchestra.run.smithi044.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 600 kB/s | 85 kB 00:00 2023-09-10T00:28:48.043 INFO:teuthology.orchestra.run.smithi044.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-09-10T00:28:48.075 INFO:teuthology.orchestra.run.smithi131.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-09-10T00:28:48.085 INFO:teuthology.orchestra.run.smithi044.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 161 kB/s | 23 kB 00:00 2023-09-10T00:28:48.142 INFO:teuthology.orchestra.run.smithi131.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 6.7 MB/s | 457 kB 00:00 2023-09-10T00:28:48.144 INFO:teuthology.orchestra.run.smithi044.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.9 MB/s | 110 kB 00:00 2023-09-10T00:28:48.184 INFO:teuthology.orchestra.run.smithi131.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-09-10T00:28:48.186 INFO:teuthology.orchestra.run.smithi044.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-09-10T00:28:48.228 INFO:teuthology.orchestra.run.smithi044.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-09-10T00:28:48.242 INFO:teuthology.orchestra.run.smithi131.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-09-10T00:28:48.268 INFO:teuthology.orchestra.run.smithi131.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 767 kB/s | 64 kB 00:00 2023-09-10T00:28:48.270 INFO:teuthology.orchestra.run.smithi044.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-09-10T00:28:48.293 INFO:teuthology.orchestra.run.smithi131.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-09-10T00:28:48.312 INFO:teuthology.orchestra.run.smithi044.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 934 kB/s | 39 kB 00:00 2023-09-10T00:28:48.318 INFO:teuthology.orchestra.run.smithi131.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-09-10T00:28:48.344 INFO:teuthology.orchestra.run.smithi131.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-09-10T00:28:48.370 INFO:teuthology.orchestra.run.smithi044.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-09-10T00:28:48.377 INFO:teuthology.orchestra.run.smithi131.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-09-10T00:28:48.421 INFO:teuthology.orchestra.run.smithi044.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 11 MB/s | 538 kB 00:00 2023-09-10T00:28:48.463 INFO:teuthology.orchestra.run.smithi044.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-09-10T00:28:48.521 INFO:teuthology.orchestra.run.smithi044.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-09-10T00:28:48.536 INFO:teuthology.orchestra.run.smithi131.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 12 MB/s | 14 MB 00:01 2023-09-10T00:28:48.561 INFO:teuthology.orchestra.run.smithi131.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2023-09-10T00:28:48.603 INFO:teuthology.orchestra.run.smithi131.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-09-10T00:28:48.645 INFO:teuthology.orchestra.run.smithi131.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-09-10T00:28:48.762 INFO:teuthology.orchestra.run.smithi131.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 12 MB/s | 4.9 MB 00:00 2023-09-10T00:28:48.772 INFO:teuthology.orchestra.run.smithi044.stdout:(80/131): python3-scipy-1.0.0-21.module+el8.5.0 19 MB/s | 14 MB 00:00 2023-09-10T00:28:48.814 INFO:teuthology.orchestra.run.smithi044.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 2.4 MB/s | 713 kB 00:00 2023-09-10T00:28:48.846 INFO:teuthology.orchestra.run.smithi131.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-09-10T00:28:48.847 INFO:teuthology.orchestra.run.smithi044.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 802 kB/s | 27 kB 00:00 2023-09-10T00:28:48.888 INFO:teuthology.orchestra.run.smithi131.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.3 MB/s | 808 kB 00:00 2023-09-10T00:28:48.889 INFO:teuthology.orchestra.run.smithi044.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-09-10T00:28:48.923 INFO:teuthology.orchestra.run.smithi044.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-09-10T00:28:48.946 INFO:teuthology.orchestra.run.smithi131.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-09-10T00:28:49.015 INFO:teuthology.orchestra.run.smithi044.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-09-10T00:28:49.063 INFO:teuthology.orchestra.run.smithi131.stdout:(128/131): postgresql-server-10.21-2.module+el8 12 MB/s | 5.1 MB 00:00 2023-09-10T00:28:49.099 INFO:teuthology.orchestra.run.smithi044.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 5.4 MB/s | 457 kB 00:00 2023-09-10T00:28:49.122 INFO:teuthology.orchestra.run.smithi131.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-09-10T00:28:49.141 INFO:teuthology.orchestra.run.smithi044.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-09-10T00:28:49.199 INFO:teuthology.orchestra.run.smithi044.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2023-09-10T00:28:49.225 INFO:teuthology.orchestra.run.smithi044.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 765 kB/s | 64 kB 00:00 2023-09-10T00:28:49.250 INFO:teuthology.orchestra.run.smithi044.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-09-10T00:28:49.275 INFO:teuthology.orchestra.run.smithi044.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-09-10T00:28:49.300 INFO:teuthology.orchestra.run.smithi044.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-09-10T00:28:49.334 INFO:teuthology.orchestra.run.smithi044.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-09-10T00:28:49.376 INFO:teuthology.orchestra.run.smithi044.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-09-10T00:28:49.397 INFO:teuthology.orchestra.run.smithi131.stdout:(130/131): librbd1-18.0.0-6025.g9670cca5.el8.x8 8.2 MB/s | 3.7 MB 00:00 2023-09-10T00:28:49.410 INFO:teuthology.orchestra.run.smithi044.stdout:(95/131): thrift-0.13.0-2.el8.x86_64.rpm 1.2 MB/s | 1.7 MB 00:01 2023-09-10T00:28:49.468 INFO:teuthology.orchestra.run.smithi044.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2023-09-10T00:28:49.510 INFO:teuthology.orchestra.run.smithi044.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-09-10T00:28:49.594 INFO:teuthology.orchestra.run.smithi044.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 17 MB/s | 4.9 MB 00:00 2023-09-10T00:28:49.694 INFO:teuthology.orchestra.run.smithi044.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-09-10T00:28:49.728 INFO:teuthology.orchestra.run.smithi044.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.9 MB/s | 808 kB 00:00 2023-09-10T00:28:49.761 INFO:teuthology.orchestra.run.smithi044.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-09-10T00:28:49.803 INFO:teuthology.orchestra.run.smithi044.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 14 MB/s | 1.5 MB 00:00 2023-09-10T00:28:49.828 INFO:teuthology.orchestra.run.smithi044.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-09-10T00:28:49.854 INFO:teuthology.orchestra.run.smithi044.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 850 kB/s | 43 kB 00:00 2023-09-10T00:28:49.879 INFO:teuthology.orchestra.run.smithi044.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-09-10T00:28:49.904 INFO:teuthology.orchestra.run.smithi044.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 954 kB/s | 48 kB 00:00 2023-09-10T00:28:49.930 INFO:teuthology.orchestra.run.smithi044.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-09-10T00:28:49.955 INFO:teuthology.orchestra.run.smithi044.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 852 kB/s | 43 kB 00:00 2023-09-10T00:28:49.995 INFO:teuthology.orchestra.run.smithi044.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 1.8 MB/s | 109 kB 00:00 2023-09-10T00:28:50.014 INFO:teuthology.orchestra.run.smithi044.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-09-10T00:28:50.031 INFO:teuthology.orchestra.run.smithi131.stdout:(131/131): librados2-18.0.0-6025.g9670cca5.el8. 3.3 MB/s | 3.7 MB 00:01 2023-09-10T00:28:50.033 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2023-09-10T00:28:50.034 INFO:teuthology.orchestra.run.smithi131.stdout:Total 11 MB/s | 208 MB 00:18 2023-09-10T00:28:50.048 INFO:teuthology.orchestra.run.smithi044.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-09-10T00:28:50.073 INFO:teuthology.orchestra.run.smithi044.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 319 kB/s | 19 kB 00:00 2023-09-10T00:28:50.098 INFO:teuthology.orchestra.run.smithi044.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-09-10T00:28:50.165 INFO:teuthology.orchestra.run.smithi044.stdout:(114/131): postgresql-server-10.21-2.module+el8 7.7 MB/s | 5.1 MB 00:00 2023-09-10T00:28:50.191 INFO:teuthology.orchestra.run.smithi044.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 194 kB/s | 18 kB 00:00 2023-09-10T00:28:50.241 INFO:teuthology.orchestra.run.smithi044.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 3.3 MB/s | 559 kB 00:00 2023-09-10T00:28:50.266 INFO:teuthology.orchestra.run.smithi044.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.9 MB/s | 605 kB 00:00 2023-09-10T00:28:50.291 INFO:teuthology.orchestra.run.smithi044.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.3 MB/s | 238 kB 00:00 2023-09-10T00:28:50.317 INFO:teuthology.orchestra.run.smithi044.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-09-10T00:28:50.342 INFO:teuthology.orchestra.run.smithi044.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-09-10T00:28:50.367 INFO:teuthology.orchestra.run.smithi044.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-09-10T00:28:50.409 INFO:teuthology.orchestra.run.smithi044.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.4 MB/s | 98 kB 00:00 2023-09-10T00:28:50.435 INFO:teuthology.orchestra.run.smithi044.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 864 kB/s | 58 kB 00:00 2023-09-10T00:28:50.460 INFO:teuthology.orchestra.run.smithi044.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 765 kB/s | 39 kB 00:00 2023-09-10T00:28:50.502 INFO:teuthology.orchestra.run.smithi044.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-09-10T00:28:50.527 INFO:teuthology.orchestra.run.smithi044.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-09-10T00:28:50.544 INFO:teuthology.orchestra.run.smithi044.stdout:(127/131): NetworkManager-initscripts-updown-1. 3.2 MB/s | 138 kB 00:00 2023-09-10T00:28:50.594 INFO:teuthology.orchestra.run.smithi044.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.6 MB/s | 2.3 MB 00:00 2023-09-10T00:28:50.645 INFO:teuthology.orchestra.run.smithi044.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-09-10T00:28:50.854 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2023-09-10T00:28:50.920 INFO:teuthology.orchestra.run.smithi044.stdout:(130/131): librados2-18.0.0-6025.g9670cca5.el8. 9.5 MB/s | 3.7 MB 00:00 2023-09-10T00:28:50.974 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2023-09-10T00:28:50.975 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2023-09-10T00:28:51.245 INFO:teuthology.orchestra.run.smithi044.stdout:(131/131): librbd1-18.0.0-6025.g9670cca5.el8.x8 5.3 MB/s | 3.7 MB 00:00 2023-09-10T00:28:51.248 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2023-09-10T00:28:51.248 INFO:teuthology.orchestra.run.smithi044.stdout:Total 12 MB/s | 208 MB 00:17 2023-09-10T00:28:51.864 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-09-10T00:28:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-10T00:28:52.098 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2023-09-10T00:28:52.099 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2023-09-10T00:28:52.102 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2023-09-10T00:28:52.230 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2023-09-10T00:28:52.230 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2023-09-10T00:28:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-10T00:28:52.519 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-09-10T00:28:52.695 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-10T00:28:52.806 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-10T00:28:53.062 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-10T00:28:53.403 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2023-09-10T00:28:53.409 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2023-09-10T00:28:53.466 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-10T00:28:53.695 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-10T00:28:54.108 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-10T00:28:54.327 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-10T00:28:54.465 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-10T00:28:54.670 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 8/134 2023-09-10T00:28:54.750 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 8/134 2023-09-10T00:28:54.971 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 9/134 2023-09-10T00:28:55.214 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-10T00:28:55.448 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-10T00:28:55.861 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-10T00:28:56.111 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-10T00:28:56.278 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 14/134 2023-09-10T00:28:56.414 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 14/134 2023-09-10T00:28:56.577 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 15/134 2023-09-10T00:28:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 15/134 2023-09-10T00:28:57.086 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-10T00:28:57.220 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-10T00:28:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-10T00:28:57.463 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2023-09-10T00:28:57.662 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-10T00:28:57.904 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-10T00:28:58.024 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-10T00:28:58.081 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-10T00:28:58.218 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-10T00:28:58.236 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-10T00:28:58.672 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2023-09-10T00:28:58.781 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-10T00:28:58.836 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-10T00:28:58.843 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-10T00:28:59.033 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-10T00:28:59.061 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-09-10T00:28:59.189 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-10T00:28:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-09-10T00:28:59.447 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 5/134 2023-09-10T00:28:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-10T00:28:59.755 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-09-10T00:28:59.853 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 5/134 2023-09-10T00:28:59.858 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-10T00:29:00.055 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-10T00:29:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-09-10T00:29:00.116 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-10T00:29:00.169 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-10T00:29:00.286 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-09-10T00:29:00.380 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-10T00:29:00.462 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-10T00:29:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-09-10T00:29:00.553 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-10T00:29:00.660 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 8/134 2023-09-10T00:29:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-09-10T00:29:00.783 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-10T00:29:00.831 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-09-10T00:29:00.941 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-09-10T00:29:01.101 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-10T00:29:01.213 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-10T00:29:01.216 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-10T00:29:01.333 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-10T00:29:01.399 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-10T00:29:01.408 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:18.0.0-6025.g9670cca5.el8. 26/134 2023-09-10T00:29:01.529 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-10T00:29:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el 27/134 2023-09-10T00:29:01.757 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 8/134 2023-09-10T00:29:01.768 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-10T00:29:01.845 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 8/134 2023-09-10T00:29:02.029 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-10T00:29:02.043 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 9/134 2023-09-10T00:29:02.205 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 14/134 2023-09-10T00:29:02.297 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-10T00:29:02.313 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 14/134 2023-09-10T00:29:02.496 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 15/134 2023-09-10T00:29:02.497 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_6 28/134 2023-09-10T00:29:02.529 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-10T00:29:02.644 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 15/134 2023-09-10T00:29:02.817 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-09-10T00:29:02.909 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-09-10T00:29:02.988 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-10T00:29:03.119 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-09-10T00:29:03.125 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-09-10T00:29:03.233 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-10T00:29:03.294 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-09-10T00:29:03.371 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-09-10T00:29:03.382 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 14/134 2023-09-10T00:29:03.438 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-10T00:29:03.519 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 14/134 2023-09-10T00:29:03.563 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-10T00:29:03.618 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-10T00:29:03.674 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 15/134 2023-09-10T00:29:03.760 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 15/134 2023-09-10T00:29:03.763 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-10T00:29:03.907 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-10T00:29:04.060 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-10T00:29:04.224 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-10T00:29:04.377 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-10T00:29:04.546 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-10T00:29:04.876 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-10T00:29:05.107 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-10T00:29:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-10T00:29:05.387 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-10T00:29:05.389 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-09-10T00:29:05.742 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-10T00:29:05.743 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-09-10T00:29:05.975 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-10T00:29:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-09-10T00:29:06.067 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-09-10T00:29:06.189 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-10T00:29:06.353 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-09-10T00:29:06.390 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 36/134 2023-09-10T00:29:06.569 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-09-10T00:29:06.604 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_6 37/134 2023-09-10T00:29:06.735 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-09-10T00:29:06.766 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-09-10T00:29:06.832 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-10T00:29:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-09-10T00:29:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-09-10T00:29:07.052 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-common-2:18.0.0-6025.g9670cca5.el8. 26/134 2023-09-10T00:29:07.153 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-09-10T00:29:07.208 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el 27/134 2023-09-10T00:29:07.297 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-09-10T00:29:07.312 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-09-10T00:29:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_6 28/134 2023-09-10T00:29:07.575 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-09-10T00:29:07.578 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-09-10T00:29:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-10T00:29:07.782 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-09-10T00:29:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-09-10T00:29:08.030 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-09-10T00:29:08.175 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-09-10T00:29:08.292 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-10T00:29:08.436 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-09-10T00:29:08.472 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-common-2:18.0.0-6025.g9670cca5.el8. 26/134 2023-09-10T00:29:08.629 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el 27/134 2023-09-10T00:29:08.668 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-09-10T00:29:08.879 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-09-10T00:29:09.155 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-09-10T00:29:09.478 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-10T00:29:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-09-10T00:29:09.625 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_6 28/134 2023-09-10T00:29:09.716 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-09-10T00:29:09.909 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-09-10T00:29:09.979 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-09-10T00:29:10.146 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-09-10T00:29:10.263 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-09-10T00:29:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-09-10T00:29:10.494 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-09-10T00:29:10.536 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-09-10T00:29:10.565 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-09-10T00:29:10.765 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-10T00:29:10.843 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-09-10T00:29:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-09-10T00:29:11.155 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-09-10T00:29:11.164 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-09-10T00:29:11.396 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-09-10T00:29:11.430 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-09-10T00:29:11.648 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-09-10T00:29:11.673 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-10T00:29:11.848 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-09-10T00:29:11.919 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-10T00:29:12.009 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-09-10T00:29:12.092 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 36/134 2023-09-10T00:29:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-09-10T00:29:12.276 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_6 37/134 2023-09-10T00:29:12.280 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-09-10T00:29:12.427 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-09-10T00:29:12.526 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-09-10T00:29:12.605 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-09-10T00:29:12.634 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-10T00:29:12.828 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-09-10T00:29:12.896 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-09-10T00:29:12.994 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-09-10T00:29:13.115 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-09-10T00:29:13.120 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-09-10T00:29:13.213 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-10T00:29:13.291 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-10T00:29:13.368 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-10T00:29:13.406 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-09-10T00:29:13.436 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-10T00:29:13.545 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-10T00:29:13.599 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-09-10T00:29:13.608 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-10T00:29:13.647 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 36/134 2023-09-10T00:29:13.784 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-09-10T00:29:13.849 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-09-10T00:29:13.861 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_6 37/134 2023-09-10T00:29:13.967 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-10T00:29:14.040 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-09-10T00:29:14.041 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-09-10T00:29:14.185 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-09-10T00:29:14.213 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-10T00:29:14.226 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-09-10T00:29:14.405 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-09-10T00:29:14.467 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-09-10T00:29:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-10T00:29:14.614 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-09-10T00:29:14.636 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-09-10T00:29:14.723 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-10T00:29:14.774 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-09-10T00:29:14.862 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-10T00:29:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-09-10T00:29:15.011 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-09-10T00:29:15.019 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-10T00:29:15.100 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-09-10T00:29:15.210 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-09-10T00:29:15.276 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-10T00:29:15.479 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-10T00:29:15.577 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-09-10T00:29:15.702 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-10T00:29:15.707 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-09-10T00:29:15.893 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-10T00:29:15.896 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-09-10T00:29:16.002 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-09-10T00:29:16.055 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-10T00:29:16.139 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-09-10T00:29:16.252 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-09-10T00:29:16.344 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-10T00:29:16.367 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-09-10T00:29:16.519 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-09-10T00:29:16.577 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-10T00:29:16.624 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-09-10T00:29:16.821 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-10T00:29:16.831 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-09-10T00:29:16.969 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-09-10T00:29:17.088 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-10T00:29:17.148 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-09-10T00:29:17.227 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-09-10T00:29:17.314 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-10T00:29:17.398 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-09-10T00:29:17.445 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-09-10T00:29:17.514 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-10T00:29:17.583 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-09-10T00:29:17.708 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-09-10T00:29:17.769 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-10T00:29:17.823 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-09-10T00:29:17.891 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-09-10T00:29:18.029 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-09-10T00:29:18.100 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-09-10T00:29:18.211 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-09-10T00:29:18.424 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-09-10T00:29:18.554 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-09-10T00:29:18.717 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-09-10T00:29:18.853 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-09-10T00:29:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-10T00:29:19.007 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-09-10T00:29:19.098 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-09-10T00:29:19.152 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-10T00:29:19.271 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-09-10T00:29:19.321 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-09-10T00:29:19.435 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-10T00:29:19.447 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-10T00:29:19.513 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-09-10T00:29:19.665 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-10T00:29:19.731 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-10T00:29:19.732 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-09-10T00:29:19.930 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-09-10T00:29:19.967 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-10T00:29:20.008 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-09-10T00:29:20.013 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-10T00:29:20.193 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-10T00:29:20.240 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-10T00:29:20.274 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-09-10T00:29:20.340 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-10T00:29:20.494 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-09-10T00:29:20.531 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-10T00:29:20.672 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-10T00:29:20.729 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-10T00:29:20.738 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-10T00:29:20.957 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-10T00:29:20.973 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-09-10T00:29:21.248 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-10T00:29:21.248 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-10T00:29:21.248 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-09-10T00:29:21.272 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 92/134 2023-09-10T00:29:21.332 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-10T00:29:21.345 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 92/134 2023-09-10T00:29:21.422 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-10T00:29:21.514 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 93/134 2023-09-10T00:29:21.563 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-10T00:29:21.575 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-10T00:29:21.828 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-10T00:29:21.830 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-10T00:29:22.089 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-10T00:29:22.098 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-10T00:29:22.252 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-10T00:29:22.275 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-10T00:29:22.426 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-10T00:29:22.583 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-10T00:29:22.685 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-10T00:29:22.826 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-10T00:29:22.904 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-10T00:29:23.060 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-10T00:29:23.110 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-10T00:29:23.292 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-10T00:29:23.320 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-10T00:29:23.470 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-10T00:29:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-10T00:29:23.993 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-10T00:29:24.219 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-10T00:29:24.445 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-10T00:29:24.518 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-10T00:29:24.756 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-10T00:29:24.777 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-10T00:29:24.973 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-10T00:29:25.043 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-10T00:29:25.201 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-10T00:29:25.203 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-10T00:29:25.360 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-10T00:29:25.438 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-10T00:29:25.698 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-10T00:29:25.904 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-10T00:29:26.045 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-10T00:29:26.338 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 95/134 2023-09-10T00:29:26.387 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-10T00:29:26.405 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 95/134 2023-09-10T00:29:26.546 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 96/134 2023-09-10T00:29:26.704 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-10T00:29:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-10T00:29:26.926 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 92/134 2023-09-10T00:29:26.974 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-10T00:29:26.993 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 92/134 2023-09-10T00:29:27.194 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 93/134 2023-09-10T00:29:27.303 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-10T00:29:27.600 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-10T00:29:27.856 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-10T00:29:28.729 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-10T00:29:28.904 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 92/134 2023-09-10T00:29:28.998 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 92/134 2023-09-10T00:29:29.188 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 93/134 2023-09-10T00:29:30.978 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-10T00:29:31.223 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-10T00:29:31.813 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-10T00:29:32.135 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 95/134 2023-09-10T00:29:32.229 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 95/134 2023-09-10T00:29:32.367 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 96/134 2023-09-10T00:29:33.070 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-10T00:29:33.304 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-10T00:29:33.941 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-10T00:29:34.270 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 95/134 2023-09-10T00:29:34.358 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 95/134 2023-09-10T00:29:34.520 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 96/134 2023-09-10T00:29:45.905 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 96/134 2023-09-10T00:29:45.905 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2023-09-10T00:29:45.906 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2023-09-10T00:29:45.906 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /dev 2023-09-10T00:29:45.906 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /run 2023-09-10T00:29:45.906 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2023-09-10T00:29:45.906 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2023-09-10T00:29:45.907 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2023-09-10T00:29:45.907 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2023-09-10T00:29:45.907 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /dev 2023-09-10T00:29:45.907 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-10T00:29:46.284 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 97/134 2023-09-10T00:29:47.872 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 97/134 2023-09-10T00:29:48.025 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 98/134 2023-09-10T00:29:48.247 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 98/134 2023-09-10T00:29:48.425 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 99/134 2023-09-10T00:29:48.881 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 99/134 2023-09-10T00:29:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 100/134 2023-09-10T00:29:49.050 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 100/134 2023-09-10T00:29:50.706 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 101/134 2023-09-10T00:29:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 101/134 2023-09-10T00:29:51.030 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.e 102/134 2023-09-10T00:29:51.197 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 96/134 2023-09-10T00:29:51.198 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /sys 2023-09-10T00:29:51.198 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /proc 2023-09-10T00:29:51.198 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /dev 2023-09-10T00:29:51.199 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /run 2023-09-10T00:29:51.199 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /mnt 2023-09-10T00:29:51.199 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /var/tmp 2023-09-10T00:29:51.199 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /home 2023-09-10T00:29:51.199 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /tmp 2023-09-10T00:29:51.200 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /dev 2023-09-10T00:29:51.200 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-10T00:29:51.306 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5. 103/134 2023-09-10T00:29:51.490 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 104/134 2023-09-10T00:29:51.581 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 97/134 2023-09-10T00:29:51.710 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 104/134 2023-09-10T00:29:52.015 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 105/134 2023-09-10T00:29:52.977 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 105/134 2023-09-10T00:29:53.157 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 106/134 2023-09-10T00:29:53.162 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 97/134 2023-09-10T00:29:53.247 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 106/134 2023-09-10T00:29:53.349 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 98/134 2023-09-10T00:29:53.571 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 98/134 2023-09-10T00:29:53.589 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el 107/134 2023-09-10T00:29:53.739 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 99/134 2023-09-10T00:29:53.851 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 108/134 2023-09-10T00:29:53.886 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 108/134 2023-09-10T00:29:54.057 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 109/134 2023-09-10T00:29:54.068 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 109/134 2023-09-10T00:29:54.192 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 99/134 2023-09-10T00:29:54.347 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 100/134 2023-09-10T00:29:54.354 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 96/134 2023-09-10T00:29:54.354 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /sys 2023-09-10T00:29:54.355 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /proc 2023-09-10T00:29:54.355 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /dev 2023-09-10T00:29:54.355 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /run 2023-09-10T00:29:54.355 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /mnt 2023-09-10T00:29:54.355 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /var/tmp 2023-09-10T00:29:54.356 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /home 2023-09-10T00:29:54.356 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /tmp 2023-09-10T00:29:54.356 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /dev 2023-09-10T00:29:54.356 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:29:54.364 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 100/134 2023-09-10T00:29:54.735 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 97/134 2023-09-10T00:29:55.391 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 110/134 2023-09-10T00:29:55.592 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 111/134 2023-09-10T00:29:55.626 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 111/134 2023-09-10T00:29:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6025.g9670c 112/134 2023-09-10T00:29:55.849 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6025.g9670c 112/134 2023-09-10T00:29:56.006 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 113/134 2023-09-10T00:29:56.118 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 101/134 2023-09-10T00:29:56.343 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 101/134 2023-09-10T00:29:56.364 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 113/134 2023-09-10T00:29:56.426 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 97/134 2023-09-10T00:29:56.469 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.e 102/134 2023-09-10T00:29:56.540 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 114/134 2023-09-10T00:29:56.584 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 98/134 2023-09-10T00:29:56.720 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5. 103/134 2023-09-10T00:29:56.819 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 98/134 2023-09-10T00:29:56.938 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 104/134 2023-09-10T00:29:57.051 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 99/134 2023-09-10T00:29:57.151 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 104/134 2023-09-10T00:29:57.348 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 105/134 2023-09-10T00:29:57.535 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 99/134 2023-09-10T00:29:57.709 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 100/134 2023-09-10T00:29:57.727 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 100/134 2023-09-10T00:29:58.306 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 105/134 2023-09-10T00:29:58.529 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 106/134 2023-09-10T00:29:58.618 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 106/134 2023-09-10T00:29:58.990 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el 107/134 2023-09-10T00:29:59.186 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 108/134 2023-09-10T00:29:59.221 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 108/134 2023-09-10T00:29:59.396 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 109/134 2023-09-10T00:29:59.408 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 109/134 2023-09-10T00:29:59.531 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 101/134 2023-09-10T00:29:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 101/134 2023-09-10T00:29:59.924 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.e 102/134 2023-09-10T00:30:00.143 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5. 103/134 2023-09-10T00:30:00.325 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 104/134 2023-09-10T00:30:00.555 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 104/134 2023-09-10T00:30:00.770 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 110/134 2023-09-10T00:30:00.808 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 105/134 2023-09-10T00:30:00.956 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 111/134 2023-09-10T00:30:00.994 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 111/134 2023-09-10T00:30:01.167 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 114/134 2023-09-10T00:30:01.179 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6025.g9670c 112/134 2023-09-10T00:30:01.221 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6025.g9670c 112/134 2023-09-10T00:30:01.404 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 113/134 2023-09-10T00:30:01.483 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 115/134 2023-09-10T00:30:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-09-10T00:30:01.762 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 113/134 2023-09-10T00:30:01.819 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-09-10T00:30:01.835 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 105/134 2023-09-10T00:30:01.913 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 114/134 2023-09-10T00:30:02.024 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 106/134 2023-09-10T00:30:02.121 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 106/134 2023-09-10T00:30:02.509 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el 107/134 2023-09-10T00:30:02.676 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-09-10T00:30:02.752 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 108/134 2023-09-10T00:30:02.789 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 108/134 2023-09-10T00:30:02.958 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 109/134 2023-09-10T00:30:02.970 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 109/134 2023-09-10T00:30:03.043 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-09-10T00:30:03.258 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 119/134 2023-09-10T00:30:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-10T00:30:03.600 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_ 121/134 2023-09-10T00:30:03.764 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-10T00:30:04.380 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 110/134 2023-09-10T00:30:04.594 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 111/134 2023-09-10T00:30:04.629 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 111/134 2023-09-10T00:30:04.808 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6025.g9670c 112/134 2023-09-10T00:30:04.852 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6025.g9670c 112/134 2023-09-10T00:30:05.025 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 113/134 2023-09-10T00:30:05.406 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 113/134 2023-09-10T00:30:05.583 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 114/134 2023-09-10T00:30:05.923 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-10T00:30:06.194 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-10T00:30:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-10T00:30:06.362 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 124/134 2023-09-10T00:30:06.549 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 114/134 2023-09-10T00:30:06.713 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 125/134 2023-09-10T00:30:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 126/134 2023-09-10T00:30:07.372 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-10T00:30:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 115/134 2023-09-10T00:30:07.520 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-10T00:30:07.529 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-10T00:30:07.719 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-09-10T00:30:07.912 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-09-10T00:30:07.922 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-09-10T00:30:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-09-10T00:30:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 117/134 2023-09-10T00:30:08.195 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-09-10T00:30:08.208 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-09-10T00:30:08.208 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-10T00:30:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-10T00:30:08.398 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-10T00:30:08.399 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-09-10T00:30:08.536 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-09-10T00:30:08.654 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-10T00:30:08.727 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-09-10T00:30:08.894 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-10T00:30:08.999 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-09-10T00:30:09.165 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_ 121/134 2023-09-10T00:30:09.346 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-10T00:30:09.390 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-10T00:30:09.390 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 1/134 2023-09-10T00:30:09.390 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 2/134 2023-09-10T00:30:09.390 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 3/134 2023-09-10T00:30:09.391 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 4/134 2023-09-10T00:30:09.391 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6025.g9670c 5/134 2023-09-10T00:30:09.391 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-10T00:30:09.391 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-10T00:30:09.391 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 8/134 2023-09-10T00:30:09.392 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 9/134 2023-09-10T00:30:09.392 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 10/134 2023-09-10T00:30:09.392 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 11/134 2023-09-10T00:30:09.392 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 12/134 2023-09-10T00:30:09.392 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 13/134 2023-09-10T00:30:09.393 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_ 14/134 2023-09-10T00:30:09.393 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 15/134 2023-09-10T00:30:09.393 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 16/134 2023-09-10T00:30:09.393 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_6 17/134 2023-09-10T00:30:09.393 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 18/134 2023-09-10T00:30:09.394 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 19/134 2023-09-10T00:30:09.394 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el 20/134 2023-09-10T00:30:09.394 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:18.0.0-6025.g9670cca5.el8. 21/134 2023-09-10T00:30:09.394 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_6 22/134 2023-09-10T00:30:09.394 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 23/134 2023-09-10T00:30:09.395 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 24/134 2023-09-10T00:30:09.395 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 25/134 2023-09-10T00:30:09.395 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 26/134 2023-09-10T00:30:09.395 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 27/134 2023-09-10T00:30:09.395 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 28/134 2023-09-10T00:30:09.396 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5. 29/134 2023-09-10T00:30:09.396 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 30/134 2023-09-10T00:30:09.396 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 31/134 2023-09-10T00:30:09.396 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 32/134 2023-09-10T00:30:09.396 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el 33/134 2023-09-10T00:30:09.397 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 34/134 2023-09-10T00:30:09.397 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.e 35/134 2023-09-10T00:30:09.397 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 36/134 2023-09-10T00:30:09.397 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 37/134 2023-09-10T00:30:09.397 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 38/134 2023-09-10T00:30:09.398 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-10T00:30:09.398 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-10T00:30:09.398 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-10T00:30:09.398 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-10T00:30:09.398 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-10T00:30:09.398 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-10T00:30:09.399 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-10T00:30:09.399 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-10T00:30:09.399 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-10T00:30:09.399 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-10T00:30:09.399 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-10T00:30:09.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-10T00:30:09.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-10T00:30:09.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-10T00:30:09.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-10T00:30:09.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-10T00:30:09.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-10T00:30:09.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-10T00:30:09.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-10T00:30:09.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-10T00:30:09.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-10T00:30:09.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-10T00:30:09.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-10T00:30:09.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-10T00:30:09.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-10T00:30:09.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-10T00:30:09.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-10T00:30:09.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-10T00:30:09.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-10T00:30:09.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-10T00:30:09.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-10T00:30:09.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-10T00:30:09.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-10T00:30:09.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-09-10T00:30:09.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-09-10T00:30:09.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-09-10T00:30:09.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-09-10T00:30:09.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-09-10T00:30:09.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-09-10T00:30:09.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-09-10T00:30:09.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-09-10T00:30:09.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-09-10T00:30:09.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-09-10T00:30:09.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-09-10T00:30:09.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-09-10T00:30:09.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-09-10T00:30:09.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-09-10T00:30:09.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-09-10T00:30:09.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-09-10T00:30:09.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-09-10T00:30:09.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-09-10T00:30:09.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-09-10T00:30:09.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-09-10T00:30:09.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-09-10T00:30:09.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-09-10T00:30:09.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-09-10T00:30:09.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-09-10T00:30:09.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-09-10T00:30:09.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-09-10T00:30:09.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-09-10T00:30:09.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-09-10T00:30:09.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-09-10T00:30:09.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-09-10T00:30:09.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-09-10T00:30:09.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-09-10T00:30:09.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-09-10T00:30:09.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-09-10T00:30:09.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-09-10T00:30:09.411 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-09-10T00:30:09.411 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-09-10T00:30:09.411 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-09-10T00:30:09.411 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-09-10T00:30:09.411 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-09-10T00:30:09.412 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-09-10T00:30:09.412 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-09-10T00:30:09.412 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-09-10T00:30:09.412 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-09-10T00:30:09.412 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-09-10T00:30:09.413 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-09-10T00:30:09.413 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-09-10T00:30:09.413 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-09-10T00:30:09.413 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-10T00:30:09.413 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-09-10T00:30:09.413 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-10T00:30:09.414 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-09-10T00:30:09.414 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-09-10T00:30:09.414 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-09-10T00:30:09.414 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-09-10T00:30:09.414 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-09-10T00:30:09.415 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-10T00:30:09.415 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 129/134 2023-09-10T00:30:09.415 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-10T00:30:09.415 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 131/134 2023-09-10T00:30:09.415 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-10T00:30:09.415 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-10T00:30:10.520 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 114/134 2023-09-10T00:30:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 115/134 2023-09-10T00:30:11.040 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-09-10T00:30:11.247 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-09-10T00:30:11.522 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-10T00:30:11.803 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-10T00:30:11.820 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-10T00:30:11.953 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-10T00:30:11.953 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:53 ago on Sun 10 Sep 2023 12:28:18 AM UTC. 2023-09-10T00:30:12.095 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 124/134 2023-09-10T00:30:12.242 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-09-10T00:30:12.327 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 125/134 2023-09-10T00:30:12.608 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-09-10T00:30:12.673 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 126/134 2023-09-10T00:30:12.828 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 119/134 2023-09-10T00:30:13.037 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-10T00:30:13.061 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-10T00:30:13.252 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-09-10T00:30:13.286 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_ 121/134 2023-09-10T00:30:13.332 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-09-10T00:30:13.442 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-10T00:30:13.561 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-09-10T00:30:13.573 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-09-10T00:30:13.727 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-10T00:30:13.735 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-10T00:30:13.913 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-09-10T00:30:13.913 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-10T00:30:14.122 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-10T00:30:14.122 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-10T00:30:14.400 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-10T00:30:14.626 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-10T00:30:15.183 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-10T00:30:15.183 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 1/134 2023-09-10T00:30:15.183 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 2/134 2023-09-10T00:30:15.183 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 3/134 2023-09-10T00:30:15.184 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 4/134 2023-09-10T00:30:15.184 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6025.g9670c 5/134 2023-09-10T00:30:15.184 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-10T00:30:15.184 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-10T00:30:15.184 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 8/134 2023-09-10T00:30:15.185 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 9/134 2023-09-10T00:30:15.185 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 10/134 2023-09-10T00:30:15.185 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 11/134 2023-09-10T00:30:15.185 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 12/134 2023-09-10T00:30:15.185 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 13/134 2023-09-10T00:30:15.186 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_ 14/134 2023-09-10T00:30:15.186 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 15/134 2023-09-10T00:30:15.186 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 16/134 2023-09-10T00:30:15.186 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_6 17/134 2023-09-10T00:30:15.186 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 18/134 2023-09-10T00:30:15.187 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 19/134 2023-09-10T00:30:15.187 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el 20/134 2023-09-10T00:30:15.187 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-common-2:18.0.0-6025.g9670cca5.el8. 21/134 2023-09-10T00:30:15.187 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_6 22/134 2023-09-10T00:30:15.187 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 23/134 2023-09-10T00:30:15.187 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 24/134 2023-09-10T00:30:15.188 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 25/134 2023-09-10T00:30:15.188 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 26/134 2023-09-10T00:30:15.188 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 27/134 2023-09-10T00:30:15.188 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 28/134 2023-09-10T00:30:15.197 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5. 29/134 2023-09-10T00:30:15.197 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 30/134 2023-09-10T00:30:15.197 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 31/134 2023-09-10T00:30:15.197 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 32/134 2023-09-10T00:30:15.197 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el 33/134 2023-09-10T00:30:15.198 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 34/134 2023-09-10T00:30:15.198 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.e 35/134 2023-09-10T00:30:15.198 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 36/134 2023-09-10T00:30:15.198 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 37/134 2023-09-10T00:30:15.199 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 38/134 2023-09-10T00:30:15.199 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-10T00:30:15.199 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-10T00:30:15.199 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-10T00:30:15.199 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-10T00:30:15.199 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-10T00:30:15.200 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-10T00:30:15.200 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-10T00:30:15.200 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-10T00:30:15.200 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-10T00:30:15.200 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-10T00:30:15.200 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-10T00:30:15.201 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-10T00:30:15.201 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-10T00:30:15.201 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-10T00:30:15.201 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-10T00:30:15.201 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-10T00:30:15.201 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-10T00:30:15.202 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-10T00:30:15.202 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-10T00:30:15.202 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-10T00:30:15.202 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-10T00:30:15.202 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-10T00:30:15.203 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-10T00:30:15.203 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-10T00:30:15.203 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-10T00:30:15.203 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-10T00:30:15.203 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-10T00:30:15.203 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-10T00:30:15.204 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-10T00:30:15.204 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-10T00:30:15.204 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-10T00:30:15.204 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-10T00:30:15.204 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-10T00:30:15.204 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-09-10T00:30:15.205 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-09-10T00:30:15.205 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-09-10T00:30:15.205 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-09-10T00:30:15.205 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-09-10T00:30:15.205 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-09-10T00:30:15.205 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-09-10T00:30:15.206 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-09-10T00:30:15.206 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-09-10T00:30:15.206 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-09-10T00:30:15.206 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-09-10T00:30:15.206 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-09-10T00:30:15.207 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-09-10T00:30:15.207 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-09-10T00:30:15.207 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-09-10T00:30:15.207 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-09-10T00:30:15.207 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-09-10T00:30:15.207 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-09-10T00:30:15.208 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-09-10T00:30:15.208 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-09-10T00:30:15.208 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-09-10T00:30:15.208 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-09-10T00:30:15.208 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-09-10T00:30:15.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-09-10T00:30:15.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-09-10T00:30:15.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-09-10T00:30:15.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-09-10T00:30:15.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-09-10T00:30:15.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-09-10T00:30:15.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-09-10T00:30:15.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-09-10T00:30:15.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-09-10T00:30:15.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-09-10T00:30:15.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-09-10T00:30:15.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-09-10T00:30:15.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-09-10T00:30:15.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-09-10T00:30:15.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-09-10T00:30:15.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-09-10T00:30:15.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-09-10T00:30:15.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-09-10T00:30:15.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-09-10T00:30:15.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-09-10T00:30:15.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-09-10T00:30:15.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-09-10T00:30:15.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-09-10T00:30:15.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-09-10T00:30:15.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-09-10T00:30:15.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-09-10T00:30:15.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-09-10T00:30:15.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-09-10T00:30:15.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-09-10T00:30:15.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-09-10T00:30:15.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-09-10T00:30:15.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-09-10T00:30:15.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-09-10T00:30:15.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-09-10T00:30:15.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 129/134 2023-09-10T00:30:15.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-10T00:30:15.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 131/134 2023-09-10T00:30:15.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-10T00:30:15.216 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-10T00:30:15.349 INFO:teuthology.orchestra.run.smithi099.stdout:Installed products updated. 2023-09-10T00:30:15.563 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-10T00:30:15.563 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2023-09-10T00:30:15.564 INFO:teuthology.orchestra.run.smithi099.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-10T00:30:15.564 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.564 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.564 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-09-10T00:30:15.564 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-10T00:30:15.565 INFO:teuthology.orchestra.run.smithi099.stdout: automake-1.16.1-7.el8.noarch 2023-09-10T00:30:15.565 INFO:teuthology.orchestra.run.smithi099.stdout: bison-3.0.4-10.el8.x86_64 2023-09-10T00:30:15.565 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.565 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.565 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.566 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.566 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:15.566 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.566 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.566 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.567 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:15.567 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:15.567 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:15.567 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:15.567 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:15.567 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.568 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.568 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:15.568 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.568 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.568 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.569 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:15.569 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:15.569 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.569 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:15.569 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-10T00:30:15.569 INFO:teuthology.orchestra.run.smithi099.stdout: flex-2.6.1-9.el8.x86_64 2023-09-10T00:30:15.570 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-10T00:30:15.570 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-10T00:30:15.570 INFO:teuthology.orchestra.run.smithi099.stdout: jq-1.6-3.el8.x86_64 2023-09-10T00:30:15.570 INFO:teuthology.orchestra.run.smithi099.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-10T00:30:15.571 INFO:teuthology.orchestra.run.smithi099.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-10T00:30:15.571 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-10T00:30:15.571 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.571 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.571 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.571 INFO:teuthology.orchestra.run.smithi099.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-10T00:30:15.572 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-10T00:30:15.572 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-10T00:30:15.572 INFO:teuthology.orchestra.run.smithi099.stdout: libpq-13.5-1.el8.x86_64 2023-09-10T00:30:15.572 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-10T00:30:15.572 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-10T00:30:15.573 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.573 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.573 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-10T00:30:15.573 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.573 INFO:teuthology.orchestra.run.smithi099.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-10T00:30:15.574 INFO:teuthology.orchestra.run.smithi099.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-10T00:30:15.574 INFO:teuthology.orchestra.run.smithi099.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-10T00:30:15.574 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-10T00:30:15.574 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-10T00:30:15.574 INFO:teuthology.orchestra.run.smithi099.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-10T00:30:15.574 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-10T00:30:15.575 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-10T00:30:15.575 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-10T00:30:15.575 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-10T00:30:15.575 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-10T00:30:15.575 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-10T00:30:15.576 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-10T00:30:15.576 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-10T00:30:15.576 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-10T00:30:15.576 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-10T00:30:15.576 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-10T00:30:15.576 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-10T00:30:15.577 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-10T00:30:15.577 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-10T00:30:15.577 INFO:teuthology.orchestra.run.smithi099.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-10T00:30:15.577 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-10T00:30:15.577 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.578 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.578 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.578 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-10T00:30:15.578 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-10T00:30:15.578 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-10T00:30:15.579 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-10T00:30:15.579 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-10T00:30:15.579 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-10T00:30:15.579 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-10T00:30:15.579 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-10T00:30:15.579 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-10T00:30:15.580 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-10T00:30:15.580 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-10T00:30:15.580 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-10T00:30:15.580 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-10T00:30:15.580 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-10T00:30:15.581 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-10T00:30:15.581 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-10T00:30:15.581 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-10T00:30:15.581 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-10T00:30:15.581 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-10T00:30:15.581 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-10T00:30:15.582 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-10T00:30:15.582 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-10T00:30:15.582 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-2.6-1.el8.noarch 2023-09-10T00:30:15.582 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-10T00:30:15.582 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-10T00:30:15.583 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-10T00:30:15.583 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-10T00:30:15.583 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-10T00:30:15.583 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-10T00:30:15.583 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.583 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.584 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-10T00:30:15.584 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-10T00:30:15.584 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.584 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-10T00:30:15.584 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-10T00:30:15.585 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-10T00:30:15.585 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-10T00:30:15.585 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-10T00:30:15.585 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-10T00:30:15.585 INFO:teuthology.orchestra.run.smithi099.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-10T00:30:15.585 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-10T00:30:15.586 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-10T00:30:15.586 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-10T00:30:15.586 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-10T00:30:15.586 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-10T00:30:15.586 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-10T00:30:15.587 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.587 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.587 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:15.587 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-10T00:30:15.587 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-10T00:30:15.587 INFO:teuthology.orchestra.run.smithi099.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-10T00:30:15.588 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-10T00:30:15.588 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-10T00:30:15.588 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-10T00:30:15.588 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-09-10T00:30:15.589 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-09-10T00:30:15.815 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-10T00:30:16.106 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-10T00:30:16.125 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-10T00:30:16.386 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 124/134 2023-09-10T00:30:16.607 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 125/134 2023-09-10T00:30:16.637 DEBUG:teuthology.parallel:result is None 2023-09-10T00:30:16.930 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 126/134 2023-09-10T00:30:17.258 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-10T00:30:17.423 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-10T00:30:17.432 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-10T00:30:17.622 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-09-10T00:30:17.778 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-10T00:30:17.779 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:58 ago on Sun 10 Sep 2023 12:28:19 AM UTC. 2023-09-10T00:30:17.839 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-09-10T00:30:17.928 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-09-10T00:30:18.156 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-09-10T00:30:18.170 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-09-10T00:30:18.170 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-10T00:30:18.526 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-10T00:30:18.527 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-10T00:30:18.787 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-10T00:30:19.006 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-10T00:30:19.566 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-10T00:30:19.567 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 1/134 2023-09-10T00:30:19.567 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 2/134 2023-09-10T00:30:19.567 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 3/134 2023-09-10T00:30:19.567 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 4/134 2023-09-10T00:30:19.568 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6025.g9670c 5/134 2023-09-10T00:30:19.568 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-10T00:30:19.568 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-10T00:30:19.568 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 8/134 2023-09-10T00:30:19.568 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 9/134 2023-09-10T00:30:19.569 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 10/134 2023-09-10T00:30:19.569 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 11/134 2023-09-10T00:30:19.569 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 12/134 2023-09-10T00:30:19.569 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 13/134 2023-09-10T00:30:19.569 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_ 14/134 2023-09-10T00:30:19.570 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 15/134 2023-09-10T00:30:19.570 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 16/134 2023-09-10T00:30:19.570 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_6 17/134 2023-09-10T00:30:19.570 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 18/134 2023-09-10T00:30:19.570 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 19/134 2023-09-10T00:30:19.571 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el 20/134 2023-09-10T00:30:19.571 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-common-2:18.0.0-6025.g9670cca5.el8. 21/134 2023-09-10T00:30:19.571 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_6 22/134 2023-09-10T00:30:19.571 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 23/134 2023-09-10T00:30:19.571 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 24/134 2023-09-10T00:30:19.572 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 25/134 2023-09-10T00:30:19.572 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 26/134 2023-09-10T00:30:19.572 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 27/134 2023-09-10T00:30:19.572 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 28/134 2023-09-10T00:30:19.573 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5. 29/134 2023-09-10T00:30:19.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 30/134 2023-09-10T00:30:19.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 31/134 2023-09-10T00:30:19.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 32/134 2023-09-10T00:30:19.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el 33/134 2023-09-10T00:30:19.583 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 34/134 2023-09-10T00:30:19.583 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.e 35/134 2023-09-10T00:30:19.583 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 36/134 2023-09-10T00:30:19.583 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 37/134 2023-09-10T00:30:19.583 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 38/134 2023-09-10T00:30:19.584 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-10T00:30:19.584 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-10T00:30:19.584 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-10T00:30:19.584 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-10T00:30:19.584 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-10T00:30:19.584 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-10T00:30:19.585 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-10T00:30:19.585 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-10T00:30:19.585 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-10T00:30:19.585 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-10T00:30:19.586 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-10T00:30:19.586 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-10T00:30:19.586 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-10T00:30:19.586 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-10T00:30:19.586 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-10T00:30:19.587 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-10T00:30:19.587 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-10T00:30:19.587 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-10T00:30:19.587 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-10T00:30:19.587 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-10T00:30:19.588 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-10T00:30:19.588 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-10T00:30:19.588 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-10T00:30:19.588 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-10T00:30:19.588 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-10T00:30:19.588 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-10T00:30:19.589 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-10T00:30:19.589 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-10T00:30:19.589 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-10T00:30:19.589 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-10T00:30:19.589 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-10T00:30:19.590 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-10T00:30:19.590 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-10T00:30:19.590 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-09-10T00:30:19.590 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-09-10T00:30:19.590 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-09-10T00:30:19.591 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-09-10T00:30:19.591 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-09-10T00:30:19.591 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-09-10T00:30:19.591 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-09-10T00:30:19.592 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-09-10T00:30:19.592 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-09-10T00:30:19.592 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-09-10T00:30:19.592 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-09-10T00:30:19.593 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-09-10T00:30:19.593 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-09-10T00:30:19.593 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-09-10T00:30:19.593 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-09-10T00:30:19.593 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-09-10T00:30:19.594 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-09-10T00:30:19.594 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-09-10T00:30:19.594 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-09-10T00:30:19.594 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-09-10T00:30:19.594 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-09-10T00:30:19.594 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-09-10T00:30:19.595 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-09-10T00:30:19.595 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-09-10T00:30:19.595 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-09-10T00:30:19.595 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-09-10T00:30:19.595 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-09-10T00:30:19.595 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-09-10T00:30:19.596 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-09-10T00:30:19.596 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-09-10T00:30:19.596 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-09-10T00:30:19.596 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-09-10T00:30:19.596 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-09-10T00:30:19.597 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-09-10T00:30:19.597 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-09-10T00:30:19.597 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-09-10T00:30:19.597 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-09-10T00:30:19.597 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-09-10T00:30:19.598 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-09-10T00:30:19.598 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-09-10T00:30:19.598 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-09-10T00:30:19.598 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-09-10T00:30:19.598 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-09-10T00:30:19.599 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-09-10T00:30:19.599 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-09-10T00:30:19.599 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-09-10T00:30:19.599 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-09-10T00:30:19.599 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-09-10T00:30:19.599 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-10T00:30:19.600 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-09-10T00:30:19.600 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-10T00:30:19.600 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-09-10T00:30:19.600 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-09-10T00:30:19.600 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-09-10T00:30:19.601 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-09-10T00:30:19.601 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-09-10T00:30:19.601 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-10T00:30:19.601 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 129/134 2023-09-10T00:30:19.601 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-10T00:30:19.602 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 131/134 2023-09-10T00:30:19.602 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-10T00:30:19.602 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-10T00:30:21.129 INFO:teuthology.orchestra.run.smithi131.stdout:Installed products updated. 2023-09-10T00:30:21.342 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-10T00:30:21.343 INFO:teuthology.orchestra.run.smithi131.stdout:Upgraded: 2023-09-10T00:30:21.343 INFO:teuthology.orchestra.run.smithi131.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-10T00:30:21.343 INFO:teuthology.orchestra.run.smithi131.stdout: librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.343 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.343 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2023-09-10T00:30:21.344 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-10T00:30:21.344 INFO:teuthology.orchestra.run.smithi131.stdout: automake-1.16.1-7.el8.noarch 2023-09-10T00:30:21.344 INFO:teuthology.orchestra.run.smithi131.stdout: bison-3.0.4-10.el8.x86_64 2023-09-10T00:30:21.344 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.345 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.345 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.345 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.345 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:21.345 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.346 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.346 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.346 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:21.346 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:21.346 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:21.347 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:21.347 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:21.347 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.347 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.347 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:21.347 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.348 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.348 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.348 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:21.348 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:21.348 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.349 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:21.349 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-10T00:30:21.349 INFO:teuthology.orchestra.run.smithi131.stdout: flex-2.6.1-9.el8.x86_64 2023-09-10T00:30:21.349 INFO:teuthology.orchestra.run.smithi131.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-10T00:30:21.349 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-10T00:30:21.350 INFO:teuthology.orchestra.run.smithi131.stdout: jq-1.6-3.el8.x86_64 2023-09-10T00:30:21.350 INFO:teuthology.orchestra.run.smithi131.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-10T00:30:21.350 INFO:teuthology.orchestra.run.smithi131.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-10T00:30:21.350 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-10T00:30:21.350 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.351 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.351 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.351 INFO:teuthology.orchestra.run.smithi131.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-10T00:30:21.351 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-10T00:30:21.351 INFO:teuthology.orchestra.run.smithi131.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-10T00:30:21.352 INFO:teuthology.orchestra.run.smithi131.stdout: libpq-13.5-1.el8.x86_64 2023-09-10T00:30:21.352 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-10T00:30:21.352 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-10T00:30:21.352 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.352 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.352 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-10T00:30:21.353 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.353 INFO:teuthology.orchestra.run.smithi131.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-10T00:30:21.353 INFO:teuthology.orchestra.run.smithi131.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-10T00:30:21.353 INFO:teuthology.orchestra.run.smithi131.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-10T00:30:21.353 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-10T00:30:21.354 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-10T00:30:21.354 INFO:teuthology.orchestra.run.smithi131.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-10T00:30:21.354 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-10T00:30:21.354 INFO:teuthology.orchestra.run.smithi131.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-10T00:30:21.355 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-10T00:30:21.355 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-10T00:30:21.355 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-10T00:30:21.355 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-10T00:30:21.355 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-10T00:30:21.356 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-10T00:30:21.356 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-10T00:30:21.356 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-10T00:30:21.356 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-10T00:30:21.356 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-10T00:30:21.357 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-10T00:30:21.357 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-10T00:30:21.357 INFO:teuthology.orchestra.run.smithi131.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-10T00:30:21.357 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-10T00:30:21.357 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.358 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.358 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.358 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-10T00:30:21.358 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-10T00:30:21.358 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-10T00:30:21.358 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-10T00:30:21.359 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-10T00:30:21.359 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-10T00:30:21.359 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-10T00:30:21.359 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-10T00:30:21.359 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-10T00:30:21.360 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-10T00:30:21.360 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-10T00:30:21.360 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-10T00:30:21.360 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-10T00:30:21.360 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-10T00:30:21.360 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-10T00:30:21.361 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-10T00:30:21.361 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-10T00:30:21.361 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-10T00:30:21.361 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-10T00:30:21.361 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-10T00:30:21.362 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-10T00:30:21.362 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-10T00:30:21.362 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend-2.6-1.el8.noarch 2023-09-10T00:30:21.362 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-10T00:30:21.362 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-10T00:30:21.363 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-10T00:30:21.363 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-10T00:30:21.363 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-10T00:30:21.363 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-10T00:30:21.363 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.364 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.364 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-10T00:30:21.364 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-10T00:30:21.364 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.364 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-10T00:30:21.365 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-10T00:30:21.365 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-10T00:30:21.365 INFO:teuthology.orchestra.run.smithi131.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-10T00:30:21.365 INFO:teuthology.orchestra.run.smithi131.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-10T00:30:21.365 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-10T00:30:21.366 INFO:teuthology.orchestra.run.smithi131.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-10T00:30:21.366 INFO:teuthology.orchestra.run.smithi131.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-10T00:30:21.366 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-10T00:30:21.366 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-10T00:30:21.366 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-10T00:30:21.367 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-10T00:30:21.367 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-10T00:30:21.367 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.367 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.367 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:21.367 INFO:teuthology.orchestra.run.smithi131.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-10T00:30:21.368 INFO:teuthology.orchestra.run.smithi131.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-10T00:30:21.368 INFO:teuthology.orchestra.run.smithi131.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-10T00:30:21.368 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-10T00:30:21.368 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-10T00:30:21.369 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-10T00:30:21.369 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-10T00:30:21.369 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2023-09-10T00:30:21.545 DEBUG:teuthology.parallel:result is None 2023-09-10T00:30:22.135 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-10T00:30:22.136 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:02:01 ago on Sun 10 Sep 2023 12:28:21 AM UTC. 2023-09-10T00:30:25.674 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2023-09-10T00:30:25.913 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:30:25.914 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2023-09-10T00:30:25.914 INFO:teuthology.orchestra.run.smithi044.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-10T00:30:25.914 INFO:teuthology.orchestra.run.smithi044.stdout: librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.914 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.914 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2023-09-10T00:30:25.915 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-10T00:30:25.915 INFO:teuthology.orchestra.run.smithi044.stdout: automake-1.16.1-7.el8.noarch 2023-09-10T00:30:25.915 INFO:teuthology.orchestra.run.smithi044.stdout: bison-3.0.4-10.el8.x86_64 2023-09-10T00:30:25.915 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.916 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.916 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.916 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.916 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:25.916 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.917 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.917 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.917 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:25.917 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:25.917 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:25.918 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:25.918 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:25.918 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.918 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.918 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:25.919 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.919 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.919 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.919 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:25.919 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:25.920 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.920 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-10T00:30:25.920 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-10T00:30:25.920 INFO:teuthology.orchestra.run.smithi044.stdout: flex-2.6.1-9.el8.x86_64 2023-09-10T00:30:25.921 INFO:teuthology.orchestra.run.smithi044.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-10T00:30:25.921 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-10T00:30:25.921 INFO:teuthology.orchestra.run.smithi044.stdout: jq-1.6-3.el8.x86_64 2023-09-10T00:30:25.921 INFO:teuthology.orchestra.run.smithi044.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-10T00:30:25.921 INFO:teuthology.orchestra.run.smithi044.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-10T00:30:25.922 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-10T00:30:25.922 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.922 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.922 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.923 INFO:teuthology.orchestra.run.smithi044.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-10T00:30:25.923 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-10T00:30:25.923 INFO:teuthology.orchestra.run.smithi044.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-10T00:30:25.923 INFO:teuthology.orchestra.run.smithi044.stdout: libpq-13.5-1.el8.x86_64 2023-09-10T00:30:25.923 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-10T00:30:25.924 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-10T00:30:25.924 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.924 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.924 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-10T00:30:25.924 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.925 INFO:teuthology.orchestra.run.smithi044.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-10T00:30:25.925 INFO:teuthology.orchestra.run.smithi044.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-10T00:30:25.925 INFO:teuthology.orchestra.run.smithi044.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-10T00:30:25.925 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-10T00:30:25.926 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-10T00:30:25.926 INFO:teuthology.orchestra.run.smithi044.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-10T00:30:25.926 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-10T00:30:25.926 INFO:teuthology.orchestra.run.smithi044.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-10T00:30:25.926 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-10T00:30:25.927 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-10T00:30:25.927 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-10T00:30:25.927 INFO:teuthology.orchestra.run.smithi044.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-10T00:30:25.927 INFO:teuthology.orchestra.run.smithi044.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-10T00:30:25.927 INFO:teuthology.orchestra.run.smithi044.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-10T00:30:25.928 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-10T00:30:25.928 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-10T00:30:25.928 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-10T00:30:25.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-10T00:30:25.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-10T00:30:25.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-10T00:30:25.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-10T00:30:25.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-10T00:30:25.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-10T00:30:25.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-10T00:30:25.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-10T00:30:25.931 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-10T00:30:25.931 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-10T00:30:25.931 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-10T00:30:25.931 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-10T00:30:25.931 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-10T00:30:25.932 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-10T00:30:25.932 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-10T00:30:25.932 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-10T00:30:25.932 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-10T00:30:25.933 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-10T00:30:25.933 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-10T00:30:25.933 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-10T00:30:25.933 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-10T00:30:25.934 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-10T00:30:25.934 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-10T00:30:25.934 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-10T00:30:25.934 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-10T00:30:25.935 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-10T00:30:25.935 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-10T00:30:25.935 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend-2.6-1.el8.noarch 2023-09-10T00:30:25.935 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-10T00:30:25.935 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-10T00:30:25.936 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-10T00:30:25.936 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-10T00:30:25.936 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-10T00:30:25.936 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-10T00:30:25.937 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.937 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.937 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-10T00:30:25.937 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-10T00:30:25.937 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.938 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-10T00:30:25.938 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-10T00:30:25.938 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-10T00:30:25.938 INFO:teuthology.orchestra.run.smithi044.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-10T00:30:25.938 INFO:teuthology.orchestra.run.smithi044.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-10T00:30:25.939 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-10T00:30:25.939 INFO:teuthology.orchestra.run.smithi044.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-10T00:30:25.939 INFO:teuthology.orchestra.run.smithi044.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-10T00:30:25.939 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-10T00:30:25.939 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-10T00:30:25.940 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-10T00:30:25.940 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-10T00:30:25.940 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-10T00:30:25.940 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.940 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.941 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-10T00:30:25.941 INFO:teuthology.orchestra.run.smithi044.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-10T00:30:25.941 INFO:teuthology.orchestra.run.smithi044.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-10T00:30:25.941 INFO:teuthology.orchestra.run.smithi044.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-10T00:30:25.942 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-10T00:30:25.942 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-10T00:30:25.942 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-10T00:30:25.943 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:30:25.943 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2023-09-10T00:30:26.147 DEBUG:teuthology.parallel:result is None 2023-09-10T00:30:26.148 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-10T00:30:26.285 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-10T00:30:26.309 INFO:teuthology.orchestra.run.smithi044.stdout:18.0.0-6025.g9670cca5.el8 2023-09-10T00:30:26.310 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6025.g9670cca5.el8 2023-09-10T00:30:26.310 INFO:teuthology.task.install:The correct ceph version 18.0.0-6025.g9670cca5 is installed. 2023-09-10T00:30:26.311 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-10T00:30:26.444 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-10T00:30:26.472 INFO:teuthology.orchestra.run.smithi099.stdout:18.0.0-6025.g9670cca5.el8 2023-09-10T00:30:26.472 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6025.g9670cca5.el8 2023-09-10T00:30:26.473 INFO:teuthology.task.install:The correct ceph version 18.0.0-6025.g9670cca5 is installed. 2023-09-10T00:30:26.473 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-10T00:30:26.606 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-10T00:30:26.634 INFO:teuthology.orchestra.run.smithi131.stdout:18.0.0-6025.g9670cca5.el8 2023-09-10T00:30:26.634 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6025.g9670cca5.el8 2023-09-10T00:30:26.634 INFO:teuthology.task.install:The correct ceph version 18.0.0-6025.g9670cca5 is installed. 2023-09-10T00:30:26.635 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-10T00:30:26.635 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:30:26.635 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-10T00:30:26.681 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-10T00:30:26.682 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-10T00:30:26.728 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-10T00:30:26.729 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-10T00:30:26.775 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-10T00:30:26.775 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:30:26.775 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2023-09-10T00:30:26.805 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-10T00:30:26.873 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-10T00:30:26.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2023-09-10T00:30:26.903 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-10T00:30:26.972 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-10T00:30:26.972 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2023-09-10T00:30:27.005 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-10T00:30:27.074 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-10T00:30:27.074 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:30:27.075 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-10T00:30:27.103 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-10T00:30:27.172 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-10T00:30:27.173 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-10T00:30:27.204 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-10T00:30:27.273 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-10T00:30:27.274 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-10T00:30:27.303 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-10T00:30:27.372 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-10T00:30:27.372 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:30:27.372 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/stdin-killer 2023-09-10T00:30:27.401 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-10T00:30:27.470 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-10T00:30:27.471 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2023-09-10T00:30:27.501 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-10T00:30:27.570 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-10T00:30:27.570 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2023-09-10T00:30:27.601 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-10T00:30:27.671 INFO:teuthology.run_tasks:Running task cephadm... 2023-09-10T00:30:27.746 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '9670cca5bc8e17916865dc84939edfc10b1f2c00', 'subvols': {'create': 2}} 2023-09-10T00:30:27.747 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-10T00:30:27.748 INFO:tasks.cephadm:Cluster fsid is 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:30:27.748 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-09-10T00:30:27.748 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.44', 'mon.b': '172.21.15.99', 'mon.c': '172.21.15.131'} 2023-09-10T00:30:27.749 INFO:tasks.cephadm:First mon is mon.a on smithi044 2023-09-10T00:30:27.749 INFO:tasks.cephadm:First mgr is x 2023-09-10T00:30:27.750 INFO:tasks.cephadm:Normalizing hostnames... 2023-09-10T00:30:27.750 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname $(hostname -s) 2023-09-10T00:30:27.781 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname $(hostname -s) 2023-09-10T00:30:27.811 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname $(hostname -s) 2023-09-10T00:30:27.842 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-09-10T00:30:27.843 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-10T00:30:27.982 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '9670cca5bc8e17916865dc84939edfc10b1f2c00', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/72502/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6025-g9670cca5', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6025.g9670cca5'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-09-09 16:32:07.237982', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20230909.145638', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2023-09-10T00:30:28.114 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-batrick-testing-20230909.145638, sha1 9670cca5bc8e17916865dc84939edfc10b1f2c00 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-10T00:30:28.114 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/x86_64/flavors/default/cephadm 2023-09-10T00:30:28.115 DEBUG:teuthology.orchestra.run.smithi044:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-10T00:30:28.659 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-rw-r--. 1 ubuntu ubuntu 724179 Sep 10 00:30 /home/ubuntu/cephtest/cephadm 2023-09-10T00:30:28.660 DEBUG:teuthology.orchestra.run.smithi099:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-10T00:30:29.293 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-rw-r--. 1 ubuntu ubuntu 724179 Sep 10 00:30 /home/ubuntu/cephtest/cephadm 2023-09-10T00:30:29.294 DEBUG:teuthology.orchestra.run.smithi131:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-10T00:30:29.836 INFO:teuthology.orchestra.run.smithi131.stdout:-rw-rw-r--. 1 ubuntu ubuntu 724179 Sep 10 00:30 /home/ubuntu/cephtest/cephadm 2023-09-10T00:30:29.837 DEBUG:teuthology.orchestra.run.smithi044:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-10T00:30:29.859 DEBUG:teuthology.orchestra.run.smithi099:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-10T00:30:29.880 DEBUG:teuthology.orchestra.run.smithi131:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-10T00:30:29.914 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 on all hosts... 2023-09-10T00:30:29.915 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 pull 2023-09-10T00:30:29.917 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 pull 2023-09-10T00:30:29.923 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 pull 2023-09-10T00:30:32.723 INFO:teuthology.orchestra.run.smithi099.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00... 2023-09-10T00:30:35.176 INFO:teuthology.orchestra.run.smithi131.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00... 2023-09-10T00:30:35.256 INFO:teuthology.orchestra.run.smithi044.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00... 2023-09-10T00:31:03.512 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-09-10T00:31:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev)", 2023-09-10T00:31:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "image_id": "20c3861ef6c981f72050dd1dd3edbdc62747039bda543319e240a6e02f850eea", 2023-09-10T00:31:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "repo_digests": [ 2023-09-10T00:31:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8bb87e43f4b5fd23ed47b24bb48a71d3a37a4d237bf8cf5ca0615963ecee90da" 2023-09-10T00:31:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-09-10T00:31:03.514 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-09-10T00:31:05.488 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-09-10T00:31:05.489 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev)", 2023-09-10T00:31:05.489 INFO:teuthology.orchestra.run.smithi131.stdout: "image_id": "20c3861ef6c981f72050dd1dd3edbdc62747039bda543319e240a6e02f850eea", 2023-09-10T00:31:05.489 INFO:teuthology.orchestra.run.smithi131.stdout: "repo_digests": [ 2023-09-10T00:31:05.489 INFO:teuthology.orchestra.run.smithi131.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8bb87e43f4b5fd23ed47b24bb48a71d3a37a4d237bf8cf5ca0615963ecee90da" 2023-09-10T00:31:05.489 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2023-09-10T00:31:05.489 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-09-10T00:31:06.265 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:31:06.266 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev)", 2023-09-10T00:31:06.266 INFO:teuthology.orchestra.run.smithi044.stdout: "image_id": "20c3861ef6c981f72050dd1dd3edbdc62747039bda543319e240a6e02f850eea", 2023-09-10T00:31:06.266 INFO:teuthology.orchestra.run.smithi044.stdout: "repo_digests": [ 2023-09-10T00:31:06.266 INFO:teuthology.orchestra.run.smithi044.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8bb87e43f4b5fd23ed47b24bb48a71d3a37a4d237bf8cf5ca0615963ecee90da" 2023-09-10T00:31:06.267 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2023-09-10T00:31:06.267 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:31:06.287 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph 2023-09-10T00:31:06.318 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph 2023-09-10T00:31:06.353 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph 2023-09-10T00:31:06.387 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 777 /etc/ceph 2023-09-10T00:31:06.414 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /etc/ceph 2023-09-10T00:31:06.445 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /etc/ceph 2023-09-10T00:31:06.474 INFO:tasks.cephadm:Writing seed config... 2023-09-10T00:31:06.475 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-09-10T00:31:06.475 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-09-10T00:31:06.476 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-09-10T00:31:06.476 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-09-10T00:31:06.476 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-09-10T00:31:06.477 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-09-10T00:31:06.477 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-09-10T00:31:06.477 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-09-10T00:31:06.477 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-09-10T00:31:06.478 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-09-10T00:31:06.478 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-09-10T00:31:06.478 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-09-10T00:31:06.478 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-09-10T00:31:06.479 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-09-10T00:31:06.479 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-09-10T00:31:06.479 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-09-10T00:31:06.479 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-09-10T00:31:06.480 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-09-10T00:31:06.480 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-09-10T00:31:06.480 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-09-10T00:31:06.480 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-09-10T00:31:06.481 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:31:06.481 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-09-10T00:31:06.499 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2023-09-10T00:31:06.499 DEBUG:teuthology.orchestra.run.smithi044:mon.a> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mon.a.service 2023-09-10T00:31:06.541 DEBUG:teuthology.orchestra.run.smithi044:mgr.x> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mgr.x.service 2023-09-10T00:31:06.569 INFO:journalctl@ceph.mon.a.smithi044.stdout:-- Logs begin at Sun 2023-09-10 00:17:15 UTC. -- 2023-09-10T00:31:06.570 INFO:tasks.cephadm:Bootstrapping... 2023-09-10T00:31:06.570 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 -v bootstrap --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.44 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-09-10T00:31:06.596 INFO:journalctl@ceph.mgr.x.smithi044.stdout:-- Logs begin at Sun 2023-09-10 00:17:15 UTC. -- 2023-09-10T00:31:06.738 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2023-09-10T00:31:06.738 INFO:teuthology.orchestra.run.smithi044.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00', '-v', 'bootstrap', '--fsid', '3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.44', '--skip-admin-label'] 2023-09-10T00:31:06.886 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stdout 4.1.1 2023-09-10T00:31:06.887 INFO:teuthology.orchestra.run.smithi044.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-09-10T00:31:06.887 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying podman|docker is present... 2023-09-10T00:31:07.032 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stdout 4.1.1 2023-09-10T00:31:07.032 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying lvm2 is present... 2023-09-10T00:31:07.033 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying time synchronization is in place... 2023-09-10T00:31:07.040 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-10T00:31:07.040 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-10T00:31:07.046 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-10T00:31:07.046 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2023-09-10T00:31:07.052 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout enabled 2023-09-10T00:31:07.058 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout active 2023-09-10T00:31:07.058 INFO:teuthology.orchestra.run.smithi044.stdout:Unit chronyd.service is enabled and running 2023-09-10T00:31:07.058 INFO:teuthology.orchestra.run.smithi044.stdout:Repeating the final host check... 2023-09-10T00:31:07.203 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stdout 4.1.1 2023-09-10T00:31:07.204 INFO:teuthology.orchestra.run.smithi044.stdout:podman (/bin/podman) version 4.1.1 is present 2023-09-10T00:31:07.204 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl is present 2023-09-10T00:31:07.204 INFO:teuthology.orchestra.run.smithi044.stdout:lvcreate is present 2023-09-10T00:31:07.210 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-10T00:31:07.210 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-10T00:31:07.215 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-10T00:31:07.216 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2023-09-10T00:31:07.221 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout enabled 2023-09-10T00:31:07.228 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout active 2023-09-10T00:31:07.228 INFO:teuthology.orchestra.run.smithi044.stdout:Unit chronyd.service is enabled and running 2023-09-10T00:31:07.228 INFO:teuthology.orchestra.run.smithi044.stdout:Host looks OK 2023-09-10T00:31:07.228 INFO:teuthology.orchestra.run.smithi044.stdout:Cluster fsid: 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:31:07.228 INFO:teuthology.orchestra.run.smithi044.stdout:Acquiring lock 139984651450856 on /run/cephadm/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da.lock 2023-09-10T00:31:07.229 INFO:teuthology.orchestra.run.smithi044.stdout:Lock 139984651450856 acquired on /run/cephadm/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da.lock 2023-09-10T00:31:07.229 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying IP 172.21.15.44 port 3300 ... 2023-09-10T00:31:07.230 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying IP 172.21.15.44 port 6789 ... 2023-09-10T00:31:07.230 INFO:teuthology.orchestra.run.smithi044.stdout:Base mon IP(s) is [172.21.15.44:3300, 172.21.15.44:6789], mon addrv is [v2:172.21.15.44:3300,v1:172.21.15.44:6789] 2023-09-10T00:31:07.232 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-09-10T00:31:07.233 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.44 metric 100 2023-09-10T00:31:07.235 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-09-10T00:31:07.235 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-09-10T00:31:07.236 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1767sec hoplimit 64 pref medium 2023-09-10T00:31:07.238 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-09-10T00:31:07.238 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-09-10T00:31:07.238 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-10T00:31:07.238 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout 3: ens1f0: mtu 1500 state UP qlen 1000 2023-09-10T00:31:07.239 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1572/64 scope link 2023-09-10T00:31:07.239 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-10T00:31:07.240 INFO:teuthology.orchestra.run.smithi044.stdout:Mon IP `172.21.15.44` is in CIDR network `172.21.0.0/20` 2023-09-10T00:31:07.240 INFO:teuthology.orchestra.run.smithi044.stdout:Mon IP `172.21.15.44` is in CIDR network `172.21.0.0/20` 2023-09-10T00:31:07.240 INFO:teuthology.orchestra.run.smithi044.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-09-10T00:31:07.240 INFO:teuthology.orchestra.run.smithi044.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-09-10T00:31:07.241 INFO:teuthology.orchestra.run.smithi044.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00... 2023-09-10T00:31:07.689 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stdout 20c3861ef6c981f72050dd1dd3edbdc62747039bda543319e240a6e02f850eea 2023-09-10T00:31:07.689 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00... 2023-09-10T00:31:07.690 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Getting image source signatures 2023-09-10T00:31:07.690 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Copying blob sha256:2d7a35eed4a09aaaa91d235ae74efd6175638ed8fde15ffafe0c3c53db478b03 2023-09-10T00:31:07.690 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Copying blob sha256:d3e1674b1d277d1d98aa3561ad0e3209c8e2f26183d8e8b51d0d9fbac358121d 2023-09-10T00:31:07.690 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Copying config sha256:20c3861ef6c981f72050dd1dd3edbdc62747039bda543319e240a6e02f850eea 2023-09-10T00:31:07.691 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Writing manifest to image destination 2023-09-10T00:31:07.691 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Storing signatures 2023-09-10T00:31:08.541 INFO:teuthology.orchestra.run.smithi044.stdout:ceph: stdout ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T00:31:08.542 INFO:teuthology.orchestra.run.smithi044.stdout:Ceph version: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T00:31:08.542 INFO:teuthology.orchestra.run.smithi044.stdout:Extracting ceph user uid/gid from container image... 2023-09-10T00:31:09.192 INFO:teuthology.orchestra.run.smithi044.stdout:stat: stdout 167 167 2023-09-10T00:31:09.193 INFO:teuthology.orchestra.run.smithi044.stdout:Creating initial keys... 2023-09-10T00:31:09.946 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph-authtool: stdout AQBNDv1kdU2aIhAA0UY+EWhUd3yochrWGk8Dug== 2023-09-10T00:31:10.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph-authtool: stdout AQBODv1kpwSADxAAdF+Gcapj8YnSN1SleF+pLA== 2023-09-10T00:31:12.661 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph-authtool: stdout AQBODv1kJ6K7NxAA0j3thOqnvBYQg1tuPVBVGw== 2023-09-10T00:31:12.662 INFO:teuthology.orchestra.run.smithi044.stdout:Creating initial monmap... 2023-09-10T00:31:14.124 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-10T00:31:14.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-09-10T00:31:14.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:31:14.125 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-10T00:31:14.125 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool for a [v2:172.21.15.44:3300,v1:172.21.15.44:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-10T00:31:14.125 INFO:teuthology.orchestra.run.smithi044.stdout:setting min_mon_release = pacific 2023-09-10T00:31:14.126 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: set fsid to 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:31:14.126 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-10T00:31:14.126 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:31:14.126 INFO:teuthology.orchestra.run.smithi044.stdout:Creating mon... 2023-09-10T00:31:14.880 INFO:teuthology.orchestra.run.smithi044.stdout:create mon.a on 2023-09-10T00:31:15.012 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-09-10T00:31:15.108 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-09-10T00:31:15.246 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da.target → /etc/systemd/system/ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da.target. 2023-09-10T00:31:15.246 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da.target → /etc/systemd/system/ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da.target. 2023-09-10T00:31:15.423 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mon.a 2023-09-10T00:31:15.424 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mon.a.service: Unit ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mon.a.service not loaded. 2023-09-10T00:31:15.517 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da.target.wants/ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mon.a.service → /etc/systemd/system/ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@.service. 2023-09-10T00:31:15.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:15 smithi044 systemd[1]: Starting Ceph mon.a for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:31:16.076 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:15 smithi044 podman[98123]: 2023-09-10T00:31:16.076 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:16 smithi044 bash[98123]: 63383b609ed5e7a3ceaabd9b28c8d12f679fc64bbf93f28e91ca34fccf0325df 2023-09-10T00:31:16.124 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-10T00:31:16.124 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout disabled 2023-09-10T00:31:16.131 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-10T00:31:16.132 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2023-09-10T00:31:16.132 INFO:teuthology.orchestra.run.smithi044.stdout:firewalld.service is not enabled 2023-09-10T00:31:16.132 INFO:teuthology.orchestra.run.smithi044.stdout:Not possible to enable service . firewalld.service is not available 2023-09-10T00:31:16.133 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mon to start... 2023-09-10T00:31:16.133 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mon... 2023-09-10T00:31:16.346 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:16 smithi044 ceph-mon[98162]: mkfs 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:31:16.347 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:16 smithi044 ceph-mon[98162]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-10T00:31:16.347 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:16 smithi044 systemd[1]: Started Ceph mon.a for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:31:17.080 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout cluster: 2023-09-10T00:31:17.082 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout id: 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:31:17.082 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-09-10T00:31:17.082 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2023-09-10T00:31:17.083 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout services: 2023-09-10T00:31:17.083 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.677247s) 2023-09-10T00:31:17.083 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-09-10T00:31:17.083 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-09-10T00:31:17.083 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2023-09-10T00:31:17.084 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout data: 2023-09-10T00:31:17.084 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-09-10T00:31:17.084 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-09-10T00:31:17.084 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-09-10T00:31:17.084 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout pgs: 2023-09-10T00:31:17.085 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2023-09-10T00:31:17.085 INFO:teuthology.orchestra.run.smithi044.stdout:mon is available 2023-09-10T00:31:17.085 INFO:teuthology.orchestra.run.smithi044.stdout:Assimilating anything we can from ceph.conf... 2023-09-10T00:31:17.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:17 smithi044 ceph-mon[98162]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-10T00:31:17.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:17 smithi044 ceph-mon[98162]: monmap e1: 1 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} 2023-09-10T00:31:17.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:17 smithi044 ceph-mon[98162]: fsmap 2023-09-10T00:31:17.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:17 smithi044 ceph-mon[98162]: osdmap e1: 0 total, 0 up, 0 in 2023-09-10T00:31:17.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:17 smithi044 ceph-mon[98162]: mgrmap e1: no daemons active 2023-09-10T00:31:17.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:17 smithi044 ceph-mon[98162]: from='client.? 172.21.15.44:0/2354041112' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-09-10T00:31:18.025 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2023-09-10T00:31:18.026 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [global] 2023-09-10T00:31:18.026 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout fsid = 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:31:18.026 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.44:3300,v1:172.21.15.44:6789] 2023-09-10T00:31:18.026 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-10T00:31:18.026 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-10T00:31:18.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-10T00:31:18.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-10T00:31:18.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2023-09-10T00:31:18.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-10T00:31:18.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-10T00:31:18.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2023-09-10T00:31:18.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [osd] 2023-09-10T00:31:18.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-10T00:31:18.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-10T00:31:18.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-10T00:31:18.028 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-10T00:31:18.029 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-10T00:31:18.029 INFO:teuthology.orchestra.run.smithi044.stdout:Generating new minimal ceph.conf... 2023-09-10T00:31:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:18 smithi044 ceph-mon[98162]: from='client.? 172.21.15.44:0/937899942' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-10T00:31:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:18 smithi044 ceph-mon[98162]: from='client.? 172.21.15.44:0/937899942' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-09-10T00:31:18.966 INFO:teuthology.orchestra.run.smithi044.stdout:Restarting the monitor... 2023-09-10T00:31:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:18 smithi044 systemd[1]: Stopping Ceph mon.a for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:31:19.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:19 smithi044 ceph-mon[98162]: from='client.? 172.21.15.44:0/2724182224' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:31:19.562 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:19 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mon-a[98138]: 2023-09-10T00:31:19.305+0000 7f870fff7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T00:31:19.563 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:19 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mon-a[98138]: 2023-09-10T00:31:19.305+0000 7f870fff7700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-09-10T00:31:19.896 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:19 smithi044 bash[98498]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mon-a 2023-09-10T00:31:20.182 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:19 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mon.a.service: Succeeded. 2023-09-10T00:31:20.183 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:19 smithi044 systemd[1]: Stopped Ceph mon.a for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:31:20.183 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:19 smithi044 systemd[1]: Starting Ceph mon.a for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:31:20.383 INFO:teuthology.orchestra.run.smithi044.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-09-10T00:31:20.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 podman[98636]: 2023-09-10T00:31:20.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: set uid:gid to 167:167 (ceph:ceph) 2023-09-10T00:31:20.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev), process ceph-mon, pid 7 2023-09-10T00:31:20.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 bash[98636]: 762f3af6a5201c9bab8994f1d40bddebe94947d0205ca5cc66a970da5c540118 2023-09-10T00:31:20.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: pidfile_write: ignore empty --pid-file 2023-09-10T00:31:20.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: load: jerasure load: lrc 2023-09-10T00:31:20.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: RocksDB version: 7.9.2 2023-09-10T00:31:20.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Git sha 0 2023-09-10T00:31:20.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Compile date 2023-09-09 15:16:58 2023-09-10T00:31:20.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: DB SUMMARY 2023-09-10T00:31:20.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: DB Session ID: 8UAVSWL0MDB054YJ7PV2 2023-09-10T00:31:20.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: CURRENT file: CURRENT 2023-09-10T00:31:20.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: IDENTITY file: IDENTITY 2023-09-10T00:31:20.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-09-10T00:31:20.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-09-10T00:31:20.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 87976 ; 2023-09-10T00:31:20.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.error_if_exists: 0 2023-09-10T00:31:20.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.create_if_missing: 0 2023-09-10T00:31:20.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.paranoid_checks: 1 2023-09-10T00:31:20.560 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-10T00:31:20.560 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-10T00:31:20.560 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-10T00:31:20.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.env: 0x55c98929bda0 2023-09-10T00:31:20.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.fs: PosixFileSystem 2023-09-10T00:31:20.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.info_log: 0x55c98b1c54e0 2023-09-10T00:31:20.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_file_opening_threads: 16 2023-09-10T00:31:20.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.statistics: (nil) 2023-09-10T00:31:20.562 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.use_fsync: 0 2023-09-10T00:31:20.562 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_log_file_size: 0 2023-09-10T00:31:20.562 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-10T00:31:20.562 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-10T00:31:20.563 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.keep_log_file_num: 1000 2023-09-10T00:31:20.563 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.recycle_log_file_num: 0 2023-09-10T00:31:20.563 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.allow_fallocate: 1 2023-09-10T00:31:20.563 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.allow_mmap_reads: 0 2023-09-10T00:31:20.563 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.allow_mmap_writes: 0 2023-09-10T00:31:20.564 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.use_direct_reads: 0 2023-09-10T00:31:20.564 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-10T00:31:20.564 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.create_missing_column_families: 0 2023-09-10T00:31:20.564 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.db_log_dir: 2023-09-10T00:31:20.566 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.wal_dir: 2023-09-10T00:31:20.566 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-10T00:31:20.566 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-10T00:31:20.566 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-10T00:31:20.567 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-10T00:31:20.567 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-10T00:31:20.567 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-10T00:31:20.567 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.advise_random_on_open: 1 2023-09-10T00:31:20.567 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.db_write_buffer_size: 0 2023-09-10T00:31:20.567 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.write_buffer_manager: 0x55c98b17f4a0 2023-09-10T00:31:20.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-10T00:31:20.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-10T00:31:20.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-10T00:31:20.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.rate_limiter: (nil) 2023-09-10T00:31:20.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-10T00:31:20.569 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.wal_recovery_mode: 2 2023-09-10T00:31:20.569 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.enable_thread_tracking: 0 2023-09-10T00:31:20.569 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.enable_pipelined_write: 0 2023-09-10T00:31:20.569 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.unordered_write: 0 2023-09-10T00:31:20.569 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-10T00:31:20.570 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-10T00:31:20.570 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-10T00:31:20.570 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-10T00:31:20.570 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.row_cache: None 2023-09-10T00:31:20.570 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.wal_filter: None 2023-09-10T00:31:20.571 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-10T00:31:20.571 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.allow_ingest_behind: 0 2023-09-10T00:31:20.571 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.two_write_queues: 0 2023-09-10T00:31:20.571 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.manual_wal_flush: 0 2023-09-10T00:31:20.571 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.wal_compression: 0 2023-09-10T00:31:20.571 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.atomic_flush: 0 2023-09-10T00:31:20.572 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-10T00:31:20.572 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-10T00:31:20.572 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-10T00:31:20.572 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.log_readahead_size: 0 2023-09-10T00:31:20.572 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-10T00:31:20.573 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.best_efforts_recovery: 0 2023-09-10T00:31:20.573 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-10T00:31:20.573 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-10T00:31:20.573 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.allow_data_in_errors: 0 2023-09-10T00:31:20.573 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.db_host_id: __hostname__ 2023-09-10T00:31:20.573 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-10T00:31:20.574 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_background_jobs: 2 2023-09-10T00:31:20.574 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_background_compactions: -1 2023-09-10T00:31:20.575 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_subcompactions: 1 2023-09-10T00:31:20.575 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-10T00:31:20.575 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-10T00:31:20.575 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-10T00:31:20.576 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_total_wal_size: 0 2023-09-10T00:31:20.576 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-10T00:31:20.576 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-10T00:31:20.576 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-10T00:31:20.577 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-10T00:31:20.577 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_open_files: -1 2023-09-10T00:31:20.577 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.bytes_per_sync: 0 2023-09-10T00:31:20.578 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-10T00:31:20.578 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-10T00:31:20.578 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compaction_readahead_size: 0 2023-09-10T00:31:20.578 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_background_flushes: -1 2023-09-10T00:31:20.579 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Compression algorithms supported: 2023-09-10T00:31:20.579 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-10T00:31:20.579 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: kZSTD supported: 0 2023-09-10T00:31:20.580 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: kXpressCompression supported: 0 2023-09-10T00:31:20.580 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: kLZ4HCCompression supported: 1 2023-09-10T00:31:20.580 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: kZlibCompression supported: 1 2023-09-10T00:31:20.580 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: kSnappyCompression supported: 1 2023-09-10T00:31:20.580 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: kLZ4Compression supported: 1 2023-09-10T00:31:20.581 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: kBZip2Compression supported: 0 2023-09-10T00:31:20.581 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-10T00:31:20.581 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-10T00:31:20.581 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-09-10T00:31:20.581 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-10T00:31:20.582 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-10T00:31:20.582 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.merge_operator: 2023-09-10T00:31:20.582 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compaction_filter: None 2023-09-10T00:31:20.582 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compaction_filter_factory: None 2023-09-10T00:31:20.582 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.sst_partitioner_factory: None 2023-09-10T00:31:20.583 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-10T00:31:20.583 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-10T00:31:20.583 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c98b1c55e0) 2023-09-10T00:31:20.583 INFO:journalctl@ceph.mon.a.smithi044.stdout: cache_index_and_filter_blocks: 1 2023-09-10T00:31:20.584 INFO:journalctl@ceph.mon.a.smithi044.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-10T00:31:20.584 INFO:journalctl@ceph.mon.a.smithi044.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-10T00:31:20.584 INFO:journalctl@ceph.mon.a.smithi044.stdout: pin_top_level_index_and_filter: 1 2023-09-10T00:31:20.584 INFO:journalctl@ceph.mon.a.smithi044.stdout: index_type: 0 2023-09-10T00:31:20.584 INFO:journalctl@ceph.mon.a.smithi044.stdout: data_block_index_type: 0 2023-09-10T00:31:20.585 INFO:journalctl@ceph.mon.a.smithi044.stdout: index_shortening: 1 2023-09-10T00:31:20.585 INFO:journalctl@ceph.mon.a.smithi044.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-10T00:31:20.585 INFO:journalctl@ceph.mon.a.smithi044.stdout: checksum: 4 2023-09-10T00:31:20.585 INFO:journalctl@ceph.mon.a.smithi044.stdout: no_block_cache: 0 2023-09-10T00:31:20.585 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_cache: 0x55c98b21d1f0 2023-09-10T00:31:20.585 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_cache_name: BinnedLRUCache 2023-09-10T00:31:20.586 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_cache_options: 2023-09-10T00:31:20.586 INFO:journalctl@ceph.mon.a.smithi044.stdout: capacity : 536870912 2023-09-10T00:31:20.586 INFO:journalctl@ceph.mon.a.smithi044.stdout: num_shard_bits : 4 2023-09-10T00:31:20.586 INFO:journalctl@ceph.mon.a.smithi044.stdout: strict_capacity_limit : 0 2023-09-10T00:31:20.586 INFO:journalctl@ceph.mon.a.smithi044.stdout: high_pri_pool_ratio: 0.000 2023-09-10T00:31:20.587 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_cache_compressed: (nil) 2023-09-10T00:31:20.587 INFO:journalctl@ceph.mon.a.smithi044.stdout: persistent_cache: (nil) 2023-09-10T00:31:20.587 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_size: 4096 2023-09-10T00:31:20.587 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_size_deviation: 10 2023-09-10T00:31:20.587 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_restart_interval: 16 2023-09-10T00:31:20.588 INFO:journalctl@ceph.mon.a.smithi044.stdout: index_block_restart_interval: 1 2023-09-10T00:31:20.588 INFO:journalctl@ceph.mon.a.smithi044.stdout: metadata_block_size: 4096 2023-09-10T00:31:20.588 INFO:journalctl@ceph.mon.a.smithi044.stdout: partition_filters: 0 2023-09-10T00:31:20.588 INFO:journalctl@ceph.mon.a.smithi044.stdout: use_delta_encoding: 1 2023-09-10T00:31:20.588 INFO:journalctl@ceph.mon.a.smithi044.stdout: filter_policy: bloomfilter 2023-09-10T00:31:20.588 INFO:journalctl@ceph.mon.a.smithi044.stdout: whole_key_filtering: 1 2023-09-10T00:31:20.589 INFO:journalctl@ceph.mon.a.smithi044.stdout: verify_compression: 0 2023-09-10T00:31:20.589 INFO:journalctl@ceph.mon.a.smithi044.stdout: read_amp_bytes_per_bit: 0 2023-09-10T00:31:20.589 INFO:journalctl@ceph.mon.a.smithi044.stdout: format_version: 5 2023-09-10T00:31:20.589 INFO:journalctl@ceph.mon.a.smithi044.stdout: enable_index_compression: 1 2023-09-10T00:31:20.589 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_align: 0 2023-09-10T00:31:20.590 INFO:journalctl@ceph.mon.a.smithi044.stdout: max_auto_readahead_size: 262144 2023-09-10T00:31:20.590 INFO:journalctl@ceph.mon.a.smithi044.stdout: prepopulate_block_cache: 0 2023-09-10T00:31:20.590 INFO:journalctl@ceph.mon.a.smithi044.stdout: initial_auto_readahead_size: 8192 2023-09-10T00:31:20.590 INFO:journalctl@ceph.mon.a.smithi044.stdout: num_file_reads_for_auto_readahead: 2 2023-09-10T00:31:20.590 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.write_buffer_size: 33554432 2023-09-10T00:31:20.591 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_write_buffer_number: 2 2023-09-10T00:31:20.591 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compression: NoCompression 2023-09-10T00:31:20.591 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.bottommost_compression: Disabled 2023-09-10T00:31:20.591 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.prefix_extractor: nullptr 2023-09-10T00:31:20.591 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-10T00:31:20.591 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.num_levels: 7 2023-09-10T00:31:20.592 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-10T00:31:20.592 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-10T00:31:20.592 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-10T00:31:20.592 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-10T00:31:20.592 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-10T00:31:20.593 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-10T00:31:20.593 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-10T00:31:20.593 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-10T00:31:20.593 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-10T00:31:20.593 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-10T00:31:20.593 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-10T00:31:20.594 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-10T00:31:20.594 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-10T00:31:20.594 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compression_opts.level: 32767 2023-09-10T00:31:20.594 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compression_opts.strategy: 0 2023-09-10T00:31:20.594 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-10T00:31:20.595 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-10T00:31:20.595 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-10T00:31:20.595 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-10T00:31:20.595 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compression_opts.enabled: false 2023-09-10T00:31:20.595 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-10T00:31:20.595 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-10T00:31:20.596 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-10T00:31:20.596 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-10T00:31:20.596 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.target_file_size_base: 67108864 2023-09-10T00:31:20.596 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-10T00:31:20.596 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-10T00:31:20.596 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-10T00:31:20.597 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-10T00:31:20.597 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-10T00:31:20.597 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-10T00:31:20.597 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-10T00:31:20.597 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-10T00:31:20.598 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-10T00:31:20.598 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-10T00:31:20.598 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-10T00:31:20.598 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-10T00:31:20.598 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-10T00:31:20.598 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-10T00:31:20.599 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.arena_block_size: 1048576 2023-09-10T00:31:20.599 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-10T00:31:20.599 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-10T00:31:20.599 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.disable_auto_compactions: 0 2023-09-10T00:31:20.599 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-10T00:31:20.600 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-10T00:31:20.600 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-10T00:31:20.600 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-10T00:31:20.600 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-10T00:31:20.600 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-10T00:31:20.600 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-10T00:31:20.601 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-10T00:31:20.601 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-10T00:31:20.601 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-10T00:31:20.601 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.table_properties_collectors: 2023-09-10T00:31:20.601 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.inplace_update_support: 0 2023-09-10T00:31:20.602 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-10T00:31:20.602 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-10T00:31:20.602 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-10T00:31:20.602 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-10T00:31:20.602 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.bloom_locality: 0 2023-09-10T00:31:20.602 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.max_successive_merges: 0 2023-09-10T00:31:20.603 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-10T00:31:20.603 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.paranoid_file_checks: 0 2023-09-10T00:31:20.603 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.force_consistency_checks: 1 2023-09-10T00:31:20.603 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.report_bg_io_stats: 0 2023-09-10T00:31:20.603 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.ttl: 2592000 2023-09-10T00:31:20.604 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-10T00:31:20.604 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-10T00:31:20.604 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-10T00:31:20.604 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.enable_blob_files: false 2023-09-10T00:31:20.604 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.min_blob_size: 0 2023-09-10T00:31:20.604 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.blob_file_size: 268435456 2023-09-10T00:31:20.605 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-10T00:31:20.605 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-10T00:31:20.605 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-10T00:31:20.605 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-10T00:31:20.605 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-10T00:31:20.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.blob_file_starting_level: 0 2023-09-10T00:31:20.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-10T00:31:20.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-09-10T00:31:20.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-09-10T00:31:20.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: de63c1f5-36d3-4f08-9d59-8de9bdb305f4 2023-09-10T00:31:20.606 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694305880351989, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-09-10T00:31:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-09-10T00:31:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694305880352908, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 84256, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 268, "table_properties": {"data_size": 82424, "index_size": 242, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10943, "raw_average_key_size": 48, "raw_value_size": 76261, "raw_average_value_size": 335, "num_data_blocks": 10, "num_entries": 227, "num_filter_entries": 227, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1694305880, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "de63c1f5-36d3-4f08-9d59-8de9bdb305f4", "db_session_id": "8UAVSWL0MDB054YJ7PV2", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-09-10T00:31:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694305880352980, "job": 1, "event": "recovery_finished"} 2023-09-10T00:31:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-09-10T00:31:20.607 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-10T00:31:20.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c98b2b8000 2023-09-10T00:31:20.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: DB pointer 0x55c98b2a4000 2023-09-10T00:31:20.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-10T00:31:20.608 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-10T00:31:20.608 INFO:journalctl@ceph.mon.a.smithi044.stdout: ** DB Stats ** 2023-09-10T00:31:20.608 INFO:journalctl@ceph.mon.a.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-10T00:31:20.609 INFO:journalctl@ceph.mon.a.smithi044.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-10T00:31:20.609 INFO:journalctl@ceph.mon.a.smithi044.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-10T00:31:20.609 INFO:journalctl@ceph.mon.a.smithi044.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-10T00:31:20.609 INFO:journalctl@ceph.mon.a.smithi044.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-10T00:31:20.609 INFO:journalctl@ceph.mon.a.smithi044.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-10T00:31:20.610 INFO:journalctl@ceph.mon.a.smithi044.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-10T00:31:20.610 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2023-09-10T00:31:20.610 INFO:journalctl@ceph.mon.a.smithi044.stdout: ** Compaction Stats [default] ** 2023-09-10T00:31:20.610 INFO:journalctl@ceph.mon.a.smithi044.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-10T00:31:20.610 INFO:journalctl@ceph.mon.a.smithi044.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-10T00:31:20.610 INFO:journalctl@ceph.mon.a.smithi044.stdout: L0 2/0 84.12 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-10T00:31:20.611 INFO:journalctl@ceph.mon.a.smithi044.stdout: Sum 2/0 84.12 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-10T00:31:20.611 INFO:journalctl@ceph.mon.a.smithi044.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-10T00:31:20.611 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2023-09-10T00:31:20.611 INFO:journalctl@ceph.mon.a.smithi044.stdout: ** Compaction Stats [default] ** 2023-09-10T00:31:20.611 INFO:journalctl@ceph.mon.a.smithi044.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-10T00:31:20.612 INFO:journalctl@ceph.mon.a.smithi044.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-10T00:31:20.612 INFO:journalctl@ceph.mon.a.smithi044.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 141.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-10T00:31:20.612 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2023-09-10T00:31:20.612 INFO:journalctl@ceph.mon.a.smithi044.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-10T00:31:20.612 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2023-09-10T00:31:20.612 INFO:journalctl@ceph.mon.a.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-10T00:31:20.613 INFO:journalctl@ceph.mon.a.smithi044.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-10T00:31:20.613 INFO:journalctl@ceph.mon.a.smithi044.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-10T00:31:20.613 INFO:journalctl@ceph.mon.a.smithi044.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-10T00:31:20.613 INFO:journalctl@ceph.mon.a.smithi044.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-10T00:31:20.613 INFO:journalctl@ceph.mon.a.smithi044.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-10T00:31:20.614 INFO:journalctl@ceph.mon.a.smithi044.stdout: Cumulative compaction: 0.00 GB write, 14.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-10T00:31:20.614 INFO:journalctl@ceph.mon.a.smithi044.stdout: Interval compaction: 0.00 GB write, 14.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-10T00:31:20.614 INFO:journalctl@ceph.mon.a.smithi044.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-10T00:31:20.614 INFO:journalctl@ceph.mon.a.smithi044.stdout: Block cache BinnedLRUCache@0x55c98b21d1f0#7 capacity: 512.00 MB usage: 1.20 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-09-10T00:31:20.614 INFO:journalctl@ceph.mon.a.smithi044.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2023-09-10T00:31:20.615 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2023-09-10T00:31:20.615 INFO:journalctl@ceph.mon.a.smithi044.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-10T00:31:20.615 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: starting mon.a rank 0 at public addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] at bind addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:31:20.615 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: mon.a@-1(???) e1 preinit fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:31:20.615 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: mon.a@-1(???).mds e1 new map 2023-09-10T00:31:20.615 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: mon.a@-1(???).mds e1 print_map 2023-09-10T00:31:20.616 INFO:journalctl@ceph.mon.a.smithi044.stdout: e1 2023-09-10T00:31:20.616 INFO:journalctl@ceph.mon.a.smithi044.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-10T00:31:20.616 INFO:journalctl@ceph.mon.a.smithi044.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-10T00:31:20.616 INFO:journalctl@ceph.mon.a.smithi044.stdout: legacy client fscid: -1 2023-09-10T00:31:20.616 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2023-09-10T00:31:20.616 INFO:journalctl@ceph.mon.a.smithi044.stdout: No filesystems configured 2023-09-10T00:31:20.617 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-10T00:31:20.617 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-10T00:31:20.617 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-10T00:31:20.617 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-10T00:31:20.617 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-09-10T00:31:20.618 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: expand_channel_meta expand map: {default=false} 2023-09-10T00:31:20.618 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: expand_channel_meta from 'false' to 'false' 2023-09-10T00:31:20.618 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: expand_channel_meta expanded map: {default=false} 2023-09-10T00:31:20.618 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: expand_channel_meta expand map: {default=info} 2023-09-10T00:31:20.618 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: expand_channel_meta from 'info' to 'info' 2023-09-10T00:31:20.619 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: expand_channel_meta expanded map: {default=info} 2023-09-10T00:31:20.619 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: expand_channel_meta expand map: {default=daemon} 2023-09-10T00:31:20.619 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-10T00:31:20.619 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: expand_channel_meta expanded map: {default=daemon} 2023-09-10T00:31:20.619 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: expand_channel_meta expand map: {default=debug} 2023-09-10T00:31:20.619 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: expand_channel_meta from 'debug' to 'debug' 2023-09-10T00:31:20.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: expand_channel_meta expanded map: {default=debug} 2023-09-10T00:31:20.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 systemd[1]: Started Ceph mon.a for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:31:20.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-10T00:31:20.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: monmap e1: 1 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} 2023-09-10T00:31:20.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: fsmap 2023-09-10T00:31:20.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: osdmap e1: 0 total, 0 up, 0 in 2023-09-10T00:31:20.621 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:20 smithi044 ceph-mon[98676]: mgrmap e1: no daemons active 2023-09-10T00:31:21.433 INFO:teuthology.orchestra.run.smithi044.stdout:Wrote config to /etc/ceph/ceph.conf 2023-09-10T00:31:21.453 INFO:teuthology.orchestra.run.smithi044.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-09-10T00:31:21.454 INFO:teuthology.orchestra.run.smithi044.stdout:Creating mgr... 2023-09-10T00:31:21.454 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying port 0.0.0.0:9283 ... 2023-09-10T00:31:21.455 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying port 0.0.0.0:8765 ... 2023-09-10T00:31:21.657 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mgr.x 2023-09-10T00:31:21.657 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mgr.x.service: Unit ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mgr.x.service not loaded. 2023-09-10T00:31:21.750 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da.target.wants/ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mgr.x.service → /etc/systemd/system/ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@.service. 2023-09-10T00:31:22.049 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:21 smithi044 systemd[1]: Starting Ceph mgr.x for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:31:22.303 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:22 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1740039955' entity='client.admin' 2023-09-10T00:31:22.304 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:22 smithi044 podman[98917]: 2023-09-10T00:31:22.360 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-10T00:31:22.360 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout disabled 2023-09-10T00:31:22.367 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-10T00:31:22.368 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2023-09-10T00:31:22.368 INFO:teuthology.orchestra.run.smithi044.stdout:firewalld.service is not enabled 2023-09-10T00:31:22.368 INFO:teuthology.orchestra.run.smithi044.stdout:Not possible to enable service . firewalld.service is not available 2023-09-10T00:31:22.376 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-10T00:31:22.376 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout disabled 2023-09-10T00:31:22.384 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-10T00:31:22.384 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2023-09-10T00:31:22.384 INFO:teuthology.orchestra.run.smithi044.stdout:firewalld.service is not enabled 2023-09-10T00:31:22.384 INFO:teuthology.orchestra.run.smithi044.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-09-10T00:31:22.385 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mgr to start... 2023-09-10T00:31:22.385 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mgr... 2023-09-10T00:31:22.554 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:22 smithi044 bash[98917]: 264933d0183fbb87955383287e5fb569bd145ce36c455c89fd717f40118866c8 2023-09-10T00:31:22.555 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:22 smithi044 systemd[1]: Started Ceph mgr.x for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:31:22.555 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:22 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:22.443+0000 7f8ebaec2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-10T00:31:22.555 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:22 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:22.540+0000 7f8ebaec2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-10T00:31:22.984 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:22 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:22.654+0000 7f8ebaec2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-10T00:31:22.984 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:22 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:22.768+0000 7f8ebaec2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-10T00:31:22.984 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:22 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:22.983+0000 7f8ebaec2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-10T00:31:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:23 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3969318264' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-10T00:31:23.335 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2023-09-10T00:31:23.336 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2023-09-10T00:31:23.336 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsid": "3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da", 2023-09-10T00:31:23.336 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "health": { 2023-09-10T00:31:23.336 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-10T00:31:23.336 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-10T00:31:23.337 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-10T00:31:23.337 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:23.337 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-10T00:31:23.337 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-10T00:31:23.337 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 0 2023-09-10T00:31:23.337 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2023-09-10T00:31:23.338 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-10T00:31:23.338 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "a" 2023-09-10T00:31:23.338 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2023-09-10T00:31:23.338 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-09-10T00:31:23.338 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-10T00:31:23.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-10T00:31:23.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-10T00:31:23.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-10T00:31:23.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:23.339 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-10T00:31:23.340 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-10T00:31:23.340 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-10T00:31:23.340 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-10T00:31:23.340 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-10T00:31:23.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-10T00:31:23.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-10T00:31:23.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-10T00:31:23.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:23.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-10T00:31:23.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-10T00:31:23.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-10T00:31:23.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-10T00:31:23.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-10T00:31:23.348 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-10T00:31:23.348 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-10T00:31:23.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-10T00:31:23.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-10T00:31:23.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:23.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-10T00:31:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-10T00:31:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-10T00:31:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-10T00:31:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-10T00:31:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-10T00:31:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-10T00:31:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-10T00:31:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-10T00:31:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-10T00:31:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "restful" 2023-09-10T00:31:23.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2023-09-10T00:31:23.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-10T00:31:23.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:23.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-10T00:31:23.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-10T00:31:23.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modified": "2023-09-10T00:31:16.074247+0000", 2023-09-10T00:31:23.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-10T00:31:23.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:23.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-10T00:31:23.353 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2023-09-10T00:31:23.354 INFO:teuthology.orchestra.run.smithi044.stdout:mgr not available, waiting (1/15)... 2023-09-10T00:31:23.622 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:23 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:23.470+0000 7f8ebaec2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-10T00:31:24.055 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:23 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:23.622+0000 7f8ebaec2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-10T00:31:24.055 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:23 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:23.730+0000 7f8ebaec2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-10T00:31:24.055 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:23 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:23.845+0000 7f8ebaec2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-10T00:31:24.805 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:24 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:24.355+0000 7f8ebaec2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-10T00:31:25.554 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:25 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:25.148+0000 7f8ebaec2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-10T00:31:25.554 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:25 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:25.247+0000 7f8ebaec2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-10T00:31:26.361 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:26 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:26.258+0000 7f8ebaec2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-10T00:31:26.485 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2023-09-10T00:31:26.486 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2023-09-10T00:31:26.486 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsid": "3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da", 2023-09-10T00:31:26.486 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "health": { 2023-09-10T00:31:26.486 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-10T00:31:26.487 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-10T00:31:26.487 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-10T00:31:26.487 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:26.487 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-10T00:31:26.487 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-10T00:31:26.488 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 0 2023-09-10T00:31:26.488 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2023-09-10T00:31:26.488 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-10T00:31:26.488 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "a" 2023-09-10T00:31:26.488 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2023-09-10T00:31:26.488 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2023-09-10T00:31:26.489 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-10T00:31:26.489 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-10T00:31:26.489 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-10T00:31:26.489 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-10T00:31:26.489 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:26.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-10T00:31:26.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-10T00:31:26.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-10T00:31:26.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-10T00:31:26.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-10T00:31:26.494 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-10T00:31:26.494 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-10T00:31:26.494 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-10T00:31:26.495 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:26.495 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-10T00:31:26.495 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-10T00:31:26.495 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-10T00:31:26.495 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-10T00:31:26.495 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-10T00:31:26.496 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-10T00:31:26.496 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-10T00:31:26.496 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-10T00:31:26.496 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-10T00:31:26.496 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:26.497 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-10T00:31:26.497 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-10T00:31:26.497 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-10T00:31:26.497 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-10T00:31:26.497 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:26.497 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-10T00:31:26.498 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-10T00:31:26.498 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-10T00:31:26.498 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-10T00:31:26.498 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-10T00:31:26.498 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-10T00:31:26.499 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "restful" 2023-09-10T00:31:26.499 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2023-09-10T00:31:26.499 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-10T00:31:26.499 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:26.499 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-10T00:31:26.500 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-10T00:31:26.500 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modified": "2023-09-10T00:31:16.074247+0000", 2023-09-10T00:31:26.500 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-10T00:31:26.500 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:26.500 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-10T00:31:26.501 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2023-09-10T00:31:26.501 INFO:teuthology.orchestra.run.smithi044.stdout:mgr not available, waiting (2/15)... 2023-09-10T00:31:26.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:26 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2344078544' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-10T00:31:26.806 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:26 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:26.370+0000 7f8ebaec2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-10T00:31:26.806 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:26 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:26.593+0000 7f8ebaec2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-10T00:31:27.195 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:26 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:26.942+0000 7f8ebaec2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-10T00:31:27.555 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:27 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:27.195+0000 7f8ebaec2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-10T00:31:27.555 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:27 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:27.377+0000 7f8ebaec2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-10T00:31:28.055 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:27 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:27.706+0000 7f8ebaec2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-10T00:31:28.984 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:28 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:28.568+0000 7f8ebaec2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-10T00:31:28.984 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:28 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:28.731+0000 7f8ebaec2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-10T00:31:28.984 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:28 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:28.983+0000 7f8ebaec2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-10T00:31:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:29 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3396915541' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-10T00:31:29.305 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:29 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:29.121+0000 7f8ebaec2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-10T00:31:29.804 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:29 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:29.398+0000 7f8ebaec2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-10T00:31:30.805 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:30 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:30.450+0000 7f8ebaec2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-10T00:31:31.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2023-09-10T00:31:31.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2023-09-10T00:31:31.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsid": "3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da", 2023-09-10T00:31:31.138 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "health": { 2023-09-10T00:31:31.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-10T00:31:31.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-10T00:31:31.139 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-10T00:31:31.141 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:31.141 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-10T00:31:31.142 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-10T00:31:31.142 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 0 2023-09-10T00:31:31.142 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2023-09-10T00:31:31.142 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-10T00:31:31.142 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "a" 2023-09-10T00:31:31.143 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2023-09-10T00:31:31.143 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2023-09-10T00:31:31.143 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-10T00:31:31.143 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-10T00:31:31.143 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-10T00:31:31.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-10T00:31:31.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:31.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-10T00:31:31.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-10T00:31:31.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-10T00:31:31.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-10T00:31:31.145 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-10T00:31:31.145 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-10T00:31:31.148 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-10T00:31:31.148 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-10T00:31:31.149 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:31.149 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-10T00:31:31.149 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-10T00:31:31.149 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-10T00:31:31.149 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-10T00:31:31.150 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-10T00:31:31.150 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-10T00:31:31.150 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-10T00:31:31.150 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-10T00:31:31.150 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-10T00:31:31.151 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:31.151 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-10T00:31:31.151 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-10T00:31:31.151 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-10T00:31:31.151 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-10T00:31:31.152 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:31.152 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-10T00:31:31.152 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-10T00:31:31.152 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-10T00:31:31.152 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-10T00:31:31.153 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-10T00:31:31.153 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-10T00:31:31.153 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "restful" 2023-09-10T00:31:31.153 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2023-09-10T00:31:31.153 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-10T00:31:31.154 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:31.154 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-10T00:31:31.154 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-10T00:31:31.154 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modified": "2023-09-10T00:31:16.074247+0000", 2023-09-10T00:31:31.154 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-10T00:31:31.155 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:31.155 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-10T00:31:31.155 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2023-09-10T00:31:31.155 INFO:teuthology.orchestra.run.smithi044.stdout:mgr not available, waiting (3/15)... 2023-09-10T00:31:31.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:31 smithi044 ceph-mon[98676]: Activating manager daemon x 2023-09-10T00:31:31.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:31 smithi044 ceph-mon[98676]: mgrmap e2: x(active, starting, since 0.00315677s) 2023-09-10T00:31:31.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:31 smithi044 ceph-mon[98676]: from='mgr.14100 172.21.15.44:0/3606743862' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-10T00:31:31.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:31 smithi044 ceph-mon[98676]: from='mgr.14100 172.21.15.44:0/3606743862' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-10T00:31:31.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:31 smithi044 ceph-mon[98676]: from='mgr.14100 172.21.15.44:0/3606743862' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-10T00:31:31.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:31 smithi044 ceph-mon[98676]: from='mgr.14100 172.21.15.44:0/3606743862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:31:31.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:31 smithi044 ceph-mon[98676]: from='mgr.14100 172.21.15.44:0/3606743862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-10T00:31:31.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:31 smithi044 ceph-mon[98676]: Manager daemon x is now available 2023-09-10T00:31:31.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:31 smithi044 ceph-mon[98676]: from='mgr.14100 172.21.15.44:0/3606743862' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-10T00:31:31.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:31 smithi044 ceph-mon[98676]: from='mgr.14100 172.21.15.44:0/3606743862' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-10T00:31:31.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:31 smithi044 ceph-mon[98676]: from='mgr.14100 172.21.15.44:0/3606743862' entity='mgr.x' 2023-09-10T00:31:31.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:31 smithi044 ceph-mon[98676]: from='mgr.14100 172.21.15.44:0/3606743862' entity='mgr.x' 2023-09-10T00:31:31.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:31 smithi044 ceph-mon[98676]: from='mgr.14100 172.21.15.44:0/3606743862' entity='mgr.x' 2023-09-10T00:31:32.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:32 smithi044 ceph-mon[98676]: mgrmap e3: x(active, since 1.00943s) 2023-09-10T00:31:34.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:34 smithi044 ceph-mon[98676]: mgrmap e4: x(active, since 3s) 2023-09-10T00:31:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:36 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/876764479' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-10T00:31:38.707 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2023-09-10T00:31:38.707 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2023-09-10T00:31:38.707 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsid": "3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da", 2023-09-10T00:31:38.708 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "health": { 2023-09-10T00:31:38.708 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-10T00:31:38.708 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-10T00:31:38.708 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-10T00:31:38.708 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:38.708 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-10T00:31:38.710 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-10T00:31:38.711 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 0 2023-09-10T00:31:38.711 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2023-09-10T00:31:38.711 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-10T00:31:38.711 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "a" 2023-09-10T00:31:38.711 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2023-09-10T00:31:38.712 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2023-09-10T00:31:38.712 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-10T00:31:38.712 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-10T00:31:38.712 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-10T00:31:38.712 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-10T00:31:38.712 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:38.713 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-10T00:31:38.713 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-10T00:31:38.713 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-10T00:31:38.713 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-10T00:31:38.713 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-10T00:31:38.714 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-10T00:31:38.717 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-10T00:31:38.717 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-10T00:31:38.717 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:38.717 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-10T00:31:38.718 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-10T00:31:38.718 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-10T00:31:38.718 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-10T00:31:38.718 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-10T00:31:38.718 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-10T00:31:38.719 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-10T00:31:38.719 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-10T00:31:38.719 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-10T00:31:38.719 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:38.719 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-10T00:31:38.720 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-10T00:31:38.720 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-10T00:31:38.720 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-10T00:31:38.720 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:38.720 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-10T00:31:38.720 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-10T00:31:38.721 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-10T00:31:38.721 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-10T00:31:38.721 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-10T00:31:38.721 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-10T00:31:38.721 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "restful" 2023-09-10T00:31:38.722 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2023-09-10T00:31:38.722 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-10T00:31:38.722 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:38.722 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-10T00:31:38.722 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-10T00:31:38.723 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modified": "2023-09-10T00:31:16.074247+0000", 2023-09-10T00:31:38.723 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-10T00:31:38.723 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2023-09-10T00:31:38.723 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-10T00:31:38.723 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2023-09-10T00:31:38.723 INFO:teuthology.orchestra.run.smithi044.stdout:mgr is available 2023-09-10T00:31:39.965 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2023-09-10T00:31:39.965 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [global] 2023-09-10T00:31:39.965 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout fsid = 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:31:39.965 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-10T00:31:39.966 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-10T00:31:39.966 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-10T00:31:39.966 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-10T00:31:39.966 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2023-09-10T00:31:39.967 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-10T00:31:39.967 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-10T00:31:39.967 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2023-09-10T00:31:39.967 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [osd] 2023-09-10T00:31:39.967 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-10T00:31:39.968 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-10T00:31:39.968 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-10T00:31:39.968 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-10T00:31:39.968 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-10T00:31:39.968 INFO:teuthology.orchestra.run.smithi044.stdout:Enabling cephadm module... 2023-09-10T00:31:39.982 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:39 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2357730707' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-10T00:31:41.658 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:41 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3493250186' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-09-10T00:31:41.658 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:41 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: ignoring --setuser ceph since I am not root 2023-09-10T00:31:41.659 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:41 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: ignoring --setgroup ceph since I am not root 2023-09-10T00:31:41.659 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:41 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:41.561+0000 7fa01046b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-10T00:31:41.928 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:41 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:41.658+0000 7fa01046b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-10T00:31:41.929 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:41 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:41.766+0000 7fa01046b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-10T00:31:41.929 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:41 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:41.882+0000 7fa01046b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-10T00:31:42.304 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:42 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:42.098+0000 7fa01046b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-10T00:31:42.728 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:42 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:42.575+0000 7fa01046b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-10T00:31:42.729 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:42 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3493250186' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-09-10T00:31:42.729 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:42 smithi044 ceph-mon[98676]: mgrmap e5: x(active, since 10s) 2023-09-10T00:31:43.054 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:42 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:42.728+0000 7fa01046b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-10T00:31:43.055 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:42 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:42.841+0000 7fa01046b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-10T00:31:43.055 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:42 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:42.959+0000 7fa01046b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-10T00:31:43.150 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2023-09-10T00:31:43.150 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-09-10T00:31:43.151 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-10T00:31:43.151 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-10T00:31:43.151 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-10T00:31:43.151 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2023-09-10T00:31:43.151 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for the mgr to restart... 2023-09-10T00:31:43.152 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mgr epoch 5... 2023-09-10T00:31:43.391 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:43 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3502269882' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-10T00:31:43.804 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:43 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:43.483+0000 7fa01046b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-10T00:31:44.554 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:44 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:44.269+0000 7fa01046b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-10T00:31:44.555 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:44 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:44.368+0000 7fa01046b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-10T00:31:45.676 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:45 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:45.342+0000 7fa01046b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-10T00:31:45.677 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:45 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:45.454+0000 7fa01046b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-10T00:31:46.278 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:45 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:45.674+0000 7fa01046b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-10T00:31:46.278 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:46 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:46.024+0000 7fa01046b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-10T00:31:46.555 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:46 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:46.277+0000 7fa01046b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-10T00:31:46.555 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:46 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:46.460+0000 7fa01046b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-10T00:31:47.054 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:46 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:46.789+0000 7fa01046b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-10T00:31:48.021 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:47 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:47.617+0000 7fa01046b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-10T00:31:48.022 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:47 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:47.775+0000 7fa01046b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-10T00:31:48.305 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:48 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:48.021+0000 7fa01046b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-10T00:31:48.305 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:48 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:48.158+0000 7fa01046b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-10T00:31:48.805 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:48 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:48.430+0000 7fa01046b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-10T00:31:49.805 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:49 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:31:49.482+0000 7fa01046b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-10T00:31:49.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:49 smithi044 ceph-mon[98676]: Active manager daemon x restarted 2023-09-10T00:31:49.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:49 smithi044 ceph-mon[98676]: Activating manager daemon x 2023-09-10T00:31:49.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:49 smithi044 ceph-mon[98676]: osdmap e2: 0 total, 0 up, 0 in 2023-09-10T00:31:49.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:49 smithi044 ceph-mon[98676]: mgrmap e6: x(active, starting, since 0.00380996s) 2023-09-10T00:31:49.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:49 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:31:49.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:49 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-10T00:31:49.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:49 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-10T00:31:49.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:49 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-10T00:31:49.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:49 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-10T00:31:49.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:49 smithi044 ceph-mon[98676]: Manager daemon x is now available 2023-09-10T00:31:49.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:49 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' 2023-09-10T00:31:49.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:49 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' 2023-09-10T00:31:49.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:49 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:31:49.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:49 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:31:50.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:50 smithi044 ceph-mon[98676]: Found migration_current of "None". Setting to last migration. 2023-09-10T00:31:50.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:50 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:31:50.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:50 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-10T00:31:50.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:50 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-10T00:31:50.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:50 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' 2023-09-10T00:31:50.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:50 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' 2023-09-10T00:31:50.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:50 smithi044 ceph-mon[98676]: mgrmap e7: x(active, since 1.00526s) 2023-09-10T00:31:50.906 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2023-09-10T00:31:50.907 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-09-10T00:31:50.907 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-10T00:31:50.907 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2023-09-10T00:31:50.907 INFO:teuthology.orchestra.run.smithi044.stdout:mgr epoch 5 is available 2023-09-10T00:31:50.908 INFO:teuthology.orchestra.run.smithi044.stdout:Setting orchestrator backend to cephadm... 2023-09-10T00:31:51.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:51 smithi044 ceph-mon[98676]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-10T00:31:51.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:51 smithi044 ceph-mon[98676]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-10T00:31:51.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:51 smithi044 ceph-mon[98676]: [10/Sep/2023:00:31:51] ENGINE Bus STARTING 2023-09-10T00:31:51.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:51 smithi044 ceph-mon[98676]: [10/Sep/2023:00:31:51] ENGINE Serving on http://172.21.15.44:8765 2023-09-10T00:31:51.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:51 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:31:52.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:52 smithi044 ceph-mon[98676]: [10/Sep/2023:00:31:51] ENGINE Serving on https://172.21.15.44:7150 2023-09-10T00:31:52.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:52 smithi044 ceph-mon[98676]: [10/Sep/2023:00:31:51] ENGINE Bus STARTED 2023-09-10T00:31:52.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:52 smithi044 ceph-mon[98676]: mgrmap e8: x(active, since 2s) 2023-09-10T00:31:52.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:52 smithi044 ceph-mon[98676]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:31:52.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:52 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' 2023-09-10T00:31:52.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:52 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:31:53.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout value unchanged 2023-09-10T00:31:53.490 INFO:teuthology.orchestra.run.smithi044.stdout:Generating ssh key... 2023-09-10T00:31:53.713 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:53 smithi044 ceph-mon[98676]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:31:54.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-mon[98676]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:31:54.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-mon[98676]: Generating ssh key... 2023-09-10T00:31:54.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' 2023-09-10T00:31:54.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' 2023-09-10T00:31:54.806 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: Generating public/private rsa key pair. 2023-09-10T00:31:54.806 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: Your identification has been saved in /tmp/tmp1d3wt3q4/key. 2023-09-10T00:31:54.806 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: Your public key has been saved in /tmp/tmp1d3wt3q4/key.pub. 2023-09-10T00:31:54.806 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: The key fingerprint is: 2023-09-10T00:31:54.807 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: SHA256:2z851oGPQyXG1PYW6N13BlTqzOa1vHtHt8NCYEImKQA ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:31:54.807 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: The key's randomart image is: 2023-09-10T00:31:54.807 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: +---[RSA 3072]----+ 2023-09-10T00:31:54.807 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: | E... . oo..| 2023-09-10T00:31:54.807 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: | . o o ..+o | 2023-09-10T00:31:54.808 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: | . + o..ooo| 2023-09-10T00:31:54.808 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: | . o+=..B| 2023-09-10T00:31:54.808 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: | S o..+=o+| 2023-09-10T00:31:54.808 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: | o o+o.+| 2023-09-10T00:31:54.808 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: | . ...=o=o| 2023-09-10T00:31:54.809 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: | .B.o++| 2023-09-10T00:31:54.809 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: | ..+.o=| 2023-09-10T00:31:54.809 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:31:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: +----[SHA256]-----+ 2023-09-10T00:31:56.175 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3rJJ8/zaqc2Dz93raGCM86UCQNC6SoYprH86QlsXLxGwBGYsg727ZyLgr3qi5lHeKt+A+umzpAgyTc4MQVM7bVsWr6Xu8UTqYXg0HO1YXWKIoRL3OYsntH7O8NGvfpZFWStd2VAOTROQ5H1wkDyQKYaT/QPgK4U/yJoyy7VEZBOU8ifYIQHxhqOFHOaQP7LCeYgiTNkC7wBfHcvLNDSJ5rYOyFSW1IkNaIhCNDzsXP/6D00c+Re5e47FnFRNAjisaPSADVFE2Dy+xZxSOfnaBOdpHY9fZo/pu+8Ni+3yWEfp9U7mO59VaZCp3+2bveHruANO8SmojVXXq2lyJWZZj5CzTWVKLZAdlkiY/lQqXOLtSu2w41GolvD9eQKyIU9ETYEYC68Ke3ZNENmW+McijeuUxqzeGOV+o5+oCwpnz6UKa8BWcMgbQ4KfgK6JVxw4u8e3MeSHDH60jZdlP6/mEU6kPLiddgjBc/jGZAGVdEhGZNImdYi5aw/pn/ffjhZk= ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:31:56.176 INFO:teuthology.orchestra.run.smithi044.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-09-10T00:31:56.176 INFO:teuthology.orchestra.run.smithi044.stdout:Adding key to root@localhost authorized_keys... 2023-09-10T00:31:56.177 INFO:teuthology.orchestra.run.smithi044.stdout:Adding host smithi044... 2023-09-10T00:31:56.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:56 smithi044 ceph-mon[98676]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:31:57.589 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:57 smithi044 ceph-mon[98676]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi044", "addr": "172.21.15.44", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:31:58.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:31:58 smithi044 ceph-mon[98676]: Deploying cephadm binary to smithi044 2023-09-10T00:32:00.327 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout Added host 'smithi044' with addr '172.21.15.44' 2023-09-10T00:32:00.327 INFO:teuthology.orchestra.run.smithi044.stdout:Deploying unmanaged mon service... 2023-09-10T00:32:01.231 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:00 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' 2023-09-10T00:32:01.231 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:00 smithi044 ceph-mon[98676]: Added host smithi044 2023-09-10T00:32:01.231 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:00 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:32:01.879 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-09-10T00:32:01.879 INFO:teuthology.orchestra.run.smithi044.stdout:Deploying unmanaged mgr service... 2023-09-10T00:32:02.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:02 smithi044 ceph-mon[98676]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:32:02.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:02 smithi044 ceph-mon[98676]: Saving service mon spec with placement count:5 2023-09-10T00:32:02.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:02 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' 2023-09-10T00:32:02.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:02 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' 2023-09-10T00:32:03.322 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-09-10T00:32:03.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:03 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' 2023-09-10T00:32:03.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:03 smithi044 ceph-mon[98676]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:32:03.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:03 smithi044 ceph-mon[98676]: Saving service mgr spec with placement count:2 2023-09-10T00:32:03.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:03 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' 2023-09-10T00:32:05.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:05 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3098997588' entity='client.admin' 2023-09-10T00:32:05.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:05 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' 2023-09-10T00:32:06.010 INFO:teuthology.orchestra.run.smithi044.stdout:Enabling the dashboard module... 2023-09-10T00:32:06.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:06 smithi044 ceph-mon[98676]: from='mgr.14120 172.21.15.44:0/4016397552' entity='mgr.x' 2023-09-10T00:32:06.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:06 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2370331927' entity='client.admin' 2023-09-10T00:32:08.008 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:07 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2339114019' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-09-10T00:32:08.009 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:07 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: ignoring --setuser ceph since I am not root 2023-09-10T00:32:08.009 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:07 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: ignoring --setgroup ceph since I am not root 2023-09-10T00:32:08.009 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:07 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:07.908+0000 7f150dadf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-10T00:32:08.010 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:08 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:08.006+0000 7f150dadf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-10T00:32:08.305 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:08 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:08.116+0000 7f150dadf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-10T00:32:08.305 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:08 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:08.231+0000 7f150dadf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-10T00:32:08.700 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:08 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:08.450+0000 7f150dadf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-10T00:32:09.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:08 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2339114019' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-09-10T00:32:09.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:08 smithi044 ceph-mon[98676]: mgrmap e9: x(active, since 18s) 2023-09-10T00:32:09.055 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:08 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:08.937+0000 7f150dadf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-10T00:32:09.555 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:09 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:09.090+0000 7f150dadf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-10T00:32:09.555 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:09 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:09.200+0000 7f150dadf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-10T00:32:09.555 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:09 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:09.320+0000 7f150dadf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-10T00:32:09.800 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2023-09-10T00:32:09.800 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-09-10T00:32:09.801 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-10T00:32:09.801 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-10T00:32:09.801 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-10T00:32:09.801 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2023-09-10T00:32:09.801 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for the mgr to restart... 2023-09-10T00:32:09.802 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mgr epoch 9... 2023-09-10T00:32:10.005 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:09 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2616339792' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-10T00:32:10.006 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:09 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:09.842+0000 7f150dadf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-10T00:32:11.055 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:10 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:10.637+0000 7f150dadf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-10T00:32:11.055 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:10 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:10.737+0000 7f150dadf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-10T00:32:12.045 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:11 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:11.709+0000 7f150dadf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-10T00:32:12.046 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:11 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:11.820+0000 7f150dadf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-10T00:32:12.304 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:12 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:12.043+0000 7f150dadf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-10T00:32:12.654 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:12 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:12.398+0000 7f150dadf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-10T00:32:13.054 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:12 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:12.652+0000 7f150dadf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-10T00:32:13.055 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:12 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:12.835+0000 7f150dadf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-10T00:32:13.555 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:13 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:13.168+0000 7f150dadf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-10T00:32:14.304 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:13 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:13.996+0000 7f150dadf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-10T00:32:14.305 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:14 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:14.158+0000 7f150dadf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-10T00:32:14.804 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:14 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:14.404+0000 7f150dadf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-10T00:32:14.805 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:14 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:14.542+0000 7f150dadf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-10T00:32:15.304 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:14 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:14.817+0000 7f150dadf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-10T00:32:16.306 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:32:15 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:32:15.898+0000 7f150dadf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-10T00:32:16.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:16 smithi044 ceph-mon[98676]: Active manager daemon x restarted 2023-09-10T00:32:16.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:16 smithi044 ceph-mon[98676]: Activating manager daemon x 2023-09-10T00:32:16.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:16 smithi044 ceph-mon[98676]: osdmap e3: 0 total, 0 up, 0 in 2023-09-10T00:32:16.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:16 smithi044 ceph-mon[98676]: mgrmap e10: x(active, starting, since 0.00314366s) 2023-09-10T00:32:16.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:32:16.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-10T00:32:16.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-10T00:32:16.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-10T00:32:16.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-10T00:32:16.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:16 smithi044 ceph-mon[98676]: Manager daemon x is now available 2023-09-10T00:32:16.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:32:16.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:32:16.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-10T00:32:16.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-10T00:32:17.256 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2023-09-10T00:32:17.256 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-09-10T00:32:17.257 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-10T00:32:17.257 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2023-09-10T00:32:17.257 INFO:teuthology.orchestra.run.smithi044.stdout:mgr epoch 9 is available 2023-09-10T00:32:17.257 INFO:teuthology.orchestra.run.smithi044.stdout:Generating a dashboard self-signed certificate... 2023-09-10T00:32:18.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:17 smithi044 ceph-mon[98676]: [10/Sep/2023:00:32:16] ENGINE Bus STARTING 2023-09-10T00:32:18.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:17 smithi044 ceph-mon[98676]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-10T00:32:18.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:17 smithi044 ceph-mon[98676]: mgrmap e11: x(active, since 1.00561s) 2023-09-10T00:32:18.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:17 smithi044 ceph-mon[98676]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-10T00:32:18.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:17 smithi044 ceph-mon[98676]: [10/Sep/2023:00:32:17] ENGINE Serving on https://172.21.15.44:7150 2023-09-10T00:32:18.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:17 smithi044 ceph-mon[98676]: [10/Sep/2023:00:32:17] ENGINE Serving on http://172.21.15.44:8765 2023-09-10T00:32:18.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:17 smithi044 ceph-mon[98676]: [10/Sep/2023:00:32:17] ENGINE Bus STARTED 2023-09-10T00:32:18.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:18.949 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-09-10T00:32:18.949 INFO:teuthology.orchestra.run.smithi044.stdout:Creating initial admin user... 2023-09-10T00:32:19.940 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:19 smithi044 ceph-mon[98676]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:32:19.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:19 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:19.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:19 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:19.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:19 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:19.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:19 smithi044 ceph-mon[98676]: mgrmap e12: x(active, since 2s) 2023-09-10T00:32:20.592 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$PgzLWgX6dhw069/7iJ0AYubMz5rF7g0zeAeDk4sVfWcEBJvX9uFb6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1694305940, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-09-10T00:32:20.592 INFO:teuthology.orchestra.run.smithi044.stdout:Fetching dashboard port number... 2023-09-10T00:32:21.477 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:21 smithi044 ceph-mon[98676]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:32:21.478 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:21.834 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 8443 2023-09-10T00:32:21.843 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-10T00:32:21.844 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout disabled 2023-09-10T00:32:21.851 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-10T00:32:21.851 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2023-09-10T00:32:21.851 INFO:teuthology.orchestra.run.smithi044.stdout:firewalld.service is not enabled 2023-09-10T00:32:21.852 INFO:teuthology.orchestra.run.smithi044.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-09-10T00:32:21.852 INFO:teuthology.orchestra.run.smithi044.stdout:Ceph Dashboard is now available at: 2023-09-10T00:32:21.853 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:32:21.853 INFO:teuthology.orchestra.run.smithi044.stdout: URL: https://smithi044:8443/ 2023-09-10T00:32:21.853 INFO:teuthology.orchestra.run.smithi044.stdout: User: admin 2023-09-10T00:32:21.853 INFO:teuthology.orchestra.run.smithi044.stdout: Password: k63ynugp0r 2023-09-10T00:32:21.854 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:32:21.854 INFO:teuthology.orchestra.run.smithi044.stdout:Saving cluster configuration to /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config directory 2023-09-10T00:32:21.854 INFO:teuthology.orchestra.run.smithi044.stdout:Enabling autotune for osd_memory_target 2023-09-10T00:32:22.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:22 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3559821193' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-09-10T00:32:23.342 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:23 smithi044 ceph-mon[98676]: mgrmap e13: x(active, since 6s) 2023-09-10T00:32:24.539 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-09-10T00:32:24.540 INFO:teuthology.orchestra.run.smithi044.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-09-10T00:32:24.540 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:32:24.540 INFO:teuthology.orchestra.run.smithi044.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-09-10T00:32:24.540 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:32:24.540 INFO:teuthology.orchestra.run.smithi044.stdout:Or, if you are only running a single cluster on this host: 2023-09-10T00:32:24.541 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:32:24.541 INFO:teuthology.orchestra.run.smithi044.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-09-10T00:32:24.541 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:32:24.541 INFO:teuthology.orchestra.run.smithi044.stdout:Please consider enabling telemetry to help improve Ceph: 2023-09-10T00:32:24.541 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:32:24.541 INFO:teuthology.orchestra.run.smithi044.stdout: ceph telemetry on 2023-09-10T00:32:24.542 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:32:24.542 INFO:teuthology.orchestra.run.smithi044.stdout:For more information see: 2023-09-10T00:32:24.542 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:32:24.542 INFO:teuthology.orchestra.run.smithi044.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-09-10T00:32:24.542 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:32:24.542 INFO:teuthology.orchestra.run.smithi044.stdout:Bootstrap complete. 2023-09-10T00:32:24.575 INFO:tasks.cephadm:Fetching config... 2023-09-10T00:32:24.575 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:32:24.576 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-09-10T00:32:24.594 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-09-10T00:32:24.594 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:32:24.594 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-09-10T00:32:24.651 INFO:tasks.cephadm:Fetching mon keyring... 2023-09-10T00:32:24.652 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:32:24.652 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/keyring of=/dev/stdout 2023-09-10T00:32:24.721 INFO:tasks.cephadm:Fetching pub ssh key... 2023-09-10T00:32:24.721 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:32:24.722 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-09-10T00:32:24.778 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-09-10T00:32:24.778 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3rJJ8/zaqc2Dz93raGCM86UCQNC6SoYprH86QlsXLxGwBGYsg727ZyLgr3qi5lHeKt+A+umzpAgyTc4MQVM7bVsWr6Xu8UTqYXg0HO1YXWKIoRL3OYsntH7O8NGvfpZFWStd2VAOTROQ5H1wkDyQKYaT/QPgK4U/yJoyy7VEZBOU8ifYIQHxhqOFHOaQP7LCeYgiTNkC7wBfHcvLNDSJ5rYOyFSW1IkNaIhCNDzsXP/6D00c+Re5e47FnFRNAjisaPSADVFE2Dy+xZxSOfnaBOdpHY9fZo/pu+8Ni+3yWEfp9U7mO59VaZCp3+2bveHruANO8SmojVXXq2lyJWZZj5CzTWVKLZAdlkiY/lQqXOLtSu2w41GolvD9eQKyIU9ETYEYC68Ke3ZNENmW+McijeuUxqzeGOV+o5+oCwpnz6UKa8BWcMgbQ4KfgK6JVxw4u8e3MeSHDH60jZdlP6/mEU6kPLiddgjBc/jGZAGVdEhGZNImdYi5aw/pn/ffjhZk= ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-10T00:32:24.861 INFO:teuthology.orchestra.run.smithi044.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3rJJ8/zaqc2Dz93raGCM86UCQNC6SoYprH86QlsXLxGwBGYsg727ZyLgr3qi5lHeKt+A+umzpAgyTc4MQVM7bVsWr6Xu8UTqYXg0HO1YXWKIoRL3OYsntH7O8NGvfpZFWStd2VAOTROQ5H1wkDyQKYaT/QPgK4U/yJoyy7VEZBOU8ifYIQHxhqOFHOaQP7LCeYgiTNkC7wBfHcvLNDSJ5rYOyFSW1IkNaIhCNDzsXP/6D00c+Re5e47FnFRNAjisaPSADVFE2Dy+xZxSOfnaBOdpHY9fZo/pu+8Ni+3yWEfp9U7mO59VaZCp3+2bveHruANO8SmojVXXq2lyJWZZj5CzTWVKLZAdlkiY/lQqXOLtSu2w41GolvD9eQKyIU9ETYEYC68Ke3ZNENmW+McijeuUxqzeGOV+o5+oCwpnz6UKa8BWcMgbQ4KfgK6JVxw4u8e3MeSHDH60jZdlP6/mEU6kPLiddgjBc/jGZAGVdEhGZNImdYi5aw/pn/ffjhZk= ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:32:24.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3rJJ8/zaqc2Dz93raGCM86UCQNC6SoYprH86QlsXLxGwBGYsg727ZyLgr3qi5lHeKt+A+umzpAgyTc4MQVM7bVsWr6Xu8UTqYXg0HO1YXWKIoRL3OYsntH7O8NGvfpZFWStd2VAOTROQ5H1wkDyQKYaT/QPgK4U/yJoyy7VEZBOU8ifYIQHxhqOFHOaQP7LCeYgiTNkC7wBfHcvLNDSJ5rYOyFSW1IkNaIhCNDzsXP/6D00c+Re5e47FnFRNAjisaPSADVFE2Dy+xZxSOfnaBOdpHY9fZo/pu+8Ni+3yWEfp9U7mO59VaZCp3+2bveHruANO8SmojVXXq2lyJWZZj5CzTWVKLZAdlkiY/lQqXOLtSu2w41GolvD9eQKyIU9ETYEYC68Ke3ZNENmW+McijeuUxqzeGOV+o5+oCwpnz6UKa8BWcMgbQ4KfgK6JVxw4u8e3MeSHDH60jZdlP6/mEU6kPLiddgjBc/jGZAGVdEhGZNImdYi5aw/pn/ffjhZk= ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-10T00:32:24.921 INFO:teuthology.orchestra.run.smithi099.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3rJJ8/zaqc2Dz93raGCM86UCQNC6SoYprH86QlsXLxGwBGYsg727ZyLgr3qi5lHeKt+A+umzpAgyTc4MQVM7bVsWr6Xu8UTqYXg0HO1YXWKIoRL3OYsntH7O8NGvfpZFWStd2VAOTROQ5H1wkDyQKYaT/QPgK4U/yJoyy7VEZBOU8ifYIQHxhqOFHOaQP7LCeYgiTNkC7wBfHcvLNDSJ5rYOyFSW1IkNaIhCNDzsXP/6D00c+Re5e47FnFRNAjisaPSADVFE2Dy+xZxSOfnaBOdpHY9fZo/pu+8Ni+3yWEfp9U7mO59VaZCp3+2bveHruANO8SmojVXXq2lyJWZZj5CzTWVKLZAdlkiY/lQqXOLtSu2w41GolvD9eQKyIU9ETYEYC68Ke3ZNENmW+McijeuUxqzeGOV+o5+oCwpnz6UKa8BWcMgbQ4KfgK6JVxw4u8e3MeSHDH60jZdlP6/mEU6kPLiddgjBc/jGZAGVdEhGZNImdYi5aw/pn/ffjhZk= ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:32:24.936 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3rJJ8/zaqc2Dz93raGCM86UCQNC6SoYprH86QlsXLxGwBGYsg727ZyLgr3qi5lHeKt+A+umzpAgyTc4MQVM7bVsWr6Xu8UTqYXg0HO1YXWKIoRL3OYsntH7O8NGvfpZFWStd2VAOTROQ5H1wkDyQKYaT/QPgK4U/yJoyy7VEZBOU8ifYIQHxhqOFHOaQP7LCeYgiTNkC7wBfHcvLNDSJ5rYOyFSW1IkNaIhCNDzsXP/6D00c+Re5e47FnFRNAjisaPSADVFE2Dy+xZxSOfnaBOdpHY9fZo/pu+8Ni+3yWEfp9U7mO59VaZCp3+2bveHruANO8SmojVXXq2lyJWZZj5CzTWVKLZAdlkiY/lQqXOLtSu2w41GolvD9eQKyIU9ETYEYC68Ke3ZNENmW+McijeuUxqzeGOV+o5+oCwpnz6UKa8BWcMgbQ4KfgK6JVxw4u8e3MeSHDH60jZdlP6/mEU6kPLiddgjBc/jGZAGVdEhGZNImdYi5aw/pn/ffjhZk= ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-10T00:32:24.983 INFO:teuthology.orchestra.run.smithi131.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3rJJ8/zaqc2Dz93raGCM86UCQNC6SoYprH86QlsXLxGwBGYsg727ZyLgr3qi5lHeKt+A+umzpAgyTc4MQVM7bVsWr6Xu8UTqYXg0HO1YXWKIoRL3OYsntH7O8NGvfpZFWStd2VAOTROQ5H1wkDyQKYaT/QPgK4U/yJoyy7VEZBOU8ifYIQHxhqOFHOaQP7LCeYgiTNkC7wBfHcvLNDSJ5rYOyFSW1IkNaIhCNDzsXP/6D00c+Re5e47FnFRNAjisaPSADVFE2Dy+xZxSOfnaBOdpHY9fZo/pu+8Ni+3yWEfp9U7mO59VaZCp3+2bveHruANO8SmojVXXq2lyJWZZj5CzTWVKLZAdlkiY/lQqXOLtSu2w41GolvD9eQKyIU9ETYEYC68Ke3ZNENmW+McijeuUxqzeGOV+o5+oCwpnz6UKa8BWcMgbQ4KfgK6JVxw4u8e3MeSHDH60jZdlP6/mEU6kPLiddgjBc/jGZAGVdEhGZNImdYi5aw/pn/ffjhZk= ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:32:24.995 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-09-10T00:32:25.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:25 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3353344966' entity='client.admin' 2023-09-10T00:32:25.563 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:32:27.535 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-09-10T00:32:27.536 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-09-10T00:32:27.962 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:27.962 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:27.962 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:32:27.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:27.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:32:27.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:27.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:27 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/4225630993' entity='client.admin' 2023-09-10T00:32:27.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:32:27.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:32:30.316 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi099 2023-09-10T00:32:30.317 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-10T00:32:30.317 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/etc/ceph/ceph.conf 2023-09-10T00:32:30.338 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-10T00:32:30.338 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:32:30.397 INFO:tasks.cephadm:Adding host smithi099 to orchestrator... 2023-09-10T00:32:30.398 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch host add smithi099 2023-09-10T00:32:30.963 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:32:31.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:30 smithi044 ceph-mon[98676]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:32:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:30 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:30 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:32:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:30 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:32:31.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:30 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:32:31.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:30 smithi044 ceph-mon[98676]: Updating smithi044:/etc/ceph/ceph.conf 2023-09-10T00:32:31.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:30 smithi044 ceph-mon[98676]: Updating smithi044:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:32:31.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:30 smithi044 ceph-mon[98676]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:32:31.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:30 smithi044 ceph-mon[98676]: Updating smithi044:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-10T00:32:32.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:31 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:32.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:31 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:32.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:31 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:33.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:32 smithi044 ceph-mon[98676]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi099", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:32:34.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:33 smithi044 ceph-mon[98676]: Deploying cephadm binary to smithi099 2023-09-10T00:32:35.236 INFO:teuthology.orchestra.run.smithi044.stdout:Added host 'smithi099' with addr '172.21.15.99' 2023-09-10T00:32:35.927 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch host ls --format=json 2023-09-10T00:32:36.498 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:32:36.554 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:36 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:36.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:36 smithi044 ceph-mon[98676]: Added host smithi099 2023-09-10T00:32:36.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:36 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:32:36.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:36 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:37.554 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:37 smithi044 ceph-mon[98676]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:32:37.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:37.917 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:32:37.917 INFO:teuthology.orchestra.run.smithi044.stdout:[{"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}, {"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}] 2023-09-10T00:32:38.258 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi131 2023-09-10T00:32:38.258 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-10T00:32:38.259 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/etc/ceph/ceph.conf 2023-09-10T00:32:38.279 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-10T00:32:38.279 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:32:38.338 INFO:tasks.cephadm:Adding host smithi131 to orchestrator... 2023-09-10T00:32:38.339 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch host add smithi131 2023-09-10T00:32:38.898 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:32:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:38 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:39.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:39 smithi044 ceph-mon[98676]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:32:39.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:39 smithi044 ceph-mon[98676]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:32:41.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:41 smithi044 ceph-mon[98676]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:32:41.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:41 smithi044 ceph-mon[98676]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi131", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:32:41.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:41.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:41.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:41.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:41.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:32:41.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:32:41.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:32:41.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:41 smithi044 ceph-mon[98676]: Updating smithi099:/etc/ceph/ceph.conf 2023-09-10T00:32:42.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:42 smithi044 ceph-mon[98676]: Updating smithi099:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:32:42.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:42 smithi044 ceph-mon[98676]: Deploying cephadm binary to smithi131 2023-09-10T00:32:42.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:42 smithi044 ceph-mon[98676]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:32:42.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:42 smithi044 ceph-mon[98676]: Updating smithi099:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-10T00:32:43.195 INFO:teuthology.orchestra.run.smithi044.stdout:Added host 'smithi131' with addr '172.21.15.131' 2023-09-10T00:32:43.554 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch host ls --format=json 2023-09-10T00:32:43.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:43 smithi044 ceph-mon[98676]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:32:43.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:43.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:43.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:43.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:43.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:32:44.121 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:32:45.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:44 smithi044 ceph-mon[98676]: Added host smithi131 2023-09-10T00:32:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:44 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:44 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:45.543 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:32:45.543 INFO:teuthology.orchestra.run.smithi044.stdout:[{"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}, {"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}, {"addr": "172.21.15.131", "hostname": "smithi131", "labels": [], "status": ""}] 2023-09-10T00:32:45.835 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:45 smithi044 ceph-mon[98676]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:32:45.835 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:45.868 INFO:tasks.cephadm:Setting crush tunables to default 2023-09-10T00:32:45.869 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd crush tunables default 2023-09-10T00:32:46.430 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:32:46.910 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:46 smithi044 ceph-mon[98676]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:32:47.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:47 smithi044 ceph-mon[98676]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:32:48.587 INFO:teuthology.orchestra.run.smithi044.stderr:adjusted tunables profile to default 2023-09-10T00:32:48.978 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:48 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3786149350' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-10T00:32:48.979 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:48.979 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:48.980 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:48.980 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:48.980 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:32:48.980 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:32:48.981 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:32:48.983 INFO:tasks.cephadm:Adding mon.a on smithi044 2023-09-10T00:32:48.983 INFO:tasks.cephadm:Adding mon.b on smithi099 2023-09-10T00:32:48.984 INFO:tasks.cephadm:Adding mon.c on smithi131 2023-09-10T00:32:48.984 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch apply mon '3;smithi044:172.21.15.44=a;smithi099:172.21.15.99=b;smithi131:172.21.15.131=c' 2023-09-10T00:32:49.573 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:32:50.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:49 smithi044 ceph-mon[98676]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:32:50.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:49 smithi044 ceph-mon[98676]: Updating smithi131:/etc/ceph/ceph.conf 2023-09-10T00:32:50.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:49 smithi044 ceph-mon[98676]: Updating smithi131:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:32:50.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:49 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3786149350' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-10T00:32:50.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:49 smithi044 ceph-mon[98676]: osdmap e4: 0 total, 0 up, 0 in 2023-09-10T00:32:50.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:50.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:50.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:50 smithi044 ceph-mon[98676]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:32:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:50 smithi044 ceph-mon[98676]: Updating smithi131:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-10T00:32:51.145 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled mon update... 2023-09-10T00:32:51.651 DEBUG:teuthology.orchestra.run.smithi099:mon.b> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mon.b.service 2023-09-10T00:32:51.654 DEBUG:teuthology.orchestra.run.smithi131:mon.c> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mon.c.service 2023-09-10T00:32:51.657 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-10T00:32:51.657 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph mon dump -f json 2023-09-10T00:32:51.685 INFO:journalctl@ceph.mon.c.smithi131.stdout:-- Logs begin at Sun 2023-09-10 00:17:05 UTC. -- 2023-09-10T00:32:51.689 INFO:journalctl@ceph.mon.b.smithi099.stdout:-- Logs begin at Sun 2023-09-10 00:17:22 UTC. -- 2023-09-10T00:32:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:51 smithi044 ceph-mon[98676]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:32:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:51 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:51 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:32:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:51 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:32:52.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:51 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:32:52.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:51 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:32:52.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:51 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-10T00:32:52.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:51 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:32:52.588 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:32:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:52 smithi044 ceph-mon[98676]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi044:172.21.15.44=a;smithi099:172.21.15.99=b;smithi131:172.21.15.131=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:32:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:52 smithi044 ceph-mon[98676]: Saving service mon spec with placement smithi044:172.21.15.44=a;smithi099:172.21.15.99=b;smithi131:172.21.15.131=c;count:3 2023-09-10T00:32:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:52 smithi044 ceph-mon[98676]: Deploying daemon mon.c on smithi131 2023-09-10T00:32:54.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:53 smithi044 ceph-mon[98676]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:32:54.446 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-10T00:32:54.447 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":1,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","modified":"2023-09-10T00:31:13.763341Z","created":"2023-09-10T00:31:13.763341Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-09-10T00:32:54.447 INFO:teuthology.orchestra.run.smithi131.stderr:dumped monmap epoch 1 2023-09-10T00:32:55.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:32:54 smithi044 ceph-mon[98676]: from='client.? 172.21.15.131:0/1387211107' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-10T00:32:55.192 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:32:55 smithi131 bash[103672]: 47c976327db9116d54235a80de769c42bca0b93ca08074b935154d733d900903 2023-09-10T00:32:55.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:32:55 smithi131 systemd[1]: Started Ceph mon.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:32:55.944 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-10T00:32:55.945 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph mon dump -f json 2023-09-10T00:32:56.509 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-10T00:33:00.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:33:00.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: mon.a calling monitor election 2023-09-10T00:33:00.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:00.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:00.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:00.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:00.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: mon.c calling monitor election 2023-09-10T00:33:00.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:00.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:00.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:00.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:00.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-10T00:33:00.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: monmap e2: 2 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],c=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0]} removed_ranks: {} 2023-09-10T00:33:00.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: fsmap 2023-09-10T00:33:00.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: osdmap e4: 0 total, 0 up, 0 in 2023-09-10T00:33:00.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: mgrmap e13: x(active, since 44s) 2023-09-10T00:33:00.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: overall HEALTH_OK 2023-09-10T00:33:00.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:00.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:00.550 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:00.550 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-10T00:33:00.550 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:00 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:00.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:33:00.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: mon.a calling monitor election 2023-09-10T00:33:00.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:00.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:00.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:00.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:00.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: mon.c calling monitor election 2023-09-10T00:33:00.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:00.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:00.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:00.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:00.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-10T00:33:00.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: monmap e2: 2 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],c=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0]} removed_ranks: {} 2023-09-10T00:33:00.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: fsmap 2023-09-10T00:33:00.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: osdmap e4: 0 total, 0 up, 0 in 2023-09-10T00:33:00.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: mgrmap e13: x(active, since 44s) 2023-09-10T00:33:00.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: overall HEALTH_OK 2023-09-10T00:33:00.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:00.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:00.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:00.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-10T00:33:00.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:00 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:01.122 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-10T00:33:01.122 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":2,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","modified":"2023-09-10T00:32:55.121271Z","created":"2023-09-10T00:31:13.763341Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-10T00:33:01.124 INFO:teuthology.orchestra.run.smithi131.stderr:dumped monmap epoch 2 2023-09-10T00:33:01.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:01 smithi131 ceph-mon[103713]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:01.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:01 smithi131 ceph-mon[103713]: Deploying daemon mon.b on smithi099 2023-09-10T00:33:01.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:01 smithi131 ceph-mon[103713]: from='client.? 172.21.15.131:0/3820096658' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-10T00:33:01.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:01 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:01.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:01 smithi044 ceph-mon[98676]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:01.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:01 smithi044 ceph-mon[98676]: Deploying daemon mon.b on smithi099 2023-09-10T00:33:01.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:01 smithi044 ceph-mon[98676]: from='client.? 172.21.15.131:0/3820096658' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-10T00:33:01.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:01 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:02.560 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-10T00:33:02.560 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph mon dump -f json 2023-09-10T00:33:03.030 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:02 smithi099 systemd[1]: Starting Ceph mon.b for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:33:03.118 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-10T00:33:03.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 podman[102652]: 2023-09-10T00:33:03.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: set uid:gid to 167:167 (ceph:ceph) 2023-09-10T00:33:03.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev), process ceph-mon, pid 7 2023-09-10T00:33:03.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 bash[102652]: a4aadc59315726a0fff1029add7d2a49d6cb28a7cd72e72f99cc5ece86c49778 2023-09-10T00:33:03.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: pidfile_write: ignore empty --pid-file 2023-09-10T00:33:03.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: load: jerasure load: lrc 2023-09-10T00:33:03.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: RocksDB version: 7.9.2 2023-09-10T00:33:03.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Git sha 0 2023-09-10T00:33:03.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Compile date 2023-09-09 15:16:58 2023-09-10T00:33:03.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: DB SUMMARY 2023-09-10T00:33:03.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: DB Session ID: I3HJ3RKVJNP1S6A6NITD 2023-09-10T00:33:03.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: CURRENT file: CURRENT 2023-09-10T00:33:03.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: IDENTITY file: IDENTITY 2023-09-10T00:33:03.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-09-10T00:33:03.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-09-10T00:33:03.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-09-10T00:33:03.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.error_if_exists: 0 2023-09-10T00:33:03.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.create_if_missing: 0 2023-09-10T00:33:03.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.paranoid_checks: 1 2023-09-10T00:33:03.361 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-10T00:33:03.361 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-10T00:33:03.361 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-10T00:33:03.361 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.env: 0x55cf81e04da0 2023-09-10T00:33:03.362 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.fs: PosixFileSystem 2023-09-10T00:33:03.362 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.info_log: 0x55cf837f54e0 2023-09-10T00:33:03.362 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_file_opening_threads: 16 2023-09-10T00:33:03.362 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.statistics: (nil) 2023-09-10T00:33:03.362 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.use_fsync: 0 2023-09-10T00:33:03.363 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_log_file_size: 0 2023-09-10T00:33:03.363 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-10T00:33:03.363 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-10T00:33:03.363 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.keep_log_file_num: 1000 2023-09-10T00:33:03.363 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.recycle_log_file_num: 0 2023-09-10T00:33:03.364 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.allow_fallocate: 1 2023-09-10T00:33:03.364 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.allow_mmap_reads: 0 2023-09-10T00:33:03.364 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.allow_mmap_writes: 0 2023-09-10T00:33:03.364 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.use_direct_reads: 0 2023-09-10T00:33:03.364 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-10T00:33:03.364 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.create_missing_column_families: 0 2023-09-10T00:33:03.365 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.db_log_dir: 2023-09-10T00:33:03.365 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.wal_dir: 2023-09-10T00:33:03.365 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-10T00:33:03.365 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-10T00:33:03.365 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-10T00:33:03.366 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-10T00:33:03.366 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-10T00:33:03.366 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-10T00:33:03.366 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.advise_random_on_open: 1 2023-09-10T00:33:03.366 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.db_write_buffer_size: 0 2023-09-10T00:33:03.367 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.write_buffer_manager: 0x55cf837af4a0 2023-09-10T00:33:03.367 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-10T00:33:03.367 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-10T00:33:03.367 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-10T00:33:03.367 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.rate_limiter: (nil) 2023-09-10T00:33:03.367 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-10T00:33:03.368 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.wal_recovery_mode: 2 2023-09-10T00:33:03.368 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.enable_thread_tracking: 0 2023-09-10T00:33:03.368 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.enable_pipelined_write: 0 2023-09-10T00:33:03.368 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.unordered_write: 0 2023-09-10T00:33:03.368 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-10T00:33:03.369 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-10T00:33:03.369 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-10T00:33:03.369 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-10T00:33:03.369 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.row_cache: None 2023-09-10T00:33:03.369 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.wal_filter: None 2023-09-10T00:33:03.370 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-10T00:33:03.370 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.allow_ingest_behind: 0 2023-09-10T00:33:03.370 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.two_write_queues: 0 2023-09-10T00:33:03.370 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.manual_wal_flush: 0 2023-09-10T00:33:03.370 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.wal_compression: 0 2023-09-10T00:33:03.370 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.atomic_flush: 0 2023-09-10T00:33:03.371 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-10T00:33:03.371 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-10T00:33:03.371 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-10T00:33:03.371 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.log_readahead_size: 0 2023-09-10T00:33:03.372 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-10T00:33:03.372 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.best_efforts_recovery: 0 2023-09-10T00:33:03.372 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-10T00:33:03.372 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-10T00:33:03.372 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.allow_data_in_errors: 0 2023-09-10T00:33:03.373 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.db_host_id: __hostname__ 2023-09-10T00:33:03.373 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-10T00:33:03.373 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_background_jobs: 2 2023-09-10T00:33:03.373 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_background_compactions: -1 2023-09-10T00:33:03.373 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_subcompactions: 1 2023-09-10T00:33:03.374 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-10T00:33:03.374 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-10T00:33:03.374 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-10T00:33:03.374 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_total_wal_size: 0 2023-09-10T00:33:03.374 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-10T00:33:03.375 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-10T00:33:03.375 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-10T00:33:03.375 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-10T00:33:03.375 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_open_files: -1 2023-09-10T00:33:03.375 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.bytes_per_sync: 0 2023-09-10T00:33:03.375 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-10T00:33:03.376 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-10T00:33:03.376 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compaction_readahead_size: 0 2023-09-10T00:33:03.376 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_background_flushes: -1 2023-09-10T00:33:03.376 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Compression algorithms supported: 2023-09-10T00:33:03.376 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-10T00:33:03.377 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: kZSTD supported: 0 2023-09-10T00:33:03.377 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: kXpressCompression supported: 0 2023-09-10T00:33:03.377 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: kLZ4HCCompression supported: 1 2023-09-10T00:33:03.377 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: kZlibCompression supported: 1 2023-09-10T00:33:03.377 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: kSnappyCompression supported: 1 2023-09-10T00:33:03.377 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: kLZ4Compression supported: 1 2023-09-10T00:33:03.378 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: kBZip2Compression supported: 0 2023-09-10T00:33:03.378 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-10T00:33:03.378 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-10T00:33:03.378 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-09-10T00:33:03.378 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-10T00:33:03.379 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-10T00:33:03.379 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.merge_operator: 2023-09-10T00:33:03.379 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compaction_filter: None 2023-09-10T00:33:03.379 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compaction_filter_factory: None 2023-09-10T00:33:03.379 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.sst_partitioner_factory: None 2023-09-10T00:33:03.380 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-10T00:33:03.380 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-10T00:33:03.380 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cf837f55e0) 2023-09-10T00:33:03.380 INFO:journalctl@ceph.mon.b.smithi099.stdout: cache_index_and_filter_blocks: 1 2023-09-10T00:33:03.380 INFO:journalctl@ceph.mon.b.smithi099.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-10T00:33:03.380 INFO:journalctl@ceph.mon.b.smithi099.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-10T00:33:03.381 INFO:journalctl@ceph.mon.b.smithi099.stdout: pin_top_level_index_and_filter: 1 2023-09-10T00:33:03.381 INFO:journalctl@ceph.mon.b.smithi099.stdout: index_type: 0 2023-09-10T00:33:03.381 INFO:journalctl@ceph.mon.b.smithi099.stdout: data_block_index_type: 0 2023-09-10T00:33:03.381 INFO:journalctl@ceph.mon.b.smithi099.stdout: index_shortening: 1 2023-09-10T00:33:03.381 INFO:journalctl@ceph.mon.b.smithi099.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-10T00:33:03.382 INFO:journalctl@ceph.mon.b.smithi099.stdout: checksum: 4 2023-09-10T00:33:03.382 INFO:journalctl@ceph.mon.b.smithi099.stdout: no_block_cache: 0 2023-09-10T00:33:03.382 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_cache: 0x55cf8384d1f0 2023-09-10T00:33:03.383 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_cache_name: BinnedLRUCache 2023-09-10T00:33:03.383 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_cache_options: 2023-09-10T00:33:03.383 INFO:journalctl@ceph.mon.b.smithi099.stdout: capacity : 536870912 2023-09-10T00:33:03.383 INFO:journalctl@ceph.mon.b.smithi099.stdout: num_shard_bits : 4 2023-09-10T00:33:03.383 INFO:journalctl@ceph.mon.b.smithi099.stdout: strict_capacity_limit : 0 2023-09-10T00:33:03.384 INFO:journalctl@ceph.mon.b.smithi099.stdout: high_pri_pool_ratio: 0.000 2023-09-10T00:33:03.384 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_cache_compressed: (nil) 2023-09-10T00:33:03.384 INFO:journalctl@ceph.mon.b.smithi099.stdout: persistent_cache: (nil) 2023-09-10T00:33:03.384 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_size: 4096 2023-09-10T00:33:03.384 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_size_deviation: 10 2023-09-10T00:33:03.385 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_restart_interval: 16 2023-09-10T00:33:03.385 INFO:journalctl@ceph.mon.b.smithi099.stdout: index_block_restart_interval: 1 2023-09-10T00:33:03.385 INFO:journalctl@ceph.mon.b.smithi099.stdout: metadata_block_size: 4096 2023-09-10T00:33:03.385 INFO:journalctl@ceph.mon.b.smithi099.stdout: partition_filters: 0 2023-09-10T00:33:03.385 INFO:journalctl@ceph.mon.b.smithi099.stdout: use_delta_encoding: 1 2023-09-10T00:33:03.385 INFO:journalctl@ceph.mon.b.smithi099.stdout: filter_policy: bloomfilter 2023-09-10T00:33:03.386 INFO:journalctl@ceph.mon.b.smithi099.stdout: whole_key_filtering: 1 2023-09-10T00:33:03.386 INFO:journalctl@ceph.mon.b.smithi099.stdout: verify_compression: 0 2023-09-10T00:33:03.386 INFO:journalctl@ceph.mon.b.smithi099.stdout: read_amp_bytes_per_bit: 0 2023-09-10T00:33:03.386 INFO:journalctl@ceph.mon.b.smithi099.stdout: format_version: 5 2023-09-10T00:33:03.386 INFO:journalctl@ceph.mon.b.smithi099.stdout: enable_index_compression: 1 2023-09-10T00:33:03.387 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_align: 0 2023-09-10T00:33:03.387 INFO:journalctl@ceph.mon.b.smithi099.stdout: max_auto_readahead_size: 262144 2023-09-10T00:33:03.387 INFO:journalctl@ceph.mon.b.smithi099.stdout: prepopulate_block_cache: 0 2023-09-10T00:33:03.387 INFO:journalctl@ceph.mon.b.smithi099.stdout: initial_auto_readahead_size: 8192 2023-09-10T00:33:03.387 INFO:journalctl@ceph.mon.b.smithi099.stdout: num_file_reads_for_auto_readahead: 2 2023-09-10T00:33:03.387 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.write_buffer_size: 33554432 2023-09-10T00:33:03.388 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_write_buffer_number: 2 2023-09-10T00:33:03.388 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compression: NoCompression 2023-09-10T00:33:03.388 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.bottommost_compression: Disabled 2023-09-10T00:33:03.388 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.prefix_extractor: nullptr 2023-09-10T00:33:03.388 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-10T00:33:03.388 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.num_levels: 7 2023-09-10T00:33:03.389 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-10T00:33:03.389 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-10T00:33:03.389 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-10T00:33:03.389 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-10T00:33:03.389 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-10T00:33:03.390 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-10T00:33:03.390 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-10T00:33:03.390 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-10T00:33:03.390 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-10T00:33:03.390 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-10T00:33:03.391 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-10T00:33:03.391 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-10T00:33:03.391 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-10T00:33:03.391 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compression_opts.level: 32767 2023-09-10T00:33:03.391 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compression_opts.strategy: 0 2023-09-10T00:33:03.391 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-10T00:33:03.392 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-10T00:33:03.392 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-10T00:33:03.392 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-10T00:33:03.392 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compression_opts.enabled: false 2023-09-10T00:33:03.392 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-10T00:33:03.393 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-10T00:33:03.393 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-10T00:33:03.393 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-10T00:33:03.393 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.target_file_size_base: 67108864 2023-09-10T00:33:03.393 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-10T00:33:03.393 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-10T00:33:03.394 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-10T00:33:03.394 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-10T00:33:03.394 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-10T00:33:03.394 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-10T00:33:03.394 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-10T00:33:03.395 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-10T00:33:03.395 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-10T00:33:03.395 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-10T00:33:03.395 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-10T00:33:03.395 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-10T00:33:03.396 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-10T00:33:03.396 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-10T00:33:03.396 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.arena_block_size: 1048576 2023-09-10T00:33:03.396 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-10T00:33:03.396 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-10T00:33:03.396 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.disable_auto_compactions: 0 2023-09-10T00:33:03.398 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-10T00:33:03.398 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-10T00:33:03.398 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-10T00:33:03.398 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-10T00:33:03.398 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-10T00:33:03.399 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-10T00:33:03.399 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-10T00:33:03.399 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-10T00:33:03.399 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-10T00:33:03.399 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-10T00:33:03.400 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.table_properties_collectors: 2023-09-10T00:33:03.400 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.inplace_update_support: 0 2023-09-10T00:33:03.400 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-10T00:33:03.400 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-10T00:33:03.400 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-10T00:33:03.401 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-10T00:33:03.401 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.bloom_locality: 0 2023-09-10T00:33:03.401 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.max_successive_merges: 0 2023-09-10T00:33:03.401 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-10T00:33:03.401 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.paranoid_file_checks: 0 2023-09-10T00:33:03.401 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.force_consistency_checks: 1 2023-09-10T00:33:03.402 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.report_bg_io_stats: 0 2023-09-10T00:33:03.402 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.ttl: 2592000 2023-09-10T00:33:03.402 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-10T00:33:03.402 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-10T00:33:03.402 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-10T00:33:03.403 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.enable_blob_files: false 2023-09-10T00:33:03.403 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.min_blob_size: 0 2023-09-10T00:33:03.403 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.blob_file_size: 268435456 2023-09-10T00:33:03.403 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-10T00:33:03.403 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-10T00:33:03.404 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-10T00:33:03.404 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-10T00:33:03.404 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-10T00:33:03.404 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.blob_file_starting_level: 0 2023-09-10T00:33:03.404 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-10T00:33:03.405 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-09-10T00:33:03.405 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-09-10T00:33:03.405 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4f88146a-acbb-42eb-bba6-87214341e0c2 2023-09-10T00:33:03.405 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694305983204932, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-09-10T00:33:03.405 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-09-10T00:33:03.405 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694305983205336, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1694305983, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4f88146a-acbb-42eb-bba6-87214341e0c2", "db_session_id": "I3HJ3RKVJNP1S6A6NITD", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-09-10T00:33:03.406 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694305983205401, "job": 1, "event": "recovery_finished"} 2023-09-10T00:33:03.406 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-09-10T00:33:03.406 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-10T00:33:03.406 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cf838e8000 2023-09-10T00:33:03.406 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: DB pointer 0x55cf838d4000 2023-09-10T00:33:03.407 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-10T00:33:03.407 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-10T00:33:03.407 INFO:journalctl@ceph.mon.b.smithi099.stdout: ** DB Stats ** 2023-09-10T00:33:03.407 INFO:journalctl@ceph.mon.b.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-10T00:33:03.407 INFO:journalctl@ceph.mon.b.smithi099.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-10T00:33:03.408 INFO:journalctl@ceph.mon.b.smithi099.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-10T00:33:03.408 INFO:journalctl@ceph.mon.b.smithi099.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-10T00:33:03.408 INFO:journalctl@ceph.mon.b.smithi099.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-10T00:33:03.408 INFO:journalctl@ceph.mon.b.smithi099.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-10T00:33:03.408 INFO:journalctl@ceph.mon.b.smithi099.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-10T00:33:03.408 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2023-09-10T00:33:03.409 INFO:journalctl@ceph.mon.b.smithi099.stdout: ** Compaction Stats [default] ** 2023-09-10T00:33:03.409 INFO:journalctl@ceph.mon.b.smithi099.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-10T00:33:03.409 INFO:journalctl@ceph.mon.b.smithi099.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-10T00:33:03.409 INFO:journalctl@ceph.mon.b.smithi099.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-10T00:33:03.409 INFO:journalctl@ceph.mon.b.smithi099.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-10T00:33:03.410 INFO:journalctl@ceph.mon.b.smithi099.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-10T00:33:03.410 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2023-09-10T00:33:03.410 INFO:journalctl@ceph.mon.b.smithi099.stdout: ** Compaction Stats [default] ** 2023-09-10T00:33:03.410 INFO:journalctl@ceph.mon.b.smithi099.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-10T00:33:03.410 INFO:journalctl@ceph.mon.b.smithi099.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-10T00:33:03.411 INFO:journalctl@ceph.mon.b.smithi099.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-10T00:33:03.411 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2023-09-10T00:33:03.411 INFO:journalctl@ceph.mon.b.smithi099.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-10T00:33:03.411 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2023-09-10T00:33:03.411 INFO:journalctl@ceph.mon.b.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-10T00:33:03.412 INFO:journalctl@ceph.mon.b.smithi099.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-10T00:33:03.412 INFO:journalctl@ceph.mon.b.smithi099.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-10T00:33:03.412 INFO:journalctl@ceph.mon.b.smithi099.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-10T00:33:03.412 INFO:journalctl@ceph.mon.b.smithi099.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-10T00:33:03.412 INFO:journalctl@ceph.mon.b.smithi099.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-10T00:33:03.412 INFO:journalctl@ceph.mon.b.smithi099.stdout: Cumulative compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-10T00:33:03.413 INFO:journalctl@ceph.mon.b.smithi099.stdout: Interval compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-10T00:33:03.413 INFO:journalctl@ceph.mon.b.smithi099.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-10T00:33:03.413 INFO:journalctl@ceph.mon.b.smithi099.stdout: Block cache BinnedLRUCache@0x55cf8384d1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-09-10T00:33:03.413 INFO:journalctl@ceph.mon.b.smithi099.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-09-10T00:33:03.413 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2023-09-10T00:33:03.413 INFO:journalctl@ceph.mon.b.smithi099.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-10T00:33:03.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-09-10T00:33:03.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: using public_addr v2:172.21.15.99:0/0 -> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] 2023-09-10T00:33:03.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: starting mon.b rank -1 at public addrs [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] at bind addrs [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:33:03.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.b@-1(???) e0 preinit fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:33:03.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 systemd[1]: Started Ceph mon.b for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:33:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.b@-1(synchronizing).mds e1 new map 2023-09-10T00:33:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.b@-1(synchronizing).mds e1 print_map 2023-09-10T00:33:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout: e1 2023-09-10T00:33:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-10T00:33:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-10T00:33:03.416 INFO:journalctl@ceph.mon.b.smithi099.stdout: legacy client fscid: -1 2023-09-10T00:33:03.416 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2023-09-10T00:33:03.416 INFO:journalctl@ceph.mon.b.smithi099.stdout: No filesystems configured 2023-09-10T00:33:03.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-10T00:33:03.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-10T00:33:03.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-10T00:33:03.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-10T00:33:03.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-10T00:33:03.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-10T00:33:03.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-10T00:33:03.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-10T00:33:03.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-10T00:33:03.418 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-10T00:33:03.418 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:03.418 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi131", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:33:03.418 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.418 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.419 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.419 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.419 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:33:03.419 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:03.419 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:03.419 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: Updating smithi099:/etc/ceph/ceph.conf 2023-09-10T00:33:03.420 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: Updating smithi099:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:33:03.420 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: Deploying cephadm binary to smithi131 2023-09-10T00:33:03.420 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:33:03.420 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: Updating smithi099:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-10T00:33:03.421 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:03.421 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.421 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.421 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.421 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:03.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: Added host smithi131 2023-09-10T00:33:03.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:03.423 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.423 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:33:03.423 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:03.423 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3786149350' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-10T00:33:03.423 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.424 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.424 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.424 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.424 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:33:03.424 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:03.424 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:03.425 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:03.425 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: Updating smithi131:/etc/ceph/ceph.conf 2023-09-10T00:33:03.425 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: Updating smithi131:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:33:03.425 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3786149350' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-10T00:33:03.425 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: osdmap e4: 0 total, 0 up, 0 in 2023-09-10T00:33:03.426 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.426 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.426 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.426 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:33:03.426 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: Updating smithi131:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-10T00:33:03.427 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:03.427 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.427 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:03.427 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:03.427 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:03.427 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.428 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-10T00:33:03.428 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:03.428 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi044:172.21.15.44=a;smithi099:172.21.15.99=b;smithi131:172.21.15.131=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:33:03.428 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: Saving service mon spec with placement smithi044:172.21.15.44=a;smithi099:172.21.15.99=b;smithi131:172.21.15.131=c;count:3 2023-09-10T00:33:03.428 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: Deploying daemon mon.c on smithi131 2023-09-10T00:33:03.428 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:03.429 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='client.? 172.21.15.131:0/1387211107' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-10T00:33:03.429 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:33:03.429 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.a calling monitor election 2023-09-10T00:33:03.429 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:03.429 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:03.430 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:03.430 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:03.430 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.c calling monitor election 2023-09-10T00:33:03.430 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:03.430 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:03.430 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:03.431 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:03.431 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-10T00:33:03.431 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: monmap e2: 2 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],c=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0]} removed_ranks: {} 2023-09-10T00:33:03.431 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: fsmap 2023-09-10T00:33:03.431 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: osdmap e4: 0 total, 0 up, 0 in 2023-09-10T00:33:03.432 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mgrmap e13: x(active, since 44s) 2023-09-10T00:33:03.432 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: overall HEALTH_OK 2023-09-10T00:33:03.432 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.432 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.432 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:03.432 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-10T00:33:03.433 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:03.433 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:03.433 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: Deploying daemon mon.b on smithi099 2023-09-10T00:33:03.433 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='client.? 172.21.15.131:0/3820096658' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-10T00:33:03.433 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:03.434 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-10T00:33:03.434 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-09-10T00:33:03.434 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: expand_channel_meta expand map: {default=false} 2023-09-10T00:33:03.434 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: expand_channel_meta from 'false' to 'false' 2023-09-10T00:33:03.434 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: expand_channel_meta expanded map: {default=false} 2023-09-10T00:33:03.435 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: expand_channel_meta expand map: {default=info} 2023-09-10T00:33:03.435 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: expand_channel_meta from 'info' to 'info' 2023-09-10T00:33:03.435 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: expand_channel_meta expanded map: {default=info} 2023-09-10T00:33:03.435 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: expand_channel_meta expand map: {default=daemon} 2023-09-10T00:33:03.435 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-10T00:33:03.435 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: expand_channel_meta expanded map: {default=daemon} 2023-09-10T00:33:03.436 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: expand_channel_meta expand map: {default=debug} 2023-09-10T00:33:03.436 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: expand_channel_meta from 'debug' to 'debug' 2023-09-10T00:33:03.436 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:03 smithi099 ceph-mon[102692]: expand_channel_meta expanded map: {default=debug} 2023-09-10T00:33:08.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:33:08.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: mon.a calling monitor election 2023-09-10T00:33:08.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: mon.c calling monitor election 2023-09-10T00:33:08.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:08.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:08.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:08.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:08.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:08.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:08.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:08.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:08.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:08.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-10T00:33:08.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: monmap e3: 3 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],b=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0]} removed_ranks: {} 2023-09-10T00:33:08.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: fsmap 2023-09-10T00:33:08.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: osdmap e4: 0 total, 0 up, 0 in 2023-09-10T00:33:08.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: mgrmap e13: x(active, since 52s) 2023-09-10T00:33:08.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-10T00:33:08.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-10T00:33:08.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-10T00:33:08.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: mon.b (rank 2) addr [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] is down (out of quorum) 2023-09-10T00:33:08.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:08.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:08 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:08.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:33:08.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: mon.a calling monitor election 2023-09-10T00:33:08.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: mon.c calling monitor election 2023-09-10T00:33:08.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:08.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:08.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:08.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:08.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:08.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:08.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:08.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:08.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:08.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-10T00:33:08.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: monmap e3: 3 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],b=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0]} removed_ranks: {} 2023-09-10T00:33:08.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: fsmap 2023-09-10T00:33:08.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: osdmap e4: 0 total, 0 up, 0 in 2023-09-10T00:33:08.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: mgrmap e13: x(active, since 52s) 2023-09-10T00:33:08.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-10T00:33:08.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-10T00:33:08.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-10T00:33:08.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: mon.b (rank 2) addr [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] is down (out of quorum) 2023-09-10T00:33:08.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:08.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:08 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:09.166 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-10T00:33:09.166 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":3,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","modified":"2023-09-10T00:33:03.235254Z","created":"2023-09-10T00:31:13.763341Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-10T00:33:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:dumped monmap epoch 3 2023-09-10T00:33:09.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:09 smithi131 ceph-mon[103713]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:09.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:09.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:09.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:09.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:09 smithi131 ceph-mon[103713]: from='client.? 172.21.15.131:0/3015642856' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-10T00:33:09.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:09.590 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-09-10T00:33:09.591 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config generate-minimal-conf 2023-09-10T00:33:09.624 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:09 smithi044 ceph-mon[98676]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:09.624 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:09.624 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:09.625 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:09.625 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:09 smithi044 ceph-mon[98676]: from='client.? 172.21.15.131:0/3015642856' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-10T00:33:09.625 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:10.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:33:10.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: mon.a calling monitor election 2023-09-10T00:33:10.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: mon.c calling monitor election 2023-09-10T00:33:10.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:10.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:33:10.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:10.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:10.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:10.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:10.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:10.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:10.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:10.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-10T00:33:10.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: monmap e3: 3 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],b=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0]} removed_ranks: {} 2023-09-10T00:33:10.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: fsmap 2023-09-10T00:33:10.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: osdmap e4: 0 total, 0 up, 0 in 2023-09-10T00:33:10.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: mgrmap e13: x(active, since 52s) 2023-09-10T00:33:10.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-10T00:33:10.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-10T00:33:10.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-10T00:33:10.611 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: mon.b (rank 2) addr [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] is down (out of quorum) 2023-09-10T00:33:10.611 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:10.611 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:10.611 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:10.611 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:10.612 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:10.612 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:10.612 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: from='client.? 172.21.15.131:0/3015642856' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-10T00:33:10.612 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:11.274 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: mon.b calling monitor election 2023-09-10T00:33:11.275 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:11.275 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: Updating smithi044:/etc/ceph/ceph.conf 2023-09-10T00:33:11.276 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: Updating smithi099:/etc/ceph/ceph.conf 2023-09-10T00:33:11.276 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: Updating smithi131:/etc/ceph/ceph.conf 2023-09-10T00:33:11.277 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: mon.b calling monitor election 2023-09-10T00:33:11.277 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: mon.c calling monitor election 2023-09-10T00:33:11.277 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: mon.a calling monitor election 2023-09-10T00:33:11.277 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-10T00:33:11.278 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: monmap e3: 3 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],b=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0]} removed_ranks: {} 2023-09-10T00:33:11.278 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: fsmap 2023-09-10T00:33:11.278 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: osdmap e4: 0 total, 0 up, 0 in 2023-09-10T00:33:11.278 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: mgrmap e13: x(active, since 54s) 2023-09-10T00:33:11.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-10T00:33:11.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: Cluster is now healthy 2023-09-10T00:33:11.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: overall HEALTH_OK 2023-09-10T00:33:11.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: Updating smithi044:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:33:11.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: Updating smithi099:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:33:11.280 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: Updating smithi131:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:33:11.280 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.280 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.280 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.280 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.281 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.281 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.281 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.281 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.282 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.282 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.282 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.282 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-10T00:33:11.282 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-10T00:33:11.283 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:11.283 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: mon.b calling monitor election 2023-09-10T00:33:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: Updating smithi044:/etc/ceph/ceph.conf 2023-09-10T00:33:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: Updating smithi099:/etc/ceph/ceph.conf 2023-09-10T00:33:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: Updating smithi131:/etc/ceph/ceph.conf 2023-09-10T00:33:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: mon.b calling monitor election 2023-09-10T00:33:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: mon.c calling monitor election 2023-09-10T00:33:11.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: mon.a calling monitor election 2023-09-10T00:33:11.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-10T00:33:11.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: monmap e3: 3 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],b=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0]} removed_ranks: {} 2023-09-10T00:33:11.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: fsmap 2023-09-10T00:33:11.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: osdmap e4: 0 total, 0 up, 0 in 2023-09-10T00:33:11.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: mgrmap e13: x(active, since 54s) 2023-09-10T00:33:11.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-10T00:33:11.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: Cluster is now healthy 2023-09-10T00:33:11.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: overall HEALTH_OK 2023-09-10T00:33:11.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: Updating smithi044:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:33:11.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: Updating smithi099:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:33:11.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: Updating smithi131:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:33:11.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.361 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.361 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.361 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.361 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.361 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.361 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.362 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.362 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.362 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-10T00:33:11.362 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-10T00:33:11.362 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:11.363 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: mon.b calling monitor election 2023-09-10T00:33:11.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:11.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: Updating smithi044:/etc/ceph/ceph.conf 2023-09-10T00:33:11.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: Updating smithi099:/etc/ceph/ceph.conf 2023-09-10T00:33:11.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: Updating smithi131:/etc/ceph/ceph.conf 2023-09-10T00:33:11.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: mon.b calling monitor election 2023-09-10T00:33:11.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: mon.c calling monitor election 2023-09-10T00:33:11.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: mon.a calling monitor election 2023-09-10T00:33:11.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-10T00:33:11.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: monmap e3: 3 mons at {a=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0],b=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0]} removed_ranks: {} 2023-09-10T00:33:11.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: fsmap 2023-09-10T00:33:11.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: osdmap e4: 0 total, 0 up, 0 in 2023-09-10T00:33:11.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: mgrmap e13: x(active, since 54s) 2023-09-10T00:33:11.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-10T00:33:11.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: Cluster is now healthy 2023-09-10T00:33:11.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: overall HEALTH_OK 2023-09-10T00:33:11.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: Updating smithi044:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:33:11.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: Updating smithi099:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:33:11.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: Updating smithi131:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:33:11.550 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.550 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.551 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.551 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.551 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.552 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.552 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.552 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-10T00:33:11.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-10T00:33:11.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:11.555 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:11.682 INFO:teuthology.orchestra.run.smithi044.stdout:# minimal ceph.conf for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:33:11.682 INFO:teuthology.orchestra.run.smithi044.stdout:[global] 2023-09-10T00:33:11.682 INFO:teuthology.orchestra.run.smithi044.stdout: fsid = 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:33:11.683 INFO:teuthology.orchestra.run.smithi044.stdout: mon_host = [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] 2023-09-10T00:33:12.227 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'] 2023-09-10T00:33:12.227 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set client client_mount_timeout 600 2023-09-10T00:33:12.269 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'debug_client', '20'] 2023-09-10T00:33:12.270 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set client debug_client 20 2023-09-10T00:33:12.293 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:12 smithi044 ceph-mon[98676]: Reconfiguring mon.a (unknown last config time)... 2023-09-10T00:33:12.294 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:12 smithi044 ceph-mon[98676]: Reconfiguring daemon mon.a on smithi044 2023-09-10T00:33:12.294 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:12 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:12.294 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:12 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2115694209' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:12.295 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'debug_ms', '1'] 2023-09-10T00:33:12.295 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set client debug_ms 1 2023-09-10T00:33:12.338 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'] 2023-09-10T00:33:12.338 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set client rados_mon_op_timeout 900 2023-09-10T00:33:12.381 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'] 2023-09-10T00:33:12.382 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set client rados_osd_op_timeout 900 2023-09-10T00:33:12.425 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'debug_mds', '20'] 2023-09-10T00:33:12.425 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set mds debug_mds 20 2023-09-10T00:33:12.469 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'] 2023-09-10T00:33:12.469 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set mds debug_mds_balancer 20 2023-09-10T00:33:12.515 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'debug_ms', '1'] 2023-09-10T00:33:12.515 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set mds debug_ms 1 2023-09-10T00:33:12.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:12 smithi131 ceph-mon[103713]: Reconfiguring mon.a (unknown last config time)... 2023-09-10T00:33:12.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:12 smithi131 ceph-mon[103713]: Reconfiguring daemon mon.a on smithi044 2023-09-10T00:33:12.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:12 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:12.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:12 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2115694209' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:12.563 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'] 2023-09-10T00:33:12.563 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_bal_fragment_size_max 10000 2023-09-10T00:33:12.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:12 smithi099 ceph-mon[102692]: Reconfiguring mon.a (unknown last config time)... 2023-09-10T00:33:12.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:12 smithi099 ceph-mon[102692]: Reconfiguring daemon mon.a on smithi044 2023-09-10T00:33:12.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:12 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:33:12.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:12 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2115694209' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:12.608 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'] 2023-09-10T00:33:12.608 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_bal_merge_size 5 2023-09-10T00:33:12.651 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'] 2023-09-10T00:33:12.651 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_bal_split_bits 3 2023-09-10T00:33:12.707 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'] 2023-09-10T00:33:12.708 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_bal_split_size 100 2023-09-10T00:33:12.751 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'] 2023-09-10T00:33:12.751 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_debug_frag True 2023-09-10T00:33:12.803 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'] 2023-09-10T00:33:12.804 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_debug_scatterstat True 2023-09-10T00:33:12.862 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'] 2023-09-10T00:33:12.862 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_op_complaint_time 180 2023-09-10T00:33:12.879 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:12.880 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'] 2023-09-10T00:33:12.880 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_verify_scatter True 2023-09-10T00:33:12.937 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'] 2023-09-10T00:33:12.938 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set mds osd_op_complaint_time 180 2023-09-10T00:33:12.981 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'] 2023-09-10T00:33:12.981 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set mds rados_mon_op_timeout 900 2023-09-10T00:33:13.021 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:13.042 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'] 2023-09-10T00:33:13.043 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set mds rados_osd_op_timeout 900 2023-09-10T00:33:13.093 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'] 2023-09-10T00:33:13.094 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set mon mon_op_complaint_time 120 2023-09-10T00:33:13.153 INFO:tasks.cephadm:['ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'] 2023-09-10T00:33:13.153 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config set osd osd_op_complaint_time 180 2023-09-10T00:33:13.221 DEBUG:tasks.cephadm:set 21 configs 2023-09-10T00:33:13.222 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:13.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:13 smithi044 ceph-mon[98676]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:13.771 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:13 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:13.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:13 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:13.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:13 smithi044 ceph-mon[98676]: Reconfiguring mon.b (monmap changed)... 2023-09-10T00:33:13.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:13 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-10T00:33:13.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:13 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-10T00:33:13.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:13 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:13.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:13 smithi044 ceph-mon[98676]: Reconfiguring daemon mon.b on smithi099 2023-09-10T00:33:13.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:13 smithi099 ceph-mon[102692]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:13.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:13 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:13.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:13 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:13.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:13 smithi099 ceph-mon[102692]: Reconfiguring mon.b (monmap changed)... 2023-09-10T00:33:13.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:13 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-10T00:33:13.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:13 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-10T00:33:13.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:13 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:13.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:13 smithi099 ceph-mon[102692]: Reconfiguring daemon mon.b on smithi099 2023-09-10T00:33:13.865 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:13.882 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:13.883 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:13.926 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:13 smithi131 ceph-mon[103713]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:13 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:13 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:14.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:13 smithi131 ceph-mon[103713]: Reconfiguring mon.b (monmap changed)... 2023-09-10T00:33:14.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:13 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-10T00:33:14.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:13 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-10T00:33:14.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:13 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:14.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:13 smithi131 ceph-mon[103713]: Reconfiguring daemon mon.b on smithi099 2023-09-10T00:33:14.870 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:14.871 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:14.871 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:14 smithi131 ceph-mon[103713]: Reconfiguring mon.c (monmap changed)... 2023-09-10T00:33:14.871 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-10T00:33:14.871 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-10T00:33:14.871 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:14.872 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:14 smithi131 ceph-mon[103713]: Reconfiguring daemon mon.c on smithi131 2023-09-10T00:33:15.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:14 smithi044 ceph-mon[98676]: Reconfiguring mon.c (monmap changed)... 2023-09-10T00:33:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-10T00:33:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-10T00:33:15.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:15.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:14 smithi044 ceph-mon[98676]: Reconfiguring daemon mon.c on smithi131 2023-09-10T00:33:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:15.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:14 smithi099 ceph-mon[102692]: Reconfiguring mon.c (monmap changed)... 2023-09-10T00:33:15.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-10T00:33:15.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-10T00:33:15.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:15.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:14 smithi099 ceph-mon[102692]: Reconfiguring daemon mon.c on smithi131 2023-09-10T00:33:15.253 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:15.882 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:15 smithi044 ceph-mon[98676]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:15.883 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:15 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:15.883 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:15 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:15.883 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:15 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:15.884 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:15 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:15.884 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:15 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:15.884 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:15 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:15.952 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:15 smithi131 ceph-mon[103713]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:15 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:16.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:15 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:16.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:15 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:16.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:15 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:16.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:15 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:16.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:15 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:16.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:15 smithi099 ceph-mon[102692]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:16.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:15 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:16.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:15 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:16.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:15 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:16.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:15 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:16.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:15 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:16.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:15 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:17.199 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:17 smithi044 ceph-mon[98676]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:17.200 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:17 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2184951961' entity='client.admin' 2023-09-10T00:33:17.200 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:17 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/390773423' entity='client.admin' 2023-09-10T00:33:17.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:17 smithi131 ceph-mon[103713]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:17.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:17 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2184951961' entity='client.admin' 2023-09-10T00:33:17.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:17 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/390773423' entity='client.admin' 2023-09-10T00:33:17.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:17 smithi099 ceph-mon[102692]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:17.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:17 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2184951961' entity='client.admin' 2023-09-10T00:33:17.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:17 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/390773423' entity='client.admin' 2023-09-10T00:33:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:17.857+0000 7fef2bd63700 1 -- 172.21.15.44:0/336149272 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fef10038630 msgr2=0x7fef1003aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:17.857+0000 7fef2bd63700 1 --2- 172.21.15.44:0/336149272 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fef10038630 0x7fef1003aaf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fef24069c10 tx=0x7fef20005b90 comp rx=0 tx=0).stop 2023-09-10T00:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:17.857+0000 7fef2bd63700 1 -- 172.21.15.44:0/336149272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef240ffee0 msgr2=0x7fef241958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:17.857+0000 7fef2bd63700 1 --2- 172.21.15.44:0/336149272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef240ffee0 0x7fef241958c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fef1400bfa0 tx=0x7fef1400fab0 comp rx=0 tx=0).stop 2023-09-10T00:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:17.858+0000 7fef2bd63700 1 -- 172.21.15.44:0/336149272 shutdown_connections 2023-09-10T00:33:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:17.858+0000 7fef2bd63700 1 --2- 172.21.15.44:0/336149272 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fef10038630 0x7fef1003aaf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:18.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:17.858+0000 7fef2bd63700 1 --2- 172.21.15.44:0/336149272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef24104530 0x7fef24199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:18.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:17.858+0000 7fef2bd63700 1 --2- 172.21.15.44:0/336149272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef240ff520 0x7fef24195380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:18.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:17.858+0000 7fef2bd63700 1 --2- 172.21.15.44:0/336149272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef240ffee0 0x7fef241958c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:18.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:17.858+0000 7fef2bd63700 1 -- 172.21.15.44:0/336149272 >> 172.21.15.44:0/336149272 conn(0x7fef24076600 msgr2=0x7fef240fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:18.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:17.858+0000 7fef2bd63700 1 -- 172.21.15.44:0/336149272 shutdown_connections 2023-09-10T00:33:18.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:17.858+0000 7fef2bd63700 1 -- 172.21.15.44:0/336149272 wait complete. 2023-09-10T00:33:18.616 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:18.617 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:18.636 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:18.702 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:18.866 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.864+0000 7f77818b6700 1 -- 172.21.15.44:0/3186655221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f777c102150 msgr2=0x7f777c1025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:18.866 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.864+0000 7f77818b6700 1 --2- 172.21.15.44:0/3186655221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f777c102150 0x7f777c1025d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7770009990 tx=0x7f7770009ca0 comp rx=0 tx=0).stop 2023-09-10T00:33:18.866 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.864+0000 7f77818b6700 1 -- 172.21.15.44:0/3186655221 shutdown_connections 2023-09-10T00:33:18.867 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.864+0000 7f77818b6700 1 --2- 172.21.15.44:0/3186655221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f777c102b10 0x7f777c10d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:18.867 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.864+0000 7f77818b6700 1 --2- 172.21.15.44:0/3186655221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f777c068d80 0x7f777c069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:18.867 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.864+0000 7f77818b6700 1 --2- 172.21.15.44:0/3186655221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f777c102150 0x7f777c1025d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:18.867 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.864+0000 7f77818b6700 1 -- 172.21.15.44:0/3186655221 >> 172.21.15.44:0/3186655221 conn(0x7f777c0765f0 msgr2=0x7f777c076a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:18.867 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.864+0000 7f77818b6700 1 -- 172.21.15.44:0/3186655221 shutdown_connections 2023-09-10T00:33:18.868 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.864+0000 7f77818b6700 1 -- 172.21.15.44:0/3186655221 wait complete. 2023-09-10T00:33:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.865+0000 7f77818b6700 1 Processor -- start 2023-09-10T00:33:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.865+0000 7f77818b6700 1 -- start start 2023-09-10T00:33:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f77818b6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f777c068d80 0x7f777c196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f77818b6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f777c102150 0x7f777c196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:18.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f77818b6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f777c102b10 0x7f777c19aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:18.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f77818b6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c19b4f0 con 0x7f777c068d80 2023-09-10T00:33:18.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f77818b6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c19b660 con 0x7f777c102150 2023-09-10T00:33:18.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f77818b6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c19b820 con 0x7f777c102b10 2023-09-10T00:33:18.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f777affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f777c068d80 0x7f777c196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:18.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f777affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f777c068d80 0x7f777c196530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45170/0 (socket says 172.21.15.44:45170) 2023-09-10T00:33:18.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f777affd700 1 -- 172.21.15.44:0/4125157066 learned_addr learned my addr 172.21.15.44:0/4125157066 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:18.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f777b7fe700 1 --2- 172.21.15.44:0/4125157066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f777c102b10 0x7f777c19aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f777a7fc700 1 --2- 172.21.15.44:0/4125157066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f777c102150 0x7f777c196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f777affd700 1 -- 172.21.15.44:0/4125157066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f777c102b10 msgr2=0x7f777c19aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f777affd700 1 --2- 172.21.15.44:0/4125157066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f777c102b10 0x7f777c19aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f777affd700 1 -- 172.21.15.44:0/4125157066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f777c102150 msgr2=0x7f777c196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f777affd700 1 --2- 172.21.15.44:0/4125157066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f777c102150 0x7f777c196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:18.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f777affd700 1 -- 172.21.15.44:0/4125157066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77700095f0 con 0x7f777c068d80 2023-09-10T00:33:18.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.866+0000 7f777affd700 1 --2- 172.21.15.44:0/4125157066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f777c068d80 0x7f777c196530 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f776c00e9a0 tx=0x7f776c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:18.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.867+0000 7f77808b4700 1 -- 172.21.15.44:0/4125157066 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c009790 con 0x7f777c068d80 2023-09-10T00:33:18.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.867+0000 7f77808b4700 1 -- 172.21.15.44:0/4125157066 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(23 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f776c00cbd0 con 0x7f777c068d80 2023-09-10T00:33:18.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.867+0000 7f77818b6700 1 -- 172.21.15.44:0/4125157066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777c19bb00 con 0x7f777c068d80 2023-09-10T00:33:18.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.867+0000 7f77808b4700 1 -- 172.21.15.44:0/4125157066 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c018420 con 0x7f777c068d80 2023-09-10T00:33:18.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.867+0000 7f77818b6700 1 -- 172.21.15.44:0/4125157066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777c1a3360 con 0x7f777c068d80 2023-09-10T00:33:18.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.868+0000 7f77818b6700 1 -- 172.21.15.44:0/4125157066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777c04e300 con 0x7f777c068d80 2023-09-10T00:33:18.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.868+0000 7f77808b4700 1 -- 172.21.15.44:0/4125157066 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f776c010430 con 0x7f777c068d80 2023-09-10T00:33:18.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.868+0000 7f77808b4700 1 --2- 172.21.15.44:0/4125157066 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7764038630 0x7f776403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:18.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.868+0000 7f77808b4700 1 -- 172.21.15.44:0/4125157066 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f776c014070 con 0x7f777c068d80 2023-09-10T00:33:18.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.868+0000 7f777a7fc700 1 --2- 172.21.15.44:0/4125157066 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7764038630 0x7f776403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:18.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.869+0000 7f777a7fc700 1 --2- 172.21.15.44:0/4125157066 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7764038630 0x7f776403aaf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f777000b2e0 tx=0x7f7770005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:18.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.873+0000 7f77808b4700 1 -- 172.21.15.44:0/4125157066 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f776c016330 con 0x7f777c068d80 2023-09-10T00:33:19.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:19.084+0000 7f77818b6700 1 -- 172.21.15.44:0/4125157066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) v1 -- 0x7f777c197890 con 0x7f777c068d80 2023-09-10T00:33:19.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:19.087+0000 7f77808b4700 1 -- 172.21.15.44:0/4125157066 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v14)=0 v14) v1 ==== 108+0+0 (secure 0 0 0) 0x7f776c01fab0 con 0x7f777c068d80 2023-09-10T00:33:19.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:19.089+0000 7f77818b6700 1 -- 172.21.15.44:0/4125157066 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7764038630 msgr2=0x7f776403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:19.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:19.089+0000 7f77818b6700 1 --2- 172.21.15.44:0/4125157066 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7764038630 0x7f776403aaf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f777000b2e0 tx=0x7f7770005fd0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:19.089+0000 7f77818b6700 1 -- 172.21.15.44:0/4125157066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f777c068d80 msgr2=0x7f777c196530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:19.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:19.089+0000 7f77818b6700 1 --2- 172.21.15.44:0/4125157066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f777c068d80 0x7f777c196530 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f776c00e9a0 tx=0x7f776c00ecb0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:19.090+0000 7f77818b6700 1 -- 172.21.15.44:0/4125157066 shutdown_connections 2023-09-10T00:33:19.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:19.090+0000 7f77818b6700 1 --2- 172.21.15.44:0/4125157066 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7764038630 0x7f776403aaf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:19.090+0000 7f77818b6700 1 --2- 172.21.15.44:0/4125157066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f777c102b10 0x7f777c19aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:19.090+0000 7f77818b6700 1 --2- 172.21.15.44:0/4125157066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f777c068d80 0x7f777c196530 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:19.090+0000 7f77818b6700 1 --2- 172.21.15.44:0/4125157066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f777c102150 0x7f777c196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:19.090+0000 7f77818b6700 1 -- 172.21.15.44:0/4125157066 >> 172.21.15.44:0/4125157066 conn(0x7f777c0765f0 msgr2=0x7f777c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:19.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:19.090+0000 7f77818b6700 1 -- 172.21.15.44:0/4125157066 shutdown_connections 2023-09-10T00:33:19.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:19.090+0000 7f77818b6700 1 -- 172.21.15.44:0/4125157066 wait complete. 2023-09-10T00:33:19.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:18 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/336149272' entity='client.admin' 2023-09-10T00:33:19.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:18 smithi131 ceph-mon[103713]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:19.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:18 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/528056401' entity='client.admin' 2023-09-10T00:33:19.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:18 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/336149272' entity='client.admin' 2023-09-10T00:33:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:18 smithi044 ceph-mon[98676]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:18 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/528056401' entity='client.admin' 2023-09-10T00:33:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:18 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/336149272' entity='client.admin' 2023-09-10T00:33:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:18 smithi099 ceph-mon[102692]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:18 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/528056401' entity='client.admin' 2023-09-10T00:33:19.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.188+0000 7f4953b87700 1 -- 172.21.15.44:0/500183393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494c1048e0 msgr2=0x7f494c10c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:19.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.188+0000 7f4953b87700 1 --2- 172.21.15.44:0/500183393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494c1048e0 0x7f494c10c460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f494400b0a0 tx=0x7f494400b3b0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.188+0000 7f4953b87700 1 -- 172.21.15.44:0/500183393 shutdown_connections 2023-09-10T00:33:19.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.188+0000 7f4953b87700 1 --2- 172.21.15.44:0/500183393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494c1048e0 0x7f494c10c460 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.188+0000 7f4953b87700 1 --2- 172.21.15.44:0/500183393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f494c0ffec0 0x7f494c1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.188+0000 7f4953b87700 1 --2- 172.21.15.44:0/500183393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c100870 0x7f494c1041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.188+0000 7f4953b87700 1 -- 172.21.15.44:0/500183393 >> 172.21.15.44:0/500183393 conn(0x7f494c075a70 msgr2=0x7f494c0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:19.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.188+0000 7f4953b87700 1 -- 172.21.15.44:0/500183393 shutdown_connections 2023-09-10T00:33:19.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.188+0000 7f4953b87700 1 -- 172.21.15.44:0/500183393 wait complete. 2023-09-10T00:33:19.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4953b87700 1 Processor -- start 2023-09-10T00:33:19.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4953b87700 1 -- start start 2023-09-10T00:33:19.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4953b87700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494c0ffec0 0x7f494c196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:19.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4953b87700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c100870 0x7f494c196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:19.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4953b87700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f494c1048e0 0x7f494c19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:19.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4953b87700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c19b5c0 con 0x7f494c0ffec0 2023-09-10T00:33:19.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4953b87700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c19b730 con 0x7f494c1048e0 2023-09-10T00:33:19.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4953b87700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c19b8f0 con 0x7f494c100870 2023-09-10T00:33:19.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4951923700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494c0ffec0 0x7f494c196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:19.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4951923700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494c0ffec0 0x7f494c196600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45156/0 (socket says 172.21.15.44:45156) 2023-09-10T00:33:19.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4951923700 1 -- 172.21.15.44:0/528056401 learned_addr learned my addr 172.21.15.44:0/528056401 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:19.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4952124700 1 --2- 172.21.15.44:0/528056401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f494c1048e0 0x7f494c19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:19.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4951122700 1 --2- 172.21.15.44:0/528056401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c100870 0x7f494c196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:19.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4951923700 1 -- 172.21.15.44:0/528056401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c100870 msgr2=0x7f494c196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:19.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4951923700 1 --2- 172.21.15.44:0/528056401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c100870 0x7f494c196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4951923700 1 -- 172.21.15.44:0/528056401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f494c1048e0 msgr2=0x7f494c19afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:19.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4951923700 1 --2- 172.21.15.44:0/528056401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f494c1048e0 0x7f494c19afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4951923700 1 -- 172.21.15.44:0/528056401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4944009d00 con 0x7f494c0ffec0 2023-09-10T00:33:19.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4951122700 1 --2- 172.21.15.44:0/528056401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c100870 0x7f494c196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:33:19.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.189+0000 7f4951923700 1 --2- 172.21.15.44:0/528056401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494c0ffec0 0x7f494c196600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f494800b530 tx=0x7f494800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:19.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.190+0000 7f4942ffd700 1 -- 172.21.15.44:0/528056401 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948011830 con 0x7f494c0ffec0 2023-09-10T00:33:19.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.190+0000 7f4942ffd700 1 -- 172.21.15.44:0/528056401 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(23 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f4948011de0 con 0x7f494c0ffec0 2023-09-10T00:33:19.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.190+0000 7f4942ffd700 1 -- 172.21.15.44:0/528056401 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948010cb0 con 0x7f494c0ffec0 2023-09-10T00:33:19.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.190+0000 7f4953b87700 1 -- 172.21.15.44:0/528056401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494c19bbd0 con 0x7f494c0ffec0 2023-09-10T00:33:19.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.190+0000 7f4953b87700 1 -- 172.21.15.44:0/528056401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494c1a3430 con 0x7f494c0ffec0 2023-09-10T00:33:19.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.191+0000 7f4942ffd700 1 -- 172.21.15.44:0/528056401 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f4948011990 con 0x7f494c0ffec0 2023-09-10T00:33:19.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.191+0000 7f4942ffd700 1 --2- 172.21.15.44:0/528056401 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4938038630 0x7f493803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:19.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.191+0000 7f4953b87700 1 -- 172.21.15.44:0/528056401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494c04e300 con 0x7f494c0ffec0 2023-09-10T00:33:19.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.191+0000 7f4942ffd700 1 -- 172.21.15.44:0/528056401 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f494804a9c0 con 0x7f494c0ffec0 2023-09-10T00:33:19.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.191+0000 7f4951122700 1 --2- 172.21.15.44:0/528056401 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4938038630 0x7f493803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:19.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.191+0000 7f4951122700 1 --2- 172.21.15.44:0/528056401 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4938038630 0x7f493803aaf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f494c197c20 tx=0x7f493c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:19.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.196+0000 7f4942ffd700 1 -- 172.21.15.44:0/528056401 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f494803de60 con 0x7f494c0ffec0 2023-09-10T00:33:19.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.401+0000 7f4953b87700 1 -- 172.21.15.44:0/528056401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) v1 -- 0x7f494c197960 con 0x7f494c0ffec0 2023-09-10T00:33:19.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.404+0000 7f4942ffd700 1 -- 172.21.15.44:0/528056401 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v13)=0 v13) v1 ==== 117+0+0 (secure 0 0 0) 0x7f4948015070 con 0x7f494c0ffec0 2023-09-10T00:33:19.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.406+0000 7f4953b87700 1 -- 172.21.15.44:0/528056401 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4938038630 msgr2=0x7f493803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:19.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.406+0000 7f4953b87700 1 --2- 172.21.15.44:0/528056401 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4938038630 0x7f493803aaf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f494c197c20 tx=0x7f493c008040 comp rx=0 tx=0).stop 2023-09-10T00:33:19.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.406+0000 7f4953b87700 1 -- 172.21.15.44:0/528056401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494c0ffec0 msgr2=0x7f494c196600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:19.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.406+0000 7f4953b87700 1 --2- 172.21.15.44:0/528056401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494c0ffec0 0x7f494c196600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f494800b530 tx=0x7f494800b8f0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.407+0000 7f4953b87700 1 -- 172.21.15.44:0/528056401 shutdown_connections 2023-09-10T00:33:19.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.407+0000 7f4953b87700 1 --2- 172.21.15.44:0/528056401 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4938038630 0x7f493803aaf0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.407+0000 7f4953b87700 1 --2- 172.21.15.44:0/528056401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f494c1048e0 0x7f494c19afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.407+0000 7f4953b87700 1 --2- 172.21.15.44:0/528056401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494c0ffec0 0x7f494c196600 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.407+0000 7f4953b87700 1 --2- 172.21.15.44:0/528056401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c100870 0x7f494c196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:19.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.407+0000 7f4953b87700 1 -- 172.21.15.44:0/528056401 >> 172.21.15.44:0/528056401 conn(0x7f494c075a70 msgr2=0x7f494c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:19.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.407+0000 7f4953b87700 1 -- 172.21.15.44:0/528056401 shutdown_connections 2023-09-10T00:33:19.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:18.407+0000 7f4953b87700 1 -- 172.21.15.44:0/528056401 wait complete. 2023-09-10T00:33:20.294 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:20 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/4125157066' entity='client.admin' 2023-09-10T00:33:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:20.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:20 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/4125157066' entity='client.admin' 2023-09-10T00:33:20.527 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:20.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:20 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/4125157066' entity='client.admin' 2023-09-10T00:33:20.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.736+0000 7fe264056700 1 -- 172.21.15.44:0/3296024422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c0ff520 msgr2=0x7fe25c0ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:20.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.736+0000 7fe264056700 1 --2- 172.21.15.44:0/3296024422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c0ff520 0x7fe25c0ff9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009990 tx=0x7fe24c009ca0 comp rx=0 tx=0).stop 2023-09-10T00:33:20.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.736+0000 7fe264056700 1 -- 172.21.15.44:0/3296024422 shutdown_connections 2023-09-10T00:33:20.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.736+0000 7fe264056700 1 --2- 172.21.15.44:0/3296024422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe25c0ffee0 0x7fe25c10c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:20.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.736+0000 7fe264056700 1 --2- 172.21.15.44:0/3296024422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe25c104530 0x7fe25c104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:20.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.736+0000 7fe264056700 1 --2- 172.21.15.44:0/3296024422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c0ff520 0x7fe25c0ff9a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:20.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.736+0000 7fe264056700 1 -- 172.21.15.44:0/3296024422 >> 172.21.15.44:0/3296024422 conn(0x7fe25c076600 msgr2=0x7fe25c076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:20.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.736+0000 7fe264056700 1 -- 172.21.15.44:0/3296024422 shutdown_connections 2023-09-10T00:33:20.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.736+0000 7fe264056700 1 -- 172.21.15.44:0/3296024422 wait complete. 2023-09-10T00:33:20.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.736+0000 7fe264056700 1 Processor -- start 2023-09-10T00:33:20.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.736+0000 7fe264056700 1 -- start start 2023-09-10T00:33:20.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe264056700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c0ff520 0x7fe25c195490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:20.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe264056700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe25c0ffee0 0x7fe25c1959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:20.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe264056700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe25c104530 0x7fe25c199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:20.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe264056700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c19a3b0 con 0x7fe25c0ff520 2023-09-10T00:33:20.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe264056700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c19a520 con 0x7fe25c104530 2023-09-10T00:33:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe264056700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c19a6e0 con 0x7fe25c0ffee0 2023-09-10T00:33:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe261df2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c0ff520 0x7fe25c195490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe261df2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c0ff520 0x7fe25c195490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45184/0 (socket says 172.21.15.44:45184) 2023-09-10T00:33:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe261df2700 1 -- 172.21.15.44:0/29910864 learned_addr learned my addr 172.21.15.44:0/29910864 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe2615f1700 1 --2- 172.21.15.44:0/29910864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe25c0ffee0 0x7fe25c1959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe2625f3700 1 --2- 172.21.15.44:0/29910864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe25c104530 0x7fe25c199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:20.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe261df2700 1 -- 172.21.15.44:0/29910864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe25c0ffee0 msgr2=0x7fe25c1959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:20.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe261df2700 1 --2- 172.21.15.44:0/29910864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe25c0ffee0 0x7fe25c1959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:20.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe261df2700 1 -- 172.21.15.44:0/29910864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe25c104530 msgr2=0x7fe25c199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:20.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe261df2700 1 --2- 172.21.15.44:0/29910864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe25c104530 0x7fe25c199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:20.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe261df2700 1 -- 172.21.15.44:0/29910864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24c0095f0 con 0x7fe25c0ff520 2023-09-10T00:33:20.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe2625f3700 1 --2- 172.21.15.44:0/29910864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe25c104530 0x7fe25c199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:33:20.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe261df2700 1 --2- 172.21.15.44:0/29910864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c0ff520 0x7fe25c195490 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe25800e970 tx=0x7fe25800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:20.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe252ffd700 1 -- 172.21.15.44:0/29910864 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258009790 con 0x7fe25c0ff520 2023-09-10T00:33:20.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe252ffd700 1 -- 172.21.15.44:0/29910864 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(23 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7fe258004470 con 0x7fe25c0ff520 2023-09-10T00:33:20.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe252ffd700 1 -- 172.21.15.44:0/29910864 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258003b50 con 0x7fe25c0ff520 2023-09-10T00:33:20.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe264056700 1 -- 172.21.15.44:0/29910864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25c19a9c0 con 0x7fe25c0ff520 2023-09-10T00:33:20.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.737+0000 7fe264056700 1 -- 172.21.15.44:0/29910864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25c1a2220 con 0x7fe25c0ff520 2023-09-10T00:33:20.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.738+0000 7fe252ffd700 1 -- 172.21.15.44:0/29910864 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fe258003cf0 con 0x7fe25c0ff520 2023-09-10T00:33:20.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.738+0000 7fe264056700 1 -- 172.21.15.44:0/29910864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe25c107ad0 con 0x7fe25c0ff520 2023-09-10T00:33:20.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.738+0000 7fe252ffd700 1 --2- 172.21.15.44:0/29910864 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe248038630 0x7fe24803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:20.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.738+0000 7fe252ffd700 1 -- 172.21.15.44:0/29910864 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe258014070 con 0x7fe25c0ff520 2023-09-10T00:33:20.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.739+0000 7fe2615f1700 1 --2- 172.21.15.44:0/29910864 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe248038630 0x7fe24803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:20.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.739+0000 7fe2615f1700 1 --2- 172.21.15.44:0/29910864 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe248038630 0x7fe24803aaf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe24c006010 tx=0x7fe24c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:20.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.743+0000 7fe252ffd700 1 -- 172.21.15.44:0/29910864 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe25800fbb0 con 0x7fe25c0ff520 2023-09-10T00:33:20.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.948+0000 7fe264056700 1 -- 172.21.15.44:0/29910864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) v1 -- 0x7fe25c04e300 con 0x7fe25c0ff520 2023-09-10T00:33:20.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.951+0000 7fe252ffd700 1 -- 172.21.15.44:0/29910864 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v15)=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe25800fbb0 con 0x7fe25c0ff520 2023-09-10T00:33:20.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.953+0000 7fe264056700 1 -- 172.21.15.44:0/29910864 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe248038630 msgr2=0x7fe24803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:20.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.953+0000 7fe264056700 1 --2- 172.21.15.44:0/29910864 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe248038630 0x7fe24803aaf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe24c006010 tx=0x7fe24c00b260 comp rx=0 tx=0).stop 2023-09-10T00:33:20.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.953+0000 7fe264056700 1 -- 172.21.15.44:0/29910864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c0ff520 msgr2=0x7fe25c195490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:20.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.953+0000 7fe264056700 1 --2- 172.21.15.44:0/29910864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c0ff520 0x7fe25c195490 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe25800e970 tx=0x7fe25800ed30 comp rx=0 tx=0).stop 2023-09-10T00:33:20.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.953+0000 7fe264056700 1 -- 172.21.15.44:0/29910864 shutdown_connections 2023-09-10T00:33:20.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.953+0000 7fe264056700 1 --2- 172.21.15.44:0/29910864 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe248038630 0x7fe24803aaf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:20.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.953+0000 7fe264056700 1 --2- 172.21.15.44:0/29910864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe25c104530 0x7fe25c199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:20.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.953+0000 7fe264056700 1 --2- 172.21.15.44:0/29910864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe25c0ff520 0x7fe25c195490 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:20.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.953+0000 7fe264056700 1 --2- 172.21.15.44:0/29910864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe25c0ffee0 0x7fe25c1959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:20.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.953+0000 7fe264056700 1 -- 172.21.15.44:0/29910864 >> 172.21.15.44:0/29910864 conn(0x7fe25c076600 msgr2=0x7fe25c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:20.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.953+0000 7fe264056700 1 -- 172.21.15.44:0/29910864 shutdown_connections 2023-09-10T00:33:20.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:20.953+0000 7fe264056700 1 -- 172.21.15.44:0/29910864 wait complete. 2023-09-10T00:33:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:21.746 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:21 smithi044 ceph-mon[98676]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:21.746 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:21 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/29910864' entity='client.admin' 2023-09-10T00:33:21.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:21 smithi131 ceph-mon[103713]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:21.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:21 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/29910864' entity='client.admin' 2023-09-10T00:33:21.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:21 smithi099 ceph-mon[102692]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:21.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:21 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/29910864' entity='client.admin' 2023-09-10T00:33:22.012 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:22.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.085+0000 7f2980e6d700 1 -- 172.21.15.44:0/2896921665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f297c1001a0 msgr2=0x7f297c101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:22.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.085+0000 7f2980e6d700 1 --2- 172.21.15.44:0/2896921665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f297c1001a0 0x7f297c101110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2970009990 tx=0x7f2970009ca0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.086+0000 7f2980e6d700 1 -- 172.21.15.44:0/2896921665 shutdown_connections 2023-09-10T00:33:22.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.086+0000 7f2980e6d700 1 --2- 172.21.15.44:0/2896921665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f297c101840 0x7f297c10c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.086+0000 7f2980e6d700 1 --2- 172.21.15.44:0/2896921665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f297c0ff880 0x7f297c0ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.086+0000 7f2980e6d700 1 --2- 172.21.15.44:0/2896921665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f297c1001a0 0x7f297c101110 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.086+0000 7f2980e6d700 1 -- 172.21.15.44:0/2896921665 >> 172.21.15.44:0/2896921665 conn(0x7f297c0765f0 msgr2=0x7f297c076a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:22.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.086+0000 7f2980e6d700 1 -- 172.21.15.44:0/2896921665 shutdown_connections 2023-09-10T00:33:22.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.086+0000 7f2980e6d700 1 -- 172.21.15.44:0/2896921665 wait complete. 2023-09-10T00:33:22.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f2980e6d700 1 Processor -- start 2023-09-10T00:33:22.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f2980e6d700 1 -- start start 2023-09-10T00:33:22.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f2980e6d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f297c0ff880 0x7f297c1954b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:22.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f2980e6d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f297c1001a0 0x7f297c1959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:22.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f2980e6d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f297c101840 0x7f297c199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:22.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f2980e6d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c19a470 con 0x7f297c1001a0 2023-09-10T00:33:22.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f2980e6d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c19a5e0 con 0x7f297c101840 2023-09-10T00:33:22.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f2980e6d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c19a7a0 con 0x7f297c0ff880 2023-09-10T00:33:22.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f2979d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f297c1001a0 0x7f297c1959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:22.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f2979d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f297c1001a0 0x7f297c1959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45196/0 (socket says 172.21.15.44:45196) 2023-09-10T00:33:22.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f2979d9b700 1 -- 172.21.15.44:0/1121985431 learned_addr learned my addr 172.21.15.44:0/1121985431 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:22.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f297ad9d700 1 --2- 172.21.15.44:0/1121985431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f297c101840 0x7f297c199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:22.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f297a59c700 1 --2- 172.21.15.44:0/1121985431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f297c0ff880 0x7f297c1954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:22.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f2979d9b700 1 -- 172.21.15.44:0/1121985431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f297c0ff880 msgr2=0x7f297c1954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:22.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f2979d9b700 1 --2- 172.21.15.44:0/1121985431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f297c0ff880 0x7f297c1954b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f2979d9b700 1 -- 172.21.15.44:0/1121985431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f297c101840 msgr2=0x7f297c199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:22.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f2979d9b700 1 --2- 172.21.15.44:0/1121985431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f297c101840 0x7f297c199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.087+0000 7f2979d9b700 1 -- 172.21.15.44:0/1121985431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29700095f0 con 0x7f297c1001a0 2023-09-10T00:33:22.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.088+0000 7f2979d9b700 1 --2- 172.21.15.44:0/1121985431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f297c1001a0 0x7f297c1959f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2970000c00 tx=0x7f2970004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:22.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.088+0000 7f296b7fe700 1 -- 172.21.15.44:0/1121985431 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297001d070 con 0x7f297c1001a0 2023-09-10T00:33:22.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.088+0000 7f296b7fe700 1 -- 172.21.15.44:0/1121985431 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(23 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f2970003c70 con 0x7f297c1001a0 2023-09-10T00:33:22.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.088+0000 7f296b7fe700 1 -- 172.21.15.44:0/1121985431 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970022d30 con 0x7f297c1001a0 2023-09-10T00:33:22.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.088+0000 7f2980e6d700 1 -- 172.21.15.44:0/1121985431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297c19aa20 con 0x7f297c1001a0 2023-09-10T00:33:22.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.088+0000 7f2980e6d700 1 -- 172.21.15.44:0/1121985431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297c1a22e0 con 0x7f297c1001a0 2023-09-10T00:33:22.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.089+0000 7f296b7fe700 1 -- 172.21.15.44:0/1121985431 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f297000f460 con 0x7f297c1001a0 2023-09-10T00:33:22.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.089+0000 7f2980e6d700 1 -- 172.21.15.44:0/1121985431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297c107bb0 con 0x7f297c1001a0 2023-09-10T00:33:22.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.089+0000 7f296b7fe700 1 --2- 172.21.15.44:0/1121985431 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2964038630 0x7f296403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:22.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.089+0000 7f296b7fe700 1 -- 172.21.15.44:0/1121985431 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f297004c4a0 con 0x7f297c1001a0 2023-09-10T00:33:22.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.090+0000 7f297a59c700 1 --2- 172.21.15.44:0/1121985431 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2964038630 0x7f296403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:22.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.090+0000 7f297a59c700 1 --2- 172.21.15.44:0/1121985431 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2964038630 0x7f296403aaf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f296c009c60 tx=0x7f296c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:22.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.095+0000 7f296b7fe700 1 -- 172.21.15.44:0/1121985431 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2970016580 con 0x7f297c1001a0 2023-09-10T00:33:22.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.310+0000 7f2980e6d700 1 -- 172.21.15.44:0/1121985431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7f297c1968b0 con 0x7f297c1001a0 2023-09-10T00:33:22.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.313+0000 7f296b7fe700 1 -- 172.21.15.44:0/1121985431 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7f2970027070 con 0x7f297c1001a0 2023-09-10T00:33:22.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.314+0000 7f296b7fe700 1 -- 172.21.15.44:0/1121985431 <== mon.0 v2:172.21.15.44:3300/0 8 ==== config(24 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f2970022e90 con 0x7f297c1001a0 2023-09-10T00:33:22.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.316+0000 7f2980e6d700 1 -- 172.21.15.44:0/1121985431 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2964038630 msgr2=0x7f296403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:22.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.316+0000 7f2980e6d700 1 --2- 172.21.15.44:0/1121985431 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2964038630 0x7f296403aaf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f296c009c60 tx=0x7f296c009260 comp rx=0 tx=0).stop 2023-09-10T00:33:22.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.316+0000 7f2980e6d700 1 -- 172.21.15.44:0/1121985431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f297c1001a0 msgr2=0x7f297c1959f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:22.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.316+0000 7f2980e6d700 1 --2- 172.21.15.44:0/1121985431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f297c1001a0 0x7f297c1959f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2970000c00 tx=0x7f2970004230 comp rx=0 tx=0).stop 2023-09-10T00:33:22.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.316+0000 7f2980e6d700 1 -- 172.21.15.44:0/1121985431 shutdown_connections 2023-09-10T00:33:22.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.316+0000 7f2980e6d700 1 --2- 172.21.15.44:0/1121985431 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2964038630 0x7f296403aaf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.316+0000 7f2980e6d700 1 --2- 172.21.15.44:0/1121985431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f297c101840 0x7f297c199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.316+0000 7f2980e6d700 1 --2- 172.21.15.44:0/1121985431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f297c0ff880 0x7f297c1954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.316+0000 7f2980e6d700 1 --2- 172.21.15.44:0/1121985431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f297c1001a0 0x7f297c1959f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.316+0000 7f2980e6d700 1 -- 172.21.15.44:0/1121985431 >> 172.21.15.44:0/1121985431 conn(0x7f297c0765f0 msgr2=0x7f297c0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:22.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.316+0000 7f2980e6d700 1 -- 172.21.15.44:0/1121985431 shutdown_connections 2023-09-10T00:33:22.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.316+0000 7f2980e6d700 1 -- 172.21.15.44:0/1121985431 wait complete. 2023-09-10T00:33:22.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.365+0000 7f15777b3700 1 -- 172.21.15.44:0/3328401568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1570104530 msgr2=0x7f1570104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.365+0000 7f15777b3700 1 --2- 172.21.15.44:0/3328401568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1570104530 0x7f1570104910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f156c009990 tx=0x7f156c009ca0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.365+0000 7f15777b3700 1 -- 172.21.15.44:0/3328401568 shutdown_connections 2023-09-10T00:33:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.365+0000 7f15777b3700 1 --2- 172.21.15.44:0/3328401568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15700ffee0 0x7f157010c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.365+0000 7f15777b3700 1 --2- 172.21.15.44:0/3328401568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1570104530 0x7f1570104910 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.365+0000 7f15777b3700 1 --2- 172.21.15.44:0/3328401568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15700ff520 0x7f15700ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.365+0000 7f15777b3700 1 -- 172.21.15.44:0/3328401568 >> 172.21.15.44:0/3328401568 conn(0x7f1570076600 msgr2=0x7f1570076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.365+0000 7f15777b3700 1 -- 172.21.15.44:0/3328401568 shutdown_connections 2023-09-10T00:33:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.365+0000 7f15777b3700 1 -- 172.21.15.44:0/3328401568 wait complete. 2023-09-10T00:33:22.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.366+0000 7f15777b3700 1 Processor -- start 2023-09-10T00:33:22.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.366+0000 7f15777b3700 1 -- start start 2023-09-10T00:33:22.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.366+0000 7f15777b3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15700ff520 0x7f1570195080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:22.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.366+0000 7f15777b3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15700ffee0 0x7f15701955c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:22.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.366+0000 7f15777b3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1570104530 0x7f1570195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:22.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.366+0000 7f15777b3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15701a48c0 con 0x7f15700ff520 2023-09-10T00:33:22.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.366+0000 7f15777b3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15701a4a30 con 0x7f15700ffee0 2023-09-10T00:33:22.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.366+0000 7f15777b3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15701a4bf0 con 0x7f1570104530 2023-09-10T00:33:22.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.367+0000 7f157554f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15700ff520 0x7f1570195080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:22.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.367+0000 7f157554f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15700ff520 0x7f1570195080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45208/0 (socket says 172.21.15.44:45208) 2023-09-10T00:33:22.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.367+0000 7f157554f700 1 -- 172.21.15.44:0/345041586 learned_addr learned my addr 172.21.15.44:0/345041586 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:22.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.367+0000 7f1574d4e700 1 --2- 172.21.15.44:0/345041586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15700ffee0 0x7f15701955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:22.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.367+0000 7f1575d50700 1 --2- 172.21.15.44:0/345041586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1570104530 0x7f1570195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:22.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.367+0000 7f157554f700 1 -- 172.21.15.44:0/345041586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1570104530 msgr2=0x7f1570195b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:22.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.367+0000 7f157554f700 1 --2- 172.21.15.44:0/345041586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1570104530 0x7f1570195b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.367+0000 7f157554f700 1 -- 172.21.15.44:0/345041586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15700ffee0 msgr2=0x7f15701955c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:22.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.367+0000 7f157554f700 1 --2- 172.21.15.44:0/345041586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15700ffee0 0x7f15701955c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.367+0000 7f157554f700 1 -- 172.21.15.44:0/345041586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0095f0 con 0x7f15700ff520 2023-09-10T00:33:22.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.367+0000 7f157554f700 1 --2- 172.21.15.44:0/345041586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15700ff520 0x7f1570195080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f156c000c00 tx=0x7f156c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:22.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.367+0000 7f15667fc700 1 -- 172.21.15.44:0/345041586 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c01d070 con 0x7f15700ff520 2023-09-10T00:33:22.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.367+0000 7f15777b3700 1 -- 172.21.15.44:0/345041586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15701a4e70 con 0x7f15700ff520 2023-09-10T00:33:22.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.367+0000 7f15667fc700 1 -- 172.21.15.44:0/345041586 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(24 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f156c00fbe0 con 0x7f15700ff520 2023-09-10T00:33:22.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.367+0000 7f15667fc700 1 -- 172.21.15.44:0/345041586 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c022c70 con 0x7f15700ff520 2023-09-10T00:33:22.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.367+0000 7f15777b3700 1 -- 172.21.15.44:0/345041586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15701a5280 con 0x7f15700ff520 2023-09-10T00:33:22.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.368+0000 7f15667fc700 1 -- 172.21.15.44:0/345041586 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f156c02c440 con 0x7f15700ff520 2023-09-10T00:33:22.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.369+0000 7f15777b3700 1 -- 172.21.15.44:0/345041586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157004e300 con 0x7f15700ff520 2023-09-10T00:33:22.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.369+0000 7f15667fc700 1 --2- 172.21.15.44:0/345041586 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f155c038630 0x7f155c03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:22.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.369+0000 7f15667fc700 1 -- 172.21.15.44:0/345041586 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f156c04c510 con 0x7f15700ff520 2023-09-10T00:33:22.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.369+0000 7f1574d4e700 1 --2- 172.21.15.44:0/345041586 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f155c038630 0x7f155c03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:22.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.369+0000 7f1574d4e700 1 --2- 172.21.15.44:0/345041586 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f155c038630 0x7f155c03aaf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1570068a40 tx=0x7f156000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:22.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.374+0000 7f15667fc700 1 -- 172.21.15.44:0/345041586 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f156c016990 con 0x7f15700ff520 2023-09-10T00:33:22.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.601+0000 7f15777b3700 1 -- 172.21.15.44:0/345041586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) v1 -- 0x7f157010f280 con 0x7f15700ff520 2023-09-10T00:33:22.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.607+0000 7f15667fc700 1 -- 172.21.15.44:0/345041586 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v17)=0 v17) v1 ==== 127+0+0 (secure 0 0 0) 0x7f156c027030 con 0x7f15700ff520 2023-09-10T00:33:22.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.610+0000 7f15777b3700 1 -- 172.21.15.44:0/345041586 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f155c038630 msgr2=0x7f155c03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:22.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.610+0000 7f15777b3700 1 --2- 172.21.15.44:0/345041586 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f155c038630 0x7f155c03aaf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1570068a40 tx=0x7f156000a380 comp rx=0 tx=0).stop 2023-09-10T00:33:22.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.610+0000 7f15777b3700 1 -- 172.21.15.44:0/345041586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15700ff520 msgr2=0x7f1570195080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:22.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.610+0000 7f15777b3700 1 --2- 172.21.15.44:0/345041586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15700ff520 0x7f1570195080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f156c000c00 tx=0x7f156c005740 comp rx=0 tx=0).stop 2023-09-10T00:33:22.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.610+0000 7f15777b3700 1 -- 172.21.15.44:0/345041586 shutdown_connections 2023-09-10T00:33:22.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.610+0000 7f15777b3700 1 --2- 172.21.15.44:0/345041586 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f155c038630 0x7f155c03aaf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.610+0000 7f15777b3700 1 --2- 172.21.15.44:0/345041586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1570104530 0x7f1570195b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.610+0000 7f15777b3700 1 --2- 172.21.15.44:0/345041586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15700ff520 0x7f1570195080 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.610+0000 7f15777b3700 1 --2- 172.21.15.44:0/345041586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15700ffee0 0x7f15701955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.610+0000 7f15777b3700 1 -- 172.21.15.44:0/345041586 >> 172.21.15.44:0/345041586 conn(0x7f1570076600 msgr2=0x7f15700fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:22.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.610+0000 7f15777b3700 1 -- 172.21.15.44:0/345041586 shutdown_connections 2023-09-10T00:33:22.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.610+0000 7f15777b3700 1 -- 172.21.15.44:0/345041586 wait complete. 2023-09-10T00:33:22.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.661+0000 7ff1a1ee9700 1 -- 172.21.15.44:0/606825569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff19c068120 msgr2=0x7ff19c0685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:22.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.661+0000 7ff1a1ee9700 1 --2- 172.21.15.44:0/606825569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff19c068120 0x7ff19c0685a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff190009990 tx=0x7ff190009ca0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.661+0000 7ff1a1ee9700 1 -- 172.21.15.44:0/606825569 shutdown_connections 2023-09-10T00:33:22.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.661+0000 7ff1a1ee9700 1 --2- 172.21.15.44:0/606825569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff19c068ae0 0x7ff19c10d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.661+0000 7ff1a1ee9700 1 --2- 172.21.15.44:0/606825569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19c106770 0x7ff19c106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.661+0000 7ff1a1ee9700 1 --2- 172.21.15.44:0/606825569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff19c068120 0x7ff19c0685a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.661+0000 7ff1a1ee9700 1 -- 172.21.15.44:0/606825569 >> 172.21.15.44:0/606825569 conn(0x7ff19c076600 msgr2=0x7ff19c076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:22.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.661+0000 7ff1a1ee9700 1 -- 172.21.15.44:0/606825569 shutdown_connections 2023-09-10T00:33:22.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.662+0000 7ff1a1ee9700 1 -- 172.21.15.44:0/606825569 wait complete. 2023-09-10T00:33:22.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.662+0000 7ff1a1ee9700 1 Processor -- start 2023-09-10T00:33:22.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.662+0000 7ff1a1ee9700 1 -- start start 2023-09-10T00:33:22.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.662+0000 7ff1a1ee9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff19c068120 0x7ff19c0ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:22.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.662+0000 7ff1a1ee9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19c068ae0 0x7ff19c0ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:22.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff1a1ee9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff19c106770 0x7ff19c100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:22.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff1a1ee9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c1a49e0 con 0x7ff19c068120 2023-09-10T00:33:22.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff1a1ee9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c1a4b50 con 0x7ff19c106770 2023-09-10T00:33:22.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff1a1ee9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c1a4d10 con 0x7ff19c068ae0 2023-09-10T00:33:22.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff19b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff19c068120 0x7ff19c0ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:22.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff19b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff19c068120 0x7ff19c0ff5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45222/0 (socket says 172.21.15.44:45222) 2023-09-10T00:33:22.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff19b7fe700 1 -- 172.21.15.44:0/1771296162 learned_addr learned my addr 172.21.15.44:0/1771296162 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:22.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff19affd700 1 --2- 172.21.15.44:0/1771296162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19c068ae0 0x7ff19c0ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:22.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff19bfff700 1 --2- 172.21.15.44:0/1771296162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff19c106770 0x7ff19c100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:22.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff19b7fe700 1 -- 172.21.15.44:0/1771296162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19c068ae0 msgr2=0x7ff19c0ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:22.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff19b7fe700 1 --2- 172.21.15.44:0/1771296162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19c068ae0 0x7ff19c0ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff19b7fe700 1 -- 172.21.15.44:0/1771296162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff19c106770 msgr2=0x7ff19c100070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:22.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff19b7fe700 1 --2- 172.21.15.44:0/1771296162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff19c106770 0x7ff19c100070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff19b7fe700 1 -- 172.21.15.44:0/1771296162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1900095f0 con 0x7ff19c068120 2023-09-10T00:33:22.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff19bfff700 1 --2- 172.21.15.44:0/1771296162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff19c106770 0x7ff19c100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:33:22.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff19b7fe700 1 --2- 172.21.15.44:0/1771296162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff19c068120 0x7ff19c0ff5b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00d950 tx=0x7ff18c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:22.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff198ff9700 1 -- 172.21.15.44:0/1771296162 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c009940 con 0x7ff19c068120 2023-09-10T00:33:22.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff198ff9700 1 -- 172.21.15.44:0/1771296162 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(24 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff18c010460 con 0x7ff19c068120 2023-09-10T00:33:22.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff1a1ee9700 1 -- 172.21.15.44:0/1771296162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c1a4ff0 con 0x7ff19c068120 2023-09-10T00:33:22.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff198ff9700 1 -- 172.21.15.44:0/1771296162 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c010ea0 con 0x7ff19c068120 2023-09-10T00:33:22.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.663+0000 7ff1a1ee9700 1 -- 172.21.15.44:0/1771296162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c1a5460 con 0x7ff19c068120 2023-09-10T00:33:22.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.664+0000 7ff198ff9700 1 -- 172.21.15.44:0/1771296162 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7ff18c00f620 con 0x7ff19c068120 2023-09-10T00:33:22.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.665+0000 7ff1a1ee9700 1 -- 172.21.15.44:0/1771296162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19c04e300 con 0x7ff19c068120 2023-09-10T00:33:22.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.665+0000 7ff198ff9700 1 --2- 172.21.15.44:0/1771296162 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff184038630 0x7ff18403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:22.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.665+0000 7ff198ff9700 1 -- 172.21.15.44:0/1771296162 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff18c04ad80 con 0x7ff19c068120 2023-09-10T00:33:22.673 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.665+0000 7ff19affd700 1 --2- 172.21.15.44:0/1771296162 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff184038630 0x7ff18403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:22.673 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.665+0000 7ff19affd700 1 --2- 172.21.15.44:0/1771296162 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff184038630 0x7ff18403aaf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff19c100da0 tx=0x7ff190005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:22.673 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.670+0000 7ff198ff9700 1 -- 172.21.15.44:0/1771296162 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff18c0228f0 con 0x7ff19c068120 2023-09-10T00:33:22.718 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:22.869 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:22.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.899+0000 7ff1a1ee9700 1 -- 172.21.15.44:0/1771296162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7ff19c110480 con 0x7ff19c068120 2023-09-10T00:33:22.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.902+0000 7ff198ff9700 1 -- 172.21.15.44:0/1771296162 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v18)=0 v18) v1 ==== 123+0+0 (secure 0 0 0) 0x7ff18c022300 con 0x7ff19c068120 2023-09-10T00:33:22.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.903+0000 7ff198ff9700 1 -- 172.21.15.44:0/1771296162 <== mon.0 v2:172.21.15.44:3300/0 8 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff18c04e030 con 0x7ff19c068120 2023-09-10T00:33:22.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.904+0000 7ff1a1ee9700 1 -- 172.21.15.44:0/1771296162 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff184038630 msgr2=0x7ff18403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:22.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.904+0000 7ff1a1ee9700 1 --2- 172.21.15.44:0/1771296162 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff184038630 0x7ff18403aaf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff19c100da0 tx=0x7ff190005fd0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.904+0000 7ff1a1ee9700 1 -- 172.21.15.44:0/1771296162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff19c068120 msgr2=0x7ff19c0ff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:22.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.904+0000 7ff1a1ee9700 1 --2- 172.21.15.44:0/1771296162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff19c068120 0x7ff19c0ff5b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00d950 tx=0x7ff18c00dc60 comp rx=0 tx=0).stop 2023-09-10T00:33:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.905+0000 7ff1a1ee9700 1 -- 172.21.15.44:0/1771296162 shutdown_connections 2023-09-10T00:33:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.905+0000 7ff1a1ee9700 1 --2- 172.21.15.44:0/1771296162 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff184038630 0x7ff18403aaf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.905+0000 7ff1a1ee9700 1 --2- 172.21.15.44:0/1771296162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff19c106770 0x7ff19c100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.905+0000 7ff1a1ee9700 1 --2- 172.21.15.44:0/1771296162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff19c068120 0x7ff19c0ff5b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.905+0000 7ff1a1ee9700 1 --2- 172.21.15.44:0/1771296162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19c068ae0 0x7ff19c0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:22.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.905+0000 7ff1a1ee9700 1 -- 172.21.15.44:0/1771296162 >> 172.21.15.44:0/1771296162 conn(0x7ff19c076600 msgr2=0x7ff19c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:22.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.905+0000 7ff1a1ee9700 1 -- 172.21.15.44:0/1771296162 shutdown_connections 2023-09-10T00:33:22.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:22.905+0000 7ff1a1ee9700 1 -- 172.21.15.44:0/1771296162 wait complete. 2023-09-10T00:33:23.507 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:23 smithi044 ceph-mon[98676]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:23.507 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:23 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1121985431' entity='client.admin' 2023-09-10T00:33:23.507 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:23 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/345041586' entity='client.admin' 2023-09-10T00:33:23.508 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:23 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1771296162' entity='client.admin' 2023-09-10T00:33:23.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:23 smithi099 ceph-mon[102692]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:23.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:23 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1121985431' entity='client.admin' 2023-09-10T00:33:23.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:23 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/345041586' entity='client.admin' 2023-09-10T00:33:23.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:23 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1771296162' entity='client.admin' 2023-09-10T00:33:23.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:23 smithi131 ceph-mon[103713]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:23.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:23 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1121985431' entity='client.admin' 2023-09-10T00:33:23.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:23 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/345041586' entity='client.admin' 2023-09-10T00:33:23.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:23 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1771296162' entity='client.admin' 2023-09-10T00:33:24.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.378+0000 7eff01ddb700 1 -- 172.21.15.44:0/2858317876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc0ff520 msgr2=0x7efefc0ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:24.380 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.378+0000 7eff01ddb700 1 --2- 172.21.15.44:0/2858317876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc0ff520 0x7efefc0ff9a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efef0009990 tx=0x7efef0009ca0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.380 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.378+0000 7eff01ddb700 1 -- 172.21.15.44:0/2858317876 shutdown_connections 2023-09-10T00:33:24.380 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.378+0000 7eff01ddb700 1 --2- 172.21.15.44:0/2858317876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0ffee0 0x7efefc10c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.380 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.378+0000 7eff01ddb700 1 --2- 172.21.15.44:0/2858317876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efefc104530 0x7efefc104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.378+0000 7eff01ddb700 1 --2- 172.21.15.44:0/2858317876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc0ff520 0x7efefc0ff9a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.378+0000 7eff01ddb700 1 -- 172.21.15.44:0/2858317876 >> 172.21.15.44:0/2858317876 conn(0x7efefc076600 msgr2=0x7efefc076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:24.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.378+0000 7eff01ddb700 1 -- 172.21.15.44:0/2858317876 shutdown_connections 2023-09-10T00:33:24.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.378+0000 7eff01ddb700 1 -- 172.21.15.44:0/2858317876 wait complete. 2023-09-10T00:33:24.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7eff01ddb700 1 Processor -- start 2023-09-10T00:33:24.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7eff01ddb700 1 -- start start 2023-09-10T00:33:24.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7eff01ddb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0ff520 0x7efefc101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:24.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7eff01ddb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efefc0ffee0 0x7efefc1016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:24.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7eff01ddb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc104530 0x7efefc101c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:24.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7eff01ddb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc1a49e0 con 0x7efefc104530 2023-09-10T00:33:24.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7eff01ddb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc1a4b50 con 0x7efefc0ff520 2023-09-10T00:33:24.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7eff01ddb700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc1a4d10 con 0x7efefc0ffee0 2023-09-10T00:33:24.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7efefbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc104530 0x7efefc101c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:24.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7efefbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc104530 0x7efefc101c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45238/0 (socket says 172.21.15.44:45238) 2023-09-10T00:33:24.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7efefbfff700 1 -- 172.21.15.44:0/527713701 learned_addr learned my addr 172.21.15.44:0/527713701 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:24.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7efefaffd700 1 --2- 172.21.15.44:0/527713701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efefc0ffee0 0x7efefc1016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:24.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7efefb7fe700 1 --2- 172.21.15.44:0/527713701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0ff520 0x7efefc101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:24.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7efefbfff700 1 -- 172.21.15.44:0/527713701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efefc0ffee0 msgr2=0x7efefc1016c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:24.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7efefbfff700 1 --2- 172.21.15.44:0/527713701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efefc0ffee0 0x7efefc1016c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7efefbfff700 1 -- 172.21.15.44:0/527713701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0ff520 msgr2=0x7efefc101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:24.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.379+0000 7efefbfff700 1 --2- 172.21.15.44:0/527713701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0ff520 0x7efefc101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.380+0000 7efefbfff700 1 -- 172.21.15.44:0/527713701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef00095f0 con 0x7efefc104530 2023-09-10T00:33:24.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.380+0000 7efefb7fe700 1 --2- 172.21.15.44:0/527713701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0ff520 0x7efefc101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:33:24.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.380+0000 7efefbfff700 1 --2- 172.21.15.44:0/527713701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc104530 0x7efefc101c40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efef400ab50 tx=0x7efef400ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:24.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.380+0000 7efef8ff9700 1 -- 172.21.15.44:0/527713701 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef400f070 con 0x7efefc104530 2023-09-10T00:33:24.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.380+0000 7efef8ff9700 1 -- 172.21.15.44:0/527713701 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efef4007a80 con 0x7efefc104530 2023-09-10T00:33:24.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.380+0000 7eff01ddb700 1 -- 172.21.15.44:0/527713701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efefc1a4ff0 con 0x7efefc104530 2023-09-10T00:33:24.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.380+0000 7efef8ff9700 1 -- 172.21.15.44:0/527713701 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4005280 con 0x7efefc104530 2023-09-10T00:33:24.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.380+0000 7eff01ddb700 1 -- 172.21.15.44:0/527713701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efefc1a54c0 con 0x7efefc104530 2023-09-10T00:33:24.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.381+0000 7eff01ddb700 1 -- 172.21.15.44:0/527713701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efefc04e300 con 0x7efefc104530 2023-09-10T00:33:24.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.381+0000 7efef8ff9700 1 -- 172.21.15.44:0/527713701 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7efef4004ad0 con 0x7efefc104530 2023-09-10T00:33:24.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.382+0000 7efef8ff9700 1 --2- 172.21.15.44:0/527713701 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7efee4038630 0x7efee403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:24.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.382+0000 7efef8ff9700 1 -- 172.21.15.44:0/527713701 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efef404a290 con 0x7efefc104530 2023-09-10T00:33:24.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.382+0000 7efefb7fe700 1 --2- 172.21.15.44:0/527713701 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7efee4038630 0x7efee403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:24.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.382+0000 7efefb7fe700 1 --2- 172.21.15.44:0/527713701 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7efee4038630 0x7efee403aaf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efefc069c10 tx=0x7efeec00a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:24.390 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:24.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.386+0000 7efef8ff9700 1 -- 172.21.15.44:0/527713701 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7efef400c620 con 0x7efefc104530 2023-09-10T00:33:24.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.614+0000 7eff01ddb700 1 -- 172.21.15.44:0/527713701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) v1 -- 0x7efefc1026b0 con 0x7efefc104530 2023-09-10T00:33:24.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.617+0000 7efef8ff9700 1 -- 172.21.15.44:0/527713701 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v19)=0 v19) v1 ==== 106+0+0 (secure 0 0 0) 0x7efef4017020 con 0x7efefc104530 2023-09-10T00:33:24.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.619+0000 7eff01ddb700 1 -- 172.21.15.44:0/527713701 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7efee4038630 msgr2=0x7efee403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:24.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.619+0000 7eff01ddb700 1 --2- 172.21.15.44:0/527713701 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7efee4038630 0x7efee403aaf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efefc069c10 tx=0x7efeec00a680 comp rx=0 tx=0).stop 2023-09-10T00:33:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.619+0000 7eff01ddb700 1 -- 172.21.15.44:0/527713701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc104530 msgr2=0x7efefc101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.619+0000 7eff01ddb700 1 --2- 172.21.15.44:0/527713701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc104530 0x7efefc101c40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efef400ab50 tx=0x7efef400ae60 comp rx=0 tx=0).stop 2023-09-10T00:33:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.619+0000 7eff01ddb700 1 -- 172.21.15.44:0/527713701 shutdown_connections 2023-09-10T00:33:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.619+0000 7eff01ddb700 1 --2- 172.21.15.44:0/527713701 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7efee4038630 0x7efee403aaf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.619+0000 7eff01ddb700 1 --2- 172.21.15.44:0/527713701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efefc104530 0x7efefc101c40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.619+0000 7eff01ddb700 1 --2- 172.21.15.44:0/527713701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0ff520 0x7efefc101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.619+0000 7eff01ddb700 1 --2- 172.21.15.44:0/527713701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efefc0ffee0 0x7efefc1016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.619+0000 7eff01ddb700 1 -- 172.21.15.44:0/527713701 >> 172.21.15.44:0/527713701 conn(0x7efefc076600 msgr2=0x7efefc1032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:24.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.619+0000 7eff01ddb700 1 -- 172.21.15.44:0/527713701 shutdown_connections 2023-09-10T00:33:24.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.619+0000 7eff01ddb700 1 -- 172.21.15.44:0/527713701 wait complete. 2023-09-10T00:33:24.646 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:24.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.747+0000 7fe3f94a8700 1 -- 172.21.15.44:0/3221467123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3f4104530 msgr2=0x7fe3f4104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:24.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.747+0000 7fe3f94a8700 1 --2- 172.21.15.44:0/3221467123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3f4104530 0x7fe3f4104910 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009990 tx=0x7fe3e4009ca0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.747+0000 7fe3f94a8700 1 -- 172.21.15.44:0/3221467123 shutdown_connections 2023-09-10T00:33:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.747+0000 7fe3f94a8700 1 --2- 172.21.15.44:0/3221467123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3f40ffee0 0x7fe3f410c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.747+0000 7fe3f94a8700 1 --2- 172.21.15.44:0/3221467123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3f4104530 0x7fe3f4104910 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.747+0000 7fe3f94a8700 1 --2- 172.21.15.44:0/3221467123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3f40ff520 0x7fe3f40ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.747+0000 7fe3f94a8700 1 -- 172.21.15.44:0/3221467123 >> 172.21.15.44:0/3221467123 conn(0x7fe3f4076600 msgr2=0x7fe3f4076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:24.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.747+0000 7fe3f94a8700 1 -- 172.21.15.44:0/3221467123 shutdown_connections 2023-09-10T00:33:24.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.748+0000 7fe3f94a8700 1 -- 172.21.15.44:0/3221467123 wait complete. 2023-09-10T00:33:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.748+0000 7fe3f94a8700 1 Processor -- start 2023-09-10T00:33:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.749+0000 7fe3f94a8700 1 -- start start 2023-09-10T00:33:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.749+0000 7fe3f94a8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3f40ff520 0x7fe3f4197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.749+0000 7fe3f94a8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3f40ffee0 0x7fe3f41979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.749+0000 7fe3f94a8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3f4104530 0x7fe3f41a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:24.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.749+0000 7fe3f94a8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f41a2a30 con 0x7fe3f4104530 2023-09-10T00:33:24.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.749+0000 7fe3f94a8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f41a2b70 con 0x7fe3f40ffee0 2023-09-10T00:33:24.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.749+0000 7fe3f94a8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f41a2cb0 con 0x7fe3f40ff520 2023-09-10T00:33:24.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.749+0000 7fe3f37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3f4104530 0x7fe3f41a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:24.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.749+0000 7fe3f2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3f40ff520 0x7fe3f4197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.749+0000 7fe3f37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3f4104530 0x7fe3f41a24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45252/0 (socket says 172.21.15.44:45252) 2023-09-10T00:33:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.749+0000 7fe3f27fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3f40ffee0 0x7fe3f41979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.749+0000 7fe3f37fe700 1 -- 172.21.15.44:0/4088617940 learned_addr learned my addr 172.21.15.44:0/4088617940 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.750+0000 7fe3f27fc700 1 -- 172.21.15.44:0/4088617940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3f40ff520 msgr2=0x7fe3f4197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.750+0000 7fe3f27fc700 1 --2- 172.21.15.44:0/4088617940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3f40ff520 0x7fe3f4197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.750+0000 7fe3f27fc700 1 -- 172.21.15.44:0/4088617940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3f4104530 msgr2=0x7fe3f41a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.750+0000 7fe3f27fc700 1 --2- 172.21.15.44:0/4088617940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3f4104530 0x7fe3f41a24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.750+0000 7fe3f27fc700 1 -- 172.21.15.44:0/4088617940 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e40095f0 con 0x7fe3f40ffee0 2023-09-10T00:33:24.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.750+0000 7fe3f37fe700 1 --2- 172.21.15.44:0/4088617940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3f4104530 0x7fe3f41a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:33:24.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.750+0000 7fe3f2ffd700 1 --2- 172.21.15.44:0/4088617940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3f40ff520 0x7fe3f4197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:33:24.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.750+0000 7fe3f27fc700 1 --2- 172.21.15.44:0/4088617940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3f40ffee0 0x7fe3f41979b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800e9f0 tx=0x7fe3e800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:24.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.750+0000 7fe3dbfff700 1 -- 172.21.15.44:0/4088617940 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e800c9c0 con 0x7fe3f40ffee0 2023-09-10T00:33:24.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.750+0000 7fe3f94a8700 1 -- 172.21.15.44:0/4088617940 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f41a2f10 con 0x7fe3f40ffee0 2023-09-10T00:33:24.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.750+0000 7fe3dbfff700 1 -- 172.21.15.44:0/4088617940 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3e8010430 con 0x7fe3f40ffee0 2023-09-10T00:33:24.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.750+0000 7fe3dbfff700 1 -- 172.21.15.44:0/4088617940 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8018680 con 0x7fe3f40ffee0 2023-09-10T00:33:24.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.750+0000 7fe3f94a8700 1 -- 172.21.15.44:0/4088617940 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f41a3380 con 0x7fe3f40ffee0 2023-09-10T00:33:24.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.751+0000 7fe3f94a8700 1 -- 172.21.15.44:0/4088617940 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f404e300 con 0x7fe3f40ffee0 2023-09-10T00:33:24.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.752+0000 7fe3dbfff700 1 -- 172.21.15.44:0/4088617940 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fe3e80105a0 con 0x7fe3f40ffee0 2023-09-10T00:33:24.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.752+0000 7fe3dbfff700 1 --2- 172.21.15.44:0/4088617940 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe3dc038630 0x7fe3dc03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:24.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.752+0000 7fe3dbfff700 1 -- 172.21.15.44:0/4088617940 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe3e8014070 con 0x7fe3f40ffee0 2023-09-10T00:33:24.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.752+0000 7fe3f2ffd700 1 --2- 172.21.15.44:0/4088617940 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe3dc038630 0x7fe3dc03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:24.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.752+0000 7fe3f2ffd700 1 --2- 172.21.15.44:0/4088617940 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe3dc038630 0x7fe3dc03aaf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4006010 tx=0x7fe3e400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.757+0000 7fe3dbfff700 1 -- 172.21.15.44:0/4088617940 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe3e80179c0 con 0x7fe3f40ffee0 2023-09-10T00:33:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.969+0000 7fe3f94a8700 1 -- 172.21.15.44:0/4088617940 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) v1 -- 0x7fe3f4068710 con 0x7fe3f40ffee0 2023-09-10T00:33:24.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.976+0000 7fe3dbfff700 1 -- 172.21.15.44:0/4088617940 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v20)=0 v20) v1 ==== 116+0+0 (secure 0 0 0) 0x7fe3e8026030 con 0x7fe3f40ffee0 2023-09-10T00:33:24.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.978+0000 7fe3f94a8700 1 -- 172.21.15.44:0/4088617940 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe3dc038630 msgr2=0x7fe3dc03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:24.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.978+0000 7fe3f94a8700 1 --2- 172.21.15.44:0/4088617940 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe3dc038630 0x7fe3dc03aaf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4006010 tx=0x7fe3e400b260 comp rx=0 tx=0).stop 2023-09-10T00:33:24.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.978+0000 7fe3f94a8700 1 -- 172.21.15.44:0/4088617940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3f40ffee0 msgr2=0x7fe3f41979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:24.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.978+0000 7fe3f94a8700 1 --2- 172.21.15.44:0/4088617940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3f40ffee0 0x7fe3f41979b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800e9f0 tx=0x7fe3e800ed00 comp rx=0 tx=0).stop 2023-09-10T00:33:24.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.978+0000 7fe3f94a8700 1 -- 172.21.15.44:0/4088617940 shutdown_connections 2023-09-10T00:33:24.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.978+0000 7fe3f94a8700 1 --2- 172.21.15.44:0/4088617940 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe3dc038630 0x7fe3dc03aaf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.978+0000 7fe3f94a8700 1 --2- 172.21.15.44:0/4088617940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3f4104530 0x7fe3f41a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.978+0000 7fe3f94a8700 1 --2- 172.21.15.44:0/4088617940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3f40ff520 0x7fe3f4197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.978+0000 7fe3f94a8700 1 --2- 172.21.15.44:0/4088617940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3f40ffee0 0x7fe3f41979b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:24.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.978+0000 7fe3f94a8700 1 -- 172.21.15.44:0/4088617940 >> 172.21.15.44:0/4088617940 conn(0x7fe3f4076600 msgr2=0x7fe3f40fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:24.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.978+0000 7fe3f94a8700 1 -- 172.21.15.44:0/4088617940 shutdown_connections 2023-09-10T00:33:24.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:24.978+0000 7fe3f94a8700 1 -- 172.21.15.44:0/4088617940 wait complete. 2023-09-10T00:33:25.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.051+0000 7f7b95e35700 1 -- 172.21.15.44:0/2733538379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b90106610 msgr2=0x7f7b901069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:25.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.051+0000 7f7b95e35700 1 --2- 172.21.15.44:0/2733538379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b90106610 0x7f7b901069f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009990 tx=0x7f7b80009ca0 comp rx=0 tx=0).stop 2023-09-10T00:33:25.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.051+0000 7f7b95e35700 1 -- 172.21.15.44:0/2733538379 shutdown_connections 2023-09-10T00:33:25.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.051+0000 7f7b95e35700 1 --2- 172.21.15.44:0/2733538379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b900ff570 0x7f7b9010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:25.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.051+0000 7f7b95e35700 1 --2- 172.21.15.44:0/2733538379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b90106610 0x7f7b901069f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:25.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.051+0000 7f7b95e35700 1 --2- 172.21.15.44:0/2733538379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b900febb0 0x7f7b900ff030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:25.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.051+0000 7f7b95e35700 1 -- 172.21.15.44:0/2733538379 >> 172.21.15.44:0/2733538379 conn(0x7f7b90075a70 msgr2=0x7f7b900fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:25.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.051+0000 7f7b95e35700 1 -- 172.21.15.44:0/2733538379 shutdown_connections 2023-09-10T00:33:25.054 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.051+0000 7f7b95e35700 1 -- 172.21.15.44:0/2733538379 wait complete. 2023-09-10T00:33:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.052+0000 7f7b95e35700 1 Processor -- start 2023-09-10T00:33:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.052+0000 7f7b95e35700 1 -- start start 2023-09-10T00:33:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b95e35700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b900febb0 0x7f7b901966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:25.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b95e35700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900ff570 0x7f7b90196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:25.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b95e35700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b90106610 0x7f7b90199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:25.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b95e35700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9019a5f0 con 0x7f7b900ff570 2023-09-10T00:33:25.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b95e35700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9019a760 con 0x7f7b900febb0 2023-09-10T00:33:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b95e35700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9019a920 con 0x7f7b90106610 2023-09-10T00:33:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b8effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900ff570 0x7f7b90196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b8effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900ff570 0x7f7b90196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45264/0 (socket says 172.21.15.44:45264) 2023-09-10T00:33:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b8effd700 1 -- 172.21.15.44:0/2759785589 learned_addr learned my addr 172.21.15.44:0/2759785589 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b8ffff700 1 --2- 172.21.15.44:0/2759785589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b90106610 0x7f7b90199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b8f7fe700 1 --2- 172.21.15.44:0/2759785589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b900febb0 0x7f7b901966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b8effd700 1 -- 172.21.15.44:0/2759785589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b90106610 msgr2=0x7f7b90199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b8effd700 1 --2- 172.21.15.44:0/2759785589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b90106610 0x7f7b90199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b8effd700 1 -- 172.21.15.44:0/2759785589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b900febb0 msgr2=0x7f7b901966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b8effd700 1 --2- 172.21.15.44:0/2759785589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b900febb0 0x7f7b901966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b8effd700 1 -- 172.21.15.44:0/2759785589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b800095f0 con 0x7f7b900ff570 2023-09-10T00:33:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b8f7fe700 1 --2- 172.21.15.44:0/2759785589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b900febb0 0x7f7b901966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:33:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.053+0000 7f7b8effd700 1 --2- 172.21.15.44:0/2759785589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900ff570 0x7f7b90196c30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400b530 tx=0x7f7b8400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.054+0000 7f7b8cff9700 1 -- 172.21.15.44:0/2759785589 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84011830 con 0x7f7b900ff570 2023-09-10T00:33:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.054+0000 7f7b8cff9700 1 -- 172.21.15.44:0/2759785589 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b84011e20 con 0x7f7b900ff570 2023-09-10T00:33:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.054+0000 7f7b8cff9700 1 -- 172.21.15.44:0/2759785589 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8400f2e0 con 0x7f7b900ff570 2023-09-10T00:33:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.054+0000 7f7b95e35700 1 -- 172.21.15.44:0/2759785589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9019ac00 con 0x7f7b900ff570 2023-09-10T00:33:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.054+0000 7f7b95e35700 1 -- 172.21.15.44:0/2759785589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b901a3550 con 0x7f7b900ff570 2023-09-10T00:33:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.055+0000 7f7b8cff9700 1 -- 172.21.15.44:0/2759785589 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f7b84011990 con 0x7f7b900ff570 2023-09-10T00:33:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.055+0000 7f7b95e35700 1 -- 172.21.15.44:0/2759785589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9004e300 con 0x7f7b900ff570 2023-09-10T00:33:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.055+0000 7f7b8cff9700 1 --2- 172.21.15.44:0/2759785589 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7b78038630 0x7f7b7803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.055+0000 7f7b8cff9700 1 -- 172.21.15.44:0/2759785589 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7b8404a940 con 0x7f7b900ff570 2023-09-10T00:33:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.055+0000 7f7b8f7fe700 1 --2- 172.21.15.44:0/2759785589 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7b78038630 0x7f7b7803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.055+0000 7f7b8f7fe700 1 --2- 172.21.15.44:0/2759785589 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7b78038630 0x7f7b7803aaf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000b5c0 tx=0x7f7b80005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.060+0000 7f7b8cff9700 1 -- 172.21.15.44:0/2759785589 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7b840105f0 con 0x7f7b900ff570 2023-09-10T00:33:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.285+0000 7f7b95e35700 1 -- 172.21.15.44:0/2759785589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f7b90197a50 con 0x7f7b900ff570 2023-09-10T00:33:25.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.288+0000 7f7b8cff9700 1 -- 172.21.15.44:0/2759785589 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v21)=0 v21) v1 ==== 120+0+0 (secure 0 0 0) 0x7f7b84021300 con 0x7f7b900ff570 2023-09-10T00:33:25.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.291+0000 7f7b95e35700 1 -- 172.21.15.44:0/2759785589 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7b78038630 msgr2=0x7f7b7803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:25.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.291+0000 7f7b95e35700 1 --2- 172.21.15.44:0/2759785589 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7b78038630 0x7f7b7803aaf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000b5c0 tx=0x7f7b80005fb0 comp rx=0 tx=0).stop 2023-09-10T00:33:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.291+0000 7f7b95e35700 1 -- 172.21.15.44:0/2759785589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900ff570 msgr2=0x7f7b90196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.291+0000 7f7b95e35700 1 --2- 172.21.15.44:0/2759785589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900ff570 0x7f7b90196c30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400b530 tx=0x7f7b8400b840 comp rx=0 tx=0).stop 2023-09-10T00:33:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.291+0000 7f7b95e35700 1 -- 172.21.15.44:0/2759785589 shutdown_connections 2023-09-10T00:33:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.291+0000 7f7b95e35700 1 --2- 172.21.15.44:0/2759785589 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7b78038630 0x7f7b7803aaf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.291+0000 7f7b95e35700 1 --2- 172.21.15.44:0/2759785589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b90106610 0x7f7b90199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:25.293 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.291+0000 7f7b95e35700 1 --2- 172.21.15.44:0/2759785589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b900febb0 0x7f7b901966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:25.293 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.291+0000 7f7b95e35700 1 --2- 172.21.15.44:0/2759785589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900ff570 0x7f7b90196c30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:25.293 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.291+0000 7f7b95e35700 1 -- 172.21.15.44:0/2759785589 >> 172.21.15.44:0/2759785589 conn(0x7f7b90075a70 msgr2=0x7f7b900fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:25.293 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.291+0000 7f7b95e35700 1 -- 172.21.15.44:0/2759785589 shutdown_connections 2023-09-10T00:33:25.293 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.291+0000 7f7b95e35700 1 -- 172.21.15.44:0/2759785589 wait complete. 2023-09-10T00:33:25.995 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:25 smithi044 ceph-mon[98676]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:25.995 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:25 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/527713701' entity='client.admin' 2023-09-10T00:33:25.995 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:25 smithi044 ceph-mon[98676]: from='client.? ' entity='client.admin' 2023-09-10T00:33:25.995 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:25 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2759785589' entity='client.admin' 2023-09-10T00:33:25.996 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:25 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3679836692' entity='client.admin' 2023-09-10T00:33:26.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:25 smithi131 ceph-mon[103713]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:26.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:25 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/527713701' entity='client.admin' 2023-09-10T00:33:26.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:25 smithi131 ceph-mon[103713]: from='client.? ' entity='client.admin' 2023-09-10T00:33:26.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:25 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2759785589' entity='client.admin' 2023-09-10T00:33:26.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:25 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3679836692' entity='client.admin' 2023-09-10T00:33:26.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:25 smithi099 ceph-mon[102692]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:26.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:25 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/527713701' entity='client.admin' 2023-09-10T00:33:26.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:25 smithi099 ceph-mon[102692]: from='client.? ' entity='client.admin' 2023-09-10T00:33:26.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:25 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2759785589' entity='client.admin' 2023-09-10T00:33:26.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:25 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3679836692' entity='client.admin' 2023-09-10T00:33:26.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.366+0000 7f5594b3d700 1 -- 172.21.15.44:0/3375920942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55900ff520 msgr2=0x7f55900ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:26.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.366+0000 7f5594b3d700 1 --2- 172.21.15.44:0/3375920942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55900ff520 0x7f55900ff9a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5580009990 tx=0x7f5580009ca0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.366+0000 7f5594b3d700 1 -- 172.21.15.44:0/3375920942 shutdown_connections 2023-09-10T00:33:26.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.366+0000 7f5594b3d700 1 --2- 172.21.15.44:0/3375920942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55900ffee0 0x7f559010c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.366+0000 7f5594b3d700 1 --2- 172.21.15.44:0/3375920942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5590104530 0x7f5590104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.366+0000 7f5594b3d700 1 --2- 172.21.15.44:0/3375920942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55900ff520 0x7f55900ff9a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.366+0000 7f5594b3d700 1 -- 172.21.15.44:0/3375920942 >> 172.21.15.44:0/3375920942 conn(0x7f5590076600 msgr2=0x7f5590076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:26.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.366+0000 7f5594b3d700 1 -- 172.21.15.44:0/3375920942 shutdown_connections 2023-09-10T00:33:26.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.366+0000 7f5594b3d700 1 -- 172.21.15.44:0/3375920942 wait complete. 2023-09-10T00:33:26.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.367+0000 7f5594b3d700 1 Processor -- start 2023-09-10T00:33:26.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.367+0000 7f5594b3d700 1 -- start start 2023-09-10T00:33:26.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.367+0000 7f5594b3d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55900ff520 0x7f559019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:26.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.367+0000 7f5594b3d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55900ffee0 0x7f559019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:26.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.367+0000 7f5594b3d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5590104530 0x7f55901985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:26.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.367+0000 7f5594b3d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5590198c60 con 0x7f55900ffee0 2023-09-10T00:33:26.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.367+0000 7f5594b3d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5590198dd0 con 0x7f55900ff520 2023-09-10T00:33:26.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.367+0000 7f5594b3d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5590198f90 con 0x7f5590104530 2023-09-10T00:33:26.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.367+0000 7f558dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55900ffee0 0x7f559019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:26.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.367+0000 7f558dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55900ffee0 0x7f559019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45290/0 (socket says 172.21.15.44:45290) 2023-09-10T00:33:26.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.367+0000 7f558dd9b700 1 -- 172.21.15.44:0/3379407907 learned_addr learned my addr 172.21.15.44:0/3379407907 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:26.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.367+0000 7f558e59c700 1 --2- 172.21.15.44:0/3379407907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55900ff520 0x7f559019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:26.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.367+0000 7f558ed9d700 1 --2- 172.21.15.44:0/3379407907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5590104530 0x7f55901985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:26.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.368+0000 7f558dd9b700 1 -- 172.21.15.44:0/3379407907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5590104530 msgr2=0x7f55901985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:26.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.368+0000 7f558dd9b700 1 --2- 172.21.15.44:0/3379407907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5590104530 0x7f55901985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.368+0000 7f558dd9b700 1 -- 172.21.15.44:0/3379407907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55900ff520 msgr2=0x7f559019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:26.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.368+0000 7f558dd9b700 1 --2- 172.21.15.44:0/3379407907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55900ff520 0x7f559019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.368+0000 7f558dd9b700 1 -- 172.21.15.44:0/3379407907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55800095f0 con 0x7f55900ffee0 2023-09-10T00:33:26.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.368+0000 7f558ed9d700 1 --2- 172.21.15.44:0/3379407907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5590104530 0x7f55901985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:33:26.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.368+0000 7f558dd9b700 1 --2- 172.21.15.44:0/3379407907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55900ffee0 0x7f559019eab0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5580006010 tx=0x7f5580004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:26.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.368+0000 7f557f7fe700 1 -- 172.21.15.44:0/3379407907 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558001d070 con 0x7f55900ffee0 2023-09-10T00:33:26.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.368+0000 7f557f7fe700 1 -- 172.21.15.44:0/3379407907 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5580004000 con 0x7f55900ffee0 2023-09-10T00:33:26.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.368+0000 7f557f7fe700 1 -- 172.21.15.44:0/3379407907 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5580022ac0 con 0x7f55900ffee0 2023-09-10T00:33:26.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.368+0000 7f5594b3d700 1 -- 172.21.15.44:0/3379407907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5590199210 con 0x7f55900ffee0 2023-09-10T00:33:26.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.368+0000 7f5594b3d700 1 -- 172.21.15.44:0/3379407907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55901a5560 con 0x7f55900ffee0 2023-09-10T00:33:26.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.369+0000 7f557f7fe700 1 -- 172.21.15.44:0/3379407907 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f5580003b70 con 0x7f55900ffee0 2023-09-10T00:33:26.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.369+0000 7f5594b3d700 1 -- 172.21.15.44:0/3379407907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f559004e300 con 0x7f55900ffee0 2023-09-10T00:33:26.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.369+0000 7f557f7fe700 1 --2- 172.21.15.44:0/3379407907 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5578038630 0x7f557803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:26.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.369+0000 7f557f7fe700 1 -- 172.21.15.44:0/3379407907 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f558004c6d0 con 0x7f55900ffee0 2023-09-10T00:33:26.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.369+0000 7f558e59c700 1 --2- 172.21.15.44:0/3379407907 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5578038630 0x7f557803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:26.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.370+0000 7f558e59c700 1 --2- 172.21.15.44:0/3379407907 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5578038630 0x7f557803aaf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5590069c10 tx=0x7f558400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:26.380 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.374+0000 7f557f7fe700 1 -- 172.21.15.44:0/3379407907 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f55800163a0 con 0x7f55900ffee0 2023-09-10T00:33:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.217+0000 7f5072f78700 1 -- 172.21.15.44:0/3027250203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f506c068ae0 msgr2=0x7f506c10d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.217+0000 7f5072f78700 1 --2- 172.21.15.44:0/3027250203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f506c068ae0 0x7f506c10d440 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f506400b0a0 tx=0x7f506400b3b0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.217+0000 7f5072f78700 1 -- 172.21.15.44:0/3027250203 shutdown_connections 2023-09-10T00:33:26.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.217+0000 7f5072f78700 1 --2- 172.21.15.44:0/3027250203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f506c068ae0 0x7f506c10d440 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.217+0000 7f5072f78700 1 --2- 172.21.15.44:0/3027250203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f506c106770 0x7f506c106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.217+0000 7f5072f78700 1 --2- 172.21.15.44:0/3027250203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c068120 0x7f506c0685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.217+0000 7f5072f78700 1 -- 172.21.15.44:0/3027250203 >> 172.21.15.44:0/3027250203 conn(0x7f506c076600 msgr2=0x7f506c076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:26.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.217+0000 7f5072f78700 1 -- 172.21.15.44:0/3027250203 shutdown_connections 2023-09-10T00:33:26.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.217+0000 7f5072f78700 1 -- 172.21.15.44:0/3027250203 wait complete. 2023-09-10T00:33:26.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.218+0000 7f5072f78700 1 Processor -- start 2023-09-10T00:33:26.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.218+0000 7f5072f78700 1 -- start start 2023-09-10T00:33:26.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.218+0000 7f5072f78700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f506c068120 0x7f506c0ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:26.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.218+0000 7f5072f78700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f506c068ae0 0x7f506c0ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:26.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.218+0000 7f5072f78700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c106770 0x7f506c100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:26.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.218+0000 7f5072f78700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c1a4b80 con 0x7f506c068ae0 2023-09-10T00:33:26.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.218+0000 7f5072f78700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c1a4cf0 con 0x7f506c068120 2023-09-10T00:33:26.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.218+0000 7f5072f78700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c1a4eb0 con 0x7f506c106770 2023-09-10T00:33:26.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.218+0000 7f506bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f506c068ae0 0x7f506c0ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:26.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.218+0000 7f506bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f506c068ae0 0x7f506c0ffaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45278/0 (socket says 172.21.15.44:45278) 2023-09-10T00:33:26.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.218+0000 7f506bfff700 1 -- 172.21.15.44:0/3679836692 learned_addr learned my addr 172.21.15.44:0/3679836692 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:26.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.218+0000 7f5071515700 1 --2- 172.21.15.44:0/3679836692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c106770 0x7f506c100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:26.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.218+0000 7f5070d14700 1 --2- 172.21.15.44:0/3679836692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f506c068120 0x7f506c0ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:26.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.219+0000 7f506bfff700 1 -- 172.21.15.44:0/3679836692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c106770 msgr2=0x7f506c100070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:26.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.219+0000 7f506bfff700 1 --2- 172.21.15.44:0/3679836692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c106770 0x7f506c100070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.219+0000 7f506bfff700 1 -- 172.21.15.44:0/3679836692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f506c068120 msgr2=0x7f506c0ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:26.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.219+0000 7f506bfff700 1 --2- 172.21.15.44:0/3679836692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f506c068120 0x7f506c0ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.219+0000 7f506bfff700 1 -- 172.21.15.44:0/3679836692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5064009d00 con 0x7f506c068ae0 2023-09-10T00:33:26.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.219+0000 7f506bfff700 1 --2- 172.21.15.44:0/3679836692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f506c068ae0 0x7f506c0ffaf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f505800e9a0 tx=0x7f505800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:26.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.219+0000 7f5069ffb700 1 -- 172.21.15.44:0/3679836692 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5058009790 con 0x7f506c068ae0 2023-09-10T00:33:26.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.219+0000 7f5069ffb700 1 -- 172.21.15.44:0/3679836692 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f505800cbd0 con 0x7f506c068ae0 2023-09-10T00:33:26.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.219+0000 7f5069ffb700 1 -- 172.21.15.44:0/3679836692 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5058018680 con 0x7f506c068ae0 2023-09-10T00:33:26.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.219+0000 7f5072f78700 1 -- 172.21.15.44:0/3679836692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f506c1a5190 con 0x7f506c068ae0 2023-09-10T00:33:26.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.219+0000 7f5072f78700 1 -- 172.21.15.44:0/3679836692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f506c1a5640 con 0x7f506c068ae0 2023-09-10T00:33:26.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.220+0000 7f5069ffb700 1 -- 172.21.15.44:0/3679836692 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f505800cd40 con 0x7f506c068ae0 2023-09-10T00:33:26.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.220+0000 7f5072f78700 1 -- 172.21.15.44:0/3679836692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f506c04e300 con 0x7f506c068ae0 2023-09-10T00:33:26.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.220+0000 7f5069ffb700 1 --2- 172.21.15.44:0/3679836692 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f505c038630 0x7f505c03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:26.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.220+0000 7f5069ffb700 1 -- 172.21.15.44:0/3679836692 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5058014070 con 0x7f506c068ae0 2023-09-10T00:33:26.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.220+0000 7f5070d14700 1 --2- 172.21.15.44:0/3679836692 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f505c038630 0x7f505c03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.221+0000 7f5070d14700 1 --2- 172.21.15.44:0/3679836692 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f505c038630 0x7f505c03aaf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5060005fd0 tx=0x7f5060005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.225+0000 7f5069ffb700 1 -- 172.21.15.44:0/3679836692 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f505801fd50 con 0x7f506c068ae0 2023-09-10T00:33:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.439+0000 7f5072f78700 1 -- 172.21.15.44:0/3679836692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) v1 -- 0x7f506c100ae0 con 0x7f506c068ae0 2023-09-10T00:33:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.442+0000 7f5069ffb700 1 -- 172.21.15.44:0/3679836692 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v22)=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5058029070 con 0x7f506c068ae0 2023-09-10T00:33:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.444+0000 7f5072f78700 1 -- 172.21.15.44:0/3679836692 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f505c038630 msgr2=0x7f505c03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.444+0000 7f5072f78700 1 --2- 172.21.15.44:0/3679836692 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f505c038630 0x7f505c03aaf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5060005fd0 tx=0x7f5060005f60 comp rx=0 tx=0).stop 2023-09-10T00:33:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.444+0000 7f5072f78700 1 -- 172.21.15.44:0/3679836692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f506c068ae0 msgr2=0x7f506c0ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.444+0000 7f5072f78700 1 --2- 172.21.15.44:0/3679836692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f506c068ae0 0x7f506c0ffaf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f505800e9a0 tx=0x7f505800ed60 comp rx=0 tx=0).stop 2023-09-10T00:33:26.480 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.444+0000 7f5072f78700 1 -- 172.21.15.44:0/3679836692 shutdown_connections 2023-09-10T00:33:26.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.444+0000 7f5072f78700 1 --2- 172.21.15.44:0/3679836692 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f505c038630 0x7f505c03aaf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.444+0000 7f5072f78700 1 --2- 172.21.15.44:0/3679836692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f506c106770 0x7f506c100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.444+0000 7f5072f78700 1 --2- 172.21.15.44:0/3679836692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f506c068120 0x7f506c0ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.444+0000 7f5072f78700 1 --2- 172.21.15.44:0/3679836692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f506c068ae0 0x7f506c0ffaf0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.444+0000 7f5072f78700 1 -- 172.21.15.44:0/3679836692 >> 172.21.15.44:0/3679836692 conn(0x7f506c076600 msgr2=0x7f506c104390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:26.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.444+0000 7f5072f78700 1 -- 172.21.15.44:0/3679836692 shutdown_connections 2023-09-10T00:33:26.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:25.444+0000 7f5072f78700 1 -- 172.21.15.44:0/3679836692 wait complete. 2023-09-10T00:33:26.588 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.587+0000 7f5594b3d700 1 -- 172.21.15.44:0/3379407907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) v1 -- 0x7f55901993a0 con 0x7f55900ffee0 2023-09-10T00:33:26.591 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.590+0000 7f557f7fe700 1 -- 172.21.15.44:0/3379407907 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v23)=0 v23) v1 ==== 121+0+0 (secure 0 0 0) 0x7f5580027030 con 0x7f55900ffee0 2023-09-10T00:33:26.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.593+0000 7f5594b3d700 1 -- 172.21.15.44:0/3379407907 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5578038630 msgr2=0x7f557803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:26.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.593+0000 7f5594b3d700 1 --2- 172.21.15.44:0/3379407907 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5578038630 0x7f557803aaf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5590069c10 tx=0x7f558400b410 comp rx=0 tx=0).stop 2023-09-10T00:33:26.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.593+0000 7f5594b3d700 1 -- 172.21.15.44:0/3379407907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55900ffee0 msgr2=0x7f559019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:26.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.593+0000 7f5594b3d700 1 --2- 172.21.15.44:0/3379407907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55900ffee0 0x7f559019eab0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5580006010 tx=0x7f5580004730 comp rx=0 tx=0).stop 2023-09-10T00:33:26.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.593+0000 7f5594b3d700 1 -- 172.21.15.44:0/3379407907 shutdown_connections 2023-09-10T00:33:26.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.593+0000 7f5594b3d700 1 --2- 172.21.15.44:0/3379407907 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5578038630 0x7f557803aaf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.593+0000 7f5594b3d700 1 --2- 172.21.15.44:0/3379407907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5590104530 0x7f55901985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.593+0000 7f5594b3d700 1 --2- 172.21.15.44:0/3379407907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55900ff520 0x7f559019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.593+0000 7f5594b3d700 1 --2- 172.21.15.44:0/3379407907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55900ffee0 0x7f559019eab0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:26.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.593+0000 7f5594b3d700 1 -- 172.21.15.44:0/3379407907 >> 172.21.15.44:0/3379407907 conn(0x7f5590076600 msgr2=0x7f55900fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:26.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.593+0000 7f5594b3d700 1 -- 172.21.15.44:0/3379407907 shutdown_connections 2023-09-10T00:33:26.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:26.593+0000 7f5594b3d700 1 -- 172.21.15.44:0/3379407907 wait complete. 2023-09-10T00:33:27.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:27 smithi044 ceph-mon[98676]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:27.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:27 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3379407907' entity='client.admin' 2023-09-10T00:33:27.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.809+0000 7efe03095700 1 -- 172.21.15.44:0/1168872607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdfc100890 msgr2=0x7efdfc100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:27.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.809+0000 7efe03095700 1 --2- 172.21.15.44:0/1168872607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdfc100890 0x7efdfc100d10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efde8009990 tx=0x7efde8009ca0 comp rx=0 tx=0).stop 2023-09-10T00:33:27.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.809+0000 7efe03095700 1 -- 172.21.15.44:0/1168872607 shutdown_connections 2023-09-10T00:33:27.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.809+0000 7efe03095700 1 --2- 172.21.15.44:0/1168872607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdfc101250 0x7efdfc10d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:27.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.809+0000 7efe03095700 1 --2- 172.21.15.44:0/1168872607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc106740 0x7efdfc106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:27.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.809+0000 7efe03095700 1 --2- 172.21.15.44:0/1168872607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdfc100890 0x7efdfc100d10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:27.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.809+0000 7efe03095700 1 -- 172.21.15.44:0/1168872607 >> 172.21.15.44:0/1168872607 conn(0x7efdfc0785f0 msgr2=0x7efdfc078a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:27.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.809+0000 7efe03095700 1 -- 172.21.15.44:0/1168872607 shutdown_connections 2023-09-10T00:33:27.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.809+0000 7efe03095700 1 -- 172.21.15.44:0/1168872607 wait complete. 2023-09-10T00:33:27.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.810+0000 7efe03095700 1 Processor -- start 2023-09-10T00:33:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.810+0000 7efe03095700 1 -- start start 2023-09-10T00:33:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.810+0000 7efe03095700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdfc100890 0x7efdfc196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.810+0000 7efe03095700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc101250 0x7efdfc196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.810+0000 7efe03095700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdfc106740 0x7efdfc199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.810+0000 7efe03095700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc19a5c0 con 0x7efdfc100890 2023-09-10T00:33:27.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.810+0000 7efe03095700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc19a730 con 0x7efdfc101250 2023-09-10T00:33:27.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.810+0000 7efe03095700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc19a8f0 con 0x7efdfc106740 2023-09-10T00:33:27.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.810+0000 7efe00e31700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdfc100890 0x7efdfc196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:27.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.810+0000 7efe00e31700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdfc100890 0x7efdfc196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45302/0 (socket says 172.21.15.44:45302) 2023-09-10T00:33:27.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.810+0000 7efe00e31700 1 -- 172.21.15.44:0/657071607 learned_addr learned my addr 172.21.15.44:0/657071607 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:27.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.810+0000 7efdfbfff700 1 --2- 172.21.15.44:0/657071607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc101250 0x7efdfc196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.810+0000 7efe00e31700 1 -- 172.21.15.44:0/657071607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdfc106740 msgr2=0x7efdfc199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.811+0000 7efe00e31700 1 --2- 172.21.15.44:0/657071607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdfc106740 0x7efdfc199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.811+0000 7efe00e31700 1 -- 172.21.15.44:0/657071607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc101250 msgr2=0x7efdfc196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.811+0000 7efe00e31700 1 --2- 172.21.15.44:0/657071607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc101250 0x7efdfc196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.811+0000 7efe00e31700 1 -- 172.21.15.44:0/657071607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde80095f0 con 0x7efdfc100890 2023-09-10T00:33:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.811+0000 7efe00e31700 1 --2- 172.21.15.44:0/657071607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdfc100890 0x7efdfc196710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efdf000c920 tx=0x7efdf000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:27.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.811+0000 7efdf9ffb700 1 -- 172.21.15.44:0/657071607 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0007b00 con 0x7efdfc100890 2023-09-10T00:33:27.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.811+0000 7efe03095700 1 -- 172.21.15.44:0/657071607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdfc19abd0 con 0x7efdfc100890 2023-09-10T00:33:27.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.811+0000 7efdf9ffb700 1 -- 172.21.15.44:0/657071607 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efdf000ce70 con 0x7efdfc100890 2023-09-10T00:33:27.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.811+0000 7efdf9ffb700 1 -- 172.21.15.44:0/657071607 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00053e0 con 0x7efdfc100890 2023-09-10T00:33:27.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.811+0000 7efe03095700 1 -- 172.21.15.44:0/657071607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc1a3520 con 0x7efdfc100890 2023-09-10T00:33:27.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.812+0000 7efdf9ffb700 1 -- 172.21.15.44:0/657071607 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7efdf0005580 con 0x7efdfc100890 2023-09-10T00:33:27.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.812+0000 7efdf9ffb700 1 --2- 172.21.15.44:0/657071607 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7efdec038630 0x7efdec03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:27.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.812+0000 7efdf9ffb700 1 -- 172.21.15.44:0/657071607 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efdf004ac70 con 0x7efdfc100890 2023-09-10T00:33:27.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.812+0000 7efe03095700 1 -- 172.21.15.44:0/657071607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdfc04e300 con 0x7efdfc100890 2023-09-10T00:33:27.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.812+0000 7efdfbfff700 1 --2- 172.21.15.44:0/657071607 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7efdec038630 0x7efdec03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:27.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.813+0000 7efdfbfff700 1 --2- 172.21.15.44:0/657071607 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7efdec038630 0x7efdec03aaf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efde800b5c0 tx=0x7efde8005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:27.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:27.818+0000 7efdf9ffb700 1 -- 172.21.15.44:0/657071607 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7efdf00173b0 con 0x7efdfc100890 2023-09-10T00:33:27.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:27 smithi099 ceph-mon[102692]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:27.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:27 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3379407907' entity='client.admin' 2023-09-10T00:33:28.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.031+0000 7efe03095700 1 -- 172.21.15.44:0/657071607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) v1 -- 0x7efdfc197a20 con 0x7efdfc100890 2023-09-10T00:33:28.034 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.034+0000 7efdf9ffb700 1 -- 172.21.15.44:0/657071607 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v24)=0 v24) v1 ==== 116+0+0 (secure 0 0 0) 0x7efdf003d070 con 0x7efdfc100890 2023-09-10T00:33:28.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.036+0000 7efe03095700 1 -- 172.21.15.44:0/657071607 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7efdec038630 msgr2=0x7efdec03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:28.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.036+0000 7efe03095700 1 --2- 172.21.15.44:0/657071607 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7efdec038630 0x7efdec03aaf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efde800b5c0 tx=0x7efde8005ee0 comp rx=0 tx=0).stop 2023-09-10T00:33:28.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.036+0000 7efe03095700 1 -- 172.21.15.44:0/657071607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdfc100890 msgr2=0x7efdfc196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:28.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.036+0000 7efe03095700 1 --2- 172.21.15.44:0/657071607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdfc100890 0x7efdfc196710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efdf000c920 tx=0x7efdf000cc30 comp rx=0 tx=0).stop 2023-09-10T00:33:28.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.036+0000 7efe03095700 1 -- 172.21.15.44:0/657071607 shutdown_connections 2023-09-10T00:33:28.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.037+0000 7efe03095700 1 --2- 172.21.15.44:0/657071607 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7efdec038630 0x7efdec03aaf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:28.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.037+0000 7efe03095700 1 --2- 172.21.15.44:0/657071607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdfc106740 0x7efdfc199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:28.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.037+0000 7efe03095700 1 --2- 172.21.15.44:0/657071607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdfc100890 0x7efdfc196710 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:28.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.037+0000 7efe03095700 1 --2- 172.21.15.44:0/657071607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc101250 0x7efdfc196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:28.039 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.037+0000 7efe03095700 1 -- 172.21.15.44:0/657071607 >> 172.21.15.44:0/657071607 conn(0x7efdfc0785f0 msgr2=0x7efdfc0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:28.039 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.037+0000 7efe03095700 1 -- 172.21.15.44:0/657071607 shutdown_connections 2023-09-10T00:33:28.039 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.037+0000 7efe03095700 1 -- 172.21.15.44:0/657071607 wait complete. 2023-09-10T00:33:28.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:27 smithi131 ceph-mon[103713]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:27 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3379407907' entity='client.admin' 2023-09-10T00:33:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.784+0000 7fda1addf700 1 -- 172.21.15.44:0/752032753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda14068120 msgr2=0x7fda140685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.784+0000 7fda1addf700 1 --2- 172.21.15.44:0/752032753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda14068120 0x7fda140685a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fda00009990 tx=0x7fda00009ca0 comp rx=0 tx=0).stop 2023-09-10T00:33:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.784+0000 7fda1addf700 1 -- 172.21.15.44:0/752032753 shutdown_connections 2023-09-10T00:33:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.784+0000 7fda1addf700 1 --2- 172.21.15.44:0/752032753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda14068ae0 0x7fda1410d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.784+0000 7fda1addf700 1 --2- 172.21.15.44:0/752032753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda14106770 0x7fda14106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.785+0000 7fda1addf700 1 --2- 172.21.15.44:0/752032753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda14068120 0x7fda140685a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.785+0000 7fda1addf700 1 -- 172.21.15.44:0/752032753 >> 172.21.15.44:0/752032753 conn(0x7fda14076600 msgr2=0x7fda14076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.785+0000 7fda1addf700 1 -- 172.21.15.44:0/752032753 shutdown_connections 2023-09-10T00:33:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.785+0000 7fda1addf700 1 -- 172.21.15.44:0/752032753 wait complete. 2023-09-10T00:33:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.785+0000 7fda1addf700 1 Processor -- start 2023-09-10T00:33:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.785+0000 7fda1addf700 1 -- start start 2023-09-10T00:33:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.785+0000 7fda1addf700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda14068120 0x7fda1419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:28.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.785+0000 7fda1addf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda14068ae0 0x7fda1419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:28.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.785+0000 7fda1addf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda14106770 0x7fda14198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:28.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.785+0000 7fda1addf700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda14198dd0 con 0x7fda14068ae0 2023-09-10T00:33:28.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.785+0000 7fda1addf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda14198f40 con 0x7fda14106770 2023-09-10T00:33:28.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.785+0000 7fda1addf700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda14199100 con 0x7fda14068120 2023-09-10T00:33:28.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.785+0000 7fda13fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda14068ae0 0x7fda1419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.785+0000 7fda13fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda14068ae0 0x7fda1419ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45316/0 (socket says 172.21.15.44:45316) 2023-09-10T00:33:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.786+0000 7fda13fff700 1 -- 172.21.15.44:0/1737128854 learned_addr learned my addr 172.21.15.44:0/1737128854 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.786+0000 7fda18b7b700 1 --2- 172.21.15.44:0/1737128854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda14068120 0x7fda1419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:28.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.786+0000 7fda13fff700 1 -- 172.21.15.44:0/1737128854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda14068120 msgr2=0x7fda1419e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:28.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.786+0000 7fda1937c700 1 --2- 172.21.15.44:0/1737128854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda14106770 0x7fda14198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:28.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.786+0000 7fda13fff700 1 --2- 172.21.15.44:0/1737128854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda14068120 0x7fda1419e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:28.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.786+0000 7fda13fff700 1 -- 172.21.15.44:0/1737128854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda14106770 msgr2=0x7fda14198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:28.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.786+0000 7fda13fff700 1 --2- 172.21.15.44:0/1737128854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda14106770 0x7fda14198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:28.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.786+0000 7fda13fff700 1 -- 172.21.15.44:0/1737128854 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda000095f0 con 0x7fda14068ae0 2023-09-10T00:33:28.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.786+0000 7fda18b7b700 1 --2- 172.21.15.44:0/1737128854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda14068120 0x7fda1419e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:33:28.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.786+0000 7fda1937c700 1 --2- 172.21.15.44:0/1737128854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda14106770 0x7fda14198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:33:28.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.786+0000 7fda13fff700 1 --2- 172.21.15.44:0/1737128854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda14068ae0 0x7fda1419ebc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda00005910 tx=0x7fda00009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:28.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.786+0000 7fda11ffb700 1 -- 172.21.15.44:0/1737128854 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0001d070 con 0x7fda14068ae0 2023-09-10T00:33:28.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.786+0000 7fda11ffb700 1 -- 172.21.15.44:0/1737128854 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda00003f40 con 0x7fda14068ae0 2023-09-10T00:33:28.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.786+0000 7fda1addf700 1 -- 172.21.15.44:0/1737128854 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda14199380 con 0x7fda14068ae0 2023-09-10T00:33:28.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.786+0000 7fda11ffb700 1 -- 172.21.15.44:0/1737128854 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0000fc60 con 0x7fda14068ae0 2023-09-10T00:33:28.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.786+0000 7fda1addf700 1 -- 172.21.15.44:0/1737128854 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda141a56b0 con 0x7fda14068ae0 2023-09-10T00:33:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.787+0000 7fda11ffb700 1 -- 172.21.15.44:0/1737128854 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fda0002b440 con 0x7fda14068ae0 2023-09-10T00:33:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.787+0000 7fda11ffb700 1 --2- 172.21.15.44:0/1737128854 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fda04038630 0x7fda0403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.787+0000 7fda11ffb700 1 -- 172.21.15.44:0/1737128854 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fda00003ab0 con 0x7fda14068ae0 2023-09-10T00:33:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.788+0000 7fda18b7b700 1 --2- 172.21.15.44:0/1737128854 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fda04038630 0x7fda0403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:28.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.788+0000 7fda1addf700 1 -- 172.21.15.44:0/1737128854 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1404e300 con 0x7fda14068ae0 2023-09-10T00:33:28.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.788+0000 7fda18b7b700 1 --2- 172.21.15.44:0/1737128854 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fda04038630 0x7fda0403aaf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fda08007900 tx=0x7fda08008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:28.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.792+0000 7fda11ffb700 1 -- 172.21.15.44:0/1737128854 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fda00026070 con 0x7fda14068ae0 2023-09-10T00:33:28.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:28.997+0000 7fda1addf700 1 -- 172.21.15.44:0/1737128854 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7fda141105d0 con 0x7fda14068ae0 2023-09-10T00:33:29.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.000+0000 7fda11ffb700 1 -- 172.21.15.44:0/1737128854 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v25)=0 v25) v1 ==== 121+0+0 (secure 0 0 0) 0x7fda000173f0 con 0x7fda14068ae0 2023-09-10T00:33:29.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.003+0000 7fda1addf700 1 -- 172.21.15.44:0/1737128854 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fda04038630 msgr2=0x7fda0403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.003+0000 7fda1addf700 1 --2- 172.21.15.44:0/1737128854 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fda04038630 0x7fda0403aaf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fda08007900 tx=0x7fda08008040 comp rx=0 tx=0).stop 2023-09-10T00:33:29.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.003+0000 7fda1addf700 1 -- 172.21.15.44:0/1737128854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda14068ae0 msgr2=0x7fda1419ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.003+0000 7fda1addf700 1 --2- 172.21.15.44:0/1737128854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda14068ae0 0x7fda1419ebc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda00005910 tx=0x7fda00009580 comp rx=0 tx=0).stop 2023-09-10T00:33:29.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.003+0000 7fda1addf700 1 -- 172.21.15.44:0/1737128854 shutdown_connections 2023-09-10T00:33:29.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.003+0000 7fda1addf700 1 --2- 172.21.15.44:0/1737128854 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fda04038630 0x7fda0403aaf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.003+0000 7fda1addf700 1 --2- 172.21.15.44:0/1737128854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda14106770 0x7fda14198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.003+0000 7fda1addf700 1 --2- 172.21.15.44:0/1737128854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda14068120 0x7fda1419e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.003+0000 7fda1addf700 1 --2- 172.21.15.44:0/1737128854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda14068ae0 0x7fda1419ebc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.003+0000 7fda1addf700 1 -- 172.21.15.44:0/1737128854 >> 172.21.15.44:0/1737128854 conn(0x7fda14076600 msgr2=0x7fda1410b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:29.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.003+0000 7fda1addf700 1 -- 172.21.15.44:0/1737128854 shutdown_connections 2023-09-10T00:33:29.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.003+0000 7fda1addf700 1 -- 172.21.15.44:0/1737128854 wait complete. 2023-09-10T00:33:29.141 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.140+0000 7fb865ec5700 1 -- 172.21.15.44:0/4016635534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb860100020 msgr2=0x7fb860100400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.140+0000 7fb865ec5700 1 --2- 172.21.15.44:0/4016635534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb860100020 0x7fb860100400 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb850009990 tx=0x7fb850009ca0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.140+0000 7fb865ec5700 1 -- 172.21.15.44:0/4016635534 shutdown_connections 2023-09-10T00:33:29.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.140+0000 7fb865ec5700 1 --2- 172.21.15.44:0/4016635534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb86010d090 0x7fb86010f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.140+0000 7fb865ec5700 1 --2- 172.21.15.44:0/4016635534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb860100020 0x7fb860100400 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.140+0000 7fb865ec5700 1 --2- 172.21.15.44:0/4016635534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb860100940 0x7fb86010cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.140+0000 7fb865ec5700 1 -- 172.21.15.44:0/4016635534 >> 172.21.15.44:0/4016635534 conn(0x7fb860076600 msgr2=0x7fb860076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:29.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.141+0000 7fb865ec5700 1 -- 172.21.15.44:0/4016635534 shutdown_connections 2023-09-10T00:33:29.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.141+0000 7fb865ec5700 1 -- 172.21.15.44:0/4016635534 wait complete. 2023-09-10T00:33:29.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.141+0000 7fb865ec5700 1 Processor -- start 2023-09-10T00:33:29.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.141+0000 7fb865ec5700 1 -- start start 2023-09-10T00:33:29.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.142+0000 7fb865ec5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb860100020 0x7fb86019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:29.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.142+0000 7fb865ec5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb860100940 0x7fb86019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:29.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.142+0000 7fb865ec5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb86010d090 0x7fb860198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:29.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.142+0000 7fb865ec5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb860198c70 con 0x7fb86010d090 2023-09-10T00:33:29.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.142+0000 7fb865ec5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb860198de0 con 0x7fb860100940 2023-09-10T00:33:29.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.142+0000 7fb865ec5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb860198fa0 con 0x7fb860100020 2023-09-10T00:33:29.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.142+0000 7fb85ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb86010d090 0x7fb860198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:29.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.142+0000 7fb85ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb86010d090 0x7fb860198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45330/0 (socket says 172.21.15.44:45330) 2023-09-10T00:33:29.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.142+0000 7fb85ffff700 1 -- 172.21.15.44:0/1162375720 learned_addr learned my addr 172.21.15.44:0/1162375720 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:29.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.142+0000 7fb85f7fe700 1 --2- 172.21.15.44:0/1162375720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb860100020 0x7fb86019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:29.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.142+0000 7fb85effd700 1 --2- 172.21.15.44:0/1162375720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb860100940 0x7fb86019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:29.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.142+0000 7fb85ffff700 1 -- 172.21.15.44:0/1162375720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb860100020 msgr2=0x7fb86019e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.142+0000 7fb85ffff700 1 --2- 172.21.15.44:0/1162375720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb860100020 0x7fb86019e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.142+0000 7fb85ffff700 1 -- 172.21.15.44:0/1162375720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb860100940 msgr2=0x7fb86019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.142+0000 7fb85ffff700 1 --2- 172.21.15.44:0/1162375720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb860100940 0x7fb86019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.142+0000 7fb85ffff700 1 -- 172.21.15.44:0/1162375720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8500095f0 con 0x7fb86010d090 2023-09-10T00:33:29.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.143+0000 7fb85ffff700 1 --2- 172.21.15.44:0/1162375720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb86010d090 0x7fb860198600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb85800cc30 tx=0x7fb85800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:29.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.143+0000 7fb85cff9700 1 -- 172.21.15.44:0/1162375720 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85800aac0 con 0x7fb86010d090 2023-09-10T00:33:29.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.143+0000 7fb85cff9700 1 -- 172.21.15.44:0/1162375720 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb85800ac20 con 0x7fb86010d090 2023-09-10T00:33:29.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.143+0000 7fb85cff9700 1 -- 172.21.15.44:0/1162375720 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8580115e0 con 0x7fb86010d090 2023-09-10T00:33:29.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.143+0000 7fb865ec5700 1 -- 172.21.15.44:0/1162375720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb860199280 con 0x7fb86010d090 2023-09-10T00:33:29.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.143+0000 7fb865ec5700 1 -- 172.21.15.44:0/1162375720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8601a5570 con 0x7fb86010d090 2023-09-10T00:33:29.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.144+0000 7fb85cff9700 1 -- 172.21.15.44:0/1162375720 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fb858011740 con 0x7fb86010d090 2023-09-10T00:33:29.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.144+0000 7fb865ec5700 1 -- 172.21.15.44:0/1162375720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86004e300 con 0x7fb86010d090 2023-09-10T00:33:29.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.144+0000 7fb85cff9700 1 --2- 172.21.15.44:0/1162375720 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb848038630 0x7fb84803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:29.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.144+0000 7fb85cff9700 1 -- 172.21.15.44:0/1162375720 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb85804bd50 con 0x7fb86010d090 2023-09-10T00:33:29.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.144+0000 7fb85f7fe700 1 --2- 172.21.15.44:0/1162375720 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb848038630 0x7fb84803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:29.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.145+0000 7fb85f7fe700 1 --2- 172.21.15.44:0/1162375720 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb848038630 0x7fb84803aaf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb85000b5c0 tx=0x7fb850005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:29.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.150+0000 7fb85cff9700 1 -- 172.21.15.44:0/1162375720 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb85801b400 con 0x7fb86010d090 2023-09-10T00:33:29.233 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:29 smithi044 ceph-mon[98676]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:29.234 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:29 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/657071607' entity='client.admin' 2023-09-10T00:33:29.234 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:29 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1737128854' entity='client.admin' 2023-09-10T00:33:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:29 smithi131 ceph-mon[103713]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:29 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/657071607' entity='client.admin' 2023-09-10T00:33:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:29 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1737128854' entity='client.admin' 2023-09-10T00:33:29.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:29 smithi099 ceph-mon[102692]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:29.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:29 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/657071607' entity='client.admin' 2023-09-10T00:33:29.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:29 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1737128854' entity='client.admin' 2023-09-10T00:33:29.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.375+0000 7fb865ec5700 1 -- 172.21.15.44:0/1162375720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) v1 -- 0x7fb860112780 con 0x7fb86010d090 2023-09-10T00:33:29.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.378+0000 7fb85cff9700 1 -- 172.21.15.44:0/1162375720 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v26)=0 v26) v1 ==== 119+0+0 (secure 0 0 0) 0x7fb858016030 con 0x7fb86010d090 2023-09-10T00:33:29.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.382+0000 7fb865ec5700 1 -- 172.21.15.44:0/1162375720 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb848038630 msgr2=0x7fb84803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.383 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.382+0000 7fb865ec5700 1 --2- 172.21.15.44:0/1162375720 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb848038630 0x7fb84803aaf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb85000b5c0 tx=0x7fb850005fb0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.383 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.382+0000 7fb865ec5700 1 -- 172.21.15.44:0/1162375720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb86010d090 msgr2=0x7fb860198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.383 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.382+0000 7fb865ec5700 1 --2- 172.21.15.44:0/1162375720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb86010d090 0x7fb860198600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb85800cc30 tx=0x7fb85800cf40 comp rx=0 tx=0).stop 2023-09-10T00:33:29.383 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.382+0000 7fb865ec5700 1 -- 172.21.15.44:0/1162375720 shutdown_connections 2023-09-10T00:33:29.383 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.382+0000 7fb865ec5700 1 --2- 172.21.15.44:0/1162375720 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb848038630 0x7fb84803aaf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.382+0000 7fb865ec5700 1 --2- 172.21.15.44:0/1162375720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb86010d090 0x7fb860198600 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.382+0000 7fb865ec5700 1 --2- 172.21.15.44:0/1162375720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb860100020 0x7fb86019e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.382+0000 7fb865ec5700 1 --2- 172.21.15.44:0/1162375720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb860100940 0x7fb86019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.382+0000 7fb865ec5700 1 -- 172.21.15.44:0/1162375720 >> 172.21.15.44:0/1162375720 conn(0x7fb860076600 msgr2=0x7fb86010e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:29.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.382+0000 7fb865ec5700 1 -- 172.21.15.44:0/1162375720 shutdown_connections 2023-09-10T00:33:29.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.382+0000 7fb865ec5700 1 -- 172.21.15.44:0/1162375720 wait complete. 2023-09-10T00:33:29.481 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.480+0000 7faef86da700 1 -- 172.21.15.44:0/8693153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faef00fff30 msgr2=0x7faef010c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.481 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.480+0000 7faef86da700 1 --2- 172.21.15.44:0/8693153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faef00fff30 0x7faef010c3e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faee800b0d0 tx=0x7faee800b3e0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.481 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.480+0000 7faef86da700 1 -- 172.21.15.44:0/8693153 shutdown_connections 2023-09-10T00:33:29.482 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.480+0000 7faef86da700 1 --2- 172.21.15.44:0/8693153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faef00fff30 0x7faef010c3e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.482 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.480+0000 7faef86da700 1 --2- 172.21.15.44:0/8693153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faef0105610 0x7faef01059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.482 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.480+0000 7faef86da700 1 --2- 172.21.15.44:0/8693153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef00ff570 0x7faef00ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.482 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.480+0000 7faef86da700 1 -- 172.21.15.44:0/8693153 >> 172.21.15.44:0/8693153 conn(0x7faef0076680 msgr2=0x7faef0076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:29.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.480+0000 7faef86da700 1 -- 172.21.15.44:0/8693153 shutdown_connections 2023-09-10T00:33:29.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.480+0000 7faef86da700 1 -- 172.21.15.44:0/8693153 wait complete. 2023-09-10T00:33:29.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.481+0000 7faef86da700 1 Processor -- start 2023-09-10T00:33:29.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.481+0000 7faef86da700 1 -- start start 2023-09-10T00:33:29.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.481+0000 7faef86da700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faef00ff570 0x7faef0197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:29.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.481+0000 7faef86da700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef00fff30 0x7faef0197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:29.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.481+0000 7faef86da700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faef0105610 0x7faef01a2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:29.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.481+0000 7faef86da700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef0198490 con 0x7faef0105610 2023-09-10T00:33:29.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.481+0000 7faef86da700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef01a2bd0 con 0x7faef00ff570 2023-09-10T00:33:29.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.481+0000 7faef86da700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef01a2d90 con 0x7faef00fff30 2023-09-10T00:33:29.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.481+0000 7faef6c77700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faef0105610 0x7faef01a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:29.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.481+0000 7faef6c77700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faef0105610 0x7faef01a2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45344/0 (socket says 172.21.15.44:45344) 2023-09-10T00:33:29.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.481+0000 7faef6c77700 1 -- 172.21.15.44:0/1971068725 learned_addr learned my addr 172.21.15.44:0/1971068725 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:29.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.481+0000 7faef6476700 1 --2- 172.21.15.44:0/1971068725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faef00ff570 0x7faef0197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.481+0000 7faef5c75700 1 --2- 172.21.15.44:0/1971068725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef00fff30 0x7faef0197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.482+0000 7faef6c77700 1 -- 172.21.15.44:0/1971068725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef00fff30 msgr2=0x7faef0197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.482+0000 7faef6c77700 1 --2- 172.21.15.44:0/1971068725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef00fff30 0x7faef0197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.482+0000 7faef6c77700 1 -- 172.21.15.44:0/1971068725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faef00ff570 msgr2=0x7faef0197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.482+0000 7faef6c77700 1 --2- 172.21.15.44:0/1971068725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faef00ff570 0x7faef0197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.482+0000 7faef6c77700 1 -- 172.21.15.44:0/1971068725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee8009d00 con 0x7faef0105610 2023-09-10T00:33:29.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.482+0000 7faef6c77700 1 --2- 172.21.15.44:0/1971068725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faef0105610 0x7faef01a2690 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faee8009fd0 tx=0x7faee800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:29.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.482+0000 7faee77fe700 1 -- 172.21.15.44:0/1971068725 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee800e040 con 0x7faef0105610 2023-09-10T00:33:29.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.482+0000 7faef86da700 1 -- 172.21.15.44:0/1971068725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef01a3010 con 0x7faef0105610 2023-09-10T00:33:29.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.482+0000 7faee77fe700 1 -- 172.21.15.44:0/1971068725 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faee8003e10 con 0x7faef0105610 2023-09-10T00:33:29.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.482+0000 7faee77fe700 1 -- 172.21.15.44:0/1971068725 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee80052a0 con 0x7faef0105610 2023-09-10T00:33:29.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.482+0000 7faef86da700 1 -- 172.21.15.44:0/1971068725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef01a34d0 con 0x7faef0105610 2023-09-10T00:33:29.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.483+0000 7faef86da700 1 -- 172.21.15.44:0/1971068725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef004e300 con 0x7faef0105610 2023-09-10T00:33:29.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.484+0000 7faee77fe700 1 -- 172.21.15.44:0/1971068725 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7faee801b450 con 0x7faef0105610 2023-09-10T00:33:29.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.484+0000 7faee77fe700 1 --2- 172.21.15.44:0/1971068725 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7faedc038630 0x7faedc03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:29.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.484+0000 7faee77fe700 1 -- 172.21.15.44:0/1971068725 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faee804c490 con 0x7faef0105610 2023-09-10T00:33:29.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.484+0000 7faef6476700 1 --2- 172.21.15.44:0/1971068725 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7faedc038630 0x7faedc03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:29.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.485+0000 7faef6476700 1 --2- 172.21.15.44:0/1971068725 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7faedc038630 0x7faedc03aaf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faeec005950 tx=0x7faeec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:29.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.490+0000 7faee77fe700 1 -- 172.21.15.44:0/1971068725 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7faee8005920 con 0x7faef0105610 2023-09-10T00:33:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.712+0000 7faef86da700 1 -- 172.21.15.44:0/1971068725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) v1 -- 0x7faef010f360 con 0x7faef0105610 2023-09-10T00:33:29.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.716+0000 7faee77fe700 1 -- 172.21.15.44:0/1971068725 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v27)=0 v27) v1 ==== 115+0+0 (secure 0 0 0) 0x7faee801f020 con 0x7faef0105610 2023-09-10T00:33:29.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.718+0000 7faef86da700 1 -- 172.21.15.44:0/1971068725 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7faedc038630 msgr2=0x7faedc03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.718+0000 7faef86da700 1 --2- 172.21.15.44:0/1971068725 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7faedc038630 0x7faedc03aaf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faeec005950 tx=0x7faeec00b410 comp rx=0 tx=0).stop 2023-09-10T00:33:29.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.718+0000 7faef86da700 1 -- 172.21.15.44:0/1971068725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faef0105610 msgr2=0x7faef01a2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.718+0000 7faef86da700 1 --2- 172.21.15.44:0/1971068725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faef0105610 0x7faef01a2690 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faee8009fd0 tx=0x7faee800cce0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.718+0000 7faef86da700 1 -- 172.21.15.44:0/1971068725 shutdown_connections 2023-09-10T00:33:29.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.718+0000 7faef86da700 1 --2- 172.21.15.44:0/1971068725 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7faedc038630 0x7faedc03aaf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.718+0000 7faef86da700 1 --2- 172.21.15.44:0/1971068725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faef0105610 0x7faef01a2690 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.718+0000 7faef86da700 1 --2- 172.21.15.44:0/1971068725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faef00ff570 0x7faef0197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.718+0000 7faef86da700 1 --2- 172.21.15.44:0/1971068725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef00fff30 0x7faef0197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.718+0000 7faef86da700 1 -- 172.21.15.44:0/1971068725 >> 172.21.15.44:0/1971068725 conn(0x7faef0076680 msgr2=0x7faef00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:29.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.718+0000 7faef86da700 1 -- 172.21.15.44:0/1971068725 shutdown_connections 2023-09-10T00:33:29.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.718+0000 7faef86da700 1 -- 172.21.15.44:0/1971068725 wait complete. 2023-09-10T00:33:29.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.749+0000 7f2ad6698700 1 -- 172.21.15.44:0/3120544311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ad0100030 msgr2=0x7f2ad01004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.749+0000 7f2ad6698700 1 --2- 172.21.15.44:0/3120544311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ad0100030 0x7f2ad01004b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00099c0 tx=0x7f2ac0009cd0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.749+0000 7f2ad6698700 1 -- 172.21.15.44:0/3120544311 shutdown_connections 2023-09-10T00:33:29.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.749+0000 7f2ad6698700 1 --2- 172.21.15.44:0/3120544311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad01009f0 0x7f2ad010c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.749+0000 7f2ad6698700 1 --2- 172.21.15.44:0/3120544311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad01055f0 0x7f2ad01059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.749+0000 7f2ad6698700 1 --2- 172.21.15.44:0/3120544311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ad0100030 0x7f2ad01004b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.749+0000 7f2ad6698700 1 -- 172.21.15.44:0/3120544311 >> 172.21.15.44:0/3120544311 conn(0x7f2ad0075a70 msgr2=0x7f2ad0075e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:29.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.749+0000 7f2ad6698700 1 -- 172.21.15.44:0/3120544311 shutdown_connections 2023-09-10T00:33:29.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.749+0000 7f2ad6698700 1 -- 172.21.15.44:0/3120544311 wait complete. 2023-09-10T00:33:29.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.750+0000 7f2ad6698700 1 Processor -- start 2023-09-10T00:33:29.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.750+0000 7f2ad6698700 1 -- start start 2023-09-10T00:33:29.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.750+0000 7f2ad6698700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad0100030 0x7f2ad01011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:29.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.750+0000 7f2ad6698700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad01009f0 0x7f2ad01016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:29.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.750+0000 7f2ad6698700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ad01055f0 0x7f2ad0101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:29.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.750+0000 7f2ad6698700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad01a49e0 con 0x7f2ad01055f0 2023-09-10T00:33:29.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.750+0000 7f2ad6698700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad01a4b50 con 0x7f2ad01009f0 2023-09-10T00:33:29.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.750+0000 7f2ad6698700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad01a4d10 con 0x7f2ad0100030 2023-09-10T00:33:29.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.750+0000 7f2ad4c35700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ad01055f0 0x7f2ad0101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:29.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.751+0000 7f2ad4c35700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ad01055f0 0x7f2ad0101c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45358/0 (socket says 172.21.15.44:45358) 2023-09-10T00:33:29.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.751+0000 7f2ad4c35700 1 -- 172.21.15.44:0/2202936678 learned_addr learned my addr 172.21.15.44:0/2202936678 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.751+0000 7f2acf7fe700 1 --2- 172.21.15.44:0/2202936678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad01009f0 0x7f2ad01016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.751+0000 7f2acffff700 1 --2- 172.21.15.44:0/2202936678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad0100030 0x7f2ad01011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.751+0000 7f2ad4c35700 1 -- 172.21.15.44:0/2202936678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad0100030 msgr2=0x7f2ad01011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.751+0000 7f2ad4c35700 1 --2- 172.21.15.44:0/2202936678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad0100030 0x7f2ad01011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.751+0000 7f2ad4c35700 1 -- 172.21.15.44:0/2202936678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad01009f0 msgr2=0x7f2ad01016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.751+0000 7f2ad4c35700 1 --2- 172.21.15.44:0/2202936678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad01009f0 0x7f2ad01016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.751+0000 7f2ad4c35700 1 -- 172.21.15.44:0/2202936678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac00095f0 con 0x7f2ad01055f0 2023-09-10T00:33:29.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.751+0000 7f2acffff700 1 --2- 172.21.15.44:0/2202936678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad0100030 0x7f2ad01011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:33:29.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.751+0000 7f2ad4c35700 1 --2- 172.21.15.44:0/2202936678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ad01055f0 0x7f2ad0101c70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800ada0 tx=0x7f2ac800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:29.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.751+0000 7f2acd7fa700 1 -- 172.21.15.44:0/2202936678 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac800f070 con 0x7f2ad01055f0 2023-09-10T00:33:29.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.751+0000 7f2acd7fa700 1 -- 172.21.15.44:0/2202936678 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ac8007b30 con 0x7f2ad01055f0 2023-09-10T00:33:29.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.751+0000 7f2acd7fa700 1 -- 172.21.15.44:0/2202936678 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac80053e0 con 0x7f2ad01055f0 2023-09-10T00:33:29.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.751+0000 7f2ad6698700 1 -- 172.21.15.44:0/2202936678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad01a4ff0 con 0x7f2ad01055f0 2023-09-10T00:33:29.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.751+0000 7f2ad6698700 1 -- 172.21.15.44:0/2202936678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad01a5470 con 0x7f2ad01055f0 2023-09-10T00:33:29.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.752+0000 7f2acd7fa700 1 -- 172.21.15.44:0/2202936678 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f2ac8005540 con 0x7f2ad01055f0 2023-09-10T00:33:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.752+0000 7f2ad6698700 1 -- 172.21.15.44:0/2202936678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad0108a70 con 0x7f2ad01055f0 2023-09-10T00:33:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.752+0000 7f2acd7fa700 1 --2- 172.21.15.44:0/2202936678 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2ab8038630 0x7f2ab803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.752+0000 7f2acd7fa700 1 -- 172.21.15.44:0/2202936678 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2ac804a5d0 con 0x7f2ad01055f0 2023-09-10T00:33:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.753+0000 7f2acffff700 1 --2- 172.21.15.44:0/2202936678 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2ab8038630 0x7f2ab803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.753+0000 7f2acffff700 1 --2- 172.21.15.44:0/2202936678 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2ab8038630 0x7f2ab803aaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4005950 tx=0x7f2ac4005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:29.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.757+0000 7f2acd7fa700 1 -- 172.21.15.44:0/2202936678 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2ac801a310 con 0x7f2ad01055f0 2023-09-10T00:33:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.972+0000 7f2ad6698700 1 -- 172.21.15.44:0/2202936678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) v1 -- 0x7f2ad004e300 con 0x7f2ad01055f0 2023-09-10T00:33:29.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.976+0000 7f2acd7fa700 1 -- 172.21.15.44:0/2202936678 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v28)=0 v28) v1 ==== 121+0+0 (secure 0 0 0) 0x7f2ac8012b80 con 0x7f2ad01055f0 2023-09-10T00:33:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.978+0000 7f2ad6698700 1 -- 172.21.15.44:0/2202936678 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2ab8038630 msgr2=0x7f2ab803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.978+0000 7f2ad6698700 1 --2- 172.21.15.44:0/2202936678 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2ab8038630 0x7f2ab803aaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4005950 tx=0x7f2ac4005f50 comp rx=0 tx=0).stop 2023-09-10T00:33:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.978+0000 7f2ad6698700 1 -- 172.21.15.44:0/2202936678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ad01055f0 msgr2=0x7f2ad0101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.978+0000 7f2ad6698700 1 --2- 172.21.15.44:0/2202936678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ad01055f0 0x7f2ad0101c70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800ada0 tx=0x7f2ac800c330 comp rx=0 tx=0).stop 2023-09-10T00:33:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.978+0000 7f2ad6698700 1 -- 172.21.15.44:0/2202936678 shutdown_connections 2023-09-10T00:33:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.978+0000 7f2ad6698700 1 --2- 172.21.15.44:0/2202936678 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2ab8038630 0x7f2ab803aaf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.978+0000 7f2ad6698700 1 --2- 172.21.15.44:0/2202936678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ad01055f0 0x7f2ad0101c70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.978+0000 7f2ad6698700 1 --2- 172.21.15.44:0/2202936678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad0100030 0x7f2ad01011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.978+0000 7f2ad6698700 1 --2- 172.21.15.44:0/2202936678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad01009f0 0x7f2ad01016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.978+0000 7f2ad6698700 1 -- 172.21.15.44:0/2202936678 >> 172.21.15.44:0/2202936678 conn(0x7f2ad0075a70 msgr2=0x7f2ad01048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.978+0000 7f2ad6698700 1 -- 172.21.15.44:0/2202936678 shutdown_connections 2023-09-10T00:33:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:29.978+0000 7f2ad6698700 1 -- 172.21.15.44:0/2202936678 wait complete. 2023-09-10T00:33:30.264 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.264+0000 7f4ec521c700 1 -- 172.21.15.44:0/4288372922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ec0068120 msgr2=0x7f4ec00685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:30.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.264+0000 7f4ec521c700 1 --2- 172.21.15.44:0/4288372922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ec0068120 0x7f4ec00685a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009990 tx=0x7f4eb4009ca0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.264+0000 7f4ec521c700 1 -- 172.21.15.44:0/4288372922 shutdown_connections 2023-09-10T00:33:30.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.264+0000 7f4ec521c700 1 --2- 172.21.15.44:0/4288372922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ec0068ae0 0x7f4ec010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.264+0000 7f4ec521c700 1 --2- 172.21.15.44:0/4288372922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec0106770 0x7f4ec0106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.264+0000 7f4ec521c700 1 --2- 172.21.15.44:0/4288372922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ec0068120 0x7f4ec00685a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.264+0000 7f4ec521c700 1 -- 172.21.15.44:0/4288372922 >> 172.21.15.44:0/4288372922 conn(0x7f4ec0076600 msgr2=0x7f4ec0076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:30.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.264+0000 7f4ec521c700 1 -- 172.21.15.44:0/4288372922 shutdown_connections 2023-09-10T00:33:30.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.264+0000 7f4ec521c700 1 -- 172.21.15.44:0/4288372922 wait complete. 2023-09-10T00:33:30.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.265+0000 7f4ec521c700 1 Processor -- start 2023-09-10T00:33:30.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.265+0000 7f4ec521c700 1 -- start start 2023-09-10T00:33:30.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.265+0000 7f4ec521c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec0068120 0x7f4ec00ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:30.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.265+0000 7f4ec521c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ec0068ae0 0x7f4ec00ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:30.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.265+0000 7f4ec521c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ec0106770 0x7f4ec0100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:30.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.265+0000 7f4ec521c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec01a4bc0 con 0x7f4ec0106770 2023-09-10T00:33:30.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.265+0000 7f4ec521c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec01a4d30 con 0x7f4ec0068ae0 2023-09-10T00:33:30.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.265+0000 7f4ec521c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec01a4ef0 con 0x7f4ec0068120 2023-09-10T00:33:30.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.265+0000 7f4ebf59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ec0106770 0x7f4ec0100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:30.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.265+0000 7f4ebf59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ec0106770 0x7f4ec0100070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45368/0 (socket says 172.21.15.44:45368) 2023-09-10T00:33:30.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.265+0000 7f4ebf59e700 1 -- 172.21.15.44:0/329531650 learned_addr learned my addr 172.21.15.44:0/329531650 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:30.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.265+0000 7f4ebed9d700 1 --2- 172.21.15.44:0/329531650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec0068120 0x7f4ec00ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:30.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.266+0000 7f4ebe59c700 1 --2- 172.21.15.44:0/329531650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ec0068ae0 0x7f4ec00ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:30.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.266+0000 7f4ebf59e700 1 -- 172.21.15.44:0/329531650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec0068120 msgr2=0x7f4ec00ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:30.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.266+0000 7f4ebf59e700 1 --2- 172.21.15.44:0/329531650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec0068120 0x7f4ec00ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.266+0000 7f4ebf59e700 1 -- 172.21.15.44:0/329531650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ec0068ae0 msgr2=0x7f4ec00ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:30.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.266+0000 7f4ebf59e700 1 --2- 172.21.15.44:0/329531650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ec0068ae0 0x7f4ec00ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.266+0000 7f4ebf59e700 1 -- 172.21.15.44:0/329531650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb40095f0 con 0x7f4ec0106770 2023-09-10T00:33:30.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.266+0000 7f4ebe59c700 1 --2- 172.21.15.44:0/329531650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ec0068ae0 0x7f4ec00ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:33:30.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.266+0000 7f4ebf59e700 1 --2- 172.21.15.44:0/329531650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ec0106770 0x7f4ec0100070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8007f10 tx=0x7f4eb800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:30.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.266+0000 7f4ea7fff700 1 -- 172.21.15.44:0/329531650 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8017070 con 0x7f4ec0106770 2023-09-10T00:33:30.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.266+0000 7f4ec521c700 1 -- 172.21.15.44:0/329531650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec01a51d0 con 0x7f4ec0106770 2023-09-10T00:33:30.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.266+0000 7f4ea7fff700 1 -- 172.21.15.44:0/329531650 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4eb800f040 con 0x7f4ec0106770 2023-09-10T00:33:30.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.266+0000 7f4ea7fff700 1 -- 172.21.15.44:0/329531650 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8005710 con 0x7f4ec0106770 2023-09-10T00:33:30.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.266+0000 7f4ec521c700 1 -- 172.21.15.44:0/329531650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec01a5680 con 0x7f4ec0106770 2023-09-10T00:33:30.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.267+0000 7f4ea7fff700 1 -- 172.21.15.44:0/329531650 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f4eb8005870 con 0x7f4ec0106770 2023-09-10T00:33:30.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.267+0000 7f4ec521c700 1 -- 172.21.15.44:0/329531650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec004e300 con 0x7f4ec0106770 2023-09-10T00:33:30.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.268+0000 7f4ea7fff700 1 --2- 172.21.15.44:0/329531650 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4ea8038630 0x7f4ea803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:30.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.268+0000 7f4ea7fff700 1 -- 172.21.15.44:0/329531650 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4eb804a5d0 con 0x7f4ec0106770 2023-09-10T00:33:30.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.268+0000 7f4ebed9d700 1 --2- 172.21.15.44:0/329531650 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4ea8038630 0x7f4ea803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:30.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.268+0000 7f4ebed9d700 1 --2- 172.21.15.44:0/329531650 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4ea8038630 0x7f4ea803aaf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009a10 tx=0x7f4eb0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:30.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.273+0000 7f4ea7fff700 1 -- 172.21.15.44:0/329531650 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4eb80112b0 con 0x7f4ec0106770 2023-09-10T00:33:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.479+0000 7f4ec521c700 1 -- 172.21.15.44:0/329531650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f4ec0100ae0 con 0x7f4ec0106770 2023-09-10T00:33:30.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.483+0000 7f4ea7fff700 1 -- 172.21.15.44:0/329531650 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v29)=0 v29) v1 ==== 121+0+0 (secure 0 0 0) 0x7f4eb801ab50 con 0x7f4ec0106770 2023-09-10T00:33:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.485+0000 7f4ec521c700 1 -- 172.21.15.44:0/329531650 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4ea8038630 msgr2=0x7f4ea803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.485+0000 7f4ec521c700 1 --2- 172.21.15.44:0/329531650 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4ea8038630 0x7f4ea803aaf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009a10 tx=0x7f4eb0009190 comp rx=0 tx=0).stop 2023-09-10T00:33:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.485+0000 7f4ec521c700 1 -- 172.21.15.44:0/329531650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ec0106770 msgr2=0x7f4ec0100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.485+0000 7f4ec521c700 1 --2- 172.21.15.44:0/329531650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ec0106770 0x7f4ec0100070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8007f10 tx=0x7f4eb800d3b0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.485+0000 7f4ec521c700 1 -- 172.21.15.44:0/329531650 shutdown_connections 2023-09-10T00:33:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.485+0000 7f4ec521c700 1 --2- 172.21.15.44:0/329531650 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4ea8038630 0x7f4ea803aaf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.485+0000 7f4ec521c700 1 --2- 172.21.15.44:0/329531650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ec0106770 0x7f4ec0100070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.485+0000 7f4ec521c700 1 --2- 172.21.15.44:0/329531650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec0068120 0x7f4ec00ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.485+0000 7f4ec521c700 1 --2- 172.21.15.44:0/329531650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ec0068ae0 0x7f4ec00ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.485+0000 7f4ec521c700 1 -- 172.21.15.44:0/329531650 >> 172.21.15.44:0/329531650 conn(0x7f4ec0076600 msgr2=0x7f4ec00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:30.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.485+0000 7f4ec521c700 1 -- 172.21.15.44:0/329531650 shutdown_connections 2023-09-10T00:33:30.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.485+0000 7f4ec521c700 1 -- 172.21.15.44:0/329531650 wait complete. 2023-09-10T00:33:30.526 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:30 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1162375720' entity='client.admin' 2023-09-10T00:33:30.526 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:30 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1971068725' entity='client.admin' 2023-09-10T00:33:30.526 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:30 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2202936678' entity='client.admin' 2023-09-10T00:33:30.600 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.600+0000 7fd959008700 1 -- 172.21.15.44:0/1675392314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540ff760 msgr2=0x7fd954105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:30.600 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.600+0000 7fd959008700 1 --2- 172.21.15.44:0/1675392314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540ff760 0x7fd954105220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd948009990 tx=0x7fd948009ca0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.601 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.600+0000 7fd959008700 1 -- 172.21.15.44:0/1675392314 shutdown_connections 2023-09-10T00:33:30.601 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.600+0000 7fd959008700 1 --2- 172.21.15.44:0/1675392314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd954105950 0x7fd95410d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.601 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.600+0000 7fd959008700 1 --2- 172.21.15.44:0/1675392314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9540fedb0 0x7fd9540ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.601 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.600+0000 7fd959008700 1 --2- 172.21.15.44:0/1675392314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540ff760 0x7fd954105220 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.601 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.600+0000 7fd959008700 1 -- 172.21.15.44:0/1675392314 >> 172.21.15.44:0/1675392314 conn(0x7fd954075a70 msgr2=0x7fd9540fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:30.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.600+0000 7fd959008700 1 -- 172.21.15.44:0/1675392314 shutdown_connections 2023-09-10T00:33:30.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.600+0000 7fd959008700 1 -- 172.21.15.44:0/1675392314 wait complete. 2023-09-10T00:33:30.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.601+0000 7fd959008700 1 Processor -- start 2023-09-10T00:33:30.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.601+0000 7fd959008700 1 -- start start 2023-09-10T00:33:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.601+0000 7fd959008700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540fedb0 0x7fd9540ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.601+0000 7fd959008700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9540ff760 0x7fd954100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.601+0000 7fd959008700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd954105950 0x7fd9541007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.601+0000 7fd959008700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9541a4940 con 0x7fd9540fedb0 2023-09-10T00:33:30.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.601+0000 7fd959008700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9541a4ab0 con 0x7fd954105950 2023-09-10T00:33:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.601+0000 7fd959008700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9541a4c70 con 0x7fd9540ff760 2023-09-10T00:33:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.601+0000 7fd952d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540fedb0 0x7fd9540ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.601+0000 7fd952d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540fedb0 0x7fd9540ffd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45378/0 (socket says 172.21.15.44:45378) 2023-09-10T00:33:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.601+0000 7fd952d9d700 1 -- 172.21.15.44:0/4056537443 learned_addr learned my addr 172.21.15.44:0/4056537443 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:30.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.602+0000 7fd95259c700 1 --2- 172.21.15.44:0/4056537443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9540ff760 0x7fd954100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.602+0000 7fd95359e700 1 --2- 172.21.15.44:0/4056537443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd954105950 0x7fd9541007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.602+0000 7fd952d9d700 1 -- 172.21.15.44:0/4056537443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9540ff760 msgr2=0x7fd954100270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.602+0000 7fd952d9d700 1 --2- 172.21.15.44:0/4056537443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9540ff760 0x7fd954100270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.602+0000 7fd952d9d700 1 -- 172.21.15.44:0/4056537443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd954105950 msgr2=0x7fd9541007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.602+0000 7fd952d9d700 1 --2- 172.21.15.44:0/4056537443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd954105950 0x7fd9541007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.602+0000 7fd952d9d700 1 -- 172.21.15.44:0/4056537443 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9480095f0 con 0x7fd9540fedb0 2023-09-10T00:33:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.602+0000 7fd95259c700 1 --2- 172.21.15.44:0/4056537443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9540ff760 0x7fd954100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:33:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.602+0000 7fd952d9d700 1 --2- 172.21.15.44:0/4056537443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540fedb0 0x7fd9540ffd30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd94400c9b0 tx=0x7fd94400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.602+0000 7fd93bfff700 1 -- 172.21.15.44:0/4056537443 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944007b00 con 0x7fd9540fedb0 2023-09-10T00:33:30.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.602+0000 7fd93bfff700 1 -- 172.21.15.44:0/4056537443 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd94400ce50 con 0x7fd9540fedb0 2023-09-10T00:33:30.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.602+0000 7fd959008700 1 -- 172.21.15.44:0/4056537443 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9541a4f50 con 0x7fd9540fedb0 2023-09-10T00:33:30.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.602+0000 7fd93bfff700 1 -- 172.21.15.44:0/4056537443 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944018420 con 0x7fd9540fedb0 2023-09-10T00:33:30.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.602+0000 7fd959008700 1 -- 172.21.15.44:0/4056537443 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9541a53c0 con 0x7fd9540fedb0 2023-09-10T00:33:30.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.603+0000 7fd93bfff700 1 -- 172.21.15.44:0/4056537443 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fd944004850 con 0x7fd9540fedb0 2023-09-10T00:33:30.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.603+0000 7fd93bfff700 1 --2- 172.21.15.44:0/4056537443 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd93c038630 0x7fd93c03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:30.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.603+0000 7fd93bfff700 1 -- 172.21.15.44:0/4056537443 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd944007c60 con 0x7fd9540fedb0 2023-09-10T00:33:30.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.604+0000 7fd95259c700 1 --2- 172.21.15.44:0/4056537443 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd93c038630 0x7fd93c03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:30.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.604+0000 7fd959008700 1 -- 172.21.15.44:0/4056537443 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd95404e300 con 0x7fd9540fedb0 2023-09-10T00:33:30.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.604+0000 7fd95259c700 1 --2- 172.21.15.44:0/4056537443 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd93c038630 0x7fd93c03aaf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd954101520 tx=0x7fd94800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:30.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.609+0000 7fd93bfff700 1 -- 172.21.15.44:0/4056537443 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd94400d750 con 0x7fd9540fedb0 2023-09-10T00:33:30.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:30 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1162375720' entity='client.admin' 2023-09-10T00:33:30.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:30 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1971068725' entity='client.admin' 2023-09-10T00:33:30.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:30 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2202936678' entity='client.admin' 2023-09-10T00:33:30.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.816+0000 7fd959008700 1 -- 172.21.15.44:0/4056537443 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7fd9541105e0 con 0x7fd9540fedb0 2023-09-10T00:33:30.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.819+0000 7fd93bfff700 1 -- 172.21.15.44:0/4056537443 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v30)=0 v30) v1 ==== 120+0+0 (secure 0 0 0) 0x7fd94403e030 con 0x7fd9540fedb0 2023-09-10T00:33:30.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.821+0000 7fd959008700 1 -- 172.21.15.44:0/4056537443 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd93c038630 msgr2=0x7fd93c03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:30.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.821+0000 7fd959008700 1 --2- 172.21.15.44:0/4056537443 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd93c038630 0x7fd93c03aaf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd954101520 tx=0x7fd94800b540 comp rx=0 tx=0).stop 2023-09-10T00:33:30.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.821+0000 7fd959008700 1 -- 172.21.15.44:0/4056537443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540fedb0 msgr2=0x7fd9540ffd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:30.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.821+0000 7fd959008700 1 --2- 172.21.15.44:0/4056537443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540fedb0 0x7fd9540ffd30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd94400c9b0 tx=0x7fd94400ccc0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.821+0000 7fd959008700 1 -- 172.21.15.44:0/4056537443 shutdown_connections 2023-09-10T00:33:30.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.822+0000 7fd959008700 1 --2- 172.21.15.44:0/4056537443 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd93c038630 0x7fd93c03aaf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.822+0000 7fd959008700 1 --2- 172.21.15.44:0/4056537443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd954105950 0x7fd9541007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.822+0000 7fd959008700 1 --2- 172.21.15.44:0/4056537443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540fedb0 0x7fd9540ffd30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.822+0000 7fd959008700 1 --2- 172.21.15.44:0/4056537443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9540ff760 0x7fd954100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:30.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.822+0000 7fd959008700 1 -- 172.21.15.44:0/4056537443 >> 172.21.15.44:0/4056537443 conn(0x7fd954075a70 msgr2=0x7fd9541041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:30.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.822+0000 7fd959008700 1 -- 172.21.15.44:0/4056537443 shutdown_connections 2023-09-10T00:33:30.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:30.822+0000 7fd959008700 1 -- 172.21.15.44:0/4056537443 wait complete. 2023-09-10T00:33:30.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:30 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1162375720' entity='client.admin' 2023-09-10T00:33:30.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:30 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1971068725' entity='client.admin' 2023-09-10T00:33:30.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:30 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2202936678' entity='client.admin' 2023-09-10T00:33:31.506 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-09-10T00:33:31.507 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:33:31.507 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.conf 2023-09-10T00:33:31.540 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:33:31.540 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:33:31.609 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-10T00:33:31.609 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.conf 2023-09-10T00:33:31.647 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-10T00:33:31.647 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:33:31.718 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-10T00:33:31.719 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.conf 2023-09-10T00:33:31.750 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:31 smithi131 ceph-mon[103713]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:31.751 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:31 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/329531650' entity='client.admin' 2023-09-10T00:33:31.751 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:31 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/4056537443' entity='client.admin' 2023-09-10T00:33:31.756 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-10T00:33:31.756 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:33:31.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:31 smithi044 ceph-mon[98676]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:31.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:31 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/329531650' entity='client.admin' 2023-09-10T00:33:31.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:31 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/4056537443' entity='client.admin' 2023-09-10T00:33:31.826 INFO:tasks.cephadm:Adding mgr.x on smithi044 2023-09-10T00:33:31.827 INFO:tasks.cephadm:Adding mgr.y on smithi099 2023-09-10T00:33:31.827 INFO:tasks.cephadm:Adding mgr.z on smithi131 2023-09-10T00:33:31.827 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch apply mgr '3;smithi044=x;smithi099=y;smithi131=z' 2023-09-10T00:33:31.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:31 smithi099 ceph-mon[102692]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:31.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:31 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/329531650' entity='client.admin' 2023-09-10T00:33:31.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:31 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/4056537443' entity='client.admin' 2023-09-10T00:33:32.418 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-10T00:33:33.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.714+0000 7f27b7a7c700 1 -- 172.21.15.131:0/705357222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0100b40 msgr2=0x7f27b0100f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:33.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.714+0000 7f27b7a7c700 1 --2- 172.21.15.131:0/705357222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0100b40 0x7f27b0100f60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0098f0 tx=0x7f27ac009c00 comp rx=0 tx=0).stop 2023-09-10T00:33:33.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.714+0000 7f27b7a7c700 1 -- 172.21.15.131:0/705357222 shutdown_connections 2023-09-10T00:33:33.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.714+0000 7f27b7a7c700 1 --2- 172.21.15.131:0/705357222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b0100160 0x7f27b0107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:33.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.714+0000 7f27b7a7c700 1 --2- 172.21.15.131:0/705357222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0100b40 0x7f27b0100f60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:33.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.714+0000 7f27b7a7c700 1 --2- 172.21.15.131:0/705357222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b00ff7a0 0x7f27b00ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:33.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.714+0000 7f27b7a7c700 1 -- 172.21.15.131:0/705357222 >> 172.21.15.131:0/705357222 conn(0x7f27b00fa2b0 msgr2=0x7f27b00fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:33.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.714+0000 7f27b7a7c700 1 -- 172.21.15.131:0/705357222 shutdown_connections 2023-09-10T00:33:33.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.714+0000 7f27b7a7c700 1 -- 172.21.15.131:0/705357222 wait complete. 2023-09-10T00:33:33.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.714+0000 7f27b7a7c700 1 Processor -- start 2023-09-10T00:33:33.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b7a7c700 1 -- start start 2023-09-10T00:33:33.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b7a7c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b00ff7a0 0x7f27b00ff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:33.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b7a7c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b0100160 0x7f27b019b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:33.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b7a7c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0100b40 0x7f27b019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:33.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b7a7c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b019ddf0 con 0x7f27b00ff7a0 2023-09-10T00:33:33.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b7a7c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b019df60 con 0x7f27b0100160 2023-09-10T00:33:33.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b7a7c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b019e120 con 0x7f27b0100b40 2023-09-10T00:33:33.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b6019700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0100b40 0x7f27b019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:33.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b6019700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0100b40 0x7f27b019d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52862/0 (socket says 172.21.15.131:52862) 2023-09-10T00:33:33.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b6019700 1 -- 172.21.15.131:0/2654930292 learned_addr learned my addr 172.21.15.131:0/2654930292 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-10T00:33:33.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b5017700 1 --2- 172.21.15.131:0/2654930292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b0100160 0x7f27b019b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:33.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b5818700 1 --2- 172.21.15.131:0/2654930292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b00ff7a0 0x7f27b00ff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:33.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b6019700 1 -- 172.21.15.131:0/2654930292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b0100160 msgr2=0x7f27b019b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:33.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b6019700 1 --2- 172.21.15.131:0/2654930292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b0100160 0x7f27b019b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:33.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b6019700 1 -- 172.21.15.131:0/2654930292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b00ff7a0 msgr2=0x7f27b00ff1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:33.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b6019700 1 --2- 172.21.15.131:0/2654930292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b00ff7a0 0x7f27b00ff1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:33.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b6019700 1 -- 172.21.15.131:0/2654930292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac009520 con 0x7f27b0100b40 2023-09-10T00:33:33.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b5818700 1 --2- 172.21.15.131:0/2654930292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b00ff7a0 0x7f27b00ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:33:33.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b5017700 1 --2- 172.21.15.131:0/2654930292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b0100160 0x7f27b019b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:33:33.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b6019700 1 --2- 172.21.15.131:0/2654930292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0100b40 0x7f27b019d650 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f27a800c110 tx=0x7f27a800c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:33.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27a6ffd700 1 -- 172.21.15.131:0/2654930292 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a8017070 con 0x7f27b0100b40 2023-09-10T00:33:33.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27a6ffd700 1 -- 172.21.15.131:0/2654930292 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27a800e050 con 0x7f27b0100b40 2023-09-10T00:33:33.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b7a7c700 1 -- 172.21.15.131:0/2654930292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b019e3a0 con 0x7f27b0100b40 2023-09-10T00:33:33.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27a6ffd700 1 -- 172.21.15.131:0/2654930292 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a8007980 con 0x7f27b0100b40 2023-09-10T00:33:33.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.715+0000 7f27b7a7c700 1 -- 172.21.15.131:0/2654930292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b019e8f0 con 0x7f27b0100b40 2023-09-10T00:33:33.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.716+0000 7f27b7a7c700 1 -- 172.21.15.131:0/2654930292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b010ac30 con 0x7f27b0100b40 2023-09-10T00:33:33.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.717+0000 7f27a6ffd700 1 -- 172.21.15.131:0/2654930292 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f27a8004ad0 con 0x7f27b0100b40 2023-09-10T00:33:33.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.717+0000 7f27a6ffd700 1 --2- 172.21.15.131:0/2654930292 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f279c038630 0x7f279c03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:33.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.717+0000 7f27a6ffd700 1 -- 172.21.15.131:0/2654930292 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f27a804a850 con 0x7f27b0100b40 2023-09-10T00:33:33.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.717+0000 7f27b5818700 1 --2- 172.21.15.131:0/2654930292 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f279c038630 0x7f279c03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:33.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.718+0000 7f27b5818700 1 --2- 172.21.15.131:0/2654930292 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f279c038630 0x7f279c03aaf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009230 tx=0x7f27ac005b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:33.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.722+0000 7f27a6ffd700 1 -- 172.21.15.131:0/2654930292 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f27a801a900 con 0x7f27b0100b40 2023-09-10T00:33:33.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:33 smithi131 ceph-mon[103713]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:33.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:33 smithi044 ceph-mon[98676]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:33.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:33 smithi099 ceph-mon[102692]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:33.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.918+0000 7f27b7a7c700 1 -- 172.21.15.131:0/2654930292 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi044=x;smithi099=y;smithi131=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f27b0060a40 con 0x7f279c038630 2023-09-10T00:33:33.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.928+0000 7f27a6ffd700 1 -- 172.21.15.131:0/2654930292 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f27b0060a40 con 0x7f279c038630 2023-09-10T00:33:33.928 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled mgr update... 2023-09-10T00:33:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.930+0000 7f27b7a7c700 1 -- 172.21.15.131:0/2654930292 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f279c038630 msgr2=0x7f279c03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.930+0000 7f27b7a7c700 1 --2- 172.21.15.131:0/2654930292 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f279c038630 0x7f279c03aaf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009230 tx=0x7f27ac005b40 comp rx=0 tx=0).stop 2023-09-10T00:33:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.930+0000 7f27b7a7c700 1 -- 172.21.15.131:0/2654930292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0100b40 msgr2=0x7f27b019d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.930+0000 7f27b7a7c700 1 --2- 172.21.15.131:0/2654930292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0100b40 0x7f27b019d650 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f27a800c110 tx=0x7f27a800c4d0 comp rx=0 tx=0).stop 2023-09-10T00:33:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.930+0000 7f27b7a7c700 1 -- 172.21.15.131:0/2654930292 shutdown_connections 2023-09-10T00:33:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.930+0000 7f27b7a7c700 1 --2- 172.21.15.131:0/2654930292 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f279c038630 0x7f279c03aaf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.930+0000 7f27b7a7c700 1 --2- 172.21.15.131:0/2654930292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b0100b40 0x7f27b019d650 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.930+0000 7f27b7a7c700 1 --2- 172.21.15.131:0/2654930292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b00ff7a0 0x7f27b00ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.930+0000 7f27b7a7c700 1 --2- 172.21.15.131:0/2654930292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b0100160 0x7f27b019b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.930+0000 7f27b7a7c700 1 -- 172.21.15.131:0/2654930292 >> 172.21.15.131:0/2654930292 conn(0x7f27b00fa2b0 msgr2=0x7f27b00fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.930+0000 7f27b7a7c700 1 -- 172.21.15.131:0/2654930292 shutdown_connections 2023-09-10T00:33:33.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:33:33.930+0000 7f27b7a7c700 1 -- 172.21.15.131:0/2654930292 wait complete. 2023-09-10T00:33:34.357 DEBUG:teuthology.orchestra.run.smithi099:mgr.y> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mgr.y.service 2023-09-10T00:33:34.362 DEBUG:teuthology.orchestra.run.smithi131:mgr.z> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mgr.z.service 2023-09-10T00:33:34.364 INFO:tasks.cephadm:Deploying OSDs... 2023-09-10T00:33:34.365 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:33:34.365 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2023-09-10T00:33:34.383 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-10T00:33:34.384 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 2023-09-10T00:33:34.391 INFO:journalctl@ceph.mgr.y.smithi099.stdout:-- Logs begin at Sun 2023-09-10 00:17:22 UTC. -- 2023-09-10T00:33:34.393 INFO:journalctl@ceph.mgr.z.smithi131.stdout:-- Logs begin at Sun 2023-09-10 00:17:05 UTC. -- 2023-09-10T00:33:34.441 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-10T00:33:34.442 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-10T00:33:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 391241 Links: 1 2023-09-10T00:33:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-10T00:33:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2023-09-10T00:33:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2023-09-10 00:32:27.587555175 +0000 2023-09-10T00:33:34.443 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2023-09-10 00:27:12.236221030 +0000 2023-09-10T00:33:34.443 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2023-09-10 00:27:12.236221030 +0000 2023-09-10T00:33:34.443 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2023-09-10T00:33:34.443 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-10T00:33:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2023-09-10T00:33:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2023-09-10T00:33:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000239735 s, 2.1 MB/s 2023-09-10T00:33:34.510 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-10T00:33:34.569 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 2023-09-10T00:33:34.627 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-10T00:33:34.628 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-10T00:33:34.628 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 391341 Links: 1 2023-09-10T00:33:34.628 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-10T00:33:34.628 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2023-09-10T00:33:34.628 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2023-09-10 00:32:27.634554454 +0000 2023-09-10T00:33:34.629 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2023-09-10 00:27:12.748213145 +0000 2023-09-10T00:33:34.629 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2023-09-10 00:27:12.748213145 +0000 2023-09-10T00:33:34.629 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2023-09-10T00:33:34.629 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-10T00:33:34.693 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2023-09-10T00:33:34.694 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2023-09-10T00:33:34.694 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000221201 s, 2.3 MB/s 2023-09-10T00:33:34.695 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-10T00:33:34.753 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 2023-09-10T00:33:34.810 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-10T00:33:34.810 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-10T00:33:34.811 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 390773 Links: 1 2023-09-10T00:33:34.811 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-10T00:33:34.811 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2023-09-10T00:33:34.811 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2023-09-10 00:32:27.684553686 +0000 2023-09-10T00:33:34.811 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2023-09-10 00:27:13.272205076 +0000 2023-09-10T00:33:34.811 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2023-09-10 00:27:13.272205076 +0000 2023-09-10T00:33:34.812 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2023-09-10T00:33:34.812 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-10T00:33:34.875 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2023-09-10T00:33:34.876 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2023-09-10T00:33:34.876 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000220678 s, 2.3 MB/s 2023-09-10T00:33:34.877 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-10T00:33:34.937 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 2023-09-10T00:33:34.995 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-10T00:33:34.995 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-10T00:33:34.995 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 390868 Links: 1 2023-09-10T00:33:34.995 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-10T00:33:34.996 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2023-09-10T00:33:34.996 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2023-09-10 00:32:27.732552949 +0000 2023-09-10T00:33:34.996 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2023-09-10 00:27:13.815196714 +0000 2023-09-10T00:33:34.996 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2023-09-10 00:27:13.815196714 +0000 2023-09-10T00:33:34.996 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2023-09-10T00:33:34.997 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-10T00:33:35.061 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2023-09-10T00:33:35.061 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2023-09-10T00:33:35.062 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.00019398 s, 2.6 MB/s 2023-09-10T00:33:35.063 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-10T00:33:35.121 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-09-10T00:33:35.121 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2023-09-10T00:33:35.142 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-10T00:33:35.143 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2023-09-10T00:33:35.203 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-10T00:33:35.203 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-10T00:33:35.203 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 385508 Links: 1 2023-09-10T00:33:35.203 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-10T00:33:35.204 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-09-10T00:33:35.204 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-09-10 00:33:02.688076338 +0000 2023-09-10T00:33:35.204 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-09-10 00:26:47.410969837 +0000 2023-09-10T00:33:35.204 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-09-10 00:26:47.410969837 +0000 2023-09-10T00:33:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2023-09-10T00:33:35.205 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-10T00:33:35.270 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:34 smithi099 ceph-mon[102692]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi044=x;smithi099=y;smithi131=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:33:35.270 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:34 smithi099 ceph-mon[102692]: Saving service mgr spec with placement smithi044=x;smithi099=y;smithi131=z;count:3 2023-09-10T00:33:35.270 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:34 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:35.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:34 smithi099 ceph-mon[102692]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:35.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:34 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:35.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:34 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:35.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:34 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:35.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:34 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:35.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:34 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-10T00:33:35.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:34 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-10T00:33:35.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:34 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-10T00:33:35.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:34 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:35.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:34 smithi099 ceph-mon[102692]: Deploying daemon mgr.z on smithi131 2023-09-10T00:33:35.273 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-09-10T00:33:35.273 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-09-10T00:33:35.273 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000250521 s, 2.0 MB/s 2023-09-10T00:33:35.274 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-10T00:33:35.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:34 smithi131 ceph-mon[103713]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi044=x;smithi099=y;smithi131=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:33:35.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:34 smithi131 ceph-mon[103713]: Saving service mgr spec with placement smithi044=x;smithi099=y;smithi131=z;count:3 2023-09-10T00:33:35.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:34 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:35.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:34 smithi131 ceph-mon[103713]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:35.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:34 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:35.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:34 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:35.285 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:34 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:35.285 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:34 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:35.285 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:34 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-10T00:33:35.285 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:34 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-10T00:33:35.285 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:34 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-10T00:33:35.285 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:34 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:35.286 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:34 smithi131 ceph-mon[103713]: Deploying daemon mgr.z on smithi131 2023-09-10T00:33:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:34 smithi044 ceph-mon[98676]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi044=x;smithi099=y;smithi131=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:33:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:34 smithi044 ceph-mon[98676]: Saving service mgr spec with placement smithi044=x;smithi099=y;smithi131=z;count:3 2023-09-10T00:33:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:34 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:34 smithi044 ceph-mon[98676]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:34 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:34 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:35.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:34 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:35.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:34 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:35.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:34 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-10T00:33:35.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:34 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-10T00:33:35.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:34 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-10T00:33:35.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:34 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:35.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:34 smithi044 ceph-mon[98676]: Deploying daemon mgr.z on smithi131 2023-09-10T00:33:35.334 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2023-09-10T00:33:35.393 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-10T00:33:35.393 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-10T00:33:35.394 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 391249 Links: 1 2023-09-10T00:33:35.394 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-10T00:33:35.394 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-09-10T00:33:35.394 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-09-10 00:33:02.688076338 +0000 2023-09-10T00:33:35.394 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-09-10 00:26:47.909957767 +0000 2023-09-10T00:33:35.395 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-09-10 00:26:47.909957767 +0000 2023-09-10T00:33:35.395 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2023-09-10T00:33:35.395 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-10T00:33:35.461 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-09-10T00:33:35.462 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-09-10T00:33:35.462 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000259641 s, 2.0 MB/s 2023-09-10T00:33:35.463 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-10T00:33:35.522 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2023-09-10T00:33:35.580 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-10T00:33:35.581 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-10T00:33:35.581 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 392378 Links: 1 2023-09-10T00:33:35.581 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-10T00:33:35.581 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-09-10T00:33:35.582 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-09-10 00:33:02.688076338 +0000 2023-09-10T00:33:35.582 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-09-10 00:26:48.438944971 +0000 2023-09-10T00:33:35.582 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-09-10 00:26:48.438944971 +0000 2023-09-10T00:33:35.582 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2023-09-10T00:33:35.583 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-10T00:33:35.649 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-09-10T00:33:35.649 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-09-10T00:33:35.649 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000246629 s, 2.1 MB/s 2023-09-10T00:33:35.650 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-10T00:33:35.709 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2023-09-10T00:33:35.767 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-10T00:33:35.767 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-10T00:33:35.768 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 384979 Links: 1 2023-09-10T00:33:35.768 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-10T00:33:35.768 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-09-10T00:33:35.768 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-09-10 00:33:02.688076338 +0000 2023-09-10T00:33:35.768 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-09-10 00:26:48.958932392 +0000 2023-09-10T00:33:35.769 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-09-10 00:26:48.958932392 +0000 2023-09-10T00:33:35.769 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2023-09-10T00:33:35.769 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-10T00:33:35.833 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-09-10T00:33:35.834 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-09-10T00:33:35.834 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000216759 s, 2.4 MB/s 2023-09-10T00:33:35.835 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-10T00:33:35.893 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-10T00:33:35.893 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2023-09-10T00:33:35.912 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-10T00:33:35.912 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2023-09-10T00:33:35.969 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-10T00:33:35.969 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-10T00:33:35.969 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 392377 Links: 1 2023-09-10T00:33:35.970 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-10T00:33:35.970 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2023-09-10T00:33:35.970 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2023-09-10 00:33:35.895246497 +0000 2023-09-10T00:33:35.970 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2023-09-10 00:26:47.403583343 +0000 2023-09-10T00:33:35.970 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2023-09-10 00:26:47.403583343 +0000 2023-09-10T00:33:35.970 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: - 2023-09-10T00:33:35.971 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-10T00:33:36.066 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2023-09-10T00:33:36.067 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2023-09-10T00:33:36.067 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000254095 s, 2.0 MB/s 2023-09-10T00:33:36.068 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-10T00:33:36.086 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2023-09-10T00:33:36.145 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-10T00:33:36.146 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-10T00:33:36.146 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 388322 Links: 1 2023-09-10T00:33:36.146 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-10T00:33:36.146 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2023-09-10T00:33:36.146 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2023-09-10 00:33:36.019243918 +0000 2023-09-10T00:33:36.147 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2023-09-10 00:26:47.900572934 +0000 2023-09-10T00:33:36.147 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2023-09-10 00:26:47.900572934 +0000 2023-09-10T00:33:36.147 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: - 2023-09-10T00:33:36.147 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-10T00:33:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2023-09-10T00:33:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2023-09-10T00:33:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000203966 s, 2.5 MB/s 2023-09-10T00:33:36.216 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-10T00:33:36.274 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2023-09-10T00:33:36.295 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:36 smithi131 systemd[1]: Starting Ceph mgr.z for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:33:36.311 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-10T00:33:36.312 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-10T00:33:36.312 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 387458 Links: 1 2023-09-10T00:33:36.312 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-10T00:33:36.312 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2023-09-10T00:33:36.312 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2023-09-10 00:33:36.019243918 +0000 2023-09-10T00:33:36.313 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2023-09-10 00:26:48.402562421 +0000 2023-09-10T00:33:36.313 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2023-09-10 00:26:48.402562421 +0000 2023-09-10T00:33:36.313 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: - 2023-09-10T00:33:36.313 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-10T00:33:36.377 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2023-09-10T00:33:36.377 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2023-09-10T00:33:36.377 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.00017402 s, 2.9 MB/s 2023-09-10T00:33:36.378 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-10T00:33:36.436 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2023-09-10T00:33:36.495 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-10T00:33:36.495 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-10T00:33:36.495 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 389956 Links: 1 2023-09-10T00:33:36.495 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-10T00:33:36.495 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2023-09-10T00:33:36.496 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2023-09-10 00:33:36.019243918 +0000 2023-09-10T00:33:36.496 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2023-09-10 00:26:48.933551300 +0000 2023-09-10T00:33:36.496 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2023-09-10 00:26:48.933551300 +0000 2023-09-10T00:33:36.496 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: - 2023-09-10T00:33:36.496 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-10T00:33:36.561 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2023-09-10T00:33:36.561 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2023-09-10T00:33:36.561 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000201083 s, 2.5 MB/s 2023-09-10T00:33:36.562 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-10T00:33:36.621 INFO:tasks.cephadm:Deploying osd.0 on smithi044 with /dev/vg_nvme/lv_4... 2023-09-10T00:33:36.621 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-10T00:33:36.668 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:36 smithi131 podman[105702]: 2023-09-10T00:33:36.960 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:36 smithi131 bash[105702]: 3246ccfe3906c7c9c9c0ea76885333b078613a8afcc3bf099c660645616b2867 2023-09-10T00:33:36.960 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:36 smithi131 ceph-mgr[105889]: -- 172.21.15.131:0/2990871880 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d21ff591e0 con 0x55d21ff2ec00 2023-09-10T00:33:36.961 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:36 smithi131 systemd[1]: Started Ceph mgr.z for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:33:36.961 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:36 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:36.787+0000 7fce2be48200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-10T00:33:36.961 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:36 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:36.871+0000 7fce2be48200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-10T00:33:36.961 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:36 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:36.959+0000 7fce2be48200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-10T00:33:37.194 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:37.220 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:37 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:37.051+0000 7fce2be48200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-10T00:33:37.545 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:37 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:37.220+0000 7fce2be48200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-10T00:33:37.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:37 smithi131 ceph-mon[103713]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:37.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:37.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:37.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:37.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-10T00:33:37.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-10T00:33:37.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-10T00:33:37.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:37.638 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:37 smithi099 ceph-mon[102692]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:37.638 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:37.639 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:37.639 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:37.639 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-10T00:33:37.639 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-10T00:33:37.639 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-10T00:33:37.640 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:37.670 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:37 smithi044 ceph-mon[98676]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:37.670 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:37.671 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:37.671 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:37.671 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-10T00:33:37.671 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-10T00:33:37.671 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-10T00:33:37.672 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:37.900 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:37 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:37.583+0000 7fce2be48200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-10T00:33:37.900 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:37 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:37.718+0000 7fce2be48200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-10T00:33:37.900 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:37 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:37.806+0000 7fce2be48200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-10T00:33:38.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:38 smithi099 ceph-mon[102692]: Deploying daemon mgr.y on smithi099 2023-09-10T00:33:38.607 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:38 smithi099 systemd[1]: Starting Ceph mgr.y for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:33:38.686 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-10T00:33:38.687 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-10T00:33:38.687 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2023-09-10T00:33:38.687 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2023-09-10T00:33:38.687 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109028 s, 962 MB/s 2023-09-10T00:33:38.687 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2023-09-10T00:33:38.723 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:38 smithi044 ceph-mon[98676]: Deploying daemon mgr.y on smithi099 2023-09-10T00:33:38.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:38 smithi131 ceph-mon[103713]: Deploying daemon mgr.y on smithi099 2023-09-10T00:33:38.795 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:37 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:37.899+0000 7fce2be48200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-10T00:33:38.795 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:38 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:38.317+0000 7fce2be48200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-10T00:33:38.919 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:38 smithi099 podman[104164]: 2023-09-10T00:33:38.919 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:38 smithi099 bash[104164]: 23557bb627c99ed2f850c63ceb100ea0c47cca18edea36f50569b10159fd76cd 2023-09-10T00:33:38.920 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:38 smithi099 ceph-mgr[104203]: -- 172.21.15.99:0/2167407531 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5581b98db1e0 con 0x5581b98b0800 2023-09-10T00:33:39.141 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi044:vg_nvme/lv_4 2023-09-10T00:33:39.259 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:38 smithi099 systemd[1]: Started Ceph mgr.y for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:33:39.259 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:38 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:38.990+0000 7f8e14d18200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-10T00:33:39.260 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:39 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:39.070+0000 7f8e14d18200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-10T00:33:39.260 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:39 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:39.162+0000 7f8e14d18200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-10T00:33:39.260 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:39 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:39.258+0000 7f8e14d18200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-10T00:33:39.295 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:38 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:38.979+0000 7fce2be48200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-10T00:33:39.295 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:39 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:39.062+0000 7fce2be48200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-10T00:33:39.606 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:39 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:39.439+0000 7f8e14d18200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-10T00:33:39.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:39 smithi099 ceph-mon[102692]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:39.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:39 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:39.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:39 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:39.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:39 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:39.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:39 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:39.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:39 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:39.705 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:33:39.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:39 smithi131 ceph-mon[103713]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:39.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:39 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:39.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:39 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:39.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:39 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:39.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:39 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:39.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:39 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:39.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:39 smithi044 ceph-mon[98676]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:39.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:39 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:39.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:39 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:39.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:39 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:39.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:39 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:39.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:39 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:40.106 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:39 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:39.815+0000 7f8e14d18200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-10T00:33:40.106 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:39 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:39.955+0000 7f8e14d18200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-10T00:33:40.106 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:40 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:40.051+0000 7f8e14d18200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-10T00:33:40.149 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:39 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:39.874+0000 7fce2be48200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-10T00:33:40.149 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:39 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:39.965+0000 7fce2be48200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-10T00:33:40.445 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:40 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:40.148+0000 7fce2be48200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-10T00:33:40.607 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:40 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:40.156+0000 7f8e14d18200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-10T00:33:40.795 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:40 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:40.444+0000 7fce2be48200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-10T00:33:40.795 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:40 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:40.683+0000 7fce2be48200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-10T00:33:40.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.917+0000 7fb8d5881700 1 -- 172.21.15.44:0/276784590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d0101160 msgr2=0x7fb8d0107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.918+0000 7fb8d5881700 1 --2- 172.21.15.44:0/276784590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d0101160 0x7fb8d0107a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800b0d0 tx=0x7fb8c800b3e0 comp rx=0 tx=0).stop 2023-09-10T00:33:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.918+0000 7fb8d5881700 1 -- 172.21.15.44:0/276784590 shutdown_connections 2023-09-10T00:33:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.918+0000 7fb8d5881700 1 --2- 172.21.15.44:0/276784590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d0101160 0x7fb8d0107a10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.918+0000 7fb8d5881700 1 --2- 172.21.15.44:0/276784590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8d00ff5c0 0x7fb8d00ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.918+0000 7fb8d5881700 1 --2- 172.21.15.44:0/276784590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8d01007c0 0x7fb8d0100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:40.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.918+0000 7fb8d5881700 1 -- 172.21.15.44:0/276784590 >> 172.21.15.44:0/276784590 conn(0x7fb8d00fab80 msgr2=0x7fb8d00fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:40.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.918+0000 7fb8d5881700 1 -- 172.21.15.44:0/276784590 shutdown_connections 2023-09-10T00:33:40.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.918+0000 7fb8d5881700 1 -- 172.21.15.44:0/276784590 wait complete. 2023-09-10T00:33:40.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.918+0000 7fb8d5881700 1 Processor -- start 2023-09-10T00:33:40.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8d5881700 1 -- start start 2023-09-10T00:33:40.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8d5881700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8d00ff5c0 0x7fb8d0103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:40.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8d5881700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8d01007c0 0x7fb8d0101ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:40.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8d5881700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d0101160 0x7fb8d01020e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:40.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8d5881700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0102860 con 0x7fb8d0101160 2023-09-10T00:33:40.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8d5881700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d01029d0 con 0x7fb8d00ff5c0 2023-09-10T00:33:40.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8d5881700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d01a03f0 con 0x7fb8d01007c0 2023-09-10T00:33:40.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8cf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d0101160 0x7fb8d01020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:40.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8ceffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8d00ff5c0 0x7fb8d0103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:40.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8ce7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8d01007c0 0x7fb8d0101ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:40.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8cf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d0101160 0x7fb8d01020e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45396/0 (socket says 172.21.15.44:45396) 2023-09-10T00:33:40.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8ce7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8d01007c0 0x7fb8d0101ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:52276/0 (socket says 172.21.15.44:52276) 2023-09-10T00:33:40.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8ceffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8d00ff5c0 0x7fb8d0103550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:47132/0 (socket says 172.21.15.44:47132) 2023-09-10T00:33:40.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8cf7fe700 1 -- 172.21.15.44:0/3387987856 learned_addr learned my addr 172.21.15.44:0/3387987856 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:33:40.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8cf7fe700 1 -- 172.21.15.44:0/3387987856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8d01007c0 msgr2=0x7fb8d0101ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:40.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8cf7fe700 1 --2- 172.21.15.44:0/3387987856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8d01007c0 0x7fb8d0101ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:40.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8cf7fe700 1 -- 172.21.15.44:0/3387987856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8d00ff5c0 msgr2=0x7fb8d0103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:40.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8cf7fe700 1 --2- 172.21.15.44:0/3387987856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8d00ff5c0 0x7fb8d0103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:40.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8cf7fe700 1 -- 172.21.15.44:0/3387987856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c8009d00 con 0x7fb8d0101160 2023-09-10T00:33:40.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8ceffd700 1 --2- 172.21.15.44:0/3387987856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8d00ff5c0 0x7fb8d0103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:33:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8cf7fe700 1 --2- 172.21.15.44:0/3387987856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d0101160 0x7fb8d01020e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009fd0 tx=0x7fb8c800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.919+0000 7fb8d487f700 1 -- 172.21.15.44:0/3387987856 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c800e040 con 0x7fb8d0101160 2023-09-10T00:33:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.920+0000 7fb8d5881700 1 -- 172.21.15.44:0/3387987856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d01a0670 con 0x7fb8d0101160 2023-09-10T00:33:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.920+0000 7fb8d487f700 1 -- 172.21.15.44:0/3387987856 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8c80094c0 con 0x7fb8d0101160 2023-09-10T00:33:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.920+0000 7fb8d487f700 1 -- 172.21.15.44:0/3387987856 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c80052a0 con 0x7fb8d0101160 2023-09-10T00:33:40.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.920+0000 7fb8d5881700 1 -- 172.21.15.44:0/3387987856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d01a0a90 con 0x7fb8d0101160 2023-09-10T00:33:40.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.920+0000 7fb8d487f700 1 -- 172.21.15.44:0/3387987856 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fb8c8005400 con 0x7fb8d0101160 2023-09-10T00:33:40.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.920+0000 7fb8d5881700 1 -- 172.21.15.44:0/3387987856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d010ac80 con 0x7fb8d0101160 2023-09-10T00:33:40.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.921+0000 7fb8d487f700 1 --2- 172.21.15.44:0/3387987856 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb8b8038630 0x7fb8b803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:33:40.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.921+0000 7fb8d487f700 1 -- 172.21.15.44:0/3387987856 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb8c804c4f0 con 0x7fb8d0101160 2023-09-10T00:33:40.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.921+0000 7fb8ceffd700 1 --2- 172.21.15.44:0/3387987856 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb8b8038630 0x7fb8b803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:33:40.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.921+0000 7fb8ceffd700 1 --2- 172.21.15.44:0/3387987856 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb8b8038630 0x7fb8b803aaf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0100620 tx=0x7fb8c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:33:40.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:40.925+0000 7fb8d487f700 1 -- 172.21.15.44:0/3387987856 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb8c8011390 con 0x7fb8d0101160 2023-09-10T00:33:41.096 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:40 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:40.834+0000 7fce2be48200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-10T00:33:41.106 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:40 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:40.612+0000 7f8e14d18200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-10T00:33:41.134 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:41.134+0000 7fb8d5881700 1 -- 172.21.15.44:0/3387987856 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8d0069230 con 0x7fb8b8038630 2023-09-10T00:33:41.545 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:41.095+0000 7fce2be48200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-10T00:33:41.606 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:41.321+0000 7f8e14d18200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-10T00:33:41.606 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:41.405+0000 7f8e14d18200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-10T00:33:41.977 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:41 smithi044 ceph-mon[98676]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:41.978 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:41.978 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:41.978 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:41.978 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:41.979 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:41.979 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:33:41.979 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:33:41.979 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:41.979 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:41.980 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:41.980 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:41.980 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-10T00:33:41.980 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-10T00:33:41.980 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:41 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:42.045 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:41.801+0000 7fce2be48200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-10T00:33:42.045 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:41.931+0000 7fce2be48200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-10T00:33:42.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-mon[103713]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:42.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:42.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:42.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:42.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:42.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:42.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:33:42.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:33:42.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:42.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:42.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:42.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:42.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-10T00:33:42.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-10T00:33:42.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:41 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:42.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-mon[102692]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:42.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:42.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:42.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:42.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:42.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:42.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:33:42.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:33:42.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:42.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:42.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:42.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:42.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-10T00:33:42.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-10T00:33:42.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:41 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:42.433 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:42 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:42.171+0000 7f8e14d18200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-10T00:33:42.433 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:42 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:42.261+0000 7f8e14d18200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-10T00:33:42.450 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:42 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:42.123+0000 7fce2be48200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-10T00:33:42.450 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:42 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:42.234+0000 7fce2be48200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-10T00:33:42.711 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:42 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:42.432+0000 7f8e14d18200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-10T00:33:42.718 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:42 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:42.449+0000 7fce2be48200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-10T00:33:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:42 smithi131 ceph-mon[103713]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:33:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:42 smithi131 ceph-mon[103713]: Reconfiguring mgr.x (unknown last config time)... 2023-09-10T00:33:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:42 smithi131 ceph-mon[103713]: Reconfiguring daemon mgr.x on smithi044 2023-09-10T00:33:43.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:42 smithi044 ceph-mon[98676]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:33:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:42 smithi044 ceph-mon[98676]: Reconfiguring mgr.x (unknown last config time)... 2023-09-10T00:33:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:42 smithi044 ceph-mon[98676]: Reconfiguring daemon mgr.x on smithi044 2023-09-10T00:33:43.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:42 smithi099 ceph-mon[102692]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:33:43.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:42 smithi099 ceph-mon[102692]: Reconfiguring mgr.x (unknown last config time)... 2023-09-10T00:33:43.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:42 smithi099 ceph-mon[102692]: Reconfiguring daemon mgr.x on smithi044 2023-09-10T00:33:43.085 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:42 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:42.711+0000 7f8e14d18200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-10T00:33:43.085 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:42 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:42.936+0000 7f8e14d18200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-10T00:33:43.085 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:43 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:43.083+0000 7f8e14d18200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-10T00:33:43.606 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:43 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:43.349+0000 7f8e14d18200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-10T00:33:43.719 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:33:43 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:33:43.311+0000 7fce2be48200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-10T00:33:44.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:43 smithi131 ceph-mon[103713]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:44.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:44.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:44.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:44.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:44.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:44.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:44.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:43 smithi131 ceph-mon[103713]: Standby manager daemon z started 2023-09-10T00:33:44.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:43 smithi131 ceph-mon[103713]: from='mgr.? 172.21.15.131:0/472030257' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-10T00:33:44.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:43 smithi131 ceph-mon[103713]: from='mgr.? 172.21.15.131:0/472030257' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-10T00:33:44.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:43 smithi131 ceph-mon[103713]: from='mgr.? 172.21.15.131:0/472030257' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-10T00:33:44.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:43 smithi131 ceph-mon[103713]: from='mgr.? 172.21.15.131:0/472030257' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-10T00:33:44.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:43 smithi099 ceph-mon[102692]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:44.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:44.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:44.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:44.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:44.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:44.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:44.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:43 smithi099 ceph-mon[102692]: Standby manager daemon z started 2023-09-10T00:33:44.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:43 smithi099 ceph-mon[102692]: from='mgr.? 172.21.15.131:0/472030257' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-10T00:33:44.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:43 smithi099 ceph-mon[102692]: from='mgr.? 172.21.15.131:0/472030257' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-10T00:33:44.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:43 smithi099 ceph-mon[102692]: from='mgr.? 172.21.15.131:0/472030257' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-10T00:33:44.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:43 smithi099 ceph-mon[102692]: from='mgr.? 172.21.15.131:0/472030257' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-10T00:33:44.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:43 smithi044 ceph-mon[98676]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:44.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:44.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:44.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:44.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:44.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:44.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:44.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:43 smithi044 ceph-mon[98676]: Standby manager daemon z started 2023-09-10T00:33:44.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:43 smithi044 ceph-mon[98676]: from='mgr.? 172.21.15.131:0/472030257' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-10T00:33:44.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:43 smithi044 ceph-mon[98676]: from='mgr.? 172.21.15.131:0/472030257' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-10T00:33:44.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:43 smithi044 ceph-mon[98676]: from='mgr.? 172.21.15.131:0/472030257' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-10T00:33:44.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:43 smithi044 ceph-mon[98676]: from='mgr.? 172.21.15.131:0/472030257' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-10T00:33:44.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:44.201+0000 7fb8d487f700 1 -- 172.21.15.44:0/3387987856 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 14) v1 ==== 90295+0+0 (secure 0 0 0) 0x7fb8c8025920 con 0x7fb8d0101160 2023-09-10T00:33:44.356 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:44 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:44.047+0000 7f8e14d18200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-10T00:33:44.356 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:44 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:44.178+0000 7f8e14d18200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-10T00:33:44.701 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:44 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:44.371+0000 7f8e14d18200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-10T00:33:44.701 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:44 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:44.484+0000 7f8e14d18200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-10T00:33:44.701 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:44 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:44.700+0000 7f8e14d18200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-10T00:33:45.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:45 smithi131 ceph-mon[103713]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:45.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:45 smithi131 ceph-mon[103713]: mgrmap e14: x(active, since 88s), standbys: z 2023-09-10T00:33:45.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-10T00:33:45.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:45 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3658522875' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a7d2e1e-5817-489e-af0e-72e9bcccc119"}]: dispatch 2023-09-10T00:33:45.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:45 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a7d2e1e-5817-489e-af0e-72e9bcccc119"}]: dispatch 2023-09-10T00:33:45.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:45 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a7d2e1e-5817-489e-af0e-72e9bcccc119"}]': finished 2023-09-10T00:33:45.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:45 smithi131 ceph-mon[103713]: osdmap e5: 1 total, 0 up, 1 in 2023-09-10T00:33:45.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:33:45.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:45 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3445666947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:33:45.554 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:45 smithi099 ceph-mon[102692]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:45.554 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:45 smithi099 ceph-mon[102692]: mgrmap e14: x(active, since 88s), standbys: z 2023-09-10T00:33:45.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-10T00:33:45.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:45 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3658522875' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a7d2e1e-5817-489e-af0e-72e9bcccc119"}]: dispatch 2023-09-10T00:33:45.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:45 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a7d2e1e-5817-489e-af0e-72e9bcccc119"}]: dispatch 2023-09-10T00:33:45.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:45 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a7d2e1e-5817-489e-af0e-72e9bcccc119"}]': finished 2023-09-10T00:33:45.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:45 smithi099 ceph-mon[102692]: osdmap e5: 1 total, 0 up, 1 in 2023-09-10T00:33:45.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:33:45.556 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:45 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3445666947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:33:45.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:45 smithi044 ceph-mon[98676]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:45.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:45 smithi044 ceph-mon[98676]: mgrmap e14: x(active, since 88s), standbys: z 2023-09-10T00:33:45.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-10T00:33:45.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:45 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3658522875' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a7d2e1e-5817-489e-af0e-72e9bcccc119"}]: dispatch 2023-09-10T00:33:45.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:45 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a7d2e1e-5817-489e-af0e-72e9bcccc119"}]: dispatch 2023-09-10T00:33:45.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:45 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a7d2e1e-5817-489e-af0e-72e9bcccc119"}]': finished 2023-09-10T00:33:45.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:45 smithi044 ceph-mon[98676]: osdmap e5: 1 total, 0 up, 1 in 2023-09-10T00:33:45.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:33:45.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:45 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3445666947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:33:45.856 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:33:45 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:33:45.554+0000 7f8e14d18200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-10T00:33:46.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:46.397+0000 7fb8d487f700 1 -- 172.21.15.44:0/3387987856 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 15) v1 ==== 135203+0+0 (secure 0 0 0) 0x7fb8c802a880 con 0x7fb8d0101160 2023-09-10T00:33:46.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:46 smithi131 ceph-mon[103713]: Standby manager daemon y started 2023-09-10T00:33:46.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:46 smithi131 ceph-mon[103713]: from='mgr.? 172.21.15.99:0/4097033424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-10T00:33:46.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:46 smithi131 ceph-mon[103713]: from='mgr.? 172.21.15.99:0/4097033424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-10T00:33:46.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:46 smithi131 ceph-mon[103713]: from='mgr.? 172.21.15.99:0/4097033424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-10T00:33:46.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:46 smithi131 ceph-mon[103713]: from='mgr.? 172.21.15.99:0/4097033424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-10T00:33:46.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:46 smithi044 ceph-mon[98676]: Standby manager daemon y started 2023-09-10T00:33:46.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:46 smithi044 ceph-mon[98676]: from='mgr.? 172.21.15.99:0/4097033424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-10T00:33:46.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:46 smithi044 ceph-mon[98676]: from='mgr.? 172.21.15.99:0/4097033424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-10T00:33:46.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:46 smithi044 ceph-mon[98676]: from='mgr.? 172.21.15.99:0/4097033424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-10T00:33:46.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:46 smithi044 ceph-mon[98676]: from='mgr.? 172.21.15.99:0/4097033424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-10T00:33:46.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:46 smithi099 ceph-mon[102692]: Standby manager daemon y started 2023-09-10T00:33:46.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:46 smithi099 ceph-mon[102692]: from='mgr.? 172.21.15.99:0/4097033424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-10T00:33:46.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:46 smithi099 ceph-mon[102692]: from='mgr.? 172.21.15.99:0/4097033424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-10T00:33:46.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:46 smithi099 ceph-mon[102692]: from='mgr.? 172.21.15.99:0/4097033424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-10T00:33:46.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:46 smithi099 ceph-mon[102692]: from='mgr.? 172.21.15.99:0/4097033424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-10T00:33:47.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:47 smithi131 ceph-mon[103713]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:47.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:47 smithi131 ceph-mon[103713]: mgrmap e15: x(active, since 90s), standbys: z, y 2023-09-10T00:33:47.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:47 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-10T00:33:47.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:47 smithi044 ceph-mon[98676]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:47.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:47 smithi044 ceph-mon[98676]: mgrmap e15: x(active, since 90s), standbys: z, y 2023-09-10T00:33:47.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:47 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-10T00:33:47.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:47 smithi099 ceph-mon[102692]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:47.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:47 smithi099 ceph-mon[102692]: mgrmap e15: x(active, since 90s), standbys: z, y 2023-09-10T00:33:47.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:47 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-10T00:33:49.664 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:49 smithi044 ceph-mon[98676]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:49.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:49 smithi131 ceph-mon[103713]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:49.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:49 smithi099 ceph-mon[102692]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:51.644 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:51 smithi044 ceph-mon[98676]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:51.644 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:51 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-10T00:33:51.644 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:51 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:51.645 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:51 smithi044 ceph-mon[98676]: Deploying daemon osd.0 on smithi044 2023-09-10T00:33:51.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:51 smithi131 ceph-mon[103713]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:51.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:51 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-10T00:33:51.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:51 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:51.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:51 smithi131 ceph-mon[103713]: Deploying daemon osd.0 on smithi044 2023-09-10T00:33:51.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:51 smithi099 ceph-mon[102692]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:51.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:51 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-10T00:33:51.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:51 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:51.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:51 smithi099 ceph-mon[102692]: Deploying daemon osd.0 on smithi044 2023-09-10T00:33:53.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:53 smithi131 ceph-mon[103713]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:53.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:53 smithi044 ceph-mon[98676]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:53.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:53 smithi099 ceph-mon[102692]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:55.708 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:55 smithi044 ceph-mon[98676]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:55.708 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:55 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:55.709 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:55 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:55.709 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:55 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:55.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:55 smithi131 ceph-mon[103713]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:55.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:55 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:55.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:55 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:55.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:55 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:55.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:55 smithi099 ceph-mon[102692]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:55.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:55 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:55.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:55 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:55.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:55 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:57.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:57 smithi044 ceph-mon[98676]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:57.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:57 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:57.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:57 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:57.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:57 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:57.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:57 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:57.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:57 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:57.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:57.602+0000 7fb8d487f700 1 -- 172.21.15.44:0/3387987856 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb8d0069230 con 0x7fb8b8038630 2023-09-10T00:33:57.603 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 0 on host 'smithi044' 2023-09-10T00:33:57.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:57.605+0000 7fb8d5881700 1 -- 172.21.15.44:0/3387987856 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb8b8038630 msgr2=0x7fb8b803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:57.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:57.605+0000 7fb8d5881700 1 --2- 172.21.15.44:0/3387987856 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb8b8038630 0x7fb8b803aaf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0100620 tx=0x7fb8c0009260 comp rx=0 tx=0).stop 2023-09-10T00:33:57.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:57.605+0000 7fb8d5881700 1 -- 172.21.15.44:0/3387987856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d0101160 msgr2=0x7fb8d01020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:33:57.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:57.605+0000 7fb8d5881700 1 --2- 172.21.15.44:0/3387987856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d0101160 0x7fb8d01020e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009fd0 tx=0x7fb8c800ccc0 comp rx=0 tx=0).stop 2023-09-10T00:33:57.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:57.605+0000 7fb8d5881700 1 -- 172.21.15.44:0/3387987856 shutdown_connections 2023-09-10T00:33:57.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:57.605+0000 7fb8d5881700 1 --2- 172.21.15.44:0/3387987856 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb8b8038630 0x7fb8b803aaf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:57.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:57.605+0000 7fb8d5881700 1 --2- 172.21.15.44:0/3387987856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d0101160 0x7fb8d01020e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:57.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:57.605+0000 7fb8d5881700 1 --2- 172.21.15.44:0/3387987856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8d00ff5c0 0x7fb8d0103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:57.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:57.605+0000 7fb8d5881700 1 --2- 172.21.15.44:0/3387987856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8d01007c0 0x7fb8d0101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:33:57.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:57.605+0000 7fb8d5881700 1 -- 172.21.15.44:0/3387987856 >> 172.21.15.44:0/3387987856 conn(0x7fb8d00fab80 msgr2=0x7fb8d0068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:33:57.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:57.605+0000 7fb8d5881700 1 -- 172.21.15.44:0/3387987856 shutdown_connections 2023-09-10T00:33:57.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:33:57.605+0000 7fb8d5881700 1 -- 172.21.15.44:0/3387987856 wait complete. 2023-09-10T00:33:57.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:57 smithi131 ceph-mon[103713]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:57.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:57 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:57.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:57 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:57.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:57 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:57.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:57 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:57.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:57 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:57.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:57 smithi099 ceph-mon[102692]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:57.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:57 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:57.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:57 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:57.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:57 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:57.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:57 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:57.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:57 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:57.999 DEBUG:teuthology.orchestra.run.smithi044:osd.0> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.0.service 2023-09-10T00:33:58.002 INFO:tasks.cephadm:Deploying osd.1 on smithi099 with /dev/vg_nvme/lv_4... 2023-09-10T00:33:58.002 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-10T00:33:58.035 INFO:journalctl@ceph.osd.0.smithi044.stdout:-- Logs begin at Sun 2023-09-10 00:17:15 UTC. -- 2023-09-10T00:33:58.563 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-10T00:33:58.794 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:58 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:58.794 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:58 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:58.795 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:58 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:58.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:58 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:58.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:58 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:58.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:58 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:59.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:58 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:59.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:58 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:33:59.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:58 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:59.055 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 00:33:58 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-0[121843]: 2023-09-10T00:33:58.907+0000 7f7c733be640 -1 osd.0 0 log_to_monitors true 2023-09-10T00:33:59.791 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:59 smithi099 ceph-mon[102692]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:33:59.792 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:59 smithi099 ceph-mon[102692]: from='osd.0 [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-10T00:33:59.792 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:33:59 smithi099 ceph-mon[102692]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-10T00:34:00.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:59 smithi131 ceph-mon[103713]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:34:00.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:59 smithi131 ceph-mon[103713]: from='osd.0 [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-10T00:34:00.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:33:59 smithi131 ceph-mon[103713]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-10T00:34:00.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:59 smithi044 ceph-mon[98676]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:34:00.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:59 smithi044 ceph-mon[98676]: from='osd.0 [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-10T00:34:00.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:33:59 smithi044 ceph-mon[98676]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-10T00:34:00.285 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-10T00:34:00.285 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-10T00:34:00.285 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2023-09-10T00:34:00.285 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2023-09-10T00:34:00.286 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111196 s, 943 MB/s 2023-09-10T00:34:00.286 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2023-09-10T00:34:00.583 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:00 smithi044 ceph-mon[98676]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-10T00:34:00.583 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:00 smithi044 ceph-mon[98676]: osdmap e6: 1 total, 0 up, 1 in 2023-09-10T00:34:00.584 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:00 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:34:00.584 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:00 smithi044 ceph-mon[98676]: from='osd.0 [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:34:00.584 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:00 smithi044 ceph-mon[98676]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:34:00.717 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi099:vg_nvme/lv_4 2023-09-10T00:34:00.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:00 smithi099 ceph-mon[102692]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-10T00:34:00.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:00 smithi099 ceph-mon[102692]: osdmap e6: 1 total, 0 up, 1 in 2023-09-10T00:34:00.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:00 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:34:00.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:00 smithi099 ceph-mon[102692]: from='osd.0 [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:34:00.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:00 smithi099 ceph-mon[102692]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:34:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:00 smithi131 ceph-mon[103713]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-10T00:34:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:00 smithi131 ceph-mon[103713]: osdmap e6: 1 total, 0 up, 1 in 2023-09-10T00:34:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:00 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:34:01.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:00 smithi131 ceph-mon[103713]: from='osd.0 [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:34:01.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:00 smithi131 ceph-mon[103713]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:34:01.054 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 00:34:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-0[121843]: 2023-09-10T00:34:00.581+0000 7f7c68a35700 -1 osd.0 0 waiting for initial osdmap 2023-09-10T00:34:01.055 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 00:34:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-0[121843]: 2023-09-10T00:34:00.584+0000 7f7c62826700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-10T00:34:01.272 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-10T00:34:01.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:01 smithi099 ceph-mon[102692]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:34:01.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:01 smithi099 ceph-mon[102692]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2023-09-10T00:34:01.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:01 smithi099 ceph-mon[102692]: osdmap e7: 1 total, 0 up, 1 in 2023-09-10T00:34:01.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:01 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:34:01.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:01 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:34:01.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:01 smithi099 ceph-mon[102692]: Detected new or changed devices on smithi044 2023-09-10T00:34:01.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:01 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:01.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:01 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:01.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:01 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:34:01.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:01 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi044 to 17173M 2023-09-10T00:34:01.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:01 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:01.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:01 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:01.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:01 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:01.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:01 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:02.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:01 smithi131 ceph-mon[103713]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:34:02.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:01 smithi131 ceph-mon[103713]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2023-09-10T00:34:02.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:01 smithi131 ceph-mon[103713]: osdmap e7: 1 total, 0 up, 1 in 2023-09-10T00:34:02.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:01 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:34:02.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:01 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:34:02.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:01 smithi131 ceph-mon[103713]: Detected new or changed devices on smithi044 2023-09-10T00:34:02.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:01 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:02.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:01 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:02.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:01 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:34:02.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:01 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi044 to 17173M 2023-09-10T00:34:02.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:01 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:02.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:01 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:02.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:01 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:02.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:01 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:02.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:01 smithi044 ceph-mon[98676]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:34:02.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:01 smithi044 ceph-mon[98676]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2023-09-10T00:34:02.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:01 smithi044 ceph-mon[98676]: osdmap e7: 1 total, 0 up, 1 in 2023-09-10T00:34:02.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:01 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:34:02.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:01 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:34:02.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:01 smithi044 ceph-mon[98676]: Detected new or changed devices on smithi044 2023-09-10T00:34:02.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:01 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:02.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:01 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:02.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:01 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:34:02.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:01 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi044 to 17173M 2023-09-10T00:34:02.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:01 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:02.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:01 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:02.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:01 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:02.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:01 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:02.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.456+0000 7f1f65aa3700 1 -- 172.21.15.99:0/3116857661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f600ff5c0 msgr2=0x7f1f600ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:34:02.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.456+0000 7f1f65aa3700 1 --2- 172.21.15.99:0/3116857661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f600ff5c0 0x7f1f600ff9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1f500099c0 tx=0x7f1f50009cd0 comp rx=0 tx=0).stop 2023-09-10T00:34:02.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.457+0000 7f1f65aa3700 1 -- 172.21.15.99:0/3116857661 shutdown_connections 2023-09-10T00:34:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.457+0000 7f1f65aa3700 1 --2- 172.21.15.99:0/3116857661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f60101160 0x7f1f60107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.457+0000 7f1f65aa3700 1 --2- 172.21.15.99:0/3116857661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f600ff5c0 0x7f1f600ff9e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.457+0000 7f1f65aa3700 1 --2- 172.21.15.99:0/3116857661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f601007c0 0x7f1f60100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.457+0000 7f1f65aa3700 1 -- 172.21.15.99:0/3116857661 >> 172.21.15.99:0/3116857661 conn(0x7f1f600fab80 msgr2=0x7f1f600fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:34:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.457+0000 7f1f65aa3700 1 -- 172.21.15.99:0/3116857661 shutdown_connections 2023-09-10T00:34:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.457+0000 7f1f65aa3700 1 -- 172.21.15.99:0/3116857661 wait complete. 2023-09-10T00:34:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.457+0000 7f1f65aa3700 1 Processor -- start 2023-09-10T00:34:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.457+0000 7f1f65aa3700 1 -- start start 2023-09-10T00:34:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.457+0000 7f1f65aa3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f600ff5c0 0x7f1f60196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:34:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.457+0000 7f1f65aa3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f601007c0 0x7f1f60196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:34:02.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.457+0000 7f1f65aa3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f60101160 0x7f1f6019d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:34:02.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.457+0000 7f1f65aa3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6019de90 con 0x7f1f600ff5c0 2023-09-10T00:34:02.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.457+0000 7f1f65aa3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6019dfd0 con 0x7f1f601007c0 2023-09-10T00:34:02.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f65aa3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6019e160 con 0x7f1f60101160 2023-09-10T00:34:02.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f5effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f601007c0 0x7f1f60196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:34:02.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f5effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f601007c0 0x7f1f60196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40602/0 (socket says 172.21.15.99:40602) 2023-09-10T00:34:02.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f5effd700 1 -- 172.21.15.99:0/1952377651 learned_addr learned my addr 172.21.15.99:0/1952377651 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-09-10T00:34:02.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f5ffff700 1 --2- 172.21.15.99:0/1952377651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f60101160 0x7f1f6019d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:34:02.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f5f7fe700 1 --2- 172.21.15.99:0/1952377651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f600ff5c0 0x7f1f60196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:34:02.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f5effd700 1 -- 172.21.15.99:0/1952377651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f60101160 msgr2=0x7f1f6019d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:34:02.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f5effd700 1 --2- 172.21.15.99:0/1952377651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f60101160 0x7f1f6019d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:02.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f5effd700 1 -- 172.21.15.99:0/1952377651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f600ff5c0 msgr2=0x7f1f60196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:34:02.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f5effd700 1 --2- 172.21.15.99:0/1952377651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f600ff5c0 0x7f1f60196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:02.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f5effd700 1 -- 172.21.15.99:0/1952377651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f500095f0 con 0x7f1f601007c0 2023-09-10T00:34:02.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f5ffff700 1 --2- 172.21.15.99:0/1952377651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f60101160 0x7f1f6019d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:34:02.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f5f7fe700 1 --2- 172.21.15.99:0/1952377651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f600ff5c0 0x7f1f60196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:34:02.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f5effd700 1 --2- 172.21.15.99:0/1952377651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f601007c0 0x7f1f60196840 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400c920 tx=0x7f1f5400cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:34:02.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f5cff9700 1 -- 172.21.15.99:0/1952377651 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54007b00 con 0x7f1f601007c0 2023-09-10T00:34:02.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f5cff9700 1 -- 172.21.15.99:0/1952377651 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f54004cc0 con 0x7f1f601007c0 2023-09-10T00:34:02.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f65aa3700 1 -- 172.21.15.99:0/1952377651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6019e440 con 0x7f1f601007c0 2023-09-10T00:34:02.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f5cff9700 1 -- 172.21.15.99:0/1952377651 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54005520 con 0x7f1f601007c0 2023-09-10T00:34:02.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.458+0000 7f1f65aa3700 1 -- 172.21.15.99:0/1952377651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6019e940 con 0x7f1f601007c0 2023-09-10T00:34:02.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.459+0000 7f1f65aa3700 1 -- 172.21.15.99:0/1952377651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f60107780 con 0x7f1f601007c0 2023-09-10T00:34:02.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.461+0000 7f1f5cff9700 1 -- 172.21.15.99:0/1952377651 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 15) v1 ==== 135203+0+0 (secure 0 0 0) 0x7f1f54004750 con 0x7f1f601007c0 2023-09-10T00:34:02.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.462+0000 7f1f5cff9700 1 --2- 172.21.15.99:0/1952377651 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1f480a05b0 0x7f1f480a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:34:02.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.462+0000 7f1f5cff9700 1 -- 172.21.15.99:0/1952377651 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f1f540ca150 con 0x7f1f601007c0 2023-09-10T00:34:02.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.462+0000 7f1f5f7fe700 1 --2- 172.21.15.99:0/1952377651 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1f480a05b0 0x7f1f480a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:34:02.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.462+0000 7f1f5f7fe700 1 --2- 172.21.15.99:0/1952377651 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1f480a05b0 0x7f1f480a2a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f50006010 tx=0x7f1f50000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:34:02.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.465+0000 7f1f5cff9700 1 -- 172.21.15.99:0/1952377651 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1f54095650 con 0x7f1f601007c0 2023-09-10T00:34:02.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:02.662+0000 7f1f65aa3700 1 -- 172.21.15.99:0/1952377651 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f6002d030 con 0x7f1f480a05b0 2023-09-10T00:34:02.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:02 smithi099 ceph-mon[102692]: purged_snaps scrub starts 2023-09-10T00:34:02.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:02 smithi099 ceph-mon[102692]: purged_snaps scrub ok 2023-09-10T00:34:02.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:02 smithi099 ceph-mon[102692]: osd.0 [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980] boot 2023-09-10T00:34:02.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:02 smithi099 ceph-mon[102692]: osdmap e8: 1 total, 1 up, 1 in 2023-09-10T00:34:02.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:02 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:34:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:02 smithi131 ceph-mon[103713]: purged_snaps scrub starts 2023-09-10T00:34:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:02 smithi131 ceph-mon[103713]: purged_snaps scrub ok 2023-09-10T00:34:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:02 smithi131 ceph-mon[103713]: osd.0 [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980] boot 2023-09-10T00:34:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:02 smithi131 ceph-mon[103713]: osdmap e8: 1 total, 1 up, 1 in 2023-09-10T00:34:03.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:02 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:34:03.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:02 smithi044 ceph-mon[98676]: purged_snaps scrub starts 2023-09-10T00:34:03.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:02 smithi044 ceph-mon[98676]: purged_snaps scrub ok 2023-09-10T00:34:03.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:02 smithi044 ceph-mon[98676]: osd.0 [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980] boot 2023-09-10T00:34:03.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:02 smithi044 ceph-mon[98676]: osdmap e8: 1 total, 1 up, 1 in 2023-09-10T00:34:03.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:02 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:34:03.720 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:03 smithi099 ceph-mon[102692]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:34:03.720 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:03 smithi099 ceph-mon[102692]: osdmap e9: 1 total, 1 up, 1 in 2023-09-10T00:34:03.721 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:03 smithi099 ceph-mon[102692]: from='client.24236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:34:03.721 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:34:03.721 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:34:03.721 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:04.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:03 smithi131 ceph-mon[103713]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:34:04.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:03 smithi131 ceph-mon[103713]: osdmap e9: 1 total, 1 up, 1 in 2023-09-10T00:34:04.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:03 smithi131 ceph-mon[103713]: from='client.24236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:34:04.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:03 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:34:04.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:03 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:34:04.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:03 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:04.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:03 smithi044 ceph-mon[98676]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-10T00:34:04.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:03 smithi044 ceph-mon[98676]: osdmap e9: 1 total, 1 up, 1 in 2023-09-10T00:34:04.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:03 smithi044 ceph-mon[98676]: from='client.24236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:34:04.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:03 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:34:04.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:03 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:34:04.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:03 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:05.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:05 smithi099 ceph-mon[102692]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:05.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:05 smithi099 ceph-mon[102692]: from='client.? 172.21.15.99:0/320512251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf261516-01eb-4f04-b7be-19f32e9e55e8"}]: dispatch 2023-09-10T00:34:05.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:05 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf261516-01eb-4f04-b7be-19f32e9e55e8"}]: dispatch 2023-09-10T00:34:05.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:05 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf261516-01eb-4f04-b7be-19f32e9e55e8"}]': finished 2023-09-10T00:34:05.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:05 smithi099 ceph-mon[102692]: osdmap e10: 2 total, 1 up, 2 in 2023-09-10T00:34:05.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:05 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:05.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:05 smithi099 ceph-mon[102692]: from='client.? 172.21.15.99:0/352311931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:34:06.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:05 smithi131 ceph-mon[103713]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:06.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:05 smithi131 ceph-mon[103713]: from='client.? 172.21.15.99:0/320512251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf261516-01eb-4f04-b7be-19f32e9e55e8"}]: dispatch 2023-09-10T00:34:06.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:05 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf261516-01eb-4f04-b7be-19f32e9e55e8"}]: dispatch 2023-09-10T00:34:06.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:05 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf261516-01eb-4f04-b7be-19f32e9e55e8"}]': finished 2023-09-10T00:34:06.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:05 smithi131 ceph-mon[103713]: osdmap e10: 2 total, 1 up, 2 in 2023-09-10T00:34:06.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:05 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:06.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:05 smithi131 ceph-mon[103713]: from='client.? 172.21.15.99:0/352311931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:34:06.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:05 smithi044 ceph-mon[98676]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:06.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:05 smithi044 ceph-mon[98676]: from='client.? 172.21.15.99:0/320512251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf261516-01eb-4f04-b7be-19f32e9e55e8"}]: dispatch 2023-09-10T00:34:06.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:05 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf261516-01eb-4f04-b7be-19f32e9e55e8"}]: dispatch 2023-09-10T00:34:06.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:05 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf261516-01eb-4f04-b7be-19f32e9e55e8"}]': finished 2023-09-10T00:34:06.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:05 smithi044 ceph-mon[98676]: osdmap e10: 2 total, 1 up, 2 in 2023-09-10T00:34:06.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:05 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:06.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:05 smithi044 ceph-mon[98676]: from='client.? 172.21.15.99:0/352311931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:34:07.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:07 smithi099 ceph-mon[102692]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:08.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:07 smithi131 ceph-mon[103713]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:08.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:07 smithi044 ceph-mon[98676]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:09.853 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:09 smithi099 ceph-mon[102692]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:10.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:09 smithi131 ceph-mon[103713]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:10.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:09 smithi044 ceph-mon[98676]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:11.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:11 smithi099 ceph-mon[102692]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:11.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-10T00:34:11.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:12.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:11 smithi131 ceph-mon[103713]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:12.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-10T00:34:12.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:12.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:11 smithi044 ceph-mon[98676]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:12.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-10T00:34:12.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:12 smithi131 ceph-mon[103713]: Deploying daemon osd.1 on smithi099 2023-09-10T00:34:13.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:12 smithi044 ceph-mon[98676]: Deploying daemon osd.1 on smithi099 2023-09-10T00:34:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:12 smithi099 ceph-mon[102692]: Deploying daemon osd.1 on smithi099 2023-09-10T00:34:13.836 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:13 smithi099 ceph-mon[102692]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:13 smithi131 ceph-mon[103713]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:14.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:13 smithi044 ceph-mon[98676]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:15.691 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:15 smithi099 ceph-mon[102692]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:15 smithi131 ceph-mon[103713]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:16.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:15 smithi044 ceph-mon[98676]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:16.850 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:16.850 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:34:16.851 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:34:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:17.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:34:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:18.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:17 smithi099 ceph-mon[102692]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:18.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:17 smithi131 ceph-mon[103713]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:18.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:17 smithi044 ceph-mon[98676]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:18.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:18.324+0000 7f1f5cff9700 1 -- 172.21.15.99:0/1952377651 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1f6002d030 con 0x7f1f480a05b0 2023-09-10T00:34:18.325 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 1 on host 'smithi099' 2023-09-10T00:34:18.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:18.326+0000 7f1f65aa3700 1 -- 172.21.15.99:0/1952377651 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1f480a05b0 msgr2=0x7f1f480a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:34:18.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:18.326+0000 7f1f65aa3700 1 --2- 172.21.15.99:0/1952377651 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1f480a05b0 0x7f1f480a2a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f50006010 tx=0x7f1f50000bc0 comp rx=0 tx=0).stop 2023-09-10T00:34:18.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:18.326+0000 7f1f65aa3700 1 -- 172.21.15.99:0/1952377651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f601007c0 msgr2=0x7f1f60196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:34:18.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:18.326+0000 7f1f65aa3700 1 --2- 172.21.15.99:0/1952377651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f601007c0 0x7f1f60196840 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400c920 tx=0x7f1f5400cce0 comp rx=0 tx=0).stop 2023-09-10T00:34:18.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:18.326+0000 7f1f65aa3700 1 -- 172.21.15.99:0/1952377651 shutdown_connections 2023-09-10T00:34:18.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:18.326+0000 7f1f65aa3700 1 --2- 172.21.15.99:0/1952377651 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1f480a05b0 0x7f1f480a2a70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:18.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:18.326+0000 7f1f65aa3700 1 --2- 172.21.15.99:0/1952377651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f60101160 0x7f1f6019d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:18.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:18.326+0000 7f1f65aa3700 1 --2- 172.21.15.99:0/1952377651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f600ff5c0 0x7f1f60196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:18.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:18.326+0000 7f1f65aa3700 1 --2- 172.21.15.99:0/1952377651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f601007c0 0x7f1f60196840 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:18.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:18.326+0000 7f1f65aa3700 1 -- 172.21.15.99:0/1952377651 >> 172.21.15.99:0/1952377651 conn(0x7f1f600fab80 msgr2=0x7f1f600fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:34:18.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:18.326+0000 7f1f65aa3700 1 -- 172.21.15.99:0/1952377651 shutdown_connections 2023-09-10T00:34:18.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:34:18.326+0000 7f1f65aa3700 1 -- 172.21.15.99:0/1952377651 wait complete. 2023-09-10T00:34:18.745 DEBUG:teuthology.orchestra.run.smithi099:osd.1> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.1.service 2023-09-10T00:34:18.747 INFO:tasks.cephadm:Deploying osd.2 on smithi131 with /dev/vg_nvme/lv_4... 2023-09-10T00:34:18.747 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-10T00:34:18.780 INFO:journalctl@ceph.osd.1.smithi099.stdout:-- Logs begin at Sun 2023-09-10 00:17:22 UTC. -- 2023-09-10T00:34:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:18 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:18 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:18 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:19.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:18 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:19.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:18 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:19.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:18 smithi131 ceph-mon[103713]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:19.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:18 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:19.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:18 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:19.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:18 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:34:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:18 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:18 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:19.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:18 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:19.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:18 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:19.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:18 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:19.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:18 smithi099 ceph-mon[102692]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:19.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:18 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:19.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:18 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:19.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:18 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:34:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:18 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:18 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:18 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:19.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:18 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:19.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:18 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:19.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:18 smithi044 ceph-mon[98676]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:19.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:18 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:19.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:18 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:19.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:18 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:34:19.312 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-10T00:34:19.834 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 00:34:19 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-1[107794]: 2023-09-10T00:34:19.666+0000 7fd02655a640 -1 osd.1 0 log_to_monitors true 2023-09-10T00:34:20.102 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:19 smithi099 ceph-mon[102692]: from='osd.1 [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-10T00:34:20.102 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:19 smithi099 ceph-mon[102692]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-10T00:34:20.136 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:19 smithi131 ceph-mon[103713]: from='osd.1 [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-10T00:34:20.137 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:19 smithi131 ceph-mon[103713]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-10T00:34:20.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:19 smithi044 ceph-mon[98676]: from='osd.1 [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-10T00:34:20.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:19 smithi044 ceph-mon[98676]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-10T00:34:20.875 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-10T00:34:20.875 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-10T00:34:20.875 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2023-09-10T00:34:20.876 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2023-09-10T00:34:20.876 INFO:teuthology.orchestra.run.smithi131.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0146899 s, 714 MB/s 2023-09-10T00:34:20.876 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2023-09-10T00:34:21.106 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 00:34:20 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-1[107794]: 2023-09-10T00:34:20.834+0000 7fd01bbd1700 -1 osd.1 0 waiting for initial osdmap 2023-09-10T00:34:21.106 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 00:34:20 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-1[107794]: 2023-09-10T00:34:20.840+0000 7fd0161c3700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-10T00:34:21.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:20 smithi099 ceph-mon[102692]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-10T00:34:21.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:20 smithi099 ceph-mon[102692]: from='osd.1 [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:34:21.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:20 smithi099 ceph-mon[102692]: osdmap e11: 2 total, 1 up, 2 in 2023-09-10T00:34:21.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:20 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:21.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:20 smithi099 ceph-mon[102692]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:34:21.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:20 smithi099 ceph-mon[102692]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:21.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:20 smithi099 ceph-mon[102692]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-09-10T00:34:21.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:20 smithi099 ceph-mon[102692]: osdmap e12: 2 total, 1 up, 2 in 2023-09-10T00:34:21.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:20 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:21.261 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:20 smithi131 ceph-mon[103713]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-10T00:34:21.262 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:20 smithi131 ceph-mon[103713]: from='osd.1 [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:34:21.262 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:20 smithi131 ceph-mon[103713]: osdmap e11: 2 total, 1 up, 2 in 2023-09-10T00:34:21.262 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:20 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:21.262 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:20 smithi131 ceph-mon[103713]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:34:21.263 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:20 smithi131 ceph-mon[103713]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:21.263 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:20 smithi131 ceph-mon[103713]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-09-10T00:34:21.263 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:20 smithi131 ceph-mon[103713]: osdmap e12: 2 total, 1 up, 2 in 2023-09-10T00:34:21.263 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:20 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:21.264 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi131:vg_nvme/lv_4 2023-09-10T00:34:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:20 smithi044 ceph-mon[98676]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-10T00:34:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:20 smithi044 ceph-mon[98676]: from='osd.1 [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:34:21.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:20 smithi044 ceph-mon[98676]: osdmap e11: 2 total, 1 up, 2 in 2023-09-10T00:34:21.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:20 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:21.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:20 smithi044 ceph-mon[98676]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:34:21.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:20 smithi044 ceph-mon[98676]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:21.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:20 smithi044 ceph-mon[98676]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-09-10T00:34:21.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:20 smithi044 ceph-mon[98676]: osdmap e12: 2 total, 1 up, 2 in 2023-09-10T00:34:21.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:20 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:21.818 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-10T00:34:22.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:22.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:21 smithi099 ceph-mon[102692]: Detected new or changed devices on smithi099 2023-09-10T00:34:22.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:22.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:22.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:34:22.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:21 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi099 to 17168M 2023-09-10T00:34:22.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:22.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:22.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:22.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:22.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:22.111 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:22.111 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:21 smithi131 ceph-mon[103713]: Detected new or changed devices on smithi099 2023-09-10T00:34:22.111 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:22.111 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:22.112 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:34:22.112 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:21 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi099 to 17168M 2023-09-10T00:34:22.112 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:22.112 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:22.112 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:22.113 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:22.113 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:22.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:22.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:21 smithi044 ceph-mon[98676]: Detected new or changed devices on smithi099 2023-09-10T00:34:22.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:22.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:22.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:34:22.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:21 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi099 to 17168M 2023-09-10T00:34:22.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:22.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:22.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:22.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:22.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:23.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.185+0000 7f9ae66d7700 1 -- 172.21.15.131:0/624846067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ae00ff5c0 msgr2=0x7f9ae00ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:34:23.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.185+0000 7f9ae66d7700 1 --2- 172.21.15.131:0/624846067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ae00ff5c0 0x7f9ae00ff9e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009990 tx=0x7f9ad0009ca0 comp rx=0 tx=0).stop 2023-09-10T00:34:23.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.185+0000 7f9ae66d7700 1 -- 172.21.15.131:0/624846067 shutdown_connections 2023-09-10T00:34:23.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.185+0000 7f9ae66d7700 1 --2- 172.21.15.131:0/624846067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ae0101160 0x7f9ae0107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:23.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.185+0000 7f9ae66d7700 1 --2- 172.21.15.131:0/624846067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ae00ff5c0 0x7f9ae00ff9e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:23.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.185+0000 7f9ae66d7700 1 --2- 172.21.15.131:0/624846067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ae01007c0 0x7f9ae0100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:23.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.185+0000 7f9ae66d7700 1 -- 172.21.15.131:0/624846067 >> 172.21.15.131:0/624846067 conn(0x7f9ae00fab80 msgr2=0x7f9ae00fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:34:23.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.186+0000 7f9ae66d7700 1 -- 172.21.15.131:0/624846067 shutdown_connections 2023-09-10T00:34:23.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.186+0000 7f9ae66d7700 1 -- 172.21.15.131:0/624846067 wait complete. 2023-09-10T00:34:23.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.186+0000 7f9ae66d7700 1 Processor -- start 2023-09-10T00:34:23.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.186+0000 7f9ae66d7700 1 -- start start 2023-09-10T00:34:23.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae66d7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ae00ff5c0 0x7f9ae0195200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:34:23.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae66d7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ae01007c0 0x7f9ae0195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:34:23.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae66d7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ae0101160 0x7f9ae019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:34:23.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae66d7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae019cd90 con 0x7f9ae00ff5c0 2023-09-10T00:34:23.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae66d7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae019ced0 con 0x7f9ae01007c0 2023-09-10T00:34:23.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae66d7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae019d060 con 0x7f9ae0101160 2023-09-10T00:34:23.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae4c74700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ae0101160 0x7f9ae019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:34:23.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae4c74700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ae0101160 0x7f9ae019c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52890/0 (socket says 172.21.15.131:52890) 2023-09-10T00:34:23.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae4c74700 1 -- 172.21.15.131:0/3426904639 learned_addr learned my addr 172.21.15.131:0/3426904639 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-10T00:34:23.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9adffff700 1 --2- 172.21.15.131:0/3426904639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ae00ff5c0 0x7f9ae0195200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:34:23.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9adf7fe700 1 --2- 172.21.15.131:0/3426904639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ae01007c0 0x7f9ae0195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:34:23.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae4c74700 1 -- 172.21.15.131:0/3426904639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ae01007c0 msgr2=0x7f9ae0195740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:34:23.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae4c74700 1 --2- 172.21.15.131:0/3426904639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ae01007c0 0x7f9ae0195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:23.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae4c74700 1 -- 172.21.15.131:0/3426904639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ae00ff5c0 msgr2=0x7f9ae0195200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:34:23.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae4c74700 1 --2- 172.21.15.131:0/3426904639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ae00ff5c0 0x7f9ae0195200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:23.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae4c74700 1 -- 172.21.15.131:0/3426904639 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad00095f0 con 0x7f9ae0101160 2023-09-10T00:34:23.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9adffff700 1 --2- 172.21.15.131:0/3426904639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ae00ff5c0 0x7f9ae0195200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:34:23.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae4c74700 1 --2- 172.21.15.131:0/3426904639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ae0101160 0x7f9ae019c7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800add0 tx=0x7f9ad800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:34:23.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9add7fa700 1 -- 172.21.15.131:0/3426904639 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad800f070 con 0x7f9ae0101160 2023-09-10T00:34:23.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9add7fa700 1 -- 172.21.15.131:0/3426904639 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ad8007b30 con 0x7f9ae0101160 2023-09-10T00:34:23.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae66d7700 1 -- 172.21.15.131:0/3426904639 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae019d340 con 0x7f9ae0101160 2023-09-10T00:34:23.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9add7fa700 1 -- 172.21.15.131:0/3426904639 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8005350 con 0x7f9ae0101160 2023-09-10T00:34:23.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.187+0000 7f9ae66d7700 1 -- 172.21.15.131:0/3426904639 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae019d7f0 con 0x7f9ae0101160 2023-09-10T00:34:23.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.188+0000 7f9ae66d7700 1 -- 172.21.15.131:0/3426904639 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae010ace0 con 0x7f9ae0101160 2023-09-10T00:34:23.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.189+0000 7f9add7fa700 1 -- 172.21.15.131:0/3426904639 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 15) v1 ==== 135203+0+0 (secure 0 0 0) 0x7f9ad800c940 con 0x7f9ae0101160 2023-09-10T00:34:23.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.190+0000 7f9add7fa700 1 --2- 172.21.15.131:0/3426904639 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9ac80a05b0 0x7f9ac80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:34:23.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.190+0000 7f9add7fa700 1 -- 172.21.15.131:0/3426904639 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f9ad80c99b0 con 0x7f9ae0101160 2023-09-10T00:34:23.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.190+0000 7f9adffff700 1 --2- 172.21.15.131:0/3426904639 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9ac80a05b0 0x7f9ac80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:34:23.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.191+0000 7f9adffff700 1 --2- 172.21.15.131:0/3426904639 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9ac80a05b0 0x7f9ac80a2a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000b5c0 tx=0x7f9ad0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:34:23.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.195+0000 7f9add7fa700 1 -- 172.21.15.131:0/3426904639 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9ad8094dc0 con 0x7f9ae0101160 2023-09-10T00:34:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:22 smithi131 ceph-mon[103713]: purged_snaps scrub starts 2023-09-10T00:34:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:22 smithi131 ceph-mon[103713]: purged_snaps scrub ok 2023-09-10T00:34:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:22 smithi131 ceph-mon[103713]: osd.1 [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758] boot 2023-09-10T00:34:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:22 smithi131 ceph-mon[103713]: osdmap e13: 2 total, 2 up, 2 in 2023-09-10T00:34:23.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:22 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:23.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:22 smithi131 ceph-mon[103713]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:23.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:22 smithi044 ceph-mon[98676]: purged_snaps scrub starts 2023-09-10T00:34:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:22 smithi044 ceph-mon[98676]: purged_snaps scrub ok 2023-09-10T00:34:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:22 smithi044 ceph-mon[98676]: osd.1 [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758] boot 2023-09-10T00:34:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:22 smithi044 ceph-mon[98676]: osdmap e13: 2 total, 2 up, 2 in 2023-09-10T00:34:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:22 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:22 smithi044 ceph-mon[98676]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:22 smithi099 ceph-mon[102692]: purged_snaps scrub starts 2023-09-10T00:34:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:22 smithi099 ceph-mon[102692]: purged_snaps scrub ok 2023-09-10T00:34:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:22 smithi099 ceph-mon[102692]: osd.1 [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758] boot 2023-09-10T00:34:23.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:22 smithi099 ceph-mon[102692]: osdmap e13: 2 total, 2 up, 2 in 2023-09-10T00:34:23.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:22 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:34:23.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:22 smithi099 ceph-mon[102692]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-10T00:34:23.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:23.392+0000 7f9ae66d7700 1 -- 172.21.15.131:0/3426904639 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ae0060a40 con 0x7f9ac80a05b0 2023-09-10T00:34:24.259 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:23 smithi131 ceph-mon[103713]: osdmap e14: 2 total, 2 up, 2 in 2023-09-10T00:34:24.259 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:23 smithi131 ceph-mon[103713]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:34:24.259 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:34:24.260 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:34:24.260 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:24.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:23 smithi044 ceph-mon[98676]: osdmap e14: 2 total, 2 up, 2 in 2023-09-10T00:34:24.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:23 smithi044 ceph-mon[98676]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:34:24.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:34:24.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:34:24.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:24.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:23 smithi099 ceph-mon[102692]: osdmap e14: 2 total, 2 up, 2 in 2023-09-10T00:34:24.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:23 smithi099 ceph-mon[102692]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:34:24.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:34:24.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:34:24.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:25.201 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:24 smithi131 ceph-mon[103713]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:24 smithi044 ceph-mon[98676]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:24 smithi099 ceph-mon[102692]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:26.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:26 smithi131 ceph-mon[103713]: from='client.? 172.21.15.131:0/3347439639' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d73ba44-ef7a-495b-97cb-550517aaab45"}]: dispatch 2023-09-10T00:34:26.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:26 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d73ba44-ef7a-495b-97cb-550517aaab45"}]: dispatch 2023-09-10T00:34:26.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:26 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d73ba44-ef7a-495b-97cb-550517aaab45"}]': finished 2023-09-10T00:34:26.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:26 smithi131 ceph-mon[103713]: osdmap e15: 3 total, 2 up, 3 in 2023-09-10T00:34:26.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:26 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:34:26.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:26 smithi044 ceph-mon[98676]: from='client.? 172.21.15.131:0/3347439639' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d73ba44-ef7a-495b-97cb-550517aaab45"}]: dispatch 2023-09-10T00:34:26.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:26 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d73ba44-ef7a-495b-97cb-550517aaab45"}]: dispatch 2023-09-10T00:34:26.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:26 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d73ba44-ef7a-495b-97cb-550517aaab45"}]': finished 2023-09-10T00:34:26.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:26 smithi044 ceph-mon[98676]: osdmap e15: 3 total, 2 up, 3 in 2023-09-10T00:34:26.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:26 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:34:26.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:26 smithi099 ceph-mon[102692]: from='client.? 172.21.15.131:0/3347439639' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d73ba44-ef7a-495b-97cb-550517aaab45"}]: dispatch 2023-09-10T00:34:26.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:26 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d73ba44-ef7a-495b-97cb-550517aaab45"}]: dispatch 2023-09-10T00:34:26.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:26 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d73ba44-ef7a-495b-97cb-550517aaab45"}]': finished 2023-09-10T00:34:26.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:26 smithi099 ceph-mon[102692]: osdmap e15: 3 total, 2 up, 3 in 2023-09-10T00:34:26.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:26 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:34:27.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:27 smithi131 ceph-mon[103713]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:27.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:27 smithi131 ceph-mon[103713]: from='client.? 172.21.15.131:0/3076452288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:34:27.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:27 smithi044 ceph-mon[98676]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:27.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:27 smithi044 ceph-mon[98676]: from='client.? 172.21.15.131:0/3076452288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:34:27.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:27 smithi099 ceph-mon[102692]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:27.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:27 smithi099 ceph-mon[102692]: from='client.? 172.21.15.131:0/3076452288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:34:29.708 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:29 smithi131 ceph-mon[103713]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:29.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:29 smithi044 ceph-mon[98676]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:29.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:29 smithi099 ceph-mon[102692]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:31.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:31 smithi131 ceph-mon[103713]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:31.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:31 smithi044 ceph-mon[98676]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:31.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:31 smithi099 ceph-mon[102692]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:33.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:33 smithi131 ceph-mon[103713]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:33.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:33 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-10T00:34:33.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:33 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:33.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:33 smithi131 ceph-mon[103713]: Deploying daemon osd.2 on smithi131 2023-09-10T00:34:33.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:33 smithi044 ceph-mon[98676]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:33.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:33 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-10T00:34:33.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:33 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:33.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:33 smithi044 ceph-mon[98676]: Deploying daemon osd.2 on smithi131 2023-09-10T00:34:33.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:33 smithi099 ceph-mon[102692]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:33.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:33 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-10T00:34:33.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:33 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:33.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:33 smithi099 ceph-mon[102692]: Deploying daemon osd.2 on smithi131 2023-09-10T00:34:35.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:35 smithi131 ceph-mon[103713]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:35.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:35 smithi044 ceph-mon[98676]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:35.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:35 smithi099 ceph-mon[102692]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:37.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:37 smithi131 ceph-mon[103713]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:37.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:37.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:37.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:34:37.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:37 smithi044 ceph-mon[98676]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:37.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:37.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:37.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:34:37.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:37 smithi099 ceph-mon[102692]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:37.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:37.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:37.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:34:39.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:39 smithi131 ceph-mon[103713]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:39.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:39 smithi044 ceph-mon[98676]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:39.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:39 smithi099 ceph-mon[102692]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:40.313 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 2 on host 'smithi131' 2023-09-10T00:34:40.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:40.311+0000 7f9add7fa700 1 -- 172.21.15.131:0/3426904639 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9ae0060a40 con 0x7f9ac80a05b0 2023-09-10T00:34:40.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:40.314+0000 7f9ae66d7700 1 -- 172.21.15.131:0/3426904639 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9ac80a05b0 msgr2=0x7f9ac80a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:34:40.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:40.314+0000 7f9ae66d7700 1 --2- 172.21.15.131:0/3426904639 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9ac80a05b0 0x7f9ac80a2a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000b5c0 tx=0x7f9ad0005fb0 comp rx=0 tx=0).stop 2023-09-10T00:34:40.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:40.314+0000 7f9ae66d7700 1 -- 172.21.15.131:0/3426904639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ae0101160 msgr2=0x7f9ae019c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:34:40.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:40.314+0000 7f9ae66d7700 1 --2- 172.21.15.131:0/3426904639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ae0101160 0x7f9ae019c7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800add0 tx=0x7f9ad800c330 comp rx=0 tx=0).stop 2023-09-10T00:34:40.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:40.314+0000 7f9ae66d7700 1 -- 172.21.15.131:0/3426904639 shutdown_connections 2023-09-10T00:34:40.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:40.314+0000 7f9ae66d7700 1 --2- 172.21.15.131:0/3426904639 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9ac80a05b0 0x7f9ac80a2a70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:40.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:40.314+0000 7f9ae66d7700 1 --2- 172.21.15.131:0/3426904639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ae0101160 0x7f9ae019c7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:40.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:40.314+0000 7f9ae66d7700 1 --2- 172.21.15.131:0/3426904639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ae00ff5c0 0x7f9ae0195200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:40.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:40.314+0000 7f9ae66d7700 1 --2- 172.21.15.131:0/3426904639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ae01007c0 0x7f9ae0195740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:40.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:40.314+0000 7f9ae66d7700 1 -- 172.21.15.131:0/3426904639 >> 172.21.15.131:0/3426904639 conn(0x7f9ae00fab80 msgr2=0x7f9ae00fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:34:40.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:40.314+0000 7f9ae66d7700 1 -- 172.21.15.131:0/3426904639 shutdown_connections 2023-09-10T00:34:40.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:34:40.314+0000 7f9ae66d7700 1 -- 172.21.15.131:0/3426904639 wait complete. 2023-09-10T00:34:40.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:40.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:40.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:40.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:40.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:40.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:40.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:40.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:34:40.823 DEBUG:teuthology.orchestra.run.smithi131:osd.2> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.2.service 2023-09-10T00:34:40.825 INFO:tasks.cephadm:Deploying osd.3 on smithi044 with /dev/vg_nvme/lv_3... 2023-09-10T00:34:40.826 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-10T00:34:40.854 INFO:journalctl@ceph.osd.2.smithi131.stdout:-- Logs begin at Sun 2023-09-10 00:17:05 UTC. -- 2023-09-10T00:34:40.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:40.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:40.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:40.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:40.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:40.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:40.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:40.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:34:40.860 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:40.860 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:40.860 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:40.860 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:40.861 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:40.861 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:40.861 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:40.861 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:34:41.404 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:34:41.410 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 00:34:41 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-2[109574]: 2023-09-10T00:34:41.082+0000 7f45a575d640 -1 osd.2 0 log_to_monitors true 2023-09-10T00:34:41.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:41 smithi131 ceph-mon[103713]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:41.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:41 smithi131 ceph-mon[103713]: from='osd.2 [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-10T00:34:41.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:41 smithi131 ceph-mon[103713]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-10T00:34:41.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:41 smithi099 ceph-mon[102692]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:41.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:41 smithi099 ceph-mon[102692]: from='osd.2 [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-10T00:34:41.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:41 smithi099 ceph-mon[102692]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-10T00:34:41.929 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:41 smithi044 ceph-mon[98676]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:41.930 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:41 smithi044 ceph-mon[98676]: from='osd.2 [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-10T00:34:41.930 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:41 smithi044 ceph-mon[98676]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-10T00:34:42.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:42 smithi044 ceph-mon[98676]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-10T00:34:42.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:42 smithi044 ceph-mon[98676]: osdmap e16: 3 total, 2 up, 3 in 2023-09-10T00:34:42.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:42 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:34:42.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:42 smithi044 ceph-mon[98676]: from='osd.2 [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:34:42.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:42 smithi044 ceph-mon[98676]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:34:42.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:42 smithi099 ceph-mon[102692]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-10T00:34:42.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:42 smithi099 ceph-mon[102692]: osdmap e16: 3 total, 2 up, 3 in 2023-09-10T00:34:42.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:42 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:34:42.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:42 smithi099 ceph-mon[102692]: from='osd.2 [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:34:42.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:42 smithi099 ceph-mon[102692]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:34:42.928 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-10T00:34:42.928 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-10T00:34:42.928 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2023-09-10T00:34:42.929 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2023-09-10T00:34:42.929 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115441 s, 908 MB/s 2023-09-10T00:34:42.929 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2023-09-10T00:34:43.045 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 00:34:42 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-2[109574]: 2023-09-10T00:34:42.606+0000 7f459add4700 -1 osd.2 0 waiting for initial osdmap 2023-09-10T00:34:43.045 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 00:34:42 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-2[109574]: 2023-09-10T00:34:42.611+0000 7f4594bc5700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-10T00:34:43.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:42 smithi131 ceph-mon[103713]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-10T00:34:43.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:42 smithi131 ceph-mon[103713]: osdmap e16: 3 total, 2 up, 3 in 2023-09-10T00:34:43.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:42 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:34:43.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:42 smithi131 ceph-mon[103713]: from='osd.2 [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:34:43.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:42 smithi131 ceph-mon[103713]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:34:43.355 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi044:vg_nvme/lv_3 2023-09-10T00:34:43.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:43 smithi131 ceph-mon[103713]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:43.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:43 smithi131 ceph-mon[103713]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2023-09-10T00:34:43.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:43 smithi131 ceph-mon[103713]: osdmap e17: 3 total, 2 up, 3 in 2023-09-10T00:34:43.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:34:43.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:34:43.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:43.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:43.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:34:43.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:43.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:43.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:43.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:43.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:43 smithi099 ceph-mon[102692]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:43.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:43 smithi099 ceph-mon[102692]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2023-09-10T00:34:43.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:43 smithi099 ceph-mon[102692]: osdmap e17: 3 total, 2 up, 3 in 2023-09-10T00:34:43.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:34:43.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:34:43.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:43.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:43.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:34:43.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:43.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:43.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:43.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:43.968 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:34:44.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:43 smithi044 ceph-mon[98676]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-10T00:34:44.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:43 smithi044 ceph-mon[98676]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2023-09-10T00:34:44.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:43 smithi044 ceph-mon[98676]: osdmap e17: 3 total, 2 up, 3 in 2023-09-10T00:34:44.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:34:44.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:34:44.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:44.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:44.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:34:44.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:44.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:44.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:44.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:44.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:44 smithi044 ceph-mon[98676]: purged_snaps scrub starts 2023-09-10T00:34:44.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:44 smithi044 ceph-mon[98676]: purged_snaps scrub ok 2023-09-10T00:34:44.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:44 smithi044 ceph-mon[98676]: Detected new or changed devices on smithi131 2023-09-10T00:34:44.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:44 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi131 to 17168M 2023-09-10T00:34:44.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:44 smithi044 ceph-mon[98676]: osd.2 [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432] boot 2023-09-10T00:34:44.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:44 smithi044 ceph-mon[98676]: osdmap e18: 3 total, 3 up, 3 in 2023-09-10T00:34:44.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:44 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:34:44.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:44 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-10T00:34:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:44 smithi131 ceph-mon[103713]: purged_snaps scrub starts 2023-09-10T00:34:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:44 smithi131 ceph-mon[103713]: purged_snaps scrub ok 2023-09-10T00:34:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:44 smithi131 ceph-mon[103713]: Detected new or changed devices on smithi131 2023-09-10T00:34:45.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:44 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi131 to 17168M 2023-09-10T00:34:45.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:44 smithi131 ceph-mon[103713]: osd.2 [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432] boot 2023-09-10T00:34:45.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:44 smithi131 ceph-mon[103713]: osdmap e18: 3 total, 3 up, 3 in 2023-09-10T00:34:45.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:44 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:34:45.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:44 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-10T00:34:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:44 smithi099 ceph-mon[102692]: purged_snaps scrub starts 2023-09-10T00:34:45.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:44 smithi099 ceph-mon[102692]: purged_snaps scrub ok 2023-09-10T00:34:45.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:44 smithi099 ceph-mon[102692]: Detected new or changed devices on smithi131 2023-09-10T00:34:45.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:44 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi131 to 17168M 2023-09-10T00:34:45.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:44 smithi099 ceph-mon[102692]: osd.2 [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432] boot 2023-09-10T00:34:45.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:44 smithi099 ceph-mon[102692]: osdmap e18: 3 total, 3 up, 3 in 2023-09-10T00:34:45.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:44 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:34:45.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:44 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-10T00:34:45.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.170+0000 7f03c7ffb700 1 -- 172.21.15.44:0/108219914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03c0101120 msgr2=0x7f03c0103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:34:45.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.170+0000 7f03c7ffb700 1 --2- 172.21.15.44:0/108219914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03c0101120 0x7f03c0103510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009990 tx=0x7f03bc009ca0 comp rx=0 tx=0).stop 2023-09-10T00:34:45.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.170+0000 7f03c7ffb700 1 -- 172.21.15.44:0/108219914 shutdown_connections 2023-09-10T00:34:45.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.170+0000 7f03c7ffb700 1 --2- 172.21.15.44:0/108219914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c0106380 0x7f03c01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:45.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.170+0000 7f03c7ffb700 1 --2- 172.21.15.44:0/108219914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03c0101120 0x7f03c0103510 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:45.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.170+0000 7f03c7ffb700 1 --2- 172.21.15.44:0/108219914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03c0103a50 0x7f03c0105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:45.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.170+0000 7f03c7ffb700 1 -- 172.21.15.44:0/108219914 >> 172.21.15.44:0/108219914 conn(0x7f03c00fab10 msgr2=0x7f03c00fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:34:45.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.170+0000 7f03c7ffb700 1 -- 172.21.15.44:0/108219914 shutdown_connections 2023-09-10T00:34:45.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.170+0000 7f03c7ffb700 1 -- 172.21.15.44:0/108219914 wait complete. 2023-09-10T00:34:45.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c7ffb700 1 Processor -- start 2023-09-10T00:34:45.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c7ffb700 1 -- start start 2023-09-10T00:34:45.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c7ffb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03c0101120 0x7f03c0196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:34:45.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c7ffb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c0103a50 0x7f03c01967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:34:45.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c7ffb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03c0106380 0x7f03c019d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:34:45.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c7ffb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c019de00 con 0x7f03c0101120 2023-09-10T00:34:45.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c7ffb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c019df40 con 0x7f03c0103a50 2023-09-10T00:34:45.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c7ffb700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c019e0d0 con 0x7f03c0106380 2023-09-10T00:34:45.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c5596700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c0103a50 0x7f03c01967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:34:45.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c5596700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c0103a50 0x7f03c01967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:47204/0 (socket says 172.21.15.44:47204) 2023-09-10T00:34:45.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c5596700 1 -- 172.21.15.44:0/2642013636 learned_addr learned my addr 172.21.15.44:0/2642013636 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:34:45.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c5d97700 1 --2- 172.21.15.44:0/2642013636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03c0101120 0x7f03c0196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:34:45.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c6598700 1 --2- 172.21.15.44:0/2642013636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03c0106380 0x7f03c019d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:34:45.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c5d97700 1 -- 172.21.15.44:0/2642013636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03c0106380 msgr2=0x7f03c019d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:34:45.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c5d97700 1 --2- 172.21.15.44:0/2642013636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03c0106380 0x7f03c019d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:45.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c5d97700 1 -- 172.21.15.44:0/2642013636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c0103a50 msgr2=0x7f03c01967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:34:45.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c5d97700 1 --2- 172.21.15.44:0/2642013636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c0103a50 0x7f03c01967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:34:45.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c5d97700 1 -- 172.21.15.44:0/2642013636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc0095f0 con 0x7f03c0101120 2023-09-10T00:34:45.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.171+0000 7f03c5d97700 1 --2- 172.21.15.44:0/2642013636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03c0101120 0x7f03c0196270 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0098e0 tx=0x7f03bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:34:45.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.172+0000 7f03b6ffd700 1 -- 172.21.15.44:0/2642013636 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc01d070 con 0x7f03c0101120 2023-09-10T00:34:45.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.172+0000 7f03b6ffd700 1 -- 172.21.15.44:0/2642013636 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03bc0054c0 con 0x7f03c0101120 2023-09-10T00:34:45.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.172+0000 7f03c7ffb700 1 -- 172.21.15.44:0/2642013636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c00fef10 con 0x7f03c0101120 2023-09-10T00:34:45.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.172+0000 7f03b6ffd700 1 -- 172.21.15.44:0/2642013636 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc003ab0 con 0x7f03c0101120 2023-09-10T00:34:45.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.172+0000 7f03c7ffb700 1 -- 172.21.15.44:0/2642013636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c00ff3b0 con 0x7f03c0101120 2023-09-10T00:34:45.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.173+0000 7f03c7ffb700 1 -- 172.21.15.44:0/2642013636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c010bd70 con 0x7f03c0101120 2023-09-10T00:34:45.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.174+0000 7f03b6ffd700 1 -- 172.21.15.44:0/2642013636 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 15) v1 ==== 135203+0+0 (secure 0 0 0) 0x7f03bc0047a0 con 0x7f03c0101120 2023-09-10T00:34:45.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.175+0000 7f03b6ffd700 1 --2- 172.21.15.44:0/2642013636 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f03ac0a05b0 0x7f03ac0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:34:45.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.175+0000 7f03c5596700 1 --2- 172.21.15.44:0/2642013636 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f03ac0a05b0 0x7f03ac0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:34:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.175+0000 7f03b6ffd700 1 -- 172.21.15.44:0/2642013636 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f03bc0cba50 con 0x7f03c0101120 2023-09-10T00:34:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.175+0000 7f03c5596700 1 --2- 172.21.15.44:0/2642013636 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f03ac0a05b0 0x7f03ac0a2a70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f03b0005950 tx=0x7f03b000a400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:34:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.178+0000 7f03b6ffd700 1 -- 172.21.15.44:0/2642013636 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f03bc096a30 con 0x7f03c0101120 2023-09-10T00:34:45.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:45.384+0000 7f03c7ffb700 1 -- 172.21.15.44:0/2642013636 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f03c0102c80 con 0x7f03ac0a05b0 2023-09-10T00:34:46.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:45 smithi131 ceph-mon[103713]: pgmap v82: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:34:46.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-10T00:34:46.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:45 smithi131 ceph-mon[103713]: osdmap e19: 3 total, 3 up, 3 in 2023-09-10T00:34:46.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-10T00:34:46.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:34:46.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:34:46.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:46.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:45 smithi044 ceph-mon[98676]: pgmap v82: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:34:46.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-10T00:34:46.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:45 smithi044 ceph-mon[98676]: osdmap e19: 3 total, 3 up, 3 in 2023-09-10T00:34:46.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-10T00:34:46.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:34:46.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:34:46.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:46.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:45 smithi099 ceph-mon[102692]: pgmap v82: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:34:46.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-10T00:34:46.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:45 smithi099 ceph-mon[102692]: osdmap e19: 3 total, 3 up, 3 in 2023-09-10T00:34:46.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-10T00:34:46.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:34:46.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:34:46.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:46.545 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 00:34:46 smithi044 sudo[128179]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:34:46.545 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 00:34:46 smithi044 sudo[128179]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:34:46.546 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 00:34:46 smithi044 sudo[128179]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:34:46.546 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 00:34:46 smithi044 sudo[128179]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:34:46.805 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 00:34:46 smithi044 sudo[128209]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:34:46.805 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 00:34:46 smithi044 sudo[128209]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:34:46.805 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 00:34:46 smithi044 sudo[128209]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:34:46.806 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 00:34:46 smithi044 sudo[128209]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:34:46.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:46 smithi044 ceph-mon[98676]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:34:46.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:46 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-10T00:34:46.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:46 smithi044 ceph-mon[98676]: osdmap e20: 3 total, 3 up, 3 in 2023-09-10T00:34:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:46 smithi131 ceph-mon[103713]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:34:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:46 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-10T00:34:47.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:46 smithi131 ceph-mon[103713]: osdmap e20: 3 total, 3 up, 3 in 2023-09-10T00:34:47.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:46 smithi099 ceph-mon[102692]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:34:47.052 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:46 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-10T00:34:47.052 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:46 smithi099 ceph-mon[102692]: osdmap e20: 3 total, 3 up, 3 in 2023-09-10T00:34:47.303 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 00:34:47 smithi099 sudo[112539]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:34:47.303 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 00:34:47 smithi099 sudo[112539]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:34:47.303 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 00:34:47 smithi099 sudo[112539]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:34:47.304 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 00:34:47 smithi099 sudo[112539]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:34:47.606 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 00:34:47 smithi099 sudo[112542]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:34:47.606 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 00:34:47 smithi099 sudo[112542]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:34:47.607 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 00:34:47 smithi099 sudo[112542]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:34:47.607 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 00:34:47 smithi099 sudo[112542]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:34:47.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:34:47.632+0000 7f03b6ffd700 1 -- 172.21.15.44:0/2642013636 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f03bc096e00 con 0x7f03c0101120 2023-09-10T00:34:47.824 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 00:34:47 smithi131 sudo[114971]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:34:47.825 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 00:34:47 smithi131 sudo[114971]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:34:47.825 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 00:34:47 smithi131 sudo[114971]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:34:47.825 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 00:34:47 smithi131 sudo[114971]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:34:47.826 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:47 smithi131 ceph-mon[103713]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:34:47.826 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:47 smithi131 ceph-mon[103713]: osdmap e21: 3 total, 3 up, 3 in 2023-09-10T00:34:47.826 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:47 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:47.826 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:47 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:48.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:47 smithi044 ceph-mon[98676]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:34:48.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:47 smithi044 ceph-mon[98676]: osdmap e21: 3 total, 3 up, 3 in 2023-09-10T00:34:48.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:47 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:48.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:47 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:48.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:47 smithi099 ceph-mon[102692]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:34:48.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:47 smithi099 ceph-mon[102692]: osdmap e21: 3 total, 3 up, 3 in 2023-09-10T00:34:48.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:47 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:48.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:47 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:48.295 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 00:34:47 smithi131 sudo[114974]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:34:48.295 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 00:34:47 smithi131 sudo[114974]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:34:48.296 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 00:34:47 smithi131 sudo[114974]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:34:48.296 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 00:34:47 smithi131 sudo[114974]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:34:48.554 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:48 smithi044 sudo[128300]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:34:48.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:48 smithi044 sudo[128300]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:34:48.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:48 smithi044 sudo[128300]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:34:48.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:48 smithi044 sudo[128300]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:34:49.004 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:48 smithi131 ceph-mon[103713]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-10T00:34:49.004 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:48 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:49.005 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:48 smithi131 ceph-mon[103713]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:34:49.005 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:48 smithi131 sudo[114977]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:34:49.005 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:48 smithi131 sudo[114977]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:34:49.005 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:48 smithi131 sudo[114977]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:34:49.005 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 sudo[114977]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:34:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:48 smithi044 sudo[128303]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:34:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:48 smithi044 sudo[128303]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:34:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:48 smithi044 sudo[128303]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:34:49.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:48 smithi044 sudo[128303]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:34:49.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:48 smithi044 ceph-mon[98676]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-10T00:34:49.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:49.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:48 smithi044 ceph-mon[98676]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:34:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:48 smithi099 ceph-mon[102692]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-10T00:34:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:48 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:34:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:48 smithi099 ceph-mon[102692]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:34:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 sudo[114980]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:34:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 sudo[114980]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:34:49.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 sudo[114980]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:34:49.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 sudo[114980]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:34:49.738 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 sudo[112545]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:34:49.738 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 sudo[112545]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:34:49.738 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 sudo[112545]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:34:49.738 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 sudo[112545]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:34:50.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:34:50.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:34:50.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:34:50.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/810961432' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6fe9647-238c-4f5c-a933-ace40dc5bc0f"}]: dispatch 2023-09-10T00:34:50.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:34:50.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:34:50.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:34:50.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/810961432' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6fe9647-238c-4f5c-a933-ace40dc5bc0f"}]': finished 2023-09-10T00:34:50.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: osdmap e22: 4 total, 3 up, 4 in 2023-09-10T00:34:50.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:34:50.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:34:50.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:34:50.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:34:50.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:34:50.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:34:50.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:34:50.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:34:50.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:34:50.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2582219580' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:34:50.050 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:49 smithi131 ceph-mon[103713]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:34:50.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:34:50.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:34:50.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:34:50.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/810961432' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6fe9647-238c-4f5c-a933-ace40dc5bc0f"}]: dispatch 2023-09-10T00:34:50.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:34:50.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:34:50.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:34:50.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/810961432' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6fe9647-238c-4f5c-a933-ace40dc5bc0f"}]': finished 2023-09-10T00:34:50.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: osdmap e22: 4 total, 3 up, 4 in 2023-09-10T00:34:50.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:34:50.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:34:50.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:34:50.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:34:50.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:34:50.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:34:50.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:34:50.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:34:50.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:34:50.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2582219580' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:34:50.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:49 smithi044 ceph-mon[98676]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:34:50.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 sudo[112548]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:34:50.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 sudo[112548]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:34:50.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 sudo[112548]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:34:50.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 sudo[112548]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:34:50.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:34:50.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:34:50.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:34:50.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/810961432' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6fe9647-238c-4f5c-a933-ace40dc5bc0f"}]: dispatch 2023-09-10T00:34:50.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:34:50.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:34:50.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:34:50.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/810961432' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6fe9647-238c-4f5c-a933-ace40dc5bc0f"}]': finished 2023-09-10T00:34:50.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: osdmap e22: 4 total, 3 up, 4 in 2023-09-10T00:34:50.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:34:50.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:34:50.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:34:50.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:34:50.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:34:50.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:34:50.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:34:50.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:34:50.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:34:50.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2582219580' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:34:50.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:49 smithi099 ceph-mon[102692]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:34:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:51 smithi131 ceph-mon[103713]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:34:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:51 smithi044 ceph-mon[98676]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:34:52.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:51 smithi099 ceph-mon[102692]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:34:54.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:53 smithi131 ceph-mon[103713]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:34:54.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:53 smithi044 ceph-mon[98676]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:34:54.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:53 smithi099 ceph-mon[102692]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:34:56.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:55 smithi131 ceph-mon[103713]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-09-10T00:34:56.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:55 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-10T00:34:56.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:55 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:56.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:55 smithi044 ceph-mon[98676]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-09-10T00:34:56.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:55 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-10T00:34:56.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:55 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:56.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:55 smithi099 ceph-mon[102692]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-09-10T00:34:56.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:55 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-10T00:34:56.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:55 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:57.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:56 smithi044 ceph-mon[98676]: Deploying daemon osd.3 on smithi044 2023-09-10T00:34:57.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:56 smithi131 ceph-mon[103713]: Deploying daemon osd.3 on smithi044 2023-09-10T00:34:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:56 smithi099 ceph-mon[102692]: Deploying daemon osd.3 on smithi044 2023-09-10T00:34:58.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:57 smithi131 ceph-mon[103713]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 73 KiB/s wr, 4 op/s 2023-09-10T00:34:58.051 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:57 smithi044 ceph-mon[98676]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 73 KiB/s wr, 4 op/s 2023-09-10T00:34:58.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:57 smithi099 ceph-mon[102692]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 73 KiB/s wr, 4 op/s 2023-09-10T00:35:00.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:34:59 smithi131 ceph-mon[103713]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 59 KiB/s wr, 3 op/s 2023-09-10T00:35:00.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:34:59 smithi044 ceph-mon[98676]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 59 KiB/s wr, 3 op/s 2023-09-10T00:35:00.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:34:59 smithi099 ceph-mon[102692]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 59 KiB/s wr, 3 op/s 2023-09-10T00:35:01.235 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:00 smithi044 ceph-mon[98676]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 52 KiB/s wr, 3 op/s 2023-09-10T00:35:01.235 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:00 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:01.236 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:00 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:01.236 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:00 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:00 smithi131 ceph-mon[103713]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 52 KiB/s wr, 3 op/s 2023-09-10T00:35:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:00 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:00 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:00 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:00 smithi099 ceph-mon[102692]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 52 KiB/s wr, 3 op/s 2023-09-10T00:35:01.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:00 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:01.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:00 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:01.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:00 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:02.538 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 3 on host 'smithi044' 2023-09-10T00:35:02.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:02.538+0000 7f03b6ffd700 1 -- 172.21.15.44:0/2642013636 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f03c0102c80 con 0x7f03ac0a05b0 2023-09-10T00:35:02.541 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:02.540+0000 7f03c7ffb700 1 -- 172.21.15.44:0/2642013636 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f03ac0a05b0 msgr2=0x7f03ac0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:35:02.541 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:02.540+0000 7f03c7ffb700 1 --2- 172.21.15.44:0/2642013636 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f03ac0a05b0 0x7f03ac0a2a70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f03b0005950 tx=0x7f03b000a400 comp rx=0 tx=0).stop 2023-09-10T00:35:02.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:02.540+0000 7f03c7ffb700 1 -- 172.21.15.44:0/2642013636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03c0101120 msgr2=0x7f03c0196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:35:02.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:02.540+0000 7f03c7ffb700 1 --2- 172.21.15.44:0/2642013636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03c0101120 0x7f03c0196270 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0098e0 tx=0x7f03bc004730 comp rx=0 tx=0).stop 2023-09-10T00:35:02.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:02.540+0000 7f03c7ffb700 1 -- 172.21.15.44:0/2642013636 shutdown_connections 2023-09-10T00:35:02.543 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:02.540+0000 7f03c7ffb700 1 --2- 172.21.15.44:0/2642013636 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f03ac0a05b0 0x7f03ac0a2a70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:02.543 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:02.540+0000 7f03c7ffb700 1 --2- 172.21.15.44:0/2642013636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03c0106380 0x7f03c019d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:02.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:02.540+0000 7f03c7ffb700 1 --2- 172.21.15.44:0/2642013636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03c0101120 0x7f03c0196270 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:02.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:02.540+0000 7f03c7ffb700 1 --2- 172.21.15.44:0/2642013636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c0103a50 0x7f03c01967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:02.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:02.540+0000 7f03c7ffb700 1 -- 172.21.15.44:0/2642013636 >> 172.21.15.44:0/2642013636 conn(0x7f03c00fab10 msgr2=0x7f03c0102570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:35:02.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:02.540+0000 7f03c7ffb700 1 -- 172.21.15.44:0/2642013636 shutdown_connections 2023-09-10T00:35:02.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:02.540+0000 7f03c7ffb700 1 -- 172.21.15.44:0/2642013636 wait complete. 2023-09-10T00:35:02.954 DEBUG:teuthology.orchestra.run.smithi044:osd.3> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.3.service 2023-09-10T00:35:02.956 INFO:tasks.cephadm:Deploying osd.4 on smithi099 with /dev/vg_nvme/lv_3... 2023-09-10T00:35:02.957 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-10T00:35:02.990 INFO:journalctl@ceph.osd.3.smithi044.stdout:-- Logs begin at Sun 2023-09-10 00:17:15 UTC. -- 2023-09-10T00:35:03.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:03 smithi099 ceph-mon[102692]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 49 KiB/s wr, 3 op/s 2023-09-10T00:35:03.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:03.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:03.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:03.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:03.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:03.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:03.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:03.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:03.474 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:03 smithi044 ceph-mon[98676]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 49 KiB/s wr, 3 op/s 2023-09-10T00:35:03.475 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:03 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:03.475 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:03 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:03.475 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:03 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:03.475 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:03 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:03.476 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:03 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:03.476 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:03 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:03.476 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:03 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:03.476 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:03 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:03.520 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-10T00:35:03.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:03 smithi131 ceph-mon[103713]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 49 KiB/s wr, 3 op/s 2023-09-10T00:35:03.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:03 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:03.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:03 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:03.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:03 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:03.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:03 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:03.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:03 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:03.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:03 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:03.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:03 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:03.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:03 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:04.038 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 10 00:35:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-3[130551]: 2023-09-10T00:35:03.922+0000 7f5bb699e640 -1 osd.3 0 log_to_monitors true 2023-09-10T00:35:04.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:04 smithi044 ceph-mon[98676]: from='osd.3 [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-10T00:35:04.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:04 smithi044 ceph-mon[98676]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-10T00:35:04.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:04 smithi099 ceph-mon[102692]: from='osd.3 [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-10T00:35:04.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:04 smithi099 ceph-mon[102692]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-10T00:35:04.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:04 smithi131 ceph-mon[103713]: from='osd.3 [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-10T00:35:04.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:04 smithi131 ceph-mon[103713]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-10T00:35:04.996 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-10T00:35:04.996 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-10T00:35:04.996 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2023-09-10T00:35:04.996 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2023-09-10T00:35:04.997 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111512 s, 940 MB/s 2023-09-10T00:35:04.997 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2023-09-10T00:35:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:05 smithi099 ceph-mon[102692]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 49 KiB/s wr, 3 op/s 2023-09-10T00:35:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:05 smithi099 ceph-mon[102692]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-10T00:35:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:05 smithi099 ceph-mon[102692]: from='osd.3 [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:35:05.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:05 smithi099 ceph-mon[102692]: osdmap e23: 4 total, 3 up, 4 in 2023-09-10T00:35:05.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:05 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:35:05.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:05 smithi099 ceph-mon[102692]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:35:05.459 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi099:vg_nvme/lv_3 2023-09-10T00:35:05.535 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 10 00:35:05 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-3[130551]: 2023-09-10T00:35:05.230+0000 7f5bac015700 -1 osd.3 0 waiting for initial osdmap 2023-09-10T00:35:05.535 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 10 00:35:05 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-3[130551]: 2023-09-10T00:35:05.235+0000 7f5ba5605700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-10T00:35:05.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:05 smithi044 ceph-mon[98676]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 49 KiB/s wr, 3 op/s 2023-09-10T00:35:05.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:05 smithi044 ceph-mon[98676]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-10T00:35:05.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:05 smithi044 ceph-mon[98676]: from='osd.3 [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:35:05.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:05 smithi044 ceph-mon[98676]: osdmap e23: 4 total, 3 up, 4 in 2023-09-10T00:35:05.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:05 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:35:05.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:05 smithi044 ceph-mon[98676]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:35:05.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:05 smithi131 ceph-mon[103713]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 49 KiB/s wr, 3 op/s 2023-09-10T00:35:05.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:05 smithi131 ceph-mon[103713]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-10T00:35:05.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:05 smithi131 ceph-mon[103713]: from='osd.3 [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:35:05.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:05 smithi131 ceph-mon[103713]: osdmap e23: 4 total, 3 up, 4 in 2023-09-10T00:35:05.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:05 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:35:05.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:05 smithi131 ceph-mon[103713]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:35:06.013 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-10T00:35:06.497 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2023-09-10T00:35:06.498 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: osdmap e24: 4 total, 3 up, 4 in 2023-09-10T00:35:06.498 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:35:06.498 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:35:06.498 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: osd.3 [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584] boot 2023-09-10T00:35:06.498 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: osdmap e25: 4 total, 4 up, 4 in 2023-09-10T00:35:06.499 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:35:06.499 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: Detected new or changed devices on smithi044 2023-09-10T00:35:06.499 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:06.499 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:06.499 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:06.500 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:06.500 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi044 to 8586M 2023-09-10T00:35:06.500 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:06.500 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:06.500 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:06.501 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:06.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2023-09-10T00:35:06.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: osdmap e24: 4 total, 3 up, 4 in 2023-09-10T00:35:06.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:35:06.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:35:06.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: osd.3 [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584] boot 2023-09-10T00:35:06.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: osdmap e25: 4 total, 4 up, 4 in 2023-09-10T00:35:06.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:35:06.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: Detected new or changed devices on smithi044 2023-09-10T00:35:06.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:06.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:06.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:06.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:06.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi044 to 8586M 2023-09-10T00:35:06.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:06.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:06.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:06.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:06.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2023-09-10T00:35:06.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: osdmap e24: 4 total, 3 up, 4 in 2023-09-10T00:35:06.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:35:06.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:35:06.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: osd.3 [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584] boot 2023-09-10T00:35:06.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: osdmap e25: 4 total, 4 up, 4 in 2023-09-10T00:35:06.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:35:06.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: Detected new or changed devices on smithi044 2023-09-10T00:35:06.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:06.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:06.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:06.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:06.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi044 to 8586M 2023-09-10T00:35:06.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:06.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:06.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:06.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:07.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.192+0000 7f3b8be2d700 1 -- 172.21.15.99:0/888371805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b841019f0 msgr2=0x7f3b84101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:35:07.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.192+0000 7f3b8be2d700 1 --2- 172.21.15.99:0/888371805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b841019f0 0x7f3b84101e50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009990 tx=0x7f3b80009ca0 comp rx=0 tx=0).stop 2023-09-10T00:35:07.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.192+0000 7f3b8be2d700 1 -- 172.21.15.99:0/888371805 shutdown_connections 2023-09-10T00:35:07.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.192+0000 7f3b8be2d700 1 --2- 172.21.15.99:0/888371805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b84102390 0x7f3b84108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:07.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.192+0000 7f3b8be2d700 1 --2- 172.21.15.99:0/888371805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b840fe970 0x7f3b840fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:07.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.192+0000 7f3b8be2d700 1 --2- 172.21.15.99:0/888371805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b841019f0 0x7f3b84101e50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:07.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.192+0000 7f3b8be2d700 1 -- 172.21.15.99:0/888371805 >> 172.21.15.99:0/888371805 conn(0x7f3b840fa2b0 msgr2=0x7f3b840fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:35:07.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.192+0000 7f3b8be2d700 1 -- 172.21.15.99:0/888371805 shutdown_connections 2023-09-10T00:35:07.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.192+0000 7f3b8be2d700 1 -- 172.21.15.99:0/888371805 wait complete. 2023-09-10T00:35:07.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b8be2d700 1 Processor -- start 2023-09-10T00:35:07.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b8be2d700 1 -- start start 2023-09-10T00:35:07.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b8be2d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b840fe970 0x7f3b84196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:35:07.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b8be2d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b841019f0 0x7f3b84196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:35:07.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b8be2d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b84102390 0x7f3b8419d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:35:07.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b8be2d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8419deb0 con 0x7f3b84102390 2023-09-10T00:35:07.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b8be2d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8419dff0 con 0x7f3b840fe970 2023-09-10T00:35:07.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b8be2d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8419e180 con 0x7f3b841019f0 2023-09-10T00:35:07.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b89bc9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b840fe970 0x7f3b84196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:35:07.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b89bc9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b840fe970 0x7f3b84196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40674/0 (socket says 172.21.15.99:40674) 2023-09-10T00:35:07.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b89bc9700 1 -- 172.21.15.99:0/1362255588 learned_addr learned my addr 172.21.15.99:0/1362255588 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-09-10T00:35:07.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b8a3ca700 1 --2- 172.21.15.99:0/1362255588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b84102390 0x7f3b8419d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:35:07.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b893c8700 1 --2- 172.21.15.99:0/1362255588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b841019f0 0x7f3b84196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:35:07.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b89bc9700 1 -- 172.21.15.99:0/1362255588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b841019f0 msgr2=0x7f3b84196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:35:07.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b89bc9700 1 --2- 172.21.15.99:0/1362255588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b841019f0 0x7f3b84196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:07.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b89bc9700 1 -- 172.21.15.99:0/1362255588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b84102390 msgr2=0x7f3b8419d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:35:07.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b89bc9700 1 --2- 172.21.15.99:0/1362255588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b84102390 0x7f3b8419d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:07.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b89bc9700 1 -- 172.21.15.99:0/1362255588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b800095f0 con 0x7f3b840fe970 2023-09-10T00:35:07.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b8a3ca700 1 --2- 172.21.15.99:0/1362255588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b84102390 0x7f3b8419d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:35:07.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.193+0000 7f3b89bc9700 1 --2- 172.21.15.99:0/1362255588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b840fe970 0x7f3b84196320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400d980 tx=0x7f3b7400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:35:07.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.194+0000 7f3b7affd700 1 -- 172.21.15.99:0/1362255588 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7400de60 con 0x7f3b840fe970 2023-09-10T00:35:07.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.194+0000 7f3b7affd700 1 -- 172.21.15.99:0/1362255588 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b740044b0 con 0x7f3b840fe970 2023-09-10T00:35:07.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.194+0000 7f3b8be2d700 1 -- 172.21.15.99:0/1362255588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8419e460 con 0x7f3b840fe970 2023-09-10T00:35:07.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.194+0000 7f3b7affd700 1 -- 172.21.15.99:0/1362255588 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b74003b20 con 0x7f3b840fe970 2023-09-10T00:35:07.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.194+0000 7f3b8be2d700 1 -- 172.21.15.99:0/1362255588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8419ea30 con 0x7f3b840fe970 2023-09-10T00:35:07.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.195+0000 7f3b8be2d700 1 -- 172.21.15.99:0/1362255588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b8410be00 con 0x7f3b840fe970 2023-09-10T00:35:07.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.196+0000 7f3b7affd700 1 -- 172.21.15.99:0/1362255588 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f3b74003c80 con 0x7f3b840fe970 2023-09-10T00:35:07.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.196+0000 7f3b7affd700 1 --2- 172.21.15.99:0/1362255588 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3b700a06c0 0x7f3b700a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:35:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.196+0000 7f3b7affd700 1 -- 172.21.15.99:0/1362255588 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f3b740ca920 con 0x7f3b840fe970 2023-09-10T00:35:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.196+0000 7f3b893c8700 1 --2- 172.21.15.99:0/1362255588 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3b700a06c0 0x7f3b700a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:35:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.197+0000 7f3b893c8700 1 --2- 172.21.15.99:0/1362255588 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3b700a06c0 0x7f3b700a2b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3b800095c0 tx=0x7f3b80009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:35:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.200+0000 7f3b7affd700 1 -- 172.21.15.99:0/1362255588 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3b740956d0 con 0x7f3b840fe970 2023-09-10T00:35:07.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:07.396+0000 7f3b8be2d700 1 -- 172.21.15.99:0/1362255588 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b84060a40 con 0x7f3b700a06c0 2023-09-10T00:35:07.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:07 smithi131 ceph-mon[103713]: purged_snaps scrub starts 2023-09-10T00:35:07.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:07 smithi131 ceph-mon[103713]: purged_snaps scrub ok 2023-09-10T00:35:07.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:07 smithi131 ceph-mon[103713]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:35:07.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:07 smithi131 ceph-mon[103713]: osdmap e26: 4 total, 4 up, 4 in 2023-09-10T00:35:07.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:07 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:35:07.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:07 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:35:07.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:07 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:07.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:07 smithi044 ceph-mon[98676]: purged_snaps scrub starts 2023-09-10T00:35:07.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:07 smithi044 ceph-mon[98676]: purged_snaps scrub ok 2023-09-10T00:35:07.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:07 smithi044 ceph-mon[98676]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:35:07.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:07 smithi044 ceph-mon[98676]: osdmap e26: 4 total, 4 up, 4 in 2023-09-10T00:35:07.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:07 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:35:07.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:07 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:35:07.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:07 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:07.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:07 smithi099 ceph-mon[102692]: purged_snaps scrub starts 2023-09-10T00:35:07.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:07 smithi099 ceph-mon[102692]: purged_snaps scrub ok 2023-09-10T00:35:07.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:07 smithi099 ceph-mon[102692]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-09-10T00:35:07.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:07 smithi099 ceph-mon[102692]: osdmap e26: 4 total, 4 up, 4 in 2023-09-10T00:35:07.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:07 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:35:07.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:07 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:35:07.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:07 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:08.472 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:08 smithi099 ceph-mon[102692]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:35:08.472 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:08 smithi099 ceph-mon[102692]: osdmap e27: 4 total, 4 up, 4 in 2023-09-10T00:35:08.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:08 smithi131 ceph-mon[103713]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:35:08.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:08 smithi131 ceph-mon[103713]: osdmap e27: 4 total, 4 up, 4 in 2023-09-10T00:35:08.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:08 smithi044 ceph-mon[98676]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:35:08.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:08 smithi044 ceph-mon[98676]: osdmap e27: 4 total, 4 up, 4 in 2023-09-10T00:35:09.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:09 smithi131 ceph-mon[103713]: pgmap v103: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 155 KiB/s, 0 objects/s recovering 2023-09-10T00:35:09.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:09 smithi131 ceph-mon[103713]: osdmap e28: 4 total, 4 up, 4 in 2023-09-10T00:35:09.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:09 smithi044 ceph-mon[98676]: pgmap v103: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 155 KiB/s, 0 objects/s recovering 2023-09-10T00:35:09.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:09 smithi044 ceph-mon[98676]: osdmap e28: 4 total, 4 up, 4 in 2023-09-10T00:35:09.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:09 smithi099 ceph-mon[102692]: pgmap v103: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 155 KiB/s, 0 objects/s recovering 2023-09-10T00:35:09.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:09 smithi099 ceph-mon[102692]: osdmap e28: 4 total, 4 up, 4 in 2023-09-10T00:35:10.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:10 smithi131 ceph-mon[103713]: osdmap e29: 4 total, 4 up, 4 in 2023-09-10T00:35:10.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:10 smithi131 ceph-mon[103713]: from='client.? 172.21.15.99:0/4112456624' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ae06a56-c8b7-4881-8ead-68835a9015a1"}]: dispatch 2023-09-10T00:35:10.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:10 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ae06a56-c8b7-4881-8ead-68835a9015a1"}]: dispatch 2023-09-10T00:35:10.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:10 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ae06a56-c8b7-4881-8ead-68835a9015a1"}]': finished 2023-09-10T00:35:10.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:10 smithi131 ceph-mon[103713]: osdmap e30: 5 total, 4 up, 5 in 2023-09-10T00:35:10.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:10 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:35:10.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:10 smithi131 ceph-mon[103713]: from='client.? 172.21.15.99:0/1764324477' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:35:10.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:10 smithi044 ceph-mon[98676]: osdmap e29: 4 total, 4 up, 4 in 2023-09-10T00:35:10.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:10 smithi044 ceph-mon[98676]: from='client.? 172.21.15.99:0/4112456624' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ae06a56-c8b7-4881-8ead-68835a9015a1"}]: dispatch 2023-09-10T00:35:10.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:10 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ae06a56-c8b7-4881-8ead-68835a9015a1"}]: dispatch 2023-09-10T00:35:10.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:10 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ae06a56-c8b7-4881-8ead-68835a9015a1"}]': finished 2023-09-10T00:35:10.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:10 smithi044 ceph-mon[98676]: osdmap e30: 5 total, 4 up, 5 in 2023-09-10T00:35:10.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:10 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:35:10.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:10 smithi044 ceph-mon[98676]: from='client.? 172.21.15.99:0/1764324477' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:35:10.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:10 smithi099 ceph-mon[102692]: osdmap e29: 4 total, 4 up, 4 in 2023-09-10T00:35:10.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:10 smithi099 ceph-mon[102692]: from='client.? 172.21.15.99:0/4112456624' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ae06a56-c8b7-4881-8ead-68835a9015a1"}]: dispatch 2023-09-10T00:35:10.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:10 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ae06a56-c8b7-4881-8ead-68835a9015a1"}]: dispatch 2023-09-10T00:35:10.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:10 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ae06a56-c8b7-4881-8ead-68835a9015a1"}]': finished 2023-09-10T00:35:10.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:10 smithi099 ceph-mon[102692]: osdmap e30: 5 total, 4 up, 5 in 2023-09-10T00:35:10.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:35:10.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:10 smithi099 ceph-mon[102692]: from='client.? 172.21.15.99:0/1764324477' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:35:12.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:11 smithi131 ceph-mon[103713]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 163 KiB/s, 0 objects/s recovering 2023-09-10T00:35:12.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:11 smithi044 ceph-mon[98676]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 163 KiB/s, 0 objects/s recovering 2023-09-10T00:35:12.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:11 smithi099 ceph-mon[102692]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 163 KiB/s, 0 objects/s recovering 2023-09-10T00:35:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:13 smithi131 ceph-mon[103713]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-09-10T00:35:14.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:13 smithi044 ceph-mon[98676]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-09-10T00:35:14.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:13 smithi099 ceph-mon[102692]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-09-10T00:35:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:15 smithi131 ceph-mon[103713]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:16.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:15 smithi044 ceph-mon[98676]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:16.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:15 smithi099 ceph-mon[102692]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:17.031 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-10T00:35:17.031 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-10T00:35:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:17.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-10T00:35:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:18.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:17 smithi131 ceph-mon[103713]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:18.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:17 smithi131 ceph-mon[103713]: Deploying daemon osd.4 on smithi099 2023-09-10T00:35:18.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:17 smithi044 ceph-mon[98676]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:18.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:17 smithi044 ceph-mon[98676]: Deploying daemon osd.4 on smithi099 2023-09-10T00:35:18.102 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:17 smithi099 ceph-mon[102692]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:18.103 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:17 smithi099 ceph-mon[102692]: Deploying daemon osd.4 on smithi099 2023-09-10T00:35:20.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:19 smithi131 ceph-mon[103713]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:20.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:19 smithi044 ceph-mon[98676]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:20.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:19 smithi099 ceph-mon[102692]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:21.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:21 smithi131 ceph-mon[103713]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:21.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:21.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:21.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:21.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:21 smithi044 ceph-mon[98676]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:21.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:21.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:21.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:21.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:21 smithi099 ceph-mon[102692]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:21.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:21.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:21.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:23.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:23.360+0000 7f3b7affd700 1 -- 172.21.15.99:0/1362255588 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3b84060a40 con 0x7f3b700a06c0 2023-09-10T00:35:23.361 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 4 on host 'smithi099' 2023-09-10T00:35:23.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:23.362+0000 7f3b8be2d700 1 -- 172.21.15.99:0/1362255588 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3b700a06c0 msgr2=0x7f3b700a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:35:23.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:23.362+0000 7f3b8be2d700 1 --2- 172.21.15.99:0/1362255588 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3b700a06c0 0x7f3b700a2b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3b800095c0 tx=0x7f3b80009550 comp rx=0 tx=0).stop 2023-09-10T00:35:23.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:23.362+0000 7f3b8be2d700 1 -- 172.21.15.99:0/1362255588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b840fe970 msgr2=0x7f3b84196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:35:23.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:23.362+0000 7f3b8be2d700 1 --2- 172.21.15.99:0/1362255588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b840fe970 0x7f3b84196320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400d980 tx=0x7f3b7400dd40 comp rx=0 tx=0).stop 2023-09-10T00:35:23.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:23.362+0000 7f3b8be2d700 1 -- 172.21.15.99:0/1362255588 shutdown_connections 2023-09-10T00:35:23.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:23.362+0000 7f3b8be2d700 1 --2- 172.21.15.99:0/1362255588 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3b700a06c0 0x7f3b700a2b80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:23.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:23.362+0000 7f3b8be2d700 1 --2- 172.21.15.99:0/1362255588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b84102390 0x7f3b8419d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:23.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:23.362+0000 7f3b8be2d700 1 --2- 172.21.15.99:0/1362255588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b840fe970 0x7f3b84196320 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:23.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:23.362+0000 7f3b8be2d700 1 --2- 172.21.15.99:0/1362255588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b841019f0 0x7f3b84196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:23.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:23.362+0000 7f3b8be2d700 1 -- 172.21.15.99:0/1362255588 >> 172.21.15.99:0/1362255588 conn(0x7f3b840fa2b0 msgr2=0x7f3b840fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:35:23.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:23.362+0000 7f3b8be2d700 1 -- 172.21.15.99:0/1362255588 shutdown_connections 2023-09-10T00:35:23.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:35:23.362+0000 7f3b8be2d700 1 -- 172.21.15.99:0/1362255588 wait complete. 2023-09-10T00:35:23.699 DEBUG:teuthology.orchestra.run.smithi099:osd.4> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.4.service 2023-09-10T00:35:23.702 INFO:tasks.cephadm:Deploying osd.5 on smithi131 with /dev/vg_nvme/lv_3... 2023-09-10T00:35:23.703 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-10T00:35:23.735 INFO:journalctl@ceph.osd.4.smithi099.stdout:-- Logs begin at Sun 2023-09-10 00:17:22 UTC. -- 2023-09-10T00:35:24.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:23 smithi131 ceph-mon[103713]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:24.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:24.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:24.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:24.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:24.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:24.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:24.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:24.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:24.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:23 smithi044 ceph-mon[98676]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:24.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:24.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:24.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:24.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:24.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:24.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:24.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:24.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:24.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:23 smithi099 ceph-mon[102692]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:24.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:24.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:24.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:24.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:24.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:24.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:24.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:24.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:24.271 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-10T00:35:24.764 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 10 00:35:24 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-4[115726]: 2023-09-10T00:35:24.502+0000 7f48891ae640 -1 osd.4 0 log_to_monitors true 2023-09-10T00:35:25.042 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:24 smithi099 ceph-mon[102692]: from='osd.4 [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-10T00:35:25.042 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:24 smithi099 ceph-mon[102692]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-10T00:35:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:24 smithi131 ceph-mon[103713]: from='osd.4 [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-10T00:35:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:24 smithi131 ceph-mon[103713]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-10T00:35:25.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:24 smithi044 ceph-mon[98676]: from='osd.4 [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-10T00:35:25.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:24 smithi044 ceph-mon[98676]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-10T00:35:25.809 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-10T00:35:25.809 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-10T00:35:25.810 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2023-09-10T00:35:25.810 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2023-09-10T00:35:25.810 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012273 s, 854 MB/s 2023-09-10T00:35:25.810 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2023-09-10T00:35:26.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:25 smithi131 ceph-mon[103713]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:26.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:25 smithi131 ceph-mon[103713]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-10T00:35:26.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:25 smithi131 ceph-mon[103713]: osdmap e31: 5 total, 4 up, 5 in 2023-09-10T00:35:26.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:25 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:35:26.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:25 smithi131 ceph-mon[103713]: from='osd.4 [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:35:26.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:25 smithi131 ceph-mon[103713]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:35:26.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:25 smithi044 ceph-mon[98676]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:26.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:25 smithi044 ceph-mon[98676]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-10T00:35:26.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:25 smithi044 ceph-mon[98676]: osdmap e31: 5 total, 4 up, 5 in 2023-09-10T00:35:26.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:25 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:35:26.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:25 smithi044 ceph-mon[98676]: from='osd.4 [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:35:26.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:25 smithi044 ceph-mon[98676]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:35:26.106 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 10 00:35:25 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-4[115726]: 2023-09-10T00:35:25.776+0000 7f487e825700 -1 osd.4 0 waiting for initial osdmap 2023-09-10T00:35:26.107 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 10 00:35:25 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-4[115726]: 2023-09-10T00:35:25.784+0000 7f4878616700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-10T00:35:26.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:25 smithi099 ceph-mon[102692]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:26.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:25 smithi099 ceph-mon[102692]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-10T00:35:26.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:25 smithi099 ceph-mon[102692]: osdmap e31: 5 total, 4 up, 5 in 2023-09-10T00:35:26.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:25 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:35:26.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:25 smithi099 ceph-mon[102692]: from='osd.4 [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:35:26.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:25 smithi099 ceph-mon[102692]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:35:26.323 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi131:vg_nvme/lv_3 2023-09-10T00:35:26.878 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-10T00:35:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:26 smithi131 ceph-mon[103713]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-09-10T00:35:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:26 smithi131 ceph-mon[103713]: osdmap e32: 5 total, 4 up, 5 in 2023-09-10T00:35:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:26 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:35:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:26 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:35:27.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:26 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:27.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:26 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:27.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:26 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:27.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:26 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:27.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:26 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:27.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:26 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:27.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:26 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:27.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:26 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:27.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:26 smithi044 ceph-mon[98676]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-09-10T00:35:27.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:26 smithi044 ceph-mon[98676]: osdmap e32: 5 total, 4 up, 5 in 2023-09-10T00:35:27.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:26 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:35:27.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:26 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:35:27.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:26 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:27.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:26 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:27.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:26 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:27.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:26 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:27.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:26 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:27.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:26 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:27.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:26 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:27.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:26 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:26 smithi099 ceph-mon[102692]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-09-10T00:35:27.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:26 smithi099 ceph-mon[102692]: osdmap e32: 5 total, 4 up, 5 in 2023-09-10T00:35:27.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:26 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:35:27.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:26 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:35:27.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:26 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:27.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:26 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:27.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:26 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:27.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:26 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:27.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:26 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:27.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:26 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:27.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:26 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:27.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:26 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:27 smithi131 ceph-mon[103713]: purged_snaps scrub starts 2023-09-10T00:35:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:27 smithi131 ceph-mon[103713]: purged_snaps scrub ok 2023-09-10T00:35:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:27 smithi131 ceph-mon[103713]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:28.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:27 smithi131 ceph-mon[103713]: Detected new or changed devices on smithi099 2023-09-10T00:35:28.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:27 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi099 to 8584M 2023-09-10T00:35:28.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:27 smithi131 ceph-mon[103713]: osd.4 [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490] boot 2023-09-10T00:35:28.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:27 smithi131 ceph-mon[103713]: osdmap e33: 5 total, 5 up, 5 in 2023-09-10T00:35:28.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:27 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:35:28.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:27 smithi044 ceph-mon[98676]: purged_snaps scrub starts 2023-09-10T00:35:28.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:27 smithi044 ceph-mon[98676]: purged_snaps scrub ok 2023-09-10T00:35:28.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:27 smithi044 ceph-mon[98676]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:28.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:27 smithi044 ceph-mon[98676]: Detected new or changed devices on smithi099 2023-09-10T00:35:28.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:27 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi099 to 8584M 2023-09-10T00:35:28.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:27 smithi044 ceph-mon[98676]: osd.4 [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490] boot 2023-09-10T00:35:28.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:27 smithi044 ceph-mon[98676]: osdmap e33: 5 total, 5 up, 5 in 2023-09-10T00:35:28.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:35:28.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:27 smithi099 ceph-mon[102692]: purged_snaps scrub starts 2023-09-10T00:35:28.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:27 smithi099 ceph-mon[102692]: purged_snaps scrub ok 2023-09-10T00:35:28.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:27 smithi099 ceph-mon[102692]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-10T00:35:28.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:27 smithi099 ceph-mon[102692]: Detected new or changed devices on smithi099 2023-09-10T00:35:28.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:27 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi099 to 8584M 2023-09-10T00:35:28.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:27 smithi099 ceph-mon[102692]: osd.4 [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490] boot 2023-09-10T00:35:28.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:27 smithi099 ceph-mon[102692]: osdmap e33: 5 total, 5 up, 5 in 2023-09-10T00:35:28.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:27 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:35:28.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.270+0000 7f1bb386e700 1 -- 172.21.15.131:0/416488453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1022d0 msgr2=0x7f1bac108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:35:28.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.270+0000 7f1bb386e700 1 --2- 172.21.15.131:0/416488453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1022d0 0x7f1bac108b80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400b0a0 tx=0x7f1ba400b3b0 comp rx=0 tx=0).stop 2023-09-10T00:35:28.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.270+0000 7f1bb386e700 1 -- 172.21.15.131:0/416488453 shutdown_connections 2023-09-10T00:35:28.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.270+0000 7f1bb386e700 1 --2- 172.21.15.131:0/416488453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1022d0 0x7f1bac108b80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:28.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.270+0000 7f1bb386e700 1 --2- 172.21.15.131:0/416488453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bac0fe950 0x7f1bac0fed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:28.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.270+0000 7f1bb386e700 1 --2- 172.21.15.131:0/416488453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bac101930 0x7f1bac101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:28.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.270+0000 7f1bb386e700 1 -- 172.21.15.131:0/416488453 >> 172.21.15.131:0/416488453 conn(0x7f1bac0fa2b0 msgr2=0x7f1bac0fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:35:28.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.270+0000 7f1bb386e700 1 -- 172.21.15.131:0/416488453 shutdown_connections 2023-09-10T00:35:28.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.270+0000 7f1bb386e700 1 -- 172.21.15.131:0/416488453 wait complete. 2023-09-10T00:35:28.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.271+0000 7f1bb386e700 1 Processor -- start 2023-09-10T00:35:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.271+0000 7f1bb386e700 1 -- start start 2023-09-10T00:35:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.271+0000 7f1bb386e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bac0fe950 0x7f1bac1962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:35:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.271+0000 7f1bb386e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bac101930 0x7f1bac196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:35:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.271+0000 7f1bb386e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1022d0 0x7f1bac19d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:35:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.271+0000 7f1bb386e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac19de70 con 0x7f1bac101930 2023-09-10T00:35:28.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.271+0000 7f1bb386e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac19dfb0 con 0x7f1bac0fe950 2023-09-10T00:35:28.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.271+0000 7f1bb386e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac19e140 con 0x7f1bac1022d0 2023-09-10T00:35:28.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.271+0000 7f1bb1e0b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1022d0 0x7f1bac19d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:35:28.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.271+0000 7f1bb1e0b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1022d0 0x7f1bac19d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52972/0 (socket says 172.21.15.131:52972) 2023-09-10T00:35:28.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.271+0000 7f1bb1e0b700 1 -- 172.21.15.131:0/1836188493 learned_addr learned my addr 172.21.15.131:0/1836188493 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-10T00:35:28.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.271+0000 7f1bb0e09700 1 --2- 172.21.15.131:0/1836188493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bac101930 0x7f1bac196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:35:28.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.271+0000 7f1bb160a700 1 --2- 172.21.15.131:0/1836188493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bac0fe950 0x7f1bac1962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:35:28.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.271+0000 7f1bb1e0b700 1 -- 172.21.15.131:0/1836188493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bac0fe950 msgr2=0x7f1bac1962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:35:28.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.272+0000 7f1bb1e0b700 1 --2- 172.21.15.131:0/1836188493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bac0fe950 0x7f1bac1962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:28.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.272+0000 7f1bb1e0b700 1 -- 172.21.15.131:0/1836188493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bac101930 msgr2=0x7f1bac196820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:35:28.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.272+0000 7f1bb1e0b700 1 --2- 172.21.15.131:0/1836188493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bac101930 0x7f1bac196820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:28.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.272+0000 7f1bb1e0b700 1 -- 172.21.15.131:0/1836188493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba4009d00 con 0x7f1bac1022d0 2023-09-10T00:35:28.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.272+0000 7f1bb160a700 1 --2- 172.21.15.131:0/1836188493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bac0fe950 0x7f1bac1962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:35:28.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.272+0000 7f1bb1e0b700 1 --2- 172.21.15.131:0/1836188493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1022d0 0x7f1bac19d8a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4007a30 tx=0x7f1ba4004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:35:28.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.272+0000 7f1ba27fc700 1 -- 172.21.15.131:0/1836188493 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba400e040 con 0x7f1bac1022d0 2023-09-10T00:35:28.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.272+0000 7f1bb386e700 1 -- 172.21.15.131:0/1836188493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bac19e3c0 con 0x7f1bac1022d0 2023-09-10T00:35:28.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.272+0000 7f1ba27fc700 1 -- 172.21.15.131:0/1836188493 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ba401db20 con 0x7f1bac1022d0 2023-09-10T00:35:28.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.272+0000 7f1ba27fc700 1 -- 172.21.15.131:0/1836188493 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba401bb90 con 0x7f1bac1022d0 2023-09-10T00:35:28.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.272+0000 7f1bb386e700 1 -- 172.21.15.131:0/1836188493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bac19e830 con 0x7f1bac1022d0 2023-09-10T00:35:28.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.273+0000 7f1bb386e700 1 -- 172.21.15.131:0/1836188493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bac10bd40 con 0x7f1bac1022d0 2023-09-10T00:35:28.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.274+0000 7f1ba27fc700 1 -- 172.21.15.131:0/1836188493 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f1ba401dc90 con 0x7f1bac1022d0 2023-09-10T00:35:28.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.274+0000 7f1ba27fc700 1 --2- 172.21.15.131:0/1836188493 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b980a06c0 0x7f1b980a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:35:28.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.275+0000 7f1ba27fc700 1 -- 172.21.15.131:0/1836188493 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f1ba40ccc20 con 0x7f1bac1022d0 2023-09-10T00:35:28.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.275+0000 7f1bb160a700 1 --2- 172.21.15.131:0/1836188493 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b980a06c0 0x7f1b980a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:35:28.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.275+0000 7f1bb160a700 1 --2- 172.21.15.131:0/1836188493 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b980a06c0 0x7f1b980a2b80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c005950 tx=0x7f1b9c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:35:28.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.279+0000 7f1ba27fc700 1 -- 172.21.15.131:0/1836188493 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1ba40977b0 con 0x7f1bac1022d0 2023-09-10T00:35:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:28.474+0000 7f1bb386e700 1 -- 172.21.15.131:0/1836188493 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bac060a40 con 0x7f1b980a06c0 2023-09-10T00:35:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:28 smithi131 ceph-mon[103713]: osdmap e34: 5 total, 5 up, 5 in 2023-09-10T00:35:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:28 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:35:29.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:28 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:35:29.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:28 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:28 smithi044 ceph-mon[98676]: osdmap e34: 5 total, 5 up, 5 in 2023-09-10T00:35:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:28 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:35:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:28 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:35:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:28 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:28 smithi099 ceph-mon[102692]: osdmap e34: 5 total, 5 up, 5 in 2023-09-10T00:35:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:28 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:35:29.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:28 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:35:29.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:28 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:29.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:29 smithi131 ceph-mon[103713]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:29.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:29 smithi131 ceph-mon[103713]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:35:30.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:29 smithi044 ceph-mon[98676]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:30.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:29 smithi044 ceph-mon[98676]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:35:30.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:29 smithi099 ceph-mon[102692]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:30.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:29 smithi099 ceph-mon[102692]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:35:32.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:31 smithi131 ceph-mon[103713]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:32.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:31 smithi131 ceph-mon[103713]: from='client.? 172.21.15.131:0/4183250473' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "917c392a-bb6c-408b-80c6-9d81c194b0b9"}]: dispatch 2023-09-10T00:35:32.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:31 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "917c392a-bb6c-408b-80c6-9d81c194b0b9"}]: dispatch 2023-09-10T00:35:32.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:31 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "917c392a-bb6c-408b-80c6-9d81c194b0b9"}]': finished 2023-09-10T00:35:32.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:31 smithi131 ceph-mon[103713]: osdmap e35: 6 total, 5 up, 6 in 2023-09-10T00:35:32.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:31 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:32.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:31 smithi131 ceph-mon[103713]: from='client.? 172.21.15.131:0/2236699301' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:35:32.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:31 smithi044 ceph-mon[98676]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:32.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:31 smithi044 ceph-mon[98676]: from='client.? 172.21.15.131:0/4183250473' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "917c392a-bb6c-408b-80c6-9d81c194b0b9"}]: dispatch 2023-09-10T00:35:32.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:31 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "917c392a-bb6c-408b-80c6-9d81c194b0b9"}]: dispatch 2023-09-10T00:35:32.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:31 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "917c392a-bb6c-408b-80c6-9d81c194b0b9"}]': finished 2023-09-10T00:35:32.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:31 smithi044 ceph-mon[98676]: osdmap e35: 6 total, 5 up, 6 in 2023-09-10T00:35:32.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:31 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:32.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:31 smithi044 ceph-mon[98676]: from='client.? 172.21.15.131:0/2236699301' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:35:32.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:31 smithi099 ceph-mon[102692]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:32.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:31 smithi099 ceph-mon[102692]: from='client.? 172.21.15.131:0/4183250473' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "917c392a-bb6c-408b-80c6-9d81c194b0b9"}]: dispatch 2023-09-10T00:35:32.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:31 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "917c392a-bb6c-408b-80c6-9d81c194b0b9"}]: dispatch 2023-09-10T00:35:32.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:31 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "917c392a-bb6c-408b-80c6-9d81c194b0b9"}]': finished 2023-09-10T00:35:32.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:31 smithi099 ceph-mon[102692]: osdmap e35: 6 total, 5 up, 6 in 2023-09-10T00:35:32.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:31 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:32.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:31 smithi099 ceph-mon[102692]: from='client.? 172.21.15.131:0/2236699301' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:35:34.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:33 smithi131 ceph-mon[103713]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:34.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:33 smithi044 ceph-mon[98676]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:34.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:33 smithi099 ceph-mon[102692]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:36.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:35 smithi044 ceph-mon[98676]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:36.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:35 smithi099 ceph-mon[102692]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:36.139 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:35 smithi131 ceph-mon[103713]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:38.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:37 smithi131 ceph-mon[103713]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:38.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-10T00:35:38.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:38.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:37 smithi044 ceph-mon[98676]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-10T00:35:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:38.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:37 smithi099 ceph-mon[102692]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:38.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-10T00:35:38.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:38.901 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:38 smithi131 ceph-mon[103713]: Deploying daemon osd.5 on smithi131 2023-09-10T00:35:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:38 smithi044 ceph-mon[98676]: Deploying daemon osd.5 on smithi131 2023-09-10T00:35:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:38 smithi099 ceph-mon[102692]: Deploying daemon osd.5 on smithi131 2023-09-10T00:35:39.990 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:39 smithi131 ceph-mon[103713]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:40.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:39 smithi099 ceph-mon[102692]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:40.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:39 smithi044 ceph-mon[98676]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:41.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:41 smithi131 ceph-mon[103713]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:41.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:41 smithi044 ceph-mon[98676]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:41.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:41 smithi099 ceph-mon[102692]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:43.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:43 smithi131 ceph-mon[103713]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:43.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:43.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:43.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:43.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:43 smithi044 ceph-mon[98676]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:43.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:43.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:43.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:43.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:43 smithi099 ceph-mon[102692]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:43.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:43.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:43.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:45.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:45.189+0000 7f1ba27fc700 1 -- 172.21.15.131:0/1836188493 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1bac060a40 con 0x7f1b980a06c0 2023-09-10T00:35:45.191 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 5 on host 'smithi131' 2023-09-10T00:35:45.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:45.192+0000 7f1bb386e700 1 -- 172.21.15.131:0/1836188493 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b980a06c0 msgr2=0x7f1b980a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:35:45.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:45.192+0000 7f1bb386e700 1 --2- 172.21.15.131:0/1836188493 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b980a06c0 0x7f1b980a2b80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c005950 tx=0x7f1b9c0058e0 comp rx=0 tx=0).stop 2023-09-10T00:35:45.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:45.192+0000 7f1bb386e700 1 -- 172.21.15.131:0/1836188493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1022d0 msgr2=0x7f1bac19d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:35:45.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:45.192+0000 7f1bb386e700 1 --2- 172.21.15.131:0/1836188493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1022d0 0x7f1bac19d8a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4007a30 tx=0x7f1ba4004d00 comp rx=0 tx=0).stop 2023-09-10T00:35:45.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:45.192+0000 7f1bb386e700 1 -- 172.21.15.131:0/1836188493 shutdown_connections 2023-09-10T00:35:45.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:45.192+0000 7f1bb386e700 1 --2- 172.21.15.131:0/1836188493 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b980a06c0 0x7f1b980a2b80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:45.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:45.192+0000 7f1bb386e700 1 --2- 172.21.15.131:0/1836188493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1022d0 0x7f1bac19d8a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:45.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:45.192+0000 7f1bb386e700 1 --2- 172.21.15.131:0/1836188493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bac0fe950 0x7f1bac1962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:45.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:45.192+0000 7f1bb386e700 1 --2- 172.21.15.131:0/1836188493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bac101930 0x7f1bac196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:45.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:45.192+0000 7f1bb386e700 1 -- 172.21.15.131:0/1836188493 >> 172.21.15.131:0/1836188493 conn(0x7f1bac0fa2b0 msgr2=0x7f1bac0fc540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:35:45.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:45.192+0000 7f1bb386e700 1 -- 172.21.15.131:0/1836188493 shutdown_connections 2023-09-10T00:35:45.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:35:45.192+0000 7f1bb386e700 1 -- 172.21.15.131:0/1836188493 wait complete. 2023-09-10T00:35:45.662 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:45 smithi131 ceph-mon[103713]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:45.663 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:45.663 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:45.663 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:45.663 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:45.664 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:45.664 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:45.664 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:45.664 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:45.674 DEBUG:teuthology.orchestra.run.smithi131:osd.5> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.5.service 2023-09-10T00:35:45.676 INFO:tasks.cephadm:Deploying osd.6 on smithi044 with /dev/vg_nvme/lv_2... 2023-09-10T00:35:45.677 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-10T00:35:45.717 INFO:journalctl@ceph.osd.5.smithi131.stdout:-- Logs begin at Sun 2023-09-10 00:17:05 UTC. -- 2023-09-10T00:35:45.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:45 smithi044 ceph-mon[98676]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:45.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:45.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:45.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:45.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:45.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:45.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:45.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:45.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:45.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:45 smithi099 ceph-mon[102692]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:45.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:45.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:45.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:45.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:45.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:45.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:45.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:45.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:46.265 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:35:46.399 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 00:35:46 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T00:35:46.137+0000 7f1a5378a640 -1 osd.5 0 log_to_monitors true 2023-09-10T00:35:46.661 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:46 smithi131 ceph-mon[103713]: from='osd.5 [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-10T00:35:46.661 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:46 smithi131 ceph-mon[103713]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-10T00:35:46.754 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:46 smithi044 ceph-mon[98676]: from='osd.5 [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-10T00:35:46.754 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:46 smithi044 ceph-mon[98676]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-10T00:35:46.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:46 smithi099 ceph-mon[102692]: from='osd.5 [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-10T00:35:46.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:46 smithi099 ceph-mon[102692]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-10T00:35:47.761 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-10T00:35:47.761 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-10T00:35:47.761 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2023-09-10T00:35:47.762 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2023-09-10T00:35:47.762 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109355 s, 959 MB/s 2023-09-10T00:35:47.762 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2023-09-10T00:35:47.795 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 00:35:47 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T00:35:47.545+0000 7f1a48e01700 -1 osd.5 0 waiting for initial osdmap 2023-09-10T00:35:47.795 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 00:35:47 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T00:35:47.554+0000 7f1a423f1700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-10T00:35:47.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:47 smithi131 ceph-mon[103713]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:47.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:47 smithi131 ceph-mon[103713]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-10T00:35:47.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:47 smithi131 ceph-mon[103713]: osdmap e36: 6 total, 5 up, 6 in 2023-09-10T00:35:47.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:47 smithi131 ceph-mon[103713]: from='osd.5 [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:35:47.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:47 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:47.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:47 smithi131 ceph-mon[103713]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:35:47.800 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:47 smithi044 ceph-mon[98676]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:47.800 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:47 smithi044 ceph-mon[98676]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-10T00:35:47.801 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:47 smithi044 ceph-mon[98676]: osdmap e36: 6 total, 5 up, 6 in 2023-09-10T00:35:47.801 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:47 smithi044 ceph-mon[98676]: from='osd.5 [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:35:47.801 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:47 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:47.801 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:47 smithi044 ceph-mon[98676]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:35:47.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:47 smithi099 ceph-mon[102692]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:47.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:47 smithi099 ceph-mon[102692]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-10T00:35:47.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:47 smithi099 ceph-mon[102692]: osdmap e36: 6 total, 5 up, 6 in 2023-09-10T00:35:47.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:47 smithi099 ceph-mon[102692]: from='osd.5 [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:35:47.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:47 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:47.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:47 smithi099 ceph-mon[102692]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:35:48.110 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi044:vg_nvme/lv_2 2023-09-10T00:35:48.687 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:35:48.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:48 smithi044 ceph-mon[98676]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2023-09-10T00:35:48.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:48 smithi044 ceph-mon[98676]: osdmap e37: 6 total, 5 up, 6 in 2023-09-10T00:35:48.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:48.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:48.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:48.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:48.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:48.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:48.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:48.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:48.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:48.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:48 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:48.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:48 smithi099 ceph-mon[102692]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2023-09-10T00:35:48.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:48 smithi099 ceph-mon[102692]: osdmap e37: 6 total, 5 up, 6 in 2023-09-10T00:35:48.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:48 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:48.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:48 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:48.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:48 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:48.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:48 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:48.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:48 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:48.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:48 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:48.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:48 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:48.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:48 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:48.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:48 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:48.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:48 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:48 smithi131 ceph-mon[103713]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2023-09-10T00:35:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:48 smithi131 ceph-mon[103713]: osdmap e37: 6 total, 5 up, 6 in 2023-09-10T00:35:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:48 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:49.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:48 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:49.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:48 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:49.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:48 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:49.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:48 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:49.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:48 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:35:49.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:48 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:49.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:48 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:49.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:48 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:49.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:48 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:35:49.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:49 smithi044 ceph-mon[98676]: purged_snaps scrub starts 2023-09-10T00:35:49.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:49 smithi044 ceph-mon[98676]: purged_snaps scrub ok 2023-09-10T00:35:49.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:49 smithi044 ceph-mon[98676]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:49.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:49 smithi044 ceph-mon[98676]: Detected new or changed devices on smithi131 2023-09-10T00:35:49.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:49 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi131 to 8584M 2023-09-10T00:35:49.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:49.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:49 smithi044 ceph-mon[98676]: osd.5 [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489] boot 2023-09-10T00:35:49.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:49 smithi044 ceph-mon[98676]: osdmap e38: 6 total, 6 up, 6 in 2023-09-10T00:35:49.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:49.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:49 smithi099 ceph-mon[102692]: purged_snaps scrub starts 2023-09-10T00:35:49.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:49 smithi099 ceph-mon[102692]: purged_snaps scrub ok 2023-09-10T00:35:49.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:49 smithi099 ceph-mon[102692]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:49.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:49 smithi099 ceph-mon[102692]: Detected new or changed devices on smithi131 2023-09-10T00:35:49.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:49 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi131 to 8584M 2023-09-10T00:35:49.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:49.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:49 smithi099 ceph-mon[102692]: osd.5 [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489] boot 2023-09-10T00:35:49.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:49 smithi099 ceph-mon[102692]: osdmap e38: 6 total, 6 up, 6 in 2023-09-10T00:35:49.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:49.867 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.865+0000 7f2edf8ec700 1 -- 172.21.15.44:0/661710763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ed8106380 msgr2=0x7f2ed81087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:35:49.867 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.865+0000 7f2edf8ec700 1 --2- 172.21.15.44:0/661710763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ed8106380 0x7f2ed81087f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000b0a0 tx=0x7f2ed000b3b0 comp rx=0 tx=0).stop 2023-09-10T00:35:49.868 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.865+0000 7f2edf8ec700 1 -- 172.21.15.44:0/661710763 shutdown_connections 2023-09-10T00:35:49.868 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.865+0000 7f2edf8ec700 1 --2- 172.21.15.44:0/661710763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ed8106380 0x7f2ed81087f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:49.868 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.865+0000 7f2edf8ec700 1 --2- 172.21.15.44:0/661710763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed8101120 0x7f2ed8103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:49.868 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.865+0000 7f2edf8ec700 1 --2- 172.21.15.44:0/661710763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8103a50 0x7f2ed8105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:49.868 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.865+0000 7f2edf8ec700 1 -- 172.21.15.44:0/661710763 >> 172.21.15.44:0/661710763 conn(0x7f2ed80fab10 msgr2=0x7f2ed80fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:35:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.865+0000 7f2edf8ec700 1 -- 172.21.15.44:0/661710763 shutdown_connections 2023-09-10T00:35:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.865+0000 7f2edf8ec700 1 -- 172.21.15.44:0/661710763 wait complete. 2023-09-10T00:35:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edf8ec700 1 Processor -- start 2023-09-10T00:35:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edf8ec700 1 -- start start 2023-09-10T00:35:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edf8ec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed8101120 0x7f2ed8100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:35:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edf8ec700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8103a50 0x7f2ed80ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:35:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edf8ec700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ed8106380 0x7f2ed80ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:35:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edf8ec700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80ffd90 con 0x7f2ed8106380 2023-09-10T00:35:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edf8ec700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80fff00 con 0x7f2ed8101120 2023-09-10T00:35:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edf8ec700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed81a0410 con 0x7f2ed8103a50 2023-09-10T00:35:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edde89700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ed8106380 0x7f2ed80ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:35:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edde89700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ed8106380 0x7f2ed80ff610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45596/0 (socket says 172.21.15.44:45596) 2023-09-10T00:35:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edde89700 1 -- 172.21.15.44:0/2168357849 learned_addr learned my addr 172.21.15.44:0/2168357849 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:35:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edce87700 1 --2- 172.21.15.44:0/2168357849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8103a50 0x7f2ed80ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:35:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edd688700 1 --2- 172.21.15.44:0/2168357849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed8101120 0x7f2ed8100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:35:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edde89700 1 -- 172.21.15.44:0/2168357849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8103a50 msgr2=0x7f2ed80ff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:35:49.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edde89700 1 --2- 172.21.15.44:0/2168357849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8103a50 0x7f2ed80ff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:49.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edde89700 1 -- 172.21.15.44:0/2168357849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed8101120 msgr2=0x7f2ed8100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:35:49.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edde89700 1 --2- 172.21.15.44:0/2168357849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed8101120 0x7f2ed8100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:35:49.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edde89700 1 -- 172.21.15.44:0/2168357849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed0009d00 con 0x7f2ed8106380 2023-09-10T00:35:49.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.866+0000 7f2edce87700 1 --2- 172.21.15.44:0/2168357849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8103a50 0x7f2ed80ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:35:49.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.867+0000 7f2edde89700 1 --2- 172.21.15.44:0/2168357849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ed8106380 0x7f2ed80ff610 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000b070 tx=0x7f2ed0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:35:49.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.867+0000 7f2ece7fc700 1 -- 172.21.15.44:0/2168357849 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed000e040 con 0x7f2ed8106380 2023-09-10T00:35:49.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.867+0000 7f2ece7fc700 1 -- 172.21.15.44:0/2168357849 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ed0003e90 con 0x7f2ed8106380 2023-09-10T00:35:49.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.867+0000 7f2edf8ec700 1 -- 172.21.15.44:0/2168357849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed81a0690 con 0x7f2ed8106380 2023-09-10T00:35:49.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.867+0000 7f2ece7fc700 1 -- 172.21.15.44:0/2168357849 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0004020 con 0x7f2ed8106380 2023-09-10T00:35:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.867+0000 7f2edf8ec700 1 -- 172.21.15.44:0/2168357849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed81a0ab0 con 0x7f2ed8106380 2023-09-10T00:35:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.868+0000 7f2ec3fff700 1 -- 172.21.15.44:0/2168357849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb80052f0 con 0x7f2ed8106380 2023-09-10T00:35:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.869+0000 7f2ece7fc700 1 -- 172.21.15.44:0/2168357849 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f2ed001d690 con 0x7f2ed8106380 2023-09-10T00:35:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.869+0000 7f2ece7fc700 1 --2- 172.21.15.44:0/2168357849 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2ec40a0640 0x7f2ec40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:35:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.870+0000 7f2edd688700 1 --2- 172.21.15.44:0/2168357849 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2ec40a0640 0x7f2ec40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:35:49.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.870+0000 7f2ece7fc700 1 -- 172.21.15.44:0/2168357849 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f2ed00cc8d0 con 0x7f2ed8106380 2023-09-10T00:35:49.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.870+0000 7f2edd688700 1 --2- 172.21.15.44:0/2168357849 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2ec40a0640 0x7f2ec40a2b00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8005950 tx=0x7f2ec800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:35:49.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:49.874+0000 7f2ece7fc700 1 -- 172.21.15.44:0/2168357849 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2ed0097330 con 0x7f2ed8106380 2023-09-10T00:35:50.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:49 smithi131 ceph-mon[103713]: purged_snaps scrub starts 2023-09-10T00:35:50.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:49 smithi131 ceph-mon[103713]: purged_snaps scrub ok 2023-09-10T00:35:50.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:49 smithi131 ceph-mon[103713]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-10T00:35:50.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:49 smithi131 ceph-mon[103713]: Detected new or changed devices on smithi131 2023-09-10T00:35:50.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:49 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi131 to 8584M 2023-09-10T00:35:50.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:50.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:49 smithi131 ceph-mon[103713]: osd.5 [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489] boot 2023-09-10T00:35:50.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:49 smithi131 ceph-mon[103713]: osdmap e38: 6 total, 6 up, 6 in 2023-09-10T00:35:50.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:35:50.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:35:50.085+0000 7f2ec3fff700 1 -- 172.21.15.44:0/2168357849 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2eb8000bc0 con 0x7f2ec40a0640 2023-09-10T00:35:50.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:50 smithi099 ceph-mon[102692]: osdmap e39: 6 total, 6 up, 6 in 2023-09-10T00:35:50.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:50 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:35:50.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:50 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:35:50.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:50 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:50.892 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:50 smithi044 ceph-mon[98676]: osdmap e39: 6 total, 6 up, 6 in 2023-09-10T00:35:50.892 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:50 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:35:50.893 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:50 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:35:50.893 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:50 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:50 smithi131 ceph-mon[103713]: osdmap e39: 6 total, 6 up, 6 in 2023-09-10T00:35:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:50 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:35:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:50 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:35:51.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:50 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:51.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:51 smithi099 ceph-mon[102692]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-10T00:35:51.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:51 smithi099 ceph-mon[102692]: from='client.14493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:35:51.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:51 smithi099 ceph-mon[102692]: osdmap e40: 6 total, 6 up, 6 in 2023-09-10T00:35:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:51 smithi131 ceph-mon[103713]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-10T00:35:52.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:51 smithi131 ceph-mon[103713]: from='client.14493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:35:52.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:51 smithi131 ceph-mon[103713]: osdmap e40: 6 total, 6 up, 6 in 2023-09-10T00:35:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:51 smithi044 ceph-mon[98676]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-10T00:35:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:51 smithi044 ceph-mon[98676]: from='client.14493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:35:52.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:51 smithi044 ceph-mon[98676]: osdmap e40: 6 total, 6 up, 6 in 2023-09-10T00:35:52.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:52 smithi099 ceph-mon[102692]: osdmap e41: 6 total, 6 up, 6 in 2023-09-10T00:35:52.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:52 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2442237072' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b"}]: dispatch 2023-09-10T00:35:52.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:52 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2442237072' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b"}]': finished 2023-09-10T00:35:52.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:52 smithi099 ceph-mon[102692]: osdmap e42: 7 total, 6 up, 7 in 2023-09-10T00:35:52.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:52 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:35:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:52 smithi131 ceph-mon[103713]: osdmap e41: 6 total, 6 up, 6 in 2023-09-10T00:35:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:52 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2442237072' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b"}]: dispatch 2023-09-10T00:35:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:52 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2442237072' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b"}]': finished 2023-09-10T00:35:53.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:52 smithi131 ceph-mon[103713]: osdmap e42: 7 total, 6 up, 7 in 2023-09-10T00:35:53.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:52 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:35:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:52 smithi044 ceph-mon[98676]: osdmap e41: 6 total, 6 up, 6 in 2023-09-10T00:35:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:52 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2442237072' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b"}]: dispatch 2023-09-10T00:35:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:52 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2442237072' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b"}]': finished 2023-09-10T00:35:53.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:52 smithi044 ceph-mon[98676]: osdmap e42: 7 total, 6 up, 7 in 2023-09-10T00:35:53.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:52 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:35:53.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:53 smithi099 ceph-mon[102692]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-10T00:35:53.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:53 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1840693578' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:35:54.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:53 smithi131 ceph-mon[103713]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-10T00:35:54.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:53 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1840693578' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:35:54.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:53 smithi044 ceph-mon[98676]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-10T00:35:54.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:53 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1840693578' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:35:55.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:55 smithi099 ceph-mon[102692]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2023-09-10T00:35:56.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:55 smithi131 ceph-mon[103713]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2023-09-10T00:35:56.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:55 smithi044 ceph-mon[98676]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2023-09-10T00:35:57.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:57 smithi044 ceph-mon[98676]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-10T00:35:57.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:57 smithi099 ceph-mon[102692]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-10T00:35:58.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:57 smithi131 ceph-mon[103713]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-10T00:35:59.847 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:59 smithi044 ceph-mon[98676]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-09-10T00:35:59.848 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:59 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-10T00:35:59.848 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:35:59 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:59.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:59 smithi099 ceph-mon[102692]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-09-10T00:35:59.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:59 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-10T00:35:59.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:35:59 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:00.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:59 smithi131 ceph-mon[103713]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-09-10T00:36:00.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:59 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-10T00:36:00.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:35:59 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:00.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:00 smithi044 ceph-mon[98676]: Deploying daemon osd.6 on smithi044 2023-09-10T00:36:00.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:00 smithi099 ceph-mon[102692]: Deploying daemon osd.6 on smithi044 2023-09-10T00:36:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:00 smithi131 ceph-mon[103713]: Deploying daemon osd.6 on smithi044 2023-09-10T00:36:01.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:01 smithi044 ceph-mon[98676]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-09-10T00:36:01.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:01 smithi099 ceph-mon[102692]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-09-10T00:36:02.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:01 smithi131 ceph-mon[103713]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-09-10T00:36:03.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:03 smithi044 ceph-mon[98676]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-10T00:36:03.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:03 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:03.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:03 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:03.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:03 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:03.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:03 smithi099 ceph-mon[102692]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-10T00:36:03.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:03.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:03.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:04.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:03 smithi131 ceph-mon[103713]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-10T00:36:04.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:03 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:04.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:03 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:04.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:03 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:05.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:05 smithi099 ceph-mon[102692]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-09-10T00:36:06.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:05 smithi131 ceph-mon[103713]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-09-10T00:36:06.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:05 smithi044 ceph-mon[98676]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-09-10T00:36:06.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:06.488+0000 7f2ece7fc700 1 -- 172.21.15.44:0/2168357849 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2eb8000bc0 con 0x7f2ec40a0640 2023-09-10T00:36:06.489 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 6 on host 'smithi044' 2023-09-10T00:36:06.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:06.490+0000 7f2ec3fff700 1 -- 172.21.15.44:0/2168357849 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2ec40a0640 msgr2=0x7f2ec40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:36:06.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:06.490+0000 7f2ec3fff700 1 --2- 172.21.15.44:0/2168357849 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2ec40a0640 0x7f2ec40a2b00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8005950 tx=0x7f2ec800a300 comp rx=0 tx=0).stop 2023-09-10T00:36:06.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:06.490+0000 7f2ec3fff700 1 -- 172.21.15.44:0/2168357849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ed8106380 msgr2=0x7f2ed80ff610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:36:06.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:06.490+0000 7f2ec3fff700 1 --2- 172.21.15.44:0/2168357849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ed8106380 0x7f2ed80ff610 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000b070 tx=0x7f2ed0004d00 comp rx=0 tx=0).stop 2023-09-10T00:36:06.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:06.490+0000 7f2ec3fff700 1 -- 172.21.15.44:0/2168357849 shutdown_connections 2023-09-10T00:36:06.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:06.490+0000 7f2ec3fff700 1 --2- 172.21.15.44:0/2168357849 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2ec40a0640 0x7f2ec40a2b00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:06.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:06.490+0000 7f2ec3fff700 1 --2- 172.21.15.44:0/2168357849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ed8106380 0x7f2ed80ff610 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:06.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:06.490+0000 7f2ec3fff700 1 --2- 172.21.15.44:0/2168357849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed8101120 0x7f2ed8100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:06.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:06.490+0000 7f2ec3fff700 1 --2- 172.21.15.44:0/2168357849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8103a50 0x7f2ed80ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:06.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:06.490+0000 7f2ec3fff700 1 -- 172.21.15.44:0/2168357849 >> 172.21.15.44:0/2168357849 conn(0x7f2ed80fab10 msgr2=0x7f2ed8102570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:36:06.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:06.490+0000 7f2ec3fff700 1 -- 172.21.15.44:0/2168357849 shutdown_connections 2023-09-10T00:36:06.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:06.490+0000 7f2ec3fff700 1 -- 172.21.15.44:0/2168357849 wait complete. 2023-09-10T00:36:06.978 DEBUG:teuthology.orchestra.run.smithi044:osd.6> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.6.service 2023-09-10T00:36:06.980 INFO:tasks.cephadm:Deploying osd.7 on smithi099 with /dev/vg_nvme/lv_2... 2023-09-10T00:36:06.981 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-10T00:36:07.010 INFO:journalctl@ceph.osd.6.smithi044.stdout:-- Logs begin at Sun 2023-09-10 00:17:15 UTC. -- 2023-09-10T00:36:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:06 smithi131 ceph-mon[103713]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-10T00:36:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:07.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:07.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:07.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:07.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:07.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:06 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:07.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:06 smithi044 ceph-mon[98676]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-10T00:36:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:07.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:07.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:06 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:07.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:06 smithi099 ceph-mon[102692]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-10T00:36:07.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:07.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:07.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:07.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:07.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:07.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:07.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:07.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:06 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:07.556 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-10T00:36:07.760 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 00:36:07 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T00:36:07.422+0000 7fc9745a6640 -1 osd.6 0 log_to_monitors true 2023-09-10T00:36:08.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:07 smithi044 ceph-mon[98676]: from='osd.6 [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-10T00:36:08.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:07 smithi099 ceph-mon[102692]: from='osd.6 [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-10T00:36:08.294 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:07 smithi131 ceph-mon[103713]: from='osd.6 [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-10T00:36:09.165 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-10T00:36:09.165 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-10T00:36:09.165 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2023-09-10T00:36:09.165 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2023-09-10T00:36:09.166 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111396 s, 941 MB/s 2023-09-10T00:36:09.166 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2023-09-10T00:36:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:08 smithi131 ceph-mon[103713]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-10T00:36:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:08 smithi131 ceph-mon[103713]: from='osd.6 [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-10T00:36:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:08 smithi131 ceph-mon[103713]: osdmap e43: 7 total, 6 up, 7 in 2023-09-10T00:36:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:08 smithi131 ceph-mon[103713]: from='osd.6 [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:36:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:08 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:36:09.299 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:08 smithi099 ceph-mon[102692]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-10T00:36:09.300 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:08 smithi099 ceph-mon[102692]: from='osd.6 [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-10T00:36:09.300 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:08 smithi099 ceph-mon[102692]: osdmap e43: 7 total, 6 up, 7 in 2023-09-10T00:36:09.300 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:08 smithi099 ceph-mon[102692]: from='osd.6 [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:36:09.300 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:08 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:36:09.301 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:08 smithi044 ceph-mon[98676]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-10T00:36:09.302 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:08 smithi044 ceph-mon[98676]: from='osd.6 [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-10T00:36:09.302 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:08 smithi044 ceph-mon[98676]: osdmap e43: 7 total, 6 up, 7 in 2023-09-10T00:36:09.302 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:08 smithi044 ceph-mon[98676]: from='osd.6 [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:36:09.302 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:08 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:36:09.302 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 00:36:08 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T00:36:08.980+0000 7fc969c1d700 -1 osd.6 0 waiting for initial osdmap 2023-09-10T00:36:09.303 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 00:36:08 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T00:36:08.992+0000 7fc964a10700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-10T00:36:09.497 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi099:vg_nvme/lv_2 2023-09-10T00:36:10.053 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-10T00:36:10.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:09 smithi131 ceph-mon[103713]: from='osd.6 [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2023-09-10T00:36:10.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:09 smithi131 ceph-mon[103713]: osdmap e44: 7 total, 6 up, 7 in 2023-09-10T00:36:10.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:36:10.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:36:10.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:09 smithi131 ceph-mon[103713]: Detected new or changed devices on smithi044 2023-09-10T00:36:10.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:10.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:10.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:10.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:10.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:10.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:09 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi044 to 5724M 2023-09-10T00:36:10.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:10.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:10.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:10.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:10.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:09 smithi099 ceph-mon[102692]: from='osd.6 [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2023-09-10T00:36:10.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:09 smithi099 ceph-mon[102692]: osdmap e44: 7 total, 6 up, 7 in 2023-09-10T00:36:10.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:36:10.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:36:10.299 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:09 smithi099 ceph-mon[102692]: Detected new or changed devices on smithi044 2023-09-10T00:36:10.299 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:10.299 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:10.299 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:10.299 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:10.299 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:10.300 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:09 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi044 to 5724M 2023-09-10T00:36:10.300 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:10.300 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:10.300 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:10.300 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:09 smithi044 ceph-mon[98676]: from='osd.6 [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2023-09-10T00:36:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:09 smithi044 ceph-mon[98676]: osdmap e44: 7 total, 6 up, 7 in 2023-09-10T00:36:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:36:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:36:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:09 smithi044 ceph-mon[98676]: Detected new or changed devices on smithi044 2023-09-10T00:36:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:10.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:10.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:10.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:10.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:10.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:09 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi044 to 5724M 2023-09-10T00:36:10.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:10.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:10.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:10.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:10 smithi131 ceph-mon[103713]: purged_snaps scrub starts 2023-09-10T00:36:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:10 smithi131 ceph-mon[103713]: purged_snaps scrub ok 2023-09-10T00:36:11.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:10 smithi131 ceph-mon[103713]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-10T00:36:11.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:10 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:36:11.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:10 smithi131 ceph-mon[103713]: osd.6 [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767] boot 2023-09-10T00:36:11.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:10 smithi131 ceph-mon[103713]: osdmap e45: 7 total, 7 up, 7 in 2023-09-10T00:36:11.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:10 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:36:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:10 smithi044 ceph-mon[98676]: purged_snaps scrub starts 2023-09-10T00:36:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:10 smithi044 ceph-mon[98676]: purged_snaps scrub ok 2023-09-10T00:36:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:10 smithi044 ceph-mon[98676]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-10T00:36:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:10 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:36:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:10 smithi044 ceph-mon[98676]: osd.6 [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767] boot 2023-09-10T00:36:11.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:10 smithi044 ceph-mon[98676]: osdmap e45: 7 total, 7 up, 7 in 2023-09-10T00:36:11.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:10 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:36:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:10 smithi099 ceph-mon[102692]: purged_snaps scrub starts 2023-09-10T00:36:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:10 smithi099 ceph-mon[102692]: purged_snaps scrub ok 2023-09-10T00:36:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:10 smithi099 ceph-mon[102692]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-10T00:36:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:36:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:10 smithi099 ceph-mon[102692]: osd.6 [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767] boot 2023-09-10T00:36:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:10 smithi099 ceph-mon[102692]: osdmap e45: 7 total, 7 up, 7 in 2023-09-10T00:36:11.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:10 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:36:11.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.364+0000 7fab2d830700 1 -- 172.21.15.99:0/1864307927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab280ff5f0 msgr2=0x7fab280ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:36:11.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.365+0000 7fab2d830700 1 --2- 172.21.15.99:0/1864307927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab280ff5f0 0x7fab280ffa10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0099c0 tx=0x7fab1c009cd0 comp rx=0 tx=0).stop 2023-09-10T00:36:11.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.365+0000 7fab2d830700 1 -- 172.21.15.99:0/1864307927 shutdown_connections 2023-09-10T00:36:11.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.365+0000 7fab2d830700 1 --2- 172.21.15.99:0/1864307927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab28101190 0x7fab28107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:11.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.365+0000 7fab2d830700 1 --2- 172.21.15.99:0/1864307927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab280ff5f0 0x7fab280ffa10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:11.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.365+0000 7fab2d830700 1 --2- 172.21.15.99:0/1864307927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab281007f0 0x7fab28100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:11.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.365+0000 7fab2d830700 1 -- 172.21.15.99:0/1864307927 >> 172.21.15.99:0/1864307927 conn(0x7fab280fab90 msgr2=0x7fab280fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:36:11.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.365+0000 7fab2d830700 1 -- 172.21.15.99:0/1864307927 shutdown_connections 2023-09-10T00:36:11.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.365+0000 7fab2d830700 1 -- 172.21.15.99:0/1864307927 wait complete. 2023-09-10T00:36:11.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.365+0000 7fab2d830700 1 Processor -- start 2023-09-10T00:36:11.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.365+0000 7fab2d830700 1 -- start start 2023-09-10T00:36:11.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab2d830700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab280ff5f0 0x7fab28069580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:36:11.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab2d830700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab281007f0 0x7fab2819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:36:11.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab2d830700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab28101190 0x7fab2819fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:36:11.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab2d830700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab281a00d0 con 0x7fab280ff5f0 2023-09-10T00:36:11.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab2d830700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab281a0240 con 0x7fab281007f0 2023-09-10T00:36:11.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab2d830700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab281a0400 con 0x7fab28101190 2023-09-10T00:36:11.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab267fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab281007f0 0x7fab2819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:36:11.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab267fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab281007f0 0x7fab2819d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40780/0 (socket says 172.21.15.99:40780) 2023-09-10T00:36:11.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab267fc700 1 -- 172.21.15.99:0/2830292294 learned_addr learned my addr 172.21.15.99:0/2830292294 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-09-10T00:36:11.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab26ffd700 1 --2- 172.21.15.99:0/2830292294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab280ff5f0 0x7fab28069580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:36:11.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab277fe700 1 --2- 172.21.15.99:0/2830292294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab28101190 0x7fab2819fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:36:11.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab267fc700 1 -- 172.21.15.99:0/2830292294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab28101190 msgr2=0x7fab2819fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:36:11.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab267fc700 1 --2- 172.21.15.99:0/2830292294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab28101190 0x7fab2819fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:11.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab267fc700 1 -- 172.21.15.99:0/2830292294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab280ff5f0 msgr2=0x7fab28069580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:36:11.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab267fc700 1 --2- 172.21.15.99:0/2830292294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab280ff5f0 0x7fab28069580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:11.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab267fc700 1 -- 172.21.15.99:0/2830292294 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c0095f0 con 0x7fab281007f0 2023-09-10T00:36:11.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab277fe700 1 --2- 172.21.15.99:0/2830292294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab28101190 0x7fab2819fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:36:11.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab26ffd700 1 --2- 172.21.15.99:0/2830292294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab280ff5f0 0x7fab28069580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:36:11.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab267fc700 1 --2- 172.21.15.99:0/2830292294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab281007f0 0x7fab2819d4d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fab1800b530 tx=0x7fab1800b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:36:11.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab2c82e700 1 -- 172.21.15.99:0/2830292294 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab18011830 con 0x7fab281007f0 2023-09-10T00:36:11.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab2c82e700 1 -- 172.21.15.99:0/2830292294 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab18011e20 con 0x7fab281007f0 2023-09-10T00:36:11.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab2d830700 1 -- 172.21.15.99:0/2830292294 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab281a06e0 con 0x7fab281007f0 2023-09-10T00:36:11.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab2c82e700 1 -- 172.21.15.99:0/2830292294 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab18010dd0 con 0x7fab281007f0 2023-09-10T00:36:11.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.366+0000 7fab2d830700 1 -- 172.21.15.99:0/2830292294 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab281a0c60 con 0x7fab281007f0 2023-09-10T00:36:11.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.367+0000 7fab2d830700 1 -- 172.21.15.99:0/2830292294 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2810ada0 con 0x7fab281007f0 2023-09-10T00:36:11.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.368+0000 7fab2c82e700 1 -- 172.21.15.99:0/2830292294 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fab18011990 con 0x7fab281007f0 2023-09-10T00:36:11.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.369+0000 7fab2c82e700 1 --2- 172.21.15.99:0/2830292294 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fab100a0640 0x7fab100a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:36:11.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.369+0000 7fab2c82e700 1 -- 172.21.15.99:0/2830292294 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fab180cade0 con 0x7fab281007f0 2023-09-10T00:36:11.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.369+0000 7fab26ffd700 1 --2- 172.21.15.99:0/2830292294 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fab100a0640 0x7fab100a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:36:11.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.370+0000 7fab26ffd700 1 --2- 172.21.15.99:0/2830292294 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fab100a0640 0x7fab100a2b00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab1c000c00 tx=0x7fab1c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:36:11.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.373+0000 7fab2c82e700 1 -- 172.21.15.99:0/2830292294 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fab18095620 con 0x7fab281007f0 2023-09-10T00:36:11.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:11.571+0000 7fab2d830700 1 -- 172.21.15.99:0/2830292294 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fab28060a40 con 0x7fab100a0640 2023-09-10T00:36:12.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:12 smithi131 ceph-mon[103713]: osdmap e46: 7 total, 7 up, 7 in 2023-09-10T00:36:12.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:12 smithi131 ceph-mon[103713]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:36:12.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:12 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:36:12.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:12 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:36:12.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:12 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:12.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:12 smithi044 ceph-mon[98676]: osdmap e46: 7 total, 7 up, 7 in 2023-09-10T00:36:12.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:12 smithi044 ceph-mon[98676]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:36:12.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:12 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:36:12.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:12 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:36:12.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:12 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:12.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:12 smithi099 ceph-mon[102692]: osdmap e46: 7 total, 7 up, 7 in 2023-09-10T00:36:12.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:12 smithi099 ceph-mon[102692]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:36:12.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:12 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:36:12.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:12 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:36:12.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:12 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:13 smithi131 ceph-mon[103713]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:13.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:13 smithi044 ceph-mon[98676]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:13 smithi099 ceph-mon[102692]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:14.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:14 smithi131 ceph-mon[103713]: from='client.? 172.21.15.99:0/103639260' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4cfd6428-45b9-4884-973d-6f980996fe99"}]: dispatch 2023-09-10T00:36:14.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:14 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4cfd6428-45b9-4884-973d-6f980996fe99"}]: dispatch 2023-09-10T00:36:14.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:14 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4cfd6428-45b9-4884-973d-6f980996fe99"}]': finished 2023-09-10T00:36:14.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:14 smithi131 ceph-mon[103713]: osdmap e47: 8 total, 7 up, 8 in 2023-09-10T00:36:14.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:36:14.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:14 smithi044 ceph-mon[98676]: from='client.? 172.21.15.99:0/103639260' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4cfd6428-45b9-4884-973d-6f980996fe99"}]: dispatch 2023-09-10T00:36:14.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:14 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4cfd6428-45b9-4884-973d-6f980996fe99"}]: dispatch 2023-09-10T00:36:14.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:14 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4cfd6428-45b9-4884-973d-6f980996fe99"}]': finished 2023-09-10T00:36:14.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:14 smithi044 ceph-mon[98676]: osdmap e47: 8 total, 7 up, 8 in 2023-09-10T00:36:14.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:36:14.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:14 smithi099 ceph-mon[102692]: from='client.? 172.21.15.99:0/103639260' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4cfd6428-45b9-4884-973d-6f980996fe99"}]: dispatch 2023-09-10T00:36:14.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:14 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4cfd6428-45b9-4884-973d-6f980996fe99"}]: dispatch 2023-09-10T00:36:14.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:14 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4cfd6428-45b9-4884-973d-6f980996fe99"}]': finished 2023-09-10T00:36:14.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:14 smithi099 ceph-mon[102692]: osdmap e47: 8 total, 7 up, 8 in 2023-09-10T00:36:14.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:36:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:15 smithi131 ceph-mon[103713]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:15 smithi131 ceph-mon[103713]: from='client.? 172.21.15.99:0/1217077005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:36:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:15 smithi044 ceph-mon[98676]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:15 smithi044 ceph-mon[98676]: from='client.? 172.21.15.99:0/1217077005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:36:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:15 smithi099 ceph-mon[102692]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:15 smithi099 ceph-mon[102692]: from='client.? 172.21.15.99:0/1217077005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:36:17.694 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:17 smithi099 ceph-mon[102692]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:17.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:17 smithi131 ceph-mon[103713]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:17.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:17 smithi044 ceph-mon[98676]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:19.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:19 smithi131 ceph-mon[103713]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:19.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:19 smithi044 ceph-mon[98676]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:19.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:19 smithi099 ceph-mon[102692]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:21.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:21 smithi099 ceph-mon[102692]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:21.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-10T00:36:21.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:21.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:21 smithi099 ceph-mon[102692]: Deploying daemon osd.7 on smithi099 2023-09-10T00:36:21.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:21 smithi131 ceph-mon[103713]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:21.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-10T00:36:21.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:21.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:21 smithi131 ceph-mon[103713]: Deploying daemon osd.7 on smithi099 2023-09-10T00:36:21.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:21 smithi044 ceph-mon[98676]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:21.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-10T00:36:21.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:21.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:21 smithi044 ceph-mon[98676]: Deploying daemon osd.7 on smithi099 2023-09-10T00:36:23.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:23 smithi131 ceph-mon[103713]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:23.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:23 smithi044 ceph-mon[98676]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:23.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:23 smithi099 ceph-mon[102692]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:25.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:25 smithi099 ceph-mon[102692]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:25.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:25 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:25.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:25 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:25.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:25 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:25.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:25 smithi131 ceph-mon[103713]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:25.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:25 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:25.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:25 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:25.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:25 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:25.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:25 smithi044 ceph-mon[98676]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:25.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:25 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:25.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:25 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:25.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:25 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:27.557 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:27 smithi099 ceph-mon[102692]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:27.557 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:27 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:27.558 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:27 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:27.558 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:27 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:27.558 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:27 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:27.558 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:27 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:27.636 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 7 on host 'smithi099' 2023-09-10T00:36:27.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:27.635+0000 7fab2c82e700 1 -- 172.21.15.99:0/2830292294 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fab28060a40 con 0x7fab100a0640 2023-09-10T00:36:27.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:27.637+0000 7fab2d830700 1 -- 172.21.15.99:0/2830292294 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fab100a0640 msgr2=0x7fab100a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:36:27.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:27.637+0000 7fab2d830700 1 --2- 172.21.15.99:0/2830292294 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fab100a0640 0x7fab100a2b00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab1c000c00 tx=0x7fab1c005fb0 comp rx=0 tx=0).stop 2023-09-10T00:36:27.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:27.637+0000 7fab2d830700 1 -- 172.21.15.99:0/2830292294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab281007f0 msgr2=0x7fab2819d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:36:27.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:27.637+0000 7fab2d830700 1 --2- 172.21.15.99:0/2830292294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab281007f0 0x7fab2819d4d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fab1800b530 tx=0x7fab1800b8f0 comp rx=0 tx=0).stop 2023-09-10T00:36:27.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:27.637+0000 7fab2d830700 1 -- 172.21.15.99:0/2830292294 shutdown_connections 2023-09-10T00:36:27.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:27.637+0000 7fab2d830700 1 --2- 172.21.15.99:0/2830292294 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fab100a0640 0x7fab100a2b00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:27.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:27.637+0000 7fab2d830700 1 --2- 172.21.15.99:0/2830292294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab28101190 0x7fab2819fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:27.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:27.637+0000 7fab2d830700 1 --2- 172.21.15.99:0/2830292294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab280ff5f0 0x7fab28069580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:27.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:27.637+0000 7fab2d830700 1 --2- 172.21.15.99:0/2830292294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab281007f0 0x7fab2819d4d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:27.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:27.637+0000 7fab2d830700 1 -- 172.21.15.99:0/2830292294 >> 172.21.15.99:0/2830292294 conn(0x7fab280fab90 msgr2=0x7fab280fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:36:27.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:27.638+0000 7fab2d830700 1 -- 172.21.15.99:0/2830292294 shutdown_connections 2023-09-10T00:36:27.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:36:27.638+0000 7fab2d830700 1 -- 172.21.15.99:0/2830292294 wait complete. 2023-09-10T00:36:27.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:27 smithi131 ceph-mon[103713]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:27.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:27 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:27.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:27 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:27.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:27 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:27.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:27 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:27.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:27 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:27.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:27 smithi044 ceph-mon[98676]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:27.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:27.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:27.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:27.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:27.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:28.039 DEBUG:teuthology.orchestra.run.smithi099:osd.7> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.7.service 2023-09-10T00:36:28.041 INFO:tasks.cephadm:Deploying osd.8 on smithi131 with /dev/vg_nvme/lv_2... 2023-09-10T00:36:28.041 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-10T00:36:28.074 INFO:journalctl@ceph.osd.7.smithi099.stdout:-- Logs begin at Sun 2023-09-10 00:17:22 UTC. -- 2023-09-10T00:36:28.609 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-10T00:36:28.854 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 00:36:28 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T00:36:28.788+0000 7f5602056640 -1 osd.7 0 log_to_monitors true 2023-09-10T00:36:28.855 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:28 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:28.855 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:28 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:28.855 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:28 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:28.879 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:28 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:28.880 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:28 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:28.880 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:28 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:28 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:28 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:28 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:29 smithi131 ceph-mon[103713]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:29 smithi131 ceph-mon[103713]: from='osd.7 [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-10T00:36:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:29 smithi131 ceph-mon[103713]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-10T00:36:30.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:29 smithi044 ceph-mon[98676]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:30.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:29 smithi044 ceph-mon[98676]: from='osd.7 [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-10T00:36:30.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:29 smithi044 ceph-mon[98676]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-10T00:36:30.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:29 smithi099 ceph-mon[102692]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:30.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:29 smithi099 ceph-mon[102692]: from='osd.7 [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-10T00:36:30.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:29 smithi099 ceph-mon[102692]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-10T00:36:30.220 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-10T00:36:30.221 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-10T00:36:30.221 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2023-09-10T00:36:30.221 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2023-09-10T00:36:30.221 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.014989 s, 700 MB/s 2023-09-10T00:36:30.222 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2023-09-10T00:36:30.663 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi131:vg_nvme/lv_2 2023-09-10T00:36:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:30 smithi131 ceph-mon[103713]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-10T00:36:31.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:30 smithi131 ceph-mon[103713]: osdmap e48: 8 total, 7 up, 8 in 2023-09-10T00:36:31.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:30 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:36:31.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:30 smithi131 ceph-mon[103713]: from='osd.7 [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:36:31.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:30 smithi131 ceph-mon[103713]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:36:31.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:30 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:31.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:30 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:31.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:30 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:31.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:30 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:31.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:30 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:31.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:30 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:31.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:30 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:31.050 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:30 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:31.050 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:30 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:30 smithi044 ceph-mon[98676]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-10T00:36:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:30 smithi044 ceph-mon[98676]: osdmap e48: 8 total, 7 up, 8 in 2023-09-10T00:36:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:30 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:36:31.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:30 smithi044 ceph-mon[98676]: from='osd.7 [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:36:31.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:30 smithi044 ceph-mon[98676]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:36:31.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:30 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:31.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:30 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:31.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:30 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:31.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:30 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:31.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:30 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:31.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:30 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:31.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:30 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:31.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:30 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:31.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:30 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:31.106 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 00:36:30 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T00:36:30.633+0000 7f55f76cd700 -1 osd.7 0 waiting for initial osdmap 2023-09-10T00:36:31.107 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 00:36:30 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T00:36:30.641+0000 7f55f1cbf700 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-10T00:36:31.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:30 smithi099 ceph-mon[102692]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-10T00:36:31.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:30 smithi099 ceph-mon[102692]: osdmap e48: 8 total, 7 up, 8 in 2023-09-10T00:36:31.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:30 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:36:31.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:30 smithi099 ceph-mon[102692]: from='osd.7 [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:36:31.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:30 smithi099 ceph-mon[102692]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:36:31.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:30 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:31.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:30 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:31.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:30 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:31.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:30 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:31.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:30 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:31.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:30 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:31.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:30 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:31.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:30 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:31.113 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:30 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:31.220 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-10T00:36:31.749 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:31 smithi131 ceph-mon[103713]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:31.750 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:31 smithi131 ceph-mon[103713]: Detected new or changed devices on smithi099 2023-09-10T00:36:31.750 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:31 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi099 to 5722M 2023-09-10T00:36:31.751 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:31 smithi131 ceph-mon[103713]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-09-10T00:36:31.751 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:31 smithi131 ceph-mon[103713]: osdmap e49: 8 total, 7 up, 8 in 2023-09-10T00:36:31.751 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:31 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:36:31.751 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:31 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:36:32.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:31 smithi044 ceph-mon[98676]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:32.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:31 smithi044 ceph-mon[98676]: Detected new or changed devices on smithi099 2023-09-10T00:36:32.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:31 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi099 to 5722M 2023-09-10T00:36:32.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:31 smithi044 ceph-mon[98676]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-09-10T00:36:32.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:31 smithi044 ceph-mon[98676]: osdmap e49: 8 total, 7 up, 8 in 2023-09-10T00:36:32.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:31 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:36:32.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:31 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:36:32.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:31 smithi099 ceph-mon[102692]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-10T00:36:32.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:31 smithi099 ceph-mon[102692]: Detected new or changed devices on smithi099 2023-09-10T00:36:32.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:31 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi099 to 5722M 2023-09-10T00:36:32.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:31 smithi099 ceph-mon[102692]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-09-10T00:36:32.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:31 smithi099 ceph-mon[102692]: osdmap e49: 8 total, 7 up, 8 in 2023-09-10T00:36:32.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:31 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:36:32.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:31 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:36:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.574+0000 7fa966bf5700 1 -- 172.21.15.131:0/4290017934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa960101150 msgr2=0x7fa960103540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:36:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.574+0000 7fa966bf5700 1 --2- 172.21.15.131:0/4290017934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa960101150 0x7fa960103540 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa9540098f0 tx=0x7fa954009c00 comp rx=0 tx=0).stop 2023-09-10T00:36:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.574+0000 7fa966bf5700 1 -- 172.21.15.131:0/4290017934 shutdown_connections 2023-09-10T00:36:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.574+0000 7fa966bf5700 1 --2- 172.21.15.131:0/4290017934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9601063b0 0x7fa960108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.574+0000 7fa966bf5700 1 --2- 172.21.15.131:0/4290017934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa960101150 0x7fa960103540 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.574+0000 7fa966bf5700 1 --2- 172.21.15.131:0/4290017934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa960103a80 0x7fa960105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.574+0000 7fa966bf5700 1 -- 172.21.15.131:0/4290017934 >> 172.21.15.131:0/4290017934 conn(0x7fa9600fab80 msgr2=0x7fa9600fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:36:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.574+0000 7fa966bf5700 1 -- 172.21.15.131:0/4290017934 shutdown_connections 2023-09-10T00:36:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.574+0000 7fa966bf5700 1 -- 172.21.15.131:0/4290017934 wait complete. 2023-09-10T00:36:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.575+0000 7fa966bf5700 1 Processor -- start 2023-09-10T00:36:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.575+0000 7fa966bf5700 1 -- start start 2023-09-10T00:36:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.575+0000 7fa966bf5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa960101150 0x7fa960196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:36:32.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.575+0000 7fa966bf5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa960103a80 0x7fa960196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:36:32.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.575+0000 7fa966bf5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9601063b0 0x7fa96019d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:36:32.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.575+0000 7fa966bf5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96019de90 con 0x7fa9601063b0 2023-09-10T00:36:32.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.575+0000 7fa966bf5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96019dfd0 con 0x7fa960103a80 2023-09-10T00:36:32.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.575+0000 7fa966bf5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96019e160 con 0x7fa960101150 2023-09-10T00:36:32.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.575+0000 7fa964991700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa960101150 0x7fa960196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:36:32.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.575+0000 7fa964991700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa960101150 0x7fa960196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53086/0 (socket says 172.21.15.131:53086) 2023-09-10T00:36:32.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.575+0000 7fa964991700 1 -- 172.21.15.131:0/4194687410 learned_addr learned my addr 172.21.15.131:0/4194687410 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-10T00:36:32.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.575+0000 7fa95ffff700 1 --2- 172.21.15.131:0/4194687410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa960103a80 0x7fa960196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:36:32.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.575+0000 7fa965192700 1 --2- 172.21.15.131:0/4194687410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9601063b0 0x7fa96019d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:36:32.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.575+0000 7fa964991700 1 -- 172.21.15.131:0/4194687410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa960103a80 msgr2=0x7fa960196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:36:32.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.576+0000 7fa964991700 1 --2- 172.21.15.131:0/4194687410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa960103a80 0x7fa960196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:32.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.576+0000 7fa964991700 1 -- 172.21.15.131:0/4194687410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9601063b0 msgr2=0x7fa96019d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:36:32.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.576+0000 7fa964991700 1 --2- 172.21.15.131:0/4194687410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9601063b0 0x7fa96019d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:32.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.576+0000 7fa964991700 1 -- 172.21.15.131:0/4194687410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa958009d00 con 0x7fa960101150 2023-09-10T00:36:32.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.576+0000 7fa964991700 1 --2- 172.21.15.131:0/4194687410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa960101150 0x7fa960196300 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa95400b7c0 tx=0x7fa95400b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:36:32.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.576+0000 7fa95dffb700 1 -- 172.21.15.131:0/4194687410 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95401c070 con 0x7fa960101150 2023-09-10T00:36:32.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.576+0000 7fa966bf5700 1 -- 172.21.15.131:0/4194687410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa954009520 con 0x7fa960101150 2023-09-10T00:36:32.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.576+0000 7fa95dffb700 1 -- 172.21.15.131:0/4194687410 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa95400f7d0 con 0x7fa960101150 2023-09-10T00:36:32.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.576+0000 7fa95dffb700 1 -- 172.21.15.131:0/4194687410 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954004e50 con 0x7fa960101150 2023-09-10T00:36:32.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.576+0000 7fa966bf5700 1 -- 172.21.15.131:0/4194687410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96019e660 con 0x7fa960101150 2023-09-10T00:36:32.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.577+0000 7fa966bf5700 1 -- 172.21.15.131:0/4194687410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96010bda0 con 0x7fa960101150 2023-09-10T00:36:32.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.578+0000 7fa95dffb700 1 -- 172.21.15.131:0/4194687410 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fa954005040 con 0x7fa960101150 2023-09-10T00:36:32.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.579+0000 7fa95dffb700 1 --2- 172.21.15.131:0/4194687410 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa9500a0640 0x7fa9500a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:36:32.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.579+0000 7fa95dffb700 1 -- 172.21.15.131:0/4194687410 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5430+0+0 (secure 0 0 0) 0x7fa9540cc590 con 0x7fa960101150 2023-09-10T00:36:32.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.579+0000 7fa95ffff700 1 --2- 172.21.15.131:0/4194687410 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa9500a0640 0x7fa9500a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:36:32.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.580+0000 7fa95ffff700 1 --2- 172.21.15.131:0/4194687410 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa9500a0640 0x7fa9500a2b00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00bbd0 tx=0x7fa94c00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:36:32.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.584+0000 7fa95dffb700 1 -- 172.21.15.131:0/4194687410 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa954096c90 con 0x7fa960101150 2023-09-10T00:36:32.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:32.788+0000 7fa966bf5700 1 -- 172.21.15.131:0/4194687410 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa960102ce0 con 0x7fa9500a0640 2023-09-10T00:36:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:32 smithi131 ceph-mon[103713]: purged_snaps scrub starts 2023-09-10T00:36:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:32 smithi131 ceph-mon[103713]: purged_snaps scrub ok 2023-09-10T00:36:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:32 smithi131 ceph-mon[103713]: osd.7 [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338] boot 2023-09-10T00:36:33.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:32 smithi131 ceph-mon[103713]: osdmap e50: 8 total, 8 up, 8 in 2023-09-10T00:36:33.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:32 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:36:33.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:32 smithi044 ceph-mon[98676]: purged_snaps scrub starts 2023-09-10T00:36:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:32 smithi044 ceph-mon[98676]: purged_snaps scrub ok 2023-09-10T00:36:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:32 smithi044 ceph-mon[98676]: osd.7 [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338] boot 2023-09-10T00:36:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:32 smithi044 ceph-mon[98676]: osdmap e50: 8 total, 8 up, 8 in 2023-09-10T00:36:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:32 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:36:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:32 smithi099 ceph-mon[102692]: purged_snaps scrub starts 2023-09-10T00:36:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:32 smithi099 ceph-mon[102692]: purged_snaps scrub ok 2023-09-10T00:36:33.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:32 smithi099 ceph-mon[102692]: osd.7 [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338] boot 2023-09-10T00:36:33.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:32 smithi099 ceph-mon[102692]: osdmap e50: 8 total, 8 up, 8 in 2023-09-10T00:36:33.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:32 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:36:33.974 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:33 smithi131 ceph-mon[103713]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:33.974 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:33 smithi131 ceph-mon[103713]: osdmap e51: 8 total, 8 up, 8 in 2023-09-10T00:36:33.975 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:33 smithi131 ceph-mon[103713]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:36:33.975 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:33 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:36:33.975 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:33 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:36:33.975 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:33 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:34.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:33 smithi044 ceph-mon[98676]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:34.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:33 smithi044 ceph-mon[98676]: osdmap e51: 8 total, 8 up, 8 in 2023-09-10T00:36:34.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:33 smithi044 ceph-mon[98676]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:36:34.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:33 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:36:34.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:33 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:36:34.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:33 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:34.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:33 smithi099 ceph-mon[102692]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:34.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:33 smithi099 ceph-mon[102692]: osdmap e51: 8 total, 8 up, 8 in 2023-09-10T00:36:34.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:33 smithi099 ceph-mon[102692]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:36:34.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:33 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:36:34.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:33 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:36:34.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:33 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:34.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:34 smithi131 ceph-mon[103713]: osdmap e52: 8 total, 8 up, 8 in 2023-09-10T00:36:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:34 smithi044 ceph-mon[98676]: osdmap e52: 8 total, 8 up, 8 in 2023-09-10T00:36:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:34 smithi099 ceph-mon[102692]: osdmap e52: 8 total, 8 up, 8 in 2023-09-10T00:36:36.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:35 smithi131 ceph-mon[103713]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:36.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:35 smithi131 ceph-mon[103713]: osdmap e53: 8 total, 8 up, 8 in 2023-09-10T00:36:36.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:35 smithi131 ceph-mon[103713]: from='client.? 172.21.15.131:0/3127082367' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b"}]: dispatch 2023-09-10T00:36:36.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:35 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b"}]: dispatch 2023-09-10T00:36:36.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:35 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b"}]': finished 2023-09-10T00:36:36.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:35 smithi131 ceph-mon[103713]: osdmap e54: 9 total, 8 up, 9 in 2023-09-10T00:36:36.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:35 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:36.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:35 smithi044 ceph-mon[98676]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:36.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:35 smithi044 ceph-mon[98676]: osdmap e53: 8 total, 8 up, 8 in 2023-09-10T00:36:36.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:35 smithi044 ceph-mon[98676]: from='client.? 172.21.15.131:0/3127082367' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b"}]: dispatch 2023-09-10T00:36:36.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:35 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b"}]: dispatch 2023-09-10T00:36:36.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:35 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b"}]': finished 2023-09-10T00:36:36.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:35 smithi044 ceph-mon[98676]: osdmap e54: 9 total, 8 up, 9 in 2023-09-10T00:36:36.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:35 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:36.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:35 smithi099 ceph-mon[102692]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:36.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:35 smithi099 ceph-mon[102692]: osdmap e53: 8 total, 8 up, 8 in 2023-09-10T00:36:36.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:35 smithi099 ceph-mon[102692]: from='client.? 172.21.15.131:0/3127082367' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b"}]: dispatch 2023-09-10T00:36:36.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:35 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b"}]: dispatch 2023-09-10T00:36:36.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:35 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b"}]': finished 2023-09-10T00:36:36.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:35 smithi099 ceph-mon[102692]: osdmap e54: 9 total, 8 up, 9 in 2023-09-10T00:36:36.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:35 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:36 smithi131 ceph-mon[103713]: from='client.? 172.21.15.131:0/1126686085' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:36:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:36 smithi044 ceph-mon[98676]: from='client.? 172.21.15.131:0/1126686085' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:36:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:36 smithi099 ceph-mon[102692]: from='client.? 172.21.15.131:0/1126686085' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:36:38.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:37 smithi131 ceph-mon[103713]: pgmap v174: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:37 smithi044 ceph-mon[98676]: pgmap v174: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:38.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:37 smithi099 ceph-mon[102692]: pgmap v174: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:40.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:39 smithi131 ceph-mon[103713]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:39 smithi044 ceph-mon[98676]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:40.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:39 smithi099 ceph-mon[102692]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:41.794 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:41 smithi131 ceph-mon[103713]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:42.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:41 smithi044 ceph-mon[98676]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:42.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:41 smithi099 ceph-mon[102692]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:42.756 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:42 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-10T00:36:42.756 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:42 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:42.757 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:42 smithi131 ceph-mon[103713]: Deploying daemon osd.8 on smithi131 2023-09-10T00:36:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:42 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-10T00:36:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:42 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:42 smithi044 ceph-mon[98676]: Deploying daemon osd.8 on smithi131 2023-09-10T00:36:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:42 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-10T00:36:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:42 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:42 smithi099 ceph-mon[102692]: Deploying daemon osd.8 on smithi131 2023-09-10T00:36:44.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:43 smithi131 ceph-mon[103713]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:44.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:43 smithi044 ceph-mon[98676]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:44.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:43 smithi099 ceph-mon[102692]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:45.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:45 smithi131 ceph-mon[103713]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:46.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:45 smithi044 ceph-mon[98676]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:46.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:45 smithi099 ceph-mon[102692]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:47.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:47 smithi131 ceph-mon[103713]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:47.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:47 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:47.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:47 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:47.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:47 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:47.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:47 smithi044 ceph-mon[98676]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:47.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:47 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:47.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:47 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:47.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:47 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:47.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:47 smithi099 ceph-mon[102692]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:47.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:47 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:47.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:47 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:47.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:47 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:49.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:49.253+0000 7fa95dffb700 1 -- 172.21.15.131:0/4194687410 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa960102ce0 con 0x7fa9500a0640 2023-09-10T00:36:49.255 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 8 on host 'smithi131' 2023-09-10T00:36:49.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:49.255+0000 7fa966bf5700 1 -- 172.21.15.131:0/4194687410 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa9500a0640 msgr2=0x7fa9500a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:36:49.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:49.255+0000 7fa966bf5700 1 --2- 172.21.15.131:0/4194687410 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa9500a0640 0x7fa9500a2b00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00bbd0 tx=0x7fa94c00b5b0 comp rx=0 tx=0).stop 2023-09-10T00:36:49.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:49.255+0000 7fa966bf5700 1 -- 172.21.15.131:0/4194687410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa960101150 msgr2=0x7fa960196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:36:49.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:49.255+0000 7fa966bf5700 1 --2- 172.21.15.131:0/4194687410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa960101150 0x7fa960196300 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa95400b7c0 tx=0x7fa95400b8a0 comp rx=0 tx=0).stop 2023-09-10T00:36:49.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:49.256+0000 7fa966bf5700 1 -- 172.21.15.131:0/4194687410 shutdown_connections 2023-09-10T00:36:49.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:49.256+0000 7fa966bf5700 1 --2- 172.21.15.131:0/4194687410 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa9500a0640 0x7fa9500a2b00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:49.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:49.256+0000 7fa966bf5700 1 --2- 172.21.15.131:0/4194687410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9601063b0 0x7fa96019d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:49.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:49.256+0000 7fa966bf5700 1 --2- 172.21.15.131:0/4194687410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa960101150 0x7fa960196300 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:49.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:49.256+0000 7fa966bf5700 1 --2- 172.21.15.131:0/4194687410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa960103a80 0x7fa960196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:49.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:49.256+0000 7fa966bf5700 1 -- 172.21.15.131:0/4194687410 >> 172.21.15.131:0/4194687410 conn(0x7fa9600fab80 msgr2=0x7fa9601025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:36:49.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:49.256+0000 7fa966bf5700 1 -- 172.21.15.131:0/4194687410 shutdown_connections 2023-09-10T00:36:49.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:36:49.256+0000 7fa966bf5700 1 -- 172.21.15.131:0/4194687410 wait complete. 2023-09-10T00:36:49.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:49 smithi131 ceph-mon[103713]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:49.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:49.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:49.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:49.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:49.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:49.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:49.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:49.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:49 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:49.715 DEBUG:teuthology.orchestra.run.smithi131:osd.8> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.8.service 2023-09-10T00:36:49.717 INFO:tasks.cephadm:Deploying osd.9 on smithi044 with /dev/vg_nvme/lv_1... 2023-09-10T00:36:49.718 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-10T00:36:49.746 INFO:journalctl@ceph.osd.8.smithi131.stdout:-- Logs begin at Sun 2023-09-10 00:17:05 UTC. -- 2023-09-10T00:36:49.751 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:49 smithi044 ceph-mon[98676]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:49.751 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:49.751 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:49.752 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:49.752 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:49.752 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:49.752 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:49.752 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:49.753 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:49 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:49.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:49 smithi099 ceph-mon[102692]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:49.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:49.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:49.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:49.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:49.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:49.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:49.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:49.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:49 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:50.293 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:36:50.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:50 smithi131 ceph-mon[103713]: from='osd.8 [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-10T00:36:50.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:50 smithi131 ceph-mon[103713]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-10T00:36:50.545 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 00:36:50 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T00:36:50.377+0000 7ff3a5751640 -1 osd.8 0 log_to_monitors true 2023-09-10T00:36:50.722 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:50 smithi044 ceph-mon[98676]: from='osd.8 [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-10T00:36:50.723 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:50 smithi044 ceph-mon[98676]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-10T00:36:50.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:50 smithi099 ceph-mon[102692]: from='osd.8 [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-10T00:36:50.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:50 smithi099 ceph-mon[102692]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-10T00:36:51.419 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:51 smithi131 ceph-mon[103713]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:51.420 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:51 smithi131 ceph-mon[103713]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-10T00:36:51.420 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:51 smithi131 ceph-mon[103713]: osdmap e55: 9 total, 8 up, 9 in 2023-09-10T00:36:51.420 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:51 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:51.420 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:51 smithi131 ceph-mon[103713]: from='osd.8 [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:36:51.420 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:51 smithi131 ceph-mon[103713]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:36:51.421 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 00:36:51 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T00:36:51.409+0000 7ff39adc8700 -1 osd.8 0 waiting for initial osdmap 2023-09-10T00:36:51.795 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 00:36:51 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T00:36:51.418+0000 7ff392bb5700 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-10T00:36:51.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:51 smithi044 ceph-mon[98676]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:51.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:51 smithi044 ceph-mon[98676]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-10T00:36:51.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:51 smithi044 ceph-mon[98676]: osdmap e55: 9 total, 8 up, 9 in 2023-09-10T00:36:51.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:51 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:51.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:51 smithi044 ceph-mon[98676]: from='osd.8 [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:36:51.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:51 smithi044 ceph-mon[98676]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:36:51.817 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-10T00:36:51.817 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-10T00:36:51.817 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2023-09-10T00:36:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2023-09-10T00:36:51.818 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130007 s, 807 MB/s 2023-09-10T00:36:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2023-09-10T00:36:51.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:51 smithi099 ceph-mon[102692]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:51.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:51 smithi099 ceph-mon[102692]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-10T00:36:51.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:51 smithi099 ceph-mon[102692]: osdmap e55: 9 total, 8 up, 9 in 2023-09-10T00:36:51.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:51 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:51.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:51 smithi099 ceph-mon[102692]: from='osd.8 [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:36:51.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:51 smithi099 ceph-mon[102692]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:36:52.157 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi044:vg_nvme/lv_1 2023-09-10T00:36:52.766 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:36:52.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:52 smithi131 ceph-mon[103713]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2023-09-10T00:36:52.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:52 smithi131 ceph-mon[103713]: osdmap e56: 9 total, 8 up, 9 in 2023-09-10T00:36:52.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:52 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:52.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:52 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:52.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:52 smithi044 ceph-mon[98676]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2023-09-10T00:36:52.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:52 smithi044 ceph-mon[98676]: osdmap e56: 9 total, 8 up, 9 in 2023-09-10T00:36:52.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:52 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:52.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:52 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:52.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:52 smithi099 ceph-mon[102692]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2023-09-10T00:36:52.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:52 smithi099 ceph-mon[102692]: osdmap e56: 9 total, 8 up, 9 in 2023-09-10T00:36:52.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:52 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:52.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:52 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:53.542 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: purged_snaps scrub starts 2023-09-10T00:36:53.542 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: purged_snaps scrub ok 2023-09-10T00:36:53.543 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:53.543 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:53.543 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: osd.8 [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267] boot 2023-09-10T00:36:53.543 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: osdmap e57: 9 total, 9 up, 9 in 2023-09-10T00:36:53.543 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:53.543 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: Detected new or changed devices on smithi131 2023-09-10T00:36:53.544 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:53.544 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:53.544 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:53.544 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:53.544 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:53.545 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi131 to 5722M 2023-09-10T00:36:53.545 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:53.545 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:53.545 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:53.545 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:53.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: purged_snaps scrub starts 2023-09-10T00:36:53.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: purged_snaps scrub ok 2023-09-10T00:36:53.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:53.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:53.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: osd.8 [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267] boot 2023-09-10T00:36:53.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: osdmap e57: 9 total, 9 up, 9 in 2023-09-10T00:36:53.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:53.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: Detected new or changed devices on smithi131 2023-09-10T00:36:53.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:53.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:53.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:53.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:53.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:53.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi131 to 5722M 2023-09-10T00:36:53.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:53.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:53.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:53.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:53.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: purged_snaps scrub starts 2023-09-10T00:36:53.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: purged_snaps scrub ok 2023-09-10T00:36:53.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-10T00:36:53.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:53.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: osd.8 [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267] boot 2023-09-10T00:36:53.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: osdmap e57: 9 total, 9 up, 9 in 2023-09-10T00:36:53.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:36:53.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: Detected new or changed devices on smithi131 2023-09-10T00:36:53.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:53.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:53.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:53.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:53.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:53.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi131 to 5722M 2023-09-10T00:36:53.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:53.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:53.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:53.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:36:53.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.965+0000 7f2940ca4700 1 -- 172.21.15.44:0/103979847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f293c0ff5c0 msgr2=0x7f293c0ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:36:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.965+0000 7f2940ca4700 1 --2- 172.21.15.44:0/103979847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f293c0ff5c0 0x7f293c0ff9e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f292c0099c0 tx=0x7f292c009cd0 comp rx=0 tx=0).stop 2023-09-10T00:36:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.965+0000 7f2940ca4700 1 -- 172.21.15.44:0/103979847 shutdown_connections 2023-09-10T00:36:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.965+0000 7f2940ca4700 1 --2- 172.21.15.44:0/103979847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f293c101160 0x7f293c107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.965+0000 7f2940ca4700 1 --2- 172.21.15.44:0/103979847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f293c0ff5c0 0x7f293c0ff9e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.965+0000 7f2940ca4700 1 --2- 172.21.15.44:0/103979847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f293c1007c0 0x7f293c100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:53.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.965+0000 7f2940ca4700 1 -- 172.21.15.44:0/103979847 >> 172.21.15.44:0/103979847 conn(0x7f293c0fab80 msgr2=0x7f293c0fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:36:53.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.965+0000 7f2940ca4700 1 -- 172.21.15.44:0/103979847 shutdown_connections 2023-09-10T00:36:53.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.965+0000 7f2940ca4700 1 -- 172.21.15.44:0/103979847 wait complete. 2023-09-10T00:36:53.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f2940ca4700 1 Processor -- start 2023-09-10T00:36:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f2940ca4700 1 -- start start 2023-09-10T00:36:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f2940ca4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f293c0ff5c0 0x7f293c069560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:36:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f2940ca4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f293c1007c0 0x7f293c19aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:36:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f2940ca4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f293c101160 0x7f293c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:36:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f2940ca4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c19dde0 con 0x7f293c0ff5c0 2023-09-10T00:36:53.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f2940ca4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c19df50 con 0x7f293c101160 2023-09-10T00:36:53.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f2940ca4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c19e110 con 0x7f293c1007c0 2023-09-10T00:36:53.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f293a59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f293c0ff5c0 0x7f293c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:36:53.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f293a59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f293c0ff5c0 0x7f293c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45730/0 (socket says 172.21.15.44:45730) 2023-09-10T00:36:53.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f293a59c700 1 -- 172.21.15.44:0/463321213 learned_addr learned my addr 172.21.15.44:0/463321213 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:36:53.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f293ad9d700 1 --2- 172.21.15.44:0/463321213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f293c101160 0x7f293c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:36:53.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f2939d9b700 1 --2- 172.21.15.44:0/463321213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f293c1007c0 0x7f293c19aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:36:53.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f293a59c700 1 -- 172.21.15.44:0/463321213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f293c1007c0 msgr2=0x7f293c19aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:36:53.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f293a59c700 1 --2- 172.21.15.44:0/463321213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f293c1007c0 0x7f293c19aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:53.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f293a59c700 1 -- 172.21.15.44:0/463321213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f293c101160 msgr2=0x7f293c19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:36:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f293a59c700 1 --2- 172.21.15.44:0/463321213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f293c101160 0x7f293c19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:36:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.966+0000 7f293a59c700 1 -- 172.21.15.44:0/463321213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292c0095f0 con 0x7f293c0ff5c0 2023-09-10T00:36:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.967+0000 7f293a59c700 1 --2- 172.21.15.44:0/463321213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f293c0ff5c0 0x7f293c069560 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f292c009f50 tx=0x7f292c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:36:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.967+0000 7f292b7fe700 1 -- 172.21.15.44:0/463321213 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c01d070 con 0x7f293c0ff5c0 2023-09-10T00:36:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.967+0000 7f2940ca4700 1 -- 172.21.15.44:0/463321213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293c194e80 con 0x7f293c0ff5c0 2023-09-10T00:36:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.967+0000 7f292b7fe700 1 -- 172.21.15.44:0/463321213 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f292c00bda0 con 0x7f293c0ff5c0 2023-09-10T00:36:53.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.967+0000 7f292b7fe700 1 -- 172.21.15.44:0/463321213 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c022de0 con 0x7f293c0ff5c0 2023-09-10T00:36:53.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.967+0000 7f2940ca4700 1 -- 172.21.15.44:0/463321213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293c195320 con 0x7f293c0ff5c0 2023-09-10T00:36:53.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.968+0000 7f2940ca4700 1 -- 172.21.15.44:0/463321213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293c1065b0 con 0x7f293c0ff5c0 2023-09-10T00:36:53.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.969+0000 7f292b7fe700 1 -- 172.21.15.44:0/463321213 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f292c0225e0 con 0x7f293c0ff5c0 2023-09-10T00:36:53.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.969+0000 7f292b7fe700 1 --2- 172.21.15.44:0/463321213 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f29240a0640 0x7f29240a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:36:53.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.969+0000 7f2939d9b700 1 --2- 172.21.15.44:0/463321213 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f29240a0640 0x7f29240a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:36:53.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.969+0000 7f292b7fe700 1 -- 172.21.15.44:0/463321213 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f292c0cc6a0 con 0x7f293c0ff5c0 2023-09-10T00:36:53.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.970+0000 7f2939d9b700 1 --2- 172.21.15.44:0/463321213 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f29240a0640 0x7f29240a2b00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2930005fd0 tx=0x7f293000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:36:53.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:53.974+0000 7f292b7fe700 1 -- 172.21.15.44:0/463321213 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f292c096d10 con 0x7f293c0ff5c0 2023-09-10T00:36:54.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:36:54.185+0000 7f2940ca4700 1 -- 172.21.15.44:0/463321213 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f293c02cff0 con 0x7f29240a0640 2023-09-10T00:36:54.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:54 smithi131 ceph-mon[103713]: osdmap e58: 9 total, 9 up, 9 in 2023-09-10T00:36:54.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:54 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:36:54.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:54 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:36:54.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:54 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:54.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:54 smithi044 ceph-mon[98676]: osdmap e58: 9 total, 9 up, 9 in 2023-09-10T00:36:54.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:54 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:36:54.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:54 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:36:54.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:54 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:54.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:54 smithi099 ceph-mon[102692]: osdmap e58: 9 total, 9 up, 9 in 2023-09-10T00:36:54.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:54 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:36:54.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:54 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:36:54.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:54 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:55.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:55 smithi044 ceph-mon[98676]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-10T00:36:55.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:55 smithi044 ceph-mon[98676]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:36:55.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:55 smithi044 ceph-mon[98676]: osdmap e59: 9 total, 9 up, 9 in 2023-09-10T00:36:55.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:55 smithi131 ceph-mon[103713]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-10T00:36:55.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:55 smithi131 ceph-mon[103713]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:36:55.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:55 smithi131 ceph-mon[103713]: osdmap e59: 9 total, 9 up, 9 in 2023-09-10T00:36:55.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:55 smithi099 ceph-mon[102692]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-10T00:36:55.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:55 smithi099 ceph-mon[102692]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:36:55.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:55 smithi099 ceph-mon[102692]: osdmap e59: 9 total, 9 up, 9 in 2023-09-10T00:36:56.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:56 smithi131 ceph-mon[103713]: osdmap e60: 9 total, 9 up, 9 in 2023-09-10T00:36:56.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:56 smithi131 ceph-mon[103713]: osdmap e61: 9 total, 9 up, 9 in 2023-09-10T00:36:56.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:56 smithi044 ceph-mon[98676]: osdmap e60: 9 total, 9 up, 9 in 2023-09-10T00:36:56.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:56 smithi044 ceph-mon[98676]: osdmap e61: 9 total, 9 up, 9 in 2023-09-10T00:36:56.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:56 smithi099 ceph-mon[102692]: osdmap e60: 9 total, 9 up, 9 in 2023-09-10T00:36:56.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:56 smithi099 ceph-mon[102692]: osdmap e61: 9 total, 9 up, 9 in 2023-09-10T00:36:57.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:57 smithi131 ceph-mon[103713]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-10T00:36:57.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:57 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3962132440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa68741f-6ffa-4525-bfce-4b29c806bc53"}]: dispatch 2023-09-10T00:36:57.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:57 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3962132440' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa68741f-6ffa-4525-bfce-4b29c806bc53"}]': finished 2023-09-10T00:36:57.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:57 smithi131 ceph-mon[103713]: osdmap e62: 10 total, 9 up, 10 in 2023-09-10T00:36:57.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:57 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:36:57.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:57 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/144561445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:36:57.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:57 smithi044 ceph-mon[98676]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-10T00:36:57.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:57 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3962132440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa68741f-6ffa-4525-bfce-4b29c806bc53"}]: dispatch 2023-09-10T00:36:57.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:57 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3962132440' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa68741f-6ffa-4525-bfce-4b29c806bc53"}]': finished 2023-09-10T00:36:57.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:57 smithi044 ceph-mon[98676]: osdmap e62: 10 total, 9 up, 10 in 2023-09-10T00:36:57.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:57 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:36:57.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:57 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/144561445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:36:57.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:57 smithi099 ceph-mon[102692]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-10T00:36:57.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:57 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3962132440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa68741f-6ffa-4525-bfce-4b29c806bc53"}]: dispatch 2023-09-10T00:36:57.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:57 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3962132440' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa68741f-6ffa-4525-bfce-4b29c806bc53"}]': finished 2023-09-10T00:36:57.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:57 smithi099 ceph-mon[102692]: osdmap e62: 10 total, 9 up, 10 in 2023-09-10T00:36:57.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:57 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:36:57.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:57 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/144561445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:36:59.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:36:59 smithi131 ceph-mon[103713]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-10T00:36:59.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:36:59 smithi044 ceph-mon[98676]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-10T00:36:59.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:36:59 smithi099 ceph-mon[102692]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-10T00:37:01.782 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:01 smithi044 ceph-mon[98676]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-09-10T00:37:01.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:01 smithi131 ceph-mon[103713]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-09-10T00:37:01.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:01 smithi099 ceph-mon[102692]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-09-10T00:37:03.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:03 smithi044 ceph-mon[98676]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-09-10T00:37:03.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:03 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-10T00:37:03.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:03 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:03.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:03 smithi131 ceph-mon[103713]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-09-10T00:37:03.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:03 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-10T00:37:03.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:03 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:03.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:03 smithi099 ceph-mon[102692]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-09-10T00:37:03.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-10T00:37:03.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:03 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:04.542 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:04 smithi044 ceph-mon[98676]: Deploying daemon osd.9 on smithi044 2023-09-10T00:37:04.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:04 smithi131 ceph-mon[103713]: Deploying daemon osd.9 on smithi044 2023-09-10T00:37:04.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:04 smithi099 ceph-mon[102692]: Deploying daemon osd.9 on smithi044 2023-09-10T00:37:05.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:05 smithi044 ceph-mon[98676]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-10T00:37:05.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:05 smithi131 ceph-mon[103713]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-10T00:37:05.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:05 smithi099 ceph-mon[102692]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-10T00:37:07.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:07 smithi131 ceph-mon[103713]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-09-10T00:37:07.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:07 smithi044 ceph-mon[98676]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-09-10T00:37:07.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:07 smithi099 ceph-mon[102692]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-09-10T00:37:09.301 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:09 smithi044 ceph-mon[98676]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-09-10T00:37:09.302 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:09.302 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:09.302 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:09.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:09 smithi099 ceph-mon[102692]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-09-10T00:37:09.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:09.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:09.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:09.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:09 smithi131 ceph-mon[103713]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-09-10T00:37:09.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:09.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:09.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:10.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:10.996+0000 7f292b7fe700 1 -- 172.21.15.44:0/463321213 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f293c02cff0 con 0x7f29240a0640 2023-09-10T00:37:10.998 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 9 on host 'smithi044' 2023-09-10T00:37:10.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:10.999+0000 7f2940ca4700 1 -- 172.21.15.44:0/463321213 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f29240a0640 msgr2=0x7f29240a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:10.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:10.999+0000 7f2940ca4700 1 --2- 172.21.15.44:0/463321213 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f29240a0640 0x7f29240a2b00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2930005fd0 tx=0x7f293000a560 comp rx=0 tx=0).stop 2023-09-10T00:37:10.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:10.999+0000 7f2940ca4700 1 -- 172.21.15.44:0/463321213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f293c0ff5c0 msgr2=0x7f293c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:11.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:10.999+0000 7f2940ca4700 1 --2- 172.21.15.44:0/463321213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f293c0ff5c0 0x7f293c069560 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f292c009f50 tx=0x7f292c005f10 comp rx=0 tx=0).stop 2023-09-10T00:37:11.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:10.999+0000 7f2940ca4700 1 -- 172.21.15.44:0/463321213 shutdown_connections 2023-09-10T00:37:11.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:10.999+0000 7f2940ca4700 1 --2- 172.21.15.44:0/463321213 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f29240a0640 0x7f29240a2b00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:11.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:10.999+0000 7f2940ca4700 1 --2- 172.21.15.44:0/463321213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f293c101160 0x7f293c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:11.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:10.999+0000 7f2940ca4700 1 --2- 172.21.15.44:0/463321213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f293c0ff5c0 0x7f293c069560 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:11.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:10.999+0000 7f2940ca4700 1 --2- 172.21.15.44:0/463321213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f293c1007c0 0x7f293c19aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:11.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:10.999+0000 7f2940ca4700 1 -- 172.21.15.44:0/463321213 >> 172.21.15.44:0/463321213 conn(0x7f293c0fab80 msgr2=0x7f293c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:37:11.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:10.999+0000 7f2940ca4700 1 -- 172.21.15.44:0/463321213 shutdown_connections 2023-09-10T00:37:11.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:10.999+0000 7f2940ca4700 1 -- 172.21.15.44:0/463321213 wait complete. 2023-09-10T00:37:11.465 DEBUG:teuthology.orchestra.run.smithi044:osd.9> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.9.service 2023-09-10T00:37:11.469 INFO:tasks.cephadm:Deploying osd.10 on smithi099 with /dev/vg_nvme/lv_1... 2023-09-10T00:37:11.470 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-10T00:37:11.500 INFO:journalctl@ceph.osd.9.smithi044.stdout:-- Logs begin at Sun 2023-09-10 00:17:15 UTC. -- 2023-09-10T00:37:11.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:11 smithi099 ceph-mon[102692]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-10T00:37:11.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:11.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:11.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:11.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:11.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:11.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:11.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:11.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:11.942 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:11 smithi044 ceph-mon[98676]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-10T00:37:11.942 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:11.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:11.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:11.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:11.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:11.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:11.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:11.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:12.035 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-10T00:37:12.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:11 smithi131 ceph-mon[103713]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-10T00:37:12.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:12.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:12.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:12.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:12.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:12.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:12.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:12.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:12.234 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 00:37:11 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T00:37:11.956+0000 7f7804d86640 -1 osd.9 0 log_to_monitors true 2023-09-10T00:37:12.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:12 smithi044 ceph-mon[98676]: from='osd.9 [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-10T00:37:12.830 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:12 smithi099 ceph-mon[102692]: from='osd.9 [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-10T00:37:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:12 smithi131 ceph-mon[103713]: from='osd.9 [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-10T00:37:13.535 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-10T00:37:13.535 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-10T00:37:13.536 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2023-09-10T00:37:13.536 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2023-09-10T00:37:13.536 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110869 s, 946 MB/s 2023-09-10T00:37:13.536 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2023-09-10T00:37:13.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:13 smithi099 ceph-mon[102692]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-10T00:37:13.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:13 smithi099 ceph-mon[102692]: from='osd.9 [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-10T00:37:13.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:13 smithi099 ceph-mon[102692]: osdmap e63: 10 total, 9 up, 10 in 2023-09-10T00:37:13.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:13 smithi099 ceph-mon[102692]: from='osd.9 [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:37:13.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:13 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:37:13.878 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi099:vg_nvme/lv_1 2023-09-10T00:37:13.974 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 00:37:13 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T00:37:13.650+0000 7f77fa3fd700 -1 osd.9 0 waiting for initial osdmap 2023-09-10T00:37:13.975 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 00:37:13 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T00:37:13.665+0000 7f77f21ea700 -1 osd.9 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-10T00:37:13.975 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:13 smithi044 ceph-mon[98676]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-10T00:37:13.976 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:13 smithi044 ceph-mon[98676]: from='osd.9 [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-10T00:37:13.976 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:13 smithi044 ceph-mon[98676]: osdmap e63: 10 total, 9 up, 10 in 2023-09-10T00:37:13.976 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:13 smithi044 ceph-mon[98676]: from='osd.9 [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:37:13.977 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:13 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:37:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:13 smithi131 ceph-mon[103713]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-10T00:37:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:13 smithi131 ceph-mon[103713]: from='osd.9 [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-10T00:37:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:13 smithi131 ceph-mon[103713]: osdmap e63: 10 total, 9 up, 10 in 2023-09-10T00:37:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:13 smithi131 ceph-mon[103713]: from='osd.9 [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2023-09-10T00:37:14.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:13 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:37:14.436 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-10T00:37:14.977 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:14 smithi099 ceph-mon[102692]: from='osd.9 [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2023-09-10T00:37:14.977 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:14 smithi099 ceph-mon[102692]: osdmap e64: 10 total, 9 up, 10 in 2023-09-10T00:37:14.977 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:37:14.978 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:37:14.978 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:14.978 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:14.978 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:14.978 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:14.979 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:14.979 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:14.979 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:14.979 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:14.979 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:14.980 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:14 smithi131 ceph-mon[103713]: from='osd.9 [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2023-09-10T00:37:15.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:14 smithi131 ceph-mon[103713]: osdmap e64: 10 total, 9 up, 10 in 2023-09-10T00:37:15.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:37:15.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:37:15.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:15.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:15.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:15.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:15.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:15.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:15.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:15.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:15.050 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:15.050 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:14 smithi044 ceph-mon[98676]: from='osd.9 [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2023-09-10T00:37:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:14 smithi044 ceph-mon[98676]: osdmap e64: 10 total, 9 up, 10 in 2023-09-10T00:37:15.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:37:15.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:37:15.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:15.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:15.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:15.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:15.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:15.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:15.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:15.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:15.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:15.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:15.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.655+0000 7fb4f2429700 1 -- 172.21.15.99:0/547900170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4ec100790 msgr2=0x7fb4ec100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:15.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.655+0000 7fb4f2429700 1 --2- 172.21.15.99:0/547900170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4ec100790 0x7fb4ec100bf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00099c0 tx=0x7fb4e0009cd0 comp rx=0 tx=0).stop 2023-09-10T00:37:15.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.655+0000 7fb4f2429700 1 -- 172.21.15.99:0/547900170 shutdown_connections 2023-09-10T00:37:15.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.655+0000 7fb4f2429700 1 --2- 172.21.15.99:0/547900170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4ec101130 0x7fb4ec1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:15.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.655+0000 7fb4f2429700 1 --2- 172.21.15.99:0/547900170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4ec0ff590 0x7fb4ec0ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:15.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.655+0000 7fb4f2429700 1 --2- 172.21.15.99:0/547900170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4ec100790 0x7fb4ec100bf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:15.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.655+0000 7fb4f2429700 1 -- 172.21.15.99:0/547900170 >> 172.21.15.99:0/547900170 conn(0x7fb4ec0fab10 msgr2=0x7fb4ec0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:37:15.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.655+0000 7fb4f2429700 1 -- 172.21.15.99:0/547900170 shutdown_connections 2023-09-10T00:37:15.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.655+0000 7fb4f2429700 1 -- 172.21.15.99:0/547900170 wait complete. 2023-09-10T00:37:15.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.656+0000 7fb4f2429700 1 Processor -- start 2023-09-10T00:37:15.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.656+0000 7fb4f2429700 1 -- start start 2023-09-10T00:37:15.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.656+0000 7fb4f2429700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4ec0ff590 0x7fb4ec103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:37:15.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.656+0000 7fb4f2429700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4ec100790 0x7fb4ec101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:37:15.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.656+0000 7fb4f2429700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4ec101130 0x7fb4ec1020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:37:15.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.656+0000 7fb4f2429700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec102830 con 0x7fb4ec0ff590 2023-09-10T00:37:15.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.656+0000 7fb4f2429700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec1029a0 con 0x7fb4ec101130 2023-09-10T00:37:15.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.656+0000 7fb4f2429700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec1a0290 con 0x7fb4ec100790 2023-09-10T00:37:15.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4f09c6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4ec101130 0x7fb4ec1020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:37:15.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4ebfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4ec0ff590 0x7fb4ec103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:37:15.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4f09c6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4ec101130 0x7fb4ec1020b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40926/0 (socket says 172.21.15.99:40926) 2023-09-10T00:37:15.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4ebfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4ec0ff590 0x7fb4ec103520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.99:33854/0 (socket says 172.21.15.99:33854) 2023-09-10T00:37:15.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4f09c6700 1 -- 172.21.15.99:0/3614463196 learned_addr learned my addr 172.21.15.99:0/3614463196 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-09-10T00:37:15.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4eb7fe700 1 --2- 172.21.15.99:0/3614463196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4ec100790 0x7fb4ec101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:37:15.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4f09c6700 1 -- 172.21.15.99:0/3614463196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4ec100790 msgr2=0x7fb4ec101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:15.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4f09c6700 1 --2- 172.21.15.99:0/3614463196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4ec100790 0x7fb4ec101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:15.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4f09c6700 1 -- 172.21.15.99:0/3614463196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4ec0ff590 msgr2=0x7fb4ec103520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:15.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4f09c6700 1 --2- 172.21.15.99:0/3614463196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4ec0ff590 0x7fb4ec103520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:15.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4f09c6700 1 -- 172.21.15.99:0/3614463196 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e00095f0 con 0x7fb4ec101130 2023-09-10T00:37:15.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4f09c6700 1 --2- 172.21.15.99:0/3614463196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4ec101130 0x7fb4ec1020b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4007ee0 tx=0x7fb4e400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:37:15.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4e97fa700 1 -- 172.21.15.99:0/3614463196 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4017070 con 0x7fb4ec101130 2023-09-10T00:37:15.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4e97fa700 1 -- 172.21.15.99:0/3614463196 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4e400f040 con 0x7fb4ec101130 2023-09-10T00:37:15.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4f2429700 1 -- 172.21.15.99:0/3614463196 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ec1a0570 con 0x7fb4ec101130 2023-09-10T00:37:15.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4e97fa700 1 -- 172.21.15.99:0/3614463196 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4005710 con 0x7fb4ec101130 2023-09-10T00:37:15.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.657+0000 7fb4f2429700 1 -- 172.21.15.99:0/3614463196 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ec1a0a20 con 0x7fb4ec101130 2023-09-10T00:37:15.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.658+0000 7fb4f2429700 1 -- 172.21.15.99:0/3614463196 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4ec10ac50 con 0x7fb4ec101130 2023-09-10T00:37:15.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.659+0000 7fb4e97fa700 1 -- 172.21.15.99:0/3614463196 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fb4e4005870 con 0x7fb4ec101130 2023-09-10T00:37:15.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.660+0000 7fb4e97fa700 1 --2- 172.21.15.99:0/3614463196 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb4d40a0640 0x7fb4d40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:37:15.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.660+0000 7fb4e97fa700 1 -- 172.21.15.99:0/3614463196 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fb4e40cabc0 con 0x7fb4ec101130 2023-09-10T00:37:15.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.660+0000 7fb4ebfff700 1 --2- 172.21.15.99:0/3614463196 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb4d40a0640 0x7fb4d40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:37:15.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.660+0000 7fb4ebfff700 1 --2- 172.21.15.99:0/3614463196 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb4d40a0640 0x7fb4d40a2b00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec1005f0 tx=0x7fb4dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:37:15.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.667+0000 7fb4e97fa700 1 -- 172.21.15.99:0/3614463196 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb4e4094ef0 con 0x7fb4ec101130 2023-09-10T00:37:15.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:15.865+0000 7fb4f2429700 1 -- 172.21.15.99:0/3614463196 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4ec068810 con 0x7fb4d40a0640 2023-09-10T00:37:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:15 smithi131 ceph-mon[103713]: purged_snaps scrub starts 2023-09-10T00:37:16.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:15 smithi131 ceph-mon[103713]: purged_snaps scrub ok 2023-09-10T00:37:16.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:15 smithi131 ceph-mon[103713]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-10T00:37:16.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:15 smithi131 ceph-mon[103713]: Detected new or changed devices on smithi044 2023-09-10T00:37:16.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:15 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi044 to 4293M 2023-09-10T00:37:16.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:15 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:37:16.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:15 smithi131 ceph-mon[103713]: osd.9 [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640] boot 2023-09-10T00:37:16.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:15 smithi131 ceph-mon[103713]: osdmap e65: 10 total, 10 up, 10 in 2023-09-10T00:37:16.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:15 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:37:16.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:15 smithi044 ceph-mon[98676]: purged_snaps scrub starts 2023-09-10T00:37:16.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:15 smithi044 ceph-mon[98676]: purged_snaps scrub ok 2023-09-10T00:37:16.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:15 smithi044 ceph-mon[98676]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-10T00:37:16.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:15 smithi044 ceph-mon[98676]: Detected new or changed devices on smithi044 2023-09-10T00:37:16.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:15 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi044 to 4293M 2023-09-10T00:37:16.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:15 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:37:16.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:15 smithi044 ceph-mon[98676]: osd.9 [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640] boot 2023-09-10T00:37:16.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:15 smithi044 ceph-mon[98676]: osdmap e65: 10 total, 10 up, 10 in 2023-09-10T00:37:16.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:15 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:37:16.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:15 smithi099 ceph-mon[102692]: purged_snaps scrub starts 2023-09-10T00:37:16.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:15 smithi099 ceph-mon[102692]: purged_snaps scrub ok 2023-09-10T00:37:16.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:15 smithi099 ceph-mon[102692]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-10T00:37:16.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:15 smithi099 ceph-mon[102692]: Detected new or changed devices on smithi044 2023-09-10T00:37:16.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:15 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi044 to 4293M 2023-09-10T00:37:16.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:15 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:37:16.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:15 smithi099 ceph-mon[102692]: osd.9 [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640] boot 2023-09-10T00:37:16.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:15 smithi099 ceph-mon[102692]: osdmap e65: 10 total, 10 up, 10 in 2023-09-10T00:37:16.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:15 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:37:16.943 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:16 smithi099 ceph-mon[102692]: osdmap e66: 10 total, 10 up, 10 in 2023-09-10T00:37:16.943 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:37:16.943 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:37:16.944 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:16 smithi131 ceph-mon[103713]: osdmap e66: 10 total, 10 up, 10 in 2023-09-10T00:37:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:37:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:37:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:16 smithi044 ceph-mon[98676]: osdmap e66: 10 total, 10 up, 10 in 2023-09-10T00:37:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:37:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:37:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:18.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:17 smithi131 ceph-mon[103713]: from='client.24434 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:37:18.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:17 smithi131 ceph-mon[103713]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:18.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:17 smithi044 ceph-mon[98676]: from='client.24434 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:37:18.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:17 smithi044 ceph-mon[98676]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:18.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:17 smithi099 ceph-mon[102692]: from='client.24434 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:37:18.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:17 smithi099 ceph-mon[102692]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:18 smithi131 ceph-mon[103713]: from='client.? 172.21.15.99:0/3411205827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aee4c5aa-0fb1-447d-948f-61cd27d5eecb"}]: dispatch 2023-09-10T00:37:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:18 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aee4c5aa-0fb1-447d-948f-61cd27d5eecb"}]: dispatch 2023-09-10T00:37:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:18 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aee4c5aa-0fb1-447d-948f-61cd27d5eecb"}]': finished 2023-09-10T00:37:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:18 smithi131 ceph-mon[103713]: osdmap e67: 11 total, 10 up, 11 in 2023-09-10T00:37:19.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:18 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:37:19.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:18 smithi044 ceph-mon[98676]: from='client.? 172.21.15.99:0/3411205827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aee4c5aa-0fb1-447d-948f-61cd27d5eecb"}]: dispatch 2023-09-10T00:37:19.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:18 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aee4c5aa-0fb1-447d-948f-61cd27d5eecb"}]: dispatch 2023-09-10T00:37:19.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:18 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aee4c5aa-0fb1-447d-948f-61cd27d5eecb"}]': finished 2023-09-10T00:37:19.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:18 smithi044 ceph-mon[98676]: osdmap e67: 11 total, 10 up, 11 in 2023-09-10T00:37:19.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:18 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:37:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:18 smithi099 ceph-mon[102692]: from='client.? 172.21.15.99:0/3411205827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aee4c5aa-0fb1-447d-948f-61cd27d5eecb"}]: dispatch 2023-09-10T00:37:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:18 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aee4c5aa-0fb1-447d-948f-61cd27d5eecb"}]: dispatch 2023-09-10T00:37:19.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:18 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aee4c5aa-0fb1-447d-948f-61cd27d5eecb"}]': finished 2023-09-10T00:37:19.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:18 smithi099 ceph-mon[102692]: osdmap e67: 11 total, 10 up, 11 in 2023-09-10T00:37:19.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:18 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:37:20.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:19 smithi131 ceph-mon[103713]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:20.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:19 smithi131 ceph-mon[103713]: from='client.? 172.21.15.99:0/3699180189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:37:20.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:19 smithi044 ceph-mon[98676]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:20.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:19 smithi044 ceph-mon[98676]: from='client.? 172.21.15.99:0/3699180189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:37:20.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:19 smithi099 ceph-mon[102692]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:20.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:19 smithi099 ceph-mon[102692]: from='client.? 172.21.15.99:0/3699180189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:37:21.952 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:21 smithi099 ceph-mon[102692]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:22.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:21 smithi131 ceph-mon[103713]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:22.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:21 smithi044 ceph-mon[98676]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:23.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:23 smithi099 ceph-mon[102692]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:24.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:23 smithi131 ceph-mon[103713]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:24.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:23 smithi044 ceph-mon[98676]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:25.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:25 smithi099 ceph-mon[102692]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:25.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:25 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-10T00:37:25.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:25 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:25.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:25 smithi099 ceph-mon[102692]: Deploying daemon osd.10 on smithi099 2023-09-10T00:37:26.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:25 smithi131 ceph-mon[103713]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:26.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:25 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-10T00:37:26.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:25 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:26.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:25 smithi131 ceph-mon[103713]: Deploying daemon osd.10 on smithi099 2023-09-10T00:37:26.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:25 smithi044 ceph-mon[98676]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:26.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:25 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-10T00:37:26.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:25 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:26.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:25 smithi044 ceph-mon[98676]: Deploying daemon osd.10 on smithi099 2023-09-10T00:37:27.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:27 smithi099 ceph-mon[102692]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:27 smithi131 ceph-mon[103713]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:28.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:27 smithi044 ceph-mon[98676]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:29.973 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:29 smithi099 ceph-mon[102692]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:29.974 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:29 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:29.974 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:29 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:29.974 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:29 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:29 smithi131 ceph-mon[103713]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:29 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:29 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:29 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:30.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:29 smithi044 ceph-mon[98676]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:30.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:29 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:30.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:29 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:30.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:29 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:31.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:31.676+0000 7fb4e97fa700 1 -- 172.21.15.99:0/3614463196 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fb4ec068810 con 0x7fb4d40a0640 2023-09-10T00:37:31.678 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 10 on host 'smithi099' 2023-09-10T00:37:31.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:31.678+0000 7fb4f2429700 1 -- 172.21.15.99:0/3614463196 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb4d40a0640 msgr2=0x7fb4d40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:31.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:31.678+0000 7fb4f2429700 1 --2- 172.21.15.99:0/3614463196 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb4d40a0640 0x7fb4d40a2b00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec1005f0 tx=0x7fb4dc00b410 comp rx=0 tx=0).stop 2023-09-10T00:37:31.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:31.678+0000 7fb4f2429700 1 -- 172.21.15.99:0/3614463196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4ec101130 msgr2=0x7fb4ec1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:31.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:31.678+0000 7fb4f2429700 1 --2- 172.21.15.99:0/3614463196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4ec101130 0x7fb4ec1020b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4007ee0 tx=0x7fb4e400d3b0 comp rx=0 tx=0).stop 2023-09-10T00:37:31.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:31.678+0000 7fb4f2429700 1 -- 172.21.15.99:0/3614463196 shutdown_connections 2023-09-10T00:37:31.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:31.678+0000 7fb4f2429700 1 --2- 172.21.15.99:0/3614463196 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb4d40a0640 0x7fb4d40a2b00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:31.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:31.678+0000 7fb4f2429700 1 --2- 172.21.15.99:0/3614463196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4ec101130 0x7fb4ec1020b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:31.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:31.678+0000 7fb4f2429700 1 --2- 172.21.15.99:0/3614463196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4ec0ff590 0x7fb4ec103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:31.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:31.678+0000 7fb4f2429700 1 --2- 172.21.15.99:0/3614463196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4ec100790 0x7fb4ec101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:31.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:31.678+0000 7fb4f2429700 1 -- 172.21.15.99:0/3614463196 >> 172.21.15.99:0/3614463196 conn(0x7fb4ec0fab10 msgr2=0x7fb4ec068100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:37:31.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:31.679+0000 7fb4f2429700 1 -- 172.21.15.99:0/3614463196 shutdown_connections 2023-09-10T00:37:31.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-09-10T00:37:31.679+0000 7fb4f2429700 1 -- 172.21.15.99:0/3614463196 wait complete. 2023-09-10T00:37:32.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:31 smithi099 ceph-mon[102692]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:32.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:31 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:32.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:31 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:32.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:31 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:32.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:31 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:32.024 DEBUG:teuthology.orchestra.run.smithi099:osd.10> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.10.service 2023-09-10T00:37:32.026 INFO:tasks.cephadm:Deploying osd.11 on smithi131 with /dev/vg_nvme/lv_1... 2023-09-10T00:37:32.027 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-10T00:37:32.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:31 smithi131 ceph-mon[103713]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:32.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:31 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:32.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:31 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:32.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:31 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:32.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:31 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:32.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:31 smithi044 ceph-mon[98676]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:32.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:31 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:32.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:31 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:32.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:31 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:32.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:31 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:32.056 INFO:journalctl@ceph.osd.10.smithi099.stdout:-- Logs begin at Sun 2023-09-10 00:17:22 UTC. -- 2023-09-10T00:37:32.592 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-10T00:37:32.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:32 smithi099 ceph-mon[102692]: from='osd.10 [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-10T00:37:32.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:32 smithi099 ceph-mon[102692]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-10T00:37:32.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 00:37:32 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T00:37:32.650+0000 7f40c26ef640 -1 osd.10 0 log_to_monitors true 2023-09-10T00:37:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:32 smithi131 ceph-mon[103713]: from='osd.10 [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-10T00:37:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:32 smithi131 ceph-mon[103713]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-10T00:37:33.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:32 smithi044 ceph-mon[98676]: from='osd.10 [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-10T00:37:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:32 smithi044 ceph-mon[98676]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-10T00:37:33.780 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:33 smithi131 ceph-mon[103713]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:33.780 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:33 smithi131 ceph-mon[103713]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-10T00:37:33.780 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:33 smithi131 ceph-mon[103713]: from='osd.10 [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:37:33.781 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:33 smithi131 ceph-mon[103713]: osdmap e68: 11 total, 10 up, 11 in 2023-09-10T00:37:33.781 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:33 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:37:33.781 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:33 smithi131 ceph-mon[103713]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:37:33.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 00:37:33 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T00:37:33.712+0000 7f40b7d66700 -1 osd.10 0 waiting for initial osdmap 2023-09-10T00:37:33.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 00:37:33 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T00:37:33.723+0000 7f40b0354700 -1 osd.10 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-10T00:37:33.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:33 smithi099 ceph-mon[102692]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:33.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:33 smithi099 ceph-mon[102692]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-10T00:37:33.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:33 smithi099 ceph-mon[102692]: from='osd.10 [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:37:33.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:33 smithi099 ceph-mon[102692]: osdmap e68: 11 total, 10 up, 11 in 2023-09-10T00:37:33.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:33 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:37:33.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:33 smithi099 ceph-mon[102692]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:37:34.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:33 smithi044 ceph-mon[98676]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:34.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:33 smithi044 ceph-mon[98676]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-10T00:37:34.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:33 smithi044 ceph-mon[98676]: from='osd.10 [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:37:34.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:33 smithi044 ceph-mon[98676]: osdmap e68: 11 total, 10 up, 11 in 2023-09-10T00:37:34.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:33 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:37:34.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:33 smithi044 ceph-mon[98676]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-09-10T00:37:34.225 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-10T00:37:34.226 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-10T00:37:34.226 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2023-09-10T00:37:34.226 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2023-09-10T00:37:34.226 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109252 s, 960 MB/s 2023-09-10T00:37:34.227 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2023-09-10T00:37:34.741 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi131:vg_nvme/lv_1 2023-09-10T00:37:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:34 smithi131 ceph-mon[103713]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-09-10T00:37:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:34 smithi131 ceph-mon[103713]: osdmap e69: 11 total, 10 up, 11 in 2023-09-10T00:37:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:34 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:37:35.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:34 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:37:35.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:34 smithi131 ceph-mon[103713]: osd.10 [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032] boot 2023-09-10T00:37:35.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:34 smithi131 ceph-mon[103713]: osdmap e70: 11 total, 11 up, 11 in 2023-09-10T00:37:35.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:34 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:37:35.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:34 smithi044 ceph-mon[98676]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-09-10T00:37:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:34 smithi044 ceph-mon[98676]: osdmap e69: 11 total, 10 up, 11 in 2023-09-10T00:37:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:34 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:37:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:34 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:37:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:34 smithi044 ceph-mon[98676]: osd.10 [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032] boot 2023-09-10T00:37:35.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:34 smithi044 ceph-mon[98676]: osdmap e70: 11 total, 11 up, 11 in 2023-09-10T00:37:35.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:34 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:37:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:34 smithi099 ceph-mon[102692]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-09-10T00:37:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:34 smithi099 ceph-mon[102692]: osdmap e69: 11 total, 10 up, 11 in 2023-09-10T00:37:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:34 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:37:35.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:34 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:37:35.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:34 smithi099 ceph-mon[102692]: osd.10 [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032] boot 2023-09-10T00:37:35.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:34 smithi099 ceph-mon[102692]: osdmap e70: 11 total, 11 up, 11 in 2023-09-10T00:37:35.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:34 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:37:35.338 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-10T00:37:35.948 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: purged_snaps scrub starts 2023-09-10T00:37:35.948 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: purged_snaps scrub ok 2023-09-10T00:37:35.949 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:35.949 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:35.949 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:35.949 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:35.949 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:35.950 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:35.950 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:35.950 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:35.950 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:35.950 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:35.950 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:35.951 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:35.951 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:35.951 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:35.951 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:35.951 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:35 smithi131 ceph-mon[103713]: osdmap e71: 11 total, 11 up, 11 in 2023-09-10T00:37:36.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: purged_snaps scrub starts 2023-09-10T00:37:36.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: purged_snaps scrub ok 2023-09-10T00:37:36.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:36.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:36.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:36.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:36.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:36.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:36.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:36.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:36.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:36.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:36.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:36.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:36.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:36.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:36.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:36.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:35 smithi044 ceph-mon[98676]: osdmap e71: 11 total, 11 up, 11 in 2023-09-10T00:37:36.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: purged_snaps scrub starts 2023-09-10T00:37:36.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: purged_snaps scrub ok 2023-09-10T00:37:36.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-10T00:37:36.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:36.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:36.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:36.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:36.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:36.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:36.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:36.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:36.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:36.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:36.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:36.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:36.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:36.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:36.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:35 smithi099 ceph-mon[102692]: osdmap e71: 11 total, 11 up, 11 in 2023-09-10T00:37:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.782+0000 7fe60139b700 1 -- 172.21.15.131:0/3700308049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5fc101990 msgr2=0x7fe5fc101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.782+0000 7fe60139b700 1 --2- 172.21.15.131:0/3700308049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5fc101990 0x7fe5fc101df0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009990 tx=0x7fe5f0009ca0 comp rx=0 tx=0).stop 2023-09-10T00:37:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.782+0000 7fe60139b700 1 -- 172.21.15.131:0/3700308049 shutdown_connections 2023-09-10T00:37:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.782+0000 7fe60139b700 1 --2- 172.21.15.131:0/3700308049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5fc102330 0x7fe5fc108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:36.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.782+0000 7fe60139b700 1 --2- 172.21.15.131:0/3700308049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5fc0fe9b0 0x7fe5fc0fedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:36.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.782+0000 7fe60139b700 1 --2- 172.21.15.131:0/3700308049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5fc101990 0x7fe5fc101df0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:36.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.782+0000 7fe60139b700 1 -- 172.21.15.131:0/3700308049 >> 172.21.15.131:0/3700308049 conn(0x7fe5fc0fa310 msgr2=0x7fe5fc0fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:37:36.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.782+0000 7fe60139b700 1 -- 172.21.15.131:0/3700308049 shutdown_connections 2023-09-10T00:37:36.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.782+0000 7fe60139b700 1 -- 172.21.15.131:0/3700308049 wait complete. 2023-09-10T00:37:36.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe60139b700 1 Processor -- start 2023-09-10T00:37:36.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe60139b700 1 -- start start 2023-09-10T00:37:36.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe60139b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5fc0fe9b0 0x7fe5fc1961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:37:36.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe60139b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5fc101990 0x7fe5fc196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:37:36.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe60139b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5fc102330 0x7fe5fc19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:37:36.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe60139b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc19dd80 con 0x7fe5fc102330 2023-09-10T00:37:36.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe60139b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc19dec0 con 0x7fe5fc0fe9b0 2023-09-10T00:37:36.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe60139b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc19e050 con 0x7fe5fc101990 2023-09-10T00:37:36.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe5fa7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5fc101990 0x7fe5fc196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:37:36.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe5fa7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5fc101990 0x7fe5fc196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53242/0 (socket says 172.21.15.131:53242) 2023-09-10T00:37:36.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe5fa7fc700 1 -- 172.21.15.131:0/2687446876 learned_addr learned my addr 172.21.15.131:0/2687446876 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-10T00:37:36.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe5fb7fe700 1 --2- 172.21.15.131:0/2687446876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5fc102330 0x7fe5fc19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:37:36.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe5faffd700 1 --2- 172.21.15.131:0/2687446876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5fc0fe9b0 0x7fe5fc1961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:37:36.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe5fa7fc700 1 -- 172.21.15.131:0/2687446876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5fc0fe9b0 msgr2=0x7fe5fc1961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:36.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe5fa7fc700 1 --2- 172.21.15.131:0/2687446876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5fc0fe9b0 0x7fe5fc1961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:36.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe5fa7fc700 1 -- 172.21.15.131:0/2687446876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5fc102330 msgr2=0x7fe5fc19d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:36.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe5fa7fc700 1 --2- 172.21.15.131:0/2687446876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5fc102330 0x7fe5fc19d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:36.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe5fa7fc700 1 -- 172.21.15.131:0/2687446876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f00095f0 con 0x7fe5fc101990 2023-09-10T00:37:36.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe5fb7fe700 1 --2- 172.21.15.131:0/2687446876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5fc102330 0x7fe5fc19d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:37:36.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.783+0000 7fe5fa7fc700 1 --2- 172.21.15.131:0/2687446876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5fc101990 0x7fe5fc196730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00059d0 tx=0x7fe5f0003b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:37:36.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.784+0000 7fe5e3fff700 1 -- 172.21.15.131:0/2687446876 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f001d070 con 0x7fe5fc101990 2023-09-10T00:37:36.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.784+0000 7fe5e3fff700 1 -- 172.21.15.131:0/2687446876 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5f000bd70 con 0x7fe5fc101990 2023-09-10T00:37:36.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.784+0000 7fe60139b700 1 -- 172.21.15.131:0/2687446876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5fc19e2d0 con 0x7fe5fc101990 2023-09-10T00:37:36.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.784+0000 7fe5e3fff700 1 -- 172.21.15.131:0/2687446876 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0022d20 con 0x7fe5fc101990 2023-09-10T00:37:36.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.784+0000 7fe60139b700 1 -- 172.21.15.131:0/2687446876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5fc19e740 con 0x7fe5fc101990 2023-09-10T00:37:36.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.785+0000 7fe60139b700 1 -- 172.21.15.131:0/2687446876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5fc107d40 con 0x7fe5fc101990 2023-09-10T00:37:36.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.786+0000 7fe5e3fff700 1 -- 172.21.15.131:0/2687446876 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fe5f0003fb0 con 0x7fe5fc101990 2023-09-10T00:37:36.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.786+0000 7fe5e3fff700 1 --2- 172.21.15.131:0/2687446876 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5e40a06c0 0x7fe5e40a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:37:36.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.786+0000 7fe5e3fff700 1 -- 172.21.15.131:0/2687446876 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fe5f00ccac0 con 0x7fe5fc101990 2023-09-10T00:37:36.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.786+0000 7fe5faffd700 1 --2- 172.21.15.131:0/2687446876 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5e40a06c0 0x7fe5e40a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:37:36.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.787+0000 7fe5faffd700 1 --2- 172.21.15.131:0/2687446876 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5e40a06c0 0x7fe5e40a2b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec005950 tx=0x7fe5ec0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:37:36.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.790+0000 7fe5e3fff700 1 -- 172.21.15.131:0/2687446876 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe5f0096cb0 con 0x7fe5fc101990 2023-09-10T00:37:36.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:36.987+0000 7fe60139b700 1 -- 172.21.15.131:0/2687446876 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5fc02cfd0 con 0x7fe5e40a06c0 2023-09-10T00:37:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:36 smithi131 ceph-mon[103713]: Detected new or changed devices on smithi099 2023-09-10T00:37:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:36 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi099 to 4292M 2023-09-10T00:37:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:36 smithi131 ceph-mon[103713]: osdmap e72: 11 total, 11 up, 11 in 2023-09-10T00:37:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:36 smithi044 ceph-mon[98676]: Detected new or changed devices on smithi099 2023-09-10T00:37:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:36 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi099 to 4292M 2023-09-10T00:37:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:36 smithi044 ceph-mon[98676]: osdmap e72: 11 total, 11 up, 11 in 2023-09-10T00:37:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:36 smithi099 ceph-mon[102692]: Detected new or changed devices on smithi099 2023-09-10T00:37:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:36 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi099 to 4292M 2023-09-10T00:37:37.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:36 smithi099 ceph-mon[102692]: osdmap e72: 11 total, 11 up, 11 in 2023-09-10T00:37:38.041 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:37 smithi131 ceph-mon[103713]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:38.042 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:37:38.042 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:37:38.042 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:38.043 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:37 smithi131 ceph-mon[103713]: osdmap e73: 11 total, 11 up, 11 in 2023-09-10T00:37:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:37 smithi044 ceph-mon[98676]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:37:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:37:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:37 smithi044 ceph-mon[98676]: osdmap e73: 11 total, 11 up, 11 in 2023-09-10T00:37:38.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:37 smithi099 ceph-mon[102692]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:38.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-10T00:37:38.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-10T00:37:38.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:38.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:37 smithi099 ceph-mon[102692]: osdmap e73: 11 total, 11 up, 11 in 2023-09-10T00:37:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:38 smithi131 ceph-mon[103713]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:37:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:38 smithi131 ceph-mon[103713]: osdmap e74: 11 total, 11 up, 11 in 2023-09-10T00:37:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:38 smithi044 ceph-mon[98676]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:37:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:38 smithi044 ceph-mon[98676]: osdmap e74: 11 total, 11 up, 11 in 2023-09-10T00:37:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:38 smithi099 ceph-mon[102692]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:37:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:38 smithi099 ceph-mon[102692]: osdmap e74: 11 total, 11 up, 11 in 2023-09-10T00:37:40.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:39 smithi131 ceph-mon[103713]: pgmap v224: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-10T00:37:40.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:39 smithi131 ceph-mon[103713]: from='client.? 172.21.15.131:0/3498604128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8d68bdc-aa91-4e82-9965-f74b9efa488a"}]: dispatch 2023-09-10T00:37:40.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:39 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8d68bdc-aa91-4e82-9965-f74b9efa488a"}]: dispatch 2023-09-10T00:37:40.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:39 smithi131 ceph-mon[103713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8d68bdc-aa91-4e82-9965-f74b9efa488a"}]': finished 2023-09-10T00:37:40.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:39 smithi131 ceph-mon[103713]: osdmap e75: 12 total, 11 up, 12 in 2023-09-10T00:37:40.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:39 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:37:40.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:39 smithi044 ceph-mon[98676]: pgmap v224: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-10T00:37:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:39 smithi044 ceph-mon[98676]: from='client.? 172.21.15.131:0/3498604128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8d68bdc-aa91-4e82-9965-f74b9efa488a"}]: dispatch 2023-09-10T00:37:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:39 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8d68bdc-aa91-4e82-9965-f74b9efa488a"}]: dispatch 2023-09-10T00:37:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:39 smithi044 ceph-mon[98676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8d68bdc-aa91-4e82-9965-f74b9efa488a"}]': finished 2023-09-10T00:37:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:39 smithi044 ceph-mon[98676]: osdmap e75: 12 total, 11 up, 12 in 2023-09-10T00:37:40.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:39 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:37:40.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:39 smithi099 ceph-mon[102692]: pgmap v224: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-10T00:37:40.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:39 smithi099 ceph-mon[102692]: from='client.? 172.21.15.131:0/3498604128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8d68bdc-aa91-4e82-9965-f74b9efa488a"}]: dispatch 2023-09-10T00:37:40.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:39 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8d68bdc-aa91-4e82-9965-f74b9efa488a"}]: dispatch 2023-09-10T00:37:40.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:39 smithi099 ceph-mon[102692]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8d68bdc-aa91-4e82-9965-f74b9efa488a"}]': finished 2023-09-10T00:37:40.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:39 smithi099 ceph-mon[102692]: osdmap e75: 12 total, 11 up, 12 in 2023-09-10T00:37:40.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:39 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:37:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:40 smithi131 ceph-mon[103713]: from='client.? 172.21.15.131:0/2126675513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:37:41.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:40 smithi044 ceph-mon[98676]: from='client.? 172.21.15.131:0/2126675513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:37:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:40 smithi099 ceph-mon[102692]: from='client.? 172.21.15.131:0/2126675513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-10T00:37:42.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:41 smithi131 ceph-mon[103713]: pgmap v227: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-10T00:37:42.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:41 smithi044 ceph-mon[98676]: pgmap v227: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-10T00:37:42.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:41 smithi099 ceph-mon[102692]: pgmap v227: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-10T00:37:44.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:43 smithi131 ceph-mon[103713]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-09-10T00:37:44.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:43 smithi044 ceph-mon[98676]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-09-10T00:37:44.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:43 smithi099 ceph-mon[102692]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-09-10T00:37:46.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:45 smithi131 ceph-mon[103713]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:46.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:45 smithi044 ceph-mon[98676]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:46.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:45 smithi099 ceph-mon[102692]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:46 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-10T00:37:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:46 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:47.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:46 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-10T00:37:47.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:46 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:46 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-10T00:37:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:46 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:48.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:47 smithi131 ceph-mon[103713]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:48.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:47 smithi131 ceph-mon[103713]: Deploying daemon osd.11 on smithi131 2023-09-10T00:37:48.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:47 smithi044 ceph-mon[98676]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:48.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:47 smithi044 ceph-mon[98676]: Deploying daemon osd.11 on smithi131 2023-09-10T00:37:48.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:47 smithi099 ceph-mon[102692]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:48.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:47 smithi099 ceph-mon[102692]: Deploying daemon osd.11 on smithi131 2023-09-10T00:37:50.021 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:49 smithi131 ceph-mon[103713]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:50.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:49 smithi044 ceph-mon[98676]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:50.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:49 smithi099 ceph-mon[102692]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:51 smithi131 ceph-mon[103713]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:51 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:51 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:51 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:52.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:51 smithi044 ceph-mon[98676]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:51 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:51 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:51 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:52.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:51 smithi099 ceph-mon[102692]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:52.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:51 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:52.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:51 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:52.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:51 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:53.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:53.652+0000 7fe5e3fff700 1 -- 172.21.15.131:0/2687446876 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fe5fc02cfd0 con 0x7fe5e40a06c0 2023-09-10T00:37:53.653 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 11 on host 'smithi131' 2023-09-10T00:37:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:53.654+0000 7fe60139b700 1 -- 172.21.15.131:0/2687446876 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5e40a06c0 msgr2=0x7fe5e40a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:53.654+0000 7fe60139b700 1 --2- 172.21.15.131:0/2687446876 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5e40a06c0 0x7fe5e40a2b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec005950 tx=0x7fe5ec0058e0 comp rx=0 tx=0).stop 2023-09-10T00:37:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:53.654+0000 7fe60139b700 1 -- 172.21.15.131:0/2687446876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5fc101990 msgr2=0x7fe5fc196730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:53.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:53.654+0000 7fe60139b700 1 --2- 172.21.15.131:0/2687446876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5fc101990 0x7fe5fc196730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00059d0 tx=0x7fe5f0003b00 comp rx=0 tx=0).stop 2023-09-10T00:37:53.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:53.654+0000 7fe60139b700 1 -- 172.21.15.131:0/2687446876 shutdown_connections 2023-09-10T00:37:53.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:53.654+0000 7fe60139b700 1 --2- 172.21.15.131:0/2687446876 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5e40a06c0 0x7fe5e40a2b80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:53.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:53.654+0000 7fe60139b700 1 --2- 172.21.15.131:0/2687446876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5fc102330 0x7fe5fc19d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:53.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:53.654+0000 7fe60139b700 1 --2- 172.21.15.131:0/2687446876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5fc0fe9b0 0x7fe5fc1961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:53.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:53.654+0000 7fe60139b700 1 --2- 172.21.15.131:0/2687446876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5fc101990 0x7fe5fc196730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:53.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:53.654+0000 7fe60139b700 1 -- 172.21.15.131:0/2687446876 >> 172.21.15.131:0/2687446876 conn(0x7fe5fc0fa310 msgr2=0x7fe5fc104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:37:53.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:53.654+0000 7fe60139b700 1 -- 172.21.15.131:0/2687446876 shutdown_connections 2023-09-10T00:37:53.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:37:53.654+0000 7fe60139b700 1 -- 172.21.15.131:0/2687446876 wait complete. 2023-09-10T00:37:54.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:53 smithi131 ceph-mon[103713]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:54.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:54.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:54.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:54.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:54.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:54.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:54.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:54.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:53 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:54.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:53 smithi044 ceph-mon[98676]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:54.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:54.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:54.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:54.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:54.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:54.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:54.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:54.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:53 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:54.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:53 smithi099 ceph-mon[102692]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:54.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:54.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:54.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:54.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:54.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:54.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:54.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:54.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:53 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:54.136 DEBUG:teuthology.orchestra.run.smithi131:osd.11> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.11.service 2023-09-10T00:37:54.138 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-09-10T00:37:54.139 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd stat -f json 2023-09-10T00:37:54.170 INFO:journalctl@ceph.osd.11.smithi131.stdout:-- Logs begin at Sun 2023-09-10 00:17:05 UTC. -- 2023-09-10T00:37:54.615 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:37:54.945 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 00:37:54 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T00:37:54.823+0000 7faedc5f7640 -1 osd.11 0 log_to_monitors true 2023-09-10T00:37:55.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.853+0000 7f8b59a61700 1 -- 172.21.15.44:0/2746132752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b541018b0 msgr2=0x7f8b54101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:55.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.853+0000 7f8b59a61700 1 --2- 172.21.15.44:0/2746132752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b541018b0 0x7f8b54101d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8b480099c0 tx=0x7f8b48009cd0 comp rx=0 tx=0).stop 2023-09-10T00:37:55.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.853+0000 7f8b59a61700 1 -- 172.21.15.44:0/2746132752 shutdown_connections 2023-09-10T00:37:55.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.853+0000 7f8b59a61700 1 --2- 172.21.15.44:0/2746132752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b54102250 0x7f8b54108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:55.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.853+0000 7f8b59a61700 1 --2- 172.21.15.44:0/2746132752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b541006b0 0x7f8b54100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:55.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.853+0000 7f8b59a61700 1 --2- 172.21.15.44:0/2746132752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b541018b0 0x7f8b54101d10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:55.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.853+0000 7f8b59a61700 1 -- 172.21.15.44:0/2746132752 >> 172.21.15.44:0/2746132752 conn(0x7f8b540fbc10 msgr2=0x7f8b540fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:37:55.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.853+0000 7f8b59a61700 1 -- 172.21.15.44:0/2746132752 shutdown_connections 2023-09-10T00:37:55.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.853+0000 7f8b59a61700 1 -- 172.21.15.44:0/2746132752 wait complete. 2023-09-10T00:37:55.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b59a61700 1 Processor -- start 2023-09-10T00:37:55.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b59a61700 1 -- start start 2023-09-10T00:37:55.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b59a61700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b541006b0 0x7f8b54196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:37:55.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b59a61700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b541018b0 0x7f8b54196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:37:55.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b59a61700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b54102250 0x7f8b5419d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:37:55.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b59a61700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5419deb0 con 0x7f8b541018b0 2023-09-10T00:37:55.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b59a61700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5419dff0 con 0x7f8b54102250 2023-09-10T00:37:55.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b59a61700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5419e180 con 0x7f8b541006b0 2023-09-10T00:37:55.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b527fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b541018b0 0x7f8b54196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:37:55.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b527fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b541018b0 0x7f8b54196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45886/0 (socket says 172.21.15.44:45886) 2023-09-10T00:37:55.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b527fc700 1 -- 172.21.15.44:0/3288255841 learned_addr learned my addr 172.21.15.44:0/3288255841 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:37:55.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b52ffd700 1 --2- 172.21.15.44:0/3288255841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b541006b0 0x7f8b54196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:37:55.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b537fe700 1 --2- 172.21.15.44:0/3288255841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b54102250 0x7f8b5419d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:37:55.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b527fc700 1 -- 172.21.15.44:0/3288255841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b541006b0 msgr2=0x7f8b54196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:55.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b527fc700 1 --2- 172.21.15.44:0/3288255841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b541006b0 0x7f8b54196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:55.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b527fc700 1 -- 172.21.15.44:0/3288255841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b54102250 msgr2=0x7f8b5419d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:55.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b527fc700 1 --2- 172.21.15.44:0/3288255841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b54102250 0x7f8b5419d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:55.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b527fc700 1 -- 172.21.15.44:0/3288255841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b480095f0 con 0x7f8b541018b0 2023-09-10T00:37:55.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.854+0000 7f8b537fe700 1 --2- 172.21.15.44:0/3288255841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b54102250 0x7f8b5419d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:37:55.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.855+0000 7f8b527fc700 1 --2- 172.21.15.44:0/3288255841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b541018b0 0x7f8b54196860 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800b580 tx=0x7f8b48003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:37:55.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.855+0000 7f8b58a5f700 1 -- 172.21.15.44:0/3288255841 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4801d070 con 0x7f8b541018b0 2023-09-10T00:37:55.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.855+0000 7f8b58a5f700 1 -- 172.21.15.44:0/3288255841 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b4800bd30 con 0x7f8b541018b0 2023-09-10T00:37:55.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.855+0000 7f8b58a5f700 1 -- 172.21.15.44:0/3288255841 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48022d40 con 0x7f8b541018b0 2023-09-10T00:37:55.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.855+0000 7f8b59a61700 1 -- 172.21.15.44:0/3288255841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5419e400 con 0x7f8b541018b0 2023-09-10T00:37:55.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.855+0000 7f8b59a61700 1 -- 172.21.15.44:0/3288255841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5419e7c0 con 0x7f8b541018b0 2023-09-10T00:37:55.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.856+0000 7f8b59a61700 1 -- 172.21.15.44:0/3288255841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b54107d50 con 0x7f8b541018b0 2023-09-10T00:37:55.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.857+0000 7f8b58a5f700 1 -- 172.21.15.44:0/3288255841 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f8b48022470 con 0x7f8b541018b0 2023-09-10T00:37:55.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.857+0000 7f8b58a5f700 1 --2- 172.21.15.44:0/3288255841 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8b3c0a0640 0x7f8b3c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:37:55.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.858+0000 7f8b52ffd700 1 --2- 172.21.15.44:0/3288255841 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8b3c0a0640 0x7f8b3c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:37:55.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.858+0000 7f8b58a5f700 1 -- 172.21.15.44:0/3288255841 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f8b480ccc10 con 0x7f8b541018b0 2023-09-10T00:37:55.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.858+0000 7f8b52ffd700 1 --2- 172.21.15.44:0/3288255841 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8b3c0a0640 0x7f8b3c0a2b00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8b54101710 tx=0x7f8b44009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:37:55.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:55.862+0000 7f8b58a5f700 1 -- 172.21.15.44:0/3288255841 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8b48096da0 con 0x7f8b541018b0 2023-09-10T00:37:56.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:55 smithi131 ceph-mon[103713]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:56.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:55 smithi131 ceph-mon[103713]: from='osd.11 [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-10T00:37:56.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:55 smithi131 ceph-mon[103713]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-10T00:37:56.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:55 smithi044 ceph-mon[98676]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:56.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:55 smithi044 ceph-mon[98676]: from='osd.11 [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-10T00:37:56.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:55 smithi044 ceph-mon[98676]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-10T00:37:56.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:56.062+0000 7f8b59a61700 1 -- 172.21.15.44:0/3288255841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f8b5410be00 con 0x7f8b541018b0 2023-09-10T00:37:56.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:56.063+0000 7f8b58a5f700 1 -- 172.21.15.44:0/3288255841 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7f8b4809a3c0 con 0x7f8b541018b0 2023-09-10T00:37:56.063 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:37:56.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:56.065+0000 7f8b59a61700 1 -- 172.21.15.44:0/3288255841 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8b3c0a0640 msgr2=0x7f8b3c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:56.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:56.065+0000 7f8b59a61700 1 --2- 172.21.15.44:0/3288255841 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8b3c0a0640 0x7f8b3c0a2b00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8b54101710 tx=0x7f8b44009190 comp rx=0 tx=0).stop 2023-09-10T00:37:56.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:56.065+0000 7f8b59a61700 1 -- 172.21.15.44:0/3288255841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b541018b0 msgr2=0x7f8b54196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:56.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:56.065+0000 7f8b59a61700 1 --2- 172.21.15.44:0/3288255841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b541018b0 0x7f8b54196860 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800b580 tx=0x7f8b48003a50 comp rx=0 tx=0).stop 2023-09-10T00:37:56.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:56.065+0000 7f8b59a61700 1 -- 172.21.15.44:0/3288255841 shutdown_connections 2023-09-10T00:37:56.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:56.065+0000 7f8b59a61700 1 --2- 172.21.15.44:0/3288255841 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8b3c0a0640 0x7f8b3c0a2b00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:56.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:56.065+0000 7f8b59a61700 1 --2- 172.21.15.44:0/3288255841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b54102250 0x7f8b5419d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:56.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:56.065+0000 7f8b59a61700 1 --2- 172.21.15.44:0/3288255841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b541006b0 0x7f8b54196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:56.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:56.065+0000 7f8b59a61700 1 --2- 172.21.15.44:0/3288255841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b541018b0 0x7f8b54196860 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:56.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:56.065+0000 7f8b59a61700 1 -- 172.21.15.44:0/3288255841 >> 172.21.15.44:0/3288255841 conn(0x7f8b540fbc10 msgr2=0x7f8b54104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:37:56.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:56.065+0000 7f8b59a61700 1 -- 172.21.15.44:0/3288255841 shutdown_connections 2023-09-10T00:37:56.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:56.065+0000 7f8b59a61700 1 -- 172.21.15.44:0/3288255841 wait complete. 2023-09-10T00:37:56.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:55 smithi099 ceph-mon[102692]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:56.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:55 smithi099 ceph-mon[102692]: from='osd.11 [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-10T00:37:56.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:55 smithi099 ceph-mon[102692]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-10T00:37:56.466 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1694306254,"num_in_osds":12,"osd_in_since":1694306259,"num_remapped_pgs":0} 2023-09-10T00:37:56.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:56 smithi044 ceph-mon[98676]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-10T00:37:56.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:56 smithi044 ceph-mon[98676]: osdmap e76: 12 total, 11 up, 12 in 2023-09-10T00:37:56.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:56 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:37:56.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:56 smithi044 ceph-mon[98676]: from='osd.11 [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:37:56.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:56 smithi044 ceph-mon[98676]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:37:56.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:56 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3288255841' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-10T00:37:56.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:56 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:56.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:56 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:56.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:56 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:56.789 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:56 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:56.789 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:56 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:56.789 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:56 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:56.789 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:56 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:56.789 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:56 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:56.790 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:56 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:56.790 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:56 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:57.045 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T00:37:56.788+0000 7faed1c6e700 -1 osd.11 0 waiting for initial osdmap 2023-09-10T00:37:57.045 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T00:37:56.805+0000 7faecd262700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-10T00:37:57.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-mon[103713]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-10T00:37:57.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-mon[103713]: osdmap e76: 12 total, 11 up, 12 in 2023-09-10T00:37:57.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:37:57.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-mon[103713]: from='osd.11 [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:37:57.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-mon[103713]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:37:57.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3288255841' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-10T00:37:57.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:57.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:57.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:57.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:57.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:57.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:57.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:57.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:57.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:57.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:56 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:56 smithi099 ceph-mon[102692]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-10T00:37:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:56 smithi099 ceph-mon[102692]: osdmap e76: 12 total, 11 up, 12 in 2023-09-10T00:37:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:56 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:37:57.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:56 smithi099 ceph-mon[102692]: from='osd.11 [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:37:57.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:56 smithi099 ceph-mon[102692]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2023-09-10T00:37:57.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:56 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3288255841' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-10T00:37:57.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:56 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:57.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:56 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:57.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:56 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:57.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:56 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:57.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:56 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:57.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:56 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:37:57.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:56 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:57.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:56 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:57.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:56 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:57.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:56 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:37:57.467 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd stat -f json 2023-09-10T00:37:57.796 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:57 smithi044 ceph-mon[98676]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:57.796 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:57 smithi044 ceph-mon[98676]: Detected new or changed devices on smithi131 2023-09-10T00:37:57.797 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:57 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi131 to 4292M 2023-09-10T00:37:57.797 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:57 smithi044 ceph-mon[98676]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2023-09-10T00:37:57.797 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:57 smithi044 ceph-mon[98676]: osdmap e77: 12 total, 11 up, 12 in 2023-09-10T00:37:57.797 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:57 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:37:57.797 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:57 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:37:58.046 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:37:58.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:57 smithi099 ceph-mon[102692]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:58.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:57 smithi099 ceph-mon[102692]: Detected new or changed devices on smithi131 2023-09-10T00:37:58.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:57 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi131 to 4292M 2023-09-10T00:37:58.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:57 smithi099 ceph-mon[102692]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2023-09-10T00:37:58.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:57 smithi099 ceph-mon[102692]: osdmap e77: 12 total, 11 up, 12 in 2023-09-10T00:37:58.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:57 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:37:58.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:57 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:37:58.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:57 smithi131 ceph-mon[103713]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:37:58.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:57 smithi131 ceph-mon[103713]: Detected new or changed devices on smithi131 2023-09-10T00:37:58.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:57 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi131 to 4292M 2023-09-10T00:37:58.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:57 smithi131 ceph-mon[103713]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2023-09-10T00:37:58.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:57 smithi131 ceph-mon[103713]: osdmap e77: 12 total, 11 up, 12 in 2023-09-10T00:37:58.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:57 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:37:58.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:57 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:37:58.842 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:58 smithi044 ceph-mon[98676]: purged_snaps scrub starts 2023-09-10T00:37:58.842 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:58 smithi044 ceph-mon[98676]: purged_snaps scrub ok 2023-09-10T00:37:58.842 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:58 smithi044 ceph-mon[98676]: osd.11 [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632] boot 2023-09-10T00:37:58.843 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:58 smithi044 ceph-mon[98676]: osdmap e78: 12 total, 12 up, 12 in 2023-09-10T00:37:58.843 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:58 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:37:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:58 smithi099 ceph-mon[102692]: purged_snaps scrub starts 2023-09-10T00:37:59.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:58 smithi099 ceph-mon[102692]: purged_snaps scrub ok 2023-09-10T00:37:59.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:58 smithi099 ceph-mon[102692]: osd.11 [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632] boot 2023-09-10T00:37:59.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:58 smithi099 ceph-mon[102692]: osdmap e78: 12 total, 12 up, 12 in 2023-09-10T00:37:59.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:58 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:37:59.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.242+0000 7fdc0bd78700 1 -- 172.21.15.44:0/1987821205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc04100790 msgr2=0x7fdc04100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:59.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.242+0000 7fdc0bd78700 1 --2- 172.21.15.44:0/1987821205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc04100790 0x7fdc04100bf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdc000099c0 tx=0x7fdc00009cd0 comp rx=0 tx=0).stop 2023-09-10T00:37:59.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.242+0000 7fdc0bd78700 1 -- 172.21.15.44:0/1987821205 shutdown_connections 2023-09-10T00:37:59.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.242+0000 7fdc0bd78700 1 --2- 172.21.15.44:0/1987821205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc04101130 0x7fdc041079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:59.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.242+0000 7fdc0bd78700 1 --2- 172.21.15.44:0/1987821205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc040ff590 0x7fdc040ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:59.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.242+0000 7fdc0bd78700 1 --2- 172.21.15.44:0/1987821205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc04100790 0x7fdc04100bf0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:59.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.242+0000 7fdc0bd78700 1 -- 172.21.15.44:0/1987821205 >> 172.21.15.44:0/1987821205 conn(0x7fdc040fab10 msgr2=0x7fdc040fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:37:59.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.242+0000 7fdc0bd78700 1 -- 172.21.15.44:0/1987821205 shutdown_connections 2023-09-10T00:37:59.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.242+0000 7fdc0bd78700 1 -- 172.21.15.44:0/1987821205 wait complete. 2023-09-10T00:37:59.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.243+0000 7fdc0bd78700 1 Processor -- start 2023-09-10T00:37:59.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.243+0000 7fdc0bd78700 1 -- start start 2023-09-10T00:37:59.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.243+0000 7fdc0bd78700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc040ff590 0x7fdc04103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:37:59.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.243+0000 7fdc0bd78700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc04100790 0x7fdc04101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:37:59.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.243+0000 7fdc0bd78700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc04101130 0x7fdc041020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:37:59.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.243+0000 7fdc0bd78700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc04102830 con 0x7fdc040ff590 2023-09-10T00:37:59.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.243+0000 7fdc0bd78700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc041029a0 con 0x7fdc04101130 2023-09-10T00:37:59.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.243+0000 7fdc0bd78700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc041a0330 con 0x7fdc04100790 2023-09-10T00:37:59.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.243+0000 7fdc09b14700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc040ff590 0x7fdc04103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:37:59.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.243+0000 7fdc09b14700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc040ff590 0x7fdc04103520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45904/0 (socket says 172.21.15.44:45904) 2023-09-10T00:37:59.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.243+0000 7fdc09b14700 1 -- 172.21.15.44:0/1859013769 learned_addr learned my addr 172.21.15.44:0/1859013769 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:37:59.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.244+0000 7fdc09313700 1 --2- 172.21.15.44:0/1859013769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc04100790 0x7fdc04101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:37:59.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.244+0000 7fdc0a315700 1 --2- 172.21.15.44:0/1859013769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc04101130 0x7fdc041020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:37:59.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.244+0000 7fdc09b14700 1 -- 172.21.15.44:0/1859013769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc04100790 msgr2=0x7fdc04101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:59.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.244+0000 7fdc09b14700 1 --2- 172.21.15.44:0/1859013769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc04100790 0x7fdc04101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:59.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.244+0000 7fdc09b14700 1 -- 172.21.15.44:0/1859013769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc04101130 msgr2=0x7fdc041020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:59.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.244+0000 7fdc09b14700 1 --2- 172.21.15.44:0/1859013769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc04101130 0x7fdc041020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:59.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.244+0000 7fdc09b14700 1 -- 172.21.15.44:0/1859013769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc000095f0 con 0x7fdc040ff590 2023-09-10T00:37:59.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.244+0000 7fdc0a315700 1 --2- 172.21.15.44:0/1859013769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc04101130 0x7fdc041020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:37:59.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.244+0000 7fdc09313700 1 --2- 172.21.15.44:0/1859013769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc04100790 0x7fdc04101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:37:59.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.244+0000 7fdc09b14700 1 --2- 172.21.15.44:0/1859013769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc040ff590 0x7fdc04103520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800c970 tx=0x7fdbf800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:37:59.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.244+0000 7fdbf6ffd700 1 -- 172.21.15.44:0/1859013769 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8004220 con 0x7fdc040ff590 2023-09-10T00:37:59.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.244+0000 7fdbf6ffd700 1 -- 172.21.15.44:0/1859013769 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbf8007d30 con 0x7fdc040ff590 2023-09-10T00:37:59.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.244+0000 7fdc0bd78700 1 -- 172.21.15.44:0/1859013769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc041a0610 con 0x7fdc040ff590 2023-09-10T00:37:59.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.244+0000 7fdbf6ffd700 1 -- 172.21.15.44:0/1859013769 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8003bf0 con 0x7fdc040ff590 2023-09-10T00:37:59.252 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.244+0000 7fdc0bd78700 1 -- 172.21.15.44:0/1859013769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc041a0b10 con 0x7fdc040ff590 2023-09-10T00:37:59.252 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.245+0000 7fdc0bd78700 1 -- 172.21.15.44:0/1859013769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0410aca0 con 0x7fdc040ff590 2023-09-10T00:37:59.253 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.246+0000 7fdbf6ffd700 1 -- 172.21.15.44:0/1859013769 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fdbf80078a0 con 0x7fdc040ff590 2023-09-10T00:37:59.253 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.247+0000 7fdbf6ffd700 1 --2- 172.21.15.44:0/1859013769 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fdbf00a0640 0x7fdbf00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:37:59.253 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.247+0000 7fdc09313700 1 --2- 172.21.15.44:0/1859013769 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fdbf00a0640 0x7fdbf00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:37:59.253 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.247+0000 7fdbf6ffd700 1 -- 172.21.15.44:0/1859013769 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdbf80cb430 con 0x7fdc040ff590 2023-09-10T00:37:59.253 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.247+0000 7fdc09313700 1 --2- 172.21.15.44:0/1859013769 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fdbf00a0640 0x7fdbf00a2b00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000c010 tx=0x7fdc00005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:37:59.254 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.251+0000 7fdbf6ffd700 1 -- 172.21.15.44:0/1859013769 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fdbf8095360 con 0x7fdc040ff590 2023-09-10T00:37:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:58 smithi131 ceph-mon[103713]: purged_snaps scrub starts 2023-09-10T00:37:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:58 smithi131 ceph-mon[103713]: purged_snaps scrub ok 2023-09-10T00:37:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:58 smithi131 ceph-mon[103713]: osd.11 [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632] boot 2023-09-10T00:37:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:58 smithi131 ceph-mon[103713]: osdmap e78: 12 total, 12 up, 12 in 2023-09-10T00:37:59.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:58 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:37:59.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.451+0000 7fdc0bd78700 1 -- 172.21.15.44:0/1859013769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fdc041a0d90 con 0x7fdc040ff590 2023-09-10T00:37:59.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.451+0000 7fdbf6ffd700 1 -- 172.21.15.44:0/1859013769 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7fdbf8098980 con 0x7fdc040ff590 2023-09-10T00:37:59.452 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:37:59.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.453+0000 7fdc0bd78700 1 -- 172.21.15.44:0/1859013769 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fdbf00a0640 msgr2=0x7fdbf00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:59.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.453+0000 7fdc0bd78700 1 --2- 172.21.15.44:0/1859013769 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fdbf00a0640 0x7fdbf00a2b00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000c010 tx=0x7fdc00005fb0 comp rx=0 tx=0).stop 2023-09-10T00:37:59.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.453+0000 7fdc0bd78700 1 -- 172.21.15.44:0/1859013769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc040ff590 msgr2=0x7fdc04103520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:37:59.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.453+0000 7fdc0bd78700 1 --2- 172.21.15.44:0/1859013769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc040ff590 0x7fdc04103520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800c970 tx=0x7fdbf800cd30 comp rx=0 tx=0).stop 2023-09-10T00:37:59.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.453+0000 7fdc0bd78700 1 -- 172.21.15.44:0/1859013769 shutdown_connections 2023-09-10T00:37:59.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.453+0000 7fdc0bd78700 1 --2- 172.21.15.44:0/1859013769 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fdbf00a0640 0x7fdbf00a2b00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:59.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.453+0000 7fdc0bd78700 1 --2- 172.21.15.44:0/1859013769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc04101130 0x7fdc041020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:59.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.453+0000 7fdc0bd78700 1 --2- 172.21.15.44:0/1859013769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc040ff590 0x7fdc04103520 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:59.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.453+0000 7fdc0bd78700 1 --2- 172.21.15.44:0/1859013769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc04100790 0x7fdc04101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:37:59.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.453+0000 7fdc0bd78700 1 -- 172.21.15.44:0/1859013769 >> 172.21.15.44:0/1859013769 conn(0x7fdc040fab10 msgr2=0x7fdc040680a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:37:59.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.454+0000 7fdc0bd78700 1 -- 172.21.15.44:0/1859013769 shutdown_connections 2023-09-10T00:37:59.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:37:59.454+0000 7fdc0bd78700 1 -- 172.21.15.44:0/1859013769 wait complete. 2023-09-10T00:37:59.905 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1694306277,"num_in_osds":12,"osd_in_since":1694306259,"num_remapped_pgs":0} 2023-09-10T00:37:59.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd dump --format=json 2023-09-10T00:38:00.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:59 smithi044 ceph-mon[98676]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:38:00.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:59 smithi044 ceph-mon[98676]: osdmap e79: 12 total, 12 up, 12 in 2023-09-10T00:38:00.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:37:59 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1859013769' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-10T00:38:00.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:59 smithi099 ceph-mon[102692]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:38:00.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:59 smithi099 ceph-mon[102692]: osdmap e79: 12 total, 12 up, 12 in 2023-09-10T00:38:00.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:37:59 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1859013769' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-10T00:38:00.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:59 smithi131 ceph-mon[103713]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-10T00:38:00.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:59 smithi131 ceph-mon[103713]: osdmap e79: 12 total, 12 up, 12 in 2023-09-10T00:38:00.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:37:59 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1859013769' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-10T00:38:00.466 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:38:01.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.692+0000 7ffb496d6700 1 -- 172.21.15.44:0/1992326887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb440ffef0 msgr2=0x7ffb4410c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:01.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.692+0000 7ffb496d6700 1 --2- 172.21.15.44:0/1992326887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb440ffef0 0x7ffb4410c3b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00b0d0 tx=0x7ffb3c00b3e0 comp rx=0 tx=0).stop 2023-09-10T00:38:01.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.692+0000 7ffb496d6700 1 -- 172.21.15.44:0/1992326887 shutdown_connections 2023-09-10T00:38:01.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.692+0000 7ffb496d6700 1 --2- 172.21.15.44:0/1992326887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb440ffef0 0x7ffb4410c3b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:01.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.692+0000 7ffb496d6700 1 --2- 172.21.15.44:0/1992326887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb441055e0 0x7ffb441059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:01.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.692+0000 7ffb496d6700 1 --2- 172.21.15.44:0/1992326887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb440ff530 0x7ffb440ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:01.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.692+0000 7ffb496d6700 1 -- 172.21.15.44:0/1992326887 >> 172.21.15.44:0/1992326887 conn(0x7ffb440fb000 msgr2=0x7ffb440fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:01.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.693+0000 7ffb496d6700 1 -- 172.21.15.44:0/1992326887 shutdown_connections 2023-09-10T00:38:01.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.693+0000 7ffb496d6700 1 -- 172.21.15.44:0/1992326887 wait complete. 2023-09-10T00:38:01.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.693+0000 7ffb496d6700 1 Processor -- start 2023-09-10T00:38:01.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb496d6700 1 -- start start 2023-09-10T00:38:01.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb496d6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb440ff530 0x7ffb44197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:01.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb496d6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb440ffef0 0x7ffb44197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:01.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb496d6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb441055e0 0x7ffb441a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:01.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb496d6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb441a2b30 con 0x7ffb441055e0 2023-09-10T00:38:01.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb496d6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb441a2c70 con 0x7ffb440ff530 2023-09-10T00:38:01.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb496d6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb441a2db0 con 0x7ffb440ffef0 2023-09-10T00:38:01.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb437fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb441055e0 0x7ffb441a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:01.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb437fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb441055e0 0x7ffb441a25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45920/0 (socket says 172.21.15.44:45920) 2023-09-10T00:38:01.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb437fe700 1 -- 172.21.15.44:0/172282536 learned_addr learned my addr 172.21.15.44:0/172282536 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:01.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb42ffd700 1 --2- 172.21.15.44:0/172282536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb440ff530 0x7ffb44197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:01.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb427fc700 1 --2- 172.21.15.44:0/172282536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb440ffef0 0x7ffb44197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:01.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb437fe700 1 -- 172.21.15.44:0/172282536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb440ffef0 msgr2=0x7ffb44197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:01.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb437fe700 1 --2- 172.21.15.44:0/172282536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb440ffef0 0x7ffb44197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:01.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb437fe700 1 -- 172.21.15.44:0/172282536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb440ff530 msgr2=0x7ffb44197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:01.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb437fe700 1 --2- 172.21.15.44:0/172282536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb440ff530 0x7ffb44197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:01.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.694+0000 7ffb437fe700 1 -- 172.21.15.44:0/172282536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c009d00 con 0x7ffb441055e0 2023-09-10T00:38:01.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.695+0000 7ffb437fe700 1 --2- 172.21.15.44:0/172282536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb441055e0 0x7ffb441a25f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0093b0 tx=0x7ffb3c0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:01.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.695+0000 7ffb2bfff700 1 -- 172.21.15.44:0/172282536 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c00e040 con 0x7ffb441055e0 2023-09-10T00:38:01.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.695+0000 7ffb2bfff700 1 -- 172.21.15.44:0/172282536 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb3c003e10 con 0x7ffb441055e0 2023-09-10T00:38:01.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.695+0000 7ffb496d6700 1 -- 172.21.15.44:0/172282536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb441a2fb0 con 0x7ffb441055e0 2023-09-10T00:38:01.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.695+0000 7ffb2bfff700 1 -- 172.21.15.44:0/172282536 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c01b890 con 0x7ffb441055e0 2023-09-10T00:38:01.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.695+0000 7ffb496d6700 1 -- 172.21.15.44:0/172282536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb441a33c0 con 0x7ffb441055e0 2023-09-10T00:38:01.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.696+0000 7ffb496d6700 1 -- 172.21.15.44:0/172282536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4404e300 con 0x7ffb441055e0 2023-09-10T00:38:01.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.697+0000 7ffb2bfff700 1 -- 172.21.15.44:0/172282536 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ffb3c024410 con 0x7ffb441055e0 2023-09-10T00:38:01.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.697+0000 7ffb2bfff700 1 --2- 172.21.15.44:0/172282536 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb2c0a06c0 0x7ffb2c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:01.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.698+0000 7ffb42ffd700 1 --2- 172.21.15.44:0/172282536 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb2c0a06c0 0x7ffb2c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:01.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.698+0000 7ffb2bfff700 1 -- 172.21.15.44:0/172282536 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ffb3c0ccd40 con 0x7ffb441055e0 2023-09-10T00:38:01.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.698+0000 7ffb42ffd700 1 --2- 172.21.15.44:0/172282536 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb2c0a06c0 0x7ffb2c0a2b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffb34005950 tx=0x7ffb3400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:01.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.702+0000 7ffb2bfff700 1 -- 172.21.15.44:0/172282536 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ffb3c096d00 con 0x7ffb441055e0 2023-09-10T00:38:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.902+0000 7ffb496d6700 1 -- 172.21.15.44:0/172282536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb440690b0 con 0x7ffb441055e0 2023-09-10T00:38:01.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.903+0000 7ffb2bfff700 1 -- 172.21.15.44:0/172282536 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18332 (secure 0 0 0) 0x7ffb3c09a320 con 0x7ffb441055e0 2023-09-10T00:38:01.921 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:01.921 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":79,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:37:58.795397+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:38:01.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.905+0000 7ffb496d6700 1 -- 172.21.15.44:0/172282536 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb2c0a06c0 msgr2=0x7ffb2c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:01.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.905+0000 7ffb496d6700 1 --2- 172.21.15.44:0/172282536 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb2c0a06c0 0x7ffb2c0a2b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffb34005950 tx=0x7ffb3400b410 comp rx=0 tx=0).stop 2023-09-10T00:38:01.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.905+0000 7ffb496d6700 1 -- 172.21.15.44:0/172282536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb441055e0 msgr2=0x7ffb441a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:01.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.905+0000 7ffb496d6700 1 --2- 172.21.15.44:0/172282536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb441055e0 0x7ffb441a25f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0093b0 tx=0x7ffb3c0093e0 comp rx=0 tx=0).stop 2023-09-10T00:38:01.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.905+0000 7ffb496d6700 1 -- 172.21.15.44:0/172282536 shutdown_connections 2023-09-10T00:38:01.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.905+0000 7ffb496d6700 1 --2- 172.21.15.44:0/172282536 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb2c0a06c0 0x7ffb2c0a2b80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:01.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.905+0000 7ffb496d6700 1 --2- 172.21.15.44:0/172282536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb441055e0 0x7ffb441a25f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:01.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.905+0000 7ffb496d6700 1 --2- 172.21.15.44:0/172282536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb440ff530 0x7ffb44197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:01.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.905+0000 7ffb496d6700 1 --2- 172.21.15.44:0/172282536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb440ffef0 0x7ffb44197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:01.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.905+0000 7ffb496d6700 1 -- 172.21.15.44:0/172282536 >> 172.21.15.44:0/172282536 conn(0x7ffb440fb000 msgr2=0x7ffb440fc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:01.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.906+0000 7ffb496d6700 1 -- 172.21.15.44:0/172282536 shutdown_connections 2023-09-10T00:38:01.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:01.906+0000 7ffb496d6700 1 -- 172.21.15.44:0/172282536 wait complete. 2023-09-10T00:38:02.071 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:01 smithi044 ceph-mon[98676]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:02.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:01 smithi099 ceph-mon[102692]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:02.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:01 smithi131 ceph-mon[103713]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:02.327 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-10T00:34:44.456741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-10T00:38:02.328 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd pool get .mgr pg_num 2023-09-10T00:38:02.893 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:38:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:02 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/172282536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:03.134 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:02 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/172282536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:02 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/172282536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:04.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:03 smithi044 ceph-mon[98676]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:04.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.095+0000 7f830359d700 1 -- 172.21.15.44:0/4127592715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0ff530 msgr2=0x7f82fc10d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:04.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.095+0000 7f830359d700 1 --2- 172.21.15.44:0/4127592715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0ff530 0x7f82fc10d4e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f82f400b0a0 tx=0x7f82f400b3b0 comp rx=0 tx=0).stop 2023-09-10T00:38:04.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.095+0000 7f830359d700 1 -- 172.21.15.44:0/4127592715 shutdown_connections 2023-09-10T00:38:04.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.095+0000 7f830359d700 1 --2- 172.21.15.44:0/4127592715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0ff530 0x7f82fc10d4e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:04.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.095+0000 7f830359d700 1 --2- 172.21.15.44:0/4127592715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82fc106610 0x7f82fc1069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:04.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.095+0000 7f830359d700 1 --2- 172.21.15.44:0/4127592715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc0feb70 0x7f82fc0feff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:04.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.095+0000 7f830359d700 1 -- 172.21.15.44:0/4127592715 >> 172.21.15.44:0/4127592715 conn(0x7f82fc075a70 msgr2=0x7f82fc0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:04.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.095+0000 7f830359d700 1 -- 172.21.15.44:0/4127592715 shutdown_connections 2023-09-10T00:38:04.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.095+0000 7f830359d700 1 -- 172.21.15.44:0/4127592715 wait complete. 2023-09-10T00:38:04.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.095+0000 7f830359d700 1 Processor -- start 2023-09-10T00:38:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f830359d700 1 -- start start 2023-09-10T00:38:04.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f830359d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0feb70 0x7f82fc0ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:04.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f830359d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82fc0ff530 0x7f82fc100290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:04.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f830359d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc106610 0x7f82fc1007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:04.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f830359d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc1a4bd0 con 0x7f82fc0feb70 2023-09-10T00:38:04.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f830359d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc1a4d40 con 0x7f82fc0ff530 2023-09-10T00:38:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f830359d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc1a4f00 con 0x7f82fc106610 2023-09-10T00:38:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f8301339700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0feb70 0x7f82fc0ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f8301339700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0feb70 0x7f82fc0ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45928/0 (socket says 172.21.15.44:45928) 2023-09-10T00:38:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f8301339700 1 -- 172.21.15.44:0/612087968 learned_addr learned my addr 172.21.15.44:0/612087968 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f8300b38700 1 --2- 172.21.15.44:0/612087968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82fc0ff530 0x7f82fc100290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f8301339700 1 -- 172.21.15.44:0/612087968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc106610 msgr2=0x7f82fc1007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f8301339700 1 --2- 172.21.15.44:0/612087968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc106610 0x7f82fc1007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f8301339700 1 -- 172.21.15.44:0/612087968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82fc0ff530 msgr2=0x7f82fc100290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f8301339700 1 --2- 172.21.15.44:0/612087968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82fc0ff530 0x7f82fc100290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f8301339700 1 -- 172.21.15.44:0/612087968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f4009d00 con 0x7f82fc0feb70 2023-09-10T00:38:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f8301339700 1 --2- 172.21.15.44:0/612087968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0feb70 0x7f82fc0ffd50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f82f800c920 tx=0x7f82f800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f82f27fc700 1 -- 172.21.15.44:0/612087968 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8007b00 con 0x7f82fc0feb70 2023-09-10T00:38:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f82f27fc700 1 -- 172.21.15.44:0/612087968 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82f8004cc0 con 0x7f82fc0feb70 2023-09-10T00:38:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.096+0000 7f830359d700 1 -- 172.21.15.44:0/612087968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82fc1a51e0 con 0x7f82fc0feb70 2023-09-10T00:38:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.097+0000 7f82f27fc700 1 -- 172.21.15.44:0/612087968 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8005590 con 0x7f82fc0feb70 2023-09-10T00:38:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.097+0000 7f830359d700 1 -- 172.21.15.44:0/612087968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc1a5730 con 0x7f82fc0feb70 2023-09-10T00:38:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.098+0000 7f830359d700 1 -- 172.21.15.44:0/612087968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82fc04e300 con 0x7f82fc0feb70 2023-09-10T00:38:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.099+0000 7f82f27fc700 1 -- 172.21.15.44:0/612087968 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f82f8004750 con 0x7f82fc0feb70 2023-09-10T00:38:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.099+0000 7f82f27fc700 1 --2- 172.21.15.44:0/612087968 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82e80a06c0 0x7f82e80a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.100+0000 7f82f27fc700 1 -- 172.21.15.44:0/612087968 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f82f80cb600 con 0x7f82fc0feb70 2023-09-10T00:38:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.100+0000 7f8300b38700 1 --2- 172.21.15.44:0/612087968 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82e80a06c0 0x7f82e80a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.100+0000 7f8300b38700 1 --2- 172.21.15.44:0/612087968 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82e80a06c0 0x7f82e80a2b80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f82ec005fd0 tx=0x7f82ec00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.103+0000 7f82f27fc700 1 -- 172.21.15.44:0/612087968 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f82f8095650 con 0x7f82fc0feb70 2023-09-10T00:38:04.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:03 smithi099 ceph-mon[102692]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:04.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:03 smithi131 ceph-mon[103713]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:04.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.301+0000 7f830359d700 1 -- 172.21.15.44:0/612087968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82fc0008d0 con 0x7f82fc0feb70 2023-09-10T00:38:04.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.302+0000 7f82f27fc700 1 -- 172.21.15.44:0/612087968 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82f801d020 con 0x7f82fc0feb70 2023-09-10T00:38:04.303 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2023-09-10T00:38:04.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.304+0000 7f830359d700 1 -- 172.21.15.44:0/612087968 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82e80a06c0 msgr2=0x7f82e80a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:04.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.304+0000 7f830359d700 1 --2- 172.21.15.44:0/612087968 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82e80a06c0 0x7f82e80a2b80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f82ec005fd0 tx=0x7f82ec00a560 comp rx=0 tx=0).stop 2023-09-10T00:38:04.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.304+0000 7f830359d700 1 -- 172.21.15.44:0/612087968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0feb70 msgr2=0x7f82fc0ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:04.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.304+0000 7f830359d700 1 --2- 172.21.15.44:0/612087968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0feb70 0x7f82fc0ffd50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f82f800c920 tx=0x7f82f800cce0 comp rx=0 tx=0).stop 2023-09-10T00:38:04.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.304+0000 7f830359d700 1 -- 172.21.15.44:0/612087968 shutdown_connections 2023-09-10T00:38:04.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.304+0000 7f830359d700 1 --2- 172.21.15.44:0/612087968 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82e80a06c0 0x7f82e80a2b80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:04.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.304+0000 7f830359d700 1 --2- 172.21.15.44:0/612087968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc106610 0x7f82fc1007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:04.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.304+0000 7f830359d700 1 --2- 172.21.15.44:0/612087968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0feb70 0x7f82fc0ffd50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:04.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.304+0000 7f830359d700 1 --2- 172.21.15.44:0/612087968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82fc0ff530 0x7f82fc100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:04.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.304+0000 7f830359d700 1 -- 172.21.15.44:0/612087968 >> 172.21.15.44:0/612087968 conn(0x7f82fc075a70 msgr2=0x7f82fc103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:04.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.304+0000 7f830359d700 1 -- 172.21.15.44:0/612087968 shutdown_connections 2023-09-10T00:38:04.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:04.304+0000 7f830359d700 1 -- 172.21.15.44:0/612087968 wait complete. 2023-09-10T00:38:04.655 INFO:tasks.cephadm:Adding mds.a on smithi044 2023-09-10T00:38:04.655 INFO:tasks.cephadm:Adding mds.d on smithi044 2023-09-10T00:38:04.656 INFO:tasks.cephadm:Adding mds.g on smithi044 2023-09-10T00:38:04.656 INFO:tasks.cephadm:Adding mds.j on smithi044 2023-09-10T00:38:04.656 INFO:tasks.cephadm:Adding mds.b on smithi099 2023-09-10T00:38:04.656 INFO:tasks.cephadm:Adding mds.e on smithi099 2023-09-10T00:38:04.657 INFO:tasks.cephadm:Adding mds.h on smithi099 2023-09-10T00:38:04.657 INFO:tasks.cephadm:Adding mds.k on smithi099 2023-09-10T00:38:04.657 INFO:tasks.cephadm:Adding mds.c on smithi131 2023-09-10T00:38:04.657 INFO:tasks.cephadm:Adding mds.f on smithi131 2023-09-10T00:38:04.658 INFO:tasks.cephadm:Adding mds.i on smithi131 2023-09-10T00:38:04.658 INFO:tasks.cephadm:Adding mds.l on smithi131 2023-09-10T00:38:04.658 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph orch apply mds all '12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l' 2023-09-10T00:38:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:04 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/612087968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:38:05.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:04 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/612087968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:38:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:04 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/612087968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:38:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-10T00:38:06.062 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:05 smithi131 ceph-mon[103713]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:06.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:05 smithi099 ceph-mon[102692]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:06.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:05 smithi044 ceph-mon[98676]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:06.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.522+0000 7f472dd6c700 1 -- 172.21.15.131:0/977819431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4728101130 msgr2=0x7f47281079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:06.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.522+0000 7f472dd6c700 1 --2- 172.21.15.131:0/977819431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4728101130 0x7f47281079e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f472000b0a0 tx=0x7f472000b3b0 comp rx=0 tx=0).stop 2023-09-10T00:38:06.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.522+0000 7f472dd6c700 1 -- 172.21.15.131:0/977819431 shutdown_connections 2023-09-10T00:38:06.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.522+0000 7f472dd6c700 1 --2- 172.21.15.131:0/977819431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4728101130 0x7f47281079e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:06.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.522+0000 7f472dd6c700 1 --2- 172.21.15.131:0/977819431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47280ff590 0x7f47280ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:06.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.522+0000 7f472dd6c700 1 --2- 172.21.15.131:0/977819431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4728100790 0x7f4728100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:06.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.522+0000 7f472dd6c700 1 -- 172.21.15.131:0/977819431 >> 172.21.15.131:0/977819431 conn(0x7f47280fab10 msgr2=0x7f47280fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:06.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.522+0000 7f472dd6c700 1 -- 172.21.15.131:0/977819431 shutdown_connections 2023-09-10T00:38:06.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.522+0000 7f472dd6c700 1 -- 172.21.15.131:0/977819431 wait complete. 2023-09-10T00:38:06.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f472dd6c700 1 Processor -- start 2023-09-10T00:38:06.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f472dd6c700 1 -- start start 2023-09-10T00:38:06.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f472dd6c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47280ff590 0x7f47280694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:06.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f472dd6c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4728100790 0x7f472819b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:06.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f472dd6c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4728101130 0x7f472819d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:06.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f472dd6c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472819df20 con 0x7f4728101130 2023-09-10T00:38:06.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f472dd6c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472819e090 con 0x7f4728100790 2023-09-10T00:38:06.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f472dd6c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472819e250 con 0x7f47280ff590 2023-09-10T00:38:06.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f47277fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47280ff590 0x7f47280694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:06.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f47277fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47280ff590 0x7f47280694d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53360/0 (socket says 172.21.15.131:53360) 2023-09-10T00:38:06.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f47277fe700 1 -- 172.21.15.131:0/1354728371 learned_addr learned my addr 172.21.15.131:0/1354728371 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-10T00:38:06.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f4726ffd700 1 --2- 172.21.15.131:0/1354728371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4728100790 0x7f472819b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:06.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f47277fe700 1 -- 172.21.15.131:0/1354728371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4728100790 msgr2=0x7f472819b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:06.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f4727fff700 1 --2- 172.21.15.131:0/1354728371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4728101130 0x7f472819d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:06.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f47277fe700 1 --2- 172.21.15.131:0/1354728371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4728100790 0x7f472819b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:06.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f47277fe700 1 -- 172.21.15.131:0/1354728371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4728101130 msgr2=0x7f472819d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:06.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f47277fe700 1 --2- 172.21.15.131:0/1354728371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4728101130 0x7f472819d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:06.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.523+0000 7f47277fe700 1 -- 172.21.15.131:0/1354728371 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4720009d00 con 0x7f47280ff590 2023-09-10T00:38:06.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.524+0000 7f47277fe700 1 --2- 172.21.15.131:0/1354728371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47280ff590 0x7f47280694d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f471c00b500 tx=0x7f471c00b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:06.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.524+0000 7f4724ff9700 1 -- 172.21.15.131:0/1354728371 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c011830 con 0x7f47280ff590 2023-09-10T00:38:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.524+0000 7f4724ff9700 1 -- 172.21.15.131:0/1354728371 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f471c011e20 con 0x7f47280ff590 2023-09-10T00:38:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.524+0000 7f472dd6c700 1 -- 172.21.15.131:0/1354728371 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472819e530 con 0x7f47280ff590 2023-09-10T00:38:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.524+0000 7f4724ff9700 1 -- 172.21.15.131:0/1354728371 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c010dd0 con 0x7f47280ff590 2023-09-10T00:38:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.524+0000 7f472dd6c700 1 -- 172.21.15.131:0/1354728371 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472819eb00 con 0x7f47280ff590 2023-09-10T00:38:06.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.525+0000 7f472dd6c700 1 -- 172.21.15.131:0/1354728371 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472810ad40 con 0x7f47280ff590 2023-09-10T00:38:06.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.526+0000 7f4724ff9700 1 -- 172.21.15.131:0/1354728371 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f471c011990 con 0x7f47280ff590 2023-09-10T00:38:06.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.526+0000 7f4724ff9700 1 --2- 172.21.15.131:0/1354728371 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f47100a06c0 0x7f47100a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:06.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.527+0000 7f4724ff9700 1 -- 172.21.15.131:0/1354728371 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f471c0cb470 con 0x7f47280ff590 2023-09-10T00:38:06.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.527+0000 7f4726ffd700 1 --2- 172.21.15.131:0/1354728371 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f47100a06c0 0x7f47100a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:06.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.527+0000 7f4726ffd700 1 --2- 172.21.15.131:0/1354728371 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f47100a06c0 0x7f47100a2b80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4728069a30 tx=0x7f471800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:06.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.530+0000 7f4724ff9700 1 -- 172.21.15.131:0/1354728371 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f471c095440 con 0x7f47280ff590 2023-09-10T00:38:06.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.726+0000 7f472dd6c700 1 -- 172.21.15.131:0/1354728371 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f4728060b40 con 0x7f47100a06c0 2023-09-10T00:38:06.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.735+0000 7f4724ff9700 1 -- 172.21.15.131:0/1354728371 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f4728060b40 con 0x7f47100a06c0 2023-09-10T00:38:06.736 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled mds.all update... 2023-09-10T00:38:06.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.738+0000 7f472dd6c700 1 -- 172.21.15.131:0/1354728371 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f47100a06c0 msgr2=0x7f47100a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:06.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.738+0000 7f472dd6c700 1 --2- 172.21.15.131:0/1354728371 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f47100a06c0 0x7f47100a2b80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4728069a30 tx=0x7f471800a380 comp rx=0 tx=0).stop 2023-09-10T00:38:06.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.738+0000 7f472dd6c700 1 -- 172.21.15.131:0/1354728371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47280ff590 msgr2=0x7f47280694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:06.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.738+0000 7f472dd6c700 1 --2- 172.21.15.131:0/1354728371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47280ff590 0x7f47280694d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f471c00b500 tx=0x7f471c00b810 comp rx=0 tx=0).stop 2023-09-10T00:38:06.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.738+0000 7f472dd6c700 1 -- 172.21.15.131:0/1354728371 shutdown_connections 2023-09-10T00:38:06.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.738+0000 7f472dd6c700 1 --2- 172.21.15.131:0/1354728371 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f47100a06c0 0x7f47100a2b80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:06.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.738+0000 7f472dd6c700 1 --2- 172.21.15.131:0/1354728371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4728101130 0x7f472819d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:06.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.738+0000 7f472dd6c700 1 --2- 172.21.15.131:0/1354728371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47280ff590 0x7f47280694d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:06.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.738+0000 7f472dd6c700 1 --2- 172.21.15.131:0/1354728371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4728100790 0x7f472819b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:06.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.738+0000 7f472dd6c700 1 -- 172.21.15.131:0/1354728371 >> 172.21.15.131:0/1354728371 conn(0x7f47280fab10 msgr2=0x7f47280fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:06.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.738+0000 7f472dd6c700 1 -- 172.21.15.131:0/1354728371 shutdown_connections 2023-09-10T00:38:06.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-10T00:38:06.738+0000 7f472dd6c700 1 -- 172.21.15.131:0/1354728371 wait complete. 2023-09-10T00:38:07.186 DEBUG:teuthology.orchestra.run.smithi044:mds.a> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.a.service 2023-09-10T00:38:07.190 DEBUG:teuthology.orchestra.run.smithi044:mds.d> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service 2023-09-10T00:38:07.228 INFO:journalctl@ceph.mds.a.smithi044.stdout:-- Logs begin at Sun 2023-09-10 00:17:15 UTC. -- 2023-09-10T00:38:07.231 DEBUG:teuthology.orchestra.run.smithi044:mds.g> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.g.service 2023-09-10T00:38:07.261 INFO:journalctl@ceph.mds.d.smithi044.stdout:-- Logs begin at Sun 2023-09-10 00:17:15 UTC. -- 2023-09-10T00:38:07.263 DEBUG:teuthology.orchestra.run.smithi044:mds.j> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.j.service 2023-09-10T00:38:07.292 INFO:journalctl@ceph.mds.g.smithi044.stdout:-- Logs begin at Sun 2023-09-10 00:17:15 UTC. -- 2023-09-10T00:38:07.295 DEBUG:teuthology.orchestra.run.smithi099:mds.b> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.b.service 2023-09-10T00:38:07.298 DEBUG:teuthology.orchestra.run.smithi099:mds.e> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.e.service 2023-09-10T00:38:07.323 INFO:journalctl@ceph.mds.j.smithi044.stdout:-- Logs begin at Sun 2023-09-10 00:17:15 UTC. -- 2023-09-10T00:38:07.335 INFO:journalctl@ceph.mds.b.smithi099.stdout:-- Logs begin at Sun 2023-09-10 00:17:22 UTC. -- 2023-09-10T00:38:07.337 DEBUG:teuthology.orchestra.run.smithi099:mds.h> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.h.service 2023-09-10T00:38:07.366 INFO:journalctl@ceph.mds.e.smithi099.stdout:-- Logs begin at Sun 2023-09-10 00:17:22 UTC. -- 2023-09-10T00:38:07.367 DEBUG:teuthology.orchestra.run.smithi099:mds.k> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.k.service 2023-09-10T00:38:07.394 INFO:journalctl@ceph.mds.h.smithi099.stdout:-- Logs begin at Sun 2023-09-10 00:17:22 UTC. -- 2023-09-10T00:38:07.395 DEBUG:teuthology.orchestra.run.smithi131:mds.c> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service 2023-09-10T00:38:07.397 DEBUG:teuthology.orchestra.run.smithi131:mds.f> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.f.service 2023-09-10T00:38:07.420 INFO:journalctl@ceph.mds.k.smithi099.stdout:-- Logs begin at Sun 2023-09-10 00:17:22 UTC. -- 2023-09-10T00:38:07.425 INFO:journalctl@ceph.mds.c.smithi131.stdout:-- Logs begin at Sun 2023-09-10 00:17:05 UTC. -- 2023-09-10T00:38:07.426 DEBUG:teuthology.orchestra.run.smithi131:mds.i> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.i.service 2023-09-10T00:38:07.452 INFO:journalctl@ceph.mds.f.smithi131.stdout:-- Logs begin at Sun 2023-09-10 00:17:05 UTC. -- 2023-09-10T00:38:07.453 DEBUG:teuthology.orchestra.run.smithi131:mds.l> sudo journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.l.service 2023-09-10T00:38:07.478 INFO:journalctl@ceph.mds.i.smithi131.stdout:-- Logs begin at Sun 2023-09-10 00:17:05 UTC. -- 2023-09-10T00:38:07.479 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-09-10T00:38:07.480 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:38:07.508 INFO:journalctl@ceph.mds.l.smithi131.stdout:-- Logs begin at Sun 2023-09-10 00:17:05 UTC. -- 2023-09-10T00:38:07.632 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.632+0000 7f4e31b03700 1 -- 172.21.15.44:0/4207560147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c1006d0 msgr2=0x7f4e2c100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.632+0000 7f4e31b03700 1 --2- 172.21.15.44:0/4207560147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c1006d0 0x7f4e2c100b30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4e200099c0 tx=0x7f4e20009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.632+0000 7f4e31b03700 1 -- 172.21.15.44:0/4207560147 shutdown_connections 2023-09-10T00:38:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.632+0000 7f4e31b03700 1 --2- 172.21.15.44:0/4207560147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e2c101070 0x7f4e2c107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.632+0000 7f4e31b03700 1 --2- 172.21.15.44:0/4207560147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e2c0ff4e0 0x7f4e2c0ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.632+0000 7f4e31b03700 1 --2- 172.21.15.44:0/4207560147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c1006d0 0x7f4e2c100b30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.632+0000 7f4e31b03700 1 -- 172.21.15.44:0/4207560147 >> 172.21.15.44:0/4207560147 conn(0x7f4e2c0faa30 msgr2=0x7f4e2c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.632+0000 7f4e31b03700 1 -- 172.21.15.44:0/4207560147 shutdown_connections 2023-09-10T00:38:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.632+0000 7f4e31b03700 1 -- 172.21.15.44:0/4207560147 wait complete. 2023-09-10T00:38:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.632+0000 7f4e31b03700 1 Processor -- start 2023-09-10T00:38:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.632+0000 7f4e31b03700 1 -- start start 2023-09-10T00:38:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e31b03700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c0ff4e0 0x7f4e2c1950b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e31b03700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e2c1006d0 0x7f4e2c1955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e31b03700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e2c101070 0x7f4e2c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:07.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e31b03700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c19cc30 con 0x7f4e2c0ff4e0 2023-09-10T00:38:07.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e31b03700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c19cd70 con 0x7f4e2c101070 2023-09-10T00:38:07.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e31b03700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c19cf00 con 0x7f4e2c1006d0 2023-09-10T00:38:07.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e2b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c0ff4e0 0x7f4e2c1950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:07.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e2b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c0ff4e0 0x7f4e2c1950b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45938/0 (socket says 172.21.15.44:45938) 2023-09-10T00:38:07.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e2b7fe700 1 -- 172.21.15.44:0/3187024199 learned_addr learned my addr 172.21.15.44:0/3187024199 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:07.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e2affd700 1 --2- 172.21.15.44:0/3187024199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e2c1006d0 0x7f4e2c1955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:07.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e2bfff700 1 --2- 172.21.15.44:0/3187024199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e2c101070 0x7f4e2c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:07.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e2b7fe700 1 -- 172.21.15.44:0/3187024199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e2c1006d0 msgr2=0x7f4e2c1955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:07.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e2b7fe700 1 --2- 172.21.15.44:0/3187024199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e2c1006d0 0x7f4e2c1955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e2b7fe700 1 -- 172.21.15.44:0/3187024199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e2c101070 msgr2=0x7f4e2c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:07.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e2b7fe700 1 --2- 172.21.15.44:0/3187024199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e2c101070 0x7f4e2c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e2b7fe700 1 -- 172.21.15.44:0/3187024199 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e200095f0 con 0x7f4e2c0ff4e0 2023-09-10T00:38:07.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e2b7fe700 1 --2- 172.21.15.44:0/3187024199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c0ff4e0 0x7f4e2c1950b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00e970 tx=0x7f4e1c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:07.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.633+0000 7f4e28ff9700 1 -- 172.21.15.44:0/3187024199 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c009790 con 0x7f4e2c0ff4e0 2023-09-10T00:38:07.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.634+0000 7f4e31b03700 1 -- 172.21.15.44:0/3187024199 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2c19d1e0 con 0x7f4e2c0ff4e0 2023-09-10T00:38:07.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.634+0000 7f4e28ff9700 1 -- 172.21.15.44:0/3187024199 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e1c00c7e0 con 0x7f4e2c0ff4e0 2023-09-10T00:38:07.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.634+0000 7f4e28ff9700 1 -- 172.21.15.44:0/3187024199 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c003bd0 con 0x7f4e2c0ff4e0 2023-09-10T00:38:07.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.634+0000 7f4e31b03700 1 -- 172.21.15.44:0/3187024199 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2c19d740 con 0x7f4e2c0ff4e0 2023-09-10T00:38:07.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.634+0000 7f4e31b03700 1 -- 172.21.15.44:0/3187024199 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e2c10abb0 con 0x7f4e2c0ff4e0 2023-09-10T00:38:07.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.636+0000 7f4e28ff9700 1 -- 172.21.15.44:0/3187024199 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f4e1c00c950 con 0x7f4e2c0ff4e0 2023-09-10T00:38:07.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.636+0000 7f4e28ff9700 1 --2- 172.21.15.44:0/3187024199 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4e140a0690 0x7f4e140a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:07.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.637+0000 7f4e2affd700 1 --2- 172.21.15.44:0/3187024199 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4e140a0690 0x7f4e140a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:07.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.637+0000 7f4e28ff9700 1 -- 172.21.15.44:0/3187024199 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4e1c014070 con 0x7f4e2c0ff4e0 2023-09-10T00:38:07.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.637+0000 7f4e2affd700 1 --2- 172.21.15.44:0/3187024199 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4e140a0690 0x7f4e140a2b50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009990 tx=0x7f4e20005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.640+0000 7f4e28ff9700 1 -- 172.21.15.44:0/3187024199 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4e1c0959f0 con 0x7f4e2c0ff4e0 2023-09-10T00:38:07.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:07 smithi044 ceph-mon[98676]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:07.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:07 smithi044 ceph-mon[98676]: from='client.24478 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:38:07.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:07 smithi044 ceph-mon[98676]: Saving service mds.all spec with placement smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l;count:12 2023-09-10T00:38:07.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:07 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:07.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:07 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:38:07.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:07 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:07.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:07 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:38:07.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:07 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:07.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:07 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:07.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:07 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:07.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:07 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:07.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:07 smithi044 ceph-mon[98676]: Deploying daemon mds.l on smithi131 2023-09-10T00:38:07.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.844+0000 7f4e31b03700 1 -- 172.21.15.44:0/3187024199 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e2c10adc0 con 0x7f4e2c0ff4e0 2023-09-10T00:38:07.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.845+0000 7f4e28ff9700 1 -- 172.21.15.44:0/3187024199 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18332 (secure 0 0 0) 0x7f4e1c099010 con 0x7f4e2c0ff4e0 2023-09-10T00:38:07.846 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:07.846 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":79,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:37:58.795397+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:38:07.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.847+0000 7f4e31b03700 1 -- 172.21.15.44:0/3187024199 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4e140a0690 msgr2=0x7f4e140a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:07.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.847+0000 7f4e31b03700 1 --2- 172.21.15.44:0/3187024199 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4e140a0690 0x7f4e140a2b50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009990 tx=0x7f4e20005be0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.847+0000 7f4e31b03700 1 -- 172.21.15.44:0/3187024199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c0ff4e0 msgr2=0x7f4e2c1950b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:07.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.847+0000 7f4e31b03700 1 --2- 172.21.15.44:0/3187024199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c0ff4e0 0x7f4e2c1950b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00e970 tx=0x7f4e1c00ec80 comp rx=0 tx=0).stop 2023-09-10T00:38:07.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.848+0000 7f4e31b03700 1 -- 172.21.15.44:0/3187024199 shutdown_connections 2023-09-10T00:38:07.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.848+0000 7f4e31b03700 1 --2- 172.21.15.44:0/3187024199 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4e140a0690 0x7f4e140a2b50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.848+0000 7f4e31b03700 1 --2- 172.21.15.44:0/3187024199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e2c101070 0x7f4e2c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.848+0000 7f4e31b03700 1 --2- 172.21.15.44:0/3187024199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c0ff4e0 0x7f4e2c1950b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.848+0000 7f4e31b03700 1 --2- 172.21.15.44:0/3187024199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e2c1006d0 0x7f4e2c1955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.848+0000 7f4e31b03700 1 -- 172.21.15.44:0/3187024199 >> 172.21.15.44:0/3187024199 conn(0x7f4e2c0faa30 msgr2=0x7f4e2c0fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:07.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.848+0000 7f4e31b03700 1 -- 172.21.15.44:0/3187024199 shutdown_connections 2023-09-10T00:38:07.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.848+0000 7f4e31b03700 1 -- 172.21.15.44:0/3187024199 wait complete. 2023-09-10T00:38:07.861 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-10T00:34:44.456741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-10T00:38:07.862 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-10T00:38:07.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.990+0000 7f76da1e4700 1 -- 172.21.15.44:0/1388754465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76d40ff4f0 msgr2=0x7f76d40ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:07.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.990+0000 7f76da1e4700 1 --2- 172.21.15.44:0/1388754465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76d40ff4f0 0x7f76d40ff910 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f76c40099c0 tx=0x7f76c4009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.990+0000 7f76da1e4700 1 -- 172.21.15.44:0/1388754465 shutdown_connections 2023-09-10T00:38:07.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.990+0000 7f76da1e4700 1 --2- 172.21.15.44:0/1388754465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76d4101080 0x7f76d4107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.990+0000 7f76da1e4700 1 --2- 172.21.15.44:0/1388754465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76d40ff4f0 0x7f76d40ff910 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.990+0000 7f76da1e4700 1 --2- 172.21.15.44:0/1388754465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76d41006e0 0x7f76d4100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.990+0000 7f76da1e4700 1 -- 172.21.15.44:0/1388754465 >> 172.21.15.44:0/1388754465 conn(0x7f76d40faa60 msgr2=0x7f76d40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:07.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.990+0000 7f76da1e4700 1 -- 172.21.15.44:0/1388754465 shutdown_connections 2023-09-10T00:38:07.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.990+0000 7f76da1e4700 1 -- 172.21.15.44:0/1388754465 wait complete. 2023-09-10T00:38:07.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.991+0000 7f76da1e4700 1 Processor -- start 2023-09-10T00:38:07.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.991+0000 7f76da1e4700 1 -- start start 2023-09-10T00:38:07.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.991+0000 7f76da1e4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76d40ff4f0 0x7f76d40682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:07.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76da1e4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76d41006e0 0x7f76d419af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:07.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76da1e4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76d4101080 0x7f76d419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:07.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76da1e4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d419dd30 con 0x7f76d4101080 2023-09-10T00:38:07.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76da1e4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d419dea0 con 0x7f76d41006e0 2023-09-10T00:38:07.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76da1e4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d419e060 con 0x7f76d40ff4f0 2023-09-10T00:38:07.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76d3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76d4101080 0x7f76d419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:07.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76d3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76d4101080 0x7f76d419d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45956/0 (socket says 172.21.15.44:45956) 2023-09-10T00:38:07.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76d3fff700 1 -- 172.21.15.44:0/351015606 learned_addr learned my addr 172.21.15.44:0/351015606 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:07.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76d37fe700 1 --2- 172.21.15.44:0/351015606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76d40ff4f0 0x7f76d40682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:07.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76d2ffd700 1 --2- 172.21.15.44:0/351015606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76d41006e0 0x7f76d419af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:07.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76d3fff700 1 -- 172.21.15.44:0/351015606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76d40ff4f0 msgr2=0x7f76d40682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:07.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76d3fff700 1 --2- 172.21.15.44:0/351015606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76d40ff4f0 0x7f76d40682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76d3fff700 1 -- 172.21.15.44:0/351015606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76d41006e0 msgr2=0x7f76d419af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:07.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76d3fff700 1 --2- 172.21.15.44:0/351015606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76d41006e0 0x7f76d419af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:07.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76d3fff700 1 -- 172.21.15.44:0/351015606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c40095f0 con 0x7f76d4101080 2023-09-10T00:38:07.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76d3fff700 1 --2- 172.21.15.44:0/351015606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76d4101080 0x7f76d419d590 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f76cc007ee0 tx=0x7f76cc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:07.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76d0ff9700 1 -- 172.21.15.44:0/351015606 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc00dd20 con 0x7f76d4101080 2023-09-10T00:38:07.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76d0ff9700 1 -- 172.21.15.44:0/351015606 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76cc00f040 con 0x7f76d4101080 2023-09-10T00:38:07.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76da1e4700 1 -- 172.21.15.44:0/351015606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d419e340 con 0x7f76d4101080 2023-09-10T00:38:07.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76d0ff9700 1 -- 172.21.15.44:0/351015606 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc004750 con 0x7f76d4101080 2023-09-10T00:38:07.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.992+0000 7f76da1e4700 1 -- 172.21.15.44:0/351015606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d419e850 con 0x7f76d4101080 2023-09-10T00:38:08.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.993+0000 7f76da1e4700 1 -- 172.21.15.44:0/351015606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d41064a0 con 0x7f76d4101080 2023-09-10T00:38:08.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.994+0000 7f76d0ff9700 1 -- 172.21.15.44:0/351015606 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f76cc003a00 con 0x7f76d4101080 2023-09-10T00:38:08.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.995+0000 7f76d0ff9700 1 --2- 172.21.15.44:0/351015606 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f76bc0a0640 0x7f76bc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:08.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.995+0000 7f76d0ff9700 1 -- 172.21.15.44:0/351015606 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f76cc0cb740 con 0x7f76d4101080 2023-09-10T00:38:08.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.995+0000 7f76d37fe700 1 --2- 172.21.15.44:0/351015606 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f76bc0a0640 0x7f76bc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:08.001 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:07 smithi131 ceph-mon[103713]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:08.001 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:07 smithi131 ceph-mon[103713]: from='client.24478 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:38:08.001 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:07 smithi131 ceph-mon[103713]: Saving service mds.all spec with placement smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l;count:12 2023-09-10T00:38:08.002 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:07 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:08.002 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:07 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:38:08.002 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:07 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:08.002 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:07 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:38:08.002 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:07 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:08.003 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:07 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:08.003 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:07 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:08.003 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:07 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:08.003 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:07 smithi131 ceph-mon[103713]: Deploying daemon mds.l on smithi131 2023-09-10T00:38:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.995+0000 7f76d37fe700 1 --2- 172.21.15.44:0/351015606 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f76bc0a0640 0x7f76bc0a2b00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f76c4000c00 tx=0x7f76c4009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:07.998+0000 7f76d0ff9700 1 -- 172.21.15.44:0/351015606 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f76cc095710 con 0x7f76d4101080 2023-09-10T00:38:08.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:07 smithi099 ceph-mon[102692]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:08.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:07 smithi099 ceph-mon[102692]: from='client.24478 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:38:08.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:07 smithi099 ceph-mon[102692]: Saving service mds.all spec with placement smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l;count:12 2023-09-10T00:38:08.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:07 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:08.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:07 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:38:08.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:07 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:08.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:07 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:38:08.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:07 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:08.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:07 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:08.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:07 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:08.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:07 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:08.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:07 smithi099 ceph-mon[102692]: Deploying daemon mds.l on smithi131 2023-09-10T00:38:08.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.197+0000 7f76da1e4700 1 -- 172.21.15.44:0/351015606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76d410ada0 con 0x7f76d4101080 2023-09-10T00:38:08.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.198+0000 7f76d0ff9700 1 -- 172.21.15.44:0/351015606 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76cc0048b0 con 0x7f76d4101080 2023-09-10T00:38:08.199 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2023-09-10T00:38:08.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.200+0000 7f76da1e4700 1 -- 172.21.15.44:0/351015606 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f76bc0a0640 msgr2=0x7f76bc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:08.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.200+0000 7f76da1e4700 1 --2- 172.21.15.44:0/351015606 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f76bc0a0640 0x7f76bc0a2b00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f76c4000c00 tx=0x7f76c4009f10 comp rx=0 tx=0).stop 2023-09-10T00:38:08.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.200+0000 7f76da1e4700 1 -- 172.21.15.44:0/351015606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76d4101080 msgr2=0x7f76d419d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:08.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.200+0000 7f76da1e4700 1 --2- 172.21.15.44:0/351015606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76d4101080 0x7f76d419d590 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f76cc007ee0 tx=0x7f76cc00d3b0 comp rx=0 tx=0).stop 2023-09-10T00:38:08.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.200+0000 7f76da1e4700 1 -- 172.21.15.44:0/351015606 shutdown_connections 2023-09-10T00:38:08.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.200+0000 7f76da1e4700 1 --2- 172.21.15.44:0/351015606 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f76bc0a0640 0x7f76bc0a2b00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:08.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.200+0000 7f76da1e4700 1 --2- 172.21.15.44:0/351015606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76d4101080 0x7f76d419d590 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:08.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.200+0000 7f76da1e4700 1 --2- 172.21.15.44:0/351015606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76d40ff4f0 0x7f76d40682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:08.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.200+0000 7f76da1e4700 1 --2- 172.21.15.44:0/351015606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76d41006e0 0x7f76d419af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:08.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.200+0000 7f76da1e4700 1 -- 172.21.15.44:0/351015606 >> 172.21.15.44:0/351015606 conn(0x7f76d40faa60 msgr2=0x7f76d40fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:08.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.200+0000 7f76da1e4700 1 -- 172.21.15.44:0/351015606 shutdown_connections 2023-09-10T00:38:08.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.200+0000 7f76da1e4700 1 -- 172.21.15.44:0/351015606 wait complete. 2023-09-10T00:38:08.213 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:38:08.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.339+0000 7f454ba08700 1 -- 172.21.15.44:0/1536458101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45440ff4e0 msgr2=0x7f45440ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:08.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.339+0000 7f454ba08700 1 --2- 172.21.15.44:0/1536458101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45440ff4e0 0x7f45440ff900 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f45340099c0 tx=0x7f4534009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:08.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.339+0000 7f454ba08700 1 -- 172.21.15.44:0/1536458101 shutdown_connections 2023-09-10T00:38:08.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.339+0000 7f454ba08700 1 --2- 172.21.15.44:0/1536458101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4544101070 0x7f4544107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:08.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.339+0000 7f454ba08700 1 --2- 172.21.15.44:0/1536458101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45440ff4e0 0x7f45440ff900 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:08.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.339+0000 7f454ba08700 1 --2- 172.21.15.44:0/1536458101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45441006d0 0x7f4544100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:08.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.339+0000 7f454ba08700 1 -- 172.21.15.44:0/1536458101 >> 172.21.15.44:0/1536458101 conn(0x7f45440faa30 msgr2=0x7f45440fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:08.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.339+0000 7f454ba08700 1 -- 172.21.15.44:0/1536458101 shutdown_connections 2023-09-10T00:38:08.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.339+0000 7f454ba08700 1 -- 172.21.15.44:0/1536458101 wait complete. 2023-09-10T00:38:08.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f454ba08700 1 Processor -- start 2023-09-10T00:38:08.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f454ba08700 1 -- start start 2023-09-10T00:38:08.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f454ba08700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45440ff4e0 0x7f4544195130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:08.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f454ba08700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45441006d0 0x7f4544195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:08.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f454ba08700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4544101070 0x7f454419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:08.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f454ba08700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454419ccb0 con 0x7f4544101070 2023-09-10T00:38:08.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f454ba08700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454419cdf0 con 0x7f45440ff4e0 2023-09-10T00:38:08.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f454ba08700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454419cf80 con 0x7f45441006d0 2023-09-10T00:38:08.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f4549fa5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4544101070 0x7f454419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:08.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f4549fa5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4544101070 0x7f454419c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45970/0 (socket says 172.21.15.44:45970) 2023-09-10T00:38:08.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f4549fa5700 1 -- 172.21.15.44:0/2386271098 learned_addr learned my addr 172.21.15.44:0/2386271098 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:08.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f4548fa3700 1 --2- 172.21.15.44:0/2386271098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45441006d0 0x7f4544195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:08.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f45497a4700 1 --2- 172.21.15.44:0/2386271098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45440ff4e0 0x7f4544195130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:08.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f4549fa5700 1 -- 172.21.15.44:0/2386271098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45441006d0 msgr2=0x7f4544195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:08.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f4549fa5700 1 --2- 172.21.15.44:0/2386271098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45441006d0 0x7f4544195670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:08.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f4549fa5700 1 -- 172.21.15.44:0/2386271098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45440ff4e0 msgr2=0x7f4544195130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:08.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f4549fa5700 1 --2- 172.21.15.44:0/2386271098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45440ff4e0 0x7f4544195130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:08.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f4549fa5700 1 -- 172.21.15.44:0/2386271098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340095f0 con 0x7f4544101070 2023-09-10T00:38:08.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.340+0000 7f45497a4700 1 --2- 172.21.15.44:0/2386271098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45440ff4e0 0x7f4544195130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:38:08.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.341+0000 7f4549fa5700 1 --2- 172.21.15.44:0/2386271098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4544101070 0x7f454419c6e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f453c007f10 tx=0x7f453c00f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:08.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.341+0000 7f453a7fc700 1 -- 172.21.15.44:0/2386271098 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c00fde0 con 0x7f4544101070 2023-09-10T00:38:08.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.341+0000 7f453a7fc700 1 -- 172.21.15.44:0/2386271098 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f453c004bb0 con 0x7f4544101070 2023-09-10T00:38:08.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.341+0000 7f454ba08700 1 -- 172.21.15.44:0/2386271098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45440677c0 con 0x7f4544101070 2023-09-10T00:38:08.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.341+0000 7f453a7fc700 1 -- 172.21.15.44:0/2386271098 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c005020 con 0x7f4544101070 2023-09-10T00:38:08.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.341+0000 7f454ba08700 1 -- 172.21.15.44:0/2386271098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4544067d20 con 0x7f4544101070 2023-09-10T00:38:08.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.342+0000 7f454ba08700 1 -- 172.21.15.44:0/2386271098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454410abf0 con 0x7f4544101070 2023-09-10T00:38:08.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.343+0000 7f453a7fc700 1 -- 172.21.15.44:0/2386271098 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f453c004d20 con 0x7f4544101070 2023-09-10T00:38:08.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.343+0000 7f453a7fc700 1 --2- 172.21.15.44:0/2386271098 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f45300a0710 0x7f45300a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:08.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.344+0000 7f453a7fc700 1 -- 172.21.15.44:0/2386271098 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f453c0cdd20 con 0x7f4544101070 2023-09-10T00:38:08.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.344+0000 7f45497a4700 1 --2- 172.21.15.44:0/2386271098 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f45300a0710 0x7f45300a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:08.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.344+0000 7f45497a4700 1 --2- 172.21.15.44:0/2386271098 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f45300a0710 0x7f45300a2bd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4534009990 tx=0x7f453401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:08.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.348+0000 7f453a7fc700 1 -- 172.21.15.44:0/2386271098 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f453c097c50 con 0x7f4544101070 2023-09-10T00:38:08.545 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 10 00:38:08 smithi131 systemd[1]: Starting Ceph mds.l for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:38:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.595+0000 7f454ba08700 1 -- 172.21.15.44:0/2386271098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f454410adb0 con 0x7f4544101070 2023-09-10T00:38:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.595+0000 7f453a7fc700 1 -- 172.21.15.44:0/2386271098 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f453c020230 con 0x7f4544101070 2023-09-10T00:38:08.596 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:08.596 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-09-10T00:38:08.597 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.597+0000 7f454ba08700 1 -- 172.21.15.44:0/2386271098 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f45300a0710 msgr2=0x7f45300a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:08.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.597+0000 7f454ba08700 1 --2- 172.21.15.44:0/2386271098 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f45300a0710 0x7f45300a2bd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4534009990 tx=0x7f453401a040 comp rx=0 tx=0).stop 2023-09-10T00:38:08.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.597+0000 7f454ba08700 1 -- 172.21.15.44:0/2386271098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4544101070 msgr2=0x7f454419c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:08.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.597+0000 7f454ba08700 1 --2- 172.21.15.44:0/2386271098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4544101070 0x7f454419c6e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f453c007f10 tx=0x7f453c00f5c0 comp rx=0 tx=0).stop 2023-09-10T00:38:08.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.597+0000 7f454ba08700 1 -- 172.21.15.44:0/2386271098 shutdown_connections 2023-09-10T00:38:08.599 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.597+0000 7f454ba08700 1 --2- 172.21.15.44:0/2386271098 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f45300a0710 0x7f45300a2bd0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:08.599 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.597+0000 7f454ba08700 1 --2- 172.21.15.44:0/2386271098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4544101070 0x7f454419c6e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:08.599 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.597+0000 7f454ba08700 1 --2- 172.21.15.44:0/2386271098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45440ff4e0 0x7f4544195130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:08.599 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.597+0000 7f454ba08700 1 --2- 172.21.15.44:0/2386271098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45441006d0 0x7f4544195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:08.599 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.597+0000 7f454ba08700 1 -- 172.21.15.44:0/2386271098 >> 172.21.15.44:0/2386271098 conn(0x7f45440faa30 msgr2=0x7f45440fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:08.599 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.597+0000 7f454ba08700 1 -- 172.21.15.44:0/2386271098 shutdown_connections 2023-09-10T00:38:08.600 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:08.597+0000 7f454ba08700 1 -- 172.21.15.44:0/2386271098 wait complete. 2023-09-10T00:38:08.600 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 1 2023-09-10T00:38:08.966 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:08 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3187024199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:08.967 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:08 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/351015606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:38:08.967 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:08 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2386271098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:08.967 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 10 00:38:08 smithi131 podman[140422]: 2023-09-10T00:38:08.968 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 10 00:38:08 smithi131 ceph-mds[140462]: -- 172.21.15.131:0/2358875215 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561d9fa971e0 con 0x561d9fa6cc00 2023-09-10T00:38:08.968 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 10 00:38:08 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-l[140438]: starting mds.l at 2023-09-10T00:38:09.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:08 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3187024199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:09.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:08 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/351015606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:38:09.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:08 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2386271098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:08 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3187024199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:08 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/351015606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:38:09.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:08 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2386271098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:09.295 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 10 00:38:08 smithi131 bash[140422]: 82aa2f832a06480e5a16ee8491e26fad43e2ae79bf3367cfd8f42dab9e1328df 2023-09-10T00:38:09.295 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 10 00:38:09 smithi131 systemd[1]: Started Ceph mds.l for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:38:10.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:09 smithi131 ceph-mon[103713]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:10.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:09 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] up:boot 2023-09-10T00:38:10.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:09 smithi131 ceph-mon[103713]: fsmap 1 up:standby 2023-09-10T00:38:10.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-10T00:38:10.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:10.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:10.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:10.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:10.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:10.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:09 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:10.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:09 smithi099 ceph-mon[102692]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:10.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:09 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] up:boot 2023-09-10T00:38:10.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:09 smithi099 ceph-mon[102692]: fsmap 1 up:standby 2023-09-10T00:38:10.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-10T00:38:10.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:10.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:10.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:10.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:10.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:10.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:09 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:10.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:09 smithi044 ceph-mon[98676]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:10.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:09 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] up:boot 2023-09-10T00:38:10.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:09 smithi044 ceph-mon[98676]: fsmap 1 up:standby 2023-09-10T00:38:10.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-10T00:38:10.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:10.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:10.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:10.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:10.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:10.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:09 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:10.564 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 10 00:38:10 smithi099 systemd[1]: Starting Ceph mds.k for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:38:10.612 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:38:10.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.753+0000 7f48b0ec3700 1 -- 172.21.15.44:0/3071652543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ac106320 msgr2=0x7f48ac108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:10.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.753+0000 7f48b0ec3700 1 --2- 172.21.15.44:0/3071652543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ac106320 0x7f48ac108730 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f48a400b0a0 tx=0x7f48a400b3b0 comp rx=0 tx=0).stop 2023-09-10T00:38:10.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.753+0000 7f48b0ec3700 1 -- 172.21.15.44:0/3071652543 shutdown_connections 2023-09-10T00:38:10.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.753+0000 7f48b0ec3700 1 --2- 172.21.15.44:0/3071652543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ac106320 0x7f48ac108730 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:10.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.753+0000 7f48b0ec3700 1 --2- 172.21.15.44:0/3071652543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ac1010c0 0x7f48ac1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:10.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.753+0000 7f48b0ec3700 1 --2- 172.21.15.44:0/3071652543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ac1039f0 0x7f48ac105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:10.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.753+0000 7f48b0ec3700 1 -- 172.21.15.44:0/3071652543 >> 172.21.15.44:0/3071652543 conn(0x7f48ac0faa30 msgr2=0x7f48ac0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:10.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.753+0000 7f48b0ec3700 1 -- 172.21.15.44:0/3071652543 shutdown_connections 2023-09-10T00:38:10.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.753+0000 7f48b0ec3700 1 -- 172.21.15.44:0/3071652543 wait complete. 2023-09-10T00:38:10.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48b0ec3700 1 Processor -- start 2023-09-10T00:38:10.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48b0ec3700 1 -- start start 2023-09-10T00:38:10.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48b0ec3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ac1010c0 0x7f48ac100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:10.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48b0ec3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ac1039f0 0x7f48ac0ff070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:10.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48b0ec3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ac106320 0x7f48ac0ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:10.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48b0ec3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac0ffd50 con 0x7f48ac106320 2023-09-10T00:38:10.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48b0ec3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac0ffec0 con 0x7f48ac1039f0 2023-09-10T00:38:10.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48b0ec3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac1a02e0 con 0x7f48ac1010c0 2023-09-10T00:38:10.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48aad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ac106320 0x7f48ac0ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:10.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48a9d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ac1039f0 0x7f48ac0ff070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:10.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48aa59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ac1010c0 0x7f48ac100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:10.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48aad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ac106320 0x7f48ac0ff5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45984/0 (socket says 172.21.15.44:45984) 2023-09-10T00:38:10.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48a9d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ac1039f0 0x7f48ac0ff070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:47722/0 (socket says 172.21.15.44:47722) 2023-09-10T00:38:10.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48aa59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ac1010c0 0x7f48ac100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:52862/0 (socket says 172.21.15.44:52862) 2023-09-10T00:38:10.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48aad9d700 1 -- 172.21.15.44:0/1099929464 learned_addr learned my addr 172.21.15.44:0/1099929464 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:10.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48aad9d700 1 -- 172.21.15.44:0/1099929464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ac1010c0 msgr2=0x7f48ac100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:10.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48aad9d700 1 --2- 172.21.15.44:0/1099929464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ac1010c0 0x7f48ac100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:10.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48aad9d700 1 -- 172.21.15.44:0/1099929464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ac1039f0 msgr2=0x7f48ac0ff070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:10.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48aad9d700 1 --2- 172.21.15.44:0/1099929464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ac1039f0 0x7f48ac0ff070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:10.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48aad9d700 1 -- 172.21.15.44:0/1099929464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a4009d00 con 0x7f48ac106320 2023-09-10T00:38:10.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48a9d9b700 1 --2- 172.21.15.44:0/1099929464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ac1039f0 0x7f48ac0ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:38:10.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.754+0000 7f48aad9d700 1 --2- 172.21.15.44:0/1099929464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ac106320 0x7f48ac0ff5b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f48a400b780 tx=0x7f48a400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:10.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.755+0000 7f48a37fe700 1 -- 172.21.15.44:0/1099929464 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a400e040 con 0x7f48ac106320 2023-09-10T00:38:10.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.755+0000 7f48a37fe700 1 -- 172.21.15.44:0/1099929464 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48a4004e60 con 0x7f48ac106320 2023-09-10T00:38:10.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.755+0000 7f48b0ec3700 1 -- 172.21.15.44:0/1099929464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ac1a0560 con 0x7f48ac106320 2023-09-10T00:38:10.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.755+0000 7f48a37fe700 1 -- 172.21.15.44:0/1099929464 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a40052a0 con 0x7f48ac106320 2023-09-10T00:38:10.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.755+0000 7f48b0ec3700 1 -- 172.21.15.44:0/1099929464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ac1a0a30 con 0x7f48ac106320 2023-09-10T00:38:10.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.756+0000 7f48b0ec3700 1 -- 172.21.15.44:0/1099929464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ac0fc5b0 con 0x7f48ac106320 2023-09-10T00:38:10.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.757+0000 7f48a37fe700 1 -- 172.21.15.44:0/1099929464 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f48a401d470 con 0x7f48ac106320 2023-09-10T00:38:10.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.758+0000 7f48a37fe700 1 --2- 172.21.15.44:0/1099929464 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f48980a0710 0x7f48980a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:10.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.758+0000 7f48a37fe700 1 -- 172.21.15.44:0/1099929464 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f48a40cd3a0 con 0x7f48ac106320 2023-09-10T00:38:10.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.758+0000 7f48aa59c700 1 --2- 172.21.15.44:0/1099929464 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f48980a0710 0x7f48980a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:10.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.758+0000 7f48aa59c700 1 --2- 172.21.15.44:0/1099929464 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f48980a0710 0x7f48980a2bd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f489c005950 tx=0x7f489c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:10.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:10.762+0000 7f48a37fe700 1 -- 172.21.15.44:0/1099929464 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f48a4097400 con 0x7f48ac106320 2023-09-10T00:38:10.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:10 smithi099 ceph-mon[102692]: Deploying daemon mds.k on smithi099 2023-09-10T00:38:11.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:11.011+0000 7f48b0ec3700 1 -- 172.21.15.44:0/1099929464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48ac10bef0 con 0x7f48ac106320 2023-09-10T00:38:11.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:11.012+0000 7f48a37fe700 1 -- 172.21.15.44:0/1099929464 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f48a409aa20 con 0x7f48ac106320 2023-09-10T00:38:11.013 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:11.013 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-10T00:38:11.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:11.014+0000 7f48b0ec3700 1 -- 172.21.15.44:0/1099929464 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f48980a0710 msgr2=0x7f48980a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:11.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:11.014+0000 7f48b0ec3700 1 --2- 172.21.15.44:0/1099929464 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f48980a0710 0x7f48980a2bd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f489c005950 tx=0x7f489c00b410 comp rx=0 tx=0).stop 2023-09-10T00:38:11.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:11.014+0000 7f48b0ec3700 1 -- 172.21.15.44:0/1099929464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ac106320 msgr2=0x7f48ac0ff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:11.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:11.014+0000 7f48b0ec3700 1 --2- 172.21.15.44:0/1099929464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ac106320 0x7f48ac0ff5b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f48a400b780 tx=0x7f48a400bd40 comp rx=0 tx=0).stop 2023-09-10T00:38:11.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:11.014+0000 7f48b0ec3700 1 -- 172.21.15.44:0/1099929464 shutdown_connections 2023-09-10T00:38:11.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:11.014+0000 7f48b0ec3700 1 --2- 172.21.15.44:0/1099929464 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f48980a0710 0x7f48980a2bd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:11.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:11.014+0000 7f48b0ec3700 1 --2- 172.21.15.44:0/1099929464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ac106320 0x7f48ac0ff5b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:11.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:11.014+0000 7f48b0ec3700 1 --2- 172.21.15.44:0/1099929464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ac1010c0 0x7f48ac100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:11.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:11.014+0000 7f48b0ec3700 1 --2- 172.21.15.44:0/1099929464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ac1039f0 0x7f48ac0ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:11.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:11.014+0000 7f48b0ec3700 1 -- 172.21.15.44:0/1099929464 >> 172.21.15.44:0/1099929464 conn(0x7f48ac0faa30 msgr2=0x7f48ac101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:11.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:11.014+0000 7f48b0ec3700 1 -- 172.21.15.44:0/1099929464 shutdown_connections 2023-09-10T00:38:11.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:11.014+0000 7f48b0ec3700 1 -- 172.21.15.44:0/1099929464 wait complete. 2023-09-10T00:38:11.017 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 2 2023-09-10T00:38:11.026 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:10 smithi044 ceph-mon[98676]: Deploying daemon mds.k on smithi099 2023-09-10T00:38:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:10 smithi131 ceph-mon[103713]: Deploying daemon mds.k on smithi099 2023-09-10T00:38:11.148 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 10 00:38:10 smithi099 podman[137805]: 2023-09-10T00:38:11.148 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 10 00:38:11 smithi099 ceph-mds[137845]: -- 172.21.15.99:0/418094688 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5600905dd1e0 con 0x5600905b2c00 2023-09-10T00:38:11.149 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 10 00:38:11 smithi099 bash[137805]: 861abd1120e688eca6e99611966ece5faaf473a61eaea48ace2e5286d8e85ca4 2023-09-10T00:38:11.149 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 10 00:38:11 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-k[137821]: starting mds.k at 2023-09-10T00:38:11.606 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 10 00:38:11 smithi099 systemd[1]: Started Ceph mds.k for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:38:12.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:11 smithi131 ceph-mon[103713]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:12.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:11 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1099929464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:12.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:12.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:12.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:12.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:12.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:12.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:11 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:12.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:11 smithi044 ceph-mon[98676]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:12.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:11 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1099929464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:12.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:12.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:12.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:12.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:12.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:12.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:11 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:12.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:11 smithi099 ceph-mon[102692]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:12.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:11 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1099929464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:12.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:12.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:12.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:12.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:12.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:12.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:11 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:12.821 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:12 smithi044 ceph-mon[98676]: Deploying daemon mds.j on smithi044 2023-09-10T00:38:12.821 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:12 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] up:boot 2023-09-10T00:38:12.822 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:12 smithi044 ceph-mon[98676]: fsmap 2 up:standby 2023-09-10T00:38:12.822 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:12 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-10T00:38:13.029 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:38:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:12 smithi131 ceph-mon[103713]: Deploying daemon mds.j on smithi044 2023-09-10T00:38:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:12 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] up:boot 2023-09-10T00:38:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:12 smithi131 ceph-mon[103713]: fsmap 2 up:standby 2023-09-10T00:38:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:12 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-10T00:38:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:12 smithi099 ceph-mon[102692]: Deploying daemon mds.j on smithi044 2023-09-10T00:38:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:12 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] up:boot 2023-09-10T00:38:13.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:12 smithi099 ceph-mon[102692]: fsmap 2 up:standby 2023-09-10T00:38:13.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:12 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-10T00:38:13.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.184+0000 7f37fb862700 1 -- 172.21.15.44:0/3694325335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37f4100600 msgr2=0x7f37f4100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.184+0000 7f37fb862700 1 --2- 172.21.15.44:0/3694325335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37f4100600 0x7f37f4100a20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f37f00099c0 tx=0x7f37f0009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.184+0000 7f37fb862700 1 -- 172.21.15.44:0/3694325335 shutdown_connections 2023-09-10T00:38:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.184+0000 7f37fb862700 1 --2- 172.21.15.44:0/3694325335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37f4102190 0x7f37f4108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.184+0000 7f37fb862700 1 --2- 172.21.15.44:0/3694325335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37f4100600 0x7f37f4100a20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.184+0000 7f37fb862700 1 --2- 172.21.15.44:0/3694325335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f41017f0 0x7f37f4101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:13.186 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.184+0000 7f37fb862700 1 -- 172.21.15.44:0/3694325335 >> 172.21.15.44:0/3694325335 conn(0x7f37f40fbb50 msgr2=0x7f37f40fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:13.186 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.184+0000 7f37fb862700 1 -- 172.21.15.44:0/3694325335 shutdown_connections 2023-09-10T00:38:13.186 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.184+0000 7f37fb862700 1 -- 172.21.15.44:0/3694325335 wait complete. 2023-09-10T00:38:13.189 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37fb862700 1 Processor -- start 2023-09-10T00:38:13.189 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37fb862700 1 -- start start 2023-09-10T00:38:13.189 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37fb862700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37f4100600 0x7f37f4196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:13.189 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37fb862700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37f41017f0 0x7f37f4196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:13.189 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37fb862700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f4102190 0x7f37f419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:13.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37fb862700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f419dd90 con 0x7f37f41017f0 2023-09-10T00:38:13.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37fb862700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f419ded0 con 0x7f37f4102190 2023-09-10T00:38:13.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37fb862700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f419e060 con 0x7f37f4100600 2023-09-10T00:38:13.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37f8dfd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37f41017f0 0x7f37f4196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:13.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37f8dfd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37f41017f0 0x7f37f4196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45994/0 (socket says 172.21.15.44:45994) 2023-09-10T00:38:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37f8dfd700 1 -- 172.21.15.44:0/3820659232 learned_addr learned my addr 172.21.15.44:0/3820659232 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37f95fe700 1 --2- 172.21.15.44:0/3820659232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37f4100600 0x7f37f4196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37f9dff700 1 --2- 172.21.15.44:0/3820659232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f4102190 0x7f37f419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37f8dfd700 1 -- 172.21.15.44:0/3820659232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37f4100600 msgr2=0x7f37f4196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37f8dfd700 1 --2- 172.21.15.44:0/3820659232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37f4100600 0x7f37f4196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:13.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37f8dfd700 1 -- 172.21.15.44:0/3820659232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f4102190 msgr2=0x7f37f419d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:13.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37f8dfd700 1 --2- 172.21.15.44:0/3820659232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f4102190 0x7f37f419d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:13.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37f8dfd700 1 -- 172.21.15.44:0/3820659232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f00095f0 con 0x7f37f41017f0 2023-09-10T00:38:13.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.185+0000 7f37f9dff700 1 --2- 172.21.15.44:0/3820659232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f4102190 0x7f37f419d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:38:13.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.186+0000 7f37f8dfd700 1 --2- 172.21.15.44:0/3820659232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37f41017f0 0x7f37f4196750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f37e400c920 tx=0x7f37e400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:13.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.186+0000 7f37ea7fc700 1 -- 172.21.15.44:0/3820659232 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e4007b00 con 0x7f37f41017f0 2023-09-10T00:38:13.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.186+0000 7f37ea7fc700 1 -- 172.21.15.44:0/3820659232 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f37e400ce70 con 0x7f37f41017f0 2023-09-10T00:38:13.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.186+0000 7f37fb862700 1 -- 172.21.15.44:0/3820659232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f419e340 con 0x7f37f41017f0 2023-09-10T00:38:13.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.186+0000 7f37ea7fc700 1 -- 172.21.15.44:0/3820659232 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e40053e0 con 0x7f37f41017f0 2023-09-10T00:38:13.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.186+0000 7f37fb862700 1 -- 172.21.15.44:0/3820659232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f419e850 con 0x7f37f41017f0 2023-09-10T00:38:13.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.187+0000 7f37fb862700 1 -- 172.21.15.44:0/3820659232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f410bce0 con 0x7f37f41017f0 2023-09-10T00:38:13.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.188+0000 7f37ea7fc700 1 -- 172.21.15.44:0/3820659232 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f37e4005580 con 0x7f37f41017f0 2023-09-10T00:38:13.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.188+0000 7f37ea7fc700 1 --2- 172.21.15.44:0/3820659232 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f37e00a0690 0x7f37e00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:13.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.189+0000 7f37f95fe700 1 --2- 172.21.15.44:0/3820659232 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f37e00a0690 0x7f37e00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:13.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.189+0000 7f37ea7fc700 1 -- 172.21.15.44:0/3820659232 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f37e40cb310 con 0x7f37f41017f0 2023-09-10T00:38:13.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.189+0000 7f37f95fe700 1 --2- 172.21.15.44:0/3820659232 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f37e00a0690 0x7f37e00a2b50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f37f000c010 tx=0x7f37f000ba70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:13.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.193+0000 7f37ea7fc700 1 -- 172.21.15.44:0/3820659232 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f37e40952e0 con 0x7f37f41017f0 2023-09-10T00:38:13.247 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 10 00:38:12 smithi044 systemd[1]: Starting Ceph mds.j for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:38:13.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.472+0000 7f37fb862700 1 -- 172.21.15.44:0/3820659232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37f410bed0 con 0x7f37f41017f0 2023-09-10T00:38:13.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.473+0000 7f37ea7fc700 1 -- 172.21.15.44:0/3820659232 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f37e4098900 con 0x7f37f41017f0 2023-09-10T00:38:13.474 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:13.474 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24515,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-09-10T00:38:13.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.476+0000 7f37fb862700 1 -- 172.21.15.44:0/3820659232 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f37e00a0690 msgr2=0x7f37e00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:13.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.476+0000 7f37fb862700 1 --2- 172.21.15.44:0/3820659232 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f37e00a0690 0x7f37e00a2b50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f37f000c010 tx=0x7f37f000ba70 comp rx=0 tx=0).stop 2023-09-10T00:38:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.476+0000 7f37fb862700 1 -- 172.21.15.44:0/3820659232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37f41017f0 msgr2=0x7f37f4196750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.476+0000 7f37fb862700 1 --2- 172.21.15.44:0/3820659232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37f41017f0 0x7f37f4196750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f37e400c920 tx=0x7f37e400cc30 comp rx=0 tx=0).stop 2023-09-10T00:38:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.476+0000 7f37fb862700 1 -- 172.21.15.44:0/3820659232 shutdown_connections 2023-09-10T00:38:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.476+0000 7f37fb862700 1 --2- 172.21.15.44:0/3820659232 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f37e00a0690 0x7f37e00a2b50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.476+0000 7f37fb862700 1 --2- 172.21.15.44:0/3820659232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f4102190 0x7f37f419d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:13.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.476+0000 7f37fb862700 1 --2- 172.21.15.44:0/3820659232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37f4100600 0x7f37f4196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:13.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.476+0000 7f37fb862700 1 --2- 172.21.15.44:0/3820659232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37f41017f0 0x7f37f4196750 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:13.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.476+0000 7f37fb862700 1 -- 172.21.15.44:0/3820659232 >> 172.21.15.44:0/3820659232 conn(0x7f37f40fbb50 msgr2=0x7f37f40fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:13.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.476+0000 7f37fb862700 1 -- 172.21.15.44:0/3820659232 shutdown_connections 2023-09-10T00:38:13.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:13.476+0000 7f37fb862700 1 -- 172.21.15.44:0/3820659232 wait complete. 2023-09-10T00:38:13.479 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 3 2023-09-10T00:38:13.552 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 10 00:38:13 smithi044 podman[153704]: 2023-09-10T00:38:13.805 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 10 00:38:13 smithi044 ceph-mds[153766]: -- 172.21.15.44:0/1235878594 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557c897951e0 con 0x557c8976b000 2023-09-10T00:38:13.805 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 10 00:38:13 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-j[153740]: starting mds.j at 2023-09-10T00:38:13.805 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 10 00:38:13 smithi044 bash[153704]: e21e54c5a077d45122d5dc55d96a46583909601e19ca5b0a3e64ca89e71a55ec 2023-09-10T00:38:13.805 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 10 00:38:13 smithi044 systemd[1]: Started Ceph mds.j for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:38:13.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:13 smithi044 ceph-mon[98676]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:13.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:13 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3820659232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:13.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:13 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:13.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:13 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:13.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:13 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:13.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:13 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:13.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:13 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:13.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:13 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:13 smithi131 ceph-mon[103713]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:13 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3820659232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:14.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:13 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:14.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:13 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:14.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:13 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:14.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:13 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:14.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:13 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:14.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:13 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:14.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:13 smithi099 ceph-mon[102692]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:14.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:13 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3820659232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:14.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:13 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:14.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:13 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:14.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:13 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:14.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:13 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:14.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:13 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:14.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:13 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:14 smithi131 ceph-mon[103713]: Deploying daemon mds.d on smithi044 2023-09-10T00:38:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:14 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.44:6834/4035626069,v1:172.21.15.44:6835/4035626069] up:boot 2023-09-10T00:38:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:14 smithi131 ceph-mon[103713]: fsmap 3 up:standby 2023-09-10T00:38:15.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:14 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-10T00:38:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:14 smithi044 ceph-mon[98676]: Deploying daemon mds.d on smithi044 2023-09-10T00:38:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:14 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.44:6834/4035626069,v1:172.21.15.44:6835/4035626069] up:boot 2023-09-10T00:38:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:14 smithi044 ceph-mon[98676]: fsmap 3 up:standby 2023-09-10T00:38:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:14 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-10T00:38:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:14 smithi099 ceph-mon[102692]: Deploying daemon mds.d on smithi044 2023-09-10T00:38:15.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:14 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.44:6834/4035626069,v1:172.21.15.44:6835/4035626069] up:boot 2023-09-10T00:38:15.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:14 smithi099 ceph-mon[102692]: fsmap 3 up:standby 2023-09-10T00:38:15.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:14 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-10T00:38:15.340 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 00:38:15 smithi044 systemd[1]: Starting Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:38:15.488 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:38:15.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.625+0000 7fb3233df700 1 -- 172.21.15.44:0/3685113122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb31c0ff4e0 msgr2=0x7fb31c0ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:15.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.625+0000 7fb3233df700 1 --2- 172.21.15.44:0/3685113122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb31c0ff4e0 0x7fb31c0ff900 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb3180099c0 tx=0x7fb318009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:15.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.626+0000 7fb3233df700 1 -- 172.21.15.44:0/3685113122 shutdown_connections 2023-09-10T00:38:15.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.626+0000 7fb3233df700 1 --2- 172.21.15.44:0/3685113122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb31c101070 0x7fb31c107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:15.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.626+0000 7fb3233df700 1 --2- 172.21.15.44:0/3685113122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb31c0ff4e0 0x7fb31c0ff900 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:15.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.626+0000 7fb3233df700 1 --2- 172.21.15.44:0/3685113122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb31c1006d0 0x7fb31c100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:15.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.626+0000 7fb3233df700 1 -- 172.21.15.44:0/3685113122 >> 172.21.15.44:0/3685113122 conn(0x7fb31c0faa30 msgr2=0x7fb31c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:15.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.626+0000 7fb3233df700 1 -- 172.21.15.44:0/3685113122 shutdown_connections 2023-09-10T00:38:15.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.626+0000 7fb3233df700 1 -- 172.21.15.44:0/3685113122 wait complete. 2023-09-10T00:38:15.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.626+0000 7fb3233df700 1 Processor -- start 2023-09-10T00:38:15.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.626+0000 7fb3233df700 1 -- start start 2023-09-10T00:38:15.631 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.626+0000 7fb3233df700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb31c0ff4e0 0x7fb31c0682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:15.631 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.626+0000 7fb3233df700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb31c1006d0 0x7fb31c19d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:15.631 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.626+0000 7fb3233df700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb31c101070 0x7fb31c19f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:15.631 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.626+0000 7fb3233df700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c19ff70 con 0x7fb31c1006d0 2023-09-10T00:38:15.631 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb3233df700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c1a00e0 con 0x7fb31c101070 2023-09-10T00:38:15.632 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb3233df700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c1a02a0 con 0x7fb31c0ff4e0 2023-09-10T00:38:15.632 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb32097a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb31c1006d0 0x7fb31c19d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:15.632 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb32097a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb31c1006d0 0x7fb31c19d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46024/0 (socket says 172.21.15.44:46024) 2023-09-10T00:38:15.632 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb32097a700 1 -- 172.21.15.44:0/1303820705 learned_addr learned my addr 172.21.15.44:0/1303820705 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:15.632 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb32117b700 1 --2- 172.21.15.44:0/1303820705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb31c0ff4e0 0x7fb31c0682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:15.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb32197c700 1 --2- 172.21.15.44:0/1303820705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb31c101070 0x7fb31c19f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:15.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb32097a700 1 -- 172.21.15.44:0/1303820705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb31c0ff4e0 msgr2=0x7fb31c0682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:15.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb32097a700 1 --2- 172.21.15.44:0/1303820705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb31c0ff4e0 0x7fb31c0682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:15.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb32097a700 1 -- 172.21.15.44:0/1303820705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb31c101070 msgr2=0x7fb31c19f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:15.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb32097a700 1 --2- 172.21.15.44:0/1303820705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb31c101070 0x7fb31c19f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:15.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb32097a700 1 -- 172.21.15.44:0/1303820705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3180095f0 con 0x7fb31c1006d0 2023-09-10T00:38:15.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb32097a700 1 --2- 172.21.15.44:0/1303820705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb31c1006d0 0x7fb31c19d370 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb31000d950 tx=0x7fb31000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:15.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb30e7fc700 1 -- 172.21.15.44:0/1303820705 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31000f890 con 0x7fb31c1006d0 2023-09-10T00:38:15.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb30e7fc700 1 -- 172.21.15.44:0/1303820705 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb31000fe80 con 0x7fb31c1006d0 2023-09-10T00:38:15.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb3233df700 1 -- 172.21.15.44:0/1303820705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31c1a0580 con 0x7fb31c1006d0 2023-09-10T00:38:15.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb30e7fc700 1 -- 172.21.15.44:0/1303820705 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31000e3b0 con 0x7fb31c1006d0 2023-09-10T00:38:15.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.627+0000 7fb3233df700 1 -- 172.21.15.44:0/1303820705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31c1a0a90 con 0x7fb31c1006d0 2023-09-10T00:38:15.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.628+0000 7fb3233df700 1 -- 172.21.15.44:0/1303820705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31c10aba0 con 0x7fb31c1006d0 2023-09-10T00:38:15.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.630+0000 7fb30e7fc700 1 -- 172.21.15.44:0/1303820705 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fb310010460 con 0x7fb31c1006d0 2023-09-10T00:38:15.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.630+0000 7fb30e7fc700 1 --2- 172.21.15.44:0/1303820705 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb3080a0690 0x7fb3080a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:15.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.630+0000 7fb32117b700 1 --2- 172.21.15.44:0/1303820705 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb3080a0690 0x7fb3080a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:15.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.630+0000 7fb30e7fc700 1 -- 172.21.15.44:0/1303820705 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb3100cb450 con 0x7fb31c1006d0 2023-09-10T00:38:15.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.631+0000 7fb32117b700 1 --2- 172.21.15.44:0/1303820705 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb3080a0690 0x7fb3080a2b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb31800c010 tx=0x7fb318005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:15.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.634+0000 7fb30e7fc700 1 -- 172.21.15.44:0/1303820705 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb310095420 con 0x7fb31c1006d0 2023-09-10T00:38:15.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.896+0000 7fb3233df700 1 -- 172.21.15.44:0/1303820705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb31c1064b0 con 0x7fb31c1006d0 2023-09-10T00:38:15.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.896+0000 7fb30e7fc700 1 -- 172.21.15.44:0/1303820705 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7fb310016b70 con 0x7fb31c1006d0 2023-09-10T00:38:15.897 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:15.898 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24515,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-09-10T00:38:15.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.899+0000 7fb3233df700 1 -- 172.21.15.44:0/1303820705 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb3080a0690 msgr2=0x7fb3080a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:15.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.899+0000 7fb3233df700 1 --2- 172.21.15.44:0/1303820705 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb3080a0690 0x7fb3080a2b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb31800c010 tx=0x7fb318005e80 comp rx=0 tx=0).stop 2023-09-10T00:38:15.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.899+0000 7fb3233df700 1 -- 172.21.15.44:0/1303820705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb31c1006d0 msgr2=0x7fb31c19d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:15.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.899+0000 7fb3233df700 1 --2- 172.21.15.44:0/1303820705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb31c1006d0 0x7fb31c19d370 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb31000d950 tx=0x7fb31000dc60 comp rx=0 tx=0).stop 2023-09-10T00:38:15.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.899+0000 7fb3233df700 1 -- 172.21.15.44:0/1303820705 shutdown_connections 2023-09-10T00:38:15.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.899+0000 7fb3233df700 1 --2- 172.21.15.44:0/1303820705 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb3080a0690 0x7fb3080a2b50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:15.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.899+0000 7fb3233df700 1 --2- 172.21.15.44:0/1303820705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb31c101070 0x7fb31c19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:15.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.899+0000 7fb3233df700 1 --2- 172.21.15.44:0/1303820705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb31c0ff4e0 0x7fb31c0682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:15.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.899+0000 7fb3233df700 1 --2- 172.21.15.44:0/1303820705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb31c1006d0 0x7fb31c19d370 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:15.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.899+0000 7fb3233df700 1 -- 172.21.15.44:0/1303820705 >> 172.21.15.44:0/1303820705 conn(0x7fb31c0faa30 msgr2=0x7fb31c0fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:15.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.899+0000 7fb3233df700 1 -- 172.21.15.44:0/1303820705 shutdown_connections 2023-09-10T00:38:15.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:15.899+0000 7fb3233df700 1 -- 172.21.15.44:0/1303820705 wait complete. 2023-09-10T00:38:15.902 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 4 2023-09-10T00:38:15.912 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:15 smithi044 ceph-mon[98676]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:15.912 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 00:38:15 smithi044 podman[154071]: 2023-09-10T00:38:15.912 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 00:38:15 smithi044 bash[154071]: 9195cc73f6c95599bbe28a3b6dcbed3360db874188ca166dfcd1dba5d932441b 2023-09-10T00:38:15.912 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 00:38:15 smithi044 ceph-mds[154154]: -- 172.21.15.44:0/1968418687 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559a4ce0b1e0 con 0x559a4cde0c00 2023-09-10T00:38:15.913 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 00:38:15 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: starting mds.d at 2023-09-10T00:38:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:15 smithi131 ceph-mon[103713]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:16.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:15 smithi099 ceph-mon[102692]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:16.305 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 00:38:15 smithi044 systemd[1]: Started Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:38:16.938 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:16 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1303820705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:16.938 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:16.939 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:16.939 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:16.939 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:16.939 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:16.939 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:16 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1303820705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:17.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:17.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:17.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:17.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:16 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1303820705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:17.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:17.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:17.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:17.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:17.607 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 10 00:38:17 smithi099 systemd[1]: Starting Ceph mds.e for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:38:17.914 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:38:18.013 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 10 00:38:17 smithi099 podman[138146]: 2023-09-10T00:38:18.013 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 10 00:38:18 smithi099 ceph-mds[138186]: -- 172.21.15.99:0/331732871 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562a562871e0 con 0x562a56248800 2023-09-10T00:38:18.013 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 10 00:38:18 smithi099 ceph-mds[138186]: -- 172.21.15.99:0/331732871 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562a562731e0 con 0x562a56248800 2023-09-10T00:38:18.014 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:17 smithi099 ceph-mon[102692]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:18.014 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:17 smithi099 ceph-mon[102692]: Deploying daemon mds.e on smithi099 2023-09-10T00:38:18.014 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:17 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.44:6836/1919207693,v1:172.21.15.44:6837/1919207693] up:boot 2023-09-10T00:38:18.014 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:17 smithi099 ceph-mon[102692]: fsmap 4 up:standby 2023-09-10T00:38:18.015 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T00:38:18.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:17 smithi131 ceph-mon[103713]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:18.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:17 smithi131 ceph-mon[103713]: Deploying daemon mds.e on smithi099 2023-09-10T00:38:18.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:17 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.44:6836/1919207693,v1:172.21.15.44:6837/1919207693] up:boot 2023-09-10T00:38:18.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:17 smithi131 ceph-mon[103713]: fsmap 4 up:standby 2023-09-10T00:38:18.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T00:38:18.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:17 smithi044 ceph-mon[98676]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:18.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:17 smithi044 ceph-mon[98676]: Deploying daemon mds.e on smithi099 2023-09-10T00:38:18.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:17 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.44:6836/1919207693,v1:172.21.15.44:6837/1919207693] up:boot 2023-09-10T00:38:18.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:17 smithi044 ceph-mon[98676]: fsmap 4 up:standby 2023-09-10T00:38:18.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T00:38:18.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.056+0000 7fcf630c5700 1 -- 172.21.15.44:0/1588831186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf5c100200 msgr2=0x7fcf5c100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:18.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.056+0000 7fcf630c5700 1 --2- 172.21.15.44:0/1588831186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf5c100200 0x7fcf5c100680 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400b0a0 tx=0x7fcf5400b3b0 comp rx=0 tx=0).stop 2023-09-10T00:38:18.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.056+0000 7fcf630c5700 1 -- 172.21.15.44:0/1588831186 shutdown_connections 2023-09-10T00:38:18.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.056+0000 7fcf630c5700 1 --2- 172.21.15.44:0/1588831186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf5c100200 0x7fcf5c100680 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:18.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.056+0000 7fcf630c5700 1 --2- 172.21.15.44:0/1588831186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5c0feee0 0x7fcf5c0ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:18.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.056+0000 7fcf630c5700 1 --2- 172.21.15.44:0/1588831186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf5c0ff840 0x7fcf5c0ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:18.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.056+0000 7fcf630c5700 1 -- 172.21.15.44:0/1588831186 >> 172.21.15.44:0/1588831186 conn(0x7fcf5c0faa20 msgr2=0x7fcf5c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:18.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.056+0000 7fcf630c5700 1 -- 172.21.15.44:0/1588831186 shutdown_connections 2023-09-10T00:38:18.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.056+0000 7fcf630c5700 1 -- 172.21.15.44:0/1588831186 wait complete. 2023-09-10T00:38:18.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf630c5700 1 Processor -- start 2023-09-10T00:38:18.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf630c5700 1 -- start start 2023-09-10T00:38:18.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf630c5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf5c0feee0 0x7fcf5c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf630c5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5c0ff840 0x7fcf5c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf630c5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf5c100200 0x7fcf5c19fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf630c5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c1a0060 con 0x7fcf5c0feee0 2023-09-10T00:38:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf630c5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c1a01a0 con 0x7fcf5c0ff840 2023-09-10T00:38:18.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf630c5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c1a0330 con 0x7fcf5c100200 2023-09-10T00:38:18.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf60e61700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf5c0feee0 0x7fcf5c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:18.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf60e61700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf5c0feee0 0x7fcf5c1984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46050/0 (socket says 172.21.15.44:46050) 2023-09-10T00:38:18.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf5bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5c0ff840 0x7fcf5c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:18.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf60e61700 1 -- 172.21.15.44:0/2027720215 learned_addr learned my addr 172.21.15.44:0/2027720215 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:18.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf61662700 1 --2- 172.21.15.44:0/2027720215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf5c100200 0x7fcf5c19fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:18.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf60e61700 1 -- 172.21.15.44:0/2027720215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf5c100200 msgr2=0x7fcf5c19fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:18.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf60e61700 1 --2- 172.21.15.44:0/2027720215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf5c100200 0x7fcf5c19fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:18.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf60e61700 1 -- 172.21.15.44:0/2027720215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5c0ff840 msgr2=0x7fcf5c198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:18.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf60e61700 1 --2- 172.21.15.44:0/2027720215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5c0ff840 0x7fcf5c198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:18.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf60e61700 1 -- 172.21.15.44:0/2027720215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf54009d00 con 0x7fcf5c0feee0 2023-09-10T00:38:18.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.057+0000 7fcf5bfff700 1 --2- 172.21.15.44:0/2027720215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5c0ff840 0x7fcf5c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:38:18.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.058+0000 7fcf60e61700 1 --2- 172.21.15.44:0/2027720215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf5c0feee0 0x7fcf5c1984e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000e9a0 tx=0x7fcf5000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:18.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.058+0000 7fcf59ffb700 1 -- 172.21.15.44:0/2027720215 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50009790 con 0x7fcf5c0feee0 2023-09-10T00:38:18.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.058+0000 7fcf59ffb700 1 -- 172.21.15.44:0/2027720215 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf50010430 con 0x7fcf5c0feee0 2023-09-10T00:38:18.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.058+0000 7fcf59ffb700 1 -- 172.21.15.44:0/2027720215 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000cea0 con 0x7fcf5c0feee0 2023-09-10T00:38:18.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.058+0000 7fcf630c5700 1 -- 172.21.15.44:0/2027720215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5c1a0610 con 0x7fcf5c0feee0 2023-09-10T00:38:18.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.058+0000 7fcf630c5700 1 -- 172.21.15.44:0/2027720215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5c1a0b20 con 0x7fcf5c0feee0 2023-09-10T00:38:18.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.059+0000 7fcf630c5700 1 -- 172.21.15.44:0/2027720215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5c109cc0 con 0x7fcf5c0feee0 2023-09-10T00:38:18.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.060+0000 7fcf59ffb700 1 -- 172.21.15.44:0/2027720215 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fcf500105a0 con 0x7fcf5c0feee0 2023-09-10T00:38:18.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.061+0000 7fcf59ffb700 1 --2- 172.21.15.44:0/2027720215 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcf4c0a0710 0x7fcf4c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:18.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.061+0000 7fcf5bfff700 1 --2- 172.21.15.44:0/2027720215 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcf4c0a0710 0x7fcf4c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:18.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.061+0000 7fcf59ffb700 1 -- 172.21.15.44:0/2027720215 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcf50014070 con 0x7fcf5c0feee0 2023-09-10T00:38:18.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.061+0000 7fcf5bfff700 1 --2- 172.21.15.44:0/2027720215 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcf4c0a0710 0x7fcf4c0a2bd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcf48011490 tx=0x7fcf48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:18.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.064+0000 7fcf59ffb700 1 -- 172.21.15.44:0/2027720215 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcf500959d0 con 0x7fcf5c0feee0 2023-09-10T00:38:18.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.331+0000 7fcf630c5700 1 -- 172.21.15.44:0/2027720215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf5c10dee0 con 0x7fcf5c0feee0 2023-09-10T00:38:18.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.331+0000 7fcf59ffb700 1 -- 172.21.15.44:0/2027720215 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7fcf50098ff0 con 0x7fcf5c0feee0 2023-09-10T00:38:18.332 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:18.332 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24515,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-09-10T00:38:18.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.333+0000 7fcf630c5700 1 -- 172.21.15.44:0/2027720215 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcf4c0a0710 msgr2=0x7fcf4c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:18.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.333+0000 7fcf630c5700 1 --2- 172.21.15.44:0/2027720215 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcf4c0a0710 0x7fcf4c0a2bd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcf48011490 tx=0x7fcf48009260 comp rx=0 tx=0).stop 2023-09-10T00:38:18.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.333+0000 7fcf630c5700 1 -- 172.21.15.44:0/2027720215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf5c0feee0 msgr2=0x7fcf5c1984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:18.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.333+0000 7fcf630c5700 1 --2- 172.21.15.44:0/2027720215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf5c0feee0 0x7fcf5c1984e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000e9a0 tx=0x7fcf5000ecb0 comp rx=0 tx=0).stop 2023-09-10T00:38:18.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.333+0000 7fcf630c5700 1 -- 172.21.15.44:0/2027720215 shutdown_connections 2023-09-10T00:38:18.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.333+0000 7fcf630c5700 1 --2- 172.21.15.44:0/2027720215 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcf4c0a0710 0x7fcf4c0a2bd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:18.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.333+0000 7fcf630c5700 1 --2- 172.21.15.44:0/2027720215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf5c100200 0x7fcf5c19fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:18.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.333+0000 7fcf630c5700 1 --2- 172.21.15.44:0/2027720215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf5c0feee0 0x7fcf5c1984e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:18.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.333+0000 7fcf630c5700 1 --2- 172.21.15.44:0/2027720215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5c0ff840 0x7fcf5c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:18.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.333+0000 7fcf630c5700 1 -- 172.21.15.44:0/2027720215 >> 172.21.15.44:0/2027720215 conn(0x7fcf5c0faa20 msgr2=0x7fcf5c0fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:18.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.333+0000 7fcf630c5700 1 -- 172.21.15.44:0/2027720215 shutdown_connections 2023-09-10T00:38:18.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:18.334+0000 7fcf630c5700 1 -- 172.21.15.44:0/2027720215 wait complete. 2023-09-10T00:38:18.337 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 5 2023-09-10T00:38:18.356 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 10 00:38:18 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-e[138162]: starting mds.e at 2023-09-10T00:38:18.356 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 10 00:38:18 smithi099 bash[138146]: dbbdc1119abc839ce99d38d44d189f485cfb7ff9412a12afe776c3d3e05780d4 2023-09-10T00:38:18.356 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 10 00:38:18 smithi099 systemd[1]: Started Ceph mds.e for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:38:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:19 smithi099 ceph-mon[102692]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:19 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:19 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:19 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:19 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:19 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:19 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:19 smithi099 ceph-mon[102692]: Deploying daemon mds.f on smithi131 2023-09-10T00:38:19.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:19 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2027720215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:19.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:19 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.99:6834/2403105238,v1:172.21.15.99:6835/2403105238] up:boot 2023-09-10T00:38:19.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:19 smithi099 ceph-mon[102692]: fsmap 5 up:standby 2023-09-10T00:38:19.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:19 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-10T00:38:19.361 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:19 smithi131 ceph-mon[103713]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:19.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:19 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:19.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:19 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:19.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:19 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:19.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:19 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:19.363 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:19 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:19.363 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:19 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:19.363 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:19 smithi131 ceph-mon[103713]: Deploying daemon mds.f on smithi131 2023-09-10T00:38:19.363 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:19 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2027720215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:19.363 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:19 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.99:6834/2403105238,v1:172.21.15.99:6835/2403105238] up:boot 2023-09-10T00:38:19.363 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:19 smithi131 ceph-mon[103713]: fsmap 5 up:standby 2023-09-10T00:38:19.364 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:19 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-10T00:38:19.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:19 smithi044 ceph-mon[98676]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:19.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:19 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:19.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:19 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:19.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:19 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:19.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:19 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:19.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:19 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:19.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:19 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:19.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:19 smithi044 ceph-mon[98676]: Deploying daemon mds.f on smithi131 2023-09-10T00:38:19.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:19 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2027720215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:19.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:19 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.99:6834/2403105238,v1:172.21.15.99:6835/2403105238] up:boot 2023-09-10T00:38:19.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:19 smithi044 ceph-mon[98676]: fsmap 5 up:standby 2023-09-10T00:38:19.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:19 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-10T00:38:19.795 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 10 00:38:19 smithi131 systemd[1]: Starting Ceph mds.f for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:38:20.188 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 10 00:38:19 smithi131 podman[140766]: 2023-09-10T00:38:20.188 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 10 00:38:20 smithi131 ceph-mds[140806]: -- 172.21.15.131:0/145655052 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564ec6ac11e0 con 0x564ec6a97000 2023-09-10T00:38:20.189 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 10 00:38:20 smithi131 bash[140766]: 8e21cee9e3993ee8b32a4a1b60a547ce8bc0dcd2ada5e2607c041086e7d157d4 2023-09-10T00:38:20.189 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 10 00:38:20 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-f[140782]: starting mds.f at 2023-09-10T00:38:20.347 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:38:20.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.488+0000 7fbaa8c6c700 1 -- 172.21.15.44:0/780995731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa40ff4e0 msgr2=0x7fbaa40ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:20.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.488+0000 7fbaa8c6c700 1 --2- 172.21.15.44:0/780995731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa40ff4e0 0x7fbaa40ff900 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fba980099c0 tx=0x7fba98009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:20.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.488+0000 7fbaa8c6c700 1 -- 172.21.15.44:0/780995731 shutdown_connections 2023-09-10T00:38:20.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.488+0000 7fbaa8c6c700 1 --2- 172.21.15.44:0/780995731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbaa4101070 0x7fbaa4107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:20.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.488+0000 7fbaa8c6c700 1 --2- 172.21.15.44:0/780995731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa40ff4e0 0x7fbaa40ff900 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:20.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.488+0000 7fbaa8c6c700 1 --2- 172.21.15.44:0/780995731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa41006d0 0x7fbaa4100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:20.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.488+0000 7fbaa8c6c700 1 -- 172.21.15.44:0/780995731 >> 172.21.15.44:0/780995731 conn(0x7fbaa40faa30 msgr2=0x7fbaa40fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:20.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.488+0000 7fbaa8c6c700 1 -- 172.21.15.44:0/780995731 shutdown_connections 2023-09-10T00:38:20.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.488+0000 7fbaa8c6c700 1 -- 172.21.15.44:0/780995731 wait complete. 2023-09-10T00:38:20.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.489+0000 7fbaa8c6c700 1 Processor -- start 2023-09-10T00:38:20.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.489+0000 7fbaa8c6c700 1 -- start start 2023-09-10T00:38:20.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fbaa8c6c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa40ff4e0 0x7fbaa4195140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:20.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fbaa8c6c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbaa41006d0 0x7fbaa4195680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:20.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fbaa8c6c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa4101070 0x7fbaa419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:20.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fbaa8c6c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa419ccc0 con 0x7fbaa40ff4e0 2023-09-10T00:38:20.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fbaa8c6c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa419ce00 con 0x7fbaa41006d0 2023-09-10T00:38:20.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fbaa8c6c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa419cf90 con 0x7fbaa4101070 2023-09-10T00:38:20.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fbaa259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa40ff4e0 0x7fbaa4195140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:20.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fbaa259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa40ff4e0 0x7fbaa4195140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46064/0 (socket says 172.21.15.44:46064) 2023-09-10T00:38:20.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fbaa259c700 1 -- 172.21.15.44:0/3386208758 learned_addr learned my addr 172.21.15.44:0/3386208758 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:20.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fbaa1d9b700 1 --2- 172.21.15.44:0/3386208758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbaa41006d0 0x7fbaa4195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:20.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fbaa259c700 1 -- 172.21.15.44:0/3386208758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa4101070 msgr2=0x7fbaa419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:20.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fbaa259c700 1 --2- 172.21.15.44:0/3386208758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa4101070 0x7fbaa419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:20.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fbaa259c700 1 -- 172.21.15.44:0/3386208758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbaa41006d0 msgr2=0x7fbaa4195680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:20.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fbaa259c700 1 --2- 172.21.15.44:0/3386208758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbaa41006d0 0x7fbaa4195680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:20.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fbaa259c700 1 -- 172.21.15.44:0/3386208758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba980095f0 con 0x7fbaa40ff4e0 2023-09-10T00:38:20.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fbaa259c700 1 --2- 172.21.15.44:0/3386208758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa40ff4e0 0x7fbaa4195140 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fba98009fd0 tx=0x7fba9800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:20.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.490+0000 7fba937fe700 1 -- 172.21.15.44:0/3386208758 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9801d070 con 0x7fbaa40ff4e0 2023-09-10T00:38:20.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.491+0000 7fbaa8c6c700 1 -- 172.21.15.44:0/3386208758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa419d210 con 0x7fbaa40ff4e0 2023-09-10T00:38:20.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.491+0000 7fba937fe700 1 -- 172.21.15.44:0/3386208758 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fba9800fe70 con 0x7fbaa40ff4e0 2023-09-10T00:38:20.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.491+0000 7fbaa8c6c700 1 -- 172.21.15.44:0/3386208758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa419d710 con 0x7fbaa40ff4e0 2023-09-10T00:38:20.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.491+0000 7fba937fe700 1 -- 172.21.15.44:0/3386208758 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba98022e70 con 0x7fbaa40ff4e0 2023-09-10T00:38:20.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.491+0000 7fbaa8c6c700 1 -- 172.21.15.44:0/3386208758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa410aba0 con 0x7fbaa40ff4e0 2023-09-10T00:38:20.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.493+0000 7fba937fe700 1 -- 172.21.15.44:0/3386208758 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fba98022470 con 0x7fbaa40ff4e0 2023-09-10T00:38:20.498 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.493+0000 7fba937fe700 1 --2- 172.21.15.44:0/3386208758 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba8c0a0690 0x7fba8c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:20.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.493+0000 7fba937fe700 1 -- 172.21.15.44:0/3386208758 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fba980cc600 con 0x7fbaa40ff4e0 2023-09-10T00:38:20.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.493+0000 7fbaa1d9b700 1 --2- 172.21.15.44:0/3386208758 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba8c0a0690 0x7fba8c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:20.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.494+0000 7fbaa1d9b700 1 --2- 172.21.15.44:0/3386208758 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba8c0a0690 0x7fba8c0a2b50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fba94009c60 tx=0x7fba94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:20.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.497+0000 7fba937fe700 1 -- 172.21.15.44:0/3386208758 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fba9805ce10 con 0x7fbaa40ff4e0 2023-09-10T00:38:20.545 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 10 00:38:20 smithi131 systemd[1]: Started Ceph mds.f for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:38:20.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.747+0000 7fbaa8c6c700 1 -- 172.21.15.44:0/3386208758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbaa419d9f0 con 0x7fbaa40ff4e0 2023-09-10T00:38:20.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.747+0000 7fba937fe700 1 -- 172.21.15.44:0/3386208758 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4492 (secure 0 0 0) 0x7fba98096660 con 0x7fbaa40ff4e0 2023-09-10T00:38:20.748 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:20.748 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24515,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24539,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-09-10T00:38:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.749+0000 7fbaa8c6c700 1 -- 172.21.15.44:0/3386208758 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba8c0a0690 msgr2=0x7fba8c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.749+0000 7fbaa8c6c700 1 --2- 172.21.15.44:0/3386208758 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba8c0a0690 0x7fba8c0a2b50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fba94009c60 tx=0x7fba94009260 comp rx=0 tx=0).stop 2023-09-10T00:38:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.749+0000 7fbaa8c6c700 1 -- 172.21.15.44:0/3386208758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa40ff4e0 msgr2=0x7fbaa4195140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.749+0000 7fbaa8c6c700 1 --2- 172.21.15.44:0/3386208758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa40ff4e0 0x7fbaa4195140 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fba98009fd0 tx=0x7fba9800fa00 comp rx=0 tx=0).stop 2023-09-10T00:38:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.749+0000 7fbaa8c6c700 1 -- 172.21.15.44:0/3386208758 shutdown_connections 2023-09-10T00:38:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.749+0000 7fbaa8c6c700 1 --2- 172.21.15.44:0/3386208758 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba8c0a0690 0x7fba8c0a2b50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.749+0000 7fbaa8c6c700 1 --2- 172.21.15.44:0/3386208758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa4101070 0x7fbaa419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.749+0000 7fbaa8c6c700 1 --2- 172.21.15.44:0/3386208758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa40ff4e0 0x7fbaa4195140 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.749+0000 7fbaa8c6c700 1 --2- 172.21.15.44:0/3386208758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbaa41006d0 0x7fbaa4195680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.750+0000 7fbaa8c6c700 1 -- 172.21.15.44:0/3386208758 >> 172.21.15.44:0/3386208758 conn(0x7fbaa40faa30 msgr2=0x7fbaa40fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.750+0000 7fbaa8c6c700 1 -- 172.21.15.44:0/3386208758 shutdown_connections 2023-09-10T00:38:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:20.750+0000 7fbaa8c6c700 1 -- 172.21.15.44:0/3386208758 wait complete. 2023-09-10T00:38:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 6 2023-09-10T00:38:21.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:21 smithi131 ceph-mon[103713]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:21.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:21.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:21.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:21.537 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:21.537 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:21.537 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:21.537 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:21 smithi131 ceph-mon[103713]: Deploying daemon mds.c on smithi131 2023-09-10T00:38:21.537 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:21 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3386208758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:21.538 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:21 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.131:6834/1445095724,v1:172.21.15.131:6835/1445095724] up:boot 2023-09-10T00:38:21.538 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:21 smithi131 ceph-mon[103713]: fsmap 6 up:standby 2023-09-10T00:38:21.538 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:21 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-10T00:38:21.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:21 smithi044 ceph-mon[98676]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:21.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:21.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:21.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:21.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:21.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:21.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:21.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:21 smithi044 ceph-mon[98676]: Deploying daemon mds.c on smithi131 2023-09-10T00:38:21.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:21 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3386208758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:21.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:21 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.131:6834/1445095724,v1:172.21.15.131:6835/1445095724] up:boot 2023-09-10T00:38:21.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:21 smithi044 ceph-mon[98676]: fsmap 6 up:standby 2023-09-10T00:38:21.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:21 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-10T00:38:21.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:21 smithi099 ceph-mon[102692]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:21.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:21.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:21.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:21.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:21.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:21.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:21.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:21 smithi099 ceph-mon[102692]: Deploying daemon mds.c on smithi131 2023-09-10T00:38:21.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:21 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3386208758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:21.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:21 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.131:6834/1445095724,v1:172.21.15.131:6835/1445095724] up:boot 2023-09-10T00:38:21.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:21 smithi099 ceph-mon[102692]: fsmap 6 up:standby 2023-09-10T00:38:21.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:21 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-10T00:38:21.795 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:38:21 smithi131 systemd[1]: Starting Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:38:22.390 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:38:22 smithi131 podman[141107]: 2023-09-10T00:38:22.763 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:38:22.795 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:38:22 smithi131 ceph-mds[141148]: -- 172.21.15.131:0/2854382116 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555c658831e0 con 0x555c65858c00 2023-09-10T00:38:22.795 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:38:22 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: starting mds.c at 2023-09-10T00:38:22.795 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:38:22 smithi131 bash[141107]: bf88bb0a140562856bf9044815b4d04f837036ca2a5bb2b1ce9255d01ed5d6f2 2023-09-10T00:38:22.795 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:38:22 smithi131 systemd[1]: Started Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:38:22.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.906+0000 7f54f4a8a700 1 -- 172.21.15.44:0/3583384171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f01021b0 msgr2=0x7f54f0108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:22.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.906+0000 7f54f4a8a700 1 --2- 172.21.15.44:0/3583384171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f01021b0 0x7f54f0108a60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f54e800b0d0 tx=0x7f54e800b3e0 comp rx=0 tx=0).stop 2023-09-10T00:38:22.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.907+0000 7f54f4a8a700 1 -- 172.21.15.44:0/3583384171 shutdown_connections 2023-09-10T00:38:22.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.907+0000 7f54f4a8a700 1 --2- 172.21.15.44:0/3583384171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f01021b0 0x7f54f0108a60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:22.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.907+0000 7f54f4a8a700 1 --2- 172.21.15.44:0/3583384171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f0100620 0x7f54f0100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:22.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.907+0000 7f54f4a8a700 1 --2- 172.21.15.44:0/3583384171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f0101810 0x7f54f0101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:22.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.907+0000 7f54f4a8a700 1 -- 172.21.15.44:0/3583384171 >> 172.21.15.44:0/3583384171 conn(0x7f54f00fbb70 msgr2=0x7f54f00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:22.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.907+0000 7f54f4a8a700 1 -- 172.21.15.44:0/3583384171 shutdown_connections 2023-09-10T00:38:22.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.907+0000 7f54f4a8a700 1 -- 172.21.15.44:0/3583384171 wait complete. 2023-09-10T00:38:22.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.907+0000 7f54f4a8a700 1 Processor -- start 2023-09-10T00:38:22.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.907+0000 7f54f4a8a700 1 -- start start 2023-09-10T00:38:22.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54f4a8a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f0100620 0x7f54f00743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:22.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54f4a8a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f0101810 0x7f54f00729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:22.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54f4a8a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f01021b0 0x7f54f0072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:22.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54f4a8a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00736d0 con 0x7f54f0100620 2023-09-10T00:38:22.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54f4a8a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f0073840 con 0x7f54f01021b0 2023-09-10T00:38:22.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54f4a8a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f01a0310 con 0x7f54f0101810 2023-09-10T00:38:22.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54ee59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f0100620 0x7f54f00743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:22.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54ee59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f0100620 0x7f54f00743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46078/0 (socket says 172.21.15.44:46078) 2023-09-10T00:38:22.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54ee59c700 1 -- 172.21.15.44:0/2120322043 learned_addr learned my addr 172.21.15.44:0/2120322043 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:22.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54edd9b700 1 --2- 172.21.15.44:0/2120322043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f0101810 0x7f54f00729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:22.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54eed9d700 1 --2- 172.21.15.44:0/2120322043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f01021b0 0x7f54f0072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:22.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54ee59c700 1 -- 172.21.15.44:0/2120322043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f0101810 msgr2=0x7f54f00729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:22.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54ee59c700 1 --2- 172.21.15.44:0/2120322043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f0101810 0x7f54f00729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:22.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54ee59c700 1 -- 172.21.15.44:0/2120322043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f01021b0 msgr2=0x7f54f0072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:22.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54ee59c700 1 --2- 172.21.15.44:0/2120322043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f01021b0 0x7f54f0072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:22.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54ee59c700 1 -- 172.21.15.44:0/2120322043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e0009520 con 0x7f54f0100620 2023-09-10T00:38:22.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54ee59c700 1 --2- 172.21.15.44:0/2120322043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f0100620 0x7f54f00743b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f54e000ea50 tx=0x7f54e000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:22.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54df7fe700 1 -- 172.21.15.44:0/2120322043 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e000cbd0 con 0x7f54f0100620 2023-09-10T00:38:22.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54f4a8a700 1 -- 172.21.15.44:0/2120322043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e8009d00 con 0x7f54f0100620 2023-09-10T00:38:22.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54df7fe700 1 -- 172.21.15.44:0/2120322043 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54e000cd30 con 0x7f54f0100620 2023-09-10T00:38:22.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.908+0000 7f54df7fe700 1 -- 172.21.15.44:0/2120322043 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e0018680 con 0x7f54f0100620 2023-09-10T00:38:22.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.909+0000 7f54f4a8a700 1 -- 172.21.15.44:0/2120322043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f01a09c0 con 0x7f54f0100620 2023-09-10T00:38:22.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.909+0000 7f54f4a8a700 1 -- 172.21.15.44:0/2120322043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f0107bc0 con 0x7f54f0100620 2023-09-10T00:38:22.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.911+0000 7f54df7fe700 1 -- 172.21.15.44:0/2120322043 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f54e0010430 con 0x7f54f0100620 2023-09-10T00:38:22.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.911+0000 7f54df7fe700 1 --2- 172.21.15.44:0/2120322043 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f54d80a0690 0x7f54d80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:22.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.911+0000 7f54edd9b700 1 --2- 172.21.15.44:0/2120322043 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f54d80a0690 0x7f54d80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:22.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.911+0000 7f54df7fe700 1 -- 172.21.15.44:0/2120322043 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f54e0014070 con 0x7f54f0100620 2023-09-10T00:38:22.917 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.912+0000 7f54edd9b700 1 --2- 172.21.15.44:0/2120322043 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f54d80a0690 0x7f54d80a2b50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f54e4007be0 tx=0x7f54e40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:22.917 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:22.915+0000 7f54df7fe700 1 -- 172.21.15.44:0/2120322043 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f54e00961d0 con 0x7f54f0100620 2023-09-10T00:38:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:23.165+0000 7f54f4a8a700 1 -- 172.21.15.44:0/2120322043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54f010bca0 con 0x7f54f0100620 2023-09-10T00:38:23.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:23.165+0000 7f54df7fe700 1 -- 172.21.15.44:0/2120322043 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5286 (secure 0 0 0) 0x7f54e00997f0 con 0x7f54f0100620 2023-09-10T00:38:23.166 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:23.166 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24515,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24539,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24547,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-09-10T00:38:23.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:23.168+0000 7f54f4a8a700 1 -- 172.21.15.44:0/2120322043 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f54d80a0690 msgr2=0x7f54d80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:23.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:23.168+0000 7f54f4a8a700 1 --2- 172.21.15.44:0/2120322043 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f54d80a0690 0x7f54d80a2b50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f54e4007be0 tx=0x7f54e40073d0 comp rx=0 tx=0).stop 2023-09-10T00:38:23.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:23.168+0000 7f54f4a8a700 1 -- 172.21.15.44:0/2120322043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f0100620 msgr2=0x7f54f00743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:23.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:23.168+0000 7f54f4a8a700 1 --2- 172.21.15.44:0/2120322043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f0100620 0x7f54f00743b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f54e000ea50 tx=0x7f54e000ed60 comp rx=0 tx=0).stop 2023-09-10T00:38:23.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:23.168+0000 7f54f4a8a700 1 -- 172.21.15.44:0/2120322043 shutdown_connections 2023-09-10T00:38:23.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:23.168+0000 7f54f4a8a700 1 --2- 172.21.15.44:0/2120322043 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f54d80a0690 0x7f54d80a2b50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:23.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:23.168+0000 7f54f4a8a700 1 --2- 172.21.15.44:0/2120322043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f01021b0 0x7f54f0072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:23.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:23.168+0000 7f54f4a8a700 1 --2- 172.21.15.44:0/2120322043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f0100620 0x7f54f00743b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:23.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:23.168+0000 7f54f4a8a700 1 --2- 172.21.15.44:0/2120322043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f0101810 0x7f54f00729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:23.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:23.168+0000 7f54f4a8a700 1 -- 172.21.15.44:0/2120322043 >> 172.21.15.44:0/2120322043 conn(0x7f54f00fbb70 msgr2=0x7f54f0104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:23.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:23.168+0000 7f54f4a8a700 1 -- 172.21.15.44:0/2120322043 shutdown_connections 2023-09-10T00:38:23.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:23.168+0000 7f54f4a8a700 1 -- 172.21.15.44:0/2120322043 wait complete. 2023-09-10T00:38:23.171 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 7 2023-09-10T00:38:23.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:23 smithi131 ceph-mon[103713]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:23.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:23.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:23.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:23.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:23.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:23.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:23.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:23 smithi131 ceph-mon[103713]: Deploying daemon mds.h on smithi099 2023-09-10T00:38:23.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:23 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2120322043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:23.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:23 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.131:6836/2397154745,v1:172.21.15.131:6837/2397154745] up:boot 2023-09-10T00:38:23.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:23 smithi131 ceph-mon[103713]: fsmap 7 up:standby 2023-09-10T00:38:23.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:23 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T00:38:23.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:23 smithi044 ceph-mon[98676]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:23.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:23.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:23.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:23.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:23.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:23.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:23.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:23 smithi044 ceph-mon[98676]: Deploying daemon mds.h on smithi099 2023-09-10T00:38:23.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:23 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2120322043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:23.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:23 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.131:6836/2397154745,v1:172.21.15.131:6837/2397154745] up:boot 2023-09-10T00:38:23.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:23 smithi044 ceph-mon[98676]: fsmap 7 up:standby 2023-09-10T00:38:23.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:23 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T00:38:23.807 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:23 smithi099 ceph-mon[102692]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:23.807 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:23.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:23.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:23.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:23.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:23.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:23.809 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:23 smithi099 ceph-mon[102692]: Deploying daemon mds.h on smithi099 2023-09-10T00:38:23.809 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:23 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2120322043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:23.809 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:23 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.131:6836/2397154745,v1:172.21.15.131:6837/2397154745] up:boot 2023-09-10T00:38:23.809 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:23 smithi099 ceph-mon[102692]: fsmap 7 up:standby 2023-09-10T00:38:23.809 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:23 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T00:38:24.106 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 10 00:38:24 smithi099 systemd[1]: Starting Ceph mds.h for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:38:24.593 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 10 00:38:24 smithi099 podman[138481]: 2023-09-10T00:38:24.594 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 10 00:38:24 smithi099 ceph-mds[138521]: -- 172.21.15.99:0/1474406654 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562d9f47b1e0 con 0x562d9f451000 2023-09-10T00:38:24.594 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 10 00:38:24 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-h[138497]: starting mds.h at 2023-09-10T00:38:24.856 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 10 00:38:24 smithi099 bash[138481]: 45b6bde08473e1f0639a8d783e13382f7c0f57ec0ab0cf4362d3b2d64ff9cb65 2023-09-10T00:38:24.856 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 10 00:38:24 smithi099 systemd[1]: Started Ceph mds.h for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:38:25.181 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:38:25.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.322+0000 7fca97278700 1 -- 172.21.15.44:0/4019954015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca900ff7b0 msgr2=0x7fca900ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:25.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.322+0000 7fca97278700 1 --2- 172.21.15.44:0/4019954015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca900ff7b0 0x7fca900ffc30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fca80009990 tx=0x7fca80009ca0 comp rx=0 tx=0).stop 2023-09-10T00:38:25.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.322+0000 7fca97278700 1 -- 172.21.15.44:0/4019954015 shutdown_connections 2023-09-10T00:38:25.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.322+0000 7fca97278700 1 --2- 172.21.15.44:0/4019954015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca90100170 0x7fca901005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:25.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.322+0000 7fca97278700 1 --2- 172.21.15.44:0/4019954015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca900fee50 0x7fca900ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:25.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.322+0000 7fca97278700 1 --2- 172.21.15.44:0/4019954015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca900ff7b0 0x7fca900ffc30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:25.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.322+0000 7fca97278700 1 -- 172.21.15.44:0/4019954015 >> 172.21.15.44:0/4019954015 conn(0x7fca900faa30 msgr2=0x7fca900fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:25.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.322+0000 7fca97278700 1 -- 172.21.15.44:0/4019954015 shutdown_connections 2023-09-10T00:38:25.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.322+0000 7fca97278700 1 -- 172.21.15.44:0/4019954015 wait complete. 2023-09-10T00:38:25.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.322+0000 7fca97278700 1 Processor -- start 2023-09-10T00:38:25.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca97278700 1 -- start start 2023-09-10T00:38:25.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca97278700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca900fee50 0x7fca90198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:25.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca97278700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca900ff7b0 0x7fca90198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca97278700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca90100170 0x7fca9019f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca97278700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9019ff90 con 0x7fca900ff7b0 2023-09-10T00:38:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca97278700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca901a00d0 con 0x7fca90100170 2023-09-10T00:38:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca97278700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca901a0260 con 0x7fca900fee50 2023-09-10T00:38:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca94813700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca900ff7b0 0x7fca90198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca94813700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca900ff7b0 0x7fca90198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46094/0 (socket says 172.21.15.44:46094) 2023-09-10T00:38:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca94813700 1 -- 172.21.15.44:0/2933757312 learned_addr learned my addr 172.21.15.44:0/2933757312 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca95014700 1 --2- 172.21.15.44:0/2933757312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca900fee50 0x7fca90198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca95815700 1 --2- 172.21.15.44:0/2933757312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca90100170 0x7fca9019f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca94813700 1 -- 172.21.15.44:0/2933757312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca900fee50 msgr2=0x7fca90198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca94813700 1 --2- 172.21.15.44:0/2933757312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca900fee50 0x7fca90198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca94813700 1 -- 172.21.15.44:0/2933757312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca90100170 msgr2=0x7fca9019f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca94813700 1 --2- 172.21.15.44:0/2933757312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca90100170 0x7fca9019f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca94813700 1 -- 172.21.15.44:0/2933757312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca800095f0 con 0x7fca900ff7b0 2023-09-10T00:38:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca94813700 1 --2- 172.21.15.44:0/2933757312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca900ff7b0 0x7fca90198950 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fca800098e0 tx=0x7fca80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.323+0000 7fca867fc700 1 -- 172.21.15.44:0/2933757312 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8001d070 con 0x7fca900ff7b0 2023-09-10T00:38:25.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.324+0000 7fca867fc700 1 -- 172.21.15.44:0/2933757312 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fca8000be30 con 0x7fca900ff7b0 2023-09-10T00:38:25.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.324+0000 7fca97278700 1 -- 172.21.15.44:0/2933757312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca901a04e0 con 0x7fca900ff7b0 2023-09-10T00:38:25.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.324+0000 7fca867fc700 1 -- 172.21.15.44:0/2933757312 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80022db0 con 0x7fca900ff7b0 2023-09-10T00:38:25.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.324+0000 7fca97278700 1 -- 172.21.15.44:0/2933757312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca901a0900 con 0x7fca900ff7b0 2023-09-10T00:38:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.324+0000 7fca97278700 1 -- 172.21.15.44:0/2933757312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca90109c30 con 0x7fca900ff7b0 2023-09-10T00:38:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.326+0000 7fca867fc700 1 -- 172.21.15.44:0/2933757312 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fca80022470 con 0x7fca900ff7b0 2023-09-10T00:38:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.326+0000 7fca867fc700 1 --2- 172.21.15.44:0/2933757312 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fca7c0a0690 0x7fca7c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.327+0000 7fca95014700 1 --2- 172.21.15.44:0/2933757312 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fca7c0a0690 0x7fca7c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.327+0000 7fca867fc700 1 -- 172.21.15.44:0/2933757312 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fca800cd120 con 0x7fca900ff7b0 2023-09-10T00:38:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.327+0000 7fca95014700 1 --2- 172.21.15.44:0/2933757312 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fca7c0a0690 0x7fca7c0a2b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fca8c005950 tx=0x7fca8c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.330+0000 7fca867fc700 1 -- 172.21.15.44:0/2933757312 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fca80097180 con 0x7fca900ff7b0 2023-09-10T00:38:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.579+0000 7fca97278700 1 -- 172.21.15.44:0/2933757312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca9010e110 con 0x7fca900ff7b0 2023-09-10T00:38:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.580+0000 7fca867fc700 1 -- 172.21.15.44:0/2933757312 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6868 (secure 0 0 0) 0x7fca8009a7a0 con 0x7fca900ff7b0 2023-09-10T00:38:25.581 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:25.581 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24515,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24539,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24547,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24556,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24572,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-09-10T00:38:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.582+0000 7fca97278700 1 -- 172.21.15.44:0/2933757312 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fca7c0a0690 msgr2=0x7fca7c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:25.583 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.582+0000 7fca97278700 1 --2- 172.21.15.44:0/2933757312 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fca7c0a0690 0x7fca7c0a2b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fca8c005950 tx=0x7fca8c00b410 comp rx=0 tx=0).stop 2023-09-10T00:38:25.583 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.582+0000 7fca97278700 1 -- 172.21.15.44:0/2933757312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca900ff7b0 msgr2=0x7fca90198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:25.583 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.582+0000 7fca97278700 1 --2- 172.21.15.44:0/2933757312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca900ff7b0 0x7fca90198950 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fca800098e0 tx=0x7fca80004730 comp rx=0 tx=0).stop 2023-09-10T00:38:25.583 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.582+0000 7fca97278700 1 -- 172.21.15.44:0/2933757312 shutdown_connections 2023-09-10T00:38:25.583 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.582+0000 7fca97278700 1 --2- 172.21.15.44:0/2933757312 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fca7c0a0690 0x7fca7c0a2b50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.582+0000 7fca97278700 1 --2- 172.21.15.44:0/2933757312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca90100170 0x7fca9019f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.582+0000 7fca97278700 1 --2- 172.21.15.44:0/2933757312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca900fee50 0x7fca90198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.582+0000 7fca97278700 1 --2- 172.21.15.44:0/2933757312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca900ff7b0 0x7fca90198950 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.582+0000 7fca97278700 1 -- 172.21.15.44:0/2933757312 >> 172.21.15.44:0/2933757312 conn(0x7fca900faa30 msgr2=0x7fca900fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.582+0000 7fca97278700 1 -- 172.21.15.44:0/2933757312 shutdown_connections 2023-09-10T00:38:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:25.582+0000 7fca97278700 1 -- 172.21.15.44:0/2933757312 wait complete. 2023-09-10T00:38:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 9 2023-09-10T00:38:25.959 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:25 smithi131 ceph-mon[103713]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:25.960 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:25 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:25.960 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:25 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:25.960 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:25 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:25.960 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:25 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:25.960 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:25 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:25.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:25 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:25.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:25 smithi131 ceph-mon[103713]: Deploying daemon mds.i on smithi131 2023-09-10T00:38:25.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:25 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] up:boot 2023-09-10T00:38:25.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:25 smithi131 ceph-mon[103713]: fsmap 8 up:standby 2023-09-10T00:38:25.962 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:25 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-10T00:38:25.962 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:25 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2933757312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:26.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:25 smithi044 ceph-mon[98676]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:26.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:25 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:26.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:25 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:26.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:25 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:26.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:25 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:26.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:25 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:26.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:25 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:26.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:25 smithi044 ceph-mon[98676]: Deploying daemon mds.i on smithi131 2023-09-10T00:38:26.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:25 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] up:boot 2023-09-10T00:38:26.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:25 smithi044 ceph-mon[98676]: fsmap 8 up:standby 2023-09-10T00:38:26.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:25 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-10T00:38:26.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:25 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2933757312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:26.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:25 smithi099 ceph-mon[102692]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:26.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:25 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:26.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:25 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:26.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:25 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:26.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:25 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:26.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:25 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:26.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:25 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:26.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:25 smithi099 ceph-mon[102692]: Deploying daemon mds.i on smithi131 2023-09-10T00:38:26.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:25 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] up:boot 2023-09-10T00:38:26.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:25 smithi099 ceph-mon[102692]: fsmap 8 up:standby 2023-09-10T00:38:26.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:25 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-10T00:38:26.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:25 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2933757312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:26.210 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 10 00:38:26 smithi131 systemd[1]: Starting Ceph mds.i for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:38:26.759 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 10 00:38:26 smithi131 podman[141446]: 2023-09-10T00:38:27.045 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 10 00:38:26 smithi131 ceph-mds[141486]: -- 172.21.15.131:0/4097393666 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5591d6a791e0 con 0x5591d6a4e800 2023-09-10T00:38:27.045 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 10 00:38:26 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-i[141462]: starting mds.i at 2023-09-10T00:38:27.045 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 10 00:38:26 smithi131 bash[141446]: d02dd914d8796b505aaf7164aab5ffd03279dff243d2dacfb4e884b49b59459b 2023-09-10T00:38:27.045 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 10 00:38:26 smithi131 systemd[1]: Started Ceph mds.i for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:38:27.595 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:38:27.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.727+0000 7fcead439700 1 -- 172.21.15.44:0/670634705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea80ff4e0 msgr2=0x7fcea80ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:27.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.727+0000 7fcead439700 1 --2- 172.21.15.44:0/670634705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea80ff4e0 0x7fcea80ff900 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fce980099c0 tx=0x7fce98009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:27.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.727+0000 7fcead439700 1 -- 172.21.15.44:0/670634705 shutdown_connections 2023-09-10T00:38:27.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.727+0000 7fcead439700 1 --2- 172.21.15.44:0/670634705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcea8101070 0x7fcea8107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:27.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.727+0000 7fcead439700 1 --2- 172.21.15.44:0/670634705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea80ff4e0 0x7fcea80ff900 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:27.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.727+0000 7fcead439700 1 --2- 172.21.15.44:0/670634705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcea81006d0 0x7fcea8100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:27.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.727+0000 7fcead439700 1 -- 172.21.15.44:0/670634705 >> 172.21.15.44:0/670634705 conn(0x7fcea80faa30 msgr2=0x7fcea80fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:27.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.727+0000 7fcead439700 1 -- 172.21.15.44:0/670634705 shutdown_connections 2023-09-10T00:38:27.730 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.727+0000 7fcead439700 1 -- 172.21.15.44:0/670634705 wait complete. 2023-09-10T00:38:27.730 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.728+0000 7fcead439700 1 Processor -- start 2023-09-10T00:38:27.730 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.728+0000 7fcead439700 1 -- start start 2023-09-10T00:38:27.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.728+0000 7fcead439700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcea80ff4e0 0x7fcea8195120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:27.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.728+0000 7fcead439700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcea81006d0 0x7fcea8195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:27.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.728+0000 7fcead439700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea8101070 0x7fcea819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:27.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.728+0000 7fcead439700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea819cca0 con 0x7fcea8101070 2023-09-10T00:38:27.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.728+0000 7fcead439700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea819cde0 con 0x7fcea81006d0 2023-09-10T00:38:27.734 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.728+0000 7fcead439700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea819cf70 con 0x7fcea80ff4e0 2023-09-10T00:38:27.734 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.728+0000 7fcea77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea8101070 0x7fcea819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:27.734 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.728+0000 7fcea77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea8101070 0x7fcea819c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46110/0 (socket says 172.21.15.44:46110) 2023-09-10T00:38:27.734 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.728+0000 7fcea77fe700 1 -- 172.21.15.44:0/3941674924 learned_addr learned my addr 172.21.15.44:0/3941674924 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:27.734 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.728+0000 7fcea6ffd700 1 --2- 172.21.15.44:0/3941674924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcea80ff4e0 0x7fcea8195120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:27.735 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.728+0000 7fce9ffff700 1 --2- 172.21.15.44:0/3941674924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcea81006d0 0x7fcea8195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:27.735 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.729+0000 7fcea77fe700 1 -- 172.21.15.44:0/3941674924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcea80ff4e0 msgr2=0x7fcea8195120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:27.735 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.729+0000 7fcea77fe700 1 --2- 172.21.15.44:0/3941674924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcea80ff4e0 0x7fcea8195120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:27.735 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.729+0000 7fcea77fe700 1 -- 172.21.15.44:0/3941674924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcea81006d0 msgr2=0x7fcea8195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:27.735 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.729+0000 7fcea77fe700 1 --2- 172.21.15.44:0/3941674924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcea81006d0 0x7fcea8195660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:27.736 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.729+0000 7fcea77fe700 1 -- 172.21.15.44:0/3941674924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce980095f0 con 0x7fcea8101070 2023-09-10T00:38:27.736 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.729+0000 7fcea77fe700 1 --2- 172.21.15.44:0/3941674924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea8101070 0x7fcea819c6d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcea000ada0 tx=0x7fcea000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:27.736 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.729+0000 7fcea4ff9700 1 -- 172.21.15.44:0/3941674924 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea000f070 con 0x7fcea8101070 2023-09-10T00:38:27.736 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.729+0000 7fcead439700 1 -- 172.21.15.44:0/3941674924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea819d250 con 0x7fcea8101070 2023-09-10T00:38:27.736 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.729+0000 7fcea4ff9700 1 -- 172.21.15.44:0/3941674924 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcea0007b30 con 0x7fcea8101070 2023-09-10T00:38:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.729+0000 7fcea4ff9700 1 -- 172.21.15.44:0/3941674924 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea0005280 con 0x7fcea8101070 2023-09-10T00:38:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.729+0000 7fcead439700 1 -- 172.21.15.44:0/3941674924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea819d760 con 0x7fcea8101070 2023-09-10T00:38:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.730+0000 7fcead439700 1 -- 172.21.15.44:0/3941674924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea8106ac0 con 0x7fcea8101070 2023-09-10T00:38:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.731+0000 7fcea4ff9700 1 -- 172.21.15.44:0/3941674924 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fcea000c8d0 con 0x7fcea8101070 2023-09-10T00:38:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.731+0000 7fcea4ff9700 1 --2- 172.21.15.44:0/3941674924 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fce940a0690 0x7fce940a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:27.737 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.732+0000 7fcea6ffd700 1 --2- 172.21.15.44:0/3941674924 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fce940a0690 0x7fce940a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:27.738 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.732+0000 7fcea4ff9700 1 -- 172.21.15.44:0/3941674924 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcea00cac40 con 0x7fcea8101070 2023-09-10T00:38:27.738 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.732+0000 7fcea6ffd700 1 --2- 172.21.15.44:0/3941674924 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fce940a0690 0x7fce940a2b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fce98000c00 tx=0x7fce98009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:27.738 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.736+0000 7fcea4ff9700 1 -- 172.21.15.44:0/3941674924 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcea0094c00 con 0x7fcea8101070 2023-09-10T00:38:27.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:27 smithi044 ceph-mon[98676]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:27.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:27.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:27.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:27.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:27.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:27.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:27 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:27.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.994+0000 7fcead439700 1 -- 172.21.15.44:0/3941674924 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcea810abf0 con 0x7fcea8101070 2023-09-10T00:38:27.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.995+0000 7fcea4ff9700 1 -- 172.21.15.44:0/3941674924 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7664 (secure 0 0 0) 0x7fcea0098220 con 0x7fcea8101070 2023-09-10T00:38:27.996 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:27.996 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24515,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24539,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24547,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24556,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24572,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24574,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/3222652693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":3222652693},{"type":"v1","addr":"172.21.15.131:6839","nonce":3222652693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-09-10T00:38:27.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.997+0000 7fcead439700 1 -- 172.21.15.44:0/3941674924 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fce940a0690 msgr2=0x7fce940a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:27.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.997+0000 7fcead439700 1 --2- 172.21.15.44:0/3941674924 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fce940a0690 0x7fce940a2b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fce98000c00 tx=0x7fce98009f10 comp rx=0 tx=0).stop 2023-09-10T00:38:27.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.997+0000 7fcead439700 1 -- 172.21.15.44:0/3941674924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea8101070 msgr2=0x7fcea819c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:27.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.997+0000 7fcead439700 1 --2- 172.21.15.44:0/3941674924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea8101070 0x7fcea819c6d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcea000ada0 tx=0x7fcea000c330 comp rx=0 tx=0).stop 2023-09-10T00:38:27.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.998+0000 7fcead439700 1 -- 172.21.15.44:0/3941674924 shutdown_connections 2023-09-10T00:38:27.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.998+0000 7fcead439700 1 --2- 172.21.15.44:0/3941674924 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fce940a0690 0x7fce940a2b50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:27.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.998+0000 7fcead439700 1 --2- 172.21.15.44:0/3941674924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcea8101070 0x7fcea819c6d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:27.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.998+0000 7fcead439700 1 --2- 172.21.15.44:0/3941674924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcea80ff4e0 0x7fcea8195120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:27.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.998+0000 7fcead439700 1 --2- 172.21.15.44:0/3941674924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcea81006d0 0x7fcea8195660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:28.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.998+0000 7fcead439700 1 -- 172.21.15.44:0/3941674924 >> 172.21.15.44:0/3941674924 conn(0x7fcea80faa30 msgr2=0x7fcea8103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:28.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.998+0000 7fcead439700 1 -- 172.21.15.44:0/3941674924 shutdown_connections 2023-09-10T00:38:28.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:27.998+0000 7fcead439700 1 -- 172.21.15.44:0/3941674924 wait complete. 2023-09-10T00:38:28.000 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 10 2023-09-10T00:38:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:27 smithi131 ceph-mon[103713]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:27 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:27 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:27 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:27 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:28.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:27 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:28.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:27 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:28.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:27 smithi099 ceph-mon[102692]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:28.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:27 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:28.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:27 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:28.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:27 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:28.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:27 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:28.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:27 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:28.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:27 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:28.687 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 10 00:38:28 smithi044 systemd[1]: Starting Ceph mds.g for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:38:28.940 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 10 00:38:28 smithi044 podman[154689]: 2023-09-10T00:38:28.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:28 smithi044 ceph-mon[98676]: Deploying daemon mds.g on smithi044 2023-09-10T00:38:28.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:28 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] up:boot 2023-09-10T00:38:28.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:28 smithi044 ceph-mon[98676]: fsmap 9 up:standby 2023-09-10T00:38:28.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:28 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-10T00:38:28.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:28 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3941674924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:28 smithi131 ceph-mon[103713]: Deploying daemon mds.g on smithi044 2023-09-10T00:38:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:28 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] up:boot 2023-09-10T00:38:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:28 smithi131 ceph-mon[103713]: fsmap 9 up:standby 2023-09-10T00:38:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:28 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-10T00:38:29.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:28 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3941674924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:28 smithi099 ceph-mon[102692]: Deploying daemon mds.g on smithi044 2023-09-10T00:38:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:28 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] up:boot 2023-09-10T00:38:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:28 smithi099 ceph-mon[102692]: fsmap 9 up:standby 2023-09-10T00:38:29.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:28 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-10T00:38:29.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:28 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3941674924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:29.305 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 10 00:38:28 smithi044 bash[154689]: 8489a55c83cdad7f50e66f6cf33b127770074973ad57fe6f37ff2587cd4d8c3f 2023-09-10T00:38:29.305 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 10 00:38:28 smithi044 ceph-mds[154729]: -- 172.21.15.44:0/276336364 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b0c241f1e0 con 0x55b0c23f4c00 2023-09-10T00:38:29.305 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 10 00:38:28 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-g[154705]: starting mds.g at 2023-09-10T00:38:29.305 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 10 00:38:28 smithi044 systemd[1]: Started Ceph mds.g for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:38:30.011 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:38:30.041 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:29 smithi044 ceph-mon[98676]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:30.041 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:29 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:30.041 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:29 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:30.042 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:29 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:30.042 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:29 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:30.042 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:29 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:30.042 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:29 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:30.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:29 smithi131 ceph-mon[103713]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:29 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:29 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:29 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:29 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:29 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:30.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:29 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:30.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:29 smithi099 ceph-mon[102692]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:30.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:29 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:30.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:29 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:30.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:29 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:30.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:29 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:30.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:29 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:30.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:29 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:30.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.142+0000 7f6d14ba8700 1 -- 172.21.15.44:0/691882136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d101006d0 msgr2=0x7f6d10100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:30.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.142+0000 7f6d14ba8700 1 --2- 172.21.15.44:0/691882136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d101006d0 0x7f6d10100b30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6d040099c0 tx=0x7f6d04009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:30.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.142+0000 7f6d14ba8700 1 -- 172.21.15.44:0/691882136 shutdown_connections 2023-09-10T00:38:30.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.142+0000 7f6d14ba8700 1 --2- 172.21.15.44:0/691882136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d10101070 0x7f6d10107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:30.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.142+0000 7f6d14ba8700 1 --2- 172.21.15.44:0/691882136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d100ff4e0 0x7f6d100ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:30.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.142+0000 7f6d14ba8700 1 --2- 172.21.15.44:0/691882136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d101006d0 0x7f6d10100b30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:30.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.142+0000 7f6d14ba8700 1 -- 172.21.15.44:0/691882136 >> 172.21.15.44:0/691882136 conn(0x7f6d100faa30 msgr2=0x7f6d100fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:30.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.142+0000 7f6d14ba8700 1 -- 172.21.15.44:0/691882136 shutdown_connections 2023-09-10T00:38:30.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.142+0000 7f6d14ba8700 1 -- 172.21.15.44:0/691882136 wait complete. 2023-09-10T00:38:30.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.143+0000 7f6d14ba8700 1 Processor -- start 2023-09-10T00:38:30.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.143+0000 7f6d14ba8700 1 -- start start 2023-09-10T00:38:30.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.143+0000 7f6d14ba8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d100ff4e0 0x7f6d10198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:30.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.143+0000 7f6d14ba8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d101006d0 0x7f6d10198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:30.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.143+0000 7f6d14ba8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d10101070 0x7f6d1019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:30.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.143+0000 7f6d14ba8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1019fea0 con 0x7f6d10101070 2023-09-10T00:38:30.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.143+0000 7f6d14ba8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1019ffe0 con 0x7f6d100ff4e0 2023-09-10T00:38:30.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.143+0000 7f6d14ba8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d101a0170 con 0x7f6d101006d0 2023-09-10T00:38:30.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.143+0000 7f6d0ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d10101070 0x7f6d1019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:30.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.143+0000 7f6d0ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d10101070 0x7f6d1019f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46136/0 (socket says 172.21.15.44:46136) 2023-09-10T00:38:30.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.143+0000 7f6d0ed9d700 1 -- 172.21.15.44:0/313567582 learned_addr learned my addr 172.21.15.44:0/313567582 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:30.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.143+0000 7f6d0e59c700 1 --2- 172.21.15.44:0/313567582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d100ff4e0 0x7f6d10198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:30.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.144+0000 7f6d0dd9b700 1 --2- 172.21.15.44:0/313567582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d101006d0 0x7f6d10198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:30.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.144+0000 7f6d0ed9d700 1 -- 172.21.15.44:0/313567582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d101006d0 msgr2=0x7f6d10198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:30.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.144+0000 7f6d0ed9d700 1 --2- 172.21.15.44:0/313567582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d101006d0 0x7f6d10198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:30.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.144+0000 7f6d0ed9d700 1 -- 172.21.15.44:0/313567582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d100ff4e0 msgr2=0x7f6d10198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:30.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.144+0000 7f6d0ed9d700 1 --2- 172.21.15.44:0/313567582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d100ff4e0 0x7f6d10198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:30.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.144+0000 7f6d0ed9d700 1 -- 172.21.15.44:0/313567582 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d040095f0 con 0x7f6d10101070 2023-09-10T00:38:30.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.144+0000 7f6d0ed9d700 1 --2- 172.21.15.44:0/313567582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d10101070 0x7f6d1019f8d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6d08007d70 tx=0x7f6d0800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:30.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.144+0000 7f6cff7fe700 1 -- 172.21.15.44:0/313567582 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0800ad40 con 0x7f6d10101070 2023-09-10T00:38:30.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.144+0000 7f6d14ba8700 1 -- 172.21.15.44:0/313567582 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d101a0450 con 0x7f6d10101070 2023-09-10T00:38:30.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.144+0000 7f6cff7fe700 1 -- 172.21.15.44:0/313567582 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d080128b0 con 0x7f6d10101070 2023-09-10T00:38:30.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.144+0000 7f6cff7fe700 1 -- 172.21.15.44:0/313567582 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080115c0 con 0x7f6d10101070 2023-09-10T00:38:30.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.144+0000 7f6d14ba8700 1 -- 172.21.15.44:0/313567582 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d101a09b0 con 0x7f6d10101070 2023-09-10T00:38:30.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.145+0000 7f6d14ba8700 1 -- 172.21.15.44:0/313567582 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d10105330 con 0x7f6d10101070 2023-09-10T00:38:30.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.146+0000 7f6cff7fe700 1 -- 172.21.15.44:0/313567582 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f6d08012a20 con 0x7f6d10101070 2023-09-10T00:38:30.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.147+0000 7f6cff7fe700 1 --2- 172.21.15.44:0/313567582 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6cf80a0690 0x7f6cf80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:30.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.147+0000 7f6cff7fe700 1 -- 172.21.15.44:0/313567582 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6d080cb9f0 con 0x7f6d10101070 2023-09-10T00:38:30.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.147+0000 7f6d0e59c700 1 --2- 172.21.15.44:0/313567582 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6cf80a0690 0x7f6cf80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:30.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.147+0000 7f6d0e59c700 1 --2- 172.21.15.44:0/313567582 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6cf80a0690 0x7f6cf80a2b50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6d10100530 tx=0x7f6d0000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:30.154 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.151+0000 7f6cff7fe700 1 -- 172.21.15.44:0/313567582 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6d08095920 con 0x7f6d10101070 2023-09-10T00:38:30.408 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.406+0000 7f6d14ba8700 1 -- 172.21.15.44:0/313567582 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d1010ac00 con 0x7f6d10101070 2023-09-10T00:38:30.408 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.407+0000 7f6cff7fe700 1 -- 172.21.15.44:0/313567582 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8456 (secure 0 0 0) 0x7f6d08098f40 con 0x7f6d10101070 2023-09-10T00:38:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:30.409 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14751,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/3958270510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":3958270510},{"type":"v1","addr":"172.21.15.44:6839","nonce":3958270510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24515,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24539,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24547,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24556,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24572,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24574,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/3222652693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":3222652693},{"type":"v1","addr":"172.21.15.131:6839","nonce":3222652693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-09-10T00:38:30.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.409+0000 7f6d14ba8700 1 -- 172.21.15.44:0/313567582 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6cf80a0690 msgr2=0x7f6cf80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:30.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.409+0000 7f6d14ba8700 1 --2- 172.21.15.44:0/313567582 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6cf80a0690 0x7f6cf80a2b50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6d10100530 tx=0x7f6d0000a3b0 comp rx=0 tx=0).stop 2023-09-10T00:38:30.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.409+0000 7f6d14ba8700 1 -- 172.21.15.44:0/313567582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d10101070 msgr2=0x7f6d1019f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:30.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.409+0000 7f6d14ba8700 1 --2- 172.21.15.44:0/313567582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d10101070 0x7f6d1019f8d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6d08007d70 tx=0x7f6d0800a340 comp rx=0 tx=0).stop 2023-09-10T00:38:30.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.409+0000 7f6d14ba8700 1 -- 172.21.15.44:0/313567582 shutdown_connections 2023-09-10T00:38:30.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.409+0000 7f6d14ba8700 1 --2- 172.21.15.44:0/313567582 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6cf80a0690 0x7f6cf80a2b50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:30.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.409+0000 7f6d14ba8700 1 --2- 172.21.15.44:0/313567582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d10101070 0x7f6d1019f8d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:30.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.409+0000 7f6d14ba8700 1 --2- 172.21.15.44:0/313567582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d100ff4e0 0x7f6d10198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:30.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.409+0000 7f6d14ba8700 1 --2- 172.21.15.44:0/313567582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d101006d0 0x7f6d10198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:30.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.409+0000 7f6d14ba8700 1 -- 172.21.15.44:0/313567582 >> 172.21.15.44:0/313567582 conn(0x7f6d100faa30 msgr2=0x7f6d100fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:30.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.409+0000 7f6d14ba8700 1 -- 172.21.15.44:0/313567582 shutdown_connections 2023-09-10T00:38:30.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:30.409+0000 7f6d14ba8700 1 -- 172.21.15.44:0/313567582 wait complete. 2023-09-10T00:38:30.414 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 11 2023-09-10T00:38:30.668 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 10 00:38:30 smithi044 systemd[1]: Starting Ceph mds.a for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:38:30.937 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:30 smithi044 ceph-mon[98676]: Deploying daemon mds.a on smithi044 2023-09-10T00:38:30.937 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:30 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.44:6838/3958270510,v1:172.21.15.44:6839/3958270510] up:boot 2023-09-10T00:38:30.937 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:30 smithi044 ceph-mon[98676]: fsmap 10 up:standby 2023-09-10T00:38:30.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:30 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-10T00:38:30.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:30 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/313567582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:30 smithi131 ceph-mon[103713]: Deploying daemon mds.a on smithi044 2023-09-10T00:38:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:30 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.44:6838/3958270510,v1:172.21.15.44:6839/3958270510] up:boot 2023-09-10T00:38:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:30 smithi131 ceph-mon[103713]: fsmap 10 up:standby 2023-09-10T00:38:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:30 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-10T00:38:31.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:30 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/313567582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:30 smithi099 ceph-mon[102692]: Deploying daemon mds.a on smithi044 2023-09-10T00:38:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:30 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.44:6838/3958270510,v1:172.21.15.44:6839/3958270510] up:boot 2023-09-10T00:38:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:30 smithi099 ceph-mon[102692]: fsmap 10 up:standby 2023-09-10T00:38:31.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:30 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-10T00:38:31.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:30 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/313567582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:31.257 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 10 00:38:30 smithi044 podman[155073]: 2023-09-10T00:38:31.257 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 10 00:38:31 smithi044 bash[155073]: 8e9860c4f4e2a57e54df3b47ccea5cd89258aef3cf85ca06211ab847c499318e 2023-09-10T00:38:31.257 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 10 00:38:31 smithi044 ceph-mds[155113]: -- 172.21.15.44:0/3841724813 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56205c75d1e0 con 0x56205c732c00 2023-09-10T00:38:31.257 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 10 00:38:31 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-a[155089]: starting mds.a at 2023-09-10T00:38:31.555 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 10 00:38:31 smithi044 systemd[1]: Started Ceph mds.a for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:38:31.974 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:31 smithi099 ceph-mon[102692]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:31.974 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:31 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:31.974 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:31 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:31.975 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:31 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:31.975 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:31 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:31.975 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:31 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:31.975 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:31 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:32.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:31 smithi131 ceph-mon[103713]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:32.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:31 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:32.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:31 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:32.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:31 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:32.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:31 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:32.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:31 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:32.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:31 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:32.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:31 smithi044 ceph-mon[98676]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:32.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:31 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:32.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:31 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:32.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:31 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:32.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:31 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-10T00:38:32.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:31 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-10T00:38:32.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:31 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:32.424 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:38:32.569 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.568+0000 7f1b40bb9700 1 -- 172.21.15.44:0/1721207646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c101800 msgr2=0x7f1b3c101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:32.570 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.568+0000 7f1b40bb9700 1 --2- 172.21.15.44:0/1721207646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c101800 0x7f1b3c101c60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1b300099c0 tx=0x7f1b30009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:32.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.568+0000 7f1b40bb9700 1 -- 172.21.15.44:0/1721207646 shutdown_connections 2023-09-10T00:38:32.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.568+0000 7f1b40bb9700 1 --2- 172.21.15.44:0/1721207646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c1021a0 0x7f1b3c108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:32.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.568+0000 7f1b40bb9700 1 --2- 172.21.15.44:0/1721207646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b3c100610 0x7f1b3c100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:32.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.568+0000 7f1b40bb9700 1 --2- 172.21.15.44:0/1721207646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c101800 0x7f1b3c101c60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:32.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.568+0000 7f1b40bb9700 1 -- 172.21.15.44:0/1721207646 >> 172.21.15.44:0/1721207646 conn(0x7f1b3c0fbb40 msgr2=0x7f1b3c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:32.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.568+0000 7f1b40bb9700 1 -- 172.21.15.44:0/1721207646 shutdown_connections 2023-09-10T00:38:32.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.568+0000 7f1b40bb9700 1 -- 172.21.15.44:0/1721207646 wait complete. 2023-09-10T00:38:32.574 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.568+0000 7f1b40bb9700 1 Processor -- start 2023-09-10T00:38:32.574 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b40bb9700 1 -- start start 2023-09-10T00:38:32.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b40bb9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c100610 0x7f1b3c196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:32.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b40bb9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b3c101800 0x7f1b3c196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:32.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b40bb9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c1021a0 0x7f1b3c19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:32.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b40bb9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c19dcb0 con 0x7f1b3c1021a0 2023-09-10T00:38:32.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b40bb9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c19ddf0 con 0x7f1b3c101800 2023-09-10T00:38:32.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b40bb9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c19df80 con 0x7f1b3c100610 2023-09-10T00:38:32.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b3ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c1021a0 0x7f1b3c19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:32.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b3ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c1021a0 0x7f1b3c19d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46162/0 (socket says 172.21.15.44:46162) 2023-09-10T00:38:32.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b3ad9d700 1 -- 172.21.15.44:0/2978028888 learned_addr learned my addr 172.21.15.44:0/2978028888 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:32.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b3a59c700 1 --2- 172.21.15.44:0/2978028888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c100610 0x7f1b3c196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:32.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b39d9b700 1 --2- 172.21.15.44:0/2978028888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b3c101800 0x7f1b3c196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:32.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b3ad9d700 1 -- 172.21.15.44:0/2978028888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c100610 msgr2=0x7f1b3c196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:32.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b3ad9d700 1 --2- 172.21.15.44:0/2978028888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c100610 0x7f1b3c196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:32.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b3ad9d700 1 -- 172.21.15.44:0/2978028888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b3c101800 msgr2=0x7f1b3c196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:32.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b3ad9d700 1 --2- 172.21.15.44:0/2978028888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b3c101800 0x7f1b3c196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:32.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b3ad9d700 1 -- 172.21.15.44:0/2978028888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b300095f0 con 0x7f1b3c1021a0 2023-09-10T00:38:32.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b3ad9d700 1 --2- 172.21.15.44:0/2978028888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c1021a0 0x7f1b3c19d6e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1b34007ee0 tx=0x7f1b3400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:32.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.569+0000 7f1b2b7fe700 1 -- 172.21.15.44:0/2978028888 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b34017070 con 0x7f1b3c1021a0 2023-09-10T00:38:32.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.570+0000 7f1b2b7fe700 1 -- 172.21.15.44:0/2978028888 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b3400f040 con 0x7f1b3c1021a0 2023-09-10T00:38:32.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.570+0000 7f1b40bb9700 1 -- 172.21.15.44:0/2978028888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b3c19e260 con 0x7f1b3c1021a0 2023-09-10T00:38:32.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.570+0000 7f1b2b7fe700 1 -- 172.21.15.44:0/2978028888 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b34005570 con 0x7f1b3c1021a0 2023-09-10T00:38:32.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.570+0000 7f1b40bb9700 1 -- 172.21.15.44:0/2978028888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3c19e770 con 0x7f1b3c1021a0 2023-09-10T00:38:32.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.570+0000 7f1b40bb9700 1 -- 172.21.15.44:0/2978028888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b3c10bce0 con 0x7f1b3c1021a0 2023-09-10T00:38:32.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.572+0000 7f1b2b7fe700 1 -- 172.21.15.44:0/2978028888 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f1b3400db20 con 0x7f1b3c1021a0 2023-09-10T00:38:32.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.572+0000 7f1b2b7fe700 1 --2- 172.21.15.44:0/2978028888 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b240a0690 0x7f1b240a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:32.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.572+0000 7f1b3a59c700 1 --2- 172.21.15.44:0/2978028888 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b240a0690 0x7f1b240a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:32.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.572+0000 7f1b2b7fe700 1 -- 172.21.15.44:0/2978028888 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1b340cb470 con 0x7f1b3c1021a0 2023-09-10T00:38:32.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.573+0000 7f1b3a59c700 1 --2- 172.21.15.44:0/2978028888 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b240a0690 0x7f1b240a2b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c101660 tx=0x7f1b2c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:32.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.576+0000 7f1b2b7fe700 1 -- 172.21.15.44:0/2978028888 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1b34095440 con 0x7f1b3c1021a0 2023-09-10T00:38:32.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.824+0000 7f1b40bb9700 1 -- 172.21.15.44:0/2978028888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b3c1074f0 con 0x7f1b3c1021a0 2023-09-10T00:38:32.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.825+0000 7f1b2b7fe700 1 -- 172.21.15.44:0/2978028888 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9248 (secure 0 0 0) 0x7f1b34098a60 con 0x7f1b3c1021a0 2023-09-10T00:38:32.826 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:32.827 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14751,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/3958270510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":3958270510},{"type":"v1","addr":"172.21.15.44:6839","nonce":3958270510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1241002552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1241002552},{"type":"v1","addr":"172.21.15.44:6841","nonce":1241002552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24515,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24539,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24547,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24556,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24572,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24574,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/3222652693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":3222652693},{"type":"v1","addr":"172.21.15.131:6839","nonce":3222652693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-09-10T00:38:32.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.827+0000 7f1b40bb9700 1 -- 172.21.15.44:0/2978028888 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b240a0690 msgr2=0x7f1b240a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:32.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.827+0000 7f1b40bb9700 1 --2- 172.21.15.44:0/2978028888 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b240a0690 0x7f1b240a2b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c101660 tx=0x7f1b2c00a3b0 comp rx=0 tx=0).stop 2023-09-10T00:38:32.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.827+0000 7f1b40bb9700 1 -- 172.21.15.44:0/2978028888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c1021a0 msgr2=0x7f1b3c19d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:32.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.827+0000 7f1b40bb9700 1 --2- 172.21.15.44:0/2978028888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c1021a0 0x7f1b3c19d6e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1b34007ee0 tx=0x7f1b3400d3b0 comp rx=0 tx=0).stop 2023-09-10T00:38:32.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.827+0000 7f1b40bb9700 1 -- 172.21.15.44:0/2978028888 shutdown_connections 2023-09-10T00:38:32.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.827+0000 7f1b40bb9700 1 --2- 172.21.15.44:0/2978028888 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b240a0690 0x7f1b240a2b50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:32.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.827+0000 7f1b40bb9700 1 --2- 172.21.15.44:0/2978028888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c1021a0 0x7f1b3c19d6e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:32.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.827+0000 7f1b40bb9700 1 --2- 172.21.15.44:0/2978028888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c100610 0x7f1b3c196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:32.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.827+0000 7f1b40bb9700 1 --2- 172.21.15.44:0/2978028888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b3c101800 0x7f1b3c196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:32.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.827+0000 7f1b40bb9700 1 -- 172.21.15.44:0/2978028888 >> 172.21.15.44:0/2978028888 conn(0x7f1b3c0fbb40 msgr2=0x7f1b3c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:32.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.827+0000 7f1b40bb9700 1 -- 172.21.15.44:0/2978028888 shutdown_connections 2023-09-10T00:38:32.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:32.827+0000 7f1b40bb9700 1 -- 172.21.15.44:0/2978028888 wait complete. 2023-09-10T00:38:32.831 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 12 2023-09-10T00:38:32.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:32 smithi099 ceph-mon[102692]: Deploying daemon mds.b on smithi099 2023-09-10T00:38:32.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:32 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.44:6840/1241002552,v1:172.21.15.44:6841/1241002552] up:boot 2023-09-10T00:38:32.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:32 smithi099 ceph-mon[102692]: fsmap 11 up:standby 2023-09-10T00:38:32.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:32 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-10T00:38:32.857 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 10 00:38:32 smithi099 systemd[1]: Starting Ceph mds.b for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:38:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:32 smithi131 ceph-mon[103713]: Deploying daemon mds.b on smithi099 2023-09-10T00:38:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:32 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.44:6840/1241002552,v1:172.21.15.44:6841/1241002552] up:boot 2023-09-10T00:38:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:32 smithi131 ceph-mon[103713]: fsmap 11 up:standby 2023-09-10T00:38:33.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:32 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-10T00:38:33.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:32 smithi044 ceph-mon[98676]: Deploying daemon mds.b on smithi099 2023-09-10T00:38:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:32 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.44:6840/1241002552,v1:172.21.15.44:6841/1241002552] up:boot 2023-09-10T00:38:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:32 smithi044 ceph-mon[98676]: fsmap 11 up:standby 2023-09-10T00:38:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:32 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-10T00:38:33.390 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 10 00:38:33 smithi099 podman[138830]: 2023-09-10T00:38:33.713 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 10 00:38:33 smithi099 ceph-mds[138870]: -- 172.21.15.99:0/571698445 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f1e66151e0 con 0x55f1e65eac00 2023-09-10T00:38:33.713 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 10 00:38:33 smithi099 bash[138830]: 2e986354a7565dcd75c629389a8472615f6926ccf146822250d90dbd6c15d1c1 2023-09-10T00:38:33.713 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 10 00:38:33 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-b[138846]: starting mds.b at 2023-09-10T00:38:33.713 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 10 00:38:33 smithi099 systemd[1]: Started Ceph mds.b for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:38:34.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:33 smithi131 ceph-mon[103713]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:34.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:33 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2978028888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:34.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:33 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:34.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:33 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:34.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:33 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:34.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:33 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:34.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:33 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:34.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:33 smithi044 ceph-mon[98676]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:34.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:33 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2978028888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:34.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:33 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:34.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:33 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:34.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:33 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:34.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:33 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:34.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:33 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:34.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:33 smithi099 ceph-mon[102692]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:34.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:33 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2978028888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:34.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:33 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:34.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:33 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:34.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:33 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:34.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:33 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:34.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:33 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:34.841 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:38:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.975+0000 7fdfe6750700 1 -- 172.21.15.44:0/129285872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe01017f0 msgr2=0x7fdfe0101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:34.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.975+0000 7fdfe6750700 1 --2- 172.21.15.44:0/129285872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe01017f0 0x7fdfe0101c50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40099c0 tx=0x7fdfd4009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:34.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.975+0000 7fdfe6750700 1 -- 172.21.15.44:0/129285872 shutdown_connections 2023-09-10T00:38:34.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.975+0000 7fdfe6750700 1 --2- 172.21.15.44:0/129285872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe0102190 0x7fdfe0108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:34.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.975+0000 7fdfe6750700 1 --2- 172.21.15.44:0/129285872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfe0100600 0x7fdfe0100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:34.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.975+0000 7fdfe6750700 1 --2- 172.21.15.44:0/129285872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe01017f0 0x7fdfe0101c50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:34.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.975+0000 7fdfe6750700 1 -- 172.21.15.44:0/129285872 >> 172.21.15.44:0/129285872 conn(0x7fdfe00fbb50 msgr2=0x7fdfe00fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:34.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.975+0000 7fdfe6750700 1 -- 172.21.15.44:0/129285872 shutdown_connections 2023-09-10T00:38:34.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.975+0000 7fdfe6750700 1 -- 172.21.15.44:0/129285872 wait complete. 2023-09-10T00:38:34.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.975+0000 7fdfe6750700 1 Processor -- start 2023-09-10T00:38:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfe6750700 1 -- start start 2023-09-10T00:38:34.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfe6750700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe0100600 0x7fdfe019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:34.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfe6750700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe01017f0 0x7fdfe019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:34.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfe6750700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfe0102190 0x7fdfe01a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:34.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfe6750700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe01a43a0 con 0x7fdfe01017f0 2023-09-10T00:38:34.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfe6750700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe01a44e0 con 0x7fdfe0100600 2023-09-10T00:38:34.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfe6750700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe01a4670 con 0x7fdfe0102190 2023-09-10T00:38:34.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfdf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe01017f0 0x7fdfe019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:34.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfdf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe01017f0 0x7fdfe019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46170/0 (socket says 172.21.15.44:46170) 2023-09-10T00:38:34.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfdf7fe700 1 -- 172.21.15.44:0/980573643 learned_addr learned my addr 172.21.15.44:0/980573643 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:34.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfdffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe0100600 0x7fdfe019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:34.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfe4ced700 1 --2- 172.21.15.44:0/980573643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfe0102190 0x7fdfe01a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:34.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfdffff700 1 -- 172.21.15.44:0/980573643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfe0102190 msgr2=0x7fdfe01a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:34.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfdffff700 1 --2- 172.21.15.44:0/980573643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfe0102190 0x7fdfe01a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:34.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfdffff700 1 -- 172.21.15.44:0/980573643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe01017f0 msgr2=0x7fdfe019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:34.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfdffff700 1 --2- 172.21.15.44:0/980573643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe01017f0 0x7fdfe019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:34.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfdffff700 1 -- 172.21.15.44:0/980573643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd40095f0 con 0x7fdfe0100600 2023-09-10T00:38:34.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfe4ced700 1 --2- 172.21.15.44:0/980573643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfe0102190 0x7fdfe01a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:38:34.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.976+0000 7fdfdf7fe700 1 --2- 172.21.15.44:0/980573643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe01017f0 0x7fdfe019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:38:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.977+0000 7fdfdffff700 1 --2- 172.21.15.44:0/980573643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe0100600 0x7fdfe019c820 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000e9c0 tx=0x7fdfd000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.977+0000 7fdfdd7fa700 1 -- 172.21.15.44:0/980573643 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0004220 con 0x7fdfe0100600 2023-09-10T00:38:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.977+0000 7fdfdd7fa700 1 -- 172.21.15.44:0/980573643 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdfd0004380 con 0x7fdfe0100600 2023-09-10T00:38:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.977+0000 7fdfe6750700 1 -- 172.21.15.44:0/980573643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe01a4950 con 0x7fdfe0100600 2023-09-10T00:38:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.977+0000 7fdfdd7fa700 1 -- 172.21.15.44:0/980573643 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0003b40 con 0x7fdfe0100600 2023-09-10T00:38:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.977+0000 7fdfe6750700 1 -- 172.21.15.44:0/980573643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe01a4e60 con 0x7fdfe0100600 2023-09-10T00:38:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.978+0000 7fdfe6750700 1 -- 172.21.15.44:0/980573643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe01055a0 con 0x7fdfe0100600 2023-09-10T00:38:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.979+0000 7fdfdd7fa700 1 -- 172.21.15.44:0/980573643 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fdfd0003ca0 con 0x7fdfe0100600 2023-09-10T00:38:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.979+0000 7fdfdd7fa700 1 --2- 172.21.15.44:0/980573643 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fdfc80a0690 0x7fdfc80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.980+0000 7fdfdf7fe700 1 --2- 172.21.15.44:0/980573643 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fdfc80a0690 0x7fdfc80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.980+0000 7fdfdd7fa700 1 -- 172.21.15.44:0/980573643 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdfd0014070 con 0x7fdfe0100600 2023-09-10T00:38:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.980+0000 7fdfdf7fe700 1 --2- 172.21.15.44:0/980573643 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fdfc80a0690 0x7fdfc80a2b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400b580 tx=0x7fdfd401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:34.983+0000 7fdfdd7fa700 1 -- 172.21.15.44:0/980573643 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fdfd0095b70 con 0x7fdfe0100600 2023-09-10T00:38:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:34 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] up:boot 2023-09-10T00:38:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:34 smithi131 ceph-mon[103713]: fsmap 12 up:standby 2023-09-10T00:38:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:34 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-10T00:38:35.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:34 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:38:35.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:34 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] up:boot 2023-09-10T00:38:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:34 smithi044 ceph-mon[98676]: fsmap 12 up:standby 2023-09-10T00:38:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:34 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-10T00:38:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:34 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:38:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:34 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] up:boot 2023-09-10T00:38:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:34 smithi099 ceph-mon[102692]: fsmap 12 up:standby 2023-09-10T00:38:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:34 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-10T00:38:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:34 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:38:35.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.239+0000 7fdfe6750700 1 -- 172.21.15.44:0/980573643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfe010bcc0 con 0x7fdfe0100600 2023-09-10T00:38:35.242 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.240+0000 7fdfdd7fa700 1 -- 172.21.15.44:0/980573643 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10040 (secure 0 0 0) 0x7fdfd0099190 con 0x7fdfe0100600 2023-09-10T00:38:35.242 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:35.242 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14751,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/3958270510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":3958270510},{"type":"v1","addr":"172.21.15.44:6839","nonce":3958270510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1241002552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1241002552},{"type":"v1","addr":"172.21.15.44:6841","nonce":1241002552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24515,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24539,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24547,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24556,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24572,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24574,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/3222652693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":3222652693},{"type":"v1","addr":"172.21.15.131:6839","nonce":3222652693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24605,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/2622966260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2622966260},{"type":"v1","addr":"172.21.15.99:6839","nonce":2622966260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-09-10T00:38:35.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.243+0000 7fdfe6750700 1 -- 172.21.15.44:0/980573643 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fdfc80a0690 msgr2=0x7fdfc80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:35.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.243+0000 7fdfe6750700 1 --2- 172.21.15.44:0/980573643 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fdfc80a0690 0x7fdfc80a2b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400b580 tx=0x7fdfd401a040 comp rx=0 tx=0).stop 2023-09-10T00:38:35.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.243+0000 7fdfe6750700 1 -- 172.21.15.44:0/980573643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe0100600 msgr2=0x7fdfe019c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:35.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.243+0000 7fdfe6750700 1 --2- 172.21.15.44:0/980573643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe0100600 0x7fdfe019c820 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000e9c0 tx=0x7fdfd000ecd0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.243+0000 7fdfe6750700 1 -- 172.21.15.44:0/980573643 shutdown_connections 2023-09-10T00:38:35.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.243+0000 7fdfe6750700 1 --2- 172.21.15.44:0/980573643 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fdfc80a0690 0x7fdfc80a2b50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.243+0000 7fdfe6750700 1 --2- 172.21.15.44:0/980573643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfe0102190 0x7fdfe01a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.243+0000 7fdfe6750700 1 --2- 172.21.15.44:0/980573643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe0100600 0x7fdfe019c820 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.243+0000 7fdfe6750700 1 --2- 172.21.15.44:0/980573643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe01017f0 0x7fdfe019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.243+0000 7fdfe6750700 1 -- 172.21.15.44:0/980573643 >> 172.21.15.44:0/980573643 conn(0x7fdfe00fbb50 msgr2=0x7fdfe00fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:35.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.243+0000 7fdfe6750700 1 -- 172.21.15.44:0/980573643 shutdown_connections 2023-09-10T00:38:35.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.243+0000 7fdfe6750700 1 -- 172.21.15.44:0/980573643 wait complete. 2023-09-10T00:38:35.246 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 13 2023-09-10T00:38:35.257 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:38:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.387+0000 7f1cb162f700 1 -- 172.21.15.44:0/1561370607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac1010c0 msgr2=0x7f1cac107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.387+0000 7f1cb162f700 1 --2- 172.21.15.44:0/1561370607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac1010c0 0x7f1cac107970 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400b0d0 tx=0x7f1ca400b3e0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.387+0000 7f1cb162f700 1 -- 172.21.15.44:0/1561370607 shutdown_connections 2023-09-10T00:38:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.387+0000 7f1cb162f700 1 --2- 172.21.15.44:0/1561370607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac1010c0 0x7f1cac107970 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.387+0000 7f1cb162f700 1 --2- 172.21.15.44:0/1561370607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cac0ff530 0x7f1cac0ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.387+0000 7f1cb162f700 1 --2- 172.21.15.44:0/1561370607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cac100720 0x7f1cac100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.387+0000 7f1cb162f700 1 -- 172.21.15.44:0/1561370607 >> 172.21.15.44:0/1561370607 conn(0x7f1cac0faa80 msgr2=0x7f1cac0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:35.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.387+0000 7f1cb162f700 1 -- 172.21.15.44:0/1561370607 shutdown_connections 2023-09-10T00:38:35.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.387+0000 7f1cb162f700 1 -- 172.21.15.44:0/1561370607 wait complete. 2023-09-10T00:38:35.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.388+0000 7f1cb162f700 1 Processor -- start 2023-09-10T00:38:35.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.388+0000 7f1cb162f700 1 -- start start 2023-09-10T00:38:35.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.388+0000 7f1cb162f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cac0ff530 0x7f1cac1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:35.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.388+0000 7f1cb162f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cac100720 0x7f1cac196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:35.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.388+0000 7f1cb162f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac1010c0 0x7f1cac19d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:35.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.388+0000 7f1cb162f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac19de50 con 0x7f1cac1010c0 2023-09-10T00:38:35.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.388+0000 7f1cb162f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac19df90 con 0x7f1cac0ff530 2023-09-10T00:38:35.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.388+0000 7f1cb162f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac19e120 con 0x7f1cac100720 2023-09-10T00:38:35.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.388+0000 7f1cab7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac1010c0 0x7f1cac19d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:35.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.388+0000 7f1cab7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac1010c0 0x7f1cac19d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46186/0 (socket says 172.21.15.44:46186) 2023-09-10T00:38:35.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.389+0000 7f1cab7fe700 1 -- 172.21.15.44:0/2726552732 learned_addr learned my addr 172.21.15.44:0/2726552732 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:35.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.389+0000 7f1caa7fc700 1 --2- 172.21.15.44:0/2726552732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cac100720 0x7f1cac196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:35.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.389+0000 7f1caaffd700 1 --2- 172.21.15.44:0/2726552732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cac0ff530 0x7f1cac1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:35.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.389+0000 7f1cab7fe700 1 -- 172.21.15.44:0/2726552732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cac100720 msgr2=0x7f1cac196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:35.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.389+0000 7f1cab7fe700 1 --2- 172.21.15.44:0/2726552732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cac100720 0x7f1cac196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.389+0000 7f1cab7fe700 1 -- 172.21.15.44:0/2726552732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cac0ff530 msgr2=0x7f1cac1962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:35.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.389+0000 7f1cab7fe700 1 --2- 172.21.15.44:0/2726552732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cac0ff530 0x7f1cac1962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.389+0000 7f1cab7fe700 1 -- 172.21.15.44:0/2726552732 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca4009d00 con 0x7f1cac1010c0 2023-09-10T00:38:35.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.389+0000 7f1cab7fe700 1 --2- 172.21.15.44:0/2726552732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac1010c0 0x7f1cac19d880 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009fd0 tx=0x7f1ca4007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:35.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.389+0000 7f1ca3fff700 1 -- 172.21.15.44:0/2726552732 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca400e040 con 0x7f1cac1010c0 2023-09-10T00:38:35.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.389+0000 7f1ca3fff700 1 -- 172.21.15.44:0/2726552732 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ca4007e30 con 0x7f1cac1010c0 2023-09-10T00:38:35.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.389+0000 7f1ca3fff700 1 -- 172.21.15.44:0/2726552732 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca401bd50 con 0x7f1cac1010c0 2023-09-10T00:38:35.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.389+0000 7f1cb162f700 1 -- 172.21.15.44:0/2726552732 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cac19e3a0 con 0x7f1cac1010c0 2023-09-10T00:38:35.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.389+0000 7f1cb162f700 1 -- 172.21.15.44:0/2726552732 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cac19e820 con 0x7f1cac1010c0 2023-09-10T00:38:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.390+0000 7f1cb162f700 1 -- 172.21.15.44:0/2726552732 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac10abf0 con 0x7f1cac1010c0 2023-09-10T00:38:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.391+0000 7f1ca3fff700 1 -- 172.21.15.44:0/2726552732 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f1ca401b450 con 0x7f1cac1010c0 2023-09-10T00:38:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.392+0000 7f1ca3fff700 1 --2- 172.21.15.44:0/2726552732 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1c980a0710 0x7f1c980a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.392+0000 7f1ca3fff700 1 -- 172.21.15.44:0/2726552732 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1ca40ccbf0 con 0x7f1cac1010c0 2023-09-10T00:38:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.392+0000 7f1caaffd700 1 --2- 172.21.15.44:0/2726552732 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1c980a0710 0x7f1c980a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:35.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.392+0000 7f1caaffd700 1 --2- 172.21.15.44:0/2726552732 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1c980a0710 0x7f1c980a2bd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1cac100580 tx=0x7f1c9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:35.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.395+0000 7f1ca3fff700 1 -- 172.21.15.44:0/2726552732 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1ca4096c50 con 0x7f1cac1010c0 2023-09-10T00:38:35.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.620+0000 7f1cb162f700 1 -- 172.21.15.44:0/2726552732 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cac19ea40 con 0x7f1cac1010c0 2023-09-10T00:38:35.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.622+0000 7f1ca3fff700 1 -- 172.21.15.44:0/2726552732 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18332 (secure 0 0 0) 0x7f1ca409a270 con 0x7f1cac1010c0 2023-09-10T00:38:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:35.624 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":79,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:37:58.795397+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:38:35.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.624+0000 7f1cb162f700 1 -- 172.21.15.44:0/2726552732 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1c980a0710 msgr2=0x7f1c980a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:35.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.624+0000 7f1cb162f700 1 --2- 172.21.15.44:0/2726552732 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1c980a0710 0x7f1c980a2bd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1cac100580 tx=0x7f1c9c009260 comp rx=0 tx=0).stop 2023-09-10T00:38:35.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.624+0000 7f1cb162f700 1 -- 172.21.15.44:0/2726552732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac1010c0 msgr2=0x7f1cac19d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:35.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.624+0000 7f1cb162f700 1 --2- 172.21.15.44:0/2726552732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac1010c0 0x7f1cac19d880 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009fd0 tx=0x7f1ca4007b90 comp rx=0 tx=0).stop 2023-09-10T00:38:35.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.624+0000 7f1cb162f700 1 -- 172.21.15.44:0/2726552732 shutdown_connections 2023-09-10T00:38:35.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.624+0000 7f1cb162f700 1 --2- 172.21.15.44:0/2726552732 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1c980a0710 0x7f1c980a2bd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.624+0000 7f1cb162f700 1 --2- 172.21.15.44:0/2726552732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac1010c0 0x7f1cac19d880 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.624+0000 7f1cb162f700 1 --2- 172.21.15.44:0/2726552732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cac0ff530 0x7f1cac1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.624+0000 7f1cb162f700 1 --2- 172.21.15.44:0/2726552732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cac100720 0x7f1cac196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.624+0000 7f1cb162f700 1 -- 172.21.15.44:0/2726552732 >> 172.21.15.44:0/2726552732 conn(0x7f1cac0faa80 msgr2=0x7f1cac068a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:35.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.624+0000 7f1cb162f700 1 -- 172.21.15.44:0/2726552732 shutdown_connections 2023-09-10T00:38:35.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.624+0000 7f1cb162f700 1 -- 172.21.15.44:0/2726552732 wait complete. 2023-09-10T00:38:35.640 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-10T00:34:44.456741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-10T00:38:35.640 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-10T00:38:35.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.771+0000 7f0dc08cc700 1 -- 172.21.15.44:0/3667992128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dbc101870 msgr2=0x7f0dbc101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:35.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.771+0000 7f0dc08cc700 1 --2- 172.21.15.44:0/3667992128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dbc101870 0x7f0dbc101cd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009990 tx=0x7f0db0009ca0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.772+0000 7f0dc08cc700 1 -- 172.21.15.44:0/3667992128 shutdown_connections 2023-09-10T00:38:35.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.772+0000 7f0dc08cc700 1 --2- 172.21.15.44:0/3667992128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dbc102210 0x7f0dbc108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.772+0000 7f0dc08cc700 1 --2- 172.21.15.44:0/3667992128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dbc0fe8a0 0x7f0dbc0fecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.772+0000 7f0dc08cc700 1 --2- 172.21.15.44:0/3667992128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dbc101870 0x7f0dbc101cd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.772+0000 7f0dc08cc700 1 -- 172.21.15.44:0/3667992128 >> 172.21.15.44:0/3667992128 conn(0x7f0dbc0fa1d0 msgr2=0x7f0dbc0fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:35.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.772+0000 7f0dc08cc700 1 -- 172.21.15.44:0/3667992128 shutdown_connections 2023-09-10T00:38:35.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.772+0000 7f0dc08cc700 1 -- 172.21.15.44:0/3667992128 wait complete. 2023-09-10T00:38:35.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.772+0000 7f0dc08cc700 1 Processor -- start 2023-09-10T00:38:35.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.772+0000 7f0dc08cc700 1 -- start start 2023-09-10T00:38:35.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dc08cc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dbc0fe8a0 0x7f0dbc196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:35.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dc08cc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dbc101870 0x7f0dbc196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:35.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dc08cc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dbc102210 0x7f0dbc19d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:35.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dc08cc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc19ddd0 con 0x7f0dbc0fe8a0 2023-09-10T00:38:35.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dc08cc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc19df10 con 0x7f0dbc102210 2023-09-10T00:38:35.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dc08cc700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc19e0a0 con 0x7f0dbc101870 2023-09-10T00:38:35.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dba59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dbc0fe8a0 0x7f0dbc196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:35.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dba59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dbc0fe8a0 0x7f0dbc196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46196/0 (socket says 172.21.15.44:46196) 2023-09-10T00:38:35.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dba59c700 1 -- 172.21.15.44:0/1195013272 learned_addr learned my addr 172.21.15.44:0/1195013272 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:35.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dbad9d700 1 --2- 172.21.15.44:0/1195013272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dbc102210 0x7f0dbc19d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:35.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0db9d9b700 1 --2- 172.21.15.44:0/1195013272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dbc101870 0x7f0dbc196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:35.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dba59c700 1 -- 172.21.15.44:0/1195013272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dbc101870 msgr2=0x7f0dbc196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:35.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dba59c700 1 --2- 172.21.15.44:0/1195013272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dbc101870 0x7f0dbc196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dba59c700 1 -- 172.21.15.44:0/1195013272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dbc102210 msgr2=0x7f0dbc19d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:35.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dba59c700 1 --2- 172.21.15.44:0/1195013272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dbc102210 0x7f0dbc19d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dba59c700 1 -- 172.21.15.44:0/1195013272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db00095f0 con 0x7f0dbc0fe8a0 2023-09-10T00:38:35.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0db9d9b700 1 --2- 172.21.15.44:0/1195013272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dbc101870 0x7f0dbc196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:38:35.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dba59c700 1 --2- 172.21.15.44:0/1195013272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dbc0fe8a0 0x7f0dbc196250 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00d980 tx=0x7f0dac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:35.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dab7fe700 1 -- 172.21.15.44:0/1195013272 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac004020 con 0x7f0dbc0fe8a0 2023-09-10T00:38:35.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.773+0000 7f0dab7fe700 1 -- 172.21.15.44:0/1195013272 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0dac009730 con 0x7f0dbc0fe8a0 2023-09-10T00:38:35.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.774+0000 7f0dc08cc700 1 -- 172.21.15.44:0/1195013272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dbc19e380 con 0x7f0dbc0fe8a0 2023-09-10T00:38:35.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.774+0000 7f0dab7fe700 1 -- 172.21.15.44:0/1195013272 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac003a00 con 0x7f0dbc0fe8a0 2023-09-10T00:38:35.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.774+0000 7f0dc08cc700 1 -- 172.21.15.44:0/1195013272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dbc19e890 con 0x7f0dbc0fe8a0 2023-09-10T00:38:35.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.775+0000 7f0dc08cc700 1 -- 172.21.15.44:0/1195013272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dbc107690 con 0x7f0dbc0fe8a0 2023-09-10T00:38:35.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.776+0000 7f0dab7fe700 1 -- 172.21.15.44:0/1195013272 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f0dac0098a0 con 0x7f0dbc0fe8a0 2023-09-10T00:38:35.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.777+0000 7f0dab7fe700 1 --2- 172.21.15.44:0/1195013272 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0da40a0640 0x7f0da40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:35.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.777+0000 7f0dab7fe700 1 -- 172.21.15.44:0/1195013272 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0dac0cb3c0 con 0x7f0dbc0fe8a0 2023-09-10T00:38:35.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.777+0000 7f0db9d9b700 1 --2- 172.21.15.44:0/1195013272 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0da40a0640 0x7f0da40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:35.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.777+0000 7f0db9d9b700 1 --2- 172.21.15.44:0/1195013272 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0da40a0640 0x7f0da40a2b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0db000b2e0 tx=0x7f0db0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:35.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.781+0000 7f0dab7fe700 1 -- 172.21.15.44:0/1195013272 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0dac095390 con 0x7f0dbc0fe8a0 2023-09-10T00:38:35.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.987+0000 7f0dc08cc700 1 -- 172.21.15.44:0/1195013272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0dbc10bf60 con 0x7f0dbc0fe8a0 2023-09-10T00:38:35.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.988+0000 7f0dab7fe700 1 -- 172.21.15.44:0/1195013272 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0dac0989b0 con 0x7f0dbc0fe8a0 2023-09-10T00:38:35.989 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2023-09-10T00:38:35.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.990+0000 7f0dc08cc700 1 -- 172.21.15.44:0/1195013272 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0da40a0640 msgr2=0x7f0da40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:35.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.990+0000 7f0dc08cc700 1 --2- 172.21.15.44:0/1195013272 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0da40a0640 0x7f0da40a2b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0db000b2e0 tx=0x7f0db0005fd0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.990+0000 7f0dc08cc700 1 -- 172.21.15.44:0/1195013272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dbc0fe8a0 msgr2=0x7f0dbc196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:35.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.990+0000 7f0dc08cc700 1 --2- 172.21.15.44:0/1195013272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dbc0fe8a0 0x7f0dbc196250 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00d980 tx=0x7f0dac00dc90 comp rx=0 tx=0).stop 2023-09-10T00:38:35.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.990+0000 7f0dc08cc700 1 -- 172.21.15.44:0/1195013272 shutdown_connections 2023-09-10T00:38:35.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.990+0000 7f0dc08cc700 1 --2- 172.21.15.44:0/1195013272 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0da40a0640 0x7f0da40a2b00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.990+0000 7f0dc08cc700 1 --2- 172.21.15.44:0/1195013272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dbc102210 0x7f0dbc19d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.990+0000 7f0dc08cc700 1 --2- 172.21.15.44:0/1195013272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dbc0fe8a0 0x7f0dbc196250 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.990+0000 7f0dc08cc700 1 --2- 172.21.15.44:0/1195013272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dbc101870 0x7f0dbc196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:35.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.990+0000 7f0dc08cc700 1 -- 172.21.15.44:0/1195013272 >> 172.21.15.44:0/1195013272 conn(0x7f0dbc0fa1d0 msgr2=0x7f0dbc0fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:35.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.990+0000 7f0dc08cc700 1 -- 172.21.15.44:0/1195013272 shutdown_connections 2023-09-10T00:38:35.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:35.990+0000 7f0dc08cc700 1 -- 172.21.15.44:0/1195013272 wait complete. 2023-09-10T00:38:36.003 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:35 smithi044 ceph-mon[98676]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:36.003 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:35 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/980573643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:36.003 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:35 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2726552732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:36.004 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-10T00:38:36.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:35 smithi131 ceph-mon[103713]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:36.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:35 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/980573643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:36.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:35 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2726552732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:36.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:35 smithi099 ceph-mon[102692]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:36.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:35 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/980573643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:36.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:35 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2726552732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:36.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.146+0000 7fcc5e2b5700 1 -- 172.21.15.44:0/2799091620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc58102130 msgr2=0x7fcc581089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:36.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.146+0000 7fcc5e2b5700 1 --2- 172.21.15.44:0/2799091620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc58102130 0x7fcc581089e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000b0d0 tx=0x7fcc5000b3e0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.146+0000 7fcc5e2b5700 1 -- 172.21.15.44:0/2799091620 shutdown_connections 2023-09-10T00:38:36.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.146+0000 7fcc5e2b5700 1 --2- 172.21.15.44:0/2799091620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc58102130 0x7fcc581089e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.146+0000 7fcc5e2b5700 1 --2- 172.21.15.44:0/2799091620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc581005a0 0x7fcc581009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.146+0000 7fcc5e2b5700 1 --2- 172.21.15.44:0/2799091620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc58101790 0x7fcc58101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.146+0000 7fcc5e2b5700 1 -- 172.21.15.44:0/2799091620 >> 172.21.15.44:0/2799091620 conn(0x7fcc580fbaf0 msgr2=0x7fcc580fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:36.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.146+0000 7fcc5e2b5700 1 -- 172.21.15.44:0/2799091620 shutdown_connections 2023-09-10T00:38:36.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.146+0000 7fcc5e2b5700 1 -- 172.21.15.44:0/2799091620 wait complete. 2023-09-10T00:38:36.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.146+0000 7fcc5e2b5700 1 Processor -- start 2023-09-10T00:38:36.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc5e2b5700 1 -- start start 2023-09-10T00:38:36.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc5e2b5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc581005a0 0x7fcc580743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:36.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc5e2b5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc58101790 0x7fcc580729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:36.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc5e2b5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc58102130 0x7fcc58072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:36.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc5e2b5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc580736d0 con 0x7fcc58101790 2023-09-10T00:38:36.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc5e2b5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc58073840 con 0x7fcc58102130 2023-09-10T00:38:36.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc5e2b5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc581a0330 con 0x7fcc581005a0 2023-09-10T00:38:36.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc58101790 0x7fcc580729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:36.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc58101790 0x7fcc580729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46210/0 (socket says 172.21.15.44:46210) 2023-09-10T00:38:36.154 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc577fe700 1 -- 172.21.15.44:0/3876300341 learned_addr learned my addr 172.21.15.44:0/3876300341 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:36.154 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc5c852700 1 --2- 172.21.15.44:0/3876300341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc58102130 0x7fcc58072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:36.154 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc57fff700 1 --2- 172.21.15.44:0/3876300341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc581005a0 0x7fcc580743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:36.154 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc577fe700 1 -- 172.21.15.44:0/3876300341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc581005a0 msgr2=0x7fcc580743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:36.154 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc577fe700 1 --2- 172.21.15.44:0/3876300341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc581005a0 0x7fcc580743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.155 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc577fe700 1 -- 172.21.15.44:0/3876300341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc58102130 msgr2=0x7fcc58072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:36.155 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc577fe700 1 --2- 172.21.15.44:0/3876300341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc58102130 0x7fcc58072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.155 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc577fe700 1 -- 172.21.15.44:0/3876300341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc50009d00 con 0x7fcc58101790 2023-09-10T00:38:36.155 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.147+0000 7fcc577fe700 1 --2- 172.21.15.44:0/3876300341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc58101790 0x7fcc580729f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00e9a0 tx=0x7fcc4c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:36.156 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.148+0000 7fcc557fa700 1 -- 172.21.15.44:0/3876300341 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c009790 con 0x7fcc58101790 2023-09-10T00:38:36.156 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.148+0000 7fcc557fa700 1 -- 172.21.15.44:0/3876300341 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc4c010430 con 0x7fcc58101790 2023-09-10T00:38:36.157 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.148+0000 7fcc5e2b5700 1 -- 172.21.15.44:0/3876300341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc581a0610 con 0x7fcc58101790 2023-09-10T00:38:36.157 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.148+0000 7fcc557fa700 1 -- 172.21.15.44:0/3876300341 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c00cea0 con 0x7fcc58101790 2023-09-10T00:38:36.157 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.148+0000 7fcc5e2b5700 1 -- 172.21.15.44:0/3876300341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc581a0b20 con 0x7fcc58101790 2023-09-10T00:38:36.157 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.149+0000 7fcc5e2b5700 1 -- 172.21.15.44:0/3876300341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc58107b40 con 0x7fcc58101790 2023-09-10T00:38:36.157 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.150+0000 7fcc557fa700 1 -- 172.21.15.44:0/3876300341 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fcc4c0105a0 con 0x7fcc58101790 2023-09-10T00:38:36.158 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.150+0000 7fcc557fa700 1 --2- 172.21.15.44:0/3876300341 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcc400a0690 0x7fcc400a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:36.158 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.151+0000 7fcc57fff700 1 --2- 172.21.15.44:0/3876300341 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcc400a0690 0x7fcc400a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:36.158 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.151+0000 7fcc557fa700 1 -- 172.21.15.44:0/3876300341 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcc4c014070 con 0x7fcc58101790 2023-09-10T00:38:36.158 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.151+0000 7fcc57fff700 1 --2- 172.21.15.44:0/3876300341 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcc400a0690 0x7fcc400a2b50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcc581015f0 tx=0x7fcc48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:36.159 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.154+0000 7fcc557fa700 1 -- 172.21.15.44:0/3876300341 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcc4c0959f0 con 0x7fcc58101790 2023-09-10T00:38:36.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.395+0000 7fcc5e2b5700 1 -- 172.21.15.44:0/3876300341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcc5810be40 con 0x7fcc58101790 2023-09-10T00:38:36.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.396+0000 7fcc557fa700 1 -- 172.21.15.44:0/3876300341 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcc4c099010 con 0x7fcc58101790 2023-09-10T00:38:36.398 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:36.398 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2023-09-10T00:38:36.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T00:38:36.398 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2023-09-10T00:38:36.399 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2023-09-10T00:38:36.399 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:38:36.399 INFO:teuthology.orchestra.run.smithi044.stdout:] 2023-09-10T00:38:36.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.399+0000 7fcc5e2b5700 1 -- 172.21.15.44:0/3876300341 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcc400a0690 msgr2=0x7fcc400a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:36.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.399+0000 7fcc5e2b5700 1 --2- 172.21.15.44:0/3876300341 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcc400a0690 0x7fcc400a2b50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcc581015f0 tx=0x7fcc48009260 comp rx=0 tx=0).stop 2023-09-10T00:38:36.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.400+0000 7fcc5e2b5700 1 -- 172.21.15.44:0/3876300341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc58101790 msgr2=0x7fcc580729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:36.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.400+0000 7fcc5e2b5700 1 --2- 172.21.15.44:0/3876300341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc58101790 0x7fcc580729f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00e9a0 tx=0x7fcc4c00ecb0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.400+0000 7fcc5e2b5700 1 -- 172.21.15.44:0/3876300341 shutdown_connections 2023-09-10T00:38:36.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.400+0000 7fcc5e2b5700 1 --2- 172.21.15.44:0/3876300341 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcc400a0690 0x7fcc400a2b50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.400+0000 7fcc5e2b5700 1 --2- 172.21.15.44:0/3876300341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc58102130 0x7fcc58072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.400+0000 7fcc5e2b5700 1 --2- 172.21.15.44:0/3876300341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc581005a0 0x7fcc580743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.400+0000 7fcc5e2b5700 1 --2- 172.21.15.44:0/3876300341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc58101790 0x7fcc580729f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.400+0000 7fcc5e2b5700 1 -- 172.21.15.44:0/3876300341 >> 172.21.15.44:0/3876300341 conn(0x7fcc580fbaf0 msgr2=0x7fcc58104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:36.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.400+0000 7fcc5e2b5700 1 -- 172.21.15.44:0/3876300341 shutdown_connections 2023-09-10T00:38:36.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.400+0000 7fcc5e2b5700 1 -- 172.21.15.44:0/3876300341 wait complete. 2023-09-10T00:38:36.415 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-10T00:38:36.415 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-10T00:38:36.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.546+0000 7fd0f761b700 1 -- 172.21.15.44:0/2270917992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0f01062d0 msgr2=0x7fd0f0108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:36.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.546+0000 7fd0f761b700 1 --2- 172.21.15.44:0/2270917992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0f01062d0 0x7fd0f0108740 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800b0a0 tx=0x7fd0e800b3b0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.546+0000 7fd0f761b700 1 -- 172.21.15.44:0/2270917992 shutdown_connections 2023-09-10T00:38:36.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.547+0000 7fd0f761b700 1 --2- 172.21.15.44:0/2270917992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0f01062d0 0x7fd0f0108740 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.547+0000 7fd0f761b700 1 --2- 172.21.15.44:0/2270917992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0f0101070 0x7fd0f0103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.547+0000 7fd0f761b700 1 --2- 172.21.15.44:0/2270917992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f01039a0 0x7fd0f0105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.547+0000 7fd0f761b700 1 -- 172.21.15.44:0/2270917992 >> 172.21.15.44:0/2270917992 conn(0x7fd0f00faa60 msgr2=0x7fd0f00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:36.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.547+0000 7fd0f761b700 1 -- 172.21.15.44:0/2270917992 shutdown_connections 2023-09-10T00:38:36.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.547+0000 7fd0f761b700 1 -- 172.21.15.44:0/2270917992 wait complete. 2023-09-10T00:38:36.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.547+0000 7fd0f761b700 1 Processor -- start 2023-09-10T00:38:36.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.547+0000 7fd0f761b700 1 -- start start 2023-09-10T00:38:36.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.547+0000 7fd0f761b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f0101070 0x7fd0f01962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:36.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.547+0000 7fd0f761b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0f01039a0 0x7fd0f01967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:36.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.547+0000 7fd0f761b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0f01062d0 0x7fd0f019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:36.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.547+0000 7fd0f761b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f019de30 con 0x7fd0f01062d0 2023-09-10T00:38:36.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.547+0000 7fd0f761b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f019df70 con 0x7fd0f0101070 2023-09-10T00:38:36.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.547+0000 7fd0f761b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f019e100 con 0x7fd0f01039a0 2023-09-10T00:38:36.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.547+0000 7fd0f5bb8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0f01062d0 0x7fd0f019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:36.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.548+0000 7fd0f53b7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f0101070 0x7fd0f01962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:36.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.548+0000 7fd0f5bb8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0f01062d0 0x7fd0f019d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46228/0 (socket says 172.21.15.44:46228) 2023-09-10T00:38:36.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.548+0000 7fd0f5bb8700 1 -- 172.21.15.44:0/1004173390 learned_addr learned my addr 172.21.15.44:0/1004173390 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:36.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.548+0000 7fd0f4bb6700 1 --2- 172.21.15.44:0/1004173390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0f01039a0 0x7fd0f01967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:36.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.548+0000 7fd0f5bb8700 1 -- 172.21.15.44:0/1004173390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0f01039a0 msgr2=0x7fd0f01967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:36.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.548+0000 7fd0f5bb8700 1 --2- 172.21.15.44:0/1004173390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0f01039a0 0x7fd0f01967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.548+0000 7fd0f5bb8700 1 -- 172.21.15.44:0/1004173390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f0101070 msgr2=0x7fd0f01962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:36.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.548+0000 7fd0f5bb8700 1 --2- 172.21.15.44:0/1004173390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f0101070 0x7fd0f01962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:36.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.548+0000 7fd0f5bb8700 1 -- 172.21.15.44:0/1004173390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e8009d00 con 0x7fd0f01062d0 2023-09-10T00:38:36.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.548+0000 7fd0f5bb8700 1 --2- 172.21.15.44:0/1004173390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0f01062d0 0x7fd0f019d860 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800d010 tx=0x7fd0e800bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:36.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.548+0000 7fd0e67fc700 1 -- 172.21.15.44:0/1004173390 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e800e040 con 0x7fd0f01062d0 2023-09-10T00:38:36.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.548+0000 7fd0e67fc700 1 -- 172.21.15.44:0/1004173390 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0e8009480 con 0x7fd0f01062d0 2023-09-10T00:38:36.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.548+0000 7fd0f761b700 1 -- 172.21.15.44:0/1004173390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f019e380 con 0x7fd0f01062d0 2023-09-10T00:38:36.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.548+0000 7fd0e67fc700 1 -- 172.21.15.44:0/1004173390 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e80052a0 con 0x7fd0f01062d0 2023-09-10T00:38:36.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.548+0000 7fd0f761b700 1 -- 172.21.15.44:0/1004173390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f019e800 con 0x7fd0f01062d0 2023-09-10T00:38:36.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.549+0000 7fd0f761b700 1 -- 172.21.15.44:0/1004173390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f00fc5e0 con 0x7fd0f01062d0 2023-09-10T00:38:36.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.550+0000 7fd0e67fc700 1 -- 172.21.15.44:0/1004173390 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fd0e8005400 con 0x7fd0f01062d0 2023-09-10T00:38:36.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.551+0000 7fd0e67fc700 1 --2- 172.21.15.44:0/1004173390 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd0dc0a06c0 0x7fd0dc0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:36.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.551+0000 7fd0f53b7700 1 --2- 172.21.15.44:0/1004173390 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd0dc0a06c0 0x7fd0dc0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:36.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.551+0000 7fd0e67fc700 1 -- 172.21.15.44:0/1004173390 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd0e80cce40 con 0x7fd0f01062d0 2023-09-10T00:38:36.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.551+0000 7fd0f53b7700 1 --2- 172.21.15.44:0/1004173390 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd0dc0a06c0 0x7fd0dc0a2b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009c60 tx=0x7fd0ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:36.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.554+0000 7fd0e67fc700 1 -- 172.21.15.44:0/1004173390 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd0e8096ea0 con 0x7fd0f01062d0 2023-09-10T00:38:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:36.755+0000 7fd0f761b700 1 -- 172.21.15.44:0/1004173390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd0f002cfc0 con 0x7fd0f01062d0 2023-09-10T00:38:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:36 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1195013272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:38:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:36 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:36 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:36 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:36 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3876300341' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-10T00:38:37.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:36 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:36 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:36 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:36 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:36 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1195013272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:38:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:36 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:36 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:36 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:36 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3876300341' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-10T00:38:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:36 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:36 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:36 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:36 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:36 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1195013272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:38:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:36 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:36 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:36 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:36 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3876300341' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-10T00:38:37.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:36 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:36 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:36 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:36 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:37.737 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.736+0000 7fd0e67fc700 1 -- 172.21.15.44:0/1004173390 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd0e809a4c0 con 0x7fd0f01062d0 2023-09-10T00:38:37.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.739+0000 7fd0f761b700 1 -- 172.21.15.44:0/1004173390 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd0dc0a06c0 msgr2=0x7fd0dc0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:37.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.739+0000 7fd0f761b700 1 --2- 172.21.15.44:0/1004173390 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd0dc0a06c0 0x7fd0dc0a2b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009c60 tx=0x7fd0ec009260 comp rx=0 tx=0).stop 2023-09-10T00:38:37.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.739+0000 7fd0f761b700 1 -- 172.21.15.44:0/1004173390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0f01062d0 msgr2=0x7fd0f019d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:37.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.739+0000 7fd0f761b700 1 --2- 172.21.15.44:0/1004173390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0f01062d0 0x7fd0f019d860 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800d010 tx=0x7fd0e800bcc0 comp rx=0 tx=0).stop 2023-09-10T00:38:37.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.739+0000 7fd0f761b700 1 -- 172.21.15.44:0/1004173390 shutdown_connections 2023-09-10T00:38:37.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.739+0000 7fd0f761b700 1 --2- 172.21.15.44:0/1004173390 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd0dc0a06c0 0x7fd0dc0a2b80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:37.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.739+0000 7fd0f761b700 1 --2- 172.21.15.44:0/1004173390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0f01062d0 0x7fd0f019d860 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:37.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.739+0000 7fd0f761b700 1 --2- 172.21.15.44:0/1004173390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f0101070 0x7fd0f01962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:37.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.739+0000 7fd0f761b700 1 --2- 172.21.15.44:0/1004173390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0f01039a0 0x7fd0f01967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:37.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.739+0000 7fd0f761b700 1 -- 172.21.15.44:0/1004173390 >> 172.21.15.44:0/1004173390 conn(0x7fd0f00faa60 msgr2=0x7fd0f0101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:37.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.739+0000 7fd0f761b700 1 -- 172.21.15.44:0/1004173390 shutdown_connections 2023-09-10T00:38:37.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.739+0000 7fd0f761b700 1 -- 172.21.15.44:0/1004173390 wait complete. 2023-09-10T00:38:37.743 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2023-09-10T00:38:37.758 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-09-10T00:38:37.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.897+0000 7fbf5d4a0700 1 -- 172.21.15.44:0/44704613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf581006e0 msgr2=0x7fbf58100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:37.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.897+0000 7fbf5d4a0700 1 --2- 172.21.15.44:0/44704613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf581006e0 0x7fbf58100b40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0099c0 tx=0x7fbf4c009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:37.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.898+0000 7fbf5d4a0700 1 -- 172.21.15.44:0/44704613 shutdown_connections 2023-09-10T00:38:37.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.898+0000 7fbf5d4a0700 1 --2- 172.21.15.44:0/44704613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf58101080 0x7fbf58107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:37.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.898+0000 7fbf5d4a0700 1 --2- 172.21.15.44:0/44704613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf580ff4f0 0x7fbf580ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:37.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.898+0000 7fbf5d4a0700 1 --2- 172.21.15.44:0/44704613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf581006e0 0x7fbf58100b40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:37.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.898+0000 7fbf5d4a0700 1 -- 172.21.15.44:0/44704613 >> 172.21.15.44:0/44704613 conn(0x7fbf580faa60 msgr2=0x7fbf580fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:37.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.898+0000 7fbf5d4a0700 1 -- 172.21.15.44:0/44704613 shutdown_connections 2023-09-10T00:38:37.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.898+0000 7fbf5d4a0700 1 -- 172.21.15.44:0/44704613 wait complete. 2023-09-10T00:38:37.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.898+0000 7fbf5d4a0700 1 Processor -- start 2023-09-10T00:38:37.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf5d4a0700 1 -- start start 2023-09-10T00:38:37.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf5d4a0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf580ff4f0 0x7fbf58195110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:37.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf5d4a0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf581006e0 0x7fbf58195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:37.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf5d4a0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf58101080 0x7fbf5819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:37.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf5d4a0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5819cc90 con 0x7fbf581006e0 2023-09-10T00:38:37.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf5d4a0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5819cdd0 con 0x7fbf58101080 2023-09-10T00:38:37.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf5d4a0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5819cf60 con 0x7fbf580ff4f0 2023-09-10T00:38:37.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf567fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf581006e0 0x7fbf58195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:37.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf567fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf581006e0 0x7fbf58195650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46240/0 (socket says 172.21.15.44:46240) 2023-09-10T00:38:37.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf567fc700 1 -- 172.21.15.44:0/1522611815 learned_addr learned my addr 172.21.15.44:0/1522611815 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:37.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf577fe700 1 --2- 172.21.15.44:0/1522611815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf58101080 0x7fbf5819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:37.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf56ffd700 1 --2- 172.21.15.44:0/1522611815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf580ff4f0 0x7fbf58195110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:37.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf567fc700 1 -- 172.21.15.44:0/1522611815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf580ff4f0 msgr2=0x7fbf58195110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:37.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf567fc700 1 --2- 172.21.15.44:0/1522611815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf580ff4f0 0x7fbf58195110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:37.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf567fc700 1 -- 172.21.15.44:0/1522611815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf58101080 msgr2=0x7fbf5819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:37.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf567fc700 1 --2- 172.21.15.44:0/1522611815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf58101080 0x7fbf5819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:37.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf567fc700 1 -- 172.21.15.44:0/1522611815 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c0095f0 con 0x7fbf581006e0 2023-09-10T00:38:37.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.899+0000 7fbf577fe700 1 --2- 172.21.15.44:0/1522611815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf58101080 0x7fbf5819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:38:37.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.900+0000 7fbf567fc700 1 --2- 172.21.15.44:0/1522611815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf581006e0 0x7fbf58195650 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009fd0 tx=0x7fbf4c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:37.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.900+0000 7fbf3ffff700 1 -- 172.21.15.44:0/1522611815 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c01d070 con 0x7fbf581006e0 2023-09-10T00:38:37.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.900+0000 7fbf5d4a0700 1 -- 172.21.15.44:0/1522611815 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf58067760 con 0x7fbf581006e0 2023-09-10T00:38:37.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.900+0000 7fbf3ffff700 1 -- 172.21.15.44:0/1522611815 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf4c00fe70 con 0x7fbf581006e0 2023-09-10T00:38:37.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.900+0000 7fbf3ffff700 1 -- 172.21.15.44:0/1522611815 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c022e90 con 0x7fbf581006e0 2023-09-10T00:38:37.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.900+0000 7fbf5d4a0700 1 -- 172.21.15.44:0/1522611815 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf58067c60 con 0x7fbf581006e0 2023-09-10T00:38:37.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.901+0000 7fbf5d4a0700 1 -- 172.21.15.44:0/1522611815 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf58068650 con 0x7fbf581006e0 2023-09-10T00:38:37.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.903+0000 7fbf3ffff700 1 -- 172.21.15.44:0/1522611815 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fbf4c022470 con 0x7fbf581006e0 2023-09-10T00:38:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.903+0000 7fbf3ffff700 1 --2- 172.21.15.44:0/1522611815 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbf400a0640 0x7fbf400a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.903+0000 7fbf3ffff700 1 -- 172.21.15.44:0/1522611815 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fbf4c0cc910 con 0x7fbf581006e0 2023-09-10T00:38:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.903+0000 7fbf56ffd700 1 --2- 172.21.15.44:0/1522611815 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbf400a0640 0x7fbf400a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.904+0000 7fbf56ffd700 1 --2- 172.21.15.44:0/1522611815 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbf400a0640 0x7fbf400a2b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbf58100540 tx=0x7fbf48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:37.908+0000 7fbf3ffff700 1 -- 172.21.15.44:0/1522611815 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbf4c096800 con 0x7fbf581006e0 2023-09-10T00:38:38.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:37 smithi131 ceph-mon[103713]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:38.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:37 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1004173390' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-10T00:38:38.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:38.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:38:38.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:38.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:37 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:38:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:37 smithi044 ceph-mon[98676]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:37 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1004173390' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-10T00:38:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:38:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:38.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:37 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:38:38.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:37 smithi099 ceph-mon[102692]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:38.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:37 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1004173390' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-10T00:38:38.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:38.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:38:38.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:38.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:37 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:38:38.121 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.120+0000 7fbf5d4a0700 1 -- 172.21.15.44:0/1522611815 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fbf58105fe0 con 0x7fbf581006e0 2023-09-10T00:38:38.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.741+0000 7fbf3ffff700 1 -- 172.21.15.44:0/1522611815 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v81) v1 ==== 171+0+0 (secure 0 0 0) 0x7fbf4c022720 con 0x7fbf581006e0 2023-09-10T00:38:38.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.748+0000 7fbf5d4a0700 1 -- 172.21.15.44:0/1522611815 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbf400a0640 msgr2=0x7fbf400a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:38.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.748+0000 7fbf5d4a0700 1 --2- 172.21.15.44:0/1522611815 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbf400a0640 0x7fbf400a2b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbf58100540 tx=0x7fbf48009260 comp rx=0 tx=0).stop 2023-09-10T00:38:38.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.748+0000 7fbf5d4a0700 1 -- 172.21.15.44:0/1522611815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf581006e0 msgr2=0x7fbf58195650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:38.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.748+0000 7fbf5d4a0700 1 --2- 172.21.15.44:0/1522611815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf581006e0 0x7fbf58195650 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009fd0 tx=0x7fbf4c00fa00 comp rx=0 tx=0).stop 2023-09-10T00:38:38.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.750+0000 7fbf5d4a0700 1 -- 172.21.15.44:0/1522611815 shutdown_connections 2023-09-10T00:38:38.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.750+0000 7fbf5d4a0700 1 --2- 172.21.15.44:0/1522611815 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbf400a0640 0x7fbf400a2b00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:38.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.750+0000 7fbf5d4a0700 1 --2- 172.21.15.44:0/1522611815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf58101080 0x7fbf5819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:38.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.750+0000 7fbf5d4a0700 1 --2- 172.21.15.44:0/1522611815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf580ff4f0 0x7fbf58195110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:38.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.750+0000 7fbf5d4a0700 1 --2- 172.21.15.44:0/1522611815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf581006e0 0x7fbf58195650 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.750+0000 7fbf5d4a0700 1 -- 172.21.15.44:0/1522611815 >> 172.21.15.44:0/1522611815 conn(0x7fbf580faa60 msgr2=0x7fbf580fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.752+0000 7fbf5d4a0700 1 -- 172.21.15.44:0/1522611815 shutdown_connections 2023-09-10T00:38:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.754+0000 7fbf5d4a0700 1 -- 172.21.15.44:0/1522611815 wait complete. 2023-09-10T00:38:38.761 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2023-09-10T00:38:38.774 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-10T00:38:38.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.912+0000 7f0e4cc8b700 1 -- 172.21.15.44:0/2833408099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e481006e0 msgr2=0x7f0e48100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:38.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.912+0000 7f0e4cc8b700 1 --2- 172.21.15.44:0/2833408099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e481006e0 0x7f0e48100b40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0e300099c0 tx=0x7f0e30009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:38.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.912+0000 7f0e4cc8b700 1 -- 172.21.15.44:0/2833408099 shutdown_connections 2023-09-10T00:38:38.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.912+0000 7f0e4cc8b700 1 --2- 172.21.15.44:0/2833408099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e48101080 0x7f0e48107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:38.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.912+0000 7f0e4cc8b700 1 --2- 172.21.15.44:0/2833408099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e480ff4f0 0x7f0e480ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:38.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.912+0000 7f0e4cc8b700 1 --2- 172.21.15.44:0/2833408099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e481006e0 0x7f0e48100b40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:38.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.912+0000 7f0e4cc8b700 1 -- 172.21.15.44:0/2833408099 >> 172.21.15.44:0/2833408099 conn(0x7f0e480faa60 msgr2=0x7f0e480fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:38.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.912+0000 7f0e4cc8b700 1 -- 172.21.15.44:0/2833408099 shutdown_connections 2023-09-10T00:38:38.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.912+0000 7f0e4cc8b700 1 -- 172.21.15.44:0/2833408099 wait complete. 2023-09-10T00:38:38.917 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.913+0000 7f0e4cc8b700 1 Processor -- start 2023-09-10T00:38:38.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.913+0000 7f0e4cc8b700 1 -- start start 2023-09-10T00:38:38.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.913+0000 7f0e4cc8b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e480ff4f0 0x7f0e48196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:38.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.913+0000 7f0e4cc8b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e481006e0 0x7f0e48196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:38.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.913+0000 7f0e4cc8b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e48101080 0x7f0e4819d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:38.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.913+0000 7f0e4cc8b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4819ddb0 con 0x7f0e481006e0 2023-09-10T00:38:38.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.913+0000 7f0e4cc8b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4819def0 con 0x7f0e48101080 2023-09-10T00:38:38.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.913+0000 7f0e4cc8b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4819e080 con 0x7f0e480ff4f0 2023-09-10T00:38:38.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.913+0000 7f0e45d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e481006e0 0x7f0e48196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:38.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.913+0000 7f0e45d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e481006e0 0x7f0e48196770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46262/0 (socket says 172.21.15.44:46262) 2023-09-10T00:38:38.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.913+0000 7f0e45d9b700 1 -- 172.21.15.44:0/45697749 learned_addr learned my addr 172.21.15.44:0/45697749 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:38.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.913+0000 7f0e4659c700 1 --2- 172.21.15.44:0/45697749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e480ff4f0 0x7f0e48196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:38.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.913+0000 7f0e46d9d700 1 --2- 172.21.15.44:0/45697749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e48101080 0x7f0e4819d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:38.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.914+0000 7f0e45d9b700 1 -- 172.21.15.44:0/45697749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e480ff4f0 msgr2=0x7f0e48196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:38.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.914+0000 7f0e45d9b700 1 --2- 172.21.15.44:0/45697749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e480ff4f0 0x7f0e48196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:38.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.914+0000 7f0e45d9b700 1 -- 172.21.15.44:0/45697749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e48101080 msgr2=0x7f0e4819d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:38.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.914+0000 7f0e45d9b700 1 --2- 172.21.15.44:0/45697749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e48101080 0x7f0e4819d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:38.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.914+0000 7f0e45d9b700 1 -- 172.21.15.44:0/45697749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e300095f0 con 0x7f0e481006e0 2023-09-10T00:38:38.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.914+0000 7f0e45d9b700 1 --2- 172.21.15.44:0/45697749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e481006e0 0x7f0e48196770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0e30009fd0 tx=0x7f0e3000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:38.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.914+0000 7f0e3f7fe700 1 -- 172.21.15.44:0/45697749 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3001d070 con 0x7f0e481006e0 2023-09-10T00:38:38.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.914+0000 7f0e4cc8b700 1 -- 172.21.15.44:0/45697749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4819e300 con 0x7f0e481006e0 2023-09-10T00:38:38.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.914+0000 7f0e3f7fe700 1 -- 172.21.15.44:0/45697749 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e3000fe70 con 0x7f0e481006e0 2023-09-10T00:38:38.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.914+0000 7f0e3f7fe700 1 -- 172.21.15.44:0/45697749 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e300173f0 con 0x7f0e481006e0 2023-09-10T00:38:38.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.914+0000 7f0e4cc8b700 1 -- 172.21.15.44:0/45697749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4819e7b0 con 0x7f0e481006e0 2023-09-10T00:38:38.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.915+0000 7f0e4cc8b700 1 -- 172.21.15.44:0/45697749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e48107760 con 0x7f0e481006e0 2023-09-10T00:38:38.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.917+0000 7f0e3f7fe700 1 -- 172.21.15.44:0/45697749 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f0e30022470 con 0x7f0e481006e0 2023-09-10T00:38:38.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.917+0000 7f0e3f7fe700 1 --2- 172.21.15.44:0/45697749 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0e340a0640 0x7f0e340a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:38.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.917+0000 7f0e4659c700 1 --2- 172.21.15.44:0/45697749 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0e340a0640 0x7f0e340a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:38.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.917+0000 7f0e3f7fe700 1 -- 172.21.15.44:0/45697749 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f0e300ccba0 con 0x7f0e481006e0 2023-09-10T00:38:38.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.918+0000 7f0e4659c700 1 --2- 172.21.15.44:0/45697749 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0e340a0640 0x7f0e340a2b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e48100540 tx=0x7f0e38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:38.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:38.921+0000 7f0e3f7fe700 1 -- 172.21.15.44:0/45697749 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0e30096a30 con 0x7f0e481006e0 2023-09-10T00:38:39.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:38 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1004173390' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-10T00:38:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:38 smithi131 ceph-mon[103713]: osdmap e80: 12 total, 12 up, 12 in 2023-09-10T00:38:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:38 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1522611815' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-10T00:38:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:38 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1004173390' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-10T00:38:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:38 smithi044 ceph-mon[98676]: osdmap e80: 12 total, 12 up, 12 in 2023-09-10T00:38:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:38 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1522611815' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-10T00:38:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:38 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1004173390' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-10T00:38:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:38 smithi099 ceph-mon[102692]: osdmap e80: 12 total, 12 up, 12 in 2023-09-10T00:38:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:38 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1522611815' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-10T00:38:39.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.181+0000 7f0e4cc8b700 1 -- 172.21.15.44:0/45697749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0e4810add0 con 0x7f0e481006e0 2023-09-10T00:38:39.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.743+0000 7f0e3f7fe700 1 -- 172.21.15.44:0/45697749 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0e3009a050 con 0x7f0e481006e0 2023-09-10T00:38:39.746 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.745+0000 7f0e4cc8b700 1 -- 172.21.15.44:0/45697749 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0e340a0640 msgr2=0x7f0e340a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:39.746 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.745+0000 7f0e4cc8b700 1 --2- 172.21.15.44:0/45697749 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0e340a0640 0x7f0e340a2b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e48100540 tx=0x7f0e38009260 comp rx=0 tx=0).stop 2023-09-10T00:38:39.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.745+0000 7f0e4cc8b700 1 -- 172.21.15.44:0/45697749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e481006e0 msgr2=0x7f0e48196770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:39.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.745+0000 7f0e4cc8b700 1 --2- 172.21.15.44:0/45697749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e481006e0 0x7f0e48196770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0e30009fd0 tx=0x7f0e3000fa00 comp rx=0 tx=0).stop 2023-09-10T00:38:39.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.745+0000 7f0e4cc8b700 1 -- 172.21.15.44:0/45697749 shutdown_connections 2023-09-10T00:38:39.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.745+0000 7f0e4cc8b700 1 --2- 172.21.15.44:0/45697749 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0e340a0640 0x7f0e340a2b00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:39.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.745+0000 7f0e4cc8b700 1 --2- 172.21.15.44:0/45697749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e48101080 0x7f0e4819d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:39.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.745+0000 7f0e4cc8b700 1 --2- 172.21.15.44:0/45697749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e480ff4f0 0x7f0e48196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:39.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.745+0000 7f0e4cc8b700 1 --2- 172.21.15.44:0/45697749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e481006e0 0x7f0e48196770 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:39.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.745+0000 7f0e4cc8b700 1 -- 172.21.15.44:0/45697749 >> 172.21.15.44:0/45697749 conn(0x7f0e480faa60 msgr2=0x7f0e480fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:39.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.746+0000 7f0e4cc8b700 1 -- 172.21.15.44:0/45697749 shutdown_connections 2023-09-10T00:38:39.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.747+0000 7f0e4cc8b700 1 -- 172.21.15.44:0/45697749 wait complete. 2023-09-10T00:38:39.750 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 2 and data pool 3 2023-09-10T00:38:39.786 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-09-10T00:38:39.786 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-09-10T00:38:39.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.920+0000 7f6ad1d67700 1 -- 172.21.15.44:0/271184968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6acc1017e0 msgr2=0x7f6acc101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:39.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.920+0000 7f6ad1d67700 1 --2- 172.21.15.44:0/271184968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6acc1017e0 0x7f6acc101c40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6ac00098f0 tx=0x7f6ac0009c00 comp rx=0 tx=0).stop 2023-09-10T00:38:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.920+0000 7f6ad1d67700 1 -- 172.21.15.44:0/271184968 shutdown_connections 2023-09-10T00:38:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.920+0000 7f6ad1d67700 1 --2- 172.21.15.44:0/271184968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6acc102180 0x7f6acc108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.920+0000 7f6ad1d67700 1 --2- 172.21.15.44:0/271184968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6acc1005f0 0x7f6acc100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.920+0000 7f6ad1d67700 1 --2- 172.21.15.44:0/271184968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6acc1017e0 0x7f6acc101c40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.920+0000 7f6ad1d67700 1 -- 172.21.15.44:0/271184968 >> 172.21.15.44:0/271184968 conn(0x7f6acc0fbb80 msgr2=0x7f6acc0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:39.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.921+0000 7f6ad1d67700 1 -- 172.21.15.44:0/271184968 shutdown_connections 2023-09-10T00:38:39.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.921+0000 7f6ad1d67700 1 -- 172.21.15.44:0/271184968 wait complete. 2023-09-10T00:38:39.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.921+0000 7f6ad1d67700 1 Processor -- start 2023-09-10T00:38:39.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.921+0000 7f6ad1d67700 1 -- start start 2023-09-10T00:38:39.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.921+0000 7f6ad1d67700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6acc1005f0 0x7f6acc196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:39.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6ad1d67700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6acc1017e0 0x7f6acc1967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6ad1d67700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6acc102180 0x7f6acc19d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6ad1d67700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc19ddf0 con 0x7f6acc102180 2023-09-10T00:38:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6ad1d67700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc19df30 con 0x7f6acc1005f0 2023-09-10T00:38:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6ad1d67700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc19e0c0 con 0x7f6acc1017e0 2023-09-10T00:38:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6acbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6acc102180 0x7f6acc19d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:39.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6acbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6acc102180 0x7f6acc19d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46278/0 (socket says 172.21.15.44:46278) 2023-09-10T00:38:39.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6acbfff700 1 -- 172.21.15.44:0/1211259285 learned_addr learned my addr 172.21.15.44:0/1211259285 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:39.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6acaffd700 1 --2- 172.21.15.44:0/1211259285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6acc1017e0 0x7f6acc1967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:39.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6acb7fe700 1 --2- 172.21.15.44:0/1211259285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6acc1005f0 0x7f6acc196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:39.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6acbfff700 1 -- 172.21.15.44:0/1211259285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6acc1017e0 msgr2=0x7f6acc1967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:39.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6acbfff700 1 --2- 172.21.15.44:0/1211259285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6acc1017e0 0x7f6acc1967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:39.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6acbfff700 1 -- 172.21.15.44:0/1211259285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6acc1005f0 msgr2=0x7f6acc196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:39.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6acbfff700 1 --2- 172.21.15.44:0/1211259285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6acc1005f0 0x7f6acc196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:39.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6acbfff700 1 -- 172.21.15.44:0/1211259285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc0095f0 con 0x7f6acc102180 2023-09-10T00:38:39.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6acb7fe700 1 --2- 172.21.15.44:0/1211259285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6acc1005f0 0x7f6acc196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:38:39.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6acbfff700 1 --2- 172.21.15.44:0/1211259285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6acc102180 0x7f6acc19d820 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400ef30 tx=0x7f6ac400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:39.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6ac8ff9700 1 -- 172.21.15.44:0/1211259285 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4018070 con 0x7f6acc102180 2023-09-10T00:38:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6ac8ff9700 1 -- 172.21.15.44:0/1211259285 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ac400f040 con 0x7f6acc102180 2023-09-10T00:38:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.922+0000 7f6ad1d67700 1 -- 172.21.15.44:0/1211259285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac0009520 con 0x7f6acc102180 2023-09-10T00:38:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.923+0000 7f6ac8ff9700 1 -- 172.21.15.44:0/1211259285 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac400cc70 con 0x7f6acc102180 2023-09-10T00:38:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.923+0000 7f6ad1d67700 1 -- 172.21.15.44:0/1211259285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6acc19e6c0 con 0x7f6acc102180 2023-09-10T00:38:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.924+0000 7f6ad1d67700 1 -- 172.21.15.44:0/1211259285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6acc10bcc0 con 0x7f6acc102180 2023-09-10T00:38:39.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.925+0000 7f6ac8ff9700 1 -- 172.21.15.44:0/1211259285 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f6ac4007500 con 0x7f6acc102180 2023-09-10T00:38:39.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.926+0000 7f6ac8ff9700 1 --2- 172.21.15.44:0/1211259285 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6ab40a0640 0x7f6ab40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:39.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.926+0000 7f6ac8ff9700 1 -- 172.21.15.44:0/1211259285 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6ac40cb5a0 con 0x7f6acc102180 2023-09-10T00:38:39.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.926+0000 7f6acb7fe700 1 --2- 172.21.15.44:0/1211259285 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6ab40a0640 0x7f6ab40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:39.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.926+0000 7f6acb7fe700 1 --2- 172.21.15.44:0/1211259285 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6ab40a0640 0x7f6ab40a2b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6acc101640 tx=0x7f6abc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:39.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:39.931+0000 7f6ac8ff9700 1 -- 172.21.15.44:0/1211259285 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6ac40952b0 con 0x7f6acc102180 2023-09-10T00:38:40.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:39 smithi131 ceph-mon[103713]: pgmap v261: 33 pgs: 4 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:40.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:39 smithi131 ceph-mon[103713]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-10T00:38:40.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:39 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1522611815' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-09-10T00:38:40.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:39 smithi131 ceph-mon[103713]: osdmap e81: 12 total, 12 up, 12 in 2023-09-10T00:38:40.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:39 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/45697749' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-10T00:38:40.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:39 smithi131 ceph-mon[103713]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-10T00:38:40.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:39 smithi044 ceph-mon[98676]: pgmap v261: 33 pgs: 4 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:39 smithi044 ceph-mon[98676]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-10T00:38:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:39 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1522611815' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-09-10T00:38:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:39 smithi044 ceph-mon[98676]: osdmap e81: 12 total, 12 up, 12 in 2023-09-10T00:38:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:39 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/45697749' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-10T00:38:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:39 smithi044 ceph-mon[98676]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-10T00:38:40.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:39 smithi099 ceph-mon[102692]: pgmap v261: 33 pgs: 4 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:40.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:39 smithi099 ceph-mon[102692]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-10T00:38:40.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:39 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1522611815' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-09-10T00:38:40.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:39 smithi099 ceph-mon[102692]: osdmap e81: 12 total, 12 up, 12 in 2023-09-10T00:38:40.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:39 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/45697749' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-10T00:38:40.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:39 smithi099 ceph-mon[102692]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-10T00:38:40.160 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.158+0000 7f6ad1d67700 1 -- 172.21.15.44:0/1211259285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f6acc10bea0 con 0x7f6acc102180 2023-09-10T00:38:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.755+0000 7f6ac8ff9700 1 -- 172.21.15.44:0/1211259285 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v83) v1 ==== 157+0+0 (secure 0 0 0) 0x7f6ac40988d0 con 0x7f6acc102180 2023-09-10T00:38:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.759+0000 7f6ad1d67700 1 -- 172.21.15.44:0/1211259285 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6ab40a0640 msgr2=0x7f6ab40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.759+0000 7f6ad1d67700 1 --2- 172.21.15.44:0/1211259285 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6ab40a0640 0x7f6ab40a2b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6acc101640 tx=0x7f6abc009310 comp rx=0 tx=0).stop 2023-09-10T00:38:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.759+0000 7f6ad1d67700 1 -- 172.21.15.44:0/1211259285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6acc102180 msgr2=0x7f6acc19d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.759+0000 7f6ad1d67700 1 --2- 172.21.15.44:0/1211259285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6acc102180 0x7f6acc19d820 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400ef30 tx=0x7f6ac400c3a0 comp rx=0 tx=0).stop 2023-09-10T00:38:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.759+0000 7f6ad1d67700 1 -- 172.21.15.44:0/1211259285 shutdown_connections 2023-09-10T00:38:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.759+0000 7f6ad1d67700 1 --2- 172.21.15.44:0/1211259285 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6ab40a0640 0x7f6ab40a2b00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.759+0000 7f6ad1d67700 1 --2- 172.21.15.44:0/1211259285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6acc102180 0x7f6acc19d820 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.759+0000 7f6ad1d67700 1 --2- 172.21.15.44:0/1211259285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6acc1005f0 0x7f6acc196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.759+0000 7f6ad1d67700 1 --2- 172.21.15.44:0/1211259285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6acc1017e0 0x7f6acc1967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.759+0000 7f6ad1d67700 1 -- 172.21.15.44:0/1211259285 >> 172.21.15.44:0/1211259285 conn(0x7f6acc0fbb80 msgr2=0x7f6acc0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.760+0000 7f6ad1d67700 1 -- 172.21.15.44:0/1211259285 shutdown_connections 2023-09-10T00:38:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.760+0000 7f6ad1d67700 1 -- 172.21.15.44:0/1211259285 wait complete. 2023-09-10T00:38:40.776 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-09-10T00:38:40.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.915+0000 7f3f87c5b700 1 -- 172.21.15.44:0/1288017487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f801021b0 msgr2=0x7f3f80108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:40.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.915+0000 7f3f87c5b700 1 --2- 172.21.15.44:0/1288017487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f801021b0 0x7f3f80108a60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800b0d0 tx=0x7f3f7800b3e0 comp rx=0 tx=0).stop 2023-09-10T00:38:40.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.915+0000 7f3f87c5b700 1 -- 172.21.15.44:0/1288017487 shutdown_connections 2023-09-10T00:38:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.915+0000 7f3f87c5b700 1 --2- 172.21.15.44:0/1288017487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f801021b0 0x7f3f80108a60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.915+0000 7f3f87c5b700 1 --2- 172.21.15.44:0/1288017487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f80100620 0x7f3f80100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.915+0000 7f3f87c5b700 1 --2- 172.21.15.44:0/1288017487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f80101810 0x7f3f80101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.915+0000 7f3f87c5b700 1 -- 172.21.15.44:0/1288017487 >> 172.21.15.44:0/1288017487 conn(0x7f3f800fbb70 msgr2=0x7f3f800fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.915+0000 7f3f87c5b700 1 -- 172.21.15.44:0/1288017487 shutdown_connections 2023-09-10T00:38:40.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.915+0000 7f3f87c5b700 1 -- 172.21.15.44:0/1288017487 wait complete. 2023-09-10T00:38:40.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.915+0000 7f3f87c5b700 1 Processor -- start 2023-09-10T00:38:40.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f87c5b700 1 -- start start 2023-09-10T00:38:40.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f87c5b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f80100620 0x7f3f80196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:40.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f87c5b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f80101810 0x7f3f80196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:40.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f87c5b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f801021b0 0x7f3f8019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:40.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f87c5b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8019ddc0 con 0x7f3f80101810 2023-09-10T00:38:40.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f87c5b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8019df00 con 0x7f3f801021b0 2023-09-10T00:38:40.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f87c5b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8019e090 con 0x7f3f80100620 2023-09-10T00:38:40.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f851f6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f80101810 0x7f3f80196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:40.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f851f6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f80101810 0x7f3f80196780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46290/0 (socket says 172.21.15.44:46290) 2023-09-10T00:38:40.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f851f6700 1 -- 172.21.15.44:0/1506285412 learned_addr learned my addr 172.21.15.44:0/1506285412 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:40.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f859f7700 1 --2- 172.21.15.44:0/1506285412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f80100620 0x7f3f80196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:40.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f861f8700 1 --2- 172.21.15.44:0/1506285412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f801021b0 0x7f3f8019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:40.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f851f6700 1 -- 172.21.15.44:0/1506285412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f80100620 msgr2=0x7f3f80196240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:40.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f851f6700 1 --2- 172.21.15.44:0/1506285412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f80100620 0x7f3f80196240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:40.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f851f6700 1 -- 172.21.15.44:0/1506285412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f801021b0 msgr2=0x7f3f8019d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:40.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f851f6700 1 --2- 172.21.15.44:0/1506285412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f801021b0 0x7f3f8019d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:40.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f851f6700 1 -- 172.21.15.44:0/1506285412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f78009d00 con 0x7f3f80101810 2023-09-10T00:38:40.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f851f6700 1 --2- 172.21.15.44:0/1506285412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f80101810 0x7f3f80196780 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00c920 tx=0x7f3f7c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:40.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.916+0000 7f3f76ffd700 1 -- 172.21.15.44:0/1506285412 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c007b00 con 0x7f3f80101810 2023-09-10T00:38:40.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.917+0000 7f3f76ffd700 1 -- 172.21.15.44:0/1506285412 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f7c00ce70 con 0x7f3f80101810 2023-09-10T00:38:40.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.917+0000 7f3f76ffd700 1 -- 172.21.15.44:0/1506285412 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c005440 con 0x7f3f80101810 2023-09-10T00:38:40.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.917+0000 7f3f87c5b700 1 -- 172.21.15.44:0/1506285412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8019e370 con 0x7f3f80101810 2023-09-10T00:38:40.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.917+0000 7f3f87c5b700 1 -- 172.21.15.44:0/1506285412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8019e830 con 0x7f3f80101810 2023-09-10T00:38:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.917+0000 7f3f87c5b700 1 -- 172.21.15.44:0/1506285412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f80107ac0 con 0x7f3f80101810 2023-09-10T00:38:40.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.919+0000 7f3f76ffd700 1 -- 172.21.15.44:0/1506285412 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f3f7c0055a0 con 0x7f3f80101810 2023-09-10T00:38:40.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.919+0000 7f3f76ffd700 1 --2- 172.21.15.44:0/1506285412 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3f6c0a06e0 0x7f3f6c0a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:40.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.919+0000 7f3f859f7700 1 --2- 172.21.15.44:0/1506285412 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3f6c0a06e0 0x7f3f6c0a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:40.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.919+0000 7f3f76ffd700 1 -- 172.21.15.44:0/1506285412 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f3f7c0cbb20 con 0x7f3f80101810 2023-09-10T00:38:40.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.920+0000 7f3f859f7700 1 --2- 172.21.15.44:0/1506285412 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3f6c0a06e0 0x7f3f6c0a2ba0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3f80101670 tx=0x7f3f7000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:40.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:40.923+0000 7f3f76ffd700 1 -- 172.21.15.44:0/1506285412 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3f7c095670 con 0x7f3f80101810 2023-09-10T00:38:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:40 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/45697749' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-10T00:38:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:40 smithi131 ceph-mon[103713]: osdmap e82: 12 total, 12 up, 12 in 2023-09-10T00:38:41.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:40 smithi131 ceph-mon[103713]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-10T00:38:41.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:40 smithi131 ceph-mon[103713]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-10T00:38:41.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:41.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:38:41.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:40 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1211259285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-10T00:38:41.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:40 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] up:active 2023-09-10T00:38:41.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:40 smithi131 ceph-mon[103713]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-10T00:38:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:40 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/45697749' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-10T00:38:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:40 smithi044 ceph-mon[98676]: osdmap e82: 12 total, 12 up, 12 in 2023-09-10T00:38:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:40 smithi044 ceph-mon[98676]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-10T00:38:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:40 smithi044 ceph-mon[98676]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-10T00:38:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:41.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:38:41.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:40 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1211259285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-10T00:38:41.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:40 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] up:active 2023-09-10T00:38:41.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:40 smithi044 ceph-mon[98676]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-10T00:38:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:40 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/45697749' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-10T00:38:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:40 smithi099 ceph-mon[102692]: osdmap e82: 12 total, 12 up, 12 in 2023-09-10T00:38:41.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:40 smithi099 ceph-mon[102692]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-10T00:38:41.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:40 smithi099 ceph-mon[102692]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-10T00:38:41.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:41.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:38:41.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:38:41.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:40 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1211259285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-10T00:38:41.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:40 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] up:active 2023-09-10T00:38:41.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:40 smithi099 ceph-mon[102692]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-10T00:38:41.129 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:41.128+0000 7f3f87c5b700 1 -- 172.21.15.44:0/1506285412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3f8002cfa0 con 0x7f3f80101810 2023-09-10T00:38:42.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:42 smithi131 ceph-mon[103713]: pgmap v264: 97 pgs: 29 creating+peering, 9 active+clean, 59 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:42.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:42 smithi131 ceph-mon[103713]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-10T00:38:42.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:42 smithi131 ceph-mon[103713]: Cluster is now healthy 2023-09-10T00:38:42.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:42 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1211259285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-09-10T00:38:42.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:42 smithi131 ceph-mon[103713]: osdmap e83: 12 total, 12 up, 12 in 2023-09-10T00:38:42.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:42 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1506285412' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-10T00:38:42.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:42 smithi044 ceph-mon[98676]: pgmap v264: 97 pgs: 29 creating+peering, 9 active+clean, 59 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:42.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:42 smithi044 ceph-mon[98676]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-10T00:38:42.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:42 smithi044 ceph-mon[98676]: Cluster is now healthy 2023-09-10T00:38:42.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:42 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1211259285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-09-10T00:38:42.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:42 smithi044 ceph-mon[98676]: osdmap e83: 12 total, 12 up, 12 in 2023-09-10T00:38:42.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:42 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1506285412' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-10T00:38:42.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:42 smithi099 ceph-mon[102692]: pgmap v264: 97 pgs: 29 creating+peering, 9 active+clean, 59 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:38:42.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:42 smithi099 ceph-mon[102692]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-10T00:38:42.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:42 smithi099 ceph-mon[102692]: Cluster is now healthy 2023-09-10T00:38:42.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:42 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1211259285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-09-10T00:38:42.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:42 smithi099 ceph-mon[102692]: osdmap e83: 12 total, 12 up, 12 in 2023-09-10T00:38:42.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:42 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1506285412' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-10T00:38:42.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.763+0000 7f3f76ffd700 1 -- 172.21.15.44:0/1506285412 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v85) v1 ==== 228+0+0 (secure 0 0 0) 0x7f3f7c01d100 con 0x7f3f80101810 2023-09-10T00:38:42.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.767+0000 7f3f87c5b700 1 -- 172.21.15.44:0/1506285412 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3f6c0a06e0 msgr2=0x7f3f6c0a2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:42.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.767+0000 7f3f87c5b700 1 --2- 172.21.15.44:0/1506285412 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3f6c0a06e0 0x7f3f6c0a2ba0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3f80101670 tx=0x7f3f7000a380 comp rx=0 tx=0).stop 2023-09-10T00:38:42.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.767+0000 7f3f87c5b700 1 -- 172.21.15.44:0/1506285412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f80101810 msgr2=0x7f3f80196780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:42.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.767+0000 7f3f87c5b700 1 --2- 172.21.15.44:0/1506285412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f80101810 0x7f3f80196780 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00c920 tx=0x7f3f7c00cc30 comp rx=0 tx=0).stop 2023-09-10T00:38:42.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.767+0000 7f3f87c5b700 1 -- 172.21.15.44:0/1506285412 shutdown_connections 2023-09-10T00:38:42.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.767+0000 7f3f87c5b700 1 --2- 172.21.15.44:0/1506285412 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3f6c0a06e0 0x7f3f6c0a2ba0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:42.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.767+0000 7f3f87c5b700 1 --2- 172.21.15.44:0/1506285412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f801021b0 0x7f3f8019d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:42.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.767+0000 7f3f87c5b700 1 --2- 172.21.15.44:0/1506285412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f80100620 0x7f3f80196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:42.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.767+0000 7f3f87c5b700 1 --2- 172.21.15.44:0/1506285412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f80101810 0x7f3f80196780 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:42.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.767+0000 7f3f87c5b700 1 -- 172.21.15.44:0/1506285412 >> 172.21.15.44:0/1506285412 conn(0x7f3f800fbb70 msgr2=0x7f3f80104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:42.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.767+0000 7f3f87c5b700 1 -- 172.21.15.44:0/1506285412 shutdown_connections 2023-09-10T00:38:42.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.767+0000 7f3f87c5b700 1 -- 172.21.15.44:0/1506285412 wait complete. 2023-09-10T00:38:42.772 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data_ec' created 2023-09-10T00:38:42.789 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-09-10T00:38:42.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.925+0000 7fc23b6e3700 1 -- 172.21.15.44:0/328974280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2341010b0 msgr2=0x7fc234107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:42.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.925+0000 7fc23b6e3700 1 --2- 172.21.15.44:0/328974280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2341010b0 0x7fc234107960 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00b0d0 tx=0x7fc22c00b3e0 comp rx=0 tx=0).stop 2023-09-10T00:38:42.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.926+0000 7fc23b6e3700 1 -- 172.21.15.44:0/328974280 shutdown_connections 2023-09-10T00:38:42.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.926+0000 7fc23b6e3700 1 --2- 172.21.15.44:0/328974280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2341010b0 0x7fc234107960 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:42.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.926+0000 7fc23b6e3700 1 --2- 172.21.15.44:0/328974280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2340ff520 0x7fc2340ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:42.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.926+0000 7fc23b6e3700 1 --2- 172.21.15.44:0/328974280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc234100710 0x7fc234100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:42.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.926+0000 7fc23b6e3700 1 -- 172.21.15.44:0/328974280 >> 172.21.15.44:0/328974280 conn(0x7fc2340faa90 msgr2=0x7fc2340fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:42.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.926+0000 7fc23b6e3700 1 -- 172.21.15.44:0/328974280 shutdown_connections 2023-09-10T00:38:42.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.926+0000 7fc23b6e3700 1 -- 172.21.15.44:0/328974280 wait complete. 2023-09-10T00:38:42.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.926+0000 7fc23b6e3700 1 Processor -- start 2023-09-10T00:38:42.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.926+0000 7fc23b6e3700 1 -- start start 2023-09-10T00:38:42.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.926+0000 7fc23b6e3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2340ff520 0x7fc234196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:42.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.926+0000 7fc23b6e3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc234100710 0x7fc234196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:42.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.926+0000 7fc23b6e3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2341010b0 0x7fc23419d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:42.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.926+0000 7fc23b6e3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23419de80 con 0x7fc2341010b0 2023-09-10T00:38:42.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.926+0000 7fc23b6e3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23419dfc0 con 0x7fc234100710 2023-09-10T00:38:42.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.926+0000 7fc23b6e3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23419e150 con 0x7fc2340ff520 2023-09-10T00:38:42.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.927+0000 7fc239c80700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2341010b0 0x7fc23419d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:42.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.927+0000 7fc239c80700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2341010b0 0x7fc23419d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46314/0 (socket says 172.21.15.44:46314) 2023-09-10T00:38:42.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.927+0000 7fc239c80700 1 -- 172.21.15.44:0/1413247135 learned_addr learned my addr 172.21.15.44:0/1413247135 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:42.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.927+0000 7fc23947f700 1 --2- 172.21.15.44:0/1413247135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2340ff520 0x7fc234196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:42.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.927+0000 7fc238c7e700 1 --2- 172.21.15.44:0/1413247135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc234100710 0x7fc234196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:42.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.927+0000 7fc239c80700 1 -- 172.21.15.44:0/1413247135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2340ff520 msgr2=0x7fc234196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:42.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.927+0000 7fc239c80700 1 --2- 172.21.15.44:0/1413247135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2340ff520 0x7fc234196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:42.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.927+0000 7fc239c80700 1 -- 172.21.15.44:0/1413247135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc234100710 msgr2=0x7fc234196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:42.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.927+0000 7fc239c80700 1 --2- 172.21.15.44:0/1413247135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc234100710 0x7fc234196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:42.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.927+0000 7fc239c80700 1 -- 172.21.15.44:0/1413247135 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c009d00 con 0x7fc2341010b0 2023-09-10T00:38:42.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.927+0000 7fc239c80700 1 --2- 172.21.15.44:0/1413247135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2341010b0 0x7fc23419d8b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00d010 tx=0x7fc22c0094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:42.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.927+0000 7fc22a7fc700 1 -- 172.21.15.44:0/1413247135 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c00e040 con 0x7fc2341010b0 2023-09-10T00:38:42.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.927+0000 7fc23b6e3700 1 -- 172.21.15.44:0/1413247135 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23419e3d0 con 0x7fc2341010b0 2023-09-10T00:38:42.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.927+0000 7fc22a7fc700 1 -- 172.21.15.44:0/1413247135 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc22c003e10 con 0x7fc2341010b0 2023-09-10T00:38:42.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.927+0000 7fc22a7fc700 1 -- 172.21.15.44:0/1413247135 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c0129f0 con 0x7fc2341010b0 2023-09-10T00:38:42.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.927+0000 7fc23b6e3700 1 -- 172.21.15.44:0/1413247135 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23419e7f0 con 0x7fc2341010b0 2023-09-10T00:38:42.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.928+0000 7fc23b6e3700 1 -- 172.21.15.44:0/1413247135 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc234107600 con 0x7fc2341010b0 2023-09-10T00:38:42.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.930+0000 7fc22a7fc700 1 -- 172.21.15.44:0/1413247135 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fc22c012b50 con 0x7fc2341010b0 2023-09-10T00:38:42.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.930+0000 7fc22a7fc700 1 --2- 172.21.15.44:0/1413247135 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc2200a0640 0x7fc2200a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:42.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.930+0000 7fc23947f700 1 --2- 172.21.15.44:0/1413247135 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc2200a0640 0x7fc2200a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:42.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.930+0000 7fc22a7fc700 1 -- 172.21.15.44:0/1413247135 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fc22c0cdd10 con 0x7fc2341010b0 2023-09-10T00:38:42.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.931+0000 7fc23947f700 1 --2- 172.21.15.44:0/1413247135 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc2200a0640 0x7fc2200a2b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc234100570 tx=0x7fc23000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:42.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:42.934+0000 7fc22a7fc700 1 -- 172.21.15.44:0/1413247135 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc22c097700 con 0x7fc2341010b0 2023-09-10T00:38:43.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.134+0000 7fc23b6e3700 1 -- 172.21.15.44:0/1413247135 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fc23402d0c0 con 0x7fc2341010b0 2023-09-10T00:38:43.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:43 smithi131 ceph-mon[103713]: pgmap v267: 97 pgs: 29 creating+peering, 44 active+clean, 24 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2023-09-10T00:38:43.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:43 smithi131 ceph-mon[103713]: osdmap e84: 12 total, 12 up, 12 in 2023-09-10T00:38:43.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:43 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1506285412' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-10T00:38:43.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:43 smithi131 ceph-mon[103713]: osdmap e85: 12 total, 12 up, 12 in 2023-09-10T00:38:43.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:43 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1413247135' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-10T00:38:43.554 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:43 smithi044 ceph-mon[98676]: pgmap v267: 97 pgs: 29 creating+peering, 44 active+clean, 24 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2023-09-10T00:38:43.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:43 smithi044 ceph-mon[98676]: osdmap e84: 12 total, 12 up, 12 in 2023-09-10T00:38:43.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:43 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1506285412' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-10T00:38:43.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:43 smithi044 ceph-mon[98676]: osdmap e85: 12 total, 12 up, 12 in 2023-09-10T00:38:43.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:43 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1413247135' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-10T00:38:43.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:43 smithi099 ceph-mon[102692]: pgmap v267: 97 pgs: 29 creating+peering, 44 active+clean, 24 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2023-09-10T00:38:43.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:43 smithi099 ceph-mon[102692]: osdmap e84: 12 total, 12 up, 12 in 2023-09-10T00:38:43.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:43 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1506285412' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-10T00:38:43.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:43 smithi099 ceph-mon[102692]: osdmap e85: 12 total, 12 up, 12 in 2023-09-10T00:38:43.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:43 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1413247135' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-10T00:38:43.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.765+0000 7fc22a7fc700 1 -- 172.21.15.44:0/1413247135 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v86) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc22c09ad20 con 0x7fc2341010b0 2023-09-10T00:38:43.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.770+0000 7fc23b6e3700 1 -- 172.21.15.44:0/1413247135 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc2200a0640 msgr2=0x7fc2200a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:43.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.770+0000 7fc23b6e3700 1 --2- 172.21.15.44:0/1413247135 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc2200a0640 0x7fc2200a2b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc234100570 tx=0x7fc23000b410 comp rx=0 tx=0).stop 2023-09-10T00:38:43.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.770+0000 7fc23b6e3700 1 -- 172.21.15.44:0/1413247135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2341010b0 msgr2=0x7fc23419d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:43.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.770+0000 7fc23b6e3700 1 --2- 172.21.15.44:0/1413247135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2341010b0 0x7fc23419d8b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00d010 tx=0x7fc22c0094c0 comp rx=0 tx=0).stop 2023-09-10T00:38:43.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.770+0000 7fc23b6e3700 1 -- 172.21.15.44:0/1413247135 shutdown_connections 2023-09-10T00:38:43.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.770+0000 7fc23b6e3700 1 --2- 172.21.15.44:0/1413247135 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc2200a0640 0x7fc2200a2b00 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc234100570 tx=0x7fc23000b410 comp rx=0 tx=0).stop 2023-09-10T00:38:43.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.770+0000 7fc23b6e3700 1 --2- 172.21.15.44:0/1413247135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2341010b0 0x7fc23419d8b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:43.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.770+0000 7fc23b6e3700 1 --2- 172.21.15.44:0/1413247135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2340ff520 0x7fc234196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:43.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.770+0000 7fc23b6e3700 1 --2- 172.21.15.44:0/1413247135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc234100710 0x7fc234196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:43.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.770+0000 7fc23b6e3700 1 -- 172.21.15.44:0/1413247135 >> 172.21.15.44:0/1413247135 conn(0x7fc2340faa90 msgr2=0x7fc2340fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:43.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.770+0000 7fc23b6e3700 1 -- 172.21.15.44:0/1413247135 shutdown_connections 2023-09-10T00:38:43.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.771+0000 7fc23b6e3700 1 -- 172.21.15.44:0/1413247135 wait complete. 2023-09-10T00:38:43.777 INFO:teuthology.orchestra.run.smithi044.stderr:set pool 4 allow_ec_overwrites to true 2023-09-10T00:38:43.800 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-09-10T00:38:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.934+0000 7f4303ba6700 1 -- 172.21.15.44:0/1297883392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42fc101c00 msgr2=0x7f42fc102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.934+0000 7f4303ba6700 1 --2- 172.21.15.44:0/1297883392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42fc101c00 0x7f42fc102020 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f42f8009990 tx=0x7f42f8009ca0 comp rx=0 tx=0).stop 2023-09-10T00:38:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.934+0000 7f4303ba6700 1 -- 172.21.15.44:0/1297883392 shutdown_connections 2023-09-10T00:38:43.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.934+0000 7f4303ba6700 1 --2- 172.21.15.44:0/1297883392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc0ff010 0x7f42fc108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.934+0000 7f4303ba6700 1 --2- 172.21.15.44:0/1297883392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42fc101c00 0x7f42fc102020 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.934+0000 7f4303ba6700 1 --2- 172.21.15.44:0/1297883392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42fc0fe670 0x7f42fc0fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.934+0000 7f4303ba6700 1 -- 172.21.15.44:0/1297883392 >> 172.21.15.44:0/1297883392 conn(0x7f42fc0fa1d0 msgr2=0x7f42fc0fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.934+0000 7f4303ba6700 1 -- 172.21.15.44:0/1297883392 shutdown_connections 2023-09-10T00:38:43.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.934+0000 7f4303ba6700 1 -- 172.21.15.44:0/1297883392 wait complete. 2023-09-10T00:38:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4303ba6700 1 Processor -- start 2023-09-10T00:38:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4303ba6700 1 -- start start 2023-09-10T00:38:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4303ba6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42fc0fe670 0x7f42fc196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4303ba6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42fc0ff010 0x7f42fc196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4303ba6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc101c00 0x7f42fc19d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4303ba6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc19dec0 con 0x7f42fc0fe670 2023-09-10T00:38:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4303ba6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc19e000 con 0x7f42fc101c00 2023-09-10T00:38:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4303ba6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc19e190 con 0x7f42fc0ff010 2023-09-10T00:38:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4301942700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42fc0fe670 0x7f42fc196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4301942700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42fc0fe670 0x7f42fc196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46328/0 (socket says 172.21.15.44:46328) 2023-09-10T00:38:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4301942700 1 -- 172.21.15.44:0/107611873 learned_addr learned my addr 172.21.15.44:0/107611873 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4301141700 1 --2- 172.21.15.44:0/107611873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42fc0ff010 0x7f42fc196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4302143700 1 --2- 172.21.15.44:0/107611873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc101c00 0x7f42fc19d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4301942700 1 -- 172.21.15.44:0/107611873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42fc0ff010 msgr2=0x7f42fc196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4301942700 1 --2- 172.21.15.44:0/107611873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42fc0ff010 0x7f42fc196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4301942700 1 -- 172.21.15.44:0/107611873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc101c00 msgr2=0x7f42fc19d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4301942700 1 --2- 172.21.15.44:0/107611873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc101c00 0x7f42fc19d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4301942700 1 -- 172.21.15.44:0/107611873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f80095f0 con 0x7f42fc0fe670 2023-09-10T00:38:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.935+0000 7f4302143700 1 --2- 172.21.15.44:0/107611873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc101c00 0x7f42fc19d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:38:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.936+0000 7f4301942700 1 --2- 172.21.15.44:0/107611873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42fc0fe670 0x7f42fc196340 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f42f8004730 tx=0x7f42f8004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:43.944 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.936+0000 7f42f2ffd700 1 -- 172.21.15.44:0/107611873 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f801d070 con 0x7f42fc0fe670 2023-09-10T00:38:43.944 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.936+0000 7f4303ba6700 1 -- 172.21.15.44:0/107611873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42fc19e410 con 0x7f42fc0fe670 2023-09-10T00:38:43.944 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.936+0000 7f42f2ffd700 1 -- 172.21.15.44:0/107611873 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f42f8004220 con 0x7f42fc0fe670 2023-09-10T00:38:43.944 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.936+0000 7f42f2ffd700 1 -- 172.21.15.44:0/107611873 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f800f460 con 0x7f42fc0fe670 2023-09-10T00:38:43.944 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.936+0000 7f4303ba6700 1 -- 172.21.15.44:0/107611873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42fc19e910 con 0x7f42fc0fe670 2023-09-10T00:38:43.945 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.937+0000 7f4303ba6700 1 -- 172.21.15.44:0/107611873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42fc107a80 con 0x7f42fc0fe670 2023-09-10T00:38:43.945 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.938+0000 7f42f2ffd700 1 -- 172.21.15.44:0/107611873 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f42f8022600 con 0x7f42fc0fe670 2023-09-10T00:38:43.946 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.938+0000 7f42f2ffd700 1 --2- 172.21.15.44:0/107611873 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42e80a0640 0x7f42e80a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:43.946 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.938+0000 7f4301141700 1 --2- 172.21.15.44:0/107611873 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42e80a0640 0x7f42e80a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:43.946 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.938+0000 7f42f2ffd700 1 -- 172.21.15.44:0/107611873 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f42f80cd340 con 0x7f42fc0fe670 2023-09-10T00:38:43.946 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.939+0000 7f4301141700 1 --2- 172.21.15.44:0/107611873 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42e80a0640 0x7f42e80a2b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009a10 tx=0x7f42ec009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:43.946 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:43.943+0000 7f42f2ffd700 1 -- 172.21.15.44:0/107611873 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f42f8096d30 con 0x7f42fc0fe670 2023-09-10T00:38:44.106 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 10 00:38:43 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-b[138846]: 2023-09-10T00:38:43.722+0000 7fe443b4f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-10T00:38:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.195+0000 7f4303ba6700 1 -- 172.21.15.44:0/107611873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f42fc10bd80 con 0x7f42fc0fe670 2023-09-10T00:38:44.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.772+0000 7f42f2ffd700 1 -- 172.21.15.44:0/107611873 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) v1 ==== 137+0+0 (secure 0 0 0) 0x7f42fc10bd80 con 0x7f42fc0fe670 2023-09-10T00:38:44.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.776+0000 7f4303ba6700 1 -- 172.21.15.44:0/107611873 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42e80a0640 msgr2=0x7f42e80a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:44.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.776+0000 7f4303ba6700 1 --2- 172.21.15.44:0/107611873 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42e80a0640 0x7f42e80a2b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009a10 tx=0x7f42ec009190 comp rx=0 tx=0).stop 2023-09-10T00:38:44.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.776+0000 7f4303ba6700 1 -- 172.21.15.44:0/107611873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42fc0fe670 msgr2=0x7f42fc196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:44.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.776+0000 7f4303ba6700 1 --2- 172.21.15.44:0/107611873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42fc0fe670 0x7f42fc196340 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f42f8004730 tx=0x7f42f8004760 comp rx=0 tx=0).stop 2023-09-10T00:38:44.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.776+0000 7f4303ba6700 1 -- 172.21.15.44:0/107611873 shutdown_connections 2023-09-10T00:38:44.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.776+0000 7f4303ba6700 1 --2- 172.21.15.44:0/107611873 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42e80a0640 0x7f42e80a2b00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:44.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.776+0000 7f4303ba6700 1 --2- 172.21.15.44:0/107611873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc101c00 0x7f42fc19d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:44.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.776+0000 7f4303ba6700 1 --2- 172.21.15.44:0/107611873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42fc0fe670 0x7f42fc196340 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:44.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.776+0000 7f4303ba6700 1 --2- 172.21.15.44:0/107611873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42fc0ff010 0x7f42fc196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:44.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.776+0000 7f4303ba6700 1 -- 172.21.15.44:0/107611873 >> 172.21.15.44:0/107611873 conn(0x7f42fc0fa1d0 msgr2=0x7f42fc104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:44.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.776+0000 7f4303ba6700 1 -- 172.21.15.44:0/107611873 shutdown_connections 2023-09-10T00:38:44.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.776+0000 7f4303ba6700 1 -- 172.21.15.44:0/107611873 wait complete. 2023-09-10T00:38:44.780 INFO:teuthology.orchestra.run.smithi044.stderr:added data pool 4 to fsmap 2023-09-10T00:38:44.792 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:38:44.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.924+0000 7fc874ddd700 1 -- 172.21.15.44:0/1646775105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8701022c0 msgr2=0x7fc870108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:44.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.924+0000 7fc874ddd700 1 --2- 172.21.15.44:0/1646775105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8701022c0 0x7fc870108b70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc86800b0a0 tx=0x7fc86800b3b0 comp rx=0 tx=0).stop 2023-09-10T00:38:44.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.924+0000 7fc874ddd700 1 -- 172.21.15.44:0/1646775105 shutdown_connections 2023-09-10T00:38:44.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.924+0000 7fc874ddd700 1 --2- 172.21.15.44:0/1646775105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8701022c0 0x7fc870108b70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:44.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.924+0000 7fc874ddd700 1 --2- 172.21.15.44:0/1646775105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8700fe8b0 0x7fc8700fecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:44.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.924+0000 7fc874ddd700 1 --2- 172.21.15.44:0/1646775105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc870101920 0x7fc870101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:44.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.924+0000 7fc874ddd700 1 -- 172.21.15.44:0/1646775105 >> 172.21.15.44:0/1646775105 conn(0x7fc8700fa1c0 msgr2=0x7fc8700fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:44.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.924+0000 7fc874ddd700 1 -- 172.21.15.44:0/1646775105 shutdown_connections 2023-09-10T00:38:44.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.924+0000 7fc874ddd700 1 -- 172.21.15.44:0/1646775105 wait complete. 2023-09-10T00:38:44.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc874ddd700 1 Processor -- start 2023-09-10T00:38:44.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc874ddd700 1 -- start start 2023-09-10T00:38:44.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc874ddd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8700fe8b0 0x7fc870101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:44.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc874ddd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc870101920 0x7fc8700ff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:44.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc874ddd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8701022c0 0x7fc8700ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:44.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc874ddd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8701005b0 con 0x7fc8700fe8b0 2023-09-10T00:38:44.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc874ddd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc870100720 con 0x7fc8701022c0 2023-09-10T00:38:44.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc874ddd700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8701a02c0 con 0x7fc870101920 2023-09-10T00:38:44.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc86e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8700fe8b0 0x7fc870101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:44.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc86e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8700fe8b0 0x7fc870101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46342/0 (socket says 172.21.15.44:46342) 2023-09-10T00:38:44.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc86e59c700 1 -- 172.21.15.44:0/3188210340 learned_addr learned my addr 172.21.15.44:0/3188210340 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:44.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc86ed9d700 1 --2- 172.21.15.44:0/3188210340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8701022c0 0x7fc8700ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:44.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc86dd9b700 1 --2- 172.21.15.44:0/3188210340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc870101920 0x7fc8700ff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:44.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc86e59c700 1 -- 172.21.15.44:0/3188210340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc870101920 msgr2=0x7fc8700ff8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:44.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc86e59c700 1 --2- 172.21.15.44:0/3188210340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc870101920 0x7fc8700ff8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:44.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc86e59c700 1 -- 172.21.15.44:0/3188210340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8701022c0 msgr2=0x7fc8700ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:44.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc86e59c700 1 --2- 172.21.15.44:0/3188210340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8701022c0 0x7fc8700ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:44.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc86e59c700 1 -- 172.21.15.44:0/3188210340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc868009d00 con 0x7fc8700fe8b0 2023-09-10T00:38:44.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.925+0000 7fc86e59c700 1 --2- 172.21.15.44:0/3188210340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8700fe8b0 0x7fc870101290 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc86000e970 tx=0x7fc86000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:44.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.926+0000 7fc85f7fe700 1 -- 172.21.15.44:0/3188210340 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860009790 con 0x7fc8700fe8b0 2023-09-10T00:38:44.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.926+0000 7fc85f7fe700 1 -- 172.21.15.44:0/3188210340 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc86000c7e0 con 0x7fc8700fe8b0 2023-09-10T00:38:44.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.926+0000 7fc874ddd700 1 -- 172.21.15.44:0/3188210340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8701a05a0 con 0x7fc8700fe8b0 2023-09-10T00:38:44.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.926+0000 7fc85f7fe700 1 -- 172.21.15.44:0/3188210340 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860003bd0 con 0x7fc8700fe8b0 2023-09-10T00:38:44.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.926+0000 7fc874ddd700 1 -- 172.21.15.44:0/3188210340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8701a0ab0 con 0x7fc8700fe8b0 2023-09-10T00:38:44.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.927+0000 7fc874ddd700 1 -- 172.21.15.44:0/3188210340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87010bc50 con 0x7fc8700fe8b0 2023-09-10T00:38:44.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.928+0000 7fc85f7fe700 1 -- 172.21.15.44:0/3188210340 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fc86000c950 con 0x7fc8700fe8b0 2023-09-10T00:38:44.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.928+0000 7fc85f7fe700 1 --2- 172.21.15.44:0/3188210340 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc8580a0690 0x7fc8580a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:44.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.929+0000 7fc86dd9b700 1 --2- 172.21.15.44:0/3188210340 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc8580a0690 0x7fc8580a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:44.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.929+0000 7fc85f7fe700 1 -- 172.21.15.44:0/3188210340 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc860014070 con 0x7fc8700fe8b0 2023-09-10T00:38:44.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.929+0000 7fc86dd9b700 1 --2- 172.21.15.44:0/3188210340 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc8580a0690 0x7fc8580a2b50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc864007be0 tx=0x7fc8640073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:44.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:44.933+0000 7fc85f7fe700 1 -- 172.21.15.44:0/3188210340 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc8600958b0 con 0x7fc8700fe8b0 2023-09-10T00:38:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:44 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1413247135' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-09-10T00:38:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:44 smithi131 ceph-mon[103713]: osdmap e86: 12 total, 12 up, 12 in 2023-09-10T00:38:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:44 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/107611873' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-10T00:38:45.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:44 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1413247135' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-09-10T00:38:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:44 smithi044 ceph-mon[98676]: osdmap e86: 12 total, 12 up, 12 in 2023-09-10T00:38:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:44 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/107611873' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-10T00:38:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:44 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1413247135' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-09-10T00:38:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:44 smithi099 ceph-mon[102692]: osdmap e86: 12 total, 12 up, 12 in 2023-09-10T00:38:45.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:44 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/107611873' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-10T00:38:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.179+0000 7fc874ddd700 1 -- 172.21.15.44:0/3188210340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc870107660 con 0x7fc8700fe8b0 2023-09-10T00:38:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.180+0000 7fc85f7fe700 1 -- 172.21.15.44:0/3188210340 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+11258 (secure 0 0 0) 0x7fc860098ed0 con 0x7fc8700fe8b0 2023-09-10T00:38:45.182 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:45.182 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14751,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/3958270510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":3958270510},{"type":"v1","addr":"172.21.15.44:6839","nonce":3958270510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1241002552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1241002552},{"type":"v1","addr":"172.21.15.44:6841","nonce":1241002552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24515,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24539,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24547,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24556,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24572,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24574,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/3222652693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":3222652693},{"type":"v1","addr":"172.21.15.131:6839","nonce":3222652693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-10T00:38:39.183533+0000","modified":"2023-09-10T00:38:44.198073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24605":{"gid":24605,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/2622966260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2622966260},{"type":"v1","addr":"172.21.15.99:6839","nonce":2622966260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-09-10T00:38:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.182+0000 7fc874ddd700 1 -- 172.21.15.44:0/3188210340 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc8580a0690 msgr2=0x7fc8580a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.182+0000 7fc874ddd700 1 --2- 172.21.15.44:0/3188210340 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc8580a0690 0x7fc8580a2b50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc864007be0 tx=0x7fc8640073d0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.182+0000 7fc874ddd700 1 -- 172.21.15.44:0/3188210340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8700fe8b0 msgr2=0x7fc870101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.182+0000 7fc874ddd700 1 --2- 172.21.15.44:0/3188210340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8700fe8b0 0x7fc870101290 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc86000e970 tx=0x7fc86000ec80 comp rx=0 tx=0).stop 2023-09-10T00:38:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.182+0000 7fc874ddd700 1 -- 172.21.15.44:0/3188210340 shutdown_connections 2023-09-10T00:38:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.182+0000 7fc874ddd700 1 --2- 172.21.15.44:0/3188210340 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc8580a0690 0x7fc8580a2b50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.182+0000 7fc874ddd700 1 --2- 172.21.15.44:0/3188210340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8701022c0 0x7fc8700ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.182+0000 7fc874ddd700 1 --2- 172.21.15.44:0/3188210340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8700fe8b0 0x7fc870101290 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.182+0000 7fc874ddd700 1 --2- 172.21.15.44:0/3188210340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc870101920 0x7fc8700ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.182+0000 7fc874ddd700 1 -- 172.21.15.44:0/3188210340 >> 172.21.15.44:0/3188210340 conn(0x7fc8700fa1c0 msgr2=0x7fc8700fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:45.186 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.182+0000 7fc874ddd700 1 -- 172.21.15.44:0/3188210340 shutdown_connections 2023-09-10T00:38:45.186 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.182+0000 7fc874ddd700 1 -- 172.21.15.44:0/3188210340 wait complete. 2023-09-10T00:38:45.186 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 16 2023-09-10T00:38:45.196 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:38:45.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.321+0000 7f8f0f7fe700 1 -- 172.21.15.44:0/2808598914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f08100600 msgr2=0x7f8f08100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:45.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.321+0000 7f8f0f7fe700 1 --2- 172.21.15.44:0/2808598914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f08100600 0x7f8f08100a20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0099c0 tx=0x7f8efc009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.321+0000 7f8f0f7fe700 1 -- 172.21.15.44:0/2808598914 shutdown_connections 2023-09-10T00:38:45.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.321+0000 7f8f0f7fe700 1 --2- 172.21.15.44:0/2808598914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f08102190 0x7f8f08108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.321+0000 7f8f0f7fe700 1 --2- 172.21.15.44:0/2808598914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f08100600 0x7f8f08100a20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.321+0000 7f8f0f7fe700 1 --2- 172.21.15.44:0/2808598914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f081017f0 0x7f8f08101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.321+0000 7f8f0f7fe700 1 -- 172.21.15.44:0/2808598914 >> 172.21.15.44:0/2808598914 conn(0x7f8f080fbb30 msgr2=0x7f8f080fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:45.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.321+0000 7f8f0f7fe700 1 -- 172.21.15.44:0/2808598914 shutdown_connections 2023-09-10T00:38:45.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.321+0000 7f8f0f7fe700 1 -- 172.21.15.44:0/2808598914 wait complete. 2023-09-10T00:38:45.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0f7fe700 1 Processor -- start 2023-09-10T00:38:45.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0f7fe700 1 -- start start 2023-09-10T00:38:45.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f08100600 0x7f8f08196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:45.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f081017f0 0x7f8f08196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:45.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f08102190 0x7f8f0819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:45.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0f7fe700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0819dda0 con 0x7f8f08102190 2023-09-10T00:38:45.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0f7fe700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0819dee0 con 0x7f8f08100600 2023-09-10T00:38:45.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0f7fe700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0819e070 con 0x7f8f081017f0 2023-09-10T00:38:45.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f08102190 0x7f8f0819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:45.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f08102190 0x7f8f0819d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46360/0 (socket says 172.21.15.44:46360) 2023-09-10T00:38:45.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0dd9b700 1 -- 172.21.15.44:0/515192634 learned_addr learned my addr 172.21.15.44:0/515192634 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:45.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0d59a700 1 --2- 172.21.15.44:0/515192634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f08100600 0x7f8f08196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:45.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0cd99700 1 --2- 172.21.15.44:0/515192634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f081017f0 0x7f8f08196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:45.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0dd9b700 1 -- 172.21.15.44:0/515192634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f081017f0 msgr2=0x7f8f08196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:45.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0dd9b700 1 --2- 172.21.15.44:0/515192634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f081017f0 0x7f8f08196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0dd9b700 1 -- 172.21.15.44:0/515192634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f08100600 msgr2=0x7f8f08196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:45.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0dd9b700 1 --2- 172.21.15.44:0/515192634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f08100600 0x7f8f08196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0dd9b700 1 -- 172.21.15.44:0/515192634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc0095f0 con 0x7f8f08102190 2023-09-10T00:38:45.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0cd99700 1 --2- 172.21.15.44:0/515192634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f081017f0 0x7f8f08196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:38:45.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.322+0000 7f8f0dd9b700 1 --2- 172.21.15.44:0/515192634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f08102190 0x7f8f0819d7d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8f00007ee0 tx=0x7f8f0000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:45.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.323+0000 7f8efa7fc700 1 -- 172.21.15.44:0/515192634 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00017070 con 0x7f8f08102190 2023-09-10T00:38:45.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.323+0000 7f8f0f7fe700 1 -- 172.21.15.44:0/515192634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0819e350 con 0x7f8f08102190 2023-09-10T00:38:45.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.323+0000 7f8efa7fc700 1 -- 172.21.15.44:0/515192634 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f0000f040 con 0x7f8f08102190 2023-09-10T00:38:45.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.323+0000 7f8efa7fc700 1 -- 172.21.15.44:0/515192634 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00005710 con 0x7f8f08102190 2023-09-10T00:38:45.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.323+0000 7f8f0f7fe700 1 -- 172.21.15.44:0/515192634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0819e810 con 0x7f8f08102190 2023-09-10T00:38:45.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.324+0000 7f8f0f7fe700 1 -- 172.21.15.44:0/515192634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0810bd20 con 0x7f8f08102190 2023-09-10T00:38:45.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.325+0000 7f8efa7fc700 1 -- 172.21.15.44:0/515192634 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f8f00005870 con 0x7f8f08102190 2023-09-10T00:38:45.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.325+0000 7f8efa7fc700 1 --2- 172.21.15.44:0/515192634 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8ef40a0690 0x7f8ef40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:45.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.326+0000 7f8f0d59a700 1 --2- 172.21.15.44:0/515192634 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8ef40a0690 0x7f8ef40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:45.333 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.326+0000 7f8efa7fc700 1 -- 172.21.15.44:0/515192634 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8f000cbc40 con 0x7f8f08102190 2023-09-10T00:38:45.333 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.326+0000 7f8f0d59a700 1 --2- 172.21.15.44:0/515192634 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8ef40a0690 0x7f8ef40a2b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8efc000c00 tx=0x7f8efc009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:45.333 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.329+0000 7f8efa7fc700 1 -- 172.21.15.44:0/515192634 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8f00095580 con 0x7f8f08102190 2023-09-10T00:38:45.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.531+0000 7f8f0f7fe700 1 -- 172.21.15.44:0/515192634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f0810bf30 con 0x7f8f08102190 2023-09-10T00:38:45.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.533+0000 7f8efa7fc700 1 -- 172.21.15.44:0/515192634 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23870 (secure 0 0 0) 0x7f8f00098ba0 con 0x7f8f08102190 2023-09-10T00:38:45.535 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:45.535 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":87,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:38:44.197902+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:38:45.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.535+0000 7f8f0f7fe700 1 -- 172.21.15.44:0/515192634 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8ef40a0690 msgr2=0x7f8ef40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:45.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.535+0000 7f8f0f7fe700 1 --2- 172.21.15.44:0/515192634 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8ef40a0690 0x7f8ef40a2b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8efc000c00 tx=0x7f8efc009f10 comp rx=0 tx=0).stop 2023-09-10T00:38:45.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.535+0000 7f8f0f7fe700 1 -- 172.21.15.44:0/515192634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f08102190 msgr2=0x7f8f0819d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:45.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.535+0000 7f8f0f7fe700 1 --2- 172.21.15.44:0/515192634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f08102190 0x7f8f0819d7d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8f00007ee0 tx=0x7f8f0000d3b0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.535+0000 7f8f0f7fe700 1 -- 172.21.15.44:0/515192634 shutdown_connections 2023-09-10T00:38:45.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.535+0000 7f8f0f7fe700 1 --2- 172.21.15.44:0/515192634 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8ef40a0690 0x7f8ef40a2b50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.535+0000 7f8f0f7fe700 1 --2- 172.21.15.44:0/515192634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f08102190 0x7f8f0819d7d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.535+0000 7f8f0f7fe700 1 --2- 172.21.15.44:0/515192634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f08100600 0x7f8f08196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.535+0000 7f8f0f7fe700 1 --2- 172.21.15.44:0/515192634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f081017f0 0x7f8f08196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.535+0000 7f8f0f7fe700 1 -- 172.21.15.44:0/515192634 >> 172.21.15.44:0/515192634 conn(0x7f8f080fbb30 msgr2=0x7f8f080fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:45.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.535+0000 7f8f0f7fe700 1 -- 172.21.15.44:0/515192634 shutdown_connections 2023-09-10T00:38:45.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.535+0000 7f8f0f7fe700 1 -- 172.21.15.44:0/515192634 wait complete. 2023-09-10T00:38:45.549 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:38:45.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.674+0000 7f755aa4f700 1 -- 172.21.15.44:0/4159878129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75540ff840 msgr2=0x7f75540ffcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:45.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.674+0000 7f755aa4f700 1 --2- 172.21.15.44:0/4159878129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75540ff840 0x7f75540ffcc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f753c0099c0 tx=0x7f753c009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.674+0000 7f755aa4f700 1 -- 172.21.15.44:0/4159878129 shutdown_connections 2023-09-10T00:38:45.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.674+0000 7f755aa4f700 1 --2- 172.21.15.44:0/4159878129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7554100200 0x7f7554100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.674+0000 7f755aa4f700 1 --2- 172.21.15.44:0/4159878129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75540feee0 0x7f75540ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.674+0000 7f755aa4f700 1 --2- 172.21.15.44:0/4159878129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75540ff840 0x7f75540ffcc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.674+0000 7f755aa4f700 1 -- 172.21.15.44:0/4159878129 >> 172.21.15.44:0/4159878129 conn(0x7f75540faa80 msgr2=0x7f75540fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:45.679 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.674+0000 7f755aa4f700 1 -- 172.21.15.44:0/4159878129 shutdown_connections 2023-09-10T00:38:45.679 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.674+0000 7f755aa4f700 1 -- 172.21.15.44:0/4159878129 wait complete. 2023-09-10T00:38:45.679 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f755aa4f700 1 Processor -- start 2023-09-10T00:38:45.679 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f755aa4f700 1 -- start start 2023-09-10T00:38:45.679 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f755aa4f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75540feee0 0x7f75541983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f755aa4f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75540ff840 0x7f7554198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f755aa4f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7554100200 0x7f755419f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f755aa4f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755419ff70 con 0x7f75540ff840 2023-09-10T00:38:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f755aa4f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75541a00b0 con 0x7f75540feee0 2023-09-10T00:38:45.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f755aa4f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75541a0240 con 0x7f7554100200 2023-09-10T00:38:45.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f754bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75540ff840 0x7f7554198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:45.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f754bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75540ff840 0x7f7554198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46372/0 (socket says 172.21.15.44:46372) 2023-09-10T00:38:45.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f754bfff700 1 -- 172.21.15.44:0/1333667150 learned_addr learned my addr 172.21.15.44:0/1333667150 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:45.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f7553fff700 1 --2- 172.21.15.44:0/1333667150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75540feee0 0x7f75541983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:45.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f754bfff700 1 -- 172.21.15.44:0/1333667150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7554100200 msgr2=0x7f755419f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:45.682 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f754bfff700 1 --2- 172.21.15.44:0/1333667150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7554100200 0x7f755419f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.682 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f754bfff700 1 -- 172.21.15.44:0/1333667150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75540feee0 msgr2=0x7f75541983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:45.682 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f754bfff700 1 --2- 172.21.15.44:0/1333667150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75540feee0 0x7f75541983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.682 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f754bfff700 1 -- 172.21.15.44:0/1333667150 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c0095f0 con 0x7f75540ff840 2023-09-10T00:38:45.683 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.675+0000 7f7553fff700 1 --2- 172.21.15.44:0/1333667150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75540feee0 0x7f75541983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:38:45.683 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.676+0000 7f754bfff700 1 --2- 172.21.15.44:0/1333667150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75540ff840 0x7f7554198930 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f753c009f50 tx=0x7f753c003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:45.683 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.676+0000 7f7551ffb700 1 -- 172.21.15.44:0/1333667150 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c01d070 con 0x7f75540ff840 2023-09-10T00:38:45.683 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.676+0000 7f7551ffb700 1 -- 172.21.15.44:0/1333667150 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f753c00bda0 con 0x7f75540ff840 2023-09-10T00:38:45.684 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.676+0000 7f755aa4f700 1 -- 172.21.15.44:0/1333667150 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75541a04c0 con 0x7f75540ff840 2023-09-10T00:38:45.684 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.676+0000 7f7551ffb700 1 -- 172.21.15.44:0/1333667150 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c005280 con 0x7f75540ff840 2023-09-10T00:38:45.684 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.676+0000 7f755aa4f700 1 -- 172.21.15.44:0/1333667150 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75541a08e0 con 0x7f75540ff840 2023-09-10T00:38:45.685 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.677+0000 7f755aa4f700 1 -- 172.21.15.44:0/1333667150 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7554109cc0 con 0x7f75540ff840 2023-09-10T00:38:45.685 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.679+0000 7f7551ffb700 1 -- 172.21.15.44:0/1333667150 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f753c0053e0 con 0x7f75540ff840 2023-09-10T00:38:45.686 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.679+0000 7f7551ffb700 1 --2- 172.21.15.44:0/1333667150 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f75400a0710 0x7f75400a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:45.686 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.679+0000 7f7553fff700 1 --2- 172.21.15.44:0/1333667150 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f75400a0710 0x7f75400a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:45.686 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.679+0000 7f7551ffb700 1 -- 172.21.15.44:0/1333667150 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f753c0cd0a0 con 0x7f75540ff840 2023-09-10T00:38:45.686 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.680+0000 7f7553fff700 1 --2- 172.21.15.44:0/1333667150 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f75400a0710 0x7f75400a2bd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7544005fd0 tx=0x7f7544005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:45.686 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.683+0000 7f7551ffb700 1 -- 172.21.15.44:0/1333667150 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f753c096b90 con 0x7f75540ff840 2023-09-10T00:38:45.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:45 smithi044 ceph-mon[98676]: pgmap v270: 129 pgs: 1 creating+activating, 22 creating+peering, 90 active+clean, 16 unknown; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 8 op/s 2023-09-10T00:38:45.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:45 smithi044 ceph-mon[98676]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-10T00:38:45.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:45 smithi044 ceph-mon[98676]: osdmap e87: 12 total, 12 up, 12 in 2023-09-10T00:38:45.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:45 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/107611873' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-09-10T00:38:45.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:45 smithi044 ceph-mon[98676]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-10T00:38:45.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:45 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3188210340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:45.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:45 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/515192634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.889+0000 7f755aa4f700 1 -- 172.21.15.44:0/1333667150 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f755410df00 con 0x7f75540ff840 2023-09-10T00:38:45.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.890+0000 7f7551ffb700 1 -- 172.21.15.44:0/1333667150 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23870 (secure 0 0 0) 0x7f753c09a1b0 con 0x7f75540ff840 2023-09-10T00:38:45.892 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:45.892 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":87,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:38:44.197902+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:38:45.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.892+0000 7f755aa4f700 1 -- 172.21.15.44:0/1333667150 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f75400a0710 msgr2=0x7f75400a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:45.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.892+0000 7f755aa4f700 1 --2- 172.21.15.44:0/1333667150 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f75400a0710 0x7f75400a2bd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7544005fd0 tx=0x7f7544005f60 comp rx=0 tx=0).stop 2023-09-10T00:38:45.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.892+0000 7f755aa4f700 1 -- 172.21.15.44:0/1333667150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75540ff840 msgr2=0x7f7554198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:45.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.892+0000 7f755aa4f700 1 --2- 172.21.15.44:0/1333667150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75540ff840 0x7f7554198930 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f753c009f50 tx=0x7f753c003b00 comp rx=0 tx=0).stop 2023-09-10T00:38:45.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.892+0000 7f755aa4f700 1 -- 172.21.15.44:0/1333667150 shutdown_connections 2023-09-10T00:38:45.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.892+0000 7f755aa4f700 1 --2- 172.21.15.44:0/1333667150 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f75400a0710 0x7f75400a2bd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.892+0000 7f755aa4f700 1 --2- 172.21.15.44:0/1333667150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7554100200 0x7f755419f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.892+0000 7f755aa4f700 1 --2- 172.21.15.44:0/1333667150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75540feee0 0x7f75541983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.893+0000 7f755aa4f700 1 --2- 172.21.15.44:0/1333667150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75540ff840 0x7f7554198930 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:45.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.893+0000 7f755aa4f700 1 -- 172.21.15.44:0/1333667150 >> 172.21.15.44:0/1333667150 conn(0x7f75540faa80 msgr2=0x7f75540fc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:45.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.893+0000 7f755aa4f700 1 -- 172.21.15.44:0/1333667150 shutdown_connections 2023-09-10T00:38:45.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:45.893+0000 7f755aa4f700 1 -- 172.21.15.44:0/1333667150 wait complete. 2023-09-10T00:38:45.922 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:38:46.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:45 smithi131 ceph-mon[103713]: pgmap v270: 129 pgs: 1 creating+activating, 22 creating+peering, 90 active+clean, 16 unknown; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 8 op/s 2023-09-10T00:38:46.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:45 smithi131 ceph-mon[103713]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-10T00:38:46.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:45 smithi131 ceph-mon[103713]: osdmap e87: 12 total, 12 up, 12 in 2023-09-10T00:38:46.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:45 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/107611873' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-09-10T00:38:46.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:45 smithi131 ceph-mon[103713]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-10T00:38:46.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:45 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3188210340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:46.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:45 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/515192634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:46.051 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.048+0000 7fac50180700 1 -- 172.21.15.44:0/1851556138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac480fe8b0 msgr2=0x7fac480fecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.051 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.048+0000 7fac50180700 1 --2- 172.21.15.44:0/1851556138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac480fe8b0 0x7fac480fecd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fac44009990 tx=0x7fac44009ca0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.051 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.048+0000 7fac50180700 1 -- 172.21.15.44:0/1851556138 shutdown_connections 2023-09-10T00:38:46.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.048+0000 7fac50180700 1 --2- 172.21.15.44:0/1851556138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac481022c0 0x7fac48108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.048+0000 7fac50180700 1 --2- 172.21.15.44:0/1851556138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac480fe8b0 0x7fac480fecd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.048+0000 7fac50180700 1 --2- 172.21.15.44:0/1851556138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac48101920 0x7fac48101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.048+0000 7fac50180700 1 -- 172.21.15.44:0/1851556138 >> 172.21.15.44:0/1851556138 conn(0x7fac480fa1c0 msgr2=0x7fac480fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:46.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.048+0000 7fac50180700 1 -- 172.21.15.44:0/1851556138 shutdown_connections 2023-09-10T00:38:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.048+0000 7fac50180700 1 -- 172.21.15.44:0/1851556138 wait complete. 2023-09-10T00:38:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.048+0000 7fac50180700 1 Processor -- start 2023-09-10T00:38:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac50180700 1 -- start start 2023-09-10T00:38:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac50180700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac480fe8b0 0x7fac48101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac50180700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac48101920 0x7fac480ff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac50180700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac481022c0 0x7fac480ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac50180700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac481005b0 con 0x7fac481022c0 2023-09-10T00:38:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac50180700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac48100720 con 0x7fac480fe8b0 2023-09-10T00:38:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac50180700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac481a0220 con 0x7fac48101920 2023-09-10T00:38:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac4e71d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac481022c0 0x7fac480ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac4e71d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac481022c0 0x7fac480ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46388/0 (socket says 172.21.15.44:46388) 2023-09-10T00:38:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac4e71d700 1 -- 172.21.15.44:0/3200331920 learned_addr learned my addr 172.21.15.44:0/3200331920 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac4d71b700 1 --2- 172.21.15.44:0/3200331920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac48101920 0x7fac480ff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac4df1c700 1 --2- 172.21.15.44:0/3200331920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac480fe8b0 0x7fac48101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac4e71d700 1 -- 172.21.15.44:0/3200331920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac48101920 msgr2=0x7fac480ff8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac4e71d700 1 --2- 172.21.15.44:0/3200331920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac48101920 0x7fac480ff8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac4e71d700 1 -- 172.21.15.44:0/3200331920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac480fe8b0 msgr2=0x7fac48101290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac4e71d700 1 --2- 172.21.15.44:0/3200331920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac480fe8b0 0x7fac48101290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac4e71d700 1 -- 172.21.15.44:0/3200331920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac440095f0 con 0x7fac481022c0 2023-09-10T00:38:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac4e71d700 1 --2- 172.21.15.44:0/3200331920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac481022c0 0x7fac480ffe10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fac4000ada0 tx=0x7fac4000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac3effd700 1 -- 172.21.15.44:0/3200331920 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4000f070 con 0x7fac481022c0 2023-09-10T00:38:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac3effd700 1 -- 172.21.15.44:0/3200331920 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fac40007b30 con 0x7fac481022c0 2023-09-10T00:38:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac3effd700 1 -- 172.21.15.44:0/3200331920 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40005280 con 0x7fac481022c0 2023-09-10T00:38:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.049+0000 7fac50180700 1 -- 172.21.15.44:0/3200331920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac481a0500 con 0x7fac481022c0 2023-09-10T00:38:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.050+0000 7fac50180700 1 -- 172.21.15.44:0/3200331920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac481a0a10 con 0x7fac481022c0 2023-09-10T00:38:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.050+0000 7fac50180700 1 -- 172.21.15.44:0/3200331920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac481077a0 con 0x7fac481022c0 2023-09-10T00:38:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.052+0000 7fac3effd700 1 -- 172.21.15.44:0/3200331920 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fac4000a090 con 0x7fac481022c0 2023-09-10T00:38:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.052+0000 7fac3effd700 1 --2- 172.21.15.44:0/3200331920 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fac340a0710 0x7fac340a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.052+0000 7fac3effd700 1 -- 172.21.15.44:0/3200331920 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fac400cb7b0 con 0x7fac481022c0 2023-09-10T00:38:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.052+0000 7fac4df1c700 1 --2- 172.21.15.44:0/3200331920 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fac340a0710 0x7fac340a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.053+0000 7fac4df1c700 1 --2- 172.21.15.44:0/3200331920 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fac340a0710 0x7fac340a2bd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fac4400b5c0 tx=0x7fac44005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.055+0000 7fac3effd700 1 -- 172.21.15.44:0/3200331920 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fac40095180 con 0x7fac481022c0 2023-09-10T00:38:46.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:45 smithi099 ceph-mon[102692]: pgmap v270: 129 pgs: 1 creating+activating, 22 creating+peering, 90 active+clean, 16 unknown; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 8 op/s 2023-09-10T00:38:46.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:45 smithi099 ceph-mon[102692]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-10T00:38:46.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:45 smithi099 ceph-mon[102692]: osdmap e87: 12 total, 12 up, 12 in 2023-09-10T00:38:46.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:45 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/107611873' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-09-10T00:38:46.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:45 smithi099 ceph-mon[102692]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-10T00:38:46.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:45 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3188210340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:46.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:45 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/515192634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:46.309 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.307+0000 7fac50180700 1 -- 172.21.15.44:0/3200331920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac4810bf10 con 0x7fac481022c0 2023-09-10T00:38:46.309 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.308+0000 7fac3effd700 1 -- 172.21.15.44:0/3200331920 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+11258 (secure 0 0 0) 0x7fac400987a0 con 0x7fac481022c0 2023-09-10T00:38:46.310 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:46.310 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14751,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/3958270510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":3958270510},{"type":"v1","addr":"172.21.15.44:6839","nonce":3958270510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/1241002552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1241002552},{"type":"v1","addr":"172.21.15.44:6841","nonce":1241002552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24515,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24539,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24547,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24556,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24572,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24574,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/3222652693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":3222652693},{"type":"v1","addr":"172.21.15.131:6839","nonce":3222652693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-10T00:38:39.183533+0000","modified":"2023-09-10T00:38:44.198073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24605":{"gid":24605,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/2622966260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2622966260},{"type":"v1","addr":"172.21.15.99:6839","nonce":2622966260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-09-10T00:38:46.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.310+0000 7fac50180700 1 -- 172.21.15.44:0/3200331920 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fac340a0710 msgr2=0x7fac340a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.310+0000 7fac50180700 1 --2- 172.21.15.44:0/3200331920 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fac340a0710 0x7fac340a2bd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fac4400b5c0 tx=0x7fac44005fb0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.310+0000 7fac50180700 1 -- 172.21.15.44:0/3200331920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac481022c0 msgr2=0x7fac480ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.310+0000 7fac50180700 1 --2- 172.21.15.44:0/3200331920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac481022c0 0x7fac480ffe10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fac4000ada0 tx=0x7fac4000c330 comp rx=0 tx=0).stop 2023-09-10T00:38:46.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.310+0000 7fac50180700 1 -- 172.21.15.44:0/3200331920 shutdown_connections 2023-09-10T00:38:46.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.310+0000 7fac50180700 1 --2- 172.21.15.44:0/3200331920 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fac340a0710 0x7fac340a2bd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.310+0000 7fac50180700 1 --2- 172.21.15.44:0/3200331920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac481022c0 0x7fac480ffe10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.310+0000 7fac50180700 1 --2- 172.21.15.44:0/3200331920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac480fe8b0 0x7fac48101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.310+0000 7fac50180700 1 --2- 172.21.15.44:0/3200331920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac48101920 0x7fac480ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.310+0000 7fac50180700 1 -- 172.21.15.44:0/3200331920 >> 172.21.15.44:0/3200331920 conn(0x7fac480fa1c0 msgr2=0x7fac480fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:46.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.311+0000 7fac50180700 1 -- 172.21.15.44:0/3200331920 shutdown_connections 2023-09-10T00:38:46.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.311+0000 7fac50180700 1 -- 172.21.15.44:0/3200331920 wait complete. 2023-09-10T00:38:46.314 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 16 2023-09-10T00:38:46.323 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:38:46.451 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.448+0000 7f9fe1261700 1 -- 172.21.15.44:0/2399146057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fdc100840 msgr2=0x7f9fdc1070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.448+0000 7f9fe1261700 1 --2- 172.21.15.44:0/2399146057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fdc100840 0x7f9fdc1070f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400b0d0 tx=0x7f9fd400b3e0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.448+0000 7f9fe1261700 1 -- 172.21.15.44:0/2399146057 shutdown_connections 2023-09-10T00:38:46.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.448+0000 7f9fe1261700 1 --2- 172.21.15.44:0/2399146057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fdc100840 0x7f9fdc1070f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.448+0000 7f9fe1261700 1 --2- 172.21.15.44:0/2399146057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fdc0fecb0 0x7f9fdc0ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.448+0000 7f9fe1261700 1 --2- 172.21.15.44:0/2399146057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fdc0ffea0 0x7f9fdc100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.448+0000 7f9fe1261700 1 -- 172.21.15.44:0/2399146057 >> 172.21.15.44:0/2399146057 conn(0x7f9fdc0fa200 msgr2=0x7f9fdc0fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:46.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.448+0000 7f9fe1261700 1 -- 172.21.15.44:0/2399146057 shutdown_connections 2023-09-10T00:38:46.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.448+0000 7f9fe1261700 1 -- 172.21.15.44:0/2399146057 wait complete. 2023-09-10T00:38:46.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.448+0000 7f9fe1261700 1 Processor -- start 2023-09-10T00:38:46.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fe1261700 1 -- start start 2023-09-10T00:38:46.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fe1261700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fdc0fecb0 0x7f9fdc1085b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fe1261700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fdc0ffea0 0x7f9fdc19afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fe1261700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fdc100840 0x7f9fdc19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fe1261700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc19ddb0 con 0x7f9fdc0ffea0 2023-09-10T00:38:46.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fe1261700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc19df20 con 0x7f9fdc100840 2023-09-10T00:38:46.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fe1261700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc19e0e0 con 0x7f9fdc0fecb0 2023-09-10T00:38:46.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fda7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fdc0ffea0 0x7f9fdc19afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fda7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fdc0ffea0 0x7f9fdc19afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46400/0 (socket says 172.21.15.44:46400) 2023-09-10T00:38:46.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fda7fc700 1 -- 172.21.15.44:0/125302933 learned_addr learned my addr 172.21.15.44:0/125302933 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:46.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fdaffd700 1 --2- 172.21.15.44:0/125302933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fdc0fecb0 0x7f9fdc1085b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fdb7fe700 1 --2- 172.21.15.44:0/125302933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fdc100840 0x7f9fdc19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fda7fc700 1 -- 172.21.15.44:0/125302933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fdc0fecb0 msgr2=0x7f9fdc1085b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fda7fc700 1 --2- 172.21.15.44:0/125302933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fdc0fecb0 0x7f9fdc1085b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fda7fc700 1 -- 172.21.15.44:0/125302933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fdc100840 msgr2=0x7f9fdc19d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fda7fc700 1 --2- 172.21.15.44:0/125302933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fdc100840 0x7f9fdc19d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fda7fc700 1 -- 172.21.15.44:0/125302933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd4009d00 con 0x7f9fdc0ffea0 2023-09-10T00:38:46.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fda7fc700 1 --2- 172.21.15.44:0/125302933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fdc0ffea0 0x7f9fdc19afd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400d980 tx=0x7f9fc400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:46.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.449+0000 7f9fd3fff700 1 -- 172.21.15.44:0/125302933 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4004220 con 0x7f9fdc0ffea0 2023-09-10T00:38:46.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.450+0000 7f9fd3fff700 1 -- 172.21.15.44:0/125302933 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fc4007640 con 0x7f9fdc0ffea0 2023-09-10T00:38:46.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.450+0000 7f9fe1261700 1 -- 172.21.15.44:0/125302933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fdc19e3c0 con 0x7f9fdc0ffea0 2023-09-10T00:38:46.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.450+0000 7f9fd3fff700 1 -- 172.21.15.44:0/125302933 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40055e0 con 0x7f9fdc0ffea0 2023-09-10T00:38:46.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.450+0000 7f9fe1261700 1 -- 172.21.15.44:0/125302933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fdc19e920 con 0x7f9fdc0ffea0 2023-09-10T00:38:46.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.450+0000 7f9fe1261700 1 -- 172.21.15.44:0/125302933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fdc1061d0 con 0x7f9fdc0ffea0 2023-09-10T00:38:46.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.452+0000 7f9fd3fff700 1 -- 172.21.15.44:0/125302933 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f9fc4004750 con 0x7f9fdc0ffea0 2023-09-10T00:38:46.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.452+0000 7f9fd3fff700 1 --2- 172.21.15.44:0/125302933 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9fc80a0710 0x7f9fc80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.452+0000 7f9fd3fff700 1 -- 172.21.15.44:0/125302933 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9fc40cbb30 con 0x7f9fdc0ffea0 2023-09-10T00:38:46.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.452+0000 7f9fdaffd700 1 --2- 172.21.15.44:0/125302933 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9fc80a0710 0x7f9fc80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.453+0000 7f9fdaffd700 1 --2- 172.21.15.44:0/125302933 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9fc80a0710 0x7f9fc80a2bd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0ffd00 tx=0x7f9fcc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:46.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.457+0000 7f9fd3fff700 1 -- 172.21.15.44:0/125302933 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9fc4095470 con 0x7f9fdc0ffea0 2023-09-10T00:38:46.659 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.658+0000 7f9fe1261700 1 -- 172.21.15.44:0/125302933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fdc10ac10 con 0x7f9fdc0ffea0 2023-09-10T00:38:46.660 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.659+0000 7f9fd3fff700 1 -- 172.21.15.44:0/125302933 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23870 (secure 0 0 0) 0x7f9fc4016020 con 0x7f9fdc0ffea0 2023-09-10T00:38:46.661 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:46.661 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":87,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:38:44.197902+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:38:46.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.661+0000 7f9fe1261700 1 -- 172.21.15.44:0/125302933 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9fc80a0710 msgr2=0x7f9fc80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.661+0000 7f9fe1261700 1 --2- 172.21.15.44:0/125302933 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9fc80a0710 0x7f9fc80a2bd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0ffd00 tx=0x7f9fcc00b410 comp rx=0 tx=0).stop 2023-09-10T00:38:46.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.661+0000 7f9fe1261700 1 -- 172.21.15.44:0/125302933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fdc0ffea0 msgr2=0x7f9fdc19afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.661+0000 7f9fe1261700 1 --2- 172.21.15.44:0/125302933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fdc0ffea0 0x7f9fdc19afd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400d980 tx=0x7f9fc400dc90 comp rx=0 tx=0).stop 2023-09-10T00:38:46.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.662+0000 7f9fe1261700 1 -- 172.21.15.44:0/125302933 shutdown_connections 2023-09-10T00:38:46.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.662+0000 7f9fe1261700 1 --2- 172.21.15.44:0/125302933 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9fc80a0710 0x7f9fc80a2bd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.662+0000 7f9fe1261700 1 --2- 172.21.15.44:0/125302933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fdc100840 0x7f9fdc19d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.662+0000 7f9fe1261700 1 --2- 172.21.15.44:0/125302933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fdc0fecb0 0x7f9fdc1085b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.662+0000 7f9fe1261700 1 --2- 172.21.15.44:0/125302933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fdc0ffea0 0x7f9fdc19afd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.662+0000 7f9fe1261700 1 -- 172.21.15.44:0/125302933 >> 172.21.15.44:0/125302933 conn(0x7f9fdc0fa200 msgr2=0x7f9fdc103310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:46.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.662+0000 7f9fe1261700 1 -- 172.21.15.44:0/125302933 shutdown_connections 2023-09-10T00:38:46.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.662+0000 7f9fe1261700 1 -- 172.21.15.44:0/125302933 wait complete. 2023-09-10T00:38:46.676 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-10T00:38:39.183533+0000', 'modified': '2023-09-10T00:38:44.198073+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24605}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24605': {'gid': 24605, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6839/2622966260', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 2622966260}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 2622966260}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-09-10T00:38:46.676 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-10T00:38:46.676 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-10T00:38:46.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.804+0000 7f1b5a191700 1 -- 172.21.15.44:0/2464689945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b541010b0 msgr2=0x7f1b54107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.804+0000 7f1b5a191700 1 --2- 172.21.15.44:0/2464689945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b541010b0 0x7f1b54107960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00b0d0 tx=0x7f1b4c00b3e0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.804+0000 7f1b5a191700 1 -- 172.21.15.44:0/2464689945 shutdown_connections 2023-09-10T00:38:46.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.804+0000 7f1b5a191700 1 --2- 172.21.15.44:0/2464689945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b541010b0 0x7f1b54107960 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.804+0000 7f1b5a191700 1 --2- 172.21.15.44:0/2464689945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b540ff520 0x7f1b540ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.804+0000 7f1b5a191700 1 --2- 172.21.15.44:0/2464689945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b54100710 0x7f1b54100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.804+0000 7f1b5a191700 1 -- 172.21.15.44:0/2464689945 >> 172.21.15.44:0/2464689945 conn(0x7f1b540faa90 msgr2=0x7f1b540fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:46.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.804+0000 7f1b5a191700 1 -- 172.21.15.44:0/2464689945 shutdown_connections 2023-09-10T00:38:46.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.804+0000 7f1b5a191700 1 -- 172.21.15.44:0/2464689945 wait complete. 2023-09-10T00:38:46.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.804+0000 7f1b5a191700 1 Processor -- start 2023-09-10T00:38:46.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b5a191700 1 -- start start 2023-09-10T00:38:46.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b5a191700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b540ff520 0x7f1b541951d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b5a191700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b54100710 0x7f1b54195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b5a191700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b541010b0 0x7f1b5419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b5a191700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5419cd50 con 0x7f1b54100710 2023-09-10T00:38:46.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b5a191700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5419ce90 con 0x7f1b540ff520 2023-09-10T00:38:46.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b5a191700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5419d020 con 0x7f1b541010b0 2023-09-10T00:38:46.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b52ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b54100710 0x7f1b54195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b52ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b54100710 0x7f1b54195710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46414/0 (socket says 172.21.15.44:46414) 2023-09-10T00:38:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b52ffd700 1 -- 172.21.15.44:0/2168242996 learned_addr learned my addr 172.21.15.44:0/2168242996 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b53fff700 1 --2- 172.21.15.44:0/2168242996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b541010b0 0x7f1b5419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b537fe700 1 --2- 172.21.15.44:0/2168242996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b540ff520 0x7f1b541951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b52ffd700 1 -- 172.21.15.44:0/2168242996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b541010b0 msgr2=0x7f1b5419c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b52ffd700 1 --2- 172.21.15.44:0/2168242996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b541010b0 0x7f1b5419c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b52ffd700 1 -- 172.21.15.44:0/2168242996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b540ff520 msgr2=0x7f1b541951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b52ffd700 1 --2- 172.21.15.44:0/2168242996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b540ff520 0x7f1b541951d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b52ffd700 1 -- 172.21.15.44:0/2168242996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c009d00 con 0x7f1b54100710 2023-09-10T00:38:46.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b52ffd700 1 --2- 172.21.15.44:0/2168242996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b54100710 0x7f1b54195710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400e9a0 tx=0x7f1b4400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:46.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.805+0000 7f1b50ff9700 1 -- 172.21.15.44:0/2168242996 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44009790 con 0x7f1b54100710 2023-09-10T00:38:46.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.806+0000 7f1b5a191700 1 -- 172.21.15.44:0/2168242996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5419d300 con 0x7f1b54100710 2023-09-10T00:38:46.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.806+0000 7f1b50ff9700 1 -- 172.21.15.44:0/2168242996 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b44010430 con 0x7f1b54100710 2023-09-10T00:38:46.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.806+0000 7f1b50ff9700 1 -- 172.21.15.44:0/2168242996 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4400cea0 con 0x7f1b54100710 2023-09-10T00:38:46.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.806+0000 7f1b5a191700 1 -- 172.21.15.44:0/2168242996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5419d860 con 0x7f1b54100710 2023-09-10T00:38:46.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.808+0000 7f1b50ff9700 1 -- 172.21.15.44:0/2168242996 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f1b440105a0 con 0x7f1b54100710 2023-09-10T00:38:46.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.808+0000 7f1b50ff9700 1 --2- 172.21.15.44:0/2168242996 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b3c0a0640 0x7f1b3c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.808+0000 7f1b50ff9700 1 -- 172.21.15.44:0/2168242996 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1b44014070 con 0x7f1b54100710 2023-09-10T00:38:46.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.808+0000 7f1b537fe700 1 --2- 172.21.15.44:0/2168242996 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b3c0a0640 0x7f1b3c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.809+0000 7f1b537fe700 1 --2- 172.21.15.44:0/2168242996 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b3c0a0640 0x7f1b3c0a2b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1b54100570 tx=0x7f1b48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:46.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.809+0000 7f1b66357b80 1 Processor -- start 2023-09-10T00:38:46.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.809+0000 7f1b66357b80 1 -- start start 2023-09-10T00:38:46.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.809+0000 7f1b66357b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114bbb0 0x55c0c114bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.809+0000 7f1b66357b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114c550 0x55c0c114a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.809+0000 7f1b66357b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c114b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.809+0000 7f1b66357b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c10a36e0 con 0x55c0c114adb0 2023-09-10T00:38:46.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.809+0000 7f1b66357b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c10a1000 con 0x55c0c114c550 2023-09-10T00:38:46.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.809+0000 7f1b66357b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c109c8f0 con 0x55c0c114bbb0 2023-09-10T00:38:46.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.809+0000 7f1b53fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c114b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.809+0000 7f1b53fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c114b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46424/0 (socket says 172.21.15.44:46424) 2023-09-10T00:38:46.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.809+0000 7f1b53fff700 1 -- 172.21.15.44:0/1036913569 learned_addr learned my addr 172.21.15.44:0/1036913569 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:46.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.809+0000 7f1b537fe700 1 --2- 172.21.15.44:0/1036913569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114bbb0 0x55c0c114bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.809+0000 7f1b52ffd700 1 --2- 172.21.15.44:0/1036913569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114c550 0x55c0c114a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.810+0000 7f1b53fff700 1 -- 172.21.15.44:0/1036913569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114bbb0 msgr2=0x55c0c114bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:38:46.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.810+0000 7f1b53fff700 1 --2- 172.21.15.44:0/1036913569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114bbb0 0x55c0c114bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.810+0000 7f1b53fff700 1 -- 172.21.15.44:0/1036913569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114c550 msgr2=0x55c0c114a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:38:46.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.810+0000 7f1b53fff700 1 --2- 172.21.15.44:0/1036913569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114c550 0x55c0c114a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.810+0000 7f1b53fff700 1 -- 172.21.15.44:0/1036913569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0c0ec8d90 con 0x55c0c114adb0 2023-09-10T00:38:46.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.810+0000 7f1b53fff700 1 --2- 172.21.15.44:0/1036913569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c114b260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c015000 tx=0x7f1b4c006210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825cd3672f052209 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:46.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.810+0000 7f1b39ffb700 1 -- 172.21.15.44:0/1036913569 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c00e040 con 0x55c0c114adb0 2023-09-10T00:38:46.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.810+0000 7f1b39ffb700 1 -- 172.21.15.44:0/1036913569 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b4c0094c0 con 0x55c0c114adb0 2023-09-10T00:38:46.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.810+0000 7f1b39ffb700 1 -- 172.21.15.44:0/1036913569 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c003dc0 con 0x55c0c114adb0 2023-09-10T00:38:46.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.811+0000 7f1b66357b80 1 -- 172.21.15.44:0/1036913569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 msgr2=0x55c0c114b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.811+0000 7f1b66357b80 1 --2- 172.21.15.44:0/1036913569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c114b260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c015000 tx=0x7f1b4c006210 comp rx=0 tx=0).stop 2023-09-10T00:38:46.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.811+0000 7f1b66357b80 1 -- 172.21.15.44:0/1036913569 shutdown_connections 2023-09-10T00:38:46.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.811+0000 7f1b66357b80 1 --2- 172.21.15.44:0/1036913569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c114b260 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.811+0000 7f1b66357b80 1 --2- 172.21.15.44:0/1036913569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114bbb0 0x55c0c114bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.811+0000 7f1b66357b80 1 --2- 172.21.15.44:0/1036913569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114c550 0x55c0c114a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.811+0000 7f1b66357b80 1 -- 172.21.15.44:0/1036913569 >> 172.21.15.44:0/1036913569 conn(0x55c0c10bf040 msgr2=0x55c0c11456d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:46.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.811+0000 7f1b66357b80 1 -- 172.21.15.44:0/1036913569 shutdown_connections 2023-09-10T00:38:46.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.811+0000 7f1b66357b80 1 -- 172.21.15.44:0/1036913569 wait complete. 2023-09-10T00:38:46.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.811+0000 7f1b66357b80 1 -- start start 2023-09-10T00:38:46.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.811+0000 7f1b66357b80 1 Processor -- start 2023-09-10T00:38:46.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b66357b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c11544b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b66357b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114bbb0 0x55c0c11549f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b66357b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114c550 0x55c0c115ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b66357b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c103ab20 con 0x55c0c114adb0 2023-09-10T00:38:46.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b66357b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c103a820 con 0x55c0c114bbb0 2023-09-10T00:38:46.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b66357b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c0f1e8d0 con 0x55c0c114c550 2023-09-10T00:38:46.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b66357b80 10 client.0 _ll_register_callbacks cb 0x55c0c0f84a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:38:46.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b537fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c11544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b537fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c11544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46426/0 (socket says 172.21.15.44:46426) 2023-09-10T00:38:46.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b537fe700 1 -- 172.21.15.44:0/3717043161 learned_addr learned my addr 172.21.15.44:0/3717043161 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:46.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b53fff700 1 --2- 172.21.15.44:0/3717043161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114c550 0x55c0c115ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b52ffd700 1 --2- 172.21.15.44:0/3717043161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114bbb0 0x55c0c11549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b537fe700 1 -- 172.21.15.44:0/3717043161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114c550 msgr2=0x55c0c115ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:38:46.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b537fe700 1 --2- 172.21.15.44:0/3717043161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114c550 0x55c0c115ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b537fe700 1 -- 172.21.15.44:0/3717043161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114bbb0 msgr2=0x55c0c11549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:38:46.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b537fe700 1 --2- 172.21.15.44:0/3717043161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114bbb0 0x55c0c11549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b537fe700 1 -- 172.21.15.44:0/3717043161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b541082b0 con 0x55c0c114adb0 2023-09-10T00:38:46.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b537fe700 1 --2- 172.21.15.44:0/3717043161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c11544b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1b48021c20 tx=0x7f1b480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b6c3a4a2f26402 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:46.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:38:46.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b32ffd700 1 -- 172.21.15.44:0/3717043161 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48026070 con 0x55c0c114adb0 2023-09-10T00:38:46.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b32ffd700 1 -- 172.21.15.44:0/3717043161 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1b4800e9d0 con 0x55c0c114adb0 2023-09-10T00:38:46.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b66357b80 1 -- 172.21.15.44:0/3717043161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0c10b1850 con 0x55c0c114adb0 2023-09-10T00:38:46.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b32ffd700 1 -- 172.21.15.44:0/3717043161 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b4800ebd0 con 0x55c0c114adb0 2023-09-10T00:38:46.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b66357b80 10 client.14877 fetch_fsmap learned FSMap version 16 2023-09-10T00:38:46.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.812+0000 7f1b66357b80 1 -- 172.21.15.44:0/3717043161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) v3 -- 0x55c0c0eb1c10 con 0x55c0c114adb0 2023-09-10T00:38:46.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.813+0000 7f1b32ffd700 1 -- 172.21.15.44:0/3717043161 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c0c10b1850 con 0x55c0c114adb0 2023-09-10T00:38:46.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.813+0000 7f1b32ffd700 1 -- 172.21.15.44:0/3717043161 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 16) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1b48003a30 con 0x55c0c114adb0 2023-09-10T00:38:46.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.813+0000 7f1b66357b80 10 client.14877 fetch_fsmap finished waiting for FSMap version 16 2023-09-10T00:38:46.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.813+0000 7f1b66357b80 10 client.14877 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-10T00:38:46.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.813+0000 7f1b66357b80 20 client.14877 populate_metadata read hostname 'smithi044' 2023-09-10T00:38:46.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.813+0000 7f1b66357b80 1 --2- 172.21.15.44:0/3717043161 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c0c115fb70 0x55c0c115e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.813+0000 7f1b66357b80 4 client.14877 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:38:46.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.813+0000 7f1b66357b80 1 -- 172.21.15.44:0/3717043161 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0c10958d0 con 0x55c0c115fb70 2023-09-10T00:38:46.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.813+0000 7f1b52ffd700 1 --2- 172.21.15.44:0/3717043161 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c0c115fb70 0x55c0c115e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.814+0000 7f1b52ffd700 1 --2- 172.21.15.44:0/3717043161 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c0c115fb70 0x55c0c115e520 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2835ee9138abd575 server_cookie=73f320c86e0fd2a3 in_seq=0 out_seq=0 2023-09-10T00:38:46.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.814+0000 7f1b32ffd700 10 client.14877 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:38:46.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.814+0000 7f1b32ffd700 1 -- 172.21.15.44:0/3717043161 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c0c10958d0 con 0x55c0c115fb70 2023-09-10T00:38:46.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.814+0000 7f1b32ffd700 10 client.14877 handle_command_reply: tid=0 2023-09-10T00:38:46.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.814+0000 7f1b66357b80 1 client.14877 shutdown 2023-09-10T00:38:46.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.815+0000 7f1b66357b80 1 -- 172.21.15.44:0/3717043161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 msgr2=0x55c0c11544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.815+0000 7f1b66357b80 1 --2- 172.21.15.44:0/3717043161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c11544b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1b48021c20 tx=0x7f1b480047c0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.815+0000 7f1b66357b80 1 -- 172.21.15.44:0/3717043161 shutdown_connections 2023-09-10T00:38:46.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.815+0000 7f1b66357b80 1 --2- 172.21.15.44:0/3717043161 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c0c115fb70 0x55c0c115e520 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.815+0000 7f1b66357b80 1 --2- 172.21.15.44:0/3717043161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114c550 0x55c0c115ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.815+0000 7f1b66357b80 1 --2- 172.21.15.44:0/3717043161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c11544b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.815+0000 7f1b32ffd700 0 client.14877 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:38:46.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.815+0000 7f1b66357b80 1 --2- 172.21.15.44:0/3717043161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114bbb0 0x55c0c11549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.815+0000 7f1b66357b80 1 -- 172.21.15.44:0/3717043161 >> 172.21.15.44:0/3717043161 conn(0x55c0c10bf040 msgr2=0x55c0c1144d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:46.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.815+0000 7f1b66357b80 1 -- 172.21.15.44:0/3717043161 shutdown_connections 2023-09-10T00:38:46.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.815+0000 7f1b66357b80 1 -- 172.21.15.44:0/3717043161 wait complete. 2023-09-10T00:38:46.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.815+0000 7f1b66357b80 20 client.14877 trim_cache size 0 max 16384 2023-09-10T00:38:46.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.832+0000 7f1b66357b80 1 Processor -- start 2023-09-10T00:38:46.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.832+0000 7f1b66357b80 1 -- start start 2023-09-10T00:38:46.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b66357b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c114ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b66357b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114bbb0 0x55c0c115b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b66357b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114c550 0x55c0c114f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b66357b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c1065f70 con 0x55c0c114adb0 2023-09-10T00:38:46.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b66357b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c1058ed0 con 0x55c0c114c550 2023-09-10T00:38:46.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b66357b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c0f2ade0 con 0x55c0c114bbb0 2023-09-10T00:38:46.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b537fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c114ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b537fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c114ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46434/0 (socket says 172.21.15.44:46434) 2023-09-10T00:38:46.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b537fe700 1 -- 172.21.15.44:0/4227517459 learned_addr learned my addr 172.21.15.44:0/4227517459 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:46.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b52ffd700 1 --2- 172.21.15.44:0/4227517459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114bbb0 0x55c0c115b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b53fff700 1 --2- 172.21.15.44:0/4227517459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114c550 0x55c0c114f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b537fe700 1 -- 172.21.15.44:0/4227517459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114bbb0 msgr2=0x55c0c115b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:38:46.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b537fe700 1 --2- 172.21.15.44:0/4227517459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114bbb0 0x55c0c115b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b537fe700 1 -- 172.21.15.44:0/4227517459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114c550 msgr2=0x55c0c114f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:38:46.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b537fe700 1 --2- 172.21.15.44:0/4227517459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114c550 0x55c0c114f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b537fe700 1 -- 172.21.15.44:0/4227517459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0c0eef200 con 0x55c0c114adb0 2023-09-10T00:38:46.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b53fff700 1 --2- 172.21.15.44:0/4227517459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114c550 0x55c0c114f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:38:46.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b537fe700 1 --2- 172.21.15.44:0/4227517459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c114ee30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1b48013010 tx=0x7f1b4800e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94eba697a262a32 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:46.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b3a7fc700 1 -- 172.21.15.44:0/4227517459 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48026070 con 0x55c0c114adb0 2023-09-10T00:38:46.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b3a7fc700 1 -- 172.21.15.44:0/4227517459 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b48003680 con 0x55c0c114adb0 2023-09-10T00:38:46.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.833+0000 7f1b3a7fc700 1 -- 172.21.15.44:0/4227517459 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48004290 con 0x55c0c114adb0 2023-09-10T00:38:46.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.834+0000 7f1b66357b80 1 -- 172.21.15.44:0/4227517459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 msgr2=0x55c0c114ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.834+0000 7f1b66357b80 1 --2- 172.21.15.44:0/4227517459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c114ee30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1b48013010 tx=0x7f1b4800e710 comp rx=0 tx=0).stop 2023-09-10T00:38:46.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.834+0000 7f1b66357b80 1 -- 172.21.15.44:0/4227517459 shutdown_connections 2023-09-10T00:38:46.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.834+0000 7f1b66357b80 1 --2- 172.21.15.44:0/4227517459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114c550 0x55c0c114f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.834+0000 7f1b66357b80 1 --2- 172.21.15.44:0/4227517459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114adb0 0x55c0c114ee30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.834+0000 7f1b66357b80 1 --2- 172.21.15.44:0/4227517459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114bbb0 0x55c0c115b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.834+0000 7f1b66357b80 1 -- 172.21.15.44:0/4227517459 >> 172.21.15.44:0/4227517459 conn(0x55c0c10bf040 msgr2=0x55c0c1152320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:46.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.834+0000 7f1b66357b80 1 -- 172.21.15.44:0/4227517459 shutdown_connections 2023-09-10T00:38:46.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.834+0000 7f1b66357b80 1 -- 172.21.15.44:0/4227517459 wait complete. 2023-09-10T00:38:46.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.834+0000 7f1b66357b80 1 -- start start 2023-09-10T00:38:46.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.834+0000 7f1b66357b80 1 Processor -- start 2023-09-10T00:38:46.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b66357b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114adb0 0x55c0c1156530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b66357b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114bbb0 0x55c0c1156a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b66357b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114c550 0x55c0c11b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b66357b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c115baa0 con 0x55c0c114bbb0 2023-09-10T00:38:46.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b66357b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c115bc10 con 0x55c0c114c550 2023-09-10T00:38:46.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b66357b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c115bdd0 con 0x55c0c114adb0 2023-09-10T00:38:46.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b66357b80 10 client.0 _ll_register_callbacks cb 0x7f1b48021820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:38:46.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b52ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114bbb0 0x55c0c1156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b52ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114bbb0 0x55c0c1156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46442/0 (socket says 172.21.15.44:46442) 2023-09-10T00:38:46.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b52ffd700 1 -- 172.21.15.44:0/3587160737 learned_addr learned my addr 172.21.15.44:0/3587160737 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:46.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b537fe700 1 --2- 172.21.15.44:0/3587160737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114adb0 0x55c0c1156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b53fff700 1 --2- 172.21.15.44:0/3587160737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114c550 0x55c0c11b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b52ffd700 1 -- 172.21.15.44:0/3587160737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114adb0 msgr2=0x55c0c1156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:38:46.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b52ffd700 1 --2- 172.21.15.44:0/3587160737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114adb0 0x55c0c1156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b52ffd700 1 -- 172.21.15.44:0/3587160737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114c550 msgr2=0x55c0c11b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:38:46.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b52ffd700 1 --2- 172.21.15.44:0/3587160737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114c550 0x55c0c11b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b52ffd700 1 -- 172.21.15.44:0/3587160737 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0c0eb1c10 con 0x55c0c114bbb0 2023-09-10T00:38:46.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b537fe700 1 --2- 172.21.15.44:0/3587160737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114adb0 0x55c0c1156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:38:46.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b52ffd700 1 --2- 172.21.15.44:0/3587160737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114bbb0 0x55c0c1156a70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400c2d0 tx=0x7f1b44095220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f84a9809f1674af server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:46.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:38:46.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.835+0000 7f1b32ffd700 1 -- 172.21.15.44:0/3587160737 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44095040 con 0x55c0c114bbb0 2023-09-10T00:38:46.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.836+0000 7f1b32ffd700 1 -- 172.21.15.44:0/3587160737 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1b440ace40 con 0x55c0c114bbb0 2023-09-10T00:38:46.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.836+0000 7f1b66357b80 1 -- 172.21.15.44:0/3587160737 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0c0ff44a0 con 0x55c0c114bbb0 2023-09-10T00:38:46.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.836+0000 7f1b32ffd700 1 -- 172.21.15.44:0/3587160737 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b4409d170 con 0x55c0c114bbb0 2023-09-10T00:38:46.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.836+0000 7f1b66357b80 10 client.14883 fetch_fsmap learned FSMap version 16 2023-09-10T00:38:46.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.836+0000 7f1b66357b80 1 -- 172.21.15.44:0/3587160737 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) v3 -- 0x55c0c0ec7260 con 0x55c0c114bbb0 2023-09-10T00:38:46.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.836+0000 7f1b32ffd700 1 -- 172.21.15.44:0/3587160737 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c0c0ff44a0 con 0x55c0c114bbb0 2023-09-10T00:38:46.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.836+0000 7f1b32ffd700 1 -- 172.21.15.44:0/3587160737 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 16) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1b440998a0 con 0x55c0c114bbb0 2023-09-10T00:38:46.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.836+0000 7f1b66357b80 10 client.14883 fetch_fsmap finished waiting for FSMap version 16 2023-09-10T00:38:46.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.836+0000 7f1b66357b80 10 client.14883 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-10T00:38:46.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.836+0000 7f1b66357b80 20 client.14883 populate_metadata read hostname 'smithi044' 2023-09-10T00:38:46.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.836+0000 7f1b66357b80 1 --2- 172.21.15.44:0/3587160737 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c0c11b91f0 0x55c0c11b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:46.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.836+0000 7f1b66357b80 4 client.14883 mds_command: new command op to 24605 tid=0[{"prefix": "status"}] 2023-09-10T00:38:46.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.836+0000 7f1b66357b80 1 -- 172.21.15.44:0/3587160737 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c0c1056060 con 0x55c0c11b91f0 2023-09-10T00:38:46.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.836+0000 7f1b537fe700 1 --2- 172.21.15.44:0/3587160737 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c0c11b91f0 0x55c0c11b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:46.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b32ffd700 10 client.14883 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:38:46.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b537fe700 1 --2- 172.21.15.44:0/3587160737 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c0c11b91f0 0x55c0c11b5670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88c59f2c2995de8e server_cookie=8bd8ead36ce076a1 in_seq=0 out_seq=0 2023-09-10T00:38:46.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b32ffd700 1 -- 172.21.15.44:0/3587160737 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55c0c1056060 con 0x55c0c11b91f0 2023-09-10T00:38:46.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b32ffd700 10 client.14883 handle_command_reply: tid=0 2023-09-10T00:38:46.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b66357b80 1 client.14883 shutdown 2023-09-10T00:38:46.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b66357b80 1 -- 172.21.15.44:0/3587160737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114bbb0 msgr2=0x55c0c1156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b66357b80 1 --2- 172.21.15.44:0/3587160737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114bbb0 0x55c0c1156a70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400c2d0 tx=0x7f1b44095220 comp rx=0 tx=0).stop 2023-09-10T00:38:46.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b66357b80 1 -- 172.21.15.44:0/3587160737 shutdown_connections 2023-09-10T00:38:46.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b66357b80 1 --2- 172.21.15.44:0/3587160737 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c0c11b91f0 0x55c0c11b5670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b66357b80 1 --2- 172.21.15.44:0/3587160737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0c114c550 0x55c0c11b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b66357b80 1 --2- 172.21.15.44:0/3587160737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0c114adb0 0x55c0c1156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b66357b80 1 --2- 172.21.15.44:0/3587160737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c0c114bbb0 0x55c0c1156a70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b32ffd700 0 client.14883 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:38:46.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b66357b80 1 -- 172.21.15.44:0/3587160737 >> 172.21.15.44:0/3587160737 conn(0x55c0c10bf040 msgr2=0x55c0c114ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:46.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b66357b80 1 -- 172.21.15.44:0/3587160737 shutdown_connections 2023-09-10T00:38:46.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b66357b80 1 -- 172.21.15.44:0/3587160737 wait complete. 2023-09-10T00:38:46.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.837+0000 7f1b66357b80 20 client.14883 trim_cache size 0 max 16384 2023-09-10T00:38:46.858 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:38:46.858 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da", 2023-09-10T00:38:46.858 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2023-09-10T00:38:46.859 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24605, 2023-09-10T00:38:46.859 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2023-09-10T00:38:46.859 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2023-09-10T00:38:46.859 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2023-09-10T00:38:46.859 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 7.0927968259999998, 2023-09-10T00:38:46.860 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 16, 2023-09-10T00:38:46.860 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 87, 2023-09-10T00:38:46.860 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 87, 2023-09-10T00:38:46.860 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 13.445170758 2023-09-10T00:38:46.860 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:38:46.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.838+0000 7f1b5a191700 1 -- 172.21.15.44:0/2168242996 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b3c0a0640 msgr2=0x7f1b3c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.838+0000 7f1b5a191700 1 --2- 172.21.15.44:0/2168242996 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b3c0a0640 0x7f1b3c0a2b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1b54100570 tx=0x7f1b48009260 comp rx=0 tx=0).stop 2023-09-10T00:38:46.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.838+0000 7f1b5a191700 1 -- 172.21.15.44:0/2168242996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b54100710 msgr2=0x7f1b54195710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:46.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.838+0000 7f1b5a191700 1 --2- 172.21.15.44:0/2168242996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b54100710 0x7f1b54195710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400e9a0 tx=0x7f1b4400ecb0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.838+0000 7f1b5a191700 1 -- 172.21.15.44:0/2168242996 shutdown_connections 2023-09-10T00:38:46.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.838+0000 7f1b5a191700 1 --2- 172.21.15.44:0/2168242996 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1b3c0a0640 0x7f1b3c0a2b00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.838+0000 7f1b5a191700 1 --2- 172.21.15.44:0/2168242996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b541010b0 0x7f1b5419c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.838+0000 7f1b5a191700 1 --2- 172.21.15.44:0/2168242996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b540ff520 0x7f1b541951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.838+0000 7f1b5a191700 1 --2- 172.21.15.44:0/2168242996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b54100710 0x7f1b54195710 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:46.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.838+0000 7f1b5a191700 1 -- 172.21.15.44:0/2168242996 >> 172.21.15.44:0/2168242996 conn(0x7f1b540faa90 msgr2=0x7f1b540fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:46.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.838+0000 7f1b5a191700 1 -- 172.21.15.44:0/2168242996 shutdown_connections 2023-09-10T00:38:46.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:46.838+0000 7f1b5a191700 1 -- 172.21.15.44:0/2168242996 wait complete. 2023-09-10T00:38:46.864 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-10T00:38:46.865 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-09-10T00:38:46.865 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph fs ls 2023-09-10T00:38:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:46 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1333667150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:46 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3200331920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:46 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/125302933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:47.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:46 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1333667150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:47.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:46 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3200331920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:47.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:46 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/125302933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:46 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1333667150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:46 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3200331920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:38:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:46 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/125302933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:47.422 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:38:47.906 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:47 smithi044 ceph-mon[98676]: pgmap v272: 129 pgs: 1 creating+activating, 13 creating+peering, 110 active+clean, 5 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 8 op/s 2023-09-10T00:38:47.906 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:47 smithi044 ceph-mon[98676]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-10T00:38:47.907 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:47 smithi044 ceph-mon[98676]: Cluster is now healthy 2023-09-10T00:38:48.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:47 smithi131 ceph-mon[103713]: pgmap v272: 129 pgs: 1 creating+activating, 13 creating+peering, 110 active+clean, 5 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 8 op/s 2023-09-10T00:38:48.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:47 smithi131 ceph-mon[103713]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-10T00:38:48.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:47 smithi131 ceph-mon[103713]: Cluster is now healthy 2023-09-10T00:38:48.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:47 smithi099 ceph-mon[102692]: pgmap v272: 129 pgs: 1 creating+activating, 13 creating+peering, 110 active+clean, 5 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 8 op/s 2023-09-10T00:38:48.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:47 smithi099 ceph-mon[102692]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-10T00:38:48.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:47 smithi099 ceph-mon[102692]: Cluster is now healthy 2023-09-10T00:38:48.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.628+0000 7f2fbf613700 1 -- 172.21.15.44:0/133654887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb80ff520 msgr2=0x7f2fb80ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:48.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.628+0000 7f2fbf613700 1 --2- 172.21.15.44:0/133654887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb80ff520 0x7f2fb80ff9a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009990 tx=0x7f2fb4009ca0 comp rx=0 tx=0).stop 2023-09-10T00:38:48.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.628+0000 7f2fbf613700 1 -- 172.21.15.44:0/133654887 shutdown_connections 2023-09-10T00:38:48.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.628+0000 7f2fbf613700 1 --2- 172.21.15.44:0/133654887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fb80ffee0 0x7f2fb810c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:48.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.628+0000 7f2fbf613700 1 --2- 172.21.15.44:0/133654887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8104530 0x7f2fb8104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:48.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.628+0000 7f2fbf613700 1 --2- 172.21.15.44:0/133654887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb80ff520 0x7f2fb80ff9a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:48.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.628+0000 7f2fbf613700 1 -- 172.21.15.44:0/133654887 >> 172.21.15.44:0/133654887 conn(0x7f2fb8076600 msgr2=0x7f2fb8076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:48.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.628+0000 7f2fbf613700 1 -- 172.21.15.44:0/133654887 shutdown_connections 2023-09-10T00:38:48.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.628+0000 7f2fbf613700 1 -- 172.21.15.44:0/133654887 wait complete. 2023-09-10T00:38:48.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.629+0000 7f2fbf613700 1 Processor -- start 2023-09-10T00:38:48.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.629+0000 7f2fbf613700 1 -- start start 2023-09-10T00:38:48.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.629+0000 7f2fbf613700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb80ff520 0x7f2fb81954d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:48.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.629+0000 7f2fbf613700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fb80ffee0 0x7f2fb8195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:48.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.629+0000 7f2fbf613700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8104530 0x7f2fb8199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:48.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.629+0000 7f2fbf613700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb819a490 con 0x7f2fb80ff520 2023-09-10T00:38:48.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.629+0000 7f2fbf613700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb819a600 con 0x7f2fb80ffee0 2023-09-10T00:38:48.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.629+0000 7f2fbf613700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb819a7c0 con 0x7f2fb8104530 2023-09-10T00:38:48.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.629+0000 7f2fbd3af700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb80ff520 0x7f2fb81954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:48.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.630+0000 7f2fbd3af700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb80ff520 0x7f2fb81954d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46452/0 (socket says 172.21.15.44:46452) 2023-09-10T00:38:48.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.630+0000 7f2fbd3af700 1 -- 172.21.15.44:0/2526620063 learned_addr learned my addr 172.21.15.44:0/2526620063 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:48.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.630+0000 7f2fbcbae700 1 --2- 172.21.15.44:0/2526620063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fb80ffee0 0x7f2fb8195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:48.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.630+0000 7f2fbdbb0700 1 --2- 172.21.15.44:0/2526620063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8104530 0x7f2fb8199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:48.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.630+0000 7f2fbd3af700 1 -- 172.21.15.44:0/2526620063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8104530 msgr2=0x7f2fb8199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:48.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.630+0000 7f2fbd3af700 1 --2- 172.21.15.44:0/2526620063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8104530 0x7f2fb8199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:48.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.630+0000 7f2fbd3af700 1 -- 172.21.15.44:0/2526620063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fb80ffee0 msgr2=0x7f2fb8195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:48.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.630+0000 7f2fbd3af700 1 --2- 172.21.15.44:0/2526620063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fb80ffee0 0x7f2fb8195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:48.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.630+0000 7f2fbd3af700 1 -- 172.21.15.44:0/2526620063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb40095f0 con 0x7f2fb80ff520 2023-09-10T00:38:48.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.630+0000 7f2fbd3af700 1 --2- 172.21.15.44:0/2526620063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb80ff520 0x7f2fb81954d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800d980 tx=0x7f2fa800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:48.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.630+0000 7f2fae7fc700 1 -- 172.21.15.44:0/2526620063 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8004020 con 0x7f2fb80ff520 2023-09-10T00:38:48.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.630+0000 7f2fae7fc700 1 -- 172.21.15.44:0/2526620063 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fa8009730 con 0x7f2fb80ff520 2023-09-10T00:38:48.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.630+0000 7f2fbf613700 1 -- 172.21.15.44:0/2526620063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb819aaa0 con 0x7f2fb80ff520 2023-09-10T00:38:48.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.630+0000 7f2fae7fc700 1 -- 172.21.15.44:0/2526620063 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8003a00 con 0x7f2fb80ff520 2023-09-10T00:38:48.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.630+0000 7f2fbf613700 1 -- 172.21.15.44:0/2526620063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb8068290 con 0x7f2fb80ff520 2023-09-10T00:38:48.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.631+0000 7f2fbf613700 1 -- 172.21.15.44:0/2526620063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb804e300 con 0x7f2fb80ff520 2023-09-10T00:38:48.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.632+0000 7f2fae7fc700 1 -- 172.21.15.44:0/2526620063 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f2fa80098a0 con 0x7f2fb80ff520 2023-09-10T00:38:48.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.633+0000 7f2fae7fc700 1 --2- 172.21.15.44:0/2526620063 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2fa40a06c0 0x7f2fa40a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:48.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.633+0000 7f2fbcbae700 1 --2- 172.21.15.44:0/2526620063 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2fa40a06c0 0x7f2fa40a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:48.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.633+0000 7f2fae7fc700 1 -- 172.21.15.44:0/2526620063 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2fa80cbd50 con 0x7f2fb80ff520 2023-09-10T00:38:48.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.633+0000 7f2fbcbae700 1 --2- 172.21.15.44:0/2526620063 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2fa40a06c0 0x7f2fa40a2b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8196af0 tx=0x7f2fb4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:48.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.636+0000 7f2fae7fc700 1 -- 172.21.15.44:0/2526620063 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2fa8095710 con 0x7f2fb80ff520 2023-09-10T00:38:48.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.885+0000 7f2fbf613700 1 -- 172.21.15.44:0/2526620063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2fb810f440 con 0x7f2fb80ff520 2023-09-10T00:38:48.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.885+0000 7f2fae7fc700 1 -- 172.21.15.44:0/2526620063 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+88 (secure 0 0 0) 0x7f2fa8098d30 con 0x7f2fb80ff520 2023-09-10T00:38:48.887 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-10T00:38:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.887+0000 7f2fbf613700 1 -- 172.21.15.44:0/2526620063 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2fa40a06c0 msgr2=0x7f2fa40a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.887+0000 7f2fbf613700 1 --2- 172.21.15.44:0/2526620063 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2fa40a06c0 0x7f2fa40a2b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8196af0 tx=0x7f2fb4005fb0 comp rx=0 tx=0).stop 2023-09-10T00:38:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.887+0000 7f2fbf613700 1 -- 172.21.15.44:0/2526620063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb80ff520 msgr2=0x7f2fb81954d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.887+0000 7f2fbf613700 1 --2- 172.21.15.44:0/2526620063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb80ff520 0x7f2fb81954d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800d980 tx=0x7f2fa800dc90 comp rx=0 tx=0).stop 2023-09-10T00:38:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.887+0000 7f2fbf613700 1 -- 172.21.15.44:0/2526620063 shutdown_connections 2023-09-10T00:38:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.887+0000 7f2fbf613700 1 --2- 172.21.15.44:0/2526620063 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2fa40a06c0 0x7f2fa40a2b80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.888+0000 7f2fbf613700 1 --2- 172.21.15.44:0/2526620063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8104530 0x7f2fb8199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.888+0000 7f2fbf613700 1 --2- 172.21.15.44:0/2526620063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb80ff520 0x7f2fb81954d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:48.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.888+0000 7f2fbf613700 1 --2- 172.21.15.44:0/2526620063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fb80ffee0 0x7f2fb8195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.888+0000 7f2fbf613700 1 -- 172.21.15.44:0/2526620063 >> 172.21.15.44:0/2526620063 conn(0x7f2fb8076600 msgr2=0x7f2fb80fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.888+0000 7f2fbf613700 1 -- 172.21.15.44:0/2526620063 shutdown_connections 2023-09-10T00:38:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:48.888+0000 7f2fbf613700 1 -- 172.21.15.44:0/2526620063 wait complete. 2023-09-10T00:38:49.282 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-09-10T00:38:49.283 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-10T00:38:49.283 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2023-09-10T00:38:49.283 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-09-10T00:38:49.284 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-09-10T00:38:49.284 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-09-10T00:38:49.284 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-10T00:38:49.285 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-10T00:38:49.285 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-09-10T00:38:49.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:49.286 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2023-09-10T00:38:49.304 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-10T00:38:49.304 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-10T00:38:49.304 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2023-09-10T00:38:49.305 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2023-09-10T00:38:49.305 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2023-09-10T00:38:49.305 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2023-09-10T00:38:49.305 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-10T00:38:49.305 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2023-09-10T00:38:49.306 INFO:teuthology.orchestra.run.smithi044.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-10T00:38:49.306 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2023-09-10T00:38:49.306 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-09-10T00:38:49.306 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 41922sec preferred_lft 41922sec 2023-09-10T00:38:49.306 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link 2023-09-10T00:38:49.306 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2023-09-10T00:38:49.307 INFO:teuthology.orchestra.run.smithi044.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-10T00:38:49.307 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2023-09-10T00:38:49.307 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-10T00:38:49.307 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2023-09-10T00:38:49.308 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-10T00:38:49.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:49.308 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2023-09-10T00:38:49.308 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2023-09-10T00:38:49.309 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add name ceph-brx type bridge 2023-09-10T00:38:49.309 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr flush dev ceph-brx 2023-09-10T00:38:49.309 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set ceph-brx up 2023-09-10T00:38:49.310 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-10T00:38:49.310 DEBUG:teuthology.orchestra.run.smithi044:> ') 2023-09-10T00:38:49.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:49.479 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-10T00:38:49.508 INFO:teuthology.orchestra.run.smithi044.stdout:1 2023-09-10T00:38:49.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:49.510 DEBUG:teuthology.orchestra.run.smithi044:> route 2023-09-10T00:38:49.577 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2023-09-10T00:38:49.578 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-10T00:38:49.578 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-09-10T00:38:49.578 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-09-10T00:38:49.578 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-10T00:38:49.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:49.579 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2023-09-10T00:38:49.580 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2023-09-10T00:38:49.580 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-09-10T00:38:49.580 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-09-10T00:38:49.581 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-09-10T00:38:49.581 DEBUG:teuthology.orchestra.run.smithi044:> ') 2023-09-10T00:38:49.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:49.863 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2023-09-10T00:38:49.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:49.882 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list-id 2023-09-10T00:38:49.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:49.938 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2023-09-10T00:38:49.939 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2023-09-10T00:38:49.939 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-10T00:38:49.940 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-09-10T00:38:49.940 DEBUG:teuthology.orchestra.run.smithi044:> ') 2023-09-10T00:38:50.030 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:49 smithi044 ceph-mon[98676]: pgmap v273: 129 pgs: 1 creating+activating, 5 creating+peering, 123 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2023-09-10T00:38:50.030 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:49 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2526620063' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-10T00:38:50.034 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-09-10T00:38:50.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:50.035 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2023-09-10T00:38:50.035 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2023-09-10T00:38:50.036 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-09-10T00:38:50.036 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-10T00:38:50.036 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-09-10T00:38:50.037 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-09-10T00:38:50.037 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-09-10T00:38:50.037 DEBUG:teuthology.orchestra.run.smithi044:> ') 2023-09-10T00:38:50.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:49 smithi131 ceph-mon[103713]: pgmap v273: 129 pgs: 1 creating+activating, 5 creating+peering, 123 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2023-09-10T00:38:50.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:49 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2526620063' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-10T00:38:50.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:49 smithi099 ceph-mon[102692]: pgmap v273: 129 pgs: 1 creating+activating, 5 creating+peering, 123 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2023-09-10T00:38:50.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:49 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2526620063' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-10T00:38:50.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:50.260 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2023-09-10T00:38:50.261 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2023-09-10T00:38:50.261 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2023-09-10T00:38:50.261 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.0 master ceph-brx 2023-09-10T00:38:50.262 DEBUG:teuthology.orchestra.run.smithi044:> ') 2023-09-10T00:38:50.362 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-09-10T00:38:50.362 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-10T00:38:50.363 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-09-10T00:38:50.380 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-09-10T00:38:50.380 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-10T00:38:50.381 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-09-10T00:38:50.436 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2023-09-10T00:38:50.505 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-10T00:38:50.505 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-10T00:38:50.573 INFO:teuthology.orchestra.run.smithi044.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-10T00:38:50.574 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-10T00:38:50.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:50.575 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2023-09-10T00:38:50.632 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-10T00:38:50.633 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-09-10T00:38:50.674 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2023-09-10T00:38:50.702 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-10T00:38:50.703 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-10T00:38:50.769 INFO:teuthology.orchestra.run.smithi044.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-10T00:38:50.770 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-10T00:38:50.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:50.771 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2023-09-10T00:38:50.782 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi044.stderr:2023-09-10T00:38:50.781+0000 7f827a4ce5c0 -1 init, newargv = 0x56520ef8c290 newargc=15 2023-09-10T00:38:50.783 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi044.stderr:ceph-fuse[157558]: starting ceph client 2023-09-10T00:38:50.788 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi044.stderr:ceph-fuse[157558]: starting fuse 2023-09-10T00:38:50.800 INFO:teuthology.orchestra.run.smithi044.stdout:124 2023-09-10T00:38:50.801 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2023-09-10T00:38:50.801 DEBUG:teuthology.orchestra.run.smithi044:> sudo stdin-killer -- python3 -c ' 2023-09-10T00:38:50.802 DEBUG:teuthology.orchestra.run.smithi044:> import glob 2023-09-10T00:38:50.802 DEBUG:teuthology.orchestra.run.smithi044:> import re 2023-09-10T00:38:50.802 DEBUG:teuthology.orchestra.run.smithi044:> import os 2023-09-10T00:38:50.803 DEBUG:teuthology.orchestra.run.smithi044:> import subprocess 2023-09-10T00:38:50.803 DEBUG:teuthology.orchestra.run.smithi044:> 2023-09-10T00:38:50.804 DEBUG:teuthology.orchestra.run.smithi044:> def _find_admin_socket(client_name): 2023-09-10T00:38:50.804 DEBUG:teuthology.orchestra.run.smithi044:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-09-10T00:38:50.804 DEBUG:teuthology.orchestra.run.smithi044:> files = glob.glob(asok_path) 2023-09-10T00:38:50.805 DEBUG:teuthology.orchestra.run.smithi044:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-09-10T00:38:50.805 DEBUG:teuthology.orchestra.run.smithi044:> 2023-09-10T00:38:50.805 DEBUG:teuthology.orchestra.run.smithi044:> # Given a non-glob path, it better be there 2023-09-10T00:38:50.806 DEBUG:teuthology.orchestra.run.smithi044:> if "*" not in asok_path: 2023-09-10T00:38:50.806 DEBUG:teuthology.orchestra.run.smithi044:> assert(len(files) == 1) 2023-09-10T00:38:50.806 DEBUG:teuthology.orchestra.run.smithi044:> return files[0] 2023-09-10T00:38:50.807 DEBUG:teuthology.orchestra.run.smithi044:> 2023-09-10T00:38:50.807 DEBUG:teuthology.orchestra.run.smithi044:> for f in files: 2023-09-10T00:38:50.808 DEBUG:teuthology.orchestra.run.smithi044:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-10T00:38:50.808 DEBUG:teuthology.orchestra.run.smithi044:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-10T00:38:50.808 DEBUG:teuthology.orchestra.run.smithi044:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-10T00:38:50.809 DEBUG:teuthology.orchestra.run.smithi044:> contents = proc_f.read() 2023-09-10T00:38:50.809 DEBUG:teuthology.orchestra.run.smithi044:> if mountpoint in contents: 2023-09-10T00:38:50.809 DEBUG:teuthology.orchestra.run.smithi044:> return f 2023-09-10T00:38:50.810 DEBUG:teuthology.orchestra.run.smithi044:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-10T00:38:50.810 DEBUG:teuthology.orchestra.run.smithi044:> 2023-09-10T00:38:50.810 DEBUG:teuthology.orchestra.run.smithi044:> print(_find_admin_socket("client.admin")) 2023-09-10T00:38:50.811 DEBUG:teuthology.orchestra.run.smithi044:> ' 2023-09-10T00:38:50.927 INFO:teuthology.orchestra.run.smithi044.stdout:/var/run/ceph/ceph-client.admin.157558.asok 2023-09-10T00:38:50.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-10T00:38:50.937 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.157558.asok 2023-09-10T00:38:50.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:50.938 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.157558.asok status 2023-09-10T00:38:51.097 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:38:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "metadata": { 2023-09-10T00:38:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_sha1": "9670cca5bc8e17916865dc84939edfc10b1f2c00", 2023-09-10T00:38:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev)", 2023-09-10T00:38:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "entity_id": "admin", 2023-09-10T00:38:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": "smithi044", 2023-09-10T00:38:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-09-10T00:38:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: "pid": "157558", 2023-09-10T00:38:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: "root": "/" 2023-09-10T00:38:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T00:38:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_count": 0, 2023-09-10T00:38:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_pinned_count": 0, 2023-09-10T00:38:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 14895, 2023-09-10T00:38:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: "inst": { 2023-09-10T00:38:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: "name": { 2023-09-10T00:38:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "client", 2023-09-10T00:38:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 14895 2023-09-10T00:38:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T00:38:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2023-09-10T00:38:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2023-09-10T00:38:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2023-09-10T00:38:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3986112082 2023-09-10T00:38:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:38:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T00:38:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2023-09-10T00:38:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2023-09-10T00:38:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2023-09-10T00:38:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3986112082 2023-09-10T00:38:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T00:38:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: "inst_str": "client.14895 192.168.0.1:0/3986112082", 2023-09-10T00:38:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: "addr_str": "192.168.0.1:0/3986112082", 2023-09-10T00:38:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: "inode_count": 1, 2023-09-10T00:38:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_epoch": 16, 2023-09-10T00:38:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch": 87, 2023-09-10T00:38:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch_barrier": 0, 2023-09-10T00:38:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklisted": false, 2023-09-10T00:38:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs" 2023-09-10T00:38:51.104 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:38:51.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:51.108 DEBUG:teuthology.orchestra.run.smithi044:> stat --file-system '--printf=%T 2023-09-10T00:38:51.108 DEBUG:teuthology.orchestra.run.smithi044:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-09-10T00:38:51.126 INFO:teuthology.orchestra.run.smithi044.stdout:fuseblk 2023-09-10T00:38:51.126 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-09-10T00:38:51.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:51.127 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-09-10T00:38:51.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:51.194 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-09-10T00:38:51.298 INFO:teuthology.orchestra.run.smithi044.stdout:# file: . 2023-09-10T00:38:51.299 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-09-10T00:38:51.299 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:51.299 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:38:51.300 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-10T00:38:51.318 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:38:51.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.483+0000 7fbd54bbb700 1 -- 172.21.15.44:0/3648496038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd501006e0 msgr2=0x7fbd50100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:51.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.483+0000 7fbd54bbb700 1 --2- 172.21.15.44:0/3648496038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd501006e0 0x7fbd50100b40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbd400099c0 tx=0x7fbd40009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.483+0000 7fbd54bbb700 1 -- 172.21.15.44:0/3648496038 shutdown_connections 2023-09-10T00:38:51.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.483+0000 7fbd54bbb700 1 --2- 172.21.15.44:0/3648496038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd50101080 0x7fbd50107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.483+0000 7fbd54bbb700 1 --2- 172.21.15.44:0/3648496038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd500ff4f0 0x7fbd500ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.485 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.483+0000 7fbd54bbb700 1 --2- 172.21.15.44:0/3648496038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd501006e0 0x7fbd50100b40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.483+0000 7fbd54bbb700 1 -- 172.21.15.44:0/3648496038 >> 172.21.15.44:0/3648496038 conn(0x7fbd500faa20 msgr2=0x7fbd500fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:51.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.483+0000 7fbd54bbb700 1 -- 172.21.15.44:0/3648496038 shutdown_connections 2023-09-10T00:38:51.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.483+0000 7fbd54bbb700 1 -- 172.21.15.44:0/3648496038 wait complete. 2023-09-10T00:38:51.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.484+0000 7fbd54bbb700 1 Processor -- start 2023-09-10T00:38:51.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.484+0000 7fbd54bbb700 1 -- start start 2023-09-10T00:38:51.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.484+0000 7fbd54bbb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd500ff4f0 0x7fbd50196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:51.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.484+0000 7fbd54bbb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd501006e0 0x7fbd501966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:51.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.484+0000 7fbd54bbb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd50101080 0x7fbd5019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:51.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.484+0000 7fbd54bbb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5019dcf0 con 0x7fbd500ff4f0 2023-09-10T00:38:51.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.484+0000 7fbd54bbb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5019de30 con 0x7fbd50101080 2023-09-10T00:38:51.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.484+0000 7fbd54bbb700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5019dfc0 con 0x7fbd501006e0 2023-09-10T00:38:51.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.484+0000 7fbd4e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd500ff4f0 0x7fbd50196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:51.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.484+0000 7fbd4e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd500ff4f0 0x7fbd50196170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46478/0 (socket says 172.21.15.44:46478) 2023-09-10T00:38:51.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.484+0000 7fbd4e59c700 1 -- 172.21.15.44:0/114839335 learned_addr learned my addr 172.21.15.44:0/114839335 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:51.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.484+0000 7fbd4ed9d700 1 --2- 172.21.15.44:0/114839335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd50101080 0x7fbd5019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:51.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.484+0000 7fbd4dd9b700 1 --2- 172.21.15.44:0/114839335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd501006e0 0x7fbd501966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:51.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.485+0000 7fbd4e59c700 1 -- 172.21.15.44:0/114839335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd501006e0 msgr2=0x7fbd501966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:51.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.485+0000 7fbd4e59c700 1 --2- 172.21.15.44:0/114839335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd501006e0 0x7fbd501966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.485+0000 7fbd4e59c700 1 -- 172.21.15.44:0/114839335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd50101080 msgr2=0x7fbd5019d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:51.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.485+0000 7fbd4e59c700 1 --2- 172.21.15.44:0/114839335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd50101080 0x7fbd5019d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.485+0000 7fbd4e59c700 1 -- 172.21.15.44:0/114839335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd400095f0 con 0x7fbd500ff4f0 2023-09-10T00:38:51.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.485+0000 7fbd4e59c700 1 --2- 172.21.15.44:0/114839335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd500ff4f0 0x7fbd50196170 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400d980 tx=0x7fbd4400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:51.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.485+0000 7fbd3f7fe700 1 -- 172.21.15.44:0/114839335 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4400de60 con 0x7fbd500ff4f0 2023-09-10T00:38:51.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.485+0000 7fbd3f7fe700 1 -- 172.21.15.44:0/114839335 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd440044b0 con 0x7fbd500ff4f0 2023-09-10T00:38:51.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.485+0000 7fbd3f7fe700 1 -- 172.21.15.44:0/114839335 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44003b40 con 0x7fbd500ff4f0 2023-09-10T00:38:51.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.485+0000 7fbd54bbb700 1 -- 172.21.15.44:0/114839335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd5019e2a0 con 0x7fbd500ff4f0 2023-09-10T00:38:51.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.485+0000 7fbd54bbb700 1 -- 172.21.15.44:0/114839335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5019e760 con 0x7fbd500ff4f0 2023-09-10T00:38:51.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.486+0000 7fbd54bbb700 1 -- 172.21.15.44:0/114839335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5010abc0 con 0x7fbd500ff4f0 2023-09-10T00:38:51.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.487+0000 7fbd3f7fe700 1 -- 172.21.15.44:0/114839335 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fbd44004870 con 0x7fbd500ff4f0 2023-09-10T00:38:51.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.488+0000 7fbd3f7fe700 1 --2- 172.21.15.44:0/114839335 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbd380a0690 0x7fbd380a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:51.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.488+0000 7fbd3f7fe700 1 -- 172.21.15.44:0/114839335 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbd440cbe40 con 0x7fbd500ff4f0 2023-09-10T00:38:51.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.488+0000 7fbd4dd9b700 1 --2- 172.21.15.44:0/114839335 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbd380a0690 0x7fbd380a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:51.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.488+0000 7fbd4dd9b700 1 --2- 172.21.15.44:0/114839335 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbd380a0690 0x7fbd380a2b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbd400098e0 tx=0x7fbd40000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:51.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.491+0000 7fbd3f7fe700 1 -- 172.21.15.44:0/114839335 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbd44095780 con 0x7fbd500ff4f0 2023-09-10T00:38:51.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.695+0000 7fbd54bbb700 1 -- 172.21.15.44:0/114839335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd5010ae30 con 0x7fbd500ff4f0 2023-09-10T00:38:51.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.697+0000 7fbd3f7fe700 1 -- 172.21.15.44:0/114839335 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23870 (secure 0 0 0) 0x7fbd44009940 con 0x7fbd500ff4f0 2023-09-10T00:38:51.697 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:51.698 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":87,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:38:44.197902+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:38:51.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.699+0000 7fbd54bbb700 1 -- 172.21.15.44:0/114839335 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbd380a0690 msgr2=0x7fbd380a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:51.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.699+0000 7fbd54bbb700 1 --2- 172.21.15.44:0/114839335 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbd380a0690 0x7fbd380a2b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbd400098e0 tx=0x7fbd40000bc0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.699+0000 7fbd54bbb700 1 -- 172.21.15.44:0/114839335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd500ff4f0 msgr2=0x7fbd50196170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:51.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.699+0000 7fbd54bbb700 1 --2- 172.21.15.44:0/114839335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd500ff4f0 0x7fbd50196170 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400d980 tx=0x7fbd4400dd40 comp rx=0 tx=0).stop 2023-09-10T00:38:51.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.699+0000 7fbd54bbb700 1 -- 172.21.15.44:0/114839335 shutdown_connections 2023-09-10T00:38:51.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.699+0000 7fbd54bbb700 1 --2- 172.21.15.44:0/114839335 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbd380a0690 0x7fbd380a2b50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.699+0000 7fbd54bbb700 1 --2- 172.21.15.44:0/114839335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd50101080 0x7fbd5019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.699+0000 7fbd54bbb700 1 --2- 172.21.15.44:0/114839335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd500ff4f0 0x7fbd50196170 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.699+0000 7fbd54bbb700 1 --2- 172.21.15.44:0/114839335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd501006e0 0x7fbd501966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.699+0000 7fbd54bbb700 1 -- 172.21.15.44:0/114839335 >> 172.21.15.44:0/114839335 conn(0x7fbd500faa20 msgr2=0x7fbd50068810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:51.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.699+0000 7fbd54bbb700 1 -- 172.21.15.44:0/114839335 shutdown_connections 2023-09-10T00:38:51.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.699+0000 7fbd54bbb700 1 -- 172.21.15.44:0/114839335 wait complete. 2023-09-10T00:38:51.711 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-10T00:34:44.456741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-10T00:38:37.194891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-10T00:38:38.573268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-10T00:38:42.198896+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-10T00:38:51.712 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-10T00:38:51.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.844+0000 7fb64b46c700 1 -- 172.21.15.44:0/261798070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6441006e0 msgr2=0x7fb644100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:51.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.844+0000 7fb64b46c700 1 --2- 172.21.15.44:0/261798070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6441006e0 0x7fb644100b40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb6380099c0 tx=0x7fb638009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.844+0000 7fb64b46c700 1 -- 172.21.15.44:0/261798070 shutdown_connections 2023-09-10T00:38:51.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.844+0000 7fb64b46c700 1 --2- 172.21.15.44:0/261798070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb644101080 0x7fb644107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.844+0000 7fb64b46c700 1 --2- 172.21.15.44:0/261798070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6440ff4f0 0x7fb6440ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.844+0000 7fb64b46c700 1 --2- 172.21.15.44:0/261798070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6441006e0 0x7fb644100b40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.844+0000 7fb64b46c700 1 -- 172.21.15.44:0/261798070 >> 172.21.15.44:0/261798070 conn(0x7fb6440faa60 msgr2=0x7fb6440fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:51.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.844+0000 7fb64b46c700 1 -- 172.21.15.44:0/261798070 shutdown_connections 2023-09-10T00:38:51.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.844+0000 7fb64b46c700 1 -- 172.21.15.44:0/261798070 wait complete. 2023-09-10T00:38:51.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb64b46c700 1 Processor -- start 2023-09-10T00:38:51.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb64b46c700 1 -- start start 2023-09-10T00:38:51.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb64b46c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6440ff4f0 0x7fb6441961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:51.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb64b46c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6441006e0 0x7fb644196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:51.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb64b46c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb644101080 0x7fb64419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:51.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb64b46c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64419dd70 con 0x7fb644101080 2023-09-10T00:38:51.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb64b46c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64419deb0 con 0x7fb6441006e0 2023-09-10T00:38:51.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb64b46c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64419e040 con 0x7fb6440ff4f0 2023-09-10T00:38:51.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb649a09700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb644101080 0x7fb64419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:51.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb649a09700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb644101080 0x7fb64419d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46504/0 (socket says 172.21.15.44:46504) 2023-09-10T00:38:51.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb649a09700 1 -- 172.21.15.44:0/1825149603 learned_addr learned my addr 172.21.15.44:0/1825149603 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:51.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb649208700 1 --2- 172.21.15.44:0/1825149603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6440ff4f0 0x7fb6441961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb648a07700 1 --2- 172.21.15.44:0/1825149603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6441006e0 0x7fb644196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb649a09700 1 -- 172.21.15.44:0/1825149603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6440ff4f0 msgr2=0x7fb6441961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb649a09700 1 --2- 172.21.15.44:0/1825149603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6440ff4f0 0x7fb6441961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb649a09700 1 -- 172.21.15.44:0/1825149603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6441006e0 msgr2=0x7fb644196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb649a09700 1 --2- 172.21.15.44:0/1825149603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6441006e0 0x7fb644196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb649a09700 1 -- 172.21.15.44:0/1825149603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6380095f0 con 0x7fb644101080 2023-09-10T00:38:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.845+0000 7fb648a07700 1 --2- 172.21.15.44:0/1825149603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6441006e0 0x7fb644196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:38:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.846+0000 7fb649a09700 1 --2- 172.21.15.44:0/1825149603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb644101080 0x7fb64419d7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb63c007d70 tx=0x7fb63c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.846+0000 7fb6367fc700 1 -- 172.21.15.44:0/1825149603 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c00ad40 con 0x7fb644101080 2023-09-10T00:38:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.846+0000 7fb6367fc700 1 -- 172.21.15.44:0/1825149603 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb63c0128b0 con 0x7fb644101080 2023-09-10T00:38:51.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.846+0000 7fb64b46c700 1 -- 172.21.15.44:0/1825149603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64419e320 con 0x7fb644101080 2023-09-10T00:38:51.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.846+0000 7fb6367fc700 1 -- 172.21.15.44:0/1825149603 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0115a0 con 0x7fb644101080 2023-09-10T00:38:51.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.846+0000 7fb64b46c700 1 -- 172.21.15.44:0/1825149603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64419e7e0 con 0x7fb644101080 2023-09-10T00:38:51.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.847+0000 7fb64b46c700 1 -- 172.21.15.44:0/1825149603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb64410ac10 con 0x7fb644101080 2023-09-10T00:38:51.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.848+0000 7fb6367fc700 1 -- 172.21.15.44:0/1825149603 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fb63c012a20 con 0x7fb644101080 2023-09-10T00:38:51.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.848+0000 7fb6367fc700 1 --2- 172.21.15.44:0/1825149603 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb6300a0640 0x7fb6300a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:51.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.848+0000 7fb649208700 1 --2- 172.21.15.44:0/1825149603 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb6300a0640 0x7fb6300a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:51.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.848+0000 7fb6367fc700 1 -- 172.21.15.44:0/1825149603 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb63c0cbfb0 con 0x7fb644101080 2023-09-10T00:38:51.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.849+0000 7fb649208700 1 --2- 172.21.15.44:0/1825149603 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb6300a0640 0x7fb6300a2b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb644100540 tx=0x7fb640009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:51.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:51.853+0000 7fb6367fc700 1 -- 172.21.15.44:0/1825149603 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb63c0fee10 con 0x7fb644101080 2023-09-10T00:38:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:51 smithi131 ceph-mon[103713]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2023-09-10T00:38:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:51 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/114839335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:52.054 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.054+0000 7fb64b46c700 1 -- 172.21.15.44:0/1825149603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb64419ea60 con 0x7fb644101080 2023-09-10T00:38:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:51 smithi044 ceph-mon[98676]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2023-09-10T00:38:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:51 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/114839335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:52.055 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.055+0000 7fb6367fc700 1 -- 172.21.15.44:0/1825149603 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb63c0958f0 con 0x7fb644101080 2023-09-10T00:38:52.056 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2023-09-10T00:38:52.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.057+0000 7fb64b46c700 1 -- 172.21.15.44:0/1825149603 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb6300a0640 msgr2=0x7fb6300a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:52.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.057+0000 7fb64b46c700 1 --2- 172.21.15.44:0/1825149603 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb6300a0640 0x7fb6300a2b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb644100540 tx=0x7fb640009260 comp rx=0 tx=0).stop 2023-09-10T00:38:52.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.057+0000 7fb64b46c700 1 -- 172.21.15.44:0/1825149603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb644101080 msgr2=0x7fb64419d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:52.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.057+0000 7fb64b46c700 1 --2- 172.21.15.44:0/1825149603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb644101080 0x7fb64419d7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb63c007d70 tx=0x7fb63c00a340 comp rx=0 tx=0).stop 2023-09-10T00:38:52.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.057+0000 7fb64b46c700 1 -- 172.21.15.44:0/1825149603 shutdown_connections 2023-09-10T00:38:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.057+0000 7fb64b46c700 1 --2- 172.21.15.44:0/1825149603 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb6300a0640 0x7fb6300a2b00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.057+0000 7fb64b46c700 1 --2- 172.21.15.44:0/1825149603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb644101080 0x7fb64419d7a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.057+0000 7fb64b46c700 1 --2- 172.21.15.44:0/1825149603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6440ff4f0 0x7fb6441961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.057+0000 7fb64b46c700 1 --2- 172.21.15.44:0/1825149603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6441006e0 0x7fb644196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.057+0000 7fb64b46c700 1 -- 172.21.15.44:0/1825149603 >> 172.21.15.44:0/1825149603 conn(0x7fb6440faa60 msgr2=0x7fb644067760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.057+0000 7fb64b46c700 1 -- 172.21.15.44:0/1825149603 shutdown_connections 2023-09-10T00:38:52.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.057+0000 7fb64b46c700 1 -- 172.21.15.44:0/1825149603 wait complete. 2023-09-10T00:38:52.069 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-10T00:38:52.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:51 smithi099 ceph-mon[102692]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2023-09-10T00:38:52.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:51 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/114839335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:52.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.198+0000 7f42487f2700 1 -- 172.21.15.44:0/12301595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42401006e0 msgr2=0x7f4240100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:52.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.198+0000 7f42487f2700 1 --2- 172.21.15.44:0/12301595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42401006e0 0x7f4240100b40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f423c0099c0 tx=0x7f423c009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.198+0000 7f42487f2700 1 -- 172.21.15.44:0/12301595 shutdown_connections 2023-09-10T00:38:52.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.198+0000 7f42487f2700 1 --2- 172.21.15.44:0/12301595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4240101080 0x7f4240107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.198+0000 7f42487f2700 1 --2- 172.21.15.44:0/12301595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42400ff4f0 0x7f42400ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.198+0000 7f42487f2700 1 --2- 172.21.15.44:0/12301595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42401006e0 0x7f4240100b40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.198+0000 7f42487f2700 1 -- 172.21.15.44:0/12301595 >> 172.21.15.44:0/12301595 conn(0x7f42400faa60 msgr2=0x7f42400fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:52.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.198+0000 7f42487f2700 1 -- 172.21.15.44:0/12301595 shutdown_connections 2023-09-10T00:38:52.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.198+0000 7f42487f2700 1 -- 172.21.15.44:0/12301595 wait complete. 2023-09-10T00:38:52.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f42487f2700 1 Processor -- start 2023-09-10T00:38:52.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f42487f2700 1 -- start start 2023-09-10T00:38:52.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f42487f2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42400ff4f0 0x7f4240103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:52.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f42487f2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42401006e0 0x7f4240101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:52.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f42487f2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4240101080 0x7f4240102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:52.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f42487f2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42401027a0 con 0x7f4240101080 2023-09-10T00:38:52.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f42487f2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4240102910 con 0x7f42400ff4f0 2023-09-10T00:38:52.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f42487f2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42401a02e0 con 0x7f42401006e0 2023-09-10T00:38:52.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f4246d8f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4240101080 0x7f4240102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:52.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f4246d8f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4240101080 0x7f4240102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46518/0 (socket says 172.21.15.44:46518) 2023-09-10T00:38:52.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f4246d8f700 1 -- 172.21.15.44:0/2453021543 learned_addr learned my addr 172.21.15.44:0/2453021543 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:52.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f424658e700 1 --2- 172.21.15.44:0/2453021543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42400ff4f0 0x7f4240103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:52.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f4245d8d700 1 --2- 172.21.15.44:0/2453021543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42401006e0 0x7f4240101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:52.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f4246d8f700 1 -- 172.21.15.44:0/2453021543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42401006e0 msgr2=0x7f4240101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:52.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f4246d8f700 1 --2- 172.21.15.44:0/2453021543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42401006e0 0x7f4240101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f4246d8f700 1 -- 172.21.15.44:0/2453021543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42400ff4f0 msgr2=0x7f4240103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:52.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f4246d8f700 1 --2- 172.21.15.44:0/2453021543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42400ff4f0 0x7f4240103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f4246d8f700 1 -- 172.21.15.44:0/2453021543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c0095f0 con 0x7f4240101080 2023-09-10T00:38:52.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.199+0000 7f4246d8f700 1 --2- 172.21.15.44:0/2453021543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4240101080 0x7f4240102000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4238007ee0 tx=0x7f423800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:52.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.200+0000 7f42377fe700 1 -- 172.21.15.44:0/2453021543 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238017070 con 0x7f4240101080 2023-09-10T00:38:52.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.200+0000 7f42377fe700 1 -- 172.21.15.44:0/2453021543 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f423800f040 con 0x7f4240101080 2023-09-10T00:38:52.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.200+0000 7f42487f2700 1 -- 172.21.15.44:0/2453021543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42401a05c0 con 0x7f4240101080 2023-09-10T00:38:52.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.200+0000 7f42377fe700 1 -- 172.21.15.44:0/2453021543 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238005570 con 0x7f4240101080 2023-09-10T00:38:52.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.200+0000 7f42487f2700 1 -- 172.21.15.44:0/2453021543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42401a0a80 con 0x7f4240101080 2023-09-10T00:38:52.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.201+0000 7f42487f2700 1 -- 172.21.15.44:0/2453021543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f424010ac10 con 0x7f4240101080 2023-09-10T00:38:52.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.202+0000 7f42377fe700 1 -- 172.21.15.44:0/2453021543 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f423800db20 con 0x7f4240101080 2023-09-10T00:38:52.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.202+0000 7f42377fe700 1 --2- 172.21.15.44:0/2453021543 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f422c0a0640 0x7f422c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:52.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.202+0000 7f424658e700 1 --2- 172.21.15.44:0/2453021543 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f422c0a0640 0x7f422c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:52.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.202+0000 7f42377fe700 1 -- 172.21.15.44:0/2453021543 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f42380cbb50 con 0x7f4240101080 2023-09-10T00:38:52.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.203+0000 7f424658e700 1 --2- 172.21.15.44:0/2453021543 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f422c0a0640 0x7f422c0a2b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4240100540 tx=0x7f423000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:52.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.206+0000 7f42377fe700 1 -- 172.21.15.44:0/2453021543 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4238095490 con 0x7f4240101080 2023-09-10T00:38:52.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.410+0000 7f42487f2700 1 -- 172.21.15.44:0/2453021543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42401a0ca0 con 0x7f4240101080 2023-09-10T00:38:52.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.410+0000 7f42377fe700 1 -- 172.21.15.44:0/2453021543 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4238098ab0 con 0x7f4240101080 2023-09-10T00:38:52.411 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:38:52.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.412+0000 7f42487f2700 1 -- 172.21.15.44:0/2453021543 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f422c0a0640 msgr2=0x7f422c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:52.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.412+0000 7f42487f2700 1 --2- 172.21.15.44:0/2453021543 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f422c0a0640 0x7f422c0a2b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4240100540 tx=0x7f423000b410 comp rx=0 tx=0).stop 2023-09-10T00:38:52.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.412+0000 7f42487f2700 1 -- 172.21.15.44:0/2453021543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4240101080 msgr2=0x7f4240102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:52.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.412+0000 7f42487f2700 1 --2- 172.21.15.44:0/2453021543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4240101080 0x7f4240102000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4238007ee0 tx=0x7f423800d3b0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.412+0000 7f42487f2700 1 -- 172.21.15.44:0/2453021543 shutdown_connections 2023-09-10T00:38:52.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.412+0000 7f42487f2700 1 --2- 172.21.15.44:0/2453021543 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f422c0a0640 0x7f422c0a2b00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.412+0000 7f42487f2700 1 --2- 172.21.15.44:0/2453021543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4240101080 0x7f4240102000 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.412+0000 7f42487f2700 1 --2- 172.21.15.44:0/2453021543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42400ff4f0 0x7f4240103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.412+0000 7f42487f2700 1 --2- 172.21.15.44:0/2453021543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42401006e0 0x7f4240101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.412+0000 7f42487f2700 1 -- 172.21.15.44:0/2453021543 >> 172.21.15.44:0/2453021543 conn(0x7f42400faa60 msgr2=0x7f42400fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:52.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.413+0000 7f42487f2700 1 -- 172.21.15.44:0/2453021543 shutdown_connections 2023-09-10T00:38:52.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.413+0000 7f42487f2700 1 -- 172.21.15.44:0/2453021543 wait complete. 2023-09-10T00:38:52.425 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-10T00:38:52.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.553+0000 7fb51fa77700 1 -- 172.21.15.44:0/868616067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5181017e0 msgr2=0x7fb518101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:52.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.553+0000 7fb51fa77700 1 --2- 172.21.15.44:0/868616067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5181017e0 0x7fb518101c40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb5140099c0 tx=0x7fb514009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.553+0000 7fb51fa77700 1 -- 172.21.15.44:0/868616067 shutdown_connections 2023-09-10T00:38:52.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.553+0000 7fb51fa77700 1 --2- 172.21.15.44:0/868616067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb518102180 0x7fb518108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.553+0000 7fb51fa77700 1 --2- 172.21.15.44:0/868616067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5181005f0 0x7fb518100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.553+0000 7fb51fa77700 1 --2- 172.21.15.44:0/868616067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5181017e0 0x7fb518101c40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.553+0000 7fb51fa77700 1 -- 172.21.15.44:0/868616067 >> 172.21.15.44:0/868616067 conn(0x7fb5180fbb40 msgr2=0x7fb5180fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:52.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.553+0000 7fb51fa77700 1 -- 172.21.15.44:0/868616067 shutdown_connections 2023-09-10T00:38:52.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.553+0000 7fb51fa77700 1 -- 172.21.15.44:0/868616067 wait complete. 2023-09-10T00:38:52.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51fa77700 1 Processor -- start 2023-09-10T00:38:52.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51fa77700 1 -- start start 2023-09-10T00:38:52.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51fa77700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5181005f0 0x7fb5180743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:52.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51fa77700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5181017e0 0x7fb5180729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:52.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51fa77700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb518102180 0x7fb518072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:52.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51fa77700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5180736d0 con 0x7fb5181005f0 2023-09-10T00:38:52.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51fa77700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb518073840 con 0x7fb518102180 2023-09-10T00:38:52.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51fa77700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5181a02c0 con 0x7fb5181017e0 2023-09-10T00:38:52.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51d813700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5181005f0 0x7fb5180743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:52.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51d813700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5181005f0 0x7fb5180743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46528/0 (socket says 172.21.15.44:46528) 2023-09-10T00:38:52.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51d813700 1 -- 172.21.15.44:0/2891475886 learned_addr learned my addr 172.21.15.44:0/2891475886 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:52.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51d012700 1 --2- 172.21.15.44:0/2891475886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5181017e0 0x7fb5180729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:52.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51e014700 1 --2- 172.21.15.44:0/2891475886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb518102180 0x7fb518072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:52.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51d813700 1 -- 172.21.15.44:0/2891475886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5181017e0 msgr2=0x7fb5180729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:52.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51d813700 1 --2- 172.21.15.44:0/2891475886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5181017e0 0x7fb5180729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51d813700 1 -- 172.21.15.44:0/2891475886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb518102180 msgr2=0x7fb518072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:52.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51d813700 1 --2- 172.21.15.44:0/2891475886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb518102180 0x7fb518072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51d813700 1 -- 172.21.15.44:0/2891475886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5140095f0 con 0x7fb5181005f0 2023-09-10T00:38:52.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51e014700 1 --2- 172.21.15.44:0/2891475886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb518102180 0x7fb518072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:38:52.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.554+0000 7fb51d813700 1 --2- 172.21.15.44:0/2891475886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5181005f0 0x7fb5180743b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb50800d980 tx=0x7fb50800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:52.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.555+0000 7fb50effd700 1 -- 172.21.15.44:0/2891475886 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50800b950 con 0x7fb5181005f0 2023-09-10T00:38:52.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.555+0000 7fb50effd700 1 -- 172.21.15.44:0/2891475886 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb508007620 con 0x7fb5181005f0 2023-09-10T00:38:52.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.555+0000 7fb51fa77700 1 -- 172.21.15.44:0/2891475886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5181a05a0 con 0x7fb5181005f0 2023-09-10T00:38:52.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.555+0000 7fb50effd700 1 -- 172.21.15.44:0/2891475886 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50800f3c0 con 0x7fb5181005f0 2023-09-10T00:38:52.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.555+0000 7fb51fa77700 1 -- 172.21.15.44:0/2891475886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5181a0b00 con 0x7fb5181005f0 2023-09-10T00:38:52.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.556+0000 7fb51fa77700 1 -- 172.21.15.44:0/2891475886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51810bd00 con 0x7fb5181005f0 2023-09-10T00:38:52.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.557+0000 7fb50effd700 1 -- 172.21.15.44:0/2891475886 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fb50800bab0 con 0x7fb5181005f0 2023-09-10T00:38:52.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.557+0000 7fb50effd700 1 --2- 172.21.15.44:0/2891475886 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb5040a06c0 0x7fb5040a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:52.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.558+0000 7fb50effd700 1 -- 172.21.15.44:0/2891475886 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb5080cc4c0 con 0x7fb5181005f0 2023-09-10T00:38:52.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.558+0000 7fb51d012700 1 --2- 172.21.15.44:0/2891475886 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb5040a06c0 0x7fb5040a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:52.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.558+0000 7fb51d012700 1 --2- 172.21.15.44:0/2891475886 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb5040a06c0 0x7fb5040a2b80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb514006010 tx=0x7fb514009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:52.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.561+0000 7fb50effd700 1 -- 172.21.15.44:0/2891475886 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb508095da0 con 0x7fb5181005f0 2023-09-10T00:38:52.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.764+0000 7fb51fa77700 1 -- 172.21.15.44:0/2891475886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb5181075c0 con 0x7fb5181005f0 2023-09-10T00:38:52.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.764+0000 7fb50effd700 1 -- 172.21.15.44:0/2891475886 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb5080160a0 con 0x7fb5181005f0 2023-09-10T00:38:52.765 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2023-09-10T00:38:52.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.766+0000 7fb51fa77700 1 -- 172.21.15.44:0/2891475886 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb5040a06c0 msgr2=0x7fb5040a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:52.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.766+0000 7fb51fa77700 1 --2- 172.21.15.44:0/2891475886 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb5040a06c0 0x7fb5040a2b80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb514006010 tx=0x7fb514009f10 comp rx=0 tx=0).stop 2023-09-10T00:38:52.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.766+0000 7fb51fa77700 1 -- 172.21.15.44:0/2891475886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5181005f0 msgr2=0x7fb5180743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:52.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.766+0000 7fb51fa77700 1 --2- 172.21.15.44:0/2891475886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5181005f0 0x7fb5180743b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb50800d980 tx=0x7fb50800dc90 comp rx=0 tx=0).stop 2023-09-10T00:38:52.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.766+0000 7fb51fa77700 1 -- 172.21.15.44:0/2891475886 shutdown_connections 2023-09-10T00:38:52.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.766+0000 7fb51fa77700 1 --2- 172.21.15.44:0/2891475886 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb5040a06c0 0x7fb5040a2b80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.766+0000 7fb51fa77700 1 --2- 172.21.15.44:0/2891475886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb518102180 0x7fb518072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.766+0000 7fb51fa77700 1 --2- 172.21.15.44:0/2891475886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5181005f0 0x7fb5180743b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.766+0000 7fb51fa77700 1 --2- 172.21.15.44:0/2891475886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5181017e0 0x7fb5180729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.766+0000 7fb51fa77700 1 -- 172.21.15.44:0/2891475886 >> 172.21.15.44:0/2891475886 conn(0x7fb5180fbb40 msgr2=0x7fb5180fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:52.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.766+0000 7fb51fa77700 1 -- 172.21.15.44:0/2891475886 shutdown_connections 2023-09-10T00:38:52.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.766+0000 7fb51fa77700 1 -- 172.21.15.44:0/2891475886 wait complete. 2023-09-10T00:38:52.779 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-10T00:38:52.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.910+0000 7f3cd7ead700 1 -- 172.21.15.44:0/3564582051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cd01006e0 msgr2=0x7f3cd0100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:52.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.910+0000 7f3cd7ead700 1 --2- 172.21.15.44:0/3564582051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cd01006e0 0x7f3cd0100b40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc0099c0 tx=0x7f3ccc009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.910+0000 7f3cd7ead700 1 -- 172.21.15.44:0/3564582051 shutdown_connections 2023-09-10T00:38:52.917 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.910+0000 7f3cd7ead700 1 --2- 172.21.15.44:0/3564582051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cd0101080 0x7f3cd0107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.917 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.910+0000 7f3cd7ead700 1 --2- 172.21.15.44:0/3564582051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cd00ff4f0 0x7f3cd00ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.910+0000 7f3cd7ead700 1 --2- 172.21.15.44:0/3564582051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cd01006e0 0x7f3cd0100b40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.910+0000 7f3cd7ead700 1 -- 172.21.15.44:0/3564582051 >> 172.21.15.44:0/3564582051 conn(0x7f3cd00faa60 msgr2=0x7f3cd00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:52.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.910+0000 7f3cd7ead700 1 -- 172.21.15.44:0/3564582051 shutdown_connections 2023-09-10T00:38:52.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.910+0000 7f3cd7ead700 1 -- 172.21.15.44:0/3564582051 wait complete. 2023-09-10T00:38:52.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.910+0000 7f3cd7ead700 1 Processor -- start 2023-09-10T00:38:52.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.910+0000 7f3cd7ead700 1 -- start start 2023-09-10T00:38:52.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.910+0000 7f3cd7ead700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cd00ff4f0 0x7f3cd00682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:52.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.910+0000 7f3cd7ead700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cd01006e0 0x7f3cd019af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:52.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.910+0000 7f3cd7ead700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cd0101080 0x7f3cd019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:52.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.910+0000 7f3cd7ead700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd019ddb0 con 0x7f3cd01006e0 2023-09-10T00:38:52.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cd7ead700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd019df20 con 0x7f3cd0101080 2023-09-10T00:38:52.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cd7ead700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd019e0e0 con 0x7f3cd00ff4f0 2023-09-10T00:38:52.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cd5448700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cd01006e0 0x7f3cd019af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:52.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cd5448700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cd01006e0 0x7f3cd019af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46544/0 (socket says 172.21.15.44:46544) 2023-09-10T00:38:52.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cd5448700 1 -- 172.21.15.44:0/1819455754 learned_addr learned my addr 172.21.15.44:0/1819455754 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:52.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cd5c49700 1 --2- 172.21.15.44:0/1819455754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cd00ff4f0 0x7f3cd00682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:52.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cd5448700 1 -- 172.21.15.44:0/1819455754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cd00ff4f0 msgr2=0x7f3cd00682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:52.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cd5448700 1 --2- 172.21.15.44:0/1819455754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cd00ff4f0 0x7f3cd00682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cd644a700 1 --2- 172.21.15.44:0/1819455754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cd0101080 0x7f3cd019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:52.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cd5448700 1 -- 172.21.15.44:0/1819455754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cd0101080 msgr2=0x7f3cd019d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:52.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cd5448700 1 --2- 172.21.15.44:0/1819455754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cd0101080 0x7f3cd019d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:52.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cd5448700 1 -- 172.21.15.44:0/1819455754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc0095f0 con 0x7f3cd01006e0 2023-09-10T00:38:52.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cd644a700 1 --2- 172.21.15.44:0/1819455754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cd0101080 0x7f3cd019d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:38:52.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cd5448700 1 --2- 172.21.15.44:0/1819455754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cd01006e0 0x7f3cd019af90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc000c00 tx=0x7f3ccc00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:52.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cc6ffd700 1 -- 172.21.15.44:0/1819455754 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc01d070 con 0x7f3cd01006e0 2023-09-10T00:38:52.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cd7ead700 1 -- 172.21.15.44:0/1819455754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd019e360 con 0x7f3cd01006e0 2023-09-10T00:38:52.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cc6ffd700 1 -- 172.21.15.44:0/1819455754 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ccc00fe70 con 0x7f3cd01006e0 2023-09-10T00:38:52.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cc6ffd700 1 -- 172.21.15.44:0/1819455754 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc0173f0 con 0x7f3cd01006e0 2023-09-10T00:38:52.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.911+0000 7f3cd7ead700 1 -- 172.21.15.44:0/1819455754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd019e780 con 0x7f3cd01006e0 2023-09-10T00:38:52.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.912+0000 7f3cd7ead700 1 -- 172.21.15.44:0/1819455754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd0106560 con 0x7f3cd01006e0 2023-09-10T00:38:52.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.913+0000 7f3cc6ffd700 1 -- 172.21.15.44:0/1819455754 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f3ccc022470 con 0x7f3cd01006e0 2023-09-10T00:38:52.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.914+0000 7f3cc6ffd700 1 --2- 172.21.15.44:0/1819455754 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3cbc0a0640 0x7f3cbc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:52.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.914+0000 7f3cc6ffd700 1 -- 172.21.15.44:0/1819455754 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3ccc0ccff0 con 0x7f3cd01006e0 2023-09-10T00:38:52.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.914+0000 7f3cd5c49700 1 --2- 172.21.15.44:0/1819455754 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3cbc0a0640 0x7f3cbc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:52.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.914+0000 7f3cd5c49700 1 --2- 172.21.15.44:0/1819455754 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3cbc0a0640 0x7f3cbc0a2b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0100540 tx=0x7f3cc0007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:52.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:52.918+0000 7f3cc6ffd700 1 -- 172.21.15.44:0/1819455754 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3ccc0969c0 con 0x7f3cd01006e0 2023-09-10T00:38:53.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:52 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1825149603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:38:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:52 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2453021543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:38:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:52 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2891475886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:38:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:52 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1825149603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:38:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:52 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2453021543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:38:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:52 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2891475886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:38:53.121 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.121+0000 7f3cd7ead700 1 -- 172.21.15.44:0/1819455754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3cd002cf70 con 0x7f3cd01006e0 2023-09-10T00:38:53.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.121+0000 7f3cc6ffd700 1 -- 172.21.15.44:0/1819455754 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3ccc099fe0 con 0x7f3cd01006e0 2023-09-10T00:38:53.122 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:38:53.124 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.123+0000 7f3cd7ead700 1 -- 172.21.15.44:0/1819455754 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3cbc0a0640 msgr2=0x7f3cbc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:53.124 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.123+0000 7f3cd7ead700 1 --2- 172.21.15.44:0/1819455754 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3cbc0a0640 0x7f3cbc0a2b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0100540 tx=0x7f3cc0007400 comp rx=0 tx=0).stop 2023-09-10T00:38:53.124 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.123+0000 7f3cd7ead700 1 -- 172.21.15.44:0/1819455754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cd01006e0 msgr2=0x7f3cd019af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:53.125 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.123+0000 7f3cd7ead700 1 --2- 172.21.15.44:0/1819455754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cd01006e0 0x7f3cd019af90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc000c00 tx=0x7f3ccc00fa00 comp rx=0 tx=0).stop 2023-09-10T00:38:53.125 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.123+0000 7f3cd7ead700 1 -- 172.21.15.44:0/1819455754 shutdown_connections 2023-09-10T00:38:53.125 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.123+0000 7f3cd7ead700 1 --2- 172.21.15.44:0/1819455754 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3cbc0a0640 0x7f3cbc0a2b00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.125 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.123+0000 7f3cd7ead700 1 --2- 172.21.15.44:0/1819455754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cd0101080 0x7f3cd019d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.125 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.123+0000 7f3cd7ead700 1 --2- 172.21.15.44:0/1819455754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cd00ff4f0 0x7f3cd00682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.126 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.123+0000 7f3cd7ead700 1 --2- 172.21.15.44:0/1819455754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cd01006e0 0x7f3cd019af90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.126 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.123+0000 7f3cd7ead700 1 -- 172.21.15.44:0/1819455754 >> 172.21.15.44:0/1819455754 conn(0x7f3cd00faa60 msgr2=0x7f3cd00fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:53.126 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.123+0000 7f3cd7ead700 1 -- 172.21.15.44:0/1819455754 shutdown_connections 2023-09-10T00:38:53.126 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.124+0000 7f3cd7ead700 1 -- 172.21.15.44:0/1819455754 wait complete. 2023-09-10T00:38:53.136 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-10T00:38:53.263 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.262+0000 7f968a5b0700 1 -- 172.21.15.44:0/2259998083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96841017e0 msgr2=0x7f9684101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:53.263 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.262+0000 7f968a5b0700 1 --2- 172.21.15.44:0/2259998083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96841017e0 0x7f9684101c40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f96780099c0 tx=0x7f9678009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.264 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.262+0000 7f968a5b0700 1 -- 172.21.15.44:0/2259998083 shutdown_connections 2023-09-10T00:38:53.264 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.262+0000 7f968a5b0700 1 --2- 172.21.15.44:0/2259998083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9684102180 0x7f9684108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.264 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.262+0000 7f968a5b0700 1 --2- 172.21.15.44:0/2259998083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96841005f0 0x7f9684100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.264 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.262+0000 7f968a5b0700 1 --2- 172.21.15.44:0/2259998083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96841017e0 0x7f9684101c40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.264 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.262+0000 7f968a5b0700 1 -- 172.21.15.44:0/2259998083 >> 172.21.15.44:0/2259998083 conn(0x7f96840fbb80 msgr2=0x7f96840fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:53.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.262+0000 7f968a5b0700 1 -- 172.21.15.44:0/2259998083 shutdown_connections 2023-09-10T00:38:53.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.262+0000 7f968a5b0700 1 -- 172.21.15.44:0/2259998083 wait complete. 2023-09-10T00:38:53.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.263+0000 7f968a5b0700 1 Processor -- start 2023-09-10T00:38:53.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.263+0000 7f968a5b0700 1 -- start start 2023-09-10T00:38:53.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.263+0000 7f968a5b0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96841005f0 0x7f96841961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:53.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.263+0000 7f968a5b0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96841017e0 0x7f9684196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:53.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.263+0000 7f968a5b0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9684102180 0x7f968419d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:53.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.263+0000 7f968a5b0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968419dd60 con 0x7f96841005f0 2023-09-10T00:38:53.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.263+0000 7f968a5b0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968419dea0 con 0x7f96841017e0 2023-09-10T00:38:53.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.263+0000 7f968a5b0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968419e030 con 0x7f9684102180 2023-09-10T00:38:53.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.264+0000 7f9683fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96841005f0 0x7f96841961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:53.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.264+0000 7f9683fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96841005f0 0x7f96841961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46556/0 (socket says 172.21.15.44:46556) 2023-09-10T00:38:53.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.264+0000 7f9683fff700 1 -- 172.21.15.44:0/711445482 learned_addr learned my addr 172.21.15.44:0/711445482 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:53.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.264+0000 7f9688b4d700 1 --2- 172.21.15.44:0/711445482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9684102180 0x7f968419d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:53.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.264+0000 7f96837fe700 1 --2- 172.21.15.44:0/711445482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96841017e0 0x7f9684196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:53.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.264+0000 7f9683fff700 1 -- 172.21.15.44:0/711445482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9684102180 msgr2=0x7f968419d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:53.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.264+0000 7f9683fff700 1 --2- 172.21.15.44:0/711445482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9684102180 0x7f968419d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.264+0000 7f9683fff700 1 -- 172.21.15.44:0/711445482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96841017e0 msgr2=0x7f9684196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:53.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.264+0000 7f9683fff700 1 --2- 172.21.15.44:0/711445482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96841017e0 0x7f9684196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.264+0000 7f9683fff700 1 -- 172.21.15.44:0/711445482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96780095f0 con 0x7f96841005f0 2023-09-10T00:38:53.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.264+0000 7f9683fff700 1 --2- 172.21.15.44:0/711445482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96841005f0 0x7f96841961e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f967400d980 tx=0x7f967400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:53.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.264+0000 7f96817fa700 1 -- 172.21.15.44:0/711445482 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674004020 con 0x7f96841005f0 2023-09-10T00:38:53.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.264+0000 7f968a5b0700 1 -- 172.21.15.44:0/711445482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968419e310 con 0x7f96841005f0 2023-09-10T00:38:53.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.264+0000 7f96817fa700 1 -- 172.21.15.44:0/711445482 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9674009730 con 0x7f96841005f0 2023-09-10T00:38:53.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.264+0000 7f96817fa700 1 -- 172.21.15.44:0/711445482 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96740039c0 con 0x7f96841005f0 2023-09-10T00:38:53.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.264+0000 7f968a5b0700 1 -- 172.21.15.44:0/711445482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968419e870 con 0x7f96841005f0 2023-09-10T00:38:53.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.265+0000 7f968a5b0700 1 -- 172.21.15.44:0/711445482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968410bcc0 con 0x7f96841005f0 2023-09-10T00:38:53.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.267+0000 7f96817fa700 1 -- 172.21.15.44:0/711445482 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f96740098a0 con 0x7f96841005f0 2023-09-10T00:38:53.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.268+0000 7f96817fa700 1 --2- 172.21.15.44:0/711445482 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f966c0a0640 0x7f966c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:53.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.268+0000 7f96837fe700 1 --2- 172.21.15.44:0/711445482 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f966c0a0640 0x7f966c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:53.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.268+0000 7f96817fa700 1 -- 172.21.15.44:0/711445482 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f96740cbe50 con 0x7f96841005f0 2023-09-10T00:38:53.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.268+0000 7f96837fe700 1 --2- 172.21.15.44:0/711445482 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f966c0a0640 0x7f966c0a2b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f96780098e0 tx=0x7f9678000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:53.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.271+0000 7f96817fa700 1 -- 172.21.15.44:0/711445482 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9674095730 con 0x7f96841005f0 2023-09-10T00:38:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:52 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1825149603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:38:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:52 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2453021543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:38:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:52 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2891475886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:38:53.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.490+0000 7f968a5b0700 1 -- 172.21.15.44:0/711445482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f96841075a0 con 0x7f96841005f0 2023-09-10T00:38:53.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.490+0000 7f96817fa700 1 -- 172.21.15.44:0/711445482 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+584 (secure 0 0 0) 0x7f9674098d50 con 0x7f96841005f0 2023-09-10T00:38:53.491 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/18127588 2023-09-11T00:32:15.913770+0000 2023-09-10T00:38:53.491 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/18127588 2023-09-11T00:32:15.913770+0000 2023-09-10T00:38:53.491 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1414498756 2023-09-11T00:32:15.913770+0000 2023-09-10T00:38:53.492 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1298831914 2023-09-11T00:32:15.913770+0000 2023-09-10T00:38:53.492 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2415771009 2023-09-11T00:31:49.496807+0000 2023-09-10T00:38:53.492 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/193660691 2023-09-11T00:31:49.496807+0000 2023-09-10T00:38:53.492 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/871815312 2023-09-11T00:31:49.496807+0000 2023-09-10T00:38:53.492 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1481976351 2023-09-11T00:32:15.913770+0000 2023-09-10T00:38:53.493 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/871815312 2023-09-11T00:31:49.496807+0000 2023-09-10T00:38:53.493 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/625830807 2023-09-11T00:31:49.496807+0000 2023-09-10T00:38:53.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.493+0000 7f968a5b0700 1 -- 172.21.15.44:0/711445482 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f966c0a0640 msgr2=0x7f966c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:53.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.493+0000 7f968a5b0700 1 --2- 172.21.15.44:0/711445482 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f966c0a0640 0x7f966c0a2b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f96780098e0 tx=0x7f9678000bc0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.493+0000 7f968a5b0700 1 -- 172.21.15.44:0/711445482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96841005f0 msgr2=0x7f96841961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:53.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.493+0000 7f968a5b0700 1 --2- 172.21.15.44:0/711445482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96841005f0 0x7f96841961e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f967400d980 tx=0x7f967400dc90 comp rx=0 tx=0).stop 2023-09-10T00:38:53.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.493+0000 7f968a5b0700 1 -- 172.21.15.44:0/711445482 shutdown_connections 2023-09-10T00:38:53.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.493+0000 7f968a5b0700 1 --2- 172.21.15.44:0/711445482 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f966c0a0640 0x7f966c0a2b00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.493+0000 7f968a5b0700 1 --2- 172.21.15.44:0/711445482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9684102180 0x7f968419d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.493+0000 7f968a5b0700 1 --2- 172.21.15.44:0/711445482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96841005f0 0x7f96841961e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.493+0000 7f968a5b0700 1 --2- 172.21.15.44:0/711445482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96841017e0 0x7f9684196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.493+0000 7f968a5b0700 1 -- 172.21.15.44:0/711445482 >> 172.21.15.44:0/711445482 conn(0x7f96840fbb80 msgr2=0x7f96840fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:53.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.493+0000 7f968a5b0700 1 -- 172.21.15.44:0/711445482 shutdown_connections 2023-09-10T00:38:53.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.493+0000 7f968a5b0700 1 -- 172.21.15.44:0/711445482 wait complete. 2023-09-10T00:38:53.496 INFO:teuthology.orchestra.run.smithi044.stderr:listed 10 entries 2023-09-10T00:38:53.505 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:38:53.505 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-10T00:38:53.524 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:38:53.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.689+0000 7f97d2208700 1 -- 172.21.15.44:0/1697695231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97cc100620 msgr2=0x7f97cc100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:53.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.689+0000 7f97d2208700 1 --2- 172.21.15.44:0/1697695231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97cc100620 0x7f97cc100a40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0099c0 tx=0x7f97bc009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.689+0000 7f97d2208700 1 -- 172.21.15.44:0/1697695231 shutdown_connections 2023-09-10T00:38:53.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.689+0000 7f97d2208700 1 --2- 172.21.15.44:0/1697695231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc1021b0 0x7f97cc108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.689+0000 7f97d2208700 1 --2- 172.21.15.44:0/1697695231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97cc100620 0x7f97cc100a40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.689+0000 7f97d2208700 1 --2- 172.21.15.44:0/1697695231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97cc101810 0x7f97cc101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.689+0000 7f97d2208700 1 -- 172.21.15.44:0/1697695231 >> 172.21.15.44:0/1697695231 conn(0x7f97cc0fbbb0 msgr2=0x7f97cc0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:53.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.689+0000 7f97d2208700 1 -- 172.21.15.44:0/1697695231 shutdown_connections 2023-09-10T00:38:53.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.689+0000 7f97d2208700 1 -- 172.21.15.44:0/1697695231 wait complete. 2023-09-10T00:38:53.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.690+0000 7f97d2208700 1 Processor -- start 2023-09-10T00:38:53.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.690+0000 7f97d2208700 1 -- start start 2023-09-10T00:38:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.690+0000 7f97d2208700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97cc100620 0x7f97cc1063a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.690+0000 7f97d2208700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc101810 0x7f97cc1068e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.690+0000 7f97d2208700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97cc1021b0 0x7f97cc102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.690+0000 7f97d2208700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc1036d0 con 0x7f97cc100620 2023-09-10T00:38:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.690+0000 7f97d2208700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc103840 con 0x7f97cc101810 2023-09-10T00:38:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.690+0000 7f97d2208700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc103a00 con 0x7f97cc1021b0 2023-09-10T00:38:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.690+0000 7f97cb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97cc100620 0x7f97cc1063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.690+0000 7f97cb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97cc100620 0x7f97cc1063a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46570/0 (socket says 172.21.15.44:46570) 2023-09-10T00:38:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.690+0000 7f97cb7fe700 1 -- 172.21.15.44:0/1068406869 learned_addr learned my addr 172.21.15.44:0/1068406869 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.690+0000 7f97cbfff700 1 --2- 172.21.15.44:0/1068406869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97cc1021b0 0x7f97cc102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.691+0000 7f97cb7fe700 1 -- 172.21.15.44:0/1068406869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97cc1021b0 msgr2=0x7f97cc102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.691+0000 7f97cb7fe700 1 --2- 172.21.15.44:0/1068406869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97cc1021b0 0x7f97cc102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.691+0000 7f97cb7fe700 1 -- 172.21.15.44:0/1068406869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc101810 msgr2=0x7f97cc1068e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.691+0000 7f97caffd700 1 --2- 172.21.15.44:0/1068406869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc101810 0x7f97cc1068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.691+0000 7f97cb7fe700 1 --2- 172.21.15.44:0/1068406869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc101810 0x7f97cc1068e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.691+0000 7f97cb7fe700 1 -- 172.21.15.44:0/1068406869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc0095f0 con 0x7f97cc100620 2023-09-10T00:38:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.691+0000 7f97caffd700 1 --2- 172.21.15.44:0/1068406869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc101810 0x7f97cc1068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:38:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.691+0000 7f97cb7fe700 1 --2- 172.21.15.44:0/1068406869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97cc100620 0x7f97cc1063a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009fd0 tx=0x7f97bc00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.691+0000 7f97c8ff9700 1 -- 172.21.15.44:0/1068406869 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc01d070 con 0x7f97cc100620 2023-09-10T00:38:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.691+0000 7f97d2208700 1 -- 172.21.15.44:0/1068406869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97cc1a49a0 con 0x7f97cc100620 2023-09-10T00:38:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.691+0000 7f97c8ff9700 1 -- 172.21.15.44:0/1068406869 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f97bc0059a0 con 0x7f97cc100620 2023-09-10T00:38:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.691+0000 7f97c8ff9700 1 -- 172.21.15.44:0/1068406869 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc017650 con 0x7f97cc100620 2023-09-10T00:38:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.691+0000 7f97d2208700 1 -- 172.21.15.44:0/1068406869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97cc1a4e20 con 0x7f97cc100620 2023-09-10T00:38:53.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.692+0000 7f97d2208700 1 -- 172.21.15.44:0/1068406869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97cc10bcf0 con 0x7f97cc100620 2023-09-10T00:38:53.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.693+0000 7f97c8ff9700 1 -- 172.21.15.44:0/1068406869 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f97bc022470 con 0x7f97cc100620 2023-09-10T00:38:53.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.693+0000 7f97c8ff9700 1 --2- 172.21.15.44:0/1068406869 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f97b80a0690 0x7f97b80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:53.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.694+0000 7f97c8ff9700 1 -- 172.21.15.44:0/1068406869 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f97bc0cd300 con 0x7f97cc100620 2023-09-10T00:38:53.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.694+0000 7f97caffd700 1 --2- 172.21.15.44:0/1068406869 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f97b80a0690 0x7f97b80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:53.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.694+0000 7f97caffd700 1 --2- 172.21.15.44:0/1068406869 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f97b80a0690 0x7f97b80a2b50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009be0 tx=0x7f97b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:53.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.697+0000 7f97c8ff9700 1 -- 172.21.15.44:0/1068406869 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f97bc096cd0 con 0x7f97cc100620 2023-09-10T00:38:53.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.899+0000 7f97d2208700 1 -- 172.21.15.44:0/1068406869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97cc1a4fb0 con 0x7f97cc100620 2023-09-10T00:38:53.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.901+0000 7f97c8ff9700 1 -- 172.21.15.44:0/1068406869 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23870 (secure 0 0 0) 0x7f97bc09a2f0 con 0x7f97cc100620 2023-09-10T00:38:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:53.902 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":87,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:38:44.197902+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:38:53.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.903+0000 7f97d2208700 1 -- 172.21.15.44:0/1068406869 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f97b80a0690 msgr2=0x7f97b80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:53.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.903+0000 7f97d2208700 1 --2- 172.21.15.44:0/1068406869 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f97b80a0690 0x7f97b80a2b50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009be0 tx=0x7f97b4009260 comp rx=0 tx=0).stop 2023-09-10T00:38:53.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.903+0000 7f97d2208700 1 -- 172.21.15.44:0/1068406869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97cc100620 msgr2=0x7f97cc1063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:53.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.903+0000 7f97d2208700 1 --2- 172.21.15.44:0/1068406869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97cc100620 0x7f97cc1063a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009fd0 tx=0x7f97bc00fab0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.903+0000 7f97d2208700 1 -- 172.21.15.44:0/1068406869 shutdown_connections 2023-09-10T00:38:53.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.903+0000 7f97d2208700 1 --2- 172.21.15.44:0/1068406869 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f97b80a0690 0x7f97b80a2b50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.903+0000 7f97d2208700 1 --2- 172.21.15.44:0/1068406869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97cc1021b0 0x7f97cc102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.903+0000 7f97d2208700 1 --2- 172.21.15.44:0/1068406869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97cc100620 0x7f97cc1063a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.903+0000 7f97d2208700 1 --2- 172.21.15.44:0/1068406869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc101810 0x7f97cc1068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:53.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.903+0000 7f97d2208700 1 -- 172.21.15.44:0/1068406869 >> 172.21.15.44:0/1068406869 conn(0x7f97cc0fbbb0 msgr2=0x7f97cc0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:53.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.903+0000 7f97d2208700 1 -- 172.21.15.44:0/1068406869 shutdown_connections 2023-09-10T00:38:53.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:53.903+0000 7f97d2208700 1 -- 172.21.15.44:0/1068406869 wait complete. 2023-09-10T00:38:53.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-10T00:34:44.456741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-10T00:38:37.194891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-10T00:38:38.573268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-10T00:38:42.198896+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-10T00:38:53.916 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-10T00:38:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.045+0000 7f587168f700 1 -- 172.21.15.44:0/1214581234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f586c0ff4f0 msgr2=0x7f586c0ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.045+0000 7f587168f700 1 --2- 172.21.15.44:0/1214581234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f586c0ff4f0 0x7f586c0ff910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f585c0099c0 tx=0x7f585c009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.045+0000 7f587168f700 1 -- 172.21.15.44:0/1214581234 shutdown_connections 2023-09-10T00:38:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.045+0000 7f587168f700 1 --2- 172.21.15.44:0/1214581234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586c101080 0x7f586c107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.045+0000 7f587168f700 1 --2- 172.21.15.44:0/1214581234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f586c0ff4f0 0x7f586c0ff910 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.045+0000 7f587168f700 1 --2- 172.21.15.44:0/1214581234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f586c1006e0 0x7f586c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.045+0000 7f587168f700 1 -- 172.21.15.44:0/1214581234 >> 172.21.15.44:0/1214581234 conn(0x7f586c0faa60 msgr2=0x7f586c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.045+0000 7f587168f700 1 -- 172.21.15.44:0/1214581234 shutdown_connections 2023-09-10T00:38:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.045+0000 7f587168f700 1 -- 172.21.15.44:0/1214581234 wait complete. 2023-09-10T00:38:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f587168f700 1 Processor -- start 2023-09-10T00:38:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f587168f700 1 -- start start 2023-09-10T00:38:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f587168f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f586c0ff4f0 0x7f586c196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f587168f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f586c1006e0 0x7f586c196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f587168f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586c101080 0x7f586c19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f587168f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c19dda0 con 0x7f586c0ff4f0 2023-09-10T00:38:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f587168f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c19dee0 con 0x7f586c1006e0 2023-09-10T00:38:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f587168f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c19e070 con 0x7f586c101080 2023-09-10T00:38:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f586affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f586c0ff4f0 0x7f586c196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f586affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f586c0ff4f0 0x7f586c196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46584/0 (socket says 172.21.15.44:46584) 2023-09-10T00:38:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f586affd700 1 -- 172.21.15.44:0/1869565844 learned_addr learned my addr 172.21.15.44:0/1869565844 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f586a7fc700 1 --2- 172.21.15.44:0/1869565844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f586c1006e0 0x7f586c196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f586b7fe700 1 --2- 172.21.15.44:0/1869565844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586c101080 0x7f586c19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f586affd700 1 -- 172.21.15.44:0/1869565844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586c101080 msgr2=0x7f586c19d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f586affd700 1 --2- 172.21.15.44:0/1869565844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586c101080 0x7f586c19d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f586affd700 1 -- 172.21.15.44:0/1869565844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f586c1006e0 msgr2=0x7f586c196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:54.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f586affd700 1 --2- 172.21.15.44:0/1869565844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f586c1006e0 0x7f586c196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.046+0000 7f586affd700 1 -- 172.21.15.44:0/1869565844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c0095f0 con 0x7f586c0ff4f0 2023-09-10T00:38:54.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.047+0000 7f586b7fe700 1 --2- 172.21.15.44:0/1869565844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586c101080 0x7f586c19d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:38:54.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.047+0000 7f586affd700 1 --2- 172.21.15.44:0/1869565844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f586c0ff4f0 0x7f586c196220 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f585c009f50 tx=0x7f585c003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:54.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.047+0000 7f5853fff700 1 -- 172.21.15.44:0/1869565844 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c01d070 con 0x7f586c0ff4f0 2023-09-10T00:38:54.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.047+0000 7f587168f700 1 -- 172.21.15.44:0/1869565844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586c19e2f0 con 0x7f586c0ff4f0 2023-09-10T00:38:54.054 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.047+0000 7f5853fff700 1 -- 172.21.15.44:0/1869565844 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f585c00bda0 con 0x7f586c0ff4f0 2023-09-10T00:38:54.054 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.047+0000 7f5853fff700 1 -- 172.21.15.44:0/1869565844 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c022d00 con 0x7f586c0ff4f0 2023-09-10T00:38:54.054 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.047+0000 7f587168f700 1 -- 172.21.15.44:0/1869565844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586c19e7a0 con 0x7f586c0ff4f0 2023-09-10T00:38:54.054 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.048+0000 7f587168f700 1 -- 172.21.15.44:0/1869565844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586c10abb0 con 0x7f586c0ff4f0 2023-09-10T00:38:54.054 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.049+0000 7f5853fff700 1 -- 172.21.15.44:0/1869565844 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f585c003ff0 con 0x7f586c0ff4f0 2023-09-10T00:38:54.055 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.049+0000 7f5853fff700 1 --2- 172.21.15.44:0/1869565844 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f58540a0640 0x7f58540a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:54.055 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.050+0000 7f586a7fc700 1 --2- 172.21.15.44:0/1869565844 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f58540a0640 0x7f58540a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:54.055 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.050+0000 7f5853fff700 1 -- 172.21.15.44:0/1869565844 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f585c0cd130 con 0x7f586c0ff4f0 2023-09-10T00:38:54.055 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.050+0000 7f586a7fc700 1 --2- 172.21.15.44:0/1869565844 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f58540a0640 0x7f58540a2b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5860005950 tx=0x7f58600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:54.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.054+0000 7f5853fff700 1 -- 172.21.15.44:0/1869565844 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f585c096bc0 con 0x7f586c0ff4f0 2023-09-10T00:38:54.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:53 smithi044 ceph-mon[98676]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 622 B/s wr, 1 op/s 2023-09-10T00:38:54.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:53 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1819455754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:38:54.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:53 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/711445482' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-10T00:38:54.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:53 smithi099 ceph-mon[102692]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 622 B/s wr, 1 op/s 2023-09-10T00:38:54.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:53 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1819455754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:38:54.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:53 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/711445482' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-10T00:38:54.253 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.253+0000 7f587168f700 1 -- 172.21.15.44:0/1869565844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f586c19e9c0 con 0x7f586c0ff4f0 2023-09-10T00:38:54.255 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.254+0000 7f5853fff700 1 -- 172.21.15.44:0/1869565844 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f585c022e60 con 0x7f586c0ff4f0 2023-09-10T00:38:54.255 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2023-09-10T00:38:54.256 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.256+0000 7f587168f700 1 -- 172.21.15.44:0/1869565844 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f58540a0640 msgr2=0x7f58540a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:54.256 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.256+0000 7f587168f700 1 --2- 172.21.15.44:0/1869565844 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f58540a0640 0x7f58540a2b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5860005950 tx=0x7f58600058e0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.256+0000 7f587168f700 1 -- 172.21.15.44:0/1869565844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f586c0ff4f0 msgr2=0x7f586c196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.256+0000 7f587168f700 1 --2- 172.21.15.44:0/1869565844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f586c0ff4f0 0x7f586c196220 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f585c009f50 tx=0x7f585c003b00 comp rx=0 tx=0).stop 2023-09-10T00:38:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.256+0000 7f587168f700 1 -- 172.21.15.44:0/1869565844 shutdown_connections 2023-09-10T00:38:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.256+0000 7f587168f700 1 --2- 172.21.15.44:0/1869565844 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f58540a0640 0x7f58540a2b00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.256+0000 7f587168f700 1 --2- 172.21.15.44:0/1869565844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586c101080 0x7f586c19d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.258 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.256+0000 7f587168f700 1 --2- 172.21.15.44:0/1869565844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f586c0ff4f0 0x7f586c196220 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.258 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.256+0000 7f587168f700 1 --2- 172.21.15.44:0/1869565844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f586c1006e0 0x7f586c196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.258 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.256+0000 7f587168f700 1 -- 172.21.15.44:0/1869565844 >> 172.21.15.44:0/1869565844 conn(0x7f586c0faa60 msgr2=0x7f586c067820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:54.258 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.256+0000 7f587168f700 1 -- 172.21.15.44:0/1869565844 shutdown_connections 2023-09-10T00:38:54.258 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.256+0000 7f587168f700 1 -- 172.21.15.44:0/1869565844 wait complete. 2023-09-10T00:38:54.268 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-10T00:38:54.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:53 smithi131 ceph-mon[103713]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 622 B/s wr, 1 op/s 2023-09-10T00:38:54.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:53 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1819455754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:38:54.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:53 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/711445482' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-10T00:38:54.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.397+0000 7f3bf23f6700 1 -- 172.21.15.44:0/1501608644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bec101080 msgr2=0x7f3bec107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:54.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.397+0000 7f3bf23f6700 1 --2- 172.21.15.44:0/1501608644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bec101080 0x7f3bec107930 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3be400b0d0 tx=0x7f3be400b3e0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.397+0000 7f3bf23f6700 1 -- 172.21.15.44:0/1501608644 shutdown_connections 2023-09-10T00:38:54.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.397+0000 7f3bf23f6700 1 --2- 172.21.15.44:0/1501608644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bec101080 0x7f3bec107930 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.397+0000 7f3bf23f6700 1 --2- 172.21.15.44:0/1501608644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bec0ff4f0 0x7f3bec0ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.397+0000 7f3bf23f6700 1 --2- 172.21.15.44:0/1501608644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bec1006e0 0x7f3bec100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.397+0000 7f3bf23f6700 1 -- 172.21.15.44:0/1501608644 >> 172.21.15.44:0/1501608644 conn(0x7f3bec0faa60 msgr2=0x7f3bec0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.397+0000 7f3bf23f6700 1 -- 172.21.15.44:0/1501608644 shutdown_connections 2023-09-10T00:38:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.397+0000 7f3bf23f6700 1 -- 172.21.15.44:0/1501608644 wait complete. 2023-09-10T00:38:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.397+0000 7f3bf23f6700 1 Processor -- start 2023-09-10T00:38:54.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.397+0000 7f3bf23f6700 1 -- start start 2023-09-10T00:38:54.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3bf23f6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bec0ff4f0 0x7f3bec1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:54.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3bf23f6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bec1006e0 0x7f3bec198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:54.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3bf23f6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bec101080 0x7f3bec19faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3bf23f6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec1a0070 con 0x7f3bec1006e0 2023-09-10T00:38:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3bf23f6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec1a01b0 con 0x7f3bec101080 2023-09-10T00:38:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3bf23f6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec1a0340 con 0x7f3bec0ff4f0 2023-09-10T00:38:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3beb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bec1006e0 0x7f3bec198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3beb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bec1006e0 0x7f3bec198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46598/0 (socket says 172.21.15.44:46598) 2023-09-10T00:38:54.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3beb7fe700 1 -- 172.21.15.44:0/1560866319 learned_addr learned my addr 172.21.15.44:0/1560866319 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:54.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3bf0993700 1 --2- 172.21.15.44:0/1560866319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bec101080 0x7f3bec19faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:54.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3bebfff700 1 --2- 172.21.15.44:0/1560866319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bec0ff4f0 0x7f3bec1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:54.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3beb7fe700 1 -- 172.21.15.44:0/1560866319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bec0ff4f0 msgr2=0x7f3bec1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:54.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3beb7fe700 1 --2- 172.21.15.44:0/1560866319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bec0ff4f0 0x7f3bec1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3beb7fe700 1 -- 172.21.15.44:0/1560866319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bec101080 msgr2=0x7f3bec19faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3beb7fe700 1 --2- 172.21.15.44:0/1560866319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bec101080 0x7f3bec19faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3beb7fe700 1 -- 172.21.15.44:0/1560866319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be4009d00 con 0x7f3bec1006e0 2023-09-10T00:38:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3bebfff700 1 --2- 172.21.15.44:0/1560866319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bec0ff4f0 0x7f3bec1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:38:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.398+0000 7f3beb7fe700 1 --2- 172.21.15.44:0/1560866319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bec1006e0 0x7f3bec198a30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3be000d950 tx=0x7f3be000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.399+0000 7f3be97fa700 1 -- 172.21.15.44:0/1560866319 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0009940 con 0x7f3bec1006e0 2023-09-10T00:38:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.399+0000 7f3be97fa700 1 -- 172.21.15.44:0/1560866319 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3be0010460 con 0x7f3bec1006e0 2023-09-10T00:38:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.399+0000 7f3bf23f6700 1 -- 172.21.15.44:0/1560866319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bec1a0620 con 0x7f3bec1006e0 2023-09-10T00:38:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.399+0000 7f3be97fa700 1 -- 172.21.15.44:0/1560866319 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be000f620 con 0x7f3bec1006e0 2023-09-10T00:38:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.399+0000 7f3bf23f6700 1 -- 172.21.15.44:0/1560866319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bec1a0b80 con 0x7f3bec1006e0 2023-09-10T00:38:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.400+0000 7f3bf23f6700 1 -- 172.21.15.44:0/1560866319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bec10abb0 con 0x7f3bec1006e0 2023-09-10T00:38:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.401+0000 7f3be97fa700 1 -- 172.21.15.44:0/1560866319 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f3be00105d0 con 0x7f3bec1006e0 2023-09-10T00:38:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.401+0000 7f3be97fa700 1 --2- 172.21.15.44:0/1560866319 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3bd40a06c0 0x7f3bd40a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.401+0000 7f3be97fa700 1 -- 172.21.15.44:0/1560866319 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3be00cbb20 con 0x7f3bec1006e0 2023-09-10T00:38:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.401+0000 7f3bebfff700 1 --2- 172.21.15.44:0/1560866319 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3bd40a06c0 0x7f3bd40a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:54.408 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.402+0000 7f3bebfff700 1 --2- 172.21.15.44:0/1560866319 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3bd40a06c0 0x7f3bd40a2b80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3bec100540 tx=0x7f3bdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:54.408 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.405+0000 7f3be97fa700 1 -- 172.21.15.44:0/1560866319 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3be0095460 con 0x7f3bec1006e0 2023-09-10T00:38:54.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.606+0000 7f3bf23f6700 1 -- 172.21.15.44:0/1560866319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3bec1a0da0 con 0x7f3bec1006e0 2023-09-10T00:38:54.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.606+0000 7f3be97fa700 1 -- 172.21.15.44:0/1560866319 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3be0020020 con 0x7f3bec1006e0 2023-09-10T00:38:54.607 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:38:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.609+0000 7f3bf23f6700 1 -- 172.21.15.44:0/1560866319 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3bd40a06c0 msgr2=0x7f3bd40a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.609+0000 7f3bf23f6700 1 --2- 172.21.15.44:0/1560866319 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3bd40a06c0 0x7f3bd40a2b80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3bec100540 tx=0x7f3bdc009260 comp rx=0 tx=0).stop 2023-09-10T00:38:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.609+0000 7f3bf23f6700 1 -- 172.21.15.44:0/1560866319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bec1006e0 msgr2=0x7f3bec198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:54.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.609+0000 7f3bf23f6700 1 --2- 172.21.15.44:0/1560866319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bec1006e0 0x7f3bec198a30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3be000d950 tx=0x7f3be000dc60 comp rx=0 tx=0).stop 2023-09-10T00:38:54.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.609+0000 7f3bf23f6700 1 -- 172.21.15.44:0/1560866319 shutdown_connections 2023-09-10T00:38:54.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.609+0000 7f3bf23f6700 1 --2- 172.21.15.44:0/1560866319 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3bd40a06c0 0x7f3bd40a2b80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.609+0000 7f3bf23f6700 1 --2- 172.21.15.44:0/1560866319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bec101080 0x7f3bec19faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.609+0000 7f3bf23f6700 1 --2- 172.21.15.44:0/1560866319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bec0ff4f0 0x7f3bec1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.609+0000 7f3bf23f6700 1 --2- 172.21.15.44:0/1560866319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bec1006e0 0x7f3bec198a30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.609+0000 7f3bf23f6700 1 -- 172.21.15.44:0/1560866319 >> 172.21.15.44:0/1560866319 conn(0x7f3bec0faa60 msgr2=0x7f3bec067740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:54.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.609+0000 7f3bf23f6700 1 -- 172.21.15.44:0/1560866319 shutdown_connections 2023-09-10T00:38:54.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.609+0000 7f3bf23f6700 1 -- 172.21.15.44:0/1560866319 wait complete. 2023-09-10T00:38:54.621 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-10T00:38:54.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.749+0000 7f6a95e1f700 1 -- 172.21.15.44:0/3277073409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a90101800 msgr2=0x7f6a90101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.749+0000 7f6a95e1f700 1 --2- 172.21.15.44:0/3277073409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a90101800 0x7f6a90101c60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6a780099c0 tx=0x7f6a78009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.749+0000 7f6a95e1f700 1 -- 172.21.15.44:0/3277073409 shutdown_connections 2023-09-10T00:38:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.749+0000 7f6a95e1f700 1 --2- 172.21.15.44:0/3277073409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a901021a0 0x7f6a90108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.749+0000 7f6a95e1f700 1 --2- 172.21.15.44:0/3277073409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a90100610 0x7f6a90100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.749+0000 7f6a95e1f700 1 --2- 172.21.15.44:0/3277073409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a90101800 0x7f6a90101c60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.749+0000 7f6a95e1f700 1 -- 172.21.15.44:0/3277073409 >> 172.21.15.44:0/3277073409 conn(0x7f6a900fbb80 msgr2=0x7f6a900fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.749+0000 7f6a95e1f700 1 -- 172.21.15.44:0/3277073409 shutdown_connections 2023-09-10T00:38:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.749+0000 7f6a95e1f700 1 -- 172.21.15.44:0/3277073409 wait complete. 2023-09-10T00:38:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.750+0000 7f6a95e1f700 1 Processor -- start 2023-09-10T00:38:54.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.750+0000 7f6a95e1f700 1 -- start start 2023-09-10T00:38:54.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.750+0000 7f6a95e1f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a90100610 0x7f6a90196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:54.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.750+0000 7f6a95e1f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a90101800 0x7f6a90196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:54.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.750+0000 7f6a95e1f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a901021a0 0x7f6a9019d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:54.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.750+0000 7f6a95e1f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9019dd70 con 0x7f6a90101800 2023-09-10T00:38:54.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.750+0000 7f6a95e1f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9019deb0 con 0x7f6a901021a0 2023-09-10T00:38:54.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.750+0000 7f6a95e1f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9019e040 con 0x7f6a90100610 2023-09-10T00:38:54.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.750+0000 7f6a8effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a90101800 0x7f6a90196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:54.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.750+0000 7f6a8effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a90101800 0x7f6a90196780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46614/0 (socket says 172.21.15.44:46614) 2023-09-10T00:38:54.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.750+0000 7f6a8effd700 1 -- 172.21.15.44:0/4240283231 learned_addr learned my addr 172.21.15.44:0/4240283231 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:54.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.750+0000 7f6a8f7fe700 1 --2- 172.21.15.44:0/4240283231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a90100610 0x7f6a90196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:54.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.751+0000 7f6a8ffff700 1 --2- 172.21.15.44:0/4240283231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a901021a0 0x7f6a9019d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:54.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.751+0000 7f6a8effd700 1 -- 172.21.15.44:0/4240283231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a90100610 msgr2=0x7f6a90196240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:54.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.751+0000 7f6a8effd700 1 --2- 172.21.15.44:0/4240283231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a90100610 0x7f6a90196240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.751+0000 7f6a8effd700 1 -- 172.21.15.44:0/4240283231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a901021a0 msgr2=0x7f6a9019d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:54.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.751+0000 7f6a8effd700 1 --2- 172.21.15.44:0/4240283231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a901021a0 0x7f6a9019d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.751+0000 7f6a8effd700 1 -- 172.21.15.44:0/4240283231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a780095f0 con 0x7f6a90101800 2023-09-10T00:38:54.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.751+0000 7f6a8effd700 1 --2- 172.21.15.44:0/4240283231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a90101800 0x7f6a90196780 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009fd0 tx=0x7f6a7800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:54.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.751+0000 7f6a8cff9700 1 -- 172.21.15.44:0/4240283231 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7801d070 con 0x7f6a90101800 2023-09-10T00:38:54.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.751+0000 7f6a8cff9700 1 -- 172.21.15.44:0/4240283231 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a780059a0 con 0x7f6a90101800 2023-09-10T00:38:54.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.751+0000 7f6a95e1f700 1 -- 172.21.15.44:0/4240283231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9019e2c0 con 0x7f6a90101800 2023-09-10T00:38:54.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.751+0000 7f6a8cff9700 1 -- 172.21.15.44:0/4240283231 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a780173f0 con 0x7f6a90101800 2023-09-10T00:38:54.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.751+0000 7f6a95e1f700 1 -- 172.21.15.44:0/4240283231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9019e770 con 0x7f6a90101800 2023-09-10T00:38:54.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.752+0000 7f6a95e1f700 1 -- 172.21.15.44:0/4240283231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a90107cd0 con 0x7f6a90101800 2023-09-10T00:38:54.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.753+0000 7f6a8cff9700 1 -- 172.21.15.44:0/4240283231 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f6a78022470 con 0x7f6a90101800 2023-09-10T00:38:54.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.754+0000 7f6a8cff9700 1 --2- 172.21.15.44:0/4240283231 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6a7c0a06c0 0x7f6a7c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:54.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.754+0000 7f6a8f7fe700 1 --2- 172.21.15.44:0/4240283231 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6a7c0a06c0 0x7f6a7c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:54.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.754+0000 7f6a8cff9700 1 -- 172.21.15.44:0/4240283231 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6a780cce40 con 0x7f6a90101800 2023-09-10T00:38:54.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.754+0000 7f6a8f7fe700 1 --2- 172.21.15.44:0/4240283231 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6a7c0a06c0 0x7f6a7c0a2b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6a90101660 tx=0x7f6a80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:54.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.758+0000 7f6a8cff9700 1 -- 172.21.15.44:0/4240283231 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6a780967b0 con 0x7f6a90101800 2023-09-10T00:38:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.965+0000 7f6a95e1f700 1 -- 172.21.15.44:0/4240283231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6a9002cfc0 con 0x7f6a90101800 2023-09-10T00:38:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.965+0000 7f6a8cff9700 1 -- 172.21.15.44:0/4240283231 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a78099dd0 con 0x7f6a90101800 2023-09-10T00:38:54.966 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2023-09-10T00:38:54.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.967+0000 7f6a95e1f700 1 -- 172.21.15.44:0/4240283231 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6a7c0a06c0 msgr2=0x7f6a7c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:54.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.967+0000 7f6a95e1f700 1 --2- 172.21.15.44:0/4240283231 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6a7c0a06c0 0x7f6a7c0a2b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6a90101660 tx=0x7f6a80009260 comp rx=0 tx=0).stop 2023-09-10T00:38:54.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.967+0000 7f6a95e1f700 1 -- 172.21.15.44:0/4240283231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a90101800 msgr2=0x7f6a90196780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:54.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.967+0000 7f6a95e1f700 1 --2- 172.21.15.44:0/4240283231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a90101800 0x7f6a90196780 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009fd0 tx=0x7f6a7800fa00 comp rx=0 tx=0).stop 2023-09-10T00:38:54.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.967+0000 7f6a95e1f700 1 -- 172.21.15.44:0/4240283231 shutdown_connections 2023-09-10T00:38:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.967+0000 7f6a95e1f700 1 --2- 172.21.15.44:0/4240283231 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6a7c0a06c0 0x7f6a7c0a2b80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.967+0000 7f6a95e1f700 1 --2- 172.21.15.44:0/4240283231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a901021a0 0x7f6a9019d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.967+0000 7f6a95e1f700 1 --2- 172.21.15.44:0/4240283231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a90100610 0x7f6a90196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.967+0000 7f6a95e1f700 1 --2- 172.21.15.44:0/4240283231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a90101800 0x7f6a90196780 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:54.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.967+0000 7f6a95e1f700 1 -- 172.21.15.44:0/4240283231 >> 172.21.15.44:0/4240283231 conn(0x7f6a900fbb80 msgr2=0x7f6a90104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.967+0000 7f6a95e1f700 1 -- 172.21.15.44:0/4240283231 shutdown_connections 2023-09-10T00:38:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:54.968+0000 7f6a95e1f700 1 -- 172.21.15.44:0/4240283231 wait complete. 2023-09-10T00:38:54.979 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:54 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1068406869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:54.979 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:54 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1869565844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:38:54.979 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:54 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1560866319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:38:54.980 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-10T00:38:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:54 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1068406869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:54 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1869565844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:38:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:54 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1560866319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:38:55.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.110+0000 7f687ce33700 1 -- 172.21.15.44:0/99034253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68780ff4f0 msgr2=0x7f68780ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.110+0000 7f687ce33700 1 --2- 172.21.15.44:0/99034253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68780ff4f0 0x7f68780ff910 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f68680099c0 tx=0x7f6868009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.110+0000 7f687ce33700 1 -- 172.21.15.44:0/99034253 shutdown_connections 2023-09-10T00:38:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.110+0000 7f687ce33700 1 --2- 172.21.15.44:0/99034253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6878101080 0x7f6878107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.110+0000 7f687ce33700 1 --2- 172.21.15.44:0/99034253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68780ff4f0 0x7f68780ff910 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.110+0000 7f687ce33700 1 --2- 172.21.15.44:0/99034253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68781006e0 0x7f6878100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.110+0000 7f687ce33700 1 -- 172.21.15.44:0/99034253 >> 172.21.15.44:0/99034253 conn(0x7f68780faa60 msgr2=0x7f68780fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:55.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.110+0000 7f687ce33700 1 -- 172.21.15.44:0/99034253 shutdown_connections 2023-09-10T00:38:55.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.110+0000 7f687ce33700 1 -- 172.21.15.44:0/99034253 wait complete. 2023-09-10T00:38:55.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.111+0000 7f687ce33700 1 Processor -- start 2023-09-10T00:38:55.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.111+0000 7f687ce33700 1 -- start start 2023-09-10T00:38:55.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.111+0000 7f687ce33700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68780ff4f0 0x7f6878103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.111+0000 7f687ce33700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68781006e0 0x7f6878101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.111+0000 7f687ce33700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6878101080 0x7f6878102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.111+0000 7f687ce33700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68781027a0 con 0x7f6878101080 2023-09-10T00:38:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.111+0000 7f687ce33700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6878102910 con 0x7f68781006e0 2023-09-10T00:38:55.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.111+0000 7f687ce33700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68781a02c0 con 0x7f68780ff4f0 2023-09-10T00:38:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.111+0000 7f6876d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6878101080 0x7f6878102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.111+0000 7f6876d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6878101080 0x7f6878102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46634/0 (socket says 172.21.15.44:46634) 2023-09-10T00:38:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.111+0000 7f6876d9d700 1 -- 172.21.15.44:0/3916022868 learned_addr learned my addr 172.21.15.44:0/3916022868 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.111+0000 7f687659c700 1 --2- 172.21.15.44:0/3916022868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68780ff4f0 0x7f6878103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.111+0000 7f6875d9b700 1 --2- 172.21.15.44:0/3916022868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68781006e0 0x7f6878101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.111+0000 7f6876d9d700 1 -- 172.21.15.44:0/3916022868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68780ff4f0 msgr2=0x7f6878103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.111+0000 7f6876d9d700 1 --2- 172.21.15.44:0/3916022868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68780ff4f0 0x7f6878103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.111+0000 7f6876d9d700 1 -- 172.21.15.44:0/3916022868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68781006e0 msgr2=0x7f6878101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.112+0000 7f6876d9d700 1 --2- 172.21.15.44:0/3916022868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68781006e0 0x7f6878101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.112+0000 7f6876d9d700 1 -- 172.21.15.44:0/3916022868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68680095f0 con 0x7f6878101080 2023-09-10T00:38:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.112+0000 7f6876d9d700 1 --2- 172.21.15.44:0/3916022868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6878101080 0x7f6878102000 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6870007ee0 tx=0x7f687000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:55.118 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.112+0000 7f68677fe700 1 -- 172.21.15.44:0/3916022868 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6870017070 con 0x7f6878101080 2023-09-10T00:38:55.118 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.112+0000 7f687ce33700 1 -- 172.21.15.44:0/3916022868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68781a05a0 con 0x7f6878101080 2023-09-10T00:38:55.118 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.112+0000 7f68677fe700 1 -- 172.21.15.44:0/3916022868 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f687000f040 con 0x7f6878101080 2023-09-10T00:38:55.118 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.112+0000 7f68677fe700 1 -- 172.21.15.44:0/3916022868 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6870005570 con 0x7f6878101080 2023-09-10T00:38:55.118 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.112+0000 7f687ce33700 1 -- 172.21.15.44:0/3916022868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68781a0ab0 con 0x7f6878101080 2023-09-10T00:38:55.119 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.113+0000 7f687ce33700 1 -- 172.21.15.44:0/3916022868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f687810ac10 con 0x7f6878101080 2023-09-10T00:38:55.119 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.114+0000 7f68677fe700 1 -- 172.21.15.44:0/3916022868 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f687000db20 con 0x7f6878101080 2023-09-10T00:38:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.114+0000 7f68677fe700 1 --2- 172.21.15.44:0/3916022868 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f68600a0640 0x7f68600a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.115+0000 7f687659c700 1 --2- 172.21.15.44:0/3916022868 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f68600a0640 0x7f68600a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.115+0000 7f68677fe700 1 -- 172.21.15.44:0/3916022868 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f68700cb530 con 0x7f6878101080 2023-09-10T00:38:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.115+0000 7f687659c700 1 --2- 172.21.15.44:0/3916022868 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f68600a0640 0x7f68600a2b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6868000c00 tx=0x7f6868009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.119+0000 7f68677fe700 1 -- 172.21.15.44:0/3916022868 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6870094e70 con 0x7f6878101080 2023-09-10T00:38:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:54 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1068406869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:54 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1869565844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:38:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:54 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1560866319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:38:55.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.325+0000 7f687ce33700 1 -- 172.21.15.44:0/3916022868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f68781a0d90 con 0x7f6878101080 2023-09-10T00:38:55.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.325+0000 7f68677fe700 1 -- 172.21.15.44:0/3916022868 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6870098490 con 0x7f6878101080 2023-09-10T00:38:55.325 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:38:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.327+0000 7f687ce33700 1 -- 172.21.15.44:0/3916022868 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f68600a0640 msgr2=0x7f68600a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.327+0000 7f687ce33700 1 --2- 172.21.15.44:0/3916022868 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f68600a0640 0x7f68600a2b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6868000c00 tx=0x7f6868009f10 comp rx=0 tx=0).stop 2023-09-10T00:38:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.327+0000 7f687ce33700 1 -- 172.21.15.44:0/3916022868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6878101080 msgr2=0x7f6878102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.327+0000 7f687ce33700 1 --2- 172.21.15.44:0/3916022868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6878101080 0x7f6878102000 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6870007ee0 tx=0x7f687000d3b0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.327+0000 7f687ce33700 1 -- 172.21.15.44:0/3916022868 shutdown_connections 2023-09-10T00:38:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.327+0000 7f687ce33700 1 --2- 172.21.15.44:0/3916022868 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f68600a0640 0x7f68600a2b00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.327+0000 7f687ce33700 1 --2- 172.21.15.44:0/3916022868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6878101080 0x7f6878102000 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.327+0000 7f687ce33700 1 --2- 172.21.15.44:0/3916022868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68780ff4f0 0x7f6878103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.327+0000 7f687ce33700 1 --2- 172.21.15.44:0/3916022868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68781006e0 0x7f6878101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.327+0000 7f687ce33700 1 -- 172.21.15.44:0/3916022868 >> 172.21.15.44:0/3916022868 conn(0x7f68780faa60 msgr2=0x7f6878067760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.327+0000 7f687ce33700 1 -- 172.21.15.44:0/3916022868 shutdown_connections 2023-09-10T00:38:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.327+0000 7f687ce33700 1 -- 172.21.15.44:0/3916022868 wait complete. 2023-09-10T00:38:55.340 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-10T00:38:55.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.466+0000 7f0c789e4700 1 -- 172.21.15.44:0/3982605697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c74101080 msgr2=0x7f0c74107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:55.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.466+0000 7f0c789e4700 1 --2- 172.21.15.44:0/3982605697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c74101080 0x7f0c74107930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00b0d0 tx=0x7f0c6c00b3e0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.466+0000 7f0c789e4700 1 -- 172.21.15.44:0/3982605697 shutdown_connections 2023-09-10T00:38:55.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.466+0000 7f0c789e4700 1 --2- 172.21.15.44:0/3982605697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c74101080 0x7f0c74107930 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.466+0000 7f0c789e4700 1 --2- 172.21.15.44:0/3982605697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c740ff4f0 0x7f0c740ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.466+0000 7f0c789e4700 1 --2- 172.21.15.44:0/3982605697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c741006e0 0x7f0c74100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.466+0000 7f0c789e4700 1 -- 172.21.15.44:0/3982605697 >> 172.21.15.44:0/3982605697 conn(0x7f0c740faa60 msgr2=0x7f0c740fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:55.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.467+0000 7f0c789e4700 1 -- 172.21.15.44:0/3982605697 shutdown_connections 2023-09-10T00:38:55.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.467+0000 7f0c789e4700 1 -- 172.21.15.44:0/3982605697 wait complete. 2023-09-10T00:38:55.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.467+0000 7f0c789e4700 1 Processor -- start 2023-09-10T00:38:55.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.467+0000 7f0c789e4700 1 -- start start 2023-09-10T00:38:55.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.467+0000 7f0c789e4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c740ff4f0 0x7f0c74196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:55.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.467+0000 7f0c789e4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c741006e0 0x7f0c74196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:55.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.467+0000 7f0c789e4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c74101080 0x7f0c7419d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:55.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.467+0000 7f0c789e4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7419ded0 con 0x7f0c740ff4f0 2023-09-10T00:38:55.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.467+0000 7f0c789e4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7419e010 con 0x7f0c741006e0 2023-09-10T00:38:55.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.467+0000 7f0c789e4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7419e1a0 con 0x7f0c74101080 2023-09-10T00:38:55.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c7259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c740ff4f0 0x7f0c74196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:55.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c7259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c740ff4f0 0x7f0c74196350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46644/0 (socket says 172.21.15.44:46644) 2023-09-10T00:38:55.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c7259c700 1 -- 172.21.15.44:0/1354028771 learned_addr learned my addr 172.21.15.44:0/1354028771 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:55.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c72d9d700 1 --2- 172.21.15.44:0/1354028771 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c74101080 0x7f0c7419d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:55.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c71d9b700 1 --2- 172.21.15.44:0/1354028771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c741006e0 0x7f0c74196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:55.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c7259c700 1 -- 172.21.15.44:0/1354028771 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c74101080 msgr2=0x7f0c7419d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:55.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c7259c700 1 --2- 172.21.15.44:0/1354028771 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c74101080 0x7f0c7419d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c7259c700 1 -- 172.21.15.44:0/1354028771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c741006e0 msgr2=0x7f0c74196890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:55.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c7259c700 1 --2- 172.21.15.44:0/1354028771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c741006e0 0x7f0c74196890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c7259c700 1 -- 172.21.15.44:0/1354028771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c009d00 con 0x7f0c740ff4f0 2023-09-10T00:38:55.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c71d9b700 1 --2- 172.21.15.44:0/1354028771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c741006e0 0x7f0c74196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:38:55.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c7259c700 1 --2- 172.21.15.44:0/1354028771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c740ff4f0 0x7f0c74196350 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400c970 tx=0x7f0c6400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:55.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c637fe700 1 -- 172.21.15.44:0/1354028771 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64004220 con 0x7f0c740ff4f0 2023-09-10T00:38:55.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c637fe700 1 -- 172.21.15.44:0/1354028771 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c64007d30 con 0x7f0c740ff4f0 2023-09-10T00:38:55.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c637fe700 1 -- 172.21.15.44:0/1354028771 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64003b80 con 0x7f0c740ff4f0 2023-09-10T00:38:55.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c789e4700 1 -- 172.21.15.44:0/1354028771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7419e480 con 0x7f0c740ff4f0 2023-09-10T00:38:55.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.468+0000 7f0c789e4700 1 -- 172.21.15.44:0/1354028771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7419e940 con 0x7f0c740ff4f0 2023-09-10T00:38:55.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.469+0000 7f0c789e4700 1 -- 172.21.15.44:0/1354028771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7410ac10 con 0x7f0c740ff4f0 2023-09-10T00:38:55.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.471+0000 7f0c637fe700 1 -- 172.21.15.44:0/1354028771 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f0c64003ce0 con 0x7f0c740ff4f0 2023-09-10T00:38:55.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.471+0000 7f0c637fe700 1 --2- 172.21.15.44:0/1354028771 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0c5c0a0640 0x7f0c5c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:55.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.471+0000 7f0c637fe700 1 -- 172.21.15.44:0/1354028771 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0c640cb9c0 con 0x7f0c740ff4f0 2023-09-10T00:38:55.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.471+0000 7f0c71d9b700 1 --2- 172.21.15.44:0/1354028771 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0c5c0a0640 0x7f0c5c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:55.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.472+0000 7f0c71d9b700 1 --2- 172.21.15.44:0/1354028771 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0c5c0a0640 0x7f0c5c0a2b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0c68005950 tx=0x7f0c680058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:55.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.476+0000 7f0c637fe700 1 -- 172.21.15.44:0/1354028771 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0c64095300 con 0x7f0c740ff4f0 2023-09-10T00:38:55.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.697+0000 7f0c789e4700 1 -- 172.21.15.44:0/1354028771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0c7410ae20 con 0x7f0c740ff4f0 2023-09-10T00:38:55.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.697+0000 7f0c637fe700 1 -- 172.21.15.44:0/1354028771 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+584 (secure 0 0 0) 0x7f0c6401d260 con 0x7f0c740ff4f0 2023-09-10T00:38:55.698 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/18127588 2023-09-11T00:32:15.913770+0000 2023-09-10T00:38:55.698 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/18127588 2023-09-11T00:32:15.913770+0000 2023-09-10T00:38:55.698 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1414498756 2023-09-11T00:32:15.913770+0000 2023-09-10T00:38:55.699 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1298831914 2023-09-11T00:32:15.913770+0000 2023-09-10T00:38:55.699 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2415771009 2023-09-11T00:31:49.496807+0000 2023-09-10T00:38:55.699 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/193660691 2023-09-11T00:31:49.496807+0000 2023-09-10T00:38:55.699 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/871815312 2023-09-11T00:31:49.496807+0000 2023-09-10T00:38:55.700 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1481976351 2023-09-11T00:32:15.913770+0000 2023-09-10T00:38:55.700 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/871815312 2023-09-11T00:31:49.496807+0000 2023-09-10T00:38:55.700 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/625830807 2023-09-11T00:31:49.496807+0000 2023-09-10T00:38:55.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.699+0000 7f0c789e4700 1 -- 172.21.15.44:0/1354028771 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0c5c0a0640 msgr2=0x7f0c5c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:55.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.699+0000 7f0c789e4700 1 --2- 172.21.15.44:0/1354028771 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0c5c0a0640 0x7f0c5c0a2b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0c68005950 tx=0x7f0c680058e0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.699+0000 7f0c789e4700 1 -- 172.21.15.44:0/1354028771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c740ff4f0 msgr2=0x7f0c74196350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:55.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.699+0000 7f0c789e4700 1 --2- 172.21.15.44:0/1354028771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c740ff4f0 0x7f0c74196350 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400c970 tx=0x7f0c6400cc80 comp rx=0 tx=0).stop 2023-09-10T00:38:55.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.699+0000 7f0c789e4700 1 -- 172.21.15.44:0/1354028771 shutdown_connections 2023-09-10T00:38:55.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.699+0000 7f0c789e4700 1 --2- 172.21.15.44:0/1354028771 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0c5c0a0640 0x7f0c5c0a2b00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.699+0000 7f0c789e4700 1 --2- 172.21.15.44:0/1354028771 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c74101080 0x7f0c7419d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.699+0000 7f0c789e4700 1 --2- 172.21.15.44:0/1354028771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c740ff4f0 0x7f0c74196350 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.699+0000 7f0c789e4700 1 --2- 172.21.15.44:0/1354028771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c741006e0 0x7f0c74196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:55.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.699+0000 7f0c789e4700 1 -- 172.21.15.44:0/1354028771 >> 172.21.15.44:0/1354028771 conn(0x7f0c740faa60 msgr2=0x7f0c740677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:55.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.699+0000 7f0c789e4700 1 -- 172.21.15.44:0/1354028771 shutdown_connections 2023-09-10T00:38:55.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:55.699+0000 7f0c789e4700 1 -- 172.21.15.44:0/1354028771 wait complete. 2023-09-10T00:38:55.703 INFO:teuthology.orchestra.run.smithi044.stderr:listed 10 entries 2023-09-10T00:38:55.711 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi044.front.sepia.ceph.com... 2023-09-10T00:38:55.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:55.712 DEBUG:teuthology.orchestra.run.smithi044:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-09-10T00:38:55.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi044.stderr:ceph-fuse[157558]: fuse finished with error 0 and tester_r 0 2023-09-10T00:38:55.752 INFO:teuthology.orchestra.run:waiting for 300 2023-09-10T00:38:55.767 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2023-09-10T00:38:55.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:38:55.768 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-09-10T00:38:55.833 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:38:56.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.000+0000 7f00596f6700 1 -- 172.21.15.44:0/4010907513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0054101080 msgr2=0x7f0054107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:56.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.000+0000 7f00596f6700 1 --2- 172.21.15.44:0/4010907513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0054101080 0x7f0054107930 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f004c00b0d0 tx=0x7f004c00b3e0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.000+0000 7f00596f6700 1 -- 172.21.15.44:0/4010907513 shutdown_connections 2023-09-10T00:38:56.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.000+0000 7f00596f6700 1 --2- 172.21.15.44:0/4010907513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0054101080 0x7f0054107930 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.000+0000 7f00596f6700 1 --2- 172.21.15.44:0/4010907513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00540ff4f0 0x7f00540ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.000+0000 7f00596f6700 1 --2- 172.21.15.44:0/4010907513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00541006e0 0x7f0054100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.000+0000 7f00596f6700 1 -- 172.21.15.44:0/4010907513 >> 172.21.15.44:0/4010907513 conn(0x7f00540faa60 msgr2=0x7f00540fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.000+0000 7f00596f6700 1 -- 172.21.15.44:0/4010907513 shutdown_connections 2023-09-10T00:38:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.000+0000 7f00596f6700 1 -- 172.21.15.44:0/4010907513 wait complete. 2023-09-10T00:38:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.001+0000 7f00596f6700 1 Processor -- start 2023-09-10T00:38:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.001+0000 7f00596f6700 1 -- start start 2023-09-10T00:38:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.001+0000 7f00596f6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00540ff4f0 0x7f0054103790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.001+0000 7f00596f6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00541006e0 0x7f0054101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.001+0000 7f00596f6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0054101080 0x7f0054102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:56.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.001+0000 7f00596f6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0054102ab0 con 0x7f00540ff4f0 2023-09-10T00:38:56.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.001+0000 7f00596f6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0054102c20 con 0x7f0054101080 2023-09-10T00:38:56.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.001+0000 7f00596f6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00541a0320 con 0x7f00541006e0 2023-09-10T00:38:56.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.001+0000 7f0052ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00540ff4f0 0x7f0054103790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:56.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.001+0000 7f0052ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00540ff4f0 0x7f0054103790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46660/0 (socket says 172.21.15.44:46660) 2023-09-10T00:38:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.001+0000 7f0052ffd700 1 -- 172.21.15.44:0/736986068 learned_addr learned my addr 172.21.15.44:0/736986068 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.002+0000 7f004a7fc700 1 --2- 172.21.15.44:0/736986068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00541006e0 0x7f0054101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.002+0000 7f00537fe700 1 --2- 172.21.15.44:0/736986068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0054101080 0x7f0054102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.002+0000 7f0052ffd700 1 -- 172.21.15.44:0/736986068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00541006e0 msgr2=0x7f0054101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.002+0000 7f0052ffd700 1 --2- 172.21.15.44:0/736986068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00541006e0 0x7f0054101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.002+0000 7f0052ffd700 1 -- 172.21.15.44:0/736986068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0054101080 msgr2=0x7f0054102310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:56.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.002+0000 7f0052ffd700 1 --2- 172.21.15.44:0/736986068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0054101080 0x7f0054102310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.002+0000 7f0052ffd700 1 -- 172.21.15.44:0/736986068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c009d00 con 0x7f00540ff4f0 2023-09-10T00:38:56.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.002+0000 7f00537fe700 1 --2- 172.21.15.44:0/736986068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0054101080 0x7f0054102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:38:56.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.002+0000 7f004a7fc700 1 --2- 172.21.15.44:0/736986068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00541006e0 0x7f0054101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:38:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.002+0000 7f0052ffd700 1 --2- 172.21.15.44:0/736986068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00540ff4f0 0x7f0054103790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f004400d980 tx=0x7f004400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.002+0000 7f0050ff9700 1 -- 172.21.15.44:0/736986068 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004400bb60 con 0x7f00540ff4f0 2023-09-10T00:38:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.002+0000 7f0050ff9700 1 -- 172.21.15.44:0/736986068 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0044007620 con 0x7f00540ff4f0 2023-09-10T00:38:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.002+0000 7f00596f6700 1 -- 172.21.15.44:0/736986068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00541a0600 con 0x7f00540ff4f0 2023-09-10T00:38:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.002+0000 7f0050ff9700 1 -- 172.21.15.44:0/736986068 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004400f3c0 con 0x7f00540ff4f0 2023-09-10T00:38:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.002+0000 7f00596f6700 1 -- 172.21.15.44:0/736986068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00541a0b10 con 0x7f00540ff4f0 2023-09-10T00:38:56.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.003+0000 7f00596f6700 1 -- 172.21.15.44:0/736986068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00541075d0 con 0x7f00540ff4f0 2023-09-10T00:38:56.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.004+0000 7f0050ff9700 1 -- 172.21.15.44:0/736986068 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f0044010460 con 0x7f00540ff4f0 2023-09-10T00:38:56.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.005+0000 7f0050ff9700 1 --2- 172.21.15.44:0/736986068 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f00400a0690 0x7f00400a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:56.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.005+0000 7f0050ff9700 1 -- 172.21.15.44:0/736986068 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f00440cc620 con 0x7f00540ff4f0 2023-09-10T00:38:56.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.005+0000 7f004a7fc700 1 --2- 172.21.15.44:0/736986068 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f00400a0690 0x7f00400a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:56.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.005+0000 7f004a7fc700 1 --2- 172.21.15.44:0/736986068 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f00400a0690 0x7f00400a2b50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f003c005e10 tx=0x7f003c005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:56.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.008+0000 7f0050ff9700 1 -- 172.21.15.44:0/736986068 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0044095f60 con 0x7f00540ff4f0 2023-09-10T00:38:56.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:55 smithi044 ceph-mon[98676]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2023-09-10T00:38:56.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:55 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/4240283231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:38:56.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:55 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3916022868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:38:56.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:55 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1354028771' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-10T00:38:56.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:55 smithi099 ceph-mon[102692]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2023-09-10T00:38:56.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:55 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/4240283231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:38:56.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:55 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3916022868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:38:56.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:55 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1354028771' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-10T00:38:56.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.212+0000 7f00596f6700 1 -- 172.21.15.44:0/736986068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f005410ae40 con 0x7f00540ff4f0 2023-09-10T00:38:56.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.213+0000 7f0050ff9700 1 -- 172.21.15.44:0/736986068 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23870 (secure 0 0 0) 0x7f0044099580 con 0x7f00540ff4f0 2023-09-10T00:38:56.214 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:56.214 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":87,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:38:44.197902+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:38:56.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.215+0000 7f00596f6700 1 -- 172.21.15.44:0/736986068 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f00400a0690 msgr2=0x7f00400a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:56.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.215+0000 7f00596f6700 1 --2- 172.21.15.44:0/736986068 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f00400a0690 0x7f00400a2b50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f003c005e10 tx=0x7f003c005d60 comp rx=0 tx=0).stop 2023-09-10T00:38:56.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.215+0000 7f00596f6700 1 -- 172.21.15.44:0/736986068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00540ff4f0 msgr2=0x7f0054103790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:56.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.215+0000 7f00596f6700 1 --2- 172.21.15.44:0/736986068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00540ff4f0 0x7f0054103790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f004400d980 tx=0x7f004400dd40 comp rx=0 tx=0).stop 2023-09-10T00:38:56.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.216+0000 7f00596f6700 1 -- 172.21.15.44:0/736986068 shutdown_connections 2023-09-10T00:38:56.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.216+0000 7f00596f6700 1 --2- 172.21.15.44:0/736986068 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f00400a0690 0x7f00400a2b50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.216+0000 7f00596f6700 1 --2- 172.21.15.44:0/736986068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0054101080 0x7f0054102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.216+0000 7f00596f6700 1 --2- 172.21.15.44:0/736986068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00540ff4f0 0x7f0054103790 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.216+0000 7f00596f6700 1 --2- 172.21.15.44:0/736986068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00541006e0 0x7f0054101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.216+0000 7f00596f6700 1 -- 172.21.15.44:0/736986068 >> 172.21.15.44:0/736986068 conn(0x7f00540faa60 msgr2=0x7f00540fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:56.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.216+0000 7f00596f6700 1 -- 172.21.15.44:0/736986068 shutdown_connections 2023-09-10T00:38:56.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.216+0000 7f00596f6700 1 -- 172.21.15.44:0/736986068 wait complete. 2023-09-10T00:38:56.229 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:38:56.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:55 smithi131 ceph-mon[103713]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2023-09-10T00:38:56.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:55 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/4240283231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:38:56.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:55 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3916022868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:38:56.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:55 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1354028771' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-10T00:38:56.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.356+0000 7ff8ccb1f700 1 -- 172.21.15.44:0/1710380378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c80ffde0 msgr2=0x7ff8c8100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:56.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.356+0000 7ff8ccb1f700 1 --2- 172.21.15.44:0/1710380378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c80ffde0 0x7ff8c8100240 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009990 tx=0x7ff8bc009ca0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.357+0000 7ff8ccb1f700 1 -- 172.21.15.44:0/1710380378 shutdown_connections 2023-09-10T00:38:56.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.357+0000 7ff8ccb1f700 1 --2- 172.21.15.44:0/1710380378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8c8100780 0x7ff8c8107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.357+0000 7ff8ccb1f700 1 --2- 172.21.15.44:0/1710380378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8c80febf0 0x7ff8c80ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.357+0000 7ff8ccb1f700 1 --2- 172.21.15.44:0/1710380378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c80ffde0 0x7ff8c8100240 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.357+0000 7ff8ccb1f700 1 -- 172.21.15.44:0/1710380378 >> 172.21.15.44:0/1710380378 conn(0x7ff8c80fa140 msgr2=0x7ff8c80fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:56.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.357+0000 7ff8ccb1f700 1 -- 172.21.15.44:0/1710380378 shutdown_connections 2023-09-10T00:38:56.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.357+0000 7ff8ccb1f700 1 -- 172.21.15.44:0/1710380378 wait complete. 2023-09-10T00:38:56.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.357+0000 7ff8ccb1f700 1 Processor -- start 2023-09-10T00:38:56.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.357+0000 7ff8ccb1f700 1 -- start start 2023-09-10T00:38:56.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.357+0000 7ff8ccb1f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8c80febf0 0x7ff8c8108240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:56.362 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.357+0000 7ff8ccb1f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8c80ffde0 0x7ff8c819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:56.362 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.357+0000 7ff8ccb1f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c8100780 0x7ff8c819d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:56.362 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8ccb1f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c819dce0 con 0x7ff8c8100780 2023-09-10T00:38:56.362 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8ccb1f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c819de50 con 0x7ff8c80febf0 2023-09-10T00:38:56.362 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8ccb1f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c819e010 con 0x7ff8c80ffde0 2023-09-10T00:38:56.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8c6d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c8100780 0x7ff8c819d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:56.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8c6d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c8100780 0x7ff8c819d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46678/0 (socket says 172.21.15.44:46678) 2023-09-10T00:38:56.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8c6d9d700 1 -- 172.21.15.44:0/34957199 learned_addr learned my addr 172.21.15.44:0/34957199 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:56.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8c5d9b700 1 --2- 172.21.15.44:0/34957199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8c80ffde0 0x7ff8c819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:56.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8c659c700 1 --2- 172.21.15.44:0/34957199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8c80febf0 0x7ff8c8108240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:56.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8c6d9d700 1 -- 172.21.15.44:0/34957199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8c80ffde0 msgr2=0x7ff8c819aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:56.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8c6d9d700 1 --2- 172.21.15.44:0/34957199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8c80ffde0 0x7ff8c819aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8c6d9d700 1 -- 172.21.15.44:0/34957199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8c80febf0 msgr2=0x7ff8c8108240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:56.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8c6d9d700 1 --2- 172.21.15.44:0/34957199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8c80febf0 0x7ff8c8108240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8c6d9d700 1 -- 172.21.15.44:0/34957199 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc0095f0 con 0x7ff8c8100780 2023-09-10T00:38:56.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8c6d9d700 1 --2- 172.21.15.44:0/34957199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c8100780 0x7ff8c819d510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000ef30 tx=0x7ff8c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:56.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8b77fe700 1 -- 172.21.15.44:0/34957199 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0018070 con 0x7ff8c8100780 2023-09-10T00:38:56.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8b77fe700 1 -- 172.21.15.44:0/34957199 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff8c000f040 con 0x7ff8c8100780 2023-09-10T00:38:56.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8ccb1f700 1 -- 172.21.15.44:0/34957199 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c819e290 con 0x7ff8c8100780 2023-09-10T00:38:56.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8b77fe700 1 -- 172.21.15.44:0/34957199 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c000cc70 con 0x7ff8c8100780 2023-09-10T00:38:56.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.358+0000 7ff8ccb1f700 1 -- 172.21.15.44:0/34957199 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c819e720 con 0x7ff8c8100780 2023-09-10T00:38:56.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.359+0000 7ff8ccb1f700 1 -- 172.21.15.44:0/34957199 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c81061d0 con 0x7ff8c8100780 2023-09-10T00:38:56.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.360+0000 7ff8b77fe700 1 -- 172.21.15.44:0/34957199 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ff8c0007500 con 0x7ff8c8100780 2023-09-10T00:38:56.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.361+0000 7ff8b77fe700 1 --2- 172.21.15.44:0/34957199 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff8b00a0690 0x7ff8b00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:56.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.361+0000 7ff8b77fe700 1 -- 172.21.15.44:0/34957199 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff8c00cb990 con 0x7ff8c8100780 2023-09-10T00:38:56.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.361+0000 7ff8c659c700 1 --2- 172.21.15.44:0/34957199 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff8b00a0690 0x7ff8b00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:56.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.361+0000 7ff8c659c700 1 --2- 172.21.15.44:0/34957199 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff8b00a0690 0x7ff8b00a2b50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80ffc40 tx=0x7ff8b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:56.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.365+0000 7ff8b77fe700 1 -- 172.21.15.44:0/34957199 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff8c0095300 con 0x7ff8c8100780 2023-09-10T00:38:56.570 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.570+0000 7ff8ccb1f700 1 -- 172.21.15.44:0/34957199 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8c802cea0 con 0x7ff8c8100780 2023-09-10T00:38:56.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.571+0000 7ff8b77fe700 1 -- 172.21.15.44:0/34957199 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23870 (secure 0 0 0) 0x7ff8c0098920 con 0x7ff8c8100780 2023-09-10T00:38:56.572 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:38:56.572 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":87,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:38:44.197902+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:38:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.573+0000 7ff8ccb1f700 1 -- 172.21.15.44:0/34957199 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff8b00a0690 msgr2=0x7ff8b00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.573+0000 7ff8ccb1f700 1 --2- 172.21.15.44:0/34957199 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff8b00a0690 0x7ff8b00a2b50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80ffc40 tx=0x7ff8b8009260 comp rx=0 tx=0).stop 2023-09-10T00:38:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.573+0000 7ff8ccb1f700 1 -- 172.21.15.44:0/34957199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c8100780 msgr2=0x7ff8c819d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.573+0000 7ff8ccb1f700 1 --2- 172.21.15.44:0/34957199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c8100780 0x7ff8c819d510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000ef30 tx=0x7ff8c000c3a0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.574+0000 7ff8ccb1f700 1 -- 172.21.15.44:0/34957199 shutdown_connections 2023-09-10T00:38:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.574+0000 7ff8ccb1f700 1 --2- 172.21.15.44:0/34957199 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff8b00a0690 0x7ff8b00a2b50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.574+0000 7ff8ccb1f700 1 --2- 172.21.15.44:0/34957199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c8100780 0x7ff8c819d510 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.574+0000 7ff8ccb1f700 1 --2- 172.21.15.44:0/34957199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8c80febf0 0x7ff8c8108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.574+0000 7ff8ccb1f700 1 --2- 172.21.15.44:0/34957199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8c80ffde0 0x7ff8c819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.574+0000 7ff8ccb1f700 1 -- 172.21.15.44:0/34957199 >> 172.21.15.44:0/34957199 conn(0x7ff8c80fa140 msgr2=0x7ff8c8103310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:56.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.574+0000 7ff8ccb1f700 1 -- 172.21.15.44:0/34957199 shutdown_connections 2023-09-10T00:38:56.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.574+0000 7ff8ccb1f700 1 -- 172.21.15.44:0/34957199 wait complete. 2023-09-10T00:38:56.587 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-10T00:38:56.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.715+0000 7f4c9a5e5700 1 -- 172.21.15.44:0/4156937090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c94101080 msgr2=0x7f4c94107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:56.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.715+0000 7f4c9a5e5700 1 --2- 172.21.15.44:0/4156937090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c94101080 0x7f4c94107930 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00b0d0 tx=0x7f4c8c00b3e0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.715+0000 7f4c9a5e5700 1 -- 172.21.15.44:0/4156937090 shutdown_connections 2023-09-10T00:38:56.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.715+0000 7f4c9a5e5700 1 --2- 172.21.15.44:0/4156937090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c94101080 0x7f4c94107930 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.715+0000 7f4c9a5e5700 1 --2- 172.21.15.44:0/4156937090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c940ff4f0 0x7f4c940ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.715+0000 7f4c9a5e5700 1 --2- 172.21.15.44:0/4156937090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c941006e0 0x7f4c94100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.715+0000 7f4c9a5e5700 1 -- 172.21.15.44:0/4156937090 >> 172.21.15.44:0/4156937090 conn(0x7f4c940faa60 msgr2=0x7f4c940fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:56.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.715+0000 7f4c9a5e5700 1 -- 172.21.15.44:0/4156937090 shutdown_connections 2023-09-10T00:38:56.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.715+0000 7f4c9a5e5700 1 -- 172.21.15.44:0/4156937090 wait complete. 2023-09-10T00:38:56.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c9a5e5700 1 Processor -- start 2023-09-10T00:38:56.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c9a5e5700 1 -- start start 2023-09-10T00:38:56.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c9a5e5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c940ff4f0 0x7f4c94196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:56.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c9a5e5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c941006e0 0x7f4c941967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:56.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c9a5e5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c94101080 0x7f4c9419d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:56.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c9a5e5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9419ddf0 con 0x7f4c940ff4f0 2023-09-10T00:38:56.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c9a5e5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9419df30 con 0x7f4c94101080 2023-09-10T00:38:56.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c9a5e5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9419e0c0 con 0x7f4c941006e0 2023-09-10T00:38:56.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c93fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c940ff4f0 0x7f4c94196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:56.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c93fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c940ff4f0 0x7f4c94196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46688/0 (socket says 172.21.15.44:46688) 2023-09-10T00:38:56.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c93fff700 1 -- 172.21.15.44:0/4224353459 learned_addr learned my addr 172.21.15.44:0/4224353459 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:56.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c937fe700 1 --2- 172.21.15.44:0/4224353459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c941006e0 0x7f4c941967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:56.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c98b82700 1 --2- 172.21.15.44:0/4224353459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c94101080 0x7f4c9419d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:56.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c93fff700 1 -- 172.21.15.44:0/4224353459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c941006e0 msgr2=0x7f4c941967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:56.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c93fff700 1 --2- 172.21.15.44:0/4224353459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c941006e0 0x7f4c941967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c93fff700 1 -- 172.21.15.44:0/4224353459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c94101080 msgr2=0x7f4c9419d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:56.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c93fff700 1 --2- 172.21.15.44:0/4224353459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c94101080 0x7f4c9419d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:56.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c93fff700 1 -- 172.21.15.44:0/4224353459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c009d00 con 0x7f4c940ff4f0 2023-09-10T00:38:56.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.716+0000 7f4c98b82700 1 --2- 172.21.15.44:0/4224353459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c94101080 0x7f4c9419d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:38:56.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.717+0000 7f4c93fff700 1 --2- 172.21.15.44:0/4224353459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c940ff4f0 0x7f4c94196270 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00d980 tx=0x7f4c7c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:56.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.717+0000 7f4c917fa700 1 -- 172.21.15.44:0/4224353459 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c00de60 con 0x7f4c940ff4f0 2023-09-10T00:38:56.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.717+0000 7f4c917fa700 1 -- 172.21.15.44:0/4224353459 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c7c0044b0 con 0x7f4c940ff4f0 2023-09-10T00:38:56.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.717+0000 7f4c9a5e5700 1 -- 172.21.15.44:0/4224353459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9419e3a0 con 0x7f4c940ff4f0 2023-09-10T00:38:56.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.717+0000 7f4c917fa700 1 -- 172.21.15.44:0/4224353459 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c003b40 con 0x7f4c940ff4f0 2023-09-10T00:38:56.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.717+0000 7f4c9a5e5700 1 -- 172.21.15.44:0/4224353459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9419e8b0 con 0x7f4c940ff4f0 2023-09-10T00:38:56.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.718+0000 7f4c9a5e5700 1 -- 172.21.15.44:0/4224353459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c941075d0 con 0x7f4c940ff4f0 2023-09-10T00:38:56.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.719+0000 7f4c917fa700 1 -- 172.21.15.44:0/4224353459 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f4c7c003ca0 con 0x7f4c940ff4f0 2023-09-10T00:38:56.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.719+0000 7f4c917fa700 1 --2- 172.21.15.44:0/4224353459 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4c800a0790 0x7f4c800a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:56.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.720+0000 7f4c937fe700 1 --2- 172.21.15.44:0/4224353459 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4c800a0790 0x7f4c800a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:56.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.720+0000 7f4c917fa700 1 -- 172.21.15.44:0/4224353459 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4c7c0cbd00 con 0x7f4c940ff4f0 2023-09-10T00:38:56.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.720+0000 7f4c937fe700 1 --2- 172.21.15.44:0/4224353459 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4c800a0790 0x7f4c800a2c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4c84005ea0 tx=0x7f4c84005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:56.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.723+0000 7f4c917fa700 1 -- 172.21.15.44:0/4224353459 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4c7c095740 con 0x7f4c940ff4f0 2023-09-10T00:38:56.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:56.975+0000 7f4c9a5e5700 1 -- 172.21.15.44:0/4224353459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4c9402cf90 con 0x7f4c940ff4f0 2023-09-10T00:38:57.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:56 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/736986068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:57.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:56 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/34957199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:56 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/736986068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:56 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/34957199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:56 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/736986068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:56 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/34957199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:38:57.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.818+0000 7f4c917fa700 1 -- 172.21.15.44:0/4224353459 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4c7c0170b0 con 0x7f4c940ff4f0 2023-09-10T00:38:57.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.821+0000 7f4c9a5e5700 1 -- 172.21.15.44:0/4224353459 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4c800a0790 msgr2=0x7f4c800a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:57.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.821+0000 7f4c9a5e5700 1 --2- 172.21.15.44:0/4224353459 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4c800a0790 0x7f4c800a2c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4c84005ea0 tx=0x7f4c84005e30 comp rx=0 tx=0).stop 2023-09-10T00:38:57.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.821+0000 7f4c9a5e5700 1 -- 172.21.15.44:0/4224353459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c940ff4f0 msgr2=0x7f4c94196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:57.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.821+0000 7f4c9a5e5700 1 --2- 172.21.15.44:0/4224353459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c940ff4f0 0x7f4c94196270 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00d980 tx=0x7f4c7c00dd40 comp rx=0 tx=0).stop 2023-09-10T00:38:57.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.821+0000 7f4c9a5e5700 1 -- 172.21.15.44:0/4224353459 shutdown_connections 2023-09-10T00:38:57.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.821+0000 7f4c9a5e5700 1 --2- 172.21.15.44:0/4224353459 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4c800a0790 0x7f4c800a2c50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:57.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.821+0000 7f4c9a5e5700 1 --2- 172.21.15.44:0/4224353459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c94101080 0x7f4c9419d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:57.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.821+0000 7f4c9a5e5700 1 --2- 172.21.15.44:0/4224353459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c940ff4f0 0x7f4c94196270 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:57.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.821+0000 7f4c9a5e5700 1 --2- 172.21.15.44:0/4224353459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c941006e0 0x7f4c941967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:57.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.821+0000 7f4c9a5e5700 1 -- 172.21.15.44:0/4224353459 >> 172.21.15.44:0/4224353459 conn(0x7f4c940faa60 msgr2=0x7f4c940fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:57.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.821+0000 7f4c9a5e5700 1 -- 172.21.15.44:0/4224353459 shutdown_connections 2023-09-10T00:38:57.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.821+0000 7f4c9a5e5700 1 -- 172.21.15.44:0/4224353459 wait complete. 2023-09-10T00:38:57.837 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2023-09-10T00:38:57.837 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-09-10T00:38:57.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.973+0000 7fba820ae700 1 -- 172.21.15.44:0/2827919245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba7c1021c0 msgr2=0x7fba7c108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:57.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.973+0000 7fba820ae700 1 --2- 172.21.15.44:0/2827919245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba7c1021c0 0x7fba7c108a70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fba7400b0d0 tx=0x7fba7400b3e0 comp rx=0 tx=0).stop 2023-09-10T00:38:57.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.973+0000 7fba820ae700 1 -- 172.21.15.44:0/2827919245 shutdown_connections 2023-09-10T00:38:57.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.973+0000 7fba820ae700 1 --2- 172.21.15.44:0/2827919245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba7c1021c0 0x7fba7c108a70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:57.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.973+0000 7fba820ae700 1 --2- 172.21.15.44:0/2827919245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba7c100630 0x7fba7c100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:57.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.973+0000 7fba820ae700 1 --2- 172.21.15.44:0/2827919245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba7c101820 0x7fba7c101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:57.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.973+0000 7fba820ae700 1 -- 172.21.15.44:0/2827919245 >> 172.21.15.44:0/2827919245 conn(0x7fba7c0fbb80 msgr2=0x7fba7c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:57.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.973+0000 7fba820ae700 1 -- 172.21.15.44:0/2827919245 shutdown_connections 2023-09-10T00:38:57.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.973+0000 7fba820ae700 1 -- 172.21.15.44:0/2827919245 wait complete. 2023-09-10T00:38:57.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.974+0000 7fba820ae700 1 Processor -- start 2023-09-10T00:38:57.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.974+0000 7fba820ae700 1 -- start start 2023-09-10T00:38:57.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.974+0000 7fba820ae700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba7c100630 0x7fba7c1962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:57.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.974+0000 7fba820ae700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba7c101820 0x7fba7c1967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:57.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.974+0000 7fba820ae700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba7c1021c0 0x7fba7c19d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:57.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.974+0000 7fba820ae700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c19de30 con 0x7fba7c100630 2023-09-10T00:38:57.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.974+0000 7fba820ae700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c19df70 con 0x7fba7c101820 2023-09-10T00:38:57.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.974+0000 7fba820ae700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c19e100 con 0x7fba7c1021c0 2023-09-10T00:38:57.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.974+0000 7fba7b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba7c100630 0x7fba7c1962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:57.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.974+0000 7fba7b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba7c100630 0x7fba7c1962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46702/0 (socket says 172.21.15.44:46702) 2023-09-10T00:38:57.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.974+0000 7fba7b7fe700 1 -- 172.21.15.44:0/4135366904 learned_addr learned my addr 172.21.15.44:0/4135366904 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:57.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.974+0000 7fba7bfff700 1 --2- 172.21.15.44:0/4135366904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba7c1021c0 0x7fba7c19d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:57.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.974+0000 7fba7affd700 1 --2- 172.21.15.44:0/4135366904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba7c101820 0x7fba7c1967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:57.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.975+0000 7fba7b7fe700 1 -- 172.21.15.44:0/4135366904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba7c1021c0 msgr2=0x7fba7c19d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:57.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.975+0000 7fba7b7fe700 1 --2- 172.21.15.44:0/4135366904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba7c1021c0 0x7fba7c19d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:57.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.975+0000 7fba7b7fe700 1 -- 172.21.15.44:0/4135366904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba7c101820 msgr2=0x7fba7c1967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:57.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.975+0000 7fba7b7fe700 1 --2- 172.21.15.44:0/4135366904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba7c101820 0x7fba7c1967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:57.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.975+0000 7fba7b7fe700 1 -- 172.21.15.44:0/4135366904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba74009d00 con 0x7fba7c100630 2023-09-10T00:38:57.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.975+0000 7fba7affd700 1 --2- 172.21.15.44:0/4135366904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba7c101820 0x7fba7c1967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:38:57.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.975+0000 7fba7b7fe700 1 --2- 172.21.15.44:0/4135366904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba7c100630 0x7fba7c1962b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fba7000d980 tx=0x7fba7000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:57.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.975+0000 7fba78ff9700 1 -- 172.21.15.44:0/4135366904 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70004020 con 0x7fba7c100630 2023-09-10T00:38:57.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.975+0000 7fba78ff9700 1 -- 172.21.15.44:0/4135366904 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fba70009730 con 0x7fba7c100630 2023-09-10T00:38:57.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.975+0000 7fba820ae700 1 -- 172.21.15.44:0/4135366904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7c19e3e0 con 0x7fba7c100630 2023-09-10T00:38:57.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.975+0000 7fba78ff9700 1 -- 172.21.15.44:0/4135366904 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70003a00 con 0x7fba7c100630 2023-09-10T00:38:57.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.975+0000 7fba820ae700 1 -- 172.21.15.44:0/4135366904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7c19e8f0 con 0x7fba7c100630 2023-09-10T00:38:57.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.976+0000 7fba820ae700 1 -- 172.21.15.44:0/4135366904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7c10bd00 con 0x7fba7c100630 2023-09-10T00:38:57.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.977+0000 7fba78ff9700 1 -- 172.21.15.44:0/4135366904 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fba700098a0 con 0x7fba7c100630 2023-09-10T00:38:57.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.977+0000 7fba78ff9700 1 --2- 172.21.15.44:0/4135366904 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba640a0640 0x7fba640a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:57.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.978+0000 7fba78ff9700 1 -- 172.21.15.44:0/4135366904 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fba700cbde0 con 0x7fba7c100630 2023-09-10T00:38:57.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.978+0000 7fba7affd700 1 --2- 172.21.15.44:0/4135366904 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba640a0640 0x7fba640a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:57.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.978+0000 7fba7affd700 1 --2- 172.21.15.44:0/4135366904 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba640a0640 0x7fba640a2b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba6c005fd0 tx=0x7fba6c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:57.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:57.981+0000 7fba78ff9700 1 -- 172.21.15.44:0/4135366904 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fba70095720 con 0x7fba7c100630 2023-09-10T00:38:58.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:57 smithi099 ceph-mon[102692]: pgmap v277: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 273 B/s wr, 0 op/s 2023-09-10T00:38:58.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:57 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/4224353459' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-10T00:38:58.230 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.229+0000 7fba820ae700 1 -- 172.21.15.44:0/4135366904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fba7c1074c0 con 0x7fba7c100630 2023-09-10T00:38:58.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:57 smithi131 ceph-mon[103713]: pgmap v277: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 273 B/s wr, 0 op/s 2023-09-10T00:38:58.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:57 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/4224353459' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-10T00:38:58.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:57 smithi044 ceph-mon[98676]: pgmap v277: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 273 B/s wr, 0 op/s 2023-09-10T00:38:58.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:57 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/4224353459' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-10T00:38:58.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.823+0000 7fba78ff9700 1 -- 172.21.15.44:0/4135366904 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7fba70098d40 con 0x7fba7c100630 2023-09-10T00:38:58.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.828+0000 7fba820ae700 1 -- 172.21.15.44:0/4135366904 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba640a0640 msgr2=0x7fba640a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:58.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.828+0000 7fba820ae700 1 --2- 172.21.15.44:0/4135366904 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba640a0640 0x7fba640a2b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba6c005fd0 tx=0x7fba6c005e30 comp rx=0 tx=0).stop 2023-09-10T00:38:58.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.828+0000 7fba820ae700 1 -- 172.21.15.44:0/4135366904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba7c100630 msgr2=0x7fba7c1962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:58.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.828+0000 7fba820ae700 1 --2- 172.21.15.44:0/4135366904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba7c100630 0x7fba7c1962b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fba7000d980 tx=0x7fba7000dc90 comp rx=0 tx=0).stop 2023-09-10T00:38:58.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.828+0000 7fba820ae700 1 -- 172.21.15.44:0/4135366904 shutdown_connections 2023-09-10T00:38:58.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.828+0000 7fba820ae700 1 --2- 172.21.15.44:0/4135366904 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba640a0640 0x7fba640a2b00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:58.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.828+0000 7fba820ae700 1 --2- 172.21.15.44:0/4135366904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba7c1021c0 0x7fba7c19d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:58.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.828+0000 7fba820ae700 1 --2- 172.21.15.44:0/4135366904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba7c100630 0x7fba7c1962b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.828+0000 7fba820ae700 1 --2- 172.21.15.44:0/4135366904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba7c101820 0x7fba7c1967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.828+0000 7fba820ae700 1 -- 172.21.15.44:0/4135366904 >> 172.21.15.44:0/4135366904 conn(0x7fba7c0fbb80 msgr2=0x7fba7c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.828+0000 7fba820ae700 1 -- 172.21.15.44:0/4135366904 shutdown_connections 2023-09-10T00:38:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.828+0000 7fba820ae700 1 -- 172.21.15.44:0/4135366904 wait complete. 2023-09-10T00:38:58.848 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-09-10T00:38:58.986 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.983+0000 7f2a1dae8700 1 -- 172.21.15.44:0/1903927763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a18101080 msgr2=0x7f2a18107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:58.986 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.983+0000 7f2a1dae8700 1 --2- 172.21.15.44:0/1903927763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a18101080 0x7f2a18107930 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000b0d0 tx=0x7f2a1000b3e0 comp rx=0 tx=0).stop 2023-09-10T00:38:58.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.983+0000 7f2a1dae8700 1 -- 172.21.15.44:0/1903927763 shutdown_connections 2023-09-10T00:38:58.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.983+0000 7f2a1dae8700 1 --2- 172.21.15.44:0/1903927763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a18101080 0x7f2a18107930 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:58.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.983+0000 7f2a1dae8700 1 --2- 172.21.15.44:0/1903927763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a180ff4f0 0x7f2a180ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:58.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.983+0000 7f2a1dae8700 1 --2- 172.21.15.44:0/1903927763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a181006e0 0x7f2a18100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:58.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.983+0000 7f2a1dae8700 1 -- 172.21.15.44:0/1903927763 >> 172.21.15.44:0/1903927763 conn(0x7f2a180faa60 msgr2=0x7f2a180fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:58.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.983+0000 7f2a1dae8700 1 -- 172.21.15.44:0/1903927763 shutdown_connections 2023-09-10T00:38:58.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.983+0000 7f2a1dae8700 1 -- 172.21.15.44:0/1903927763 wait complete. 2023-09-10T00:38:58.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.984+0000 7f2a1dae8700 1 Processor -- start 2023-09-10T00:38:58.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.984+0000 7f2a1dae8700 1 -- start start 2023-09-10T00:38:58.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.984+0000 7f2a1dae8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a180ff4f0 0x7f2a180682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:58.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.984+0000 7f2a1dae8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a181006e0 0x7f2a1819b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:58.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.984+0000 7f2a1dae8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a18101080 0x7f2a1819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:58.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.984+0000 7f2a1dae8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1819de30 con 0x7f2a181006e0 2023-09-10T00:38:58.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.984+0000 7f2a1dae8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1819dfa0 con 0x7f2a180ff4f0 2023-09-10T00:38:58.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.984+0000 7f2a1dae8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1819e160 con 0x7f2a18101080 2023-09-10T00:38:58.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.984+0000 7f2a16ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a181006e0 0x7f2a1819b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:58.990 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.984+0000 7f2a16ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a181006e0 0x7f2a1819b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46718/0 (socket says 172.21.15.44:46718) 2023-09-10T00:38:58.990 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.984+0000 7f2a16ffd700 1 -- 172.21.15.44:0/1140030129 learned_addr learned my addr 172.21.15.44:0/1140030129 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:58.990 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.984+0000 7f2a177fe700 1 --2- 172.21.15.44:0/1140030129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a180ff4f0 0x7f2a180682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:58.990 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.984+0000 7f2a17fff700 1 --2- 172.21.15.44:0/1140030129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a18101080 0x7f2a1819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:58.990 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.985+0000 7f2a16ffd700 1 -- 172.21.15.44:0/1140030129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a18101080 msgr2=0x7f2a1819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:58.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.985+0000 7f2a16ffd700 1 --2- 172.21.15.44:0/1140030129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a18101080 0x7f2a1819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:58.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.985+0000 7f2a16ffd700 1 -- 172.21.15.44:0/1140030129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a180ff4f0 msgr2=0x7f2a180682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:58.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.985+0000 7f2a16ffd700 1 --2- 172.21.15.44:0/1140030129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a180ff4f0 0x7f2a180682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:58.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.985+0000 7f2a16ffd700 1 -- 172.21.15.44:0/1140030129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a10009d00 con 0x7f2a181006e0 2023-09-10T00:38:58.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.985+0000 7f2a16ffd700 1 --2- 172.21.15.44:0/1140030129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a181006e0 0x7f2a1819b010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800c920 tx=0x7f2a0800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:58.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.985+0000 7f2a14ff9700 1 -- 172.21.15.44:0/1140030129 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a08004a30 con 0x7f2a181006e0 2023-09-10T00:38:58.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.985+0000 7f2a14ff9700 1 -- 172.21.15.44:0/1140030129 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2a080054b0 con 0x7f2a181006e0 2023-09-10T00:38:58.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.985+0000 7f2a14ff9700 1 -- 172.21.15.44:0/1140030129 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a08007ab0 con 0x7f2a181006e0 2023-09-10T00:38:58.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.985+0000 7f2a1dae8700 1 -- 172.21.15.44:0/1140030129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1819e440 con 0x7f2a181006e0 2023-09-10T00:38:58.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.985+0000 7f2a1dae8700 1 -- 172.21.15.44:0/1140030129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1819e900 con 0x7f2a181006e0 2023-09-10T00:38:58.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.986+0000 7f2a1dae8700 1 -- 172.21.15.44:0/1140030129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a18106470 con 0x7f2a181006e0 2023-09-10T00:38:58.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.987+0000 7f2a14ff9700 1 -- 172.21.15.44:0/1140030129 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f2a08007c50 con 0x7f2a181006e0 2023-09-10T00:38:58.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.988+0000 7f2a14ff9700 1 --2- 172.21.15.44:0/1140030129 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2a000a0640 0x7f2a000a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:58.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.988+0000 7f2a14ff9700 1 -- 172.21.15.44:0/1140030129 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2a080cbd30 con 0x7f2a181006e0 2023-09-10T00:38:58.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.988+0000 7f2a177fe700 1 --2- 172.21.15.44:0/1140030129 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2a000a0640 0x7f2a000a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:58.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.988+0000 7f2a177fe700 1 --2- 172.21.15.44:0/1140030129 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2a000a0640 0x7f2a000a2b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2a18100540 tx=0x7f2a0c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:58.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:58.991+0000 7f2a14ff9700 1 -- 172.21.15.44:0/1140030129 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2a08095670 con 0x7f2a181006e0 2023-09-10T00:38:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:58 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/4224353459' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-10T00:38:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:58 smithi099 ceph-mon[102692]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-10T00:38:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:58 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/4135366904' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-10T00:38:59.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.239+0000 7f2a1dae8700 1 -- 172.21.15.44:0/1140030129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f2a1810ae00 con 0x7f2a181006e0 2023-09-10T00:38:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:58 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/4224353459' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-10T00:38:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:58 smithi131 ceph-mon[103713]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-10T00:38:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:58 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/4135366904' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-10T00:38:59.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:58 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/4224353459' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-10T00:38:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:58 smithi044 ceph-mon[98676]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-10T00:38:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:58 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/4135366904' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-10T00:38:59.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.834+0000 7f2a14ff9700 1 -- 172.21.15.44:0/1140030129 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f2a0801d0c0 con 0x7f2a181006e0 2023-09-10T00:38:59.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.836+0000 7f2a1dae8700 1 -- 172.21.15.44:0/1140030129 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2a000a0640 msgr2=0x7f2a000a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:59.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.836+0000 7f2a1dae8700 1 --2- 172.21.15.44:0/1140030129 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2a000a0640 0x7f2a000a2b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2a18100540 tx=0x7f2a0c00a380 comp rx=0 tx=0).stop 2023-09-10T00:38:59.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.836+0000 7f2a1dae8700 1 -- 172.21.15.44:0/1140030129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a181006e0 msgr2=0x7f2a1819b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:59.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.836+0000 7f2a1dae8700 1 --2- 172.21.15.44:0/1140030129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a181006e0 0x7f2a1819b010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800c920 tx=0x7f2a0800cc30 comp rx=0 tx=0).stop 2023-09-10T00:38:59.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.837+0000 7f2a1dae8700 1 -- 172.21.15.44:0/1140030129 shutdown_connections 2023-09-10T00:38:59.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.837+0000 7f2a1dae8700 1 --2- 172.21.15.44:0/1140030129 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2a000a0640 0x7f2a000a2b00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:59.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.837+0000 7f2a1dae8700 1 --2- 172.21.15.44:0/1140030129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a18101080 0x7f2a1819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:59.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.837+0000 7f2a1dae8700 1 --2- 172.21.15.44:0/1140030129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a180ff4f0 0x7f2a180682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:59.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.837+0000 7f2a1dae8700 1 --2- 172.21.15.44:0/1140030129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a181006e0 0x7f2a1819b010 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:59.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.837+0000 7f2a1dae8700 1 -- 172.21.15.44:0/1140030129 >> 172.21.15.44:0/1140030129 conn(0x7f2a180faa60 msgr2=0x7f2a180fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:59.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.837+0000 7f2a1dae8700 1 -- 172.21.15.44:0/1140030129 shutdown_connections 2023-09-10T00:38:59.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.837+0000 7f2a1dae8700 1 -- 172.21.15.44:0/1140030129 wait complete. 2023-09-10T00:38:59.856 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-09-10T00:38:59.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.990+0000 7ff192059700 1 -- 172.21.15.44:0/3477634365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff18c1006e0 msgr2=0x7ff18c100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:59.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.990+0000 7ff192059700 1 --2- 172.21.15.44:0/3477634365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff18c1006e0 0x7ff18c100b40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0099c0 tx=0x7ff17c009cd0 comp rx=0 tx=0).stop 2023-09-10T00:38:59.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.990+0000 7ff192059700 1 -- 172.21.15.44:0/3477634365 shutdown_connections 2023-09-10T00:38:59.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.990+0000 7ff192059700 1 --2- 172.21.15.44:0/3477634365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff18c101080 0x7ff18c107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:59.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.990+0000 7ff192059700 1 --2- 172.21.15.44:0/3477634365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff18c0ff4f0 0x7ff18c0ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:59.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.990+0000 7ff192059700 1 --2- 172.21.15.44:0/3477634365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff18c1006e0 0x7ff18c100b40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:59.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.990+0000 7ff192059700 1 -- 172.21.15.44:0/3477634365 >> 172.21.15.44:0/3477634365 conn(0x7ff18c0faa60 msgr2=0x7ff18c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:38:59.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.990+0000 7ff192059700 1 -- 172.21.15.44:0/3477634365 shutdown_connections 2023-09-10T00:38:59.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.990+0000 7ff192059700 1 -- 172.21.15.44:0/3477634365 wait complete. 2023-09-10T00:38:59.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff192059700 1 Processor -- start 2023-09-10T00:38:59.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff192059700 1 -- start start 2023-09-10T00:38:59.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff192059700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff18c0ff4f0 0x7ff18c195110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:59.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff192059700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff18c1006e0 0x7ff18c195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:59.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff192059700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff18c101080 0x7ff18c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:38:59.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff192059700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c19cc90 con 0x7ff18c0ff4f0 2023-09-10T00:38:59.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff192059700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c19cdd0 con 0x7ff18c1006e0 2023-09-10T00:38:59.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff192059700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c19cf60 con 0x7ff18c101080 2023-09-10T00:38:59.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff18b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff18c0ff4f0 0x7ff18c195110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:59.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff18b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff18c0ff4f0 0x7ff18c195110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46730/0 (socket says 172.21.15.44:46730) 2023-09-10T00:38:59.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff18b7fe700 1 -- 172.21.15.44:0/2961733839 learned_addr learned my addr 172.21.15.44:0/2961733839 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:38:59.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff18affd700 1 --2- 172.21.15.44:0/2961733839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff18c1006e0 0x7ff18c195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:59.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff18bfff700 1 --2- 172.21.15.44:0/2961733839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff18c101080 0x7ff18c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:38:59.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff18b7fe700 1 -- 172.21.15.44:0/2961733839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff18c101080 msgr2=0x7ff18c19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:59.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff18b7fe700 1 --2- 172.21.15.44:0/2961733839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff18c101080 0x7ff18c19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:59.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff18b7fe700 1 -- 172.21.15.44:0/2961733839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff18c1006e0 msgr2=0x7ff18c195650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:38:59.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff18b7fe700 1 --2- 172.21.15.44:0/2961733839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff18c1006e0 0x7ff18c195650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:38:59.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff18b7fe700 1 -- 172.21.15.44:0/2961733839 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c0095f0 con 0x7ff18c0ff4f0 2023-09-10T00:38:59.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.991+0000 7ff18bfff700 1 --2- 172.21.15.44:0/2961733839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff18c101080 0x7ff18c19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:38:59.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.992+0000 7ff18b7fe700 1 --2- 172.21.15.44:0/2961733839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff18c0ff4f0 0x7ff18c195110 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff17400d980 tx=0x7ff17400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:38:59.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.992+0000 7ff188ff9700 1 -- 172.21.15.44:0/2961733839 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17400de60 con 0x7ff18c0ff4f0 2023-09-10T00:38:59.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.992+0000 7ff192059700 1 -- 172.21.15.44:0/2961733839 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18c19d240 con 0x7ff18c0ff4f0 2023-09-10T00:38:59.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.992+0000 7ff188ff9700 1 -- 172.21.15.44:0/2961733839 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1740044b0 con 0x7ff18c0ff4f0 2023-09-10T00:38:59.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.992+0000 7ff188ff9700 1 -- 172.21.15.44:0/2961733839 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174003b40 con 0x7ff18c0ff4f0 2023-09-10T00:38:59.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.992+0000 7ff192059700 1 -- 172.21.15.44:0/2961733839 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18c19d750 con 0x7ff18c0ff4f0 2023-09-10T00:39:00.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.993+0000 7ff192059700 1 -- 172.21.15.44:0/2961733839 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18c106560 con 0x7ff18c0ff4f0 2023-09-10T00:39:00.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.994+0000 7ff188ff9700 1 -- 172.21.15.44:0/2961733839 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ff174003ca0 con 0x7ff18c0ff4f0 2023-09-10T00:39:00.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.994+0000 7ff188ff9700 1 --2- 172.21.15.44:0/2961733839 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff1780a0640 0x7ff1780a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:00.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.994+0000 7ff188ff9700 1 -- 172.21.15.44:0/2961733839 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff1740cba70 con 0x7ff18c0ff4f0 2023-09-10T00:39:00.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.994+0000 7ff18affd700 1 --2- 172.21.15.44:0/2961733839 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff1780a0640 0x7ff1780a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:00.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.995+0000 7ff18affd700 1 --2- 172.21.15.44:0/2961733839 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff1780a0640 0x7ff1780a2b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00c010 tx=0x7ff17c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:00.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:38:59.999+0000 7ff188ff9700 1 -- 172.21.15.44:0/2961733839 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff1740953b0 con 0x7ff18c0ff4f0 2023-09-10T00:39:00.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:59 smithi099 ceph-mon[102692]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-09-10T00:39:00.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:59 smithi099 ceph-mon[102692]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-10T00:39:00.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:59 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/4135366904' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-09-10T00:39:00.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:59 smithi099 ceph-mon[102692]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-10T00:39:00.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:59 smithi099 ceph-mon[102692]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-10T00:39:00.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:59 smithi099 ceph-mon[102692]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-09-10T00:39:00.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:59 smithi099 ceph-mon[102692]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-09-10T00:39:00.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:38:59 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1140030129' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-10T00:39:00.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.250+0000 7ff192059700 1 -- 172.21.15.44:0/2961733839 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7ff18c10abc0 con 0x7ff18c0ff4f0 2023-09-10T00:39:00.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:59 smithi131 ceph-mon[103713]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-09-10T00:39:00.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:59 smithi131 ceph-mon[103713]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-10T00:39:00.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:59 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/4135366904' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-09-10T00:39:00.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:59 smithi131 ceph-mon[103713]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-10T00:39:00.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:59 smithi131 ceph-mon[103713]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-10T00:39:00.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:59 smithi131 ceph-mon[103713]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-09-10T00:39:00.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:59 smithi131 ceph-mon[103713]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-09-10T00:39:00.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:38:59 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1140030129' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-10T00:39:00.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:59 smithi044 ceph-mon[98676]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-09-10T00:39:00.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:59 smithi044 ceph-mon[98676]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-10T00:39:00.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:59 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/4135366904' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-09-10T00:39:00.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:59 smithi044 ceph-mon[98676]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-10T00:39:00.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:59 smithi044 ceph-mon[98676]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-10T00:39:00.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:59 smithi044 ceph-mon[98676]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-09-10T00:39:00.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:59 smithi044 ceph-mon[98676]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-09-10T00:39:00.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:38:59 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1140030129' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-10T00:39:00.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.458+0000 7ff188ff9700 1 -- 172.21.15.44:0/2961733839 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff174009940 con 0x7ff18c0ff4f0 2023-09-10T00:39:00.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.460+0000 7ff192059700 1 -- 172.21.15.44:0/2961733839 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff1780a0640 msgr2=0x7ff1780a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:00.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.460+0000 7ff192059700 1 --2- 172.21.15.44:0/2961733839 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff1780a0640 0x7ff1780a2b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00c010 tx=0x7ff17c005fb0 comp rx=0 tx=0).stop 2023-09-10T00:39:00.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.460+0000 7ff192059700 1 -- 172.21.15.44:0/2961733839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff18c0ff4f0 msgr2=0x7ff18c195110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:00.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.460+0000 7ff192059700 1 --2- 172.21.15.44:0/2961733839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff18c0ff4f0 0x7ff18c195110 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff17400d980 tx=0x7ff17400dd40 comp rx=0 tx=0).stop 2023-09-10T00:39:00.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.461+0000 7ff192059700 1 -- 172.21.15.44:0/2961733839 shutdown_connections 2023-09-10T00:39:00.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.461+0000 7ff192059700 1 --2- 172.21.15.44:0/2961733839 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff1780a0640 0x7ff1780a2b00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:00.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.461+0000 7ff192059700 1 --2- 172.21.15.44:0/2961733839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff18c101080 0x7ff18c19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:00.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.461+0000 7ff192059700 1 --2- 172.21.15.44:0/2961733839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff18c0ff4f0 0x7ff18c195110 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:00.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.461+0000 7ff192059700 1 --2- 172.21.15.44:0/2961733839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff18c1006e0 0x7ff18c195650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:00.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.461+0000 7ff192059700 1 -- 172.21.15.44:0/2961733839 >> 172.21.15.44:0/2961733839 conn(0x7ff18c0faa60 msgr2=0x7ff18c0fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:00.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.461+0000 7ff192059700 1 -- 172.21.15.44:0/2961733839 shutdown_connections 2023-09-10T00:39:00.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.461+0000 7ff192059700 1 -- 172.21.15.44:0/2961733839 wait complete. 2023-09-10T00:39:00.478 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2023-09-10T00:39:00.479 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-09-10T00:39:00.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.610+0000 7ffb64e19700 1 -- 172.21.15.44:0/4240004019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb60100200 msgr2=0x7ffb60100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:00.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.610+0000 7ffb64e19700 1 --2- 172.21.15.44:0/4240004019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb60100200 0x7ffb60100680 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800b0a0 tx=0x7ffb5800b3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:00.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.610+0000 7ffb64e19700 1 -- 172.21.15.44:0/4240004019 shutdown_connections 2023-09-10T00:39:00.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.610+0000 7ffb64e19700 1 --2- 172.21.15.44:0/4240004019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb60100200 0x7ffb60100680 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:00.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.610+0000 7ffb64e19700 1 --2- 172.21.15.44:0/4240004019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb600feee0 0x7ffb600ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:00.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.610+0000 7ffb64e19700 1 --2- 172.21.15.44:0/4240004019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb600ff840 0x7ffb600ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:00.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.610+0000 7ffb64e19700 1 -- 172.21.15.44:0/4240004019 >> 172.21.15.44:0/4240004019 conn(0x7ffb600faa60 msgr2=0x7ffb600fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:00.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.610+0000 7ffb64e19700 1 -- 172.21.15.44:0/4240004019 shutdown_connections 2023-09-10T00:39:00.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.611+0000 7ffb64e19700 1 -- 172.21.15.44:0/4240004019 wait complete. 2023-09-10T00:39:00.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.611+0000 7ffb64e19700 1 Processor -- start 2023-09-10T00:39:00.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.611+0000 7ffb64e19700 1 -- start start 2023-09-10T00:39:00.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.611+0000 7ffb64e19700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb600feee0 0x7ffb60198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:00.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.611+0000 7ffb64e19700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb600ff840 0x7ffb60198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:00.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.611+0000 7ffb64e19700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb60100200 0x7ffb6019fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:00.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.611+0000 7ffb64e19700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb601a0080 con 0x7ffb600ff840 2023-09-10T00:39:00.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.611+0000 7ffb64e19700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb601a01c0 con 0x7ffb600feee0 2023-09-10T00:39:00.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.611+0000 7ffb64e19700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb601a0350 con 0x7ffb60100200 2023-09-10T00:39:00.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb5dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb600ff840 0x7ffb60198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:00.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb5dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb600ff840 0x7ffb60198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46746/0 (socket says 172.21.15.44:46746) 2023-09-10T00:39:00.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb5dd9b700 1 -- 172.21.15.44:0/2058760459 learned_addr learned my addr 172.21.15.44:0/2058760459 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:00.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb5e59c700 1 --2- 172.21.15.44:0/2058760459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb600feee0 0x7ffb60198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:00.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb5ed9d700 1 --2- 172.21.15.44:0/2058760459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb60100200 0x7ffb6019fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:00.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb5dd9b700 1 -- 172.21.15.44:0/2058760459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb60100200 msgr2=0x7ffb6019fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:00.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb5dd9b700 1 --2- 172.21.15.44:0/2058760459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb60100200 0x7ffb6019fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:00.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb5dd9b700 1 -- 172.21.15.44:0/2058760459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb600feee0 msgr2=0x7ffb60198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:00.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb5dd9b700 1 --2- 172.21.15.44:0/2058760459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb600feee0 0x7ffb60198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:00.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb5dd9b700 1 -- 172.21.15.44:0/2058760459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb58009d00 con 0x7ffb600ff840 2023-09-10T00:39:00.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb5ed9d700 1 --2- 172.21.15.44:0/2058760459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb60100200 0x7ffb6019fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:39:00.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb5dd9b700 1 --2- 172.21.15.44:0/2058760459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb600ff840 0x7ffb60198a40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800e9a0 tx=0x7ffb4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:00.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb577fe700 1 -- 172.21.15.44:0/2058760459 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48009790 con 0x7ffb600ff840 2023-09-10T00:39:00.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb577fe700 1 -- 172.21.15.44:0/2058760459 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb4800cbd0 con 0x7ffb600ff840 2023-09-10T00:39:00.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb64e19700 1 -- 172.21.15.44:0/2058760459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb601a0630 con 0x7ffb600ff840 2023-09-10T00:39:00.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb577fe700 1 -- 172.21.15.44:0/2058760459 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48018680 con 0x7ffb600ff840 2023-09-10T00:39:00.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.612+0000 7ffb64e19700 1 -- 172.21.15.44:0/2058760459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb601a0b40 con 0x7ffb600ff840 2023-09-10T00:39:00.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.613+0000 7ffb64e19700 1 -- 172.21.15.44:0/2058760459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb60109cc0 con 0x7ffb600ff840 2023-09-10T00:39:00.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.614+0000 7ffb577fe700 1 -- 172.21.15.44:0/2058760459 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ffb4800cd40 con 0x7ffb600ff840 2023-09-10T00:39:00.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.615+0000 7ffb577fe700 1 --2- 172.21.15.44:0/2058760459 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb4c0a0640 0x7ffb4c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:00.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.615+0000 7ffb5e59c700 1 --2- 172.21.15.44:0/2058760459 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb4c0a0640 0x7ffb4c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:00.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.615+0000 7ffb577fe700 1 -- 172.21.15.44:0/2058760459 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ffb48014070 con 0x7ffb600ff840 2023-09-10T00:39:00.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.615+0000 7ffb5e59c700 1 --2- 172.21.15.44:0/2058760459 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb4c0a0640 0x7ffb4c0a2b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009c60 tx=0x7ffb50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:00.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.619+0000 7ffb577fe700 1 -- 172.21.15.44:0/2058760459 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ffb48051e10 con 0x7ffb600ff840 2023-09-10T00:39:00.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:00.858+0000 7ffb64e19700 1 -- 172.21.15.44:0/2058760459 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb6002cf10 con 0x7ffb4c0a0640 2023-09-10T00:39:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:00 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1140030129' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-10T00:39:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:00 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] up:active 2023-09-10T00:39:01.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:00 smithi099 ceph-mon[102692]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-10T00:39:01.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:00 smithi099 ceph-mon[102692]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-09-10T00:39:01.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:00 smithi099 ceph-mon[102692]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-09-10T00:39:01.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:00 smithi099 ceph-mon[102692]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-09-10T00:39:01.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:00 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2961733839' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-10T00:39:01.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:00 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2961733839' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-10T00:39:01.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:00 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] up:active 2023-09-10T00:39:01.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:00 smithi099 ceph-mon[102692]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-09-10T00:39:01.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:00 smithi099 ceph-mon[102692]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-10T00:39:01.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:00 smithi099 ceph-mon[102692]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2023-09-10T00:39:01.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:00 smithi099 ceph-mon[102692]: daemon mds.k is now active in filesystem cephfs as rank 3 2023-09-10T00:39:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:00 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1140030129' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-10T00:39:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:00 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] up:active 2023-09-10T00:39:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:00 smithi131 ceph-mon[103713]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-10T00:39:01.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:00 smithi131 ceph-mon[103713]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-09-10T00:39:01.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:00 smithi131 ceph-mon[103713]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-09-10T00:39:01.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:00 smithi131 ceph-mon[103713]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-09-10T00:39:01.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:00 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2961733839' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-10T00:39:01.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:00 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2961733839' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-10T00:39:01.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:00 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] up:active 2023-09-10T00:39:01.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:00 smithi131 ceph-mon[103713]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-09-10T00:39:01.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:00 smithi131 ceph-mon[103713]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-10T00:39:01.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:00 smithi131 ceph-mon[103713]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2023-09-10T00:39:01.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:00 smithi131 ceph-mon[103713]: daemon mds.k is now active in filesystem cephfs as rank 3 2023-09-10T00:39:01.305 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:39:00.864+0000 7f14706d9700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:39:01.305 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:39:00.864+0000 7f14706d9700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:39:01.305 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:39:00.864+0000 7f14706d9700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:39:01.305 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:39:00.864+0000 7f14706d9700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:39:01.306 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:39:00.864+0000 7f14706d9700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:39:01.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1140030129' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-10T00:39:01.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] up:active 2023-09-10T00:39:01.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-mon[98676]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-10T00:39:01.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-mon[98676]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-09-10T00:39:01.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-mon[98676]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-09-10T00:39:01.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-mon[98676]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-09-10T00:39:01.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2961733839' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-10T00:39:01.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2961733839' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-10T00:39:01.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] up:active 2023-09-10T00:39:01.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-mon[98676]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-09-10T00:39:01.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-mon[98676]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-10T00:39:01.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-mon[98676]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2023-09-10T00:39:01.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:00 smithi044 ceph-mon[98676]: daemon mds.k is now active in filesystem cephfs as rank 3 2023-09-10T00:39:01.504 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.504+0000 7ffb577fe700 1 -- 172.21.15.44:0/2058760459 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ffb6002cf10 con 0x7ffb4c0a0640 2023-09-10T00:39:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.507+0000 7ffb64e19700 1 -- 172.21.15.44:0/2058760459 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb4c0a0640 msgr2=0x7ffb4c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.507+0000 7ffb64e19700 1 --2- 172.21.15.44:0/2058760459 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb4c0a0640 0x7ffb4c0a2b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009c60 tx=0x7ffb50009260 comp rx=0 tx=0).stop 2023-09-10T00:39:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.507+0000 7ffb64e19700 1 -- 172.21.15.44:0/2058760459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb600ff840 msgr2=0x7ffb60198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:01.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.507+0000 7ffb64e19700 1 --2- 172.21.15.44:0/2058760459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb600ff840 0x7ffb60198a40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800e9a0 tx=0x7ffb4800ecb0 comp rx=0 tx=0).stop 2023-09-10T00:39:01.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.507+0000 7ffb64e19700 1 -- 172.21.15.44:0/2058760459 shutdown_connections 2023-09-10T00:39:01.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.507+0000 7ffb64e19700 1 --2- 172.21.15.44:0/2058760459 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb4c0a0640 0x7ffb4c0a2b00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:01.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.507+0000 7ffb64e19700 1 --2- 172.21.15.44:0/2058760459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb60100200 0x7ffb6019fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:01.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.507+0000 7ffb64e19700 1 --2- 172.21.15.44:0/2058760459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb600feee0 0x7ffb60198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:01.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.507+0000 7ffb64e19700 1 --2- 172.21.15.44:0/2058760459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb600ff840 0x7ffb60198a40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:01.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.507+0000 7ffb64e19700 1 -- 172.21.15.44:0/2058760459 >> 172.21.15.44:0/2058760459 conn(0x7ffb600faa60 msgr2=0x7ffb600fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:01.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.507+0000 7ffb64e19700 1 -- 172.21.15.44:0/2058760459 shutdown_connections 2023-09-10T00:39:01.510 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.507+0000 7ffb64e19700 1 -- 172.21.15.44:0/2058760459 wait complete. 2023-09-10T00:39:01.522 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-09-10T00:39:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.659+0000 7f6ddde81700 1 -- 172.21.15.44:0/977334031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd81010b0 msgr2=0x7f6dd8107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.659+0000 7f6ddde81700 1 --2- 172.21.15.44:0/977334031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd81010b0 0x7f6dd8107960 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000b0d0 tx=0x7f6dd000b3e0 comp rx=0 tx=0).stop 2023-09-10T00:39:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.659+0000 7f6ddde81700 1 -- 172.21.15.44:0/977334031 shutdown_connections 2023-09-10T00:39:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.659+0000 7f6ddde81700 1 --2- 172.21.15.44:0/977334031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd81010b0 0x7f6dd8107960 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.659+0000 7f6ddde81700 1 --2- 172.21.15.44:0/977334031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dd80ff520 0x7f6dd80ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.659+0000 7f6ddde81700 1 --2- 172.21.15.44:0/977334031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dd8100710 0x7f6dd8100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:01.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.659+0000 7f6ddde81700 1 -- 172.21.15.44:0/977334031 >> 172.21.15.44:0/977334031 conn(0x7f6dd80faa90 msgr2=0x7f6dd80fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:01.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.659+0000 7f6ddde81700 1 -- 172.21.15.44:0/977334031 shutdown_connections 2023-09-10T00:39:01.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.659+0000 7f6ddde81700 1 -- 172.21.15.44:0/977334031 wait complete. 2023-09-10T00:39:01.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.659+0000 7f6ddde81700 1 Processor -- start 2023-09-10T00:39:01.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6ddde81700 1 -- start start 2023-09-10T00:39:01.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6ddde81700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dd80ff520 0x7f6dd81034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:01.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6ddde81700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd8100710 0x7f6dd8101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6ddde81700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dd81010b0 0x7f6dd8102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6ddde81700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd81027d0 con 0x7f6dd8100710 2023-09-10T00:39:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6ddde81700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd8102940 con 0x7f6dd80ff520 2023-09-10T00:39:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6ddde81700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd81a0370 con 0x7f6dd81010b0 2023-09-10T00:39:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6dd6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd8100710 0x7f6dd8101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6dd6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd8100710 0x7f6dd8101af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46778/0 (socket says 172.21.15.44:46778) 2023-09-10T00:39:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6dd6ffd700 1 -- 172.21.15.44:0/1257303780 learned_addr learned my addr 172.21.15.44:0/1257303780 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6dd77fe700 1 --2- 172.21.15.44:0/1257303780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dd80ff520 0x7f6dd81034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6dd7fff700 1 --2- 172.21.15.44:0/1257303780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dd81010b0 0x7f6dd8102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6dd6ffd700 1 -- 172.21.15.44:0/1257303780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dd81010b0 msgr2=0x7f6dd8102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6dd6ffd700 1 --2- 172.21.15.44:0/1257303780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dd81010b0 0x7f6dd8102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6dd6ffd700 1 -- 172.21.15.44:0/1257303780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dd80ff520 msgr2=0x7f6dd81034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6dd6ffd700 1 --2- 172.21.15.44:0/1257303780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dd80ff520 0x7f6dd81034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6dd6ffd700 1 -- 172.21.15.44:0/1257303780 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd0009d00 con 0x7f6dd8100710 2023-09-10T00:39:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6dd6ffd700 1 --2- 172.21.15.44:0/1257303780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd8100710 0x7f6dd8101af0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000d950 tx=0x7f6dc000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6dd4ff9700 1 -- 172.21.15.44:0/1257303780 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0009940 con 0x7f6dd8100710 2023-09-10T00:39:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.660+0000 7f6dd4ff9700 1 -- 172.21.15.44:0/1257303780 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6dc0010460 con 0x7f6dd8100710 2023-09-10T00:39:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.661+0000 7f6ddde81700 1 -- 172.21.15.44:0/1257303780 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd81a0650 con 0x7f6dd8100710 2023-09-10T00:39:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.661+0000 7f6dd4ff9700 1 -- 172.21.15.44:0/1257303780 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc000f620 con 0x7f6dd8100710 2023-09-10T00:39:01.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.661+0000 7f6ddde81700 1 -- 172.21.15.44:0/1257303780 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd81a0bb0 con 0x7f6dd8100710 2023-09-10T00:39:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.661+0000 7f6ddde81700 1 -- 172.21.15.44:0/1257303780 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd810aba0 con 0x7f6dd8100710 2023-09-10T00:39:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.663+0000 7f6dd4ff9700 1 -- 172.21.15.44:0/1257303780 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f6dc00105d0 con 0x7f6dd8100710 2023-09-10T00:39:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.663+0000 7f6dd4ff9700 1 --2- 172.21.15.44:0/1257303780 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6dc40a06c0 0x7f6dc40a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.663+0000 7f6dd4ff9700 1 -- 172.21.15.44:0/1257303780 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6dc00cbb20 con 0x7f6dd8100710 2023-09-10T00:39:01.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.663+0000 7f6dd77fe700 1 --2- 172.21.15.44:0/1257303780 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6dc40a06c0 0x7f6dc40a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.664+0000 7f6dd77fe700 1 --2- 172.21.15.44:0/1257303780 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6dc40a06c0 0x7f6dc40a2b80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8100570 tx=0x7f6dc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:01.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.667+0000 7f6dd4ff9700 1 -- 172.21.15.44:0/1257303780 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6dc0095460 con 0x7f6dd8100710 2023-09-10T00:39:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.904+0000 7f6ddde81700 1 -- 172.21.15.44:0/1257303780 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f6dd8068f00 con 0x7f6dc40a06c0 2023-09-10T00:39:01.910 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.909+0000 7f6dd4ff9700 1 -- 172.21.15.44:0/1257303780 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f6dd8068f00 con 0x7f6dc40a06c0 2023-09-10T00:39:01.910 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv_0/18ae2dc5-fa62-40a4-9c11-44763760865a 2023-09-10T00:39:01.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.911+0000 7f6ddde81700 1 -- 172.21.15.44:0/1257303780 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6dc40a06c0 msgr2=0x7f6dc40a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:01.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.912+0000 7f6ddde81700 1 --2- 172.21.15.44:0/1257303780 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6dc40a06c0 0x7f6dc40a2b80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8100570 tx=0x7f6dc8009260 comp rx=0 tx=0).stop 2023-09-10T00:39:01.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.912+0000 7f6ddde81700 1 -- 172.21.15.44:0/1257303780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd8100710 msgr2=0x7f6dd8101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:01.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.912+0000 7f6ddde81700 1 --2- 172.21.15.44:0/1257303780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd8100710 0x7f6dd8101af0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000d950 tx=0x7f6dc000dc60 comp rx=0 tx=0).stop 2023-09-10T00:39:01.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.912+0000 7f6ddde81700 1 -- 172.21.15.44:0/1257303780 shutdown_connections 2023-09-10T00:39:01.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.912+0000 7f6ddde81700 1 --2- 172.21.15.44:0/1257303780 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6dc40a06c0 0x7f6dc40a2b80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:01.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.912+0000 7f6ddde81700 1 --2- 172.21.15.44:0/1257303780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dd81010b0 0x7f6dd8102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:01.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.912+0000 7f6ddde81700 1 --2- 172.21.15.44:0/1257303780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dd80ff520 0x7f6dd81034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:01.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.912+0000 7f6ddde81700 1 --2- 172.21.15.44:0/1257303780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd8100710 0x7f6dd8101af0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:01.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.912+0000 7f6ddde81700 1 -- 172.21.15.44:0/1257303780 >> 172.21.15.44:0/1257303780 conn(0x7f6dd80faa90 msgr2=0x7f6dd80687f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:01.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.912+0000 7f6ddde81700 1 -- 172.21.15.44:0/1257303780 shutdown_connections 2023-09-10T00:39:01.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:01.912+0000 7f6ddde81700 1 -- 172.21.15.44:0/1257303780 wait complete. 2023-09-10T00:39:01.924 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-09-10T00:39:02.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.054+0000 7fd747992700 1 -- 172.21.15.44:0/364785560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd740100710 msgr2=0x7fd740100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:02.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.054+0000 7fd747992700 1 --2- 172.21.15.44:0/364785560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd740100710 0x7fd740100b70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd7300099c0 tx=0x7fd730009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.055+0000 7fd747992700 1 -- 172.21.15.44:0/364785560 shutdown_connections 2023-09-10T00:39:02.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.055+0000 7fd747992700 1 --2- 172.21.15.44:0/364785560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7401010b0 0x7fd740107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.055+0000 7fd747992700 1 --2- 172.21.15.44:0/364785560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7400ff520 0x7fd7400ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.055+0000 7fd747992700 1 --2- 172.21.15.44:0/364785560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd740100710 0x7fd740100b70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.055+0000 7fd747992700 1 -- 172.21.15.44:0/364785560 >> 172.21.15.44:0/364785560 conn(0x7fd7400faa90 msgr2=0x7fd7400fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:02.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.055+0000 7fd747992700 1 -- 172.21.15.44:0/364785560 shutdown_connections 2023-09-10T00:39:02.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.055+0000 7fd747992700 1 -- 172.21.15.44:0/364785560 wait complete. 2023-09-10T00:39:02.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.055+0000 7fd747992700 1 Processor -- start 2023-09-10T00:39:02.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.055+0000 7fd747992700 1 -- start start 2023-09-10T00:39:02.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.055+0000 7fd747992700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7400ff520 0x7fd7400692a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:02.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.055+0000 7fd747992700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd740100710 0x7fd74019af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:02.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.055+0000 7fd747992700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7401010b0 0x7fd74019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:02.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.055+0000 7fd747992700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74019dd40 con 0x7fd7401010b0 2023-09-10T00:39:02.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.055+0000 7fd747992700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74019deb0 con 0x7fd740100710 2023-09-10T00:39:02.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.055+0000 7fd747992700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74019e070 con 0x7fd7400ff520 2023-09-10T00:39:02.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd745f2f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7401010b0 0x7fd74019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:02.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd745f2f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7401010b0 0x7fd74019d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46794/0 (socket says 172.21.15.44:46794) 2023-09-10T00:39:02.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd745f2f700 1 -- 172.21.15.44:0/264305881 learned_addr learned my addr 172.21.15.44:0/264305881 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:02.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd744f2d700 1 --2- 172.21.15.44:0/264305881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd740100710 0x7fd74019af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:02.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd74572e700 1 --2- 172.21.15.44:0/264305881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7400ff520 0x7fd7400692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:02.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd745f2f700 1 -- 172.21.15.44:0/264305881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7400ff520 msgr2=0x7fd7400692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:02.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd745f2f700 1 --2- 172.21.15.44:0/264305881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7400ff520 0x7fd7400692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd745f2f700 1 -- 172.21.15.44:0/264305881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd740100710 msgr2=0x7fd74019af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:02.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd745f2f700 1 --2- 172.21.15.44:0/264305881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd740100710 0x7fd74019af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd745f2f700 1 -- 172.21.15.44:0/264305881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7300095f0 con 0x7fd7401010b0 2023-09-10T00:39:02.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd74572e700 1 --2- 172.21.15.44:0/264305881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7400ff520 0x7fd7400692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:02.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd745f2f700 1 --2- 172.21.15.44:0/264305881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7401010b0 0x7fd74019d5a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd738007ee0 tx=0x7fd73800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:02.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd7367fc700 1 -- 172.21.15.44:0/264305881 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738017070 con 0x7fd7401010b0 2023-09-10T00:39:02.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd7367fc700 1 -- 172.21.15.44:0/264305881 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd73800f040 con 0x7fd7401010b0 2023-09-10T00:39:02.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd747992700 1 -- 172.21.15.44:0/264305881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74019e350 con 0x7fd7401010b0 2023-09-10T00:39:02.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd7367fc700 1 -- 172.21.15.44:0/264305881 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738005710 con 0x7fd7401010b0 2023-09-10T00:39:02.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.056+0000 7fd747992700 1 -- 172.21.15.44:0/264305881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74019e860 con 0x7fd7401010b0 2023-09-10T00:39:02.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.057+0000 7fd747992700 1 -- 172.21.15.44:0/264305881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74010abf0 con 0x7fd7401010b0 2023-09-10T00:39:02.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.058+0000 7fd7367fc700 1 -- 172.21.15.44:0/264305881 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fd738005870 con 0x7fd7401010b0 2023-09-10T00:39:02.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.059+0000 7fd7367fc700 1 --2- 172.21.15.44:0/264305881 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd72c0a06c0 0x7fd72c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:02.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.059+0000 7fd7367fc700 1 -- 172.21.15.44:0/264305881 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd7380cbb70 con 0x7fd7401010b0 2023-09-10T00:39:02.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.059+0000 7fd74572e700 1 --2- 172.21.15.44:0/264305881 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd72c0a06c0 0x7fd72c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:02.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.059+0000 7fd74572e700 1 --2- 172.21.15.44:0/264305881 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd72c0a06c0 0x7fd72c0a2b80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd740100570 tx=0x7fd73c00b670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:02.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.062+0000 7fd7367fc700 1 -- 172.21.15.44:0/264305881 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd738095450 con 0x7fd7401010b0 2023-09-10T00:39:02.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:01 smithi099 ceph-mon[102692]: pgmap v279: 129 pgs: 129 active+clean; 582 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2023-09-10T00:39:02.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:01 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] up:active 2023-09-10T00:39:02.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:01 smithi099 ceph-mon[102692]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-09-10T00:39:02.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:01 smithi099 ceph-mon[102692]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-10T00:39:02.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:01 smithi099 ceph-mon[102692]: Cluster is now healthy 2023-09-10T00:39:02.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:01 smithi099 ceph-mon[102692]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2023-09-10T00:39:02.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:01 smithi099 ceph-mon[102692]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-09-10T00:39:02.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:01 smithi099 ceph-mon[102692]: daemon mds.l is now active in filesystem cephfs as rank 4 2023-09-10T00:39:02.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:01 smithi131 ceph-mon[103713]: pgmap v279: 129 pgs: 129 active+clean; 582 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2023-09-10T00:39:02.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:01 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] up:active 2023-09-10T00:39:02.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:01 smithi131 ceph-mon[103713]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-09-10T00:39:02.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:01 smithi131 ceph-mon[103713]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-10T00:39:02.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:01 smithi131 ceph-mon[103713]: Cluster is now healthy 2023-09-10T00:39:02.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:01 smithi131 ceph-mon[103713]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2023-09-10T00:39:02.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:01 smithi131 ceph-mon[103713]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-09-10T00:39:02.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:01 smithi131 ceph-mon[103713]: daemon mds.l is now active in filesystem cephfs as rank 4 2023-09-10T00:39:02.297 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.297+0000 7fd747992700 1 -- 172.21.15.44:0/264305881 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fd740060a40 con 0x7fd72c0a06c0 2023-09-10T00:39:02.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:01 smithi044 ceph-mon[98676]: pgmap v279: 129 pgs: 129 active+clean; 582 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2023-09-10T00:39:02.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:01 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] up:active 2023-09-10T00:39:02.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:01 smithi044 ceph-mon[98676]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-09-10T00:39:02.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:01 smithi044 ceph-mon[98676]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-10T00:39:02.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:01 smithi044 ceph-mon[98676]: Cluster is now healthy 2023-09-10T00:39:02.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:01 smithi044 ceph-mon[98676]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2023-09-10T00:39:02.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:01 smithi044 ceph-mon[98676]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-09-10T00:39:02.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:01 smithi044 ceph-mon[98676]: daemon mds.l is now active in filesystem cephfs as rank 4 2023-09-10T00:39:02.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.321+0000 7fd7367fc700 1 -- 172.21.15.44:0/264305881 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd740060a40 con 0x7fd72c0a06c0 2023-09-10T00:39:02.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.323+0000 7fd747992700 1 -- 172.21.15.44:0/264305881 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd72c0a06c0 msgr2=0x7fd72c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:02.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.323+0000 7fd747992700 1 --2- 172.21.15.44:0/264305881 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd72c0a06c0 0x7fd72c0a2b80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd740100570 tx=0x7fd73c00b670 comp rx=0 tx=0).stop 2023-09-10T00:39:02.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.323+0000 7fd747992700 1 -- 172.21.15.44:0/264305881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7401010b0 msgr2=0x7fd74019d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:02.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.323+0000 7fd747992700 1 --2- 172.21.15.44:0/264305881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7401010b0 0x7fd74019d5a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd738007ee0 tx=0x7fd73800d3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.324+0000 7fd747992700 1 -- 172.21.15.44:0/264305881 shutdown_connections 2023-09-10T00:39:02.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.324+0000 7fd747992700 1 --2- 172.21.15.44:0/264305881 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd72c0a06c0 0x7fd72c0a2b80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.324+0000 7fd747992700 1 --2- 172.21.15.44:0/264305881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7401010b0 0x7fd74019d5a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.324+0000 7fd747992700 1 --2- 172.21.15.44:0/264305881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7400ff520 0x7fd7400692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.324+0000 7fd747992700 1 --2- 172.21.15.44:0/264305881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd740100710 0x7fd74019af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.324+0000 7fd747992700 1 -- 172.21.15.44:0/264305881 >> 172.21.15.44:0/264305881 conn(0x7fd7400faa90 msgr2=0x7fd7400fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:02.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.324+0000 7fd747992700 1 -- 172.21.15.44:0/264305881 shutdown_connections 2023-09-10T00:39:02.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.324+0000 7fd747992700 1 -- 172.21.15.44:0/264305881 wait complete. 2023-09-10T00:39:02.336 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-09-10T00:39:02.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.465+0000 7f91c27ae700 1 -- 172.21.15.44:0/2684252983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc103a00 msgr2=0x7f91bc105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:02.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.465+0000 7f91c27ae700 1 --2- 172.21.15.44:0/2684252983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc103a00 0x7f91bc105df0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009990 tx=0x7f91b0009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.465+0000 7f91c27ae700 1 -- 172.21.15.44:0/2684252983 shutdown_connections 2023-09-10T00:39:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.465+0000 7f91c27ae700 1 --2- 172.21.15.44:0/2684252983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91bc106330 0x7f91bc108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.465+0000 7f91c27ae700 1 --2- 172.21.15.44:0/2684252983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91bc1010d0 0x7f91bc1034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.465+0000 7f91c27ae700 1 --2- 172.21.15.44:0/2684252983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc103a00 0x7f91bc105df0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.465+0000 7f91c27ae700 1 -- 172.21.15.44:0/2684252983 >> 172.21.15.44:0/2684252983 conn(0x7f91bc0faa60 msgr2=0x7f91bc0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:02.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.465+0000 7f91c27ae700 1 -- 172.21.15.44:0/2684252983 shutdown_connections 2023-09-10T00:39:02.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.465+0000 7f91c27ae700 1 -- 172.21.15.44:0/2684252983 wait complete. 2023-09-10T00:39:02.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.466+0000 7f91c27ae700 1 Processor -- start 2023-09-10T00:39:02.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.466+0000 7f91c27ae700 1 -- start start 2023-09-10T00:39:02.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.466+0000 7f91c27ae700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91bc1010d0 0x7f91bc100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:02.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.466+0000 7f91c27ae700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91bc103a00 0x7f91bc0ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:02.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.466+0000 7f91c27ae700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc106330 0x7f91bc0ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:02.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.466+0000 7f91c27ae700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0ffd60 con 0x7f91bc106330 2023-09-10T00:39:02.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.466+0000 7f91c27ae700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0ffed0 con 0x7f91bc1010d0 2023-09-10T00:39:02.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.466+0000 7f91c27ae700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc1a0280 con 0x7f91bc103a00 2023-09-10T00:39:02.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.466+0000 7f91c0d4b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc106330 0x7f91bc0ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:02.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.466+0000 7f91c0d4b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc106330 0x7f91bc0ff5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46810/0 (socket says 172.21.15.44:46810) 2023-09-10T00:39:02.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.466+0000 7f91c0d4b700 1 -- 172.21.15.44:0/3353260088 learned_addr learned my addr 172.21.15.44:0/3353260088 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:02.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.467+0000 7f91bbfff700 1 --2- 172.21.15.44:0/3353260088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91bc1010d0 0x7f91bc100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:02.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.467+0000 7f91bb7fe700 1 --2- 172.21.15.44:0/3353260088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91bc103a00 0x7f91bc0ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:02.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.467+0000 7f91c0d4b700 1 -- 172.21.15.44:0/3353260088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91bc103a00 msgr2=0x7f91bc0ff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:02.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.467+0000 7f91c0d4b700 1 --2- 172.21.15.44:0/3353260088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91bc103a00 0x7f91bc0ff080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.467+0000 7f91c0d4b700 1 -- 172.21.15.44:0/3353260088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91bc1010d0 msgr2=0x7f91bc100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:02.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.467+0000 7f91c0d4b700 1 --2- 172.21.15.44:0/3353260088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91bc1010d0 0x7f91bc100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.467+0000 7f91c0d4b700 1 -- 172.21.15.44:0/3353260088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b00095f0 con 0x7f91bc106330 2023-09-10T00:39:02.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.467+0000 7f91c0d4b700 1 --2- 172.21.15.44:0/3353260088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc106330 0x7f91bc0ff5c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f91b4007ee0 tx=0x7f91b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:02.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.467+0000 7f91b97fa700 1 -- 172.21.15.44:0/3353260088 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b4017070 con 0x7f91bc106330 2023-09-10T00:39:02.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.467+0000 7f91c27ae700 1 -- 172.21.15.44:0/3353260088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91bc1a0560 con 0x7f91bc106330 2023-09-10T00:39:02.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.467+0000 7f91b97fa700 1 -- 172.21.15.44:0/3353260088 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f91b400f040 con 0x7f91bc106330 2023-09-10T00:39:02.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.467+0000 7f91b97fa700 1 -- 172.21.15.44:0/3353260088 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b4005710 con 0x7f91bc106330 2023-09-10T00:39:02.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.467+0000 7f91c27ae700 1 -- 172.21.15.44:0/3353260088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91bc1a0a70 con 0x7f91bc106330 2023-09-10T00:39:02.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.468+0000 7f91c27ae700 1 -- 172.21.15.44:0/3353260088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91bc0fc5e0 con 0x7f91bc106330 2023-09-10T00:39:02.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.469+0000 7f91b97fa700 1 -- 172.21.15.44:0/3353260088 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f91b4005870 con 0x7f91bc106330 2023-09-10T00:39:02.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.470+0000 7f91b97fa700 1 --2- 172.21.15.44:0/3353260088 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f91a40a0640 0x7f91a40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:02.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.470+0000 7f91bbfff700 1 --2- 172.21.15.44:0/3353260088 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f91a40a0640 0x7f91a40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:02.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.470+0000 7f91b97fa700 1 -- 172.21.15.44:0/3353260088 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f91b40cb5b0 con 0x7f91bc106330 2023-09-10T00:39:02.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.470+0000 7f91bbfff700 1 --2- 172.21.15.44:0/3353260088 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f91a40a0640 0x7f91a40a2b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009c60 tx=0x7f91ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:02.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.474+0000 7f91b97fa700 1 -- 172.21.15.44:0/3353260088 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f91b4094ef0 con 0x7f91bc106330 2023-09-10T00:39:02.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.720+0000 7f91c27ae700 1 -- 172.21.15.44:0/3353260088 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f91bc02cf40 con 0x7f91a40a0640 2023-09-10T00:39:02.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.724+0000 7f91b97fa700 1 -- 172.21.15.44:0/3353260088 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f91bc02cf40 con 0x7f91a40a0640 2023-09-10T00:39:02.725 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv_1/2963f505-7e8a-43b6-a9da-ebc7a2f05636 2023-09-10T00:39:02.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.726+0000 7f91c27ae700 1 -- 172.21.15.44:0/3353260088 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f91a40a0640 msgr2=0x7f91a40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:02.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.726+0000 7f91c27ae700 1 --2- 172.21.15.44:0/3353260088 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f91a40a0640 0x7f91a40a2b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009c60 tx=0x7f91ac009260 comp rx=0 tx=0).stop 2023-09-10T00:39:02.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.726+0000 7f91c27ae700 1 -- 172.21.15.44:0/3353260088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc106330 msgr2=0x7f91bc0ff5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:02.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.726+0000 7f91c27ae700 1 --2- 172.21.15.44:0/3353260088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc106330 0x7f91bc0ff5c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f91b4007ee0 tx=0x7f91b400d3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.727+0000 7f91c27ae700 1 -- 172.21.15.44:0/3353260088 shutdown_connections 2023-09-10T00:39:02.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.727+0000 7f91c27ae700 1 --2- 172.21.15.44:0/3353260088 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f91a40a0640 0x7f91a40a2b00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.727+0000 7f91c27ae700 1 --2- 172.21.15.44:0/3353260088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc106330 0x7f91bc0ff5c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.727+0000 7f91c27ae700 1 --2- 172.21.15.44:0/3353260088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91bc1010d0 0x7f91bc100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.727+0000 7f91c27ae700 1 --2- 172.21.15.44:0/3353260088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91bc103a00 0x7f91bc0ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.727+0000 7f91c27ae700 1 -- 172.21.15.44:0/3353260088 >> 172.21.15.44:0/3353260088 conn(0x7f91bc0faa60 msgr2=0x7f91bc101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:02.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.727+0000 7f91c27ae700 1 -- 172.21.15.44:0/3353260088 shutdown_connections 2023-09-10T00:39:02.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.727+0000 7f91c27ae700 1 -- 172.21.15.44:0/3353260088 wait complete. 2023-09-10T00:39:02.739 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:39:02.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.872+0000 7fc20bad0700 1 -- 172.21.15.44:0/2348250765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2041006d0 msgr2=0x7fc204100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:02.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.872+0000 7fc20bad0700 1 --2- 172.21.15.44:0/2348250765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2041006d0 0x7fc204100b30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc2000099c0 tx=0x7fc200009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.872+0000 7fc20bad0700 1 -- 172.21.15.44:0/2348250765 shutdown_connections 2023-09-10T00:39:02.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.872+0000 7fc20bad0700 1 --2- 172.21.15.44:0/2348250765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc204101070 0x7fc204107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.872+0000 7fc20bad0700 1 --2- 172.21.15.44:0/2348250765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2040ff4e0 0x7fc2040ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.872+0000 7fc20bad0700 1 --2- 172.21.15.44:0/2348250765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2041006d0 0x7fc204100b30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.872+0000 7fc20bad0700 1 -- 172.21.15.44:0/2348250765 >> 172.21.15.44:0/2348250765 conn(0x7fc2040faa30 msgr2=0x7fc2040fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:02.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.872+0000 7fc20bad0700 1 -- 172.21.15.44:0/2348250765 shutdown_connections 2023-09-10T00:39:02.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.872+0000 7fc20bad0700 1 -- 172.21.15.44:0/2348250765 wait complete. 2023-09-10T00:39:02.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20bad0700 1 Processor -- start 2023-09-10T00:39:02.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20bad0700 1 -- start start 2023-09-10T00:39:02.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20bad0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2040ff4e0 0x7fc204196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:02.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20bad0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2041006d0 0x7fc2041966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:02.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20bad0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc204101070 0x7fc20419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:02.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20bad0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20419dd00 con 0x7fc2041006d0 2023-09-10T00:39:02.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20bad0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20419de40 con 0x7fc2040ff4e0 2023-09-10T00:39:02.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20bad0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20419dfd0 con 0x7fc204101070 2023-09-10T00:39:02.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20906b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2041006d0 0x7fc2041966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:02.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20906b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2041006d0 0x7fc2041966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46844/0 (socket says 172.21.15.44:46844) 2023-09-10T00:39:02.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20906b700 1 -- 172.21.15.44:0/797462926 learned_addr learned my addr 172.21.15.44:0/797462926 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:02.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20986c700 1 --2- 172.21.15.44:0/797462926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2040ff4e0 0x7fc204196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:02.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20a06d700 1 --2- 172.21.15.44:0/797462926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc204101070 0x7fc20419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:02.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20906b700 1 -- 172.21.15.44:0/797462926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc204101070 msgr2=0x7fc20419d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:02.881 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20906b700 1 --2- 172.21.15.44:0/797462926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc204101070 0x7fc20419d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.881 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20906b700 1 -- 172.21.15.44:0/797462926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2040ff4e0 msgr2=0x7fc204196180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:02.881 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20906b700 1 --2- 172.21.15.44:0/797462926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2040ff4e0 0x7fc204196180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:02.882 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20906b700 1 -- 172.21.15.44:0/797462926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2000095f0 con 0x7fc2041006d0 2023-09-10T00:39:02.882 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.873+0000 7fc20a06d700 1 --2- 172.21.15.44:0/797462926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc204101070 0x7fc20419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:02.882 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.874+0000 7fc20906b700 1 --2- 172.21.15.44:0/797462926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2041006d0 0x7fc2041966c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc200005910 tx=0x7fc200003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:02.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.874+0000 7fc1faffd700 1 -- 172.21.15.44:0/797462926 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20001d070 con 0x7fc2041006d0 2023-09-10T00:39:02.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.874+0000 7fc1faffd700 1 -- 172.21.15.44:0/797462926 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc20000bd30 con 0x7fc2041006d0 2023-09-10T00:39:02.884 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.874+0000 7fc20bad0700 1 -- 172.21.15.44:0/797462926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20419e250 con 0x7fc2041006d0 2023-09-10T00:39:02.884 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.874+0000 7fc1faffd700 1 -- 172.21.15.44:0/797462926 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200022d90 con 0x7fc2041006d0 2023-09-10T00:39:02.884 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.874+0000 7fc20bad0700 1 -- 172.21.15.44:0/797462926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20419e670 con 0x7fc2041006d0 2023-09-10T00:39:02.885 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.875+0000 7fc20bad0700 1 -- 172.21.15.44:0/797462926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20410abb0 con 0x7fc2041006d0 2023-09-10T00:39:02.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.876+0000 7fc1faffd700 1 -- 172.21.15.44:0/797462926 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fc200003fd0 con 0x7fc2041006d0 2023-09-10T00:39:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.876+0000 7fc1faffd700 1 --2- 172.21.15.44:0/797462926 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc1f00a0710 0x7fc1f00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.877+0000 7fc1faffd700 1 -- 172.21.15.44:0/797462926 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc2000cd420 con 0x7fc2041006d0 2023-09-10T00:39:02.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.877+0000 7fc20986c700 1 --2- 172.21.15.44:0/797462926 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc1f00a0710 0x7fc1f00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:02.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.877+0000 7fc20986c700 1 --2- 172.21.15.44:0/797462926 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc1f00a0710 0x7fc1f00a2bd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc204100530 tx=0x7fc1f400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:02.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:02.881+0000 7fc1faffd700 1 -- 172.21.15.44:0/797462926 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc200096e60 con 0x7fc2041006d0 2023-09-10T00:39:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:02 smithi044 ceph-mon[98676]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:39:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:02 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] up:active 2023-09-10T00:39:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:02 smithi044 ceph-mon[98676]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-09-10T00:39:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:02 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:39:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:02 smithi099 ceph-mon[102692]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:39:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:02 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] up:active 2023-09-10T00:39:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:02 smithi099 ceph-mon[102692]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-09-10T00:39:03.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:02 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:39:03.127 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.127+0000 7fc20bad0700 1 -- 172.21.15.44:0/797462926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc20410ada0 con 0x7fc2041006d0 2023-09-10T00:39:03.128 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.128+0000 7fc1faffd700 1 -- 172.21.15.44:0/797462926 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11369 (secure 0 0 0) 0x7fc200027090 con 0x7fc2041006d0 2023-09-10T00:39:03.129 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:03.129 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-10T00:38:39.183533+0000","modified":"2023-09-10T00:39:02.480227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24605,"mds_1":24574,"mds_2":24572,"mds_3":24515,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14751":{"gid":14751,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/3958270510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":3958270510},{"type":"v1","addr":"172.21.15.44:6839","nonce":3958270510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14763":{"gid":14763,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/1241002552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1241002552},{"type":"v1","addr":"172.21.15.44:6841","nonce":1241002552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24556":{"gid":24556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/3222652693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":3222652693},{"type":"v1","addr":"172.21.15.131:6839","nonce":3222652693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24605":{"gid":24605,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/2622966260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2622966260},{"type":"v1","addr":"172.21.15.99:6839","nonce":2622966260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-10T00:39:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.130+0000 7fc20bad0700 1 -- 172.21.15.44:0/797462926 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc1f00a0710 msgr2=0x7fc1f00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.130+0000 7fc20bad0700 1 --2- 172.21.15.44:0/797462926 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc1f00a0710 0x7fc1f00a2bd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc204100530 tx=0x7fc1f400b410 comp rx=0 tx=0).stop 2023-09-10T00:39:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.130+0000 7fc20bad0700 1 -- 172.21.15.44:0/797462926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2041006d0 msgr2=0x7fc2041966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.130+0000 7fc20bad0700 1 --2- 172.21.15.44:0/797462926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2041006d0 0x7fc2041966c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc200005910 tx=0x7fc200003a50 comp rx=0 tx=0).stop 2023-09-10T00:39:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.130+0000 7fc20bad0700 1 -- 172.21.15.44:0/797462926 shutdown_connections 2023-09-10T00:39:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.130+0000 7fc20bad0700 1 --2- 172.21.15.44:0/797462926 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fc1f00a0710 0x7fc1f00a2bd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.130+0000 7fc20bad0700 1 --2- 172.21.15.44:0/797462926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc204101070 0x7fc20419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.130+0000 7fc20bad0700 1 --2- 172.21.15.44:0/797462926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2040ff4e0 0x7fc204196180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.130+0000 7fc20bad0700 1 --2- 172.21.15.44:0/797462926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2041006d0 0x7fc2041966c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.130+0000 7fc20bad0700 1 -- 172.21.15.44:0/797462926 >> 172.21.15.44:0/797462926 conn(0x7fc2040faa30 msgr2=0x7fc2040fb350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:03.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.130+0000 7fc20bad0700 1 -- 172.21.15.44:0/797462926 shutdown_connections 2023-09-10T00:39:03.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.130+0000 7fc20bad0700 1 -- 172.21.15.44:0/797462926 wait complete. 2023-09-10T00:39:03.133 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2023-09-10T00:39:03.143 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:39:03.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.269+0000 7f3a81030700 1 -- 172.21.15.44:0/4008455156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c1010b0 msgr2=0x7f3a7c107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:03.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.269+0000 7f3a81030700 1 --2- 172.21.15.44:0/4008455156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c1010b0 0x7f3a7c107960 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400b0d0 tx=0x7f3a7400b3e0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.269+0000 7f3a81030700 1 -- 172.21.15.44:0/4008455156 shutdown_connections 2023-09-10T00:39:03.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.269+0000 7f3a81030700 1 --2- 172.21.15.44:0/4008455156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c1010b0 0x7f3a7c107960 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.269+0000 7f3a81030700 1 --2- 172.21.15.44:0/4008455156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a7c0ff520 0x7f3a7c0ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.269+0000 7f3a81030700 1 --2- 172.21.15.44:0/4008455156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c100710 0x7f3a7c100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.269+0000 7f3a81030700 1 -- 172.21.15.44:0/4008455156 >> 172.21.15.44:0/4008455156 conn(0x7f3a7c0faa90 msgr2=0x7f3a7c0fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:03.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.269+0000 7f3a81030700 1 -- 172.21.15.44:0/4008455156 shutdown_connections 2023-09-10T00:39:03.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.269+0000 7f3a81030700 1 -- 172.21.15.44:0/4008455156 wait complete. 2023-09-10T00:39:03.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.270+0000 7f3a81030700 1 Processor -- start 2023-09-10T00:39:03.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.270+0000 7f3a81030700 1 -- start start 2023-09-10T00:39:03.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.270+0000 7f3a81030700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a7c0ff520 0x7f3a7c1034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:03.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.270+0000 7f3a81030700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c100710 0x7f3a7c101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:03.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.270+0000 7f3a81030700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c1010b0 0x7f3a7c102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:03.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.270+0000 7f3a81030700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c1027d0 con 0x7f3a7c1010b0 2023-09-10T00:39:03.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.270+0000 7f3a81030700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c102940 con 0x7f3a7c100710 2023-09-10T00:39:03.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.270+0000 7f3a81030700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c1a02e0 con 0x7f3a7c0ff520 2023-09-10T00:39:03.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.270+0000 7f3a7b59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c1010b0 0x7f3a7c102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:03.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.270+0000 7f3a7b59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c1010b0 0x7f3a7c102030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46860/0 (socket says 172.21.15.44:46860) 2023-09-10T00:39:03.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.270+0000 7f3a7b59e700 1 -- 172.21.15.44:0/3200927266 learned_addr learned my addr 172.21.15.44:0/3200927266 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:03.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.270+0000 7f3a7a59c700 1 --2- 172.21.15.44:0/3200927266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c100710 0x7f3a7c101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:03.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.270+0000 7f3a7ad9d700 1 --2- 172.21.15.44:0/3200927266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a7c0ff520 0x7f3a7c1034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:03.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.271+0000 7f3a7b59e700 1 -- 172.21.15.44:0/3200927266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a7c0ff520 msgr2=0x7f3a7c1034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:03.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.271+0000 7f3a7b59e700 1 --2- 172.21.15.44:0/3200927266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a7c0ff520 0x7f3a7c1034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.271+0000 7f3a7b59e700 1 -- 172.21.15.44:0/3200927266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c100710 msgr2=0x7f3a7c101af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.271+0000 7f3a7b59e700 1 --2- 172.21.15.44:0/3200927266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c100710 0x7f3a7c101af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.271+0000 7f3a7b59e700 1 -- 172.21.15.44:0/3200927266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a74009d00 con 0x7f3a7c1010b0 2023-09-10T00:39:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.271+0000 7f3a7b59e700 1 --2- 172.21.15.44:0/3200927266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c1010b0 0x7f3a7c102030 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009fd0 tx=0x7f3a7400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:03.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.271+0000 7f3a73fff700 1 -- 172.21.15.44:0/3200927266 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7400e040 con 0x7f3a7c1010b0 2023-09-10T00:39:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.271+0000 7f3a73fff700 1 -- 172.21.15.44:0/3200927266 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a7400ce80 con 0x7f3a7c1010b0 2023-09-10T00:39:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.271+0000 7f3a81030700 1 -- 172.21.15.44:0/3200927266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c1a0560 con 0x7f3a7c1010b0 2023-09-10T00:39:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.271+0000 7f3a73fff700 1 -- 172.21.15.44:0/3200927266 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a740052a0 con 0x7f3a7c1010b0 2023-09-10T00:39:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.271+0000 7f3a81030700 1 -- 172.21.15.44:0/3200927266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c1a09e0 con 0x7f3a7c1010b0 2023-09-10T00:39:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.272+0000 7f3a81030700 1 -- 172.21.15.44:0/3200927266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7c10abe0 con 0x7f3a7c1010b0 2023-09-10T00:39:03.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.273+0000 7f3a73fff700 1 -- 172.21.15.44:0/3200927266 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f3a7401b450 con 0x7f3a7c1010b0 2023-09-10T00:39:03.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.274+0000 7f3a73fff700 1 --2- 172.21.15.44:0/3200927266 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3a680a0710 0x7f3a680a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:03.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.274+0000 7f3a7ad9d700 1 --2- 172.21.15.44:0/3200927266 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3a680a0710 0x7f3a680a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:03.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.274+0000 7f3a73fff700 1 -- 172.21.15.44:0/3200927266 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3a740cd450 con 0x7f3a7c1010b0 2023-09-10T00:39:03.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.274+0000 7f3a7ad9d700 1 --2- 172.21.15.44:0/3200927266 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3a680a0710 0x7f3a680a2bd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c100570 tx=0x7f3a6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:03.281 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.278+0000 7f3a73fff700 1 -- 172.21.15.44:0/3200927266 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3a74096cc0 con 0x7f3a7c1010b0 2023-09-10T00:39:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:02 smithi131 ceph-mon[103713]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:39:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:02 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] up:active 2023-09-10T00:39:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:02 smithi131 ceph-mon[103713]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-09-10T00:39:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:02 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:39:03.482 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.482+0000 7f3a81030700 1 -- 172.21.15.44:0/3200927266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a7c1a0c00 con 0x7f3a7c1010b0 2023-09-10T00:39:03.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.483+0000 7f3a73fff700 1 -- 172.21.15.44:0/3200927266 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23870 (secure 0 0 0) 0x7f3a7409a2e0 con 0x7f3a7c1010b0 2023-09-10T00:39:03.484 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:03.484 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":87,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:38:44.197902+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:39:03.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.486+0000 7f3a81030700 1 -- 172.21.15.44:0/3200927266 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3a680a0710 msgr2=0x7f3a680a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:03.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.486+0000 7f3a81030700 1 --2- 172.21.15.44:0/3200927266 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3a680a0710 0x7f3a680a2bd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c100570 tx=0x7f3a6c009260 comp rx=0 tx=0).stop 2023-09-10T00:39:03.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.486+0000 7f3a81030700 1 -- 172.21.15.44:0/3200927266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c1010b0 msgr2=0x7f3a7c102030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:03.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.486+0000 7f3a81030700 1 --2- 172.21.15.44:0/3200927266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c1010b0 0x7f3a7c102030 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009fd0 tx=0x7f3a7400cab0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.486+0000 7f3a81030700 1 -- 172.21.15.44:0/3200927266 shutdown_connections 2023-09-10T00:39:03.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.486+0000 7f3a81030700 1 --2- 172.21.15.44:0/3200927266 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f3a680a0710 0x7f3a680a2bd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.486+0000 7f3a81030700 1 --2- 172.21.15.44:0/3200927266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c1010b0 0x7f3a7c102030 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.486+0000 7f3a81030700 1 --2- 172.21.15.44:0/3200927266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a7c0ff520 0x7f3a7c1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.486+0000 7f3a81030700 1 --2- 172.21.15.44:0/3200927266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c100710 0x7f3a7c101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.486+0000 7f3a81030700 1 -- 172.21.15.44:0/3200927266 >> 172.21.15.44:0/3200927266 conn(0x7f3a7c0faa90 msgr2=0x7f3a7c068750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:03.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.486+0000 7f3a81030700 1 -- 172.21.15.44:0/3200927266 shutdown_connections 2023-09-10T00:39:03.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.486+0000 7f3a81030700 1 -- 172.21.15.44:0/3200927266 wait complete. 2023-09-10T00:39:03.499 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-10T00:39:03.500 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-10T00:39:03.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.625+0000 7ff33b17b700 1 -- 172.21.15.44:0/1834959573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff334100600 msgr2=0x7ff334100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:03.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.625+0000 7ff33b17b700 1 --2- 172.21.15.44:0/1834959573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff334100600 0x7ff334100a20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff3280099c0 tx=0x7ff328009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.625+0000 7ff33b17b700 1 -- 172.21.15.44:0/1834959573 shutdown_connections 2023-09-10T00:39:03.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.625+0000 7ff33b17b700 1 --2- 172.21.15.44:0/1834959573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff334102190 0x7ff334108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.625+0000 7ff33b17b700 1 --2- 172.21.15.44:0/1834959573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff334100600 0x7ff334100a20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.625+0000 7ff33b17b700 1 --2- 172.21.15.44:0/1834959573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3341017f0 0x7ff334101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.625+0000 7ff33b17b700 1 -- 172.21.15.44:0/1834959573 >> 172.21.15.44:0/1834959573 conn(0x7ff3340fbb50 msgr2=0x7ff3340fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.625+0000 7ff33b17b700 1 -- 172.21.15.44:0/1834959573 shutdown_connections 2023-09-10T00:39:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.625+0000 7ff33b17b700 1 -- 172.21.15.44:0/1834959573 wait complete. 2023-09-10T00:39:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff33b17b700 1 Processor -- start 2023-09-10T00:39:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff33b17b700 1 -- start start 2023-09-10T00:39:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff33b17b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff334100600 0x7ff3341961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:03.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff33b17b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3341017f0 0x7ff3341966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:03.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff33b17b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff334102190 0x7ff33419d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:03.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff33b17b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33419dd30 con 0x7ff334102190 2023-09-10T00:39:03.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff33b17b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33419de70 con 0x7ff3341017f0 2023-09-10T00:39:03.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff33b17b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33419e000 con 0x7ff334100600 2023-09-10T00:39:03.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff339718700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff334102190 0x7ff33419d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:03.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff339718700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff334102190 0x7ff33419d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46874/0 (socket says 172.21.15.44:46874) 2023-09-10T00:39:03.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff339718700 1 -- 172.21.15.44:0/3738453348 learned_addr learned my addr 172.21.15.44:0/3738453348 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:03.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff338f17700 1 --2- 172.21.15.44:0/3738453348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff334100600 0x7ff3341961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:03.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff333fff700 1 --2- 172.21.15.44:0/3738453348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3341017f0 0x7ff3341966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:03.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff339718700 1 -- 172.21.15.44:0/3738453348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff334100600 msgr2=0x7ff3341961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff339718700 1 --2- 172.21.15.44:0/3738453348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff334100600 0x7ff3341961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff339718700 1 -- 172.21.15.44:0/3738453348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3341017f0 msgr2=0x7ff3341966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff339718700 1 --2- 172.21.15.44:0/3738453348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3341017f0 0x7ff3341966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff339718700 1 -- 172.21.15.44:0/3738453348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3280095f0 con 0x7ff334102190 2023-09-10T00:39:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff333fff700 1 --2- 172.21.15.44:0/3738453348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3341017f0 0x7ff3341966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.626+0000 7ff339718700 1 --2- 172.21.15.44:0/3738453348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff334102190 0x7ff33419d760 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff32c007ee0 tx=0x7ff32c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.627+0000 7ff331ffb700 1 -- 172.21.15.44:0/3738453348 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c017070 con 0x7ff334102190 2023-09-10T00:39:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.627+0000 7ff331ffb700 1 -- 172.21.15.44:0/3738453348 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff32c00f040 con 0x7ff334102190 2023-09-10T00:39:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.627+0000 7ff33b17b700 1 -- 172.21.15.44:0/3738453348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33419e2e0 con 0x7ff334102190 2023-09-10T00:39:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.627+0000 7ff331ffb700 1 -- 172.21.15.44:0/3738453348 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c005710 con 0x7ff334102190 2023-09-10T00:39:03.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.627+0000 7ff33b17b700 1 -- 172.21.15.44:0/3738453348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33419e7f0 con 0x7ff334102190 2023-09-10T00:39:03.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.628+0000 7ff33b17b700 1 -- 172.21.15.44:0/3738453348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33410bd20 con 0x7ff334102190 2023-09-10T00:39:03.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.629+0000 7ff331ffb700 1 -- 172.21.15.44:0/3738453348 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7ff32c005870 con 0x7ff334102190 2023-09-10T00:39:03.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.629+0000 7ff331ffb700 1 --2- 172.21.15.44:0/3738453348 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff3240a0790 0x7ff3240a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:03.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.629+0000 7ff331ffb700 1 -- 172.21.15.44:0/3738453348 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff32c0cb6a0 con 0x7ff334102190 2023-09-10T00:39:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.629+0000 7ff338f17700 1 --2- 172.21.15.44:0/3738453348 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff3240a0790 0x7ff3240a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.630+0000 7ff338f17700 1 --2- 172.21.15.44:0/3738453348 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff3240a0790 0x7ff3240a2c50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff328000c00 tx=0x7ff328009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.634+0000 7ff331ffb700 1 -- 172.21.15.44:0/3738453348 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff32c094e80 con 0x7ff334102190 2023-09-10T00:39:03.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.836+0000 7ff33b17b700 1 -- 172.21.15.44:0/3738453348 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff334060a40 con 0x7ff3240a0790 2023-09-10T00:39:03.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.854+0000 7ff331ffb700 1 -- 172.21.15.44:0/3738453348 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+417957 (secure 0 0 0) 0x7ff334060a40 con 0x7ff3240a0790 2023-09-10T00:39:03.859 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:03.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.858+0000 7ff33b17b700 1 -- 172.21.15.44:0/3738453348 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff3240a0790 msgr2=0x7ff3240a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:03.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.858+0000 7ff33b17b700 1 --2- 172.21.15.44:0/3738453348 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff3240a0790 0x7ff3240a2c50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff328000c00 tx=0x7ff328009f10 comp rx=0 tx=0).stop 2023-09-10T00:39:03.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.858+0000 7ff33b17b700 1 -- 172.21.15.44:0/3738453348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff334102190 msgr2=0x7ff33419d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:03.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.858+0000 7ff33b17b700 1 --2- 172.21.15.44:0/3738453348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff334102190 0x7ff33419d760 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff32c007ee0 tx=0x7ff32c00d3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.858+0000 7ff33b17b700 1 -- 172.21.15.44:0/3738453348 shutdown_connections 2023-09-10T00:39:03.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.858+0000 7ff33b17b700 1 --2- 172.21.15.44:0/3738453348 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff3240a0790 0x7ff3240a2c50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.858+0000 7ff33b17b700 1 --2- 172.21.15.44:0/3738453348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff334102190 0x7ff33419d760 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.858+0000 7ff33b17b700 1 --2- 172.21.15.44:0/3738453348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff334100600 0x7ff3341961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.858+0000 7ff33b17b700 1 --2- 172.21.15.44:0/3738453348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3341017f0 0x7ff3341966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:03.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.858+0000 7ff33b17b700 1 -- 172.21.15.44:0/3738453348 >> 172.21.15.44:0/3738453348 conn(0x7ff3340fbb50 msgr2=0x7ff3340fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:03.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.858+0000 7ff33b17b700 1 -- 172.21.15.44:0/3738453348 shutdown_connections 2023-09-10T00:39:03.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:03.858+0000 7ff33b17b700 1 -- 172.21.15.44:0/3738453348 wait complete. 2023-09-10T00:39:03.865 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2023-09-10T00:39:03.871 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2023-09-10T00:39:01.998410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597193,"num_objects":49,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":138,"num_read_kb":121,"num_write":289,"num_write_kb":2965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337304,"kb_used_data":13096,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124604520,"statfs":{"total":1151940427776,"available":1151595028480,"internally_reserved":0,"allocated":13410304,"data_stored":7383726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4415,"num_objects":25,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":63,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003234"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.777120+0000","last_change":"2023-09-10T00:38:39.768218+0000","last_active":"2023-09-10T00:38:43.777120+0000","last_peered":"2023-09-10T00:38:43.777120+0000","last_clean":"2023-09-10T00:38:43.777120+0000","last_became_active":"2023-09-10T00:38:39.767454+0000","last_became_peered":"2023-09-10T00:38:39.767454+0000","last_unstale":"2023-09-10T00:38:43.777120+0000","last_undegraded":"2023-09-10T00:38:43.777120+0000","last_fullsized":"2023-09-10T00:38:43.777120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:17:30.287403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778165+0000","last_change":"2023-09-10T00:38:39.765384+0000","last_active":"2023-09-10T00:38:43.778165+0000","last_peered":"2023-09-10T00:38:43.778165+0000","last_clean":"2023-09-10T00:38:43.778165+0000","last_became_active":"2023-09-10T00:38:39.765090+0000","last_became_peered":"2023-09-10T00:38:39.765090+0000","last_unstale":"2023-09-10T00:38:43.778165+0000","last_undegraded":"2023-09-10T00:38:43.778165+0000","last_fullsized":"2023-09-10T00:38:43.778165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:02:46.025627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778228+0000","last_change":"2023-09-10T00:38:39.769222+0000","last_active":"2023-09-10T00:38:43.778228+0000","last_peered":"2023-09-10T00:38:43.778228+0000","last_clean":"2023-09-10T00:38:43.778228+0000","last_became_active":"2023-09-10T00:38:39.768801+0000","last_became_peered":"2023-09-10T00:38:39.768801+0000","last_unstale":"2023-09-10T00:38:43.778228+0000","last_undegraded":"2023-09-10T00:38:43.778228+0000","last_fullsized":"2023-09-10T00:38:43.778228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:38:53.394209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.777846+0000","last_change":"2023-09-10T00:38:39.764963+0000","last_active":"2023-09-10T00:38:43.777846+0000","last_peered":"2023-09-10T00:38:43.777846+0000","last_clean":"2023-09-10T00:38:43.777846+0000","last_became_active":"2023-09-10T00:38:39.764466+0000","last_became_peered":"2023-09-10T00:38:39.764466+0000","last_unstale":"2023-09-10T00:38:43.777846+0000","last_undegraded":"2023-09-10T00:38:43.777846+0000","last_fullsized":"2023-09-10T00:38:43.777846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:31.553321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.775719+0000","last_change":"2023-09-10T00:38:39.774933+0000","last_active":"2023-09-10T00:38:43.775719+0000","last_peered":"2023-09-10T00:38:43.775719+0000","last_clean":"2023-09-10T00:38:43.775719+0000","last_became_active":"2023-09-10T00:38:39.774606+0000","last_became_peered":"2023-09-10T00:38:39.774606+0000","last_unstale":"2023-09-10T00:38:43.775719+0000","last_undegraded":"2023-09-10T00:38:43.775719+0000","last_fullsized":"2023-09-10T00:38:43.775719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:31:58.902544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.837124+0000","last_change":"2023-09-10T00:38:39.761681+0000","last_active":"2023-09-10T00:38:43.837124+0000","last_peered":"2023-09-10T00:38:43.837124+0000","last_clean":"2023-09-10T00:38:43.837124+0000","last_became_active":"2023-09-10T00:38:39.761383+0000","last_became_peered":"2023-09-10T00:38:39.761383+0000","last_unstale":"2023-09-10T00:38:43.837124+0000","last_undegraded":"2023-09-10T00:38:43.837124+0000","last_fullsized":"2023-09-10T00:38:43.837124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:38:04.534656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772151+0000","last_change":"2023-09-10T00:38:39.765028+0000","last_active":"2023-09-10T00:38:43.772151+0000","last_peered":"2023-09-10T00:38:43.772151+0000","last_clean":"2023-09-10T00:38:43.772151+0000","last_became_active":"2023-09-10T00:38:39.764663+0000","last_became_peered":"2023-09-10T00:38:39.764663+0000","last_unstale":"2023-09-10T00:38:43.772151+0000","last_undegraded":"2023-09-10T00:38:43.772151+0000","last_fullsized":"2023-09-10T00:38:43.772151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:14:49.645129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.782249+0000","last_change":"2023-09-10T00:38:39.770135+0000","last_active":"2023-09-10T00:38:43.782249+0000","last_peered":"2023-09-10T00:38:43.782249+0000","last_clean":"2023-09-10T00:38:43.782249+0000","last_became_active":"2023-09-10T00:38:39.769720+0000","last_became_peered":"2023-09-10T00:38:39.769720+0000","last_unstale":"2023-09-10T00:38:43.782249+0000","last_undegraded":"2023-09-10T00:38:43.782249+0000","last_fullsized":"2023-09-10T00:38:43.782249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:46:21.166566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.779651+0000","last_change":"2023-09-10T00:38:39.770170+0000","last_active":"2023-09-10T00:38:43.779651+0000","last_peered":"2023-09-10T00:38:43.779651+0000","last_clean":"2023-09-10T00:38:43.779651+0000","last_became_active":"2023-09-10T00:38:39.769840+0000","last_became_peered":"2023-09-10T00:38:39.769840+0000","last_unstale":"2023-09-10T00:38:43.779651+0000","last_undegraded":"2023-09-10T00:38:43.779651+0000","last_fullsized":"2023-09-10T00:38:43.779651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:48.117670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.837797+0000","last_change":"2023-09-10T00:38:39.769621+0000","last_active":"2023-09-10T00:38:43.837797+0000","last_peered":"2023-09-10T00:38:43.837797+0000","last_clean":"2023-09-10T00:38:43.837797+0000","last_became_active":"2023-09-10T00:38:39.769300+0000","last_became_peered":"2023-09-10T00:38:39.769300+0000","last_unstale":"2023-09-10T00:38:43.837797+0000","last_undegraded":"2023-09-10T00:38:43.837797+0000","last_fullsized":"2023-09-10T00:38:43.837797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:37.100671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772431+0000","last_change":"2023-09-10T00:38:39.772200+0000","last_active":"2023-09-10T00:38:43.772431+0000","last_peered":"2023-09-10T00:38:43.772431+0000","last_clean":"2023-09-10T00:38:43.772431+0000","last_became_active":"2023-09-10T00:38:39.771503+0000","last_became_peered":"2023-09-10T00:38:39.771503+0000","last_unstale":"2023-09-10T00:38:43.772431+0000","last_undegraded":"2023-09-10T00:38:43.772431+0000","last_fullsized":"2023-09-10T00:38:43.772431+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:13:38.830642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.777810+0000","last_change":"2023-09-10T00:38:39.769832+0000","last_active":"2023-09-10T00:38:43.777810+0000","last_peered":"2023-09-10T00:38:43.777810+0000","last_clean":"2023-09-10T00:38:43.777810+0000","last_became_active":"2023-09-10T00:38:39.769546+0000","last_became_peered":"2023-09-10T00:38:39.769546+0000","last_unstale":"2023-09-10T00:38:43.777810+0000","last_undegraded":"2023-09-10T00:38:43.777810+0000","last_fullsized":"2023-09-10T00:38:43.777810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:53:05.219877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.774568+0000","last_change":"2023-09-10T00:38:39.766509+0000","last_active":"2023-09-10T00:38:43.774568+0000","last_peered":"2023-09-10T00:38:43.774568+0000","last_clean":"2023-09-10T00:38:43.774568+0000","last_became_active":"2023-09-10T00:38:39.766017+0000","last_became_peered":"2023-09-10T00:38:39.766017+0000","last_unstale":"2023-09-10T00:38:43.774568+0000","last_undegraded":"2023-09-10T00:38:43.774568+0000","last_fullsized":"2023-09-10T00:38:43.774568+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:53:05.246343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.777959+0000","last_change":"2023-09-10T00:38:39.764751+0000","last_active":"2023-09-10T00:38:43.777959+0000","last_peered":"2023-09-10T00:38:43.777959+0000","last_clean":"2023-09-10T00:38:43.777959+0000","last_became_active":"2023-09-10T00:38:39.764464+0000","last_became_peered":"2023-09-10T00:38:39.764464+0000","last_unstale":"2023-09-10T00:38:43.777959+0000","last_undegraded":"2023-09-10T00:38:43.777959+0000","last_fullsized":"2023-09-10T00:38:43.777959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:03:51.096126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778426+0000","last_change":"2023-09-10T00:38:39.777487+0000","last_active":"2023-09-10T00:38:43.778426+0000","last_peered":"2023-09-10T00:38:43.778426+0000","last_clean":"2023-09-10T00:38:43.778426+0000","last_became_active":"2023-09-10T00:38:39.777134+0000","last_became_peered":"2023-09-10T00:38:39.777134+0000","last_unstale":"2023-09-10T00:38:43.778426+0000","last_undegraded":"2023-09-10T00:38:43.778426+0000","last_fullsized":"2023-09-10T00:38:43.778426+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:17:55.356440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778881+0000","last_change":"2023-09-10T00:38:39.770787+0000","last_active":"2023-09-10T00:38:43.778881+0000","last_peered":"2023-09-10T00:38:43.778881+0000","last_clean":"2023-09-10T00:38:43.778881+0000","last_became_active":"2023-09-10T00:38:39.770497+0000","last_became_peered":"2023-09-10T00:38:39.770497+0000","last_unstale":"2023-09-10T00:38:43.778881+0000","last_undegraded":"2023-09-10T00:38:43.778881+0000","last_fullsized":"2023-09-10T00:38:43.778881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:43:11.066766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.776880+0000","last_change":"2023-09-10T00:38:39.768008+0000","last_active":"2023-09-10T00:38:43.776880+0000","last_peered":"2023-09-10T00:38:43.776880+0000","last_clean":"2023-09-10T00:38:43.776880+0000","last_became_active":"2023-09-10T00:38:39.767079+0000","last_became_peered":"2023-09-10T00:38:39.767079+0000","last_unstale":"2023-09-10T00:38:43.776880+0000","last_undegraded":"2023-09-10T00:38:43.776880+0000","last_fullsized":"2023-09-10T00:38:43.776880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:34:00.689881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.769405+0000","last_change":"2023-09-10T00:38:39.775326+0000","last_active":"2023-09-10T00:38:43.769405+0000","last_peered":"2023-09-10T00:38:43.769405+0000","last_clean":"2023-09-10T00:38:43.769405+0000","last_became_active":"2023-09-10T00:38:39.775007+0000","last_became_peered":"2023-09-10T00:38:39.775007+0000","last_unstale":"2023-09-10T00:38:43.769405+0000","last_undegraded":"2023-09-10T00:38:43.769405+0000","last_fullsized":"2023-09-10T00:38:43.769405+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:59:27.711722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.780709+0000","last_change":"2023-09-10T00:38:39.764974+0000","last_active":"2023-09-10T00:38:43.780709+0000","last_peered":"2023-09-10T00:38:43.780709+0000","last_clean":"2023-09-10T00:38:43.780709+0000","last_became_active":"2023-09-10T00:38:39.764368+0000","last_became_peered":"2023-09-10T00:38:39.764368+0000","last_unstale":"2023-09-10T00:38:43.780709+0000","last_undegraded":"2023-09-10T00:38:43.780709+0000","last_fullsized":"2023-09-10T00:38:43.780709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:40:28.046021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253622+0000","last_change":"2023-09-10T00:38:43.793589+0000","last_active":"2023-09-10T00:38:46.253622+0000","last_peered":"2023-09-10T00:38:46.253622+0000","last_clean":"2023-09-10T00:38:46.253622+0000","last_became_active":"2023-09-10T00:38:43.793166+0000","last_became_peered":"2023-09-10T00:38:43.793166+0000","last_unstale":"2023-09-10T00:38:46.253622+0000","last_undegraded":"2023-09-10T00:38:46.253622+0000","last_fullsized":"2023-09-10T00:38:46.253622+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:51:47.128925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:00.475366+0000","last_change":"2023-09-10T00:38:38.759909+0000","last_active":"2023-09-10T00:39:00.475366+0000","last_peered":"2023-09-10T00:39:00.475366+0000","last_clean":"2023-09-10T00:39:00.475366+0000","last_became_active":"2023-09-10T00:38:38.759561+0000","last_became_peered":"2023-09-10T00:38:38.759561+0000","last_unstale":"2023-09-10T00:39:00.475366+0000","last_undegraded":"2023-09-10T00:39:00.475366+0000","last_fullsized":"2023-09-10T00:39:00.475366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:56:18.009497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.783633+0000","last_change":"2023-09-10T00:38:39.774916+0000","last_active":"2023-09-10T00:38:43.783633+0000","last_peered":"2023-09-10T00:38:43.783633+0000","last_clean":"2023-09-10T00:38:43.783633+0000","last_became_active":"2023-09-10T00:38:39.774584+0000","last_became_peered":"2023-09-10T00:38:39.774584+0000","last_unstale":"2023-09-10T00:38:43.783633+0000","last_undegraded":"2023-09-10T00:38:43.783633+0000","last_fullsized":"2023-09-10T00:38:43.783633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:36:33.125742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253516+0000","last_change":"2023-09-10T00:38:43.789421+0000","last_active":"2023-09-10T00:38:46.253516+0000","last_peered":"2023-09-10T00:38:46.253516+0000","last_clean":"2023-09-10T00:38:46.253516+0000","last_became_active":"2023-09-10T00:38:43.788569+0000","last_became_peered":"2023-09-10T00:38:43.788569+0000","last_unstale":"2023-09-10T00:38:46.253516+0000","last_undegraded":"2023-09-10T00:38:46.253516+0000","last_fullsized":"2023-09-10T00:38:46.253516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:07:38.529811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.254348+0000","last_change":"2023-09-10T00:38:38.767381+0000","last_active":"2023-09-10T00:38:46.254348+0000","last_peered":"2023-09-10T00:38:46.254348+0000","last_clean":"2023-09-10T00:38:46.254348+0000","last_became_active":"2023-09-10T00:38:38.767082+0000","last_became_peered":"2023-09-10T00:38:38.767082+0000","last_unstale":"2023-09-10T00:38:46.254348+0000","last_undegraded":"2023-09-10T00:38:46.254348+0000","last_fullsized":"2023-09-10T00:38:46.254348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:13:32.533215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.779079+0000","last_change":"2023-09-10T00:38:39.772645+0000","last_active":"2023-09-10T00:38:43.779079+0000","last_peered":"2023-09-10T00:38:43.779079+0000","last_clean":"2023-09-10T00:38:43.779079+0000","last_became_active":"2023-09-10T00:38:39.771554+0000","last_became_peered":"2023-09-10T00:38:39.771554+0000","last_unstale":"2023-09-10T00:38:43.779079+0000","last_undegraded":"2023-09-10T00:38:43.779079+0000","last_fullsized":"2023-09-10T00:38:43.779079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:25:06.237309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253732+0000","last_change":"2023-09-10T00:38:43.795764+0000","last_active":"2023-09-10T00:38:46.253732+0000","last_peered":"2023-09-10T00:38:46.253732+0000","last_clean":"2023-09-10T00:38:46.253732+0000","last_became_active":"2023-09-10T00:38:43.795112+0000","last_became_peered":"2023-09-10T00:38:43.795112+0000","last_unstale":"2023-09-10T00:38:46.253732+0000","last_undegraded":"2023-09-10T00:38:46.253732+0000","last_fullsized":"2023-09-10T00:38:46.253732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:18:55.460752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.776802+0000","last_change":"2023-09-10T00:38:38.765522+0000","last_active":"2023-09-10T00:38:43.776802+0000","last_peered":"2023-09-10T00:38:43.776802+0000","last_clean":"2023-09-10T00:38:43.776802+0000","last_became_active":"2023-09-10T00:38:38.765219+0000","last_became_peered":"2023-09-10T00:38:38.765219+0000","last_unstale":"2023-09-10T00:38:43.776802+0000","last_undegraded":"2023-09-10T00:38:43.776802+0000","last_fullsized":"2023-09-10T00:38:43.776802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:00:27.275269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.780883+0000","last_change":"2023-09-10T00:38:39.767790+0000","last_active":"2023-09-10T00:38:43.780883+0000","last_peered":"2023-09-10T00:38:43.780883+0000","last_clean":"2023-09-10T00:38:43.780883+0000","last_became_active":"2023-09-10T00:38:39.767513+0000","last_became_peered":"2023-09-10T00:38:39.767513+0000","last_unstale":"2023-09-10T00:38:43.780883+0000","last_undegraded":"2023-09-10T00:38:43.780883+0000","last_fullsized":"2023-09-10T00:38:43.780883+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:05:48.223951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253610+0000","last_change":"2023-09-10T00:38:43.789089+0000","last_active":"2023-09-10T00:38:46.253610+0000","last_peered":"2023-09-10T00:38:46.253610+0000","last_clean":"2023-09-10T00:38:46.253610+0000","last_became_active":"2023-09-10T00:38:43.788434+0000","last_became_peered":"2023-09-10T00:38:43.788434+0000","last_unstale":"2023-09-10T00:38:46.253610+0000","last_undegraded":"2023-09-10T00:38:46.253610+0000","last_fullsized":"2023-09-10T00:38:46.253610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:55:00.236088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.777343+0000","last_change":"2023-09-10T00:38:38.769478+0000","last_active":"2023-09-10T00:38:43.777343+0000","last_peered":"2023-09-10T00:38:43.777343+0000","last_clean":"2023-09-10T00:38:43.777343+0000","last_became_active":"2023-09-10T00:38:38.769168+0000","last_became_peered":"2023-09-10T00:38:38.769168+0000","last_unstale":"2023-09-10T00:38:43.777343+0000","last_undegraded":"2023-09-10T00:38:43.777343+0000","last_fullsized":"2023-09-10T00:38:43.777343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:03:48.133561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.773628+0000","last_change":"2023-09-10T00:38:39.771843+0000","last_active":"2023-09-10T00:38:43.773628+0000","last_peered":"2023-09-10T00:38:43.773628+0000","last_clean":"2023-09-10T00:38:43.773628+0000","last_became_active":"2023-09-10T00:38:39.771510+0000","last_became_peered":"2023-09-10T00:38:39.771510+0000","last_unstale":"2023-09-10T00:38:43.773628+0000","last_undegraded":"2023-09-10T00:38:43.773628+0000","last_fullsized":"2023-09-10T00:38:43.773628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:17:39.996490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.254062+0000","last_change":"2023-09-10T00:38:43.786853+0000","last_active":"2023-09-10T00:38:46.254062+0000","last_peered":"2023-09-10T00:38:46.254062+0000","last_clean":"2023-09-10T00:38:46.254062+0000","last_became_active":"2023-09-10T00:38:43.786386+0000","last_became_peered":"2023-09-10T00:38:43.786386+0000","last_unstale":"2023-09-10T00:38:46.254062+0000","last_undegraded":"2023-09-10T00:38:46.254062+0000","last_fullsized":"2023-09-10T00:38:46.254062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:23:53.724653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:58.843879+0000","last_change":"2023-09-10T00:38:38.769004+0000","last_active":"2023-09-10T00:38:58.843879+0000","last_peered":"2023-09-10T00:38:58.843879+0000","last_clean":"2023-09-10T00:38:58.843879+0000","last_became_active":"2023-09-10T00:38:38.768690+0000","last_became_peered":"2023-09-10T00:38:38.768690+0000","last_unstale":"2023-09-10T00:38:58.843879+0000","last_undegraded":"2023-09-10T00:38:58.843879+0000","last_fullsized":"2023-09-10T00:38:58.843879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:04:42.531092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.777391+0000","last_change":"2023-09-10T00:38:39.773174+0000","last_active":"2023-09-10T00:38:43.777391+0000","last_peered":"2023-09-10T00:38:43.777391+0000","last_clean":"2023-09-10T00:38:43.777391+0000","last_became_active":"2023-09-10T00:38:39.771192+0000","last_became_peered":"2023-09-10T00:38:39.771192+0000","last_unstale":"2023-09-10T00:38:43.777391+0000","last_undegraded":"2023-09-10T00:38:43.777391+0000","last_fullsized":"2023-09-10T00:38:43.777391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:16:25.034024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253678+0000","last_change":"2023-09-10T00:38:43.786661+0000","last_active":"2023-09-10T00:38:46.253678+0000","last_peered":"2023-09-10T00:38:46.253678+0000","last_clean":"2023-09-10T00:38:46.253678+0000","last_became_active":"2023-09-10T00:38:43.786242+0000","last_became_peered":"2023-09-10T00:38:43.786242+0000","last_unstale":"2023-09-10T00:38:46.253678+0000","last_undegraded":"2023-09-10T00:38:46.253678+0000","last_fullsized":"2023-09-10T00:38:46.253678+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:36:14.490085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.837126+0000","last_change":"2023-09-10T00:38:38.763416+0000","last_active":"2023-09-10T00:38:43.837126+0000","last_peered":"2023-09-10T00:38:43.837126+0000","last_clean":"2023-09-10T00:38:43.837126+0000","last_became_active":"2023-09-10T00:38:38.763054+0000","last_became_peered":"2023-09-10T00:38:38.763054+0000","last_unstale":"2023-09-10T00:38:43.837126+0000","last_undegraded":"2023-09-10T00:38:43.837126+0000","last_fullsized":"2023-09-10T00:38:43.837126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:47:00.179467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.773330+0000","last_change":"2023-09-10T00:38:39.773284+0000","last_active":"2023-09-10T00:38:43.773330+0000","last_peered":"2023-09-10T00:38:43.773330+0000","last_clean":"2023-09-10T00:38:43.773330+0000","last_became_active":"2023-09-10T00:38:39.772969+0000","last_became_peered":"2023-09-10T00:38:39.772969+0000","last_unstale":"2023-09-10T00:38:43.773330+0000","last_undegraded":"2023-09-10T00:38:43.773330+0000","last_fullsized":"2023-09-10T00:38:43.773330+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:46:45.013879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.254229+0000","last_change":"2023-09-10T00:38:43.790810+0000","last_active":"2023-09-10T00:38:46.254229+0000","last_peered":"2023-09-10T00:38:46.254229+0000","last_clean":"2023-09-10T00:38:46.254229+0000","last_became_active":"2023-09-10T00:38:43.790429+0000","last_became_peered":"2023-09-10T00:38:43.790429+0000","last_unstale":"2023-09-10T00:38:46.254229+0000","last_undegraded":"2023-09-10T00:38:46.254229+0000","last_fullsized":"2023-09-10T00:38:46.254229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:30:56.072587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:59.851971+0000","last_change":"2023-09-10T00:38:38.754326+0000","last_active":"2023-09-10T00:38:59.851971+0000","last_peered":"2023-09-10T00:38:59.851971+0000","last_clean":"2023-09-10T00:38:59.851971+0000","last_became_active":"2023-09-10T00:38:38.754006+0000","last_became_peered":"2023-09-10T00:38:38.754006+0000","last_unstale":"2023-09-10T00:38:59.851971+0000","last_undegraded":"2023-09-10T00:38:59.851971+0000","last_fullsized":"2023-09-10T00:38:59.851971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:09:26.983606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.782137+0000","last_change":"2023-09-10T00:38:39.769699+0000","last_active":"2023-09-10T00:38:43.782137+0000","last_peered":"2023-09-10T00:38:43.782137+0000","last_clean":"2023-09-10T00:38:43.782137+0000","last_became_active":"2023-09-10T00:38:39.769362+0000","last_became_peered":"2023-09-10T00:38:39.769362+0000","last_unstale":"2023-09-10T00:38:43.782137+0000","last_undegraded":"2023-09-10T00:38:43.782137+0000","last_fullsized":"2023-09-10T00:38:43.782137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:30:20.938437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:45.752807+0000","last_change":"2023-09-10T00:38:43.785669+0000","last_active":"2023-09-10T00:38:45.752807+0000","last_peered":"2023-09-10T00:38:45.752807+0000","last_clean":"2023-09-10T00:38:45.752807+0000","last_became_active":"2023-09-10T00:38:43.785255+0000","last_became_peered":"2023-09-10T00:38:43.785255+0000","last_unstale":"2023-09-10T00:38:45.752807+0000","last_undegraded":"2023-09-10T00:38:45.752807+0000","last_fullsized":"2023-09-10T00:38:45.752807+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:52:57.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.066564+0000","last_change":"2023-09-10T00:38:38.767997+0000","last_active":"2023-09-10T00:38:46.066564+0000","last_peered":"2023-09-10T00:38:46.066564+0000","last_clean":"2023-09-10T00:38:46.066564+0000","last_became_active":"2023-09-10T00:38:38.767715+0000","last_became_peered":"2023-09-10T00:38:38.767715+0000","last_unstale":"2023-09-10T00:38:46.066564+0000","last_undegraded":"2023-09-10T00:38:46.066564+0000","last_fullsized":"2023-09-10T00:38:46.066564+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:11:17.047280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.769273+0000","last_change":"2023-09-10T00:38:39.778004+0000","last_active":"2023-09-10T00:38:43.769273+0000","last_peered":"2023-09-10T00:38:43.769273+0000","last_clean":"2023-09-10T00:38:43.769273+0000","last_became_active":"2023-09-10T00:38:39.777632+0000","last_became_peered":"2023-09-10T00:38:39.777632+0000","last_unstale":"2023-09-10T00:38:43.769273+0000","last_undegraded":"2023-09-10T00:38:43.769273+0000","last_fullsized":"2023-09-10T00:38:43.769273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:46:40.804385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253113+0000","last_change":"2023-09-10T00:38:43.791397+0000","last_active":"2023-09-10T00:38:46.253113+0000","last_peered":"2023-09-10T00:38:46.253113+0000","last_clean":"2023-09-10T00:38:46.253113+0000","last_became_active":"2023-09-10T00:38:43.791010+0000","last_became_peered":"2023-09-10T00:38:43.791010+0000","last_unstale":"2023-09-10T00:38:46.253113+0000","last_undegraded":"2023-09-10T00:38:46.253113+0000","last_fullsized":"2023-09-10T00:38:46.253113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:52:26.015718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:58.841224+0000","last_change":"2023-09-10T00:38:38.767399+0000","last_active":"2023-09-10T00:38:58.841224+0000","last_peered":"2023-09-10T00:38:58.841224+0000","last_clean":"2023-09-10T00:38:58.841224+0000","last_became_active":"2023-09-10T00:38:38.767176+0000","last_became_peered":"2023-09-10T00:38:38.767176+0000","last_unstale":"2023-09-10T00:38:58.841224+0000","last_undegraded":"2023-09-10T00:38:58.841224+0000","last_fullsized":"2023-09-10T00:38:58.841224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:17:24.481985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.779324+0000","last_change":"2023-09-10T00:38:39.766013+0000","last_active":"2023-09-10T00:38:43.779324+0000","last_peered":"2023-09-10T00:38:43.779324+0000","last_clean":"2023-09-10T00:38:43.779324+0000","last_became_active":"2023-09-10T00:38:39.765235+0000","last_became_peered":"2023-09-10T00:38:39.765235+0000","last_unstale":"2023-09-10T00:38:43.779324+0000","last_undegraded":"2023-09-10T00:38:43.779324+0000","last_fullsized":"2023-09-10T00:38:43.779324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:01:48.874845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.252793+0000","last_change":"2023-09-10T00:38:43.790361+0000","last_active":"2023-09-10T00:38:46.252793+0000","last_peered":"2023-09-10T00:38:46.252793+0000","last_clean":"2023-09-10T00:38:46.252793+0000","last_became_active":"2023-09-10T00:38:43.789966+0000","last_became_peered":"2023-09-10T00:38:43.789966+0000","last_unstale":"2023-09-10T00:38:46.252793+0000","last_undegraded":"2023-09-10T00:38:46.252793+0000","last_fullsized":"2023-09-10T00:38:46.252793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:46:37.978258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:01.482417+0000","last_change":"2023-09-10T00:38:38.771670+0000","last_active":"2023-09-10T00:39:01.482417+0000","last_peered":"2023-09-10T00:39:01.482417+0000","last_clean":"2023-09-10T00:39:01.482417+0000","last_became_active":"2023-09-10T00:38:38.771339+0000","last_became_peered":"2023-09-10T00:38:38.771339+0000","last_unstale":"2023-09-10T00:39:01.482417+0000","last_undegraded":"2023-09-10T00:39:01.482417+0000","last_fullsized":"2023-09-10T00:39:01.482417+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:22:32.658544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.779874+0000","last_change":"2023-09-10T00:38:39.772857+0000","last_active":"2023-09-10T00:38:43.779874+0000","last_peered":"2023-09-10T00:38:43.779874+0000","last_clean":"2023-09-10T00:38:43.779874+0000","last_became_active":"2023-09-10T00:38:39.771990+0000","last_became_peered":"2023-09-10T00:38:39.771990+0000","last_unstale":"2023-09-10T00:38:43.779874+0000","last_undegraded":"2023-09-10T00:38:43.779874+0000","last_fullsized":"2023-09-10T00:38:43.779874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:16:15.767748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253930+0000","last_change":"2023-09-10T00:38:43.795482+0000","last_active":"2023-09-10T00:38:46.253930+0000","last_peered":"2023-09-10T00:38:46.253930+0000","last_clean":"2023-09-10T00:38:46.253930+0000","last_became_active":"2023-09-10T00:38:43.795048+0000","last_became_peered":"2023-09-10T00:38:43.795048+0000","last_unstale":"2023-09-10T00:38:46.253930+0000","last_undegraded":"2023-09-10T00:38:46.253930+0000","last_fullsized":"2023-09-10T00:38:46.253930+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:13:18.715274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.066839+0000","last_change":"2023-09-10T00:38:38.771008+0000","last_active":"2023-09-10T00:38:46.066839+0000","last_peered":"2023-09-10T00:38:46.066839+0000","last_clean":"2023-09-10T00:38:46.066839+0000","last_became_active":"2023-09-10T00:38:38.770714+0000","last_became_peered":"2023-09-10T00:38:38.770714+0000","last_unstale":"2023-09-10T00:38:46.066839+0000","last_undegraded":"2023-09-10T00:38:46.066839+0000","last_fullsized":"2023-09-10T00:38:46.066839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:04:30.148141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.781087+0000","last_change":"2023-09-10T00:38:39.760234+0000","last_active":"2023-09-10T00:38:43.781087+0000","last_peered":"2023-09-10T00:38:43.781087+0000","last_clean":"2023-09-10T00:38:43.781087+0000","last_became_active":"2023-09-10T00:38:39.759874+0000","last_became_peered":"2023-09-10T00:38:39.759874+0000","last_unstale":"2023-09-10T00:38:43.781087+0000","last_undegraded":"2023-09-10T00:38:43.781087+0000","last_fullsized":"2023-09-10T00:38:43.781087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:38:40.826776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.252965+0000","last_change":"2023-09-10T00:38:43.788877+0000","last_active":"2023-09-10T00:38:46.252965+0000","last_peered":"2023-09-10T00:38:46.252965+0000","last_clean":"2023-09-10T00:38:46.252965+0000","last_became_active":"2023-09-10T00:38:43.788438+0000","last_became_peered":"2023-09-10T00:38:43.788438+0000","last_unstale":"2023-09-10T00:38:46.252965+0000","last_undegraded":"2023-09-10T00:38:46.252965+0000","last_fullsized":"2023-09-10T00:38:46.252965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:33:37.757020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'12","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:58.839927+0000","last_change":"2023-09-10T00:38:38.761072+0000","last_active":"2023-09-10T00:38:58.839927+0000","last_peered":"2023-09-10T00:38:58.839927+0000","last_clean":"2023-09-10T00:38:58.839927+0000","last_became_active":"2023-09-10T00:38:38.760752+0000","last_became_peered":"2023-09-10T00:38:38.760752+0000","last_unstale":"2023-09-10T00:38:58.839927+0000","last_undegraded":"2023-09-10T00:38:58.839927+0000","last_fullsized":"2023-09-10T00:38:58.839927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:55:18.272741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.776667+0000","last_change":"2023-09-10T00:38:39.767507+0000","last_active":"2023-09-10T00:38:43.776667+0000","last_peered":"2023-09-10T00:38:43.776667+0000","last_clean":"2023-09-10T00:38:43.776667+0000","last_became_active":"2023-09-10T00:38:39.767223+0000","last_became_peered":"2023-09-10T00:38:39.767223+0000","last_unstale":"2023-09-10T00:38:43.776667+0000","last_undegraded":"2023-09-10T00:38:43.776667+0000","last_fullsized":"2023-09-10T00:38:43.776667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:46:30.901818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.067240+0000","last_change":"2023-09-10T00:38:43.785396+0000","last_active":"2023-09-10T00:38:46.067240+0000","last_peered":"2023-09-10T00:38:46.067240+0000","last_clean":"2023-09-10T00:38:46.067240+0000","last_became_active":"2023-09-10T00:38:43.785038+0000","last_became_peered":"2023-09-10T00:38:43.785038+0000","last_unstale":"2023-09-10T00:38:46.067240+0000","last_undegraded":"2023-09-10T00:38:46.067240+0000","last_fullsized":"2023-09-10T00:38:46.067240+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:30:31.629703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"82'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778339+0000","last_change":"2023-09-10T00:38:38.766742+0000","last_active":"2023-09-10T00:38:43.778339+0000","last_peered":"2023-09-10T00:38:43.778339+0000","last_clean":"2023-09-10T00:38:43.778339+0000","last_became_active":"2023-09-10T00:38:38.766220+0000","last_became_peered":"2023-09-10T00:38:38.766220+0000","last_unstale":"2023-09-10T00:38:43.778339+0000","last_undegraded":"2023-09-10T00:38:43.778339+0000","last_fullsized":"2023-09-10T00:38:43.778339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:41:22.030448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.774373+0000","last_change":"2023-09-10T00:38:39.766022+0000","last_active":"2023-09-10T00:38:43.774373+0000","last_peered":"2023-09-10T00:38:43.774373+0000","last_clean":"2023-09-10T00:38:43.774373+0000","last_became_active":"2023-09-10T00:38:39.765697+0000","last_became_peered":"2023-09-10T00:38:39.765697+0000","last_unstale":"2023-09-10T00:38:43.774373+0000","last_undegraded":"2023-09-10T00:38:43.774373+0000","last_fullsized":"2023-09-10T00:38:43.774373+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:19:39.394299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253786+0000","last_change":"2023-09-10T00:38:43.795492+0000","last_active":"2023-09-10T00:38:46.253786+0000","last_peered":"2023-09-10T00:38:46.253786+0000","last_clean":"2023-09-10T00:38:46.253786+0000","last_became_active":"2023-09-10T00:38:43.794968+0000","last_became_peered":"2023-09-10T00:38:43.794968+0000","last_unstale":"2023-09-10T00:38:46.253786+0000","last_undegraded":"2023-09-10T00:38:46.253786+0000","last_fullsized":"2023-09-10T00:38:46.253786+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:40:43.558870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:58.843552+0000","last_change":"2023-09-10T00:38:38.767666+0000","last_active":"2023-09-10T00:38:58.843552+0000","last_peered":"2023-09-10T00:38:58.843552+0000","last_clean":"2023-09-10T00:38:58.843552+0000","last_became_active":"2023-09-10T00:38:38.767333+0000","last_became_peered":"2023-09-10T00:38:38.767333+0000","last_unstale":"2023-09-10T00:38:58.843552+0000","last_undegraded":"2023-09-10T00:38:58.843552+0000","last_fullsized":"2023-09-10T00:38:58.843552+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:07:53.363196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772282+0000","last_change":"2023-09-10T00:38:39.766036+0000","last_active":"2023-09-10T00:38:43.772282+0000","last_peered":"2023-09-10T00:38:43.772282+0000","last_clean":"2023-09-10T00:38:43.772282+0000","last_became_active":"2023-09-10T00:38:39.765683+0000","last_became_peered":"2023-09-10T00:38:39.765683+0000","last_unstale":"2023-09-10T00:38:43.772282+0000","last_undegraded":"2023-09-10T00:38:43.772282+0000","last_fullsized":"2023-09-10T00:38:43.772282+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:58:04.429960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253407+0000","last_change":"2023-09-10T00:38:43.787870+0000","last_active":"2023-09-10T00:38:46.253407+0000","last_peered":"2023-09-10T00:38:46.253407+0000","last_clean":"2023-09-10T00:38:46.253407+0000","last_became_active":"2023-09-10T00:38:43.785915+0000","last_became_peered":"2023-09-10T00:38:43.785915+0000","last_unstale":"2023-09-10T00:38:46.253407+0000","last_undegraded":"2023-09-10T00:38:46.253407+0000","last_fullsized":"2023-09-10T00:38:46.253407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:34:08.623967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.776032+0000","last_change":"2023-09-10T00:38:38.756925+0000","last_active":"2023-09-10T00:38:43.776032+0000","last_peered":"2023-09-10T00:38:43.776032+0000","last_clean":"2023-09-10T00:38:43.776032+0000","last_became_active":"2023-09-10T00:38:38.756610+0000","last_became_peered":"2023-09-10T00:38:38.756610+0000","last_unstale":"2023-09-10T00:38:43.776032+0000","last_undegraded":"2023-09-10T00:38:43.776032+0000","last_fullsized":"2023-09-10T00:38:43.776032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:48:15.385230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772314+0000","last_change":"2023-09-10T00:38:39.766750+0000","last_active":"2023-09-10T00:38:43.772314+0000","last_peered":"2023-09-10T00:38:43.772314+0000","last_clean":"2023-09-10T00:38:43.772314+0000","last_became_active":"2023-09-10T00:38:39.766458+0000","last_became_peered":"2023-09-10T00:38:39.766458+0000","last_unstale":"2023-09-10T00:38:43.772314+0000","last_undegraded":"2023-09-10T00:38:43.772314+0000","last_fullsized":"2023-09-10T00:38:43.772314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:59:26.691487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:44.771371+0000","last_change":"2023-09-10T00:38:43.788814+0000","last_active":"2023-09-10T00:38:44.771371+0000","last_peered":"2023-09-10T00:38:44.771371+0000","last_clean":"2023-09-10T00:38:44.771371+0000","last_became_active":"2023-09-10T00:38:43.788446+0000","last_became_peered":"2023-09-10T00:38:43.788446+0000","last_unstale":"2023-09-10T00:38:44.771371+0000","last_undegraded":"2023-09-10T00:38:44.771371+0000","last_fullsized":"2023-09-10T00:38:44.771371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:38:53.308160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"82'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.775907+0000","last_change":"2023-09-10T00:38:38.766020+0000","last_active":"2023-09-10T00:38:43.775907+0000","last_peered":"2023-09-10T00:38:43.775907+0000","last_clean":"2023-09-10T00:38:43.775907+0000","last_became_active":"2023-09-10T00:38:38.765719+0000","last_became_peered":"2023-09-10T00:38:38.765719+0000","last_unstale":"2023-09-10T00:38:43.775907+0000","last_undegraded":"2023-09-10T00:38:43.775907+0000","last_fullsized":"2023-09-10T00:38:43.775907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:25:30.488014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.773146+0000","last_change":"2023-09-10T00:38:39.771091+0000","last_active":"2023-09-10T00:38:43.773146+0000","last_peered":"2023-09-10T00:38:43.773146+0000","last_clean":"2023-09-10T00:38:43.773146+0000","last_became_active":"2023-09-10T00:38:39.770745+0000","last_became_peered":"2023-09-10T00:38:39.770745+0000","last_unstale":"2023-09-10T00:38:43.773146+0000","last_undegraded":"2023-09-10T00:38:43.773146+0000","last_fullsized":"2023-09-10T00:38:43.773146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:51:17.978466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.066591+0000","last_change":"2023-09-10T00:38:43.793757+0000","last_active":"2023-09-10T00:38:46.066591+0000","last_peered":"2023-09-10T00:38:46.066591+0000","last_clean":"2023-09-10T00:38:46.066591+0000","last_became_active":"2023-09-10T00:38:43.793336+0000","last_became_peered":"2023-09-10T00:38:43.793336+0000","last_unstale":"2023-09-10T00:38:46.066591+0000","last_undegraded":"2023-09-10T00:38:46.066591+0000","last_fullsized":"2023-09-10T00:38:46.066591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:22:36.544950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.776942+0000","last_change":"2023-09-10T00:38:38.769373+0000","last_active":"2023-09-10T00:38:43.776942+0000","last_peered":"2023-09-10T00:38:43.776942+0000","last_clean":"2023-09-10T00:38:43.776942+0000","last_became_active":"2023-09-10T00:38:38.769054+0000","last_became_peered":"2023-09-10T00:38:38.769054+0000","last_unstale":"2023-09-10T00:38:43.776942+0000","last_undegraded":"2023-09-10T00:38:43.776942+0000","last_fullsized":"2023-09-10T00:38:43.776942+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:12:27.982405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778698+0000","last_change":"2023-09-10T00:38:39.766068+0000","last_active":"2023-09-10T00:38:43.778698+0000","last_peered":"2023-09-10T00:38:43.778698+0000","last_clean":"2023-09-10T00:38:43.778698+0000","last_became_active":"2023-09-10T00:38:39.765773+0000","last_became_peered":"2023-09-10T00:38:39.765773+0000","last_unstale":"2023-09-10T00:38:43.778698+0000","last_undegraded":"2023-09-10T00:38:43.778698+0000","last_fullsized":"2023-09-10T00:38:43.778698+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:28:34.172288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:44.770828+0000","last_change":"2023-09-10T00:38:43.838689+0000","last_active":"2023-09-10T00:38:44.770828+0000","last_peered":"2023-09-10T00:38:44.770828+0000","last_clean":"2023-09-10T00:38:44.770828+0000","last_became_active":"2023-09-10T00:38:43.838244+0000","last_became_peered":"2023-09-10T00:38:43.838244+0000","last_unstale":"2023-09-10T00:38:44.770828+0000","last_undegraded":"2023-09-10T00:38:44.770828+0000","last_fullsized":"2023-09-10T00:38:44.770828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:54:00.033719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253225+0000","last_change":"2023-09-10T00:38:38.765140+0000","last_active":"2023-09-10T00:38:46.253225+0000","last_peered":"2023-09-10T00:38:46.253225+0000","last_clean":"2023-09-10T00:38:46.253225+0000","last_became_active":"2023-09-10T00:38:38.764730+0000","last_became_peered":"2023-09-10T00:38:38.764730+0000","last_unstale":"2023-09-10T00:38:46.253225+0000","last_undegraded":"2023-09-10T00:38:46.253225+0000","last_fullsized":"2023-09-10T00:38:46.253225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:49:41.129763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.837565+0000","last_change":"2023-09-10T00:38:39.773260+0000","last_active":"2023-09-10T00:38:43.837565+0000","last_peered":"2023-09-10T00:38:43.837565+0000","last_clean":"2023-09-10T00:38:43.837565+0000","last_became_active":"2023-09-10T00:38:39.772947+0000","last_became_peered":"2023-09-10T00:38:39.772947+0000","last_unstale":"2023-09-10T00:38:43.837565+0000","last_undegraded":"2023-09-10T00:38:43.837565+0000","last_fullsized":"2023-09-10T00:38:43.837565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:08:10.791610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:44.771317+0000","last_change":"2023-09-10T00:38:43.787882+0000","last_active":"2023-09-10T00:38:44.771317+0000","last_peered":"2023-09-10T00:38:44.771317+0000","last_clean":"2023-09-10T00:38:44.771317+0000","last_became_active":"2023-09-10T00:38:43.787257+0000","last_became_peered":"2023-09-10T00:38:43.787257+0000","last_unstale":"2023-09-10T00:38:44.771317+0000","last_undegraded":"2023-09-10T00:38:44.771317+0000","last_fullsized":"2023-09-10T00:38:44.771317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:11:19.171881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:01.483506+0000","last_change":"2023-09-10T00:38:38.760014+0000","last_active":"2023-09-10T00:39:01.483506+0000","last_peered":"2023-09-10T00:39:01.483506+0000","last_clean":"2023-09-10T00:39:01.483506+0000","last_became_active":"2023-09-10T00:38:38.759637+0000","last_became_peered":"2023-09-10T00:38:38.759637+0000","last_unstale":"2023-09-10T00:39:01.483506+0000","last_undegraded":"2023-09-10T00:39:01.483506+0000","last_fullsized":"2023-09-10T00:39:01.483506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:15.704929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.837304+0000","last_change":"2023-09-10T00:38:39.765800+0000","last_active":"2023-09-10T00:38:43.837304+0000","last_peered":"2023-09-10T00:38:43.837304+0000","last_clean":"2023-09-10T00:38:43.837304+0000","last_became_active":"2023-09-10T00:38:39.765468+0000","last_became_peered":"2023-09-10T00:38:39.765468+0000","last_unstale":"2023-09-10T00:38:43.837304+0000","last_undegraded":"2023-09-10T00:38:43.837304+0000","last_fullsized":"2023-09-10T00:38:43.837304+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:57:56.429365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:44.771364+0000","last_change":"2023-09-10T00:38:43.838285+0000","last_active":"2023-09-10T00:38:44.771364+0000","last_peered":"2023-09-10T00:38:44.771364+0000","last_clean":"2023-09-10T00:38:44.771364+0000","last_became_active":"2023-09-10T00:38:43.837821+0000","last_became_peered":"2023-09-10T00:38:43.837821+0000","last_unstale":"2023-09-10T00:38:44.771364+0000","last_undegraded":"2023-09-10T00:38:44.771364+0000","last_fullsized":"2023-09-10T00:38:44.771364+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:01:03.354281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.773101+0000","last_change":"2023-09-10T00:38:39.763480+0000","last_active":"2023-09-10T00:38:43.773101+0000","last_peered":"2023-09-10T00:38:43.773101+0000","last_clean":"2023-09-10T00:38:43.773101+0000","last_became_active":"2023-09-10T00:38:39.763177+0000","last_became_peered":"2023-09-10T00:38:39.763177+0000","last_unstale":"2023-09-10T00:38:43.773101+0000","last_undegraded":"2023-09-10T00:38:43.773101+0000","last_fullsized":"2023-09-10T00:38:43.773101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:44:14.800236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:59.854768+0000","last_change":"2023-09-10T00:38:38.758745+0000","last_active":"2023-09-10T00:38:59.854768+0000","last_peered":"2023-09-10T00:38:59.854768+0000","last_clean":"2023-09-10T00:38:59.854768+0000","last_became_active":"2023-09-10T00:38:38.758427+0000","last_became_peered":"2023-09-10T00:38:38.758427+0000","last_unstale":"2023-09-10T00:38:59.854768+0000","last_undegraded":"2023-09-10T00:38:59.854768+0000","last_fullsized":"2023-09-10T00:38:59.854768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:02:49.238175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.658168+0000","last_change":"2023-09-10T00:38:43.784058+0000","last_active":"2023-09-10T00:38:46.658168+0000","last_peered":"2023-09-10T00:38:46.658168+0000","last_clean":"2023-09-10T00:38:46.658168+0000","last_became_active":"2023-09-10T00:38:43.783571+0000","last_became_peered":"2023-09-10T00:38:43.783571+0000","last_unstale":"2023-09-10T00:38:46.658168+0000","last_undegraded":"2023-09-10T00:38:46.658168+0000","last_fullsized":"2023-09-10T00:38:46.658168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:53:50.992393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.779510+0000","last_change":"2023-09-10T00:38:39.775659+0000","last_active":"2023-09-10T00:38:43.779510+0000","last_peered":"2023-09-10T00:38:43.779510+0000","last_clean":"2023-09-10T00:38:43.779510+0000","last_became_active":"2023-09-10T00:38:39.775151+0000","last_became_peered":"2023-09-10T00:38:39.775151+0000","last_unstale":"2023-09-10T00:38:43.779510+0000","last_undegraded":"2023-09-10T00:38:43.779510+0000","last_fullsized":"2023-09-10T00:38:43.779510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:07.415418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778416+0000","last_change":"2023-09-10T00:38:38.755009+0000","last_active":"2023-09-10T00:38:43.778416+0000","last_peered":"2023-09-10T00:38:43.778416+0000","last_clean":"2023-09-10T00:38:43.778416+0000","last_became_active":"2023-09-10T00:38:38.754699+0000","last_became_peered":"2023-09-10T00:38:38.754699+0000","last_unstale":"2023-09-10T00:38:43.778416+0000","last_undegraded":"2023-09-10T00:38:43.778416+0000","last_fullsized":"2023-09-10T00:38:43.778416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:12:02.075256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":320,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772183+0000","last_change":"2023-09-10T00:37:38.459695+0000","last_active":"2023-09-10T00:38:43.772183+0000","last_peered":"2023-09-10T00:38:43.772183+0000","last_clean":"2023-09-10T00:38:43.772183+0000","last_became_active":"2023-09-10T00:37:38.459220+0000","last_became_peered":"2023-09-10T00:37:38.459220+0000","last_unstale":"2023-09-10T00:38:43.772183+0000","last_undegraded":"2023-09-10T00:38:43.772183+0000","last_fullsized":"2023-09-10T00:38:43.772183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:34:44.602334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:34:44.602334+0000","last_clean_scrub_stamp":"2023-09-10T00:34:44.602334+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:21:32.648725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.066939+0000","last_change":"2023-09-10T00:38:43.839001+0000","last_active":"2023-09-10T00:38:46.066939+0000","last_peered":"2023-09-10T00:38:46.066939+0000","last_clean":"2023-09-10T00:38:46.066939+0000","last_became_active":"2023-09-10T00:38:43.838497+0000","last_became_peered":"2023-09-10T00:38:43.838497+0000","last_unstale":"2023-09-10T00:38:46.066939+0000","last_undegraded":"2023-09-10T00:38:46.066939+0000","last_fullsized":"2023-09-10T00:38:46.066939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:57:23.349632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.773180+0000","last_change":"2023-09-10T00:38:39.760150+0000","last_active":"2023-09-10T00:38:43.773180+0000","last_peered":"2023-09-10T00:38:43.773180+0000","last_clean":"2023-09-10T00:38:43.773180+0000","last_became_active":"2023-09-10T00:38:39.759811+0000","last_became_peered":"2023-09-10T00:38:39.759811+0000","last_unstale":"2023-09-10T00:38:43.773180+0000","last_undegraded":"2023-09-10T00:38:43.773180+0000","last_fullsized":"2023-09-10T00:38:43.773180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:21:19.359057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:59.853234+0000","last_change":"2023-09-10T00:38:38.771066+0000","last_active":"2023-09-10T00:38:59.853234+0000","last_peered":"2023-09-10T00:38:59.853234+0000","last_clean":"2023-09-10T00:38:59.853234+0000","last_became_active":"2023-09-10T00:38:38.770810+0000","last_became_peered":"2023-09-10T00:38:38.770810+0000","last_unstale":"2023-09-10T00:38:59.853234+0000","last_undegraded":"2023-09-10T00:38:59.853234+0000","last_fullsized":"2023-09-10T00:38:59.853234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:55:06.923842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253778+0000","last_change":"2023-09-10T00:38:43.786272+0000","last_active":"2023-09-10T00:38:46.253778+0000","last_peered":"2023-09-10T00:38:46.253778+0000","last_clean":"2023-09-10T00:38:46.253778+0000","last_became_active":"2023-09-10T00:38:43.785934+0000","last_became_peered":"2023-09-10T00:38:43.785934+0000","last_unstale":"2023-09-10T00:38:46.253778+0000","last_undegraded":"2023-09-10T00:38:46.253778+0000","last_fullsized":"2023-09-10T00:38:46.253778+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:31:49.721133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.776720+0000","last_change":"2023-09-10T00:38:38.763263+0000","last_active":"2023-09-10T00:38:43.776720+0000","last_peered":"2023-09-10T00:38:43.776720+0000","last_clean":"2023-09-10T00:38:43.776720+0000","last_became_active":"2023-09-10T00:38:38.762911+0000","last_became_peered":"2023-09-10T00:38:38.762911+0000","last_unstale":"2023-09-10T00:38:43.776720+0000","last_undegraded":"2023-09-10T00:38:43.776720+0000","last_fullsized":"2023-09-10T00:38:43.776720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:19:10.292311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.774636+0000","last_change":"2023-09-10T00:38:39.759171+0000","last_active":"2023-09-10T00:38:43.774636+0000","last_peered":"2023-09-10T00:38:43.774636+0000","last_clean":"2023-09-10T00:38:43.774636+0000","last_became_active":"2023-09-10T00:38:39.758843+0000","last_became_peered":"2023-09-10T00:38:39.758843+0000","last_unstale":"2023-09-10T00:38:43.774636+0000","last_undegraded":"2023-09-10T00:38:43.774636+0000","last_fullsized":"2023-09-10T00:38:43.774636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:50:36.064937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253682+0000","last_change":"2023-09-10T00:38:43.837998+0000","last_active":"2023-09-10T00:38:46.253682+0000","last_peered":"2023-09-10T00:38:46.253682+0000","last_clean":"2023-09-10T00:38:46.253682+0000","last_became_active":"2023-09-10T00:38:43.837556+0000","last_became_peered":"2023-09-10T00:38:43.837556+0000","last_unstale":"2023-09-10T00:38:46.253682+0000","last_undegraded":"2023-09-10T00:38:46.253682+0000","last_fullsized":"2023-09-10T00:38:46.253682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:18:40.488279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:00.475148+0000","last_change":"2023-09-10T00:38:38.759819+0000","last_active":"2023-09-10T00:39:00.475148+0000","last_peered":"2023-09-10T00:39:00.475148+0000","last_clean":"2023-09-10T00:39:00.475148+0000","last_became_active":"2023-09-10T00:38:38.759490+0000","last_became_peered":"2023-09-10T00:38:38.759490+0000","last_unstale":"2023-09-10T00:39:00.475148+0000","last_undegraded":"2023-09-10T00:39:00.475148+0000","last_fullsized":"2023-09-10T00:39:00.475148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:39:01.206817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.787641+0000","last_change":"2023-09-10T00:38:39.769669+0000","last_active":"2023-09-10T00:38:43.787641+0000","last_peered":"2023-09-10T00:38:43.787641+0000","last_clean":"2023-09-10T00:38:43.787641+0000","last_became_active":"2023-09-10T00:38:39.769402+0000","last_became_peered":"2023-09-10T00:38:39.769402+0000","last_unstale":"2023-09-10T00:38:43.787641+0000","last_undegraded":"2023-09-10T00:38:43.787641+0000","last_fullsized":"2023-09-10T00:38:43.787641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:00:50.215103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253291+0000","last_change":"2023-09-10T00:38:43.787518+0000","last_active":"2023-09-10T00:38:46.253291+0000","last_peered":"2023-09-10T00:38:46.253291+0000","last_clean":"2023-09-10T00:38:46.253291+0000","last_became_active":"2023-09-10T00:38:43.786410+0000","last_became_peered":"2023-09-10T00:38:43.786410+0000","last_unstale":"2023-09-10T00:38:46.253291+0000","last_undegraded":"2023-09-10T00:38:46.253291+0000","last_fullsized":"2023-09-10T00:38:46.253291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:13:37.903875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778088+0000","last_change":"2023-09-10T00:38:38.760992+0000","last_active":"2023-09-10T00:38:43.778088+0000","last_peered":"2023-09-10T00:38:43.778088+0000","last_clean":"2023-09-10T00:38:43.778088+0000","last_became_active":"2023-09-10T00:38:38.760635+0000","last_became_peered":"2023-09-10T00:38:38.760635+0000","last_unstale":"2023-09-10T00:38:43.778088+0000","last_undegraded":"2023-09-10T00:38:43.778088+0000","last_fullsized":"2023-09-10T00:38:43.778088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:29:36.070477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778256+0000","last_change":"2023-09-10T00:38:39.773771+0000","last_active":"2023-09-10T00:38:43.778256+0000","last_peered":"2023-09-10T00:38:43.778256+0000","last_clean":"2023-09-10T00:38:43.778256+0000","last_became_active":"2023-09-10T00:38:39.773296+0000","last_became_peered":"2023-09-10T00:38:39.773296+0000","last_unstale":"2023-09-10T00:38:43.778256+0000","last_undegraded":"2023-09-10T00:38:43.778256+0000","last_fullsized":"2023-09-10T00:38:43.778256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:13:01.194358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.254303+0000","last_change":"2023-09-10T00:38:43.839879+0000","last_active":"2023-09-10T00:38:46.254303+0000","last_peered":"2023-09-10T00:38:46.254303+0000","last_clean":"2023-09-10T00:38:46.254303+0000","last_became_active":"2023-09-10T00:38:43.839167+0000","last_became_peered":"2023-09-10T00:38:43.839167+0000","last_unstale":"2023-09-10T00:38:46.254303+0000","last_undegraded":"2023-09-10T00:38:46.254303+0000","last_fullsized":"2023-09-10T00:38:46.254303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:11:44.779730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778983+0000","last_change":"2023-09-10T00:38:38.764954+0000","last_active":"2023-09-10T00:38:43.778983+0000","last_peered":"2023-09-10T00:38:43.778983+0000","last_clean":"2023-09-10T00:38:43.778983+0000","last_became_active":"2023-09-10T00:38:38.764589+0000","last_became_peered":"2023-09-10T00:38:38.764589+0000","last_unstale":"2023-09-10T00:38:43.778983+0000","last_undegraded":"2023-09-10T00:38:43.778983+0000","last_fullsized":"2023-09-10T00:38:43.778983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:40:19.559285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.775598+0000","last_change":"2023-09-10T00:38:39.766141+0000","last_active":"2023-09-10T00:38:43.775598+0000","last_peered":"2023-09-10T00:38:43.775598+0000","last_clean":"2023-09-10T00:38:43.775598+0000","last_became_active":"2023-09-10T00:38:39.765832+0000","last_became_peered":"2023-09-10T00:38:39.765832+0000","last_unstale":"2023-09-10T00:38:43.775598+0000","last_undegraded":"2023-09-10T00:38:43.775598+0000","last_fullsized":"2023-09-10T00:38:43.775598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:08:45.149973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253630+0000","last_change":"2023-09-10T00:38:43.785111+0000","last_active":"2023-09-10T00:38:46.253630+0000","last_peered":"2023-09-10T00:38:46.253630+0000","last_clean":"2023-09-10T00:38:46.253630+0000","last_became_active":"2023-09-10T00:38:43.784542+0000","last_became_peered":"2023-09-10T00:38:43.784542+0000","last_unstale":"2023-09-10T00:38:46.253630+0000","last_undegraded":"2023-09-10T00:38:46.253630+0000","last_fullsized":"2023-09-10T00:38:46.253630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:34:51.611608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:00.475001+0000","last_change":"2023-09-10T00:38:38.759984+0000","last_active":"2023-09-10T00:39:00.475001+0000","last_peered":"2023-09-10T00:39:00.475001+0000","last_clean":"2023-09-10T00:39:00.475001+0000","last_became_active":"2023-09-10T00:38:38.759660+0000","last_became_peered":"2023-09-10T00:38:38.759660+0000","last_unstale":"2023-09-10T00:39:00.475001+0000","last_undegraded":"2023-09-10T00:39:00.475001+0000","last_fullsized":"2023-09-10T00:39:00.475001+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:02:11.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.776657+0000","last_change":"2023-09-10T00:38:39.770765+0000","last_active":"2023-09-10T00:38:43.776657+0000","last_peered":"2023-09-10T00:38:43.776657+0000","last_clean":"2023-09-10T00:38:43.776657+0000","last_became_active":"2023-09-10T00:38:39.770483+0000","last_became_peered":"2023-09-10T00:38:39.770483+0000","last_unstale":"2023-09-10T00:38:43.776657+0000","last_undegraded":"2023-09-10T00:38:43.776657+0000","last_fullsized":"2023-09-10T00:38:43.776657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:57:28.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.254029+0000","last_change":"2023-09-10T00:38:43.790945+0000","last_active":"2023-09-10T00:38:46.254029+0000","last_peered":"2023-09-10T00:38:46.254029+0000","last_clean":"2023-09-10T00:38:46.254029+0000","last_became_active":"2023-09-10T00:38:43.790467+0000","last_became_peered":"2023-09-10T00:38:43.790467+0000","last_unstale":"2023-09-10T00:38:46.254029+0000","last_undegraded":"2023-09-10T00:38:46.254029+0000","last_fullsized":"2023-09-10T00:38:46.254029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:44:07.041302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.776418+0000","last_change":"2023-09-10T00:38:38.765912+0000","last_active":"2023-09-10T00:38:43.776418+0000","last_peered":"2023-09-10T00:38:43.776418+0000","last_clean":"2023-09-10T00:38:43.776418+0000","last_became_active":"2023-09-10T00:38:38.765586+0000","last_became_peered":"2023-09-10T00:38:38.765586+0000","last_unstale":"2023-09-10T00:38:43.776418+0000","last_undegraded":"2023-09-10T00:38:43.776418+0000","last_fullsized":"2023-09-10T00:38:43.776418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:25:05.974158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.780661+0000","last_change":"2023-09-10T00:38:39.771462+0000","last_active":"2023-09-10T00:38:43.780661+0000","last_peered":"2023-09-10T00:38:43.780661+0000","last_clean":"2023-09-10T00:38:43.780661+0000","last_became_active":"2023-09-10T00:38:39.771153+0000","last_became_peered":"2023-09-10T00:38:39.771153+0000","last_unstale":"2023-09-10T00:38:43.780661+0000","last_undegraded":"2023-09-10T00:38:43.780661+0000","last_fullsized":"2023-09-10T00:38:43.780661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:03:35.312186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253881+0000","last_change":"2023-09-10T00:38:43.838446+0000","last_active":"2023-09-10T00:38:46.253881+0000","last_peered":"2023-09-10T00:38:46.253881+0000","last_clean":"2023-09-10T00:38:46.253881+0000","last_became_active":"2023-09-10T00:38:43.838119+0000","last_became_peered":"2023-09-10T00:38:43.838119+0000","last_unstale":"2023-09-10T00:38:46.253881+0000","last_undegraded":"2023-09-10T00:38:46.253881+0000","last_fullsized":"2023-09-10T00:38:46.253881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:12:20.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"82'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.775830+0000","last_change":"2023-09-10T00:38:38.765548+0000","last_active":"2023-09-10T00:38:43.775830+0000","last_peered":"2023-09-10T00:38:43.775830+0000","last_clean":"2023-09-10T00:38:43.775830+0000","last_became_active":"2023-09-10T00:38:38.765281+0000","last_became_peered":"2023-09-10T00:38:38.765281+0000","last_unstale":"2023-09-10T00:38:43.775830+0000","last_undegraded":"2023-09-10T00:38:43.775830+0000","last_fullsized":"2023-09-10T00:38:43.775830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:50:44.422327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772694+0000","last_change":"2023-09-10T00:38:39.771995+0000","last_active":"2023-09-10T00:38:43.772694+0000","last_peered":"2023-09-10T00:38:43.772694+0000","last_clean":"2023-09-10T00:38:43.772694+0000","last_became_active":"2023-09-10T00:38:39.771693+0000","last_became_peered":"2023-09-10T00:38:39.771693+0000","last_unstale":"2023-09-10T00:38:43.772694+0000","last_undegraded":"2023-09-10T00:38:43.772694+0000","last_fullsized":"2023-09-10T00:38:43.772694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:41:45.621060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:44.771267+0000","last_change":"2023-09-10T00:38:43.789159+0000","last_active":"2023-09-10T00:38:44.771267+0000","last_peered":"2023-09-10T00:38:44.771267+0000","last_clean":"2023-09-10T00:38:44.771267+0000","last_became_active":"2023-09-10T00:38:43.788675+0000","last_became_peered":"2023-09-10T00:38:43.788675+0000","last_unstale":"2023-09-10T00:38:44.771267+0000","last_undegraded":"2023-09-10T00:38:44.771267+0000","last_fullsized":"2023-09-10T00:38:44.771267+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:30:39.545561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.777401+0000","last_change":"2023-09-10T00:38:38.767245+0000","last_active":"2023-09-10T00:38:43.777401+0000","last_peered":"2023-09-10T00:38:43.777401+0000","last_clean":"2023-09-10T00:38:43.777401+0000","last_became_active":"2023-09-10T00:38:38.766952+0000","last_became_peered":"2023-09-10T00:38:38.766952+0000","last_unstale":"2023-09-10T00:38:43.777401+0000","last_undegraded":"2023-09-10T00:38:43.777401+0000","last_fullsized":"2023-09-10T00:38:43.777401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:57:23.309838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772636+0000","last_change":"2023-09-10T00:38:39.772199+0000","last_active":"2023-09-10T00:38:43.772636+0000","last_peered":"2023-09-10T00:38:43.772636+0000","last_clean":"2023-09-10T00:38:43.772636+0000","last_became_active":"2023-09-10T00:38:39.771917+0000","last_became_peered":"2023-09-10T00:38:39.771917+0000","last_unstale":"2023-09-10T00:38:43.772636+0000","last_undegraded":"2023-09-10T00:38:43.772636+0000","last_fullsized":"2023-09-10T00:38:43.772636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:00:52.824581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:45.752923+0000","last_change":"2023-09-10T00:38:43.797107+0000","last_active":"2023-09-10T00:38:45.752923+0000","last_peered":"2023-09-10T00:38:45.752923+0000","last_clean":"2023-09-10T00:38:45.752923+0000","last_became_active":"2023-09-10T00:38:43.796606+0000","last_became_peered":"2023-09-10T00:38:43.796606+0000","last_unstale":"2023-09-10T00:38:45.752923+0000","last_undegraded":"2023-09-10T00:38:45.752923+0000","last_fullsized":"2023-09-10T00:38:45.752923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:17:00.444238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.773571+0000","last_change":"2023-09-10T00:38:38.768045+0000","last_active":"2023-09-10T00:38:43.773571+0000","last_peered":"2023-09-10T00:38:43.773571+0000","last_clean":"2023-09-10T00:38:43.773571+0000","last_became_active":"2023-09-10T00:38:38.767344+0000","last_became_peered":"2023-09-10T00:38:38.767344+0000","last_unstale":"2023-09-10T00:38:43.773571+0000","last_undegraded":"2023-09-10T00:38:43.773571+0000","last_fullsized":"2023-09-10T00:38:43.773571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:51:08.765412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772316+0000","last_change":"2023-09-10T00:38:39.772931+0000","last_active":"2023-09-10T00:38:43.772316+0000","last_peered":"2023-09-10T00:38:43.772316+0000","last_clean":"2023-09-10T00:38:43.772316+0000","last_became_active":"2023-09-10T00:38:39.772631+0000","last_became_peered":"2023-09-10T00:38:39.772631+0000","last_unstale":"2023-09-10T00:38:43.772316+0000","last_undegraded":"2023-09-10T00:38:43.772316+0000","last_fullsized":"2023-09-10T00:38:43.772316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:12:23.948287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.066355+0000","last_change":"2023-09-10T00:38:43.784928+0000","last_active":"2023-09-10T00:38:46.066355+0000","last_peered":"2023-09-10T00:38:46.066355+0000","last_clean":"2023-09-10T00:38:46.066355+0000","last_became_active":"2023-09-10T00:38:43.784479+0000","last_became_peered":"2023-09-10T00:38:43.784479+0000","last_unstale":"2023-09-10T00:38:46.066355+0000","last_undegraded":"2023-09-10T00:38:46.066355+0000","last_fullsized":"2023-09-10T00:38:46.066355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:14:45.019988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:44.771265+0000","last_change":"2023-09-10T00:38:38.770730+0000","last_active":"2023-09-10T00:38:44.771265+0000","last_peered":"2023-09-10T00:38:44.771265+0000","last_clean":"2023-09-10T00:38:44.771265+0000","last_became_active":"2023-09-10T00:38:38.770344+0000","last_became_peered":"2023-09-10T00:38:38.770344+0000","last_unstale":"2023-09-10T00:38:44.771265+0000","last_undegraded":"2023-09-10T00:38:44.771265+0000","last_fullsized":"2023-09-10T00:38:44.771265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:49:22.881956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772728+0000","last_change":"2023-09-10T00:38:39.766149+0000","last_active":"2023-09-10T00:38:43.772728+0000","last_peered":"2023-09-10T00:38:43.772728+0000","last_clean":"2023-09-10T00:38:43.772728+0000","last_became_active":"2023-09-10T00:38:39.765773+0000","last_became_peered":"2023-09-10T00:38:39.765773+0000","last_unstale":"2023-09-10T00:38:43.772728+0000","last_undegraded":"2023-09-10T00:38:43.772728+0000","last_fullsized":"2023-09-10T00:38:43.772728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:00:37.854168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.776553+0000","last_change":"2023-09-10T00:38:39.765197+0000","last_active":"2023-09-10T00:38:43.776553+0000","last_peered":"2023-09-10T00:38:43.776553+0000","last_clean":"2023-09-10T00:38:43.776553+0000","last_became_active":"2023-09-10T00:38:39.764891+0000","last_became_peered":"2023-09-10T00:38:39.764891+0000","last_unstale":"2023-09-10T00:38:43.776553+0000","last_undegraded":"2023-09-10T00:38:43.776553+0000","last_fullsized":"2023-09-10T00:38:43.776553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:47:27.429355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.781389+0000","last_change":"2023-09-10T00:38:39.764722+0000","last_active":"2023-09-10T00:38:43.781389+0000","last_peered":"2023-09-10T00:38:43.781389+0000","last_clean":"2023-09-10T00:38:43.781389+0000","last_became_active":"2023-09-10T00:38:39.764376+0000","last_became_peered":"2023-09-10T00:38:39.764376+0000","last_unstale":"2023-09-10T00:38:43.781389+0000","last_undegraded":"2023-09-10T00:38:43.781389+0000","last_fullsized":"2023-09-10T00:38:43.781389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:27:36.708573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.776325+0000","last_change":"2023-09-10T00:38:39.771997+0000","last_active":"2023-09-10T00:38:43.776325+0000","last_peered":"2023-09-10T00:38:43.776325+0000","last_clean":"2023-09-10T00:38:43.776325+0000","last_became_active":"2023-09-10T00:38:39.771588+0000","last_became_peered":"2023-09-10T00:38:39.771588+0000","last_unstale":"2023-09-10T00:38:43.776325+0000","last_undegraded":"2023-09-10T00:38:43.776325+0000","last_fullsized":"2023-09-10T00:38:43.776325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:20:07.171333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778627+0000","last_change":"2023-09-10T00:38:39.773393+0000","last_active":"2023-09-10T00:38:43.778627+0000","last_peered":"2023-09-10T00:38:43.778627+0000","last_clean":"2023-09-10T00:38:43.778627+0000","last_became_active":"2023-09-10T00:38:39.772972+0000","last_became_peered":"2023-09-10T00:38:39.772972+0000","last_unstale":"2023-09-10T00:38:43.778627+0000","last_undegraded":"2023-09-10T00:38:43.778627+0000","last_fullsized":"2023-09-10T00:38:43.778627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:27:29.916970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.775135+0000","last_change":"2023-09-10T00:38:39.767871+0000","last_active":"2023-09-10T00:38:43.775135+0000","last_peered":"2023-09-10T00:38:43.775135+0000","last_clean":"2023-09-10T00:38:43.775135+0000","last_became_active":"2023-09-10T00:38:39.767563+0000","last_became_peered":"2023-09-10T00:38:39.767563+0000","last_unstale":"2023-09-10T00:38:43.775135+0000","last_undegraded":"2023-09-10T00:38:43.775135+0000","last_fullsized":"2023-09-10T00:38:43.775135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:22:30.400441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778083+0000","last_change":"2023-09-10T00:38:39.764664+0000","last_active":"2023-09-10T00:38:43.778083+0000","last_peered":"2023-09-10T00:38:43.778083+0000","last_clean":"2023-09-10T00:38:43.778083+0000","last_became_active":"2023-09-10T00:38:39.764364+0000","last_became_peered":"2023-09-10T00:38:39.764364+0000","last_unstale":"2023-09-10T00:38:43.778083+0000","last_undegraded":"2023-09-10T00:38:43.778083+0000","last_fullsized":"2023-09-10T00:38:43.778083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:41.656472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778638+0000","last_change":"2023-09-10T00:38:39.770315+0000","last_active":"2023-09-10T00:38:43.778638+0000","last_peered":"2023-09-10T00:38:43.778638+0000","last_clean":"2023-09-10T00:38:43.778638+0000","last_became_active":"2023-09-10T00:38:39.769977+0000","last_became_peered":"2023-09-10T00:38:39.769977+0000","last_unstale":"2023-09-10T00:38:43.778638+0000","last_undegraded":"2023-09-10T00:38:43.778638+0000","last_fullsized":"2023-09-10T00:38:43.778638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:30:13.380487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.787768+0000","last_change":"2023-09-10T00:38:39.768734+0000","last_active":"2023-09-10T00:38:43.787768+0000","last_peered":"2023-09-10T00:38:43.787768+0000","last_clean":"2023-09-10T00:38:43.787768+0000","last_became_active":"2023-09-10T00:38:39.768383+0000","last_became_peered":"2023-09-10T00:38:39.768383+0000","last_unstale":"2023-09-10T00:38:43.787768+0000","last_undegraded":"2023-09-10T00:38:43.787768+0000","last_fullsized":"2023-09-10T00:38:43.787768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:09.974447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.773027+0000","last_change":"2023-09-10T00:38:39.773294+0000","last_active":"2023-09-10T00:38:43.773027+0000","last_peered":"2023-09-10T00:38:43.773027+0000","last_clean":"2023-09-10T00:38:43.773027+0000","last_became_active":"2023-09-10T00:38:39.772925+0000","last_became_peered":"2023-09-10T00:38:39.772925+0000","last_unstale":"2023-09-10T00:38:43.773027+0000","last_undegraded":"2023-09-10T00:38:43.773027+0000","last_fullsized":"2023-09-10T00:38:43.773027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:58:27.525511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.782371+0000","last_change":"2023-09-10T00:38:39.773839+0000","last_active":"2023-09-10T00:38:43.782371+0000","last_peered":"2023-09-10T00:38:43.782371+0000","last_clean":"2023-09-10T00:38:43.782371+0000","last_became_active":"2023-09-10T00:38:39.773521+0000","last_became_peered":"2023-09-10T00:38:39.773521+0000","last_unstale":"2023-09-10T00:38:43.782371+0000","last_undegraded":"2023-09-10T00:38:43.782371+0000","last_fullsized":"2023-09-10T00:38:43.782371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:15:49.033374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.779484+0000","last_change":"2023-09-10T00:38:39.775721+0000","last_active":"2023-09-10T00:38:43.779484+0000","last_peered":"2023-09-10T00:38:43.779484+0000","last_clean":"2023-09-10T00:38:43.779484+0000","last_became_active":"2023-09-10T00:38:39.775424+0000","last_became_peered":"2023-09-10T00:38:39.775424+0000","last_unstale":"2023-09-10T00:38:43.779484+0000","last_undegraded":"2023-09-10T00:38:43.779484+0000","last_fullsized":"2023-09-10T00:38:43.779484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:56:26.079332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.774663+0000","last_change":"2023-09-10T00:38:39.768767+0000","last_active":"2023-09-10T00:38:43.774663+0000","last_peered":"2023-09-10T00:38:43.774663+0000","last_clean":"2023-09-10T00:38:43.774663+0000","last_became_active":"2023-09-10T00:38:39.768452+0000","last_became_peered":"2023-09-10T00:38:39.768452+0000","last_unstale":"2023-09-10T00:38:43.774663+0000","last_undegraded":"2023-09-10T00:38:43.774663+0000","last_fullsized":"2023-09-10T00:38:43.774663+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:31:53.578574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778044+0000","last_change":"2023-09-10T00:38:39.761998+0000","last_active":"2023-09-10T00:38:43.778044+0000","last_peered":"2023-09-10T00:38:43.778044+0000","last_clean":"2023-09-10T00:38:43.778044+0000","last_became_active":"2023-09-10T00:38:39.761672+0000","last_became_peered":"2023-09-10T00:38:39.761672+0000","last_unstale":"2023-09-10T00:38:43.778044+0000","last_undegraded":"2023-09-10T00:38:43.778044+0000","last_fullsized":"2023-09-10T00:38:43.778044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:03:24.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6702,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":107,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":20058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449102,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":466233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]}]},{"osd":10,"up_from":70,"seq":300647710739,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1061399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.442}]},{"osd":2,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.65100000000000002}]},{"osd":3,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.626}]},{"osd":4,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60199999999999998}]},{"osd":5,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.752}]},{"osd":6,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71299999999999997}]},{"osd":8,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.82499999999999996}]},{"osd":9,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71799999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]}]},{"osd":9,"up_from":65,"seq":279172874264,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":467051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.313}]},{"osd":1,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.36299999999999999}]},{"osd":2,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46300000000000002}]},{"osd":3,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.48299999999999998}]},{"osd":4,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.54400000000000004}]},{"osd":5,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.61299999999999999}]},{"osd":6,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.42499999999999999}]},{"osd":7,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.64500000000000002}]},{"osd":8,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.59799999999999998}]},{"osd":10,"last update":"Sun Sep 10 00:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.628}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":8,"up_from":57,"seq":244813135900,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":466506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57199999999999995,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.67800000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55800000000000005}]},{"osd":1,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.67500000000000004,"15min":0.628},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":6.5759999999999996,"5min":6.5759999999999996,"15min":6.5759999999999996},"last":0.69099999999999995}]},{"osd":2,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.745},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.73299999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.82799999999999996}]},{"osd":3,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83099999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.85099999999999998}]},{"osd":4,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.41199999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.439,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.90000000000000002}]},{"osd":5,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.34699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94399999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.871}]},{"osd":6,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.79100000000000004,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":6.5940000000000003,"5min":6.5940000000000003,"15min":6.5940000000000003},"last":1.026}]},{"osd":7,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.76000000000000001,"15min":0.70999999999999996},"min":{"1min":0.51300000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.5439999999999996,"5min":6.5439999999999996,"15min":6.5439999999999996},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.441,"5min":0.435,"15min":0.435},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.997}]},{"osd":9,"last update":"Sun Sep 10 00:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":1.012},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.93400000000000005}]},{"osd":10,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":1.044}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]}]},{"osd":7,"up_from":50,"seq":214748364831,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1058734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54100000000000004,"15min":0.53200000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.39800000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.27600000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.60499999999999998,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.28299999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.75800000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.36699999999999999}]},{"osd":2,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.59199999999999997}]},{"osd":3,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.38400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.37,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73599999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.50800000000000001}]},{"osd":4,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.32300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78400000000000003,"5min":1.056,"15min":1.056},"last":0.34899999999999998}]},{"osd":5,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.629,"15min":0.627},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73499999999999999}]},{"osd":6,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65300000000000002,"15min":0.65000000000000002},"min":{"1min":0.50800000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.87,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.877,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.434}]},{"osd":8,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.63500000000000001,"15min":0.62},"min":{"1min":0.50600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.66200000000000003,"15min":0.65200000000000002},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.66900000000000004}]},{"osd":9,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.62}]},{"osd":10,"last update":"Sun Sep 10 00:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.65200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528356,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":466323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.41199999999999998,"15min":0.40000000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.33800000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.33800000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.57499999999999996,"15min":0.55600000000000005},"min":{"1min":0.436,"5min":0.378,"15min":0.378},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.48999999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.49099999999999999}]},{"osd":2,"last update":"Sun Sep 10 00:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.441,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.432,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.624}]},{"osd":3,"last update":"Sun Sep 10 00:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.33700000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.442}]},{"osd":4,"last update":"Sun Sep 10 00:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65400000000000003,"15min":0.64500000000000002},"min":{"1min":0.47799999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.69799999999999995,"15min":0.68200000000000005},"min":{"1min":0.51500000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.50700000000000001}]},{"osd":5,"last update":"Sun Sep 10 00:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69199999999999995,"15min":0.68000000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.56499999999999995,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.74099999999999999,"15min":0.73199999999999998},"min":{"1min":0.53600000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.73699999999999999,"15min":0.72499999999999998},"min":{"1min":0.52100000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.53200000000000003}]},{"osd":8,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.81499999999999995,"15min":0.83299999999999996},"min":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.999,"5min":1.27,"15min":1.27},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.81699999999999995,"15min":0.82799999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0389999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.64700000000000002}]},{"osd":9,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.54000000000000004}]},{"osd":10,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.65400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":470075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.52500000000000002,"15min":0.50900000000000001},"min":{"1min":0.32600000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.59299999999999997}]},{"osd":2,"last update":"Sun Sep 10 00:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.38100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73299999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.56399999999999995,"15min":0.53700000000000003},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":0.89100000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.77200000000000002}]},{"osd":3,"last update":"Sun Sep 10 00:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.36599999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.53700000000000003,"15min":0.52800000000000002},"min":{"1min":0.36299999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.65600000000000003}]},{"osd":4,"last update":"Sun Sep 10 00:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.42299999999999999},"min":{"1min":0.35099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.504},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.46000000000000002,"15min":0.437},"min":{"1min":0.33800000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.62}]},{"osd":5,"last update":"Sun Sep 10 00:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.755,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.65800000000000003,"15min":0.626},"min":{"1min":0.62,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92700000000000005,"5min":1.044,"15min":1.044},"last":1.0409999999999999}]},{"osd":6,"last update":"Sun Sep 10 00:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.623,"15min":0.60799999999999998},"min":{"1min":0.44400000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.629,"15min":0.61199999999999999},"min":{"1min":0.46400000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.83999999999999997}]},{"osd":7,"last update":"Sun Sep 10 00:38:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.59099999999999997,"15min":0.56599999999999995},"min":{"1min":0.49099999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.40899999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.86499999999999999}]},{"osd":8,"last update":"Sun Sep 10 00:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71699999999999997,"15min":0.71299999999999997},"min":{"1min":0.57799999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94999999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.54000000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.86599999999999999,"5min":1.014,"15min":1.014},"last":0.878}]},{"osd":9,"last update":"Sun Sep 10 00:38:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.93000000000000005}]},{"osd":10,"last update":"Sun Sep 10 00:38:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.90500000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":466323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 10 00:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52100000000000002,"15min":0.49199999999999999},"min":{"1min":0.371,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.623},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.432,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.54700000000000004}]},{"osd":2,"last update":"Sun Sep 10 00:38:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.39500000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80700000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58599999999999997,"15min":0.56799999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0329999999999999,"5min":1.621,"15min":1.621},"last":0.69699999999999995}]},{"osd":3,"last update":"Sun Sep 10 00:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.41799999999999998,"15min":0.41699999999999998},"min":{"1min":0.249,"5min":0.245,"15min":0.245},"max":{"1min":0.66700000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46000000000000002,"15min":0.45600000000000002},"min":{"1min":0.29099999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.79100000000000004,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.50900000000000001}]},{"osd":4,"last update":"Sun Sep 10 00:38:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68899999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.38900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.91300000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.66900000000000004}]},{"osd":5,"last update":"Sun Sep 10 00:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64000000000000001,"15min":0.60999999999999999},"min":{"1min":0.49199999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.96999999999999997,"5min":1.075,"15min":1.075},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64700000000000002,"15min":0.63700000000000001},"min":{"1min":0.48699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.81100000000000005}]},{"osd":6,"last update":"Sun Sep 10 00:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.52500000000000002,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.74099999999999999}]},{"osd":7,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65700000000000003,"15min":0.66400000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73899999999999999,"15min":0.74399999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.98499999999999999,"5min":1.111,"15min":1.111},"last":0.76600000000000001}]},{"osd":8,"last update":"Sun Sep 10 00:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77300000000000002,"15min":0.77900000000000003},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0640000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.499,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.88}]},{"osd":9,"last update":"Sun Sep 10 00:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.64300000000000002}]},{"osd":10,"last update":"Sun Sep 10 00:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.76000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411381,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":470109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.70499999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.39800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72099999999999997,"5min":0.753,"15min":0.753},"last":0.69499999999999995}]},{"osd":1,"last update":"Sun Sep 10 00:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.42499999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.746,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82599999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.54600000000000004}]},{"osd":3,"last update":"Sun Sep 10 00:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.54800000000000004,"15min":0.53300000000000003},"min":{"1min":0.47799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.442}]},{"osd":4,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.39000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.82299999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79700000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.90900000000000003}]},{"osd":5,"last update":"Sun Sep 10 00:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.45500000000000002,"15min":0.435},"min":{"1min":0.29899999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52400000000000002,"15min":0.5},"min":{"1min":0.35699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.71799999999999997}]},{"osd":6,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59999999999999998,"15min":0.59499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.80600000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.746},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.64200000000000002,"15min":0.63600000000000001},"min":{"1min":0.50800000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.65700000000000003}]},{"osd":7,"last update":"Sun Sep 10 00:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.75800000000000001}]},{"osd":8,"last update":"Sun Sep 10 00:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.87}]},{"osd":9,"last update":"Sun Sep 10 00:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.81100000000000005}]},{"osd":10,"last update":"Sun Sep 10 00:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.84399999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":466641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.26000000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.64300000000000002,"5min":1.409,"15min":1.409},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.42299999999999999,"15min":0.41899999999999998},"min":{"1min":0.30399999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.93000000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.34599999999999997}]},{"osd":1,"last update":"Sun Sep 10 00:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.40600000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85899999999999999,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.57299999999999995,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.90500000000000003,"5min":1.581,"15min":1.581},"last":0.39500000000000002}]},{"osd":2,"last update":"Sun Sep 10 00:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.61099999999999999,"15min":0.60299999999999998},"min":{"1min":0.5,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64500000000000002,"15min":0.65900000000000003},"min":{"1min":0.438,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.036,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.55500000000000005}]},{"osd":4,"last update":"Sun Sep 10 00:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88100000000000001,"5min":1.536,"15min":1.536},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67900000000000005,"15min":0.69299999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95099999999999996,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.625}]},{"osd":5,"last update":"Sun Sep 10 00:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88100000000000001,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67800000000000005,"15min":0.67500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88700000000000001,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Sun Sep 10 00:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.35299999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.438},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.59599999999999997,"15min":0.57799999999999996},"min":{"1min":0.44400000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59499999999999997}]},{"osd":7,"last update":"Sun Sep 10 00:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.755,"15min":0.749},"min":{"1min":0.51600000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.74099999999999999,"15min":0.73699999999999999},"min":{"1min":0.53500000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.50700000000000001}]},{"osd":8,"last update":"Sun Sep 10 00:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.90000000000000002}]},{"osd":9,"last update":"Sun Sep 10 00:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.69599999999999995}]},{"osd":10,"last update":"Sun Sep 10 00:38:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.61299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920813,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":466255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45000000000000001}]},{"osd":1,"last update":"Sun Sep 10 00:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.35299999999999998,"15min":0.33900000000000002},"min":{"1min":0.245,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.371,"15min":0.35899999999999999},"min":{"1min":0.27200000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.505,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.41799999999999998}]},{"osd":2,"last update":"Sun Sep 10 00:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.45600000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72199999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.60799999999999998}]},{"osd":3,"last update":"Sun Sep 10 00:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53800000000000003,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.57299999999999995,"15min":0.55000000000000004},"min":{"1min":0.44900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.73099999999999998}]},{"osd":5,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.439,"5min":0.376,"15min":0.376},"max":{"1min":0.80600000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.52600000000000002}]},{"osd":6,"last update":"Sun Sep 10 00:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":0.77600000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64200000000000002,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.70599999999999996}]},{"osd":7,"last update":"Sun Sep 10 00:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.53500000000000003,"15min":0.51800000000000002},"min":{"1min":0.41099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.48599999999999999}]},{"osd":8,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63900000000000001,"15min":0.629},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91100000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.624},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.879,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.68700000000000006}]},{"osd":9,"last update":"Sun Sep 10 00:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.75700000000000001}]},{"osd":10,"last update":"Sun Sep 10 00:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.44},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.45900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757289,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1058077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.40699999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.755,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54700000000000004}]},{"osd":1,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53400000000000003,"15min":0.51600000000000001},"min":{"1min":0.33700000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86099999999999999,"5min":0.873,"15min":0.873},"last":0.747}]},{"osd":2,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40699999999999997,"15min":0.39600000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.71699999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.438,"15min":0.42999999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.47199999999999998}]},{"osd":3,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54900000000000004,"15min":0.54200000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.81499999999999995,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.92400000000000004,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.57399999999999995}]},{"osd":4,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76000000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.70599999999999996}]},{"osd":6,"last update":"Sun Sep 10 00:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.45600000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82799999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.621}]},{"osd":7,"last update":"Sun Sep 10 00:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61499999999999999,"15min":0.61899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.45400000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.69299999999999995}]},{"osd":8,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.42499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53400000000000003,"15min":0.52000000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.65200000000000002}]},{"osd":9,"last update":"Sun Sep 10 00:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71799999999999997}]},{"osd":10,"last update":"Sun Sep 10 00:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.77600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-10T00:39:03.880 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-10T00:39:04.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.036+0000 7febe0811700 1 -- 172.21.15.44:0/681151445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febdc103a50 msgr2=0x7febdc105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:04.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.036+0000 7febe0811700 1 --2- 172.21.15.44:0/681151445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febdc103a50 0x7febdc105e40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7febcc009990 tx=0x7febcc009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:04.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.036+0000 7febe0811700 1 -- 172.21.15.44:0/681151445 shutdown_connections 2023-09-10T00:39:04.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.036+0000 7febe0811700 1 --2- 172.21.15.44:0/681151445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febdc106380 0x7febdc108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:04.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.036+0000 7febe0811700 1 --2- 172.21.15.44:0/681151445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febdc101120 0x7febdc103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:04.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.036+0000 7febe0811700 1 --2- 172.21.15.44:0/681151445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febdc103a50 0x7febdc105e40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:04.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.036+0000 7febe0811700 1 -- 172.21.15.44:0/681151445 >> 172.21.15.44:0/681151445 conn(0x7febdc0faa90 msgr2=0x7febdc0fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:04.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.036+0000 7febe0811700 1 -- 172.21.15.44:0/681151445 shutdown_connections 2023-09-10T00:39:04.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.036+0000 7febe0811700 1 -- 172.21.15.44:0/681151445 wait complete. 2023-09-10T00:39:04.041 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.037+0000 7febe0811700 1 Processor -- start 2023-09-10T00:39:04.041 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.037+0000 7febe0811700 1 -- start start 2023-09-10T00:39:04.041 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.037+0000 7febe0811700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febdc101120 0x7febdc100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:04.041 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.037+0000 7febe0811700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febdc103a50 0x7febdc0ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:04.041 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.037+0000 7febe0811700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febdc106380 0x7febdc0ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:04.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.037+0000 7febe0811700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc0ffdb0 con 0x7febdc106380 2023-09-10T00:39:04.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.037+0000 7febe0811700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc0fff20 con 0x7febdc101120 2023-09-10T00:39:04.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.037+0000 7febe0811700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc1a0120 con 0x7febdc103a50 2023-09-10T00:39:04.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.037+0000 7febdad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febdc106380 0x7febdc0ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:04.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.037+0000 7febdad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febdc106380 0x7febdc0ff610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46888/0 (socket says 172.21.15.44:46888) 2023-09-10T00:39:04.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.037+0000 7febdad9d700 1 -- 172.21.15.44:0/3235794514 learned_addr learned my addr 172.21.15.44:0/3235794514 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:04.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.037+0000 7febda59c700 1 --2- 172.21.15.44:0/3235794514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febdc101120 0x7febdc100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:04.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.037+0000 7febd9d9b700 1 --2- 172.21.15.44:0/3235794514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febdc103a50 0x7febdc0ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:04.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.038+0000 7febdad9d700 1 -- 172.21.15.44:0/3235794514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febdc103a50 msgr2=0x7febdc0ff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:04.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.038+0000 7febdad9d700 1 --2- 172.21.15.44:0/3235794514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febdc103a50 0x7febdc0ff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:04.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.038+0000 7febdad9d700 1 -- 172.21.15.44:0/3235794514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febdc101120 msgr2=0x7febdc100a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:04.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.038+0000 7febdad9d700 1 --2- 172.21.15.44:0/3235794514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febdc101120 0x7febdc100a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:04.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.038+0000 7febdad9d700 1 -- 172.21.15.44:0/3235794514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc0095f0 con 0x7febdc106380 2023-09-10T00:39:04.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.038+0000 7febd9d9b700 1 --2- 172.21.15.44:0/3235794514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febdc103a50 0x7febdc0ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:04.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.038+0000 7febdad9d700 1 --2- 172.21.15.44:0/3235794514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febdc106380 0x7febdc0ff610 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7febd4007d70 tx=0x7febd400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:04.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.038+0000 7febd37fe700 1 -- 172.21.15.44:0/3235794514 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd400ad40 con 0x7febdc106380 2023-09-10T00:39:04.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.038+0000 7febd37fe700 1 -- 172.21.15.44:0/3235794514 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febd40128b0 con 0x7febdc106380 2023-09-10T00:39:04.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.038+0000 7febd37fe700 1 -- 172.21.15.44:0/3235794514 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd40115a0 con 0x7febdc106380 2023-09-10T00:39:04.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.038+0000 7febe0811700 1 -- 172.21.15.44:0/3235794514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febdc1a0400 con 0x7febdc106380 2023-09-10T00:39:04.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.038+0000 7febe0811700 1 -- 172.21.15.44:0/3235794514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc1a08c0 con 0x7febdc106380 2023-09-10T00:39:04.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.039+0000 7febe0811700 1 -- 172.21.15.44:0/3235794514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febdc0fc610 con 0x7febdc106380 2023-09-10T00:39:04.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.040+0000 7febd37fe700 1 -- 172.21.15.44:0/3235794514 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7febd4012a20 con 0x7febdc106380 2023-09-10T00:39:04.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.041+0000 7febd37fe700 1 --2- 172.21.15.44:0/3235794514 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7febc80a0790 0x7febc80a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:04.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.041+0000 7febda59c700 1 --2- 172.21.15.44:0/3235794514 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7febc80a0790 0x7febc80a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:04.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.041+0000 7febd37fe700 1 -- 172.21.15.44:0/3235794514 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7febd40cc090 con 0x7febdc106380 2023-09-10T00:39:04.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.041+0000 7febda59c700 1 --2- 172.21.15.44:0/3235794514 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7febc80a0790 0x7febc80a2c50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7febc400ac50 tx=0x7febc400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:04.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.045+0000 7febd37fe700 1 -- 172.21.15.44:0/3235794514 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7febd4095920 con 0x7febdc106380 2023-09-10T00:39:04.106 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 10 00:39:03 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-b[138846]: 2023-09-10T00:39:03.723+0000 7fe443b4f700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-09-10T00:39:04.106 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 10 00:39:03 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-b[138846]: 2023-09-10T00:39:03.723+0000 7fe443b4f700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-09-10T00:39:04.106 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 10 00:39:03 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-b[138846]: 2023-09-10T00:39:03.723+0000 7fe443b4f700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-09-10T00:39:04.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:03 smithi099 ceph-mon[102692]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:39:04.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:03 smithi099 ceph-mon[102692]: pgmap v280: 129 pgs: 129 active+clean; 583 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 1.1 KiB/s wr, 5 op/s 2023-09-10T00:39:04.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:03 smithi099 ceph-mon[102692]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:39:04.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:03 smithi099 ceph-mon[102692]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:39:04.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:03 smithi099 ceph-mon[102692]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-10T00:39:04.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:03 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/797462926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:39:04.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:03 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3200927266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:04.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:03 smithi099 ceph-mon[102692]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:39:04.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.241+0000 7febe0811700 1 -- 172.21.15.44:0/3235794514 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7febdc02cfc0 con 0x7febc80a0790 2023-09-10T00:39:04.256 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.256+0000 7febd37fe700 1 -- 172.21.15.44:0/3235794514 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+421938 (secure 0 0 0) 0x7febdc02cfc0 con 0x7febc80a0790 2023-09-10T00:39:04.261 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:04.264 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.260+0000 7febe0811700 1 -- 172.21.15.44:0/3235794514 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7febc80a0790 msgr2=0x7febc80a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:04.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.260+0000 7febe0811700 1 --2- 172.21.15.44:0/3235794514 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7febc80a0790 0x7febc80a2c50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7febc400ac50 tx=0x7febc400a380 comp rx=0 tx=0).stop 2023-09-10T00:39:04.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.260+0000 7febe0811700 1 -- 172.21.15.44:0/3235794514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febdc106380 msgr2=0x7febdc0ff610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:04.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.260+0000 7febe0811700 1 --2- 172.21.15.44:0/3235794514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febdc106380 0x7febdc0ff610 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7febd4007d70 tx=0x7febd400a340 comp rx=0 tx=0).stop 2023-09-10T00:39:04.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.260+0000 7febe0811700 1 -- 172.21.15.44:0/3235794514 shutdown_connections 2023-09-10T00:39:04.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.260+0000 7febe0811700 1 --2- 172.21.15.44:0/3235794514 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7febc80a0790 0x7febc80a2c50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:04.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.260+0000 7febe0811700 1 --2- 172.21.15.44:0/3235794514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febdc106380 0x7febdc0ff610 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:04.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.260+0000 7febe0811700 1 --2- 172.21.15.44:0/3235794514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febdc101120 0x7febdc100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:04.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.260+0000 7febe0811700 1 --2- 172.21.15.44:0/3235794514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febdc103a50 0x7febdc0ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:04.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.260+0000 7febe0811700 1 -- 172.21.15.44:0/3235794514 >> 172.21.15.44:0/3235794514 conn(0x7febdc0faa90 msgr2=0x7febdc101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:04.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.260+0000 7febe0811700 1 -- 172.21.15.44:0/3235794514 shutdown_connections 2023-09-10T00:39:04.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:04.260+0000 7febe0811700 1 -- 172.21.15.44:0/3235794514 wait complete. 2023-09-10T00:39:04.267 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2023-09-10T00:39:04.272 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:03 smithi044 ceph-mon[98676]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:39:04.272 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:03 smithi044 ceph-mon[98676]: pgmap v280: 129 pgs: 129 active+clean; 583 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 1.1 KiB/s wr, 5 op/s 2023-09-10T00:39:04.272 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:03 smithi044 ceph-mon[98676]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:39:04.272 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:03 smithi044 ceph-mon[98676]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:39:04.272 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:03 smithi044 ceph-mon[98676]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-10T00:39:04.273 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:03 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/797462926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:39:04.273 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:03 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3200927266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:04.273 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:03 smithi044 ceph-mon[98676]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:39:04.274 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2023-09-10T00:39:03.998942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600394,"num_objects":85,"num_object_clones":0,"num_object_copies":257,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":164,"num_read_kb":142,"num_write":362,"num_write_kb":2991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":54,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337488,"kb_used_data":13296,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124604336,"statfs":{"total":1151940427776,"available":1151594840064,"internally_reserved":0,"allocated":13615104,"data_stored":7462053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7616,"num_objects":61,"num_object_clones":0,"num_object_copies":185,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":136,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003239"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.777120+0000","last_change":"2023-09-10T00:38:39.768218+0000","last_active":"2023-09-10T00:38:43.777120+0000","last_peered":"2023-09-10T00:38:43.777120+0000","last_clean":"2023-09-10T00:38:43.777120+0000","last_became_active":"2023-09-10T00:38:39.767454+0000","last_became_peered":"2023-09-10T00:38:39.767454+0000","last_unstale":"2023-09-10T00:38:43.777120+0000","last_undegraded":"2023-09-10T00:38:43.777120+0000","last_fullsized":"2023-09-10T00:38:43.777120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:17:30.287403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778165+0000","last_change":"2023-09-10T00:38:39.765384+0000","last_active":"2023-09-10T00:38:43.778165+0000","last_peered":"2023-09-10T00:38:43.778165+0000","last_clean":"2023-09-10T00:38:43.778165+0000","last_became_active":"2023-09-10T00:38:39.765090+0000","last_became_peered":"2023-09-10T00:38:39.765090+0000","last_unstale":"2023-09-10T00:38:43.778165+0000","last_undegraded":"2023-09-10T00:38:43.778165+0000","last_fullsized":"2023-09-10T00:38:43.778165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:02:46.025627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778228+0000","last_change":"2023-09-10T00:38:39.769222+0000","last_active":"2023-09-10T00:38:43.778228+0000","last_peered":"2023-09-10T00:38:43.778228+0000","last_clean":"2023-09-10T00:38:43.778228+0000","last_became_active":"2023-09-10T00:38:39.768801+0000","last_became_peered":"2023-09-10T00:38:39.768801+0000","last_unstale":"2023-09-10T00:38:43.778228+0000","last_undegraded":"2023-09-10T00:38:43.778228+0000","last_fullsized":"2023-09-10T00:38:43.778228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:38:53.394209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.777846+0000","last_change":"2023-09-10T00:38:39.764963+0000","last_active":"2023-09-10T00:38:43.777846+0000","last_peered":"2023-09-10T00:38:43.777846+0000","last_clean":"2023-09-10T00:38:43.777846+0000","last_became_active":"2023-09-10T00:38:39.764466+0000","last_became_peered":"2023-09-10T00:38:39.764466+0000","last_unstale":"2023-09-10T00:38:43.777846+0000","last_undegraded":"2023-09-10T00:38:43.777846+0000","last_fullsized":"2023-09-10T00:38:43.777846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:31.553321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.775719+0000","last_change":"2023-09-10T00:38:39.774933+0000","last_active":"2023-09-10T00:38:43.775719+0000","last_peered":"2023-09-10T00:38:43.775719+0000","last_clean":"2023-09-10T00:38:43.775719+0000","last_became_active":"2023-09-10T00:38:39.774606+0000","last_became_peered":"2023-09-10T00:38:39.774606+0000","last_unstale":"2023-09-10T00:38:43.775719+0000","last_undegraded":"2023-09-10T00:38:43.775719+0000","last_fullsized":"2023-09-10T00:38:43.775719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:31:58.902544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.837124+0000","last_change":"2023-09-10T00:38:39.761681+0000","last_active":"2023-09-10T00:38:43.837124+0000","last_peered":"2023-09-10T00:38:43.837124+0000","last_clean":"2023-09-10T00:38:43.837124+0000","last_became_active":"2023-09-10T00:38:39.761383+0000","last_became_peered":"2023-09-10T00:38:39.761383+0000","last_unstale":"2023-09-10T00:38:43.837124+0000","last_undegraded":"2023-09-10T00:38:43.837124+0000","last_fullsized":"2023-09-10T00:38:43.837124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:38:04.534656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772151+0000","last_change":"2023-09-10T00:38:39.765028+0000","last_active":"2023-09-10T00:38:43.772151+0000","last_peered":"2023-09-10T00:38:43.772151+0000","last_clean":"2023-09-10T00:38:43.772151+0000","last_became_active":"2023-09-10T00:38:39.764663+0000","last_became_peered":"2023-09-10T00:38:39.764663+0000","last_unstale":"2023-09-10T00:38:43.772151+0000","last_undegraded":"2023-09-10T00:38:43.772151+0000","last_fullsized":"2023-09-10T00:38:43.772151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:14:49.645129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.782249+0000","last_change":"2023-09-10T00:38:39.770135+0000","last_active":"2023-09-10T00:38:43.782249+0000","last_peered":"2023-09-10T00:38:43.782249+0000","last_clean":"2023-09-10T00:38:43.782249+0000","last_became_active":"2023-09-10T00:38:39.769720+0000","last_became_peered":"2023-09-10T00:38:39.769720+0000","last_unstale":"2023-09-10T00:38:43.782249+0000","last_undegraded":"2023-09-10T00:38:43.782249+0000","last_fullsized":"2023-09-10T00:38:43.782249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:46:21.166566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.779651+0000","last_change":"2023-09-10T00:38:39.770170+0000","last_active":"2023-09-10T00:38:43.779651+0000","last_peered":"2023-09-10T00:38:43.779651+0000","last_clean":"2023-09-10T00:38:43.779651+0000","last_became_active":"2023-09-10T00:38:39.769840+0000","last_became_peered":"2023-09-10T00:38:39.769840+0000","last_unstale":"2023-09-10T00:38:43.779651+0000","last_undegraded":"2023-09-10T00:38:43.779651+0000","last_fullsized":"2023-09-10T00:38:43.779651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:48.117670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.837797+0000","last_change":"2023-09-10T00:38:39.769621+0000","last_active":"2023-09-10T00:38:43.837797+0000","last_peered":"2023-09-10T00:38:43.837797+0000","last_clean":"2023-09-10T00:38:43.837797+0000","last_became_active":"2023-09-10T00:38:39.769300+0000","last_became_peered":"2023-09-10T00:38:39.769300+0000","last_unstale":"2023-09-10T00:38:43.837797+0000","last_undegraded":"2023-09-10T00:38:43.837797+0000","last_fullsized":"2023-09-10T00:38:43.837797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:37.100671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772431+0000","last_change":"2023-09-10T00:38:39.772200+0000","last_active":"2023-09-10T00:38:43.772431+0000","last_peered":"2023-09-10T00:38:43.772431+0000","last_clean":"2023-09-10T00:38:43.772431+0000","last_became_active":"2023-09-10T00:38:39.771503+0000","last_became_peered":"2023-09-10T00:38:39.771503+0000","last_unstale":"2023-09-10T00:38:43.772431+0000","last_undegraded":"2023-09-10T00:38:43.772431+0000","last_fullsized":"2023-09-10T00:38:43.772431+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:13:38.830642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.777810+0000","last_change":"2023-09-10T00:38:39.769832+0000","last_active":"2023-09-10T00:38:43.777810+0000","last_peered":"2023-09-10T00:38:43.777810+0000","last_clean":"2023-09-10T00:38:43.777810+0000","last_became_active":"2023-09-10T00:38:39.769546+0000","last_became_peered":"2023-09-10T00:38:39.769546+0000","last_unstale":"2023-09-10T00:38:43.777810+0000","last_undegraded":"2023-09-10T00:38:43.777810+0000","last_fullsized":"2023-09-10T00:38:43.777810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:53:05.219877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.774568+0000","last_change":"2023-09-10T00:38:39.766509+0000","last_active":"2023-09-10T00:38:43.774568+0000","last_peered":"2023-09-10T00:38:43.774568+0000","last_clean":"2023-09-10T00:38:43.774568+0000","last_became_active":"2023-09-10T00:38:39.766017+0000","last_became_peered":"2023-09-10T00:38:39.766017+0000","last_unstale":"2023-09-10T00:38:43.774568+0000","last_undegraded":"2023-09-10T00:38:43.774568+0000","last_fullsized":"2023-09-10T00:38:43.774568+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:53:05.246343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.777959+0000","last_change":"2023-09-10T00:38:39.764751+0000","last_active":"2023-09-10T00:38:43.777959+0000","last_peered":"2023-09-10T00:38:43.777959+0000","last_clean":"2023-09-10T00:38:43.777959+0000","last_became_active":"2023-09-10T00:38:39.764464+0000","last_became_peered":"2023-09-10T00:38:39.764464+0000","last_unstale":"2023-09-10T00:38:43.777959+0000","last_undegraded":"2023-09-10T00:38:43.777959+0000","last_fullsized":"2023-09-10T00:38:43.777959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:03:51.096126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778426+0000","last_change":"2023-09-10T00:38:39.777487+0000","last_active":"2023-09-10T00:38:43.778426+0000","last_peered":"2023-09-10T00:38:43.778426+0000","last_clean":"2023-09-10T00:38:43.778426+0000","last_became_active":"2023-09-10T00:38:39.777134+0000","last_became_peered":"2023-09-10T00:38:39.777134+0000","last_unstale":"2023-09-10T00:38:43.778426+0000","last_undegraded":"2023-09-10T00:38:43.778426+0000","last_fullsized":"2023-09-10T00:38:43.778426+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:17:55.356440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778881+0000","last_change":"2023-09-10T00:38:39.770787+0000","last_active":"2023-09-10T00:38:43.778881+0000","last_peered":"2023-09-10T00:38:43.778881+0000","last_clean":"2023-09-10T00:38:43.778881+0000","last_became_active":"2023-09-10T00:38:39.770497+0000","last_became_peered":"2023-09-10T00:38:39.770497+0000","last_unstale":"2023-09-10T00:38:43.778881+0000","last_undegraded":"2023-09-10T00:38:43.778881+0000","last_fullsized":"2023-09-10T00:38:43.778881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:43:11.066766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.776880+0000","last_change":"2023-09-10T00:38:39.768008+0000","last_active":"2023-09-10T00:38:43.776880+0000","last_peered":"2023-09-10T00:38:43.776880+0000","last_clean":"2023-09-10T00:38:43.776880+0000","last_became_active":"2023-09-10T00:38:39.767079+0000","last_became_peered":"2023-09-10T00:38:39.767079+0000","last_unstale":"2023-09-10T00:38:43.776880+0000","last_undegraded":"2023-09-10T00:38:43.776880+0000","last_fullsized":"2023-09-10T00:38:43.776880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:34:00.689881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.769405+0000","last_change":"2023-09-10T00:38:39.775326+0000","last_active":"2023-09-10T00:38:43.769405+0000","last_peered":"2023-09-10T00:38:43.769405+0000","last_clean":"2023-09-10T00:38:43.769405+0000","last_became_active":"2023-09-10T00:38:39.775007+0000","last_became_peered":"2023-09-10T00:38:39.775007+0000","last_unstale":"2023-09-10T00:38:43.769405+0000","last_undegraded":"2023-09-10T00:38:43.769405+0000","last_fullsized":"2023-09-10T00:38:43.769405+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:59:27.711722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.780709+0000","last_change":"2023-09-10T00:38:39.764974+0000","last_active":"2023-09-10T00:38:43.780709+0000","last_peered":"2023-09-10T00:38:43.780709+0000","last_clean":"2023-09-10T00:38:43.780709+0000","last_became_active":"2023-09-10T00:38:39.764368+0000","last_became_peered":"2023-09-10T00:38:39.764368+0000","last_unstale":"2023-09-10T00:38:43.780709+0000","last_undegraded":"2023-09-10T00:38:43.780709+0000","last_fullsized":"2023-09-10T00:38:43.780709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:40:28.046021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253622+0000","last_change":"2023-09-10T00:38:43.793589+0000","last_active":"2023-09-10T00:38:46.253622+0000","last_peered":"2023-09-10T00:38:46.253622+0000","last_clean":"2023-09-10T00:38:46.253622+0000","last_became_active":"2023-09-10T00:38:43.793166+0000","last_became_peered":"2023-09-10T00:38:43.793166+0000","last_unstale":"2023-09-10T00:38:46.253622+0000","last_undegraded":"2023-09-10T00:38:46.253622+0000","last_fullsized":"2023-09-10T00:38:46.253622+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:51:47.128925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:00.475366+0000","last_change":"2023-09-10T00:38:38.759909+0000","last_active":"2023-09-10T00:39:00.475366+0000","last_peered":"2023-09-10T00:39:00.475366+0000","last_clean":"2023-09-10T00:39:00.475366+0000","last_became_active":"2023-09-10T00:38:38.759561+0000","last_became_peered":"2023-09-10T00:38:38.759561+0000","last_unstale":"2023-09-10T00:39:00.475366+0000","last_undegraded":"2023-09-10T00:39:00.475366+0000","last_fullsized":"2023-09-10T00:39:00.475366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:56:18.009497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.783633+0000","last_change":"2023-09-10T00:38:39.774916+0000","last_active":"2023-09-10T00:38:43.783633+0000","last_peered":"2023-09-10T00:38:43.783633+0000","last_clean":"2023-09-10T00:38:43.783633+0000","last_became_active":"2023-09-10T00:38:39.774584+0000","last_became_peered":"2023-09-10T00:38:39.774584+0000","last_unstale":"2023-09-10T00:38:43.783633+0000","last_undegraded":"2023-09-10T00:38:43.783633+0000","last_fullsized":"2023-09-10T00:38:43.783633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:36:33.125742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253516+0000","last_change":"2023-09-10T00:38:43.789421+0000","last_active":"2023-09-10T00:38:46.253516+0000","last_peered":"2023-09-10T00:38:46.253516+0000","last_clean":"2023-09-10T00:38:46.253516+0000","last_became_active":"2023-09-10T00:38:43.788569+0000","last_became_peered":"2023-09-10T00:38:43.788569+0000","last_unstale":"2023-09-10T00:38:46.253516+0000","last_undegraded":"2023-09-10T00:38:46.253516+0000","last_fullsized":"2023-09-10T00:38:46.253516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:07:38.529811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.254348+0000","last_change":"2023-09-10T00:38:38.767381+0000","last_active":"2023-09-10T00:38:46.254348+0000","last_peered":"2023-09-10T00:38:46.254348+0000","last_clean":"2023-09-10T00:38:46.254348+0000","last_became_active":"2023-09-10T00:38:38.767082+0000","last_became_peered":"2023-09-10T00:38:38.767082+0000","last_unstale":"2023-09-10T00:38:46.254348+0000","last_undegraded":"2023-09-10T00:38:46.254348+0000","last_fullsized":"2023-09-10T00:38:46.254348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:13:32.533215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.779079+0000","last_change":"2023-09-10T00:38:39.772645+0000","last_active":"2023-09-10T00:38:43.779079+0000","last_peered":"2023-09-10T00:38:43.779079+0000","last_clean":"2023-09-10T00:38:43.779079+0000","last_became_active":"2023-09-10T00:38:39.771554+0000","last_became_peered":"2023-09-10T00:38:39.771554+0000","last_unstale":"2023-09-10T00:38:43.779079+0000","last_undegraded":"2023-09-10T00:38:43.779079+0000","last_fullsized":"2023-09-10T00:38:43.779079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:25:06.237309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253732+0000","last_change":"2023-09-10T00:38:43.795764+0000","last_active":"2023-09-10T00:38:46.253732+0000","last_peered":"2023-09-10T00:38:46.253732+0000","last_clean":"2023-09-10T00:38:46.253732+0000","last_became_active":"2023-09-10T00:38:43.795112+0000","last_became_peered":"2023-09-10T00:38:43.795112+0000","last_unstale":"2023-09-10T00:38:46.253732+0000","last_undegraded":"2023-09-10T00:38:46.253732+0000","last_fullsized":"2023-09-10T00:38:46.253732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:18:55.460752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:01.483705+0000","last_change":"2023-09-10T00:38:38.765522+0000","last_active":"2023-09-10T00:39:01.483705+0000","last_peered":"2023-09-10T00:39:01.483705+0000","last_clean":"2023-09-10T00:39:01.483705+0000","last_became_active":"2023-09-10T00:38:38.765219+0000","last_became_peered":"2023-09-10T00:38:38.765219+0000","last_unstale":"2023-09-10T00:39:01.483705+0000","last_undegraded":"2023-09-10T00:39:01.483705+0000","last_fullsized":"2023-09-10T00:39:01.483705+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:00:27.275269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.780883+0000","last_change":"2023-09-10T00:38:39.767790+0000","last_active":"2023-09-10T00:38:43.780883+0000","last_peered":"2023-09-10T00:38:43.780883+0000","last_clean":"2023-09-10T00:38:43.780883+0000","last_became_active":"2023-09-10T00:38:39.767513+0000","last_became_peered":"2023-09-10T00:38:39.767513+0000","last_unstale":"2023-09-10T00:38:43.780883+0000","last_undegraded":"2023-09-10T00:38:43.780883+0000","last_fullsized":"2023-09-10T00:38:43.780883+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:05:48.223951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253610+0000","last_change":"2023-09-10T00:38:43.789089+0000","last_active":"2023-09-10T00:38:46.253610+0000","last_peered":"2023-09-10T00:38:46.253610+0000","last_clean":"2023-09-10T00:38:46.253610+0000","last_became_active":"2023-09-10T00:38:43.788434+0000","last_became_peered":"2023-09-10T00:38:43.788434+0000","last_unstale":"2023-09-10T00:38:46.253610+0000","last_undegraded":"2023-09-10T00:38:46.253610+0000","last_fullsized":"2023-09-10T00:38:46.253610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:55:00.236088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:01.483269+0000","last_change":"2023-09-10T00:38:38.769478+0000","last_active":"2023-09-10T00:39:01.483269+0000","last_peered":"2023-09-10T00:39:01.483269+0000","last_clean":"2023-09-10T00:39:01.483269+0000","last_became_active":"2023-09-10T00:38:38.769168+0000","last_became_peered":"2023-09-10T00:38:38.769168+0000","last_unstale":"2023-09-10T00:39:01.483269+0000","last_undegraded":"2023-09-10T00:39:01.483269+0000","last_fullsized":"2023-09-10T00:39:01.483269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:03:48.133561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.773628+0000","last_change":"2023-09-10T00:38:39.771843+0000","last_active":"2023-09-10T00:38:43.773628+0000","last_peered":"2023-09-10T00:38:43.773628+0000","last_clean":"2023-09-10T00:38:43.773628+0000","last_became_active":"2023-09-10T00:38:39.771510+0000","last_became_peered":"2023-09-10T00:38:39.771510+0000","last_unstale":"2023-09-10T00:38:43.773628+0000","last_undegraded":"2023-09-10T00:38:43.773628+0000","last_fullsized":"2023-09-10T00:38:43.773628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:17:39.996490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.254062+0000","last_change":"2023-09-10T00:38:43.786853+0000","last_active":"2023-09-10T00:38:46.254062+0000","last_peered":"2023-09-10T00:38:46.254062+0000","last_clean":"2023-09-10T00:38:46.254062+0000","last_became_active":"2023-09-10T00:38:43.786386+0000","last_became_peered":"2023-09-10T00:38:43.786386+0000","last_unstale":"2023-09-10T00:38:46.254062+0000","last_undegraded":"2023-09-10T00:38:46.254062+0000","last_fullsized":"2023-09-10T00:38:46.254062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:23:53.724653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:58.843879+0000","last_change":"2023-09-10T00:38:38.769004+0000","last_active":"2023-09-10T00:38:58.843879+0000","last_peered":"2023-09-10T00:38:58.843879+0000","last_clean":"2023-09-10T00:38:58.843879+0000","last_became_active":"2023-09-10T00:38:38.768690+0000","last_became_peered":"2023-09-10T00:38:38.768690+0000","last_unstale":"2023-09-10T00:38:58.843879+0000","last_undegraded":"2023-09-10T00:38:58.843879+0000","last_fullsized":"2023-09-10T00:38:58.843879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:04:42.531092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.777391+0000","last_change":"2023-09-10T00:38:39.773174+0000","last_active":"2023-09-10T00:38:43.777391+0000","last_peered":"2023-09-10T00:38:43.777391+0000","last_clean":"2023-09-10T00:38:43.777391+0000","last_became_active":"2023-09-10T00:38:39.771192+0000","last_became_peered":"2023-09-10T00:38:39.771192+0000","last_unstale":"2023-09-10T00:38:43.777391+0000","last_undegraded":"2023-09-10T00:38:43.777391+0000","last_fullsized":"2023-09-10T00:38:43.777391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:16:25.034024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253678+0000","last_change":"2023-09-10T00:38:43.786661+0000","last_active":"2023-09-10T00:38:46.253678+0000","last_peered":"2023-09-10T00:38:46.253678+0000","last_clean":"2023-09-10T00:38:46.253678+0000","last_became_active":"2023-09-10T00:38:43.786242+0000","last_became_peered":"2023-09-10T00:38:43.786242+0000","last_unstale":"2023-09-10T00:38:46.253678+0000","last_undegraded":"2023-09-10T00:38:46.253678+0000","last_fullsized":"2023-09-10T00:38:46.253678+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:36:14.490085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.837126+0000","last_change":"2023-09-10T00:38:38.763416+0000","last_active":"2023-09-10T00:38:43.837126+0000","last_peered":"2023-09-10T00:38:43.837126+0000","last_clean":"2023-09-10T00:38:43.837126+0000","last_became_active":"2023-09-10T00:38:38.763054+0000","last_became_peered":"2023-09-10T00:38:38.763054+0000","last_unstale":"2023-09-10T00:38:43.837126+0000","last_undegraded":"2023-09-10T00:38:43.837126+0000","last_fullsized":"2023-09-10T00:38:43.837126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:47:00.179467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.773330+0000","last_change":"2023-09-10T00:38:39.773284+0000","last_active":"2023-09-10T00:38:43.773330+0000","last_peered":"2023-09-10T00:38:43.773330+0000","last_clean":"2023-09-10T00:38:43.773330+0000","last_became_active":"2023-09-10T00:38:39.772969+0000","last_became_peered":"2023-09-10T00:38:39.772969+0000","last_unstale":"2023-09-10T00:38:43.773330+0000","last_undegraded":"2023-09-10T00:38:43.773330+0000","last_fullsized":"2023-09-10T00:38:43.773330+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:46:45.013879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.254229+0000","last_change":"2023-09-10T00:38:43.790810+0000","last_active":"2023-09-10T00:38:46.254229+0000","last_peered":"2023-09-10T00:38:46.254229+0000","last_clean":"2023-09-10T00:38:46.254229+0000","last_became_active":"2023-09-10T00:38:43.790429+0000","last_became_peered":"2023-09-10T00:38:43.790429+0000","last_unstale":"2023-09-10T00:38:46.254229+0000","last_undegraded":"2023-09-10T00:38:46.254229+0000","last_fullsized":"2023-09-10T00:38:46.254229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:30:56.072587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:59.851971+0000","last_change":"2023-09-10T00:38:38.754326+0000","last_active":"2023-09-10T00:38:59.851971+0000","last_peered":"2023-09-10T00:38:59.851971+0000","last_clean":"2023-09-10T00:38:59.851971+0000","last_became_active":"2023-09-10T00:38:38.754006+0000","last_became_peered":"2023-09-10T00:38:38.754006+0000","last_unstale":"2023-09-10T00:38:59.851971+0000","last_undegraded":"2023-09-10T00:38:59.851971+0000","last_fullsized":"2023-09-10T00:38:59.851971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:09:26.983606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.782137+0000","last_change":"2023-09-10T00:38:39.769699+0000","last_active":"2023-09-10T00:38:43.782137+0000","last_peered":"2023-09-10T00:38:43.782137+0000","last_clean":"2023-09-10T00:38:43.782137+0000","last_became_active":"2023-09-10T00:38:39.769362+0000","last_became_peered":"2023-09-10T00:38:39.769362+0000","last_unstale":"2023-09-10T00:38:43.782137+0000","last_undegraded":"2023-09-10T00:38:43.782137+0000","last_fullsized":"2023-09-10T00:38:43.782137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:30:20.938437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:45.752807+0000","last_change":"2023-09-10T00:38:43.785669+0000","last_active":"2023-09-10T00:38:45.752807+0000","last_peered":"2023-09-10T00:38:45.752807+0000","last_clean":"2023-09-10T00:38:45.752807+0000","last_became_active":"2023-09-10T00:38:43.785255+0000","last_became_peered":"2023-09-10T00:38:43.785255+0000","last_unstale":"2023-09-10T00:38:45.752807+0000","last_undegraded":"2023-09-10T00:38:45.752807+0000","last_fullsized":"2023-09-10T00:38:45.752807+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:52:57.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.066564+0000","last_change":"2023-09-10T00:38:38.767997+0000","last_active":"2023-09-10T00:38:46.066564+0000","last_peered":"2023-09-10T00:38:46.066564+0000","last_clean":"2023-09-10T00:38:46.066564+0000","last_became_active":"2023-09-10T00:38:38.767715+0000","last_became_peered":"2023-09-10T00:38:38.767715+0000","last_unstale":"2023-09-10T00:38:46.066564+0000","last_undegraded":"2023-09-10T00:38:46.066564+0000","last_fullsized":"2023-09-10T00:38:46.066564+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:11:17.047280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.769273+0000","last_change":"2023-09-10T00:38:39.778004+0000","last_active":"2023-09-10T00:38:43.769273+0000","last_peered":"2023-09-10T00:38:43.769273+0000","last_clean":"2023-09-10T00:38:43.769273+0000","last_became_active":"2023-09-10T00:38:39.777632+0000","last_became_peered":"2023-09-10T00:38:39.777632+0000","last_unstale":"2023-09-10T00:38:43.769273+0000","last_undegraded":"2023-09-10T00:38:43.769273+0000","last_fullsized":"2023-09-10T00:38:43.769273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:46:40.804385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253113+0000","last_change":"2023-09-10T00:38:43.791397+0000","last_active":"2023-09-10T00:38:46.253113+0000","last_peered":"2023-09-10T00:38:46.253113+0000","last_clean":"2023-09-10T00:38:46.253113+0000","last_became_active":"2023-09-10T00:38:43.791010+0000","last_became_peered":"2023-09-10T00:38:43.791010+0000","last_unstale":"2023-09-10T00:38:46.253113+0000","last_undegraded":"2023-09-10T00:38:46.253113+0000","last_fullsized":"2023-09-10T00:38:46.253113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:52:26.015718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:58.841224+0000","last_change":"2023-09-10T00:38:38.767399+0000","last_active":"2023-09-10T00:38:58.841224+0000","last_peered":"2023-09-10T00:38:58.841224+0000","last_clean":"2023-09-10T00:38:58.841224+0000","last_became_active":"2023-09-10T00:38:38.767176+0000","last_became_peered":"2023-09-10T00:38:38.767176+0000","last_unstale":"2023-09-10T00:38:58.841224+0000","last_undegraded":"2023-09-10T00:38:58.841224+0000","last_fullsized":"2023-09-10T00:38:58.841224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:17:24.481985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.779324+0000","last_change":"2023-09-10T00:38:39.766013+0000","last_active":"2023-09-10T00:38:43.779324+0000","last_peered":"2023-09-10T00:38:43.779324+0000","last_clean":"2023-09-10T00:38:43.779324+0000","last_became_active":"2023-09-10T00:38:39.765235+0000","last_became_peered":"2023-09-10T00:38:39.765235+0000","last_unstale":"2023-09-10T00:38:43.779324+0000","last_undegraded":"2023-09-10T00:38:43.779324+0000","last_fullsized":"2023-09-10T00:38:43.779324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:01:48.874845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.252793+0000","last_change":"2023-09-10T00:38:43.790361+0000","last_active":"2023-09-10T00:38:46.252793+0000","last_peered":"2023-09-10T00:38:46.252793+0000","last_clean":"2023-09-10T00:38:46.252793+0000","last_became_active":"2023-09-10T00:38:43.789966+0000","last_became_peered":"2023-09-10T00:38:43.789966+0000","last_unstale":"2023-09-10T00:38:46.252793+0000","last_undegraded":"2023-09-10T00:38:46.252793+0000","last_fullsized":"2023-09-10T00:38:46.252793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:46:37.978258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:01.482417+0000","last_change":"2023-09-10T00:38:38.771670+0000","last_active":"2023-09-10T00:39:01.482417+0000","last_peered":"2023-09-10T00:39:01.482417+0000","last_clean":"2023-09-10T00:39:01.482417+0000","last_became_active":"2023-09-10T00:38:38.771339+0000","last_became_peered":"2023-09-10T00:38:38.771339+0000","last_unstale":"2023-09-10T00:39:01.482417+0000","last_undegraded":"2023-09-10T00:39:01.482417+0000","last_fullsized":"2023-09-10T00:39:01.482417+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:22:32.658544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.779874+0000","last_change":"2023-09-10T00:38:39.772857+0000","last_active":"2023-09-10T00:38:43.779874+0000","last_peered":"2023-09-10T00:38:43.779874+0000","last_clean":"2023-09-10T00:38:43.779874+0000","last_became_active":"2023-09-10T00:38:39.771990+0000","last_became_peered":"2023-09-10T00:38:39.771990+0000","last_unstale":"2023-09-10T00:38:43.779874+0000","last_undegraded":"2023-09-10T00:38:43.779874+0000","last_fullsized":"2023-09-10T00:38:43.779874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:16:15.767748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253930+0000","last_change":"2023-09-10T00:38:43.795482+0000","last_active":"2023-09-10T00:38:46.253930+0000","last_peered":"2023-09-10T00:38:46.253930+0000","last_clean":"2023-09-10T00:38:46.253930+0000","last_became_active":"2023-09-10T00:38:43.795048+0000","last_became_peered":"2023-09-10T00:38:43.795048+0000","last_unstale":"2023-09-10T00:38:46.253930+0000","last_undegraded":"2023-09-10T00:38:46.253930+0000","last_fullsized":"2023-09-10T00:38:46.253930+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:13:18.715274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.066839+0000","last_change":"2023-09-10T00:38:38.771008+0000","last_active":"2023-09-10T00:38:46.066839+0000","last_peered":"2023-09-10T00:38:46.066839+0000","last_clean":"2023-09-10T00:38:46.066839+0000","last_became_active":"2023-09-10T00:38:38.770714+0000","last_became_peered":"2023-09-10T00:38:38.770714+0000","last_unstale":"2023-09-10T00:38:46.066839+0000","last_undegraded":"2023-09-10T00:38:46.066839+0000","last_fullsized":"2023-09-10T00:38:46.066839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:04:30.148141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.781087+0000","last_change":"2023-09-10T00:38:39.760234+0000","last_active":"2023-09-10T00:38:43.781087+0000","last_peered":"2023-09-10T00:38:43.781087+0000","last_clean":"2023-09-10T00:38:43.781087+0000","last_became_active":"2023-09-10T00:38:39.759874+0000","last_became_peered":"2023-09-10T00:38:39.759874+0000","last_unstale":"2023-09-10T00:38:43.781087+0000","last_undegraded":"2023-09-10T00:38:43.781087+0000","last_fullsized":"2023-09-10T00:38:43.781087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:38:40.826776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.252965+0000","last_change":"2023-09-10T00:38:43.788877+0000","last_active":"2023-09-10T00:38:46.252965+0000","last_peered":"2023-09-10T00:38:46.252965+0000","last_clean":"2023-09-10T00:38:46.252965+0000","last_became_active":"2023-09-10T00:38:43.788438+0000","last_became_peered":"2023-09-10T00:38:43.788438+0000","last_unstale":"2023-09-10T00:38:46.252965+0000","last_undegraded":"2023-09-10T00:38:46.252965+0000","last_fullsized":"2023-09-10T00:38:46.252965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:33:37.757020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'12","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:58.839927+0000","last_change":"2023-09-10T00:38:38.761072+0000","last_active":"2023-09-10T00:38:58.839927+0000","last_peered":"2023-09-10T00:38:58.839927+0000","last_clean":"2023-09-10T00:38:58.839927+0000","last_became_active":"2023-09-10T00:38:38.760752+0000","last_became_peered":"2023-09-10T00:38:38.760752+0000","last_unstale":"2023-09-10T00:38:58.839927+0000","last_undegraded":"2023-09-10T00:38:58.839927+0000","last_fullsized":"2023-09-10T00:38:58.839927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:55:18.272741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.776667+0000","last_change":"2023-09-10T00:38:39.767507+0000","last_active":"2023-09-10T00:38:43.776667+0000","last_peered":"2023-09-10T00:38:43.776667+0000","last_clean":"2023-09-10T00:38:43.776667+0000","last_became_active":"2023-09-10T00:38:39.767223+0000","last_became_peered":"2023-09-10T00:38:39.767223+0000","last_unstale":"2023-09-10T00:38:43.776667+0000","last_undegraded":"2023-09-10T00:38:43.776667+0000","last_fullsized":"2023-09-10T00:38:43.776667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:46:30.901818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.067240+0000","last_change":"2023-09-10T00:38:43.785396+0000","last_active":"2023-09-10T00:38:46.067240+0000","last_peered":"2023-09-10T00:38:46.067240+0000","last_clean":"2023-09-10T00:38:46.067240+0000","last_became_active":"2023-09-10T00:38:43.785038+0000","last_became_peered":"2023-09-10T00:38:43.785038+0000","last_unstale":"2023-09-10T00:38:46.067240+0000","last_undegraded":"2023-09-10T00:38:46.067240+0000","last_fullsized":"2023-09-10T00:38:46.067240+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:30:31.629703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:01.484419+0000","last_change":"2023-09-10T00:38:38.766742+0000","last_active":"2023-09-10T00:39:01.484419+0000","last_peered":"2023-09-10T00:39:01.484419+0000","last_clean":"2023-09-10T00:39:01.484419+0000","last_became_active":"2023-09-10T00:38:38.766220+0000","last_became_peered":"2023-09-10T00:38:38.766220+0000","last_unstale":"2023-09-10T00:39:01.484419+0000","last_undegraded":"2023-09-10T00:39:01.484419+0000","last_fullsized":"2023-09-10T00:39:01.484419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:41:22.030448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.774373+0000","last_change":"2023-09-10T00:38:39.766022+0000","last_active":"2023-09-10T00:38:43.774373+0000","last_peered":"2023-09-10T00:38:43.774373+0000","last_clean":"2023-09-10T00:38:43.774373+0000","last_became_active":"2023-09-10T00:38:39.765697+0000","last_became_peered":"2023-09-10T00:38:39.765697+0000","last_unstale":"2023-09-10T00:38:43.774373+0000","last_undegraded":"2023-09-10T00:38:43.774373+0000","last_fullsized":"2023-09-10T00:38:43.774373+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:19:39.394299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253786+0000","last_change":"2023-09-10T00:38:43.795492+0000","last_active":"2023-09-10T00:38:46.253786+0000","last_peered":"2023-09-10T00:38:46.253786+0000","last_clean":"2023-09-10T00:38:46.253786+0000","last_became_active":"2023-09-10T00:38:43.794968+0000","last_became_peered":"2023-09-10T00:38:43.794968+0000","last_unstale":"2023-09-10T00:38:46.253786+0000","last_undegraded":"2023-09-10T00:38:46.253786+0000","last_fullsized":"2023-09-10T00:38:46.253786+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:40:43.558870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:58.843552+0000","last_change":"2023-09-10T00:38:38.767666+0000","last_active":"2023-09-10T00:38:58.843552+0000","last_peered":"2023-09-10T00:38:58.843552+0000","last_clean":"2023-09-10T00:38:58.843552+0000","last_became_active":"2023-09-10T00:38:38.767333+0000","last_became_peered":"2023-09-10T00:38:38.767333+0000","last_unstale":"2023-09-10T00:38:58.843552+0000","last_undegraded":"2023-09-10T00:38:58.843552+0000","last_fullsized":"2023-09-10T00:38:58.843552+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:07:53.363196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772282+0000","last_change":"2023-09-10T00:38:39.766036+0000","last_active":"2023-09-10T00:38:43.772282+0000","last_peered":"2023-09-10T00:38:43.772282+0000","last_clean":"2023-09-10T00:38:43.772282+0000","last_became_active":"2023-09-10T00:38:39.765683+0000","last_became_peered":"2023-09-10T00:38:39.765683+0000","last_unstale":"2023-09-10T00:38:43.772282+0000","last_undegraded":"2023-09-10T00:38:43.772282+0000","last_fullsized":"2023-09-10T00:38:43.772282+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:58:04.429960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253407+0000","last_change":"2023-09-10T00:38:43.787870+0000","last_active":"2023-09-10T00:38:46.253407+0000","last_peered":"2023-09-10T00:38:46.253407+0000","last_clean":"2023-09-10T00:38:46.253407+0000","last_became_active":"2023-09-10T00:38:43.785915+0000","last_became_peered":"2023-09-10T00:38:43.785915+0000","last_unstale":"2023-09-10T00:38:46.253407+0000","last_undegraded":"2023-09-10T00:38:46.253407+0000","last_fullsized":"2023-09-10T00:38:46.253407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:34:08.623967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:01.476401+0000","last_change":"2023-09-10T00:38:38.756925+0000","last_active":"2023-09-10T00:39:01.476401+0000","last_peered":"2023-09-10T00:39:01.476401+0000","last_clean":"2023-09-10T00:39:01.476401+0000","last_became_active":"2023-09-10T00:38:38.756610+0000","last_became_peered":"2023-09-10T00:38:38.756610+0000","last_unstale":"2023-09-10T00:39:01.476401+0000","last_undegraded":"2023-09-10T00:39:01.476401+0000","last_fullsized":"2023-09-10T00:39:01.476401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:48:15.385230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772314+0000","last_change":"2023-09-10T00:38:39.766750+0000","last_active":"2023-09-10T00:38:43.772314+0000","last_peered":"2023-09-10T00:38:43.772314+0000","last_clean":"2023-09-10T00:38:43.772314+0000","last_became_active":"2023-09-10T00:38:39.766458+0000","last_became_peered":"2023-09-10T00:38:39.766458+0000","last_unstale":"2023-09-10T00:38:43.772314+0000","last_undegraded":"2023-09-10T00:38:43.772314+0000","last_fullsized":"2023-09-10T00:38:43.772314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:59:26.691487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:44.771371+0000","last_change":"2023-09-10T00:38:43.788814+0000","last_active":"2023-09-10T00:38:44.771371+0000","last_peered":"2023-09-10T00:38:44.771371+0000","last_clean":"2023-09-10T00:38:44.771371+0000","last_became_active":"2023-09-10T00:38:43.788446+0000","last_became_peered":"2023-09-10T00:38:43.788446+0000","last_unstale":"2023-09-10T00:38:44.771371+0000","last_undegraded":"2023-09-10T00:38:44.771371+0000","last_fullsized":"2023-09-10T00:38:44.771371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:38:53.308160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:00.475099+0000","last_change":"2023-09-10T00:38:38.766020+0000","last_active":"2023-09-10T00:39:00.475099+0000","last_peered":"2023-09-10T00:39:00.475099+0000","last_clean":"2023-09-10T00:39:00.475099+0000","last_became_active":"2023-09-10T00:38:38.765719+0000","last_became_peered":"2023-09-10T00:38:38.765719+0000","last_unstale":"2023-09-10T00:39:00.475099+0000","last_undegraded":"2023-09-10T00:39:00.475099+0000","last_fullsized":"2023-09-10T00:39:00.475099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:25:30.488014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.773146+0000","last_change":"2023-09-10T00:38:39.771091+0000","last_active":"2023-09-10T00:38:43.773146+0000","last_peered":"2023-09-10T00:38:43.773146+0000","last_clean":"2023-09-10T00:38:43.773146+0000","last_became_active":"2023-09-10T00:38:39.770745+0000","last_became_peered":"2023-09-10T00:38:39.770745+0000","last_unstale":"2023-09-10T00:38:43.773146+0000","last_undegraded":"2023-09-10T00:38:43.773146+0000","last_fullsized":"2023-09-10T00:38:43.773146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:51:17.978466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.066591+0000","last_change":"2023-09-10T00:38:43.793757+0000","last_active":"2023-09-10T00:38:46.066591+0000","last_peered":"2023-09-10T00:38:46.066591+0000","last_clean":"2023-09-10T00:38:46.066591+0000","last_became_active":"2023-09-10T00:38:43.793336+0000","last_became_peered":"2023-09-10T00:38:43.793336+0000","last_unstale":"2023-09-10T00:38:46.066591+0000","last_undegraded":"2023-09-10T00:38:46.066591+0000","last_fullsized":"2023-09-10T00:38:46.066591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:22:36.544950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:44.771028+0000","last_change":"2023-09-10T00:38:38.769373+0000","last_active":"2023-09-10T00:38:44.771028+0000","last_peered":"2023-09-10T00:38:44.771028+0000","last_clean":"2023-09-10T00:38:44.771028+0000","last_became_active":"2023-09-10T00:38:38.769054+0000","last_became_peered":"2023-09-10T00:38:38.769054+0000","last_unstale":"2023-09-10T00:38:44.771028+0000","last_undegraded":"2023-09-10T00:38:44.771028+0000","last_fullsized":"2023-09-10T00:38:44.771028+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:12:27.982405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778698+0000","last_change":"2023-09-10T00:38:39.766068+0000","last_active":"2023-09-10T00:38:43.778698+0000","last_peered":"2023-09-10T00:38:43.778698+0000","last_clean":"2023-09-10T00:38:43.778698+0000","last_became_active":"2023-09-10T00:38:39.765773+0000","last_became_peered":"2023-09-10T00:38:39.765773+0000","last_unstale":"2023-09-10T00:38:43.778698+0000","last_undegraded":"2023-09-10T00:38:43.778698+0000","last_fullsized":"2023-09-10T00:38:43.778698+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:28:34.172288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:44.770828+0000","last_change":"2023-09-10T00:38:43.838689+0000","last_active":"2023-09-10T00:38:44.770828+0000","last_peered":"2023-09-10T00:38:44.770828+0000","last_clean":"2023-09-10T00:38:44.770828+0000","last_became_active":"2023-09-10T00:38:43.838244+0000","last_became_peered":"2023-09-10T00:38:43.838244+0000","last_unstale":"2023-09-10T00:38:44.770828+0000","last_undegraded":"2023-09-10T00:38:44.770828+0000","last_fullsized":"2023-09-10T00:38:44.770828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:54:00.033719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253225+0000","last_change":"2023-09-10T00:38:38.765140+0000","last_active":"2023-09-10T00:38:46.253225+0000","last_peered":"2023-09-10T00:38:46.253225+0000","last_clean":"2023-09-10T00:38:46.253225+0000","last_became_active":"2023-09-10T00:38:38.764730+0000","last_became_peered":"2023-09-10T00:38:38.764730+0000","last_unstale":"2023-09-10T00:38:46.253225+0000","last_undegraded":"2023-09-10T00:38:46.253225+0000","last_fullsized":"2023-09-10T00:38:46.253225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:49:41.129763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.837565+0000","last_change":"2023-09-10T00:38:39.773260+0000","last_active":"2023-09-10T00:38:43.837565+0000","last_peered":"2023-09-10T00:38:43.837565+0000","last_clean":"2023-09-10T00:38:43.837565+0000","last_became_active":"2023-09-10T00:38:39.772947+0000","last_became_peered":"2023-09-10T00:38:39.772947+0000","last_unstale":"2023-09-10T00:38:43.837565+0000","last_undegraded":"2023-09-10T00:38:43.837565+0000","last_fullsized":"2023-09-10T00:38:43.837565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:08:10.791610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:44.771317+0000","last_change":"2023-09-10T00:38:43.787882+0000","last_active":"2023-09-10T00:38:44.771317+0000","last_peered":"2023-09-10T00:38:44.771317+0000","last_clean":"2023-09-10T00:38:44.771317+0000","last_became_active":"2023-09-10T00:38:43.787257+0000","last_became_peered":"2023-09-10T00:38:43.787257+0000","last_unstale":"2023-09-10T00:38:44.771317+0000","last_undegraded":"2023-09-10T00:38:44.771317+0000","last_fullsized":"2023-09-10T00:38:44.771317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:11:19.171881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:01.483506+0000","last_change":"2023-09-10T00:38:38.760014+0000","last_active":"2023-09-10T00:39:01.483506+0000","last_peered":"2023-09-10T00:39:01.483506+0000","last_clean":"2023-09-10T00:39:01.483506+0000","last_became_active":"2023-09-10T00:38:38.759637+0000","last_became_peered":"2023-09-10T00:38:38.759637+0000","last_unstale":"2023-09-10T00:39:01.483506+0000","last_undegraded":"2023-09-10T00:39:01.483506+0000","last_fullsized":"2023-09-10T00:39:01.483506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:15.704929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.837304+0000","last_change":"2023-09-10T00:38:39.765800+0000","last_active":"2023-09-10T00:38:43.837304+0000","last_peered":"2023-09-10T00:38:43.837304+0000","last_clean":"2023-09-10T00:38:43.837304+0000","last_became_active":"2023-09-10T00:38:39.765468+0000","last_became_peered":"2023-09-10T00:38:39.765468+0000","last_unstale":"2023-09-10T00:38:43.837304+0000","last_undegraded":"2023-09-10T00:38:43.837304+0000","last_fullsized":"2023-09-10T00:38:43.837304+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:57:56.429365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:44.771364+0000","last_change":"2023-09-10T00:38:43.838285+0000","last_active":"2023-09-10T00:38:44.771364+0000","last_peered":"2023-09-10T00:38:44.771364+0000","last_clean":"2023-09-10T00:38:44.771364+0000","last_became_active":"2023-09-10T00:38:43.837821+0000","last_became_peered":"2023-09-10T00:38:43.837821+0000","last_unstale":"2023-09-10T00:38:44.771364+0000","last_undegraded":"2023-09-10T00:38:44.771364+0000","last_fullsized":"2023-09-10T00:38:44.771364+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:01:03.354281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.773101+0000","last_change":"2023-09-10T00:38:39.763480+0000","last_active":"2023-09-10T00:38:43.773101+0000","last_peered":"2023-09-10T00:38:43.773101+0000","last_clean":"2023-09-10T00:38:43.773101+0000","last_became_active":"2023-09-10T00:38:39.763177+0000","last_became_peered":"2023-09-10T00:38:39.763177+0000","last_unstale":"2023-09-10T00:38:43.773101+0000","last_undegraded":"2023-09-10T00:38:43.773101+0000","last_fullsized":"2023-09-10T00:38:43.773101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:44:14.800236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:59.854768+0000","last_change":"2023-09-10T00:38:38.758745+0000","last_active":"2023-09-10T00:38:59.854768+0000","last_peered":"2023-09-10T00:38:59.854768+0000","last_clean":"2023-09-10T00:38:59.854768+0000","last_became_active":"2023-09-10T00:38:38.758427+0000","last_became_peered":"2023-09-10T00:38:38.758427+0000","last_unstale":"2023-09-10T00:38:59.854768+0000","last_undegraded":"2023-09-10T00:38:59.854768+0000","last_fullsized":"2023-09-10T00:38:59.854768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:02:49.238175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.658168+0000","last_change":"2023-09-10T00:38:43.784058+0000","last_active":"2023-09-10T00:38:46.658168+0000","last_peered":"2023-09-10T00:38:46.658168+0000","last_clean":"2023-09-10T00:38:46.658168+0000","last_became_active":"2023-09-10T00:38:43.783571+0000","last_became_peered":"2023-09-10T00:38:43.783571+0000","last_unstale":"2023-09-10T00:38:46.658168+0000","last_undegraded":"2023-09-10T00:38:46.658168+0000","last_fullsized":"2023-09-10T00:38:46.658168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:53:50.992393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.779510+0000","last_change":"2023-09-10T00:38:39.775659+0000","last_active":"2023-09-10T00:38:43.779510+0000","last_peered":"2023-09-10T00:38:43.779510+0000","last_clean":"2023-09-10T00:38:43.779510+0000","last_became_active":"2023-09-10T00:38:39.775151+0000","last_became_peered":"2023-09-10T00:38:39.775151+0000","last_unstale":"2023-09-10T00:38:43.779510+0000","last_undegraded":"2023-09-10T00:38:43.779510+0000","last_fullsized":"2023-09-10T00:38:43.779510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:07.415418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:01.481424+0000","last_change":"2023-09-10T00:38:38.755009+0000","last_active":"2023-09-10T00:39:01.481424+0000","last_peered":"2023-09-10T00:39:01.481424+0000","last_clean":"2023-09-10T00:39:01.481424+0000","last_became_active":"2023-09-10T00:38:38.754699+0000","last_became_peered":"2023-09-10T00:38:38.754699+0000","last_unstale":"2023-09-10T00:39:01.481424+0000","last_undegraded":"2023-09-10T00:39:01.481424+0000","last_fullsized":"2023-09-10T00:39:01.481424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:12:02.075256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":320,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772183+0000","last_change":"2023-09-10T00:37:38.459695+0000","last_active":"2023-09-10T00:38:43.772183+0000","last_peered":"2023-09-10T00:38:43.772183+0000","last_clean":"2023-09-10T00:38:43.772183+0000","last_became_active":"2023-09-10T00:37:38.459220+0000","last_became_peered":"2023-09-10T00:37:38.459220+0000","last_unstale":"2023-09-10T00:38:43.772183+0000","last_undegraded":"2023-09-10T00:38:43.772183+0000","last_fullsized":"2023-09-10T00:38:43.772183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:34:44.602334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:34:44.602334+0000","last_clean_scrub_stamp":"2023-09-10T00:34:44.602334+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:21:32.648725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.066939+0000","last_change":"2023-09-10T00:38:43.839001+0000","last_active":"2023-09-10T00:38:46.066939+0000","last_peered":"2023-09-10T00:38:46.066939+0000","last_clean":"2023-09-10T00:38:46.066939+0000","last_became_active":"2023-09-10T00:38:43.838497+0000","last_became_peered":"2023-09-10T00:38:43.838497+0000","last_unstale":"2023-09-10T00:38:46.066939+0000","last_undegraded":"2023-09-10T00:38:46.066939+0000","last_fullsized":"2023-09-10T00:38:46.066939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:57:23.349632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.773180+0000","last_change":"2023-09-10T00:38:39.760150+0000","last_active":"2023-09-10T00:38:43.773180+0000","last_peered":"2023-09-10T00:38:43.773180+0000","last_clean":"2023-09-10T00:38:43.773180+0000","last_became_active":"2023-09-10T00:38:39.759811+0000","last_became_peered":"2023-09-10T00:38:39.759811+0000","last_unstale":"2023-09-10T00:38:43.773180+0000","last_undegraded":"2023-09-10T00:38:43.773180+0000","last_fullsized":"2023-09-10T00:38:43.773180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:21:19.359057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:59.853234+0000","last_change":"2023-09-10T00:38:38.771066+0000","last_active":"2023-09-10T00:38:59.853234+0000","last_peered":"2023-09-10T00:38:59.853234+0000","last_clean":"2023-09-10T00:38:59.853234+0000","last_became_active":"2023-09-10T00:38:38.770810+0000","last_became_peered":"2023-09-10T00:38:38.770810+0000","last_unstale":"2023-09-10T00:38:59.853234+0000","last_undegraded":"2023-09-10T00:38:59.853234+0000","last_fullsized":"2023-09-10T00:38:59.853234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:55:06.923842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253778+0000","last_change":"2023-09-10T00:38:43.786272+0000","last_active":"2023-09-10T00:38:46.253778+0000","last_peered":"2023-09-10T00:38:46.253778+0000","last_clean":"2023-09-10T00:38:46.253778+0000","last_became_active":"2023-09-10T00:38:43.785934+0000","last_became_peered":"2023-09-10T00:38:43.785934+0000","last_unstale":"2023-09-10T00:38:46.253778+0000","last_undegraded":"2023-09-10T00:38:46.253778+0000","last_fullsized":"2023-09-10T00:38:46.253778+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:31:49.721133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:00.476223+0000","last_change":"2023-09-10T00:38:38.763263+0000","last_active":"2023-09-10T00:39:00.476223+0000","last_peered":"2023-09-10T00:39:00.476223+0000","last_clean":"2023-09-10T00:39:00.476223+0000","last_became_active":"2023-09-10T00:38:38.762911+0000","last_became_peered":"2023-09-10T00:38:38.762911+0000","last_unstale":"2023-09-10T00:39:00.476223+0000","last_undegraded":"2023-09-10T00:39:00.476223+0000","last_fullsized":"2023-09-10T00:39:00.476223+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:19:10.292311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.774636+0000","last_change":"2023-09-10T00:38:39.759171+0000","last_active":"2023-09-10T00:38:43.774636+0000","last_peered":"2023-09-10T00:38:43.774636+0000","last_clean":"2023-09-10T00:38:43.774636+0000","last_became_active":"2023-09-10T00:38:39.758843+0000","last_became_peered":"2023-09-10T00:38:39.758843+0000","last_unstale":"2023-09-10T00:38:43.774636+0000","last_undegraded":"2023-09-10T00:38:43.774636+0000","last_fullsized":"2023-09-10T00:38:43.774636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:50:36.064937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253682+0000","last_change":"2023-09-10T00:38:43.837998+0000","last_active":"2023-09-10T00:38:46.253682+0000","last_peered":"2023-09-10T00:38:46.253682+0000","last_clean":"2023-09-10T00:38:46.253682+0000","last_became_active":"2023-09-10T00:38:43.837556+0000","last_became_peered":"2023-09-10T00:38:43.837556+0000","last_unstale":"2023-09-10T00:38:46.253682+0000","last_undegraded":"2023-09-10T00:38:46.253682+0000","last_fullsized":"2023-09-10T00:38:46.253682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:18:40.488279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:00.475148+0000","last_change":"2023-09-10T00:38:38.759819+0000","last_active":"2023-09-10T00:39:00.475148+0000","last_peered":"2023-09-10T00:39:00.475148+0000","last_clean":"2023-09-10T00:39:00.475148+0000","last_became_active":"2023-09-10T00:38:38.759490+0000","last_became_peered":"2023-09-10T00:38:38.759490+0000","last_unstale":"2023-09-10T00:39:00.475148+0000","last_undegraded":"2023-09-10T00:39:00.475148+0000","last_fullsized":"2023-09-10T00:39:00.475148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:39:01.206817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.787641+0000","last_change":"2023-09-10T00:38:39.769669+0000","last_active":"2023-09-10T00:38:43.787641+0000","last_peered":"2023-09-10T00:38:43.787641+0000","last_clean":"2023-09-10T00:38:43.787641+0000","last_became_active":"2023-09-10T00:38:39.769402+0000","last_became_peered":"2023-09-10T00:38:39.769402+0000","last_unstale":"2023-09-10T00:38:43.787641+0000","last_undegraded":"2023-09-10T00:38:43.787641+0000","last_fullsized":"2023-09-10T00:38:43.787641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:00:50.215103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253291+0000","last_change":"2023-09-10T00:38:43.787518+0000","last_active":"2023-09-10T00:38:46.253291+0000","last_peered":"2023-09-10T00:38:46.253291+0000","last_clean":"2023-09-10T00:38:46.253291+0000","last_became_active":"2023-09-10T00:38:43.786410+0000","last_became_peered":"2023-09-10T00:38:43.786410+0000","last_unstale":"2023-09-10T00:38:46.253291+0000","last_undegraded":"2023-09-10T00:38:46.253291+0000","last_fullsized":"2023-09-10T00:38:46.253291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:13:37.903875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:01.482196+0000","last_change":"2023-09-10T00:38:38.760992+0000","last_active":"2023-09-10T00:39:01.482196+0000","last_peered":"2023-09-10T00:39:01.482196+0000","last_clean":"2023-09-10T00:39:01.482196+0000","last_became_active":"2023-09-10T00:38:38.760635+0000","last_became_peered":"2023-09-10T00:38:38.760635+0000","last_unstale":"2023-09-10T00:39:01.482196+0000","last_undegraded":"2023-09-10T00:39:01.482196+0000","last_fullsized":"2023-09-10T00:39:01.482196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:29:36.070477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778256+0000","last_change":"2023-09-10T00:38:39.773771+0000","last_active":"2023-09-10T00:38:43.778256+0000","last_peered":"2023-09-10T00:38:43.778256+0000","last_clean":"2023-09-10T00:38:43.778256+0000","last_became_active":"2023-09-10T00:38:39.773296+0000","last_became_peered":"2023-09-10T00:38:39.773296+0000","last_unstale":"2023-09-10T00:38:43.778256+0000","last_undegraded":"2023-09-10T00:38:43.778256+0000","last_fullsized":"2023-09-10T00:38:43.778256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:13:01.194358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.254303+0000","last_change":"2023-09-10T00:38:43.839879+0000","last_active":"2023-09-10T00:38:46.254303+0000","last_peered":"2023-09-10T00:38:46.254303+0000","last_clean":"2023-09-10T00:38:46.254303+0000","last_became_active":"2023-09-10T00:38:43.839167+0000","last_became_peered":"2023-09-10T00:38:43.839167+0000","last_unstale":"2023-09-10T00:38:46.254303+0000","last_undegraded":"2023-09-10T00:38:46.254303+0000","last_fullsized":"2023-09-10T00:38:46.254303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:11:44.779730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:58.842766+0000","last_change":"2023-09-10T00:38:38.764954+0000","last_active":"2023-09-10T00:38:58.842766+0000","last_peered":"2023-09-10T00:38:58.842766+0000","last_clean":"2023-09-10T00:38:58.842766+0000","last_became_active":"2023-09-10T00:38:38.764589+0000","last_became_peered":"2023-09-10T00:38:38.764589+0000","last_unstale":"2023-09-10T00:38:58.842766+0000","last_undegraded":"2023-09-10T00:38:58.842766+0000","last_fullsized":"2023-09-10T00:38:58.842766+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:40:19.559285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.775598+0000","last_change":"2023-09-10T00:38:39.766141+0000","last_active":"2023-09-10T00:38:43.775598+0000","last_peered":"2023-09-10T00:38:43.775598+0000","last_clean":"2023-09-10T00:38:43.775598+0000","last_became_active":"2023-09-10T00:38:39.765832+0000","last_became_peered":"2023-09-10T00:38:39.765832+0000","last_unstale":"2023-09-10T00:38:43.775598+0000","last_undegraded":"2023-09-10T00:38:43.775598+0000","last_fullsized":"2023-09-10T00:38:43.775598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:08:45.149973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253630+0000","last_change":"2023-09-10T00:38:43.785111+0000","last_active":"2023-09-10T00:38:46.253630+0000","last_peered":"2023-09-10T00:38:46.253630+0000","last_clean":"2023-09-10T00:38:46.253630+0000","last_became_active":"2023-09-10T00:38:43.784542+0000","last_became_peered":"2023-09-10T00:38:43.784542+0000","last_unstale":"2023-09-10T00:38:46.253630+0000","last_undegraded":"2023-09-10T00:38:46.253630+0000","last_fullsized":"2023-09-10T00:38:46.253630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:34:51.611608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:00.475001+0000","last_change":"2023-09-10T00:38:38.759984+0000","last_active":"2023-09-10T00:39:00.475001+0000","last_peered":"2023-09-10T00:39:00.475001+0000","last_clean":"2023-09-10T00:39:00.475001+0000","last_became_active":"2023-09-10T00:38:38.759660+0000","last_became_peered":"2023-09-10T00:38:38.759660+0000","last_unstale":"2023-09-10T00:39:00.475001+0000","last_undegraded":"2023-09-10T00:39:00.475001+0000","last_fullsized":"2023-09-10T00:39:00.475001+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:02:11.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.776657+0000","last_change":"2023-09-10T00:38:39.770765+0000","last_active":"2023-09-10T00:38:43.776657+0000","last_peered":"2023-09-10T00:38:43.776657+0000","last_clean":"2023-09-10T00:38:43.776657+0000","last_became_active":"2023-09-10T00:38:39.770483+0000","last_became_peered":"2023-09-10T00:38:39.770483+0000","last_unstale":"2023-09-10T00:38:43.776657+0000","last_undegraded":"2023-09-10T00:38:43.776657+0000","last_fullsized":"2023-09-10T00:38:43.776657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:57:28.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.254094+0000","last_change":"2023-09-10T00:38:43.790945+0000","last_active":"2023-09-10T00:38:46.254094+0000","last_peered":"2023-09-10T00:38:46.254094+0000","last_clean":"2023-09-10T00:38:46.254094+0000","last_became_active":"2023-09-10T00:38:43.790467+0000","last_became_peered":"2023-09-10T00:38:43.790467+0000","last_unstale":"2023-09-10T00:38:46.254094+0000","last_undegraded":"2023-09-10T00:38:46.254094+0000","last_fullsized":"2023-09-10T00:38:46.254094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:44:07.041302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:00.473318+0000","last_change":"2023-09-10T00:38:38.765912+0000","last_active":"2023-09-10T00:39:00.473318+0000","last_peered":"2023-09-10T00:39:00.473318+0000","last_clean":"2023-09-10T00:39:00.473318+0000","last_became_active":"2023-09-10T00:38:38.765586+0000","last_became_peered":"2023-09-10T00:38:38.765586+0000","last_unstale":"2023-09-10T00:39:00.473318+0000","last_undegraded":"2023-09-10T00:39:00.473318+0000","last_fullsized":"2023-09-10T00:39:00.473318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:25:05.974158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.780661+0000","last_change":"2023-09-10T00:38:39.771462+0000","last_active":"2023-09-10T00:38:43.780661+0000","last_peered":"2023-09-10T00:38:43.780661+0000","last_clean":"2023-09-10T00:38:43.780661+0000","last_became_active":"2023-09-10T00:38:39.771153+0000","last_became_peered":"2023-09-10T00:38:39.771153+0000","last_unstale":"2023-09-10T00:38:43.780661+0000","last_undegraded":"2023-09-10T00:38:43.780661+0000","last_fullsized":"2023-09-10T00:38:43.780661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:03:35.312186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.253881+0000","last_change":"2023-09-10T00:38:43.838446+0000","last_active":"2023-09-10T00:38:46.253881+0000","last_peered":"2023-09-10T00:38:46.253881+0000","last_clean":"2023-09-10T00:38:46.253881+0000","last_became_active":"2023-09-10T00:38:43.838119+0000","last_became_peered":"2023-09-10T00:38:43.838119+0000","last_unstale":"2023-09-10T00:38:46.253881+0000","last_undegraded":"2023-09-10T00:38:46.253881+0000","last_fullsized":"2023-09-10T00:38:46.253881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:12:20.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:58.841558+0000","last_change":"2023-09-10T00:38:38.765548+0000","last_active":"2023-09-10T00:38:58.841558+0000","last_peered":"2023-09-10T00:38:58.841558+0000","last_clean":"2023-09-10T00:38:58.841558+0000","last_became_active":"2023-09-10T00:38:38.765281+0000","last_became_peered":"2023-09-10T00:38:38.765281+0000","last_unstale":"2023-09-10T00:38:58.841558+0000","last_undegraded":"2023-09-10T00:38:58.841558+0000","last_fullsized":"2023-09-10T00:38:58.841558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:50:44.422327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772694+0000","last_change":"2023-09-10T00:38:39.771995+0000","last_active":"2023-09-10T00:38:43.772694+0000","last_peered":"2023-09-10T00:38:43.772694+0000","last_clean":"2023-09-10T00:38:43.772694+0000","last_became_active":"2023-09-10T00:38:39.771693+0000","last_became_peered":"2023-09-10T00:38:39.771693+0000","last_unstale":"2023-09-10T00:38:43.772694+0000","last_undegraded":"2023-09-10T00:38:43.772694+0000","last_fullsized":"2023-09-10T00:38:43.772694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:41:45.621060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:44.771267+0000","last_change":"2023-09-10T00:38:43.789159+0000","last_active":"2023-09-10T00:38:44.771267+0000","last_peered":"2023-09-10T00:38:44.771267+0000","last_clean":"2023-09-10T00:38:44.771267+0000","last_became_active":"2023-09-10T00:38:43.788675+0000","last_became_peered":"2023-09-10T00:38:43.788675+0000","last_unstale":"2023-09-10T00:38:44.771267+0000","last_undegraded":"2023-09-10T00:38:44.771267+0000","last_fullsized":"2023-09-10T00:38:44.771267+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:30:39.545561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:39:02.492475+0000","last_change":"2023-09-10T00:38:38.767245+0000","last_active":"2023-09-10T00:39:02.492475+0000","last_peered":"2023-09-10T00:39:02.492475+0000","last_clean":"2023-09-10T00:39:02.492475+0000","last_became_active":"2023-09-10T00:38:38.766952+0000","last_became_peered":"2023-09-10T00:38:38.766952+0000","last_unstale":"2023-09-10T00:39:02.492475+0000","last_undegraded":"2023-09-10T00:39:02.492475+0000","last_fullsized":"2023-09-10T00:39:02.492475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:57:23.309838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772636+0000","last_change":"2023-09-10T00:38:39.772199+0000","last_active":"2023-09-10T00:38:43.772636+0000","last_peered":"2023-09-10T00:38:43.772636+0000","last_clean":"2023-09-10T00:38:43.772636+0000","last_became_active":"2023-09-10T00:38:39.771917+0000","last_became_peered":"2023-09-10T00:38:39.771917+0000","last_unstale":"2023-09-10T00:38:43.772636+0000","last_undegraded":"2023-09-10T00:38:43.772636+0000","last_fullsized":"2023-09-10T00:38:43.772636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:00:52.824581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:45.752923+0000","last_change":"2023-09-10T00:38:43.797107+0000","last_active":"2023-09-10T00:38:45.752923+0000","last_peered":"2023-09-10T00:38:45.752923+0000","last_clean":"2023-09-10T00:38:45.752923+0000","last_became_active":"2023-09-10T00:38:43.796606+0000","last_became_peered":"2023-09-10T00:38:43.796606+0000","last_unstale":"2023-09-10T00:38:45.752923+0000","last_undegraded":"2023-09-10T00:38:45.752923+0000","last_fullsized":"2023-09-10T00:38:45.752923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:17:00.444238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.773571+0000","last_change":"2023-09-10T00:38:38.768045+0000","last_active":"2023-09-10T00:38:43.773571+0000","last_peered":"2023-09-10T00:38:43.773571+0000","last_clean":"2023-09-10T00:38:43.773571+0000","last_became_active":"2023-09-10T00:38:38.767344+0000","last_became_peered":"2023-09-10T00:38:38.767344+0000","last_unstale":"2023-09-10T00:38:43.773571+0000","last_undegraded":"2023-09-10T00:38:43.773571+0000","last_fullsized":"2023-09-10T00:38:43.773571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:51:08.765412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772316+0000","last_change":"2023-09-10T00:38:39.772931+0000","last_active":"2023-09-10T00:38:43.772316+0000","last_peered":"2023-09-10T00:38:43.772316+0000","last_clean":"2023-09-10T00:38:43.772316+0000","last_became_active":"2023-09-10T00:38:39.772631+0000","last_became_peered":"2023-09-10T00:38:39.772631+0000","last_unstale":"2023-09-10T00:38:43.772316+0000","last_undegraded":"2023-09-10T00:38:43.772316+0000","last_fullsized":"2023-09-10T00:38:43.772316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:12:23.948287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:46.066355+0000","last_change":"2023-09-10T00:38:43.784928+0000","last_active":"2023-09-10T00:38:46.066355+0000","last_peered":"2023-09-10T00:38:46.066355+0000","last_clean":"2023-09-10T00:38:46.066355+0000","last_became_active":"2023-09-10T00:38:43.784479+0000","last_became_peered":"2023-09-10T00:38:43.784479+0000","last_unstale":"2023-09-10T00:38:46.066355+0000","last_undegraded":"2023-09-10T00:38:46.066355+0000","last_fullsized":"2023-09-10T00:38:46.066355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:14:45.019988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-10T00:38:44.771265+0000","last_change":"2023-09-10T00:38:38.770730+0000","last_active":"2023-09-10T00:38:44.771265+0000","last_peered":"2023-09-10T00:38:44.771265+0000","last_clean":"2023-09-10T00:38:44.771265+0000","last_became_active":"2023-09-10T00:38:38.770344+0000","last_became_peered":"2023-09-10T00:38:38.770344+0000","last_unstale":"2023-09-10T00:38:44.771265+0000","last_undegraded":"2023-09-10T00:38:44.771265+0000","last_fullsized":"2023-09-10T00:38:44.771265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:49:22.881956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.772728+0000","last_change":"2023-09-10T00:38:39.766149+0000","last_active":"2023-09-10T00:38:43.772728+0000","last_peered":"2023-09-10T00:38:43.772728+0000","last_clean":"2023-09-10T00:38:43.772728+0000","last_became_active":"2023-09-10T00:38:39.765773+0000","last_became_peered":"2023-09-10T00:38:39.765773+0000","last_unstale":"2023-09-10T00:38:43.772728+0000","last_undegraded":"2023-09-10T00:38:43.772728+0000","last_fullsized":"2023-09-10T00:38:43.772728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:00:37.854168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.776553+0000","last_change":"2023-09-10T00:38:39.765197+0000","last_active":"2023-09-10T00:38:43.776553+0000","last_peered":"2023-09-10T00:38:43.776553+0000","last_clean":"2023-09-10T00:38:43.776553+0000","last_became_active":"2023-09-10T00:38:39.764891+0000","last_became_peered":"2023-09-10T00:38:39.764891+0000","last_unstale":"2023-09-10T00:38:43.776553+0000","last_undegraded":"2023-09-10T00:38:43.776553+0000","last_fullsized":"2023-09-10T00:38:43.776553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:47:27.429355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.781389+0000","last_change":"2023-09-10T00:38:39.764722+0000","last_active":"2023-09-10T00:38:43.781389+0000","last_peered":"2023-09-10T00:38:43.781389+0000","last_clean":"2023-09-10T00:38:43.781389+0000","last_became_active":"2023-09-10T00:38:39.764376+0000","last_became_peered":"2023-09-10T00:38:39.764376+0000","last_unstale":"2023-09-10T00:38:43.781389+0000","last_undegraded":"2023-09-10T00:38:43.781389+0000","last_fullsized":"2023-09-10T00:38:43.781389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:27:36.708573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.776325+0000","last_change":"2023-09-10T00:38:39.771997+0000","last_active":"2023-09-10T00:38:43.776325+0000","last_peered":"2023-09-10T00:38:43.776325+0000","last_clean":"2023-09-10T00:38:43.776325+0000","last_became_active":"2023-09-10T00:38:39.771588+0000","last_became_peered":"2023-09-10T00:38:39.771588+0000","last_unstale":"2023-09-10T00:38:43.776325+0000","last_undegraded":"2023-09-10T00:38:43.776325+0000","last_fullsized":"2023-09-10T00:38:43.776325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:20:07.171333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778627+0000","last_change":"2023-09-10T00:38:39.773393+0000","last_active":"2023-09-10T00:38:43.778627+0000","last_peered":"2023-09-10T00:38:43.778627+0000","last_clean":"2023-09-10T00:38:43.778627+0000","last_became_active":"2023-09-10T00:38:39.772972+0000","last_became_peered":"2023-09-10T00:38:39.772972+0000","last_unstale":"2023-09-10T00:38:43.778627+0000","last_undegraded":"2023-09-10T00:38:43.778627+0000","last_fullsized":"2023-09-10T00:38:43.778627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:27:29.916970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.775135+0000","last_change":"2023-09-10T00:38:39.767871+0000","last_active":"2023-09-10T00:38:43.775135+0000","last_peered":"2023-09-10T00:38:43.775135+0000","last_clean":"2023-09-10T00:38:43.775135+0000","last_became_active":"2023-09-10T00:38:39.767563+0000","last_became_peered":"2023-09-10T00:38:39.767563+0000","last_unstale":"2023-09-10T00:38:43.775135+0000","last_undegraded":"2023-09-10T00:38:43.775135+0000","last_fullsized":"2023-09-10T00:38:43.775135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:22:30.400441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778083+0000","last_change":"2023-09-10T00:38:39.764664+0000","last_active":"2023-09-10T00:38:43.778083+0000","last_peered":"2023-09-10T00:38:43.778083+0000","last_clean":"2023-09-10T00:38:43.778083+0000","last_became_active":"2023-09-10T00:38:39.764364+0000","last_became_peered":"2023-09-10T00:38:39.764364+0000","last_unstale":"2023-09-10T00:38:43.778083+0000","last_undegraded":"2023-09-10T00:38:43.778083+0000","last_fullsized":"2023-09-10T00:38:43.778083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:41.656472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778638+0000","last_change":"2023-09-10T00:38:39.770315+0000","last_active":"2023-09-10T00:38:43.778638+0000","last_peered":"2023-09-10T00:38:43.778638+0000","last_clean":"2023-09-10T00:38:43.778638+0000","last_became_active":"2023-09-10T00:38:39.769977+0000","last_became_peered":"2023-09-10T00:38:39.769977+0000","last_unstale":"2023-09-10T00:38:43.778638+0000","last_undegraded":"2023-09-10T00:38:43.778638+0000","last_fullsized":"2023-09-10T00:38:43.778638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:30:13.380487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.787768+0000","last_change":"2023-09-10T00:38:39.768734+0000","last_active":"2023-09-10T00:38:43.787768+0000","last_peered":"2023-09-10T00:38:43.787768+0000","last_clean":"2023-09-10T00:38:43.787768+0000","last_became_active":"2023-09-10T00:38:39.768383+0000","last_became_peered":"2023-09-10T00:38:39.768383+0000","last_unstale":"2023-09-10T00:38:43.787768+0000","last_undegraded":"2023-09-10T00:38:43.787768+0000","last_fullsized":"2023-09-10T00:38:43.787768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:09.974447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.773027+0000","last_change":"2023-09-10T00:38:39.773294+0000","last_active":"2023-09-10T00:38:43.773027+0000","last_peered":"2023-09-10T00:38:43.773027+0000","last_clean":"2023-09-10T00:38:43.773027+0000","last_became_active":"2023-09-10T00:38:39.772925+0000","last_became_peered":"2023-09-10T00:38:39.772925+0000","last_unstale":"2023-09-10T00:38:43.773027+0000","last_undegraded":"2023-09-10T00:38:43.773027+0000","last_fullsized":"2023-09-10T00:38:43.773027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:58:27.525511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.782371+0000","last_change":"2023-09-10T00:38:39.773839+0000","last_active":"2023-09-10T00:38:43.782371+0000","last_peered":"2023-09-10T00:38:43.782371+0000","last_clean":"2023-09-10T00:38:43.782371+0000","last_became_active":"2023-09-10T00:38:39.773521+0000","last_became_peered":"2023-09-10T00:38:39.773521+0000","last_unstale":"2023-09-10T00:38:43.782371+0000","last_undegraded":"2023-09-10T00:38:43.782371+0000","last_fullsized":"2023-09-10T00:38:43.782371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:15:49.033374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.779484+0000","last_change":"2023-09-10T00:38:39.775721+0000","last_active":"2023-09-10T00:38:43.779484+0000","last_peered":"2023-09-10T00:38:43.779484+0000","last_clean":"2023-09-10T00:38:43.779484+0000","last_became_active":"2023-09-10T00:38:39.775424+0000","last_became_peered":"2023-09-10T00:38:39.775424+0000","last_unstale":"2023-09-10T00:38:43.779484+0000","last_undegraded":"2023-09-10T00:38:43.779484+0000","last_fullsized":"2023-09-10T00:38:43.779484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:56:26.079332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.774663+0000","last_change":"2023-09-10T00:38:39.768767+0000","last_active":"2023-09-10T00:38:43.774663+0000","last_peered":"2023-09-10T00:38:43.774663+0000","last_clean":"2023-09-10T00:38:43.774663+0000","last_became_active":"2023-09-10T00:38:39.768452+0000","last_became_peered":"2023-09-10T00:38:39.768452+0000","last_unstale":"2023-09-10T00:38:43.774663+0000","last_undegraded":"2023-09-10T00:38:43.774663+0000","last_fullsized":"2023-09-10T00:38:43.774663+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:31:53.578574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-10T00:38:43.778044+0000","last_change":"2023-09-10T00:38:39.761998+0000","last_active":"2023-09-10T00:38:43.778044+0000","last_peered":"2023-09-10T00:38:43.778044+0000","last_clean":"2023-09-10T00:38:43.778044+0000","last_became_active":"2023-09-10T00:38:39.761672+0000","last_became_peered":"2023-09-10T00:38:39.761672+0000","last_unstale":"2023-09-10T00:38:43.778044+0000","last_undegraded":"2023-09-10T00:38:43.778044+0000","last_fullsized":"2023-09-10T00:38:43.778044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:03:24.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9780,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":179,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":54,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":103437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449103,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":471521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.35199999999999998}]},{"osd":1,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.47099999999999997}]},{"osd":2,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":4,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.503},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.38}]},{"osd":6,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.60999999999999999}]},{"osd":7,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63900000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":9,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.67800000000000005}]},{"osd":10,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.73799999999999999}]}]},{"osd":10,"up_from":70,"seq":300647710739,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1061399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.442}]},{"osd":2,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.65100000000000002}]},{"osd":3,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.626}]},{"osd":4,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60199999999999998}]},{"osd":5,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.752}]},{"osd":6,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71299999999999997}]},{"osd":8,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.82499999999999996}]},{"osd":9,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71799999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]}]},{"osd":9,"up_from":65,"seq":279172874264,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":467051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.313}]},{"osd":1,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.36299999999999999}]},{"osd":2,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46300000000000002}]},{"osd":3,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.48299999999999998}]},{"osd":4,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.54400000000000004}]},{"osd":5,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.61299999999999999}]},{"osd":6,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.42499999999999999}]},{"osd":7,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.64500000000000002}]},{"osd":8,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.59799999999999998}]},{"osd":10,"last update":"Sun Sep 10 00:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.628}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":8,"up_from":57,"seq":244813135901,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":471498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57199999999999995,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.67800000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.37}]},{"osd":1,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.67500000000000004,"15min":0.628},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":6.5759999999999996,"5min":6.5759999999999996,"15min":6.5759999999999996},"last":0.60299999999999998}]},{"osd":2,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.73299999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.34899999999999998}]},{"osd":3,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.435},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83099999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.39400000000000002}]},{"osd":4,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.41199999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.439,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.60999999999999999}]},{"osd":5,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.34699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94399999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.504}]},{"osd":6,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.79100000000000004,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":6.5940000000000003,"5min":6.5940000000000003,"15min":6.5940000000000003},"last":0.51800000000000002}]},{"osd":7,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.76000000000000001,"15min":0.70999999999999996},"min":{"1min":0.51300000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.5439999999999996,"5min":6.5439999999999996,"15min":6.5439999999999996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.441,"5min":0.435,"15min":0.435},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.622}]},{"osd":9,"last update":"Sun Sep 10 00:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66200000000000003}]},{"osd":10,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.70299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]}]},{"osd":7,"up_from":50,"seq":214748364831,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1058734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54100000000000004,"15min":0.53200000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.39800000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.27600000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.60499999999999998,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.28299999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.75800000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.36699999999999999}]},{"osd":2,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.59199999999999997}]},{"osd":3,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.38400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.37,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73599999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.50800000000000001}]},{"osd":4,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.32300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78400000000000003,"5min":1.056,"15min":1.056},"last":0.34899999999999998}]},{"osd":5,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.629,"15min":0.627},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73499999999999999}]},{"osd":6,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65300000000000002,"15min":0.65000000000000002},"min":{"1min":0.50800000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.87,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.877,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.434}]},{"osd":8,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.63500000000000001,"15min":0.62},"min":{"1min":0.50600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.66200000000000003,"15min":0.65200000000000002},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.66900000000000004}]},{"osd":9,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.62}]},{"osd":10,"last update":"Sun Sep 10 00:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.65200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528356,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":466323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.41199999999999998,"15min":0.40000000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.33800000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.33800000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.57499999999999996,"15min":0.55600000000000005},"min":{"1min":0.436,"5min":0.378,"15min":0.378},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.48999999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.49099999999999999}]},{"osd":2,"last update":"Sun Sep 10 00:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.441,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.432,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.624}]},{"osd":3,"last update":"Sun Sep 10 00:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.33700000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.442}]},{"osd":4,"last update":"Sun Sep 10 00:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65400000000000003,"15min":0.64500000000000002},"min":{"1min":0.47799999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.69799999999999995,"15min":0.68200000000000005},"min":{"1min":0.51500000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.50700000000000001}]},{"osd":5,"last update":"Sun Sep 10 00:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69199999999999995,"15min":0.68000000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.56499999999999995,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.74099999999999999,"15min":0.73199999999999998},"min":{"1min":0.53600000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.73699999999999999,"15min":0.72499999999999998},"min":{"1min":0.52100000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.53200000000000003}]},{"osd":8,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.81499999999999995,"15min":0.83299999999999996},"min":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.999,"5min":1.27,"15min":1.27},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.81699999999999995,"15min":0.82799999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0389999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.64700000000000002}]},{"osd":9,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.54000000000000004}]},{"osd":10,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.65400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":489208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.52500000000000002,"15min":0.50900000000000001},"min":{"1min":0.32600000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.501},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52700000000000002}]},{"osd":2,"last update":"Sun Sep 10 00:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.38100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73299999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.56399999999999995,"15min":0.53700000000000003},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":0.89100000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.84599999999999997}]},{"osd":3,"last update":"Sun Sep 10 00:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.36599999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.53700000000000003,"15min":0.52800000000000002},"min":{"1min":0.36299999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56499999999999995}]},{"osd":4,"last update":"Sun Sep 10 00:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.42299999999999999},"min":{"1min":0.35099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.46000000000000002,"15min":0.437},"min":{"1min":0.33800000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.65700000000000003}]},{"osd":5,"last update":"Sun Sep 10 00:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.755,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.65800000000000003,"15min":0.626},"min":{"1min":0.62,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92700000000000005,"5min":1.044,"15min":1.044},"last":0.88}]},{"osd":6,"last update":"Sun Sep 10 00:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.623,"15min":0.60799999999999998},"min":{"1min":0.44400000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.629,"15min":0.61199999999999999},"min":{"1min":0.46400000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.70999999999999996}]},{"osd":7,"last update":"Sun Sep 10 00:38:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.59099999999999997,"15min":0.56599999999999995},"min":{"1min":0.49099999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.747},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.40899999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.76700000000000002}]},{"osd":8,"last update":"Sun Sep 10 00:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71699999999999997,"15min":0.71299999999999997},"min":{"1min":0.57799999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94999999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.54000000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.86599999999999999,"5min":1.014,"15min":1.014},"last":0.82799999999999996}]},{"osd":9,"last update":"Sun Sep 10 00:38:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.73499999999999999}]},{"osd":10,"last update":"Sun Sep 10 00:38:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.90400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738431,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":468411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 10 00:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52100000000000002,"15min":0.49199999999999999},"min":{"1min":0.371,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.625},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.432,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56599999999999995}]},{"osd":2,"last update":"Sun Sep 10 00:38:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.39500000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80700000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58599999999999997,"15min":0.56799999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0329999999999999,"5min":1.621,"15min":1.621},"last":0.69899999999999995}]},{"osd":3,"last update":"Sun Sep 10 00:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.41799999999999998,"15min":0.41699999999999998},"min":{"1min":0.249,"5min":0.245,"15min":0.245},"max":{"1min":0.66700000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46000000000000002,"15min":0.45600000000000002},"min":{"1min":0.29099999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.79100000000000004,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.53900000000000003}]},{"osd":4,"last update":"Sun Sep 10 00:38:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68899999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.38900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.91300000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65800000000000003}]},{"osd":5,"last update":"Sun Sep 10 00:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64000000000000001,"15min":0.60999999999999999},"min":{"1min":0.49199999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.96999999999999997,"5min":1.075,"15min":1.075},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64700000000000002,"15min":0.63700000000000001},"min":{"1min":0.48699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.753}]},{"osd":6,"last update":"Sun Sep 10 00:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.52500000000000002,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.71599999999999997}]},{"osd":7,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65700000000000003,"15min":0.66400000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73899999999999999,"15min":0.74399999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.98499999999999999,"5min":1.111,"15min":1.111},"last":0.83499999999999996}]},{"osd":8,"last update":"Sun Sep 10 00:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77300000000000002,"15min":0.77900000000000003},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0640000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.499,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.97199999999999998}]},{"osd":9,"last update":"Sun Sep 10 00:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.878}]},{"osd":10,"last update":"Sun Sep 10 00:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.86299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411382,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":511771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.70499999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.39800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72099999999999997,"5min":0.753,"15min":0.753},"last":0.56000000000000005}]},{"osd":1,"last update":"Sun Sep 10 00:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.42499999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.746,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82599999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.82899999999999996}]},{"osd":3,"last update":"Sun Sep 10 00:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.75},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.54800000000000004,"15min":0.53300000000000003},"min":{"1min":0.47799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.68000000000000005}]},{"osd":4,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.39000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.82299999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.879},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79700000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.87}]},{"osd":5,"last update":"Sun Sep 10 00:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.45500000000000002,"15min":0.435},"min":{"1min":0.29899999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52400000000000002,"15min":0.5},"min":{"1min":0.35699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.70599999999999996}]},{"osd":6,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59999999999999998,"15min":0.59499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.80600000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.64200000000000002,"15min":0.63600000000000001},"min":{"1min":0.50800000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.92200000000000004}]},{"osd":7,"last update":"Sun Sep 10 00:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.99099999999999999}]},{"osd":8,"last update":"Sun Sep 10 00:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.35099999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78400000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.36899999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.80800000000000005}]},{"osd":9,"last update":"Sun Sep 10 00:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.85699999999999998}]},{"osd":10,"last update":"Sun Sep 10 00:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":1.032}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":466641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.26000000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.64300000000000002,"5min":1.409,"15min":1.409},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.42299999999999999,"15min":0.41899999999999998},"min":{"1min":0.30399999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.93000000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.34599999999999997}]},{"osd":1,"last update":"Sun Sep 10 00:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.40600000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85899999999999999,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.57299999999999995,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.90500000000000003,"5min":1.581,"15min":1.581},"last":0.39500000000000002}]},{"osd":2,"last update":"Sun Sep 10 00:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.61099999999999999,"15min":0.60299999999999998},"min":{"1min":0.5,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64500000000000002,"15min":0.65900000000000003},"min":{"1min":0.438,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.036,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.55500000000000005}]},{"osd":4,"last update":"Sun Sep 10 00:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88100000000000001,"5min":1.536,"15min":1.536},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67900000000000005,"15min":0.69299999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95099999999999996,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.625}]},{"osd":5,"last update":"Sun Sep 10 00:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88100000000000001,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67800000000000005,"15min":0.67500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88700000000000001,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Sun Sep 10 00:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.35299999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.438},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.59599999999999997,"15min":0.57799999999999996},"min":{"1min":0.44400000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59499999999999997}]},{"osd":7,"last update":"Sun Sep 10 00:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.755,"15min":0.749},"min":{"1min":0.51600000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.74099999999999999,"15min":0.73699999999999999},"min":{"1min":0.53500000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.50700000000000001}]},{"osd":8,"last update":"Sun Sep 10 00:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.90000000000000002}]},{"osd":9,"last update":"Sun Sep 10 00:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.69599999999999995}]},{"osd":10,"last update":"Sun Sep 10 00:38:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.61299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920814,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":471419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45000000000000001}]},{"osd":1,"last update":"Sun Sep 10 00:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.35299999999999998,"15min":0.33900000000000002},"min":{"1min":0.245,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.371,"15min":0.35899999999999999},"min":{"1min":0.27200000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.505,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.46000000000000002}]},{"osd":2,"last update":"Sun Sep 10 00:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.45600000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72199999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59199999999999997}]},{"osd":3,"last update":"Sun Sep 10 00:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53800000000000003,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.57299999999999995,"15min":0.55000000000000004},"min":{"1min":0.44900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.80300000000000005}]},{"osd":5,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.439,"5min":0.376,"15min":0.376},"max":{"1min":0.80600000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60799999999999998}]},{"osd":6,"last update":"Sun Sep 10 00:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":0.77600000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64200000000000002,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72599999999999998}]},{"osd":7,"last update":"Sun Sep 10 00:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.53500000000000003,"15min":0.51800000000000002},"min":{"1min":0.41099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.67400000000000004}]},{"osd":8,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63900000000000001,"15min":0.629},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91100000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.879,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.69699999999999995}]},{"osd":9,"last update":"Sun Sep 10 00:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.77800000000000002}]},{"osd":10,"last update":"Sun Sep 10 00:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.748},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.81599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757289,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1058077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.40699999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.755,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54700000000000004}]},{"osd":1,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53400000000000003,"15min":0.51600000000000001},"min":{"1min":0.33700000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86099999999999999,"5min":0.873,"15min":0.873},"last":0.747}]},{"osd":2,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40699999999999997,"15min":0.39600000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.71699999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.438,"15min":0.42999999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.47199999999999998}]},{"osd":3,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54900000000000004,"15min":0.54200000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.81499999999999995,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.92400000000000004,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.57399999999999995}]},{"osd":4,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76000000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.70599999999999996}]},{"osd":6,"last update":"Sun Sep 10 00:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.45600000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82799999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.621}]},{"osd":7,"last update":"Sun Sep 10 00:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61499999999999999,"15min":0.61899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.45400000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.69299999999999995}]},{"osd":8,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.42499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53400000000000003,"15min":0.52000000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.65200000000000002}]},{"osd":9,"last update":"Sun Sep 10 00:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71799999999999997}]},{"osd":10,"last update":"Sun Sep 10 00:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.77600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":40315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":45538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-10T00:39:04.283 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-10T00:39:04.284 INFO:tasks.cephadm:Setting up client nodes... 2023-09-10T00:39:04.284 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-09-10T00:39:04.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:03 smithi131 ceph-mon[103713]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:39:04.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:03 smithi131 ceph-mon[103713]: pgmap v280: 129 pgs: 129 active+clean; 583 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 1.1 KiB/s wr, 5 op/s 2023-09-10T00:39:04.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:03 smithi131 ceph-mon[103713]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:39:04.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:03 smithi131 ceph-mon[103713]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:39:04.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:03 smithi131 ceph-mon[103713]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-10T00:39:04.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:03 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/797462926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:39:04.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:03 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3200927266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:04.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:03 smithi131 ceph-mon[103713]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:39:04.876 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:04 smithi099 ceph-mon[102692]: pgmap v281: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2023-09-10T00:39:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:04 smithi099 ceph-mon[102692]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:39:05.126 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:04 smithi044 ceph-mon[98676]: pgmap v281: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2023-09-10T00:39:05.127 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:04 smithi044 ceph-mon[98676]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:39:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:04 smithi131 ceph-mon[103713]: pgmap v281: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2023-09-10T00:39:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:04 smithi131 ceph-mon[103713]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:39:06.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.077+0000 7f91f180e700 1 -- 172.21.15.44:0/3799097468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91ec103b80 msgr2=0x7f91ec105f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:06.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.077+0000 7f91f180e700 1 --2- 172.21.15.44:0/3799097468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91ec103b80 0x7f91ec105f70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f91dc009990 tx=0x7f91dc009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:06.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.077+0000 7f91f180e700 1 -- 172.21.15.44:0/3799097468 shutdown_connections 2023-09-10T00:39:06.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.077+0000 7f91f180e700 1 --2- 172.21.15.44:0/3799097468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ec1064b0 0x7f91ec1088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:06.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.077+0000 7f91f180e700 1 --2- 172.21.15.44:0/3799097468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91ec101250 0x7f91ec103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:06.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.077+0000 7f91f180e700 1 --2- 172.21.15.44:0/3799097468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91ec103b80 0x7f91ec105f70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:06.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.077+0000 7f91f180e700 1 -- 172.21.15.44:0/3799097468 >> 172.21.15.44:0/3799097468 conn(0x7f91ec0fac00 msgr2=0x7f91ec0fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:06.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.077+0000 7f91f180e700 1 -- 172.21.15.44:0/3799097468 shutdown_connections 2023-09-10T00:39:06.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.077+0000 7f91f180e700 1 -- 172.21.15.44:0/3799097468 wait complete. 2023-09-10T00:39:06.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.077+0000 7f91f180e700 1 Processor -- start 2023-09-10T00:39:06.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.077+0000 7f91f180e700 1 -- start start 2023-09-10T00:39:06.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91f180e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91ec101250 0x7f91ec1962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:06.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91f180e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91ec103b80 0x7f91ec196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:06.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91f180e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ec1064b0 0x7f91ec19d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:06.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91f180e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec19de50 con 0x7f91ec101250 2023-09-10T00:39:06.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91f180e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec19df90 con 0x7f91ec103b80 2023-09-10T00:39:06.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91f180e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec19e120 con 0x7f91ec1064b0 2023-09-10T00:39:06.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91eaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91ec101250 0x7f91ec1962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:06.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91eaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91ec101250 0x7f91ec1962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46896/0 (socket says 172.21.15.44:46896) 2023-09-10T00:39:06.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91eaffd700 1 -- 172.21.15.44:0/2776095298 learned_addr learned my addr 172.21.15.44:0/2776095298 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:06.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91ea7fc700 1 --2- 172.21.15.44:0/2776095298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91ec103b80 0x7f91ec196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:06.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91eaffd700 1 -- 172.21.15.44:0/2776095298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ec1064b0 msgr2=0x7f91ec19d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:06.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91eaffd700 1 --2- 172.21.15.44:0/2776095298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ec1064b0 0x7f91ec19d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:06.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91eaffd700 1 -- 172.21.15.44:0/2776095298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91ec103b80 msgr2=0x7f91ec196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:06.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91eaffd700 1 --2- 172.21.15.44:0/2776095298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91ec103b80 0x7f91ec196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:06.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91eaffd700 1 -- 172.21.15.44:0/2776095298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc0095f0 con 0x7f91ec101250 2023-09-10T00:39:06.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91eaffd700 1 --2- 172.21.15.44:0/2776095298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91ec101250 0x7f91ec1962c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f91e000d980 tx=0x7f91e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:06.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91f080c700 1 -- 172.21.15.44:0/2776095298 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0009b40 con 0x7f91ec101250 2023-09-10T00:39:06.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91f080c700 1 -- 172.21.15.44:0/2776095298 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f91e000b6e0 con 0x7f91ec101250 2023-09-10T00:39:06.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.078+0000 7f91f180e700 1 -- 172.21.15.44:0/2776095298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ec19e400 con 0x7f91ec101250 2023-09-10T00:39:06.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.079+0000 7f91f080c700 1 -- 172.21.15.44:0/2776095298 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e000f3c0 con 0x7f91ec101250 2023-09-10T00:39:06.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.079+0000 7f91f180e700 1 -- 172.21.15.44:0/2776095298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ec19e950 con 0x7f91ec101250 2023-09-10T00:39:06.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.079+0000 7f91f180e700 1 -- 172.21.15.44:0/2776095298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91ec0fc780 con 0x7f91ec101250 2023-09-10T00:39:06.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.081+0000 7f91f080c700 1 -- 172.21.15.44:0/2776095298 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f91e0010460 con 0x7f91ec101250 2023-09-10T00:39:06.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.081+0000 7f91f080c700 1 --2- 172.21.15.44:0/2776095298 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f91d40a0750 0x7f91d40a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:06.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.081+0000 7f91ea7fc700 1 --2- 172.21.15.44:0/2776095298 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f91d40a0750 0x7f91d40a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:06.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.081+0000 7f91f080c700 1 -- 172.21.15.44:0/2776095298 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f91e00cbdd0 con 0x7f91ec101250 2023-09-10T00:39:06.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.082+0000 7f91ea7fc700 1 --2- 172.21.15.44:0/2776095298 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f91d40a0750 0x7f91d40a2c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00b5c0 tx=0x7f91dc009f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:06.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.085+0000 7f91f080c700 1 -- 172.21.15.44:0/2776095298 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f91e00955b0 con 0x7f91ec101250 2023-09-10T00:39:06.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.370+0000 7f91f180e700 1 -- 172.21.15.44:0/2776095298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f91ec10be00 con 0x7f91ec101250 2023-09-10T00:39:06.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.372+0000 7f91f080c700 1 -- 172.21.15.44:0/2776095298 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f91e00200f0 con 0x7f91ec101250 2023-09-10T00:39:06.373 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2023-09-10T00:39:06.373 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQAqEP1kPGAjFhAA1mirJgUs1kcNJfS7jkg2cQ== 2023-09-10T00:39:06.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.374+0000 7f91f180e700 1 -- 172.21.15.44:0/2776095298 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f91d40a0750 msgr2=0x7f91d40a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:06.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.374+0000 7f91f180e700 1 --2- 172.21.15.44:0/2776095298 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f91d40a0750 0x7f91d40a2c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00b5c0 tx=0x7f91dc009f00 comp rx=0 tx=0).stop 2023-09-10T00:39:06.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.374+0000 7f91f180e700 1 -- 172.21.15.44:0/2776095298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91ec101250 msgr2=0x7f91ec1962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:06.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.374+0000 7f91f180e700 1 --2- 172.21.15.44:0/2776095298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91ec101250 0x7f91ec1962c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f91e000d980 tx=0x7f91e000dc90 comp rx=0 tx=0).stop 2023-09-10T00:39:06.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.374+0000 7f91f180e700 1 -- 172.21.15.44:0/2776095298 shutdown_connections 2023-09-10T00:39:06.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.374+0000 7f91f180e700 1 --2- 172.21.15.44:0/2776095298 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f91d40a0750 0x7f91d40a2c10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:06.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.374+0000 7f91f180e700 1 --2- 172.21.15.44:0/2776095298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ec1064b0 0x7f91ec19d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:06.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.374+0000 7f91f180e700 1 --2- 172.21.15.44:0/2776095298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91ec101250 0x7f91ec1962c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:06.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.374+0000 7f91f180e700 1 --2- 172.21.15.44:0/2776095298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91ec103b80 0x7f91ec196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:06.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.374+0000 7f91f180e700 1 -- 172.21.15.44:0/2776095298 >> 172.21.15.44:0/2776095298 conn(0x7f91ec0fac00 msgr2=0x7f91ec101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:06.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.374+0000 7f91f180e700 1 -- 172.21.15.44:0/2776095298 shutdown_connections 2023-09-10T00:39:06.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:06.374+0000 7f91f180e700 1 -- 172.21.15.44:0/2776095298 wait complete. 2023-09-10T00:39:06.716 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:06 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2776095298' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-10T00:39:06.716 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:06 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2776095298' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-10T00:39:06.729 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T00:39:06.730 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-09-10T00:39:06.730 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-10T00:39:06.772 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph config log 1 --format=json 2023-09-10T00:39:06.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:06 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2776095298' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-10T00:39:06.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:06 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2776095298' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-10T00:39:06.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:06 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2776095298' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-10T00:39:06.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:06 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2776095298' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-10T00:39:07.371 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:07.744 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:07 smithi044 ceph-mon[98676]: pgmap v282: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.3 KiB/s wr, 20 op/s 2023-09-10T00:39:07.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:07 smithi131 ceph-mon[103713]: pgmap v282: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.3 KiB/s wr, 20 op/s 2023-09-10T00:39:07.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:07 smithi099 ceph-mon[102692]: pgmap v282: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.3 KiB/s wr, 20 op/s 2023-09-10T00:39:08.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.543+0000 7fcc796ee700 1 -- 172.21.15.44:0/633815201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc741048e0 msgr2=0x7fcc7410c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:08.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.543+0000 7fcc796ee700 1 --2- 172.21.15.44:0/633815201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc741048e0 0x7fcc7410c460 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00b0a0 tx=0x7fcc6c00b3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:08.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.543+0000 7fcc796ee700 1 -- 172.21.15.44:0/633815201 shutdown_connections 2023-09-10T00:39:08.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.543+0000 7fcc796ee700 1 --2- 172.21.15.44:0/633815201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc741048e0 0x7fcc7410c460 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:08.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.543+0000 7fcc796ee700 1 --2- 172.21.15.44:0/633815201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc740ffec0 0x7fcc741002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:08.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.543+0000 7fcc796ee700 1 --2- 172.21.15.44:0/633815201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc74100870 0x7fcc741041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:08.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.543+0000 7fcc796ee700 1 -- 172.21.15.44:0/633815201 >> 172.21.15.44:0/633815201 conn(0x7fcc74075a70 msgr2=0x7fcc740fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:08.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.543+0000 7fcc796ee700 1 -- 172.21.15.44:0/633815201 shutdown_connections 2023-09-10T00:39:08.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.543+0000 7fcc796ee700 1 -- 172.21.15.44:0/633815201 wait complete. 2023-09-10T00:39:08.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.544+0000 7fcc796ee700 1 Processor -- start 2023-09-10T00:39:08.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.544+0000 7fcc796ee700 1 -- start start 2023-09-10T00:39:08.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.544+0000 7fcc796ee700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc740ffec0 0x7fcc74195560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:08.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.544+0000 7fcc796ee700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc74100870 0x7fcc74195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:08.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.544+0000 7fcc796ee700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc741048e0 0x7fcc74199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:08.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.544+0000 7fcc796ee700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7419a520 con 0x7fcc741048e0 2023-09-10T00:39:08.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.544+0000 7fcc796ee700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7419a690 con 0x7fcc740ffec0 2023-09-10T00:39:08.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.544+0000 7fcc796ee700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7419a850 con 0x7fcc74100870 2023-09-10T00:39:08.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.545+0000 7fcc737fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc741048e0 0x7fcc74199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:08.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.545+0000 7fcc72ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc740ffec0 0x7fcc74195560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:08.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.545+0000 7fcc737fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc741048e0 0x7fcc74199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46914/0 (socket says 172.21.15.44:46914) 2023-09-10T00:39:08.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.545+0000 7fcc737fe700 1 -- 172.21.15.44:0/146688262 learned_addr learned my addr 172.21.15.44:0/146688262 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:08.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.545+0000 7fcc727fc700 1 --2- 172.21.15.44:0/146688262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc74100870 0x7fcc74195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:08.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.545+0000 7fcc737fe700 1 -- 172.21.15.44:0/146688262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc74100870 msgr2=0x7fcc74195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:08.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.545+0000 7fcc737fe700 1 --2- 172.21.15.44:0/146688262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc74100870 0x7fcc74195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:08.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.545+0000 7fcc737fe700 1 -- 172.21.15.44:0/146688262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc740ffec0 msgr2=0x7fcc74195560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:08.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.545+0000 7fcc737fe700 1 --2- 172.21.15.44:0/146688262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc740ffec0 0x7fcc74195560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:08.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.545+0000 7fcc737fe700 1 -- 172.21.15.44:0/146688262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c009d00 con 0x7fcc741048e0 2023-09-10T00:39:08.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.545+0000 7fcc737fe700 1 --2- 172.21.15.44:0/146688262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc741048e0 0x7fcc74199f10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c007990 tx=0x7fcc6c0079c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:08.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.545+0000 7fcc5bfff700 1 -- 172.21.15.44:0/146688262 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c00e040 con 0x7fcc741048e0 2023-09-10T00:39:08.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.545+0000 7fcc5bfff700 1 -- 172.21.15.44:0/146688262 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc6c01a950 con 0x7fcc741048e0 2023-09-10T00:39:08.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.545+0000 7fcc796ee700 1 -- 172.21.15.44:0/146688262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7419aad0 con 0x7fcc741048e0 2023-09-10T00:39:08.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.545+0000 7fcc5bfff700 1 -- 172.21.15.44:0/146688262 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c024980 con 0x7fcc741048e0 2023-09-10T00:39:08.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.545+0000 7fcc796ee700 1 -- 172.21.15.44:0/146688262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc740fed20 con 0x7fcc741048e0 2023-09-10T00:39:08.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.546+0000 7fcc796ee700 1 -- 172.21.15.44:0/146688262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7404e300 con 0x7fcc741048e0 2023-09-10T00:39:08.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.547+0000 7fcc5bfff700 1 -- 172.21.15.44:0/146688262 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fcc6c01a4c0 con 0x7fcc741048e0 2023-09-10T00:39:08.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.548+0000 7fcc5bfff700 1 --2- 172.21.15.44:0/146688262 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcc5c0a06d0 0x7fcc5c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:08.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.548+0000 7fcc72ffd700 1 --2- 172.21.15.44:0/146688262 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcc5c0a06d0 0x7fcc5c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:08.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.548+0000 7fcc5bfff700 1 -- 172.21.15.44:0/146688262 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcc6c0cf660 con 0x7fcc741048e0 2023-09-10T00:39:08.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.549+0000 7fcc72ffd700 1 --2- 172.21.15.44:0/146688262 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcc5c0a06d0 0x7fcc5c0a2b90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcc68005950 tx=0x7fcc6800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:08.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.552+0000 7fcc5bfff700 1 -- 172.21.15.44:0/146688262 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcc6c098e70 con 0x7fcc741048e0 2023-09-10T00:39:08.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.753+0000 7fcc796ee700 1 -- 172.21.15.44:0/146688262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7fcc7410f340 con 0x7fcc741048e0 2023-09-10T00:39:08.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.753+0000 7fcc5bfff700 1 -- 172.21.15.44:0/146688262 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v43) v1 ==== 86+0+133 (secure 0 0 0) 0x7fcc6c09c490 con 0x7fcc741048e0 2023-09-10T00:39:08.754 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:08.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.755+0000 7fcc796ee700 1 -- 172.21.15.44:0/146688262 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcc5c0a06d0 msgr2=0x7fcc5c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:08.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.755+0000 7fcc796ee700 1 --2- 172.21.15.44:0/146688262 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcc5c0a06d0 0x7fcc5c0a2b90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcc68005950 tx=0x7fcc6800b410 comp rx=0 tx=0).stop 2023-09-10T00:39:08.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.755+0000 7fcc796ee700 1 -- 172.21.15.44:0/146688262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc741048e0 msgr2=0x7fcc74199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:08.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.755+0000 7fcc796ee700 1 --2- 172.21.15.44:0/146688262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc741048e0 0x7fcc74199f10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c007990 tx=0x7fcc6c0079c0 comp rx=0 tx=0).stop 2023-09-10T00:39:08.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.755+0000 7fcc796ee700 1 -- 172.21.15.44:0/146688262 shutdown_connections 2023-09-10T00:39:08.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.755+0000 7fcc796ee700 1 --2- 172.21.15.44:0/146688262 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fcc5c0a06d0 0x7fcc5c0a2b90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:08.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.755+0000 7fcc796ee700 1 --2- 172.21.15.44:0/146688262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc741048e0 0x7fcc74199f10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:08.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.755+0000 7fcc796ee700 1 --2- 172.21.15.44:0/146688262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc740ffec0 0x7fcc74195560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:08.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.755+0000 7fcc796ee700 1 --2- 172.21.15.44:0/146688262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc74100870 0x7fcc74195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:08.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.755+0000 7fcc796ee700 1 -- 172.21.15.44:0/146688262 >> 172.21.15.44:0/146688262 conn(0x7fcc74075a70 msgr2=0x7fcc74103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:08.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.756+0000 7fcc796ee700 1 -- 172.21.15.44:0/146688262 shutdown_connections 2023-09-10T00:39:08.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:08.756+0000 7fcc796ee700 1 -- 172.21.15.44:0/146688262 wait complete. 2023-09-10T00:39:09.123 INFO:teuthology.orchestra.run.smithi044.stdout:[{"version":43,"timestamp":"2023-09-10T00:38:33.526044+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2023-09-10T00:39:09.124 INFO:tasks.ceph_manager:config epoch is 43 2023-09-10T00:39:09.124 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-10T00:39:09.125 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-09-10T00:39:09.126 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph mgr dump --format=json 2023-09-10T00:39:09.693 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:09.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:09 smithi131 ceph-mon[103713]: pgmap v283: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.1 KiB/s wr, 21 op/s 2023-09-10T00:39:09.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:09 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/146688262' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-09-10T00:39:09.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:09 smithi044 ceph-mon[98676]: pgmap v283: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.1 KiB/s wr, 21 op/s 2023-09-10T00:39:09.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:09 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/146688262' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-09-10T00:39:09.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:09 smithi099 ceph-mon[102692]: pgmap v283: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.1 KiB/s wr, 21 op/s 2023-09-10T00:39:09.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:09 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/146688262' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-09-10T00:39:11.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.011+0000 7f72d0a3a700 1 -- 172.21.15.44:0/1537368922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72cc106760 msgr2=0x7f72cc106b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:11.013 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.011+0000 7f72d0a3a700 1 --2- 172.21.15.44:0/1537368922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72cc106760 0x7f72cc106b40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f72c00098f0 tx=0x7f72c0009c00 comp rx=0 tx=0).stop 2023-09-10T00:39:11.013 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.011+0000 7f72d0a3a700 1 -- 172.21.15.44:0/1537368922 shutdown_connections 2023-09-10T00:39:11.013 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.011+0000 7f72d0a3a700 1 --2- 172.21.15.44:0/1537368922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72cc101250 0x7f72cc10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:11.013 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.011+0000 7f72d0a3a700 1 --2- 172.21.15.44:0/1537368922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72cc106760 0x7f72cc106b40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:11.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.011+0000 7f72d0a3a700 1 --2- 172.21.15.44:0/1537368922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72cc100890 0x7f72cc100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:11.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.011+0000 7f72d0a3a700 1 -- 172.21.15.44:0/1537368922 >> 172.21.15.44:0/1537368922 conn(0x7f72cc078620 msgr2=0x7f72cc078a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:11.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.011+0000 7f72d0a3a700 1 -- 172.21.15.44:0/1537368922 shutdown_connections 2023-09-10T00:39:11.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.011+0000 7f72d0a3a700 1 -- 172.21.15.44:0/1537368922 wait complete. 2023-09-10T00:39:11.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.012+0000 7f72d0a3a700 1 Processor -- start 2023-09-10T00:39:11.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.012+0000 7f72d0a3a700 1 -- start start 2023-09-10T00:39:11.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72d0a3a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72cc100890 0x7f72cc19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:11.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72d0a3a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72cc101250 0x7f72cc19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:11.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72d0a3a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72cc106760 0x7f72cc196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:11.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72d0a3a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc196c00 con 0x7f72cc100890 2023-09-10T00:39:11.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72d0a3a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc196d70 con 0x7f72cc106760 2023-09-10T00:39:11.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72d0a3a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc196f30 con 0x7f72cc101250 2023-09-10T00:39:11.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72ca59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72cc100890 0x7f72cc19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:11.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72ca59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72cc100890 0x7f72cc19c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46922/0 (socket says 172.21.15.44:46922) 2023-09-10T00:39:11.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72ca59c700 1 -- 172.21.15.44:0/657905400 learned_addr learned my addr 172.21.15.44:0/657905400 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:11.019 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72c9d9b700 1 --2- 172.21.15.44:0/657905400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72cc101250 0x7f72cc19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:11.019 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72cad9d700 1 --2- 172.21.15.44:0/657905400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72cc106760 0x7f72cc196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:11.019 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72ca59c700 1 -- 172.21.15.44:0/657905400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72cc101250 msgr2=0x7f72cc19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:11.019 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72ca59c700 1 --2- 172.21.15.44:0/657905400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72cc101250 0x7f72cc19ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:11.019 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72ca59c700 1 -- 172.21.15.44:0/657905400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72cc106760 msgr2=0x7f72cc196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72ca59c700 1 --2- 172.21.15.44:0/657905400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72cc106760 0x7f72cc196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72ca59c700 1 -- 172.21.15.44:0/657905400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c0009520 con 0x7f72cc100890 2023-09-10T00:39:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72ca59c700 1 --2- 172.21.15.44:0/657905400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72cc100890 0x7f72cc19c510 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009810 tx=0x7f72c000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.013+0000 7f72bb7fe700 1 -- 172.21.15.44:0/657905400 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c001c070 con 0x7f72cc100890 2023-09-10T00:39:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.014+0000 7f72d0a3a700 1 -- 172.21.15.44:0/657905400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc1971b0 con 0x7f72cc100890 2023-09-10T00:39:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.014+0000 7f72bb7fe700 1 -- 172.21.15.44:0/657905400 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72c0005040 con 0x7f72cc100890 2023-09-10T00:39:11.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.014+0000 7f72bb7fe700 1 -- 172.21.15.44:0/657905400 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0004930 con 0x7f72cc100890 2023-09-10T00:39:11.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.014+0000 7f72d0a3a700 1 -- 172.21.15.44:0/657905400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc1a3500 con 0x7f72cc100890 2023-09-10T00:39:11.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.014+0000 7f72d0a3a700 1 -- 172.21.15.44:0/657905400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72cc04e300 con 0x7f72cc100890 2023-09-10T00:39:11.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.016+0000 7f72bb7fe700 1 -- 172.21.15.44:0/657905400 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f72c0020070 con 0x7f72cc100890 2023-09-10T00:39:11.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.016+0000 7f72bb7fe700 1 --2- 172.21.15.44:0/657905400 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f72b40a06d0 0x7f72b40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:11.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.016+0000 7f72bb7fe700 1 -- 172.21.15.44:0/657905400 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f72c00cd000 con 0x7f72cc100890 2023-09-10T00:39:11.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.016+0000 7f72c9d9b700 1 --2- 172.21.15.44:0/657905400 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f72b40a06d0 0x7f72b40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:11.023 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.017+0000 7f72c9d9b700 1 --2- 172.21.15.44:0/657905400 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f72b40a06d0 0x7f72b40a2b90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f72cc078f50 tx=0x7f72bc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:11.023 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.020+0000 7f72bb7fe700 1 -- 172.21.15.44:0/657905400 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f72c0096870 con 0x7f72cc100890 2023-09-10T00:39:11.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.280+0000 7f72d0a3a700 1 -- 172.21.15.44:0/657905400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f72cc110470 con 0x7f72cc100890 2023-09-10T00:39:11.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.285+0000 7f72bb7fe700 1 -- 172.21.15.44:0/657905400 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+259018 (secure 0 0 0) 0x7f72c0099e90 con 0x7f72cc100890 2023-09-10T00:39:11.288 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:11.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.287+0000 7f72d0a3a700 1 -- 172.21.15.44:0/657905400 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f72b40a06d0 msgr2=0x7f72b40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:11.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.287+0000 7f72d0a3a700 1 --2- 172.21.15.44:0/657905400 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f72b40a06d0 0x7f72b40a2b90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f72cc078f50 tx=0x7f72bc009310 comp rx=0 tx=0).stop 2023-09-10T00:39:11.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.287+0000 7f72d0a3a700 1 -- 172.21.15.44:0/657905400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72cc100890 msgr2=0x7f72cc19c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:11.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.287+0000 7f72d0a3a700 1 --2- 172.21.15.44:0/657905400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72cc100890 0x7f72cc19c510 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009810 tx=0x7f72c000fa00 comp rx=0 tx=0).stop 2023-09-10T00:39:11.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.287+0000 7f72d0a3a700 1 -- 172.21.15.44:0/657905400 shutdown_connections 2023-09-10T00:39:11.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.287+0000 7f72d0a3a700 1 --2- 172.21.15.44:0/657905400 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f72b40a06d0 0x7f72b40a2b90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:11.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.287+0000 7f72d0a3a700 1 --2- 172.21.15.44:0/657905400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72cc106760 0x7f72cc196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:11.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.287+0000 7f72d0a3a700 1 --2- 172.21.15.44:0/657905400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72cc100890 0x7f72cc19c510 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:11.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.287+0000 7f72d0a3a700 1 --2- 172.21.15.44:0/657905400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72cc101250 0x7f72cc19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:11.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.287+0000 7f72d0a3a700 1 -- 172.21.15.44:0/657905400 >> 172.21.15.44:0/657905400 conn(0x7f72cc078620 msgr2=0x7f72cc0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:11.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.288+0000 7f72d0a3a700 1 -- 172.21.15.44:0/657905400 shutdown_connections 2023-09-10T00:39:11.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:11.288+0000 7f72d0a3a700 1 -- 172.21.15.44:0/657905400 wait complete. 2023-09-10T00:39:11.624 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:11 smithi044 ceph-mon[98676]: pgmap v284: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2023-09-10T00:39:11.624 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:11 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/657905400' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-10T00:39:11.655 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1488156576},{"type":"v1","addr":"172.21.15.44:6801","nonce":1488156576}]},"active_addr":"172.21.15.44:6801/1488156576","active_change":"2023-09-10T00:32:15.913880+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24184,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24203,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.44:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":3766354728}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":3642136197}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":902389643}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":1968699888}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":3738465235}]}]} 2023-09-10T00:39:11.662 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-09-10T00:39:11.663 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-09-10T00:39:11.663 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd dump --format=json 2023-09-10T00:39:11.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:11 smithi131 ceph-mon[103713]: pgmap v284: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2023-09-10T00:39:11.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:11 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/657905400' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-10T00:39:11.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:11 smithi099 ceph-mon[102692]: pgmap v284: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2023-09-10T00:39:11.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:11 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/657905400' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-10T00:39:12.229 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:13.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.395+0000 7f830c8b8700 1 -- 172.21.15.44:0/1647629483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83080ff520 msgr2=0x7f83080ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:13.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.395+0000 7f830c8b8700 1 --2- 172.21.15.44:0/1647629483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83080ff520 0x7f83080ff9a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009990 tx=0x7f82f8009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:13.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.395+0000 7f830c8b8700 1 -- 172.21.15.44:0/1647629483 shutdown_connections 2023-09-10T00:39:13.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.395+0000 7f830c8b8700 1 --2- 172.21.15.44:0/1647629483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83080ffee0 0x7f830810c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:13.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.395+0000 7f830c8b8700 1 --2- 172.21.15.44:0/1647629483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8308104530 0x7f8308104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:13.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.395+0000 7f830c8b8700 1 --2- 172.21.15.44:0/1647629483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83080ff520 0x7f83080ff9a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:13.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.395+0000 7f830c8b8700 1 -- 172.21.15.44:0/1647629483 >> 172.21.15.44:0/1647629483 conn(0x7f8308076600 msgr2=0x7f8308076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:13.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.395+0000 7f830c8b8700 1 -- 172.21.15.44:0/1647629483 shutdown_connections 2023-09-10T00:39:13.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.395+0000 7f830c8b8700 1 -- 172.21.15.44:0/1647629483 wait complete. 2023-09-10T00:39:13.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.396+0000 7f830c8b8700 1 Processor -- start 2023-09-10T00:39:13.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.396+0000 7f830c8b8700 1 -- start start 2023-09-10T00:39:13.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.396+0000 7f830c8b8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83080ff520 0x7f83081951d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:13.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.396+0000 7f830c8b8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83080ffee0 0x7f8308195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:13.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.396+0000 7f830c8b8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8308104530 0x7f8308195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:13.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.396+0000 7f830c8b8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83081a4a10 con 0x7f8308104530 2023-09-10T00:39:13.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.396+0000 7f830c8b8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83081a4b80 con 0x7f83080ff520 2023-09-10T00:39:13.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.396+0000 7f830c8b8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83081a4d40 con 0x7f83080ffee0 2023-09-10T00:39:13.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.396+0000 7f8306d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8308104530 0x7f8308195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:13.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.396+0000 7f8306d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8308104530 0x7f8308195c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46940/0 (socket says 172.21.15.44:46940) 2023-09-10T00:39:13.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.396+0000 7f8306d9d700 1 -- 172.21.15.44:0/3970783979 learned_addr learned my addr 172.21.15.44:0/3970783979 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:13.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.397+0000 7f830659c700 1 --2- 172.21.15.44:0/3970783979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83080ff520 0x7f83081951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:13.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.397+0000 7f8305d9b700 1 --2- 172.21.15.44:0/3970783979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83080ffee0 0x7f8308195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:13.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.397+0000 7f8306d9d700 1 -- 172.21.15.44:0/3970783979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83080ffee0 msgr2=0x7f8308195710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:13.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.397+0000 7f8306d9d700 1 --2- 172.21.15.44:0/3970783979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83080ffee0 0x7f8308195710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:13.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.397+0000 7f8306d9d700 1 -- 172.21.15.44:0/3970783979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83080ff520 msgr2=0x7f83081951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:13.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.397+0000 7f8306d9d700 1 --2- 172.21.15.44:0/3970783979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83080ff520 0x7f83081951d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:13.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.397+0000 7f8306d9d700 1 -- 172.21.15.44:0/3970783979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f80095f0 con 0x7f8308104530 2023-09-10T00:39:13.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.397+0000 7f830659c700 1 --2- 172.21.15.44:0/3970783979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83080ff520 0x7f83081951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:13.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.397+0000 7f8306d9d700 1 --2- 172.21.15.44:0/3970783979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8308104530 0x7f8308195c50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f830000cca0 tx=0x7f830000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:13.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.397+0000 7f82f77fe700 1 -- 172.21.15.44:0/3970783979 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830000cea0 con 0x7f8308104530 2023-09-10T00:39:13.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.397+0000 7f82f77fe700 1 -- 172.21.15.44:0/3970783979 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f830000ad60 con 0x7f8308104530 2023-09-10T00:39:13.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.397+0000 7f830c8b8700 1 -- 172.21.15.44:0/3970783979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83081a5020 con 0x7f8308104530 2023-09-10T00:39:13.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.397+0000 7f82f77fe700 1 -- 172.21.15.44:0/3970783979 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83000115e0 con 0x7f8308104530 2023-09-10T00:39:13.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.397+0000 7f830c8b8700 1 -- 172.21.15.44:0/3970783979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83081a54a0 con 0x7f8308104530 2023-09-10T00:39:13.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.398+0000 7f830c8b8700 1 -- 172.21.15.44:0/3970783979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830804e300 con 0x7f8308104530 2023-09-10T00:39:13.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.400+0000 7f82f77fe700 1 -- 172.21.15.44:0/3970783979 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f8300012420 con 0x7f8308104530 2023-09-10T00:39:13.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.400+0000 7f82f77fe700 1 --2- 172.21.15.44:0/3970783979 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82f00a06d0 0x7f82f00a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:13.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.400+0000 7f830659c700 1 --2- 172.21.15.44:0/3970783979 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82f00a06d0 0x7f82f00a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:13.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.400+0000 7f82f77fe700 1 -- 172.21.15.44:0/3970783979 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f83000cc610 con 0x7f8308104530 2023-09-10T00:39:13.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.401+0000 7f830659c700 1 --2- 172.21.15.44:0/3970783979 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82f00a06d0 0x7f82f00a2b90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8308069c10 tx=0x7f82fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:13.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.404+0000 7f82f77fe700 1 -- 172.21.15.44:0/3970783979 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8300095f00 con 0x7f8308104530 2023-09-10T00:39:13.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.605+0000 7f830c8b8700 1 -- 172.21.15.44:0/3970783979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83080686b0 con 0x7f8308104530 2023-09-10T00:39:13.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.606+0000 7f82f77fe700 1 -- 172.21.15.44:0/3970783979 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23870 (secure 0 0 0) 0x7f8300099520 con 0x7f8308104530 2023-09-10T00:39:13.607 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:13.607 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":87,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:38:44.197902+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:39:13.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.608+0000 7f830c8b8700 1 -- 172.21.15.44:0/3970783979 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82f00a06d0 msgr2=0x7f82f00a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:13.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.608+0000 7f830c8b8700 1 --2- 172.21.15.44:0/3970783979 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82f00a06d0 0x7f82f00a2b90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8308069c10 tx=0x7f82fc005f90 comp rx=0 tx=0).stop 2023-09-10T00:39:13.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.608+0000 7f830c8b8700 1 -- 172.21.15.44:0/3970783979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8308104530 msgr2=0x7f8308195c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:13.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.608+0000 7f830c8b8700 1 --2- 172.21.15.44:0/3970783979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8308104530 0x7f8308195c50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f830000cca0 tx=0x7f830000a340 comp rx=0 tx=0).stop 2023-09-10T00:39:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.608+0000 7f830c8b8700 1 -- 172.21.15.44:0/3970783979 shutdown_connections 2023-09-10T00:39:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.608+0000 7f830c8b8700 1 --2- 172.21.15.44:0/3970783979 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82f00a06d0 0x7f82f00a2b90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.608+0000 7f830c8b8700 1 --2- 172.21.15.44:0/3970783979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8308104530 0x7f8308195c50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.608+0000 7f830c8b8700 1 --2- 172.21.15.44:0/3970783979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83080ff520 0x7f83081951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.608+0000 7f830c8b8700 1 --2- 172.21.15.44:0/3970783979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83080ffee0 0x7f8308195710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:13.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.608+0000 7f830c8b8700 1 -- 172.21.15.44:0/3970783979 >> 172.21.15.44:0/3970783979 conn(0x7f8308076600 msgr2=0x7f83080fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:13.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.609+0000 7f830c8b8700 1 -- 172.21.15.44:0/3970783979 shutdown_connections 2023-09-10T00:39:13.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:13.609+0000 7f830c8b8700 1 -- 172.21.15.44:0/3970783979 wait complete. 2023-09-10T00:39:13.736 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:13 smithi044 ceph-mon[98676]: pgmap v285: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2023-09-10T00:39:13.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:13 smithi131 ceph-mon[103713]: pgmap v285: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2023-09-10T00:39:13.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:13 smithi099 ceph-mon[102692]: pgmap v285: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2023-09-10T00:39:13.983 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-09-10T00:39:13.984 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd dump --format=json 2023-09-10T00:39:14.545 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:14.752 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:14 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3970783979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:14.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:14 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3970783979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:14.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:14 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3970783979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:15.714 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.712+0000 7f8a0771f700 1 -- 172.21.15.44:0/3628632763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a000ff8a0 msgr2=0x7f8a001038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:15.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.712+0000 7f8a0771f700 1 --2- 172.21.15.44:0/3628632763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a000ff8a0 0x7f8a001038f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009990 tx=0x7f89fc009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:15.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.712+0000 7f8a0771f700 1 -- 172.21.15.44:0/3628632763 shutdown_connections 2023-09-10T00:39:15.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.712+0000 7f8a0771f700 1 --2- 172.21.15.44:0/3628632763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a00103fd0 0x7f8a0010bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:15.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.712+0000 7f8a0771f700 1 --2- 172.21.15.44:0/3628632763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a000feef0 0x7f8a000ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:15.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.712+0000 7f8a0771f700 1 --2- 172.21.15.44:0/3628632763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a000ff8a0 0x7f8a001038f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:15.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.712+0000 7f8a0771f700 1 -- 172.21.15.44:0/3628632763 >> 172.21.15.44:0/3628632763 conn(0x7f8a00075a70 msgr2=0x7f8a000fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:15.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.712+0000 7f8a0771f700 1 -- 172.21.15.44:0/3628632763 shutdown_connections 2023-09-10T00:39:15.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.712+0000 7f8a0771f700 1 -- 172.21.15.44:0/3628632763 wait complete. 2023-09-10T00:39:15.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.713+0000 7f8a0771f700 1 Processor -- start 2023-09-10T00:39:15.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.713+0000 7f8a0771f700 1 -- start start 2023-09-10T00:39:15.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.713+0000 7f8a0771f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a000feef0 0x7f8a00196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:15.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.713+0000 7f8a0771f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a000ff8a0 0x7f8a00196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:15.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.713+0000 7f8a0771f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a00103fd0 0x7f8a0019aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:15.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.713+0000 7f8a0771f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0019b600 con 0x7f8a00103fd0 2023-09-10T00:39:15.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.713+0000 7f8a0771f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0019b770 con 0x7f8a000feef0 2023-09-10T00:39:15.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.713+0000 7f8a0771f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0019b930 con 0x7f8a000ff8a0 2023-09-10T00:39:15.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.713+0000 7f8a05cbc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a00103fd0 0x7f8a0019aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:15.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.713+0000 7f8a05cbc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a00103fd0 0x7f8a0019aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46952/0 (socket says 172.21.15.44:46952) 2023-09-10T00:39:15.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.713+0000 7f8a05cbc700 1 -- 172.21.15.44:0/4277030134 learned_addr learned my addr 172.21.15.44:0/4277030134 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:15.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.714+0000 7f8a054bb700 1 --2- 172.21.15.44:0/4277030134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a000feef0 0x7f8a00196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:15.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.714+0000 7f8a04cba700 1 --2- 172.21.15.44:0/4277030134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a000ff8a0 0x7f8a00196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:15.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.714+0000 7f8a05cbc700 1 -- 172.21.15.44:0/4277030134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a000ff8a0 msgr2=0x7f8a00196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:15.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.714+0000 7f8a05cbc700 1 --2- 172.21.15.44:0/4277030134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a000ff8a0 0x7f8a00196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:15.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.714+0000 7f8a05cbc700 1 -- 172.21.15.44:0/4277030134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a000feef0 msgr2=0x7f8a00196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:15.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.714+0000 7f8a05cbc700 1 --2- 172.21.15.44:0/4277030134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a000feef0 0x7f8a00196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:15.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.714+0000 7f8a05cbc700 1 -- 172.21.15.44:0/4277030134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc0095f0 con 0x7f8a00103fd0 2023-09-10T00:39:15.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.714+0000 7f8a054bb700 1 --2- 172.21.15.44:0/4277030134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a000feef0 0x7f8a00196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:15.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.714+0000 7f8a05cbc700 1 --2- 172.21.15.44:0/4277030134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a00103fd0 0x7f8a0019aff0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f89f800ada0 tx=0x7f89f800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:15.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.714+0000 7f89f27fc700 1 -- 172.21.15.44:0/4277030134 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f800f070 con 0x7f8a00103fd0 2023-09-10T00:39:15.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.714+0000 7f89f27fc700 1 -- 172.21.15.44:0/4277030134 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89f8007b30 con 0x7f8a00103fd0 2023-09-10T00:39:15.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.714+0000 7f89f27fc700 1 -- 172.21.15.44:0/4277030134 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f80053e0 con 0x7f8a00103fd0 2023-09-10T00:39:15.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.714+0000 7f8a0771f700 1 -- 172.21.15.44:0/4277030134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0019bc10 con 0x7f8a00103fd0 2023-09-10T00:39:15.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.714+0000 7f8a0771f700 1 -- 172.21.15.44:0/4277030134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a001a3470 con 0x7f8a00103fd0 2023-09-10T00:39:15.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.715+0000 7f8a0771f700 1 -- 172.21.15.44:0/4277030134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a00107240 con 0x7f8a00103fd0 2023-09-10T00:39:15.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.717+0000 7f89f27fc700 1 -- 172.21.15.44:0/4277030134 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f89f800c980 con 0x7f8a00103fd0 2023-09-10T00:39:15.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.717+0000 7f89f27fc700 1 --2- 172.21.15.44:0/4277030134 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f89ec0a0750 0x7f89ec0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:15.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.717+0000 7f89f27fc700 1 -- 172.21.15.44:0/4277030134 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f89f80cb8d0 con 0x7f8a00103fd0 2023-09-10T00:39:15.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.717+0000 7f8a054bb700 1 --2- 172.21.15.44:0/4277030134 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f89ec0a0750 0x7f89ec0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:15.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.718+0000 7f8a054bb700 1 --2- 172.21.15.44:0/4277030134 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f89ec0a0750 0x7f89ec0a2c10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f89f4005950 tx=0x7f89f40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:15.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.721+0000 7f89f27fc700 1 -- 172.21.15.44:0/4277030134 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f89f8095140 con 0x7f8a00103fd0 2023-09-10T00:39:15.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:15 smithi131 ceph-mon[103713]: pgmap v286: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 21 op/s 2023-09-10T00:39:15.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:15 smithi044 ceph-mon[98676]: pgmap v286: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 21 op/s 2023-09-10T00:39:15.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:15 smithi099 ceph-mon[102692]: pgmap v286: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 21 op/s 2023-09-10T00:39:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.924+0000 7f8a0771f700 1 -- 172.21.15.44:0/4277030134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a0004e300 con 0x7f8a00103fd0 2023-09-10T00:39:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.925+0000 7f89f27fc700 1 -- 172.21.15.44:0/4277030134 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23870 (secure 0 0 0) 0x7f89f800cc30 con 0x7f8a00103fd0 2023-09-10T00:39:15.926 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:15.926 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":87,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:38:44.197902+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:39:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.928+0000 7f8a0771f700 1 -- 172.21.15.44:0/4277030134 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f89ec0a0750 msgr2=0x7f89ec0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.928+0000 7f8a0771f700 1 --2- 172.21.15.44:0/4277030134 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f89ec0a0750 0x7f89ec0a2c10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f89f4005950 tx=0x7f89f40058e0 comp rx=0 tx=0).stop 2023-09-10T00:39:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.928+0000 7f8a0771f700 1 -- 172.21.15.44:0/4277030134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a00103fd0 msgr2=0x7f8a0019aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.928+0000 7f8a0771f700 1 --2- 172.21.15.44:0/4277030134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a00103fd0 0x7f8a0019aff0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f89f800ada0 tx=0x7f89f800c330 comp rx=0 tx=0).stop 2023-09-10T00:39:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.928+0000 7f8a0771f700 1 -- 172.21.15.44:0/4277030134 shutdown_connections 2023-09-10T00:39:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.928+0000 7f8a0771f700 1 --2- 172.21.15.44:0/4277030134 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f89ec0a0750 0x7f89ec0a2c10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.928+0000 7f8a0771f700 1 --2- 172.21.15.44:0/4277030134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a00103fd0 0x7f8a0019aff0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.928+0000 7f8a0771f700 1 --2- 172.21.15.44:0/4277030134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a000feef0 0x7f8a00196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.928+0000 7f8a0771f700 1 --2- 172.21.15.44:0/4277030134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a000ff8a0 0x7f8a00196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:15.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.928+0000 7f8a0771f700 1 -- 172.21.15.44:0/4277030134 >> 172.21.15.44:0/4277030134 conn(0x7f8a00075a70 msgr2=0x7f8a001029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:15.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.928+0000 7f8a0771f700 1 -- 172.21.15.44:0/4277030134 shutdown_connections 2023-09-10T00:39:15.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:15.928+0000 7f8a0771f700 1 -- 172.21.15.44:0/4277030134 wait complete. 2023-09-10T00:39:16.400 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph tell osd.0 flush_pg_stats 2023-09-10T00:39:16.400 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph tell osd.1 flush_pg_stats 2023-09-10T00:39:16.401 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph tell osd.2 flush_pg_stats 2023-09-10T00:39:16.401 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph tell osd.3 flush_pg_stats 2023-09-10T00:39:16.402 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph tell osd.4 flush_pg_stats 2023-09-10T00:39:16.402 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph tell osd.5 flush_pg_stats 2023-09-10T00:39:16.403 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph tell osd.6 flush_pg_stats 2023-09-10T00:39:16.404 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph tell osd.7 flush_pg_stats 2023-09-10T00:39:16.404 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph tell osd.8 flush_pg_stats 2023-09-10T00:39:16.405 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph tell osd.9 flush_pg_stats 2023-09-10T00:39:16.405 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph tell osd.10 flush_pg_stats 2023-09-10T00:39:16.406 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph tell osd.11 flush_pg_stats 2023-09-10T00:39:16.567 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:16 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/4277030134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:16.567 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2023-09-10T00:39:16.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-09-10T00:39:16.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2023-09-10T00:39:16.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2023-09-10T00:39:16.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2023-09-10T00:39:16.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2023-09-10T00:39:16.569 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-09-10T00:39:16.569 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2023-09-10T00:39:16.569 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2023-09-10T00:39:16.569 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2023-09-10T00:39:16.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:16 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/4277030134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:16.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2023-09-10T00:39:16.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-09-10T00:39:16.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2023-09-10T00:39:16.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2023-09-10T00:39:16.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2023-09-10T00:39:16.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2023-09-10T00:39:16.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-09-10T00:39:16.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2023-09-10T00:39:16.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2023-09-10T00:39:16.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2023-09-10T00:39:16.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:16 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/4277030134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:16.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2023-09-10T00:39:16.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-09-10T00:39:16.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2023-09-10T00:39:16.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2023-09-10T00:39:16.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2023-09-10T00:39:16.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2023-09-10T00:39:16.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-09-10T00:39:16.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2023-09-10T00:39:16.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2023-09-10T00:39:16.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2023-09-10T00:39:17.526 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:17.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:17 smithi044 ceph-mon[98676]: pgmap v287: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2023-09-10T00:39:17.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2023-09-10T00:39:17.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-09-10T00:39:17.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2023-09-10T00:39:17.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2023-09-10T00:39:17.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2023-09-10T00:39:17.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2023-09-10T00:39:17.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-09-10T00:39:17.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2023-09-10T00:39:17.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2023-09-10T00:39:17.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2023-09-10T00:39:17.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:17 smithi044 ceph-mon[98676]: osdmap e88: 12 total, 12 up, 12 in 2023-09-10T00:39:17.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:17 smithi131 ceph-mon[103713]: pgmap v287: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2023-09-10T00:39:17.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2023-09-10T00:39:17.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-09-10T00:39:17.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2023-09-10T00:39:17.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2023-09-10T00:39:17.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2023-09-10T00:39:17.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2023-09-10T00:39:17.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-09-10T00:39:17.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2023-09-10T00:39:17.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2023-09-10T00:39:17.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2023-09-10T00:39:17.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:17 smithi131 ceph-mon[103713]: osdmap e88: 12 total, 12 up, 12 in 2023-09-10T00:39:17.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:17 smithi099 ceph-mon[102692]: pgmap v287: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2023-09-10T00:39:17.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2023-09-10T00:39:17.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-09-10T00:39:17.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2023-09-10T00:39:17.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2023-09-10T00:39:17.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2023-09-10T00:39:17.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2023-09-10T00:39:17.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-09-10T00:39:17.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2023-09-10T00:39:17.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2023-09-10T00:39:17.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2023-09-10T00:39:17.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:17 smithi099 ceph-mon[102692]: osdmap e88: 12 total, 12 up, 12 in 2023-09-10T00:39:17.922 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:18.044 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:18.270 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:18.303 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:18.410 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:18.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:18 smithi044 ceph-mon[98676]: osdmap e89: 12 total, 12 up, 12 in 2023-09-10T00:39:18.688 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:18.690 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:18.743 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:18.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:18 smithi131 ceph-mon[103713]: osdmap e89: 12 total, 12 up, 12 in 2023-09-10T00:39:18.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:18 smithi099 ceph-mon[102692]: osdmap e89: 12 total, 12 up, 12 in 2023-09-10T00:39:19.760 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:19.760 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:19.794 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:19 smithi131 ceph-mon[103713]: pgmap v290: 129 pgs: 1 peering, 128 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 8 op/s 2023-09-10T00:39:19.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:19 smithi044 ceph-mon[98676]: pgmap v290: 129 pgs: 1 peering, 128 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 8 op/s 2023-09-10T00:39:19.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:19 smithi099 ceph-mon[102692]: pgmap v290: 129 pgs: 1 peering, 128 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 8 op/s 2023-09-10T00:39:20.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.663+0000 7fad157c1700 1 -- 172.21.15.44:0/2718338420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad10068120 msgr2=0x7fad100685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:20.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.663+0000 7fad157c1700 1 --2- 172.21.15.44:0/2718338420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad10068120 0x7fad100685a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fad04009990 tx=0x7fad04009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:20.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.663+0000 7fad157c1700 1 -- 172.21.15.44:0/2718338420 shutdown_connections 2023-09-10T00:39:20.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.663+0000 7fad157c1700 1 --2- 172.21.15.44:0/2718338420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad10068ae0 0x7fad1010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:20.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.663+0000 7fad157c1700 1 --2- 172.21.15.44:0/2718338420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad10106770 0x7fad10106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:20.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.663+0000 7fad157c1700 1 --2- 172.21.15.44:0/2718338420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad10068120 0x7fad100685a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:20.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.663+0000 7fad157c1700 1 -- 172.21.15.44:0/2718338420 >> 172.21.15.44:0/2718338420 conn(0x7fad10076600 msgr2=0x7fad10076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:20.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.663+0000 7fad157c1700 1 -- 172.21.15.44:0/2718338420 shutdown_connections 2023-09-10T00:39:20.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.663+0000 7fad157c1700 1 -- 172.21.15.44:0/2718338420 wait complete. 2023-09-10T00:39:20.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.664+0000 7fad157c1700 1 Processor -- start 2023-09-10T00:39:20.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.664+0000 7fad157c1700 1 -- start start 2023-09-10T00:39:20.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad157c1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad10068120 0x7fad10196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:20.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad157c1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad10068ae0 0x7fad10196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:20.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad157c1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad10106770 0x7fad10199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:20.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad157c1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1019a590 con 0x7fad10068ae0 2023-09-10T00:39:20.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad157c1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1019a700 con 0x7fad10068120 2023-09-10T00:39:20.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad157c1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1019a8c0 con 0x7fad10106770 2023-09-10T00:39:20.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad0e7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad10068ae0 0x7fad10196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:20.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad0e7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad10068ae0 0x7fad10196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46962/0 (socket says 172.21.15.44:46962) 2023-09-10T00:39:20.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad0e7fc700 1 -- 172.21.15.44:0/741220572 learned_addr learned my addr 172.21.15.44:0/741220572 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:20.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad0f7fe700 1 --2- 172.21.15.44:0/741220572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad10106770 0x7fad10199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:20.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad0effd700 1 --2- 172.21.15.44:0/741220572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad10068120 0x7fad10196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:20.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad0e7fc700 1 -- 172.21.15.44:0/741220572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad10106770 msgr2=0x7fad10199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:20.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad0e7fc700 1 --2- 172.21.15.44:0/741220572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad10106770 0x7fad10199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:20.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad0e7fc700 1 -- 172.21.15.44:0/741220572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad10068120 msgr2=0x7fad10196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:20.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad0e7fc700 1 --2- 172.21.15.44:0/741220572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad10068120 0x7fad10196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:20.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad0e7fc700 1 -- 172.21.15.44:0/741220572 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad040095f0 con 0x7fad10068ae0 2023-09-10T00:39:20.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad0effd700 1 --2- 172.21.15.44:0/741220572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad10068120 0x7fad10196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:20.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.665+0000 7fad0e7fc700 1 --2- 172.21.15.44:0/741220572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad10068ae0 0x7fad10196bd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fad04005910 tx=0x7fad040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.666+0000 7facf7fff700 1 -- 172.21.15.44:0/741220572 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0401d070 con 0x7fad10068ae0 2023-09-10T00:39:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.666+0000 7fad157c1700 1 -- 172.21.15.44:0/741220572 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1019ab40 con 0x7fad10068ae0 2023-09-10T00:39:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.666+0000 7facf7fff700 1 -- 172.21.15.44:0/741220572 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad04021410 con 0x7fad10068ae0 2023-09-10T00:39:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.666+0000 7facf7fff700 1 -- 172.21.15.44:0/741220572 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad04017650 con 0x7fad10068ae0 2023-09-10T00:39:20.673 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.666+0000 7fad157c1700 1 -- 172.21.15.44:0/741220572 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1019ae10 con 0x7fad10068ae0 2023-09-10T00:39:20.673 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.667+0000 7fad157c1700 1 -- 172.21.15.44:0/741220572 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad1004e300 con 0x7fad10068ae0 2023-09-10T00:39:20.673 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.668+0000 7facf7fff700 1 -- 172.21.15.44:0/741220572 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fad0400fb00 con 0x7fad10068ae0 2023-09-10T00:39:20.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.668+0000 7facf7fff700 1 --2- 172.21.15.44:0/741220572 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7facf80a06d0 0x7facf80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:20.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.668+0000 7facf7fff700 1 -- 172.21.15.44:0/741220572 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fad040cdf20 con 0x7fad10068ae0 2023-09-10T00:39:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.668+0000 7fad0effd700 1 --2- 172.21.15.44:0/741220572 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7facf80a06d0 0x7facf80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.669+0000 7facf7fff700 1 --2- 172.21.15.44:0/741220572 >> [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338] conn(0x7facf80ab6f0 0x7facf80adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.669+0000 7facf7fff700 1 -- 172.21.15.44:0/741220572 --> [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7facf80ae260 con 0x7facf80ab6f0 2023-09-10T00:39:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.669+0000 7facf7fff700 1 -- 172.21.15.44:0/741220572 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad040ce330 con 0x7fad10068ae0 2023-09-10T00:39:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.669+0000 7fad0effd700 1 --2- 172.21.15.44:0/741220572 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7facf80a06d0 0x7facf80a2b90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fad000099d0 tx=0x7fad00008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.669+0000 7fad0f7fe700 1 --2- 172.21.15.44:0/741220572 >> [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338] conn(0x7facf80ab6f0 0x7facf80adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.670+0000 7fad0f7fe700 1 --2- 172.21.15.44:0/741220572 >> [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338] conn(0x7facf80ab6f0 0x7facf80adb90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.670+0000 7facf7fff700 1 -- 172.21.15.44:0/741220572 <== osd.7 v2:172.21.15.99:6816/107594338 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7facf80ae260 con 0x7facf80ab6f0 2023-09-10T00:39:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.690+0000 7fad157c1700 1 -- 172.21.15.44:0/741220572 --> [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fad10110820 con 0x7facf80ab6f0 2023-09-10T00:39:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.691+0000 7facf7fff700 1 -- 172.21.15.44:0/741220572 <== osd.7 v2:172.21.15.99:6816/107594338 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fad0800f040 con 0x7facf80ab6f0 2023-09-10T00:39:20.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.692+0000 7fad157c1700 1 -- 172.21.15.44:0/741220572 >> [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338] conn(0x7facf80ab6f0 msgr2=0x7facf80adb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.692+0000 7fad157c1700 1 --2- 172.21.15.44:0/741220572 >> [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338] conn(0x7facf80ab6f0 0x7facf80adb90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.692+0000 7fad157c1700 1 -- 172.21.15.44:0/741220572 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7facf80a06d0 msgr2=0x7facf80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:20.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.692+0000 7fad157c1700 1 --2- 172.21.15.44:0/741220572 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7facf80a06d0 0x7facf80a2b90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fad000099d0 tx=0x7fad00008040 comp rx=0 tx=0).stop 2023-09-10T00:39:20.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.692+0000 7fad157c1700 1 -- 172.21.15.44:0/741220572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad10068ae0 msgr2=0x7fad10196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:20.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.692+0000 7fad157c1700 1 --2- 172.21.15.44:0/741220572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad10068ae0 0x7fad10196bd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fad04005910 tx=0x7fad040047b0 comp rx=0 tx=0).stop 2023-09-10T00:39:20.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.692+0000 7fad0f7fe700 1 -- 172.21.15.44:0/741220572 reap_dead start 2023-09-10T00:39:20.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.692+0000 7fad157c1700 1 -- 172.21.15.44:0/741220572 shutdown_connections 2023-09-10T00:39:20.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.692+0000 7fad157c1700 1 -- 172.21.15.44:0/741220572 >> 172.21.15.44:0/741220572 conn(0x7fad10076600 msgr2=0x7fad100fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:20.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.692+0000 7fad157c1700 1 -- 172.21.15.44:0/741220572 shutdown_connections 2023-09-10T00:39:20.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:20.692+0000 7fad157c1700 1 -- 172.21.15.44:0/741220572 wait complete. 2023-09-10T00:39:21.793 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:21 smithi044 ceph-mon[98676]: pgmap v291: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 639 B/s wr, 11 op/s 2023-09-10T00:39:21.794 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:21 smithi131 ceph-mon[103713]: pgmap v291: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 639 B/s wr, 11 op/s 2023-09-10T00:39:21.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:21 smithi099 ceph-mon[102692]: pgmap v291: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 639 B/s wr, 11 op/s 2023-09-10T00:39:22.864 INFO:teuthology.orchestra.run.smithi044.stdout:214748364835 2023-09-10T00:39:22.865 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.7 2023-09-10T00:39:23.702 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:23 smithi044 ceph-mon[98676]: pgmap v292: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 639 B/s wr, 11 op/s 2023-09-10T00:39:23.794 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:23 smithi131 ceph-mon[103713]: pgmap v292: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 639 B/s wr, 11 op/s 2023-09-10T00:39:23.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:23 smithi099 ceph-mon[102692]: pgmap v292: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 639 B/s wr, 11 op/s 2023-09-10T00:39:23.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.753+0000 7fee8aa68700 1 -- 172.21.15.44:0/629459260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee84104160 msgr2=0x7fee841045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:23.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.753+0000 7fee8aa68700 1 --2- 172.21.15.44:0/629459260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee84104160 0x7fee841045e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fee700099c0 tx=0x7fee70009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:23.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.753+0000 7fee8aa68700 1 -- 172.21.15.44:0/629459260 shutdown_connections 2023-09-10T00:39:23.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.753+0000 7fee8aa68700 1 --2- 172.21.15.44:0/629459260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee84104b20 0x7fee8410c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.753+0000 7fee8aa68700 1 --2- 172.21.15.44:0/629459260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee840ffa30 0x7fee840ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.753+0000 7fee8aa68700 1 --2- 172.21.15.44:0/629459260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee84104160 0x7fee841045e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.753+0000 7fee8aa68700 1 -- 172.21.15.44:0/629459260 >> 172.21.15.44:0/629459260 conn(0x7fee84076430 msgr2=0x7fee84076840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.753+0000 7fee8aa68700 1 -- 172.21.15.44:0/629459260 shutdown_connections 2023-09-10T00:39:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.753+0000 7fee8aa68700 1 -- 172.21.15.44:0/629459260 wait complete. 2023-09-10T00:39:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.754+0000 7fee8aa68700 1 Processor -- start 2023-09-10T00:39:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.754+0000 7fee8aa68700 1 -- start start 2023-09-10T00:39:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.754+0000 7fee8aa68700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee840ffa30 0x7fee841965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.754+0000 7fee8aa68700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee84104160 0x7fee84196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.754+0000 7fee8aa68700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee84104b20 0x7fee8419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.754+0000 7fee8aa68700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8419b570 con 0x7fee84104160 2023-09-10T00:39:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.754+0000 7fee8aa68700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8419b6e0 con 0x7fee840ffa30 2023-09-10T00:39:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.754+0000 7fee8aa68700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8419b8a0 con 0x7fee84104b20 2023-09-10T00:39:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.754+0000 7fee83fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee84104160 0x7fee84196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.754+0000 7fee83fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee84104160 0x7fee84196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46990/0 (socket says 172.21.15.44:46990) 2023-09-10T00:39:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.754+0000 7fee83fff700 1 -- 172.21.15.44:0/1490635463 learned_addr learned my addr 172.21.15.44:0/1490635463 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.755+0000 7fee88804700 1 --2- 172.21.15.44:0/1490635463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee840ffa30 0x7fee841965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.755+0000 7fee89005700 1 --2- 172.21.15.44:0/1490635463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee84104b20 0x7fee8419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.755+0000 7fee83fff700 1 -- 172.21.15.44:0/1490635463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee84104b20 msgr2=0x7fee8419af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.755+0000 7fee83fff700 1 --2- 172.21.15.44:0/1490635463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee84104b20 0x7fee8419af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:23.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.755+0000 7fee83fff700 1 -- 172.21.15.44:0/1490635463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee840ffa30 msgr2=0x7fee841965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:23.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.755+0000 7fee83fff700 1 --2- 172.21.15.44:0/1490635463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee840ffa30 0x7fee841965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:23.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.755+0000 7fee83fff700 1 -- 172.21.15.44:0/1490635463 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee700095f0 con 0x7fee84104160 2023-09-10T00:39:23.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.755+0000 7fee88804700 1 --2- 172.21.15.44:0/1490635463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee840ffa30 0x7fee841965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:23.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.755+0000 7fee83fff700 1 --2- 172.21.15.44:0/1490635463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee84104160 0x7fee84196af0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fee70009fd0 tx=0x7fee7000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:23.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.755+0000 7fee81ffb700 1 -- 172.21.15.44:0/1490635463 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7001d070 con 0x7fee84104160 2023-09-10T00:39:23.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.755+0000 7fee81ffb700 1 -- 172.21.15.44:0/1490635463 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee700054c0 con 0x7fee84104160 2023-09-10T00:39:23.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.755+0000 7fee8aa68700 1 -- 172.21.15.44:0/1490635463 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee8419bb20 con 0x7fee84104160 2023-09-10T00:39:23.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.755+0000 7fee81ffb700 1 -- 172.21.15.44:0/1490635463 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70004b30 con 0x7fee84104160 2023-09-10T00:39:23.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.755+0000 7fee8aa68700 1 -- 172.21.15.44:0/1490635463 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee841a33e0 con 0x7fee84104160 2023-09-10T00:39:23.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.756+0000 7fee8aa68700 1 -- 172.21.15.44:0/1490635463 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee8404e300 con 0x7fee84104160 2023-09-10T00:39:23.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.757+0000 7fee81ffb700 1 -- 172.21.15.44:0/1490635463 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fee7000fc30 con 0x7fee84104160 2023-09-10T00:39:23.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.758+0000 7fee81ffb700 1 --2- 172.21.15.44:0/1490635463 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fee740a0750 0x7fee740a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:23.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.758+0000 7fee88804700 1 --2- 172.21.15.44:0/1490635463 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fee740a0750 0x7fee740a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:23.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.759+0000 7fee81ffb700 1 -- 172.21.15.44:0/1490635463 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fee700cd5b0 con 0x7fee84104160 2023-09-10T00:39:23.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.759+0000 7fee81ffb700 1 --2- 172.21.15.44:0/1490635463 >> [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640] conn(0x7fee740ab770 0x7fee740adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:23.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.759+0000 7fee81ffb700 1 -- 172.21.15.44:0/1490635463 --> [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee740ae2e0 con 0x7fee740ab770 2023-09-10T00:39:23.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.759+0000 7fee81ffb700 1 -- 172.21.15.44:0/1490635463 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee700cd960 con 0x7fee84104160 2023-09-10T00:39:23.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.759+0000 7fee88804700 1 --2- 172.21.15.44:0/1490635463 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fee740a0750 0x7fee740a2c10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fee78005950 tx=0x7fee78005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:23.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.759+0000 7fee89005700 1 --2- 172.21.15.44:0/1490635463 >> [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640] conn(0x7fee740ab770 0x7fee740adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:23.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.759+0000 7fee89005700 1 --2- 172.21.15.44:0/1490635463 >> [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640] conn(0x7fee740ab770 0x7fee740adc10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:23.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.760+0000 7fee81ffb700 1 -- 172.21.15.44:0/1490635463 <== osd.9 v2:172.21.15.44:6826/1885092640 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fee740ae2e0 con 0x7fee740ab770 2023-09-10T00:39:23.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.777+0000 7fee8aa68700 1 -- 172.21.15.44:0/1490635463 --> [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fee8410f630 con 0x7fee740ab770 2023-09-10T00:39:23.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.777+0000 7fee81ffb700 1 -- 172.21.15.44:0/1490635463 <== osd.9 v2:172.21.15.44:6826/1885092640 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fee7c0092c0 con 0x7fee740ab770 2023-09-10T00:39:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.777+0000 7fee8aa68700 1 -- 172.21.15.44:0/1490635463 >> [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640] conn(0x7fee740ab770 msgr2=0x7fee740adc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.777+0000 7fee8aa68700 1 --2- 172.21.15.44:0/1490635463 >> [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640] conn(0x7fee740ab770 0x7fee740adc10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.777+0000 7fee8aa68700 1 -- 172.21.15.44:0/1490635463 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fee740a0750 msgr2=0x7fee740a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.777+0000 7fee8aa68700 1 --2- 172.21.15.44:0/1490635463 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fee740a0750 0x7fee740a2c10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fee78005950 tx=0x7fee78005f50 comp rx=0 tx=0).stop 2023-09-10T00:39:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.777+0000 7fee8aa68700 1 -- 172.21.15.44:0/1490635463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee84104160 msgr2=0x7fee84196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.777+0000 7fee8aa68700 1 --2- 172.21.15.44:0/1490635463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee84104160 0x7fee84196af0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fee70009fd0 tx=0x7fee7000f740 comp rx=0 tx=0).stop 2023-09-10T00:39:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.777+0000 7fee89005700 1 -- 172.21.15.44:0/1490635463 reap_dead start 2023-09-10T00:39:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.777+0000 7fee8aa68700 1 -- 172.21.15.44:0/1490635463 shutdown_connections 2023-09-10T00:39:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.777+0000 7fee8aa68700 1 -- 172.21.15.44:0/1490635463 >> 172.21.15.44:0/1490635463 conn(0x7fee84076430 msgr2=0x7fee841038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.777+0000 7fee8aa68700 1 -- 172.21.15.44:0/1490635463 shutdown_connections 2023-09-10T00:39:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.778+0000 7fee8aa68700 1 -- 172.21.15.44:0/1490635463 wait complete. 2023-09-10T00:39:23.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.991+0000 7fa8ce9d5700 1 -- 172.21.15.44:0/1767697328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c8068120 msgr2=0x7fa8c80685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:23.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.991+0000 7fa8ce9d5700 1 --2- 172.21.15.44:0/1767697328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c8068120 0x7fa8c80685a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009990 tx=0x7fa8bc009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.992+0000 7fa8ce9d5700 1 -- 172.21.15.44:0/1767697328 shutdown_connections 2023-09-10T00:39:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.992+0000 7fa8ce9d5700 1 --2- 172.21.15.44:0/1767697328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8c8068ae0 0x7fa8c810d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.992+0000 7fa8ce9d5700 1 --2- 172.21.15.44:0/1767697328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8c8106770 0x7fa8c8106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.992+0000 7fa8ce9d5700 1 --2- 172.21.15.44:0/1767697328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c8068120 0x7fa8c80685a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.992+0000 7fa8ce9d5700 1 -- 172.21.15.44:0/1767697328 >> 172.21.15.44:0/1767697328 conn(0x7fa8c8076600 msgr2=0x7fa8c8076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.992+0000 7fa8ce9d5700 1 -- 172.21.15.44:0/1767697328 shutdown_connections 2023-09-10T00:39:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.992+0000 7fa8ce9d5700 1 -- 172.21.15.44:0/1767697328 wait complete. 2023-09-10T00:39:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.992+0000 7fa8ce9d5700 1 Processor -- start 2023-09-10T00:39:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.992+0000 7fa8ce9d5700 1 -- start start 2023-09-10T00:39:23.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8ce9d5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c8068120 0x7fa8c819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:23.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8ce9d5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8c8068ae0 0x7fa8c819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:23.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8ce9d5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8c8106770 0x7fa8c81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:23.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8ce9d5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8198e90 con 0x7fa8c8068120 2023-09-10T00:39:23.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8ce9d5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8199000 con 0x7fa8c8068ae0 2023-09-10T00:39:23.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8ce9d5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c81991c0 con 0x7fa8c8106770 2023-09-10T00:39:23.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8c7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c8068120 0x7fa8c819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:23.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8c7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c8068120 0x7fa8c819e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47004/0 (socket says 172.21.15.44:47004) 2023-09-10T00:39:23.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8c7fff700 1 -- 172.21.15.44:0/341436064 learned_addr learned my addr 172.21.15.44:0/341436064 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:23.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8ccf72700 1 --2- 172.21.15.44:0/341436064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8c8106770 0x7fa8c81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:23.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8c77fe700 1 --2- 172.21.15.44:0/341436064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8c8068ae0 0x7fa8c819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:23.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8c7fff700 1 -- 172.21.15.44:0/341436064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8c8106770 msgr2=0x7fa8c81986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:23.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8c7fff700 1 --2- 172.21.15.44:0/341436064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8c8106770 0x7fa8c81986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:23.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8c7fff700 1 -- 172.21.15.44:0/341436064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8c8068ae0 msgr2=0x7fa8c819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:23.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8c7fff700 1 --2- 172.21.15.44:0/341436064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8c8068ae0 0x7fa8c819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:23.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8c7fff700 1 -- 172.21.15.44:0/341436064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8bc0095f0 con 0x7fa8c8068120 2023-09-10T00:39:23.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8c7fff700 1 --2- 172.21.15.44:0/341436064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c8068120 0x7fa8c819e650 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800d950 tx=0x7fa8b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:23.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8c57fa700 1 -- 172.21.15.44:0/341436064 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b800f890 con 0x7fa8c8068120 2023-09-10T00:39:24.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8c57fa700 1 -- 172.21.15.44:0/341436064 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8b800fe80 con 0x7fa8c8068120 2023-09-10T00:39:24.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8ce9d5700 1 -- 172.21.15.44:0/341436064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c81994a0 con 0x7fa8c8068120 2023-09-10T00:39:24.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.993+0000 7fa8c57fa700 1 -- 172.21.15.44:0/341436064 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b800e3b0 con 0x7fa8c8068120 2023-09-10T00:39:24.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.994+0000 7fa8ce9d5700 1 -- 172.21.15.44:0/341436064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c81a55d0 con 0x7fa8c8068120 2023-09-10T00:39:24.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.994+0000 7fa8ce9d5700 1 -- 172.21.15.44:0/341436064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa8c81107e0 con 0x7fa8c8068120 2023-09-10T00:39:24.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.996+0000 7fa8c57fa700 1 -- 172.21.15.44:0/341436064 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fa8b8010460 con 0x7fa8c8068120 2023-09-10T00:39:24.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.996+0000 7fa8c57fa700 1 --2- 172.21.15.44:0/341436064 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa8b00a0750 0x7fa8b00a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:24.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.996+0000 7fa8c57fa700 1 -- 172.21.15.44:0/341436064 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa8b80cc2d0 con 0x7fa8c8068120 2023-09-10T00:39:24.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.996+0000 7fa8c77fe700 1 --2- 172.21.15.44:0/341436064 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa8b00a0750 0x7fa8b00a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:24.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.996+0000 7fa8c57fa700 1 --2- 172.21.15.44:0/341436064 >> [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758] conn(0x7fa8b00ab770 0x7fa8b00adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:24.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.996+0000 7fa8c57fa700 1 -- 172.21.15.44:0/341436064 --> [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8b00ae2e0 con 0x7fa8b00ab770 2023-09-10T00:39:24.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.996+0000 7fa8c57fa700 1 -- 172.21.15.44:0/341436064 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8b80cc640 con 0x7fa8c8068120 2023-09-10T00:39:24.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.997+0000 7fa8c77fe700 1 --2- 172.21.15.44:0/341436064 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa8b00a0750 0x7fa8b00a2c10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8199f70 tx=0x7fa8bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:24.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.997+0000 7fa8ccf72700 1 --2- 172.21.15.44:0/341436064 >> [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758] conn(0x7fa8b00ab770 0x7fa8b00adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:24.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.997+0000 7fa8ccf72700 1 --2- 172.21.15.44:0/341436064 >> [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758] conn(0x7fa8b00ab770 0x7fa8b00adc10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:24.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.998+0000 7fa8c57fa700 1 -- 172.21.15.44:0/341436064 <== osd.1 v2:172.21.15.99:6800/63099758 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fa8b00ae2e0 con 0x7fa8b00ab770 2023-09-10T00:39:24.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.015+0000 7fa8ce9d5700 1 -- 172.21.15.44:0/341436064 --> [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa8c804e300 con 0x7fa8b00ab770 2023-09-10T00:39:24.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.015+0000 7fa8c57fa700 1 -- 172.21.15.44:0/341436064 <== osd.1 v2:172.21.15.99:6800/63099758 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa8c804e300 con 0x7fa8b00ab770 2023-09-10T00:39:24.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.016+0000 7fa8ce9d5700 1 -- 172.21.15.44:0/341436064 >> [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758] conn(0x7fa8b00ab770 msgr2=0x7fa8b00adc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:24.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.016+0000 7fa8ce9d5700 1 --2- 172.21.15.44:0/341436064 >> [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758] conn(0x7fa8b00ab770 0x7fa8b00adc10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:24.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.016+0000 7fa8ce9d5700 1 -- 172.21.15.44:0/341436064 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa8b00a0750 msgr2=0x7fa8b00a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:24.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.016+0000 7fa8ce9d5700 1 --2- 172.21.15.44:0/341436064 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa8b00a0750 0x7fa8b00a2c10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8199f70 tx=0x7fa8bc005fb0 comp rx=0 tx=0).stop 2023-09-10T00:39:24.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.016+0000 7fa8ce9d5700 1 -- 172.21.15.44:0/341436064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c8068120 msgr2=0x7fa8c819e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:24.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.016+0000 7fa8ce9d5700 1 --2- 172.21.15.44:0/341436064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c8068120 0x7fa8c819e650 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800d950 tx=0x7fa8b800dc60 comp rx=0 tx=0).stop 2023-09-10T00:39:24.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.016+0000 7fa8ccf72700 1 -- 172.21.15.44:0/341436064 reap_dead start 2023-09-10T00:39:24.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.017+0000 7fa8ce9d5700 1 -- 172.21.15.44:0/341436064 shutdown_connections 2023-09-10T00:39:24.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.017+0000 7fa8ce9d5700 1 -- 172.21.15.44:0/341436064 >> 172.21.15.44:0/341436064 conn(0x7fa8c8076600 msgr2=0x7fa8c810b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:24.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.017+0000 7fa8ce9d5700 1 -- 172.21.15.44:0/341436064 shutdown_connections 2023-09-10T00:39:24.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.017+0000 7fa8ce9d5700 1 -- 172.21.15.44:0/341436064 wait complete. 2023-09-10T00:39:24.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.996+0000 7f0c7f0d3700 1 -- 172.21.15.44:0/2787544888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c780feef0 msgr2=0x7f0c780ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:24.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.996+0000 7f0c7f0d3700 1 --2- 172.21.15.44:0/2787544888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c780feef0 0x7f0c780ff2d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009990 tx=0x7f0c6c009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:24.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.996+0000 7f0c7f0d3700 1 -- 172.21.15.44:0/2787544888 shutdown_connections 2023-09-10T00:39:24.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.996+0000 7f0c7f0d3700 1 --2- 172.21.15.44:0/2787544888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c78103fd0 0x7f0c7810bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:24.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.996+0000 7f0c7f0d3700 1 --2- 172.21.15.44:0/2787544888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c780feef0 0x7f0c780ff2d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:24.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.996+0000 7f0c7f0d3700 1 --2- 172.21.15.44:0/2787544888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c780ff8a0 0x7f0c781038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:24.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.996+0000 7f0c7f0d3700 1 -- 172.21.15.44:0/2787544888 >> 172.21.15.44:0/2787544888 conn(0x7f0c78075a70 msgr2=0x7f0c780fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:24.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.996+0000 7f0c7f0d3700 1 -- 172.21.15.44:0/2787544888 shutdown_connections 2023-09-10T00:39:24.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.996+0000 7f0c7f0d3700 1 -- 172.21.15.44:0/2787544888 wait complete. 2023-09-10T00:39:24.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.997+0000 7f0c7f0d3700 1 Processor -- start 2023-09-10T00:39:24.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.997+0000 7f0c7f0d3700 1 -- start start 2023-09-10T00:39:24.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.997+0000 7f0c7f0d3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c780feef0 0x7f0c78197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:24.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.997+0000 7f0c7f0d3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c780ff8a0 0x7f0c78197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:24.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.997+0000 7f0c7f0d3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c78103fd0 0x7f0c781a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:24.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.997+0000 7f0c7f0d3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c78198380 con 0x7f0c780feef0 2023-09-10T00:39:24.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.997+0000 7f0c7f0d3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c781a2ac0 con 0x7f0c78103fd0 2023-09-10T00:39:24.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.997+0000 7f0c7f0d3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c781a2c80 con 0x7f0c780ff8a0 2023-09-10T00:39:24.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.997+0000 7f0c7ce6f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c780feef0 0x7f0c78197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:24.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.997+0000 7f0c7ce6f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c780feef0 0x7f0c78197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46974/0 (socket says 172.21.15.44:46974) 2023-09-10T00:39:24.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.997+0000 7f0c7ce6f700 1 -- 172.21.15.44:0/2069190358 learned_addr learned my addr 172.21.15.44:0/2069190358 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:24.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.997+0000 7f0c7d670700 1 --2- 172.21.15.44:0/2069190358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c78103fd0 0x7f0c781a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:24.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.998+0000 7f0c77fff700 1 --2- 172.21.15.44:0/2069190358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c780ff8a0 0x7f0c78197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:24.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.998+0000 7f0c7ce6f700 1 -- 172.21.15.44:0/2069190358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c780ff8a0 msgr2=0x7f0c78197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:24.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.998+0000 7f0c7ce6f700 1 --2- 172.21.15.44:0/2069190358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c780ff8a0 0x7f0c78197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:24.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.998+0000 7f0c7ce6f700 1 -- 172.21.15.44:0/2069190358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c78103fd0 msgr2=0x7f0c781a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:24.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.998+0000 7f0c7ce6f700 1 --2- 172.21.15.44:0/2069190358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c78103fd0 0x7f0c781a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:24.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.998+0000 7f0c7ce6f700 1 -- 172.21.15.44:0/2069190358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c0095f0 con 0x7f0c780feef0 2023-09-10T00:39:24.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.998+0000 7f0c77fff700 1 --2- 172.21.15.44:0/2069190358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c780ff8a0 0x7f0c78197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:39:24.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.998+0000 7f0c7ce6f700 1 --2- 172.21.15.44:0/2069190358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c780feef0 0x7f0c78197500 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c000c00 tx=0x7f0c6c009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:24.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.998+0000 7f0c75ffb700 1 -- 172.21.15.44:0/2069190358 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c01d070 con 0x7f0c780feef0 2023-09-10T00:39:24.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.998+0000 7f0c75ffb700 1 -- 172.21.15.44:0/2069190358 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c6c022470 con 0x7f0c780feef0 2023-09-10T00:39:24.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.998+0000 7f0c7f0d3700 1 -- 172.21.15.44:0/2069190358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c781a2f00 con 0x7f0c780feef0 2023-09-10T00:39:24.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.998+0000 7f0c75ffb700 1 -- 172.21.15.44:0/2069190358 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c017650 con 0x7f0c780feef0 2023-09-10T00:39:24.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.998+0000 7f0c7f0d3700 1 -- 172.21.15.44:0/2069190358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c781a32b0 con 0x7f0c780feef0 2023-09-10T00:39:24.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:22.999+0000 7f0c7f0d3700 1 -- 172.21.15.44:0/2069190358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c7810edb0 con 0x7f0c780feef0 2023-09-10T00:39:24.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.001+0000 7f0c75ffb700 1 -- 172.21.15.44:0/2069190358 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f0c6c00fb00 con 0x7f0c780feef0 2023-09-10T00:39:24.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.002+0000 7f0c75ffb700 1 --2- 172.21.15.44:0/2069190358 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0c680a06d0 0x7f0c680a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:24.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.002+0000 7f0c75ffb700 1 -- 172.21.15.44:0/2069190358 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0c6c0cdf90 con 0x7f0c780feef0 2023-09-10T00:39:24.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.002+0000 7f0c77fff700 1 --2- 172.21.15.44:0/2069190358 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0c680a06d0 0x7f0c680a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:24.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.002+0000 7f0c75ffb700 1 --2- 172.21.15.44:0/2069190358 >> [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490] conn(0x7f0c680ab6f0 0x7f0c680adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:24.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.002+0000 7f0c75ffb700 1 -- 172.21.15.44:0/2069190358 --> [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c680ae260 con 0x7f0c680ab6f0 2023-09-10T00:39:24.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.002+0000 7f0c75ffb700 1 -- 172.21.15.44:0/2069190358 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c6c0ce3a0 con 0x7f0c780feef0 2023-09-10T00:39:24.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.002+0000 7f0c7d670700 1 --2- 172.21.15.44:0/2069190358 >> [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490] conn(0x7f0c680ab6f0 0x7f0c680adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:24.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.002+0000 7f0c77fff700 1 --2- 172.21.15.44:0/2069190358 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0c680a06d0 0x7f0c680a2b90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0c781028e0 tx=0x7f0c64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:24.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.003+0000 7f0c7d670700 1 --2- 172.21.15.44:0/2069190358 >> [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490] conn(0x7f0c680ab6f0 0x7f0c680adb90 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:24.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.004+0000 7f0c75ffb700 1 -- 172.21.15.44:0/2069190358 <== osd.4 v2:172.21.15.99:6808/532684490 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0c680ae260 con 0x7f0c680ab6f0 2023-09-10T00:39:24.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.022+0000 7f0c7f0d3700 1 -- 172.21.15.44:0/2069190358 --> [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0c7804e300 con 0x7f0c680ab6f0 2023-09-10T00:39:24.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.023+0000 7f0c75ffb700 1 -- 172.21.15.44:0/2069190358 <== osd.4 v2:172.21.15.99:6808/532684490 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0c7804e300 con 0x7f0c680ab6f0 2023-09-10T00:39:24.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.024+0000 7f0c7f0d3700 1 -- 172.21.15.44:0/2069190358 >> [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490] conn(0x7f0c680ab6f0 msgr2=0x7f0c680adb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:24.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.024+0000 7f0c7f0d3700 1 --2- 172.21.15.44:0/2069190358 >> [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490] conn(0x7f0c680ab6f0 0x7f0c680adb90 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:24.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.024+0000 7f0c7f0d3700 1 -- 172.21.15.44:0/2069190358 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0c680a06d0 msgr2=0x7f0c680a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:24.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.024+0000 7f0c7f0d3700 1 --2- 172.21.15.44:0/2069190358 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0c680a06d0 0x7f0c680a2b90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0c781028e0 tx=0x7f0c64009260 comp rx=0 tx=0).stop 2023-09-10T00:39:24.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.024+0000 7f0c7f0d3700 1 -- 172.21.15.44:0/2069190358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c780feef0 msgr2=0x7f0c78197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:24.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.024+0000 7f0c7f0d3700 1 --2- 172.21.15.44:0/2069190358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c780feef0 0x7f0c78197500 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c000c00 tx=0x7f0c6c009570 comp rx=0 tx=0).stop 2023-09-10T00:39:24.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.025+0000 7f0c7d670700 1 -- 172.21.15.44:0/2069190358 reap_dead start 2023-09-10T00:39:24.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.025+0000 7f0c7f0d3700 1 -- 172.21.15.44:0/2069190358 shutdown_connections 2023-09-10T00:39:24.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.025+0000 7f0c7f0d3700 1 -- 172.21.15.44:0/2069190358 >> 172.21.15.44:0/2069190358 conn(0x7f0c78075a70 msgr2=0x7f0c780fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:24.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.025+0000 7f0c7f0d3700 1 -- 172.21.15.44:0/2069190358 shutdown_connections 2023-09-10T00:39:24.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:23.025+0000 7f0c7f0d3700 1 -- 172.21.15.44:0/2069190358 wait complete. 2023-09-10T00:39:24.686 INFO:teuthology.orchestra.run.smithi044.stdout:279172874269 2023-09-10T00:39:24.687 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.9 2023-09-10T00:39:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.806+0000 7f1ddec0e700 1 -- 172.21.15.44:0/1995123745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dd80ff120 msgr2=0x7f1dd80ff5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:24.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.806+0000 7f1ddec0e700 1 --2- 172.21.15.44:0/1995123745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dd80ff120 0x7f1dd80ff5a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009990 tx=0x7f1dc4009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:24.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.806+0000 7f1ddec0e700 1 -- 172.21.15.44:0/1995123745 shutdown_connections 2023-09-10T00:39:24.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.806+0000 7f1ddec0e700 1 --2- 172.21.15.44:0/1995123745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8105b50 0x7f1dd8113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:24.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.806+0000 7f1ddec0e700 1 --2- 172.21.15.44:0/1995123745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd8101560 0x7f1dd80feb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:24.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.806+0000 7f1ddec0e700 1 --2- 172.21.15.44:0/1995123745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dd80ff120 0x7f1dd80ff5a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:24.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.806+0000 7f1ddec0e700 1 -- 172.21.15.44:0/1995123745 >> 172.21.15.44:0/1995123745 conn(0x7f1dd8075a70 msgr2=0x7f1dd80fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:24.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.806+0000 7f1ddec0e700 1 -- 172.21.15.44:0/1995123745 shutdown_connections 2023-09-10T00:39:24.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.806+0000 7f1ddec0e700 1 -- 172.21.15.44:0/1995123745 wait complete. 2023-09-10T00:39:24.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.807+0000 7f1ddec0e700 1 Processor -- start 2023-09-10T00:39:24.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.807+0000 7f1ddec0e700 1 -- start start 2023-09-10T00:39:24.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1ddec0e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd80ff120 0x7f1dd81a2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:24.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1ddec0e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dd8101560 0x7f1dd81a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:24.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1ddec0e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8105b50 0x7f1dd819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:24.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1ddec0e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd819d070 con 0x7f1dd8101560 2023-09-10T00:39:24.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1ddec0e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd819d1e0 con 0x7f1dd8105b50 2023-09-10T00:39:24.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1ddec0e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd819d3a0 con 0x7f1dd80ff120 2023-09-10T00:39:24.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1dd7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dd8101560 0x7f1dd81a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:24.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1dd7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dd8101560 0x7f1dd81a2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47022/0 (socket says 172.21.15.44:47022) 2023-09-10T00:39:24.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1dd7fff700 1 -- 172.21.15.44:0/3968609419 learned_addr learned my addr 172.21.15.44:0/3968609419 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:24.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1ddc9aa700 1 --2- 172.21.15.44:0/3968609419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd80ff120 0x7f1dd81a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:24.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1ddd1ab700 1 --2- 172.21.15.44:0/3968609419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8105b50 0x7f1dd819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:24.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1dd7fff700 1 -- 172.21.15.44:0/3968609419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd80ff120 msgr2=0x7f1dd81a2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:24.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1dd7fff700 1 --2- 172.21.15.44:0/3968609419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd80ff120 0x7f1dd81a2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:24.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1dd7fff700 1 -- 172.21.15.44:0/3968609419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8105b50 msgr2=0x7f1dd819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:24.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1dd7fff700 1 --2- 172.21.15.44:0/3968609419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8105b50 0x7f1dd819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:24.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1dd7fff700 1 -- 172.21.15.44:0/3968609419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc40095f0 con 0x7f1dd8101560 2023-09-10T00:39:24.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.808+0000 7f1dd7fff700 1 --2- 172.21.15.44:0/3968609419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dd8101560 0x7f1dd81a2ec0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40036c0 tx=0x7f1dc4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:24.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.809+0000 7f1dd5ffb700 1 -- 172.21.15.44:0/3968609419 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc401d070 con 0x7f1dd8101560 2023-09-10T00:39:24.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.809+0000 7f1ddec0e700 1 -- 172.21.15.44:0/3968609419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd819d620 con 0x7f1dd8101560 2023-09-10T00:39:24.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.809+0000 7f1dd5ffb700 1 -- 172.21.15.44:0/3968609419 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1dc4003ec0 con 0x7f1dd8101560 2023-09-10T00:39:24.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.809+0000 7f1dd5ffb700 1 -- 172.21.15.44:0/3968609419 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4022b00 con 0x7f1dd8101560 2023-09-10T00:39:24.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.809+0000 7f1ddec0e700 1 -- 172.21.15.44:0/3968609419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd81a9970 con 0x7f1dd8101560 2023-09-10T00:39:24.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.810+0000 7f1ddec0e700 1 -- 172.21.15.44:0/3968609419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1dd804e300 con 0x7f1dd8101560 2023-09-10T00:39:24.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.811+0000 7f1dd5ffb700 1 -- 172.21.15.44:0/3968609419 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f1dc4004030 con 0x7f1dd8101560 2023-09-10T00:39:24.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.812+0000 7f1dd5ffb700 1 --2- 172.21.15.44:0/3968609419 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1dc80a06d0 0x7f1dc80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:24.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.813+0000 7f1ddc9aa700 1 --2- 172.21.15.44:0/3968609419 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1dc80a06d0 0x7f1dc80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:24.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.813+0000 7f1dd5ffb700 1 -- 172.21.15.44:0/3968609419 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1dc40ce2e0 con 0x7f1dd8101560 2023-09-10T00:39:24.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.813+0000 7f1dd5ffb700 1 --2- 172.21.15.44:0/3968609419 >> [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032] conn(0x7f1dc80ab6f0 0x7f1dc80adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:24.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.813+0000 7f1dd5ffb700 1 -- 172.21.15.44:0/3968609419 --> [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1dc80ae260 con 0x7f1dc80ab6f0 2023-09-10T00:39:24.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.813+0000 7f1dd5ffb700 1 -- 172.21.15.44:0/3968609419 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dc40ce6f0 con 0x7f1dd8101560 2023-09-10T00:39:24.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.813+0000 7f1ddc9aa700 1 --2- 172.21.15.44:0/3968609419 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1dc80a06d0 0x7f1dc80a2b90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00ac50 tx=0x7f1dcc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:24.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.813+0000 7f1ddd1ab700 1 --2- 172.21.15.44:0/3968609419 >> [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032] conn(0x7f1dc80ab6f0 0x7f1dc80adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:24.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.814+0000 7f1ddd1ab700 1 --2- 172.21.15.44:0/3968609419 >> [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032] conn(0x7f1dc80ab6f0 0x7f1dc80adb90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:24.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.815+0000 7f1dd5ffb700 1 -- 172.21.15.44:0/3968609419 <== osd.10 v2:172.21.15.99:6824/1454470032 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1dc80ae260 con 0x7f1dc80ab6f0 2023-09-10T00:39:24.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.831+0000 7f1ddec0e700 1 -- 172.21.15.44:0/3968609419 --> [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1dd8113600 con 0x7f1dc80ab6f0 2023-09-10T00:39:24.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.832+0000 7f1dd5ffb700 1 -- 172.21.15.44:0/3968609419 <== osd.10 v2:172.21.15.99:6824/1454470032 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1dd8113600 con 0x7f1dc80ab6f0 2023-09-10T00:39:24.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.832+0000 7f1ddec0e700 1 -- 172.21.15.44:0/3968609419 >> [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032] conn(0x7f1dc80ab6f0 msgr2=0x7f1dc80adb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:24.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.832+0000 7f1ddec0e700 1 --2- 172.21.15.44:0/3968609419 >> [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032] conn(0x7f1dc80ab6f0 0x7f1dc80adb90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:24.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.832+0000 7f1ddec0e700 1 -- 172.21.15.44:0/3968609419 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1dc80a06d0 msgr2=0x7f1dc80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:24.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.832+0000 7f1ddec0e700 1 --2- 172.21.15.44:0/3968609419 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1dc80a06d0 0x7f1dc80a2b90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00ac50 tx=0x7f1dcc00a380 comp rx=0 tx=0).stop 2023-09-10T00:39:24.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.832+0000 7f1ddec0e700 1 -- 172.21.15.44:0/3968609419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dd8101560 msgr2=0x7f1dd81a2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:24.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.832+0000 7f1ddec0e700 1 --2- 172.21.15.44:0/3968609419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dd8101560 0x7f1dd81a2ec0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40036c0 tx=0x7f1dc4005740 comp rx=0 tx=0).stop 2023-09-10T00:39:24.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.832+0000 7f1ddd1ab700 1 -- 172.21.15.44:0/3968609419 reap_dead start 2023-09-10T00:39:24.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.832+0000 7f1ddec0e700 1 -- 172.21.15.44:0/3968609419 shutdown_connections 2023-09-10T00:39:24.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.832+0000 7f1ddec0e700 1 -- 172.21.15.44:0/3968609419 >> 172.21.15.44:0/3968609419 conn(0x7f1dd8075a70 msgr2=0x7f1dd8112cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:24.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.832+0000 7f1ddec0e700 1 -- 172.21.15.44:0/3968609419 shutdown_connections 2023-09-10T00:39:24.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:24.832+0000 7f1ddec0e700 1 -- 172.21.15.44:0/3968609419 wait complete. 2023-09-10T00:39:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.174+0000 7f0fd34cb700 1 -- 172.21.15.44:0/426223997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0ffee0 msgr2=0x7f0fcc10c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:25.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.174+0000 7f0fd34cb700 1 --2- 172.21.15.44:0/426223997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0ffee0 0x7f0fcc10c3f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400b0a0 tx=0x7f0fc400b3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:25.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.174+0000 7f0fd34cb700 1 -- 172.21.15.44:0/426223997 shutdown_connections 2023-09-10T00:39:25.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.174+0000 7f0fd34cb700 1 --2- 172.21.15.44:0/426223997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0ffee0 0x7f0fcc10c3f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:25.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.174+0000 7f0fd34cb700 1 --2- 172.21.15.44:0/426223997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fcc104530 0x7f0fcc104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:25.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.174+0000 7f0fd34cb700 1 --2- 172.21.15.44:0/426223997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fcc0ff520 0x7f0fcc0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:25.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.175+0000 7f0fd34cb700 1 -- 172.21.15.44:0/426223997 >> 172.21.15.44:0/426223997 conn(0x7f0fcc076600 msgr2=0x7f0fcc076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:25.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.175+0000 7f0fd34cb700 1 -- 172.21.15.44:0/426223997 shutdown_connections 2023-09-10T00:39:25.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.175+0000 7f0fd34cb700 1 -- 172.21.15.44:0/426223997 wait complete. 2023-09-10T00:39:25.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.175+0000 7f0fd34cb700 1 Processor -- start 2023-09-10T00:39:25.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.175+0000 7f0fd34cb700 1 -- start start 2023-09-10T00:39:25.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.175+0000 7f0fd34cb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fcc0ff520 0x7f0fcc19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:25.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.175+0000 7f0fd34cb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0ffee0 0x7f0fcc19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:25.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.175+0000 7f0fd34cb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fcc104530 0x7f0fcc1965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:25.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.175+0000 7f0fd34cb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc196c50 con 0x7f0fcc0ffee0 2023-09-10T00:39:25.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.175+0000 7f0fd34cb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc196dc0 con 0x7f0fcc0ff520 2023-09-10T00:39:25.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.175+0000 7f0fd34cb700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc196f80 con 0x7f0fcc104530 2023-09-10T00:39:25.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.176+0000 7f0fd0a66700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0ffee0 0x7f0fcc19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:25.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.176+0000 7f0fd0a66700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0ffee0 0x7f0fcc19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47038/0 (socket says 172.21.15.44:47038) 2023-09-10T00:39:25.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.176+0000 7f0fd0a66700 1 -- 172.21.15.44:0/2296152304 learned_addr learned my addr 172.21.15.44:0/2296152304 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:25.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.176+0000 7f0fd1267700 1 --2- 172.21.15.44:0/2296152304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fcc0ff520 0x7f0fcc19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:25.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.176+0000 7f0fd1a68700 1 --2- 172.21.15.44:0/2296152304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fcc104530 0x7f0fcc1965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:25.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.176+0000 7f0fd0a66700 1 -- 172.21.15.44:0/2296152304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fcc104530 msgr2=0x7f0fcc1965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:25.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.176+0000 7f0fd0a66700 1 --2- 172.21.15.44:0/2296152304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fcc104530 0x7f0fcc1965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:25.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.176+0000 7f0fd0a66700 1 -- 172.21.15.44:0/2296152304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fcc0ff520 msgr2=0x7f0fcc19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:25.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.176+0000 7f0fd0a66700 1 --2- 172.21.15.44:0/2296152304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fcc0ff520 0x7f0fcc19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:25.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.176+0000 7f0fd0a66700 1 -- 172.21.15.44:0/2296152304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc4009d00 con 0x7f0fcc0ffee0 2023-09-10T00:39:25.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.176+0000 7f0fd0a66700 1 --2- 172.21.15.44:0/2296152304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0ffee0 0x7f0fcc19caa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00e9a0 tx=0x7f0fbc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:25.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.176+0000 7f0fc27fc700 1 -- 172.21.15.44:0/2296152304 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc009790 con 0x7f0fcc0ffee0 2023-09-10T00:39:25.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.176+0000 7f0fc27fc700 1 -- 172.21.15.44:0/2296152304 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fbc00cbd0 con 0x7f0fcc0ffee0 2023-09-10T00:39:25.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.176+0000 7f0fc27fc700 1 -- 172.21.15.44:0/2296152304 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc018680 con 0x7f0fcc0ffee0 2023-09-10T00:39:25.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.176+0000 7f0fd34cb700 1 -- 172.21.15.44:0/2296152304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fcc197260 con 0x7f0fcc0ffee0 2023-09-10T00:39:25.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.176+0000 7f0fd34cb700 1 -- 172.21.15.44:0/2296152304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc1a3550 con 0x7f0fcc0ffee0 2023-09-10T00:39:25.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.177+0000 7f0fd34cb700 1 -- 172.21.15.44:0/2296152304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0fcc110730 con 0x7f0fcc0ffee0 2023-09-10T00:39:25.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.178+0000 7f0fc27fc700 1 -- 172.21.15.44:0/2296152304 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f0fbc00cd40 con 0x7f0fcc0ffee0 2023-09-10T00:39:25.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.179+0000 7f0fc27fc700 1 --2- 172.21.15.44:0/2296152304 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0fb80a0750 0x7f0fb80a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:25.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.179+0000 7f0fd1267700 1 --2- 172.21.15.44:0/2296152304 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0fb80a0750 0x7f0fb80a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:25.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.179+0000 7f0fc27fc700 1 -- 172.21.15.44:0/2296152304 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0fbc014070 con 0x7f0fcc0ffee0 2023-09-10T00:39:25.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.179+0000 7f0fc27fc700 1 --2- 172.21.15.44:0/2296152304 >> [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432] conn(0x7f0fb80ab770 0x7f0fb80adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:25.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.179+0000 7f0fc27fc700 1 -- 172.21.15.44:0/2296152304 --> [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fb80ae2e0 con 0x7f0fb80ab770 2023-09-10T00:39:25.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.179+0000 7f0fc27fc700 1 -- 172.21.15.44:0/2296152304 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fbc0cca70 con 0x7f0fcc0ffee0 2023-09-10T00:39:25.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.179+0000 7f0fd1267700 1 --2- 172.21.15.44:0/2296152304 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0fb80a0750 0x7f0fb80a2c10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc069c10 tx=0x7f0fc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:25.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.179+0000 7f0fd1a68700 1 --2- 172.21.15.44:0/2296152304 >> [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432] conn(0x7f0fb80ab770 0x7f0fb80adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:25.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.180+0000 7f0fd1a68700 1 --2- 172.21.15.44:0/2296152304 >> [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432] conn(0x7f0fb80ab770 0x7f0fb80adc10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:25.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.181+0000 7f0fc27fc700 1 -- 172.21.15.44:0/2296152304 <== osd.2 v2:172.21.15.131:6800/3245285432 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0fb80ae2e0 con 0x7f0fb80ab770 2023-09-10T00:39:25.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.198+0000 7f0fd34cb700 1 -- 172.21.15.44:0/2296152304 --> [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0fcc04e300 con 0x7f0fb80ab770 2023-09-10T00:39:25.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.199+0000 7f0fc27fc700 1 -- 172.21.15.44:0/2296152304 <== osd.2 v2:172.21.15.131:6800/3245285432 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0fcc04e300 con 0x7f0fb80ab770 2023-09-10T00:39:25.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.199+0000 7f0fd34cb700 1 -- 172.21.15.44:0/2296152304 >> [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432] conn(0x7f0fb80ab770 msgr2=0x7f0fb80adc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:25.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.199+0000 7f0fd34cb700 1 --2- 172.21.15.44:0/2296152304 >> [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432] conn(0x7f0fb80ab770 0x7f0fb80adc10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:25.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.199+0000 7f0fd34cb700 1 -- 172.21.15.44:0/2296152304 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0fb80a0750 msgr2=0x7f0fb80a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:25.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.199+0000 7f0fd34cb700 1 --2- 172.21.15.44:0/2296152304 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0fb80a0750 0x7f0fb80a2c10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc069c10 tx=0x7f0fc8009260 comp rx=0 tx=0).stop 2023-09-10T00:39:25.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.199+0000 7f0fd34cb700 1 -- 172.21.15.44:0/2296152304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0ffee0 msgr2=0x7f0fcc19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:25.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.199+0000 7f0fd34cb700 1 --2- 172.21.15.44:0/2296152304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0ffee0 0x7f0fcc19caa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00e9a0 tx=0x7f0fbc00ecb0 comp rx=0 tx=0).stop 2023-09-10T00:39:25.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.199+0000 7f0fd1a68700 1 -- 172.21.15.44:0/2296152304 reap_dead start 2023-09-10T00:39:25.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.199+0000 7f0fd34cb700 1 -- 172.21.15.44:0/2296152304 shutdown_connections 2023-09-10T00:39:25.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.199+0000 7f0fd34cb700 1 -- 172.21.15.44:0/2296152304 >> 172.21.15.44:0/2296152304 conn(0x7f0fcc076600 msgr2=0x7f0fcc0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:25.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.199+0000 7f0fd34cb700 1 -- 172.21.15.44:0/2296152304 shutdown_connections 2023-09-10T00:39:25.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:25.199+0000 7f0fd34cb700 1 -- 172.21.15.44:0/2296152304 wait complete. 2023-09-10T00:39:25.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:25 smithi131 ceph-mon[103713]: pgmap v293: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2023-09-10T00:39:25.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:25 smithi044 ceph-mon[98676]: pgmap v293: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2023-09-10T00:39:25.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:25 smithi099 ceph-mon[102692]: pgmap v293: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2023-09-10T00:39:25.879 INFO:teuthology.orchestra.run.smithi044.stdout:141733920819 2023-09-10T00:39:25.879 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.4 2023-09-10T00:39:26.317 INFO:teuthology.orchestra.run.smithi044.stdout:55834574912 2023-09-10T00:39:26.317 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.1 2023-09-10T00:39:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.410+0000 7f1a0e3a0700 1 -- 172.21.15.44:0/596828287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a080ffef0 msgr2=0x7f1a0810c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.410+0000 7f1a0e3a0700 1 --2- 172.21.15.44:0/596828287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a080ffef0 0x7f1a0810c3b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000b0d0 tx=0x7f1a0000b3e0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.410+0000 7f1a0e3a0700 1 -- 172.21.15.44:0/596828287 shutdown_connections 2023-09-10T00:39:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.410+0000 7f1a0e3a0700 1 --2- 172.21.15.44:0/596828287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a080ffef0 0x7f1a0810c3b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.410+0000 7f1a0e3a0700 1 --2- 172.21.15.44:0/596828287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a081055e0 0x7f1a081059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.410+0000 7f1a0e3a0700 1 --2- 172.21.15.44:0/596828287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a080ff530 0x7f1a080ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.410+0000 7f1a0e3a0700 1 -- 172.21.15.44:0/596828287 >> 172.21.15.44:0/596828287 conn(0x7f1a080fafc0 msgr2=0x7f1a080fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:26.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.410+0000 7f1a0e3a0700 1 -- 172.21.15.44:0/596828287 shutdown_connections 2023-09-10T00:39:26.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.410+0000 7f1a0e3a0700 1 -- 172.21.15.44:0/596828287 wait complete. 2023-09-10T00:39:26.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.411+0000 7f1a0e3a0700 1 Processor -- start 2023-09-10T00:39:26.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.411+0000 7f1a0e3a0700 1 -- start start 2023-09-10T00:39:26.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.411+0000 7f1a0e3a0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a080ff530 0x7f1a0810cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:26.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.411+0000 7f1a0e3a0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a080ffef0 0x7f1a0810d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:26.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.411+0000 7f1a0e3a0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a081055e0 0x7f1a0810d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:26.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.411+0000 7f1a0e3a0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a081a4ae0 con 0x7f1a081055e0 2023-09-10T00:39:26.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.411+0000 7f1a0e3a0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a081a4c50 con 0x7f1a080ff530 2023-09-10T00:39:26.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.411+0000 7f1a0e3a0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a081a4e10 con 0x7f1a080ffef0 2023-09-10T00:39:26.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.412+0000 7f1a0c93d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a081055e0 0x7f1a0810d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:26.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.412+0000 7f1a0c93d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a081055e0 0x7f1a0810d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47070/0 (socket says 172.21.15.44:47070) 2023-09-10T00:39:26.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.412+0000 7f1a0c93d700 1 -- 172.21.15.44:0/3254018085 learned_addr learned my addr 172.21.15.44:0/3254018085 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:26.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.412+0000 7f1a07fff700 1 --2- 172.21.15.44:0/3254018085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a080ff530 0x7f1a0810cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:26.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.412+0000 7f1a077fe700 1 --2- 172.21.15.44:0/3254018085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a080ffef0 0x7f1a0810d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:26.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.412+0000 7f1a0c93d700 1 -- 172.21.15.44:0/3254018085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a080ffef0 msgr2=0x7f1a0810d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.412+0000 7f1a0c93d700 1 --2- 172.21.15.44:0/3254018085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a080ffef0 0x7f1a0810d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.412+0000 7f1a0c93d700 1 -- 172.21.15.44:0/3254018085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a080ff530 msgr2=0x7f1a0810cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.412+0000 7f1a0c93d700 1 --2- 172.21.15.44:0/3254018085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a080ff530 0x7f1a0810cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.412+0000 7f1a0c93d700 1 -- 172.21.15.44:0/3254018085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a00009d00 con 0x7f1a081055e0 2023-09-10T00:39:26.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.412+0000 7f1a0c93d700 1 --2- 172.21.15.44:0/3254018085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a081055e0 0x7f1a0810d840 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1a00008ec0 tx=0x7f1a00008ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:26.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.412+0000 7f1a057fa700 1 -- 172.21.15.44:0/3254018085 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0000e040 con 0x7f1a081055e0 2023-09-10T00:39:26.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.413+0000 7f1a0e3a0700 1 -- 172.21.15.44:0/3254018085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a081a5090 con 0x7f1a081055e0 2023-09-10T00:39:26.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.413+0000 7f1a0e3a0700 1 -- 172.21.15.44:0/3254018085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a081a54a0 con 0x7f1a081055e0 2023-09-10T00:39:26.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.413+0000 7f1a057fa700 1 -- 172.21.15.44:0/3254018085 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a00003e90 con 0x7f1a081055e0 2023-09-10T00:39:26.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.413+0000 7f1a057fa700 1 -- 172.21.15.44:0/3254018085 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0001b6c0 con 0x7f1a081055e0 2023-09-10T00:39:26.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.414+0000 7f1a0e3a0700 1 -- 172.21.15.44:0/3254018085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1a08068290 con 0x7f1a081055e0 2023-09-10T00:39:26.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.415+0000 7f1a057fa700 1 -- 172.21.15.44:0/3254018085 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f1a00003a00 con 0x7f1a081055e0 2023-09-10T00:39:26.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.416+0000 7f1a057fa700 1 --2- 172.21.15.44:0/3254018085 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f19f00a0750 0x7f19f00a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:26.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.416+0000 7f1a057fa700 1 -- 172.21.15.44:0/3254018085 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1a000cd1b0 con 0x7f1a081055e0 2023-09-10T00:39:26.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.416+0000 7f1a07fff700 1 --2- 172.21.15.44:0/3254018085 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f19f00a0750 0x7f19f00a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:26.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.416+0000 7f1a057fa700 1 --2- 172.21.15.44:0/3254018085 >> [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632] conn(0x7f19f00ab770 0x7f19f00adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:26.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.416+0000 7f1a057fa700 1 -- 172.21.15.44:0/3254018085 --> [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f19f00ae2e0 con 0x7f19f00ab770 2023-09-10T00:39:26.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.416+0000 7f1a057fa700 1 -- 172.21.15.44:0/3254018085 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a000cf830 con 0x7f1a081055e0 2023-09-10T00:39:26.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.417+0000 7f1a07fff700 1 --2- 172.21.15.44:0/3254018085 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f19f00a0750 0x7f19f00a2c10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009c60 tx=0x7f19f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:26.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.417+0000 7f1a077fe700 1 --2- 172.21.15.44:0/3254018085 >> [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632] conn(0x7f19f00ab770 0x7f19f00adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:26.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.417+0000 7f1a077fe700 1 --2- 172.21.15.44:0/3254018085 >> [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632] conn(0x7f19f00ab770 0x7f19f00adc10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:26.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.418+0000 7f1a057fa700 1 -- 172.21.15.44:0/3254018085 <== osd.11 v2:172.21.15.131:6824/1396964632 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f19f00ae2e0 con 0x7f19f00ab770 2023-09-10T00:39:26.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.435+0000 7f1a0e3a0700 1 -- 172.21.15.44:0/3254018085 --> [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1a08110750 con 0x7f19f00ab770 2023-09-10T00:39:26.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.436+0000 7f1a057fa700 1 -- 172.21.15.44:0/3254018085 <== osd.11 v2:172.21.15.131:6824/1396964632 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f19fc00bd30 con 0x7f19f00ab770 2023-09-10T00:39:26.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.436+0000 7f1a0e3a0700 1 -- 172.21.15.44:0/3254018085 >> [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632] conn(0x7f19f00ab770 msgr2=0x7f19f00adc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.436+0000 7f1a0e3a0700 1 --2- 172.21.15.44:0/3254018085 >> [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632] conn(0x7f19f00ab770 0x7f19f00adc10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.436+0000 7f1a0e3a0700 1 -- 172.21.15.44:0/3254018085 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f19f00a0750 msgr2=0x7f19f00a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.436+0000 7f1a0e3a0700 1 --2- 172.21.15.44:0/3254018085 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f19f00a0750 0x7f19f00a2c10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009c60 tx=0x7f19f8009260 comp rx=0 tx=0).stop 2023-09-10T00:39:26.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.436+0000 7f1a0e3a0700 1 -- 172.21.15.44:0/3254018085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a081055e0 msgr2=0x7f1a0810d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.436+0000 7f1a0e3a0700 1 --2- 172.21.15.44:0/3254018085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a081055e0 0x7f1a0810d840 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1a00008ec0 tx=0x7f1a00008ef0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.436+0000 7f1a0c93d700 1 -- 172.21.15.44:0/3254018085 reap_dead start 2023-09-10T00:39:26.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.436+0000 7f1a0e3a0700 1 -- 172.21.15.44:0/3254018085 shutdown_connections 2023-09-10T00:39:26.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.436+0000 7f1a0e3a0700 1 -- 172.21.15.44:0/3254018085 >> 172.21.15.44:0/3254018085 conn(0x7f1a080fafc0 msgr2=0x7f1a080fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:26.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.436+0000 7f1a0e3a0700 1 -- 172.21.15.44:0/3254018085 shutdown_connections 2023-09-10T00:39:26.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.436+0000 7f1a0e3a0700 1 -- 172.21.15.44:0/3254018085 wait complete. 2023-09-10T00:39:26.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.333+0000 7f0fc4aaa700 1 -- 172.21.15.44:0/240113602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00ff4b0 msgr2=0x7f0fc010d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.333+0000 7f0fc4aaa700 1 --2- 172.21.15.44:0/240113602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00ff4b0 0x7f0fc010d4e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800b0a0 tx=0x7f0fb800b3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.333+0000 7f0fc4aaa700 1 -- 172.21.15.44:0/240113602 shutdown_connections 2023-09-10T00:39:26.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.333+0000 7f0fc4aaa700 1 --2- 172.21.15.44:0/240113602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00ff4b0 0x7f0fc010d4e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.333+0000 7f0fc4aaa700 1 --2- 172.21.15.44:0/240113602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fc0106610 0x7f0fc01069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.333+0000 7f0fc4aaa700 1 --2- 172.21.15.44:0/240113602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc00feaf0 0x7f0fc00fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.333+0000 7f0fc4aaa700 1 -- 172.21.15.44:0/240113602 >> 172.21.15.44:0/240113602 conn(0x7f0fc0075a70 msgr2=0x7f0fc00fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:26.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.333+0000 7f0fc4aaa700 1 -- 172.21.15.44:0/240113602 shutdown_connections 2023-09-10T00:39:26.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.333+0000 7f0fc4aaa700 1 -- 172.21.15.44:0/240113602 wait complete. 2023-09-10T00:39:26.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.334+0000 7f0fc4aaa700 1 Processor -- start 2023-09-10T00:39:26.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.334+0000 7f0fc4aaa700 1 -- start start 2023-09-10T00:39:26.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.334+0000 7f0fc4aaa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00feaf0 0x7f0fc019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:26.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.334+0000 7f0fc4aaa700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc00ff4b0 0x7f0fc019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:26.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.334+0000 7f0fc4aaa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fc0106610 0x7f0fc0196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:26.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.334+0000 7f0fc4aaa700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc0196be0 con 0x7f0fc00feaf0 2023-09-10T00:39:26.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.334+0000 7f0fc4aaa700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc0196d50 con 0x7f0fc0106610 2023-09-10T00:39:26.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.334+0000 7f0fc4aaa700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc0196f10 con 0x7f0fc00ff4b0 2023-09-10T00:39:26.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.335+0000 7f0fbe59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00feaf0 0x7f0fc019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:26.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.335+0000 7f0fbe59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00feaf0 0x7f0fc019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47052/0 (socket says 172.21.15.44:47052) 2023-09-10T00:39:26.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.335+0000 7f0fbe59c700 1 -- 172.21.15.44:0/51851998 learned_addr learned my addr 172.21.15.44:0/51851998 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:26.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.335+0000 7f0fbed9d700 1 --2- 172.21.15.44:0/51851998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fc0106610 0x7f0fc0196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:26.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.335+0000 7f0fbdd9b700 1 --2- 172.21.15.44:0/51851998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc00ff4b0 0x7f0fc019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:26.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.335+0000 7f0fbe59c700 1 -- 172.21.15.44:0/51851998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc00ff4b0 msgr2=0x7f0fc019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.335+0000 7f0fbe59c700 1 --2- 172.21.15.44:0/51851998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc00ff4b0 0x7f0fc019ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.335+0000 7f0fbe59c700 1 -- 172.21.15.44:0/51851998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fc0106610 msgr2=0x7f0fc0196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.335+0000 7f0fbe59c700 1 --2- 172.21.15.44:0/51851998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fc0106610 0x7f0fc0196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.335+0000 7f0fbe59c700 1 -- 172.21.15.44:0/51851998 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb8009d00 con 0x7f0fc00feaf0 2023-09-10T00:39:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.335+0000 7f0fbed9d700 1 --2- 172.21.15.44:0/51851998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fc0106610 0x7f0fc0196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:39:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.335+0000 7f0fbe59c700 1 --2- 172.21.15.44:0/51851998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00feaf0 0x7f0fc019c4f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000e9a0 tx=0x7f0fb000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.336+0000 7f0faf7fe700 1 -- 172.21.15.44:0/51851998 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0009790 con 0x7f0fc00feaf0 2023-09-10T00:39:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.336+0000 7f0fc4aaa700 1 -- 172.21.15.44:0/51851998 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc01971f0 con 0x7f0fc00feaf0 2023-09-10T00:39:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.336+0000 7f0faf7fe700 1 -- 172.21.15.44:0/51851998 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fb0010430 con 0x7f0fc00feaf0 2023-09-10T00:39:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.336+0000 7f0faf7fe700 1 -- 172.21.15.44:0/51851998 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb000cea0 con 0x7f0fc00feaf0 2023-09-10T00:39:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.336+0000 7f0fc4aaa700 1 -- 172.21.15.44:0/51851998 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc00ffc00 con 0x7f0fc00feaf0 2023-09-10T00:39:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.337+0000 7f0fc4aaa700 1 -- 172.21.15.44:0/51851998 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0fc0110750 con 0x7f0fc00feaf0 2023-09-10T00:39:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.340+0000 7f0faf7fe700 1 -- 172.21.15.44:0/51851998 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f0fb00105a0 con 0x7f0fc00feaf0 2023-09-10T00:39:26.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.341+0000 7f0faf7fe700 1 --2- 172.21.15.44:0/51851998 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0fa80a4ae0 0x7f0fa80a6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.341+0000 7f0faf7fe700 1 -- 172.21.15.44:0/51851998 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0fb0014070 con 0x7f0fc00feaf0 2023-09-10T00:39:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.341+0000 7f0fbdd9b700 1 --2- 172.21.15.44:0/51851998 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0fa80a4ae0 0x7f0fa80a6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.341+0000 7f0faf7fe700 1 --2- 172.21.15.44:0/51851998 >> [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584] conn(0x7f0fa80afb00 0x7f0fa80b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.341+0000 7f0faf7fe700 1 -- 172.21.15.44:0/51851998 --> [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fa80b2670 con 0x7f0fa80afb00 2023-09-10T00:39:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.341+0000 7f0faf7fe700 1 -- 172.21.15.44:0/51851998 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fb00cc850 con 0x7f0fc00feaf0 2023-09-10T00:39:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.341+0000 7f0fbed9d700 1 --2- 172.21.15.44:0/51851998 >> [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584] conn(0x7f0fa80afb00 0x7f0fa80b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:26.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.341+0000 7f0fbdd9b700 1 --2- 172.21.15.44:0/51851998 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0fa80a4ae0 0x7f0fa80a6fa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00fcbb0 tx=0x7f0fb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:26.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.342+0000 7f0fbed9d700 1 --2- 172.21.15.44:0/51851998 >> [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584] conn(0x7f0fa80afb00 0x7f0fa80b1fa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:26.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.342+0000 7f0faf7fe700 1 -- 172.21.15.44:0/51851998 <== osd.3 v2:172.21.15.44:6810/2588997584 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0fa80b2670 con 0x7f0fa80afb00 2023-09-10T00:39:26.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.376+0000 7f0fc4aaa700 1 -- 172.21.15.44:0/51851998 --> [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0fc004e300 con 0x7f0fa80afb00 2023-09-10T00:39:26.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.377+0000 7f0faf7fe700 1 -- 172.21.15.44:0/51851998 <== osd.3 v2:172.21.15.44:6810/2588997584 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0fc004e300 con 0x7f0fa80afb00 2023-09-10T00:39:26.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.377+0000 7f0fc4aaa700 1 -- 172.21.15.44:0/51851998 >> [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584] conn(0x7f0fa80afb00 msgr2=0x7f0fa80b1fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.377+0000 7f0fc4aaa700 1 --2- 172.21.15.44:0/51851998 >> [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584] conn(0x7f0fa80afb00 0x7f0fa80b1fa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.377+0000 7f0fc4aaa700 1 -- 172.21.15.44:0/51851998 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0fa80a4ae0 msgr2=0x7f0fa80a6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.377+0000 7f0fc4aaa700 1 --2- 172.21.15.44:0/51851998 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0fa80a4ae0 0x7f0fa80a6fa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00fcbb0 tx=0x7f0fb4009260 comp rx=0 tx=0).stop 2023-09-10T00:39:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.377+0000 7f0fc4aaa700 1 -- 172.21.15.44:0/51851998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00feaf0 msgr2=0x7f0fc019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.377+0000 7f0fc4aaa700 1 --2- 172.21.15.44:0/51851998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00feaf0 0x7f0fc019c4f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000e9a0 tx=0x7f0fb000ecb0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.377+0000 7f0fbed9d700 1 -- 172.21.15.44:0/51851998 reap_dead start 2023-09-10T00:39:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.377+0000 7f0fc4aaa700 1 -- 172.21.15.44:0/51851998 shutdown_connections 2023-09-10T00:39:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.377+0000 7f0fc4aaa700 1 -- 172.21.15.44:0/51851998 >> 172.21.15.44:0/51851998 conn(0x7f0fc0075a70 msgr2=0x7f0fc00fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.377+0000 7f0fc4aaa700 1 -- 172.21.15.44:0/51851998 shutdown_connections 2023-09-10T00:39:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.377+0000 7f0fc4aaa700 1 -- 172.21.15.44:0/51851998 wait complete. 2023-09-10T00:39:26.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.609+0000 7fad07e05700 1 -- 172.21.15.44:0/2088598655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad00104530 msgr2=0x7fad00104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.609+0000 7fad07e05700 1 --2- 172.21.15.44:0/2088598655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad00104530 0x7fad00104910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7facfc00b0a0 tx=0x7facfc00b3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.609+0000 7fad07e05700 1 -- 172.21.15.44:0/2088598655 shutdown_connections 2023-09-10T00:39:26.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.609+0000 7fad07e05700 1 --2- 172.21.15.44:0/2088598655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad000ffee0 0x7fad0010c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.609+0000 7fad07e05700 1 --2- 172.21.15.44:0/2088598655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad00104530 0x7fad00104910 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.609+0000 7fad07e05700 1 --2- 172.21.15.44:0/2088598655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad000ff520 0x7fad000ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.609+0000 7fad07e05700 1 -- 172.21.15.44:0/2088598655 >> 172.21.15.44:0/2088598655 conn(0x7fad00076600 msgr2=0x7fad00076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:26.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.609+0000 7fad07e05700 1 -- 172.21.15.44:0/2088598655 shutdown_connections 2023-09-10T00:39:26.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.609+0000 7fad07e05700 1 -- 172.21.15.44:0/2088598655 wait complete. 2023-09-10T00:39:26.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.609+0000 7fad07e05700 1 Processor -- start 2023-09-10T00:39:26.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad07e05700 1 -- start start 2023-09-10T00:39:26.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad07e05700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad000ff520 0x7fad001976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:26.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad07e05700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad000ffee0 0x7fad00197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:26.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad07e05700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad00104530 0x7fad001a2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:26.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad07e05700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad00198560 con 0x7fad000ffee0 2023-09-10T00:39:26.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad07e05700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad001a2ca0 con 0x7fad00104530 2023-09-10T00:39:26.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad07e05700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad001a2e60 con 0x7fad000ff520 2023-09-10T00:39:26.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad053a0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad000ffee0 0x7fad00197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:26.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad053a0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad000ffee0 0x7fad00197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47086/0 (socket says 172.21.15.44:47086) 2023-09-10T00:39:26.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad053a0700 1 -- 172.21.15.44:0/292438749 learned_addr learned my addr 172.21.15.44:0/292438749 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:26.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad063a2700 1 --2- 172.21.15.44:0/292438749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad00104530 0x7fad001a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:26.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad05ba1700 1 --2- 172.21.15.44:0/292438749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad000ff520 0x7fad001976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:26.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad053a0700 1 -- 172.21.15.44:0/292438749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad000ff520 msgr2=0x7fad001976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad053a0700 1 --2- 172.21.15.44:0/292438749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad000ff520 0x7fad001976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad053a0700 1 -- 172.21.15.44:0/292438749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad00104530 msgr2=0x7fad001a2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad053a0700 1 --2- 172.21.15.44:0/292438749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad00104530 0x7fad001a2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.610+0000 7fad053a0700 1 -- 172.21.15.44:0/292438749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc009d00 con 0x7fad000ffee0 2023-09-10T00:39:26.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.611+0000 7fad053a0700 1 --2- 172.21.15.44:0/292438749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad000ffee0 0x7fad00197c20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7facf000d950 tx=0x7facf000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:26.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.611+0000 7facf6ffd700 1 -- 172.21.15.44:0/292438749 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf000f890 con 0x7fad000ffee0 2023-09-10T00:39:26.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.611+0000 7facf6ffd700 1 -- 172.21.15.44:0/292438749 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facf000fe80 con 0x7fad000ffee0 2023-09-10T00:39:26.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.611+0000 7fad07e05700 1 -- 172.21.15.44:0/292438749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad001a3140 con 0x7fad000ffee0 2023-09-10T00:39:26.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.611+0000 7facf6ffd700 1 -- 172.21.15.44:0/292438749 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf000e3b0 con 0x7fad000ffee0 2023-09-10T00:39:26.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.611+0000 7fad07e05700 1 -- 172.21.15.44:0/292438749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad001a3690 con 0x7fad000ffee0 2023-09-10T00:39:26.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.612+0000 7fad07e05700 1 -- 172.21.15.44:0/292438749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad0010f5c0 con 0x7fad000ffee0 2023-09-10T00:39:26.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.613+0000 7facf6ffd700 1 -- 172.21.15.44:0/292438749 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7facf0010460 con 0x7fad000ffee0 2023-09-10T00:39:26.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.613+0000 7facf6ffd700 1 --2- 172.21.15.44:0/292438749 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7facec0a0750 0x7facec0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:26.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.614+0000 7fad05ba1700 1 --2- 172.21.15.44:0/292438749 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7facec0a0750 0x7facec0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:26.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.614+0000 7facf6ffd700 1 -- 172.21.15.44:0/292438749 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7facf00cc270 con 0x7fad000ffee0 2023-09-10T00:39:26.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.614+0000 7facf6ffd700 1 --2- 172.21.15.44:0/292438749 >> [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767] conn(0x7facec0ab770 0x7facec0adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:26.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.614+0000 7facf6ffd700 1 -- 172.21.15.44:0/292438749 --> [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7facec0ae2e0 con 0x7facec0ab770 2023-09-10T00:39:26.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.614+0000 7facf6ffd700 1 -- 172.21.15.44:0/292438749 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7facf00cc680 con 0x7fad000ffee0 2023-09-10T00:39:26.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.614+0000 7fad063a2700 1 --2- 172.21.15.44:0/292438749 >> [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767] conn(0x7facec0ab770 0x7facec0adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:26.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.614+0000 7fad05ba1700 1 --2- 172.21.15.44:0/292438749 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7facec0a0750 0x7facec0a2c10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7facfc009fd0 tx=0x7facfc00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:26.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.614+0000 7fad063a2700 1 --2- 172.21.15.44:0/292438749 >> [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767] conn(0x7facec0ab770 0x7facec0adc10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:26.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.615+0000 7facf6ffd700 1 -- 172.21.15.44:0/292438749 <== osd.6 v2:172.21.15.44:6818/1858317767 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7facec0ae2e0 con 0x7facec0ab770 2023-09-10T00:39:26.632 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.631+0000 7fad07e05700 1 -- 172.21.15.44:0/292438749 --> [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fad0004e300 con 0x7facec0ab770 2023-09-10T00:39:26.632 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.632+0000 7facf6ffd700 1 -- 172.21.15.44:0/292438749 <== osd.6 v2:172.21.15.44:6818/1858317767 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fad0004e300 con 0x7facec0ab770 2023-09-10T00:39:26.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.632+0000 7fad07e05700 1 -- 172.21.15.44:0/292438749 >> [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767] conn(0x7facec0ab770 msgr2=0x7facec0adc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.632+0000 7fad07e05700 1 --2- 172.21.15.44:0/292438749 >> [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767] conn(0x7facec0ab770 0x7facec0adc10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.632+0000 7fad07e05700 1 -- 172.21.15.44:0/292438749 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7facec0a0750 msgr2=0x7facec0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.632+0000 7fad07e05700 1 --2- 172.21.15.44:0/292438749 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7facec0a0750 0x7facec0a2c10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7facfc009fd0 tx=0x7facfc00b7b0 comp rx=0 tx=0).stop 2023-09-10T00:39:26.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.632+0000 7fad07e05700 1 -- 172.21.15.44:0/292438749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad000ffee0 msgr2=0x7fad00197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:26.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.632+0000 7fad07e05700 1 --2- 172.21.15.44:0/292438749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad000ffee0 0x7fad00197c20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7facf000d950 tx=0x7facf000dc60 comp rx=0 tx=0).stop 2023-09-10T00:39:26.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.632+0000 7fad063a2700 1 -- 172.21.15.44:0/292438749 reap_dead start 2023-09-10T00:39:26.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.632+0000 7fad07e05700 1 -- 172.21.15.44:0/292438749 shutdown_connections 2023-09-10T00:39:26.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.632+0000 7fad07e05700 1 -- 172.21.15.44:0/292438749 >> 172.21.15.44:0/292438749 conn(0x7fad00076600 msgr2=0x7fad000fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:26.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.632+0000 7fad07e05700 1 -- 172.21.15.44:0/292438749 shutdown_connections 2023-09-10T00:39:26.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:26.632+0000 7fad07e05700 1 -- 172.21.15.44:0/292438749 wait complete. 2023-09-10T00:39:27.076 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.076+0000 7f840fafb700 1 -- 172.21.15.44:0/48057131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84080ff760 msgr2=0x7f8408105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:27.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.076+0000 7f840fafb700 1 --2- 172.21.15.44:0/48057131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84080ff760 0x7f8408105220 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009990 tx=0x7f83fc009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:27.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.076+0000 7f840fafb700 1 -- 172.21.15.44:0/48057131 shutdown_connections 2023-09-10T00:39:27.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.076+0000 7f840fafb700 1 --2- 172.21.15.44:0/48057131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8408105950 0x7f840810d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:27.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.076+0000 7f840fafb700 1 --2- 172.21.15.44:0/48057131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84080fedb0 0x7f84080ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:27.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.076+0000 7f840fafb700 1 --2- 172.21.15.44:0/48057131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84080ff760 0x7f8408105220 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:27.078 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.076+0000 7f840fafb700 1 -- 172.21.15.44:0/48057131 >> 172.21.15.44:0/48057131 conn(0x7f8408075a70 msgr2=0x7f84080fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:27.078 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.076+0000 7f840fafb700 1 -- 172.21.15.44:0/48057131 shutdown_connections 2023-09-10T00:39:27.078 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.076+0000 7f840fafb700 1 -- 172.21.15.44:0/48057131 wait complete. 2023-09-10T00:39:27.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.077+0000 7f840fafb700 1 Processor -- start 2023-09-10T00:39:27.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.077+0000 7f840fafb700 1 -- start start 2023-09-10T00:39:27.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.077+0000 7f840fafb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84080fedb0 0x7f8408196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:27.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.077+0000 7f840fafb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84080ff760 0x7f8408196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:27.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.077+0000 7f840fafb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8408105950 0x7f840819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:27.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.077+0000 7f840fafb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840819b4e0 con 0x7f8408105950 2023-09-10T00:39:27.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.077+0000 7f840fafb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840819b650 con 0x7f84080ff760 2023-09-10T00:39:27.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.077+0000 7f840fafb700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840819b810 con 0x7f84080fedb0 2023-09-10T00:39:27.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.077+0000 7f840e098700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8408105950 0x7f840819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:27.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.077+0000 7f840e098700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8408105950 0x7f840819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47104/0 (socket says 172.21.15.44:47104) 2023-09-10T00:39:27.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.077+0000 7f840e098700 1 -- 172.21.15.44:0/804338736 learned_addr learned my addr 172.21.15.44:0/804338736 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:27.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.077+0000 7f840d096700 1 --2- 172.21.15.44:0/804338736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84080ff760 0x7f8408196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:27.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.078+0000 7f840d897700 1 --2- 172.21.15.44:0/804338736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84080fedb0 0x7f8408196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:27.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.078+0000 7f840e098700 1 -- 172.21.15.44:0/804338736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84080fedb0 msgr2=0x7f8408196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:27.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.078+0000 7f840e098700 1 --2- 172.21.15.44:0/804338736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84080fedb0 0x7f8408196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:27.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.078+0000 7f840e098700 1 -- 172.21.15.44:0/804338736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84080ff760 msgr2=0x7f8408196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:27.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.078+0000 7f840e098700 1 --2- 172.21.15.44:0/804338736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84080ff760 0x7f8408196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:27.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.078+0000 7f840e098700 1 -- 172.21.15.44:0/804338736 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc0095f0 con 0x7f8408105950 2023-09-10T00:39:27.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.078+0000 7f840e098700 1 --2- 172.21.15.44:0/804338736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8408105950 0x7f840819aed0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f83f800d980 tx=0x7f83f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:27.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.078+0000 7f8406ffd700 1 -- 172.21.15.44:0/804338736 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f8004a30 con 0x7f8408105950 2023-09-10T00:39:27.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.078+0000 7f8406ffd700 1 -- 172.21.15.44:0/804338736 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f83f800b5c0 con 0x7f8408105950 2023-09-10T00:39:27.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.078+0000 7f840fafb700 1 -- 172.21.15.44:0/804338736 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840819baf0 con 0x7f8408105950 2023-09-10T00:39:27.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.078+0000 7f8406ffd700 1 -- 172.21.15.44:0/804338736 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f8009a40 con 0x7f8408105950 2023-09-10T00:39:27.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.078+0000 7f840fafb700 1 -- 172.21.15.44:0/804338736 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84081a3350 con 0x7f8408105950 2023-09-10T00:39:27.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.079+0000 7f840fafb700 1 -- 172.21.15.44:0/804338736 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f840804e300 con 0x7f8408105950 2023-09-10T00:39:27.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.080+0000 7f8406ffd700 1 -- 172.21.15.44:0/804338736 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f83f8009ba0 con 0x7f8408105950 2023-09-10T00:39:27.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.081+0000 7f8406ffd700 1 --2- 172.21.15.44:0/804338736 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f83f00a0750 0x7f83f00a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:27.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.081+0000 7f840d897700 1 --2- 172.21.15.44:0/804338736 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f83f00a0750 0x7f83f00a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:27.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.081+0000 7f8406ffd700 1 -- 172.21.15.44:0/804338736 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f83f80cc4c0 con 0x7f8408105950 2023-09-10T00:39:27.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.081+0000 7f8406ffd700 1 --2- 172.21.15.44:0/804338736 >> [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267] conn(0x7f83f00ab770 0x7f83f00adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:27.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.081+0000 7f8406ffd700 1 -- 172.21.15.44:0/804338736 --> [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f83f00ae2e0 con 0x7f83f00ab770 2023-09-10T00:39:27.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.081+0000 7f8406ffd700 1 -- 172.21.15.44:0/804338736 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83f80cc8d0 con 0x7f8408105950 2023-09-10T00:39:27.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.081+0000 7f840d096700 1 --2- 172.21.15.44:0/804338736 >> [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267] conn(0x7f83f00ab770 0x7f83f00adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:27.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.082+0000 7f840d897700 1 --2- 172.21.15.44:0/804338736 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f83f00a0750 0x7f83f00a2c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8400007fd0 tx=0x7f8400009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:27.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.082+0000 7f840d096700 1 --2- 172.21.15.44:0/804338736 >> [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267] conn(0x7f83f00ab770 0x7f83f00adc10 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:27.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.083+0000 7f8406ffd700 1 -- 172.21.15.44:0/804338736 <== osd.8 v2:172.21.15.131:6816/1528398267 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f83f00ae2e0 con 0x7f83f00ab770 2023-09-10T00:39:27.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.100+0000 7f840fafb700 1 -- 172.21.15.44:0/804338736 --> [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8408110730 con 0x7f83f00ab770 2023-09-10T00:39:27.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.101+0000 7f8406ffd700 1 -- 172.21.15.44:0/804338736 <== osd.8 v2:172.21.15.131:6816/1528398267 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f83fc01e040 con 0x7f83f00ab770 2023-09-10T00:39:27.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.101+0000 7f840fafb700 1 -- 172.21.15.44:0/804338736 >> [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267] conn(0x7f83f00ab770 msgr2=0x7f83f00adc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:27.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.101+0000 7f840fafb700 1 --2- 172.21.15.44:0/804338736 >> [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267] conn(0x7f83f00ab770 0x7f83f00adc10 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:27.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.101+0000 7f840fafb700 1 -- 172.21.15.44:0/804338736 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f83f00a0750 msgr2=0x7f83f00a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:27.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.101+0000 7f840fafb700 1 --2- 172.21.15.44:0/804338736 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f83f00a0750 0x7f83f00a2c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8400007fd0 tx=0x7f8400009040 comp rx=0 tx=0).stop 2023-09-10T00:39:27.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.101+0000 7f840fafb700 1 -- 172.21.15.44:0/804338736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8408105950 msgr2=0x7f840819aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:27.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.101+0000 7f840fafb700 1 --2- 172.21.15.44:0/804338736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8408105950 0x7f840819aed0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f83f800d980 tx=0x7f83f800dc90 comp rx=0 tx=0).stop 2023-09-10T00:39:27.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.101+0000 7f840e098700 1 -- 172.21.15.44:0/804338736 reap_dead start 2023-09-10T00:39:27.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.102+0000 7f840fafb700 1 -- 172.21.15.44:0/804338736 shutdown_connections 2023-09-10T00:39:27.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.102+0000 7f840fafb700 1 -- 172.21.15.44:0/804338736 >> 172.21.15.44:0/804338736 conn(0x7f8408075a70 msgr2=0x7f8408104210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:27.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.102+0000 7f840fafb700 1 -- 172.21.15.44:0/804338736 shutdown_connections 2023-09-10T00:39:27.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.102+0000 7f840fafb700 1 -- 172.21.15.44:0/804338736 wait complete. 2023-09-10T00:39:27.579 INFO:teuthology.orchestra.run.smithi044.stdout:107374182455 2023-09-10T00:39:27.579 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.3 2023-09-10T00:39:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.399+0000 7fe109cb9700 1 -- 172.21.15.44:0/3211477013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe104068ae0 msgr2=0x7fe10410d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:27.583 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.399+0000 7fe109cb9700 1 --2- 172.21.15.44:0/3211477013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe104068ae0 0x7fe10410d440 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00b0a0 tx=0x7fe0fc00b3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:27.583 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.399+0000 7fe109cb9700 1 -- 172.21.15.44:0/3211477013 shutdown_connections 2023-09-10T00:39:27.583 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.399+0000 7fe109cb9700 1 --2- 172.21.15.44:0/3211477013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe104068ae0 0x7fe10410d440 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:27.583 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.399+0000 7fe109cb9700 1 --2- 172.21.15.44:0/3211477013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe104106770 0x7fe104106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:27.583 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.399+0000 7fe109cb9700 1 --2- 172.21.15.44:0/3211477013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe104068120 0x7fe1040685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:27.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.399+0000 7fe109cb9700 1 -- 172.21.15.44:0/3211477013 >> 172.21.15.44:0/3211477013 conn(0x7fe104076600 msgr2=0x7fe104076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:27.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.399+0000 7fe109cb9700 1 -- 172.21.15.44:0/3211477013 shutdown_connections 2023-09-10T00:39:27.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.399+0000 7fe109cb9700 1 -- 172.21.15.44:0/3211477013 wait complete. 2023-09-10T00:39:27.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.400+0000 7fe109cb9700 1 Processor -- start 2023-09-10T00:39:27.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.400+0000 7fe109cb9700 1 -- start start 2023-09-10T00:39:27.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.400+0000 7fe109cb9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe104068120 0x7fe104196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:27.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.400+0000 7fe109cb9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe104068ae0 0x7fe104196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:27.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.400+0000 7fe109cb9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe104106770 0x7fe104199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:27.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.400+0000 7fe109cb9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10419a590 con 0x7fe104068120 2023-09-10T00:39:27.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.400+0000 7fe109cb9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10419a700 con 0x7fe104068ae0 2023-09-10T00:39:27.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.400+0000 7fe109cb9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10419a8c0 con 0x7fe104106770 2023-09-10T00:39:27.586 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.400+0000 7fe1037fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe104068120 0x7fe104196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:27.586 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.400+0000 7fe1037fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe104068120 0x7fe104196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47116/0 (socket says 172.21.15.44:47116) 2023-09-10T00:39:27.586 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.400+0000 7fe1037fe700 1 -- 172.21.15.44:0/2497868346 learned_addr learned my addr 172.21.15.44:0/2497868346 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:27.586 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.401+0000 7fe102ffd700 1 --2- 172.21.15.44:0/2497868346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe104068ae0 0x7fe104196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:27.586 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.401+0000 7fe1037fe700 1 -- 172.21.15.44:0/2497868346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe104106770 msgr2=0x7fe104199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:27.587 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.401+0000 7fe103fff700 1 --2- 172.21.15.44:0/2497868346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe104106770 0x7fe104199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:27.587 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.401+0000 7fe1037fe700 1 --2- 172.21.15.44:0/2497868346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe104106770 0x7fe104199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:27.587 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.401+0000 7fe1037fe700 1 -- 172.21.15.44:0/2497868346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe104068ae0 msgr2=0x7fe104196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:27.587 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.401+0000 7fe1037fe700 1 --2- 172.21.15.44:0/2497868346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe104068ae0 0x7fe104196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:27.587 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.401+0000 7fe1037fe700 1 -- 172.21.15.44:0/2497868346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc009d00 con 0x7fe104068120 2023-09-10T00:39:27.587 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.401+0000 7fe103fff700 1 --2- 172.21.15.44:0/2497868346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe104106770 0x7fe104199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:39:27.588 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.401+0000 7fe1037fe700 1 --2- 172.21.15.44:0/2497868346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe104068120 0x7fe104196690 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400c920 tx=0x7fe0f400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:27.588 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.401+0000 7fe100ff9700 1 -- 172.21.15.44:0/2497868346 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4007b00 con 0x7fe104068120 2023-09-10T00:39:27.588 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.401+0000 7fe100ff9700 1 -- 172.21.15.44:0/2497868346 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe0f400ce70 con 0x7fe104068120 2023-09-10T00:39:27.588 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.401+0000 7fe109cb9700 1 -- 172.21.15.44:0/2497868346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10419aba0 con 0x7fe104068120 2023-09-10T00:39:27.588 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.401+0000 7fe100ff9700 1 -- 172.21.15.44:0/2497868346 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4005440 con 0x7fe104068120 2023-09-10T00:39:27.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.401+0000 7fe109cb9700 1 -- 172.21.15.44:0/2497868346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1041a34f0 con 0x7fe104068120 2023-09-10T00:39:27.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.402+0000 7fe109cb9700 1 -- 172.21.15.44:0/2497868346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1041107e0 con 0x7fe104068120 2023-09-10T00:39:27.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.403+0000 7fe100ff9700 1 -- 172.21.15.44:0/2497868346 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fe0f40055a0 con 0x7fe104068120 2023-09-10T00:39:27.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.404+0000 7fe100ff9700 1 --2- 172.21.15.44:0/2497868346 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe0ec0a0750 0x7fe0ec0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:27.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.404+0000 7fe100ff9700 1 -- 172.21.15.44:0/2497868346 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe0f40cbf60 con 0x7fe104068120 2023-09-10T00:39:27.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.404+0000 7fe102ffd700 1 --2- 172.21.15.44:0/2497868346 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe0ec0a0750 0x7fe0ec0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:27.590 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.404+0000 7fe100ff9700 1 --2- 172.21.15.44:0/2497868346 >> [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489] conn(0x7fe0ec0ab770 0x7fe0ec0adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:27.590 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.404+0000 7fe100ff9700 1 -- 172.21.15.44:0/2497868346 --> [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0ec0ae2e0 con 0x7fe0ec0ab770 2023-09-10T00:39:27.590 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.404+0000 7fe100ff9700 1 -- 172.21.15.44:0/2497868346 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0f40cc370 con 0x7fe104068120 2023-09-10T00:39:27.590 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.404+0000 7fe102ffd700 1 --2- 172.21.15.44:0/2497868346 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe0ec0a0750 0x7fe0ec0a2c10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe104197cb0 tx=0x7fe0f800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:27.590 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.404+0000 7fe103fff700 1 --2- 172.21.15.44:0/2497868346 >> [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489] conn(0x7fe0ec0ab770 0x7fe0ec0adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:27.591 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.405+0000 7fe103fff700 1 --2- 172.21.15.44:0/2497868346 >> [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489] conn(0x7fe0ec0ab770 0x7fe0ec0adc10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:27.591 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.405+0000 7fe100ff9700 1 -- 172.21.15.44:0/2497868346 <== osd.5 v2:172.21.15.131:6808/979810489 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe0ec0ae2e0 con 0x7fe0ec0ab770 2023-09-10T00:39:27.591 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.422+0000 7fe109cb9700 1 -- 172.21.15.44:0/2497868346 --> [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe10404e300 con 0x7fe0ec0ab770 2023-09-10T00:39:27.592 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.422+0000 7fe100ff9700 1 -- 172.21.15.44:0/2497868346 <== osd.5 v2:172.21.15.131:6808/979810489 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe10404e300 con 0x7fe0ec0ab770 2023-09-10T00:39:27.592 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.423+0000 7fe109cb9700 1 -- 172.21.15.44:0/2497868346 >> [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489] conn(0x7fe0ec0ab770 msgr2=0x7fe0ec0adc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:27.592 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.423+0000 7fe109cb9700 1 --2- 172.21.15.44:0/2497868346 >> [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489] conn(0x7fe0ec0ab770 0x7fe0ec0adc10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:27.592 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.423+0000 7fe109cb9700 1 -- 172.21.15.44:0/2497868346 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe0ec0a0750 msgr2=0x7fe0ec0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:27.592 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.423+0000 7fe109cb9700 1 --2- 172.21.15.44:0/2497868346 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe0ec0a0750 0x7fe0ec0a2c10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe104197cb0 tx=0x7fe0f800a380 comp rx=0 tx=0).stop 2023-09-10T00:39:27.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.423+0000 7fe109cb9700 1 -- 172.21.15.44:0/2497868346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe104068120 msgr2=0x7fe104196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:27.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.423+0000 7fe109cb9700 1 --2- 172.21.15.44:0/2497868346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe104068120 0x7fe104196690 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400c920 tx=0x7fe0f400cc30 comp rx=0 tx=0).stop 2023-09-10T00:39:27.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.423+0000 7fe103fff700 1 -- 172.21.15.44:0/2497868346 reap_dead start 2023-09-10T00:39:27.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.423+0000 7fe109cb9700 1 -- 172.21.15.44:0/2497868346 shutdown_connections 2023-09-10T00:39:27.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.423+0000 7fe109cb9700 1 -- 172.21.15.44:0/2497868346 >> 172.21.15.44:0/2497868346 conn(0x7fe104076600 msgr2=0x7fe1040fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:27.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.423+0000 7fe109cb9700 1 -- 172.21.15.44:0/2497868346 shutdown_connections 2023-09-10T00:39:27.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:27.423+0000 7fe109cb9700 1 -- 172.21.15.44:0/2497868346 wait complete. 2023-09-10T00:39:27.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:27 smithi131 ceph-mon[103713]: pgmap v294: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 540 B/s wr, 11 op/s 2023-09-10T00:39:27.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:27 smithi044 ceph-mon[98676]: pgmap v294: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 540 B/s wr, 11 op/s 2023-09-10T00:39:27.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:27 smithi099 ceph-mon[102692]: pgmap v294: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 540 B/s wr, 11 op/s 2023-09-10T00:39:27.976 INFO:teuthology.orchestra.run.smithi044.stdout:77309411387 2023-09-10T00:39:27.976 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.2 2023-09-10T00:39:28.151 INFO:teuthology.orchestra.run.smithi044.stdout:300647710745 2023-09-10T00:39:28.151 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.10 2023-09-10T00:39:28.288 INFO:teuthology.orchestra.run.smithi044.stdout:193273528362 2023-09-10T00:39:28.289 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.6 2023-09-10T00:39:28.320 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:28.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.689+0000 7f63e53a5700 1 -- 172.21.15.44:0/2472401891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e0106770 msgr2=0x7f63e0106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.689+0000 7f63e53a5700 1 --2- 172.21.15.44:0/2472401891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e0106770 0x7f63e0106b50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f63d00098f0 tx=0x7f63d0009c00 comp rx=0 tx=0).stop 2023-09-10T00:39:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.690+0000 7f63e53a5700 1 -- 172.21.15.44:0/2472401891 shutdown_connections 2023-09-10T00:39:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.690+0000 7f63e53a5700 1 --2- 172.21.15.44:0/2472401891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e0068ae0 0x7f63e010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.690+0000 7f63e53a5700 1 --2- 172.21.15.44:0/2472401891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e0106770 0x7f63e0106b50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.690+0000 7f63e53a5700 1 --2- 172.21.15.44:0/2472401891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63e0068120 0x7f63e00685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.690+0000 7f63e53a5700 1 -- 172.21.15.44:0/2472401891 >> 172.21.15.44:0/2472401891 conn(0x7f63e0076600 msgr2=0x7f63e0076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.690+0000 7f63e53a5700 1 -- 172.21.15.44:0/2472401891 shutdown_connections 2023-09-10T00:39:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.690+0000 7f63e53a5700 1 -- 172.21.15.44:0/2472401891 wait complete. 2023-09-10T00:39:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.690+0000 7f63e53a5700 1 Processor -- start 2023-09-10T00:39:28.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63e53a5700 1 -- start start 2023-09-10T00:39:28.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63e53a5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e0068120 0x7f63e00ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:28.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63e53a5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63e0068ae0 0x7f63e00ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:28.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63e53a5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e0106770 0x7f63e0100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:28.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63e53a5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e01a4ad0 con 0x7f63e0068120 2023-09-10T00:39:28.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63e53a5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e01a4c40 con 0x7f63e0068ae0 2023-09-10T00:39:28.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63e53a5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e01a4e00 con 0x7f63e0106770 2023-09-10T00:39:28.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63deffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e0068120 0x7f63e00ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:28.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63deffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e0068120 0x7f63e00ff5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47132/0 (socket says 172.21.15.44:47132) 2023-09-10T00:39:28.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63deffd700 1 -- 172.21.15.44:0/4103911029 learned_addr learned my addr 172.21.15.44:0/4103911029 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:28.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63de7fc700 1 --2- 172.21.15.44:0/4103911029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63e0068ae0 0x7f63e00ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:28.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63df7fe700 1 --2- 172.21.15.44:0/4103911029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e0106770 0x7f63e0100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:28.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63deffd700 1 -- 172.21.15.44:0/4103911029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e0106770 msgr2=0x7f63e0100070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:28.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63deffd700 1 --2- 172.21.15.44:0/4103911029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e0106770 0x7f63e0100070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:28.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63deffd700 1 -- 172.21.15.44:0/4103911029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63e0068ae0 msgr2=0x7f63e00ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:28.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63deffd700 1 --2- 172.21.15.44:0/4103911029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63e0068ae0 0x7f63e00ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:28.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63deffd700 1 -- 172.21.15.44:0/4103911029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d0009520 con 0x7f63e0068120 2023-09-10T00:39:28.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63df7fe700 1 --2- 172.21.15.44:0/4103911029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e0106770 0x7f63e0100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:39:28.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.691+0000 7f63deffd700 1 --2- 172.21.15.44:0/4103911029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e0068120 0x7f63e00ff5b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009810 tx=0x7f63d000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:28.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.692+0000 7f63c7fff700 1 -- 172.21.15.44:0/4103911029 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d001c070 con 0x7f63e0068120 2023-09-10T00:39:28.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.692+0000 7f63e53a5700 1 -- 172.21.15.44:0/4103911029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e01a5080 con 0x7f63e0068120 2023-09-10T00:39:28.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.692+0000 7f63c7fff700 1 -- 172.21.15.44:0/4103911029 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63d000bb70 con 0x7f63e0068120 2023-09-10T00:39:28.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.692+0000 7f63c7fff700 1 -- 172.21.15.44:0/4103911029 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0004930 con 0x7f63e0068120 2023-09-10T00:39:28.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.692+0000 7f63e53a5700 1 -- 172.21.15.44:0/4103911029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e01a54a0 con 0x7f63e0068120 2023-09-10T00:39:28.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.692+0000 7f63e53a5700 1 -- 172.21.15.44:0/4103911029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f63e0110770 con 0x7f63e0068120 2023-09-10T00:39:28.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.694+0000 7f63c7fff700 1 -- 172.21.15.44:0/4103911029 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f63d0020070 con 0x7f63e0068120 2023-09-10T00:39:28.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.694+0000 7f63c7fff700 1 --2- 172.21.15.44:0/4103911029 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f63c80a06d0 0x7f63c80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:28.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.695+0000 7f63de7fc700 1 --2- 172.21.15.44:0/4103911029 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f63c80a06d0 0x7f63c80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:28.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.695+0000 7f63c7fff700 1 -- 172.21.15.44:0/4103911029 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f63d00cd470 con 0x7f63e0068120 2023-09-10T00:39:28.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.695+0000 7f63c7fff700 1 --2- 172.21.15.44:0/4103911029 >> [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980] conn(0x7f63c80ab6f0 0x7f63c80adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:28.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.695+0000 7f63c7fff700 1 -- 172.21.15.44:0/4103911029 --> [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f63c80ae260 con 0x7f63c80ab6f0 2023-09-10T00:39:28.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.695+0000 7f63c7fff700 1 -- 172.21.15.44:0/4103911029 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63d00cd880 con 0x7f63e0068120 2023-09-10T00:39:28.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.695+0000 7f63df7fe700 1 --2- 172.21.15.44:0/4103911029 >> [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980] conn(0x7f63c80ab6f0 0x7f63c80adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:28.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.695+0000 7f63de7fc700 1 --2- 172.21.15.44:0/4103911029 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f63c80a06d0 0x7f63c80a2b90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f63e0100da0 tx=0x7f63d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:28.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.695+0000 7f63df7fe700 1 --2- 172.21.15.44:0/4103911029 >> [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980] conn(0x7f63c80ab6f0 0x7f63c80adb90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:28.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.696+0000 7f63c7fff700 1 -- 172.21.15.44:0/4103911029 <== osd.0 v2:172.21.15.44:6802/424109980 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f63c80ae260 con 0x7f63c80ab6f0 2023-09-10T00:39:28.712 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.712+0000 7f63e53a5700 1 -- 172.21.15.44:0/4103911029 --> [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f63e004e300 con 0x7f63c80ab6f0 2023-09-10T00:39:28.713 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.712+0000 7f63c7fff700 1 -- 172.21.15.44:0/4103911029 <== osd.0 v2:172.21.15.44:6802/424109980 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f63e004e300 con 0x7f63c80ab6f0 2023-09-10T00:39:28.713 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.713+0000 7f63e53a5700 1 -- 172.21.15.44:0/4103911029 >> [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980] conn(0x7f63c80ab6f0 msgr2=0x7f63c80adb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:28.714 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.713+0000 7f63e53a5700 1 --2- 172.21.15.44:0/4103911029 >> [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980] conn(0x7f63c80ab6f0 0x7f63c80adb90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:28.714 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.713+0000 7f63e53a5700 1 -- 172.21.15.44:0/4103911029 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f63c80a06d0 msgr2=0x7f63c80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:28.714 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.713+0000 7f63e53a5700 1 --2- 172.21.15.44:0/4103911029 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f63c80a06d0 0x7f63c80a2b90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f63e0100da0 tx=0x7f63d4009260 comp rx=0 tx=0).stop 2023-09-10T00:39:28.714 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.713+0000 7f63e53a5700 1 -- 172.21.15.44:0/4103911029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e0068120 msgr2=0x7f63e00ff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:28.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.713+0000 7f63e53a5700 1 --2- 172.21.15.44:0/4103911029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e0068120 0x7f63e00ff5b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009810 tx=0x7f63d000fa00 comp rx=0 tx=0).stop 2023-09-10T00:39:28.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.713+0000 7f63df7fe700 1 -- 172.21.15.44:0/4103911029 reap_dead start 2023-09-10T00:39:28.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.713+0000 7f63e53a5700 1 -- 172.21.15.44:0/4103911029 shutdown_connections 2023-09-10T00:39:28.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.713+0000 7f63e53a5700 1 -- 172.21.15.44:0/4103911029 >> 172.21.15.44:0/4103911029 conn(0x7f63e0076600 msgr2=0x7f63e00fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:28.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.713+0000 7f63e53a5700 1 -- 172.21.15.44:0/4103911029 shutdown_connections 2023-09-10T00:39:28.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:28.713+0000 7f63e53a5700 1 -- 172.21.15.44:0/4103911029 wait complete. 2023-09-10T00:39:28.789 INFO:teuthology.orchestra.run.smithi044.stdout:335007449108 2023-09-10T00:39:28.790 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.11 2023-09-10T00:39:29.329 INFO:teuthology.orchestra.run.smithi044.stdout:244813135906 2023-09-10T00:39:29.329 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.8 2023-09-10T00:39:29.347 INFO:teuthology.orchestra.run.smithi044.stdout:163208757295 2023-09-10T00:39:29.348 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.5 2023-09-10T00:39:29.366 INFO:teuthology.orchestra.run.smithi044.stdout:34359738437 2023-09-10T00:39:29.367 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.0 2023-09-10T00:39:29.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:29 smithi131 ceph-mon[103713]: pgmap v295: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 488 B/s wr, 10 op/s 2023-09-10T00:39:29.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:29 smithi044 ceph-mon[98676]: pgmap v295: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 488 B/s wr, 10 op/s 2023-09-10T00:39:29.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:29 smithi099 ceph-mon[102692]: pgmap v295: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 488 B/s wr, 10 op/s 2023-09-10T00:39:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:30.176 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:30.176 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:30.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.457+0000 7f5257450700 1 -- 172.21.15.44:0/1411938968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5250100030 msgr2=0x7f52501004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.457+0000 7f5257450700 1 --2- 172.21.15.44:0/1411938968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5250100030 0x7f52501004b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f52400099c0 tx=0x7f5240009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.457+0000 7f5257450700 1 -- 172.21.15.44:0/1411938968 shutdown_connections 2023-09-10T00:39:30.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.457+0000 7f5257450700 1 --2- 172.21.15.44:0/1411938968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52501009f0 0x7f525010c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.457+0000 7f5257450700 1 --2- 172.21.15.44:0/1411938968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5250104590 0x7f5250104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.457+0000 7f5257450700 1 --2- 172.21.15.44:0/1411938968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5250100030 0x7f52501004b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.457+0000 7f5257450700 1 -- 172.21.15.44:0/1411938968 >> 172.21.15.44:0/1411938968 conn(0x7f5250075a00 msgr2=0x7f5250075e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.457+0000 7f5257450700 1 -- 172.21.15.44:0/1411938968 shutdown_connections 2023-09-10T00:39:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.457+0000 7f5257450700 1 -- 172.21.15.44:0/1411938968 wait complete. 2023-09-10T00:39:30.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f5257450700 1 Processor -- start 2023-09-10T00:39:30.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f5257450700 1 -- start start 2023-09-10T00:39:30.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f5257450700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5250100030 0x7f52501973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:30.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f5257450700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52501009f0 0x7f5250197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:30.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f5257450700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5250104590 0x7f52501a2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:30.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f5257450700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5250198250 con 0x7f52501009f0 2023-09-10T00:39:30.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f5257450700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52501a2990 con 0x7f5250100030 2023-09-10T00:39:30.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f5257450700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52501a2b50 con 0x7f5250104590 2023-09-10T00:39:30.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f52549eb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52501009f0 0x7f5250197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:30.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f52549eb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52501009f0 0x7f5250197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47148/0 (socket says 172.21.15.44:47148) 2023-09-10T00:39:30.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f52549eb700 1 -- 172.21.15.44:0/1220944836 learned_addr learned my addr 172.21.15.44:0/1220944836 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:30.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f52551ec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5250100030 0x7f52501973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:30.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f52559ed700 1 --2- 172.21.15.44:0/1220944836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5250104590 0x7f52501a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:30.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f52549eb700 1 -- 172.21.15.44:0/1220944836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5250104590 msgr2=0x7f52501a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:30.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f52549eb700 1 --2- 172.21.15.44:0/1220944836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5250104590 0x7f52501a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:30.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f52549eb700 1 -- 172.21.15.44:0/1220944836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5250100030 msgr2=0x7f52501973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:30.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f52549eb700 1 --2- 172.21.15.44:0/1220944836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5250100030 0x7f52501973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:30.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.458+0000 7f52549eb700 1 -- 172.21.15.44:0/1220944836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52400095f0 con 0x7f52501009f0 2023-09-10T00:39:30.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.459+0000 7f52549eb700 1 --2- 172.21.15.44:0/1220944836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52501009f0 0x7f5250197910 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5240009fd0 tx=0x7f524000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:30.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.459+0000 7f52467fc700 1 -- 172.21.15.44:0/1220944836 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524001d070 con 0x7f52501009f0 2023-09-10T00:39:30.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.459+0000 7f52467fc700 1 -- 172.21.15.44:0/1220944836 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52400054c0 con 0x7f52501009f0 2023-09-10T00:39:30.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.459+0000 7f52467fc700 1 -- 172.21.15.44:0/1220944836 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240004a90 con 0x7f52501009f0 2023-09-10T00:39:30.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.459+0000 7f5257450700 1 -- 172.21.15.44:0/1220944836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52501a2dd0 con 0x7f52501009f0 2023-09-10T00:39:30.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.459+0000 7f5257450700 1 -- 172.21.15.44:0/1220944836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52501a3190 con 0x7f52501009f0 2023-09-10T00:39:30.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.460+0000 7f5257450700 1 -- 172.21.15.44:0/1220944836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525004e300 con 0x7f52501009f0 2023-09-10T00:39:30.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.461+0000 7f52467fc700 1 -- 172.21.15.44:0/1220944836 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f524000fb80 con 0x7f52501009f0 2023-09-10T00:39:30.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.462+0000 7f52467fc700 1 --2- 172.21.15.44:0/1220944836 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f523c0a0750 0x7f523c0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:30.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.462+0000 7f52551ec700 1 --2- 172.21.15.44:0/1220944836 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f523c0a0750 0x7f523c0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:30.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.462+0000 7f52467fc700 1 -- 172.21.15.44:0/1220944836 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f52400cd200 con 0x7f52501009f0 2023-09-10T00:39:30.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.462+0000 7f52551ec700 1 --2- 172.21.15.44:0/1220944836 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f523c0a0750 0x7f523c0a2c10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f524c009c60 tx=0x7f524c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:30.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.465+0000 7f52467fc700 1 -- 172.21.15.44:0/1220944836 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5240096950 con 0x7f52501009f0 2023-09-10T00:39:30.496 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:30.579 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:30.582 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:30.633 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:30.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.716+0000 7f5257450700 1 -- 172.21.15.44:0/1220944836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f525010f5a0 con 0x7f52501009f0 2023-09-10T00:39:30.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.717+0000 7f52467fc700 1 -- 172.21.15.44:0/1220944836 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5240026020 con 0x7f52501009f0 2023-09-10T00:39:30.717 INFO:teuthology.orchestra.run.smithi044.stdout:214748364837 2023-09-10T00:39:30.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.719+0000 7f5257450700 1 -- 172.21.15.44:0/1220944836 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f523c0a0750 msgr2=0x7f523c0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:30.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.719+0000 7f5257450700 1 --2- 172.21.15.44:0/1220944836 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f523c0a0750 0x7f523c0a2c10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f524c009c60 tx=0x7f524c009260 comp rx=0 tx=0).stop 2023-09-10T00:39:30.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.719+0000 7f5257450700 1 -- 172.21.15.44:0/1220944836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52501009f0 msgr2=0x7f5250197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:30.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.719+0000 7f5257450700 1 --2- 172.21.15.44:0/1220944836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52501009f0 0x7f5250197910 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5240009fd0 tx=0x7f524000f690 comp rx=0 tx=0).stop 2023-09-10T00:39:30.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.719+0000 7f5257450700 1 -- 172.21.15.44:0/1220944836 shutdown_connections 2023-09-10T00:39:30.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.719+0000 7f5257450700 1 --2- 172.21.15.44:0/1220944836 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f523c0a0750 0x7f523c0a2c10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:30.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.719+0000 7f5257450700 1 --2- 172.21.15.44:0/1220944836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5250104590 0x7f52501a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:30.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.719+0000 7f5257450700 1 --2- 172.21.15.44:0/1220944836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5250100030 0x7f52501973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:30.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.719+0000 7f5257450700 1 --2- 172.21.15.44:0/1220944836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52501009f0 0x7f5250197910 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:30.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.719+0000 7f5257450700 1 -- 172.21.15.44:0/1220944836 >> 172.21.15.44:0/1220944836 conn(0x7f5250075a00 msgr2=0x7f52500fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:30.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.719+0000 7f5257450700 1 -- 172.21.15.44:0/1220944836 shutdown_connections 2023-09-10T00:39:30.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:30.719+0000 7f5257450700 1 -- 172.21.15.44:0/1220944836 wait complete. 2023-09-10T00:39:31.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:31 smithi044 ceph-mon[98676]: pgmap v296: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2023-09-10T00:39:31.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:31 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1220944836' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-10T00:39:31.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:31 smithi099 ceph-mon[102692]: pgmap v296: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2023-09-10T00:39:31.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:31 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1220944836' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-10T00:39:32.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:31 smithi131 ceph-mon[103713]: pgmap v296: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2023-09-10T00:39:32.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:31 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1220944836' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-10T00:39:32.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.267+0000 7ff359404700 1 -- 172.21.15.44:0/2847283855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3540ff520 msgr2=0x7ff3540ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:32.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.267+0000 7ff359404700 1 --2- 172.21.15.44:0/2847283855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3540ff520 0x7ff3540ff9a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff348009990 tx=0x7ff348009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:32.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.267+0000 7ff359404700 1 -- 172.21.15.44:0/2847283855 shutdown_connections 2023-09-10T00:39:32.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.267+0000 7ff359404700 1 --2- 172.21.15.44:0/2847283855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3540ffee0 0x7ff35410c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:32.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.267+0000 7ff359404700 1 --2- 172.21.15.44:0/2847283855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff354104530 0x7ff354104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:32.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.267+0000 7ff359404700 1 --2- 172.21.15.44:0/2847283855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3540ff520 0x7ff3540ff9a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:32.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.267+0000 7ff359404700 1 -- 172.21.15.44:0/2847283855 >> 172.21.15.44:0/2847283855 conn(0x7ff354076600 msgr2=0x7ff354076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:32.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.267+0000 7ff359404700 1 -- 172.21.15.44:0/2847283855 shutdown_connections 2023-09-10T00:39:32.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.267+0000 7ff359404700 1 -- 172.21.15.44:0/2847283855 wait complete. 2023-09-10T00:39:32.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.267+0000 7ff359404700 1 Processor -- start 2023-09-10T00:39:32.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.267+0000 7ff359404700 1 -- start start 2023-09-10T00:39:32.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff359404700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3540ff520 0x7ff35419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:32.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff359404700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3540ffee0 0x7ff35419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:32.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff359404700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff354104530 0x7ff3541986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:32.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff359404700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff354198d10 con 0x7ff3540ff520 2023-09-10T00:39:32.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff359404700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff354198e80 con 0x7ff354104530 2023-09-10T00:39:32.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff359404700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff354199040 con 0x7ff3540ffee0 2023-09-10T00:39:32.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff352ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3540ff520 0x7ff35419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:32.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff352ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3540ff520 0x7ff35419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47160/0 (socket says 172.21.15.44:47160) 2023-09-10T00:39:32.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff352ffd700 1 -- 172.21.15.44:0/2944576541 learned_addr learned my addr 172.21.15.44:0/2944576541 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:32.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff3537fe700 1 --2- 172.21.15.44:0/2944576541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff354104530 0x7ff3541986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:32.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff3527fc700 1 --2- 172.21.15.44:0/2944576541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3540ffee0 0x7ff35419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:32.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff352ffd700 1 -- 172.21.15.44:0/2944576541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3540ffee0 msgr2=0x7ff35419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:32.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff352ffd700 1 --2- 172.21.15.44:0/2944576541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3540ffee0 0x7ff35419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:32.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff352ffd700 1 -- 172.21.15.44:0/2944576541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff354104530 msgr2=0x7ff3541986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:32.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff352ffd700 1 --2- 172.21.15.44:0/2944576541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff354104530 0x7ff3541986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:32.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff352ffd700 1 -- 172.21.15.44:0/2944576541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3480095f0 con 0x7ff3540ff520 2023-09-10T00:39:32.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff3527fc700 1 --2- 172.21.15.44:0/2944576541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3540ffee0 0x7ff35419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:32.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.268+0000 7ff352ffd700 1 --2- 172.21.15.44:0/2944576541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3540ff520 0x7ff35419e620 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff34400d980 tx=0x7ff34400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:32.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.269+0000 7ff33bfff700 1 -- 172.21.15.44:0/2944576541 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344004020 con 0x7ff3540ff520 2023-09-10T00:39:32.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.269+0000 7ff33bfff700 1 -- 172.21.15.44:0/2944576541 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff344009730 con 0x7ff3540ff520 2023-09-10T00:39:32.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.269+0000 7ff359404700 1 -- 172.21.15.44:0/2944576541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff354199320 con 0x7ff3540ff520 2023-09-10T00:39:32.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.269+0000 7ff33bfff700 1 -- 172.21.15.44:0/2944576541 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344003a00 con 0x7ff3540ff520 2023-09-10T00:39:32.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.269+0000 7ff359404700 1 -- 172.21.15.44:0/2944576541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3541a5610 con 0x7ff3540ff520 2023-09-10T00:39:32.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.270+0000 7ff359404700 1 -- 172.21.15.44:0/2944576541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff35404e300 con 0x7ff3540ff520 2023-09-10T00:39:32.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.271+0000 7ff33bfff700 1 -- 172.21.15.44:0/2944576541 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7ff3440098a0 con 0x7ff3540ff520 2023-09-10T00:39:32.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.271+0000 7ff33bfff700 1 --2- 172.21.15.44:0/2944576541 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff33c0a0750 0x7ff33c0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:32.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.271+0000 7ff3527fc700 1 --2- 172.21.15.44:0/2944576541 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff33c0a0750 0x7ff33c0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:32.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.271+0000 7ff33bfff700 1 -- 172.21.15.44:0/2944576541 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff3440cbc10 con 0x7ff3540ff520 2023-09-10T00:39:32.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.272+0000 7ff3527fc700 1 --2- 172.21.15.44:0/2944576541 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff33c0a0750 0x7ff33c0a2c10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff34800b5c0 tx=0x7ff348005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:32.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.276+0000 7ff33bfff700 1 -- 172.21.15.44:0/2944576541 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff3440952d0 con 0x7ff3540ff520 2023-09-10T00:39:32.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.484+0000 7ff359404700 1 -- 172.21.15.44:0/2944576541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7ff35410f570 con 0x7ff3540ff520 2023-09-10T00:39:32.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.484+0000 7ff33bfff700 1 -- 172.21.15.44:0/2944576541 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff3440988f0 con 0x7ff3540ff520 2023-09-10T00:39:32.485 INFO:teuthology.orchestra.run.smithi044.stdout:279172874270 2023-09-10T00:39:32.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.486+0000 7ff359404700 1 -- 172.21.15.44:0/2944576541 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff33c0a0750 msgr2=0x7ff33c0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:32.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.486+0000 7ff359404700 1 --2- 172.21.15.44:0/2944576541 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff33c0a0750 0x7ff33c0a2c10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff34800b5c0 tx=0x7ff348005fd0 comp rx=0 tx=0).stop 2023-09-10T00:39:32.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.486+0000 7ff359404700 1 -- 172.21.15.44:0/2944576541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3540ff520 msgr2=0x7ff35419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:32.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.486+0000 7ff359404700 1 --2- 172.21.15.44:0/2944576541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3540ff520 0x7ff35419e620 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff34400d980 tx=0x7ff34400dc90 comp rx=0 tx=0).stop 2023-09-10T00:39:32.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.486+0000 7ff359404700 1 -- 172.21.15.44:0/2944576541 shutdown_connections 2023-09-10T00:39:32.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.486+0000 7ff359404700 1 --2- 172.21.15.44:0/2944576541 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff33c0a0750 0x7ff33c0a2c10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:32.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.486+0000 7ff359404700 1 --2- 172.21.15.44:0/2944576541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff354104530 0x7ff3541986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:32.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.486+0000 7ff359404700 1 --2- 172.21.15.44:0/2944576541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3540ff520 0x7ff35419e620 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:32.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.486+0000 7ff359404700 1 --2- 172.21.15.44:0/2944576541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3540ffee0 0x7ff35419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:32.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.486+0000 7ff359404700 1 -- 172.21.15.44:0/2944576541 >> 172.21.15.44:0/2944576541 conn(0x7ff354076600 msgr2=0x7ff3540680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:32.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.486+0000 7ff359404700 1 -- 172.21.15.44:0/2944576541 shutdown_connections 2023-09-10T00:39:32.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:32.486+0000 7ff359404700 1 -- 172.21.15.44:0/2944576541 wait complete. 2023-09-10T00:39:32.782 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:32.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:32 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2944576541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-10T00:39:32.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:32 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2944576541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-10T00:39:32.858 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364835 got 214748364837 for osd.7 2023-09-10T00:39:32.858 DEBUG:teuthology.parallel:result is None 2023-09-10T00:39:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:32 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2944576541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-10T00:39:33.512 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:33.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:33 smithi044 ceph-mon[98676]: pgmap v297: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-10T00:39:33.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:33 smithi099 ceph-mon[102692]: pgmap v297: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-10T00:39:34.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:33 smithi131 ceph-mon[103713]: pgmap v297: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-10T00:39:34.093 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:34.668 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:34.707 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874269 got 279172874270 for osd.9 2023-09-10T00:39:34.708 DEBUG:teuthology.parallel:result is None 2023-09-10T00:39:35.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:35 smithi044 ceph-mon[98676]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-10T00:39:35.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:35 smithi099 ceph-mon[102692]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-10T00:39:36.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:35 smithi131 ceph-mon[103713]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-10T00:39:36.164 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.039+0000 7f747270a700 1 -- 172.21.15.44:0/714379181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c100ff0 msgr2=0x7f746c10d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:36.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.039+0000 7f747270a700 1 --2- 172.21.15.44:0/714379181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c100ff0 0x7f746c10d500 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f746400b0a0 tx=0x7f746400b3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.039+0000 7f747270a700 1 -- 172.21.15.44:0/714379181 shutdown_connections 2023-09-10T00:39:36.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.039+0000 7f747270a700 1 --2- 172.21.15.44:0/714379181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c100ff0 0x7f746c10d500 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.039+0000 7f747270a700 1 --2- 172.21.15.44:0/714379181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746c106630 0x7f746c106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.039+0000 7f747270a700 1 --2- 172.21.15.44:0/714379181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c100630 0x7f746c100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.039+0000 7f747270a700 1 -- 172.21.15.44:0/714379181 >> 172.21.15.44:0/714379181 conn(0x7f746c0fc120 msgr2=0x7f746c0fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:36.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.039+0000 7f747270a700 1 -- 172.21.15.44:0/714379181 shutdown_connections 2023-09-10T00:39:36.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.039+0000 7f747270a700 1 -- 172.21.15.44:0/714379181 wait complete. 2023-09-10T00:39:36.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.040+0000 7f747270a700 1 Processor -- start 2023-09-10T00:39:36.167 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.040+0000 7f747270a700 1 -- start start 2023-09-10T00:39:36.167 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f747270a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c100630 0x7f746c19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:36.167 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f747270a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c100ff0 0x7f746c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:36.167 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f747270a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746c106630 0x7f746c196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:36.167 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f747270a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c196be0 con 0x7f746c100ff0 2023-09-10T00:39:36.167 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f747270a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c196d50 con 0x7f746c106630 2023-09-10T00:39:36.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f747270a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c196f10 con 0x7f746c100630 2023-09-10T00:39:36.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f746b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c100ff0 0x7f746c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:36.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f746b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c100ff0 0x7f746c19ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47188/0 (socket says 172.21.15.44:47188) 2023-09-10T00:39:36.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f746b7fe700 1 -- 172.21.15.44:0/1444989649 learned_addr learned my addr 172.21.15.44:0/1444989649 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:36.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f7470ca7700 1 --2- 172.21.15.44:0/1444989649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746c106630 0x7f746c196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:36.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f746bfff700 1 --2- 172.21.15.44:0/1444989649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c100630 0x7f746c19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:36.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f746b7fe700 1 -- 172.21.15.44:0/1444989649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c100630 msgr2=0x7f746c19c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:36.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f746b7fe700 1 --2- 172.21.15.44:0/1444989649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c100630 0x7f746c19c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f746b7fe700 1 -- 172.21.15.44:0/1444989649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746c106630 msgr2=0x7f746c196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:36.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f746b7fe700 1 --2- 172.21.15.44:0/1444989649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746c106630 0x7f746c196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f746b7fe700 1 -- 172.21.15.44:0/1444989649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7464009d00 con 0x7f746c100ff0 2023-09-10T00:39:36.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f746b7fe700 1 --2- 172.21.15.44:0/1444989649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c100ff0 0x7f746c19ca30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f746000d950 tx=0x7f746000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:36.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.041+0000 7f74697fa700 1 -- 172.21.15.44:0/1444989649 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7460009940 con 0x7f746c100ff0 2023-09-10T00:39:36.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.042+0000 7f74697fa700 1 -- 172.21.15.44:0/1444989649 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7460010460 con 0x7f746c100ff0 2023-09-10T00:39:36.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.042+0000 7f747270a700 1 -- 172.21.15.44:0/1444989649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746c1971f0 con 0x7f746c100ff0 2023-09-10T00:39:36.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.042+0000 7f74697fa700 1 -- 172.21.15.44:0/1444989649 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746000f620 con 0x7f746c100ff0 2023-09-10T00:39:36.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.042+0000 7f747270a700 1 -- 172.21.15.44:0/1444989649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746c1a34e0 con 0x7f746c100ff0 2023-09-10T00:39:36.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.043+0000 7f747270a700 1 -- 172.21.15.44:0/1444989649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f746c04e300 con 0x7f746c100ff0 2023-09-10T00:39:36.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.044+0000 7f74697fa700 1 -- 172.21.15.44:0/1444989649 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f74600105d0 con 0x7f746c100ff0 2023-09-10T00:39:36.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.045+0000 7f74697fa700 1 --2- 172.21.15.44:0/1444989649 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f74540a06d0 0x7f74540a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:36.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.045+0000 7f746bfff700 1 --2- 172.21.15.44:0/1444989649 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f74540a06d0 0x7f74540a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:36.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.045+0000 7f74697fa700 1 -- 172.21.15.44:0/1444989649 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f74600cbe80 con 0x7f746c100ff0 2023-09-10T00:39:36.173 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.045+0000 7f746bfff700 1 --2- 172.21.15.44:0/1444989649 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f74540a06d0 0x7f74540a2b90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f745c009bd0 tx=0x7f745c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:36.173 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.049+0000 7f74697fa700 1 -- 172.21.15.44:0/1444989649 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7460095540 con 0x7f746c100ff0 2023-09-10T00:39:36.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.132+0000 7f6e1cafb700 1 -- 172.21.15.44:0/945505328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e181098c0 msgr2=0x7f6e18112010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:36.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.132+0000 7f6e1cafb700 1 --2- 172.21.15.44:0/945505328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e181098c0 0x7f6e18112010 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800b0a0 tx=0x7f6e0800b3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.132+0000 7f6e1cafb700 1 -- 172.21.15.44:0/945505328 shutdown_connections 2023-09-10T00:39:36.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.132+0000 7f6e1cafb700 1 --2- 172.21.15.44:0/945505328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e18112550 0x7f6e18114940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.132+0000 7f6e1cafb700 1 --2- 172.21.15.44:0/945505328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e18108fa0 0x7f6e18109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.132+0000 7f6e1cafb700 1 --2- 172.21.15.44:0/945505328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e181098c0 0x7f6e18112010 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.132+0000 7f6e1cafb700 1 -- 172.21.15.44:0/945505328 >> 172.21.15.44:0/945505328 conn(0x7f6e1806cb80 msgr2=0x7f6e1806cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:36.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.132+0000 7f6e1cafb700 1 -- 172.21.15.44:0/945505328 shutdown_connections 2023-09-10T00:39:36.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.132+0000 7f6e1cafb700 1 -- 172.21.15.44:0/945505328 wait complete. 2023-09-10T00:39:36.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.133+0000 7f6e1cafb700 1 Processor -- start 2023-09-10T00:39:36.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.133+0000 7f6e1cafb700 1 -- start start 2023-09-10T00:39:36.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.133+0000 7f6e1cafb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e18108fa0 0x7f6e181a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:36.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.133+0000 7f6e1cafb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e181098c0 0x7f6e181a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:36.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e1cafb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e18112550 0x7f6e181a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:36.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e1cafb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e181a6110 con 0x7f6e18108fa0 2023-09-10T00:39:36.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e1cafb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e181a6280 con 0x7f6e18112550 2023-09-10T00:39:36.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e1cafb700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e181a6440 con 0x7f6e181098c0 2023-09-10T00:39:36.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e1659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e18108fa0 0x7f6e181a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:36.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e1659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e18108fa0 0x7f6e181a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47200/0 (socket says 172.21.15.44:47200) 2023-09-10T00:39:36.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e1659c700 1 -- 172.21.15.44:0/2673521047 learned_addr learned my addr 172.21.15.44:0/2673521047 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:36.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e0ffff700 1 --2- 172.21.15.44:0/2673521047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e181098c0 0x7f6e181a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:36.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e16d9d700 1 --2- 172.21.15.44:0/2673521047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e18112550 0x7f6e181a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:36.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e1659c700 1 -- 172.21.15.44:0/2673521047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e181098c0 msgr2=0x7f6e181a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:36.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e1659c700 1 --2- 172.21.15.44:0/2673521047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e181098c0 0x7f6e181a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e1659c700 1 -- 172.21.15.44:0/2673521047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e18112550 msgr2=0x7f6e181a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e1659c700 1 --2- 172.21.15.44:0/2673521047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e18112550 0x7f6e181a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e1659c700 1 -- 172.21.15.44:0/2673521047 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e08009d00 con 0x7f6e18108fa0 2023-09-10T00:39:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e16d9d700 1 --2- 172.21.15.44:0/2673521047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e18112550 0x7f6e181a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e0ffff700 1 --2- 172.21.15.44:0/2673521047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e181098c0 0x7f6e181a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:39:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.134+0000 7f6e1659c700 1 --2- 172.21.15.44:0/2673521047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e18108fa0 0x7f6e181a1150 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000e9a0 tx=0x7f6e0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.135+0000 7f6e0f7fe700 1 -- 172.21.15.44:0/2673521047 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00009790 con 0x7f6e18108fa0 2023-09-10T00:39:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.135+0000 7f6e1cafb700 1 -- 172.21.15.44:0/2673521047 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e181a6720 con 0x7f6e18108fa0 2023-09-10T00:39:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.135+0000 7f6e0f7fe700 1 -- 172.21.15.44:0/2673521047 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6e0000cbd0 con 0x7f6e18108fa0 2023-09-10T00:39:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.135+0000 7f6e0f7fe700 1 -- 172.21.15.44:0/2673521047 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00018680 con 0x7f6e18108fa0 2023-09-10T00:39:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.135+0000 7f6e1cafb700 1 -- 172.21.15.44:0/2673521047 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e181adf80 con 0x7f6e18108fa0 2023-09-10T00:39:36.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.136+0000 7f6e1cafb700 1 -- 172.21.15.44:0/2673521047 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1804e300 con 0x7f6e18108fa0 2023-09-10T00:39:36.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.138+0000 7f6e0f7fe700 1 -- 172.21.15.44:0/2673521047 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f6e0000cd40 con 0x7f6e18108fa0 2023-09-10T00:39:36.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.138+0000 7f6e0f7fe700 1 --2- 172.21.15.44:0/2673521047 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6e040a06d0 0x7f6e040a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:36.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.138+0000 7f6e0f7fe700 1 -- 172.21.15.44:0/2673521047 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6e00014070 con 0x7f6e18108fa0 2023-09-10T00:39:36.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.138+0000 7f6e0ffff700 1 --2- 172.21.15.44:0/2673521047 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6e040a06d0 0x7f6e040a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:36.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.139+0000 7f6e0ffff700 1 --2- 172.21.15.44:0/2673521047 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6e040a06d0 0x7f6e040a2b90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6e181a2770 tx=0x7f6e0800b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:36.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.144+0000 7f6e0f7fe700 1 -- 172.21.15.44:0/2673521047 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6e00051e10 con 0x7f6e18108fa0 2023-09-10T00:39:36.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.807+0000 7f426102c700 1 -- 172.21.15.44:0/3703013135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c0ff4f0 msgr2=0x7f425c0ff970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:36.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.807+0000 7f426102c700 1 --2- 172.21.15.44:0/3703013135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c0ff4f0 0x7f425c0ff970 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4250009990 tx=0x7f4250009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.807+0000 7f426102c700 1 -- 172.21.15.44:0/3703013135 shutdown_connections 2023-09-10T00:39:36.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.807+0000 7f426102c700 1 --2- 172.21.15.44:0/3703013135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f425c0ffeb0 0x7f425c10c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.807+0000 7f426102c700 1 --2- 172.21.15.44:0/3703013135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425c104500 0x7f425c1048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.807+0000 7f426102c700 1 --2- 172.21.15.44:0/3703013135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c0ff4f0 0x7f425c0ff970 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.807+0000 7f426102c700 1 -- 172.21.15.44:0/3703013135 >> 172.21.15.44:0/3703013135 conn(0x7f425c076650 msgr2=0x7f425c076a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:36.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.807+0000 7f426102c700 1 -- 172.21.15.44:0/3703013135 shutdown_connections 2023-09-10T00:39:36.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.807+0000 7f426102c700 1 -- 172.21.15.44:0/3703013135 wait complete. 2023-09-10T00:39:36.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.808+0000 7f426102c700 1 Processor -- start 2023-09-10T00:39:36.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.808+0000 7f426102c700 1 -- start start 2023-09-10T00:39:36.242 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.808+0000 7f426102c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425c0ff4f0 0x7f425c1954a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:36.242 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.808+0000 7f426102c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f425c0ffeb0 0x7f425c1959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:36.242 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.808+0000 7f426102c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c104500 0x7f425c199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:36.242 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.808+0000 7f426102c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c19a460 con 0x7f425c104500 2023-09-10T00:39:36.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.808+0000 7f426102c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c19a5d0 con 0x7f425c0ff4f0 2023-09-10T00:39:36.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.808+0000 7f426102c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c19a790 con 0x7f425c0ffeb0 2023-09-10T00:39:36.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.808+0000 7f425b59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c104500 0x7f425c199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:36.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.808+0000 7f425b59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c104500 0x7f425c199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47176/0 (socket says 172.21.15.44:47176) 2023-09-10T00:39:36.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.808+0000 7f425b59e700 1 -- 172.21.15.44:0/1352990380 learned_addr learned my addr 172.21.15.44:0/1352990380 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:36.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.808+0000 7f425ad9d700 1 --2- 172.21.15.44:0/1352990380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425c0ff4f0 0x7f425c1954a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:36.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.808+0000 7f425a59c700 1 --2- 172.21.15.44:0/1352990380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f425c0ffeb0 0x7f425c1959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:36.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.809+0000 7f425b59e700 1 -- 172.21.15.44:0/1352990380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f425c0ffeb0 msgr2=0x7f425c1959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:36.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.809+0000 7f425b59e700 1 --2- 172.21.15.44:0/1352990380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f425c0ffeb0 0x7f425c1959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.809+0000 7f425b59e700 1 -- 172.21.15.44:0/1352990380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425c0ff4f0 msgr2=0x7f425c1954a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:36.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.809+0000 7f425b59e700 1 --2- 172.21.15.44:0/1352990380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425c0ff4f0 0x7f425c1954a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.809+0000 7f425b59e700 1 -- 172.21.15.44:0/1352990380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42500095f0 con 0x7f425c104500 2023-09-10T00:39:36.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.809+0000 7f425b59e700 1 --2- 172.21.15.44:0/1352990380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c104500 0x7f425c199e50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f425400cfb0 tx=0x7f425400e330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:36.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.809+0000 7f4243fff700 1 -- 172.21.15.44:0/1352990380 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254011070 con 0x7f425c104500 2023-09-10T00:39:36.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.809+0000 7f4243fff700 1 -- 172.21.15.44:0/1352990380 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4254007b30 con 0x7f425c104500 2023-09-10T00:39:36.246 INFO:teuthology.orchestra.run.smithi044.stdout:55834574914 2023-09-10T00:39:36.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.809+0000 7f4243fff700 1 -- 172.21.15.44:0/1352990380 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425400abd0 con 0x7f425c104500 2023-09-10T00:39:36.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.809+0000 7f426102c700 1 -- 172.21.15.44:0/1352990380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425c19aa70 con 0x7f425c104500 2023-09-10T00:39:36.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.809+0000 7f426102c700 1 -- 172.21.15.44:0/1352990380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425c068290 con 0x7f425c104500 2023-09-10T00:39:36.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.810+0000 7f426102c700 1 -- 172.21.15.44:0/1352990380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425c19ac90 con 0x7f425c104500 2023-09-10T00:39:36.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.811+0000 7f4243fff700 1 -- 172.21.15.44:0/1352990380 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f425400ad30 con 0x7f425c104500 2023-09-10T00:39:36.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.812+0000 7f4243fff700 1 --2- 172.21.15.44:0/1352990380 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42440a06d0 0x7f42440a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:36.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.812+0000 7f4243fff700 1 -- 172.21.15.44:0/1352990380 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f42540ce130 con 0x7f425c104500 2023-09-10T00:39:36.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.812+0000 7f425ad9d700 1 --2- 172.21.15.44:0/1352990380 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42440a06d0 0x7f42440a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:36.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.812+0000 7f425ad9d700 1 --2- 172.21.15.44:0/1352990380 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42440a06d0 0x7f42440a2b90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f425c069b20 tx=0x7f424c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:36.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:35.815+0000 7f4243fff700 1 -- 172.21.15.44:0/1352990380 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f42540977f0 con 0x7f425c104500 2023-09-10T00:39:36.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.025+0000 7f426102c700 1 -- 172.21.15.44:0/1352990380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f425c196800 con 0x7f425c104500 2023-09-10T00:39:36.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.026+0000 7f4243fff700 1 -- 172.21.15.44:0/1352990380 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f425409ae10 con 0x7f425c104500 2023-09-10T00:39:36.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.029+0000 7f426102c700 1 -- 172.21.15.44:0/1352990380 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42440a06d0 msgr2=0x7f42440a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:36.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.029+0000 7f426102c700 1 --2- 172.21.15.44:0/1352990380 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42440a06d0 0x7f42440a2b90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f425c069b20 tx=0x7f424c00b410 comp rx=0 tx=0).stop 2023-09-10T00:39:36.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.029+0000 7f426102c700 1 -- 172.21.15.44:0/1352990380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c104500 msgr2=0x7f425c199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:36.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.029+0000 7f426102c700 1 --2- 172.21.15.44:0/1352990380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c104500 0x7f425c199e50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f425400cfb0 tx=0x7f425400e330 comp rx=0 tx=0).stop 2023-09-10T00:39:36.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.029+0000 7f426102c700 1 -- 172.21.15.44:0/1352990380 shutdown_connections 2023-09-10T00:39:36.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.029+0000 7f426102c700 1 --2- 172.21.15.44:0/1352990380 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42440a06d0 0x7f42440a2b90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.029+0000 7f426102c700 1 --2- 172.21.15.44:0/1352990380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f425c104500 0x7f425c199e50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.029+0000 7f426102c700 1 --2- 172.21.15.44:0/1352990380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425c0ff4f0 0x7f425c1954a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.029+0000 7f426102c700 1 --2- 172.21.15.44:0/1352990380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f425c0ffeb0 0x7f425c1959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.029+0000 7f426102c700 1 -- 172.21.15.44:0/1352990380 >> 172.21.15.44:0/1352990380 conn(0x7f425c076650 msgr2=0x7f425c101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:36.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.029+0000 7f426102c700 1 -- 172.21.15.44:0/1352990380 shutdown_connections 2023-09-10T00:39:36.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.029+0000 7f426102c700 1 -- 172.21.15.44:0/1352990380 wait complete. 2023-09-10T00:39:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.297+0000 7f747270a700 1 -- 172.21.15.44:0/1444989649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f746c1104d0 con 0x7f746c100ff0 2023-09-10T00:39:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.298+0000 7f74697fa700 1 -- 172.21.15.44:0/1444989649 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7460016090 con 0x7f746c100ff0 2023-09-10T00:39:36.300 INFO:teuthology.orchestra.run.smithi044.stdout:141733920821 2023-09-10T00:39:36.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.300+0000 7f747270a700 1 -- 172.21.15.44:0/1444989649 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f74540a06d0 msgr2=0x7f74540a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:36.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.300+0000 7f747270a700 1 --2- 172.21.15.44:0/1444989649 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f74540a06d0 0x7f74540a2b90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f745c009bd0 tx=0x7f745c009260 comp rx=0 tx=0).stop 2023-09-10T00:39:36.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.300+0000 7f747270a700 1 -- 172.21.15.44:0/1444989649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c100ff0 msgr2=0x7f746c19ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:36.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.300+0000 7f747270a700 1 --2- 172.21.15.44:0/1444989649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c100ff0 0x7f746c19ca30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f746000d950 tx=0x7f746000dc60 comp rx=0 tx=0).stop 2023-09-10T00:39:36.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.300+0000 7f747270a700 1 -- 172.21.15.44:0/1444989649 shutdown_connections 2023-09-10T00:39:36.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.300+0000 7f747270a700 1 --2- 172.21.15.44:0/1444989649 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f74540a06d0 0x7f74540a2b90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.300+0000 7f747270a700 1 --2- 172.21.15.44:0/1444989649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746c106630 0x7f746c196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.300+0000 7f747270a700 1 --2- 172.21.15.44:0/1444989649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c100630 0x7f746c19c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.300+0000 7f747270a700 1 --2- 172.21.15.44:0/1444989649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c100ff0 0x7f746c19ca30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.300+0000 7f747270a700 1 -- 172.21.15.44:0/1444989649 >> 172.21.15.44:0/1444989649 conn(0x7f746c0fc120 msgr2=0x7f746c0fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.300+0000 7f747270a700 1 -- 172.21.15.44:0/1444989649 shutdown_connections 2023-09-10T00:39:36.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.300+0000 7f747270a700 1 -- 172.21.15.44:0/1444989649 wait complete. 2023-09-10T00:39:36.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.363+0000 7f6e1cafb700 1 -- 172.21.15.44:0/2673521047 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f6e181a24b0 con 0x7f6e18108fa0 2023-09-10T00:39:36.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.364+0000 7f6e0f7fe700 1 -- 172.21.15.44:0/2673521047 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6e00095980 con 0x7f6e18108fa0 2023-09-10T00:39:36.365 INFO:teuthology.orchestra.run.smithi044.stdout:244813135908 2023-09-10T00:39:36.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.367+0000 7f6e1cafb700 1 -- 172.21.15.44:0/2673521047 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6e040a06d0 msgr2=0x7f6e040a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:36.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.367+0000 7f6e1cafb700 1 --2- 172.21.15.44:0/2673521047 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6e040a06d0 0x7f6e040a2b90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6e181a2770 tx=0x7f6e0800b700 comp rx=0 tx=0).stop 2023-09-10T00:39:36.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.367+0000 7f6e1cafb700 1 -- 172.21.15.44:0/2673521047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e18108fa0 msgr2=0x7f6e181a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:36.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.367+0000 7f6e1cafb700 1 --2- 172.21.15.44:0/2673521047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e18108fa0 0x7f6e181a1150 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000e9a0 tx=0x7f6e0000ecb0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.367+0000 7f6e1cafb700 1 -- 172.21.15.44:0/2673521047 shutdown_connections 2023-09-10T00:39:36.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.367+0000 7f6e1cafb700 1 --2- 172.21.15.44:0/2673521047 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f6e040a06d0 0x7f6e040a2b90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.367+0000 7f6e1cafb700 1 --2- 172.21.15.44:0/2673521047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e18112550 0x7f6e181a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.367+0000 7f6e1cafb700 1 --2- 172.21.15.44:0/2673521047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e18108fa0 0x7f6e181a1150 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.367+0000 7f6e1cafb700 1 --2- 172.21.15.44:0/2673521047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e181098c0 0x7f6e181a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:36.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.367+0000 7f6e1cafb700 1 -- 172.21.15.44:0/2673521047 >> 172.21.15.44:0/2673521047 conn(0x7f6e1806cb80 msgr2=0x7f6e1806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:36.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.367+0000 7f6e1cafb700 1 -- 172.21.15.44:0/2673521047 shutdown_connections 2023-09-10T00:39:36.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:36.367+0000 7f6e1cafb700 1 -- 172.21.15.44:0/2673521047 wait complete. 2023-09-10T00:39:36.594 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:36 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1352990380' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-10T00:39:36.594 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:36 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1444989649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-10T00:39:36.595 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:36 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2673521047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-10T00:39:36.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:36 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1352990380' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-10T00:39:36.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:36 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1444989649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-10T00:39:36.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:36 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2673521047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-10T00:39:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:36 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1352990380' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-10T00:39:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:36 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1444989649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-10T00:39:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:36 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2673521047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-10T00:39:37.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.071+0000 7f9a25471700 1 -- 172.21.15.44:0/2090385812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a201055e0 msgr2=0x7f9a201059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:37.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.071+0000 7f9a25471700 1 --2- 172.21.15.44:0/2090385812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a201055e0 0x7f9a201059c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9a100099c0 tx=0x7f9a10009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.071+0000 7f9a25471700 1 -- 172.21.15.44:0/2090385812 shutdown_connections 2023-09-10T00:39:37.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.071+0000 7f9a25471700 1 --2- 172.21.15.44:0/2090385812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a200ffef0 0x7f9a2010c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.071+0000 7f9a25471700 1 --2- 172.21.15.44:0/2090385812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a201055e0 0x7f9a201059c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.071+0000 7f9a25471700 1 --2- 172.21.15.44:0/2090385812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a200ff530 0x7f9a200ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.071+0000 7f9a25471700 1 -- 172.21.15.44:0/2090385812 >> 172.21.15.44:0/2090385812 conn(0x7f9a200fb000 msgr2=0x7f9a200fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.071+0000 7f9a25471700 1 -- 172.21.15.44:0/2090385812 shutdown_connections 2023-09-10T00:39:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.071+0000 7f9a25471700 1 -- 172.21.15.44:0/2090385812 wait complete. 2023-09-10T00:39:37.076 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.072+0000 7f9a25471700 1 Processor -- start 2023-09-10T00:39:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.072+0000 7f9a25471700 1 -- start start 2023-09-10T00:39:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.072+0000 7f9a25471700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a200ff530 0x7f9a201011a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.072+0000 7f9a25471700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a200ffef0 0x7f9a201016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.072+0000 7f9a25471700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a201055e0 0x7f9a20101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.072+0000 7f9a25471700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a201a49b0 con 0x7f9a200ffef0 2023-09-10T00:39:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.072+0000 7f9a25471700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a201a4b20 con 0x7f9a201055e0 2023-09-10T00:39:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.072+0000 7f9a25471700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a201a4ce0 con 0x7f9a200ff530 2023-09-10T00:39:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.072+0000 7f9a1e7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a200ffef0 0x7f9a201016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.073+0000 7f9a1e7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a200ffef0 0x7f9a201016e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47214/0 (socket says 172.21.15.44:47214) 2023-09-10T00:39:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.073+0000 7f9a1e7fc700 1 -- 172.21.15.44:0/3027332711 learned_addr learned my addr 172.21.15.44:0/3027332711 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.073+0000 7f9a1effd700 1 --2- 172.21.15.44:0/3027332711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a200ff530 0x7f9a201011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.073+0000 7f9a1f7fe700 1 --2- 172.21.15.44:0/3027332711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a201055e0 0x7f9a20101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.073+0000 7f9a1e7fc700 1 -- 172.21.15.44:0/3027332711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a200ff530 msgr2=0x7f9a201011a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:37.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.073+0000 7f9a1e7fc700 1 --2- 172.21.15.44:0/3027332711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a200ff530 0x7f9a201011a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.073+0000 7f9a1e7fc700 1 -- 172.21.15.44:0/3027332711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a201055e0 msgr2=0x7f9a20101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:37.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.073+0000 7f9a1e7fc700 1 --2- 172.21.15.44:0/3027332711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a201055e0 0x7f9a20101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.073+0000 7f9a1e7fc700 1 -- 172.21.15.44:0/3027332711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a100095f0 con 0x7f9a200ffef0 2023-09-10T00:39:37.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.073+0000 7f9a1e7fc700 1 --2- 172.21.15.44:0/3027332711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a200ffef0 0x7f9a201016e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400d950 tx=0x7f9a1400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.073+0000 7f9a07fff700 1 -- 172.21.15.44:0/3027332711 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14009940 con 0x7f9a200ffef0 2023-09-10T00:39:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.073+0000 7f9a07fff700 1 -- 172.21.15.44:0/3027332711 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9a14010460 con 0x7f9a200ffef0 2023-09-10T00:39:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.073+0000 7f9a25471700 1 -- 172.21.15.44:0/3027332711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a201a4fc0 con 0x7f9a200ffef0 2023-09-10T00:39:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.073+0000 7f9a07fff700 1 -- 172.21.15.44:0/3027332711 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1400f620 con 0x7f9a200ffef0 2023-09-10T00:39:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.073+0000 7f9a25471700 1 -- 172.21.15.44:0/3027332711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a201a5430 con 0x7f9a200ffef0 2023-09-10T00:39:37.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.074+0000 7f9a25471700 1 -- 172.21.15.44:0/3027332711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a2004e300 con 0x7f9a200ffef0 2023-09-10T00:39:37.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.075+0000 7f9a07fff700 1 -- 172.21.15.44:0/3027332711 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f9a140105d0 con 0x7f9a200ffef0 2023-09-10T00:39:37.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.076+0000 7f9a07fff700 1 --2- 172.21.15.44:0/3027332711 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a080a06d0 0x7f9a080a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:37.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.076+0000 7f9a07fff700 1 -- 172.21.15.44:0/3027332711 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9a140cbe80 con 0x7f9a200ffef0 2023-09-10T00:39:37.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.076+0000 7f9a1effd700 1 --2- 172.21.15.44:0/3027332711 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a080a06d0 0x7f9a080a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:37.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.076+0000 7f9a1effd700 1 --2- 172.21.15.44:0/3027332711 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a080a06d0 0x7f9a080a2b90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000b580 tx=0x7f9a10009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:37.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.080+0000 7f9a07fff700 1 -- 172.21.15.44:0/3027332711 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9a14095540 con 0x7f9a200ffef0 2023-09-10T00:39:37.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.286+0000 7f9a25471700 1 -- 172.21.15.44:0/3027332711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9a201026d0 con 0x7f9a200ffef0 2023-09-10T00:39:37.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.287+0000 7f9a07fff700 1 -- 172.21.15.44:0/3027332711 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9a14016090 con 0x7f9a200ffef0 2023-09-10T00:39:37.288 INFO:teuthology.orchestra.run.smithi044.stdout:193273528364 2023-09-10T00:39:37.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.289+0000 7f9a25471700 1 -- 172.21.15.44:0/3027332711 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a080a06d0 msgr2=0x7f9a080a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:37.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.289+0000 7f9a25471700 1 --2- 172.21.15.44:0/3027332711 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a080a06d0 0x7f9a080a2b90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000b580 tx=0x7f9a10009f10 comp rx=0 tx=0).stop 2023-09-10T00:39:37.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.289+0000 7f9a25471700 1 -- 172.21.15.44:0/3027332711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a200ffef0 msgr2=0x7f9a201016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:37.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.289+0000 7f9a25471700 1 --2- 172.21.15.44:0/3027332711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a200ffef0 0x7f9a201016e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400d950 tx=0x7f9a1400dc60 comp rx=0 tx=0).stop 2023-09-10T00:39:37.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.289+0000 7f9a25471700 1 -- 172.21.15.44:0/3027332711 shutdown_connections 2023-09-10T00:39:37.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.289+0000 7f9a25471700 1 --2- 172.21.15.44:0/3027332711 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a080a06d0 0x7f9a080a2b90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.289+0000 7f9a25471700 1 --2- 172.21.15.44:0/3027332711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a201055e0 0x7f9a20101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.289+0000 7f9a25471700 1 --2- 172.21.15.44:0/3027332711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a200ff530 0x7f9a201011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.289+0000 7f9a25471700 1 --2- 172.21.15.44:0/3027332711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a200ffef0 0x7f9a201016e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.289+0000 7f9a25471700 1 -- 172.21.15.44:0/3027332711 >> 172.21.15.44:0/3027332711 conn(0x7f9a200fb000 msgr2=0x7f9a20068ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:37.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.289+0000 7f9a25471700 1 -- 172.21.15.44:0/3027332711 shutdown_connections 2023-09-10T00:39:37.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.289+0000 7f9a25471700 1 -- 172.21.15.44:0/3027332711 wait complete. 2023-09-10T00:39:37.552 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920819 got 141733920821 for osd.4 2023-09-10T00:39:37.552 DEBUG:teuthology.parallel:result is None 2023-09-10T00:39:37.576 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:37 smithi044 ceph-mon[98676]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-10T00:39:37.577 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:37 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3027332711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-10T00:39:37.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:37 smithi099 ceph-mon[102692]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-10T00:39:37.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:37 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3027332711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-10T00:39:37.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.850+0000 7fb5fcba1700 1 -- 172.21.15.44:0/3591803420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f8106620 msgr2=0x7fb5f8106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:37.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.850+0000 7fb5fcba1700 1 --2- 172.21.15.44:0/3591803420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f8106620 0x7fb5f8106a00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009990 tx=0x7fb5e8009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.850+0000 7fb5fcba1700 1 -- 172.21.15.44:0/3591803420 shutdown_connections 2023-09-10T00:39:37.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.850+0000 7fb5fcba1700 1 --2- 172.21.15.44:0/3591803420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f8100fe0 0x7fb5f810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.850+0000 7fb5fcba1700 1 --2- 172.21.15.44:0/3591803420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f8106620 0x7fb5f8106a00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.850+0000 7fb5fcba1700 1 --2- 172.21.15.44:0/3591803420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5f8100620 0x7fb5f8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.850+0000 7fb5fcba1700 1 -- 172.21.15.44:0/3591803420 >> 172.21.15.44:0/3591803420 conn(0x7fb5f80fc0f0 msgr2=0x7fb5f80fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:37.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.850+0000 7fb5fcba1700 1 -- 172.21.15.44:0/3591803420 shutdown_connections 2023-09-10T00:39:37.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.851+0000 7fb5fcba1700 1 -- 172.21.15.44:0/3591803420 wait complete. 2023-09-10T00:39:37.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.851+0000 7fb5fcba1700 1 Processor -- start 2023-09-10T00:39:37.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.851+0000 7fb5fcba1700 1 -- start start 2023-09-10T00:39:37.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.851+0000 7fb5fcba1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f8100620 0x7fb5f81966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.851+0000 7fb5fcba1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5f8100fe0 0x7fb5f8196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.851+0000 7fb5fcba1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f8106620 0x7fb5f819afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.851+0000 7fb5fcba1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f819b5d0 con 0x7fb5f8106620 2023-09-10T00:39:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5fcba1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f819b740 con 0x7fb5f8100620 2023-09-10T00:39:37.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5fcba1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f819b900 con 0x7fb5f8100fe0 2023-09-10T00:39:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5f6d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f8106620 0x7fb5f819afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5f659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f8100620 0x7fb5f81966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5f6d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f8106620 0x7fb5f819afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47244/0 (socket says 172.21.15.44:47244) 2023-09-10T00:39:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5f6d9d700 1 -- 172.21.15.44:0/2164179281 learned_addr learned my addr 172.21.15.44:0/2164179281 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:37.910 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5f5d9b700 1 --2- 172.21.15.44:0/2164179281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5f8100fe0 0x7fb5f8196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5f6d9d700 1 -- 172.21.15.44:0/2164179281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5f8100fe0 msgr2=0x7fb5f8196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5f6d9d700 1 --2- 172.21.15.44:0/2164179281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5f8100fe0 0x7fb5f8196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5f6d9d700 1 -- 172.21.15.44:0/2164179281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f8100620 msgr2=0x7fb5f81966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5f6d9d700 1 --2- 172.21.15.44:0/2164179281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f8100620 0x7fb5f81966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5f6d9d700 1 -- 172.21.15.44:0/2164179281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e80095f0 con 0x7fb5f8106620 2023-09-10T00:39:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5f659c700 1 --2- 172.21.15.44:0/2164179281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f8100620 0x7fb5f81966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:39:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5f6d9d700 1 --2- 172.21.15.44:0/2164179281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f8106620 0x7fb5f819afc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000ef30 tx=0x7fb5f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5e77fe700 1 -- 172.21.15.44:0/2164179281 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0018070 con 0x7fb5f8106620 2023-09-10T00:39:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5e77fe700 1 -- 172.21.15.44:0/2164179281 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5f000f040 con 0x7fb5f8106620 2023-09-10T00:39:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5fcba1700 1 -- 172.21.15.44:0/2164179281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f819bb80 con 0x7fb5f8106620 2023-09-10T00:39:37.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.852+0000 7fb5e77fe700 1 -- 172.21.15.44:0/2164179281 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000cbc0 con 0x7fb5f8106620 2023-09-10T00:39:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.853+0000 7fb5fcba1700 1 -- 172.21.15.44:0/2164179281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f81a3440 con 0x7fb5f8106620 2023-09-10T00:39:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.853+0000 7fb5fcba1700 1 -- 172.21.15.44:0/2164179281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f804e300 con 0x7fb5f8106620 2023-09-10T00:39:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.855+0000 7fb5e77fe700 1 -- 172.21.15.44:0/2164179281 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fb5f000cd80 con 0x7fb5f8106620 2023-09-10T00:39:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.856+0000 7fb5e77fe700 1 --2- 172.21.15.44:0/2164179281 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb5e00a06d0 0x7fb5e00a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:37.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.856+0000 7fb5f659c700 1 --2- 172.21.15.44:0/2164179281 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb5e00a06d0 0x7fb5e00a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:37.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.856+0000 7fb5e77fe700 1 -- 172.21.15.44:0/2164179281 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb5f00cc0e0 con 0x7fb5f8106620 2023-09-10T00:39:37.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.857+0000 7fb5f659c700 1 --2- 172.21.15.44:0/2164179281 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb5e00a06d0 0x7fb5e00a2b90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80059d0 tx=0x7fb5e8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:37.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.859+0000 7fb5e77fe700 1 -- 172.21.15.44:0/2164179281 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb5f0095830 con 0x7fb5f8106620 2023-09-10T00:39:37.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.773+0000 7f67dd2cc700 1 -- 172.21.15.44:0/1171491549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67d80ff880 msgr2=0x7f67d80ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:37.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.773+0000 7f67dd2cc700 1 --2- 172.21.15.44:0/1171491549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67d80ff880 0x7f67d80ffc60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009990 tx=0x7f67c8009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.773+0000 7f67dd2cc700 1 -- 172.21.15.44:0/1171491549 shutdown_connections 2023-09-10T00:39:37.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.773+0000 7f67dd2cc700 1 --2- 172.21.15.44:0/1171491549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67d8101840 0x7f67d810c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.773+0000 7f67dd2cc700 1 --2- 172.21.15.44:0/1171491549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67d80ff880 0x7f67d80ffc60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.773+0000 7f67dd2cc700 1 --2- 172.21.15.44:0/1171491549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67d81001a0 0x7f67d8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.773+0000 7f67dd2cc700 1 -- 172.21.15.44:0/1171491549 >> 172.21.15.44:0/1171491549 conn(0x7f67d80765f0 msgr2=0x7f67d8076a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:37.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.773+0000 7f67dd2cc700 1 -- 172.21.15.44:0/1171491549 shutdown_connections 2023-09-10T00:39:37.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.773+0000 7f67dd2cc700 1 -- 172.21.15.44:0/1171491549 wait complete. 2023-09-10T00:39:37.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.774+0000 7f67dd2cc700 1 Processor -- start 2023-09-10T00:39:37.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.774+0000 7f67dd2cc700 1 -- start start 2023-09-10T00:39:37.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.774+0000 7f67dd2cc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67d80ff880 0x7f67d8197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:37.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.774+0000 7f67dd2cc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67d81001a0 0x7f67d8197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:37.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.774+0000 7f67dd2cc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67d8101840 0x7f67d81a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:37.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.774+0000 7f67dd2cc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d81983c0 con 0x7f67d81001a0 2023-09-10T00:39:37.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.774+0000 7f67dd2cc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d81a2b00 con 0x7f67d80ff880 2023-09-10T00:39:37.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.774+0000 7f67dd2cc700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d81a2cc0 con 0x7f67d8101840 2023-09-10T00:39:37.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.774+0000 7f67d67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67d81001a0 0x7f67d8197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:37.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.774+0000 7f67d67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67d81001a0 0x7f67d8197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47228/0 (socket says 172.21.15.44:47228) 2023-09-10T00:39:37.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.774+0000 7f67d67fc700 1 -- 172.21.15.44:0/1663978434 learned_addr learned my addr 172.21.15.44:0/1663978434 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:37.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.775+0000 7f67d6ffd700 1 --2- 172.21.15.44:0/1663978434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67d80ff880 0x7f67d8197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:37.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.775+0000 7f67d77fe700 1 --2- 172.21.15.44:0/1663978434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67d8101840 0x7f67d81a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:37.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.775+0000 7f67d67fc700 1 -- 172.21.15.44:0/1663978434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67d8101840 msgr2=0x7f67d81a25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:37.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.775+0000 7f67d67fc700 1 --2- 172.21.15.44:0/1663978434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67d8101840 0x7f67d81a25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.775+0000 7f67d67fc700 1 -- 172.21.15.44:0/1663978434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67d80ff880 msgr2=0x7f67d8197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:37.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.775+0000 7f67d67fc700 1 --2- 172.21.15.44:0/1663978434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67d80ff880 0x7f67d8197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.775+0000 7f67d67fc700 1 -- 172.21.15.44:0/1663978434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c80095f0 con 0x7f67d81001a0 2023-09-10T00:39:37.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.775+0000 7f67d67fc700 1 --2- 172.21.15.44:0/1663978434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67d81001a0 0x7f67d8197a80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f67c000e970 tx=0x7f67c000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:37.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.775+0000 7f67cffff700 1 -- 172.21.15.44:0/1663978434 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0009790 con 0x7f67d81001a0 2023-09-10T00:39:37.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.775+0000 7f67dd2cc700 1 -- 172.21.15.44:0/1663978434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d81a2fa0 con 0x7f67d81001a0 2023-09-10T00:39:37.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.775+0000 7f67cffff700 1 -- 172.21.15.44:0/1663978434 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67c000c7e0 con 0x7f67d81001a0 2023-09-10T00:39:37.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.775+0000 7f67cffff700 1 -- 172.21.15.44:0/1663978434 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0005440 con 0x7f67d81001a0 2023-09-10T00:39:37.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.775+0000 7f67dd2cc700 1 -- 172.21.15.44:0/1663978434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d81a3410 con 0x7f67d81001a0 2023-09-10T00:39:37.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.776+0000 7f67dd2cc700 1 -- 172.21.15.44:0/1663978434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d804e300 con 0x7f67d81001a0 2023-09-10T00:39:37.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.777+0000 7f67cffff700 1 -- 172.21.15.44:0/1663978434 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f67c000c950 con 0x7f67d81001a0 2023-09-10T00:39:37.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.778+0000 7f67cffff700 1 --2- 172.21.15.44:0/1663978434 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f67c40a06d0 0x7f67c40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:37.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.778+0000 7f67d6ffd700 1 --2- 172.21.15.44:0/1663978434 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f67c40a06d0 0x7f67c40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:37.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.778+0000 7f67cffff700 1 -- 172.21.15.44:0/1663978434 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f67c0014070 con 0x7f67d81001a0 2023-09-10T00:39:37.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.779+0000 7f67d6ffd700 1 --2- 172.21.15.44:0/1663978434 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f67c40a06d0 0x7f67c40a2b90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f67c800b2e0 tx=0x7f67c8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:37.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.782+0000 7f67cffff700 1 -- 172.21.15.44:0/1663978434 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f67c0095a70 con 0x7f67d81001a0 2023-09-10T00:39:37.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.991+0000 7f67dd2cc700 1 -- 172.21.15.44:0/1663978434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f67d8068f60 con 0x7f67d81001a0 2023-09-10T00:39:37.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.991+0000 7f67cffff700 1 -- 172.21.15.44:0/1663978434 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67c0099090 con 0x7f67d81001a0 2023-09-10T00:39:37.993 INFO:teuthology.orchestra.run.smithi044.stdout:107374182456 2023-09-10T00:39:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.993+0000 7f67dd2cc700 1 -- 172.21.15.44:0/1663978434 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f67c40a06d0 msgr2=0x7f67c40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.993+0000 7f67dd2cc700 1 --2- 172.21.15.44:0/1663978434 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f67c40a06d0 0x7f67c40a2b90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f67c800b2e0 tx=0x7f67c8005fb0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.993+0000 7f67dd2cc700 1 -- 172.21.15.44:0/1663978434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67d81001a0 msgr2=0x7f67d8197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.993+0000 7f67dd2cc700 1 --2- 172.21.15.44:0/1663978434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67d81001a0 0x7f67d8197a80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f67c000e970 tx=0x7f67c000ec80 comp rx=0 tx=0).stop 2023-09-10T00:39:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.993+0000 7f67dd2cc700 1 -- 172.21.15.44:0/1663978434 shutdown_connections 2023-09-10T00:39:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.993+0000 7f67dd2cc700 1 --2- 172.21.15.44:0/1663978434 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f67c40a06d0 0x7f67c40a2b90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.993+0000 7f67dd2cc700 1 --2- 172.21.15.44:0/1663978434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67d8101840 0x7f67d81a25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.993+0000 7f67dd2cc700 1 --2- 172.21.15.44:0/1663978434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67d80ff880 0x7f67d8197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.993+0000 7f67dd2cc700 1 --2- 172.21.15.44:0/1663978434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67d81001a0 0x7f67d8197a80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.994+0000 7f67dd2cc700 1 -- 172.21.15.44:0/1663978434 >> 172.21.15.44:0/1663978434 conn(0x7f67d80765f0 msgr2=0x7f67d80fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.994+0000 7f67dd2cc700 1 -- 172.21.15.44:0/1663978434 shutdown_connections 2023-09-10T00:39:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:37.994+0000 7f67dd2cc700 1 -- 172.21.15.44:0/1663978434 wait complete. 2023-09-10T00:39:38.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:37 smithi131 ceph-mon[103713]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-10T00:39:38.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:37 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3027332711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-10T00:39:38.078 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.077+0000 7fb5fcba1700 1 -- 172.21.15.44:0/2164179281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb5f8197970 con 0x7fb5f8106620 2023-09-10T00:39:38.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.077+0000 7fb5e77fe700 1 -- 172.21.15.44:0/2164179281 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb5f0098e50 con 0x7fb5f8106620 2023-09-10T00:39:38.079 INFO:teuthology.orchestra.run.smithi044.stdout:163208757297 2023-09-10T00:39:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.079+0000 7fb5fcba1700 1 -- 172.21.15.44:0/2164179281 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb5e00a06d0 msgr2=0x7fb5e00a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.079+0000 7fb5fcba1700 1 --2- 172.21.15.44:0/2164179281 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb5e00a06d0 0x7fb5e00a2b90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80059d0 tx=0x7fb5e8005960 comp rx=0 tx=0).stop 2023-09-10T00:39:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.080+0000 7fb5fcba1700 1 -- 172.21.15.44:0/2164179281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f8106620 msgr2=0x7fb5f819afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.080+0000 7fb5fcba1700 1 --2- 172.21.15.44:0/2164179281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f8106620 0x7fb5f819afc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000ef30 tx=0x7fb5f000c3a0 comp rx=0 tx=0).stop 2023-09-10T00:39:38.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.080+0000 7fb5fcba1700 1 -- 172.21.15.44:0/2164179281 shutdown_connections 2023-09-10T00:39:38.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.080+0000 7fb5fcba1700 1 --2- 172.21.15.44:0/2164179281 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb5e00a06d0 0x7fb5e00a2b90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:38.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.080+0000 7fb5fcba1700 1 --2- 172.21.15.44:0/2164179281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f8106620 0x7fb5f819afc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:38.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.080+0000 7fb5fcba1700 1 --2- 172.21.15.44:0/2164179281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f8100620 0x7fb5f81966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:38.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.080+0000 7fb5fcba1700 1 --2- 172.21.15.44:0/2164179281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5f8100fe0 0x7fb5f8196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:38.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.080+0000 7fb5fcba1700 1 -- 172.21.15.44:0/2164179281 >> 172.21.15.44:0/2164179281 conn(0x7fb5f80fc0f0 msgr2=0x7fb5f80fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:38.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.080+0000 7fb5fcba1700 1 -- 172.21.15.44:0/2164179281 shutdown_connections 2023-09-10T00:39:38.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.080+0000 7fb5fcba1700 1 -- 172.21.15.44:0/2164179281 wait complete. 2023-09-10T00:39:38.350 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528362 got 193273528364 for osd.6 2023-09-10T00:39:38.351 DEBUG:teuthology.parallel:result is None 2023-09-10T00:39:38.677 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:38 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1663978434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-10T00:39:38.677 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:38 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2164179281' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-10T00:39:38.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.786+0000 7f0bd9169700 1 -- 172.21.15.44:0/1360011505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd40ffee0 msgr2=0x7f0bd410c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:38.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.786+0000 7f0bd9169700 1 --2- 172.21.15.44:0/1360011505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd40ffee0 0x7f0bd410c3f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00b0a0 tx=0x7f0bcc00b3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:38.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.786+0000 7f0bd9169700 1 -- 172.21.15.44:0/1360011505 shutdown_connections 2023-09-10T00:39:38.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.786+0000 7f0bd9169700 1 --2- 172.21.15.44:0/1360011505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd40ffee0 0x7f0bd410c3f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:38.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.786+0000 7f0bd9169700 1 --2- 172.21.15.44:0/1360011505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bd4104530 0x7f0bd4104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:38.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.786+0000 7f0bd9169700 1 --2- 172.21.15.44:0/1360011505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bd40ff520 0x7f0bd40ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:38.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.786+0000 7f0bd9169700 1 -- 172.21.15.44:0/1360011505 >> 172.21.15.44:0/1360011505 conn(0x7f0bd4076600 msgr2=0x7f0bd4076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:38.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.786+0000 7f0bd9169700 1 -- 172.21.15.44:0/1360011505 shutdown_connections 2023-09-10T00:39:38.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.787+0000 7f0bd9169700 1 -- 172.21.15.44:0/1360011505 wait complete. 2023-09-10T00:39:38.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.788+0000 7f0bd9169700 1 Processor -- start 2023-09-10T00:39:38.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.788+0000 7f0bd9169700 1 -- start start 2023-09-10T00:39:38.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.788+0000 7f0bd9169700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd40ff520 0x7f0bd4101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:38.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.788+0000 7f0bd9169700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bd40ffee0 0x7f0bd41016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:38.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.788+0000 7f0bd9169700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bd4104530 0x7f0bd4101c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:38.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.788+0000 7f0bd9169700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd41a4b50 con 0x7f0bd40ff520 2023-09-10T00:39:38.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.788+0000 7f0bd9169700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd41a4cc0 con 0x7f0bd4104530 2023-09-10T00:39:38.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.788+0000 7f0bd9169700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd41a4e80 con 0x7f0bd40ffee0 2023-09-10T00:39:38.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.788+0000 7f0bd2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd40ff520 0x7f0bd4101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:38.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.788+0000 7f0bd359e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bd4104530 0x7f0bd4101c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:38.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.788+0000 7f0bd359e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bd4104530 0x7f0bd4101c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:48998/0 (socket says 172.21.15.44:48998) 2023-09-10T00:39:38.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.788+0000 7f0bd2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd40ff520 0x7f0bd4101180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47254/0 (socket says 172.21.15.44:47254) 2023-09-10T00:39:38.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.788+0000 7f0bd359e700 1 -- 172.21.15.44:0/194126544 learned_addr learned my addr 172.21.15.44:0/194126544 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:38.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.788+0000 7f0bd259c700 1 --2- 172.21.15.44:0/194126544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bd40ffee0 0x7f0bd41016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:38.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.789+0000 7f0bd359e700 1 -- 172.21.15.44:0/194126544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bd40ffee0 msgr2=0x7f0bd41016c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:38.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.789+0000 7f0bd359e700 1 --2- 172.21.15.44:0/194126544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bd40ffee0 0x7f0bd41016c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:38.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.789+0000 7f0bd359e700 1 -- 172.21.15.44:0/194126544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd40ff520 msgr2=0x7f0bd4101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:38.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.789+0000 7f0bd359e700 1 --2- 172.21.15.44:0/194126544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd40ff520 0x7f0bd4101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:38.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.789+0000 7f0bd359e700 1 -- 172.21.15.44:0/194126544 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc009d00 con 0x7f0bd4104530 2023-09-10T00:39:38.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.789+0000 7f0bd359e700 1 --2- 172.21.15.44:0/194126544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bd4104530 0x7f0bd4101c40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00b830 tx=0x7f0bcc0094c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:38.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.789+0000 7f0bbbfff700 1 -- 172.21.15.44:0/194126544 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc00e040 con 0x7f0bd4104530 2023-09-10T00:39:38.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.789+0000 7f0bd9169700 1 -- 172.21.15.44:0/194126544 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd41a5100 con 0x7f0bd4104530 2023-09-10T00:39:38.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.789+0000 7f0bbbfff700 1 -- 172.21.15.44:0/194126544 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bcc003e90 con 0x7f0bd4104530 2023-09-10T00:39:38.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.789+0000 7f0bbbfff700 1 -- 172.21.15.44:0/194126544 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0047e0 con 0x7f0bd4104530 2023-09-10T00:39:38.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.789+0000 7f0bd9169700 1 -- 172.21.15.44:0/194126544 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd41a5570 con 0x7f0bd4104530 2023-09-10T00:39:38.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.790+0000 7f0bd9169700 1 -- 172.21.15.44:0/194126544 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd404e300 con 0x7f0bd4104530 2023-09-10T00:39:38.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.792+0000 7f0bbbfff700 1 -- 172.21.15.44:0/194126544 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f0bcc003a00 con 0x7f0bd4104530 2023-09-10T00:39:38.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.792+0000 7f0bbbfff700 1 --2- 172.21.15.44:0/194126544 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0bbc0a06d0 0x7f0bbc0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:38.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.792+0000 7f0bd2d9d700 1 --2- 172.21.15.44:0/194126544 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0bbc0a06d0 0x7f0bbc0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:38.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.792+0000 7f0bbbfff700 1 -- 172.21.15.44:0/194126544 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0bcc0cdf40 con 0x7f0bd4104530 2023-09-10T00:39:38.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.793+0000 7f0bd2d9d700 1 --2- 172.21.15.44:0/194126544 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0bbc0a06d0 0x7f0bbc0a2b90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4069c10 tx=0x7f0bc400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:38.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:38.796+0000 7f0bbbfff700 1 -- 172.21.15.44:0/194126544 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0bcc097630 con 0x7f0bd4104530 2023-09-10T00:39:38.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:38 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1663978434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-10T00:39:38.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:38 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2164179281' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-10T00:39:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.005+0000 7f0bd9169700 1 -- 172.21.15.44:0/194126544 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f0bd410f570 con 0x7f0bd4104530 2023-09-10T00:39:39.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.005+0000 7f0bbbfff700 1 -- 172.21.15.44:0/194126544 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f0bcc09fe60 con 0x7f0bd4104530 2023-09-10T00:39:39.007 INFO:teuthology.orchestra.run.smithi044.stdout:335007449110 2023-09-10T00:39:39.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.007+0000 7f0bd9169700 1 -- 172.21.15.44:0/194126544 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0bbc0a06d0 msgr2=0x7f0bbc0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:39.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.007+0000 7f0bd9169700 1 --2- 172.21.15.44:0/194126544 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0bbc0a06d0 0x7f0bbc0a2b90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4069c10 tx=0x7f0bc400a380 comp rx=0 tx=0).stop 2023-09-10T00:39:39.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.007+0000 7f0bd9169700 1 -- 172.21.15.44:0/194126544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bd4104530 msgr2=0x7f0bd4101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:39.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.007+0000 7f0bd9169700 1 --2- 172.21.15.44:0/194126544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bd4104530 0x7f0bd4101c40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00b830 tx=0x7f0bcc0094c0 comp rx=0 tx=0).stop 2023-09-10T00:39:39.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.008+0000 7f0bd9169700 1 -- 172.21.15.44:0/194126544 shutdown_connections 2023-09-10T00:39:39.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.008+0000 7f0bd9169700 1 --2- 172.21.15.44:0/194126544 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f0bbc0a06d0 0x7f0bbc0a2b90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:39.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.008+0000 7f0bd9169700 1 --2- 172.21.15.44:0/194126544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bd4104530 0x7f0bd4101c40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:39.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.008+0000 7f0bd9169700 1 --2- 172.21.15.44:0/194126544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd40ff520 0x7f0bd4101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:39.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.008+0000 7f0bd9169700 1 --2- 172.21.15.44:0/194126544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bd40ffee0 0x7f0bd41016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:39.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.008+0000 7f0bd9169700 1 -- 172.21.15.44:0/194126544 >> 172.21.15.44:0/194126544 conn(0x7f0bd4076600 msgr2=0x7f0bd4068160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:39.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.008+0000 7f0bd9169700 1 -- 172.21.15.44:0/194126544 shutdown_connections 2023-09-10T00:39:39.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.008+0000 7f0bd9169700 1 -- 172.21.15.44:0/194126544 wait complete. 2023-09-10T00:39:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:38 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1663978434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-10T00:39:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:38 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2164179281' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-10T00:39:39.117 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574912 got 55834574914 for osd.1 2023-09-10T00:39:39.117 DEBUG:teuthology.parallel:result is None 2023-09-10T00:39:39.227 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182455 got 107374182456 for osd.3 2023-09-10T00:39:39.228 DEBUG:teuthology.parallel:result is None 2023-09-10T00:39:39.809 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449108 got 335007449110 for osd.11 2023-09-10T00:39:39.810 DEBUG:teuthology.parallel:result is None 2023-09-10T00:39:39.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:39 smithi099 ceph-mon[102692]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-10T00:39:39.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:39 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/194126544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-10T00:39:39.857 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:39 smithi044 ceph-mon[98676]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-10T00:39:39.858 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:39 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/194126544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-10T00:39:39.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.932+0000 7f830326b700 1 -- 172.21.15.44:0/3711513669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc104530 msgr2=0x7f82fc104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:39.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.932+0000 7f830326b700 1 --2- 172.21.15.44:0/3711513669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc104530 0x7f82fc104910 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009990 tx=0x7f82f8009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:39.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.932+0000 7f830326b700 1 -- 172.21.15.44:0/3711513669 shutdown_connections 2023-09-10T00:39:39.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.932+0000 7f830326b700 1 --2- 172.21.15.44:0/3711513669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc0ffee0 0x7f82fc10c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:39.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.932+0000 7f830326b700 1 --2- 172.21.15.44:0/3711513669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc104530 0x7f82fc104910 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:39.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.932+0000 7f830326b700 1 --2- 172.21.15.44:0/3711513669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82fc0ff520 0x7f82fc0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:39.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.932+0000 7f830326b700 1 -- 172.21.15.44:0/3711513669 >> 172.21.15.44:0/3711513669 conn(0x7f82fc076600 msgr2=0x7f82fc076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:39.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.932+0000 7f830326b700 1 -- 172.21.15.44:0/3711513669 shutdown_connections 2023-09-10T00:39:39.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.932+0000 7f830326b700 1 -- 172.21.15.44:0/3711513669 wait complete. 2023-09-10T00:39:39.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.932+0000 7f830326b700 1 Processor -- start 2023-09-10T00:39:39.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.932+0000 7f830326b700 1 -- start start 2023-09-10T00:39:39.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f830326b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc0ff520 0x7f82fc196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:39.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f830326b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0ffee0 0x7f82fc196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:39.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f830326b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82fc104530 0x7f82fc19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:39.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f830326b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc19b5d0 con 0x7f82fc0ffee0 2023-09-10T00:39:39.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f830326b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc19b740 con 0x7f82fc104530 2023-09-10T00:39:39.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f830326b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc19b900 con 0x7f82fc0ff520 2023-09-10T00:39:39.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f8300806700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0ffee0 0x7f82fc196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:39.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f8300806700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0ffee0 0x7f82fc196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47268/0 (socket says 172.21.15.44:47268) 2023-09-10T00:39:39.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f8300806700 1 -- 172.21.15.44:0/1398381942 learned_addr learned my addr 172.21.15.44:0/1398381942 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f8301007700 1 --2- 172.21.15.44:0/1398381942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc0ff520 0x7f82fc196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f8301808700 1 --2- 172.21.15.44:0/1398381942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82fc104530 0x7f82fc19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:39.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f8300806700 1 -- 172.21.15.44:0/1398381942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc0ff520 msgr2=0x7f82fc196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:39.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f8300806700 1 --2- 172.21.15.44:0/1398381942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc0ff520 0x7f82fc196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:39.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f8300806700 1 -- 172.21.15.44:0/1398381942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82fc104530 msgr2=0x7f82fc19afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:39.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f8300806700 1 --2- 172.21.15.44:0/1398381942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82fc104530 0x7f82fc19afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:39.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f8300806700 1 -- 172.21.15.44:0/1398381942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f80095f0 con 0x7f82fc0ffee0 2023-09-10T00:39:39.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f8300806700 1 --2- 172.21.15.44:0/1398381942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0ffee0 0x7f82fc196b50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00d980 tx=0x7f82ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:39.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.933+0000 7f82f27fc700 1 -- 172.21.15.44:0/1398381942 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec004220 con 0x7f82fc0ffee0 2023-09-10T00:39:39.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.934+0000 7f82f27fc700 1 -- 172.21.15.44:0/1398381942 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82ec007640 con 0x7f82fc0ffee0 2023-09-10T00:39:39.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.934+0000 7f82f27fc700 1 -- 172.21.15.44:0/1398381942 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec005660 con 0x7f82fc0ffee0 2023-09-10T00:39:39.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.934+0000 7f830326b700 1 -- 172.21.15.44:0/1398381942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82fc19bbe0 con 0x7f82fc0ffee0 2023-09-10T00:39:39.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.934+0000 7f830326b700 1 -- 172.21.15.44:0/1398381942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc1a3440 con 0x7f82fc0ffee0 2023-09-10T00:39:39.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.934+0000 7f830326b700 1 -- 172.21.15.44:0/1398381942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82fc04e300 con 0x7f82fc0ffee0 2023-09-10T00:39:39.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.936+0000 7f82f27fc700 1 -- 172.21.15.44:0/1398381942 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f82ec004750 con 0x7f82fc0ffee0 2023-09-10T00:39:39.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.936+0000 7f82f27fc700 1 --2- 172.21.15.44:0/1398381942 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82e80a0750 0x7f82e80a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:39.986 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.936+0000 7f82f27fc700 1 -- 172.21.15.44:0/1398381942 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f82ec0cc2c0 con 0x7f82fc0ffee0 2023-09-10T00:39:39.986 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.936+0000 7f8301007700 1 --2- 172.21.15.44:0/1398381942 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82e80a0750 0x7f82e80a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:39.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.937+0000 7f8301007700 1 --2- 172.21.15.44:0/1398381942 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82e80a0750 0x7f82e80a2c10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f82f800b5c0 tx=0x7f82f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:39.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:39.940+0000 7f82f27fc700 1 -- 172.21.15.44:0/1398381942 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f82ec095980 con 0x7f82fc0ffee0 2023-09-10T00:39:40.045 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135906 got 244813135908 for osd.8 2023-09-10T00:39:40.045 DEBUG:teuthology.parallel:result is None 2023-09-10T00:39:40.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:39 smithi131 ceph-mon[103713]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-10T00:39:40.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:39 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/194126544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-10T00:39:40.161 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.160+0000 7f830326b700 1 -- 172.21.15.44:0/1398381942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f82fc197970 con 0x7f82fc0ffee0 2023-09-10T00:39:40.162 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.160+0000 7f82f27fc700 1 -- 172.21.15.44:0/1398381942 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f82ec016070 con 0x7f82fc0ffee0 2023-09-10T00:39:40.162 INFO:teuthology.orchestra.run.smithi044.stdout:300647710747 2023-09-10T00:39:40.164 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.162+0000 7f830326b700 1 -- 172.21.15.44:0/1398381942 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82e80a0750 msgr2=0x7f82e80a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:40.164 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.162+0000 7f830326b700 1 --2- 172.21.15.44:0/1398381942 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82e80a0750 0x7f82e80a2c10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f82f800b5c0 tx=0x7f82f8005fb0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.164 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.162+0000 7f830326b700 1 -- 172.21.15.44:0/1398381942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0ffee0 msgr2=0x7f82fc196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:40.164 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.162+0000 7f830326b700 1 --2- 172.21.15.44:0/1398381942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0ffee0 0x7f82fc196b50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00d980 tx=0x7f82ec00dc90 comp rx=0 tx=0).stop 2023-09-10T00:39:40.164 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.162+0000 7f830326b700 1 -- 172.21.15.44:0/1398381942 shutdown_connections 2023-09-10T00:39:40.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.162+0000 7f830326b700 1 --2- 172.21.15.44:0/1398381942 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f82e80a0750 0x7f82e80a2c10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.162+0000 7f830326b700 1 --2- 172.21.15.44:0/1398381942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82fc104530 0x7f82fc19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.162+0000 7f830326b700 1 --2- 172.21.15.44:0/1398381942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc0ff520 0x7f82fc196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.162+0000 7f830326b700 1 --2- 172.21.15.44:0/1398381942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82fc0ffee0 0x7f82fc196b50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.162+0000 7f830326b700 1 -- 172.21.15.44:0/1398381942 >> 172.21.15.44:0/1398381942 conn(0x7f82fc076600 msgr2=0x7f82fc0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:40.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.162+0000 7f830326b700 1 -- 172.21.15.44:0/1398381942 shutdown_connections 2023-09-10T00:39:40.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.162+0000 7f830326b700 1 -- 172.21.15.44:0/1398381942 wait complete. 2023-09-10T00:39:40.197 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757295 got 163208757297 for osd.5 2023-09-10T00:39:40.197 DEBUG:teuthology.parallel:result is None 2023-09-10T00:39:40.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.200+0000 7f54719ac700 1 -- 172.21.15.44:0/369122992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f546c10d240 msgr2=0x7f546c10f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:40.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.200+0000 7f54719ac700 1 --2- 172.21.15.44:0/369122992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f546c10d240 0x7f546c10f630 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f546400b0d0 tx=0x7f546400b3e0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.200+0000 7f54719ac700 1 -- 172.21.15.44:0/369122992 shutdown_connections 2023-09-10T00:39:40.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.200+0000 7f54719ac700 1 --2- 172.21.15.44:0/369122992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f546c10d240 0x7f546c10f630 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.200+0000 7f54719ac700 1 --2- 172.21.15.44:0/369122992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f546c072830 0x7f546c072c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.200+0000 7f54719ac700 1 --2- 172.21.15.44:0/369122992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c073150 0x7f546c10cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.200+0000 7f54719ac700 1 -- 172.21.15.44:0/369122992 >> 172.21.15.44:0/369122992 conn(0x7f546c078620 msgr2=0x7f546c078a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.200+0000 7f54719ac700 1 -- 172.21.15.44:0/369122992 shutdown_connections 2023-09-10T00:39:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.200+0000 7f54719ac700 1 -- 172.21.15.44:0/369122992 wait complete. 2023-09-10T00:39:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.201+0000 7f54719ac700 1 Processor -- start 2023-09-10T00:39:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.201+0000 7f54719ac700 1 -- start start 2023-09-10T00:39:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.201+0000 7f54719ac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f546c072830 0x7f546c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.201+0000 7f54719ac700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c073150 0x7f546c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.201+0000 7f54719ac700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f546c10d240 0x7f546c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.201+0000 7f54719ac700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c198dc0 con 0x7f546c10d240 2023-09-10T00:39:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.201+0000 7f54719ac700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c198f30 con 0x7f546c072830 2023-09-10T00:39:40.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.201+0000 7f54719ac700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c1990f0 con 0x7f546c073150 2023-09-10T00:39:40.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.202+0000 7f546b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f546c10d240 0x7f546c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:40.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.202+0000 7f546b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f546c10d240 0x7f546c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47284/0 (socket says 172.21.15.44:47284) 2023-09-10T00:39:40.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.202+0000 7f546b7fe700 1 -- 172.21.15.44:0/1693385411 learned_addr learned my addr 172.21.15.44:0/1693385411 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:40.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.202+0000 7f546a7fc700 1 --2- 172.21.15.44:0/1693385411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c073150 0x7f546c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:40.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.202+0000 7f546affd700 1 --2- 172.21.15.44:0/1693385411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f546c072830 0x7f546c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:40.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.202+0000 7f546b7fe700 1 -- 172.21.15.44:0/1693385411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c073150 msgr2=0x7f546c19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:40.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.202+0000 7f546b7fe700 1 --2- 172.21.15.44:0/1693385411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c073150 0x7f546c19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.202+0000 7f546b7fe700 1 -- 172.21.15.44:0/1693385411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f546c072830 msgr2=0x7f546c19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:40.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.202+0000 7f546b7fe700 1 --2- 172.21.15.44:0/1693385411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f546c072830 0x7f546c19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.202+0000 7f546b7fe700 1 -- 172.21.15.44:0/1693385411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5464009d00 con 0x7f546c10d240 2023-09-10T00:39:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.202+0000 7f546b7fe700 1 --2- 172.21.15.44:0/1693385411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f546c10d240 0x7f546c198750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5464009fd0 tx=0x7f546400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.202+0000 7f54709aa700 1 -- 172.21.15.44:0/1693385411 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546400e040 con 0x7f546c10d240 2023-09-10T00:39:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.202+0000 7f54709aa700 1 -- 172.21.15.44:0/1693385411 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54640094c0 con 0x7f546c10d240 2023-09-10T00:39:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.202+0000 7f54709aa700 1 -- 172.21.15.44:0/1693385411 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464005040 con 0x7f546c10d240 2023-09-10T00:39:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.202+0000 7f54719ac700 1 -- 172.21.15.44:0/1693385411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546c199370 con 0x7f546c10d240 2023-09-10T00:39:40.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.202+0000 7f54719ac700 1 -- 172.21.15.44:0/1693385411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546c1a56c0 con 0x7f546c10d240 2023-09-10T00:39:40.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.204+0000 7f54719ac700 1 -- 172.21.15.44:0/1693385411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f546c04e300 con 0x7f546c10d240 2023-09-10T00:39:40.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.205+0000 7f54709aa700 1 -- 172.21.15.44:0/1693385411 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f546401b450 con 0x7f546c10d240 2023-09-10T00:39:40.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.206+0000 7f54709aa700 1 --2- 172.21.15.44:0/1693385411 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f54540a0750 0x7f54540a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:40.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.206+0000 7f546affd700 1 --2- 172.21.15.44:0/1693385411 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f54540a0750 0x7f54540a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:40.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.206+0000 7f54709aa700 1 -- 172.21.15.44:0/1693385411 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f54640cdc30 con 0x7f546c10d240 2023-09-10T00:39:40.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.207+0000 7f546affd700 1 --2- 172.21.15.44:0/1693385411 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f54540a0750 0x7f54540a2c10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f545c005950 tx=0x7f545c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:40.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.210+0000 7f54709aa700 1 -- 172.21.15.44:0/1693385411 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5464097380 con 0x7f546c10d240 2023-09-10T00:39:40.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.315+0000 7ffb1c23d700 1 -- 172.21.15.44:0/3212371670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb14068120 msgr2=0x7ffb140685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:40.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.315+0000 7ffb1c23d700 1 --2- 172.21.15.44:0/3212371670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb14068120 0x7ffb140685a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009990 tx=0x7ffb04009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.315+0000 7ffb1c23d700 1 -- 172.21.15.44:0/3212371670 shutdown_connections 2023-09-10T00:39:40.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.315+0000 7ffb1c23d700 1 --2- 172.21.15.44:0/3212371670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb14068ae0 0x7ffb1410d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.315+0000 7ffb1c23d700 1 --2- 172.21.15.44:0/3212371670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb14106770 0x7ffb14106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.315+0000 7ffb1c23d700 1 --2- 172.21.15.44:0/3212371670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb14068120 0x7ffb140685a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.315+0000 7ffb1c23d700 1 -- 172.21.15.44:0/3212371670 >> 172.21.15.44:0/3212371670 conn(0x7ffb14076600 msgr2=0x7ffb14076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:40.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.315+0000 7ffb1c23d700 1 -- 172.21.15.44:0/3212371670 shutdown_connections 2023-09-10T00:39:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.315+0000 7ffb1c23d700 1 -- 172.21.15.44:0/3212371670 wait complete. 2023-09-10T00:39:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.315+0000 7ffb1c23d700 1 Processor -- start 2023-09-10T00:39:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb1c23d700 1 -- start start 2023-09-10T00:39:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb1c23d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb14068120 0x7ffb140ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb1c23d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb14068ae0 0x7ffb140ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb1c23d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb14106770 0x7ffb14100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb1c23d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb141a49f0 con 0x7ffb14068ae0 2023-09-10T00:39:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb1c23d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb141a4b60 con 0x7ffb14106770 2023-09-10T00:39:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb1c23d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb141a4d20 con 0x7ffb14068120 2023-09-10T00:39:40.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb197d8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb14068ae0 0x7ffb140ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:40.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb197d8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb14068ae0 0x7ffb140ffaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47294/0 (socket says 172.21.15.44:47294) 2023-09-10T00:39:40.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb19fd9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb14068120 0x7ffb140ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:40.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb197d8700 1 -- 172.21.15.44:0/1661743620 learned_addr learned my addr 172.21.15.44:0/1661743620 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:40.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb1a7da700 1 --2- 172.21.15.44:0/1661743620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb14106770 0x7ffb14100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:40.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb197d8700 1 -- 172.21.15.44:0/1661743620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb14068120 msgr2=0x7ffb140ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:40.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb197d8700 1 --2- 172.21.15.44:0/1661743620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb14068120 0x7ffb140ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb197d8700 1 -- 172.21.15.44:0/1661743620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb14106770 msgr2=0x7ffb14100070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:40.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb197d8700 1 --2- 172.21.15.44:0/1661743620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb14106770 0x7ffb14100070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb197d8700 1 -- 172.21.15.44:0/1661743620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb040095f0 con 0x7ffb14068ae0 2023-09-10T00:39:40.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb19fd9700 1 --2- 172.21.15.44:0/1661743620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb14068120 0x7ffb140ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:39:40.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.316+0000 7ffb197d8700 1 --2- 172.21.15.44:0/1661743620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb14068ae0 0x7ffb140ffaf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffb04006010 tx=0x7ffb04005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:40.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.317+0000 7ffb0affd700 1 -- 172.21.15.44:0/1661743620 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0401d070 con 0x7ffb14068ae0 2023-09-10T00:39:40.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.317+0000 7ffb0affd700 1 -- 172.21.15.44:0/1661743620 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb04003f80 con 0x7ffb14068ae0 2023-09-10T00:39:40.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.317+0000 7ffb0affd700 1 -- 172.21.15.44:0/1661743620 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04022b90 con 0x7ffb14068ae0 2023-09-10T00:39:40.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.317+0000 7ffb1c23d700 1 -- 172.21.15.44:0/1661743620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb141a4fa0 con 0x7ffb14068ae0 2023-09-10T00:39:40.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.317+0000 7ffb1c23d700 1 -- 172.21.15.44:0/1661743620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb141a5300 con 0x7ffb14068ae0 2023-09-10T00:39:40.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.318+0000 7ffb1c23d700 1 -- 172.21.15.44:0/1661743620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb1404e300 con 0x7ffb14068ae0 2023-09-10T00:39:40.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.319+0000 7ffb0affd700 1 -- 172.21.15.44:0/1661743620 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7ffb04003af0 con 0x7ffb14068ae0 2023-09-10T00:39:40.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.320+0000 7ffb0affd700 1 --2- 172.21.15.44:0/1661743620 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb000a0750 0x7ffb000a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:40.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.320+0000 7ffb19fd9700 1 --2- 172.21.15.44:0/1661743620 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb000a0750 0x7ffb000a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:40.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.320+0000 7ffb0affd700 1 -- 172.21.15.44:0/1661743620 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ffb040cd4c0 con 0x7ffb14068ae0 2023-09-10T00:39:40.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.320+0000 7ffb19fd9700 1 --2- 172.21.15.44:0/1661743620 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb000a0750 0x7ffb000a2c10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000afd0 tx=0x7ffb1000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:40.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.324+0000 7ffb0affd700 1 -- 172.21.15.44:0/1661743620 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ffb04096c10 con 0x7ffb14068ae0 2023-09-10T00:39:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.443+0000 7f54719ac700 1 -- 172.21.15.44:0/1693385411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f546c112720 con 0x7f546c10d240 2023-09-10T00:39:40.446 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.443+0000 7f54709aa700 1 -- 172.21.15.44:0/1693385411 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f546409a9a0 con 0x7f546c10d240 2023-09-10T00:39:40.446 INFO:teuthology.orchestra.run.smithi044.stdout:77309411389 2023-09-10T00:39:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.446+0000 7f54719ac700 1 -- 172.21.15.44:0/1693385411 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f54540a0750 msgr2=0x7f54540a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.446+0000 7f54719ac700 1 --2- 172.21.15.44:0/1693385411 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f54540a0750 0x7f54540a2c10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f545c005950 tx=0x7f545c00b410 comp rx=0 tx=0).stop 2023-09-10T00:39:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.446+0000 7f54719ac700 1 -- 172.21.15.44:0/1693385411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f546c10d240 msgr2=0x7f546c198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.446+0000 7f54719ac700 1 --2- 172.21.15.44:0/1693385411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f546c10d240 0x7f546c198750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5464009fd0 tx=0x7f546400cce0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.446+0000 7f54719ac700 1 -- 172.21.15.44:0/1693385411 shutdown_connections 2023-09-10T00:39:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.446+0000 7f54719ac700 1 --2- 172.21.15.44:0/1693385411 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f54540a0750 0x7f54540a2c10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.446+0000 7f54719ac700 1 --2- 172.21.15.44:0/1693385411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f546c10d240 0x7f546c198750 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.446+0000 7f54719ac700 1 --2- 172.21.15.44:0/1693385411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f546c072830 0x7f546c19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.446+0000 7f54719ac700 1 --2- 172.21.15.44:0/1693385411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c073150 0x7f546c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.446+0000 7f54719ac700 1 -- 172.21.15.44:0/1693385411 >> 172.21.15.44:0/1693385411 conn(0x7f546c078620 msgr2=0x7f546c0fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.446+0000 7f54719ac700 1 -- 172.21.15.44:0/1693385411 shutdown_connections 2023-09-10T00:39:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.446+0000 7f54719ac700 1 -- 172.21.15.44:0/1693385411 wait complete. 2023-09-10T00:39:40.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.556+0000 7ffb1c23d700 1 -- 172.21.15.44:0/1661743620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffb14110740 con 0x7ffb14068ae0 2023-09-10T00:39:40.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.556+0000 7ffb0affd700 1 -- 172.21.15.44:0/1661743620 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffb0409a230 con 0x7ffb14068ae0 2023-09-10T00:39:40.558 INFO:teuthology.orchestra.run.smithi044.stdout:34359738439 2023-09-10T00:39:40.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.560+0000 7ffb1c23d700 1 -- 172.21.15.44:0/1661743620 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb000a0750 msgr2=0x7ffb000a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:40.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.560+0000 7ffb1c23d700 1 --2- 172.21.15.44:0/1661743620 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb000a0750 0x7ffb000a2c10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000afd0 tx=0x7ffb1000a380 comp rx=0 tx=0).stop 2023-09-10T00:39:40.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.560+0000 7ffb1c23d700 1 -- 172.21.15.44:0/1661743620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb14068ae0 msgr2=0x7ffb140ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:40.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.560+0000 7ffb1c23d700 1 --2- 172.21.15.44:0/1661743620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb14068ae0 0x7ffb140ffaf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffb04006010 tx=0x7ffb04005740 comp rx=0 tx=0).stop 2023-09-10T00:39:40.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.560+0000 7ffb1c23d700 1 -- 172.21.15.44:0/1661743620 shutdown_connections 2023-09-10T00:39:40.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.560+0000 7ffb1c23d700 1 --2- 172.21.15.44:0/1661743620 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffb000a0750 0x7ffb000a2c10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.560+0000 7ffb1c23d700 1 --2- 172.21.15.44:0/1661743620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb14106770 0x7ffb14100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.560+0000 7ffb1c23d700 1 --2- 172.21.15.44:0/1661743620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb14068120 0x7ffb140ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.560+0000 7ffb1c23d700 1 --2- 172.21.15.44:0/1661743620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb14068ae0 0x7ffb140ffaf0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:40.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.560+0000 7ffb1c23d700 1 -- 172.21.15.44:0/1661743620 >> 172.21.15.44:0/1661743620 conn(0x7ffb14076600 msgr2=0x7ffb140fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:40.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.562+0000 7ffb1c23d700 1 -- 172.21.15.44:0/1661743620 shutdown_connections 2023-09-10T00:39:40.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:40.562+0000 7ffb1c23d700 1 -- 172.21.15.44:0/1661743620 wait complete. 2023-09-10T00:39:40.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:40 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1398381942' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-10T00:39:40.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:40 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:39:40.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:40 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1693385411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-10T00:39:40.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:40 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1661743620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-10T00:39:40.891 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:40 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1398381942' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-10T00:39:40.891 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:40 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:39:40.891 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:40 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1693385411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-10T00:39:40.892 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:40 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1661743620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-10T00:39:40.956 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710745 got 300647710747 for osd.10 2023-09-10T00:39:40.956 DEBUG:teuthology.parallel:result is None 2023-09-10T00:39:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:40 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1398381942' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-10T00:39:41.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:40 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:39:41.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:40 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1693385411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-10T00:39:41.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:40 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1661743620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-10T00:39:41.415 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738437 got 34359738439 for osd.0 2023-09-10T00:39:41.415 DEBUG:teuthology.parallel:result is None 2023-09-10T00:39:41.515 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411387 got 77309411389 for osd.2 2023-09-10T00:39:41.516 DEBUG:teuthology.parallel:result is None 2023-09-10T00:39:41.516 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-09-10T00:39:41.516 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph pg dump --format=json 2023-09-10T00:39:41.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:41 smithi044 ceph-mon[98676]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s 2023-09-10T00:39:41.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:41 smithi099 ceph-mon[102692]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s 2023-09-10T00:39:42.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:41 smithi131 ceph-mon[103713]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s 2023-09-10T00:39:42.151 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:43.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.353+0000 7fe99ca03700 1 -- 172.21.15.44:0/3060526446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe998104530 msgr2=0x7fe998104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:43.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.353+0000 7fe99ca03700 1 --2- 172.21.15.44:0/3060526446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe998104530 0x7fe998104910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe988009990 tx=0x7fe988009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:43.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.353+0000 7fe99ca03700 1 -- 172.21.15.44:0/3060526446 shutdown_connections 2023-09-10T00:39:43.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.353+0000 7fe99ca03700 1 --2- 172.21.15.44:0/3060526446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9980ffee0 0x7fe99810c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:43.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.353+0000 7fe99ca03700 1 --2- 172.21.15.44:0/3060526446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe998104530 0x7fe998104910 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:43.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.353+0000 7fe99ca03700 1 --2- 172.21.15.44:0/3060526446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9980ff520 0x7fe9980ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:43.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.353+0000 7fe99ca03700 1 -- 172.21.15.44:0/3060526446 >> 172.21.15.44:0/3060526446 conn(0x7fe998076600 msgr2=0x7fe998076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:43.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.353+0000 7fe99ca03700 1 -- 172.21.15.44:0/3060526446 shutdown_connections 2023-09-10T00:39:43.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.353+0000 7fe99ca03700 1 -- 172.21.15.44:0/3060526446 wait complete. 2023-09-10T00:39:43.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.354+0000 7fe99ca03700 1 Processor -- start 2023-09-10T00:39:43.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.354+0000 7fe99ca03700 1 -- start start 2023-09-10T00:39:43.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.354+0000 7fe99ca03700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9980ff520 0x7fe998197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:43.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.354+0000 7fe99ca03700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9980ffee0 0x7fe998197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:43.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.354+0000 7fe99ca03700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe998104530 0x7fe9981a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:43.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.354+0000 7fe99ca03700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9981982d0 con 0x7fe998104530 2023-09-10T00:39:43.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.354+0000 7fe99ca03700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9981a2a10 con 0x7fe9980ff520 2023-09-10T00:39:43.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.354+0000 7fe99ca03700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9981a2bd0 con 0x7fe9980ffee0 2023-09-10T00:39:43.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe99659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9980ff520 0x7fe998197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:43.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe996d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe998104530 0x7fe9981a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:43.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe99659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9980ff520 0x7fe998197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:49044/0 (socket says 172.21.15.44:49044) 2023-09-10T00:39:43.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe99659c700 1 -- 172.21.15.44:0/2157343652 learned_addr learned my addr 172.21.15.44:0/2157343652 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:43.362 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe995d9b700 1 --2- 172.21.15.44:0/2157343652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9980ffee0 0x7fe998197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:43.362 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe996d9d700 1 -- 172.21.15.44:0/2157343652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9980ffee0 msgr2=0x7fe998197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:43.362 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe996d9d700 1 --2- 172.21.15.44:0/2157343652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9980ffee0 0x7fe998197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:43.362 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe996d9d700 1 -- 172.21.15.44:0/2157343652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9980ff520 msgr2=0x7fe998197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe996d9d700 1 --2- 172.21.15.44:0/2157343652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9980ff520 0x7fe998197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe996d9d700 1 -- 172.21.15.44:0/2157343652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9880095f0 con 0x7fe998104530 2023-09-10T00:39:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe99659c700 1 --2- 172.21.15.44:0/2157343652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9980ff520 0x7fe998197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:39:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe996d9d700 1 --2- 172.21.15.44:0/2157343652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe998104530 0x7fe9981a24d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe99000efc0 tx=0x7fe99000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe98f7fe700 1 -- 172.21.15.44:0/2157343652 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe990018070 con 0x7fe998104530 2023-09-10T00:39:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe98f7fe700 1 -- 172.21.15.44:0/2157343652 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe99000f040 con 0x7fe998104530 2023-09-10T00:39:43.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe99ca03700 1 -- 172.21.15.44:0/2157343652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9981a2e50 con 0x7fe998104530 2023-09-10T00:39:43.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe98f7fe700 1 -- 172.21.15.44:0/2157343652 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99000cbd0 con 0x7fe998104530 2023-09-10T00:39:43.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.355+0000 7fe99ca03700 1 -- 172.21.15.44:0/2157343652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9981a32d0 con 0x7fe998104530 2023-09-10T00:39:43.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.356+0000 7fe99ca03700 1 -- 172.21.15.44:0/2157343652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99804e300 con 0x7fe998104530 2023-09-10T00:39:43.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.357+0000 7fe98f7fe700 1 -- 172.21.15.44:0/2157343652 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fe990007500 con 0x7fe998104530 2023-09-10T00:39:43.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.358+0000 7fe98f7fe700 1 --2- 172.21.15.44:0/2157343652 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe9840a06d0 0x7fe9840a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:43.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.358+0000 7fe99659c700 1 --2- 172.21.15.44:0/2157343652 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe9840a06d0 0x7fe9840a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:43.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.358+0000 7fe98f7fe700 1 -- 172.21.15.44:0/2157343652 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe9900cbb00 con 0x7fe998104530 2023-09-10T00:39:43.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.359+0000 7fe99659c700 1 --2- 172.21.15.44:0/2157343652 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe9840a06d0 0x7fe9840a2b90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe988006010 tx=0x7fe988005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:43.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.363+0000 7fe98f7fe700 1 -- 172.21.15.44:0/2157343652 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe9900cde60 con 0x7fe998104530 2023-09-10T00:39:43.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.565+0000 7fe99ca03700 1 -- 172.21.15.44:0/2157343652 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe99810f5c0 con 0x7fe9840a06d0 2023-09-10T00:39:43.587 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.586+0000 7fe98f7fe700 1 -- 172.21.15.44:0/2157343652 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+424263 (secure 0 0 0) 0x7fe99810f5c0 con 0x7fe9840a06d0 2023-09-10T00:39:43.592 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:43.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.589+0000 7fe99ca03700 1 -- 172.21.15.44:0/2157343652 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe9840a06d0 msgr2=0x7fe9840a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:43.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.589+0000 7fe99ca03700 1 --2- 172.21.15.44:0/2157343652 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe9840a06d0 0x7fe9840a2b90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe988006010 tx=0x7fe988005960 comp rx=0 tx=0).stop 2023-09-10T00:39:43.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.589+0000 7fe99ca03700 1 -- 172.21.15.44:0/2157343652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe998104530 msgr2=0x7fe9981a24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:43.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.589+0000 7fe99ca03700 1 --2- 172.21.15.44:0/2157343652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe998104530 0x7fe9981a24d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe99000efc0 tx=0x7fe99000c3a0 comp rx=0 tx=0).stop 2023-09-10T00:39:43.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.589+0000 7fe99ca03700 1 -- 172.21.15.44:0/2157343652 shutdown_connections 2023-09-10T00:39:43.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.589+0000 7fe99ca03700 1 --2- 172.21.15.44:0/2157343652 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe9840a06d0 0x7fe9840a2b90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:43.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.589+0000 7fe99ca03700 1 --2- 172.21.15.44:0/2157343652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe998104530 0x7fe9981a24d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:43.597 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.589+0000 7fe99ca03700 1 --2- 172.21.15.44:0/2157343652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9980ff520 0x7fe998197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:43.597 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.589+0000 7fe99ca03700 1 --2- 172.21.15.44:0/2157343652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9980ffee0 0x7fe998197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:43.597 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.590+0000 7fe99ca03700 1 -- 172.21.15.44:0/2157343652 >> 172.21.15.44:0/2157343652 conn(0x7fe998076600 msgr2=0x7fe9980fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:43.597 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.590+0000 7fe99ca03700 1 -- 172.21.15.44:0/2157343652 shutdown_connections 2023-09-10T00:39:43.597 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:43.590+0000 7fe99ca03700 1 -- 172.21.15.44:0/2157343652 wait complete. 2023-09-10T00:39:43.597 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2023-09-10T00:39:43.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:43 smithi099 ceph-mon[102692]: pgmap v302: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-09-10T00:39:43.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:43.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:43.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:43.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:43 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:43.910 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:43 smithi044 ceph-mon[98676]: pgmap v302: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-09-10T00:39:43.910 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:43.911 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:43.911 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:43.911 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:43 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:43.912 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2023-09-10T00:39:42.010625+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656415,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":569,"num_read_kb":410,"num_write":417,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338132,"kb_used_data":13884,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603692,"statfs":{"total":1151940427776,"available":1151594180608,"internally_reserved":0,"allocated":14217216,"data_stored":7824513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":59,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003372"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.520087+0000","last_change":"2023-09-10T00:38:39.768218+0000","last_active":"2023-09-10T00:39:16.520087+0000","last_peered":"2023-09-10T00:39:16.520087+0000","last_clean":"2023-09-10T00:39:16.520087+0000","last_became_active":"2023-09-10T00:38:39.767454+0000","last_became_peered":"2023-09-10T00:38:39.767454+0000","last_unstale":"2023-09-10T00:39:16.520087+0000","last_undegraded":"2023-09-10T00:39:16.520087+0000","last_fullsized":"2023-09-10T00:39:16.520087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:17:30.287403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.528889+0000","last_change":"2023-09-10T00:38:39.765384+0000","last_active":"2023-09-10T00:39:16.528889+0000","last_peered":"2023-09-10T00:39:16.528889+0000","last_clean":"2023-09-10T00:39:16.528889+0000","last_became_active":"2023-09-10T00:38:39.765090+0000","last_became_peered":"2023-09-10T00:38:39.765090+0000","last_unstale":"2023-09-10T00:39:16.528889+0000","last_undegraded":"2023-09-10T00:39:16.528889+0000","last_fullsized":"2023-09-10T00:39:16.528889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:02:46.025627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.528127+0000","last_change":"2023-09-10T00:38:39.769222+0000","last_active":"2023-09-10T00:39:16.528127+0000","last_peered":"2023-09-10T00:39:16.528127+0000","last_clean":"2023-09-10T00:39:16.528127+0000","last_became_active":"2023-09-10T00:38:39.768801+0000","last_became_peered":"2023-09-10T00:38:39.768801+0000","last_unstale":"2023-09-10T00:39:16.528127+0000","last_undegraded":"2023-09-10T00:39:16.528127+0000","last_fullsized":"2023-09-10T00:39:16.528127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:38:53.394209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522938+0000","last_change":"2023-09-10T00:38:39.764963+0000","last_active":"2023-09-10T00:39:16.522938+0000","last_peered":"2023-09-10T00:39:16.522938+0000","last_clean":"2023-09-10T00:39:16.522938+0000","last_became_active":"2023-09-10T00:38:39.764466+0000","last_became_peered":"2023-09-10T00:38:39.764466+0000","last_unstale":"2023-09-10T00:39:16.522938+0000","last_undegraded":"2023-09-10T00:39:16.522938+0000","last_fullsized":"2023-09-10T00:39:16.522938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:31.553321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.534684+0000","last_change":"2023-09-10T00:39:17.534684+0000","last_active":"2023-09-10T00:39:17.534684+0000","last_peered":"2023-09-10T00:39:17.534684+0000","last_clean":"2023-09-10T00:39:17.534684+0000","last_became_active":"2023-09-10T00:39:17.534325+0000","last_became_peered":"2023-09-10T00:39:17.534325+0000","last_unstale":"2023-09-10T00:39:17.534684+0000","last_undegraded":"2023-09-10T00:39:17.534684+0000","last_fullsized":"2023-09-10T00:39:17.534684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:39:11.902221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526669+0000","last_change":"2023-09-10T00:38:39.761681+0000","last_active":"2023-09-10T00:39:16.526669+0000","last_peered":"2023-09-10T00:39:16.526669+0000","last_clean":"2023-09-10T00:39:16.526669+0000","last_became_active":"2023-09-10T00:38:39.761383+0000","last_became_peered":"2023-09-10T00:38:39.761383+0000","last_unstale":"2023-09-10T00:39:16.526669+0000","last_undegraded":"2023-09-10T00:39:16.526669+0000","last_fullsized":"2023-09-10T00:39:16.526669+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:38:04.534656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527509+0000","last_change":"2023-09-10T00:38:39.765028+0000","last_active":"2023-09-10T00:39:16.527509+0000","last_peered":"2023-09-10T00:39:16.527509+0000","last_clean":"2023-09-10T00:39:16.527509+0000","last_became_active":"2023-09-10T00:38:39.764663+0000","last_became_peered":"2023-09-10T00:38:39.764663+0000","last_unstale":"2023-09-10T00:39:16.527509+0000","last_undegraded":"2023-09-10T00:39:16.527509+0000","last_fullsized":"2023-09-10T00:39:16.527509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:14:49.645129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.536787+0000","last_change":"2023-09-10T00:38:39.770135+0000","last_active":"2023-09-10T00:39:16.536787+0000","last_peered":"2023-09-10T00:39:16.536787+0000","last_clean":"2023-09-10T00:39:16.536787+0000","last_became_active":"2023-09-10T00:38:39.769720+0000","last_became_peered":"2023-09-10T00:38:39.769720+0000","last_unstale":"2023-09-10T00:39:16.536787+0000","last_undegraded":"2023-09-10T00:39:16.536787+0000","last_fullsized":"2023-09-10T00:39:16.536787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:46:21.166566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526931+0000","last_change":"2023-09-10T00:38:39.770170+0000","last_active":"2023-09-10T00:39:16.526931+0000","last_peered":"2023-09-10T00:39:16.526931+0000","last_clean":"2023-09-10T00:39:16.526931+0000","last_became_active":"2023-09-10T00:38:39.769840+0000","last_became_peered":"2023-09-10T00:38:39.769840+0000","last_unstale":"2023-09-10T00:39:16.526931+0000","last_undegraded":"2023-09-10T00:39:16.526931+0000","last_fullsized":"2023-09-10T00:39:16.526931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:48.117670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527116+0000","last_change":"2023-09-10T00:38:39.769621+0000","last_active":"2023-09-10T00:39:16.527116+0000","last_peered":"2023-09-10T00:39:16.527116+0000","last_clean":"2023-09-10T00:39:16.527116+0000","last_became_active":"2023-09-10T00:38:39.769300+0000","last_became_peered":"2023-09-10T00:38:39.769300+0000","last_unstale":"2023-09-10T00:39:16.527116+0000","last_undegraded":"2023-09-10T00:39:16.527116+0000","last_fullsized":"2023-09-10T00:39:16.527116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:37.100671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527669+0000","last_change":"2023-09-10T00:38:39.772200+0000","last_active":"2023-09-10T00:39:16.527669+0000","last_peered":"2023-09-10T00:39:16.527669+0000","last_clean":"2023-09-10T00:39:16.527669+0000","last_became_active":"2023-09-10T00:38:39.771503+0000","last_became_peered":"2023-09-10T00:38:39.771503+0000","last_unstale":"2023-09-10T00:39:16.527669+0000","last_undegraded":"2023-09-10T00:39:16.527669+0000","last_fullsized":"2023-09-10T00:39:16.527669+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:13:38.830642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522841+0000","last_change":"2023-09-10T00:38:39.769832+0000","last_active":"2023-09-10T00:39:16.522841+0000","last_peered":"2023-09-10T00:39:16.522841+0000","last_clean":"2023-09-10T00:39:16.522841+0000","last_became_active":"2023-09-10T00:38:39.769546+0000","last_became_peered":"2023-09-10T00:38:39.769546+0000","last_unstale":"2023-09-10T00:39:16.522841+0000","last_undegraded":"2023-09-10T00:39:16.522841+0000","last_fullsized":"2023-09-10T00:39:16.522841+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:53:05.219877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.531430+0000","last_change":"2023-09-10T00:39:17.531430+0000","last_active":"2023-09-10T00:39:17.531430+0000","last_peered":"2023-09-10T00:39:17.531430+0000","last_clean":"2023-09-10T00:39:17.531430+0000","last_became_active":"2023-09-10T00:39:17.531145+0000","last_became_peered":"2023-09-10T00:39:17.531145+0000","last_unstale":"2023-09-10T00:39:17.531430+0000","last_undegraded":"2023-09-10T00:39:17.531430+0000","last_fullsized":"2023-09-10T00:39:17.531430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:34:50.163747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527248+0000","last_change":"2023-09-10T00:38:39.764751+0000","last_active":"2023-09-10T00:39:16.527248+0000","last_peered":"2023-09-10T00:39:16.527248+0000","last_clean":"2023-09-10T00:39:16.527248+0000","last_became_active":"2023-09-10T00:38:39.764464+0000","last_became_peered":"2023-09-10T00:38:39.764464+0000","last_unstale":"2023-09-10T00:39:16.527248+0000","last_undegraded":"2023-09-10T00:39:16.527248+0000","last_fullsized":"2023-09-10T00:39:16.527248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:03:51.096126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.532977+0000","last_change":"2023-09-10T00:39:17.532977+0000","last_active":"2023-09-10T00:39:17.532977+0000","last_peered":"2023-09-10T00:39:17.532977+0000","last_clean":"2023-09-10T00:39:17.532977+0000","last_became_active":"2023-09-10T00:39:17.532547+0000","last_became_peered":"2023-09-10T00:39:17.532547+0000","last_unstale":"2023-09-10T00:39:17.532977+0000","last_undegraded":"2023-09-10T00:39:17.532977+0000","last_fullsized":"2023-09-10T00:39:17.532977+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:33:01.685743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.525442+0000","last_change":"2023-09-10T00:38:39.770787+0000","last_active":"2023-09-10T00:39:16.525442+0000","last_peered":"2023-09-10T00:39:16.525442+0000","last_clean":"2023-09-10T00:39:16.525442+0000","last_became_active":"2023-09-10T00:38:39.770497+0000","last_became_peered":"2023-09-10T00:38:39.770497+0000","last_unstale":"2023-09-10T00:39:16.525442+0000","last_undegraded":"2023-09-10T00:39:16.525442+0000","last_fullsized":"2023-09-10T00:39:16.525442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:43:11.066766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.519978+0000","last_change":"2023-09-10T00:38:39.768008+0000","last_active":"2023-09-10T00:39:16.519978+0000","last_peered":"2023-09-10T00:39:16.519978+0000","last_clean":"2023-09-10T00:39:16.519978+0000","last_became_active":"2023-09-10T00:38:39.767079+0000","last_became_peered":"2023-09-10T00:38:39.767079+0000","last_unstale":"2023-09-10T00:39:16.519978+0000","last_undegraded":"2023-09-10T00:39:16.519978+0000","last_fullsized":"2023-09-10T00:39:16.519978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:34:00.689881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.532177+0000","last_change":"2023-09-10T00:38:39.775326+0000","last_active":"2023-09-10T00:39:16.532177+0000","last_peered":"2023-09-10T00:39:16.532177+0000","last_clean":"2023-09-10T00:39:16.532177+0000","last_became_active":"2023-09-10T00:38:39.775007+0000","last_became_peered":"2023-09-10T00:38:39.775007+0000","last_unstale":"2023-09-10T00:39:16.532177+0000","last_undegraded":"2023-09-10T00:39:16.532177+0000","last_fullsized":"2023-09-10T00:39:16.532177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:59:27.711722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527686+0000","last_change":"2023-09-10T00:38:39.764974+0000","last_active":"2023-09-10T00:39:16.527686+0000","last_peered":"2023-09-10T00:39:16.527686+0000","last_clean":"2023-09-10T00:39:16.527686+0000","last_became_active":"2023-09-10T00:38:39.764368+0000","last_became_peered":"2023-09-10T00:38:39.764368+0000","last_unstale":"2023-09-10T00:39:16.527686+0000","last_undegraded":"2023-09-10T00:39:16.527686+0000","last_fullsized":"2023-09-10T00:39:16.527686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:40:28.046021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526817+0000","last_change":"2023-09-10T00:38:43.793589+0000","last_active":"2023-09-10T00:39:16.526817+0000","last_peered":"2023-09-10T00:39:16.526817+0000","last_clean":"2023-09-10T00:39:16.526817+0000","last_became_active":"2023-09-10T00:38:43.793166+0000","last_became_peered":"2023-09-10T00:38:43.793166+0000","last_unstale":"2023-09-10T00:39:16.526817+0000","last_undegraded":"2023-09-10T00:39:16.526817+0000","last_fullsized":"2023-09-10T00:39:16.526817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:51:47.128925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":71,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:39.589556+0000","last_change":"2023-09-10T00:38:38.759909+0000","last_active":"2023-09-10T00:39:39.589556+0000","last_peered":"2023-09-10T00:39:39.589556+0000","last_clean":"2023-09-10T00:39:39.589556+0000","last_became_active":"2023-09-10T00:38:38.759561+0000","last_became_peered":"2023-09-10T00:38:38.759561+0000","last_unstale":"2023-09-10T00:39:39.589556+0000","last_undegraded":"2023-09-10T00:39:39.589556+0000","last_fullsized":"2023-09-10T00:39:39.589556+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:56:18.009497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.534652+0000","last_change":"2023-09-10T00:38:39.774916+0000","last_active":"2023-09-10T00:39:16.534652+0000","last_peered":"2023-09-10T00:39:16.534652+0000","last_clean":"2023-09-10T00:39:16.534652+0000","last_became_active":"2023-09-10T00:38:39.774584+0000","last_became_peered":"2023-09-10T00:38:39.774584+0000","last_unstale":"2023-09-10T00:39:16.534652+0000","last_undegraded":"2023-09-10T00:39:16.534652+0000","last_fullsized":"2023-09-10T00:39:16.534652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:36:33.125742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522181+0000","last_change":"2023-09-10T00:38:43.789421+0000","last_active":"2023-09-10T00:39:16.522181+0000","last_peered":"2023-09-10T00:39:16.522181+0000","last_clean":"2023-09-10T00:39:16.522181+0000","last_became_active":"2023-09-10T00:38:43.788569+0000","last_became_peered":"2023-09-10T00:38:43.788569+0000","last_unstale":"2023-09-10T00:39:16.522181+0000","last_undegraded":"2023-09-10T00:39:16.522181+0000","last_fullsized":"2023-09-10T00:39:16.522181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:07:38.529811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.534752+0000","last_change":"2023-09-10T00:38:38.767381+0000","last_active":"2023-09-10T00:39:16.534752+0000","last_peered":"2023-09-10T00:39:16.534752+0000","last_clean":"2023-09-10T00:39:16.534752+0000","last_became_active":"2023-09-10T00:38:38.767082+0000","last_became_peered":"2023-09-10T00:38:38.767082+0000","last_unstale":"2023-09-10T00:39:16.534752+0000","last_undegraded":"2023-09-10T00:39:16.534752+0000","last_fullsized":"2023-09-10T00:39:16.534752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:13:32.533215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.528233+0000","last_change":"2023-09-10T00:38:39.772645+0000","last_active":"2023-09-10T00:39:16.528233+0000","last_peered":"2023-09-10T00:39:16.528233+0000","last_clean":"2023-09-10T00:39:16.528233+0000","last_became_active":"2023-09-10T00:38:39.771554+0000","last_became_peered":"2023-09-10T00:38:39.771554+0000","last_unstale":"2023-09-10T00:39:16.528233+0000","last_undegraded":"2023-09-10T00:39:16.528233+0000","last_fullsized":"2023-09-10T00:39:16.528233+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:25:06.237309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522555+0000","last_change":"2023-09-10T00:38:43.795764+0000","last_active":"2023-09-10T00:39:16.522555+0000","last_peered":"2023-09-10T00:39:16.522555+0000","last_clean":"2023-09-10T00:39:16.522555+0000","last_became_active":"2023-09-10T00:38:43.795112+0000","last_became_peered":"2023-09-10T00:38:43.795112+0000","last_unstale":"2023-09-10T00:39:16.522555+0000","last_undegraded":"2023-09-10T00:39:16.522555+0000","last_fullsized":"2023-09-10T00:39:16.522555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:18:55.460752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.524242+0000","last_change":"2023-09-10T00:38:38.765522+0000","last_active":"2023-09-10T00:39:16.524242+0000","last_peered":"2023-09-10T00:39:16.524242+0000","last_clean":"2023-09-10T00:39:16.524242+0000","last_became_active":"2023-09-10T00:38:38.765219+0000","last_became_peered":"2023-09-10T00:38:38.765219+0000","last_unstale":"2023-09-10T00:39:16.524242+0000","last_undegraded":"2023-09-10T00:39:16.524242+0000","last_fullsized":"2023-09-10T00:39:16.524242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:00:27.275269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526550+0000","last_change":"2023-09-10T00:38:39.767790+0000","last_active":"2023-09-10T00:39:16.526550+0000","last_peered":"2023-09-10T00:39:16.526550+0000","last_clean":"2023-09-10T00:39:16.526550+0000","last_became_active":"2023-09-10T00:38:39.767513+0000","last_became_peered":"2023-09-10T00:38:39.767513+0000","last_unstale":"2023-09-10T00:39:16.526550+0000","last_undegraded":"2023-09-10T00:39:16.526550+0000","last_fullsized":"2023-09-10T00:39:16.526550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:05:48.223951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522271+0000","last_change":"2023-09-10T00:38:43.789089+0000","last_active":"2023-09-10T00:39:16.522271+0000","last_peered":"2023-09-10T00:39:16.522271+0000","last_clean":"2023-09-10T00:39:16.522271+0000","last_became_active":"2023-09-10T00:38:43.788434+0000","last_became_peered":"2023-09-10T00:38:43.788434+0000","last_unstale":"2023-09-10T00:39:16.522271+0000","last_undegraded":"2023-09-10T00:39:16.522271+0000","last_fullsized":"2023-09-10T00:39:16.522271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:55:00.236088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.523380+0000","last_change":"2023-09-10T00:38:38.769478+0000","last_active":"2023-09-10T00:39:16.523380+0000","last_peered":"2023-09-10T00:39:16.523380+0000","last_clean":"2023-09-10T00:39:16.523380+0000","last_became_active":"2023-09-10T00:38:38.769168+0000","last_became_peered":"2023-09-10T00:38:38.769168+0000","last_unstale":"2023-09-10T00:39:16.523380+0000","last_undegraded":"2023-09-10T00:39:16.523380+0000","last_fullsized":"2023-09-10T00:39:16.523380+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:03:48.133561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.525752+0000","last_change":"2023-09-10T00:38:39.771843+0000","last_active":"2023-09-10T00:39:16.525752+0000","last_peered":"2023-09-10T00:39:16.525752+0000","last_clean":"2023-09-10T00:39:16.525752+0000","last_became_active":"2023-09-10T00:38:39.771510+0000","last_became_peered":"2023-09-10T00:38:39.771510+0000","last_unstale":"2023-09-10T00:39:16.525752+0000","last_undegraded":"2023-09-10T00:39:16.525752+0000","last_fullsized":"2023-09-10T00:39:16.525752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:17:39.996490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527399+0000","last_change":"2023-09-10T00:38:43.786853+0000","last_active":"2023-09-10T00:39:16.527399+0000","last_peered":"2023-09-10T00:39:16.527399+0000","last_clean":"2023-09-10T00:39:16.527399+0000","last_became_active":"2023-09-10T00:38:43.786386+0000","last_became_peered":"2023-09-10T00:38:43.786386+0000","last_unstale":"2023-09-10T00:39:16.527399+0000","last_undegraded":"2023-09-10T00:39:16.527399+0000","last_fullsized":"2023-09-10T00:39:16.527399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:23:53.724653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":68,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:38.529849+0000","last_change":"2023-09-10T00:38:38.769004+0000","last_active":"2023-09-10T00:39:38.529849+0000","last_peered":"2023-09-10T00:39:38.529849+0000","last_clean":"2023-09-10T00:39:38.529849+0000","last_became_active":"2023-09-10T00:38:38.768690+0000","last_became_peered":"2023-09-10T00:38:38.768690+0000","last_unstale":"2023-09-10T00:39:38.529849+0000","last_undegraded":"2023-09-10T00:39:38.529849+0000","last_fullsized":"2023-09-10T00:39:38.529849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:04:42.531092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.523666+0000","last_change":"2023-09-10T00:38:39.773174+0000","last_active":"2023-09-10T00:39:16.523666+0000","last_peered":"2023-09-10T00:39:16.523666+0000","last_clean":"2023-09-10T00:39:16.523666+0000","last_became_active":"2023-09-10T00:38:39.771192+0000","last_became_peered":"2023-09-10T00:38:39.771192+0000","last_unstale":"2023-09-10T00:39:16.523666+0000","last_undegraded":"2023-09-10T00:39:16.523666+0000","last_fullsized":"2023-09-10T00:39:16.523666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:16:25.034024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527162+0000","last_change":"2023-09-10T00:38:43.786661+0000","last_active":"2023-09-10T00:39:16.527162+0000","last_peered":"2023-09-10T00:39:16.527162+0000","last_clean":"2023-09-10T00:39:16.527162+0000","last_became_active":"2023-09-10T00:38:43.786242+0000","last_became_peered":"2023-09-10T00:38:43.786242+0000","last_unstale":"2023-09-10T00:39:16.527162+0000","last_undegraded":"2023-09-10T00:39:16.527162+0000","last_fullsized":"2023-09-10T00:39:16.527162+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:36:14.490085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526572+0000","last_change":"2023-09-10T00:38:38.763416+0000","last_active":"2023-09-10T00:39:16.526572+0000","last_peered":"2023-09-10T00:39:16.526572+0000","last_clean":"2023-09-10T00:39:16.526572+0000","last_became_active":"2023-09-10T00:38:38.763054+0000","last_became_peered":"2023-09-10T00:38:38.763054+0000","last_unstale":"2023-09-10T00:39:16.526572+0000","last_undegraded":"2023-09-10T00:39:16.526572+0000","last_fullsized":"2023-09-10T00:39:16.526572+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:47:00.179467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.523311+0000","last_change":"2023-09-10T00:38:39.773284+0000","last_active":"2023-09-10T00:39:16.523311+0000","last_peered":"2023-09-10T00:39:16.523311+0000","last_clean":"2023-09-10T00:39:16.523311+0000","last_became_active":"2023-09-10T00:38:39.772969+0000","last_became_peered":"2023-09-10T00:38:39.772969+0000","last_unstale":"2023-09-10T00:39:16.523311+0000","last_undegraded":"2023-09-10T00:39:16.523311+0000","last_fullsized":"2023-09-10T00:39:16.523311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:46:45.013879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.519914+0000","last_change":"2023-09-10T00:38:43.790810+0000","last_active":"2023-09-10T00:39:16.519914+0000","last_peered":"2023-09-10T00:39:16.519914+0000","last_clean":"2023-09-10T00:39:16.519914+0000","last_became_active":"2023-09-10T00:38:43.790429+0000","last_became_peered":"2023-09-10T00:38:43.790429+0000","last_unstale":"2023-09-10T00:39:16.519914+0000","last_undegraded":"2023-09-10T00:39:16.519914+0000","last_fullsized":"2023-09-10T00:39:16.519914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:30:56.072587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.553599+0000","last_change":"2023-09-10T00:38:38.754326+0000","last_active":"2023-09-10T00:39:16.553599+0000","last_peered":"2023-09-10T00:39:16.553599+0000","last_clean":"2023-09-10T00:39:16.553599+0000","last_became_active":"2023-09-10T00:38:38.754006+0000","last_became_peered":"2023-09-10T00:38:38.754006+0000","last_unstale":"2023-09-10T00:39:16.553599+0000","last_undegraded":"2023-09-10T00:39:16.553599+0000","last_fullsized":"2023-09-10T00:39:16.553599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:09:26.983606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.536700+0000","last_change":"2023-09-10T00:38:39.769699+0000","last_active":"2023-09-10T00:39:16.536700+0000","last_peered":"2023-09-10T00:39:16.536700+0000","last_clean":"2023-09-10T00:39:16.536700+0000","last_became_active":"2023-09-10T00:38:39.769362+0000","last_became_peered":"2023-09-10T00:38:39.769362+0000","last_unstale":"2023-09-10T00:39:16.536700+0000","last_undegraded":"2023-09-10T00:39:16.536700+0000","last_fullsized":"2023-09-10T00:39:16.536700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:30:20.938437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.528806+0000","last_change":"2023-09-10T00:38:43.785669+0000","last_active":"2023-09-10T00:39:16.528806+0000","last_peered":"2023-09-10T00:39:16.528806+0000","last_clean":"2023-09-10T00:39:16.528806+0000","last_became_active":"2023-09-10T00:38:43.785255+0000","last_became_peered":"2023-09-10T00:38:43.785255+0000","last_unstale":"2023-09-10T00:39:16.528806+0000","last_undegraded":"2023-09-10T00:39:16.528806+0000","last_fullsized":"2023-09-10T00:39:16.528806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:52:57.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522146+0000","last_change":"2023-09-10T00:38:38.767997+0000","last_active":"2023-09-10T00:39:16.522146+0000","last_peered":"2023-09-10T00:39:16.522146+0000","last_clean":"2023-09-10T00:39:16.522146+0000","last_became_active":"2023-09-10T00:38:38.767715+0000","last_became_peered":"2023-09-10T00:38:38.767715+0000","last_unstale":"2023-09-10T00:39:16.522146+0000","last_undegraded":"2023-09-10T00:39:16.522146+0000","last_fullsized":"2023-09-10T00:39:16.522146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:11:17.047280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527273+0000","last_change":"2023-09-10T00:38:39.778004+0000","last_active":"2023-09-10T00:39:16.527273+0000","last_peered":"2023-09-10T00:39:16.527273+0000","last_clean":"2023-09-10T00:39:16.527273+0000","last_became_active":"2023-09-10T00:38:39.777632+0000","last_became_peered":"2023-09-10T00:38:39.777632+0000","last_unstale":"2023-09-10T00:39:16.527273+0000","last_undegraded":"2023-09-10T00:39:16.527273+0000","last_fullsized":"2023-09-10T00:39:16.527273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:46:40.804385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.533338+0000","last_change":"2023-09-10T00:38:43.791397+0000","last_active":"2023-09-10T00:39:16.533338+0000","last_peered":"2023-09-10T00:39:16.533338+0000","last_clean":"2023-09-10T00:39:16.533338+0000","last_became_active":"2023-09-10T00:38:43.791010+0000","last_became_peered":"2023-09-10T00:38:43.791010+0000","last_unstale":"2023-09-10T00:39:16.533338+0000","last_undegraded":"2023-09-10T00:39:16.533338+0000","last_fullsized":"2023-09-10T00:39:16.533338+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:52:26.015718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":100,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:39.530817+0000","last_change":"2023-09-10T00:38:38.767399+0000","last_active":"2023-09-10T00:39:39.530817+0000","last_peered":"2023-09-10T00:39:39.530817+0000","last_clean":"2023-09-10T00:39:39.530817+0000","last_became_active":"2023-09-10T00:38:38.767176+0000","last_became_peered":"2023-09-10T00:38:38.767176+0000","last_unstale":"2023-09-10T00:39:39.530817+0000","last_undegraded":"2023-09-10T00:39:39.530817+0000","last_fullsized":"2023-09-10T00:39:39.530817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:17:24.481985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.525581+0000","last_change":"2023-09-10T00:38:39.766013+0000","last_active":"2023-09-10T00:39:16.525581+0000","last_peered":"2023-09-10T00:39:16.525581+0000","last_clean":"2023-09-10T00:39:16.525581+0000","last_became_active":"2023-09-10T00:38:39.765235+0000","last_became_peered":"2023-09-10T00:38:39.765235+0000","last_unstale":"2023-09-10T00:39:16.525581+0000","last_undegraded":"2023-09-10T00:39:16.525581+0000","last_fullsized":"2023-09-10T00:39:16.525581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:01:48.874845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526918+0000","last_change":"2023-09-10T00:38:43.790361+0000","last_active":"2023-09-10T00:39:16.526918+0000","last_peered":"2023-09-10T00:39:16.526918+0000","last_clean":"2023-09-10T00:39:16.526918+0000","last_became_active":"2023-09-10T00:38:43.789966+0000","last_became_peered":"2023-09-10T00:38:43.789966+0000","last_unstale":"2023-09-10T00:39:16.526918+0000","last_undegraded":"2023-09-10T00:39:16.526918+0000","last_fullsized":"2023-09-10T00:39:16.526918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:46:37.978258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527984+0000","last_change":"2023-09-10T00:38:38.771670+0000","last_active":"2023-09-10T00:39:16.527984+0000","last_peered":"2023-09-10T00:39:16.527984+0000","last_clean":"2023-09-10T00:39:16.527984+0000","last_became_active":"2023-09-10T00:38:38.771339+0000","last_became_peered":"2023-09-10T00:38:38.771339+0000","last_unstale":"2023-09-10T00:39:16.527984+0000","last_undegraded":"2023-09-10T00:39:16.527984+0000","last_fullsized":"2023-09-10T00:39:16.527984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:22:32.658544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.528315+0000","last_change":"2023-09-10T00:38:39.772857+0000","last_active":"2023-09-10T00:39:16.528315+0000","last_peered":"2023-09-10T00:39:16.528315+0000","last_clean":"2023-09-10T00:39:16.528315+0000","last_became_active":"2023-09-10T00:38:39.771990+0000","last_became_peered":"2023-09-10T00:38:39.771990+0000","last_unstale":"2023-09-10T00:39:16.528315+0000","last_undegraded":"2023-09-10T00:39:16.528315+0000","last_fullsized":"2023-09-10T00:39:16.528315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:16:15.767748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522691+0000","last_change":"2023-09-10T00:38:43.795482+0000","last_active":"2023-09-10T00:39:16.522691+0000","last_peered":"2023-09-10T00:39:16.522691+0000","last_clean":"2023-09-10T00:39:16.522691+0000","last_became_active":"2023-09-10T00:38:43.795048+0000","last_became_peered":"2023-09-10T00:38:43.795048+0000","last_unstale":"2023-09-10T00:39:16.522691+0000","last_undegraded":"2023-09-10T00:39:16.522691+0000","last_fullsized":"2023-09-10T00:39:16.522691+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:13:18.715274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.523679+0000","last_change":"2023-09-10T00:38:38.771008+0000","last_active":"2023-09-10T00:39:16.523679+0000","last_peered":"2023-09-10T00:39:16.523679+0000","last_clean":"2023-09-10T00:39:16.523679+0000","last_became_active":"2023-09-10T00:38:38.770714+0000","last_became_peered":"2023-09-10T00:38:38.770714+0000","last_unstale":"2023-09-10T00:39:16.523679+0000","last_undegraded":"2023-09-10T00:39:16.523679+0000","last_fullsized":"2023-09-10T00:39:16.523679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:04:30.148141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.533500+0000","last_change":"2023-09-10T00:39:17.533500+0000","last_active":"2023-09-10T00:39:17.533500+0000","last_peered":"2023-09-10T00:39:17.533500+0000","last_clean":"2023-09-10T00:39:17.533500+0000","last_became_active":"2023-09-10T00:39:17.533175+0000","last_became_peered":"2023-09-10T00:39:17.533175+0000","last_unstale":"2023-09-10T00:39:17.533500+0000","last_undegraded":"2023-09-10T00:39:17.533500+0000","last_fullsized":"2023-09-10T00:39:17.533500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:13:42.121699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.537288+0000","last_change":"2023-09-10T00:39:17.536684+0000","last_active":"2023-09-10T00:39:17.537288+0000","last_peered":"2023-09-10T00:39:17.537288+0000","last_clean":"2023-09-10T00:39:17.537288+0000","last_became_active":"2023-09-10T00:39:17.536170+0000","last_became_peered":"2023-09-10T00:39:17.536170+0000","last_unstale":"2023-09-10T00:39:17.537288+0000","last_undegraded":"2023-09-10T00:39:17.537288+0000","last_fullsized":"2023-09-10T00:39:17.537288+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:47:16.264619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":139,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:39.530802+0000","last_change":"2023-09-10T00:38:38.761072+0000","last_active":"2023-09-10T00:39:39.530802+0000","last_peered":"2023-09-10T00:39:39.530802+0000","last_clean":"2023-09-10T00:39:39.530802+0000","last_became_active":"2023-09-10T00:38:38.760752+0000","last_became_peered":"2023-09-10T00:38:38.760752+0000","last_unstale":"2023-09-10T00:39:39.530802+0000","last_undegraded":"2023-09-10T00:39:39.530802+0000","last_fullsized":"2023-09-10T00:39:39.530802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:55:18.272741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58599,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":88,"num_read_kb":101,"num_write":35,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.523956+0000","last_change":"2023-09-10T00:38:39.767507+0000","last_active":"2023-09-10T00:39:16.523956+0000","last_peered":"2023-09-10T00:39:16.523956+0000","last_clean":"2023-09-10T00:39:16.523956+0000","last_became_active":"2023-09-10T00:38:39.767223+0000","last_became_peered":"2023-09-10T00:38:39.767223+0000","last_unstale":"2023-09-10T00:39:16.523956+0000","last_undegraded":"2023-09-10T00:39:16.523956+0000","last_fullsized":"2023-09-10T00:39:16.523956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:46:30.901818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526139+0000","last_change":"2023-09-10T00:38:43.785396+0000","last_active":"2023-09-10T00:39:16.526139+0000","last_peered":"2023-09-10T00:39:16.526139+0000","last_clean":"2023-09-10T00:39:16.526139+0000","last_became_active":"2023-09-10T00:38:43.785038+0000","last_became_peered":"2023-09-10T00:38:43.785038+0000","last_unstale":"2023-09-10T00:39:16.526139+0000","last_undegraded":"2023-09-10T00:39:16.526139+0000","last_fullsized":"2023-09-10T00:39:16.526139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:30:31.629703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527248+0000","last_change":"2023-09-10T00:38:38.766742+0000","last_active":"2023-09-10T00:39:16.527248+0000","last_peered":"2023-09-10T00:39:16.527248+0000","last_clean":"2023-09-10T00:39:16.527248+0000","last_became_active":"2023-09-10T00:38:38.766220+0000","last_became_peered":"2023-09-10T00:38:38.766220+0000","last_unstale":"2023-09-10T00:39:16.527248+0000","last_undegraded":"2023-09-10T00:39:16.527248+0000","last_fullsized":"2023-09-10T00:39:16.527248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:41:22.030448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.537227+0000","last_change":"2023-09-10T00:38:39.766022+0000","last_active":"2023-09-10T00:39:16.537227+0000","last_peered":"2023-09-10T00:39:16.537227+0000","last_clean":"2023-09-10T00:39:16.537227+0000","last_became_active":"2023-09-10T00:38:39.765697+0000","last_became_peered":"2023-09-10T00:38:39.765697+0000","last_unstale":"2023-09-10T00:39:16.537227+0000","last_undegraded":"2023-09-10T00:39:16.537227+0000","last_fullsized":"2023-09-10T00:39:16.537227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:19:39.394299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.519687+0000","last_change":"2023-09-10T00:38:43.795492+0000","last_active":"2023-09-10T00:39:16.519687+0000","last_peered":"2023-09-10T00:39:16.519687+0000","last_clean":"2023-09-10T00:39:16.519687+0000","last_became_active":"2023-09-10T00:38:43.794968+0000","last_became_peered":"2023-09-10T00:38:43.794968+0000","last_unstale":"2023-09-10T00:39:16.519687+0000","last_undegraded":"2023-09-10T00:39:16.519687+0000","last_fullsized":"2023-09-10T00:39:16.519687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:40:43.558870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522912+0000","last_change":"2023-09-10T00:38:38.767666+0000","last_active":"2023-09-10T00:39:16.522912+0000","last_peered":"2023-09-10T00:39:16.522912+0000","last_clean":"2023-09-10T00:39:16.522912+0000","last_became_active":"2023-09-10T00:38:38.767333+0000","last_became_peered":"2023-09-10T00:38:38.767333+0000","last_unstale":"2023-09-10T00:39:16.522912+0000","last_undegraded":"2023-09-10T00:39:16.522912+0000","last_fullsized":"2023-09-10T00:39:16.522912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:07:53.363196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527421+0000","last_change":"2023-09-10T00:38:39.766036+0000","last_active":"2023-09-10T00:39:16.527421+0000","last_peered":"2023-09-10T00:39:16.527421+0000","last_clean":"2023-09-10T00:39:16.527421+0000","last_became_active":"2023-09-10T00:38:39.765683+0000","last_became_peered":"2023-09-10T00:38:39.765683+0000","last_unstale":"2023-09-10T00:39:16.527421+0000","last_undegraded":"2023-09-10T00:39:16.527421+0000","last_fullsized":"2023-09-10T00:39:16.527421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:58:04.429960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.532739+0000","last_change":"2023-09-10T00:38:43.787870+0000","last_active":"2023-09-10T00:39:16.532739+0000","last_peered":"2023-09-10T00:39:16.532739+0000","last_clean":"2023-09-10T00:39:16.532739+0000","last_became_active":"2023-09-10T00:38:43.785915+0000","last_became_peered":"2023-09-10T00:38:43.785915+0000","last_unstale":"2023-09-10T00:39:16.532739+0000","last_undegraded":"2023-09-10T00:39:16.532739+0000","last_fullsized":"2023-09-10T00:39:16.532739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:34:08.623967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":65,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:36.532596+0000","last_change":"2023-09-10T00:38:38.756925+0000","last_active":"2023-09-10T00:39:36.532596+0000","last_peered":"2023-09-10T00:39:36.532596+0000","last_clean":"2023-09-10T00:39:36.532596+0000","last_became_active":"2023-09-10T00:38:38.756610+0000","last_became_peered":"2023-09-10T00:38:38.756610+0000","last_unstale":"2023-09-10T00:39:36.532596+0000","last_undegraded":"2023-09-10T00:39:36.532596+0000","last_fullsized":"2023-09-10T00:39:36.532596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:48:15.385230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526743+0000","last_change":"2023-09-10T00:38:39.766750+0000","last_active":"2023-09-10T00:39:16.526743+0000","last_peered":"2023-09-10T00:39:16.526743+0000","last_clean":"2023-09-10T00:39:16.526743+0000","last_became_active":"2023-09-10T00:38:39.766458+0000","last_became_peered":"2023-09-10T00:38:39.766458+0000","last_unstale":"2023-09-10T00:39:16.526743+0000","last_undegraded":"2023-09-10T00:39:16.526743+0000","last_fullsized":"2023-09-10T00:39:16.526743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:59:26.691487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.538243+0000","last_change":"2023-09-10T00:38:43.788814+0000","last_active":"2023-09-10T00:39:16.538243+0000","last_peered":"2023-09-10T00:39:16.538243+0000","last_clean":"2023-09-10T00:39:16.538243+0000","last_became_active":"2023-09-10T00:38:43.788446+0000","last_became_peered":"2023-09-10T00:38:43.788446+0000","last_unstale":"2023-09-10T00:39:16.538243+0000","last_undegraded":"2023-09-10T00:39:16.538243+0000","last_fullsized":"2023-09-10T00:39:16.538243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:38:53.308160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522627+0000","last_change":"2023-09-10T00:38:38.766020+0000","last_active":"2023-09-10T00:39:16.522627+0000","last_peered":"2023-09-10T00:39:16.522627+0000","last_clean":"2023-09-10T00:39:16.522627+0000","last_became_active":"2023-09-10T00:38:38.765719+0000","last_became_peered":"2023-09-10T00:38:38.765719+0000","last_unstale":"2023-09-10T00:39:16.522627+0000","last_undegraded":"2023-09-10T00:39:16.522627+0000","last_fullsized":"2023-09-10T00:39:16.522627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:25:30.488014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522546+0000","last_change":"2023-09-10T00:38:39.771091+0000","last_active":"2023-09-10T00:39:16.522546+0000","last_peered":"2023-09-10T00:39:16.522546+0000","last_clean":"2023-09-10T00:39:16.522546+0000","last_became_active":"2023-09-10T00:38:39.770745+0000","last_became_peered":"2023-09-10T00:38:39.770745+0000","last_unstale":"2023-09-10T00:39:16.522546+0000","last_undegraded":"2023-09-10T00:39:16.522546+0000","last_fullsized":"2023-09-10T00:39:16.522546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:51:17.978466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522440+0000","last_change":"2023-09-10T00:38:43.793757+0000","last_active":"2023-09-10T00:39:16.522440+0000","last_peered":"2023-09-10T00:39:16.522440+0000","last_clean":"2023-09-10T00:39:16.522440+0000","last_became_active":"2023-09-10T00:38:43.793336+0000","last_became_peered":"2023-09-10T00:38:43.793336+0000","last_unstale":"2023-09-10T00:39:16.522440+0000","last_undegraded":"2023-09-10T00:39:16.522440+0000","last_fullsized":"2023-09-10T00:39:16.522440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:22:36.544950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.525478+0000","last_change":"2023-09-10T00:38:38.769373+0000","last_active":"2023-09-10T00:39:16.525478+0000","last_peered":"2023-09-10T00:39:16.525478+0000","last_clean":"2023-09-10T00:39:16.525478+0000","last_became_active":"2023-09-10T00:38:38.769054+0000","last_became_peered":"2023-09-10T00:38:38.769054+0000","last_unstale":"2023-09-10T00:39:16.525478+0000","last_undegraded":"2023-09-10T00:39:16.525478+0000","last_fullsized":"2023-09-10T00:39:16.525478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:12:27.982405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.537427+0000","last_change":"2023-09-10T00:39:17.537427+0000","last_active":"2023-09-10T00:39:17.537427+0000","last_peered":"2023-09-10T00:39:17.537427+0000","last_clean":"2023-09-10T00:39:17.537427+0000","last_became_active":"2023-09-10T00:39:17.537023+0000","last_became_peered":"2023-09-10T00:39:17.537023+0000","last_unstale":"2023-09-10T00:39:17.537427+0000","last_undegraded":"2023-09-10T00:39:17.537427+0000","last_fullsized":"2023-09-10T00:39:17.537427+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:54:43.720224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.523517+0000","last_change":"2023-09-10T00:38:43.838689+0000","last_active":"2023-09-10T00:39:16.523517+0000","last_peered":"2023-09-10T00:39:16.523517+0000","last_clean":"2023-09-10T00:39:16.523517+0000","last_became_active":"2023-09-10T00:38:43.838244+0000","last_became_peered":"2023-09-10T00:38:43.838244+0000","last_unstale":"2023-09-10T00:39:16.523517+0000","last_undegraded":"2023-09-10T00:39:16.523517+0000","last_fullsized":"2023-09-10T00:39:16.523517+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:54:00.033719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.528925+0000","last_change":"2023-09-10T00:38:38.765140+0000","last_active":"2023-09-10T00:39:16.528925+0000","last_peered":"2023-09-10T00:39:16.528925+0000","last_clean":"2023-09-10T00:39:16.528925+0000","last_became_active":"2023-09-10T00:38:38.764730+0000","last_became_peered":"2023-09-10T00:38:38.764730+0000","last_unstale":"2023-09-10T00:39:16.528925+0000","last_undegraded":"2023-09-10T00:39:16.528925+0000","last_fullsized":"2023-09-10T00:39:16.528925+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:49:41.129763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526914+0000","last_change":"2023-09-10T00:38:39.773260+0000","last_active":"2023-09-10T00:39:16.526914+0000","last_peered":"2023-09-10T00:39:16.526914+0000","last_clean":"2023-09-10T00:39:16.526914+0000","last_became_active":"2023-09-10T00:38:39.772947+0000","last_became_peered":"2023-09-10T00:38:39.772947+0000","last_unstale":"2023-09-10T00:39:16.526914+0000","last_undegraded":"2023-09-10T00:39:16.526914+0000","last_fullsized":"2023-09-10T00:39:16.526914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:08:10.791610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.537466+0000","last_change":"2023-09-10T00:38:43.787882+0000","last_active":"2023-09-10T00:39:16.537466+0000","last_peered":"2023-09-10T00:39:16.537466+0000","last_clean":"2023-09-10T00:39:16.537466+0000","last_became_active":"2023-09-10T00:38:43.787257+0000","last_became_peered":"2023-09-10T00:38:43.787257+0000","last_unstale":"2023-09-10T00:39:16.537466+0000","last_undegraded":"2023-09-10T00:39:16.537466+0000","last_fullsized":"2023-09-10T00:39:16.537466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:11:19.171881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":65,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:40.592452+0000","last_change":"2023-09-10T00:38:38.760014+0000","last_active":"2023-09-10T00:39:40.592452+0000","last_peered":"2023-09-10T00:39:40.592452+0000","last_clean":"2023-09-10T00:39:40.592452+0000","last_became_active":"2023-09-10T00:38:38.759637+0000","last_became_peered":"2023-09-10T00:38:38.759637+0000","last_unstale":"2023-09-10T00:39:40.592452+0000","last_undegraded":"2023-09-10T00:39:40.592452+0000","last_fullsized":"2023-09-10T00:39:40.592452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:15.704929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527234+0000","last_change":"2023-09-10T00:38:39.765800+0000","last_active":"2023-09-10T00:39:16.527234+0000","last_peered":"2023-09-10T00:39:16.527234+0000","last_clean":"2023-09-10T00:39:16.527234+0000","last_became_active":"2023-09-10T00:38:39.765468+0000","last_became_peered":"2023-09-10T00:38:39.765468+0000","last_unstale":"2023-09-10T00:39:16.527234+0000","last_undegraded":"2023-09-10T00:39:16.527234+0000","last_fullsized":"2023-09-10T00:39:16.527234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:57:56.429365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.538360+0000","last_change":"2023-09-10T00:38:43.838285+0000","last_active":"2023-09-10T00:39:16.538360+0000","last_peered":"2023-09-10T00:39:16.538360+0000","last_clean":"2023-09-10T00:39:16.538360+0000","last_became_active":"2023-09-10T00:38:43.837821+0000","last_became_peered":"2023-09-10T00:38:43.837821+0000","last_unstale":"2023-09-10T00:39:16.538360+0000","last_undegraded":"2023-09-10T00:39:16.538360+0000","last_fullsized":"2023-09-10T00:39:16.538360+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:01:03.354281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.536361+0000","last_change":"2023-09-10T00:39:17.536361+0000","last_active":"2023-09-10T00:39:17.536361+0000","last_peered":"2023-09-10T00:39:17.536361+0000","last_clean":"2023-09-10T00:39:17.536361+0000","last_became_active":"2023-09-10T00:39:17.535930+0000","last_became_peered":"2023-09-10T00:39:17.535930+0000","last_unstale":"2023-09-10T00:39:17.536361+0000","last_undegraded":"2023-09-10T00:39:17.536361+0000","last_fullsized":"2023-09-10T00:39:17.536361+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:53.576115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":66,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:39.536268+0000","last_change":"2023-09-10T00:38:38.758745+0000","last_active":"2023-09-10T00:39:39.536268+0000","last_peered":"2023-09-10T00:39:39.536268+0000","last_clean":"2023-09-10T00:39:39.536268+0000","last_became_active":"2023-09-10T00:38:38.758427+0000","last_became_peered":"2023-09-10T00:38:38.758427+0000","last_unstale":"2023-09-10T00:39:39.536268+0000","last_undegraded":"2023-09-10T00:39:39.536268+0000","last_fullsized":"2023-09-10T00:39:39.536268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:02:49.238175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527226+0000","last_change":"2023-09-10T00:38:43.784058+0000","last_active":"2023-09-10T00:39:16.527226+0000","last_peered":"2023-09-10T00:39:16.527226+0000","last_clean":"2023-09-10T00:39:16.527226+0000","last_became_active":"2023-09-10T00:38:43.783571+0000","last_became_peered":"2023-09-10T00:38:43.783571+0000","last_unstale":"2023-09-10T00:39:16.527226+0000","last_undegraded":"2023-09-10T00:39:16.527226+0000","last_fullsized":"2023-09-10T00:39:16.527226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:53:50.992393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.533823+0000","last_change":"2023-09-10T00:38:39.775659+0000","last_active":"2023-09-10T00:39:16.533823+0000","last_peered":"2023-09-10T00:39:16.533823+0000","last_clean":"2023-09-10T00:39:16.533823+0000","last_became_active":"2023-09-10T00:38:39.775151+0000","last_became_peered":"2023-09-10T00:38:39.775151+0000","last_unstale":"2023-09-10T00:39:16.533823+0000","last_undegraded":"2023-09-10T00:39:16.533823+0000","last_fullsized":"2023-09-10T00:39:16.533823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:07.415418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522562+0000","last_change":"2023-09-10T00:38:38.755009+0000","last_active":"2023-09-10T00:39:16.522562+0000","last_peered":"2023-09-10T00:39:16.522562+0000","last_clean":"2023-09-10T00:39:16.522562+0000","last_became_active":"2023-09-10T00:38:38.754699+0000","last_became_peered":"2023-09-10T00:38:38.754699+0000","last_unstale":"2023-09-10T00:39:16.522562+0000","last_undegraded":"2023-09-10T00:39:16.522562+0000","last_fullsized":"2023-09-10T00:39:16.522562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:12:02.075256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":322,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526552+0000","last_change":"2023-09-10T00:37:38.459695+0000","last_active":"2023-09-10T00:39:16.526552+0000","last_peered":"2023-09-10T00:39:16.526552+0000","last_clean":"2023-09-10T00:39:16.526552+0000","last_became_active":"2023-09-10T00:37:38.459220+0000","last_became_peered":"2023-09-10T00:37:38.459220+0000","last_unstale":"2023-09-10T00:39:16.526552+0000","last_undegraded":"2023-09-10T00:39:16.526552+0000","last_fullsized":"2023-09-10T00:39:16.526552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:34:44.602334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:34:44.602334+0000","last_clean_scrub_stamp":"2023-09-10T00:34:44.602334+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:21:32.648725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526212+0000","last_change":"2023-09-10T00:38:43.839001+0000","last_active":"2023-09-10T00:39:16.526212+0000","last_peered":"2023-09-10T00:39:16.526212+0000","last_clean":"2023-09-10T00:39:16.526212+0000","last_became_active":"2023-09-10T00:38:43.838497+0000","last_became_peered":"2023-09-10T00:38:43.838497+0000","last_unstale":"2023-09-10T00:39:16.526212+0000","last_undegraded":"2023-09-10T00:39:16.526212+0000","last_fullsized":"2023-09-10T00:39:16.526212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:57:23.349632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.553332+0000","last_change":"2023-09-10T00:38:39.760150+0000","last_active":"2023-09-10T00:39:16.553332+0000","last_peered":"2023-09-10T00:39:16.553332+0000","last_clean":"2023-09-10T00:39:16.553332+0000","last_became_active":"2023-09-10T00:38:39.759811+0000","last_became_peered":"2023-09-10T00:38:39.759811+0000","last_unstale":"2023-09-10T00:39:16.553332+0000","last_undegraded":"2023-09-10T00:39:16.553332+0000","last_fullsized":"2023-09-10T00:39:16.553332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:21:19.359057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526651+0000","last_change":"2023-09-10T00:38:38.771066+0000","last_active":"2023-09-10T00:39:16.526651+0000","last_peered":"2023-09-10T00:39:16.526651+0000","last_clean":"2023-09-10T00:39:16.526651+0000","last_became_active":"2023-09-10T00:38:38.770810+0000","last_became_peered":"2023-09-10T00:38:38.770810+0000","last_unstale":"2023-09-10T00:39:16.526651+0000","last_undegraded":"2023-09-10T00:39:16.526651+0000","last_fullsized":"2023-09-10T00:39:16.526651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:55:06.923842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.519639+0000","last_change":"2023-09-10T00:38:43.786272+0000","last_active":"2023-09-10T00:39:16.519639+0000","last_peered":"2023-09-10T00:39:16.519639+0000","last_clean":"2023-09-10T00:39:16.519639+0000","last_became_active":"2023-09-10T00:38:43.785934+0000","last_became_peered":"2023-09-10T00:38:43.785934+0000","last_unstale":"2023-09-10T00:39:16.519639+0000","last_undegraded":"2023-09-10T00:39:16.519639+0000","last_fullsized":"2023-09-10T00:39:16.519639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:31:49.721133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.524061+0000","last_change":"2023-09-10T00:38:38.763263+0000","last_active":"2023-09-10T00:39:16.524061+0000","last_peered":"2023-09-10T00:39:16.524061+0000","last_clean":"2023-09-10T00:39:16.524061+0000","last_became_active":"2023-09-10T00:38:38.762911+0000","last_became_peered":"2023-09-10T00:38:38.762911+0000","last_unstale":"2023-09-10T00:39:16.524061+0000","last_undegraded":"2023-09-10T00:39:16.524061+0000","last_fullsized":"2023-09-10T00:39:16.524061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:19:10.292311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.537610+0000","last_change":"2023-09-10T00:38:39.759171+0000","last_active":"2023-09-10T00:39:16.537610+0000","last_peered":"2023-09-10T00:39:16.537610+0000","last_clean":"2023-09-10T00:39:16.537610+0000","last_became_active":"2023-09-10T00:38:39.758843+0000","last_became_peered":"2023-09-10T00:38:39.758843+0000","last_unstale":"2023-09-10T00:39:16.537610+0000","last_undegraded":"2023-09-10T00:39:16.537610+0000","last_fullsized":"2023-09-10T00:39:16.537610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:50:36.064937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.536606+0000","last_change":"2023-09-10T00:38:43.837998+0000","last_active":"2023-09-10T00:39:16.536606+0000","last_peered":"2023-09-10T00:39:16.536606+0000","last_clean":"2023-09-10T00:39:16.536606+0000","last_became_active":"2023-09-10T00:38:43.837556+0000","last_became_peered":"2023-09-10T00:38:43.837556+0000","last_unstale":"2023-09-10T00:39:16.536606+0000","last_undegraded":"2023-09-10T00:39:16.536606+0000","last_fullsized":"2023-09-10T00:39:16.536606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:18:40.488279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.532279+0000","last_change":"2023-09-10T00:38:38.759819+0000","last_active":"2023-09-10T00:39:16.532279+0000","last_peered":"2023-09-10T00:39:16.532279+0000","last_clean":"2023-09-10T00:39:16.532279+0000","last_became_active":"2023-09-10T00:38:38.759490+0000","last_became_peered":"2023-09-10T00:38:38.759490+0000","last_unstale":"2023-09-10T00:39:16.532279+0000","last_undegraded":"2023-09-10T00:39:16.532279+0000","last_fullsized":"2023-09-10T00:39:16.532279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:39:01.206817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.535116+0000","last_change":"2023-09-10T00:38:39.769669+0000","last_active":"2023-09-10T00:39:16.535116+0000","last_peered":"2023-09-10T00:39:16.535116+0000","last_clean":"2023-09-10T00:39:16.535116+0000","last_became_active":"2023-09-10T00:38:39.769402+0000","last_became_peered":"2023-09-10T00:38:39.769402+0000","last_unstale":"2023-09-10T00:39:16.535116+0000","last_undegraded":"2023-09-10T00:39:16.535116+0000","last_fullsized":"2023-09-10T00:39:16.535116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:00:50.215103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526606+0000","last_change":"2023-09-10T00:38:43.787518+0000","last_active":"2023-09-10T00:39:16.526606+0000","last_peered":"2023-09-10T00:39:16.526606+0000","last_clean":"2023-09-10T00:39:16.526606+0000","last_became_active":"2023-09-10T00:38:43.786410+0000","last_became_peered":"2023-09-10T00:38:43.786410+0000","last_unstale":"2023-09-10T00:39:16.526606+0000","last_undegraded":"2023-09-10T00:39:16.526606+0000","last_fullsized":"2023-09-10T00:39:16.526606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:13:37.903875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522600+0000","last_change":"2023-09-10T00:38:38.760992+0000","last_active":"2023-09-10T00:39:16.522600+0000","last_peered":"2023-09-10T00:39:16.522600+0000","last_clean":"2023-09-10T00:39:16.522600+0000","last_became_active":"2023-09-10T00:38:38.760635+0000","last_became_peered":"2023-09-10T00:38:38.760635+0000","last_unstale":"2023-09-10T00:39:16.522600+0000","last_undegraded":"2023-09-10T00:39:16.522600+0000","last_fullsized":"2023-09-10T00:39:16.522600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:29:36.070477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.528974+0000","last_change":"2023-09-10T00:38:39.773771+0000","last_active":"2023-09-10T00:39:16.528974+0000","last_peered":"2023-09-10T00:39:16.528974+0000","last_clean":"2023-09-10T00:39:16.528974+0000","last_became_active":"2023-09-10T00:38:39.773296+0000","last_became_peered":"2023-09-10T00:38:39.773296+0000","last_unstale":"2023-09-10T00:39:16.528974+0000","last_undegraded":"2023-09-10T00:39:16.528974+0000","last_fullsized":"2023-09-10T00:39:16.528974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:13:01.194358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.536853+0000","last_change":"2023-09-10T00:39:17.536853+0000","last_active":"2023-09-10T00:39:17.536853+0000","last_peered":"2023-09-10T00:39:17.536853+0000","last_clean":"2023-09-10T00:39:17.536853+0000","last_became_active":"2023-09-10T00:39:17.536287+0000","last_became_peered":"2023-09-10T00:39:17.536287+0000","last_unstale":"2023-09-10T00:39:17.536853+0000","last_undegraded":"2023-09-10T00:39:17.536853+0000","last_fullsized":"2023-09-10T00:39:17.536853+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:50:35.294055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.525550+0000","last_change":"2023-09-10T00:38:38.764954+0000","last_active":"2023-09-10T00:39:16.525550+0000","last_peered":"2023-09-10T00:39:16.525550+0000","last_clean":"2023-09-10T00:39:16.525550+0000","last_became_active":"2023-09-10T00:38:38.764589+0000","last_became_peered":"2023-09-10T00:38:38.764589+0000","last_unstale":"2023-09-10T00:39:16.525550+0000","last_undegraded":"2023-09-10T00:39:16.525550+0000","last_fullsized":"2023-09-10T00:39:16.525550+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:40:19.559285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522698+0000","last_change":"2023-09-10T00:38:39.766141+0000","last_active":"2023-09-10T00:39:16.522698+0000","last_peered":"2023-09-10T00:39:16.522698+0000","last_clean":"2023-09-10T00:39:16.522698+0000","last_became_active":"2023-09-10T00:38:39.765832+0000","last_became_peered":"2023-09-10T00:38:39.765832+0000","last_unstale":"2023-09-10T00:39:16.522698+0000","last_undegraded":"2023-09-10T00:39:16.522698+0000","last_fullsized":"2023-09-10T00:39:16.522698+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:08:45.149973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.532882+0000","last_change":"2023-09-10T00:38:43.785111+0000","last_active":"2023-09-10T00:39:16.532882+0000","last_peered":"2023-09-10T00:39:16.532882+0000","last_clean":"2023-09-10T00:39:16.532882+0000","last_became_active":"2023-09-10T00:38:43.784542+0000","last_became_peered":"2023-09-10T00:38:43.784542+0000","last_unstale":"2023-09-10T00:39:16.532882+0000","last_undegraded":"2023-09-10T00:39:16.532882+0000","last_fullsized":"2023-09-10T00:39:16.532882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:34:51.611608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.533431+0000","last_change":"2023-09-10T00:38:38.759984+0000","last_active":"2023-09-10T00:39:16.533431+0000","last_peered":"2023-09-10T00:39:16.533431+0000","last_clean":"2023-09-10T00:39:16.533431+0000","last_became_active":"2023-09-10T00:38:38.759660+0000","last_became_peered":"2023-09-10T00:38:38.759660+0000","last_unstale":"2023-09-10T00:39:16.533431+0000","last_undegraded":"2023-09-10T00:39:16.533431+0000","last_fullsized":"2023-09-10T00:39:16.533431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:02:11.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.519505+0000","last_change":"2023-09-10T00:38:39.770765+0000","last_active":"2023-09-10T00:39:16.519505+0000","last_peered":"2023-09-10T00:39:16.519505+0000","last_clean":"2023-09-10T00:39:16.519505+0000","last_became_active":"2023-09-10T00:38:39.770483+0000","last_became_peered":"2023-09-10T00:38:39.770483+0000","last_unstale":"2023-09-10T00:39:16.519505+0000","last_undegraded":"2023-09-10T00:39:16.519505+0000","last_fullsized":"2023-09-10T00:39:16.519505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:57:28.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522403+0000","last_change":"2023-09-10T00:38:43.790945+0000","last_active":"2023-09-10T00:39:16.522403+0000","last_peered":"2023-09-10T00:39:16.522403+0000","last_clean":"2023-09-10T00:39:16.522403+0000","last_became_active":"2023-09-10T00:38:43.790467+0000","last_became_peered":"2023-09-10T00:38:43.790467+0000","last_unstale":"2023-09-10T00:39:16.522403+0000","last_undegraded":"2023-09-10T00:39:16.522403+0000","last_fullsized":"2023-09-10T00:39:16.522403+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:44:07.041302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526857+0000","last_change":"2023-09-10T00:38:38.765912+0000","last_active":"2023-09-10T00:39:16.526857+0000","last_peered":"2023-09-10T00:39:16.526857+0000","last_clean":"2023-09-10T00:39:16.526857+0000","last_became_active":"2023-09-10T00:38:38.765586+0000","last_became_peered":"2023-09-10T00:38:38.765586+0000","last_unstale":"2023-09-10T00:39:16.526857+0000","last_undegraded":"2023-09-10T00:39:16.526857+0000","last_fullsized":"2023-09-10T00:39:16.526857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:25:05.974158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.533090+0000","last_change":"2023-09-10T00:38:39.771462+0000","last_active":"2023-09-10T00:39:16.533090+0000","last_peered":"2023-09-10T00:39:16.533090+0000","last_clean":"2023-09-10T00:39:16.533090+0000","last_became_active":"2023-09-10T00:38:39.771153+0000","last_became_peered":"2023-09-10T00:38:39.771153+0000","last_unstale":"2023-09-10T00:39:16.533090+0000","last_undegraded":"2023-09-10T00:39:16.533090+0000","last_fullsized":"2023-09-10T00:39:16.533090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:03:35.312186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522619+0000","last_change":"2023-09-10T00:38:43.838446+0000","last_active":"2023-09-10T00:39:16.522619+0000","last_peered":"2023-09-10T00:39:16.522619+0000","last_clean":"2023-09-10T00:39:16.522619+0000","last_became_active":"2023-09-10T00:38:43.838119+0000","last_became_peered":"2023-09-10T00:38:43.838119+0000","last_unstale":"2023-09-10T00:39:16.522619+0000","last_undegraded":"2023-09-10T00:39:16.522619+0000","last_fullsized":"2023-09-10T00:39:16.522619+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:12:20.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.524372+0000","last_change":"2023-09-10T00:38:38.765548+0000","last_active":"2023-09-10T00:39:16.524372+0000","last_peered":"2023-09-10T00:39:16.524372+0000","last_clean":"2023-09-10T00:39:16.524372+0000","last_became_active":"2023-09-10T00:38:38.765281+0000","last_became_peered":"2023-09-10T00:38:38.765281+0000","last_unstale":"2023-09-10T00:39:16.524372+0000","last_undegraded":"2023-09-10T00:39:16.524372+0000","last_fullsized":"2023-09-10T00:39:16.524372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:50:44.422327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527816+0000","last_change":"2023-09-10T00:38:39.771995+0000","last_active":"2023-09-10T00:39:16.527816+0000","last_peered":"2023-09-10T00:39:16.527816+0000","last_clean":"2023-09-10T00:39:16.527816+0000","last_became_active":"2023-09-10T00:38:39.771693+0000","last_became_peered":"2023-09-10T00:38:39.771693+0000","last_unstale":"2023-09-10T00:39:16.527816+0000","last_undegraded":"2023-09-10T00:39:16.527816+0000","last_fullsized":"2023-09-10T00:39:16.527816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:41:45.621060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526180+0000","last_change":"2023-09-10T00:38:43.789159+0000","last_active":"2023-09-10T00:39:16.526180+0000","last_peered":"2023-09-10T00:39:16.526180+0000","last_clean":"2023-09-10T00:39:16.526180+0000","last_became_active":"2023-09-10T00:38:43.788675+0000","last_became_peered":"2023-09-10T00:38:43.788675+0000","last_unstale":"2023-09-10T00:39:16.526180+0000","last_undegraded":"2023-09-10T00:39:16.526180+0000","last_fullsized":"2023-09-10T00:39:16.526180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:30:39.545561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:37.584481+0000","last_change":"2023-09-10T00:38:38.767245+0000","last_active":"2023-09-10T00:39:37.584481+0000","last_peered":"2023-09-10T00:39:37.584481+0000","last_clean":"2023-09-10T00:39:37.584481+0000","last_became_active":"2023-09-10T00:38:38.766952+0000","last_became_peered":"2023-09-10T00:38:38.766952+0000","last_unstale":"2023-09-10T00:39:37.584481+0000","last_undegraded":"2023-09-10T00:39:37.584481+0000","last_fullsized":"2023-09-10T00:39:37.584481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:57:23.309838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527738+0000","last_change":"2023-09-10T00:38:39.772199+0000","last_active":"2023-09-10T00:39:16.527738+0000","last_peered":"2023-09-10T00:39:16.527738+0000","last_clean":"2023-09-10T00:39:16.527738+0000","last_became_active":"2023-09-10T00:38:39.771917+0000","last_became_peered":"2023-09-10T00:38:39.771917+0000","last_unstale":"2023-09-10T00:39:16.527738+0000","last_undegraded":"2023-09-10T00:39:16.527738+0000","last_fullsized":"2023-09-10T00:39:16.527738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:00:52.824581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527350+0000","last_change":"2023-09-10T00:38:43.797107+0000","last_active":"2023-09-10T00:39:16.527350+0000","last_peered":"2023-09-10T00:39:16.527350+0000","last_clean":"2023-09-10T00:39:16.527350+0000","last_became_active":"2023-09-10T00:38:43.796606+0000","last_became_peered":"2023-09-10T00:38:43.796606+0000","last_unstale":"2023-09-10T00:39:16.527350+0000","last_undegraded":"2023-09-10T00:39:16.527350+0000","last_fullsized":"2023-09-10T00:39:16.527350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:17:00.444238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.525888+0000","last_change":"2023-09-10T00:38:38.768045+0000","last_active":"2023-09-10T00:39:16.525888+0000","last_peered":"2023-09-10T00:39:16.525888+0000","last_clean":"2023-09-10T00:39:16.525888+0000","last_became_active":"2023-09-10T00:38:38.767344+0000","last_became_peered":"2023-09-10T00:38:38.767344+0000","last_unstale":"2023-09-10T00:39:16.525888+0000","last_undegraded":"2023-09-10T00:39:16.525888+0000","last_fullsized":"2023-09-10T00:39:16.525888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:51:08.765412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.536408+0000","last_change":"2023-09-10T00:39:17.536408+0000","last_active":"2023-09-10T00:39:17.536408+0000","last_peered":"2023-09-10T00:39:17.536408+0000","last_clean":"2023-09-10T00:39:17.536408+0000","last_became_active":"2023-09-10T00:39:17.535980+0000","last_became_peered":"2023-09-10T00:39:17.535980+0000","last_unstale":"2023-09-10T00:39:17.536408+0000","last_undegraded":"2023-09-10T00:39:17.536408+0000","last_fullsized":"2023-09-10T00:39:17.536408+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:08:10.517535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.521961+0000","last_change":"2023-09-10T00:38:43.784928+0000","last_active":"2023-09-10T00:39:16.521961+0000","last_peered":"2023-09-10T00:39:16.521961+0000","last_clean":"2023-09-10T00:39:16.521961+0000","last_became_active":"2023-09-10T00:38:43.784479+0000","last_became_peered":"2023-09-10T00:38:43.784479+0000","last_unstale":"2023-09-10T00:39:16.521961+0000","last_undegraded":"2023-09-10T00:39:16.521961+0000","last_fullsized":"2023-09-10T00:39:16.521961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:14:45.019988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.538128+0000","last_change":"2023-09-10T00:38:38.770730+0000","last_active":"2023-09-10T00:39:16.538128+0000","last_peered":"2023-09-10T00:39:16.538128+0000","last_clean":"2023-09-10T00:39:16.538128+0000","last_became_active":"2023-09-10T00:38:38.770344+0000","last_became_peered":"2023-09-10T00:38:38.770344+0000","last_unstale":"2023-09-10T00:39:16.538128+0000","last_undegraded":"2023-09-10T00:39:16.538128+0000","last_fullsized":"2023-09-10T00:39:16.538128+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:49:22.881956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527892+0000","last_change":"2023-09-10T00:38:39.766149+0000","last_active":"2023-09-10T00:39:16.527892+0000","last_peered":"2023-09-10T00:39:16.527892+0000","last_clean":"2023-09-10T00:39:16.527892+0000","last_became_active":"2023-09-10T00:38:39.765773+0000","last_became_peered":"2023-09-10T00:38:39.765773+0000","last_unstale":"2023-09-10T00:39:16.527892+0000","last_undegraded":"2023-09-10T00:39:16.527892+0000","last_fullsized":"2023-09-10T00:39:16.527892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:00:37.854168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.519425+0000","last_change":"2023-09-10T00:38:39.765197+0000","last_active":"2023-09-10T00:39:16.519425+0000","last_peered":"2023-09-10T00:39:16.519425+0000","last_clean":"2023-09-10T00:39:16.519425+0000","last_became_active":"2023-09-10T00:38:39.764891+0000","last_became_peered":"2023-09-10T00:38:39.764891+0000","last_unstale":"2023-09-10T00:39:16.519425+0000","last_undegraded":"2023-09-10T00:39:16.519425+0000","last_fullsized":"2023-09-10T00:39:16.519425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:47:27.429355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527189+0000","last_change":"2023-09-10T00:38:39.764722+0000","last_active":"2023-09-10T00:39:16.527189+0000","last_peered":"2023-09-10T00:39:16.527189+0000","last_clean":"2023-09-10T00:39:16.527189+0000","last_became_active":"2023-09-10T00:38:39.764376+0000","last_became_peered":"2023-09-10T00:38:39.764376+0000","last_unstale":"2023-09-10T00:39:16.527189+0000","last_undegraded":"2023-09-10T00:39:16.527189+0000","last_fullsized":"2023-09-10T00:39:16.527189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:27:36.708573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526720+0000","last_change":"2023-09-10T00:38:39.771997+0000","last_active":"2023-09-10T00:39:16.526720+0000","last_peered":"2023-09-10T00:39:16.526720+0000","last_clean":"2023-09-10T00:39:16.526720+0000","last_became_active":"2023-09-10T00:38:39.771588+0000","last_became_peered":"2023-09-10T00:38:39.771588+0000","last_unstale":"2023-09-10T00:39:16.526720+0000","last_undegraded":"2023-09-10T00:39:16.526720+0000","last_fullsized":"2023-09-10T00:39:16.526720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:20:07.171333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527554+0000","last_change":"2023-09-10T00:38:39.773393+0000","last_active":"2023-09-10T00:39:16.527554+0000","last_peered":"2023-09-10T00:39:16.527554+0000","last_clean":"2023-09-10T00:39:16.527554+0000","last_became_active":"2023-09-10T00:38:39.772972+0000","last_became_peered":"2023-09-10T00:38:39.772972+0000","last_unstale":"2023-09-10T00:39:16.527554+0000","last_undegraded":"2023-09-10T00:39:16.527554+0000","last_fullsized":"2023-09-10T00:39:16.527554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:27:29.916970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.537695+0000","last_change":"2023-09-10T00:38:39.767871+0000","last_active":"2023-09-10T00:39:16.537695+0000","last_peered":"2023-09-10T00:39:16.537695+0000","last_clean":"2023-09-10T00:39:16.537695+0000","last_became_active":"2023-09-10T00:38:39.767563+0000","last_became_peered":"2023-09-10T00:38:39.767563+0000","last_unstale":"2023-09-10T00:39:16.537695+0000","last_undegraded":"2023-09-10T00:39:16.537695+0000","last_fullsized":"2023-09-10T00:39:16.537695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:22:30.400441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.520036+0000","last_change":"2023-09-10T00:38:39.764664+0000","last_active":"2023-09-10T00:39:16.520036+0000","last_peered":"2023-09-10T00:39:16.520036+0000","last_clean":"2023-09-10T00:39:16.520036+0000","last_became_active":"2023-09-10T00:38:39.764364+0000","last_became_peered":"2023-09-10T00:38:39.764364+0000","last_unstale":"2023-09-10T00:39:16.520036+0000","last_undegraded":"2023-09-10T00:39:16.520036+0000","last_fullsized":"2023-09-10T00:39:16.520036+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:41.656472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527797+0000","last_change":"2023-09-10T00:38:39.770315+0000","last_active":"2023-09-10T00:39:16.527797+0000","last_peered":"2023-09-10T00:39:16.527797+0000","last_clean":"2023-09-10T00:39:16.527797+0000","last_became_active":"2023-09-10T00:38:39.769977+0000","last_became_peered":"2023-09-10T00:38:39.769977+0000","last_unstale":"2023-09-10T00:39:16.527797+0000","last_undegraded":"2023-09-10T00:39:16.527797+0000","last_fullsized":"2023-09-10T00:39:16.527797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:30:13.380487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.535264+0000","last_change":"2023-09-10T00:38:39.768734+0000","last_active":"2023-09-10T00:39:16.535264+0000","last_peered":"2023-09-10T00:39:16.535264+0000","last_clean":"2023-09-10T00:39:16.535264+0000","last_became_active":"2023-09-10T00:38:39.768383+0000","last_became_peered":"2023-09-10T00:38:39.768383+0000","last_unstale":"2023-09-10T00:39:16.535264+0000","last_undegraded":"2023-09-10T00:39:16.535264+0000","last_fullsized":"2023-09-10T00:39:16.535264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:09.974447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.732943+0000","last_change":"2023-09-10T00:39:17.732561+0000","last_active":"2023-09-10T00:39:17.732943+0000","last_peered":"2023-09-10T00:39:17.732943+0000","last_clean":"2023-09-10T00:39:17.732943+0000","last_became_active":"2023-09-10T00:39:17.732138+0000","last_became_peered":"2023-09-10T00:39:17.732138+0000","last_unstale":"2023-09-10T00:39:17.732943+0000","last_undegraded":"2023-09-10T00:39:17.732943+0000","last_fullsized":"2023-09-10T00:39:17.732943+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:11:58.866881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.533521+0000","last_change":"2023-09-10T00:38:39.773839+0000","last_active":"2023-09-10T00:39:16.533521+0000","last_peered":"2023-09-10T00:39:16.533521+0000","last_clean":"2023-09-10T00:39:16.533521+0000","last_became_active":"2023-09-10T00:38:39.773521+0000","last_became_peered":"2023-09-10T00:38:39.773521+0000","last_unstale":"2023-09-10T00:39:16.533521+0000","last_undegraded":"2023-09-10T00:39:16.533521+0000","last_fullsized":"2023-09-10T00:39:16.533521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:15:49.033374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526122+0000","last_change":"2023-09-10T00:38:39.775721+0000","last_active":"2023-09-10T00:39:16.526122+0000","last_peered":"2023-09-10T00:39:16.526122+0000","last_clean":"2023-09-10T00:39:16.526122+0000","last_became_active":"2023-09-10T00:38:39.775424+0000","last_became_peered":"2023-09-10T00:38:39.775424+0000","last_unstale":"2023-09-10T00:39:16.526122+0000","last_undegraded":"2023-09-10T00:39:16.526122+0000","last_fullsized":"2023-09-10T00:39:16.526122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:56:26.079332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.536682+0000","last_change":"2023-09-10T00:38:39.768767+0000","last_active":"2023-09-10T00:39:16.536682+0000","last_peered":"2023-09-10T00:39:16.536682+0000","last_clean":"2023-09-10T00:39:16.536682+0000","last_became_active":"2023-09-10T00:38:39.768452+0000","last_became_peered":"2023-09-10T00:38:39.768452+0000","last_unstale":"2023-09-10T00:39:16.536682+0000","last_undegraded":"2023-09-10T00:39:16.536682+0000","last_fullsized":"2023-09-10T00:39:16.536682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:31:53.578574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.519685+0000","last_change":"2023-09-10T00:38:39.761998+0000","last_active":"2023-09-10T00:39:16.519685+0000","last_peered":"2023-09-10T00:39:16.519685+0000","last_clean":"2023-09-10T00:39:16.519685+0000","last_became_active":"2023-09-10T00:38:39.761672+0000","last_became_peered":"2023-09-10T00:38:39.761672+0000","last_unstale":"2023-09-10T00:39:16.519685+0000","last_undegraded":"2023-09-10T00:39:16.519685+0000","last_fullsized":"2023-09-10T00:39:16.519685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:03:24.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65801,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":433,"num_read_kb":291,"num_write":234,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":197403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449111,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":490613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.501},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.46300000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.74399999999999999}]},{"osd":2,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.53900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":4,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.65100000000000002}]},{"osd":5,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.626},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.61199999999999999}]},{"osd":6,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.68799999999999994}]},{"osd":7,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.70799999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":9,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.81000000000000005}]},{"osd":10,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.76700000000000002}]}]},{"osd":10,"up_from":70,"seq":300647710748,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1695744,"data_stored":1136418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.46899999999999997}]},{"osd":1,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51500000000000001,"15min":0.51900000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78200000000000003,"5min":0.873,"15min":0.873},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.498,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.36499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50700000000000001}]},{"osd":2,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.629,"15min":0.629},"min":{"1min":0.501,"5min":0.439,"15min":0.439},"max":{"1min":0.75800000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65400000000000003,"15min":0.64400000000000002},"min":{"1min":0.52400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.64500000000000002}]},{"osd":3,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.61899999999999999,"15min":0.63700000000000001},"min":{"1min":0.39300000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70099999999999996,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.443},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.48899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.69499999999999995}]},{"osd":4,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.625},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.71599999999999997}]},{"osd":5,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.54600000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78100000000000003,"15min":0.78300000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.94299999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.82299999999999995}]},{"osd":6,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.995,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.59999999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73999999999999999}]},{"osd":7,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71299999999999997}]},{"osd":8,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77700000000000002,"15min":0.77500000000000002},"min":{"1min":0.63900000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78600000000000003,"15min":0.78800000000000003},"min":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.84399999999999997}]},{"osd":9,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.86099999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55600000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.752}]},{"osd":11,"last update":"Sun Sep 10 00:39:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.86799999999999999}]}]},{"osd":9,"up_from":65,"seq":279172874273,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":494639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.441,"15min":0.44900000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.58399999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.504},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.44900000000000001,"15min":0.46300000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.60899999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.33200000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.77800000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78600000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.432}]},{"osd":2,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.46000000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.83099999999999996,"5min":0.879,"15min":0.879},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.59499999999999997,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79100000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40200000000000002}]},{"osd":3,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.55800000000000005,"15min":0.57799999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76000000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.60899999999999999,"15min":0.63100000000000001},"min":{"1min":0.375,"5min":0.314,"15min":0.314},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.55800000000000005}]},{"osd":4,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.52500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.85499999999999998,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.61899999999999999}]},{"osd":5,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.73399999999999999,"15min":0.749},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91500000000000004,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":3.3109999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.75900000000000001,"15min":0.77700000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.028,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.68799999999999994}]},{"osd":6,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.42499999999999999}]},{"osd":7,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.79200000000000004,"15min":0.80900000000000005},"min":{"1min":0.504,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.071,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.81000000000000005,"15min":0.83199999999999996},"min":{"1min":0.498,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0580000000000001,"5min":1.071,"15min":1.071},"last":0.63300000000000001}]},{"osd":8,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.77500000000000002,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.83499999999999996,"15min":0.85999999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96699999999999997,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.65900000000000003}]},{"osd":10,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.72899999999999998,"15min":0.749},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.77600000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.54000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.82799999999999996,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":3.359}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":8,"up_from":57,"seq":244813135909,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":490590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57199999999999995,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.67800000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.47799999999999998}]},{"osd":1,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.67500000000000004,"15min":0.628},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":6.5759999999999996,"5min":6.5759999999999996,"15min":6.5759999999999996},"last":0.71899999999999997}]},{"osd":2,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.73299999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.59299999999999997}]},{"osd":3,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83099999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.63}]},{"osd":4,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.41199999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.439,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.77600000000000002}]},{"osd":5,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.34699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94399999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60699999999999998}]},{"osd":6,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.79100000000000004,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":6.5940000000000003,"5min":6.5940000000000003,"15min":6.5940000000000003},"last":0.67400000000000004}]},{"osd":7,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.76000000000000001,"15min":0.70999999999999996},"min":{"1min":0.51300000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.5439999999999996,"5min":6.5439999999999996,"15min":6.5439999999999996},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.441,"5min":0.435,"15min":0.435},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.79900000000000004}]},{"osd":9,"last update":"Sun Sep 10 00:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.73899999999999999,"15min":0.73199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.73399999999999999,"15min":0.72199999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.78800000000000003}]},{"osd":10,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.752},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.88800000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]}]},{"osd":7,"up_from":50,"seq":214748364840,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1617920,"data_stored":1087476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54100000000000004,"15min":0.53200000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.39800000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40500000000000003,"15min":0.40000000000000002},"min":{"1min":0.32800000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.52600000000000002,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.40999999999999998,"15min":0.40200000000000002},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63600000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.311}]},{"osd":2,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.79800000000000004,"5min":1.091,"15min":1.091},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.51400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85999999999999999,"5min":1.069,"15min":1.069},"last":0.51000000000000001}]},{"osd":3,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.36699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76900000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.46999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.39900000000000002}]},{"osd":4,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.38300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.66300000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.497,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.59999999999999998,"5min":1.056,"15min":1.056},"last":0.29299999999999998}]},{"osd":5,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":0.628},"min":{"1min":0.49399999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.83299999999999996,"5min":1.026,"15min":1.026},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.48099999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98399999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.46100000000000002}]},{"osd":6,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.48899999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.84999999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89800000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.55200000000000005}]},{"osd":8,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.63500000000000001,"15min":0.62},"min":{"1min":0.50600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.66200000000000003,"15min":0.65200000000000002},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.496}]},{"osd":9,"last update":"Sun Sep 10 00:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.53500000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80000000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69299999999999995,"15min":0.69599999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.83599999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.48299999999999998}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65700000000000003,"15min":0.65200000000000002},"min":{"1min":0.46100000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83299999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66600000000000004,"15min":0.65700000000000003},"min":{"1min":0.54000000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.57099999999999995}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.624}]}]},{"osd":6,"up_from":45,"seq":193273528365,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":486047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.41199999999999998,"15min":0.40000000000000002},"min":{"1min":0.311,"5min":0.247,"15min":0.247},"max":{"1min":0.52000000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42299999999999999,"15min":0.40799999999999997},"min":{"1min":0.33100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59199999999999997,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.42199999999999999}]},{"osd":1,"last update":"Sun Sep 10 00:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57799999999999996,"15min":0.55700000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80600000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.71599999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53800000000000003}]},{"osd":2,"last update":"Sun Sep 10 00:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.47799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74299999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.624,"15min":0.60999999999999999},"min":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76300000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55500000000000005}]},{"osd":3,"last update":"Sun Sep 10 00:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.49299999999999999,"15min":0.48099999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68999999999999995,"5min":0.875,"15min":0.875},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.42899999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67700000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.47999999999999998}]},{"osd":4,"last update":"Sun Sep 10 00:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64500000000000002,"15min":0.64200000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88700000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.68899999999999995,"15min":0.67900000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81000000000000005,"5min":1.024,"15min":1.024},"last":0.59999999999999998}]},{"osd":5,"last update":"Sun Sep 10 00:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.46800000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82999999999999996,"5min":1.048,"15min":1.048},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70799999999999996,"15min":0.69699999999999995},"min":{"1min":0.48299999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91300000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.65900000000000003}]},{"osd":7,"last update":"Sun Sep 10 00:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.73199999999999998,"15min":0.72899999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94899999999999995,"5min":1.069,"15min":1.069},"last":0.625},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72999999999999998,"15min":0.72299999999999998},"min":{"1min":0.47799999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.82799999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.70899999999999996}]},{"osd":8,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.81499999999999995,"15min":0.83299999999999996},"min":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.999,"5min":1.27,"15min":1.27},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.81699999999999995,"15min":0.82799999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0389999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.754}]},{"osd":9,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.82799999999999996,"15min":0.85899999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":1.389,"15min":1.389},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.76000000000000001,"15min":0.78200000000000003},"min":{"1min":0.5,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.80700000000000005,"5min":1.099,"15min":1.099},"last":0.59299999999999997}]},{"osd":10,"last update":"Sun Sep 10 00:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.82599999999999996,"15min":0.83899999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.96299999999999997,"5min":1.181,"15min":1.181},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.91300000000000003,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.73899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":545676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52600000000000002,"15min":0.50900000000000001},"min":{"1min":0.32100000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72199999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.53200000000000003,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.495}]},{"osd":2,"last update":"Sun Sep 10 00:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.38100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73299999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.56399999999999995,"15min":0.53700000000000003},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":0.89100000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64300000000000002}]},{"osd":3,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.315,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.79300000000000004,"5min":0.879,"15min":0.879},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53400000000000003,"15min":0.52700000000000002},"min":{"1min":0.34699999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78800000000000003,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60499999999999998}]},{"osd":4,"last update":"Sun Sep 10 00:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.42299999999999999},"min":{"1min":0.35099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.46000000000000002,"15min":0.437},"min":{"1min":0.33800000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58299999999999996}]},{"osd":5,"last update":"Sun Sep 10 00:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.755,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.65800000000000003,"15min":0.626},"min":{"1min":0.62,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92700000000000005,"5min":1.044,"15min":1.044},"last":0.71899999999999997}]},{"osd":6,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64000000000000001,"15min":0.61399999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86899999999999999,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.78500000000000003}]},{"osd":7,"last update":"Sun Sep 10 00:38:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.59099999999999997,"15min":0.56599999999999995},"min":{"1min":0.49099999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.40899999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.82699999999999996}]},{"osd":8,"last update":"Sun Sep 10 00:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71699999999999997,"15min":0.71299999999999997},"min":{"1min":0.57799999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94999999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.879},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.54000000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.86599999999999999,"5min":1.014,"15min":1.014},"last":0.84399999999999997}]},{"osd":9,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.998,"5min":1.097,"15min":1.097},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77300000000000002,"15min":0.78600000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.93000000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.90600000000000003}]},{"osd":10,"last update":"Sun Sep 10 00:38:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.89500000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":487503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 10 00:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54400000000000004,"15min":0.5},"min":{"1min":0.36399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.52500000000000002},"min":{"1min":0.376,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.46899999999999997}]},{"osd":2,"last update":"Sun Sep 10 00:38:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.39500000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80700000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58599999999999997,"15min":0.56799999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0329999999999999,"5min":1.621,"15min":1.621},"last":0.83999999999999997}]},{"osd":3,"last update":"Sun Sep 10 00:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.41799999999999998,"15min":0.41699999999999998},"min":{"1min":0.249,"5min":0.245,"15min":0.245},"max":{"1min":0.66700000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.436},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46000000000000002,"15min":0.45600000000000002},"min":{"1min":0.29099999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.79100000000000004,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.37}]},{"osd":4,"last update":"Sun Sep 10 00:38:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68899999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.38900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.91300000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.48499999999999999}]},{"osd":5,"last update":"Sun Sep 10 00:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64000000000000001,"15min":0.60999999999999999},"min":{"1min":0.49199999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.96999999999999997,"5min":1.075,"15min":1.075},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64700000000000002,"15min":0.63700000000000001},"min":{"1min":0.48699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.54400000000000004}]},{"osd":6,"last update":"Sun Sep 10 00:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.34699999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.70699999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76100000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.55700000000000005}]},{"osd":7,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65700000000000003,"15min":0.66400000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73899999999999999,"15min":0.74399999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.98499999999999999,"5min":1.111,"15min":1.111},"last":0.71699999999999997}]},{"osd":8,"last update":"Sun Sep 10 00:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77300000000000002,"15min":0.77900000000000003},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0640000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.499,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.80300000000000005}]},{"osd":9,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.746,"15min":0.754},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0309999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.71799999999999997},"min":{"1min":0.45500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.621}]},{"osd":10,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.746,"15min":0.74299999999999999},"min":{"1min":0.57999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.53400000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.90600000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.745}]},{"osd":11,"last update":"Sun Sep 10 00:39:02 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.755}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":546286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.70499999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.439},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.39800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72099999999999997,"5min":0.753,"15min":0.753},"last":0.40799999999999997}]},{"osd":1,"last update":"Sun Sep 10 00:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.42499999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.746,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82599999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68000000000000005}]},{"osd":3,"last update":"Sun Sep 10 00:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.45000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.442,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68999999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58599999999999997}]},{"osd":4,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.39000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.82299999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79700000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.79400000000000004}]},{"osd":5,"last update":"Sun Sep 10 00:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.45500000000000002,"15min":0.435},"min":{"1min":0.29899999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52400000000000002,"15min":0.5},"min":{"1min":0.35699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56799999999999995}]},{"osd":6,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.47699999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.66400000000000003,"15min":0.64300000000000002},"min":{"1min":0.53700000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.77300000000000002}]},{"osd":7,"last update":"Sun Sep 10 00:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.746},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.81499999999999995}]},{"osd":8,"last update":"Sun Sep 10 00:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.35099999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78400000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.36899999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63700000000000001}]},{"osd":9,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.61199999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.53400000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.80600000000000005}]},{"osd":10,"last update":"Sun Sep 10 00:38:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.90100000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":485755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40200000000000002,"15min":0.39800000000000002},"min":{"1min":0.27900000000000003,"5min":0.217,"15min":0.217},"max":{"1min":0.83199999999999996,"5min":1.409,"15min":1.409},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.41899999999999998,"15min":0.41799999999999998},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.57699999999999996,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.40899999999999997}]},{"osd":1,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57999999999999996,"15min":0.59899999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92200000000000004,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.39500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88400000000000001,"5min":1.581,"15min":1.581},"last":0.52900000000000003}]},{"osd":2,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60599999999999998,"15min":0.60099999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78400000000000003,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63,"15min":0.65400000000000003},"min":{"1min":0.41299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85099999999999998,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.54100000000000004}]},{"osd":4,"last update":"Sun Sep 10 00:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.39300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80600000000000005,"5min":1.536,"15min":1.536},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.66500000000000004,"15min":0.68799999999999994},"min":{"1min":0.38300000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.748,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.51600000000000001}]},{"osd":5,"last update":"Sun Sep 10 00:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88100000000000001,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67800000000000005,"15min":0.67500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88700000000000001,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.65300000000000002}]},{"osd":6,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78900000000000003,"5min":1,"15min":1},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60199999999999998,"15min":0.57999999999999996},"min":{"1min":0.35699999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.84299999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.498}]},{"osd":7,"last update":"Sun Sep 10 00:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.74399999999999999,"15min":0.745},"min":{"1min":0.47699999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90800000000000003,"5min":1.022,"15min":1.022},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73799999999999999,"15min":0.73599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.69799999999999995}]},{"osd":8,"last update":"Sun Sep 10 00:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.90000000000000002}]},{"osd":9,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.73199999999999998,"15min":0.76400000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.98899999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.81200000000000006,"15min":0.83599999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96499999999999997,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.58199999999999996}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.82999999999999996,"15min":0.83999999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.75,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.71999999999999997}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.753}]}]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":490511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45000000000000001}]},{"osd":1,"last update":"Sun Sep 10 00:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.37,"15min":0.34399999999999997},"min":{"1min":0.29199999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.317,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.51400000000000001}]},{"osd":2,"last update":"Sun Sep 10 00:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80200000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58799999999999997,"15min":0.55500000000000005},"min":{"1min":0.497,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.83099999999999996}]},{"osd":3,"last update":"Sun Sep 10 00:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.39900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.60999999999999999,"15min":0.56200000000000006},"min":{"1min":0.35099999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.55000000000000004}]},{"osd":5,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.439,"5min":0.376,"15min":0.376},"max":{"1min":0.80600000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.67900000000000005}]},{"osd":6,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58099999999999996,"15min":0.56799999999999995},"min":{"1min":0.50600000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.78400000000000003,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.871},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.63700000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79900000000000004,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.71899999999999997}]},{"osd":7,"last update":"Sun Sep 10 00:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.53500000000000003,"15min":0.51800000000000002},"min":{"1min":0.41099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.748}]},{"osd":8,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63900000000000001,"15min":0.629},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91100000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.879,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.76800000000000002}]},{"osd":9,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.63100000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73499999999999999,"15min":0.73399999999999999},"min":{"1min":0.56000000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94799999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.78400000000000003}]},{"osd":10,"last update":"Sun Sep 10 00:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.94199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1630208,"data_stored":1082999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.40699999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.755,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52900000000000003}]},{"osd":1,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53400000000000003,"15min":0.51600000000000001},"min":{"1min":0.33700000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86099999999999999,"5min":0.873,"15min":0.873},"last":0.61599999999999999}]},{"osd":2,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40699999999999997,"15min":0.39600000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.71699999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.438,"15min":0.42999999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40699999999999997}]},{"osd":3,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54900000000000004,"15min":0.54200000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.81499999999999995,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.92400000000000004,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.51300000000000001}]},{"osd":4,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76000000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.68799999999999994}]},{"osd":6,"last update":"Sun Sep 10 00:39:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61699999999999999,"15min":0.61199999999999999},"min":{"1min":0.38900000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.502},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66000000000000003,"15min":0.65300000000000002},"min":{"1min":0.45700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96799999999999997,"5min":0.996,"15min":0.996},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Sep 10 00:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.64100000000000001,"15min":0.622},"min":{"1min":0.53700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.98399999999999999,"5min":1.046,"15min":1.046},"last":0.74199999999999999}]},{"osd":8,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.42499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53400000000000003,"15min":0.52000000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55600000000000005}]},{"osd":9,"last update":"Sun Sep 10 00:39:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.72199999999999998,"15min":0.71099999999999997},"min":{"1min":0.61899999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.71699999999999997,"15min":0.70499999999999996},"min":{"1min":0.61799999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.64900000000000002}]},{"osd":10,"last update":"Sun Sep 10 00:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.77600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-10T00:39:43.922 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph pg dump --format=json 2023-09-10T00:39:44.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:43 smithi131 ceph-mon[103713]: pgmap v302: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-09-10T00:39:44.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:44.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:44.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:44.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:43 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:44.488 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:44.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:44 smithi099 ceph-mon[102692]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:39:44.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:44 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:44.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:44 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:44.870 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:44 smithi044 ceph-mon[98676]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:39:44.870 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:44 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:44.870 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:44 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:44 smithi131 ceph-mon[103713]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:39:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:44 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:44 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:45.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.697+0000 7fabfaff3700 1 -- 172.21.15.44:0/1893253157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf4068ae0 msgr2=0x7fabf410d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:45.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.697+0000 7fabfaff3700 1 --2- 172.21.15.44:0/1893253157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf4068ae0 0x7fabf410d440 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fabec00b0a0 tx=0x7fabec00b3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:45.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.697+0000 7fabfaff3700 1 -- 172.21.15.44:0/1893253157 shutdown_connections 2023-09-10T00:39:45.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.697+0000 7fabfaff3700 1 --2- 172.21.15.44:0/1893253157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf4068ae0 0x7fabf410d440 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:45.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.697+0000 7fabfaff3700 1 --2- 172.21.15.44:0/1893253157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabf4106770 0x7fabf4106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:45.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.697+0000 7fabfaff3700 1 --2- 172.21.15.44:0/1893253157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabf4068120 0x7fabf40685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:45.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.697+0000 7fabfaff3700 1 -- 172.21.15.44:0/1893253157 >> 172.21.15.44:0/1893253157 conn(0x7fabf4076600 msgr2=0x7fabf4076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:45.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.697+0000 7fabfaff3700 1 -- 172.21.15.44:0/1893253157 shutdown_connections 2023-09-10T00:39:45.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.697+0000 7fabfaff3700 1 -- 172.21.15.44:0/1893253157 wait complete. 2023-09-10T00:39:45.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.697+0000 7fabfaff3700 1 Processor -- start 2023-09-10T00:39:45.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.697+0000 7fabfaff3700 1 -- start start 2023-09-10T00:39:45.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabfaff3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf4068120 0x7fabf41966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:45.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabfaff3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabf4068ae0 0x7fabf4196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:45.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabfaff3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabf4106770 0x7fabf4199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:45.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabfaff3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf419a5f0 con 0x7fabf4068120 2023-09-10T00:39:45.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabfaff3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf419a760 con 0x7fabf4106770 2023-09-10T00:39:45.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabfaff3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf419a920 con 0x7fabf4068ae0 2023-09-10T00:39:45.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabf8d8f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf4068120 0x7fabf41966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:45.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabf8d8f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf4068120 0x7fabf41966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47316/0 (socket says 172.21.15.44:47316) 2023-09-10T00:39:45.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabf8d8f700 1 -- 172.21.15.44:0/3011870667 learned_addr learned my addr 172.21.15.44:0/3011870667 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:45.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabf3fff700 1 --2- 172.21.15.44:0/3011870667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabf4068ae0 0x7fabf4196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:45.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabf9590700 1 --2- 172.21.15.44:0/3011870667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabf4106770 0x7fabf4199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:45.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabf8d8f700 1 -- 172.21.15.44:0/3011870667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabf4068ae0 msgr2=0x7fabf4196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:45.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabf8d8f700 1 --2- 172.21.15.44:0/3011870667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabf4068ae0 0x7fabf4196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:45.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabf8d8f700 1 -- 172.21.15.44:0/3011870667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabf4106770 msgr2=0x7fabf4199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:45.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabf8d8f700 1 --2- 172.21.15.44:0/3011870667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabf4106770 0x7fabf4199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:45.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabf8d8f700 1 -- 172.21.15.44:0/3011870667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec009d00 con 0x7fabf4068120 2023-09-10T00:39:45.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabf8d8f700 1 --2- 172.21.15.44:0/3011870667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf4068120 0x7fabf41966f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fabe800d950 tx=0x7fabe800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:45.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.698+0000 7fabf1ffb700 1 -- 172.21.15.44:0/3011870667 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe800f890 con 0x7fabf4068120 2023-09-10T00:39:45.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.699+0000 7fabf1ffb700 1 -- 172.21.15.44:0/3011870667 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fabe800fe80 con 0x7fabf4068120 2023-09-10T00:39:45.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.699+0000 7fabfaff3700 1 -- 172.21.15.44:0/3011870667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf419ac00 con 0x7fabf4068120 2023-09-10T00:39:45.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.699+0000 7fabf1ffb700 1 -- 172.21.15.44:0/3011870667 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe800e3b0 con 0x7fabf4068120 2023-09-10T00:39:45.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.699+0000 7fabfaff3700 1 -- 172.21.15.44:0/3011870667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf41a3550 con 0x7fabf4068120 2023-09-10T00:39:45.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.700+0000 7fabfaff3700 1 -- 172.21.15.44:0/3011870667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf404e300 con 0x7fabf4068120 2023-09-10T00:39:45.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.701+0000 7fabf1ffb700 1 -- 172.21.15.44:0/3011870667 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fabe8010460 con 0x7fabf4068120 2023-09-10T00:39:45.708 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.701+0000 7fabf1ffb700 1 --2- 172.21.15.44:0/3011870667 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fabe40a06d0 0x7fabe40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:45.708 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.701+0000 7fabf3fff700 1 --2- 172.21.15.44:0/3011870667 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fabe40a06d0 0x7fabe40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:45.708 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.701+0000 7fabf1ffb700 1 -- 172.21.15.44:0/3011870667 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fabe80cbdd0 con 0x7fabf4068120 2023-09-10T00:39:45.709 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.702+0000 7fabf3fff700 1 --2- 172.21.15.44:0/3011870667 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fabe40a06d0 0x7fabe40a2b90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fabf4197d10 tx=0x7fabe000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:45.709 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.706+0000 7fabf1ffb700 1 -- 172.21.15.44:0/3011870667 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fabe8095490 con 0x7fabf4068120 2023-09-10T00:39:45.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:45 smithi044 ceph-mon[98676]: pgmap v303: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:39:45.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:39:45.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:39:45.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:45 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:45.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:45 smithi099 ceph-mon[102692]: pgmap v303: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:39:45.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:39:45.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:39:45.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:45 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.900+0000 7fabfaff3700 1 -- 172.21.15.44:0/3011870667 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fabf41105c0 con 0x7fabe40a06d0 2023-09-10T00:39:45.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.915+0000 7fabf1ffb700 1 -- 172.21.15.44:0/3011870667 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426252 (secure 0 0 0) 0x7fabf41105c0 con 0x7fabe40a06d0 2023-09-10T00:39:45.921 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:45.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.918+0000 7fabfaff3700 1 -- 172.21.15.44:0/3011870667 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fabe40a06d0 msgr2=0x7fabe40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:45.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.918+0000 7fabfaff3700 1 --2- 172.21.15.44:0/3011870667 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fabe40a06d0 0x7fabe40a2b90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fabf4197d10 tx=0x7fabe000a300 comp rx=0 tx=0).stop 2023-09-10T00:39:45.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.918+0000 7fabfaff3700 1 -- 172.21.15.44:0/3011870667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf4068120 msgr2=0x7fabf41966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:45.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.918+0000 7fabfaff3700 1 --2- 172.21.15.44:0/3011870667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf4068120 0x7fabf41966f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fabe800d950 tx=0x7fabe800dc60 comp rx=0 tx=0).stop 2023-09-10T00:39:45.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.918+0000 7fabfaff3700 1 -- 172.21.15.44:0/3011870667 shutdown_connections 2023-09-10T00:39:45.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.918+0000 7fabfaff3700 1 --2- 172.21.15.44:0/3011870667 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fabe40a06d0 0x7fabe40a2b90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:45.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.918+0000 7fabfaff3700 1 --2- 172.21.15.44:0/3011870667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabf4106770 0x7fabf4199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:45.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.918+0000 7fabfaff3700 1 --2- 172.21.15.44:0/3011870667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf4068120 0x7fabf41966f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:45.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.918+0000 7fabfaff3700 1 --2- 172.21.15.44:0/3011870667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabf4068ae0 0x7fabf4196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:45.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.918+0000 7fabfaff3700 1 -- 172.21.15.44:0/3011870667 >> 172.21.15.44:0/3011870667 conn(0x7fabf4076600 msgr2=0x7fabf40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:45.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.919+0000 7fabfaff3700 1 -- 172.21.15.44:0/3011870667 shutdown_connections 2023-09-10T00:39:45.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:45.919+0000 7fabfaff3700 1 -- 172.21.15.44:0/3011870667 wait complete. 2023-09-10T00:39:45.927 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2023-09-10T00:39:46.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:45 smithi131 ceph-mon[103713]: pgmap v303: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:39:46.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:39:46.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:39:46.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:45 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' 2023-09-10T00:39:46.360 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2023-09-10T00:39:44.011157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656415,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":579,"num_read_kb":410,"num_write":417,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338132,"kb_used_data":13884,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603692,"statfs":{"total":1151940427776,"available":1151594180608,"internally_reserved":0,"allocated":14217216,"data_stored":7824513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003355"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.520087+0000","last_change":"2023-09-10T00:38:39.768218+0000","last_active":"2023-09-10T00:39:16.520087+0000","last_peered":"2023-09-10T00:39:16.520087+0000","last_clean":"2023-09-10T00:39:16.520087+0000","last_became_active":"2023-09-10T00:38:39.767454+0000","last_became_peered":"2023-09-10T00:38:39.767454+0000","last_unstale":"2023-09-10T00:39:16.520087+0000","last_undegraded":"2023-09-10T00:39:16.520087+0000","last_fullsized":"2023-09-10T00:39:16.520087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:17:30.287403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.528889+0000","last_change":"2023-09-10T00:38:39.765384+0000","last_active":"2023-09-10T00:39:16.528889+0000","last_peered":"2023-09-10T00:39:16.528889+0000","last_clean":"2023-09-10T00:39:16.528889+0000","last_became_active":"2023-09-10T00:38:39.765090+0000","last_became_peered":"2023-09-10T00:38:39.765090+0000","last_unstale":"2023-09-10T00:39:16.528889+0000","last_undegraded":"2023-09-10T00:39:16.528889+0000","last_fullsized":"2023-09-10T00:39:16.528889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:02:46.025627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.528127+0000","last_change":"2023-09-10T00:38:39.769222+0000","last_active":"2023-09-10T00:39:16.528127+0000","last_peered":"2023-09-10T00:39:16.528127+0000","last_clean":"2023-09-10T00:39:16.528127+0000","last_became_active":"2023-09-10T00:38:39.768801+0000","last_became_peered":"2023-09-10T00:38:39.768801+0000","last_unstale":"2023-09-10T00:39:16.528127+0000","last_undegraded":"2023-09-10T00:39:16.528127+0000","last_fullsized":"2023-09-10T00:39:16.528127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:38:53.394209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522938+0000","last_change":"2023-09-10T00:38:39.764963+0000","last_active":"2023-09-10T00:39:16.522938+0000","last_peered":"2023-09-10T00:39:16.522938+0000","last_clean":"2023-09-10T00:39:16.522938+0000","last_became_active":"2023-09-10T00:38:39.764466+0000","last_became_peered":"2023-09-10T00:38:39.764466+0000","last_unstale":"2023-09-10T00:39:16.522938+0000","last_undegraded":"2023-09-10T00:39:16.522938+0000","last_fullsized":"2023-09-10T00:39:16.522938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:31.553321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.534684+0000","last_change":"2023-09-10T00:39:17.534684+0000","last_active":"2023-09-10T00:39:17.534684+0000","last_peered":"2023-09-10T00:39:17.534684+0000","last_clean":"2023-09-10T00:39:17.534684+0000","last_became_active":"2023-09-10T00:39:17.534325+0000","last_became_peered":"2023-09-10T00:39:17.534325+0000","last_unstale":"2023-09-10T00:39:17.534684+0000","last_undegraded":"2023-09-10T00:39:17.534684+0000","last_fullsized":"2023-09-10T00:39:17.534684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:39:11.902221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526669+0000","last_change":"2023-09-10T00:38:39.761681+0000","last_active":"2023-09-10T00:39:16.526669+0000","last_peered":"2023-09-10T00:39:16.526669+0000","last_clean":"2023-09-10T00:39:16.526669+0000","last_became_active":"2023-09-10T00:38:39.761383+0000","last_became_peered":"2023-09-10T00:38:39.761383+0000","last_unstale":"2023-09-10T00:39:16.526669+0000","last_undegraded":"2023-09-10T00:39:16.526669+0000","last_fullsized":"2023-09-10T00:39:16.526669+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:38:04.534656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527509+0000","last_change":"2023-09-10T00:38:39.765028+0000","last_active":"2023-09-10T00:39:16.527509+0000","last_peered":"2023-09-10T00:39:16.527509+0000","last_clean":"2023-09-10T00:39:16.527509+0000","last_became_active":"2023-09-10T00:38:39.764663+0000","last_became_peered":"2023-09-10T00:38:39.764663+0000","last_unstale":"2023-09-10T00:39:16.527509+0000","last_undegraded":"2023-09-10T00:39:16.527509+0000","last_fullsized":"2023-09-10T00:39:16.527509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:14:49.645129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.536787+0000","last_change":"2023-09-10T00:38:39.770135+0000","last_active":"2023-09-10T00:39:16.536787+0000","last_peered":"2023-09-10T00:39:16.536787+0000","last_clean":"2023-09-10T00:39:16.536787+0000","last_became_active":"2023-09-10T00:38:39.769720+0000","last_became_peered":"2023-09-10T00:38:39.769720+0000","last_unstale":"2023-09-10T00:39:16.536787+0000","last_undegraded":"2023-09-10T00:39:16.536787+0000","last_fullsized":"2023-09-10T00:39:16.536787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:46:21.166566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526931+0000","last_change":"2023-09-10T00:38:39.770170+0000","last_active":"2023-09-10T00:39:16.526931+0000","last_peered":"2023-09-10T00:39:16.526931+0000","last_clean":"2023-09-10T00:39:16.526931+0000","last_became_active":"2023-09-10T00:38:39.769840+0000","last_became_peered":"2023-09-10T00:38:39.769840+0000","last_unstale":"2023-09-10T00:39:16.526931+0000","last_undegraded":"2023-09-10T00:39:16.526931+0000","last_fullsized":"2023-09-10T00:39:16.526931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:48.117670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527116+0000","last_change":"2023-09-10T00:38:39.769621+0000","last_active":"2023-09-10T00:39:16.527116+0000","last_peered":"2023-09-10T00:39:16.527116+0000","last_clean":"2023-09-10T00:39:16.527116+0000","last_became_active":"2023-09-10T00:38:39.769300+0000","last_became_peered":"2023-09-10T00:38:39.769300+0000","last_unstale":"2023-09-10T00:39:16.527116+0000","last_undegraded":"2023-09-10T00:39:16.527116+0000","last_fullsized":"2023-09-10T00:39:16.527116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:37.100671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527669+0000","last_change":"2023-09-10T00:38:39.772200+0000","last_active":"2023-09-10T00:39:16.527669+0000","last_peered":"2023-09-10T00:39:16.527669+0000","last_clean":"2023-09-10T00:39:16.527669+0000","last_became_active":"2023-09-10T00:38:39.771503+0000","last_became_peered":"2023-09-10T00:38:39.771503+0000","last_unstale":"2023-09-10T00:39:16.527669+0000","last_undegraded":"2023-09-10T00:39:16.527669+0000","last_fullsized":"2023-09-10T00:39:16.527669+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:13:38.830642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522841+0000","last_change":"2023-09-10T00:38:39.769832+0000","last_active":"2023-09-10T00:39:16.522841+0000","last_peered":"2023-09-10T00:39:16.522841+0000","last_clean":"2023-09-10T00:39:16.522841+0000","last_became_active":"2023-09-10T00:38:39.769546+0000","last_became_peered":"2023-09-10T00:38:39.769546+0000","last_unstale":"2023-09-10T00:39:16.522841+0000","last_undegraded":"2023-09-10T00:39:16.522841+0000","last_fullsized":"2023-09-10T00:39:16.522841+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:53:05.219877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.531430+0000","last_change":"2023-09-10T00:39:17.531430+0000","last_active":"2023-09-10T00:39:17.531430+0000","last_peered":"2023-09-10T00:39:17.531430+0000","last_clean":"2023-09-10T00:39:17.531430+0000","last_became_active":"2023-09-10T00:39:17.531145+0000","last_became_peered":"2023-09-10T00:39:17.531145+0000","last_unstale":"2023-09-10T00:39:17.531430+0000","last_undegraded":"2023-09-10T00:39:17.531430+0000","last_fullsized":"2023-09-10T00:39:17.531430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:34:50.163747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527248+0000","last_change":"2023-09-10T00:38:39.764751+0000","last_active":"2023-09-10T00:39:16.527248+0000","last_peered":"2023-09-10T00:39:16.527248+0000","last_clean":"2023-09-10T00:39:16.527248+0000","last_became_active":"2023-09-10T00:38:39.764464+0000","last_became_peered":"2023-09-10T00:38:39.764464+0000","last_unstale":"2023-09-10T00:39:16.527248+0000","last_undegraded":"2023-09-10T00:39:16.527248+0000","last_fullsized":"2023-09-10T00:39:16.527248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:03:51.096126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.532977+0000","last_change":"2023-09-10T00:39:17.532977+0000","last_active":"2023-09-10T00:39:17.532977+0000","last_peered":"2023-09-10T00:39:17.532977+0000","last_clean":"2023-09-10T00:39:17.532977+0000","last_became_active":"2023-09-10T00:39:17.532547+0000","last_became_peered":"2023-09-10T00:39:17.532547+0000","last_unstale":"2023-09-10T00:39:17.532977+0000","last_undegraded":"2023-09-10T00:39:17.532977+0000","last_fullsized":"2023-09-10T00:39:17.532977+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:33:01.685743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.525442+0000","last_change":"2023-09-10T00:38:39.770787+0000","last_active":"2023-09-10T00:39:16.525442+0000","last_peered":"2023-09-10T00:39:16.525442+0000","last_clean":"2023-09-10T00:39:16.525442+0000","last_became_active":"2023-09-10T00:38:39.770497+0000","last_became_peered":"2023-09-10T00:38:39.770497+0000","last_unstale":"2023-09-10T00:39:16.525442+0000","last_undegraded":"2023-09-10T00:39:16.525442+0000","last_fullsized":"2023-09-10T00:39:16.525442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:43:11.066766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.519978+0000","last_change":"2023-09-10T00:38:39.768008+0000","last_active":"2023-09-10T00:39:16.519978+0000","last_peered":"2023-09-10T00:39:16.519978+0000","last_clean":"2023-09-10T00:39:16.519978+0000","last_became_active":"2023-09-10T00:38:39.767079+0000","last_became_peered":"2023-09-10T00:38:39.767079+0000","last_unstale":"2023-09-10T00:39:16.519978+0000","last_undegraded":"2023-09-10T00:39:16.519978+0000","last_fullsized":"2023-09-10T00:39:16.519978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:34:00.689881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.532177+0000","last_change":"2023-09-10T00:38:39.775326+0000","last_active":"2023-09-10T00:39:16.532177+0000","last_peered":"2023-09-10T00:39:16.532177+0000","last_clean":"2023-09-10T00:39:16.532177+0000","last_became_active":"2023-09-10T00:38:39.775007+0000","last_became_peered":"2023-09-10T00:38:39.775007+0000","last_unstale":"2023-09-10T00:39:16.532177+0000","last_undegraded":"2023-09-10T00:39:16.532177+0000","last_fullsized":"2023-09-10T00:39:16.532177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:59:27.711722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527686+0000","last_change":"2023-09-10T00:38:39.764974+0000","last_active":"2023-09-10T00:39:16.527686+0000","last_peered":"2023-09-10T00:39:16.527686+0000","last_clean":"2023-09-10T00:39:16.527686+0000","last_became_active":"2023-09-10T00:38:39.764368+0000","last_became_peered":"2023-09-10T00:38:39.764368+0000","last_unstale":"2023-09-10T00:39:16.527686+0000","last_undegraded":"2023-09-10T00:39:16.527686+0000","last_fullsized":"2023-09-10T00:39:16.527686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:40:28.046021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526817+0000","last_change":"2023-09-10T00:38:43.793589+0000","last_active":"2023-09-10T00:39:16.526817+0000","last_peered":"2023-09-10T00:39:16.526817+0000","last_clean":"2023-09-10T00:39:16.526817+0000","last_became_active":"2023-09-10T00:38:43.793166+0000","last_became_peered":"2023-09-10T00:38:43.793166+0000","last_unstale":"2023-09-10T00:39:16.526817+0000","last_undegraded":"2023-09-10T00:39:16.526817+0000","last_fullsized":"2023-09-10T00:39:16.526817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:51:47.128925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":71,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:39.589556+0000","last_change":"2023-09-10T00:38:38.759909+0000","last_active":"2023-09-10T00:39:39.589556+0000","last_peered":"2023-09-10T00:39:39.589556+0000","last_clean":"2023-09-10T00:39:39.589556+0000","last_became_active":"2023-09-10T00:38:38.759561+0000","last_became_peered":"2023-09-10T00:38:38.759561+0000","last_unstale":"2023-09-10T00:39:39.589556+0000","last_undegraded":"2023-09-10T00:39:39.589556+0000","last_fullsized":"2023-09-10T00:39:39.589556+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:56:18.009497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.534652+0000","last_change":"2023-09-10T00:38:39.774916+0000","last_active":"2023-09-10T00:39:16.534652+0000","last_peered":"2023-09-10T00:39:16.534652+0000","last_clean":"2023-09-10T00:39:16.534652+0000","last_became_active":"2023-09-10T00:38:39.774584+0000","last_became_peered":"2023-09-10T00:38:39.774584+0000","last_unstale":"2023-09-10T00:39:16.534652+0000","last_undegraded":"2023-09-10T00:39:16.534652+0000","last_fullsized":"2023-09-10T00:39:16.534652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:36:33.125742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522181+0000","last_change":"2023-09-10T00:38:43.789421+0000","last_active":"2023-09-10T00:39:16.522181+0000","last_peered":"2023-09-10T00:39:16.522181+0000","last_clean":"2023-09-10T00:39:16.522181+0000","last_became_active":"2023-09-10T00:38:43.788569+0000","last_became_peered":"2023-09-10T00:38:43.788569+0000","last_unstale":"2023-09-10T00:39:16.522181+0000","last_undegraded":"2023-09-10T00:39:16.522181+0000","last_fullsized":"2023-09-10T00:39:16.522181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:07:38.529811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.534752+0000","last_change":"2023-09-10T00:38:38.767381+0000","last_active":"2023-09-10T00:39:16.534752+0000","last_peered":"2023-09-10T00:39:16.534752+0000","last_clean":"2023-09-10T00:39:16.534752+0000","last_became_active":"2023-09-10T00:38:38.767082+0000","last_became_peered":"2023-09-10T00:38:38.767082+0000","last_unstale":"2023-09-10T00:39:16.534752+0000","last_undegraded":"2023-09-10T00:39:16.534752+0000","last_fullsized":"2023-09-10T00:39:16.534752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:13:32.533215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.528233+0000","last_change":"2023-09-10T00:38:39.772645+0000","last_active":"2023-09-10T00:39:16.528233+0000","last_peered":"2023-09-10T00:39:16.528233+0000","last_clean":"2023-09-10T00:39:16.528233+0000","last_became_active":"2023-09-10T00:38:39.771554+0000","last_became_peered":"2023-09-10T00:38:39.771554+0000","last_unstale":"2023-09-10T00:39:16.528233+0000","last_undegraded":"2023-09-10T00:39:16.528233+0000","last_fullsized":"2023-09-10T00:39:16.528233+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:25:06.237309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522555+0000","last_change":"2023-09-10T00:38:43.795764+0000","last_active":"2023-09-10T00:39:16.522555+0000","last_peered":"2023-09-10T00:39:16.522555+0000","last_clean":"2023-09-10T00:39:16.522555+0000","last_became_active":"2023-09-10T00:38:43.795112+0000","last_became_peered":"2023-09-10T00:38:43.795112+0000","last_unstale":"2023-09-10T00:39:16.522555+0000","last_undegraded":"2023-09-10T00:39:16.522555+0000","last_fullsized":"2023-09-10T00:39:16.522555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:18:55.460752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.524242+0000","last_change":"2023-09-10T00:38:38.765522+0000","last_active":"2023-09-10T00:39:16.524242+0000","last_peered":"2023-09-10T00:39:16.524242+0000","last_clean":"2023-09-10T00:39:16.524242+0000","last_became_active":"2023-09-10T00:38:38.765219+0000","last_became_peered":"2023-09-10T00:38:38.765219+0000","last_unstale":"2023-09-10T00:39:16.524242+0000","last_undegraded":"2023-09-10T00:39:16.524242+0000","last_fullsized":"2023-09-10T00:39:16.524242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:00:27.275269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526550+0000","last_change":"2023-09-10T00:38:39.767790+0000","last_active":"2023-09-10T00:39:16.526550+0000","last_peered":"2023-09-10T00:39:16.526550+0000","last_clean":"2023-09-10T00:39:16.526550+0000","last_became_active":"2023-09-10T00:38:39.767513+0000","last_became_peered":"2023-09-10T00:38:39.767513+0000","last_unstale":"2023-09-10T00:39:16.526550+0000","last_undegraded":"2023-09-10T00:39:16.526550+0000","last_fullsized":"2023-09-10T00:39:16.526550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:05:48.223951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522271+0000","last_change":"2023-09-10T00:38:43.789089+0000","last_active":"2023-09-10T00:39:16.522271+0000","last_peered":"2023-09-10T00:39:16.522271+0000","last_clean":"2023-09-10T00:39:16.522271+0000","last_became_active":"2023-09-10T00:38:43.788434+0000","last_became_peered":"2023-09-10T00:38:43.788434+0000","last_unstale":"2023-09-10T00:39:16.522271+0000","last_undegraded":"2023-09-10T00:39:16.522271+0000","last_fullsized":"2023-09-10T00:39:16.522271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:55:00.236088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.523380+0000","last_change":"2023-09-10T00:38:38.769478+0000","last_active":"2023-09-10T00:39:16.523380+0000","last_peered":"2023-09-10T00:39:16.523380+0000","last_clean":"2023-09-10T00:39:16.523380+0000","last_became_active":"2023-09-10T00:38:38.769168+0000","last_became_peered":"2023-09-10T00:38:38.769168+0000","last_unstale":"2023-09-10T00:39:16.523380+0000","last_undegraded":"2023-09-10T00:39:16.523380+0000","last_fullsized":"2023-09-10T00:39:16.523380+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:03:48.133561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.525752+0000","last_change":"2023-09-10T00:38:39.771843+0000","last_active":"2023-09-10T00:39:16.525752+0000","last_peered":"2023-09-10T00:39:16.525752+0000","last_clean":"2023-09-10T00:39:16.525752+0000","last_became_active":"2023-09-10T00:38:39.771510+0000","last_became_peered":"2023-09-10T00:38:39.771510+0000","last_unstale":"2023-09-10T00:39:16.525752+0000","last_undegraded":"2023-09-10T00:39:16.525752+0000","last_fullsized":"2023-09-10T00:39:16.525752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:17:39.996490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527399+0000","last_change":"2023-09-10T00:38:43.786853+0000","last_active":"2023-09-10T00:39:16.527399+0000","last_peered":"2023-09-10T00:39:16.527399+0000","last_clean":"2023-09-10T00:39:16.527399+0000","last_became_active":"2023-09-10T00:38:43.786386+0000","last_became_peered":"2023-09-10T00:38:43.786386+0000","last_unstale":"2023-09-10T00:39:16.527399+0000","last_undegraded":"2023-09-10T00:39:16.527399+0000","last_fullsized":"2023-09-10T00:39:16.527399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:23:53.724653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":68,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:38.529849+0000","last_change":"2023-09-10T00:38:38.769004+0000","last_active":"2023-09-10T00:39:38.529849+0000","last_peered":"2023-09-10T00:39:38.529849+0000","last_clean":"2023-09-10T00:39:38.529849+0000","last_became_active":"2023-09-10T00:38:38.768690+0000","last_became_peered":"2023-09-10T00:38:38.768690+0000","last_unstale":"2023-09-10T00:39:38.529849+0000","last_undegraded":"2023-09-10T00:39:38.529849+0000","last_fullsized":"2023-09-10T00:39:38.529849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:04:42.531092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.523666+0000","last_change":"2023-09-10T00:38:39.773174+0000","last_active":"2023-09-10T00:39:16.523666+0000","last_peered":"2023-09-10T00:39:16.523666+0000","last_clean":"2023-09-10T00:39:16.523666+0000","last_became_active":"2023-09-10T00:38:39.771192+0000","last_became_peered":"2023-09-10T00:38:39.771192+0000","last_unstale":"2023-09-10T00:39:16.523666+0000","last_undegraded":"2023-09-10T00:39:16.523666+0000","last_fullsized":"2023-09-10T00:39:16.523666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:16:25.034024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527162+0000","last_change":"2023-09-10T00:38:43.786661+0000","last_active":"2023-09-10T00:39:16.527162+0000","last_peered":"2023-09-10T00:39:16.527162+0000","last_clean":"2023-09-10T00:39:16.527162+0000","last_became_active":"2023-09-10T00:38:43.786242+0000","last_became_peered":"2023-09-10T00:38:43.786242+0000","last_unstale":"2023-09-10T00:39:16.527162+0000","last_undegraded":"2023-09-10T00:39:16.527162+0000","last_fullsized":"2023-09-10T00:39:16.527162+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:36:14.490085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526572+0000","last_change":"2023-09-10T00:38:38.763416+0000","last_active":"2023-09-10T00:39:16.526572+0000","last_peered":"2023-09-10T00:39:16.526572+0000","last_clean":"2023-09-10T00:39:16.526572+0000","last_became_active":"2023-09-10T00:38:38.763054+0000","last_became_peered":"2023-09-10T00:38:38.763054+0000","last_unstale":"2023-09-10T00:39:16.526572+0000","last_undegraded":"2023-09-10T00:39:16.526572+0000","last_fullsized":"2023-09-10T00:39:16.526572+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:47:00.179467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.523311+0000","last_change":"2023-09-10T00:38:39.773284+0000","last_active":"2023-09-10T00:39:16.523311+0000","last_peered":"2023-09-10T00:39:16.523311+0000","last_clean":"2023-09-10T00:39:16.523311+0000","last_became_active":"2023-09-10T00:38:39.772969+0000","last_became_peered":"2023-09-10T00:38:39.772969+0000","last_unstale":"2023-09-10T00:39:16.523311+0000","last_undegraded":"2023-09-10T00:39:16.523311+0000","last_fullsized":"2023-09-10T00:39:16.523311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:46:45.013879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.519914+0000","last_change":"2023-09-10T00:38:43.790810+0000","last_active":"2023-09-10T00:39:16.519914+0000","last_peered":"2023-09-10T00:39:16.519914+0000","last_clean":"2023-09-10T00:39:16.519914+0000","last_became_active":"2023-09-10T00:38:43.790429+0000","last_became_peered":"2023-09-10T00:38:43.790429+0000","last_unstale":"2023-09-10T00:39:16.519914+0000","last_undegraded":"2023-09-10T00:39:16.519914+0000","last_fullsized":"2023-09-10T00:39:16.519914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:30:56.072587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.553599+0000","last_change":"2023-09-10T00:38:38.754326+0000","last_active":"2023-09-10T00:39:16.553599+0000","last_peered":"2023-09-10T00:39:16.553599+0000","last_clean":"2023-09-10T00:39:16.553599+0000","last_became_active":"2023-09-10T00:38:38.754006+0000","last_became_peered":"2023-09-10T00:38:38.754006+0000","last_unstale":"2023-09-10T00:39:16.553599+0000","last_undegraded":"2023-09-10T00:39:16.553599+0000","last_fullsized":"2023-09-10T00:39:16.553599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:09:26.983606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.536700+0000","last_change":"2023-09-10T00:38:39.769699+0000","last_active":"2023-09-10T00:39:16.536700+0000","last_peered":"2023-09-10T00:39:16.536700+0000","last_clean":"2023-09-10T00:39:16.536700+0000","last_became_active":"2023-09-10T00:38:39.769362+0000","last_became_peered":"2023-09-10T00:38:39.769362+0000","last_unstale":"2023-09-10T00:39:16.536700+0000","last_undegraded":"2023-09-10T00:39:16.536700+0000","last_fullsized":"2023-09-10T00:39:16.536700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:30:20.938437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.528806+0000","last_change":"2023-09-10T00:38:43.785669+0000","last_active":"2023-09-10T00:39:16.528806+0000","last_peered":"2023-09-10T00:39:16.528806+0000","last_clean":"2023-09-10T00:39:16.528806+0000","last_became_active":"2023-09-10T00:38:43.785255+0000","last_became_peered":"2023-09-10T00:38:43.785255+0000","last_unstale":"2023-09-10T00:39:16.528806+0000","last_undegraded":"2023-09-10T00:39:16.528806+0000","last_fullsized":"2023-09-10T00:39:16.528806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:52:57.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522146+0000","last_change":"2023-09-10T00:38:38.767997+0000","last_active":"2023-09-10T00:39:16.522146+0000","last_peered":"2023-09-10T00:39:16.522146+0000","last_clean":"2023-09-10T00:39:16.522146+0000","last_became_active":"2023-09-10T00:38:38.767715+0000","last_became_peered":"2023-09-10T00:38:38.767715+0000","last_unstale":"2023-09-10T00:39:16.522146+0000","last_undegraded":"2023-09-10T00:39:16.522146+0000","last_fullsized":"2023-09-10T00:39:16.522146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:11:17.047280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527273+0000","last_change":"2023-09-10T00:38:39.778004+0000","last_active":"2023-09-10T00:39:16.527273+0000","last_peered":"2023-09-10T00:39:16.527273+0000","last_clean":"2023-09-10T00:39:16.527273+0000","last_became_active":"2023-09-10T00:38:39.777632+0000","last_became_peered":"2023-09-10T00:38:39.777632+0000","last_unstale":"2023-09-10T00:39:16.527273+0000","last_undegraded":"2023-09-10T00:39:16.527273+0000","last_fullsized":"2023-09-10T00:39:16.527273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:46:40.804385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.533338+0000","last_change":"2023-09-10T00:38:43.791397+0000","last_active":"2023-09-10T00:39:16.533338+0000","last_peered":"2023-09-10T00:39:16.533338+0000","last_clean":"2023-09-10T00:39:16.533338+0000","last_became_active":"2023-09-10T00:38:43.791010+0000","last_became_peered":"2023-09-10T00:38:43.791010+0000","last_unstale":"2023-09-10T00:39:16.533338+0000","last_undegraded":"2023-09-10T00:39:16.533338+0000","last_fullsized":"2023-09-10T00:39:16.533338+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:52:26.015718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":100,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:39.530817+0000","last_change":"2023-09-10T00:38:38.767399+0000","last_active":"2023-09-10T00:39:39.530817+0000","last_peered":"2023-09-10T00:39:39.530817+0000","last_clean":"2023-09-10T00:39:39.530817+0000","last_became_active":"2023-09-10T00:38:38.767176+0000","last_became_peered":"2023-09-10T00:38:38.767176+0000","last_unstale":"2023-09-10T00:39:39.530817+0000","last_undegraded":"2023-09-10T00:39:39.530817+0000","last_fullsized":"2023-09-10T00:39:39.530817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:17:24.481985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.525581+0000","last_change":"2023-09-10T00:38:39.766013+0000","last_active":"2023-09-10T00:39:16.525581+0000","last_peered":"2023-09-10T00:39:16.525581+0000","last_clean":"2023-09-10T00:39:16.525581+0000","last_became_active":"2023-09-10T00:38:39.765235+0000","last_became_peered":"2023-09-10T00:38:39.765235+0000","last_unstale":"2023-09-10T00:39:16.525581+0000","last_undegraded":"2023-09-10T00:39:16.525581+0000","last_fullsized":"2023-09-10T00:39:16.525581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:01:48.874845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526918+0000","last_change":"2023-09-10T00:38:43.790361+0000","last_active":"2023-09-10T00:39:16.526918+0000","last_peered":"2023-09-10T00:39:16.526918+0000","last_clean":"2023-09-10T00:39:16.526918+0000","last_became_active":"2023-09-10T00:38:43.789966+0000","last_became_peered":"2023-09-10T00:38:43.789966+0000","last_unstale":"2023-09-10T00:39:16.526918+0000","last_undegraded":"2023-09-10T00:39:16.526918+0000","last_fullsized":"2023-09-10T00:39:16.526918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:46:37.978258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527984+0000","last_change":"2023-09-10T00:38:38.771670+0000","last_active":"2023-09-10T00:39:16.527984+0000","last_peered":"2023-09-10T00:39:16.527984+0000","last_clean":"2023-09-10T00:39:16.527984+0000","last_became_active":"2023-09-10T00:38:38.771339+0000","last_became_peered":"2023-09-10T00:38:38.771339+0000","last_unstale":"2023-09-10T00:39:16.527984+0000","last_undegraded":"2023-09-10T00:39:16.527984+0000","last_fullsized":"2023-09-10T00:39:16.527984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:22:32.658544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.528315+0000","last_change":"2023-09-10T00:38:39.772857+0000","last_active":"2023-09-10T00:39:16.528315+0000","last_peered":"2023-09-10T00:39:16.528315+0000","last_clean":"2023-09-10T00:39:16.528315+0000","last_became_active":"2023-09-10T00:38:39.771990+0000","last_became_peered":"2023-09-10T00:38:39.771990+0000","last_unstale":"2023-09-10T00:39:16.528315+0000","last_undegraded":"2023-09-10T00:39:16.528315+0000","last_fullsized":"2023-09-10T00:39:16.528315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:16:15.767748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522691+0000","last_change":"2023-09-10T00:38:43.795482+0000","last_active":"2023-09-10T00:39:16.522691+0000","last_peered":"2023-09-10T00:39:16.522691+0000","last_clean":"2023-09-10T00:39:16.522691+0000","last_became_active":"2023-09-10T00:38:43.795048+0000","last_became_peered":"2023-09-10T00:38:43.795048+0000","last_unstale":"2023-09-10T00:39:16.522691+0000","last_undegraded":"2023-09-10T00:39:16.522691+0000","last_fullsized":"2023-09-10T00:39:16.522691+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:13:18.715274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.523679+0000","last_change":"2023-09-10T00:38:38.771008+0000","last_active":"2023-09-10T00:39:16.523679+0000","last_peered":"2023-09-10T00:39:16.523679+0000","last_clean":"2023-09-10T00:39:16.523679+0000","last_became_active":"2023-09-10T00:38:38.770714+0000","last_became_peered":"2023-09-10T00:38:38.770714+0000","last_unstale":"2023-09-10T00:39:16.523679+0000","last_undegraded":"2023-09-10T00:39:16.523679+0000","last_fullsized":"2023-09-10T00:39:16.523679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:04:30.148141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.533500+0000","last_change":"2023-09-10T00:39:17.533500+0000","last_active":"2023-09-10T00:39:17.533500+0000","last_peered":"2023-09-10T00:39:17.533500+0000","last_clean":"2023-09-10T00:39:17.533500+0000","last_became_active":"2023-09-10T00:39:17.533175+0000","last_became_peered":"2023-09-10T00:39:17.533175+0000","last_unstale":"2023-09-10T00:39:17.533500+0000","last_undegraded":"2023-09-10T00:39:17.533500+0000","last_fullsized":"2023-09-10T00:39:17.533500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:13:42.121699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.537288+0000","last_change":"2023-09-10T00:39:17.536684+0000","last_active":"2023-09-10T00:39:17.537288+0000","last_peered":"2023-09-10T00:39:17.537288+0000","last_clean":"2023-09-10T00:39:17.537288+0000","last_became_active":"2023-09-10T00:39:17.536170+0000","last_became_peered":"2023-09-10T00:39:17.536170+0000","last_unstale":"2023-09-10T00:39:17.537288+0000","last_undegraded":"2023-09-10T00:39:17.537288+0000","last_fullsized":"2023-09-10T00:39:17.537288+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:47:16.264619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":139,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:39.530802+0000","last_change":"2023-09-10T00:38:38.761072+0000","last_active":"2023-09-10T00:39:39.530802+0000","last_peered":"2023-09-10T00:39:39.530802+0000","last_clean":"2023-09-10T00:39:39.530802+0000","last_became_active":"2023-09-10T00:38:38.760752+0000","last_became_peered":"2023-09-10T00:38:38.760752+0000","last_unstale":"2023-09-10T00:39:39.530802+0000","last_undegraded":"2023-09-10T00:39:39.530802+0000","last_fullsized":"2023-09-10T00:39:39.530802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:55:18.272741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58599,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":88,"num_read_kb":101,"num_write":35,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.523956+0000","last_change":"2023-09-10T00:38:39.767507+0000","last_active":"2023-09-10T00:39:16.523956+0000","last_peered":"2023-09-10T00:39:16.523956+0000","last_clean":"2023-09-10T00:39:16.523956+0000","last_became_active":"2023-09-10T00:38:39.767223+0000","last_became_peered":"2023-09-10T00:38:39.767223+0000","last_unstale":"2023-09-10T00:39:16.523956+0000","last_undegraded":"2023-09-10T00:39:16.523956+0000","last_fullsized":"2023-09-10T00:39:16.523956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:46:30.901818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526139+0000","last_change":"2023-09-10T00:38:43.785396+0000","last_active":"2023-09-10T00:39:16.526139+0000","last_peered":"2023-09-10T00:39:16.526139+0000","last_clean":"2023-09-10T00:39:16.526139+0000","last_became_active":"2023-09-10T00:38:43.785038+0000","last_became_peered":"2023-09-10T00:38:43.785038+0000","last_unstale":"2023-09-10T00:39:16.526139+0000","last_undegraded":"2023-09-10T00:39:16.526139+0000","last_fullsized":"2023-09-10T00:39:16.526139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:30:31.629703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527248+0000","last_change":"2023-09-10T00:38:38.766742+0000","last_active":"2023-09-10T00:39:16.527248+0000","last_peered":"2023-09-10T00:39:16.527248+0000","last_clean":"2023-09-10T00:39:16.527248+0000","last_became_active":"2023-09-10T00:38:38.766220+0000","last_became_peered":"2023-09-10T00:38:38.766220+0000","last_unstale":"2023-09-10T00:39:16.527248+0000","last_undegraded":"2023-09-10T00:39:16.527248+0000","last_fullsized":"2023-09-10T00:39:16.527248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:41:22.030448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.537227+0000","last_change":"2023-09-10T00:38:39.766022+0000","last_active":"2023-09-10T00:39:16.537227+0000","last_peered":"2023-09-10T00:39:16.537227+0000","last_clean":"2023-09-10T00:39:16.537227+0000","last_became_active":"2023-09-10T00:38:39.765697+0000","last_became_peered":"2023-09-10T00:38:39.765697+0000","last_unstale":"2023-09-10T00:39:16.537227+0000","last_undegraded":"2023-09-10T00:39:16.537227+0000","last_fullsized":"2023-09-10T00:39:16.537227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:19:39.394299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.519687+0000","last_change":"2023-09-10T00:38:43.795492+0000","last_active":"2023-09-10T00:39:16.519687+0000","last_peered":"2023-09-10T00:39:16.519687+0000","last_clean":"2023-09-10T00:39:16.519687+0000","last_became_active":"2023-09-10T00:38:43.794968+0000","last_became_peered":"2023-09-10T00:38:43.794968+0000","last_unstale":"2023-09-10T00:39:16.519687+0000","last_undegraded":"2023-09-10T00:39:16.519687+0000","last_fullsized":"2023-09-10T00:39:16.519687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:40:43.558870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522912+0000","last_change":"2023-09-10T00:38:38.767666+0000","last_active":"2023-09-10T00:39:16.522912+0000","last_peered":"2023-09-10T00:39:16.522912+0000","last_clean":"2023-09-10T00:39:16.522912+0000","last_became_active":"2023-09-10T00:38:38.767333+0000","last_became_peered":"2023-09-10T00:38:38.767333+0000","last_unstale":"2023-09-10T00:39:16.522912+0000","last_undegraded":"2023-09-10T00:39:16.522912+0000","last_fullsized":"2023-09-10T00:39:16.522912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:07:53.363196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527421+0000","last_change":"2023-09-10T00:38:39.766036+0000","last_active":"2023-09-10T00:39:16.527421+0000","last_peered":"2023-09-10T00:39:16.527421+0000","last_clean":"2023-09-10T00:39:16.527421+0000","last_became_active":"2023-09-10T00:38:39.765683+0000","last_became_peered":"2023-09-10T00:38:39.765683+0000","last_unstale":"2023-09-10T00:39:16.527421+0000","last_undegraded":"2023-09-10T00:39:16.527421+0000","last_fullsized":"2023-09-10T00:39:16.527421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:58:04.429960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.532739+0000","last_change":"2023-09-10T00:38:43.787870+0000","last_active":"2023-09-10T00:39:16.532739+0000","last_peered":"2023-09-10T00:39:16.532739+0000","last_clean":"2023-09-10T00:39:16.532739+0000","last_became_active":"2023-09-10T00:38:43.785915+0000","last_became_peered":"2023-09-10T00:38:43.785915+0000","last_unstale":"2023-09-10T00:39:16.532739+0000","last_undegraded":"2023-09-10T00:39:16.532739+0000","last_fullsized":"2023-09-10T00:39:16.532739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:34:08.623967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":70,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:41.539298+0000","last_change":"2023-09-10T00:38:38.756925+0000","last_active":"2023-09-10T00:39:41.539298+0000","last_peered":"2023-09-10T00:39:41.539298+0000","last_clean":"2023-09-10T00:39:41.539298+0000","last_became_active":"2023-09-10T00:38:38.756610+0000","last_became_peered":"2023-09-10T00:38:38.756610+0000","last_unstale":"2023-09-10T00:39:41.539298+0000","last_undegraded":"2023-09-10T00:39:41.539298+0000","last_fullsized":"2023-09-10T00:39:41.539298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:48:15.385230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526743+0000","last_change":"2023-09-10T00:38:39.766750+0000","last_active":"2023-09-10T00:39:16.526743+0000","last_peered":"2023-09-10T00:39:16.526743+0000","last_clean":"2023-09-10T00:39:16.526743+0000","last_became_active":"2023-09-10T00:38:39.766458+0000","last_became_peered":"2023-09-10T00:38:39.766458+0000","last_unstale":"2023-09-10T00:39:16.526743+0000","last_undegraded":"2023-09-10T00:39:16.526743+0000","last_fullsized":"2023-09-10T00:39:16.526743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:59:26.691487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.538243+0000","last_change":"2023-09-10T00:38:43.788814+0000","last_active":"2023-09-10T00:39:16.538243+0000","last_peered":"2023-09-10T00:39:16.538243+0000","last_clean":"2023-09-10T00:39:16.538243+0000","last_became_active":"2023-09-10T00:38:43.788446+0000","last_became_peered":"2023-09-10T00:38:43.788446+0000","last_unstale":"2023-09-10T00:39:16.538243+0000","last_undegraded":"2023-09-10T00:39:16.538243+0000","last_fullsized":"2023-09-10T00:39:16.538243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:38:53.308160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522627+0000","last_change":"2023-09-10T00:38:38.766020+0000","last_active":"2023-09-10T00:39:16.522627+0000","last_peered":"2023-09-10T00:39:16.522627+0000","last_clean":"2023-09-10T00:39:16.522627+0000","last_became_active":"2023-09-10T00:38:38.765719+0000","last_became_peered":"2023-09-10T00:38:38.765719+0000","last_unstale":"2023-09-10T00:39:16.522627+0000","last_undegraded":"2023-09-10T00:39:16.522627+0000","last_fullsized":"2023-09-10T00:39:16.522627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:25:30.488014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522546+0000","last_change":"2023-09-10T00:38:39.771091+0000","last_active":"2023-09-10T00:39:16.522546+0000","last_peered":"2023-09-10T00:39:16.522546+0000","last_clean":"2023-09-10T00:39:16.522546+0000","last_became_active":"2023-09-10T00:38:39.770745+0000","last_became_peered":"2023-09-10T00:38:39.770745+0000","last_unstale":"2023-09-10T00:39:16.522546+0000","last_undegraded":"2023-09-10T00:39:16.522546+0000","last_fullsized":"2023-09-10T00:39:16.522546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:51:17.978466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522440+0000","last_change":"2023-09-10T00:38:43.793757+0000","last_active":"2023-09-10T00:39:16.522440+0000","last_peered":"2023-09-10T00:39:16.522440+0000","last_clean":"2023-09-10T00:39:16.522440+0000","last_became_active":"2023-09-10T00:38:43.793336+0000","last_became_peered":"2023-09-10T00:38:43.793336+0000","last_unstale":"2023-09-10T00:39:16.522440+0000","last_undegraded":"2023-09-10T00:39:16.522440+0000","last_fullsized":"2023-09-10T00:39:16.522440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:22:36.544950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.525478+0000","last_change":"2023-09-10T00:38:38.769373+0000","last_active":"2023-09-10T00:39:16.525478+0000","last_peered":"2023-09-10T00:39:16.525478+0000","last_clean":"2023-09-10T00:39:16.525478+0000","last_became_active":"2023-09-10T00:38:38.769054+0000","last_became_peered":"2023-09-10T00:38:38.769054+0000","last_unstale":"2023-09-10T00:39:16.525478+0000","last_undegraded":"2023-09-10T00:39:16.525478+0000","last_fullsized":"2023-09-10T00:39:16.525478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:12:27.982405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.537427+0000","last_change":"2023-09-10T00:39:17.537427+0000","last_active":"2023-09-10T00:39:17.537427+0000","last_peered":"2023-09-10T00:39:17.537427+0000","last_clean":"2023-09-10T00:39:17.537427+0000","last_became_active":"2023-09-10T00:39:17.537023+0000","last_became_peered":"2023-09-10T00:39:17.537023+0000","last_unstale":"2023-09-10T00:39:17.537427+0000","last_undegraded":"2023-09-10T00:39:17.537427+0000","last_fullsized":"2023-09-10T00:39:17.537427+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:54:43.720224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.523517+0000","last_change":"2023-09-10T00:38:43.838689+0000","last_active":"2023-09-10T00:39:16.523517+0000","last_peered":"2023-09-10T00:39:16.523517+0000","last_clean":"2023-09-10T00:39:16.523517+0000","last_became_active":"2023-09-10T00:38:43.838244+0000","last_became_peered":"2023-09-10T00:38:43.838244+0000","last_unstale":"2023-09-10T00:39:16.523517+0000","last_undegraded":"2023-09-10T00:39:16.523517+0000","last_fullsized":"2023-09-10T00:39:16.523517+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:54:00.033719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.528925+0000","last_change":"2023-09-10T00:38:38.765140+0000","last_active":"2023-09-10T00:39:16.528925+0000","last_peered":"2023-09-10T00:39:16.528925+0000","last_clean":"2023-09-10T00:39:16.528925+0000","last_became_active":"2023-09-10T00:38:38.764730+0000","last_became_peered":"2023-09-10T00:38:38.764730+0000","last_unstale":"2023-09-10T00:39:16.528925+0000","last_undegraded":"2023-09-10T00:39:16.528925+0000","last_fullsized":"2023-09-10T00:39:16.528925+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:49:41.129763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526914+0000","last_change":"2023-09-10T00:38:39.773260+0000","last_active":"2023-09-10T00:39:16.526914+0000","last_peered":"2023-09-10T00:39:16.526914+0000","last_clean":"2023-09-10T00:39:16.526914+0000","last_became_active":"2023-09-10T00:38:39.772947+0000","last_became_peered":"2023-09-10T00:38:39.772947+0000","last_unstale":"2023-09-10T00:39:16.526914+0000","last_undegraded":"2023-09-10T00:39:16.526914+0000","last_fullsized":"2023-09-10T00:39:16.526914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:08:10.791610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.537466+0000","last_change":"2023-09-10T00:38:43.787882+0000","last_active":"2023-09-10T00:39:16.537466+0000","last_peered":"2023-09-10T00:39:16.537466+0000","last_clean":"2023-09-10T00:39:16.537466+0000","last_became_active":"2023-09-10T00:38:43.787257+0000","last_became_peered":"2023-09-10T00:38:43.787257+0000","last_unstale":"2023-09-10T00:39:16.537466+0000","last_undegraded":"2023-09-10T00:39:16.537466+0000","last_fullsized":"2023-09-10T00:39:16.537466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:11:19.171881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":65,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:40.592452+0000","last_change":"2023-09-10T00:38:38.760014+0000","last_active":"2023-09-10T00:39:40.592452+0000","last_peered":"2023-09-10T00:39:40.592452+0000","last_clean":"2023-09-10T00:39:40.592452+0000","last_became_active":"2023-09-10T00:38:38.759637+0000","last_became_peered":"2023-09-10T00:38:38.759637+0000","last_unstale":"2023-09-10T00:39:40.592452+0000","last_undegraded":"2023-09-10T00:39:40.592452+0000","last_fullsized":"2023-09-10T00:39:40.592452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:15.704929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527234+0000","last_change":"2023-09-10T00:38:39.765800+0000","last_active":"2023-09-10T00:39:16.527234+0000","last_peered":"2023-09-10T00:39:16.527234+0000","last_clean":"2023-09-10T00:39:16.527234+0000","last_became_active":"2023-09-10T00:38:39.765468+0000","last_became_peered":"2023-09-10T00:38:39.765468+0000","last_unstale":"2023-09-10T00:39:16.527234+0000","last_undegraded":"2023-09-10T00:39:16.527234+0000","last_fullsized":"2023-09-10T00:39:16.527234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:57:56.429365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.538360+0000","last_change":"2023-09-10T00:38:43.838285+0000","last_active":"2023-09-10T00:39:16.538360+0000","last_peered":"2023-09-10T00:39:16.538360+0000","last_clean":"2023-09-10T00:39:16.538360+0000","last_became_active":"2023-09-10T00:38:43.837821+0000","last_became_peered":"2023-09-10T00:38:43.837821+0000","last_unstale":"2023-09-10T00:39:16.538360+0000","last_undegraded":"2023-09-10T00:39:16.538360+0000","last_fullsized":"2023-09-10T00:39:16.538360+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:01:03.354281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.536361+0000","last_change":"2023-09-10T00:39:17.536361+0000","last_active":"2023-09-10T00:39:17.536361+0000","last_peered":"2023-09-10T00:39:17.536361+0000","last_clean":"2023-09-10T00:39:17.536361+0000","last_became_active":"2023-09-10T00:39:17.535930+0000","last_became_peered":"2023-09-10T00:39:17.535930+0000","last_unstale":"2023-09-10T00:39:17.536361+0000","last_undegraded":"2023-09-10T00:39:17.536361+0000","last_fullsized":"2023-09-10T00:39:17.536361+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:53.576115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":66,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:39.536268+0000","last_change":"2023-09-10T00:38:38.758745+0000","last_active":"2023-09-10T00:39:39.536268+0000","last_peered":"2023-09-10T00:39:39.536268+0000","last_clean":"2023-09-10T00:39:39.536268+0000","last_became_active":"2023-09-10T00:38:38.758427+0000","last_became_peered":"2023-09-10T00:38:38.758427+0000","last_unstale":"2023-09-10T00:39:39.536268+0000","last_undegraded":"2023-09-10T00:39:39.536268+0000","last_fullsized":"2023-09-10T00:39:39.536268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:02:49.238175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527226+0000","last_change":"2023-09-10T00:38:43.784058+0000","last_active":"2023-09-10T00:39:16.527226+0000","last_peered":"2023-09-10T00:39:16.527226+0000","last_clean":"2023-09-10T00:39:16.527226+0000","last_became_active":"2023-09-10T00:38:43.783571+0000","last_became_peered":"2023-09-10T00:38:43.783571+0000","last_unstale":"2023-09-10T00:39:16.527226+0000","last_undegraded":"2023-09-10T00:39:16.527226+0000","last_fullsized":"2023-09-10T00:39:16.527226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:53:50.992393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.533823+0000","last_change":"2023-09-10T00:38:39.775659+0000","last_active":"2023-09-10T00:39:16.533823+0000","last_peered":"2023-09-10T00:39:16.533823+0000","last_clean":"2023-09-10T00:39:16.533823+0000","last_became_active":"2023-09-10T00:38:39.775151+0000","last_became_peered":"2023-09-10T00:38:39.775151+0000","last_unstale":"2023-09-10T00:39:16.533823+0000","last_undegraded":"2023-09-10T00:39:16.533823+0000","last_fullsized":"2023-09-10T00:39:16.533823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:07.415418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522562+0000","last_change":"2023-09-10T00:38:38.755009+0000","last_active":"2023-09-10T00:39:16.522562+0000","last_peered":"2023-09-10T00:39:16.522562+0000","last_clean":"2023-09-10T00:39:16.522562+0000","last_became_active":"2023-09-10T00:38:38.754699+0000","last_became_peered":"2023-09-10T00:38:38.754699+0000","last_unstale":"2023-09-10T00:39:16.522562+0000","last_undegraded":"2023-09-10T00:39:16.522562+0000","last_fullsized":"2023-09-10T00:39:16.522562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:12:02.075256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":322,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526552+0000","last_change":"2023-09-10T00:37:38.459695+0000","last_active":"2023-09-10T00:39:16.526552+0000","last_peered":"2023-09-10T00:39:16.526552+0000","last_clean":"2023-09-10T00:39:16.526552+0000","last_became_active":"2023-09-10T00:37:38.459220+0000","last_became_peered":"2023-09-10T00:37:38.459220+0000","last_unstale":"2023-09-10T00:39:16.526552+0000","last_undegraded":"2023-09-10T00:39:16.526552+0000","last_fullsized":"2023-09-10T00:39:16.526552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:34:44.602334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:34:44.602334+0000","last_clean_scrub_stamp":"2023-09-10T00:34:44.602334+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:21:32.648725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526212+0000","last_change":"2023-09-10T00:38:43.839001+0000","last_active":"2023-09-10T00:39:16.526212+0000","last_peered":"2023-09-10T00:39:16.526212+0000","last_clean":"2023-09-10T00:39:16.526212+0000","last_became_active":"2023-09-10T00:38:43.838497+0000","last_became_peered":"2023-09-10T00:38:43.838497+0000","last_unstale":"2023-09-10T00:39:16.526212+0000","last_undegraded":"2023-09-10T00:39:16.526212+0000","last_fullsized":"2023-09-10T00:39:16.526212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:57:23.349632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.553332+0000","last_change":"2023-09-10T00:38:39.760150+0000","last_active":"2023-09-10T00:39:16.553332+0000","last_peered":"2023-09-10T00:39:16.553332+0000","last_clean":"2023-09-10T00:39:16.553332+0000","last_became_active":"2023-09-10T00:38:39.759811+0000","last_became_peered":"2023-09-10T00:38:39.759811+0000","last_unstale":"2023-09-10T00:39:16.553332+0000","last_undegraded":"2023-09-10T00:39:16.553332+0000","last_fullsized":"2023-09-10T00:39:16.553332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:21:19.359057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526651+0000","last_change":"2023-09-10T00:38:38.771066+0000","last_active":"2023-09-10T00:39:16.526651+0000","last_peered":"2023-09-10T00:39:16.526651+0000","last_clean":"2023-09-10T00:39:16.526651+0000","last_became_active":"2023-09-10T00:38:38.770810+0000","last_became_peered":"2023-09-10T00:38:38.770810+0000","last_unstale":"2023-09-10T00:39:16.526651+0000","last_undegraded":"2023-09-10T00:39:16.526651+0000","last_fullsized":"2023-09-10T00:39:16.526651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:55:06.923842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.519639+0000","last_change":"2023-09-10T00:38:43.786272+0000","last_active":"2023-09-10T00:39:16.519639+0000","last_peered":"2023-09-10T00:39:16.519639+0000","last_clean":"2023-09-10T00:39:16.519639+0000","last_became_active":"2023-09-10T00:38:43.785934+0000","last_became_peered":"2023-09-10T00:38:43.785934+0000","last_unstale":"2023-09-10T00:39:16.519639+0000","last_undegraded":"2023-09-10T00:39:16.519639+0000","last_fullsized":"2023-09-10T00:39:16.519639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:31:49.721133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.524061+0000","last_change":"2023-09-10T00:38:38.763263+0000","last_active":"2023-09-10T00:39:16.524061+0000","last_peered":"2023-09-10T00:39:16.524061+0000","last_clean":"2023-09-10T00:39:16.524061+0000","last_became_active":"2023-09-10T00:38:38.762911+0000","last_became_peered":"2023-09-10T00:38:38.762911+0000","last_unstale":"2023-09-10T00:39:16.524061+0000","last_undegraded":"2023-09-10T00:39:16.524061+0000","last_fullsized":"2023-09-10T00:39:16.524061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:19:10.292311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.537610+0000","last_change":"2023-09-10T00:38:39.759171+0000","last_active":"2023-09-10T00:39:16.537610+0000","last_peered":"2023-09-10T00:39:16.537610+0000","last_clean":"2023-09-10T00:39:16.537610+0000","last_became_active":"2023-09-10T00:38:39.758843+0000","last_became_peered":"2023-09-10T00:38:39.758843+0000","last_unstale":"2023-09-10T00:39:16.537610+0000","last_undegraded":"2023-09-10T00:39:16.537610+0000","last_fullsized":"2023-09-10T00:39:16.537610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:50:36.064937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.536606+0000","last_change":"2023-09-10T00:38:43.837998+0000","last_active":"2023-09-10T00:39:16.536606+0000","last_peered":"2023-09-10T00:39:16.536606+0000","last_clean":"2023-09-10T00:39:16.536606+0000","last_became_active":"2023-09-10T00:38:43.837556+0000","last_became_peered":"2023-09-10T00:38:43.837556+0000","last_unstale":"2023-09-10T00:39:16.536606+0000","last_undegraded":"2023-09-10T00:39:16.536606+0000","last_fullsized":"2023-09-10T00:39:16.536606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:18:40.488279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'9","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.532279+0000","last_change":"2023-09-10T00:38:38.759819+0000","last_active":"2023-09-10T00:39:16.532279+0000","last_peered":"2023-09-10T00:39:16.532279+0000","last_clean":"2023-09-10T00:39:16.532279+0000","last_became_active":"2023-09-10T00:38:38.759490+0000","last_became_peered":"2023-09-10T00:38:38.759490+0000","last_unstale":"2023-09-10T00:39:16.532279+0000","last_undegraded":"2023-09-10T00:39:16.532279+0000","last_fullsized":"2023-09-10T00:39:16.532279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:39:01.206817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.535116+0000","last_change":"2023-09-10T00:38:39.769669+0000","last_active":"2023-09-10T00:39:16.535116+0000","last_peered":"2023-09-10T00:39:16.535116+0000","last_clean":"2023-09-10T00:39:16.535116+0000","last_became_active":"2023-09-10T00:38:39.769402+0000","last_became_peered":"2023-09-10T00:38:39.769402+0000","last_unstale":"2023-09-10T00:39:16.535116+0000","last_undegraded":"2023-09-10T00:39:16.535116+0000","last_fullsized":"2023-09-10T00:39:16.535116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:00:50.215103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526606+0000","last_change":"2023-09-10T00:38:43.787518+0000","last_active":"2023-09-10T00:39:16.526606+0000","last_peered":"2023-09-10T00:39:16.526606+0000","last_clean":"2023-09-10T00:39:16.526606+0000","last_became_active":"2023-09-10T00:38:43.786410+0000","last_became_peered":"2023-09-10T00:38:43.786410+0000","last_unstale":"2023-09-10T00:39:16.526606+0000","last_undegraded":"2023-09-10T00:39:16.526606+0000","last_fullsized":"2023-09-10T00:39:16.526606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:13:37.903875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522600+0000","last_change":"2023-09-10T00:38:38.760992+0000","last_active":"2023-09-10T00:39:16.522600+0000","last_peered":"2023-09-10T00:39:16.522600+0000","last_clean":"2023-09-10T00:39:16.522600+0000","last_became_active":"2023-09-10T00:38:38.760635+0000","last_became_peered":"2023-09-10T00:38:38.760635+0000","last_unstale":"2023-09-10T00:39:16.522600+0000","last_undegraded":"2023-09-10T00:39:16.522600+0000","last_fullsized":"2023-09-10T00:39:16.522600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:29:36.070477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.528974+0000","last_change":"2023-09-10T00:38:39.773771+0000","last_active":"2023-09-10T00:39:16.528974+0000","last_peered":"2023-09-10T00:39:16.528974+0000","last_clean":"2023-09-10T00:39:16.528974+0000","last_became_active":"2023-09-10T00:38:39.773296+0000","last_became_peered":"2023-09-10T00:38:39.773296+0000","last_unstale":"2023-09-10T00:39:16.528974+0000","last_undegraded":"2023-09-10T00:39:16.528974+0000","last_fullsized":"2023-09-10T00:39:16.528974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:13:01.194358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.536853+0000","last_change":"2023-09-10T00:39:17.536853+0000","last_active":"2023-09-10T00:39:17.536853+0000","last_peered":"2023-09-10T00:39:17.536853+0000","last_clean":"2023-09-10T00:39:17.536853+0000","last_became_active":"2023-09-10T00:39:17.536287+0000","last_became_peered":"2023-09-10T00:39:17.536287+0000","last_unstale":"2023-09-10T00:39:17.536853+0000","last_undegraded":"2023-09-10T00:39:17.536853+0000","last_fullsized":"2023-09-10T00:39:17.536853+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:50:35.294055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.525550+0000","last_change":"2023-09-10T00:38:38.764954+0000","last_active":"2023-09-10T00:39:16.525550+0000","last_peered":"2023-09-10T00:39:16.525550+0000","last_clean":"2023-09-10T00:39:16.525550+0000","last_became_active":"2023-09-10T00:38:38.764589+0000","last_became_peered":"2023-09-10T00:38:38.764589+0000","last_unstale":"2023-09-10T00:39:16.525550+0000","last_undegraded":"2023-09-10T00:39:16.525550+0000","last_fullsized":"2023-09-10T00:39:16.525550+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:40:19.559285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522698+0000","last_change":"2023-09-10T00:38:39.766141+0000","last_active":"2023-09-10T00:39:16.522698+0000","last_peered":"2023-09-10T00:39:16.522698+0000","last_clean":"2023-09-10T00:39:16.522698+0000","last_became_active":"2023-09-10T00:38:39.765832+0000","last_became_peered":"2023-09-10T00:38:39.765832+0000","last_unstale":"2023-09-10T00:39:16.522698+0000","last_undegraded":"2023-09-10T00:39:16.522698+0000","last_fullsized":"2023-09-10T00:39:16.522698+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:08:45.149973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.532882+0000","last_change":"2023-09-10T00:38:43.785111+0000","last_active":"2023-09-10T00:39:16.532882+0000","last_peered":"2023-09-10T00:39:16.532882+0000","last_clean":"2023-09-10T00:39:16.532882+0000","last_became_active":"2023-09-10T00:38:43.784542+0000","last_became_peered":"2023-09-10T00:38:43.784542+0000","last_unstale":"2023-09-10T00:39:16.532882+0000","last_undegraded":"2023-09-10T00:39:16.532882+0000","last_fullsized":"2023-09-10T00:39:16.532882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:34:51.611608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.533431+0000","last_change":"2023-09-10T00:38:38.759984+0000","last_active":"2023-09-10T00:39:16.533431+0000","last_peered":"2023-09-10T00:39:16.533431+0000","last_clean":"2023-09-10T00:39:16.533431+0000","last_became_active":"2023-09-10T00:38:38.759660+0000","last_became_peered":"2023-09-10T00:38:38.759660+0000","last_unstale":"2023-09-10T00:39:16.533431+0000","last_undegraded":"2023-09-10T00:39:16.533431+0000","last_fullsized":"2023-09-10T00:39:16.533431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:02:11.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.519505+0000","last_change":"2023-09-10T00:38:39.770765+0000","last_active":"2023-09-10T00:39:16.519505+0000","last_peered":"2023-09-10T00:39:16.519505+0000","last_clean":"2023-09-10T00:39:16.519505+0000","last_became_active":"2023-09-10T00:38:39.770483+0000","last_became_peered":"2023-09-10T00:38:39.770483+0000","last_unstale":"2023-09-10T00:39:16.519505+0000","last_undegraded":"2023-09-10T00:39:16.519505+0000","last_fullsized":"2023-09-10T00:39:16.519505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:57:28.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522403+0000","last_change":"2023-09-10T00:38:43.790945+0000","last_active":"2023-09-10T00:39:16.522403+0000","last_peered":"2023-09-10T00:39:16.522403+0000","last_clean":"2023-09-10T00:39:16.522403+0000","last_became_active":"2023-09-10T00:38:43.790467+0000","last_became_peered":"2023-09-10T00:38:43.790467+0000","last_unstale":"2023-09-10T00:39:16.522403+0000","last_undegraded":"2023-09-10T00:39:16.522403+0000","last_fullsized":"2023-09-10T00:39:16.522403+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:44:07.041302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526857+0000","last_change":"2023-09-10T00:38:38.765912+0000","last_active":"2023-09-10T00:39:16.526857+0000","last_peered":"2023-09-10T00:39:16.526857+0000","last_clean":"2023-09-10T00:39:16.526857+0000","last_became_active":"2023-09-10T00:38:38.765586+0000","last_became_peered":"2023-09-10T00:38:38.765586+0000","last_unstale":"2023-09-10T00:39:16.526857+0000","last_undegraded":"2023-09-10T00:39:16.526857+0000","last_fullsized":"2023-09-10T00:39:16.526857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:25:05.974158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.533090+0000","last_change":"2023-09-10T00:38:39.771462+0000","last_active":"2023-09-10T00:39:16.533090+0000","last_peered":"2023-09-10T00:39:16.533090+0000","last_clean":"2023-09-10T00:39:16.533090+0000","last_became_active":"2023-09-10T00:38:39.771153+0000","last_became_peered":"2023-09-10T00:38:39.771153+0000","last_unstale":"2023-09-10T00:39:16.533090+0000","last_undegraded":"2023-09-10T00:39:16.533090+0000","last_fullsized":"2023-09-10T00:39:16.533090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:03:35.312186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.522619+0000","last_change":"2023-09-10T00:38:43.838446+0000","last_active":"2023-09-10T00:39:16.522619+0000","last_peered":"2023-09-10T00:39:16.522619+0000","last_clean":"2023-09-10T00:39:16.522619+0000","last_became_active":"2023-09-10T00:38:43.838119+0000","last_became_peered":"2023-09-10T00:38:43.838119+0000","last_unstale":"2023-09-10T00:39:16.522619+0000","last_undegraded":"2023-09-10T00:39:16.522619+0000","last_fullsized":"2023-09-10T00:39:16.522619+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:12:20.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.524372+0000","last_change":"2023-09-10T00:38:38.765548+0000","last_active":"2023-09-10T00:39:16.524372+0000","last_peered":"2023-09-10T00:39:16.524372+0000","last_clean":"2023-09-10T00:39:16.524372+0000","last_became_active":"2023-09-10T00:38:38.765281+0000","last_became_peered":"2023-09-10T00:38:38.765281+0000","last_unstale":"2023-09-10T00:39:16.524372+0000","last_undegraded":"2023-09-10T00:39:16.524372+0000","last_fullsized":"2023-09-10T00:39:16.524372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:50:44.422327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527816+0000","last_change":"2023-09-10T00:38:39.771995+0000","last_active":"2023-09-10T00:39:16.527816+0000","last_peered":"2023-09-10T00:39:16.527816+0000","last_clean":"2023-09-10T00:39:16.527816+0000","last_became_active":"2023-09-10T00:38:39.771693+0000","last_became_peered":"2023-09-10T00:38:39.771693+0000","last_unstale":"2023-09-10T00:39:16.527816+0000","last_undegraded":"2023-09-10T00:39:16.527816+0000","last_fullsized":"2023-09-10T00:39:16.527816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:41:45.621060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526180+0000","last_change":"2023-09-10T00:38:43.789159+0000","last_active":"2023-09-10T00:39:16.526180+0000","last_peered":"2023-09-10T00:39:16.526180+0000","last_clean":"2023-09-10T00:39:16.526180+0000","last_became_active":"2023-09-10T00:38:43.788675+0000","last_became_peered":"2023-09-10T00:38:43.788675+0000","last_unstale":"2023-09-10T00:39:16.526180+0000","last_undegraded":"2023-09-10T00:39:16.526180+0000","last_fullsized":"2023-09-10T00:39:16.526180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:30:39.545561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":67,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:42.593170+0000","last_change":"2023-09-10T00:38:38.767245+0000","last_active":"2023-09-10T00:39:42.593170+0000","last_peered":"2023-09-10T00:39:42.593170+0000","last_clean":"2023-09-10T00:39:42.593170+0000","last_became_active":"2023-09-10T00:38:38.766952+0000","last_became_peered":"2023-09-10T00:38:38.766952+0000","last_unstale":"2023-09-10T00:39:42.593170+0000","last_undegraded":"2023-09-10T00:39:42.593170+0000","last_fullsized":"2023-09-10T00:39:42.593170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:57:23.309838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527738+0000","last_change":"2023-09-10T00:38:39.772199+0000","last_active":"2023-09-10T00:39:16.527738+0000","last_peered":"2023-09-10T00:39:16.527738+0000","last_clean":"2023-09-10T00:39:16.527738+0000","last_became_active":"2023-09-10T00:38:39.771917+0000","last_became_peered":"2023-09-10T00:38:39.771917+0000","last_unstale":"2023-09-10T00:39:16.527738+0000","last_undegraded":"2023-09-10T00:39:16.527738+0000","last_fullsized":"2023-09-10T00:39:16.527738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:00:52.824581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527350+0000","last_change":"2023-09-10T00:38:43.797107+0000","last_active":"2023-09-10T00:39:16.527350+0000","last_peered":"2023-09-10T00:39:16.527350+0000","last_clean":"2023-09-10T00:39:16.527350+0000","last_became_active":"2023-09-10T00:38:43.796606+0000","last_became_peered":"2023-09-10T00:38:43.796606+0000","last_unstale":"2023-09-10T00:39:16.527350+0000","last_undegraded":"2023-09-10T00:39:16.527350+0000","last_fullsized":"2023-09-10T00:39:16.527350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:17:00.444238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.525888+0000","last_change":"2023-09-10T00:38:38.768045+0000","last_active":"2023-09-10T00:39:16.525888+0000","last_peered":"2023-09-10T00:39:16.525888+0000","last_clean":"2023-09-10T00:39:16.525888+0000","last_became_active":"2023-09-10T00:38:38.767344+0000","last_became_peered":"2023-09-10T00:38:38.767344+0000","last_unstale":"2023-09-10T00:39:16.525888+0000","last_undegraded":"2023-09-10T00:39:16.525888+0000","last_fullsized":"2023-09-10T00:39:16.525888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:51:08.765412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.536408+0000","last_change":"2023-09-10T00:39:17.536408+0000","last_active":"2023-09-10T00:39:17.536408+0000","last_peered":"2023-09-10T00:39:17.536408+0000","last_clean":"2023-09-10T00:39:17.536408+0000","last_became_active":"2023-09-10T00:39:17.535980+0000","last_became_peered":"2023-09-10T00:39:17.535980+0000","last_unstale":"2023-09-10T00:39:17.536408+0000","last_undegraded":"2023-09-10T00:39:17.536408+0000","last_fullsized":"2023-09-10T00:39:17.536408+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:08:10.517535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.521961+0000","last_change":"2023-09-10T00:38:43.784928+0000","last_active":"2023-09-10T00:39:16.521961+0000","last_peered":"2023-09-10T00:39:16.521961+0000","last_clean":"2023-09-10T00:39:16.521961+0000","last_became_active":"2023-09-10T00:38:43.784479+0000","last_became_peered":"2023-09-10T00:38:43.784479+0000","last_unstale":"2023-09-10T00:39:16.521961+0000","last_undegraded":"2023-09-10T00:39:16.521961+0000","last_fullsized":"2023-09-10T00:39:16.521961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:14:45.019988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.538128+0000","last_change":"2023-09-10T00:38:38.770730+0000","last_active":"2023-09-10T00:39:16.538128+0000","last_peered":"2023-09-10T00:39:16.538128+0000","last_clean":"2023-09-10T00:39:16.538128+0000","last_became_active":"2023-09-10T00:38:38.770344+0000","last_became_peered":"2023-09-10T00:38:38.770344+0000","last_unstale":"2023-09-10T00:39:16.538128+0000","last_undegraded":"2023-09-10T00:39:16.538128+0000","last_fullsized":"2023-09-10T00:39:16.538128+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:49:22.881956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527892+0000","last_change":"2023-09-10T00:38:39.766149+0000","last_active":"2023-09-10T00:39:16.527892+0000","last_peered":"2023-09-10T00:39:16.527892+0000","last_clean":"2023-09-10T00:39:16.527892+0000","last_became_active":"2023-09-10T00:38:39.765773+0000","last_became_peered":"2023-09-10T00:38:39.765773+0000","last_unstale":"2023-09-10T00:39:16.527892+0000","last_undegraded":"2023-09-10T00:39:16.527892+0000","last_fullsized":"2023-09-10T00:39:16.527892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:00:37.854168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.519425+0000","last_change":"2023-09-10T00:38:39.765197+0000","last_active":"2023-09-10T00:39:16.519425+0000","last_peered":"2023-09-10T00:39:16.519425+0000","last_clean":"2023-09-10T00:39:16.519425+0000","last_became_active":"2023-09-10T00:38:39.764891+0000","last_became_peered":"2023-09-10T00:38:39.764891+0000","last_unstale":"2023-09-10T00:39:16.519425+0000","last_undegraded":"2023-09-10T00:39:16.519425+0000","last_fullsized":"2023-09-10T00:39:16.519425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:47:27.429355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527189+0000","last_change":"2023-09-10T00:38:39.764722+0000","last_active":"2023-09-10T00:39:16.527189+0000","last_peered":"2023-09-10T00:39:16.527189+0000","last_clean":"2023-09-10T00:39:16.527189+0000","last_became_active":"2023-09-10T00:38:39.764376+0000","last_became_peered":"2023-09-10T00:38:39.764376+0000","last_unstale":"2023-09-10T00:39:16.527189+0000","last_undegraded":"2023-09-10T00:39:16.527189+0000","last_fullsized":"2023-09-10T00:39:16.527189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:27:36.708573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526720+0000","last_change":"2023-09-10T00:38:39.771997+0000","last_active":"2023-09-10T00:39:16.526720+0000","last_peered":"2023-09-10T00:39:16.526720+0000","last_clean":"2023-09-10T00:39:16.526720+0000","last_became_active":"2023-09-10T00:38:39.771588+0000","last_became_peered":"2023-09-10T00:38:39.771588+0000","last_unstale":"2023-09-10T00:39:16.526720+0000","last_undegraded":"2023-09-10T00:39:16.526720+0000","last_fullsized":"2023-09-10T00:39:16.526720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:20:07.171333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527554+0000","last_change":"2023-09-10T00:38:39.773393+0000","last_active":"2023-09-10T00:39:16.527554+0000","last_peered":"2023-09-10T00:39:16.527554+0000","last_clean":"2023-09-10T00:39:16.527554+0000","last_became_active":"2023-09-10T00:38:39.772972+0000","last_became_peered":"2023-09-10T00:38:39.772972+0000","last_unstale":"2023-09-10T00:39:16.527554+0000","last_undegraded":"2023-09-10T00:39:16.527554+0000","last_fullsized":"2023-09-10T00:39:16.527554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:27:29.916970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.537695+0000","last_change":"2023-09-10T00:38:39.767871+0000","last_active":"2023-09-10T00:39:16.537695+0000","last_peered":"2023-09-10T00:39:16.537695+0000","last_clean":"2023-09-10T00:39:16.537695+0000","last_became_active":"2023-09-10T00:38:39.767563+0000","last_became_peered":"2023-09-10T00:38:39.767563+0000","last_unstale":"2023-09-10T00:39:16.537695+0000","last_undegraded":"2023-09-10T00:39:16.537695+0000","last_fullsized":"2023-09-10T00:39:16.537695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:22:30.400441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.520036+0000","last_change":"2023-09-10T00:38:39.764664+0000","last_active":"2023-09-10T00:39:16.520036+0000","last_peered":"2023-09-10T00:39:16.520036+0000","last_clean":"2023-09-10T00:39:16.520036+0000","last_became_active":"2023-09-10T00:38:39.764364+0000","last_became_peered":"2023-09-10T00:38:39.764364+0000","last_unstale":"2023-09-10T00:39:16.520036+0000","last_undegraded":"2023-09-10T00:39:16.520036+0000","last_fullsized":"2023-09-10T00:39:16.520036+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:41.656472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.527797+0000","last_change":"2023-09-10T00:38:39.770315+0000","last_active":"2023-09-10T00:39:16.527797+0000","last_peered":"2023-09-10T00:39:16.527797+0000","last_clean":"2023-09-10T00:39:16.527797+0000","last_became_active":"2023-09-10T00:38:39.769977+0000","last_became_peered":"2023-09-10T00:38:39.769977+0000","last_unstale":"2023-09-10T00:39:16.527797+0000","last_undegraded":"2023-09-10T00:39:16.527797+0000","last_fullsized":"2023-09-10T00:39:16.527797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:30:13.380487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.535264+0000","last_change":"2023-09-10T00:38:39.768734+0000","last_active":"2023-09-10T00:39:16.535264+0000","last_peered":"2023-09-10T00:39:16.535264+0000","last_clean":"2023-09-10T00:39:16.535264+0000","last_became_active":"2023-09-10T00:38:39.768383+0000","last_became_peered":"2023-09-10T00:38:39.768383+0000","last_unstale":"2023-09-10T00:39:16.535264+0000","last_undegraded":"2023-09-10T00:39:16.535264+0000","last_fullsized":"2023-09-10T00:39:16.535264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:09.974447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-10T00:39:17.732943+0000","last_change":"2023-09-10T00:39:17.732561+0000","last_active":"2023-09-10T00:39:17.732943+0000","last_peered":"2023-09-10T00:39:17.732943+0000","last_clean":"2023-09-10T00:39:17.732943+0000","last_became_active":"2023-09-10T00:39:17.732138+0000","last_became_peered":"2023-09-10T00:39:17.732138+0000","last_unstale":"2023-09-10T00:39:17.732943+0000","last_undegraded":"2023-09-10T00:39:17.732943+0000","last_fullsized":"2023-09-10T00:39:17.732943+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:11:58.866881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.533521+0000","last_change":"2023-09-10T00:38:39.773839+0000","last_active":"2023-09-10T00:39:16.533521+0000","last_peered":"2023-09-10T00:39:16.533521+0000","last_clean":"2023-09-10T00:39:16.533521+0000","last_became_active":"2023-09-10T00:38:39.773521+0000","last_became_peered":"2023-09-10T00:38:39.773521+0000","last_unstale":"2023-09-10T00:39:16.533521+0000","last_undegraded":"2023-09-10T00:39:16.533521+0000","last_fullsized":"2023-09-10T00:39:16.533521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:15:49.033374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.526122+0000","last_change":"2023-09-10T00:38:39.775721+0000","last_active":"2023-09-10T00:39:16.526122+0000","last_peered":"2023-09-10T00:39:16.526122+0000","last_clean":"2023-09-10T00:39:16.526122+0000","last_became_active":"2023-09-10T00:38:39.775424+0000","last_became_peered":"2023-09-10T00:38:39.775424+0000","last_unstale":"2023-09-10T00:39:16.526122+0000","last_undegraded":"2023-09-10T00:39:16.526122+0000","last_fullsized":"2023-09-10T00:39:16.526122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:56:26.079332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.536682+0000","last_change":"2023-09-10T00:38:39.768767+0000","last_active":"2023-09-10T00:39:16.536682+0000","last_peered":"2023-09-10T00:39:16.536682+0000","last_clean":"2023-09-10T00:39:16.536682+0000","last_became_active":"2023-09-10T00:38:39.768452+0000","last_became_peered":"2023-09-10T00:38:39.768452+0000","last_unstale":"2023-09-10T00:39:16.536682+0000","last_undegraded":"2023-09-10T00:39:16.536682+0000","last_fullsized":"2023-09-10T00:39:16.536682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:31:53.578574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-10T00:39:16.519685+0000","last_change":"2023-09-10T00:38:39.761998+0000","last_active":"2023-09-10T00:39:16.519685+0000","last_peered":"2023-09-10T00:39:16.519685+0000","last_clean":"2023-09-10T00:39:16.519685+0000","last_became_active":"2023-09-10T00:38:39.761672+0000","last_became_peered":"2023-09-10T00:38:39.761672+0000","last_unstale":"2023-09-10T00:39:16.519685+0000","last_undegraded":"2023-09-10T00:39:16.519685+0000","last_fullsized":"2023-09-10T00:39:16.519685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:03:24.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65801,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":443,"num_read_kb":291,"num_write":234,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":197403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449112,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":490613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.36499999999999999}]},{"osd":1,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.499},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.44800000000000001}]},{"osd":2,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.39100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":4,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.435},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.53900000000000003}]},{"osd":5,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.379},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.313}]},{"osd":6,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.50900000000000001}]},{"osd":7,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.47199999999999998}]},{"osd":8,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.46100000000000002}]},{"osd":9,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.52900000000000003}]},{"osd":10,"last update":"Sun Sep 10 00:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.56699999999999995}]}]},{"osd":10,"up_from":70,"seq":300647710748,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1695744,"data_stored":1136418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.46899999999999997}]},{"osd":1,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51500000000000001,"15min":0.51900000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78200000000000003,"5min":0.873,"15min":0.873},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.498,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.36499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50700000000000001}]},{"osd":2,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.629,"15min":0.629},"min":{"1min":0.501,"5min":0.439,"15min":0.439},"max":{"1min":0.75800000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65400000000000003,"15min":0.64400000000000002},"min":{"1min":0.52400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.64500000000000002}]},{"osd":3,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.61899999999999999,"15min":0.63700000000000001},"min":{"1min":0.39300000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70099999999999996,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.443},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.48899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.69499999999999995}]},{"osd":4,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.625},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.71599999999999997}]},{"osd":5,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.54600000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78100000000000003,"15min":0.78300000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.94299999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.82299999999999995}]},{"osd":6,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.995,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.59999999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73999999999999999}]},{"osd":7,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71299999999999997}]},{"osd":8,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77700000000000002,"15min":0.77500000000000002},"min":{"1min":0.63900000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78600000000000003,"15min":0.78800000000000003},"min":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.84399999999999997}]},{"osd":9,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.86099999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55600000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.752}]},{"osd":11,"last update":"Sun Sep 10 00:39:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.86799999999999999}]}]},{"osd":9,"up_from":65,"seq":279172874273,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":494639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.441,"15min":0.44900000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.58399999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.504},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.44900000000000001,"15min":0.46300000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.60899999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.33200000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.77800000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78600000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.432}]},{"osd":2,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.46000000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.83099999999999996,"5min":0.879,"15min":0.879},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.59499999999999997,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79100000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40200000000000002}]},{"osd":3,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.55800000000000005,"15min":0.57799999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76000000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.60899999999999999,"15min":0.63100000000000001},"min":{"1min":0.375,"5min":0.314,"15min":0.314},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.55800000000000005}]},{"osd":4,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.52500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.85499999999999998,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.61899999999999999}]},{"osd":5,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.73399999999999999,"15min":0.749},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91500000000000004,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":3.3109999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.75900000000000001,"15min":0.77700000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.028,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.68799999999999994}]},{"osd":6,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.42499999999999999}]},{"osd":7,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.79200000000000004,"15min":0.80900000000000005},"min":{"1min":0.504,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.071,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.81000000000000005,"15min":0.83199999999999996},"min":{"1min":0.498,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0580000000000001,"5min":1.071,"15min":1.071},"last":0.63300000000000001}]},{"osd":8,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.77500000000000002,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.83499999999999996,"15min":0.85999999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96699999999999997,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.65900000000000003}]},{"osd":10,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.72899999999999998,"15min":0.749},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.77600000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.54000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.82799999999999996,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":3.359}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":8,"up_from":57,"seq":244813135910,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":490590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57199999999999995,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.67800000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55300000000000005}]},{"osd":1,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.67500000000000004,"15min":0.628},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":6.5759999999999996,"5min":6.5759999999999996,"15min":6.5759999999999996},"last":0.56200000000000006}]},{"osd":2,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.73299999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.42599999999999999}]},{"osd":3,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.624},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83099999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.41199999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.439,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.53700000000000003}]},{"osd":5,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.34699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94399999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.44600000000000001}]},{"osd":6,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.79100000000000004,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":6.5940000000000003,"5min":6.5940000000000003,"15min":6.5940000000000003},"last":0.66700000000000004}]},{"osd":7,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.76000000000000001,"15min":0.70999999999999996},"min":{"1min":0.51300000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.5439999999999996,"5min":6.5439999999999996,"15min":6.5439999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.441,"5min":0.435,"15min":0.435},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.64900000000000002}]},{"osd":9,"last update":"Sun Sep 10 00:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.73899999999999999,"15min":0.73199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.73399999999999999,"15min":0.72199999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70399999999999996}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.57199999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.73999999999999999,"15min":0.71299999999999997},"min":{"1min":0.64200000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.64000000000000001}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.65900000000000003}]}]},{"osd":7,"up_from":50,"seq":214748364840,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1617920,"data_stored":1087476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54100000000000004,"15min":0.53200000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.39800000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40500000000000003,"15min":0.40000000000000002},"min":{"1min":0.32800000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.52600000000000002,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.40999999999999998,"15min":0.40200000000000002},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63600000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.311}]},{"osd":2,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.79800000000000004,"5min":1.091,"15min":1.091},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.51400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85999999999999999,"5min":1.069,"15min":1.069},"last":0.51000000000000001}]},{"osd":3,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.36699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76900000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.46999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.39900000000000002}]},{"osd":4,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.38300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.66300000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.497,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.59999999999999998,"5min":1.056,"15min":1.056},"last":0.29299999999999998}]},{"osd":5,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":0.628},"min":{"1min":0.49399999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.83299999999999996,"5min":1.026,"15min":1.026},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.48099999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98399999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.46100000000000002}]},{"osd":6,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.48899999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.84999999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89800000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.55200000000000005}]},{"osd":8,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.63500000000000001,"15min":0.62},"min":{"1min":0.50600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.66200000000000003,"15min":0.65200000000000002},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.496}]},{"osd":9,"last update":"Sun Sep 10 00:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.53500000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80000000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69299999999999995,"15min":0.69599999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.83599999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.48299999999999998}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65700000000000003,"15min":0.65200000000000002},"min":{"1min":0.46100000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83299999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66600000000000004,"15min":0.65700000000000003},"min":{"1min":0.54000000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.57099999999999995}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.624}]}]},{"osd":6,"up_from":45,"seq":193273528365,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":486047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.41199999999999998,"15min":0.40000000000000002},"min":{"1min":0.311,"5min":0.247,"15min":0.247},"max":{"1min":0.52000000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42299999999999999,"15min":0.40799999999999997},"min":{"1min":0.33100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59199999999999997,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.42199999999999999}]},{"osd":1,"last update":"Sun Sep 10 00:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57799999999999996,"15min":0.55700000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80600000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.71599999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53800000000000003}]},{"osd":2,"last update":"Sun Sep 10 00:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.47799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74299999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.624,"15min":0.60999999999999999},"min":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76300000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55500000000000005}]},{"osd":3,"last update":"Sun Sep 10 00:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.49299999999999999,"15min":0.48099999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68999999999999995,"5min":0.875,"15min":0.875},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.42899999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67700000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.47999999999999998}]},{"osd":4,"last update":"Sun Sep 10 00:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64500000000000002,"15min":0.64200000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88700000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.68899999999999995,"15min":0.67900000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81000000000000005,"5min":1.024,"15min":1.024},"last":0.59999999999999998}]},{"osd":5,"last update":"Sun Sep 10 00:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.46800000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82999999999999996,"5min":1.048,"15min":1.048},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70799999999999996,"15min":0.69699999999999995},"min":{"1min":0.48299999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91300000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.65900000000000003}]},{"osd":7,"last update":"Sun Sep 10 00:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.73199999999999998,"15min":0.72899999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94899999999999995,"5min":1.069,"15min":1.069},"last":0.625},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72999999999999998,"15min":0.72299999999999998},"min":{"1min":0.47799999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.82799999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.70899999999999996}]},{"osd":8,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.81499999999999995,"15min":0.83299999999999996},"min":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.999,"5min":1.27,"15min":1.27},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.81699999999999995,"15min":0.82799999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0389999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.754}]},{"osd":9,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.82799999999999996,"15min":0.85899999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":1.389,"15min":1.389},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.76000000000000001,"15min":0.78200000000000003},"min":{"1min":0.5,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.80700000000000005,"5min":1.099,"15min":1.099},"last":0.59299999999999997}]},{"osd":10,"last update":"Sun Sep 10 00:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.82599999999999996,"15min":0.83899999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.96299999999999997,"5min":1.181,"15min":1.181},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.91300000000000003,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.73899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":545676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52600000000000002,"15min":0.50900000000000001},"min":{"1min":0.32100000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72199999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.53200000000000003,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.64900000000000002}]},{"osd":2,"last update":"Sun Sep 10 00:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.38100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73299999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.56399999999999995,"15min":0.53700000000000003},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":0.89100000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.501}]},{"osd":3,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.315,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.79300000000000004,"5min":0.879,"15min":0.879},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53400000000000003,"15min":0.52700000000000002},"min":{"1min":0.34699999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78800000000000003,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.69599999999999995}]},{"osd":4,"last update":"Sun Sep 10 00:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.42299999999999999},"min":{"1min":0.35099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.46000000000000002,"15min":0.437},"min":{"1min":0.33800000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58299999999999996}]},{"osd":5,"last update":"Sun Sep 10 00:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.755,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.65800000000000003,"15min":0.626},"min":{"1min":0.62,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92700000000000005,"5min":1.044,"15min":1.044},"last":0.52500000000000002}]},{"osd":6,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64000000000000001,"15min":0.61399999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86899999999999999,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.76900000000000002}]},{"osd":7,"last update":"Sun Sep 10 00:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61399999999999999,"15min":0.57399999999999995},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89200000000000002,"5min":0.999,"15min":0.999},"last":0.311},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.621,"15min":0.58299999999999996},"min":{"1min":0.54300000000000004,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.878,"5min":0.996,"15min":0.996},"last":0.34499999999999997}]},{"osd":8,"last update":"Sun Sep 10 00:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71699999999999997,"15min":0.71299999999999997},"min":{"1min":0.57799999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94999999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.54000000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.86599999999999999,"5min":1.014,"15min":1.014},"last":0.51000000000000001}]},{"osd":9,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.998,"5min":1.097,"15min":1.097},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77300000000000002,"15min":0.78600000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.93000000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.58599999999999997}]},{"osd":10,"last update":"Sun Sep 10 00:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.76000000000000001,"15min":0.77200000000000002},"min":{"1min":0.46700000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93600000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.86199999999999999,"15min":0.86799999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0109999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.42899999999999999}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.55900000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":487503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 10 00:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54400000000000004,"15min":0.5},"min":{"1min":0.36399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.52500000000000002},"min":{"1min":0.376,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.40500000000000003}]},{"osd":2,"last update":"Sun Sep 10 00:38:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.39500000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80700000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58599999999999997,"15min":0.56799999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0329999999999999,"5min":1.621,"15min":1.621},"last":0.45300000000000001}]},{"osd":3,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.627},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.501}]},{"osd":4,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.46300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73699999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.374,"15min":0.374},"max":{"1min":0.83999999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.83999999999999997}]},{"osd":5,"last update":"Sun Sep 10 00:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64000000000000001,"15min":0.60999999999999999},"min":{"1min":0.49199999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.96999999999999997,"5min":1.075,"15min":1.075},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64700000000000002,"15min":0.63700000000000001},"min":{"1min":0.48699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.54100000000000004}]},{"osd":6,"last update":"Sun Sep 10 00:39:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.34699999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.70699999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76100000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.97999999999999998}]},{"osd":7,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65700000000000003,"15min":0.66400000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73899999999999999,"15min":0.74399999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.98499999999999999,"5min":1.111,"15min":1.111},"last":0.82499999999999996}]},{"osd":8,"last update":"Sun Sep 10 00:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77300000000000002,"15min":0.77900000000000003},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0640000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":1.022},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.499,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.85999999999999999}]},{"osd":9,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.746,"15min":0.754},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0309999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.71799999999999997},"min":{"1min":0.45500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.90300000000000002}]},{"osd":10,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.746,"15min":0.74299999999999999},"min":{"1min":0.57999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.53400000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.90600000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.70999999999999996}]},{"osd":11,"last update":"Sun Sep 10 00:39:02 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":1.085}]}]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":546286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.70499999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.39800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72099999999999997,"5min":0.753,"15min":0.753},"last":0.29899999999999999}]},{"osd":1,"last update":"Sun Sep 10 00:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.42499999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.746,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82599999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.34200000000000003}]},{"osd":3,"last update":"Sun Sep 10 00:39:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.45000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.442,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68999999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.47599999999999998}]},{"osd":4,"last update":"Sun Sep 10 00:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.5},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.58499999999999996}]},{"osd":5,"last update":"Sun Sep 10 00:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.45500000000000002,"15min":0.435},"min":{"1min":0.29899999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52400000000000002,"15min":0.5},"min":{"1min":0.35699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52000000000000002}]},{"osd":6,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.47699999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.66400000000000003,"15min":0.64300000000000002},"min":{"1min":0.53700000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.38700000000000001}]},{"osd":7,"last update":"Sun Sep 10 00:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.54800000000000004}]},{"osd":8,"last update":"Sun Sep 10 00:39:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.35099999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78400000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.36899999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.67000000000000004}]},{"osd":9,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.61199999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.53400000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.56999999999999995}]},{"osd":10,"last update":"Sun Sep 10 00:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.70799999999999996,"15min":0.68500000000000005},"min":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.745,"15min":0.71899999999999997},"min":{"1min":0.68700000000000006,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.68700000000000006}]},{"osd":11,"last update":"Sun Sep 10 00:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.624},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.69899999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":485755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40200000000000002,"15min":0.39800000000000002},"min":{"1min":0.27900000000000003,"5min":0.217,"15min":0.217},"max":{"1min":0.83199999999999996,"5min":1.409,"15min":1.409},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.41899999999999998,"15min":0.41799999999999998},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.57699999999999996,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.40899999999999997}]},{"osd":1,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57999999999999996,"15min":0.59899999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92200000000000004,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.39500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88400000000000001,"5min":1.581,"15min":1.581},"last":0.52900000000000003}]},{"osd":2,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60599999999999998,"15min":0.60099999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78400000000000003,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63,"15min":0.65400000000000003},"min":{"1min":0.41299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85099999999999998,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.54100000000000004}]},{"osd":4,"last update":"Sun Sep 10 00:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.39300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80600000000000005,"5min":1.536,"15min":1.536},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.66500000000000004,"15min":0.68799999999999994},"min":{"1min":0.38300000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.748,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.51600000000000001}]},{"osd":5,"last update":"Sun Sep 10 00:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88100000000000001,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67800000000000005,"15min":0.67500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88700000000000001,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.65300000000000002}]},{"osd":6,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78900000000000003,"5min":1,"15min":1},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60199999999999998,"15min":0.57999999999999996},"min":{"1min":0.35699999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.84299999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.498}]},{"osd":7,"last update":"Sun Sep 10 00:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.74399999999999999,"15min":0.745},"min":{"1min":0.47699999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90800000000000003,"5min":1.022,"15min":1.022},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73799999999999999,"15min":0.73599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.69799999999999995}]},{"osd":8,"last update":"Sun Sep 10 00:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.90000000000000002}]},{"osd":9,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.73199999999999998,"15min":0.76400000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.98899999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.81200000000000006,"15min":0.83599999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96499999999999997,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.58199999999999996}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.82999999999999996,"15min":0.83999999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.75,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.71999999999999997}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.753}]}]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":490511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45000000000000001}]},{"osd":1,"last update":"Sun Sep 10 00:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.37,"15min":0.34399999999999997},"min":{"1min":0.29199999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.317,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.41099999999999998}]},{"osd":2,"last update":"Sun Sep 10 00:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80200000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58799999999999997,"15min":0.55500000000000005},"min":{"1min":0.497,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.54700000000000004}]},{"osd":3,"last update":"Sun Sep 10 00:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.39900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.60999999999999999,"15min":0.56200000000000006},"min":{"1min":0.35099999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.63800000000000001}]},{"osd":5,"last update":"Sun Sep 10 00:38:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.439,"5min":0.376,"15min":0.376},"max":{"1min":0.80600000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.52900000000000003}]},{"osd":6,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58099999999999996,"15min":0.56799999999999995},"min":{"1min":0.50600000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.78400000000000003,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.63700000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79900000000000004,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.69899999999999995}]},{"osd":7,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.39300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.88500000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.56499999999999995,"15min":0.52800000000000002},"min":{"1min":0.38700000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85199999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.503}]},{"osd":8,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63900000000000001,"15min":0.629},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91100000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.879,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.621}]},{"osd":9,"last update":"Sun Sep 10 00:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.63100000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73499999999999999,"15min":0.73399999999999999},"min":{"1min":0.56000000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94799999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67400000000000004}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74399999999999999,"15min":0.75},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94999999999999996,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.74099999999999999,"15min":0.73699999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.627}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.65000000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1630208,"data_stored":1082999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.40699999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.755,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52900000000000003}]},{"osd":1,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53400000000000003,"15min":0.51600000000000001},"min":{"1min":0.33700000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86099999999999999,"5min":0.873,"15min":0.873},"last":0.61599999999999999}]},{"osd":2,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40699999999999997,"15min":0.39600000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.71699999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.438,"15min":0.42999999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40699999999999997}]},{"osd":3,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54900000000000004,"15min":0.54200000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.81499999999999995,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.92400000000000004,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.51300000000000001}]},{"osd":4,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76000000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.68799999999999994}]},{"osd":6,"last update":"Sun Sep 10 00:39:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61699999999999999,"15min":0.61199999999999999},"min":{"1min":0.38900000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.502},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66000000000000003,"15min":0.65300000000000002},"min":{"1min":0.45700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96799999999999997,"5min":0.996,"15min":0.996},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Sep 10 00:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.64100000000000001,"15min":0.622},"min":{"1min":0.53700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.98399999999999999,"5min":1.046,"15min":1.046},"last":0.74199999999999999}]},{"osd":8,"last update":"Sun Sep 10 00:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.42499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53400000000000003,"15min":0.52000000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55600000000000005}]},{"osd":9,"last update":"Sun Sep 10 00:39:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.72199999999999998,"15min":0.71099999999999997},"min":{"1min":0.61899999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.71699999999999997,"15min":0.70499999999999996},"min":{"1min":0.61799999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.64900000000000002}]},{"osd":10,"last update":"Sun Sep 10 00:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.77600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-10T00:39:46.370 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-09-10T00:39:46.370 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-10T00:39:46.371 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-09-10T00:39:46.371 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph health --format=json 2023-09-10T00:39:46.934 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:47.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:47 smithi044 ceph-mon[98676]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:39:47.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:47 smithi044 ceph-mon[98676]: pgmap v304: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-09-10T00:39:47.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:47 smithi099 ceph-mon[102692]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:39:47.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:47 smithi099 ceph-mon[102692]: pgmap v304: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-09-10T00:39:48.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:47 smithi131 ceph-mon[103713]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:39:48.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:47 smithi131 ceph-mon[103713]: pgmap v304: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-09-10T00:39:48.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.171+0000 7f1e3ad49700 1 -- 172.21.15.44:0/3774636148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e340ff520 msgr2=0x7f1e340ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:48.173 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.171+0000 7f1e3ad49700 1 --2- 172.21.15.44:0/3774636148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e340ff520 0x7f1e340ff9a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009990 tx=0x7f1e20009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:48.173 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.171+0000 7f1e3ad49700 1 -- 172.21.15.44:0/3774636148 shutdown_connections 2023-09-10T00:39:48.173 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.171+0000 7f1e3ad49700 1 --2- 172.21.15.44:0/3774636148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e340ffee0 0x7f1e3410c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:48.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.171+0000 7f1e3ad49700 1 --2- 172.21.15.44:0/3774636148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e34104530 0x7f1e34104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:48.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.171+0000 7f1e3ad49700 1 --2- 172.21.15.44:0/3774636148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e340ff520 0x7f1e340ff9a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:48.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.171+0000 7f1e3ad49700 1 -- 172.21.15.44:0/3774636148 >> 172.21.15.44:0/3774636148 conn(0x7f1e34076600 msgr2=0x7f1e34076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:48.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.171+0000 7f1e3ad49700 1 -- 172.21.15.44:0/3774636148 shutdown_connections 2023-09-10T00:39:48.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.171+0000 7f1e3ad49700 1 -- 172.21.15.44:0/3774636148 wait complete. 2023-09-10T00:39:48.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e3ad49700 1 Processor -- start 2023-09-10T00:39:48.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e3ad49700 1 -- start start 2023-09-10T00:39:48.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e3ad49700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e340ff520 0x7f1e34101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:48.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e3ad49700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e340ffee0 0x7f1e341016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:48.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e3ad49700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e34104530 0x7f1e34101c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:48.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e3ad49700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e341a4af0 con 0x7f1e340ffee0 2023-09-10T00:39:48.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e3ad49700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e341a4c60 con 0x7f1e340ff520 2023-09-10T00:39:48.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e3ad49700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e341a4e20 con 0x7f1e34104530 2023-09-10T00:39:48.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e33fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e340ffee0 0x7f1e341016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:48.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e33fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e340ffee0 0x7f1e341016c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47330/0 (socket says 172.21.15.44:47330) 2023-09-10T00:39:48.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e33fff700 1 -- 172.21.15.44:0/477827794 learned_addr learned my addr 172.21.15.44:0/477827794 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:48.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e38ae5700 1 --2- 172.21.15.44:0/477827794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e340ff520 0x7f1e34101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:48.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e392e6700 1 --2- 172.21.15.44:0/477827794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e34104530 0x7f1e34101c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:48.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e33fff700 1 -- 172.21.15.44:0/477827794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e34104530 msgr2=0x7f1e34101c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:48.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e33fff700 1 --2- 172.21.15.44:0/477827794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e34104530 0x7f1e34101c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:48.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e33fff700 1 -- 172.21.15.44:0/477827794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e340ff520 msgr2=0x7f1e34101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:48.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e33fff700 1 --2- 172.21.15.44:0/477827794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e340ff520 0x7f1e34101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:48.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e33fff700 1 -- 172.21.15.44:0/477827794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e200095f0 con 0x7f1e340ffee0 2023-09-10T00:39:48.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.172+0000 7f1e392e6700 1 --2- 172.21.15.44:0/477827794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e34104530 0x7f1e34101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:39:48.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.173+0000 7f1e33fff700 1 --2- 172.21.15.44:0/477827794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e340ffee0 0x7f1e341016c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1e20006010 tx=0x7f1e20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:48.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.173+0000 7f1e31ffb700 1 -- 172.21.15.44:0/477827794 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2001d070 con 0x7f1e340ffee0 2023-09-10T00:39:48.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.173+0000 7f1e31ffb700 1 -- 172.21.15.44:0/477827794 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1e20004130 con 0x7f1e340ffee0 2023-09-10T00:39:48.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.173+0000 7f1e3ad49700 1 -- 172.21.15.44:0/477827794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e341a50a0 con 0x7f1e340ffee0 2023-09-10T00:39:48.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.173+0000 7f1e31ffb700 1 -- 172.21.15.44:0/477827794 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20022e30 con 0x7f1e340ffee0 2023-09-10T00:39:48.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.173+0000 7f1e3ad49700 1 -- 172.21.15.44:0/477827794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e341a5510 con 0x7f1e340ffee0 2023-09-10T00:39:48.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.174+0000 7f1e3ad49700 1 -- 172.21.15.44:0/477827794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e3404e300 con 0x7f1e340ffee0 2023-09-10T00:39:48.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.175+0000 7f1e31ffb700 1 -- 172.21.15.44:0/477827794 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f1e20022600 con 0x7f1e340ffee0 2023-09-10T00:39:48.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.175+0000 7f1e31ffb700 1 --2- 172.21.15.44:0/477827794 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1e240a0750 0x7f1e240a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:48.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.176+0000 7f1e38ae5700 1 --2- 172.21.15.44:0/477827794 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1e240a0750 0x7f1e240a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:48.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.176+0000 7f1e31ffb700 1 -- 172.21.15.44:0/477827794 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1e200cd7c0 con 0x7f1e340ffee0 2023-09-10T00:39:48.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.176+0000 7f1e38ae5700 1 --2- 172.21.15.44:0/477827794 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1e240a0750 0x7f1e240a2c10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1e34069c10 tx=0x7f1e28009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:48.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.179+0000 7f1e31ffb700 1 -- 172.21.15.44:0/477827794 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1e20096f10 con 0x7f1e340ffee0 2023-09-10T00:39:48.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.423+0000 7f1e3ad49700 1 -- 172.21.15.44:0/477827794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1e341026b0 con 0x7f1e340ffee0 2023-09-10T00:39:48.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.424+0000 7f1e31ffb700 1 -- 172.21.15.44:0/477827794 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1e2009a530 con 0x7f1e340ffee0 2023-09-10T00:39:48.425 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:48.425 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-10T00:39:48.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.426+0000 7f1e3ad49700 1 -- 172.21.15.44:0/477827794 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1e240a0750 msgr2=0x7f1e240a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:48.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.426+0000 7f1e3ad49700 1 --2- 172.21.15.44:0/477827794 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1e240a0750 0x7f1e240a2c10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1e34069c10 tx=0x7f1e28009190 comp rx=0 tx=0).stop 2023-09-10T00:39:48.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.426+0000 7f1e3ad49700 1 -- 172.21.15.44:0/477827794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e340ffee0 msgr2=0x7f1e341016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:48.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.426+0000 7f1e3ad49700 1 --2- 172.21.15.44:0/477827794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e340ffee0 0x7f1e341016c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1e20006010 tx=0x7f1e20004730 comp rx=0 tx=0).stop 2023-09-10T00:39:48.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.426+0000 7f1e3ad49700 1 -- 172.21.15.44:0/477827794 shutdown_connections 2023-09-10T00:39:48.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.426+0000 7f1e3ad49700 1 --2- 172.21.15.44:0/477827794 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1e240a0750 0x7f1e240a2c10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:48.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.426+0000 7f1e3ad49700 1 --2- 172.21.15.44:0/477827794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e34104530 0x7f1e34101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:48.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.426+0000 7f1e3ad49700 1 --2- 172.21.15.44:0/477827794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e340ff520 0x7f1e34101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:48.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.426+0000 7f1e3ad49700 1 --2- 172.21.15.44:0/477827794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e340ffee0 0x7f1e341016c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:48.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.426+0000 7f1e3ad49700 1 -- 172.21.15.44:0/477827794 >> 172.21.15.44:0/477827794 conn(0x7f1e34076600 msgr2=0x7f1e340680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:48.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.426+0000 7f1e3ad49700 1 -- 172.21.15.44:0/477827794 shutdown_connections 2023-09-10T00:39:48.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.426+0000 7f1e3ad49700 1 -- 172.21.15.44:0/477827794 wait complete. 2023-09-10T00:39:48.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:48 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/477827794' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-10T00:39:48.787 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-09-10T00:39:48.788 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:39:48.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:48 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/477827794' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-10T00:39:48.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.918+0000 7fccf4f02700 1 -- 172.21.15.44:0/1662755820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccf00ff4e0 msgr2=0x7fccf00ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:48.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.918+0000 7fccf4f02700 1 --2- 172.21.15.44:0/1662755820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccf00ff4e0 0x7fccf00ff900 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcce40099c0 tx=0x7fcce4009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:48.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.918+0000 7fccf4f02700 1 -- 172.21.15.44:0/1662755820 shutdown_connections 2023-09-10T00:39:48.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.918+0000 7fccf4f02700 1 --2- 172.21.15.44:0/1662755820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf0101070 0x7fccf0107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:48.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.918+0000 7fccf4f02700 1 --2- 172.21.15.44:0/1662755820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccf00ff4e0 0x7fccf00ff900 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:48.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.918+0000 7fccf4f02700 1 --2- 172.21.15.44:0/1662755820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccf01006d0 0x7fccf0100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:48.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.918+0000 7fccf4f02700 1 -- 172.21.15.44:0/1662755820 >> 172.21.15.44:0/1662755820 conn(0x7fccf00faa30 msgr2=0x7fccf00fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:48.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.918+0000 7fccf4f02700 1 -- 172.21.15.44:0/1662755820 shutdown_connections 2023-09-10T00:39:48.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.918+0000 7fccf4f02700 1 -- 172.21.15.44:0/1662755820 wait complete. 2023-09-10T00:39:48.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.918+0000 7fccf4f02700 1 Processor -- start 2023-09-10T00:39:48.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.918+0000 7fccf4f02700 1 -- start start 2023-09-10T00:39:48.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccf4f02700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf00ff4e0 0x7fccf00682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:48.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccf4f02700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccf01006d0 0x7fccf019af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:48.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccf4f02700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccf0101070 0x7fccf019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:48.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccf4f02700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf019dd40 con 0x7fccf01006d0 2023-09-10T00:39:48.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccf4f02700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf019deb0 con 0x7fccf0101070 2023-09-10T00:39:48.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccf4f02700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf019e070 con 0x7fccf00ff4e0 2023-09-10T00:39:48.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccedd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccf01006d0 0x7fccf019af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:48.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccedd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccf01006d0 0x7fccf019af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47344/0 (socket says 172.21.15.44:47344) 2023-09-10T00:39:48.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccedd9b700 1 -- 172.21.15.44:0/1747041133 learned_addr learned my addr 172.21.15.44:0/1747041133 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:48.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccee59c700 1 --2- 172.21.15.44:0/1747041133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf00ff4e0 0x7fccf00682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:48.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fcceed9d700 1 --2- 172.21.15.44:0/1747041133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccf0101070 0x7fccf019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:48.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccedd9b700 1 -- 172.21.15.44:0/1747041133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf00ff4e0 msgr2=0x7fccf00682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:48.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccedd9b700 1 --2- 172.21.15.44:0/1747041133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf00ff4e0 0x7fccf00682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:48.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccedd9b700 1 -- 172.21.15.44:0/1747041133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccf0101070 msgr2=0x7fccf019d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:48.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccedd9b700 1 --2- 172.21.15.44:0/1747041133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccf0101070 0x7fccf019d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:48.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccedd9b700 1 -- 172.21.15.44:0/1747041133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce40095f0 con 0x7fccf01006d0 2023-09-10T00:39:48.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fcceed9d700 1 --2- 172.21.15.44:0/1747041133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccf0101070 0x7fccf019d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:48.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccedd9b700 1 --2- 172.21.15.44:0/1747041133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccf01006d0 0x7fccf019af20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcce000d950 tx=0x7fcce000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:48.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.919+0000 7fccdf7fe700 1 -- 172.21.15.44:0/1747041133 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce0009940 con 0x7fccf01006d0 2023-09-10T00:39:48.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.920+0000 7fccdf7fe700 1 -- 172.21.15.44:0/1747041133 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcce0010460 con 0x7fccf01006d0 2023-09-10T00:39:48.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.920+0000 7fccf4f02700 1 -- 172.21.15.44:0/1747041133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf019e350 con 0x7fccf01006d0 2023-09-10T00:39:48.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.920+0000 7fccdf7fe700 1 -- 172.21.15.44:0/1747041133 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce000f620 con 0x7fccf01006d0 2023-09-10T00:39:48.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.920+0000 7fccf4f02700 1 -- 172.21.15.44:0/1747041133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf019e860 con 0x7fccf01006d0 2023-09-10T00:39:48.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.920+0000 7fccf4f02700 1 -- 172.21.15.44:0/1747041133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf010abc0 con 0x7fccf01006d0 2023-09-10T00:39:48.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.922+0000 7fccdf7fe700 1 -- 172.21.15.44:0/1747041133 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fcce00105d0 con 0x7fccf01006d0 2023-09-10T00:39:48.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.922+0000 7fccdf7fe700 1 --2- 172.21.15.44:0/1747041133 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fccd80a0710 0x7fccd80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:48.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.922+0000 7fccdf7fe700 1 -- 172.21.15.44:0/1747041133 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcce00cbe90 con 0x7fccf01006d0 2023-09-10T00:39:48.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.922+0000 7fccee59c700 1 --2- 172.21.15.44:0/1747041133 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fccd80a0710 0x7fccd80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:48.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.923+0000 7fccee59c700 1 --2- 172.21.15.44:0/1747041133 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fccd80a0710 0x7fccd80a2bd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcce400c010 tx=0x7fcce4005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:48.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:48.926+0000 7fccdf7fe700 1 -- 172.21.15.44:0/1747041133 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcce0095550 con 0x7fccf01006d0 2023-09-10T00:39:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:48 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/477827794' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-10T00:39:49.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.132+0000 7fccf4f02700 1 -- 172.21.15.44:0/1747041133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccf019ea80 con 0x7fccf01006d0 2023-09-10T00:39:49.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.134+0000 7fccdf7fe700 1 -- 172.21.15.44:0/1747041133 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24340 (secure 0 0 0) 0x7fcce0016090 con 0x7fccf01006d0 2023-09-10T00:39:49.135 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:49.136 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":89,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:39:17.515738+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:39:49.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.136+0000 7fccf4f02700 1 -- 172.21.15.44:0/1747041133 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fccd80a0710 msgr2=0x7fccd80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:49.138 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.136+0000 7fccf4f02700 1 --2- 172.21.15.44:0/1747041133 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fccd80a0710 0x7fccd80a2bd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcce400c010 tx=0x7fcce4005e80 comp rx=0 tx=0).stop 2023-09-10T00:39:49.138 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.136+0000 7fccf4f02700 1 -- 172.21.15.44:0/1747041133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccf01006d0 msgr2=0x7fccf019af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:49.138 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.136+0000 7fccf4f02700 1 --2- 172.21.15.44:0/1747041133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccf01006d0 0x7fccf019af20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcce000d950 tx=0x7fcce000dc60 comp rx=0 tx=0).stop 2023-09-10T00:39:49.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.136+0000 7fccf4f02700 1 -- 172.21.15.44:0/1747041133 shutdown_connections 2023-09-10T00:39:49.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.136+0000 7fccf4f02700 1 --2- 172.21.15.44:0/1747041133 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fccd80a0710 0x7fccd80a2bd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.136+0000 7fccf4f02700 1 --2- 172.21.15.44:0/1747041133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccf0101070 0x7fccf019d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.136+0000 7fccf4f02700 1 --2- 172.21.15.44:0/1747041133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf00ff4e0 0x7fccf00682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.136+0000 7fccf4f02700 1 --2- 172.21.15.44:0/1747041133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccf01006d0 0x7fccf019af20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.136+0000 7fccf4f02700 1 -- 172.21.15.44:0/1747041133 >> 172.21.15.44:0/1747041133 conn(0x7fccf00faa30 msgr2=0x7fccf00fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:49.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.136+0000 7fccf4f02700 1 -- 172.21.15.44:0/1747041133 shutdown_connections 2023-09-10T00:39:49.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.136+0000 7fccf4f02700 1 -- 172.21.15.44:0/1747041133 wait complete. 2023-09-10T00:39:49.150 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-10T00:34:44.456741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-10T00:38:37.194891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-10T00:38:38.573268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-10T00:38:42.198896+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-10T00:39:49.151 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-10T00:39:49.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.279+0000 7f79682b0700 1 -- 172.21.15.44:0/4254057505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79600fecf0 msgr2=0x7f79600ff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:49.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.279+0000 7f79682b0700 1 --2- 172.21.15.44:0/4254057505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79600fecf0 0x7f79600ff110 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f79540099c0 tx=0x7f7954009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.279+0000 7f79682b0700 1 -- 172.21.15.44:0/4254057505 shutdown_connections 2023-09-10T00:39:49.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.279+0000 7f79682b0700 1 --2- 172.21.15.44:0/4254057505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7960100880 0x7f7960107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.279+0000 7f79682b0700 1 --2- 172.21.15.44:0/4254057505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79600fecf0 0x7f79600ff110 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.279+0000 7f79682b0700 1 --2- 172.21.15.44:0/4254057505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79600ffee0 0x7f7960100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.279+0000 7f79682b0700 1 -- 172.21.15.44:0/4254057505 >> 172.21.15.44:0/4254057505 conn(0x7f79600fa260 msgr2=0x7f79600fc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:49.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.279+0000 7f79682b0700 1 -- 172.21.15.44:0/4254057505 shutdown_connections 2023-09-10T00:39:49.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.279+0000 7f79682b0700 1 -- 172.21.15.44:0/4254057505 wait complete. 2023-09-10T00:39:49.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f79682b0700 1 Processor -- start 2023-09-10T00:39:49.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f79682b0700 1 -- start start 2023-09-10T00:39:49.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f79682b0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79600fecf0 0x7f79600fc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:49.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f79682b0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79600ffee0 0x7f796019af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:49.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f79682b0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7960100880 0x7f796019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:49.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f79682b0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796019dd40 con 0x7f7960100880 2023-09-10T00:39:49.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f79682b0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796019deb0 con 0x7f79600fecf0 2023-09-10T00:39:49.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f79682b0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796019e070 con 0x7f79600ffee0 2023-09-10T00:39:49.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f796684d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7960100880 0x7f796019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:49.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f796684d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7960100880 0x7f796019d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47358/0 (socket says 172.21.15.44:47358) 2023-09-10T00:39:49.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f796684d700 1 -- 172.21.15.44:0/3033292574 learned_addr learned my addr 172.21.15.44:0/3033292574 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:49.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f796604c700 1 --2- 172.21.15.44:0/3033292574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79600fecf0 0x7f79600fc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:49.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f796584b700 1 --2- 172.21.15.44:0/3033292574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79600ffee0 0x7f796019af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:49.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f796684d700 1 -- 172.21.15.44:0/3033292574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79600ffee0 msgr2=0x7f796019af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:49.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f796684d700 1 --2- 172.21.15.44:0/3033292574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79600ffee0 0x7f796019af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f796684d700 1 -- 172.21.15.44:0/3033292574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79600fecf0 msgr2=0x7f79600fc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:49.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f796684d700 1 --2- 172.21.15.44:0/3033292574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79600fecf0 0x7f79600fc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f796684d700 1 -- 172.21.15.44:0/3033292574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79540095f0 con 0x7f7960100880 2023-09-10T00:39:49.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f796684d700 1 --2- 172.21.15.44:0/3033292574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7960100880 0x7f796019d5a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f795800ef30 tx=0x7f795800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:49.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.280+0000 7f79537fe700 1 -- 172.21.15.44:0/3033292574 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958018070 con 0x7f7960100880 2023-09-10T00:39:49.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.281+0000 7f79537fe700 1 -- 172.21.15.44:0/3033292574 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f795800f040 con 0x7f7960100880 2023-09-10T00:39:49.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.281+0000 7f79682b0700 1 -- 172.21.15.44:0/3033292574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796019e2f0 con 0x7f7960100880 2023-09-10T00:39:49.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.281+0000 7f79537fe700 1 -- 172.21.15.44:0/3033292574 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795800cc70 con 0x7f7960100880 2023-09-10T00:39:49.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.281+0000 7f79682b0700 1 -- 172.21.15.44:0/3033292574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796019e7b0 con 0x7f7960100880 2023-09-10T00:39:49.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.282+0000 7f79682b0700 1 -- 172.21.15.44:0/3033292574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7960106860 con 0x7f7960100880 2023-09-10T00:39:49.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.283+0000 7f79537fe700 1 -- 172.21.15.44:0/3033292574 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f7958007500 con 0x7f7960100880 2023-09-10T00:39:49.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.283+0000 7f79537fe700 1 --2- 172.21.15.44:0/3033292574 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f794c0a06d0 0x7f794c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:49.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.283+0000 7f79537fe700 1 -- 172.21.15.44:0/3033292574 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f79580cbbe0 con 0x7f7960100880 2023-09-10T00:39:49.293 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.283+0000 7f796604c700 1 --2- 172.21.15.44:0/3033292574 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f794c0a06d0 0x7f794c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:49.293 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.284+0000 7f796604c700 1 --2- 172.21.15.44:0/3033292574 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f794c0a06d0 0x7f794c0a2b90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7954000c00 tx=0x7f795400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:49.293 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.287+0000 7f79537fe700 1 -- 172.21.15.44:0/3033292574 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7958095330 con 0x7f7960100880 2023-09-10T00:39:49.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.492+0000 7f79682b0700 1 -- 172.21.15.44:0/3033292574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f796010a590 con 0x7f7960100880 2023-09-10T00:39:49.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.493+0000 7f79537fe700 1 -- 172.21.15.44:0/3033292574 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7958098950 con 0x7f7960100880 2023-09-10T00:39:49.495 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2023-09-10T00:39:49.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.495+0000 7f79682b0700 1 -- 172.21.15.44:0/3033292574 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f794c0a06d0 msgr2=0x7f794c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:49.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.495+0000 7f79682b0700 1 --2- 172.21.15.44:0/3033292574 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f794c0a06d0 0x7f794c0a2b90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7954000c00 tx=0x7f795400b540 comp rx=0 tx=0).stop 2023-09-10T00:39:49.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.495+0000 7f79682b0700 1 -- 172.21.15.44:0/3033292574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7960100880 msgr2=0x7f796019d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:49.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.495+0000 7f79682b0700 1 --2- 172.21.15.44:0/3033292574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7960100880 0x7f796019d5a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f795800ef30 tx=0x7f795800c3a0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.498 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.495+0000 7f79682b0700 1 -- 172.21.15.44:0/3033292574 shutdown_connections 2023-09-10T00:39:49.498 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.495+0000 7f79682b0700 1 --2- 172.21.15.44:0/3033292574 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f794c0a06d0 0x7f794c0a2b90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.498 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.495+0000 7f79682b0700 1 --2- 172.21.15.44:0/3033292574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7960100880 0x7f796019d5a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.498 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.495+0000 7f79682b0700 1 --2- 172.21.15.44:0/3033292574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79600fecf0 0x7f79600fc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.495+0000 7f79682b0700 1 --2- 172.21.15.44:0/3033292574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79600ffee0 0x7f796019af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.495+0000 7f79682b0700 1 -- 172.21.15.44:0/3033292574 >> 172.21.15.44:0/3033292574 conn(0x7f79600fa260 msgr2=0x7f79601039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:49.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.495+0000 7f79682b0700 1 -- 172.21.15.44:0/3033292574 shutdown_connections 2023-09-10T00:39:49.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.495+0000 7f79682b0700 1 -- 172.21.15.44:0/3033292574 wait complete. 2023-09-10T00:39:49.509 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-10T00:39:49.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.638+0000 7fe19556c700 1 -- 172.21.15.44:0/1409571274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1901019c0 msgr2=0x7fe190101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:49.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.638+0000 7fe19556c700 1 --2- 172.21.15.44:0/1409571274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1901019c0 0x7fe190101e20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe180009990 tx=0x7fe180009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.638+0000 7fe19556c700 1 -- 172.21.15.44:0/1409571274 shutdown_connections 2023-09-10T00:39:49.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.638+0000 7fe19556c700 1 --2- 172.21.15.44:0/1409571274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe190102360 0x7fe190108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.638+0000 7fe19556c700 1 --2- 172.21.15.44:0/1409571274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1900fe950 0x7fe1900fed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.638+0000 7fe19556c700 1 --2- 172.21.15.44:0/1409571274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1901019c0 0x7fe190101e20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.638+0000 7fe19556c700 1 -- 172.21.15.44:0/1409571274 >> 172.21.15.44:0/1409571274 conn(0x7fe1900fa260 msgr2=0x7fe1900fc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:49.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.638+0000 7fe19556c700 1 -- 172.21.15.44:0/1409571274 shutdown_connections 2023-09-10T00:39:49.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.638+0000 7fe19556c700 1 -- 172.21.15.44:0/1409571274 wait complete. 2023-09-10T00:39:49.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.639+0000 7fe19556c700 1 Processor -- start 2023-09-10T00:39:49.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.639+0000 7fe19556c700 1 -- start start 2023-09-10T00:39:49.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.639+0000 7fe19556c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1900fe950 0x7fe1901962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:49.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.639+0000 7fe19556c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1901019c0 0x7fe1901967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:49.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.639+0000 7fe19556c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe190102360 0x7fe19019d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:49.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.639+0000 7fe19556c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19019de20 con 0x7fe1901019c0 2023-09-10T00:39:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.639+0000 7fe19556c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19019df60 con 0x7fe190102360 2023-09-10T00:39:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.639+0000 7fe19556c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19019e0f0 con 0x7fe1900fe950 2023-09-10T00:39:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.639+0000 7fe18e7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1901019c0 0x7fe1901967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.639+0000 7fe18e7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1901019c0 0x7fe1901967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47368/0 (socket says 172.21.15.44:47368) 2023-09-10T00:39:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.639+0000 7fe18e7fc700 1 -- 172.21.15.44:0/2296001381 learned_addr learned my addr 172.21.15.44:0/2296001381 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:49.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.639+0000 7fe18f7fe700 1 --2- 172.21.15.44:0/2296001381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe190102360 0x7fe19019d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:49.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.640+0000 7fe18effd700 1 --2- 172.21.15.44:0/2296001381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1900fe950 0x7fe1901962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:49.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.640+0000 7fe18e7fc700 1 -- 172.21.15.44:0/2296001381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1900fe950 msgr2=0x7fe1901962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:49.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.640+0000 7fe18e7fc700 1 --2- 172.21.15.44:0/2296001381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1900fe950 0x7fe1901962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.640+0000 7fe18e7fc700 1 -- 172.21.15.44:0/2296001381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe190102360 msgr2=0x7fe19019d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:49.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.640+0000 7fe18e7fc700 1 --2- 172.21.15.44:0/2296001381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe190102360 0x7fe19019d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.640+0000 7fe18e7fc700 1 -- 172.21.15.44:0/2296001381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1800095f0 con 0x7fe1901019c0 2023-09-10T00:39:49.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.640+0000 7fe18f7fe700 1 --2- 172.21.15.44:0/2296001381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe190102360 0x7fe19019d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:49.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.640+0000 7fe18e7fc700 1 --2- 172.21.15.44:0/2296001381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1901019c0 0x7fe1901967e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe1800098e0 tx=0x7fe180005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:49.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.640+0000 7fe177fff700 1 -- 172.21.15.44:0/2296001381 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18001d070 con 0x7fe1901019c0 2023-09-10T00:39:49.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.640+0000 7fe177fff700 1 -- 172.21.15.44:0/2296001381 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe18000f460 con 0x7fe1901019c0 2023-09-10T00:39:49.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.640+0000 7fe177fff700 1 -- 172.21.15.44:0/2296001381 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe180021670 con 0x7fe1901019c0 2023-09-10T00:39:49.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.640+0000 7fe19556c700 1 -- 172.21.15.44:0/2296001381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe19019e370 con 0x7fe1901019c0 2023-09-10T00:39:49.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.640+0000 7fe19556c700 1 -- 172.21.15.44:0/2296001381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe19019e740 con 0x7fe1901019c0 2023-09-10T00:39:49.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.641+0000 7fe19556c700 1 -- 172.21.15.44:0/2296001381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe19010bcf0 con 0x7fe1901019c0 2023-09-10T00:39:49.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.643+0000 7fe177fff700 1 -- 172.21.15.44:0/2296001381 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fe18000f5d0 con 0x7fe1901019c0 2023-09-10T00:39:49.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.643+0000 7fe177fff700 1 --2- 172.21.15.44:0/2296001381 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe1780a06d0 0x7fe1780a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:49.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.643+0000 7fe18effd700 1 --2- 172.21.15.44:0/2296001381 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe1780a06d0 0x7fe1780a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:49.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.643+0000 7fe177fff700 1 -- 172.21.15.44:0/2296001381 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe1800cd610 con 0x7fe1901019c0 2023-09-10T00:39:49.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.644+0000 7fe18effd700 1 --2- 172.21.15.44:0/2296001381 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe1780a06d0 0x7fe1780a2b90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe184005950 tx=0x7fe1840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:49.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.647+0000 7fe177fff700 1 -- 172.21.15.44:0/2296001381 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe180096e80 con 0x7fe1901019c0 2023-09-10T00:39:49.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:49 smithi044 ceph-mon[98676]: pgmap v305: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:39:49.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:49 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1747041133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:49.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:49 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3033292574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:39:49.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.848+0000 7fe19556c700 1 -- 172.21.15.44:0/2296001381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe190107810 con 0x7fe1901019c0 2023-09-10T00:39:49.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.848+0000 7fe177fff700 1 -- 172.21.15.44:0/2296001381 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe1800260a0 con 0x7fe1901019c0 2023-09-10T00:39:49.850 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:39:49.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.850+0000 7fe19556c700 1 -- 172.21.15.44:0/2296001381 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe1780a06d0 msgr2=0x7fe1780a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:49.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.850+0000 7fe19556c700 1 --2- 172.21.15.44:0/2296001381 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe1780a06d0 0x7fe1780a2b90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe184005950 tx=0x7fe1840058e0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.850+0000 7fe19556c700 1 -- 172.21.15.44:0/2296001381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1901019c0 msgr2=0x7fe1901967e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:49.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.850+0000 7fe19556c700 1 --2- 172.21.15.44:0/2296001381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1901019c0 0x7fe1901967e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe1800098e0 tx=0x7fe180005ef0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.850+0000 7fe19556c700 1 -- 172.21.15.44:0/2296001381 shutdown_connections 2023-09-10T00:39:49.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.850+0000 7fe19556c700 1 --2- 172.21.15.44:0/2296001381 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe1780a06d0 0x7fe1780a2b90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.850+0000 7fe19556c700 1 --2- 172.21.15.44:0/2296001381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe190102360 0x7fe19019d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.850+0000 7fe19556c700 1 --2- 172.21.15.44:0/2296001381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1900fe950 0x7fe1901962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.850+0000 7fe19556c700 1 --2- 172.21.15.44:0/2296001381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1901019c0 0x7fe1901967e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.850+0000 7fe19556c700 1 -- 172.21.15.44:0/2296001381 >> 172.21.15.44:0/2296001381 conn(0x7fe1900fa260 msgr2=0x7fe1900fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:49.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.850+0000 7fe19556c700 1 -- 172.21.15.44:0/2296001381 shutdown_connections 2023-09-10T00:39:49.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.850+0000 7fe19556c700 1 -- 172.21.15.44:0/2296001381 wait complete. 2023-09-10T00:39:49.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:49 smithi099 ceph-mon[102692]: pgmap v305: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:39:49.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:49 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1747041133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:49.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:49 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3033292574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:39:49.864 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-10T00:39:49.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.993+0000 7f87f38c8700 1 -- 172.21.15.44:0/4030788686 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ec101080 msgr2=0x7f87ec107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:49.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.993+0000 7f87f38c8700 1 --2- 172.21.15.44:0/4030788686 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ec101080 0x7f87ec107930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f87e400b0d0 tx=0x7f87e400b3e0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.993+0000 7f87f38c8700 1 -- 172.21.15.44:0/4030788686 shutdown_connections 2023-09-10T00:39:49.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.993+0000 7f87f38c8700 1 --2- 172.21.15.44:0/4030788686 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ec101080 0x7f87ec107930 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.993+0000 7f87f38c8700 1 --2- 172.21.15.44:0/4030788686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87ec0ff4f0 0x7f87ec0ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.993+0000 7f87f38c8700 1 --2- 172.21.15.44:0/4030788686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87ec1006e0 0x7f87ec100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:49.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.993+0000 7f87f38c8700 1 -- 172.21.15.44:0/4030788686 >> 172.21.15.44:0/4030788686 conn(0x7f87ec0faa60 msgr2=0x7f87ec0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.993+0000 7f87f38c8700 1 -- 172.21.15.44:0/4030788686 shutdown_connections 2023-09-10T00:39:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.993+0000 7f87f38c8700 1 -- 172.21.15.44:0/4030788686 wait complete. 2023-09-10T00:39:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f38c8700 1 Processor -- start 2023-09-10T00:39:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f38c8700 1 -- start start 2023-09-10T00:39:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f38c8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87ec0ff4f0 0x7f87ec068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f38c8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ec1006e0 0x7f87ec19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f38c8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87ec101080 0x7f87ec19f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f38c8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1a0000 con 0x7f87ec1006e0 2023-09-10T00:39:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f38c8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1a0170 con 0x7f87ec0ff4f0 2023-09-10T00:39:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f38c8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1a0330 con 0x7f87ec101080 2023-09-10T00:39:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f0e63700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ec1006e0 0x7f87ec19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f0e63700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ec1006e0 0x7f87ec19d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47382/0 (socket says 172.21.15.44:47382) 2023-09-10T00:39:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f0e63700 1 -- 172.21.15.44:0/3116094949 learned_addr learned my addr 172.21.15.44:0/3116094949 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f1664700 1 --2- 172.21.15.44:0/3116094949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87ec0ff4f0 0x7f87ec068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f0e63700 1 -- 172.21.15.44:0/3116094949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87ec101080 msgr2=0x7f87ec19f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f1e65700 1 --2- 172.21.15.44:0/3116094949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87ec101080 0x7f87ec19f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f0e63700 1 --2- 172.21.15.44:0/3116094949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87ec101080 0x7f87ec19f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f0e63700 1 -- 172.21.15.44:0/3116094949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87ec0ff4f0 msgr2=0x7f87ec068330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f0e63700 1 --2- 172.21.15.44:0/3116094949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87ec0ff4f0 0x7f87ec068330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.994+0000 7f87f0e63700 1 -- 172.21.15.44:0/3116094949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e4009d00 con 0x7f87ec1006e0 2023-09-10T00:39:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.995+0000 7f87f0e63700 1 --2- 172.21.15.44:0/3116094949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ec1006e0 0x7f87ec19d3d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f87e800d980 tx=0x7f87e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.995+0000 7f87e27fc700 1 -- 172.21.15.44:0/3116094949 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8004220 con 0x7f87ec1006e0 2023-09-10T00:39:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.995+0000 7f87e27fc700 1 -- 172.21.15.44:0/3116094949 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87e8007640 con 0x7f87ec1006e0 2023-09-10T00:39:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.995+0000 7f87f38c8700 1 -- 172.21.15.44:0/3116094949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec1a0610 con 0x7f87ec1006e0 2023-09-10T00:39:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.995+0000 7f87e27fc700 1 -- 172.21.15.44:0/3116094949 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e80055e0 con 0x7f87ec1006e0 2023-09-10T00:39:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.995+0000 7f87f38c8700 1 -- 172.21.15.44:0/3116094949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec1a0b70 con 0x7f87ec1006e0 2023-09-10T00:39:50.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.996+0000 7f87f38c8700 1 -- 172.21.15.44:0/3116094949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ec106470 con 0x7f87ec1006e0 2023-09-10T00:39:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.998+0000 7f87e27fc700 1 -- 172.21.15.44:0/3116094949 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f87e8004750 con 0x7f87ec1006e0 2023-09-10T00:39:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.998+0000 7f87e27fc700 1 --2- 172.21.15.44:0/3116094949 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f87d80a0750 0x7f87d80a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.998+0000 7f87f1664700 1 --2- 172.21.15.44:0/3116094949 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f87d80a0750 0x7f87d80a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.998+0000 7f87e27fc700 1 -- 172.21.15.44:0/3116094949 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f87e80cc290 con 0x7f87ec1006e0 2023-09-10T00:39:50.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:49.999+0000 7f87f1664700 1 --2- 172.21.15.44:0/3116094949 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f87d80a0750 0x7f87d80a2c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f87ec100540 tx=0x7f87dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:50.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.002+0000 7f87e27fc700 1 -- 172.21.15.44:0/3116094949 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f87e8095950 con 0x7f87ec1006e0 2023-09-10T00:39:50.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:49 smithi131 ceph-mon[103713]: pgmap v305: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:39:50.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:49 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1747041133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:50.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:49 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3033292574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:39:50.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.201+0000 7f87f38c8700 1 -- 172.21.15.44:0/3116094949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f87ec10ad40 con 0x7f87ec1006e0 2023-09-10T00:39:50.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.202+0000 7f87e27fc700 1 -- 172.21.15.44:0/3116094949 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f87e8016070 con 0x7f87ec1006e0 2023-09-10T00:39:50.204 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2023-09-10T00:39:50.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.204+0000 7f87f38c8700 1 -- 172.21.15.44:0/3116094949 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f87d80a0750 msgr2=0x7f87d80a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:50.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.204+0000 7f87f38c8700 1 --2- 172.21.15.44:0/3116094949 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f87d80a0750 0x7f87d80a2c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f87ec100540 tx=0x7f87dc009260 comp rx=0 tx=0).stop 2023-09-10T00:39:50.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.204+0000 7f87f38c8700 1 -- 172.21.15.44:0/3116094949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ec1006e0 msgr2=0x7f87ec19d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:50.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.204+0000 7f87f38c8700 1 --2- 172.21.15.44:0/3116094949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ec1006e0 0x7f87ec19d3d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f87e800d980 tx=0x7f87e800dc90 comp rx=0 tx=0).stop 2023-09-10T00:39:50.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.204+0000 7f87f38c8700 1 -- 172.21.15.44:0/3116094949 shutdown_connections 2023-09-10T00:39:50.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.204+0000 7f87f38c8700 1 --2- 172.21.15.44:0/3116094949 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f87d80a0750 0x7f87d80a2c10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.204+0000 7f87f38c8700 1 --2- 172.21.15.44:0/3116094949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87ec101080 0x7f87ec19f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.204+0000 7f87f38c8700 1 --2- 172.21.15.44:0/3116094949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87ec0ff4f0 0x7f87ec068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.204+0000 7f87f38c8700 1 --2- 172.21.15.44:0/3116094949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ec1006e0 0x7f87ec19d3d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.204+0000 7f87f38c8700 1 -- 172.21.15.44:0/3116094949 >> 172.21.15.44:0/3116094949 conn(0x7f87ec0faa60 msgr2=0x7f87ec0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:50.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.204+0000 7f87f38c8700 1 -- 172.21.15.44:0/3116094949 shutdown_connections 2023-09-10T00:39:50.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.204+0000 7f87f38c8700 1 -- 172.21.15.44:0/3116094949 wait complete. 2023-09-10T00:39:50.218 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-10T00:39:50.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.345+0000 7fe63fdd0700 1 -- 172.21.15.44:0/4214818959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6381021f0 msgr2=0x7fe638108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:50.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.345+0000 7fe63fdd0700 1 --2- 172.21.15.44:0/4214818959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6381021f0 0x7fe638108aa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe63000b0d0 tx=0x7fe63000b3e0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.345+0000 7fe63fdd0700 1 -- 172.21.15.44:0/4214818959 shutdown_connections 2023-09-10T00:39:50.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.345+0000 7fe63fdd0700 1 --2- 172.21.15.44:0/4214818959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6381021f0 0x7fe638108aa0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.346+0000 7fe63fdd0700 1 --2- 172.21.15.44:0/4214818959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe638100660 0x7fe638100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.346+0000 7fe63fdd0700 1 --2- 172.21.15.44:0/4214818959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe638101850 0x7fe638101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.346+0000 7fe63fdd0700 1 -- 172.21.15.44:0/4214818959 >> 172.21.15.44:0/4214818959 conn(0x7fe6380fbbd0 msgr2=0x7fe6380fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:50.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.346+0000 7fe63fdd0700 1 -- 172.21.15.44:0/4214818959 shutdown_connections 2023-09-10T00:39:50.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.346+0000 7fe63fdd0700 1 -- 172.21.15.44:0/4214818959 wait complete. 2023-09-10T00:39:50.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.346+0000 7fe63fdd0700 1 Processor -- start 2023-09-10T00:39:50.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.346+0000 7fe63fdd0700 1 -- start start 2023-09-10T00:39:50.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.346+0000 7fe63fdd0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe638100660 0x7fe6380743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:50.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.346+0000 7fe63fdd0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe638101850 0x7fe6380729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:50.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.346+0000 7fe63fdd0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6381021f0 0x7fe638072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:50.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.346+0000 7fe63fdd0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6380736d0 con 0x7fe638101850 2023-09-10T00:39:50.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.346+0000 7fe63fdd0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe638073840 con 0x7fe6381021f0 2023-09-10T00:39:50.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.346+0000 7fe63fdd0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6381a0370 con 0x7fe638100660 2023-09-10T00:39:50.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe63d36b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe638101850 0x7fe6380729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:50.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe63d36b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe638101850 0x7fe6380729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47396/0 (socket says 172.21.15.44:47396) 2023-09-10T00:39:50.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe63d36b700 1 -- 172.21.15.44:0/1361559818 learned_addr learned my addr 172.21.15.44:0/1361559818 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:50.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe63db6c700 1 --2- 172.21.15.44:0/1361559818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe638100660 0x7fe6380743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:50.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe63e36d700 1 --2- 172.21.15.44:0/1361559818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6381021f0 0x7fe638072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:50.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe63d36b700 1 -- 172.21.15.44:0/1361559818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe638100660 msgr2=0x7fe6380743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:50.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe63d36b700 1 --2- 172.21.15.44:0/1361559818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe638100660 0x7fe6380743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe63d36b700 1 -- 172.21.15.44:0/1361559818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6381021f0 msgr2=0x7fe638072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:50.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe63d36b700 1 --2- 172.21.15.44:0/1361559818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6381021f0 0x7fe638072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe63d36b700 1 -- 172.21.15.44:0/1361559818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe634009520 con 0x7fe638101850 2023-09-10T00:39:50.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe63db6c700 1 --2- 172.21.15.44:0/1361559818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe638100660 0x7fe6380743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:39:50.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe63d36b700 1 --2- 172.21.15.44:0/1361559818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe638101850 0x7fe6380729f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe63400eb30 tx=0x7fe63400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:50.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe62affd700 1 -- 172.21.15.44:0/1361559818 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe634009880 con 0x7fe638101850 2023-09-10T00:39:50.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe63fdd0700 1 -- 172.21.15.44:0/1361559818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe630009d00 con 0x7fe638101850 2023-09-10T00:39:50.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe62affd700 1 -- 172.21.15.44:0/1361559818 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe6340044b0 con 0x7fe638101850 2023-09-10T00:39:50.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe62affd700 1 -- 172.21.15.44:0/1361559818 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe634003e40 con 0x7fe638101850 2023-09-10T00:39:50.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.347+0000 7fe63fdd0700 1 -- 172.21.15.44:0/1361559818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6381a0a20 con 0x7fe638101850 2023-09-10T00:39:50.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.348+0000 7fe63fdd0700 1 -- 172.21.15.44:0/1361559818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63810bc70 con 0x7fe638101850 2023-09-10T00:39:50.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.349+0000 7fe62affd700 1 -- 172.21.15.44:0/1361559818 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fe634003fa0 con 0x7fe638101850 2023-09-10T00:39:50.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.350+0000 7fe62affd700 1 --2- 172.21.15.44:0/1361559818 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe6240a06d0 0x7fe6240a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:50.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.350+0000 7fe63db6c700 1 --2- 172.21.15.44:0/1361559818 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe6240a06d0 0x7fe6240a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:50.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.350+0000 7fe62affd700 1 -- 172.21.15.44:0/1361559818 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe634014070 con 0x7fe638101850 2023-09-10T00:39:50.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.350+0000 7fe63db6c700 1 --2- 172.21.15.44:0/1361559818 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe6240a06d0 0x7fe6240a2b90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe62c005fd0 tx=0x7fe62c005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:50.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.353+0000 7fe62affd700 1 -- 172.21.15.44:0/1361559818 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe634050e10 con 0x7fe638101850 2023-09-10T00:39:50.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.555+0000 7fe63fdd0700 1 -- 172.21.15.44:0/1361559818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe638107600 con 0x7fe638101850 2023-09-10T00:39:50.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.556+0000 7fe62affd700 1 -- 172.21.15.44:0/1361559818 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe634095cf0 con 0x7fe638101850 2023-09-10T00:39:50.557 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:39:50.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.558+0000 7fe63fdd0700 1 -- 172.21.15.44:0/1361559818 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe6240a06d0 msgr2=0x7fe6240a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:50.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.558+0000 7fe63fdd0700 1 --2- 172.21.15.44:0/1361559818 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe6240a06d0 0x7fe6240a2b90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe62c005fd0 tx=0x7fe62c005f00 comp rx=0 tx=0).stop 2023-09-10T00:39:50.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.558+0000 7fe63fdd0700 1 -- 172.21.15.44:0/1361559818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe638101850 msgr2=0x7fe6380729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:50.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.558+0000 7fe63fdd0700 1 --2- 172.21.15.44:0/1361559818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe638101850 0x7fe6380729f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe63400eb30 tx=0x7fe63400eef0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.558+0000 7fe63fdd0700 1 -- 172.21.15.44:0/1361559818 shutdown_connections 2023-09-10T00:39:50.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.558+0000 7fe63fdd0700 1 --2- 172.21.15.44:0/1361559818 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe6240a06d0 0x7fe6240a2b90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.558+0000 7fe63fdd0700 1 --2- 172.21.15.44:0/1361559818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6381021f0 0x7fe638072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.558+0000 7fe63fdd0700 1 --2- 172.21.15.44:0/1361559818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe638100660 0x7fe6380743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.558+0000 7fe63fdd0700 1 --2- 172.21.15.44:0/1361559818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe638101850 0x7fe6380729f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.558+0000 7fe63fdd0700 1 -- 172.21.15.44:0/1361559818 >> 172.21.15.44:0/1361559818 conn(0x7fe6380fbbd0 msgr2=0x7fe6380fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:50.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.558+0000 7fe63fdd0700 1 -- 172.21.15.44:0/1361559818 shutdown_connections 2023-09-10T00:39:50.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.558+0000 7fe63fdd0700 1 -- 172.21.15.44:0/1361559818 wait complete. 2023-09-10T00:39:50.571 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:39:50.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.698+0000 7fd85f733700 1 -- 172.21.15.44:0/1729278442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8581006d0 msgr2=0x7fd858100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:50.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.698+0000 7fd85f733700 1 --2- 172.21.15.44:0/1729278442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8581006d0 0x7fd858100b30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd8480099c0 tx=0x7fd848009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.698+0000 7fd85f733700 1 -- 172.21.15.44:0/1729278442 shutdown_connections 2023-09-10T00:39:50.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.698+0000 7fd85f733700 1 --2- 172.21.15.44:0/1729278442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd858101070 0x7fd858107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.698+0000 7fd85f733700 1 --2- 172.21.15.44:0/1729278442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8580ff4e0 0x7fd8580ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.698+0000 7fd85f733700 1 --2- 172.21.15.44:0/1729278442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8581006d0 0x7fd858100b30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.698+0000 7fd85f733700 1 -- 172.21.15.44:0/1729278442 >> 172.21.15.44:0/1729278442 conn(0x7fd8580faa30 msgr2=0x7fd8580fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:50.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.698+0000 7fd85f733700 1 -- 172.21.15.44:0/1729278442 shutdown_connections 2023-09-10T00:39:50.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.698+0000 7fd85f733700 1 -- 172.21.15.44:0/1729278442 wait complete. 2023-09-10T00:39:50.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.698+0000 7fd85f733700 1 Processor -- start 2023-09-10T00:39:50.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.698+0000 7fd85f733700 1 -- start start 2023-09-10T00:39:50.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85f733700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8580ff4e0 0x7fd858103470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:50.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85f733700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8581006d0 0x7fd858101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:50.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85f733700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd858101070 0x7fd858101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:50.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85f733700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd858102790 con 0x7fd8581006d0 2023-09-10T00:39:50.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85f733700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd858102900 con 0x7fd8580ff4e0 2023-09-10T00:39:50.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85f733700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8581a02f0 con 0x7fd858101070 2023-09-10T00:39:50.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85ccce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8581006d0 0x7fd858101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:50.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85ccce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8581006d0 0x7fd858101ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47410/0 (socket says 172.21.15.44:47410) 2023-09-10T00:39:50.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85ccce700 1 -- 172.21.15.44:0/2191886179 learned_addr learned my addr 172.21.15.44:0/2191886179 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:50.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85d4cf700 1 --2- 172.21.15.44:0/2191886179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8580ff4e0 0x7fd858103470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:50.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85dcd0700 1 --2- 172.21.15.44:0/2191886179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd858101070 0x7fd858101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:50.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85ccce700 1 -- 172.21.15.44:0/2191886179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd858101070 msgr2=0x7fd858101ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:50.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85ccce700 1 --2- 172.21.15.44:0/2191886179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd858101070 0x7fd858101ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85ccce700 1 -- 172.21.15.44:0/2191886179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8580ff4e0 msgr2=0x7fd858103470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:50.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85ccce700 1 --2- 172.21.15.44:0/2191886179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8580ff4e0 0x7fd858103470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85ccce700 1 -- 172.21.15.44:0/2191886179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8480095f0 con 0x7fd8581006d0 2023-09-10T00:39:50.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85dcd0700 1 --2- 172.21.15.44:0/2191886179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd858101070 0x7fd858101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:50.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85ccce700 1 --2- 172.21.15.44:0/2191886179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8581006d0 0x7fd858101ab0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd848009f50 tx=0x7fd848005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:50.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd84e7fc700 1 -- 172.21.15.44:0/2191886179 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84801d070 con 0x7fd8581006d0 2023-09-10T00:39:50.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd84e7fc700 1 -- 172.21.15.44:0/2191886179 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd848004830 con 0x7fd8581006d0 2023-09-10T00:39:50.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85f733700 1 -- 172.21.15.44:0/2191886179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8581a0570 con 0x7fd8581006d0 2023-09-10T00:39:50.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd84e7fc700 1 -- 172.21.15.44:0/2191886179 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848005310 con 0x7fd8581006d0 2023-09-10T00:39:50.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.699+0000 7fd85f733700 1 -- 172.21.15.44:0/2191886179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8581a0930 con 0x7fd8581006d0 2023-09-10T00:39:50.708 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.700+0000 7fd85f733700 1 -- 172.21.15.44:0/2191886179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85810ac00 con 0x7fd8581006d0 2023-09-10T00:39:50.709 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.701+0000 7fd84e7fc700 1 -- 172.21.15.44:0/2191886179 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fd8480049a0 con 0x7fd8581006d0 2023-09-10T00:39:50.709 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.702+0000 7fd84e7fc700 1 --2- 172.21.15.44:0/2191886179 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd8440a0710 0x7fd8440a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:50.709 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.702+0000 7fd85d4cf700 1 --2- 172.21.15.44:0/2191886179 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd8440a0710 0x7fd8440a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:50.709 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.702+0000 7fd84e7fc700 1 -- 172.21.15.44:0/2191886179 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd8480cd3e0 con 0x7fd8581006d0 2023-09-10T00:39:50.709 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.703+0000 7fd85d4cf700 1 --2- 172.21.15.44:0/2191886179 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd8440a0710 0x7fd8440a2bd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd858100530 tx=0x7fd85400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:50.710 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.707+0000 7fd84e7fc700 1 -- 172.21.15.44:0/2191886179 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd848096b30 con 0x7fd8581006d0 2023-09-10T00:39:50.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:50 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2296001381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:39:50.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:50 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3116094949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:39:50.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:50 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1361559818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:39:50.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.956+0000 7fd85f733700 1 -- 172.21.15.44:0/2191886179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8581a0bb0 con 0x7fd8581006d0 2023-09-10T00:39:50.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.957+0000 7fd84e7fc700 1 -- 172.21.15.44:0/2191886179 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11369 (secure 0 0 0) 0x7fd84809a150 con 0x7fd8581006d0 2023-09-10T00:39:50.958 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:50.958 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-10T00:38:39.183533+0000","modified":"2023-09-10T00:39:02.480227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24605,"mds_1":24574,"mds_2":24572,"mds_3":24515,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14751":{"gid":14751,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/3958270510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":3958270510},{"type":"v1","addr":"172.21.15.44:6839","nonce":3958270510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14763":{"gid":14763,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/1241002552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1241002552},{"type":"v1","addr":"172.21.15.44:6841","nonce":1241002552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24556":{"gid":24556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/3222652693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":3222652693},{"type":"v1","addr":"172.21.15.131:6839","nonce":3222652693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24605":{"gid":24605,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/2622966260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2622966260},{"type":"v1","addr":"172.21.15.99:6839","nonce":2622966260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-10T00:39:50.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.959+0000 7fd85f733700 1 -- 172.21.15.44:0/2191886179 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd8440a0710 msgr2=0x7fd8440a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:50.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.959+0000 7fd85f733700 1 --2- 172.21.15.44:0/2191886179 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd8440a0710 0x7fd8440a2bd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd858100530 tx=0x7fd85400a380 comp rx=0 tx=0).stop 2023-09-10T00:39:50.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.959+0000 7fd85f733700 1 -- 172.21.15.44:0/2191886179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8581006d0 msgr2=0x7fd858101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:50.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.959+0000 7fd85f733700 1 --2- 172.21.15.44:0/2191886179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8581006d0 0x7fd858101ab0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd848009f50 tx=0x7fd848005f10 comp rx=0 tx=0).stop 2023-09-10T00:39:50.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.959+0000 7fd85f733700 1 -- 172.21.15.44:0/2191886179 shutdown_connections 2023-09-10T00:39:50.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.959+0000 7fd85f733700 1 --2- 172.21.15.44:0/2191886179 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd8440a0710 0x7fd8440a2bd0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.959+0000 7fd85f733700 1 --2- 172.21.15.44:0/2191886179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd858101070 0x7fd858101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.959+0000 7fd85f733700 1 --2- 172.21.15.44:0/2191886179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8580ff4e0 0x7fd858103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.959+0000 7fd85f733700 1 --2- 172.21.15.44:0/2191886179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8581006d0 0x7fd858101ab0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:50.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.959+0000 7fd85f733700 1 -- 172.21.15.44:0/2191886179 >> 172.21.15.44:0/2191886179 conn(0x7fd8580faa30 msgr2=0x7fd858067760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:50.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.959+0000 7fd85f733700 1 -- 172.21.15.44:0/2191886179 shutdown_connections 2023-09-10T00:39:50.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:50.959+0000 7fd85f733700 1 -- 172.21.15.44:0/2191886179 wait complete. 2023-09-10T00:39:50.964 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2023-09-10T00:39:50.972 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:50 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2296001381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:39:50.972 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:50 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3116094949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:39:50.972 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:50 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1361559818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:39:50.974 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:39:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:50 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2296001381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:39:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:50 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3116094949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:39:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:50 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1361559818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:39:51.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.098+0000 7f76bca7b700 1 -- 172.21.15.44:0/732010537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b8101070 msgr2=0x7f76b8107920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:51.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.098+0000 7f76bca7b700 1 --2- 172.21.15.44:0/732010537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b8101070 0x7f76b8107920 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f76b000b0d0 tx=0x7f76b000b3e0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.098+0000 7f76bca7b700 1 -- 172.21.15.44:0/732010537 shutdown_connections 2023-09-10T00:39:51.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.098+0000 7f76bca7b700 1 --2- 172.21.15.44:0/732010537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b8101070 0x7f76b8107920 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.098+0000 7f76bca7b700 1 --2- 172.21.15.44:0/732010537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b80ff4e0 0x7f76b80ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.098+0000 7f76bca7b700 1 --2- 172.21.15.44:0/732010537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76b81006d0 0x7f76b8100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.098+0000 7f76bca7b700 1 -- 172.21.15.44:0/732010537 >> 172.21.15.44:0/732010537 conn(0x7f76b80faa30 msgr2=0x7f76b80fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:51.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.098+0000 7f76bca7b700 1 -- 172.21.15.44:0/732010537 shutdown_connections 2023-09-10T00:39:51.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.098+0000 7f76bca7b700 1 -- 172.21.15.44:0/732010537 wait complete. 2023-09-10T00:39:51.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.099+0000 7f76bca7b700 1 Processor -- start 2023-09-10T00:39:51.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.099+0000 7f76bca7b700 1 -- start start 2023-09-10T00:39:51.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.099+0000 7f76bca7b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b80ff4e0 0x7f76b8103470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:51.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.099+0000 7f76bca7b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b81006d0 0x7f76b8101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:51.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.099+0000 7f76bca7b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76b8101070 0x7f76b8101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:51.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.099+0000 7f76bca7b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b8102790 con 0x7f76b80ff4e0 2023-09-10T00:39:51.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.099+0000 7f76bca7b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b8102900 con 0x7f76b81006d0 2023-09-10T00:39:51.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.099+0000 7f76bca7b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b81a0260 con 0x7f76b8101070 2023-09-10T00:39:51.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.099+0000 7f76b659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b80ff4e0 0x7f76b8103470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:51.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.099+0000 7f76b659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b80ff4e0 0x7f76b8103470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47420/0 (socket says 172.21.15.44:47420) 2023-09-10T00:39:51.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.099+0000 7f76b659c700 1 -- 172.21.15.44:0/2895636345 learned_addr learned my addr 172.21.15.44:0/2895636345 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:51.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.099+0000 7f76b5d9b700 1 --2- 172.21.15.44:0/2895636345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b81006d0 0x7f76b8101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:51.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.099+0000 7f76b6d9d700 1 --2- 172.21.15.44:0/2895636345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76b8101070 0x7f76b8101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:51.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.100+0000 7f76b659c700 1 -- 172.21.15.44:0/2895636345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76b8101070 msgr2=0x7f76b8101ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:51.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.100+0000 7f76b659c700 1 --2- 172.21.15.44:0/2895636345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76b8101070 0x7f76b8101ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.100+0000 7f76b659c700 1 -- 172.21.15.44:0/2895636345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b81006d0 msgr2=0x7f76b8101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:51.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.100+0000 7f76b659c700 1 --2- 172.21.15.44:0/2895636345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b81006d0 0x7f76b8101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.100+0000 7f76b659c700 1 -- 172.21.15.44:0/2895636345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b0009d00 con 0x7f76b80ff4e0 2023-09-10T00:39:51.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.100+0000 7f76b5d9b700 1 --2- 172.21.15.44:0/2895636345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b81006d0 0x7f76b8101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:51.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.100+0000 7f76b6d9d700 1 --2- 172.21.15.44:0/2895636345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76b8101070 0x7f76b8101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:51.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.100+0000 7f76b659c700 1 --2- 172.21.15.44:0/2895636345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b80ff4e0 0x7f76b8103470 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f76a800c990 tx=0x7f76a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:51.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.100+0000 7f76af7fe700 1 -- 172.21.15.44:0/2895636345 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8007b00 con 0x7f76b80ff4e0 2023-09-10T00:39:51.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.100+0000 7f76bca7b700 1 -- 172.21.15.44:0/2895636345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b81a0540 con 0x7f76b80ff4e0 2023-09-10T00:39:51.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.100+0000 7f76af7fe700 1 -- 172.21.15.44:0/2895636345 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76a8007c60 con 0x7f76b80ff4e0 2023-09-10T00:39:51.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.100+0000 7f76af7fe700 1 -- 172.21.15.44:0/2895636345 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8018660 con 0x7f76b80ff4e0 2023-09-10T00:39:51.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.100+0000 7f76bca7b700 1 -- 172.21.15.44:0/2895636345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b81a0aa0 con 0x7f76b80ff4e0 2023-09-10T00:39:51.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.101+0000 7f76bca7b700 1 -- 172.21.15.44:0/2895636345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b810aba0 con 0x7f76b80ff4e0 2023-09-10T00:39:51.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.102+0000 7f76af7fe700 1 -- 172.21.15.44:0/2895636345 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f76a800f450 con 0x7f76b80ff4e0 2023-09-10T00:39:51.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.103+0000 7f76af7fe700 1 --2- 172.21.15.44:0/2895636345 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f76a40a0710 0x7f76a40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:51.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.103+0000 7f76b5d9b700 1 --2- 172.21.15.44:0/2895636345 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f76a40a0710 0x7f76a40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:51.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.103+0000 7f76af7fe700 1 -- 172.21.15.44:0/2895636345 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f76a80cc050 con 0x7f76b80ff4e0 2023-09-10T00:39:51.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.103+0000 7f76b5d9b700 1 --2- 172.21.15.44:0/2895636345 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f76a40a0710 0x7f76a40a2bd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f76a0005950 tx=0x7f76a00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:51.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.107+0000 7f76af7fe700 1 -- 172.21.15.44:0/2895636345 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f76a8095710 con 0x7f76b80ff4e0 2023-09-10T00:39:51.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.312+0000 7f76bca7b700 1 -- 172.21.15.44:0/2895636345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76b810ad60 con 0x7f76b80ff4e0 2023-09-10T00:39:51.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.314+0000 7f76af7fe700 1 -- 172.21.15.44:0/2895636345 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24340 (secure 0 0 0) 0x7f76a8098d30 con 0x7f76b80ff4e0 2023-09-10T00:39:51.315 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:51.316 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":89,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:39:17.515738+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:39:51.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.316+0000 7f76bca7b700 1 -- 172.21.15.44:0/2895636345 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f76a40a0710 msgr2=0x7f76a40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.316+0000 7f76bca7b700 1 --2- 172.21.15.44:0/2895636345 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f76a40a0710 0x7f76a40a2bd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f76a0005950 tx=0x7f76a00058e0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.316+0000 7f76bca7b700 1 -- 172.21.15.44:0/2895636345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b80ff4e0 msgr2=0x7f76b8103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.316+0000 7f76bca7b700 1 --2- 172.21.15.44:0/2895636345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b80ff4e0 0x7f76b8103470 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f76a800c990 tx=0x7f76a800cd50 comp rx=0 tx=0).stop 2023-09-10T00:39:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.316+0000 7f76bca7b700 1 -- 172.21.15.44:0/2895636345 shutdown_connections 2023-09-10T00:39:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.316+0000 7f76bca7b700 1 --2- 172.21.15.44:0/2895636345 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f76a40a0710 0x7f76a40a2bd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.316+0000 7f76bca7b700 1 --2- 172.21.15.44:0/2895636345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76b8101070 0x7f76b8101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.316+0000 7f76bca7b700 1 --2- 172.21.15.44:0/2895636345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b80ff4e0 0x7f76b8103470 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.316+0000 7f76bca7b700 1 --2- 172.21.15.44:0/2895636345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b81006d0 0x7f76b8101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.316+0000 7f76bca7b700 1 -- 172.21.15.44:0/2895636345 >> 172.21.15.44:0/2895636345 conn(0x7f76b80faa30 msgr2=0x7f76b8067800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.316+0000 7f76bca7b700 1 -- 172.21.15.44:0/2895636345 shutdown_connections 2023-09-10T00:39:51.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.316+0000 7f76bca7b700 1 -- 172.21.15.44:0/2895636345 wait complete. 2023-09-10T00:39:51.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-10T00:34:44.456741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-10T00:38:37.194891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-10T00:38:38.573268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-10T00:38:42.198896+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-10T00:39:51.331 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-10T00:39:51.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.458+0000 7f0066d93700 1 -- 172.21.15.44:0/3434915053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00600ff4f0 msgr2=0x7f00600ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:51.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.458+0000 7f0066d93700 1 --2- 172.21.15.44:0/3434915053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00600ff4f0 0x7f00600ff910 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f00540099c0 tx=0x7f0054009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.458+0000 7f0066d93700 1 -- 172.21.15.44:0/3434915053 shutdown_connections 2023-09-10T00:39:51.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.458+0000 7f0066d93700 1 --2- 172.21.15.44:0/3434915053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0060101080 0x7f0060107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.458+0000 7f0066d93700 1 --2- 172.21.15.44:0/3434915053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00600ff4f0 0x7f00600ff910 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.458+0000 7f0066d93700 1 --2- 172.21.15.44:0/3434915053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00601006e0 0x7f0060100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.458+0000 7f0066d93700 1 -- 172.21.15.44:0/3434915053 >> 172.21.15.44:0/3434915053 conn(0x7f00600faa60 msgr2=0x7f00600fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.458+0000 7f0066d93700 1 -- 172.21.15.44:0/3434915053 shutdown_connections 2023-09-10T00:39:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.458+0000 7f0066d93700 1 -- 172.21.15.44:0/3434915053 wait complete. 2023-09-10T00:39:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.459+0000 7f0066d93700 1 Processor -- start 2023-09-10T00:39:51.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.459+0000 7f0066d93700 1 -- start start 2023-09-10T00:39:51.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.459+0000 7f0066d93700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00600ff4f0 0x7f006010a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:51.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.459+0000 7f0066d93700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00601006e0 0x7f0060108690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:51.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.459+0000 7f0066d93700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0060101080 0x7f0060108bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:51.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.459+0000 7f0066d93700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0060109370 con 0x7f0060101080 2023-09-10T00:39:51.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.459+0000 7f0066d93700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00601094e0 con 0x7f00600ff4f0 2023-09-10T00:39:51.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.459+0000 7f0066d93700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00601a02a0 con 0x7f00601006e0 2023-09-10T00:39:51.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.459+0000 7f0065330700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0060101080 0x7f0060108bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:51.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.459+0000 7f005ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00601006e0 0x7f0060108690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:51.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.459+0000 7f0065330700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0060101080 0x7f0060108bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47436/0 (socket says 172.21.15.44:47436) 2023-09-10T00:39:51.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.459+0000 7f0065330700 1 -- 172.21.15.44:0/3819474508 learned_addr learned my addr 172.21.15.44:0/3819474508 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:51.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.460+0000 7f0064b2f700 1 --2- 172.21.15.44:0/3819474508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00600ff4f0 0x7f006010a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:51.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.460+0000 7f0065330700 1 -- 172.21.15.44:0/3819474508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00601006e0 msgr2=0x7f0060108690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:51.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.460+0000 7f0065330700 1 --2- 172.21.15.44:0/3819474508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00601006e0 0x7f0060108690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.460+0000 7f0065330700 1 -- 172.21.15.44:0/3819474508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00600ff4f0 msgr2=0x7f006010a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:51.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.460+0000 7f0065330700 1 --2- 172.21.15.44:0/3819474508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00600ff4f0 0x7f006010a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.460+0000 7f0065330700 1 -- 172.21.15.44:0/3819474508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00540095f0 con 0x7f0060101080 2023-09-10T00:39:51.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.460+0000 7f0064b2f700 1 --2- 172.21.15.44:0/3819474508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00600ff4f0 0x7f006010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:51.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.460+0000 7f0065330700 1 --2- 172.21.15.44:0/3819474508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0060101080 0x7f0060108bd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0058007ee0 tx=0x7f005800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:51.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.460+0000 7f005dffb700 1 -- 172.21.15.44:0/3819474508 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0058017070 con 0x7f0060101080 2023-09-10T00:39:51.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.460+0000 7f005dffb700 1 -- 172.21.15.44:0/3819474508 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f005800f040 con 0x7f0060101080 2023-09-10T00:39:51.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.460+0000 7f0066d93700 1 -- 172.21.15.44:0/3819474508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00601a0580 con 0x7f0060101080 2023-09-10T00:39:51.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.460+0000 7f005dffb700 1 -- 172.21.15.44:0/3819474508 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0058005710 con 0x7f0060101080 2023-09-10T00:39:51.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.460+0000 7f0066d93700 1 -- 172.21.15.44:0/3819474508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00601a0a40 con 0x7f0060101080 2023-09-10T00:39:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.461+0000 7f0066d93700 1 -- 172.21.15.44:0/3819474508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0060106bb0 con 0x7f0060101080 2023-09-10T00:39:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.462+0000 7f005dffb700 1 -- 172.21.15.44:0/3819474508 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f0058005870 con 0x7f0060101080 2023-09-10T00:39:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.463+0000 7f005dffb700 1 --2- 172.21.15.44:0/3819474508 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f00500a06d0 0x7f00500a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.463+0000 7f005dffb700 1 -- 172.21.15.44:0/3819474508 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f00580cb860 con 0x7f0060101080 2023-09-10T00:39:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.463+0000 7f0064b2f700 1 --2- 172.21.15.44:0/3819474508 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f00500a06d0 0x7f00500a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.464+0000 7f0064b2f700 1 --2- 172.21.15.44:0/3819474508 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f00500a06d0 0x7f00500a2b90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0054009990 tx=0x7f005401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.467+0000 7f005dffb700 1 -- 172.21.15.44:0/3819474508 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0058094f20 con 0x7f0060101080 2023-09-10T00:39:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.670+0000 7f0066d93700 1 -- 172.21.15.44:0/3819474508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f006002ce70 con 0x7f0060101080 2023-09-10T00:39:51.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.670+0000 7f005dffb700 1 -- 172.21.15.44:0/3819474508 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0058098540 con 0x7f0060101080 2023-09-10T00:39:51.672 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2023-09-10T00:39:51.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.672+0000 7f0066d93700 1 -- 172.21.15.44:0/3819474508 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f00500a06d0 msgr2=0x7f00500a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:51.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.672+0000 7f0066d93700 1 --2- 172.21.15.44:0/3819474508 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f00500a06d0 0x7f00500a2b90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0054009990 tx=0x7f005401a040 comp rx=0 tx=0).stop 2023-09-10T00:39:51.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.672+0000 7f0066d93700 1 -- 172.21.15.44:0/3819474508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0060101080 msgr2=0x7f0060108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:51.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.672+0000 7f0066d93700 1 --2- 172.21.15.44:0/3819474508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0060101080 0x7f0060108bd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0058007ee0 tx=0x7f005800d3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.672+0000 7f0066d93700 1 -- 172.21.15.44:0/3819474508 shutdown_connections 2023-09-10T00:39:51.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.672+0000 7f0066d93700 1 --2- 172.21.15.44:0/3819474508 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f00500a06d0 0x7f00500a2b90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.672+0000 7f0066d93700 1 --2- 172.21.15.44:0/3819474508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0060101080 0x7f0060108bd0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.672+0000 7f0066d93700 1 --2- 172.21.15.44:0/3819474508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00600ff4f0 0x7f006010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.672+0000 7f0066d93700 1 --2- 172.21.15.44:0/3819474508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00601006e0 0x7f0060108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.672+0000 7f0066d93700 1 -- 172.21.15.44:0/3819474508 >> 172.21.15.44:0/3819474508 conn(0x7f00600faa60 msgr2=0x7f0060103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:51.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.672+0000 7f0066d93700 1 -- 172.21.15.44:0/3819474508 shutdown_connections 2023-09-10T00:39:51.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.672+0000 7f0066d93700 1 -- 172.21.15.44:0/3819474508 wait complete. 2023-09-10T00:39:51.686 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-10T00:39:51.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.816+0000 7f9c7e548700 1 -- 172.21.15.44:0/3815586326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c781062d0 msgr2=0x7f9c78108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.816+0000 7f9c7e548700 1 --2- 172.21.15.44:0/3815586326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c781062d0 0x7f9c78108740 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000b0a0 tx=0x7f9c7000b3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.816+0000 7f9c7e548700 1 -- 172.21.15.44:0/3815586326 shutdown_connections 2023-09-10T00:39:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.816+0000 7f9c7e548700 1 --2- 172.21.15.44:0/3815586326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c781062d0 0x7f9c78108740 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.816+0000 7f9c7e548700 1 --2- 172.21.15.44:0/3815586326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c78101070 0x7f9c78103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.816+0000 7f9c7e548700 1 --2- 172.21.15.44:0/3815586326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c781039a0 0x7f9c78105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.816+0000 7f9c7e548700 1 -- 172.21.15.44:0/3815586326 >> 172.21.15.44:0/3815586326 conn(0x7f9c780faa60 msgr2=0x7f9c780fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.816+0000 7f9c7e548700 1 -- 172.21.15.44:0/3815586326 shutdown_connections 2023-09-10T00:39:51.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.816+0000 7f9c7e548700 1 -- 172.21.15.44:0/3815586326 wait complete. 2023-09-10T00:39:51.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.817+0000 7f9c7e548700 1 Processor -- start 2023-09-10T00:39:51.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.817+0000 7f9c7e548700 1 -- start start 2023-09-10T00:39:51.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.817+0000 7f9c7e548700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c78101070 0x7f9c78196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:51.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.817+0000 7f9c7e548700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c781039a0 0x7f9c78196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:51.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.817+0000 7f9c7e548700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c781062d0 0x7f9c7819d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:51.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.817+0000 7f9c7e548700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7819dea0 con 0x7f9c781062d0 2023-09-10T00:39:51.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.817+0000 7f9c7e548700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7819dfe0 con 0x7f9c78101070 2023-09-10T00:39:51.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.817+0000 7f9c7e548700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7819e170 con 0x7f9c781039a0 2023-09-10T00:39:51.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.817+0000 7f9c7cae5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c781062d0 0x7f9c7819d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:51.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.817+0000 7f9c7cae5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c781062d0 0x7f9c7819d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47450/0 (socket says 172.21.15.44:47450) 2023-09-10T00:39:51.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.817+0000 7f9c7cae5700 1 -- 172.21.15.44:0/3033034600 learned_addr learned my addr 172.21.15.44:0/3033034600 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:51.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.817+0000 7f9c77fff700 1 --2- 172.21.15.44:0/3033034600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c78101070 0x7f9c78196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:51.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.817+0000 7f9c777fe700 1 --2- 172.21.15.44:0/3033034600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c781039a0 0x7f9c78196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:51.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.818+0000 7f9c7cae5700 1 -- 172.21.15.44:0/3033034600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c781039a0 msgr2=0x7f9c78196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:51.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.818+0000 7f9c7cae5700 1 --2- 172.21.15.44:0/3033034600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c781039a0 0x7f9c78196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.818+0000 7f9c7cae5700 1 -- 172.21.15.44:0/3033034600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c78101070 msgr2=0x7f9c78196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:51.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.818+0000 7f9c7cae5700 1 --2- 172.21.15.44:0/3033034600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c78101070 0x7f9c78196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.818+0000 7f9c7cae5700 1 -- 172.21.15.44:0/3033034600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c70009d00 con 0x7f9c781062d0 2023-09-10T00:39:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.818+0000 7f9c7cae5700 1 --2- 172.21.15.44:0/3033034600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c781062d0 0x7f9c7819d8d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000b780 tx=0x7f9c700078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.818+0000 7f9c757fa700 1 -- 172.21.15.44:0/3033034600 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7000e040 con 0x7f9c781062d0 2023-09-10T00:39:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.818+0000 7f9c757fa700 1 -- 172.21.15.44:0/3033034600 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c7000be80 con 0x7f9c781062d0 2023-09-10T00:39:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.818+0000 7f9c7e548700 1 -- 172.21.15.44:0/3033034600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7819e3f0 con 0x7f9c781062d0 2023-09-10T00:39:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.818+0000 7f9c757fa700 1 -- 172.21.15.44:0/3033034600 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7001dea0 con 0x7f9c781062d0 2023-09-10T00:39:51.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.818+0000 7f9c7e548700 1 -- 172.21.15.44:0/3033034600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7819e820 con 0x7f9c781062d0 2023-09-10T00:39:51.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.819+0000 7f9c7e548700 1 -- 172.21.15.44:0/3033034600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c780fc5e0 con 0x7f9c781062d0 2023-09-10T00:39:51.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.820+0000 7f9c757fa700 1 -- 172.21.15.44:0/3033034600 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f9c7001d470 con 0x7f9c781062d0 2023-09-10T00:39:51.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.821+0000 7f9c757fa700 1 --2- 172.21.15.44:0/3033034600 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9c640a06d0 0x7f9c640a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:51.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.821+0000 7f9c77fff700 1 --2- 172.21.15.44:0/3033034600 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9c640a06d0 0x7f9c640a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:51.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.821+0000 7f9c757fa700 1 -- 172.21.15.44:0/3033034600 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9c700cda40 con 0x7f9c781062d0 2023-09-10T00:39:51.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.822+0000 7f9c77fff700 1 --2- 172.21.15.44:0/3033034600 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9c640a06d0 0x7f9c640a2b90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009c60 tx=0x7f9c60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:51.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:51.825+0000 7f9c757fa700 1 -- 172.21.15.44:0/3033034600 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9c70097190 con 0x7f9c781062d0 2023-09-10T00:39:52.027 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.026+0000 7f9c7e548700 1 -- 172.21.15.44:0/3033034600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c7810bde0 con 0x7f9c781062d0 2023-09-10T00:39:52.027 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.026+0000 7f9c757fa700 1 -- 172.21.15.44:0/3033034600 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9c7810bde0 con 0x7f9c781062d0 2023-09-10T00:39:52.028 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:39:52.030 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.028+0000 7f9c7e548700 1 -- 172.21.15.44:0/3033034600 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9c640a06d0 msgr2=0x7f9c640a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:52.030 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.028+0000 7f9c7e548700 1 --2- 172.21.15.44:0/3033034600 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9c640a06d0 0x7f9c640a2b90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009c60 tx=0x7f9c60009260 comp rx=0 tx=0).stop 2023-09-10T00:39:52.030 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.028+0000 7f9c7e548700 1 -- 172.21.15.44:0/3033034600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c781062d0 msgr2=0x7f9c7819d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:52.030 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.028+0000 7f9c7e548700 1 --2- 172.21.15.44:0/3033034600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c781062d0 0x7f9c7819d8d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000b780 tx=0x7f9c700078e0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.028+0000 7f9c7e548700 1 -- 172.21.15.44:0/3033034600 shutdown_connections 2023-09-10T00:39:52.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.028+0000 7f9c7e548700 1 --2- 172.21.15.44:0/3033034600 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9c640a06d0 0x7f9c640a2b90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.028+0000 7f9c7e548700 1 --2- 172.21.15.44:0/3033034600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c781062d0 0x7f9c7819d8d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.029+0000 7f9c7e548700 1 --2- 172.21.15.44:0/3033034600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c78101070 0x7f9c78196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.029+0000 7f9c7e548700 1 --2- 172.21.15.44:0/3033034600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c781039a0 0x7f9c78196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.029+0000 7f9c7e548700 1 -- 172.21.15.44:0/3033034600 >> 172.21.15.44:0/3033034600 conn(0x7f9c780faa60 msgr2=0x7f9c78101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:52.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.029+0000 7f9c7e548700 1 -- 172.21.15.44:0/3033034600 shutdown_connections 2023-09-10T00:39:52.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.029+0000 7f9c7e548700 1 -- 172.21.15.44:0/3033034600 wait complete. 2023-09-10T00:39:52.041 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:51 smithi044 ceph-mon[98676]: pgmap v306: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-10T00:39:52.041 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:51 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2191886179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:39:52.042 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:51 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2895636345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:52.043 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-10T00:39:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:51 smithi131 ceph-mon[103713]: pgmap v306: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-10T00:39:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:51 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2191886179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:39:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:51 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2895636345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:52.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:51 smithi099 ceph-mon[102692]: pgmap v306: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-10T00:39:52.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:51 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2191886179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:39:52.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:51 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2895636345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:52.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.171+0000 7f5d630da700 1 -- 172.21.15.44:0/3037478294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c0ff4f0 msgr2=0x7f5d5c0ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:52.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.171+0000 7f5d630da700 1 --2- 172.21.15.44:0/3037478294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c0ff4f0 0x7f5d5c0ff910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5d500099c0 tx=0x7f5d50009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.171+0000 7f5d630da700 1 -- 172.21.15.44:0/3037478294 shutdown_connections 2023-09-10T00:39:52.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.171+0000 7f5d630da700 1 --2- 172.21.15.44:0/3037478294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d5c101080 0x7f5d5c107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.171+0000 7f5d630da700 1 --2- 172.21.15.44:0/3037478294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c0ff4f0 0x7f5d5c0ff910 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.171+0000 7f5d630da700 1 --2- 172.21.15.44:0/3037478294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d5c1006e0 0x7f5d5c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.171+0000 7f5d630da700 1 -- 172.21.15.44:0/3037478294 >> 172.21.15.44:0/3037478294 conn(0x7f5d5c0faa60 msgr2=0x7f5d5c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:52.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.171+0000 7f5d630da700 1 -- 172.21.15.44:0/3037478294 shutdown_connections 2023-09-10T00:39:52.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.171+0000 7f5d630da700 1 -- 172.21.15.44:0/3037478294 wait complete. 2023-09-10T00:39:52.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.171+0000 7f5d630da700 1 Processor -- start 2023-09-10T00:39:52.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.171+0000 7f5d630da700 1 -- start start 2023-09-10T00:39:52.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d630da700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d5c0ff4f0 0x7f5d5c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:52.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d630da700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c1006e0 0x7f5d5c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:52.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d630da700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d5c101080 0x7f5d5c19fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:52.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d630da700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c1a0000 con 0x7f5d5c1006e0 2023-09-10T00:39:52.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d630da700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c1a0140 con 0x7f5d5c0ff4f0 2023-09-10T00:39:52.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d630da700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c1a02d0 con 0x7f5d5c101080 2023-09-10T00:39:52.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d5bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c1006e0 0x7f5d5c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:52.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d5bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c1006e0 0x7f5d5c1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47462/0 (socket says 172.21.15.44:47462) 2023-09-10T00:39:52.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d5bfff700 1 -- 172.21.15.44:0/3049030257 learned_addr learned my addr 172.21.15.44:0/3049030257 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:52.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d61677700 1 --2- 172.21.15.44:0/3049030257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d5c101080 0x7f5d5c19fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:52.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d60e76700 1 --2- 172.21.15.44:0/3049030257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d5c0ff4f0 0x7f5d5c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:52.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d5bfff700 1 -- 172.21.15.44:0/3049030257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d5c101080 msgr2=0x7f5d5c19fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:52.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d5bfff700 1 --2- 172.21.15.44:0/3049030257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d5c101080 0x7f5d5c19fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d5bfff700 1 -- 172.21.15.44:0/3049030257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d5c0ff4f0 msgr2=0x7f5d5c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:52.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d5bfff700 1 --2- 172.21.15.44:0/3049030257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d5c0ff4f0 0x7f5d5c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d5bfff700 1 -- 172.21.15.44:0/3049030257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d500095f0 con 0x7f5d5c1006e0 2023-09-10T00:39:52.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d5bfff700 1 --2- 172.21.15.44:0/3049030257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c1006e0 0x7f5d5c1989c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800c920 tx=0x7f5d4800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:52.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.172+0000 7f5d59ffb700 1 -- 172.21.15.44:0/3049030257 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48004a30 con 0x7f5d5c1006e0 2023-09-10T00:39:52.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.173+0000 7f5d59ffb700 1 -- 172.21.15.44:0/3049030257 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d480054b0 con 0x7f5d5c1006e0 2023-09-10T00:39:52.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.173+0000 7f5d630da700 1 -- 172.21.15.44:0/3049030257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c1a05b0 con 0x7f5d5c1006e0 2023-09-10T00:39:52.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.173+0000 7f5d59ffb700 1 -- 172.21.15.44:0/3049030257 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48007b50 con 0x7f5d5c1006e0 2023-09-10T00:39:52.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.173+0000 7f5d630da700 1 -- 172.21.15.44:0/3049030257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c1a0ac0 con 0x7f5d5c1006e0 2023-09-10T00:39:52.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.174+0000 7f5d630da700 1 -- 172.21.15.44:0/3049030257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5c10ac10 con 0x7f5d5c1006e0 2023-09-10T00:39:52.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.175+0000 7f5d59ffb700 1 -- 172.21.15.44:0/3049030257 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f5d48007cb0 con 0x7f5d5c1006e0 2023-09-10T00:39:52.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.175+0000 7f5d59ffb700 1 --2- 172.21.15.44:0/3049030257 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5d4c0a06d0 0x7f5d4c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:52.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.175+0000 7f5d59ffb700 1 -- 172.21.15.44:0/3049030257 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5d480cbb60 con 0x7f5d5c1006e0 2023-09-10T00:39:52.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.175+0000 7f5d60e76700 1 --2- 172.21.15.44:0/3049030257 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5d4c0a06d0 0x7f5d4c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:52.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.176+0000 7f5d60e76700 1 --2- 172.21.15.44:0/3049030257 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5d4c0a06d0 0x7f5d4c0a2b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5d50006010 tx=0x7f5d50005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:52.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.179+0000 7f5d59ffb700 1 -- 172.21.15.44:0/3049030257 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5d48095220 con 0x7f5d5c1006e0 2023-09-10T00:39:52.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.383+0000 7f5d630da700 1 -- 172.21.15.44:0/3049030257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5d5c1a0d40 con 0x7f5d5c1006e0 2023-09-10T00:39:52.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.384+0000 7f5d59ffb700 1 -- 172.21.15.44:0/3049030257 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5d4801d0a0 con 0x7f5d5c1006e0 2023-09-10T00:39:52.385 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2023-09-10T00:39:52.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.386+0000 7f5d630da700 1 -- 172.21.15.44:0/3049030257 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5d4c0a06d0 msgr2=0x7f5d4c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:52.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.386+0000 7f5d630da700 1 --2- 172.21.15.44:0/3049030257 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5d4c0a06d0 0x7f5d4c0a2b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5d50006010 tx=0x7f5d50005e80 comp rx=0 tx=0).stop 2023-09-10T00:39:52.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.386+0000 7f5d630da700 1 -- 172.21.15.44:0/3049030257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c1006e0 msgr2=0x7f5d5c1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:52.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.386+0000 7f5d630da700 1 --2- 172.21.15.44:0/3049030257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c1006e0 0x7f5d5c1989c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800c920 tx=0x7f5d4800cc30 comp rx=0 tx=0).stop 2023-09-10T00:39:52.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.386+0000 7f5d630da700 1 -- 172.21.15.44:0/3049030257 shutdown_connections 2023-09-10T00:39:52.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.386+0000 7f5d630da700 1 --2- 172.21.15.44:0/3049030257 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5d4c0a06d0 0x7f5d4c0a2b90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.386+0000 7f5d630da700 1 --2- 172.21.15.44:0/3049030257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d5c101080 0x7f5d5c19fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.386+0000 7f5d630da700 1 --2- 172.21.15.44:0/3049030257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d5c0ff4f0 0x7f5d5c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.386+0000 7f5d630da700 1 --2- 172.21.15.44:0/3049030257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c1006e0 0x7f5d5c1989c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.386+0000 7f5d630da700 1 -- 172.21.15.44:0/3049030257 >> 172.21.15.44:0/3049030257 conn(0x7f5d5c0faa60 msgr2=0x7f5d5c067820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:52.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.386+0000 7f5d630da700 1 -- 172.21.15.44:0/3049030257 shutdown_connections 2023-09-10T00:39:52.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.386+0000 7f5d630da700 1 -- 172.21.15.44:0/3049030257 wait complete. 2023-09-10T00:39:52.400 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-10T00:39:52.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.527+0000 7fd84027b700 1 -- 172.21.15.44:0/1475110188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd838100200 msgr2=0x7fd838100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:52.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.527+0000 7fd84027b700 1 --2- 172.21.15.44:0/1475110188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd838100200 0x7fd838100680 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd83000b0a0 tx=0x7fd83000b3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.527+0000 7fd84027b700 1 -- 172.21.15.44:0/1475110188 shutdown_connections 2023-09-10T00:39:52.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.527+0000 7fd84027b700 1 --2- 172.21.15.44:0/1475110188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd838100200 0x7fd838100680 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.527+0000 7fd84027b700 1 --2- 172.21.15.44:0/1475110188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380feee0 0x7fd8380ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.527+0000 7fd84027b700 1 --2- 172.21.15.44:0/1475110188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8380ff840 0x7fd8380ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.527+0000 7fd84027b700 1 -- 172.21.15.44:0/1475110188 >> 172.21.15.44:0/1475110188 conn(0x7fd8380faa60 msgr2=0x7fd8380fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:52.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.527+0000 7fd84027b700 1 -- 172.21.15.44:0/1475110188 shutdown_connections 2023-09-10T00:39:52.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.527+0000 7fd84027b700 1 -- 172.21.15.44:0/1475110188 wait complete. 2023-09-10T00:39:52.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd84027b700 1 Processor -- start 2023-09-10T00:39:52.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd84027b700 1 -- start start 2023-09-10T00:39:52.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd84027b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380feee0 0x7fd838198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:52.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd84027b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8380ff840 0x7fd838198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:52.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd84027b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd838100200 0x7fd83819faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:52.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd84027b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8381a00c0 con 0x7fd838100200 2023-09-10T00:39:52.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd84027b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8381a0200 con 0x7fd8380ff840 2023-09-10T00:39:52.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd84027b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8381a0390 con 0x7fd8380feee0 2023-09-10T00:39:52.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd83e818700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd838100200 0x7fd83819faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:52.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd83e818700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd838100200 0x7fd83819faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47478/0 (socket says 172.21.15.44:47478) 2023-09-10T00:39:52.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd83e818700 1 -- 172.21.15.44:0/737197013 learned_addr learned my addr 172.21.15.44:0/737197013 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:52.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd83d816700 1 --2- 172.21.15.44:0/737197013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8380ff840 0x7fd838198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:52.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd83e017700 1 --2- 172.21.15.44:0/737197013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380feee0 0x7fd838198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:52.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd83e818700 1 -- 172.21.15.44:0/737197013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380feee0 msgr2=0x7fd838198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd83e818700 1 --2- 172.21.15.44:0/737197013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380feee0 0x7fd838198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd83e818700 1 -- 172.21.15.44:0/737197013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8380ff840 msgr2=0x7fd838198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.528+0000 7fd83e818700 1 --2- 172.21.15.44:0/737197013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8380ff840 0x7fd838198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.529+0000 7fd83e818700 1 -- 172.21.15.44:0/737197013 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd830009d00 con 0x7fd838100200 2023-09-10T00:39:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.529+0000 7fd83d816700 1 --2- 172.21.15.44:0/737197013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8380ff840 0x7fd838198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.529+0000 7fd83e017700 1 --2- 172.21.15.44:0/737197013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380feee0 0x7fd838198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:39:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.529+0000 7fd83e818700 1 --2- 172.21.15.44:0/737197013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd838100200 0x7fd83819faf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd83000b070 tx=0x7fd830004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.529+0000 7fd82b7fe700 1 -- 172.21.15.44:0/737197013 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83000e040 con 0x7fd838100200 2023-09-10T00:39:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.529+0000 7fd82b7fe700 1 -- 172.21.15.44:0/737197013 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd830003e90 con 0x7fd838100200 2023-09-10T00:39:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.529+0000 7fd84027b700 1 -- 172.21.15.44:0/737197013 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8381a0610 con 0x7fd838100200 2023-09-10T00:39:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.529+0000 7fd82b7fe700 1 -- 172.21.15.44:0/737197013 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830004020 con 0x7fd838100200 2023-09-10T00:39:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.529+0000 7fd84027b700 1 -- 172.21.15.44:0/737197013 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8381a0a90 con 0x7fd838100200 2023-09-10T00:39:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.530+0000 7fd84027b700 1 -- 172.21.15.44:0/737197013 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd838109cc0 con 0x7fd838100200 2023-09-10T00:39:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.531+0000 7fd82b7fe700 1 -- 172.21.15.44:0/737197013 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fd83001b660 con 0x7fd838100200 2023-09-10T00:39:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.532+0000 7fd82b7fe700 1 --2- 172.21.15.44:0/737197013 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd8240a06d0 0x7fd8240a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.532+0000 7fd82b7fe700 1 -- 172.21.15.44:0/737197013 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd8300cde20 con 0x7fd838100200 2023-09-10T00:39:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.532+0000 7fd83e017700 1 --2- 172.21.15.44:0/737197013 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd8240a06d0 0x7fd8240a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.532+0000 7fd83e017700 1 --2- 172.21.15.44:0/737197013 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd8240a06d0 0x7fd8240a2b90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd82c005fd0 tx=0x7fd82c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.535+0000 7fd82b7fe700 1 -- 172.21.15.44:0/737197013 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd830097510 con 0x7fd838100200 2023-09-10T00:39:52.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.740+0000 7fd84027b700 1 -- 172.21.15.44:0/737197013 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd83810e070 con 0x7fd838100200 2023-09-10T00:39:52.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.740+0000 7fd82b7fe700 1 -- 172.21.15.44:0/737197013 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd830011390 con 0x7fd838100200 2023-09-10T00:39:52.742 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:39:52.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.742+0000 7fd84027b700 1 -- 172.21.15.44:0/737197013 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd8240a06d0 msgr2=0x7fd8240a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:52.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.742+0000 7fd84027b700 1 --2- 172.21.15.44:0/737197013 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd8240a06d0 0x7fd8240a2b90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd82c005fd0 tx=0x7fd82c005dc0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.742+0000 7fd84027b700 1 -- 172.21.15.44:0/737197013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd838100200 msgr2=0x7fd83819faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:52.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.742+0000 7fd84027b700 1 --2- 172.21.15.44:0/737197013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd838100200 0x7fd83819faf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd83000b070 tx=0x7fd830004d30 comp rx=0 tx=0).stop 2023-09-10T00:39:52.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.742+0000 7fd84027b700 1 -- 172.21.15.44:0/737197013 shutdown_connections 2023-09-10T00:39:52.746 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.742+0000 7fd84027b700 1 --2- 172.21.15.44:0/737197013 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd8240a06d0 0x7fd8240a2b90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.746 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.742+0000 7fd84027b700 1 --2- 172.21.15.44:0/737197013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd838100200 0x7fd83819faf0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.746 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.742+0000 7fd84027b700 1 --2- 172.21.15.44:0/737197013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380feee0 0x7fd838198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.742+0000 7fd84027b700 1 --2- 172.21.15.44:0/737197013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8380ff840 0x7fd838198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.742+0000 7fd84027b700 1 -- 172.21.15.44:0/737197013 >> 172.21.15.44:0/737197013 conn(0x7fd8380faa60 msgr2=0x7fd8380fc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:52.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.742+0000 7fd84027b700 1 -- 172.21.15.44:0/737197013 shutdown_connections 2023-09-10T00:39:52.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.742+0000 7fd84027b700 1 -- 172.21.15.44:0/737197013 wait complete. 2023-09-10T00:39:52.755 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:52 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3819474508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:39:52.755 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:52 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3033034600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:39:52.755 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:52 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3049030257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:39:52.757 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:39:52.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.882+0000 7f94b9a54700 1 -- 172.21.15.44:0/3928521845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b4106350 msgr2=0x7f94b4108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:52.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.882+0000 7f94b9a54700 1 --2- 172.21.15.44:0/3928521845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b4106350 0x7f94b4108780 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00b0a0 tx=0x7f94ac00b3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.882+0000 7f94b9a54700 1 -- 172.21.15.44:0/3928521845 shutdown_connections 2023-09-10T00:39:52.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.882+0000 7f94b9a54700 1 --2- 172.21.15.44:0/3928521845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b4106350 0x7f94b4108780 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.882+0000 7f94b9a54700 1 --2- 172.21.15.44:0/3928521845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94b41010f0 0x7f94b41034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.882+0000 7f94b9a54700 1 --2- 172.21.15.44:0/3928521845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94b4103a20 0x7f94b4105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.882+0000 7f94b9a54700 1 -- 172.21.15.44:0/3928521845 >> 172.21.15.44:0/3928521845 conn(0x7f94b40faac0 msgr2=0x7f94b40fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:52.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.882+0000 7f94b9a54700 1 -- 172.21.15.44:0/3928521845 shutdown_connections 2023-09-10T00:39:52.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.882+0000 7f94b9a54700 1 -- 172.21.15.44:0/3928521845 wait complete. 2023-09-10T00:39:52.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.883+0000 7f94b9a54700 1 Processor -- start 2023-09-10T00:39:52.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.883+0000 7f94b9a54700 1 -- start start 2023-09-10T00:39:52.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.883+0000 7f94b9a54700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b41010f0 0x7f94b4100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:52.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.883+0000 7f94b9a54700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94b4103a20 0x7f94b40ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:52.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.883+0000 7f94b9a54700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94b4106350 0x7f94b40ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:52.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.883+0000 7f94b9a54700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b40ffd80 con 0x7f94b41010f0 2023-09-10T00:39:52.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.883+0000 7f94b9a54700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b40ffef0 con 0x7f94b4103a20 2023-09-10T00:39:52.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.883+0000 7f94b9a54700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b41a0330 con 0x7f94b4106350 2023-09-10T00:39:52.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b41010f0 0x7f94b4100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:52.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b41010f0 0x7f94b4100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47488/0 (socket says 172.21.15.44:47488) 2023-09-10T00:39:52.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b2ffd700 1 -- 172.21.15.44:0/2378062304 learned_addr learned my addr 172.21.15.44:0/2378062304 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:52.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b27fc700 1 --2- 172.21.15.44:0/2378062304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94b4103a20 0x7f94b40ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:52.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b37fe700 1 --2- 172.21.15.44:0/2378062304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94b4106350 0x7f94b40ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:52.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b2ffd700 1 -- 172.21.15.44:0/2378062304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94b4106350 msgr2=0x7f94b40ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:52.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b2ffd700 1 --2- 172.21.15.44:0/2378062304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94b4106350 0x7f94b40ff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b2ffd700 1 -- 172.21.15.44:0/2378062304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94b4103a20 msgr2=0x7f94b40ff0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:52.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b2ffd700 1 --2- 172.21.15.44:0/2378062304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94b4103a20 0x7f94b40ff0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:52.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b2ffd700 1 -- 172.21.15.44:0/2378062304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac009d00 con 0x7f94b41010f0 2023-09-10T00:39:52.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b37fe700 1 --2- 172.21.15.44:0/2378062304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94b4106350 0x7f94b40ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:39:52.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b2ffd700 1 --2- 172.21.15.44:0/2378062304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b41010f0 0x7f94b4100a10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f94a800e9a0 tx=0x7f94a800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:52.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b8a52700 1 -- 172.21.15.44:0/2378062304 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8009790 con 0x7f94b41010f0 2023-09-10T00:39:52.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b8a52700 1 -- 172.21.15.44:0/2378062304 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f94a8010430 con 0x7f94b41010f0 2023-09-10T00:39:52.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b9a54700 1 -- 172.21.15.44:0/2378062304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b41a05c0 con 0x7f94b41010f0 2023-09-10T00:39:52.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b8a52700 1 -- 172.21.15.44:0/2378062304 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8018680 con 0x7f94b41010f0 2023-09-10T00:39:52.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.884+0000 7f94b9a54700 1 -- 172.21.15.44:0/2378062304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b41a0b20 con 0x7f94b41010f0 2023-09-10T00:39:52.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.885+0000 7f94b9a54700 1 -- 172.21.15.44:0/2378062304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b40fc640 con 0x7f94b41010f0 2023-09-10T00:39:52.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.886+0000 7f94b8a52700 1 -- 172.21.15.44:0/2378062304 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f94a80105a0 con 0x7f94b41010f0 2023-09-10T00:39:52.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.887+0000 7f94b8a52700 1 --2- 172.21.15.44:0/2378062304 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f949c0a0790 0x7f949c0a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:52.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.887+0000 7f94b8a52700 1 -- 172.21.15.44:0/2378062304 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f94a8014070 con 0x7f94b41010f0 2023-09-10T00:39:52.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.887+0000 7f94b27fc700 1 --2- 172.21.15.44:0/2378062304 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f949c0a0790 0x7f949c0a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:52.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.887+0000 7f94b27fc700 1 --2- 172.21.15.44:0/2378062304 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f949c0a0790 0x7f949c0a2c50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f94a4005fd0 tx=0x7f94a4005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:52.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:52.890+0000 7f94b8a52700 1 -- 172.21.15.44:0/2378062304 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f94a8095c30 con 0x7f94b41010f0 2023-09-10T00:39:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:52 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3819474508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:39:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:52 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3033034600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:39:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:52 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3049030257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:39:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:52 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3819474508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:39:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:52 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3033034600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:39:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:52 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3049030257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:39:53.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.138+0000 7f94b9a54700 1 -- 172.21.15.44:0/2378062304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94b410bf80 con 0x7f94b41010f0 2023-09-10T00:39:53.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.139+0000 7f94b8a52700 1 -- 172.21.15.44:0/2378062304 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11369 (secure 0 0 0) 0x7f94a8099250 con 0x7f94b41010f0 2023-09-10T00:39:53.140 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:53.141 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-10T00:38:39.183533+0000","modified":"2023-09-10T00:39:02.480227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24605,"mds_1":24574,"mds_2":24572,"mds_3":24515,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14751":{"gid":14751,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/3958270510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":3958270510},{"type":"v1","addr":"172.21.15.44:6839","nonce":3958270510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14763":{"gid":14763,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/1241002552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1241002552},{"type":"v1","addr":"172.21.15.44:6841","nonce":1241002552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24556":{"gid":24556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/3222652693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":3222652693},{"type":"v1","addr":"172.21.15.131:6839","nonce":3222652693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24605":{"gid":24605,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/2622966260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2622966260},{"type":"v1","addr":"172.21.15.99:6839","nonce":2622966260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-10T00:39:53.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.141+0000 7f94b9a54700 1 -- 172.21.15.44:0/2378062304 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f949c0a0790 msgr2=0x7f949c0a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:53.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.141+0000 7f94b9a54700 1 --2- 172.21.15.44:0/2378062304 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f949c0a0790 0x7f949c0a2c50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f94a4005fd0 tx=0x7f94a4005f60 comp rx=0 tx=0).stop 2023-09-10T00:39:53.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.141+0000 7f94b9a54700 1 -- 172.21.15.44:0/2378062304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b41010f0 msgr2=0x7f94b4100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:53.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.141+0000 7f94b9a54700 1 --2- 172.21.15.44:0/2378062304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b41010f0 0x7f94b4100a10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f94a800e9a0 tx=0x7f94a800ed60 comp rx=0 tx=0).stop 2023-09-10T00:39:53.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.141+0000 7f94b9a54700 1 -- 172.21.15.44:0/2378062304 shutdown_connections 2023-09-10T00:39:53.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.141+0000 7f94b9a54700 1 --2- 172.21.15.44:0/2378062304 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f949c0a0790 0x7f949c0a2c50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.141+0000 7f94b9a54700 1 --2- 172.21.15.44:0/2378062304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94b4106350 0x7f94b40ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.141+0000 7f94b9a54700 1 --2- 172.21.15.44:0/2378062304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b41010f0 0x7f94b4100a10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.141+0000 7f94b9a54700 1 --2- 172.21.15.44:0/2378062304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94b4103a20 0x7f94b40ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.141+0000 7f94b9a54700 1 -- 172.21.15.44:0/2378062304 >> 172.21.15.44:0/2378062304 conn(0x7f94b40faac0 msgr2=0x7f94b4101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:53.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.141+0000 7f94b9a54700 1 -- 172.21.15.44:0/2378062304 shutdown_connections 2023-09-10T00:39:53.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.141+0000 7f94b9a54700 1 -- 172.21.15.44:0/2378062304 wait complete. 2023-09-10T00:39:53.147 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2023-09-10T00:39:53.155 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:39:53.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.280+0000 7f8660978700 1 -- 172.21.15.44:0/1608679819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f865c0ff4e0 msgr2=0x7f865c0ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:53.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.280+0000 7f8660978700 1 --2- 172.21.15.44:0/1608679819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f865c0ff4e0 0x7f865c0ff900 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f864c0099c0 tx=0x7f864c009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.280+0000 7f8660978700 1 -- 172.21.15.44:0/1608679819 shutdown_connections 2023-09-10T00:39:53.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.280+0000 7f8660978700 1 --2- 172.21.15.44:0/1608679819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c101070 0x7f865c107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.280+0000 7f8660978700 1 --2- 172.21.15.44:0/1608679819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f865c0ff4e0 0x7f865c0ff900 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.280+0000 7f8660978700 1 --2- 172.21.15.44:0/1608679819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865c1006d0 0x7f865c100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.280+0000 7f8660978700 1 -- 172.21.15.44:0/1608679819 >> 172.21.15.44:0/1608679819 conn(0x7f865c0faa30 msgr2=0x7f865c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:53.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.280+0000 7f8660978700 1 -- 172.21.15.44:0/1608679819 shutdown_connections 2023-09-10T00:39:53.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.280+0000 7f8660978700 1 -- 172.21.15.44:0/1608679819 wait complete. 2023-09-10T00:39:53.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.280+0000 7f8660978700 1 Processor -- start 2023-09-10T00:39:53.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f8660978700 1 -- start start 2023-09-10T00:39:53.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f8660978700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c0ff4e0 0x7f865c0682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:53.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f8660978700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865c1006d0 0x7f865c19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:53.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f8660978700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f865c101070 0x7f865c19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:53.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f8660978700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c19dd00 con 0x7f865c101070 2023-09-10T00:39:53.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f8660978700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c19de70 con 0x7f865c0ff4e0 2023-09-10T00:39:53.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f8660978700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c19e030 con 0x7f865c1006d0 2023-09-10T00:39:53.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f865ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f865c101070 0x7f865c19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:53.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f865ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f865c101070 0x7f865c19d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47504/0 (socket says 172.21.15.44:47504) 2023-09-10T00:39:53.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f865ad9d700 1 -- 172.21.15.44:0/2456065970 learned_addr learned my addr 172.21.15.44:0/2456065970 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:53.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f8659d9b700 1 --2- 172.21.15.44:0/2456065970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865c1006d0 0x7f865c19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:53.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f865a59c700 1 --2- 172.21.15.44:0/2456065970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c0ff4e0 0x7f865c0682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:53.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f865ad9d700 1 -- 172.21.15.44:0/2456065970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865c1006d0 msgr2=0x7f865c19af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:53.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f865ad9d700 1 --2- 172.21.15.44:0/2456065970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865c1006d0 0x7f865c19af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f865ad9d700 1 -- 172.21.15.44:0/2456065970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c0ff4e0 msgr2=0x7f865c0682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:53.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f865ad9d700 1 --2- 172.21.15.44:0/2456065970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c0ff4e0 0x7f865c0682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f865ad9d700 1 -- 172.21.15.44:0/2456065970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c0095f0 con 0x7f865c101070 2023-09-10T00:39:53.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.281+0000 7f865ad9d700 1 --2- 172.21.15.44:0/2456065970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f865c101070 0x7f865c19d560 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8654007d70 tx=0x7f865400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:53.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.282+0000 7f864b7fe700 1 -- 172.21.15.44:0/2456065970 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865400ad40 con 0x7f865c101070 2023-09-10T00:39:53.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.282+0000 7f864b7fe700 1 -- 172.21.15.44:0/2456065970 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86540128b0 con 0x7f865c101070 2023-09-10T00:39:53.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.282+0000 7f8660978700 1 -- 172.21.15.44:0/2456065970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c19e310 con 0x7f865c101070 2023-09-10T00:39:53.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.282+0000 7f864b7fe700 1 -- 172.21.15.44:0/2456065970 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86540115a0 con 0x7f865c101070 2023-09-10T00:39:53.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.282+0000 7f8660978700 1 -- 172.21.15.44:0/2456065970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c19e820 con 0x7f865c101070 2023-09-10T00:39:53.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.282+0000 7f8660978700 1 -- 172.21.15.44:0/2456065970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c106ac0 con 0x7f865c101070 2023-09-10T00:39:53.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.284+0000 7f864b7fe700 1 -- 172.21.15.44:0/2456065970 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f8654012a20 con 0x7f865c101070 2023-09-10T00:39:53.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.284+0000 7f864b7fe700 1 --2- 172.21.15.44:0/2456065970 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f86440a0710 0x7f86440a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:53.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.284+0000 7f864b7fe700 1 -- 172.21.15.44:0/2456065970 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f86540cc520 con 0x7f865c101070 2023-09-10T00:39:53.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.284+0000 7f865a59c700 1 --2- 172.21.15.44:0/2456065970 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f86440a0710 0x7f86440a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:53.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.285+0000 7f865a59c700 1 --2- 172.21.15.44:0/2456065970 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f86440a0710 0x7f86440a2bd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f864c000c00 tx=0x7f864c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:53.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.288+0000 7f864b7fe700 1 -- 172.21.15.44:0/2456065970 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8654095cf0 con 0x7f865c101070 2023-09-10T00:39:53.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.490+0000 7f8660978700 1 -- 172.21.15.44:0/2456065970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f865c10ad70 con 0x7f865c101070 2023-09-10T00:39:53.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.491+0000 7f864b7fe700 1 -- 172.21.15.44:0/2456065970 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24340 (secure 0 0 0) 0x7f8654099310 con 0x7f865c101070 2023-09-10T00:39:53.493 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:53.493 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":89,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:39:17.515738+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:39:53.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.493+0000 7f8660978700 1 -- 172.21.15.44:0/2456065970 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f86440a0710 msgr2=0x7f86440a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:53.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.493+0000 7f8660978700 1 --2- 172.21.15.44:0/2456065970 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f86440a0710 0x7f86440a2bd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f864c000c00 tx=0x7f864c005fb0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.493+0000 7f8660978700 1 -- 172.21.15.44:0/2456065970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f865c101070 msgr2=0x7f865c19d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:53.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.493+0000 7f8660978700 1 --2- 172.21.15.44:0/2456065970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f865c101070 0x7f865c19d560 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8654007d70 tx=0x7f865400a340 comp rx=0 tx=0).stop 2023-09-10T00:39:53.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.494+0000 7f8660978700 1 -- 172.21.15.44:0/2456065970 shutdown_connections 2023-09-10T00:39:53.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.494+0000 7f8660978700 1 --2- 172.21.15.44:0/2456065970 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f86440a0710 0x7f86440a2bd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.494+0000 7f8660978700 1 --2- 172.21.15.44:0/2456065970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f865c101070 0x7f865c19d560 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.494+0000 7f8660978700 1 --2- 172.21.15.44:0/2456065970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c0ff4e0 0x7f865c0682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.494+0000 7f8660978700 1 --2- 172.21.15.44:0/2456065970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865c1006d0 0x7f865c19af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.494+0000 7f8660978700 1 -- 172.21.15.44:0/2456065970 >> 172.21.15.44:0/2456065970 conn(0x7f865c0faa30 msgr2=0x7f865c103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:53.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.494+0000 7f8660978700 1 -- 172.21.15.44:0/2456065970 shutdown_connections 2023-09-10T00:39:53.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.494+0000 7f8660978700 1 -- 172.21.15.44:0/2456065970 wait complete. 2023-09-10T00:39:53.508 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:39:53.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.634+0000 7f2301b5f700 1 -- 172.21.15.44:0/1120938177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22fc1006d0 msgr2=0x7f22fc100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:53.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.634+0000 7f2301b5f700 1 --2- 172.21.15.44:0/1120938177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22fc1006d0 0x7f22fc100b30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0099c0 tx=0x7f22ec009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.634+0000 7f2301b5f700 1 -- 172.21.15.44:0/1120938177 shutdown_connections 2023-09-10T00:39:53.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.634+0000 7f2301b5f700 1 --2- 172.21.15.44:0/1120938177 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22fc101070 0x7f22fc107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.634+0000 7f2301b5f700 1 --2- 172.21.15.44:0/1120938177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22fc0ff4e0 0x7f22fc0ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.634+0000 7f2301b5f700 1 --2- 172.21.15.44:0/1120938177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22fc1006d0 0x7f22fc100b30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.634+0000 7f2301b5f700 1 -- 172.21.15.44:0/1120938177 >> 172.21.15.44:0/1120938177 conn(0x7f22fc0faa30 msgr2=0x7f22fc0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:53.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.634+0000 7f2301b5f700 1 -- 172.21.15.44:0/1120938177 shutdown_connections 2023-09-10T00:39:53.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.634+0000 7f2301b5f700 1 -- 172.21.15.44:0/1120938177 wait complete. 2023-09-10T00:39:53.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.634+0000 7f2301b5f700 1 Processor -- start 2023-09-10T00:39:53.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f2301b5f700 1 -- start start 2023-09-10T00:39:53.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f2301b5f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22fc0ff4e0 0x7f22fc103470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:53.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f2301b5f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22fc1006d0 0x7f22fc101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:53.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f2301b5f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22fc101070 0x7f22fc101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:53.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f2301b5f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc102790 con 0x7f22fc0ff4e0 2023-09-10T00:39:53.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f2301b5f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc102900 con 0x7f22fc101070 2023-09-10T00:39:53.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f2301b5f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc1a0230 con 0x7f22fc1006d0 2023-09-10T00:39:53.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f22fb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22fc0ff4e0 0x7f22fc103470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:53.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f22fb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22fc0ff4e0 0x7f22fc103470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47514/0 (socket says 172.21.15.44:47514) 2023-09-10T00:39:53.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f22fb7fe700 1 -- 172.21.15.44:0/126096701 learned_addr learned my addr 172.21.15.44:0/126096701 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:53.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f22faffd700 1 --2- 172.21.15.44:0/126096701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22fc1006d0 0x7f22fc101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:53.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f22fbfff700 1 --2- 172.21.15.44:0/126096701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22fc101070 0x7f22fc101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:53.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f22fb7fe700 1 -- 172.21.15.44:0/126096701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22fc1006d0 msgr2=0x7f22fc101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:53.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f22fb7fe700 1 --2- 172.21.15.44:0/126096701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22fc1006d0 0x7f22fc101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f22fb7fe700 1 -- 172.21.15.44:0/126096701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22fc101070 msgr2=0x7f22fc101ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:53.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f22fb7fe700 1 --2- 172.21.15.44:0/126096701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22fc101070 0x7f22fc101ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.635+0000 7f22fb7fe700 1 -- 172.21.15.44:0/126096701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec0095f0 con 0x7f22fc0ff4e0 2023-09-10T00:39:53.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.636+0000 7f22fb7fe700 1 --2- 172.21.15.44:0/126096701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22fc0ff4e0 0x7f22fc103470 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f22e400d980 tx=0x7f22e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:53.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.636+0000 7f22f8ff9700 1 -- 172.21.15.44:0/126096701 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e400f890 con 0x7f22fc0ff4e0 2023-09-10T00:39:53.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.636+0000 7f22f8ff9700 1 -- 172.21.15.44:0/126096701 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22e400fe80 con 0x7f22fc0ff4e0 2023-09-10T00:39:53.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.636+0000 7f2301b5f700 1 -- 172.21.15.44:0/126096701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22fc1a0510 con 0x7f22fc0ff4e0 2023-09-10T00:39:53.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.636+0000 7f22f8ff9700 1 -- 172.21.15.44:0/126096701 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4019360 con 0x7f22fc0ff4e0 2023-09-10T00:39:53.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.636+0000 7f2301b5f700 1 -- 172.21.15.44:0/126096701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22fc1a0a70 con 0x7f22fc0ff4e0 2023-09-10T00:39:53.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.637+0000 7f2301b5f700 1 -- 172.21.15.44:0/126096701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22fc10abb0 con 0x7f22fc0ff4e0 2023-09-10T00:39:53.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.638+0000 7f22f8ff9700 1 -- 172.21.15.44:0/126096701 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f22e400f9f0 con 0x7f22fc0ff4e0 2023-09-10T00:39:53.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.638+0000 7f22f8ff9700 1 --2- 172.21.15.44:0/126096701 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f22e80a0790 0x7f22e80a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:53.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.638+0000 7f22f8ff9700 1 -- 172.21.15.44:0/126096701 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f22e40cc2a0 con 0x7f22fc0ff4e0 2023-09-10T00:39:53.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.639+0000 7f22faffd700 1 --2- 172.21.15.44:0/126096701 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f22e80a0790 0x7f22e80a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:53.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.639+0000 7f22faffd700 1 --2- 172.21.15.44:0/126096701 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f22e80a0790 0x7f22e80a2c50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00b580 tx=0x7f22ec005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:53.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.642+0000 7f22f8ff9700 1 -- 172.21.15.44:0/126096701 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f22e4095960 con 0x7f22fc0ff4e0 2023-09-10T00:39:53.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:53 smithi044 ceph-mon[98676]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-10T00:39:53.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:53 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/737197013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:39:53.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:53 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2378062304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:39:53.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:53 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2456065970' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:53.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.895+0000 7f2301b5f700 1 -- 172.21.15.44:0/126096701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22fc10adc0 con 0x7f22fc0ff4e0 2023-09-10T00:39:53.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.895+0000 7f22f8ff9700 1 -- 172.21.15.44:0/126096701 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11369 (secure 0 0 0) 0x7f22e4020090 con 0x7f22fc0ff4e0 2023-09-10T00:39:53.897 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:39:53.897 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-10T00:38:39.183533+0000","modified":"2023-09-10T00:39:02.480227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24605,"mds_1":24574,"mds_2":24572,"mds_3":24515,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14751":{"gid":14751,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/3958270510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":3958270510},{"type":"v1","addr":"172.21.15.44:6839","nonce":3958270510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14763":{"gid":14763,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/1241002552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1241002552},{"type":"v1","addr":"172.21.15.44:6841","nonce":1241002552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24556":{"gid":24556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/3222652693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":3222652693},{"type":"v1","addr":"172.21.15.131:6839","nonce":3222652693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24605":{"gid":24605,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/2622966260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2622966260},{"type":"v1","addr":"172.21.15.99:6839","nonce":2622966260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-10T00:39:53.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.898+0000 7f2301b5f700 1 -- 172.21.15.44:0/126096701 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f22e80a0790 msgr2=0x7f22e80a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:53.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.898+0000 7f2301b5f700 1 --2- 172.21.15.44:0/126096701 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f22e80a0790 0x7f22e80a2c50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00b580 tx=0x7f22ec005ea0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.898+0000 7f2301b5f700 1 -- 172.21.15.44:0/126096701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22fc0ff4e0 msgr2=0x7f22fc103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:53.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.898+0000 7f2301b5f700 1 --2- 172.21.15.44:0/126096701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22fc0ff4e0 0x7f22fc103470 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f22e400d980 tx=0x7f22e400dc90 comp rx=0 tx=0).stop 2023-09-10T00:39:53.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.898+0000 7f2301b5f700 1 -- 172.21.15.44:0/126096701 shutdown_connections 2023-09-10T00:39:53.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.898+0000 7f2301b5f700 1 --2- 172.21.15.44:0/126096701 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f22e80a0790 0x7f22e80a2c50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.898+0000 7f2301b5f700 1 --2- 172.21.15.44:0/126096701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22fc101070 0x7f22fc101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.898+0000 7f2301b5f700 1 --2- 172.21.15.44:0/126096701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22fc0ff4e0 0x7f22fc103470 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.898+0000 7f2301b5f700 1 --2- 172.21.15.44:0/126096701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22fc1006d0 0x7f22fc101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:53.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.898+0000 7f2301b5f700 1 -- 172.21.15.44:0/126096701 >> 172.21.15.44:0/126096701 conn(0x7f22fc0faa30 msgr2=0x7f22fc0677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:53.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.898+0000 7f2301b5f700 1 -- 172.21.15.44:0/126096701 shutdown_connections 2023-09-10T00:39:53.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:53.898+0000 7f2301b5f700 1 -- 172.21.15.44:0/126096701 wait complete. 2023-09-10T00:39:53.902 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2023-09-10T00:39:53.912 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-10T00:38:39.183533+0000', 'modified': '2023-09-10T00:39:02.480227+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24605, 'mds_1': 24574, 'mds_2': 24572, 'mds_3': 24515, 'mds_4': 24502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14751': {'gid': 14751, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6839/3958270510', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6838', 'nonce': 3958270510}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 3958270510}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14763': {'gid': 14763, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6841/1241002552', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6840', 'nonce': 1241002552}, {'type': 'v1', 'addr': '172.21.15.44:6841', 'nonce': 1241002552}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24502': {'gid': 24502, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.131:6833/3395466517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 3395466517}, {'type': 'v1', 'addr': '172.21.15.131:6833', 'nonce': 3395466517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24515': {'gid': 24515, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.99:6833/1934109590', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 1934109590}, {'type': 'v1', 'addr': '172.21.15.99:6833', 'nonce': 1934109590}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24539': {'gid': 24539, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6835/2403105238', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 2403105238}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 2403105238}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24547': {'gid': 24547, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/1445095724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 1445095724}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1445095724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24556': {'gid': 24556, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6837/2397154745', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6836', 'nonce': 2397154745}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 2397154745}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24572': {'gid': 24572, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.99:6837/566283638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 566283638}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 566283638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24574': {'gid': 24574, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.131:6839/3222652693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 3222652693}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 3222652693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24605': {'gid': 24605, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6839/2622966260', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 2622966260}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 2622966260}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-10T00:39:53.912 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-09-10T00:39:53.913 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2023-09-10T00:39:54.041 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.038+0000 7eff2748a700 1 -- 172.21.15.44:0/677094542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff20101080 msgr2=0x7eff20107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.038+0000 7eff2748a700 1 --2- 172.21.15.44:0/677094542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff20101080 0x7eff20107930 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7eff1800b0d0 tx=0x7eff1800b3e0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.038+0000 7eff2748a700 1 -- 172.21.15.44:0/677094542 shutdown_connections 2023-09-10T00:39:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.038+0000 7eff2748a700 1 --2- 172.21.15.44:0/677094542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff20101080 0x7eff20107930 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.038+0000 7eff2748a700 1 --2- 172.21.15.44:0/677094542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff200ff4f0 0x7eff200ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.038+0000 7eff2748a700 1 --2- 172.21.15.44:0/677094542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff201006e0 0x7eff20100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.038+0000 7eff2748a700 1 -- 172.21.15.44:0/677094542 >> 172.21.15.44:0/677094542 conn(0x7eff200faa60 msgr2=0x7eff200fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.038+0000 7eff2748a700 1 -- 172.21.15.44:0/677094542 shutdown_connections 2023-09-10T00:39:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.038+0000 7eff2748a700 1 -- 172.21.15.44:0/677094542 wait complete. 2023-09-10T00:39:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff2748a700 1 Processor -- start 2023-09-10T00:39:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff2748a700 1 -- start start 2023-09-10T00:39:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff2748a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff200ff4f0 0x7eff200682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff2748a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff201006e0 0x7eff2019b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff2748a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff20101080 0x7eff2019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff2748a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2019de10 con 0x7eff20101080 2023-09-10T00:39:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff2748a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2019df80 con 0x7eff200ff4f0 2023-09-10T00:39:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff2748a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2019e140 con 0x7eff201006e0 2023-09-10T00:39:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff25a27700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff20101080 0x7eff2019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff25a27700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff20101080 0x7eff2019d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47532/0 (socket says 172.21.15.44:47532) 2023-09-10T00:39:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff25a27700 1 -- 172.21.15.44:0/161986299 learned_addr learned my addr 172.21.15.44:0/161986299 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff25226700 1 --2- 172.21.15.44:0/161986299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff200ff4f0 0x7eff200682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff24a25700 1 --2- 172.21.15.44:0/161986299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff201006e0 0x7eff2019b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff25a27700 1 -- 172.21.15.44:0/161986299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff201006e0 msgr2=0x7eff2019b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff25a27700 1 --2- 172.21.15.44:0/161986299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff201006e0 0x7eff2019b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff25a27700 1 -- 172.21.15.44:0/161986299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff200ff4f0 msgr2=0x7eff200682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff25a27700 1 --2- 172.21.15.44:0/161986299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff200ff4f0 0x7eff200682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff25a27700 1 -- 172.21.15.44:0/161986299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff18009d00 con 0x7eff20101080 2023-09-10T00:39:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff25226700 1 --2- 172.21.15.44:0/161986299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff200ff4f0 0x7eff200682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.039+0000 7eff25a27700 1 --2- 172.21.15.44:0/161986299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff20101080 0x7eff2019d670 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7eff18009fd0 tx=0x7eff1800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.040+0000 7eff167fc700 1 -- 172.21.15.44:0/161986299 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1800e040 con 0x7eff20101080 2023-09-10T00:39:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.040+0000 7eff167fc700 1 -- 172.21.15.44:0/161986299 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff18003e10 con 0x7eff20101080 2023-09-10T00:39:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.040+0000 7eff167fc700 1 -- 172.21.15.44:0/161986299 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff180052a0 con 0x7eff20101080 2023-09-10T00:39:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.040+0000 7eff2748a700 1 -- 172.21.15.44:0/161986299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2019e3c0 con 0x7eff20101080 2023-09-10T00:39:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.040+0000 7eff2748a700 1 -- 172.21.15.44:0/161986299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2019e840 con 0x7eff20101080 2023-09-10T00:39:54.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:53 smithi131 ceph-mon[103713]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-10T00:39:54.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:53 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/737197013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:39:54.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:53 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2378062304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:39:54.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:53 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2456065970' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:54.055 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.042+0000 7eff167fc700 1 -- 172.21.15.44:0/161986299 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7eff1801b450 con 0x7eff20101080 2023-09-10T00:39:54.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.043+0000 7eff167fc700 1 --2- 172.21.15.44:0/161986299 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7eff0c0a0750 0x7eff0c0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.043+0000 7eff25226700 1 --2- 172.21.15.44:0/161986299 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7eff0c0a0750 0x7eff0c0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.043+0000 7eff167fc700 1 -- 172.21.15.44:0/161986299 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7eff180cd7d0 con 0x7eff20101080 2023-09-10T00:39:54.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.043+0000 7eff25226700 1 --2- 172.21.15.44:0/161986299 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7eff0c0a0750 0x7eff0c0a2c10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7eff20100540 tx=0x7eff10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.043+0000 7eff33650b80 1 Processor -- start 2023-09-10T00:39:54.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff33650b80 1 -- start start 2023-09-10T00:39:54.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff33650b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195ebb0 0x5589c195efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff33650b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195f550 0x5589c195d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff33650b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ddb0 0x5589c195e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff33650b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589c18b66e0 con 0x5589c195f550 2023-09-10T00:39:54.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff33650b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589c18b4000 con 0x5589c195ddb0 2023-09-10T00:39:54.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff33650b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589c18af8f0 con 0x5589c195ebb0 2023-09-10T00:39:54.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff24a25700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195f550 0x5589c195d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff24a25700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195f550 0x5589c195d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47538/0 (socket says 172.21.15.44:47538) 2023-09-10T00:39:54.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff24a25700 1 -- 172.21.15.44:0/763708707 learned_addr learned my addr 172.21.15.44:0/763708707 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff25226700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195ebb0 0x5589c195efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff25a27700 1 --2- 172.21.15.44:0/763708707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ddb0 0x5589c195e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff24a25700 1 -- 172.21.15.44:0/763708707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195ebb0 msgr2=0x5589c195efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff24a25700 1 --2- 172.21.15.44:0/763708707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195ebb0 0x5589c195efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff24a25700 1 -- 172.21.15.44:0/763708707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ddb0 msgr2=0x5589c195e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff24a25700 1 --2- 172.21.15.44:0/763708707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ddb0 0x5589c195e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff24a25700 1 -- 172.21.15.44:0/763708707 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589c16dbd90 con 0x5589c195f550 2023-09-10T00:39:54.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff25a27700 1 --2- 172.21.15.44:0/763708707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ddb0 0x5589c195e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:54.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff25226700 1 --2- 172.21.15.44:0/763708707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195ebb0 0x5589c195efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:39:54.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff24a25700 1 --2- 172.21.15.44:0/763708707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195f550 0x5589c195d7b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00dc70 tx=0x7eff1c00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ec2f18d06025ea server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.044+0000 7eff0b7fe700 1 -- 172.21.15.44:0/763708707 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c00fb70 con 0x5589c195f550 2023-09-10T00:39:54.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.045+0000 7eff0b7fe700 1 -- 172.21.15.44:0/763708707 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff1c004cc0 con 0x5589c195f550 2023-09-10T00:39:54.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.045+0000 7eff0b7fe700 1 -- 172.21.15.44:0/763708707 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c005680 con 0x5589c195f550 2023-09-10T00:39:54.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.045+0000 7eff33650b80 1 -- 172.21.15.44:0/763708707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195f550 msgr2=0x5589c195d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.045+0000 7eff33650b80 1 --2- 172.21.15.44:0/763708707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195f550 0x5589c195d7b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00dc70 tx=0x7eff1c00df80 comp rx=0 tx=0).stop 2023-09-10T00:39:54.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.045+0000 7eff33650b80 1 -- 172.21.15.44:0/763708707 shutdown_connections 2023-09-10T00:39:54.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.045+0000 7eff33650b80 1 --2- 172.21.15.44:0/763708707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ddb0 0x5589c195e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.045+0000 7eff33650b80 1 --2- 172.21.15.44:0/763708707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195ebb0 0x5589c195efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.045+0000 7eff33650b80 1 --2- 172.21.15.44:0/763708707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195f550 0x5589c195d7b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.045+0000 7eff33650b80 1 -- 172.21.15.44:0/763708707 >> 172.21.15.44:0/763708707 conn(0x5589c18d2040 msgr2=0x5589c19586d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.045+0000 7eff33650b80 1 -- 172.21.15.44:0/763708707 shutdown_connections 2023-09-10T00:39:54.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.045+0000 7eff33650b80 1 -- 172.21.15.44:0/763708707 wait complete. 2023-09-10T00:39:54.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.046+0000 7eff33650b80 1 -- start start 2023-09-10T00:39:54.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.046+0000 7eff33650b80 1 Processor -- start 2023-09-10T00:39:54.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.046+0000 7eff33650b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ddb0 0x5589c19675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.046+0000 7eff33650b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ebb0 0x5589c1967af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.046+0000 7eff33650b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195f550 0x5589c196eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.046+0000 7eff33650b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589c184db20 con 0x5589c195ddb0 2023-09-10T00:39:54.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.046+0000 7eff33650b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589c184d820 con 0x5589c195ebb0 2023-09-10T00:39:54.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.046+0000 7eff33650b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589c17318d0 con 0x5589c195f550 2023-09-10T00:39:54.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.046+0000 7eff33650b80 10 client.0 _ll_register_callbacks cb 0x5589c1797a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:39:54.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.046+0000 7eff25226700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ddb0 0x5589c19675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.046+0000 7eff25226700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ddb0 0x5589c19675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47542/0 (socket says 172.21.15.44:47542) 2023-09-10T00:39:54.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.046+0000 7eff25226700 1 -- 172.21.15.44:0/2275976198 learned_addr learned my addr 172.21.15.44:0/2275976198 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.046+0000 7eff24a25700 1 --2- 172.21.15.44:0/2275976198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ebb0 0x5589c1967af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.046+0000 7eff25a27700 1 --2- 172.21.15.44:0/2275976198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195f550 0x5589c196eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff25226700 1 -- 172.21.15.44:0/2275976198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195f550 msgr2=0x5589c196eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff25226700 1 --2- 172.21.15.44:0/2275976198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195f550 0x5589c196eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff25226700 1 -- 172.21.15.44:0/2275976198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ebb0 msgr2=0x5589c1967af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff25226700 1 --2- 172.21.15.44:0/2275976198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ebb0 0x5589c1967af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff25226700 1 -- 172.21.15.44:0/2275976198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff10011c60 con 0x5589c195ddb0 2023-09-10T00:39:54.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff24a25700 1 --2- 172.21.15.44:0/2275976198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ebb0 0x5589c1967af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:54.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff25226700 1 --2- 172.21.15.44:0/2275976198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ddb0 0x5589c19675b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7eff10012c10 tx=0x7eff100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70bdbd24aea415f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff08ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:39:54.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff08ff9700 1 -- 172.21.15.44:0/2275976198 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff10011eb0 con 0x5589c195ddb0 2023-09-10T00:39:54.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff08ff9700 1 -- 172.21.15.44:0/2275976198 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7eff10018070 con 0x5589c195ddb0 2023-09-10T00:39:54.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff33650b80 1 -- 172.21.15.44:0/2275976198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589c18074a0 con 0x5589c195ddb0 2023-09-10T00:39:54.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff08ff9700 1 -- 172.21.15.44:0/2275976198 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff10004830 con 0x5589c195ddb0 2023-09-10T00:39:54.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff33650b80 10 client.15342 fetch_fsmap learned FSMap version 27 2023-09-10T00:39:54.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff33650b80 1 -- 172.21.15.44:0/2275976198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7eff1c00d8a0 con 0x5589c195ddb0 2023-09-10T00:39:54.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff08ff9700 1 -- 172.21.15.44:0/2275976198 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff10032020 con 0x5589c195ddb0 2023-09-10T00:39:54.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff08ff9700 1 -- 172.21.15.44:0/2275976198 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7eff10005020 con 0x5589c195ddb0 2023-09-10T00:39:54.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff33650b80 10 client.15342 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:39:54.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff33650b80 10 client.15342 resolve_mds: resolved name 'l' to daemon mds.l 2023-09-10T00:39:54.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff33650b80 20 client.15342 populate_metadata read hostname 'smithi044' 2023-09-10T00:39:54.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff33650b80 1 --2- 172.21.15.44:0/2275976198 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x5589c1972dc0 0x5589c1971ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff33650b80 4 client.15342 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:39:54.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.047+0000 7eff33650b80 1 -- 172.21.15.44:0/2275976198 --> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589c18c4850 con 0x5589c1972dc0 2023-09-10T00:39:54.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.048+0000 7eff24a25700 1 --2- 172.21.15.44:0/2275976198 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x5589c1972dc0 0x5589c1971ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.048+0000 7eff08ff9700 10 client.15342 ms_handle_connect on v2:172.21.15.131:6832/3395466517 2023-09-10T00:39:54.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.048+0000 7eff24a25700 1 --2- 172.21.15.44:0/2275976198 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x5589c1972dc0 0x5589c1971ce0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=867236e2cf1c1a6c server_cookie=b6ea3ac3bbca1179 in_seq=0 out_seq=0 2023-09-10T00:39:54.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.049+0000 7eff08ff9700 1 -- 172.21.15.44:0/2275976198 <== mds.4 v2:172.21.15.131:6832/3395466517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5589c18c4850 con 0x5589c1972dc0 2023-09-10T00:39:54.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.049+0000 7eff08ff9700 10 client.15342 handle_command_reply: tid=0 2023-09-10T00:39:54.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.049+0000 7eff33650b80 1 client.15342 shutdown 2023-09-10T00:39:54.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.049+0000 7eff33650b80 1 -- 172.21.15.44:0/2275976198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ddb0 msgr2=0x5589c19675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.049+0000 7eff33650b80 1 --2- 172.21.15.44:0/2275976198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ddb0 0x5589c19675b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7eff10012c10 tx=0x7eff100047c0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.049+0000 7eff33650b80 1 -- 172.21.15.44:0/2275976198 shutdown_connections 2023-09-10T00:39:54.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.049+0000 7eff33650b80 1 --2- 172.21.15.44:0/2275976198 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x5589c1972dc0 0x5589c1971ce0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.049+0000 7eff33650b80 1 --2- 172.21.15.44:0/2275976198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195f550 0x5589c196eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.049+0000 7eff08ff9700 0 client.15342 ms_handle_reset on v2:172.21.15.131:6832/3395466517 2023-09-10T00:39:54.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.049+0000 7eff33650b80 1 --2- 172.21.15.44:0/2275976198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ddb0 0x5589c19675b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.049+0000 7eff33650b80 1 --2- 172.21.15.44:0/2275976198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ebb0 0x5589c1967af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.049+0000 7eff33650b80 1 -- 172.21.15.44:0/2275976198 >> 172.21.15.44:0/2275976198 conn(0x5589c18d2040 msgr2=0x5589c1957d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.049+0000 7eff33650b80 1 -- 172.21.15.44:0/2275976198 shutdown_connections 2023-09-10T00:39:54.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.049+0000 7eff33650b80 1 -- 172.21.15.44:0/2275976198 wait complete. 2023-09-10T00:39:54.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.049+0000 7eff33650b80 20 client.15342 trim_cache size 0 max 16384 2023-09-10T00:39:54.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.067+0000 7eff33650b80 1 Processor -- start 2023-09-10T00:39:54.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.067+0000 7eff33650b80 1 -- start start 2023-09-10T00:39:54.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.067+0000 7eff33650b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ddb0 0x5589c1966280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.067+0000 7eff33650b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ebb0 0x5589c196eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff33650b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195f550 0x5589c19653e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff33650b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589c18574d0 con 0x5589c195ebb0 2023-09-10T00:39:54.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff33650b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589c1959640 con 0x5589c195ddb0 2023-09-10T00:39:54.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff33650b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589c1849e30 con 0x5589c195f550 2023-09-10T00:39:54.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff24a25700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ebb0 0x5589c196eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff24a25700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ebb0 0x5589c196eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47552/0 (socket says 172.21.15.44:47552) 2023-09-10T00:39:54.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff24a25700 1 -- 172.21.15.44:0/1904401587 learned_addr learned my addr 172.21.15.44:0/1904401587 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff25226700 1 --2- 172.21.15.44:0/1904401587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ddb0 0x5589c1966280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff25a27700 1 --2- 172.21.15.44:0/1904401587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195f550 0x5589c19653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff24a25700 1 -- 172.21.15.44:0/1904401587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195f550 msgr2=0x5589c19653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff24a25700 1 --2- 172.21.15.44:0/1904401587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195f550 0x5589c19653e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff24a25700 1 -- 172.21.15.44:0/1904401587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ddb0 msgr2=0x5589c1966280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff24a25700 1 --2- 172.21.15.44:0/1904401587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ddb0 0x5589c1966280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff24a25700 1 -- 172.21.15.44:0/1904401587 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589c16c4c10 con 0x5589c195ebb0 2023-09-10T00:39:54.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff25a27700 1 --2- 172.21.15.44:0/1904401587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195f550 0x5589c19653e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:39:54.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff25226700 1 --2- 172.21.15.44:0/1904401587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ddb0 0x5589c1966280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:54.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff24a25700 1 --2- 172.21.15.44:0/1904401587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ebb0 0x5589c196eac0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00d350 tx=0x7eff1c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aceeed6a40a72e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff0bfff700 1 -- 172.21.15.44:0/1904401587 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c005720 con 0x5589c195ebb0 2023-09-10T00:39:54.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff0bfff700 1 -- 172.21.15.44:0/1904401587 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff1c021040 con 0x5589c195ebb0 2023-09-10T00:39:54.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.068+0000 7eff0bfff700 1 -- 172.21.15.44:0/1904401587 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c0280d0 con 0x5589c195ebb0 2023-09-10T00:39:54.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.069+0000 7eff33650b80 1 -- 172.21.15.44:0/1904401587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ebb0 msgr2=0x5589c196eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.069+0000 7eff33650b80 1 --2- 172.21.15.44:0/1904401587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ebb0 0x5589c196eac0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00d350 tx=0x7eff1c013040 comp rx=0 tx=0).stop 2023-09-10T00:39:54.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.069+0000 7eff33650b80 1 -- 172.21.15.44:0/1904401587 shutdown_connections 2023-09-10T00:39:54.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.069+0000 7eff33650b80 1 --2- 172.21.15.44:0/1904401587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195f550 0x5589c19653e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.069+0000 7eff33650b80 1 --2- 172.21.15.44:0/1904401587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195ddb0 0x5589c1966280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.069+0000 7eff33650b80 1 --2- 172.21.15.44:0/1904401587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ebb0 0x5589c196eac0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.069+0000 7eff33650b80 1 -- 172.21.15.44:0/1904401587 >> 172.21.15.44:0/1904401587 conn(0x5589c18d2040 msgr2=0x5589c1971ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.069+0000 7eff33650b80 1 -- 172.21.15.44:0/1904401587 shutdown_connections 2023-09-10T00:39:54.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.069+0000 7eff33650b80 1 -- 172.21.15.44:0/1904401587 wait complete. 2023-09-10T00:39:54.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.069+0000 7eff33650b80 1 -- start start 2023-09-10T00:39:54.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.069+0000 7eff33650b80 1 Processor -- start 2023-09-10T00:39:54.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff33650b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ddb0 0x5589c196b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff33650b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195ebb0 0x5589c196bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff33650b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195f550 0x5589c19c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff33650b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589c196eb40 con 0x5589c195ddb0 2023-09-10T00:39:54.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff33650b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589c196ecb0 con 0x5589c195f550 2023-09-10T00:39:54.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff33650b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589c196ee70 con 0x5589c195ebb0 2023-09-10T00:39:54.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff33650b80 10 client.0 _ll_register_callbacks cb 0x7eff100127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:39:54.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff25226700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ddb0 0x5589c196b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff25226700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ddb0 0x5589c196b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47556/0 (socket says 172.21.15.44:47556) 2023-09-10T00:39:54.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff25226700 1 -- 172.21.15.44:0/4248285905 learned_addr learned my addr 172.21.15.44:0/4248285905 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff24a25700 1 --2- 172.21.15.44:0/4248285905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195ebb0 0x5589c196bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff25a27700 1 --2- 172.21.15.44:0/4248285905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195f550 0x5589c19c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff25226700 1 -- 172.21.15.44:0/4248285905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195ebb0 msgr2=0x5589c196bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff25226700 1 --2- 172.21.15.44:0/4248285905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195ebb0 0x5589c196bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff25226700 1 -- 172.21.15.44:0/4248285905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195f550 msgr2=0x5589c19c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff25226700 1 --2- 172.21.15.44:0/4248285905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195f550 0x5589c19c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff25226700 1 -- 172.21.15.44:0/4248285905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589c16dbd90 con 0x5589c195ddb0 2023-09-10T00:39:54.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff24a25700 1 --2- 172.21.15.44:0/4248285905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195ebb0 0x5589c196bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:54.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff25a27700 1 --2- 172.21.15.44:0/4248285905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195f550 0x5589c19c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:39:54.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff25226700 1 --2- 172.21.15.44:0/4248285905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ddb0 0x5589c196b5e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7eff10011ad0 tx=0x7eff10005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225443090aa9a21d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff08ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:39:54.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff08ff9700 1 -- 172.21.15.44:0/4248285905 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1001fe50 con 0x5589c195ddb0 2023-09-10T00:39:54.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff08ff9700 1 -- 172.21.15.44:0/4248285905 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7eff10018040 con 0x5589c195ddb0 2023-09-10T00:39:54.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.070+0000 7eff33650b80 1 -- 172.21.15.44:0/4248285905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff1c004cc0 con 0x5589c195ddb0 2023-09-10T00:39:54.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.071+0000 7eff08ff9700 1 -- 172.21.15.44:0/4248285905 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff10004830 con 0x5589c195ddb0 2023-09-10T00:39:54.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.071+0000 7eff33650b80 10 client.15348 fetch_fsmap learned FSMap version 27 2023-09-10T00:39:54.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.071+0000 7eff33650b80 1 -- 172.21.15.44:0/4248285905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5589c1702200 con 0x5589c195ddb0 2023-09-10T00:39:54.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.071+0000 7eff08ff9700 1 -- 172.21.15.44:0/4248285905 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff1c004cc0 con 0x5589c195ddb0 2023-09-10T00:39:54.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.071+0000 7eff08ff9700 1 -- 172.21.15.44:0/4248285905 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7eff100205d0 con 0x5589c195ddb0 2023-09-10T00:39:54.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.071+0000 7eff33650b80 10 client.15348 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:39:54.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.071+0000 7eff33650b80 10 client.15348 resolve_mds: resolved name 'l' to daemon mds.l 2023-09-10T00:39:54.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.071+0000 7eff33650b80 20 client.15348 populate_metadata read hostname 'smithi044' 2023-09-10T00:39:54.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.071+0000 7eff33650b80 1 --2- 172.21.15.44:0/4248285905 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x5589c19cc440 0x5589c19c8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.071+0000 7eff33650b80 4 client.15348 mds_command: new command op to 24502 tid=0[{"prefix": "status"}] 2023-09-10T00:39:54.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.071+0000 7eff33650b80 1 -- 172.21.15.44:0/4248285905 --> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5589c18a88d0 con 0x5589c19cc440 2023-09-10T00:39:54.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.071+0000 7eff24a25700 1 --2- 172.21.15.44:0/4248285905 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x5589c19cc440 0x5589c19c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff08ff9700 10 client.15348 ms_handle_connect on v2:172.21.15.131:6832/3395466517 2023-09-10T00:39:54.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff24a25700 1 --2- 172.21.15.44:0/4248285905 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x5589c19cc440 0x5589c19c8880 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a39ec38889f613fb server_cookie=d4b24cab38c956bd in_seq=0 out_seq=0 2023-09-10T00:39:54.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff08ff9700 1 -- 172.21.15.44:0/4248285905 <== mds.4 v2:172.21.15.131:6832/3395466517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5589c18a88d0 con 0x5589c19cc440 2023-09-10T00:39:54.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff08ff9700 10 client.15348 handle_command_reply: tid=0 2023-09-10T00:39:54.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff33650b80 1 client.15348 shutdown 2023-09-10T00:39:54.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff33650b80 1 -- 172.21.15.44:0/4248285905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ddb0 msgr2=0x5589c196b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff33650b80 1 --2- 172.21.15.44:0/4248285905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ddb0 0x5589c196b5e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7eff10011ad0 tx=0x7eff10005870 comp rx=0 tx=0).stop 2023-09-10T00:39:54.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff33650b80 1 -- 172.21.15.44:0/4248285905 shutdown_connections 2023-09-10T00:39:54.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff33650b80 1 --2- 172.21.15.44:0/4248285905 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x5589c19cc440 0x5589c19c8880 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff33650b80 1 --2- 172.21.15.44:0/4248285905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589c195f550 0x5589c19c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff33650b80 1 --2- 172.21.15.44:0/4248285905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589c195ddb0 0x5589c196b5e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff33650b80 1 --2- 172.21.15.44:0/4248285905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589c195ebb0 0x5589c196bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff08ff9700 0 client.15348 ms_handle_reset on v2:172.21.15.131:6832/3395466517 2023-09-10T00:39:54.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff33650b80 1 -- 172.21.15.44:0/4248285905 >> 172.21.15.44:0/4248285905 conn(0x5589c18d2040 msgr2=0x5589c19653e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff33650b80 1 -- 172.21.15.44:0/4248285905 shutdown_connections 2023-09-10T00:39:54.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff33650b80 1 -- 172.21.15.44:0/4248285905 wait complete. 2023-09-10T00:39:54.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.072+0000 7eff33650b80 20 client.15348 trim_cache size 0 max 16384 2023-09-10T00:39:54.096 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:39:54.096 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da", 2023-09-10T00:39:54.097 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 4, 2023-09-10T00:39:54.097 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24502, 2023-09-10T00:39:54.097 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2023-09-10T00:39:54.097 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2023-09-10T00:39:54.097 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2023-09-10T00:39:54.098 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 52.601773669000004, 2023-09-10T00:39:54.098 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2023-09-10T00:39:54.098 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 89, 2023-09-10T00:39:54.098 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 87, 2023-09-10T00:39:54.098 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 105.14651011300001 2023-09-10T00:39:54.099 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:39:54.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.073+0000 7eff2748a700 1 -- 172.21.15.44:0/161986299 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7eff0c0a0750 msgr2=0x7eff0c0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.073+0000 7eff2748a700 1 --2- 172.21.15.44:0/161986299 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7eff0c0a0750 0x7eff0c0a2c10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7eff20100540 tx=0x7eff10009260 comp rx=0 tx=0).stop 2023-09-10T00:39:54.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.073+0000 7eff2748a700 1 -- 172.21.15.44:0/161986299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff20101080 msgr2=0x7eff2019d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.073+0000 7eff2748a700 1 --2- 172.21.15.44:0/161986299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff20101080 0x7eff2019d670 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7eff18009fd0 tx=0x7eff1800cd70 comp rx=0 tx=0).stop 2023-09-10T00:39:54.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.073+0000 7eff2748a700 1 -- 172.21.15.44:0/161986299 shutdown_connections 2023-09-10T00:39:54.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.073+0000 7eff2748a700 1 --2- 172.21.15.44:0/161986299 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7eff0c0a0750 0x7eff0c0a2c10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.073+0000 7eff2748a700 1 --2- 172.21.15.44:0/161986299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff20101080 0x7eff2019d670 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.073+0000 7eff2748a700 1 --2- 172.21.15.44:0/161986299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff200ff4f0 0x7eff200682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.073+0000 7eff2748a700 1 --2- 172.21.15.44:0/161986299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff201006e0 0x7eff2019b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.073+0000 7eff2748a700 1 -- 172.21.15.44:0/161986299 >> 172.21.15.44:0/161986299 conn(0x7eff200faa60 msgr2=0x7eff200fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.073+0000 7eff2748a700 1 -- 172.21.15.44:0/161986299 shutdown_connections 2023-09-10T00:39:54.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.073+0000 7eff2748a700 1 -- 172.21.15.44:0/161986299 wait complete. 2023-09-10T00:39:54.102 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2023-09-10T00:39:54.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:53 smithi099 ceph-mon[102692]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-10T00:39:54.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:53 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/737197013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:39:54.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:53 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2378062304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:39:54.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:53 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2456065970' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:39:54.229 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.228+0000 7f1b0081b700 1 -- 172.21.15.44:0/2528471408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1afc101080 msgr2=0x7f1afc107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.230 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.228+0000 7f1b0081b700 1 --2- 172.21.15.44:0/2528471408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1afc101080 0x7f1afc107930 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1af400b0d0 tx=0x7f1af400b3e0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.230 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.228+0000 7f1b0081b700 1 -- 172.21.15.44:0/2528471408 shutdown_connections 2023-09-10T00:39:54.230 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.228+0000 7f1b0081b700 1 --2- 172.21.15.44:0/2528471408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1afc101080 0x7f1afc107930 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.230 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.228+0000 7f1b0081b700 1 --2- 172.21.15.44:0/2528471408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1afc0ff4f0 0x7f1afc0ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.231 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.228+0000 7f1b0081b700 1 --2- 172.21.15.44:0/2528471408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1afc1006e0 0x7f1afc100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.231 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.228+0000 7f1b0081b700 1 -- 172.21.15.44:0/2528471408 >> 172.21.15.44:0/2528471408 conn(0x7f1afc0faa60 msgr2=0x7f1afc0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.231 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.228+0000 7f1b0081b700 1 -- 172.21.15.44:0/2528471408 shutdown_connections 2023-09-10T00:39:54.231 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.228+0000 7f1b0081b700 1 -- 172.21.15.44:0/2528471408 wait complete. 2023-09-10T00:39:54.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.229+0000 7f1b0081b700 1 Processor -- start 2023-09-10T00:39:54.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.229+0000 7f1b0081b700 1 -- start start 2023-09-10T00:39:54.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.229+0000 7f1b0081b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1afc0ff4f0 0x7f1afc196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.229+0000 7f1b0081b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1afc1006e0 0x7f1afc196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.229+0000 7f1b0081b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1afc101080 0x7f1afc19d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.229+0000 7f1b0081b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc19de80 con 0x7f1afc1006e0 2023-09-10T00:39:54.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.229+0000 7f1b0081b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc19dfc0 con 0x7f1afc101080 2023-09-10T00:39:54.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.229+0000 7f1b0081b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc19e150 con 0x7f1afc0ff4f0 2023-09-10T00:39:54.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.229+0000 7f1af9d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1afc1006e0 0x7f1afc196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.229+0000 7f1afa59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1afc0ff4f0 0x7f1afc196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.229+0000 7f1af9d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1afc1006e0 0x7f1afc196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47572/0 (socket says 172.21.15.44:47572) 2023-09-10T00:39:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.229+0000 7f1af9d9b700 1 -- 172.21.15.44:0/2988313677 learned_addr learned my addr 172.21.15.44:0/2988313677 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.229+0000 7f1afad9d700 1 --2- 172.21.15.44:0/2988313677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1afc101080 0x7f1afc19d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.230+0000 7f1af9d9b700 1 -- 172.21.15.44:0/2988313677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1afc0ff4f0 msgr2=0x7f1afc196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.230+0000 7f1af9d9b700 1 --2- 172.21.15.44:0/2988313677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1afc0ff4f0 0x7f1afc196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.230+0000 7f1af9d9b700 1 -- 172.21.15.44:0/2988313677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1afc101080 msgr2=0x7f1afc19d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.230+0000 7f1af9d9b700 1 --2- 172.21.15.44:0/2988313677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1afc101080 0x7f1afc19d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.230+0000 7f1af9d9b700 1 -- 172.21.15.44:0/2988313677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af4009d00 con 0x7f1afc1006e0 2023-09-10T00:39:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.230+0000 7f1af9d9b700 1 --2- 172.21.15.44:0/2988313677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1afc1006e0 0x7f1afc196840 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400d950 tx=0x7f1ae400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.230+0000 7f1af37fe700 1 -- 172.21.15.44:0/2988313677 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae400f890 con 0x7f1afc1006e0 2023-09-10T00:39:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.230+0000 7f1af37fe700 1 -- 172.21.15.44:0/2988313677 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ae400fe80 con 0x7f1afc1006e0 2023-09-10T00:39:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.230+0000 7f1b0081b700 1 -- 172.21.15.44:0/2988313677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1afc19e430 con 0x7f1afc1006e0 2023-09-10T00:39:54.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.230+0000 7f1af37fe700 1 -- 172.21.15.44:0/2988313677 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae400e3b0 con 0x7f1afc1006e0 2023-09-10T00:39:54.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.230+0000 7f1b0081b700 1 -- 172.21.15.44:0/2988313677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1afc19e940 con 0x7f1afc1006e0 2023-09-10T00:39:54.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.232+0000 7f1af37fe700 1 -- 172.21.15.44:0/2988313677 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f1ae4010460 con 0x7f1afc1006e0 2023-09-10T00:39:54.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.233+0000 7f1af37fe700 1 --2- 172.21.15.44:0/2988313677 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1ae80a06d0 0x7f1ae80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.233+0000 7f1afa59c700 1 --2- 172.21.15.44:0/2988313677 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1ae80a06d0 0x7f1ae80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.233+0000 7f1af37fe700 1 -- 172.21.15.44:0/2988313677 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1ae40cbe30 con 0x7f1afc1006e0 2023-09-10T00:39:54.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.233+0000 7f1afa59c700 1 --2- 172.21.15.44:0/2988313677 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1ae80a06d0 0x7f1ae80a2b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1afc100540 tx=0x7f1aec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.233+0000 7f1b0c9e1b80 1 Processor -- start 2023-09-10T00:39:54.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1b0c9e1b80 1 -- start start 2023-09-10T00:39:54.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1b0c9e1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcaafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1b0c9e1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbcab550 0x55c3cbca97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1b0c9e1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbca9db0 0x55c3cbcaa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1b0c9e1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3cbc026e0 con 0x55c3cbca9db0 2023-09-10T00:39:54.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1b0c9e1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3cbc00000 con 0x55c3cbcab550 2023-09-10T00:39:54.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1b0c9e1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3cbbfb8f0 con 0x55c3cbcaabb0 2023-09-10T00:39:54.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1afad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbca9db0 0x55c3cbcaa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1afad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbca9db0 0x55c3cbcaa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47582/0 (socket says 172.21.15.44:47582) 2023-09-10T00:39:54.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1afad9d700 1 -- 172.21.15.44:0/3706109247 learned_addr learned my addr 172.21.15.44:0/3706109247 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1afa59c700 1 --2- 172.21.15.44:0/3706109247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcaafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1af9d9b700 1 --2- 172.21.15.44:0/3706109247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbcab550 0x55c3cbca97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1afad9d700 1 -- 172.21.15.44:0/3706109247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbcaabb0 msgr2=0x55c3cbcaafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1afad9d700 1 --2- 172.21.15.44:0/3706109247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcaafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1afad9d700 1 -- 172.21.15.44:0/3706109247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbcab550 msgr2=0x55c3cbca97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1afad9d700 1 --2- 172.21.15.44:0/3706109247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbcab550 0x55c3cbca97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1afad9d700 1 -- 172.21.15.44:0/3706109247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3cba27d90 con 0x55c3cbca9db0 2023-09-10T00:39:54.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1afa59c700 1 --2- 172.21.15.44:0/3706109247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcaafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:54.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1af9d9b700 1 --2- 172.21.15.44:0/3706109247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbcab550 0x55c3cbca97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:54.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.234+0000 7f1afad9d700 1 --2- 172.21.15.44:0/3706109247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbca9db0 0x55c3cbcaa260 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1af40074e0 tx=0x7f1af4007510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5092f5547ecd3f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.235+0000 7f1af0ff9700 1 -- 172.21.15.44:0/3706109247 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af400e070 con 0x55c3cbca9db0 2023-09-10T00:39:54.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.235+0000 7f1af0ff9700 1 -- 172.21.15.44:0/3706109247 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1af4003e90 con 0x55c3cbca9db0 2023-09-10T00:39:54.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.235+0000 7f1af0ff9700 1 -- 172.21.15.44:0/3706109247 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af40055a0 con 0x55c3cbca9db0 2023-09-10T00:39:54.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.235+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/3706109247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbca9db0 msgr2=0x55c3cbcaa260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.235+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/3706109247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbca9db0 0x55c3cbcaa260 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1af40074e0 tx=0x7f1af4007510 comp rx=0 tx=0).stop 2023-09-10T00:39:54.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.235+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/3706109247 shutdown_connections 2023-09-10T00:39:54.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.235+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/3706109247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbca9db0 0x55c3cbcaa260 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.235+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/3706109247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcaafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.235+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/3706109247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbcab550 0x55c3cbca97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.235+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/3706109247 >> 172.21.15.44:0/3706109247 conn(0x55c3cbc1e040 msgr2=0x55c3cbca46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.235+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/3706109247 shutdown_connections 2023-09-10T00:39:54.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.235+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/3706109247 wait complete. 2023-09-10T00:39:54.252 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1b0c9e1b80 1 -- start start 2023-09-10T00:39:54.252 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1b0c9e1b80 1 Processor -- start 2023-09-10T00:39:54.252 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1b0c9e1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbca9db0 0x55c3cbcb3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.252 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1b0c9e1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcb39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.252 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1b0c9e1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbcab550 0x55c3cbcbaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.253 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1b0c9e1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3cbb99b20 con 0x55c3cbcab550 2023-09-10T00:39:54.253 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1b0c9e1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3cbb99820 con 0x55c3cbcaabb0 2023-09-10T00:39:54.253 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1b0c9e1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3cba7d8d0 con 0x55c3cbca9db0 2023-09-10T00:39:54.253 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1b0c9e1b80 10 client.0 _ll_register_callbacks cb 0x55c3cbae3a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:39:54.253 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1afad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbcab550 0x55c3cbcbaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.253 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1afad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbcab550 0x55c3cbcbaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47588/0 (socket says 172.21.15.44:47588) 2023-09-10T00:39:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1afad9d700 1 -- 172.21.15.44:0/1341284806 learned_addr learned my addr 172.21.15.44:0/1341284806 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1af9d9b700 1 --2- 172.21.15.44:0/1341284806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcb39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1afa59c700 1 --2- 172.21.15.44:0/1341284806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbca9db0 0x55c3cbcb3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1afad9d700 1 -- 172.21.15.44:0/1341284806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbca9db0 msgr2=0x55c3cbcb3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1afad9d700 1 --2- 172.21.15.44:0/1341284806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbca9db0 0x55c3cbcb3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.255 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1afad9d700 1 -- 172.21.15.44:0/1341284806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbcaabb0 msgr2=0x55c3cbcb39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.255 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1afad9d700 1 --2- 172.21.15.44:0/1341284806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcb39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.255 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1afad9d700 1 -- 172.21.15.44:0/1341284806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1afc108280 con 0x55c3cbcab550 2023-09-10T00:39:54.255 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.236+0000 7f1afa59c700 1 --2- 172.21.15.44:0/1341284806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbca9db0 0x55c3cbcb3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:54.255 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.237+0000 7f1afad9d700 1 --2- 172.21.15.44:0/1341284806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbcab550 0x55c3cbcbaa10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1af400d010 tx=0x7f1af4003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cbc6f153b6d762 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.256 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.237+0000 7f1ae1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:39:54.256 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.237+0000 7f1ae1ffb700 1 -- 172.21.15.44:0/1341284806 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af400e040 con 0x55c3cbcab550 2023-09-10T00:39:54.256 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.237+0000 7f1ae1ffb700 1 -- 172.21.15.44:0/1341284806 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1af4005730 con 0x55c3cbcab550 2023-09-10T00:39:54.256 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.237+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/1341284806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3cbb534a0 con 0x55c3cbcab550 2023-09-10T00:39:54.256 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.237+0000 7f1ae1ffb700 1 -- 172.21.15.44:0/1341284806 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1af40058f0 con 0x55c3cbcab550 2023-09-10T00:39:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.237+0000 7f1b0c9e1b80 10 client.15360 fetch_fsmap learned FSMap version 27 2023-09-10T00:39:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.237+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/1341284806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c3cba10c10 con 0x55c3cbcab550 2023-09-10T00:39:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.237+0000 7f1ae1ffb700 1 -- 172.21.15.44:0/1341284806 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c3cbb534a0 con 0x55c3cbcab550 2023-09-10T00:39:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.237+0000 7f1ae1ffb700 1 -- 172.21.15.44:0/1341284806 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1af4019050 con 0x55c3cbcab550 2023-09-10T00:39:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.237+0000 7f1b0c9e1b80 10 client.15360 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:39:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.237+0000 7f1b0c9e1b80 10 client.15360 resolve_mds: resolved name 'k' to daemon mds.k 2023-09-10T00:39:54.258 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.237+0000 7f1b0c9e1b80 20 client.15360 populate_metadata read hostname 'smithi044' 2023-09-10T00:39:54.258 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.237+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/1341284806 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x55c3cbcbe810 0x55c3cbcbd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.258 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.237+0000 7f1b0c9e1b80 4 client.15360 mds_command: new command op to 24515 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:39:54.258 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.237+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/1341284806 --> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3cbc10850 con 0x55c3cbcbe810 2023-09-10T00:39:54.258 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.238+0000 7f1afa59c700 1 --2- 172.21.15.44:0/1341284806 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x55c3cbcbe810 0x55c3cbcbd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.259 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.238+0000 7f1afa59c700 1 --2- 172.21.15.44:0/1341284806 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x55c3cbcbe810 0x55c3cbcbd940 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9b0a02c87053d208 server_cookie=d46d10fdb73c4dcf in_seq=0 out_seq=0 2023-09-10T00:39:54.259 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.238+0000 7f1ae1ffb700 10 client.15360 ms_handle_connect on v2:172.21.15.99:6832/1934109590 2023-09-10T00:39:54.259 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.239+0000 7f1ae1ffb700 1 -- 172.21.15.44:0/1341284806 <== mds.3 v2:172.21.15.99:6832/1934109590 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c3cbc10850 con 0x55c3cbcbe810 2023-09-10T00:39:54.259 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.239+0000 7f1ae1ffb700 10 client.15360 handle_command_reply: tid=0 2023-09-10T00:39:54.259 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.239+0000 7f1b0c9e1b80 1 client.15360 shutdown 2023-09-10T00:39:54.260 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.239+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/1341284806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbcab550 msgr2=0x55c3cbcbaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.260 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.239+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/1341284806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbcab550 0x55c3cbcbaa10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1af400d010 tx=0x7f1af4003cb0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.260 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.239+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/1341284806 shutdown_connections 2023-09-10T00:39:54.260 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.239+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/1341284806 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x55c3cbcbe810 0x55c3cbcbd940 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.260 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.239+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/1341284806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbcab550 0x55c3cbcbaa10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.261 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.239+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/1341284806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbca9db0 0x55c3cbcb3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.261 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.239+0000 7f1ae1ffb700 0 client.15360 ms_handle_reset on v2:172.21.15.99:6832/1934109590 2023-09-10T00:39:54.261 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.239+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/1341284806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcb39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.261 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.239+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/1341284806 >> 172.21.15.44:0/1341284806 conn(0x55c3cbc1e040 msgr2=0x55c3cbca3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.261 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.239+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/1341284806 shutdown_connections 2023-09-10T00:39:54.261 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.239+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/1341284806 wait complete. 2023-09-10T00:39:54.262 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.239+0000 7f1b0c9e1b80 20 client.15360 trim_cache size 0 max 16384 2023-09-10T00:39:54.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.257+0000 7f1b0c9e1b80 1 Processor -- start 2023-09-10T00:39:54.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.257+0000 7f1b0c9e1b80 1 -- start start 2023-09-10T00:39:54.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.257+0000 7f1b0c9e1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbca9db0 0x55c3cbca96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.257+0000 7f1b0c9e1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcaa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.257+0000 7f1b0c9e1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbcab550 0x55c3cbcbb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.257+0000 7f1b0c9e1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3cbcb4390 con 0x55c3cbca9db0 2023-09-10T00:39:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.257+0000 7f1b0c9e1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3cbc10850 con 0x55c3cbcab550 2023-09-10T00:39:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.257+0000 7f1b0c9e1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3cbb534a0 con 0x55c3cbcaabb0 2023-09-10T00:39:54.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.258+0000 7f1afa59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbca9db0 0x55c3cbca96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.258+0000 7f1afa59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbca9db0 0x55c3cbca96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47592/0 (socket says 172.21.15.44:47592) 2023-09-10T00:39:54.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.258+0000 7f1afa59c700 1 -- 172.21.15.44:0/890042058 learned_addr learned my addr 172.21.15.44:0/890042058 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.258+0000 7f1afad9d700 1 --2- 172.21.15.44:0/890042058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbcab550 0x55c3cbcbb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.258+0000 7f1af9d9b700 1 --2- 172.21.15.44:0/890042058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcaa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.258+0000 7f1afa59c700 1 -- 172.21.15.44:0/890042058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbcaabb0 msgr2=0x55c3cbcaa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.258+0000 7f1afa59c700 1 --2- 172.21.15.44:0/890042058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcaa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.258+0000 7f1afa59c700 1 -- 172.21.15.44:0/890042058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbcab550 msgr2=0x55c3cbcbb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.258+0000 7f1afa59c700 1 --2- 172.21.15.44:0/890042058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbcab550 0x55c3cbcbb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.258+0000 7f1afa59c700 1 -- 172.21.15.44:0/890042058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3cba4e200 con 0x55c3cbca9db0 2023-09-10T00:39:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.258+0000 7f1afa59c700 1 --2- 172.21.15.44:0/890042058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbca9db0 0x55c3cbca96c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00b6c0 tx=0x7f1aec013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ee656d52d3d992 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.258+0000 7f1af17fa700 1 -- 172.21.15.44:0/890042058 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec02a070 con 0x55c3cbca9db0 2023-09-10T00:39:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.258+0000 7f1af17fa700 1 -- 172.21.15.44:0/890042058 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1aec015070 con 0x55c3cbca9db0 2023-09-10T00:39:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.258+0000 7f1af17fa700 1 -- 172.21.15.44:0/890042058 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec024070 con 0x55c3cbca9db0 2023-09-10T00:39:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/890042058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbca9db0 msgr2=0x55c3cbca96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/890042058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbca9db0 0x55c3cbca96c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00b6c0 tx=0x7f1aec013000 comp rx=0 tx=0).stop 2023-09-10T00:39:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/890042058 shutdown_connections 2023-09-10T00:39:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/890042058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbcab550 0x55c3cbcbb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/890042058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbca9db0 0x55c3cbca96c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/890042058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcaa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/890042058 >> 172.21.15.44:0/890042058 conn(0x55c3cbc1e040 msgr2=0x55c3cbca4640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/890042058 shutdown_connections 2023-09-10T00:39:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/890042058 wait complete. 2023-09-10T00:39:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 -- start start 2023-09-10T00:39:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 Processor -- start 2023-09-10T00:39:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbca9db0 0x55c3cbcb0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcb0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbcab550 0x55c3cbd16880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3cbd16df0 con 0x55c3cbcaabb0 2023-09-10T00:39:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3cbd16f60 con 0x55c3cbca9db0 2023-09-10T00:39:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3cbd17120 con 0x55c3cbcab550 2023-09-10T00:39:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.259+0000 7f1b0c9e1b80 10 client.0 _ll_register_callbacks cb 0x7f1af4009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:39:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1af9d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcb0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1af9d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcb0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47600/0 (socket says 172.21.15.44:47600) 2023-09-10T00:39:54.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1af9d9b700 1 -- 172.21.15.44:0/3351925791 learned_addr learned my addr 172.21.15.44:0/3351925791 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1afa59c700 1 --2- 172.21.15.44:0/3351925791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbca9db0 0x55c3cbcb0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1afad9d700 1 --2- 172.21.15.44:0/3351925791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbcab550 0x55c3cbd16880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1af9d9b700 1 -- 172.21.15.44:0/3351925791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbcab550 msgr2=0x55c3cbd16880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1af9d9b700 1 --2- 172.21.15.44:0/3351925791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbcab550 0x55c3cbd16880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1af9d9b700 1 -- 172.21.15.44:0/3351925791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbca9db0 msgr2=0x55c3cbcb0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1af9d9b700 1 --2- 172.21.15.44:0/3351925791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbca9db0 0x55c3cbcb0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1af9d9b700 1 -- 172.21.15.44:0/3351925791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aec00bd90 con 0x55c3cbcaabb0 2023-09-10T00:39:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1afa59c700 1 --2- 172.21.15.44:0/3351925791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbca9db0 0x55c3cbcb0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1af9d9b700 1 --2- 172.21.15.44:0/3351925791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcb0640 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4010710 tx=0x7f1ae4097700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e592e7668bb116 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1ae1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:39:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1ae1ffb700 1 -- 172.21.15.44:0/3351925791 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40a11c0 con 0x55c3cbcaabb0 2023-09-10T00:39:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/3351925791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1af4003e90 con 0x55c3cbcaabb0 2023-09-10T00:39:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1ae1ffb700 1 -- 172.21.15.44:0/3351925791 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1ae40995c0 con 0x55c3cbcaabb0 2023-09-10T00:39:54.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1ae1ffb700 1 -- 172.21.15.44:0/3351925791 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ae4098640 con 0x55c3cbcaabb0 2023-09-10T00:39:54.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1b0c9e1b80 10 client.15366 fetch_fsmap learned FSMap version 27 2023-09-10T00:39:54.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/3351925791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c3cba26260 con 0x55c3cbcaabb0 2023-09-10T00:39:54.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.260+0000 7f1ae1ffb700 1 -- 172.21.15.44:0/3351925791 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1af4003e90 con 0x55c3cbcaabb0 2023-09-10T00:39:54.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.261+0000 7f1ae1ffb700 1 -- 172.21.15.44:0/3351925791 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1ae40b1220 con 0x55c3cbcaabb0 2023-09-10T00:39:54.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.261+0000 7f1b0c9e1b80 10 client.15366 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:39:54.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.261+0000 7f1b0c9e1b80 10 client.15366 resolve_mds: resolved name 'k' to daemon mds.k 2023-09-10T00:39:54.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.261+0000 7f1b0c9e1b80 20 client.15366 populate_metadata read hostname 'smithi044' 2023-09-10T00:39:54.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.261+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/3351925791 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x55c3cbd1c390 0x55c3cbd187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.261+0000 7f1b0c9e1b80 4 client.15366 mds_command: new command op to 24515 tid=0[{"prefix": "status"}] 2023-09-10T00:39:54.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.261+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/3351925791 --> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c3cbbf48d0 con 0x55c3cbd1c390 2023-09-10T00:39:54.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.261+0000 7f1afa59c700 1 --2- 172.21.15.44:0/3351925791 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x55c3cbd1c390 0x55c3cbd187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.261+0000 7f1ae1ffb700 10 client.15366 ms_handle_connect on v2:172.21.15.99:6832/1934109590 2023-09-10T00:39:54.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.261+0000 7f1afa59c700 1 --2- 172.21.15.44:0/3351925791 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x55c3cbd1c390 0x55c3cbd187f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=83177781e6f221f1 server_cookie=554b3bda2761391a in_seq=0 out_seq=0 2023-09-10T00:39:54.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1ae1ffb700 1 -- 172.21.15.44:0/3351925791 <== mds.3 v2:172.21.15.99:6832/1934109590 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c3cbbf48d0 con 0x55c3cbd1c390 2023-09-10T00:39:54.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1ae1ffb700 10 client.15366 handle_command_reply: tid=0 2023-09-10T00:39:54.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0c9e1b80 1 client.15366 shutdown 2023-09-10T00:39:54.281 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/3351925791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbcaabb0 msgr2=0x55c3cbcb0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.281 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/3351925791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcb0640 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4010710 tx=0x7f1ae4097700 comp rx=0 tx=0).stop 2023-09-10T00:39:54.281 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/3351925791 shutdown_connections 2023-09-10T00:39:54.281 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/3351925791 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x55c3cbd1c390 0x55c3cbd187f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.281 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/3351925791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3cbcab550 0x55c3cbd16880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/3351925791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3cbca9db0 0x55c3cbcb0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0c9e1b80 1 --2- 172.21.15.44:0/3351925791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3cbcaabb0 0x55c3cbcb0640 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1ae1ffb700 0 client.15366 ms_handle_reset on v2:172.21.15.99:6832/1934109590 2023-09-10T00:39:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/3351925791 >> 172.21.15.44:0/3351925791 conn(0x55c3cbc1e040 msgr2=0x55c3cbcaa190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/3351925791 shutdown_connections 2023-09-10T00:39:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0c9e1b80 1 -- 172.21.15.44:0/3351925791 wait complete. 2023-09-10T00:39:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0c9e1b80 20 client.15366 trim_cache size 0 max 16384 2023-09-10T00:39:54.283 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:39:54.283 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da", 2023-09-10T00:39:54.284 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 3, 2023-09-10T00:39:54.284 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24515, 2023-09-10T00:39:54.284 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2023-09-10T00:39:54.284 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2023-09-10T00:39:54.284 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2023-09-10T00:39:54.285 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 53.798494302999998, 2023-09-10T00:39:54.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2023-09-10T00:39:54.285 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 89, 2023-09-10T00:39:54.285 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 87, 2023-09-10T00:39:54.285 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 103.16516909800001 2023-09-10T00:39:54.285 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:39:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0081b700 1 -- 172.21.15.44:0/2988313677 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1ae80a06d0 msgr2=0x7f1ae80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0081b700 1 --2- 172.21.15.44:0/2988313677 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1ae80a06d0 0x7f1ae80a2b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1afc100540 tx=0x7f1aec00b410 comp rx=0 tx=0).stop 2023-09-10T00:39:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0081b700 1 -- 172.21.15.44:0/2988313677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1afc1006e0 msgr2=0x7f1afc196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0081b700 1 --2- 172.21.15.44:0/2988313677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1afc1006e0 0x7f1afc196840 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400d950 tx=0x7f1ae400dc60 comp rx=0 tx=0).stop 2023-09-10T00:39:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0081b700 1 -- 172.21.15.44:0/2988313677 shutdown_connections 2023-09-10T00:39:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0081b700 1 --2- 172.21.15.44:0/2988313677 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1ae80a06d0 0x7f1ae80a2b90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0081b700 1 --2- 172.21.15.44:0/2988313677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1afc101080 0x7f1afc19d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0081b700 1 --2- 172.21.15.44:0/2988313677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1afc0ff4f0 0x7f1afc196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0081b700 1 --2- 172.21.15.44:0/2988313677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1afc1006e0 0x7f1afc196840 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0081b700 1 -- 172.21.15.44:0/2988313677 >> 172.21.15.44:0/2988313677 conn(0x7f1afc0faa60 msgr2=0x7f1afc0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.262+0000 7f1b0081b700 1 -- 172.21.15.44:0/2988313677 shutdown_connections 2023-09-10T00:39:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.263+0000 7f1b0081b700 1 -- 172.21.15.44:0/2988313677 wait complete. 2023-09-10T00:39:54.289 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-09-10T00:39:54.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.415+0000 7f96e73d4700 1 -- 172.21.15.44:0/2051066323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e01017e0 msgr2=0x7f96e0101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.415+0000 7f96e73d4700 1 --2- 172.21.15.44:0/2051066323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e01017e0 0x7f96e0101c40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0099c0 tx=0x7f96dc009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.415+0000 7f96e73d4700 1 -- 172.21.15.44:0/2051066323 shutdown_connections 2023-09-10T00:39:54.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.415+0000 7f96e73d4700 1 --2- 172.21.15.44:0/2051066323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96e0102180 0x7f96e0108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.415+0000 7f96e73d4700 1 --2- 172.21.15.44:0/2051066323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e01005f0 0x7f96e0100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.415+0000 7f96e73d4700 1 --2- 172.21.15.44:0/2051066323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e01017e0 0x7f96e0101c40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.415+0000 7f96e73d4700 1 -- 172.21.15.44:0/2051066323 >> 172.21.15.44:0/2051066323 conn(0x7f96e00fbb80 msgr2=0x7f96e00fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.415+0000 7f96e73d4700 1 -- 172.21.15.44:0/2051066323 shutdown_connections 2023-09-10T00:39:54.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.415+0000 7f96e73d4700 1 -- 172.21.15.44:0/2051066323 wait complete. 2023-09-10T00:39:54.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.415+0000 7f96e73d4700 1 Processor -- start 2023-09-10T00:39:54.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e73d4700 1 -- start start 2023-09-10T00:39:54.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e73d4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e01005f0 0x7f96e0196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e73d4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e01017e0 0x7f96e0196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e73d4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96e0102180 0x7f96e019d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e73d4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e019dda0 con 0x7f96e01017e0 2023-09-10T00:39:54.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e73d4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e019dee0 con 0x7f96e0102180 2023-09-10T00:39:54.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e73d4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e019e070 con 0x7f96e01005f0 2023-09-10T00:39:54.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e496f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e01017e0 0x7f96e0196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e496f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e01017e0 0x7f96e0196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47614/0 (socket says 172.21.15.44:47614) 2023-09-10T00:39:54.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e496f700 1 -- 172.21.15.44:0/2695784718 learned_addr learned my addr 172.21.15.44:0/2695784718 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e5170700 1 --2- 172.21.15.44:0/2695784718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e01005f0 0x7f96e0196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e5971700 1 --2- 172.21.15.44:0/2695784718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96e0102180 0x7f96e019d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e496f700 1 -- 172.21.15.44:0/2695784718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e01005f0 msgr2=0x7f96e0196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e496f700 1 --2- 172.21.15.44:0/2695784718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e01005f0 0x7f96e0196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e496f700 1 -- 172.21.15.44:0/2695784718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96e0102180 msgr2=0x7f96e019d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e496f700 1 --2- 172.21.15.44:0/2695784718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96e0102180 0x7f96e019d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e496f700 1 -- 172.21.15.44:0/2695784718 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc0095f0 con 0x7f96e01017e0 2023-09-10T00:39:54.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e5170700 1 --2- 172.21.15.44:0/2695784718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e01005f0 0x7f96e0196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:39:54.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.416+0000 7f96e496f700 1 --2- 172.21.15.44:0/2695784718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e01017e0 0x7f96e0196760 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f96dc000c00 tx=0x7f96dc00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.417+0000 7f96d67fc700 1 -- 172.21.15.44:0/2695784718 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc01d070 con 0x7f96e01017e0 2023-09-10T00:39:54.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.417+0000 7f96d67fc700 1 -- 172.21.15.44:0/2695784718 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96dc00fdc0 con 0x7f96e01017e0 2023-09-10T00:39:54.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.417+0000 7f96e73d4700 1 -- 172.21.15.44:0/2695784718 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e019e2f0 con 0x7f96e01017e0 2023-09-10T00:39:54.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.417+0000 7f96d67fc700 1 -- 172.21.15.44:0/2695784718 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc0173f0 con 0x7f96e01017e0 2023-09-10T00:39:54.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.417+0000 7f96e73d4700 1 -- 172.21.15.44:0/2695784718 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e019e710 con 0x7f96e01017e0 2023-09-10T00:39:54.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.419+0000 7f96d67fc700 1 -- 172.21.15.44:0/2695784718 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f96dc022470 con 0x7f96e01017e0 2023-09-10T00:39:54.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.419+0000 7f96d67fc700 1 --2- 172.21.15.44:0/2695784718 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f96cc0a06d0 0x7f96cc0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.419+0000 7f96e5170700 1 --2- 172.21.15.44:0/2695784718 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f96cc0a06d0 0x7f96cc0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.420+0000 7f96d67fc700 1 -- 172.21.15.44:0/2695784718 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f96dc0cd310 con 0x7f96e01017e0 2023-09-10T00:39:54.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.420+0000 7f96e5170700 1 --2- 172.21.15.44:0/2695784718 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f96cc0a06d0 0x7f96cc0a2b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f96e0101640 tx=0x7f96d000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.420+0000 7f96f359ab80 1 Processor -- start 2023-09-10T00:39:54.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.420+0000 7f96f359ab80 1 -- start start 2023-09-10T00:39:54.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96f359ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6370bb0 0x55a5c6370fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96f359ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c6371550 0x55a5c636f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96f359ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c636fdb0 0x55a5c6370260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96f359ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c62c86e0 con 0x55a5c6370bb0 2023-09-10T00:39:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96f359ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c62c6000 con 0x55a5c636fdb0 2023-09-10T00:39:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96f359ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c62c18f0 con 0x55a5c6371550 2023-09-10T00:39:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96e5170700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6370bb0 0x55a5c6370fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96e5170700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6370bb0 0x55a5c6370fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47620/0 (socket says 172.21.15.44:47620) 2023-09-10T00:39:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96e5170700 1 -- 172.21.15.44:0/3040774982 learned_addr learned my addr 172.21.15.44:0/3040774982 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96e5971700 1 --2- 172.21.15.44:0/3040774982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c636fdb0 0x55a5c6370260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96e496f700 1 --2- 172.21.15.44:0/3040774982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c6371550 0x55a5c636f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96e5170700 1 -- 172.21.15.44:0/3040774982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c6371550 msgr2=0x55a5c636f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96e5170700 1 --2- 172.21.15.44:0/3040774982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c6371550 0x55a5c636f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96e5170700 1 -- 172.21.15.44:0/3040774982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c636fdb0 msgr2=0x55a5c6370260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96e5170700 1 --2- 172.21.15.44:0/3040774982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c636fdb0 0x55a5c6370260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96e5170700 1 -- 172.21.15.44:0/3040774982 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5c60edd90 con 0x55a5c6370bb0 2023-09-10T00:39:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96e5971700 1 --2- 172.21.15.44:0/3040774982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c636fdb0 0x55a5c6370260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:39:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96e5170700 1 --2- 172.21.15.44:0/3040774982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6370bb0 0x55a5c6370fd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f96d0004350 tx=0x7f96d0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bf41e5dc032f80 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96cb7fe700 1 -- 172.21.15.44:0/3040774982 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0003a90 con 0x55a5c6370bb0 2023-09-10T00:39:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96cb7fe700 1 -- 172.21.15.44:0/3040774982 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96d0003bf0 con 0x55a5c6370bb0 2023-09-10T00:39:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.421+0000 7f96cb7fe700 1 -- 172.21.15.44:0/3040774982 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d001f430 con 0x55a5c6370bb0 2023-09-10T00:39:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.422+0000 7f96f359ab80 1 -- 172.21.15.44:0/3040774982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6370bb0 msgr2=0x55a5c6370fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.422+0000 7f96f359ab80 1 --2- 172.21.15.44:0/3040774982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6370bb0 0x55a5c6370fd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f96d0004350 tx=0x7f96d0013000 comp rx=0 tx=0).stop 2023-09-10T00:39:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.422+0000 7f96f359ab80 1 -- 172.21.15.44:0/3040774982 shutdown_connections 2023-09-10T00:39:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.422+0000 7f96f359ab80 1 --2- 172.21.15.44:0/3040774982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c636fdb0 0x55a5c6370260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.422+0000 7f96f359ab80 1 --2- 172.21.15.44:0/3040774982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6370bb0 0x55a5c6370fd0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.422+0000 7f96f359ab80 1 --2- 172.21.15.44:0/3040774982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c6371550 0x55a5c636f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.422+0000 7f96f359ab80 1 -- 172.21.15.44:0/3040774982 >> 172.21.15.44:0/3040774982 conn(0x55a5c62e4040 msgr2=0x55a5c636a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.422+0000 7f96f359ab80 1 -- 172.21.15.44:0/3040774982 shutdown_connections 2023-09-10T00:39:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.422+0000 7f96f359ab80 1 -- 172.21.15.44:0/3040774982 wait complete. 2023-09-10T00:39:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.422+0000 7f96f359ab80 1 -- start start 2023-09-10T00:39:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96f359ab80 1 Processor -- start 2023-09-10T00:39:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96f359ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c636fdb0 0x55a5c6379600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96f359ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6370bb0 0x55a5c6379b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96f359ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c6371550 0x55a5c6380b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96f359ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c625fb20 con 0x55a5c6370bb0 2023-09-10T00:39:54.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96f359ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c625f820 con 0x55a5c636fdb0 2023-09-10T00:39:54.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96f359ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c61438d0 con 0x55a5c6371550 2023-09-10T00:39:54.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96f359ab80 10 client.0 _ll_register_callbacks cb 0x55a5c61a9a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:39:54.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96e496f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6370bb0 0x55a5c6379b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96e496f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6370bb0 0x55a5c6379b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47628/0 (socket says 172.21.15.44:47628) 2023-09-10T00:39:54.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96e496f700 1 -- 172.21.15.44:0/2477931725 learned_addr learned my addr 172.21.15.44:0/2477931725 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96e5170700 1 --2- 172.21.15.44:0/2477931725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c636fdb0 0x55a5c6379600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96e5971700 1 --2- 172.21.15.44:0/2477931725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c6371550 0x55a5c6380b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96e496f700 1 -- 172.21.15.44:0/2477931725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c6371550 msgr2=0x55a5c6380b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96e496f700 1 --2- 172.21.15.44:0/2477931725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c6371550 0x55a5c6380b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96e496f700 1 -- 172.21.15.44:0/2477931725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c636fdb0 msgr2=0x55a5c6379600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96e496f700 1 --2- 172.21.15.44:0/2477931725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c636fdb0 0x55a5c6379600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.423+0000 7f96e496f700 1 -- 172.21.15.44:0/2477931725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d0004030 con 0x55a5c6370bb0 2023-09-10T00:39:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.424+0000 7f96e496f700 1 --2- 172.21.15.44:0/2477931725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6370bb0 0x55a5c6379b40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f96dc006010 tx=0x7f96dc022a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c423c416ffe518 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.424+0000 7f96c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:39:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.424+0000 7f96c8ff9700 1 -- 172.21.15.44:0/2477931725 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc09af40 con 0x55a5c6370bb0 2023-09-10T00:39:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.424+0000 7f96c8ff9700 1 -- 172.21.15.44:0/2477931725 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f96dc09a840 con 0x55a5c6370bb0 2023-09-10T00:39:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.424+0000 7f96f359ab80 1 -- 172.21.15.44:0/2477931725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5c62194a0 con 0x55a5c6370bb0 2023-09-10T00:39:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.424+0000 7f96c8ff9700 1 -- 172.21.15.44:0/2477931725 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96dc09e670 con 0x55a5c6370bb0 2023-09-10T00:39:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.424+0000 7f96f359ab80 10 client.15378 fetch_fsmap learned FSMap version 27 2023-09-10T00:39:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.424+0000 7f96f359ab80 1 -- 172.21.15.44:0/2477931725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a5c60d6c10 con 0x55a5c6370bb0 2023-09-10T00:39:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.424+0000 7f96c8ff9700 1 -- 172.21.15.44:0/2477931725 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a5c62194a0 con 0x55a5c6370bb0 2023-09-10T00:39:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.424+0000 7f96c8ff9700 1 -- 172.21.15.44:0/2477931725 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f96dc0a2390 con 0x55a5c6370bb0 2023-09-10T00:39:54.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.424+0000 7f96f359ab80 10 client.15378 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:39:54.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.424+0000 7f96f359ab80 10 client.15378 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-10T00:39:54.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.424+0000 7f96f359ab80 20 client.15378 populate_metadata read hostname 'smithi044' 2023-09-10T00:39:54.445 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.424+0000 7f96f359ab80 1 --2- 172.21.15.44:0/2477931725 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55a5c63848d0 0x55a5c6383a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.445 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.424+0000 7f96f359ab80 4 client.15378 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:39:54.445 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.424+0000 7f96f359ab80 1 -- 172.21.15.44:0/2477931725 --> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5c62d6850 con 0x55a5c63848d0 2023-09-10T00:39:54.445 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.425+0000 7f96e5170700 1 --2- 172.21.15.44:0/2477931725 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55a5c63848d0 0x55a5c6383a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.445 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.425+0000 7f96c8ff9700 10 client.15378 ms_handle_connect on v2:172.21.15.99:6836/566283638 2023-09-10T00:39:54.446 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.425+0000 7f96e5170700 1 --2- 172.21.15.44:0/2477931725 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55a5c63848d0 0x55a5c6383a20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=adb271f085e5c061 server_cookie=2d8e4308f76dada1 in_seq=0 out_seq=0 2023-09-10T00:39:54.446 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.426+0000 7f96c8ff9700 1 -- 172.21.15.44:0/2477931725 <== mds.2 v2:172.21.15.99:6836/566283638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a5c62d6850 con 0x55a5c63848d0 2023-09-10T00:39:54.446 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.426+0000 7f96c8ff9700 10 client.15378 handle_command_reply: tid=0 2023-09-10T00:39:54.446 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.426+0000 7f96f359ab80 1 client.15378 shutdown 2023-09-10T00:39:54.446 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.426+0000 7f96f359ab80 1 -- 172.21.15.44:0/2477931725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6370bb0 msgr2=0x55a5c6379b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.447 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.426+0000 7f96f359ab80 1 --2- 172.21.15.44:0/2477931725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6370bb0 0x55a5c6379b40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f96dc006010 tx=0x7f96dc022a00 comp rx=0 tx=0).stop 2023-09-10T00:39:54.447 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.426+0000 7f96f359ab80 1 -- 172.21.15.44:0/2477931725 shutdown_connections 2023-09-10T00:39:54.447 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.426+0000 7f96f359ab80 1 --2- 172.21.15.44:0/2477931725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c6371550 0x55a5c6380b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.447 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.426+0000 7f96f359ab80 1 --2- 172.21.15.44:0/2477931725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c636fdb0 0x55a5c6379600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.447 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.426+0000 7f96f359ab80 1 --2- 172.21.15.44:0/2477931725 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55a5c63848d0 0x55a5c6383a20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.426+0000 7f96f359ab80 1 --2- 172.21.15.44:0/2477931725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6370bb0 0x55a5c6379b40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.426+0000 7f96c8ff9700 0 client.15378 ms_handle_reset on v2:172.21.15.99:6836/566283638 2023-09-10T00:39:54.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.426+0000 7f96f359ab80 1 -- 172.21.15.44:0/2477931725 >> 172.21.15.44:0/2477931725 conn(0x55a5c62e4040 msgr2=0x55a5c6369d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.426+0000 7f96f359ab80 1 -- 172.21.15.44:0/2477931725 shutdown_connections 2023-09-10T00:39:54.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.426+0000 7f96f359ab80 1 -- 172.21.15.44:0/2477931725 wait complete. 2023-09-10T00:39:54.449 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.426+0000 7f96f359ab80 20 client.15378 trim_cache size 0 max 16384 2023-09-10T00:39:54.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96f359ab80 1 Processor -- start 2023-09-10T00:39:54.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96f359ab80 1 -- start start 2023-09-10T00:39:54.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96f359ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c636fdb0 0x55a5c6377420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96f359ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c6370bb0 0x55a5c6380b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96f359ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6371550 0x55a5c6373f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96f359ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c62d6850 con 0x55a5c6371550 2023-09-10T00:39:54.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96f359ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c62194a0 con 0x55a5c6370bb0 2023-09-10T00:39:54.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96f359ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c62694d0 con 0x55a5c636fdb0 2023-09-10T00:39:54.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96e5971700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6371550 0x55a5c6373f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96e5971700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6371550 0x55a5c6373f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47638/0 (socket says 172.21.15.44:47638) 2023-09-10T00:39:54.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96e5971700 1 -- 172.21.15.44:0/384621860 learned_addr learned my addr 172.21.15.44:0/384621860 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96e5170700 1 --2- 172.21.15.44:0/384621860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c636fdb0 0x55a5c6377420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96e496f700 1 --2- 172.21.15.44:0/384621860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c6370bb0 0x55a5c6380b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96e5971700 1 -- 172.21.15.44:0/384621860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c636fdb0 msgr2=0x55a5c6377420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96e5971700 1 --2- 172.21.15.44:0/384621860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c636fdb0 0x55a5c6377420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96e5971700 1 -- 172.21.15.44:0/384621860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c6370bb0 msgr2=0x55a5c6380b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96e5971700 1 --2- 172.21.15.44:0/384621860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c6370bb0 0x55a5c6380b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96e5971700 1 -- 172.21.15.44:0/384621860 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5c6114200 con 0x55a5c6371550 2023-09-10T00:39:54.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.444+0000 7f96e496f700 1 --2- 172.21.15.44:0/384621860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c6370bb0 0x55a5c6380b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:54.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.445+0000 7f96e5971700 1 --2- 172.21.15.44:0/384621860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6371550 0x55a5c6373f60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f96d8007530 tx=0x7f96d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f79ebdfbd286d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.445+0000 7f96cbfff700 1 -- 172.21.15.44:0/384621860 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d800e040 con 0x55a5c6371550 2023-09-10T00:39:54.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.445+0000 7f96cbfff700 1 -- 172.21.15.44:0/384621860 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96d800f040 con 0x55a5c6371550 2023-09-10T00:39:54.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.445+0000 7f96cbfff700 1 -- 172.21.15.44:0/384621860 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8017670 con 0x55a5c6371550 2023-09-10T00:39:54.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.445+0000 7f96f359ab80 1 -- 172.21.15.44:0/384621860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6371550 msgr2=0x55a5c6373f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.445+0000 7f96f359ab80 1 --2- 172.21.15.44:0/384621860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6371550 0x55a5c6373f60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f96d8007530 tx=0x7f96d800b040 comp rx=0 tx=0).stop 2023-09-10T00:39:54.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.445+0000 7f96f359ab80 1 -- 172.21.15.44:0/384621860 shutdown_connections 2023-09-10T00:39:54.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.445+0000 7f96f359ab80 1 --2- 172.21.15.44:0/384621860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6371550 0x55a5c6373f60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.445+0000 7f96f359ab80 1 --2- 172.21.15.44:0/384621860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c636fdb0 0x55a5c6377420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.445+0000 7f96f359ab80 1 --2- 172.21.15.44:0/384621860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c6370bb0 0x55a5c6380b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.445+0000 7f96f359ab80 1 -- 172.21.15.44:0/384621860 >> 172.21.15.44:0/384621860 conn(0x55a5c62e4040 msgr2=0x55a5c6378280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.445+0000 7f96f359ab80 1 -- 172.21.15.44:0/384621860 shutdown_connections 2023-09-10T00:39:54.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.445+0000 7f96f359ab80 1 -- 172.21.15.44:0/384621860 wait complete. 2023-09-10T00:39:54.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96f359ab80 1 -- start start 2023-09-10T00:39:54.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96f359ab80 1 Processor -- start 2023-09-10T00:39:54.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96f359ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c636fdb0 0x55a5c637d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96f359ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c6370bb0 0x55a5c637dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96f359ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6371550 0x55a5c63d8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96f359ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c637e580 con 0x55a5c6371550 2023-09-10T00:39:54.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96f359ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c63d8e60 con 0x55a5c6370bb0 2023-09-10T00:39:54.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96f359ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c63d8fd0 con 0x55a5c636fdb0 2023-09-10T00:39:54.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96f359ab80 10 client.0 _ll_register_callbacks cb 0x7f96dc096c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:39:54.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96e5971700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6371550 0x55a5c63d8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96e5971700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6371550 0x55a5c63d8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47644/0 (socket says 172.21.15.44:47644) 2023-09-10T00:39:54.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96e5971700 1 -- 172.21.15.44:0/4025578415 learned_addr learned my addr 172.21.15.44:0/4025578415 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96e496f700 1 --2- 172.21.15.44:0/4025578415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c6370bb0 0x55a5c637dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96e5170700 1 --2- 172.21.15.44:0/4025578415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c636fdb0 0x55a5c637d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96e5971700 1 -- 172.21.15.44:0/4025578415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c636fdb0 msgr2=0x55a5c637d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96e5971700 1 --2- 172.21.15.44:0/4025578415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c636fdb0 0x55a5c637d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96e5971700 1 -- 172.21.15.44:0/4025578415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c6370bb0 msgr2=0x55a5c637dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96e5971700 1 --2- 172.21.15.44:0/4025578415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c6370bb0 0x55a5c637dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.446+0000 7f96e5971700 1 -- 172.21.15.44:0/4025578415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d8009220 con 0x55a5c6371550 2023-09-10T00:39:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.447+0000 7f96e5971700 1 --2- 172.21.15.44:0/4025578415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6371550 0x55a5c63d8920 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009010 tx=0x7f96d8005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a19afa0b057079 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.447+0000 7f96c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:39:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.447+0000 7f96c8ff9700 1 -- 172.21.15.44:0/4025578415 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d800e070 con 0x55a5c6371550 2023-09-10T00:39:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.447+0000 7f96c8ff9700 1 -- 172.21.15.44:0/4025578415 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f96d80056a0 con 0x55a5c6371550 2023-09-10T00:39:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.447+0000 7f96f359ab80 1 -- 172.21.15.44:0/4025578415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96d0003bf0 con 0x55a5c6371550 2023-09-10T00:39:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.447+0000 7f96c8ff9700 1 -- 172.21.15.44:0/4025578415 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96d8024070 con 0x55a5c6371550 2023-09-10T00:39:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.447+0000 7f96f359ab80 10 client.15384 fetch_fsmap learned FSMap version 27 2023-09-10T00:39:54.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.447+0000 7f96f359ab80 1 -- 172.21.15.44:0/4025578415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a5c60ec260 con 0x55a5c6371550 2023-09-10T00:39:54.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.447+0000 7f96c8ff9700 1 -- 172.21.15.44:0/4025578415 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96d0003bf0 con 0x55a5c6371550 2023-09-10T00:39:54.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.447+0000 7f96c8ff9700 1 -- 172.21.15.44:0/4025578415 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f96d8017410 con 0x55a5c6371550 2023-09-10T00:39:54.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.447+0000 7f96f359ab80 10 client.15384 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:39:54.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.447+0000 7f96f359ab80 10 client.15384 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-10T00:39:54.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.447+0000 7f96f359ab80 20 client.15384 populate_metadata read hostname 'smithi044' 2023-09-10T00:39:54.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.447+0000 7f96f359ab80 1 --2- 172.21.15.44:0/4025578415 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55a5c63de210 0x55a5c63da670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.447+0000 7f96f359ab80 4 client.15384 mds_command: new command op to 24572 tid=0[{"prefix": "status"}] 2023-09-10T00:39:54.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.447+0000 7f96f359ab80 1 -- 172.21.15.44:0/4025578415 --> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a5c62ba8d0 con 0x55a5c63de210 2023-09-10T00:39:54.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96e5170700 1 --2- 172.21.15.44:0/4025578415 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55a5c63de210 0x55a5c63da670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96c8ff9700 10 client.15384 ms_handle_connect on v2:172.21.15.99:6836/566283638 2023-09-10T00:39:54.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96e5170700 1 --2- 172.21.15.44:0/4025578415 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55a5c63de210 0x55a5c63da670 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b836922cac961228 server_cookie=41339ca7c2164865 in_seq=0 out_seq=0 2023-09-10T00:39:54.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96c8ff9700 1 -- 172.21.15.44:0/4025578415 <== mds.2 v2:172.21.15.99:6836/566283638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a5c62ba8d0 con 0x55a5c63de210 2023-09-10T00:39:54.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96c8ff9700 10 client.15384 handle_command_reply: tid=0 2023-09-10T00:39:54.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96f359ab80 1 client.15384 shutdown 2023-09-10T00:39:54.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96f359ab80 1 -- 172.21.15.44:0/4025578415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6371550 msgr2=0x55a5c63d8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96f359ab80 1 --2- 172.21.15.44:0/4025578415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6371550 0x55a5c63d8920 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009010 tx=0x7f96d8005270 comp rx=0 tx=0).stop 2023-09-10T00:39:54.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96f359ab80 1 -- 172.21.15.44:0/4025578415 shutdown_connections 2023-09-10T00:39:54.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96f359ab80 1 --2- 172.21.15.44:0/4025578415 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55a5c63de210 0x55a5c63da670 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96f359ab80 1 --2- 172.21.15.44:0/4025578415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a5c6371550 0x55a5c63d8920 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96f359ab80 1 --2- 172.21.15.44:0/4025578415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5c636fdb0 0x55a5c637d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96f359ab80 1 --2- 172.21.15.44:0/4025578415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5c6370bb0 0x55a5c637dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96f359ab80 1 -- 172.21.15.44:0/4025578415 >> 172.21.15.44:0/4025578415 conn(0x55a5c62e4040 msgr2=0x55a5c6378280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96c8ff9700 0 client.15384 ms_handle_reset on v2:172.21.15.99:6836/566283638 2023-09-10T00:39:54.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96f359ab80 1 -- 172.21.15.44:0/4025578415 shutdown_connections 2023-09-10T00:39:54.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.448+0000 7f96f359ab80 1 -- 172.21.15.44:0/4025578415 wait complete. 2023-09-10T00:39:54.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.449+0000 7f96f359ab80 20 client.15384 trim_cache size 0 max 16384 2023-09-10T00:39:54.470 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:39:54.470 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da", 2023-09-10T00:39:54.470 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 2, 2023-09-10T00:39:54.471 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24572, 2023-09-10T00:39:54.471 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2023-09-10T00:39:54.471 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2023-09-10T00:39:54.471 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2023-09-10T00:39:54.471 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 54.606325716000001, 2023-09-10T00:39:54.472 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2023-09-10T00:39:54.472 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 89, 2023-09-10T00:39:54.472 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 87, 2023-09-10T00:39:54.472 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 89.871403567000002 2023-09-10T00:39:54.472 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:39:54.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.449+0000 7f96e73d4700 1 -- 172.21.15.44:0/2695784718 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f96cc0a06d0 msgr2=0x7f96cc0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.449+0000 7f96e73d4700 1 --2- 172.21.15.44:0/2695784718 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f96cc0a06d0 0x7f96cc0a2b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f96e0101640 tx=0x7f96d000b410 comp rx=0 tx=0).stop 2023-09-10T00:39:54.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.449+0000 7f96e73d4700 1 -- 172.21.15.44:0/2695784718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e01017e0 msgr2=0x7f96e0196760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.449+0000 7f96e73d4700 1 --2- 172.21.15.44:0/2695784718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e01017e0 0x7f96e0196760 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f96dc000c00 tx=0x7f96dc00fa00 comp rx=0 tx=0).stop 2023-09-10T00:39:54.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.449+0000 7f96e73d4700 1 -- 172.21.15.44:0/2695784718 shutdown_connections 2023-09-10T00:39:54.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.449+0000 7f96e73d4700 1 --2- 172.21.15.44:0/2695784718 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f96cc0a06d0 0x7f96cc0a2b90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.449+0000 7f96e73d4700 1 --2- 172.21.15.44:0/2695784718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96e0102180 0x7f96e019d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.449+0000 7f96e73d4700 1 --2- 172.21.15.44:0/2695784718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e01005f0 0x7f96e0196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.449+0000 7f96e73d4700 1 --2- 172.21.15.44:0/2695784718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e01017e0 0x7f96e0196760 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.449+0000 7f96e73d4700 1 -- 172.21.15.44:0/2695784718 >> 172.21.15.44:0/2695784718 conn(0x7f96e00fbb80 msgr2=0x7f96e00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.449+0000 7f96e73d4700 1 -- 172.21.15.44:0/2695784718 shutdown_connections 2023-09-10T00:39:54.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.449+0000 7f96e73d4700 1 -- 172.21.15.44:0/2695784718 wait complete. 2023-09-10T00:39:54.476 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-09-10T00:39:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.602+0000 7f9a40deb700 1 -- 172.21.15.44:0/3860394413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c100630 msgr2=0x7f9a3c100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.602+0000 7f9a40deb700 1 --2- 172.21.15.44:0/3860394413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c100630 0x7f9a3c100a50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9a300099c0 tx=0x7f9a30009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.602+0000 7f9a40deb700 1 -- 172.21.15.44:0/3860394413 shutdown_connections 2023-09-10T00:39:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.602+0000 7f9a40deb700 1 --2- 172.21.15.44:0/3860394413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a3c1021c0 0x7f9a3c108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.602+0000 7f9a40deb700 1 --2- 172.21.15.44:0/3860394413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c100630 0x7f9a3c100a50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.602+0000 7f9a40deb700 1 --2- 172.21.15.44:0/3860394413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a3c101820 0x7f9a3c101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.602+0000 7f9a40deb700 1 -- 172.21.15.44:0/3860394413 >> 172.21.15.44:0/3860394413 conn(0x7f9a3c0fbb80 msgr2=0x7f9a3c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.603+0000 7f9a40deb700 1 -- 172.21.15.44:0/3860394413 shutdown_connections 2023-09-10T00:39:54.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.603+0000 7f9a40deb700 1 -- 172.21.15.44:0/3860394413 wait complete. 2023-09-10T00:39:54.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.603+0000 7f9a40deb700 1 Processor -- start 2023-09-10T00:39:54.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a40deb700 1 -- start start 2023-09-10T00:39:54.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a40deb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a3c100630 0x7f9a3c1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a40deb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c101820 0x7f9a3c1967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a40deb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a3c1021c0 0x7f9a3c19d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a40deb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c19de20 con 0x7f9a3c101820 2023-09-10T00:39:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a40deb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c19df60 con 0x7f9a3c100630 2023-09-10T00:39:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a40deb700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c19e0f0 con 0x7f9a3c1021c0 2023-09-10T00:39:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a39d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c101820 0x7f9a3c1967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a39d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c101820 0x7f9a3c1967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47656/0 (socket says 172.21.15.44:47656) 2023-09-10T00:39:54.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a39d9b700 1 -- 172.21.15.44:0/4226151618 learned_addr learned my addr 172.21.15.44:0/4226151618 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a3a59c700 1 --2- 172.21.15.44:0/4226151618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a3c100630 0x7f9a3c1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a3ad9d700 1 --2- 172.21.15.44:0/4226151618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a3c1021c0 0x7f9a3c19d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a39d9b700 1 -- 172.21.15.44:0/4226151618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a3c1021c0 msgr2=0x7f9a3c19d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a39d9b700 1 --2- 172.21.15.44:0/4226151618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a3c1021c0 0x7f9a3c19d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a39d9b700 1 -- 172.21.15.44:0/4226151618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a3c100630 msgr2=0x7f9a3c1962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a39d9b700 1 --2- 172.21.15.44:0/4226151618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a3c100630 0x7f9a3c1962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a39d9b700 1 -- 172.21.15.44:0/4226151618 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a300095f0 con 0x7f9a3c101820 2023-09-10T00:39:54.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a3a59c700 1 --2- 172.21.15.44:0/4226151618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a3c100630 0x7f9a3c1962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:39:54.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.604+0000 7f9a39d9b700 1 --2- 172.21.15.44:0/4226151618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c101820 0x7f9a3c1967e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00d950 tx=0x7f9a2c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.605+0000 7f9a2b7fe700 1 -- 172.21.15.44:0/4226151618 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c00f890 con 0x7f9a3c101820 2023-09-10T00:39:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.605+0000 7f9a2b7fe700 1 -- 172.21.15.44:0/4226151618 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9a2c00fe80 con 0x7f9a3c101820 2023-09-10T00:39:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.605+0000 7f9a40deb700 1 -- 172.21.15.44:0/4226151618 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3c19e3d0 con 0x7f9a3c101820 2023-09-10T00:39:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.605+0000 7f9a2b7fe700 1 -- 172.21.15.44:0/4226151618 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c00e3b0 con 0x7f9a3c101820 2023-09-10T00:39:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.605+0000 7f9a40deb700 1 -- 172.21.15.44:0/4226151618 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3c19e930 con 0x7f9a3c101820 2023-09-10T00:39:54.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.607+0000 7f9a2b7fe700 1 -- 172.21.15.44:0/4226151618 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f9a2c010460 con 0x7f9a3c101820 2023-09-10T00:39:54.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.608+0000 7f9a2b7fe700 1 --2- 172.21.15.44:0/4226151618 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a240a0750 0x7f9a240a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.608+0000 7f9a3a59c700 1 --2- 172.21.15.44:0/4226151618 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a240a0750 0x7f9a240a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.608+0000 7f9a2b7fe700 1 -- 172.21.15.44:0/4226151618 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9a2c0cbd90 con 0x7f9a3c101820 2023-09-10T00:39:54.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.608+0000 7f9a3a59c700 1 --2- 172.21.15.44:0/4226151618 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a240a0750 0x7f9a240a2c10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9a30006010 tx=0x7f9a30005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.608+0000 7f9a4cfb1b80 1 Processor -- start 2023-09-10T00:39:54.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a4cfb1b80 1 -- start start 2023-09-10T00:39:54.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a4cfb1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed78bb0 0x55e47ed78fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a4cfb1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed79550 0x55e47ed777b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a4cfb1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed77db0 0x55e47ed78260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a4cfb1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47ecd06e0 con 0x55e47ed79550 2023-09-10T00:39:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a4cfb1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47ecce000 con 0x55e47ed78bb0 2023-09-10T00:39:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a4cfb1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47ecc98f0 con 0x55e47ed77db0 2023-09-10T00:39:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a39d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed79550 0x55e47ed777b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a39d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed79550 0x55e47ed777b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47664/0 (socket says 172.21.15.44:47664) 2023-09-10T00:39:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a39d9b700 1 -- 172.21.15.44:0/986953768 learned_addr learned my addr 172.21.15.44:0/986953768 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a3ad9d700 1 --2- 172.21.15.44:0/986953768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed77db0 0x55e47ed78260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a3a59c700 1 --2- 172.21.15.44:0/986953768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed78bb0 0x55e47ed78fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a39d9b700 1 -- 172.21.15.44:0/986953768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed77db0 msgr2=0x55e47ed78260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a39d9b700 1 --2- 172.21.15.44:0/986953768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed77db0 0x55e47ed78260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a39d9b700 1 -- 172.21.15.44:0/986953768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed78bb0 msgr2=0x55e47ed78fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a39d9b700 1 --2- 172.21.15.44:0/986953768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed78bb0 0x55e47ed78fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a39d9b700 1 -- 172.21.15.44:0/986953768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e47eaf5d90 con 0x55e47ed79550 2023-09-10T00:39:54.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a3a59c700 1 --2- 172.21.15.44:0/986953768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed78bb0 0x55e47ed78fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:39:54.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a39d9b700 1 --2- 172.21.15.44:0/986953768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed79550 0x55e47ed777b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c095610 tx=0x7f9a2c098d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671025730dd65e34 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.609+0000 7f9a28ff9700 1 -- 172.21.15.44:0/986953768 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c099110 con 0x55e47ed79550 2023-09-10T00:39:54.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.610+0000 7f9a28ff9700 1 -- 172.21.15.44:0/986953768 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9a2c09cf40 con 0x55e47ed79550 2023-09-10T00:39:54.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.610+0000 7f9a28ff9700 1 -- 172.21.15.44:0/986953768 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c098800 con 0x55e47ed79550 2023-09-10T00:39:54.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.610+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/986953768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed79550 msgr2=0x55e47ed777b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.610+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/986953768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed79550 0x55e47ed777b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c095610 tx=0x7f9a2c098d80 comp rx=0 tx=0).stop 2023-09-10T00:39:54.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.610+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/986953768 shutdown_connections 2023-09-10T00:39:54.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.610+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/986953768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed77db0 0x55e47ed78260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.610+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/986953768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed78bb0 0x55e47ed78fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.610+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/986953768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed79550 0x55e47ed777b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.610+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/986953768 >> 172.21.15.44:0/986953768 conn(0x55e47ecec040 msgr2=0x55e47ed726d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.610+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/986953768 shutdown_connections 2023-09-10T00:39:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.610+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/986953768 wait complete. 2023-09-10T00:39:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.611+0000 7f9a4cfb1b80 1 -- start start 2023-09-10T00:39:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.611+0000 7f9a4cfb1b80 1 Processor -- start 2023-09-10T00:39:54.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.611+0000 7f9a4cfb1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed77db0 0x55e47ed815e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.611+0000 7f9a4cfb1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed78bb0 0x55e47ed81b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.611+0000 7f9a4cfb1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed79550 0x55e47ed88b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.611+0000 7f9a4cfb1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47ec67b20 con 0x55e47ed78bb0 2023-09-10T00:39:54.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.611+0000 7f9a4cfb1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47ec67820 con 0x55e47ed77db0 2023-09-10T00:39:54.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.611+0000 7f9a4cfb1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47eb4b8d0 con 0x55e47ed79550 2023-09-10T00:39:54.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.611+0000 7f9a4cfb1b80 10 client.0 _ll_register_callbacks cb 0x55e47ebb1a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:39:54.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.611+0000 7f9a39d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed78bb0 0x55e47ed81b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.611+0000 7f9a39d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed78bb0 0x55e47ed81b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47670/0 (socket says 172.21.15.44:47670) 2023-09-10T00:39:54.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.611+0000 7f9a39d9b700 1 -- 172.21.15.44:0/655388072 learned_addr learned my addr 172.21.15.44:0/655388072 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.611+0000 7f9a3ad9d700 1 --2- 172.21.15.44:0/655388072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed79550 0x55e47ed88b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.611+0000 7f9a3a59c700 1 --2- 172.21.15.44:0/655388072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed77db0 0x55e47ed815e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.612+0000 7f9a39d9b700 1 -- 172.21.15.44:0/655388072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed79550 msgr2=0x55e47ed88b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.612+0000 7f9a39d9b700 1 --2- 172.21.15.44:0/655388072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed79550 0x55e47ed88b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.612+0000 7f9a39d9b700 1 -- 172.21.15.44:0/655388072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed77db0 msgr2=0x55e47ed815e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.612+0000 7f9a39d9b700 1 --2- 172.21.15.44:0/655388072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed77db0 0x55e47ed815e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.612+0000 7f9a39d9b700 1 -- 172.21.15.44:0/655388072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3c19e930 con 0x55e47ed78bb0 2023-09-10T00:39:54.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.612+0000 7f9a3a59c700 1 --2- 172.21.15.44:0/655388072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed77db0 0x55e47ed815e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:54.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.612+0000 7f9a39d9b700 1 --2- 172.21.15.44:0/655388072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed78bb0 0x55e47ed81b20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c000c00 tx=0x7f9a2c010a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4691028a9a6cf93 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.612+0000 7f9a21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:39:54.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.614+0000 7f9a21ffb700 1 -- 172.21.15.44:0/655388072 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c07b020 con 0x55e47ed78bb0 2023-09-10T00:39:54.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.614+0000 7f9a21ffb700 1 -- 172.21.15.44:0/655388072 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9a2c0974b0 con 0x55e47ed78bb0 2023-09-10T00:39:54.629 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.614+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/655388072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e47ec214a0 con 0x55e47ed78bb0 2023-09-10T00:39:54.629 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.614+0000 7f9a21ffb700 1 -- 172.21.15.44:0/655388072 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a2c098600 con 0x55e47ed78bb0 2023-09-10T00:39:54.629 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.615+0000 7f9a4cfb1b80 10 client.15396 fetch_fsmap learned FSMap version 27 2023-09-10T00:39:54.629 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.615+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/655388072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e47eadec10 con 0x55e47ed78bb0 2023-09-10T00:39:54.629 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.615+0000 7f9a21ffb700 1 -- 172.21.15.44:0/655388072 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e47ec214a0 con 0x55e47ed78bb0 2023-09-10T00:39:54.629 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.615+0000 7f9a21ffb700 1 -- 172.21.15.44:0/655388072 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9a2c0a0cf0 con 0x55e47ed78bb0 2023-09-10T00:39:54.630 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.615+0000 7f9a4cfb1b80 10 client.15396 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:39:54.630 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.615+0000 7f9a4cfb1b80 10 client.15396 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-10T00:39:54.630 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.615+0000 7f9a4cfb1b80 20 client.15396 populate_metadata read hostname 'smithi044' 2023-09-10T00:39:54.630 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.615+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/655388072 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55e47ed8c7c0 0x55e47ed8ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.630 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.615+0000 7f9a4cfb1b80 4 client.15396 mds_command: new command op to 24574 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:39:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.615+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/655388072 --> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e47ecde850 con 0x55e47ed8c7c0 2023-09-10T00:39:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.615+0000 7f9a3a59c700 1 --2- 172.21.15.44:0/655388072 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55e47ed8c7c0 0x55e47ed8ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.616+0000 7f9a3a59c700 1 --2- 172.21.15.44:0/655388072 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55e47ed8c7c0 0x55e47ed8ba00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6edb23f92ec28aee server_cookie=173554e57fdc544c in_seq=0 out_seq=0 2023-09-10T00:39:54.631 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.616+0000 7f9a21ffb700 10 client.15396 ms_handle_connect on v2:172.21.15.131:6838/3222652693 2023-09-10T00:39:54.632 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.616+0000 7f9a21ffb700 1 -- 172.21.15.44:0/655388072 <== mds.1 v2:172.21.15.131:6838/3222652693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e47ecde850 con 0x55e47ed8c7c0 2023-09-10T00:39:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.616+0000 7f9a21ffb700 10 client.15396 handle_command_reply: tid=0 2023-09-10T00:39:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.616+0000 7f9a4cfb1b80 1 client.15396 shutdown 2023-09-10T00:39:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.617+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/655388072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed78bb0 msgr2=0x55e47ed81b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.617+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/655388072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed78bb0 0x55e47ed81b20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c000c00 tx=0x7f9a2c010a60 comp rx=0 tx=0).stop 2023-09-10T00:39:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.617+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/655388072 shutdown_connections 2023-09-10T00:39:54.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.617+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/655388072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed79550 0x55e47ed88b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.617+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/655388072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed77db0 0x55e47ed815e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.617+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/655388072 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55e47ed8c7c0 0x55e47ed8ba00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.617+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/655388072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed78bb0 0x55e47ed81b20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.617+0000 7f9a21ffb700 0 client.15396 ms_handle_reset on v2:172.21.15.131:6838/3222652693 2023-09-10T00:39:54.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.617+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/655388072 >> 172.21.15.44:0/655388072 conn(0x55e47ecec040 msgr2=0x55e47ed71d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.617+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/655388072 shutdown_connections 2023-09-10T00:39:54.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.617+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/655388072 wait complete. 2023-09-10T00:39:54.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.617+0000 7f9a4cfb1b80 20 client.15396 trim_cache size 0 max 16384 2023-09-10T00:39:54.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a4cfb1b80 1 Processor -- start 2023-09-10T00:39:54.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a4cfb1b80 1 -- start start 2023-09-10T00:39:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a4cfb1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed77db0 0x55e47ed7f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a4cfb1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed78bb0 0x55e47ed7f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a4cfb1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed79550 0x55e47ed7bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a4cfb1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47ed827d0 con 0x55e47ed78bb0 2023-09-10T00:39:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a4cfb1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47ec714d0 con 0x55e47ed77db0 2023-09-10T00:39:54.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a4cfb1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47eb57de0 con 0x55e47ed79550 2023-09-10T00:39:54.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a39d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed78bb0 0x55e47ed7f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a39d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed78bb0 0x55e47ed7f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47678/0 (socket says 172.21.15.44:47678) 2023-09-10T00:39:54.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a39d9b700 1 -- 172.21.15.44:0/814312895 learned_addr learned my addr 172.21.15.44:0/814312895 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a3a59c700 1 --2- 172.21.15.44:0/814312895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed77db0 0x55e47ed7f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a39d9b700 1 -- 172.21.15.44:0/814312895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed79550 msgr2=0x55e47ed7bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a39d9b700 1 --2- 172.21.15.44:0/814312895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed79550 0x55e47ed7bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a39d9b700 1 -- 172.21.15.44:0/814312895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed77db0 msgr2=0x55e47ed7f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a39d9b700 1 --2- 172.21.15.44:0/814312895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed77db0 0x55e47ed7f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a39d9b700 1 -- 172.21.15.44:0/814312895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e47eb1c200 con 0x55e47ed78bb0 2023-09-10T00:39:54.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.635+0000 7f9a3a59c700 1 --2- 172.21.15.44:0/814312895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed77db0 0x55e47ed7f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:39:54.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.636+0000 7f9a39d9b700 1 --2- 172.21.15.44:0/814312895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed78bb0 0x55e47ed7f8d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0990a0 tx=0x7f9a2c010ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b5d20a1d0d2e21 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.636+0000 7f9a297fa700 1 -- 172.21.15.44:0/814312895 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c07b070 con 0x55e47ed78bb0 2023-09-10T00:39:54.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.636+0000 7f9a297fa700 1 -- 172.21.15.44:0/814312895 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9a2c010760 con 0x55e47ed78bb0 2023-09-10T00:39:54.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.636+0000 7f9a297fa700 1 -- 172.21.15.44:0/814312895 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c0a0cf0 con 0x55e47ed78bb0 2023-09-10T00:39:54.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.636+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/814312895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed78bb0 msgr2=0x55e47ed7f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.636+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/814312895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed78bb0 0x55e47ed7f8d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0990a0 tx=0x7f9a2c010ad0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.636+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/814312895 shutdown_connections 2023-09-10T00:39:54.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.636+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/814312895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed79550 0x55e47ed7bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/814312895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed77db0 0x55e47ed7f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/814312895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed78bb0 0x55e47ed7f8d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/814312895 >> 172.21.15.44:0/814312895 conn(0x55e47ecec040 msgr2=0x55e47ed892d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/814312895 shutdown_connections 2023-09-10T00:39:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/814312895 wait complete. 2023-09-10T00:39:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a4cfb1b80 1 -- start start 2023-09-10T00:39:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a4cfb1b80 1 Processor -- start 2023-09-10T00:39:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a4cfb1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed77db0 0x55e47ed84580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a4cfb1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed78bb0 0x55e47ed82b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a4cfb1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed79550 0x55e47ed831a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a4cfb1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47ed838e0 con 0x55e47ed77db0 2023-09-10T00:39:54.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a4cfb1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47ed83a50 con 0x55e47ed78bb0 2023-09-10T00:39:54.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a4cfb1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47ede48b0 con 0x55e47ed79550 2023-09-10T00:39:54.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a4cfb1b80 10 client.0 _ll_register_callbacks cb 0x7f9a34007c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:39:54.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a3a59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed77db0 0x55e47ed84580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a3a59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed77db0 0x55e47ed84580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47682/0 (socket says 172.21.15.44:47682) 2023-09-10T00:39:54.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a3a59c700 1 -- 172.21.15.44:0/1322889981 learned_addr learned my addr 172.21.15.44:0/1322889981 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a39d9b700 1 --2- 172.21.15.44:0/1322889981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed78bb0 0x55e47ed82b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.637+0000 7f9a3ad9d700 1 --2- 172.21.15.44:0/1322889981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed79550 0x55e47ed831a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a3a59c700 1 -- 172.21.15.44:0/1322889981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed79550 msgr2=0x55e47ed831a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a3a59c700 1 --2- 172.21.15.44:0/1322889981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed79550 0x55e47ed831a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a3a59c700 1 -- 172.21.15.44:0/1322889981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed78bb0 msgr2=0x55e47ed82b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a3a59c700 1 --2- 172.21.15.44:0/1322889981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed78bb0 0x55e47ed82b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a3a59c700 1 -- 172.21.15.44:0/1322889981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e47eadec10 con 0x55e47ed77db0 2023-09-10T00:39:54.652 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a3a59c700 1 --2- 172.21.15.44:0/1322889981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed77db0 0x55e47ed84580 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000b580 tx=0x7f9a3001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa03271641ec49f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.653 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:39:54.653 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a21ffb700 1 -- 172.21.15.44:0/1322889981 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a300329a0 con 0x55e47ed77db0 2023-09-10T00:39:54.653 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/1322889981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a2c09cf40 con 0x55e47ed77db0 2023-09-10T00:39:54.653 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a21ffb700 1 -- 172.21.15.44:0/1322889981 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9a30032b00 con 0x55e47ed77db0 2023-09-10T00:39:54.653 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a21ffb700 1 -- 172.21.15.44:0/1322889981 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a30032cc0 con 0x55e47ed77db0 2023-09-10T00:39:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a4cfb1b80 10 client.15402 fetch_fsmap learned FSMap version 27 2023-09-10T00:39:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/1322889981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e47eaf4260 con 0x55e47ed77db0 2023-09-10T00:39:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a21ffb700 1 -- 172.21.15.44:0/1322889981 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9a2c09cf40 con 0x55e47ed77db0 2023-09-10T00:39:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a21ffb700 1 -- 172.21.15.44:0/1322889981 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9a3002f050 con 0x55e47ed77db0 2023-09-10T00:39:54.654 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a4cfb1b80 10 client.15402 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:39:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a4cfb1b80 10 client.15402 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-10T00:39:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a4cfb1b80 20 client.15402 populate_metadata read hostname 'smithi044' 2023-09-10T00:39:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/1322889981 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55e47ede9af0 0x55e47ede5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a4cfb1b80 4 client.15402 mds_command: new command op to 24574 tid=0[{"prefix": "status"}] 2023-09-10T00:39:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.638+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/1322889981 --> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e47ecc28d0 con 0x55e47ede9af0 2023-09-10T00:39:54.655 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.639+0000 7f9a39d9b700 1 --2- 172.21.15.44:0/1322889981 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55e47ede9af0 0x55e47ede5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.639+0000 7f9a39d9b700 1 --2- 172.21.15.44:0/1322889981 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55e47ede9af0 0x55e47ede5f30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=76823b536602a271 server_cookie=332cc38eb3036c3 in_seq=0 out_seq=0 2023-09-10T00:39:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.639+0000 7f9a21ffb700 10 client.15402 ms_handle_connect on v2:172.21.15.131:6838/3222652693 2023-09-10T00:39:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.639+0000 7f9a21ffb700 1 -- 172.21.15.44:0/1322889981 <== mds.1 v2:172.21.15.131:6838/3222652693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55e47ecc28d0 con 0x55e47ede9af0 2023-09-10T00:39:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.639+0000 7f9a21ffb700 10 client.15402 handle_command_reply: tid=0 2023-09-10T00:39:54.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.639+0000 7f9a4cfb1b80 1 client.15402 shutdown 2023-09-10T00:39:54.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.639+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/1322889981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed77db0 msgr2=0x55e47ed84580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.639+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/1322889981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed77db0 0x55e47ed84580 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000b580 tx=0x7f9a3001a050 comp rx=0 tx=0).stop 2023-09-10T00:39:54.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/1322889981 shutdown_connections 2023-09-10T00:39:54.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/1322889981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e47ed79550 0x55e47ed831a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/1322889981 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55e47ede9af0 0x55e47ede5f30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/1322889981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e47ed77db0 0x55e47ed84580 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a21ffb700 0 client.15402 ms_handle_reset on v2:172.21.15.131:6838/3222652693 2023-09-10T00:39:54.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a4cfb1b80 1 --2- 172.21.15.44:0/1322889981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47ed78bb0 0x55e47ed82b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/1322889981 >> 172.21.15.44:0/1322889981 conn(0x55e47ecec040 msgr2=0x55e47ed892d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/1322889981 shutdown_connections 2023-09-10T00:39:54.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a4cfb1b80 1 -- 172.21.15.44:0/1322889981 wait complete. 2023-09-10T00:39:54.659 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a4cfb1b80 20 client.15402 trim_cache size 0 max 16384 2023-09-10T00:39:54.659 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:39:54.659 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da", 2023-09-10T00:39:54.659 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 1, 2023-09-10T00:39:54.660 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24574, 2023-09-10T00:39:54.660 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2023-09-10T00:39:54.660 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2023-09-10T00:39:54.660 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2023-09-10T00:39:54.660 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 55.808900598000001, 2023-09-10T00:39:54.660 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2023-09-10T00:39:54.661 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 89, 2023-09-10T00:39:54.661 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 87, 2023-09-10T00:39:54.661 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 87.878564315000006 2023-09-10T00:39:54.661 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:39:54.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a40deb700 1 -- 172.21.15.44:0/4226151618 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a240a0750 msgr2=0x7f9a240a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a40deb700 1 --2- 172.21.15.44:0/4226151618 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a240a0750 0x7f9a240a2c10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9a30006010 tx=0x7f9a30005ea0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a40deb700 1 -- 172.21.15.44:0/4226151618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c101820 msgr2=0x7f9a3c1967e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a40deb700 1 --2- 172.21.15.44:0/4226151618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c101820 0x7f9a3c1967e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00d950 tx=0x7f9a2c00dc60 comp rx=0 tx=0).stop 2023-09-10T00:39:54.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a40deb700 1 -- 172.21.15.44:0/4226151618 shutdown_connections 2023-09-10T00:39:54.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a40deb700 1 --2- 172.21.15.44:0/4226151618 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a240a0750 0x7f9a240a2c10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a40deb700 1 --2- 172.21.15.44:0/4226151618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a3c1021c0 0x7f9a3c19d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a40deb700 1 --2- 172.21.15.44:0/4226151618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a3c100630 0x7f9a3c1962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a40deb700 1 --2- 172.21.15.44:0/4226151618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a3c101820 0x7f9a3c1967e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a40deb700 1 -- 172.21.15.44:0/4226151618 >> 172.21.15.44:0/4226151618 conn(0x7f9a3c0fbb80 msgr2=0x7f9a3c104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a40deb700 1 -- 172.21.15.44:0/4226151618 shutdown_connections 2023-09-10T00:39:54.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.640+0000 7f9a40deb700 1 -- 172.21.15.44:0/4226151618 wait complete. 2023-09-10T00:39:54.665 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-10T00:39:54.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.791+0000 7f1298331700 1 -- 172.21.15.44:0/3321353087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12901006e0 msgr2=0x7f1290100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.791+0000 7f1298331700 1 --2- 172.21.15.44:0/3321353087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12901006e0 0x7f1290100b40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f12800099c0 tx=0x7f1280009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.791+0000 7f1298331700 1 -- 172.21.15.44:0/3321353087 shutdown_connections 2023-09-10T00:39:54.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.791+0000 7f1298331700 1 --2- 172.21.15.44:0/3321353087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1290101080 0x7f1290107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.791+0000 7f1298331700 1 --2- 172.21.15.44:0/3321353087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12900ff4f0 0x7f12900ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.791+0000 7f1298331700 1 --2- 172.21.15.44:0/3321353087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12901006e0 0x7f1290100b40 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.791+0000 7f1298331700 1 -- 172.21.15.44:0/3321353087 >> 172.21.15.44:0/3321353087 conn(0x7f12900faa60 msgr2=0x7f12900fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.791+0000 7f1298331700 1 -- 172.21.15.44:0/3321353087 shutdown_connections 2023-09-10T00:39:54.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.791+0000 7f1298331700 1 -- 172.21.15.44:0/3321353087 wait complete. 2023-09-10T00:39:54.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.791+0000 7f1298331700 1 Processor -- start 2023-09-10T00:39:54.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.791+0000 7f1298331700 1 -- start start 2023-09-10T00:39:54.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f1298331700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12900ff4f0 0x7f1290196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f1298331700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12901006e0 0x7f1290196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f1298331700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1290101080 0x7f129019d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f1298331700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129019dda0 con 0x7f1290101080 2023-09-10T00:39:54.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f1298331700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129019dee0 con 0x7f12900ff4f0 2023-09-10T00:39:54.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f1298331700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129019e070 con 0x7f12901006e0 2023-09-10T00:39:54.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f12968ce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1290101080 0x7f129019d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f12968ce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1290101080 0x7f129019d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47700/0 (socket says 172.21.15.44:47700) 2023-09-10T00:39:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f12968ce700 1 -- 172.21.15.44:0/342066785 learned_addr learned my addr 172.21.15.44:0/342066785 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f12960cd700 1 --2- 172.21.15.44:0/342066785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12900ff4f0 0x7f1290196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f12958cc700 1 --2- 172.21.15.44:0/342066785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12901006e0 0x7f1290196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f12968ce700 1 -- 172.21.15.44:0/342066785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12901006e0 msgr2=0x7f1290196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f12968ce700 1 --2- 172.21.15.44:0/342066785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12901006e0 0x7f1290196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f12968ce700 1 -- 172.21.15.44:0/342066785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12900ff4f0 msgr2=0x7f1290196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f12968ce700 1 --2- 172.21.15.44:0/342066785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12900ff4f0 0x7f1290196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f12968ce700 1 -- 172.21.15.44:0/342066785 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12800095f0 con 0x7f1290101080 2023-09-10T00:39:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f12968ce700 1 --2- 172.21.15.44:0/342066785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1290101080 0x7f129019d7d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1288007ee0 tx=0x7f128800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f12877fe700 1 -- 172.21.15.44:0/342066785 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288017070 con 0x7f1290101080 2023-09-10T00:39:54.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f12877fe700 1 -- 172.21.15.44:0/342066785 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f128800f040 con 0x7f1290101080 2023-09-10T00:39:54.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f1298331700 1 -- 172.21.15.44:0/342066785 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129019e350 con 0x7f1290101080 2023-09-10T00:39:54.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.792+0000 7f12877fe700 1 -- 172.21.15.44:0/342066785 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288005710 con 0x7f1290101080 2023-09-10T00:39:54.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.793+0000 7f1298331700 1 -- 172.21.15.44:0/342066785 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129019e860 con 0x7f1290101080 2023-09-10T00:39:54.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.794+0000 7f12877fe700 1 -- 172.21.15.44:0/342066785 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f1288005870 con 0x7f1290101080 2023-09-10T00:39:54.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.795+0000 7f12877fe700 1 --2- 172.21.15.44:0/342066785 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f127c0a0750 0x7f127c0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.795+0000 7f12877fe700 1 -- 172.21.15.44:0/342066785 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f12880cb6f0 con 0x7f1290101080 2023-09-10T00:39:54.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.795+0000 7f12960cd700 1 --2- 172.21.15.44:0/342066785 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f127c0a0750 0x7f127c0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.796+0000 7f12960cd700 1 --2- 172.21.15.44:0/342066785 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f127c0a0750 0x7f127c0a2c10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1290100540 tx=0x7f128c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.796+0000 7f12a44f7b80 1 Processor -- start 2023-09-10T00:39:54.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.796+0000 7f12a44f7b80 1 -- start start 2023-09-10T00:39:54.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12a44f7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c35bb0 0x564b23c35fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12a44f7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c36550 0x564b23c347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12a44f7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c34db0 0x564b23c35260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12a44f7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b23b8d6e0 con 0x564b23c34db0 2023-09-10T00:39:54.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12a44f7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b23b8b000 con 0x564b23c36550 2023-09-10T00:39:54.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12a44f7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b23b868f0 con 0x564b23c35bb0 2023-09-10T00:39:54.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12968ce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c34db0 0x564b23c35260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12968ce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c34db0 0x564b23c35260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47708/0 (socket says 172.21.15.44:47708) 2023-09-10T00:39:54.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12968ce700 1 -- 172.21.15.44:0/1264803091 learned_addr learned my addr 172.21.15.44:0/1264803091 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12960cd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c35bb0 0x564b23c35fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12958cc700 1 --2- 172.21.15.44:0/1264803091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c36550 0x564b23c347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12968ce700 1 -- 172.21.15.44:0/1264803091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c35bb0 msgr2=0x564b23c35fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12968ce700 1 --2- 172.21.15.44:0/1264803091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c35bb0 0x564b23c35fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12968ce700 1 -- 172.21.15.44:0/1264803091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c36550 msgr2=0x564b23c347b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12968ce700 1 --2- 172.21.15.44:0/1264803091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c36550 0x564b23c347b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12968ce700 1 -- 172.21.15.44:0/1264803091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b239b2d90 con 0x564b23c34db0 2023-09-10T00:39:54.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12960cd700 1 --2- 172.21.15.44:0/1264803091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c35bb0 0x564b23c35fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:39:54.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12958cc700 1 --2- 172.21.15.44:0/1264803091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c36550 0x564b23c347b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:39:54.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.797+0000 7f12968ce700 1 --2- 172.21.15.44:0/1264803091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c34db0 0x564b23c35260 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1288094f70 tx=0x7f1288098950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b7110c8ca61000 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.798+0000 7f1284d89700 1 -- 172.21.15.44:0/1264803091 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288098a70 con 0x564b23c34db0 2023-09-10T00:39:54.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.798+0000 7f1284d89700 1 -- 172.21.15.44:0/1264803091 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f128809c8a0 con 0x564b23c34db0 2023-09-10T00:39:54.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.798+0000 7f1284d89700 1 -- 172.21.15.44:0/1264803091 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288098ef0 con 0x564b23c34db0 2023-09-10T00:39:54.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.798+0000 7f12a44f7b80 1 -- 172.21.15.44:0/1264803091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c34db0 msgr2=0x564b23c35260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.798+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/1264803091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c34db0 0x564b23c35260 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1288094f70 tx=0x7f1288098950 comp rx=0 tx=0).stop 2023-09-10T00:39:54.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.798+0000 7f12a44f7b80 1 -- 172.21.15.44:0/1264803091 shutdown_connections 2023-09-10T00:39:54.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.798+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/1264803091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c34db0 0x564b23c35260 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.798+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/1264803091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c35bb0 0x564b23c35fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.798+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/1264803091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c36550 0x564b23c347b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.798+0000 7f12a44f7b80 1 -- 172.21.15.44:0/1264803091 >> 172.21.15.44:0/1264803091 conn(0x564b23ba9040 msgr2=0x564b23c2f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.798+0000 7f12a44f7b80 1 -- 172.21.15.44:0/1264803091 shutdown_connections 2023-09-10T00:39:54.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.798+0000 7f12a44f7b80 1 -- 172.21.15.44:0/1264803091 wait complete. 2023-09-10T00:39:54.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12a44f7b80 1 -- start start 2023-09-10T00:39:54.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12a44f7b80 1 Processor -- start 2023-09-10T00:39:54.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12a44f7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c34db0 0x564b23c3e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12a44f7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c35bb0 0x564b23c3eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12a44f7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c36550 0x564b23c45b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12a44f7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b23b24b20 con 0x564b23c35bb0 2023-09-10T00:39:54.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12a44f7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b23b24820 con 0x564b23c36550 2023-09-10T00:39:54.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12a44f7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b23a088d0 con 0x564b23c34db0 2023-09-10T00:39:54.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12a44f7b80 10 client.0 _ll_register_callbacks cb 0x564b23a6ea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:39:54.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12958cc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c35bb0 0x564b23c3eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12958cc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c35bb0 0x564b23c3eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47712/0 (socket says 172.21.15.44:47712) 2023-09-10T00:39:54.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12958cc700 1 -- 172.21.15.44:0/2862002162 learned_addr learned my addr 172.21.15.44:0/2862002162 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12968ce700 1 --2- 172.21.15.44:0/2862002162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c36550 0x564b23c45b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12958cc700 1 -- 172.21.15.44:0/2862002162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c34db0 msgr2=0x564b23c3e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12958cc700 1 --2- 172.21.15.44:0/2862002162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c34db0 0x564b23c3e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12958cc700 1 -- 172.21.15.44:0/2862002162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c36550 msgr2=0x564b23c45b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12958cc700 1 --2- 172.21.15.44:0/2862002162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c36550 0x564b23c45b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.799+0000 7f12958cc700 1 -- 172.21.15.44:0/2862002162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128c010a70 con 0x564b23c35bb0 2023-09-10T00:39:54.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.800+0000 7f12958cc700 1 --2- 172.21.15.44:0/2862002162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c35bb0 0x564b23c3eae0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f128000b580 tx=0x7f12800058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f87e1e225e78eb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.800+0000 7f1279ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:39:54.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.800+0000 7f1279ffb700 1 -- 172.21.15.44:0/2862002162 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12800055d0 con 0x564b23c35bb0 2023-09-10T00:39:54.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.800+0000 7f1279ffb700 1 -- 172.21.15.44:0/2862002162 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1280004500 con 0x564b23c35bb0 2023-09-10T00:39:54.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.800+0000 7f12a44f7b80 1 -- 172.21.15.44:0/2862002162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b23ade4a0 con 0x564b23c35bb0 2023-09-10T00:39:54.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.800+0000 7f1279ffb700 1 -- 172.21.15.44:0/2862002162 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12800046c0 con 0x564b23c35bb0 2023-09-10T00:39:54.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.800+0000 7f12a44f7b80 10 client.15414 fetch_fsmap learned FSMap version 27 2023-09-10T00:39:54.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.800+0000 7f12a44f7b80 1 -- 172.21.15.44:0/2862002162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f129019e860 con 0x564b23c35bb0 2023-09-10T00:39:54.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.800+0000 7f1279ffb700 1 -- 172.21.15.44:0/2862002162 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564b23ade4a0 con 0x564b23c35bb0 2023-09-10T00:39:54.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.800+0000 7f1279ffb700 1 -- 172.21.15.44:0/2862002162 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1280003c80 con 0x564b23c35bb0 2023-09-10T00:39:54.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.800+0000 7f12a44f7b80 10 client.15414 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:39:54.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.800+0000 7f12a44f7b80 10 client.15414 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-10T00:39:54.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.800+0000 7f12a44f7b80 20 client.15414 populate_metadata read hostname 'smithi044' 2023-09-10T00:39:54.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.800+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/2862002162 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564b23c49d40 0x564b23c48c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.800+0000 7f12a44f7b80 4 client.15414 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:39:54.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.800+0000 7f12a44f7b80 1 -- 172.21.15.44:0/2862002162 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b23b9b850 con 0x564b23c49d40 2023-09-10T00:39:54.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.801+0000 7f12960cd700 1 --2- 172.21.15.44:0/2862002162 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564b23c49d40 0x564b23c48c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.801+0000 7f12960cd700 1 --2- 172.21.15.44:0/2862002162 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564b23c49d40 0x564b23c48c70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22ead25bc1864810 server_cookie=953bb1a8af2f061 in_seq=0 out_seq=0 2023-09-10T00:39:54.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.801+0000 7f1279ffb700 10 client.15414 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:39:54.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.802+0000 7f1279ffb700 1 -- 172.21.15.44:0/2862002162 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564b23b9b850 con 0x564b23c49d40 2023-09-10T00:39:54.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.802+0000 7f1279ffb700 10 client.15414 handle_command_reply: tid=0 2023-09-10T00:39:54.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.802+0000 7f12a44f7b80 1 client.15414 shutdown 2023-09-10T00:39:54.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.802+0000 7f12a44f7b80 1 -- 172.21.15.44:0/2862002162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c35bb0 msgr2=0x564b23c3eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.802+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/2862002162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c35bb0 0x564b23c3eae0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f128000b580 tx=0x7f12800058e0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.802+0000 7f12a44f7b80 1 -- 172.21.15.44:0/2862002162 shutdown_connections 2023-09-10T00:39:54.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.802+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/2862002162 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564b23c49d40 0x564b23c48c70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.802+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/2862002162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c36550 0x564b23c45b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.802+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/2862002162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c34db0 0x564b23c3e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.802+0000 7f1279ffb700 0 client.15414 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:39:54.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.802+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/2862002162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c35bb0 0x564b23c3eae0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.802+0000 7f12a44f7b80 1 -- 172.21.15.44:0/2862002162 >> 172.21.15.44:0/2862002162 conn(0x564b23ba9040 msgr2=0x564b23c2ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.802+0000 7f12a44f7b80 1 -- 172.21.15.44:0/2862002162 shutdown_connections 2023-09-10T00:39:54.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.802+0000 7f12a44f7b80 1 -- 172.21.15.44:0/2862002162 wait complete. 2023-09-10T00:39:54.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.802+0000 7f12a44f7b80 20 client.15414 trim_cache size 0 max 16384 2023-09-10T00:39:54.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.820+0000 7f12a44f7b80 1 Processor -- start 2023-09-10T00:39:54.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.820+0000 7f12a44f7b80 1 -- start start 2023-09-10T00:39:54.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.820+0000 7f12a44f7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c34db0 0x564b23c3c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.820+0000 7f12a44f7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c35bb0 0x564b23c45ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.820+0000 7f12a44f7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c36550 0x564b23c38ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.820+0000 7f12a44f7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b23b4ff70 con 0x564b23c35bb0 2023-09-10T00:39:54.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.820+0000 7f12a44f7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b23b42ed0 con 0x564b23c36550 2023-09-10T00:39:54.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.820+0000 7f12a44f7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b23b2e4d0 con 0x564b23c34db0 2023-09-10T00:39:54.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.820+0000 7f12958cc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c35bb0 0x564b23c45ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.820+0000 7f12958cc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c35bb0 0x564b23c45ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47720/0 (socket says 172.21.15.44:47720) 2023-09-10T00:39:54.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.820+0000 7f12958cc700 1 -- 172.21.15.44:0/3117404157 learned_addr learned my addr 172.21.15.44:0/3117404157 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.820+0000 7f12960cd700 1 --2- 172.21.15.44:0/3117404157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c34db0 0x564b23c3c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.820+0000 7f12958cc700 1 -- 172.21.15.44:0/3117404157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c34db0 msgr2=0x564b23c3c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f12958cc700 1 --2- 172.21.15.44:0/3117404157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c34db0 0x564b23c3c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f12958cc700 1 -- 172.21.15.44:0/3117404157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c36550 msgr2=0x564b23c38ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f12958cc700 1 --2- 172.21.15.44:0/3117404157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c36550 0x564b23c38ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f12958cc700 1 -- 172.21.15.44:0/3117404157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b2399bc10 con 0x564b23c35bb0 2023-09-10T00:39:54.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f12958cc700 1 --2- 172.21.15.44:0/3117404157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c35bb0 0x564b23c45ae0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1280003fe0 tx=0x7f1280009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a693525609ba86ea server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f128558a700 1 -- 172.21.15.44:0/3117404157 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128003a070 con 0x564b23c35bb0 2023-09-10T00:39:54.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f128558a700 1 -- 172.21.15.44:0/3117404157 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f128001e050 con 0x564b23c35bb0 2023-09-10T00:39:54.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f128558a700 1 -- 172.21.15.44:0/3117404157 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128000f670 con 0x564b23c35bb0 2023-09-10T00:39:54.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f12a44f7b80 1 -- 172.21.15.44:0/3117404157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c35bb0 msgr2=0x564b23c45ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/3117404157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c35bb0 0x564b23c45ae0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1280003fe0 tx=0x7f1280009cd0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f12a44f7b80 1 -- 172.21.15.44:0/3117404157 shutdown_connections 2023-09-10T00:39:54.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/3117404157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c36550 0x564b23c38ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/3117404157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c34db0 0x564b23c3c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/3117404157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c35bb0 0x564b23c45ae0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f12a44f7b80 1 -- 172.21.15.44:0/3117404157 >> 172.21.15.44:0/3117404157 conn(0x564b23ba9040 msgr2=0x564b23c48c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f12a44f7b80 1 -- 172.21.15.44:0/3117404157 shutdown_connections 2023-09-10T00:39:54.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.821+0000 7f12a44f7b80 1 -- 172.21.15.44:0/3117404157 wait complete. 2023-09-10T00:39:54.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.822+0000 7f12a44f7b80 1 -- start start 2023-09-10T00:39:54.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.822+0000 7f12a44f7b80 1 Processor -- start 2023-09-10T00:39:54.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.822+0000 7f12a44f7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c34db0 0x564b23c407d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.822+0000 7f12a44f7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c35bb0 0x564b23c40d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.822+0000 7f12a44f7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c36550 0x564b23c9dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.822+0000 7f12a44f7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b23c41250 con 0x564b23c36550 2023-09-10T00:39:54.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.822+0000 7f12a44f7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b23c413c0 con 0x564b23c34db0 2023-09-10T00:39:54.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.822+0000 7f12a44f7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b23c9e2d0 con 0x564b23c35bb0 2023-09-10T00:39:54.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.822+0000 7f12a44f7b80 10 client.0 _ll_register_callbacks cb 0x7f1280009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:39:54.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.822+0000 7f12968ce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c36550 0x564b23c9dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.822+0000 7f12968ce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c36550 0x564b23c9dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47728/0 (socket says 172.21.15.44:47728) 2023-09-10T00:39:54.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.822+0000 7f12968ce700 1 -- 172.21.15.44:0/274966209 learned_addr learned my addr 172.21.15.44:0/274966209 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:54.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.822+0000 7f12960cd700 1 --2- 172.21.15.44:0/274966209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c34db0 0x564b23c407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.822+0000 7f12958cc700 1 --2- 172.21.15.44:0/274966209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c35bb0 0x564b23c40d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f12968ce700 1 -- 172.21.15.44:0/274966209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c35bb0 msgr2=0x564b23c40d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f12968ce700 1 --2- 172.21.15.44:0/274966209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c35bb0 0x564b23c40d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f12968ce700 1 -- 172.21.15.44:0/274966209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c34db0 msgr2=0x564b23c407d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:39:54.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f12968ce700 1 --2- 172.21.15.44:0/274966209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c34db0 0x564b23c407d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f12968ce700 1 -- 172.21.15.44:0/274966209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129019e860 con 0x564b23c36550 2023-09-10T00:39:54.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f12960cd700 1 --2- 172.21.15.44:0/274966209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c34db0 0x564b23c407d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:39:54.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f12968ce700 1 --2- 172.21.15.44:0/274966209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c36550 0x564b23c9dd90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f128800d7b0 tx=0x7f1288094db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecc3c62e9f18574 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:54.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f1279ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:39:54.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f1279ffb700 1 -- 172.21.15.44:0/274966209 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288098a70 con 0x564b23c36550 2023-09-10T00:39:54.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f1279ffb700 1 -- 172.21.15.44:0/274966209 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f12880a0ae0 con 0x564b23c36550 2023-09-10T00:39:54.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f12a44f7b80 1 -- 172.21.15.44:0/274966209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f128809c8a0 con 0x564b23c36550 2023-09-10T00:39:54.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f1279ffb700 1 -- 172.21.15.44:0/274966209 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1288098cd0 con 0x564b23c36550 2023-09-10T00:39:54.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f12a44f7b80 10 client.15420 fetch_fsmap learned FSMap version 27 2023-09-10T00:39:54.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f12a44f7b80 1 -- 172.21.15.44:0/274966209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564b239d9200 con 0x564b23c36550 2023-09-10T00:39:54.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f1279ffb700 1 -- 172.21.15.44:0/274966209 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f128809c8a0 con 0x564b23c36550 2023-09-10T00:39:54.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f1279ffb700 1 -- 172.21.15.44:0/274966209 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f12880a0650 con 0x564b23c36550 2023-09-10T00:39:54.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f12a44f7b80 10 client.15420 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:39:54.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f12a44f7b80 10 client.15420 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-10T00:39:54.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f12a44f7b80 20 client.15420 populate_metadata read hostname 'smithi044' 2023-09-10T00:39:54.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/274966209 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564b23ca3220 0x564b23c9f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:54.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f12a44f7b80 4 client.15420 mds_command: new command op to 24605 tid=0[{"prefix": "status"}] 2023-09-10T00:39:54.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.823+0000 7f12a44f7b80 1 -- 172.21.15.44:0/274966209 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564b23b7f8d0 con 0x564b23ca3220 2023-09-10T00:39:54.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.824+0000 7f12960cd700 1 --2- 172.21.15.44:0/274966209 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564b23ca3220 0x564b23c9f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:54.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.824+0000 7f12960cd700 1 --2- 172.21.15.44:0/274966209 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564b23ca3220 0x564b23c9f6c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f4fc56c3f16ad2b server_cookie=9108ec807d9c7ef5 in_seq=0 out_seq=0 2023-09-10T00:39:54.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.824+0000 7f1279ffb700 10 client.15420 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:39:54.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.824+0000 7f1279ffb700 1 -- 172.21.15.44:0/274966209 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x564b23b7f8d0 con 0x564b23ca3220 2023-09-10T00:39:54.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.824+0000 7f1279ffb700 10 client.15420 handle_command_reply: tid=0 2023-09-10T00:39:54.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.824+0000 7f12a44f7b80 1 client.15420 shutdown 2023-09-10T00:39:54.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f12a44f7b80 1 -- 172.21.15.44:0/274966209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c36550 msgr2=0x564b23c9dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/274966209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c36550 0x564b23c9dd90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f128800d7b0 tx=0x7f1288094db0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f12a44f7b80 1 -- 172.21.15.44:0/274966209 shutdown_connections 2023-09-10T00:39:54.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/274966209 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564b23ca3220 0x564b23c9f6c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/274966209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b23c36550 0x564b23c9dd90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/274966209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b23c34db0 0x564b23c407d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f1279ffb700 0 client.15420 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:39:54.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f12a44f7b80 1 --2- 172.21.15.44:0/274966209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b23c35bb0 0x564b23c40d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f12a44f7b80 1 -- 172.21.15.44:0/274966209 >> 172.21.15.44:0/274966209 conn(0x564b23ba9040 msgr2=0x564b23c3c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f12a44f7b80 1 -- 172.21.15.44:0/274966209 shutdown_connections 2023-09-10T00:39:54.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f12a44f7b80 1 -- 172.21.15.44:0/274966209 wait complete. 2023-09-10T00:39:54.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f12a44f7b80 20 client.15420 trim_cache size 0 max 16384 2023-09-10T00:39:54.846 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:39:54.846 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da", 2023-09-10T00:39:54.846 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2023-09-10T00:39:54.847 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24605, 2023-09-10T00:39:54.847 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2023-09-10T00:39:54.847 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2023-09-10T00:39:54.847 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2023-09-10T00:39:54.848 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 75.080333933000006, 2023-09-10T00:39:54.848 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2023-09-10T00:39:54.848 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 89, 2023-09-10T00:39:54.848 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 87, 2023-09-10T00:39:54.848 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 81.432699502999995 2023-09-10T00:39:54.848 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:39:54.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f1298331700 1 -- 172.21.15.44:0/342066785 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f127c0a0750 msgr2=0x7f127c0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f1298331700 1 --2- 172.21.15.44:0/342066785 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f127c0a0750 0x7f127c0a2c10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1290100540 tx=0x7f128c007400 comp rx=0 tx=0).stop 2023-09-10T00:39:54.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f1298331700 1 -- 172.21.15.44:0/342066785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1290101080 msgr2=0x7f129019d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:54.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f1298331700 1 --2- 172.21.15.44:0/342066785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1290101080 0x7f129019d7d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1288007ee0 tx=0x7f128800d3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f1298331700 1 -- 172.21.15.44:0/342066785 shutdown_connections 2023-09-10T00:39:54.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f1298331700 1 --2- 172.21.15.44:0/342066785 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f127c0a0750 0x7f127c0a2c10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f1298331700 1 --2- 172.21.15.44:0/342066785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1290101080 0x7f129019d7d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f1298331700 1 --2- 172.21.15.44:0/342066785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12900ff4f0 0x7f1290196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f1298331700 1 --2- 172.21.15.44:0/342066785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12901006e0 0x7f1290196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:54.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f1298331700 1 -- 172.21.15.44:0/342066785 >> 172.21.15.44:0/342066785 conn(0x7f12900faa60 msgr2=0x7f12900fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:54.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f1298331700 1 -- 172.21.15.44:0/342066785 shutdown_connections 2023-09-10T00:39:54.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:54.825+0000 7f1298331700 1 -- 172.21.15.44:0/342066785 wait complete. 2023-09-10T00:39:54.852 INFO:tasks.cephadm:Setup complete, yielding 2023-09-10T00:39:54.853 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-10T00:39:54.866 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2023-09-10T00:39:54.867 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch status' 2023-09-10T00:39:54.892 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:54 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/126096701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:39:55.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:54 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/126096701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:39:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:54 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/126096701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:39:55.426 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:55.980 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:55 smithi044 ceph-mon[98676]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:39:56.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:55 smithi131 ceph-mon[103713]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:39:56.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:55 smithi099 ceph-mon[102692]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:39:56.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.605+0000 7f9b09897700 1 -- 172.21.15.44:0/411107770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b04102230 msgr2=0x7f9b04108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:56.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.605+0000 7f9b09897700 1 --2- 172.21.15.44:0/411107770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b04102230 0x7f9b04108ae0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00b0d0 tx=0x7f9afc00b3e0 comp rx=0 tx=0).stop 2023-09-10T00:39:56.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.605+0000 7f9b09897700 1 -- 172.21.15.44:0/411107770 shutdown_connections 2023-09-10T00:39:56.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.605+0000 7f9b09897700 1 --2- 172.21.15.44:0/411107770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b04102230 0x7f9b04108ae0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:56.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.605+0000 7f9b09897700 1 --2- 172.21.15.44:0/411107770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b04100690 0x7f9b04100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:56.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.605+0000 7f9b09897700 1 --2- 172.21.15.44:0/411107770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b04101890 0x7f9b04101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:56.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.605+0000 7f9b09897700 1 -- 172.21.15.44:0/411107770 >> 172.21.15.44:0/411107770 conn(0x7f9b040fbbf0 msgr2=0x7f9b040fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:56.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.605+0000 7f9b09897700 1 -- 172.21.15.44:0/411107770 shutdown_connections 2023-09-10T00:39:56.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.605+0000 7f9b09897700 1 -- 172.21.15.44:0/411107770 wait complete. 2023-09-10T00:39:56.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b09897700 1 Processor -- start 2023-09-10T00:39:56.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b09897700 1 -- start start 2023-09-10T00:39:56.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b09897700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b04100690 0x7f9b040743a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:56.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b09897700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b04101890 0x7f9b040729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:56.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b09897700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b04102230 0x7f9b04072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:56.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b09897700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040736b0 con 0x7f9b04100690 2023-09-10T00:39:56.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b09897700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b04073820 con 0x7f9b04101890 2023-09-10T00:39:56.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b09897700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b041a0350 con 0x7f9b04102230 2023-09-10T00:39:56.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b02ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b04100690 0x7f9b040743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:56.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b02ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b04100690 0x7f9b040743a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47738/0 (socket says 172.21.15.44:47738) 2023-09-10T00:39:56.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b02ffd700 1 -- 172.21.15.44:0/950983374 learned_addr learned my addr 172.21.15.44:0/950983374 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:56.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b027fc700 1 --2- 172.21.15.44:0/950983374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b04101890 0x7f9b040729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:56.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b037fe700 1 --2- 172.21.15.44:0/950983374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b04102230 0x7f9b04072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:56.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b02ffd700 1 -- 172.21.15.44:0/950983374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b04102230 msgr2=0x7f9b04072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:56.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b02ffd700 1 --2- 172.21.15.44:0/950983374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b04102230 0x7f9b04072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:56.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b02ffd700 1 -- 172.21.15.44:0/950983374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b04101890 msgr2=0x7f9b040729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:56.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.606+0000 7f9b02ffd700 1 --2- 172.21.15.44:0/950983374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b04101890 0x7f9b040729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:56.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.607+0000 7f9b02ffd700 1 -- 172.21.15.44:0/950983374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc009d00 con 0x7f9b04100690 2023-09-10T00:39:56.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.607+0000 7f9b02ffd700 1 --2- 172.21.15.44:0/950983374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b04100690 0x7f9b040743a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9af800c990 tx=0x7f9af800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:56.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.607+0000 7f9b08895700 1 -- 172.21.15.44:0/950983374 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af8007b00 con 0x7f9b04100690 2023-09-10T00:39:56.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.607+0000 7f9b09897700 1 -- 172.21.15.44:0/950983374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b041a0630 con 0x7f9b04100690 2023-09-10T00:39:56.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.607+0000 7f9b08895700 1 -- 172.21.15.44:0/950983374 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9af8007c60 con 0x7f9b04100690 2023-09-10T00:39:56.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.607+0000 7f9b08895700 1 -- 172.21.15.44:0/950983374 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af8018420 con 0x7f9b04100690 2023-09-10T00:39:56.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.607+0000 7f9b09897700 1 -- 172.21.15.44:0/950983374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b041a0b80 con 0x7f9b04100690 2023-09-10T00:39:56.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.608+0000 7f9b09897700 1 -- 172.21.15.44:0/950983374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b04107c30 con 0x7f9b04100690 2023-09-10T00:39:56.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.609+0000 7f9b08895700 1 -- 172.21.15.44:0/950983374 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f9af800f450 con 0x7f9b04100690 2023-09-10T00:39:56.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.610+0000 7f9b08895700 1 --2- 172.21.15.44:0/950983374 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9aec0a06d0 0x7f9aec0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:56.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.610+0000 7f9b027fc700 1 --2- 172.21.15.44:0/950983374 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9aec0a06d0 0x7f9aec0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:56.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.610+0000 7f9b08895700 1 -- 172.21.15.44:0/950983374 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9af80cc1f0 con 0x7f9b04100690 2023-09-10T00:39:56.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.610+0000 7f9b027fc700 1 --2- 172.21.15.44:0/950983374 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9aec0a06d0 0x7f9aec0a2b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9af4007be0 tx=0x7f9af40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:56.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.613+0000 7f9b08895700 1 -- 172.21.15.44:0/950983374 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9af8095850 con 0x7f9b04100690 2023-09-10T00:39:56.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.832+0000 7f9b09897700 1 -- 172.21.15.44:0/950983374 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b0402cee0 con 0x7f9aec0a06d0 2023-09-10T00:39:56.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.833+0000 7f9b08895700 1 -- 172.21.15.44:0/950983374 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f9b0402cee0 con 0x7f9aec0a06d0 2023-09-10T00:39:56.834 INFO:teuthology.orchestra.run.smithi044.stdout:Backend: cephadm 2023-09-10T00:39:56.834 INFO:teuthology.orchestra.run.smithi044.stdout:Available: Yes 2023-09-10T00:39:56.834 INFO:teuthology.orchestra.run.smithi044.stdout:Paused: No 2023-09-10T00:39:56.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.836+0000 7f9b09897700 1 -- 172.21.15.44:0/950983374 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9aec0a06d0 msgr2=0x7f9aec0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:56.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.836+0000 7f9b09897700 1 --2- 172.21.15.44:0/950983374 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9aec0a06d0 0x7f9aec0a2b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9af4007be0 tx=0x7f9af40073d0 comp rx=0 tx=0).stop 2023-09-10T00:39:56.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.836+0000 7f9b09897700 1 -- 172.21.15.44:0/950983374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b04100690 msgr2=0x7f9b040743a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:56.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.836+0000 7f9b09897700 1 --2- 172.21.15.44:0/950983374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b04100690 0x7f9b040743a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9af800c990 tx=0x7f9af800cca0 comp rx=0 tx=0).stop 2023-09-10T00:39:56.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.836+0000 7f9b09897700 1 -- 172.21.15.44:0/950983374 shutdown_connections 2023-09-10T00:39:56.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.836+0000 7f9b09897700 1 --2- 172.21.15.44:0/950983374 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9aec0a06d0 0x7f9aec0a2b90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:56.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.836+0000 7f9b09897700 1 --2- 172.21.15.44:0/950983374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b04102230 0x7f9b04072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:56.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.836+0000 7f9b09897700 1 --2- 172.21.15.44:0/950983374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b04100690 0x7f9b040743a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:56.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.836+0000 7f9b09897700 1 --2- 172.21.15.44:0/950983374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b04101890 0x7f9b040729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:56.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.836+0000 7f9b09897700 1 -- 172.21.15.44:0/950983374 >> 172.21.15.44:0/950983374 conn(0x7f9b040fbbf0 msgr2=0x7f9b04104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:56.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.836+0000 7f9b09897700 1 -- 172.21.15.44:0/950983374 shutdown_connections 2023-09-10T00:39:56.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:56.836+0000 7f9b09897700 1 -- 172.21.15.44:0/950983374 wait complete. 2023-09-10T00:39:57.190 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch ps' 2023-09-10T00:39:57.793 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:39:58.002 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:57 smithi044 ceph-mon[98676]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-09-10T00:39:58.002 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:57 smithi044 ceph-mon[98676]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:39:58.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:57 smithi131 ceph-mon[103713]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-09-10T00:39:58.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:57 smithi131 ceph-mon[103713]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:39:58.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:57 smithi099 ceph-mon[102692]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-09-10T00:39:58.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:57 smithi099 ceph-mon[102692]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:39:59.026 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.026+0000 7f1d566e5700 1 -- 172.21.15.44:0/1992011153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d500fef10 msgr2=0x7f1d500ff330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:59.027 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.026+0000 7f1d566e5700 1 --2- 172.21.15.44:0/1992011153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d500fef10 0x7f1d500ff330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1d40009990 tx=0x7f1d40009ca0 comp rx=0 tx=0).stop 2023-09-10T00:39:59.027 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.026+0000 7f1d566e5700 1 -- 172.21.15.44:0/1992011153 shutdown_connections 2023-09-10T00:39:59.027 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.026+0000 7f1d566e5700 1 --2- 172.21.15.44:0/1992011153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d50100230 0x7f1d501006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:59.028 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.026+0000 7f1d566e5700 1 --2- 172.21.15.44:0/1992011153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d500fef10 0x7f1d500ff330 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:59.028 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.026+0000 7f1d566e5700 1 --2- 172.21.15.44:0/1992011153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d500ff870 0x7f1d500ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:59.028 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.026+0000 7f1d566e5700 1 -- 172.21.15.44:0/1992011153 >> 172.21.15.44:0/1992011153 conn(0x7f1d500fab10 msgr2=0x7f1d500fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:59.028 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.026+0000 7f1d566e5700 1 -- 172.21.15.44:0/1992011153 shutdown_connections 2023-09-10T00:39:59.028 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.026+0000 7f1d566e5700 1 -- 172.21.15.44:0/1992011153 wait complete. 2023-09-10T00:39:59.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.027+0000 7f1d566e5700 1 Processor -- start 2023-09-10T00:39:59.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.027+0000 7f1d566e5700 1 -- start start 2023-09-10T00:39:59.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.027+0000 7f1d566e5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d500fef10 0x7f1d50198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:59.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.027+0000 7f1d566e5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d500ff870 0x7f1d50198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:59.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.027+0000 7f1d566e5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d50100230 0x7f1d5019fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:59.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.027+0000 7f1d566e5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5019ffd0 con 0x7f1d50100230 2023-09-10T00:39:59.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.027+0000 7f1d566e5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d501a0110 con 0x7f1d500fef10 2023-09-10T00:39:59.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.027+0000 7f1d566e5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d501a02a0 con 0x7f1d500ff870 2023-09-10T00:39:59.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.028+0000 7f1d54c82700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d50100230 0x7f1d5019fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:59.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.028+0000 7f1d54c82700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d50100230 0x7f1d5019fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47754/0 (socket says 172.21.15.44:47754) 2023-09-10T00:39:59.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.028+0000 7f1d54c82700 1 -- 172.21.15.44:0/2744949527 learned_addr learned my addr 172.21.15.44:0/2744949527 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:39:59.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.028+0000 7f1d4ffff700 1 --2- 172.21.15.44:0/2744949527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d500fef10 0x7f1d50198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:59.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.028+0000 7f1d4f7fe700 1 --2- 172.21.15.44:0/2744949527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d500ff870 0x7f1d50198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:59.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.028+0000 7f1d54c82700 1 -- 172.21.15.44:0/2744949527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d500ff870 msgr2=0x7f1d50198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:59.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.028+0000 7f1d54c82700 1 --2- 172.21.15.44:0/2744949527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d500ff870 0x7f1d50198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:59.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.028+0000 7f1d54c82700 1 -- 172.21.15.44:0/2744949527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d500fef10 msgr2=0x7f1d50198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:59.034 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.028+0000 7f1d54c82700 1 --2- 172.21.15.44:0/2744949527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d500fef10 0x7f1d50198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:59.034 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.028+0000 7f1d54c82700 1 -- 172.21.15.44:0/2744949527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d400095f0 con 0x7f1d50100230 2023-09-10T00:39:59.034 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.028+0000 7f1d54c82700 1 --2- 172.21.15.44:0/2744949527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d50100230 0x7f1d5019fa00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1d48007f10 tx=0x7f1d4800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:59.034 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.028+0000 7f1d4d7fa700 1 -- 172.21.15.44:0/2744949527 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d48017070 con 0x7f1d50100230 2023-09-10T00:39:59.034 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.028+0000 7f1d4d7fa700 1 -- 172.21.15.44:0/2744949527 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d4800f040 con 0x7f1d50100230 2023-09-10T00:39:59.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.028+0000 7f1d566e5700 1 -- 172.21.15.44:0/2744949527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d501a0580 con 0x7f1d50100230 2023-09-10T00:39:59.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.028+0000 7f1d4d7fa700 1 -- 172.21.15.44:0/2744949527 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d48005570 con 0x7f1d50100230 2023-09-10T00:39:59.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.028+0000 7f1d566e5700 1 -- 172.21.15.44:0/2744949527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d501a0a80 con 0x7f1d50100230 2023-09-10T00:39:59.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.029+0000 7f1d566e5700 1 -- 172.21.15.44:0/2744949527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d50109cf0 con 0x7f1d50100230 2023-09-10T00:39:59.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.030+0000 7f1d4d7fa700 1 -- 172.21.15.44:0/2744949527 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f1d4800db20 con 0x7f1d50100230 2023-09-10T00:39:59.036 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.031+0000 7f1d4d7fa700 1 --2- 172.21.15.44:0/2744949527 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1d380a0750 0x7f1d380a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:39:59.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.031+0000 7f1d4ffff700 1 --2- 172.21.15.44:0/2744949527 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1d380a0750 0x7f1d380a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:39:59.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.031+0000 7f1d4d7fa700 1 -- 172.21.15.44:0/2744949527 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1d480cb730 con 0x7f1d50100230 2023-09-10T00:39:59.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.032+0000 7f1d4ffff700 1 --2- 172.21.15.44:0/2744949527 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1d380a0750 0x7f1d380a2c10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000b5c0 tx=0x7f1d40005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:39:59.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.035+0000 7f1d4d7fa700 1 -- 172.21.15.44:0/2744949527 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1d48094df0 con 0x7f1d50100230 2023-09-10T00:39:59.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.248+0000 7f1d566e5700 1 -- 172.21.15.44:0/2744949527 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d5002cfd0 con 0x7f1d380a0750 2023-09-10T00:39:59.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.268+0000 7f1d4d7fa700 1 -- 172.21.15.44:0/2744949527 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f1d5002cfd0 con 0x7f1d380a0750 2023-09-10T00:39:59.269 INFO:teuthology.orchestra.run.smithi044.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-09-10T00:39:59.269 INFO:teuthology.orchestra.run.smithi044.stdout:mds.a smithi044 running (88s) - 87s 17.6M - 18.0.0-6025-g9670cca5 20c3861ef6c9 8e9860c4f4e2 2023-09-10T00:39:59.269 INFO:teuthology.orchestra.run.smithi044.stdout:mds.b smithi099 running (86s) - 85s 33.6M - 18.0.0-6025-g9670cca5 20c3861ef6c9 2e986354a756 2023-09-10T00:39:59.269 INFO:teuthology.orchestra.run.smithi044.stdout:mds.c smithi131 running (97s) - 96s 20.4M - 18.0.0-6025-g9670cca5 20c3861ef6c9 bf88bb0a1405 2023-09-10T00:39:59.269 INFO:teuthology.orchestra.run.smithi044.stdout:mds.d smithi044 running (103s) - 103s 17.5M - 18.0.0-6025-g9670cca5 20c3861ef6c9 9195cc73f6c9 2023-09-10T00:39:59.270 INFO:teuthology.orchestra.run.smithi044.stdout:mds.e smithi099 running (101s) - 101s 17.5M - 18.0.0-6025-g9670cca5 20c3861ef6c9 dbbdc1119abc 2023-09-10T00:39:59.270 INFO:teuthology.orchestra.run.smithi044.stdout:mds.f smithi131 running (99s) - 99s 19.0M - 18.0.0-6025-g9670cca5 20c3861ef6c9 8e21cee9e399 2023-09-10T00:39:59.270 INFO:teuthology.orchestra.run.smithi044.stdout:mds.g smithi044 running (90s) - 90s 20.0M - 18.0.0-6025-g9670cca5 20c3861ef6c9 8489a55c83cd 2023-09-10T00:39:59.270 INFO:teuthology.orchestra.run.smithi044.stdout:mds.h smithi099 running (95s) - 94s 18.4M - 18.0.0-6025-g9670cca5 20c3861ef6c9 45b6bde08473 2023-09-10T00:39:59.270 INFO:teuthology.orchestra.run.smithi044.stdout:mds.i smithi131 running (92s) - 92s 18.8M - 18.0.0-6025-g9670cca5 20c3861ef6c9 d02dd914d879 2023-09-10T00:39:59.271 INFO:teuthology.orchestra.run.smithi044.stdout:mds.j smithi044 running (106s) - 105s 15.2M - 18.0.0-6025-g9670cca5 20c3861ef6c9 e21e54c5a077 2023-09-10T00:39:59.271 INFO:teuthology.orchestra.run.smithi044.stdout:mds.k smithi099 running (108s) - 108s 21.3M - 18.0.0-6025-g9670cca5 20c3861ef6c9 861abd1120e6 2023-09-10T00:39:59.271 INFO:teuthology.orchestra.run.smithi044.stdout:mds.l smithi131 running (110s) - 110s 20.0M - 18.0.0-6025-g9670cca5 20c3861ef6c9 82aa2f832a06 2023-09-10T00:39:59.271 INFO:teuthology.orchestra.run.smithi044.stdout:mgr.x smithi044 *:9283,8765 running (8m) - 8m 522M - 18.0.0-6025-g9670cca5 20c3861ef6c9 264933d0183f 2023-09-10T00:39:59.271 INFO:teuthology.orchestra.run.smithi044.stdout:mgr.y smithi099 *:8443,8765 running (6m) - 6m 451M - 18.0.0-6025-g9670cca5 20c3861ef6c9 23557bb627c9 2023-09-10T00:39:59.271 INFO:teuthology.orchestra.run.smithi044.stdout:mgr.z smithi131 *:8443,8765 running (6m) - 6m 450M - 18.0.0-6025-g9670cca5 20c3861ef6c9 3246ccfe3906 2023-09-10T00:39:59.272 INFO:teuthology.orchestra.run.smithi044.stdout:mon.a smithi044 running (8m) - 8m 96.7M 2048M 18.0.0-6025-g9670cca5 20c3861ef6c9 762f3af6a520 2023-09-10T00:39:59.272 INFO:teuthology.orchestra.run.smithi044.stdout:mon.b smithi099 running (6m) - 6m 68.4M 2048M 18.0.0-6025-g9670cca5 20c3861ef6c9 a4aadc593157 2023-09-10T00:39:59.272 INFO:teuthology.orchestra.run.smithi044.stdout:mon.c smithi131 running (7m) - 7m 64.0M 2048M 18.0.0-6025-g9670cca5 20c3861ef6c9 47c976327db9 2023-09-10T00:39:59.272 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 smithi044 running (6m) - 6m 74.7M 4293M 18.0.0-6025-g9670cca5 20c3861ef6c9 dc25c5523a3c 2023-09-10T00:39:59.272 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 smithi099 running (5m) - 5m 77.5M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 305624f0a966 2023-09-10T00:39:59.273 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 smithi131 running (5m) - 5m 75.4M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 fa931b7fa5bd 2023-09-10T00:39:59.273 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 smithi044 running (4m) - 4m 66.1M 4293M 18.0.0-6025-g9670cca5 20c3861ef6c9 ce536b854dcf 2023-09-10T00:39:59.273 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 smithi099 running (4m) - 4m 64.4M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 a958eafa55f2 2023-09-10T00:39:59.273 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 smithi131 running (4m) - 4m 69.1M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 86f26e355c2e 2023-09-10T00:39:59.273 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 smithi044 running (3m) - 3m 61.4M 4293M 18.0.0-6025-g9670cca5 20c3861ef6c9 5fb33e8e90b1 2023-09-10T00:39:59.273 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 smithi099 running (3m) - 3m 68.5M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 69610e5e0829 2023-09-10T00:39:59.274 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8 smithi131 running (3m) - 3m 64.3M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 29f0d5c269a9 2023-09-10T00:39:59.274 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9 smithi044 running (2m) - 2m 60.8M 4293M 18.0.0-6025-g9670cca5 20c3861ef6c9 35932f7205b8 2023-09-10T00:39:59.274 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10 smithi099 running (2m) - 2m 63.6M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 9793b126487a 2023-09-10T00:39:59.274 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11 smithi131 running (2m) - 2m 57.5M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 7690c70a60f2 2023-09-10T00:39:59.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.270+0000 7f1d566e5700 1 -- 172.21.15.44:0/2744949527 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1d380a0750 msgr2=0x7f1d380a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:59.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.270+0000 7f1d566e5700 1 --2- 172.21.15.44:0/2744949527 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1d380a0750 0x7f1d380a2c10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000b5c0 tx=0x7f1d40005fb0 comp rx=0 tx=0).stop 2023-09-10T00:39:59.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.270+0000 7f1d566e5700 1 -- 172.21.15.44:0/2744949527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d50100230 msgr2=0x7f1d5019fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:39:59.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.270+0000 7f1d566e5700 1 --2- 172.21.15.44:0/2744949527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d50100230 0x7f1d5019fa00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1d48007f10 tx=0x7f1d4800d3b0 comp rx=0 tx=0).stop 2023-09-10T00:39:59.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.270+0000 7f1d566e5700 1 -- 172.21.15.44:0/2744949527 shutdown_connections 2023-09-10T00:39:59.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.270+0000 7f1d566e5700 1 --2- 172.21.15.44:0/2744949527 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1d380a0750 0x7f1d380a2c10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:59.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.270+0000 7f1d566e5700 1 --2- 172.21.15.44:0/2744949527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d50100230 0x7f1d5019fa00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:59.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.270+0000 7f1d566e5700 1 --2- 172.21.15.44:0/2744949527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d500fef10 0x7f1d50198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:59.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.270+0000 7f1d566e5700 1 --2- 172.21.15.44:0/2744949527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d500ff870 0x7f1d50198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:39:59.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.270+0000 7f1d566e5700 1 -- 172.21.15.44:0/2744949527 >> 172.21.15.44:0/2744949527 conn(0x7f1d500fab10 msgr2=0x7f1d500fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:39:59.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.270+0000 7f1d566e5700 1 -- 172.21.15.44:0/2744949527 shutdown_connections 2023-09-10T00:39:59.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:39:59.270+0000 7f1d566e5700 1 -- 172.21.15.44:0/2744949527 wait complete. 2023-09-10T00:39:59.665 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch ls' 2023-09-10T00:40:00.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:39:59 smithi131 ceph-mon[103713]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:40:00.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:39:59 smithi044 ceph-mon[98676]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:40:00.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:39:59 smithi099 ceph-mon[102692]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:40:00.236 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:40:00.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:00 smithi044 ceph-mon[98676]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:00.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:00 smithi044 ceph-mon[98676]: overall HEALTH_OK 2023-09-10T00:40:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:00 smithi131 ceph-mon[103713]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:00 smithi131 ceph-mon[103713]: overall HEALTH_OK 2023-09-10T00:40:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:00 smithi099 ceph-mon[102692]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:00 smithi099 ceph-mon[102692]: overall HEALTH_OK 2023-09-10T00:40:01.529 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.529+0000 7f96745e1700 1 -- 172.21.15.44:0/3289132560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f966c0ff590 msgr2=0x7f966c0ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:01.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.529+0000 7f96745e1700 1 --2- 172.21.15.44:0/3289132560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f966c0ff590 0x7f966c0ff9b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f96680099c0 tx=0x7f9668009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:01.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.529+0000 7f96745e1700 1 -- 172.21.15.44:0/3289132560 shutdown_connections 2023-09-10T00:40:01.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.529+0000 7f96745e1700 1 --2- 172.21.15.44:0/3289132560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966c101130 0x7f966c1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:01.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.529+0000 7f96745e1700 1 --2- 172.21.15.44:0/3289132560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f966c0ff590 0x7f966c0ff9b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:01.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.529+0000 7f96745e1700 1 --2- 172.21.15.44:0/3289132560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f966c100790 0x7f966c100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:01.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.529+0000 7f96745e1700 1 -- 172.21.15.44:0/3289132560 >> 172.21.15.44:0/3289132560 conn(0x7f966c0fab10 msgr2=0x7f966c0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:01.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.529+0000 7f96745e1700 1 -- 172.21.15.44:0/3289132560 shutdown_connections 2023-09-10T00:40:01.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.529+0000 7f96745e1700 1 -- 172.21.15.44:0/3289132560 wait complete. 2023-09-10T00:40:01.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.530+0000 7f96745e1700 1 Processor -- start 2023-09-10T00:40:01.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.530+0000 7f96745e1700 1 -- start start 2023-09-10T00:40:01.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.530+0000 7f96745e1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966c0ff590 0x7f966c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:01.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.530+0000 7f96745e1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f966c100790 0x7f966c19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:01.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.530+0000 7f96745e1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f966c101130 0x7f966c19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:01.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.530+0000 7f96745e1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c19ddb0 con 0x7f966c100790 2023-09-10T00:40:01.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.530+0000 7f96745e1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c19df20 con 0x7f966c0ff590 2023-09-10T00:40:01.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.530+0000 7f96745e1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c19e0e0 con 0x7f966c101130 2023-09-10T00:40:01.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.530+0000 7f9671b7c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f966c100790 0x7f966c19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:01.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.530+0000 7f9671b7c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f966c100790 0x7f966c19afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47764/0 (socket says 172.21.15.44:47764) 2023-09-10T00:40:01.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.530+0000 7f9671b7c700 1 -- 172.21.15.44:0/921948911 learned_addr learned my addr 172.21.15.44:0/921948911 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.531+0000 7f967237d700 1 --2- 172.21.15.44:0/921948911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966c0ff590 0x7f966c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.531+0000 7f9672b7e700 1 --2- 172.21.15.44:0/921948911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f966c101130 0x7f966c19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.531+0000 7f9671b7c700 1 -- 172.21.15.44:0/921948911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f966c101130 msgr2=0x7f966c19d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.531+0000 7f9671b7c700 1 --2- 172.21.15.44:0/921948911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f966c101130 0x7f966c19d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.531+0000 7f9671b7c700 1 -- 172.21.15.44:0/921948911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966c0ff590 msgr2=0x7f966c068c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.531+0000 7f9671b7c700 1 --2- 172.21.15.44:0/921948911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966c0ff590 0x7f966c068c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.531+0000 7f9671b7c700 1 -- 172.21.15.44:0/921948911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96680095f0 con 0x7f966c100790 2023-09-10T00:40:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.531+0000 7f9672b7e700 1 --2- 172.21.15.44:0/921948911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f966c101130 0x7f966c19d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:40:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.531+0000 7f9671b7c700 1 --2- 172.21.15.44:0/921948911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f966c100790 0x7f966c19afc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f965c00d950 tx=0x7f965c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.531+0000 7f96637fe700 1 -- 172.21.15.44:0/921948911 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c009940 con 0x7f966c100790 2023-09-10T00:40:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.531+0000 7f96745e1700 1 -- 172.21.15.44:0/921948911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966c19e3c0 con 0x7f966c100790 2023-09-10T00:40:01.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.531+0000 7f96637fe700 1 -- 172.21.15.44:0/921948911 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f965c010460 con 0x7f966c100790 2023-09-10T00:40:01.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.531+0000 7f96637fe700 1 -- 172.21.15.44:0/921948911 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c00f620 con 0x7f966c100790 2023-09-10T00:40:01.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.531+0000 7f96745e1700 1 -- 172.21.15.44:0/921948911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966c19e8c0 con 0x7f966c100790 2023-09-10T00:40:01.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.532+0000 7f96745e1700 1 -- 172.21.15.44:0/921948911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f966c10ac60 con 0x7f966c100790 2023-09-10T00:40:01.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.536+0000 7f96637fe700 1 -- 172.21.15.44:0/921948911 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f965c0105d0 con 0x7f966c100790 2023-09-10T00:40:01.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.536+0000 7f96637fe700 1 --2- 172.21.15.44:0/921948911 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f96580a0750 0x7f96580a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:01.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.536+0000 7f96637fe700 1 -- 172.21.15.44:0/921948911 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f965c0cbe90 con 0x7f966c100790 2023-09-10T00:40:01.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.536+0000 7f967237d700 1 --2- 172.21.15.44:0/921948911 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f96580a0750 0x7f96580a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:01.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.537+0000 7f967237d700 1 --2- 172.21.15.44:0/921948911 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f96580a0750 0x7f96580a2c10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f966800c010 tx=0x7f9668005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:01.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.538+0000 7f96637fe700 1 -- 172.21.15.44:0/921948911 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f965c0954f0 con 0x7f966c100790 2023-09-10T00:40:01.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.747+0000 7f96745e1700 1 -- 172.21.15.44:0/921948911 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f966c060a40 con 0x7f96580a0750 2023-09-10T00:40:01.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.754+0000 7f96637fe700 1 -- 172.21.15.44:0/921948911 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7f966c060a40 con 0x7f96580a0750 2023-09-10T00:40:01.754 INFO:teuthology.orchestra.run.smithi044.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-09-10T00:40:01.755 INFO:teuthology.orchestra.run.smithi044.stdout:mds.all 12/12 - 115s smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l;count:12 2023-09-10T00:40:01.755 INFO:teuthology.orchestra.run.smithi044.stdout:mgr 3/3 - 6m smithi044=x;smithi099=y;smithi131=z;count:3 2023-09-10T00:40:01.755 INFO:teuthology.orchestra.run.smithi044.stdout:mon 3/3 - 7m smithi044:172.21.15.44=a;smithi099:172.21.15.99=b;smithi131:172.21.15.131=c;count:3 2023-09-10T00:40:01.755 INFO:teuthology.orchestra.run.smithi044.stdout:osd 12 - - 2023-09-10T00:40:01.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.756+0000 7f96745e1700 1 -- 172.21.15.44:0/921948911 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f96580a0750 msgr2=0x7f96580a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:01.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.756+0000 7f96745e1700 1 --2- 172.21.15.44:0/921948911 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f96580a0750 0x7f96580a2c10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f966800c010 tx=0x7f9668005e80 comp rx=0 tx=0).stop 2023-09-10T00:40:01.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.756+0000 7f96745e1700 1 -- 172.21.15.44:0/921948911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f966c100790 msgr2=0x7f966c19afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:01.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.756+0000 7f96745e1700 1 --2- 172.21.15.44:0/921948911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f966c100790 0x7f966c19afc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f965c00d950 tx=0x7f965c00dc60 comp rx=0 tx=0).stop 2023-09-10T00:40:01.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.756+0000 7f96745e1700 1 -- 172.21.15.44:0/921948911 shutdown_connections 2023-09-10T00:40:01.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.756+0000 7f96745e1700 1 --2- 172.21.15.44:0/921948911 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f96580a0750 0x7f96580a2c10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:01.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.756+0000 7f96745e1700 1 --2- 172.21.15.44:0/921948911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f966c101130 0x7f966c19d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:01.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.756+0000 7f96745e1700 1 --2- 172.21.15.44:0/921948911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966c0ff590 0x7f966c068c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:01.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.756+0000 7f96745e1700 1 --2- 172.21.15.44:0/921948911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f966c100790 0x7f966c19afc0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:01.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.756+0000 7f96745e1700 1 -- 172.21.15.44:0/921948911 >> 172.21.15.44:0/921948911 conn(0x7f966c0fab10 msgr2=0x7f966c0fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:01.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.756+0000 7f96745e1700 1 -- 172.21.15.44:0/921948911 shutdown_connections 2023-09-10T00:40:01.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:01.756+0000 7f96745e1700 1 -- 172.21.15.44:0/921948911 wait complete. 2023-09-10T00:40:01.904 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:01 smithi044 ceph-mon[98676]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-10T00:40:02.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:01 smithi131 ceph-mon[103713]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-10T00:40:02.104 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch host ls' 2023-09-10T00:40:02.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:01 smithi099 ceph-mon[102692]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-10T00:40:02.670 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:40:02.895 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:02 smithi044 ceph-mon[98676]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:02 smithi131 ceph-mon[103713]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:02 smithi099 ceph-mon[102692]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:03.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.889+0000 7fe5a34e1700 1 -- 172.21.15.44:0/2456972114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe59c103ab0 msgr2=0x7fe59c103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:03.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.889+0000 7fe5a34e1700 1 --2- 172.21.15.44:0/2456972114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe59c103ab0 0x7fe59c103f10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe5980099c0 tx=0x7fe598009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:03.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.889+0000 7fe5a34e1700 1 -- 172.21.15.44:0/2456972114 shutdown_connections 2023-09-10T00:40:03.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.889+0000 7fe5a34e1700 1 --2- 172.21.15.44:0/2456972114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe59c104450 0x7fe59c10ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:03.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.889+0000 7fe5a34e1700 1 --2- 172.21.15.44:0/2456972114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe59c1028b0 0x7fe59c102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:03.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.889+0000 7fe5a34e1700 1 --2- 172.21.15.44:0/2456972114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe59c103ab0 0x7fe59c103f10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:03.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.889+0000 7fe5a34e1700 1 -- 172.21.15.44:0/2456972114 >> 172.21.15.44:0/2456972114 conn(0x7fe59c0fde10 msgr2=0x7fe59c100290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:03.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.889+0000 7fe5a34e1700 1 -- 172.21.15.44:0/2456972114 shutdown_connections 2023-09-10T00:40:03.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.889+0000 7fe5a34e1700 1 -- 172.21.15.44:0/2456972114 wait complete. 2023-09-10T00:40:03.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a34e1700 1 Processor -- start 2023-09-10T00:40:03.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a34e1700 1 -- start start 2023-09-10T00:40:03.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a34e1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe59c1028b0 0x7fe59c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:03.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a34e1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe59c103ab0 0x7fe59c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:03.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a34e1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe59c104450 0x7fe59c19faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:03.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a34e1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c1a0070 con 0x7fe59c103ab0 2023-09-10T00:40:03.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a34e1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c1a01b0 con 0x7fe59c104450 2023-09-10T00:40:03.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a34e1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c1a0340 con 0x7fe59c1028b0 2023-09-10T00:40:03.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a0a7c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe59c103ab0 0x7fe59c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:03.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a0a7c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe59c103ab0 0x7fe59c198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47774/0 (socket says 172.21.15.44:47774) 2023-09-10T00:40:03.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a0a7c700 1 -- 172.21.15.44:0/3680953807 learned_addr learned my addr 172.21.15.44:0/3680953807 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:03.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a1a7e700 1 --2- 172.21.15.44:0/3680953807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe59c104450 0x7fe59c19faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:03.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a127d700 1 --2- 172.21.15.44:0/3680953807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe59c1028b0 0x7fe59c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:03.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a0a7c700 1 -- 172.21.15.44:0/3680953807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe59c1028b0 msgr2=0x7fe59c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:03.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a0a7c700 1 --2- 172.21.15.44:0/3680953807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe59c1028b0 0x7fe59c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:03.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a0a7c700 1 -- 172.21.15.44:0/3680953807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe59c104450 msgr2=0x7fe59c19faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:03.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a0a7c700 1 --2- 172.21.15.44:0/3680953807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe59c104450 0x7fe59c19faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:03.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a0a7c700 1 -- 172.21.15.44:0/3680953807 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5980095f0 con 0x7fe59c103ab0 2023-09-10T00:40:03.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.890+0000 7fe5a0a7c700 1 --2- 172.21.15.44:0/3680953807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe59c103ab0 0x7fe59c198a20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe59800b580 tx=0x7fe598003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:03.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.891+0000 7fe58e7fc700 1 -- 172.21.15.44:0/3680953807 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59801d070 con 0x7fe59c103ab0 2023-09-10T00:40:03.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.891+0000 7fe58e7fc700 1 -- 172.21.15.44:0/3680953807 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe59800bd30 con 0x7fe59c103ab0 2023-09-10T00:40:03.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.891+0000 7fe5a34e1700 1 -- 172.21.15.44:0/3680953807 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59c1a05c0 con 0x7fe59c103ab0 2023-09-10T00:40:03.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.891+0000 7fe58e7fc700 1 -- 172.21.15.44:0/3680953807 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598022cd0 con 0x7fe59c103ab0 2023-09-10T00:40:03.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.891+0000 7fe5a34e1700 1 -- 172.21.15.44:0/3680953807 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59c1a09d0 con 0x7fe59c103ab0 2023-09-10T00:40:03.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.892+0000 7fe5a34e1700 1 -- 172.21.15.44:0/3680953807 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59c109f80 con 0x7fe59c103ab0 2023-09-10T00:40:03.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.893+0000 7fe58e7fc700 1 -- 172.21.15.44:0/3680953807 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fe598022470 con 0x7fe59c103ab0 2023-09-10T00:40:03.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.893+0000 7fe58e7fc700 1 --2- 172.21.15.44:0/3680953807 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5880a06d0 0x7fe5880a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:03.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.894+0000 7fe58e7fc700 1 -- 172.21.15.44:0/3680953807 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe5980cd910 con 0x7fe59c103ab0 2023-09-10T00:40:03.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.894+0000 7fe5a127d700 1 --2- 172.21.15.44:0/3680953807 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5880a06d0 0x7fe5880a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:03.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.894+0000 7fe5a127d700 1 --2- 172.21.15.44:0/3680953807 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5880a06d0 0x7fe5880a2b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe59c103910 tx=0x7fe59000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:03.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:03.897+0000 7fe58e7fc700 1 -- 172.21.15.44:0/3680953807 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe598097060 con 0x7fe59c103ab0 2023-09-10T00:40:04.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:03 smithi131 ceph-mon[103713]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-10T00:40:04.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:03 smithi044 ceph-mon[98676]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-10T00:40:04.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:03 smithi099 ceph-mon[102692]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-10T00:40:04.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:04.106+0000 7fe5a34e1700 1 -- 172.21.15.44:0/3680953807 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe59c02d090 con 0x7fe5880a06d0 2023-09-10T00:40:04.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:04.107+0000 7fe58e7fc700 1 -- 172.21.15.44:0/3680953807 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fe59c02d090 con 0x7fe5880a06d0 2023-09-10T00:40:04.108 INFO:teuthology.orchestra.run.smithi044.stdout:HOST ADDR LABELS STATUS 2023-09-10T00:40:04.108 INFO:teuthology.orchestra.run.smithi044.stdout:smithi044 172.21.15.44 2023-09-10T00:40:04.108 INFO:teuthology.orchestra.run.smithi044.stdout:smithi099 172.21.15.99 2023-09-10T00:40:04.109 INFO:teuthology.orchestra.run.smithi044.stdout:smithi131 172.21.15.131 2023-09-10T00:40:04.109 INFO:teuthology.orchestra.run.smithi044.stdout:3 hosts in cluster 2023-09-10T00:40:04.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:04.109+0000 7fe5a34e1700 1 -- 172.21.15.44:0/3680953807 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5880a06d0 msgr2=0x7fe5880a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:04.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:04.109+0000 7fe5a34e1700 1 --2- 172.21.15.44:0/3680953807 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5880a06d0 0x7fe5880a2b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe59c103910 tx=0x7fe59000b410 comp rx=0 tx=0).stop 2023-09-10T00:40:04.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:04.109+0000 7fe5a34e1700 1 -- 172.21.15.44:0/3680953807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe59c103ab0 msgr2=0x7fe59c198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:04.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:04.109+0000 7fe5a34e1700 1 --2- 172.21.15.44:0/3680953807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe59c103ab0 0x7fe59c198a20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe59800b580 tx=0x7fe598003a50 comp rx=0 tx=0).stop 2023-09-10T00:40:04.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:04.109+0000 7fe5a34e1700 1 -- 172.21.15.44:0/3680953807 shutdown_connections 2023-09-10T00:40:04.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:04.109+0000 7fe5a34e1700 1 --2- 172.21.15.44:0/3680953807 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5880a06d0 0x7fe5880a2b90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:04.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:04.109+0000 7fe5a34e1700 1 --2- 172.21.15.44:0/3680953807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe59c104450 0x7fe59c19faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:04.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:04.109+0000 7fe5a34e1700 1 --2- 172.21.15.44:0/3680953807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe59c1028b0 0x7fe59c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:04.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:04.109+0000 7fe5a34e1700 1 --2- 172.21.15.44:0/3680953807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe59c103ab0 0x7fe59c198a20 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:04.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:04.110+0000 7fe5a34e1700 1 -- 172.21.15.44:0/3680953807 >> 172.21.15.44:0/3680953807 conn(0x7fe59c0fde10 msgr2=0x7fe59c1070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:04.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:04.110+0000 7fe5a34e1700 1 -- 172.21.15.44:0/3680953807 shutdown_connections 2023-09-10T00:40:04.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:04.110+0000 7fe5a34e1700 1 -- 172.21.15.44:0/3680953807 wait complete. 2023-09-10T00:40:04.424 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch device ls' 2023-09-10T00:40:04.987 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:40:05.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:05 smithi044 ceph-mon[98676]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:40:05.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:05 smithi044 ceph-mon[98676]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:06.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:05 smithi131 ceph-mon[103713]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:40:06.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:05 smithi131 ceph-mon[103713]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:06.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:05 smithi099 ceph-mon[102692]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:40:06.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:05 smithi099 ceph-mon[102692]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.214+0000 7fd693193700 1 -- 172.21.15.44:0/179307849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c0ff590 msgr2=0x7fd68c0ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.214+0000 7fd693193700 1 --2- 172.21.15.44:0/179307849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c0ff590 0x7fd68c0ff9b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd6800099c0 tx=0x7fd680009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.214+0000 7fd693193700 1 -- 172.21.15.44:0/179307849 shutdown_connections 2023-09-10T00:40:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.214+0000 7fd693193700 1 --2- 172.21.15.44:0/179307849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68c101130 0x7fd68c1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.214+0000 7fd693193700 1 --2- 172.21.15.44:0/179307849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c0ff590 0x7fd68c0ff9b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.214+0000 7fd693193700 1 --2- 172.21.15.44:0/179307849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c100790 0x7fd68c100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.214+0000 7fd693193700 1 -- 172.21.15.44:0/179307849 >> 172.21.15.44:0/179307849 conn(0x7fd68c0fab10 msgr2=0x7fd68c0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.215+0000 7fd693193700 1 -- 172.21.15.44:0/179307849 shutdown_connections 2023-09-10T00:40:06.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.215+0000 7fd693193700 1 -- 172.21.15.44:0/179307849 wait complete. 2023-09-10T00:40:06.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.215+0000 7fd693193700 1 Processor -- start 2023-09-10T00:40:06.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.215+0000 7fd693193700 1 -- start start 2023-09-10T00:40:06.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.215+0000 7fd693193700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c0ff590 0x7fd68c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:06.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.215+0000 7fd693193700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68c100790 0x7fd68c19b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:06.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.215+0000 7fd693193700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c101130 0x7fd68c19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.215+0000 7fd693193700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c19de80 con 0x7fd68c101130 2023-09-10T00:40:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.215+0000 7fd693193700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c19dff0 con 0x7fd68c100790 2023-09-10T00:40:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.215+0000 7fd693193700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c19e1b0 con 0x7fd68c0ff590 2023-09-10T00:40:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.216+0000 7fd691730700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c101130 0x7fd68c19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.216+0000 7fd691730700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c101130 0x7fd68c19d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47788/0 (socket says 172.21.15.44:47788) 2023-09-10T00:40:06.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.216+0000 7fd691730700 1 -- 172.21.15.44:0/2657976499 learned_addr learned my addr 172.21.15.44:0/2657976499 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:06.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.216+0000 7fd68bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68c100790 0x7fd68c19b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:06.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.216+0000 7fd690f2f700 1 --2- 172.21.15.44:0/2657976499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c0ff590 0x7fd68c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:06.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.216+0000 7fd691730700 1 -- 172.21.15.44:0/2657976499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c0ff590 msgr2=0x7fd68c068c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:06.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.216+0000 7fd691730700 1 --2- 172.21.15.44:0/2657976499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c0ff590 0x7fd68c068c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:06.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.216+0000 7fd691730700 1 -- 172.21.15.44:0/2657976499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68c100790 msgr2=0x7fd68c19b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:06.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.216+0000 7fd691730700 1 --2- 172.21.15.44:0/2657976499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68c100790 0x7fd68c19b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:06.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.216+0000 7fd691730700 1 -- 172.21.15.44:0/2657976499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6800095f0 con 0x7fd68c101130 2023-09-10T00:40:06.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.216+0000 7fd691730700 1 --2- 172.21.15.44:0/2657976499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c101130 0x7fd68c19d6e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd68400ef30 tx=0x7fd68400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:06.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.216+0000 7fd689ffb700 1 -- 172.21.15.44:0/2657976499 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684018070 con 0x7fd68c101130 2023-09-10T00:40:06.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.216+0000 7fd689ffb700 1 -- 172.21.15.44:0/2657976499 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd68400f040 con 0x7fd68c101130 2023-09-10T00:40:06.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.216+0000 7fd693193700 1 -- 172.21.15.44:0/2657976499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68c19e430 con 0x7fd68c101130 2023-09-10T00:40:06.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.216+0000 7fd689ffb700 1 -- 172.21.15.44:0/2657976499 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68400cbc0 con 0x7fd68c101130 2023-09-10T00:40:06.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.216+0000 7fd693193700 1 -- 172.21.15.44:0/2657976499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd68c19e930 con 0x7fd68c101130 2023-09-10T00:40:06.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.217+0000 7fd693193700 1 -- 172.21.15.44:0/2657976499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd68c106c20 con 0x7fd68c101130 2023-09-10T00:40:06.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.218+0000 7fd689ffb700 1 -- 172.21.15.44:0/2657976499 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fd684007500 con 0x7fd68c101130 2023-09-10T00:40:06.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.219+0000 7fd689ffb700 1 --2- 172.21.15.44:0/2657976499 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd67c0a0750 0x7fd67c0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:06.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.219+0000 7fd689ffb700 1 -- 172.21.15.44:0/2657976499 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd6840cbac0 con 0x7fd68c101130 2023-09-10T00:40:06.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.219+0000 7fd690f2f700 1 --2- 172.21.15.44:0/2657976499 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd67c0a0750 0x7fd67c0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:06.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.219+0000 7fd690f2f700 1 --2- 172.21.15.44:0/2657976499 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd67c0a0750 0x7fd67c0a2c10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd680009990 tx=0x7fd68000ba90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:06.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.223+0000 7fd689ffb700 1 -- 172.21.15.44:0/2657976499 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd684095210 con 0x7fd68c101130 2023-09-10T00:40:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.431+0000 7fd693193700 1 -- 172.21.15.44:0/2657976499 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd68c02d010 con 0x7fd67c0a0750 2023-09-10T00:40:06.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.433+0000 7fd689ffb700 1 -- 172.21.15.44:0/2657976499 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fd68c02d010 con 0x7fd67c0a0750 2023-09-10T00:40:06.434 INFO:teuthology.orchestra.run.smithi044.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-09-10T00:40:06.434 INFO:teuthology.orchestra.run.smithi044.stdout:smithi044 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001K400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-09-10T00:40:06.434 INFO:teuthology.orchestra.run.smithi044.stdout:smithi099 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040101400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-09-10T00:40:06.434 INFO:teuthology.orchestra.run.smithi044.stdout:smithi131 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300N4400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-09-10T00:40:06.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.436+0000 7fd693193700 1 -- 172.21.15.44:0/2657976499 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd67c0a0750 msgr2=0x7fd67c0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:06.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.436+0000 7fd693193700 1 --2- 172.21.15.44:0/2657976499 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd67c0a0750 0x7fd67c0a2c10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd680009990 tx=0x7fd68000ba90 comp rx=0 tx=0).stop 2023-09-10T00:40:06.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.436+0000 7fd693193700 1 -- 172.21.15.44:0/2657976499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c101130 msgr2=0x7fd68c19d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:06.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.436+0000 7fd693193700 1 --2- 172.21.15.44:0/2657976499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c101130 0x7fd68c19d6e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd68400ef30 tx=0x7fd68400c3a0 comp rx=0 tx=0).stop 2023-09-10T00:40:06.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.436+0000 7fd693193700 1 -- 172.21.15.44:0/2657976499 shutdown_connections 2023-09-10T00:40:06.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.436+0000 7fd693193700 1 --2- 172.21.15.44:0/2657976499 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd67c0a0750 0x7fd67c0a2c10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:06.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.436+0000 7fd693193700 1 --2- 172.21.15.44:0/2657976499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c101130 0x7fd68c19d6e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:06.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.436+0000 7fd693193700 1 --2- 172.21.15.44:0/2657976499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c0ff590 0x7fd68c068c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:06.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.436+0000 7fd693193700 1 --2- 172.21.15.44:0/2657976499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68c100790 0x7fd68c19b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:06.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.436+0000 7fd693193700 1 -- 172.21.15.44:0/2657976499 >> 172.21.15.44:0/2657976499 conn(0x7fd68c0fab10 msgr2=0x7fd68c103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:06.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.436+0000 7fd693193700 1 -- 172.21.15.44:0/2657976499 shutdown_connections 2023-09-10T00:40:06.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:06.436+0000 7fd693193700 1 -- 172.21.15.44:0/2657976499 wait complete. 2023-09-10T00:40:06.739 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-10T00:40:06.749 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2023-09-10T00:40:06.749 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph fs dump' 2023-09-10T00:40:07.314 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:40:07.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:07 smithi044 ceph-mon[98676]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-09-10T00:40:07.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:07 smithi044 ceph-mon[98676]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:08.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:07 smithi131 ceph-mon[103713]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-09-10T00:40:08.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:07 smithi131 ceph-mon[103713]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:08.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:07 smithi099 ceph-mon[102692]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-09-10T00:40:08.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:07 smithi099 ceph-mon[102692]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:08.510 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.510+0000 7f9dd653a700 1 -- 172.21.15.44:0/3245438433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd01007c0 msgr2=0x7f9dd0100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:08.511 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.510+0000 7f9dd653a700 1 --2- 172.21.15.44:0/3245438433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd01007c0 0x7f9dd0100c20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00099c0 tx=0x7f9dc0009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:08.511 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.510+0000 7f9dd653a700 1 -- 172.21.15.44:0/3245438433 shutdown_connections 2023-09-10T00:40:08.511 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.510+0000 7f9dd653a700 1 --2- 172.21.15.44:0/3245438433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd0101160 0x7f9dd0107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:08.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.510+0000 7f9dd653a700 1 --2- 172.21.15.44:0/3245438433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd00ff5c0 0x7f9dd00ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:08.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.510+0000 7f9dd653a700 1 --2- 172.21.15.44:0/3245438433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd01007c0 0x7f9dd0100c20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:08.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.510+0000 7f9dd653a700 1 -- 172.21.15.44:0/3245438433 >> 172.21.15.44:0/3245438433 conn(0x7f9dd00fab80 msgr2=0x7f9dd00fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:08.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.510+0000 7f9dd653a700 1 -- 172.21.15.44:0/3245438433 shutdown_connections 2023-09-10T00:40:08.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.510+0000 7f9dd653a700 1 -- 172.21.15.44:0/3245438433 wait complete. 2023-09-10T00:40:08.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.511+0000 7f9dd653a700 1 Processor -- start 2023-09-10T00:40:08.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.511+0000 7f9dd653a700 1 -- start start 2023-09-10T00:40:08.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.511+0000 7f9dd653a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd00ff5c0 0x7f9dd0069540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:08.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.511+0000 7f9dd653a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd01007c0 0x7f9dd019b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:08.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.511+0000 7f9dd653a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd0101160 0x7f9dd019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:08.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.511+0000 7f9dd653a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd019de10 con 0x7f9dd0101160 2023-09-10T00:40:08.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.511+0000 7f9dd653a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd019df80 con 0x7f9dd00ff5c0 2023-09-10T00:40:08.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.511+0000 7f9dd653a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd019e140 con 0x7f9dd01007c0 2023-09-10T00:40:08.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.511+0000 7f9dd4ad7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd0101160 0x7f9dd019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:08.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.511+0000 7f9dd4ad7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd0101160 0x7f9dd019d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47800/0 (socket says 172.21.15.44:47800) 2023-09-10T00:40:08.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.511+0000 7f9dd4ad7700 1 -- 172.21.15.44:0/3359293194 learned_addr learned my addr 172.21.15.44:0/3359293194 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:08.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.511+0000 7f9dcffff700 1 --2- 172.21.15.44:0/3359293194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd00ff5c0 0x7f9dd0069540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:08.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.512+0000 7f9dcf7fe700 1 --2- 172.21.15.44:0/3359293194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd01007c0 0x7f9dd019b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:08.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.512+0000 7f9dd4ad7700 1 -- 172.21.15.44:0/3359293194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd01007c0 msgr2=0x7f9dd019b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:08.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.512+0000 7f9dd4ad7700 1 --2- 172.21.15.44:0/3359293194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd01007c0 0x7f9dd019b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:08.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.512+0000 7f9dd4ad7700 1 -- 172.21.15.44:0/3359293194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd00ff5c0 msgr2=0x7f9dd0069540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:08.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.512+0000 7f9dd4ad7700 1 --2- 172.21.15.44:0/3359293194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd00ff5c0 0x7f9dd0069540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:08.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.512+0000 7f9dd4ad7700 1 -- 172.21.15.44:0/3359293194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc00095f0 con 0x7f9dd0101160 2023-09-10T00:40:08.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.512+0000 7f9dd4ad7700 1 --2- 172.21.15.44:0/3359293194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd0101160 0x7f9dd019d670 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8007ee0 tx=0x7f9dc800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:08.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.512+0000 7f9dcd7fa700 1 -- 172.21.15.44:0/3359293194 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8017070 con 0x7f9dd0101160 2023-09-10T00:40:08.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.512+0000 7f9dcd7fa700 1 -- 172.21.15.44:0/3359293194 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9dc800f040 con 0x7f9dd0101160 2023-09-10T00:40:08.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.512+0000 7f9dd653a700 1 -- 172.21.15.44:0/3359293194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd019e420 con 0x7f9dd0101160 2023-09-10T00:40:08.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.512+0000 7f9dcd7fa700 1 -- 172.21.15.44:0/3359293194 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8005710 con 0x7f9dd0101160 2023-09-10T00:40:08.519 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.512+0000 7f9dd653a700 1 -- 172.21.15.44:0/3359293194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd019e920 con 0x7f9dd0101160 2023-09-10T00:40:08.519 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.513+0000 7f9dd653a700 1 -- 172.21.15.44:0/3359293194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd010acd0 con 0x7f9dd0101160 2023-09-10T00:40:08.519 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.514+0000 7f9dcd7fa700 1 -- 172.21.15.44:0/3359293194 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f9dc8005870 con 0x7f9dd0101160 2023-09-10T00:40:08.520 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.515+0000 7f9dcd7fa700 1 --2- 172.21.15.44:0/3359293194 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9dbc0a0750 0x7f9dbc0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:08.520 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.515+0000 7f9dcffff700 1 --2- 172.21.15.44:0/3359293194 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9dbc0a0750 0x7f9dbc0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:08.520 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.515+0000 7f9dcd7fa700 1 -- 172.21.15.44:0/3359293194 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9dc80cb820 con 0x7f9dd0101160 2023-09-10T00:40:08.521 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.515+0000 7f9dcffff700 1 --2- 172.21.15.44:0/3359293194 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9dbc0a0750 0x7f9dbc0a2c10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0100620 tx=0x7f9db800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:08.521 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.519+0000 7f9dcd7fa700 1 -- 172.21.15.44:0/3359293194 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9dc8094e80 con 0x7f9dd0101160 2023-09-10T00:40:08.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.766+0000 7f9dd653a700 1 -- 172.21.15.44:0/3359293194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9dd01065b0 con 0x7f9dd0101160 2023-09-10T00:40:08.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.767+0000 7f9dcd7fa700 1 -- 172.21.15.44:0/3359293194 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2973 (secure 0 0 0) 0x7f9dc80984a0 con 0x7f9dd0101160 2023-09-10T00:40:08.767 INFO:teuthology.orchestra.run.smithi044.stdout:e27 2023-09-10T00:40:08.768 INFO:teuthology.orchestra.run.smithi044.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-09-10T00:40:08.768 INFO:teuthology.orchestra.run.smithi044.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-10T00:40:08.768 INFO:teuthology.orchestra.run.smithi044.stdout:legacy client fscid: 1 2023-09-10T00:40:08.768 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:08.768 INFO:teuthology.orchestra.run.smithi044.stdout:Filesystem 'cephfs' (1) 2023-09-10T00:40:08.769 INFO:teuthology.orchestra.run.smithi044.stdout:fs_name cephfs 2023-09-10T00:40:08.769 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 27 2023-09-10T00:40:08.769 INFO:teuthology.orchestra.run.smithi044.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-09-10T00:40:08.769 INFO:teuthology.orchestra.run.smithi044.stdout:created 2023-09-10T00:38:39.183533+0000 2023-09-10T00:40:08.769 INFO:teuthology.orchestra.run.smithi044.stdout:modified 2023-09-10T00:39:02.480227+0000 2023-09-10T00:40:08.770 INFO:teuthology.orchestra.run.smithi044.stdout:tableserver 0 2023-09-10T00:40:08.770 INFO:teuthology.orchestra.run.smithi044.stdout:root 0 2023-09-10T00:40:08.770 INFO:teuthology.orchestra.run.smithi044.stdout:session_timeout 300 2023-09-10T00:40:08.770 INFO:teuthology.orchestra.run.smithi044.stdout:session_autoclose 300 2023-09-10T00:40:08.770 INFO:teuthology.orchestra.run.smithi044.stdout:max_file_size 1099511627776 2023-09-10T00:40:08.771 INFO:teuthology.orchestra.run.smithi044.stdout:max_xattr_size 65536 2023-09-10T00:40:08.771 INFO:teuthology.orchestra.run.smithi044.stdout:required_client_features {} 2023-09-10T00:40:08.771 INFO:teuthology.orchestra.run.smithi044.stdout:last_failure 0 2023-09-10T00:40:08.771 INFO:teuthology.orchestra.run.smithi044.stdout:last_failure_osd_epoch 0 2023-09-10T00:40:08.771 INFO:teuthology.orchestra.run.smithi044.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-10T00:40:08.771 INFO:teuthology.orchestra.run.smithi044.stdout:max_mds 5 2023-09-10T00:40:08.772 INFO:teuthology.orchestra.run.smithi044.stdout:in 0,1,2,3,4 2023-09-10T00:40:08.772 INFO:teuthology.orchestra.run.smithi044.stdout:up {0=24605,1=24574,2=24572,3=24515,4=24502} 2023-09-10T00:40:08.772 INFO:teuthology.orchestra.run.smithi044.stdout:failed 2023-09-10T00:40:08.772 INFO:teuthology.orchestra.run.smithi044.stdout:damaged 2023-09-10T00:40:08.772 INFO:teuthology.orchestra.run.smithi044.stdout:stopped 2023-09-10T00:40:08.773 INFO:teuthology.orchestra.run.smithi044.stdout:data_pools [3,4] 2023-09-10T00:40:08.773 INFO:teuthology.orchestra.run.smithi044.stdout:metadata_pool 2 2023-09-10T00:40:08.773 INFO:teuthology.orchestra.run.smithi044.stdout:inline_data disabled 2023-09-10T00:40:08.773 INFO:teuthology.orchestra.run.smithi044.stdout:balancer 2023-09-10T00:40:08.773 INFO:teuthology.orchestra.run.smithi044.stdout:bal_rank_mask -1 2023-09-10T00:40:08.773 INFO:teuthology.orchestra.run.smithi044.stdout:standby_count_wanted 0 2023-09-10T00:40:08.774 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.b{0:24605} state up:active seq 3 addr [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] compat {c=[1],r=[1],i=[fff]}] 2023-09-10T00:40:08.774 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.c{0:24556} state up:standby-replay seq 1 addr [v2:172.21.15.131:6836/2397154745,v1:172.21.15.131:6837/2397154745] compat {c=[1],r=[1],i=[fff]}] 2023-09-10T00:40:08.774 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.i{1:24574} state up:active seq 10 addr [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] compat {c=[1],r=[1],i=[fff]}] 2023-09-10T00:40:08.774 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.f{1:24547} state up:standby-replay seq 1 addr [v2:172.21.15.131:6834/1445095724,v1:172.21.15.131:6835/1445095724] compat {c=[1],r=[1],i=[fff]}] 2023-09-10T00:40:08.774 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.h{2:24572} state up:active seq 10 addr [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] compat {c=[1],r=[1],i=[fff]}] 2023-09-10T00:40:08.775 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.e{2:24539} state up:standby-replay seq 1 addr [v2:172.21.15.99:6834/2403105238,v1:172.21.15.99:6835/2403105238] compat {c=[1],r=[1],i=[fff]}] 2023-09-10T00:40:08.775 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.k{3:24515} state up:active seq 14 addr [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] compat {c=[1],r=[1],i=[fff]}] 2023-09-10T00:40:08.775 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.a{3:14763} state up:standby-replay seq 1 addr [v2:172.21.15.44:6840/1241002552,v1:172.21.15.44:6841/1241002552] compat {c=[1],r=[1],i=[fff]}] 2023-09-10T00:40:08.775 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.l{4:24502} state up:active seq 15 addr [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] compat {c=[1],r=[1],i=[fff]}] 2023-09-10T00:40:08.775 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.g{4:14751} state up:standby-replay seq 1 addr [v2:172.21.15.44:6838/3958270510,v1:172.21.15.44:6839/3958270510] compat {c=[1],r=[1],i=[fff]}] 2023-09-10T00:40:08.775 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:08.776 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:08.776 INFO:teuthology.orchestra.run.smithi044.stdout:Standby daemons: 2023-09-10T00:40:08.776 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:08.776 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.j{-1:14685} state up:standby seq 1 addr [v2:172.21.15.44:6834/4035626069,v1:172.21.15.44:6835/4035626069] compat {c=[1],r=[1],i=[fff]}] 2023-09-10T00:40:08.776 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.d{-1:14697} state up:standby seq 1 addr [v2:172.21.15.44:6836/1919207693,v1:172.21.15.44:6837/1919207693] compat {c=[1],r=[1],i=[fff]}] 2023-09-10T00:40:08.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.769+0000 7f9dd653a700 1 -- 172.21.15.44:0/3359293194 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9dbc0a0750 msgr2=0x7f9dbc0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:08.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.769+0000 7f9dd653a700 1 --2- 172.21.15.44:0/3359293194 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9dbc0a0750 0x7f9dbc0a2c10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0100620 tx=0x7f9db800b410 comp rx=0 tx=0).stop 2023-09-10T00:40:08.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.769+0000 7f9dd653a700 1 -- 172.21.15.44:0/3359293194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd0101160 msgr2=0x7f9dd019d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:08.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.769+0000 7f9dd653a700 1 --2- 172.21.15.44:0/3359293194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd0101160 0x7f9dd019d670 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8007ee0 tx=0x7f9dc800d3b0 comp rx=0 tx=0).stop 2023-09-10T00:40:08.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.769+0000 7f9dd653a700 1 -- 172.21.15.44:0/3359293194 shutdown_connections 2023-09-10T00:40:08.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.769+0000 7f9dd653a700 1 --2- 172.21.15.44:0/3359293194 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9dbc0a0750 0x7f9dbc0a2c10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:08.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.769+0000 7f9dd653a700 1 --2- 172.21.15.44:0/3359293194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd0101160 0x7f9dd019d670 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:08.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.769+0000 7f9dd653a700 1 --2- 172.21.15.44:0/3359293194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd00ff5c0 0x7f9dd0069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:08.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.769+0000 7f9dd653a700 1 --2- 172.21.15.44:0/3359293194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd01007c0 0x7f9dd019b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:08.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.769+0000 7f9dd653a700 1 -- 172.21.15.44:0/3359293194 >> 172.21.15.44:0/3359293194 conn(0x7f9dd00fab80 msgr2=0x7f9dd00fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:08.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.769+0000 7f9dd653a700 1 -- 172.21.15.44:0/3359293194 shutdown_connections 2023-09-10T00:40:08.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:08.769+0000 7f9dd653a700 1 -- 172.21.15.44:0/3359293194 wait complete. 2023-09-10T00:40:08.780 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2023-09-10T00:40:09.167 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph osd dump' 2023-09-10T00:40:09.732 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:40:09.967 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:09 smithi044 ceph-mon[98676]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:40:09.968 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:09 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3359293194' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-10T00:40:10.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:09 smithi131 ceph-mon[103713]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:40:10.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:09 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3359293194' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-10T00:40:10.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:09 smithi099 ceph-mon[102692]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:40:10.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:09 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3359293194' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-10T00:40:11.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.056+0000 7f8b96cc4700 1 -- 172.21.15.44:0/2961274162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90100790 msgr2=0x7f8b90100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:11.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.056+0000 7f8b96cc4700 1 --2- 172.21.15.44:0/2961274162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90100790 0x7f8b90100bf0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0099c0 tx=0x7f8b7c009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.056+0000 7f8b96cc4700 1 -- 172.21.15.44:0/2961274162 shutdown_connections 2023-09-10T00:40:11.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.056+0000 7f8b96cc4700 1 --2- 172.21.15.44:0/2961274162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b90101130 0x7f8b901079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.056+0000 7f8b96cc4700 1 --2- 172.21.15.44:0/2961274162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b900ff590 0x7f8b900ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.056+0000 7f8b96cc4700 1 --2- 172.21.15.44:0/2961274162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90100790 0x7f8b90100bf0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.056+0000 7f8b96cc4700 1 -- 172.21.15.44:0/2961274162 >> 172.21.15.44:0/2961274162 conn(0x7f8b900fab10 msgr2=0x7f8b900fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:11.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.056+0000 7f8b96cc4700 1 -- 172.21.15.44:0/2961274162 shutdown_connections 2023-09-10T00:40:11.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.056+0000 7f8b96cc4700 1 -- 172.21.15.44:0/2961274162 wait complete. 2023-09-10T00:40:11.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b96cc4700 1 Processor -- start 2023-09-10T00:40:11.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b96cc4700 1 -- start start 2023-09-10T00:40:11.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b96cc4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b900ff590 0x7f8b90196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:11.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b96cc4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b90100790 0x7f8b90196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:11.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b96cc4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90101130 0x7f8b9019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:11.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b96cc4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9019ddc0 con 0x7f8b90101130 2023-09-10T00:40:11.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b96cc4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9019df00 con 0x7f8b90100790 2023-09-10T00:40:11.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b96cc4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9019e090 con 0x7f8b900ff590 2023-09-10T00:40:11.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b95261700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90101130 0x7f8b9019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:11.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b95261700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90101130 0x7f8b9019d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47812/0 (socket says 172.21.15.44:47812) 2023-09-10T00:40:11.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b8ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b90100790 0x7f8b90196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:11.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b95261700 1 -- 172.21.15.44:0/1615348129 learned_addr learned my addr 172.21.15.44:0/1615348129 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:11.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b94a60700 1 --2- 172.21.15.44:0/1615348129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b900ff590 0x7f8b90196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:11.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b95261700 1 -- 172.21.15.44:0/1615348129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b900ff590 msgr2=0x7f8b90196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:11.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b95261700 1 --2- 172.21.15.44:0/1615348129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b900ff590 0x7f8b90196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b95261700 1 -- 172.21.15.44:0/1615348129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b90100790 msgr2=0x7f8b90196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:11.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b95261700 1 --2- 172.21.15.44:0/1615348129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b90100790 0x7f8b90196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.057+0000 7f8b95261700 1 -- 172.21.15.44:0/1615348129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c0095f0 con 0x7f8b90101130 2023-09-10T00:40:11.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.058+0000 7f8b95261700 1 --2- 172.21.15.44:0/1615348129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90101130 0x7f8b9019d7f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8b88007f10 tx=0x7f8b8800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:11.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.058+0000 7f8b8dffb700 1 -- 172.21.15.44:0/1615348129 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88017070 con 0x7f8b90101130 2023-09-10T00:40:11.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.058+0000 7f8b8dffb700 1 -- 172.21.15.44:0/1615348129 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b8800f040 con 0x7f8b90101130 2023-09-10T00:40:11.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.058+0000 7f8b96cc4700 1 -- 172.21.15.44:0/1615348129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9019e370 con 0x7f8b90101130 2023-09-10T00:40:11.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.058+0000 7f8b8dffb700 1 -- 172.21.15.44:0/1615348129 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88005570 con 0x7f8b90101130 2023-09-10T00:40:11.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.058+0000 7f8b96cc4700 1 -- 172.21.15.44:0/1615348129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9019e870 con 0x7f8b90101130 2023-09-10T00:40:11.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.059+0000 7f8b96cc4700 1 -- 172.21.15.44:0/1615348129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b9010ac50 con 0x7f8b90101130 2023-09-10T00:40:11.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.060+0000 7f8b8dffb700 1 -- 172.21.15.44:0/1615348129 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f8b8800db20 con 0x7f8b90101130 2023-09-10T00:40:11.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.060+0000 7f8b8dffb700 1 --2- 172.21.15.44:0/1615348129 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8b800a0750 0x7f8b800a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:11.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.061+0000 7f8b94a60700 1 --2- 172.21.15.44:0/1615348129 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8b800a0750 0x7f8b800a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:11.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.061+0000 7f8b8dffb700 1 -- 172.21.15.44:0/1615348129 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8b880cb730 con 0x7f8b90101130 2023-09-10T00:40:11.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.061+0000 7f8b94a60700 1 --2- 172.21.15.44:0/1615348129 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8b800a0750 0x7f8b800a2c10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8b901005f0 tx=0x7f8b8400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:11.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.065+0000 7f8b8dffb700 1 -- 172.21.15.44:0/1615348129 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8b88094df0 con 0x7f8b90101130 2023-09-10T00:40:11.264 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.264+0000 7f8b96cc4700 1 -- 172.21.15.44:0/1615348129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f8b9010ae70 con 0x7f8b90101130 2023-09-10T00:40:11.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.265+0000 7f8b8dffb700 1 -- 172.21.15.44:0/1615348129 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) v1 ==== 56+0+5566 (secure 0 0 0) 0x7f8b88098410 con 0x7f8b90101130 2023-09-10T00:40:11.265 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 89 2023-09-10T00:40:11.265 INFO:teuthology.orchestra.run.smithi044.stdout:fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T00:40:11.266 INFO:teuthology.orchestra.run.smithi044.stdout:created 2023-09-10T00:31:16.073750+0000 2023-09-10T00:40:11.266 INFO:teuthology.orchestra.run.smithi044.stdout:modified 2023-09-10T00:39:17.515738+0000 2023-09-10T00:40:11.266 INFO:teuthology.orchestra.run.smithi044.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-09-10T00:40:11.266 INFO:teuthology.orchestra.run.smithi044.stdout:crush_version 27 2023-09-10T00:40:11.266 INFO:teuthology.orchestra.run.smithi044.stdout:full_ratio 0.95 2023-09-10T00:40:11.267 INFO:teuthology.orchestra.run.smithi044.stdout:backfillfull_ratio 0.9 2023-09-10T00:40:11.267 INFO:teuthology.orchestra.run.smithi044.stdout:nearfull_ratio 0.85 2023-09-10T00:40:11.267 INFO:teuthology.orchestra.run.smithi044.stdout:require_min_compat_client luminous 2023-09-10T00:40:11.267 INFO:teuthology.orchestra.run.smithi044.stdout:min_compat_client luminous 2023-09-10T00:40:11.267 INFO:teuthology.orchestra.run.smithi044.stdout:require_osd_release reef 2023-09-10T00:40:11.268 INFO:teuthology.orchestra.run.smithi044.stdout:stretch_mode_enabled false 2023-09-10T00:40:11.268 INFO:teuthology.orchestra.run.smithi044.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-09-10T00:40:11.268 INFO:teuthology.orchestra.run.smithi044.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-09-10T00:40:11.268 INFO:teuthology.orchestra.run.smithi044.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-09-10T00:40:11.268 INFO:teuthology.orchestra.run.smithi044.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-09-10T00:40:11.269 INFO:teuthology.orchestra.run.smithi044.stdout:max_osd 12 2023-09-10T00:40:11.269 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6802/424109980,v1:172.21.15.44:6803/424109980] [v2:172.21.15.44:6804/424109980,v1:172.21.15.44:6805/424109980] exists,up 4a7d2e1e-5817-489e-af0e-72e9bcccc119 2023-09-10T00:40:11.269 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6800/63099758,v1:172.21.15.99:6801/63099758] [v2:172.21.15.99:6802/63099758,v1:172.21.15.99:6803/63099758] exists,up cf261516-01eb-4f04-b7be-19f32e9e55e8 2023-09-10T00:40:11.269 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6800/3245285432,v1:172.21.15.131:6801/3245285432] [v2:172.21.15.131:6802/3245285432,v1:172.21.15.131:6803/3245285432] exists,up 1d73ba44-ef7a-495b-97cb-550517aaab45 2023-09-10T00:40:11.269 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6810/2588997584,v1:172.21.15.44:6811/2588997584] [v2:172.21.15.44:6812/2588997584,v1:172.21.15.44:6813/2588997584] exists,up e6fe9647-238c-4f5c-a933-ace40dc5bc0f 2023-09-10T00:40:11.270 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 up in weight 1 up_from 33 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6808/532684490,v1:172.21.15.99:6809/532684490] [v2:172.21.15.99:6810/532684490,v1:172.21.15.99:6811/532684490] exists,up 1ae06a56-c8b7-4881-8ead-68835a9015a1 2023-09-10T00:40:11.270 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 up in weight 1 up_from 38 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6808/979810489,v1:172.21.15.131:6809/979810489] [v2:172.21.15.131:6810/979810489,v1:172.21.15.131:6811/979810489] exists,up 917c392a-bb6c-408b-80c6-9d81c194b0b9 2023-09-10T00:40:11.270 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 up in weight 1 up_from 45 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6818/1858317767,v1:172.21.15.44:6819/1858317767] [v2:172.21.15.44:6820/1858317767,v1:172.21.15.44:6821/1858317767] exists,up 4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b 2023-09-10T00:40:11.270 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 up in weight 1 up_from 50 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6816/107594338,v1:172.21.15.99:6817/107594338] [v2:172.21.15.99:6818/107594338,v1:172.21.15.99:6819/107594338] exists,up 4cfd6428-45b9-4884-973d-6f980996fe99 2023-09-10T00:40:11.270 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8 up in weight 1 up_from 57 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6816/1528398267,v1:172.21.15.131:6817/1528398267] [v2:172.21.15.131:6818/1528398267,v1:172.21.15.131:6819/1528398267] exists,up 81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b 2023-09-10T00:40:11.271 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9 up in weight 1 up_from 65 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6826/1885092640,v1:172.21.15.44:6827/1885092640] [v2:172.21.15.44:6828/1885092640,v1:172.21.15.44:6829/1885092640] exists,up fa68741f-6ffa-4525-bfce-4b29c806bc53 2023-09-10T00:40:11.271 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10 up in weight 1 up_from 70 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6824/1454470032,v1:172.21.15.99:6825/1454470032] [v2:172.21.15.99:6826/1454470032,v1:172.21.15.99:6827/1454470032] exists,up aee4c5aa-0fb1-447d-948f-61cd27d5eecb 2023-09-10T00:40:11.271 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11 up in weight 1 up_from 78 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6824/1396964632,v1:172.21.15.131:6825/1396964632] [v2:172.21.15.131:6826/1396964632,v1:172.21.15.131:6827/1396964632] exists,up f8d68bdc-aa91-4e82-9965-f74b9efa488a 2023-09-10T00:40:11.271 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.0 [4,6] 2023-09-10T00:40:11.271 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.c [3,9] 2023-09-10T00:40:11.272 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.e [4,8] 2023-09-10T00:40:11.272 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.14 [4,6] 2023-09-10T00:40:11.272 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.28 [5,8] 2023-09-10T00:40:11.272 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.31 [4,7] 2023-09-10T00:40:11.272 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.33 [5,9] 2023-09-10T00:40:11.273 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.3b [3,9] 2023-09-10T00:40:11.273 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 4.e [9,11] 2023-09-10T00:40:11.273 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 4.12 [9,10] 2023-09-10T00:40:11.273 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6801/18127588 expires 2023-09-11T00:32:15.913770+0000 2023-09-10T00:40:11.273 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6800/18127588 expires 2023-09-11T00:32:15.913770+0000 2023-09-10T00:40:11.274 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/1414498756 expires 2023-09-11T00:32:15.913770+0000 2023-09-10T00:40:11.274 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/1298831914 expires 2023-09-11T00:32:15.913770+0000 2023-09-10T00:40:11.274 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/2415771009 expires 2023-09-11T00:31:49.496807+0000 2023-09-10T00:40:11.274 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/193660691 expires 2023-09-11T00:31:49.496807+0000 2023-09-10T00:40:11.274 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6800/871815312 expires 2023-09-11T00:31:49.496807+0000 2023-09-10T00:40:11.275 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/1481976351 expires 2023-09-11T00:32:15.913770+0000 2023-09-10T00:40:11.275 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6801/871815312 expires 2023-09-11T00:31:49.496807+0000 2023-09-10T00:40:11.275 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/625830807 expires 2023-09-11T00:31:49.496807+0000 2023-09-10T00:40:11.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.267+0000 7f8b96cc4700 1 -- 172.21.15.44:0/1615348129 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8b800a0750 msgr2=0x7f8b800a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:11.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.267+0000 7f8b96cc4700 1 --2- 172.21.15.44:0/1615348129 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8b800a0750 0x7f8b800a2c10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8b901005f0 tx=0x7f8b8400b410 comp rx=0 tx=0).stop 2023-09-10T00:40:11.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.267+0000 7f8b96cc4700 1 -- 172.21.15.44:0/1615348129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90101130 msgr2=0x7f8b9019d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:11.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.267+0000 7f8b96cc4700 1 --2- 172.21.15.44:0/1615348129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90101130 0x7f8b9019d7f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8b88007f10 tx=0x7f8b8800d3b0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.267+0000 7f8b96cc4700 1 -- 172.21.15.44:0/1615348129 shutdown_connections 2023-09-10T00:40:11.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.267+0000 7f8b96cc4700 1 --2- 172.21.15.44:0/1615348129 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8b800a0750 0x7f8b800a2c10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.267+0000 7f8b96cc4700 1 --2- 172.21.15.44:0/1615348129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90101130 0x7f8b9019d7f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.267+0000 7f8b96cc4700 1 --2- 172.21.15.44:0/1615348129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b900ff590 0x7f8b90196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.267+0000 7f8b96cc4700 1 --2- 172.21.15.44:0/1615348129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b90100790 0x7f8b90196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.267+0000 7f8b96cc4700 1 -- 172.21.15.44:0/1615348129 >> 172.21.15.44:0/1615348129 conn(0x7f8b900fab10 msgr2=0x7f8b90068100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:11.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.267+0000 7f8b96cc4700 1 -- 172.21.15.44:0/1615348129 shutdown_connections 2023-09-10T00:40:11.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.267+0000 7f8b96cc4700 1 -- 172.21.15.44:0/1615348129 wait complete. 2023-09-10T00:40:11.690 INFO:teuthology.run_tasks:Running task fs.ready... 2023-09-10T00:40:11.704 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:40:11.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.837+0000 7f9ab3b30700 1 -- 172.21.15.44:0/3881840924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9aac102190 msgr2=0x7f9aac108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:11.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.837+0000 7f9ab3b30700 1 --2- 172.21.15.44:0/3881840924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9aac102190 0x7f9aac108a40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400b0d0 tx=0x7f9aa400b3e0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.837+0000 7f9ab3b30700 1 -- 172.21.15.44:0/3881840924 shutdown_connections 2023-09-10T00:40:11.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.837+0000 7f9ab3b30700 1 --2- 172.21.15.44:0/3881840924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9aac102190 0x7f9aac108a40 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.837+0000 7f9ab3b30700 1 --2- 172.21.15.44:0/3881840924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9aac100600 0x7f9aac100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.837+0000 7f9ab3b30700 1 --2- 172.21.15.44:0/3881840924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9aac1017f0 0x7f9aac101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.837+0000 7f9ab3b30700 1 -- 172.21.15.44:0/3881840924 >> 172.21.15.44:0/3881840924 conn(0x7f9aac0fbb50 msgr2=0x7f9aac0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:11.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.837+0000 7f9ab3b30700 1 -- 172.21.15.44:0/3881840924 shutdown_connections 2023-09-10T00:40:11.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.837+0000 7f9ab3b30700 1 -- 172.21.15.44:0/3881840924 wait complete. 2023-09-10T00:40:11.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.838+0000 7f9ab3b30700 1 Processor -- start 2023-09-10T00:40:11.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.838+0000 7f9ab3b30700 1 -- start start 2023-09-10T00:40:11.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.838+0000 7f9ab3b30700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9aac100600 0x7f9aac1962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:11.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.838+0000 7f9ab3b30700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9aac1017f0 0x7f9aac196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:11.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.838+0000 7f9ab3b30700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9aac102190 0x7f9aac19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:11.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.838+0000 7f9ab3b30700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac19de40 con 0x7f9aac1017f0 2023-09-10T00:40:11.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.838+0000 7f9ab3b30700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac19df80 con 0x7f9aac100600 2023-09-10T00:40:11.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.838+0000 7f9ab3b30700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac19e110 con 0x7f9aac102190 2023-09-10T00:40:11.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.838+0000 7f9ab10cb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9aac1017f0 0x7f9aac196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:11.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.838+0000 7f9ab10cb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9aac1017f0 0x7f9aac196800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47822/0 (socket says 172.21.15.44:47822) 2023-09-10T00:40:11.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.838+0000 7f9ab10cb700 1 -- 172.21.15.44:0/94581442 learned_addr learned my addr 172.21.15.44:0/94581442 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:11.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.839+0000 7f9ab20cd700 1 --2- 172.21.15.44:0/94581442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9aac102190 0x7f9aac19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:11.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.839+0000 7f9ab18cc700 1 --2- 172.21.15.44:0/94581442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9aac100600 0x7f9aac1962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:11.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.839+0000 7f9ab10cb700 1 -- 172.21.15.44:0/94581442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9aac102190 msgr2=0x7f9aac19d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:11.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.839+0000 7f9ab10cb700 1 --2- 172.21.15.44:0/94581442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9aac102190 0x7f9aac19d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.839+0000 7f9ab10cb700 1 -- 172.21.15.44:0/94581442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9aac100600 msgr2=0x7f9aac1962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:11.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.839+0000 7f9ab10cb700 1 --2- 172.21.15.44:0/94581442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9aac100600 0x7f9aac1962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:11.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.839+0000 7f9ab10cb700 1 -- 172.21.15.44:0/94581442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa4009d00 con 0x7f9aac1017f0 2023-09-10T00:40:11.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.839+0000 7f9ab18cc700 1 --2- 172.21.15.44:0/94581442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9aac100600 0x7f9aac1962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:11.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.839+0000 7f9ab20cd700 1 --2- 172.21.15.44:0/94581442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9aac102190 0x7f9aac19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:11.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.839+0000 7f9ab10cb700 1 --2- 172.21.15.44:0/94581442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9aac1017f0 0x7f9aac196800 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800e9a0 tx=0x7f9aa800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:11.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.839+0000 7f9aa2ffd700 1 -- 172.21.15.44:0/94581442 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8009790 con 0x7f9aac1017f0 2023-09-10T00:40:11.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.839+0000 7f9aa2ffd700 1 -- 172.21.15.44:0/94581442 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9aa800cbd0 con 0x7f9aac1017f0 2023-09-10T00:40:11.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.839+0000 7f9aa2ffd700 1 -- 172.21.15.44:0/94581442 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8018680 con 0x7f9aac1017f0 2023-09-10T00:40:11.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.839+0000 7f9ab3b30700 1 -- 172.21.15.44:0/94581442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aac19e3f0 con 0x7f9aac1017f0 2023-09-10T00:40:11.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.839+0000 7f9ab3b30700 1 -- 172.21.15.44:0/94581442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aac19e8b0 con 0x7f9aac1017f0 2023-09-10T00:40:11.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.840+0000 7f9ab3b30700 1 -- 172.21.15.44:0/94581442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aac107bf0 con 0x7f9aac1017f0 2023-09-10T00:40:11.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.842+0000 7f9aa2ffd700 1 -- 172.21.15.44:0/94581442 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f9aa800cd40 con 0x7f9aac1017f0 2023-09-10T00:40:11.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.842+0000 7f9aa2ffd700 1 --2- 172.21.15.44:0/94581442 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a980a0710 0x7f9a980a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:11.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.842+0000 7f9ab18cc700 1 --2- 172.21.15.44:0/94581442 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a980a0710 0x7f9a980a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:11.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.842+0000 7f9aa2ffd700 1 -- 172.21.15.44:0/94581442 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9aa8014070 con 0x7f9aac1017f0 2023-09-10T00:40:11.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.843+0000 7f9ab18cc700 1 --2- 172.21.15.44:0/94581442 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a980a0710 0x7f9a980a2bd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9aac101650 tx=0x7f9a9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:11.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:11.846+0000 7f9aa2ffd700 1 -- 172.21.15.44:0/94581442 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9aa8095c20 con 0x7f9aac1017f0 2023-09-10T00:40:12.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:11 smithi131 ceph-mon[103713]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-10T00:40:12.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:11 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1615348129' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-10T00:40:12.051 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.051+0000 7f9ab3b30700 1 -- 172.21.15.44:0/94581442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9aac10bc80 con 0x7f9aac1017f0 2023-09-10T00:40:12.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.052+0000 7f9aa2ffd700 1 -- 172.21.15.44:0/94581442 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24340 (secure 0 0 0) 0x7f9aa8099240 con 0x7f9aac1017f0 2023-09-10T00:40:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:12.053 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":89,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:39:17.515738+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:40:12.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:11 smithi044 ceph-mon[98676]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-10T00:40:12.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:11 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1615348129' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-10T00:40:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.055+0000 7f9ab3b30700 1 -- 172.21.15.44:0/94581442 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a980a0710 msgr2=0x7f9a980a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.055+0000 7f9ab3b30700 1 --2- 172.21.15.44:0/94581442 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a980a0710 0x7f9a980a2bd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9aac101650 tx=0x7f9a9c009260 comp rx=0 tx=0).stop 2023-09-10T00:40:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.055+0000 7f9ab3b30700 1 -- 172.21.15.44:0/94581442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9aac1017f0 msgr2=0x7f9aac196800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.055+0000 7f9ab3b30700 1 --2- 172.21.15.44:0/94581442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9aac1017f0 0x7f9aac196800 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800e9a0 tx=0x7f9aa800ed60 comp rx=0 tx=0).stop 2023-09-10T00:40:12.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.055+0000 7f9ab3b30700 1 -- 172.21.15.44:0/94581442 shutdown_connections 2023-09-10T00:40:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.055+0000 7f9ab3b30700 1 --2- 172.21.15.44:0/94581442 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f9a980a0710 0x7f9a980a2bd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.055+0000 7f9ab3b30700 1 --2- 172.21.15.44:0/94581442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9aac102190 0x7f9aac19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.055+0000 7f9ab3b30700 1 --2- 172.21.15.44:0/94581442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9aac100600 0x7f9aac1962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.055+0000 7f9ab3b30700 1 --2- 172.21.15.44:0/94581442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9aac1017f0 0x7f9aac196800 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.055+0000 7f9ab3b30700 1 -- 172.21.15.44:0/94581442 >> 172.21.15.44:0/94581442 conn(0x7f9aac0fbb50 msgr2=0x7f9aac104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.055+0000 7f9ab3b30700 1 -- 172.21.15.44:0/94581442 shutdown_connections 2023-09-10T00:40:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.055+0000 7f9ab3b30700 1 -- 172.21.15.44:0/94581442 wait complete. 2023-09-10T00:40:12.068 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-10T00:34:44.456741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-10T00:38:37.194891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-10T00:38:38.573268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-10T00:38:42.198896+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-10T00:40:12.069 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-10T00:40:12.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:11 smithi099 ceph-mon[102692]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-10T00:40:12.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:11 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1615348129' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-10T00:40:12.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.199+0000 7f412040a700 1 -- 172.21.15.44:0/1688084632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4118101800 msgr2=0x7f4118101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.199+0000 7f412040a700 1 --2- 172.21.15.44:0/1688084632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4118101800 0x7f4118101c60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f41080099c0 tx=0x7f4108009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.199+0000 7f412040a700 1 -- 172.21.15.44:0/1688084632 shutdown_connections 2023-09-10T00:40:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.199+0000 7f412040a700 1 --2- 172.21.15.44:0/1688084632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41181021a0 0x7f4118108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.199+0000 7f412040a700 1 --2- 172.21.15.44:0/1688084632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4118100610 0x7f4118100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.199+0000 7f412040a700 1 --2- 172.21.15.44:0/1688084632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4118101800 0x7f4118101c60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.199+0000 7f412040a700 1 -- 172.21.15.44:0/1688084632 >> 172.21.15.44:0/1688084632 conn(0x7f41180fbb80 msgr2=0x7f41180fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:12.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.199+0000 7f412040a700 1 -- 172.21.15.44:0/1688084632 shutdown_connections 2023-09-10T00:40:12.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.199+0000 7f412040a700 1 -- 172.21.15.44:0/1688084632 wait complete. 2023-09-10T00:40:12.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.199+0000 7f412040a700 1 Processor -- start 2023-09-10T00:40:12.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.199+0000 7f412040a700 1 -- start start 2023-09-10T00:40:12.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f412040a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4118100610 0x7f41180743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:12.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f412040a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4118101800 0x7f41180729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:12.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f412040a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41181021a0 0x7f4118072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:12.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f412040a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41180736d0 con 0x7f41181021a0 2023-09-10T00:40:12.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f412040a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4118073840 con 0x7f4118100610 2023-09-10T00:40:12.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f412040a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41181a0160 con 0x7f4118101800 2023-09-10T00:40:12.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f411e9a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41181021a0 0x7f4118072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:12.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f411e9a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41181021a0 0x7f4118072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47838/0 (socket says 172.21.15.44:47838) 2023-09-10T00:40:12.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f411e9a7700 1 -- 172.21.15.44:0/1596940695 learned_addr learned my addr 172.21.15.44:0/1596940695 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:12.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f411d9a5700 1 --2- 172.21.15.44:0/1596940695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4118101800 0x7f41180729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:12.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f411e1a6700 1 --2- 172.21.15.44:0/1596940695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4118100610 0x7f41180743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:12.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f411e9a7700 1 -- 172.21.15.44:0/1596940695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4118101800 msgr2=0x7f41180729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:12.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f411e9a7700 1 --2- 172.21.15.44:0/1596940695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4118101800 0x7f41180729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f411e9a7700 1 -- 172.21.15.44:0/1596940695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4118100610 msgr2=0x7f41180743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:12.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f411e9a7700 1 --2- 172.21.15.44:0/1596940695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4118100610 0x7f41180743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f411e9a7700 1 -- 172.21.15.44:0/1596940695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41080095f0 con 0x7f41181021a0 2023-09-10T00:40:12.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f411e1a6700 1 --2- 172.21.15.44:0/1596940695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4118100610 0x7f41180743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:12.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f411e9a7700 1 --2- 172.21.15.44:0/1596940695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41181021a0 0x7f4118072f30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4110007d70 tx=0x7f411000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:12.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.200+0000 7f410f7fe700 1 -- 172.21.15.44:0/1596940695 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411000ad40 con 0x7f41181021a0 2023-09-10T00:40:12.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.201+0000 7f410f7fe700 1 -- 172.21.15.44:0/1596940695 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41100128b0 con 0x7f41181021a0 2023-09-10T00:40:12.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.201+0000 7f410f7fe700 1 -- 172.21.15.44:0/1596940695 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41100115a0 con 0x7f41181021a0 2023-09-10T00:40:12.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.201+0000 7f412040a700 1 -- 172.21.15.44:0/1596940695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41181a0440 con 0x7f41181021a0 2023-09-10T00:40:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.201+0000 7f412040a700 1 -- 172.21.15.44:0/1596940695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41181a0900 con 0x7f41181021a0 2023-09-10T00:40:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.202+0000 7f412040a700 1 -- 172.21.15.44:0/1596940695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4118107b60 con 0x7f41181021a0 2023-09-10T00:40:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.203+0000 7f410f7fe700 1 -- 172.21.15.44:0/1596940695 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f4110012a20 con 0x7f41181021a0 2023-09-10T00:40:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.204+0000 7f410f7fe700 1 --2- 172.21.15.44:0/1596940695 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f41040a0750 0x7f41040a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.204+0000 7f411e1a6700 1 --2- 172.21.15.44:0/1596940695 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f41040a0750 0x7f41040a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.204+0000 7f410f7fe700 1 -- 172.21.15.44:0/1596940695 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f41100cc510 con 0x7f41181021a0 2023-09-10T00:40:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.204+0000 7f411e1a6700 1 --2- 172.21.15.44:0/1596940695 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f41040a0750 0x7f41040a2c10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4118101660 tx=0x7f411400a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.207+0000 7f410f7fe700 1 -- 172.21.15.44:0/1596940695 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4110095ce0 con 0x7f41181021a0 2023-09-10T00:40:12.408 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.408+0000 7f412040a700 1 -- 172.21.15.44:0/1596940695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f411810be70 con 0x7f41181021a0 2023-09-10T00:40:12.409 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.408+0000 7f410f7fe700 1 -- 172.21.15.44:0/1596940695 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41100160a0 con 0x7f41181021a0 2023-09-10T00:40:12.409 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2023-09-10T00:40:12.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.411+0000 7f412040a700 1 -- 172.21.15.44:0/1596940695 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f41040a0750 msgr2=0x7f41040a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:12.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.411+0000 7f412040a700 1 --2- 172.21.15.44:0/1596940695 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f41040a0750 0x7f41040a2c10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4118101660 tx=0x7f411400a680 comp rx=0 tx=0).stop 2023-09-10T00:40:12.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.411+0000 7f412040a700 1 -- 172.21.15.44:0/1596940695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41181021a0 msgr2=0x7f4118072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:12.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.411+0000 7f412040a700 1 --2- 172.21.15.44:0/1596940695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41181021a0 0x7f4118072f30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4110007d70 tx=0x7f411000a340 comp rx=0 tx=0).stop 2023-09-10T00:40:12.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.411+0000 7f412040a700 1 -- 172.21.15.44:0/1596940695 shutdown_connections 2023-09-10T00:40:12.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.411+0000 7f412040a700 1 --2- 172.21.15.44:0/1596940695 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f41040a0750 0x7f41040a2c10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.411+0000 7f412040a700 1 --2- 172.21.15.44:0/1596940695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41181021a0 0x7f4118072f30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.411+0000 7f412040a700 1 --2- 172.21.15.44:0/1596940695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4118100610 0x7f41180743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.411+0000 7f412040a700 1 --2- 172.21.15.44:0/1596940695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4118101800 0x7f41180729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.411+0000 7f412040a700 1 -- 172.21.15.44:0/1596940695 >> 172.21.15.44:0/1596940695 conn(0x7f41180fbb80 msgr2=0x7f4118104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:12.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.411+0000 7f412040a700 1 -- 172.21.15.44:0/1596940695 shutdown_connections 2023-09-10T00:40:12.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.411+0000 7f412040a700 1 -- 172.21.15.44:0/1596940695 wait complete. 2023-09-10T00:40:12.423 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-10T00:40:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.553+0000 7ffae1a0c700 1 -- 172.21.15.44:0/1408663921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc0ff520 msgr2=0x7ffadc0ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:12.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.553+0000 7ffae1a0c700 1 --2- 172.21.15.44:0/1408663921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc0ff520 0x7ffadc0ff940 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0099c0 tx=0x7ffacc009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.553+0000 7ffae1a0c700 1 -- 172.21.15.44:0/1408663921 shutdown_connections 2023-09-10T00:40:12.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.553+0000 7ffae1a0c700 1 --2- 172.21.15.44:0/1408663921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffadc1010b0 0x7ffadc107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.553+0000 7ffae1a0c700 1 --2- 172.21.15.44:0/1408663921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc0ff520 0x7ffadc0ff940 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.553+0000 7ffae1a0c700 1 --2- 172.21.15.44:0/1408663921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffadc100710 0x7ffadc100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.553+0000 7ffae1a0c700 1 -- 172.21.15.44:0/1408663921 >> 172.21.15.44:0/1408663921 conn(0x7ffadc0faa90 msgr2=0x7ffadc0fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.553+0000 7ffae1a0c700 1 -- 172.21.15.44:0/1408663921 shutdown_connections 2023-09-10T00:40:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.553+0000 7ffae1a0c700 1 -- 172.21.15.44:0/1408663921 wait complete. 2023-09-10T00:40:12.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.554+0000 7ffae1a0c700 1 Processor -- start 2023-09-10T00:40:12.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.554+0000 7ffae1a0c700 1 -- start start 2023-09-10T00:40:12.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.554+0000 7ffae1a0c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffadc0ff520 0x7ffadc195150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:12.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.554+0000 7ffae1a0c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffadc100710 0x7ffadc195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:12.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.554+0000 7ffae1a0c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc1010b0 0x7ffadc19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:12.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.554+0000 7ffae1a0c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc19ccd0 con 0x7ffadc1010b0 2023-09-10T00:40:12.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.554+0000 7ffae1a0c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc19ce10 con 0x7ffadc0ff520 2023-09-10T00:40:12.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.554+0000 7ffae1a0c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc19cfa0 con 0x7ffadc100710 2023-09-10T00:40:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.554+0000 7ffadb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc1010b0 0x7ffadc19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.554+0000 7ffadb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc1010b0 0x7ffadc19c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47852/0 (socket says 172.21.15.44:47852) 2023-09-10T00:40:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.554+0000 7ffadb7fe700 1 -- 172.21.15.44:0/2820181446 learned_addr learned my addr 172.21.15.44:0/2820181446 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.554+0000 7ffada7fc700 1 --2- 172.21.15.44:0/2820181446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffadc100710 0x7ffadc195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.555+0000 7ffadaffd700 1 --2- 172.21.15.44:0/2820181446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffadc0ff520 0x7ffadc195150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.555+0000 7ffadb7fe700 1 -- 172.21.15.44:0/2820181446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffadc100710 msgr2=0x7ffadc195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.555+0000 7ffadb7fe700 1 --2- 172.21.15.44:0/2820181446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffadc100710 0x7ffadc195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.555+0000 7ffadb7fe700 1 -- 172.21.15.44:0/2820181446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffadc0ff520 msgr2=0x7ffadc195150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.555+0000 7ffadb7fe700 1 --2- 172.21.15.44:0/2820181446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffadc0ff520 0x7ffadc195150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.555+0000 7ffadb7fe700 1 -- 172.21.15.44:0/2820181446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc0095f0 con 0x7ffadc1010b0 2023-09-10T00:40:12.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.555+0000 7ffadb7fe700 1 --2- 172.21.15.44:0/2820181446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc1010b0 0x7ffadc19c700 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffad4007d70 tx=0x7ffad400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:12.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.555+0000 7ffae0a0a700 1 -- 172.21.15.44:0/2820181446 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad400ad40 con 0x7ffadc1010b0 2023-09-10T00:40:12.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.555+0000 7ffae0a0a700 1 -- 172.21.15.44:0/2820181446 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffad40128b0 con 0x7ffadc1010b0 2023-09-10T00:40:12.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.555+0000 7ffae1a0c700 1 -- 172.21.15.44:0/2820181446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffadc19d280 con 0x7ffadc1010b0 2023-09-10T00:40:12.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.555+0000 7ffae0a0a700 1 -- 172.21.15.44:0/2820181446 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad40115a0 con 0x7ffadc1010b0 2023-09-10T00:40:12.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.555+0000 7ffae1a0c700 1 -- 172.21.15.44:0/2820181446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffadc19d740 con 0x7ffadc1010b0 2023-09-10T00:40:12.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.556+0000 7ffae1a0c700 1 -- 172.21.15.44:0/2820181446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffadc106500 con 0x7ffadc1010b0 2023-09-10T00:40:12.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.557+0000 7ffae0a0a700 1 -- 172.21.15.44:0/2820181446 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7ffad4012a20 con 0x7ffadc1010b0 2023-09-10T00:40:12.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.558+0000 7ffae0a0a700 1 --2- 172.21.15.44:0/2820181446 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffac40a06d0 0x7ffac40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:12.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.558+0000 7ffadaffd700 1 --2- 172.21.15.44:0/2820181446 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffac40a06d0 0x7ffac40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:12.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.558+0000 7ffae0a0a700 1 -- 172.21.15.44:0/2820181446 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ffad40cc220 con 0x7ffadc1010b0 2023-09-10T00:40:12.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.558+0000 7ffadaffd700 1 --2- 172.21.15.44:0/2820181446 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffac40a06d0 0x7ffac40a2b90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffacc000c00 tx=0x7ffacc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:12.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.562+0000 7ffae0a0a700 1 -- 172.21.15.44:0/2820181446 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ffad4095990 con 0x7ffadc1010b0 2023-09-10T00:40:12.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.762+0000 7ffae1a0c700 1 -- 172.21.15.44:0/2820181446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffadc10ae50 con 0x7ffadc1010b0 2023-09-10T00:40:12.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.762+0000 7ffae0a0a700 1 -- 172.21.15.44:0/2820181446 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffad4098fb0 con 0x7ffadc1010b0 2023-09-10T00:40:12.763 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:40:12.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.764+0000 7ffae1a0c700 1 -- 172.21.15.44:0/2820181446 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffac40a06d0 msgr2=0x7ffac40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:12.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.764+0000 7ffae1a0c700 1 --2- 172.21.15.44:0/2820181446 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffac40a06d0 0x7ffac40a2b90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffacc000c00 tx=0x7ffacc005fb0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.764+0000 7ffae1a0c700 1 -- 172.21.15.44:0/2820181446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc1010b0 msgr2=0x7ffadc19c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:12.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.764+0000 7ffae1a0c700 1 --2- 172.21.15.44:0/2820181446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc1010b0 0x7ffadc19c700 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffad4007d70 tx=0x7ffad400a340 comp rx=0 tx=0).stop 2023-09-10T00:40:12.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.764+0000 7ffae1a0c700 1 -- 172.21.15.44:0/2820181446 shutdown_connections 2023-09-10T00:40:12.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.764+0000 7ffae1a0c700 1 --2- 172.21.15.44:0/2820181446 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffac40a06d0 0x7ffac40a2b90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.764+0000 7ffae1a0c700 1 --2- 172.21.15.44:0/2820181446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc1010b0 0x7ffadc19c700 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.764+0000 7ffae1a0c700 1 --2- 172.21.15.44:0/2820181446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffadc0ff520 0x7ffadc195150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.764+0000 7ffae1a0c700 1 --2- 172.21.15.44:0/2820181446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffadc100710 0x7ffadc195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.764+0000 7ffae1a0c700 1 -- 172.21.15.44:0/2820181446 >> 172.21.15.44:0/2820181446 conn(0x7ffadc0faa90 msgr2=0x7ffadc0fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:12.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.765+0000 7ffae1a0c700 1 -- 172.21.15.44:0/2820181446 shutdown_connections 2023-09-10T00:40:12.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.765+0000 7ffae1a0c700 1 -- 172.21.15.44:0/2820181446 wait complete. 2023-09-10T00:40:12.776 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:12 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/94581442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:12.776 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:12 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1596940695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:40:12.777 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-10T00:40:12.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.906+0000 7f16c9df2700 1 -- 172.21.15.44:0/186728589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16c41010d0 msgr2=0x7f16c41034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:12.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.906+0000 7f16c9df2700 1 --2- 172.21.15.44:0/186728589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16c41010d0 0x7f16c41034c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009990 tx=0x7f16b4009ca0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.907+0000 7f16c9df2700 1 -- 172.21.15.44:0/186728589 shutdown_connections 2023-09-10T00:40:12.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.907+0000 7f16c9df2700 1 --2- 172.21.15.44:0/186728589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16c4106330 0x7f16c4108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.907+0000 7f16c9df2700 1 --2- 172.21.15.44:0/186728589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16c41010d0 0x7f16c41034c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.907+0000 7f16c9df2700 1 --2- 172.21.15.44:0/186728589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16c4103a00 0x7f16c4105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.907+0000 7f16c9df2700 1 -- 172.21.15.44:0/186728589 >> 172.21.15.44:0/186728589 conn(0x7f16c40faa60 msgr2=0x7f16c40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:12.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.907+0000 7f16c9df2700 1 -- 172.21.15.44:0/186728589 shutdown_connections 2023-09-10T00:40:12.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.907+0000 7f16c9df2700 1 -- 172.21.15.44:0/186728589 wait complete. 2023-09-10T00:40:12.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.907+0000 7f16c9df2700 1 Processor -- start 2023-09-10T00:40:12.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.907+0000 7f16c9df2700 1 -- start start 2023-09-10T00:40:12.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c9df2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16c41010d0 0x7f16c4100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:12.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c9df2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16c4103a00 0x7f16c40ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:12.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c9df2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16c4106330 0x7f16c40ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:12.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c9df2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40ffd60 con 0x7f16c4103a00 2023-09-10T00:40:12.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c9df2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40ffed0 con 0x7f16c41010d0 2023-09-10T00:40:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c9df2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c41a01f0 con 0x7f16c4106330 2023-09-10T00:40:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16c4103a00 0x7f16c40ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16c4103a00 0x7f16c40ff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47864/0 (socket says 172.21.15.44:47864) 2023-09-10T00:40:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c2ffd700 1 -- 172.21.15.44:0/4029414469 learned_addr learned my addr 172.21.15.44:0/4029414469 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c37fe700 1 --2- 172.21.15.44:0/4029414469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16c41010d0 0x7f16c4100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:12.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c3fff700 1 --2- 172.21.15.44:0/4029414469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16c4106330 0x7f16c40ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:12.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c2ffd700 1 -- 172.21.15.44:0/4029414469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16c4106330 msgr2=0x7f16c40ff5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:12.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c2ffd700 1 --2- 172.21.15.44:0/4029414469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16c4106330 0x7f16c40ff5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c2ffd700 1 -- 172.21.15.44:0/4029414469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16c41010d0 msgr2=0x7f16c4100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:12.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c2ffd700 1 --2- 172.21.15.44:0/4029414469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16c41010d0 0x7f16c4100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:12.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c2ffd700 1 -- 172.21.15.44:0/4029414469 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b40095f0 con 0x7f16c4103a00 2023-09-10T00:40:12.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c3fff700 1 --2- 172.21.15.44:0/4029414469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16c4106330 0x7f16c40ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:40:12.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c2ffd700 1 --2- 172.21.15.44:0/4029414469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16c4103a00 0x7f16c40ff080 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f16b800e9a0 tx=0x7f16b800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:12.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c0ff9700 1 -- 172.21.15.44:0/4029414469 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8009790 con 0x7f16c4103a00 2023-09-10T00:40:12.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c0ff9700 1 -- 172.21.15.44:0/4029414469 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16b800cbd0 con 0x7f16c4103a00 2023-09-10T00:40:12.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c9df2700 1 -- 172.21.15.44:0/4029414469 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c41a04d0 con 0x7f16c4103a00 2023-09-10T00:40:12.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c0ff9700 1 -- 172.21.15.44:0/4029414469 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8018680 con 0x7f16c4103a00 2023-09-10T00:40:12.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.908+0000 7f16c9df2700 1 -- 172.21.15.44:0/4029414469 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c41a0a30 con 0x7f16c4103a00 2023-09-10T00:40:12.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.909+0000 7f16c9df2700 1 -- 172.21.15.44:0/4029414469 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c40fc5e0 con 0x7f16c4103a00 2023-09-10T00:40:12.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.911+0000 7f16c0ff9700 1 -- 172.21.15.44:0/4029414469 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f16b800cd40 con 0x7f16c4103a00 2023-09-10T00:40:12.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.911+0000 7f16c0ff9700 1 --2- 172.21.15.44:0/4029414469 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16ac0a0750 0x7f16ac0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:12.917 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.911+0000 7f16c37fe700 1 --2- 172.21.15.44:0/4029414469 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16ac0a0750 0x7f16ac0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:12.917 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.911+0000 7f16c0ff9700 1 -- 172.21.15.44:0/4029414469 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f16b8014070 con 0x7f16c4103a00 2023-09-10T00:40:12.917 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.912+0000 7f16c37fe700 1 --2- 172.21.15.44:0/4029414469 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16ac0a0750 0x7f16ac0a2c10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f16b4006010 tx=0x7f16b400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:12.915+0000 7f16c0ff9700 1 -- 172.21.15.44:0/4029414469 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f16b8051e10 con 0x7f16c4103a00 2023-09-10T00:40:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:12 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/94581442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:12 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1596940695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:40:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:12 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/94581442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:12 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1596940695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:40:13.118 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.118+0000 7f16c9df2700 1 -- 172.21.15.44:0/4029414469 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f16c410bef0 con 0x7f16c4103a00 2023-09-10T00:40:13.119 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.119+0000 7f16c0ff9700 1 -- 172.21.15.44:0/4029414469 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f16b8095970 con 0x7f16c4103a00 2023-09-10T00:40:13.119 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2023-09-10T00:40:13.121 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.121+0000 7f16c9df2700 1 -- 172.21.15.44:0/4029414469 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16ac0a0750 msgr2=0x7f16ac0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:13.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.121+0000 7f16c9df2700 1 --2- 172.21.15.44:0/4029414469 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16ac0a0750 0x7f16ac0a2c10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f16b4006010 tx=0x7f16b400b260 comp rx=0 tx=0).stop 2023-09-10T00:40:13.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.121+0000 7f16c9df2700 1 -- 172.21.15.44:0/4029414469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16c4103a00 msgr2=0x7f16c40ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:13.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.121+0000 7f16c9df2700 1 --2- 172.21.15.44:0/4029414469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16c4103a00 0x7f16c40ff080 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f16b800e9a0 tx=0x7f16b800ed60 comp rx=0 tx=0).stop 2023-09-10T00:40:13.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.121+0000 7f16c9df2700 1 -- 172.21.15.44:0/4029414469 shutdown_connections 2023-09-10T00:40:13.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.121+0000 7f16c9df2700 1 --2- 172.21.15.44:0/4029414469 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16ac0a0750 0x7f16ac0a2c10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.121+0000 7f16c9df2700 1 --2- 172.21.15.44:0/4029414469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16c4106330 0x7f16c40ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.121+0000 7f16c9df2700 1 --2- 172.21.15.44:0/4029414469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16c41010d0 0x7f16c4100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.121+0000 7f16c9df2700 1 --2- 172.21.15.44:0/4029414469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16c4103a00 0x7f16c40ff080 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.121+0000 7f16c9df2700 1 -- 172.21.15.44:0/4029414469 >> 172.21.15.44:0/4029414469 conn(0x7f16c40faa60 msgr2=0x7f16c4101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:13.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.121+0000 7f16c9df2700 1 -- 172.21.15.44:0/4029414469 shutdown_connections 2023-09-10T00:40:13.124 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.121+0000 7f16c9df2700 1 -- 172.21.15.44:0/4029414469 wait complete. 2023-09-10T00:40:13.134 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-10T00:40:13.264 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.262+0000 7fe0bb4ea700 1 -- 172.21.15.44:0/515357818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b4100710 msgr2=0x7fe0b4100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:13.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.262+0000 7fe0bb4ea700 1 --2- 172.21.15.44:0/515357818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b4100710 0x7fe0b4100b70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80099c0 tx=0x7fe0a8009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.262+0000 7fe0bb4ea700 1 -- 172.21.15.44:0/515357818 shutdown_connections 2023-09-10T00:40:13.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.262+0000 7fe0bb4ea700 1 --2- 172.21.15.44:0/515357818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0b41010b0 0x7fe0b4107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.262+0000 7fe0bb4ea700 1 --2- 172.21.15.44:0/515357818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b40ff520 0x7fe0b40ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.262+0000 7fe0bb4ea700 1 --2- 172.21.15.44:0/515357818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b4100710 0x7fe0b4100b70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.262+0000 7fe0bb4ea700 1 -- 172.21.15.44:0/515357818 >> 172.21.15.44:0/515357818 conn(0x7fe0b40faa90 msgr2=0x7fe0b40fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:13.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.262+0000 7fe0bb4ea700 1 -- 172.21.15.44:0/515357818 shutdown_connections 2023-09-10T00:40:13.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.262+0000 7fe0bb4ea700 1 -- 172.21.15.44:0/515357818 wait complete. 2023-09-10T00:40:13.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.263+0000 7fe0bb4ea700 1 Processor -- start 2023-09-10T00:40:13.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.263+0000 7fe0bb4ea700 1 -- start start 2023-09-10T00:40:13.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.263+0000 7fe0bb4ea700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b40ff520 0x7fe0b4195150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:13.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.263+0000 7fe0bb4ea700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b4100710 0x7fe0b4195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:13.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.263+0000 7fe0bb4ea700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0b41010b0 0x7fe0b419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:13.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.263+0000 7fe0bb4ea700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b419ccd0 con 0x7fe0b40ff520 2023-09-10T00:40:13.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.263+0000 7fe0bb4ea700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b419ce10 con 0x7fe0b4100710 2023-09-10T00:40:13.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.263+0000 7fe0bb4ea700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b419cfa0 con 0x7fe0b41010b0 2023-09-10T00:40:13.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.263+0000 7fe0b9286700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b40ff520 0x7fe0b4195150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:13.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.263+0000 7fe0b9286700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b40ff520 0x7fe0b4195150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47876/0 (socket says 172.21.15.44:47876) 2023-09-10T00:40:13.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.263+0000 7fe0b9286700 1 -- 172.21.15.44:0/1153509642 learned_addr learned my addr 172.21.15.44:0/1153509642 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:13.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.263+0000 7fe0b9a87700 1 --2- 172.21.15.44:0/1153509642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0b41010b0 0x7fe0b419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:13.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.263+0000 7fe0b8a85700 1 --2- 172.21.15.44:0/1153509642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b4100710 0x7fe0b4195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:13.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.264+0000 7fe0b9286700 1 -- 172.21.15.44:0/1153509642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0b41010b0 msgr2=0x7fe0b419c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:13.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.264+0000 7fe0b9286700 1 --2- 172.21.15.44:0/1153509642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0b41010b0 0x7fe0b419c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.264+0000 7fe0b9286700 1 -- 172.21.15.44:0/1153509642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b4100710 msgr2=0x7fe0b4195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:13.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.264+0000 7fe0b9286700 1 --2- 172.21.15.44:0/1153509642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b4100710 0x7fe0b4195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.264+0000 7fe0b9286700 1 -- 172.21.15.44:0/1153509642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a80095f0 con 0x7fe0b40ff520 2023-09-10T00:40:13.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.264+0000 7fe0b9286700 1 --2- 172.21.15.44:0/1153509642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b40ff520 0x7fe0b4195150 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000d980 tx=0x7fe0b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:13.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.264+0000 7fe0a67fc700 1 -- 172.21.15.44:0/1153509642 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b000b950 con 0x7fe0b40ff520 2023-09-10T00:40:13.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.264+0000 7fe0bb4ea700 1 -- 172.21.15.44:0/1153509642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b419d280 con 0x7fe0b40ff520 2023-09-10T00:40:13.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.264+0000 7fe0a67fc700 1 -- 172.21.15.44:0/1153509642 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe0b0007620 con 0x7fe0b40ff520 2023-09-10T00:40:13.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.264+0000 7fe0a67fc700 1 -- 172.21.15.44:0/1153509642 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b000f3c0 con 0x7fe0b40ff520 2023-09-10T00:40:13.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.264+0000 7fe0bb4ea700 1 -- 172.21.15.44:0/1153509642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b419d790 con 0x7fe0b40ff520 2023-09-10T00:40:13.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.265+0000 7fe0bb4ea700 1 -- 172.21.15.44:0/1153509642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b410ac40 con 0x7fe0b40ff520 2023-09-10T00:40:13.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.266+0000 7fe0a67fc700 1 -- 172.21.15.44:0/1153509642 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fe0b000bab0 con 0x7fe0b40ff520 2023-09-10T00:40:13.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.267+0000 7fe0a67fc700 1 --2- 172.21.15.44:0/1153509642 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe0a00a06d0 0x7fe0a00a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:13.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.267+0000 7fe0b8a85700 1 --2- 172.21.15.44:0/1153509642 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe0a00a06d0 0x7fe0a00a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:13.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.267+0000 7fe0a67fc700 1 -- 172.21.15.44:0/1153509642 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe0b00cc790 con 0x7fe0b40ff520 2023-09-10T00:40:13.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.267+0000 7fe0b8a85700 1 --2- 172.21.15.44:0/1153509642 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe0a00a06d0 0x7fe0a00a2b90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800c010 tx=0x7fe0a8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:13.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.271+0000 7fe0a67fc700 1 -- 172.21.15.44:0/1153509642 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe0b0095e50 con 0x7fe0b40ff520 2023-09-10T00:40:13.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.473+0000 7fe0bb4ea700 1 -- 172.21.15.44:0/1153509642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe0b4106520 con 0x7fe0b40ff520 2023-09-10T00:40:13.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.474+0000 7fe0a67fc700 1 -- 172.21.15.44:0/1153509642 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe0b0099470 con 0x7fe0b40ff520 2023-09-10T00:40:13.474 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:40:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.476+0000 7fe0bb4ea700 1 -- 172.21.15.44:0/1153509642 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe0a00a06d0 msgr2=0x7fe0a00a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.476+0000 7fe0bb4ea700 1 --2- 172.21.15.44:0/1153509642 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe0a00a06d0 0x7fe0a00a2b90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800c010 tx=0x7fe0a8005fb0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.476+0000 7fe0bb4ea700 1 -- 172.21.15.44:0/1153509642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b40ff520 msgr2=0x7fe0b4195150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.476+0000 7fe0bb4ea700 1 --2- 172.21.15.44:0/1153509642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b40ff520 0x7fe0b4195150 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000d980 tx=0x7fe0b000dc90 comp rx=0 tx=0).stop 2023-09-10T00:40:13.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.476+0000 7fe0bb4ea700 1 -- 172.21.15.44:0/1153509642 shutdown_connections 2023-09-10T00:40:13.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.476+0000 7fe0bb4ea700 1 --2- 172.21.15.44:0/1153509642 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe0a00a06d0 0x7fe0a00a2b90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.476+0000 7fe0bb4ea700 1 --2- 172.21.15.44:0/1153509642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0b41010b0 0x7fe0b419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.476+0000 7fe0bb4ea700 1 --2- 172.21.15.44:0/1153509642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b40ff520 0x7fe0b4195150 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.476+0000 7fe0bb4ea700 1 --2- 172.21.15.44:0/1153509642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b4100710 0x7fe0b4195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.476+0000 7fe0bb4ea700 1 -- 172.21.15.44:0/1153509642 >> 172.21.15.44:0/1153509642 conn(0x7fe0b40faa90 msgr2=0x7fe0b40fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:13.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.476+0000 7fe0bb4ea700 1 -- 172.21.15.44:0/1153509642 shutdown_connections 2023-09-10T00:40:13.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.476+0000 7fe0bb4ea700 1 -- 172.21.15.44:0/1153509642 wait complete. 2023-09-10T00:40:13.489 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:40:13.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.616+0000 7f63e614e700 1 -- 172.21.15.44:0/336968933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e0101070 msgr2=0x7f63e0107920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:13.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.616+0000 7f63e614e700 1 --2- 172.21.15.44:0/336968933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e0101070 0x7f63e0107920 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f63d800b0d0 tx=0x7f63d800b3e0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.616+0000 7f63e614e700 1 -- 172.21.15.44:0/336968933 shutdown_connections 2023-09-10T00:40:13.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.616+0000 7f63e614e700 1 --2- 172.21.15.44:0/336968933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e0101070 0x7f63e0107920 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.616+0000 7f63e614e700 1 --2- 172.21.15.44:0/336968933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e00ff4e0 0x7f63e00ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.616+0000 7f63e614e700 1 --2- 172.21.15.44:0/336968933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63e01006d0 0x7f63e0100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.616+0000 7f63e614e700 1 -- 172.21.15.44:0/336968933 >> 172.21.15.44:0/336968933 conn(0x7f63e00faa30 msgr2=0x7f63e00fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:13.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.616+0000 7f63e614e700 1 -- 172.21.15.44:0/336968933 shutdown_connections 2023-09-10T00:40:13.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.616+0000 7f63e614e700 1 -- 172.21.15.44:0/336968933 wait complete. 2023-09-10T00:40:13.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.616+0000 7f63e614e700 1 Processor -- start 2023-09-10T00:40:13.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63e614e700 1 -- start start 2023-09-10T00:40:13.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63e614e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e00ff4e0 0x7f63e00682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:13.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63e614e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63e01006d0 0x7f63e019aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:13.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63e614e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e0101070 0x7f63e019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:13.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63e614e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e019de10 con 0x7f63e00ff4e0 2023-09-10T00:40:13.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63e614e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e019df80 con 0x7f63e01006d0 2023-09-10T00:40:13.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63e614e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e019e140 con 0x7f63e0101070 2023-09-10T00:40:13.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63df7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e00ff4e0 0x7f63e00682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:13.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63df7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e00ff4e0 0x7f63e00682f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47890/0 (socket says 172.21.15.44:47890) 2023-09-10T00:40:13.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63df7fe700 1 -- 172.21.15.44:0/4003554115 learned_addr learned my addr 172.21.15.44:0/4003554115 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:13.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63dffff700 1 --2- 172.21.15.44:0/4003554115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e0101070 0x7f63e019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:13.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63deffd700 1 --2- 172.21.15.44:0/4003554115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63e01006d0 0x7f63e019aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:13.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63df7fe700 1 -- 172.21.15.44:0/4003554115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e0101070 msgr2=0x7f63e019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:13.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63df7fe700 1 --2- 172.21.15.44:0/4003554115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e0101070 0x7f63e019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63df7fe700 1 -- 172.21.15.44:0/4003554115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63e01006d0 msgr2=0x7f63e019aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:13.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63df7fe700 1 --2- 172.21.15.44:0/4003554115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63e01006d0 0x7f63e019aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63df7fe700 1 -- 172.21.15.44:0/4003554115 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d8009d00 con 0x7f63e00ff4e0 2023-09-10T00:40:13.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.617+0000 7f63df7fe700 1 --2- 172.21.15.44:0/4003554115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e00ff4e0 0x7f63e00682f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f63c800d980 tx=0x7f63c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:13.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.618+0000 7f63dcff9700 1 -- 172.21.15.44:0/4003554115 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8004020 con 0x7f63e00ff4e0 2023-09-10T00:40:13.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.618+0000 7f63dcff9700 1 -- 172.21.15.44:0/4003554115 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63c8009730 con 0x7f63e00ff4e0 2023-09-10T00:40:13.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.618+0000 7f63e614e700 1 -- 172.21.15.44:0/4003554115 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e019e420 con 0x7f63e00ff4e0 2023-09-10T00:40:13.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.618+0000 7f63dcff9700 1 -- 172.21.15.44:0/4003554115 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8003a00 con 0x7f63e00ff4e0 2023-09-10T00:40:13.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.618+0000 7f63e614e700 1 -- 172.21.15.44:0/4003554115 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e019e980 con 0x7f63e00ff4e0 2023-09-10T00:40:13.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.619+0000 7f63e614e700 1 -- 172.21.15.44:0/4003554115 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e0106a70 con 0x7f63e00ff4e0 2023-09-10T00:40:13.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.620+0000 7f63dcff9700 1 -- 172.21.15.44:0/4003554115 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f63c80098a0 con 0x7f63e00ff4e0 2023-09-10T00:40:13.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.620+0000 7f63dcff9700 1 --2- 172.21.15.44:0/4003554115 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f63cc0a0790 0x7f63cc0a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:13.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.621+0000 7f63deffd700 1 --2- 172.21.15.44:0/4003554115 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f63cc0a0790 0x7f63cc0a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:13.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.621+0000 7f63dcff9700 1 -- 172.21.15.44:0/4003554115 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f63c80cc000 con 0x7f63e00ff4e0 2023-09-10T00:40:13.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.621+0000 7f63deffd700 1 --2- 172.21.15.44:0/4003554115 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f63cc0a0790 0x7f63cc0a2c50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f63d0005950 tx=0x7f63d00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:13.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.624+0000 7f63dcff9700 1 -- 172.21.15.44:0/4003554115 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f63c8095740 con 0x7f63e00ff4e0 2023-09-10T00:40:13.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:13 smithi044 ceph-mon[98676]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-10T00:40:13.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:13 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2820181446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:40:13.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:13 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/4029414469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:40:13.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:13 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1153509642' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:40:13.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.874+0000 7f63e614e700 1 -- 172.21.15.44:0/4003554115 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63e002cfc0 con 0x7f63e00ff4e0 2023-09-10T00:40:13.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.875+0000 7f63dcff9700 1 -- 172.21.15.44:0/4003554115 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11369 (secure 0 0 0) 0x7f63c8098d60 con 0x7f63e00ff4e0 2023-09-10T00:40:13.876 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:13.876 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-10T00:38:39.183533+0000","modified":"2023-09-10T00:39:02.480227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24605,"mds_1":24574,"mds_2":24572,"mds_3":24515,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14751":{"gid":14751,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/3958270510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":3958270510},{"type":"v1","addr":"172.21.15.44:6839","nonce":3958270510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14763":{"gid":14763,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/1241002552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1241002552},{"type":"v1","addr":"172.21.15.44:6841","nonce":1241002552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24556":{"gid":24556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/3222652693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":3222652693},{"type":"v1","addr":"172.21.15.131:6839","nonce":3222652693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24605":{"gid":24605,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/2622966260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2622966260},{"type":"v1","addr":"172.21.15.99:6839","nonce":2622966260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-10T00:40:13.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.877+0000 7f63e614e700 1 -- 172.21.15.44:0/4003554115 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f63cc0a0790 msgr2=0x7f63cc0a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:13.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.877+0000 7f63e614e700 1 --2- 172.21.15.44:0/4003554115 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f63cc0a0790 0x7f63cc0a2c50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f63d0005950 tx=0x7f63d00058e0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.877+0000 7f63e614e700 1 -- 172.21.15.44:0/4003554115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e00ff4e0 msgr2=0x7f63e00682f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:13.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.877+0000 7f63e614e700 1 --2- 172.21.15.44:0/4003554115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e00ff4e0 0x7f63e00682f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f63c800d980 tx=0x7f63c800dc90 comp rx=0 tx=0).stop 2023-09-10T00:40:13.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.878+0000 7f63e614e700 1 -- 172.21.15.44:0/4003554115 shutdown_connections 2023-09-10T00:40:13.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.878+0000 7f63e614e700 1 --2- 172.21.15.44:0/4003554115 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f63cc0a0790 0x7f63cc0a2c50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.878+0000 7f63e614e700 1 --2- 172.21.15.44:0/4003554115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e0101070 0x7f63e019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.878+0000 7f63e614e700 1 --2- 172.21.15.44:0/4003554115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63e00ff4e0 0x7f63e00682f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.878+0000 7f63e614e700 1 --2- 172.21.15.44:0/4003554115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63e01006d0 0x7f63e019aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:13.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.878+0000 7f63e614e700 1 -- 172.21.15.44:0/4003554115 >> 172.21.15.44:0/4003554115 conn(0x7f63e00faa30 msgr2=0x7f63e0103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:13.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.878+0000 7f63e614e700 1 -- 172.21.15.44:0/4003554115 shutdown_connections 2023-09-10T00:40:13.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:13.878+0000 7f63e614e700 1 -- 172.21.15.44:0/4003554115 wait complete. 2023-09-10T00:40:13.880 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2023-09-10T00:40:13.893 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:40:14.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.019+0000 7f5281678700 1 -- 172.21.15.44:0/1665514157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c101060 msgr2=0x7f527c103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:14.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.019+0000 7f5281678700 1 --2- 172.21.15.44:0/1665514157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c101060 0x7f527c103450 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f526c009990 tx=0x7f526c009ca0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.019+0000 7f5281678700 1 -- 172.21.15.44:0/1665514157 shutdown_connections 2023-09-10T00:40:14.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.019+0000 7f5281678700 1 --2- 172.21.15.44:0/1665514157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f527c1062c0 0x7f527c108730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.019+0000 7f5281678700 1 --2- 172.21.15.44:0/1665514157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c101060 0x7f527c103450 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.019+0000 7f5281678700 1 --2- 172.21.15.44:0/1665514157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f527c103990 0x7f527c105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.019+0000 7f5281678700 1 -- 172.21.15.44:0/1665514157 >> 172.21.15.44:0/1665514157 conn(0x7f527c0faa30 msgr2=0x7f527c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:14.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.019+0000 7f5281678700 1 -- 172.21.15.44:0/1665514157 shutdown_connections 2023-09-10T00:40:14.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.019+0000 7f5281678700 1 -- 172.21.15.44:0/1665514157 wait complete. 2023-09-10T00:40:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.019+0000 7f5281678700 1 Processor -- start 2023-09-10T00:40:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f5281678700 1 -- start start 2023-09-10T00:40:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f5281678700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f527c101060 0x7f527c1009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f5281678700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c103990 0x7f527c0ff010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f5281678700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f527c1062c0 0x7f527c0ff550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:14.026 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f5281678700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c0ffcf0 con 0x7f527c103990 2023-09-10T00:40:14.026 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f5281678700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c0ffe60 con 0x7f527c101060 2023-09-10T00:40:14.026 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f5281678700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c1a01c0 con 0x7f527c1062c0 2023-09-10T00:40:14.026 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f527a7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c103990 0x7f527c0ff010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:14.027 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f527a7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c103990 0x7f527c0ff010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47906/0 (socket says 172.21.15.44:47906) 2023-09-10T00:40:14.027 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f527a7fc700 1 -- 172.21.15.44:0/1999601268 learned_addr learned my addr 172.21.15.44:0/1999601268 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:14.027 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f527affd700 1 --2- 172.21.15.44:0/1999601268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f527c101060 0x7f527c1009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:14.027 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f527b7fe700 1 --2- 172.21.15.44:0/1999601268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f527c1062c0 0x7f527c0ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:14.027 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f527a7fc700 1 -- 172.21.15.44:0/1999601268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f527c1062c0 msgr2=0x7f527c0ff550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:14.028 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f527a7fc700 1 --2- 172.21.15.44:0/1999601268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f527c1062c0 0x7f527c0ff550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.028 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f527a7fc700 1 -- 172.21.15.44:0/1999601268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f527c101060 msgr2=0x7f527c1009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:14.028 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f527a7fc700 1 --2- 172.21.15.44:0/1999601268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f527c101060 0x7f527c1009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.028 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f527a7fc700 1 -- 172.21.15.44:0/1999601268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526c0095f0 con 0x7f527c103990 2023-09-10T00:40:14.028 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.020+0000 7f527b7fe700 1 --2- 172.21.15.44:0/1999601268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f527c1062c0 0x7f527c0ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.021+0000 7f527a7fc700 1 --2- 172.21.15.44:0/1999601268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c103990 0x7f527c0ff010 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f527000d950 tx=0x7f527000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.021+0000 7f5263fff700 1 -- 172.21.15.44:0/1999601268 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270009940 con 0x7f527c103990 2023-09-10T00:40:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.021+0000 7f5263fff700 1 -- 172.21.15.44:0/1999601268 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5270010460 con 0x7f527c103990 2023-09-10T00:40:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.021+0000 7f5281678700 1 -- 172.21.15.44:0/1999601268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527c1a04a0 con 0x7f527c103990 2023-09-10T00:40:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.021+0000 7f5263fff700 1 -- 172.21.15.44:0/1999601268 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527000f620 con 0x7f527c103990 2023-09-10T00:40:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.021+0000 7f5281678700 1 -- 172.21.15.44:0/1999601268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527c1a09b0 con 0x7f527c103990 2023-09-10T00:40:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.022+0000 7f5281678700 1 -- 172.21.15.44:0/1999601268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527c0fc5b0 con 0x7f527c103990 2023-09-10T00:40:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.023+0000 7f5263fff700 1 -- 172.21.15.44:0/1999601268 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f52700105d0 con 0x7f527c103990 2023-09-10T00:40:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.024+0000 7f5263fff700 1 --2- 172.21.15.44:0/1999601268 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f52640a0710 0x7f52640a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:14.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.024+0000 7f527affd700 1 --2- 172.21.15.44:0/1999601268 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f52640a0710 0x7f52640a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:14.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.024+0000 7f5263fff700 1 -- 172.21.15.44:0/1999601268 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f52700cbe90 con 0x7f527c103990 2023-09-10T00:40:14.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.024+0000 7f527affd700 1 --2- 172.21.15.44:0/1999601268 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f52640a0710 0x7f52640a2bd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f526c00b5c0 tx=0x7f526c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:14.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.027+0000 7f5263fff700 1 -- 172.21.15.44:0/1999601268 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f52700954f0 con 0x7f527c103990 2023-09-10T00:40:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:13 smithi131 ceph-mon[103713]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-10T00:40:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:13 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2820181446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:40:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:13 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/4029414469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:40:14.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:13 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1153509642' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:40:14.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:13 smithi099 ceph-mon[102692]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-10T00:40:14.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:13 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2820181446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:40:14.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:13 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/4029414469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:40:14.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:13 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1153509642' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:40:14.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.233+0000 7f5281678700 1 -- 172.21.15.44:0/1999601268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f527c10bee0 con 0x7f527c103990 2023-09-10T00:40:14.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.235+0000 7f5263fff700 1 -- 172.21.15.44:0/1999601268 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24340 (secure 0 0 0) 0x7f5270016090 con 0x7f527c103990 2023-09-10T00:40:14.236 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:14.236 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":89,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:39:17.515738+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:40:14.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.237+0000 7f5281678700 1 -- 172.21.15.44:0/1999601268 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f52640a0710 msgr2=0x7f52640a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:14.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.237+0000 7f5281678700 1 --2- 172.21.15.44:0/1999601268 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f52640a0710 0x7f52640a2bd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f526c00b5c0 tx=0x7f526c005fb0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.237+0000 7f5281678700 1 -- 172.21.15.44:0/1999601268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c103990 msgr2=0x7f527c0ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:14.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.237+0000 7f5281678700 1 --2- 172.21.15.44:0/1999601268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c103990 0x7f527c0ff010 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f527000d950 tx=0x7f527000dc60 comp rx=0 tx=0).stop 2023-09-10T00:40:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.237+0000 7f5281678700 1 -- 172.21.15.44:0/1999601268 shutdown_connections 2023-09-10T00:40:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.237+0000 7f5281678700 1 --2- 172.21.15.44:0/1999601268 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f52640a0710 0x7f52640a2bd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.237+0000 7f5281678700 1 --2- 172.21.15.44:0/1999601268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f527c1062c0 0x7f527c0ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.237+0000 7f5281678700 1 --2- 172.21.15.44:0/1999601268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f527c101060 0x7f527c1009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.237+0000 7f5281678700 1 --2- 172.21.15.44:0/1999601268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c103990 0x7f527c0ff010 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.237+0000 7f5281678700 1 -- 172.21.15.44:0/1999601268 >> 172.21.15.44:0/1999601268 conn(0x7f527c0faa30 msgr2=0x7f527c101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:14.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.237+0000 7f5281678700 1 -- 172.21.15.44:0/1999601268 shutdown_connections 2023-09-10T00:40:14.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.237+0000 7f5281678700 1 -- 172.21.15.44:0/1999601268 wait complete. 2023-09-10T00:40:14.250 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-10T00:34:44.456741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-10T00:38:37.194891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-10T00:38:38.573268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-10T00:38:42.198896+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-10T00:40:14.251 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-10T00:40:14.380 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.379+0000 7ff4b4ef6700 1 -- 172.21.15.44:0/1417331622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4b0102210 msgr2=0x7ff4b0108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:14.380 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.379+0000 7ff4b4ef6700 1 --2- 172.21.15.44:0/1417331622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4b0102210 0x7ff4b0108ac0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800b0a0 tx=0x7ff4a800b3b0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.379+0000 7ff4b4ef6700 1 -- 172.21.15.44:0/1417331622 shutdown_connections 2023-09-10T00:40:14.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.379+0000 7ff4b4ef6700 1 --2- 172.21.15.44:0/1417331622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4b0102210 0x7ff4b0108ac0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.379+0000 7ff4b4ef6700 1 --2- 172.21.15.44:0/1417331622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4b00fe8a0 0x7ff4b00fecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.379+0000 7ff4b4ef6700 1 --2- 172.21.15.44:0/1417331622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b0101870 0x7ff4b0101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.379+0000 7ff4b4ef6700 1 -- 172.21.15.44:0/1417331622 >> 172.21.15.44:0/1417331622 conn(0x7ff4b00fa1d0 msgr2=0x7ff4b00fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:14.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.379+0000 7ff4b4ef6700 1 -- 172.21.15.44:0/1417331622 shutdown_connections 2023-09-10T00:40:14.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.379+0000 7ff4b4ef6700 1 -- 172.21.15.44:0/1417331622 wait complete. 2023-09-10T00:40:14.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.380+0000 7ff4b4ef6700 1 Processor -- start 2023-09-10T00:40:14.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.380+0000 7ff4b4ef6700 1 -- start start 2023-09-10T00:40:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4b4ef6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4b00fe8a0 0x7ff4b01962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4b4ef6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b0101870 0x7ff4b01967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4b4ef6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4b0102210 0x7ff4b019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4b4ef6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b019de30 con 0x7ff4b0102210 2023-09-10T00:40:14.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4b4ef6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b019df70 con 0x7ff4b0101870 2023-09-10T00:40:14.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4b4ef6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b019e100 con 0x7ff4b00fe8a0 2023-09-10T00:40:14.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4aed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4b0102210 0x7ff4b019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:14.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4aed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4b0102210 0x7ff4b019d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47922/0 (socket says 172.21.15.44:47922) 2023-09-10T00:40:14.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4aed9d700 1 -- 172.21.15.44:0/1873298365 learned_addr learned my addr 172.21.15.44:0/1873298365 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:14.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4add9b700 1 --2- 172.21.15.44:0/1873298365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b0101870 0x7ff4b01967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:14.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4ae59c700 1 --2- 172.21.15.44:0/1873298365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4b00fe8a0 0x7ff4b01962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:14.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4aed9d700 1 -- 172.21.15.44:0/1873298365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4b00fe8a0 msgr2=0x7ff4b01962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:14.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4aed9d700 1 --2- 172.21.15.44:0/1873298365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4b00fe8a0 0x7ff4b01962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4aed9d700 1 -- 172.21.15.44:0/1873298365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b0101870 msgr2=0x7ff4b01967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:14.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4aed9d700 1 --2- 172.21.15.44:0/1873298365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b0101870 0x7ff4b01967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4aed9d700 1 -- 172.21.15.44:0/1873298365 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a8009d00 con 0x7ff4b0102210 2023-09-10T00:40:14.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff4aed9d700 1 --2- 172.21.15.44:0/1873298365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4b0102210 0x7ff4b019d860 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800b070 tx=0x7ff4a8004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:14.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.381+0000 7ff49f7fe700 1 -- 172.21.15.44:0/1873298365 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800e040 con 0x7ff4b0102210 2023-09-10T00:40:14.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.382+0000 7ff49f7fe700 1 -- 172.21.15.44:0/1873298365 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4a8003e90 con 0x7ff4b0102210 2023-09-10T00:40:14.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.382+0000 7ff4b4ef6700 1 -- 172.21.15.44:0/1873298365 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b019e380 con 0x7ff4b0102210 2023-09-10T00:40:14.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.382+0000 7ff49f7fe700 1 -- 172.21.15.44:0/1873298365 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8004020 con 0x7ff4b0102210 2023-09-10T00:40:14.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.382+0000 7ff4b4ef6700 1 -- 172.21.15.44:0/1873298365 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b019e800 con 0x7ff4b0102210 2023-09-10T00:40:14.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.383+0000 7ff4b4ef6700 1 -- 172.21.15.44:0/1873298365 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b01075b0 con 0x7ff4b0102210 2023-09-10T00:40:14.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.384+0000 7ff49f7fe700 1 -- 172.21.15.44:0/1873298365 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7ff4a8003a00 con 0x7ff4b0102210 2023-09-10T00:40:14.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.384+0000 7ff49f7fe700 1 --2- 172.21.15.44:0/1873298365 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff4980a06d0 0x7ff4980a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:14.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.384+0000 7ff49f7fe700 1 -- 172.21.15.44:0/1873298365 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff4a80cd5e0 con 0x7ff4b0102210 2023-09-10T00:40:14.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.384+0000 7ff4ae59c700 1 --2- 172.21.15.44:0/1873298365 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff4980a06d0 0x7ff4980a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:14.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.385+0000 7ff4ae59c700 1 --2- 172.21.15.44:0/1873298365 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff4980a06d0 0x7ff4980a2b90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0067560 tx=0x7ff4a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:14.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.388+0000 7ff49f7fe700 1 -- 172.21.15.44:0/1873298365 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff4a8096d30 con 0x7ff4b0102210 2023-09-10T00:40:14.590 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.590+0000 7ff4b4ef6700 1 -- 172.21.15.44:0/1873298365 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4b010bf00 con 0x7ff4b0102210 2023-09-10T00:40:14.591 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.590+0000 7ff49f7fe700 1 -- 172.21.15.44:0/1873298365 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff4a809a350 con 0x7ff4b0102210 2023-09-10T00:40:14.591 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2023-09-10T00:40:14.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.592+0000 7ff4b4ef6700 1 -- 172.21.15.44:0/1873298365 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff4980a06d0 msgr2=0x7ff4980a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:14.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.592+0000 7ff4b4ef6700 1 --2- 172.21.15.44:0/1873298365 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff4980a06d0 0x7ff4980a2b90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0067560 tx=0x7ff4a000b410 comp rx=0 tx=0).stop 2023-09-10T00:40:14.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.592+0000 7ff4b4ef6700 1 -- 172.21.15.44:0/1873298365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4b0102210 msgr2=0x7ff4b019d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.592+0000 7ff4b4ef6700 1 --2- 172.21.15.44:0/1873298365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4b0102210 0x7ff4b019d860 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800b070 tx=0x7ff4a8004c80 comp rx=0 tx=0).stop 2023-09-10T00:40:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.592+0000 7ff4b4ef6700 1 -- 172.21.15.44:0/1873298365 shutdown_connections 2023-09-10T00:40:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.592+0000 7ff4b4ef6700 1 --2- 172.21.15.44:0/1873298365 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff4980a06d0 0x7ff4980a2b90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.592+0000 7ff4b4ef6700 1 --2- 172.21.15.44:0/1873298365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4b0102210 0x7ff4b019d860 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.592+0000 7ff4b4ef6700 1 --2- 172.21.15.44:0/1873298365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4b00fe8a0 0x7ff4b01962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.592+0000 7ff4b4ef6700 1 --2- 172.21.15.44:0/1873298365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b0101870 0x7ff4b01967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.592+0000 7ff4b4ef6700 1 -- 172.21.15.44:0/1873298365 >> 172.21.15.44:0/1873298365 conn(0x7ff4b00fa1d0 msgr2=0x7ff4b00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:14.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.593+0000 7ff4b4ef6700 1 -- 172.21.15.44:0/1873298365 shutdown_connections 2023-09-10T00:40:14.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.593+0000 7ff4b4ef6700 1 -- 172.21.15.44:0/1873298365 wait complete. 2023-09-10T00:40:14.605 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-10T00:40:14.739 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.737+0000 7eff84902700 1 -- 172.21.15.44:0/2192279420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff801006e0 msgr2=0x7eff80100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:14.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.737+0000 7eff84902700 1 --2- 172.21.15.44:0/2192279420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff801006e0 0x7eff80100b40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7eff680099c0 tx=0x7eff68009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.737+0000 7eff84902700 1 -- 172.21.15.44:0/2192279420 shutdown_connections 2023-09-10T00:40:14.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.737+0000 7eff84902700 1 --2- 172.21.15.44:0/2192279420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff80101080 0x7eff80107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.737+0000 7eff84902700 1 --2- 172.21.15.44:0/2192279420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff800ff4f0 0x7eff800ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.737+0000 7eff84902700 1 --2- 172.21.15.44:0/2192279420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff801006e0 0x7eff80100b40 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.737+0000 7eff84902700 1 -- 172.21.15.44:0/2192279420 >> 172.21.15.44:0/2192279420 conn(0x7eff800faa60 msgr2=0x7eff800fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.737+0000 7eff84902700 1 -- 172.21.15.44:0/2192279420 shutdown_connections 2023-09-10T00:40:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.737+0000 7eff84902700 1 -- 172.21.15.44:0/2192279420 wait complete. 2023-09-10T00:40:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.738+0000 7eff84902700 1 Processor -- start 2023-09-10T00:40:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.738+0000 7eff84902700 1 -- start start 2023-09-10T00:40:14.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.738+0000 7eff84902700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff800ff4f0 0x7eff800682e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:14.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.738+0000 7eff84902700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff801006e0 0x7eff8019af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:14.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.738+0000 7eff84902700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff80101080 0x7eff8019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:14.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.738+0000 7eff84902700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8019dd40 con 0x7eff800ff4f0 2023-09-10T00:40:14.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.738+0000 7eff84902700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8019deb0 con 0x7eff80101080 2023-09-10T00:40:14.743 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.738+0000 7eff84902700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8019e070 con 0x7eff801006e0 2023-09-10T00:40:14.743 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.738+0000 7eff7e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff800ff4f0 0x7eff800682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:14.743 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.738+0000 7eff7e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff800ff4f0 0x7eff800682e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47932/0 (socket says 172.21.15.44:47932) 2023-09-10T00:40:14.743 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.738+0000 7eff7e59c700 1 -- 172.21.15.44:0/336211637 learned_addr learned my addr 172.21.15.44:0/336211637 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:14.743 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.739+0000 7eff7ed9d700 1 --2- 172.21.15.44:0/336211637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff80101080 0x7eff8019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:14.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.739+0000 7eff7dd9b700 1 --2- 172.21.15.44:0/336211637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff801006e0 0x7eff8019af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:14.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.739+0000 7eff7e59c700 1 -- 172.21.15.44:0/336211637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff801006e0 msgr2=0x7eff8019af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:14.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.739+0000 7eff7e59c700 1 --2- 172.21.15.44:0/336211637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff801006e0 0x7eff8019af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.739+0000 7eff7e59c700 1 -- 172.21.15.44:0/336211637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff80101080 msgr2=0x7eff8019d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:14.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.739+0000 7eff7e59c700 1 --2- 172.21.15.44:0/336211637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff80101080 0x7eff8019d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.739+0000 7eff7e59c700 1 -- 172.21.15.44:0/336211637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff680095f0 con 0x7eff800ff4f0 2023-09-10T00:40:14.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.739+0000 7eff7e59c700 1 --2- 172.21.15.44:0/336211637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff800ff4f0 0x7eff800682e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7eff7000e970 tx=0x7eff7000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:14.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.739+0000 7eff777fe700 1 -- 172.21.15.44:0/336211637 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70009790 con 0x7eff800ff4f0 2023-09-10T00:40:14.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.739+0000 7eff84902700 1 -- 172.21.15.44:0/336211637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8019e350 con 0x7eff800ff4f0 2023-09-10T00:40:14.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.739+0000 7eff777fe700 1 -- 172.21.15.44:0/336211637 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff7000eec0 con 0x7eff800ff4f0 2023-09-10T00:40:14.746 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.739+0000 7eff777fe700 1 -- 172.21.15.44:0/336211637 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70003c10 con 0x7eff800ff4f0 2023-09-10T00:40:14.746 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.739+0000 7eff84902700 1 -- 172.21.15.44:0/336211637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8019e880 con 0x7eff800ff4f0 2023-09-10T00:40:14.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.740+0000 7eff84902700 1 -- 172.21.15.44:0/336211637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff801067d0 con 0x7eff800ff4f0 2023-09-10T00:40:14.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.741+0000 7eff777fe700 1 -- 172.21.15.44:0/336211637 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7eff70003d90 con 0x7eff800ff4f0 2023-09-10T00:40:14.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.742+0000 7eff777fe700 1 --2- 172.21.15.44:0/336211637 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7eff6c0a06d0 0x7eff6c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:14.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.742+0000 7eff777fe700 1 -- 172.21.15.44:0/336211637 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7eff70014070 con 0x7eff800ff4f0 2023-09-10T00:40:14.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.742+0000 7eff7dd9b700 1 --2- 172.21.15.44:0/336211637 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7eff6c0a06d0 0x7eff6c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:14.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.743+0000 7eff7dd9b700 1 --2- 172.21.15.44:0/336211637 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7eff6c0a06d0 0x7eff6c0a2b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7eff68009990 tx=0x7eff68005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:14.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.745+0000 7eff777fe700 1 -- 172.21.15.44:0/336211637 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7eff70095ac0 con 0x7eff800ff4f0 2023-09-10T00:40:14.950 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.950+0000 7eff84902700 1 -- 172.21.15.44:0/336211637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff8010ad50 con 0x7eff800ff4f0 2023-09-10T00:40:14.951 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.951+0000 7eff777fe700 1 -- 172.21.15.44:0/336211637 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff700990e0 con 0x7eff800ff4f0 2023-09-10T00:40:14.951 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:40:14.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.953+0000 7eff84902700 1 -- 172.21.15.44:0/336211637 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7eff6c0a06d0 msgr2=0x7eff6c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:14.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.953+0000 7eff84902700 1 --2- 172.21.15.44:0/336211637 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7eff6c0a06d0 0x7eff6c0a2b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7eff68009990 tx=0x7eff68005be0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.953+0000 7eff84902700 1 -- 172.21.15.44:0/336211637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff800ff4f0 msgr2=0x7eff800682e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:14.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.953+0000 7eff84902700 1 --2- 172.21.15.44:0/336211637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff800ff4f0 0x7eff800682e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7eff7000e970 tx=0x7eff7000ec80 comp rx=0 tx=0).stop 2023-09-10T00:40:14.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.953+0000 7eff84902700 1 -- 172.21.15.44:0/336211637 shutdown_connections 2023-09-10T00:40:14.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.953+0000 7eff84902700 1 --2- 172.21.15.44:0/336211637 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7eff6c0a06d0 0x7eff6c0a2b90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.953+0000 7eff84902700 1 --2- 172.21.15.44:0/336211637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff80101080 0x7eff8019d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.953+0000 7eff84902700 1 --2- 172.21.15.44:0/336211637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff800ff4f0 0x7eff800682e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.953+0000 7eff84902700 1 --2- 172.21.15.44:0/336211637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff801006e0 0x7eff8019af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:14.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.953+0000 7eff84902700 1 -- 172.21.15.44:0/336211637 >> 172.21.15.44:0/336211637 conn(0x7eff800faa60 msgr2=0x7eff80103910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:14.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.953+0000 7eff84902700 1 -- 172.21.15.44:0/336211637 shutdown_connections 2023-09-10T00:40:14.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:14.953+0000 7eff84902700 1 -- 172.21.15.44:0/336211637 wait complete. 2023-09-10T00:40:14.964 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:14 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/4003554115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:40:14.965 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:14 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1999601268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:14.965 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:14 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1873298365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:40:14.966 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-10T00:40:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:14 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/4003554115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:40:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:14 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1999601268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:14 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1873298365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:40:15.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.095+0000 7f2bc9a59700 1 -- 172.21.15.44:0/4073586148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bc41006e0 msgr2=0x7f2bc4100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:15.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.095+0000 7f2bc9a59700 1 --2- 172.21.15.44:0/4073586148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bc41006e0 0x7f2bc4100b40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40099c0 tx=0x7f2bb4009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.095+0000 7f2bc9a59700 1 -- 172.21.15.44:0/4073586148 shutdown_connections 2023-09-10T00:40:15.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.095+0000 7f2bc9a59700 1 --2- 172.21.15.44:0/4073586148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc4101080 0x7f2bc4107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.095+0000 7f2bc9a59700 1 --2- 172.21.15.44:0/4073586148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bc40ff4f0 0x7f2bc40ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.095+0000 7f2bc9a59700 1 --2- 172.21.15.44:0/4073586148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bc41006e0 0x7f2bc4100b40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.095+0000 7f2bc9a59700 1 -- 172.21.15.44:0/4073586148 >> 172.21.15.44:0/4073586148 conn(0x7f2bc40faa60 msgr2=0x7f2bc40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:15.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.095+0000 7f2bc9a59700 1 -- 172.21.15.44:0/4073586148 shutdown_connections 2023-09-10T00:40:15.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.095+0000 7f2bc9a59700 1 -- 172.21.15.44:0/4073586148 wait complete. 2023-09-10T00:40:15.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.096+0000 7f2bc9a59700 1 Processor -- start 2023-09-10T00:40:15.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.096+0000 7f2bc9a59700 1 -- start start 2023-09-10T00:40:15.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.096+0000 7f2bc9a59700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bc40ff4f0 0x7f2bc4196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:15.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.096+0000 7f2bc9a59700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bc41006e0 0x7f2bc41967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:15.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.096+0000 7f2bc9a59700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc4101080 0x7f2bc419d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:15.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.096+0000 7f2bc9a59700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc419de00 con 0x7f2bc40ff4f0 2023-09-10T00:40:15.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.096+0000 7f2bc9a59700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc419df40 con 0x7f2bc41006e0 2023-09-10T00:40:15.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.096+0000 7f2bc9a59700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc419e0d0 con 0x7f2bc4101080 2023-09-10T00:40:15.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.096+0000 7f2bc2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bc40ff4f0 0x7f2bc4196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:15.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.096+0000 7f2bc2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bc40ff4f0 0x7f2bc4196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47946/0 (socket says 172.21.15.44:47946) 2023-09-10T00:40:15.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.096+0000 7f2bc2ffd700 1 -- 172.21.15.44:0/1726408782 learned_addr learned my addr 172.21.15.44:0/1726408782 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.096+0000 7f2bc37fe700 1 --2- 172.21.15.44:0/1726408782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc4101080 0x7f2bc419d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.096+0000 7f2bc27fc700 1 --2- 172.21.15.44:0/1726408782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bc41006e0 0x7f2bc41967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.097+0000 7f2bc2ffd700 1 -- 172.21.15.44:0/1726408782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc4101080 msgr2=0x7f2bc419d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.097+0000 7f2bc2ffd700 1 --2- 172.21.15.44:0/1726408782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc4101080 0x7f2bc419d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.097+0000 7f2bc2ffd700 1 -- 172.21.15.44:0/1726408782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bc41006e0 msgr2=0x7f2bc41967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.097+0000 7f2bc2ffd700 1 --2- 172.21.15.44:0/1726408782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bc41006e0 0x7f2bc41967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.097+0000 7f2bc2ffd700 1 -- 172.21.15.44:0/1726408782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb40095f0 con 0x7f2bc40ff4f0 2023-09-10T00:40:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.097+0000 7f2bc2ffd700 1 --2- 172.21.15.44:0/1726408782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bc40ff4f0 0x7f2bc4196280 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800d980 tx=0x7f2bb800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.097+0000 7f2bc8a57700 1 -- 172.21.15.44:0/1726408782 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8004020 con 0x7f2bc40ff4f0 2023-09-10T00:40:15.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.097+0000 7f2bc8a57700 1 -- 172.21.15.44:0/1726408782 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2bb8009730 con 0x7f2bc40ff4f0 2023-09-10T00:40:15.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.097+0000 7f2bc9a59700 1 -- 172.21.15.44:0/1726408782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc419e3b0 con 0x7f2bc40ff4f0 2023-09-10T00:40:15.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.097+0000 7f2bc8a57700 1 -- 172.21.15.44:0/1726408782 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8003a00 con 0x7f2bc40ff4f0 2023-09-10T00:40:15.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.097+0000 7f2bc9a59700 1 -- 172.21.15.44:0/1726408782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc419e8c0 con 0x7f2bc40ff4f0 2023-09-10T00:40:15.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.098+0000 7f2bc9a59700 1 -- 172.21.15.44:0/1726408782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc410ac10 con 0x7f2bc40ff4f0 2023-09-10T00:40:15.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.099+0000 7f2bc8a57700 1 -- 172.21.15.44:0/1726408782 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f2bb80098a0 con 0x7f2bc40ff4f0 2023-09-10T00:40:15.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.100+0000 7f2bc8a57700 1 --2- 172.21.15.44:0/1726408782 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2bac0a06d0 0x7f2bac0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:15.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.100+0000 7f2bc27fc700 1 --2- 172.21.15.44:0/1726408782 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2bac0a06d0 0x7f2bac0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:15.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.100+0000 7f2bc8a57700 1 -- 172.21.15.44:0/1726408782 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2bb80cbd00 con 0x7f2bc40ff4f0 2023-09-10T00:40:15.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.100+0000 7f2bc27fc700 1 --2- 172.21.15.44:0/1726408782 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2bac0a06d0 0x7f2bac0a2b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400c010 tx=0x7f2bb4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:15.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:14 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/4003554115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:40:15.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:14 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1999601268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:15.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:14 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1873298365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:40:15.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.104+0000 7f2bc8a57700 1 -- 172.21.15.44:0/1726408782 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2bb80953c0 con 0x7f2bc40ff4f0 2023-09-10T00:40:15.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.306+0000 7f2bc9a59700 1 -- 172.21.15.44:0/1726408782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2bc419eb40 con 0x7f2bc40ff4f0 2023-09-10T00:40:15.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.307+0000 7f2bc8a57700 1 -- 172.21.15.44:0/1726408782 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2bb80989e0 con 0x7f2bc40ff4f0 2023-09-10T00:40:15.308 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2023-09-10T00:40:15.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.309+0000 7f2bc9a59700 1 -- 172.21.15.44:0/1726408782 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2bac0a06d0 msgr2=0x7f2bac0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:15.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.309+0000 7f2bc9a59700 1 --2- 172.21.15.44:0/1726408782 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2bac0a06d0 0x7f2bac0a2b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400c010 tx=0x7f2bb4005fb0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.309+0000 7f2bc9a59700 1 -- 172.21.15.44:0/1726408782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bc40ff4f0 msgr2=0x7f2bc4196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.309+0000 7f2bc9a59700 1 --2- 172.21.15.44:0/1726408782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bc40ff4f0 0x7f2bc4196280 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800d980 tx=0x7f2bb800dc90 comp rx=0 tx=0).stop 2023-09-10T00:40:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.309+0000 7f2bc9a59700 1 -- 172.21.15.44:0/1726408782 shutdown_connections 2023-09-10T00:40:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.309+0000 7f2bc9a59700 1 --2- 172.21.15.44:0/1726408782 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2bac0a06d0 0x7f2bac0a2b90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.309+0000 7f2bc9a59700 1 --2- 172.21.15.44:0/1726408782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc4101080 0x7f2bc419d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.309+0000 7f2bc9a59700 1 --2- 172.21.15.44:0/1726408782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bc40ff4f0 0x7f2bc4196280 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.309+0000 7f2bc9a59700 1 --2- 172.21.15.44:0/1726408782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bc41006e0 0x7f2bc41967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.309+0000 7f2bc9a59700 1 -- 172.21.15.44:0/1726408782 >> 172.21.15.44:0/1726408782 conn(0x7f2bc40faa60 msgr2=0x7f2bc40fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:15.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.309+0000 7f2bc9a59700 1 -- 172.21.15.44:0/1726408782 shutdown_connections 2023-09-10T00:40:15.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.309+0000 7f2bc9a59700 1 -- 172.21.15.44:0/1726408782 wait complete. 2023-09-10T00:40:15.321 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-10T00:40:15.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.452+0000 7fd3c8cd8700 1 -- 172.21.15.44:0/4070973339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c4101820 msgr2=0x7fd3c4101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:15.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.452+0000 7fd3c8cd8700 1 --2- 172.21.15.44:0/4070973339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c4101820 0x7fd3c4101c80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0099c0 tx=0x7fd3ac009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.453+0000 7fd3c8cd8700 1 -- 172.21.15.44:0/4070973339 shutdown_connections 2023-09-10T00:40:15.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.453+0000 7fd3c8cd8700 1 --2- 172.21.15.44:0/4070973339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3c41021c0 0x7fd3c4108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.453+0000 7fd3c8cd8700 1 --2- 172.21.15.44:0/4070973339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3c4100630 0x7fd3c4100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.453+0000 7fd3c8cd8700 1 --2- 172.21.15.44:0/4070973339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c4101820 0x7fd3c4101c80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.453+0000 7fd3c8cd8700 1 -- 172.21.15.44:0/4070973339 >> 172.21.15.44:0/4070973339 conn(0x7fd3c40fbb80 msgr2=0x7fd3c40fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:15.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.453+0000 7fd3c8cd8700 1 -- 172.21.15.44:0/4070973339 shutdown_connections 2023-09-10T00:40:15.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.453+0000 7fd3c8cd8700 1 -- 172.21.15.44:0/4070973339 wait complete. 2023-09-10T00:40:15.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.453+0000 7fd3c8cd8700 1 Processor -- start 2023-09-10T00:40:15.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.453+0000 7fd3c8cd8700 1 -- start start 2023-09-10T00:40:15.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c8cd8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3c4100630 0x7fd3c41961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:15.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c8cd8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c4101820 0x7fd3c4196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:15.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c8cd8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3c41021c0 0x7fd3c419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:15.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c8cd8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c419dd70 con 0x7fd3c4101820 2023-09-10T00:40:15.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c8cd8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c419deb0 con 0x7fd3c41021c0 2023-09-10T00:40:15.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c8cd8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c419e040 con 0x7fd3c4100630 2023-09-10T00:40:15.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c4101820 0x7fd3c4196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:15.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c4101820 0x7fd3c4196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47964/0 (socket says 172.21.15.44:47964) 2023-09-10T00:40:15.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c1d9b700 1 -- 172.21.15.44:0/2156969952 learned_addr learned my addr 172.21.15.44:0/2156969952 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:15.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c2d9d700 1 --2- 172.21.15.44:0/2156969952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3c41021c0 0x7fd3c419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:15.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c259c700 1 --2- 172.21.15.44:0/2156969952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3c4100630 0x7fd3c41961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:15.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c1d9b700 1 -- 172.21.15.44:0/2156969952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3c4100630 msgr2=0x7fd3c41961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:15.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c1d9b700 1 --2- 172.21.15.44:0/2156969952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3c4100630 0x7fd3c41961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c1d9b700 1 -- 172.21.15.44:0/2156969952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3c41021c0 msgr2=0x7fd3c419d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:15.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c1d9b700 1 --2- 172.21.15.44:0/2156969952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3c41021c0 0x7fd3c419d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c1d9b700 1 -- 172.21.15.44:0/2156969952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac0095f0 con 0x7fd3c4101820 2023-09-10T00:40:15.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3c1d9b700 1 --2- 172.21.15.44:0/2156969952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c4101820 0x7fd3c4196730 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac005910 tx=0x7fd3ac003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:15.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.454+0000 7fd3bb7fe700 1 -- 172.21.15.44:0/2156969952 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac01d070 con 0x7fd3c4101820 2023-09-10T00:40:15.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.455+0000 7fd3c8cd8700 1 -- 172.21.15.44:0/2156969952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c419e2c0 con 0x7fd3c4101820 2023-09-10T00:40:15.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.455+0000 7fd3bb7fe700 1 -- 172.21.15.44:0/2156969952 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd3ac00bd30 con 0x7fd3c4101820 2023-09-10T00:40:15.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.455+0000 7fd3bb7fe700 1 -- 172.21.15.44:0/2156969952 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac022ca0 con 0x7fd3c4101820 2023-09-10T00:40:15.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.455+0000 7fd3c8cd8700 1 -- 172.21.15.44:0/2156969952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c419e6f0 con 0x7fd3c4101820 2023-09-10T00:40:15.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.455+0000 7fd3c8cd8700 1 -- 172.21.15.44:0/2156969952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c410bcf0 con 0x7fd3c4101820 2023-09-10T00:40:15.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.457+0000 7fd3bb7fe700 1 -- 172.21.15.44:0/2156969952 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fd3ac022470 con 0x7fd3c4101820 2023-09-10T00:40:15.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.457+0000 7fd3bb7fe700 1 --2- 172.21.15.44:0/2156969952 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd3b00a06d0 0x7fd3b00a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:15.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.457+0000 7fd3bb7fe700 1 -- 172.21.15.44:0/2156969952 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd3ac0cdce0 con 0x7fd3c4101820 2023-09-10T00:40:15.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.457+0000 7fd3c259c700 1 --2- 172.21.15.44:0/2156969952 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd3b00a06d0 0x7fd3b00a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:15.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.458+0000 7fd3c259c700 1 --2- 172.21.15.44:0/2156969952 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd3b00a06d0 0x7fd3b00a2b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4101680 tx=0x7fd3b400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:15.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.461+0000 7fd3bb7fe700 1 -- 172.21.15.44:0/2156969952 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd3ac097430 con 0x7fd3c4101820 2023-09-10T00:40:15.662 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.662+0000 7fd3c8cd8700 1 -- 172.21.15.44:0/2156969952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd3c41075a0 con 0x7fd3c4101820 2023-09-10T00:40:15.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.662+0000 7fd3bb7fe700 1 -- 172.21.15.44:0/2156969952 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd3ac09aa50 con 0x7fd3c4101820 2023-09-10T00:40:15.663 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:40:15.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.665+0000 7fd3c8cd8700 1 -- 172.21.15.44:0/2156969952 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd3b00a06d0 msgr2=0x7fd3b00a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:15.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.665+0000 7fd3c8cd8700 1 --2- 172.21.15.44:0/2156969952 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd3b00a06d0 0x7fd3b00a2b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4101680 tx=0x7fd3b400a380 comp rx=0 tx=0).stop 2023-09-10T00:40:15.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.665+0000 7fd3c8cd8700 1 -- 172.21.15.44:0/2156969952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c4101820 msgr2=0x7fd3c4196730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:15.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.665+0000 7fd3c8cd8700 1 --2- 172.21.15.44:0/2156969952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c4101820 0x7fd3c4196730 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac005910 tx=0x7fd3ac003a50 comp rx=0 tx=0).stop 2023-09-10T00:40:15.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.665+0000 7fd3c8cd8700 1 -- 172.21.15.44:0/2156969952 shutdown_connections 2023-09-10T00:40:15.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.665+0000 7fd3c8cd8700 1 --2- 172.21.15.44:0/2156969952 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fd3b00a06d0 0x7fd3b00a2b90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.665+0000 7fd3c8cd8700 1 --2- 172.21.15.44:0/2156969952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3c41021c0 0x7fd3c419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.665+0000 7fd3c8cd8700 1 --2- 172.21.15.44:0/2156969952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3c4100630 0x7fd3c41961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.665+0000 7fd3c8cd8700 1 --2- 172.21.15.44:0/2156969952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c4101820 0x7fd3c4196730 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.665+0000 7fd3c8cd8700 1 -- 172.21.15.44:0/2156969952 >> 172.21.15.44:0/2156969952 conn(0x7fd3c40fbb80 msgr2=0x7fd3c40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:15.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.665+0000 7fd3c8cd8700 1 -- 172.21.15.44:0/2156969952 shutdown_connections 2023-09-10T00:40:15.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.665+0000 7fd3c8cd8700 1 -- 172.21.15.44:0/2156969952 wait complete. 2023-09-10T00:40:15.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:40:15.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.810+0000 7f0387066700 1 -- 172.21.15.44:0/1292454177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0380106320 msgr2=0x7f0380108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:15.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.810+0000 7f0387066700 1 --2- 172.21.15.44:0/1292454177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0380106320 0x7f0380108730 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0378009f90 tx=0x7f037800b330 comp rx=0 tx=0).stop 2023-09-10T00:40:15.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.810+0000 7f0387066700 1 -- 172.21.15.44:0/1292454177 shutdown_connections 2023-09-10T00:40:15.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.810+0000 7f0387066700 1 --2- 172.21.15.44:0/1292454177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0380106320 0x7f0380108730 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.810+0000 7f0387066700 1 --2- 172.21.15.44:0/1292454177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03801010c0 0x7f03801034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.810+0000 7f0387066700 1 --2- 172.21.15.44:0/1292454177 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03801039f0 0x7f0380105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.810+0000 7f0387066700 1 -- 172.21.15.44:0/1292454177 >> 172.21.15.44:0/1292454177 conn(0x7f03800faa30 msgr2=0x7f03800fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:15.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.810+0000 7f0387066700 1 -- 172.21.15.44:0/1292454177 shutdown_connections 2023-09-10T00:40:15.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.810+0000 7f0387066700 1 -- 172.21.15.44:0/1292454177 wait complete. 2023-09-10T00:40:15.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.810+0000 7f0387066700 1 Processor -- start 2023-09-10T00:40:15.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0387066700 1 -- start start 2023-09-10T00:40:15.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0387066700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03801010c0 0x7f0380100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:15.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0387066700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03801039f0 0x7f03800ff070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:15.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0387066700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0380106320 0x7f03800ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:15.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0387066700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03800ffd50 con 0x7f0380106320 2023-09-10T00:40:15.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0387066700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03800ffec0 con 0x7f03801010c0 2023-09-10T00:40:15.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0387066700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03801a0310 con 0x7f03801039f0 2023-09-10T00:40:15.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0385603700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0380106320 0x7f03800ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:15.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0385603700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0380106320 0x7f03800ff5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47978/0 (socket says 172.21.15.44:47978) 2023-09-10T00:40:15.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0384e02700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03801010c0 0x7f0380100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:15.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f037ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03801039f0 0x7f03800ff070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:15.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0385603700 1 -- 172.21.15.44:0/2841530742 learned_addr learned my addr 172.21.15.44:0/2841530742 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:15.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0385603700 1 -- 172.21.15.44:0/2841530742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03801039f0 msgr2=0x7f03800ff070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:15.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0385603700 1 --2- 172.21.15.44:0/2841530742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03801039f0 0x7f03800ff070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0385603700 1 -- 172.21.15.44:0/2841530742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03801010c0 msgr2=0x7f0380100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:15.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0385603700 1 --2- 172.21.15.44:0/2841530742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03801010c0 0x7f0380100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:15.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0385603700 1 -- 172.21.15.44:0/2841530742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c0095f0 con 0x7f0380106320 2023-09-10T00:40:15.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.811+0000 7f0385603700 1 --2- 172.21.15.44:0/2841530742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0380106320 0x7f03800ff5b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0378009f60 tx=0x7f0378008d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:15.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.812+0000 7f037dffb700 1 -- 172.21.15.44:0/2841530742 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037800e040 con 0x7f0380106320 2023-09-10T00:40:15.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.812+0000 7f037dffb700 1 -- 172.21.15.44:0/2841530742 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f037800bc00 con 0x7f0380106320 2023-09-10T00:40:15.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.812+0000 7f0387066700 1 -- 172.21.15.44:0/2841530742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0378009c40 con 0x7f0380106320 2023-09-10T00:40:15.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.812+0000 7f037dffb700 1 -- 172.21.15.44:0/2841530742 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03780052a0 con 0x7f0380106320 2023-09-10T00:40:15.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.812+0000 7f0387066700 1 -- 172.21.15.44:0/2841530742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03801a0880 con 0x7f0380106320 2023-09-10T00:40:15.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.813+0000 7f0387066700 1 -- 172.21.15.44:0/2841530742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03800fc5b0 con 0x7f0380106320 2023-09-10T00:40:15.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.814+0000 7f037dffb700 1 -- 172.21.15.44:0/2841530742 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f037801d470 con 0x7f0380106320 2023-09-10T00:40:15.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.814+0000 7f037dffb700 1 --2- 172.21.15.44:0/2841530742 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f03700a0790 0x7f03700a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:15.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.815+0000 7f037dffb700 1 -- 172.21.15.44:0/2841530742 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f03780cdbf0 con 0x7f0380106320 2023-09-10T00:40:15.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.815+0000 7f0384e02700 1 --2- 172.21.15.44:0/2841530742 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f03700a0790 0x7f03700a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:15.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.815+0000 7f0384e02700 1 --2- 172.21.15.44:0/2841530742 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f03700a0790 0x7f03700a2c50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0374009c60 tx=0x7f0374009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:15.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:15.818+0000 7f037dffb700 1 -- 172.21.15.44:0/2841530742 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0378097460 con 0x7f0380106320 2023-09-10T00:40:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:15 smithi131 ceph-mon[103713]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:40:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:15 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/336211637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:40:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:15 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1726408782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:40:16.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:15 smithi044 ceph-mon[98676]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:40:16.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:15 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/336211637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:40:16.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:15 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1726408782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:40:16.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.097+0000 7f0387066700 1 -- 172.21.15.44:0/2841530742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f038010bcf0 con 0x7f0380106320 2023-09-10T00:40:16.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.098+0000 7f037dffb700 1 -- 172.21.15.44:0/2841530742 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11369 (secure 0 0 0) 0x7f0378021090 con 0x7f0380106320 2023-09-10T00:40:16.098 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:16.099 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-10T00:38:39.183533+0000","modified":"2023-09-10T00:39:02.480227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24605,"mds_1":24574,"mds_2":24572,"mds_3":24515,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14751":{"gid":14751,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/3958270510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":3958270510},{"type":"v1","addr":"172.21.15.44:6839","nonce":3958270510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14763":{"gid":14763,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/1241002552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1241002552},{"type":"v1","addr":"172.21.15.44:6841","nonce":1241002552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24556":{"gid":24556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/3222652693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":3222652693},{"type":"v1","addr":"172.21.15.131:6839","nonce":3222652693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24605":{"gid":24605,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/2622966260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2622966260},{"type":"v1","addr":"172.21.15.99:6839","nonce":2622966260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-10T00:40:16.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.100+0000 7f0387066700 1 -- 172.21.15.44:0/2841530742 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f03700a0790 msgr2=0x7f03700a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.100+0000 7f0387066700 1 --2- 172.21.15.44:0/2841530742 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f03700a0790 0x7f03700a2c50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0374009c60 tx=0x7f0374009260 comp rx=0 tx=0).stop 2023-09-10T00:40:16.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.100+0000 7f0387066700 1 -- 172.21.15.44:0/2841530742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0380106320 msgr2=0x7f03800ff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.100+0000 7f0387066700 1 --2- 172.21.15.44:0/2841530742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0380106320 0x7f03800ff5b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0378009f60 tx=0x7f0378008d00 comp rx=0 tx=0).stop 2023-09-10T00:40:16.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.100+0000 7f0387066700 1 -- 172.21.15.44:0/2841530742 shutdown_connections 2023-09-10T00:40:16.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.100+0000 7f0387066700 1 --2- 172.21.15.44:0/2841530742 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f03700a0790 0x7f03700a2c50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.100+0000 7f0387066700 1 --2- 172.21.15.44:0/2841530742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0380106320 0x7f03800ff5b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.100+0000 7f0387066700 1 --2- 172.21.15.44:0/2841530742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03801010c0 0x7f0380100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.100+0000 7f0387066700 1 --2- 172.21.15.44:0/2841530742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03801039f0 0x7f03800ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.100+0000 7f0387066700 1 -- 172.21.15.44:0/2841530742 >> 172.21.15.44:0/2841530742 conn(0x7f03800faa30 msgr2=0x7f0380101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:16.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.100+0000 7f0387066700 1 -- 172.21.15.44:0/2841530742 shutdown_connections 2023-09-10T00:40:16.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.100+0000 7f0387066700 1 -- 172.21.15.44:0/2841530742 wait complete. 2023-09-10T00:40:16.103 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2023-09-10T00:40:16.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:15 smithi099 ceph-mon[102692]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T00:40:16.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:15 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/336211637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:40:16.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:15 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1726408782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:40:16.113 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:40:16.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.239+0000 7feaf0e24700 1 -- 172.21.15.44:0/4172647943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaec101070 msgr2=0x7feaec107920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.239+0000 7feaf0e24700 1 --2- 172.21.15.44:0/4172647943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaec101070 0x7feaec107920 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7feae400b0d0 tx=0x7feae400b3e0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.239+0000 7feaf0e24700 1 -- 172.21.15.44:0/4172647943 shutdown_connections 2023-09-10T00:40:16.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.239+0000 7feaf0e24700 1 --2- 172.21.15.44:0/4172647943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaec101070 0x7feaec107920 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.239+0000 7feaf0e24700 1 --2- 172.21.15.44:0/4172647943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaec0ff4e0 0x7feaec0ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.239+0000 7feaf0e24700 1 --2- 172.21.15.44:0/4172647943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec1006d0 0x7feaec100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.239+0000 7feaf0e24700 1 -- 172.21.15.44:0/4172647943 >> 172.21.15.44:0/4172647943 conn(0x7feaec0faa30 msgr2=0x7feaec0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:16.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.239+0000 7feaf0e24700 1 -- 172.21.15.44:0/4172647943 shutdown_connections 2023-09-10T00:40:16.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.239+0000 7feaf0e24700 1 -- 172.21.15.44:0/4172647943 wait complete. 2023-09-10T00:40:16.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.240+0000 7feaf0e24700 1 Processor -- start 2023-09-10T00:40:16.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.240+0000 7feaf0e24700 1 -- start start 2023-09-10T00:40:16.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.240+0000 7feaf0e24700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaec0ff4e0 0x7feaec196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.240+0000 7feaf0e24700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec1006d0 0x7feaec1967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.240+0000 7feaf0e24700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaec101070 0x7feaec19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.240+0000 7feaf0e24700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec19de00 con 0x7feaec0ff4e0 2023-09-10T00:40:16.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.240+0000 7feaf0e24700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec19df40 con 0x7feaec101070 2023-09-10T00:40:16.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.240+0000 7feaf0e24700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec19e0d0 con 0x7feaec1006d0 2023-09-10T00:40:16.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.240+0000 7feaea59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaec0ff4e0 0x7feaec196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.240+0000 7feaea59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaec0ff4e0 0x7feaec196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47986/0 (socket says 172.21.15.44:47986) 2023-09-10T00:40:16.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.240+0000 7feaea59c700 1 -- 172.21.15.44:0/616912973 learned_addr learned my addr 172.21.15.44:0/616912973 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:16.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.240+0000 7feaead9d700 1 --2- 172.21.15.44:0/616912973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaec101070 0x7feaec19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.241+0000 7feae9d9b700 1 --2- 172.21.15.44:0/616912973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec1006d0 0x7feaec1967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.241+0000 7feaea59c700 1 -- 172.21.15.44:0/616912973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec1006d0 msgr2=0x7feaec1967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.241+0000 7feaea59c700 1 --2- 172.21.15.44:0/616912973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec1006d0 0x7feaec1967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.241+0000 7feaea59c700 1 -- 172.21.15.44:0/616912973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaec101070 msgr2=0x7feaec19d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.241+0000 7feaea59c700 1 --2- 172.21.15.44:0/616912973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaec101070 0x7feaec19d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.241+0000 7feaea59c700 1 -- 172.21.15.44:0/616912973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae4009d00 con 0x7feaec0ff4e0 2023-09-10T00:40:16.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.241+0000 7feaea59c700 1 --2- 172.21.15.44:0/616912973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaec0ff4e0 0x7feaec196280 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7feadc00d980 tx=0x7feadc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:16.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.241+0000 7feae37fe700 1 -- 172.21.15.44:0/616912973 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc004020 con 0x7feaec0ff4e0 2023-09-10T00:40:16.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.241+0000 7feaf0e24700 1 -- 172.21.15.44:0/616912973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaec19e3b0 con 0x7feaec0ff4e0 2023-09-10T00:40:16.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.241+0000 7feae37fe700 1 -- 172.21.15.44:0/616912973 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feadc00f690 con 0x7feaec0ff4e0 2023-09-10T00:40:16.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.241+0000 7feae37fe700 1 -- 172.21.15.44:0/616912973 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc0039e0 con 0x7feaec0ff4e0 2023-09-10T00:40:16.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.241+0000 7feaf0e24700 1 -- 172.21.15.44:0/616912973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaec19e8c0 con 0x7feaec0ff4e0 2023-09-10T00:40:16.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.242+0000 7feaf0e24700 1 -- 172.21.15.44:0/616912973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaec107560 con 0x7feaec0ff4e0 2023-09-10T00:40:16.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.244+0000 7feae37fe700 1 -- 172.21.15.44:0/616912973 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7feadc00f800 con 0x7feaec0ff4e0 2023-09-10T00:40:16.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.244+0000 7feae37fe700 1 --2- 172.21.15.44:0/616912973 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fead80a0710 0x7fead80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.245+0000 7feae9d9b700 1 --2- 172.21.15.44:0/616912973 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fead80a0710 0x7fead80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.245+0000 7feae37fe700 1 -- 172.21.15.44:0/616912973 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feadc0cbf40 con 0x7feaec0ff4e0 2023-09-10T00:40:16.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.245+0000 7feae9d9b700 1 --2- 172.21.15.44:0/616912973 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fead80a0710 0x7fead80a2bd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fead4007be0 tx=0x7fead40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:16.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.247+0000 7feae37fe700 1 -- 172.21.15.44:0/616912973 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7feadc095600 con 0x7feaec0ff4e0 2023-09-10T00:40:16.451 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.451+0000 7feaf0e24700 1 -- 172.21.15.44:0/616912973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feaec10aba0 con 0x7feaec0ff4e0 2023-09-10T00:40:16.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.452+0000 7feae37fe700 1 -- 172.21.15.44:0/616912973 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24340 (secure 0 0 0) 0x7feadc098c20 con 0x7feaec0ff4e0 2023-09-10T00:40:16.453 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:16.453 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":89,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:39:17.515738+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:40:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.454+0000 7feaf0e24700 1 -- 172.21.15.44:0/616912973 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fead80a0710 msgr2=0x7fead80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.454+0000 7feaf0e24700 1 --2- 172.21.15.44:0/616912973 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fead80a0710 0x7fead80a2bd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fead4007be0 tx=0x7fead40073d0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.454+0000 7feaf0e24700 1 -- 172.21.15.44:0/616912973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaec0ff4e0 msgr2=0x7feaec196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.454+0000 7feaf0e24700 1 --2- 172.21.15.44:0/616912973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaec0ff4e0 0x7feaec196280 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7feadc00d980 tx=0x7feadc00dc90 comp rx=0 tx=0).stop 2023-09-10T00:40:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.454+0000 7feaf0e24700 1 -- 172.21.15.44:0/616912973 shutdown_connections 2023-09-10T00:40:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.454+0000 7feaf0e24700 1 --2- 172.21.15.44:0/616912973 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fead80a0710 0x7fead80a2bd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.454+0000 7feaf0e24700 1 --2- 172.21.15.44:0/616912973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaec101070 0x7feaec19d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.454+0000 7feaf0e24700 1 --2- 172.21.15.44:0/616912973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaec0ff4e0 0x7feaec196280 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.454+0000 7feaf0e24700 1 --2- 172.21.15.44:0/616912973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec1006d0 0x7feaec1967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.454+0000 7feaf0e24700 1 -- 172.21.15.44:0/616912973 >> 172.21.15.44:0/616912973 conn(0x7feaec0faa30 msgr2=0x7feaec0fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:16.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.455+0000 7feaf0e24700 1 -- 172.21.15.44:0/616912973 shutdown_connections 2023-09-10T00:40:16.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.455+0000 7feaf0e24700 1 -- 172.21.15.44:0/616912973 wait complete. 2023-09-10T00:40:16.468 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-10T00:38:39.183533+0000', 'modified': '2023-09-10T00:39:02.480227+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24605, 'mds_1': 24574, 'mds_2': 24572, 'mds_3': 24515, 'mds_4': 24502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14751': {'gid': 14751, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6839/3958270510', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6838', 'nonce': 3958270510}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 3958270510}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14763': {'gid': 14763, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6841/1241002552', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6840', 'nonce': 1241002552}, {'type': 'v1', 'addr': '172.21.15.44:6841', 'nonce': 1241002552}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24502': {'gid': 24502, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.131:6833/3395466517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 3395466517}, {'type': 'v1', 'addr': '172.21.15.131:6833', 'nonce': 3395466517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24515': {'gid': 24515, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.99:6833/1934109590', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 1934109590}, {'type': 'v1', 'addr': '172.21.15.99:6833', 'nonce': 1934109590}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24539': {'gid': 24539, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6835/2403105238', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 2403105238}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 2403105238}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24547': {'gid': 24547, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/1445095724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 1445095724}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1445095724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24556': {'gid': 24556, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6837/2397154745', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6836', 'nonce': 2397154745}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 2397154745}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24572': {'gid': 24572, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.99:6837/566283638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 566283638}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 566283638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24574': {'gid': 24574, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.131:6839/3222652693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 3222652693}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 3222652693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24605': {'gid': 24605, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6839/2622966260', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 2622966260}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 2622966260}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-10T00:40:16.468 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-09-10T00:40:16.469 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2023-09-10T00:40:16.600 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.598+0000 7f06355dd700 1 -- 172.21.15.44:0/2087651825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06300ff740 msgr2=0x7f06300ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.601 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.598+0000 7f06355dd700 1 --2- 172.21.15.44:0/2087651825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06300ff740 0x7f06300ffbc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0618009990 tx=0x7f0618009ca0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.601 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.598+0000 7f06355dd700 1 -- 172.21.15.44:0/2087651825 shutdown_connections 2023-09-10T00:40:16.601 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.598+0000 7f06355dd700 1 --2- 172.21.15.44:0/2087651825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0630100100 0x7f06301079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.601 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.598+0000 7f06355dd700 1 --2- 172.21.15.44:0/2087651825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0630100af0 0x7f0630100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.601 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.598+0000 7f06355dd700 1 --2- 172.21.15.44:0/2087651825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06300ff740 0x7f06300ffbc0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.598+0000 7f06355dd700 1 -- 172.21.15.44:0/2087651825 >> 172.21.15.44:0/2087651825 conn(0x7f06300fa1d0 msgr2=0x7f06300fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:16.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.598+0000 7f06355dd700 1 -- 172.21.15.44:0/2087651825 shutdown_connections 2023-09-10T00:40:16.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.598+0000 7f06355dd700 1 -- 172.21.15.44:0/2087651825 wait complete. 2023-09-10T00:40:16.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.598+0000 7f06355dd700 1 Processor -- start 2023-09-10T00:40:16.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f06355dd700 1 -- start start 2023-09-10T00:40:16.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f06355dd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06300ff740 0x7f06300ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f06355dd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0630100100 0x7f063019af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f06355dd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0630100af0 0x7f063019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f06355dd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063019dd50 con 0x7f0630100100 2023-09-10T00:40:16.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f06355dd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063019dec0 con 0x7f0630100af0 2023-09-10T00:40:16.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f06355dd700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063019e080 con 0x7f06300ff740 2023-09-10T00:40:16.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f062e7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0630100100 0x7f063019af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f062e7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0630100100 0x7f063019af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48002/0 (socket says 172.21.15.44:48002) 2023-09-10T00:40:16.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f062e7fc700 1 -- 172.21.15.44:0/3373561244 learned_addr learned my addr 172.21.15.44:0/3373561244 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:16.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f062f7fe700 1 --2- 172.21.15.44:0/3373561244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0630100af0 0x7f063019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f062effd700 1 --2- 172.21.15.44:0/3373561244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06300ff740 0x7f06300ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f062e7fc700 1 -- 172.21.15.44:0/3373561244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06300ff740 msgr2=0x7f06300ff1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f062e7fc700 1 --2- 172.21.15.44:0/3373561244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06300ff740 0x7f06300ff1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f062e7fc700 1 -- 172.21.15.44:0/3373561244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0630100af0 msgr2=0x7f063019d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f062e7fc700 1 --2- 172.21.15.44:0/3373561244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0630100af0 0x7f063019d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f062e7fc700 1 -- 172.21.15.44:0/3373561244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06180095f0 con 0x7f0630100100 2023-09-10T00:40:16.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.599+0000 7f062effd700 1 --2- 172.21.15.44:0/3373561244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06300ff740 0x7f06300ff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:16.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.600+0000 7f062e7fc700 1 --2- 172.21.15.44:0/3373561244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0630100100 0x7f063019af30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0618006010 tx=0x7f0618003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:16.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.600+0000 7f0627fff700 1 -- 172.21.15.44:0/3373561244 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061801d070 con 0x7f0630100100 2023-09-10T00:40:16.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.600+0000 7f0627fff700 1 -- 172.21.15.44:0/3373561244 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0618004640 con 0x7f0630100100 2023-09-10T00:40:16.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.600+0000 7f06355dd700 1 -- 172.21.15.44:0/3373561244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063019e300 con 0x7f0630100100 2023-09-10T00:40:16.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.600+0000 7f0627fff700 1 -- 172.21.15.44:0/3373561244 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061800f460 con 0x7f0630100100 2023-09-10T00:40:16.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.600+0000 7f06355dd700 1 -- 172.21.15.44:0/3373561244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063019e7a0 con 0x7f0630100100 2023-09-10T00:40:16.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.602+0000 7f0627fff700 1 -- 172.21.15.44:0/3373561244 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f0618022470 con 0x7f0630100100 2023-09-10T00:40:16.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.602+0000 7f0627fff700 1 --2- 172.21.15.44:0/3373561244 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f061c0a06d0 0x7f061c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.603+0000 7f0627fff700 1 -- 172.21.15.44:0/3373561244 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f06180cd600 con 0x7f0630100100 2023-09-10T00:40:16.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.603+0000 7f062effd700 1 --2- 172.21.15.44:0/3373561244 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f061c0a06d0 0x7f061c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.603+0000 7f062effd700 1 --2- 172.21.15.44:0/3373561244 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f061c0a06d0 0x7f061c0a2b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0630067540 tx=0x7f062000a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:16.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.603+0000 7f06417a3b80 1 Processor -- start 2023-09-10T00:40:16.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.603+0000 7f06417a3b80 1 -- start start 2023-09-10T00:40:16.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f06417a3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a464bb0 0x55b85a464fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f06417a3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a465550 0x55b85a4637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f06417a3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a463db0 0x55b85a464260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f06417a3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85a3bc6e0 con 0x55b85a464bb0 2023-09-10T00:40:16.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f06417a3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85a3ba000 con 0x55b85a463db0 2023-09-10T00:40:16.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f06417a3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85a3b58f0 con 0x55b85a465550 2023-09-10T00:40:16.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f062effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a464bb0 0x55b85a464fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f062effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a464bb0 0x55b85a464fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48008/0 (socket says 172.21.15.44:48008) 2023-09-10T00:40:16.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f062effd700 1 -- 172.21.15.44:0/1601361981 learned_addr learned my addr 172.21.15.44:0/1601361981 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:16.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f062f7fe700 1 --2- 172.21.15.44:0/1601361981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a463db0 0x55b85a464260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f062e7fc700 1 --2- 172.21.15.44:0/1601361981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a465550 0x55b85a4637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f062effd700 1 -- 172.21.15.44:0/1601361981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a465550 msgr2=0x55b85a4637b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:16.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f062effd700 1 --2- 172.21.15.44:0/1601361981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a465550 0x55b85a4637b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f062effd700 1 -- 172.21.15.44:0/1601361981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a463db0 msgr2=0x55b85a464260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:16.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f062effd700 1 --2- 172.21.15.44:0/1601361981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a463db0 0x55b85a464260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f062effd700 1 -- 172.21.15.44:0/1601361981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b85a1e1d90 con 0x55b85a464bb0 2023-09-10T00:40:16.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.604+0000 7f062effd700 1 --2- 172.21.15.44:0/1601361981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a464bb0 0x55b85a464fd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f062000ed90 tx=0x7f0620011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c140bcc3fd523360 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:16.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.605+0000 7f06257fa700 1 -- 172.21.15.44:0/1601361981 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0620010da0 con 0x55b85a464bb0 2023-09-10T00:40:16.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.605+0000 7f06257fa700 1 -- 172.21.15.44:0/1601361981 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f062001d8e0 con 0x55b85a464bb0 2023-09-10T00:40:16.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.605+0000 7f06257fa700 1 -- 172.21.15.44:0/1601361981 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062000f7a0 con 0x55b85a464bb0 2023-09-10T00:40:16.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.605+0000 7f06417a3b80 1 -- 172.21.15.44:0/1601361981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a464bb0 msgr2=0x55b85a464fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.605+0000 7f06417a3b80 1 --2- 172.21.15.44:0/1601361981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a464bb0 0x55b85a464fd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f062000ed90 tx=0x7f0620011000 comp rx=0 tx=0).stop 2023-09-10T00:40:16.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.605+0000 7f06417a3b80 1 -- 172.21.15.44:0/1601361981 shutdown_connections 2023-09-10T00:40:16.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.605+0000 7f06417a3b80 1 --2- 172.21.15.44:0/1601361981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a463db0 0x55b85a464260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.605+0000 7f06417a3b80 1 --2- 172.21.15.44:0/1601361981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a464bb0 0x55b85a464fd0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.605+0000 7f06417a3b80 1 --2- 172.21.15.44:0/1601361981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a465550 0x55b85a4637b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.605+0000 7f06417a3b80 1 -- 172.21.15.44:0/1601361981 >> 172.21.15.44:0/1601361981 conn(0x55b85a3d8040 msgr2=0x55b85a45e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:16.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.605+0000 7f06417a3b80 1 -- 172.21.15.44:0/1601361981 shutdown_connections 2023-09-10T00:40:16.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.605+0000 7f06417a3b80 1 -- 172.21.15.44:0/1601361981 wait complete. 2023-09-10T00:40:16.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f06417a3b80 1 -- start start 2023-09-10T00:40:16.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f06417a3b80 1 Processor -- start 2023-09-10T00:40:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f06417a3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a463db0 0x55b85a46d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f06417a3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a464bb0 0x55b85a46dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f06417a3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a465550 0x55b85a474af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f06417a3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85a353b20 con 0x55b85a463db0 2023-09-10T00:40:16.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f06417a3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85a353820 con 0x55b85a464bb0 2023-09-10T00:40:16.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f06417a3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85a2378d0 con 0x55b85a465550 2023-09-10T00:40:16.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f06417a3b80 10 client.0 _ll_register_callbacks cb 0x55b85a29da30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:40:16.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f062effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a463db0 0x55b85a46d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f062effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a463db0 0x55b85a46d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48014/0 (socket says 172.21.15.44:48014) 2023-09-10T00:40:16.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f062effd700 1 -- 172.21.15.44:0/2727952444 learned_addr learned my addr 172.21.15.44:0/2727952444 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:16.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f062e7fc700 1 --2- 172.21.15.44:0/2727952444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a464bb0 0x55b85a46dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f062f7fe700 1 --2- 172.21.15.44:0/2727952444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a465550 0x55b85a474af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f062effd700 1 -- 172.21.15.44:0/2727952444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a465550 msgr2=0x55b85a474af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:16.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f062effd700 1 --2- 172.21.15.44:0/2727952444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a465550 0x55b85a474af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f062effd700 1 -- 172.21.15.44:0/2727952444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a464bb0 msgr2=0x55b85a46dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:16.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f062effd700 1 --2- 172.21.15.44:0/2727952444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a464bb0 0x55b85a46dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.606+0000 7f062effd700 1 -- 172.21.15.44:0/2727952444 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062000ae70 con 0x55b85a463db0 2023-09-10T00:40:16.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.607+0000 7f062effd700 1 --2- 172.21.15.44:0/2727952444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a463db0 0x55b85a46d570 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f062000f010 tx=0x7f062001d6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5d29aa1341c297 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:16.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.607+0000 7f06167fc700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:40:16.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.607+0000 7f06167fc700 1 -- 172.21.15.44:0/2727952444 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062002c070 con 0x55b85a463db0 2023-09-10T00:40:16.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.607+0000 7f06167fc700 1 -- 172.21.15.44:0/2727952444 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f062001dc60 con 0x55b85a463db0 2023-09-10T00:40:16.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.607+0000 7f06417a3b80 1 -- 172.21.15.44:0/2727952444 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b85a30d4a0 con 0x55b85a463db0 2023-09-10T00:40:16.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.607+0000 7f06167fc700 1 -- 172.21.15.44:0/2727952444 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f062001de20 con 0x55b85a463db0 2023-09-10T00:40:16.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.607+0000 7f06417a3b80 10 client.15552 fetch_fsmap learned FSMap version 27 2023-09-10T00:40:16.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.607+0000 7f06417a3b80 1 -- 172.21.15.44:0/2727952444 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b85a1cac10 con 0x55b85a463db0 2023-09-10T00:40:16.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.607+0000 7f06167fc700 1 -- 172.21.15.44:0/2727952444 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b85a30d4a0 con 0x55b85a463db0 2023-09-10T00:40:16.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.607+0000 7f06167fc700 1 -- 172.21.15.44:0/2727952444 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0620003cc0 con 0x55b85a463db0 2023-09-10T00:40:16.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.607+0000 7f06417a3b80 10 client.15552 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:40:16.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.607+0000 7f06417a3b80 10 client.15552 resolve_mds: resolved name 'l' to daemon mds.l 2023-09-10T00:40:16.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.607+0000 7f06417a3b80 20 client.15552 populate_metadata read hostname 'smithi044' 2023-09-10T00:40:16.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.607+0000 7f06417a3b80 1 --2- 172.21.15.44:0/2727952444 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x55b85a478b90 0x55b85a477b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.607+0000 7f06417a3b80 4 client.15552 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:40:16.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.607+0000 7f06417a3b80 1 -- 172.21.15.44:0/2727952444 --> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b85a3ca850 con 0x55b85a478b90 2023-09-10T00:40:16.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.608+0000 7f062e7fc700 1 --2- 172.21.15.44:0/2727952444 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x55b85a478b90 0x55b85a477b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.608+0000 7f06167fc700 10 client.15552 ms_handle_connect on v2:172.21.15.131:6832/3395466517 2023-09-10T00:40:16.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.608+0000 7f062e7fc700 1 --2- 172.21.15.44:0/2727952444 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x55b85a478b90 0x55b85a477b90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=70072b7a0d53ba6c server_cookie=b98c073e4a31df15 in_seq=0 out_seq=0 2023-09-10T00:40:16.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.609+0000 7f06167fc700 1 -- 172.21.15.44:0/2727952444 <== mds.4 v2:172.21.15.131:6832/3395466517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b85a3ca850 con 0x55b85a478b90 2023-09-10T00:40:16.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.609+0000 7f06167fc700 10 client.15552 handle_command_reply: tid=0 2023-09-10T00:40:16.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.609+0000 7f06417a3b80 1 client.15552 shutdown 2023-09-10T00:40:16.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.609+0000 7f06417a3b80 1 -- 172.21.15.44:0/2727952444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a463db0 msgr2=0x55b85a46d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.609+0000 7f06417a3b80 1 --2- 172.21.15.44:0/2727952444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a463db0 0x55b85a46d570 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f062000f010 tx=0x7f062001d6d0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.609+0000 7f06417a3b80 1 -- 172.21.15.44:0/2727952444 shutdown_connections 2023-09-10T00:40:16.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.609+0000 7f06417a3b80 1 --2- 172.21.15.44:0/2727952444 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x55b85a478b90 0x55b85a477b90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.609+0000 7f06417a3b80 1 --2- 172.21.15.44:0/2727952444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a465550 0x55b85a474af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.629 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.609+0000 7f06167fc700 0 client.15552 ms_handle_reset on v2:172.21.15.131:6832/3395466517 2023-09-10T00:40:16.629 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.609+0000 7f06417a3b80 1 --2- 172.21.15.44:0/2727952444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a463db0 0x55b85a46d570 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.629 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.609+0000 7f06417a3b80 1 --2- 172.21.15.44:0/2727952444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a464bb0 0x55b85a46dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.629 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.609+0000 7f06417a3b80 1 -- 172.21.15.44:0/2727952444 >> 172.21.15.44:0/2727952444 conn(0x55b85a3d8040 msgr2=0x55b85a45dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:16.629 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.609+0000 7f06417a3b80 1 -- 172.21.15.44:0/2727952444 shutdown_connections 2023-09-10T00:40:16.630 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.609+0000 7f06417a3b80 1 -- 172.21.15.44:0/2727952444 wait complete. 2023-09-10T00:40:16.630 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.609+0000 7f06417a3b80 20 client.15552 trim_cache size 0 max 16384 2023-09-10T00:40:16.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.627+0000 7f06417a3b80 1 Processor -- start 2023-09-10T00:40:16.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.627+0000 7f06417a3b80 1 -- start start 2023-09-10T00:40:16.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.627+0000 7f06417a3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a463db0 0x55b85a4636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.627+0000 7f06417a3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a464bb0 0x55b85a474a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.627+0000 7f06417a3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a465550 0x55b85a464190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.627+0000 7f06417a3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85a30d4a0 con 0x55b85a465550 2023-09-10T00:40:16.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.627+0000 7f06417a3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85a46e760 con 0x55b85a463db0 2023-09-10T00:40:16.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.627+0000 7f06417a3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85a243de0 con 0x55b85a464bb0 2023-09-10T00:40:16.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.627+0000 7f062f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a465550 0x55b85a464190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.627+0000 7f062f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a465550 0x55b85a464190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48026/0 (socket says 172.21.15.44:48026) 2023-09-10T00:40:16.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.627+0000 7f062f7fe700 1 -- 172.21.15.44:0/3548254166 learned_addr learned my addr 172.21.15.44:0/3548254166 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:16.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.628+0000 7f062effd700 1 --2- 172.21.15.44:0/3548254166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a463db0 0x55b85a4636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.628+0000 7f062e7fc700 1 --2- 172.21.15.44:0/3548254166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a464bb0 0x55b85a474a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.628+0000 7f062f7fe700 1 -- 172.21.15.44:0/3548254166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a464bb0 msgr2=0x55b85a474a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:16.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.628+0000 7f062f7fe700 1 --2- 172.21.15.44:0/3548254166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a464bb0 0x55b85a474a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.628+0000 7f062f7fe700 1 -- 172.21.15.44:0/3548254166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a463db0 msgr2=0x55b85a4636c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:16.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.628+0000 7f062f7fe700 1 --2- 172.21.15.44:0/3548254166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a463db0 0x55b85a4636c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.628+0000 7f062f7fe700 1 -- 172.21.15.44:0/3548254166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b85a208200 con 0x55b85a465550 2023-09-10T00:40:16.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.628+0000 7f062effd700 1 --2- 172.21.15.44:0/3548254166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a463db0 0x55b85a4636c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:16.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.628+0000 7f062f7fe700 1 --2- 172.21.15.44:0/3548254166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a465550 0x55b85a464190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f062800a400 tx=0x7f0628009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ae594828186c4f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:16.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.628+0000 7f0625ffb700 1 -- 172.21.15.44:0/3548254166 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628010040 con 0x55b85a465550 2023-09-10T00:40:16.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.628+0000 7f0625ffb700 1 -- 172.21.15.44:0/3548254166 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f062800ae80 con 0x55b85a465550 2023-09-10T00:40:16.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.628+0000 7f0625ffb700 1 -- 172.21.15.44:0/3548254166 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06280167b0 con 0x55b85a465550 2023-09-10T00:40:16.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 -- 172.21.15.44:0/3548254166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a465550 msgr2=0x55b85a464190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 --2- 172.21.15.44:0/3548254166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a465550 0x55b85a464190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f062800a400 tx=0x7f0628009650 comp rx=0 tx=0).stop 2023-09-10T00:40:16.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 -- 172.21.15.44:0/3548254166 shutdown_connections 2023-09-10T00:40:16.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 --2- 172.21.15.44:0/3548254166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a465550 0x55b85a464190 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 --2- 172.21.15.44:0/3548254166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a463db0 0x55b85a4636c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 --2- 172.21.15.44:0/3548254166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a464bb0 0x55b85a474a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 -- 172.21.15.44:0/3548254166 >> 172.21.15.44:0/3548254166 conn(0x55b85a3d8040 msgr2=0x55b85a477b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:16.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 -- 172.21.15.44:0/3548254166 shutdown_connections 2023-09-10T00:40:16.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 -- 172.21.15.44:0/3548254166 wait complete. 2023-09-10T00:40:16.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 -- start start 2023-09-10T00:40:16.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 Processor -- start 2023-09-10T00:40:16.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a463db0 0x55b85a471d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a464bb0 0x55b85a474250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a465550 0x55b85a474880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85a474f50 con 0x55b85a464bb0 2023-09-10T00:40:16.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85a4ccbe0 con 0x55b85a463db0 2023-09-10T00:40:16.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85a4ccd70 con 0x55b85a465550 2023-09-10T00:40:16.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f06417a3b80 10 client.0 _ll_register_callbacks cb 0x7f0628009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:40:16.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.629+0000 7f062e7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a464bb0 0x55b85a474250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f062e7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a464bb0 0x55b85a474250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48030/0 (socket says 172.21.15.44:48030) 2023-09-10T00:40:16.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f062e7fc700 1 -- 172.21.15.44:0/2148504540 learned_addr learned my addr 172.21.15.44:0/2148504540 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:16.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f062effd700 1 --2- 172.21.15.44:0/2148504540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a463db0 0x55b85a471d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f062f7fe700 1 --2- 172.21.15.44:0/2148504540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a465550 0x55b85a474880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f062e7fc700 1 -- 172.21.15.44:0/2148504540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a465550 msgr2=0x55b85a474880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:16.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f062e7fc700 1 --2- 172.21.15.44:0/2148504540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a465550 0x55b85a474880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f062e7fc700 1 -- 172.21.15.44:0/2148504540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a463db0 msgr2=0x55b85a471d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:16.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f062e7fc700 1 --2- 172.21.15.44:0/2148504540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a463db0 0x55b85a471d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f062e7fc700 1 -- 172.21.15.44:0/2148504540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0628009cb0 con 0x55b85a464bb0 2023-09-10T00:40:16.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f062f7fe700 1 --2- 172.21.15.44:0/2148504540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a465550 0x55b85a474880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:40:16.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f062e7fc700 1 --2- 172.21.15.44:0/2148504540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a464bb0 0x55b85a474250 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f061809aa60 tx=0x7f061801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c148f93a36ca86d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:16.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f06167fc700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:40:16.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f06167fc700 1 -- 172.21.15.44:0/2148504540 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061809e900 con 0x55b85a464bb0 2023-09-10T00:40:16.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f06167fc700 1 -- 172.21.15.44:0/2148504540 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f06180a2ab0 con 0x55b85a464bb0 2023-09-10T00:40:16.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f06417a3b80 1 -- 172.21.15.44:0/2148504540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b85a3ae8d0 con 0x55b85a464bb0 2023-09-10T00:40:16.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f06167fc700 1 -- 172.21.15.44:0/2148504540 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06180040d0 con 0x55b85a464bb0 2023-09-10T00:40:16.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f06417a3b80 10 client.15558 fetch_fsmap learned FSMap version 27 2023-09-10T00:40:16.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f06417a3b80 1 -- 172.21.15.44:0/2148504540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b85a1e0260 con 0x55b85a464bb0 2023-09-10T00:40:16.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f06167fc700 1 -- 172.21.15.44:0/2148504540 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b85a3ae8d0 con 0x55b85a464bb0 2023-09-10T00:40:16.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.630+0000 7f06167fc700 1 -- 172.21.15.44:0/2148504540 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f061809a050 con 0x55b85a464bb0 2023-09-10T00:40:16.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.631+0000 7f06417a3b80 10 client.15558 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:40:16.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.631+0000 7f06417a3b80 10 client.15558 resolve_mds: resolved name 'l' to daemon mds.l 2023-09-10T00:40:16.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.631+0000 7f06417a3b80 20 client.15558 populate_metadata read hostname 'smithi044' 2023-09-10T00:40:16.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.631+0000 7f06417a3b80 1 --2- 172.21.15.44:0/2148504540 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x55b85a4d1d20 0x55b85a4ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.631+0000 7f06417a3b80 4 client.15558 mds_command: new command op to 24502 tid=0[{"prefix": "status"}] 2023-09-10T00:40:16.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.631+0000 7f06417a3b80 1 -- 172.21.15.44:0/2148504540 --> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b85a36f060 con 0x55b85a4d1d20 2023-09-10T00:40:16.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.631+0000 7f062effd700 1 --2- 172.21.15.44:0/2148504540 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x55b85a4d1d20 0x55b85a4ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.631+0000 7f06167fc700 10 client.15558 ms_handle_connect on v2:172.21.15.131:6832/3395466517 2023-09-10T00:40:16.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.631+0000 7f062effd700 1 --2- 172.21.15.44:0/2148504540 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x55b85a4d1d20 0x55b85a4ce250 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fe7abddb5ea60b1e server_cookie=2e12ebe10030187 in_seq=0 out_seq=0 2023-09-10T00:40:16.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.631+0000 7f06167fc700 1 -- 172.21.15.44:0/2148504540 <== mds.4 v2:172.21.15.131:6832/3395466517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b85a36f060 con 0x55b85a4d1d20 2023-09-10T00:40:16.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06167fc700 10 client.15558 handle_command_reply: tid=0 2023-09-10T00:40:16.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06417a3b80 1 client.15558 shutdown 2023-09-10T00:40:16.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06417a3b80 1 -- 172.21.15.44:0/2148504540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a464bb0 msgr2=0x55b85a474250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06417a3b80 1 --2- 172.21.15.44:0/2148504540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a464bb0 0x55b85a474250 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f061809aa60 tx=0x7f061801a040 comp rx=0 tx=0).stop 2023-09-10T00:40:16.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06417a3b80 1 -- 172.21.15.44:0/2148504540 shutdown_connections 2023-09-10T00:40:16.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06417a3b80 1 --2- 172.21.15.44:0/2148504540 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x55b85a4d1d20 0x55b85a4ce250 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06417a3b80 1 --2- 172.21.15.44:0/2148504540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b85a465550 0x55b85a474880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06417a3b80 1 --2- 172.21.15.44:0/2148504540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b85a463db0 0x55b85a471d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06167fc700 0 client.15558 ms_handle_reset on v2:172.21.15.131:6832/3395466517 2023-09-10T00:40:16.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06417a3b80 1 --2- 172.21.15.44:0/2148504540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b85a464bb0 0x55b85a474250 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06417a3b80 1 -- 172.21.15.44:0/2148504540 >> 172.21.15.44:0/2148504540 conn(0x55b85a3d8040 msgr2=0x55b85a464190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:16.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06417a3b80 1 -- 172.21.15.44:0/2148504540 shutdown_connections 2023-09-10T00:40:16.652 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06417a3b80 1 -- 172.21.15.44:0/2148504540 wait complete. 2023-09-10T00:40:16.652 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06417a3b80 20 client.15558 trim_cache size 0 max 16384 2023-09-10T00:40:16.652 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:40:16.652 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da", 2023-09-10T00:40:16.653 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 4, 2023-09-10T00:40:16.653 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24502, 2023-09-10T00:40:16.653 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2023-09-10T00:40:16.653 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2023-09-10T00:40:16.653 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2023-09-10T00:40:16.654 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 75.160333887999997, 2023-09-10T00:40:16.654 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2023-09-10T00:40:16.654 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 89, 2023-09-10T00:40:16.654 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 87, 2023-09-10T00:40:16.654 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 127.705071426 2023-09-10T00:40:16.655 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:40:16.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06355dd700 1 -- 172.21.15.44:0/3373561244 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f061c0a06d0 msgr2=0x7f061c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06355dd700 1 --2- 172.21.15.44:0/3373561244 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f061c0a06d0 0x7f061c0a2b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0630067540 tx=0x7f062000a680 comp rx=0 tx=0).stop 2023-09-10T00:40:16.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06355dd700 1 -- 172.21.15.44:0/3373561244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0630100100 msgr2=0x7f063019af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06355dd700 1 --2- 172.21.15.44:0/3373561244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0630100100 0x7f063019af30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0618006010 tx=0x7f0618003a50 comp rx=0 tx=0).stop 2023-09-10T00:40:16.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06355dd700 1 -- 172.21.15.44:0/3373561244 shutdown_connections 2023-09-10T00:40:16.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06355dd700 1 --2- 172.21.15.44:0/3373561244 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f061c0a06d0 0x7f061c0a2b90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06355dd700 1 --2- 172.21.15.44:0/3373561244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0630100af0 0x7f063019d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06355dd700 1 --2- 172.21.15.44:0/3373561244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06300ff740 0x7f06300ff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06355dd700 1 --2- 172.21.15.44:0/3373561244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0630100100 0x7f063019af30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.632+0000 7f06355dd700 1 -- 172.21.15.44:0/3373561244 >> 172.21.15.44:0/3373561244 conn(0x7f06300fa1d0 msgr2=0x7f0630103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:16.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.633+0000 7f06355dd700 1 -- 172.21.15.44:0/3373561244 shutdown_connections 2023-09-10T00:40:16.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.633+0000 7f06355dd700 1 -- 172.21.15.44:0/3373561244 wait complete. 2023-09-10T00:40:16.658 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2023-09-10T00:40:16.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.792+0000 7f79907d3700 1 -- 172.21.15.44:0/1585782677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79880ff520 msgr2=0x7f79880ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.792+0000 7f79907d3700 1 --2- 172.21.15.44:0/1585782677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79880ff520 0x7f79880ff940 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f79840099c0 tx=0x7f7984009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.793+0000 7f79907d3700 1 -- 172.21.15.44:0/1585782677 shutdown_connections 2023-09-10T00:40:16.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.793+0000 7f79907d3700 1 --2- 172.21.15.44:0/1585782677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79881010b0 0x7f7988107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.793+0000 7f79907d3700 1 --2- 172.21.15.44:0/1585782677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79880ff520 0x7f79880ff940 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.793+0000 7f79907d3700 1 --2- 172.21.15.44:0/1585782677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7988100710 0x7f7988100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.793+0000 7f79907d3700 1 -- 172.21.15.44:0/1585782677 >> 172.21.15.44:0/1585782677 conn(0x7f79880faa90 msgr2=0x7f79880fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:16.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.793+0000 7f79907d3700 1 -- 172.21.15.44:0/1585782677 shutdown_connections 2023-09-10T00:40:16.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.793+0000 7f79907d3700 1 -- 172.21.15.44:0/1585782677 wait complete. 2023-09-10T00:40:16.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.793+0000 7f79907d3700 1 Processor -- start 2023-09-10T00:40:16.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.793+0000 7f79907d3700 1 -- start start 2023-09-10T00:40:16.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.793+0000 7f79907d3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79880ff520 0x7f7988195160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.793+0000 7f79907d3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7988100710 0x7f79881956a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f79907d3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79881010b0 0x7f798819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f79907d3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798819cce0 con 0x7f79880ff520 2023-09-10T00:40:16.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f79907d3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798819ce20 con 0x7f79881010b0 2023-09-10T00:40:16.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f79907d3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798819cfb0 con 0x7f7988100710 2023-09-10T00:40:16.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f798e56f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79880ff520 0x7f7988195160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f798e56f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79880ff520 0x7f7988195160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48046/0 (socket says 172.21.15.44:48046) 2023-09-10T00:40:16.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f798e56f700 1 -- 172.21.15.44:0/1923779822 learned_addr learned my addr 172.21.15.44:0/1923779822 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:16.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f798dd6e700 1 --2- 172.21.15.44:0/1923779822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7988100710 0x7f79881956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f798ed70700 1 --2- 172.21.15.44:0/1923779822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79881010b0 0x7f798819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f798e56f700 1 -- 172.21.15.44:0/1923779822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7988100710 msgr2=0x7f79881956a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f798e56f700 1 --2- 172.21.15.44:0/1923779822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7988100710 0x7f79881956a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f798e56f700 1 -- 172.21.15.44:0/1923779822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79881010b0 msgr2=0x7f798819c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f798e56f700 1 --2- 172.21.15.44:0/1923779822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79881010b0 0x7f798819c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f798e56f700 1 -- 172.21.15.44:0/1923779822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79840095f0 con 0x7f79880ff520 2023-09-10T00:40:16.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f798e56f700 1 --2- 172.21.15.44:0/1923779822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79880ff520 0x7f7988195160 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f79840098e0 tx=0x7f7984009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:16.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f797f7fe700 1 -- 172.21.15.44:0/1923779822 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798401d070 con 0x7f79880ff520 2023-09-10T00:40:16.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f797f7fe700 1 -- 172.21.15.44:0/1923779822 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7984003f80 con 0x7f79880ff520 2023-09-10T00:40:16.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f797f7fe700 1 -- 172.21.15.44:0/1923779822 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984022c40 con 0x7f79880ff520 2023-09-10T00:40:16.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f79907d3700 1 -- 172.21.15.44:0/1923779822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f798819d230 con 0x7f79880ff520 2023-09-10T00:40:16.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.794+0000 7f79907d3700 1 -- 172.21.15.44:0/1923779822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f798819d650 con 0x7f79880ff520 2023-09-10T00:40:16.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.797+0000 7f797f7fe700 1 -- 172.21.15.44:0/1923779822 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f7984022470 con 0x7f79880ff520 2023-09-10T00:40:16.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.797+0000 7f797f7fe700 1 --2- 172.21.15.44:0/1923779822 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f79740a06c0 0x7f79740a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.798+0000 7f798dd6e700 1 --2- 172.21.15.44:0/1923779822 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f79740a06c0 0x7f79740a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.798+0000 7f797f7fe700 1 -- 172.21.15.44:0/1923779822 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f79840cd3d0 con 0x7f79880ff520 2023-09-10T00:40:16.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.798+0000 7f798dd6e700 1 --2- 172.21.15.44:0/1923779822 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f79740a06c0 0x7f79740a2b80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7978005950 tx=0x7f797800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:16.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.798+0000 7f799c999b80 1 Processor -- start 2023-09-10T00:40:16.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f799c999b80 1 -- start start 2023-09-10T00:40:16.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f799c999b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83959e20 0x559d8395a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f799c999b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d8395a780 0x559d83958a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f799c999b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83958fe0 0x559d83959460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f799c999b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d838af6e0 con 0x559d83958fe0 2023-09-10T00:40:16.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f799c999b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d838ad000 con 0x559d83959e20 2023-09-10T00:40:16.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f799c999b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d838a88f0 con 0x559d8395a780 2023-09-10T00:40:16.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f798ed70700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83958fe0 0x559d83959460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f798ed70700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83958fe0 0x559d83959460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48058/0 (socket says 172.21.15.44:48058) 2023-09-10T00:40:16.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f798ed70700 1 -- 172.21.15.44:0/2116669388 learned_addr learned my addr 172.21.15.44:0/2116669388 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:16.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f798e56f700 1 --2- 172.21.15.44:0/2116669388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83959e20 0x559d8395a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f798dd6e700 1 --2- 172.21.15.44:0/2116669388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d8395a780 0x559d83958a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f798e56f700 1 -- 172.21.15.44:0/2116669388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d8395a780 msgr2=0x559d83958a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:16.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f798e56f700 1 --2- 172.21.15.44:0/2116669388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d8395a780 0x559d83958a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f798e56f700 1 -- 172.21.15.44:0/2116669388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83958fe0 msgr2=0x559d83959460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:16.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f798e56f700 1 --2- 172.21.15.44:0/2116669388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83958fe0 0x559d83959460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f798e56f700 1 -- 172.21.15.44:0/2116669388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d836d4d90 con 0x559d83959e20 2023-09-10T00:40:16.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.799+0000 7f798ed70700 1 --2- 172.21.15.44:0/2116669388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83958fe0 0x559d83959460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:40:16.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.800+0000 7f798e56f700 1 --2- 172.21.15.44:0/2116669388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83959e20 0x559d8395a240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f79840cab10 tx=0x7f7984005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a4eed0f9e212a45 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:16.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.800+0000 7f797cff9700 1 -- 172.21.15.44:0/2116669388 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798409b0b0 con 0x559d83959e20 2023-09-10T00:40:16.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.800+0000 7f797cff9700 1 -- 172.21.15.44:0/2116669388 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f798409a9b0 con 0x559d83959e20 2023-09-10T00:40:16.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.800+0000 7f797cff9700 1 -- 172.21.15.44:0/2116669388 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984099ea0 con 0x559d83959e20 2023-09-10T00:40:16.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.800+0000 7f799c999b80 1 -- 172.21.15.44:0/2116669388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83959e20 msgr2=0x559d8395a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.800+0000 7f799c999b80 1 --2- 172.21.15.44:0/2116669388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83959e20 0x559d8395a240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f79840cab10 tx=0x7f7984005a00 comp rx=0 tx=0).stop 2023-09-10T00:40:16.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f799c999b80 1 -- 172.21.15.44:0/2116669388 shutdown_connections 2023-09-10T00:40:16.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f799c999b80 1 --2- 172.21.15.44:0/2116669388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83958fe0 0x559d83959460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f799c999b80 1 --2- 172.21.15.44:0/2116669388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83959e20 0x559d8395a240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f799c999b80 1 --2- 172.21.15.44:0/2116669388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d8395a780 0x559d83958a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f799c999b80 1 -- 172.21.15.44:0/2116669388 >> 172.21.15.44:0/2116669388 conn(0x559d838cb040 msgr2=0x559d839516d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:16.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f799c999b80 1 -- 172.21.15.44:0/2116669388 shutdown_connections 2023-09-10T00:40:16.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f799c999b80 1 -- 172.21.15.44:0/2116669388 wait complete. 2023-09-10T00:40:16.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f799c999b80 1 -- start start 2023-09-10T00:40:16.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f799c999b80 1 Processor -- start 2023-09-10T00:40:16.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f799c999b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83958fe0 0x559d83962400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f799c999b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d83959e20 0x559d83962940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f799c999b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d8395a780 0x559d83969980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f799c999b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d83846b20 con 0x559d8395a780 2023-09-10T00:40:16.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f799c999b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d83846820 con 0x559d83958fe0 2023-09-10T00:40:16.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f799c999b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8372a8d0 con 0x559d83959e20 2023-09-10T00:40:16.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f799c999b80 10 client.0 _ll_register_callbacks cb 0x559d83790a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:40:16.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f798ed70700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d8395a780 0x559d83969980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.801+0000 7f798ed70700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d8395a780 0x559d83969980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48064/0 (socket says 172.21.15.44:48064) 2023-09-10T00:40:16.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f798ed70700 1 -- 172.21.15.44:0/1263676566 learned_addr learned my addr 172.21.15.44:0/1263676566 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:16.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f798dd6e700 1 --2- 172.21.15.44:0/1263676566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d83959e20 0x559d83962940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f798e56f700 1 --2- 172.21.15.44:0/1263676566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83958fe0 0x559d83962400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f798ed70700 1 -- 172.21.15.44:0/1263676566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d83959e20 msgr2=0x559d83962940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:16.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f798ed70700 1 --2- 172.21.15.44:0/1263676566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d83959e20 0x559d83962940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f798ed70700 1 -- 172.21.15.44:0/1263676566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83958fe0 msgr2=0x559d83962400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:16.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f798ed70700 1 --2- 172.21.15.44:0/1263676566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83958fe0 0x559d83962400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f798ed70700 1 -- 172.21.15.44:0/1263676566 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7980003c50 con 0x559d8395a780 2023-09-10T00:40:16.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f798e56f700 1 --2- 172.21.15.44:0/1263676566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83958fe0 0x559d83962400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:16.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f798ed70700 1 --2- 172.21.15.44:0/1263676566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d8395a780 0x559d83969980 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7980004270 tx=0x7f79800042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a20e900e4c146ee server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:16.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f7971ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:40:16.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f7971ffb700 1 -- 172.21.15.44:0/1263676566 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7980007910 con 0x559d8395a780 2023-09-10T00:40:16.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f7971ffb700 1 -- 172.21.15.44:0/1263676566 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f7980007a70 con 0x559d8395a780 2023-09-10T00:40:16.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f799c999b80 1 -- 172.21.15.44:0/1263676566 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d838004a0 con 0x559d8395a780 2023-09-10T00:40:16.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f7971ffb700 1 -- 172.21.15.44:0/1263676566 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7980007c30 con 0x559d8395a780 2023-09-10T00:40:16.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f799c999b80 10 client.15570 fetch_fsmap learned FSMap version 27 2023-09-10T00:40:16.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.802+0000 7f799c999b80 1 -- 172.21.15.44:0/1263676566 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f798819d650 con 0x559d8395a780 2023-09-10T00:40:16.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.803+0000 7f7971ffb700 1 -- 172.21.15.44:0/1263676566 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559d838004a0 con 0x559d8395a780 2023-09-10T00:40:16.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.803+0000 7f7971ffb700 1 -- 172.21.15.44:0/1263676566 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f798001e3b0 con 0x559d8395a780 2023-09-10T00:40:16.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.803+0000 7f799c999b80 10 client.15570 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:40:16.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.803+0000 7f799c999b80 10 client.15570 resolve_mds: resolved name 'k' to daemon mds.k 2023-09-10T00:40:16.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.803+0000 7f799c999b80 20 client.15570 populate_metadata read hostname 'smithi044' 2023-09-10T00:40:16.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.803+0000 7f799c999b80 1 --2- 172.21.15.44:0/1263676566 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x559d8396db80 0x559d8396cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.803+0000 7f799c999b80 4 client.15570 mds_command: new command op to 24515 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:40:16.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.803+0000 7f799c999b80 1 -- 172.21.15.44:0/1263676566 --> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d838bd850 con 0x559d8396db80 2023-09-10T00:40:16.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.803+0000 7f798e56f700 1 --2- 172.21.15.44:0/1263676566 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x559d8396db80 0x559d8396cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.803+0000 7f798e56f700 1 --2- 172.21.15.44:0/1263676566 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x559d8396db80 0x559d8396cae0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c60ca71db94925e server_cookie=e3b45f179ed03eff in_seq=0 out_seq=0 2023-09-10T00:40:16.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.803+0000 7f7971ffb700 10 client.15570 ms_handle_connect on v2:172.21.15.99:6832/1934109590 2023-09-10T00:40:16.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.804+0000 7f7971ffb700 1 -- 172.21.15.44:0/1263676566 <== mds.3 v2:172.21.15.99:6832/1934109590 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559d838bd850 con 0x559d8396db80 2023-09-10T00:40:16.821 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.804+0000 7f7971ffb700 10 client.15570 handle_command_reply: tid=0 2023-09-10T00:40:16.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.804+0000 7f799c999b80 1 client.15570 shutdown 2023-09-10T00:40:16.822 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:16 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2156969952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:40:16.822 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:16 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2841530742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:40:16.822 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:16 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/616912973' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:16.823 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-09-10T00:40:16.823 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-09-10T00:40:16.823 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-09-10T00:40:16.823 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-09-10T00:40:16.824 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2023-09-10T00:40:16.824 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:16 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2023-09-10T00:40:16.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.805+0000 7f799c999b80 1 -- 172.21.15.44:0/1263676566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d8395a780 msgr2=0x559d83969980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.805+0000 7f799c999b80 1 --2- 172.21.15.44:0/1263676566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d8395a780 0x559d83969980 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7980004270 tx=0x7f79800042a0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.805+0000 7f799c999b80 1 -- 172.21.15.44:0/1263676566 shutdown_connections 2023-09-10T00:40:16.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.805+0000 7f799c999b80 1 --2- 172.21.15.44:0/1263676566 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x559d8396db80 0x559d8396cae0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.805+0000 7f799c999b80 1 --2- 172.21.15.44:0/1263676566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d8395a780 0x559d83969980 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.805+0000 7f799c999b80 1 --2- 172.21.15.44:0/1263676566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83958fe0 0x559d83962400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.805+0000 7f799c999b80 1 --2- 172.21.15.44:0/1263676566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d83959e20 0x559d83962940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.805+0000 7f7971ffb700 0 client.15570 ms_handle_reset on v2:172.21.15.99:6832/1934109590 2023-09-10T00:40:16.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.805+0000 7f799c999b80 1 -- 172.21.15.44:0/1263676566 >> 172.21.15.44:0/1263676566 conn(0x559d838cb040 msgr2=0x559d83950d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:16.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.805+0000 7f799c999b80 1 -- 172.21.15.44:0/1263676566 shutdown_connections 2023-09-10T00:40:16.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.805+0000 7f799c999b80 1 -- 172.21.15.44:0/1263676566 wait complete. 2023-09-10T00:40:16.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.806+0000 7f799c999b80 20 client.15570 trim_cache size 0 max 16384 2023-09-10T00:40:16.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.823+0000 7f799c999b80 1 Processor -- start 2023-09-10T00:40:16.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f799c999b80 1 -- start start 2023-09-10T00:40:16.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f799c999b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83958fe0 0x559d839593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f799c999b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83959e20 0x559d839698a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f799c999b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d8395a780 0x559d8396cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f799c999b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d83736de0 con 0x559d83958fe0 2023-09-10T00:40:16.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f799c999b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d83871f70 con 0x559d83959e20 2023-09-10T00:40:16.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f799c999b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d83864ed0 con 0x559d8395a780 2023-09-10T00:40:16.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f798e56f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83958fe0 0x559d839593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f798e56f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83958fe0 0x559d839593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48068/0 (socket says 172.21.15.44:48068) 2023-09-10T00:40:16.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f798e56f700 1 -- 172.21.15.44:0/3250229207 learned_addr learned my addr 172.21.15.44:0/3250229207 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:16.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f798ed70700 1 --2- 172.21.15.44:0/3250229207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d8395a780 0x559d8396cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f798dd6e700 1 --2- 172.21.15.44:0/3250229207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83959e20 0x559d839698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f798e56f700 1 -- 172.21.15.44:0/3250229207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d8395a780 msgr2=0x559d8396cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:16.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f798e56f700 1 --2- 172.21.15.44:0/3250229207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d8395a780 0x559d8396cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f798e56f700 1 -- 172.21.15.44:0/3250229207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83959e20 msgr2=0x559d839698a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:16.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f798e56f700 1 --2- 172.21.15.44:0/3250229207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83959e20 0x559d839698a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f798e56f700 1 -- 172.21.15.44:0/3250229207 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d836bdc10 con 0x559d83958fe0 2023-09-10T00:40:16.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f798dd6e700 1 --2- 172.21.15.44:0/3250229207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83959e20 0x559d839698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:40:16.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f798e56f700 1 --2- 172.21.15.44:0/3250229207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83958fe0 0x559d839593c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7984006010 tx=0x7f79840b4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7b5a0ffac0f6bd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:16.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.824+0000 7f797d7fa700 1 -- 172.21.15.44:0/3250229207 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79840b8040 con 0x559d83958fe0 2023-09-10T00:40:16.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.825+0000 7f797d7fa700 1 -- 172.21.15.44:0/3250229207 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f79840cd170 con 0x559d83958fe0 2023-09-10T00:40:16.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.825+0000 7f797d7fa700 1 -- 172.21.15.44:0/3250229207 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984098e70 con 0x559d83958fe0 2023-09-10T00:40:16.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.825+0000 7f799c999b80 1 -- 172.21.15.44:0/3250229207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83958fe0 msgr2=0x559d839593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.825+0000 7f799c999b80 1 --2- 172.21.15.44:0/3250229207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83958fe0 0x559d839593c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7984006010 tx=0x7f79840b4040 comp rx=0 tx=0).stop 2023-09-10T00:40:16.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.825+0000 7f799c999b80 1 -- 172.21.15.44:0/3250229207 shutdown_connections 2023-09-10T00:40:16.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.825+0000 7f799c999b80 1 --2- 172.21.15.44:0/3250229207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d8395a780 0x559d8396cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.825+0000 7f799c999b80 1 --2- 172.21.15.44:0/3250229207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83958fe0 0x559d839593c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.825+0000 7f799c999b80 1 --2- 172.21.15.44:0/3250229207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d83959e20 0x559d839698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.825+0000 7f799c999b80 1 -- 172.21.15.44:0/3250229207 >> 172.21.15.44:0/3250229207 conn(0x559d838cb040 msgr2=0x559d839516a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.825+0000 7f799c999b80 1 -- 172.21.15.44:0/3250229207 shutdown_connections 2023-09-10T00:40:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.825+0000 7f799c999b80 1 -- 172.21.15.44:0/3250229207 wait complete. 2023-09-10T00:40:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.825+0000 7f799c999b80 1 -- start start 2023-09-10T00:40:16.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f799c999b80 1 Processor -- start 2023-09-10T00:40:16.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f799c999b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d83958fe0 0x559d83964b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f799c999b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83959e20 0x559d83965050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f799c999b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d8395a780 0x559d839c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f799c999b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d83969bb0 con 0x559d83959e20 2023-09-10T00:40:16.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f799c999b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d83969d20 con 0x559d8395a780 2023-09-10T00:40:16.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f799c999b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d839c2110 con 0x559d83958fe0 2023-09-10T00:40:16.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f799c999b80 10 client.0 _ll_register_callbacks cb 0x7f7980009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:40:16.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f798dd6e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83959e20 0x559d83965050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f798dd6e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83959e20 0x559d83965050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48076/0 (socket says 172.21.15.44:48076) 2023-09-10T00:40:16.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f798dd6e700 1 -- 172.21.15.44:0/301766681 learned_addr learned my addr 172.21.15.44:0/301766681 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:16.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f798e56f700 1 --2- 172.21.15.44:0/301766681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d83958fe0 0x559d83964b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f798ed70700 1 --2- 172.21.15.44:0/301766681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d8395a780 0x559d839c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f798dd6e700 1 -- 172.21.15.44:0/301766681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d83958fe0 msgr2=0x559d83964b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:16.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f798dd6e700 1 --2- 172.21.15.44:0/301766681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d83958fe0 0x559d83964b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f798dd6e700 1 -- 172.21.15.44:0/301766681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d8395a780 msgr2=0x559d839c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:16.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f798dd6e700 1 --2- 172.21.15.44:0/301766681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d8395a780 0x559d839c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f798dd6e700 1 -- 172.21.15.44:0/301766681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d836d4d90 con 0x559d83959e20 2023-09-10T00:40:16.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.826+0000 7f798ed70700 1 --2- 172.21.15.44:0/301766681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d8395a780 0x559d839c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:16.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.827+0000 7f798dd6e700 1 --2- 172.21.15.44:0/301766681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83959e20 0x559d83965050 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f79780117c0 tx=0x7f79780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b852e5f7b3a9393 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:16.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.827+0000 7f7971ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:40:16.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.827+0000 7f7971ffb700 1 -- 172.21.15.44:0/301766681 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978024070 con 0x559d83959e20 2023-09-10T00:40:16.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.827+0000 7f7971ffb700 1 -- 172.21.15.44:0/301766681 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f7978017070 con 0x559d83959e20 2023-09-10T00:40:16.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.827+0000 7f799c999b80 1 -- 172.21.15.44:0/301766681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f798409a9b0 con 0x559d83959e20 2023-09-10T00:40:16.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.827+0000 7f7971ffb700 1 -- 172.21.15.44:0/301766681 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7978004830 con 0x559d83959e20 2023-09-10T00:40:16.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.827+0000 7f799c999b80 10 client.15576 fetch_fsmap learned FSMap version 27 2023-09-10T00:40:16.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.827+0000 7f799c999b80 1 -- 172.21.15.44:0/301766681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559d836fb200 con 0x559d83959e20 2023-09-10T00:40:16.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.827+0000 7f7971ffb700 1 -- 172.21.15.44:0/301766681 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f798409a9b0 con 0x559d83959e20 2023-09-10T00:40:16.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.827+0000 7f7971ffb700 1 -- 172.21.15.44:0/301766681 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f79780055d0 con 0x559d83959e20 2023-09-10T00:40:16.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.827+0000 7f799c999b80 10 client.15576 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:40:16.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.827+0000 7f799c999b80 10 client.15576 resolve_mds: resolved name 'k' to daemon mds.k 2023-09-10T00:40:16.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.827+0000 7f799c999b80 20 client.15576 populate_metadata read hostname 'smithi044' 2023-09-10T00:40:16.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.827+0000 7f799c999b80 1 --2- 172.21.15.44:0/301766681 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x559d839c7330 0x559d839c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.827+0000 7f799c999b80 4 client.15576 mds_command: new command op to 24515 tid=0[{"prefix": "status"}] 2023-09-10T00:40:16.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.827+0000 7f799c999b80 1 -- 172.21.15.44:0/301766681 --> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559d838a18d0 con 0x559d839c7330 2023-09-10T00:40:16.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.828+0000 7f798e56f700 1 --2- 172.21.15.44:0/301766681 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x559d839c7330 0x559d839c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:16.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.828+0000 7f798e56f700 1 --2- 172.21.15.44:0/301766681 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x559d839c7330 0x559d839c37b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=615b2cd54c4da0ad server_cookie=fd9522409da7000a in_seq=0 out_seq=0 2023-09-10T00:40:16.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.828+0000 7f7971ffb700 10 client.15576 ms_handle_connect on v2:172.21.15.99:6832/1934109590 2023-09-10T00:40:16.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.828+0000 7f7971ffb700 1 -- 172.21.15.44:0/301766681 <== mds.3 v2:172.21.15.99:6832/1934109590 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x559d838a18d0 con 0x559d839c7330 2023-09-10T00:40:16.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.828+0000 7f7971ffb700 10 client.15576 handle_command_reply: tid=0 2023-09-10T00:40:16.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.828+0000 7f799c999b80 1 client.15576 shutdown 2023-09-10T00:40:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.828+0000 7f799c999b80 1 -- 172.21.15.44:0/301766681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83959e20 msgr2=0x559d83965050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.828+0000 7f799c999b80 1 --2- 172.21.15.44:0/301766681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83959e20 0x559d83965050 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f79780117c0 tx=0x7f79780047c0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f799c999b80 1 -- 172.21.15.44:0/301766681 shutdown_connections 2023-09-10T00:40:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f799c999b80 1 --2- 172.21.15.44:0/301766681 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x559d839c7330 0x559d839c37b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f799c999b80 1 --2- 172.21.15.44:0/301766681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d8395a780 0x559d839c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f799c999b80 1 --2- 172.21.15.44:0/301766681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d83958fe0 0x559d83964b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f799c999b80 1 --2- 172.21.15.44:0/301766681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d83959e20 0x559d83965050 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f799c999b80 1 -- 172.21.15.44:0/301766681 >> 172.21.15.44:0/301766681 conn(0x559d838cb040 msgr2=0x559d839593c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:16.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f7971ffb700 0 client.15576 ms_handle_reset on v2:172.21.15.99:6832/1934109590 2023-09-10T00:40:16.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f799c999b80 1 -- 172.21.15.44:0/301766681 shutdown_connections 2023-09-10T00:40:16.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f799c999b80 1 -- 172.21.15.44:0/301766681 wait complete. 2023-09-10T00:40:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f799c999b80 20 client.15576 trim_cache size 0 max 16384 2023-09-10T00:40:16.860 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:40:16.860 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da", 2023-09-10T00:40:16.861 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 3, 2023-09-10T00:40:16.861 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24515, 2023-09-10T00:40:16.861 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2023-09-10T00:40:16.861 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2023-09-10T00:40:16.862 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2023-09-10T00:40:16.862 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 76.364334103000004, 2023-09-10T00:40:16.862 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2023-09-10T00:40:16.862 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 90, 2023-09-10T00:40:16.862 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 87, 2023-09-10T00:40:16.863 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 125.731008695 2023-09-10T00:40:16.863 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:40:16.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f79907d3700 1 -- 172.21.15.44:0/1923779822 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f79740a06c0 msgr2=0x7f79740a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f79907d3700 1 --2- 172.21.15.44:0/1923779822 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f79740a06c0 0x7f79740a2b80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7978005950 tx=0x7f797800b410 comp rx=0 tx=0).stop 2023-09-10T00:40:16.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f79907d3700 1 -- 172.21.15.44:0/1923779822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79880ff520 msgr2=0x7f7988195160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f79907d3700 1 --2- 172.21.15.44:0/1923779822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79880ff520 0x7f7988195160 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f79840098e0 tx=0x7f7984009580 comp rx=0 tx=0).stop 2023-09-10T00:40:16.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f79907d3700 1 -- 172.21.15.44:0/1923779822 shutdown_connections 2023-09-10T00:40:16.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f79907d3700 1 --2- 172.21.15.44:0/1923779822 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f79740a06c0 0x7f79740a2b80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f79907d3700 1 --2- 172.21.15.44:0/1923779822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79881010b0 0x7f798819c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f79907d3700 1 --2- 172.21.15.44:0/1923779822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79880ff520 0x7f7988195160 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.866 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f79907d3700 1 --2- 172.21.15.44:0/1923779822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7988100710 0x7f79881956a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.866 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f79907d3700 1 -- 172.21.15.44:0/1923779822 >> 172.21.15.44:0/1923779822 conn(0x7f79880faa90 msgr2=0x7f79880fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:16.866 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f79907d3700 1 -- 172.21.15.44:0/1923779822 shutdown_connections 2023-09-10T00:40:16.866 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.829+0000 7f79907d3700 1 -- 172.21.15.44:0/1923779822 wait complete. 2023-09-10T00:40:16.867 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-09-10T00:40:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.994+0000 7fb246b9e700 1 -- 172.21.15.44:0/2438148841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2401021a0 msgr2=0x7fb240108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.994+0000 7fb246b9e700 1 --2- 172.21.15.44:0/2438148841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2401021a0 0x7fb240108a50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb23800b0d0 tx=0x7fb23800b3e0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.994+0000 7fb246b9e700 1 -- 172.21.15.44:0/2438148841 shutdown_connections 2023-09-10T00:40:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.994+0000 7fb246b9e700 1 --2- 172.21.15.44:0/2438148841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2401021a0 0x7fb240108a50 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.994+0000 7fb246b9e700 1 --2- 172.21.15.44:0/2438148841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb240100610 0x7fb240100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.994+0000 7fb246b9e700 1 --2- 172.21.15.44:0/2438148841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb240101800 0x7fb240101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.994+0000 7fb246b9e700 1 -- 172.21.15.44:0/2438148841 >> 172.21.15.44:0/2438148841 conn(0x7fb2400fbb80 msgr2=0x7fb2400fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.994+0000 7fb246b9e700 1 -- 172.21.15.44:0/2438148841 shutdown_connections 2023-09-10T00:40:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.994+0000 7fb246b9e700 1 -- 172.21.15.44:0/2438148841 wait complete. 2023-09-10T00:40:16.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.995+0000 7fb246b9e700 1 Processor -- start 2023-09-10T00:40:16.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.995+0000 7fb246b9e700 1 -- start start 2023-09-10T00:40:16.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.995+0000 7fb246b9e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb240100610 0x7fb2401962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.995+0000 7fb246b9e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb240101800 0x7fb240196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:16.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.995+0000 7fb246b9e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2401021a0 0x7fb24019d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.995+0000 7fb246b9e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24019de40 con 0x7fb2401021a0 2023-09-10T00:40:17.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.995+0000 7fb246b9e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24019df80 con 0x7fb240100610 2023-09-10T00:40:17.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.995+0000 7fb246b9e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24019e110 con 0x7fb240101800 2023-09-10T00:40:17.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.995+0000 7fb24513b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2401021a0 0x7fb24019d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.995+0000 7fb24513b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2401021a0 0x7fb24019d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48090/0 (socket says 172.21.15.44:48090) 2023-09-10T00:40:17.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.995+0000 7fb24513b700 1 -- 172.21.15.44:0/478134658 learned_addr learned my addr 172.21.15.44:0/478134658 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:17.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.995+0000 7fb23ffff700 1 --2- 172.21.15.44:0/478134658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb240101800 0x7fb240196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.995+0000 7fb24493a700 1 --2- 172.21.15.44:0/478134658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb240100610 0x7fb2401962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.996+0000 7fb24513b700 1 -- 172.21.15.44:0/478134658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb240101800 msgr2=0x7fb240196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.996+0000 7fb24513b700 1 --2- 172.21.15.44:0/478134658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb240101800 0x7fb240196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.996+0000 7fb24513b700 1 -- 172.21.15.44:0/478134658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb240100610 msgr2=0x7fb2401962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.996+0000 7fb24513b700 1 --2- 172.21.15.44:0/478134658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb240100610 0x7fb2401962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.996+0000 7fb24513b700 1 -- 172.21.15.44:0/478134658 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb238009d00 con 0x7fb2401021a0 2023-09-10T00:40:17.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.996+0000 7fb24513b700 1 --2- 172.21.15.44:0/478134658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2401021a0 0x7fb24019d870 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb23800d010 tx=0x7fb2380094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.996+0000 7fb23dffb700 1 -- 172.21.15.44:0/478134658 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23800e040 con 0x7fb2401021a0 2023-09-10T00:40:17.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.996+0000 7fb23dffb700 1 -- 172.21.15.44:0/478134658 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb238003e10 con 0x7fb2401021a0 2023-09-10T00:40:17.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.996+0000 7fb23dffb700 1 -- 172.21.15.44:0/478134658 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb238012a30 con 0x7fb2401021a0 2023-09-10T00:40:17.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.996+0000 7fb246b9e700 1 -- 172.21.15.44:0/478134658 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb24019e390 con 0x7fb2401021a0 2023-09-10T00:40:17.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.996+0000 7fb246b9e700 1 -- 172.21.15.44:0/478134658 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24019e760 con 0x7fb2401021a0 2023-09-10T00:40:17.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.999+0000 7fb23dffb700 1 -- 172.21.15.44:0/478134658 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fb238012b90 con 0x7fb2401021a0 2023-09-10T00:40:17.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:16.999+0000 7fb23dffb700 1 --2- 172.21.15.44:0/478134658 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb2300a0640 0x7fb2300a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.000+0000 7fb23dffb700 1 -- 172.21.15.44:0/478134658 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9194+0+0 (secure 0 0 0) 0x7fb2380cd6a0 con 0x7fb2401021a0 2023-09-10T00:40:17.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.000+0000 7fb24493a700 1 --2- 172.21.15.44:0/478134658 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb2300a0640 0x7fb2300a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.000+0000 7fb24493a700 1 --2- 172.21.15.44:0/478134658 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb2300a0640 0x7fb2300a2b00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb240101660 tx=0x7fb23400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.000+0000 7fb252d64b80 1 Processor -- start 2023-09-10T00:40:17.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.000+0000 7fb252d64b80 1 -- start start 2023-09-10T00:40:17.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.000+0000 7fb252d64b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880fbb0 0x55947880ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.000+0000 7fb252d64b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559478810550 0x55947880e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.000+0000 7fb252d64b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880edb0 0x55947880f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.000+0000 7fb252d64b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594787676e0 con 0x55947880fbb0 2023-09-10T00:40:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.000+0000 7fb252d64b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559478765000 con 0x55947880edb0 2023-09-10T00:40:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.000+0000 7fb252d64b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594787608f0 con 0x559478810550 2023-09-10T00:40:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.001+0000 7fb24493a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880fbb0 0x55947880ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.001+0000 7fb24493a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880fbb0 0x55947880ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48094/0 (socket says 172.21.15.44:48094) 2023-09-10T00:40:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.001+0000 7fb24493a700 1 -- 172.21.15.44:0/2558418345 learned_addr learned my addr 172.21.15.44:0/2558418345 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:17.013 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.001+0000 7fb23ffff700 1 --2- 172.21.15.44:0/2558418345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559478810550 0x55947880e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.013 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.001+0000 7fb24513b700 1 --2- 172.21.15.44:0/2558418345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880edb0 0x55947880f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.013 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.001+0000 7fb24493a700 1 -- 172.21.15.44:0/2558418345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559478810550 msgr2=0x55947880e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.013 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.001+0000 7fb24493a700 1 --2- 172.21.15.44:0/2558418345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559478810550 0x55947880e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.013 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.001+0000 7fb24493a700 1 -- 172.21.15.44:0/2558418345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880edb0 msgr2=0x55947880f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.001+0000 7fb24493a700 1 --2- 172.21.15.44:0/2558418345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880edb0 0x55947880f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.001+0000 7fb24493a700 1 -- 172.21.15.44:0/2558418345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55947858cd90 con 0x55947880fbb0 2023-09-10T00:40:17.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.001+0000 7fb24493a700 1 --2- 172.21.15.44:0/2558418345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880fbb0 0x55947880ffd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb23400fe20 tx=0x7fb234012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663be93fb5ae9bf7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.001+0000 7fb22affd700 1 -- 172.21.15.44:0/2558418345 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23400adf0 con 0x55947880fbb0 2023-09-10T00:40:17.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.001+0000 7fb22affd700 1 -- 172.21.15.44:0/2558418345 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb234020070 con 0x55947880fbb0 2023-09-10T00:40:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.001+0000 7fb22affd700 1 -- 172.21.15.44:0/2558418345 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb234025d60 con 0x55947880fbb0 2023-09-10T00:40:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.002+0000 7fb252d64b80 1 -- 172.21.15.44:0/2558418345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880fbb0 msgr2=0x55947880ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.002+0000 7fb252d64b80 1 --2- 172.21.15.44:0/2558418345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880fbb0 0x55947880ffd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb23400fe20 tx=0x7fb234012000 comp rx=0 tx=0).stop 2023-09-10T00:40:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.002+0000 7fb252d64b80 1 -- 172.21.15.44:0/2558418345 shutdown_connections 2023-09-10T00:40:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.002+0000 7fb252d64b80 1 --2- 172.21.15.44:0/2558418345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880edb0 0x55947880f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.002+0000 7fb252d64b80 1 --2- 172.21.15.44:0/2558418345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880fbb0 0x55947880ffd0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.002+0000 7fb252d64b80 1 --2- 172.21.15.44:0/2558418345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559478810550 0x55947880e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.002+0000 7fb252d64b80 1 -- 172.21.15.44:0/2558418345 >> 172.21.15.44:0/2558418345 conn(0x559478783040 msgr2=0x5594788096d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.002+0000 7fb252d64b80 1 -- 172.21.15.44:0/2558418345 shutdown_connections 2023-09-10T00:40:17.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.002+0000 7fb252d64b80 1 -- 172.21.15.44:0/2558418345 wait complete. 2023-09-10T00:40:17.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.002+0000 7fb252d64b80 1 -- start start 2023-09-10T00:40:17.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.002+0000 7fb252d64b80 1 Processor -- start 2023-09-10T00:40:17.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb252d64b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880edb0 0x5594788185f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb252d64b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55947880fbb0 0x559478818b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb252d64b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559478810550 0x55947881fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb252d64b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594786feb20 con 0x559478810550 2023-09-10T00:40:17.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb252d64b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594786fe820 con 0x55947880edb0 2023-09-10T00:40:17.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb252d64b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594785e28d0 con 0x55947880fbb0 2023-09-10T00:40:17.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb252d64b80 10 client.0 _ll_register_callbacks cb 0x559478648a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:40:17.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb24513b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559478810550 0x55947881fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb24513b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559478810550 0x55947881fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48104/0 (socket says 172.21.15.44:48104) 2023-09-10T00:40:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb24513b700 1 -- 172.21.15.44:0/3471606420 learned_addr learned my addr 172.21.15.44:0/3471606420 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb24493a700 1 --2- 172.21.15.44:0/3471606420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880edb0 0x5594788185f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb23ffff700 1 --2- 172.21.15.44:0/3471606420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55947880fbb0 0x559478818b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb24513b700 1 -- 172.21.15.44:0/3471606420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55947880fbb0 msgr2=0x559478818b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb24513b700 1 --2- 172.21.15.44:0/3471606420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55947880fbb0 0x559478818b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb24513b700 1 -- 172.21.15.44:0/3471606420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880edb0 msgr2=0x5594788185f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb24513b700 1 --2- 172.21.15.44:0/3471606420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880edb0 0x5594788185f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb24513b700 1 -- 172.21.15.44:0/3471606420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23400fc40 con 0x559478810550 2023-09-10T00:40:17.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb24493a700 1 --2- 172.21.15.44:0/3471606420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880edb0 0x5594788185f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:17.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb24513b700 1 --2- 172.21.15.44:0/3471606420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559478810550 0x55947881fb70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb238009fd0 tx=0x7fb23809a770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb65666faee0aa7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.003+0000 7fb21ffff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:40:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.004+0000 7fb21ffff700 1 -- 172.21.15.44:0/3471606420 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23809ab00 con 0x559478810550 2023-09-10T00:40:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.004+0000 7fb21ffff700 1 -- 172.21.15.44:0/3471606420 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9194+0+0 (secure 0 0 0) 0x7fb23809e930 con 0x559478810550 2023-09-10T00:40:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.004+0000 7fb252d64b80 1 -- 172.21.15.44:0/3471606420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594786b84a0 con 0x559478810550 2023-09-10T00:40:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.004+0000 7fb21ffff700 1 -- 172.21.15.44:0/3471606420 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2380a2ae0 con 0x559478810550 2023-09-10T00:40:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.004+0000 7fb252d64b80 10 client.15588 fetch_fsmap learned FSMap version 27 2023-09-10T00:40:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.004+0000 7fb252d64b80 1 -- 172.21.15.44:0/3471606420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559478575c10 con 0x559478810550 2023-09-10T00:40:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.004+0000 7fb21ffff700 1 -- 172.21.15.44:0/3471606420 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5594786b84a0 con 0x559478810550 2023-09-10T00:40:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.004+0000 7fb21ffff700 1 -- 172.21.15.44:0/3471606420 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb2380cd070 con 0x559478810550 2023-09-10T00:40:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.004+0000 7fb252d64b80 10 client.15588 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:40:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.004+0000 7fb252d64b80 10 client.15588 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-10T00:40:17.023 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.004+0000 7fb252d64b80 20 client.15588 populate_metadata read hostname 'smithi044' 2023-09-10T00:40:17.023 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.004+0000 7fb252d64b80 1 --2- 172.21.15.44:0/3471606420 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x559478823ab0 0x559478822aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.023 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.004+0000 7fb252d64b80 4 client.15588 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:40:17.023 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.004+0000 7fb252d64b80 1 -- 172.21.15.44:0/3471606420 --> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559478775850 con 0x559478823ab0 2023-09-10T00:40:17.023 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.005+0000 7fb24493a700 1 --2- 172.21.15.44:0/3471606420 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x559478823ab0 0x559478822aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.024 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.005+0000 7fb21ffff700 10 client.15588 ms_handle_connect on v2:172.21.15.99:6836/566283638 2023-09-10T00:40:17.024 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.005+0000 7fb24493a700 1 --2- 172.21.15.44:0/3471606420 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x559478823ab0 0x559478822aa0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2442aeb49ad5c90 server_cookie=f07072fb8b5bb7f7 in_seq=0 out_seq=0 2023-09-10T00:40:17.024 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.005+0000 7fb21ffff700 1 -- 172.21.15.44:0/3471606420 <== mds.2 v2:172.21.15.99:6836/566283638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559478775850 con 0x559478823ab0 2023-09-10T00:40:17.024 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.005+0000 7fb21ffff700 10 client.15588 handle_command_reply: tid=0 2023-09-10T00:40:17.024 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.006+0000 7fb252d64b80 1 client.15588 shutdown 2023-09-10T00:40:17.024 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.006+0000 7fb252d64b80 1 -- 172.21.15.44:0/3471606420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559478810550 msgr2=0x55947881fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.025 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.006+0000 7fb252d64b80 1 --2- 172.21.15.44:0/3471606420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559478810550 0x55947881fb70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb238009fd0 tx=0x7fb23809a770 comp rx=0 tx=0).stop 2023-09-10T00:40:17.025 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.006+0000 7fb252d64b80 1 -- 172.21.15.44:0/3471606420 shutdown_connections 2023-09-10T00:40:17.025 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.006+0000 7fb252d64b80 1 --2- 172.21.15.44:0/3471606420 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x559478823ab0 0x559478822aa0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.025 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.006+0000 7fb252d64b80 1 --2- 172.21.15.44:0/3471606420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559478810550 0x55947881fb70 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.025 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.006+0000 7fb21ffff700 0 client.15588 ms_handle_reset on v2:172.21.15.99:6836/566283638 2023-09-10T00:40:17.026 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.006+0000 7fb252d64b80 1 --2- 172.21.15.44:0/3471606420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880edb0 0x5594788185f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.026 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.006+0000 7fb252d64b80 1 --2- 172.21.15.44:0/3471606420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55947880fbb0 0x559478818b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.026 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.006+0000 7fb252d64b80 1 -- 172.21.15.44:0/3471606420 >> 172.21.15.44:0/3471606420 conn(0x559478783040 msgr2=0x559478808d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.026 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.006+0000 7fb252d64b80 1 -- 172.21.15.44:0/3471606420 shutdown_connections 2023-09-10T00:40:17.026 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.006+0000 7fb252d64b80 1 -- 172.21.15.44:0/3471606420 wait complete. 2023-09-10T00:40:17.026 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.006+0000 7fb252d64b80 20 client.15588 trim_cache size 0 max 16384 2023-09-10T00:40:17.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb252d64b80 1 Processor -- start 2023-09-10T00:40:17.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb252d64b80 1 -- start start 2023-09-10T00:40:17.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb252d64b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880edb0 0x559478822aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb252d64b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880fbb0 0x55947881fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb252d64b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559478810550 0x5594788202e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb252d64b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559478819380 con 0x55947880edb0 2023-09-10T00:40:17.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb252d64b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55947881ffd0 con 0x55947880fbb0 2023-09-10T00:40:17.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb252d64b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594788013b0 con 0x559478810550 2023-09-10T00:40:17.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb24493a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880edb0 0x559478822aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb24493a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880edb0 0x559478822aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48108/0 (socket says 172.21.15.44:48108) 2023-09-10T00:40:17.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb24493a700 1 -- 172.21.15.44:0/3116389567 learned_addr learned my addr 172.21.15.44:0/3116389567 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:17.034 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb24493a700 1 -- 172.21.15.44:0/3116389567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559478810550 msgr2=0x5594788202e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.034 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb24493a700 1 --2- 172.21.15.44:0/3116389567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559478810550 0x5594788202e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.034 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb24493a700 1 -- 172.21.15.44:0/3116389567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880fbb0 msgr2=0x55947881fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.034 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb24493a700 1 --2- 172.21.15.44:0/3116389567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880fbb0 0x55947881fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.034 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb24493a700 1 -- 172.21.15.44:0/3116389567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594785b3200 con 0x55947880edb0 2023-09-10T00:40:17.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb24493a700 1 --2- 172.21.15.44:0/3116389567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880edb0 0x559478822aa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb234005ec0 tx=0x7fb234025740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf2aaa177796842 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb22b7fe700 1 -- 172.21.15.44:0/3116389567 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23400fe50 con 0x55947880edb0 2023-09-10T00:40:17.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb22b7fe700 1 -- 172.21.15.44:0/3116389567 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb23402b040 con 0x55947880edb0 2023-09-10T00:40:17.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.024+0000 7fb22b7fe700 1 -- 172.21.15.44:0/3116389567 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb234010b20 con 0x55947880edb0 2023-09-10T00:40:17.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.025+0000 7fb252d64b80 1 -- 172.21.15.44:0/3116389567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880edb0 msgr2=0x559478822aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.025+0000 7fb252d64b80 1 --2- 172.21.15.44:0/3116389567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880edb0 0x559478822aa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb234005ec0 tx=0x7fb234025740 comp rx=0 tx=0).stop 2023-09-10T00:40:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.025+0000 7fb252d64b80 1 -- 172.21.15.44:0/3116389567 shutdown_connections 2023-09-10T00:40:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.025+0000 7fb252d64b80 1 --2- 172.21.15.44:0/3116389567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559478810550 0x5594788202e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.025+0000 7fb252d64b80 1 --2- 172.21.15.44:0/3116389567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880edb0 0x559478822aa0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.025+0000 7fb252d64b80 1 --2- 172.21.15.44:0/3116389567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880fbb0 0x55947881fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.025+0000 7fb252d64b80 1 -- 172.21.15.44:0/3116389567 >> 172.21.15.44:0/3116389567 conn(0x559478783040 msgr2=0x559478817280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.025+0000 7fb252d64b80 1 -- 172.21.15.44:0/3116389567 shutdown_connections 2023-09-10T00:40:17.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.025+0000 7fb252d64b80 1 -- 172.21.15.44:0/3116389567 wait complete. 2023-09-10T00:40:17.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.025+0000 7fb252d64b80 1 -- start start 2023-09-10T00:40:17.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.025+0000 7fb252d64b80 1 Processor -- start 2023-09-10T00:40:17.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb252d64b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880edb0 0x55947881ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb252d64b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880fbb0 0x55947881b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb252d64b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559478810550 0x559478877b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb252d64b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559478878040 con 0x55947880fbb0 2023-09-10T00:40:17.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb252d64b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559478878180 con 0x55947880edb0 2023-09-10T00:40:17.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb252d64b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594788782c0 con 0x559478810550 2023-09-10T00:40:17.039 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb252d64b80 10 client.0 _ll_register_callbacks cb 0x559478648a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:40:17.039 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb23ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880fbb0 0x55947881b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.039 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb23ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880fbb0 0x55947881b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48116/0 (socket says 172.21.15.44:48116) 2023-09-10T00:40:17.039 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb23ffff700 1 -- 172.21.15.44:0/2673964534 learned_addr learned my addr 172.21.15.44:0/2673964534 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:17.039 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb24513b700 1 --2- 172.21.15.44:0/2673964534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559478810550 0x559478877b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.039 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb24493a700 1 --2- 172.21.15.44:0/2673964534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880edb0 0x55947881ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.040 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb23ffff700 1 -- 172.21.15.44:0/2673964534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559478810550 msgr2=0x559478877b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.040 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb23ffff700 1 --2- 172.21.15.44:0/2673964534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559478810550 0x559478877b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.040 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb23ffff700 1 -- 172.21.15.44:0/2673964534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880edb0 msgr2=0x55947881ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.040 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb23ffff700 1 --2- 172.21.15.44:0/2673964534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880edb0 0x55947881ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb23ffff700 1 -- 172.21.15.44:0/2673964534 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55947858cd90 con 0x55947880fbb0 2023-09-10T00:40:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb23ffff700 1 --2- 172.21.15.44:0/2673964534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880fbb0 0x55947881b240 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb22c005950 tx=0x7fb22c00bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee715fb4f0dc734 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.026+0000 7fb21ffff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:40:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.027+0000 7fb21ffff700 1 -- 172.21.15.44:0/2673964534 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c017070 con 0x55947880fbb0 2023-09-10T00:40:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.027+0000 7fb21ffff700 1 -- 172.21.15.44:0/2673964534 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9194+0+0 (secure 0 0 0) 0x7fb22c004830 con 0x55947880fbb0 2023-09-10T00:40:17.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.027+0000 7fb252d64b80 1 -- 172.21.15.44:0/2673964534 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb234020070 con 0x55947880fbb0 2023-09-10T00:40:17.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.027+0000 7fb21ffff700 1 -- 172.21.15.44:0/2673964534 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb22c0049f0 con 0x55947880fbb0 2023-09-10T00:40:17.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.027+0000 7fb252d64b80 10 client.15594 fetch_fsmap learned FSMap version 27 2023-09-10T00:40:17.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.027+0000 7fb252d64b80 1 -- 172.21.15.44:0/2673964534 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55947858b260 con 0x55947880fbb0 2023-09-10T00:40:17.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.027+0000 7fb21ffff700 1 -- 172.21.15.44:0/2673964534 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb234020070 con 0x55947880fbb0 2023-09-10T00:40:17.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.027+0000 7fb21ffff700 1 -- 172.21.15.44:0/2673964534 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb22c01c980 con 0x55947880fbb0 2023-09-10T00:40:17.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.027+0000 7fb252d64b80 10 client.15594 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:40:17.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.027+0000 7fb252d64b80 10 client.15594 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-10T00:40:17.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.027+0000 7fb252d64b80 20 client.15594 populate_metadata read hostname 'smithi044' 2023-09-10T00:40:17.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.027+0000 7fb252d64b80 1 --2- 172.21.15.44:0/2673964534 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55947887d5a0 0x559478879a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.027+0000 7fb252d64b80 4 client.15594 mds_command: new command op to 24572 tid=0[{"prefix": "status"}] 2023-09-10T00:40:17.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.027+0000 7fb252d64b80 1 -- 172.21.15.44:0/2673964534 --> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5594787598d0 con 0x55947887d5a0 2023-09-10T00:40:17.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.028+0000 7fb24493a700 1 --2- 172.21.15.44:0/2673964534 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55947887d5a0 0x559478879a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.028+0000 7fb21ffff700 10 client.15594 ms_handle_connect on v2:172.21.15.99:6836/566283638 2023-09-10T00:40:17.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.028+0000 7fb24493a700 1 --2- 172.21.15.44:0/2673964534 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55947887d5a0 0x559478879a80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34d033b84ea7164e server_cookie=a7fa83893fbbda18 in_seq=0 out_seq=0 2023-09-10T00:40:17.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.028+0000 7fb21ffff700 1 -- 172.21.15.44:0/2673964534 <== mds.2 v2:172.21.15.99:6836/566283638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5594787598d0 con 0x55947887d5a0 2023-09-10T00:40:17.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.028+0000 7fb21ffff700 10 client.15594 handle_command_reply: tid=0 2023-09-10T00:40:17.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.028+0000 7fb252d64b80 1 client.15594 shutdown 2023-09-10T00:40:17.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.028+0000 7fb252d64b80 1 -- 172.21.15.44:0/2673964534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880fbb0 msgr2=0x55947881b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.028+0000 7fb252d64b80 1 --2- 172.21.15.44:0/2673964534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880fbb0 0x55947881b240 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb22c005950 tx=0x7fb22c00bac0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.028+0000 7fb252d64b80 1 -- 172.21.15.44:0/2673964534 shutdown_connections 2023-09-10T00:40:17.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.028+0000 7fb252d64b80 1 --2- 172.21.15.44:0/2673964534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559478810550 0x559478877b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.028+0000 7fb252d64b80 1 --2- 172.21.15.44:0/2673964534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55947880edb0 0x55947881ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.028+0000 7fb252d64b80 1 --2- 172.21.15.44:0/2673964534 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55947887d5a0 0x559478879a80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.028+0000 7fb252d64b80 1 --2- 172.21.15.44:0/2673964534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55947880fbb0 0x55947881b240 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.028+0000 7fb252d64b80 1 -- 172.21.15.44:0/2673964534 >> 172.21.15.44:0/2673964534 conn(0x559478783040 msgr2=0x559478822aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.028+0000 7fb21ffff700 0 client.15594 ms_handle_reset on v2:172.21.15.99:6836/566283638 2023-09-10T00:40:17.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.029+0000 7fb252d64b80 1 -- 172.21.15.44:0/2673964534 shutdown_connections 2023-09-10T00:40:17.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.029+0000 7fb252d64b80 1 -- 172.21.15.44:0/2673964534 wait complete. 2023-09-10T00:40:17.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.029+0000 7fb252d64b80 20 client.15594 trim_cache size 0 max 16384 2023-09-10T00:40:17.047 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:40:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da", 2023-09-10T00:40:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 2, 2023-09-10T00:40:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24572, 2023-09-10T00:40:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2023-09-10T00:40:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2023-09-10T00:40:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2023-09-10T00:40:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 77.185379605999998, 2023-09-10T00:40:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2023-09-10T00:40:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 90, 2023-09-10T00:40:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 87, 2023-09-10T00:40:17.050 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 112.45045951900001 2023-09-10T00:40:17.050 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:40:17.051 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.029+0000 7fb246b9e700 1 -- 172.21.15.44:0/478134658 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb2300a0640 msgr2=0x7fb2300a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.051 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.029+0000 7fb246b9e700 1 --2- 172.21.15.44:0/478134658 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb2300a0640 0x7fb2300a2b00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb240101660 tx=0x7fb23400a3b0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.029+0000 7fb246b9e700 1 -- 172.21.15.44:0/478134658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2401021a0 msgr2=0x7fb24019d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.029+0000 7fb246b9e700 1 --2- 172.21.15.44:0/478134658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2401021a0 0x7fb24019d870 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb23800d010 tx=0x7fb2380094c0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.029+0000 7fb246b9e700 1 -- 172.21.15.44:0/478134658 shutdown_connections 2023-09-10T00:40:17.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.029+0000 7fb246b9e700 1 --2- 172.21.15.44:0/478134658 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb2300a0640 0x7fb2300a2b00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.052 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.029+0000 7fb246b9e700 1 --2- 172.21.15.44:0/478134658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2401021a0 0x7fb24019d870 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.029+0000 7fb246b9e700 1 --2- 172.21.15.44:0/478134658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb240100610 0x7fb2401962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.029+0000 7fb246b9e700 1 --2- 172.21.15.44:0/478134658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb240101800 0x7fb240196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.029+0000 7fb246b9e700 1 -- 172.21.15.44:0/478134658 >> 172.21.15.44:0/478134658 conn(0x7fb2400fbb80 msgr2=0x7fb240104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.029+0000 7fb246b9e700 1 -- 172.21.15.44:0/478134658 shutdown_connections 2023-09-10T00:40:17.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.029+0000 7fb246b9e700 1 -- 172.21.15.44:0/478134658 wait complete. 2023-09-10T00:40:17.054 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-09-10T00:40:17.055 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:16 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2156969952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:40:17.055 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:16 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2841530742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:40:17.055 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:16 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/616912973' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:17.055 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-09-10T00:40:17.055 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-09-10T00:40:17.056 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-09-10T00:40:17.056 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-09-10T00:40:17.056 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2023-09-10T00:40:17.056 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:16 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2023-09-10T00:40:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:16 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2156969952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:40:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:16 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2841530742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:40:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:16 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/616912973' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:17.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-09-10T00:40:17.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-09-10T00:40:17.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-09-10T00:40:17.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-09-10T00:40:17.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2023-09-10T00:40:17.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:16 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2023-09-10T00:40:17.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.182+0000 7fe2b1551700 1 -- 172.21.15.44:0/1047895150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2ac1010f0 msgr2=0x7fe2ac1079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.182+0000 7fe2b1551700 1 --2- 172.21.15.44:0/1047895150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2ac1010f0 0x7fe2ac1079a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400b0d0 tx=0x7fe2a400b3e0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.182+0000 7fe2b1551700 1 -- 172.21.15.44:0/1047895150 shutdown_connections 2023-09-10T00:40:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.182+0000 7fe2b1551700 1 --2- 172.21.15.44:0/1047895150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2ac1010f0 0x7fe2ac1079a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.182+0000 7fe2b1551700 1 --2- 172.21.15.44:0/1047895150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2ac0ff560 0x7fe2ac0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.182+0000 7fe2b1551700 1 --2- 172.21.15.44:0/1047895150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2ac100750 0x7fe2ac100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.183+0000 7fe2b1551700 1 -- 172.21.15.44:0/1047895150 >> 172.21.15.44:0/1047895150 conn(0x7fe2ac0faad0 msgr2=0x7fe2ac0fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.183+0000 7fe2b1551700 1 -- 172.21.15.44:0/1047895150 shutdown_connections 2023-09-10T00:40:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.183+0000 7fe2b1551700 1 -- 172.21.15.44:0/1047895150 wait complete. 2023-09-10T00:40:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.183+0000 7fe2b1551700 1 Processor -- start 2023-09-10T00:40:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.183+0000 7fe2b1551700 1 -- start start 2023-09-10T00:40:17.189 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2b1551700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2ac0ff560 0x7fe2ac1962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.189 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2b1551700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2ac100750 0x7fe2ac196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.189 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2b1551700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2ac1010f0 0x7fe2ac19d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2b1551700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac19de60 con 0x7fe2ac100750 2023-09-10T00:40:17.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2b1551700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac19dfa0 con 0x7fe2ac0ff560 2023-09-10T00:40:17.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2b1551700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac19e130 con 0x7fe2ac1010f0 2023-09-10T00:40:17.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2aa7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2ac100750 0x7fe2ac196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2aa7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2ac100750 0x7fe2ac196820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48130/0 (socket says 172.21.15.44:48130) 2023-09-10T00:40:17.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/1951484413 learned_addr learned my addr 172.21.15.44:0/1951484413 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:17.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2aaffd700 1 --2- 172.21.15.44:0/1951484413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2ac0ff560 0x7fe2ac1962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2ab7fe700 1 --2- 172.21.15.44:0/1951484413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2ac1010f0 0x7fe2ac19d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/1951484413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2ac1010f0 msgr2=0x7fe2ac19d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2aa7fc700 1 --2- 172.21.15.44:0/1951484413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2ac1010f0 0x7fe2ac19d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/1951484413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2ac0ff560 msgr2=0x7fe2ac1962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2aa7fc700 1 --2- 172.21.15.44:0/1951484413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2ac0ff560 0x7fe2ac1962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/1951484413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a4009d00 con 0x7fe2ac100750 2023-09-10T00:40:17.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2ab7fe700 1 --2- 172.21.15.44:0/1951484413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2ac1010f0 0x7fe2ac19d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:40:17.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2aa7fc700 1 --2- 172.21.15.44:0/1951484413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2ac100750 0x7fe2ac196820 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000d950 tx=0x7fe2a000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe293fff700 1 -- 172.21.15.44:0/1951484413 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0009b40 con 0x7fe2ac100750 2023-09-10T00:40:17.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe293fff700 1 -- 172.21.15.44:0/1951484413 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2a0009ca0 con 0x7fe2ac100750 2023-09-10T00:40:17.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2b1551700 1 -- 172.21.15.44:0/1951484413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ac19e410 con 0x7fe2ac100750 2023-09-10T00:40:17.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe293fff700 1 -- 172.21.15.44:0/1951484413 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a000f620 con 0x7fe2ac100750 2023-09-10T00:40:17.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.184+0000 7fe2b1551700 1 -- 172.21.15.44:0/1951484413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ac19e920 con 0x7fe2ac100750 2023-09-10T00:40:17.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.186+0000 7fe293fff700 1 -- 172.21.15.44:0/1951484413 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fe2a0010460 con 0x7fe2ac100750 2023-09-10T00:40:17.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.187+0000 7fe293fff700 1 --2- 172.21.15.44:0/1951484413 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe2940a0640 0x7fe2940a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.187+0000 7fe293fff700 1 -- 172.21.15.44:0/1951484413 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9194+0+0 (secure 0 0 0) 0x7fe2a00cc300 con 0x7fe2ac100750 2023-09-10T00:40:17.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.187+0000 7fe2aaffd700 1 --2- 172.21.15.44:0/1951484413 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe2940a0640 0x7fe2940a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.188+0000 7fe2aaffd700 1 --2- 172.21.15.44:0/1951484413 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe2940a0640 0x7fe2940a2b00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe29c005950 tx=0x7fe29c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.188+0000 7fe2bd717b80 1 Processor -- start 2023-09-10T00:40:17.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.188+0000 7fe2bd717b80 1 -- start start 2023-09-10T00:40:17.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.188+0000 7fe2bd717b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd63bb0 0x55895bd63fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.188+0000 7fe2bd717b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd64550 0x55895bd627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.188+0000 7fe2bd717b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd62db0 0x55895bd63260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.188+0000 7fe2bd717b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55895bcbb6e0 con 0x55895bd64550 2023-09-10T00:40:17.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.188+0000 7fe2bd717b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55895bcb9000 con 0x55895bd63bb0 2023-09-10T00:40:17.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.188+0000 7fe2bd717b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55895bcb48f0 con 0x55895bd62db0 2023-09-10T00:40:17.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.188+0000 7fe2aa7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd64550 0x55895bd627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.188+0000 7fe2aa7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd64550 0x55895bd627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48138/0 (socket says 172.21.15.44:48138) 2023-09-10T00:40:17.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.188+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/1999124097 learned_addr learned my addr 172.21.15.44:0/1999124097 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:17.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.188+0000 7fe2aaffd700 1 --2- 172.21.15.44:0/1999124097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd63bb0 0x55895bd63fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.188+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/1999124097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd62db0 msgr2=0x55895bd63260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.189+0000 7fe2aa7fc700 1 --2- 172.21.15.44:0/1999124097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd62db0 0x55895bd63260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.189+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/1999124097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd63bb0 msgr2=0x55895bd63fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.189+0000 7fe2aa7fc700 1 --2- 172.21.15.44:0/1999124097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd63bb0 0x55895bd63fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.189+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/1999124097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55895bae0d90 con 0x55895bd64550 2023-09-10T00:40:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.189+0000 7fe2aa7fc700 1 --2- 172.21.15.44:0/1999124097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd64550 0x55895bd627b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00961a0 tx=0x7fe2a0095b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e53170be8f7fe3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.189+0000 7fe2917fa700 1 -- 172.21.15.44:0/1999124097 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00cc170 con 0x55895bd64550 2023-09-10T00:40:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.189+0000 7fe2917fa700 1 -- 172.21.15.44:0/1999124097 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2a0099e30 con 0x55895bd64550 2023-09-10T00:40:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.189+0000 7fe2917fa700 1 -- 172.21.15.44:0/1999124097 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0010710 con 0x55895bd64550 2023-09-10T00:40:17.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.189+0000 7fe2bd717b80 1 -- 172.21.15.44:0/1999124097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd64550 msgr2=0x55895bd627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.189+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/1999124097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd64550 0x55895bd627b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00961a0 tx=0x7fe2a0095b70 comp rx=0 tx=0).stop 2023-09-10T00:40:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2bd717b80 1 -- 172.21.15.44:0/1999124097 shutdown_connections 2023-09-10T00:40:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/1999124097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd62db0 0x55895bd63260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/1999124097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd63bb0 0x55895bd63fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/1999124097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd64550 0x55895bd627b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2bd717b80 1 -- 172.21.15.44:0/1999124097 >> 172.21.15.44:0/1999124097 conn(0x55895bcd7040 msgr2=0x55895bd5d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2bd717b80 1 -- 172.21.15.44:0/1999124097 shutdown_connections 2023-09-10T00:40:17.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2bd717b80 1 -- 172.21.15.44:0/1999124097 wait complete. 2023-09-10T00:40:17.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2bd717b80 1 -- start start 2023-09-10T00:40:17.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2bd717b80 1 Processor -- start 2023-09-10T00:40:17.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2bd717b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd62db0 0x55895bd6c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2bd717b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd63bb0 0x55895bd6ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2bd717b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd64550 0x55895bd73ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2bd717b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55895bc52b20 con 0x55895bd62db0 2023-09-10T00:40:17.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2bd717b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55895bc52820 con 0x55895bd64550 2023-09-10T00:40:17.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2bd717b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55895bb368d0 con 0x55895bd63bb0 2023-09-10T00:40:17.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2bd717b80 10 client.0 _ll_register_callbacks cb 0x55895bb9ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:40:17.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2aaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd62db0 0x55895bd6c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2aaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd62db0 0x55895bd6c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48142/0 (socket says 172.21.15.44:48142) 2023-09-10T00:40:17.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.190+0000 7fe2aaffd700 1 -- 172.21.15.44:0/3577567188 learned_addr learned my addr 172.21.15.44:0/3577567188 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:17.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.191+0000 7fe2ab7fe700 1 --2- 172.21.15.44:0/3577567188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd64550 0x55895bd73ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.191+0000 7fe2aa7fc700 1 --2- 172.21.15.44:0/3577567188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd63bb0 0x55895bd6ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.191+0000 7fe2aaffd700 1 -- 172.21.15.44:0/3577567188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd63bb0 msgr2=0x55895bd6ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.191+0000 7fe2aaffd700 1 --2- 172.21.15.44:0/3577567188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd63bb0 0x55895bd6ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.191+0000 7fe2aaffd700 1 -- 172.21.15.44:0/3577567188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd64550 msgr2=0x55895bd73ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.191+0000 7fe2aaffd700 1 --2- 172.21.15.44:0/3577567188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd64550 0x55895bd73ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.191+0000 7fe2aaffd700 1 -- 172.21.15.44:0/3577567188 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ac19e920 con 0x55895bd62db0 2023-09-10T00:40:17.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.191+0000 7fe2aaffd700 1 --2- 172.21.15.44:0/3577567188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd62db0 0x55895bd6c540 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00fde0 tx=0x7fe29c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81ba5eadde90790 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.191+0000 7fe28eff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:40:17.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.191+0000 7fe28eff5700 1 -- 172.21.15.44:0/3577567188 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c00b910 con 0x55895bd62db0 2023-09-10T00:40:17.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.191+0000 7fe28eff5700 1 -- 172.21.15.44:0/3577567188 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9194+0+0 (secure 0 0 0) 0x7fe29c01e420 con 0x55895bd62db0 2023-09-10T00:40:17.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.191+0000 7fe2bd717b80 1 -- 172.21.15.44:0/3577567188 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55895bc0c4a0 con 0x55895bd62db0 2023-09-10T00:40:17.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.191+0000 7fe28eff5700 1 -- 172.21.15.44:0/3577567188 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe29c011e60 con 0x55895bd62db0 2023-09-10T00:40:17.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.191+0000 7fe2bd717b80 10 client.15606 fetch_fsmap learned FSMap version 27 2023-09-10T00:40:17.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.191+0000 7fe2bd717b80 1 -- 172.21.15.44:0/3577567188 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55895bac9c10 con 0x55895bd62db0 2023-09-10T00:40:17.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.192+0000 7fe28eff5700 1 -- 172.21.15.44:0/3577567188 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55895bc0c4a0 con 0x55895bd62db0 2023-09-10T00:40:17.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.192+0000 7fe28eff5700 1 -- 172.21.15.44:0/3577567188 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe29c024020 con 0x55895bd62db0 2023-09-10T00:40:17.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.192+0000 7fe2bd717b80 10 client.15606 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:40:17.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.192+0000 7fe2bd717b80 10 client.15606 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-10T00:40:17.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.192+0000 7fe2bd717b80 20 client.15606 populate_metadata read hostname 'smithi044' 2023-09-10T00:40:17.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.192+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/3577567188 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55895bd77cc0 0x55895bd76c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.192+0000 7fe2bd717b80 4 client.15606 mds_command: new command op to 24574 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:40:17.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.192+0000 7fe2bd717b80 1 -- 172.21.15.44:0/3577567188 --> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55895bcc9850 con 0x55895bd77cc0 2023-09-10T00:40:17.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.192+0000 7fe2aa7fc700 1 --2- 172.21.15.44:0/3577567188 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55895bd77cc0 0x55895bd76c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.193+0000 7fe2aa7fc700 1 --2- 172.21.15.44:0/3577567188 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55895bd77cc0 0x55895bd76c70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fac760db0435e3ad server_cookie=766f186a159d0e5e in_seq=0 out_seq=0 2023-09-10T00:40:17.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.193+0000 7fe28eff5700 10 client.15606 ms_handle_connect on v2:172.21.15.131:6838/3222652693 2023-09-10T00:40:17.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.193+0000 7fe28eff5700 1 -- 172.21.15.44:0/3577567188 <== mds.1 v2:172.21.15.131:6838/3222652693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55895bcc9850 con 0x55895bd77cc0 2023-09-10T00:40:17.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.193+0000 7fe28eff5700 10 client.15606 handle_command_reply: tid=0 2023-09-10T00:40:17.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.193+0000 7fe2bd717b80 1 client.15606 shutdown 2023-09-10T00:40:17.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.193+0000 7fe2bd717b80 1 -- 172.21.15.44:0/3577567188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd62db0 msgr2=0x55895bd6c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.194+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/3577567188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd62db0 0x55895bd6c540 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00fde0 tx=0x7fe29c004750 comp rx=0 tx=0).stop 2023-09-10T00:40:17.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.194+0000 7fe2bd717b80 1 -- 172.21.15.44:0/3577567188 shutdown_connections 2023-09-10T00:40:17.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.194+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/3577567188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd64550 0x55895bd73ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.194+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/3577567188 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55895bd77cc0 0x55895bd76c70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.194+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/3577567188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd62db0 0x55895bd6c540 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.194+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/3577567188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd63bb0 0x55895bd6ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.194+0000 7fe28eff5700 0 client.15606 ms_handle_reset on v2:172.21.15.131:6838/3222652693 2023-09-10T00:40:17.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.194+0000 7fe2bd717b80 1 -- 172.21.15.44:0/3577567188 >> 172.21.15.44:0/3577567188 conn(0x55895bcd7040 msgr2=0x55895bd5cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.194+0000 7fe2bd717b80 1 -- 172.21.15.44:0/3577567188 shutdown_connections 2023-09-10T00:40:17.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.194+0000 7fe2bd717b80 1 -- 172.21.15.44:0/3577567188 wait complete. 2023-09-10T00:40:17.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.194+0000 7fe2bd717b80 20 client.15606 trim_cache size 0 max 16384 2023-09-10T00:40:17.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.211+0000 7fe2bd717b80 1 Processor -- start 2023-09-10T00:40:17.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.211+0000 7fe2bd717b80 1 -- start start 2023-09-10T00:40:17.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2bd717b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd62db0 0x55895bd6a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2bd717b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd63bb0 0x55895bd73a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2bd717b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd64550 0x55895bd6b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2bd717b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55895bcc9850 con 0x55895bd63bb0 2023-09-10T00:40:17.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2bd717b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55895bb42de0 con 0x55895bd64550 2023-09-10T00:40:17.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2bd717b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55895bc7df70 con 0x55895bd62db0 2023-09-10T00:40:17.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2aa7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd63bb0 0x55895bd73a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2aa7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd63bb0 0x55895bd73a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48152/0 (socket says 172.21.15.44:48152) 2023-09-10T00:40:17.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/3068838270 learned_addr learned my addr 172.21.15.44:0/3068838270 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:17.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2aaffd700 1 --2- 172.21.15.44:0/3068838270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd62db0 0x55895bd6a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2ab7fe700 1 --2- 172.21.15.44:0/3068838270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd64550 0x55895bd6b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/3068838270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd62db0 msgr2=0x55895bd6a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2aa7fc700 1 --2- 172.21.15.44:0/3068838270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd62db0 0x55895bd6a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/3068838270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd64550 msgr2=0x55895bd6b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2aa7fc700 1 --2- 172.21.15.44:0/3068838270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd64550 0x55895bd6b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/3068838270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55895bb07200 con 0x55895bd63bb0 2023-09-10T00:40:17.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2ab7fe700 1 --2- 172.21.15.44:0/3068838270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd64550 0x55895bd6b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:40:17.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe2aa7fc700 1 --2- 172.21.15.44:0/3068838270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd63bb0 0x55895bd73a80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0095be0 tx=0x7fe2a0084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6ee78dcc8e62fb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe291ffb700 1 -- 172.21.15.44:0/3068838270 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0099730 con 0x55895bd63bb0 2023-09-10T00:40:17.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe291ffb700 1 -- 172.21.15.44:0/3068838270 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2a0097ad0 con 0x55895bd63bb0 2023-09-10T00:40:17.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.212+0000 7fe291ffb700 1 -- 172.21.15.44:0/3068838270 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0099990 con 0x55895bd63bb0 2023-09-10T00:40:17.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.213+0000 7fe2bd717b80 1 -- 172.21.15.44:0/3068838270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd63bb0 msgr2=0x55895bd73a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.213+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/3068838270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd63bb0 0x55895bd73a80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0095be0 tx=0x7fe2a0084040 comp rx=0 tx=0).stop 2023-09-10T00:40:17.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.213+0000 7fe2bd717b80 1 -- 172.21.15.44:0/3068838270 shutdown_connections 2023-09-10T00:40:17.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.213+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/3068838270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd64550 0x55895bd6b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.213+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/3068838270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd62db0 0x55895bd6a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.213+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/3068838270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd63bb0 0x55895bd73a80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.213+0000 7fe2bd717b80 1 -- 172.21.15.44:0/3068838270 >> 172.21.15.44:0/3068838270 conn(0x55895bcd7040 msgr2=0x55895bd76c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.213+0000 7fe2bd717b80 1 -- 172.21.15.44:0/3068838270 shutdown_connections 2023-09-10T00:40:17.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.213+0000 7fe2bd717b80 1 -- 172.21.15.44:0/3068838270 wait complete. 2023-09-10T00:40:17.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.213+0000 7fe2bd717b80 1 -- start start 2023-09-10T00:40:17.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.213+0000 7fe2bd717b80 1 Processor -- start 2023-09-10T00:40:17.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.213+0000 7fe2bd717b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd62db0 0x55895bd6e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2bd717b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd63bb0 0x55895bd6eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.227 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2bd717b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd64550 0x55895bdcbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.227 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2bd717b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55895bd73ae0 con 0x55895bd62db0 2023-09-10T00:40:17.227 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2bd717b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55895bd73c50 con 0x55895bd64550 2023-09-10T00:40:17.227 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2bd717b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55895bd73e10 con 0x55895bd63bb0 2023-09-10T00:40:17.227 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2bd717b80 10 client.0 _ll_register_callbacks cb 0x7fe29c00f9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:40:17.228 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2aaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd62db0 0x55895bd6e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.228 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2aaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd62db0 0x55895bd6e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48156/0 (socket says 172.21.15.44:48156) 2023-09-10T00:40:17.228 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2aaffd700 1 -- 172.21.15.44:0/4243507673 learned_addr learned my addr 172.21.15.44:0/4243507673 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:17.228 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2aa7fc700 1 --2- 172.21.15.44:0/4243507673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd63bb0 0x55895bd6eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.228 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2ab7fe700 1 --2- 172.21.15.44:0/4243507673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd64550 0x55895bdcbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.228 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2aaffd700 1 -- 172.21.15.44:0/4243507673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd63bb0 msgr2=0x55895bd6eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.229 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2aaffd700 1 --2- 172.21.15.44:0/4243507673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd63bb0 0x55895bd6eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.229 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2aaffd700 1 -- 172.21.15.44:0/4243507673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd64550 msgr2=0x55895bdcbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.229 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2aaffd700 1 --2- 172.21.15.44:0/4243507673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd64550 0x55895bdcbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.229 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2aaffd700 1 -- 172.21.15.44:0/4243507673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55895bae0d90 con 0x55895bd62db0 2023-09-10T00:40:17.229 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2aa7fc700 1 --2- 172.21.15.44:0/4243507673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd63bb0 0x55895bd6eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:17.230 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2aaffd700 1 --2- 172.21.15.44:0/4243507673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd62db0 0x55895bd6e570 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe29c005a80 tx=0x7fe29c010a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d5216ed3b98c50 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.230 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe28eff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:40:17.230 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe28eff5700 1 -- 172.21.15.44:0/4243507673 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c005040 con 0x55895bd62db0 2023-09-10T00:40:17.230 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe28eff5700 1 -- 172.21.15.44:0/4243507673 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9194+0+0 (secure 0 0 0) 0x7fe29c0051a0 con 0x55895bd62db0 2023-09-10T00:40:17.230 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe2bd717b80 1 -- 172.21.15.44:0/4243507673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2a0099e30 con 0x55895bd62db0 2023-09-10T00:40:17.231 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.214+0000 7fe28eff5700 1 -- 172.21.15.44:0/4243507673 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe29c010cd0 con 0x55895bd62db0 2023-09-10T00:40:17.231 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.215+0000 7fe2bd717b80 10 client.15612 fetch_fsmap learned FSMap version 27 2023-09-10T00:40:17.231 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.215+0000 7fe2bd717b80 1 -- 172.21.15.44:0/4243507673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55895badf260 con 0x55895bd62db0 2023-09-10T00:40:17.231 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.215+0000 7fe28eff5700 1 -- 172.21.15.44:0/4243507673 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2a0099e30 con 0x55895bd62db0 2023-09-10T00:40:17.231 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.215+0000 7fe28eff5700 1 -- 172.21.15.44:0/4243507673 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe29c011440 con 0x55895bd62db0 2023-09-10T00:40:17.231 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.215+0000 7fe2bd717b80 10 client.15612 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:40:17.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.215+0000 7fe2bd717b80 10 client.15612 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-10T00:40:17.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.215+0000 7fe2bd717b80 20 client.15612 populate_metadata read hostname 'smithi044' 2023-09-10T00:40:17.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.215+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/4243507673 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55895bdd11b0 0x55895bdcd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.215+0000 7fe2bd717b80 4 client.15612 mds_command: new command op to 24574 tid=0[{"prefix": "status"}] 2023-09-10T00:40:17.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.215+0000 7fe2bd717b80 1 -- 172.21.15.44:0/4243507673 --> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55895bcad8d0 con 0x55895bdd11b0 2023-09-10T00:40:17.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.215+0000 7fe2aa7fc700 1 --2- 172.21.15.44:0/4243507673 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55895bdd11b0 0x55895bdcd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe28eff5700 10 client.15612 ms_handle_connect on v2:172.21.15.131:6838/3222652693 2023-09-10T00:40:17.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe2aa7fc700 1 --2- 172.21.15.44:0/4243507673 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55895bdd11b0 0x55895bdcd630 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=476b7a71a4bac92a server_cookie=c7a15f0748496a48 in_seq=0 out_seq=0 2023-09-10T00:40:17.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe28eff5700 1 -- 172.21.15.44:0/4243507673 <== mds.1 v2:172.21.15.131:6838/3222652693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55895bcad8d0 con 0x55895bdd11b0 2023-09-10T00:40:17.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe28eff5700 10 client.15612 handle_command_reply: tid=0 2023-09-10T00:40:17.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe2bd717b80 1 client.15612 shutdown 2023-09-10T00:40:17.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe2bd717b80 1 -- 172.21.15.44:0/4243507673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd62db0 msgr2=0x55895bd6e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/4243507673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd62db0 0x55895bd6e570 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe29c005a80 tx=0x7fe29c010a30 comp rx=0 tx=0).stop 2023-09-10T00:40:17.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe2bd717b80 1 -- 172.21.15.44:0/4243507673 shutdown_connections 2023-09-10T00:40:17.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/4243507673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895bd64550 0x55895bdcbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/4243507673 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55895bdd11b0 0x55895bdcd630 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/4243507673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55895bd62db0 0x55895bd6e570 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe2bd717b80 1 --2- 172.21.15.44:0/4243507673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55895bd63bb0 0x55895bd6eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe28eff5700 0 client.15612 ms_handle_reset on v2:172.21.15.131:6838/3222652693 2023-09-10T00:40:17.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe2bd717b80 1 -- 172.21.15.44:0/4243507673 >> 172.21.15.44:0/4243507673 conn(0x55895bcd7040 msgr2=0x55895bd6a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe2bd717b80 1 -- 172.21.15.44:0/4243507673 shutdown_connections 2023-09-10T00:40:17.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe2bd717b80 1 -- 172.21.15.44:0/4243507673 wait complete. 2023-09-10T00:40:17.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.216+0000 7fe2bd717b80 20 client.15612 trim_cache size 0 max 16384 2023-09-10T00:40:17.236 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:40:17.236 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da", 2023-09-10T00:40:17.236 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 1, 2023-09-10T00:40:17.237 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24574, 2023-09-10T00:40:17.237 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2023-09-10T00:40:17.237 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2023-09-10T00:40:17.237 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2023-09-10T00:40:17.237 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 78.384328445999998, 2023-09-10T00:40:17.238 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2023-09-10T00:40:17.238 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 90, 2023-09-10T00:40:17.238 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 87, 2023-09-10T00:40:17.238 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 110.453992367 2023-09-10T00:40:17.238 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:40:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.217+0000 7fe2b1551700 1 -- 172.21.15.44:0/1951484413 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe2940a0640 msgr2=0x7fe2940a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.217+0000 7fe2b1551700 1 --2- 172.21.15.44:0/1951484413 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe2940a0640 0x7fe2940a2b00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe29c005950 tx=0x7fe29c0058e0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.217+0000 7fe2b1551700 1 -- 172.21.15.44:0/1951484413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2ac100750 msgr2=0x7fe2ac196820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.217+0000 7fe2b1551700 1 --2- 172.21.15.44:0/1951484413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2ac100750 0x7fe2ac196820 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000d950 tx=0x7fe2a000dd10 comp rx=0 tx=0).stop 2023-09-10T00:40:17.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.217+0000 7fe2b1551700 1 -- 172.21.15.44:0/1951484413 shutdown_connections 2023-09-10T00:40:17.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.217+0000 7fe2b1551700 1 --2- 172.21.15.44:0/1951484413 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe2940a0640 0x7fe2940a2b00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.217+0000 7fe2b1551700 1 --2- 172.21.15.44:0/1951484413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2ac1010f0 0x7fe2ac19d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.217+0000 7fe2b1551700 1 --2- 172.21.15.44:0/1951484413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2ac0ff560 0x7fe2ac1962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.217+0000 7fe2b1551700 1 --2- 172.21.15.44:0/1951484413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2ac100750 0x7fe2ac196820 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.217+0000 7fe2b1551700 1 -- 172.21.15.44:0/1951484413 >> 172.21.15.44:0/1951484413 conn(0x7fe2ac0faad0 msgr2=0x7fe2ac0fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.217+0000 7fe2b1551700 1 -- 172.21.15.44:0/1951484413 shutdown_connections 2023-09-10T00:40:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.217+0000 7fe2b1551700 1 -- 172.21.15.44:0/1951484413 wait complete. 2023-09-10T00:40:17.242 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-10T00:40:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.369+0000 7f167bf17700 1 -- 172.21.15.44:0/2979240289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16741039d0 msgr2=0x7f1674105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.369+0000 7f167bf17700 1 --2- 172.21.15.44:0/2979240289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16741039d0 0x7f1674105dc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1664009990 tx=0x7f1664009ca0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.369+0000 7f167bf17700 1 -- 172.21.15.44:0/2979240289 shutdown_connections 2023-09-10T00:40:17.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.369+0000 7f167bf17700 1 --2- 172.21.15.44:0/2979240289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1674106300 0x7f1674108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.369+0000 7f167bf17700 1 --2- 172.21.15.44:0/2979240289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16741010a0 0x7f1674103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.369+0000 7f167bf17700 1 --2- 172.21.15.44:0/2979240289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16741039d0 0x7f1674105dc0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.369+0000 7f167bf17700 1 -- 172.21.15.44:0/2979240289 >> 172.21.15.44:0/2979240289 conn(0x7f16740faa90 msgr2=0x7f16740fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.369+0000 7f167bf17700 1 -- 172.21.15.44:0/2979240289 shutdown_connections 2023-09-10T00:40:17.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.369+0000 7f167bf17700 1 -- 172.21.15.44:0/2979240289 wait complete. 2023-09-10T00:40:17.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.369+0000 7f167bf17700 1 Processor -- start 2023-09-10T00:40:17.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f167bf17700 1 -- start start 2023-09-10T00:40:17.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f167bf17700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16741010a0 0x7f1674100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f167bf17700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16741039d0 0x7f16740ff050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f167bf17700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1674106300 0x7f16740ff590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f167bf17700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16740ffd30 con 0x7f1674106300 2023-09-10T00:40:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f167bf17700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16740ffea0 con 0x7f16741010a0 2023-09-10T00:40:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f167bf17700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16741a0270 con 0x7f16741039d0 2023-09-10T00:40:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f167a4b4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1674106300 0x7f16740ff590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f167a4b4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1674106300 0x7f16740ff590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48174/0 (socket says 172.21.15.44:48174) 2023-09-10T00:40:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f167a4b4700 1 -- 172.21.15.44:0/2431357532 learned_addr learned my addr 172.21.15.44:0/2431357532 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:17.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f16794b2700 1 --2- 172.21.15.44:0/2431357532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16741039d0 0x7f16740ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f1679cb3700 1 --2- 172.21.15.44:0/2431357532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16741010a0 0x7f1674100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f167a4b4700 1 -- 172.21.15.44:0/2431357532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16741039d0 msgr2=0x7f16740ff050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f167a4b4700 1 --2- 172.21.15.44:0/2431357532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16741039d0 0x7f16740ff050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f167a4b4700 1 -- 172.21.15.44:0/2431357532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16741010a0 msgr2=0x7f1674100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f167a4b4700 1 --2- 172.21.15.44:0/2431357532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16741010a0 0x7f1674100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f167a4b4700 1 -- 172.21.15.44:0/2431357532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16640095f0 con 0x7f1674106300 2023-09-10T00:40:17.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f167a4b4700 1 --2- 172.21.15.44:0/2431357532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1674106300 0x7f16740ff590 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f166c007ee0 tx=0x7f166c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.370+0000 7f166affd700 1 -- 172.21.15.44:0/2431357532 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c017070 con 0x7f1674106300 2023-09-10T00:40:17.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.371+0000 7f166affd700 1 -- 172.21.15.44:0/2431357532 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f166c00f040 con 0x7f1674106300 2023-09-10T00:40:17.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.371+0000 7f167bf17700 1 -- 172.21.15.44:0/2431357532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16741a0550 con 0x7f1674106300 2023-09-10T00:40:17.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.371+0000 7f166affd700 1 -- 172.21.15.44:0/2431357532 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c005570 con 0x7f1674106300 2023-09-10T00:40:17.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.371+0000 7f167bf17700 1 -- 172.21.15.44:0/2431357532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16741a0a10 con 0x7f1674106300 2023-09-10T00:40:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.373+0000 7f166affd700 1 -- 172.21.15.44:0/2431357532 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f166c00db20 con 0x7f1674106300 2023-09-10T00:40:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.373+0000 7f166affd700 1 --2- 172.21.15.44:0/2431357532 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16600a0640 0x7f16600a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.373+0000 7f1679cb3700 1 --2- 172.21.15.44:0/2431357532 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16600a0640 0x7f16600a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.373+0000 7f166affd700 1 -- 172.21.15.44:0/2431357532 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f166c0cb700 con 0x7f1674106300 2023-09-10T00:40:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.374+0000 7f1679cb3700 1 --2- 172.21.15.44:0/2431357532 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16600a0640 0x7f16600a2b00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1670005950 tx=0x7f167000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.374+0000 7f16880ddb80 1 Processor -- start 2023-09-10T00:40:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.374+0000 7f16880ddb80 1 -- start start 2023-09-10T00:40:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f16880ddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cebb0 0x558f976cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f16880ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cf550 0x558f976cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f16880ddb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cddb0 0x558f976ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f16880ddb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f976266e0 con 0x558f976cddb0 2023-09-10T00:40:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f16880ddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f97624000 con 0x558f976cebb0 2023-09-10T00:40:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f16880ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9761f8f0 con 0x558f976cf550 2023-09-10T00:40:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f167a4b4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cddb0 0x558f976ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f167a4b4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cddb0 0x558f976ce260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48182/0 (socket says 172.21.15.44:48182) 2023-09-10T00:40:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f167a4b4700 1 -- 172.21.15.44:0/2918095392 learned_addr learned my addr 172.21.15.44:0/2918095392 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f16794b2700 1 --2- 172.21.15.44:0/2918095392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cf550 0x558f976cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f1679cb3700 1 --2- 172.21.15.44:0/2918095392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cebb0 0x558f976cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f167a4b4700 1 -- 172.21.15.44:0/2918095392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cf550 msgr2=0x558f976cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f167a4b4700 1 --2- 172.21.15.44:0/2918095392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cf550 0x558f976cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f167a4b4700 1 -- 172.21.15.44:0/2918095392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cebb0 msgr2=0x558f976cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f167a4b4700 1 --2- 172.21.15.44:0/2918095392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cebb0 0x558f976cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f167a4b4700 1 -- 172.21.15.44:0/2918095392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f9744bd90 con 0x558f976cddb0 2023-09-10T00:40:17.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f1679cb3700 1 --2- 172.21.15.44:0/2918095392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cebb0 0x558f976cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:17.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f167a4b4700 1 --2- 172.21.15.44:0/2918095392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cddb0 0x558f976ce260 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f166c098ac0 tx=0x7f166c098760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f1fb474389f64b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f165ffff700 1 -- 172.21.15.44:0/2918095392 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c00ddd0 con 0x558f976cddb0 2023-09-10T00:40:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f165ffff700 1 -- 172.21.15.44:0/2918095392 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f166c099230 con 0x558f976cddb0 2023-09-10T00:40:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.375+0000 7f165ffff700 1 -- 172.21.15.44:0/2918095392 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c0b80f0 con 0x558f976cddb0 2023-09-10T00:40:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.376+0000 7f16880ddb80 1 -- 172.21.15.44:0/2918095392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cddb0 msgr2=0x558f976ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.376+0000 7f16880ddb80 1 --2- 172.21.15.44:0/2918095392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cddb0 0x558f976ce260 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f166c098ac0 tx=0x7f166c098760 comp rx=0 tx=0).stop 2023-09-10T00:40:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.376+0000 7f16880ddb80 1 -- 172.21.15.44:0/2918095392 shutdown_connections 2023-09-10T00:40:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.376+0000 7f16880ddb80 1 --2- 172.21.15.44:0/2918095392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cddb0 0x558f976ce260 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.376+0000 7f16880ddb80 1 --2- 172.21.15.44:0/2918095392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cebb0 0x558f976cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.376+0000 7f16880ddb80 1 --2- 172.21.15.44:0/2918095392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cf550 0x558f976cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.376+0000 7f16880ddb80 1 -- 172.21.15.44:0/2918095392 >> 172.21.15.44:0/2918095392 conn(0x558f97642040 msgr2=0x558f976c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.376+0000 7f16880ddb80 1 -- 172.21.15.44:0/2918095392 shutdown_connections 2023-09-10T00:40:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.376+0000 7f16880ddb80 1 -- 172.21.15.44:0/2918095392 wait complete. 2023-09-10T00:40:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.377+0000 7f16880ddb80 1 -- start start 2023-09-10T00:40:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.377+0000 7f16880ddb80 1 Processor -- start 2023-09-10T00:40:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.377+0000 7f16880ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cddb0 0x558f976d75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.377+0000 7f16880ddb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cebb0 0x558f976d7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.377+0000 7f16880ddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cf550 0x558f976deb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.377+0000 7f16880ddb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f975bdb20 con 0x558f976cebb0 2023-09-10T00:40:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.377+0000 7f16880ddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f975bd820 con 0x558f976cf550 2023-09-10T00:40:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.377+0000 7f16880ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f974a18d0 con 0x558f976cddb0 2023-09-10T00:40:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.377+0000 7f16880ddb80 10 client.0 _ll_register_callbacks cb 0x558f97507a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:40:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.377+0000 7f16794b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cebb0 0x558f976d7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.377+0000 7f16794b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cebb0 0x558f976d7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48186/0 (socket says 172.21.15.44:48186) 2023-09-10T00:40:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.377+0000 7f16794b2700 1 -- 172.21.15.44:0/3294072313 learned_addr learned my addr 172.21.15.44:0/3294072313 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f1679cb3700 1 --2- 172.21.15.44:0/3294072313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cddb0 0x558f976d75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f167a4b4700 1 --2- 172.21.15.44:0/3294072313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cf550 0x558f976deb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f16794b2700 1 -- 172.21.15.44:0/3294072313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cddb0 msgr2=0x558f976d75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f16794b2700 1 --2- 172.21.15.44:0/3294072313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cddb0 0x558f976d75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f16794b2700 1 -- 172.21.15.44:0/3294072313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cf550 msgr2=0x558f976deb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f16794b2700 1 --2- 172.21.15.44:0/3294072313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cf550 0x558f976deb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f16794b2700 1 -- 172.21.15.44:0/3294072313 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16741a0a10 con 0x558f976cebb0 2023-09-10T00:40:17.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f1679cb3700 1 --2- 172.21.15.44:0/3294072313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cddb0 0x558f976d75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:17.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f167a4b4700 1 --2- 172.21.15.44:0/3294072313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cf550 0x558f976deb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:40:17.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f16794b2700 1 --2- 172.21.15.44:0/3294072313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cebb0 0x558f976d7b20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f16640098e0 tx=0x7f16640050f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8184a64252e3c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f165d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:40:17.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f165d7fa700 1 -- 172.21.15.44:0/3294072313 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166401d070 con 0x558f976cebb0 2023-09-10T00:40:17.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f16880ddb80 1 -- 172.21.15.44:0/3294072313 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f975774a0 con 0x558f976cebb0 2023-09-10T00:40:17.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f165d7fa700 1 -- 172.21.15.44:0/3294072313 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f1664004500 con 0x558f976cebb0 2023-09-10T00:40:17.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f165d7fa700 1 -- 172.21.15.44:0/3294072313 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16640046c0 con 0x558f976cebb0 2023-09-10T00:40:17.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f16880ddb80 10 client.15624 fetch_fsmap learned FSMap version 27 2023-09-10T00:40:17.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f16880ddb80 1 -- 172.21.15.44:0/3294072313 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558f97434c10 con 0x558f976cebb0 2023-09-10T00:40:17.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.378+0000 7f165d7fa700 1 -- 172.21.15.44:0/3294072313 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558f975774a0 con 0x558f976cebb0 2023-09-10T00:40:17.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.379+0000 7f165d7fa700 1 -- 172.21.15.44:0/3294072313 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1664003ab0 con 0x558f976cebb0 2023-09-10T00:40:17.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.379+0000 7f16880ddb80 10 client.15624 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:40:17.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.379+0000 7f16880ddb80 10 client.15624 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-10T00:40:17.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.379+0000 7f16880ddb80 20 client.15624 populate_metadata read hostname 'smithi044' 2023-09-10T00:40:17.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.379+0000 7f16880ddb80 1 --2- 172.21.15.44:0/3294072313 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f976e2d40 0x558f976e1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.379+0000 7f16880ddb80 4 client.15624 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:40:17.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.379+0000 7f16880ddb80 1 -- 172.21.15.44:0/3294072313 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f97634850 con 0x558f976e2d40 2023-09-10T00:40:17.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.379+0000 7f1679cb3700 1 --2- 172.21.15.44:0/3294072313 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f976e2d40 0x558f976e1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.379+0000 7f1679cb3700 1 --2- 172.21.15.44:0/3294072313 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f976e2d40 0x558f976e1cb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b613e032cda2270 server_cookie=e6342c623dceb898 in_seq=0 out_seq=0 2023-09-10T00:40:17.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.379+0000 7f165d7fa700 10 client.15624 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:40:17.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.380+0000 7f165d7fa700 1 -- 172.21.15.44:0/3294072313 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558f97634850 con 0x558f976e2d40 2023-09-10T00:40:17.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.380+0000 7f165d7fa700 10 client.15624 handle_command_reply: tid=0 2023-09-10T00:40:17.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.380+0000 7f16880ddb80 1 client.15624 shutdown 2023-09-10T00:40:17.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.380+0000 7f16880ddb80 1 -- 172.21.15.44:0/3294072313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cebb0 msgr2=0x558f976d7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.380+0000 7f16880ddb80 1 --2- 172.21.15.44:0/3294072313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cebb0 0x558f976d7b20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f16640098e0 tx=0x7f16640050f0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.380+0000 7f16880ddb80 1 -- 172.21.15.44:0/3294072313 shutdown_connections 2023-09-10T00:40:17.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.380+0000 7f16880ddb80 1 --2- 172.21.15.44:0/3294072313 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f976e2d40 0x558f976e1cb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.380+0000 7f16880ddb80 1 --2- 172.21.15.44:0/3294072313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cf550 0x558f976deb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.380+0000 7f16880ddb80 1 --2- 172.21.15.44:0/3294072313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cddb0 0x558f976d75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.380+0000 7f165d7fa700 0 client.15624 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:40:17.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.380+0000 7f16880ddb80 1 --2- 172.21.15.44:0/3294072313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cebb0 0x558f976d7b20 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.380+0000 7f16880ddb80 1 -- 172.21.15.44:0/3294072313 >> 172.21.15.44:0/3294072313 conn(0x558f97642040 msgr2=0x558f976c7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.380+0000 7f16880ddb80 1 -- 172.21.15.44:0/3294072313 shutdown_connections 2023-09-10T00:40:17.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.380+0000 7f16880ddb80 1 -- 172.21.15.44:0/3294072313 wait complete. 2023-09-10T00:40:17.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.381+0000 7f16880ddb80 20 client.15624 trim_cache size 0 max 16384 2023-09-10T00:40:17.409 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.398+0000 7f16880ddb80 1 Processor -- start 2023-09-10T00:40:17.409 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.398+0000 7f16880ddb80 1 -- start start 2023-09-10T00:40:17.409 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.398+0000 7f16880ddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cddb0 0x558f976d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.398+0000 7f16880ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cebb0 0x558f976deb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.398+0000 7f16880ddb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cf550 0x558f976d5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.398+0000 7f16880ddb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f975e8f70 con 0x558f976cf550 2023-09-10T00:40:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.398+0000 7f16880ddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f975dbed0 con 0x558f976cddb0 2023-09-10T00:40:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.398+0000 7f16880ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f975c74d0 con 0x558f976cebb0 2023-09-10T00:40:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.399+0000 7f167a4b4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cf550 0x558f976d5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.399+0000 7f167a4b4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cf550 0x558f976d5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48196/0 (socket says 172.21.15.44:48196) 2023-09-10T00:40:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.399+0000 7f167a4b4700 1 -- 172.21.15.44:0/1936719836 learned_addr learned my addr 172.21.15.44:0/1936719836 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.399+0000 7f1679cb3700 1 --2- 172.21.15.44:0/1936719836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cddb0 0x558f976d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.399+0000 7f16794b2700 1 --2- 172.21.15.44:0/1936719836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cebb0 0x558f976deb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.399+0000 7f167a4b4700 1 -- 172.21.15.44:0/1936719836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cebb0 msgr2=0x558f976deb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.399+0000 7f167a4b4700 1 --2- 172.21.15.44:0/1936719836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cebb0 0x558f976deb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.399+0000 7f167a4b4700 1 -- 172.21.15.44:0/1936719836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cddb0 msgr2=0x558f976d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.399+0000 7f167a4b4700 1 --2- 172.21.15.44:0/1936719836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cddb0 0x558f976d1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.399+0000 7f167a4b4700 1 -- 172.21.15.44:0/1936719836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f97472200 con 0x558f976cf550 2023-09-10T00:40:17.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.399+0000 7f167a4b4700 1 --2- 172.21.15.44:0/1936719836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cf550 0x558f976d5440 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f166c0a0670 tx=0x7f166c098690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b85edee7d93e26 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.399+0000 7f1668ff9700 1 -- 172.21.15.44:0/1936719836 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c0b2070 con 0x558f976cf550 2023-09-10T00:40:17.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.399+0000 7f1668ff9700 1 -- 172.21.15.44:0/1936719836 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f166c00ddd0 con 0x558f976cf550 2023-09-10T00:40:17.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.399+0000 7f1668ff9700 1 -- 172.21.15.44:0/1936719836 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c096ed0 con 0x558f976cf550 2023-09-10T00:40:17.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.400+0000 7f16880ddb80 1 -- 172.21.15.44:0/1936719836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cf550 msgr2=0x558f976d5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.400+0000 7f16880ddb80 1 --2- 172.21.15.44:0/1936719836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cf550 0x558f976d5440 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f166c0a0670 tx=0x7f166c098690 comp rx=0 tx=0).stop 2023-09-10T00:40:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.400+0000 7f16880ddb80 1 -- 172.21.15.44:0/1936719836 shutdown_connections 2023-09-10T00:40:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.400+0000 7f16880ddb80 1 --2- 172.21.15.44:0/1936719836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cf550 0x558f976d5440 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.400+0000 7f16880ddb80 1 --2- 172.21.15.44:0/1936719836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cddb0 0x558f976d1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.400+0000 7f16880ddb80 1 --2- 172.21.15.44:0/1936719836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cebb0 0x558f976deb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.400+0000 7f16880ddb80 1 -- 172.21.15.44:0/1936719836 >> 172.21.15.44:0/1936719836 conn(0x558f97642040 msgr2=0x558f976d6280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.400+0000 7f16880ddb80 1 -- 172.21.15.44:0/1936719836 shutdown_connections 2023-09-10T00:40:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.400+0000 7f16880ddb80 1 -- 172.21.15.44:0/1936719836 wait complete. 2023-09-10T00:40:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.400+0000 7f16880ddb80 1 -- start start 2023-09-10T00:40:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.400+0000 7f16880ddb80 1 Processor -- start 2023-09-10T00:40:17.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16880ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cddb0 0x558f976d9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16880ddb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cebb0 0x558f976da2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16880ddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cf550 0x558f97736d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16880ddb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f976da880 con 0x558f976cebb0 2023-09-10T00:40:17.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16880ddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f976da9f0 con 0x558f976cf550 2023-09-10T00:40:17.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16880ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f97737350 con 0x558f976cddb0 2023-09-10T00:40:17.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16880ddb80 10 client.0 _ll_register_callbacks cb 0x7f1664009f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:40:17.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16794b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cebb0 0x558f976da2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16794b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cebb0 0x558f976da2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48200/0 (socket says 172.21.15.44:48200) 2023-09-10T00:40:17.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16794b2700 1 -- 172.21.15.44:0/2991562110 learned_addr learned my addr 172.21.15.44:0/2991562110 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:17.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f1679cb3700 1 --2- 172.21.15.44:0/2991562110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cddb0 0x558f976d9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f167a4b4700 1 --2- 172.21.15.44:0/2991562110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cf550 0x558f97736d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16794b2700 1 -- 172.21.15.44:0/2991562110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cddb0 msgr2=0x558f976d9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16794b2700 1 --2- 172.21.15.44:0/2991562110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cddb0 0x558f976d9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16794b2700 1 -- 172.21.15.44:0/2991562110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cf550 msgr2=0x558f97736d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:17.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16794b2700 1 --2- 172.21.15.44:0/2991562110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cf550 0x558f97736d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16794b2700 1 -- 172.21.15.44:0/2991562110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f9744bd90 con 0x558f976cebb0 2023-09-10T00:40:17.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f167a4b4700 1 --2- 172.21.15.44:0/2991562110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cf550 0x558f97736d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:40:17.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16794b2700 1 --2- 172.21.15.44:0/2991562110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cebb0 0x558f976da2d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f166400c010 tx=0x7f1664005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382b79fe7bff7db server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:17.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f165d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:40:17.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f165d7fa700 1 -- 172.21.15.44:0/2991562110 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166401d070 con 0x558f976cebb0 2023-09-10T00:40:17.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f165d7fa700 1 -- 172.21.15.44:0/2991562110 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f16640041b0 con 0x558f976cebb0 2023-09-10T00:40:17.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f16880ddb80 1 -- 172.21.15.44:0/2991562110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f166c099230 con 0x558f976cebb0 2023-09-10T00:40:17.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.401+0000 7f165d7fa700 1 -- 172.21.15.44:0/2991562110 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1664003a20 con 0x558f976cebb0 2023-09-10T00:40:17.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.402+0000 7f16880ddb80 10 client.15630 fetch_fsmap learned FSMap version 27 2023-09-10T00:40:17.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.402+0000 7f16880ddb80 1 -- 172.21.15.44:0/2991562110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558f9744a260 con 0x558f976cebb0 2023-09-10T00:40:17.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.402+0000 7f165d7fa700 1 -- 172.21.15.44:0/2991562110 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f166c099230 con 0x558f976cebb0 2023-09-10T00:40:17.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.402+0000 7f165d7fa700 1 -- 172.21.15.44:0/2991562110 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1664004500 con 0x558f976cebb0 2023-09-10T00:40:17.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.402+0000 7f16880ddb80 10 client.15630 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:40:17.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.402+0000 7f16880ddb80 10 client.15630 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-10T00:40:17.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.402+0000 7f16880ddb80 20 client.15630 populate_metadata read hostname 'smithi044' 2023-09-10T00:40:17.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.402+0000 7f16880ddb80 1 --2- 172.21.15.44:0/2991562110 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f9773c130 0x558f977385e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:17.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.402+0000 7f16880ddb80 4 client.15630 mds_command: new command op to 24605 tid=0[{"prefix": "status"}] 2023-09-10T00:40:17.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.402+0000 7f16880ddb80 1 -- 172.21.15.44:0/2991562110 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558f976188d0 con 0x558f9773c130 2023-09-10T00:40:17.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.402+0000 7f1679cb3700 1 --2- 172.21.15.44:0/2991562110 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f9773c130 0x558f977385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:17.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.403+0000 7f165d7fa700 10 client.15630 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:40:17.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.402+0000 7f1679cb3700 1 --2- 172.21.15.44:0/2991562110 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f9773c130 0x558f977385e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65c1a3d680d1d020 server_cookie=4280d156a45d1151 in_seq=0 out_seq=0 2023-09-10T00:40:17.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.403+0000 7f165d7fa700 1 -- 172.21.15.44:0/2991562110 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x558f976188d0 con 0x558f9773c130 2023-09-10T00:40:17.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.403+0000 7f165d7fa700 10 client.15630 handle_command_reply: tid=0 2023-09-10T00:40:17.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.403+0000 7f16880ddb80 1 client.15630 shutdown 2023-09-10T00:40:17.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.403+0000 7f16880ddb80 1 -- 172.21.15.44:0/2991562110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cebb0 msgr2=0x558f976da2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.403+0000 7f16880ddb80 1 --2- 172.21.15.44:0/2991562110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cebb0 0x558f976da2d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f166400c010 tx=0x7f1664005c40 comp rx=0 tx=0).stop 2023-09-10T00:40:17.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.403+0000 7f16880ddb80 1 -- 172.21.15.44:0/2991562110 shutdown_connections 2023-09-10T00:40:17.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.403+0000 7f16880ddb80 1 --2- 172.21.15.44:0/2991562110 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f9773c130 0x558f977385e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.403+0000 7f16880ddb80 1 --2- 172.21.15.44:0/2991562110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f976cf550 0x558f97736d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.403+0000 7f16880ddb80 1 --2- 172.21.15.44:0/2991562110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f976cddb0 0x558f976d9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.403+0000 7f165d7fa700 0 client.15630 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:40:17.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.403+0000 7f16880ddb80 1 --2- 172.21.15.44:0/2991562110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f976cebb0 0x558f976da2d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.403+0000 7f16880ddb80 1 -- 172.21.15.44:0/2991562110 >> 172.21.15.44:0/2991562110 conn(0x558f97642040 msgr2=0x558f976d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.403+0000 7f16880ddb80 1 -- 172.21.15.44:0/2991562110 shutdown_connections 2023-09-10T00:40:17.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.403+0000 7f16880ddb80 1 -- 172.21.15.44:0/2991562110 wait complete. 2023-09-10T00:40:17.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.403+0000 7f16880ddb80 20 client.15630 trim_cache size 0 max 16384 2023-09-10T00:40:17.426 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:40:17.426 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da", 2023-09-10T00:40:17.426 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2023-09-10T00:40:17.427 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24605, 2023-09-10T00:40:17.427 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2023-09-10T00:40:17.427 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2023-09-10T00:40:17.427 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2023-09-10T00:40:17.427 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 97.657694446999997, 2023-09-10T00:40:17.428 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2023-09-10T00:40:17.428 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 90, 2023-09-10T00:40:17.428 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 87, 2023-09-10T00:40:17.428 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 104.01005988 2023-09-10T00:40:17.428 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:40:17.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.404+0000 7f167bf17700 1 -- 172.21.15.44:0/2431357532 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16600a0640 msgr2=0x7f16600a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.404+0000 7f167bf17700 1 --2- 172.21.15.44:0/2431357532 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16600a0640 0x7f16600a2b00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1670005950 tx=0x7f167000b410 comp rx=0 tx=0).stop 2023-09-10T00:40:17.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.404+0000 7f167bf17700 1 -- 172.21.15.44:0/2431357532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1674106300 msgr2=0x7f16740ff590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:17.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.404+0000 7f167bf17700 1 --2- 172.21.15.44:0/2431357532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1674106300 0x7f16740ff590 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f166c007ee0 tx=0x7f166c00d3b0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.404+0000 7f167bf17700 1 -- 172.21.15.44:0/2431357532 shutdown_connections 2023-09-10T00:40:17.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.404+0000 7f167bf17700 1 --2- 172.21.15.44:0/2431357532 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16600a0640 0x7f16600a2b00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.404+0000 7f167bf17700 1 --2- 172.21.15.44:0/2431357532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1674106300 0x7f16740ff590 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.404+0000 7f167bf17700 1 --2- 172.21.15.44:0/2431357532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16741010a0 0x7f1674100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.404+0000 7f167bf17700 1 --2- 172.21.15.44:0/2431357532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16741039d0 0x7f16740ff050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:17.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.404+0000 7f167bf17700 1 -- 172.21.15.44:0/2431357532 >> 172.21.15.44:0/2431357532 conn(0x7f16740faa90 msgr2=0x7f1674101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:17.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.404+0000 7f167bf17700 1 -- 172.21.15.44:0/2431357532 shutdown_connections 2023-09-10T00:40:17.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:17.404+0000 7f167bf17700 1 -- 172.21.15.44:0/2431357532 wait complete. 2023-09-10T00:40:17.432 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-09-10T00:40:17.445 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-09-10T00:40:17.445 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2023-09-10T00:40:17.445 INFO:tasks.ceph_fuse:client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2023-09-10T00:40:17.445 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-10T00:40:17.446 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2023-09-10T00:40:17.446 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2023-09-10T00:40:17.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:17.447 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2023-09-10T00:40:17.464 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-09-10T00:40:17.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:17.465 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-10T00:40:17.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:17.549 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2023-09-10T00:40:17.653 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-09-10T00:40:17.653 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph fs ls 2023-09-10T00:40:17.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:17 smithi044 ceph-mon[98676]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-10T00:40:17.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-09-10T00:40:17.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-09-10T00:40:17.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-09-10T00:40:17.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-09-10T00:40:17.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2023-09-10T00:40:17.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:17 smithi044 ceph-mon[98676]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2023-09-10T00:40:17.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:17 smithi044 ceph-mon[98676]: osdmap e90: 12 total, 12 up, 12 in 2023-09-10T00:40:17.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:17 smithi044 ceph-mon[98676]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-09-10T00:40:18.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:17 smithi131 ceph-mon[103713]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-10T00:40:18.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-09-10T00:40:18.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-09-10T00:40:18.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-09-10T00:40:18.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-09-10T00:40:18.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2023-09-10T00:40:18.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:17 smithi131 ceph-mon[103713]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2023-09-10T00:40:18.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:17 smithi131 ceph-mon[103713]: osdmap e90: 12 total, 12 up, 12 in 2023-09-10T00:40:18.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:17 smithi131 ceph-mon[103713]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-09-10T00:40:18.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:17 smithi099 ceph-mon[102692]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-10T00:40:18.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-09-10T00:40:18.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-09-10T00:40:18.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-09-10T00:40:18.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-09-10T00:40:18.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2023-09-10T00:40:18.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:17 smithi099 ceph-mon[102692]: from='mgr.14152 172.21.15.44:0/4208591411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2023-09-10T00:40:18.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:17 smithi099 ceph-mon[102692]: osdmap e90: 12 total, 12 up, 12 in 2023-09-10T00:40:18.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:17 smithi099 ceph-mon[102692]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-09-10T00:40:18.219 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T00:40:19.006 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:18 smithi044 ceph-mon[98676]: osdmap e91: 12 total, 12 up, 12 in 2023-09-10T00:40:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:18 smithi131 ceph-mon[103713]: osdmap e91: 12 total, 12 up, 12 in 2023-09-10T00:40:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:18 smithi099 ceph-mon[102692]: osdmap e91: 12 total, 12 up, 12 in 2023-09-10T00:40:19.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.491+0000 7f5019214700 1 -- 172.21.15.44:0/3619654067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50140ff520 msgr2=0x7f50140ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:19.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.491+0000 7f5019214700 1 --2- 172.21.15.44:0/3619654067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50140ff520 0x7f50140ff9a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5008009990 tx=0x7f5008009ca0 comp rx=0 tx=0).stop 2023-09-10T00:40:19.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.491+0000 7f5019214700 1 -- 172.21.15.44:0/3619654067 shutdown_connections 2023-09-10T00:40:19.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.491+0000 7f5019214700 1 --2- 172.21.15.44:0/3619654067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50140ffee0 0x7f501410c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:19.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.491+0000 7f5019214700 1 --2- 172.21.15.44:0/3619654067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5014104530 0x7f5014104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:19.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.491+0000 7f5019214700 1 --2- 172.21.15.44:0/3619654067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50140ff520 0x7f50140ff9a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:19.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.491+0000 7f5019214700 1 -- 172.21.15.44:0/3619654067 >> 172.21.15.44:0/3619654067 conn(0x7f5014076600 msgr2=0x7f5014076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:19.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.491+0000 7f5019214700 1 -- 172.21.15.44:0/3619654067 shutdown_connections 2023-09-10T00:40:19.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.491+0000 7f5019214700 1 -- 172.21.15.44:0/3619654067 wait complete. 2023-09-10T00:40:19.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.492+0000 7f5019214700 1 Processor -- start 2023-09-10T00:40:19.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.492+0000 7f5019214700 1 -- start start 2023-09-10T00:40:19.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.492+0000 7f5019214700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50140ff520 0x7f50141974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:19.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.492+0000 7f5019214700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50140ffee0 0x7f50141979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:19.498 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.492+0000 7f5019214700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5014104530 0x7f50141a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:19.498 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.492+0000 7f5019214700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5014198320 con 0x7f50140ffee0 2023-09-10T00:40:19.498 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.492+0000 7f5019214700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50141a2a60 con 0x7f50140ff520 2023-09-10T00:40:19.498 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.492+0000 7f5019214700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50141a2c20 con 0x7f5014104530 2023-09-10T00:40:19.498 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.493+0000 7f501259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50140ffee0 0x7f50141979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:19.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.493+0000 7f501259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50140ffee0 0x7f50141979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48214/0 (socket says 172.21.15.44:48214) 2023-09-10T00:40:19.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.493+0000 7f501259c700 1 -- 172.21.15.44:0/420944500 learned_addr learned my addr 172.21.15.44:0/420944500 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:19.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.493+0000 7f501359e700 1 --2- 172.21.15.44:0/420944500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5014104530 0x7f50141a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:19.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.493+0000 7f5012d9d700 1 --2- 172.21.15.44:0/420944500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50140ff520 0x7f50141974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:19.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.493+0000 7f501259c700 1 -- 172.21.15.44:0/420944500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5014104530 msgr2=0x7f50141a2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:19.500 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.493+0000 7f501259c700 1 --2- 172.21.15.44:0/420944500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5014104530 0x7f50141a2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:19.500 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.493+0000 7f501259c700 1 -- 172.21.15.44:0/420944500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50140ff520 msgr2=0x7f50141974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:19.500 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.493+0000 7f501259c700 1 --2- 172.21.15.44:0/420944500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50140ff520 0x7f50141974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:19.500 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.493+0000 7f501259c700 1 -- 172.21.15.44:0/420944500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50080095f0 con 0x7f50140ffee0 2023-09-10T00:40:19.500 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.493+0000 7f501259c700 1 --2- 172.21.15.44:0/420944500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50140ffee0 0x7f50141979e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5008006010 tx=0x7f5008004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:19.501 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.493+0000 7f4ffbfff700 1 -- 172.21.15.44:0/420944500 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500801d070 con 0x7f50140ffee0 2023-09-10T00:40:19.501 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.493+0000 7f4ffbfff700 1 -- 172.21.15.44:0/420944500 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5008004000 con 0x7f50140ffee0 2023-09-10T00:40:19.501 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.493+0000 7f5019214700 1 -- 172.21.15.44:0/420944500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50141a2ea0 con 0x7f50140ffee0 2023-09-10T00:40:19.501 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.493+0000 7f4ffbfff700 1 -- 172.21.15.44:0/420944500 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5008022ac0 con 0x7f50140ffee0 2023-09-10T00:40:19.501 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.493+0000 7f5019214700 1 -- 172.21.15.44:0/420944500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50141a3250 con 0x7f50140ffee0 2023-09-10T00:40:19.501 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.494+0000 7f5019214700 1 -- 172.21.15.44:0/420944500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f501404e300 con 0x7f50140ffee0 2023-09-10T00:40:19.502 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.496+0000 7f4ffbfff700 1 -- 172.21.15.44:0/420944500 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f5008003b70 con 0x7f50140ffee0 2023-09-10T00:40:19.502 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.496+0000 7f4ffbfff700 1 --2- 172.21.15.44:0/420944500 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4ffc0a0640 0x7f4ffc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:19.502 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.496+0000 7f4ffbfff700 1 -- 172.21.15.44:0/420944500 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f50080cdcf0 con 0x7f50140ffee0 2023-09-10T00:40:19.502 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.496+0000 7f5012d9d700 1 --2- 172.21.15.44:0/420944500 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4ffc0a0640 0x7f4ffc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:19.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.497+0000 7f5012d9d700 1 --2- 172.21.15.44:0/420944500 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4ffc0a0640 0x7f4ffc0a2b00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5004005950 tx=0x7f50040058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:19.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.500+0000 7f4ffbfff700 1 -- 172.21.15.44:0/420944500 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f50080974f0 con 0x7f50140ffee0 2023-09-10T00:40:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.744+0000 7f5019214700 1 -- 172.21.15.44:0/420944500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f50140686d0 con 0x7f50140ffee0 2023-09-10T00:40:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.745+0000 7f4ffbfff700 1 -- 172.21.15.44:0/420944500 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+88 (secure 0 0 0) 0x7f500809ab10 con 0x7f50140ffee0 2023-09-10T00:40:19.746 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-10T00:40:19.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.747+0000 7f5019214700 1 -- 172.21.15.44:0/420944500 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4ffc0a0640 msgr2=0x7f4ffc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:19.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.747+0000 7f5019214700 1 --2- 172.21.15.44:0/420944500 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4ffc0a0640 0x7f4ffc0a2b00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5004005950 tx=0x7f50040058e0 comp rx=0 tx=0).stop 2023-09-10T00:40:19.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.747+0000 7f5019214700 1 -- 172.21.15.44:0/420944500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50140ffee0 msgr2=0x7f50141979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:19.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.747+0000 7f5019214700 1 --2- 172.21.15.44:0/420944500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50140ffee0 0x7f50141979e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5008006010 tx=0x7f5008004730 comp rx=0 tx=0).stop 2023-09-10T00:40:19.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.747+0000 7f5019214700 1 -- 172.21.15.44:0/420944500 shutdown_connections 2023-09-10T00:40:19.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.747+0000 7f5019214700 1 --2- 172.21.15.44:0/420944500 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4ffc0a0640 0x7f4ffc0a2b00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:19.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.747+0000 7f5019214700 1 --2- 172.21.15.44:0/420944500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5014104530 0x7f50141a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:19.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.747+0000 7f5019214700 1 --2- 172.21.15.44:0/420944500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50140ff520 0x7f50141974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:19.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.747+0000 7f5019214700 1 --2- 172.21.15.44:0/420944500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50140ffee0 0x7f50141979e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:19.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.747+0000 7f5019214700 1 -- 172.21.15.44:0/420944500 >> 172.21.15.44:0/420944500 conn(0x7f5014076600 msgr2=0x7f50140fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:19.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.747+0000 7f5019214700 1 -- 172.21.15.44:0/420944500 shutdown_connections 2023-09-10T00:40:19.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:19.747+0000 7f5019214700 1 -- 172.21.15.44:0/420944500 wait complete. 2023-09-10T00:40:20.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:19 smithi131 ceph-mon[103713]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-10T00:40:20.052 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:19 smithi044 ceph-mon[98676]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-10T00:40:20.077 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-10T00:40:20.078 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-10T00:40:20.078 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2023-09-10T00:40:20.079 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-10T00:40:20.079 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2023-09-10T00:40:20.080 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-10T00:40:20.080 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-10T00:40:20.080 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-10T00:40:20.081 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2023-09-10T00:40:20.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:20.082 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2023-09-10T00:40:20.101 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-10T00:40:20.101 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-10T00:40:20.102 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2023-09-10T00:40:20.102 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2023-09-10T00:40:20.102 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2023-09-10T00:40:20.102 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2023-09-10T00:40:20.102 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-10T00:40:20.103 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2023-09-10T00:40:20.103 INFO:teuthology.orchestra.run.smithi044.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-10T00:40:20.103 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2023-09-10T00:40:20.103 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-09-10T00:40:20.104 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 41831sec preferred_lft 41831sec 2023-09-10T00:40:20.104 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link 2023-09-10T00:40:20.104 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2023-09-10T00:40:20.104 INFO:teuthology.orchestra.run.smithi044.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-10T00:40:20.105 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2023-09-10T00:40:20.105 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-10T00:40:20.105 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2023-09-10T00:40:20.105 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-10T00:40:20.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:20.106 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2023-09-10T00:40:20.107 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2023-09-10T00:40:20.107 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add name ceph-brx type bridge 2023-09-10T00:40:20.107 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr flush dev ceph-brx 2023-09-10T00:40:20.107 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set ceph-brx up 2023-09-10T00:40:20.108 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-10T00:40:20.108 DEBUG:teuthology.orchestra.run.smithi044:> ') 2023-09-10T00:40:20.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:19 smithi099 ceph-mon[102692]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-10T00:40:20.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:20.226 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-10T00:40:20.254 INFO:teuthology.orchestra.run.smithi044.stdout:1 2023-09-10T00:40:20.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:20.255 DEBUG:teuthology.orchestra.run.smithi044:> route 2023-09-10T00:40:20.315 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2023-09-10T00:40:20.315 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-10T00:40:20.315 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-09-10T00:40:20.316 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-09-10T00:40:20.316 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-10T00:40:20.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:20.317 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2023-09-10T00:40:20.317 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2023-09-10T00:40:20.318 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-09-10T00:40:20.318 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-09-10T00:40:20.318 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-09-10T00:40:20.319 DEBUG:teuthology.orchestra.run.smithi044:> ') 2023-09-10T00:40:20.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:20.424 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2023-09-10T00:40:20.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:20.442 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list-id 2023-09-10T00:40:20.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:20.499 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2023-09-10T00:40:20.500 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2023-09-10T00:40:20.500 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--tmp-cephfs 2023-09-10T00:40:20.500 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--tmp-cephfs 0 2023-09-10T00:40:20.501 DEBUG:teuthology.orchestra.run.smithi044:> ') 2023-09-10T00:40:20.595 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2023-09-10T00:40:20.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:20.596 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2023-09-10T00:40:20.596 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2023-09-10T00:40:20.596 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2023-09-10T00:40:20.597 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-10T00:40:20.597 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2023-09-10T00:40:20.597 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2023-09-10T00:40:20.597 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2023-09-10T00:40:20.598 DEBUG:teuthology.orchestra.run.smithi044:> ') 2023-09-10T00:40:20.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:20.804 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec bash -c ' 2023-09-10T00:40:20.805 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2023-09-10T00:40:20.805 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2023-09-10T00:40:20.805 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.0 master ceph-brx 2023-09-10T00:40:20.805 DEBUG:teuthology.orchestra.run.smithi044:> ') 2023-09-10T00:40:20.911 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:20 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/420944500' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-10T00:40:20.928 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2023-09-10T00:40:20.929 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-10T00:40:20.929 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /tmp/cephfs 2023-09-10T00:40:20.957 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/tmp/cephfs' 2023-09-10T00:40:20.957 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-10T00:40:20.958 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /tmp/cephfs 2023-09-10T00:40:21.014 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2023-09-10T00:40:21.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:20 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/420944500' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-10T00:40:21.084 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-10T00:40:21.084 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-10T00:40:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:20 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/420944500' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-10T00:40:21.151 INFO:teuthology.orchestra.run.smithi044.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-10T00:40:21.152 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-10T00:40:21.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:21.153 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2023-09-10T00:40:21.210 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-10T00:40:21.211 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /tmp/cephfs --id 0 --client_mountpoint=/volumes/_nogroup/sv_0/18ae2dc5-fa62-40a4-9c11-44763760865a --client_fs=cephfs -o context=system_u:object_r:postgresql_db_t:s0) 2023-09-10T00:40:21.253 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2023-09-10T00:40:21.281 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-10T00:40:21.282 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-10T00:40:21.346 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:2023-09-10T00:40:21.346+0000 7f7c3fb0d5c0 -1 init, newargv = 0x558a954bd610 newargc=17 2023-09-10T00:40:21.347 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:ceph-fuse[170792]: starting ceph client 2023-09-10T00:40:21.351 INFO:teuthology.orchestra.run.smithi044.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-10T00:40:21.352 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-10T00:40:21.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:21.353 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2023-09-10T00:40:21.371 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:ceph-fuse[170792]: starting fuse 2023-09-10T00:40:21.388 INFO:teuthology.orchestra.run.smithi044.stdout:124 2023-09-10T00:40:21.388 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2023-09-10T00:40:21.389 DEBUG:teuthology.orchestra.run.smithi044:> sudo stdin-killer -- python3 -c ' 2023-09-10T00:40:21.389 DEBUG:teuthology.orchestra.run.smithi044:> import glob 2023-09-10T00:40:21.389 DEBUG:teuthology.orchestra.run.smithi044:> import re 2023-09-10T00:40:21.390 DEBUG:teuthology.orchestra.run.smithi044:> import os 2023-09-10T00:40:21.390 DEBUG:teuthology.orchestra.run.smithi044:> import subprocess 2023-09-10T00:40:21.390 DEBUG:teuthology.orchestra.run.smithi044:> 2023-09-10T00:40:21.391 DEBUG:teuthology.orchestra.run.smithi044:> def _find_admin_socket(client_name): 2023-09-10T00:40:21.391 DEBUG:teuthology.orchestra.run.smithi044:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-10T00:40:21.391 DEBUG:teuthology.orchestra.run.smithi044:> files = glob.glob(asok_path) 2023-09-10T00:40:21.391 DEBUG:teuthology.orchestra.run.smithi044:> mountpoint = "/tmp/cephfs" 2023-09-10T00:40:21.392 DEBUG:teuthology.orchestra.run.smithi044:> 2023-09-10T00:40:21.392 DEBUG:teuthology.orchestra.run.smithi044:> # Given a non-glob path, it better be there 2023-09-10T00:40:21.392 DEBUG:teuthology.orchestra.run.smithi044:> if "*" not in asok_path: 2023-09-10T00:40:21.393 DEBUG:teuthology.orchestra.run.smithi044:> assert(len(files) == 1) 2023-09-10T00:40:21.393 DEBUG:teuthology.orchestra.run.smithi044:> return files[0] 2023-09-10T00:40:21.393 DEBUG:teuthology.orchestra.run.smithi044:> 2023-09-10T00:40:21.393 DEBUG:teuthology.orchestra.run.smithi044:> for f in files: 2023-09-10T00:40:21.394 DEBUG:teuthology.orchestra.run.smithi044:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-10T00:40:21.394 DEBUG:teuthology.orchestra.run.smithi044:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-10T00:40:21.394 DEBUG:teuthology.orchestra.run.smithi044:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-10T00:40:21.395 DEBUG:teuthology.orchestra.run.smithi044:> contents = proc_f.read() 2023-09-10T00:40:21.395 DEBUG:teuthology.orchestra.run.smithi044:> if mountpoint in contents: 2023-09-10T00:40:21.395 DEBUG:teuthology.orchestra.run.smithi044:> return f 2023-09-10T00:40:21.395 DEBUG:teuthology.orchestra.run.smithi044:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-10T00:40:21.396 DEBUG:teuthology.orchestra.run.smithi044:> 2023-09-10T00:40:21.396 DEBUG:teuthology.orchestra.run.smithi044:> print(_find_admin_socket("client.0")) 2023-09-10T00:40:21.396 DEBUG:teuthology.orchestra.run.smithi044:> ' 2023-09-10T00:40:21.514 INFO:teuthology.orchestra.run.smithi044.stdout:/var/run/ceph/ceph-client.0.170792.asok 2023-09-10T00:40:21.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-10T00:40:21.524 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.170792.asok 2023-09-10T00:40:21.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:21.525 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.170792.asok status 2023-09-10T00:40:21.687 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:40:21.688 INFO:teuthology.orchestra.run.smithi044.stdout: "metadata": { 2023-09-10T00:40:21.688 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_sha1": "9670cca5bc8e17916865dc84939edfc10b1f2c00", 2023-09-10T00:40:21.688 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev)", 2023-09-10T00:40:21.688 INFO:teuthology.orchestra.run.smithi044.stdout: "entity_id": "0", 2023-09-10T00:40:21.689 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": "smithi044", 2023-09-10T00:40:21.689 INFO:teuthology.orchestra.run.smithi044.stdout: "mount_point": "/tmp/cephfs", 2023-09-10T00:40:21.689 INFO:teuthology.orchestra.run.smithi044.stdout: "pid": "170792", 2023-09-10T00:40:21.689 INFO:teuthology.orchestra.run.smithi044.stdout: "root": "/volumes/_nogroup/sv_0/18ae2dc5-fa62-40a4-9c11-44763760865a" 2023-09-10T00:40:21.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T00:40:21.689 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_count": 0, 2023-09-10T00:40:21.690 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_pinned_count": 0, 2023-09-10T00:40:21.690 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 15642, 2023-09-10T00:40:21.690 INFO:teuthology.orchestra.run.smithi044.stdout: "inst": { 2023-09-10T00:40:21.690 INFO:teuthology.orchestra.run.smithi044.stdout: "name": { 2023-09-10T00:40:21.690 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "client", 2023-09-10T00:40:21.691 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 15642 2023-09-10T00:40:21.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T00:40:21.691 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2023-09-10T00:40:21.691 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2023-09-10T00:40:21.691 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2023-09-10T00:40:21.692 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 981857120 2023-09-10T00:40:21.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:40:21.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T00:40:21.692 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2023-09-10T00:40:21.692 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2023-09-10T00:40:21.693 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2023-09-10T00:40:21.693 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 981857120 2023-09-10T00:40:21.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T00:40:21.693 INFO:teuthology.orchestra.run.smithi044.stdout: "inst_str": "client.15642 192.168.0.1:0/981857120", 2023-09-10T00:40:21.693 INFO:teuthology.orchestra.run.smithi044.stdout: "addr_str": "192.168.0.1:0/981857120", 2023-09-10T00:40:21.693 INFO:teuthology.orchestra.run.smithi044.stdout: "inode_count": 5, 2023-09-10T00:40:21.694 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_epoch": 27, 2023-09-10T00:40:21.694 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch": 91, 2023-09-10T00:40:21.694 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch_barrier": 0, 2023-09-10T00:40:21.694 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklisted": false, 2023-09-10T00:40:21.694 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs" 2023-09-10T00:40:21.695 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:40:21.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:21.698 DEBUG:teuthology.orchestra.run.smithi044:> stat --file-system '--printf=%T 2023-09-10T00:40:21.698 DEBUG:teuthology.orchestra.run.smithi044:> ' -- /tmp/cephfs 2023-09-10T00:40:21.716 INFO:teuthology.orchestra.run.smithi044.stdout:fuseblk 2023-09-10T00:40:21.716 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/cephfs 2023-09-10T00:40:21.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T00:40:21.717 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /tmp/cephfs 2023-09-10T00:40:21.786 INFO:teuthology.run_tasks:Running task check-counter... 2023-09-10T00:40:21.798 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-09-10T00:40:21.809 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:40:21.851 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:21 smithi044 ceph-mon[98676]: pgmap v323: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 895 B/s wr, 11 op/s; 7.2 KiB/s, 1 keys/s, 1 objects/s recovering 2023-09-10T00:40:21.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.953+0000 7f4ac50cc700 1 -- 172.21.15.44:0/2720707445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ac01006d0 msgr2=0x7f4ac0100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:21.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.953+0000 7f4ac50cc700 1 --2- 172.21.15.44:0/2720707445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ac01006d0 0x7f4ac0100b30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40099c0 tx=0x7f4ab4009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:21.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.953+0000 7f4ac50cc700 1 -- 172.21.15.44:0/2720707445 shutdown_connections 2023-09-10T00:40:21.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.953+0000 7f4ac50cc700 1 --2- 172.21.15.44:0/2720707445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac0101070 0x7f4ac0107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:21.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.953+0000 7f4ac50cc700 1 --2- 172.21.15.44:0/2720707445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ac00ff4e0 0x7f4ac00ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:21.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.953+0000 7f4ac50cc700 1 --2- 172.21.15.44:0/2720707445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ac01006d0 0x7f4ac0100b30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:21.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.953+0000 7f4ac50cc700 1 -- 172.21.15.44:0/2720707445 >> 172.21.15.44:0/2720707445 conn(0x7f4ac00faa30 msgr2=0x7f4ac00fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:21.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.953+0000 7f4ac50cc700 1 -- 172.21.15.44:0/2720707445 shutdown_connections 2023-09-10T00:40:21.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.953+0000 7f4ac50cc700 1 -- 172.21.15.44:0/2720707445 wait complete. 2023-09-10T00:40:21.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.954+0000 7f4ac50cc700 1 Processor -- start 2023-09-10T00:40:21.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.954+0000 7f4ac50cc700 1 -- start start 2023-09-10T00:40:21.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.954+0000 7f4ac50cc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ac00ff4e0 0x7f4ac00682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:21.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.954+0000 7f4ac50cc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ac01006d0 0x7f4ac019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:21.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.954+0000 7f4ac50cc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac0101070 0x7f4ac019d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:21.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.954+0000 7f4ac50cc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac019dd00 con 0x7f4ac01006d0 2023-09-10T00:40:21.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.954+0000 7f4ac50cc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac019de70 con 0x7f4ac0101070 2023-09-10T00:40:21.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.954+0000 7f4ac50cc700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac019e030 con 0x7f4ac00ff4e0 2023-09-10T00:40:21.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.954+0000 7f4abe59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ac01006d0 0x7f4ac019aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:21.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.954+0000 7f4abe59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ac01006d0 0x7f4ac019aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48250/0 (socket says 172.21.15.44:48250) 2023-09-10T00:40:21.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.954+0000 7f4abe59c700 1 -- 172.21.15.44:0/46380110 learned_addr learned my addr 172.21.15.44:0/46380110 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:21.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.955+0000 7f4abf59e700 1 --2- 172.21.15.44:0/46380110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac0101070 0x7f4ac019d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:21.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.955+0000 7f4abed9d700 1 --2- 172.21.15.44:0/46380110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ac00ff4e0 0x7f4ac00682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:21.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.955+0000 7f4abe59c700 1 -- 172.21.15.44:0/46380110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ac00ff4e0 msgr2=0x7f4ac00682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:21.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.955+0000 7f4abe59c700 1 --2- 172.21.15.44:0/46380110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ac00ff4e0 0x7f4ac00682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:21.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.955+0000 7f4abe59c700 1 -- 172.21.15.44:0/46380110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac0101070 msgr2=0x7f4ac019d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:21.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.955+0000 7f4abe59c700 1 --2- 172.21.15.44:0/46380110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac0101070 0x7f4ac019d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:21.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.955+0000 7f4abe59c700 1 -- 172.21.15.44:0/46380110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab40095f0 con 0x7f4ac01006d0 2023-09-10T00:40:21.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.955+0000 7f4abe59c700 1 --2- 172.21.15.44:0/46380110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ac01006d0 0x7f4ac019aee0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4005910 tx=0x7f4ab4003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:21.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.955+0000 7f4aa7fff700 1 -- 172.21.15.44:0/46380110 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab401d070 con 0x7f4ac01006d0 2023-09-10T00:40:21.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.955+0000 7f4ac50cc700 1 -- 172.21.15.44:0/46380110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac019e2b0 con 0x7f4ac01006d0 2023-09-10T00:40:21.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.955+0000 7f4aa7fff700 1 -- 172.21.15.44:0/46380110 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ab400bd30 con 0x7f4ac01006d0 2023-09-10T00:40:21.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.955+0000 7f4aa7fff700 1 -- 172.21.15.44:0/46380110 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4022d90 con 0x7f4ac01006d0 2023-09-10T00:40:21.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.955+0000 7f4ac50cc700 1 -- 172.21.15.44:0/46380110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac019e6e0 con 0x7f4ac01006d0 2023-09-10T00:40:21.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.956+0000 7f4ac50cc700 1 -- 172.21.15.44:0/46380110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac010abb0 con 0x7f4ac01006d0 2023-09-10T00:40:21.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.957+0000 7f4aa7fff700 1 -- 172.21.15.44:0/46380110 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f4ab4003fd0 con 0x7f4ac01006d0 2023-09-10T00:40:21.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.958+0000 7f4aa7fff700 1 --2- 172.21.15.44:0/46380110 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4aa80a0690 0x7f4aa80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:21.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.958+0000 7f4aa7fff700 1 -- 172.21.15.44:0/46380110 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f4ab40cd510 con 0x7f4ac01006d0 2023-09-10T00:40:21.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.958+0000 7f4abed9d700 1 --2- 172.21.15.44:0/46380110 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4aa80a0690 0x7f4aa80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:21.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.958+0000 7f4abed9d700 1 --2- 172.21.15.44:0/46380110 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4aa80a0690 0x7f4aa80a2b50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0005950 tx=0x7f4ab00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:21.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:21.962+0000 7f4aa7fff700 1 -- 172.21.15.44:0/46380110 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4ab4096e30 con 0x7f4ac01006d0 2023-09-10T00:40:22.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:21 smithi131 ceph-mon[103713]: pgmap v323: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 895 B/s wr, 11 op/s; 7.2 KiB/s, 1 keys/s, 1 objects/s recovering 2023-09-10T00:40:22.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:21 smithi099 ceph-mon[102692]: pgmap v323: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 895 B/s wr, 11 op/s; 7.2 KiB/s, 1 keys/s, 1 objects/s recovering 2023-09-10T00:40:22.167 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.167+0000 7f4ac50cc700 1 -- 172.21.15.44:0/46380110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ac019e900 con 0x7f4ac01006d0 2023-09-10T00:40:22.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.169+0000 7f4aa7fff700 1 -- 172.21.15.44:0/46380110 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24678 (secure 0 0 0) 0x7f4ab4027090 con 0x7f4ac01006d0 2023-09-10T00:40:22.169 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:22.170 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":91,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:40:17.671364+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:40:22.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.171+0000 7f4ac50cc700 1 -- 172.21.15.44:0/46380110 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4aa80a0690 msgr2=0x7f4aa80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:22.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.171+0000 7f4ac50cc700 1 --2- 172.21.15.44:0/46380110 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4aa80a0690 0x7f4aa80a2b50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0005950 tx=0x7f4ab00058e0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.171+0000 7f4ac50cc700 1 -- 172.21.15.44:0/46380110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ac01006d0 msgr2=0x7f4ac019aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:22.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.171+0000 7f4ac50cc700 1 --2- 172.21.15.44:0/46380110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ac01006d0 0x7f4ac019aee0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4005910 tx=0x7f4ab4003a50 comp rx=0 tx=0).stop 2023-09-10T00:40:22.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.171+0000 7f4ac50cc700 1 -- 172.21.15.44:0/46380110 shutdown_connections 2023-09-10T00:40:22.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.171+0000 7f4ac50cc700 1 --2- 172.21.15.44:0/46380110 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f4aa80a0690 0x7f4aa80a2b50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.173 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.171+0000 7f4ac50cc700 1 --2- 172.21.15.44:0/46380110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac0101070 0x7f4ac019d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.173 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.171+0000 7f4ac50cc700 1 --2- 172.21.15.44:0/46380110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ac00ff4e0 0x7f4ac00682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.173 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.171+0000 7f4ac50cc700 1 --2- 172.21.15.44:0/46380110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ac01006d0 0x7f4ac019aee0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.173 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.171+0000 7f4ac50cc700 1 -- 172.21.15.44:0/46380110 >> 172.21.15.44:0/46380110 conn(0x7f4ac00faa30 msgr2=0x7f4ac00fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:22.173 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.171+0000 7f4ac50cc700 1 -- 172.21.15.44:0/46380110 shutdown_connections 2023-09-10T00:40:22.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.171+0000 7f4ac50cc700 1 -- 172.21.15.44:0/46380110 wait complete. 2023-09-10T00:40:22.184 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-10T00:34:44.456741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-10T00:38:37.194891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-10T00:38:38.573268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-10T00:38:42.198896+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-10T00:40:22.185 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-10T00:40:22.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.353+0000 7f1109f9a700 1 -- 172.21.15.44:0/1617943693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1104101080 msgr2=0x7f1104107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:22.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.353+0000 7f1109f9a700 1 --2- 172.21.15.44:0/1617943693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1104101080 0x7f1104107930 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00b0d0 tx=0x7f10fc00b3e0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.353+0000 7f1109f9a700 1 -- 172.21.15.44:0/1617943693 shutdown_connections 2023-09-10T00:40:22.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.353+0000 7f1109f9a700 1 --2- 172.21.15.44:0/1617943693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1104101080 0x7f1104107930 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.353+0000 7f1109f9a700 1 --2- 172.21.15.44:0/1617943693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11040ff4f0 0x7f11040ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.353+0000 7f1109f9a700 1 --2- 172.21.15.44:0/1617943693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11041006e0 0x7f1104100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.353+0000 7f1109f9a700 1 -- 172.21.15.44:0/1617943693 >> 172.21.15.44:0/1617943693 conn(0x7f11040faa60 msgr2=0x7f11040fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:22.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.353+0000 7f1109f9a700 1 -- 172.21.15.44:0/1617943693 shutdown_connections 2023-09-10T00:40:22.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.353+0000 7f1109f9a700 1 -- 172.21.15.44:0/1617943693 wait complete. 2023-09-10T00:40:22.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.354+0000 7f1109f9a700 1 Processor -- start 2023-09-10T00:40:22.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.354+0000 7f1109f9a700 1 -- start start 2023-09-10T00:40:22.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.354+0000 7f1109f9a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11040ff4f0 0x7f1104196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.354+0000 7f1109f9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11041006e0 0x7f1104196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.354+0000 7f1109f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1104101080 0x7f110419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.354+0000 7f1109f9a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110419dea0 con 0x7f11040ff4f0 2023-09-10T00:40:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.354+0000 7f1109f9a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110419dfe0 con 0x7f11041006e0 2023-09-10T00:40:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.354+0000 7f1109f9a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110419e170 con 0x7f1104101080 2023-09-10T00:40:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.354+0000 7f11037fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11040ff4f0 0x7f1104196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.354+0000 7f11037fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11040ff4f0 0x7f1104196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48262/0 (socket says 172.21.15.44:48262) 2023-09-10T00:40:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.354+0000 7f11037fe700 1 -- 172.21.15.44:0/1886955602 learned_addr learned my addr 172.21.15.44:0/1886955602 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.354+0000 7f1102ffd700 1 --2- 172.21.15.44:0/1886955602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11041006e0 0x7f1104196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.354+0000 7f1103fff700 1 --2- 172.21.15.44:0/1886955602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1104101080 0x7f110419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.355+0000 7f11037fe700 1 -- 172.21.15.44:0/1886955602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1104101080 msgr2=0x7f110419d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.355+0000 7f11037fe700 1 --2- 172.21.15.44:0/1886955602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1104101080 0x7f110419d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.355+0000 7f11037fe700 1 -- 172.21.15.44:0/1886955602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11041006e0 msgr2=0x7f1104196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.355+0000 7f11037fe700 1 --2- 172.21.15.44:0/1886955602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11041006e0 0x7f1104196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.355+0000 7f11037fe700 1 -- 172.21.15.44:0/1886955602 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc009d00 con 0x7f11040ff4f0 2023-09-10T00:40:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.355+0000 7f11037fe700 1 --2- 172.21.15.44:0/1886955602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11040ff4f0 0x7f1104196320 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f10f400e970 tx=0x7f10f400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:22.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.355+0000 7f1100ff9700 1 -- 172.21.15.44:0/1886955602 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4009790 con 0x7f11040ff4f0 2023-09-10T00:40:22.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.355+0000 7f1109f9a700 1 -- 172.21.15.44:0/1886955602 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110419e450 con 0x7f11040ff4f0 2023-09-10T00:40:22.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.355+0000 7f1100ff9700 1 -- 172.21.15.44:0/1886955602 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10f400c7e0 con 0x7f11040ff4f0 2023-09-10T00:40:22.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.355+0000 7f1100ff9700 1 -- 172.21.15.44:0/1886955602 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4003bd0 con 0x7f11040ff4f0 2023-09-10T00:40:22.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.355+0000 7f1109f9a700 1 -- 172.21.15.44:0/1886955602 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110419e9b0 con 0x7f11040ff4f0 2023-09-10T00:40:22.362 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.356+0000 7f1109f9a700 1 -- 172.21.15.44:0/1886955602 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110410ab70 con 0x7f11040ff4f0 2023-09-10T00:40:22.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.357+0000 7f1100ff9700 1 -- 172.21.15.44:0/1886955602 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f10f400c950 con 0x7f11040ff4f0 2023-09-10T00:40:22.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.358+0000 7f1100ff9700 1 --2- 172.21.15.44:0/1886955602 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f10f00a0640 0x7f10f00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:22.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.358+0000 7f1100ff9700 1 -- 172.21.15.44:0/1886955602 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f10f4014070 con 0x7f11040ff4f0 2023-09-10T00:40:22.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.358+0000 7f1102ffd700 1 --2- 172.21.15.44:0/1886955602 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f10f00a0640 0x7f10f00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:22.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.358+0000 7f1102ffd700 1 --2- 172.21.15.44:0/1886955602 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f10f00a0640 0x7f10f00a2b00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f10ec005950 tx=0x7f10ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:22.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.362+0000 7f1100ff9700 1 -- 172.21.15.44:0/1886955602 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f10f40958b0 con 0x7f11040ff4f0 2023-09-10T00:40:22.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.563+0000 7f1109f9a700 1 -- 172.21.15.44:0/1886955602 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f110419ec30 con 0x7f11040ff4f0 2023-09-10T00:40:22.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.563+0000 7f1100ff9700 1 -- 172.21.15.44:0/1886955602 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f10f4098ed0 con 0x7f11040ff4f0 2023-09-10T00:40:22.564 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2023-09-10T00:40:22.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.565+0000 7f1109f9a700 1 -- 172.21.15.44:0/1886955602 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f10f00a0640 msgr2=0x7f10f00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:22.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.565+0000 7f1109f9a700 1 --2- 172.21.15.44:0/1886955602 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f10f00a0640 0x7f10f00a2b00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f10ec005950 tx=0x7f10ec00b410 comp rx=0 tx=0).stop 2023-09-10T00:40:22.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.565+0000 7f1109f9a700 1 -- 172.21.15.44:0/1886955602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11040ff4f0 msgr2=0x7f1104196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:22.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.565+0000 7f1109f9a700 1 --2- 172.21.15.44:0/1886955602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11040ff4f0 0x7f1104196320 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f10f400e970 tx=0x7f10f400ec80 comp rx=0 tx=0).stop 2023-09-10T00:40:22.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.565+0000 7f1109f9a700 1 -- 172.21.15.44:0/1886955602 shutdown_connections 2023-09-10T00:40:22.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.565+0000 7f1109f9a700 1 --2- 172.21.15.44:0/1886955602 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f10f00a0640 0x7f10f00a2b00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.565+0000 7f1109f9a700 1 --2- 172.21.15.44:0/1886955602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1104101080 0x7f110419d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.565+0000 7f1109f9a700 1 --2- 172.21.15.44:0/1886955602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11040ff4f0 0x7f1104196320 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.565+0000 7f1109f9a700 1 --2- 172.21.15.44:0/1886955602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11041006e0 0x7f1104196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.565+0000 7f1109f9a700 1 -- 172.21.15.44:0/1886955602 >> 172.21.15.44:0/1886955602 conn(0x7f11040faa60 msgr2=0x7f1104067760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:22.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.565+0000 7f1109f9a700 1 -- 172.21.15.44:0/1886955602 shutdown_connections 2023-09-10T00:40:22.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.566+0000 7f1109f9a700 1 -- 172.21.15.44:0/1886955602 wait complete. 2023-09-10T00:40:22.578 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-10T00:40:22.712 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.709+0000 7f017f207700 1 -- 172.21.15.44:0/2819335656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01780ff530 msgr2=0x7f01780ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:22.712 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.709+0000 7f017f207700 1 --2- 172.21.15.44:0/2819335656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01780ff530 0x7f01780ff950 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f016800b0d0 tx=0x7f016800b3e0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.713 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.709+0000 7f017f207700 1 -- 172.21.15.44:0/2819335656 shutdown_connections 2023-09-10T00:40:22.713 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.709+0000 7f017f207700 1 --2- 172.21.15.44:0/2819335656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01781010c0 0x7f0178107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.713 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.709+0000 7f017f207700 1 --2- 172.21.15.44:0/2819335656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01780ff530 0x7f01780ff950 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.713 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.709+0000 7f017f207700 1 --2- 172.21.15.44:0/2819335656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0178100720 0x7f0178100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.714 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.709+0000 7f017f207700 1 -- 172.21.15.44:0/2819335656 >> 172.21.15.44:0/2819335656 conn(0x7f01780faac0 msgr2=0x7f01780fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:22.714 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.709+0000 7f017f207700 1 -- 172.21.15.44:0/2819335656 shutdown_connections 2023-09-10T00:40:22.714 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.709+0000 7f017f207700 1 -- 172.21.15.44:0/2819335656 wait complete. 2023-09-10T00:40:22.714 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.710+0000 7f017f207700 1 Processor -- start 2023-09-10T00:40:22.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.710+0000 7f017f207700 1 -- start start 2023-09-10T00:40:22.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.710+0000 7f017f207700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01780ff530 0x7f0178196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:22.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.710+0000 7f017f207700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0178100720 0x7f0178196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:22.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.710+0000 7f017f207700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01781010c0 0x7f017819d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:22.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.710+0000 7f017f207700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017819df90 con 0x7f0178100720 2023-09-10T00:40:22.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.710+0000 7f017f207700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017819e0d0 con 0x7f01781010c0 2023-09-10T00:40:22.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.710+0000 7f017f207700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017819e260 con 0x7f01780ff530 2023-09-10T00:40:22.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.710+0000 7f0177fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0178100720 0x7f0178196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:22.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.710+0000 7f0177fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0178100720 0x7f0178196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48276/0 (socket says 172.21.15.44:48276) 2023-09-10T00:40:22.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.710+0000 7f0177fff700 1 -- 172.21.15.44:0/941860795 learned_addr learned my addr 172.21.15.44:0/941860795 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:22.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.710+0000 7f017d7a4700 1 --2- 172.21.15.44:0/941860795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01781010c0 0x7f017819d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:22.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.710+0000 7f017cfa3700 1 --2- 172.21.15.44:0/941860795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01780ff530 0x7f0178196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:22.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.711+0000 7f0177fff700 1 -- 172.21.15.44:0/941860795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01780ff530 msgr2=0x7f0178196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:22.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.711+0000 7f0177fff700 1 --2- 172.21.15.44:0/941860795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01780ff530 0x7f0178196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.711+0000 7f0177fff700 1 -- 172.21.15.44:0/941860795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01781010c0 msgr2=0x7f017819d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:22.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.711+0000 7f0177fff700 1 --2- 172.21.15.44:0/941860795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01781010c0 0x7f017819d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.711+0000 7f0177fff700 1 -- 172.21.15.44:0/941860795 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0168009d00 con 0x7f0178100720 2023-09-10T00:40:22.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.711+0000 7f0177fff700 1 --2- 172.21.15.44:0/941860795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0178100720 0x7f0178196950 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f016c00e9a0 tx=0x7f016c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:22.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.711+0000 7f0175ffb700 1 -- 172.21.15.44:0/941860795 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c009790 con 0x7f0178100720 2023-09-10T00:40:22.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.711+0000 7f0175ffb700 1 -- 172.21.15.44:0/941860795 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f016c010430 con 0x7f0178100720 2023-09-10T00:40:22.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.711+0000 7f017f207700 1 -- 172.21.15.44:0/941860795 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017819e540 con 0x7f0178100720 2023-09-10T00:40:22.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.711+0000 7f0175ffb700 1 -- 172.21.15.44:0/941860795 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c00cea0 con 0x7f0178100720 2023-09-10T00:40:22.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.711+0000 7f017f207700 1 -- 172.21.15.44:0/941860795 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017819ea50 con 0x7f0178100720 2023-09-10T00:40:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.712+0000 7f017f207700 1 -- 172.21.15.44:0/941860795 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f017810bcf0 con 0x7f0178100720 2023-09-10T00:40:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.713+0000 7f0175ffb700 1 -- 172.21.15.44:0/941860795 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f016c0105a0 con 0x7f0178100720 2023-09-10T00:40:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.714+0000 7f0175ffb700 1 --2- 172.21.15.44:0/941860795 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f01600a0640 0x7f01600a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.714+0000 7f017cfa3700 1 --2- 172.21.15.44:0/941860795 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f01600a0640 0x7f01600a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.714+0000 7f0175ffb700 1 -- 172.21.15.44:0/941860795 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f016c014070 con 0x7f0178100720 2023-09-10T00:40:22.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.714+0000 7f017cfa3700 1 --2- 172.21.15.44:0/941860795 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f01600a0640 0x7f01600a2b00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f01680062a0 tx=0x7f016800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:22.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.717+0000 7f0175ffb700 1 -- 172.21.15.44:0/941860795 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f016c095a70 con 0x7f0178100720 2023-09-10T00:40:22.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.921+0000 7f017f207700 1 -- 172.21.15.44:0/941860795 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0178106550 con 0x7f0178100720 2023-09-10T00:40:22.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.921+0000 7f0175ffb700 1 -- 172.21.15.44:0/941860795 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f016c026020 con 0x7f0178100720 2023-09-10T00:40:22.922 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:40:22.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.923+0000 7f017f207700 1 -- 172.21.15.44:0/941860795 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f01600a0640 msgr2=0x7f01600a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:22.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.923+0000 7f017f207700 1 --2- 172.21.15.44:0/941860795 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f01600a0640 0x7f01600a2b00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f01680062a0 tx=0x7f016800b040 comp rx=0 tx=0).stop 2023-09-10T00:40:22.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.923+0000 7f017f207700 1 -- 172.21.15.44:0/941860795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0178100720 msgr2=0x7f0178196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:22.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.923+0000 7f017f207700 1 --2- 172.21.15.44:0/941860795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0178100720 0x7f0178196950 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f016c00e9a0 tx=0x7f016c00ecb0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.923+0000 7f017f207700 1 -- 172.21.15.44:0/941860795 shutdown_connections 2023-09-10T00:40:22.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.923+0000 7f017f207700 1 --2- 172.21.15.44:0/941860795 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f01600a0640 0x7f01600a2b00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.923+0000 7f017f207700 1 --2- 172.21.15.44:0/941860795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01781010c0 0x7f017819d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.923+0000 7f017f207700 1 --2- 172.21.15.44:0/941860795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01780ff530 0x7f0178196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.923+0000 7f017f207700 1 --2- 172.21.15.44:0/941860795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0178100720 0x7f0178196950 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:22.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.923+0000 7f017f207700 1 -- 172.21.15.44:0/941860795 >> 172.21.15.44:0/941860795 conn(0x7f01780faac0 msgr2=0x7f01780fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:22.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.923+0000 7f017f207700 1 -- 172.21.15.44:0/941860795 shutdown_connections 2023-09-10T00:40:22.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:22.924+0000 7f017f207700 1 -- 172.21.15.44:0/941860795 wait complete. 2023-09-10T00:40:22.935 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:22 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/46380110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:22.935 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:22 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1886955602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:40:22.936 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-10T00:40:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:22 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/46380110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:22 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1886955602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:40:23.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.065+0000 7fb7c10b6700 1 -- 172.21.15.44:0/3498564317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc0ff4f0 msgr2=0x7fb7bc0ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.065+0000 7fb7c10b6700 1 --2- 172.21.15.44:0/3498564317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc0ff4f0 0x7fb7bc0ff910 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00099c0 tx=0x7fb7b0009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.065+0000 7fb7c10b6700 1 -- 172.21.15.44:0/3498564317 shutdown_connections 2023-09-10T00:40:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.065+0000 7fb7c10b6700 1 --2- 172.21.15.44:0/3498564317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc101080 0x7fb7bc107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.065+0000 7fb7c10b6700 1 --2- 172.21.15.44:0/3498564317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc0ff4f0 0x7fb7bc0ff910 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.065+0000 7fb7c10b6700 1 --2- 172.21.15.44:0/3498564317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7bc1006e0 0x7fb7bc100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.065+0000 7fb7c10b6700 1 -- 172.21.15.44:0/3498564317 >> 172.21.15.44:0/3498564317 conn(0x7fb7bc0faa60 msgr2=0x7fb7bc0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.065+0000 7fb7c10b6700 1 -- 172.21.15.44:0/3498564317 shutdown_connections 2023-09-10T00:40:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.065+0000 7fb7c10b6700 1 -- 172.21.15.44:0/3498564317 wait complete. 2023-09-10T00:40:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.065+0000 7fb7c10b6700 1 Processor -- start 2023-09-10T00:40:23.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7c10b6700 1 -- start start 2023-09-10T00:40:23.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7c10b6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc0ff4f0 0x7fb7bc196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:23.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7c10b6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc1006e0 0x7fb7bc1967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:23.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7c10b6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7bc101080 0x7fb7bc19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7c10b6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc19de00 con 0x7fb7bc1006e0 2023-09-10T00:40:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7c10b6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc19df40 con 0x7fb7bc0ff4f0 2023-09-10T00:40:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7c10b6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc19e0d0 con 0x7fb7bc101080 2023-09-10T00:40:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7ba59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc1006e0 0x7fb7bc1967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7ba59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc1006e0 0x7fb7bc1967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48290/0 (socket says 172.21.15.44:48290) 2023-09-10T00:40:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7ba59c700 1 -- 172.21.15.44:0/4127439502 learned_addr learned my addr 172.21.15.44:0/4127439502 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7bad9d700 1 --2- 172.21.15.44:0/4127439502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc0ff4f0 0x7fb7bc196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7bb59e700 1 --2- 172.21.15.44:0/4127439502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7bc101080 0x7fb7bc19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7ba59c700 1 -- 172.21.15.44:0/4127439502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7bc101080 msgr2=0x7fb7bc19d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7ba59c700 1 --2- 172.21.15.44:0/4127439502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7bc101080 0x7fb7bc19d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7ba59c700 1 -- 172.21.15.44:0/4127439502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc0ff4f0 msgr2=0x7fb7bc196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7ba59c700 1 --2- 172.21.15.44:0/4127439502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc0ff4f0 0x7fb7bc196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7ba59c700 1 -- 172.21.15.44:0/4127439502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b00095f0 con 0x7fb7bc1006e0 2023-09-10T00:40:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7bb59e700 1 --2- 172.21.15.44:0/4127439502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7bc101080 0x7fb7bc19d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.066+0000 7fb7ba59c700 1 --2- 172.21.15.44:0/4127439502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc1006e0 0x7fb7bc1967c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00d950 tx=0x7fb7ac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.067+0000 7fb7a3fff700 1 -- 172.21.15.44:0/4127439502 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac009940 con 0x7fb7bc1006e0 2023-09-10T00:40:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.067+0000 7fb7c10b6700 1 -- 172.21.15.44:0/4127439502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7bc19e3b0 con 0x7fb7bc1006e0 2023-09-10T00:40:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.067+0000 7fb7a3fff700 1 -- 172.21.15.44:0/4127439502 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7ac010460 con 0x7fb7bc1006e0 2023-09-10T00:40:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.067+0000 7fb7a3fff700 1 -- 172.21.15.44:0/4127439502 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00f620 con 0x7fb7bc1006e0 2023-09-10T00:40:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.067+0000 7fb7c10b6700 1 -- 172.21.15.44:0/4127439502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7bc19e8c0 con 0x7fb7bc1006e0 2023-09-10T00:40:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.068+0000 7fb7c10b6700 1 -- 172.21.15.44:0/4127439502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7bc10ac10 con 0x7fb7bc1006e0 2023-09-10T00:40:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.069+0000 7fb7a3fff700 1 -- 172.21.15.44:0/4127439502 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fb7ac0105d0 con 0x7fb7bc1006e0 2023-09-10T00:40:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.069+0000 7fb7a3fff700 1 --2- 172.21.15.44:0/4127439502 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb7a40a0640 0x7fb7a40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.069+0000 7fb7a3fff700 1 -- 172.21.15.44:0/4127439502 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7fb7ac0cbd50 con 0x7fb7bc1006e0 2023-09-10T00:40:23.075 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.069+0000 7fb7bad9d700 1 --2- 172.21.15.44:0/4127439502 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb7a40a0640 0x7fb7a40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.070+0000 7fb7bad9d700 1 --2- 172.21.15.44:0/4127439502 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb7a40a0640 0x7fb7a40a2b00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0000c00 tx=0x7fb7b0009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.073+0000 7fb7a3fff700 1 -- 172.21.15.44:0/4127439502 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb7ac0954c0 con 0x7fb7bc1006e0 2023-09-10T00:40:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:22 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/46380110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:22 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1886955602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:40:23.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.273+0000 7fb7c10b6700 1 -- 172.21.15.44:0/4127439502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb7bc19eae0 con 0x7fb7bc1006e0 2023-09-10T00:40:23.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.274+0000 7fb7a3fff700 1 -- 172.21.15.44:0/4127439502 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb7ac020020 con 0x7fb7bc1006e0 2023-09-10T00:40:23.275 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2023-09-10T00:40:23.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.276+0000 7fb7c10b6700 1 -- 172.21.15.44:0/4127439502 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb7a40a0640 msgr2=0x7fb7a40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:23.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.276+0000 7fb7c10b6700 1 --2- 172.21.15.44:0/4127439502 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb7a40a0640 0x7fb7a40a2b00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0000c00 tx=0x7fb7b0009f10 comp rx=0 tx=0).stop 2023-09-10T00:40:23.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.276+0000 7fb7c10b6700 1 -- 172.21.15.44:0/4127439502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc1006e0 msgr2=0x7fb7bc1967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:23.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.276+0000 7fb7c10b6700 1 --2- 172.21.15.44:0/4127439502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc1006e0 0x7fb7bc1967c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00d950 tx=0x7fb7ac00dc60 comp rx=0 tx=0).stop 2023-09-10T00:40:23.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.276+0000 7fb7c10b6700 1 -- 172.21.15.44:0/4127439502 shutdown_connections 2023-09-10T00:40:23.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.276+0000 7fb7c10b6700 1 --2- 172.21.15.44:0/4127439502 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb7a40a0640 0x7fb7a40a2b00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.276+0000 7fb7c10b6700 1 --2- 172.21.15.44:0/4127439502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7bc101080 0x7fb7bc19d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.276+0000 7fb7c10b6700 1 --2- 172.21.15.44:0/4127439502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc0ff4f0 0x7fb7bc196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.276+0000 7fb7c10b6700 1 --2- 172.21.15.44:0/4127439502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc1006e0 0x7fb7bc1967c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.276+0000 7fb7c10b6700 1 -- 172.21.15.44:0/4127439502 >> 172.21.15.44:0/4127439502 conn(0x7fb7bc0faa60 msgr2=0x7fb7bc0677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:23.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.276+0000 7fb7c10b6700 1 -- 172.21.15.44:0/4127439502 shutdown_connections 2023-09-10T00:40:23.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.276+0000 7fb7c10b6700 1 -- 172.21.15.44:0/4127439502 wait complete. 2023-09-10T00:40:23.288 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-10T00:40:23.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.418+0000 7f5d39598700 1 -- 172.21.15.44:0/3502244725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d34103a60 msgr2=0x7f5d34105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:23.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.418+0000 7f5d39598700 1 --2- 172.21.15.44:0/3502244725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d34103a60 0x7f5d34105e50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009990 tx=0x7f5d28009ca0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.418+0000 7f5d39598700 1 -- 172.21.15.44:0/3502244725 shutdown_connections 2023-09-10T00:40:23.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.418+0000 7f5d39598700 1 --2- 172.21.15.44:0/3502244725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d34106390 0x7f5d341087a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.418+0000 7f5d39598700 1 --2- 172.21.15.44:0/3502244725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d34101130 0x7f5d34103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.418+0000 7f5d39598700 1 --2- 172.21.15.44:0/3502244725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d34103a60 0x7f5d34105e50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.418+0000 7f5d39598700 1 -- 172.21.15.44:0/3502244725 >> 172.21.15.44:0/3502244725 conn(0x7f5d340fab00 msgr2=0x7f5d340fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:23.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.418+0000 7f5d39598700 1 -- 172.21.15.44:0/3502244725 shutdown_connections 2023-09-10T00:40:23.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.418+0000 7f5d39598700 1 -- 172.21.15.44:0/3502244725 wait complete. 2023-09-10T00:40:23.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.418+0000 7f5d39598700 1 Processor -- start 2023-09-10T00:40:23.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.418+0000 7f5d39598700 1 -- start start 2023-09-10T00:40:23.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d39598700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d34101130 0x7f5d34100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:23.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d39598700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d34103a60 0x7f5d340ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:23.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d39598700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d34106390 0x7f5d340ff620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d39598700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d340ffdc0 con 0x7f5d34101130 2023-09-10T00:40:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d39598700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d340fff30 con 0x7f5d34103a60 2023-09-10T00:40:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d39598700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d341a0270 con 0x7f5d34106390 2023-09-10T00:40:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d32ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d34101130 0x7f5d34100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d32ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d34101130 0x7f5d34100aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48302/0 (socket says 172.21.15.44:48302) 2023-09-10T00:40:23.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d32ffd700 1 -- 172.21.15.44:0/3280972764 learned_addr learned my addr 172.21.15.44:0/3280972764 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:23.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d327fc700 1 --2- 172.21.15.44:0/3280972764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d34103a60 0x7f5d340ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:23.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d337fe700 1 --2- 172.21.15.44:0/3280972764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d34106390 0x7f5d340ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:23.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d32ffd700 1 -- 172.21.15.44:0/3280972764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d34106390 msgr2=0x7f5d340ff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:23.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d32ffd700 1 --2- 172.21.15.44:0/3280972764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d34106390 0x7f5d340ff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d32ffd700 1 -- 172.21.15.44:0/3280972764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d34103a60 msgr2=0x7f5d340ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:23.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d32ffd700 1 --2- 172.21.15.44:0/3280972764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d34103a60 0x7f5d340ff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d32ffd700 1 -- 172.21.15.44:0/3280972764 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d280095f0 con 0x7f5d34101130 2023-09-10T00:40:23.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d32ffd700 1 --2- 172.21.15.44:0/3280972764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d34101130 0x7f5d34100aa0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400e9a0 tx=0x7f5d2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:23.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.419+0000 7f5d1bfff700 1 -- 172.21.15.44:0/3280972764 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d24009790 con 0x7f5d34101130 2023-09-10T00:40:23.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.420+0000 7f5d1bfff700 1 -- 172.21.15.44:0/3280972764 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d2400cbd0 con 0x7f5d34101130 2023-09-10T00:40:23.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.420+0000 7f5d1bfff700 1 -- 172.21.15.44:0/3280972764 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d24018680 con 0x7f5d34101130 2023-09-10T00:40:23.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.420+0000 7f5d39598700 1 -- 172.21.15.44:0/3280972764 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d341a0550 con 0x7f5d34101130 2023-09-10T00:40:23.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.420+0000 7f5d39598700 1 -- 172.21.15.44:0/3280972764 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d341a0a10 con 0x7f5d34101130 2023-09-10T00:40:23.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.421+0000 7f5d39598700 1 -- 172.21.15.44:0/3280972764 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d340fc680 con 0x7f5d34101130 2023-09-10T00:40:23.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.422+0000 7f5d1bfff700 1 -- 172.21.15.44:0/3280972764 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f5d2400cd40 con 0x7f5d34101130 2023-09-10T00:40:23.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.422+0000 7f5d1bfff700 1 --2- 172.21.15.44:0/3280972764 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5d1c0a0640 0x7f5d1c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:23.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.422+0000 7f5d327fc700 1 --2- 172.21.15.44:0/3280972764 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5d1c0a0640 0x7f5d1c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:23.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.422+0000 7f5d1bfff700 1 -- 172.21.15.44:0/3280972764 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f5d24014070 con 0x7f5d34101130 2023-09-10T00:40:23.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.423+0000 7f5d327fc700 1 --2- 172.21.15.44:0/3280972764 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5d1c0a0640 0x7f5d1c0a2b00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5d28006010 tx=0x7f5d28005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:23.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.426+0000 7f5d1bfff700 1 -- 172.21.15.44:0/3280972764 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5d24051e10 con 0x7f5d34101130 2023-09-10T00:40:23.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.633+0000 7f5d39598700 1 -- 172.21.15.44:0/3280972764 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5d3402cf90 con 0x7f5d34101130 2023-09-10T00:40:23.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.633+0000 7f5d1bfff700 1 -- 172.21.15.44:0/3280972764 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5d24095970 con 0x7f5d34101130 2023-09-10T00:40:23.634 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:40:23.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.635+0000 7f5d39598700 1 -- 172.21.15.44:0/3280972764 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5d1c0a0640 msgr2=0x7f5d1c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:23.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.635+0000 7f5d39598700 1 --2- 172.21.15.44:0/3280972764 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5d1c0a0640 0x7f5d1c0a2b00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5d28006010 tx=0x7f5d28005960 comp rx=0 tx=0).stop 2023-09-10T00:40:23.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.635+0000 7f5d39598700 1 -- 172.21.15.44:0/3280972764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d34101130 msgr2=0x7f5d34100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:23.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.635+0000 7f5d39598700 1 --2- 172.21.15.44:0/3280972764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d34101130 0x7f5d34100aa0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400e9a0 tx=0x7f5d2400ecb0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.636+0000 7f5d39598700 1 -- 172.21.15.44:0/3280972764 shutdown_connections 2023-09-10T00:40:23.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.636+0000 7f5d39598700 1 --2- 172.21.15.44:0/3280972764 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f5d1c0a0640 0x7f5d1c0a2b00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.636+0000 7f5d39598700 1 --2- 172.21.15.44:0/3280972764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d34106390 0x7f5d340ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.636+0000 7f5d39598700 1 --2- 172.21.15.44:0/3280972764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d34101130 0x7f5d34100aa0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.636+0000 7f5d39598700 1 --2- 172.21.15.44:0/3280972764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d34103a60 0x7f5d340ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.636+0000 7f5d39598700 1 -- 172.21.15.44:0/3280972764 >> 172.21.15.44:0/3280972764 conn(0x7f5d340fab00 msgr2=0x7f5d34101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:23.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.636+0000 7f5d39598700 1 -- 172.21.15.44:0/3280972764 shutdown_connections 2023-09-10T00:40:23.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.636+0000 7f5d39598700 1 -- 172.21.15.44:0/3280972764 wait complete. 2023-09-10T00:40:23.648 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:40:23.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.776+0000 7f92680cf700 1 -- 172.21.15.44:0/3622873920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92601006d0 msgr2=0x7f9260100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:23.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.776+0000 7f92680cf700 1 --2- 172.21.15.44:0/3622873920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92601006d0 0x7f9260100b30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f92500099c0 tx=0x7f9250009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.776+0000 7f92680cf700 1 -- 172.21.15.44:0/3622873920 shutdown_connections 2023-09-10T00:40:23.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.776+0000 7f92680cf700 1 --2- 172.21.15.44:0/3622873920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9260101070 0x7f9260107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.776+0000 7f92680cf700 1 --2- 172.21.15.44:0/3622873920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92600ff4e0 0x7f92600ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.776+0000 7f92680cf700 1 --2- 172.21.15.44:0/3622873920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92601006d0 0x7f9260100b30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.776+0000 7f92680cf700 1 -- 172.21.15.44:0/3622873920 >> 172.21.15.44:0/3622873920 conn(0x7f92600faa30 msgr2=0x7f92600fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:23.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.776+0000 7f92680cf700 1 -- 172.21.15.44:0/3622873920 shutdown_connections 2023-09-10T00:40:23.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.776+0000 7f92680cf700 1 -- 172.21.15.44:0/3622873920 wait complete. 2023-09-10T00:40:23.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f92680cf700 1 Processor -- start 2023-09-10T00:40:23.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f92680cf700 1 -- start start 2023-09-10T00:40:23.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f92680cf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92600ff4e0 0x7f92600682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:23.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f92680cf700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92601006d0 0x7f926019afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:23.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f92680cf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9260101070 0x7f926019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:23.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f92680cf700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926019ddc0 con 0x7f9260101070 2023-09-10T00:40:23.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f92680cf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926019df30 con 0x7f92600ff4e0 2023-09-10T00:40:23.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f92680cf700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926019e0f0 con 0x7f92601006d0 2023-09-10T00:40:23.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f9265e6b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92600ff4e0 0x7f92600682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:23.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f926666c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9260101070 0x7f926019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:23.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f9265e6b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92600ff4e0 0x7f92600682d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:50054/0 (socket says 172.21.15.44:50054) 2023-09-10T00:40:23.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f9265e6b700 1 -- 172.21.15.44:0/1178411167 learned_addr learned my addr 172.21.15.44:0/1178411167 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:23.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f926566a700 1 --2- 172.21.15.44:0/1178411167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92601006d0 0x7f926019afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:23.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f926666c700 1 -- 172.21.15.44:0/1178411167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92601006d0 msgr2=0x7f926019afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:23.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f926666c700 1 --2- 172.21.15.44:0/1178411167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92601006d0 0x7f926019afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f926666c700 1 -- 172.21.15.44:0/1178411167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92600ff4e0 msgr2=0x7f92600682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:23.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f926666c700 1 --2- 172.21.15.44:0/1178411167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92600ff4e0 0x7f92600682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.777+0000 7f926666c700 1 -- 172.21.15.44:0/1178411167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92500095f0 con 0x7f9260101070 2023-09-10T00:40:23.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.778+0000 7f926666c700 1 --2- 172.21.15.44:0/1178411167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9260101070 0x7f926019d620 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f925800be20 tx=0x7f9258009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:23.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.778+0000 7f9256ffd700 1 -- 172.21.15.44:0/1178411167 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925800cb90 con 0x7f9260101070 2023-09-10T00:40:23.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.778+0000 7f92680cf700 1 -- 172.21.15.44:0/1178411167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926019e3d0 con 0x7f9260101070 2023-09-10T00:40:23.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.778+0000 7f9256ffd700 1 -- 172.21.15.44:0/1178411167 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f925800ccf0 con 0x7f9260101070 2023-09-10T00:40:23.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.778+0000 7f9256ffd700 1 -- 172.21.15.44:0/1178411167 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92580125a0 con 0x7f9260101070 2023-09-10T00:40:23.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.778+0000 7f92680cf700 1 -- 172.21.15.44:0/1178411167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926019e890 con 0x7f9260101070 2023-09-10T00:40:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.779+0000 7f92680cf700 1 -- 172.21.15.44:0/1178411167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926010ac70 con 0x7f9260101070 2023-09-10T00:40:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.780+0000 7f9256ffd700 1 -- 172.21.15.44:0/1178411167 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f9258014440 con 0x7f9260101070 2023-09-10T00:40:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.781+0000 7f9256ffd700 1 --2- 172.21.15.44:0/1178411167 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f924c0a0690 0x7f924c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.781+0000 7f9265e6b700 1 --2- 172.21.15.44:0/1178411167 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f924c0a0690 0x7f924c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:23.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.781+0000 7f9256ffd700 1 -- 172.21.15.44:0/1178411167 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f9258006e40 con 0x7f9260101070 2023-09-10T00:40:23.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.781+0000 7f9265e6b700 1 --2- 172.21.15.44:0/1178411167 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f924c0a0690 0x7f924c0a2b50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f925c005950 tx=0x7f925c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:23.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.785+0000 7f9256ffd700 1 -- 172.21.15.44:0/1178411167 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f925804ee10 con 0x7f9260101070 2023-09-10T00:40:23.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.992+0000 7f92680cf700 1 -- 172.21.15.44:0/1178411167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9260106490 con 0x7f9260101070 2023-09-10T00:40:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.994+0000 7f9256ffd700 1 -- 172.21.15.44:0/1178411167 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24678 (secure 0 0 0) 0x7f92580b6e60 con 0x7f9260101070 2023-09-10T00:40:23.995 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:23.995 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":91,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:40:17.671364+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:40:23.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.996+0000 7f92680cf700 1 -- 172.21.15.44:0/1178411167 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f924c0a0690 msgr2=0x7f924c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:23.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.996+0000 7f92680cf700 1 --2- 172.21.15.44:0/1178411167 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f924c0a0690 0x7f924c0a2b50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f925c005950 tx=0x7f925c009260 comp rx=0 tx=0).stop 2023-09-10T00:40:23.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.996+0000 7f92680cf700 1 -- 172.21.15.44:0/1178411167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9260101070 msgr2=0x7f926019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:23.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.996+0000 7f92680cf700 1 --2- 172.21.15.44:0/1178411167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9260101070 0x7f926019d620 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f925800be20 tx=0x7f9258009ed0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.996+0000 7f92680cf700 1 -- 172.21.15.44:0/1178411167 shutdown_connections 2023-09-10T00:40:23.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.996+0000 7f92680cf700 1 --2- 172.21.15.44:0/1178411167 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f924c0a0690 0x7f924c0a2b50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.996+0000 7f92680cf700 1 --2- 172.21.15.44:0/1178411167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9260101070 0x7f926019d620 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.996+0000 7f92680cf700 1 --2- 172.21.15.44:0/1178411167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92600ff4e0 0x7f92600682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.996+0000 7f92680cf700 1 --2- 172.21.15.44:0/1178411167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92601006d0 0x7f926019afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:23.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.996+0000 7f92680cf700 1 -- 172.21.15.44:0/1178411167 >> 172.21.15.44:0/1178411167 conn(0x7f92600faa30 msgr2=0x7f92600fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:23.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.996+0000 7f92680cf700 1 -- 172.21.15.44:0/1178411167 shutdown_connections 2023-09-10T00:40:23.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:23.996+0000 7f92680cf700 1 -- 172.21.15.44:0/1178411167 wait complete. 2023-09-10T00:40:24.007 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:23 smithi044 ceph-mon[98676]: pgmap v324: 129 pgs: 129 active+clean; 642 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 12 op/s; 7.2 KiB/s, 1 keys/s, 1 objects/s recovering 2023-09-10T00:40:24.008 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:23 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/941860795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:40:24.008 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:23 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/4127439502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:40:24.008 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:23 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3280972764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:40:24.009 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-10T00:34:44.456741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-10T00:38:37.194891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-10T00:38:38.573268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-10T00:38:42.198896+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-10T00:40:24.010 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-10T00:40:24.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:23 smithi131 ceph-mon[103713]: pgmap v324: 129 pgs: 129 active+clean; 642 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 12 op/s; 7.2 KiB/s, 1 keys/s, 1 objects/s recovering 2023-09-10T00:40:24.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:23 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/941860795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:40:24.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:23 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/4127439502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:40:24.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:23 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3280972764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:40:24.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:23 smithi099 ceph-mon[102692]: pgmap v324: 129 pgs: 129 active+clean; 642 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 12 op/s; 7.2 KiB/s, 1 keys/s, 1 objects/s recovering 2023-09-10T00:40:24.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:23 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/941860795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:40:24.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:23 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/4127439502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:40:24.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:23 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3280972764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:40:24.141 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.139+0000 7fba3bef2700 1 -- 172.21.15.44:0/1298314272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba341006e0 msgr2=0x7fba34100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:24.141 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.139+0000 7fba3bef2700 1 --2- 172.21.15.44:0/1298314272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba341006e0 0x7fba34100b40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fba240099c0 tx=0x7fba24009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.139+0000 7fba3bef2700 1 -- 172.21.15.44:0/1298314272 shutdown_connections 2023-09-10T00:40:24.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.139+0000 7fba3bef2700 1 --2- 172.21.15.44:0/1298314272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba34101080 0x7fba34107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.139+0000 7fba3bef2700 1 --2- 172.21.15.44:0/1298314272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba340ff4f0 0x7fba340ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.139+0000 7fba3bef2700 1 --2- 172.21.15.44:0/1298314272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba341006e0 0x7fba34100b40 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.139+0000 7fba3bef2700 1 -- 172.21.15.44:0/1298314272 >> 172.21.15.44:0/1298314272 conn(0x7fba340faa60 msgr2=0x7fba340fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:24.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.139+0000 7fba3bef2700 1 -- 172.21.15.44:0/1298314272 shutdown_connections 2023-09-10T00:40:24.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.139+0000 7fba3bef2700 1 -- 172.21.15.44:0/1298314272 wait complete. 2023-09-10T00:40:24.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.139+0000 7fba3bef2700 1 Processor -- start 2023-09-10T00:40:24.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.139+0000 7fba3bef2700 1 -- start start 2023-09-10T00:40:24.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.139+0000 7fba3bef2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba340ff4f0 0x7fba340682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3bef2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba341006e0 0x7fba3419af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3bef2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba34101080 0x7fba3419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3bef2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3419dd90 con 0x7fba34101080 2023-09-10T00:40:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3bef2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3419df00 con 0x7fba340ff4f0 2023-09-10T00:40:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3bef2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3419e0c0 con 0x7fba341006e0 2023-09-10T00:40:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3a48f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba34101080 0x7fba3419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3a48f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba34101080 0x7fba3419d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48332/0 (socket says 172.21.15.44:48332) 2023-09-10T00:40:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3a48f700 1 -- 172.21.15.44:0/2778327089 learned_addr learned my addr 172.21.15.44:0/2778327089 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3948d700 1 --2- 172.21.15.44:0/2778327089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba341006e0 0x7fba3419af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba39c8e700 1 --2- 172.21.15.44:0/2778327089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba340ff4f0 0x7fba340682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3a48f700 1 -- 172.21.15.44:0/2778327089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba341006e0 msgr2=0x7fba3419af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3a48f700 1 --2- 172.21.15.44:0/2778327089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba341006e0 0x7fba3419af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3a48f700 1 -- 172.21.15.44:0/2778327089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba340ff4f0 msgr2=0x7fba340682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3a48f700 1 --2- 172.21.15.44:0/2778327089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba340ff4f0 0x7fba340682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3a48f700 1 -- 172.21.15.44:0/2778327089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba240095f0 con 0x7fba34101080 2023-09-10T00:40:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba39c8e700 1 --2- 172.21.15.44:0/2778327089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba340ff4f0 0x7fba340682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3a48f700 1 --2- 172.21.15.44:0/2778327089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba34101080 0x7fba3419d5c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00ada0 tx=0x7fba2c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba2affd700 1 -- 172.21.15.44:0/2778327089 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c00f070 con 0x7fba34101080 2023-09-10T00:40:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3bef2700 1 -- 172.21.15.44:0/2778327089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3419e3a0 con 0x7fba34101080 2023-09-10T00:40:24.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba2affd700 1 -- 172.21.15.44:0/2778327089 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fba2c007b30 con 0x7fba34101080 2023-09-10T00:40:24.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba2affd700 1 -- 172.21.15.44:0/2778327089 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c005280 con 0x7fba34101080 2023-09-10T00:40:24.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.140+0000 7fba3bef2700 1 -- 172.21.15.44:0/2778327089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3419e830 con 0x7fba34101080 2023-09-10T00:40:24.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.141+0000 7fba3bef2700 1 -- 172.21.15.44:0/2778327089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba341067d0 con 0x7fba34101080 2023-09-10T00:40:24.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.142+0000 7fba2affd700 1 -- 172.21.15.44:0/2778327089 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fba2c00c980 con 0x7fba34101080 2023-09-10T00:40:24.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.143+0000 7fba2affd700 1 --2- 172.21.15.44:0/2778327089 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba200a0640 0x7fba200a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:24.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.143+0000 7fba2affd700 1 -- 172.21.15.44:0/2778327089 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7fba2c0cbbc0 con 0x7fba34101080 2023-09-10T00:40:24.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.143+0000 7fba39c8e700 1 --2- 172.21.15.44:0/2778327089 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba200a0640 0x7fba200a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:24.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.143+0000 7fba39c8e700 1 --2- 172.21.15.44:0/2778327089 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba200a0640 0x7fba200a2b00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fba30005950 tx=0x7fba300058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:24.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.147+0000 7fba2affd700 1 -- 172.21.15.44:0/2778327089 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fba2c0953c0 con 0x7fba34101080 2023-09-10T00:40:24.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.348+0000 7fba3bef2700 1 -- 172.21.15.44:0/2778327089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba3410ad50 con 0x7fba34101080 2023-09-10T00:40:24.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.348+0000 7fba2affd700 1 -- 172.21.15.44:0/2778327089 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba2c00cc30 con 0x7fba34101080 2023-09-10T00:40:24.349 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2023-09-10T00:40:24.351 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.350+0000 7fba3bef2700 1 -- 172.21.15.44:0/2778327089 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba200a0640 msgr2=0x7fba200a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:24.351 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.350+0000 7fba3bef2700 1 --2- 172.21.15.44:0/2778327089 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba200a0640 0x7fba200a2b00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fba30005950 tx=0x7fba300058e0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.351 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.350+0000 7fba3bef2700 1 -- 172.21.15.44:0/2778327089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba34101080 msgr2=0x7fba3419d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:24.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.350+0000 7fba3bef2700 1 --2- 172.21.15.44:0/2778327089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba34101080 0x7fba3419d5c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00ada0 tx=0x7fba2c00c330 comp rx=0 tx=0).stop 2023-09-10T00:40:24.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.350+0000 7fba3bef2700 1 -- 172.21.15.44:0/2778327089 shutdown_connections 2023-09-10T00:40:24.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.351+0000 7fba3bef2700 1 --2- 172.21.15.44:0/2778327089 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fba200a0640 0x7fba200a2b00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.351+0000 7fba3bef2700 1 --2- 172.21.15.44:0/2778327089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba34101080 0x7fba3419d5c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.351+0000 7fba3bef2700 1 --2- 172.21.15.44:0/2778327089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba340ff4f0 0x7fba340682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.351+0000 7fba3bef2700 1 --2- 172.21.15.44:0/2778327089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba341006e0 0x7fba3419af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.351+0000 7fba3bef2700 1 -- 172.21.15.44:0/2778327089 >> 172.21.15.44:0/2778327089 conn(0x7fba340faa60 msgr2=0x7fba34103910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:24.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.351+0000 7fba3bef2700 1 -- 172.21.15.44:0/2778327089 shutdown_connections 2023-09-10T00:40:24.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.351+0000 7fba3bef2700 1 -- 172.21.15.44:0/2778327089 wait complete. 2023-09-10T00:40:24.364 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-10T00:40:24.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.493+0000 7f6738b4c700 1 -- 172.21.15.44:0/2579823345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67341021a0 msgr2=0x7f6734108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:24.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.493+0000 7f6738b4c700 1 --2- 172.21.15.44:0/2579823345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67341021a0 0x7f6734108a50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f672c00b0d0 tx=0x7f672c00b3e0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.500 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.493+0000 7f6738b4c700 1 -- 172.21.15.44:0/2579823345 shutdown_connections 2023-09-10T00:40:24.500 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.493+0000 7f6738b4c700 1 --2- 172.21.15.44:0/2579823345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67341021a0 0x7f6734108a50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.500 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.493+0000 7f6738b4c700 1 --2- 172.21.15.44:0/2579823345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6734100610 0x7f6734100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.493+0000 7f6738b4c700 1 --2- 172.21.15.44:0/2579823345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6734101800 0x7f6734101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.493+0000 7f6738b4c700 1 -- 172.21.15.44:0/2579823345 >> 172.21.15.44:0/2579823345 conn(0x7f67340fbb80 msgr2=0x7f67340fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:24.502 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.493+0000 7f6738b4c700 1 -- 172.21.15.44:0/2579823345 shutdown_connections 2023-09-10T00:40:24.502 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.493+0000 7f6738b4c700 1 -- 172.21.15.44:0/2579823345 wait complete. 2023-09-10T00:40:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.494+0000 7f6738b4c700 1 Processor -- start 2023-09-10T00:40:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.494+0000 7f6738b4c700 1 -- start start 2023-09-10T00:40:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.494+0000 7f6738b4c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6734100610 0x7f67340743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.494+0000 7f6738b4c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6734101800 0x7f67340729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:24.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.494+0000 7f6738b4c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67341021a0 0x7f6734072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:24.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.494+0000 7f6738b4c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67340736d0 con 0x7f6734101800 2023-09-10T00:40:24.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.494+0000 7f6738b4c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6734073840 con 0x7f6734100610 2023-09-10T00:40:24.506 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.494+0000 7f6738b4c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67341a0360 con 0x7f67341021a0 2023-09-10T00:40:24.506 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.494+0000 7f6731d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6734101800 0x7f67340729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:24.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.494+0000 7f6731d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6734101800 0x7f67340729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48344/0 (socket says 172.21.15.44:48344) 2023-09-10T00:40:24.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.494+0000 7f6731d9b700 1 -- 172.21.15.44:0/1059428469 learned_addr learned my addr 172.21.15.44:0/1059428469 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.494+0000 7f6732d9d700 1 --2- 172.21.15.44:0/1059428469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67341021a0 0x7f6734072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.494+0000 7f673259c700 1 --2- 172.21.15.44:0/1059428469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6734100610 0x7f67340743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.495+0000 7f6731d9b700 1 -- 172.21.15.44:0/1059428469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67341021a0 msgr2=0x7f6734072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.495+0000 7f6731d9b700 1 --2- 172.21.15.44:0/1059428469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67341021a0 0x7f6734072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.495+0000 7f6731d9b700 1 -- 172.21.15.44:0/1059428469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6734100610 msgr2=0x7f67340743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.495+0000 7f6731d9b700 1 --2- 172.21.15.44:0/1059428469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6734100610 0x7f67340743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.495+0000 7f6731d9b700 1 -- 172.21.15.44:0/1059428469 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c009d00 con 0x7f6734101800 2023-09-10T00:40:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.495+0000 7f6732d9d700 1 --2- 172.21.15.44:0/1059428469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67341021a0 0x7f6734072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.495+0000 7f6731d9b700 1 --2- 172.21.15.44:0/1059428469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6734101800 0x7f67340729f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f672400e9a0 tx=0x7f672400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.495+0000 7f67237fe700 1 -- 172.21.15.44:0/1059428469 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724009790 con 0x7f6734101800 2023-09-10T00:40:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.495+0000 7f67237fe700 1 -- 172.21.15.44:0/1059428469 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f672400cbd0 con 0x7f6734101800 2023-09-10T00:40:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.495+0000 7f6738b4c700 1 -- 172.21.15.44:0/1059428469 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67341a0640 con 0x7f6734101800 2023-09-10T00:40:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.495+0000 7f67237fe700 1 -- 172.21.15.44:0/1059428469 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724018680 con 0x7f6734101800 2023-09-10T00:40:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.495+0000 7f6738b4c700 1 -- 172.21.15.44:0/1059428469 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67341a0ba0 con 0x7f6734101800 2023-09-10T00:40:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.496+0000 7f6738b4c700 1 -- 172.21.15.44:0/1059428469 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6734107c20 con 0x7f6734101800 2023-09-10T00:40:24.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.497+0000 7f67237fe700 1 -- 172.21.15.44:0/1059428469 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f672400cd40 con 0x7f6734101800 2023-09-10T00:40:24.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.497+0000 7f67237fe700 1 --2- 172.21.15.44:0/1059428469 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f671c0a06c0 0x7f671c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.498+0000 7f67237fe700 1 -- 172.21.15.44:0/1059428469 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f6724014070 con 0x7f6734101800 2023-09-10T00:40:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.498+0000 7f673259c700 1 --2- 172.21.15.44:0/1059428469 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f671c0a06c0 0x7f671c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.498+0000 7f673259c700 1 --2- 172.21.15.44:0/1059428469 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f671c0a06c0 0x7f671c0a2b80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6728005950 tx=0x7f67280058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.502+0000 7f67237fe700 1 -- 172.21.15.44:0/1059428469 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6724095970 con 0x7f6734101800 2023-09-10T00:40:24.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.706+0000 7f6738b4c700 1 -- 172.21.15.44:0/1059428469 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f673402cf80 con 0x7f6734101800 2023-09-10T00:40:24.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.706+0000 7f67237fe700 1 -- 172.21.15.44:0/1059428469 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6724098f90 con 0x7f6734101800 2023-09-10T00:40:24.708 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:40:24.710 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.709+0000 7f6738b4c700 1 -- 172.21.15.44:0/1059428469 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f671c0a06c0 msgr2=0x7f671c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:24.711 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.709+0000 7f6738b4c700 1 --2- 172.21.15.44:0/1059428469 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f671c0a06c0 0x7f671c0a2b80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6728005950 tx=0x7f67280058e0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.711 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.709+0000 7f6738b4c700 1 -- 172.21.15.44:0/1059428469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6734101800 msgr2=0x7f67340729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:24.711 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.709+0000 7f6738b4c700 1 --2- 172.21.15.44:0/1059428469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6734101800 0x7f67340729f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f672400e9a0 tx=0x7f672400ed60 comp rx=0 tx=0).stop 2023-09-10T00:40:24.712 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.709+0000 7f6738b4c700 1 -- 172.21.15.44:0/1059428469 shutdown_connections 2023-09-10T00:40:24.712 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.709+0000 7f6738b4c700 1 --2- 172.21.15.44:0/1059428469 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f671c0a06c0 0x7f671c0a2b80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.713 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.709+0000 7f6738b4c700 1 --2- 172.21.15.44:0/1059428469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67341021a0 0x7f6734072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.714 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.709+0000 7f6738b4c700 1 --2- 172.21.15.44:0/1059428469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6734100610 0x7f67340743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.714 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.709+0000 7f6738b4c700 1 --2- 172.21.15.44:0/1059428469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6734101800 0x7f67340729f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.714 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.709+0000 7f6738b4c700 1 -- 172.21.15.44:0/1059428469 >> 172.21.15.44:0/1059428469 conn(0x7f67340fbb80 msgr2=0x7f6734104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:24.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.709+0000 7f6738b4c700 1 -- 172.21.15.44:0/1059428469 shutdown_connections 2023-09-10T00:40:24.715 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.709+0000 7f6738b4c700 1 -- 172.21.15.44:0/1059428469 wait complete. 2023-09-10T00:40:24.716 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:24 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1178411167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:24.717 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:24 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2778327089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:40:24.717 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:24 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1059428469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:40:24.722 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-10T00:40:24.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.851+0000 7f1f8157c700 1 -- 172.21.15.44:0/2324439248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1039a0 msgr2=0x7f1f7c105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.851+0000 7f1f8157c700 1 --2- 172.21.15.44:0/2324439248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1039a0 0x7f1f7c105d90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009990 tx=0x7f1f6c009ca0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.851+0000 7f1f8157c700 1 -- 172.21.15.44:0/2324439248 shutdown_connections 2023-09-10T00:40:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.851+0000 7f1f8157c700 1 --2- 172.21.15.44:0/2324439248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f7c1062d0 0x7f1f7c108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.851+0000 7f1f8157c700 1 --2- 172.21.15.44:0/2324439248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c101070 0x7f1f7c103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.851+0000 7f1f8157c700 1 --2- 172.21.15.44:0/2324439248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1039a0 0x7f1f7c105d90 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.851+0000 7f1f8157c700 1 -- 172.21.15.44:0/2324439248 >> 172.21.15.44:0/2324439248 conn(0x7f1f7c0faa60 msgr2=0x7f1f7c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.851+0000 7f1f8157c700 1 -- 172.21.15.44:0/2324439248 shutdown_connections 2023-09-10T00:40:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.851+0000 7f1f8157c700 1 -- 172.21.15.44:0/2324439248 wait complete. 2023-09-10T00:40:24.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.851+0000 7f1f8157c700 1 Processor -- start 2023-09-10T00:40:24.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f8157c700 1 -- start start 2023-09-10T00:40:24.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f8157c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c101070 0x7f1f7c1009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:24.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f8157c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1039a0 0x7f1f7c0ff020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:24.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f8157c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f7c1062d0 0x7f1f7c0ff560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:24.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f8157c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0ffd00 con 0x7f1f7c1039a0 2023-09-10T00:40:24.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f8157c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0ffe70 con 0x7f1f7c101070 2023-09-10T00:40:24.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f8157c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c1a0230 con 0x7f1f7c1062d0 2023-09-10T00:40:24.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f7a7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1039a0 0x7f1f7c0ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:24.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f7a7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1039a0 0x7f1f7c0ff020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48358/0 (socket says 172.21.15.44:48358) 2023-09-10T00:40:24.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f7a7fc700 1 -- 172.21.15.44:0/1376547286 learned_addr learned my addr 172.21.15.44:0/1376547286 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:24.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f7affd700 1 --2- 172.21.15.44:0/1376547286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c101070 0x7f1f7c1009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:24.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f7b7fe700 1 --2- 172.21.15.44:0/1376547286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f7c1062d0 0x7f1f7c0ff560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:24.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f7a7fc700 1 -- 172.21.15.44:0/1376547286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f7c1062d0 msgr2=0x7f1f7c0ff560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:24.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f7a7fc700 1 --2- 172.21.15.44:0/1376547286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f7c1062d0 0x7f1f7c0ff560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f7a7fc700 1 -- 172.21.15.44:0/1376547286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c101070 msgr2=0x7f1f7c1009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:24.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f7a7fc700 1 --2- 172.21.15.44:0/1376547286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c101070 0x7f1f7c1009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:24.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.852+0000 7f1f7a7fc700 1 -- 172.21.15.44:0/1376547286 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c0095f0 con 0x7f1f7c1039a0 2023-09-10T00:40:24.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.853+0000 7f1f7a7fc700 1 --2- 172.21.15.44:0/1376547286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1039a0 0x7f1f7c0ff020 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0059d0 tx=0x7f1f6c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:24.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.853+0000 7f1f63fff700 1 -- 172.21.15.44:0/1376547286 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c01d070 con 0x7f1f7c1039a0 2023-09-10T00:40:24.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.853+0000 7f1f63fff700 1 -- 172.21.15.44:0/1376547286 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f6c00beb0 con 0x7f1f7c1039a0 2023-09-10T00:40:24.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.853+0000 7f1f8157c700 1 -- 172.21.15.44:0/1376547286 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c1a04b0 con 0x7f1f7c1039a0 2023-09-10T00:40:24.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.853+0000 7f1f63fff700 1 -- 172.21.15.44:0/1376547286 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c00fdb0 con 0x7f1f7c1039a0 2023-09-10T00:40:24.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.853+0000 7f1f8157c700 1 -- 172.21.15.44:0/1376547286 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c1a0960 con 0x7f1f7c1039a0 2023-09-10T00:40:24.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.854+0000 7f1f8157c700 1 -- 172.21.15.44:0/1376547286 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7c0fc5e0 con 0x7f1f7c1039a0 2023-09-10T00:40:24.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.855+0000 7f1f63fff700 1 -- 172.21.15.44:0/1376547286 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f1f6c00f460 con 0x7f1f7c1039a0 2023-09-10T00:40:24.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.855+0000 7f1f63fff700 1 --2- 172.21.15.44:0/1376547286 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1f640a0640 0x7f1f640a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:24.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.856+0000 7f1f7affd700 1 --2- 172.21.15.44:0/1376547286 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1f640a0640 0x7f1f640a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:24.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.856+0000 7f1f63fff700 1 -- 172.21.15.44:0/1376547286 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f1f6c0cd380 con 0x7f1f7c1039a0 2023-09-10T00:40:24.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.856+0000 7f1f7affd700 1 --2- 172.21.15.44:0/1376547286 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1f640a0640 0x7f1f640a2b00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1f70007c60 tx=0x7f1f700073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:24.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:24.860+0000 7f1f63fff700 1 -- 172.21.15.44:0/1376547286 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1f6c096b80 con 0x7f1f7c1039a0 2023-09-10T00:40:24.965 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:24 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1178411167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:24.965 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:24 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2778327089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:40:24.965 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:24 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1059428469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:40:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.062+0000 7f1f8157c700 1 -- 172.21.15.44:0/1376547286 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1f7c02ce70 con 0x7f1f7c1039a0 2023-09-10T00:40:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.062+0000 7f1f63fff700 1 -- 172.21.15.44:0/1376547286 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1f6c09a1a0 con 0x7f1f7c1039a0 2023-09-10T00:40:25.063 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2023-09-10T00:40:25.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.064+0000 7f1f8157c700 1 -- 172.21.15.44:0/1376547286 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1f640a0640 msgr2=0x7f1f640a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:25.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.064+0000 7f1f8157c700 1 --2- 172.21.15.44:0/1376547286 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1f640a0640 0x7f1f640a2b00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1f70007c60 tx=0x7f1f700073d0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.064+0000 7f1f8157c700 1 -- 172.21.15.44:0/1376547286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1039a0 msgr2=0x7f1f7c0ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:25.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.064+0000 7f1f8157c700 1 --2- 172.21.15.44:0/1376547286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1039a0 0x7f1f7c0ff020 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0059d0 tx=0x7f1f6c004790 comp rx=0 tx=0).stop 2023-09-10T00:40:25.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.064+0000 7f1f8157c700 1 -- 172.21.15.44:0/1376547286 shutdown_connections 2023-09-10T00:40:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.064+0000 7f1f8157c700 1 --2- 172.21.15.44:0/1376547286 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1f640a0640 0x7f1f640a2b00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.064+0000 7f1f8157c700 1 --2- 172.21.15.44:0/1376547286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f7c1062d0 0x7f1f7c0ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.064+0000 7f1f8157c700 1 --2- 172.21.15.44:0/1376547286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c101070 0x7f1f7c1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.064+0000 7f1f8157c700 1 --2- 172.21.15.44:0/1376547286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1039a0 0x7f1f7c0ff020 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.064+0000 7f1f8157c700 1 -- 172.21.15.44:0/1376547286 >> 172.21.15.44:0/1376547286 conn(0x7f1f7c0faa60 msgr2=0x7f1f7c101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:25.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.065+0000 7f1f8157c700 1 -- 172.21.15.44:0/1376547286 shutdown_connections 2023-09-10T00:40:25.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.065+0000 7f1f8157c700 1 -- 172.21.15.44:0/1376547286 wait complete. 2023-09-10T00:40:25.077 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-10T00:40:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:24 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1178411167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:24 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2778327089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:40:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:24 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1059428469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:40:25.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.205+0000 7f8ef339f700 1 -- 172.21.15.44:0/975885196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8eec1006f0 msgr2=0x7f8eec100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:25.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.205+0000 7f8ef339f700 1 --2- 172.21.15.44:0/975885196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8eec1006f0 0x7f8eec100b50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80099c0 tx=0x7f8ee8009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.205+0000 7f8ef339f700 1 -- 172.21.15.44:0/975885196 shutdown_connections 2023-09-10T00:40:25.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.205+0000 7f8ef339f700 1 --2- 172.21.15.44:0/975885196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8eec101090 0x7f8eec107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.205+0000 7f8ef339f700 1 --2- 172.21.15.44:0/975885196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8eec0ff500 0x7f8eec0ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.205+0000 7f8ef339f700 1 --2- 172.21.15.44:0/975885196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8eec1006f0 0x7f8eec100b50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.205+0000 7f8ef339f700 1 -- 172.21.15.44:0/975885196 >> 172.21.15.44:0/975885196 conn(0x7f8eec0faa90 msgr2=0x7f8eec0fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:25.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.205+0000 7f8ef339f700 1 -- 172.21.15.44:0/975885196 shutdown_connections 2023-09-10T00:40:25.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.205+0000 7f8ef339f700 1 -- 172.21.15.44:0/975885196 wait complete. 2023-09-10T00:40:25.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.206+0000 7f8ef339f700 1 Processor -- start 2023-09-10T00:40:25.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.206+0000 7f8ef339f700 1 -- start start 2023-09-10T00:40:25.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.206+0000 7f8ef339f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8eec0ff500 0x7f8eec195130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:25.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.206+0000 7f8ef339f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8eec1006f0 0x7f8eec195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:25.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.206+0000 7f8ef339f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8eec101090 0x7f8eec19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.206+0000 7f8ef339f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec19ccb0 con 0x7f8eec1006f0 2023-09-10T00:40:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.206+0000 7f8ef339f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec19cdf0 con 0x7f8eec0ff500 2023-09-10T00:40:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.206+0000 7f8ef339f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec19cf80 con 0x7f8eec101090 2023-09-10T00:40:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.206+0000 7f8ef093a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8eec1006f0 0x7f8eec195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.207+0000 7f8ef093a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8eec1006f0 0x7f8eec195670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48372/0 (socket says 172.21.15.44:48372) 2023-09-10T00:40:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.207+0000 7f8ef093a700 1 -- 172.21.15.44:0/367040477 learned_addr learned my addr 172.21.15.44:0/367040477 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.207+0000 7f8ef113b700 1 --2- 172.21.15.44:0/367040477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8eec0ff500 0x7f8eec195130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.207+0000 7f8ef093a700 1 -- 172.21.15.44:0/367040477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8eec101090 msgr2=0x7f8eec19c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.207+0000 7f8ef193c700 1 --2- 172.21.15.44:0/367040477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8eec101090 0x7f8eec19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.207+0000 7f8ef093a700 1 --2- 172.21.15.44:0/367040477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8eec101090 0x7f8eec19c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.207+0000 7f8ef093a700 1 -- 172.21.15.44:0/367040477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8eec0ff500 msgr2=0x7f8eec195130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.207+0000 7f8ef093a700 1 --2- 172.21.15.44:0/367040477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8eec0ff500 0x7f8eec195130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.207+0000 7f8ef093a700 1 -- 172.21.15.44:0/367040477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee80095f0 con 0x7f8eec1006f0 2023-09-10T00:40:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.207+0000 7f8ef113b700 1 --2- 172.21.15.44:0/367040477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8eec0ff500 0x7f8eec195130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:40:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.207+0000 7f8ef093a700 1 --2- 172.21.15.44:0/367040477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8eec1006f0 0x7f8eec195670 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009f50 tx=0x7f8ee801a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:25.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.207+0000 7f8ee27fc700 1 -- 172.21.15.44:0/367040477 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee801f070 con 0x7f8eec1006f0 2023-09-10T00:40:25.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.207+0000 7f8ee27fc700 1 -- 172.21.15.44:0/367040477 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ee800bda0 con 0x7f8eec1006f0 2023-09-10T00:40:25.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.207+0000 7f8ef339f700 1 -- 172.21.15.44:0/367040477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eec068770 con 0x7f8eec1006f0 2023-09-10T00:40:25.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.207+0000 7f8ee27fc700 1 -- 172.21.15.44:0/367040477 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8005280 con 0x7f8eec1006f0 2023-09-10T00:40:25.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.207+0000 7f8ef339f700 1 -- 172.21.15.44:0/367040477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eec068c10 con 0x7f8eec1006f0 2023-09-10T00:40:25.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.208+0000 7f8ef339f700 1 -- 172.21.15.44:0/367040477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eec069660 con 0x7f8eec1006f0 2023-09-10T00:40:25.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.209+0000 7f8ee27fc700 1 -- 172.21.15.44:0/367040477 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f8ee8023070 con 0x7f8eec1006f0 2023-09-10T00:40:25.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.210+0000 7f8ee27fc700 1 --2- 172.21.15.44:0/367040477 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8ed80a06c0 0x7f8ed80a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:25.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.210+0000 7f8ee27fc700 1 -- 172.21.15.44:0/367040477 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f8ee80cf330 con 0x7f8eec1006f0 2023-09-10T00:40:25.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.210+0000 7f8ef113b700 1 --2- 172.21.15.44:0/367040477 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8ed80a06c0 0x7f8ed80a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:25.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.210+0000 7f8ef113b700 1 --2- 172.21.15.44:0/367040477 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8ed80a06c0 0x7f8ed80a2b80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8edc005fd0 tx=0x7f8edc005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:25.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.214+0000 7f8ee27fc700 1 -- 172.21.15.44:0/367040477 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8ee8098b30 con 0x7f8eec1006f0 2023-09-10T00:40:25.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.415+0000 7f8ef339f700 1 -- 172.21.15.44:0/367040477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8eec10ae50 con 0x7f8eec1006f0 2023-09-10T00:40:25.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.415+0000 7f8ee27fc700 1 -- 172.21.15.44:0/367040477 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8ee809c150 con 0x7f8eec1006f0 2023-09-10T00:40:25.416 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:40:25.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.417+0000 7f8ef339f700 1 -- 172.21.15.44:0/367040477 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8ed80a06c0 msgr2=0x7f8ed80a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:25.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.417+0000 7f8ef339f700 1 --2- 172.21.15.44:0/367040477 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8ed80a06c0 0x7f8ed80a2b80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8edc005fd0 tx=0x7f8edc005f00 comp rx=0 tx=0).stop 2023-09-10T00:40:25.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.417+0000 7f8ef339f700 1 -- 172.21.15.44:0/367040477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8eec1006f0 msgr2=0x7f8eec195670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:25.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.417+0000 7f8ef339f700 1 --2- 172.21.15.44:0/367040477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8eec1006f0 0x7f8eec195670 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009f50 tx=0x7f8ee801a8f0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.417+0000 7f8ef339f700 1 -- 172.21.15.44:0/367040477 shutdown_connections 2023-09-10T00:40:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.417+0000 7f8ef339f700 1 --2- 172.21.15.44:0/367040477 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f8ed80a06c0 0x7f8ed80a2b80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.417+0000 7f8ef339f700 1 --2- 172.21.15.44:0/367040477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8eec101090 0x7f8eec19c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.417+0000 7f8ef339f700 1 --2- 172.21.15.44:0/367040477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8eec0ff500 0x7f8eec195130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.417+0000 7f8ef339f700 1 --2- 172.21.15.44:0/367040477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8eec1006f0 0x7f8eec195670 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.417+0000 7f8ef339f700 1 -- 172.21.15.44:0/367040477 >> 172.21.15.44:0/367040477 conn(0x7f8eec0faa90 msgr2=0x7f8eec0fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:25.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.417+0000 7f8ef339f700 1 -- 172.21.15.44:0/367040477 shutdown_connections 2023-09-10T00:40:25.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.417+0000 7f8ef339f700 1 -- 172.21.15.44:0/367040477 wait complete. 2023-09-10T00:40:25.430 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-09-10T00:40:25.430 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:40:25.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.556+0000 7fb3bbd1f700 1 -- 172.21.15.44:0/483010800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b40ff520 msgr2=0x7fb3b40ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:25.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.556+0000 7fb3bbd1f700 1 --2- 172.21.15.44:0/483010800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b40ff520 0x7fb3b40ff940 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40099c0 tx=0x7fb3a4009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.556+0000 7fb3bbd1f700 1 -- 172.21.15.44:0/483010800 shutdown_connections 2023-09-10T00:40:25.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.556+0000 7fb3bbd1f700 1 --2- 172.21.15.44:0/483010800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3b41010b0 0x7fb3b4107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.556+0000 7fb3bbd1f700 1 --2- 172.21.15.44:0/483010800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b40ff520 0x7fb3b40ff940 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.556+0000 7fb3bbd1f700 1 --2- 172.21.15.44:0/483010800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3b4100710 0x7fb3b4100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.556+0000 7fb3bbd1f700 1 -- 172.21.15.44:0/483010800 >> 172.21.15.44:0/483010800 conn(0x7fb3b40faa90 msgr2=0x7fb3b40fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:25.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.556+0000 7fb3bbd1f700 1 -- 172.21.15.44:0/483010800 shutdown_connections 2023-09-10T00:40:25.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.556+0000 7fb3bbd1f700 1 -- 172.21.15.44:0/483010800 wait complete. 2023-09-10T00:40:25.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.556+0000 7fb3bbd1f700 1 Processor -- start 2023-09-10T00:40:25.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3bbd1f700 1 -- start start 2023-09-10T00:40:25.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3bbd1f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3b40ff520 0x7fb3b41037c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:25.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3bbd1f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3b4100710 0x7fb3b4101e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:25.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3bbd1f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b41010b0 0x7fb3b4102340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:25.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3bbd1f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b4102ae0 con 0x7fb3b41010b0 2023-09-10T00:40:25.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3bbd1f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b4102c50 con 0x7fb3b4100710 2023-09-10T00:40:25.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3bbd1f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b41a0260 con 0x7fb3b40ff520 2023-09-10T00:40:25.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3ba2bc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b41010b0 0x7fb3b4102340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:25.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3ba2bc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b41010b0 0x7fb3b4102340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48388/0 (socket says 172.21.15.44:48388) 2023-09-10T00:40:25.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3ba2bc700 1 -- 172.21.15.44:0/4012451869 learned_addr learned my addr 172.21.15.44:0/4012451869 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:25.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3b9abb700 1 --2- 172.21.15.44:0/4012451869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3b40ff520 0x7fb3b41037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:25.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3b92ba700 1 --2- 172.21.15.44:0/4012451869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3b4100710 0x7fb3b4101e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:25.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3ba2bc700 1 -- 172.21.15.44:0/4012451869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3b40ff520 msgr2=0x7fb3b41037c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3ba2bc700 1 --2- 172.21.15.44:0/4012451869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3b40ff520 0x7fb3b41037c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3ba2bc700 1 -- 172.21.15.44:0/4012451869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3b4100710 msgr2=0x7fb3b4101e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3ba2bc700 1 --2- 172.21.15.44:0/4012451869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3b4100710 0x7fb3b4101e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3ba2bc700 1 -- 172.21.15.44:0/4012451869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a40095f0 con 0x7fb3b41010b0 2023-09-10T00:40:25.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.557+0000 7fb3ba2bc700 1 --2- 172.21.15.44:0/4012451869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b41010b0 0x7fb3b4102340 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac007ee0 tx=0x7fb3ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:25.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.558+0000 7fb3aaffd700 1 -- 172.21.15.44:0/4012451869 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac017070 con 0x7fb3b41010b0 2023-09-10T00:40:25.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.558+0000 7fb3aaffd700 1 -- 172.21.15.44:0/4012451869 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb3ac00f040 con 0x7fb3b41010b0 2023-09-10T00:40:25.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.558+0000 7fb3aaffd700 1 -- 172.21.15.44:0/4012451869 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac005570 con 0x7fb3b41010b0 2023-09-10T00:40:25.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.558+0000 7fb3bbd1f700 1 -- 172.21.15.44:0/4012451869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b41a0540 con 0x7fb3b41010b0 2023-09-10T00:40:25.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.558+0000 7fb3bbd1f700 1 -- 172.21.15.44:0/4012451869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b41a0a00 con 0x7fb3b41010b0 2023-09-10T00:40:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.559+0000 7fb3bbd1f700 1 -- 172.21.15.44:0/4012451869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b4107600 con 0x7fb3b41010b0 2023-09-10T00:40:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.560+0000 7fb3aaffd700 1 -- 172.21.15.44:0/4012451869 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fb3ac00db20 con 0x7fb3b41010b0 2023-09-10T00:40:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.561+0000 7fb3aaffd700 1 --2- 172.21.15.44:0/4012451869 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb3a00a0710 0x7fb3a00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.561+0000 7fb3b9abb700 1 --2- 172.21.15.44:0/4012451869 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb3a00a0710 0x7fb3a00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.561+0000 7fb3aaffd700 1 -- 172.21.15.44:0/4012451869 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7fb3ac0cbc40 con 0x7fb3b41010b0 2023-09-10T00:40:25.569 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.561+0000 7fb3b9abb700 1 --2- 172.21.15.44:0/4012451869 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb3a00a0710 0x7fb3a00a2bd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4000c00 tx=0x7fb3a4009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:25.569 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.564+0000 7fb3aaffd700 1 -- 172.21.15.44:0/4012451869 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb3ac0953b0 con 0x7fb3b41010b0 2023-09-10T00:40:25.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.815+0000 7fb3bbd1f700 1 -- 172.21.15.44:0/4012451869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3b402cf80 con 0x7fb3b41010b0 2023-09-10T00:40:25.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.816+0000 7fb3aaffd700 1 -- 172.21.15.44:0/4012451869 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11369 (secure 0 0 0) 0x7fb3ac0989d0 con 0x7fb3b41010b0 2023-09-10T00:40:25.817 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:25.817 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-10T00:38:39.183533+0000","modified":"2023-09-10T00:39:02.480227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24605,"mds_1":24574,"mds_2":24572,"mds_3":24515,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14751":{"gid":14751,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/3958270510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":3958270510},{"type":"v1","addr":"172.21.15.44:6839","nonce":3958270510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14763":{"gid":14763,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/1241002552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1241002552},{"type":"v1","addr":"172.21.15.44:6841","nonce":1241002552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24556":{"gid":24556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/3222652693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":3222652693},{"type":"v1","addr":"172.21.15.131:6839","nonce":3222652693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24605":{"gid":24605,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/2622966260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2622966260},{"type":"v1","addr":"172.21.15.99:6839","nonce":2622966260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-10T00:40:25.818 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.818+0000 7fb3bbd1f700 1 -- 172.21.15.44:0/4012451869 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb3a00a0710 msgr2=0x7fb3a00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:25.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.818+0000 7fb3bbd1f700 1 --2- 172.21.15.44:0/4012451869 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb3a00a0710 0x7fb3a00a2bd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4000c00 tx=0x7fb3a4009f10 comp rx=0 tx=0).stop 2023-09-10T00:40:25.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.818+0000 7fb3bbd1f700 1 -- 172.21.15.44:0/4012451869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b41010b0 msgr2=0x7fb3b4102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:25.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.818+0000 7fb3bbd1f700 1 --2- 172.21.15.44:0/4012451869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b41010b0 0x7fb3b4102340 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac007ee0 tx=0x7fb3ac00d3b0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.818+0000 7fb3bbd1f700 1 -- 172.21.15.44:0/4012451869 shutdown_connections 2023-09-10T00:40:25.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.818+0000 7fb3bbd1f700 1 --2- 172.21.15.44:0/4012451869 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fb3a00a0710 0x7fb3a00a2bd0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.818+0000 7fb3bbd1f700 1 --2- 172.21.15.44:0/4012451869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b41010b0 0x7fb3b4102340 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.818+0000 7fb3bbd1f700 1 --2- 172.21.15.44:0/4012451869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3b40ff520 0x7fb3b41037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.818+0000 7fb3bbd1f700 1 --2- 172.21.15.44:0/4012451869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3b4100710 0x7fb3b4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.818+0000 7fb3bbd1f700 1 -- 172.21.15.44:0/4012451869 >> 172.21.15.44:0/4012451869 conn(0x7fb3b40faa90 msgr2=0x7fb3b40fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:25.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.818+0000 7fb3bbd1f700 1 -- 172.21.15.44:0/4012451869 shutdown_connections 2023-09-10T00:40:25.820 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.818+0000 7fb3bbd1f700 1 -- 172.21.15.44:0/4012451869 wait complete. 2023-09-10T00:40:25.821 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2023-09-10T00:40:25.831 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-09-10T00:40:25.831 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-09-10T00:40:25.832 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-10T00:40:25.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.957+0000 7ffbc22de700 1 -- 172.21.15.44:0/3286101546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbbc0ff4e0 msgr2=0x7ffbbc0ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:25.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.957+0000 7ffbc22de700 1 --2- 172.21.15.44:0/3286101546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbbc0ff4e0 0x7ffbbc0ff900 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0099c0 tx=0x7ffbac009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.957+0000 7ffbc22de700 1 -- 172.21.15.44:0/3286101546 shutdown_connections 2023-09-10T00:40:25.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.957+0000 7ffbc22de700 1 --2- 172.21.15.44:0/3286101546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbbc101070 0x7ffbbc107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.957+0000 7ffbc22de700 1 --2- 172.21.15.44:0/3286101546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbbc0ff4e0 0x7ffbbc0ff900 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.957+0000 7ffbc22de700 1 --2- 172.21.15.44:0/3286101546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbbc1006d0 0x7ffbbc100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.957+0000 7ffbc22de700 1 -- 172.21.15.44:0/3286101546 >> 172.21.15.44:0/3286101546 conn(0x7ffbbc0faa30 msgr2=0x7ffbbc0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:25.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.957+0000 7ffbc22de700 1 -- 172.21.15.44:0/3286101546 shutdown_connections 2023-09-10T00:40:25.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.957+0000 7ffbc22de700 1 -- 172.21.15.44:0/3286101546 wait complete. 2023-09-10T00:40:25.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.958+0000 7ffbc22de700 1 Processor -- start 2023-09-10T00:40:25.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.958+0000 7ffbc22de700 1 -- start start 2023-09-10T00:40:25.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.958+0000 7ffbc22de700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbbc0ff4e0 0x7ffbbc196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:25.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.958+0000 7ffbc22de700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbbc1006d0 0x7ffbbc196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:25.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.958+0000 7ffbc22de700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbbc101070 0x7ffbbc19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:25.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.958+0000 7ffbc22de700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc19dd90 con 0x7ffbbc101070 2023-09-10T00:40:25.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.958+0000 7ffbc22de700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc19ded0 con 0x7ffbbc1006d0 2023-09-10T00:40:25.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.958+0000 7ffbc22de700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc19e060 con 0x7ffbbc0ff4e0 2023-09-10T00:40:25.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.959+0000 7ffbc087b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbbc101070 0x7ffbbc19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:25.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.959+0000 7ffbc087b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbbc101070 0x7ffbbc19d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48402/0 (socket says 172.21.15.44:48402) 2023-09-10T00:40:25.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.959+0000 7ffbc087b700 1 -- 172.21.15.44:0/2880294509 learned_addr learned my addr 172.21.15.44:0/2880294509 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:25.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.959+0000 7ffbbbfff700 1 --2- 172.21.15.44:0/2880294509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbbc0ff4e0 0x7ffbbc196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:25.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.959+0000 7ffbbb7fe700 1 --2- 172.21.15.44:0/2880294509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbbc1006d0 0x7ffbbc196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:25.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.959+0000 7ffbc087b700 1 -- 172.21.15.44:0/2880294509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbbc0ff4e0 msgr2=0x7ffbbc196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.959+0000 7ffbc087b700 1 --2- 172.21.15.44:0/2880294509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbbc0ff4e0 0x7ffbbc196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.959+0000 7ffbc087b700 1 -- 172.21.15.44:0/2880294509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbbc1006d0 msgr2=0x7ffbbc196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.959+0000 7ffbc087b700 1 --2- 172.21.15.44:0/2880294509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbbc1006d0 0x7ffbbc196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.959+0000 7ffbc087b700 1 -- 172.21.15.44:0/2880294509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac0095f0 con 0x7ffbbc101070 2023-09-10T00:40:25.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.959+0000 7ffbc087b700 1 --2- 172.21.15.44:0/2880294509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbbc101070 0x7ffbbc19d7c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4007ee0 tx=0x7ffbb400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.959+0000 7ffbb97fa700 1 -- 172.21.15.44:0/2880294509 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4017070 con 0x7ffbbc101070 2023-09-10T00:40:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.959+0000 7ffbb97fa700 1 -- 172.21.15.44:0/2880294509 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffbb400f040 con 0x7ffbbc101070 2023-09-10T00:40:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.959+0000 7ffbc22de700 1 -- 172.21.15.44:0/2880294509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbbc19e340 con 0x7ffbbc101070 2023-09-10T00:40:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.959+0000 7ffbb97fa700 1 -- 172.21.15.44:0/2880294509 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4005710 con 0x7ffbbc101070 2023-09-10T00:40:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.959+0000 7ffbc22de700 1 -- 172.21.15.44:0/2880294509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbbc19e800 con 0x7ffbbc101070 2023-09-10T00:40:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.960+0000 7ffbc22de700 1 -- 172.21.15.44:0/2880294509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbbc10aba0 con 0x7ffbbc101070 2023-09-10T00:40:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.961+0000 7ffbb97fa700 1 -- 172.21.15.44:0/2880294509 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ffbb4005870 con 0x7ffbbc101070 2023-09-10T00:40:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.962+0000 7ffbb97fa700 1 --2- 172.21.15.44:0/2880294509 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffba40a0690 0x7ffba40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:25.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.962+0000 7ffbb97fa700 1 -- 172.21.15.44:0/2880294509 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7ffbb40cb880 con 0x7ffbbc101070 2023-09-10T00:40:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.962+0000 7ffbbbfff700 1 --2- 172.21.15.44:0/2880294509 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffba40a0690 0x7ffba40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.962+0000 7ffbbbfff700 1 --2- 172.21.15.44:0/2880294509 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffba40a0690 0x7ffba40a2b50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffbac000c00 tx=0x7ffbac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:25.966+0000 7ffbb97fa700 1 -- 172.21.15.44:0/2880294509 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ffbb4094f90 con 0x7ffbbc101070 2023-09-10T00:40:26.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:25 smithi131 ceph-mon[103713]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 9 op/s; 7.2 KiB/s, 1 keys/s, 1 objects/s recovering 2023-09-10T00:40:26.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:25 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1376547286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:40:26.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:25 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/367040477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:40:26.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:25 smithi044 ceph-mon[98676]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 9 op/s; 7.2 KiB/s, 1 keys/s, 1 objects/s recovering 2023-09-10T00:40:26.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:25 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1376547286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:40:26.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:25 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/367040477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:40:26.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:25 smithi099 ceph-mon[102692]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 9 op/s; 7.2 KiB/s, 1 keys/s, 1 objects/s recovering 2023-09-10T00:40:26.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:25 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1376547286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:40:26.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:25 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/367040477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:40:26.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.165+0000 7ffbc22de700 1 -- 172.21.15.44:0/2880294509 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffbbc067eb0 con 0x7ffba40a0690 2023-09-10T00:40:26.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.180+0000 7ffbb97fa700 1 -- 172.21.15.44:0/2880294509 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427736 (secure 0 0 0) 0x7ffbbc067eb0 con 0x7ffba40a0690 2023-09-10T00:40:26.185 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:26.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.183+0000 7ffbc22de700 1 -- 172.21.15.44:0/2880294509 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffba40a0690 msgr2=0x7ffba40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:26.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.183+0000 7ffbc22de700 1 --2- 172.21.15.44:0/2880294509 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffba40a0690 0x7ffba40a2b50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffbac000c00 tx=0x7ffbac005fb0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.184+0000 7ffbc22de700 1 -- 172.21.15.44:0/2880294509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbbc101070 msgr2=0x7ffbbc19d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:26.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.184+0000 7ffbc22de700 1 --2- 172.21.15.44:0/2880294509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbbc101070 0x7ffbbc19d7c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4007ee0 tx=0x7ffbb400d3b0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.184+0000 7ffbc22de700 1 -- 172.21.15.44:0/2880294509 shutdown_connections 2023-09-10T00:40:26.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.184+0000 7ffbc22de700 1 --2- 172.21.15.44:0/2880294509 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ffba40a0690 0x7ffba40a2b50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.184+0000 7ffbc22de700 1 --2- 172.21.15.44:0/2880294509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbbc101070 0x7ffbbc19d7c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.184+0000 7ffbc22de700 1 --2- 172.21.15.44:0/2880294509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbbc0ff4e0 0x7ffbbc196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.184+0000 7ffbc22de700 1 --2- 172.21.15.44:0/2880294509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbbc1006d0 0x7ffbbc196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.184+0000 7ffbc22de700 1 -- 172.21.15.44:0/2880294509 >> 172.21.15.44:0/2880294509 conn(0x7ffbbc0faa30 msgr2=0x7ffbbc0677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:26.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.184+0000 7ffbc22de700 1 -- 172.21.15.44:0/2880294509 shutdown_connections 2023-09-10T00:40:26.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.184+0000 7ffbc22de700 1 -- 172.21.15.44:0/2880294509 wait complete. 2023-09-10T00:40:26.195 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2023-09-10T00:40:26.196 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2023-09-10T00:40:26.023414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663681,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1016,"num_read_kb":645,"num_write":438,"num_write_kb":3093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1239584,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":863,"ondisk_log_size":863,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338488,"kb_used_data":14272,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603336,"statfs":{"total":1151940427776,"available":1151593816064,"internally_reserved":0,"allocated":14614528,"data_stored":8073294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7266,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":60,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":58848,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.351178"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682673+0000","last_change":"2023-09-10T00:38:39.768218+0000","last_active":"2023-09-10T00:40:16.682673+0000","last_peered":"2023-09-10T00:40:16.682673+0000","last_clean":"2023-09-10T00:40:16.682673+0000","last_became_active":"2023-09-10T00:38:39.767454+0000","last_became_peered":"2023-09-10T00:38:39.767454+0000","last_unstale":"2023-09-10T00:40:16.682673+0000","last_undegraded":"2023-09-10T00:40:16.682673+0000","last_fullsized":"2023-09-10T00:40:16.682673+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:17:30.287403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689406+0000","last_change":"2023-09-10T00:38:39.765384+0000","last_active":"2023-09-10T00:40:16.689406+0000","last_peered":"2023-09-10T00:40:16.689406+0000","last_clean":"2023-09-10T00:40:16.689406+0000","last_became_active":"2023-09-10T00:38:39.765090+0000","last_became_peered":"2023-09-10T00:38:39.765090+0000","last_unstale":"2023-09-10T00:40:16.689406+0000","last_undegraded":"2023-09-10T00:40:16.689406+0000","last_fullsized":"2023-09-10T00:40:16.689406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:02:46.025627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689356+0000","last_change":"2023-09-10T00:38:39.769222+0000","last_active":"2023-09-10T00:40:16.689356+0000","last_peered":"2023-09-10T00:40:16.689356+0000","last_clean":"2023-09-10T00:40:16.689356+0000","last_became_active":"2023-09-10T00:38:39.768801+0000","last_became_peered":"2023-09-10T00:38:39.768801+0000","last_unstale":"2023-09-10T00:40:16.689356+0000","last_undegraded":"2023-09-10T00:40:16.689356+0000","last_fullsized":"2023-09-10T00:40:16.689356+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:38:53.394209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686934+0000","last_change":"2023-09-10T00:38:39.764963+0000","last_active":"2023-09-10T00:40:16.686934+0000","last_peered":"2023-09-10T00:40:16.686934+0000","last_clean":"2023-09-10T00:40:16.686934+0000","last_became_active":"2023-09-10T00:38:39.764466+0000","last_became_peered":"2023-09-10T00:38:39.764466+0000","last_unstale":"2023-09-10T00:40:16.686934+0000","last_undegraded":"2023-09-10T00:40:16.686934+0000","last_fullsized":"2023-09-10T00:40:16.686934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:31.553321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684022+0000","last_change":"2023-09-10T00:39:17.534684+0000","last_active":"2023-09-10T00:40:16.684022+0000","last_peered":"2023-09-10T00:40:16.684022+0000","last_clean":"2023-09-10T00:40:16.684022+0000","last_became_active":"2023-09-10T00:39:17.534325+0000","last_became_peered":"2023-09-10T00:39:17.534325+0000","last_unstale":"2023-09-10T00:40:16.684022+0000","last_undegraded":"2023-09-10T00:40:16.684022+0000","last_fullsized":"2023-09-10T00:40:16.684022+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:39:11.902221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684935+0000","last_change":"2023-09-10T00:38:39.761681+0000","last_active":"2023-09-10T00:40:16.684935+0000","last_peered":"2023-09-10T00:40:16.684935+0000","last_clean":"2023-09-10T00:40:16.684935+0000","last_became_active":"2023-09-10T00:38:39.761383+0000","last_became_peered":"2023-09-10T00:38:39.761383+0000","last_unstale":"2023-09-10T00:40:16.684935+0000","last_undegraded":"2023-09-10T00:40:16.684935+0000","last_fullsized":"2023-09-10T00:40:16.684935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:38:04.534656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684507+0000","last_change":"2023-09-10T00:38:39.765028+0000","last_active":"2023-09-10T00:40:16.684507+0000","last_peered":"2023-09-10T00:40:16.684507+0000","last_clean":"2023-09-10T00:40:16.684507+0000","last_became_active":"2023-09-10T00:38:39.764663+0000","last_became_peered":"2023-09-10T00:38:39.764663+0000","last_unstale":"2023-09-10T00:40:16.684507+0000","last_undegraded":"2023-09-10T00:40:16.684507+0000","last_fullsized":"2023-09-10T00:40:16.684507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:14:49.645129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674630+0000","last_change":"2023-09-10T00:38:39.770135+0000","last_active":"2023-09-10T00:40:16.674630+0000","last_peered":"2023-09-10T00:40:16.674630+0000","last_clean":"2023-09-10T00:40:16.674630+0000","last_became_active":"2023-09-10T00:38:39.769720+0000","last_became_peered":"2023-09-10T00:38:39.769720+0000","last_unstale":"2023-09-10T00:40:16.674630+0000","last_undegraded":"2023-09-10T00:40:16.674630+0000","last_fullsized":"2023-09-10T00:40:16.674630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:46:21.166566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.683408+0000","last_change":"2023-09-10T00:38:39.770170+0000","last_active":"2023-09-10T00:40:16.683408+0000","last_peered":"2023-09-10T00:40:16.683408+0000","last_clean":"2023-09-10T00:40:16.683408+0000","last_became_active":"2023-09-10T00:38:39.769840+0000","last_became_peered":"2023-09-10T00:38:39.769840+0000","last_unstale":"2023-09-10T00:40:16.683408+0000","last_undegraded":"2023-09-10T00:40:16.683408+0000","last_fullsized":"2023-09-10T00:40:16.683408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:48.117670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685644+0000","last_change":"2023-09-10T00:38:39.769621+0000","last_active":"2023-09-10T00:40:16.685644+0000","last_peered":"2023-09-10T00:40:16.685644+0000","last_clean":"2023-09-10T00:40:16.685644+0000","last_became_active":"2023-09-10T00:38:39.769300+0000","last_became_peered":"2023-09-10T00:38:39.769300+0000","last_unstale":"2023-09-10T00:40:16.685644+0000","last_undegraded":"2023-09-10T00:40:16.685644+0000","last_fullsized":"2023-09-10T00:40:16.685644+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:37.100671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682752+0000","last_change":"2023-09-10T00:38:39.772200+0000","last_active":"2023-09-10T00:40:16.682752+0000","last_peered":"2023-09-10T00:40:16.682752+0000","last_clean":"2023-09-10T00:40:16.682752+0000","last_became_active":"2023-09-10T00:38:39.771503+0000","last_became_peered":"2023-09-10T00:38:39.771503+0000","last_unstale":"2023-09-10T00:40:16.682752+0000","last_undegraded":"2023-09-10T00:40:16.682752+0000","last_fullsized":"2023-09-10T00:40:16.682752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:13:38.830642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686922+0000","last_change":"2023-09-10T00:38:39.769832+0000","last_active":"2023-09-10T00:40:16.686922+0000","last_peered":"2023-09-10T00:40:16.686922+0000","last_clean":"2023-09-10T00:40:16.686922+0000","last_became_active":"2023-09-10T00:38:39.769546+0000","last_became_peered":"2023-09-10T00:38:39.769546+0000","last_unstale":"2023-09-10T00:40:16.686922+0000","last_undegraded":"2023-09-10T00:40:16.686922+0000","last_fullsized":"2023-09-10T00:40:16.686922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:53:05.219877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674379+0000","last_change":"2023-09-10T00:39:17.531430+0000","last_active":"2023-09-10T00:40:16.674379+0000","last_peered":"2023-09-10T00:40:16.674379+0000","last_clean":"2023-09-10T00:40:16.674379+0000","last_became_active":"2023-09-10T00:39:17.531145+0000","last_became_peered":"2023-09-10T00:39:17.531145+0000","last_unstale":"2023-09-10T00:40:16.674379+0000","last_undegraded":"2023-09-10T00:40:16.674379+0000","last_fullsized":"2023-09-10T00:40:16.674379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:34:50.163747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.688856+0000","last_change":"2023-09-10T00:38:39.764751+0000","last_active":"2023-09-10T00:40:16.688856+0000","last_peered":"2023-09-10T00:40:16.688856+0000","last_clean":"2023-09-10T00:40:16.688856+0000","last_became_active":"2023-09-10T00:38:39.764464+0000","last_became_peered":"2023-09-10T00:38:39.764464+0000","last_unstale":"2023-09-10T00:40:16.688856+0000","last_undegraded":"2023-09-10T00:40:16.688856+0000","last_fullsized":"2023-09-10T00:40:16.688856+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:03:51.096126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.688911+0000","last_change":"2023-09-10T00:39:17.532977+0000","last_active":"2023-09-10T00:40:16.688911+0000","last_peered":"2023-09-10T00:40:16.688911+0000","last_clean":"2023-09-10T00:40:16.688911+0000","last_became_active":"2023-09-10T00:39:17.532547+0000","last_became_peered":"2023-09-10T00:39:17.532547+0000","last_unstale":"2023-09-10T00:40:16.688911+0000","last_undegraded":"2023-09-10T00:40:16.688911+0000","last_fullsized":"2023-09-10T00:40:16.688911+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:33:01.685743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681772+0000","last_change":"2023-09-10T00:38:39.770787+0000","last_active":"2023-09-10T00:40:16.681772+0000","last_peered":"2023-09-10T00:40:16.681772+0000","last_clean":"2023-09-10T00:40:16.681772+0000","last_became_active":"2023-09-10T00:38:39.770497+0000","last_became_peered":"2023-09-10T00:38:39.770497+0000","last_unstale":"2023-09-10T00:40:16.681772+0000","last_undegraded":"2023-09-10T00:40:16.681772+0000","last_fullsized":"2023-09-10T00:40:16.681772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:43:11.066766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682433+0000","last_change":"2023-09-10T00:38:39.768008+0000","last_active":"2023-09-10T00:40:16.682433+0000","last_peered":"2023-09-10T00:40:16.682433+0000","last_clean":"2023-09-10T00:40:16.682433+0000","last_became_active":"2023-09-10T00:38:39.767079+0000","last_became_peered":"2023-09-10T00:38:39.767079+0000","last_unstale":"2023-09-10T00:40:16.682433+0000","last_undegraded":"2023-09-10T00:40:16.682433+0000","last_fullsized":"2023-09-10T00:40:16.682433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:34:00.689881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685567+0000","last_change":"2023-09-10T00:38:39.775326+0000","last_active":"2023-09-10T00:40:16.685567+0000","last_peered":"2023-09-10T00:40:16.685567+0000","last_clean":"2023-09-10T00:40:16.685567+0000","last_became_active":"2023-09-10T00:38:39.775007+0000","last_became_peered":"2023-09-10T00:38:39.775007+0000","last_unstale":"2023-09-10T00:40:16.685567+0000","last_undegraded":"2023-09-10T00:40:16.685567+0000","last_fullsized":"2023-09-10T00:40:16.685567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:59:27.711722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681440+0000","last_change":"2023-09-10T00:38:39.764974+0000","last_active":"2023-09-10T00:40:16.681440+0000","last_peered":"2023-09-10T00:40:16.681440+0000","last_clean":"2023-09-10T00:40:16.681440+0000","last_became_active":"2023-09-10T00:38:39.764368+0000","last_became_peered":"2023-09-10T00:38:39.764368+0000","last_unstale":"2023-09-10T00:40:16.681440+0000","last_undegraded":"2023-09-10T00:40:16.681440+0000","last_fullsized":"2023-09-10T00:40:16.681440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:40:28.046021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686233+0000","last_change":"2023-09-10T00:38:43.793589+0000","last_active":"2023-09-10T00:40:16.686233+0000","last_peered":"2023-09-10T00:40:16.686233+0000","last_clean":"2023-09-10T00:40:16.686233+0000","last_became_active":"2023-09-10T00:38:43.793166+0000","last_became_peered":"2023-09-10T00:38:43.793166+0000","last_unstale":"2023-09-10T00:40:16.686233+0000","last_undegraded":"2023-09-10T00:40:16.686233+0000","last_fullsized":"2023-09-10T00:40:16.686233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:51:47.128925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":119,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:24.649354+0000","last_change":"2023-09-10T00:38:38.759909+0000","last_active":"2023-09-10T00:40:24.649354+0000","last_peered":"2023-09-10T00:40:24.649354+0000","last_clean":"2023-09-10T00:40:24.649354+0000","last_became_active":"2023-09-10T00:38:38.759561+0000","last_became_peered":"2023-09-10T00:38:38.759561+0000","last_unstale":"2023-09-10T00:40:24.649354+0000","last_undegraded":"2023-09-10T00:40:24.649354+0000","last_fullsized":"2023-09-10T00:40:24.649354+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:56:18.009497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676177+0000","last_change":"2023-09-10T00:38:39.774916+0000","last_active":"2023-09-10T00:40:16.676177+0000","last_peered":"2023-09-10T00:40:16.676177+0000","last_clean":"2023-09-10T00:40:16.676177+0000","last_became_active":"2023-09-10T00:38:39.774584+0000","last_became_peered":"2023-09-10T00:38:39.774584+0000","last_unstale":"2023-09-10T00:40:16.676177+0000","last_undegraded":"2023-09-10T00:40:16.676177+0000","last_fullsized":"2023-09-10T00:40:16.676177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:36:33.125742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686686+0000","last_change":"2023-09-10T00:38:43.789421+0000","last_active":"2023-09-10T00:40:16.686686+0000","last_peered":"2023-09-10T00:40:16.686686+0000","last_clean":"2023-09-10T00:40:16.686686+0000","last_became_active":"2023-09-10T00:38:43.788569+0000","last_became_peered":"2023-09-10T00:38:43.788569+0000","last_unstale":"2023-09-10T00:40:16.686686+0000","last_undegraded":"2023-09-10T00:40:16.686686+0000","last_fullsized":"2023-09-10T00:40:16.686686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:07:38.529811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676261+0000","last_change":"2023-09-10T00:38:38.767381+0000","last_active":"2023-09-10T00:40:16.676261+0000","last_peered":"2023-09-10T00:40:16.676261+0000","last_clean":"2023-09-10T00:40:16.676261+0000","last_became_active":"2023-09-10T00:38:38.767082+0000","last_became_peered":"2023-09-10T00:38:38.767082+0000","last_unstale":"2023-09-10T00:40:16.676261+0000","last_undegraded":"2023-09-10T00:40:16.676261+0000","last_fullsized":"2023-09-10T00:40:16.676261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:13:32.533215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689263+0000","last_change":"2023-09-10T00:38:39.772645+0000","last_active":"2023-09-10T00:40:16.689263+0000","last_peered":"2023-09-10T00:40:16.689263+0000","last_clean":"2023-09-10T00:40:16.689263+0000","last_became_active":"2023-09-10T00:38:39.771554+0000","last_became_peered":"2023-09-10T00:38:39.771554+0000","last_unstale":"2023-09-10T00:40:16.689263+0000","last_undegraded":"2023-09-10T00:40:16.689263+0000","last_fullsized":"2023-09-10T00:40:16.689263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:25:06.237309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686805+0000","last_change":"2023-09-10T00:38:43.795764+0000","last_active":"2023-09-10T00:40:16.686805+0000","last_peered":"2023-09-10T00:40:16.686805+0000","last_clean":"2023-09-10T00:40:16.686805+0000","last_became_active":"2023-09-10T00:38:43.795112+0000","last_became_peered":"2023-09-10T00:38:43.795112+0000","last_unstale":"2023-09-10T00:40:16.686805+0000","last_undegraded":"2023-09-10T00:40:16.686805+0000","last_fullsized":"2023-09-10T00:40:16.686805+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:18:55.460752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.699271+0000","last_change":"2023-09-10T00:40:17.698716+0000","last_active":"2023-09-10T00:40:17.699271+0000","last_peered":"2023-09-10T00:40:17.699271+0000","last_clean":"2023-09-10T00:40:17.699271+0000","last_became_active":"2023-09-10T00:40:17.687287+0000","last_became_peered":"2023-09-10T00:40:17.687287+0000","last_unstale":"2023-09-10T00:40:17.699271+0000","last_undegraded":"2023-09-10T00:40:17.699271+0000","last_fullsized":"2023-09-10T00:40:17.699271+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:07:29.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681429+0000","last_change":"2023-09-10T00:38:39.767790+0000","last_active":"2023-09-10T00:40:16.681429+0000","last_peered":"2023-09-10T00:40:16.681429+0000","last_clean":"2023-09-10T00:40:16.681429+0000","last_became_active":"2023-09-10T00:38:39.767513+0000","last_became_peered":"2023-09-10T00:38:39.767513+0000","last_unstale":"2023-09-10T00:40:16.681429+0000","last_undegraded":"2023-09-10T00:40:16.681429+0000","last_fullsized":"2023-09-10T00:40:16.681429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:05:48.223951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686757+0000","last_change":"2023-09-10T00:38:43.789089+0000","last_active":"2023-09-10T00:40:16.686757+0000","last_peered":"2023-09-10T00:40:16.686757+0000","last_clean":"2023-09-10T00:40:16.686757+0000","last_became_active":"2023-09-10T00:38:43.788434+0000","last_became_peered":"2023-09-10T00:38:43.788434+0000","last_unstale":"2023-09-10T00:40:16.686757+0000","last_undegraded":"2023-09-10T00:40:16.686757+0000","last_fullsized":"2023-09-10T00:40:16.686757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:55:00.236088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685274+0000","last_change":"2023-09-10T00:38:38.769478+0000","last_active":"2023-09-10T00:40:16.685274+0000","last_peered":"2023-09-10T00:40:16.685274+0000","last_clean":"2023-09-10T00:40:16.685274+0000","last_became_active":"2023-09-10T00:38:38.769168+0000","last_became_peered":"2023-09-10T00:38:38.769168+0000","last_unstale":"2023-09-10T00:40:16.685274+0000","last_undegraded":"2023-09-10T00:40:16.685274+0000","last_fullsized":"2023-09-10T00:40:16.685274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:03:48.133561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675591+0000","last_change":"2023-09-10T00:38:39.771843+0000","last_active":"2023-09-10T00:40:16.675591+0000","last_peered":"2023-09-10T00:40:16.675591+0000","last_clean":"2023-09-10T00:40:16.675591+0000","last_became_active":"2023-09-10T00:38:39.771510+0000","last_became_peered":"2023-09-10T00:38:39.771510+0000","last_unstale":"2023-09-10T00:40:16.675591+0000","last_undegraded":"2023-09-10T00:40:16.675591+0000","last_fullsized":"2023-09-10T00:40:16.675591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:17:39.996490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.680942+0000","last_change":"2023-09-10T00:38:43.786853+0000","last_active":"2023-09-10T00:40:16.680942+0000","last_peered":"2023-09-10T00:40:16.680942+0000","last_clean":"2023-09-10T00:40:16.680942+0000","last_became_active":"2023-09-10T00:38:43.786386+0000","last_became_peered":"2023-09-10T00:38:43.786386+0000","last_unstale":"2023-09-10T00:40:16.680942+0000","last_undegraded":"2023-09-10T00:40:16.680942+0000","last_fullsized":"2023-09-10T00:40:16.680942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:23:53.724653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:22.600360+0000","last_change":"2023-09-10T00:38:38.769004+0000","last_active":"2023-09-10T00:40:22.600360+0000","last_peered":"2023-09-10T00:40:22.600360+0000","last_clean":"2023-09-10T00:40:22.600360+0000","last_became_active":"2023-09-10T00:38:38.768690+0000","last_became_peered":"2023-09-10T00:38:38.768690+0000","last_unstale":"2023-09-10T00:40:22.600360+0000","last_undegraded":"2023-09-10T00:40:22.600360+0000","last_fullsized":"2023-09-10T00:40:22.600360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:04:42.531092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1636,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684636+0000","last_change":"2023-09-10T00:38:39.773174+0000","last_active":"2023-09-10T00:40:16.684636+0000","last_peered":"2023-09-10T00:40:16.684636+0000","last_clean":"2023-09-10T00:40:16.684636+0000","last_became_active":"2023-09-10T00:38:39.771192+0000","last_became_peered":"2023-09-10T00:38:39.771192+0000","last_unstale":"2023-09-10T00:40:16.684636+0000","last_undegraded":"2023-09-10T00:40:16.684636+0000","last_fullsized":"2023-09-10T00:40:16.684636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:16:25.034024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.687731+0000","last_change":"2023-09-10T00:40:17.687731+0000","last_active":"2023-09-10T00:40:17.687731+0000","last_peered":"2023-09-10T00:40:17.687731+0000","last_clean":"2023-09-10T00:40:17.687731+0000","last_became_active":"2023-09-10T00:40:17.687285+0000","last_became_peered":"2023-09-10T00:40:17.687285+0000","last_unstale":"2023-09-10T00:40:17.687731+0000","last_undegraded":"2023-09-10T00:40:17.687731+0000","last_fullsized":"2023-09-10T00:40:17.687731+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:16:29.995198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684888+0000","last_change":"2023-09-10T00:38:38.763416+0000","last_active":"2023-09-10T00:40:16.684888+0000","last_peered":"2023-09-10T00:40:16.684888+0000","last_clean":"2023-09-10T00:40:16.684888+0000","last_became_active":"2023-09-10T00:38:38.763054+0000","last_became_peered":"2023-09-10T00:38:38.763054+0000","last_unstale":"2023-09-10T00:40:16.684888+0000","last_undegraded":"2023-09-10T00:40:16.684888+0000","last_fullsized":"2023-09-10T00:40:16.684888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:47:00.179467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675938+0000","last_change":"2023-09-10T00:38:39.773284+0000","last_active":"2023-09-10T00:40:16.675938+0000","last_peered":"2023-09-10T00:40:16.675938+0000","last_clean":"2023-09-10T00:40:16.675938+0000","last_became_active":"2023-09-10T00:38:39.772969+0000","last_became_peered":"2023-09-10T00:38:39.772969+0000","last_unstale":"2023-09-10T00:40:16.675938+0000","last_undegraded":"2023-09-10T00:40:16.675938+0000","last_fullsized":"2023-09-10T00:40:16.675938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:46:45.013879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682333+0000","last_change":"2023-09-10T00:38:43.790810+0000","last_active":"2023-09-10T00:40:16.682333+0000","last_peered":"2023-09-10T00:40:16.682333+0000","last_clean":"2023-09-10T00:40:16.682333+0000","last_became_active":"2023-09-10T00:38:43.790429+0000","last_became_peered":"2023-09-10T00:38:43.790429+0000","last_unstale":"2023-09-10T00:40:16.682333+0000","last_undegraded":"2023-09-10T00:40:16.682333+0000","last_fullsized":"2023-09-10T00:40:16.682333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:30:56.072587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.673630+0000","last_change":"2023-09-10T00:38:38.754326+0000","last_active":"2023-09-10T00:40:16.673630+0000","last_peered":"2023-09-10T00:40:16.673630+0000","last_clean":"2023-09-10T00:40:16.673630+0000","last_became_active":"2023-09-10T00:38:38.754006+0000","last_became_peered":"2023-09-10T00:38:38.754006+0000","last_unstale":"2023-09-10T00:40:16.673630+0000","last_undegraded":"2023-09-10T00:40:16.673630+0000","last_fullsized":"2023-09-10T00:40:16.673630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:09:26.983606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674527+0000","last_change":"2023-09-10T00:38:39.769699+0000","last_active":"2023-09-10T00:40:16.674527+0000","last_peered":"2023-09-10T00:40:16.674527+0000","last_clean":"2023-09-10T00:40:16.674527+0000","last_became_active":"2023-09-10T00:38:39.769362+0000","last_became_peered":"2023-09-10T00:38:39.769362+0000","last_unstale":"2023-09-10T00:40:16.674527+0000","last_undegraded":"2023-09-10T00:40:16.674527+0000","last_fullsized":"2023-09-10T00:40:16.674527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:30:20.938437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689355+0000","last_change":"2023-09-10T00:38:43.785669+0000","last_active":"2023-09-10T00:40:16.689355+0000","last_peered":"2023-09-10T00:40:16.689355+0000","last_clean":"2023-09-10T00:40:16.689355+0000","last_became_active":"2023-09-10T00:38:43.785255+0000","last_became_peered":"2023-09-10T00:38:43.785255+0000","last_unstale":"2023-09-10T00:40:16.689355+0000","last_undegraded":"2023-09-10T00:40:16.689355+0000","last_fullsized":"2023-09-10T00:40:16.689355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:52:57.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675650+0000","last_change":"2023-09-10T00:38:38.767997+0000","last_active":"2023-09-10T00:40:16.675650+0000","last_peered":"2023-09-10T00:40:16.675650+0000","last_clean":"2023-09-10T00:40:16.675650+0000","last_became_active":"2023-09-10T00:38:38.767715+0000","last_became_peered":"2023-09-10T00:38:38.767715+0000","last_unstale":"2023-09-10T00:40:16.675650+0000","last_undegraded":"2023-09-10T00:40:16.675650+0000","last_fullsized":"2023-09-10T00:40:16.675650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:11:17.047280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685405+0000","last_change":"2023-09-10T00:38:39.778004+0000","last_active":"2023-09-10T00:40:16.685405+0000","last_peered":"2023-09-10T00:40:16.685405+0000","last_clean":"2023-09-10T00:40:16.685405+0000","last_became_active":"2023-09-10T00:38:39.777632+0000","last_became_peered":"2023-09-10T00:38:39.777632+0000","last_unstale":"2023-09-10T00:40:16.685405+0000","last_undegraded":"2023-09-10T00:40:16.685405+0000","last_fullsized":"2023-09-10T00:40:16.685405+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:46:40.804385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.682227+0000","last_change":"2023-09-10T00:40:17.682227+0000","last_active":"2023-09-10T00:40:17.682227+0000","last_peered":"2023-09-10T00:40:17.682227+0000","last_clean":"2023-09-10T00:40:17.682227+0000","last_became_active":"2023-09-10T00:40:17.681849+0000","last_became_peered":"2023-09-10T00:40:17.681849+0000","last_unstale":"2023-09-10T00:40:17.682227+0000","last_undegraded":"2023-09-10T00:40:17.682227+0000","last_fullsized":"2023-09-10T00:40:17.682227+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:59.568162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":194,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:24.602900+0000","last_change":"2023-09-10T00:38:38.767399+0000","last_active":"2023-09-10T00:40:24.602900+0000","last_peered":"2023-09-10T00:40:24.602900+0000","last_clean":"2023-09-10T00:40:24.602900+0000","last_became_active":"2023-09-10T00:38:38.767176+0000","last_became_peered":"2023-09-10T00:38:38.767176+0000","last_unstale":"2023-09-10T00:40:24.602900+0000","last_undegraded":"2023-09-10T00:40:24.602900+0000","last_fullsized":"2023-09-10T00:40:24.602900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:17:24.481985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681626+0000","last_change":"2023-09-10T00:38:39.766013+0000","last_active":"2023-09-10T00:40:16.681626+0000","last_peered":"2023-09-10T00:40:16.681626+0000","last_clean":"2023-09-10T00:40:16.681626+0000","last_became_active":"2023-09-10T00:38:39.765235+0000","last_became_peered":"2023-09-10T00:38:39.765235+0000","last_unstale":"2023-09-10T00:40:16.681626+0000","last_undegraded":"2023-09-10T00:40:16.681626+0000","last_fullsized":"2023-09-10T00:40:16.681626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:01:48.874845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685291+0000","last_change":"2023-09-10T00:38:43.790361+0000","last_active":"2023-09-10T00:40:16.685291+0000","last_peered":"2023-09-10T00:40:16.685291+0000","last_clean":"2023-09-10T00:40:16.685291+0000","last_became_active":"2023-09-10T00:38:43.789966+0000","last_became_peered":"2023-09-10T00:38:43.789966+0000","last_unstale":"2023-09-10T00:40:16.685291+0000","last_undegraded":"2023-09-10T00:40:16.685291+0000","last_fullsized":"2023-09-10T00:40:16.685291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:46:37.978258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682397+0000","last_change":"2023-09-10T00:38:38.771670+0000","last_active":"2023-09-10T00:40:16.682397+0000","last_peered":"2023-09-10T00:40:16.682397+0000","last_clean":"2023-09-10T00:40:16.682397+0000","last_became_active":"2023-09-10T00:38:38.771339+0000","last_became_peered":"2023-09-10T00:38:38.771339+0000","last_unstale":"2023-09-10T00:40:16.682397+0000","last_undegraded":"2023-09-10T00:40:16.682397+0000","last_fullsized":"2023-09-10T00:40:16.682397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:22:32.658544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689361+0000","last_change":"2023-09-10T00:38:39.772857+0000","last_active":"2023-09-10T00:40:16.689361+0000","last_peered":"2023-09-10T00:40:16.689361+0000","last_clean":"2023-09-10T00:40:16.689361+0000","last_became_active":"2023-09-10T00:38:39.771990+0000","last_became_peered":"2023-09-10T00:38:39.771990+0000","last_unstale":"2023-09-10T00:40:16.689361+0000","last_undegraded":"2023-09-10T00:40:16.689361+0000","last_fullsized":"2023-09-10T00:40:16.689361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:16:15.767748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686883+0000","last_change":"2023-09-10T00:38:43.795482+0000","last_active":"2023-09-10T00:40:16.686883+0000","last_peered":"2023-09-10T00:40:16.686883+0000","last_clean":"2023-09-10T00:40:16.686883+0000","last_became_active":"2023-09-10T00:38:43.795048+0000","last_became_peered":"2023-09-10T00:38:43.795048+0000","last_unstale":"2023-09-10T00:40:16.686883+0000","last_undegraded":"2023-09-10T00:40:16.686883+0000","last_fullsized":"2023-09-10T00:40:16.686883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:13:18.715274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675457+0000","last_change":"2023-09-10T00:38:38.771008+0000","last_active":"2023-09-10T00:40:16.675457+0000","last_peered":"2023-09-10T00:40:16.675457+0000","last_clean":"2023-09-10T00:40:16.675457+0000","last_became_active":"2023-09-10T00:38:38.770714+0000","last_became_peered":"2023-09-10T00:38:38.770714+0000","last_unstale":"2023-09-10T00:40:16.675457+0000","last_undegraded":"2023-09-10T00:40:16.675457+0000","last_fullsized":"2023-09-10T00:40:16.675457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:04:30.148141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682324+0000","last_change":"2023-09-10T00:39:17.533500+0000","last_active":"2023-09-10T00:40:16.682324+0000","last_peered":"2023-09-10T00:40:16.682324+0000","last_clean":"2023-09-10T00:40:16.682324+0000","last_became_active":"2023-09-10T00:39:17.533175+0000","last_became_peered":"2023-09-10T00:39:17.533175+0000","last_unstale":"2023-09-10T00:40:16.682324+0000","last_undegraded":"2023-09-10T00:40:16.682324+0000","last_fullsized":"2023-09-10T00:40:16.682324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:13:42.121699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684740+0000","last_change":"2023-09-10T00:39:17.536684+0000","last_active":"2023-09-10T00:40:16.684740+0000","last_peered":"2023-09-10T00:40:16.684740+0000","last_clean":"2023-09-10T00:40:16.684740+0000","last_became_active":"2023-09-10T00:39:17.536170+0000","last_became_peered":"2023-09-10T00:39:17.536170+0000","last_unstale":"2023-09-10T00:40:16.684740+0000","last_undegraded":"2023-09-10T00:40:16.684740+0000","last_fullsized":"2023-09-10T00:40:16.684740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:47:16.264619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":259,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:23.744670+0000","last_change":"2023-09-10T00:40:17.741148+0000","last_active":"2023-09-10T00:40:23.744670+0000","last_peered":"2023-09-10T00:40:23.744670+0000","last_clean":"2023-09-10T00:40:23.744670+0000","last_became_active":"2023-09-10T00:40:17.731033+0000","last_became_peered":"2023-09-10T00:40:17.731033+0000","last_unstale":"2023-09-10T00:40:23.744670+0000","last_undegraded":"2023-09-10T00:40:23.744670+0000","last_fullsized":"2023-09-10T00:40:23.744670+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:20:42.067739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63651,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":176,"num_read_kb":150,"num_write":40,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":58848,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684860+0000","last_change":"2023-09-10T00:38:39.767507+0000","last_active":"2023-09-10T00:40:16.684860+0000","last_peered":"2023-09-10T00:40:16.684860+0000","last_clean":"2023-09-10T00:40:16.684860+0000","last_became_active":"2023-09-10T00:38:39.767223+0000","last_became_peered":"2023-09-10T00:38:39.767223+0000","last_unstale":"2023-09-10T00:40:16.684860+0000","last_undegraded":"2023-09-10T00:40:16.684860+0000","last_fullsized":"2023-09-10T00:40:16.684860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:46:30.901818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676490+0000","last_change":"2023-09-10T00:38:43.785396+0000","last_active":"2023-09-10T00:40:16.676490+0000","last_peered":"2023-09-10T00:40:16.676490+0000","last_clean":"2023-09-10T00:40:16.676490+0000","last_became_active":"2023-09-10T00:38:43.785038+0000","last_became_peered":"2023-09-10T00:38:43.785038+0000","last_unstale":"2023-09-10T00:40:16.676490+0000","last_undegraded":"2023-09-10T00:40:16.676490+0000","last_fullsized":"2023-09-10T00:40:16.676490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:30:31.629703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.680630+0000","last_change":"2023-09-10T00:38:38.766742+0000","last_active":"2023-09-10T00:40:16.680630+0000","last_peered":"2023-09-10T00:40:16.680630+0000","last_clean":"2023-09-10T00:40:16.680630+0000","last_became_active":"2023-09-10T00:38:38.766220+0000","last_became_peered":"2023-09-10T00:38:38.766220+0000","last_unstale":"2023-09-10T00:40:16.680630+0000","last_undegraded":"2023-09-10T00:40:16.680630+0000","last_fullsized":"2023-09-10T00:40:16.680630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:41:22.030448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.673843+0000","last_change":"2023-09-10T00:38:39.766022+0000","last_active":"2023-09-10T00:40:16.673843+0000","last_peered":"2023-09-10T00:40:16.673843+0000","last_clean":"2023-09-10T00:40:16.673843+0000","last_became_active":"2023-09-10T00:38:39.765697+0000","last_became_peered":"2023-09-10T00:38:39.765697+0000","last_unstale":"2023-09-10T00:40:16.673843+0000","last_undegraded":"2023-09-10T00:40:16.673843+0000","last_fullsized":"2023-09-10T00:40:16.673843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:19:39.394299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681669+0000","last_change":"2023-09-10T00:38:43.795492+0000","last_active":"2023-09-10T00:40:16.681669+0000","last_peered":"2023-09-10T00:40:16.681669+0000","last_clean":"2023-09-10T00:40:16.681669+0000","last_became_active":"2023-09-10T00:38:43.794968+0000","last_became_peered":"2023-09-10T00:38:43.794968+0000","last_unstale":"2023-09-10T00:40:16.681669+0000","last_undegraded":"2023-09-10T00:40:16.681669+0000","last_fullsized":"2023-09-10T00:40:16.681669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:40:43.558870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676085+0000","last_change":"2023-09-10T00:38:38.767666+0000","last_active":"2023-09-10T00:40:16.676085+0000","last_peered":"2023-09-10T00:40:16.676085+0000","last_clean":"2023-09-10T00:40:16.676085+0000","last_became_active":"2023-09-10T00:38:38.767333+0000","last_became_peered":"2023-09-10T00:38:38.767333+0000","last_unstale":"2023-09-10T00:40:16.676085+0000","last_undegraded":"2023-09-10T00:40:16.676085+0000","last_fullsized":"2023-09-10T00:40:16.676085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:07:53.363196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682945+0000","last_change":"2023-09-10T00:38:39.766036+0000","last_active":"2023-09-10T00:40:16.682945+0000","last_peered":"2023-09-10T00:40:16.682945+0000","last_clean":"2023-09-10T00:40:16.682945+0000","last_became_active":"2023-09-10T00:38:39.765683+0000","last_became_peered":"2023-09-10T00:38:39.765683+0000","last_unstale":"2023-09-10T00:40:16.682945+0000","last_undegraded":"2023-09-10T00:40:16.682945+0000","last_fullsized":"2023-09-10T00:40:16.682945+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:58:04.429960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686204+0000","last_change":"2023-09-10T00:38:43.787870+0000","last_active":"2023-09-10T00:40:16.686204+0000","last_peered":"2023-09-10T00:40:16.686204+0000","last_clean":"2023-09-10T00:40:16.686204+0000","last_became_active":"2023-09-10T00:38:43.785915+0000","last_became_peered":"2023-09-10T00:38:43.785915+0000","last_unstale":"2023-09-10T00:40:16.686204+0000","last_undegraded":"2023-09-10T00:40:16.686204+0000","last_fullsized":"2023-09-10T00:40:16.686204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:34:08.623967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'8","reported_seq":116,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:21.597796+0000","last_change":"2023-09-10T00:38:38.756925+0000","last_active":"2023-09-10T00:40:21.597796+0000","last_peered":"2023-09-10T00:40:21.597796+0000","last_clean":"2023-09-10T00:40:21.597796+0000","last_became_active":"2023-09-10T00:38:38.756610+0000","last_became_peered":"2023-09-10T00:38:38.756610+0000","last_unstale":"2023-09-10T00:40:21.597796+0000","last_undegraded":"2023-09-10T00:40:21.597796+0000","last_fullsized":"2023-09-10T00:40:21.597796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:48:15.385230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1602,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682147+0000","last_change":"2023-09-10T00:38:39.766750+0000","last_active":"2023-09-10T00:40:16.682147+0000","last_peered":"2023-09-10T00:40:16.682147+0000","last_clean":"2023-09-10T00:40:16.682147+0000","last_became_active":"2023-09-10T00:38:39.766458+0000","last_became_peered":"2023-09-10T00:38:39.766458+0000","last_unstale":"2023-09-10T00:40:16.682147+0000","last_undegraded":"2023-09-10T00:40:16.682147+0000","last_fullsized":"2023-09-10T00:40:16.682147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:59:26.691487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674310+0000","last_change":"2023-09-10T00:38:43.788814+0000","last_active":"2023-09-10T00:40:16.674310+0000","last_peered":"2023-09-10T00:40:16.674310+0000","last_clean":"2023-09-10T00:40:16.674310+0000","last_became_active":"2023-09-10T00:38:43.788446+0000","last_became_peered":"2023-09-10T00:38:43.788446+0000","last_unstale":"2023-09-10T00:40:16.674310+0000","last_undegraded":"2023-09-10T00:40:16.674310+0000","last_fullsized":"2023-09-10T00:40:16.674310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:38:53.308160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684332+0000","last_change":"2023-09-10T00:38:38.766020+0000","last_active":"2023-09-10T00:40:16.684332+0000","last_peered":"2023-09-10T00:40:16.684332+0000","last_clean":"2023-09-10T00:40:16.684332+0000","last_became_active":"2023-09-10T00:38:38.765719+0000","last_became_peered":"2023-09-10T00:38:38.765719+0000","last_unstale":"2023-09-10T00:40:16.684332+0000","last_undegraded":"2023-09-10T00:40:16.684332+0000","last_fullsized":"2023-09-10T00:40:16.684332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:25:30.488014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676849+0000","last_change":"2023-09-10T00:38:39.771091+0000","last_active":"2023-09-10T00:40:16.676849+0000","last_peered":"2023-09-10T00:40:16.676849+0000","last_clean":"2023-09-10T00:40:16.676849+0000","last_became_active":"2023-09-10T00:38:39.770745+0000","last_became_peered":"2023-09-10T00:38:39.770745+0000","last_unstale":"2023-09-10T00:40:16.676849+0000","last_undegraded":"2023-09-10T00:40:16.676849+0000","last_fullsized":"2023-09-10T00:40:16.676849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:51:17.978466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676732+0000","last_change":"2023-09-10T00:38:43.793757+0000","last_active":"2023-09-10T00:40:16.676732+0000","last_peered":"2023-09-10T00:40:16.676732+0000","last_clean":"2023-09-10T00:40:16.676732+0000","last_became_active":"2023-09-10T00:38:43.793336+0000","last_became_peered":"2023-09-10T00:38:43.793336+0000","last_unstale":"2023-09-10T00:40:16.676732+0000","last_undegraded":"2023-09-10T00:40:16.676732+0000","last_fullsized":"2023-09-10T00:40:16.676732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:22:36.544950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686902+0000","last_change":"2023-09-10T00:38:38.769373+0000","last_active":"2023-09-10T00:40:16.686902+0000","last_peered":"2023-09-10T00:40:16.686902+0000","last_clean":"2023-09-10T00:40:16.686902+0000","last_became_active":"2023-09-10T00:38:38.769054+0000","last_became_peered":"2023-09-10T00:38:38.769054+0000","last_unstale":"2023-09-10T00:40:16.686902+0000","last_undegraded":"2023-09-10T00:40:16.686902+0000","last_fullsized":"2023-09-10T00:40:16.686902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:12:27.982405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681789+0000","last_change":"2023-09-10T00:39:17.537427+0000","last_active":"2023-09-10T00:40:16.681789+0000","last_peered":"2023-09-10T00:40:16.681789+0000","last_clean":"2023-09-10T00:40:16.681789+0000","last_became_active":"2023-09-10T00:39:17.537023+0000","last_became_peered":"2023-09-10T00:39:17.537023+0000","last_unstale":"2023-09-10T00:40:16.681789+0000","last_undegraded":"2023-09-10T00:40:16.681789+0000","last_fullsized":"2023-09-10T00:40:16.681789+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:54:43.720224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684523+0000","last_change":"2023-09-10T00:38:43.838689+0000","last_active":"2023-09-10T00:40:16.684523+0000","last_peered":"2023-09-10T00:40:16.684523+0000","last_clean":"2023-09-10T00:40:16.684523+0000","last_became_active":"2023-09-10T00:38:43.838244+0000","last_became_peered":"2023-09-10T00:38:43.838244+0000","last_unstale":"2023-09-10T00:40:16.684523+0000","last_undegraded":"2023-09-10T00:40:16.684523+0000","last_fullsized":"2023-09-10T00:40:16.684523+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:54:00.033719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685917+0000","last_change":"2023-09-10T00:38:38.765140+0000","last_active":"2023-09-10T00:40:16.685917+0000","last_peered":"2023-09-10T00:40:16.685917+0000","last_clean":"2023-09-10T00:40:16.685917+0000","last_became_active":"2023-09-10T00:38:38.764730+0000","last_became_peered":"2023-09-10T00:38:38.764730+0000","last_unstale":"2023-09-10T00:40:16.685917+0000","last_undegraded":"2023-09-10T00:40:16.685917+0000","last_fullsized":"2023-09-10T00:40:16.685917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:49:41.129763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685440+0000","last_change":"2023-09-10T00:38:39.773260+0000","last_active":"2023-09-10T00:40:16.685440+0000","last_peered":"2023-09-10T00:40:16.685440+0000","last_clean":"2023-09-10T00:40:16.685440+0000","last_became_active":"2023-09-10T00:38:39.772947+0000","last_became_peered":"2023-09-10T00:38:39.772947+0000","last_unstale":"2023-09-10T00:40:16.685440+0000","last_undegraded":"2023-09-10T00:40:16.685440+0000","last_fullsized":"2023-09-10T00:40:16.685440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:08:10.791610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674639+0000","last_change":"2023-09-10T00:38:43.787882+0000","last_active":"2023-09-10T00:40:16.674639+0000","last_peered":"2023-09-10T00:40:16.674639+0000","last_clean":"2023-09-10T00:40:16.674639+0000","last_became_active":"2023-09-10T00:38:43.787257+0000","last_became_peered":"2023-09-10T00:38:43.787257+0000","last_unstale":"2023-09-10T00:40:16.674639+0000","last_undegraded":"2023-09-10T00:40:16.674639+0000","last_fullsized":"2023-09-10T00:40:16.674639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:11:19.171881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:21.645218+0000","last_change":"2023-09-10T00:38:38.760014+0000","last_active":"2023-09-10T00:40:21.645218+0000","last_peered":"2023-09-10T00:40:21.645218+0000","last_clean":"2023-09-10T00:40:21.645218+0000","last_became_active":"2023-09-10T00:38:38.759637+0000","last_became_peered":"2023-09-10T00:38:38.759637+0000","last_unstale":"2023-09-10T00:40:21.645218+0000","last_undegraded":"2023-09-10T00:40:21.645218+0000","last_fullsized":"2023-09-10T00:40:21.645218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:15.704929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1071,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684955+0000","last_change":"2023-09-10T00:38:39.765800+0000","last_active":"2023-09-10T00:40:16.684955+0000","last_peered":"2023-09-10T00:40:16.684955+0000","last_clean":"2023-09-10T00:40:16.684955+0000","last_became_active":"2023-09-10T00:38:39.765468+0000","last_became_peered":"2023-09-10T00:38:39.765468+0000","last_unstale":"2023-09-10T00:40:16.684955+0000","last_undegraded":"2023-09-10T00:40:16.684955+0000","last_fullsized":"2023-09-10T00:40:16.684955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:57:56.429365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674169+0000","last_change":"2023-09-10T00:38:43.838285+0000","last_active":"2023-09-10T00:40:16.674169+0000","last_peered":"2023-09-10T00:40:16.674169+0000","last_clean":"2023-09-10T00:40:16.674169+0000","last_became_active":"2023-09-10T00:38:43.837821+0000","last_became_peered":"2023-09-10T00:38:43.837821+0000","last_unstale":"2023-09-10T00:40:16.674169+0000","last_undegraded":"2023-09-10T00:40:16.674169+0000","last_fullsized":"2023-09-10T00:40:16.674169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:01:03.354281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675958+0000","last_change":"2023-09-10T00:39:17.536361+0000","last_active":"2023-09-10T00:40:16.675958+0000","last_peered":"2023-09-10T00:40:16.675958+0000","last_clean":"2023-09-10T00:40:16.675958+0000","last_became_active":"2023-09-10T00:39:17.535930+0000","last_became_peered":"2023-09-10T00:39:17.535930+0000","last_unstale":"2023-09-10T00:40:16.675958+0000","last_undegraded":"2023-09-10T00:40:16.675958+0000","last_fullsized":"2023-09-10T00:40:16.675958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:53.576115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":114,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:24.601702+0000","last_change":"2023-09-10T00:38:38.758745+0000","last_active":"2023-09-10T00:40:24.601702+0000","last_peered":"2023-09-10T00:40:24.601702+0000","last_clean":"2023-09-10T00:40:24.601702+0000","last_became_active":"2023-09-10T00:38:38.758427+0000","last_became_peered":"2023-09-10T00:38:38.758427+0000","last_unstale":"2023-09-10T00:40:24.601702+0000","last_undegraded":"2023-09-10T00:40:24.601702+0000","last_fullsized":"2023-09-10T00:40:24.601702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:02:49.238175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.683243+0000","last_change":"2023-09-10T00:38:43.784058+0000","last_active":"2023-09-10T00:40:16.683243+0000","last_peered":"2023-09-10T00:40:16.683243+0000","last_clean":"2023-09-10T00:40:16.683243+0000","last_became_active":"2023-09-10T00:38:43.783571+0000","last_became_peered":"2023-09-10T00:38:43.783571+0000","last_unstale":"2023-09-10T00:40:16.683243+0000","last_undegraded":"2023-09-10T00:40:16.683243+0000","last_fullsized":"2023-09-10T00:40:16.683243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:53:50.992393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676566+0000","last_change":"2023-09-10T00:38:39.775659+0000","last_active":"2023-09-10T00:40:16.676566+0000","last_peered":"2023-09-10T00:40:16.676566+0000","last_clean":"2023-09-10T00:40:16.676566+0000","last_became_active":"2023-09-10T00:38:39.775151+0000","last_became_peered":"2023-09-10T00:38:39.775151+0000","last_unstale":"2023-09-10T00:40:16.676566+0000","last_undegraded":"2023-09-10T00:40:16.676566+0000","last_fullsized":"2023-09-10T00:40:16.676566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:07.415418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686900+0000","last_change":"2023-09-10T00:38:38.755009+0000","last_active":"2023-09-10T00:40:16.686900+0000","last_peered":"2023-09-10T00:40:16.686900+0000","last_clean":"2023-09-10T00:40:16.686900+0000","last_became_active":"2023-09-10T00:38:38.754699+0000","last_became_peered":"2023-09-10T00:38:38.754699+0000","last_unstale":"2023-09-10T00:40:16.686900+0000","last_undegraded":"2023-09-10T00:40:16.686900+0000","last_fullsized":"2023-09-10T00:40:16.686900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:12:02.075256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":324,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681263+0000","last_change":"2023-09-10T00:37:38.459695+0000","last_active":"2023-09-10T00:40:16.681263+0000","last_peered":"2023-09-10T00:40:16.681263+0000","last_clean":"2023-09-10T00:40:16.681263+0000","last_became_active":"2023-09-10T00:37:38.459220+0000","last_became_peered":"2023-09-10T00:37:38.459220+0000","last_unstale":"2023-09-10T00:40:16.681263+0000","last_undegraded":"2023-09-10T00:40:16.681263+0000","last_fullsized":"2023-09-10T00:40:16.681263+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:34:44.602334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:34:44.602334+0000","last_clean_scrub_stamp":"2023-09-10T00:34:44.602334+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:21:32.648725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675426+0000","last_change":"2023-09-10T00:38:43.839001+0000","last_active":"2023-09-10T00:40:16.675426+0000","last_peered":"2023-09-10T00:40:16.675426+0000","last_clean":"2023-09-10T00:40:16.675426+0000","last_became_active":"2023-09-10T00:38:43.838497+0000","last_became_peered":"2023-09-10T00:38:43.838497+0000","last_unstale":"2023-09-10T00:40:16.675426+0000","last_undegraded":"2023-09-10T00:40:16.675426+0000","last_fullsized":"2023-09-10T00:40:16.675426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:57:23.349632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.673565+0000","last_change":"2023-09-10T00:38:39.760150+0000","last_active":"2023-09-10T00:40:16.673565+0000","last_peered":"2023-09-10T00:40:16.673565+0000","last_clean":"2023-09-10T00:40:16.673565+0000","last_became_active":"2023-09-10T00:38:39.759811+0000","last_became_peered":"2023-09-10T00:38:39.759811+0000","last_unstale":"2023-09-10T00:40:16.673565+0000","last_undegraded":"2023-09-10T00:40:16.673565+0000","last_fullsized":"2023-09-10T00:40:16.673565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:21:19.359057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.701421+0000","last_change":"2023-09-10T00:40:17.700994+0000","last_active":"2023-09-10T00:40:17.701421+0000","last_peered":"2023-09-10T00:40:17.701421+0000","last_clean":"2023-09-10T00:40:17.701421+0000","last_became_active":"2023-09-10T00:40:17.690073+0000","last_became_peered":"2023-09-10T00:40:17.690073+0000","last_unstale":"2023-09-10T00:40:17.701421+0000","last_undegraded":"2023-09-10T00:40:17.701421+0000","last_fullsized":"2023-09-10T00:40:17.701421+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:43:32.478865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681570+0000","last_change":"2023-09-10T00:38:43.786272+0000","last_active":"2023-09-10T00:40:16.681570+0000","last_peered":"2023-09-10T00:40:16.681570+0000","last_clean":"2023-09-10T00:40:16.681570+0000","last_became_active":"2023-09-10T00:38:43.785934+0000","last_became_peered":"2023-09-10T00:38:43.785934+0000","last_unstale":"2023-09-10T00:40:16.681570+0000","last_undegraded":"2023-09-10T00:40:16.681570+0000","last_fullsized":"2023-09-10T00:40:16.681570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:31:49.721133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684970+0000","last_change":"2023-09-10T00:38:38.763263+0000","last_active":"2023-09-10T00:40:16.684970+0000","last_peered":"2023-09-10T00:40:16.684970+0000","last_clean":"2023-09-10T00:40:16.684970+0000","last_became_active":"2023-09-10T00:38:38.762911+0000","last_became_peered":"2023-09-10T00:38:38.762911+0000","last_unstale":"2023-09-10T00:40:16.684970+0000","last_undegraded":"2023-09-10T00:40:16.684970+0000","last_fullsized":"2023-09-10T00:40:16.684970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:19:10.292311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674491+0000","last_change":"2023-09-10T00:38:39.759171+0000","last_active":"2023-09-10T00:40:16.674491+0000","last_peered":"2023-09-10T00:40:16.674491+0000","last_clean":"2023-09-10T00:40:16.674491+0000","last_became_active":"2023-09-10T00:38:39.758843+0000","last_became_peered":"2023-09-10T00:38:39.758843+0000","last_unstale":"2023-09-10T00:40:16.674491+0000","last_undegraded":"2023-09-10T00:40:16.674491+0000","last_fullsized":"2023-09-10T00:40:16.674491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:50:36.064937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674438+0000","last_change":"2023-09-10T00:38:43.837998+0000","last_active":"2023-09-10T00:40:16.674438+0000","last_peered":"2023-09-10T00:40:16.674438+0000","last_clean":"2023-09-10T00:40:16.674438+0000","last_became_active":"2023-09-10T00:38:43.837556+0000","last_became_peered":"2023-09-10T00:38:43.837556+0000","last_unstale":"2023-09-10T00:40:16.674438+0000","last_undegraded":"2023-09-10T00:40:16.674438+0000","last_fullsized":"2023-09-10T00:40:16.674438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:18:40.488279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:18.399592+0000","last_change":"2023-09-10T00:38:38.759819+0000","last_active":"2023-09-10T00:40:18.399592+0000","last_peered":"2023-09-10T00:40:18.399592+0000","last_clean":"2023-09-10T00:40:18.399592+0000","last_became_active":"2023-09-10T00:38:38.759490+0000","last_became_peered":"2023-09-10T00:38:38.759490+0000","last_unstale":"2023-09-10T00:40:18.399592+0000","last_undegraded":"2023-09-10T00:40:18.399592+0000","last_fullsized":"2023-09-10T00:40:18.399592+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:39:01.206817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675634+0000","last_change":"2023-09-10T00:38:39.769669+0000","last_active":"2023-09-10T00:40:16.675634+0000","last_peered":"2023-09-10T00:40:16.675634+0000","last_clean":"2023-09-10T00:40:16.675634+0000","last_became_active":"2023-09-10T00:38:39.769402+0000","last_became_peered":"2023-09-10T00:38:39.769402+0000","last_unstale":"2023-09-10T00:40:16.675634+0000","last_undegraded":"2023-09-10T00:40:16.675634+0000","last_fullsized":"2023-09-10T00:40:16.675634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:00:50.215103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685982+0000","last_change":"2023-09-10T00:38:43.787518+0000","last_active":"2023-09-10T00:40:16.685982+0000","last_peered":"2023-09-10T00:40:16.685982+0000","last_clean":"2023-09-10T00:40:16.685982+0000","last_became_active":"2023-09-10T00:38:43.786410+0000","last_became_peered":"2023-09-10T00:38:43.786410+0000","last_unstale":"2023-09-10T00:40:16.685982+0000","last_undegraded":"2023-09-10T00:40:16.685982+0000","last_fullsized":"2023-09-10T00:40:16.685982+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:13:37.903875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.687579+0000","last_change":"2023-09-10T00:38:38.760992+0000","last_active":"2023-09-10T00:40:16.687579+0000","last_peered":"2023-09-10T00:40:16.687579+0000","last_clean":"2023-09-10T00:40:16.687579+0000","last_became_active":"2023-09-10T00:38:38.760635+0000","last_became_peered":"2023-09-10T00:38:38.760635+0000","last_unstale":"2023-09-10T00:40:16.687579+0000","last_undegraded":"2023-09-10T00:40:16.687579+0000","last_fullsized":"2023-09-10T00:40:16.687579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:29:36.070477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689409+0000","last_change":"2023-09-10T00:38:39.773771+0000","last_active":"2023-09-10T00:40:16.689409+0000","last_peered":"2023-09-10T00:40:16.689409+0000","last_clean":"2023-09-10T00:40:16.689409+0000","last_became_active":"2023-09-10T00:38:39.773296+0000","last_became_peered":"2023-09-10T00:38:39.773296+0000","last_unstale":"2023-09-10T00:40:16.689409+0000","last_undegraded":"2023-09-10T00:40:16.689409+0000","last_fullsized":"2023-09-10T00:40:16.689409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:13:01.194358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.687211+0000","last_change":"2023-09-10T00:39:17.536853+0000","last_active":"2023-09-10T00:40:16.687211+0000","last_peered":"2023-09-10T00:40:16.687211+0000","last_clean":"2023-09-10T00:40:16.687211+0000","last_became_active":"2023-09-10T00:39:17.536287+0000","last_became_peered":"2023-09-10T00:39:17.536287+0000","last_unstale":"2023-09-10T00:40:16.687211+0000","last_undegraded":"2023-09-10T00:40:16.687211+0000","last_fullsized":"2023-09-10T00:40:16.687211+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:50:35.294055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681867+0000","last_change":"2023-09-10T00:38:38.764954+0000","last_active":"2023-09-10T00:40:16.681867+0000","last_peered":"2023-09-10T00:40:16.681867+0000","last_clean":"2023-09-10T00:40:16.681867+0000","last_became_active":"2023-09-10T00:38:38.764589+0000","last_became_peered":"2023-09-10T00:38:38.764589+0000","last_unstale":"2023-09-10T00:40:16.681867+0000","last_undegraded":"2023-09-10T00:40:16.681867+0000","last_fullsized":"2023-09-10T00:40:16.681867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:40:19.559285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684134+0000","last_change":"2023-09-10T00:38:39.766141+0000","last_active":"2023-09-10T00:40:16.684134+0000","last_peered":"2023-09-10T00:40:16.684134+0000","last_clean":"2023-09-10T00:40:16.684134+0000","last_became_active":"2023-09-10T00:38:39.765832+0000","last_became_peered":"2023-09-10T00:38:39.765832+0000","last_unstale":"2023-09-10T00:40:16.684134+0000","last_undegraded":"2023-09-10T00:40:16.684134+0000","last_fullsized":"2023-09-10T00:40:16.684134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:08:45.149973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686352+0000","last_change":"2023-09-10T00:38:43.785111+0000","last_active":"2023-09-10T00:40:16.686352+0000","last_peered":"2023-09-10T00:40:16.686352+0000","last_clean":"2023-09-10T00:40:16.686352+0000","last_became_active":"2023-09-10T00:38:43.784542+0000","last_became_peered":"2023-09-10T00:38:43.784542+0000","last_unstale":"2023-09-10T00:40:16.686352+0000","last_undegraded":"2023-09-10T00:40:16.686352+0000","last_fullsized":"2023-09-10T00:40:16.686352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:34:51.611608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674667+0000","last_change":"2023-09-10T00:38:38.759984+0000","last_active":"2023-09-10T00:40:16.674667+0000","last_peered":"2023-09-10T00:40:16.674667+0000","last_clean":"2023-09-10T00:40:16.674667+0000","last_became_active":"2023-09-10T00:38:38.759660+0000","last_became_peered":"2023-09-10T00:38:38.759660+0000","last_unstale":"2023-09-10T00:40:16.674667+0000","last_undegraded":"2023-09-10T00:40:16.674667+0000","last_fullsized":"2023-09-10T00:40:16.674667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:02:11.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.683063+0000","last_change":"2023-09-10T00:38:39.770765+0000","last_active":"2023-09-10T00:40:16.683063+0000","last_peered":"2023-09-10T00:40:16.683063+0000","last_clean":"2023-09-10T00:40:16.683063+0000","last_became_active":"2023-09-10T00:38:39.770483+0000","last_became_peered":"2023-09-10T00:38:39.770483+0000","last_unstale":"2023-09-10T00:40:16.683063+0000","last_undegraded":"2023-09-10T00:40:16.683063+0000","last_fullsized":"2023-09-10T00:40:16.683063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:57:28.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.687049+0000","last_change":"2023-09-10T00:38:43.790945+0000","last_active":"2023-09-10T00:40:16.687049+0000","last_peered":"2023-09-10T00:40:16.687049+0000","last_clean":"2023-09-10T00:40:16.687049+0000","last_became_active":"2023-09-10T00:38:43.790467+0000","last_became_peered":"2023-09-10T00:38:43.790467+0000","last_unstale":"2023-09-10T00:40:16.687049+0000","last_undegraded":"2023-09-10T00:40:16.687049+0000","last_fullsized":"2023-09-10T00:40:16.687049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:44:07.041302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.676240+0000","last_change":"2023-09-10T00:38:38.765912+0000","last_active":"2023-09-10T00:40:17.676240+0000","last_peered":"2023-09-10T00:40:17.676240+0000","last_clean":"2023-09-10T00:40:17.676240+0000","last_became_active":"2023-09-10T00:38:38.765586+0000","last_became_peered":"2023-09-10T00:38:38.765586+0000","last_unstale":"2023-09-10T00:40:17.676240+0000","last_undegraded":"2023-09-10T00:40:17.676240+0000","last_fullsized":"2023-09-10T00:40:17.676240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:25:05.974158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675205+0000","last_change":"2023-09-10T00:38:39.771462+0000","last_active":"2023-09-10T00:40:16.675205+0000","last_peered":"2023-09-10T00:40:16.675205+0000","last_clean":"2023-09-10T00:40:16.675205+0000","last_became_active":"2023-09-10T00:38:39.771153+0000","last_became_peered":"2023-09-10T00:38:39.771153+0000","last_unstale":"2023-09-10T00:40:16.675205+0000","last_undegraded":"2023-09-10T00:40:16.675205+0000","last_fullsized":"2023-09-10T00:40:16.675205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:03:35.312186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.687296+0000","last_change":"2023-09-10T00:38:43.838446+0000","last_active":"2023-09-10T00:40:16.687296+0000","last_peered":"2023-09-10T00:40:16.687296+0000","last_clean":"2023-09-10T00:40:16.687296+0000","last_became_active":"2023-09-10T00:38:43.838119+0000","last_became_peered":"2023-09-10T00:38:43.838119+0000","last_unstale":"2023-09-10T00:40:16.687296+0000","last_undegraded":"2023-09-10T00:40:16.687296+0000","last_fullsized":"2023-09-10T00:40:16.687296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:12:20.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.734911+0000","last_change":"2023-09-10T00:40:17.734911+0000","last_active":"2023-09-10T00:40:17.734911+0000","last_peered":"2023-09-10T00:40:17.734911+0000","last_clean":"2023-09-10T00:40:17.734911+0000","last_became_active":"2023-09-10T00:40:17.730465+0000","last_became_peered":"2023-09-10T00:40:17.730465+0000","last_unstale":"2023-09-10T00:40:17.734911+0000","last_undegraded":"2023-09-10T00:40:17.734911+0000","last_fullsized":"2023-09-10T00:40:17.734911+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:42:17.592032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684224+0000","last_change":"2023-09-10T00:38:39.771995+0000","last_active":"2023-09-10T00:40:16.684224+0000","last_peered":"2023-09-10T00:40:16.684224+0000","last_clean":"2023-09-10T00:40:16.684224+0000","last_became_active":"2023-09-10T00:38:39.771693+0000","last_became_peered":"2023-09-10T00:38:39.771693+0000","last_unstale":"2023-09-10T00:40:16.684224+0000","last_undegraded":"2023-09-10T00:40:16.684224+0000","last_fullsized":"2023-09-10T00:40:16.684224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:41:45.621060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.680916+0000","last_change":"2023-09-10T00:38:43.789159+0000","last_active":"2023-09-10T00:40:16.680916+0000","last_peered":"2023-09-10T00:40:16.680916+0000","last_clean":"2023-09-10T00:40:16.680916+0000","last_became_active":"2023-09-10T00:38:43.788675+0000","last_became_peered":"2023-09-10T00:38:43.788675+0000","last_unstale":"2023-09-10T00:40:16.680916+0000","last_undegraded":"2023-09-10T00:40:16.680916+0000","last_fullsized":"2023-09-10T00:40:16.680916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:30:39.545561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'6","reported_seq":110,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:21.647139+0000","last_change":"2023-09-10T00:38:38.767245+0000","last_active":"2023-09-10T00:40:21.647139+0000","last_peered":"2023-09-10T00:40:21.647139+0000","last_clean":"2023-09-10T00:40:21.647139+0000","last_became_active":"2023-09-10T00:38:38.766952+0000","last_became_peered":"2023-09-10T00:38:38.766952+0000","last_unstale":"2023-09-10T00:40:21.647139+0000","last_undegraded":"2023-09-10T00:40:21.647139+0000","last_fullsized":"2023-09-10T00:40:21.647139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:57:23.309838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1653,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.683345+0000","last_change":"2023-09-10T00:38:39.772199+0000","last_active":"2023-09-10T00:40:16.683345+0000","last_peered":"2023-09-10T00:40:16.683345+0000","last_clean":"2023-09-10T00:40:16.683345+0000","last_became_active":"2023-09-10T00:38:39.771917+0000","last_became_peered":"2023-09-10T00:38:39.771917+0000","last_unstale":"2023-09-10T00:40:16.683345+0000","last_undegraded":"2023-09-10T00:40:16.683345+0000","last_fullsized":"2023-09-10T00:40:16.683345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:00:52.824581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.688815+0000","last_change":"2023-09-10T00:38:43.797107+0000","last_active":"2023-09-10T00:40:16.688815+0000","last_peered":"2023-09-10T00:40:16.688815+0000","last_clean":"2023-09-10T00:40:16.688815+0000","last_became_active":"2023-09-10T00:38:43.796606+0000","last_became_peered":"2023-09-10T00:38:43.796606+0000","last_unstale":"2023-09-10T00:40:16.688815+0000","last_undegraded":"2023-09-10T00:40:16.688815+0000","last_fullsized":"2023-09-10T00:40:16.688815+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:17:00.444238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676254+0000","last_change":"2023-09-10T00:38:38.768045+0000","last_active":"2023-09-10T00:40:16.676254+0000","last_peered":"2023-09-10T00:40:16.676254+0000","last_clean":"2023-09-10T00:40:16.676254+0000","last_became_active":"2023-09-10T00:38:38.767344+0000","last_became_peered":"2023-09-10T00:38:38.767344+0000","last_unstale":"2023-09-10T00:40:16.676254+0000","last_undegraded":"2023-09-10T00:40:16.676254+0000","last_fullsized":"2023-09-10T00:40:16.676254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:51:08.765412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.683838+0000","last_change":"2023-09-10T00:39:17.536408+0000","last_active":"2023-09-10T00:40:16.683838+0000","last_peered":"2023-09-10T00:40:16.683838+0000","last_clean":"2023-09-10T00:40:16.683838+0000","last_became_active":"2023-09-10T00:39:17.535980+0000","last_became_peered":"2023-09-10T00:39:17.535980+0000","last_unstale":"2023-09-10T00:40:16.683838+0000","last_undegraded":"2023-09-10T00:40:16.683838+0000","last_fullsized":"2023-09-10T00:40:16.683838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:08:10.517535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675080+0000","last_change":"2023-09-10T00:38:43.784928+0000","last_active":"2023-09-10T00:40:16.675080+0000","last_peered":"2023-09-10T00:40:16.675080+0000","last_clean":"2023-09-10T00:40:16.675080+0000","last_became_active":"2023-09-10T00:38:43.784479+0000","last_became_peered":"2023-09-10T00:38:43.784479+0000","last_unstale":"2023-09-10T00:40:16.675080+0000","last_undegraded":"2023-09-10T00:40:16.675080+0000","last_fullsized":"2023-09-10T00:40:16.675080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:14:45.019988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674218+0000","last_change":"2023-09-10T00:38:38.770730+0000","last_active":"2023-09-10T00:40:16.674218+0000","last_peered":"2023-09-10T00:40:16.674218+0000","last_clean":"2023-09-10T00:40:16.674218+0000","last_became_active":"2023-09-10T00:38:38.770344+0000","last_became_peered":"2023-09-10T00:38:38.770344+0000","last_unstale":"2023-09-10T00:40:16.674218+0000","last_undegraded":"2023-09-10T00:40:16.674218+0000","last_fullsized":"2023-09-10T00:40:16.674218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:49:22.881956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682628+0000","last_change":"2023-09-10T00:38:39.766149+0000","last_active":"2023-09-10T00:40:16.682628+0000","last_peered":"2023-09-10T00:40:16.682628+0000","last_clean":"2023-09-10T00:40:16.682628+0000","last_became_active":"2023-09-10T00:38:39.765773+0000","last_became_peered":"2023-09-10T00:38:39.765773+0000","last_unstale":"2023-09-10T00:40:16.682628+0000","last_undegraded":"2023-09-10T00:40:16.682628+0000","last_fullsized":"2023-09-10T00:40:16.682628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:00:37.854168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682825+0000","last_change":"2023-09-10T00:38:39.765197+0000","last_active":"2023-09-10T00:40:16.682825+0000","last_peered":"2023-09-10T00:40:16.682825+0000","last_clean":"2023-09-10T00:40:16.682825+0000","last_became_active":"2023-09-10T00:38:39.764891+0000","last_became_peered":"2023-09-10T00:38:39.764891+0000","last_unstale":"2023-09-10T00:40:16.682825+0000","last_undegraded":"2023-09-10T00:40:16.682825+0000","last_fullsized":"2023-09-10T00:40:16.682825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:47:27.429355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686326+0000","last_change":"2023-09-10T00:38:39.764722+0000","last_active":"2023-09-10T00:40:16.686326+0000","last_peered":"2023-09-10T00:40:16.686326+0000","last_clean":"2023-09-10T00:40:16.686326+0000","last_became_active":"2023-09-10T00:38:39.764376+0000","last_became_peered":"2023-09-10T00:38:39.764376+0000","last_unstale":"2023-09-10T00:40:16.686326+0000","last_undegraded":"2023-09-10T00:40:16.686326+0000","last_fullsized":"2023-09-10T00:40:16.686326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:27:36.708573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.679566+0000","last_change":"2023-09-10T00:38:39.771997+0000","last_active":"2023-09-10T00:40:16.679566+0000","last_peered":"2023-09-10T00:40:16.679566+0000","last_clean":"2023-09-10T00:40:16.679566+0000","last_became_active":"2023-09-10T00:38:39.771588+0000","last_became_peered":"2023-09-10T00:38:39.771588+0000","last_unstale":"2023-09-10T00:40:16.679566+0000","last_undegraded":"2023-09-10T00:40:16.679566+0000","last_fullsized":"2023-09-10T00:40:16.679566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:20:07.171333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689101+0000","last_change":"2023-09-10T00:38:39.773393+0000","last_active":"2023-09-10T00:40:16.689101+0000","last_peered":"2023-09-10T00:40:16.689101+0000","last_clean":"2023-09-10T00:40:16.689101+0000","last_became_active":"2023-09-10T00:38:39.772972+0000","last_became_peered":"2023-09-10T00:38:39.772972+0000","last_unstale":"2023-09-10T00:40:16.689101+0000","last_undegraded":"2023-09-10T00:40:16.689101+0000","last_fullsized":"2023-09-10T00:40:16.689101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:27:29.916970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674401+0000","last_change":"2023-09-10T00:38:39.767871+0000","last_active":"2023-09-10T00:40:16.674401+0000","last_peered":"2023-09-10T00:40:16.674401+0000","last_clean":"2023-09-10T00:40:16.674401+0000","last_became_active":"2023-09-10T00:38:39.767563+0000","last_became_peered":"2023-09-10T00:38:39.767563+0000","last_unstale":"2023-09-10T00:40:16.674401+0000","last_undegraded":"2023-09-10T00:40:16.674401+0000","last_fullsized":"2023-09-10T00:40:16.674401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:22:30.400441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682225+0000","last_change":"2023-09-10T00:38:39.764664+0000","last_active":"2023-09-10T00:40:16.682225+0000","last_peered":"2023-09-10T00:40:16.682225+0000","last_clean":"2023-09-10T00:40:16.682225+0000","last_became_active":"2023-09-10T00:38:39.764364+0000","last_became_peered":"2023-09-10T00:38:39.764364+0000","last_unstale":"2023-09-10T00:40:16.682225+0000","last_undegraded":"2023-09-10T00:40:16.682225+0000","last_fullsized":"2023-09-10T00:40:16.682225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:41.656472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685830+0000","last_change":"2023-09-10T00:38:39.770315+0000","last_active":"2023-09-10T00:40:16.685830+0000","last_peered":"2023-09-10T00:40:16.685830+0000","last_clean":"2023-09-10T00:40:16.685830+0000","last_became_active":"2023-09-10T00:38:39.769977+0000","last_became_peered":"2023-09-10T00:38:39.769977+0000","last_unstale":"2023-09-10T00:40:16.685830+0000","last_undegraded":"2023-09-10T00:40:16.685830+0000","last_fullsized":"2023-09-10T00:40:16.685830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:30:13.380487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675805+0000","last_change":"2023-09-10T00:38:39.768734+0000","last_active":"2023-09-10T00:40:16.675805+0000","last_peered":"2023-09-10T00:40:16.675805+0000","last_clean":"2023-09-10T00:40:16.675805+0000","last_became_active":"2023-09-10T00:38:39.768383+0000","last_became_peered":"2023-09-10T00:38:39.768383+0000","last_unstale":"2023-09-10T00:40:16.675805+0000","last_undegraded":"2023-09-10T00:40:16.675805+0000","last_fullsized":"2023-09-10T00:40:16.675805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:09.974447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684233+0000","last_change":"2023-09-10T00:39:17.732561+0000","last_active":"2023-09-10T00:40:16.684233+0000","last_peered":"2023-09-10T00:40:16.684233+0000","last_clean":"2023-09-10T00:40:16.684233+0000","last_became_active":"2023-09-10T00:39:17.732138+0000","last_became_peered":"2023-09-10T00:39:17.732138+0000","last_unstale":"2023-09-10T00:40:16.684233+0000","last_undegraded":"2023-09-10T00:40:16.684233+0000","last_fullsized":"2023-09-10T00:40:16.684233+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:11:58.866881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674781+0000","last_change":"2023-09-10T00:38:39.773839+0000","last_active":"2023-09-10T00:40:16.674781+0000","last_peered":"2023-09-10T00:40:16.674781+0000","last_clean":"2023-09-10T00:40:16.674781+0000","last_became_active":"2023-09-10T00:38:39.773521+0000","last_became_peered":"2023-09-10T00:38:39.773521+0000","last_unstale":"2023-09-10T00:40:16.674781+0000","last_undegraded":"2023-09-10T00:40:16.674781+0000","last_fullsized":"2023-09-10T00:40:16.674781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:15:49.033374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682785+0000","last_change":"2023-09-10T00:38:39.775721+0000","last_active":"2023-09-10T00:40:16.682785+0000","last_peered":"2023-09-10T00:40:16.682785+0000","last_clean":"2023-09-10T00:40:16.682785+0000","last_became_active":"2023-09-10T00:38:39.775424+0000","last_became_peered":"2023-09-10T00:38:39.775424+0000","last_unstale":"2023-09-10T00:40:16.682785+0000","last_undegraded":"2023-09-10T00:40:16.682785+0000","last_fullsized":"2023-09-10T00:40:16.682785+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:56:26.079332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674146+0000","last_change":"2023-09-10T00:38:39.768767+0000","last_active":"2023-09-10T00:40:16.674146+0000","last_peered":"2023-09-10T00:40:16.674146+0000","last_clean":"2023-09-10T00:40:16.674146+0000","last_became_active":"2023-09-10T00:38:39.768452+0000","last_became_peered":"2023-09-10T00:38:39.768452+0000","last_unstale":"2023-09-10T00:40:16.674146+0000","last_undegraded":"2023-09-10T00:40:16.674146+0000","last_fullsized":"2023-09-10T00:40:16.674146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:31:53.578574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681759+0000","last_change":"2023-09-10T00:38:39.761998+0000","last_active":"2023-09-10T00:40:16.681759+0000","last_peered":"2023-09-10T00:40:16.681759+0000","last_clean":"2023-09-10T00:40:16.681759+0000","last_became_active":"2023-09-10T00:38:39.761672+0000","last_became_peered":"2023-09-10T00:38:39.761672+0000","last_unstale":"2023-09-10T00:40:16.681759+0000","last_undegraded":"2023-09-10T00:40:16.681759+0000","last_fullsized":"2023-09-10T00:40:16.681759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:03:24.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73067,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":880,"num_read_kb":526,"num_write":255,"num_write_kb":153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":58848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":210561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449120,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":510852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.3150000000000004,"5min":4.3150000000000004,"15min":4.3150000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.54700000000000004,"15min":0.50800000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":6.3470000000000004,"5min":6.3470000000000004,"15min":6.3470000000000004},"last":0.41199999999999998}]},{"osd":1,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.499,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81899999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.64100000000000001},"min":{"1min":0.435,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.60499999999999998}]},{"osd":2,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48699999999999999,"15min":0.48799999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.47999999999999998,"15min":0.47699999999999998},"min":{"1min":0.39100000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":4,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67100000000000004,"15min":0.66900000000000004},"min":{"1min":0.51700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.83999999999999997,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.65200000000000002}]},{"osd":5,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77600000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.434}]},{"osd":6,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.747,"15min":0.71999999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":6.4260000000000002,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.79900000000000004,"15min":0.76800000000000002},"min":{"1min":0.50900000000000001,"5min":0.498,"15min":0.498},"max":{"1min":6.4720000000000004,"5min":6.4720000000000004,"15min":6.4720000000000004},"last":0.67300000000000004}]},{"osd":7,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76500000000000001,"15min":0.77000000000000002},"min":{"1min":0.54500000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89900000000000002,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.755,"15min":0.75800000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90900000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.75600000000000001}]},{"osd":8,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71199999999999997}]},{"osd":9,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.81299999999999994,"15min":0.78900000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":4.4859999999999998,"5min":4.4859999999999998,"15min":4.4859999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.75600000000000001,"15min":0.748},"min":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.622}]},{"osd":10,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.57599999999999996,"5min":0.5,"15min":0.5},"max":{"1min":0.97899999999999998,"5min":1.486,"15min":1.486},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81499999999999995,"15min":0.81699999999999995},"min":{"1min":0.56699999999999995,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96799999999999997,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.78900000000000003}]}]},{"osd":10,"up_from":70,"seq":300647710757,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28708,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716444,"statfs":{"total":95995035648,"available":95965638656,"internally_reserved":0,"allocated":1732608,"data_stored":1161850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.502}]},{"osd":1,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51500000000000001,"15min":0.51900000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78200000000000003,"5min":0.873,"15min":0.873},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.36499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52900000000000003}]},{"osd":2,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.629,"15min":0.629},"min":{"1min":0.501,"5min":0.439,"15min":0.439},"max":{"1min":0.75800000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65400000000000003,"15min":0.64400000000000002},"min":{"1min":0.52400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61699999999999999}]},{"osd":3,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.61899999999999999,"15min":0.63700000000000001},"min":{"1min":0.39300000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70099999999999996,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.48899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.56899999999999995}]},{"osd":4,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.54600000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.624},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78100000000000003,"15min":0.78300000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.94299999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65400000000000003}]},{"osd":6,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.995,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.59999999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69399999999999995}]},{"osd":7,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71299999999999997}]},{"osd":8,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77700000000000002,"15min":0.77500000000000002},"min":{"1min":0.63900000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78600000000000003,"15min":0.78800000000000003},"min":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.70399999999999996}]},{"osd":9,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.86099999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55600000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.72799999999999998}]},{"osd":11,"last update":"Sun Sep 10 00:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80400000000000005,"15min":0.80200000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.71699999999999997}]}]},{"osd":9,"up_from":65,"seq":279172874282,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":514395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42499999999999999,"15min":0.44400000000000001},"min":{"1min":0.32300000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.504,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.42799999999999999,"15min":0.45600000000000002},"min":{"1min":0.28999999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.55500000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.59699999999999998,"15min":0.61299999999999999},"min":{"1min":0.43099999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81100000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70099999999999996,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59899999999999998}]},{"osd":2,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63600000000000001,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.74199999999999999,"5min":0.879,"15min":0.879},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77300000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.51200000000000001}]},{"osd":3,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.52900000000000003,"15min":0.56899999999999995},"min":{"1min":0.29899999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.74399999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.442,"5min":0.56899999999999995,"15min":0.61699999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72499999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42199999999999999}]},{"osd":4,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.60599999999999998,"15min":0.63200000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80500000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.66400000000000003,"15min":0.68799999999999994},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75900000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.621}]},{"osd":5,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72699999999999998,"15min":0.747},"min":{"1min":0.46000000000000002,"5min":0.435,"15min":0.435},"max":{"1min":3.3109999999999999,"5min":3.3109999999999999,"15min":3.3109999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.72899999999999998,"15min":0.76700000000000002},"min":{"1min":0.47399999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.78100000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.57399999999999995}]},{"osd":6,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.42499999999999999}]},{"osd":7,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.751,"15min":0.79500000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.79500000000000004,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.76800000000000002,"15min":0.81799999999999995},"min":{"1min":0.499,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.89200000000000002,"5min":1.071,"15min":1.071},"last":0.66300000000000003}]},{"osd":8,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73599999999999999,"15min":0.78600000000000003},"min":{"1min":0.47799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79000000000000004,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.79800000000000004,"15min":0.84799999999999998},"min":{"1min":0.53400000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88900000000000001,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.78600000000000003}]},{"osd":10,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.72899999999999998,"15min":0.749},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.77600000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.54000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.82799999999999996,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.76700000000000002}]},{"osd":11,"last update":"Sun Sep 10 00:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.69899999999999995}]}]},{"osd":8,"up_from":57,"seq":244813135918,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":510346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76100000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64400000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61499999999999999,"15min":0.59699999999999998},"min":{"1min":0.49299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.874,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.69899999999999995,"15min":0.63600000000000001},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93999999999999995,"5min":6.5759999999999996,"15min":6.5759999999999996},"last":0.78900000000000003}]},{"osd":2,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.31,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.49399999999999999,"15min":0.48499999999999999},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59799999999999998}]},{"osd":3,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.85199999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.871,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.82999999999999996}]},{"osd":4,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.61399999999999999,"15min":0.59199999999999997},"min":{"1min":0.48299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.63600000000000001,"15min":0.61699999999999999},"min":{"1min":0.53700000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.76500000000000001}]},{"osd":5,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90000000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.74099999999999999}]},{"osd":6,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79800000000000004,"15min":0.755},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.026,"5min":6.5940000000000003,"15min":6.5940000000000003},"last":0.93799999999999994}]},{"osd":7,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79000000000000004,"15min":0.71999999999999997},"min":{"1min":0.58799999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0920000000000001,"5min":6.5439999999999996,"15min":6.5439999999999996},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.69199999999999995,"15min":0.67400000000000004},"min":{"1min":0.60399999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":1.028}]},{"osd":9,"last update":"Sun Sep 10 00:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.75700000000000001,"15min":0.73799999999999999},"min":{"1min":0.65700000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.002,"5min":1.103,"15min":1.103},"last":1.002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.76100000000000001,"15min":0.73099999999999998},"min":{"1min":0.70099999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.038,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":1.0149999999999999}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.57199999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.73999999999999999,"15min":0.71299999999999997},"min":{"1min":0.64200000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.95599999999999996}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.92000000000000004}]}]},{"osd":7,"up_from":50,"seq":214748364848,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1650688,"data_stored":1107514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54100000000000004,"15min":0.53200000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.39800000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40500000000000003,"15min":0.40000000000000002},"min":{"1min":0.32800000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.52600000000000002,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.40999999999999998,"15min":0.40200000000000002},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63600000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.38700000000000001}]},{"osd":2,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.79800000000000004,"5min":1.091,"15min":1.091},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.51400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85999999999999999,"5min":1.069,"15min":1.069},"last":0.81899999999999995}]},{"osd":3,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.36699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76900000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.46999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54900000000000004}]},{"osd":4,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.38300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.66300000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.497,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.59999999999999998,"5min":1.056,"15min":1.056},"last":0.47899999999999998}]},{"osd":5,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":0.628},"min":{"1min":0.49399999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.83299999999999996,"5min":1.026,"15min":1.026},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.48099999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98399999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.74199999999999999}]},{"osd":6,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.48899999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.84999999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89800000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.42399999999999999}]},{"osd":8,"last update":"Sun Sep 10 00:39:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.65200000000000002,"15min":0.626},"min":{"1min":0.44400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.86399999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67800000000000005,"15min":0.65700000000000003},"min":{"1min":0.496,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.86499999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72599999999999998}]},{"osd":9,"last update":"Sun Sep 10 00:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.53500000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80000000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69299999999999995,"15min":0.69599999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.83599999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.70999999999999996}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65700000000000003,"15min":0.65200000000000002},"min":{"1min":0.46100000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83299999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66600000000000004,"15min":0.65700000000000003},"min":{"1min":0.54000000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.64800000000000002}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77300000000000002}]}]},{"osd":6,"up_from":45,"seq":193273528374,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":506286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41699999999999998,"15min":0.40100000000000002},"min":{"1min":0.34399999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.63700000000000001,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.40799999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59699999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.34499999999999997}]},{"osd":1,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59799999999999998,"15min":0.56399999999999995},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82999999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81499999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.73799999999999999}]},{"osd":2,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69499999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.627,"15min":0.61099999999999999},"min":{"1min":0.39500000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80100000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.50600000000000001}]},{"osd":3,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48699999999999999,"15min":0.47899999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66400000000000003,"5min":0.875,"15min":0.875},"last":0.37},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68799999999999994,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40400000000000003}]},{"osd":4,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.46800000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82199999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69099999999999995,"15min":0.68000000000000005},"min":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93400000000000005,"5min":1.024,"15min":1.024},"last":0.69699999999999995}]},{"osd":5,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68000000000000005,"15min":0.67600000000000005},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.879,"5min":1.048,"15min":1.048},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.69899999999999995,"15min":0.69399999999999995},"min":{"1min":0.48699999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.48699999999999999}]},{"osd":7,"last update":"Sun Sep 10 00:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.73199999999999998,"15min":0.72899999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94899999999999995,"5min":1.069,"15min":1.069},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72999999999999998,"15min":0.72299999999999998},"min":{"1min":0.47799999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.82799999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.78200000000000003}]},{"osd":8,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.79100000000000004,"15min":0.82499999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.876,"5min":1.27,"15min":1.27},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.79500000000000004,"15min":0.82099999999999995},"min":{"1min":0.55600000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.92800000000000005,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.625}]},{"osd":9,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.78200000000000003,"15min":0.84399999999999997},"min":{"1min":0.438,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94399999999999995,"5min":1.389,"15min":1.389},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.72499999999999998,"15min":0.77000000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86199999999999999,"5min":1.099,"15min":1.099},"last":0.63600000000000001}]},{"osd":10,"last update":"Sun Sep 10 00:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.82599999999999996,"15min":0.83899999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.96299999999999997,"5min":1.181,"15min":1.181},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.91300000000000003,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.79900000000000004}]},{"osd":11,"last update":"Sun Sep 10 00:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":566647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52600000000000002,"15min":0.50900000000000001},"min":{"1min":0.32100000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72199999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.53200000000000003,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53500000000000003}]},{"osd":2,"last update":"Sun Sep 10 00:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53900000000000003,"15min":0.52200000000000002},"min":{"1min":0.308,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.58799999999999997,"15min":0.54500000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84599999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.86799999999999999}]},{"osd":3,"last update":"Sun Sep 10 00:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.70399999999999996,"5min":0.879,"15min":0.879},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53100000000000003,"15min":0.52600000000000002},"min":{"1min":0.38,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69599999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56100000000000005}]},{"osd":4,"last update":"Sun Sep 10 00:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.42299999999999999},"min":{"1min":0.35099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.46000000000000002,"15min":0.437},"min":{"1min":0.33800000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58299999999999996}]},{"osd":5,"last update":"Sun Sep 10 00:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60399999999999998,"15min":0.57299999999999995},"min":{"1min":0.38700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89400000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.63500000000000001},"min":{"1min":0.45700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0409999999999999,"5min":1.044,"15min":1.044},"last":0.81399999999999995}]},{"osd":6,"last update":"Sun Sep 10 00:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63900000000000001,"15min":0.61299999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86199999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63900000000000001,"15min":0.61499999999999999},"min":{"1min":0.40400000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83699999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68100000000000005}]},{"osd":7,"last update":"Sun Sep 10 00:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61399999999999999,"15min":0.57399999999999995},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89200000000000002,"5min":0.999,"15min":0.999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.621,"15min":0.58299999999999996},"min":{"1min":0.54300000000000004,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.878,"5min":0.996,"15min":0.996},"last":0.72799999999999998}]},{"osd":8,"last update":"Sun Sep 10 00:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94499999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88200000000000001,"5min":1.014,"15min":1.014},"last":0.88400000000000001}]},{"osd":9,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.998,"5min":1.097,"15min":1.097},"last":0.75},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77300000000000002,"15min":0.78600000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.93000000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65900000000000003}]},{"osd":10,"last update":"Sun Sep 10 00:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.76000000000000001,"15min":0.77200000000000002},"min":{"1min":0.46700000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93600000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.86199999999999999,"15min":0.86799999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0109999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.90800000000000003}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.93400000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":507742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 10 00:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54400000000000004,"15min":0.5},"min":{"1min":0.36399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.629},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.52500000000000002},"min":{"1min":0.376,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.64700000000000002}]},{"osd":2,"last update":"Sun Sep 10 00:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.53800000000000003},"min":{"1min":0.39400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.56899999999999995},"min":{"1min":0.372,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83999999999999997,"5min":1.621,"15min":1.621},"last":0.78100000000000003}]},{"osd":3,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.375},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.39900000000000002}]},{"osd":4,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.46300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73699999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.374,"15min":0.374},"max":{"1min":0.83999999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59499999999999997}]},{"osd":5,"last update":"Sun Sep 10 00:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65700000000000003,"15min":0.61599999999999999},"min":{"1min":0.42799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.91000000000000003,"5min":1.075,"15min":1.075},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64700000000000002,"15min":0.63700000000000001},"min":{"1min":0.47799999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85399999999999998,"5min":1.123,"15min":1.123},"last":0.86499999999999999}]},{"osd":6,"last update":"Sun Sep 10 00:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53900000000000003,"15min":0.52000000000000002},"min":{"1min":0.34000000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.42899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.97999999999999998,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.52300000000000002}]},{"osd":7,"last update":"Sun Sep 10 00:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.72199999999999998}]},{"osd":8,"last update":"Sun Sep 10 00:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76900000000000002,"15min":0.77800000000000002},"min":{"1min":0.60499999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.022,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":0.74199999999999999},"min":{"1min":0.49099999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97199999999999998,"5min":1.151,"15min":1.151},"last":0.88500000000000001}]},{"osd":9,"last update":"Sun Sep 10 00:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73899999999999999,"15min":0.752},"min":{"1min":0.54800000000000004,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0549999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.69399999999999995,"15min":0.71299999999999997},"min":{"1min":0.44800000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90300000000000002,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.54400000000000004}]},{"osd":10,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.746,"15min":0.74299999999999999},"min":{"1min":0.57999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.53400000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.90600000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.749}]},{"osd":11,"last update":"Sun Sep 10 00:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75,"15min":0.749},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.67400000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.085,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.81699999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":508550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.51300000000000001},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52700000000000002,"15min":0.52500000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72699999999999998,"5min":0.753,"15min":0.753},"last":0.64000000000000001}]},{"osd":1,"last update":"Sun Sep 10 00:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.58599999999999997,"15min":0.53900000000000003},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.58999999999999997,"15min":0.55000000000000004},"min":{"1min":0.34200000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.75700000000000001}]},{"osd":3,"last update":"Sun Sep 10 00:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57599999999999996,"15min":0.54000000000000004},"min":{"1min":0.43099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.501}]},{"osd":4,"last update":"Sun Sep 10 00:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.77700000000000002}]},{"osd":5,"last update":"Sun Sep 10 00:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.499,"15min":0.44900000000000001},"min":{"1min":0.41399999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.55800000000000005,"15min":0.51200000000000001},"min":{"1min":0.46200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.71699999999999997}]},{"osd":6,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.45800000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.97599999999999998,"5min":1.018,"15min":1.018},"last":0.621},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67500000000000004,"15min":0.64700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96499999999999997,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.68799999999999994}]},{"osd":7,"last update":"Sun Sep 10 00:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.80700000000000005}]},{"osd":8,"last update":"Sun Sep 10 00:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.58999999999999997,"15min":0.55600000000000005},"min":{"1min":0.45400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.63300000000000001,"15min":0.60699999999999998},"min":{"1min":0.47899999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.53600000000000003}]},{"osd":9,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76200000000000001,"15min":0.74099999999999999},"min":{"1min":0.60999999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0429999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.73699999999999999,"15min":0.71499999999999997},"min":{"1min":0.56999999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98799999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.79000000000000004}]},{"osd":10,"last update":"Sun Sep 10 00:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.70799999999999996,"15min":0.68500000000000005},"min":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.745,"15min":0.71899999999999997},"min":{"1min":0.68700000000000006,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.875}]},{"osd":11,"last update":"Sun Sep 10 00:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.83199999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":505511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40899999999999997,"15min":0.40100000000000002},"min":{"1min":0.26700000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.92000000000000004,"5min":1.409,"15min":1.409},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.42799999999999999,"15min":0.42099999999999999},"min":{"1min":0.31900000000000001,"5min":0.255,"15min":0.255},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.51400000000000001}]},{"osd":1,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.6699999999999999,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58699999999999997,"15min":0.55600000000000005},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78500000000000003,"5min":1.581,"15min":1.581},"last":0.746}]},{"osd":2,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.60399999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.65400000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.7110000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.79200000000000004}]},{"osd":4,"last update":"Sun Sep 10 00:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.39300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80600000000000005,"5min":1.536,"15min":1.536},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.66500000000000004,"15min":0.68799999999999994},"min":{"1min":0.38300000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.748,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.68899999999999995}]},{"osd":5,"last update":"Sun Sep 10 00:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.73299999999999998},"min":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7809999999999999,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68100000000000005,"15min":0.67600000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.872}]},{"osd":6,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60099999999999998,"15min":0.57999999999999996},"min":{"1min":0.36499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.65600000000000003}]},{"osd":7,"last update":"Sun Sep 10 00:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.74399999999999999,"15min":0.745},"min":{"1min":0.47699999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90800000000000003,"5min":1.022,"15min":1.022},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73799999999999999,"15min":0.73599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.82499999999999996}]},{"osd":8,"last update":"Sun Sep 10 00:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.89400000000000002}]},{"osd":9,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71399999999999997,"15min":0.75800000000000001},"min":{"1min":0.36499999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.77900000000000003,"15min":0.82499999999999996},"min":{"1min":0.47599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95899999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.84399999999999997}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.82999999999999996,"15min":0.83999999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.90600000000000003}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.95299999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":570222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45000000000000001}]},{"osd":1,"last update":"Sun Sep 10 00:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.37,"15min":0.34399999999999997},"min":{"1min":0.29199999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.317,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.56000000000000005}]},{"osd":2,"last update":"Sun Sep 10 00:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80200000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58799999999999997,"15min":0.55500000000000005},"min":{"1min":0.497,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.68200000000000005}]},{"osd":3,"last update":"Sun Sep 10 00:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.39900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.60999999999999999,"15min":0.56200000000000006},"min":{"1min":0.35099999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.59799999999999998}]},{"osd":5,"last update":"Sun Sep 10 00:39:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85299999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.59499999999999997},"min":{"1min":0.49399999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63400000000000001}]},{"osd":6,"last update":"Sun Sep 10 00:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.61599999999999999,"15min":0.57999999999999996},"min":{"1min":0.50700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64100000000000001,"15min":0.63800000000000001},"min":{"1min":0.40799999999999997,"5min":0.375,"15min":0.375},"max":{"1min":0.91300000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72599999999999998}]},{"osd":7,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.39300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.88500000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.56499999999999995,"15min":0.52800000000000002},"min":{"1min":0.38700000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85199999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.877}]},{"osd":8,"last update":"Sun Sep 10 00:39:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.44800000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91200000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.60499999999999998,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97699999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.93100000000000005}]},{"osd":9,"last update":"Sun Sep 10 00:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.73199999999999998,"15min":0.70599999999999996},"min":{"1min":0.63200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.74299999999999999,"15min":0.73699999999999999},"min":{"1min":0.65600000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":1.0760000000000001}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74399999999999999,"15min":0.75},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94999999999999996,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.74099999999999999,"15min":0.73699999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.94499999999999995}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.97599999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1662976,"data_stored":1103379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.37,"5min":0.315,"15min":0.315},"max":{"1min":0.746,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.376,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74399999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52400000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.52500000000000002},"min":{"1min":0.51800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.46000000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.747}]},{"osd":2,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.36899999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.60999999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.373},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.45900000000000002,"15min":0.437},"min":{"1min":0.38600000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.39600000000000002}]},{"osd":3,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.371,"5min":0.31,"15min":0.31},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.504},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.48899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.91100000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46999999999999997}]},{"osd":4,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.59999999999999998,"15min":0.56000000000000005},"min":{"1min":0.53200000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92900000000000005,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58799999999999997,"15min":0.55900000000000005},"min":{"1min":0.47999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78100000000000003,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57599999999999996}]},{"osd":6,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63300000000000001,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.9100000000000001,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":0.442},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.68400000000000005,"15min":0.66100000000000003},"min":{"1min":0.53600000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Sun Sep 10 00:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.629},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.64100000000000001,"15min":0.622},"min":{"1min":0.53700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.98399999999999999,"5min":1.046,"15min":1.046},"last":0.61499999999999999}]},{"osd":8,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.60499999999999998,"15min":0.56499999999999995},"min":{"1min":0.56699999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.56799999999999995,"15min":0.53200000000000003},"min":{"1min":0.48899999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.68400000000000005}]},{"osd":9,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.75900000000000001,"15min":0.72299999999999998},"min":{"1min":0.59999999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.9590000000000001,"5min":2.9590000000000001,"15min":2.9590000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.77200000000000002,"15min":0.72399999999999998},"min":{"1min":0.63100000000000001,"5min":0.435,"15min":0.435},"max":{"1min":2.9129999999999998,"5min":2.9129999999999998,"15min":2.9129999999999998},"last":0.77900000000000003}]},{"osd":10,"last update":"Sun Sep 10 00:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.69699999999999995,"15min":0.68300000000000005},"min":{"1min":0.54600000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.02,"5min":1.026,"15min":1.026},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.58599999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.76600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-10T00:40:26.206 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-10T00:40:26.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.361+0000 7fbbc4344700 1 -- 172.21.15.44:0/3562589556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc0ff4e0 msgr2=0x7fbbbc0ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:26.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.361+0000 7fbbc4344700 1 --2- 172.21.15.44:0/3562589556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc0ff4e0 0x7fbbbc0ff900 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00099c0 tx=0x7fbbb0009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.361+0000 7fbbc4344700 1 -- 172.21.15.44:0/3562589556 shutdown_connections 2023-09-10T00:40:26.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.361+0000 7fbbc4344700 1 --2- 172.21.15.44:0/3562589556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbbc101070 0x7fbbbc107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.361+0000 7fbbc4344700 1 --2- 172.21.15.44:0/3562589556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc0ff4e0 0x7fbbbc0ff900 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.361+0000 7fbbc4344700 1 --2- 172.21.15.44:0/3562589556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbbc1006d0 0x7fbbbc100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.361+0000 7fbbc4344700 1 -- 172.21.15.44:0/3562589556 >> 172.21.15.44:0/3562589556 conn(0x7fbbbc0faa30 msgr2=0x7fbbbc0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:26.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.361+0000 7fbbc4344700 1 -- 172.21.15.44:0/3562589556 shutdown_connections 2023-09-10T00:40:26.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.361+0000 7fbbc4344700 1 -- 172.21.15.44:0/3562589556 wait complete. 2023-09-10T00:40:26.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc4344700 1 Processor -- start 2023-09-10T00:40:26.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc4344700 1 -- start start 2023-09-10T00:40:26.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc4344700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbbc0ff4e0 0x7fbbbc103470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:26.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc4344700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbbc1006d0 0x7fbbbc101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:26.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc4344700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc101070 0x7fbbbc101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:26.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc4344700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc102790 con 0x7fbbbc101070 2023-09-10T00:40:26.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc4344700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc102900 con 0x7fbbbc1006d0 2023-09-10T00:40:26.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc4344700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc1a02c0 con 0x7fbbbc0ff4e0 2023-09-10T00:40:26.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc28e1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc101070 0x7fbbbc101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:26.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc28e1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc101070 0x7fbbbc101ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48416/0 (socket says 172.21.15.44:48416) 2023-09-10T00:40:26.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc28e1700 1 -- 172.21.15.44:0/521358577 learned_addr learned my addr 172.21.15.44:0/521358577 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:26.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc18df700 1 --2- 172.21.15.44:0/521358577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbbc1006d0 0x7fbbbc101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:26.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc20e0700 1 --2- 172.21.15.44:0/521358577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbbc0ff4e0 0x7fbbbc103470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:26.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc28e1700 1 -- 172.21.15.44:0/521358577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbbc0ff4e0 msgr2=0x7fbbbc103470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:26.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc28e1700 1 --2- 172.21.15.44:0/521358577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbbc0ff4e0 0x7fbbbc103470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc28e1700 1 -- 172.21.15.44:0/521358577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbbc1006d0 msgr2=0x7fbbbc101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:26.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc28e1700 1 --2- 172.21.15.44:0/521358577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbbc1006d0 0x7fbbbc101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.362+0000 7fbbc28e1700 1 -- 172.21.15.44:0/521358577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb00095f0 con 0x7fbbbc101070 2023-09-10T00:40:26.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.363+0000 7fbbc28e1700 1 --2- 172.21.15.44:0/521358577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc101070 0x7fbbbc101ff0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4007ee0 tx=0x7fbbb400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:26.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.363+0000 7fbbaf7fe700 1 -- 172.21.15.44:0/521358577 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb400dd20 con 0x7fbbbc101070 2023-09-10T00:40:26.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.363+0000 7fbbaf7fe700 1 -- 172.21.15.44:0/521358577 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbbb400f040 con 0x7fbbbc101070 2023-09-10T00:40:26.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.363+0000 7fbbaf7fe700 1 -- 172.21.15.44:0/521358577 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4004750 con 0x7fbbbc101070 2023-09-10T00:40:26.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.363+0000 7fbbc4344700 1 -- 172.21.15.44:0/521358577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbbc1a05a0 con 0x7fbbbc101070 2023-09-10T00:40:26.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.363+0000 7fbbc4344700 1 -- 172.21.15.44:0/521358577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbbc1a0a60 con 0x7fbbbc101070 2023-09-10T00:40:26.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.364+0000 7fbbc4344700 1 -- 172.21.15.44:0/521358577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbbc10ac00 con 0x7fbbbc101070 2023-09-10T00:40:26.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.365+0000 7fbbaf7fe700 1 -- 172.21.15.44:0/521358577 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fbbb4003a00 con 0x7fbbbc101070 2023-09-10T00:40:26.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.366+0000 7fbbaf7fe700 1 --2- 172.21.15.44:0/521358577 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbba80a0690 0x7fbba80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:26.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.366+0000 7fbbc20e0700 1 --2- 172.21.15.44:0/521358577 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbba80a0690 0x7fbba80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:26.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.366+0000 7fbbaf7fe700 1 -- 172.21.15.44:0/521358577 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7fbbb40cb7f0 con 0x7fbbbc101070 2023-09-10T00:40:26.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.366+0000 7fbbc20e0700 1 --2- 172.21.15.44:0/521358577 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbba80a0690 0x7fbba80a2b50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0006010 tx=0x7fbbb0005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:26.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.371+0000 7fbbaf7fe700 1 -- 172.21.15.44:0/521358577 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbbb404ee10 con 0x7fbbbc101070 2023-09-10T00:40:26.569 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.569+0000 7fbbc4344700 1 -- 172.21.15.44:0/521358577 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbbbc067eb0 con 0x7fbba80a0690 2023-09-10T00:40:26.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.584+0000 7fbbaf7fe700 1 -- 172.21.15.44:0/521358577 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427736 (secure 0 0 0) 0x7fbbbc067eb0 con 0x7fbba80a0690 2023-09-10T00:40:26.590 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:26.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.588+0000 7fbbc4344700 1 -- 172.21.15.44:0/521358577 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbba80a0690 msgr2=0x7fbba80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:26.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.588+0000 7fbbc4344700 1 --2- 172.21.15.44:0/521358577 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbba80a0690 0x7fbba80a2b50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0006010 tx=0x7fbbb0005e80 comp rx=0 tx=0).stop 2023-09-10T00:40:26.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.588+0000 7fbbc4344700 1 -- 172.21.15.44:0/521358577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc101070 msgr2=0x7fbbbc101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:26.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.588+0000 7fbbc4344700 1 --2- 172.21.15.44:0/521358577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc101070 0x7fbbbc101ff0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4007ee0 tx=0x7fbbb400d3b0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.588+0000 7fbbc4344700 1 -- 172.21.15.44:0/521358577 shutdown_connections 2023-09-10T00:40:26.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.588+0000 7fbbc4344700 1 --2- 172.21.15.44:0/521358577 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fbba80a0690 0x7fbba80a2b50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.597 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.588+0000 7fbbc4344700 1 --2- 172.21.15.44:0/521358577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbbc101070 0x7fbbbc101ff0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.588+0000 7fbbc4344700 1 --2- 172.21.15.44:0/521358577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbbc0ff4e0 0x7fbbbc103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.588+0000 7fbbc4344700 1 --2- 172.21.15.44:0/521358577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbbc1006d0 0x7fbbbc101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.588+0000 7fbbc4344700 1 -- 172.21.15.44:0/521358577 >> 172.21.15.44:0/521358577 conn(0x7fbbbc0faa30 msgr2=0x7fbbbc0677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:26.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.588+0000 7fbbc4344700 1 -- 172.21.15.44:0/521358577 shutdown_connections 2023-09-10T00:40:26.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.588+0000 7fbbc4344700 1 -- 172.21.15.44:0/521358577 wait complete. 2023-09-10T00:40:26.599 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2023-09-10T00:40:26.601 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2023-09-10T00:40:26.023414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663681,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1016,"num_read_kb":645,"num_write":438,"num_write_kb":3093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1239584,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":863,"ondisk_log_size":863,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338488,"kb_used_data":14272,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603336,"statfs":{"total":1151940427776,"available":1151593816064,"internally_reserved":0,"allocated":14614528,"data_stored":8073294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7266,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":60,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":58848,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.351178"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682673+0000","last_change":"2023-09-10T00:38:39.768218+0000","last_active":"2023-09-10T00:40:16.682673+0000","last_peered":"2023-09-10T00:40:16.682673+0000","last_clean":"2023-09-10T00:40:16.682673+0000","last_became_active":"2023-09-10T00:38:39.767454+0000","last_became_peered":"2023-09-10T00:38:39.767454+0000","last_unstale":"2023-09-10T00:40:16.682673+0000","last_undegraded":"2023-09-10T00:40:16.682673+0000","last_fullsized":"2023-09-10T00:40:16.682673+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:17:30.287403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689406+0000","last_change":"2023-09-10T00:38:39.765384+0000","last_active":"2023-09-10T00:40:16.689406+0000","last_peered":"2023-09-10T00:40:16.689406+0000","last_clean":"2023-09-10T00:40:16.689406+0000","last_became_active":"2023-09-10T00:38:39.765090+0000","last_became_peered":"2023-09-10T00:38:39.765090+0000","last_unstale":"2023-09-10T00:40:16.689406+0000","last_undegraded":"2023-09-10T00:40:16.689406+0000","last_fullsized":"2023-09-10T00:40:16.689406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:02:46.025627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689356+0000","last_change":"2023-09-10T00:38:39.769222+0000","last_active":"2023-09-10T00:40:16.689356+0000","last_peered":"2023-09-10T00:40:16.689356+0000","last_clean":"2023-09-10T00:40:16.689356+0000","last_became_active":"2023-09-10T00:38:39.768801+0000","last_became_peered":"2023-09-10T00:38:39.768801+0000","last_unstale":"2023-09-10T00:40:16.689356+0000","last_undegraded":"2023-09-10T00:40:16.689356+0000","last_fullsized":"2023-09-10T00:40:16.689356+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:38:53.394209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686934+0000","last_change":"2023-09-10T00:38:39.764963+0000","last_active":"2023-09-10T00:40:16.686934+0000","last_peered":"2023-09-10T00:40:16.686934+0000","last_clean":"2023-09-10T00:40:16.686934+0000","last_became_active":"2023-09-10T00:38:39.764466+0000","last_became_peered":"2023-09-10T00:38:39.764466+0000","last_unstale":"2023-09-10T00:40:16.686934+0000","last_undegraded":"2023-09-10T00:40:16.686934+0000","last_fullsized":"2023-09-10T00:40:16.686934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:31.553321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684022+0000","last_change":"2023-09-10T00:39:17.534684+0000","last_active":"2023-09-10T00:40:16.684022+0000","last_peered":"2023-09-10T00:40:16.684022+0000","last_clean":"2023-09-10T00:40:16.684022+0000","last_became_active":"2023-09-10T00:39:17.534325+0000","last_became_peered":"2023-09-10T00:39:17.534325+0000","last_unstale":"2023-09-10T00:40:16.684022+0000","last_undegraded":"2023-09-10T00:40:16.684022+0000","last_fullsized":"2023-09-10T00:40:16.684022+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:39:11.902221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684935+0000","last_change":"2023-09-10T00:38:39.761681+0000","last_active":"2023-09-10T00:40:16.684935+0000","last_peered":"2023-09-10T00:40:16.684935+0000","last_clean":"2023-09-10T00:40:16.684935+0000","last_became_active":"2023-09-10T00:38:39.761383+0000","last_became_peered":"2023-09-10T00:38:39.761383+0000","last_unstale":"2023-09-10T00:40:16.684935+0000","last_undegraded":"2023-09-10T00:40:16.684935+0000","last_fullsized":"2023-09-10T00:40:16.684935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:38:04.534656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684507+0000","last_change":"2023-09-10T00:38:39.765028+0000","last_active":"2023-09-10T00:40:16.684507+0000","last_peered":"2023-09-10T00:40:16.684507+0000","last_clean":"2023-09-10T00:40:16.684507+0000","last_became_active":"2023-09-10T00:38:39.764663+0000","last_became_peered":"2023-09-10T00:38:39.764663+0000","last_unstale":"2023-09-10T00:40:16.684507+0000","last_undegraded":"2023-09-10T00:40:16.684507+0000","last_fullsized":"2023-09-10T00:40:16.684507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:14:49.645129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674630+0000","last_change":"2023-09-10T00:38:39.770135+0000","last_active":"2023-09-10T00:40:16.674630+0000","last_peered":"2023-09-10T00:40:16.674630+0000","last_clean":"2023-09-10T00:40:16.674630+0000","last_became_active":"2023-09-10T00:38:39.769720+0000","last_became_peered":"2023-09-10T00:38:39.769720+0000","last_unstale":"2023-09-10T00:40:16.674630+0000","last_undegraded":"2023-09-10T00:40:16.674630+0000","last_fullsized":"2023-09-10T00:40:16.674630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:46:21.166566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.683408+0000","last_change":"2023-09-10T00:38:39.770170+0000","last_active":"2023-09-10T00:40:16.683408+0000","last_peered":"2023-09-10T00:40:16.683408+0000","last_clean":"2023-09-10T00:40:16.683408+0000","last_became_active":"2023-09-10T00:38:39.769840+0000","last_became_peered":"2023-09-10T00:38:39.769840+0000","last_unstale":"2023-09-10T00:40:16.683408+0000","last_undegraded":"2023-09-10T00:40:16.683408+0000","last_fullsized":"2023-09-10T00:40:16.683408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:48.117670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685644+0000","last_change":"2023-09-10T00:38:39.769621+0000","last_active":"2023-09-10T00:40:16.685644+0000","last_peered":"2023-09-10T00:40:16.685644+0000","last_clean":"2023-09-10T00:40:16.685644+0000","last_became_active":"2023-09-10T00:38:39.769300+0000","last_became_peered":"2023-09-10T00:38:39.769300+0000","last_unstale":"2023-09-10T00:40:16.685644+0000","last_undegraded":"2023-09-10T00:40:16.685644+0000","last_fullsized":"2023-09-10T00:40:16.685644+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:37.100671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682752+0000","last_change":"2023-09-10T00:38:39.772200+0000","last_active":"2023-09-10T00:40:16.682752+0000","last_peered":"2023-09-10T00:40:16.682752+0000","last_clean":"2023-09-10T00:40:16.682752+0000","last_became_active":"2023-09-10T00:38:39.771503+0000","last_became_peered":"2023-09-10T00:38:39.771503+0000","last_unstale":"2023-09-10T00:40:16.682752+0000","last_undegraded":"2023-09-10T00:40:16.682752+0000","last_fullsized":"2023-09-10T00:40:16.682752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:13:38.830642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686922+0000","last_change":"2023-09-10T00:38:39.769832+0000","last_active":"2023-09-10T00:40:16.686922+0000","last_peered":"2023-09-10T00:40:16.686922+0000","last_clean":"2023-09-10T00:40:16.686922+0000","last_became_active":"2023-09-10T00:38:39.769546+0000","last_became_peered":"2023-09-10T00:38:39.769546+0000","last_unstale":"2023-09-10T00:40:16.686922+0000","last_undegraded":"2023-09-10T00:40:16.686922+0000","last_fullsized":"2023-09-10T00:40:16.686922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:53:05.219877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674379+0000","last_change":"2023-09-10T00:39:17.531430+0000","last_active":"2023-09-10T00:40:16.674379+0000","last_peered":"2023-09-10T00:40:16.674379+0000","last_clean":"2023-09-10T00:40:16.674379+0000","last_became_active":"2023-09-10T00:39:17.531145+0000","last_became_peered":"2023-09-10T00:39:17.531145+0000","last_unstale":"2023-09-10T00:40:16.674379+0000","last_undegraded":"2023-09-10T00:40:16.674379+0000","last_fullsized":"2023-09-10T00:40:16.674379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:34:50.163747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.688856+0000","last_change":"2023-09-10T00:38:39.764751+0000","last_active":"2023-09-10T00:40:16.688856+0000","last_peered":"2023-09-10T00:40:16.688856+0000","last_clean":"2023-09-10T00:40:16.688856+0000","last_became_active":"2023-09-10T00:38:39.764464+0000","last_became_peered":"2023-09-10T00:38:39.764464+0000","last_unstale":"2023-09-10T00:40:16.688856+0000","last_undegraded":"2023-09-10T00:40:16.688856+0000","last_fullsized":"2023-09-10T00:40:16.688856+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:03:51.096126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.688911+0000","last_change":"2023-09-10T00:39:17.532977+0000","last_active":"2023-09-10T00:40:16.688911+0000","last_peered":"2023-09-10T00:40:16.688911+0000","last_clean":"2023-09-10T00:40:16.688911+0000","last_became_active":"2023-09-10T00:39:17.532547+0000","last_became_peered":"2023-09-10T00:39:17.532547+0000","last_unstale":"2023-09-10T00:40:16.688911+0000","last_undegraded":"2023-09-10T00:40:16.688911+0000","last_fullsized":"2023-09-10T00:40:16.688911+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:33:01.685743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681772+0000","last_change":"2023-09-10T00:38:39.770787+0000","last_active":"2023-09-10T00:40:16.681772+0000","last_peered":"2023-09-10T00:40:16.681772+0000","last_clean":"2023-09-10T00:40:16.681772+0000","last_became_active":"2023-09-10T00:38:39.770497+0000","last_became_peered":"2023-09-10T00:38:39.770497+0000","last_unstale":"2023-09-10T00:40:16.681772+0000","last_undegraded":"2023-09-10T00:40:16.681772+0000","last_fullsized":"2023-09-10T00:40:16.681772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:43:11.066766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682433+0000","last_change":"2023-09-10T00:38:39.768008+0000","last_active":"2023-09-10T00:40:16.682433+0000","last_peered":"2023-09-10T00:40:16.682433+0000","last_clean":"2023-09-10T00:40:16.682433+0000","last_became_active":"2023-09-10T00:38:39.767079+0000","last_became_peered":"2023-09-10T00:38:39.767079+0000","last_unstale":"2023-09-10T00:40:16.682433+0000","last_undegraded":"2023-09-10T00:40:16.682433+0000","last_fullsized":"2023-09-10T00:40:16.682433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:34:00.689881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685567+0000","last_change":"2023-09-10T00:38:39.775326+0000","last_active":"2023-09-10T00:40:16.685567+0000","last_peered":"2023-09-10T00:40:16.685567+0000","last_clean":"2023-09-10T00:40:16.685567+0000","last_became_active":"2023-09-10T00:38:39.775007+0000","last_became_peered":"2023-09-10T00:38:39.775007+0000","last_unstale":"2023-09-10T00:40:16.685567+0000","last_undegraded":"2023-09-10T00:40:16.685567+0000","last_fullsized":"2023-09-10T00:40:16.685567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:59:27.711722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681440+0000","last_change":"2023-09-10T00:38:39.764974+0000","last_active":"2023-09-10T00:40:16.681440+0000","last_peered":"2023-09-10T00:40:16.681440+0000","last_clean":"2023-09-10T00:40:16.681440+0000","last_became_active":"2023-09-10T00:38:39.764368+0000","last_became_peered":"2023-09-10T00:38:39.764368+0000","last_unstale":"2023-09-10T00:40:16.681440+0000","last_undegraded":"2023-09-10T00:40:16.681440+0000","last_fullsized":"2023-09-10T00:40:16.681440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:40:28.046021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686233+0000","last_change":"2023-09-10T00:38:43.793589+0000","last_active":"2023-09-10T00:40:16.686233+0000","last_peered":"2023-09-10T00:40:16.686233+0000","last_clean":"2023-09-10T00:40:16.686233+0000","last_became_active":"2023-09-10T00:38:43.793166+0000","last_became_peered":"2023-09-10T00:38:43.793166+0000","last_unstale":"2023-09-10T00:40:16.686233+0000","last_undegraded":"2023-09-10T00:40:16.686233+0000","last_fullsized":"2023-09-10T00:40:16.686233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:51:47.128925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":119,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:24.649354+0000","last_change":"2023-09-10T00:38:38.759909+0000","last_active":"2023-09-10T00:40:24.649354+0000","last_peered":"2023-09-10T00:40:24.649354+0000","last_clean":"2023-09-10T00:40:24.649354+0000","last_became_active":"2023-09-10T00:38:38.759561+0000","last_became_peered":"2023-09-10T00:38:38.759561+0000","last_unstale":"2023-09-10T00:40:24.649354+0000","last_undegraded":"2023-09-10T00:40:24.649354+0000","last_fullsized":"2023-09-10T00:40:24.649354+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:56:18.009497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676177+0000","last_change":"2023-09-10T00:38:39.774916+0000","last_active":"2023-09-10T00:40:16.676177+0000","last_peered":"2023-09-10T00:40:16.676177+0000","last_clean":"2023-09-10T00:40:16.676177+0000","last_became_active":"2023-09-10T00:38:39.774584+0000","last_became_peered":"2023-09-10T00:38:39.774584+0000","last_unstale":"2023-09-10T00:40:16.676177+0000","last_undegraded":"2023-09-10T00:40:16.676177+0000","last_fullsized":"2023-09-10T00:40:16.676177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:36:33.125742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686686+0000","last_change":"2023-09-10T00:38:43.789421+0000","last_active":"2023-09-10T00:40:16.686686+0000","last_peered":"2023-09-10T00:40:16.686686+0000","last_clean":"2023-09-10T00:40:16.686686+0000","last_became_active":"2023-09-10T00:38:43.788569+0000","last_became_peered":"2023-09-10T00:38:43.788569+0000","last_unstale":"2023-09-10T00:40:16.686686+0000","last_undegraded":"2023-09-10T00:40:16.686686+0000","last_fullsized":"2023-09-10T00:40:16.686686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:07:38.529811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676261+0000","last_change":"2023-09-10T00:38:38.767381+0000","last_active":"2023-09-10T00:40:16.676261+0000","last_peered":"2023-09-10T00:40:16.676261+0000","last_clean":"2023-09-10T00:40:16.676261+0000","last_became_active":"2023-09-10T00:38:38.767082+0000","last_became_peered":"2023-09-10T00:38:38.767082+0000","last_unstale":"2023-09-10T00:40:16.676261+0000","last_undegraded":"2023-09-10T00:40:16.676261+0000","last_fullsized":"2023-09-10T00:40:16.676261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:13:32.533215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689263+0000","last_change":"2023-09-10T00:38:39.772645+0000","last_active":"2023-09-10T00:40:16.689263+0000","last_peered":"2023-09-10T00:40:16.689263+0000","last_clean":"2023-09-10T00:40:16.689263+0000","last_became_active":"2023-09-10T00:38:39.771554+0000","last_became_peered":"2023-09-10T00:38:39.771554+0000","last_unstale":"2023-09-10T00:40:16.689263+0000","last_undegraded":"2023-09-10T00:40:16.689263+0000","last_fullsized":"2023-09-10T00:40:16.689263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:25:06.237309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686805+0000","last_change":"2023-09-10T00:38:43.795764+0000","last_active":"2023-09-10T00:40:16.686805+0000","last_peered":"2023-09-10T00:40:16.686805+0000","last_clean":"2023-09-10T00:40:16.686805+0000","last_became_active":"2023-09-10T00:38:43.795112+0000","last_became_peered":"2023-09-10T00:38:43.795112+0000","last_unstale":"2023-09-10T00:40:16.686805+0000","last_undegraded":"2023-09-10T00:40:16.686805+0000","last_fullsized":"2023-09-10T00:40:16.686805+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:18:55.460752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.699271+0000","last_change":"2023-09-10T00:40:17.698716+0000","last_active":"2023-09-10T00:40:17.699271+0000","last_peered":"2023-09-10T00:40:17.699271+0000","last_clean":"2023-09-10T00:40:17.699271+0000","last_became_active":"2023-09-10T00:40:17.687287+0000","last_became_peered":"2023-09-10T00:40:17.687287+0000","last_unstale":"2023-09-10T00:40:17.699271+0000","last_undegraded":"2023-09-10T00:40:17.699271+0000","last_fullsized":"2023-09-10T00:40:17.699271+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:07:29.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681429+0000","last_change":"2023-09-10T00:38:39.767790+0000","last_active":"2023-09-10T00:40:16.681429+0000","last_peered":"2023-09-10T00:40:16.681429+0000","last_clean":"2023-09-10T00:40:16.681429+0000","last_became_active":"2023-09-10T00:38:39.767513+0000","last_became_peered":"2023-09-10T00:38:39.767513+0000","last_unstale":"2023-09-10T00:40:16.681429+0000","last_undegraded":"2023-09-10T00:40:16.681429+0000","last_fullsized":"2023-09-10T00:40:16.681429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:05:48.223951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686757+0000","last_change":"2023-09-10T00:38:43.789089+0000","last_active":"2023-09-10T00:40:16.686757+0000","last_peered":"2023-09-10T00:40:16.686757+0000","last_clean":"2023-09-10T00:40:16.686757+0000","last_became_active":"2023-09-10T00:38:43.788434+0000","last_became_peered":"2023-09-10T00:38:43.788434+0000","last_unstale":"2023-09-10T00:40:16.686757+0000","last_undegraded":"2023-09-10T00:40:16.686757+0000","last_fullsized":"2023-09-10T00:40:16.686757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:55:00.236088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685274+0000","last_change":"2023-09-10T00:38:38.769478+0000","last_active":"2023-09-10T00:40:16.685274+0000","last_peered":"2023-09-10T00:40:16.685274+0000","last_clean":"2023-09-10T00:40:16.685274+0000","last_became_active":"2023-09-10T00:38:38.769168+0000","last_became_peered":"2023-09-10T00:38:38.769168+0000","last_unstale":"2023-09-10T00:40:16.685274+0000","last_undegraded":"2023-09-10T00:40:16.685274+0000","last_fullsized":"2023-09-10T00:40:16.685274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:03:48.133561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675591+0000","last_change":"2023-09-10T00:38:39.771843+0000","last_active":"2023-09-10T00:40:16.675591+0000","last_peered":"2023-09-10T00:40:16.675591+0000","last_clean":"2023-09-10T00:40:16.675591+0000","last_became_active":"2023-09-10T00:38:39.771510+0000","last_became_peered":"2023-09-10T00:38:39.771510+0000","last_unstale":"2023-09-10T00:40:16.675591+0000","last_undegraded":"2023-09-10T00:40:16.675591+0000","last_fullsized":"2023-09-10T00:40:16.675591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:17:39.996490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.680942+0000","last_change":"2023-09-10T00:38:43.786853+0000","last_active":"2023-09-10T00:40:16.680942+0000","last_peered":"2023-09-10T00:40:16.680942+0000","last_clean":"2023-09-10T00:40:16.680942+0000","last_became_active":"2023-09-10T00:38:43.786386+0000","last_became_peered":"2023-09-10T00:38:43.786386+0000","last_unstale":"2023-09-10T00:40:16.680942+0000","last_undegraded":"2023-09-10T00:40:16.680942+0000","last_fullsized":"2023-09-10T00:40:16.680942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:23:53.724653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:22.600360+0000","last_change":"2023-09-10T00:38:38.769004+0000","last_active":"2023-09-10T00:40:22.600360+0000","last_peered":"2023-09-10T00:40:22.600360+0000","last_clean":"2023-09-10T00:40:22.600360+0000","last_became_active":"2023-09-10T00:38:38.768690+0000","last_became_peered":"2023-09-10T00:38:38.768690+0000","last_unstale":"2023-09-10T00:40:22.600360+0000","last_undegraded":"2023-09-10T00:40:22.600360+0000","last_fullsized":"2023-09-10T00:40:22.600360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:04:42.531092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1636,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684636+0000","last_change":"2023-09-10T00:38:39.773174+0000","last_active":"2023-09-10T00:40:16.684636+0000","last_peered":"2023-09-10T00:40:16.684636+0000","last_clean":"2023-09-10T00:40:16.684636+0000","last_became_active":"2023-09-10T00:38:39.771192+0000","last_became_peered":"2023-09-10T00:38:39.771192+0000","last_unstale":"2023-09-10T00:40:16.684636+0000","last_undegraded":"2023-09-10T00:40:16.684636+0000","last_fullsized":"2023-09-10T00:40:16.684636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:16:25.034024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.687731+0000","last_change":"2023-09-10T00:40:17.687731+0000","last_active":"2023-09-10T00:40:17.687731+0000","last_peered":"2023-09-10T00:40:17.687731+0000","last_clean":"2023-09-10T00:40:17.687731+0000","last_became_active":"2023-09-10T00:40:17.687285+0000","last_became_peered":"2023-09-10T00:40:17.687285+0000","last_unstale":"2023-09-10T00:40:17.687731+0000","last_undegraded":"2023-09-10T00:40:17.687731+0000","last_fullsized":"2023-09-10T00:40:17.687731+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:16:29.995198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684888+0000","last_change":"2023-09-10T00:38:38.763416+0000","last_active":"2023-09-10T00:40:16.684888+0000","last_peered":"2023-09-10T00:40:16.684888+0000","last_clean":"2023-09-10T00:40:16.684888+0000","last_became_active":"2023-09-10T00:38:38.763054+0000","last_became_peered":"2023-09-10T00:38:38.763054+0000","last_unstale":"2023-09-10T00:40:16.684888+0000","last_undegraded":"2023-09-10T00:40:16.684888+0000","last_fullsized":"2023-09-10T00:40:16.684888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:47:00.179467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675938+0000","last_change":"2023-09-10T00:38:39.773284+0000","last_active":"2023-09-10T00:40:16.675938+0000","last_peered":"2023-09-10T00:40:16.675938+0000","last_clean":"2023-09-10T00:40:16.675938+0000","last_became_active":"2023-09-10T00:38:39.772969+0000","last_became_peered":"2023-09-10T00:38:39.772969+0000","last_unstale":"2023-09-10T00:40:16.675938+0000","last_undegraded":"2023-09-10T00:40:16.675938+0000","last_fullsized":"2023-09-10T00:40:16.675938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:46:45.013879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682333+0000","last_change":"2023-09-10T00:38:43.790810+0000","last_active":"2023-09-10T00:40:16.682333+0000","last_peered":"2023-09-10T00:40:16.682333+0000","last_clean":"2023-09-10T00:40:16.682333+0000","last_became_active":"2023-09-10T00:38:43.790429+0000","last_became_peered":"2023-09-10T00:38:43.790429+0000","last_unstale":"2023-09-10T00:40:16.682333+0000","last_undegraded":"2023-09-10T00:40:16.682333+0000","last_fullsized":"2023-09-10T00:40:16.682333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:30:56.072587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.673630+0000","last_change":"2023-09-10T00:38:38.754326+0000","last_active":"2023-09-10T00:40:16.673630+0000","last_peered":"2023-09-10T00:40:16.673630+0000","last_clean":"2023-09-10T00:40:16.673630+0000","last_became_active":"2023-09-10T00:38:38.754006+0000","last_became_peered":"2023-09-10T00:38:38.754006+0000","last_unstale":"2023-09-10T00:40:16.673630+0000","last_undegraded":"2023-09-10T00:40:16.673630+0000","last_fullsized":"2023-09-10T00:40:16.673630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:09:26.983606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674527+0000","last_change":"2023-09-10T00:38:39.769699+0000","last_active":"2023-09-10T00:40:16.674527+0000","last_peered":"2023-09-10T00:40:16.674527+0000","last_clean":"2023-09-10T00:40:16.674527+0000","last_became_active":"2023-09-10T00:38:39.769362+0000","last_became_peered":"2023-09-10T00:38:39.769362+0000","last_unstale":"2023-09-10T00:40:16.674527+0000","last_undegraded":"2023-09-10T00:40:16.674527+0000","last_fullsized":"2023-09-10T00:40:16.674527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:30:20.938437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689355+0000","last_change":"2023-09-10T00:38:43.785669+0000","last_active":"2023-09-10T00:40:16.689355+0000","last_peered":"2023-09-10T00:40:16.689355+0000","last_clean":"2023-09-10T00:40:16.689355+0000","last_became_active":"2023-09-10T00:38:43.785255+0000","last_became_peered":"2023-09-10T00:38:43.785255+0000","last_unstale":"2023-09-10T00:40:16.689355+0000","last_undegraded":"2023-09-10T00:40:16.689355+0000","last_fullsized":"2023-09-10T00:40:16.689355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:52:57.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675650+0000","last_change":"2023-09-10T00:38:38.767997+0000","last_active":"2023-09-10T00:40:16.675650+0000","last_peered":"2023-09-10T00:40:16.675650+0000","last_clean":"2023-09-10T00:40:16.675650+0000","last_became_active":"2023-09-10T00:38:38.767715+0000","last_became_peered":"2023-09-10T00:38:38.767715+0000","last_unstale":"2023-09-10T00:40:16.675650+0000","last_undegraded":"2023-09-10T00:40:16.675650+0000","last_fullsized":"2023-09-10T00:40:16.675650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:11:17.047280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685405+0000","last_change":"2023-09-10T00:38:39.778004+0000","last_active":"2023-09-10T00:40:16.685405+0000","last_peered":"2023-09-10T00:40:16.685405+0000","last_clean":"2023-09-10T00:40:16.685405+0000","last_became_active":"2023-09-10T00:38:39.777632+0000","last_became_peered":"2023-09-10T00:38:39.777632+0000","last_unstale":"2023-09-10T00:40:16.685405+0000","last_undegraded":"2023-09-10T00:40:16.685405+0000","last_fullsized":"2023-09-10T00:40:16.685405+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:46:40.804385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.682227+0000","last_change":"2023-09-10T00:40:17.682227+0000","last_active":"2023-09-10T00:40:17.682227+0000","last_peered":"2023-09-10T00:40:17.682227+0000","last_clean":"2023-09-10T00:40:17.682227+0000","last_became_active":"2023-09-10T00:40:17.681849+0000","last_became_peered":"2023-09-10T00:40:17.681849+0000","last_unstale":"2023-09-10T00:40:17.682227+0000","last_undegraded":"2023-09-10T00:40:17.682227+0000","last_fullsized":"2023-09-10T00:40:17.682227+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:59.568162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":194,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:24.602900+0000","last_change":"2023-09-10T00:38:38.767399+0000","last_active":"2023-09-10T00:40:24.602900+0000","last_peered":"2023-09-10T00:40:24.602900+0000","last_clean":"2023-09-10T00:40:24.602900+0000","last_became_active":"2023-09-10T00:38:38.767176+0000","last_became_peered":"2023-09-10T00:38:38.767176+0000","last_unstale":"2023-09-10T00:40:24.602900+0000","last_undegraded":"2023-09-10T00:40:24.602900+0000","last_fullsized":"2023-09-10T00:40:24.602900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:17:24.481985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681626+0000","last_change":"2023-09-10T00:38:39.766013+0000","last_active":"2023-09-10T00:40:16.681626+0000","last_peered":"2023-09-10T00:40:16.681626+0000","last_clean":"2023-09-10T00:40:16.681626+0000","last_became_active":"2023-09-10T00:38:39.765235+0000","last_became_peered":"2023-09-10T00:38:39.765235+0000","last_unstale":"2023-09-10T00:40:16.681626+0000","last_undegraded":"2023-09-10T00:40:16.681626+0000","last_fullsized":"2023-09-10T00:40:16.681626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:01:48.874845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685291+0000","last_change":"2023-09-10T00:38:43.790361+0000","last_active":"2023-09-10T00:40:16.685291+0000","last_peered":"2023-09-10T00:40:16.685291+0000","last_clean":"2023-09-10T00:40:16.685291+0000","last_became_active":"2023-09-10T00:38:43.789966+0000","last_became_peered":"2023-09-10T00:38:43.789966+0000","last_unstale":"2023-09-10T00:40:16.685291+0000","last_undegraded":"2023-09-10T00:40:16.685291+0000","last_fullsized":"2023-09-10T00:40:16.685291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:46:37.978258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682397+0000","last_change":"2023-09-10T00:38:38.771670+0000","last_active":"2023-09-10T00:40:16.682397+0000","last_peered":"2023-09-10T00:40:16.682397+0000","last_clean":"2023-09-10T00:40:16.682397+0000","last_became_active":"2023-09-10T00:38:38.771339+0000","last_became_peered":"2023-09-10T00:38:38.771339+0000","last_unstale":"2023-09-10T00:40:16.682397+0000","last_undegraded":"2023-09-10T00:40:16.682397+0000","last_fullsized":"2023-09-10T00:40:16.682397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:22:32.658544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689361+0000","last_change":"2023-09-10T00:38:39.772857+0000","last_active":"2023-09-10T00:40:16.689361+0000","last_peered":"2023-09-10T00:40:16.689361+0000","last_clean":"2023-09-10T00:40:16.689361+0000","last_became_active":"2023-09-10T00:38:39.771990+0000","last_became_peered":"2023-09-10T00:38:39.771990+0000","last_unstale":"2023-09-10T00:40:16.689361+0000","last_undegraded":"2023-09-10T00:40:16.689361+0000","last_fullsized":"2023-09-10T00:40:16.689361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:16:15.767748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686883+0000","last_change":"2023-09-10T00:38:43.795482+0000","last_active":"2023-09-10T00:40:16.686883+0000","last_peered":"2023-09-10T00:40:16.686883+0000","last_clean":"2023-09-10T00:40:16.686883+0000","last_became_active":"2023-09-10T00:38:43.795048+0000","last_became_peered":"2023-09-10T00:38:43.795048+0000","last_unstale":"2023-09-10T00:40:16.686883+0000","last_undegraded":"2023-09-10T00:40:16.686883+0000","last_fullsized":"2023-09-10T00:40:16.686883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:13:18.715274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675457+0000","last_change":"2023-09-10T00:38:38.771008+0000","last_active":"2023-09-10T00:40:16.675457+0000","last_peered":"2023-09-10T00:40:16.675457+0000","last_clean":"2023-09-10T00:40:16.675457+0000","last_became_active":"2023-09-10T00:38:38.770714+0000","last_became_peered":"2023-09-10T00:38:38.770714+0000","last_unstale":"2023-09-10T00:40:16.675457+0000","last_undegraded":"2023-09-10T00:40:16.675457+0000","last_fullsized":"2023-09-10T00:40:16.675457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:04:30.148141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682324+0000","last_change":"2023-09-10T00:39:17.533500+0000","last_active":"2023-09-10T00:40:16.682324+0000","last_peered":"2023-09-10T00:40:16.682324+0000","last_clean":"2023-09-10T00:40:16.682324+0000","last_became_active":"2023-09-10T00:39:17.533175+0000","last_became_peered":"2023-09-10T00:39:17.533175+0000","last_unstale":"2023-09-10T00:40:16.682324+0000","last_undegraded":"2023-09-10T00:40:16.682324+0000","last_fullsized":"2023-09-10T00:40:16.682324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:13:42.121699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684740+0000","last_change":"2023-09-10T00:39:17.536684+0000","last_active":"2023-09-10T00:40:16.684740+0000","last_peered":"2023-09-10T00:40:16.684740+0000","last_clean":"2023-09-10T00:40:16.684740+0000","last_became_active":"2023-09-10T00:39:17.536170+0000","last_became_peered":"2023-09-10T00:39:17.536170+0000","last_unstale":"2023-09-10T00:40:16.684740+0000","last_undegraded":"2023-09-10T00:40:16.684740+0000","last_fullsized":"2023-09-10T00:40:16.684740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:47:16.264619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":259,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:23.744670+0000","last_change":"2023-09-10T00:40:17.741148+0000","last_active":"2023-09-10T00:40:23.744670+0000","last_peered":"2023-09-10T00:40:23.744670+0000","last_clean":"2023-09-10T00:40:23.744670+0000","last_became_active":"2023-09-10T00:40:17.731033+0000","last_became_peered":"2023-09-10T00:40:17.731033+0000","last_unstale":"2023-09-10T00:40:23.744670+0000","last_undegraded":"2023-09-10T00:40:23.744670+0000","last_fullsized":"2023-09-10T00:40:23.744670+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:20:42.067739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63651,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":176,"num_read_kb":150,"num_write":40,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":58848,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684860+0000","last_change":"2023-09-10T00:38:39.767507+0000","last_active":"2023-09-10T00:40:16.684860+0000","last_peered":"2023-09-10T00:40:16.684860+0000","last_clean":"2023-09-10T00:40:16.684860+0000","last_became_active":"2023-09-10T00:38:39.767223+0000","last_became_peered":"2023-09-10T00:38:39.767223+0000","last_unstale":"2023-09-10T00:40:16.684860+0000","last_undegraded":"2023-09-10T00:40:16.684860+0000","last_fullsized":"2023-09-10T00:40:16.684860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:46:30.901818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676490+0000","last_change":"2023-09-10T00:38:43.785396+0000","last_active":"2023-09-10T00:40:16.676490+0000","last_peered":"2023-09-10T00:40:16.676490+0000","last_clean":"2023-09-10T00:40:16.676490+0000","last_became_active":"2023-09-10T00:38:43.785038+0000","last_became_peered":"2023-09-10T00:38:43.785038+0000","last_unstale":"2023-09-10T00:40:16.676490+0000","last_undegraded":"2023-09-10T00:40:16.676490+0000","last_fullsized":"2023-09-10T00:40:16.676490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:30:31.629703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.680630+0000","last_change":"2023-09-10T00:38:38.766742+0000","last_active":"2023-09-10T00:40:16.680630+0000","last_peered":"2023-09-10T00:40:16.680630+0000","last_clean":"2023-09-10T00:40:16.680630+0000","last_became_active":"2023-09-10T00:38:38.766220+0000","last_became_peered":"2023-09-10T00:38:38.766220+0000","last_unstale":"2023-09-10T00:40:16.680630+0000","last_undegraded":"2023-09-10T00:40:16.680630+0000","last_fullsized":"2023-09-10T00:40:16.680630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:41:22.030448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.673843+0000","last_change":"2023-09-10T00:38:39.766022+0000","last_active":"2023-09-10T00:40:16.673843+0000","last_peered":"2023-09-10T00:40:16.673843+0000","last_clean":"2023-09-10T00:40:16.673843+0000","last_became_active":"2023-09-10T00:38:39.765697+0000","last_became_peered":"2023-09-10T00:38:39.765697+0000","last_unstale":"2023-09-10T00:40:16.673843+0000","last_undegraded":"2023-09-10T00:40:16.673843+0000","last_fullsized":"2023-09-10T00:40:16.673843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:19:39.394299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681669+0000","last_change":"2023-09-10T00:38:43.795492+0000","last_active":"2023-09-10T00:40:16.681669+0000","last_peered":"2023-09-10T00:40:16.681669+0000","last_clean":"2023-09-10T00:40:16.681669+0000","last_became_active":"2023-09-10T00:38:43.794968+0000","last_became_peered":"2023-09-10T00:38:43.794968+0000","last_unstale":"2023-09-10T00:40:16.681669+0000","last_undegraded":"2023-09-10T00:40:16.681669+0000","last_fullsized":"2023-09-10T00:40:16.681669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:40:43.558870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676085+0000","last_change":"2023-09-10T00:38:38.767666+0000","last_active":"2023-09-10T00:40:16.676085+0000","last_peered":"2023-09-10T00:40:16.676085+0000","last_clean":"2023-09-10T00:40:16.676085+0000","last_became_active":"2023-09-10T00:38:38.767333+0000","last_became_peered":"2023-09-10T00:38:38.767333+0000","last_unstale":"2023-09-10T00:40:16.676085+0000","last_undegraded":"2023-09-10T00:40:16.676085+0000","last_fullsized":"2023-09-10T00:40:16.676085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:07:53.363196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682945+0000","last_change":"2023-09-10T00:38:39.766036+0000","last_active":"2023-09-10T00:40:16.682945+0000","last_peered":"2023-09-10T00:40:16.682945+0000","last_clean":"2023-09-10T00:40:16.682945+0000","last_became_active":"2023-09-10T00:38:39.765683+0000","last_became_peered":"2023-09-10T00:38:39.765683+0000","last_unstale":"2023-09-10T00:40:16.682945+0000","last_undegraded":"2023-09-10T00:40:16.682945+0000","last_fullsized":"2023-09-10T00:40:16.682945+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:58:04.429960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686204+0000","last_change":"2023-09-10T00:38:43.787870+0000","last_active":"2023-09-10T00:40:16.686204+0000","last_peered":"2023-09-10T00:40:16.686204+0000","last_clean":"2023-09-10T00:40:16.686204+0000","last_became_active":"2023-09-10T00:38:43.785915+0000","last_became_peered":"2023-09-10T00:38:43.785915+0000","last_unstale":"2023-09-10T00:40:16.686204+0000","last_undegraded":"2023-09-10T00:40:16.686204+0000","last_fullsized":"2023-09-10T00:40:16.686204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:34:08.623967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'8","reported_seq":116,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:21.597796+0000","last_change":"2023-09-10T00:38:38.756925+0000","last_active":"2023-09-10T00:40:21.597796+0000","last_peered":"2023-09-10T00:40:21.597796+0000","last_clean":"2023-09-10T00:40:21.597796+0000","last_became_active":"2023-09-10T00:38:38.756610+0000","last_became_peered":"2023-09-10T00:38:38.756610+0000","last_unstale":"2023-09-10T00:40:21.597796+0000","last_undegraded":"2023-09-10T00:40:21.597796+0000","last_fullsized":"2023-09-10T00:40:21.597796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:48:15.385230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1602,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682147+0000","last_change":"2023-09-10T00:38:39.766750+0000","last_active":"2023-09-10T00:40:16.682147+0000","last_peered":"2023-09-10T00:40:16.682147+0000","last_clean":"2023-09-10T00:40:16.682147+0000","last_became_active":"2023-09-10T00:38:39.766458+0000","last_became_peered":"2023-09-10T00:38:39.766458+0000","last_unstale":"2023-09-10T00:40:16.682147+0000","last_undegraded":"2023-09-10T00:40:16.682147+0000","last_fullsized":"2023-09-10T00:40:16.682147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:59:26.691487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674310+0000","last_change":"2023-09-10T00:38:43.788814+0000","last_active":"2023-09-10T00:40:16.674310+0000","last_peered":"2023-09-10T00:40:16.674310+0000","last_clean":"2023-09-10T00:40:16.674310+0000","last_became_active":"2023-09-10T00:38:43.788446+0000","last_became_peered":"2023-09-10T00:38:43.788446+0000","last_unstale":"2023-09-10T00:40:16.674310+0000","last_undegraded":"2023-09-10T00:40:16.674310+0000","last_fullsized":"2023-09-10T00:40:16.674310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:38:53.308160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684332+0000","last_change":"2023-09-10T00:38:38.766020+0000","last_active":"2023-09-10T00:40:16.684332+0000","last_peered":"2023-09-10T00:40:16.684332+0000","last_clean":"2023-09-10T00:40:16.684332+0000","last_became_active":"2023-09-10T00:38:38.765719+0000","last_became_peered":"2023-09-10T00:38:38.765719+0000","last_unstale":"2023-09-10T00:40:16.684332+0000","last_undegraded":"2023-09-10T00:40:16.684332+0000","last_fullsized":"2023-09-10T00:40:16.684332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:25:30.488014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676849+0000","last_change":"2023-09-10T00:38:39.771091+0000","last_active":"2023-09-10T00:40:16.676849+0000","last_peered":"2023-09-10T00:40:16.676849+0000","last_clean":"2023-09-10T00:40:16.676849+0000","last_became_active":"2023-09-10T00:38:39.770745+0000","last_became_peered":"2023-09-10T00:38:39.770745+0000","last_unstale":"2023-09-10T00:40:16.676849+0000","last_undegraded":"2023-09-10T00:40:16.676849+0000","last_fullsized":"2023-09-10T00:40:16.676849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:51:17.978466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676732+0000","last_change":"2023-09-10T00:38:43.793757+0000","last_active":"2023-09-10T00:40:16.676732+0000","last_peered":"2023-09-10T00:40:16.676732+0000","last_clean":"2023-09-10T00:40:16.676732+0000","last_became_active":"2023-09-10T00:38:43.793336+0000","last_became_peered":"2023-09-10T00:38:43.793336+0000","last_unstale":"2023-09-10T00:40:16.676732+0000","last_undegraded":"2023-09-10T00:40:16.676732+0000","last_fullsized":"2023-09-10T00:40:16.676732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:22:36.544950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686902+0000","last_change":"2023-09-10T00:38:38.769373+0000","last_active":"2023-09-10T00:40:16.686902+0000","last_peered":"2023-09-10T00:40:16.686902+0000","last_clean":"2023-09-10T00:40:16.686902+0000","last_became_active":"2023-09-10T00:38:38.769054+0000","last_became_peered":"2023-09-10T00:38:38.769054+0000","last_unstale":"2023-09-10T00:40:16.686902+0000","last_undegraded":"2023-09-10T00:40:16.686902+0000","last_fullsized":"2023-09-10T00:40:16.686902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:12:27.982405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681789+0000","last_change":"2023-09-10T00:39:17.537427+0000","last_active":"2023-09-10T00:40:16.681789+0000","last_peered":"2023-09-10T00:40:16.681789+0000","last_clean":"2023-09-10T00:40:16.681789+0000","last_became_active":"2023-09-10T00:39:17.537023+0000","last_became_peered":"2023-09-10T00:39:17.537023+0000","last_unstale":"2023-09-10T00:40:16.681789+0000","last_undegraded":"2023-09-10T00:40:16.681789+0000","last_fullsized":"2023-09-10T00:40:16.681789+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:54:43.720224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684523+0000","last_change":"2023-09-10T00:38:43.838689+0000","last_active":"2023-09-10T00:40:16.684523+0000","last_peered":"2023-09-10T00:40:16.684523+0000","last_clean":"2023-09-10T00:40:16.684523+0000","last_became_active":"2023-09-10T00:38:43.838244+0000","last_became_peered":"2023-09-10T00:38:43.838244+0000","last_unstale":"2023-09-10T00:40:16.684523+0000","last_undegraded":"2023-09-10T00:40:16.684523+0000","last_fullsized":"2023-09-10T00:40:16.684523+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:54:00.033719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685917+0000","last_change":"2023-09-10T00:38:38.765140+0000","last_active":"2023-09-10T00:40:16.685917+0000","last_peered":"2023-09-10T00:40:16.685917+0000","last_clean":"2023-09-10T00:40:16.685917+0000","last_became_active":"2023-09-10T00:38:38.764730+0000","last_became_peered":"2023-09-10T00:38:38.764730+0000","last_unstale":"2023-09-10T00:40:16.685917+0000","last_undegraded":"2023-09-10T00:40:16.685917+0000","last_fullsized":"2023-09-10T00:40:16.685917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:49:41.129763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685440+0000","last_change":"2023-09-10T00:38:39.773260+0000","last_active":"2023-09-10T00:40:16.685440+0000","last_peered":"2023-09-10T00:40:16.685440+0000","last_clean":"2023-09-10T00:40:16.685440+0000","last_became_active":"2023-09-10T00:38:39.772947+0000","last_became_peered":"2023-09-10T00:38:39.772947+0000","last_unstale":"2023-09-10T00:40:16.685440+0000","last_undegraded":"2023-09-10T00:40:16.685440+0000","last_fullsized":"2023-09-10T00:40:16.685440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:08:10.791610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674639+0000","last_change":"2023-09-10T00:38:43.787882+0000","last_active":"2023-09-10T00:40:16.674639+0000","last_peered":"2023-09-10T00:40:16.674639+0000","last_clean":"2023-09-10T00:40:16.674639+0000","last_became_active":"2023-09-10T00:38:43.787257+0000","last_became_peered":"2023-09-10T00:38:43.787257+0000","last_unstale":"2023-09-10T00:40:16.674639+0000","last_undegraded":"2023-09-10T00:40:16.674639+0000","last_fullsized":"2023-09-10T00:40:16.674639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:11:19.171881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:21.645218+0000","last_change":"2023-09-10T00:38:38.760014+0000","last_active":"2023-09-10T00:40:21.645218+0000","last_peered":"2023-09-10T00:40:21.645218+0000","last_clean":"2023-09-10T00:40:21.645218+0000","last_became_active":"2023-09-10T00:38:38.759637+0000","last_became_peered":"2023-09-10T00:38:38.759637+0000","last_unstale":"2023-09-10T00:40:21.645218+0000","last_undegraded":"2023-09-10T00:40:21.645218+0000","last_fullsized":"2023-09-10T00:40:21.645218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:15.704929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1071,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684955+0000","last_change":"2023-09-10T00:38:39.765800+0000","last_active":"2023-09-10T00:40:16.684955+0000","last_peered":"2023-09-10T00:40:16.684955+0000","last_clean":"2023-09-10T00:40:16.684955+0000","last_became_active":"2023-09-10T00:38:39.765468+0000","last_became_peered":"2023-09-10T00:38:39.765468+0000","last_unstale":"2023-09-10T00:40:16.684955+0000","last_undegraded":"2023-09-10T00:40:16.684955+0000","last_fullsized":"2023-09-10T00:40:16.684955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:57:56.429365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674169+0000","last_change":"2023-09-10T00:38:43.838285+0000","last_active":"2023-09-10T00:40:16.674169+0000","last_peered":"2023-09-10T00:40:16.674169+0000","last_clean":"2023-09-10T00:40:16.674169+0000","last_became_active":"2023-09-10T00:38:43.837821+0000","last_became_peered":"2023-09-10T00:38:43.837821+0000","last_unstale":"2023-09-10T00:40:16.674169+0000","last_undegraded":"2023-09-10T00:40:16.674169+0000","last_fullsized":"2023-09-10T00:40:16.674169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:01:03.354281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675958+0000","last_change":"2023-09-10T00:39:17.536361+0000","last_active":"2023-09-10T00:40:16.675958+0000","last_peered":"2023-09-10T00:40:16.675958+0000","last_clean":"2023-09-10T00:40:16.675958+0000","last_became_active":"2023-09-10T00:39:17.535930+0000","last_became_peered":"2023-09-10T00:39:17.535930+0000","last_unstale":"2023-09-10T00:40:16.675958+0000","last_undegraded":"2023-09-10T00:40:16.675958+0000","last_fullsized":"2023-09-10T00:40:16.675958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:53.576115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":114,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:24.601702+0000","last_change":"2023-09-10T00:38:38.758745+0000","last_active":"2023-09-10T00:40:24.601702+0000","last_peered":"2023-09-10T00:40:24.601702+0000","last_clean":"2023-09-10T00:40:24.601702+0000","last_became_active":"2023-09-10T00:38:38.758427+0000","last_became_peered":"2023-09-10T00:38:38.758427+0000","last_unstale":"2023-09-10T00:40:24.601702+0000","last_undegraded":"2023-09-10T00:40:24.601702+0000","last_fullsized":"2023-09-10T00:40:24.601702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:02:49.238175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.683243+0000","last_change":"2023-09-10T00:38:43.784058+0000","last_active":"2023-09-10T00:40:16.683243+0000","last_peered":"2023-09-10T00:40:16.683243+0000","last_clean":"2023-09-10T00:40:16.683243+0000","last_became_active":"2023-09-10T00:38:43.783571+0000","last_became_peered":"2023-09-10T00:38:43.783571+0000","last_unstale":"2023-09-10T00:40:16.683243+0000","last_undegraded":"2023-09-10T00:40:16.683243+0000","last_fullsized":"2023-09-10T00:40:16.683243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:53:50.992393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676566+0000","last_change":"2023-09-10T00:38:39.775659+0000","last_active":"2023-09-10T00:40:16.676566+0000","last_peered":"2023-09-10T00:40:16.676566+0000","last_clean":"2023-09-10T00:40:16.676566+0000","last_became_active":"2023-09-10T00:38:39.775151+0000","last_became_peered":"2023-09-10T00:38:39.775151+0000","last_unstale":"2023-09-10T00:40:16.676566+0000","last_undegraded":"2023-09-10T00:40:16.676566+0000","last_fullsized":"2023-09-10T00:40:16.676566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:07.415418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686900+0000","last_change":"2023-09-10T00:38:38.755009+0000","last_active":"2023-09-10T00:40:16.686900+0000","last_peered":"2023-09-10T00:40:16.686900+0000","last_clean":"2023-09-10T00:40:16.686900+0000","last_became_active":"2023-09-10T00:38:38.754699+0000","last_became_peered":"2023-09-10T00:38:38.754699+0000","last_unstale":"2023-09-10T00:40:16.686900+0000","last_undegraded":"2023-09-10T00:40:16.686900+0000","last_fullsized":"2023-09-10T00:40:16.686900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:12:02.075256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":324,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681263+0000","last_change":"2023-09-10T00:37:38.459695+0000","last_active":"2023-09-10T00:40:16.681263+0000","last_peered":"2023-09-10T00:40:16.681263+0000","last_clean":"2023-09-10T00:40:16.681263+0000","last_became_active":"2023-09-10T00:37:38.459220+0000","last_became_peered":"2023-09-10T00:37:38.459220+0000","last_unstale":"2023-09-10T00:40:16.681263+0000","last_undegraded":"2023-09-10T00:40:16.681263+0000","last_fullsized":"2023-09-10T00:40:16.681263+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:34:44.602334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:34:44.602334+0000","last_clean_scrub_stamp":"2023-09-10T00:34:44.602334+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:21:32.648725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675426+0000","last_change":"2023-09-10T00:38:43.839001+0000","last_active":"2023-09-10T00:40:16.675426+0000","last_peered":"2023-09-10T00:40:16.675426+0000","last_clean":"2023-09-10T00:40:16.675426+0000","last_became_active":"2023-09-10T00:38:43.838497+0000","last_became_peered":"2023-09-10T00:38:43.838497+0000","last_unstale":"2023-09-10T00:40:16.675426+0000","last_undegraded":"2023-09-10T00:40:16.675426+0000","last_fullsized":"2023-09-10T00:40:16.675426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:57:23.349632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.673565+0000","last_change":"2023-09-10T00:38:39.760150+0000","last_active":"2023-09-10T00:40:16.673565+0000","last_peered":"2023-09-10T00:40:16.673565+0000","last_clean":"2023-09-10T00:40:16.673565+0000","last_became_active":"2023-09-10T00:38:39.759811+0000","last_became_peered":"2023-09-10T00:38:39.759811+0000","last_unstale":"2023-09-10T00:40:16.673565+0000","last_undegraded":"2023-09-10T00:40:16.673565+0000","last_fullsized":"2023-09-10T00:40:16.673565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:21:19.359057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.701421+0000","last_change":"2023-09-10T00:40:17.700994+0000","last_active":"2023-09-10T00:40:17.701421+0000","last_peered":"2023-09-10T00:40:17.701421+0000","last_clean":"2023-09-10T00:40:17.701421+0000","last_became_active":"2023-09-10T00:40:17.690073+0000","last_became_peered":"2023-09-10T00:40:17.690073+0000","last_unstale":"2023-09-10T00:40:17.701421+0000","last_undegraded":"2023-09-10T00:40:17.701421+0000","last_fullsized":"2023-09-10T00:40:17.701421+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:43:32.478865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681570+0000","last_change":"2023-09-10T00:38:43.786272+0000","last_active":"2023-09-10T00:40:16.681570+0000","last_peered":"2023-09-10T00:40:16.681570+0000","last_clean":"2023-09-10T00:40:16.681570+0000","last_became_active":"2023-09-10T00:38:43.785934+0000","last_became_peered":"2023-09-10T00:38:43.785934+0000","last_unstale":"2023-09-10T00:40:16.681570+0000","last_undegraded":"2023-09-10T00:40:16.681570+0000","last_fullsized":"2023-09-10T00:40:16.681570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:31:49.721133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684970+0000","last_change":"2023-09-10T00:38:38.763263+0000","last_active":"2023-09-10T00:40:16.684970+0000","last_peered":"2023-09-10T00:40:16.684970+0000","last_clean":"2023-09-10T00:40:16.684970+0000","last_became_active":"2023-09-10T00:38:38.762911+0000","last_became_peered":"2023-09-10T00:38:38.762911+0000","last_unstale":"2023-09-10T00:40:16.684970+0000","last_undegraded":"2023-09-10T00:40:16.684970+0000","last_fullsized":"2023-09-10T00:40:16.684970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:19:10.292311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674491+0000","last_change":"2023-09-10T00:38:39.759171+0000","last_active":"2023-09-10T00:40:16.674491+0000","last_peered":"2023-09-10T00:40:16.674491+0000","last_clean":"2023-09-10T00:40:16.674491+0000","last_became_active":"2023-09-10T00:38:39.758843+0000","last_became_peered":"2023-09-10T00:38:39.758843+0000","last_unstale":"2023-09-10T00:40:16.674491+0000","last_undegraded":"2023-09-10T00:40:16.674491+0000","last_fullsized":"2023-09-10T00:40:16.674491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:50:36.064937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674438+0000","last_change":"2023-09-10T00:38:43.837998+0000","last_active":"2023-09-10T00:40:16.674438+0000","last_peered":"2023-09-10T00:40:16.674438+0000","last_clean":"2023-09-10T00:40:16.674438+0000","last_became_active":"2023-09-10T00:38:43.837556+0000","last_became_peered":"2023-09-10T00:38:43.837556+0000","last_unstale":"2023-09-10T00:40:16.674438+0000","last_undegraded":"2023-09-10T00:40:16.674438+0000","last_fullsized":"2023-09-10T00:40:16.674438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:18:40.488279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:18.399592+0000","last_change":"2023-09-10T00:38:38.759819+0000","last_active":"2023-09-10T00:40:18.399592+0000","last_peered":"2023-09-10T00:40:18.399592+0000","last_clean":"2023-09-10T00:40:18.399592+0000","last_became_active":"2023-09-10T00:38:38.759490+0000","last_became_peered":"2023-09-10T00:38:38.759490+0000","last_unstale":"2023-09-10T00:40:18.399592+0000","last_undegraded":"2023-09-10T00:40:18.399592+0000","last_fullsized":"2023-09-10T00:40:18.399592+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:39:01.206817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675634+0000","last_change":"2023-09-10T00:38:39.769669+0000","last_active":"2023-09-10T00:40:16.675634+0000","last_peered":"2023-09-10T00:40:16.675634+0000","last_clean":"2023-09-10T00:40:16.675634+0000","last_became_active":"2023-09-10T00:38:39.769402+0000","last_became_peered":"2023-09-10T00:38:39.769402+0000","last_unstale":"2023-09-10T00:40:16.675634+0000","last_undegraded":"2023-09-10T00:40:16.675634+0000","last_fullsized":"2023-09-10T00:40:16.675634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:00:50.215103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685982+0000","last_change":"2023-09-10T00:38:43.787518+0000","last_active":"2023-09-10T00:40:16.685982+0000","last_peered":"2023-09-10T00:40:16.685982+0000","last_clean":"2023-09-10T00:40:16.685982+0000","last_became_active":"2023-09-10T00:38:43.786410+0000","last_became_peered":"2023-09-10T00:38:43.786410+0000","last_unstale":"2023-09-10T00:40:16.685982+0000","last_undegraded":"2023-09-10T00:40:16.685982+0000","last_fullsized":"2023-09-10T00:40:16.685982+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:13:37.903875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.687579+0000","last_change":"2023-09-10T00:38:38.760992+0000","last_active":"2023-09-10T00:40:16.687579+0000","last_peered":"2023-09-10T00:40:16.687579+0000","last_clean":"2023-09-10T00:40:16.687579+0000","last_became_active":"2023-09-10T00:38:38.760635+0000","last_became_peered":"2023-09-10T00:38:38.760635+0000","last_unstale":"2023-09-10T00:40:16.687579+0000","last_undegraded":"2023-09-10T00:40:16.687579+0000","last_fullsized":"2023-09-10T00:40:16.687579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:29:36.070477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689409+0000","last_change":"2023-09-10T00:38:39.773771+0000","last_active":"2023-09-10T00:40:16.689409+0000","last_peered":"2023-09-10T00:40:16.689409+0000","last_clean":"2023-09-10T00:40:16.689409+0000","last_became_active":"2023-09-10T00:38:39.773296+0000","last_became_peered":"2023-09-10T00:38:39.773296+0000","last_unstale":"2023-09-10T00:40:16.689409+0000","last_undegraded":"2023-09-10T00:40:16.689409+0000","last_fullsized":"2023-09-10T00:40:16.689409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:13:01.194358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.687211+0000","last_change":"2023-09-10T00:39:17.536853+0000","last_active":"2023-09-10T00:40:16.687211+0000","last_peered":"2023-09-10T00:40:16.687211+0000","last_clean":"2023-09-10T00:40:16.687211+0000","last_became_active":"2023-09-10T00:39:17.536287+0000","last_became_peered":"2023-09-10T00:39:17.536287+0000","last_unstale":"2023-09-10T00:40:16.687211+0000","last_undegraded":"2023-09-10T00:40:16.687211+0000","last_fullsized":"2023-09-10T00:40:16.687211+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:50:35.294055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681867+0000","last_change":"2023-09-10T00:38:38.764954+0000","last_active":"2023-09-10T00:40:16.681867+0000","last_peered":"2023-09-10T00:40:16.681867+0000","last_clean":"2023-09-10T00:40:16.681867+0000","last_became_active":"2023-09-10T00:38:38.764589+0000","last_became_peered":"2023-09-10T00:38:38.764589+0000","last_unstale":"2023-09-10T00:40:16.681867+0000","last_undegraded":"2023-09-10T00:40:16.681867+0000","last_fullsized":"2023-09-10T00:40:16.681867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:40:19.559285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684134+0000","last_change":"2023-09-10T00:38:39.766141+0000","last_active":"2023-09-10T00:40:16.684134+0000","last_peered":"2023-09-10T00:40:16.684134+0000","last_clean":"2023-09-10T00:40:16.684134+0000","last_became_active":"2023-09-10T00:38:39.765832+0000","last_became_peered":"2023-09-10T00:38:39.765832+0000","last_unstale":"2023-09-10T00:40:16.684134+0000","last_undegraded":"2023-09-10T00:40:16.684134+0000","last_fullsized":"2023-09-10T00:40:16.684134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:08:45.149973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686352+0000","last_change":"2023-09-10T00:38:43.785111+0000","last_active":"2023-09-10T00:40:16.686352+0000","last_peered":"2023-09-10T00:40:16.686352+0000","last_clean":"2023-09-10T00:40:16.686352+0000","last_became_active":"2023-09-10T00:38:43.784542+0000","last_became_peered":"2023-09-10T00:38:43.784542+0000","last_unstale":"2023-09-10T00:40:16.686352+0000","last_undegraded":"2023-09-10T00:40:16.686352+0000","last_fullsized":"2023-09-10T00:40:16.686352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:34:51.611608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674667+0000","last_change":"2023-09-10T00:38:38.759984+0000","last_active":"2023-09-10T00:40:16.674667+0000","last_peered":"2023-09-10T00:40:16.674667+0000","last_clean":"2023-09-10T00:40:16.674667+0000","last_became_active":"2023-09-10T00:38:38.759660+0000","last_became_peered":"2023-09-10T00:38:38.759660+0000","last_unstale":"2023-09-10T00:40:16.674667+0000","last_undegraded":"2023-09-10T00:40:16.674667+0000","last_fullsized":"2023-09-10T00:40:16.674667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:02:11.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.683063+0000","last_change":"2023-09-10T00:38:39.770765+0000","last_active":"2023-09-10T00:40:16.683063+0000","last_peered":"2023-09-10T00:40:16.683063+0000","last_clean":"2023-09-10T00:40:16.683063+0000","last_became_active":"2023-09-10T00:38:39.770483+0000","last_became_peered":"2023-09-10T00:38:39.770483+0000","last_unstale":"2023-09-10T00:40:16.683063+0000","last_undegraded":"2023-09-10T00:40:16.683063+0000","last_fullsized":"2023-09-10T00:40:16.683063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:57:28.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.687049+0000","last_change":"2023-09-10T00:38:43.790945+0000","last_active":"2023-09-10T00:40:16.687049+0000","last_peered":"2023-09-10T00:40:16.687049+0000","last_clean":"2023-09-10T00:40:16.687049+0000","last_became_active":"2023-09-10T00:38:43.790467+0000","last_became_peered":"2023-09-10T00:38:43.790467+0000","last_unstale":"2023-09-10T00:40:16.687049+0000","last_undegraded":"2023-09-10T00:40:16.687049+0000","last_fullsized":"2023-09-10T00:40:16.687049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:44:07.041302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.676240+0000","last_change":"2023-09-10T00:38:38.765912+0000","last_active":"2023-09-10T00:40:17.676240+0000","last_peered":"2023-09-10T00:40:17.676240+0000","last_clean":"2023-09-10T00:40:17.676240+0000","last_became_active":"2023-09-10T00:38:38.765586+0000","last_became_peered":"2023-09-10T00:38:38.765586+0000","last_unstale":"2023-09-10T00:40:17.676240+0000","last_undegraded":"2023-09-10T00:40:17.676240+0000","last_fullsized":"2023-09-10T00:40:17.676240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:25:05.974158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675205+0000","last_change":"2023-09-10T00:38:39.771462+0000","last_active":"2023-09-10T00:40:16.675205+0000","last_peered":"2023-09-10T00:40:16.675205+0000","last_clean":"2023-09-10T00:40:16.675205+0000","last_became_active":"2023-09-10T00:38:39.771153+0000","last_became_peered":"2023-09-10T00:38:39.771153+0000","last_unstale":"2023-09-10T00:40:16.675205+0000","last_undegraded":"2023-09-10T00:40:16.675205+0000","last_fullsized":"2023-09-10T00:40:16.675205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:03:35.312186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.687296+0000","last_change":"2023-09-10T00:38:43.838446+0000","last_active":"2023-09-10T00:40:16.687296+0000","last_peered":"2023-09-10T00:40:16.687296+0000","last_clean":"2023-09-10T00:40:16.687296+0000","last_became_active":"2023-09-10T00:38:43.838119+0000","last_became_peered":"2023-09-10T00:38:43.838119+0000","last_unstale":"2023-09-10T00:40:16.687296+0000","last_undegraded":"2023-09-10T00:40:16.687296+0000","last_fullsized":"2023-09-10T00:40:16.687296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:12:20.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.734911+0000","last_change":"2023-09-10T00:40:17.734911+0000","last_active":"2023-09-10T00:40:17.734911+0000","last_peered":"2023-09-10T00:40:17.734911+0000","last_clean":"2023-09-10T00:40:17.734911+0000","last_became_active":"2023-09-10T00:40:17.730465+0000","last_became_peered":"2023-09-10T00:40:17.730465+0000","last_unstale":"2023-09-10T00:40:17.734911+0000","last_undegraded":"2023-09-10T00:40:17.734911+0000","last_fullsized":"2023-09-10T00:40:17.734911+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:42:17.592032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684224+0000","last_change":"2023-09-10T00:38:39.771995+0000","last_active":"2023-09-10T00:40:16.684224+0000","last_peered":"2023-09-10T00:40:16.684224+0000","last_clean":"2023-09-10T00:40:16.684224+0000","last_became_active":"2023-09-10T00:38:39.771693+0000","last_became_peered":"2023-09-10T00:38:39.771693+0000","last_unstale":"2023-09-10T00:40:16.684224+0000","last_undegraded":"2023-09-10T00:40:16.684224+0000","last_fullsized":"2023-09-10T00:40:16.684224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:41:45.621060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.680916+0000","last_change":"2023-09-10T00:38:43.789159+0000","last_active":"2023-09-10T00:40:16.680916+0000","last_peered":"2023-09-10T00:40:16.680916+0000","last_clean":"2023-09-10T00:40:16.680916+0000","last_became_active":"2023-09-10T00:38:43.788675+0000","last_became_peered":"2023-09-10T00:38:43.788675+0000","last_unstale":"2023-09-10T00:40:16.680916+0000","last_undegraded":"2023-09-10T00:40:16.680916+0000","last_fullsized":"2023-09-10T00:40:16.680916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:30:39.545561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'6","reported_seq":110,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:21.647139+0000","last_change":"2023-09-10T00:38:38.767245+0000","last_active":"2023-09-10T00:40:21.647139+0000","last_peered":"2023-09-10T00:40:21.647139+0000","last_clean":"2023-09-10T00:40:21.647139+0000","last_became_active":"2023-09-10T00:38:38.766952+0000","last_became_peered":"2023-09-10T00:38:38.766952+0000","last_unstale":"2023-09-10T00:40:21.647139+0000","last_undegraded":"2023-09-10T00:40:21.647139+0000","last_fullsized":"2023-09-10T00:40:21.647139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:57:23.309838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1653,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.683345+0000","last_change":"2023-09-10T00:38:39.772199+0000","last_active":"2023-09-10T00:40:16.683345+0000","last_peered":"2023-09-10T00:40:16.683345+0000","last_clean":"2023-09-10T00:40:16.683345+0000","last_became_active":"2023-09-10T00:38:39.771917+0000","last_became_peered":"2023-09-10T00:38:39.771917+0000","last_unstale":"2023-09-10T00:40:16.683345+0000","last_undegraded":"2023-09-10T00:40:16.683345+0000","last_fullsized":"2023-09-10T00:40:16.683345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:00:52.824581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.688815+0000","last_change":"2023-09-10T00:38:43.797107+0000","last_active":"2023-09-10T00:40:16.688815+0000","last_peered":"2023-09-10T00:40:16.688815+0000","last_clean":"2023-09-10T00:40:16.688815+0000","last_became_active":"2023-09-10T00:38:43.796606+0000","last_became_peered":"2023-09-10T00:38:43.796606+0000","last_unstale":"2023-09-10T00:40:16.688815+0000","last_undegraded":"2023-09-10T00:40:16.688815+0000","last_fullsized":"2023-09-10T00:40:16.688815+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:17:00.444238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676254+0000","last_change":"2023-09-10T00:38:38.768045+0000","last_active":"2023-09-10T00:40:16.676254+0000","last_peered":"2023-09-10T00:40:16.676254+0000","last_clean":"2023-09-10T00:40:16.676254+0000","last_became_active":"2023-09-10T00:38:38.767344+0000","last_became_peered":"2023-09-10T00:38:38.767344+0000","last_unstale":"2023-09-10T00:40:16.676254+0000","last_undegraded":"2023-09-10T00:40:16.676254+0000","last_fullsized":"2023-09-10T00:40:16.676254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:51:08.765412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.683838+0000","last_change":"2023-09-10T00:39:17.536408+0000","last_active":"2023-09-10T00:40:16.683838+0000","last_peered":"2023-09-10T00:40:16.683838+0000","last_clean":"2023-09-10T00:40:16.683838+0000","last_became_active":"2023-09-10T00:39:17.535980+0000","last_became_peered":"2023-09-10T00:39:17.535980+0000","last_unstale":"2023-09-10T00:40:16.683838+0000","last_undegraded":"2023-09-10T00:40:16.683838+0000","last_fullsized":"2023-09-10T00:40:16.683838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:08:10.517535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675080+0000","last_change":"2023-09-10T00:38:43.784928+0000","last_active":"2023-09-10T00:40:16.675080+0000","last_peered":"2023-09-10T00:40:16.675080+0000","last_clean":"2023-09-10T00:40:16.675080+0000","last_became_active":"2023-09-10T00:38:43.784479+0000","last_became_peered":"2023-09-10T00:38:43.784479+0000","last_unstale":"2023-09-10T00:40:16.675080+0000","last_undegraded":"2023-09-10T00:40:16.675080+0000","last_fullsized":"2023-09-10T00:40:16.675080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:14:45.019988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674218+0000","last_change":"2023-09-10T00:38:38.770730+0000","last_active":"2023-09-10T00:40:16.674218+0000","last_peered":"2023-09-10T00:40:16.674218+0000","last_clean":"2023-09-10T00:40:16.674218+0000","last_became_active":"2023-09-10T00:38:38.770344+0000","last_became_peered":"2023-09-10T00:38:38.770344+0000","last_unstale":"2023-09-10T00:40:16.674218+0000","last_undegraded":"2023-09-10T00:40:16.674218+0000","last_fullsized":"2023-09-10T00:40:16.674218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:49:22.881956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682628+0000","last_change":"2023-09-10T00:38:39.766149+0000","last_active":"2023-09-10T00:40:16.682628+0000","last_peered":"2023-09-10T00:40:16.682628+0000","last_clean":"2023-09-10T00:40:16.682628+0000","last_became_active":"2023-09-10T00:38:39.765773+0000","last_became_peered":"2023-09-10T00:38:39.765773+0000","last_unstale":"2023-09-10T00:40:16.682628+0000","last_undegraded":"2023-09-10T00:40:16.682628+0000","last_fullsized":"2023-09-10T00:40:16.682628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:00:37.854168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682825+0000","last_change":"2023-09-10T00:38:39.765197+0000","last_active":"2023-09-10T00:40:16.682825+0000","last_peered":"2023-09-10T00:40:16.682825+0000","last_clean":"2023-09-10T00:40:16.682825+0000","last_became_active":"2023-09-10T00:38:39.764891+0000","last_became_peered":"2023-09-10T00:38:39.764891+0000","last_unstale":"2023-09-10T00:40:16.682825+0000","last_undegraded":"2023-09-10T00:40:16.682825+0000","last_fullsized":"2023-09-10T00:40:16.682825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:47:27.429355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686326+0000","last_change":"2023-09-10T00:38:39.764722+0000","last_active":"2023-09-10T00:40:16.686326+0000","last_peered":"2023-09-10T00:40:16.686326+0000","last_clean":"2023-09-10T00:40:16.686326+0000","last_became_active":"2023-09-10T00:38:39.764376+0000","last_became_peered":"2023-09-10T00:38:39.764376+0000","last_unstale":"2023-09-10T00:40:16.686326+0000","last_undegraded":"2023-09-10T00:40:16.686326+0000","last_fullsized":"2023-09-10T00:40:16.686326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:27:36.708573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.679566+0000","last_change":"2023-09-10T00:38:39.771997+0000","last_active":"2023-09-10T00:40:16.679566+0000","last_peered":"2023-09-10T00:40:16.679566+0000","last_clean":"2023-09-10T00:40:16.679566+0000","last_became_active":"2023-09-10T00:38:39.771588+0000","last_became_peered":"2023-09-10T00:38:39.771588+0000","last_unstale":"2023-09-10T00:40:16.679566+0000","last_undegraded":"2023-09-10T00:40:16.679566+0000","last_fullsized":"2023-09-10T00:40:16.679566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:20:07.171333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689101+0000","last_change":"2023-09-10T00:38:39.773393+0000","last_active":"2023-09-10T00:40:16.689101+0000","last_peered":"2023-09-10T00:40:16.689101+0000","last_clean":"2023-09-10T00:40:16.689101+0000","last_became_active":"2023-09-10T00:38:39.772972+0000","last_became_peered":"2023-09-10T00:38:39.772972+0000","last_unstale":"2023-09-10T00:40:16.689101+0000","last_undegraded":"2023-09-10T00:40:16.689101+0000","last_fullsized":"2023-09-10T00:40:16.689101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:27:29.916970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674401+0000","last_change":"2023-09-10T00:38:39.767871+0000","last_active":"2023-09-10T00:40:16.674401+0000","last_peered":"2023-09-10T00:40:16.674401+0000","last_clean":"2023-09-10T00:40:16.674401+0000","last_became_active":"2023-09-10T00:38:39.767563+0000","last_became_peered":"2023-09-10T00:38:39.767563+0000","last_unstale":"2023-09-10T00:40:16.674401+0000","last_undegraded":"2023-09-10T00:40:16.674401+0000","last_fullsized":"2023-09-10T00:40:16.674401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:22:30.400441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682225+0000","last_change":"2023-09-10T00:38:39.764664+0000","last_active":"2023-09-10T00:40:16.682225+0000","last_peered":"2023-09-10T00:40:16.682225+0000","last_clean":"2023-09-10T00:40:16.682225+0000","last_became_active":"2023-09-10T00:38:39.764364+0000","last_became_peered":"2023-09-10T00:38:39.764364+0000","last_unstale":"2023-09-10T00:40:16.682225+0000","last_undegraded":"2023-09-10T00:40:16.682225+0000","last_fullsized":"2023-09-10T00:40:16.682225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:41.656472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685830+0000","last_change":"2023-09-10T00:38:39.770315+0000","last_active":"2023-09-10T00:40:16.685830+0000","last_peered":"2023-09-10T00:40:16.685830+0000","last_clean":"2023-09-10T00:40:16.685830+0000","last_became_active":"2023-09-10T00:38:39.769977+0000","last_became_peered":"2023-09-10T00:38:39.769977+0000","last_unstale":"2023-09-10T00:40:16.685830+0000","last_undegraded":"2023-09-10T00:40:16.685830+0000","last_fullsized":"2023-09-10T00:40:16.685830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:30:13.380487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675805+0000","last_change":"2023-09-10T00:38:39.768734+0000","last_active":"2023-09-10T00:40:16.675805+0000","last_peered":"2023-09-10T00:40:16.675805+0000","last_clean":"2023-09-10T00:40:16.675805+0000","last_became_active":"2023-09-10T00:38:39.768383+0000","last_became_peered":"2023-09-10T00:38:39.768383+0000","last_unstale":"2023-09-10T00:40:16.675805+0000","last_undegraded":"2023-09-10T00:40:16.675805+0000","last_fullsized":"2023-09-10T00:40:16.675805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:09.974447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684233+0000","last_change":"2023-09-10T00:39:17.732561+0000","last_active":"2023-09-10T00:40:16.684233+0000","last_peered":"2023-09-10T00:40:16.684233+0000","last_clean":"2023-09-10T00:40:16.684233+0000","last_became_active":"2023-09-10T00:39:17.732138+0000","last_became_peered":"2023-09-10T00:39:17.732138+0000","last_unstale":"2023-09-10T00:40:16.684233+0000","last_undegraded":"2023-09-10T00:40:16.684233+0000","last_fullsized":"2023-09-10T00:40:16.684233+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:11:58.866881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674781+0000","last_change":"2023-09-10T00:38:39.773839+0000","last_active":"2023-09-10T00:40:16.674781+0000","last_peered":"2023-09-10T00:40:16.674781+0000","last_clean":"2023-09-10T00:40:16.674781+0000","last_became_active":"2023-09-10T00:38:39.773521+0000","last_became_peered":"2023-09-10T00:38:39.773521+0000","last_unstale":"2023-09-10T00:40:16.674781+0000","last_undegraded":"2023-09-10T00:40:16.674781+0000","last_fullsized":"2023-09-10T00:40:16.674781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:15:49.033374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682785+0000","last_change":"2023-09-10T00:38:39.775721+0000","last_active":"2023-09-10T00:40:16.682785+0000","last_peered":"2023-09-10T00:40:16.682785+0000","last_clean":"2023-09-10T00:40:16.682785+0000","last_became_active":"2023-09-10T00:38:39.775424+0000","last_became_peered":"2023-09-10T00:38:39.775424+0000","last_unstale":"2023-09-10T00:40:16.682785+0000","last_undegraded":"2023-09-10T00:40:16.682785+0000","last_fullsized":"2023-09-10T00:40:16.682785+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:56:26.079332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674146+0000","last_change":"2023-09-10T00:38:39.768767+0000","last_active":"2023-09-10T00:40:16.674146+0000","last_peered":"2023-09-10T00:40:16.674146+0000","last_clean":"2023-09-10T00:40:16.674146+0000","last_became_active":"2023-09-10T00:38:39.768452+0000","last_became_peered":"2023-09-10T00:38:39.768452+0000","last_unstale":"2023-09-10T00:40:16.674146+0000","last_undegraded":"2023-09-10T00:40:16.674146+0000","last_fullsized":"2023-09-10T00:40:16.674146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:31:53.578574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681759+0000","last_change":"2023-09-10T00:38:39.761998+0000","last_active":"2023-09-10T00:40:16.681759+0000","last_peered":"2023-09-10T00:40:16.681759+0000","last_clean":"2023-09-10T00:40:16.681759+0000","last_became_active":"2023-09-10T00:38:39.761672+0000","last_became_peered":"2023-09-10T00:38:39.761672+0000","last_unstale":"2023-09-10T00:40:16.681759+0000","last_undegraded":"2023-09-10T00:40:16.681759+0000","last_fullsized":"2023-09-10T00:40:16.681759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:03:24.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73067,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":880,"num_read_kb":526,"num_write":255,"num_write_kb":153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":58848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":210561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449120,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":510852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.3150000000000004,"5min":4.3150000000000004,"15min":4.3150000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.54700000000000004,"15min":0.50800000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":6.3470000000000004,"5min":6.3470000000000004,"15min":6.3470000000000004},"last":0.41199999999999998}]},{"osd":1,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.499,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81899999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.64100000000000001},"min":{"1min":0.435,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.60499999999999998}]},{"osd":2,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48699999999999999,"15min":0.48799999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.47999999999999998,"15min":0.47699999999999998},"min":{"1min":0.39100000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":4,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67100000000000004,"15min":0.66900000000000004},"min":{"1min":0.51700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.83999999999999997,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.65200000000000002}]},{"osd":5,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77600000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.434}]},{"osd":6,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.747,"15min":0.71999999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":6.4260000000000002,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.79900000000000004,"15min":0.76800000000000002},"min":{"1min":0.50900000000000001,"5min":0.498,"15min":0.498},"max":{"1min":6.4720000000000004,"5min":6.4720000000000004,"15min":6.4720000000000004},"last":0.67300000000000004}]},{"osd":7,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76500000000000001,"15min":0.77000000000000002},"min":{"1min":0.54500000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89900000000000002,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.755,"15min":0.75800000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90900000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.75600000000000001}]},{"osd":8,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71199999999999997}]},{"osd":9,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.81299999999999994,"15min":0.78900000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":4.4859999999999998,"5min":4.4859999999999998,"15min":4.4859999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.75600000000000001,"15min":0.748},"min":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.622}]},{"osd":10,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.57599999999999996,"5min":0.5,"15min":0.5},"max":{"1min":0.97899999999999998,"5min":1.486,"15min":1.486},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81499999999999995,"15min":0.81699999999999995},"min":{"1min":0.56699999999999995,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96799999999999997,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.78900000000000003}]}]},{"osd":10,"up_from":70,"seq":300647710757,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28708,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716444,"statfs":{"total":95995035648,"available":95965638656,"internally_reserved":0,"allocated":1732608,"data_stored":1161850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.502}]},{"osd":1,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51500000000000001,"15min":0.51900000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78200000000000003,"5min":0.873,"15min":0.873},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.36499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52900000000000003}]},{"osd":2,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.629,"15min":0.629},"min":{"1min":0.501,"5min":0.439,"15min":0.439},"max":{"1min":0.75800000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65400000000000003,"15min":0.64400000000000002},"min":{"1min":0.52400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61699999999999999}]},{"osd":3,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.61899999999999999,"15min":0.63700000000000001},"min":{"1min":0.39300000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70099999999999996,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.48899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.56899999999999995}]},{"osd":4,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.54600000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.624},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78100000000000003,"15min":0.78300000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.94299999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65400000000000003}]},{"osd":6,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.995,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.59999999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69399999999999995}]},{"osd":7,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71299999999999997}]},{"osd":8,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77700000000000002,"15min":0.77500000000000002},"min":{"1min":0.63900000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78600000000000003,"15min":0.78800000000000003},"min":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.70399999999999996}]},{"osd":9,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.86099999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55600000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.72799999999999998}]},{"osd":11,"last update":"Sun Sep 10 00:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80400000000000005,"15min":0.80200000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.71699999999999997}]}]},{"osd":9,"up_from":65,"seq":279172874282,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":514395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42499999999999999,"15min":0.44400000000000001},"min":{"1min":0.32300000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.504,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.42799999999999999,"15min":0.45600000000000002},"min":{"1min":0.28999999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.55500000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.59699999999999998,"15min":0.61299999999999999},"min":{"1min":0.43099999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81100000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70099999999999996,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59899999999999998}]},{"osd":2,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63600000000000001,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.74199999999999999,"5min":0.879,"15min":0.879},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77300000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.51200000000000001}]},{"osd":3,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.52900000000000003,"15min":0.56899999999999995},"min":{"1min":0.29899999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.74399999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.442,"5min":0.56899999999999995,"15min":0.61699999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72499999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42199999999999999}]},{"osd":4,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.60599999999999998,"15min":0.63200000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80500000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.66400000000000003,"15min":0.68799999999999994},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75900000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.621}]},{"osd":5,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72699999999999998,"15min":0.747},"min":{"1min":0.46000000000000002,"5min":0.435,"15min":0.435},"max":{"1min":3.3109999999999999,"5min":3.3109999999999999,"15min":3.3109999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.72899999999999998,"15min":0.76700000000000002},"min":{"1min":0.47399999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.78100000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.57399999999999995}]},{"osd":6,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.42499999999999999}]},{"osd":7,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.751,"15min":0.79500000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.79500000000000004,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.76800000000000002,"15min":0.81799999999999995},"min":{"1min":0.499,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.89200000000000002,"5min":1.071,"15min":1.071},"last":0.66300000000000003}]},{"osd":8,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73599999999999999,"15min":0.78600000000000003},"min":{"1min":0.47799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79000000000000004,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.79800000000000004,"15min":0.84799999999999998},"min":{"1min":0.53400000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88900000000000001,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.78600000000000003}]},{"osd":10,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.72899999999999998,"15min":0.749},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.77600000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.54000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.82799999999999996,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.76700000000000002}]},{"osd":11,"last update":"Sun Sep 10 00:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.69899999999999995}]}]},{"osd":8,"up_from":57,"seq":244813135918,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":510346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76100000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64400000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61499999999999999,"15min":0.59699999999999998},"min":{"1min":0.49299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.874,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.69899999999999995,"15min":0.63600000000000001},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93999999999999995,"5min":6.5759999999999996,"15min":6.5759999999999996},"last":0.78900000000000003}]},{"osd":2,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.31,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.49399999999999999,"15min":0.48499999999999999},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59799999999999998}]},{"osd":3,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.85199999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.871,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.82999999999999996}]},{"osd":4,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.61399999999999999,"15min":0.59199999999999997},"min":{"1min":0.48299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.63600000000000001,"15min":0.61699999999999999},"min":{"1min":0.53700000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.76500000000000001}]},{"osd":5,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90000000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.74099999999999999}]},{"osd":6,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79800000000000004,"15min":0.755},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.026,"5min":6.5940000000000003,"15min":6.5940000000000003},"last":0.93799999999999994}]},{"osd":7,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79000000000000004,"15min":0.71999999999999997},"min":{"1min":0.58799999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0920000000000001,"5min":6.5439999999999996,"15min":6.5439999999999996},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.69199999999999995,"15min":0.67400000000000004},"min":{"1min":0.60399999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":1.028}]},{"osd":9,"last update":"Sun Sep 10 00:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.75700000000000001,"15min":0.73799999999999999},"min":{"1min":0.65700000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.002,"5min":1.103,"15min":1.103},"last":1.002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.76100000000000001,"15min":0.73099999999999998},"min":{"1min":0.70099999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.038,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":1.0149999999999999}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.57199999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.73999999999999999,"15min":0.71299999999999997},"min":{"1min":0.64200000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.95599999999999996}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.92000000000000004}]}]},{"osd":7,"up_from":50,"seq":214748364848,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1650688,"data_stored":1107514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54100000000000004,"15min":0.53200000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.39800000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40500000000000003,"15min":0.40000000000000002},"min":{"1min":0.32800000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.52600000000000002,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.40999999999999998,"15min":0.40200000000000002},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63600000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.38700000000000001}]},{"osd":2,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.79800000000000004,"5min":1.091,"15min":1.091},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.51400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85999999999999999,"5min":1.069,"15min":1.069},"last":0.81899999999999995}]},{"osd":3,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.36699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76900000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.46999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54900000000000004}]},{"osd":4,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.38300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.66300000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.497,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.59999999999999998,"5min":1.056,"15min":1.056},"last":0.47899999999999998}]},{"osd":5,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":0.628},"min":{"1min":0.49399999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.83299999999999996,"5min":1.026,"15min":1.026},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.48099999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98399999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.74199999999999999}]},{"osd":6,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.48899999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.84999999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89800000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.42399999999999999}]},{"osd":8,"last update":"Sun Sep 10 00:39:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.65200000000000002,"15min":0.626},"min":{"1min":0.44400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.86399999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67800000000000005,"15min":0.65700000000000003},"min":{"1min":0.496,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.86499999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72599999999999998}]},{"osd":9,"last update":"Sun Sep 10 00:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.53500000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80000000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69299999999999995,"15min":0.69599999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.83599999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.70999999999999996}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65700000000000003,"15min":0.65200000000000002},"min":{"1min":0.46100000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83299999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66600000000000004,"15min":0.65700000000000003},"min":{"1min":0.54000000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.64800000000000002}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77300000000000002}]}]},{"osd":6,"up_from":45,"seq":193273528374,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":506286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41699999999999998,"15min":0.40100000000000002},"min":{"1min":0.34399999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.63700000000000001,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.40799999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59699999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.34499999999999997}]},{"osd":1,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59799999999999998,"15min":0.56399999999999995},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82999999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81499999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.73799999999999999}]},{"osd":2,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69499999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.627,"15min":0.61099999999999999},"min":{"1min":0.39500000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80100000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.50600000000000001}]},{"osd":3,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48699999999999999,"15min":0.47899999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66400000000000003,"5min":0.875,"15min":0.875},"last":0.37},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68799999999999994,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40400000000000003}]},{"osd":4,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.46800000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82199999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69099999999999995,"15min":0.68000000000000005},"min":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93400000000000005,"5min":1.024,"15min":1.024},"last":0.69699999999999995}]},{"osd":5,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68000000000000005,"15min":0.67600000000000005},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.879,"5min":1.048,"15min":1.048},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.69899999999999995,"15min":0.69399999999999995},"min":{"1min":0.48699999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.48699999999999999}]},{"osd":7,"last update":"Sun Sep 10 00:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.73199999999999998,"15min":0.72899999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94899999999999995,"5min":1.069,"15min":1.069},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72999999999999998,"15min":0.72299999999999998},"min":{"1min":0.47799999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.82799999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.78200000000000003}]},{"osd":8,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.79100000000000004,"15min":0.82499999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.876,"5min":1.27,"15min":1.27},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.79500000000000004,"15min":0.82099999999999995},"min":{"1min":0.55600000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.92800000000000005,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.625}]},{"osd":9,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.78200000000000003,"15min":0.84399999999999997},"min":{"1min":0.438,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94399999999999995,"5min":1.389,"15min":1.389},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.72499999999999998,"15min":0.77000000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86199999999999999,"5min":1.099,"15min":1.099},"last":0.63600000000000001}]},{"osd":10,"last update":"Sun Sep 10 00:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.82599999999999996,"15min":0.83899999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.96299999999999997,"5min":1.181,"15min":1.181},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.91300000000000003,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.79900000000000004}]},{"osd":11,"last update":"Sun Sep 10 00:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":566647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52600000000000002,"15min":0.50900000000000001},"min":{"1min":0.32100000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72199999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.53200000000000003,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53500000000000003}]},{"osd":2,"last update":"Sun Sep 10 00:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53900000000000003,"15min":0.52200000000000002},"min":{"1min":0.308,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.58799999999999997,"15min":0.54500000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84599999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.86799999999999999}]},{"osd":3,"last update":"Sun Sep 10 00:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.70399999999999996,"5min":0.879,"15min":0.879},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53100000000000003,"15min":0.52600000000000002},"min":{"1min":0.38,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69599999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56100000000000005}]},{"osd":4,"last update":"Sun Sep 10 00:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.42299999999999999},"min":{"1min":0.35099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.46000000000000002,"15min":0.437},"min":{"1min":0.33800000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58299999999999996}]},{"osd":5,"last update":"Sun Sep 10 00:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60399999999999998,"15min":0.57299999999999995},"min":{"1min":0.38700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89400000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.63500000000000001},"min":{"1min":0.45700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0409999999999999,"5min":1.044,"15min":1.044},"last":0.81399999999999995}]},{"osd":6,"last update":"Sun Sep 10 00:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63900000000000001,"15min":0.61299999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86199999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63900000000000001,"15min":0.61499999999999999},"min":{"1min":0.40400000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83699999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68100000000000005}]},{"osd":7,"last update":"Sun Sep 10 00:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61399999999999999,"15min":0.57399999999999995},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89200000000000002,"5min":0.999,"15min":0.999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.621,"15min":0.58299999999999996},"min":{"1min":0.54300000000000004,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.878,"5min":0.996,"15min":0.996},"last":0.72799999999999998}]},{"osd":8,"last update":"Sun Sep 10 00:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94499999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88200000000000001,"5min":1.014,"15min":1.014},"last":0.88400000000000001}]},{"osd":9,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.998,"5min":1.097,"15min":1.097},"last":0.75},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77300000000000002,"15min":0.78600000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.93000000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65900000000000003}]},{"osd":10,"last update":"Sun Sep 10 00:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.76000000000000001,"15min":0.77200000000000002},"min":{"1min":0.46700000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93600000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.86199999999999999,"15min":0.86799999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0109999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.90800000000000003}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.93400000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":507742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 10 00:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54400000000000004,"15min":0.5},"min":{"1min":0.36399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.629},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.52500000000000002},"min":{"1min":0.376,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.64700000000000002}]},{"osd":2,"last update":"Sun Sep 10 00:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.53800000000000003},"min":{"1min":0.39400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.56899999999999995},"min":{"1min":0.372,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83999999999999997,"5min":1.621,"15min":1.621},"last":0.78100000000000003}]},{"osd":3,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.375},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.39900000000000002}]},{"osd":4,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.46300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73699999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.374,"15min":0.374},"max":{"1min":0.83999999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59499999999999997}]},{"osd":5,"last update":"Sun Sep 10 00:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65700000000000003,"15min":0.61599999999999999},"min":{"1min":0.42799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.91000000000000003,"5min":1.075,"15min":1.075},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64700000000000002,"15min":0.63700000000000001},"min":{"1min":0.47799999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85399999999999998,"5min":1.123,"15min":1.123},"last":0.86499999999999999}]},{"osd":6,"last update":"Sun Sep 10 00:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53900000000000003,"15min":0.52000000000000002},"min":{"1min":0.34000000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.42899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.97999999999999998,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.52300000000000002}]},{"osd":7,"last update":"Sun Sep 10 00:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.72199999999999998}]},{"osd":8,"last update":"Sun Sep 10 00:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76900000000000002,"15min":0.77800000000000002},"min":{"1min":0.60499999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.022,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":0.74199999999999999},"min":{"1min":0.49099999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97199999999999998,"5min":1.151,"15min":1.151},"last":0.88500000000000001}]},{"osd":9,"last update":"Sun Sep 10 00:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73899999999999999,"15min":0.752},"min":{"1min":0.54800000000000004,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0549999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.69399999999999995,"15min":0.71299999999999997},"min":{"1min":0.44800000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90300000000000002,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.54400000000000004}]},{"osd":10,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.746,"15min":0.74299999999999999},"min":{"1min":0.57999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.53400000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.90600000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.749}]},{"osd":11,"last update":"Sun Sep 10 00:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75,"15min":0.749},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.67400000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.085,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.81699999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":508550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.51300000000000001},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52700000000000002,"15min":0.52500000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72699999999999998,"5min":0.753,"15min":0.753},"last":0.64000000000000001}]},{"osd":1,"last update":"Sun Sep 10 00:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.58599999999999997,"15min":0.53900000000000003},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.58999999999999997,"15min":0.55000000000000004},"min":{"1min":0.34200000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.75700000000000001}]},{"osd":3,"last update":"Sun Sep 10 00:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57599999999999996,"15min":0.54000000000000004},"min":{"1min":0.43099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.501}]},{"osd":4,"last update":"Sun Sep 10 00:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.77700000000000002}]},{"osd":5,"last update":"Sun Sep 10 00:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.499,"15min":0.44900000000000001},"min":{"1min":0.41399999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.55800000000000005,"15min":0.51200000000000001},"min":{"1min":0.46200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.71699999999999997}]},{"osd":6,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.45800000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.97599999999999998,"5min":1.018,"15min":1.018},"last":0.621},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67500000000000004,"15min":0.64700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96499999999999997,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.68799999999999994}]},{"osd":7,"last update":"Sun Sep 10 00:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.80700000000000005}]},{"osd":8,"last update":"Sun Sep 10 00:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.58999999999999997,"15min":0.55600000000000005},"min":{"1min":0.45400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.63300000000000001,"15min":0.60699999999999998},"min":{"1min":0.47899999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.53600000000000003}]},{"osd":9,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76200000000000001,"15min":0.74099999999999999},"min":{"1min":0.60999999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0429999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.73699999999999999,"15min":0.71499999999999997},"min":{"1min":0.56999999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98799999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.79000000000000004}]},{"osd":10,"last update":"Sun Sep 10 00:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.70799999999999996,"15min":0.68500000000000005},"min":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.745,"15min":0.71899999999999997},"min":{"1min":0.68700000000000006,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.875}]},{"osd":11,"last update":"Sun Sep 10 00:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.83199999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":505511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40899999999999997,"15min":0.40100000000000002},"min":{"1min":0.26700000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.92000000000000004,"5min":1.409,"15min":1.409},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.42799999999999999,"15min":0.42099999999999999},"min":{"1min":0.31900000000000001,"5min":0.255,"15min":0.255},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.51400000000000001}]},{"osd":1,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.6699999999999999,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58699999999999997,"15min":0.55600000000000005},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78500000000000003,"5min":1.581,"15min":1.581},"last":0.746}]},{"osd":2,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.60399999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.65400000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.7110000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.79200000000000004}]},{"osd":4,"last update":"Sun Sep 10 00:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.39300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80600000000000005,"5min":1.536,"15min":1.536},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.66500000000000004,"15min":0.68799999999999994},"min":{"1min":0.38300000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.748,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.68899999999999995}]},{"osd":5,"last update":"Sun Sep 10 00:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.73299999999999998},"min":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7809999999999999,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68100000000000005,"15min":0.67600000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.872}]},{"osd":6,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60099999999999998,"15min":0.57999999999999996},"min":{"1min":0.36499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.65600000000000003}]},{"osd":7,"last update":"Sun Sep 10 00:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.74399999999999999,"15min":0.745},"min":{"1min":0.47699999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90800000000000003,"5min":1.022,"15min":1.022},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73799999999999999,"15min":0.73599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.82499999999999996}]},{"osd":8,"last update":"Sun Sep 10 00:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.89400000000000002}]},{"osd":9,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71399999999999997,"15min":0.75800000000000001},"min":{"1min":0.36499999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.77900000000000003,"15min":0.82499999999999996},"min":{"1min":0.47599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95899999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.84399999999999997}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.82999999999999996,"15min":0.83999999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.90600000000000003}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.95299999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":570222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45000000000000001}]},{"osd":1,"last update":"Sun Sep 10 00:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.37,"15min":0.34399999999999997},"min":{"1min":0.29199999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.317,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.56000000000000005}]},{"osd":2,"last update":"Sun Sep 10 00:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80200000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58799999999999997,"15min":0.55500000000000005},"min":{"1min":0.497,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.68200000000000005}]},{"osd":3,"last update":"Sun Sep 10 00:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.39900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.60999999999999999,"15min":0.56200000000000006},"min":{"1min":0.35099999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.59799999999999998}]},{"osd":5,"last update":"Sun Sep 10 00:39:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85299999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.59499999999999997},"min":{"1min":0.49399999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63400000000000001}]},{"osd":6,"last update":"Sun Sep 10 00:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.61599999999999999,"15min":0.57999999999999996},"min":{"1min":0.50700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64100000000000001,"15min":0.63800000000000001},"min":{"1min":0.40799999999999997,"5min":0.375,"15min":0.375},"max":{"1min":0.91300000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72599999999999998}]},{"osd":7,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.39300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.88500000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.56499999999999995,"15min":0.52800000000000002},"min":{"1min":0.38700000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85199999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.877}]},{"osd":8,"last update":"Sun Sep 10 00:39:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.44800000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91200000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.60499999999999998,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97699999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.93100000000000005}]},{"osd":9,"last update":"Sun Sep 10 00:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.73199999999999998,"15min":0.70599999999999996},"min":{"1min":0.63200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.74299999999999999,"15min":0.73699999999999999},"min":{"1min":0.65600000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":1.0760000000000001}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74399999999999999,"15min":0.75},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94999999999999996,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.74099999999999999,"15min":0.73699999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.94499999999999995}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.97599999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1662976,"data_stored":1103379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.37,"5min":0.315,"15min":0.315},"max":{"1min":0.746,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.376,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74399999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52400000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.52500000000000002},"min":{"1min":0.51800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.46000000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.747}]},{"osd":2,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.36899999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.60999999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.373},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.45900000000000002,"15min":0.437},"min":{"1min":0.38600000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.39600000000000002}]},{"osd":3,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.371,"5min":0.31,"15min":0.31},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.504},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.48899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.91100000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46999999999999997}]},{"osd":4,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.59999999999999998,"15min":0.56000000000000005},"min":{"1min":0.53200000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92900000000000005,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58799999999999997,"15min":0.55900000000000005},"min":{"1min":0.47999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78100000000000003,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57599999999999996}]},{"osd":6,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63300000000000001,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.9100000000000001,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":0.442},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.68400000000000005,"15min":0.66100000000000003},"min":{"1min":0.53600000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Sun Sep 10 00:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.629},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.64100000000000001,"15min":0.622},"min":{"1min":0.53700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.98399999999999999,"5min":1.046,"15min":1.046},"last":0.61499999999999999}]},{"osd":8,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.60499999999999998,"15min":0.56499999999999995},"min":{"1min":0.56699999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.56799999999999995,"15min":0.53200000000000003},"min":{"1min":0.48899999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.68400000000000005}]},{"osd":9,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.75900000000000001,"15min":0.72299999999999998},"min":{"1min":0.59999999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.9590000000000001,"5min":2.9590000000000001,"15min":2.9590000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.77200000000000002,"15min":0.72399999999999998},"min":{"1min":0.63100000000000001,"5min":0.435,"15min":0.435},"max":{"1min":2.9129999999999998,"5min":2.9129999999999998,"15min":2.9129999999999998},"last":0.77900000000000003}]},{"osd":10,"last update":"Sun Sep 10 00:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.69699999999999995,"15min":0.68300000000000005},"min":{"1min":0.54600000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.02,"5min":1.026,"15min":1.026},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.58599999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.76600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-10T00:40:26.610 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-09-10T00:40:26.611 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-10T00:40:26.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.768+0000 7ff09634e700 1 -- 172.21.15.44:0/924152075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0901006d0 msgr2=0x7ff090100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:26.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.768+0000 7ff09634e700 1 --2- 172.21.15.44:0/924152075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0901006d0 0x7ff090100b30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff0840099c0 tx=0x7ff084009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.768+0000 7ff09634e700 1 -- 172.21.15.44:0/924152075 shutdown_connections 2023-09-10T00:40:26.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.768+0000 7ff09634e700 1 --2- 172.21.15.44:0/924152075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090101070 0x7ff090107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.768+0000 7ff09634e700 1 --2- 172.21.15.44:0/924152075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0900ff4e0 0x7ff0900ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.768+0000 7ff09634e700 1 --2- 172.21.15.44:0/924152075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0901006d0 0x7ff090100b30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.768+0000 7ff09634e700 1 -- 172.21.15.44:0/924152075 >> 172.21.15.44:0/924152075 conn(0x7ff0900faa30 msgr2=0x7ff0900fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.768+0000 7ff09634e700 1 -- 172.21.15.44:0/924152075 shutdown_connections 2023-09-10T00:40:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.768+0000 7ff09634e700 1 -- 172.21.15.44:0/924152075 wait complete. 2023-09-10T00:40:26.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff09634e700 1 Processor -- start 2023-09-10T00:40:26.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff09634e700 1 -- start start 2023-09-10T00:40:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff09634e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0900ff4e0 0x7ff090196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff09634e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0901006d0 0x7ff0901966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff09634e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff090101070 0x7ff09019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff09634e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09019dd10 con 0x7ff090101070 2023-09-10T00:40:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff09634e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09019de50 con 0x7ff0901006d0 2023-09-10T00:40:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff09634e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09019dfe0 con 0x7ff0900ff4e0 2023-09-10T00:40:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff0948eb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff090101070 0x7ff09019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff0948eb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff090101070 0x7ff09019d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48430/0 (socket says 172.21.15.44:48430) 2023-09-10T00:40:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff0948eb700 1 -- 172.21.15.44:0/4029480052 learned_addr learned my addr 172.21.15.44:0/4029480052 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff08ffff700 1 --2- 172.21.15.44:0/4029480052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0900ff4e0 0x7ff090196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff08f7fe700 1 --2- 172.21.15.44:0/4029480052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0901006d0 0x7ff0901966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff0948eb700 1 -- 172.21.15.44:0/4029480052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0900ff4e0 msgr2=0x7ff090196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:26.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff0948eb700 1 --2- 172.21.15.44:0/4029480052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0900ff4e0 0x7ff090196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff0948eb700 1 -- 172.21.15.44:0/4029480052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0901006d0 msgr2=0x7ff0901966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:26.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff0948eb700 1 --2- 172.21.15.44:0/4029480052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0901006d0 0x7ff0901966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:26.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff0948eb700 1 -- 172.21.15.44:0/4029480052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0840095f0 con 0x7ff090101070 2023-09-10T00:40:26.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.769+0000 7ff08f7fe700 1 --2- 172.21.15.44:0/4029480052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0901006d0 0x7ff0901966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:40:26.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.770+0000 7ff0948eb700 1 --2- 172.21.15.44:0/4029480052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff090101070 0x7ff09019d740 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff08800ef30 tx=0x7ff08800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:26.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.770+0000 7ff08d7fa700 1 -- 172.21.15.44:0/4029480052 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088018070 con 0x7ff090101070 2023-09-10T00:40:26.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.770+0000 7ff08d7fa700 1 -- 172.21.15.44:0/4029480052 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff08800f040 con 0x7ff090101070 2023-09-10T00:40:26.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.770+0000 7ff09634e700 1 -- 172.21.15.44:0/4029480052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09019e260 con 0x7ff090101070 2023-09-10T00:40:26.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.770+0000 7ff08d7fa700 1 -- 172.21.15.44:0/4029480052 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08800cc70 con 0x7ff090101070 2023-09-10T00:40:26.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.770+0000 7ff09634e700 1 -- 172.21.15.44:0/4029480052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09019e720 con 0x7ff090101070 2023-09-10T00:40:26.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.771+0000 7ff09634e700 1 -- 172.21.15.44:0/4029480052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0901075e0 con 0x7ff090101070 2023-09-10T00:40:26.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.772+0000 7ff08d7fa700 1 -- 172.21.15.44:0/4029480052 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ff088007500 con 0x7ff090101070 2023-09-10T00:40:26.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.772+0000 7ff08d7fa700 1 --2- 172.21.15.44:0/4029480052 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff0780a0690 0x7ff0780a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:26.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.773+0000 7ff08d7fa700 1 -- 172.21.15.44:0/4029480052 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7ff0880cbb60 con 0x7ff090101070 2023-09-10T00:40:26.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.773+0000 7ff08ffff700 1 --2- 172.21.15.44:0/4029480052 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff0780a0690 0x7ff0780a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:26.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.773+0000 7ff08ffff700 1 --2- 172.21.15.44:0/4029480052 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff0780a0690 0x7ff0780a2b50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff090100530 tx=0x7ff08000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:26.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.777+0000 7ff08d7fa700 1 -- 172.21.15.44:0/4029480052 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff088095300 con 0x7ff090101070 2023-09-10T00:40:26.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.975+0000 7ff09634e700 1 -- 172.21.15.44:0/4029480052 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff09002cf40 con 0x7ff0780a0690 2023-09-10T00:40:26.990 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.990+0000 7ff08d7fa700 1 -- 172.21.15.44:0/4029480052 <== mgr.14152 v2:172.21.15.44:6800/1488156576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427736 (secure 0 0 0) 0x7ff09002cf40 con 0x7ff0780a0690 2023-09-10T00:40:26.995 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:26.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.994+0000 7ff09634e700 1 -- 172.21.15.44:0/4029480052 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff0780a0690 msgr2=0x7ff0780a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:26.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.994+0000 7ff09634e700 1 --2- 172.21.15.44:0/4029480052 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff0780a0690 0x7ff0780a2b50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff090100530 tx=0x7ff08000b410 comp rx=0 tx=0).stop 2023-09-10T00:40:26.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.994+0000 7ff09634e700 1 -- 172.21.15.44:0/4029480052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff090101070 msgr2=0x7ff09019d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:27.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.994+0000 7ff09634e700 1 --2- 172.21.15.44:0/4029480052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff090101070 0x7ff09019d740 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff08800ef30 tx=0x7ff08800c3a0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.994+0000 7ff09634e700 1 -- 172.21.15.44:0/4029480052 shutdown_connections 2023-09-10T00:40:27.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.994+0000 7ff09634e700 1 --2- 172.21.15.44:0/4029480052 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7ff0780a0690 0x7ff0780a2b50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.994+0000 7ff09634e700 1 --2- 172.21.15.44:0/4029480052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff090101070 0x7ff09019d740 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.994+0000 7ff09634e700 1 --2- 172.21.15.44:0/4029480052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0900ff4e0 0x7ff090196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.994+0000 7ff09634e700 1 --2- 172.21.15.44:0/4029480052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0901006d0 0x7ff0901966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.994+0000 7ff09634e700 1 -- 172.21.15.44:0/4029480052 >> 172.21.15.44:0/4029480052 conn(0x7ff0900faa30 msgr2=0x7ff0900fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:27.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.994+0000 7ff09634e700 1 -- 172.21.15.44:0/4029480052 shutdown_connections 2023-09-10T00:40:27.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:26.994+0000 7ff09634e700 1 -- 172.21.15.44:0/4029480052 wait complete. 2023-09-10T00:40:27.001 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2023-09-10T00:40:27.006 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:26 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/4012451869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:40:27.007 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2023-09-10T00:40:26.023414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663681,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1016,"num_read_kb":645,"num_write":438,"num_write_kb":3093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1239584,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":863,"ondisk_log_size":863,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338488,"kb_used_data":14272,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603336,"statfs":{"total":1151940427776,"available":1151593816064,"internally_reserved":0,"allocated":14614528,"data_stored":8073294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7266,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":60,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":58848,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.351178"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682673+0000","last_change":"2023-09-10T00:38:39.768218+0000","last_active":"2023-09-10T00:40:16.682673+0000","last_peered":"2023-09-10T00:40:16.682673+0000","last_clean":"2023-09-10T00:40:16.682673+0000","last_became_active":"2023-09-10T00:38:39.767454+0000","last_became_peered":"2023-09-10T00:38:39.767454+0000","last_unstale":"2023-09-10T00:40:16.682673+0000","last_undegraded":"2023-09-10T00:40:16.682673+0000","last_fullsized":"2023-09-10T00:40:16.682673+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:17:30.287403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689406+0000","last_change":"2023-09-10T00:38:39.765384+0000","last_active":"2023-09-10T00:40:16.689406+0000","last_peered":"2023-09-10T00:40:16.689406+0000","last_clean":"2023-09-10T00:40:16.689406+0000","last_became_active":"2023-09-10T00:38:39.765090+0000","last_became_peered":"2023-09-10T00:38:39.765090+0000","last_unstale":"2023-09-10T00:40:16.689406+0000","last_undegraded":"2023-09-10T00:40:16.689406+0000","last_fullsized":"2023-09-10T00:40:16.689406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:02:46.025627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689356+0000","last_change":"2023-09-10T00:38:39.769222+0000","last_active":"2023-09-10T00:40:16.689356+0000","last_peered":"2023-09-10T00:40:16.689356+0000","last_clean":"2023-09-10T00:40:16.689356+0000","last_became_active":"2023-09-10T00:38:39.768801+0000","last_became_peered":"2023-09-10T00:38:39.768801+0000","last_unstale":"2023-09-10T00:40:16.689356+0000","last_undegraded":"2023-09-10T00:40:16.689356+0000","last_fullsized":"2023-09-10T00:40:16.689356+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:38:53.394209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686934+0000","last_change":"2023-09-10T00:38:39.764963+0000","last_active":"2023-09-10T00:40:16.686934+0000","last_peered":"2023-09-10T00:40:16.686934+0000","last_clean":"2023-09-10T00:40:16.686934+0000","last_became_active":"2023-09-10T00:38:39.764466+0000","last_became_peered":"2023-09-10T00:38:39.764466+0000","last_unstale":"2023-09-10T00:40:16.686934+0000","last_undegraded":"2023-09-10T00:40:16.686934+0000","last_fullsized":"2023-09-10T00:40:16.686934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:31.553321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684022+0000","last_change":"2023-09-10T00:39:17.534684+0000","last_active":"2023-09-10T00:40:16.684022+0000","last_peered":"2023-09-10T00:40:16.684022+0000","last_clean":"2023-09-10T00:40:16.684022+0000","last_became_active":"2023-09-10T00:39:17.534325+0000","last_became_peered":"2023-09-10T00:39:17.534325+0000","last_unstale":"2023-09-10T00:40:16.684022+0000","last_undegraded":"2023-09-10T00:40:16.684022+0000","last_fullsized":"2023-09-10T00:40:16.684022+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:39:11.902221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684935+0000","last_change":"2023-09-10T00:38:39.761681+0000","last_active":"2023-09-10T00:40:16.684935+0000","last_peered":"2023-09-10T00:40:16.684935+0000","last_clean":"2023-09-10T00:40:16.684935+0000","last_became_active":"2023-09-10T00:38:39.761383+0000","last_became_peered":"2023-09-10T00:38:39.761383+0000","last_unstale":"2023-09-10T00:40:16.684935+0000","last_undegraded":"2023-09-10T00:40:16.684935+0000","last_fullsized":"2023-09-10T00:40:16.684935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:38:04.534656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684507+0000","last_change":"2023-09-10T00:38:39.765028+0000","last_active":"2023-09-10T00:40:16.684507+0000","last_peered":"2023-09-10T00:40:16.684507+0000","last_clean":"2023-09-10T00:40:16.684507+0000","last_became_active":"2023-09-10T00:38:39.764663+0000","last_became_peered":"2023-09-10T00:38:39.764663+0000","last_unstale":"2023-09-10T00:40:16.684507+0000","last_undegraded":"2023-09-10T00:40:16.684507+0000","last_fullsized":"2023-09-10T00:40:16.684507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:14:49.645129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674630+0000","last_change":"2023-09-10T00:38:39.770135+0000","last_active":"2023-09-10T00:40:16.674630+0000","last_peered":"2023-09-10T00:40:16.674630+0000","last_clean":"2023-09-10T00:40:16.674630+0000","last_became_active":"2023-09-10T00:38:39.769720+0000","last_became_peered":"2023-09-10T00:38:39.769720+0000","last_unstale":"2023-09-10T00:40:16.674630+0000","last_undegraded":"2023-09-10T00:40:16.674630+0000","last_fullsized":"2023-09-10T00:40:16.674630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:46:21.166566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.683408+0000","last_change":"2023-09-10T00:38:39.770170+0000","last_active":"2023-09-10T00:40:16.683408+0000","last_peered":"2023-09-10T00:40:16.683408+0000","last_clean":"2023-09-10T00:40:16.683408+0000","last_became_active":"2023-09-10T00:38:39.769840+0000","last_became_peered":"2023-09-10T00:38:39.769840+0000","last_unstale":"2023-09-10T00:40:16.683408+0000","last_undegraded":"2023-09-10T00:40:16.683408+0000","last_fullsized":"2023-09-10T00:40:16.683408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:48.117670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685644+0000","last_change":"2023-09-10T00:38:39.769621+0000","last_active":"2023-09-10T00:40:16.685644+0000","last_peered":"2023-09-10T00:40:16.685644+0000","last_clean":"2023-09-10T00:40:16.685644+0000","last_became_active":"2023-09-10T00:38:39.769300+0000","last_became_peered":"2023-09-10T00:38:39.769300+0000","last_unstale":"2023-09-10T00:40:16.685644+0000","last_undegraded":"2023-09-10T00:40:16.685644+0000","last_fullsized":"2023-09-10T00:40:16.685644+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:37.100671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682752+0000","last_change":"2023-09-10T00:38:39.772200+0000","last_active":"2023-09-10T00:40:16.682752+0000","last_peered":"2023-09-10T00:40:16.682752+0000","last_clean":"2023-09-10T00:40:16.682752+0000","last_became_active":"2023-09-10T00:38:39.771503+0000","last_became_peered":"2023-09-10T00:38:39.771503+0000","last_unstale":"2023-09-10T00:40:16.682752+0000","last_undegraded":"2023-09-10T00:40:16.682752+0000","last_fullsized":"2023-09-10T00:40:16.682752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:13:38.830642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686922+0000","last_change":"2023-09-10T00:38:39.769832+0000","last_active":"2023-09-10T00:40:16.686922+0000","last_peered":"2023-09-10T00:40:16.686922+0000","last_clean":"2023-09-10T00:40:16.686922+0000","last_became_active":"2023-09-10T00:38:39.769546+0000","last_became_peered":"2023-09-10T00:38:39.769546+0000","last_unstale":"2023-09-10T00:40:16.686922+0000","last_undegraded":"2023-09-10T00:40:16.686922+0000","last_fullsized":"2023-09-10T00:40:16.686922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:53:05.219877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674379+0000","last_change":"2023-09-10T00:39:17.531430+0000","last_active":"2023-09-10T00:40:16.674379+0000","last_peered":"2023-09-10T00:40:16.674379+0000","last_clean":"2023-09-10T00:40:16.674379+0000","last_became_active":"2023-09-10T00:39:17.531145+0000","last_became_peered":"2023-09-10T00:39:17.531145+0000","last_unstale":"2023-09-10T00:40:16.674379+0000","last_undegraded":"2023-09-10T00:40:16.674379+0000","last_fullsized":"2023-09-10T00:40:16.674379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:34:50.163747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.688856+0000","last_change":"2023-09-10T00:38:39.764751+0000","last_active":"2023-09-10T00:40:16.688856+0000","last_peered":"2023-09-10T00:40:16.688856+0000","last_clean":"2023-09-10T00:40:16.688856+0000","last_became_active":"2023-09-10T00:38:39.764464+0000","last_became_peered":"2023-09-10T00:38:39.764464+0000","last_unstale":"2023-09-10T00:40:16.688856+0000","last_undegraded":"2023-09-10T00:40:16.688856+0000","last_fullsized":"2023-09-10T00:40:16.688856+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:03:51.096126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.688911+0000","last_change":"2023-09-10T00:39:17.532977+0000","last_active":"2023-09-10T00:40:16.688911+0000","last_peered":"2023-09-10T00:40:16.688911+0000","last_clean":"2023-09-10T00:40:16.688911+0000","last_became_active":"2023-09-10T00:39:17.532547+0000","last_became_peered":"2023-09-10T00:39:17.532547+0000","last_unstale":"2023-09-10T00:40:16.688911+0000","last_undegraded":"2023-09-10T00:40:16.688911+0000","last_fullsized":"2023-09-10T00:40:16.688911+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:33:01.685743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681772+0000","last_change":"2023-09-10T00:38:39.770787+0000","last_active":"2023-09-10T00:40:16.681772+0000","last_peered":"2023-09-10T00:40:16.681772+0000","last_clean":"2023-09-10T00:40:16.681772+0000","last_became_active":"2023-09-10T00:38:39.770497+0000","last_became_peered":"2023-09-10T00:38:39.770497+0000","last_unstale":"2023-09-10T00:40:16.681772+0000","last_undegraded":"2023-09-10T00:40:16.681772+0000","last_fullsized":"2023-09-10T00:40:16.681772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:43:11.066766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682433+0000","last_change":"2023-09-10T00:38:39.768008+0000","last_active":"2023-09-10T00:40:16.682433+0000","last_peered":"2023-09-10T00:40:16.682433+0000","last_clean":"2023-09-10T00:40:16.682433+0000","last_became_active":"2023-09-10T00:38:39.767079+0000","last_became_peered":"2023-09-10T00:38:39.767079+0000","last_unstale":"2023-09-10T00:40:16.682433+0000","last_undegraded":"2023-09-10T00:40:16.682433+0000","last_fullsized":"2023-09-10T00:40:16.682433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:34:00.689881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685567+0000","last_change":"2023-09-10T00:38:39.775326+0000","last_active":"2023-09-10T00:40:16.685567+0000","last_peered":"2023-09-10T00:40:16.685567+0000","last_clean":"2023-09-10T00:40:16.685567+0000","last_became_active":"2023-09-10T00:38:39.775007+0000","last_became_peered":"2023-09-10T00:38:39.775007+0000","last_unstale":"2023-09-10T00:40:16.685567+0000","last_undegraded":"2023-09-10T00:40:16.685567+0000","last_fullsized":"2023-09-10T00:40:16.685567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:59:27.711722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681440+0000","last_change":"2023-09-10T00:38:39.764974+0000","last_active":"2023-09-10T00:40:16.681440+0000","last_peered":"2023-09-10T00:40:16.681440+0000","last_clean":"2023-09-10T00:40:16.681440+0000","last_became_active":"2023-09-10T00:38:39.764368+0000","last_became_peered":"2023-09-10T00:38:39.764368+0000","last_unstale":"2023-09-10T00:40:16.681440+0000","last_undegraded":"2023-09-10T00:40:16.681440+0000","last_fullsized":"2023-09-10T00:40:16.681440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:40:28.046021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686233+0000","last_change":"2023-09-10T00:38:43.793589+0000","last_active":"2023-09-10T00:40:16.686233+0000","last_peered":"2023-09-10T00:40:16.686233+0000","last_clean":"2023-09-10T00:40:16.686233+0000","last_became_active":"2023-09-10T00:38:43.793166+0000","last_became_peered":"2023-09-10T00:38:43.793166+0000","last_unstale":"2023-09-10T00:40:16.686233+0000","last_undegraded":"2023-09-10T00:40:16.686233+0000","last_fullsized":"2023-09-10T00:40:16.686233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:51:47.128925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":119,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:24.649354+0000","last_change":"2023-09-10T00:38:38.759909+0000","last_active":"2023-09-10T00:40:24.649354+0000","last_peered":"2023-09-10T00:40:24.649354+0000","last_clean":"2023-09-10T00:40:24.649354+0000","last_became_active":"2023-09-10T00:38:38.759561+0000","last_became_peered":"2023-09-10T00:38:38.759561+0000","last_unstale":"2023-09-10T00:40:24.649354+0000","last_undegraded":"2023-09-10T00:40:24.649354+0000","last_fullsized":"2023-09-10T00:40:24.649354+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:56:18.009497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676177+0000","last_change":"2023-09-10T00:38:39.774916+0000","last_active":"2023-09-10T00:40:16.676177+0000","last_peered":"2023-09-10T00:40:16.676177+0000","last_clean":"2023-09-10T00:40:16.676177+0000","last_became_active":"2023-09-10T00:38:39.774584+0000","last_became_peered":"2023-09-10T00:38:39.774584+0000","last_unstale":"2023-09-10T00:40:16.676177+0000","last_undegraded":"2023-09-10T00:40:16.676177+0000","last_fullsized":"2023-09-10T00:40:16.676177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:36:33.125742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686686+0000","last_change":"2023-09-10T00:38:43.789421+0000","last_active":"2023-09-10T00:40:16.686686+0000","last_peered":"2023-09-10T00:40:16.686686+0000","last_clean":"2023-09-10T00:40:16.686686+0000","last_became_active":"2023-09-10T00:38:43.788569+0000","last_became_peered":"2023-09-10T00:38:43.788569+0000","last_unstale":"2023-09-10T00:40:16.686686+0000","last_undegraded":"2023-09-10T00:40:16.686686+0000","last_fullsized":"2023-09-10T00:40:16.686686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:07:38.529811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676261+0000","last_change":"2023-09-10T00:38:38.767381+0000","last_active":"2023-09-10T00:40:16.676261+0000","last_peered":"2023-09-10T00:40:16.676261+0000","last_clean":"2023-09-10T00:40:16.676261+0000","last_became_active":"2023-09-10T00:38:38.767082+0000","last_became_peered":"2023-09-10T00:38:38.767082+0000","last_unstale":"2023-09-10T00:40:16.676261+0000","last_undegraded":"2023-09-10T00:40:16.676261+0000","last_fullsized":"2023-09-10T00:40:16.676261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:13:32.533215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689263+0000","last_change":"2023-09-10T00:38:39.772645+0000","last_active":"2023-09-10T00:40:16.689263+0000","last_peered":"2023-09-10T00:40:16.689263+0000","last_clean":"2023-09-10T00:40:16.689263+0000","last_became_active":"2023-09-10T00:38:39.771554+0000","last_became_peered":"2023-09-10T00:38:39.771554+0000","last_unstale":"2023-09-10T00:40:16.689263+0000","last_undegraded":"2023-09-10T00:40:16.689263+0000","last_fullsized":"2023-09-10T00:40:16.689263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:25:06.237309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686805+0000","last_change":"2023-09-10T00:38:43.795764+0000","last_active":"2023-09-10T00:40:16.686805+0000","last_peered":"2023-09-10T00:40:16.686805+0000","last_clean":"2023-09-10T00:40:16.686805+0000","last_became_active":"2023-09-10T00:38:43.795112+0000","last_became_peered":"2023-09-10T00:38:43.795112+0000","last_unstale":"2023-09-10T00:40:16.686805+0000","last_undegraded":"2023-09-10T00:40:16.686805+0000","last_fullsized":"2023-09-10T00:40:16.686805+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:18:55.460752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.699271+0000","last_change":"2023-09-10T00:40:17.698716+0000","last_active":"2023-09-10T00:40:17.699271+0000","last_peered":"2023-09-10T00:40:17.699271+0000","last_clean":"2023-09-10T00:40:17.699271+0000","last_became_active":"2023-09-10T00:40:17.687287+0000","last_became_peered":"2023-09-10T00:40:17.687287+0000","last_unstale":"2023-09-10T00:40:17.699271+0000","last_undegraded":"2023-09-10T00:40:17.699271+0000","last_fullsized":"2023-09-10T00:40:17.699271+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:07:29.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681429+0000","last_change":"2023-09-10T00:38:39.767790+0000","last_active":"2023-09-10T00:40:16.681429+0000","last_peered":"2023-09-10T00:40:16.681429+0000","last_clean":"2023-09-10T00:40:16.681429+0000","last_became_active":"2023-09-10T00:38:39.767513+0000","last_became_peered":"2023-09-10T00:38:39.767513+0000","last_unstale":"2023-09-10T00:40:16.681429+0000","last_undegraded":"2023-09-10T00:40:16.681429+0000","last_fullsized":"2023-09-10T00:40:16.681429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:05:48.223951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686757+0000","last_change":"2023-09-10T00:38:43.789089+0000","last_active":"2023-09-10T00:40:16.686757+0000","last_peered":"2023-09-10T00:40:16.686757+0000","last_clean":"2023-09-10T00:40:16.686757+0000","last_became_active":"2023-09-10T00:38:43.788434+0000","last_became_peered":"2023-09-10T00:38:43.788434+0000","last_unstale":"2023-09-10T00:40:16.686757+0000","last_undegraded":"2023-09-10T00:40:16.686757+0000","last_fullsized":"2023-09-10T00:40:16.686757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:55:00.236088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685274+0000","last_change":"2023-09-10T00:38:38.769478+0000","last_active":"2023-09-10T00:40:16.685274+0000","last_peered":"2023-09-10T00:40:16.685274+0000","last_clean":"2023-09-10T00:40:16.685274+0000","last_became_active":"2023-09-10T00:38:38.769168+0000","last_became_peered":"2023-09-10T00:38:38.769168+0000","last_unstale":"2023-09-10T00:40:16.685274+0000","last_undegraded":"2023-09-10T00:40:16.685274+0000","last_fullsized":"2023-09-10T00:40:16.685274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:03:48.133561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675591+0000","last_change":"2023-09-10T00:38:39.771843+0000","last_active":"2023-09-10T00:40:16.675591+0000","last_peered":"2023-09-10T00:40:16.675591+0000","last_clean":"2023-09-10T00:40:16.675591+0000","last_became_active":"2023-09-10T00:38:39.771510+0000","last_became_peered":"2023-09-10T00:38:39.771510+0000","last_unstale":"2023-09-10T00:40:16.675591+0000","last_undegraded":"2023-09-10T00:40:16.675591+0000","last_fullsized":"2023-09-10T00:40:16.675591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:17:39.996490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.680942+0000","last_change":"2023-09-10T00:38:43.786853+0000","last_active":"2023-09-10T00:40:16.680942+0000","last_peered":"2023-09-10T00:40:16.680942+0000","last_clean":"2023-09-10T00:40:16.680942+0000","last_became_active":"2023-09-10T00:38:43.786386+0000","last_became_peered":"2023-09-10T00:38:43.786386+0000","last_unstale":"2023-09-10T00:40:16.680942+0000","last_undegraded":"2023-09-10T00:40:16.680942+0000","last_fullsized":"2023-09-10T00:40:16.680942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:23:53.724653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:22.600360+0000","last_change":"2023-09-10T00:38:38.769004+0000","last_active":"2023-09-10T00:40:22.600360+0000","last_peered":"2023-09-10T00:40:22.600360+0000","last_clean":"2023-09-10T00:40:22.600360+0000","last_became_active":"2023-09-10T00:38:38.768690+0000","last_became_peered":"2023-09-10T00:38:38.768690+0000","last_unstale":"2023-09-10T00:40:22.600360+0000","last_undegraded":"2023-09-10T00:40:22.600360+0000","last_fullsized":"2023-09-10T00:40:22.600360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:04:42.531092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1636,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684636+0000","last_change":"2023-09-10T00:38:39.773174+0000","last_active":"2023-09-10T00:40:16.684636+0000","last_peered":"2023-09-10T00:40:16.684636+0000","last_clean":"2023-09-10T00:40:16.684636+0000","last_became_active":"2023-09-10T00:38:39.771192+0000","last_became_peered":"2023-09-10T00:38:39.771192+0000","last_unstale":"2023-09-10T00:40:16.684636+0000","last_undegraded":"2023-09-10T00:40:16.684636+0000","last_fullsized":"2023-09-10T00:40:16.684636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:16:25.034024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.687731+0000","last_change":"2023-09-10T00:40:17.687731+0000","last_active":"2023-09-10T00:40:17.687731+0000","last_peered":"2023-09-10T00:40:17.687731+0000","last_clean":"2023-09-10T00:40:17.687731+0000","last_became_active":"2023-09-10T00:40:17.687285+0000","last_became_peered":"2023-09-10T00:40:17.687285+0000","last_unstale":"2023-09-10T00:40:17.687731+0000","last_undegraded":"2023-09-10T00:40:17.687731+0000","last_fullsized":"2023-09-10T00:40:17.687731+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:16:29.995198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684888+0000","last_change":"2023-09-10T00:38:38.763416+0000","last_active":"2023-09-10T00:40:16.684888+0000","last_peered":"2023-09-10T00:40:16.684888+0000","last_clean":"2023-09-10T00:40:16.684888+0000","last_became_active":"2023-09-10T00:38:38.763054+0000","last_became_peered":"2023-09-10T00:38:38.763054+0000","last_unstale":"2023-09-10T00:40:16.684888+0000","last_undegraded":"2023-09-10T00:40:16.684888+0000","last_fullsized":"2023-09-10T00:40:16.684888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:47:00.179467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675938+0000","last_change":"2023-09-10T00:38:39.773284+0000","last_active":"2023-09-10T00:40:16.675938+0000","last_peered":"2023-09-10T00:40:16.675938+0000","last_clean":"2023-09-10T00:40:16.675938+0000","last_became_active":"2023-09-10T00:38:39.772969+0000","last_became_peered":"2023-09-10T00:38:39.772969+0000","last_unstale":"2023-09-10T00:40:16.675938+0000","last_undegraded":"2023-09-10T00:40:16.675938+0000","last_fullsized":"2023-09-10T00:40:16.675938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:46:45.013879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682333+0000","last_change":"2023-09-10T00:38:43.790810+0000","last_active":"2023-09-10T00:40:16.682333+0000","last_peered":"2023-09-10T00:40:16.682333+0000","last_clean":"2023-09-10T00:40:16.682333+0000","last_became_active":"2023-09-10T00:38:43.790429+0000","last_became_peered":"2023-09-10T00:38:43.790429+0000","last_unstale":"2023-09-10T00:40:16.682333+0000","last_undegraded":"2023-09-10T00:40:16.682333+0000","last_fullsized":"2023-09-10T00:40:16.682333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:30:56.072587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.673630+0000","last_change":"2023-09-10T00:38:38.754326+0000","last_active":"2023-09-10T00:40:16.673630+0000","last_peered":"2023-09-10T00:40:16.673630+0000","last_clean":"2023-09-10T00:40:16.673630+0000","last_became_active":"2023-09-10T00:38:38.754006+0000","last_became_peered":"2023-09-10T00:38:38.754006+0000","last_unstale":"2023-09-10T00:40:16.673630+0000","last_undegraded":"2023-09-10T00:40:16.673630+0000","last_fullsized":"2023-09-10T00:40:16.673630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:09:26.983606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674527+0000","last_change":"2023-09-10T00:38:39.769699+0000","last_active":"2023-09-10T00:40:16.674527+0000","last_peered":"2023-09-10T00:40:16.674527+0000","last_clean":"2023-09-10T00:40:16.674527+0000","last_became_active":"2023-09-10T00:38:39.769362+0000","last_became_peered":"2023-09-10T00:38:39.769362+0000","last_unstale":"2023-09-10T00:40:16.674527+0000","last_undegraded":"2023-09-10T00:40:16.674527+0000","last_fullsized":"2023-09-10T00:40:16.674527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:30:20.938437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689355+0000","last_change":"2023-09-10T00:38:43.785669+0000","last_active":"2023-09-10T00:40:16.689355+0000","last_peered":"2023-09-10T00:40:16.689355+0000","last_clean":"2023-09-10T00:40:16.689355+0000","last_became_active":"2023-09-10T00:38:43.785255+0000","last_became_peered":"2023-09-10T00:38:43.785255+0000","last_unstale":"2023-09-10T00:40:16.689355+0000","last_undegraded":"2023-09-10T00:40:16.689355+0000","last_fullsized":"2023-09-10T00:40:16.689355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:52:57.535721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675650+0000","last_change":"2023-09-10T00:38:38.767997+0000","last_active":"2023-09-10T00:40:16.675650+0000","last_peered":"2023-09-10T00:40:16.675650+0000","last_clean":"2023-09-10T00:40:16.675650+0000","last_became_active":"2023-09-10T00:38:38.767715+0000","last_became_peered":"2023-09-10T00:38:38.767715+0000","last_unstale":"2023-09-10T00:40:16.675650+0000","last_undegraded":"2023-09-10T00:40:16.675650+0000","last_fullsized":"2023-09-10T00:40:16.675650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:11:17.047280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685405+0000","last_change":"2023-09-10T00:38:39.778004+0000","last_active":"2023-09-10T00:40:16.685405+0000","last_peered":"2023-09-10T00:40:16.685405+0000","last_clean":"2023-09-10T00:40:16.685405+0000","last_became_active":"2023-09-10T00:38:39.777632+0000","last_became_peered":"2023-09-10T00:38:39.777632+0000","last_unstale":"2023-09-10T00:40:16.685405+0000","last_undegraded":"2023-09-10T00:40:16.685405+0000","last_fullsized":"2023-09-10T00:40:16.685405+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:46:40.804385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.682227+0000","last_change":"2023-09-10T00:40:17.682227+0000","last_active":"2023-09-10T00:40:17.682227+0000","last_peered":"2023-09-10T00:40:17.682227+0000","last_clean":"2023-09-10T00:40:17.682227+0000","last_became_active":"2023-09-10T00:40:17.681849+0000","last_became_peered":"2023-09-10T00:40:17.681849+0000","last_unstale":"2023-09-10T00:40:17.682227+0000","last_undegraded":"2023-09-10T00:40:17.682227+0000","last_fullsized":"2023-09-10T00:40:17.682227+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:59.568162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":194,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:24.602900+0000","last_change":"2023-09-10T00:38:38.767399+0000","last_active":"2023-09-10T00:40:24.602900+0000","last_peered":"2023-09-10T00:40:24.602900+0000","last_clean":"2023-09-10T00:40:24.602900+0000","last_became_active":"2023-09-10T00:38:38.767176+0000","last_became_peered":"2023-09-10T00:38:38.767176+0000","last_unstale":"2023-09-10T00:40:24.602900+0000","last_undegraded":"2023-09-10T00:40:24.602900+0000","last_fullsized":"2023-09-10T00:40:24.602900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:17:24.481985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681626+0000","last_change":"2023-09-10T00:38:39.766013+0000","last_active":"2023-09-10T00:40:16.681626+0000","last_peered":"2023-09-10T00:40:16.681626+0000","last_clean":"2023-09-10T00:40:16.681626+0000","last_became_active":"2023-09-10T00:38:39.765235+0000","last_became_peered":"2023-09-10T00:38:39.765235+0000","last_unstale":"2023-09-10T00:40:16.681626+0000","last_undegraded":"2023-09-10T00:40:16.681626+0000","last_fullsized":"2023-09-10T00:40:16.681626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:01:48.874845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685291+0000","last_change":"2023-09-10T00:38:43.790361+0000","last_active":"2023-09-10T00:40:16.685291+0000","last_peered":"2023-09-10T00:40:16.685291+0000","last_clean":"2023-09-10T00:40:16.685291+0000","last_became_active":"2023-09-10T00:38:43.789966+0000","last_became_peered":"2023-09-10T00:38:43.789966+0000","last_unstale":"2023-09-10T00:40:16.685291+0000","last_undegraded":"2023-09-10T00:40:16.685291+0000","last_fullsized":"2023-09-10T00:40:16.685291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:46:37.978258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682397+0000","last_change":"2023-09-10T00:38:38.771670+0000","last_active":"2023-09-10T00:40:16.682397+0000","last_peered":"2023-09-10T00:40:16.682397+0000","last_clean":"2023-09-10T00:40:16.682397+0000","last_became_active":"2023-09-10T00:38:38.771339+0000","last_became_peered":"2023-09-10T00:38:38.771339+0000","last_unstale":"2023-09-10T00:40:16.682397+0000","last_undegraded":"2023-09-10T00:40:16.682397+0000","last_fullsized":"2023-09-10T00:40:16.682397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:22:32.658544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689361+0000","last_change":"2023-09-10T00:38:39.772857+0000","last_active":"2023-09-10T00:40:16.689361+0000","last_peered":"2023-09-10T00:40:16.689361+0000","last_clean":"2023-09-10T00:40:16.689361+0000","last_became_active":"2023-09-10T00:38:39.771990+0000","last_became_peered":"2023-09-10T00:38:39.771990+0000","last_unstale":"2023-09-10T00:40:16.689361+0000","last_undegraded":"2023-09-10T00:40:16.689361+0000","last_fullsized":"2023-09-10T00:40:16.689361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:16:15.767748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686883+0000","last_change":"2023-09-10T00:38:43.795482+0000","last_active":"2023-09-10T00:40:16.686883+0000","last_peered":"2023-09-10T00:40:16.686883+0000","last_clean":"2023-09-10T00:40:16.686883+0000","last_became_active":"2023-09-10T00:38:43.795048+0000","last_became_peered":"2023-09-10T00:38:43.795048+0000","last_unstale":"2023-09-10T00:40:16.686883+0000","last_undegraded":"2023-09-10T00:40:16.686883+0000","last_fullsized":"2023-09-10T00:40:16.686883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:13:18.715274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675457+0000","last_change":"2023-09-10T00:38:38.771008+0000","last_active":"2023-09-10T00:40:16.675457+0000","last_peered":"2023-09-10T00:40:16.675457+0000","last_clean":"2023-09-10T00:40:16.675457+0000","last_became_active":"2023-09-10T00:38:38.770714+0000","last_became_peered":"2023-09-10T00:38:38.770714+0000","last_unstale":"2023-09-10T00:40:16.675457+0000","last_undegraded":"2023-09-10T00:40:16.675457+0000","last_fullsized":"2023-09-10T00:40:16.675457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:04:30.148141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682324+0000","last_change":"2023-09-10T00:39:17.533500+0000","last_active":"2023-09-10T00:40:16.682324+0000","last_peered":"2023-09-10T00:40:16.682324+0000","last_clean":"2023-09-10T00:40:16.682324+0000","last_became_active":"2023-09-10T00:39:17.533175+0000","last_became_peered":"2023-09-10T00:39:17.533175+0000","last_unstale":"2023-09-10T00:40:16.682324+0000","last_undegraded":"2023-09-10T00:40:16.682324+0000","last_fullsized":"2023-09-10T00:40:16.682324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:13:42.121699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684740+0000","last_change":"2023-09-10T00:39:17.536684+0000","last_active":"2023-09-10T00:40:16.684740+0000","last_peered":"2023-09-10T00:40:16.684740+0000","last_clean":"2023-09-10T00:40:16.684740+0000","last_became_active":"2023-09-10T00:39:17.536170+0000","last_became_peered":"2023-09-10T00:39:17.536170+0000","last_unstale":"2023-09-10T00:40:16.684740+0000","last_undegraded":"2023-09-10T00:40:16.684740+0000","last_fullsized":"2023-09-10T00:40:16.684740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:47:16.264619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":259,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:23.744670+0000","last_change":"2023-09-10T00:40:17.741148+0000","last_active":"2023-09-10T00:40:23.744670+0000","last_peered":"2023-09-10T00:40:23.744670+0000","last_clean":"2023-09-10T00:40:23.744670+0000","last_became_active":"2023-09-10T00:40:17.731033+0000","last_became_peered":"2023-09-10T00:40:17.731033+0000","last_unstale":"2023-09-10T00:40:23.744670+0000","last_undegraded":"2023-09-10T00:40:23.744670+0000","last_fullsized":"2023-09-10T00:40:23.744670+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:20:42.067739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63651,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":176,"num_read_kb":150,"num_write":40,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":58848,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684860+0000","last_change":"2023-09-10T00:38:39.767507+0000","last_active":"2023-09-10T00:40:16.684860+0000","last_peered":"2023-09-10T00:40:16.684860+0000","last_clean":"2023-09-10T00:40:16.684860+0000","last_became_active":"2023-09-10T00:38:39.767223+0000","last_became_peered":"2023-09-10T00:38:39.767223+0000","last_unstale":"2023-09-10T00:40:16.684860+0000","last_undegraded":"2023-09-10T00:40:16.684860+0000","last_fullsized":"2023-09-10T00:40:16.684860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:46:30.901818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676490+0000","last_change":"2023-09-10T00:38:43.785396+0000","last_active":"2023-09-10T00:40:16.676490+0000","last_peered":"2023-09-10T00:40:16.676490+0000","last_clean":"2023-09-10T00:40:16.676490+0000","last_became_active":"2023-09-10T00:38:43.785038+0000","last_became_peered":"2023-09-10T00:38:43.785038+0000","last_unstale":"2023-09-10T00:40:16.676490+0000","last_undegraded":"2023-09-10T00:40:16.676490+0000","last_fullsized":"2023-09-10T00:40:16.676490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:30:31.629703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.680630+0000","last_change":"2023-09-10T00:38:38.766742+0000","last_active":"2023-09-10T00:40:16.680630+0000","last_peered":"2023-09-10T00:40:16.680630+0000","last_clean":"2023-09-10T00:40:16.680630+0000","last_became_active":"2023-09-10T00:38:38.766220+0000","last_became_peered":"2023-09-10T00:38:38.766220+0000","last_unstale":"2023-09-10T00:40:16.680630+0000","last_undegraded":"2023-09-10T00:40:16.680630+0000","last_fullsized":"2023-09-10T00:40:16.680630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:41:22.030448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.673843+0000","last_change":"2023-09-10T00:38:39.766022+0000","last_active":"2023-09-10T00:40:16.673843+0000","last_peered":"2023-09-10T00:40:16.673843+0000","last_clean":"2023-09-10T00:40:16.673843+0000","last_became_active":"2023-09-10T00:38:39.765697+0000","last_became_peered":"2023-09-10T00:38:39.765697+0000","last_unstale":"2023-09-10T00:40:16.673843+0000","last_undegraded":"2023-09-10T00:40:16.673843+0000","last_fullsized":"2023-09-10T00:40:16.673843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:19:39.394299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681669+0000","last_change":"2023-09-10T00:38:43.795492+0000","last_active":"2023-09-10T00:40:16.681669+0000","last_peered":"2023-09-10T00:40:16.681669+0000","last_clean":"2023-09-10T00:40:16.681669+0000","last_became_active":"2023-09-10T00:38:43.794968+0000","last_became_peered":"2023-09-10T00:38:43.794968+0000","last_unstale":"2023-09-10T00:40:16.681669+0000","last_undegraded":"2023-09-10T00:40:16.681669+0000","last_fullsized":"2023-09-10T00:40:16.681669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:40:43.558870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676085+0000","last_change":"2023-09-10T00:38:38.767666+0000","last_active":"2023-09-10T00:40:16.676085+0000","last_peered":"2023-09-10T00:40:16.676085+0000","last_clean":"2023-09-10T00:40:16.676085+0000","last_became_active":"2023-09-10T00:38:38.767333+0000","last_became_peered":"2023-09-10T00:38:38.767333+0000","last_unstale":"2023-09-10T00:40:16.676085+0000","last_undegraded":"2023-09-10T00:40:16.676085+0000","last_fullsized":"2023-09-10T00:40:16.676085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:07:53.363196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682945+0000","last_change":"2023-09-10T00:38:39.766036+0000","last_active":"2023-09-10T00:40:16.682945+0000","last_peered":"2023-09-10T00:40:16.682945+0000","last_clean":"2023-09-10T00:40:16.682945+0000","last_became_active":"2023-09-10T00:38:39.765683+0000","last_became_peered":"2023-09-10T00:38:39.765683+0000","last_unstale":"2023-09-10T00:40:16.682945+0000","last_undegraded":"2023-09-10T00:40:16.682945+0000","last_fullsized":"2023-09-10T00:40:16.682945+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:58:04.429960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686204+0000","last_change":"2023-09-10T00:38:43.787870+0000","last_active":"2023-09-10T00:40:16.686204+0000","last_peered":"2023-09-10T00:40:16.686204+0000","last_clean":"2023-09-10T00:40:16.686204+0000","last_became_active":"2023-09-10T00:38:43.785915+0000","last_became_peered":"2023-09-10T00:38:43.785915+0000","last_unstale":"2023-09-10T00:40:16.686204+0000","last_undegraded":"2023-09-10T00:40:16.686204+0000","last_fullsized":"2023-09-10T00:40:16.686204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:34:08.623967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'8","reported_seq":116,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:21.597796+0000","last_change":"2023-09-10T00:38:38.756925+0000","last_active":"2023-09-10T00:40:21.597796+0000","last_peered":"2023-09-10T00:40:21.597796+0000","last_clean":"2023-09-10T00:40:21.597796+0000","last_became_active":"2023-09-10T00:38:38.756610+0000","last_became_peered":"2023-09-10T00:38:38.756610+0000","last_unstale":"2023-09-10T00:40:21.597796+0000","last_undegraded":"2023-09-10T00:40:21.597796+0000","last_fullsized":"2023-09-10T00:40:21.597796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:48:15.385230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1602,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682147+0000","last_change":"2023-09-10T00:38:39.766750+0000","last_active":"2023-09-10T00:40:16.682147+0000","last_peered":"2023-09-10T00:40:16.682147+0000","last_clean":"2023-09-10T00:40:16.682147+0000","last_became_active":"2023-09-10T00:38:39.766458+0000","last_became_peered":"2023-09-10T00:38:39.766458+0000","last_unstale":"2023-09-10T00:40:16.682147+0000","last_undegraded":"2023-09-10T00:40:16.682147+0000","last_fullsized":"2023-09-10T00:40:16.682147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:59:26.691487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674310+0000","last_change":"2023-09-10T00:38:43.788814+0000","last_active":"2023-09-10T00:40:16.674310+0000","last_peered":"2023-09-10T00:40:16.674310+0000","last_clean":"2023-09-10T00:40:16.674310+0000","last_became_active":"2023-09-10T00:38:43.788446+0000","last_became_peered":"2023-09-10T00:38:43.788446+0000","last_unstale":"2023-09-10T00:40:16.674310+0000","last_undegraded":"2023-09-10T00:40:16.674310+0000","last_fullsized":"2023-09-10T00:40:16.674310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:38:53.308160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684332+0000","last_change":"2023-09-10T00:38:38.766020+0000","last_active":"2023-09-10T00:40:16.684332+0000","last_peered":"2023-09-10T00:40:16.684332+0000","last_clean":"2023-09-10T00:40:16.684332+0000","last_became_active":"2023-09-10T00:38:38.765719+0000","last_became_peered":"2023-09-10T00:38:38.765719+0000","last_unstale":"2023-09-10T00:40:16.684332+0000","last_undegraded":"2023-09-10T00:40:16.684332+0000","last_fullsized":"2023-09-10T00:40:16.684332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:25:30.488014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676849+0000","last_change":"2023-09-10T00:38:39.771091+0000","last_active":"2023-09-10T00:40:16.676849+0000","last_peered":"2023-09-10T00:40:16.676849+0000","last_clean":"2023-09-10T00:40:16.676849+0000","last_became_active":"2023-09-10T00:38:39.770745+0000","last_became_peered":"2023-09-10T00:38:39.770745+0000","last_unstale":"2023-09-10T00:40:16.676849+0000","last_undegraded":"2023-09-10T00:40:16.676849+0000","last_fullsized":"2023-09-10T00:40:16.676849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:51:17.978466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676732+0000","last_change":"2023-09-10T00:38:43.793757+0000","last_active":"2023-09-10T00:40:16.676732+0000","last_peered":"2023-09-10T00:40:16.676732+0000","last_clean":"2023-09-10T00:40:16.676732+0000","last_became_active":"2023-09-10T00:38:43.793336+0000","last_became_peered":"2023-09-10T00:38:43.793336+0000","last_unstale":"2023-09-10T00:40:16.676732+0000","last_undegraded":"2023-09-10T00:40:16.676732+0000","last_fullsized":"2023-09-10T00:40:16.676732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:22:36.544950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686902+0000","last_change":"2023-09-10T00:38:38.769373+0000","last_active":"2023-09-10T00:40:16.686902+0000","last_peered":"2023-09-10T00:40:16.686902+0000","last_clean":"2023-09-10T00:40:16.686902+0000","last_became_active":"2023-09-10T00:38:38.769054+0000","last_became_peered":"2023-09-10T00:38:38.769054+0000","last_unstale":"2023-09-10T00:40:16.686902+0000","last_undegraded":"2023-09-10T00:40:16.686902+0000","last_fullsized":"2023-09-10T00:40:16.686902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:12:27.982405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681789+0000","last_change":"2023-09-10T00:39:17.537427+0000","last_active":"2023-09-10T00:40:16.681789+0000","last_peered":"2023-09-10T00:40:16.681789+0000","last_clean":"2023-09-10T00:40:16.681789+0000","last_became_active":"2023-09-10T00:39:17.537023+0000","last_became_peered":"2023-09-10T00:39:17.537023+0000","last_unstale":"2023-09-10T00:40:16.681789+0000","last_undegraded":"2023-09-10T00:40:16.681789+0000","last_fullsized":"2023-09-10T00:40:16.681789+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:54:43.720224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684523+0000","last_change":"2023-09-10T00:38:43.838689+0000","last_active":"2023-09-10T00:40:16.684523+0000","last_peered":"2023-09-10T00:40:16.684523+0000","last_clean":"2023-09-10T00:40:16.684523+0000","last_became_active":"2023-09-10T00:38:43.838244+0000","last_became_peered":"2023-09-10T00:38:43.838244+0000","last_unstale":"2023-09-10T00:40:16.684523+0000","last_undegraded":"2023-09-10T00:40:16.684523+0000","last_fullsized":"2023-09-10T00:40:16.684523+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:54:00.033719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685917+0000","last_change":"2023-09-10T00:38:38.765140+0000","last_active":"2023-09-10T00:40:16.685917+0000","last_peered":"2023-09-10T00:40:16.685917+0000","last_clean":"2023-09-10T00:40:16.685917+0000","last_became_active":"2023-09-10T00:38:38.764730+0000","last_became_peered":"2023-09-10T00:38:38.764730+0000","last_unstale":"2023-09-10T00:40:16.685917+0000","last_undegraded":"2023-09-10T00:40:16.685917+0000","last_fullsized":"2023-09-10T00:40:16.685917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:49:41.129763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685440+0000","last_change":"2023-09-10T00:38:39.773260+0000","last_active":"2023-09-10T00:40:16.685440+0000","last_peered":"2023-09-10T00:40:16.685440+0000","last_clean":"2023-09-10T00:40:16.685440+0000","last_became_active":"2023-09-10T00:38:39.772947+0000","last_became_peered":"2023-09-10T00:38:39.772947+0000","last_unstale":"2023-09-10T00:40:16.685440+0000","last_undegraded":"2023-09-10T00:40:16.685440+0000","last_fullsized":"2023-09-10T00:40:16.685440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:08:10.791610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674639+0000","last_change":"2023-09-10T00:38:43.787882+0000","last_active":"2023-09-10T00:40:16.674639+0000","last_peered":"2023-09-10T00:40:16.674639+0000","last_clean":"2023-09-10T00:40:16.674639+0000","last_became_active":"2023-09-10T00:38:43.787257+0000","last_became_peered":"2023-09-10T00:38:43.787257+0000","last_unstale":"2023-09-10T00:40:16.674639+0000","last_undegraded":"2023-09-10T00:40:16.674639+0000","last_fullsized":"2023-09-10T00:40:16.674639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:11:19.171881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:21.645218+0000","last_change":"2023-09-10T00:38:38.760014+0000","last_active":"2023-09-10T00:40:21.645218+0000","last_peered":"2023-09-10T00:40:21.645218+0000","last_clean":"2023-09-10T00:40:21.645218+0000","last_became_active":"2023-09-10T00:38:38.759637+0000","last_became_peered":"2023-09-10T00:38:38.759637+0000","last_unstale":"2023-09-10T00:40:21.645218+0000","last_undegraded":"2023-09-10T00:40:21.645218+0000","last_fullsized":"2023-09-10T00:40:21.645218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:14:15.704929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1071,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684955+0000","last_change":"2023-09-10T00:38:39.765800+0000","last_active":"2023-09-10T00:40:16.684955+0000","last_peered":"2023-09-10T00:40:16.684955+0000","last_clean":"2023-09-10T00:40:16.684955+0000","last_became_active":"2023-09-10T00:38:39.765468+0000","last_became_peered":"2023-09-10T00:38:39.765468+0000","last_unstale":"2023-09-10T00:40:16.684955+0000","last_undegraded":"2023-09-10T00:40:16.684955+0000","last_fullsized":"2023-09-10T00:40:16.684955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:57:56.429365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674169+0000","last_change":"2023-09-10T00:38:43.838285+0000","last_active":"2023-09-10T00:40:16.674169+0000","last_peered":"2023-09-10T00:40:16.674169+0000","last_clean":"2023-09-10T00:40:16.674169+0000","last_became_active":"2023-09-10T00:38:43.837821+0000","last_became_peered":"2023-09-10T00:38:43.837821+0000","last_unstale":"2023-09-10T00:40:16.674169+0000","last_undegraded":"2023-09-10T00:40:16.674169+0000","last_fullsized":"2023-09-10T00:40:16.674169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:01:03.354281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675958+0000","last_change":"2023-09-10T00:39:17.536361+0000","last_active":"2023-09-10T00:40:16.675958+0000","last_peered":"2023-09-10T00:40:16.675958+0000","last_clean":"2023-09-10T00:40:16.675958+0000","last_became_active":"2023-09-10T00:39:17.535930+0000","last_became_peered":"2023-09-10T00:39:17.535930+0000","last_unstale":"2023-09-10T00:40:16.675958+0000","last_undegraded":"2023-09-10T00:40:16.675958+0000","last_fullsized":"2023-09-10T00:40:16.675958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:53.576115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":114,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:24.601702+0000","last_change":"2023-09-10T00:38:38.758745+0000","last_active":"2023-09-10T00:40:24.601702+0000","last_peered":"2023-09-10T00:40:24.601702+0000","last_clean":"2023-09-10T00:40:24.601702+0000","last_became_active":"2023-09-10T00:38:38.758427+0000","last_became_peered":"2023-09-10T00:38:38.758427+0000","last_unstale":"2023-09-10T00:40:24.601702+0000","last_undegraded":"2023-09-10T00:40:24.601702+0000","last_fullsized":"2023-09-10T00:40:24.601702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:02:49.238175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.683243+0000","last_change":"2023-09-10T00:38:43.784058+0000","last_active":"2023-09-10T00:40:16.683243+0000","last_peered":"2023-09-10T00:40:16.683243+0000","last_clean":"2023-09-10T00:40:16.683243+0000","last_became_active":"2023-09-10T00:38:43.783571+0000","last_became_peered":"2023-09-10T00:38:43.783571+0000","last_unstale":"2023-09-10T00:40:16.683243+0000","last_undegraded":"2023-09-10T00:40:16.683243+0000","last_fullsized":"2023-09-10T00:40:16.683243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:53:50.992393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676566+0000","last_change":"2023-09-10T00:38:39.775659+0000","last_active":"2023-09-10T00:40:16.676566+0000","last_peered":"2023-09-10T00:40:16.676566+0000","last_clean":"2023-09-10T00:40:16.676566+0000","last_became_active":"2023-09-10T00:38:39.775151+0000","last_became_peered":"2023-09-10T00:38:39.775151+0000","last_unstale":"2023-09-10T00:40:16.676566+0000","last_undegraded":"2023-09-10T00:40:16.676566+0000","last_fullsized":"2023-09-10T00:40:16.676566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:07.415418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686900+0000","last_change":"2023-09-10T00:38:38.755009+0000","last_active":"2023-09-10T00:40:16.686900+0000","last_peered":"2023-09-10T00:40:16.686900+0000","last_clean":"2023-09-10T00:40:16.686900+0000","last_became_active":"2023-09-10T00:38:38.754699+0000","last_became_peered":"2023-09-10T00:38:38.754699+0000","last_unstale":"2023-09-10T00:40:16.686900+0000","last_undegraded":"2023-09-10T00:40:16.686900+0000","last_fullsized":"2023-09-10T00:40:16.686900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:12:02.075256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":324,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681263+0000","last_change":"2023-09-10T00:37:38.459695+0000","last_active":"2023-09-10T00:40:16.681263+0000","last_peered":"2023-09-10T00:40:16.681263+0000","last_clean":"2023-09-10T00:40:16.681263+0000","last_became_active":"2023-09-10T00:37:38.459220+0000","last_became_peered":"2023-09-10T00:37:38.459220+0000","last_unstale":"2023-09-10T00:40:16.681263+0000","last_undegraded":"2023-09-10T00:40:16.681263+0000","last_fullsized":"2023-09-10T00:40:16.681263+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:34:44.602334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:34:44.602334+0000","last_clean_scrub_stamp":"2023-09-10T00:34:44.602334+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:21:32.648725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675426+0000","last_change":"2023-09-10T00:38:43.839001+0000","last_active":"2023-09-10T00:40:16.675426+0000","last_peered":"2023-09-10T00:40:16.675426+0000","last_clean":"2023-09-10T00:40:16.675426+0000","last_became_active":"2023-09-10T00:38:43.838497+0000","last_became_peered":"2023-09-10T00:38:43.838497+0000","last_unstale":"2023-09-10T00:40:16.675426+0000","last_undegraded":"2023-09-10T00:40:16.675426+0000","last_fullsized":"2023-09-10T00:40:16.675426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:57:23.349632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.673565+0000","last_change":"2023-09-10T00:38:39.760150+0000","last_active":"2023-09-10T00:40:16.673565+0000","last_peered":"2023-09-10T00:40:16.673565+0000","last_clean":"2023-09-10T00:40:16.673565+0000","last_became_active":"2023-09-10T00:38:39.759811+0000","last_became_peered":"2023-09-10T00:38:39.759811+0000","last_unstale":"2023-09-10T00:40:16.673565+0000","last_undegraded":"2023-09-10T00:40:16.673565+0000","last_fullsized":"2023-09-10T00:40:16.673565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:21:19.359057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.701421+0000","last_change":"2023-09-10T00:40:17.700994+0000","last_active":"2023-09-10T00:40:17.701421+0000","last_peered":"2023-09-10T00:40:17.701421+0000","last_clean":"2023-09-10T00:40:17.701421+0000","last_became_active":"2023-09-10T00:40:17.690073+0000","last_became_peered":"2023-09-10T00:40:17.690073+0000","last_unstale":"2023-09-10T00:40:17.701421+0000","last_undegraded":"2023-09-10T00:40:17.701421+0000","last_fullsized":"2023-09-10T00:40:17.701421+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:43:32.478865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681570+0000","last_change":"2023-09-10T00:38:43.786272+0000","last_active":"2023-09-10T00:40:16.681570+0000","last_peered":"2023-09-10T00:40:16.681570+0000","last_clean":"2023-09-10T00:40:16.681570+0000","last_became_active":"2023-09-10T00:38:43.785934+0000","last_became_peered":"2023-09-10T00:38:43.785934+0000","last_unstale":"2023-09-10T00:40:16.681570+0000","last_undegraded":"2023-09-10T00:40:16.681570+0000","last_fullsized":"2023-09-10T00:40:16.681570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:31:49.721133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684970+0000","last_change":"2023-09-10T00:38:38.763263+0000","last_active":"2023-09-10T00:40:16.684970+0000","last_peered":"2023-09-10T00:40:16.684970+0000","last_clean":"2023-09-10T00:40:16.684970+0000","last_became_active":"2023-09-10T00:38:38.762911+0000","last_became_peered":"2023-09-10T00:38:38.762911+0000","last_unstale":"2023-09-10T00:40:16.684970+0000","last_undegraded":"2023-09-10T00:40:16.684970+0000","last_fullsized":"2023-09-10T00:40:16.684970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:19:10.292311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674491+0000","last_change":"2023-09-10T00:38:39.759171+0000","last_active":"2023-09-10T00:40:16.674491+0000","last_peered":"2023-09-10T00:40:16.674491+0000","last_clean":"2023-09-10T00:40:16.674491+0000","last_became_active":"2023-09-10T00:38:39.758843+0000","last_became_peered":"2023-09-10T00:38:39.758843+0000","last_unstale":"2023-09-10T00:40:16.674491+0000","last_undegraded":"2023-09-10T00:40:16.674491+0000","last_fullsized":"2023-09-10T00:40:16.674491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:50:36.064937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674438+0000","last_change":"2023-09-10T00:38:43.837998+0000","last_active":"2023-09-10T00:40:16.674438+0000","last_peered":"2023-09-10T00:40:16.674438+0000","last_clean":"2023-09-10T00:40:16.674438+0000","last_became_active":"2023-09-10T00:38:43.837556+0000","last_became_peered":"2023-09-10T00:38:43.837556+0000","last_unstale":"2023-09-10T00:40:16.674438+0000","last_undegraded":"2023-09-10T00:40:16.674438+0000","last_fullsized":"2023-09-10T00:40:16.674438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:18:40.488279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:18.399592+0000","last_change":"2023-09-10T00:38:38.759819+0000","last_active":"2023-09-10T00:40:18.399592+0000","last_peered":"2023-09-10T00:40:18.399592+0000","last_clean":"2023-09-10T00:40:18.399592+0000","last_became_active":"2023-09-10T00:38:38.759490+0000","last_became_peered":"2023-09-10T00:38:38.759490+0000","last_unstale":"2023-09-10T00:40:18.399592+0000","last_undegraded":"2023-09-10T00:40:18.399592+0000","last_fullsized":"2023-09-10T00:40:18.399592+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:39:01.206817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675634+0000","last_change":"2023-09-10T00:38:39.769669+0000","last_active":"2023-09-10T00:40:16.675634+0000","last_peered":"2023-09-10T00:40:16.675634+0000","last_clean":"2023-09-10T00:40:16.675634+0000","last_became_active":"2023-09-10T00:38:39.769402+0000","last_became_peered":"2023-09-10T00:38:39.769402+0000","last_unstale":"2023-09-10T00:40:16.675634+0000","last_undegraded":"2023-09-10T00:40:16.675634+0000","last_fullsized":"2023-09-10T00:40:16.675634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:00:50.215103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685982+0000","last_change":"2023-09-10T00:38:43.787518+0000","last_active":"2023-09-10T00:40:16.685982+0000","last_peered":"2023-09-10T00:40:16.685982+0000","last_clean":"2023-09-10T00:40:16.685982+0000","last_became_active":"2023-09-10T00:38:43.786410+0000","last_became_peered":"2023-09-10T00:38:43.786410+0000","last_unstale":"2023-09-10T00:40:16.685982+0000","last_undegraded":"2023-09-10T00:40:16.685982+0000","last_fullsized":"2023-09-10T00:40:16.685982+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:13:37.903875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.687579+0000","last_change":"2023-09-10T00:38:38.760992+0000","last_active":"2023-09-10T00:40:16.687579+0000","last_peered":"2023-09-10T00:40:16.687579+0000","last_clean":"2023-09-10T00:40:16.687579+0000","last_became_active":"2023-09-10T00:38:38.760635+0000","last_became_peered":"2023-09-10T00:38:38.760635+0000","last_unstale":"2023-09-10T00:40:16.687579+0000","last_undegraded":"2023-09-10T00:40:16.687579+0000","last_fullsized":"2023-09-10T00:40:16.687579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:29:36.070477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689409+0000","last_change":"2023-09-10T00:38:39.773771+0000","last_active":"2023-09-10T00:40:16.689409+0000","last_peered":"2023-09-10T00:40:16.689409+0000","last_clean":"2023-09-10T00:40:16.689409+0000","last_became_active":"2023-09-10T00:38:39.773296+0000","last_became_peered":"2023-09-10T00:38:39.773296+0000","last_unstale":"2023-09-10T00:40:16.689409+0000","last_undegraded":"2023-09-10T00:40:16.689409+0000","last_fullsized":"2023-09-10T00:40:16.689409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:13:01.194358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.687211+0000","last_change":"2023-09-10T00:39:17.536853+0000","last_active":"2023-09-10T00:40:16.687211+0000","last_peered":"2023-09-10T00:40:16.687211+0000","last_clean":"2023-09-10T00:40:16.687211+0000","last_became_active":"2023-09-10T00:39:17.536287+0000","last_became_peered":"2023-09-10T00:39:17.536287+0000","last_unstale":"2023-09-10T00:40:16.687211+0000","last_undegraded":"2023-09-10T00:40:16.687211+0000","last_fullsized":"2023-09-10T00:40:16.687211+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:50:35.294055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681867+0000","last_change":"2023-09-10T00:38:38.764954+0000","last_active":"2023-09-10T00:40:16.681867+0000","last_peered":"2023-09-10T00:40:16.681867+0000","last_clean":"2023-09-10T00:40:16.681867+0000","last_became_active":"2023-09-10T00:38:38.764589+0000","last_became_peered":"2023-09-10T00:38:38.764589+0000","last_unstale":"2023-09-10T00:40:16.681867+0000","last_undegraded":"2023-09-10T00:40:16.681867+0000","last_fullsized":"2023-09-10T00:40:16.681867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:40:19.559285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684134+0000","last_change":"2023-09-10T00:38:39.766141+0000","last_active":"2023-09-10T00:40:16.684134+0000","last_peered":"2023-09-10T00:40:16.684134+0000","last_clean":"2023-09-10T00:40:16.684134+0000","last_became_active":"2023-09-10T00:38:39.765832+0000","last_became_peered":"2023-09-10T00:38:39.765832+0000","last_unstale":"2023-09-10T00:40:16.684134+0000","last_undegraded":"2023-09-10T00:40:16.684134+0000","last_fullsized":"2023-09-10T00:40:16.684134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:08:45.149973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686352+0000","last_change":"2023-09-10T00:38:43.785111+0000","last_active":"2023-09-10T00:40:16.686352+0000","last_peered":"2023-09-10T00:40:16.686352+0000","last_clean":"2023-09-10T00:40:16.686352+0000","last_became_active":"2023-09-10T00:38:43.784542+0000","last_became_peered":"2023-09-10T00:38:43.784542+0000","last_unstale":"2023-09-10T00:40:16.686352+0000","last_undegraded":"2023-09-10T00:40:16.686352+0000","last_fullsized":"2023-09-10T00:40:16.686352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:34:51.611608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674667+0000","last_change":"2023-09-10T00:38:38.759984+0000","last_active":"2023-09-10T00:40:16.674667+0000","last_peered":"2023-09-10T00:40:16.674667+0000","last_clean":"2023-09-10T00:40:16.674667+0000","last_became_active":"2023-09-10T00:38:38.759660+0000","last_became_peered":"2023-09-10T00:38:38.759660+0000","last_unstale":"2023-09-10T00:40:16.674667+0000","last_undegraded":"2023-09-10T00:40:16.674667+0000","last_fullsized":"2023-09-10T00:40:16.674667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:02:11.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.683063+0000","last_change":"2023-09-10T00:38:39.770765+0000","last_active":"2023-09-10T00:40:16.683063+0000","last_peered":"2023-09-10T00:40:16.683063+0000","last_clean":"2023-09-10T00:40:16.683063+0000","last_became_active":"2023-09-10T00:38:39.770483+0000","last_became_peered":"2023-09-10T00:38:39.770483+0000","last_unstale":"2023-09-10T00:40:16.683063+0000","last_undegraded":"2023-09-10T00:40:16.683063+0000","last_fullsized":"2023-09-10T00:40:16.683063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:57:28.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.687049+0000","last_change":"2023-09-10T00:38:43.790945+0000","last_active":"2023-09-10T00:40:16.687049+0000","last_peered":"2023-09-10T00:40:16.687049+0000","last_clean":"2023-09-10T00:40:16.687049+0000","last_became_active":"2023-09-10T00:38:43.790467+0000","last_became_peered":"2023-09-10T00:38:43.790467+0000","last_unstale":"2023-09-10T00:40:16.687049+0000","last_undegraded":"2023-09-10T00:40:16.687049+0000","last_fullsized":"2023-09-10T00:40:16.687049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:44:07.041302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.676240+0000","last_change":"2023-09-10T00:38:38.765912+0000","last_active":"2023-09-10T00:40:17.676240+0000","last_peered":"2023-09-10T00:40:17.676240+0000","last_clean":"2023-09-10T00:40:17.676240+0000","last_became_active":"2023-09-10T00:38:38.765586+0000","last_became_peered":"2023-09-10T00:38:38.765586+0000","last_unstale":"2023-09-10T00:40:17.676240+0000","last_undegraded":"2023-09-10T00:40:17.676240+0000","last_fullsized":"2023-09-10T00:40:17.676240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:25:05.974158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675205+0000","last_change":"2023-09-10T00:38:39.771462+0000","last_active":"2023-09-10T00:40:16.675205+0000","last_peered":"2023-09-10T00:40:16.675205+0000","last_clean":"2023-09-10T00:40:16.675205+0000","last_became_active":"2023-09-10T00:38:39.771153+0000","last_became_peered":"2023-09-10T00:38:39.771153+0000","last_unstale":"2023-09-10T00:40:16.675205+0000","last_undegraded":"2023-09-10T00:40:16.675205+0000","last_fullsized":"2023-09-10T00:40:16.675205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:03:35.312186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.687296+0000","last_change":"2023-09-10T00:38:43.838446+0000","last_active":"2023-09-10T00:40:16.687296+0000","last_peered":"2023-09-10T00:40:16.687296+0000","last_clean":"2023-09-10T00:40:16.687296+0000","last_became_active":"2023-09-10T00:38:43.838119+0000","last_became_peered":"2023-09-10T00:38:43.838119+0000","last_unstale":"2023-09-10T00:40:16.687296+0000","last_undegraded":"2023-09-10T00:40:16.687296+0000","last_fullsized":"2023-09-10T00:40:16.687296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:12:20.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:17.734911+0000","last_change":"2023-09-10T00:40:17.734911+0000","last_active":"2023-09-10T00:40:17.734911+0000","last_peered":"2023-09-10T00:40:17.734911+0000","last_clean":"2023-09-10T00:40:17.734911+0000","last_became_active":"2023-09-10T00:40:17.730465+0000","last_became_peered":"2023-09-10T00:40:17.730465+0000","last_unstale":"2023-09-10T00:40:17.734911+0000","last_undegraded":"2023-09-10T00:40:17.734911+0000","last_fullsized":"2023-09-10T00:40:17.734911+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:42:17.592032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684224+0000","last_change":"2023-09-10T00:38:39.771995+0000","last_active":"2023-09-10T00:40:16.684224+0000","last_peered":"2023-09-10T00:40:16.684224+0000","last_clean":"2023-09-10T00:40:16.684224+0000","last_became_active":"2023-09-10T00:38:39.771693+0000","last_became_peered":"2023-09-10T00:38:39.771693+0000","last_unstale":"2023-09-10T00:40:16.684224+0000","last_undegraded":"2023-09-10T00:40:16.684224+0000","last_fullsized":"2023-09-10T00:40:16.684224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:41:45.621060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.680916+0000","last_change":"2023-09-10T00:38:43.789159+0000","last_active":"2023-09-10T00:40:16.680916+0000","last_peered":"2023-09-10T00:40:16.680916+0000","last_clean":"2023-09-10T00:40:16.680916+0000","last_became_active":"2023-09-10T00:38:43.788675+0000","last_became_peered":"2023-09-10T00:38:43.788675+0000","last_unstale":"2023-09-10T00:40:16.680916+0000","last_undegraded":"2023-09-10T00:40:16.680916+0000","last_fullsized":"2023-09-10T00:40:16.680916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:30:39.545561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'6","reported_seq":110,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-10T00:40:21.647139+0000","last_change":"2023-09-10T00:38:38.767245+0000","last_active":"2023-09-10T00:40:21.647139+0000","last_peered":"2023-09-10T00:40:21.647139+0000","last_clean":"2023-09-10T00:40:21.647139+0000","last_became_active":"2023-09-10T00:38:38.766952+0000","last_became_peered":"2023-09-10T00:38:38.766952+0000","last_unstale":"2023-09-10T00:40:21.647139+0000","last_undegraded":"2023-09-10T00:40:21.647139+0000","last_fullsized":"2023-09-10T00:40:21.647139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:57:23.309838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1653,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.683345+0000","last_change":"2023-09-10T00:38:39.772199+0000","last_active":"2023-09-10T00:40:16.683345+0000","last_peered":"2023-09-10T00:40:16.683345+0000","last_clean":"2023-09-10T00:40:16.683345+0000","last_became_active":"2023-09-10T00:38:39.771917+0000","last_became_peered":"2023-09-10T00:38:39.771917+0000","last_unstale":"2023-09-10T00:40:16.683345+0000","last_undegraded":"2023-09-10T00:40:16.683345+0000","last_fullsized":"2023-09-10T00:40:16.683345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:00:52.824581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.688815+0000","last_change":"2023-09-10T00:38:43.797107+0000","last_active":"2023-09-10T00:40:16.688815+0000","last_peered":"2023-09-10T00:40:16.688815+0000","last_clean":"2023-09-10T00:40:16.688815+0000","last_became_active":"2023-09-10T00:38:43.796606+0000","last_became_peered":"2023-09-10T00:38:43.796606+0000","last_unstale":"2023-09-10T00:40:16.688815+0000","last_undegraded":"2023-09-10T00:40:16.688815+0000","last_fullsized":"2023-09-10T00:40:16.688815+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:17:00.444238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.676254+0000","last_change":"2023-09-10T00:38:38.768045+0000","last_active":"2023-09-10T00:40:16.676254+0000","last_peered":"2023-09-10T00:40:16.676254+0000","last_clean":"2023-09-10T00:40:16.676254+0000","last_became_active":"2023-09-10T00:38:38.767344+0000","last_became_peered":"2023-09-10T00:38:38.767344+0000","last_unstale":"2023-09-10T00:40:16.676254+0000","last_undegraded":"2023-09-10T00:40:16.676254+0000","last_fullsized":"2023-09-10T00:40:16.676254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:51:08.765412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.683838+0000","last_change":"2023-09-10T00:39:17.536408+0000","last_active":"2023-09-10T00:40:16.683838+0000","last_peered":"2023-09-10T00:40:16.683838+0000","last_clean":"2023-09-10T00:40:16.683838+0000","last_became_active":"2023-09-10T00:39:17.535980+0000","last_became_peered":"2023-09-10T00:39:17.535980+0000","last_unstale":"2023-09-10T00:40:16.683838+0000","last_undegraded":"2023-09-10T00:40:16.683838+0000","last_fullsized":"2023-09-10T00:40:16.683838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:08:10.517535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675080+0000","last_change":"2023-09-10T00:38:43.784928+0000","last_active":"2023-09-10T00:40:16.675080+0000","last_peered":"2023-09-10T00:40:16.675080+0000","last_clean":"2023-09-10T00:40:16.675080+0000","last_became_active":"2023-09-10T00:38:43.784479+0000","last_became_peered":"2023-09-10T00:38:43.784479+0000","last_unstale":"2023-09-10T00:40:16.675080+0000","last_undegraded":"2023-09-10T00:40:16.675080+0000","last_fullsized":"2023-09-10T00:40:16.675080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:42.759662+0000","last_clean_scrub_stamp":"2023-09-10T00:38:42.759662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:14:45.019988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674218+0000","last_change":"2023-09-10T00:38:38.770730+0000","last_active":"2023-09-10T00:40:16.674218+0000","last_peered":"2023-09-10T00:40:16.674218+0000","last_clean":"2023-09-10T00:40:16.674218+0000","last_became_active":"2023-09-10T00:38:38.770344+0000","last_became_peered":"2023-09-10T00:38:38.770344+0000","last_unstale":"2023-09-10T00:40:16.674218+0000","last_undegraded":"2023-09-10T00:40:16.674218+0000","last_fullsized":"2023-09-10T00:40:16.674218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:37.731579+0000","last_clean_scrub_stamp":"2023-09-10T00:38:37.731579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:49:22.881956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682628+0000","last_change":"2023-09-10T00:38:39.766149+0000","last_active":"2023-09-10T00:40:16.682628+0000","last_peered":"2023-09-10T00:40:16.682628+0000","last_clean":"2023-09-10T00:40:16.682628+0000","last_became_active":"2023-09-10T00:38:39.765773+0000","last_became_peered":"2023-09-10T00:38:39.765773+0000","last_unstale":"2023-09-10T00:40:16.682628+0000","last_undegraded":"2023-09-10T00:40:16.682628+0000","last_fullsized":"2023-09-10T00:40:16.682628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:00:37.854168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682825+0000","last_change":"2023-09-10T00:38:39.765197+0000","last_active":"2023-09-10T00:40:16.682825+0000","last_peered":"2023-09-10T00:40:16.682825+0000","last_clean":"2023-09-10T00:40:16.682825+0000","last_became_active":"2023-09-10T00:38:39.764891+0000","last_became_peered":"2023-09-10T00:38:39.764891+0000","last_unstale":"2023-09-10T00:40:16.682825+0000","last_undegraded":"2023-09-10T00:40:16.682825+0000","last_fullsized":"2023-09-10T00:40:16.682825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:47:27.429355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.686326+0000","last_change":"2023-09-10T00:38:39.764722+0000","last_active":"2023-09-10T00:40:16.686326+0000","last_peered":"2023-09-10T00:40:16.686326+0000","last_clean":"2023-09-10T00:40:16.686326+0000","last_became_active":"2023-09-10T00:38:39.764376+0000","last_became_peered":"2023-09-10T00:38:39.764376+0000","last_unstale":"2023-09-10T00:40:16.686326+0000","last_undegraded":"2023-09-10T00:40:16.686326+0000","last_fullsized":"2023-09-10T00:40:16.686326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:27:36.708573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.679566+0000","last_change":"2023-09-10T00:38:39.771997+0000","last_active":"2023-09-10T00:40:16.679566+0000","last_peered":"2023-09-10T00:40:16.679566+0000","last_clean":"2023-09-10T00:40:16.679566+0000","last_became_active":"2023-09-10T00:38:39.771588+0000","last_became_peered":"2023-09-10T00:38:39.771588+0000","last_unstale":"2023-09-10T00:40:16.679566+0000","last_undegraded":"2023-09-10T00:40:16.679566+0000","last_fullsized":"2023-09-10T00:40:16.679566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:20:07.171333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.689101+0000","last_change":"2023-09-10T00:38:39.773393+0000","last_active":"2023-09-10T00:40:16.689101+0000","last_peered":"2023-09-10T00:40:16.689101+0000","last_clean":"2023-09-10T00:40:16.689101+0000","last_became_active":"2023-09-10T00:38:39.772972+0000","last_became_peered":"2023-09-10T00:38:39.772972+0000","last_unstale":"2023-09-10T00:40:16.689101+0000","last_undegraded":"2023-09-10T00:40:16.689101+0000","last_fullsized":"2023-09-10T00:40:16.689101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:27:29.916970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674401+0000","last_change":"2023-09-10T00:38:39.767871+0000","last_active":"2023-09-10T00:40:16.674401+0000","last_peered":"2023-09-10T00:40:16.674401+0000","last_clean":"2023-09-10T00:40:16.674401+0000","last_became_active":"2023-09-10T00:38:39.767563+0000","last_became_peered":"2023-09-10T00:38:39.767563+0000","last_unstale":"2023-09-10T00:40:16.674401+0000","last_undegraded":"2023-09-10T00:40:16.674401+0000","last_fullsized":"2023-09-10T00:40:16.674401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:22:30.400441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682225+0000","last_change":"2023-09-10T00:38:39.764664+0000","last_active":"2023-09-10T00:40:16.682225+0000","last_peered":"2023-09-10T00:40:16.682225+0000","last_clean":"2023-09-10T00:40:16.682225+0000","last_became_active":"2023-09-10T00:38:39.764364+0000","last_became_peered":"2023-09-10T00:38:39.764364+0000","last_unstale":"2023-09-10T00:40:16.682225+0000","last_undegraded":"2023-09-10T00:40:16.682225+0000","last_fullsized":"2023-09-10T00:40:16.682225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:41.656472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.685830+0000","last_change":"2023-09-10T00:38:39.770315+0000","last_active":"2023-09-10T00:40:16.685830+0000","last_peered":"2023-09-10T00:40:16.685830+0000","last_clean":"2023-09-10T00:40:16.685830+0000","last_became_active":"2023-09-10T00:38:39.769977+0000","last_became_peered":"2023-09-10T00:38:39.769977+0000","last_unstale":"2023-09-10T00:40:16.685830+0000","last_undegraded":"2023-09-10T00:40:16.685830+0000","last_fullsized":"2023-09-10T00:40:16.685830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:30:13.380487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.675805+0000","last_change":"2023-09-10T00:38:39.768734+0000","last_active":"2023-09-10T00:40:16.675805+0000","last_peered":"2023-09-10T00:40:16.675805+0000","last_clean":"2023-09-10T00:40:16.675805+0000","last_became_active":"2023-09-10T00:38:39.768383+0000","last_became_peered":"2023-09-10T00:38:39.768383+0000","last_unstale":"2023-09-10T00:40:16.675805+0000","last_undegraded":"2023-09-10T00:40:16.675805+0000","last_fullsized":"2023-09-10T00:40:16.675805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:30:09.974447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.684233+0000","last_change":"2023-09-10T00:39:17.732561+0000","last_active":"2023-09-10T00:40:16.684233+0000","last_peered":"2023-09-10T00:40:16.684233+0000","last_clean":"2023-09-10T00:40:16.684233+0000","last_became_active":"2023-09-10T00:39:17.732138+0000","last_became_peered":"2023-09-10T00:39:17.732138+0000","last_unstale":"2023-09-10T00:40:16.684233+0000","last_undegraded":"2023-09-10T00:40:16.684233+0000","last_fullsized":"2023-09-10T00:40:16.684233+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:11:58.866881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674781+0000","last_change":"2023-09-10T00:38:39.773839+0000","last_active":"2023-09-10T00:40:16.674781+0000","last_peered":"2023-09-10T00:40:16.674781+0000","last_clean":"2023-09-10T00:40:16.674781+0000","last_became_active":"2023-09-10T00:38:39.773521+0000","last_became_peered":"2023-09-10T00:38:39.773521+0000","last_unstale":"2023-09-10T00:40:16.674781+0000","last_undegraded":"2023-09-10T00:40:16.674781+0000","last_fullsized":"2023-09-10T00:40:16.674781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:15:49.033374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.682785+0000","last_change":"2023-09-10T00:38:39.775721+0000","last_active":"2023-09-10T00:40:16.682785+0000","last_peered":"2023-09-10T00:40:16.682785+0000","last_clean":"2023-09-10T00:40:16.682785+0000","last_became_active":"2023-09-10T00:38:39.775424+0000","last_became_peered":"2023-09-10T00:38:39.775424+0000","last_unstale":"2023-09-10T00:40:16.682785+0000","last_undegraded":"2023-09-10T00:40:16.682785+0000","last_fullsized":"2023-09-10T00:40:16.682785+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:56:26.079332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.674146+0000","last_change":"2023-09-10T00:38:39.768767+0000","last_active":"2023-09-10T00:40:16.674146+0000","last_peered":"2023-09-10T00:40:16.674146+0000","last_clean":"2023-09-10T00:40:16.674146+0000","last_became_active":"2023-09-10T00:38:39.768452+0000","last_became_peered":"2023-09-10T00:38:39.768452+0000","last_unstale":"2023-09-10T00:40:16.674146+0000","last_undegraded":"2023-09-10T00:40:16.674146+0000","last_fullsized":"2023-09-10T00:40:16.674146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:31:53.578574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-10T00:40:16.681759+0000","last_change":"2023-09-10T00:38:39.761998+0000","last_active":"2023-09-10T00:40:16.681759+0000","last_peered":"2023-09-10T00:40:16.681759+0000","last_clean":"2023-09-10T00:40:16.681759+0000","last_became_active":"2023-09-10T00:38:39.761672+0000","last_became_peered":"2023-09-10T00:38:39.761672+0000","last_unstale":"2023-09-10T00:40:16.681759+0000","last_undegraded":"2023-09-10T00:40:16.681759+0000","last_fullsized":"2023-09-10T00:40:16.681759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-10T00:38:38.738053+0000","last_clean_scrub_stamp":"2023-09-10T00:38:38.738053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T12:03:24.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73067,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":880,"num_read_kb":526,"num_write":255,"num_write_kb":153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":58848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":210561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449120,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":510852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.3150000000000004,"5min":4.3150000000000004,"15min":4.3150000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.54700000000000004,"15min":0.50800000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":6.3470000000000004,"5min":6.3470000000000004,"15min":6.3470000000000004},"last":0.41199999999999998}]},{"osd":1,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.499,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81899999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.64100000000000001},"min":{"1min":0.435,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.60499999999999998}]},{"osd":2,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48699999999999999,"15min":0.48799999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.47999999999999998,"15min":0.47699999999999998},"min":{"1min":0.39100000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":4,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67100000000000004,"15min":0.66900000000000004},"min":{"1min":0.51700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.83999999999999997,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.65200000000000002}]},{"osd":5,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77600000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.434}]},{"osd":6,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.747,"15min":0.71999999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":6.4260000000000002,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.79900000000000004,"15min":0.76800000000000002},"min":{"1min":0.50900000000000001,"5min":0.498,"15min":0.498},"max":{"1min":6.4720000000000004,"5min":6.4720000000000004,"15min":6.4720000000000004},"last":0.67300000000000004}]},{"osd":7,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76500000000000001,"15min":0.77000000000000002},"min":{"1min":0.54500000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89900000000000002,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.755,"15min":0.75800000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90900000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.75600000000000001}]},{"osd":8,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71199999999999997}]},{"osd":9,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.81299999999999994,"15min":0.78900000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":4.4859999999999998,"5min":4.4859999999999998,"15min":4.4859999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.75600000000000001,"15min":0.748},"min":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.622}]},{"osd":10,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.57599999999999996,"5min":0.5,"15min":0.5},"max":{"1min":0.97899999999999998,"5min":1.486,"15min":1.486},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81499999999999995,"15min":0.81699999999999995},"min":{"1min":0.56699999999999995,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96799999999999997,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.78900000000000003}]}]},{"osd":10,"up_from":70,"seq":300647710757,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28708,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716444,"statfs":{"total":95995035648,"available":95965638656,"internally_reserved":0,"allocated":1732608,"data_stored":1161850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.502}]},{"osd":1,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51500000000000001,"15min":0.51900000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78200000000000003,"5min":0.873,"15min":0.873},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.36499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52900000000000003}]},{"osd":2,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.629,"15min":0.629},"min":{"1min":0.501,"5min":0.439,"15min":0.439},"max":{"1min":0.75800000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65400000000000003,"15min":0.64400000000000002},"min":{"1min":0.52400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61699999999999999}]},{"osd":3,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.61899999999999999,"15min":0.63700000000000001},"min":{"1min":0.39300000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70099999999999996,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.48899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.56899999999999995}]},{"osd":4,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.54600000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.624},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78100000000000003,"15min":0.78300000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.94299999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65400000000000003}]},{"osd":6,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.995,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.59999999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69399999999999995}]},{"osd":7,"last update":"Sun Sep 10 00:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71299999999999997}]},{"osd":8,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77700000000000002,"15min":0.77500000000000002},"min":{"1min":0.63900000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78600000000000003,"15min":0.78800000000000003},"min":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.70399999999999996}]},{"osd":9,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.86099999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55600000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.72799999999999998}]},{"osd":11,"last update":"Sun Sep 10 00:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80400000000000005,"15min":0.80200000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.71699999999999997}]}]},{"osd":9,"up_from":65,"seq":279172874282,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":514395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42499999999999999,"15min":0.44400000000000001},"min":{"1min":0.32300000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.504,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.42799999999999999,"15min":0.45600000000000002},"min":{"1min":0.28999999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.55500000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.59699999999999998,"15min":0.61299999999999999},"min":{"1min":0.43099999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81100000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70099999999999996,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59899999999999998}]},{"osd":2,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63600000000000001,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.74199999999999999,"5min":0.879,"15min":0.879},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77300000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.51200000000000001}]},{"osd":3,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.52900000000000003,"15min":0.56899999999999995},"min":{"1min":0.29899999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.74399999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.442,"5min":0.56899999999999995,"15min":0.61699999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72499999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42199999999999999}]},{"osd":4,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.60599999999999998,"15min":0.63200000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80500000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.66400000000000003,"15min":0.68799999999999994},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75900000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.621}]},{"osd":5,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72699999999999998,"15min":0.747},"min":{"1min":0.46000000000000002,"5min":0.435,"15min":0.435},"max":{"1min":3.3109999999999999,"5min":3.3109999999999999,"15min":3.3109999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.72899999999999998,"15min":0.76700000000000002},"min":{"1min":0.47399999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.78100000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.57399999999999995}]},{"osd":6,"last update":"Sun Sep 10 00:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.42499999999999999}]},{"osd":7,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.751,"15min":0.79500000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.79500000000000004,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.76800000000000002,"15min":0.81799999999999995},"min":{"1min":0.499,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.89200000000000002,"5min":1.071,"15min":1.071},"last":0.66300000000000003}]},{"osd":8,"last update":"Sun Sep 10 00:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73599999999999999,"15min":0.78600000000000003},"min":{"1min":0.47799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79000000000000004,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.79800000000000004,"15min":0.84799999999999998},"min":{"1min":0.53400000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88900000000000001,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.78600000000000003}]},{"osd":10,"last update":"Sun Sep 10 00:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.72899999999999998,"15min":0.749},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.77600000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.54000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.82799999999999996,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.76700000000000002}]},{"osd":11,"last update":"Sun Sep 10 00:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.69899999999999995}]}]},{"osd":8,"up_from":57,"seq":244813135918,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":510346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76100000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64400000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61499999999999999,"15min":0.59699999999999998},"min":{"1min":0.49299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.874,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.69899999999999995,"15min":0.63600000000000001},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93999999999999995,"5min":6.5759999999999996,"15min":6.5759999999999996},"last":0.78900000000000003}]},{"osd":2,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.31,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.49399999999999999,"15min":0.48499999999999999},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59799999999999998}]},{"osd":3,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.85199999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.871,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.82999999999999996}]},{"osd":4,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.61399999999999999,"15min":0.59199999999999997},"min":{"1min":0.48299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.63600000000000001,"15min":0.61699999999999999},"min":{"1min":0.53700000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.76500000000000001}]},{"osd":5,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90000000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.74099999999999999}]},{"osd":6,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79800000000000004,"15min":0.755},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.026,"5min":6.5940000000000003,"15min":6.5940000000000003},"last":0.93799999999999994}]},{"osd":7,"last update":"Sun Sep 10 00:39:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79000000000000004,"15min":0.71999999999999997},"min":{"1min":0.58799999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0920000000000001,"5min":6.5439999999999996,"15min":6.5439999999999996},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.69199999999999995,"15min":0.67400000000000004},"min":{"1min":0.60399999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":1.028}]},{"osd":9,"last update":"Sun Sep 10 00:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.75700000000000001,"15min":0.73799999999999999},"min":{"1min":0.65700000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.002,"5min":1.103,"15min":1.103},"last":1.002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.76100000000000001,"15min":0.73099999999999998},"min":{"1min":0.70099999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.038,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":1.0149999999999999}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.57199999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.73999999999999999,"15min":0.71299999999999997},"min":{"1min":0.64200000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.95599999999999996}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.92000000000000004}]}]},{"osd":7,"up_from":50,"seq":214748364848,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1650688,"data_stored":1107514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54100000000000004,"15min":0.53200000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.39800000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40500000000000003,"15min":0.40000000000000002},"min":{"1min":0.32800000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.52600000000000002,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.40999999999999998,"15min":0.40200000000000002},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63600000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.38700000000000001}]},{"osd":2,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.79800000000000004,"5min":1.091,"15min":1.091},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.51400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85999999999999999,"5min":1.069,"15min":1.069},"last":0.81899999999999995}]},{"osd":3,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.36699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76900000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.46999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54900000000000004}]},{"osd":4,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.38300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.66300000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.497,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.59999999999999998,"5min":1.056,"15min":1.056},"last":0.47899999999999998}]},{"osd":5,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":0.628},"min":{"1min":0.49399999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.83299999999999996,"5min":1.026,"15min":1.026},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.48099999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98399999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.74199999999999999}]},{"osd":6,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.48899999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.84999999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89800000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.42399999999999999}]},{"osd":8,"last update":"Sun Sep 10 00:39:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.65200000000000002,"15min":0.626},"min":{"1min":0.44400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.86399999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67800000000000005,"15min":0.65700000000000003},"min":{"1min":0.496,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.86499999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72599999999999998}]},{"osd":9,"last update":"Sun Sep 10 00:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.53500000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80000000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69299999999999995,"15min":0.69599999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.83599999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.70999999999999996}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65700000000000003,"15min":0.65200000000000002},"min":{"1min":0.46100000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83299999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66600000000000004,"15min":0.65700000000000003},"min":{"1min":0.54000000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.64800000000000002}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77300000000000002}]}]},{"osd":6,"up_from":45,"seq":193273528374,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":506286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41699999999999998,"15min":0.40100000000000002},"min":{"1min":0.34399999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.63700000000000001,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.40799999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59699999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.34499999999999997}]},{"osd":1,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59799999999999998,"15min":0.56399999999999995},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82999999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81499999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.73799999999999999}]},{"osd":2,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69499999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.627,"15min":0.61099999999999999},"min":{"1min":0.39500000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80100000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.50600000000000001}]},{"osd":3,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48699999999999999,"15min":0.47899999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66400000000000003,"5min":0.875,"15min":0.875},"last":0.37},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68799999999999994,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40400000000000003}]},{"osd":4,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.46800000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82199999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69099999999999995,"15min":0.68000000000000005},"min":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93400000000000005,"5min":1.024,"15min":1.024},"last":0.69699999999999995}]},{"osd":5,"last update":"Sun Sep 10 00:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68000000000000005,"15min":0.67600000000000005},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.879,"5min":1.048,"15min":1.048},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.69899999999999995,"15min":0.69399999999999995},"min":{"1min":0.48699999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.48699999999999999}]},{"osd":7,"last update":"Sun Sep 10 00:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.73199999999999998,"15min":0.72899999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94899999999999995,"5min":1.069,"15min":1.069},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72999999999999998,"15min":0.72299999999999998},"min":{"1min":0.47799999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.82799999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.78200000000000003}]},{"osd":8,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.79100000000000004,"15min":0.82499999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.876,"5min":1.27,"15min":1.27},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.79500000000000004,"15min":0.82099999999999995},"min":{"1min":0.55600000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.92800000000000005,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.625}]},{"osd":9,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.78200000000000003,"15min":0.84399999999999997},"min":{"1min":0.438,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94399999999999995,"5min":1.389,"15min":1.389},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.72499999999999998,"15min":0.77000000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86199999999999999,"5min":1.099,"15min":1.099},"last":0.63600000000000001}]},{"osd":10,"last update":"Sun Sep 10 00:39:37 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.82599999999999996,"15min":0.83899999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.96299999999999997,"5min":1.181,"15min":1.181},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.91300000000000003,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.79900000000000004}]},{"osd":11,"last update":"Sun Sep 10 00:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":566647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52600000000000002,"15min":0.50900000000000001},"min":{"1min":0.32100000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72199999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.53200000000000003,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53500000000000003}]},{"osd":2,"last update":"Sun Sep 10 00:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53900000000000003,"15min":0.52200000000000002},"min":{"1min":0.308,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.58799999999999997,"15min":0.54500000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84599999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.86799999999999999}]},{"osd":3,"last update":"Sun Sep 10 00:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.70399999999999996,"5min":0.879,"15min":0.879},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53100000000000003,"15min":0.52600000000000002},"min":{"1min":0.38,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69599999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56100000000000005}]},{"osd":4,"last update":"Sun Sep 10 00:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.42299999999999999},"min":{"1min":0.35099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.46000000000000002,"15min":0.437},"min":{"1min":0.33800000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58299999999999996}]},{"osd":5,"last update":"Sun Sep 10 00:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60399999999999998,"15min":0.57299999999999995},"min":{"1min":0.38700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89400000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.63500000000000001},"min":{"1min":0.45700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0409999999999999,"5min":1.044,"15min":1.044},"last":0.81399999999999995}]},{"osd":6,"last update":"Sun Sep 10 00:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63900000000000001,"15min":0.61299999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86199999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63900000000000001,"15min":0.61499999999999999},"min":{"1min":0.40400000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83699999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68100000000000005}]},{"osd":7,"last update":"Sun Sep 10 00:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61399999999999999,"15min":0.57399999999999995},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89200000000000002,"5min":0.999,"15min":0.999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.621,"15min":0.58299999999999996},"min":{"1min":0.54300000000000004,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.878,"5min":0.996,"15min":0.996},"last":0.72799999999999998}]},{"osd":8,"last update":"Sun Sep 10 00:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94499999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88200000000000001,"5min":1.014,"15min":1.014},"last":0.88400000000000001}]},{"osd":9,"last update":"Sun Sep 10 00:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.998,"5min":1.097,"15min":1.097},"last":0.75},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77300000000000002,"15min":0.78600000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.93000000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65900000000000003}]},{"osd":10,"last update":"Sun Sep 10 00:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.76000000000000001,"15min":0.77200000000000002},"min":{"1min":0.46700000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93600000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.86199999999999999,"15min":0.86799999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0109999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.90800000000000003}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.93400000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":507742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 10 00:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54400000000000004,"15min":0.5},"min":{"1min":0.36399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.629},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.52500000000000002},"min":{"1min":0.376,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.64700000000000002}]},{"osd":2,"last update":"Sun Sep 10 00:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.53800000000000003},"min":{"1min":0.39400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.56899999999999995},"min":{"1min":0.372,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83999999999999997,"5min":1.621,"15min":1.621},"last":0.78100000000000003}]},{"osd":3,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.375},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.39900000000000002}]},{"osd":4,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.46300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73699999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.374,"15min":0.374},"max":{"1min":0.83999999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59499999999999997}]},{"osd":5,"last update":"Sun Sep 10 00:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65700000000000003,"15min":0.61599999999999999},"min":{"1min":0.42799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.91000000000000003,"5min":1.075,"15min":1.075},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64700000000000002,"15min":0.63700000000000001},"min":{"1min":0.47799999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85399999999999998,"5min":1.123,"15min":1.123},"last":0.86499999999999999}]},{"osd":6,"last update":"Sun Sep 10 00:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53900000000000003,"15min":0.52000000000000002},"min":{"1min":0.34000000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.42899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.97999999999999998,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.52300000000000002}]},{"osd":7,"last update":"Sun Sep 10 00:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.72199999999999998}]},{"osd":8,"last update":"Sun Sep 10 00:39:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76900000000000002,"15min":0.77800000000000002},"min":{"1min":0.60499999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.022,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":0.74199999999999999},"min":{"1min":0.49099999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97199999999999998,"5min":1.151,"15min":1.151},"last":0.88500000000000001}]},{"osd":9,"last update":"Sun Sep 10 00:40:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73899999999999999,"15min":0.752},"min":{"1min":0.54800000000000004,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0549999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.69399999999999995,"15min":0.71299999999999997},"min":{"1min":0.44800000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90300000000000002,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.54400000000000004}]},{"osd":10,"last update":"Sun Sep 10 00:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.746,"15min":0.74299999999999999},"min":{"1min":0.57999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.53400000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.90600000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.749}]},{"osd":11,"last update":"Sun Sep 10 00:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75,"15min":0.749},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.67400000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.085,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.81699999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":508550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.51300000000000001},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52700000000000002,"15min":0.52500000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72699999999999998,"5min":0.753,"15min":0.753},"last":0.64000000000000001}]},{"osd":1,"last update":"Sun Sep 10 00:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.58599999999999997,"15min":0.53900000000000003},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.58999999999999997,"15min":0.55000000000000004},"min":{"1min":0.34200000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.75700000000000001}]},{"osd":3,"last update":"Sun Sep 10 00:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57599999999999996,"15min":0.54000000000000004},"min":{"1min":0.43099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.501}]},{"osd":4,"last update":"Sun Sep 10 00:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.77700000000000002}]},{"osd":5,"last update":"Sun Sep 10 00:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.499,"15min":0.44900000000000001},"min":{"1min":0.41399999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.55800000000000005,"15min":0.51200000000000001},"min":{"1min":0.46200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.71699999999999997}]},{"osd":6,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.45800000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.97599999999999998,"5min":1.018,"15min":1.018},"last":0.621},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67500000000000004,"15min":0.64700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96499999999999997,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.68799999999999994}]},{"osd":7,"last update":"Sun Sep 10 00:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.80700000000000005}]},{"osd":8,"last update":"Sun Sep 10 00:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.58999999999999997,"15min":0.55600000000000005},"min":{"1min":0.45400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.63300000000000001,"15min":0.60699999999999998},"min":{"1min":0.47899999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.53600000000000003}]},{"osd":9,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76200000000000001,"15min":0.74099999999999999},"min":{"1min":0.60999999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0429999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.73699999999999999,"15min":0.71499999999999997},"min":{"1min":0.56999999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98799999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.79000000000000004}]},{"osd":10,"last update":"Sun Sep 10 00:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.70799999999999996,"15min":0.68500000000000005},"min":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.745,"15min":0.71899999999999997},"min":{"1min":0.68700000000000006,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.875}]},{"osd":11,"last update":"Sun Sep 10 00:39:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.83199999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":505511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40899999999999997,"15min":0.40100000000000002},"min":{"1min":0.26700000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.92000000000000004,"5min":1.409,"15min":1.409},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.42799999999999999,"15min":0.42099999999999999},"min":{"1min":0.31900000000000001,"5min":0.255,"15min":0.255},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.51400000000000001}]},{"osd":1,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.6699999999999999,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58699999999999997,"15min":0.55600000000000005},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78500000000000003,"5min":1.581,"15min":1.581},"last":0.746}]},{"osd":2,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.60399999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.65400000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.7110000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.79200000000000004}]},{"osd":4,"last update":"Sun Sep 10 00:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.39300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80600000000000005,"5min":1.536,"15min":1.536},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.66500000000000004,"15min":0.68799999999999994},"min":{"1min":0.38300000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.748,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.68899999999999995}]},{"osd":5,"last update":"Sun Sep 10 00:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.73299999999999998},"min":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7809999999999999,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68100000000000005,"15min":0.67600000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.872}]},{"osd":6,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60099999999999998,"15min":0.57999999999999996},"min":{"1min":0.36499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.65600000000000003}]},{"osd":7,"last update":"Sun Sep 10 00:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.74399999999999999,"15min":0.745},"min":{"1min":0.47699999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90800000000000003,"5min":1.022,"15min":1.022},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73799999999999999,"15min":0.73599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.82499999999999996}]},{"osd":8,"last update":"Sun Sep 10 00:37:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.89400000000000002}]},{"osd":9,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71399999999999997,"15min":0.75800000000000001},"min":{"1min":0.36499999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.77900000000000003,"15min":0.82499999999999996},"min":{"1min":0.47599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95899999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.84399999999999997}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.82999999999999996,"15min":0.83999999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.90600000000000003}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.95299999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":570222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45000000000000001}]},{"osd":1,"last update":"Sun Sep 10 00:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.37,"15min":0.34399999999999997},"min":{"1min":0.29199999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.317,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.56000000000000005}]},{"osd":2,"last update":"Sun Sep 10 00:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80200000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58799999999999997,"15min":0.55500000000000005},"min":{"1min":0.497,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.68200000000000005}]},{"osd":3,"last update":"Sun Sep 10 00:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.39900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.60999999999999999,"15min":0.56200000000000006},"min":{"1min":0.35099999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.59799999999999998}]},{"osd":5,"last update":"Sun Sep 10 00:39:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85299999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.59499999999999997},"min":{"1min":0.49399999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63400000000000001}]},{"osd":6,"last update":"Sun Sep 10 00:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.61599999999999999,"15min":0.57999999999999996},"min":{"1min":0.50700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64100000000000001,"15min":0.63800000000000001},"min":{"1min":0.40799999999999997,"5min":0.375,"15min":0.375},"max":{"1min":0.91300000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72599999999999998}]},{"osd":7,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.39300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.88500000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.56499999999999995,"15min":0.52800000000000002},"min":{"1min":0.38700000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85199999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.877}]},{"osd":8,"last update":"Sun Sep 10 00:39:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.44800000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91200000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.60499999999999998,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97699999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.93100000000000005}]},{"osd":9,"last update":"Sun Sep 10 00:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.73199999999999998,"15min":0.70599999999999996},"min":{"1min":0.63200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.74299999999999999,"15min":0.73699999999999999},"min":{"1min":0.65600000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":1.0760000000000001}]},{"osd":10,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74399999999999999,"15min":0.75},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94999999999999996,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.74099999999999999,"15min":0.73699999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.94499999999999995}]},{"osd":11,"last update":"Sun Sep 10 00:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.97599999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1662976,"data_stored":1103379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.37,"5min":0.315,"15min":0.315},"max":{"1min":0.746,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.376,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74399999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52400000000000002}]},{"osd":1,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.52500000000000002},"min":{"1min":0.51800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.46000000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.747}]},{"osd":2,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.36899999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.60999999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.373},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.45900000000000002,"15min":0.437},"min":{"1min":0.38600000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.39600000000000002}]},{"osd":3,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.371,"5min":0.31,"15min":0.31},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.504},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.48899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.91100000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46999999999999997}]},{"osd":4,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.59999999999999998,"15min":0.56000000000000005},"min":{"1min":0.53200000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92900000000000005,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58799999999999997,"15min":0.55900000000000005},"min":{"1min":0.47999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78100000000000003,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57599999999999996}]},{"osd":6,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63300000000000001,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.9100000000000001,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":0.442},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.68400000000000005,"15min":0.66100000000000003},"min":{"1min":0.53600000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Sun Sep 10 00:39:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.629},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.64100000000000001,"15min":0.622},"min":{"1min":0.53700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.98399999999999999,"5min":1.046,"15min":1.046},"last":0.61499999999999999}]},{"osd":8,"last update":"Sun Sep 10 00:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.60499999999999998,"15min":0.56499999999999995},"min":{"1min":0.56699999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.56799999999999995,"15min":0.53200000000000003},"min":{"1min":0.48899999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.68400000000000005}]},{"osd":9,"last update":"Sun Sep 10 00:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.75900000000000001,"15min":0.72299999999999998},"min":{"1min":0.59999999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.9590000000000001,"5min":2.9590000000000001,"15min":2.9590000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.77200000000000002,"15min":0.72399999999999998},"min":{"1min":0.63100000000000001,"5min":0.435,"15min":0.435},"max":{"1min":2.9129999999999998,"5min":2.9129999999999998,"15min":2.9129999999999998},"last":0.77900000000000003}]},{"osd":10,"last update":"Sun Sep 10 00:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.69699999999999995,"15min":0.68300000000000005},"min":{"1min":0.54600000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.02,"5min":1.026,"15min":1.026},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.58599999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.76600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-10T00:40:27.018 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:40:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:26 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/4012451869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:40:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:26 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/4012451869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:40:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.172+0000 7f42f2ee7700 1 -- 172.21.15.44:0/2572513461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ec0ff4e0 msgr2=0x7f42ec0ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.172+0000 7f42f2ee7700 1 --2- 172.21.15.44:0/2572513461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ec0ff4e0 0x7f42ec0ff900 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f42e00099c0 tx=0x7f42e0009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.172+0000 7f42f2ee7700 1 -- 172.21.15.44:0/2572513461 shutdown_connections 2023-09-10T00:40:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.172+0000 7f42f2ee7700 1 --2- 172.21.15.44:0/2572513461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42ec101070 0x7f42ec107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.172+0000 7f42f2ee7700 1 --2- 172.21.15.44:0/2572513461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ec0ff4e0 0x7f42ec0ff900 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.172+0000 7f42f2ee7700 1 --2- 172.21.15.44:0/2572513461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42ec1006d0 0x7f42ec100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.172+0000 7f42f2ee7700 1 -- 172.21.15.44:0/2572513461 >> 172.21.15.44:0/2572513461 conn(0x7f42ec0faa30 msgr2=0x7f42ec0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.172+0000 7f42f2ee7700 1 -- 172.21.15.44:0/2572513461 shutdown_connections 2023-09-10T00:40:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.173+0000 7f42f2ee7700 1 -- 172.21.15.44:0/2572513461 wait complete. 2023-09-10T00:40:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.173+0000 7f42f2ee7700 1 Processor -- start 2023-09-10T00:40:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.173+0000 7f42f2ee7700 1 -- start start 2023-09-10T00:40:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f2ee7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ec0ff4e0 0x7f42ec196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:27.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f2ee7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42ec1006d0 0x7f42ec1967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:27.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f2ee7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42ec101070 0x7f42ec19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:27.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f2ee7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec19de00 con 0x7f42ec0ff4e0 2023-09-10T00:40:27.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f2ee7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec19df40 con 0x7f42ec101070 2023-09-10T00:40:27.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f2ee7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec19e0d0 con 0x7f42ec1006d0 2023-09-10T00:40:27.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f0c83700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ec0ff4e0 0x7f42ec196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:27.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f0c83700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ec0ff4e0 0x7f42ec196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48440/0 (socket says 172.21.15.44:48440) 2023-09-10T00:40:27.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f0c83700 1 -- 172.21.15.44:0/2611093619 learned_addr learned my addr 172.21.15.44:0/2611093619 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:27.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42ebfff700 1 --2- 172.21.15.44:0/2611093619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42ec1006d0 0x7f42ec1967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:27.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f1484700 1 --2- 172.21.15.44:0/2611093619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42ec101070 0x7f42ec19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f0c83700 1 -- 172.21.15.44:0/2611093619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42ec1006d0 msgr2=0x7f42ec1967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f0c83700 1 --2- 172.21.15.44:0/2611093619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42ec1006d0 0x7f42ec1967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f0c83700 1 -- 172.21.15.44:0/2611093619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42ec101070 msgr2=0x7f42ec19d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f0c83700 1 --2- 172.21.15.44:0/2611093619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42ec101070 0x7f42ec19d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f0c83700 1 -- 172.21.15.44:0/2611093619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e00095f0 con 0x7f42ec0ff4e0 2023-09-10T00:40:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f0c83700 1 --2- 172.21.15.44:0/2611093619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ec0ff4e0 0x7f42ec196280 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009fd0 tx=0x7f42e000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:27.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42e9ffb700 1 -- 172.21.15.44:0/2611093619 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e001d070 con 0x7f42ec0ff4e0 2023-09-10T00:40:27.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42e9ffb700 1 -- 172.21.15.44:0/2611093619 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f42e00059a0 con 0x7f42ec0ff4e0 2023-09-10T00:40:27.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42f2ee7700 1 -- 172.21.15.44:0/2611093619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ec19e350 con 0x7f42ec0ff4e0 2023-09-10T00:40:27.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.174+0000 7f42e9ffb700 1 -- 172.21.15.44:0/2611093619 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e00173f0 con 0x7f42ec0ff4e0 2023-09-10T00:40:27.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.175+0000 7f42f2ee7700 1 -- 172.21.15.44:0/2611093619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ec19e800 con 0x7f42ec0ff4e0 2023-09-10T00:40:27.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.176+0000 7f42f2ee7700 1 -- 172.21.15.44:0/2611093619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ec106bd0 con 0x7f42ec0ff4e0 2023-09-10T00:40:27.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.177+0000 7f42e9ffb700 1 -- 172.21.15.44:0/2611093619 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f42e0022470 con 0x7f42ec0ff4e0 2023-09-10T00:40:27.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.177+0000 7f42e9ffb700 1 --2- 172.21.15.44:0/2611093619 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42dc0a0690 0x7f42dc0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:27.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.177+0000 7f42e9ffb700 1 -- 172.21.15.44:0/2611093619 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f42e00cd2e0 con 0x7f42ec0ff4e0 2023-09-10T00:40:27.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.178+0000 7f42ebfff700 1 --2- 172.21.15.44:0/2611093619 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42dc0a0690 0x7f42dc0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:27.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.178+0000 7f42ebfff700 1 --2- 172.21.15.44:0/2611093619 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42dc0a0690 0x7f42dc0a2b50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009c60 tx=0x7f42d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:27.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.181+0000 7f42e9ffb700 1 -- 172.21.15.44:0/2611093619 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f42e0096a80 con 0x7f42ec0ff4e0 2023-09-10T00:40:27.383 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.383+0000 7f42f2ee7700 1 -- 172.21.15.44:0/2611093619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42ec10bef0 con 0x7f42ec0ff4e0 2023-09-10T00:40:27.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.384+0000 7f42e9ffb700 1 -- 172.21.15.44:0/2611093619 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24678 (secure 0 0 0) 0x7f42e009a0a0 con 0x7f42ec0ff4e0 2023-09-10T00:40:27.385 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:27.385 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":91,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:40:17.671364+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:40:27.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.386+0000 7f42f2ee7700 1 -- 172.21.15.44:0/2611093619 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42dc0a0690 msgr2=0x7f42dc0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:27.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.386+0000 7f42f2ee7700 1 --2- 172.21.15.44:0/2611093619 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42dc0a0690 0x7f42dc0a2b50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009c60 tx=0x7f42d8009260 comp rx=0 tx=0).stop 2023-09-10T00:40:27.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.386+0000 7f42f2ee7700 1 -- 172.21.15.44:0/2611093619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ec0ff4e0 msgr2=0x7f42ec196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:27.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.386+0000 7f42f2ee7700 1 --2- 172.21.15.44:0/2611093619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ec0ff4e0 0x7f42ec196280 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009fd0 tx=0x7f42e000fa00 comp rx=0 tx=0).stop 2023-09-10T00:40:27.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.387+0000 7f42f2ee7700 1 -- 172.21.15.44:0/2611093619 shutdown_connections 2023-09-10T00:40:27.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.387+0000 7f42f2ee7700 1 --2- 172.21.15.44:0/2611093619 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f42dc0a0690 0x7f42dc0a2b50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.387+0000 7f42f2ee7700 1 --2- 172.21.15.44:0/2611093619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42ec101070 0x7f42ec19d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.387+0000 7f42f2ee7700 1 --2- 172.21.15.44:0/2611093619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ec0ff4e0 0x7f42ec196280 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.387+0000 7f42f2ee7700 1 --2- 172.21.15.44:0/2611093619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42ec1006d0 0x7f42ec1967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.387+0000 7f42f2ee7700 1 -- 172.21.15.44:0/2611093619 >> 172.21.15.44:0/2611093619 conn(0x7f42ec0faa30 msgr2=0x7f42ec103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.387+0000 7f42f2ee7700 1 -- 172.21.15.44:0/2611093619 shutdown_connections 2023-09-10T00:40:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.387+0000 7f42f2ee7700 1 -- 172.21.15.44:0/2611093619 wait complete. 2023-09-10T00:40:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-10T00:34:44.456741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-10T00:38:37.194891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-10T00:38:38.573268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-10T00:38:42.198896+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-10T00:40:27.400 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-10T00:40:27.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.530+0000 7f117c23c700 1 -- 172.21.15.44:0/3018644549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11741006e0 msgr2=0x7f1174100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:27.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.530+0000 7f117c23c700 1 --2- 172.21.15.44:0/3018644549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11741006e0 0x7f1174100b40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f11700099c0 tx=0x7f1170009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.530+0000 7f117c23c700 1 -- 172.21.15.44:0/3018644549 shutdown_connections 2023-09-10T00:40:27.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.530+0000 7f117c23c700 1 --2- 172.21.15.44:0/3018644549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1174101080 0x7f1174107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.530+0000 7f117c23c700 1 --2- 172.21.15.44:0/3018644549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11740ff4f0 0x7f11740ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.530+0000 7f117c23c700 1 --2- 172.21.15.44:0/3018644549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11741006e0 0x7f1174100b40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.530+0000 7f117c23c700 1 -- 172.21.15.44:0/3018644549 >> 172.21.15.44:0/3018644549 conn(0x7f11740faa60 msgr2=0x7f11740fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:27.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.530+0000 7f117c23c700 1 -- 172.21.15.44:0/3018644549 shutdown_connections 2023-09-10T00:40:27.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.530+0000 7f117c23c700 1 -- 172.21.15.44:0/3018644549 wait complete. 2023-09-10T00:40:27.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f117c23c700 1 Processor -- start 2023-09-10T00:40:27.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f117c23c700 1 -- start start 2023-09-10T00:40:27.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f117c23c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11740ff4f0 0x7f1174198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:27.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f117c23c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11741006e0 0x7f1174198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:27.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f117c23c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1174101080 0x7f117419f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:27.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f117c23c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117419ffb0 con 0x7f11741006e0 2023-09-10T00:40:27.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f117c23c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741a00f0 con 0x7f1174101080 2023-09-10T00:40:27.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f117c23c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741a0280 con 0x7f11740ff4f0 2023-09-10T00:40:27.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f11797d7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11741006e0 0x7f1174198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:27.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f11797d7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11741006e0 0x7f1174198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48458/0 (socket says 172.21.15.44:48458) 2023-09-10T00:40:27.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f11797d7700 1 -- 172.21.15.44:0/36516851 learned_addr learned my addr 172.21.15.44:0/36516851 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:27.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f117a7d9700 1 --2- 172.21.15.44:0/36516851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1174101080 0x7f117419f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:27.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f1179fd8700 1 --2- 172.21.15.44:0/36516851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11740ff4f0 0x7f1174198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:27.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f11797d7700 1 -- 172.21.15.44:0/36516851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11740ff4f0 msgr2=0x7f1174198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:27.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f11797d7700 1 --2- 172.21.15.44:0/36516851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11740ff4f0 0x7f1174198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f11797d7700 1 -- 172.21.15.44:0/36516851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1174101080 msgr2=0x7f117419f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:27.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f11797d7700 1 --2- 172.21.15.44:0/36516851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1174101080 0x7f117419f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f11797d7700 1 -- 172.21.15.44:0/36516851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11700095f0 con 0x7f11741006e0 2023-09-10T00:40:27.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.531+0000 7f11797d7700 1 --2- 172.21.15.44:0/36516851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11741006e0 0x7f1174198970 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1170000c00 tx=0x7f117000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:27.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.532+0000 7f116affd700 1 -- 172.21.15.44:0/36516851 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117001d070 con 0x7f11741006e0 2023-09-10T00:40:27.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.532+0000 7f116affd700 1 -- 172.21.15.44:0/36516851 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f117000fe70 con 0x7f11741006e0 2023-09-10T00:40:27.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.532+0000 7f117c23c700 1 -- 172.21.15.44:0/36516851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11741a0500 con 0x7f11741006e0 2023-09-10T00:40:27.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.532+0000 7f116affd700 1 -- 172.21.15.44:0/36516851 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11700173f0 con 0x7f11741006e0 2023-09-10T00:40:27.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.532+0000 7f117c23c700 1 -- 172.21.15.44:0/36516851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11741a0920 con 0x7f11741006e0 2023-09-10T00:40:27.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.533+0000 7f117c23c700 1 -- 172.21.15.44:0/36516851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117410ac30 con 0x7f11741006e0 2023-09-10T00:40:27.541 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.534+0000 7f116affd700 1 -- 172.21.15.44:0/36516851 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f1170022470 con 0x7f11741006e0 2023-09-10T00:40:27.541 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.534+0000 7f116affd700 1 --2- 172.21.15.44:0/36516851 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f11600a0640 0x7f11600a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:27.541 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.534+0000 7f116affd700 1 -- 172.21.15.44:0/36516851 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f11700cd340 con 0x7f11741006e0 2023-09-10T00:40:27.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.534+0000 7f1179fd8700 1 --2- 172.21.15.44:0/36516851 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f11600a0640 0x7f11600a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:27.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.535+0000 7f1179fd8700 1 --2- 172.21.15.44:0/36516851 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f11600a0640 0x7f11600a2b00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1174100540 tx=0x7f1164009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:27.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.538+0000 7f116affd700 1 -- 172.21.15.44:0/36516851 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1170096b40 con 0x7f11741006e0 2023-09-10T00:40:27.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.740+0000 7f117c23c700 1 -- 172.21.15.44:0/36516851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11741a0b40 con 0x7f11741006e0 2023-09-10T00:40:27.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.740+0000 7f116affd700 1 -- 172.21.15.44:0/36516851 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f117009a160 con 0x7f11741006e0 2023-09-10T00:40:27.741 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2023-09-10T00:40:27.743 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.742+0000 7f117c23c700 1 -- 172.21.15.44:0/36516851 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f11600a0640 msgr2=0x7f11600a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:27.743 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.742+0000 7f117c23c700 1 --2- 172.21.15.44:0/36516851 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f11600a0640 0x7f11600a2b00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1174100540 tx=0x7f1164009260 comp rx=0 tx=0).stop 2023-09-10T00:40:27.743 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.742+0000 7f117c23c700 1 -- 172.21.15.44:0/36516851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11741006e0 msgr2=0x7f1174198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:27.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.742+0000 7f117c23c700 1 --2- 172.21.15.44:0/36516851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11741006e0 0x7f1174198970 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1170000c00 tx=0x7f117000fa00 comp rx=0 tx=0).stop 2023-09-10T00:40:27.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.743+0000 7f117c23c700 1 -- 172.21.15.44:0/36516851 shutdown_connections 2023-09-10T00:40:27.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.743+0000 7f117c23c700 1 --2- 172.21.15.44:0/36516851 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f11600a0640 0x7f11600a2b00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.743+0000 7f117c23c700 1 --2- 172.21.15.44:0/36516851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1174101080 0x7f117419f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.743+0000 7f117c23c700 1 --2- 172.21.15.44:0/36516851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11740ff4f0 0x7f1174198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.743+0000 7f117c23c700 1 --2- 172.21.15.44:0/36516851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11741006e0 0x7f1174198970 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.743+0000 7f117c23c700 1 -- 172.21.15.44:0/36516851 >> 172.21.15.44:0/36516851 conn(0x7f11740faa60 msgr2=0x7f11740fb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:27.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.743+0000 7f117c23c700 1 -- 172.21.15.44:0/36516851 shutdown_connections 2023-09-10T00:40:27.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.743+0000 7f117c23c700 1 -- 172.21.15.44:0/36516851 wait complete. 2023-09-10T00:40:27.755 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-10T00:40:27.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.886+0000 7f1646f73700 1 -- 172.21.15.44:0/676865554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1640101080 msgr2=0x7f1640107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:27.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.886+0000 7f1646f73700 1 --2- 172.21.15.44:0/676865554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1640101080 0x7f1640107930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f163800b0d0 tx=0x7f163800b3e0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.886+0000 7f1646f73700 1 -- 172.21.15.44:0/676865554 shutdown_connections 2023-09-10T00:40:27.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.886+0000 7f1646f73700 1 --2- 172.21.15.44:0/676865554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1640101080 0x7f1640107930 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.886+0000 7f1646f73700 1 --2- 172.21.15.44:0/676865554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16400ff4f0 0x7f16400ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.886+0000 7f1646f73700 1 --2- 172.21.15.44:0/676865554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16401006e0 0x7f1640100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.886+0000 7f1646f73700 1 -- 172.21.15.44:0/676865554 >> 172.21.15.44:0/676865554 conn(0x7f16400faa60 msgr2=0x7f16400fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:27.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.886+0000 7f1646f73700 1 -- 172.21.15.44:0/676865554 shutdown_connections 2023-09-10T00:40:27.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.886+0000 7f1646f73700 1 -- 172.21.15.44:0/676865554 wait complete. 2023-09-10T00:40:27.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f1646f73700 1 Processor -- start 2023-09-10T00:40:27.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f1646f73700 1 -- start start 2023-09-10T00:40:27.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f1646f73700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16400ff4f0 0x7f16400682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:27.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f1646f73700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16401006e0 0x7f164019b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:27.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f1646f73700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1640101080 0x7f164019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:27.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f1646f73700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164019de30 con 0x7f1640101080 2023-09-10T00:40:27.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f1646f73700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164019dfa0 con 0x7f16400ff4f0 2023-09-10T00:40:27.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f1646f73700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164019e160 con 0x7f16401006e0 2023-09-10T00:40:27.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f1645510700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1640101080 0x7f164019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:27.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f1645510700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1640101080 0x7f164019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48472/0 (socket says 172.21.15.44:48472) 2023-09-10T00:40:27.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f1645510700 1 -- 172.21.15.44:0/230712805 learned_addr learned my addr 172.21.15.44:0/230712805 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:27.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f163ffff700 1 --2- 172.21.15.44:0/230712805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16401006e0 0x7f164019b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:27.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f1644d0f700 1 --2- 172.21.15.44:0/230712805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16400ff4f0 0x7f16400682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:27.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f1645510700 1 -- 172.21.15.44:0/230712805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16401006e0 msgr2=0x7f164019b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:27.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f1645510700 1 --2- 172.21.15.44:0/230712805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16401006e0 0x7f164019b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f1645510700 1 -- 172.21.15.44:0/230712805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16400ff4f0 msgr2=0x7f16400682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:27.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.887+0000 7f1645510700 1 --2- 172.21.15.44:0/230712805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16400ff4f0 0x7f16400682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:27.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.888+0000 7f1645510700 1 -- 172.21.15.44:0/230712805 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1638009d00 con 0x7f1640101080 2023-09-10T00:40:27.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.888+0000 7f1645510700 1 --2- 172.21.15.44:0/230712805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1640101080 0x7f164019d660 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1638009fd0 tx=0x7f163800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:27.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.888+0000 7f163dffb700 1 -- 172.21.15.44:0/230712805 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163800e040 con 0x7f1640101080 2023-09-10T00:40:27.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.888+0000 7f163dffb700 1 -- 172.21.15.44:0/230712805 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1638003e10 con 0x7f1640101080 2023-09-10T00:40:27.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.888+0000 7f1646f73700 1 -- 172.21.15.44:0/230712805 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164019e3e0 con 0x7f1640101080 2023-09-10T00:40:27.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.888+0000 7f163dffb700 1 -- 172.21.15.44:0/230712805 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16380052a0 con 0x7f1640101080 2023-09-10T00:40:27.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.888+0000 7f1646f73700 1 -- 172.21.15.44:0/230712805 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164019e860 con 0x7f1640101080 2023-09-10T00:40:27.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.889+0000 7f1646f73700 1 -- 172.21.15.44:0/230712805 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1640106470 con 0x7f1640101080 2023-09-10T00:40:27.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.890+0000 7f163dffb700 1 -- 172.21.15.44:0/230712805 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f1638005400 con 0x7f1640101080 2023-09-10T00:40:27.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.891+0000 7f163dffb700 1 --2- 172.21.15.44:0/230712805 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16300a0640 0x7f16300a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:27.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.891+0000 7f1644d0f700 1 --2- 172.21.15.44:0/230712805 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16300a0640 0x7f16300a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:27.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.891+0000 7f163dffb700 1 -- 172.21.15.44:0/230712805 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f16380cd300 con 0x7f1640101080 2023-09-10T00:40:27.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.891+0000 7f1644d0f700 1 --2- 172.21.15.44:0/230712805 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16300a0640 0x7f16300a2b00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1640100540 tx=0x7f1634009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:27.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:27.895+0000 7f163dffb700 1 -- 172.21.15.44:0/230712805 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1638096b00 con 0x7f1640101080 2023-09-10T00:40:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:27 smithi131 ceph-mon[103713]: pgmap v326: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 12 op/s; 6.1 KiB/s, 1 keys/s, 1 objects/s recovering 2023-09-10T00:40:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:27 smithi131 ceph-mon[103713]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:40:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:27 smithi131 ceph-mon[103713]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:40:28.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:27 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2611093619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:28.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:27 smithi044 ceph-mon[98676]: pgmap v326: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 12 op/s; 6.1 KiB/s, 1 keys/s, 1 objects/s recovering 2023-09-10T00:40:28.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:27 smithi044 ceph-mon[98676]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:40:28.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:27 smithi044 ceph-mon[98676]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:40:28.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:27 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2611093619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:28.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.097+0000 7f1646f73700 1 -- 172.21.15.44:0/230712805 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f164002cfa0 con 0x7f1640101080 2023-09-10T00:40:28.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.098+0000 7f163dffb700 1 -- 172.21.15.44:0/230712805 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f163809a120 con 0x7f1640101080 2023-09-10T00:40:28.098 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:40:28.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.100+0000 7f1646f73700 1 -- 172.21.15.44:0/230712805 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16300a0640 msgr2=0x7f16300a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:28.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.100+0000 7f1646f73700 1 --2- 172.21.15.44:0/230712805 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16300a0640 0x7f16300a2b00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1640100540 tx=0x7f1634009260 comp rx=0 tx=0).stop 2023-09-10T00:40:28.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.100+0000 7f1646f73700 1 -- 172.21.15.44:0/230712805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1640101080 msgr2=0x7f164019d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:28.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.100+0000 7f1646f73700 1 --2- 172.21.15.44:0/230712805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1640101080 0x7f164019d660 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1638009fd0 tx=0x7f163800ccc0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.100+0000 7f1646f73700 1 -- 172.21.15.44:0/230712805 shutdown_connections 2023-09-10T00:40:28.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.100+0000 7f1646f73700 1 --2- 172.21.15.44:0/230712805 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f16300a0640 0x7f16300a2b00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.100+0000 7f1646f73700 1 --2- 172.21.15.44:0/230712805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1640101080 0x7f164019d660 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.100+0000 7f1646f73700 1 --2- 172.21.15.44:0/230712805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16400ff4f0 0x7f16400682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.100+0000 7f1646f73700 1 --2- 172.21.15.44:0/230712805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16401006e0 0x7f164019b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.100+0000 7f1646f73700 1 -- 172.21.15.44:0/230712805 >> 172.21.15.44:0/230712805 conn(0x7f16400faa60 msgr2=0x7f16400fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:28.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.100+0000 7f1646f73700 1 -- 172.21.15.44:0/230712805 shutdown_connections 2023-09-10T00:40:28.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.100+0000 7f1646f73700 1 -- 172.21.15.44:0/230712805 wait complete. 2023-09-10T00:40:28.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:27 smithi099 ceph-mon[102692]: pgmap v326: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 12 op/s; 6.1 KiB/s, 1 keys/s, 1 objects/s recovering 2023-09-10T00:40:28.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:27 smithi099 ceph-mon[102692]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:40:28.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:27 smithi099 ceph-mon[102692]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:40:28.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:27 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2611093619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:28.113 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-10T00:40:28.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.241+0000 7f354a6d3700 1 -- 172.21.15.44:0/3172679776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35440ff4f0 msgr2=0x7f35440ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:28.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.241+0000 7f354a6d3700 1 --2- 172.21.15.44:0/3172679776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35440ff4f0 0x7f35440ff910 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f35340099c0 tx=0x7f3534009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.241+0000 7f354a6d3700 1 -- 172.21.15.44:0/3172679776 shutdown_connections 2023-09-10T00:40:28.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.241+0000 7f354a6d3700 1 --2- 172.21.15.44:0/3172679776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3544101080 0x7f3544107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.241+0000 7f354a6d3700 1 --2- 172.21.15.44:0/3172679776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35440ff4f0 0x7f35440ff910 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.241+0000 7f354a6d3700 1 --2- 172.21.15.44:0/3172679776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35441006e0 0x7f3544100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.241+0000 7f354a6d3700 1 -- 172.21.15.44:0/3172679776 >> 172.21.15.44:0/3172679776 conn(0x7f35440faa60 msgr2=0x7f35440fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:28.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.241+0000 7f354a6d3700 1 -- 172.21.15.44:0/3172679776 shutdown_connections 2023-09-10T00:40:28.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.241+0000 7f354a6d3700 1 -- 172.21.15.44:0/3172679776 wait complete. 2023-09-10T00:40:28.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.242+0000 7f354a6d3700 1 Processor -- start 2023-09-10T00:40:28.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.242+0000 7f354a6d3700 1 -- start start 2023-09-10T00:40:28.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.242+0000 7f354a6d3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35440ff4f0 0x7f35440682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:28.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.242+0000 7f354a6d3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35441006e0 0x7f354419af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:28.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.242+0000 7f354a6d3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3544101080 0x7f354419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:28.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.242+0000 7f354a6d3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354419dd70 con 0x7f35441006e0 2023-09-10T00:40:28.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.242+0000 7f354a6d3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354419dee0 con 0x7f3544101080 2023-09-10T00:40:28.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.242+0000 7f354a6d3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354419e0a0 con 0x7f35440ff4f0 2023-09-10T00:40:28.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.242+0000 7f35437fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35441006e0 0x7f354419af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:28.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.242+0000 7f35437fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35441006e0 0x7f354419af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48484/0 (socket says 172.21.15.44:48484) 2023-09-10T00:40:28.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.243+0000 7f35437fe700 1 -- 172.21.15.44:0/3838994973 learned_addr learned my addr 172.21.15.44:0/3838994973 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:28.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.243+0000 7f3543fff700 1 --2- 172.21.15.44:0/3838994973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35440ff4f0 0x7f35440682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:28.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.243+0000 7f3548c70700 1 --2- 172.21.15.44:0/3838994973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3544101080 0x7f354419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:28.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.243+0000 7f35437fe700 1 -- 172.21.15.44:0/3838994973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35440ff4f0 msgr2=0x7f35440682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:28.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.243+0000 7f35437fe700 1 --2- 172.21.15.44:0/3838994973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35440ff4f0 0x7f35440682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.243+0000 7f35437fe700 1 -- 172.21.15.44:0/3838994973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3544101080 msgr2=0x7f354419d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:28.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.243+0000 7f35437fe700 1 --2- 172.21.15.44:0/3838994973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3544101080 0x7f354419d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.243+0000 7f35437fe700 1 -- 172.21.15.44:0/3838994973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35340095f0 con 0x7f35441006e0 2023-09-10T00:40:28.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.243+0000 7f3548c70700 1 --2- 172.21.15.44:0/3838994973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3544101080 0x7f354419d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:28.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.243+0000 7f35437fe700 1 --2- 172.21.15.44:0/3838994973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35441006e0 0x7f354419af90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f353800d950 tx=0x7f353800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:28.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.243+0000 7f35417fa700 1 -- 172.21.15.44:0/3838994973 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3538009940 con 0x7f35441006e0 2023-09-10T00:40:28.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.243+0000 7f354a6d3700 1 -- 172.21.15.44:0/3838994973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354419e380 con 0x7f35441006e0 2023-09-10T00:40:28.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.243+0000 7f35417fa700 1 -- 172.21.15.44:0/3838994973 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3538010460 con 0x7f35441006e0 2023-09-10T00:40:28.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.243+0000 7f35417fa700 1 -- 172.21.15.44:0/3838994973 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353800f620 con 0x7f35441006e0 2023-09-10T00:40:28.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.243+0000 7f354a6d3700 1 -- 172.21.15.44:0/3838994973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354419e8e0 con 0x7f35441006e0 2023-09-10T00:40:28.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.244+0000 7f354a6d3700 1 -- 172.21.15.44:0/3838994973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354410abc0 con 0x7f35441006e0 2023-09-10T00:40:28.252 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.245+0000 7f35417fa700 1 -- 172.21.15.44:0/3838994973 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f35380105d0 con 0x7f35441006e0 2023-09-10T00:40:28.252 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.246+0000 7f35417fa700 1 --2- 172.21.15.44:0/3838994973 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f352c0a0640 0x7f352c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:28.252 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.246+0000 7f35417fa700 1 -- 172.21.15.44:0/3838994973 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f35380cbe00 con 0x7f35441006e0 2023-09-10T00:40:28.252 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.246+0000 7f3543fff700 1 --2- 172.21.15.44:0/3838994973 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f352c0a0640 0x7f352c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:28.252 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.246+0000 7f3543fff700 1 --2- 172.21.15.44:0/3838994973 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f352c0a0640 0x7f352c0a2b00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3534000c00 tx=0x7f353400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:28.253 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.250+0000 7f35417fa700 1 -- 172.21.15.44:0/3838994973 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3538095510 con 0x7f35441006e0 2023-09-10T00:40:28.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.455+0000 7f354a6d3700 1 -- 172.21.15.44:0/3838994973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3544106500 con 0x7f35441006e0 2023-09-10T00:40:28.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.455+0000 7f35417fa700 1 -- 172.21.15.44:0/3838994973 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f35380160a0 con 0x7f35441006e0 2023-09-10T00:40:28.456 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2023-09-10T00:40:28.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.458+0000 7f354a6d3700 1 -- 172.21.15.44:0/3838994973 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f352c0a0640 msgr2=0x7f352c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:28.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.458+0000 7f354a6d3700 1 --2- 172.21.15.44:0/3838994973 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f352c0a0640 0x7f352c0a2b00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3534000c00 tx=0x7f353400b540 comp rx=0 tx=0).stop 2023-09-10T00:40:28.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.458+0000 7f354a6d3700 1 -- 172.21.15.44:0/3838994973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35441006e0 msgr2=0x7f354419af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:28.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.458+0000 7f354a6d3700 1 --2- 172.21.15.44:0/3838994973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35441006e0 0x7f354419af90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f353800d950 tx=0x7f353800dc60 comp rx=0 tx=0).stop 2023-09-10T00:40:28.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.458+0000 7f354a6d3700 1 -- 172.21.15.44:0/3838994973 shutdown_connections 2023-09-10T00:40:28.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.458+0000 7f354a6d3700 1 --2- 172.21.15.44:0/3838994973 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f352c0a0640 0x7f352c0a2b00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.458+0000 7f354a6d3700 1 --2- 172.21.15.44:0/3838994973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3544101080 0x7f354419d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.458+0000 7f354a6d3700 1 --2- 172.21.15.44:0/3838994973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35440ff4f0 0x7f35440682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.458+0000 7f354a6d3700 1 --2- 172.21.15.44:0/3838994973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35441006e0 0x7f354419af90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.458+0000 7f354a6d3700 1 -- 172.21.15.44:0/3838994973 >> 172.21.15.44:0/3838994973 conn(0x7f35440faa60 msgr2=0x7f35440fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:28.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.458+0000 7f354a6d3700 1 -- 172.21.15.44:0/3838994973 shutdown_connections 2023-09-10T00:40:28.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.458+0000 7f354a6d3700 1 -- 172.21.15.44:0/3838994973 wait complete. 2023-09-10T00:40:28.470 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-10T00:40:28.601 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.599+0000 7f0687bb5700 1 -- 172.21.15.44:0/2967795315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06801039a0 msgr2=0x7f0680105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:28.601 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.599+0000 7f0687bb5700 1 --2- 172.21.15.44:0/2967795315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06801039a0 0x7f0680105d90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f067c009990 tx=0x7f067c009ca0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.599+0000 7f0687bb5700 1 -- 172.21.15.44:0/2967795315 shutdown_connections 2023-09-10T00:40:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.599+0000 7f0687bb5700 1 --2- 172.21.15.44:0/2967795315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06801062d0 0x7f0680108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.599+0000 7f0687bb5700 1 --2- 172.21.15.44:0/2967795315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0680101070 0x7f0680103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.599+0000 7f0687bb5700 1 --2- 172.21.15.44:0/2967795315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06801039a0 0x7f0680105d90 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.599+0000 7f0687bb5700 1 -- 172.21.15.44:0/2967795315 >> 172.21.15.44:0/2967795315 conn(0x7f06800faa60 msgr2=0x7f06800fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.599+0000 7f0687bb5700 1 -- 172.21.15.44:0/2967795315 shutdown_connections 2023-09-10T00:40:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.599+0000 7f0687bb5700 1 -- 172.21.15.44:0/2967795315 wait complete. 2023-09-10T00:40:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.599+0000 7f0687bb5700 1 Processor -- start 2023-09-10T00:40:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0687bb5700 1 -- start start 2023-09-10T00:40:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0687bb5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0680101070 0x7f06801961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0687bb5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06801039a0 0x7f0680196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:28.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0687bb5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06801062d0 0x7f068019d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:28.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0687bb5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068019dd40 con 0x7f0680101070 2023-09-10T00:40:28.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0687bb5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068019de80 con 0x7f06801039a0 2023-09-10T00:40:28.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0687bb5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068019e010 con 0x7f06801062d0 2023-09-10T00:40:28.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0685951700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0680101070 0x7f06801961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:28.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0685951700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0680101070 0x7f06801961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48496/0 (socket says 172.21.15.44:48496) 2023-09-10T00:40:28.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0685951700 1 -- 172.21.15.44:0/283942998 learned_addr learned my addr 172.21.15.44:0/283942998 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:28.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0686152700 1 --2- 172.21.15.44:0/283942998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06801062d0 0x7f068019d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:28.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0685150700 1 --2- 172.21.15.44:0/283942998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06801039a0 0x7f0680196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:28.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0685951700 1 -- 172.21.15.44:0/283942998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06801062d0 msgr2=0x7f068019d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:28.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0685951700 1 --2- 172.21.15.44:0/283942998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06801062d0 0x7f068019d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0685951700 1 -- 172.21.15.44:0/283942998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06801039a0 msgr2=0x7f0680196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:28.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0685951700 1 --2- 172.21.15.44:0/283942998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06801039a0 0x7f0680196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0685951700 1 -- 172.21.15.44:0/283942998 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c0095f0 con 0x7f0680101070 2023-09-10T00:40:28.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0686152700 1 --2- 172.21.15.44:0/283942998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06801062d0 0x7f068019d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:28.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0685951700 1 --2- 172.21.15.44:0/283942998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0680101070 0x7f06801961c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f067000c9b0 tx=0x7f067000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:28.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.600+0000 7f0676ffd700 1 -- 172.21.15.44:0/283942998 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670007b00 con 0x7f0680101070 2023-09-10T00:40:28.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.601+0000 7f0676ffd700 1 -- 172.21.15.44:0/283942998 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0670007c60 con 0x7f0680101070 2023-09-10T00:40:28.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.601+0000 7f0676ffd700 1 -- 172.21.15.44:0/283942998 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670018660 con 0x7f0680101070 2023-09-10T00:40:28.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.601+0000 7f0687bb5700 1 -- 172.21.15.44:0/283942998 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068019e2f0 con 0x7f0680101070 2023-09-10T00:40:28.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.601+0000 7f0687bb5700 1 -- 172.21.15.44:0/283942998 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068019e7b0 con 0x7f0680101070 2023-09-10T00:40:28.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.601+0000 7f0687bb5700 1 -- 172.21.15.44:0/283942998 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06800fc5e0 con 0x7f0680101070 2023-09-10T00:40:28.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.602+0000 7f0676ffd700 1 -- 172.21.15.44:0/283942998 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f067000f450 con 0x7f0680101070 2023-09-10T00:40:28.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.603+0000 7f0676ffd700 1 --2- 172.21.15.44:0/283942998 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f066c0a0640 0x7f066c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:28.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.603+0000 7f0685150700 1 --2- 172.21.15.44:0/283942998 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f066c0a0640 0x7f066c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:28.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.603+0000 7f0676ffd700 1 -- 172.21.15.44:0/283942998 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f06700cc200 con 0x7f0680101070 2023-09-10T00:40:28.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.604+0000 7f0685150700 1 --2- 172.21.15.44:0/283942998 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f066c0a0640 0x7f066c0a2b00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f067c00b5c0 tx=0x7f067c009f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:28.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.606+0000 7f0676ffd700 1 -- 172.21.15.44:0/283942998 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0670095a00 con 0x7f0680101070 2023-09-10T00:40:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.811+0000 7f0687bb5700 1 -- 172.21.15.44:0/283942998 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f068010bfc0 con 0x7f0680101070 2023-09-10T00:40:28.812 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.811+0000 7f0676ffd700 1 -- 172.21.15.44:0/283942998 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f067000fe60 con 0x7f0680101070 2023-09-10T00:40:28.812 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T00:40:28.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.814+0000 7f0687bb5700 1 -- 172.21.15.44:0/283942998 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f066c0a0640 msgr2=0x7f066c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:28.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.814+0000 7f0687bb5700 1 --2- 172.21.15.44:0/283942998 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f066c0a0640 0x7f066c0a2b00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f067c00b5c0 tx=0x7f067c009f00 comp rx=0 tx=0).stop 2023-09-10T00:40:28.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.814+0000 7f0687bb5700 1 -- 172.21.15.44:0/283942998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0680101070 msgr2=0x7f06801961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:28.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.814+0000 7f0687bb5700 1 --2- 172.21.15.44:0/283942998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0680101070 0x7f06801961c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f067000c9b0 tx=0x7f067000ccc0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.814+0000 7f0687bb5700 1 -- 172.21.15.44:0/283942998 shutdown_connections 2023-09-10T00:40:28.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.814+0000 7f0687bb5700 1 --2- 172.21.15.44:0/283942998 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f066c0a0640 0x7f066c0a2b00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.814+0000 7f0687bb5700 1 --2- 172.21.15.44:0/283942998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06801062d0 0x7f068019d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.814+0000 7f0687bb5700 1 --2- 172.21.15.44:0/283942998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0680101070 0x7f06801961c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.814+0000 7f0687bb5700 1 --2- 172.21.15.44:0/283942998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06801039a0 0x7f0680196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.814+0000 7f0687bb5700 1 -- 172.21.15.44:0/283942998 >> 172.21.15.44:0/283942998 conn(0x7f06800faa60 msgr2=0x7f0680101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:28.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.814+0000 7f0687bb5700 1 -- 172.21.15.44:0/283942998 shutdown_connections 2023-09-10T00:40:28.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.814+0000 7f0687bb5700 1 -- 172.21.15.44:0/283942998 wait complete. 2023-09-10T00:40:28.826 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:40:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.952+0000 7f1c085a5700 1 -- 172.21.15.44:0/1910401523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c001006d0 msgr2=0x7f1c00100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.952+0000 7f1c085a5700 1 --2- 172.21.15.44:0/1910401523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c001006d0 0x7f1c00100b30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0099c0 tx=0x7f1bfc009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.952+0000 7f1c085a5700 1 -- 172.21.15.44:0/1910401523 shutdown_connections 2023-09-10T00:40:28.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.952+0000 7f1c085a5700 1 --2- 172.21.15.44:0/1910401523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c00101070 0x7f1c00107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.952+0000 7f1c085a5700 1 --2- 172.21.15.44:0/1910401523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c000ff4e0 0x7f1c000ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.952+0000 7f1c085a5700 1 --2- 172.21.15.44:0/1910401523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c001006d0 0x7f1c00100b30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.952+0000 7f1c085a5700 1 -- 172.21.15.44:0/1910401523 >> 172.21.15.44:0/1910401523 conn(0x7f1c000faa30 msgr2=0x7f1c000fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:28.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.952+0000 7f1c085a5700 1 -- 172.21.15.44:0/1910401523 shutdown_connections 2023-09-10T00:40:28.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.952+0000 7f1c085a5700 1 -- 172.21.15.44:0/1910401523 wait complete. 2023-09-10T00:40:28.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.952+0000 7f1c085a5700 1 Processor -- start 2023-09-10T00:40:28.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.952+0000 7f1c085a5700 1 -- start start 2023-09-10T00:40:28.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c085a5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c000ff4e0 0x7f1c00068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:28.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c085a5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c001006d0 0x7f1c0019aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:28.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c085a5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c00101070 0x7f1c0019d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:28.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c085a5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0019dcf0 con 0x7f1c000ff4e0 2023-09-10T00:40:28.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c085a5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0019de60 con 0x7f1c00101070 2023-09-10T00:40:28.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c085a5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0019e020 con 0x7f1c001006d0 2023-09-10T00:40:28.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c06341700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c000ff4e0 0x7f1c00068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:28.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c06341700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c000ff4e0 0x7f1c00068330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48510/0 (socket says 172.21.15.44:48510) 2023-09-10T00:40:28.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c06341700 1 -- 172.21.15.44:0/3811064607 learned_addr learned my addr 172.21.15.44:0/3811064607 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:28.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c05b40700 1 --2- 172.21.15.44:0/3811064607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c001006d0 0x7f1c0019aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:28.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c06b42700 1 --2- 172.21.15.44:0/3811064607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c00101070 0x7f1c0019d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:28.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c06341700 1 -- 172.21.15.44:0/3811064607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c001006d0 msgr2=0x7f1c0019aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:28.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c06341700 1 --2- 172.21.15.44:0/3811064607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c001006d0 0x7f1c0019aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c06341700 1 -- 172.21.15.44:0/3811064607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c00101070 msgr2=0x7f1c0019d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:28.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c06341700 1 --2- 172.21.15.44:0/3811064607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c00101070 0x7f1c0019d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:28.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c06341700 1 -- 172.21.15.44:0/3811064607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc0095f0 con 0x7f1c000ff4e0 2023-09-10T00:40:28.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.953+0000 7f1c06341700 1 --2- 172.21.15.44:0/3811064607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c000ff4e0 0x7f1c00068330 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000d980 tx=0x7f1bf000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:28.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.954+0000 7f1bf77fe700 1 -- 172.21.15.44:0/3811064607 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0004020 con 0x7f1c000ff4e0 2023-09-10T00:40:28.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.954+0000 7f1bf77fe700 1 -- 172.21.15.44:0/3811064607 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1bf000f690 con 0x7f1c000ff4e0 2023-09-10T00:40:28.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.954+0000 7f1bf77fe700 1 -- 172.21.15.44:0/3811064607 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf00039e0 con 0x7f1c000ff4e0 2023-09-10T00:40:28.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.954+0000 7f1c085a5700 1 -- 172.21.15.44:0/3811064607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c0019e300 con 0x7f1c000ff4e0 2023-09-10T00:40:28.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.954+0000 7f1c085a5700 1 -- 172.21.15.44:0/3811064607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0019e810 con 0x7f1c000ff4e0 2023-09-10T00:40:28.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.954+0000 7f1c085a5700 1 -- 172.21.15.44:0/3811064607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0010abb0 con 0x7f1c000ff4e0 2023-09-10T00:40:28.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.956+0000 7f1bf77fe700 1 -- 172.21.15.44:0/3811064607 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f1bf000f800 con 0x7f1c000ff4e0 2023-09-10T00:40:28.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.956+0000 7f1bf77fe700 1 --2- 172.21.15.44:0/3811064607 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1bec0a0710 0x7f1bec0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:28.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.956+0000 7f1c05b40700 1 --2- 172.21.15.44:0/3811064607 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1bec0a0710 0x7f1bec0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:28.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.956+0000 7f1bf77fe700 1 -- 172.21.15.44:0/3811064607 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f1bf00cbe20 con 0x7f1c000ff4e0 2023-09-10T00:40:28.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.957+0000 7f1c05b40700 1 --2- 172.21.15.44:0/3811064607 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1bec0a0710 0x7f1bec0a2bd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00b580 tx=0x7f1bfc005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:28.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:28.960+0000 7f1bf77fe700 1 -- 172.21.15.44:0/3811064607 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1bf0095610 con 0x7f1c000ff4e0 2023-09-10T00:40:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:28 smithi131 ceph-mon[103713]: from='client.15726 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:40:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:28 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/36516851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:40:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:28 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/230712805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:40:29.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:28 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3838994973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:40:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:28 smithi044 ceph-mon[98676]: from='client.15726 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:40:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:28 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/36516851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:40:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:28 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/230712805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:40:29.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:28 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3838994973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:40:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:28 smithi099 ceph-mon[102692]: from='client.15726 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-10T00:40:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:28 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/36516851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T00:40:29.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:28 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/230712805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T00:40:29.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:28 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3838994973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T00:40:29.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.205+0000 7f1c085a5700 1 -- 172.21.15.44:0/3811064607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c00106490 con 0x7f1c000ff4e0 2023-09-10T00:40:29.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.206+0000 7f1bf77fe700 1 -- 172.21.15.44:0/3811064607 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11369 (secure 0 0 0) 0x7f1bf0098c30 con 0x7f1c000ff4e0 2023-09-10T00:40:29.206 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:29.206 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-10T00:38:39.183533+0000","modified":"2023-09-10T00:39:02.480227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24605,"mds_1":24574,"mds_2":24572,"mds_3":24515,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14751":{"gid":14751,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/3958270510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":3958270510},{"type":"v1","addr":"172.21.15.44:6839","nonce":3958270510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14763":{"gid":14763,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/1241002552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1241002552},{"type":"v1","addr":"172.21.15.44:6841","nonce":1241002552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24556":{"gid":24556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/3222652693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":3222652693},{"type":"v1","addr":"172.21.15.131:6839","nonce":3222652693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24605":{"gid":24605,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/2622966260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2622966260},{"type":"v1","addr":"172.21.15.99:6839","nonce":2622966260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-10T00:40:29.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.208+0000 7f1c085a5700 1 -- 172.21.15.44:0/3811064607 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1bec0a0710 msgr2=0x7f1bec0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.208+0000 7f1c085a5700 1 --2- 172.21.15.44:0/3811064607 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1bec0a0710 0x7f1bec0a2bd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00b580 tx=0x7f1bfc005ea0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.208+0000 7f1c085a5700 1 -- 172.21.15.44:0/3811064607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c000ff4e0 msgr2=0x7f1c00068330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.208+0000 7f1c085a5700 1 --2- 172.21.15.44:0/3811064607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c000ff4e0 0x7f1c00068330 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000d980 tx=0x7f1bf000dc90 comp rx=0 tx=0).stop 2023-09-10T00:40:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.208+0000 7f1c085a5700 1 -- 172.21.15.44:0/3811064607 shutdown_connections 2023-09-10T00:40:29.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.208+0000 7f1c085a5700 1 --2- 172.21.15.44:0/3811064607 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1bec0a0710 0x7f1bec0a2bd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.208+0000 7f1c085a5700 1 --2- 172.21.15.44:0/3811064607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c00101070 0x7f1c0019d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.208+0000 7f1c085a5700 1 --2- 172.21.15.44:0/3811064607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c000ff4e0 0x7f1c00068330 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.208+0000 7f1c085a5700 1 --2- 172.21.15.44:0/3811064607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c001006d0 0x7f1c0019aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.208+0000 7f1c085a5700 1 -- 172.21.15.44:0/3811064607 >> 172.21.15.44:0/3811064607 conn(0x7f1c000faa30 msgr2=0x7f1c000fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:29.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.208+0000 7f1c085a5700 1 -- 172.21.15.44:0/3811064607 shutdown_connections 2023-09-10T00:40:29.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.208+0000 7f1c085a5700 1 -- 172.21.15.44:0/3811064607 wait complete. 2023-09-10T00:40:29.211 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2023-09-10T00:40:29.221 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T00:40:29.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.347+0000 7f7d3270d700 1 -- 172.21.15.44:0/4253517567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d2c1006d0 msgr2=0x7f7d2c100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.347+0000 7f7d3270d700 1 --2- 172.21.15.44:0/4253517567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d2c1006d0 0x7f7d2c100b30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7d200099c0 tx=0x7f7d20009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.347+0000 7f7d3270d700 1 -- 172.21.15.44:0/4253517567 shutdown_connections 2023-09-10T00:40:29.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.347+0000 7f7d3270d700 1 --2- 172.21.15.44:0/4253517567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d2c101070 0x7f7d2c107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.347+0000 7f7d3270d700 1 --2- 172.21.15.44:0/4253517567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d2c0ff4e0 0x7f7d2c0ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.347+0000 7f7d3270d700 1 --2- 172.21.15.44:0/4253517567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d2c1006d0 0x7f7d2c100b30 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.347+0000 7f7d3270d700 1 -- 172.21.15.44:0/4253517567 >> 172.21.15.44:0/4253517567 conn(0x7f7d2c0faa30 msgr2=0x7f7d2c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:29.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.347+0000 7f7d3270d700 1 -- 172.21.15.44:0/4253517567 shutdown_connections 2023-09-10T00:40:29.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.347+0000 7f7d3270d700 1 -- 172.21.15.44:0/4253517567 wait complete. 2023-09-10T00:40:29.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.347+0000 7f7d3270d700 1 Processor -- start 2023-09-10T00:40:29.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d3270d700 1 -- start start 2023-09-10T00:40:29.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d3270d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d2c0ff4e0 0x7f7d2c0682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d3270d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d2c1006d0 0x7f7d2c19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d3270d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d2c101070 0x7f7d2c19d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d3270d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c19dcd0 con 0x7f7d2c0ff4e0 2023-09-10T00:40:29.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d3270d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c19de40 con 0x7f7d2c1006d0 2023-09-10T00:40:29.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d3270d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c19e000 con 0x7f7d2c101070 2023-09-10T00:40:29.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d2bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d2c0ff4e0 0x7f7d2c0682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d2bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d2c0ff4e0 0x7f7d2c0682d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48524/0 (socket says 172.21.15.44:48524) 2023-09-10T00:40:29.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d2bfff700 1 -- 172.21.15.44:0/3501313723 learned_addr learned my addr 172.21.15.44:0/3501313723 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:29.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d2b7fe700 1 --2- 172.21.15.44:0/3501313723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d2c1006d0 0x7f7d2c19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d30caa700 1 --2- 172.21.15.44:0/3501313723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d2c101070 0x7f7d2c19d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d2bfff700 1 -- 172.21.15.44:0/3501313723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d2c101070 msgr2=0x7f7d2c19d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d2bfff700 1 --2- 172.21.15.44:0/3501313723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d2c101070 0x7f7d2c19d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d2bfff700 1 -- 172.21.15.44:0/3501313723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d2c1006d0 msgr2=0x7f7d2c19aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d2bfff700 1 --2- 172.21.15.44:0/3501313723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d2c1006d0 0x7f7d2c19aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.348+0000 7f7d2bfff700 1 -- 172.21.15.44:0/3501313723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d200095f0 con 0x7f7d2c0ff4e0 2023-09-10T00:40:29.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.349+0000 7f7d2bfff700 1 --2- 172.21.15.44:0/3501313723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d2c0ff4e0 0x7f7d2c0682d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00e970 tx=0x7f7d1c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:29.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.349+0000 7f7d297fa700 1 -- 172.21.15.44:0/3501313723 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c009790 con 0x7f7d2c0ff4e0 2023-09-10T00:40:29.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.349+0000 7f7d297fa700 1 -- 172.21.15.44:0/3501313723 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d1c00c7e0 con 0x7f7d2c0ff4e0 2023-09-10T00:40:29.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.349+0000 7f7d3270d700 1 -- 172.21.15.44:0/3501313723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2c19e2e0 con 0x7f7d2c0ff4e0 2023-09-10T00:40:29.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.349+0000 7f7d297fa700 1 -- 172.21.15.44:0/3501313723 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c003bd0 con 0x7f7d2c0ff4e0 2023-09-10T00:40:29.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.349+0000 7f7d3270d700 1 -- 172.21.15.44:0/3501313723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2c19e7f0 con 0x7f7d2c0ff4e0 2023-09-10T00:40:29.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.350+0000 7f7d3270d700 1 -- 172.21.15.44:0/3501313723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2c106af0 con 0x7f7d2c0ff4e0 2023-09-10T00:40:29.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.351+0000 7f7d297fa700 1 -- 172.21.15.44:0/3501313723 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f7d1c00c950 con 0x7f7d2c0ff4e0 2023-09-10T00:40:29.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.352+0000 7f7d297fa700 1 --2- 172.21.15.44:0/3501313723 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7d140a0690 0x7f7d140a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.352+0000 7f7d2b7fe700 1 --2- 172.21.15.44:0/3501313723 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7d140a0690 0x7f7d140a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.352+0000 7f7d297fa700 1 -- 172.21.15.44:0/3501313723 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f7d1c014070 con 0x7f7d2c0ff4e0 2023-09-10T00:40:29.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.352+0000 7f7d2b7fe700 1 --2- 172.21.15.44:0/3501313723 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7d140a0690 0x7f7d140a2b50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009990 tx=0x7f7d20005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:29.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.355+0000 7f7d297fa700 1 -- 172.21.15.44:0/3501313723 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7d1c095a40 con 0x7f7d2c0ff4e0 2023-09-10T00:40:29.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.560+0000 7f7d3270d700 1 -- 172.21.15.44:0/3501313723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d2c10ad80 con 0x7f7d2c0ff4e0 2023-09-10T00:40:29.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.562+0000 7f7d297fa700 1 -- 172.21.15.44:0/3501313723 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24678 (secure 0 0 0) 0x7f7d1c099060 con 0x7f7d2c0ff4e0 2023-09-10T00:40:29.563 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:29.563 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":91,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T00:40:17.671364+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T00:40:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.564+0000 7f7d3270d700 1 -- 172.21.15.44:0/3501313723 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7d140a0690 msgr2=0x7f7d140a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.564+0000 7f7d3270d700 1 --2- 172.21.15.44:0/3501313723 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7d140a0690 0x7f7d140a2b50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009990 tx=0x7f7d20005be0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.564+0000 7f7d3270d700 1 -- 172.21.15.44:0/3501313723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d2c0ff4e0 msgr2=0x7f7d2c0682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.564+0000 7f7d3270d700 1 --2- 172.21.15.44:0/3501313723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d2c0ff4e0 0x7f7d2c0682d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00e970 tx=0x7f7d1c00ec80 comp rx=0 tx=0).stop 2023-09-10T00:40:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.564+0000 7f7d3270d700 1 -- 172.21.15.44:0/3501313723 shutdown_connections 2023-09-10T00:40:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.564+0000 7f7d3270d700 1 --2- 172.21.15.44:0/3501313723 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f7d140a0690 0x7f7d140a2b50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.564+0000 7f7d3270d700 1 --2- 172.21.15.44:0/3501313723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d2c101070 0x7f7d2c19d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.564+0000 7f7d3270d700 1 --2- 172.21.15.44:0/3501313723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d2c0ff4e0 0x7f7d2c0682d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.564+0000 7f7d3270d700 1 --2- 172.21.15.44:0/3501313723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d2c1006d0 0x7f7d2c19aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.564+0000 7f7d3270d700 1 -- 172.21.15.44:0/3501313723 >> 172.21.15.44:0/3501313723 conn(0x7f7d2c0faa30 msgr2=0x7f7d2c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.564+0000 7f7d3270d700 1 -- 172.21.15.44:0/3501313723 shutdown_connections 2023-09-10T00:40:29.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.564+0000 7f7d3270d700 1 -- 172.21.15.44:0/3501313723 wait complete. 2023-09-10T00:40:29.578 DEBUG:tasks.fwd_scrub:Yielding 2023-09-10T00:40:29.578 INFO:teuthology.run_tasks:Running task exec... 2023-09-10T00:40:29.590 INFO:teuthology.task.exec:Executing custom commands... 2023-09-10T00:40:29.590 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2023-09-10T00:40:29.591 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-09-10T00:40:29.591 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-09-10T00:40:29.592 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-10T00:40:29.592 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-10T00:40:29.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.721+0000 7f1bceee7700 1 -- 172.21.15.44:0/2891908776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc81018d0 msgr2=0x7f1bc8101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.721+0000 7f1bceee7700 1 --2- 172.21.15.44:0/2891908776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc81018d0 0x7f1bc8101d30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009990 tx=0x7f1bb4009ca0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.721+0000 7f1bceee7700 1 -- 172.21.15.44:0/2891908776 shutdown_connections 2023-09-10T00:40:29.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.721+0000 7f1bceee7700 1 --2- 172.21.15.44:0/2891908776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc8102270 0x7f1bc8108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.721+0000 7f1bceee7700 1 --2- 172.21.15.44:0/2891908776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bc80fe900 0x7f1bc80fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.721+0000 7f1bceee7700 1 --2- 172.21.15.44:0/2891908776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc81018d0 0x7f1bc8101d30 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.721+0000 7f1bceee7700 1 -- 172.21.15.44:0/2891908776 >> 172.21.15.44:0/2891908776 conn(0x7f1bc80fa210 msgr2=0x7f1bc80fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:29.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.721+0000 7f1bceee7700 1 -- 172.21.15.44:0/2891908776 shutdown_connections 2023-09-10T00:40:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.721+0000 7f1bceee7700 1 -- 172.21.15.44:0/2891908776 wait complete. 2023-09-10T00:40:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.721+0000 7f1bceee7700 1 Processor -- start 2023-09-10T00:40:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.721+0000 7f1bceee7700 1 -- start start 2023-09-10T00:40:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.721+0000 7f1bceee7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc80fe900 0x7f1bc81962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.721+0000 7f1bceee7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bc81018d0 0x7f1bc8196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bceee7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc8102270 0x7f1bc8197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bceee7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc8197ce0 con 0x7f1bc8102270 2023-09-10T00:40:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bceee7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc8197e50 con 0x7f1bc80fe900 2023-09-10T00:40:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bceee7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc8198010 con 0x7f1bc81018d0 2023-09-10T00:40:29.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bcd484700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc8102270 0x7f1bc8197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bcd484700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc8102270 0x7f1bc8197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48540/0 (socket says 172.21.15.44:48540) 2023-09-10T00:40:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bcd484700 1 -- 172.21.15.44:0/914189993 learned_addr learned my addr 172.21.15.44:0/914189993 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bccc83700 1 --2- 172.21.15.44:0/914189993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc80fe900 0x7f1bc81962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bc7fff700 1 --2- 172.21.15.44:0/914189993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bc81018d0 0x7f1bc8196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bcd484700 1 -- 172.21.15.44:0/914189993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bc81018d0 msgr2=0x7f1bc8196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bcd484700 1 --2- 172.21.15.44:0/914189993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bc81018d0 0x7f1bc8196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bcd484700 1 -- 172.21.15.44:0/914189993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc80fe900 msgr2=0x7f1bc81962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bcd484700 1 --2- 172.21.15.44:0/914189993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc80fe900 0x7f1bc81962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bcd484700 1 -- 172.21.15.44:0/914189993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb40095f0 con 0x7f1bc8102270 2023-09-10T00:40:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bcd484700 1 --2- 172.21.15.44:0/914189993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc8102270 0x7f1bc8197680 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0007ee0 tx=0x7f1bc000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bc5ffb700 1 -- 172.21.15.44:0/914189993 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0017070 con 0x7f1bc8102270 2023-09-10T00:40:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bceee7700 1 -- 172.21.15.44:0/914189993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc81982f0 con 0x7f1bc8102270 2023-09-10T00:40:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bc5ffb700 1 -- 172.21.15.44:0/914189993 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1bc000f040 con 0x7f1bc8102270 2023-09-10T00:40:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.722+0000 7f1bc5ffb700 1 -- 172.21.15.44:0/914189993 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0005710 con 0x7f1bc8102270 2023-09-10T00:40:29.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.723+0000 7f1bceee7700 1 -- 172.21.15.44:0/914189993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc819eab0 con 0x7f1bc8102270 2023-09-10T00:40:29.730 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.724+0000 7f1bceee7700 1 -- 172.21.15.44:0/914189993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc8107660 con 0x7f1bc8102270 2023-09-10T00:40:29.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.725+0000 7fa87ed95700 1 -- 172.21.15.44:0/1095443611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8780feea0 msgr2=0x7fa8781012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.725+0000 7fa87ed95700 1 --2- 172.21.15.44:0/1095443611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8780feea0 0x7fa8781012d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0098f0 tx=0x7fa86c009c00 comp rx=0 tx=0).stop 2023-09-10T00:40:29.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.725+0000 7fa87ed95700 1 -- 172.21.15.44:0/1095443611 shutdown_connections 2023-09-10T00:40:29.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.725+0000 7fa87ed95700 1 --2- 172.21.15.44:0/1095443611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8781041e0 0x7fa878104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.725+0000 7fa87ed95700 1 --2- 172.21.15.44:0/1095443611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8780feea0 0x7fa8781012d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.725+0000 7fa87ed95700 1 --2- 172.21.15.44:0/1095443611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa878101810 0x7fa878103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.725+0000 7fa87ed95700 1 -- 172.21.15.44:0/1095443611 >> 172.21.15.44:0/1095443611 conn(0x7fa8780faac0 msgr2=0x7fa8780fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:29.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.725+0000 7fa87ed95700 1 -- 172.21.15.44:0/1095443611 shutdown_connections 2023-09-10T00:40:29.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.725+0000 7fa87ed95700 1 -- 172.21.15.44:0/1095443611 wait complete. 2023-09-10T00:40:29.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.725+0000 7f1bc5ffb700 1 -- 172.21.15.44:0/914189993 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f1bc0005870 con 0x7f1bc8102270 2023-09-10T00:40:29.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.725+0000 7fa87ed95700 1 Processor -- start 2023-09-10T00:40:29.734 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.725+0000 7fa87ed95700 1 -- start start 2023-09-10T00:40:29.734 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.725+0000 7f1bc5ffb700 1 --2- 172.21.15.44:0/914189993 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1bb80a0640 0x7f1bb80a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.735 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.725+0000 7fa87ed95700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8780feea0 0x7fa87819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.735 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa87ed95700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa878101810 0x7fa87819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.735 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa87ed95700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8781041e0 0x7fa8781a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.735 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa87ed95700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8781a43d0 con 0x7fa8780feea0 2023-09-10T00:40:29.735 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa87ed95700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8781a4510 con 0x7fa8781041e0 2023-09-10T00:40:29.736 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa87ed95700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8781a46a0 con 0x7fa878101810 2023-09-10T00:40:29.736 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7f1bc5ffb700 1 -- 172.21.15.44:0/914189993 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f1bc00cb220 con 0x7f1bc8102270 2023-09-10T00:40:29.737 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa87cb31700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8780feea0 0x7fa87819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.737 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa87cb31700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8780feea0 0x7fa87819c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48554/0 (socket says 172.21.15.44:48554) 2023-09-10T00:40:29.737 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa87cb31700 1 -- 172.21.15.44:0/4111442538 learned_addr learned my addr 172.21.15.44:0/4111442538 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:29.737 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7f1bccc83700 1 --2- 172.21.15.44:0/914189993 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1bb80a0640 0x7f1bb80a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.739 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa877fff700 1 --2- 172.21.15.44:0/4111442538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa878101810 0x7fa87819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.739 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa87d332700 1 --2- 172.21.15.44:0/4111442538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8781041e0 0x7fa8781a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.739 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa87cb31700 1 -- 172.21.15.44:0/4111442538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa878101810 msgr2=0x7fa87819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.739 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa87cb31700 1 --2- 172.21.15.44:0/4111442538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa878101810 0x7fa87819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.739 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa87cb31700 1 -- 172.21.15.44:0/4111442538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8781041e0 msgr2=0x7fa8781a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa87cb31700 1 --2- 172.21.15.44:0/4111442538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8781041e0 0x7fa8781a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa87cb31700 1 -- 172.21.15.44:0/4111442538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c009520 con 0x7fa8780feea0 2023-09-10T00:40:29.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa877fff700 1 --2- 172.21.15.44:0/4111442538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa878101810 0x7fa87819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:40:29.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7f1bccc83700 1 --2- 172.21.15.44:0/914189993 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1bb80a0640 0x7f1bb80a2b00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc005fd0 tx=0x7f1bbc005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:29.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.726+0000 7fa87cb31700 1 --2- 172.21.15.44:0/4111442538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8780feea0 0x7fa87819c850 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009230 tx=0x7fa86c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:29.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.730+0000 7fa875ffb700 1 -- 172.21.15.44:0/4111442538 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c01c070 con 0x7fa8780feea0 2023-09-10T00:40:29.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.730+0000 7fa87ed95700 1 -- 172.21.15.44:0/4111442538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8781a4920 con 0x7fa8780feea0 2023-09-10T00:40:29.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.730+0000 7f1bc5ffb700 1 -- 172.21.15.44:0/914189993 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1bc0094990 con 0x7f1bc8102270 2023-09-10T00:40:29.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.730+0000 7fa87ed95700 1 -- 172.21.15.44:0/4111442538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8781a4da0 con 0x7fa8780feea0 2023-09-10T00:40:29.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.730+0000 7fa875ffb700 1 -- 172.21.15.44:0/4111442538 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa86c005040 con 0x7fa8780feea0 2023-09-10T00:40:29.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.730+0000 7fa875ffb700 1 -- 172.21.15.44:0/4111442538 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0038e0 con 0x7fa8780feea0 2023-09-10T00:40:29.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.733+0000 7fa875ffb700 1 -- 172.21.15.44:0/4111442538 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fa86c020030 con 0x7fa8780feea0 2023-09-10T00:40:29.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.734+0000 7fa875ffb700 1 --2- 172.21.15.44:0/4111442538 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa8680a0640 0x7fa8680a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.734+0000 7fa877fff700 1 --2- 172.21.15.44:0/4111442538 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa8680a0640 0x7fa8680a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.734+0000 7fa88af5bb80 1 Processor -- start 2023-09-10T00:40:29.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.734+0000 7fa88af5bb80 1 -- start start 2023-09-10T00:40:29.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa88af5bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cddc80 0x557571cde0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa88af5bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571cdc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa88af5bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cdce80 0x557571cdd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa88af5bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557571c357b0 con 0x557571cde620 2023-09-10T00:40:29.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa88af5bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557571c330d0 con 0x557571cddc80 2023-09-10T00:40:29.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa88af5bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557571c2e9c0 con 0x557571cdce80 2023-09-10T00:40:29.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa877fff700 1 --2- 172.21.15.44:0/4111442538 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa8680a0640 0x7fa8680a2b00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa864011490 tx=0x7fa864009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:29.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa87d332700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cdce80 0x557571cdd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa87d332700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cdce80 0x557571cdd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:55444/0 (socket says 172.21.15.44:55444) 2023-09-10T00:40:29.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa87d332700 1 -- 172.21.15.44:0/4030972868 learned_addr learned my addr 172.21.15.44:0/4030972868 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:29.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa877fff700 1 --2- 172.21.15.44:0/4030972868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571cdc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa87cb31700 1 --2- 172.21.15.44:0/4030972868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cddc80 0x557571cde0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa877fff700 1 -- 172.21.15.44:0/4030972868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cdce80 msgr2=0x557571cdd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:29.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa877fff700 1 --2- 172.21.15.44:0/4030972868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cdce80 0x557571cdd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa877fff700 1 -- 172.21.15.44:0/4030972868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cddc80 msgr2=0x557571cde0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:29.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa877fff700 1 --2- 172.21.15.44:0/4030972868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cddc80 0x557571cde0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa877fff700 1 -- 172.21.15.44:0/4030972868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557571a5ae60 con 0x557571cde620 2023-09-10T00:40:29.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.735+0000 7fa877fff700 1 --2- 172.21.15.44:0/4030972868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571cdc880 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa864004270 tx=0x7fa864014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ca83841bd73517 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:29.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.736+0000 7fa875ffb700 1 -- 172.21.15.44:0/4111442538 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7fa86c021380 con 0x7fa8780feea0 2023-09-10T00:40:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.736+0000 7fa862ffd700 1 -- 172.21.15.44:0/4030972868 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864003d10 con 0x557571cde620 2023-09-10T00:40:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.736+0000 7fa862ffd700 1 -- 172.21.15.44:0/4030972868 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa864003e70 con 0x557571cde620 2023-09-10T00:40:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.736+0000 7fa862ffd700 1 -- 172.21.15.44:0/4030972868 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86401e650 con 0x557571cde620 2023-09-10T00:40:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.737+0000 7fa88af5bb80 1 -- 172.21.15.44:0/4030972868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 msgr2=0x557571cdc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.737+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/4030972868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571cdc880 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa864004270 tx=0x7fa864014000 comp rx=0 tx=0).stop 2023-09-10T00:40:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.737+0000 7fa88af5bb80 1 -- 172.21.15.44:0/4030972868 shutdown_connections 2023-09-10T00:40:29.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.737+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/4030972868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cdce80 0x557571cdd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.737+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/4030972868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cddc80 0x557571cde0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.737+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/4030972868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571cdc880 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.737+0000 7fa88af5bb80 1 -- 172.21.15.44:0/4030972868 >> 172.21.15.44:0/4030972868 conn(0x557571c51110 msgr2=0x557571cd77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:29.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.737+0000 7fa88af5bb80 1 -- 172.21.15.44:0/4030972868 shutdown_connections 2023-09-10T00:40:29.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.737+0000 7fa88af5bb80 1 -- 172.21.15.44:0/4030972868 wait complete. 2023-09-10T00:40:29.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.737+0000 7fa88af5bb80 1 -- start start 2023-09-10T00:40:29.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.738+0000 7fa88af5bb80 1 Processor -- start 2023-09-10T00:40:29.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.738+0000 7fa88af5bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cdce80 0x557571ce66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.738+0000 7fa88af5bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cddc80 0x557571ce6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.738+0000 7fa88af5bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571cedc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.738+0000 7fa88af5bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557571bccbf0 con 0x557571cde620 2023-09-10T00:40:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.738+0000 7fa88af5bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557571bcc8f0 con 0x557571cdce80 2023-09-10T00:40:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.738+0000 7fa88af5bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557571ab09a0 con 0x557571cddc80 2023-09-10T00:40:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.738+0000 7fa88af5bb80 10 client.0 _ll_register_callbacks cb 0x557571b16b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:40:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.738+0000 7fa87d332700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571cedc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.738+0000 7fa87cb31700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cdce80 0x557571ce66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.738+0000 7fa87cb31700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cdce80 0x557571ce66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:50306/0 (socket says 172.21.15.44:50306) 2023-09-10T00:40:29.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.738+0000 7fa87d332700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571cedc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48570/0 (socket says 172.21.15.44:48570) 2023-09-10T00:40:29.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.738+0000 7fa87cb31700 1 -- 172.21.15.44:0/897135305 learned_addr learned my addr 172.21.15.44:0/897135305 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:29.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.738+0000 7fa877fff700 1 --2- 172.21.15.44:0/897135305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cddc80 0x557571ce6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.739+0000 7fa87d332700 1 -- 172.21.15.44:0/897135305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cddc80 msgr2=0x557571ce6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:29.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.739+0000 7fa87d332700 1 --2- 172.21.15.44:0/897135305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cddc80 0x557571ce6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.739+0000 7fa87d332700 1 -- 172.21.15.44:0/897135305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cdce80 msgr2=0x557571ce66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:29.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.739+0000 7fa87d332700 1 --2- 172.21.15.44:0/897135305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cdce80 0x557571ce66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.739+0000 7fa87d332700 1 -- 172.21.15.44:0/897135305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa864017070 con 0x557571cde620 2023-09-10T00:40:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.739+0000 7fa87d332700 1 --2- 172.21.15.44:0/897135305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571cedc20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa870006220 tx=0x7fa870007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2efaf6e402a9aa server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.739+0000 7fa857fff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:40:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.739+0000 7fa857fff700 1 -- 172.21.15.44:0/897135305 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000e040 con 0x557571cde620 2023-09-10T00:40:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.739+0000 7fa857fff700 1 -- 172.21.15.44:0/897135305 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7fa870014ca0 con 0x557571cde620 2023-09-10T00:40:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.739+0000 7fa88af5bb80 1 -- 172.21.15.44:0/897135305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557571b86570 con 0x557571cde620 2023-09-10T00:40:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.739+0000 7fa857fff700 1 -- 172.21.15.44:0/897135305 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa870014e60 con 0x557571cde620 2023-09-10T00:40:29.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.739+0000 7fa88af5bb80 10 client.15786 fetch_fsmap learned FSMap version 27 2023-09-10T00:40:29.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.739+0000 7fa88af5bb80 1 -- 172.21.15.44:0/897135305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557571a43ce0 con 0x557571cde620 2023-09-10T00:40:29.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.739+0000 7fa857fff700 1 -- 172.21.15.44:0/897135305 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557571b86570 con 0x557571cde620 2023-09-10T00:40:29.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.740+0000 7fa857fff700 1 -- 172.21.15.44:0/897135305 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa870004640 con 0x557571cde620 2023-09-10T00:40:29.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.740+0000 7fa88af5bb80 10 client.15786 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:40:29.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.740+0000 7fa88af5bb80 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:40:29.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.740+0000 7fa88af5bb80 20 client.15786 populate_metadata read hostname 'smithi044' 2023-09-10T00:40:29.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.740+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/897135305 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557571cf24e0 0x557571cf4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.740+0000 7fa88af5bb80 4 client.15786 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:40:29.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.740+0000 7fa88af5bb80 1 -- 172.21.15.44:0/897135305 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557571c43920 con 0x557571cf24e0 2023-09-10T00:40:29.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.740+0000 7fa87cb31700 1 --2- 172.21.15.44:0/897135305 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557571cf24e0 0x557571cf4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.740+0000 7fa857fff700 10 client.15786 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:40:29.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.740+0000 7fa87cb31700 1 --2- 172.21.15.44:0/897135305 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557571cf24e0 0x557571cf4900 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd78d38e3932344c server_cookie=bcd7f2829787f1c4 in_seq=0 out_seq=0 2023-09-10T00:40:29.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.741+0000 7fa857fff700 1 -- 172.21.15.44:0/897135305 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557571c43920 con 0x557571cf24e0 2023-09-10T00:40:29.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.741+0000 7fa857fff700 10 client.15786 handle_command_reply: tid=0 2023-09-10T00:40:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.741+0000 7fa88af5bb80 1 client.15786 shutdown 2023-09-10T00:40:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.741+0000 7fa88af5bb80 1 -- 172.21.15.44:0/897135305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 msgr2=0x557571cedc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.741+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/897135305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571cedc20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa870006220 tx=0x7fa870007ec0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.741+0000 7fa88af5bb80 1 -- 172.21.15.44:0/897135305 shutdown_connections 2023-09-10T00:40:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.741+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/897135305 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557571cf24e0 0x557571cf4900 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.741+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/897135305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571cedc20 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.741+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/897135305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cdce80 0x557571ce66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.741+0000 7fa857fff700 0 client.15786 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:40:29.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.741+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/897135305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cddc80 0x557571ce6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.741+0000 7fa88af5bb80 1 -- 172.21.15.44:0/897135305 >> 172.21.15.44:0/897135305 conn(0x557571c51110 msgr2=0x557571cd6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:29.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.741+0000 7fa88af5bb80 1 -- 172.21.15.44:0/897135305 shutdown_connections 2023-09-10T00:40:29.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.741+0000 7fa88af5bb80 1 -- 172.21.15.44:0/897135305 wait complete. 2023-09-10T00:40:29.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.741+0000 7fa88af5bb80 20 client.15786 trim_cache size 0 max 16384 2023-09-10T00:40:29.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa88af5bb80 1 Processor -- start 2023-09-10T00:40:29.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa88af5bb80 1 -- start start 2023-09-10T00:40:29.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa88af5bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cdce80 0x557571ce6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa88af5bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cddc80 0x557571cedbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa88af5bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571ce6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa88af5bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557571ce4cb0 con 0x557571cde620 2023-09-10T00:40:29.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa88af5bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557571ce41d0 con 0x557571cdce80 2023-09-10T00:40:29.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa88af5bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557571cf21e0 con 0x557571cddc80 2023-09-10T00:40:29.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa87d332700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571ce6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa87d332700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571ce6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48576/0 (socket says 172.21.15.44:48576) 2023-09-10T00:40:29.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa87d332700 1 -- 172.21.15.44:0/3502569847 learned_addr learned my addr 172.21.15.44:0/3502569847 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:29.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa877fff700 1 --2- 172.21.15.44:0/3502569847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cddc80 0x557571cedbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa87cb31700 1 --2- 172.21.15.44:0/3502569847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cdce80 0x557571ce6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa87d332700 1 -- 172.21.15.44:0/3502569847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cddc80 msgr2=0x557571cedbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:29.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa87d332700 1 --2- 172.21.15.44:0/3502569847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cddc80 0x557571cedbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa87d332700 1 -- 172.21.15.44:0/3502569847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cdce80 msgr2=0x557571ce6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:29.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa87d332700 1 --2- 172.21.15.44:0/3502569847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cdce80 0x557571ce6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa87d332700 1 -- 172.21.15.44:0/3502569847 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557571cf9550 con 0x557571cde620 2023-09-10T00:40:29.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa87cb31700 1 --2- 172.21.15.44:0/3502569847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cdce80 0x557571ce6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:29.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.759+0000 7fa87d332700 1 --2- 172.21.15.44:0/3502569847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571ce6bc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa870020920 tx=0x7fa870006080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a65b757ae3a6ac3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:29.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.760+0000 7fa8637fe700 1 -- 172.21.15.44:0/3502569847 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000e070 con 0x557571cde620 2023-09-10T00:40:29.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.760+0000 7fa8637fe700 1 -- 172.21.15.44:0/3502569847 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa87000f040 con 0x557571cde620 2023-09-10T00:40:29.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.760+0000 7fa8637fe700 1 -- 172.21.15.44:0/3502569847 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870020c00 con 0x557571cde620 2023-09-10T00:40:29.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.760+0000 7fa88af5bb80 1 -- 172.21.15.44:0/3502569847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 msgr2=0x557571ce6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.760+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/3502569847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571ce6bc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa870020920 tx=0x7fa870006080 comp rx=0 tx=0).stop 2023-09-10T00:40:29.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.760+0000 7fa88af5bb80 1 -- 172.21.15.44:0/3502569847 shutdown_connections 2023-09-10T00:40:29.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.760+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/3502569847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cde620 0x557571ce6bc0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.760+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/3502569847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cdce80 0x557571ce6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.760+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/3502569847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cddc80 0x557571cedbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.760+0000 7fa88af5bb80 1 -- 172.21.15.44:0/3502569847 >> 172.21.15.44:0/3502569847 conn(0x557571c51110 msgr2=0x557571ce0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:29.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.760+0000 7fa88af5bb80 1 -- 172.21.15.44:0/3502569847 shutdown_connections 2023-09-10T00:40:29.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.761+0000 7fa88af5bb80 1 -- 172.21.15.44:0/3502569847 wait complete. 2023-09-10T00:40:29.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.761+0000 7fa88af5bb80 1 -- start start 2023-09-10T00:40:29.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.761+0000 7fa88af5bb80 1 Processor -- start 2023-09-10T00:40:29.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.761+0000 7fa88af5bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cdce80 0x557571ceacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.761+0000 7fa88af5bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cddc80 0x557571ceb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.761+0000 7fa88af5bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cde620 0x557571cec210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.761+0000 7fa88af5bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557571cec750 con 0x557571cdce80 2023-09-10T00:40:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.761+0000 7fa88af5bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557571cec8c0 con 0x557571cde620 2023-09-10T00:40:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.761+0000 7fa88af5bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557571ceca80 con 0x557571cddc80 2023-09-10T00:40:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.761+0000 7fa88af5bb80 10 client.0 _ll_register_callbacks cb 0x7fa8700094e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:40:29.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.761+0000 7fa87cb31700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cdce80 0x557571ceacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.761+0000 7fa87cb31700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cdce80 0x557571ceacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48578/0 (socket says 172.21.15.44:48578) 2023-09-10T00:40:29.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.761+0000 7fa87cb31700 1 -- 172.21.15.44:0/2618665833 learned_addr learned my addr 172.21.15.44:0/2618665833 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:29.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa877fff700 1 --2- 172.21.15.44:0/2618665833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cddc80 0x557571ceb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa87d332700 1 --2- 172.21.15.44:0/2618665833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cde620 0x557571cec210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa87cb31700 1 -- 172.21.15.44:0/2618665833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cddc80 msgr2=0x557571ceb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:29.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa87cb31700 1 --2- 172.21.15.44:0/2618665833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cddc80 0x557571ceb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa87cb31700 1 -- 172.21.15.44:0/2618665833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cde620 msgr2=0x557571cec210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:29.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa87cb31700 1 --2- 172.21.15.44:0/2618665833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cde620 0x557571cec210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa87cb31700 1 -- 172.21.15.44:0/2618665833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557571a43ce0 con 0x557571cdce80 2023-09-10T00:40:29.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa87d332700 1 --2- 172.21.15.44:0/2618665833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cde620 0x557571cec210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:40:29.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa87cb31700 1 --2- 172.21.15.44:0/2618665833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cdce80 0x557571ceacc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00fa60 tx=0x7fa86c096040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4def3639f016e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:29.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa857fff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:40:29.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa857fff700 1 -- 172.21.15.44:0/2618665833 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0b6070 con 0x557571cdce80 2023-09-10T00:40:29.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa857fff700 1 -- 172.21.15.44:0/2618665833 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7fa86c0ca750 con 0x557571cdce80 2023-09-10T00:40:29.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa88af5bb80 1 -- 172.21.15.44:0/2618665833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa864003e70 con 0x557571cdce80 2023-09-10T00:40:29.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa857fff700 1 -- 172.21.15.44:0/2618665833 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa86c0661a0 con 0x557571cdce80 2023-09-10T00:40:29.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa88af5bb80 10 client.15792 fetch_fsmap learned FSMap version 27 2023-09-10T00:40:29.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa88af5bb80 1 -- 172.21.15.44:0/2618665833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557571a812d0 con 0x557571cdce80 2023-09-10T00:40:29.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.762+0000 7fa857fff700 1 -- 172.21.15.44:0/2618665833 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa864003e70 con 0x557571cdce80 2023-09-10T00:40:29.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.763+0000 7fa857fff700 1 -- 172.21.15.44:0/2618665833 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa86c096ad0 con 0x557571cdce80 2023-09-10T00:40:29.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.763+0000 7fa88af5bb80 10 client.15792 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:40:29.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.763+0000 7fa88af5bb80 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:40:29.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.763+0000 7fa88af5bb80 20 client.15792 populate_metadata read hostname 'smithi044' 2023-09-10T00:40:29.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.763+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/2618665833 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557571d48330 0x557571d4a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.763+0000 7fa88af5bb80 4 client.15792 mds_command: new command op to 24605 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-10T00:40:29.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.763+0000 7fa88af5bb80 1 -- 172.21.15.44:0/2618665833 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557571c279a0 con 0x557571d48330 2023-09-10T00:40:29.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.763+0000 7fa877fff700 1 --2- 172.21.15.44:0/2618665833 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557571d48330 0x557571d4a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.763+0000 7fa857fff700 10 client.15792 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:40:29.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.763+0000 7fa877fff700 1 --2- 172.21.15.44:0/2618665833 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557571d48330 0x557571d4a720 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1c9e82b762ec6c1 server_cookie=307abbcc5194a3a4 in_seq=0 out_seq=0 2023-09-10T00:40:29.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.764+0000 7fa857fff700 1 -- 172.21.15.44:0/2618665833 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557571c279a0 con 0x557571d48330 2023-09-10T00:40:29.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.764+0000 7fa857fff700 10 client.15792 handle_command_reply: tid=0 2023-09-10T00:40:29.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.764+0000 7fa88af5bb80 1 client.15792 shutdown 2023-09-10T00:40:29.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.765+0000 7fa88af5bb80 1 -- 172.21.15.44:0/2618665833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cdce80 msgr2=0x557571ceacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.765+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/2618665833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cdce80 0x557571ceacc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00fa60 tx=0x7fa86c096040 comp rx=0 tx=0).stop 2023-09-10T00:40:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.765+0000 7fa88af5bb80 1 -- 172.21.15.44:0/2618665833 shutdown_connections 2023-09-10T00:40:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.765+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/2618665833 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557571d48330 0x557571d4a720 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.765+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/2618665833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557571cde620 0x557571cec210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.765+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/2618665833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557571cdce80 0x557571ceacc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.765+0000 7fa88af5bb80 1 --2- 172.21.15.44:0/2618665833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557571cddc80 0x557571ceb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.765+0000 7fa88af5bb80 1 -- 172.21.15.44:0/2618665833 >> 172.21.15.44:0/2618665833 conn(0x557571c51110 msgr2=0x557571ce0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:29.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.765+0000 7fa857fff700 0 client.15792 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:40:29.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.765+0000 7fa88af5bb80 1 -- 172.21.15.44:0/2618665833 shutdown_connections 2023-09-10T00:40:29.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.765+0000 7fa88af5bb80 1 -- 172.21.15.44:0/2618665833 wait complete. 2023-09-10T00:40:29.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.765+0000 7fa88af5bb80 20 client.15792 trim_cache size 0 max 16384 2023-09-10T00:40:29.797 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:40:29.797 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2023-09-10T00:40:29.797 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "9c9b99ba-82ac-48f8-8e9f-389243527364", 2023-09-10T00:40:29.797 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2023-09-10T00:40:29.797 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:40:29.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.766+0000 7fa87ed95700 1 -- 172.21.15.44:0/4111442538 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa8680a0640 msgr2=0x7fa8680a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.766+0000 7fa87ed95700 1 --2- 172.21.15.44:0/4111442538 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa8680a0640 0x7fa8680a2b00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa864011490 tx=0x7fa864009260 comp rx=0 tx=0).stop 2023-09-10T00:40:29.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.766+0000 7fa87ed95700 1 -- 172.21.15.44:0/4111442538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8780feea0 msgr2=0x7fa87819c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.766+0000 7fa87ed95700 1 --2- 172.21.15.44:0/4111442538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8780feea0 0x7fa87819c850 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009230 tx=0x7fa86c005e50 comp rx=0 tx=0).stop 2023-09-10T00:40:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.766+0000 7fa87ed95700 1 -- 172.21.15.44:0/4111442538 shutdown_connections 2023-09-10T00:40:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.766+0000 7fa87ed95700 1 --2- 172.21.15.44:0/4111442538 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fa8680a0640 0x7fa8680a2b00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.766+0000 7fa87ed95700 1 --2- 172.21.15.44:0/4111442538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8781041e0 0x7fa8781a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.766+0000 7fa87ed95700 1 --2- 172.21.15.44:0/4111442538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8780feea0 0x7fa87819c850 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.766+0000 7fa87ed95700 1 --2- 172.21.15.44:0/4111442538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa878101810 0x7fa87819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.766+0000 7fa87ed95700 1 -- 172.21.15.44:0/4111442538 >> 172.21.15.44:0/4111442538 conn(0x7fa8780faac0 msgr2=0x7fa878102520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.766+0000 7fa87ed95700 1 -- 172.21.15.44:0/4111442538 shutdown_connections 2023-09-10T00:40:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.766+0000 7fa87ed95700 1 -- 172.21.15.44:0/4111442538 wait complete. 2023-09-10T00:40:29.801 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:40:29.827 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:29 smithi044 ceph-mon[98676]: pgmap v327: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 12 op/s; 5.6 KiB/s, 0 keys/s, 1 objects/s recovering 2023-09-10T00:40:29.827 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:29 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/283942998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:40:29.828 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:29 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3811064607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:40:29.828 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:29 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3501313723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.931+0000 7f0283abd700 1 -- 172.21.15.44:0/2790720522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f027c100630 msgr2=0x7f027c100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.931+0000 7f0283abd700 1 --2- 172.21.15.44:0/2790720522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f027c100630 0x7f027c100a50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f02780099c0 tx=0x7f0278009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.931+0000 7f0283abd700 1 -- 172.21.15.44:0/2790720522 shutdown_connections 2023-09-10T00:40:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.931+0000 7f0283abd700 1 --2- 172.21.15.44:0/2790720522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f027c1021c0 0x7f027c108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.931+0000 7f0283abd700 1 --2- 172.21.15.44:0/2790720522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f027c100630 0x7f027c100a50 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.931+0000 7f0283abd700 1 --2- 172.21.15.44:0/2790720522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f027c101820 0x7f027c101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.931+0000 7f0283abd700 1 -- 172.21.15.44:0/2790720522 >> 172.21.15.44:0/2790720522 conn(0x7f027c0fbb80 msgr2=0x7f027c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.931+0000 7f0283abd700 1 -- 172.21.15.44:0/2790720522 shutdown_connections 2023-09-10T00:40:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.931+0000 7f0283abd700 1 -- 172.21.15.44:0/2790720522 wait complete. 2023-09-10T00:40:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.931+0000 7f0283abd700 1 Processor -- start 2023-09-10T00:40:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0283abd700 1 -- start start 2023-09-10T00:40:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0283abd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f027c100630 0x7f027c196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0283abd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f027c101820 0x7f027c1967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0283abd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f027c1021c0 0x7f027c19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0283abd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c19dde0 con 0x7f027c100630 2023-09-10T00:40:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0283abd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c19df20 con 0x7f027c1021c0 2023-09-10T00:40:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0283abd700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c19e0b0 con 0x7f027c101820 2023-09-10T00:40:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0281859700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f027c100630 0x7f027c196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0281859700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f027c100630 0x7f027c196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48592/0 (socket says 172.21.15.44:48592) 2023-09-10T00:40:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0281859700 1 -- 172.21.15.44:0/1917410975 learned_addr learned my addr 172.21.15.44:0/1917410975 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f028205a700 1 --2- 172.21.15.44:0/1917410975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f027c1021c0 0x7f027c19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0281058700 1 --2- 172.21.15.44:0/1917410975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f027c101820 0x7f027c1967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0281859700 1 -- 172.21.15.44:0/1917410975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f027c101820 msgr2=0x7f027c1967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0281859700 1 --2- 172.21.15.44:0/1917410975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f027c101820 0x7f027c1967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0281859700 1 -- 172.21.15.44:0/1917410975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f027c1021c0 msgr2=0x7f027c19d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0281859700 1 --2- 172.21.15.44:0/1917410975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f027c1021c0 0x7f027c19d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0281859700 1 -- 172.21.15.44:0/1917410975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02780095f0 con 0x7f027c100630 2023-09-10T00:40:29.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.932+0000 7f0281859700 1 --2- 172.21.15.44:0/1917410975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f027c100630 0x7f027c196260 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0278009f50 tx=0x7f0278003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:29.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.933+0000 7f0272ffd700 1 -- 172.21.15.44:0/1917410975 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027801d070 con 0x7f027c100630 2023-09-10T00:40:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.933+0000 7f0272ffd700 1 -- 172.21.15.44:0/1917410975 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f027800bda0 con 0x7f027c100630 2023-09-10T00:40:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.933+0000 7f0283abd700 1 -- 172.21.15.44:0/1917410975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027c19e330 con 0x7f027c100630 2023-09-10T00:40:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.933+0000 7f0272ffd700 1 -- 172.21.15.44:0/1917410975 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0278022ba0 con 0x7f027c100630 2023-09-10T00:40:29.945 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.933+0000 7f0283abd700 1 -- 172.21.15.44:0/1917410975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f027c19e7e0 con 0x7f027c100630 2023-09-10T00:40:29.945 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.935+0000 7f0272ffd700 1 -- 172.21.15.44:0/1917410975 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f0278003e90 con 0x7f027c100630 2023-09-10T00:40:29.946 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.936+0000 7f0272ffd700 1 --2- 172.21.15.44:0/1917410975 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f02680a0790 0x7f02680a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.946 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.936+0000 7f0281058700 1 --2- 172.21.15.44:0/1917410975 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f02680a0790 0x7f02680a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.946 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.936+0000 7f0272ffd700 1 -- 172.21.15.44:0/1917410975 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f02780cd350 con 0x7f027c100630 2023-09-10T00:40:29.946 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.936+0000 7f028fc83b80 1 Processor -- start 2023-09-10T00:40:29.946 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f0281058700 1 --2- 172.21.15.44:0/1917410975 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f02680a0790 0x7f02680a2c50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f026c005950 tx=0x7f026c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f028fc83b80 1 -- start start 2023-09-10T00:40:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f028fc83b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473abc00 0x55bc473ac020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f028fc83b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473ac5a0 0x55bc473aa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f028fc83b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473aae00 0x55bc473ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f028fc83b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc47303730 con 0x55bc473ac5a0 2023-09-10T00:40:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f028fc83b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc47301050 con 0x55bc473aae00 2023-09-10T00:40:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f028fc83b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc472fc940 con 0x55bc473abc00 2023-09-10T00:40:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f0281058700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473ac5a0 0x55bc473aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f0281859700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473abc00 0x55bc473ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f0281859700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473abc00 0x55bc473ac020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:55482/0 (socket says 172.21.15.44:55482) 2023-09-10T00:40:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f0281058700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473ac5a0 0x55bc473aa800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48602/0 (socket says 172.21.15.44:48602) 2023-09-10T00:40:29.949 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f0281859700 1 -- 172.21.15.44:0/74183200 learned_addr learned my addr 172.21.15.44:0/74183200 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:29.949 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f028205a700 1 --2- 172.21.15.44:0/74183200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473aae00 0x55bc473ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.949 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f0281859700 1 -- 172.21.15.44:0/74183200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473aae00 msgr2=0x55bc473ab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:29.949 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f0281859700 1 --2- 172.21.15.44:0/74183200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473aae00 0x55bc473ab2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.949 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f0281859700 1 -- 172.21.15.44:0/74183200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473ac5a0 msgr2=0x55bc473aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:29.950 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f0281859700 1 --2- 172.21.15.44:0/74183200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473ac5a0 0x55bc473aa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.950 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f0281859700 1 -- 172.21.15.44:0/74183200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc47128de0 con 0x55bc473abc00 2023-09-10T00:40:29.950 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.937+0000 7f0281058700 1 --2- 172.21.15.44:0/74183200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473ac5a0 0x55bc473aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:40:29.950 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.938+0000 7f0281859700 1 --2- 172.21.15.44:0/74183200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473abc00 0x55bc473ac020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f027809a630 tx=0x7f0278009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dd42dac32b4d99d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:29.950 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.938+0000 7f0267fff700 1 -- 172.21.15.44:0/74183200 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027809b030 con 0x55bc473abc00 2023-09-10T00:40:29.950 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.938+0000 7f0267fff700 1 -- 172.21.15.44:0/74183200 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f027809a930 con 0x55bc473abc00 2023-09-10T00:40:29.951 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.938+0000 7f0267fff700 1 -- 172.21.15.44:0/74183200 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0278098cd0 con 0x55bc473abc00 2023-09-10T00:40:29.951 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.938+0000 7f028fc83b80 1 -- 172.21.15.44:0/74183200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473abc00 msgr2=0x55bc473ac020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.951 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.938+0000 7f028fc83b80 1 --2- 172.21.15.44:0/74183200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473abc00 0x55bc473ac020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f027809a630 tx=0x7f0278009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.951 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.938+0000 7f028fc83b80 1 -- 172.21.15.44:0/74183200 shutdown_connections 2023-09-10T00:40:29.951 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.938+0000 7f028fc83b80 1 --2- 172.21.15.44:0/74183200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473aae00 0x55bc473ab2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.938+0000 7f028fc83b80 1 --2- 172.21.15.44:0/74183200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473abc00 0x55bc473ac020 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.938+0000 7f028fc83b80 1 --2- 172.21.15.44:0/74183200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473ac5a0 0x55bc473aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.938+0000 7f028fc83b80 1 -- 172.21.15.44:0/74183200 >> 172.21.15.44:0/74183200 conn(0x55bc4731f090 msgr2=0x55bc473a5720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:29.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.939+0000 7f028fc83b80 1 -- 172.21.15.44:0/74183200 shutdown_connections 2023-09-10T00:40:29.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.939+0000 7f028fc83b80 1 -- 172.21.15.44:0/74183200 wait complete. 2023-09-10T00:40:29.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.939+0000 7f028fc83b80 1 -- start start 2023-09-10T00:40:29.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.939+0000 7f028fc83b80 1 Processor -- start 2023-09-10T00:40:29.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f028fc83b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473aae00 0x55bc473b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f028fc83b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473abc00 0x55bc473b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f028fc83b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473ac5a0 0x55bc473bb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f028fc83b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc4729ab70 con 0x55bc473ac5a0 2023-09-10T00:40:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f028fc83b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc4729a870 con 0x55bc473abc00 2023-09-10T00:40:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f028fc83b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc4717e920 con 0x55bc473aae00 2023-09-10T00:40:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f028fc83b80 10 client.0 _ll_register_callbacks cb 0x55bc471e4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:40:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f028205a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473ac5a0 0x55bc473bb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f028205a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473ac5a0 0x55bc473bb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48610/0 (socket says 172.21.15.44:48610) 2023-09-10T00:40:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f028205a700 1 -- 172.21.15.44:0/389565064 learned_addr learned my addr 172.21.15.44:0/389565064 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f0281859700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473aae00 0x55bc473b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f0281058700 1 --2- 172.21.15.44:0/389565064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473abc00 0x55bc473b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f028205a700 1 -- 172.21.15.44:0/389565064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473aae00 msgr2=0x55bc473b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f028205a700 1 --2- 172.21.15.44:0/389565064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473aae00 0x55bc473b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f028205a700 1 -- 172.21.15.44:0/389565064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473abc00 msgr2=0x55bc473b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f028205a700 1 --2- 172.21.15.44:0/389565064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473abc00 0x55bc473b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f028205a700 1 -- 172.21.15.44:0/389565064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026c010c50 con 0x55bc473ac5a0 2023-09-10T00:40:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f0281859700 1 --2- 172.21.15.44:0/389565064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473aae00 0x55bc473b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:40:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f028205a700 1 --2- 172.21.15.44:0/389565064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473ac5a0 0x55bc473bb7e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0274009f50 tx=0x7f0274004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ea6c30d323771f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f02657fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:40:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.940+0000 7f02657fa700 1 -- 172.21.15.44:0/389565064 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0274007910 con 0x55bc473ac5a0 2023-09-10T00:40:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.941+0000 7f02657fa700 1 -- 172.21.15.44:0/389565064 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f0274007a70 con 0x55bc473ac5a0 2023-09-10T00:40:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.941+0000 7f028fc83b80 1 -- 172.21.15.44:0/389565064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc472544f0 con 0x55bc473ac5a0 2023-09-10T00:40:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.941+0000 7f02657fa700 1 -- 172.21.15.44:0/389565064 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0274004920 con 0x55bc473ac5a0 2023-09-10T00:40:29.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.941+0000 7f028fc83b80 10 client.15804 fetch_fsmap learned FSMap version 27 2023-09-10T00:40:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.941+0000 7f028fc83b80 1 -- 172.21.15.44:0/389565064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f027c19e7e0 con 0x55bc473ac5a0 2023-09-10T00:40:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.941+0000 7f02657fa700 1 -- 172.21.15.44:0/389565064 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bc472544f0 con 0x55bc473ac5a0 2023-09-10T00:40:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.941+0000 7f02657fa700 1 -- 172.21.15.44:0/389565064 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0274014040 con 0x55bc473ac5a0 2023-09-10T00:40:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.941+0000 7f028fc83b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:40:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.941+0000 7f028fc83b80 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:40:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.941+0000 7f028fc83b80 20 client.15804 populate_metadata read hostname 'smithi044' 2023-09-10T00:40:29.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.941+0000 7f028fc83b80 1 --2- 172.21.15.44:0/389565064 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc473c00c0 0x55bc473c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.941+0000 7f028fc83b80 4 client.15804 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:40:29.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.941+0000 7f028fc83b80 1 -- 172.21.15.44:0/389565064 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc473118a0 con 0x55bc473c00c0 2023-09-10T00:40:29.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.941+0000 7f0281859700 1 --2- 172.21.15.44:0/389565064 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc473c00c0 0x55bc473c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.942+0000 7f02657fa700 10 client.15804 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:40:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.942+0000 7f0281859700 1 --2- 172.21.15.44:0/389565064 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc473c00c0 0x55bc473c24e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=413f04119b4d6b15 server_cookie=42bf60f939f2be0a in_seq=0 out_seq=0 2023-09-10T00:40:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.943+0000 7f02657fa700 1 -- 172.21.15.44:0/389565064 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bc473118a0 con 0x55bc473c00c0 2023-09-10T00:40:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.943+0000 7f02657fa700 10 client.15804 handle_command_reply: tid=0 2023-09-10T00:40:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.943+0000 7f028fc83b80 1 client.15804 shutdown 2023-09-10T00:40:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.943+0000 7f028fc83b80 1 -- 172.21.15.44:0/389565064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473ac5a0 msgr2=0x55bc473bb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.943+0000 7f028fc83b80 1 --2- 172.21.15.44:0/389565064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473ac5a0 0x55bc473bb7e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0274009f50 tx=0x7f0274004320 comp rx=0 tx=0).stop 2023-09-10T00:40:29.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.943+0000 7f028fc83b80 1 -- 172.21.15.44:0/389565064 shutdown_connections 2023-09-10T00:40:29.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.943+0000 7f028fc83b80 1 --2- 172.21.15.44:0/389565064 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc473c00c0 0x55bc473c24e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.943+0000 7f028fc83b80 1 --2- 172.21.15.44:0/389565064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473ac5a0 0x55bc473bb7e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.943+0000 7f028fc83b80 1 --2- 172.21.15.44:0/389565064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473aae00 0x55bc473b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.943+0000 7f02657fa700 0 client.15804 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:40:29.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.943+0000 7f028fc83b80 1 --2- 172.21.15.44:0/389565064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473abc00 0x55bc473b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.943+0000 7f028fc83b80 1 -- 172.21.15.44:0/389565064 >> 172.21.15.44:0/389565064 conn(0x55bc4731f090 msgr2=0x55bc473a4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:29.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.943+0000 7f028fc83b80 1 -- 172.21.15.44:0/389565064 shutdown_connections 2023-09-10T00:40:29.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.943+0000 7f028fc83b80 1 -- 172.21.15.44:0/389565064 wait complete. 2023-09-10T00:40:29.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.943+0000 7f028fc83b80 20 client.15804 trim_cache size 0 max 16384 2023-09-10T00:40:29.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.960+0000 7f028fc83b80 1 Processor -- start 2023-09-10T00:40:29.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.960+0000 7f028fc83b80 1 -- start start 2023-09-10T00:40:29.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.960+0000 7f028fc83b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473aae00 0x55bc473aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.960+0000 7f028fc83b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473abc00 0x55bc473af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.960+0000 7f028fc83b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473ac5a0 0x55bc473b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.960+0000 7f028fc83b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc473b3710 con 0x55bc473abc00 2023-09-10T00:40:29.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.960+0000 7f028fc83b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc473bbb90 con 0x55bc473aae00 2023-09-10T00:40:29.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.960+0000 7f028fc83b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc473b3e00 con 0x55bc473ac5a0 2023-09-10T00:40:29.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.960+0000 7f0281058700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473abc00 0x55bc473af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.960+0000 7f0281058700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473abc00 0x55bc473af3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48616/0 (socket says 172.21.15.44:48616) 2023-09-10T00:40:29.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.960+0000 7f0281058700 1 -- 172.21.15.44:0/4126652107 learned_addr learned my addr 172.21.15.44:0/4126652107 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:29.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.960+0000 7f0281859700 1 --2- 172.21.15.44:0/4126652107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473aae00 0x55bc473aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.961+0000 7f028205a700 1 --2- 172.21.15.44:0/4126652107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473ac5a0 0x55bc473b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.961+0000 7f0281058700 1 -- 172.21.15.44:0/4126652107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473ac5a0 msgr2=0x55bc473b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:29.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.961+0000 7f0281058700 1 --2- 172.21.15.44:0/4126652107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473ac5a0 0x55bc473b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.961+0000 7f0281058700 1 -- 172.21.15.44:0/4126652107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473aae00 msgr2=0x55bc473aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:29.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.961+0000 7f0281058700 1 --2- 172.21.15.44:0/4126652107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473aae00 0x55bc473aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.961+0000 7f0281058700 1 -- 172.21.15.44:0/4126652107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc47111c60 con 0x55bc473abc00 2023-09-10T00:40:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.961+0000 7f0281058700 1 --2- 172.21.15.44:0/4126652107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473abc00 0x55bc473af3c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f026c011790 tx=0x7f026c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e6917ff91dff00 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.961+0000 7f0270d89700 1 -- 172.21.15.44:0/4126652107 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c024070 con 0x55bc473abc00 2023-09-10T00:40:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.961+0000 7f0270d89700 1 -- 172.21.15.44:0/4126652107 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f026c015070 con 0x55bc473abc00 2023-09-10T00:40:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.961+0000 7f0270d89700 1 -- 172.21.15.44:0/4126652107 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c0056b0 con 0x55bc473abc00 2023-09-10T00:40:29.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 -- 172.21.15.44:0/4126652107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473abc00 msgr2=0x55bc473af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 --2- 172.21.15.44:0/4126652107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473abc00 0x55bc473af3c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f026c011790 tx=0x7f026c013000 comp rx=0 tx=0).stop 2023-09-10T00:40:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 -- 172.21.15.44:0/4126652107 shutdown_connections 2023-09-10T00:40:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 --2- 172.21.15.44:0/4126652107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473ac5a0 0x55bc473b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 --2- 172.21.15.44:0/4126652107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473aae00 0x55bc473aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 --2- 172.21.15.44:0/4126652107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473abc00 0x55bc473af3c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 -- 172.21.15.44:0/4126652107 >> 172.21.15.44:0/4126652107 conn(0x55bc4731f090 msgr2=0x55bc473b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:29.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 -- 172.21.15.44:0/4126652107 shutdown_connections 2023-09-10T00:40:29.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 -- 172.21.15.44:0/4126652107 wait complete. 2023-09-10T00:40:29.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 -- start start 2023-09-10T00:40:29.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 Processor -- start 2023-09-10T00:40:29.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473aae00 0x55bc47410110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473abc00 0x55bc474106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473ac5a0 0x55bc47417460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc474179a0 con 0x55bc473aae00 2023-09-10T00:40:29.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc47417ae0 con 0x55bc473abc00 2023-09-10T00:40:29.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.962+0000 7f028fc83b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc47417ca0 con 0x55bc473ac5a0 2023-09-10T00:40:29.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f028fc83b80 10 client.0 _ll_register_callbacks cb 0x7f0274009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:40:29.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f0281859700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473aae00 0x55bc47410110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f0281859700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473aae00 0x55bc47410110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48620/0 (socket says 172.21.15.44:48620) 2023-09-10T00:40:29.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f0281859700 1 -- 172.21.15.44:0/2878690054 learned_addr learned my addr 172.21.15.44:0/2878690054 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:29.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f0281058700 1 --2- 172.21.15.44:0/2878690054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473abc00 0x55bc474106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f028205a700 1 --2- 172.21.15.44:0/2878690054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473ac5a0 0x55bc47417460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f0281859700 1 -- 172.21.15.44:0/2878690054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473ac5a0 msgr2=0x55bc47417460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:29.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f0281859700 1 --2- 172.21.15.44:0/2878690054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473ac5a0 0x55bc47417460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f0281859700 1 -- 172.21.15.44:0/2878690054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473abc00 msgr2=0x55bc474106d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:29.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f0281859700 1 --2- 172.21.15.44:0/2878690054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473abc00 0x55bc474106d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f0281859700 1 -- 172.21.15.44:0/2878690054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026c011440 con 0x55bc473aae00 2023-09-10T00:40:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f028205a700 1 --2- 172.21.15.44:0/2878690054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473ac5a0 0x55bc47417460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f0281859700 1 --2- 172.21.15.44:0/2878690054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473aae00 0x55bc47410110 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f027800c010 tx=0x7f027809a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d12720d17b3a8f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f02657fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:40:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f02657fa700 1 -- 172.21.15.44:0/2878690054 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02780b2ea0 con 0x55bc473aae00 2023-09-10T00:40:29.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f02657fa700 1 -- 172.21.15.44:0/2878690054 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f02780a2910 con 0x55bc473aae00 2023-09-10T00:40:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f028fc83b80 1 -- 172.21.15.44:0/2878690054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f027809a930 con 0x55bc473aae00 2023-09-10T00:40:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f02657fa700 1 -- 172.21.15.44:0/2878690054 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02780cd160 con 0x55bc473aae00 2023-09-10T00:40:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f028fc83b80 10 client.15810 fetch_fsmap learned FSMap version 27 2023-09-10T00:40:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.963+0000 7f028fc83b80 1 -- 172.21.15.44:0/2878690054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bc4714f250 con 0x55bc473aae00 2023-09-10T00:40:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.964+0000 7f02657fa700 1 -- 172.21.15.44:0/2878690054 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f027809a930 con 0x55bc473aae00 2023-09-10T00:40:29.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.964+0000 7f02657fa700 1 -- 172.21.15.44:0/2878690054 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f02780b1070 con 0x55bc473aae00 2023-09-10T00:40:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.964+0000 7f028fc83b80 10 client.15810 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:40:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.964+0000 7f028fc83b80 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:40:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.964+0000 7f028fc83b80 20 client.15810 populate_metadata read hostname 'smithi044' 2023-09-10T00:40:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.964+0000 7f028fc83b80 1 --2- 172.21.15.44:0/2878690054 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc47419320 0x55bc4741b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.964+0000 7f028fc83b80 4 client.15810 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:40:29.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.964+0000 7f028fc83b80 1 -- 172.21.15.44:0/2878690054 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bc472f5920 con 0x55bc47419320 2023-09-10T00:40:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.964+0000 7f0281058700 1 --2- 172.21.15.44:0/2878690054 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc47419320 0x55bc4741b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f02657fa700 10 client.15810 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:40:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f0281058700 1 --2- 172.21.15.44:0/2878690054 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc47419320 0x55bc4741b740 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8e10cf81ab37284 server_cookie=c34fec445b04707c in_seq=0 out_seq=0 2023-09-10T00:40:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f02657fa700 1 -- 172.21.15.44:0/2878690054 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bc472f5920 con 0x55bc47419320 2023-09-10T00:40:29.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f02657fa700 10 client.15810 handle_command_reply: tid=0 2023-09-10T00:40:29.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f028fc83b80 1 client.15810 shutdown 2023-09-10T00:40:29.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f028fc83b80 1 -- 172.21.15.44:0/2878690054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473aae00 msgr2=0x55bc47410110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f028fc83b80 1 --2- 172.21.15.44:0/2878690054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473aae00 0x55bc47410110 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f027800c010 tx=0x7f027809a570 comp rx=0 tx=0).stop 2023-09-10T00:40:29.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f028fc83b80 1 -- 172.21.15.44:0/2878690054 shutdown_connections 2023-09-10T00:40:29.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f028fc83b80 1 --2- 172.21.15.44:0/2878690054 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc47419320 0x55bc4741b740 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f028fc83b80 1 --2- 172.21.15.44:0/2878690054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc473ac5a0 0x55bc47417460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f02657fa700 0 client.15810 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:40:29.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f028fc83b80 1 --2- 172.21.15.44:0/2878690054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc473aae00 0x55bc47410110 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f028fc83b80 1 --2- 172.21.15.44:0/2878690054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc473abc00 0x55bc474106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f028fc83b80 1 -- 172.21.15.44:0/2878690054 >> 172.21.15.44:0/2878690054 conn(0x55bc4731f090 msgr2=0x55bc473aee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:29.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f028fc83b80 1 -- 172.21.15.44:0/2878690054 shutdown_connections 2023-09-10T00:40:29.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f028fc83b80 1 -- 172.21.15.44:0/2878690054 wait complete. 2023-09-10T00:40:29.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.965+0000 7f028fc83b80 20 client.15810 trim_cache size 0 max 16384 2023-09-10T00:40:29.984 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:40:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:40:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:40:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "9c9b99ba-82ac-48f8-8e9f-389243527364": { 2023-09-10T00:40:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:40:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "9c9b99ba-82ac-48f8-8e9f-389243527364", 2023-09-10T00:40:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:40:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:40:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:40:29.986 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:40:29.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.966+0000 7f0283abd700 1 -- 172.21.15.44:0/1917410975 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f02680a0790 msgr2=0x7f02680a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.966+0000 7f0283abd700 1 --2- 172.21.15.44:0/1917410975 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f02680a0790 0x7f02680a2c50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f026c005950 tx=0x7f026c00b410 comp rx=0 tx=0).stop 2023-09-10T00:40:29.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.966+0000 7f0283abd700 1 -- 172.21.15.44:0/1917410975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f027c100630 msgr2=0x7f027c196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:29.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.966+0000 7f0283abd700 1 --2- 172.21.15.44:0/1917410975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f027c100630 0x7f027c196260 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0278009f50 tx=0x7f0278003a50 comp rx=0 tx=0).stop 2023-09-10T00:40:29.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.966+0000 7f0283abd700 1 -- 172.21.15.44:0/1917410975 shutdown_connections 2023-09-10T00:40:29.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.966+0000 7f0283abd700 1 --2- 172.21.15.44:0/1917410975 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f02680a0790 0x7f02680a2c50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.966+0000 7f0283abd700 1 --2- 172.21.15.44:0/1917410975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f027c1021c0 0x7f027c19d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.966+0000 7f0283abd700 1 --2- 172.21.15.44:0/1917410975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f027c100630 0x7f027c196260 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.966+0000 7f0283abd700 1 --2- 172.21.15.44:0/1917410975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f027c101820 0x7f027c1967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:29.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.966+0000 7f0283abd700 1 -- 172.21.15.44:0/1917410975 >> 172.21.15.44:0/1917410975 conn(0x7f027c0fbb80 msgr2=0x7f027c104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:29.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.966+0000 7f0283abd700 1 -- 172.21.15.44:0/1917410975 shutdown_connections 2023-09-10T00:40:29.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:29.966+0000 7f0283abd700 1 -- 172.21.15.44:0/1917410975 wait complete. 2023-09-10T00:40:29.990 INFO:tasks.cephfs.filesystem:scrub status for tag:9c9b99ba-82ac-48f8-8e9f-389243527364 - {'path': '/', 'tag': '9c9b99ba-82ac-48f8-8e9f-389243527364', 'options': 'recursive,force'} 2023-09-10T00:40:30.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.006+0000 7f1bceee7700 1 -- 172.21.15.44:0/914189993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f1bc8107040 con 0x7f1bc8102270 2023-09-10T00:40:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:29 smithi131 ceph-mon[103713]: pgmap v327: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 12 op/s; 5.6 KiB/s, 0 keys/s, 1 objects/s recovering 2023-09-10T00:40:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:29 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/283942998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:40:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:29 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3811064607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:40:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:29 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3501313723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:30.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:29 smithi099 ceph-mon[102692]: pgmap v327: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 12 op/s; 5.6 KiB/s, 0 keys/s, 1 objects/s recovering 2023-09-10T00:40:30.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:29 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/283942998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T00:40:30.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:29 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3811064607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:40:30.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:29 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3501313723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T00:40:30.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.741+0000 7f1bc5ffb700 1 -- 172.21.15.44:0/914189993 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f1bc0097fb0 con 0x7f1bc8102270 2023-09-10T00:40:30.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.744+0000 7f1bceee7700 1 -- 172.21.15.44:0/914189993 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1bb80a0640 msgr2=0x7f1bb80a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:30.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.744+0000 7f1bceee7700 1 --2- 172.21.15.44:0/914189993 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1bb80a0640 0x7f1bb80a2b00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc005fd0 tx=0x7f1bbc005e50 comp rx=0 tx=0).stop 2023-09-10T00:40:30.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.744+0000 7f1bceee7700 1 -- 172.21.15.44:0/914189993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc8102270 msgr2=0x7f1bc8197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:30.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.744+0000 7f1bceee7700 1 --2- 172.21.15.44:0/914189993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc8102270 0x7f1bc8197680 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0007ee0 tx=0x7f1bc000d3b0 comp rx=0 tx=0).stop 2023-09-10T00:40:30.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.746+0000 7f1bceee7700 1 -- 172.21.15.44:0/914189993 shutdown_connections 2023-09-10T00:40:30.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.746+0000 7f1bceee7700 1 --2- 172.21.15.44:0/914189993 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f1bb80a0640 0x7f1bb80a2b00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:30.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.746+0000 7f1bceee7700 1 --2- 172.21.15.44:0/914189993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc8102270 0x7f1bc8197680 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:30.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.746+0000 7f1bceee7700 1 --2- 172.21.15.44:0/914189993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc80fe900 0x7f1bc81962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:30.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.746+0000 7f1bceee7700 1 --2- 172.21.15.44:0/914189993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bc81018d0 0x7f1bc8196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:30.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.746+0000 7f1bceee7700 1 -- 172.21.15.44:0/914189993 >> 172.21.15.44:0/914189993 conn(0x7f1bc80fa210 msgr2=0x7f1bc80fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:30.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.746+0000 7f1bceee7700 1 -- 172.21.15.44:0/914189993 shutdown_connections 2023-09-10T00:40:30.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.746+0000 7f1bceee7700 1 -- 172.21.15.44:0/914189993 wait complete. 2023-09-10T00:40:30.767 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-09-10T00:40:30.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.951+0000 7fe61a0cd700 1 -- 172.21.15.44:0/2394988050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6140ff830 msgr2=0x7fe6140ffcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:30.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.951+0000 7fe61a0cd700 1 --2- 172.21.15.44:0/2394988050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6140ff830 0x7fe6140ffcb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe608009920 tx=0x7fe608009c30 comp rx=0 tx=0).stop 2023-09-10T00:40:30.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.951+0000 7fe61a0cd700 1 -- 172.21.15.44:0/2394988050 shutdown_connections 2023-09-10T00:40:30.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.951+0000 7fe61a0cd700 1 --2- 172.21.15.44:0/2394988050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6141001f0 0x7fe614100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:30.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.951+0000 7fe61a0cd700 1 --2- 172.21.15.44:0/2394988050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6140feed0 0x7fe6140ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:30.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.951+0000 7fe61a0cd700 1 --2- 172.21.15.44:0/2394988050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6140ff830 0x7fe6140ffcb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:30.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.951+0000 7fe61a0cd700 1 -- 172.21.15.44:0/2394988050 >> 172.21.15.44:0/2394988050 conn(0x7fe6140faa50 msgr2=0x7fe6140fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:30.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.951+0000 7fe61a0cd700 1 -- 172.21.15.44:0/2394988050 shutdown_connections 2023-09-10T00:40:30.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.951+0000 7fe61a0cd700 1 -- 172.21.15.44:0/2394988050 wait complete. 2023-09-10T00:40:30.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.952+0000 7fe61a0cd700 1 Processor -- start 2023-09-10T00:40:30.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.952+0000 7fe61a0cd700 1 -- start start 2023-09-10T00:40:30.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.952+0000 7fe61a0cd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6140feed0 0x7fe6141985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:30.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.952+0000 7fe61a0cd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6140ff830 0x7fe614198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:30.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.952+0000 7fe61a0cd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6141001f0 0x7fe6141999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:30.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.952+0000 7fe61a0cd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61419a010 con 0x7fe6140ff830 2023-09-10T00:40:30.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.952+0000 7fe61a0cd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61419a180 con 0x7fe6141001f0 2023-09-10T00:40:30.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.952+0000 7fe61a0cd700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61419a340 con 0x7fe6140feed0 2023-09-10T00:40:30.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.953+0000 7fe612ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6140ff830 0x7fe614198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:30.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.953+0000 7fe613fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6141001f0 0x7fe6141999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:30.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.953+0000 7fe612ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6140ff830 0x7fe614198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48660/0 (socket says 172.21.15.44:48660) 2023-09-10T00:40:30.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.953+0000 7fe612ffd700 1 -- 172.21.15.44:0/1850799773 learned_addr learned my addr 172.21.15.44:0/1850799773 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:30.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.953+0000 7fe6137fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6140feed0 0x7fe6141985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:30.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.953+0000 7fe613fff700 1 -- 172.21.15.44:0/1850799773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6140feed0 msgr2=0x7fe6141985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:30.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.953+0000 7fe613fff700 1 --2- 172.21.15.44:0/1850799773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6140feed0 0x7fe6141985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:30.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.953+0000 7fe613fff700 1 -- 172.21.15.44:0/1850799773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6140ff830 msgr2=0x7fe614198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:30.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.953+0000 7fe613fff700 1 --2- 172.21.15.44:0/1850799773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6140ff830 0x7fe614198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:30.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.953+0000 7fe613fff700 1 -- 172.21.15.44:0/1850799773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6040095f0 con 0x7fe6141001f0 2023-09-10T00:40:30.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.953+0000 7fe613fff700 1 --2- 172.21.15.44:0/1850799773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6141001f0 0x7fe6141999b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00efc0 tx=0x7fe60c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:30.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.953+0000 7fe610ff9700 1 -- 172.21.15.44:0/1850799773 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c018070 con 0x7fe6141001f0 2023-09-10T00:40:30.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.953+0000 7fe610ff9700 1 -- 172.21.15.44:0/1850799773 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe60c00f040 con 0x7fe6141001f0 2023-09-10T00:40:30.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.953+0000 7fe610ff9700 1 -- 172.21.15.44:0/1850799773 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00cba0 con 0x7fe6141001f0 2023-09-10T00:40:30.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.953+0000 7fe61a0cd700 1 -- 172.21.15.44:0/1850799773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe608009520 con 0x7fe6141001f0 2023-09-10T00:40:30.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.953+0000 7fe61a0cd700 1 -- 172.21.15.44:0/1850799773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6141a0c00 con 0x7fe6141001f0 2023-09-10T00:40:30.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.954+0000 7fe61a0cd700 1 -- 172.21.15.44:0/1850799773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe614109cb0 con 0x7fe6141001f0 2023-09-10T00:40:30.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.955+0000 7fe610ff9700 1 -- 172.21.15.44:0/1850799773 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7fe60c003a00 con 0x7fe6141001f0 2023-09-10T00:40:30.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.956+0000 7fe610ff9700 1 --2- 172.21.15.44:0/1850799773 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5fc0a0690 0x7fe5fc0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:30.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.956+0000 7fe6137fe700 1 -- 172.21.15.44:0/1850799773 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5fc0a0690 msgr2=0x7fe5fc0a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1488156576 2023-09-10T00:40:30.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.956+0000 7fe610ff9700 1 -- 172.21.15.44:0/1850799773 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7fe60c0cba50 con 0x7fe6141001f0 2023-09-10T00:40:30.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.956+0000 7fe6137fe700 1 --2- 172.21.15.44:0/1850799773 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5fc0a0690 0x7fe5fc0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-10T00:40:30.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:30.960+0000 7fe610ff9700 1 -- 172.21.15.44:0/1850799773 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe60c095250 con 0x7fe6141001f0 2023-09-10T00:40:31.004 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:30 smithi131 ceph-mon[103713]: scrub queued for path: / 2023-09-10T00:40:31.004 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:30 smithi131 ceph-mon[103713]: scrub summary: idle+waiting paths [/] 2023-09-10T00:40:31.005 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:30 smithi131 ceph-mon[103713]: scrub summary: active paths [/] 2023-09-10T00:40:31.005 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:30 smithi131 ceph-mon[103713]: scrub summary: idle+waiting paths [/] 2023-09-10T00:40:31.005 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:30 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/914189993' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-10T00:40:31.005 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:30 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: ignoring --setuser ceph since I am not root 2023-09-10T00:40:31.006 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:30 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: ignoring --setgroup ceph since I am not root 2023-09-10T00:40:31.006 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:30 smithi131 ceph-mgr[105889]: -- 172.21.15.131:0/2756826865 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5591883471e0 con 0x55918831cc00 2023-09-10T00:40:31.006 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:30 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:30.922+0000 7fa9fb48b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-10T00:40:31.006 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:31 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:31.004+0000 7fa9fb48b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-10T00:40:31.007 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:30 smithi099 ceph-mon[102692]: scrub queued for path: / 2023-09-10T00:40:31.007 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:30 smithi099 ceph-mon[102692]: scrub summary: idle+waiting paths [/] 2023-09-10T00:40:31.007 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:30 smithi099 ceph-mon[102692]: scrub summary: active paths [/] 2023-09-10T00:40:31.007 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:30 smithi099 ceph-mon[102692]: scrub summary: idle+waiting paths [/] 2023-09-10T00:40:31.008 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:30 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/914189993' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-10T00:40:31.008 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:30 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: ignoring --setuser ceph since I am not root 2023-09-10T00:40:31.008 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:30 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: ignoring --setgroup ceph since I am not root 2023-09-10T00:40:31.008 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:30 smithi099 ceph-mgr[104203]: -- 172.21.15.99:0/2097997163 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a01d3171e0 con 0x55a01d2ecc00 2023-09-10T00:40:31.008 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:30 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:30.922+0000 7f13458fb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-10T00:40:31.009 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:31 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:31.004+0000 7f13458fb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-10T00:40:31.055 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:30 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: ignoring --setuser ceph since I am not root 2023-09-10T00:40:31.055 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:30 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: ignoring --setgroup ceph since I am not root 2023-09-10T00:40:31.055 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:30 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:30.961+0000 7f34ba828200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-10T00:40:31.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:30 smithi044 ceph-mon[98676]: scrub queued for path: / 2023-09-10T00:40:31.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:30 smithi044 ceph-mon[98676]: scrub summary: idle+waiting paths [/] 2023-09-10T00:40:31.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:30 smithi044 ceph-mon[98676]: scrub summary: active paths [/] 2023-09-10T00:40:31.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:30 smithi044 ceph-mon[98676]: scrub summary: idle+waiting paths [/] 2023-09-10T00:40:31.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:30 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/914189993' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-10T00:40:31.157 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.157+0000 7fe6137fe700 1 -- 172.21.15.44:0/1850799773 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5fc0a0690 msgr2=0x7fe5fc0a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1488156576 2023-09-10T00:40:31.157 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.157+0000 7fe6137fe700 1 --2- 172.21.15.44:0/1850799773 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5fc0a0690 0x7fe5fc0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-10T00:40:31.163 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.163+0000 7fe61a0cd700 1 -- 172.21.15.44:0/1850799773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fe61404e3e0 con 0x7fe6141001f0 2023-09-10T00:40:31.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.171+0000 7fe610ff9700 1 -- 172.21.15.44:0/1850799773 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v44)=0 v44) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe60c098870 con 0x7fe6141001f0 2023-09-10T00:40:31.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.173+0000 7fe61a0cd700 1 -- 172.21.15.44:0/1850799773 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5fc0a0690 msgr2=0x7fe5fc0a2b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T00:40:31.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.173+0000 7fe61a0cd700 1 --2- 172.21.15.44:0/1850799773 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5fc0a0690 0x7fe5fc0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.173+0000 7fe61a0cd700 1 -- 172.21.15.44:0/1850799773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6141001f0 msgr2=0x7fe6141999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:31.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.173+0000 7fe61a0cd700 1 --2- 172.21.15.44:0/1850799773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6141001f0 0x7fe6141999b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00efc0 tx=0x7fe60c00c3a0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.173+0000 7fe61a0cd700 1 -- 172.21.15.44:0/1850799773 shutdown_connections 2023-09-10T00:40:31.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.173+0000 7fe61a0cd700 1 --2- 172.21.15.44:0/1850799773 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7fe5fc0a0690 0x7fe5fc0a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.173+0000 7fe61a0cd700 1 --2- 172.21.15.44:0/1850799773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6141001f0 0x7fe6141999b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.173+0000 7fe61a0cd700 1 --2- 172.21.15.44:0/1850799773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6140feed0 0x7fe6141985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.173+0000 7fe61a0cd700 1 --2- 172.21.15.44:0/1850799773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6140ff830 0x7fe614198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.173+0000 7fe61a0cd700 1 -- 172.21.15.44:0/1850799773 >> 172.21.15.44:0/1850799773 conn(0x7fe6140faa50 msgr2=0x7fe6140fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.173+0000 7fe61a0cd700 1 -- 172.21.15.44:0/1850799773 shutdown_connections 2023-09-10T00:40:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.173+0000 7fe61a0cd700 1 -- 172.21.15.44:0/1850799773 wait complete. 2023-09-10T00:40:31.187 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-09-10T00:40:31.295 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:31 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:31.093+0000 7fa9fb48b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-10T00:40:31.295 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:31 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:31.186+0000 7fa9fb48b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-10T00:40:31.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.314+0000 7f2e6916b700 1 -- 172.21.15.44:0/3883300080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e641010e0 msgr2=0x7f2e641034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:31.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.314+0000 7f2e6916b700 1 --2- 172.21.15.44:0/3883300080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e641010e0 0x7f2e641034d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009990 tx=0x7f2e54009ca0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.314+0000 7f2e6916b700 1 -- 172.21.15.44:0/3883300080 shutdown_connections 2023-09-10T00:40:31.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.314+0000 7f2e6916b700 1 --2- 172.21.15.44:0/3883300080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e64106340 0x7f2e64108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.314+0000 7f2e6916b700 1 --2- 172.21.15.44:0/3883300080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e641010e0 0x7f2e641034d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.314+0000 7f2e6916b700 1 --2- 172.21.15.44:0/3883300080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e64103a10 0x7f2e64105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.314+0000 7f2e6916b700 1 -- 172.21.15.44:0/3883300080 >> 172.21.15.44:0/3883300080 conn(0x7f2e640faa50 msgr2=0x7f2e640fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:31.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.314+0000 7f2e6916b700 1 -- 172.21.15.44:0/3883300080 shutdown_connections 2023-09-10T00:40:31.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.314+0000 7f2e6916b700 1 -- 172.21.15.44:0/3883300080 wait complete. 2023-09-10T00:40:31.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.314+0000 7f2e6916b700 1 Processor -- start 2023-09-10T00:40:31.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6916b700 1 -- start start 2023-09-10T00:40:31.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6916b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e641010e0 0x7f2e640ff090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:31.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6916b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e64103a10 0x7f2e640ff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:31.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6916b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e64106340 0x7f2e640ffb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:31.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6916b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e641a0240 con 0x7f2e64106340 2023-09-10T00:40:31.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6916b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e641a0380 con 0x7f2e641010e0 2023-09-10T00:40:31.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6916b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e641a04c0 con 0x7f2e64103a10 2023-09-10T00:40:31.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6359e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e64106340 0x7f2e640ffb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:31.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e62d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e641010e0 0x7f2e640ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:31.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6359e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e64106340 0x7f2e640ffb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48702/0 (socket says 172.21.15.44:48702) 2023-09-10T00:40:31.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e62d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e641010e0 0x7f2e640ff090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:50438/0 (socket says 172.21.15.44:50438) 2023-09-10T00:40:31.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e64103a10 0x7f2e640ff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:31.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6359e700 1 -- 172.21.15.44:0/841367979 learned_addr learned my addr 172.21.15.44:0/841367979 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:31.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6359e700 1 -- 172.21.15.44:0/841367979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e64103a10 msgr2=0x7f2e640ff5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:31.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6359e700 1 --2- 172.21.15.44:0/841367979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e64103a10 0x7f2e640ff5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6359e700 1 -- 172.21.15.44:0/841367979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e641010e0 msgr2=0x7f2e640ff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:31.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6359e700 1 --2- 172.21.15.44:0/841367979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e641010e0 0x7f2e640ff090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6359e700 1 -- 172.21.15.44:0/841367979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e540095f0 con 0x7f2e64106340 2023-09-10T00:40:31.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e62d9d700 1 --2- 172.21.15.44:0/841367979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e641010e0 0x7f2e640ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:40:31.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.315+0000 7f2e6259c700 1 --2- 172.21.15.44:0/841367979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e64103a10 0x7f2e640ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:40:31.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.316+0000 7f2e6359e700 1 --2- 172.21.15.44:0/841367979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e64106340 0x7f2e640ffb10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c007ee0 tx=0x7f2e5c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:31.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.316+0000 7f2e4bfff700 1 -- 172.21.15.44:0/841367979 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c017070 con 0x7f2e64106340 2023-09-10T00:40:31.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.316+0000 7f2e6916b700 1 -- 172.21.15.44:0/841367979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e641a0750 con 0x7f2e64106340 2023-09-10T00:40:31.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.316+0000 7f2e4bfff700 1 -- 172.21.15.44:0/841367979 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e5c00f040 con 0x7f2e64106340 2023-09-10T00:40:31.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.316+0000 7f2e4bfff700 1 -- 172.21.15.44:0/841367979 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c005570 con 0x7f2e64106340 2023-09-10T00:40:31.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.316+0000 7f2e6916b700 1 -- 172.21.15.44:0/841367979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e641a0be0 con 0x7f2e64106340 2023-09-10T00:40:31.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.317+0000 7f2e6916b700 1 -- 172.21.15.44:0/841367979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e640fc5d0 con 0x7f2e64106340 2023-09-10T00:40:31.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.318+0000 7f2e4bfff700 1 -- 172.21.15.44:0/841367979 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7f2e5c00db20 con 0x7f2e64106340 2023-09-10T00:40:31.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.318+0000 7f2e4bfff700 1 --2- 172.21.15.44:0/841367979 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2e4c0a0690 0x7f2e4c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:31.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.319+0000 7f2e62d9d700 1 -- 172.21.15.44:0/841367979 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2e4c0a0690 msgr2=0x7f2e4c0a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1488156576 2023-09-10T00:40:31.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.319+0000 7f2e4bfff700 1 -- 172.21.15.44:0/841367979 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f2e5c0cbd80 con 0x7f2e64106340 2023-09-10T00:40:31.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.319+0000 7f2e62d9d700 1 --2- 172.21.15.44:0/841367979 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2e4c0a0690 0x7f2e4c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-10T00:40:31.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.322+0000 7f2e4bfff700 1 -- 172.21.15.44:0/841367979 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2e5c0954f0 con 0x7f2e64106340 2023-09-10T00:40:31.356 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:31 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:31.094+0000 7f13458fb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-10T00:40:31.356 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:31 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:31.187+0000 7f13458fb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-10T00:40:31.500 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:31 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:31.059+0000 7f34ba828200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-10T00:40:31.500 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:31 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:31.169+0000 7f34ba828200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-10T00:40:31.501 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:31 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:31.287+0000 7f34ba828200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-10T00:40:31.520 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.520+0000 7f2e62d9d700 1 -- 172.21.15.44:0/841367979 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2e4c0a0690 msgr2=0x7f2e4c0a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1488156576 2023-09-10T00:40:31.520 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.520+0000 7f2e62d9d700 1 --2- 172.21.15.44:0/841367979 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2e4c0a0690 0x7f2e4c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-10T00:40:31.527 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.527+0000 7f2e6916b700 1 -- 172.21.15.44:0/841367979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f2e6404e3e0 con 0x7f2e64106340 2023-09-10T00:40:31.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.533+0000 7f2e4bfff700 1 -- 172.21.15.44:0/841367979 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v45)=0 v45) v1 ==== 133+0+0 (secure 0 0 0) 0x7f2e5c098b10 con 0x7f2e64106340 2023-09-10T00:40:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.535+0000 7f2e6916b700 1 -- 172.21.15.44:0/841367979 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2e4c0a0690 msgr2=0x7f2e4c0a2b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T00:40:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.535+0000 7f2e6916b700 1 --2- 172.21.15.44:0/841367979 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2e4c0a0690 0x7f2e4c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.535+0000 7f2e6916b700 1 -- 172.21.15.44:0/841367979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e64106340 msgr2=0x7f2e640ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.535+0000 7f2e6916b700 1 --2- 172.21.15.44:0/841367979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e64106340 0x7f2e640ffb10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c007ee0 tx=0x7f2e5c00d3b0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.535+0000 7f2e6916b700 1 -- 172.21.15.44:0/841367979 shutdown_connections 2023-09-10T00:40:31.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.535+0000 7f2e6916b700 1 --2- 172.21.15.44:0/841367979 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f2e4c0a0690 0x7f2e4c0a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.535+0000 7f2e6916b700 1 --2- 172.21.15.44:0/841367979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e64106340 0x7f2e640ffb10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.535+0000 7f2e6916b700 1 --2- 172.21.15.44:0/841367979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e641010e0 0x7f2e640ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.535+0000 7f2e6916b700 1 --2- 172.21.15.44:0/841367979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e64103a10 0x7f2e640ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.535+0000 7f2e6916b700 1 -- 172.21.15.44:0/841367979 >> 172.21.15.44:0/841367979 conn(0x7f2e640faa50 msgr2=0x7f2e64101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:31.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.535+0000 7f2e6916b700 1 -- 172.21.15.44:0/841367979 shutdown_connections 2023-09-10T00:40:31.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.535+0000 7f2e6916b700 1 -- 172.21.15.44:0/841367979 wait complete. 2023-09-10T00:40:31.548 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-09-10T00:40:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.679+0000 7f9448419700 1 -- 172.21.15.44:0/3031824789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9440106340 msgr2=0x7f9440108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.679+0000 7f9448419700 1 --2- 172.21.15.44:0/3031824789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9440106340 0x7f9440108750 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f943800b0a0 tx=0x7f943800b3b0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.679+0000 7f9448419700 1 -- 172.21.15.44:0/3031824789 shutdown_connections 2023-09-10T00:40:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.679+0000 7f9448419700 1 --2- 172.21.15.44:0/3031824789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9440106340 0x7f9440108750 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.679+0000 7f9448419700 1 --2- 172.21.15.44:0/3031824789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94401010e0 0x7f94401034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.679+0000 7f9448419700 1 --2- 172.21.15.44:0/3031824789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9440103a10 0x7f9440105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.679+0000 7f9448419700 1 -- 172.21.15.44:0/3031824789 >> 172.21.15.44:0/3031824789 conn(0x7f94400faa50 msgr2=0x7f94400fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:31.682 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.679+0000 7f9448419700 1 -- 172.21.15.44:0/3031824789 shutdown_connections 2023-09-10T00:40:31.682 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.679+0000 7f9448419700 1 -- 172.21.15.44:0/3031824789 wait complete. 2023-09-10T00:40:31.682 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.680+0000 7f9448419700 1 Processor -- start 2023-09-10T00:40:31.682 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.680+0000 7f9448419700 1 -- start start 2023-09-10T00:40:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.680+0000 7f9448419700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94401010e0 0x7f9440196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.680+0000 7f9448419700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9440103a10 0x7f94401968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.680+0000 7f9448419700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9440106340 0x7f9440197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.680+0000 7f9448419700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9440197da0 con 0x7f94401010e0 2023-09-10T00:40:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.680+0000 7f9448419700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9440197f10 con 0x7f9440106340 2023-09-10T00:40:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.680+0000 7f9448419700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94401980d0 con 0x7f9440103a10 2023-09-10T00:40:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.680+0000 7f94461b5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94401010e0 0x7f9440196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.680+0000 7f94461b5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94401010e0 0x7f9440196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48740/0 (socket says 172.21.15.44:48740) 2023-09-10T00:40:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.680+0000 7f94461b5700 1 -- 172.21.15.44:0/3192000487 learned_addr learned my addr 172.21.15.44:0/3192000487 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.681+0000 7f94459b4700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9440103a10 0x7f94401968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.681+0000 7f94469b6700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9440106340 0x7f9440197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.681+0000 7f94461b5700 1 -- 172.21.15.44:0/3192000487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9440103a10 msgr2=0x7f94401968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:31.687 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.681+0000 7f94461b5700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9440103a10 0x7f94401968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.687 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.681+0000 7f94461b5700 1 -- 172.21.15.44:0/3192000487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9440106340 msgr2=0x7f9440197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:31.687 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.681+0000 7f94461b5700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9440106340 0x7f9440197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:31.687 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.681+0000 7f94461b5700 1 -- 172.21.15.44:0/3192000487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9438009d00 con 0x7f94401010e0 2023-09-10T00:40:31.687 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.681+0000 7f94461b5700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94401010e0 0x7f9440196380 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f943c00e9a0 tx=0x7f943c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:31.688 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.681+0000 7f94377fe700 1 -- 172.21.15.44:0/3192000487 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c009790 con 0x7f94401010e0 2023-09-10T00:40:31.688 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.681+0000 7f94377fe700 1 -- 172.21.15.44:0/3192000487 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f943c010430 con 0x7f94401010e0 2023-09-10T00:40:31.688 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.681+0000 7f9448419700 1 -- 172.21.15.44:0/3192000487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94401983b0 con 0x7f94401010e0 2023-09-10T00:40:31.688 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.681+0000 7f94377fe700 1 -- 172.21.15.44:0/3192000487 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c00cea0 con 0x7f94401010e0 2023-09-10T00:40:31.688 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.681+0000 7f9448419700 1 -- 172.21.15.44:0/3192000487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944019eb70 con 0x7f94401010e0 2023-09-10T00:40:31.689 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.682+0000 7f9448419700 1 -- 172.21.15.44:0/3192000487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94400fc5d0 con 0x7f94401010e0 2023-09-10T00:40:31.689 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.684+0000 7f94377fe700 1 -- 172.21.15.44:0/3192000487 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7f943c0105a0 con 0x7f94401010e0 2023-09-10T00:40:31.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.685+0000 7f94377fe700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f942c0a0710 0x7f942c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:31.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.685+0000 7f94459b4700 1 -- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f942c0a0710 msgr2=0x7f942c0a2bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1488156576 2023-09-10T00:40:31.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.685+0000 7f94377fe700 1 -- 172.21.15.44:0/3192000487 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f943c014070 con 0x7f94401010e0 2023-09-10T00:40:31.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.685+0000 7f94459b4700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f942c0a0710 0x7f942c0a2bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-10T00:40:31.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.687+0000 7f94377fe700 1 -- 172.21.15.44:0/3192000487 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f943c095a30 con 0x7f94401010e0 2023-09-10T00:40:31.722 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:31 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:31.356+0000 7fa9fb48b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-10T00:40:31.724 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:31 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:31.358+0000 7f13458fb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-10T00:40:31.805 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:31 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:31.500+0000 7f34ba828200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-10T00:40:31.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:31 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/914189993' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-10T00:40:31.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:31 smithi044 ceph-mon[98676]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-09-10T00:40:31.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:31 smithi044 ceph-mon[98676]: from='client.? ' entity='client.admin' 2023-09-10T00:40:31.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:31 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/841367979' entity='client.admin' 2023-09-10T00:40:31.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.886+0000 7f94459b4700 1 -- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f942c0a0710 msgr2=0x7f942c0a2bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1488156576 2023-09-10T00:40:31.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.886+0000 7f94459b4700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f942c0a0710 0x7f942c0a2bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-10T00:40:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:31.925+0000 7f9448419700 1 -- 172.21.15.44:0/3192000487 --> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f944002cdc0 con 0x7f942c0a0710 2023-09-10T00:40:32.041 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:31 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/914189993' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-10T00:40:32.041 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:31 smithi131 ceph-mon[103713]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-09-10T00:40:32.041 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:31 smithi131 ceph-mon[103713]: from='client.? ' entity='client.admin' 2023-09-10T00:40:32.041 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:31 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/841367979' entity='client.admin' 2023-09-10T00:40:32.042 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:31 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:31.722+0000 7fa9fb48b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-10T00:40:32.042 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:31 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:31.856+0000 7fa9fb48b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-10T00:40:32.042 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:31 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:31.945+0000 7fa9fb48b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-10T00:40:32.042 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:32 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:32.040+0000 7fa9fb48b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-10T00:40:32.043 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:31 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/914189993' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-10T00:40:32.043 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:31 smithi099 ceph-mon[102692]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-09-10T00:40:32.043 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:31 smithi099 ceph-mon[102692]: from='client.? ' entity='client.admin' 2023-09-10T00:40:32.043 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:31 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/841367979' entity='client.admin' 2023-09-10T00:40:32.044 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:31 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:31.723+0000 7f13458fb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-10T00:40:32.044 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:31 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:31.858+0000 7f13458fb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-10T00:40:32.044 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:31 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:31.946+0000 7f13458fb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-10T00:40:32.044 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:32 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:32.040+0000 7f13458fb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-10T00:40:32.252 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:31 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:31.989+0000 7f34ba828200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-10T00:40:32.253 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:32 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:32.143+0000 7f34ba828200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-10T00:40:32.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:32.287+0000 7f94459b4700 1 -- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f942c0a0710 msgr2=0x7f942c0a2bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1488156576 2023-09-10T00:40:32.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:32.287+0000 7f94459b4700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f942c0a0710 0x7f942c0a2bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-10T00:40:32.554 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:32 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:32.252+0000 7f34ba828200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-10T00:40:32.555 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:32 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:32.368+0000 7f34ba828200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-10T00:40:32.795 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:32 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:32.462+0000 7fa9fb48b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-10T00:40:32.856 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:32 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:32.460+0000 7f13458fb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-10T00:40:33.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:33.088+0000 7f94459b4700 1 -- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f942c0a0710 msgr2=0x7f942c0a2bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1488156576 2023-09-10T00:40:33.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:33.088+0000 7f94459b4700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f942c0a0710 0x7f942c0a2bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-10T00:40:33.304 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:32 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:32.880+0000 7f34ba828200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-10T00:40:33.545 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:33 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:33.122+0000 7fa9fb48b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-10T00:40:33.545 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:33 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:33.203+0000 7fa9fb48b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-10T00:40:33.606 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:33 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:33.113+0000 7f13458fb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-10T00:40:33.606 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:33 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:33.195+0000 7f13458fb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-10T00:40:34.054 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:33 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:33.665+0000 7f34ba828200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-10T00:40:34.055 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:33 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:33.765+0000 7f34ba828200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-10T00:40:34.226 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:33 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:33.963+0000 7f13458fb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-10T00:40:34.227 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:34 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:34.054+0000 7f13458fb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-10T00:40:34.243 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:33 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:33.978+0000 7fa9fb48b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-10T00:40:34.243 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:34 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:34.069+0000 7fa9fb48b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-10T00:40:34.503 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:34 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:34.226+0000 7f13458fb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-10T00:40:34.523 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:34 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:34.242+0000 7fa9fb48b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-10T00:40:34.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:34.689+0000 7f94459b4700 1 -- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f942c0a0710 msgr2=0x7f942c0a2bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1488156576 2023-09-10T00:40:34.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:34.689+0000 7f94459b4700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f942c0a0710 0x7f942c0a2bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-09-10T00:40:34.794 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:34 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:34.522+0000 7fa9fb48b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-10T00:40:34.795 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:34 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:34.751+0000 7fa9fb48b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-10T00:40:34.856 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:34 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:34.503+0000 7f13458fb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-10T00:40:34.856 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:34 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:34.729+0000 7f13458fb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-10T00:40:35.054 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:34 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:34.744+0000 7f34ba828200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-10T00:40:35.055 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:34 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:34.856+0000 7f34ba828200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-10T00:40:35.140 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:34 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:34.877+0000 7f13458fb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-10T00:40:35.162 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:34 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:34.900+0000 7fa9fb48b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-10T00:40:35.438 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:35 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:35.082+0000 7f34ba828200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-10T00:40:35.545 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:35 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:35.162+0000 7fa9fb48b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-10T00:40:35.606 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:35 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:35.139+0000 7f13458fb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-10T00:40:35.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:35 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:35.437+0000 7f34ba828200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-10T00:40:36.054 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:35 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:35.694+0000 7f34ba828200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-10T00:40:36.055 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:35 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:35.881+0000 7f34ba828200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-10T00:40:36.106 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:35 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:35.841+0000 7f13458fb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-10T00:40:36.106 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:35 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:35.972+0000 7f13458fb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-10T00:40:36.193 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:35 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:35.870+0000 7fa9fb48b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-10T00:40:36.193 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:36 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:36.000+0000 7fa9fb48b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-10T00:40:36.490 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:36 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:36.163+0000 7f13458fb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-10T00:40:36.490 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:36 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:36.274+0000 7f13458fb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-10T00:40:36.524 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:36 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:36.192+0000 7fa9fb48b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-10T00:40:36.545 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:36 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:36.305+0000 7fa9fb48b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-10T00:40:36.555 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:36 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:36.217+0000 7f34ba828200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-10T00:40:37.304 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:37.052+0000 7f34ba828200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-10T00:40:37.305 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:37.215+0000 7f34ba828200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-10T00:40:37.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:37.420+0000 7f94377fe700 1 -- 172.21.15.44:0/3192000487 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 20) v1 ==== 135276+0+0 (secure 0 0 0) 0x7f943c095e00 con 0x7f94401010e0 2023-09-10T00:40:37.606 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:36 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:36.489+0000 7f13458fb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-10T00:40:37.606 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 00:40:37 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y[104179]: 2023-09-10T00:40:37.347+0000 7f13458fb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-10T00:40:37.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:37 smithi099 ceph-mon[102692]: Standby manager daemon y restarted 2023-09-10T00:40:37.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:37 smithi099 ceph-mon[102692]: Standby manager daemon y started 2023-09-10T00:40:37.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:37 smithi099 ceph-mon[102692]: from='mgr.? 172.21.15.99:0/3772758170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-10T00:40:37.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:37 smithi099 ceph-mon[102692]: from='mgr.? 172.21.15.99:0/3772758170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-10T00:40:37.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:37 smithi099 ceph-mon[102692]: from='mgr.? 172.21.15.99:0/3772758170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-10T00:40:37.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:37 smithi099 ceph-mon[102692]: from='mgr.? 172.21.15.99:0/3772758170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-10T00:40:37.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:37 smithi099 ceph-mon[102692]: Standby manager daemon z restarted 2023-09-10T00:40:37.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:37 smithi099 ceph-mon[102692]: Standby manager daemon z started 2023-09-10T00:40:37.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:37 smithi099 ceph-mon[102692]: from='mgr.? 172.21.15.131:0/2641766832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-10T00:40:37.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:37 smithi099 ceph-mon[102692]: from='mgr.? 172.21.15.131:0/2641766832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-10T00:40:37.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:37 smithi099 ceph-mon[102692]: from='mgr.? 172.21.15.131:0/2641766832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-10T00:40:37.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:37 smithi099 ceph-mon[102692]: from='mgr.? 172.21.15.131:0/2641766832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-10T00:40:37.795 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:36 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:36.523+0000 7fa9fb48b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-10T00:40:37.795 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 00:40:37 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z[105802]: 2023-09-10T00:40:37.389+0000 7fa9fb48b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-10T00:40:37.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:37 smithi131 ceph-mon[103713]: Standby manager daemon y restarted 2023-09-10T00:40:37.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:37 smithi131 ceph-mon[103713]: Standby manager daemon y started 2023-09-10T00:40:37.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:37 smithi131 ceph-mon[103713]: from='mgr.? 172.21.15.99:0/3772758170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-10T00:40:37.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:37 smithi131 ceph-mon[103713]: from='mgr.? 172.21.15.99:0/3772758170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-10T00:40:37.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:37 smithi131 ceph-mon[103713]: from='mgr.? 172.21.15.99:0/3772758170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-10T00:40:37.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:37 smithi131 ceph-mon[103713]: from='mgr.? 172.21.15.99:0/3772758170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-10T00:40:37.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:37 smithi131 ceph-mon[103713]: Standby manager daemon z restarted 2023-09-10T00:40:37.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:37 smithi131 ceph-mon[103713]: Standby manager daemon z started 2023-09-10T00:40:37.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:37 smithi131 ceph-mon[103713]: from='mgr.? 172.21.15.131:0/2641766832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-10T00:40:37.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:37 smithi131 ceph-mon[103713]: from='mgr.? 172.21.15.131:0/2641766832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-10T00:40:37.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:37 smithi131 ceph-mon[103713]: from='mgr.? 172.21.15.131:0/2641766832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-10T00:40:37.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:37 smithi131 ceph-mon[103713]: from='mgr.? 172.21.15.131:0/2641766832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-10T00:40:37.804 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:37.471+0000 7f34ba828200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-10T00:40:37.805 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:37.614+0000 7f34ba828200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-10T00:40:37.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-mon[98676]: Standby manager daemon y restarted 2023-09-10T00:40:37.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-mon[98676]: Standby manager daemon y started 2023-09-10T00:40:37.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-mon[98676]: from='mgr.? 172.21.15.99:0/3772758170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-10T00:40:37.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-mon[98676]: from='mgr.? 172.21.15.99:0/3772758170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-10T00:40:37.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-mon[98676]: from='mgr.? 172.21.15.99:0/3772758170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-10T00:40:37.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-mon[98676]: from='mgr.? 172.21.15.99:0/3772758170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-10T00:40:37.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-mon[98676]: Standby manager daemon z restarted 2023-09-10T00:40:37.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-mon[98676]: Standby manager daemon z started 2023-09-10T00:40:37.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-mon[98676]: from='mgr.? 172.21.15.131:0/2641766832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-10T00:40:37.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-mon[98676]: from='mgr.? 172.21.15.131:0/2641766832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-10T00:40:37.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-mon[98676]: from='mgr.? 172.21.15.131:0/2641766832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-10T00:40:37.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-mon[98676]: from='mgr.? 172.21.15.131:0/2641766832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-10T00:40:37.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:37.893+0000 7f94459b4700 1 -- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f942c0a0710 msgr2=0x7f942c0a2bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1488156576 2023-09-10T00:40:37.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:37.893+0000 7f94459b4700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f942c0a0710 0x7f942c0a2bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-09-10T00:40:38.304 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:37 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:37.894+0000 7f34ba828200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-10T00:40:38.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:38 smithi131 ceph-mon[103713]: mgrmap e20: x(active, since 8m), standbys: z, y 2023-09-10T00:40:38.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:38 smithi044 ceph-mon[98676]: mgrmap e20: x(active, since 8m), standbys: z, y 2023-09-10T00:40:38.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:38 smithi099 ceph-mon[102692]: mgrmap e20: x(active, since 8m), standbys: z, y 2023-09-10T00:40:38.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:38.982+0000 7f94377fe700 1 -- 172.21.15.44:0/3192000487 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 21) v1 ==== 134944+0+0 (secure 0 0 0) 0x7f943c0c9a50 con 0x7f94401010e0 2023-09-10T00:40:38.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:38.982+0000 7f94377fe700 1 -- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f942c0a0710 msgr2=0x7f942c0a2bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T00:40:38.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:38.982+0000 7f94377fe700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/1488156576,v1:172.21.15.44:6801/1488156576] conn(0x7f942c0a0710 0x7f942c0a2bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:39.305 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:38 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:38.955+0000 7f34ba828200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-10T00:40:39.306 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:39.187+0000 7f341a95d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:40:39.306 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:39.187+0000 7f341a95d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:40:39.306 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:39.187+0000 7f341a95d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:40:39.306 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:39.187+0000 7f341a95d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:40:39.306 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:39.187+0000 7f341d963700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:40:39.307 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:39.187+0000 7f341a95d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:40:39.307 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:39.187+0000 7f341d963700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:40:39.307 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:39.187+0000 7f341d963700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:40:39.307 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:39.187+0000 7f341d963700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:40:39.307 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:40:39.187+0000 7f341d963700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:40:39.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: Active manager daemon x restarted 2023-09-10T00:40:39.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: Activating manager daemon x 2023-09-10T00:40:39.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: osdmap e92: 12 total, 12 up, 12 in 2023-09-10T00:40:39.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: mgrmap e21: x(active, starting, since 0.0136054s), standbys: z, y 2023-09-10T00:40:39.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:40:39.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:40:39.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:40:39.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-10T00:40:39.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T00:40:39.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-10T00:40:39.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-10T00:40:39.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-10T00:40:39.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-10T00:40:39.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-10T00:40:39.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-10T00:40:39.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T00:40:39.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-10T00:40:39.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-10T00:40:39.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-10T00:40:39.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-10T00:40:39.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-10T00:40:39.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-10T00:40:39.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:40:39.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:40:39.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:40:39.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:40:39.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:40:39.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:40:39.801 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:40:39.801 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:40:39.801 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:40:39.801 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:40:39.801 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:40:39.802 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:40:39.802 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-10T00:40:39.802 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-10T00:40:39.802 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-10T00:40:39.802 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: Manager daemon x is now available 2023-09-10T00:40:39.802 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:40:39.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:40:39.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-10T00:40:39.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-10T00:40:39.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-10T00:40:39.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:39 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-10T00:40:39.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: Active manager daemon x restarted 2023-09-10T00:40:39.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: Activating manager daemon x 2023-09-10T00:40:39.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: osdmap e92: 12 total, 12 up, 12 in 2023-09-10T00:40:39.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: mgrmap e21: x(active, starting, since 0.0136054s), standbys: z, y 2023-09-10T00:40:39.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:40:39.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:40:39.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:40:39.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-10T00:40:39.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T00:40:39.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-10T00:40:39.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-10T00:40:39.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-10T00:40:39.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-10T00:40:39.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-10T00:40:39.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-10T00:40:39.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T00:40:39.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-10T00:40:39.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-10T00:40:39.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-10T00:40:39.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-10T00:40:39.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-10T00:40:39.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-10T00:40:39.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:40:39.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:40:39.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:40:39.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:40:39.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:40:39.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:40:39.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:40:39.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:40:39.812 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:40:39.812 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:40:39.812 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:40:39.812 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:40:39.812 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-10T00:40:39.813 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-10T00:40:39.813 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-10T00:40:39.813 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: Manager daemon x is now available 2023-09-10T00:40:39.813 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:40:39.813 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:40:39.814 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-10T00:40:39.814 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-10T00:40:39.814 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-10T00:40:39.814 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:39 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-10T00:40:39.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: Active manager daemon x restarted 2023-09-10T00:40:39.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: Activating manager daemon x 2023-09-10T00:40:39.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: osdmap e92: 12 total, 12 up, 12 in 2023-09-10T00:40:39.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: mgrmap e21: x(active, starting, since 0.0136054s), standbys: z, y 2023-09-10T00:40:39.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:40:39.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:40:39.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:40:39.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-10T00:40:39.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T00:40:39.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-10T00:40:39.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-10T00:40:39.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-10T00:40:39.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-10T00:40:39.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-10T00:40:39.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-10T00:40:39.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T00:40:39.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-10T00:40:39.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-10T00:40:39.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-10T00:40:39.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-10T00:40:39.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-10T00:40:39.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-10T00:40:39.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-10T00:40:39.862 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-10T00:40:39.862 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-10T00:40:39.862 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-10T00:40:39.862 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-10T00:40:39.862 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-10T00:40:39.863 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-10T00:40:39.863 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-10T00:40:39.863 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-10T00:40:39.863 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-10T00:40:39.863 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-10T00:40:39.864 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-10T00:40:39.864 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-10T00:40:39.865 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-10T00:40:39.865 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-10T00:40:39.865 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: Manager daemon x is now available 2023-09-10T00:40:39.865 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:40:39.865 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:40:39.866 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-10T00:40:39.866 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-10T00:40:39.866 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-10T00:40:39.866 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:39 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-10T00:40:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:39.987+0000 7f94377fe700 1 -- 172.21.15.44:0/3192000487 <== mon.0 v2:172.21.15.44:3300/0 9 ==== mgrmap(e 22) v1 ==== 135071+0+0 (secure 0 0 0) 0x7f943c0ba620 con 0x7f94401010e0 2023-09-10T00:40:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:39.987+0000 7f94377fe700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f942c01b9e0 0x7f942c01bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:39.987+0000 7f94377fe700 1 -- 172.21.15.44:0/3192000487 --> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f944002cdc0 con 0x7f942c01b9e0 2023-09-10T00:40:39.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:39.988+0000 7f94459b4700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f942c01b9e0 0x7f942c01bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:39.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:39.991+0000 7f94459b4700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f942c01b9e0 0x7f942c01bdc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9430009880 tx=0x7f9430009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:40.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.046+0000 7f94377fe700 1 -- 172.21.15.44:0/3192000487 <== mgr.15825 v2:172.21.15.44:6800/605958391 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f944002cdc0 con 0x7f942c01b9e0 2023-09-10T00:40:40.047 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path / 2023-09-10T00:40:40.049 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.048+0000 7f9448419700 1 -- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f942c01b9e0 msgr2=0x7f942c01bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:40.049 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.048+0000 7f9448419700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f942c01b9e0 0x7f942c01bdc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9430009880 tx=0x7f9430009260 comp rx=0 tx=0).stop 2023-09-10T00:40:40.049 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.048+0000 7f9448419700 1 -- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94401010e0 msgr2=0x7f9440196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:40.049 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.048+0000 7f9448419700 1 --2- 172.21.15.44:0/3192000487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94401010e0 0x7f9440196380 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f943c00e9a0 tx=0x7f943c00ecb0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.048+0000 7f94469b6700 1 -- 172.21.15.44:0/3192000487 reap_dead start 2023-09-10T00:40:40.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.049+0000 7f9448419700 1 -- 172.21.15.44:0/3192000487 shutdown_connections 2023-09-10T00:40:40.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.049+0000 7f9448419700 1 -- 172.21.15.44:0/3192000487 >> 172.21.15.44:0/3192000487 conn(0x7f94400faa50 msgr2=0x7f9440101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:40.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.049+0000 7f9448419700 1 -- 172.21.15.44:0/3192000487 shutdown_connections 2023-09-10T00:40:40.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.049+0000 7f9448419700 1 -- 172.21.15.44:0/3192000487 wait complete. 2023-09-10T00:40:40.062 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-09-10T00:40:40.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.197+0000 7f0ee9a2f700 1 -- 172.21.15.44:0/4025513103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ee4108740 msgr2=0x7f0ee410eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:40.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.197+0000 7f0ee9a2f700 1 --2- 172.21.15.44:0/4025513103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ee4108740 0x7f0ee410eff0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00b0d0 tx=0x7f0edc00b3e0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.197+0000 7f0ee9a2f700 1 -- 172.21.15.44:0/4025513103 shutdown_connections 2023-09-10T00:40:40.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.197+0000 7f0ee9a2f700 1 --2- 172.21.15.44:0/4025513103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ee4108740 0x7f0ee410eff0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.197+0000 7f0ee9a2f700 1 --2- 172.21.15.44:0/4025513103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee4106bb0 0x7f0ee4106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.197+0000 7f0ee9a2f700 1 --2- 172.21.15.44:0/4025513103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ee4107da0 0x7f0ee4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.197+0000 7f0ee9a2f700 1 -- 172.21.15.44:0/4025513103 >> 172.21.15.44:0/4025513103 conn(0x7f0ee4075b30 msgr2=0x7f0ee4077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:40.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.197+0000 7f0ee9a2f700 1 -- 172.21.15.44:0/4025513103 shutdown_connections 2023-09-10T00:40:40.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.197+0000 7f0ee9a2f700 1 -- 172.21.15.44:0/4025513103 wait complete. 2023-09-10T00:40:40.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.198+0000 7f0ee9a2f700 1 Processor -- start 2023-09-10T00:40:40.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.198+0000 7f0ee9a2f700 1 -- start start 2023-09-10T00:40:40.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.198+0000 7f0ee9a2f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ee4106bb0 0x7f0ee419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:40.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.198+0000 7f0ee9a2f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ee4107da0 0x7f0ee419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:40.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.198+0000 7f0ee9a2f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee4108740 0x7f0ee41a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:40.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.198+0000 7f0ee9a2f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee41a4410 con 0x7f0ee4107da0 2023-09-10T00:40:40.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.198+0000 7f0ee9a2f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee41a4550 con 0x7f0ee4108740 2023-09-10T00:40:40.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.198+0000 7f0ee9a2f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee41a46e0 con 0x7f0ee4106bb0 2023-09-10T00:40:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.198+0000 7f0ee27fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ee4107da0 0x7f0ee419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.198+0000 7f0ee27fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ee4107da0 0x7f0ee419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48972/0 (socket says 172.21.15.44:48972) 2023-09-10T00:40:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.198+0000 7f0ee27fc700 1 -- 172.21.15.44:0/4039383319 learned_addr learned my addr 172.21.15.44:0/4039383319 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.198+0000 7f0ee2ffd700 1 --2- 172.21.15.44:0/4039383319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ee4106bb0 0x7f0ee419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.199+0000 7f0ee37fe700 1 --2- 172.21.15.44:0/4039383319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee4108740 0x7f0ee41a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.199+0000 7f0ee2ffd700 1 -- 172.21.15.44:0/4039383319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee4108740 msgr2=0x7f0ee41a3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.199+0000 7f0ee2ffd700 1 --2- 172.21.15.44:0/4039383319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee4108740 0x7f0ee41a3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.199+0000 7f0ee2ffd700 1 -- 172.21.15.44:0/4039383319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ee4107da0 msgr2=0x7f0ee419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.199+0000 7f0ee2ffd700 1 --2- 172.21.15.44:0/4039383319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ee4107da0 0x7f0ee419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.199+0000 7f0ee2ffd700 1 -- 172.21.15.44:0/4039383319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc009d00 con 0x7f0ee4106bb0 2023-09-10T00:40:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.199+0000 7f0ee27fc700 1 --2- 172.21.15.44:0/4039383319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ee4107da0 0x7f0ee419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:40:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.199+0000 7f0ee2ffd700 1 --2- 172.21.15.44:0/4039383319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ee4106bb0 0x7f0ee419c890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400e9c0 tx=0x7f0ed400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.199+0000 7f0ee8a2d700 1 -- 172.21.15.44:0/4039383319 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4004220 con 0x7f0ee4106bb0 2023-09-10T00:40:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.199+0000 7f0ee9a2f700 1 -- 172.21.15.44:0/4039383319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee41a49c0 con 0x7f0ee4106bb0 2023-09-10T00:40:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.199+0000 7f0ee8a2d700 1 -- 172.21.15.44:0/4039383319 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ed4004380 con 0x7f0ee4106bb0 2023-09-10T00:40:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.199+0000 7f0ee8a2d700 1 -- 172.21.15.44:0/4039383319 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4003b40 con 0x7f0ee4106bb0 2023-09-10T00:40:40.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.199+0000 7f0ee9a2f700 1 -- 172.21.15.44:0/4039383319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee41a4ed0 con 0x7f0ee4106bb0 2023-09-10T00:40:40.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.200+0000 7f0ee9a2f700 1 -- 172.21.15.44:0/4039383319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee4112280 con 0x7f0ee4106bb0 2023-09-10T00:40:40.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.201+0000 7f0ee8a2d700 1 -- 172.21.15.44:0/4039383319 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 22) v1 ==== 135071+0+0 (secure 0 0 0) 0x7f0ed4003ca0 con 0x7f0ee4106bb0 2023-09-10T00:40:40.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.202+0000 7f0ee8a2d700 1 --2- 172.21.15.44:0/4039383319 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0ecc0a0440 0x7f0ecc0a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:40.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.202+0000 7f0ee8a2d700 1 -- 172.21.15.44:0/4039383319 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f0ed4014070 con 0x7f0ee4106bb0 2023-09-10T00:40:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.202+0000 7f0ee27fc700 1 --2- 172.21.15.44:0/4039383319 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0ecc0a0440 0x7f0ecc0a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.202+0000 7f0ee27fc700 1 --2- 172.21.15.44:0/4039383319 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0ecc0a0440 0x7f0ecc0a2900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8005950 tx=0x7f0ed800b500 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.205+0000 7f0ee8a2d700 1 -- 172.21.15.44:0/4039383319 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0ed40958a0 con 0x7f0ee4106bb0 2023-09-10T00:40:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.450+0000 7f0ee9a2f700 1 -- 172.21.15.44:0/4039383319 --> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ee4060a40 con 0x7f0ecc0a0440 2023-09-10T00:40:40.510 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.509+0000 7f0ee8a2d700 1 -- 172.21.15.44:0/4039383319 <== mgr.15825 v2:172.21.15.44:6800/605958391 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f0ee4060a40 con 0x7f0ecc0a0440 2023-09-10T00:40:40.511 INFO:teuthology.orchestra.run.smithi044.stdout:Retention added to path / 2023-09-10T00:40:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.511+0000 7f0ee9a2f700 1 -- 172.21.15.44:0/4039383319 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0ecc0a0440 msgr2=0x7f0ecc0a2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.511+0000 7f0ee9a2f700 1 --2- 172.21.15.44:0/4039383319 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0ecc0a0440 0x7f0ecc0a2900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8005950 tx=0x7f0ed800b500 comp rx=0 tx=0).stop 2023-09-10T00:40:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.511+0000 7f0ee9a2f700 1 -- 172.21.15.44:0/4039383319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ee4106bb0 msgr2=0x7f0ee419c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.511+0000 7f0ee9a2f700 1 --2- 172.21.15.44:0/4039383319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ee4106bb0 0x7f0ee419c890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400e9c0 tx=0x7f0ed400ecd0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.511+0000 7f0ee9a2f700 1 -- 172.21.15.44:0/4039383319 shutdown_connections 2023-09-10T00:40:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.511+0000 7f0ee9a2f700 1 --2- 172.21.15.44:0/4039383319 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0ecc0a0440 0x7f0ecc0a2900 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.511+0000 7f0ee9a2f700 1 --2- 172.21.15.44:0/4039383319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee4108740 0x7f0ee41a3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.511+0000 7f0ee9a2f700 1 --2- 172.21.15.44:0/4039383319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ee4106bb0 0x7f0ee419c890 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.511+0000 7f0ee9a2f700 1 --2- 172.21.15.44:0/4039383319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ee4107da0 0x7f0ee419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.511+0000 7f0ee9a2f700 1 -- 172.21.15.44:0/4039383319 >> 172.21.15.44:0/4039383319 conn(0x7f0ee4075b30 msgr2=0x7f0ee4077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.512+0000 7f0ee9a2f700 1 -- 172.21.15.44:0/4039383319 shutdown_connections 2023-09-10T00:40:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.512+0000 7f0ee9a2f700 1 -- 172.21.15.44:0/4039383319 wait complete. 2023-09-10T00:40:40.527 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-09-10T00:40:40.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.659+0000 7f1a3d13c700 1 -- 172.21.15.44:0/3041062089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a38101040 msgr2=0x7f1a381078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:40.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.659+0000 7f1a3d13c700 1 --2- 172.21.15.44:0/3041062089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a38101040 0x7f1a381078f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1a300099c0 tx=0x7f1a30009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.659+0000 7f1a3d13c700 1 -- 172.21.15.44:0/3041062089 shutdown_connections 2023-09-10T00:40:40.662 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.659+0000 7f1a3d13c700 1 --2- 172.21.15.44:0/3041062089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a38101040 0x7f1a381078f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.662 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.659+0000 7f1a3d13c700 1 --2- 172.21.15.44:0/3041062089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a380ff4b0 0x7f1a380ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.662 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.659+0000 7f1a3d13c700 1 --2- 172.21.15.44:0/3041062089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a381006a0 0x7f1a38100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.662 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.659+0000 7f1a3d13c700 1 -- 172.21.15.44:0/3041062089 >> 172.21.15.44:0/3041062089 conn(0x7f1a380faa00 msgr2=0x7f1a380fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:40.662 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.659+0000 7f1a3d13c700 1 -- 172.21.15.44:0/3041062089 shutdown_connections 2023-09-10T00:40:40.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.659+0000 7f1a3d13c700 1 -- 172.21.15.44:0/3041062089 wait complete. 2023-09-10T00:40:40.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.660+0000 7f1a3d13c700 1 Processor -- start 2023-09-10T00:40:40.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.660+0000 7f1a3d13c700 1 -- start start 2023-09-10T00:40:40.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.660+0000 7f1a3d13c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a380ff4b0 0x7f1a38069490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:40.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.660+0000 7f1a3d13c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a381006a0 0x7f1a3819aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:40.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.660+0000 7f1a3d13c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a38101040 0x7f1a3819d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:40.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.660+0000 7f1a3d13c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3819dd00 con 0x7f1a38101040 2023-09-10T00:40:40.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.660+0000 7f1a3d13c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3819de70 con 0x7f1a381006a0 2023-09-10T00:40:40.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.660+0000 7f1a3d13c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3819e030 con 0x7f1a380ff4b0 2023-09-10T00:40:40.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.661+0000 7f1a3759e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a38101040 0x7f1a3819d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:40.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.661+0000 7f1a3759e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a38101040 0x7f1a3819d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48986/0 (socket says 172.21.15.44:48986) 2023-09-10T00:40:40.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.661+0000 7f1a3759e700 1 -- 172.21.15.44:0/4217446850 learned_addr learned my addr 172.21.15.44:0/4217446850 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:40.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.661+0000 7f1a36d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a380ff4b0 0x7f1a38069490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:40.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.661+0000 7f1a3659c700 1 --2- 172.21.15.44:0/4217446850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a381006a0 0x7f1a3819aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:40.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.661+0000 7f1a3759e700 1 -- 172.21.15.44:0/4217446850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a380ff4b0 msgr2=0x7f1a38069490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:40.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.661+0000 7f1a3759e700 1 --2- 172.21.15.44:0/4217446850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a380ff4b0 0x7f1a38069490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.661+0000 7f1a3759e700 1 -- 172.21.15.44:0/4217446850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a381006a0 msgr2=0x7f1a3819aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:40.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.661+0000 7f1a3759e700 1 --2- 172.21.15.44:0/4217446850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a381006a0 0x7f1a3819aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.661+0000 7f1a3759e700 1 -- 172.21.15.44:0/4217446850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a300095f0 con 0x7f1a38101040 2023-09-10T00:40:40.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.661+0000 7f1a3759e700 1 --2- 172.21.15.44:0/4217446850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a38101040 0x7f1a3819d530 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009990 tx=0x7f1a30003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:40.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.661+0000 7f1a1ffff700 1 -- 172.21.15.44:0/4217446850 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3001d070 con 0x7f1a38101040 2023-09-10T00:40:40.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.661+0000 7f1a1ffff700 1 -- 172.21.15.44:0/4217446850 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a3000bd50 con 0x7f1a38101040 2023-09-10T00:40:40.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.661+0000 7f1a3d13c700 1 -- 172.21.15.44:0/4217446850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3819e2b0 con 0x7f1a38101040 2023-09-10T00:40:40.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.661+0000 7f1a1ffff700 1 -- 172.21.15.44:0/4217446850 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a30022c30 con 0x7f1a38101040 2023-09-10T00:40:40.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.661+0000 7f1a3d13c700 1 -- 172.21.15.44:0/4217446850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3819e6d0 con 0x7f1a38101040 2023-09-10T00:40:40.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.662+0000 7f1a3d13c700 1 -- 172.21.15.44:0/4217446850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a38106a30 con 0x7f1a38101040 2023-09-10T00:40:40.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.663+0000 7f1a1ffff700 1 -- 172.21.15.44:0/4217446850 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 22) v1 ==== 135071+0+0 (secure 0 0 0) 0x7f1a30003e90 con 0x7f1a38101040 2023-09-10T00:40:40.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.664+0000 7f1a1ffff700 1 --2- 172.21.15.44:0/4217446850 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1a200a0440 0x7f1a200a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:40.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.664+0000 7f1a36d9d700 1 --2- 172.21.15.44:0/4217446850 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1a200a0440 0x7f1a200a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:40.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.664+0000 7f1a1ffff700 1 -- 172.21.15.44:0/4217446850 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f1a300cd900 con 0x7f1a38101040 2023-09-10T00:40:40.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.665+0000 7f1a36d9d700 1 --2- 172.21.15.44:0/4217446850 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1a200a0440 0x7f1a200a2900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1a38100500 tx=0x7f1a280091c0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:40.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.668+0000 7f1a1ffff700 1 -- 172.21.15.44:0/4217446850 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1a300972f0 con 0x7f1a38101040 2023-09-10T00:40:40.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:40 smithi131 ceph-mon[103713]: [10/Sep/2023:00:40:39] ENGINE Bus STARTING 2023-09-10T00:40:40.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:40 smithi131 ceph-mon[103713]: [10/Sep/2023:00:40:39] ENGINE Serving on http://172.21.15.44:8765 2023-09-10T00:40:40.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:40 smithi131 ceph-mon[103713]: [10/Sep/2023:00:40:39] ENGINE Serving on https://172.21.15.44:7150 2023-09-10T00:40:40.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:40 smithi131 ceph-mon[103713]: [10/Sep/2023:00:40:39] ENGINE Bus STARTED 2023-09-10T00:40:40.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:40 smithi131 ceph-mon[103713]: mgrmap e22: x(active, since 1.01751s), standbys: z, y 2023-09-10T00:40:40.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:40 smithi044 ceph-mon[98676]: [10/Sep/2023:00:40:39] ENGINE Bus STARTING 2023-09-10T00:40:40.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:40 smithi044 ceph-mon[98676]: [10/Sep/2023:00:40:39] ENGINE Serving on http://172.21.15.44:8765 2023-09-10T00:40:40.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:40 smithi044 ceph-mon[98676]: [10/Sep/2023:00:40:39] ENGINE Serving on https://172.21.15.44:7150 2023-09-10T00:40:40.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:40 smithi044 ceph-mon[98676]: [10/Sep/2023:00:40:39] ENGINE Bus STARTED 2023-09-10T00:40:40.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:40 smithi044 ceph-mon[98676]: mgrmap e22: x(active, since 1.01751s), standbys: z, y 2023-09-10T00:40:40.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:40 smithi099 ceph-mon[102692]: [10/Sep/2023:00:40:39] ENGINE Bus STARTING 2023-09-10T00:40:40.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:40 smithi099 ceph-mon[102692]: [10/Sep/2023:00:40:39] ENGINE Serving on http://172.21.15.44:8765 2023-09-10T00:40:40.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:40 smithi099 ceph-mon[102692]: [10/Sep/2023:00:40:39] ENGINE Serving on https://172.21.15.44:7150 2023-09-10T00:40:40.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:40 smithi099 ceph-mon[102692]: [10/Sep/2023:00:40:39] ENGINE Bus STARTED 2023-09-10T00:40:40.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:40 smithi099 ceph-mon[102692]: mgrmap e22: x(active, since 1.01751s), standbys: z, y 2023-09-10T00:40:40.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.915+0000 7f1a3d13c700 1 -- 172.21.15.44:0/4217446850 --> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a3802d0c0 con 0x7f1a200a0440 2023-09-10T00:40:40.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.924+0000 7f1a1ffff700 1 -- 172.21.15.44:0/4217446850 <== mgr.15825 v2:172.21.15.44:6800/605958391 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f1a3802d0c0 con 0x7f1a200a0440 2023-09-10T00:40:40.926 INFO:teuthology.orchestra.run.smithi044.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-09-10T00:00:00", "created": "2023-09-10T00:40:39", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-09-10T00:40:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.926+0000 7f1a3d13c700 1 -- 172.21.15.44:0/4217446850 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1a200a0440 msgr2=0x7f1a200a2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:40.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.926+0000 7f1a3d13c700 1 --2- 172.21.15.44:0/4217446850 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1a200a0440 0x7f1a200a2900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1a38100500 tx=0x7f1a280091c0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.926+0000 7f1a3d13c700 1 -- 172.21.15.44:0/4217446850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a38101040 msgr2=0x7f1a3819d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:40.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.926+0000 7f1a3d13c700 1 --2- 172.21.15.44:0/4217446850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a38101040 0x7f1a3819d530 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009990 tx=0x7f1a30003a50 comp rx=0 tx=0).stop 2023-09-10T00:40:40.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.926+0000 7f1a3d13c700 1 -- 172.21.15.44:0/4217446850 shutdown_connections 2023-09-10T00:40:40.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.926+0000 7f1a3d13c700 1 --2- 172.21.15.44:0/4217446850 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1a200a0440 0x7f1a200a2900 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.926+0000 7f1a3d13c700 1 --2- 172.21.15.44:0/4217446850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a38101040 0x7f1a3819d530 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.926+0000 7f1a3d13c700 1 --2- 172.21.15.44:0/4217446850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a380ff4b0 0x7f1a38069490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.926+0000 7f1a3d13c700 1 --2- 172.21.15.44:0/4217446850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a381006a0 0x7f1a3819aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:40.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.926+0000 7f1a3d13c700 1 -- 172.21.15.44:0/4217446850 >> 172.21.15.44:0/4217446850 conn(0x7f1a380faa00 msgr2=0x7f1a38103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:40.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.926+0000 7f1a3d13c700 1 -- 172.21.15.44:0/4217446850 shutdown_connections 2023-09-10T00:40:40.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:40.926+0000 7f1a3d13c700 1 -- 172.21.15.44:0/4217446850 wait complete. 2023-09-10T00:40:40.939 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-09-10T00:40:41.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.077+0000 7f29c1f47700 1 -- 172.21.15.44:0/2198118699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29bc1006f0 msgr2=0x7f29bc100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:41.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.077+0000 7f29c1f47700 1 --2- 172.21.15.44:0/2198118699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29bc1006f0 0x7f29bc100b50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f29b00099c0 tx=0x7f29b0009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:41.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.077+0000 7f29c1f47700 1 -- 172.21.15.44:0/2198118699 shutdown_connections 2023-09-10T00:40:41.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.077+0000 7f29c1f47700 1 --2- 172.21.15.44:0/2198118699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29bc101090 0x7f29bc107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:41.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.077+0000 7f29c1f47700 1 --2- 172.21.15.44:0/2198118699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29bc0ff500 0x7f29bc0ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:41.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.077+0000 7f29c1f47700 1 --2- 172.21.15.44:0/2198118699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29bc1006f0 0x7f29bc100b50 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:41.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.077+0000 7f29c1f47700 1 -- 172.21.15.44:0/2198118699 >> 172.21.15.44:0/2198118699 conn(0x7f29bc0faa50 msgr2=0x7f29bc0fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:41.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.077+0000 7f29c1f47700 1 -- 172.21.15.44:0/2198118699 shutdown_connections 2023-09-10T00:40:41.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.077+0000 7f29c1f47700 1 -- 172.21.15.44:0/2198118699 wait complete. 2023-09-10T00:40:41.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.077+0000 7f29c1f47700 1 Processor -- start 2023-09-10T00:40:41.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.077+0000 7f29c1f47700 1 -- start start 2023-09-10T00:40:41.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29c1f47700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29bc0ff500 0x7f29bc101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:41.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29c1f47700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29bc1006f0 0x7f29bc102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:41.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29c1f47700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29bc101090 0x7f29bc102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:41.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29c1f47700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc1a0250 con 0x7f29bc0ff500 2023-09-10T00:40:41.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29c1f47700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc1a0390 con 0x7f29bc1006f0 2023-09-10T00:40:41.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29c1f47700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc1a04d0 con 0x7f29bc101090 2023-09-10T00:40:41.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29bb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29bc0ff500 0x7f29bc101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:41.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29bb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29bc0ff500 0x7f29bc101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48998/0 (socket says 172.21.15.44:48998) 2023-09-10T00:40:41.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29bb7fe700 1 -- 172.21.15.44:0/3660873497 learned_addr learned my addr 172.21.15.44:0/3660873497 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:41.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29baffd700 1 --2- 172.21.15.44:0/3660873497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29bc1006f0 0x7f29bc102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:41.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29bbfff700 1 --2- 172.21.15.44:0/3660873497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29bc101090 0x7f29bc102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:41.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29bb7fe700 1 -- 172.21.15.44:0/3660873497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29bc101090 msgr2=0x7f29bc102550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:41.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29bb7fe700 1 --2- 172.21.15.44:0/3660873497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29bc101090 0x7f29bc102550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:41.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29bb7fe700 1 -- 172.21.15.44:0/3660873497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29bc1006f0 msgr2=0x7f29bc102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:41.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29bb7fe700 1 --2- 172.21.15.44:0/3660873497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29bc1006f0 0x7f29bc102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:41.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29bb7fe700 1 -- 172.21.15.44:0/3660873497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b00095f0 con 0x7f29bc0ff500 2023-09-10T00:40:41.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29baffd700 1 --2- 172.21.15.44:0/3660873497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29bc1006f0 0x7f29bc102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:40:41.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29bb7fe700 1 --2- 172.21.15.44:0/3660873497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29bc0ff500 0x7f29bc101ad0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00e970 tx=0x7f29ac00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:41.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.078+0000 7f29b8ff9700 1 -- 172.21.15.44:0/3660873497 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac009790 con 0x7f29bc0ff500 2023-09-10T00:40:41.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.079+0000 7f29c1f47700 1 -- 172.21.15.44:0/3660873497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc1a0760 con 0x7f29bc0ff500 2023-09-10T00:40:41.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.079+0000 7f29b8ff9700 1 -- 172.21.15.44:0/3660873497 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f29ac00c7e0 con 0x7f29bc0ff500 2023-09-10T00:40:41.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.079+0000 7f29b8ff9700 1 -- 172.21.15.44:0/3660873497 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac003bd0 con 0x7f29bc0ff500 2023-09-10T00:40:41.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.079+0000 7f29c1f47700 1 -- 172.21.15.44:0/3660873497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29bc1a0cc0 con 0x7f29bc0ff500 2023-09-10T00:40:41.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.079+0000 7f29c1f47700 1 -- 172.21.15.44:0/3660873497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29bc04e3e0 con 0x7f29bc0ff500 2023-09-10T00:40:41.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.081+0000 7f29b8ff9700 1 -- 172.21.15.44:0/3660873497 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 22) v1 ==== 135071+0+0 (secure 0 0 0) 0x7f29ac00c950 con 0x7f29bc0ff500 2023-09-10T00:40:41.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.081+0000 7f29b8ff9700 1 --2- 172.21.15.44:0/3660873497 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29a40a0440 0x7f29a40a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:41.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.082+0000 7f29baffd700 1 --2- 172.21.15.44:0/3660873497 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29a40a0440 0x7f29a40a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:41.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.082+0000 7f29b8ff9700 1 -- 172.21.15.44:0/3660873497 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f29ac014070 con 0x7f29bc0ff500 2023-09-10T00:40:41.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.082+0000 7f29baffd700 1 --2- 172.21.15.44:0/3660873497 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29a40a0440 0x7f29a40a2900 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f29bc102dd0 tx=0x7f29b001a040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:41.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.085+0000 7f29b8ff9700 1 -- 172.21.15.44:0/3660873497 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f29ac0958e0 con 0x7f29bc0ff500 2023-09-10T00:40:41.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.322+0000 7f29c1f47700 1 -- 172.21.15.44:0/3660873497 --> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f29bc068100 con 0x7f29a40a0440 2023-09-10T00:40:41.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.329+0000 7f29b8ff9700 1 -- 172.21.15.44:0/3660873497 <== mgr.15825 v2:172.21.15.44:6800/605958391 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f29bc068100 con 0x7f29a40a0440 2023-09-10T00:40:41.330 INFO:teuthology.orchestra.run.smithi044.stdout:/ 1M 6M3h 2023-09-10T00:40:41.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.331+0000 7f29c1f47700 1 -- 172.21.15.44:0/3660873497 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29a40a0440 msgr2=0x7f29a40a2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:41.333 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.331+0000 7f29c1f47700 1 --2- 172.21.15.44:0/3660873497 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29a40a0440 0x7f29a40a2900 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f29bc102dd0 tx=0x7f29b001a040 comp rx=0 tx=0).stop 2023-09-10T00:40:41.333 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.331+0000 7f29c1f47700 1 -- 172.21.15.44:0/3660873497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29bc0ff500 msgr2=0x7f29bc101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:41.333 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.331+0000 7f29c1f47700 1 --2- 172.21.15.44:0/3660873497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29bc0ff500 0x7f29bc101ad0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00e970 tx=0x7f29ac00ec80 comp rx=0 tx=0).stop 2023-09-10T00:40:41.333 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.331+0000 7f29c1f47700 1 -- 172.21.15.44:0/3660873497 shutdown_connections 2023-09-10T00:40:41.333 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.331+0000 7f29c1f47700 1 --2- 172.21.15.44:0/3660873497 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29a40a0440 0x7f29a40a2900 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:41.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.331+0000 7f29c1f47700 1 --2- 172.21.15.44:0/3660873497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29bc101090 0x7f29bc102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:41.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.331+0000 7f29c1f47700 1 --2- 172.21.15.44:0/3660873497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29bc0ff500 0x7f29bc101ad0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:41.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.331+0000 7f29c1f47700 1 --2- 172.21.15.44:0/3660873497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29bc1006f0 0x7f29bc102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:41.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.331+0000 7f29c1f47700 1 -- 172.21.15.44:0/3660873497 >> 172.21.15.44:0/3660873497 conn(0x7f29bc0faa50 msgr2=0x7f29bc0679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:41.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.331+0000 7f29c1f47700 1 -- 172.21.15.44:0/3660873497 shutdown_connections 2023-09-10T00:40:41.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:40:41.331+0000 7f29c1f47700 1 -- 172.21.15.44:0/3660873497 wait complete. 2023-09-10T00:40:41.348 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-09-10T00:40:41.380 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-09-10T00:40:41.391 INFO:teuthology.run_tasks:Running task background_exec... 2023-09-10T00:40:41.475 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2023-09-10T00:40:41.476 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-09-10T00:40:41.480 INFO:teuthology.run_tasks:Running task exec... 2023-09-10T00:40:41.489 INFO:teuthology.task.exec:Executing custom commands... 2023-09-10T00:40:41.490 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi044.front.sepia.ceph.com 2023-09-10T00:40:41.490 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2023-09-10T00:40:41.535 INFO:teuthology.orchestra.run.smithi044.stdout:total 0 2023-09-10T00:40:41.547 INFO:teuthology.orchestra.run.smithi044.stdout:Filesystem Size Used Avail Use% Mounted on 2023-09-10T00:40:41.547 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-fuse 1.1T 332M 1.1T 1% /tmp/cephfs 2023-09-10T00:40:41.549 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2023-09-10T00:40:41.614 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:41 smithi044 ceph-mon[98676]: from='client.15843 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:41.614 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:41 smithi044 ceph-mon[98676]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:40:41.614 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:41 smithi044 ceph-mon[98676]: from='client.25492 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:41.650 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2023-09-10T00:40:41.736 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2023-09-10T00:40:41.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:41 smithi131 ceph-mon[103713]: from='client.15843 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:41.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:41 smithi131 ceph-mon[103713]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:40:41.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:41 smithi131 ceph-mon[103713]: from='client.25492 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:41.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:41 smithi099 ceph-mon[102692]: from='client.15843 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:41.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:41 smithi099 ceph-mon[102692]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:40:41.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:41 smithi099 ceph-mon[102692]: from='client.25492 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:42.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:42 smithi131 ceph-mon[103713]: from='client.15888 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:42.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:42 smithi131 ceph-mon[103713]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:40:42.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:42 smithi131 ceph-mon[103713]: from='client.15894 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:42.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:42 smithi131 ceph-mon[103713]: mgrmap e23: x(active, since 3s), standbys: z, y 2023-09-10T00:40:42.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:42 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:42 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:42 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:42 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:42 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:42 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:42 smithi044 ceph-mon[98676]: from='client.15888 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:42.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:42 smithi044 ceph-mon[98676]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:40:42.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:42 smithi044 ceph-mon[98676]: from='client.15894 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:42.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:42 smithi044 ceph-mon[98676]: mgrmap e23: x(active, since 3s), standbys: z, y 2023-09-10T00:40:42.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:42 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:42 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:42 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:42 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:42 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:42 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:42 smithi099 ceph-mon[102692]: from='client.15888 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:42.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:42 smithi099 ceph-mon[102692]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:40:42.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:42 smithi099 ceph-mon[102692]: from='client.15894 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T00:40:42.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:42 smithi099 ceph-mon[102692]: mgrmap e23: x(active, since 3s), standbys: z, y 2023-09-10T00:40:42.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:42 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:42 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:42 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:42 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:42 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:42.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:42 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.319 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2023-09-10T00:40:44.365 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2023-09-10T00:40:44.367 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2023-09-10T00:40:44.392 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:40:44.392 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.393 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.393 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.393 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.393 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.394 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.394 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.394 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.394 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.395 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.395 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.395 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.395 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.396 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.396 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.396 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.396 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T00:40:44.396 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.397 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.397 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:40:44.397 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.397 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.398 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T00:40:44.398 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.398 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T00:40:44.398 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.399 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.399 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.399 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.399 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.399 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.400 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.400 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.400 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.401 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T00:40:44.401 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T00:40:44.401 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:40:44.401 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:40:44.401 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: Updating smithi044:/etc/ceph/ceph.conf 2023-09-10T00:40:44.402 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: Updating smithi099:/etc/ceph/ceph.conf 2023-09-10T00:40:44.402 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: Updating smithi131:/etc/ceph/ceph.conf 2023-09-10T00:40:44.402 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: scrub summary: idle 2023-09-10T00:40:44.403 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: Updating smithi099:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:40:44.403 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: Updating smithi131:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:40:44.403 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: Updating smithi044:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:40:44.403 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: mgrmap e24: x(active, since 4s), standbys: z, y 2023-09-10T00:40:44.404 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:40:44.404 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:44 smithi044 ceph-mon[98676]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:40:44.411 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-09-10T00:40:44.495 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/: 2023-09-10T00:40:44.495 INFO:teuthology.orchestra.run.smithi044.stdout:total 2 2023-09-10T00:40:44.495 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T00:40:44.495 INFO:teuthology.orchestra.run.smithi044.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 .. 2023-09-10T00:40:44.496 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 data 2023-09-10T00:40:44.496 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:44.496 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data: 2023-09-10T00:40:44.496 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T00:40:44.497 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T00:40:44.497 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 .. 2023-09-10T00:40:44.498 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2023-09-10T00:40:44.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:40:44.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T00:40:44.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.550 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.550 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:40:44.550 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.550 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.551 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T00:40:44.551 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.551 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T00:40:44.551 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.551 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.552 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.552 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.552 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.552 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T00:40:44.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T00:40:44.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:40:44.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:40:44.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: Updating smithi044:/etc/ceph/ceph.conf 2023-09-10T00:40:44.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: Updating smithi099:/etc/ceph/ceph.conf 2023-09-10T00:40:44.555 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: Updating smithi131:/etc/ceph/ceph.conf 2023-09-10T00:40:44.555 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: scrub summary: idle 2023-09-10T00:40:44.555 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: Updating smithi099:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:40:44.555 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: Updating smithi131:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:40:44.556 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: Updating smithi044:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:40:44.556 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: mgrmap e24: x(active, since 4s), standbys: z, y 2023-09-10T00:40:44.556 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:40:44.556 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:44 smithi131 ceph-mon[103713]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:40:44.558 INFO:teuthology.orchestra.run.smithi044.stdout:[Service] 2023-09-10T00:40:44.558 INFO:teuthology.orchestra.run.smithi044.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2023-09-10T00:40:44.558 INFO:teuthology.orchestra.run.smithi044.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2023-09-10T00:40:44.559 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2023-09-10T00:40:44.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-10T00:40:44.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T00:40:44.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:40:44.611 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.611 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.611 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T00:40:44.611 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.611 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T00:40:44.612 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:44.612 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.612 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.612 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.613 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.613 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.613 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.613 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.613 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:40:44.614 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T00:40:44.614 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T00:40:44.614 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:40:44.614 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:40:44.614 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: Updating smithi044:/etc/ceph/ceph.conf 2023-09-10T00:40:44.615 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: Updating smithi099:/etc/ceph/ceph.conf 2023-09-10T00:40:44.615 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: Updating smithi131:/etc/ceph/ceph.conf 2023-09-10T00:40:44.615 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: scrub summary: idle 2023-09-10T00:40:44.615 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: Updating smithi099:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:40:44.615 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: Updating smithi131:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:40:44.616 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: Updating smithi044:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-10T00:40:44.616 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: mgrmap e24: x(active, since 4s), standbys: z, y 2023-09-10T00:40:44.616 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:40:44.616 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:44 smithi099 ceph-mon[102692]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:40:44.648 INFO:teuthology.orchestra.run.smithi044.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2023-09-10T00:40:45.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:45 smithi099 ceph-mon[102692]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:40:45.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:45 smithi099 ceph-mon[102692]: Updating smithi131:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-10T00:40:45.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:45 smithi099 ceph-mon[102692]: Updating smithi099:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-10T00:40:45.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:45 smithi099 ceph-mon[102692]: Updating smithi044:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-10T00:40:45.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:45 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:45 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:45 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:45 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:45 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:45 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:45 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:45 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:40:45.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:45 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:40:45.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:45 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:40:45.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:45 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:45 smithi131 ceph-mon[103713]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:40:45.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:45 smithi131 ceph-mon[103713]: Updating smithi131:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-10T00:40:45.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:45 smithi131 ceph-mon[103713]: Updating smithi099:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-10T00:40:45.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:45 smithi131 ceph-mon[103713]: Updating smithi044:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-10T00:40:45.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:45 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:45 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:45 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:45 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:45 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:45 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:45 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:45 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:40:45.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:45 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:40:45.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:45 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:40:45.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:45 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:45 smithi044 ceph-mon[98676]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2023-09-10T00:40:45.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:45 smithi044 ceph-mon[98676]: Updating smithi131:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-10T00:40:45.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:45 smithi044 ceph-mon[98676]: Updating smithi099:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-10T00:40:45.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:45 smithi044 ceph-mon[98676]: Updating smithi044:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-10T00:40:45.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:45 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:45 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:45 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:45 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:45 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:45 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:45 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:45.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:45 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:40:45.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:45 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:40:45.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:45 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:40:45.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:45 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:40:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:46 smithi131 ceph-mon[103713]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 615 B/s wr, 1 op/s 2023-09-10T00:40:47.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:46 smithi044 ceph-mon[98676]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 615 B/s wr, 1 op/s 2023-09-10T00:40:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:46 smithi099 ceph-mon[102692]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 615 B/s wr, 1 op/s 2023-09-10T00:40:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:48 smithi131 ceph-mon[103713]: pgmap v7: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 160 KiB/s wr, 74 op/s 2023-09-10T00:40:49.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:48 smithi044 ceph-mon[98676]: pgmap v7: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 160 KiB/s wr, 74 op/s 2023-09-10T00:40:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:48 smithi099 ceph-mon[102692]: pgmap v7: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 160 KiB/s wr, 74 op/s 2023-09-10T00:40:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:50 smithi131 ceph-mon[103713]: pgmap v8: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 124 KiB/s wr, 58 op/s 2023-09-10T00:40:51.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:50 smithi044 ceph-mon[98676]: pgmap v8: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 124 KiB/s wr, 58 op/s 2023-09-10T00:40:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:50 smithi099 ceph-mon[102692]: pgmap v8: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 124 KiB/s wr, 58 op/s 2023-09-10T00:40:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:52 smithi131 ceph-mon[103713]: pgmap v9: 129 pgs: 129 active+clean; 30 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 3.5 MiB/s wr, 124 op/s 2023-09-10T00:40:53.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:52 smithi044 ceph-mon[98676]: pgmap v9: 129 pgs: 129 active+clean; 30 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 3.5 MiB/s wr, 124 op/s 2023-09-10T00:40:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:52 smithi099 ceph-mon[102692]: pgmap v9: 129 pgs: 129 active+clean; 30 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 3.5 MiB/s wr, 124 op/s 2023-09-10T00:40:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.653+0000 7f7322d1c700 1 -- 172.21.15.44:0/2309516451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c0ffa30 msgr2=0x7f731c1038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.653+0000 7f7322d1c700 1 --2- 172.21.15.44:0/2309516451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c0ffa30 0x7f731c1038b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f73080099c0 tx=0x7f7308009cd0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.654+0000 7f7322d1c700 1 -- 172.21.15.44:0/2309516451 shutdown_connections 2023-09-10T00:40:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.654+0000 7f7322d1c700 1 --2- 172.21.15.44:0/2309516451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f731c103fe0 0x7f731c10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.654+0000 7f7322d1c700 1 --2- 172.21.15.44:0/2309516451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f731c0ff080 0x7f731c0ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.654+0000 7f7322d1c700 1 --2- 172.21.15.44:0/2309516451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c0ffa30 0x7f731c1038b0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.654+0000 7f7322d1c700 1 -- 172.21.15.44:0/2309516451 >> 172.21.15.44:0/2309516451 conn(0x7f731c075920 msgr2=0x7f731c075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.654+0000 7f7322d1c700 1 -- 172.21.15.44:0/2309516451 shutdown_connections 2023-09-10T00:40:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.654+0000 7f7322d1c700 1 -- 172.21.15.44:0/2309516451 wait complete. 2023-09-10T00:40:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.654+0000 7f7322d1c700 1 Processor -- start 2023-09-10T00:40:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.654+0000 7f7322d1c700 1 -- start start 2023-09-10T00:40:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.654+0000 7f7322d1c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f731c0ff080 0x7f731c19b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.654+0000 7f7322d1c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c103fe0 0x7f731c19ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.654+0000 7f7322d1c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f731c1953b0 0x7f731c195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.654+0000 7f7322d1c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c195e30 con 0x7f731c103fe0 2023-09-10T00:40:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f7322d1c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c195fa0 con 0x7f731c1953b0 2023-09-10T00:40:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f7322d1c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c196160 con 0x7f731c0ff080 2023-09-10T00:40:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f731bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c103fe0 0x7f731c19ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f731bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c103fe0 0x7f731c19ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49052/0 (socket says 172.21.15.44:49052) 2023-09-10T00:40:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f731bfff700 1 -- 172.21.15.44:0/3781560064 learned_addr learned my addr 172.21.15.44:0/3781560064 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f73212b9700 1 --2- 172.21.15.44:0/3781560064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f731c1953b0 0x7f731c195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f7320ab8700 1 --2- 172.21.15.44:0/3781560064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f731c0ff080 0x7f731c19b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f731bfff700 1 -- 172.21.15.44:0/3781560064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f731c0ff080 msgr2=0x7f731c19b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f731bfff700 1 --2- 172.21.15.44:0/3781560064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f731c0ff080 0x7f731c19b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f731bfff700 1 -- 172.21.15.44:0/3781560064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f731c1953b0 msgr2=0x7f731c195860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f731bfff700 1 --2- 172.21.15.44:0/3781560064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f731c1953b0 0x7f731c195860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f731bfff700 1 -- 172.21.15.44:0/3781560064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73080095f0 con 0x7f731c103fe0 2023-09-10T00:40:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f7320ab8700 1 --2- 172.21.15.44:0/3781560064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f731c0ff080 0x7f731c19b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f731bfff700 1 --2- 172.21.15.44:0/3781560064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c103fe0 0x7f731c19ba90 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7308009fd0 tx=0x7f730800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f7319ffb700 1 -- 172.21.15.44:0/3781560064 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730801d070 con 0x7f731c103fe0 2023-09-10T00:40:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f7319ffb700 1 -- 172.21.15.44:0/3781560064 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f73080059a0 con 0x7f731c103fe0 2023-09-10T00:40:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f7322d1c700 1 -- 172.21.15.44:0/3781560064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731c1a23c0 con 0x7f731c103fe0 2023-09-10T00:40:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.655+0000 7f7319ffb700 1 -- 172.21.15.44:0/3781560064 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308022ea0 con 0x7f731c103fe0 2023-09-10T00:40:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.656+0000 7f7322d1c700 1 -- 172.21.15.44:0/3781560064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731c1a2840 con 0x7f731c103fe0 2023-09-10T00:40:54.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.659+0000 7f732eee2b80 1 Processor -- start 2023-09-10T00:40:54.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.659+0000 7f7319ffb700 1 -- 172.21.15.44:0/3781560064 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f730800fb80 con 0x7f731c103fe0 2023-09-10T00:40:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.659+0000 7f732eee2b80 1 -- start start 2023-09-10T00:40:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.659+0000 7f732eee2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea5484c700 0x55ea5484cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.659+0000 7f732eee2b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea54850f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.659+0000 7f732eee2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea54851480 0x55ea548518e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.659+0000 7f732eee2b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea547a36d0 con 0x55ea5484d020 2023-09-10T00:40:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.659+0000 7f732eee2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea547a0ff0 con 0x55ea54851480 2023-09-10T00:40:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.659+0000 7f732eee2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea5479c8e0 con 0x55ea5484c700 2023-09-10T00:40:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.659+0000 7f7320ab8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea54850f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.659+0000 7f7320ab8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea54850f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49058/0 (socket says 172.21.15.44:49058) 2023-09-10T00:40:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.659+0000 7f7320ab8700 1 -- 172.21.15.44:0/2189894461 learned_addr learned my addr 172.21.15.44:0/2189894461 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.660+0000 7f73212b9700 1 --2- 172.21.15.44:0/2189894461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea5484c700 0x55ea5484cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.660+0000 7f731bfff700 1 --2- 172.21.15.44:0/2189894461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea54851480 0x55ea548518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.660+0000 7f7319ffb700 1 --2- 172.21.15.44:0/3781560064 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f730c0a0840 0x7f730c0a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.660+0000 7f7320ab8700 1 -- 172.21.15.44:0/2189894461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea5484c700 msgr2=0x55ea5484cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.660+0000 7f7320ab8700 1 --2- 172.21.15.44:0/2189894461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea5484c700 0x55ea5484cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.660+0000 7f7320ab8700 1 -- 172.21.15.44:0/2189894461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea54851480 msgr2=0x55ea548518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.660+0000 7f7320ab8700 1 --2- 172.21.15.44:0/2189894461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea54851480 0x55ea548518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.660+0000 7f7320ab8700 1 -- 172.21.15.44:0/2189894461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea545c8d80 con 0x55ea5484d020 2023-09-10T00:40:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.660+0000 7f731bfff700 1 --2- 172.21.15.44:0/2189894461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea54851480 0x55ea548518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.660+0000 7f73212b9700 1 --2- 172.21.15.44:0/3781560064 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f730c0a0840 0x7f730c0a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.660+0000 7f7319ffb700 1 -- 172.21.15.44:0/3781560064 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f73080cd440 con 0x7f731c103fe0 2023-09-10T00:40:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.660+0000 7f7320ab8700 1 --2- 172.21.15.44:0/2189894461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea54850f40 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f731000bd40 tx=0x7f731000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86063232697caf1e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.660+0000 7f7306ffd700 1 -- 172.21.15.44:0/2189894461 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731000fe50 con 0x55ea5484d020 2023-09-10T00:40:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.660+0000 7f7306ffd700 1 -- 172.21.15.44:0/2189894461 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f731000be90 con 0x55ea5484d020 2023-09-10T00:40:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.660+0000 7f7306ffd700 1 -- 172.21.15.44:0/2189894461 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73100116f0 con 0x55ea5484d020 2023-09-10T00:40:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.661+0000 7f732eee2b80 1 -- 172.21.15.44:0/2189894461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 msgr2=0x55ea54850f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.661+0000 7f73212b9700 1 --2- 172.21.15.44:0/3781560064 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f730c0a0840 0x7f730c0a2d00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7314009ec0 tx=0x7f7314009040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.661+0000 7f732eee2b80 1 --2- 172.21.15.44:0/2189894461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea54850f40 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f731000bd40 tx=0x7f731000f640 comp rx=0 tx=0).stop 2023-09-10T00:40:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.661+0000 7f732eee2b80 1 -- 172.21.15.44:0/2189894461 shutdown_connections 2023-09-10T00:40:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.661+0000 7f732eee2b80 1 --2- 172.21.15.44:0/2189894461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea54851480 0x55ea548518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.661+0000 7f732eee2b80 1 --2- 172.21.15.44:0/2189894461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea5484c700 0x55ea5484cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.661+0000 7f732eee2b80 1 --2- 172.21.15.44:0/2189894461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea54850f40 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.661+0000 7f732eee2b80 1 -- 172.21.15.44:0/2189894461 >> 172.21.15.44:0/2189894461 conn(0x55ea547bf030 msgr2=0x55ea54845560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.661+0000 7f732eee2b80 1 -- 172.21.15.44:0/2189894461 shutdown_connections 2023-09-10T00:40:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.661+0000 7f732eee2b80 1 -- 172.21.15.44:0/2189894461 wait complete. 2023-09-10T00:40:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.661+0000 7f732eee2b80 1 -- start start 2023-09-10T00:40:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.661+0000 7f732eee2b80 1 Processor -- start 2023-09-10T00:40:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f732eee2b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484c700 0x55ea54858bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f732eee2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea5484d020 0x55ea54855e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f732eee2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea54851480 0x55ea54856440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f732eee2b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea5473ab10 con 0x55ea5484c700 2023-09-10T00:40:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f732eee2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea5473a810 con 0x55ea5484d020 2023-09-10T00:40:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f732eee2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea5461e8c0 con 0x55ea54851480 2023-09-10T00:40:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f732eee2b80 10 client.0 _ll_register_callbacks cb 0x55ea54684a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:40:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f7320ab8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484c700 0x55ea54858bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f7320ab8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484c700 0x55ea54858bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49064/0 (socket says 172.21.15.44:49064) 2023-09-10T00:40:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f7320ab8700 1 -- 172.21.15.44:0/2622013992 learned_addr learned my addr 172.21.15.44:0/2622013992 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f73212b9700 1 --2- 172.21.15.44:0/2622013992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea54851480 0x55ea54856440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f731bfff700 1 --2- 172.21.15.44:0/2622013992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea5484d020 0x55ea54855e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f7320ab8700 1 -- 172.21.15.44:0/2622013992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea54851480 msgr2=0x55ea54856440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f7320ab8700 1 --2- 172.21.15.44:0/2622013992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea54851480 0x55ea54856440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f7320ab8700 1 -- 172.21.15.44:0/2622013992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea5484d020 msgr2=0x55ea54855e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f7320ab8700 1 --2- 172.21.15.44:0/2622013992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea5484d020 0x55ea54855e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f7320ab8700 1 -- 172.21.15.44:0/2622013992 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731000b9a0 con 0x55ea5484c700 2023-09-10T00:40:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f731bfff700 1 --2- 172.21.15.44:0/2622013992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea5484d020 0x55ea54855e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:40:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f7320ab8700 1 --2- 172.21.15.44:0/2622013992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484c700 0x55ea54858bb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f731001af50 tx=0x7f731000ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b33b2dc5178e6cb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f72fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:40:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.662+0000 7f72fbfff700 1 -- 172.21.15.44:0/2622013992 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731001a870 con 0x55ea5484c700 2023-09-10T00:40:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.663+0000 7f72fbfff700 1 -- 172.21.15.44:0/2622013992 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f731001a9d0 con 0x55ea5484c700 2023-09-10T00:40:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.663+0000 7f732eee2b80 1 -- 172.21.15.44:0/2622013992 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea546f4490 con 0x55ea5484c700 2023-09-10T00:40:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.663+0000 7f72fbfff700 1 -- 172.21.15.44:0/2622013992 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7310005040 con 0x55ea5484c700 2023-09-10T00:40:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.663+0000 7f732eee2b80 10 client.15912 fetch_fsmap learned FSMap version 27 2023-09-10T00:40:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.663+0000 7f732eee2b80 1 -- 172.21.15.44:0/2622013992 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ea545b1c00 con 0x55ea5484c700 2023-09-10T00:40:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.663+0000 7f72fbfff700 1 -- 172.21.15.44:0/2622013992 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ea546f4490 con 0x55ea5484c700 2023-09-10T00:40:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.663+0000 7f72fbfff700 1 -- 172.21.15.44:0/2622013992 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7310023030 con 0x55ea5484c700 2023-09-10T00:40:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.663+0000 7f732eee2b80 10 client.15912 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:40:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.663+0000 7f732eee2b80 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:40:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.663+0000 7f732eee2b80 20 client.15912 populate_metadata read hostname 'smithi044' 2023-09-10T00:40:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.663+0000 7f732eee2b80 1 --2- 172.21.15.44:0/2622013992 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ea54866420 0x55ea54868810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.663+0000 7f732eee2b80 4 client.15912 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:40:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.663+0000 7f732eee2b80 1 -- 172.21.15.44:0/2622013992 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea547b1840 con 0x55ea54866420 2023-09-10T00:40:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.664+0000 7f731bfff700 1 --2- 172.21.15.44:0/2622013992 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ea54866420 0x55ea54868810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.664+0000 7f72fbfff700 10 client.15912 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:40:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.664+0000 7f731bfff700 1 --2- 172.21.15.44:0/2622013992 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ea54866420 0x55ea54868810 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7420b8ce500c6db server_cookie=c33ae727935360aa in_seq=0 out_seq=0 2023-09-10T00:40:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.664+0000 7f72fbfff700 1 -- 172.21.15.44:0/2622013992 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ea547b1840 con 0x55ea54866420 2023-09-10T00:40:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.664+0000 7f72fbfff700 10 client.15912 handle_command_reply: tid=0 2023-09-10T00:40:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.664+0000 7f732eee2b80 1 client.15912 shutdown 2023-09-10T00:40:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.665+0000 7f732eee2b80 1 -- 172.21.15.44:0/2622013992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484c700 msgr2=0x55ea54858bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.665+0000 7f732eee2b80 1 --2- 172.21.15.44:0/2622013992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484c700 0x55ea54858bb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f731001af50 tx=0x7f731000ff00 comp rx=0 tx=0).stop 2023-09-10T00:40:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.665+0000 7f732eee2b80 1 -- 172.21.15.44:0/2622013992 shutdown_connections 2023-09-10T00:40:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.665+0000 7f732eee2b80 1 --2- 172.21.15.44:0/2622013992 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ea54866420 0x55ea54868810 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.665+0000 7f732eee2b80 1 --2- 172.21.15.44:0/2622013992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea54851480 0x55ea54856440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.665+0000 7f72fbfff700 0 client.15912 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:40:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.665+0000 7f732eee2b80 1 --2- 172.21.15.44:0/2622013992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484c700 0x55ea54858bb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.665+0000 7f732eee2b80 1 --2- 172.21.15.44:0/2622013992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea5484d020 0x55ea54855e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.665+0000 7f732eee2b80 1 -- 172.21.15.44:0/2622013992 >> 172.21.15.44:0/2622013992 conn(0x55ea547bf030 msgr2=0x55ea548528e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.665+0000 7f732eee2b80 1 -- 172.21.15.44:0/2622013992 shutdown_connections 2023-09-10T00:40:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.665+0000 7f732eee2b80 1 -- 172.21.15.44:0/2622013992 wait complete. 2023-09-10T00:40:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.665+0000 7f732eee2b80 20 client.15912 trim_cache size 0 max 16384 2023-09-10T00:40:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.680+0000 7f732eee2b80 1 Processor -- start 2023-09-10T00:40:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.680+0000 7f732eee2b80 1 -- start start 2023-09-10T00:40:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.680+0000 7f732eee2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea5484c700 0x55ea54845610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.680+0000 7f732eee2b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea5485d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.680+0000 7f732eee2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea54851480 0x55ea54858750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.680+0000 7f732eee2b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea54859300 con 0x55ea5484d020 2023-09-10T00:40:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.680+0000 7f732eee2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea5483f8a0 con 0x55ea54851480 2023-09-10T00:40:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.680+0000 7f732eee2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea5484d580 con 0x55ea5484c700 2023-09-10T00:40:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.680+0000 7f731bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea5485d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.680+0000 7f731bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea5485d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49072/0 (socket says 172.21.15.44:49072) 2023-09-10T00:40:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.680+0000 7f731bfff700 1 -- 172.21.15.44:0/3113030468 learned_addr learned my addr 172.21.15.44:0/3113030468 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.680+0000 7f73212b9700 1 --2- 172.21.15.44:0/3113030468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea54851480 0x55ea54858750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.680+0000 7f7320ab8700 1 --2- 172.21.15.44:0/3113030468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea5484c700 0x55ea54845610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.681+0000 7f731bfff700 1 -- 172.21.15.44:0/3113030468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea5484c700 msgr2=0x55ea54845610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.681+0000 7f731bfff700 1 --2- 172.21.15.44:0/3113030468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea5484c700 0x55ea54845610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.681+0000 7f731bfff700 1 -- 172.21.15.44:0/3113030468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea54851480 msgr2=0x55ea54858750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.681+0000 7f731bfff700 1 --2- 172.21.15.44:0/3113030468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea54851480 0x55ea54858750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.681+0000 7f731bfff700 1 -- 172.21.15.44:0/3113030468 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea545ef1f0 con 0x55ea5484d020 2023-09-10T00:40:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.681+0000 7f73212b9700 1 --2- 172.21.15.44:0/3113030468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea54851480 0x55ea54858750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:40:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.681+0000 7f731bfff700 1 --2- 172.21.15.44:0/3113030468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea5485d640 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f730809a170 tx=0x7f730801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69704c5e718c2337 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.681+0000 7f73077fe700 1 -- 172.21.15.44:0/3113030468 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308099fb0 con 0x55ea5484d020 2023-09-10T00:40:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.681+0000 7f73077fe700 1 -- 172.21.15.44:0/3113030468 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7308099dd0 con 0x55ea5484d020 2023-09-10T00:40:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.681+0000 7f73077fe700 1 -- 172.21.15.44:0/3113030468 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73080a2450 con 0x55ea5484d020 2023-09-10T00:40:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.682+0000 7f732eee2b80 1 -- 172.21.15.44:0/3113030468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 msgr2=0x55ea5485d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.682+0000 7f732eee2b80 1 --2- 172.21.15.44:0/3113030468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea5485d640 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f730809a170 tx=0x7f730801a040 comp rx=0 tx=0).stop 2023-09-10T00:40:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.682+0000 7f732eee2b80 1 -- 172.21.15.44:0/3113030468 shutdown_connections 2023-09-10T00:40:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.682+0000 7f732eee2b80 1 --2- 172.21.15.44:0/3113030468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea54851480 0x55ea54858750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.682+0000 7f732eee2b80 1 --2- 172.21.15.44:0/3113030468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea5484c700 0x55ea54845610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.682+0000 7f732eee2b80 1 --2- 172.21.15.44:0/3113030468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea5485d640 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.682+0000 7f732eee2b80 1 -- 172.21.15.44:0/3113030468 >> 172.21.15.44:0/3113030468 conn(0x55ea547bf030 msgr2=0x55ea54862820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.682+0000 7f732eee2b80 1 -- 172.21.15.44:0/3113030468 shutdown_connections 2023-09-10T00:40:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.682+0000 7f732eee2b80 1 -- 172.21.15.44:0/3113030468 wait complete. 2023-09-10T00:40:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.682+0000 7f732eee2b80 1 -- start start 2023-09-10T00:40:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.682+0000 7f732eee2b80 1 Processor -- start 2023-09-10T00:40:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f732eee2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea5484c700 0x55ea54859620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f732eee2b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea54859b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f732eee2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea54851480 0x55ea5485a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f732eee2b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea548bb7e0 con 0x55ea5484d020 2023-09-10T00:40:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f732eee2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea548bb950 con 0x55ea5484c700 2023-09-10T00:40:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f732eee2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea548bbb10 con 0x55ea54851480 2023-09-10T00:40:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f732eee2b80 10 client.0 _ll_register_callbacks cb 0x7f731000bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:40:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f731bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea54859b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f731bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea54859b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49080/0 (socket says 172.21.15.44:49080) 2023-09-10T00:40:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f731bfff700 1 -- 172.21.15.44:0/3059656754 learned_addr learned my addr 172.21.15.44:0/3059656754 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:40:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f73212b9700 1 --2- 172.21.15.44:0/3059656754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea54851480 0x55ea5485a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f7320ab8700 1 --2- 172.21.15.44:0/3059656754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea5484c700 0x55ea54859620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f731bfff700 1 -- 172.21.15.44:0/3059656754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea54851480 msgr2=0x55ea5485a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f731bfff700 1 --2- 172.21.15.44:0/3059656754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea54851480 0x55ea5485a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f731bfff700 1 -- 172.21.15.44:0/3059656754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea5484c700 msgr2=0x55ea54859620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:40:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f731bfff700 1 --2- 172.21.15.44:0/3059656754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea5484c700 0x55ea54859620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f731bfff700 1 -- 172.21.15.44:0/3059656754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731c1a2840 con 0x55ea5484d020 2023-09-10T00:40:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f731bfff700 1 --2- 172.21.15.44:0/3059656754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea54859b60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f73080ca810 tx=0x7f73080ca700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe227bf11b683f20 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:40:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f72fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:40:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f72fbfff700 1 -- 172.21.15.44:0/3059656754 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730809a470 con 0x55ea5484d020 2023-09-10T00:40:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f72fbfff700 1 -- 172.21.15.44:0/3059656754 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f730809ab70 con 0x55ea5484d020 2023-09-10T00:40:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f732eee2b80 1 -- 172.21.15.44:0/3059656754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f731000be90 con 0x55ea5484d020 2023-09-10T00:40:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.683+0000 7f72fbfff700 1 -- 172.21.15.44:0/3059656754 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73080a2450 con 0x55ea5484d020 2023-09-10T00:40:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.684+0000 7f732eee2b80 10 client.15918 fetch_fsmap learned FSMap version 27 2023-09-10T00:40:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.684+0000 7f732eee2b80 1 -- 172.21.15.44:0/3059656754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ea545c7250 con 0x55ea5484d020 2023-09-10T00:40:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.684+0000 7f72fbfff700 1 -- 172.21.15.44:0/3059656754 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f731000be90 con 0x55ea5484d020 2023-09-10T00:40:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.684+0000 7f72fbfff700 1 -- 172.21.15.44:0/3059656754 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f730809a760 con 0x55ea5484d020 2023-09-10T00:40:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.684+0000 7f732eee2b80 10 client.15918 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:40:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.684+0000 7f732eee2b80 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:40:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.684+0000 7f732eee2b80 20 client.15918 populate_metadata read hostname 'smithi044' 2023-09-10T00:40:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.684+0000 7f732eee2b80 1 --2- 172.21.15.44:0/3059656754 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ea548bcd80 0x55ea548bf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:40:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.684+0000 7f732eee2b80 4 client.15918 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:40:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.684+0000 7f732eee2b80 1 -- 172.21.15.44:0/3059656754 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ea547958c0 con 0x55ea548bcd80 2023-09-10T00:40:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.684+0000 7f7320ab8700 1 --2- 172.21.15.44:0/3059656754 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ea548bcd80 0x55ea548bf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:40:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.684+0000 7f72fbfff700 10 client.15918 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:40:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:54.684+0000 7f7320ab8700 1 --2- 172.21.15.44:0/3059656754 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ea548bcd80 0x55ea548bf1a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29d6b2b1f5816bab server_cookie=1493adf304b3f800 in_seq=0 out_seq=0 2023-09-10T00:40:55.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:54 smithi131 ceph-mon[103713]: pgmap v10: 129 pgs: 129 active+clean; 30 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 3.2 MiB/s wr, 114 op/s 2023-09-10T00:40:55.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:54 smithi044 ceph-mon[98676]: pgmap v10: 129 pgs: 129 active+clean; 30 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 3.2 MiB/s wr, 114 op/s 2023-09-10T00:40:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:54 smithi099 ceph-mon[102692]: pgmap v10: 129 pgs: 129 active+clean; 30 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 3.2 MiB/s wr, 114 op/s 2023-09-10T00:40:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.109+0000 7f72fbfff700 1 -- 172.21.15.44:0/3059656754 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ea547958c0 con 0x55ea548bcd80 2023-09-10T00:40:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.109+0000 7f72fbfff700 10 client.15918 handle_command_reply: tid=0 2023-09-10T00:40:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.109+0000 7f732eee2b80 1 client.15918 shutdown 2023-09-10T00:40:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.109+0000 7f732eee2b80 1 -- 172.21.15.44:0/3059656754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 msgr2=0x55ea54859b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.109+0000 7f732eee2b80 1 --2- 172.21.15.44:0/3059656754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea54859b60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f73080ca810 tx=0x7f73080ca700 comp rx=0 tx=0).stop 2023-09-10T00:40:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.109+0000 7f732eee2b80 1 -- 172.21.15.44:0/3059656754 shutdown_connections 2023-09-10T00:40:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.109+0000 7f732eee2b80 1 --2- 172.21.15.44:0/3059656754 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ea548bcd80 0x55ea548bf1a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.109+0000 7f732eee2b80 1 --2- 172.21.15.44:0/3059656754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea54851480 0x55ea5485a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.109+0000 7f72fbfff700 0 client.15918 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:40:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.109+0000 7f732eee2b80 1 --2- 172.21.15.44:0/3059656754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea5484c700 0x55ea54859620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.109+0000 7f732eee2b80 1 --2- 172.21.15.44:0/3059656754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5484d020 0x55ea54859b60 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.109+0000 7f732eee2b80 1 -- 172.21.15.44:0/3059656754 >> 172.21.15.44:0/3059656754 conn(0x55ea547bf030 msgr2=0x55ea548503e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.109+0000 7f732eee2b80 1 -- 172.21.15.44:0/3059656754 shutdown_connections 2023-09-10T00:40:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.110+0000 7f732eee2b80 1 -- 172.21.15.44:0/3059656754 wait complete. 2023-09-10T00:40:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.110+0000 7f732eee2b80 20 client.15918 trim_cache size 0 max 16384 2023-09-10T00:40:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:40:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:40:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:40:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:40:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.110+0000 7f7322d1c700 1 -- 172.21.15.44:0/3781560064 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f730c0a0840 msgr2=0x7f730c0a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.110+0000 7f7322d1c700 1 --2- 172.21.15.44:0/3781560064 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f730c0a0840 0x7f730c0a2d00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7314009ec0 tx=0x7f7314009040 comp rx=0 tx=0).stop 2023-09-10T00:40:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.110+0000 7f7322d1c700 1 -- 172.21.15.44:0/3781560064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c103fe0 msgr2=0x7f731c19ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:40:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.110+0000 7f7322d1c700 1 --2- 172.21.15.44:0/3781560064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c103fe0 0x7f731c19ba90 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7308009fd0 tx=0x7f730800f690 comp rx=0 tx=0).stop 2023-09-10T00:40:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.111+0000 7f7322d1c700 1 -- 172.21.15.44:0/3781560064 shutdown_connections 2023-09-10T00:40:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.111+0000 7f7322d1c700 1 --2- 172.21.15.44:0/3781560064 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f730c0a0840 0x7f730c0a2d00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.111+0000 7f7322d1c700 1 --2- 172.21.15.44:0/3781560064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f731c1953b0 0x7f731c195860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.111+0000 7f7322d1c700 1 --2- 172.21.15.44:0/3781560064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f731c0ff080 0x7f731c19b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.111+0000 7f7322d1c700 1 --2- 172.21.15.44:0/3781560064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c103fe0 0x7f731c19ba90 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:40:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.111+0000 7f7322d1c700 1 -- 172.21.15.44:0/3781560064 >> 172.21.15.44:0/3781560064 conn(0x7f731c075920 msgr2=0x7f731c0fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:40:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.111+0000 7f7322d1c700 1 -- 172.21.15.44:0/3781560064 shutdown_connections 2023-09-10T00:40:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:40:55.111+0000 7f7322d1c700 1 -- 172.21.15.44:0/3781560064 wait complete. 2023-09-10T00:40:57.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:56 smithi131 ceph-mon[103713]: pgmap v11: 129 pgs: 129 active+clean; 32 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 3.6 MiB/s wr, 158 op/s 2023-09-10T00:40:57.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:56 smithi044 ceph-mon[98676]: pgmap v11: 129 pgs: 129 active+clean; 32 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 3.6 MiB/s wr, 158 op/s 2023-09-10T00:40:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:56 smithi099 ceph-mon[102692]: pgmap v11: 129 pgs: 129 active+clean; 32 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 3.6 MiB/s wr, 158 op/s 2023-09-10T00:40:58.765 INFO:teuthology.orchestra.run.smithi044.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2023-09-10T00:40:58.768 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-09-10T00:40:59.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:40:58 smithi131 ceph-mon[103713]: pgmap v12: 129 pgs: 129 active+clean; 38 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 4.3 MiB/s wr, 303 op/s 2023-09-10T00:40:59.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:40:58 smithi044 ceph-mon[98676]: pgmap v12: 129 pgs: 129 active+clean; 38 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 4.3 MiB/s wr, 303 op/s 2023-09-10T00:40:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:40:58 smithi099 ceph-mon[102692]: pgmap v12: 129 pgs: 129 active+clean; 38 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 4.3 MiB/s wr, 303 op/s 2023-09-10T00:40:59.121 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/: 2023-09-10T00:40:59.121 INFO:teuthology.orchestra.run.smithi044.stdout:total 2 2023-09-10T00:40:59.122 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 . 2023-09-10T00:40:59.122 INFO:teuthology.orchestra.run.smithi044.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.122 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 data 2023-09-10T00:40:59.123 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.124 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data: 2023-09-10T00:40:59.124 INFO:teuthology.orchestra.run.smithi044.stdout:total 40 2023-09-10T00:40:59.124 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 . 2023-09-10T00:40:59.124 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.125 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22095369 Sep 10 00:40 base 2023-09-10T00:40:59.125 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 565760 Sep 10 00:40 global 2023-09-10T00:40:59.125 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 log 2023-09-10T00:40:59.125 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 pg_commit_ts 2023-09-10T00:40:59.125 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 pg_dynshmem 2023-09-10T00:40:59.126 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Sep 10 00:40 pg_hba.conf 2023-09-10T00:40:59.126 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 10 00:40 pg_ident.conf 2023-09-10T00:40:59.126 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 10 00:40 pg_logical 2023-09-10T00:40:59.126 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 pg_multixact 2023-09-10T00:40:59.126 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 pg_notify 2023-09-10T00:40:59.127 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 pg_replslot 2023-09-10T00:40:59.127 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 pg_serial 2023-09-10T00:40:59.127 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 pg_snapshots 2023-09-10T00:40:59.127 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 pg_stat 2023-09-10T00:40:59.128 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 pg_stat_tmp 2023-09-10T00:40:59.128 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 pg_subtrans 2023-09-10T00:40:59.128 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 pg_tblspc 2023-09-10T00:40:59.128 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 pg_twophase 2023-09-10T00:40:59.128 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 00:40 PG_VERSION 2023-09-10T00:40:59.129 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 00:40 pg_wal 2023-09-10T00:40:59.129 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 pg_xact 2023-09-10T00:40:59.129 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 10 00:40 postgresql.auto.conf 2023-09-10T00:40:59.129 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Sep 10 00:40 postgresql.conf 2023-09-10T00:40:59.129 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.129 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/base: 2023-09-10T00:40:59.130 INFO:teuthology.orchestra.run.smithi044.stdout:total 3 2023-09-10T00:40:59.130 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22095369 Sep 10 00:40 . 2023-09-10T00:40:59.130 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.130 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7365123 Sep 10 00:40 1 2023-09-10T00:40:59.130 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7365123 Sep 10 00:40 12346 2023-09-10T00:40:59.131 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7365123 Sep 10 00:40 12347 2023-09-10T00:40:59.131 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.131 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/base/1: 2023-09-10T00:40:59.131 INFO:teuthology.orchestra.run.smithi044.stdout:total 7194 2023-09-10T00:40:59.131 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7365123 Sep 10 00:40 . 2023-09-10T00:40:59.132 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22095369 Sep 10 00:40 .. 2023-09-10T00:40:59.132 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 112 2023-09-10T00:40:59.132 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 113 2023-09-10T00:40:59.132 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 00:40 12182 2023-09-10T00:40:59.132 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12182_fsm 2023-09-10T00:40:59.133 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12182_vm 2023-09-10T00:40:59.133 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12184 2023-09-10T00:40:59.133 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12186 2023-09-10T00:40:59.133 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12187 2023-09-10T00:40:59.133 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12187_fsm 2023-09-10T00:40:59.134 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12187_vm 2023-09-10T00:40:59.134 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12189 2023-09-10T00:40:59.134 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12191 2023-09-10T00:40:59.134 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12192 2023-09-10T00:40:59.134 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12192_fsm 2023-09-10T00:40:59.135 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12192_vm 2023-09-10T00:40:59.135 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12194 2023-09-10T00:40:59.135 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12196 2023-09-10T00:40:59.135 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12197 2023-09-10T00:40:59.135 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12197_fsm 2023-09-10T00:40:59.136 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12197_vm 2023-09-10T00:40:59.136 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12199 2023-09-10T00:40:59.136 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12201 2023-09-10T00:40:59.136 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12202 2023-09-10T00:40:59.136 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12202_fsm 2023-09-10T00:40:59.137 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12202_vm 2023-09-10T00:40:59.137 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12204 2023-09-10T00:40:59.137 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12206 2023-09-10T00:40:59.137 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12207 2023-09-10T00:40:59.137 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12207_fsm 2023-09-10T00:40:59.137 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12207_vm 2023-09-10T00:40:59.138 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12209 2023-09-10T00:40:59.138 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12211 2023-09-10T00:40:59.138 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12212 2023-09-10T00:40:59.138 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12214 2023-09-10T00:40:59.138 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12216 2023-09-10T00:40:59.139 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 10 00:40 1247 2023-09-10T00:40:59.139 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1247_fsm 2023-09-10T00:40:59.139 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1247_vm 2023-09-10T00:40:59.139 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 10 00:40 1249 2023-09-10T00:40:59.139 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1249_fsm 2023-09-10T00:40:59.139 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1249_vm 2023-09-10T00:40:59.140 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 10 00:40 1255 2023-09-10T00:40:59.140 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1255_fsm 2023-09-10T00:40:59.140 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1255_vm 2023-09-10T00:40:59.140 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 00:40 1259 2023-09-10T00:40:59.140 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1259_fsm 2023-09-10T00:40:59.141 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1259_vm 2023-09-10T00:40:59.141 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1417 2023-09-10T00:40:59.141 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1417_vm 2023-09-10T00:40:59.141 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1418 2023-09-10T00:40:59.141 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1418_vm 2023-09-10T00:40:59.142 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 174 2023-09-10T00:40:59.142 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 175 2023-09-10T00:40:59.142 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2187 2023-09-10T00:40:59.142 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2224 2023-09-10T00:40:59.142 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2224_vm 2023-09-10T00:40:59.143 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2328 2023-09-10T00:40:59.143 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2328_vm 2023-09-10T00:40:59.143 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2336 2023-09-10T00:40:59.143 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2336_vm 2023-09-10T00:40:59.143 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2337 2023-09-10T00:40:59.143 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2600 2023-09-10T00:40:59.144 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2600_fsm 2023-09-10T00:40:59.144 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2600_vm 2023-09-10T00:40:59.144 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2601 2023-09-10T00:40:59.144 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2601_fsm 2023-09-10T00:40:59.144 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2601_vm 2023-09-10T00:40:59.145 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2602 2023-09-10T00:40:59.145 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2602_fsm 2023-09-10T00:40:59.145 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2602_vm 2023-09-10T00:40:59.145 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2603 2023-09-10T00:40:59.145 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2603_fsm 2023-09-10T00:40:59.145 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2603_vm 2023-09-10T00:40:59.146 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2604 2023-09-10T00:40:59.146 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2604_vm 2023-09-10T00:40:59.146 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2605 2023-09-10T00:40:59.146 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2605_fsm 2023-09-10T00:40:59.146 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2605_vm 2023-09-10T00:40:59.147 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2606 2023-09-10T00:40:59.147 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2606_fsm 2023-09-10T00:40:59.147 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2606_vm 2023-09-10T00:40:59.147 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2607 2023-09-10T00:40:59.147 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2607_fsm 2023-09-10T00:40:59.148 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2607_vm 2023-09-10T00:40:59.148 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 10 00:40 2608 2023-09-10T00:40:59.148 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2608_fsm 2023-09-10T00:40:59.148 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2608_vm 2023-09-10T00:40:59.148 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 10 00:40 2609 2023-09-10T00:40:59.148 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2609_fsm 2023-09-10T00:40:59.149 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2609_vm 2023-09-10T00:40:59.149 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2610 2023-09-10T00:40:59.149 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2610_fsm 2023-09-10T00:40:59.149 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2610_vm 2023-09-10T00:40:59.149 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2611 2023-09-10T00:40:59.150 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2611_vm 2023-09-10T00:40:59.150 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2612 2023-09-10T00:40:59.150 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2612_fsm 2023-09-10T00:40:59.150 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2612_vm 2023-09-10T00:40:59.150 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2613 2023-09-10T00:40:59.151 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2613_vm 2023-09-10T00:40:59.151 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2615 2023-09-10T00:40:59.151 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2615_fsm 2023-09-10T00:40:59.151 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2615_vm 2023-09-10T00:40:59.151 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2616 2023-09-10T00:40:59.151 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2616_fsm 2023-09-10T00:40:59.152 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2616_vm 2023-09-10T00:40:59.152 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 00:40 2617 2023-09-10T00:40:59.152 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2617_fsm 2023-09-10T00:40:59.152 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2617_vm 2023-09-10T00:40:59.152 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 00:40 2618 2023-09-10T00:40:59.153 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2618_fsm 2023-09-10T00:40:59.153 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2618_vm 2023-09-10T00:40:59.153 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 10 00:40 2619 2023-09-10T00:40:59.153 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2619_fsm 2023-09-10T00:40:59.153 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2619_vm 2023-09-10T00:40:59.154 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2620 2023-09-10T00:40:59.154 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2620_vm 2023-09-10T00:40:59.154 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2650 2023-09-10T00:40:59.154 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2651 2023-09-10T00:40:59.154 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2652 2023-09-10T00:40:59.154 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2653 2023-09-10T00:40:59.155 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2654 2023-09-10T00:40:59.155 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2655 2023-09-10T00:40:59.155 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2656 2023-09-10T00:40:59.155 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2657 2023-09-10T00:40:59.155 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 00:40 2658 2023-09-10T00:40:59.156 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 00:40 2659 2023-09-10T00:40:59.156 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2660 2023-09-10T00:40:59.156 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2661 2023-09-10T00:40:59.156 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2662 2023-09-10T00:40:59.156 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2663 2023-09-10T00:40:59.157 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2664 2023-09-10T00:40:59.157 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2665 2023-09-10T00:40:59.157 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2666 2023-09-10T00:40:59.157 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2667 2023-09-10T00:40:59.157 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2668 2023-09-10T00:40:59.157 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2669 2023-09-10T00:40:59.158 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2670 2023-09-10T00:40:59.158 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Sep 10 00:40 2673 2023-09-10T00:40:59.158 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 10 00:40 2674 2023-09-10T00:40:59.158 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 10 00:40 2675 2023-09-10T00:40:59.158 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2678 2023-09-10T00:40:59.159 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2679 2023-09-10T00:40:59.159 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2680 2023-09-10T00:40:59.159 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2681 2023-09-10T00:40:59.159 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2682 2023-09-10T00:40:59.159 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2683 2023-09-10T00:40:59.160 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2684 2023-09-10T00:40:59.160 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2685 2023-09-10T00:40:59.160 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2686 2023-09-10T00:40:59.160 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2687 2023-09-10T00:40:59.161 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2688 2023-09-10T00:40:59.161 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2689 2023-09-10T00:40:59.161 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 00:40 2690 2023-09-10T00:40:59.161 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 10 00:40 2691 2023-09-10T00:40:59.161 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2692 2023-09-10T00:40:59.161 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2693 2023-09-10T00:40:59.162 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2696 2023-09-10T00:40:59.162 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2699 2023-09-10T00:40:59.162 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2701 2023-09-10T00:40:59.162 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2702 2023-09-10T00:40:59.163 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2703 2023-09-10T00:40:59.163 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2704 2023-09-10T00:40:59.163 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2753 2023-09-10T00:40:59.163 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2753_fsm 2023-09-10T00:40:59.163 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2753_vm 2023-09-10T00:40:59.164 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2754 2023-09-10T00:40:59.164 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2755 2023-09-10T00:40:59.164 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2756 2023-09-10T00:40:59.164 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2757 2023-09-10T00:40:59.164 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2830 2023-09-10T00:40:59.164 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2830_vm 2023-09-10T00:40:59.165 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2831 2023-09-10T00:40:59.165 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2832 2023-09-10T00:40:59.165 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2832_vm 2023-09-10T00:40:59.165 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2833 2023-09-10T00:40:59.165 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2834 2023-09-10T00:40:59.166 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2834_vm 2023-09-10T00:40:59.166 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2835 2023-09-10T00:40:59.166 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2836 2023-09-10T00:40:59.166 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2836_vm 2023-09-10T00:40:59.166 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2837 2023-09-10T00:40:59.167 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 10 00:40 2838 2023-09-10T00:40:59.167 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2838_fsm 2023-09-10T00:40:59.167 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2838_vm 2023-09-10T00:40:59.167 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2839 2023-09-10T00:40:59.167 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2840 2023-09-10T00:40:59.168 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2840_fsm 2023-09-10T00:40:59.168 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2840_vm 2023-09-10T00:40:59.168 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2841 2023-09-10T00:40:59.168 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2995 2023-09-10T00:40:59.168 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2995_vm 2023-09-10T00:40:59.168 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2996 2023-09-10T00:40:59.169 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3079 2023-09-10T00:40:59.169 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3079_fsm 2023-09-10T00:40:59.169 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3079_vm 2023-09-10T00:40:59.169 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3080 2023-09-10T00:40:59.169 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3081 2023-09-10T00:40:59.170 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3085 2023-09-10T00:40:59.170 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3118 2023-09-10T00:40:59.170 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3118_vm 2023-09-10T00:40:59.170 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3119 2023-09-10T00:40:59.170 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3164 2023-09-10T00:40:59.170 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3256 2023-09-10T00:40:59.171 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3256_vm 2023-09-10T00:40:59.171 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3257 2023-09-10T00:40:59.171 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3258 2023-09-10T00:40:59.171 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3350 2023-09-10T00:40:59.171 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3350_vm 2023-09-10T00:40:59.172 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3351 2023-09-10T00:40:59.172 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3379 2023-09-10T00:40:59.172 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3380 2023-09-10T00:40:59.172 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3381 2023-09-10T00:40:59.172 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3381_vm 2023-09-10T00:40:59.172 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3394 2023-09-10T00:40:59.173 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3394_fsm 2023-09-10T00:40:59.173 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3394_vm 2023-09-10T00:40:59.173 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3395 2023-09-10T00:40:59.173 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3439 2023-09-10T00:40:59.173 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3439_vm 2023-09-10T00:40:59.174 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3440 2023-09-10T00:40:59.174 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 3455 2023-09-10T00:40:59.174 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3456 2023-09-10T00:40:59.174 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3456_fsm 2023-09-10T00:40:59.174 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3456_vm 2023-09-10T00:40:59.175 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3466 2023-09-10T00:40:59.175 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3466_vm 2023-09-10T00:40:59.175 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3467 2023-09-10T00:40:59.175 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3468 2023-09-10T00:40:59.175 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3501 2023-09-10T00:40:59.176 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3501_vm 2023-09-10T00:40:59.176 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3502 2023-09-10T00:40:59.176 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3503 2023-09-10T00:40:59.176 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3534 2023-09-10T00:40:59.176 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3541 2023-09-10T00:40:59.176 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3541_fsm 2023-09-10T00:40:59.177 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3541_vm 2023-09-10T00:40:59.177 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3542 2023-09-10T00:40:59.177 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3574 2023-09-10T00:40:59.177 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3575 2023-09-10T00:40:59.177 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3576 2023-09-10T00:40:59.178 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3576_vm 2023-09-10T00:40:59.178 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3596 2023-09-10T00:40:59.178 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3596_vm 2023-09-10T00:40:59.178 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3597 2023-09-10T00:40:59.178 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3598 2023-09-10T00:40:59.179 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3598_vm 2023-09-10T00:40:59.179 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3599 2023-09-10T00:40:59.179 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3600 2023-09-10T00:40:59.179 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3600_fsm 2023-09-10T00:40:59.179 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3600_vm 2023-09-10T00:40:59.179 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3601 2023-09-10T00:40:59.180 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3601_fsm 2023-09-10T00:40:59.180 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3601_vm 2023-09-10T00:40:59.180 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3602 2023-09-10T00:40:59.180 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3602_fsm 2023-09-10T00:40:59.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3602_vm 2023-09-10T00:40:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3603 2023-09-10T00:40:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3603_fsm 2023-09-10T00:40:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3603_vm 2023-09-10T00:40:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3604 2023-09-10T00:40:59.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3605 2023-09-10T00:40:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3606 2023-09-10T00:40:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3607 2023-09-10T00:40:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3608 2023-09-10T00:40:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 3609 2023-09-10T00:40:59.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3712 2023-09-10T00:40:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3764 2023-09-10T00:40:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3764_fsm 2023-09-10T00:40:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3764_vm 2023-09-10T00:40:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3766 2023-09-10T00:40:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3767 2023-09-10T00:40:59.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3997 2023-09-10T00:40:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 5002 2023-09-10T00:40:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 548 2023-09-10T00:40:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 549 2023-09-10T00:40:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6102 2023-09-10T00:40:59.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6102_vm 2023-09-10T00:40:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6104 2023-09-10T00:40:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6104_vm 2023-09-10T00:40:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6106 2023-09-10T00:40:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6106_vm 2023-09-10T00:40:59.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6110 2023-09-10T00:40:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6111 2023-09-10T00:40:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6112 2023-09-10T00:40:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6113 2023-09-10T00:40:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6117 2023-09-10T00:40:59.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 826 2023-09-10T00:40:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 826_vm 2023-09-10T00:40:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 827 2023-09-10T00:40:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 828 2023-09-10T00:40:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 00:40 pg_filenode.map 2023-09-10T00:40:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 00:40 PG_VERSION 2023-09-10T00:40:59.396 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-09-10T00:40:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:total 7194 2023-09-10T00:40:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7365123 Sep 10 00:40 . 2023-09-10T00:40:59.396 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22095369 Sep 10 00:40 .. 2023-09-10T00:40:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 112 2023-09-10T00:40:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 113 2023-09-10T00:40:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 00:40 12182 2023-09-10T00:40:59.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12182_fsm 2023-09-10T00:40:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12182_vm 2023-09-10T00:40:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12184 2023-09-10T00:40:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12186 2023-09-10T00:40:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12187 2023-09-10T00:40:59.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12187_fsm 2023-09-10T00:40:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12187_vm 2023-09-10T00:40:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12189 2023-09-10T00:40:59.399 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12191 2023-09-10T00:40:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12192 2023-09-10T00:40:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12192_fsm 2023-09-10T00:40:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12192_vm 2023-09-10T00:40:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12194 2023-09-10T00:40:59.400 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12196 2023-09-10T00:40:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12197 2023-09-10T00:40:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12197_fsm 2023-09-10T00:40:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12197_vm 2023-09-10T00:40:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12199 2023-09-10T00:40:59.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12201 2023-09-10T00:40:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12202 2023-09-10T00:40:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12202_fsm 2023-09-10T00:40:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12202_vm 2023-09-10T00:40:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12204 2023-09-10T00:40:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12206 2023-09-10T00:40:59.402 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12207 2023-09-10T00:40:59.403 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12207_fsm 2023-09-10T00:40:59.403 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12207_vm 2023-09-10T00:40:59.403 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12209 2023-09-10T00:40:59.403 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12211 2023-09-10T00:40:59.403 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12212 2023-09-10T00:40:59.404 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12214 2023-09-10T00:40:59.404 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12216 2023-09-10T00:40:59.404 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 10 00:40 1247 2023-09-10T00:40:59.404 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1247_fsm 2023-09-10T00:40:59.404 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1247_vm 2023-09-10T00:40:59.405 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 10 00:40 1249 2023-09-10T00:40:59.405 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1249_fsm 2023-09-10T00:40:59.405 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1249_vm 2023-09-10T00:40:59.405 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 10 00:40 1255 2023-09-10T00:40:59.405 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1255_fsm 2023-09-10T00:40:59.405 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1255_vm 2023-09-10T00:40:59.406 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 00:40 1259 2023-09-10T00:40:59.406 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1259_fsm 2023-09-10T00:40:59.406 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1259_vm 2023-09-10T00:40:59.406 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1417 2023-09-10T00:40:59.406 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1417_vm 2023-09-10T00:40:59.407 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1418 2023-09-10T00:40:59.407 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1418_vm 2023-09-10T00:40:59.407 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 174 2023-09-10T00:40:59.407 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 175 2023-09-10T00:40:59.407 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2187 2023-09-10T00:40:59.408 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2224 2023-09-10T00:40:59.408 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2224_vm 2023-09-10T00:40:59.408 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2328 2023-09-10T00:40:59.408 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2328_vm 2023-09-10T00:40:59.408 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2336 2023-09-10T00:40:59.409 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2336_vm 2023-09-10T00:40:59.409 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2337 2023-09-10T00:40:59.409 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2600 2023-09-10T00:40:59.409 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2600_fsm 2023-09-10T00:40:59.409 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2600_vm 2023-09-10T00:40:59.409 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2601 2023-09-10T00:40:59.410 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2601_fsm 2023-09-10T00:40:59.410 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2601_vm 2023-09-10T00:40:59.410 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2602 2023-09-10T00:40:59.410 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2602_fsm 2023-09-10T00:40:59.410 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2602_vm 2023-09-10T00:40:59.411 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2603 2023-09-10T00:40:59.411 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2603_fsm 2023-09-10T00:40:59.411 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2603_vm 2023-09-10T00:40:59.411 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2604 2023-09-10T00:40:59.411 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2604_vm 2023-09-10T00:40:59.412 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2605 2023-09-10T00:40:59.412 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2605_fsm 2023-09-10T00:40:59.412 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2605_vm 2023-09-10T00:40:59.412 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2606 2023-09-10T00:40:59.412 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2606_fsm 2023-09-10T00:40:59.412 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2606_vm 2023-09-10T00:40:59.413 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2607 2023-09-10T00:40:59.413 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2607_fsm 2023-09-10T00:40:59.413 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2607_vm 2023-09-10T00:40:59.413 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 10 00:40 2608 2023-09-10T00:40:59.413 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2608_fsm 2023-09-10T00:40:59.414 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2608_vm 2023-09-10T00:40:59.414 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 10 00:40 2609 2023-09-10T00:40:59.414 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2609_fsm 2023-09-10T00:40:59.414 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2609_vm 2023-09-10T00:40:59.414 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2610 2023-09-10T00:40:59.415 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2610_fsm 2023-09-10T00:40:59.415 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2610_vm 2023-09-10T00:40:59.415 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2611 2023-09-10T00:40:59.415 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2611_vm 2023-09-10T00:40:59.415 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2612 2023-09-10T00:40:59.416 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2612_fsm 2023-09-10T00:40:59.416 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2612_vm 2023-09-10T00:40:59.416 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2613 2023-09-10T00:40:59.416 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2613_vm 2023-09-10T00:40:59.416 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2615 2023-09-10T00:40:59.416 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2615_fsm 2023-09-10T00:40:59.417 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2615_vm 2023-09-10T00:40:59.417 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2616 2023-09-10T00:40:59.417 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2616_fsm 2023-09-10T00:40:59.417 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2616_vm 2023-09-10T00:40:59.417 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 00:40 2617 2023-09-10T00:40:59.418 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2617_fsm 2023-09-10T00:40:59.418 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2617_vm 2023-09-10T00:40:59.418 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 00:40 2618 2023-09-10T00:40:59.418 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2618_fsm 2023-09-10T00:40:59.418 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2618_vm 2023-09-10T00:40:59.418 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 10 00:40 2619 2023-09-10T00:40:59.419 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2619_fsm 2023-09-10T00:40:59.419 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2619_vm 2023-09-10T00:40:59.419 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2620 2023-09-10T00:40:59.419 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2620_vm 2023-09-10T00:40:59.419 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2650 2023-09-10T00:40:59.420 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2651 2023-09-10T00:40:59.420 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2652 2023-09-10T00:40:59.420 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2653 2023-09-10T00:40:59.420 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2654 2023-09-10T00:40:59.420 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2655 2023-09-10T00:40:59.421 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2656 2023-09-10T00:40:59.421 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2657 2023-09-10T00:40:59.421 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 00:40 2658 2023-09-10T00:40:59.421 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 00:40 2659 2023-09-10T00:40:59.421 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2660 2023-09-10T00:40:59.421 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2661 2023-09-10T00:40:59.422 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2662 2023-09-10T00:40:59.422 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2663 2023-09-10T00:40:59.422 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2664 2023-09-10T00:40:59.422 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2665 2023-09-10T00:40:59.422 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2666 2023-09-10T00:40:59.423 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2667 2023-09-10T00:40:59.423 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2668 2023-09-10T00:40:59.423 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2669 2023-09-10T00:40:59.423 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2670 2023-09-10T00:40:59.423 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Sep 10 00:40 2673 2023-09-10T00:40:59.424 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 10 00:40 2674 2023-09-10T00:40:59.424 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 10 00:40 2675 2023-09-10T00:40:59.424 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2678 2023-09-10T00:40:59.424 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2679 2023-09-10T00:40:59.424 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2680 2023-09-10T00:40:59.424 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2681 2023-09-10T00:40:59.425 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2682 2023-09-10T00:40:59.425 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2683 2023-09-10T00:40:59.425 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2684 2023-09-10T00:40:59.425 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2685 2023-09-10T00:40:59.426 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2686 2023-09-10T00:40:59.426 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2687 2023-09-10T00:40:59.426 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2688 2023-09-10T00:40:59.426 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2689 2023-09-10T00:40:59.426 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 00:40 2690 2023-09-10T00:40:59.427 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 10 00:40 2691 2023-09-10T00:40:59.427 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2692 2023-09-10T00:40:59.427 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2693 2023-09-10T00:40:59.427 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2696 2023-09-10T00:40:59.428 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2699 2023-09-10T00:40:59.428 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2701 2023-09-10T00:40:59.428 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2702 2023-09-10T00:40:59.428 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2703 2023-09-10T00:40:59.428 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2704 2023-09-10T00:40:59.428 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2753 2023-09-10T00:40:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2753_fsm 2023-09-10T00:40:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2753_vm 2023-09-10T00:40:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2754 2023-09-10T00:40:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2755 2023-09-10T00:40:59.429 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2756 2023-09-10T00:40:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2757 2023-09-10T00:40:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2830 2023-09-10T00:40:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2830_vm 2023-09-10T00:40:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2831 2023-09-10T00:40:59.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2832 2023-09-10T00:40:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2832_vm 2023-09-10T00:40:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2833 2023-09-10T00:40:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2834 2023-09-10T00:40:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2834_vm 2023-09-10T00:40:59.431 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2835 2023-09-10T00:40:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2836 2023-09-10T00:40:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2836_vm 2023-09-10T00:40:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2837 2023-09-10T00:40:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 10 00:40 2838 2023-09-10T00:40:59.432 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2838_fsm 2023-09-10T00:40:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2838_vm 2023-09-10T00:40:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2839 2023-09-10T00:40:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2840 2023-09-10T00:40:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2840_fsm 2023-09-10T00:40:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2840_vm 2023-09-10T00:40:59.433 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2841 2023-09-10T00:40:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2995 2023-09-10T00:40:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2995_vm 2023-09-10T00:40:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2996 2023-09-10T00:40:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3079 2023-09-10T00:40:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3079_fsm 2023-09-10T00:40:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3079_vm 2023-09-10T00:40:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3080 2023-09-10T00:40:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3081 2023-09-10T00:40:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3085 2023-09-10T00:40:59.435 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3118 2023-09-10T00:40:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3118_vm 2023-09-10T00:40:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3119 2023-09-10T00:40:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3164 2023-09-10T00:40:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3256 2023-09-10T00:40:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3256_vm 2023-09-10T00:40:59.436 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3257 2023-09-10T00:40:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3258 2023-09-10T00:40:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3350 2023-09-10T00:40:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3350_vm 2023-09-10T00:40:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3351 2023-09-10T00:40:59.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3379 2023-09-10T00:40:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3380 2023-09-10T00:40:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3381 2023-09-10T00:40:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3381_vm 2023-09-10T00:40:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3394 2023-09-10T00:40:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3394_fsm 2023-09-10T00:40:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3394_vm 2023-09-10T00:40:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3395 2023-09-10T00:40:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3439 2023-09-10T00:40:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3439_vm 2023-09-10T00:40:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3440 2023-09-10T00:40:59.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 3455 2023-09-10T00:40:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3456 2023-09-10T00:40:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3456_fsm 2023-09-10T00:40:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3456_vm 2023-09-10T00:40:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3466 2023-09-10T00:40:59.440 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3466_vm 2023-09-10T00:40:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3467 2023-09-10T00:40:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3468 2023-09-10T00:40:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3501 2023-09-10T00:40:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3501_vm 2023-09-10T00:40:59.441 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3502 2023-09-10T00:40:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3503 2023-09-10T00:40:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3534 2023-09-10T00:40:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3541 2023-09-10T00:40:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3541_fsm 2023-09-10T00:40:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3541_vm 2023-09-10T00:40:59.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3542 2023-09-10T00:40:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3574 2023-09-10T00:40:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3575 2023-09-10T00:40:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3576 2023-09-10T00:40:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3576_vm 2023-09-10T00:40:59.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3596 2023-09-10T00:40:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3596_vm 2023-09-10T00:40:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3597 2023-09-10T00:40:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3598 2023-09-10T00:40:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3598_vm 2023-09-10T00:40:59.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3599 2023-09-10T00:40:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3600 2023-09-10T00:40:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3600_fsm 2023-09-10T00:40:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3600_vm 2023-09-10T00:40:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3601 2023-09-10T00:40:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3601_fsm 2023-09-10T00:40:59.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3601_vm 2023-09-10T00:40:59.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3602 2023-09-10T00:40:59.667 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3602_fsm 2023-09-10T00:40:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3602_vm 2023-09-10T00:40:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3603 2023-09-10T00:40:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3603_fsm 2023-09-10T00:40:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3603_vm 2023-09-10T00:40:59.668 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3604 2023-09-10T00:40:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3605 2023-09-10T00:40:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3606 2023-09-10T00:40:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3607 2023-09-10T00:40:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3608 2023-09-10T00:40:59.669 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 3609 2023-09-10T00:40:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3712 2023-09-10T00:40:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3764 2023-09-10T00:40:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3764_fsm 2023-09-10T00:40:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3764_vm 2023-09-10T00:40:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3766 2023-09-10T00:40:59.670 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3767 2023-09-10T00:40:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3997 2023-09-10T00:40:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 5002 2023-09-10T00:40:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 548 2023-09-10T00:40:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 549 2023-09-10T00:40:59.671 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6102 2023-09-10T00:40:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6102_vm 2023-09-10T00:40:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6104 2023-09-10T00:40:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6104_vm 2023-09-10T00:40:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6106 2023-09-10T00:40:59.672 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6106_vm 2023-09-10T00:40:59.673 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6110 2023-09-10T00:40:59.673 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6111 2023-09-10T00:40:59.674 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6112 2023-09-10T00:40:59.674 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6113 2023-09-10T00:40:59.674 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6117 2023-09-10T00:40:59.674 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 826 2023-09-10T00:40:59.674 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 826_vm 2023-09-10T00:40:59.675 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 827 2023-09-10T00:40:59.675 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 828 2023-09-10T00:40:59.675 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 00:40 pg_filenode.map 2023-09-10T00:40:59.675 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 00:40 PG_VERSION 2023-09-10T00:40:59.675 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.676 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-09-10T00:40:59.676 INFO:teuthology.orchestra.run.smithi044.stdout:total 7194 2023-09-10T00:40:59.676 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7365123 Sep 10 00:40 . 2023-09-10T00:40:59.676 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22095369 Sep 10 00:40 .. 2023-09-10T00:40:59.676 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 112 2023-09-10T00:40:59.676 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 113 2023-09-10T00:40:59.677 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 00:40 12182 2023-09-10T00:40:59.677 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12182_fsm 2023-09-10T00:40:59.677 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12182_vm 2023-09-10T00:40:59.677 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12184 2023-09-10T00:40:59.677 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12186 2023-09-10T00:40:59.678 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12187 2023-09-10T00:40:59.678 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12187_fsm 2023-09-10T00:40:59.678 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12187_vm 2023-09-10T00:40:59.678 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12189 2023-09-10T00:40:59.678 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12191 2023-09-10T00:40:59.679 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12192 2023-09-10T00:40:59.679 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12192_fsm 2023-09-10T00:40:59.679 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12192_vm 2023-09-10T00:40:59.679 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12194 2023-09-10T00:40:59.679 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12196 2023-09-10T00:40:59.679 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12197 2023-09-10T00:40:59.680 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12197_fsm 2023-09-10T00:40:59.680 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12197_vm 2023-09-10T00:40:59.680 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12199 2023-09-10T00:40:59.680 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12201 2023-09-10T00:40:59.680 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12202 2023-09-10T00:40:59.681 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12202_fsm 2023-09-10T00:40:59.681 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12202_vm 2023-09-10T00:40:59.681 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12204 2023-09-10T00:40:59.681 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12206 2023-09-10T00:40:59.681 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12207 2023-09-10T00:40:59.682 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12207_fsm 2023-09-10T00:40:59.682 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12207_vm 2023-09-10T00:40:59.682 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12209 2023-09-10T00:40:59.682 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12211 2023-09-10T00:40:59.682 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12212 2023-09-10T00:40:59.682 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12214 2023-09-10T00:40:59.683 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12216 2023-09-10T00:40:59.683 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 10 00:40 1247 2023-09-10T00:40:59.683 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1247_fsm 2023-09-10T00:40:59.683 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1247_vm 2023-09-10T00:40:59.683 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 10 00:40 1249 2023-09-10T00:40:59.684 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1249_fsm 2023-09-10T00:40:59.684 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1249_vm 2023-09-10T00:40:59.684 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 10 00:40 1255 2023-09-10T00:40:59.684 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1255_fsm 2023-09-10T00:40:59.684 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1255_vm 2023-09-10T00:40:59.684 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 00:40 1259 2023-09-10T00:40:59.685 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1259_fsm 2023-09-10T00:40:59.685 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1259_vm 2023-09-10T00:40:59.685 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1417 2023-09-10T00:40:59.685 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1417_vm 2023-09-10T00:40:59.685 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1418 2023-09-10T00:40:59.686 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1418_vm 2023-09-10T00:40:59.686 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 174 2023-09-10T00:40:59.686 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 175 2023-09-10T00:40:59.686 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2187 2023-09-10T00:40:59.686 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2224 2023-09-10T00:40:59.687 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2224_vm 2023-09-10T00:40:59.687 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2328 2023-09-10T00:40:59.687 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2328_vm 2023-09-10T00:40:59.687 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2336 2023-09-10T00:40:59.687 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2336_vm 2023-09-10T00:40:59.688 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2337 2023-09-10T00:40:59.688 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2600 2023-09-10T00:40:59.688 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2600_fsm 2023-09-10T00:40:59.688 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2600_vm 2023-09-10T00:40:59.688 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2601 2023-09-10T00:40:59.688 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2601_fsm 2023-09-10T00:40:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2601_vm 2023-09-10T00:40:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2602 2023-09-10T00:40:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2602_fsm 2023-09-10T00:40:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2602_vm 2023-09-10T00:40:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2603 2023-09-10T00:40:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2603_fsm 2023-09-10T00:40:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2603_vm 2023-09-10T00:40:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2604 2023-09-10T00:40:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2604_vm 2023-09-10T00:40:59.690 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2605 2023-09-10T00:40:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2605_fsm 2023-09-10T00:40:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2605_vm 2023-09-10T00:40:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2606 2023-09-10T00:40:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2606_fsm 2023-09-10T00:40:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2606_vm 2023-09-10T00:40:59.691 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2607 2023-09-10T00:40:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2607_fsm 2023-09-10T00:40:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2607_vm 2023-09-10T00:40:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 10 00:40 2608 2023-09-10T00:40:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2608_fsm 2023-09-10T00:40:59.692 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2608_vm 2023-09-10T00:40:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 10 00:40 2609 2023-09-10T00:40:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2609_fsm 2023-09-10T00:40:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2609_vm 2023-09-10T00:40:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2610 2023-09-10T00:40:59.693 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2610_fsm 2023-09-10T00:40:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2610_vm 2023-09-10T00:40:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2611 2023-09-10T00:40:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2611_vm 2023-09-10T00:40:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2612 2023-09-10T00:40:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2612_fsm 2023-09-10T00:40:59.694 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2612_vm 2023-09-10T00:40:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2613 2023-09-10T00:40:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2613_vm 2023-09-10T00:40:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2615 2023-09-10T00:40:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2615_fsm 2023-09-10T00:40:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2615_vm 2023-09-10T00:40:59.695 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2616 2023-09-10T00:40:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2616_fsm 2023-09-10T00:40:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2616_vm 2023-09-10T00:40:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 00:40 2617 2023-09-10T00:40:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2617_fsm 2023-09-10T00:40:59.696 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2617_vm 2023-09-10T00:40:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 00:40 2618 2023-09-10T00:40:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2618_fsm 2023-09-10T00:40:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2618_vm 2023-09-10T00:40:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 10 00:40 2619 2023-09-10T00:40:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2619_fsm 2023-09-10T00:40:59.697 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2619_vm 2023-09-10T00:40:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2620 2023-09-10T00:40:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2620_vm 2023-09-10T00:40:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2650 2023-09-10T00:40:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2651 2023-09-10T00:40:59.698 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2652 2023-09-10T00:40:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2653 2023-09-10T00:40:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2654 2023-09-10T00:40:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2655 2023-09-10T00:40:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2656 2023-09-10T00:40:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2657 2023-09-10T00:40:59.699 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 00:40 2658 2023-09-10T00:40:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 00:40 2659 2023-09-10T00:40:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2660 2023-09-10T00:40:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2661 2023-09-10T00:40:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2662 2023-09-10T00:40:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2663 2023-09-10T00:40:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2664 2023-09-10T00:40:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2665 2023-09-10T00:40:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2666 2023-09-10T00:40:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2667 2023-09-10T00:40:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2668 2023-09-10T00:40:59.701 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2669 2023-09-10T00:40:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2670 2023-09-10T00:40:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Sep 10 00:40 2673 2023-09-10T00:40:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 10 00:40 2674 2023-09-10T00:40:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 10 00:40 2675 2023-09-10T00:40:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2678 2023-09-10T00:40:59.702 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2679 2023-09-10T00:40:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2680 2023-09-10T00:40:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2681 2023-09-10T00:40:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2682 2023-09-10T00:40:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2683 2023-09-10T00:40:59.703 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2684 2023-09-10T00:40:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2685 2023-09-10T00:40:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2686 2023-09-10T00:40:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2687 2023-09-10T00:40:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2688 2023-09-10T00:40:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2689 2023-09-10T00:40:59.704 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 00:40 2690 2023-09-10T00:40:59.705 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 10 00:40 2691 2023-09-10T00:40:59.705 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2692 2023-09-10T00:40:59.705 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2693 2023-09-10T00:40:59.705 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2696 2023-09-10T00:40:59.705 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2699 2023-09-10T00:40:59.706 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2701 2023-09-10T00:40:59.706 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2702 2023-09-10T00:40:59.706 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2703 2023-09-10T00:40:59.706 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2704 2023-09-10T00:40:59.706 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2753 2023-09-10T00:40:59.707 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2753_fsm 2023-09-10T00:40:59.707 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2753_vm 2023-09-10T00:40:59.707 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2754 2023-09-10T00:40:59.707 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2755 2023-09-10T00:40:59.707 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2756 2023-09-10T00:40:59.707 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2757 2023-09-10T00:40:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2830 2023-09-10T00:40:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2830_vm 2023-09-10T00:40:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2831 2023-09-10T00:40:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2832 2023-09-10T00:40:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2832_vm 2023-09-10T00:40:59.708 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2833 2023-09-10T00:40:59.709 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2834 2023-09-10T00:40:59.709 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2834_vm 2023-09-10T00:40:59.709 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2835 2023-09-10T00:40:59.709 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2836 2023-09-10T00:40:59.709 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2836_vm 2023-09-10T00:40:59.710 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2837 2023-09-10T00:40:59.710 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 10 00:40 2838 2023-09-10T00:40:59.710 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2838_fsm 2023-09-10T00:40:59.710 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2838_vm 2023-09-10T00:40:59.710 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2839 2023-09-10T00:40:59.710 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2840 2023-09-10T00:40:59.711 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2840_fsm 2023-09-10T00:40:59.711 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2840_vm 2023-09-10T00:40:59.711 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2841 2023-09-10T00:40:59.711 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2995 2023-09-10T00:40:59.711 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2995_vm 2023-09-10T00:40:59.712 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2996 2023-09-10T00:40:59.712 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3079 2023-09-10T00:40:59.712 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3079_fsm 2023-09-10T00:40:59.712 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3079_vm 2023-09-10T00:40:59.712 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3080 2023-09-10T00:40:59.712 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3081 2023-09-10T00:40:59.713 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3085 2023-09-10T00:40:59.713 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3118 2023-09-10T00:40:59.713 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3118_vm 2023-09-10T00:40:59.713 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3119 2023-09-10T00:40:59.713 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3164 2023-09-10T00:40:59.713 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3256 2023-09-10T00:40:59.714 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3256_vm 2023-09-10T00:40:59.714 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3257 2023-09-10T00:40:59.714 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3258 2023-09-10T00:40:59.714 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3350 2023-09-10T00:40:59.714 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3350_vm 2023-09-10T00:40:59.715 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3351 2023-09-10T00:40:59.715 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3379 2023-09-10T00:40:59.715 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3380 2023-09-10T00:40:59.715 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3381 2023-09-10T00:40:59.715 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3381_vm 2023-09-10T00:40:59.716 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3394 2023-09-10T00:40:59.716 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3394_fsm 2023-09-10T00:40:59.716 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3394_vm 2023-09-10T00:40:59.716 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3395 2023-09-10T00:40:59.716 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3439 2023-09-10T00:40:59.716 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3439_vm 2023-09-10T00:40:59.717 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3440 2023-09-10T00:40:59.717 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 3455 2023-09-10T00:40:59.717 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3456 2023-09-10T00:40:59.717 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3456_fsm 2023-09-10T00:40:59.717 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3456_vm 2023-09-10T00:40:59.718 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3466 2023-09-10T00:40:59.718 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3466_vm 2023-09-10T00:40:59.718 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3467 2023-09-10T00:40:59.718 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3468 2023-09-10T00:40:59.718 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3501 2023-09-10T00:40:59.718 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3501_vm 2023-09-10T00:40:59.719 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3502 2023-09-10T00:40:59.719 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3503 2023-09-10T00:40:59.719 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3534 2023-09-10T00:40:59.719 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3541 2023-09-10T00:40:59.719 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3541_fsm 2023-09-10T00:40:59.720 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3541_vm 2023-09-10T00:40:59.720 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3542 2023-09-10T00:40:59.720 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3574 2023-09-10T00:40:59.720 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3575 2023-09-10T00:40:59.720 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3576 2023-09-10T00:40:59.720 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3576_vm 2023-09-10T00:40:59.721 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3596 2023-09-10T00:40:59.721 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3596_vm 2023-09-10T00:40:59.721 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3597 2023-09-10T00:40:59.721 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3598 2023-09-10T00:40:59.721 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3598_vm 2023-09-10T00:40:59.722 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3599 2023-09-10T00:40:59.722 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3600 2023-09-10T00:40:59.722 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3600_fsm 2023-09-10T00:40:59.722 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3600_vm 2023-09-10T00:40:59.722 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3601 2023-09-10T00:40:59.722 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3601_fsm 2023-09-10T00:40:59.723 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3601_vm 2023-09-10T00:40:59.723 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3602 2023-09-10T00:40:59.724 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3602_fsm 2023-09-10T00:40:59.724 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3602_vm 2023-09-10T00:40:59.724 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3603 2023-09-10T00:40:59.724 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3603_fsm 2023-09-10T00:40:59.724 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3603_vm 2023-09-10T00:40:59.725 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3604 2023-09-10T00:40:59.725 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3605 2023-09-10T00:40:59.725 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3606 2023-09-10T00:40:59.725 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3607 2023-09-10T00:40:59.725 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3608 2023-09-10T00:40:59.726 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 3609 2023-09-10T00:40:59.726 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3712 2023-09-10T00:40:59.726 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3764 2023-09-10T00:40:59.726 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3764_fsm 2023-09-10T00:40:59.726 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3764_vm 2023-09-10T00:40:59.726 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3766 2023-09-10T00:40:59.727 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3767 2023-09-10T00:40:59.727 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3997 2023-09-10T00:40:59.727 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 5002 2023-09-10T00:40:59.727 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 548 2023-09-10T00:40:59.727 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 549 2023-09-10T00:40:59.728 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6102 2023-09-10T00:40:59.728 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6102_vm 2023-09-10T00:40:59.728 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6104 2023-09-10T00:40:59.728 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6104_vm 2023-09-10T00:40:59.728 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6106 2023-09-10T00:40:59.728 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6106_vm 2023-09-10T00:40:59.729 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6110 2023-09-10T00:40:59.729 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6111 2023-09-10T00:40:59.729 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6112 2023-09-10T00:40:59.729 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6113 2023-09-10T00:40:59.729 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6117 2023-09-10T00:40:59.730 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 826 2023-09-10T00:40:59.730 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 826_vm 2023-09-10T00:40:59.730 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 827 2023-09-10T00:40:59.730 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 828 2023-09-10T00:40:59.730 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 00:40 pg_filenode.map 2023-09-10T00:40:59.730 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 00:40 PG_VERSION 2023-09-10T00:40:59.731 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.731 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/global: 2023-09-10T00:40:59.731 INFO:teuthology.orchestra.run.smithi044.stdout:total 554 2023-09-10T00:40:59.731 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 565760 Sep 10 00:40 . 2023-09-10T00:40:59.731 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.732 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1136 2023-09-10T00:40:59.732 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1136_fsm 2023-09-10T00:40:59.732 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1136_vm 2023-09-10T00:40:59.732 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 1137 2023-09-10T00:40:59.732 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1213 2023-09-10T00:40:59.732 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1213_fsm 2023-09-10T00:40:59.733 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1213_vm 2023-09-10T00:40:59.733 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1214 2023-09-10T00:40:59.733 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1214_fsm 2023-09-10T00:40:59.733 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1214_vm 2023-09-10T00:40:59.733 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 1232 2023-09-10T00:40:59.734 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 1233 2023-09-10T00:40:59.734 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1260 2023-09-10T00:40:59.734 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1260_fsm 2023-09-10T00:40:59.734 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1260_vm 2023-09-10T00:40:59.734 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1261 2023-09-10T00:40:59.734 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1261_fsm 2023-09-10T00:40:59.735 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1261_vm 2023-09-10T00:40:59.735 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1262 2023-09-10T00:40:59.735 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1262_fsm 2023-09-10T00:40:59.735 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1262_vm 2023-09-10T00:40:59.735 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2396 2023-09-10T00:40:59.736 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2396_fsm 2023-09-10T00:40:59.736 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2396_vm 2023-09-10T00:40:59.736 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2397 2023-09-10T00:40:59.736 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2671 2023-09-10T00:40:59.736 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2672 2023-09-10T00:40:59.736 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2676 2023-09-10T00:40:59.737 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2677 2023-09-10T00:40:59.737 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2694 2023-09-10T00:40:59.737 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2695 2023-09-10T00:40:59.737 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2697 2023-09-10T00:40:59.737 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2698 2023-09-10T00:40:59.737 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2846 2023-09-10T00:40:59.738 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2846_vm 2023-09-10T00:40:59.738 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2847 2023-09-10T00:40:59.738 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2964 2023-09-10T00:40:59.738 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2964_vm 2023-09-10T00:40:59.738 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2965 2023-09-10T00:40:59.739 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2966 2023-09-10T00:40:59.739 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2966_vm 2023-09-10T00:40:59.739 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2967 2023-09-10T00:40:59.739 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3592 2023-09-10T00:40:59.748 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3592_vm 2023-09-10T00:40:59.748 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3593 2023-09-10T00:40:59.748 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 4060 2023-09-10T00:40:59.749 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 4060_vm 2023-09-10T00:40:59.749 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 4061 2023-09-10T00:40:59.749 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6000 2023-09-10T00:40:59.749 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6000_vm 2023-09-10T00:40:59.749 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6001 2023-09-10T00:40:59.749 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6002 2023-09-10T00:40:59.750 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6100 2023-09-10T00:40:59.750 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6100_vm 2023-09-10T00:40:59.750 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6114 2023-09-10T00:40:59.750 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6115 2023-09-10T00:40:59.750 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 pg_control 2023-09-10T00:40:59.751 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 00:40 pg_filenode.map 2023-09-10T00:40:59.751 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.751 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/log: 2023-09-10T00:40:59.751 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T00:40:59.751 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T00:40:59.752 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.752 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.752 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-09-10T00:40:59.752 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T00:40:59.752 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T00:40:59.752 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.753 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.753 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-09-10T00:40:59.753 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T00:40:59.753 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T00:40:59.753 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.754 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.754 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-09-10T00:40:59.754 INFO:teuthology.orchestra.run.smithi044.stdout:total 3 2023-09-10T00:40:59.754 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 10 00:40 . 2023-09-10T00:40:59.754 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.754 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 mappings 2023-09-10T00:40:59.755 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 10 00:40 replorigin_checkpoint 2023-09-10T00:40:59.755 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 snapshots 2023-09-10T00:40:59.755 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.755 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-09-10T00:40:59.755 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T00:40:59.755 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T00:40:59.756 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 10 00:40 .. 2023-09-10T00:40:59.756 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.756 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-09-10T00:40:59.756 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T00:40:59.756 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T00:40:59.757 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 10 00:40 .. 2023-09-10T00:40:59.757 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.757 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-09-10T00:40:59.757 INFO:teuthology.orchestra.run.smithi044.stdout:total 2 2023-09-10T00:40:59.757 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 . 2023-09-10T00:40:59.757 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.758 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 members 2023-09-10T00:40:59.758 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 offsets 2023-09-10T00:40:59.758 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.758 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-09-10T00:40:59.758 INFO:teuthology.orchestra.run.smithi044.stdout:total 9 2023-09-10T00:40:59.759 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 . 2023-09-10T00:40:59.759 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 .. 2023-09-10T00:40:59.759 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 0000 2023-09-10T00:40:59.759 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.759 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-09-10T00:40:59.759 INFO:teuthology.orchestra.run.smithi044.stdout:total 9 2023-09-10T00:40:59.760 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 . 2023-09-10T00:40:59.786 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 .. 2023-09-10T00:40:59.788 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 0000 2023-09-10T00:40:59.788 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.788 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-09-10T00:40:59.788 INFO:teuthology.orchestra.run.smithi044.stdout:total 9 2023-09-10T00:40:59.788 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 . 2023-09-10T00:40:59.789 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.789 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 0000 2023-09-10T00:40:59.789 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.789 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-09-10T00:40:59.789 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T00:40:59.790 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T00:40:59.790 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.790 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.790 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-09-10T00:40:59.790 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T00:40:59.791 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T00:40:59.791 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.791 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.791 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-09-10T00:40:59.791 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T00:40:59.791 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T00:40:59.792 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.792 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.792 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-09-10T00:40:59.792 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T00:40:59.792 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T00:40:59.793 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.793 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.793 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-09-10T00:40:59.793 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T00:40:59.794 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T00:40:59.794 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.794 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.794 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-09-10T00:40:59.795 INFO:teuthology.orchestra.run.smithi044.stdout:total 9 2023-09-10T00:40:59.795 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 . 2023-09-10T00:40:59.795 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.795 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 0000 2023-09-10T00:40:59.795 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.796 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-09-10T00:40:59.796 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T00:40:59.796 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T00:40:59.796 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.796 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.797 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-09-10T00:40:59.797 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T00:40:59.797 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T00:40:59.797 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.797 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.798 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-09-10T00:40:59.798 INFO:teuthology.orchestra.run.smithi044.stdout:total 16386 2023-09-10T00:40:59.798 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 00:40 . 2023-09-10T00:40:59.798 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.798 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 00:40 000000010000000000000001 2023-09-10T00:40:59.799 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 archive_status 2023-09-10T00:40:59.799 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.799 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-09-10T00:40:59.799 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T00:40:59.799 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T00:40:59.800 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 00:40 .. 2023-09-10T00:40:59.800 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:40:59.800 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-09-10T00:40:59.800 INFO:teuthology.orchestra.run.smithi044.stdout:total 9 2023-09-10T00:40:59.800 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 . 2023-09-10T00:40:59.800 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39508298 Sep 10 00:40 .. 2023-09-10T00:40:59.801 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 0000 2023-09-10T00:40:59.802 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2023-09-10T00:40:59.982 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 500 -i' 2023-09-10T00:40:59.992 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:41:00.074 INFO:teuthology.orchestra.run.smithi044.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2023-09-10T00:41:00.113 INFO:teuthology.orchestra.run.smithi044.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2023-09-10T00:41:00.118 INFO:teuthology.orchestra.run.smithi044.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2023-09-10T00:41:00.124 INFO:teuthology.orchestra.run.smithi044.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2023-09-10T00:41:00.174 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.169+0000 7f86da01a700 1 -- 172.21.15.44:0/1934748284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d41054e0 msgr2=0x7f86d41058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.169+0000 7f86da01a700 1 --2- 172.21.15.44:0/1934748284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d41054e0 0x7f86d41058c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009990 tx=0x7f86c4009ca0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.169+0000 7f86da01a700 1 -- 172.21.15.44:0/1934748284 shutdown_connections 2023-09-10T00:41:00.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.169+0000 7f86da01a700 1 --2- 172.21.15.44:0/1934748284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d40ffe00 0x7f86d410c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.169+0000 7f86da01a700 1 --2- 172.21.15.44:0/1934748284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d41054e0 0x7f86d41058c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.169+0000 7f86da01a700 1 --2- 172.21.15.44:0/1934748284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86d40ff440 0x7f86d40ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.169+0000 7f86da01a700 1 -- 172.21.15.44:0/1934748284 >> 172.21.15.44:0/1934748284 conn(0x7f86d40faf50 msgr2=0x7f86d40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:00.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.169+0000 7f86da01a700 1 -- 172.21.15.44:0/1934748284 shutdown_connections 2023-09-10T00:41:00.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.169+0000 7f86da01a700 1 -- 172.21.15.44:0/1934748284 wait complete. 2023-09-10T00:41:00.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.170+0000 7f86da01a700 1 Processor -- start 2023-09-10T00:41:00.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.170+0000 7f86da01a700 1 -- start start 2023-09-10T00:41:00.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.170+0000 7f86da01a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d40ff440 0x7f86d41973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.170+0000 7f86da01a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86d40ffe00 0x7f86d4197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.170+0000 7f86da01a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d41054e0 0x7f86d41a2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.170+0000 7f86da01a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d41981f0 con 0x7f86d40ff440 2023-09-10T00:41:00.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.170+0000 7f86da01a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d41a2990 con 0x7f86d41054e0 2023-09-10T00:41:00.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.170+0000 7f86da01a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d41a2ad0 con 0x7f86d40ffe00 2023-09-10T00:41:00.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.170+0000 7f86d37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d40ff440 0x7f86d41973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.170+0000 7f86d37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d40ff440 0x7f86d41973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49092/0 (socket says 172.21.15.44:49092) 2023-09-10T00:41:00.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.170+0000 7f86d37fe700 1 -- 172.21.15.44:0/1974897142 learned_addr learned my addr 172.21.15.44:0/1974897142 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:00.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.170+0000 7f86d2ffd700 1 --2- 172.21.15.44:0/1974897142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86d40ffe00 0x7f86d4197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.170+0000 7f86d3fff700 1 --2- 172.21.15.44:0/1974897142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d41054e0 0x7f86d41a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.171+0000 7f86d37fe700 1 -- 172.21.15.44:0/1974897142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86d40ffe00 msgr2=0x7f86d4197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.171+0000 7f86d37fe700 1 --2- 172.21.15.44:0/1974897142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86d40ffe00 0x7f86d4197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.171+0000 7f86d37fe700 1 -- 172.21.15.44:0/1974897142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d41054e0 msgr2=0x7f86d41a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.171+0000 7f86d37fe700 1 --2- 172.21.15.44:0/1974897142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d41054e0 0x7f86d41a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.171+0000 7f86d37fe700 1 -- 172.21.15.44:0/1974897142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c40095f0 con 0x7f86d40ff440 2023-09-10T00:41:00.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.171+0000 7f86d37fe700 1 --2- 172.21.15.44:0/1974897142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d40ff440 0x7f86d41973d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f86c4005910 tx=0x7f86c40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:00.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.171+0000 7f86d0ff9700 1 -- 172.21.15.44:0/1974897142 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c401d070 con 0x7f86d40ff440 2023-09-10T00:41:00.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.171+0000 7f86da01a700 1 -- 172.21.15.44:0/1974897142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d41a2db0 con 0x7f86d40ff440 2023-09-10T00:41:00.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.171+0000 7f86d0ff9700 1 -- 172.21.15.44:0/1974897142 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86c4021410 con 0x7f86d40ff440 2023-09-10T00:41:00.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.171+0000 7f86d0ff9700 1 -- 172.21.15.44:0/1974897142 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4017650 con 0x7f86d40ff440 2023-09-10T00:41:00.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.171+0000 7f86da01a700 1 -- 172.21.15.44:0/1974897142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d41a3280 con 0x7f86d40ff440 2023-09-10T00:41:00.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.173+0000 7f86d0ff9700 1 -- 172.21.15.44:0/1974897142 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f86c400fb00 con 0x7f86d40ff440 2023-09-10T00:41:00.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.174+0000 7f86d0ff9700 1 --2- 172.21.15.44:0/1974897142 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f86bc0a0840 0x7f86bc0a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.174+0000 7f86d2ffd700 1 --2- 172.21.15.44:0/1974897142 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f86bc0a0840 0x7f86bc0a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.174+0000 7f86d0ff9700 1 -- 172.21.15.44:0/1974897142 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f86c40cd900 con 0x7f86d40ff440 2023-09-10T00:41:00.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.174+0000 7f86d2ffd700 1 --2- 172.21.15.44:0/1974897142 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f86bc0a0840 0x7f86bc0a2d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f86d4067fe0 tx=0x7f86c800b410 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:00.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.175+0000 7f86e61e0b80 1 Processor -- start 2023-09-10T00:41:00.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.175+0000 7f86e61e0b80 1 -- start start 2023-09-10T00:41:00.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.175+0000 7f86e61e0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 0x55f302ff1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.175+0000 7f86e61e0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ff2030 0x55f302ffb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.175+0000 7f86e61e0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ffbe40 0x55f302ffe230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.175+0000 7f86e61e0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f302f4a730 con 0x55f302ffbe40 2023-09-10T00:41:00.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.175+0000 7f86e61e0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f302f48050 con 0x55f302ff2030 2023-09-10T00:41:00.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.175+0000 7f86e61e0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f302f43940 con 0x55f302ff1710 2023-09-10T00:41:00.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.176+0000 7f86d37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 0x55f302ff1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.176+0000 7f86d3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ffbe40 0x55f302ffe230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.176+0000 7f86d37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 0x55f302ff1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:55982/0 (socket says 172.21.15.44:55982) 2023-09-10T00:41:00.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.176+0000 7f86d37fe700 1 -- 172.21.15.44:0/525342490 learned_addr learned my addr 172.21.15.44:0/525342490 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:00.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.176+0000 7f86d2ffd700 1 --2- 172.21.15.44:0/525342490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ff2030 0x55f302ffb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.176+0000 7f86d37fe700 1 -- 172.21.15.44:0/525342490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ff2030 msgr2=0x55f302ffb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:00.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.176+0000 7f86d37fe700 1 --2- 172.21.15.44:0/525342490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ff2030 0x55f302ffb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.197 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.176+0000 7f86d37fe700 1 -- 172.21.15.44:0/525342490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ffbe40 msgr2=0x55f302ffe230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:00.197 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.176+0000 7f86d37fe700 1 --2- 172.21.15.44:0/525342490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ffbe40 0x55f302ffe230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.197 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.176+0000 7f86d37fe700 1 -- 172.21.15.44:0/525342490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f302d6fde0 con 0x55f302ff1710 2023-09-10T00:41:00.197 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.176+0000 7f86d37fe700 1 --2- 172.21.15.44:0/525342490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 0x55f302ff1af0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f86c40cb2b0 tx=0x7f86c409a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=616d74c02c63abae server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:00.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.176+0000 7f86b9ffb700 1 -- 172.21.15.44:0/525342490 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4085040 con 0x55f302ff1710 2023-09-10T00:41:00.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.176+0000 7f86b9ffb700 1 -- 172.21.15.44:0/525342490 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86c409a290 con 0x55f302ff1710 2023-09-10T00:41:00.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.176+0000 7f86b9ffb700 1 -- 172.21.15.44:0/525342490 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4021670 con 0x55f302ff1710 2023-09-10T00:41:00.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.177+0000 7f86e61e0b80 1 -- 172.21.15.44:0/525342490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 msgr2=0x55f302ff1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.177+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/525342490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 0x55f302ff1af0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f86c40cb2b0 tx=0x7f86c409a780 comp rx=0 tx=0).stop 2023-09-10T00:41:00.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.177+0000 7f86e61e0b80 1 -- 172.21.15.44:0/525342490 shutdown_connections 2023-09-10T00:41:00.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.177+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/525342490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ffbe40 0x55f302ffe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.177+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/525342490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 0x55f302ff1af0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.177+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/525342490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ff2030 0x55f302ffb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.177+0000 7f86e61e0b80 1 -- 172.21.15.44:0/525342490 >> 172.21.15.44:0/525342490 conn(0x55f302f66090 msgr2=0x55f302fecbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:00.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.177+0000 7f86e61e0b80 1 -- 172.21.15.44:0/525342490 shutdown_connections 2023-09-10T00:41:00.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.177+0000 7f86e61e0b80 1 -- 172.21.15.44:0/525342490 wait complete. 2023-09-10T00:41:00.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.177+0000 7f86e61e0b80 1 -- start start 2023-09-10T00:41:00.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.178+0000 7f86e61e0b80 1 Processor -- start 2023-09-10T00:41:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.178+0000 7f86e61e0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff1710 0x55f302ffc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.178+0000 7f86e61e0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ff2030 0x55f302ffc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.178+0000 7f86e61e0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ffbe40 0x55f302ffcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.178+0000 7f86e61e0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f302ee1b70 con 0x55f302ff1710 2023-09-10T00:41:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.178+0000 7f86e61e0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f302ee1870 con 0x55f302ff2030 2023-09-10T00:41:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.178+0000 7f86e61e0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f302dc5920 con 0x55f302ffbe40 2023-09-10T00:41:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.178+0000 7f86e61e0b80 10 client.0 _ll_register_callbacks cb 0x55f302e2ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.178+0000 7f86d37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff1710 0x55f302ffc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.178+0000 7f86d37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff1710 0x55f302ffc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49106/0 (socket says 172.21.15.44:49106) 2023-09-10T00:41:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.178+0000 7f86d37fe700 1 -- 172.21.15.44:0/1425039446 learned_addr learned my addr 172.21.15.44:0/1425039446 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.178+0000 7f86d2ffd700 1 --2- 172.21.15.44:0/1425039446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ff2030 0x55f302ffc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.178+0000 7f86d3fff700 1 --2- 172.21.15.44:0/1425039446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ffbe40 0x55f302ffcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.179+0000 7f86d37fe700 1 -- 172.21.15.44:0/1425039446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ffbe40 msgr2=0x55f302ffcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.179+0000 7f86d37fe700 1 --2- 172.21.15.44:0/1425039446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ffbe40 0x55f302ffcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.179+0000 7f86d37fe700 1 -- 172.21.15.44:0/1425039446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ff2030 msgr2=0x55f302ffc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.179+0000 7f86d37fe700 1 --2- 172.21.15.44:0/1425039446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ff2030 0x55f302ffc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.179+0000 7f86d37fe700 1 -- 172.21.15.44:0/1425039446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d41a3280 con 0x55f302ff1710 2023-09-10T00:41:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.179+0000 7f86d2ffd700 1 --2- 172.21.15.44:0/1425039446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ff2030 0x55f302ffc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.179+0000 7f86d37fe700 1 --2- 172.21.15.44:0/1425039446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff1710 0x55f302ffc220 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f86c40cac80 tx=0x7f86c4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead5021f61ac7e9f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.179+0000 7f86b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.179+0000 7f86b2ffd700 1 -- 172.21.15.44:0/1425039446 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c40b2ea0 con 0x55f302ff1710 2023-09-10T00:41:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.179+0000 7f86b2ffd700 1 -- 172.21.15.44:0/1425039446 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f86c409b030 con 0x55f302ff1710 2023-09-10T00:41:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.179+0000 7f86e61e0b80 1 -- 172.21.15.44:0/1425039446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f302e9b4f0 con 0x55f302ff1710 2023-09-10T00:41:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.179+0000 7f86b2ffd700 1 -- 172.21.15.44:0/1425039446 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c4085040 con 0x55f302ff1710 2023-09-10T00:41:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.179+0000 7f86e61e0b80 10 client.15930 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.179+0000 7f86e61e0b80 1 -- 172.21.15.44:0/1425039446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f302d58c60 con 0x55f302ff1710 2023-09-10T00:41:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.179+0000 7f86b2ffd700 1 -- 172.21.15.44:0/1425039446 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86c407d080 con 0x55f302ff1710 2023-09-10T00:41:00.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.180+0000 7f86b2ffd700 1 -- 172.21.15.44:0/1425039446 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f86c40b1070 con 0x55f302ff1710 2023-09-10T00:41:00.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.180+0000 7f86e61e0b80 10 client.15930 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:00.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.180+0000 7f86e61e0b80 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:00.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.180+0000 7f86e61e0b80 20 client.15930 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:00.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.180+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/1425039446 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f30300b210 0x55f30300d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.180+0000 7f86e61e0b80 4 client.15930 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:00.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.180+0000 7f86e61e0b80 1 -- 172.21.15.44:0/1425039446 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f302f588a0 con 0x55f30300b210 2023-09-10T00:41:00.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.180+0000 7f86d2ffd700 1 --2- 172.21.15.44:0/1425039446 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f30300b210 0x55f30300d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.180+0000 7f86b2ffd700 10 client.15930 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:00.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.180+0000 7f86d2ffd700 1 --2- 172.21.15.44:0/1425039446 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f30300b210 0x55f30300d630 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=950e5ed870b29885 server_cookie=62d127539796c63e in_seq=0 out_seq=0 2023-09-10T00:41:00.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.181+0000 7f86b2ffd700 1 -- 172.21.15.44:0/1425039446 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f302f588a0 con 0x55f30300b210 2023-09-10T00:41:00.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.181+0000 7f86b2ffd700 10 client.15930 handle_command_reply: tid=0 2023-09-10T00:41:00.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.181+0000 7f86e61e0b80 1 client.15930 shutdown 2023-09-10T00:41:00.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.181+0000 7f86e61e0b80 1 -- 172.21.15.44:0/1425039446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff1710 msgr2=0x55f302ffc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.181+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/1425039446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff1710 0x55f302ffc220 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f86c40cac80 tx=0x7f86c4009310 comp rx=0 tx=0).stop 2023-09-10T00:41:00.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.181+0000 7f86e61e0b80 1 -- 172.21.15.44:0/1425039446 shutdown_connections 2023-09-10T00:41:00.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.181+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/1425039446 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f30300b210 0x55f30300d630 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.181+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/1425039446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ffbe40 0x55f302ffcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.181+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/1425039446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff1710 0x55f302ffc220 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.181+0000 7f86b2ffd700 0 client.15930 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:00.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.181+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/1425039446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ff2030 0x55f302ffc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.181+0000 7f86e61e0b80 1 -- 172.21.15.44:0/1425039446 >> 172.21.15.44:0/1425039446 conn(0x55f302f66090 msgr2=0x55f302fec7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:00.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.181+0000 7f86e61e0b80 1 -- 172.21.15.44:0/1425039446 shutdown_connections 2023-09-10T00:41:00.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.181+0000 7f86e61e0b80 1 -- 172.21.15.44:0/1425039446 wait complete. 2023-09-10T00:41:00.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.181+0000 7f86e61e0b80 20 client.15930 trim_cache size 0 max 16384 2023-09-10T00:41:00.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.199+0000 7f86e61e0b80 1 Processor -- start 2023-09-10T00:41:00.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.199+0000 7f86e61e0b80 1 -- start start 2023-09-10T00:41:00.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.199+0000 7f86e61e0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 0x55f303000d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.199+0000 7f86e61e0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff2030 0x55f302ffa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.199+0000 7f86e61e0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ffbe40 0x55f303007720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.199+0000 7f86e61e0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f303001250 con 0x55f302ff2030 2023-09-10T00:41:00.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.199+0000 7f86e61e0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30300d850 con 0x55f302ffbe40 2023-09-10T00:41:00.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.199+0000 7f86e61e0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30300da10 con 0x55f302ff1710 2023-09-10T00:41:00.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.199+0000 7f86d2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff2030 0x55f302ffa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.199+0000 7f86d2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff2030 0x55f302ffa6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49116/0 (socket says 172.21.15.44:49116) 2023-09-10T00:41:00.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.199+0000 7f86d2ffd700 1 -- 172.21.15.44:0/4155410401 learned_addr learned my addr 172.21.15.44:0/4155410401 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:00.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.200+0000 7f86d37fe700 1 --2- 172.21.15.44:0/4155410401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 0x55f303000d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.200+0000 7f86d3fff700 1 --2- 172.21.15.44:0/4155410401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ffbe40 0x55f303007720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.200+0000 7f86d2ffd700 1 -- 172.21.15.44:0/4155410401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 msgr2=0x55f303000d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.200+0000 7f86d2ffd700 1 --2- 172.21.15.44:0/4155410401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 0x55f303000d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.200+0000 7f86d2ffd700 1 -- 172.21.15.44:0/4155410401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ffbe40 msgr2=0x55f303007720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.200+0000 7f86d2ffd700 1 --2- 172.21.15.44:0/4155410401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ffbe40 0x55f303007720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.200+0000 7f86d2ffd700 1 -- 172.21.15.44:0/4155410401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f302d96250 con 0x55f302ff2030 2023-09-10T00:41:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.200+0000 7f86d2ffd700 1 --2- 172.21.15.44:0/4155410401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff2030 0x55f302ffa6e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f86c800ba80 tx=0x7f86c8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94395cc63339d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.200+0000 7f86ba7fc700 1 -- 172.21.15.44:0/4155410401 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c802e070 con 0x55f302ff2030 2023-09-10T00:41:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.200+0000 7f86ba7fc700 1 -- 172.21.15.44:0/4155410401 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86c8015070 con 0x55f302ff2030 2023-09-10T00:41:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.200+0000 7f86ba7fc700 1 -- 172.21.15.44:0/4155410401 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8024040 con 0x55f302ff2030 2023-09-10T00:41:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.201+0000 7f86e61e0b80 1 -- 172.21.15.44:0/4155410401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff2030 msgr2=0x55f302ffa6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.201+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/4155410401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff2030 0x55f302ffa6e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f86c800ba80 tx=0x7f86c8013000 comp rx=0 tx=0).stop 2023-09-10T00:41:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.201+0000 7f86e61e0b80 1 -- 172.21.15.44:0/4155410401 shutdown_connections 2023-09-10T00:41:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.201+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/4155410401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ffbe40 0x55f303007720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.201+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/4155410401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 0x55f303000d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.201+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/4155410401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff2030 0x55f302ffa6e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.201+0000 7f86e61e0b80 1 -- 172.21.15.44:0/4155410401 >> 172.21.15.44:0/4155410401 conn(0x55f302f66090 msgr2=0x55f303011f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.201+0000 7f86e61e0b80 1 -- 172.21.15.44:0/4155410401 shutdown_connections 2023-09-10T00:41:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.201+0000 7f86e61e0b80 1 -- 172.21.15.44:0/4155410401 wait complete. 2023-09-10T00:41:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.201+0000 7f86e61e0b80 1 -- start start 2023-09-10T00:41:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.201+0000 7f86e61e0b80 1 Processor -- start 2023-09-10T00:41:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86e61e0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 0x55f302ffea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86e61e0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff2030 0x55f302ffef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86e61e0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ffbe40 0x55f303058080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86e61e0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3030585c0 con 0x55f302ff2030 2023-09-10T00:41:00.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86e61e0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f303058730 con 0x55f302ffbe40 2023-09-10T00:41:00.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86e61e0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3030588a0 con 0x55f302ff1710 2023-09-10T00:41:00.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86e61e0b80 10 client.0 _ll_register_callbacks cb 0x7f86cc009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:00.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86d2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff2030 0x55f302ffef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86d2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff2030 0x55f302ffef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49122/0 (socket says 172.21.15.44:49122) 2023-09-10T00:41:00.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86d2ffd700 1 -- 172.21.15.44:0/1986838221 learned_addr learned my addr 172.21.15.44:0/1986838221 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:00.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86d3fff700 1 --2- 172.21.15.44:0/1986838221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ffbe40 0x55f303058080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86d37fe700 1 --2- 172.21.15.44:0/1986838221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 0x55f302ffea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86d2ffd700 1 -- 172.21.15.44:0/1986838221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 msgr2=0x55f302ffea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:00.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86d2ffd700 1 --2- 172.21.15.44:0/1986838221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 0x55f302ffea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86d2ffd700 1 -- 172.21.15.44:0/1986838221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ffbe40 msgr2=0x55f303058080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:00.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86d2ffd700 1 --2- 172.21.15.44:0/1986838221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ffbe40 0x55f303058080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86d2ffd700 1 -- 172.21.15.44:0/1986838221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c8010e30 con 0x55f302ff2030 2023-09-10T00:41:00.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86d3fff700 1 --2- 172.21.15.44:0/1986838221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ffbe40 0x55f303058080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:00.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86d2ffd700 1 --2- 172.21.15.44:0/1986838221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff2030 0x55f302ffef50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f86c800bf20 tx=0x7f86c8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcd295a057866c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:00.227 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:00.227 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.202+0000 7f86b2ffd700 1 -- 172.21.15.44:0/1986838221 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c802e040 con 0x55f302ff2030 2023-09-10T00:41:00.227 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.203+0000 7f86b2ffd700 1 -- 172.21.15.44:0/1986838221 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f86c8017070 con 0x55f302ff2030 2023-09-10T00:41:00.227 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.203+0000 7f86e61e0b80 1 -- 172.21.15.44:0/1986838221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86c409a290 con 0x55f302ff2030 2023-09-10T00:41:00.227 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.203+0000 7f86b2ffd700 1 -- 172.21.15.44:0/1986838221 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c8024040 con 0x55f302ff2030 2023-09-10T00:41:00.228 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.203+0000 7f86e61e0b80 10 client.15936 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:00.228 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.203+0000 7f86e61e0b80 1 -- 172.21.15.44:0/1986838221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f302d6e2b0 con 0x55f302ff2030 2023-09-10T00:41:00.228 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.203+0000 7f86b2ffd700 1 -- 172.21.15.44:0/1986838221 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86c409a290 con 0x55f302ff2030 2023-09-10T00:41:00.228 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.203+0000 7f86b2ffd700 1 -- 172.21.15.44:0/1986838221 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f86c801f940 con 0x55f302ff2030 2023-09-10T00:41:00.228 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.203+0000 7f86e61e0b80 10 client.15936 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:00.228 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.203+0000 7f86e61e0b80 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:00.229 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.203+0000 7f86e61e0b80 20 client.15936 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:00.229 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.203+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/1986838221 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f3030600d0 0x55f3030624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.229 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.203+0000 7f86e61e0b80 4 client.15936 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:41:00.229 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.203+0000 7f86e61e0b80 1 -- 172.21.15.44:0/1986838221 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f302f3c920 con 0x55f3030600d0 2023-09-10T00:41:00.229 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.203+0000 7f86d37fe700 1 --2- 172.21.15.44:0/1986838221 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f3030600d0 0x55f3030624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.230 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86d37fe700 1 --2- 172.21.15.44:0/1986838221 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f3030600d0 0x55f3030624f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e44d29e87ac3d306 server_cookie=18d3edde9ffc3a95 in_seq=0 out_seq=0 2023-09-10T00:41:00.230 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86b2ffd700 10 client.15936 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:00.230 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86b2ffd700 1 -- 172.21.15.44:0/1986838221 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f302f3c920 con 0x55f3030600d0 2023-09-10T00:41:00.230 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86b2ffd700 10 client.15936 handle_command_reply: tid=0 2023-09-10T00:41:00.230 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86e61e0b80 1 client.15936 shutdown 2023-09-10T00:41:00.231 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86e61e0b80 1 -- 172.21.15.44:0/1986838221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff2030 msgr2=0x55f302ffef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.231 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/1986838221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff2030 0x55f302ffef50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f86c800bf20 tx=0x7f86c8005850 comp rx=0 tx=0).stop 2023-09-10T00:41:00.231 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86e61e0b80 1 -- 172.21.15.44:0/1986838221 shutdown_connections 2023-09-10T00:41:00.231 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/1986838221 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f3030600d0 0x55f3030624f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.231 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/1986838221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f302ffbe40 0x55f303058080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86b2ffd700 0 client.15936 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:00.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/1986838221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f302ff1710 0x55f302ffea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86e61e0b80 1 --2- 172.21.15.44:0/1986838221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f302ff2030 0x55f302ffef50 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86e61e0b80 1 -- 172.21.15.44:0/1986838221 >> 172.21.15.44:0/1986838221 conn(0x55f302f66090 msgr2=0x55f303007720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:00.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86e61e0b80 1 -- 172.21.15.44:0/1986838221 shutdown_connections 2023-09-10T00:41:00.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86e61e0b80 1 -- 172.21.15.44:0/1986838221 wait complete. 2023-09-10T00:41:00.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.204+0000 7f86e61e0b80 20 client.15936 trim_cache size 0 max 16384 2023-09-10T00:41:00.233 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:41:00.234 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2023-09-10T00:41:00.234 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2023-09-10T00:41:00.234 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:41:00.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.205+0000 7f86da01a700 1 -- 172.21.15.44:0/1974897142 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f86bc0a0840 msgr2=0x7f86bc0a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.205+0000 7f86da01a700 1 --2- 172.21.15.44:0/1974897142 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f86bc0a0840 0x7f86bc0a2d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f86d4067fe0 tx=0x7f86c800b410 comp rx=0 tx=0).stop 2023-09-10T00:41:00.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.205+0000 7f86da01a700 1 -- 172.21.15.44:0/1974897142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d40ff440 msgr2=0x7f86d41973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.205+0000 7f86da01a700 1 --2- 172.21.15.44:0/1974897142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d40ff440 0x7f86d41973d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f86c4005910 tx=0x7f86c40047b0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.205+0000 7f86da01a700 1 -- 172.21.15.44:0/1974897142 shutdown_connections 2023-09-10T00:41:00.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.205+0000 7f86da01a700 1 --2- 172.21.15.44:0/1974897142 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f86bc0a0840 0x7f86bc0a2d00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.205+0000 7f86da01a700 1 --2- 172.21.15.44:0/1974897142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d41054e0 0x7f86d41a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.205+0000 7f86da01a700 1 --2- 172.21.15.44:0/1974897142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d40ff440 0x7f86d41973d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.205+0000 7f86da01a700 1 --2- 172.21.15.44:0/1974897142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86d40ffe00 0x7f86d4197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.205+0000 7f86da01a700 1 -- 172.21.15.44:0/1974897142 >> 172.21.15.44:0/1974897142 conn(0x7f86d40faf50 msgr2=0x7f86d40fb820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:00.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.205+0000 7f86da01a700 1 -- 172.21.15.44:0/1974897142 shutdown_connections 2023-09-10T00:41:00.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.205+0000 7f86da01a700 1 -- 172.21.15.44:0/1974897142 wait complete. 2023-09-10T00:41:00.238 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-10T00:41:00.238 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:41:00.363 INFO:teuthology.orchestra.run.smithi044.stderr:creating tables... 2023-09-10T00:41:00.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.364+0000 7efefccda700 1 -- 172.21.15.44:0/1404630546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efef8106640 msgr2=0x7efef8106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.364+0000 7efefccda700 1 --2- 172.21.15.44:0/1404630546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efef8106640 0x7efef8106a20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efee8009920 tx=0x7efee8009c30 comp rx=0 tx=0).stop 2023-09-10T00:41:00.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.365+0000 7efefccda700 1 -- 172.21.15.44:0/1404630546 shutdown_connections 2023-09-10T00:41:00.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.365+0000 7efefccda700 1 --2- 172.21.15.44:0/1404630546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef8100f40 0x7efef810d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.365+0000 7efefccda700 1 --2- 172.21.15.44:0/1404630546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efef8106640 0x7efef8106a20 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.365+0000 7efefccda700 1 --2- 172.21.15.44:0/1404630546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efef8100580 0x7efef8100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.365+0000 7efefccda700 1 -- 172.21.15.44:0/1404630546 >> 172.21.15.44:0/1404630546 conn(0x7efef80fc070 msgr2=0x7efef80fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:00.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.365+0000 7efefccda700 1 -- 172.21.15.44:0/1404630546 shutdown_connections 2023-09-10T00:41:00.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.365+0000 7efefccda700 1 -- 172.21.15.44:0/1404630546 wait complete. 2023-09-10T00:41:00.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efefccda700 1 Processor -- start 2023-09-10T00:41:00.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efefccda700 1 -- start start 2023-09-10T00:41:00.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efefccda700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef8100580 0x7efef81965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efefccda700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efef8100f40 0x7efef8196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efefccda700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efef8106640 0x7efef819af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efefccda700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef819b5a0 con 0x7efef8106640 2023-09-10T00:41:00.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efefccda700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef819b710 con 0x7efef8100f40 2023-09-10T00:41:00.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efefccda700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef819b8d0 con 0x7efef8100580 2023-09-10T00:41:00.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efef5d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efef8100f40 0x7efef8196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efef6d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efef8106640 0x7efef819af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efef659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef8100580 0x7efef81965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efef5d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efef8100f40 0x7efef8196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:50876/0 (socket says 172.21.15.44:50876) 2023-09-10T00:41:00.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efef5d9b700 1 -- 172.21.15.44:0/1961362737 learned_addr learned my addr 172.21.15.44:0/1961362737 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:00.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efef659c700 1 -- 172.21.15.44:0/1961362737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efef8100f40 msgr2=0x7efef8196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efef659c700 1 --2- 172.21.15.44:0/1961362737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efef8100f40 0x7efef8196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efef659c700 1 -- 172.21.15.44:0/1961362737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efef8106640 msgr2=0x7efef819af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efef659c700 1 --2- 172.21.15.44:0/1961362737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efef8106640 0x7efef819af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.366+0000 7efef659c700 1 -- 172.21.15.44:0/1961362737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec0095f0 con 0x7efef8100580 2023-09-10T00:41:00.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.367+0000 7efef6d9d700 1 --2- 172.21.15.44:0/1961362737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efef8106640 0x7efef819af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.367+0000 7efef5d9b700 1 --2- 172.21.15.44:0/1961362737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efef8100f40 0x7efef8196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.367+0000 7efef659c700 1 --2- 172.21.15.44:0/1961362737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef8100580 0x7efef81965c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efee8009230 tx=0x7efee800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.367+0000 7efee77fe700 1 -- 172.21.15.44:0/1961362737 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee801c070 con 0x7efef8100580 2023-09-10T00:41:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.367+0000 7efee77fe700 1 -- 172.21.15.44:0/1961362737 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efee800bab0 con 0x7efef8100580 2023-09-10T00:41:00.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.367+0000 7efee77fe700 1 -- 172.21.15.44:0/1961362737 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8004950 con 0x7efef8100580 2023-09-10T00:41:00.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.367+0000 7efefccda700 1 -- 172.21.15.44:0/1961362737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee8009520 con 0x7efef8100580 2023-09-10T00:41:00.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.367+0000 7efefccda700 1 -- 172.21.15.44:0/1961362737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef819bd90 con 0x7efef8100580 2023-09-10T00:41:00.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.368+0000 7efefccda700 1 -- 172.21.15.44:0/1961362737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef8067840 con 0x7efef8100580 2023-09-10T00:41:00.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.369+0000 7efee77fe700 1 -- 172.21.15.44:0/1961362737 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7efee8020070 con 0x7efef8100580 2023-09-10T00:41:00.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.370+0000 7efee77fe700 1 --2- 172.21.15.44:0/1961362737 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efee00a0880 0x7efee00a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.370+0000 7efef5d9b700 1 --2- 172.21.15.44:0/1961362737 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efee00a0880 0x7efee00a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.370+0000 7efee77fe700 1 -- 172.21.15.44:0/1961362737 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7efee80cd370 con 0x7efef8100580 2023-09-10T00:41:00.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.370+0000 7efef5d9b700 1 --2- 172.21.15.44:0/1961362737 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efee00a0880 0x7efee00a2d40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efef8197bd0 tx=0x7efeec009310 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:00.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.374+0000 7efee77fe700 1 -- 172.21.15.44:0/1961362737 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7efee80966e0 con 0x7efef8100580 2023-09-10T00:41:00.662 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.661+0000 7efefccda700 1 -- 172.21.15.44:0/1961362737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efef8197910 con 0x7efef8100580 2023-09-10T00:41:00.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.662+0000 7efee77fe700 1 -- 172.21.15.44:0/1961362737 <== mon.1 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11369 (secure 0 0 0) 0x7efee8099d00 con 0x7efef8100580 2023-09-10T00:41:00.664 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:41:00.664 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-10T00:38:39.183533+0000","modified":"2023-09-10T00:39:02.480227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24605,"mds_1":24574,"mds_2":24572,"mds_3":24515,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14751":{"gid":14751,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/3958270510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":3958270510},{"type":"v1","addr":"172.21.15.44:6839","nonce":3958270510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14763":{"gid":14763,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/1241002552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1241002552},{"type":"v1","addr":"172.21.15.44:6841","nonce":1241002552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24556":{"gid":24556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/3222652693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":3222652693},{"type":"v1","addr":"172.21.15.131:6839","nonce":3222652693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24605":{"gid":24605,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/2622966260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2622966260},{"type":"v1","addr":"172.21.15.99:6839","nonce":2622966260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-10T00:41:00.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.664+0000 7efefccda700 1 -- 172.21.15.44:0/1961362737 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efee00a0880 msgr2=0x7efee00a2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.664+0000 7efefccda700 1 --2- 172.21.15.44:0/1961362737 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efee00a0880 0x7efee00a2d40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efef8197bd0 tx=0x7efeec009310 comp rx=0 tx=0).stop 2023-09-10T00:41:00.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.664+0000 7efefccda700 1 -- 172.21.15.44:0/1961362737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef8100580 msgr2=0x7efef81965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.664+0000 7efefccda700 1 --2- 172.21.15.44:0/1961362737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef8100580 0x7efef81965c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efee8009230 tx=0x7efee800fa00 comp rx=0 tx=0).stop 2023-09-10T00:41:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.664+0000 7efefccda700 1 -- 172.21.15.44:0/1961362737 shutdown_connections 2023-09-10T00:41:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.664+0000 7efefccda700 1 --2- 172.21.15.44:0/1961362737 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efee00a0880 0x7efee00a2d40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.664+0000 7efefccda700 1 --2- 172.21.15.44:0/1961362737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efef8106640 0x7efef819af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.664+0000 7efefccda700 1 --2- 172.21.15.44:0/1961362737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef8100580 0x7efef81965c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.664+0000 7efefccda700 1 --2- 172.21.15.44:0/1961362737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efef8100f40 0x7efef8196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.664+0000 7efefccda700 1 -- 172.21.15.44:0/1961362737 >> 172.21.15.44:0/1961362737 conn(0x7efef80fc070 msgr2=0x7efef80fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:00.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.665+0000 7efefccda700 1 -- 172.21.15.44:0/1961362737 shutdown_connections 2023-09-10T00:41:00.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.665+0000 7efefccda700 1 -- 172.21.15.44:0/1961362737 wait complete. 2023-09-10T00:41:00.669 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2023-09-10T00:41:00.679 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-10T00:41:00.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:00 smithi044 ceph-mon[98676]: pgmap v13: 129 pgs: 129 active+clean; 38 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 4.2 MiB/s wr, 261 op/s 2023-09-10T00:41:00.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:00 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1961362737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:41:00.806 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:41:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:41:00.521+0000 7f33fb55f700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:41:00.806 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:41:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:41:00.521+0000 7f33fb55f700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:41:00.806 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:41:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:41:00.521+0000 7f33fb55f700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:41:00.806 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:41:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:41:00.521+0000 7f33fb55f700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:41:00.807 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 00:41:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T00:41:00.521+0000 7f33fb55f700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-10T00:41:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.828+0000 7f3ed89a5700 1 -- 172.21.15.44:0/3124148180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ed40ffe20 msgr2=0x7f3ed410c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.828+0000 7f3ed89a5700 1 --2- 172.21.15.44:0/3124148180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ed40ffe20 0x7f3ed410c330 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00b0a0 tx=0x7f3ecc00b3b0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.828+0000 7f3ed89a5700 1 -- 172.21.15.44:0/3124148180 shutdown_connections 2023-09-10T00:41:00.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.828+0000 7f3ed89a5700 1 --2- 172.21.15.44:0/3124148180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ed40ffe20 0x7f3ed410c330 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.828+0000 7f3ed89a5700 1 --2- 172.21.15.44:0/3124148180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ed4105500 0x7f3ed41058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.828+0000 7f3ed89a5700 1 --2- 172.21.15.44:0/3124148180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed40ff460 0x7f3ed40ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.828+0000 7f3ed89a5700 1 -- 172.21.15.44:0/3124148180 >> 172.21.15.44:0/3124148180 conn(0x7f3ed40faf50 msgr2=0x7f3ed40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:00.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.828+0000 7f3ed89a5700 1 -- 172.21.15.44:0/3124148180 shutdown_connections 2023-09-10T00:41:00.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.828+0000 7f3ed89a5700 1 -- 172.21.15.44:0/3124148180 wait complete. 2023-09-10T00:41:00.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.829+0000 7f3ed89a5700 1 Processor -- start 2023-09-10T00:41:00.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.829+0000 7f3ed89a5700 1 -- start start 2023-09-10T00:41:00.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.829+0000 7f3ed89a5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed40ff460 0x7f3ed41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.829+0000 7f3ed89a5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ed40ffe20 0x7f3ed4198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.829+0000 7f3ed89a5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ed4105500 0x7f3ed419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.829+0000 7f3ed89a5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed419c5c0 con 0x7f3ed4105500 2023-09-10T00:41:00.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.829+0000 7f3ed89a5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed419c730 con 0x7f3ed40ff460 2023-09-10T00:41:00.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.829+0000 7f3ed89a5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed419c8f0 con 0x7f3ed40ffe20 2023-09-10T00:41:00.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.829+0000 7f3ed2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ed4105500 0x7f3ed419bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.829+0000 7f3ed2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ed4105500 0x7f3ed419bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49168/0 (socket says 172.21.15.44:49168) 2023-09-10T00:41:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.829+0000 7f3ed2d9d700 1 -- 172.21.15.44:0/3577954849 learned_addr learned my addr 172.21.15.44:0/3577954849 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.829+0000 7f3ed1d9b700 1 --2- 172.21.15.44:0/3577954849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ed40ffe20 0x7f3ed4198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.829+0000 7f3ed259c700 1 --2- 172.21.15.44:0/3577954849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed40ff460 0x7f3ed41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.830+0000 7f3ed2d9d700 1 -- 172.21.15.44:0/3577954849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ed40ffe20 msgr2=0x7f3ed4198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.830+0000 7f3ed2d9d700 1 --2- 172.21.15.44:0/3577954849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ed40ffe20 0x7f3ed4198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.830+0000 7f3ed2d9d700 1 -- 172.21.15.44:0/3577954849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed40ff460 msgr2=0x7f3ed41986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.830+0000 7f3ed2d9d700 1 --2- 172.21.15.44:0/3577954849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed40ff460 0x7f3ed41986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.830+0000 7f3ed2d9d700 1 -- 172.21.15.44:0/3577954849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc009d00 con 0x7f3ed4105500 2023-09-10T00:41:00.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.830+0000 7f3ed2d9d700 1 --2- 172.21.15.44:0/3577954849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ed4105500 0x7f3ed419bfc0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00b830 tx=0x7f3ecc009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:00.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.830+0000 7f3ec37fe700 1 -- 172.21.15.44:0/3577954849 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc00e040 con 0x7f3ed4105500 2023-09-10T00:41:00.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.830+0000 7f3ec37fe700 1 -- 172.21.15.44:0/3577954849 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ecc012c10 con 0x7f3ed4105500 2023-09-10T00:41:00.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.830+0000 7f3ed89a5700 1 -- 172.21.15.44:0/3577954849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed419cb70 con 0x7f3ed4105500 2023-09-10T00:41:00.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.830+0000 7f3ec37fe700 1 -- 172.21.15.44:0/3577954849 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc005560 con 0x7f3ed4105500 2023-09-10T00:41:00.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.830+0000 7f3ed89a5700 1 -- 172.21.15.44:0/3577954849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed41a5530 con 0x7f3ed4105500 2023-09-10T00:41:00.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.833+0000 7f3ec37fe700 1 -- 172.21.15.44:0/3577954849 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f3ecc0056c0 con 0x7f3ed4105500 2023-09-10T00:41:00.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.834+0000 7f3ec37fe700 1 --2- 172.21.15.44:0/3577954849 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3ebc0a08c0 0x7f3ebc0a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.834+0000 7f3ec37fe700 1 -- 172.21.15.44:0/3577954849 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f3ecc0cde40 con 0x7f3ed4105500 2023-09-10T00:41:00.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.834+0000 7f3ed259c700 1 --2- 172.21.15.44:0/3577954849 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3ebc0a08c0 0x7f3ebc0a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.834+0000 7f3ed259c700 1 --2- 172.21.15.44:0/3577954849 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3ebc0a08c0 0x7f3ebc0a2d80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4005950 tx=0x7f3ec40058e0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:00.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.834+0000 7f3ee4b6bb80 1 Processor -- start 2023-09-10T00:41:00.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ee4b6bb80 1 -- start start 2023-09-10T00:41:00.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ee4b6bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce3710 0x5609d2ce3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ee4b6bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2ce4030 0x5609d2ced900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ee4b6bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2cede40 0x5609d2cf0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ee4b6bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609d2c3c730 con 0x5609d2ce4030 2023-09-10T00:41:00.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ee4b6bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609d2c3a050 con 0x5609d2cede40 2023-09-10T00:41:00.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ee4b6bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609d2c35940 con 0x5609d2ce3710 2023-09-10T00:41:00.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ed1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2ce4030 0x5609d2ced900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ed1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2ce4030 0x5609d2ced900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49174/0 (socket says 172.21.15.44:49174) 2023-09-10T00:41:00.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ed2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2cede40 0x5609d2cf0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ed1d9b700 1 -- 172.21.15.44:0/3381281653 learned_addr learned my addr 172.21.15.44:0/3381281653 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:00.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ed259c700 1 --2- 172.21.15.44:0/3381281653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce3710 0x5609d2ce3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ed1d9b700 1 -- 172.21.15.44:0/3381281653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce3710 msgr2=0x5609d2ce3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:00.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ed1d9b700 1 --2- 172.21.15.44:0/3381281653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce3710 0x5609d2ce3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ed1d9b700 1 -- 172.21.15.44:0/3381281653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2cede40 msgr2=0x5609d2cf0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:00.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ed1d9b700 1 --2- 172.21.15.44:0/3381281653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2cede40 0x5609d2cf0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ed1d9b700 1 -- 172.21.15.44:0/3381281653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609d2a61de0 con 0x5609d2ce4030 2023-09-10T00:41:00.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.835+0000 7f3ed2d9d700 1 --2- 172.21.15.44:0/3381281653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2cede40 0x5609d2cf0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:00.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.836+0000 7f3ed1d9b700 1 --2- 172.21.15.44:0/3381281653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2ce4030 0x5609d2ced900 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800bcf0 tx=0x7f3ec80103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d829ad97cffc7f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:00.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.836+0000 7f3ec0ff9700 1 -- 172.21.15.44:0/3381281653 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8010d10 con 0x5609d2ce4030 2023-09-10T00:41:00.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.836+0000 7f3ec0ff9700 1 -- 172.21.15.44:0/3381281653 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ec8015070 con 0x5609d2ce4030 2023-09-10T00:41:00.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.836+0000 7f3ec0ff9700 1 -- 172.21.15.44:0/3381281653 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8005020 con 0x5609d2ce4030 2023-09-10T00:41:00.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.837+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/3381281653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2ce4030 msgr2=0x5609d2ced900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.837+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/3381281653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2ce4030 0x5609d2ced900 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800bcf0 tx=0x7f3ec80103d0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.837+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/3381281653 shutdown_connections 2023-09-10T00:41:00.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.837+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/3381281653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2cede40 0x5609d2cf0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.837+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/3381281653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce3710 0x5609d2ce3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.837+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/3381281653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2ce4030 0x5609d2ced900 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.837+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/3381281653 >> 172.21.15.44:0/3381281653 conn(0x5609d2c58090 msgr2=0x5609d2cdebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:00.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.837+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/3381281653 shutdown_connections 2023-09-10T00:41:00.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.837+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/3381281653 wait complete. 2023-09-10T00:41:00.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.837+0000 7f3ee4b6bb80 1 -- start start 2023-09-10T00:41:00.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.837+0000 7f3ee4b6bb80 1 Processor -- start 2023-09-10T00:41:00.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ee4b6bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce3710 0x5609d2ced820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ee4b6bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce4030 0x5609d2ceaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ee4b6bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2cede40 0x5609d2ceb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ee4b6bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609d2bd3b70 con 0x5609d2cede40 2023-09-10T00:41:00.866 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ee4b6bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609d2bd3870 con 0x5609d2ce3710 2023-09-10T00:41:00.866 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ee4b6bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609d2ab7920 con 0x5609d2ce4030 2023-09-10T00:41:00.866 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ee4b6bb80 10 client.0 _ll_register_callbacks cb 0x5609d2b1da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:00.866 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ed1d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce4030 0x5609d2ceaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.867 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ed2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2cede40 0x5609d2ceb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.867 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ed1d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce4030 0x5609d2ceaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56062/0 (socket says 172.21.15.44:56062) 2023-09-10T00:41:00.867 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ed1d9b700 1 -- 172.21.15.44:0/2869057264 learned_addr learned my addr 172.21.15.44:0/2869057264 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:00.868 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ed259c700 1 --2- 172.21.15.44:0/2869057264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce3710 0x5609d2ced820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.868 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ed1d9b700 1 -- 172.21.15.44:0/2869057264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce3710 msgr2=0x5609d2ced820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:00.868 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ed1d9b700 1 --2- 172.21.15.44:0/2869057264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce3710 0x5609d2ced820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.868 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ed1d9b700 1 -- 172.21.15.44:0/2869057264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2cede40 msgr2=0x5609d2ceb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:00.868 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ed1d9b700 1 --2- 172.21.15.44:0/2869057264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2cede40 0x5609d2ceb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ed1d9b700 1 -- 172.21.15.44:0/2869057264 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed41a5530 con 0x5609d2ce4030 2023-09-10T00:41:00.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3ed1d9b700 1 --2- 172.21.15.44:0/2869057264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce4030 0x5609d2ceaa30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8000c00 tx=0x7f3ec8010b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0ac30c5aec1832 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:00.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.838+0000 7f3eb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:41:00.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.839+0000 7f3eb9ffb700 1 -- 172.21.15.44:0/2869057264 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8020070 con 0x5609d2ce4030 2023-09-10T00:41:00.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.839+0000 7f3eb9ffb700 1 -- 172.21.15.44:0/2869057264 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f3ec8017070 con 0x5609d2ce4030 2023-09-10T00:41:00.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.839+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/2869057264 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609d2b8d4f0 con 0x5609d2ce4030 2023-09-10T00:41:00.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.839+0000 7f3eb9ffb700 1 -- 172.21.15.44:0/2869057264 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ec8004040 con 0x5609d2ce4030 2023-09-10T00:41:00.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.839+0000 7f3ee4b6bb80 10 client.25570 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:00.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.839+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/2869057264 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f3ec800b920 con 0x5609d2ce4030 2023-09-10T00:41:00.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.839+0000 7f3eb9ffb700 1 -- 172.21.15.44:0/2869057264 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5609d2b8d4f0 con 0x5609d2ce4030 2023-09-10T00:41:00.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.839+0000 7f3eb9ffb700 1 -- 172.21.15.44:0/2869057264 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3ec8011440 con 0x5609d2ce4030 2023-09-10T00:41:00.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.839+0000 7f3ee4b6bb80 10 client.25570 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:00.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.839+0000 7f3ee4b6bb80 10 client.25570 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2023-09-10T00:41:00.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.839+0000 7f3ee4b6bb80 20 client.25570 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:00.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.839+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/2869057264 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x5609d2cfd7e0 0x5609d2cffc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.839+0000 7f3ee4b6bb80 4 client.25570 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:00.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.839+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/2869057264 --> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609d2c4a8a0 con 0x5609d2cfd7e0 2023-09-10T00:41:00.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.839+0000 7f3ed259c700 1 --2- 172.21.15.44:0/2869057264 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x5609d2cfd7e0 0x5609d2cffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.840+0000 7f3eb9ffb700 10 client.25570 ms_handle_connect on v2:172.21.15.131:6832/3395466517 2023-09-10T00:41:00.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.840+0000 7f3ed259c700 1 --2- 172.21.15.44:0/2869057264 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x5609d2cfd7e0 0x5609d2cffc00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a255776128cdd8be server_cookie=3cf8d2cd73ccd9be in_seq=0 out_seq=0 2023-09-10T00:41:00.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.840+0000 7f3eb9ffb700 1 -- 172.21.15.44:0/2869057264 <== mds.4 v2:172.21.15.131:6832/3395466517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5609d2c4a8a0 con 0x5609d2cfd7e0 2023-09-10T00:41:00.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.840+0000 7f3eb9ffb700 10 client.25570 handle_command_reply: tid=0 2023-09-10T00:41:00.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.840+0000 7f3ee4b6bb80 1 client.25570 shutdown 2023-09-10T00:41:00.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.841+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/2869057264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce4030 msgr2=0x5609d2ceaa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.841+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/2869057264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce4030 0x5609d2ceaa30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8000c00 tx=0x7f3ec8010b10 comp rx=0 tx=0).stop 2023-09-10T00:41:00.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.841+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/2869057264 shutdown_connections 2023-09-10T00:41:00.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.841+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/2869057264 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x5609d2cfd7e0 0x5609d2cffc00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.841+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/2869057264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2cede40 0x5609d2ceb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.841+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/2869057264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce3710 0x5609d2ced820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.841+0000 7f3eb9ffb700 0 client.25570 ms_handle_reset on v2:172.21.15.131:6832/3395466517 2023-09-10T00:41:00.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.841+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/2869057264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce4030 0x5609d2ceaa30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.841+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/2869057264 >> 172.21.15.44:0/2869057264 conn(0x5609d2c58090 msgr2=0x5609d2cde7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:00.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.841+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/2869057264 shutdown_connections 2023-09-10T00:41:00.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.841+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/2869057264 wait complete. 2023-09-10T00:41:00.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.841+0000 7f3ee4b6bb80 20 client.25570 trim_cache size 0 max 16384 2023-09-10T00:41:00.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.860+0000 7f3ee4b6bb80 1 Processor -- start 2023-09-10T00:41:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.860+0000 7f3ee4b6bb80 1 -- start start 2023-09-10T00:41:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.860+0000 7f3ee4b6bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce3710 0x5609d2ced350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.860+0000 7f3ee4b6bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce4030 0x5609d2ce5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.860+0000 7f3ee4b6bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2cede40 0x5609d2ce6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.860+0000 7f3ee4b6bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609d2ce4410 con 0x5609d2cede40 2023-09-10T00:41:00.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.860+0000 7f3ee4b6bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609d2ce8f90 con 0x5609d2ce4030 2023-09-10T00:41:00.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.860+0000 7f3ee4b6bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609d2ce4990 con 0x5609d2ce3710 2023-09-10T00:41:00.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.860+0000 7f3ed2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2cede40 0x5609d2ce6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.860+0000 7f3ed2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2cede40 0x5609d2ce6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49190/0 (socket says 172.21.15.44:49190) 2023-09-10T00:41:00.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.860+0000 7f3ed2d9d700 1 -- 172.21.15.44:0/1413381518 learned_addr learned my addr 172.21.15.44:0/1413381518 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:00.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.860+0000 7f3ed1d9b700 1 --2- 172.21.15.44:0/1413381518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce4030 0x5609d2ce5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.861+0000 7f3ed259c700 1 --2- 172.21.15.44:0/1413381518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce3710 0x5609d2ced350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.861+0000 7f3ed2d9d700 1 -- 172.21.15.44:0/1413381518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce3710 msgr2=0x5609d2ced350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:00.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.861+0000 7f3ed2d9d700 1 --2- 172.21.15.44:0/1413381518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce3710 0x5609d2ced350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.861+0000 7f3ed2d9d700 1 -- 172.21.15.44:0/1413381518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce4030 msgr2=0x5609d2ce5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:00.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.861+0000 7f3ed2d9d700 1 --2- 172.21.15.44:0/1413381518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce4030 0x5609d2ce5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.861+0000 7f3ed2d9d700 1 -- 172.21.15.44:0/1413381518 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609d2a4ac60 con 0x5609d2cede40 2023-09-10T00:41:00.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.861+0000 7f3ed1d9b700 1 --2- 172.21.15.44:0/1413381518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce4030 0x5609d2ce5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:00.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.861+0000 7f3ed2d9d700 1 --2- 172.21.15.44:0/1413381518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2cede40 0x5609d2ce6260 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0cb250 tx=0x7f3ecc0cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553517eecc999616 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:00.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.861+0000 7f3ec17fa700 1 -- 172.21.15.44:0/1413381518 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc0b3260 con 0x5609d2cede40 2023-09-10T00:41:00.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.861+0000 7f3ec17fa700 1 -- 172.21.15.44:0/1413381518 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ecc012d80 con 0x5609d2cede40 2023-09-10T00:41:00.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.861+0000 7f3ec17fa700 1 -- 172.21.15.44:0/1413381518 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc097610 con 0x5609d2cede40 2023-09-10T00:41:00.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.862+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/1413381518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2cede40 msgr2=0x5609d2ce6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.862+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/1413381518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2cede40 0x5609d2ce6260 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0cb250 tx=0x7f3ecc0cd040 comp rx=0 tx=0).stop 2023-09-10T00:41:00.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.862+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/1413381518 shutdown_connections 2023-09-10T00:41:00.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.862+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/1413381518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2cede40 0x5609d2ce6260 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.862+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/1413381518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2ce3710 0x5609d2ced350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.862+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/1413381518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce4030 0x5609d2ce5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.862+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/1413381518 >> 172.21.15.44:0/1413381518 conn(0x5609d2c58090 msgr2=0x5609d2d04560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:00.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.862+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/1413381518 shutdown_connections 2023-09-10T00:41:00.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.862+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/1413381518 wait complete. 2023-09-10T00:41:00.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.862+0000 7f3ee4b6bb80 1 -- start start 2023-09-10T00:41:00.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.862+0000 7f3ee4b6bb80 1 Processor -- start 2023-09-10T00:41:00.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ee4b6bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce3710 0x5609d2cf2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ee4b6bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2ce4030 0x5609d2cf3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ee4b6bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2cede40 0x5609d2cf40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ee4b6bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609d2cf4810 con 0x5609d2ce4030 2023-09-10T00:41:00.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ee4b6bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609d2cf4980 con 0x5609d2ce3710 2023-09-10T00:41:00.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ee4b6bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609d2cf4b40 con 0x5609d2cede40 2023-09-10T00:41:00.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ee4b6bb80 10 client.0 _ll_register_callbacks cb 0x7f3ecc0cb050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:00.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ed1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2ce4030 0x5609d2cf3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ed1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2ce4030 0x5609d2cf3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49194/0 (socket says 172.21.15.44:49194) 2023-09-10T00:41:00.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ed1d9b700 1 -- 172.21.15.44:0/1582356008 learned_addr learned my addr 172.21.15.44:0/1582356008 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:00.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ed259c700 1 --2- 172.21.15.44:0/1582356008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce3710 0x5609d2cf2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ed2d9d700 1 --2- 172.21.15.44:0/1582356008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2cede40 0x5609d2cf40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ed1d9b700 1 -- 172.21.15.44:0/1582356008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2cede40 msgr2=0x5609d2cf40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:00.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ed1d9b700 1 --2- 172.21.15.44:0/1582356008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2cede40 0x5609d2cf40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ed1d9b700 1 -- 172.21.15.44:0/1582356008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce3710 msgr2=0x5609d2cf2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:00.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ed1d9b700 1 --2- 172.21.15.44:0/1582356008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce3710 0x5609d2cf2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ed1d9b700 1 -- 172.21.15.44:0/1582356008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec800b920 con 0x5609d2ce4030 2023-09-10T00:41:00.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ed259c700 1 --2- 172.21.15.44:0/1582356008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce3710 0x5609d2cf2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:00.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3ed1d9b700 1 --2- 172.21.15.44:0/1582356008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2ce4030 0x5609d2cf3150 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x5609d2cebfe0 tx=0x7f3ec8010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d26e9e5d71287b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:00.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.863+0000 7f3eb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:00.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.864+0000 7f3eb9ffb700 1 -- 172.21.15.44:0/1582356008 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8020020 con 0x5609d2ce4030 2023-09-10T00:41:00.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.864+0000 7f3eb9ffb700 1 -- 172.21.15.44:0/1582356008 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f3ec8017070 con 0x5609d2ce4030 2023-09-10T00:41:00.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.864+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/1582356008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ec8015070 con 0x5609d2ce4030 2023-09-10T00:41:00.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.864+0000 7f3eb9ffb700 1 -- 172.21.15.44:0/1582356008 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ec8005ee0 con 0x5609d2ce4030 2023-09-10T00:41:00.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.864+0000 7f3ee4b6bb80 10 client.15966 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:00.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.864+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/1582356008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f3ed419cb70 con 0x5609d2ce4030 2023-09-10T00:41:00.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.864+0000 7f3eb9ffb700 1 -- 172.21.15.44:0/1582356008 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ec8015070 con 0x5609d2ce4030 2023-09-10T00:41:00.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.864+0000 7f3eb9ffb700 1 -- 172.21.15.44:0/1582356008 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3ec802c070 con 0x5609d2ce4030 2023-09-10T00:41:00.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.864+0000 7f3ee4b6bb80 10 client.15966 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:00.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.864+0000 7f3ee4b6bb80 10 client.15966 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2023-09-10T00:41:00.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.864+0000 7f3ee4b6bb80 20 client.15966 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:00.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.864+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/1582356008 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x5609d2d525c0 0x5609d2d549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:00.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.864+0000 7f3ee4b6bb80 4 client.15966 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2023-09-10T00:41:00.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.864+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/1582356008 --> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5609d2c2e920 con 0x5609d2d525c0 2023-09-10T00:41:00.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.864+0000 7f3ed259c700 1 --2- 172.21.15.44:0/1582356008 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x5609d2d525c0 0x5609d2d549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:00.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.865+0000 7f3eb9ffb700 10 client.15966 ms_handle_connect on v2:172.21.15.131:6832/3395466517 2023-09-10T00:41:00.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.865+0000 7f3ed259c700 1 --2- 172.21.15.44:0/1582356008 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x5609d2d525c0 0x5609d2d549e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=165aac7e5c7af0a6 server_cookie=ec3d0b3186635544 in_seq=0 out_seq=0 2023-09-10T00:41:00.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.865+0000 7f3eb9ffb700 1 -- 172.21.15.44:0/1582356008 <== mds.4 v2:172.21.15.131:6832/3395466517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5609d2c2e920 con 0x5609d2d525c0 2023-09-10T00:41:00.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.865+0000 7f3eb9ffb700 10 client.15966 handle_command_reply: tid=0 2023-09-10T00:41:00.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.865+0000 7f3ee4b6bb80 1 client.15966 shutdown 2023-09-10T00:41:00.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.865+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/1582356008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2ce4030 msgr2=0x5609d2cf3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.865+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/1582356008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2ce4030 0x5609d2cf3150 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x5609d2cebfe0 tx=0x7f3ec8010fd0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.865+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/1582356008 shutdown_connections 2023-09-10T00:41:00.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.865+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/1582356008 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x5609d2d525c0 0x5609d2d549e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.865+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/1582356008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609d2cede40 0x5609d2cf40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.865+0000 7f3eb9ffb700 0 client.15966 ms_handle_reset on v2:172.21.15.131:6832/3395466517 2023-09-10T00:41:00.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.865+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/1582356008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609d2ce3710 0x5609d2cf2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.865+0000 7f3ee4b6bb80 1 --2- 172.21.15.44:0/1582356008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609d2ce4030 0x5609d2cf3150 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.865+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/1582356008 >> 172.21.15.44:0/1582356008 conn(0x5609d2c58090 msgr2=0x5609d2ce5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:00.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.865+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/1582356008 shutdown_connections 2023-09-10T00:41:00.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.865+0000 7f3ee4b6bb80 1 -- 172.21.15.44:0/1582356008 wait complete. 2023-09-10T00:41:00.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.866+0000 7f3ee4b6bb80 20 client.15966 trim_cache size 0 max 16384 2023-09-10T00:41:00.909 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2023-09-10T00:41:00.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.866+0000 7f3ed89a5700 1 -- 172.21.15.44:0/3577954849 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3ebc0a08c0 msgr2=0x7f3ebc0a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.866+0000 7f3ed89a5700 1 --2- 172.21.15.44:0/3577954849 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3ebc0a08c0 0x7f3ebc0a2d80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4005950 tx=0x7f3ec40058e0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.866+0000 7f3ed89a5700 1 -- 172.21.15.44:0/3577954849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ed4105500 msgr2=0x7f3ed419bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:00.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.866+0000 7f3ed89a5700 1 --2- 172.21.15.44:0/3577954849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ed4105500 0x7f3ed419bfc0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00b830 tx=0x7f3ecc009c90 comp rx=0 tx=0).stop 2023-09-10T00:41:00.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.866+0000 7f3ed89a5700 1 -- 172.21.15.44:0/3577954849 shutdown_connections 2023-09-10T00:41:00.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.866+0000 7f3ed89a5700 1 --2- 172.21.15.44:0/3577954849 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3ebc0a08c0 0x7f3ebc0a2d80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.866+0000 7f3ed89a5700 1 --2- 172.21.15.44:0/3577954849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ed4105500 0x7f3ed419bfc0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.866+0000 7f3ed89a5700 1 --2- 172.21.15.44:0/3577954849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed40ff460 0x7f3ed41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.866+0000 7f3ed89a5700 1 --2- 172.21.15.44:0/3577954849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ed40ffe20 0x7f3ed4198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:00.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.866+0000 7f3ed89a5700 1 -- 172.21.15.44:0/3577954849 >> 172.21.15.44:0/3577954849 conn(0x7f3ed40faf50 msgr2=0x7f3ed410ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:00.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.866+0000 7f3ed89a5700 1 -- 172.21.15.44:0/3577954849 shutdown_connections 2023-09-10T00:41:00.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:00.866+0000 7f3ed89a5700 1 -- 172.21.15.44:0/3577954849 wait complete. 2023-09-10T00:41:00.915 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-10T00:41:00.940 INFO:teuthology.orchestra.run.smithi044.stderr:100000 of 50000000 tuples (0%) done (elapsed 0.57 s, remaining 286.52 s) 2023-09-10T00:41:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:00 smithi131 ceph-mon[103713]: pgmap v13: 129 pgs: 129 active+clean; 38 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 4.2 MiB/s wr, 261 op/s 2023-09-10T00:41:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:00 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1961362737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:41:01.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.062+0000 7f015a684700 1 -- 172.21.15.44:0/1893842959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01540ff4b0 msgr2=0x7f01540ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.062+0000 7f015a684700 1 --2- 172.21.15.44:0/1893842959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01540ff4b0 0x7f01540ff930 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f01480099c0 tx=0x7f0148009cd0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.062+0000 7f015a684700 1 -- 172.21.15.44:0/1893842959 shutdown_connections 2023-09-10T00:41:01.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.062+0000 7f015a684700 1 --2- 172.21.15.44:0/1893842959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01540ffe70 0x7f015410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.062+0000 7f015a684700 1 --2- 172.21.15.44:0/1893842959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0154105570 0x7f0154105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.062+0000 7f015a684700 1 --2- 172.21.15.44:0/1893842959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01540ff4b0 0x7f01540ff930 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.062+0000 7f015a684700 1 -- 172.21.15.44:0/1893842959 >> 172.21.15.44:0/1893842959 conn(0x7f01540fafa0 msgr2=0x7f01540fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.062+0000 7f015a684700 1 -- 172.21.15.44:0/1893842959 shutdown_connections 2023-09-10T00:41:01.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.062+0000 7f015a684700 1 -- 172.21.15.44:0/1893842959 wait complete. 2023-09-10T00:41:01.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.062+0000 7f015a684700 1 Processor -- start 2023-09-10T00:41:01.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f015a684700 1 -- start start 2023-09-10T00:41:01.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f015a684700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01540ff4b0 0x7f0154197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f015a684700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01540ffe70 0x7f0154197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f015a684700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0154105570 0x7f01541a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f015a684700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01541981b0 con 0x7f01540ffe70 2023-09-10T00:41:01.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f015a684700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01541a28f0 con 0x7f0154105570 2023-09-10T00:41:01.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f015a684700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01541a2ab0 con 0x7f01540ff4b0 2023-09-10T00:41:01.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f01537fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01540ffe70 0x7f0154197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f01537fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01540ffe70 0x7f0154197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49208/0 (socket says 172.21.15.44:49208) 2023-09-10T00:41:01.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f01537fe700 1 -- 172.21.15.44:0/1526387276 learned_addr learned my addr 172.21.15.44:0/1526387276 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f0153fff700 1 --2- 172.21.15.44:0/1526387276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01540ff4b0 0x7f0154197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f0158c21700 1 --2- 172.21.15.44:0/1526387276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0154105570 0x7f01541a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f01537fe700 1 -- 172.21.15.44:0/1526387276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01540ff4b0 msgr2=0x7f0154197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f01537fe700 1 --2- 172.21.15.44:0/1526387276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01540ff4b0 0x7f0154197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f01537fe700 1 -- 172.21.15.44:0/1526387276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0154105570 msgr2=0x7f01541a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f01537fe700 1 --2- 172.21.15.44:0/1526387276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0154105570 0x7f01541a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f01537fe700 1 -- 172.21.15.44:0/1526387276 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01480095f0 con 0x7f01540ffe70 2023-09-10T00:41:01.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.063+0000 7f0158c21700 1 --2- 172.21.15.44:0/1526387276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0154105570 0x7f01541a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:01.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.064+0000 7f01537fe700 1 --2- 172.21.15.44:0/1526387276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01540ffe70 0x7f0154197870 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0148009f50 tx=0x7f0148005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.064+0000 7f01517fa700 1 -- 172.21.15.44:0/1526387276 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014801d070 con 0x7f01540ffe70 2023-09-10T00:41:01.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.064+0000 7f015a684700 1 -- 172.21.15.44:0/1526387276 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01541a2d30 con 0x7f01540ffe70 2023-09-10T00:41:01.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.064+0000 7f01517fa700 1 -- 172.21.15.44:0/1526387276 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0148003fc0 con 0x7f01540ffe70 2023-09-10T00:41:01.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.064+0000 7f01517fa700 1 -- 172.21.15.44:0/1526387276 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148022d10 con 0x7f01540ffe70 2023-09-10T00:41:01.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.064+0000 7f015a684700 1 -- 172.21.15.44:0/1526387276 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01541a31b0 con 0x7f01540ffe70 2023-09-10T00:41:01.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.066+0000 7f01517fa700 1 -- 172.21.15.44:0/1526387276 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f0148022470 con 0x7f01540ffe70 2023-09-10T00:41:01.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.067+0000 7f01517fa700 1 --2- 172.21.15.44:0/1526387276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f013c0a0840 0x7f013c0a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.067+0000 7f0153fff700 1 --2- 172.21.15.44:0/1526387276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f013c0a0840 0x7f013c0a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.067+0000 7f01517fa700 1 -- 172.21.15.44:0/1526387276 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f01480cd860 con 0x7f01540ffe70 2023-09-10T00:41:01.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.068+0000 7f0153fff700 1 --2- 172.21.15.44:0/1526387276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f013c0a0840 0x7f013c0a2d00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0144005950 tx=0x7f014400b410 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.068+0000 7f016684ab80 1 Processor -- start 2023-09-10T00:41:01.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.068+0000 7f016684ab80 1 -- start start 2023-09-10T00:41:01.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.068+0000 7f016684ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433c7710 0x5574433c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.068+0000 7f016684ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c8030 0x5574433d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.068+0000 7f016684ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433d1e40 0x5574433d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.068+0000 7f016684ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557443320730 con 0x5574433c7710 2023-09-10T00:41:01.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.068+0000 7f016684ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744331e050 con 0x5574433d1e40 2023-09-10T00:41:01.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.068+0000 7f016684ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557443319940 con 0x5574433c8030 2023-09-10T00:41:01.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.069+0000 7f0153fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433c7710 0x5574433c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.069+0000 7f0153fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433c7710 0x5574433c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49214/0 (socket says 172.21.15.44:49214) 2023-09-10T00:41:01.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.069+0000 7f0153fff700 1 -- 172.21.15.44:0/3770488424 learned_addr learned my addr 172.21.15.44:0/3770488424 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.069+0000 7f01537fe700 1 --2- 172.21.15.44:0/3770488424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c8030 0x5574433d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.069+0000 7f0158c21700 1 --2- 172.21.15.44:0/3770488424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433d1e40 0x5574433d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.069+0000 7f0153fff700 1 -- 172.21.15.44:0/3770488424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c8030 msgr2=0x5574433d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.069+0000 7f0153fff700 1 --2- 172.21.15.44:0/3770488424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c8030 0x5574433d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.069+0000 7f0153fff700 1 -- 172.21.15.44:0/3770488424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433d1e40 msgr2=0x5574433d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.069+0000 7f0153fff700 1 --2- 172.21.15.44:0/3770488424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433d1e40 0x5574433d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.069+0000 7f0153fff700 1 -- 172.21.15.44:0/3770488424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557443145de0 con 0x5574433c7710 2023-09-10T00:41:01.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.069+0000 7f0158c21700 1 --2- 172.21.15.44:0/3770488424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433d1e40 0x5574433d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:41:01.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.069+0000 7f0153fff700 1 --2- 172.21.15.44:0/3770488424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433c7710 0x5574433c7af0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0144010e60 tx=0x7f0144013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bb32cf471e4f7b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.069+0000 7f013a7fc700 1 -- 172.21.15.44:0/3770488424 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014401e620 con 0x5574433c7710 2023-09-10T00:41:01.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.069+0000 7f013a7fc700 1 -- 172.21.15.44:0/3770488424 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0144015070 con 0x5574433c7710 2023-09-10T00:41:01.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.069+0000 7f013a7fc700 1 -- 172.21.15.44:0/3770488424 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014401c940 con 0x5574433c7710 2023-09-10T00:41:01.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.070+0000 7f016684ab80 1 -- 172.21.15.44:0/3770488424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433c7710 msgr2=0x5574433c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.070+0000 7f016684ab80 1 --2- 172.21.15.44:0/3770488424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433c7710 0x5574433c7af0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0144010e60 tx=0x7f0144013000 comp rx=0 tx=0).stop 2023-09-10T00:41:01.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.070+0000 7f016684ab80 1 -- 172.21.15.44:0/3770488424 shutdown_connections 2023-09-10T00:41:01.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.070+0000 7f016684ab80 1 --2- 172.21.15.44:0/3770488424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433d1e40 0x5574433d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.070+0000 7f016684ab80 1 --2- 172.21.15.44:0/3770488424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433c7710 0x5574433c7af0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.070+0000 7f016684ab80 1 --2- 172.21.15.44:0/3770488424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c8030 0x5574433d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.070+0000 7f016684ab80 1 -- 172.21.15.44:0/3770488424 >> 172.21.15.44:0/3770488424 conn(0x55744333c090 msgr2=0x5574433c2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.070+0000 7f016684ab80 1 -- 172.21.15.44:0/3770488424 shutdown_connections 2023-09-10T00:41:01.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.070+0000 7f016684ab80 1 -- 172.21.15.44:0/3770488424 wait complete. 2023-09-10T00:41:01.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.070+0000 7f016684ab80 1 -- start start 2023-09-10T00:41:01.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f016684ab80 1 Processor -- start 2023-09-10T00:41:01.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f016684ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433c7710 0x5574433d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f016684ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433c8030 0x5574433cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f016684ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433d1e40 0x5574433cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f016684ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574432b7b70 con 0x5574433c8030 2023-09-10T00:41:01.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f016684ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574432b7870 con 0x5574433c7710 2023-09-10T00:41:01.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f016684ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744319b920 con 0x5574433d1e40 2023-09-10T00:41:01.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f016684ab80 10 client.0 _ll_register_callbacks cb 0x557443201a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:01.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f01537fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433c8030 0x5574433cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f01537fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433c8030 0x5574433cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49222/0 (socket says 172.21.15.44:49222) 2023-09-10T00:41:01.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f0153fff700 1 --2- 172.21.15.44:0/247139339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433c7710 0x5574433d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f01537fe700 1 -- 172.21.15.44:0/247139339 learned_addr learned my addr 172.21.15.44:0/247139339 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f0158c21700 1 --2- 172.21.15.44:0/247139339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433d1e40 0x5574433cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f01537fe700 1 -- 172.21.15.44:0/247139339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433d1e40 msgr2=0x5574433cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f01537fe700 1 --2- 172.21.15.44:0/247139339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433d1e40 0x5574433cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f01537fe700 1 -- 172.21.15.44:0/247139339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433c7710 msgr2=0x5574433d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f01537fe700 1 --2- 172.21.15.44:0/247139339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433c7710 0x5574433d17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f01537fe700 1 -- 172.21.15.44:0/247139339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0144010b10 con 0x5574433c8030 2023-09-10T00:41:01.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f0153fff700 1 --2- 172.21.15.44:0/247139339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433c7710 0x5574433d17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:01.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.071+0000 7f0158c21700 1 --2- 172.21.15.44:0/247139339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433d1e40 0x5574433cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:01.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.072+0000 7f01537fe700 1 --2- 172.21.15.44:0/247139339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433c8030 0x5574433cea60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f014809a590 tx=0x7f01480cb0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4e6baac5c56728 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.072+0000 7f01337fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:01.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.072+0000 7f01337fe700 1 -- 172.21.15.44:0/247139339 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148099f80 con 0x5574433c8030 2023-09-10T00:41:01.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.072+0000 7f01337fe700 1 -- 172.21.15.44:0/247139339 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f014809a890 con 0x5574433c8030 2023-09-10T00:41:01.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.072+0000 7f016684ab80 1 -- 172.21.15.44:0/247139339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574432714f0 con 0x5574433c8030 2023-09-10T00:41:01.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.072+0000 7f01337fe700 1 -- 172.21.15.44:0/247139339 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f014809e6c0 con 0x5574433c8030 2023-09-10T00:41:01.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.072+0000 7f016684ab80 10 client.15978 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:01.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.072+0000 7f016684ab80 1 -- 172.21.15.44:0/247139339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55744312ec60 con 0x5574433c8030 2023-09-10T00:41:01.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.072+0000 7f01337fe700 1 -- 172.21.15.44:0/247139339 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5574432714f0 con 0x5574433c8030 2023-09-10T00:41:01.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.072+0000 7f01337fe700 1 -- 172.21.15.44:0/247139339 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0148060ae0 con 0x5574433c8030 2023-09-10T00:41:01.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.072+0000 7f016684ab80 10 client.15978 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:01.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.072+0000 7f016684ab80 10 client.15978 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2023-09-10T00:41:01.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.072+0000 7f016684ab80 20 client.15978 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:01.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.073+0000 7f016684ab80 1 --2- 172.21.15.44:0/247139339 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x5574433e1500 0x5574433e38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.073+0000 7f016684ab80 4 client.15978 mds_command: new command op to 24515 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:01.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.073+0000 7f016684ab80 1 -- 172.21.15.44:0/247139339 --> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55744332e8a0 con 0x5574433e1500 2023-09-10T00:41:01.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.073+0000 7f0153fff700 1 --2- 172.21.15.44:0/247139339 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x5574433e1500 0x5574433e38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.073+0000 7f0153fff700 1 --2- 172.21.15.44:0/247139339 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x5574433e1500 0x5574433e38f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=65259bd9fba14b03 server_cookie=cfa234c705e87bfe in_seq=0 out_seq=0 2023-09-10T00:41:01.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.073+0000 7f01337fe700 10 client.15978 ms_handle_connect on v2:172.21.15.99:6832/1934109590 2023-09-10T00:41:01.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.074+0000 7f01337fe700 1 -- 172.21.15.44:0/247139339 <== mds.3 v2:172.21.15.99:6832/1934109590 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55744332e8a0 con 0x5574433e1500 2023-09-10T00:41:01.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.074+0000 7f01337fe700 10 client.15978 handle_command_reply: tid=0 2023-09-10T00:41:01.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.074+0000 7f016684ab80 1 client.15978 shutdown 2023-09-10T00:41:01.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.074+0000 7f016684ab80 1 -- 172.21.15.44:0/247139339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433c8030 msgr2=0x5574433cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.074+0000 7f016684ab80 1 --2- 172.21.15.44:0/247139339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433c8030 0x5574433cea60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f014809a590 tx=0x7f01480cb0e0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.074+0000 7f016684ab80 1 -- 172.21.15.44:0/247139339 shutdown_connections 2023-09-10T00:41:01.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.074+0000 7f016684ab80 1 --2- 172.21.15.44:0/247139339 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x5574433e1500 0x5574433e38f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.074+0000 7f016684ab80 1 --2- 172.21.15.44:0/247139339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433d1e40 0x5574433cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.074+0000 7f016684ab80 1 --2- 172.21.15.44:0/247139339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433c7710 0x5574433d17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.074+0000 7f016684ab80 1 --2- 172.21.15.44:0/247139339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433c8030 0x5574433cea60 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.074+0000 7f01337fe700 0 client.15978 ms_handle_reset on v2:172.21.15.99:6832/1934109590 2023-09-10T00:41:01.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.074+0000 7f016684ab80 1 -- 172.21.15.44:0/247139339 >> 172.21.15.44:0/247139339 conn(0x55744333c090 msgr2=0x5574433c27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.074+0000 7f016684ab80 1 -- 172.21.15.44:0/247139339 shutdown_connections 2023-09-10T00:41:01.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.074+0000 7f016684ab80 1 -- 172.21.15.44:0/247139339 wait complete. 2023-09-10T00:41:01.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.074+0000 7f016684ab80 20 client.15978 trim_cache size 0 max 16384 2023-09-10T00:41:01.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.089+0000 7f016684ab80 1 Processor -- start 2023-09-10T00:41:01.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.089+0000 7f016684ab80 1 -- start start 2023-09-10T00:41:01.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.089+0000 7f016684ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c7710 0x5574433d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.089+0000 7f016684ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433c8030 0x5574433c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.089+0000 7f016684ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433d1e40 0x5574433ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.089+0000 7f016684ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574433c8410 con 0x5574433d1e40 2023-09-10T00:41:01.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.089+0000 7f016684ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574433c4240 con 0x5574433c8030 2023-09-10T00:41:01.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.089+0000 7f016684ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574433bcd70 con 0x5574433c7710 2023-09-10T00:41:01.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.089+0000 7f0158c21700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433d1e40 0x5574433ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.089+0000 7f0158c21700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433d1e40 0x5574433ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49230/0 (socket says 172.21.15.44:49230) 2023-09-10T00:41:01.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.089+0000 7f0158c21700 1 -- 172.21.15.44:0/202155384 learned_addr learned my addr 172.21.15.44:0/202155384 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.089+0000 7f0153fff700 1 --2- 172.21.15.44:0/202155384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c7710 0x5574433d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.089+0000 7f0158c21700 1 -- 172.21.15.44:0/202155384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c7710 msgr2=0x5574433d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.089+0000 7f0158c21700 1 --2- 172.21.15.44:0/202155384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c7710 0x5574433d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.089+0000 7f0158c21700 1 -- 172.21.15.44:0/202155384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433c8030 msgr2=0x5574433c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.090+0000 7f0158c21700 1 --2- 172.21.15.44:0/202155384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433c8030 0x5574433c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.090+0000 7f0158c21700 1 -- 172.21.15.44:0/202155384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55744316c250 con 0x5574433d1e40 2023-09-10T00:41:01.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.090+0000 7f0158c21700 1 --2- 172.21.15.44:0/202155384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433d1e40 0x5574433ce300 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f014c009e10 tx=0x7f014c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97c3787ce55c2d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.090+0000 7f013affd700 1 -- 172.21.15.44:0/202155384 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c00e070 con 0x5574433d1e40 2023-09-10T00:41:01.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.090+0000 7f013affd700 1 -- 172.21.15.44:0/202155384 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f014c003e90 con 0x5574433d1e40 2023-09-10T00:41:01.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.090+0000 7f013affd700 1 -- 172.21.15.44:0/202155384 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c004030 con 0x5574433d1e40 2023-09-10T00:41:01.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.091+0000 7f016684ab80 1 -- 172.21.15.44:0/202155384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433d1e40 msgr2=0x5574433ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.091+0000 7f016684ab80 1 --2- 172.21.15.44:0/202155384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433d1e40 0x5574433ce300 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f014c009e10 tx=0x7f014c009040 comp rx=0 tx=0).stop 2023-09-10T00:41:01.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.091+0000 7f016684ab80 1 -- 172.21.15.44:0/202155384 shutdown_connections 2023-09-10T00:41:01.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.091+0000 7f016684ab80 1 --2- 172.21.15.44:0/202155384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433d1e40 0x5574433ce300 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.091+0000 7f016684ab80 1 --2- 172.21.15.44:0/202155384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c7710 0x5574433d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.091+0000 7f016684ab80 1 --2- 172.21.15.44:0/202155384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433c8030 0x5574433c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.091+0000 7f016684ab80 1 -- 172.21.15.44:0/202155384 >> 172.21.15.44:0/202155384 conn(0x55744333c090 msgr2=0x5574433c1320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.091+0000 7f016684ab80 1 -- 172.21.15.44:0/202155384 shutdown_connections 2023-09-10T00:41:01.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.091+0000 7f016684ab80 1 -- 172.21.15.44:0/202155384 wait complete. 2023-09-10T00:41:01.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.091+0000 7f016684ab80 1 -- start start 2023-09-10T00:41:01.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.091+0000 7f016684ab80 1 Processor -- start 2023-09-10T00:41:01.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f016684ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c7710 0x5574433d8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f016684ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433c8030 0x5574433d6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f016684ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433d1e40 0x5574433d7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f016684ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574433d76c0 con 0x5574433d1e40 2023-09-10T00:41:01.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f016684ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574433d7830 con 0x5574433c8030 2023-09-10T00:41:01.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f016684ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574433d79f0 con 0x5574433c7710 2023-09-10T00:41:01.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f016684ab80 10 client.0 _ll_register_callbacks cb 0x557443201a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:01.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f0153fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c7710 0x5574433d8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f0153fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c7710 0x5574433d8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56116/0 (socket says 172.21.15.44:56116) 2023-09-10T00:41:01.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f0153fff700 1 -- 172.21.15.44:0/3183228234 learned_addr learned my addr 172.21.15.44:0/3183228234 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f01537fe700 1 --2- 172.21.15.44:0/3183228234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433c8030 0x5574433d6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f0158c21700 1 --2- 172.21.15.44:0/3183228234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433d1e40 0x5574433d7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f0153fff700 1 -- 172.21.15.44:0/3183228234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433c8030 msgr2=0x5574433d6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f0153fff700 1 --2- 172.21.15.44:0/3183228234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433c8030 0x5574433d6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f0153fff700 1 -- 172.21.15.44:0/3183228234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433d1e40 msgr2=0x5574433d7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f0153fff700 1 --2- 172.21.15.44:0/3183228234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433d1e40 0x5574433d7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.092+0000 7f0153fff700 1 -- 172.21.15.44:0/3183228234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014c011040 con 0x5574433c7710 2023-09-10T00:41:01.113 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.093+0000 7f0153fff700 1 --2- 172.21.15.44:0/3183228234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c7710 0x5574433d8900 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0144011ab0 tx=0x7f0144005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea7835a5ca36f4fb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.113 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.093+0000 7f01337fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:41:01.113 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.093+0000 7f01337fe700 1 -- 172.21.15.44:0/3183228234 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014402d040 con 0x5574433c7710 2023-09-10T00:41:01.113 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.093+0000 7f01337fe700 1 -- 172.21.15.44:0/3183228234 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f0144017070 con 0x5574433c7710 2023-09-10T00:41:01.113 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.093+0000 7f016684ab80 1 -- 172.21.15.44:0/3183228234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0144015070 con 0x5574433c7710 2023-09-10T00:41:01.113 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.093+0000 7f01337fe700 1 -- 172.21.15.44:0/3183228234 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0144005740 con 0x5574433c7710 2023-09-10T00:41:01.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.093+0000 7f016684ab80 10 client.25594 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:01.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.093+0000 7f016684ab80 1 -- 172.21.15.44:0/3183228234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5574431442b0 con 0x5574433c7710 2023-09-10T00:41:01.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.093+0000 7f01337fe700 1 -- 172.21.15.44:0/3183228234 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0144015070 con 0x5574433c7710 2023-09-10T00:41:01.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.093+0000 7f01337fe700 1 -- 172.21.15.44:0/3183228234 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f014403b020 con 0x5574433c7710 2023-09-10T00:41:01.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.093+0000 7f016684ab80 10 client.25594 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:01.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.093+0000 7f016684ab80 10 client.25594 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2023-09-10T00:41:01.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.093+0000 7f016684ab80 20 client.25594 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:01.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.093+0000 7f016684ab80 1 --2- 172.21.15.44:0/3183228234 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x5574434366e0 0x557443438b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.093+0000 7f016684ab80 4 client.25594 mds_command: new command op to 24515 tid=0[{"prefix": "damage ls"}] 2023-09-10T00:41:01.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.093+0000 7f016684ab80 1 -- 172.21.15.44:0/3183228234 --> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557443312920 con 0x5574434366e0 2023-09-10T00:41:01.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.094+0000 7f01537fe700 1 --2- 172.21.15.44:0/3183228234 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x5574434366e0 0x557443438b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.094+0000 7f01337fe700 10 client.25594 ms_handle_connect on v2:172.21.15.99:6832/1934109590 2023-09-10T00:41:01.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.094+0000 7f01537fe700 1 --2- 172.21.15.44:0/3183228234 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x5574434366e0 0x557443438b00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1c041973b23615ec server_cookie=5dc775672cbd4d36 in_seq=0 out_seq=0 2023-09-10T00:41:01.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.094+0000 7f01337fe700 1 -- 172.21.15.44:0/3183228234 <== mds.3 v2:172.21.15.99:6832/1934109590 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557443312920 con 0x5574434366e0 2023-09-10T00:41:01.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.094+0000 7f01337fe700 10 client.25594 handle_command_reply: tid=0 2023-09-10T00:41:01.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.094+0000 7f016684ab80 1 client.25594 shutdown 2023-09-10T00:41:01.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.094+0000 7f016684ab80 1 -- 172.21.15.44:0/3183228234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c7710 msgr2=0x5574433d8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.094+0000 7f016684ab80 1 --2- 172.21.15.44:0/3183228234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c7710 0x5574433d8900 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0144011ab0 tx=0x7f0144005250 comp rx=0 tx=0).stop 2023-09-10T00:41:01.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.094+0000 7f016684ab80 1 -- 172.21.15.44:0/3183228234 shutdown_connections 2023-09-10T00:41:01.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.094+0000 7f016684ab80 1 --2- 172.21.15.44:0/3183228234 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x5574434366e0 0x557443438b00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.094+0000 7f016684ab80 1 --2- 172.21.15.44:0/3183228234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574433d1e40 0x5574433d7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.118 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.094+0000 7f016684ab80 1 --2- 172.21.15.44:0/3183228234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574433c7710 0x5574433d8900 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.118 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.094+0000 7f01337fe700 0 client.25594 ms_handle_reset on v2:172.21.15.99:6832/1934109590 2023-09-10T00:41:01.118 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.094+0000 7f016684ab80 1 --2- 172.21.15.44:0/3183228234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574433c8030 0x5574433d6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.118 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.094+0000 7f016684ab80 1 -- 172.21.15.44:0/3183228234 >> 172.21.15.44:0/3183228234 conn(0x55744333c090 msgr2=0x5574433c9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.118 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.095+0000 7f016684ab80 1 -- 172.21.15.44:0/3183228234 shutdown_connections 2023-09-10T00:41:01.119 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.095+0000 7f016684ab80 1 -- 172.21.15.44:0/3183228234 wait complete. 2023-09-10T00:41:01.119 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.095+0000 7f016684ab80 20 client.25594 trim_cache size 0 max 16384 2023-09-10T00:41:01.119 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2023-09-10T00:41:01.121 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.095+0000 7f015a684700 1 -- 172.21.15.44:0/1526387276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f013c0a0840 msgr2=0x7f013c0a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.121 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.095+0000 7f015a684700 1 --2- 172.21.15.44:0/1526387276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f013c0a0840 0x7f013c0a2d00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0144005950 tx=0x7f014400b410 comp rx=0 tx=0).stop 2023-09-10T00:41:01.121 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.095+0000 7f015a684700 1 -- 172.21.15.44:0/1526387276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01540ffe70 msgr2=0x7f0154197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.121 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.095+0000 7f015a684700 1 --2- 172.21.15.44:0/1526387276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01540ffe70 0x7f0154197870 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0148009f50 tx=0x7f0148005740 comp rx=0 tx=0).stop 2023-09-10T00:41:01.121 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.095+0000 7f015a684700 1 -- 172.21.15.44:0/1526387276 shutdown_connections 2023-09-10T00:41:01.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.095+0000 7f015a684700 1 --2- 172.21.15.44:0/1526387276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f013c0a0840 0x7f013c0a2d00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.095+0000 7f015a684700 1 --2- 172.21.15.44:0/1526387276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0154105570 0x7f01541a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.095+0000 7f015a684700 1 --2- 172.21.15.44:0/1526387276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01540ff4b0 0x7f0154197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.095+0000 7f015a684700 1 --2- 172.21.15.44:0/1526387276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01540ffe70 0x7f0154197870 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.095+0000 7f015a684700 1 -- 172.21.15.44:0/1526387276 >> 172.21.15.44:0/1526387276 conn(0x7f01540fafa0 msgr2=0x7f01540fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.095+0000 7f015a684700 1 -- 172.21.15.44:0/1526387276 shutdown_connections 2023-09-10T00:41:01.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.095+0000 7f015a684700 1 -- 172.21.15.44:0/1526387276 wait complete. 2023-09-10T00:41:01.123 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-10T00:41:01.124 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:00 smithi099 ceph-mon[102692]: pgmap v13: 129 pgs: 129 active+clean; 38 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 4.2 MiB/s wr, 261 op/s 2023-09-10T00:41:01.124 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:00 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1961362737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:41:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.321+0000 7fc1c9c70700 1 -- 172.21.15.44:0/1787678966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1c41005a0 msgr2=0x7fc1c4100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.321+0000 7fc1c9c70700 1 --2- 172.21.15.44:0/1787678966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1c41005a0 0x7fc1c4100a20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80099c0 tx=0x7fc1b8009cd0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.321+0000 7fc1c9c70700 1 -- 172.21.15.44:0/1787678966 shutdown_connections 2023-09-10T00:41:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.321+0000 7fc1c9c70700 1 --2- 172.21.15.44:0/1787678966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c4100f60 0x7fc1c410d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.321+0000 7fc1c9c70700 1 --2- 172.21.15.44:0/1787678966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1c4106660 0x7fc1c4106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.321+0000 7fc1c9c70700 1 --2- 172.21.15.44:0/1787678966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1c41005a0 0x7fc1c4100a20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.321+0000 7fc1c9c70700 1 -- 172.21.15.44:0/1787678966 >> 172.21.15.44:0/1787678966 conn(0x7fc1c40fc070 msgr2=0x7fc1c40fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.321+0000 7fc1c9c70700 1 -- 172.21.15.44:0/1787678966 shutdown_connections 2023-09-10T00:41:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.321+0000 7fc1c9c70700 1 -- 172.21.15.44:0/1787678966 wait complete. 2023-09-10T00:41:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.322+0000 7fc1c9c70700 1 Processor -- start 2023-09-10T00:41:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.322+0000 7fc1c9c70700 1 -- start start 2023-09-10T00:41:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c9c70700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1c41005a0 0x7fc1c4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c9c70700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c4100f60 0x7fc1c4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c9c70700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1c4106660 0x7fc1c41a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c9c70700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c4073670 con 0x7fc1c41005a0 2023-09-10T00:41:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c9c70700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c41a4c40 con 0x7fc1c4100f60 2023-09-10T00:41:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c9c70700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c41a4d80 con 0x7fc1c4106660 2023-09-10T00:41:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1c41005a0 0x7fc1c4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1c41005a0 0x7fc1c4072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49248/0 (socket says 172.21.15.44:49248) 2023-09-10T00:41:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c37fe700 1 -- 172.21.15.44:0/1758569819 learned_addr learned my addr 172.21.15.44:0/1758569819 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c2ffd700 1 --2- 172.21.15.44:0/1758569819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c4100f60 0x7fc1c4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c3fff700 1 --2- 172.21.15.44:0/1758569819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1c4106660 0x7fc1c41a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c37fe700 1 -- 172.21.15.44:0/1758569819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1c4106660 msgr2=0x7fc1c41a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c37fe700 1 --2- 172.21.15.44:0/1758569819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1c4106660 0x7fc1c41a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c37fe700 1 -- 172.21.15.44:0/1758569819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c4100f60 msgr2=0x7fc1c4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c37fe700 1 --2- 172.21.15.44:0/1758569819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c4100f60 0x7fc1c4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c37fe700 1 -- 172.21.15.44:0/1758569819 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b80095f0 con 0x7fc1c41005a0 2023-09-10T00:41:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c2ffd700 1 --2- 172.21.15.44:0/1758569819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c4100f60 0x7fc1c4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c37fe700 1 --2- 172.21.15.44:0/1758569819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1c41005a0 0x7fc1c4072900 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400d950 tx=0x7fc1b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.323+0000 7fc1c0ff9700 1 -- 172.21.15.44:0/1758569819 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400f890 con 0x7fc1c41005a0 2023-09-10T00:41:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.324+0000 7fc1c0ff9700 1 -- 172.21.15.44:0/1758569819 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1b400fe80 con 0x7fc1c41005a0 2023-09-10T00:41:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.324+0000 7fc1c9c70700 1 -- 172.21.15.44:0/1758569819 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c41a4fe0 con 0x7fc1c41005a0 2023-09-10T00:41:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.324+0000 7fc1c0ff9700 1 -- 172.21.15.44:0/1758569819 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400e3b0 con 0x7fc1c41005a0 2023-09-10T00:41:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.324+0000 7fc1c9c70700 1 -- 172.21.15.44:0/1758569819 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c41a5460 con 0x7fc1c41005a0 2023-09-10T00:41:01.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.326+0000 7fc1c0ff9700 1 -- 172.21.15.44:0/1758569819 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7fc1b4010460 con 0x7fc1c41005a0 2023-09-10T00:41:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.327+0000 7fc1c0ff9700 1 --2- 172.21.15.44:0/1758569819 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1ac0a08c0 0x7fc1ac0a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.327+0000 7fc1c0ff9700 1 -- 172.21.15.44:0/1758569819 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc1b40cc210 con 0x7fc1c41005a0 2023-09-10T00:41:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.327+0000 7fc1c2ffd700 1 --2- 172.21.15.44:0/1758569819 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1ac0a08c0 0x7fc1ac0a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.327+0000 7fc1c2ffd700 1 --2- 172.21.15.44:0/1758569819 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1ac0a08c0 0x7fc1ac0a2d80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800b580 tx=0x7fc1b801a040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.327+0000 7fc1d5e36b80 1 Processor -- start 2023-09-10T00:41:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1d5e36b80 1 -- start start 2023-09-10T00:41:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1d5e36b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc770710 0x55d7fc770af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1d5e36b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc77a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1d5e36b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc77ae40 0x55d7fc77d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1d5e36b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7fc6c9730 con 0x55d7fc770710 2023-09-10T00:41:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1d5e36b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7fc6c7050 con 0x55d7fc77ae40 2023-09-10T00:41:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1d5e36b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7fc6c2940 con 0x55d7fc771030 2023-09-10T00:41:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1c37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc770710 0x55d7fc770af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1c37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc770710 0x55d7fc770af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49256/0 (socket says 172.21.15.44:49256) 2023-09-10T00:41:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1c37fe700 1 -- 172.21.15.44:0/2554842843 learned_addr learned my addr 172.21.15.44:0/2554842843 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1c3fff700 1 --2- 172.21.15.44:0/2554842843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc77ae40 0x55d7fc77d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1c2ffd700 1 --2- 172.21.15.44:0/2554842843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc77a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1c37fe700 1 -- 172.21.15.44:0/2554842843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 msgr2=0x55d7fc77a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1c37fe700 1 --2- 172.21.15.44:0/2554842843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc77a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1c37fe700 1 -- 172.21.15.44:0/2554842843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc77ae40 msgr2=0x55d7fc77d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1c37fe700 1 --2- 172.21.15.44:0/2554842843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc77ae40 0x55d7fc77d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1c37fe700 1 -- 172.21.15.44:0/2554842843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7fc4eede0 con 0x55d7fc770710 2023-09-10T00:41:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.328+0000 7fc1c37fe700 1 --2- 172.21.15.44:0/2554842843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc770710 0x55d7fc770af0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4099140 tx=0x7fc1b4098b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74473a2e0f953d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.329+0000 7fc1a9ffb700 1 -- 172.21.15.44:0/2554842843 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40991b0 con 0x55d7fc770710 2023-09-10T00:41:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.329+0000 7fc1a9ffb700 1 -- 172.21.15.44:0/2554842843 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1b409cfe0 con 0x55d7fc770710 2023-09-10T00:41:01.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.329+0000 7fc1a9ffb700 1 -- 172.21.15.44:0/2554842843 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40988e0 con 0x55d7fc770710 2023-09-10T00:41:01.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.329+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/2554842843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc770710 msgr2=0x55d7fc770af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.329+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/2554842843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc770710 0x55d7fc770af0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4099140 tx=0x7fc1b4098b90 comp rx=0 tx=0).stop 2023-09-10T00:41:01.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.329+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/2554842843 shutdown_connections 2023-09-10T00:41:01.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.329+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/2554842843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc77ae40 0x55d7fc77d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.329+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/2554842843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc770710 0x55d7fc770af0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.329+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/2554842843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc77a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.329+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/2554842843 >> 172.21.15.44:0/2554842843 conn(0x55d7fc6e5090 msgr2=0x55d7fc76bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.329+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/2554842843 shutdown_connections 2023-09-10T00:41:01.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.330+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/2554842843 wait complete. 2023-09-10T00:41:01.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.330+0000 7fc1d5e36b80 1 -- start start 2023-09-10T00:41:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.330+0000 7fc1d5e36b80 1 Processor -- start 2023-09-10T00:41:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1d5e36b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc770710 0x55d7fc77a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1d5e36b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc777b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1d5e36b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc77ae40 0x55d7fc778240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1d5e36b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7fc660b70 con 0x55d7fc770710 2023-09-10T00:41:01.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1d5e36b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7fc660870 con 0x55d7fc77ae40 2023-09-10T00:41:01.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1d5e36b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7fc544920 con 0x55d7fc771030 2023-09-10T00:41:01.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1d5e36b80 10 client.0 _ll_register_callbacks cb 0x55d7fc5aaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:01.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1c37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc770710 0x55d7fc77a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1c37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc770710 0x55d7fc77a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49262/0 (socket says 172.21.15.44:49262) 2023-09-10T00:41:01.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1c37fe700 1 -- 172.21.15.44:0/3775258466 learned_addr learned my addr 172.21.15.44:0/3775258466 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1c2ffd700 1 --2- 172.21.15.44:0/3775258466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc777b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1c3fff700 1 --2- 172.21.15.44:0/3775258466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc77ae40 0x55d7fc778240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1c37fe700 1 -- 172.21.15.44:0/3775258466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 msgr2=0x55d7fc777b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1c37fe700 1 --2- 172.21.15.44:0/3775258466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc777b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1c37fe700 1 -- 172.21.15.44:0/3775258466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc77ae40 msgr2=0x55d7fc778240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1c37fe700 1 --2- 172.21.15.44:0/3775258466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc77ae40 0x55d7fc778240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1c37fe700 1 -- 172.21.15.44:0/3775258466 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c41a5460 con 0x55d7fc770710 2023-09-10T00:41:01.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1c2ffd700 1 --2- 172.21.15.44:0/3775258466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc777b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:01.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1c3fff700 1 --2- 172.21.15.44:0/3775258466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc77ae40 0x55d7fc778240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:01.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1c37fe700 1 --2- 172.21.15.44:0/3775258466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc770710 0x55d7fc77a880 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4095660 tx=0x7fc1b40988d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808ac7082bf8efb5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:01.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.331+0000 7fc1a2ffd700 1 -- 172.21.15.44:0/3775258466 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40a1220 con 0x55d7fc770710 2023-09-10T00:41:01.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.332+0000 7fc1a2ffd700 1 -- 172.21.15.44:0/3775258466 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc1b40991b0 con 0x55d7fc770710 2023-09-10T00:41:01.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.332+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/3775258466 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7fc61a4f0 con 0x55d7fc770710 2023-09-10T00:41:01.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.332+0000 7fc1a2ffd700 1 -- 172.21.15.44:0/3775258466 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1b4099620 con 0x55d7fc770710 2023-09-10T00:41:01.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.332+0000 7fc1d5e36b80 10 client.15996 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:01.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.332+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/3775258466 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d7fc4d7c60 con 0x55d7fc770710 2023-09-10T00:41:01.351 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.332+0000 7fc1a2ffd700 1 -- 172.21.15.44:0/3775258466 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1b4080020 con 0x55d7fc770710 2023-09-10T00:41:01.351 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.332+0000 7fc1a2ffd700 1 -- 172.21.15.44:0/3775258466 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc1b40acad0 con 0x55d7fc770710 2023-09-10T00:41:01.351 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.332+0000 7fc1d5e36b80 10 client.15996 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:01.351 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.332+0000 7fc1d5e36b80 10 client.15996 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-10T00:41:01.351 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.332+0000 7fc1d5e36b80 20 client.15996 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.332+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/3775258466 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55d7fc78a380 0x55d7fc78c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.332+0000 7fc1d5e36b80 4 client.15996 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.332+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/3775258466 --> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7fc6d78a0 con 0x55d7fc78a380 2023-09-10T00:41:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.332+0000 7fc1c2ffd700 1 --2- 172.21.15.44:0/3775258466 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55d7fc78a380 0x55d7fc78c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.333+0000 7fc1a2ffd700 10 client.15996 ms_handle_connect on v2:172.21.15.99:6836/566283638 2023-09-10T00:41:01.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.333+0000 7fc1c2ffd700 1 --2- 172.21.15.44:0/3775258466 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55d7fc78a380 0x55d7fc78c770 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a565ae12d77a7fb1 server_cookie=8663615fb390597f in_seq=0 out_seq=0 2023-09-10T00:41:01.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.334+0000 7fc1a2ffd700 1 -- 172.21.15.44:0/3775258466 <== mds.2 v2:172.21.15.99:6836/566283638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d7fc6d78a0 con 0x55d7fc78a380 2023-09-10T00:41:01.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.334+0000 7fc1a2ffd700 10 client.15996 handle_command_reply: tid=0 2023-09-10T00:41:01.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.334+0000 7fc1d5e36b80 1 client.15996 shutdown 2023-09-10T00:41:01.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.334+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/3775258466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc770710 msgr2=0x55d7fc77a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.334+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/3775258466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc770710 0x55d7fc77a880 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4095660 tx=0x7fc1b40988d0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.334+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/3775258466 shutdown_connections 2023-09-10T00:41:01.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.334+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/3775258466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc77ae40 0x55d7fc778240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.334+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/3775258466 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55d7fc78a380 0x55d7fc78c770 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.334+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/3775258466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc770710 0x55d7fc77a880 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.334+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/3775258466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc777b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.334+0000 7fc1a2ffd700 0 client.15996 ms_handle_reset on v2:172.21.15.99:6836/566283638 2023-09-10T00:41:01.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.334+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/3775258466 >> 172.21.15.44:0/3775258466 conn(0x55d7fc6e5090 msgr2=0x55d7fc76b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.334+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/3775258466 shutdown_connections 2023-09-10T00:41:01.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.334+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/3775258466 wait complete. 2023-09-10T00:41:01.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.334+0000 7fc1d5e36b80 20 client.15996 trim_cache size 0 max 16384 2023-09-10T00:41:01.362 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.351+0000 7fc1d5e36b80 1 Processor -- start 2023-09-10T00:41:01.362 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.351+0000 7fc1d5e36b80 1 -- start start 2023-09-10T00:41:01.362 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1d5e36b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc770710 0x55d7fc77a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1d5e36b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc772e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1d5e36b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc77ae40 0x55d7fc773340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1d5e36b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7fc763400 con 0x55d7fc77ae40 2023-09-10T00:41:01.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1d5e36b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7fc776060 con 0x55d7fc770710 2023-09-10T00:41:01.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1d5e36b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7fc76d190 con 0x55d7fc771030 2023-09-10T00:41:01.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1c3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc77ae40 0x55d7fc773340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1c3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc77ae40 0x55d7fc773340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49274/0 (socket says 172.21.15.44:49274) 2023-09-10T00:41:01.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1c3fff700 1 -- 172.21.15.44:0/3230229943 learned_addr learned my addr 172.21.15.44:0/3230229943 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1c2ffd700 1 --2- 172.21.15.44:0/3230229943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc772e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1c37fe700 1 --2- 172.21.15.44:0/3230229943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc770710 0x55d7fc77a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1c3fff700 1 -- 172.21.15.44:0/3230229943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 msgr2=0x55d7fc772e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1c3fff700 1 --2- 172.21.15.44:0/3230229943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc772e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1c3fff700 1 -- 172.21.15.44:0/3230229943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc770710 msgr2=0x55d7fc77a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1c3fff700 1 --2- 172.21.15.44:0/3230229943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc770710 0x55d7fc77a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1c3fff700 1 -- 172.21.15.44:0/3230229943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7fc515250 con 0x55d7fc77ae40 2023-09-10T00:41:01.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1c37fe700 1 --2- 172.21.15.44:0/3230229943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc770710 0x55d7fc77a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:01.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.352+0000 7fc1c2ffd700 1 --2- 172.21.15.44:0/3230229943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc772e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:01.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.353+0000 7fc1c3fff700 1 --2- 172.21.15.44:0/3230229943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc77ae40 0x55d7fc773340 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009e20 tx=0x7fc1bc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76c051a0bd934f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.353+0000 7fc1aa7fc700 1 -- 172.21.15.44:0/3230229943 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc00e040 con 0x55d7fc77ae40 2023-09-10T00:41:01.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.353+0000 7fc1aa7fc700 1 -- 172.21.15.44:0/3230229943 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1bc00f040 con 0x55d7fc77ae40 2023-09-10T00:41:01.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.353+0000 7fc1aa7fc700 1 -- 172.21.15.44:0/3230229943 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc017850 con 0x55d7fc77ae40 2023-09-10T00:41:01.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.353+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/3230229943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc77ae40 msgr2=0x55d7fc773340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.353+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/3230229943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc77ae40 0x55d7fc773340 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009e20 tx=0x7fc1bc009920 comp rx=0 tx=0).stop 2023-09-10T00:41:01.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.354+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/3230229943 shutdown_connections 2023-09-10T00:41:01.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.354+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/3230229943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc77ae40 0x55d7fc773340 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.354+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/3230229943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc770710 0x55d7fc77a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.354+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/3230229943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc772e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.354+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/3230229943 >> 172.21.15.44:0/3230229943 conn(0x55d7fc6e5090 msgr2=0x55d7fc786b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.354+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/3230229943 shutdown_connections 2023-09-10T00:41:01.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.354+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/3230229943 wait complete. 2023-09-10T00:41:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.354+0000 7fc1d5e36b80 1 -- start start 2023-09-10T00:41:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.354+0000 7fc1d5e36b80 1 Processor -- start 2023-09-10T00:41:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1d5e36b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc770710 0x55d7fc7814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1d5e36b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc781a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1d5e36b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc77ae40 0x55d7fc77f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1d5e36b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7fc7800b0 con 0x55d7fc77ae40 2023-09-10T00:41:01.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1d5e36b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7fc780220 con 0x55d7fc770710 2023-09-10T00:41:01.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1d5e36b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7fc7803e0 con 0x55d7fc771030 2023-09-10T00:41:01.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1d5e36b80 10 client.0 _ll_register_callbacks cb 0x7fc1bc009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:01.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1c37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc770710 0x55d7fc7814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1c3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc77ae40 0x55d7fc77f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1c2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc781a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1c37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc770710 0x55d7fc7814d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51016/0 (socket says 172.21.15.44:51016) 2023-09-10T00:41:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1c37fe700 1 -- 172.21.15.44:0/2718895345 learned_addr learned my addr 172.21.15.44:0/2718895345 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1c37fe700 1 -- 172.21.15.44:0/2718895345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 msgr2=0x55d7fc781a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1c37fe700 1 --2- 172.21.15.44:0/2718895345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc781a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1c37fe700 1 -- 172.21.15.44:0/2718895345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc77ae40 msgr2=0x55d7fc77f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1c37fe700 1 --2- 172.21.15.44:0/2718895345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc77ae40 0x55d7fc77f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1c37fe700 1 -- 172.21.15.44:0/2718895345 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7fc4d7c60 con 0x55d7fc770710 2023-09-10T00:41:01.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.355+0000 7fc1c3fff700 1 --2- 172.21.15.44:0/2718895345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc77ae40 0x55d7fc77f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:01.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.356+0000 7fc1c37fe700 1 --2- 172.21.15.44:0/2718895345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc770710 0x55d7fc7814d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40c9bc0 tx=0x7fc1b40a1120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3192924c556eaa9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.356+0000 7fc1a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:41:01.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.356+0000 7fc1a2ffd700 1 -- 172.21.15.44:0/2718895345 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40954f0 con 0x55d7fc770710 2023-09-10T00:41:01.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.356+0000 7fc1a2ffd700 1 -- 172.21.15.44:0/2718895345 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc1b4081030 con 0x55d7fc770710 2023-09-10T00:41:01.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.356+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/2718895345 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1b409cfe0 con 0x55d7fc770710 2023-09-10T00:41:01.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.356+0000 7fc1a2ffd700 1 -- 172.21.15.44:0/2718895345 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1b4098b10 con 0x55d7fc770710 2023-09-10T00:41:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.356+0000 7fc1d5e36b80 10 client.25676 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.356+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/2718895345 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fc1c410dd20 con 0x55d7fc770710 2023-09-10T00:41:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.356+0000 7fc1a2ffd700 1 -- 172.21.15.44:0/2718895345 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1b409cfe0 con 0x55d7fc770710 2023-09-10T00:41:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.356+0000 7fc1a2ffd700 1 -- 172.21.15.44:0/2718895345 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc1b4095030 con 0x55d7fc770710 2023-09-10T00:41:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.356+0000 7fc1d5e36b80 10 client.25676 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.356+0000 7fc1d5e36b80 10 client.25676 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-10T00:41:01.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.356+0000 7fc1d5e36b80 20 client.25676 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:01.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.356+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/2718895345 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55d7fc7defa0 0x55d7fc7e1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.356+0000 7fc1d5e36b80 4 client.25676 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-09-10T00:41:01.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.356+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/2718895345 --> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d7fc6bb920 con 0x55d7fc7defa0 2023-09-10T00:41:01.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1c2ffd700 1 --2- 172.21.15.44:0/2718895345 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55d7fc7defa0 0x55d7fc7e1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1a2ffd700 10 client.25676 ms_handle_connect on v2:172.21.15.99:6836/566283638 2023-09-10T00:41:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1c2ffd700 1 --2- 172.21.15.44:0/2718895345 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55d7fc7defa0 0x55d7fc7e1390 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f348be6e8c7d6320 server_cookie=d2fc717e2b0fb154 in_seq=0 out_seq=0 2023-09-10T00:41:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1a2ffd700 1 -- 172.21.15.44:0/2718895345 <== mds.2 v2:172.21.15.99:6836/566283638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d7fc6bb920 con 0x55d7fc7defa0 2023-09-10T00:41:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1a2ffd700 10 client.25676 handle_command_reply: tid=0 2023-09-10T00:41:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1d5e36b80 1 client.25676 shutdown 2023-09-10T00:41:01.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/2718895345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc770710 msgr2=0x55d7fc7814d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/2718895345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc770710 0x55d7fc7814d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40c9bc0 tx=0x7fc1b40a1120 comp rx=0 tx=0).stop 2023-09-10T00:41:01.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/2718895345 shutdown_connections 2023-09-10T00:41:01.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/2718895345 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x55d7fc7defa0 0x55d7fc7e1390 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/2718895345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7fc77ae40 0x55d7fc77f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1a2ffd700 0 client.25676 ms_handle_reset on v2:172.21.15.99:6836/566283638 2023-09-10T00:41:01.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/2718895345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7fc770710 0x55d7fc7814d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1d5e36b80 1 --2- 172.21.15.44:0/2718895345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7fc771030 0x55d7fc781a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/2718895345 >> 172.21.15.44:0/2718895345 conn(0x55d7fc6e5090 msgr2=0x55d7fc772e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/2718895345 shutdown_connections 2023-09-10T00:41:01.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.357+0000 7fc1d5e36b80 1 -- 172.21.15.44:0/2718895345 wait complete. 2023-09-10T00:41:01.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.358+0000 7fc1d5e36b80 20 client.25676 trim_cache size 0 max 16384 2023-09-10T00:41:01.379 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2023-09-10T00:41:01.380 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.358+0000 7fc1c9c70700 1 -- 172.21.15.44:0/1758569819 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1ac0a08c0 msgr2=0x7fc1ac0a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.358+0000 7fc1c9c70700 1 --2- 172.21.15.44:0/1758569819 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1ac0a08c0 0x7fc1ac0a2d80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800b580 tx=0x7fc1b801a040 comp rx=0 tx=0).stop 2023-09-10T00:41:01.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.358+0000 7fc1c9c70700 1 -- 172.21.15.44:0/1758569819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1c41005a0 msgr2=0x7fc1c4072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.358+0000 7fc1c9c70700 1 --2- 172.21.15.44:0/1758569819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1c41005a0 0x7fc1c4072900 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400d950 tx=0x7fc1b400dc60 comp rx=0 tx=0).stop 2023-09-10T00:41:01.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.358+0000 7fc1c9c70700 1 -- 172.21.15.44:0/1758569819 shutdown_connections 2023-09-10T00:41:01.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.358+0000 7fc1c9c70700 1 --2- 172.21.15.44:0/1758569819 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1ac0a08c0 0x7fc1ac0a2d80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.358+0000 7fc1c9c70700 1 --2- 172.21.15.44:0/1758569819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1c4106660 0x7fc1c41a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.358+0000 7fc1c9c70700 1 --2- 172.21.15.44:0/1758569819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1c41005a0 0x7fc1c4072900 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.358+0000 7fc1c9c70700 1 --2- 172.21.15.44:0/1758569819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c4100f60 0x7fc1c4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.358+0000 7fc1c9c70700 1 -- 172.21.15.44:0/1758569819 >> 172.21.15.44:0/1758569819 conn(0x7fc1c40fc070 msgr2=0x7fc1c40fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.358+0000 7fc1c9c70700 1 -- 172.21.15.44:0/1758569819 shutdown_connections 2023-09-10T00:41:01.383 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.358+0000 7fc1c9c70700 1 -- 172.21.15.44:0/1758569819 wait complete. 2023-09-10T00:41:01.383 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-10T00:41:01.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.530+0000 7f67b9eff700 1 -- 172.21.15.44:0/1113200461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b4106640 msgr2=0x7f67b4106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.531+0000 7f67b9eff700 1 --2- 172.21.15.44:0/1113200461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b4106640 0x7f67b4106a20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f67a40099c0 tx=0x7f67a4009cd0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.531+0000 7f67b9eff700 1 -- 172.21.15.44:0/1113200461 shutdown_connections 2023-09-10T00:41:01.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.531+0000 7f67b9eff700 1 --2- 172.21.15.44:0/1113200461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67b4100f40 0x7f67b410d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.531+0000 7f67b9eff700 1 --2- 172.21.15.44:0/1113200461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b4106640 0x7f67b4106a20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.531+0000 7f67b9eff700 1 --2- 172.21.15.44:0/1113200461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67b4100580 0x7f67b4100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.531+0000 7f67b9eff700 1 -- 172.21.15.44:0/1113200461 >> 172.21.15.44:0/1113200461 conn(0x7f67b40fc070 msgr2=0x7f67b40fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.531+0000 7f67b9eff700 1 -- 172.21.15.44:0/1113200461 shutdown_connections 2023-09-10T00:41:01.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.531+0000 7f67b9eff700 1 -- 172.21.15.44:0/1113200461 wait complete. 2023-09-10T00:41:01.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.531+0000 7f67b9eff700 1 Processor -- start 2023-09-10T00:41:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.531+0000 7f67b9eff700 1 -- start start 2023-09-10T00:41:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b9eff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b4100580 0x7f67b4196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b9eff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67b4100f40 0x7f67b4196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b9eff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67b4106640 0x7f67b419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b9eff700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b419b4f0 con 0x7f67b4100580 2023-09-10T00:41:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b9eff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b419b660 con 0x7f67b4106640 2023-09-10T00:41:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b9eff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b419b820 con 0x7f67b4100f40 2023-09-10T00:41:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b4100580 0x7f67b4196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b4100580 0x7f67b4196510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49290/0 (socket says 172.21.15.44:49290) 2023-09-10T00:41:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b37fe700 1 -- 172.21.15.44:0/1004877584 learned_addr learned my addr 172.21.15.44:0/1004877584 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b2ffd700 1 --2- 172.21.15.44:0/1004877584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67b4100f40 0x7f67b4196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b3fff700 1 --2- 172.21.15.44:0/1004877584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67b4106640 0x7f67b419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b37fe700 1 -- 172.21.15.44:0/1004877584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67b4100f40 msgr2=0x7f67b4196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b37fe700 1 --2- 172.21.15.44:0/1004877584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67b4100f40 0x7f67b4196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b37fe700 1 -- 172.21.15.44:0/1004877584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67b4106640 msgr2=0x7f67b419aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b37fe700 1 --2- 172.21.15.44:0/1004877584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67b4106640 0x7f67b419aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b37fe700 1 -- 172.21.15.44:0/1004877584 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a40095f0 con 0x7f67b4100580 2023-09-10T00:41:01.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.532+0000 7f67b3fff700 1 --2- 172.21.15.44:0/1004877584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67b4106640 0x7f67b419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:41:01.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.533+0000 7f67b37fe700 1 --2- 172.21.15.44:0/1004877584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b4100580 0x7f67b4196510 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009fd0 tx=0x7f67a400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.533+0000 7f67b0ff9700 1 -- 172.21.15.44:0/1004877584 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a401d070 con 0x7f67b4100580 2023-09-10T00:41:01.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.533+0000 7f67b9eff700 1 -- 172.21.15.44:0/1004877584 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b419baa0 con 0x7f67b4100580 2023-09-10T00:41:01.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.533+0000 7f67b0ff9700 1 -- 172.21.15.44:0/1004877584 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67a40054c0 con 0x7f67b4100580 2023-09-10T00:41:01.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.533+0000 7f67b0ff9700 1 -- 172.21.15.44:0/1004877584 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4004a90 con 0x7f67b4100580 2023-09-10T00:41:01.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.533+0000 7f67b9eff700 1 -- 172.21.15.44:0/1004877584 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b4072a10 con 0x7f67b4100580 2023-09-10T00:41:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.536+0000 7f67b0ff9700 1 -- 172.21.15.44:0/1004877584 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f67a400fb80 con 0x7f67b4100580 2023-09-10T00:41:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.536+0000 7f67b0ff9700 1 --2- 172.21.15.44:0/1004877584 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f679c0a0840 0x7f679c0a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.536+0000 7f67b0ff9700 1 -- 172.21.15.44:0/1004877584 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f67a40cd320 con 0x7f67b4100580 2023-09-10T00:41:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.536+0000 7f67b2ffd700 1 --2- 172.21.15.44:0/1004877584 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f679c0a0840 0x7f679c0a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.537+0000 7f67b2ffd700 1 --2- 172.21.15.44:0/1004877584 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f679c0a0840 0x7f679c0a2d00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f67b4197b20 tx=0x7f67a800b410 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.538+0000 7f67c60c5b80 1 Processor -- start 2023-09-10T00:41:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.538+0000 7f67c60c5b80 1 -- start start 2023-09-10T00:41:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.538+0000 7f67c60c5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 0x561b6f6b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.538+0000 7f67c60c5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b8030 0x561b6f6c1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.538+0000 7f67c60c5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6c1e40 0x561b6f6c4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.538+0000 7f67c60c5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6f610730 con 0x561b6f6c1e40 2023-09-10T00:41:01.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.538+0000 7f67c60c5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6f60e050 con 0x561b6f6b8030 2023-09-10T00:41:01.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.538+0000 7f67c60c5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6f609940 con 0x561b6f6b7710 2023-09-10T00:41:01.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.538+0000 7f67b37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 0x561b6f6b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.538+0000 7f67b2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b8030 0x561b6f6c1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.538+0000 7f67b3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6c1e40 0x561b6f6c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.538+0000 7f67b37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 0x561b6f6b7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56180/0 (socket says 172.21.15.44:56180) 2023-09-10T00:41:01.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.538+0000 7f67b37fe700 1 -- 172.21.15.44:0/2737047577 learned_addr learned my addr 172.21.15.44:0/2737047577 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.539+0000 7f67b37fe700 1 -- 172.21.15.44:0/2737047577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b8030 msgr2=0x561b6f6c1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.539+0000 7f67b37fe700 1 --2- 172.21.15.44:0/2737047577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b8030 0x561b6f6c1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.539+0000 7f67b37fe700 1 -- 172.21.15.44:0/2737047577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6c1e40 msgr2=0x561b6f6c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.539+0000 7f67b37fe700 1 --2- 172.21.15.44:0/2737047577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6c1e40 0x561b6f6c4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.539+0000 7f67b37fe700 1 -- 172.21.15.44:0/2737047577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b6f435de0 con 0x561b6f6b7710 2023-09-10T00:41:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.539+0000 7f67b2ffd700 1 --2- 172.21.15.44:0/2737047577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b8030 0x561b6f6c1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.539+0000 7f67b37fe700 1 --2- 172.21.15.44:0/2737047577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 0x561b6f6b7af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67a4096850 tx=0x7f67a4099f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81c80859e01a8709 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.539+0000 7f6799ffb700 1 -- 172.21.15.44:0/2737047577 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a409a350 con 0x561b6f6b7710 2023-09-10T00:41:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.539+0000 7f6799ffb700 1 -- 172.21.15.44:0/2737047577 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67a409e180 con 0x561b6f6b7710 2023-09-10T00:41:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.539+0000 7f6799ffb700 1 -- 172.21.15.44:0/2737047577 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a409a5b0 con 0x561b6f6b7710 2023-09-10T00:41:01.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.540+0000 7f67c60c5b80 1 -- 172.21.15.44:0/2737047577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 msgr2=0x561b6f6b7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.540+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/2737047577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 0x561b6f6b7af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67a4096850 tx=0x7f67a4099f60 comp rx=0 tx=0).stop 2023-09-10T00:41:01.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.540+0000 7f67c60c5b80 1 -- 172.21.15.44:0/2737047577 shutdown_connections 2023-09-10T00:41:01.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.540+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/2737047577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6c1e40 0x561b6f6c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.540+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/2737047577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 0x561b6f6b7af0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.540+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/2737047577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b8030 0x561b6f6c1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.540+0000 7f67c60c5b80 1 -- 172.21.15.44:0/2737047577 >> 172.21.15.44:0/2737047577 conn(0x561b6f62c090 msgr2=0x561b6f6b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.540+0000 7f67c60c5b80 1 -- 172.21.15.44:0/2737047577 shutdown_connections 2023-09-10T00:41:01.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.540+0000 7f67c60c5b80 1 -- 172.21.15.44:0/2737047577 wait complete. 2023-09-10T00:41:01.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.541+0000 7f67c60c5b80 1 -- start start 2023-09-10T00:41:01.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.541+0000 7f67c60c5b80 1 Processor -- start 2023-09-10T00:41:01.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.541+0000 7f67c60c5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 0x561b6f6c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.541+0000 7f67c60c5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b8030 0x561b6f6c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.541+0000 7f67c60c5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6c1e40 0x561b6f6cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.541+0000 7f67c60c5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6f5a7b70 con 0x561b6f6c1e40 2023-09-10T00:41:01.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.541+0000 7f67c60c5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6f5a7870 con 0x561b6f6b8030 2023-09-10T00:41:01.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.541+0000 7f67c60c5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6f48b920 con 0x561b6f6b7710 2023-09-10T00:41:01.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.541+0000 7f67c60c5b80 10 client.0 _ll_register_callbacks cb 0x561b6f4f1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:01.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.541+0000 7f67b37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 0x561b6f6c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.541+0000 7f67b3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6c1e40 0x561b6f6cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f67b37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 0x561b6f6c2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56186/0 (socket says 172.21.15.44:56186) 2023-09-10T00:41:01.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f67b37fe700 1 -- 172.21.15.44:0/1559431317 learned_addr learned my addr 172.21.15.44:0/1559431317 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f67b2ffd700 1 --2- 172.21.15.44:0/1559431317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b8030 0x561b6f6c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f67b3fff700 1 -- 172.21.15.44:0/1559431317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 msgr2=0x561b6f6c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f67b3fff700 1 --2- 172.21.15.44:0/1559431317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 0x561b6f6c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f67b3fff700 1 -- 172.21.15.44:0/1559431317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b8030 msgr2=0x561b6f6c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f67b3fff700 1 --2- 172.21.15.44:0/1559431317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b8030 0x561b6f6c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f67b3fff700 1 -- 172.21.15.44:0/1559431317 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b4072a10 con 0x561b6f6c1e40 2023-09-10T00:41:01.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f67b2ffd700 1 --2- 172.21.15.44:0/1559431317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b8030 0x561b6f6c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:01.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f67b3fff700 1 --2- 172.21.15.44:0/1559431317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6c1e40 0x561b6f6cd2a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f67ac007ab0 tx=0x7f67ac007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a40f71ff563304 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f6792ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:01.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f6792ffd700 1 -- 172.21.15.44:0/1559431317 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac004db0 con 0x561b6f6c1e40 2023-09-10T00:41:01.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f67c60c5b80 1 -- 172.21.15.44:0/1559431317 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b6f5614f0 con 0x561b6f6c1e40 2023-09-10T00:41:01.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f6792ffd700 1 -- 172.21.15.44:0/1559431317 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f67ac0128c0 con 0x561b6f6c1e40 2023-09-10T00:41:01.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f6792ffd700 1 -- 172.21.15.44:0/1559431317 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67ac012a80 con 0x561b6f6c1e40 2023-09-10T00:41:01.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f67c60c5b80 10 client.16014 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:01.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.542+0000 7f67c60c5b80 1 -- 172.21.15.44:0/1559431317 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561b6f41ec60 con 0x561b6f6c1e40 2023-09-10T00:41:01.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.543+0000 7f6792ffd700 1 -- 172.21.15.44:0/1559431317 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561b6f5614f0 con 0x561b6f6c1e40 2023-09-10T00:41:01.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.543+0000 7f6792ffd700 1 -- 172.21.15.44:0/1559431317 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f67ac014040 con 0x561b6f6c1e40 2023-09-10T00:41:01.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.543+0000 7f67c60c5b80 10 client.16014 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:01.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.543+0000 7f67c60c5b80 10 client.16014 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-10T00:41:01.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.543+0000 7f67c60c5b80 20 client.16014 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:01.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.543+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/1559431317 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x561b6f6d1590 0x561b6f6d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.543+0000 7f67c60c5b80 4 client.16014 mds_command: new command op to 24574 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:01.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.543+0000 7f67c60c5b80 1 -- 172.21.15.44:0/1559431317 --> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b6f61e8a0 con 0x561b6f6d1590 2023-09-10T00:41:01.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.543+0000 7f67b37fe700 1 --2- 172.21.15.44:0/1559431317 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x561b6f6d1590 0x561b6f6d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.544+0000 7f6792ffd700 10 client.16014 ms_handle_connect on v2:172.21.15.131:6838/3222652693 2023-09-10T00:41:01.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.544+0000 7f67b37fe700 1 --2- 172.21.15.44:0/1559431317 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x561b6f6d1590 0x561b6f6d39b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e0f30e2f669b1b2 server_cookie=170076f163004f8e in_seq=0 out_seq=0 2023-09-10T00:41:01.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.544+0000 7f6792ffd700 1 -- 172.21.15.44:0/1559431317 <== mds.1 v2:172.21.15.131:6838/3222652693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561b6f61e8a0 con 0x561b6f6d1590 2023-09-10T00:41:01.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.544+0000 7f6792ffd700 10 client.16014 handle_command_reply: tid=0 2023-09-10T00:41:01.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.544+0000 7f67c60c5b80 1 client.16014 shutdown 2023-09-10T00:41:01.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.545+0000 7f67c60c5b80 1 -- 172.21.15.44:0/1559431317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6c1e40 msgr2=0x561b6f6cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.545+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/1559431317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6c1e40 0x561b6f6cd2a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f67ac007ab0 tx=0x7f67ac007dc0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.545+0000 7f67c60c5b80 1 -- 172.21.15.44:0/1559431317 shutdown_connections 2023-09-10T00:41:01.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.545+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/1559431317 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x561b6f6d1590 0x561b6f6d39b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.545+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/1559431317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6c1e40 0x561b6f6cd2a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.545+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/1559431317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 0x561b6f6c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.545+0000 7f6792ffd700 0 client.16014 ms_handle_reset on v2:172.21.15.131:6838/3222652693 2023-09-10T00:41:01.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.545+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/1559431317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b8030 0x561b6f6c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.545+0000 7f67c60c5b80 1 -- 172.21.15.44:0/1559431317 >> 172.21.15.44:0/1559431317 conn(0x561b6f62c090 msgr2=0x561b6f6b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.545+0000 7f67c60c5b80 1 -- 172.21.15.44:0/1559431317 shutdown_connections 2023-09-10T00:41:01.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.545+0000 7f67c60c5b80 1 -- 172.21.15.44:0/1559431317 wait complete. 2023-09-10T00:41:01.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.545+0000 7f67c60c5b80 20 client.16014 trim_cache size 0 max 16384 2023-09-10T00:41:01.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.560+0000 7f67c60c5b80 1 Processor -- start 2023-09-10T00:41:01.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.560+0000 7f67c60c5b80 1 -- start start 2023-09-10T00:41:01.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67c60c5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 0x561b6f6c7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67c60c5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6b8030 0x561b6f6c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67c60c5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6c1e40 0x561b6f6c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67c60c5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6f6af3d0 con 0x561b6f6b8030 2023-09-10T00:41:01.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67c60c5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6f6c2da0 con 0x561b6f6c1e40 2023-09-10T00:41:01.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67c60c5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6f6b89f0 con 0x561b6f6b7710 2023-09-10T00:41:01.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67b2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6b8030 0x561b6f6c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67b2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6b8030 0x561b6f6c2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49314/0 (socket says 172.21.15.44:49314) 2023-09-10T00:41:01.573 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67b2ffd700 1 -- 172.21.15.44:0/2601889806 learned_addr learned my addr 172.21.15.44:0/2601889806 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.573 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67b37fe700 1 --2- 172.21.15.44:0/2601889806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 0x561b6f6c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.573 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67b3fff700 1 --2- 172.21.15.44:0/2601889806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6c1e40 0x561b6f6c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.573 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67b37fe700 1 -- 172.21.15.44:0/2601889806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6c1e40 msgr2=0x561b6f6c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.573 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67b37fe700 1 --2- 172.21.15.44:0/2601889806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6c1e40 0x561b6f6c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.574 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67b37fe700 1 -- 172.21.15.44:0/2601889806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6b8030 msgr2=0x561b6f6c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.574 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67b37fe700 1 --2- 172.21.15.44:0/2601889806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6b8030 0x561b6f6c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.574 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67b37fe700 1 -- 172.21.15.44:0/2601889806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b6f45c250 con 0x561b6f6b7710 2023-09-10T00:41:01.574 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67b2ffd700 1 --2- 172.21.15.44:0/2601889806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6b8030 0x561b6f6c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:01.574 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.561+0000 7f67b37fe700 1 --2- 172.21.15.44:0/2601889806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 0x561b6f6c7200 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f67a409a2e0 tx=0x7f67a4005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=867e139164773069 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.562+0000 7f679a7fc700 1 -- 172.21.15.44:0/2601889806 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a409a350 con 0x561b6f6b7710 2023-09-10T00:41:01.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.562+0000 7f679a7fc700 1 -- 172.21.15.44:0/2601889806 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67a40a2330 con 0x561b6f6b7710 2023-09-10T00:41:01.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.562+0000 7f679a7fc700 1 -- 172.21.15.44:0/2601889806 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a40adbe0 con 0x561b6f6b7710 2023-09-10T00:41:01.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.562+0000 7f67c60c5b80 1 -- 172.21.15.44:0/2601889806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 msgr2=0x561b6f6c7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.562+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/2601889806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 0x561b6f6c7200 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f67a409a2e0 tx=0x7f67a4005f10 comp rx=0 tx=0).stop 2023-09-10T00:41:01.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.563+0000 7f67c60c5b80 1 -- 172.21.15.44:0/2601889806 shutdown_connections 2023-09-10T00:41:01.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.563+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/2601889806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6c1e40 0x561b6f6c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.563+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/2601889806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6b7710 0x561b6f6c7200 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.563+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/2601889806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6b8030 0x561b6f6c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.563+0000 7f67c60c5b80 1 -- 172.21.15.44:0/2601889806 >> 172.21.15.44:0/2601889806 conn(0x561b6f62c090 msgr2=0x561b6f6cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.563+0000 7f67c60c5b80 1 -- 172.21.15.44:0/2601889806 shutdown_connections 2023-09-10T00:41:01.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.563+0000 7f67c60c5b80 1 -- 172.21.15.44:0/2601889806 wait complete. 2023-09-10T00:41:01.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.563+0000 7f67c60c5b80 1 -- start start 2023-09-10T00:41:01.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.563+0000 7f67c60c5b80 1 Processor -- start 2023-09-10T00:41:01.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.564+0000 7f67c60c5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b7710 0x561b6f6c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.564+0000 7f67c60c5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6b8030 0x561b6f6c3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.564+0000 7f67c60c5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6c1e40 0x561b6f6c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.564+0000 7f67c60c5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6f7243e0 con 0x561b6f6b8030 2023-09-10T00:41:01.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.564+0000 7f67c60c5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6f724550 con 0x561b6f6b7710 2023-09-10T00:41:01.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.564+0000 7f67c60c5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6f724710 con 0x561b6f6c1e40 2023-09-10T00:41:01.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.564+0000 7f67c60c5b80 10 client.0 _ll_register_callbacks cb 0x7f67ac009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:01.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.564+0000 7f67b37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b7710 0x561b6f6c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.564+0000 7f67b37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b7710 0x561b6f6c30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51058/0 (socket says 172.21.15.44:51058) 2023-09-10T00:41:01.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.564+0000 7f67b37fe700 1 -- 172.21.15.44:0/1861747969 learned_addr learned my addr 172.21.15.44:0/1861747969 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.564+0000 7f67b3fff700 1 --2- 172.21.15.44:0/1861747969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6c1e40 0x561b6f6c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.564+0000 7f67b37fe700 1 -- 172.21.15.44:0/1861747969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6c1e40 msgr2=0x561b6f6c3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.564+0000 7f67b37fe700 1 --2- 172.21.15.44:0/1861747969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6c1e40 0x561b6f6c3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.564+0000 7f67b37fe700 1 -- 172.21.15.44:0/1861747969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6b8030 msgr2=0x561b6f6c3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.564+0000 7f67b37fe700 1 --2- 172.21.15.44:0/1861747969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6b8030 0x561b6f6c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.564+0000 7f67b37fe700 1 -- 172.21.15.44:0/1861747969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a8010c20 con 0x561b6f6b7710 2023-09-10T00:41:01.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.565+0000 7f67b37fe700 1 --2- 172.21.15.44:0/1861747969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b7710 0x561b6f6c30d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67a409a050 tx=0x7f67a40a24a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8019891218882425 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.581 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.565+0000 7f6792ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:41:01.581 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.565+0000 7f6792ffd700 1 -- 172.21.15.44:0/1861747969 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a40b82c0 con 0x561b6f6b7710 2023-09-10T00:41:01.581 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.565+0000 7f6792ffd700 1 -- 172.21.15.44:0/1861747969 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f67a409a350 con 0x561b6f6b7710 2023-09-10T00:41:01.581 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.565+0000 7f67c60c5b80 1 -- 172.21.15.44:0/1861747969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67a409e180 con 0x561b6f6b7710 2023-09-10T00:41:01.581 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.565+0000 7f6792ffd700 1 -- 172.21.15.44:0/1861747969 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67a40660d0 con 0x561b6f6b7710 2023-09-10T00:41:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.565+0000 7f67c60c5b80 10 client.25691 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.565+0000 7f67c60c5b80 1 -- 172.21.15.44:0/1861747969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561b6f435de0 con 0x561b6f6b7710 2023-09-10T00:41:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.565+0000 7f6792ffd700 1 -- 172.21.15.44:0/1861747969 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67a409e180 con 0x561b6f6b7710 2023-09-10T00:41:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.565+0000 7f6792ffd700 1 -- 172.21.15.44:0/1861747969 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f67a4099840 con 0x561b6f6b7710 2023-09-10T00:41:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.565+0000 7f67c60c5b80 10 client.25691 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.565+0000 7f67c60c5b80 10 client.25691 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-10T00:41:01.583 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.565+0000 7f67c60c5b80 20 client.25691 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:01.583 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.566+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/1861747969 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x561b6f725a10 0x561b6f727e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.583 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.566+0000 7f67c60c5b80 4 client.25691 mds_command: new command op to 24574 tid=0[{"prefix": "damage ls"}] 2023-09-10T00:41:01.583 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.566+0000 7f67c60c5b80 1 -- 172.21.15.44:0/1861747969 --> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561b6f602920 con 0x561b6f725a10 2023-09-10T00:41:01.583 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.566+0000 7f67b2ffd700 1 --2- 172.21.15.44:0/1861747969 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x561b6f725a10 0x561b6f727e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.566+0000 7f6792ffd700 10 client.25691 ms_handle_connect on v2:172.21.15.131:6838/3222652693 2023-09-10T00:41:01.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.566+0000 7f67b2ffd700 1 --2- 172.21.15.44:0/1861747969 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x561b6f725a10 0x561b6f727e30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84ed4512d6f0c2bd server_cookie=67aec60de7e01ca5 in_seq=0 out_seq=0 2023-09-10T00:41:01.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.566+0000 7f6792ffd700 1 -- 172.21.15.44:0/1861747969 <== mds.1 v2:172.21.15.131:6838/3222652693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561b6f602920 con 0x561b6f725a10 2023-09-10T00:41:01.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.566+0000 7f6792ffd700 10 client.25691 handle_command_reply: tid=0 2023-09-10T00:41:01.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.566+0000 7f67c60c5b80 1 client.25691 shutdown 2023-09-10T00:41:01.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.567+0000 7f67c60c5b80 1 -- 172.21.15.44:0/1861747969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b7710 msgr2=0x561b6f6c30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.567+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/1861747969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b7710 0x561b6f6c30d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67a409a050 tx=0x7f67a40a24a0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.567+0000 7f67c60c5b80 1 -- 172.21.15.44:0/1861747969 shutdown_connections 2023-09-10T00:41:01.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.567+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/1861747969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b6f6c1e40 0x561b6f6c3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.567+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/1861747969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6f6b7710 0x561b6f6c30d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.567+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/1861747969 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x561b6f725a10 0x561b6f727e30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.586 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.567+0000 7f67c60c5b80 1 --2- 172.21.15.44:0/1861747969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b6f6b8030 0x561b6f6c3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.586 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.567+0000 7f6792ffd700 0 client.25691 ms_handle_reset on v2:172.21.15.131:6838/3222652693 2023-09-10T00:41:01.586 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.567+0000 7f67c60c5b80 1 -- 172.21.15.44:0/1861747969 >> 172.21.15.44:0/1861747969 conn(0x561b6f62c090 msgr2=0x561b6f6c2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.586 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.567+0000 7f67c60c5b80 1 -- 172.21.15.44:0/1861747969 shutdown_connections 2023-09-10T00:41:01.586 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.567+0000 7f67c60c5b80 1 -- 172.21.15.44:0/1861747969 wait complete. 2023-09-10T00:41:01.587 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.567+0000 7f67c60c5b80 20 client.25691 trim_cache size 0 max 16384 2023-09-10T00:41:01.587 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2023-09-10T00:41:01.588 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.568+0000 7f67b9eff700 1 -- 172.21.15.44:0/1004877584 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f679c0a0840 msgr2=0x7f679c0a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.568+0000 7f67b9eff700 1 --2- 172.21.15.44:0/1004877584 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f679c0a0840 0x7f679c0a2d00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f67b4197b20 tx=0x7f67a800b410 comp rx=0 tx=0).stop 2023-09-10T00:41:01.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.568+0000 7f67b9eff700 1 -- 172.21.15.44:0/1004877584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b4100580 msgr2=0x7f67b4196510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.568+0000 7f67b9eff700 1 --2- 172.21.15.44:0/1004877584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b4100580 0x7f67b4196510 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009fd0 tx=0x7f67a400f690 comp rx=0 tx=0).stop 2023-09-10T00:41:01.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.568+0000 7f67b9eff700 1 -- 172.21.15.44:0/1004877584 shutdown_connections 2023-09-10T00:41:01.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.568+0000 7f67b9eff700 1 --2- 172.21.15.44:0/1004877584 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f679c0a0840 0x7f679c0a2d00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.590 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.568+0000 7f67b9eff700 1 --2- 172.21.15.44:0/1004877584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67b4106640 0x7f67b419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.591 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.568+0000 7f67b9eff700 1 --2- 172.21.15.44:0/1004877584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b4100580 0x7f67b4196510 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.591 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.568+0000 7f67b9eff700 1 --2- 172.21.15.44:0/1004877584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67b4100f40 0x7f67b4196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.591 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.568+0000 7f67b9eff700 1 -- 172.21.15.44:0/1004877584 >> 172.21.15.44:0/1004877584 conn(0x7f67b40fc070 msgr2=0x7f67b40fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.591 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.568+0000 7f67b9eff700 1 -- 172.21.15.44:0/1004877584 shutdown_connections 2023-09-10T00:41:01.591 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.568+0000 7f67b9eff700 1 -- 172.21.15.44:0/1004877584 wait complete. 2023-09-10T00:41:01.592 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-10T00:41:01.743 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.742+0000 7f60a5b5b700 1 -- 172.21.15.44:0/743141179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a00ffe00 msgr2=0x7f60a010c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.743 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.742+0000 7f60a5b5b700 1 --2- 172.21.15.44:0/743141179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a00ffe00 0x7f60a010c310 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f609800b0a0 tx=0x7f609800b3b0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.742+0000 7f60a5b5b700 1 -- 172.21.15.44:0/743141179 shutdown_connections 2023-09-10T00:41:01.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.742+0000 7f60a5b5b700 1 --2- 172.21.15.44:0/743141179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a00ffe00 0x7f60a010c310 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.742+0000 7f60a5b5b700 1 --2- 172.21.15.44:0/743141179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a01054e0 0x7f60a01058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.742+0000 7f60a5b5b700 1 --2- 172.21.15.44:0/743141179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a00ff440 0x7f60a00ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.742+0000 7f60a5b5b700 1 -- 172.21.15.44:0/743141179 >> 172.21.15.44:0/743141179 conn(0x7f60a00faf50 msgr2=0x7f60a00fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.743+0000 7f60a5b5b700 1 -- 172.21.15.44:0/743141179 shutdown_connections 2023-09-10T00:41:01.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.743+0000 7f60a5b5b700 1 -- 172.21.15.44:0/743141179 wait complete. 2023-09-10T00:41:01.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.743+0000 7f60a5b5b700 1 Processor -- start 2023-09-10T00:41:01.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.743+0000 7f60a5b5b700 1 -- start start 2023-09-10T00:41:01.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f60a5b5b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a00ff440 0x7f60a0197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f60a5b5b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a00ffe00 0x7f60a01978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f60a5b5b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a01054e0 0x7f60a01a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f60a5b5b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a01981e0 con 0x7f60a01054e0 2023-09-10T00:41:01.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f60a5b5b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a01a2950 con 0x7f60a00ffe00 2023-09-10T00:41:01.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f60a5b5b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a01a2ac0 con 0x7f60a00ff440 2023-09-10T00:41:01.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f609ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a01054e0 0x7f60a01a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f609f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a00ff440 0x7f60a0197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f609ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a01054e0 0x7f60a01a2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49334/0 (socket says 172.21.15.44:49334) 2023-09-10T00:41:01.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f609ffff700 1 -- 172.21.15.44:0/1038268008 learned_addr learned my addr 172.21.15.44:0/1038268008 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f609f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a00ff440 0x7f60a0197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56214/0 (socket says 172.21.15.44:56214) 2023-09-10T00:41:01.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f609effd700 1 --2- 172.21.15.44:0/1038268008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a00ffe00 0x7f60a01978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f609ffff700 1 -- 172.21.15.44:0/1038268008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a00ff440 msgr2=0x7f60a0197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f609ffff700 1 --2- 172.21.15.44:0/1038268008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a00ff440 0x7f60a0197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f609ffff700 1 -- 172.21.15.44:0/1038268008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a00ffe00 msgr2=0x7f60a01978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f609ffff700 1 --2- 172.21.15.44:0/1038268008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a00ffe00 0x7f60a01978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f609ffff700 1 -- 172.21.15.44:0/1038268008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6098009d00 con 0x7f60a01054e0 2023-09-10T00:41:01.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.744+0000 7f609ffff700 1 --2- 172.21.15.44:0/1038268008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a01054e0 0x7f60a01a2410 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f609800b780 tx=0x7f609800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.745+0000 7f609cff9700 1 -- 172.21.15.44:0/1038268008 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609800e040 con 0x7f60a01054e0 2023-09-10T00:41:01.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.745+0000 7f60a5b5b700 1 -- 172.21.15.44:0/1038268008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a01a2d40 con 0x7f60a01054e0 2023-09-10T00:41:01.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.745+0000 7f609cff9700 1 -- 172.21.15.44:0/1038268008 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f60980094c0 con 0x7f60a01054e0 2023-09-10T00:41:01.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.745+0000 7f609cff9700 1 -- 172.21.15.44:0/1038268008 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60980052a0 con 0x7f60a01054e0 2023-09-10T00:41:01.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.745+0000 7f60a5b5b700 1 -- 172.21.15.44:0/1038268008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a01a31c0 con 0x7f60a01054e0 2023-09-10T00:41:01.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.747+0000 7f609cff9700 1 -- 172.21.15.44:0/1038268008 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f609801d470 con 0x7f60a01054e0 2023-09-10T00:41:01.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.748+0000 7f609cff9700 1 --2- 172.21.15.44:0/1038268008 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f60880a08d0 0x7f60880a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.748+0000 7f609cff9700 1 -- 172.21.15.44:0/1038268008 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f60980cdc30 con 0x7f60a01054e0 2023-09-10T00:41:01.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.748+0000 7f609f7fe700 1 --2- 172.21.15.44:0/1038268008 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f60880a08d0 0x7f60880a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.748+0000 7f609f7fe700 1 --2- 172.21.15.44:0/1038268008 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f60880a08d0 0x7f60880a2d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6090007b60 tx=0x7f60900073d0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.749+0000 7f60b1d21b80 1 Processor -- start 2023-09-10T00:41:01.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.749+0000 7f60b1d21b80 1 -- start start 2023-09-10T00:41:01.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.750+0000 7f60b1d21b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ece710 0x55cff7eceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.750+0000 7f60b1d21b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 0x55cff7edcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.750+0000 7f60b1d21b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7edd250 0x55cff7edf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.750+0000 7f60b1d21b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff7e27730 con 0x55cff7ece710 2023-09-10T00:41:01.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.750+0000 7f60b1d21b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff7e25050 con 0x55cff7edd250 2023-09-10T00:41:01.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.750+0000 7f60b1d21b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff7e20940 con 0x55cff7ecf030 2023-09-10T00:41:01.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.750+0000 7f609effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 0x55cff7edcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.750+0000 7f609effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 0x55cff7edcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56224/0 (socket says 172.21.15.44:56224) 2023-09-10T00:41:01.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.750+0000 7f609effd700 1 -- 172.21.15.44:0/1406980301 learned_addr learned my addr 172.21.15.44:0/1406980301 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.750+0000 7f609ffff700 1 --2- 172.21.15.44:0/1406980301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7edd250 0x55cff7edf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.750+0000 7f609effd700 1 -- 172.21.15.44:0/1406980301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7edd250 msgr2=0x55cff7edf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.750+0000 7f609effd700 1 --2- 172.21.15.44:0/1406980301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7edd250 0x55cff7edf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.750+0000 7f609effd700 1 -- 172.21.15.44:0/1406980301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ece710 msgr2=0x55cff7eceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.750+0000 7f609effd700 1 --2- 172.21.15.44:0/1406980301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ece710 0x55cff7eceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.750+0000 7f609effd700 1 -- 172.21.15.44:0/1406980301 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cff7c4cde0 con 0x55cff7ecf030 2023-09-10T00:41:01.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.750+0000 7f609effd700 1 --2- 172.21.15.44:0/1406980301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 0x55cff7edcd10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6094009d20 tx=0x7f6094011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f13549cffe37ff8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.751+0000 7f6085ffb700 1 -- 172.21.15.44:0/1406980301 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6094019630 con 0x55cff7ecf030 2023-09-10T00:41:01.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.751+0000 7f6085ffb700 1 -- 172.21.15.44:0/1406980301 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6094019c20 con 0x55cff7ecf030 2023-09-10T00:41:01.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.751+0000 7f6085ffb700 1 -- 172.21.15.44:0/1406980301 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609400eab0 con 0x55cff7ecf030 2023-09-10T00:41:01.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.752+0000 7f60b1d21b80 1 -- 172.21.15.44:0/1406980301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 msgr2=0x55cff7edcd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.752+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/1406980301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 0x55cff7edcd10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6094009d20 tx=0x7f6094011720 comp rx=0 tx=0).stop 2023-09-10T00:41:01.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.752+0000 7f60b1d21b80 1 -- 172.21.15.44:0/1406980301 shutdown_connections 2023-09-10T00:41:01.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.752+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/1406980301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7edd250 0x55cff7edf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.752+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/1406980301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ece710 0x55cff7eceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.752+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/1406980301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 0x55cff7edcd10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.752+0000 7f60b1d21b80 1 -- 172.21.15.44:0/1406980301 >> 172.21.15.44:0/1406980301 conn(0x55cff7e43090 msgr2=0x55cff7ec9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.752+0000 7f60b1d21b80 1 -- 172.21.15.44:0/1406980301 shutdown_connections 2023-09-10T00:41:01.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.752+0000 7f60b1d21b80 1 -- 172.21.15.44:0/1406980301 wait complete. 2023-09-10T00:41:01.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.752+0000 7f60b1d21b80 1 -- start start 2023-09-10T00:41:01.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.752+0000 7f60b1d21b80 1 Processor -- start 2023-09-10T00:41:01.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f60b1d21b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ece710 0x55cff7ed9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f60b1d21b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ecf030 0x55cff7eda490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f60b1d21b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7edd250 0x55cff7eda9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f60b1d21b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff7dbeb70 con 0x55cff7ecf030 2023-09-10T00:41:01.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f60b1d21b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff7dbe870 con 0x55cff7edd250 2023-09-10T00:41:01.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f60b1d21b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff7ca2920 con 0x55cff7ece710 2023-09-10T00:41:01.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f60b1d21b80 10 client.0 _ll_register_callbacks cb 0x55cff7d08a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:01.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f609effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ecf030 0x55cff7eda490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f609effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ecf030 0x55cff7eda490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49348/0 (socket says 172.21.15.44:49348) 2023-09-10T00:41:01.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f609effd700 1 -- 172.21.15.44:0/2368924880 learned_addr learned my addr 172.21.15.44:0/2368924880 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f609f7fe700 1 --2- 172.21.15.44:0/2368924880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ece710 0x55cff7ed9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f609ffff700 1 --2- 172.21.15.44:0/2368924880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7edd250 0x55cff7eda9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f609effd700 1 -- 172.21.15.44:0/2368924880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ece710 msgr2=0x55cff7ed9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f609effd700 1 --2- 172.21.15.44:0/2368924880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ece710 0x55cff7ed9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f609effd700 1 -- 172.21.15.44:0/2368924880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7edd250 msgr2=0x55cff7eda9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f609effd700 1 --2- 172.21.15.44:0/2368924880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7edd250 0x55cff7eda9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f609effd700 1 -- 172.21.15.44:0/2368924880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6094009980 con 0x55cff7ecf030 2023-09-10T00:41:01.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f609effd700 1 --2- 172.21.15.44:0/2368924880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ecf030 0x55cff7eda490 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f609401c040 tx=0x7f6094019b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4872bbde274e3915 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.753+0000 7f607effd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:01.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.754+0000 7f607effd700 1 -- 172.21.15.44:0/2368924880 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6094004230 con 0x55cff7ecf030 2023-09-10T00:41:01.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.754+0000 7f60b1d21b80 1 -- 172.21.15.44:0/2368924880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cff7d784f0 con 0x55cff7ecf030 2023-09-10T00:41:01.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.754+0000 7f607effd700 1 -- 172.21.15.44:0/2368924880 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f6094004390 con 0x55cff7ecf030 2023-09-10T00:41:01.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.754+0000 7f607effd700 1 -- 172.21.15.44:0/2368924880 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6094019e50 con 0x55cff7ecf030 2023-09-10T00:41:01.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.754+0000 7f60b1d21b80 10 client.16026 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:01.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.754+0000 7f60b1d21b80 1 -- 172.21.15.44:0/2368924880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cff7c35c60 con 0x55cff7ecf030 2023-09-10T00:41:01.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.754+0000 7f607effd700 1 -- 172.21.15.44:0/2368924880 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cff7d784f0 con 0x55cff7ecf030 2023-09-10T00:41:01.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.754+0000 7f607effd700 1 -- 172.21.15.44:0/2368924880 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6094004ab0 con 0x55cff7ecf030 2023-09-10T00:41:01.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.754+0000 7f60b1d21b80 10 client.16026 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:01.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.754+0000 7f60b1d21b80 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:01.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.754+0000 7f60b1d21b80 20 client.16026 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:01.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.754+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/2368924880 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cff7eecc80 0x55cff7eef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.754+0000 7f60b1d21b80 4 client.16026 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:01.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.754+0000 7f60b1d21b80 1 -- 172.21.15.44:0/2368924880 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cff7e358a0 con 0x55cff7eecc80 2023-09-10T00:41:01.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.755+0000 7f609f7fe700 1 --2- 172.21.15.44:0/2368924880 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cff7eecc80 0x55cff7eef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.755+0000 7f609f7fe700 1 --2- 172.21.15.44:0/2368924880 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cff7eecc80 0x55cff7eef0a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c521d4d5cbb420e9 server_cookie=5620b44a21b75a55 in_seq=0 out_seq=0 2023-09-10T00:41:01.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.755+0000 7f607effd700 10 client.16026 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:01.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.756+0000 7f607effd700 1 -- 172.21.15.44:0/2368924880 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cff7e358a0 con 0x55cff7eecc80 2023-09-10T00:41:01.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.756+0000 7f607effd700 10 client.16026 handle_command_reply: tid=0 2023-09-10T00:41:01.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.756+0000 7f60b1d21b80 1 client.16026 shutdown 2023-09-10T00:41:01.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.756+0000 7f60b1d21b80 1 -- 172.21.15.44:0/2368924880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ecf030 msgr2=0x55cff7eda490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.756+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/2368924880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ecf030 0x55cff7eda490 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f609401c040 tx=0x7f6094019b10 comp rx=0 tx=0).stop 2023-09-10T00:41:01.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.756+0000 7f60b1d21b80 1 -- 172.21.15.44:0/2368924880 shutdown_connections 2023-09-10T00:41:01.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.756+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/2368924880 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cff7eecc80 0x55cff7eef0a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.756+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/2368924880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7edd250 0x55cff7eda9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.756+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/2368924880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ece710 0x55cff7ed9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.756+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/2368924880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ecf030 0x55cff7eda490 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.756+0000 7f60b1d21b80 1 -- 172.21.15.44:0/2368924880 >> 172.21.15.44:0/2368924880 conn(0x55cff7e43090 msgr2=0x55cff7ec97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.756+0000 7f607effd700 0 client.16026 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:01.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.756+0000 7f60b1d21b80 1 -- 172.21.15.44:0/2368924880 shutdown_connections 2023-09-10T00:41:01.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.756+0000 7f60b1d21b80 1 -- 172.21.15.44:0/2368924880 wait complete. 2023-09-10T00:41:01.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.756+0000 7f60b1d21b80 20 client.16026 trim_cache size 0 max 16384 2023-09-10T00:41:01.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f60b1d21b80 1 Processor -- start 2023-09-10T00:41:01.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f60b1d21b80 1 -- start start 2023-09-10T00:41:01.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f60b1d21b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ece710 0x55cff7ee3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f60b1d21b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 0x55cff7ed71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f60b1d21b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7edd250 0x55cff7ed7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f60b1d21b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff7ecb240 con 0x55cff7ece710 2023-09-10T00:41:01.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f60b1d21b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff7edfae0 con 0x55cff7edd250 2023-09-10T00:41:01.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f60b1d21b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff7ee4330 con 0x55cff7ecf030 2023-09-10T00:41:01.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f609f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ece710 0x55cff7ee3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f609f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ece710 0x55cff7ee3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49354/0 (socket says 172.21.15.44:49354) 2023-09-10T00:41:01.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f609f7fe700 1 -- 172.21.15.44:0/3644322230 learned_addr learned my addr 172.21.15.44:0/3644322230 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f609ffff700 1 --2- 172.21.15.44:0/3644322230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7edd250 0x55cff7ed7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f609effd700 1 --2- 172.21.15.44:0/3644322230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 0x55cff7ed71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f609f7fe700 1 -- 172.21.15.44:0/3644322230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 msgr2=0x55cff7ed71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f609f7fe700 1 --2- 172.21.15.44:0/3644322230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 0x55cff7ed71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f609f7fe700 1 -- 172.21.15.44:0/3644322230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7edd250 msgr2=0x55cff7ed7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f609f7fe700 1 --2- 172.21.15.44:0/3644322230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7edd250 0x55cff7ed7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.771+0000 7f609f7fe700 1 -- 172.21.15.44:0/3644322230 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cff7c73250 con 0x55cff7ece710 2023-09-10T00:41:01.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.772+0000 7f609f7fe700 1 --2- 172.21.15.44:0/3644322230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ece710 0x55cff7ee3df0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6090007940 tx=0x7f6090013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc54d8321b2bc898 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.772+0000 7f60867fc700 1 -- 172.21.15.44:0/3644322230 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090024070 con 0x55cff7ece710 2023-09-10T00:41:01.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.772+0000 7f60867fc700 1 -- 172.21.15.44:0/3644322230 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6090025040 con 0x55cff7ece710 2023-09-10T00:41:01.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.772+0000 7f60867fc700 1 -- 172.21.15.44:0/3644322230 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60900038e0 con 0x55cff7ece710 2023-09-10T00:41:01.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.772+0000 7f60b1d21b80 1 -- 172.21.15.44:0/3644322230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ece710 msgr2=0x55cff7ee3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.772+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/3644322230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ece710 0x55cff7ee3df0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6090007940 tx=0x7f6090013000 comp rx=0 tx=0).stop 2023-09-10T00:41:01.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.772+0000 7f60b1d21b80 1 -- 172.21.15.44:0/3644322230 shutdown_connections 2023-09-10T00:41:01.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.772+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/3644322230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7edd250 0x55cff7ed7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.772+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/3644322230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7ece710 0x55cff7ee3df0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.772+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/3644322230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 0x55cff7ed71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.772+0000 7f60b1d21b80 1 -- 172.21.15.44:0/3644322230 >> 172.21.15.44:0/3644322230 conn(0x55cff7e43090 msgr2=0x55cff7ed5250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.773+0000 7f60b1d21b80 1 -- 172.21.15.44:0/3644322230 shutdown_connections 2023-09-10T00:41:01.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.773+0000 7f60b1d21b80 1 -- 172.21.15.44:0/3644322230 wait complete. 2023-09-10T00:41:01.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.773+0000 7f60b1d21b80 1 -- start start 2023-09-10T00:41:01.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.773+0000 7f60b1d21b80 1 Processor -- start 2023-09-10T00:41:01.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.773+0000 7f60b1d21b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7ece710 0x55cff7ee3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.773+0000 7f60b1d21b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 0x55cff7ee2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.773+0000 7f60b1d21b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7edd250 0x55cff7ee2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.773+0000 7f60b1d21b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff7ee2e30 con 0x55cff7edd250 2023-09-10T00:41:01.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.773+0000 7f60b1d21b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff7f40ac0 con 0x55cff7ece710 2023-09-10T00:41:01.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.773+0000 7f60b1d21b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff7f40c00 con 0x55cff7ecf030 2023-09-10T00:41:01.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.773+0000 7f60b1d21b80 10 client.0 _ll_register_callbacks cb 0x7f6098004ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:01.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f609ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7edd250 0x55cff7ee2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f609f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7ece710 0x55cff7ee3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f609f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7ece710 0x55cff7ee3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51100/0 (socket says 172.21.15.44:51100) 2023-09-10T00:41:01.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f609ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7edd250 0x55cff7ee2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49364/0 (socket says 172.21.15.44:49364) 2023-09-10T00:41:01.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f609f7fe700 1 -- 172.21.15.44:0/3777962116 learned_addr learned my addr 172.21.15.44:0/3777962116 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:01.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f609effd700 1 --2- 172.21.15.44:0/3777962116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 0x55cff7ee2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f609f7fe700 1 -- 172.21.15.44:0/3777962116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 msgr2=0x55cff7ee2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f609f7fe700 1 --2- 172.21.15.44:0/3777962116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 0x55cff7ee2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f609f7fe700 1 -- 172.21.15.44:0/3777962116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7edd250 msgr2=0x55cff7ee2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:01.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f609f7fe700 1 --2- 172.21.15.44:0/3777962116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7edd250 0x55cff7ee2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f609f7fe700 1 -- 172.21.15.44:0/3777962116 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60900109b0 con 0x55cff7ece710 2023-09-10T00:41:01.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f609ffff700 1 --2- 172.21.15.44:0/3777962116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7edd250 0x55cff7ee2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:01.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f609f7fe700 1 --2- 172.21.15.44:0/3777962116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7ece710 0x55cff7ee3d70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f60900076a0 tx=0x7f6090026cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35a30c19f05387d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f607effd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:41:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f607effd700 1 -- 172.21.15.44:0/3777962116 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090024030 con 0x55cff7ece710 2023-09-10T00:41:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f607effd700 1 -- 172.21.15.44:0/3777962116 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f6090017070 con 0x55cff7ece710 2023-09-10T00:41:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.774+0000 7f60b1d21b80 1 -- 172.21.15.44:0/3777962116 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cff7ee9800 con 0x55cff7ece710 2023-09-10T00:41:01.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.775+0000 7f607effd700 1 -- 172.21.15.44:0/3777962116 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6090026420 con 0x55cff7ece710 2023-09-10T00:41:01.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.775+0000 7f60b1d21b80 10 client.25709 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:01.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.775+0000 7f60b1d21b80 1 -- 172.21.15.44:0/3777962116 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cff7c4b2b0 con 0x55cff7ece710 2023-09-10T00:41:01.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.775+0000 7f607effd700 1 -- 172.21.15.44:0/3777962116 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cff7ee9800 con 0x55cff7ece710 2023-09-10T00:41:01.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.775+0000 7f607effd700 1 -- 172.21.15.44:0/3777962116 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6090011480 con 0x55cff7ece710 2023-09-10T00:41:01.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.775+0000 7f60b1d21b80 10 client.25709 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:01.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.775+0000 7f60b1d21b80 10 client.25709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.775+0000 7f60b1d21b80 20 client.25709 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.775+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/3777962116 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cff7ed0ef0 0x55cff7ed3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.775+0000 7f60b1d21b80 4 client.25709 mds_command: new command op to 24605 tid=0[{"prefix": "damage ls"}] 2023-09-10T00:41:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.775+0000 7f60b1d21b80 1 -- 172.21.15.44:0/3777962116 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f6094019c20 con 0x55cff7ed0ef0 2023-09-10T00:41:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.775+0000 7f609effd700 1 --2- 172.21.15.44:0/3777962116 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cff7ed0ef0 0x55cff7ed3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.776+0000 7f609effd700 1 --2- 172.21.15.44:0/3777962116 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cff7ed0ef0 0x55cff7ed3310 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4bc814854a8911a server_cookie=1e4402b40d0b8660 in_seq=0 out_seq=0 2023-09-10T00:41:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.776+0000 7f607effd700 10 client.25709 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.776+0000 7f607effd700 1 -- 172.21.15.44:0/3777962116 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f6094019c20 con 0x55cff7ed0ef0 2023-09-10T00:41:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.776+0000 7f607effd700 10 client.25709 handle_command_reply: tid=0 2023-09-10T00:41:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.776+0000 7f60b1d21b80 1 client.25709 shutdown 2023-09-10T00:41:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.776+0000 7f60b1d21b80 1 -- 172.21.15.44:0/3777962116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7ece710 msgr2=0x55cff7ee3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.776+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/3777962116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7ece710 0x55cff7ee3d70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f60900076a0 tx=0x7f6090026cb0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.776+0000 7f60b1d21b80 1 -- 172.21.15.44:0/3777962116 shutdown_connections 2023-09-10T00:41:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.776+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/3777962116 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cff7ed0ef0 0x55cff7ed3310 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.776+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/3777962116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cff7edd250 0x55cff7ee2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.776+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/3777962116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cff7ece710 0x55cff7ee3d70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.776+0000 7f60b1d21b80 1 --2- 172.21.15.44:0/3777962116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cff7ecf030 0x55cff7ee2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.776+0000 7f607effd700 0 client.25709 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:01.802 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.776+0000 7f60b1d21b80 1 -- 172.21.15.44:0/3777962116 >> 172.21.15.44:0/3777962116 conn(0x55cff7e43090 msgr2=0x55cff7ed9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.802 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.776+0000 7f60b1d21b80 1 -- 172.21.15.44:0/3777962116 shutdown_connections 2023-09-10T00:41:01.802 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.776+0000 7f60b1d21b80 1 -- 172.21.15.44:0/3777962116 wait complete. 2023-09-10T00:41:01.802 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.777+0000 7f60b1d21b80 20 client.25709 trim_cache size 0 max 16384 2023-09-10T00:41:01.803 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2023-09-10T00:41:01.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.777+0000 7f60a5b5b700 1 -- 172.21.15.44:0/1038268008 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f60880a08d0 msgr2=0x7f60880a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.777+0000 7f60a5b5b700 1 --2- 172.21.15.44:0/1038268008 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f60880a08d0 0x7f60880a2d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6090007b60 tx=0x7f60900073d0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.777+0000 7f60a5b5b700 1 -- 172.21.15.44:0/1038268008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a01054e0 msgr2=0x7f60a01a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.777+0000 7f60a5b5b700 1 --2- 172.21.15.44:0/1038268008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a01054e0 0x7f60a01a2410 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f609800b780 tx=0x7f609800bd60 comp rx=0 tx=0).stop 2023-09-10T00:41:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.777+0000 7f60a5b5b700 1 -- 172.21.15.44:0/1038268008 shutdown_connections 2023-09-10T00:41:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.777+0000 7f60a5b5b700 1 --2- 172.21.15.44:0/1038268008 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f60880a08d0 0x7f60880a2d90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.777+0000 7f60a5b5b700 1 --2- 172.21.15.44:0/1038268008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a01054e0 0x7f60a01a2410 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.777+0000 7f60a5b5b700 1 --2- 172.21.15.44:0/1038268008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a00ff440 0x7f60a0197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.777+0000 7f60a5b5b700 1 --2- 172.21.15.44:0/1038268008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a00ffe00 0x7f60a01978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.777+0000 7f60a5b5b700 1 -- 172.21.15.44:0/1038268008 >> 172.21.15.44:0/1038268008 conn(0x7f60a00faf50 msgr2=0x7f60a00fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.777+0000 7f60a5b5b700 1 -- 172.21.15.44:0/1038268008 shutdown_connections 2023-09-10T00:41:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:01.777+0000 7f60a5b5b700 1 -- 172.21.15.44:0/1038268008 wait complete. 2023-09-10T00:41:02.232 INFO:teuthology.orchestra.run.smithi044.stderr:200000 of 50000000 tuples (0%) done (elapsed 1.87 s, remaining 464.72 s) 2023-09-10T00:41:02.807 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-10T00:41:02.808 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-10T00:41:02.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.951+0000 7f7f09a85700 1 -- 172.21.15.44:0/1022070991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f040ffe00 msgr2=0x7f7f0410c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:02.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.951+0000 7f7f09a85700 1 --2- 172.21.15.44:0/1022070991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f040ffe00 0x7f7f0410c310 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00b0a0 tx=0x7f7efc00b3b0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.952+0000 7f7f09a85700 1 -- 172.21.15.44:0/1022070991 shutdown_connections 2023-09-10T00:41:02.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.952+0000 7f7f09a85700 1 --2- 172.21.15.44:0/1022070991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f040ffe00 0x7f7f0410c310 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.952+0000 7f7f09a85700 1 --2- 172.21.15.44:0/1022070991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f041054e0 0x7f7f041058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.952+0000 7f7f09a85700 1 --2- 172.21.15.44:0/1022070991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f040ff440 0x7f7f040ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.952+0000 7f7f09a85700 1 -- 172.21.15.44:0/1022070991 >> 172.21.15.44:0/1022070991 conn(0x7f7f040faf50 msgr2=0x7f7f040fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:02.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.952+0000 7f7f09a85700 1 -- 172.21.15.44:0/1022070991 shutdown_connections 2023-09-10T00:41:02.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.952+0000 7f7f09a85700 1 -- 172.21.15.44:0/1022070991 wait complete. 2023-09-10T00:41:02.955 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.952+0000 7f7f09a85700 1 Processor -- start 2023-09-10T00:41:02.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f09a85700 1 -- start start 2023-09-10T00:41:02.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f09a85700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f040ff440 0x7f7f04197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f09a85700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f040ffe00 0x7f7f04197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f09a85700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f041054e0 0x7f7f041a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.956 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f09a85700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f04198190 con 0x7f7f041054e0 2023-09-10T00:41:02.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f09a85700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f041a28d0 con 0x7f7f040ffe00 2023-09-10T00:41:02.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f09a85700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f041a2a90 con 0x7f7f040ff440 2023-09-10T00:41:02.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f037fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f040ff440 0x7f7f04197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:02.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f037fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f040ff440 0x7f7f04197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56256/0 (socket says 172.21.15.44:56256) 2023-09-10T00:41:02.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f037fe700 1 -- 172.21.15.44:0/4042159932 learned_addr learned my addr 172.21.15.44:0/4042159932 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:02.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/4042159932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f040ffe00 0x7f7f04197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:02.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f03fff700 1 --2- 172.21.15.44:0/4042159932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f041054e0 0x7f7f041a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:02.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f037fe700 1 -- 172.21.15.44:0/4042159932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f040ffe00 msgr2=0x7f7f04197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:02.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f037fe700 1 --2- 172.21.15.44:0/4042159932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f040ffe00 0x7f7f04197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f037fe700 1 -- 172.21.15.44:0/4042159932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f041054e0 msgr2=0x7f7f041a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:02.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f037fe700 1 --2- 172.21.15.44:0/4042159932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f041054e0 0x7f7f041a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f037fe700 1 -- 172.21.15.44:0/4042159932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc009d00 con 0x7f7f040ff440 2023-09-10T00:41:02.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.953+0000 7f7f037fe700 1 --2- 172.21.15.44:0/4042159932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f040ff440 0x7f7f04197310 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400e9a0 tx=0x7f7ef400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:02.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.954+0000 7f7f00ff9700 1 -- 172.21.15.44:0/4042159932 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4009790 con 0x7f7f040ff440 2023-09-10T00:41:02.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.954+0000 7f7f09a85700 1 -- 172.21.15.44:0/4042159932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f041a2d70 con 0x7f7f040ff440 2023-09-10T00:41:02.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.954+0000 7f7f09a85700 1 -- 172.21.15.44:0/4042159932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f041a3270 con 0x7f7f040ff440 2023-09-10T00:41:02.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.954+0000 7f7f00ff9700 1 -- 172.21.15.44:0/4042159932 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ef4010430 con 0x7f7f040ff440 2023-09-10T00:41:02.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.954+0000 7f7f00ff9700 1 -- 172.21.15.44:0/4042159932 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef400cea0 con 0x7f7f040ff440 2023-09-10T00:41:02.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.957+0000 7f7f00ff9700 1 -- 172.21.15.44:0/4042159932 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f7ef400ca10 con 0x7f7f040ff440 2023-09-10T00:41:02.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.957+0000 7f7f00ff9700 1 --2- 172.21.15.44:0/4042159932 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7eec0a08d0 0x7f7eec0a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.957+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/4042159932 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7eec0a08d0 0x7f7eec0a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:02.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.957+0000 7f7f00ff9700 1 -- 172.21.15.44:0/4042159932 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7ef4014070 con 0x7f7f040ff440 2023-09-10T00:41:02.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.958+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/4042159932 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7eec0a08d0 0x7f7eec0a2d90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7f04068030 tx=0x7f7ef800b410 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:02.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.958+0000 7f7f15c4bb80 1 Processor -- start 2023-09-10T00:41:02.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.958+0000 7f7f15c4bb80 1 -- start start 2023-09-10T00:41:02.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.958+0000 7f7f15c4bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff816e790 0x560ff816eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.958+0000 7f7f15c4bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816f0b0 0x560ff817cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.958+0000 7f7f15c4bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff817d2d0 0x560ff817f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.958+0000 7f7f15c4bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff80c77b0 con 0x560ff816f0b0 2023-09-10T00:41:02.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.958+0000 7f7f15c4bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff80c50d0 con 0x560ff816e790 2023-09-10T00:41:02.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.958+0000 7f7f15c4bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff80c09c0 con 0x560ff817d2d0 2023-09-10T00:41:02.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.958+0000 7f7f02ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816f0b0 0x560ff817cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:02.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.959+0000 7f7f02ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816f0b0 0x560ff817cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49384/0 (socket says 172.21.15.44:49384) 2023-09-10T00:41:02.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.959+0000 7f7f02ffd700 1 -- 172.21.15.44:0/3027334084 learned_addr learned my addr 172.21.15.44:0/3027334084 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:02.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.959+0000 7f7f03fff700 1 --2- 172.21.15.44:0/3027334084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff817d2d0 0x560ff817f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:02.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.959+0000 7f7f037fe700 1 --2- 172.21.15.44:0/3027334084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff816e790 0x560ff816eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:02.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.959+0000 7f7f02ffd700 1 -- 172.21.15.44:0/3027334084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff817d2d0 msgr2=0x560ff817f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:02.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.959+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/3027334084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff817d2d0 0x560ff817f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.959+0000 7f7f02ffd700 1 -- 172.21.15.44:0/3027334084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff816e790 msgr2=0x560ff816eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:02.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.959+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/3027334084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff816e790 0x560ff816eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.959+0000 7f7f02ffd700 1 -- 172.21.15.44:0/3027334084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ff7eece60 con 0x560ff816f0b0 2023-09-10T00:41:02.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.959+0000 7f7f037fe700 1 --2- 172.21.15.44:0/3027334084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff816e790 0x560ff816eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:02.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.959+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/3027334084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816f0b0 0x560ff817cd90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8010f40 tx=0x7f7ef8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c6c44089e51b9b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:02.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.959+0000 7f7ee9ffb700 1 -- 172.21.15.44:0/3027334084 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8011db0 con 0x560ff816f0b0 2023-09-10T00:41:02.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.959+0000 7f7ee9ffb700 1 -- 172.21.15.44:0/3027334084 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ef8015070 con 0x560ff816f0b0 2023-09-10T00:41:02.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.959+0000 7f7ee9ffb700 1 -- 172.21.15.44:0/3027334084 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef801c610 con 0x560ff816f0b0 2023-09-10T00:41:02.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.960+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/3027334084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816f0b0 msgr2=0x560ff817cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:02.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.960+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/3027334084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816f0b0 0x560ff817cd90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8010f40 tx=0x7f7ef8013000 comp rx=0 tx=0).stop 2023-09-10T00:41:02.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.960+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/3027334084 shutdown_connections 2023-09-10T00:41:02.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.960+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/3027334084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff817d2d0 0x560ff817f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.960+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/3027334084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff816e790 0x560ff816eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.960+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/3027334084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816f0b0 0x560ff817cd90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.960+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/3027334084 >> 172.21.15.44:0/3027334084 conn(0x560ff80e3110 msgr2=0x560ff8169c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:02.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.960+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/3027334084 shutdown_connections 2023-09-10T00:41:02.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.960+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/3027334084 wait complete. 2023-09-10T00:41:02.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.960+0000 7f7f15c4bb80 1 -- start start 2023-09-10T00:41:02.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.961+0000 7f7f15c4bb80 1 Processor -- start 2023-09-10T00:41:02.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.961+0000 7f7f15c4bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff817cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.961+0000 7f7f15c4bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff816f0b0 0x560ff8179eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.961+0000 7f7f15c4bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff817d2d0 0x560ff817a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.961+0000 7f7f15c4bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff805ebf0 con 0x560ff816e790 2023-09-10T00:41:02.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.961+0000 7f7f15c4bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff805e8f0 con 0x560ff817d2d0 2023-09-10T00:41:02.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.961+0000 7f7f15c4bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff7f429a0 con 0x560ff816f0b0 2023-09-10T00:41:02.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.961+0000 7f7f15c4bb80 10 client.0 _ll_register_callbacks cb 0x560ff7fa8b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:02.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.961+0000 7f7f02ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff816f0b0 0x560ff8179eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:02.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.961+0000 7f7f037fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff817cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:02.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.961+0000 7f7f02ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff816f0b0 0x560ff8179eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56274/0 (socket says 172.21.15.44:56274) 2023-09-10T00:41:02.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.961+0000 7f7f02ffd700 1 -- 172.21.15.44:0/1382095844 learned_addr learned my addr 172.21.15.44:0/1382095844 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:02.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.961+0000 7f7f03fff700 1 --2- 172.21.15.44:0/1382095844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff817d2d0 0x560ff817a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:02.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.962+0000 7f7f02ffd700 1 -- 172.21.15.44:0/1382095844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff817d2d0 msgr2=0x560ff817a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:02.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.962+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/1382095844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff817d2d0 0x560ff817a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.962+0000 7f7f02ffd700 1 -- 172.21.15.44:0/1382095844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 msgr2=0x560ff817cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:02.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.962+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/1382095844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff817cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.962+0000 7f7f02ffd700 1 -- 172.21.15.44:0/1382095844 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef8010c20 con 0x560ff816f0b0 2023-09-10T00:41:02.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.962+0000 7f7f037fe700 1 --2- 172.21.15.44:0/1382095844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff817cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:02.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.962+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/1382095844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff816f0b0 0x560ff8179eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800bfc0 tx=0x7f7ef801ebc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63bce6de7ec3dd8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:02.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.962+0000 7f7ee2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:41:02.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.962+0000 7f7ee2ffd700 1 -- 172.21.15.44:0/1382095844 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef802c070 con 0x560ff816f0b0 2023-09-10T00:41:02.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.962+0000 7f7ee2ffd700 1 -- 172.21.15.44:0/1382095844 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7ef8017070 con 0x560ff816f0b0 2023-09-10T00:41:02.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.962+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/1382095844 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ff8018570 con 0x560ff816f0b0 2023-09-10T00:41:02.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.962+0000 7f7ee2ffd700 1 -- 172.21.15.44:0/1382095844 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ef801e630 con 0x560ff816f0b0 2023-09-10T00:41:02.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.962+0000 7f7f15c4bb80 10 client.25660 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:02.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.962+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/1382095844 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560ff7ed5ce0 con 0x560ff816f0b0 2023-09-10T00:41:02.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.962+0000 7f7ee2ffd700 1 -- 172.21.15.44:0/1382095844 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560ff8018570 con 0x560ff816f0b0 2023-09-10T00:41:02.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.963+0000 7f7ee2ffd700 1 -- 172.21.15.44:0/1382095844 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7ef8004910 con 0x560ff816f0b0 2023-09-10T00:41:02.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.963+0000 7f7f15c4bb80 10 client.25660 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:02.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.963+0000 7f7f15c4bb80 10 client.25660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:02.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.963+0000 7f7f15c4bb80 20 client.25660 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:02.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.963+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/1382095844 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560ff818cc70 0x560ff818f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.963+0000 7f7f15c4bb80 4 client.25660 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:02.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.963+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/1382095844 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ff80d5920 con 0x560ff818cc70 2023-09-10T00:41:02.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.963+0000 7f7f037fe700 1 --2- 172.21.15.44:0/1382095844 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560ff818cc70 0x560ff818f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:02.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.963+0000 7f7f037fe700 1 --2- 172.21.15.44:0/1382095844 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560ff818cc70 0x560ff818f090 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2be08bd696ddaf server_cookie=a53da5c24fa6e6c3 in_seq=0 out_seq=0 2023-09-10T00:41:02.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.963+0000 7f7ee2ffd700 10 client.25660 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:02.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.964+0000 7f7ee2ffd700 1 -- 172.21.15.44:0/1382095844 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560ff80d5920 con 0x560ff818cc70 2023-09-10T00:41:02.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.964+0000 7f7ee2ffd700 10 client.25660 handle_command_reply: tid=0 2023-09-10T00:41:02.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.964+0000 7f7f15c4bb80 1 client.25660 shutdown 2023-09-10T00:41:02.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.964+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/1382095844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff816f0b0 msgr2=0x560ff8179eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:02.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.964+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/1382095844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff816f0b0 0x560ff8179eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800bfc0 tx=0x7f7ef801ebc0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.964+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/1382095844 shutdown_connections 2023-09-10T00:41:02.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.964+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/1382095844 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560ff818cc70 0x560ff818f090 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.964+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/1382095844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff817d2d0 0x560ff817a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.964+0000 7f7ee2ffd700 0 client.25660 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:02.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.964+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/1382095844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff817cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.964+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/1382095844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff816f0b0 0x560ff8179eb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.964+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/1382095844 >> 172.21.15.44:0/1382095844 conn(0x560ff80e3110 msgr2=0x560ff8169860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:02.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.964+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/1382095844 shutdown_connections 2023-09-10T00:41:02.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.964+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/1382095844 wait complete. 2023-09-10T00:41:02.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.964+0000 7f7f15c4bb80 20 client.25660 trim_cache size 0 max 16384 2023-09-10T00:41:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.986+0000 7f7f15c4bb80 1 Processor -- start 2023-09-10T00:41:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.986+0000 7f7f15c4bb80 1 -- start start 2023-09-10T00:41:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.986+0000 7f7f15c4bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff8183f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.986+0000 7f7f15c4bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff816f0b0 0x560ff816de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.986+0000 7f7f15c4bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff817d2d0 0x560ff817c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.986+0000 7f7f15c4bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff8018570 con 0x560ff816e790 2023-09-10T00:41:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.986+0000 7f7f15c4bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff81706c0 con 0x560ff816f0b0 2023-09-10T00:41:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.986+0000 7f7f15c4bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff8166450 con 0x560ff817d2d0 2023-09-10T00:41:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.987+0000 7f7f037fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff8183f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.987+0000 7f7f037fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff8183f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49396/0 (socket says 172.21.15.44:49396) 2023-09-10T00:41:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.987+0000 7f7f037fe700 1 -- 172.21.15.44:0/103978342 learned_addr learned my addr 172.21.15.44:0/103978342 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.987+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/103978342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff816f0b0 0x560ff816de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:02.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.987+0000 7f7f03fff700 1 --2- 172.21.15.44:0/103978342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff817d2d0 0x560ff817c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:02.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.987+0000 7f7f02ffd700 1 -- 172.21.15.44:0/103978342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff817d2d0 msgr2=0x560ff817c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:02.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.987+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/103978342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff817d2d0 0x560ff817c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.987+0000 7f7f02ffd700 1 -- 172.21.15.44:0/103978342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 msgr2=0x560ff8183f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:02.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.987+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/103978342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff8183f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.987+0000 7f7f02ffd700 1 -- 172.21.15.44:0/103978342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ff7f132d0 con 0x560ff816f0b0 2023-09-10T00:41:02.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.987+0000 7f7f037fe700 1 --2- 172.21.15.44:0/103978342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff8183f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:02.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.987+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/103978342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff816f0b0 0x560ff816de90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8004cd0 tx=0x7f7ef8004e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a77c9efb2f6eea6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:02.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.987+0000 7f7eea7fc700 1 -- 172.21.15.44:0/103978342 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef802c070 con 0x560ff816f0b0 2023-09-10T00:41:02.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.987+0000 7f7eea7fc700 1 -- 172.21.15.44:0/103978342 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ef8030410 con 0x560ff816f0b0 2023-09-10T00:41:02.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.987+0000 7f7eea7fc700 1 -- 172.21.15.44:0/103978342 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8004fc0 con 0x560ff816f0b0 2023-09-10T00:41:02.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.988+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/103978342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff816f0b0 msgr2=0x560ff816de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:02.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.988+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/103978342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff816f0b0 0x560ff816de90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8004cd0 tx=0x7f7ef8004e40 comp rx=0 tx=0).stop 2023-09-10T00:41:02.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.988+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/103978342 shutdown_connections 2023-09-10T00:41:02.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.988+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/103978342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff817d2d0 0x560ff817c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.993 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.988+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/103978342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff8183f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.988+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/103978342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff816f0b0 0x560ff816de90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:02.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.988+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/103978342 >> 172.21.15.44:0/103978342 conn(0x560ff80e3110 msgr2=0x560ff816d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:02.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.988+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/103978342 shutdown_connections 2023-09-10T00:41:02.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.988+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/103978342 wait complete. 2023-09-10T00:41:02.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.988+0000 7f7f15c4bb80 1 -- start start 2023-09-10T00:41:02.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.989+0000 7f7f15c4bb80 1 Processor -- start 2023-09-10T00:41:02.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.989+0000 7f7f15c4bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff8182bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.989+0000 7f7f15c4bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff816f0b0 0x560ff8183100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.989+0000 7f7f15c4bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff817d2d0 0x560ff81e0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:02.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.989+0000 7f7f15c4bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff8183870 con 0x560ff816e790 2023-09-10T00:41:02.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.989+0000 7f7f15c4bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff81e1010 con 0x560ff817d2d0 2023-09-10T00:41:02.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.989+0000 7f7f15c4bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff81e11d0 con 0x560ff816f0b0 2023-09-10T00:41:02.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.989+0000 7f7f15c4bb80 10 client.0 _ll_register_callbacks cb 0x7f7ef4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:03.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.989+0000 7f7f037fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff8182bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.989+0000 7f7f037fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff8182bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49402/0 (socket says 172.21.15.44:49402) 2023-09-10T00:41:03.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.989+0000 7f7f037fe700 1 -- 172.21.15.44:0/1686518066 learned_addr learned my addr 172.21.15.44:0/1686518066 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:03.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.989+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/1686518066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff816f0b0 0x560ff8183100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.989+0000 7f7f03fff700 1 --2- 172.21.15.44:0/1686518066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff817d2d0 0x560ff81e0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.989+0000 7f7f037fe700 1 -- 172.21.15.44:0/1686518066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff816f0b0 msgr2=0x560ff8183100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:03.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7f037fe700 1 --2- 172.21.15.44:0/1686518066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff816f0b0 0x560ff8183100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7f037fe700 1 -- 172.21.15.44:0/1686518066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff817d2d0 msgr2=0x560ff81e0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:03.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7f037fe700 1 --2- 172.21.15.44:0/1686518066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff817d2d0 0x560ff81e0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7f037fe700 1 -- 172.21.15.44:0/1686518066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef4095040 con 0x560ff816e790 2023-09-10T00:41:03.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/1686518066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff816f0b0 0x560ff8183100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:03.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7f037fe700 1 --2- 172.21.15.44:0/1686518066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff8182bc0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40994e0 tx=0x7f7ef4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6e8bdafd8737d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:03.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7ee2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:03.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7ee2ffd700 1 -- 172.21.15.44:0/1686518066 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40b5070 con 0x560ff816e790 2023-09-10T00:41:03.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7ee2ffd700 1 -- 172.21.15.44:0/1686518066 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7ef40cb040 con 0x560ff816e790 2023-09-10T00:41:03.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/1686518066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ef8015070 con 0x560ff816e790 2023-09-10T00:41:03.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7ee2ffd700 1 -- 172.21.15.44:0/1686518066 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ef40a15c0 con 0x560ff816e790 2023-09-10T00:41:03.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7f15c4bb80 10 client.16050 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:03.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/1686518066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560ff7ed5ce0 con 0x560ff816e790 2023-09-10T00:41:03.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7ee2ffd700 1 -- 172.21.15.44:0/1686518066 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ef8015070 con 0x560ff816e790 2023-09-10T00:41:03.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7ee2ffd700 1 -- 172.21.15.44:0/1686518066 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7ef40998d0 con 0x560ff816e790 2023-09-10T00:41:03.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7f15c4bb80 10 client.16050 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:03.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7f15c4bb80 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:03.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7f15c4bb80 20 client.16050 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:03.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/1686518066 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560ff81e2910 0x560ff81e4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.990+0000 7f7f15c4bb80 4 client.16050 mds_command: new command op to 24605 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-10T00:41:03.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.991+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/1686518066 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560ff80b99a0 con 0x560ff81e2910 2023-09-10T00:41:03.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.991+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/1686518066 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560ff81e2910 0x560ff81e4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.991+0000 7f7f02ffd700 1 --2- 172.21.15.44:0/1686518066 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560ff81e2910 0x560ff81e4d30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=940053fcf315217d server_cookie=7a07e66dd134ab37 in_seq=0 out_seq=0 2023-09-10T00:41:03.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.991+0000 7f7ee2ffd700 10 client.16050 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:03.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.992+0000 7f7ee2ffd700 1 -- 172.21.15.44:0/1686518066 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560ff80b99a0 con 0x560ff81e2910 2023-09-10T00:41:03.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.992+0000 7f7ee2ffd700 10 client.16050 handle_command_reply: tid=0 2023-09-10T00:41:03.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.992+0000 7f7f15c4bb80 1 client.16050 shutdown 2023-09-10T00:41:03.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.992+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/1686518066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 msgr2=0x560ff8182bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:03.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.992+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/1686518066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff8182bc0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40994e0 tx=0x7f7ef4000bc0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.992+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/1686518066 shutdown_connections 2023-09-10T00:41:03.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.992+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/1686518066 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560ff81e2910 0x560ff81e4d30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.992+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/1686518066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ff817d2d0 0x560ff81e0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.992+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/1686518066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ff816e790 0x560ff8182bc0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.993+0000 7f7ee2ffd700 0 client.16050 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:03.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.993+0000 7f7f15c4bb80 1 --2- 172.21.15.44:0/1686518066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ff816f0b0 0x560ff8183100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.993+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/1686518066 >> 172.21.15.44:0/1686518066 conn(0x560ff80e3110 msgr2=0x560ff816e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:03.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.993+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/1686518066 shutdown_connections 2023-09-10T00:41:03.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.993+0000 7f7f15c4bb80 1 -- 172.21.15.44:0/1686518066 wait complete. 2023-09-10T00:41:03.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.993+0000 7f7f15c4bb80 20 client.16050 trim_cache size 0 max 16384 2023-09-10T00:41:03.009 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:41:03.009 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2023-09-10T00:41:03.009 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "24dbd23a-27ba-458e-88c3-b2f208d376e7", 2023-09-10T00:41:03.009 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2023-09-10T00:41:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:41:03.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.993+0000 7f7f09a85700 1 -- 172.21.15.44:0/4042159932 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7eec0a08d0 msgr2=0x7f7eec0a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:03.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.993+0000 7f7f09a85700 1 --2- 172.21.15.44:0/4042159932 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7eec0a08d0 0x7f7eec0a2d90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7f04068030 tx=0x7f7ef800b410 comp rx=0 tx=0).stop 2023-09-10T00:41:03.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.993+0000 7f7f09a85700 1 -- 172.21.15.44:0/4042159932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f040ff440 msgr2=0x7f7f04197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:03.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.993+0000 7f7f09a85700 1 --2- 172.21.15.44:0/4042159932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f040ff440 0x7f7f04197310 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400e9a0 tx=0x7f7ef400ecb0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.993+0000 7f7f09a85700 1 -- 172.21.15.44:0/4042159932 shutdown_connections 2023-09-10T00:41:03.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.993+0000 7f7f09a85700 1 --2- 172.21.15.44:0/4042159932 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7eec0a08d0 0x7f7eec0a2d90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.993+0000 7f7f09a85700 1 --2- 172.21.15.44:0/4042159932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f041054e0 0x7f7f041a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.993+0000 7f7f09a85700 1 --2- 172.21.15.44:0/4042159932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f040ff440 0x7f7f04197310 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.993+0000 7f7f09a85700 1 --2- 172.21.15.44:0/4042159932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f040ffe00 0x7f7f04197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.993+0000 7f7f09a85700 1 -- 172.21.15.44:0/4042159932 >> 172.21.15.44:0/4042159932 conn(0x7f7f040faf50 msgr2=0x7f7f040fb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:03.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.994+0000 7f7f09a85700 1 -- 172.21.15.44:0/4042159932 shutdown_connections 2023-09-10T00:41:03.013 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:02.994+0000 7f7f09a85700 1 -- 172.21.15.44:0/4042159932 wait complete. 2023-09-10T00:41:03.013 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:02 smithi044 ceph-mon[98676]: pgmap v14: 129 pgs: 129 active+clean; 47 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 5.1 MiB/s wr, 394 op/s 2023-09-10T00:41:03.013 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:02 smithi044 ceph-mon[98676]: mgrmap e25: x(active, since 22s), standbys: z, y 2023-09-10T00:41:03.014 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:41:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:02 smithi131 ceph-mon[103713]: pgmap v14: 129 pgs: 129 active+clean; 47 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 5.1 MiB/s wr, 394 op/s 2023-09-10T00:41:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:02 smithi131 ceph-mon[103713]: mgrmap e25: x(active, since 22s), standbys: z, y 2023-09-10T00:41:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:02 smithi099 ceph-mon[102692]: pgmap v14: 129 pgs: 129 active+clean; 47 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 5.1 MiB/s wr, 394 op/s 2023-09-10T00:41:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:02 smithi099 ceph-mon[102692]: mgrmap e25: x(active, since 22s), standbys: z, y 2023-09-10T00:41:03.189 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.188+0000 7f11ebe48700 1 -- 172.21.15.44:0/1148750728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e40ffe00 msgr2=0x7f11e410c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:03.189 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.188+0000 7f11ebe48700 1 --2- 172.21.15.44:0/1148750728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e40ffe00 0x7f11e410c310 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00b0a0 tx=0x7f11dc00b3b0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.188+0000 7f11ebe48700 1 -- 172.21.15.44:0/1148750728 shutdown_connections 2023-09-10T00:41:03.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.188+0000 7f11ebe48700 1 --2- 172.21.15.44:0/1148750728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e40ffe00 0x7f11e410c310 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.188+0000 7f11ebe48700 1 --2- 172.21.15.44:0/1148750728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e41054e0 0x7f11e41058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.188+0000 7f11ebe48700 1 --2- 172.21.15.44:0/1148750728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11e40ff440 0x7f11e40ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.188+0000 7f11ebe48700 1 -- 172.21.15.44:0/1148750728 >> 172.21.15.44:0/1148750728 conn(0x7f11e40faf50 msgr2=0x7f11e40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:03.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.189+0000 7f11ebe48700 1 -- 172.21.15.44:0/1148750728 shutdown_connections 2023-09-10T00:41:03.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.189+0000 7f11ebe48700 1 -- 172.21.15.44:0/1148750728 wait complete. 2023-09-10T00:41:03.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.189+0000 7f11ebe48700 1 Processor -- start 2023-09-10T00:41:03.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.189+0000 7f11ebe48700 1 -- start start 2023-09-10T00:41:03.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11ebe48700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e40ff440 0x7f11e410cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11ebe48700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11e40ffe00 0x7f11e410d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11ebe48700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41054e0 0x7f11e41a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11ebe48700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e410da50 con 0x7f11e41054e0 2023-09-10T00:41:03.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11ebe48700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e41a4dc0 con 0x7f11e40ff440 2023-09-10T00:41:03.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11ebe48700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e41a4f00 con 0x7f11e40ffe00 2023-09-10T00:41:03.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11e9be4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e40ff440 0x7f11e410cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11ea3e5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41054e0 0x7f11e41a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11e9be4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e40ff440 0x7f11e410cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51156/0 (socket says 172.21.15.44:51156) 2023-09-10T00:41:03.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11e9be4700 1 -- 172.21.15.44:0/4233417454 learned_addr learned my addr 172.21.15.44:0/4233417454 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:03.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11e93e3700 1 --2- 172.21.15.44:0/4233417454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11e40ffe00 0x7f11e410d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11e9be4700 1 -- 172.21.15.44:0/4233417454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11e40ffe00 msgr2=0x7f11e410d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:03.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11e9be4700 1 --2- 172.21.15.44:0/4233417454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11e40ffe00 0x7f11e410d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11e9be4700 1 -- 172.21.15.44:0/4233417454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41054e0 msgr2=0x7f11e41a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:03.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11e9be4700 1 --2- 172.21.15.44:0/4233417454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41054e0 0x7f11e41a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11e9be4700 1 -- 172.21.15.44:0/4233417454 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11dc009d00 con 0x7f11e40ff440 2023-09-10T00:41:03.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11ea3e5700 1 --2- 172.21.15.44:0/4233417454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41054e0 0x7f11e41a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:03.197 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.190+0000 7f11e9be4700 1 --2- 172.21.15.44:0/4233417454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e40ff440 0x7f11e410cce0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f11e000e9a0 tx=0x7f11e000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:03.197 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.191+0000 7f11daffd700 1 -- 172.21.15.44:0/4233417454 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0009790 con 0x7f11e40ff440 2023-09-10T00:41:03.197 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.191+0000 7f11ebe48700 1 -- 172.21.15.44:0/4233417454 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e41a5160 con 0x7f11e40ff440 2023-09-10T00:41:03.197 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.191+0000 7f11ebe48700 1 -- 172.21.15.44:0/4233417454 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e41a5690 con 0x7f11e40ff440 2023-09-10T00:41:03.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.191+0000 7f11daffd700 1 -- 172.21.15.44:0/4233417454 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11e0010430 con 0x7f11e40ff440 2023-09-10T00:41:03.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.191+0000 7f11daffd700 1 -- 172.21.15.44:0/4233417454 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e000cea0 con 0x7f11e40ff440 2023-09-10T00:41:03.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.193+0000 7f11daffd700 1 -- 172.21.15.44:0/4233417454 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f11e000ca10 con 0x7f11e40ff440 2023-09-10T00:41:03.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.194+0000 7f11daffd700 1 --2- 172.21.15.44:0/4233417454 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f11d00a08d0 0x7f11d00a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.194+0000 7f11f800eb80 1 Processor -- start 2023-09-10T00:41:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.194+0000 7f11daffd700 1 -- 172.21.15.44:0/4233417454 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f11e0014070 con 0x7f11e40ff440 2023-09-10T00:41:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11f800eb80 1 -- start start 2023-09-10T00:41:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11f800eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f766a710 0x55b8f766aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11f800eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f7674900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11f800eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 0x55b8f7677230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11f800eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f75c3730 con 0x55b8f766b030 2023-09-10T00:41:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11f800eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f75c1050 con 0x55b8f7674e40 2023-09-10T00:41:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11f800eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f75bc940 con 0x55b8f766a710 2023-09-10T00:41:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11e93e3700 1 --2- 172.21.15.44:0/4233417454 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f11d00a08d0 0x7f11d00a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11ea3e5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 0x55b8f7677230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11ea3e5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 0x55b8f7677230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51166/0 (socket says 172.21.15.44:51166) 2023-09-10T00:41:03.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11ea3e5700 1 -- 172.21.15.44:0/3425157003 learned_addr learned my addr 172.21.15.44:0/3425157003 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:03.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11e9be4700 1 --2- 172.21.15.44:0/3425157003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f766a710 0x55b8f766aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.206 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11ea3e5700 1 -- 172.21.15.44:0/3425157003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f766a710 msgr2=0x55b8f766aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:03.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11ea3e5700 1 --2- 172.21.15.44:0/3425157003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f766a710 0x55b8f766aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11ea3e5700 1 -- 172.21.15.44:0/3425157003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 msgr2=0x55b8f7674900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:03.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11e93e3700 1 --2- 172.21.15.44:0/3425157003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f7674900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11ea3e5700 1 --2- 172.21.15.44:0/3425157003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f7674900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11ea3e5700 1 -- 172.21.15.44:0/3425157003 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8f73e8de0 con 0x55b8f7674e40 2023-09-10T00:41:03.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11e9be4700 1 --2- 172.21.15.44:0/3425157003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f766a710 0x55b8f766aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:03.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.195+0000 7f11e93e3700 1 --2- 172.21.15.44:0/3425157003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f7674900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:41:03.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.196+0000 7f11e93e3700 1 --2- 172.21.15.44:0/4233417454 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f11d00a08d0 0x7f11d00a2d90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f11e410e4d0 tx=0x7f11d40098c0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:03.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.196+0000 7f11ea3e5700 1 --2- 172.21.15.44:0/3425157003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 0x55b8f7677230 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00b0a0 tx=0x7f11dc009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d3ead07e6adc7a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:03.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.196+0000 7f11cffff700 1 -- 172.21.15.44:0/3425157003 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc00e040 con 0x55b8f7674e40 2023-09-10T00:41:03.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.196+0000 7f11cffff700 1 -- 172.21.15.44:0/3425157003 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11dc0094a0 con 0x55b8f7674e40 2023-09-10T00:41:03.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.196+0000 7f11cffff700 1 -- 172.21.15.44:0/3425157003 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0052a0 con 0x55b8f7674e40 2023-09-10T00:41:03.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.197+0000 7f11f800eb80 1 -- 172.21.15.44:0/3425157003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 msgr2=0x55b8f7677230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:03.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.197+0000 7f11f800eb80 1 --2- 172.21.15.44:0/3425157003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 0x55b8f7677230 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00b0a0 tx=0x7f11dc009ba0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.197+0000 7f11f800eb80 1 -- 172.21.15.44:0/3425157003 shutdown_connections 2023-09-10T00:41:03.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.197+0000 7f11f800eb80 1 --2- 172.21.15.44:0/3425157003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 0x55b8f7677230 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.197+0000 7f11f800eb80 1 --2- 172.21.15.44:0/3425157003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f766a710 0x55b8f766aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.197+0000 7f11f800eb80 1 --2- 172.21.15.44:0/3425157003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f7674900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.197+0000 7f11f800eb80 1 -- 172.21.15.44:0/3425157003 >> 172.21.15.44:0/3425157003 conn(0x55b8f75df090 msgr2=0x55b8f7665bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:03.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.197+0000 7f11f800eb80 1 -- 172.21.15.44:0/3425157003 shutdown_connections 2023-09-10T00:41:03.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.197+0000 7f11f800eb80 1 -- 172.21.15.44:0/3425157003 wait complete. 2023-09-10T00:41:03.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.197+0000 7f11f800eb80 1 -- start start 2023-09-10T00:41:03.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.197+0000 7f11f800eb80 1 Processor -- start 2023-09-10T00:41:03.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11f800eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766a710 0x55b8f76747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11f800eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f766b030 0x55b8f7671a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11f800eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 0x55b8f7671fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11f800eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f755ab70 con 0x55b8f766a710 2023-09-10T00:41:03.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11f800eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f755a870 con 0x55b8f7674e40 2023-09-10T00:41:03.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11f800eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f743e920 con 0x55b8f766b030 2023-09-10T00:41:03.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11f800eb80 10 client.0 _ll_register_callbacks cb 0x55b8f74a4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:03.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11e9be4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766a710 0x55b8f76747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11ea3e5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 0x55b8f7671fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11e9be4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766a710 0x55b8f76747b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49430/0 (socket says 172.21.15.44:49430) 2023-09-10T00:41:03.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11e9be4700 1 -- 172.21.15.44:0/3420841155 learned_addr learned my addr 172.21.15.44:0/3420841155 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:03.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11e93e3700 1 --2- 172.21.15.44:0/3420841155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f766b030 0x55b8f7671a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11ea3e5700 1 -- 172.21.15.44:0/3420841155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f766b030 msgr2=0x55b8f7671a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:03.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11ea3e5700 1 --2- 172.21.15.44:0/3420841155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f766b030 0x55b8f7671a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11ea3e5700 1 -- 172.21.15.44:0/3420841155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766a710 msgr2=0x55b8f76747b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:03.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11ea3e5700 1 --2- 172.21.15.44:0/3420841155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766a710 0x55b8f76747b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.198+0000 7f11ea3e5700 1 -- 172.21.15.44:0/3420841155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e4067760 con 0x55b8f7674e40 2023-09-10T00:41:03.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.199+0000 7f11ea3e5700 1 --2- 172.21.15.44:0/3420841155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 0x55b8f7671fb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00d010 tx=0x7f11dc00bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa5a41a40db1b857 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:03.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.199+0000 7f11cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:41:03.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.199+0000 7f11cd7fa700 1 -- 172.21.15.44:0/3420841155 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc00e040 con 0x55b8f7674e40 2023-09-10T00:41:03.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.199+0000 7f11f800eb80 1 -- 172.21.15.44:0/3420841155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8f75144f0 con 0x55b8f7674e40 2023-09-10T00:41:03.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.199+0000 7f11cd7fa700 1 -- 172.21.15.44:0/3420841155 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f11dc003a00 con 0x55b8f7674e40 2023-09-10T00:41:03.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.199+0000 7f11cd7fa700 1 -- 172.21.15.44:0/3420841155 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11dc003bc0 con 0x55b8f7674e40 2023-09-10T00:41:03.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.199+0000 7f11f800eb80 10 client.25739 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:03.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.199+0000 7f11f800eb80 1 -- 172.21.15.44:0/3420841155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b8f73d1c60 con 0x55b8f7674e40 2023-09-10T00:41:03.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.199+0000 7f11cd7fa700 1 -- 172.21.15.44:0/3420841155 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b8f75144f0 con 0x55b8f7674e40 2023-09-10T00:41:03.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.200+0000 7f11cd7fa700 1 -- 172.21.15.44:0/3420841155 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f11dc00b0d0 con 0x55b8f7674e40 2023-09-10T00:41:03.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.200+0000 7f11f800eb80 10 client.25739 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:03.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.200+0000 7f11f800eb80 10 client.25739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.200+0000 7f11f800eb80 20 client.25739 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.200+0000 7f11f800eb80 1 --2- 172.21.15.44:0/3420841155 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8f76843f0 0x55b8f7686810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.200+0000 7f11f800eb80 4 client.25739 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.200+0000 7f11f800eb80 1 -- 172.21.15.44:0/3420841155 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8f75d18a0 con 0x55b8f76843f0 2023-09-10T00:41:03.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.200+0000 7f11e9be4700 1 --2- 172.21.15.44:0/3420841155 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8f76843f0 0x55b8f7686810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.200+0000 7f11cd7fa700 10 client.25739 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:03.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.200+0000 7f11e9be4700 1 --2- 172.21.15.44:0/3420841155 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8f76843f0 0x55b8f7686810 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc37e22020e11f37 server_cookie=86267441f49db63a in_seq=0 out_seq=0 2023-09-10T00:41:03.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.201+0000 7f11cd7fa700 1 -- 172.21.15.44:0/3420841155 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b8f75d18a0 con 0x55b8f76843f0 2023-09-10T00:41:03.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.201+0000 7f11cd7fa700 10 client.25739 handle_command_reply: tid=0 2023-09-10T00:41:03.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.201+0000 7f11f800eb80 1 client.25739 shutdown 2023-09-10T00:41:03.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.201+0000 7f11f800eb80 1 -- 172.21.15.44:0/3420841155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 msgr2=0x55b8f7671fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:03.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.201+0000 7f11f800eb80 1 --2- 172.21.15.44:0/3420841155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 0x55b8f7671fb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00d010 tx=0x7f11dc00bce0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.201+0000 7f11f800eb80 1 -- 172.21.15.44:0/3420841155 shutdown_connections 2023-09-10T00:41:03.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.201+0000 7f11f800eb80 1 --2- 172.21.15.44:0/3420841155 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8f76843f0 0x55b8f7686810 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.201+0000 7f11f800eb80 1 --2- 172.21.15.44:0/3420841155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 0x55b8f7671fb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.201+0000 7f11f800eb80 1 --2- 172.21.15.44:0/3420841155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766a710 0x55b8f76747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.201+0000 7f11cd7fa700 0 client.25739 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:03.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.201+0000 7f11f800eb80 1 --2- 172.21.15.44:0/3420841155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f766b030 0x55b8f7671a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.201+0000 7f11f800eb80 1 -- 172.21.15.44:0/3420841155 >> 172.21.15.44:0/3420841155 conn(0x55b8f75df090 msgr2=0x55b8f7665780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:03.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.201+0000 7f11f800eb80 1 -- 172.21.15.44:0/3420841155 shutdown_connections 2023-09-10T00:41:03.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.201+0000 7f11f800eb80 1 -- 172.21.15.44:0/3420841155 wait complete. 2023-09-10T00:41:03.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.201+0000 7f11f800eb80 20 client.25739 trim_cache size 0 max 16384 2023-09-10T00:41:03.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.223+0000 7f11f800eb80 1 Processor -- start 2023-09-10T00:41:03.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.223+0000 7f11f800eb80 1 -- start start 2023-09-10T00:41:03.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.223+0000 7f11f800eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f766a710 0x55b8f76711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.223+0000 7f11f800eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f76742e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.223+0000 7f11f800eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 0x55b8f766cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.223+0000 7f11f800eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f76725c0 con 0x55b8f766b030 2023-09-10T00:41:03.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.223+0000 7f11f800eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f7681220 con 0x55b8f7674e40 2023-09-10T00:41:03.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.223+0000 7f11f800eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f7671760 con 0x55b8f766a710 2023-09-10T00:41:03.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.223+0000 7f11e93e3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f76742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.223+0000 7f11e93e3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f76742e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49442/0 (socket says 172.21.15.44:49442) 2023-09-10T00:41:03.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.223+0000 7f11e93e3700 1 -- 172.21.15.44:0/74175670 learned_addr learned my addr 172.21.15.44:0/74175670 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:03.227 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.223+0000 7f11ea3e5700 1 --2- 172.21.15.44:0/74175670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 0x55b8f766cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.227 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.223+0000 7f11e9be4700 1 --2- 172.21.15.44:0/74175670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f766a710 0x55b8f76711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.227 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.224+0000 7f11e93e3700 1 -- 172.21.15.44:0/74175670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f766a710 msgr2=0x55b8f76711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:03.227 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.224+0000 7f11e93e3700 1 --2- 172.21.15.44:0/74175670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f766a710 0x55b8f76711f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.227 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.224+0000 7f11e93e3700 1 -- 172.21.15.44:0/74175670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 msgr2=0x55b8f766cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:03.228 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.224+0000 7f11e93e3700 1 --2- 172.21.15.44:0/74175670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 0x55b8f766cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.228 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.224+0000 7f11e93e3700 1 -- 172.21.15.44:0/74175670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8f740f250 con 0x55b8f766b030 2023-09-10T00:41:03.228 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.224+0000 7f11ea3e5700 1 --2- 172.21.15.44:0/74175670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 0x55b8f766cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:03.229 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.224+0000 7f11e93e3700 1 --2- 172.21.15.44:0/74175670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f76742e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f11d4013ad0 tx=0x7f11d4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ada19925e35366 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:03.229 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.224+0000 7f11d8ff9700 1 -- 172.21.15.44:0/74175670 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d40040f0 con 0x55b8f766b030 2023-09-10T00:41:03.229 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.224+0000 7f11d8ff9700 1 -- 172.21.15.44:0/74175670 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11d40108c0 con 0x55b8f766b030 2023-09-10T00:41:03.229 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.224+0000 7f11d8ff9700 1 -- 172.21.15.44:0/74175670 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4004cd0 con 0x55b8f766b030 2023-09-10T00:41:03.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.225+0000 7f11f800eb80 1 -- 172.21.15.44:0/74175670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 msgr2=0x55b8f76742e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:03.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.225+0000 7f11f800eb80 1 --2- 172.21.15.44:0/74175670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f76742e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f11d4013ad0 tx=0x7f11d4016000 comp rx=0 tx=0).stop 2023-09-10T00:41:03.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.225+0000 7f11f800eb80 1 -- 172.21.15.44:0/74175670 shutdown_connections 2023-09-10T00:41:03.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.225+0000 7f11f800eb80 1 --2- 172.21.15.44:0/74175670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f7674e40 0x55b8f766cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.225+0000 7f11f800eb80 1 --2- 172.21.15.44:0/74175670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f766a710 0x55b8f76711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.225+0000 7f11f800eb80 1 --2- 172.21.15.44:0/74175670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f76742e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.225+0000 7f11f800eb80 1 -- 172.21.15.44:0/74175670 >> 172.21.15.44:0/74175670 conn(0x55b8f75df090 msgr2=0x55b8f76719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:03.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.225+0000 7f11f800eb80 1 -- 172.21.15.44:0/74175670 shutdown_connections 2023-09-10T00:41:03.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.225+0000 7f11f800eb80 1 -- 172.21.15.44:0/74175670 wait complete. 2023-09-10T00:41:03.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.225+0000 7f11f800eb80 1 -- start start 2023-09-10T00:41:03.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.226+0000 7f11f800eb80 1 Processor -- start 2023-09-10T00:41:03.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.226+0000 7f11f800eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f766a710 0x55b8f767a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.226+0000 7f11f800eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f767a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.226+0000 7f11f800eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f7674e40 0x55b8f767ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.226+0000 7f11f800eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f76d83f0 con 0x55b8f766b030 2023-09-10T00:41:03.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.226+0000 7f11f800eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f76d8560 con 0x55b8f766a710 2023-09-10T00:41:03.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.226+0000 7f11f800eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f76d8720 con 0x55b8f7674e40 2023-09-10T00:41:03.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.226+0000 7f11f800eb80 10 client.0 _ll_register_callbacks cb 0x7f11dc009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:03.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.226+0000 7f11e93e3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f767a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.226+0000 7f11e93e3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f767a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49446/0 (socket says 172.21.15.44:49446) 2023-09-10T00:41:03.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.226+0000 7f11e93e3700 1 -- 172.21.15.44:0/2639116879 learned_addr learned my addr 172.21.15.44:0/2639116879 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:03.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.226+0000 7f11e9be4700 1 --2- 172.21.15.44:0/2639116879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f766a710 0x55b8f767a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.226+0000 7f11ea3e5700 1 --2- 172.21.15.44:0/2639116879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f7674e40 0x55b8f767ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.227+0000 7f11e93e3700 1 -- 172.21.15.44:0/2639116879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f7674e40 msgr2=0x55b8f767ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:03.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.227+0000 7f11e93e3700 1 --2- 172.21.15.44:0/2639116879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f7674e40 0x55b8f767ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.227+0000 7f11e93e3700 1 -- 172.21.15.44:0/2639116879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f766a710 msgr2=0x55b8f767a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:03.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.227+0000 7f11e93e3700 1 --2- 172.21.15.44:0/2639116879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f766a710 0x55b8f767a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.227+0000 7f11e93e3700 1 -- 172.21.15.44:0/2639116879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8f73d1c60 con 0x55b8f766b030 2023-09-10T00:41:03.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.227+0000 7f11ea3e5700 1 --2- 172.21.15.44:0/2639116879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f7674e40 0x55b8f767ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:03.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.227+0000 7f11e93e3700 1 --2- 172.21.15.44:0/2639116879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f767a7f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f11d4005920 tx=0x7f11d4011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39082c82ab24a4ab server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:03.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.227+0000 7f11cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:03.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.227+0000 7f11cd7fa700 1 -- 172.21.15.44:0/2639116879 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4004500 con 0x55b8f766b030 2023-09-10T00:41:03.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.227+0000 7f11f800eb80 1 -- 172.21.15.44:0/2639116879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11dc0094a0 con 0x55b8f766b030 2023-09-10T00:41:03.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.227+0000 7f11cd7fa700 1 -- 172.21.15.44:0/2639116879 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f11d4003680 con 0x55b8f766b030 2023-09-10T00:41:03.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.227+0000 7f11cd7fa700 1 -- 172.21.15.44:0/2639116879 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11d4003840 con 0x55b8f766b030 2023-09-10T00:41:03.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.227+0000 7f11f800eb80 10 client.16065 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:03.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.227+0000 7f11f800eb80 1 -- 172.21.15.44:0/2639116879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f11d4019050 con 0x55b8f766b030 2023-09-10T00:41:03.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.227+0000 7f11cd7fa700 1 -- 172.21.15.44:0/2639116879 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11dc0094a0 con 0x55b8f766b030 2023-09-10T00:41:03.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.228+0000 7f11cd7fa700 1 -- 172.21.15.44:0/2639116879 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f11d4029a30 con 0x55b8f766b030 2023-09-10T00:41:03.242 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.228+0000 7f11f800eb80 10 client.16065 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:03.242 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.228+0000 7f11f800eb80 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:03.242 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.228+0000 7f11f800eb80 20 client.16065 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:03.242 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.228+0000 7f11f800eb80 1 --2- 172.21.15.44:0/2639116879 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8f76d9db0 0x55b8f76dc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:03.242 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.228+0000 7f11f800eb80 4 client.16065 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:41:03.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.228+0000 7f11f800eb80 1 -- 172.21.15.44:0/2639116879 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b8f75b5920 con 0x55b8f76d9db0 2023-09-10T00:41:03.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.228+0000 7f11e9be4700 1 --2- 172.21.15.44:0/2639116879 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8f76d9db0 0x55b8f76dc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:03.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.228+0000 7f11cd7fa700 10 client.16065 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:03.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.228+0000 7f11e9be4700 1 --2- 172.21.15.44:0/2639116879 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8f76d9db0 0x55b8f76dc1d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96fd841adfe4673a server_cookie=9fc3183a83cda17a in_seq=0 out_seq=0 2023-09-10T00:41:03.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.228+0000 7f11cd7fa700 1 -- 172.21.15.44:0/2639116879 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b8f75b5920 con 0x55b8f76d9db0 2023-09-10T00:41:03.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.228+0000 7f11cd7fa700 10 client.16065 handle_command_reply: tid=0 2023-09-10T00:41:03.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11f800eb80 1 client.16065 shutdown 2023-09-10T00:41:03.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11f800eb80 1 -- 172.21.15.44:0/2639116879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 msgr2=0x55b8f767a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:03.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11f800eb80 1 --2- 172.21.15.44:0/2639116879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f767a7f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f11d4005920 tx=0x7f11d4011f20 comp rx=0 tx=0).stop 2023-09-10T00:41:03.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11f800eb80 1 -- 172.21.15.44:0/2639116879 shutdown_connections 2023-09-10T00:41:03.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11f800eb80 1 --2- 172.21.15.44:0/2639116879 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8f76d9db0 0x55b8f76dc1d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11f800eb80 1 --2- 172.21.15.44:0/2639116879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8f7674e40 0x55b8f767ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11f800eb80 1 --2- 172.21.15.44:0/2639116879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8f766a710 0x55b8f767a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11cd7fa700 0 client.16065 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:03.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11f800eb80 1 --2- 172.21.15.44:0/2639116879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8f766b030 0x55b8f767a7f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11f800eb80 1 -- 172.21.15.44:0/2639116879 >> 172.21.15.44:0/2639116879 conn(0x55b8f75df090 msgr2=0x55b8f766cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:03.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11f800eb80 1 -- 172.21.15.44:0/2639116879 shutdown_connections 2023-09-10T00:41:03.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11f800eb80 1 -- 172.21.15.44:0/2639116879 wait complete. 2023-09-10T00:41:03.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11f800eb80 20 client.16065 trim_cache size 0 max 16384 2023-09-10T00:41:03.246 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:41:03.247 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:41:03.247 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:41:03.247 INFO:teuthology.orchestra.run.smithi044.stdout: "24dbd23a-27ba-458e-88c3-b2f208d376e7": { 2023-09-10T00:41:03.247 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:41:03.247 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "24dbd23a-27ba-458e-88c3-b2f208d376e7", 2023-09-10T00:41:03.248 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:41:03.248 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:41:03.248 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:41:03.249 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:41:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11ebe48700 1 -- 172.21.15.44:0/4233417454 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f11d00a08d0 msgr2=0x7f11d00a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11ebe48700 1 --2- 172.21.15.44:0/4233417454 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f11d00a08d0 0x7f11d00a2d90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f11e410e4d0 tx=0x7f11d40098c0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11ebe48700 1 -- 172.21.15.44:0/4233417454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e40ff440 msgr2=0x7f11e410cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:03.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.229+0000 7f11ebe48700 1 --2- 172.21.15.44:0/4233417454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e40ff440 0x7f11e410cce0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f11e000e9a0 tx=0x7f11e000ecb0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.230+0000 7f11ebe48700 1 -- 172.21.15.44:0/4233417454 shutdown_connections 2023-09-10T00:41:03.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.230+0000 7f11ebe48700 1 --2- 172.21.15.44:0/4233417454 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f11d00a08d0 0x7f11d00a2d90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.230+0000 7f11ebe48700 1 --2- 172.21.15.44:0/4233417454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11e41054e0 0x7f11e41a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.230+0000 7f11ebe48700 1 --2- 172.21.15.44:0/4233417454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e40ff440 0x7f11e410cce0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.252 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.230+0000 7f11ebe48700 1 --2- 172.21.15.44:0/4233417454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11e40ffe00 0x7f11e410d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:03.252 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.230+0000 7f11ebe48700 1 -- 172.21.15.44:0/4233417454 >> 172.21.15.44:0/4233417454 conn(0x7f11e40faf50 msgr2=0x7f11e4102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:03.252 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.230+0000 7f11ebe48700 1 -- 172.21.15.44:0/4233417454 shutdown_connections 2023-09-10T00:41:03.252 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:03.230+0000 7f11ebe48700 1 -- 172.21.15.44:0/4233417454 wait complete. 2023-09-10T00:41:03.253 INFO:tasks.cephfs.filesystem:scrub status for tag:24dbd23a-27ba-458e-88c3-b2f208d376e7 - {'path': '/', 'tag': '24dbd23a-27ba-458e-88c3-b2f208d376e7', 'options': 'recursive,force'} 2023-09-10T00:41:03.605 INFO:teuthology.orchestra.run.smithi044.stderr:300000 of 50000000 tuples (0%) done (elapsed 3.24 s, remaining 536.63 s) 2023-09-10T00:41:04.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:03 smithi131 ceph-mon[103713]: scrub queued for path: / 2023-09-10T00:41:04.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:03 smithi131 ceph-mon[103713]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:04.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:03 smithi131 ceph-mon[103713]: scrub summary: active paths [/] 2023-09-10T00:41:04.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:03 smithi131 ceph-mon[103713]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:04.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:03 smithi131 ceph-mon[103713]: scrub summary: idle 2023-09-10T00:41:04.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:03 smithi044 ceph-mon[98676]: scrub queued for path: / 2023-09-10T00:41:04.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:03 smithi044 ceph-mon[98676]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:04.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:03 smithi044 ceph-mon[98676]: scrub summary: active paths [/] 2023-09-10T00:41:04.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:03 smithi044 ceph-mon[98676]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:04.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:03 smithi044 ceph-mon[98676]: scrub summary: idle 2023-09-10T00:41:04.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:03 smithi099 ceph-mon[102692]: scrub queued for path: / 2023-09-10T00:41:04.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:03 smithi099 ceph-mon[102692]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:04.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:03 smithi099 ceph-mon[102692]: scrub summary: active paths [/] 2023-09-10T00:41:04.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:03 smithi099 ceph-mon[102692]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:04.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:03 smithi099 ceph-mon[102692]: scrub summary: idle 2023-09-10T00:41:04.463 INFO:teuthology.orchestra.run.smithi044.stderr:400000 of 50000000 tuples (0%) done (elapsed 4.10 s, remaining 508.11 s) 2023-09-10T00:41:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:04 smithi131 ceph-mon[103713]: pgmap v15: 129 pgs: 129 active+clean; 47 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 2.0 MiB/s wr, 323 op/s 2023-09-10T00:41:05.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:04 smithi044 ceph-mon[98676]: pgmap v15: 129 pgs: 129 active+clean; 47 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 2.0 MiB/s wr, 323 op/s 2023-09-10T00:41:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:04 smithi099 ceph-mon[102692]: pgmap v15: 129 pgs: 129 active+clean; 47 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 2.0 MiB/s wr, 323 op/s 2023-09-10T00:41:05.781 INFO:teuthology.orchestra.run.smithi044.stderr:500000 of 50000000 tuples (1%) done (elapsed 5.42 s, remaining 536.14 s) 2023-09-10T00:41:06.629 INFO:teuthology.orchestra.run.smithi044.stderr:600000 of 50000000 tuples (1%) done (elapsed 6.26 s, remaining 515.71 s) 2023-09-10T00:41:07.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:06 smithi131 ceph-mon[103713]: pgmap v16: 129 pgs: 129 active+clean; 100 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 10 MiB/s wr, 392 op/s 2023-09-10T00:41:07.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:06 smithi044 ceph-mon[98676]: pgmap v16: 129 pgs: 129 active+clean; 100 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 10 MiB/s wr, 392 op/s 2023-09-10T00:41:07.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:06 smithi099 ceph-mon[102692]: pgmap v16: 129 pgs: 129 active+clean; 100 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 10 MiB/s wr, 392 op/s 2023-09-10T00:41:07.841 INFO:teuthology.orchestra.run.smithi044.stderr:700000 of 50000000 tuples (1%) done (elapsed 7.47 s, remaining 526.44 s) 2023-09-10T00:41:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.246+0000 7f526e885700 1 -- 172.21.15.44:0/2153917467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52680ff760 msgr2=0x7f52680ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.246+0000 7f526e885700 1 --2- 172.21.15.44:0/2153917467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52680ff760 0x7f52680ffbe0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f525c0099c0 tx=0x7f525c009cd0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.247+0000 7f526e885700 1 -- 172.21.15.44:0/2153917467 shutdown_connections 2023-09-10T00:41:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.247+0000 7f526e885700 1 --2- 172.21.15.44:0/2153917467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5268100120 0x7f526810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.247+0000 7f526e885700 1 --2- 172.21.15.44:0/2153917467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52681055b0 0x7f5268105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.247+0000 7f526e885700 1 --2- 172.21.15.44:0/2153917467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52680ff760 0x7f52680ffbe0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.247+0000 7f526e885700 1 -- 172.21.15.44:0/2153917467 >> 172.21.15.44:0/2153917467 conn(0x7f5268074f30 msgr2=0x7f5268075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.247+0000 7f526e885700 1 -- 172.21.15.44:0/2153917467 shutdown_connections 2023-09-10T00:41:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.247+0000 7f526e885700 1 -- 172.21.15.44:0/2153917467 wait complete. 2023-09-10T00:41:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.247+0000 7f526e885700 1 Processor -- start 2023-09-10T00:41:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.247+0000 7f526e885700 1 -- start start 2023-09-10T00:41:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.248+0000 7f526e885700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52680ff760 0x7f52681975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.248+0000 7f526e885700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5268100120 0x7f5268197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.248+0000 7f526e885700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52681055b0 0x7f52681a2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.248+0000 7f526e885700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52681a2a10 con 0x7f52680ff760 2023-09-10T00:41:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.248+0000 7f526e885700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52681a2b80 con 0x7f52681055b0 2023-09-10T00:41:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.248+0000 7f526e885700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52681a2d40 con 0x7f5268100120 2023-09-10T00:41:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.248+0000 7f5267fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52680ff760 0x7f52681975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.248+0000 7f5267fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52680ff760 0x7f52681975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49458/0 (socket says 172.21.15.44:49458) 2023-09-10T00:41:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.248+0000 7f5267fff700 1 -- 172.21.15.44:0/2936101673 learned_addr learned my addr 172.21.15.44:0/2936101673 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.248+0000 7f52677fe700 1 --2- 172.21.15.44:0/2936101673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5268100120 0x7f5268197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.248+0000 7f526ce22700 1 --2- 172.21.15.44:0/2936101673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52681055b0 0x7f52681a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.248+0000 7f5267fff700 1 -- 172.21.15.44:0/2936101673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5268100120 msgr2=0x7f5268197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.248+0000 7f5267fff700 1 --2- 172.21.15.44:0/2936101673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5268100120 0x7f5268197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.248+0000 7f5267fff700 1 -- 172.21.15.44:0/2936101673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52681055b0 msgr2=0x7f52681a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.248+0000 7f5267fff700 1 --2- 172.21.15.44:0/2936101673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52681055b0 0x7f52681a2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.248+0000 7f5267fff700 1 -- 172.21.15.44:0/2936101673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c0095f0 con 0x7f52680ff760 2023-09-10T00:41:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.249+0000 7f526ce22700 1 --2- 172.21.15.44:0/2936101673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52681055b0 0x7f52681a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.249+0000 7f5267fff700 1 --2- 172.21.15.44:0/2936101673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52680ff760 0x7f52681975a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f525800e9a0 tx=0x7f525800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.249+0000 7f52657fa700 1 -- 172.21.15.44:0/2936101673 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5258009790 con 0x7f52680ff760 2023-09-10T00:41:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.249+0000 7f526e885700 1 -- 172.21.15.44:0/2936101673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52681a2fc0 con 0x7f52680ff760 2023-09-10T00:41:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.249+0000 7f52657fa700 1 -- 172.21.15.44:0/2936101673 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f525800cbd0 con 0x7f52680ff760 2023-09-10T00:41:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.249+0000 7f52657fa700 1 -- 172.21.15.44:0/2936101673 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5258018680 con 0x7f52680ff760 2023-09-10T00:41:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.249+0000 7f526e885700 1 -- 172.21.15.44:0/2936101673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52681a3440 con 0x7f52680ff760 2023-09-10T00:41:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.252+0000 7f52657fa700 1 -- 172.21.15.44:0/2936101673 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f525800cd40 con 0x7f52680ff760 2023-09-10T00:41:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.252+0000 7f52657fa700 1 --2- 172.21.15.44:0/2936101673 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f52500a08d0 0x7f52500a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.253+0000 7f52677fe700 1 --2- 172.21.15.44:0/2936101673 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f52500a08d0 0x7f52500a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.253+0000 7f52657fa700 1 -- 172.21.15.44:0/2936101673 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f5258014070 con 0x7f52680ff760 2023-09-10T00:41:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.253+0000 7f52677fe700 1 --2- 172.21.15.44:0/2936101673 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f52500a08d0 0x7f52500a2d90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f525c00b580 tx=0x7f525c005be0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.253+0000 7f527aa4bb80 1 Processor -- start 2023-09-10T00:41:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.253+0000 7f527aa4bb80 1 -- start start 2023-09-10T00:41:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f527aa4bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf321a0 0x55adddf32580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f527aa4bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2c300 0x55adddf2c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f527aa4bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2ccc0 0x55adddf38f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f527aa4bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55addde856d0 con 0x55adddf2c300 2023-09-10T00:41:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f527aa4bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55addde82ff0 con 0x55adddf321a0 2023-09-10T00:41:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f527aa4bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55addde7e8e0 con 0x55adddf2ccc0 2023-09-10T00:41:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f52677fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2c300 0x55adddf2c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f52677fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2c300 0x55adddf2c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49468/0 (socket says 172.21.15.44:49468) 2023-09-10T00:41:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f52677fe700 1 -- 172.21.15.44:0/2685165200 learned_addr learned my addr 172.21.15.44:0/2685165200 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f526ce22700 1 --2- 172.21.15.44:0/2685165200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2ccc0 0x55adddf38f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f5267fff700 1 --2- 172.21.15.44:0/2685165200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf321a0 0x55adddf32580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f52677fe700 1 -- 172.21.15.44:0/2685165200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2ccc0 msgr2=0x55adddf38f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f52677fe700 1 --2- 172.21.15.44:0/2685165200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2ccc0 0x55adddf38f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f52677fe700 1 -- 172.21.15.44:0/2685165200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf321a0 msgr2=0x55adddf32580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f52677fe700 1 --2- 172.21.15.44:0/2685165200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf321a0 0x55adddf32580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f52677fe700 1 -- 172.21.15.44:0/2685165200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adddcaad80 con 0x55adddf2c300 2023-09-10T00:41:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f526ce22700 1 --2- 172.21.15.44:0/2685165200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2ccc0 0x55adddf38f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f52677fe700 1 --2- 172.21.15.44:0/2685165200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2c300 0x55adddf2c780 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f525c015000 tx=0x7f525c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebc2c1c4cf90f92 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.254+0000 7f524e7fc700 1 -- 172.21.15.44:0/2685165200 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c028070 con 0x55adddf2c300 2023-09-10T00:41:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.255+0000 7f524e7fc700 1 -- 172.21.15.44:0/2685165200 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f525c01e040 con 0x55adddf2c300 2023-09-10T00:41:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.255+0000 7f524e7fc700 1 -- 172.21.15.44:0/2685165200 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c00f660 con 0x55adddf2c300 2023-09-10T00:41:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.255+0000 7f527aa4bb80 1 -- 172.21.15.44:0/2685165200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2c300 msgr2=0x55adddf2c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.255+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/2685165200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2c300 0x55adddf2c780 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f525c015000 tx=0x7f525c01f000 comp rx=0 tx=0).stop 2023-09-10T00:41:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.255+0000 7f527aa4bb80 1 -- 172.21.15.44:0/2685165200 shutdown_connections 2023-09-10T00:41:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.255+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/2685165200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2ccc0 0x55adddf38f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.255+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/2685165200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf321a0 0x55adddf32580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.255+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/2685165200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2c300 0x55adddf2c780 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.255+0000 7f527aa4bb80 1 -- 172.21.15.44:0/2685165200 >> 172.21.15.44:0/2685165200 conn(0x55adddea1030 msgr2=0x55adddf27560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.255+0000 7f527aa4bb80 1 -- 172.21.15.44:0/2685165200 shutdown_connections 2023-09-10T00:41:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.255+0000 7f527aa4bb80 1 -- 172.21.15.44:0/2685165200 wait complete. 2023-09-10T00:41:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.256+0000 7f527aa4bb80 1 -- start start 2023-09-10T00:41:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.256+0000 7f527aa4bb80 1 Processor -- start 2023-09-10T00:41:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.256+0000 7f527aa4bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2c300 0x55adddf38e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.256+0000 7f527aa4bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2ccc0 0x55adddf36110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.256+0000 7f527aa4bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf321a0 0x55adddf36740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.256+0000 7f527aa4bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55addde1cb10 con 0x55adddf2c300 2023-09-10T00:41:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.256+0000 7f527aa4bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55addde1c810 con 0x55adddf321a0 2023-09-10T00:41:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.256+0000 7f527aa4bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adddd008c0 con 0x55adddf2ccc0 2023-09-10T00:41:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.256+0000 7f527aa4bb80 10 client.0 _ll_register_callbacks cb 0x55adddd66a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.256+0000 7f5267fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2c300 0x55adddf38e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.256+0000 7f5267fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2c300 0x55adddf38e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49472/0 (socket says 172.21.15.44:49472) 2023-09-10T00:41:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.257+0000 7f5267fff700 1 -- 172.21.15.44:0/3252184300 learned_addr learned my addr 172.21.15.44:0/3252184300 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.257+0000 7f52677fe700 1 --2- 172.21.15.44:0/3252184300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2ccc0 0x55adddf36110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.257+0000 7f526ce22700 1 --2- 172.21.15.44:0/3252184300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf321a0 0x55adddf36740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.257+0000 7f5267fff700 1 -- 172.21.15.44:0/3252184300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2ccc0 msgr2=0x55adddf36110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.257+0000 7f5267fff700 1 --2- 172.21.15.44:0/3252184300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2ccc0 0x55adddf36110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.257+0000 7f5267fff700 1 -- 172.21.15.44:0/3252184300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf321a0 msgr2=0x55adddf36740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.257+0000 7f5267fff700 1 --2- 172.21.15.44:0/3252184300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf321a0 0x55adddf36740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.257+0000 7f5267fff700 1 -- 172.21.15.44:0/3252184300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52600099c0 con 0x55adddf2c300 2023-09-10T00:41:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.257+0000 7f5267fff700 1 --2- 172.21.15.44:0/3252184300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2c300 0x55adddf38e20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5258096180 tx=0x7f5258099c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce8e2120701f51a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.257+0000 7f52477fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.257+0000 7f52477fe700 1 -- 172.21.15.44:0/3252184300 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525809d540 con 0x55adddf2c300 2023-09-10T00:41:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.257+0000 7f527aa4bb80 1 -- 172.21.15.44:0/3252184300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55addddd6490 con 0x55adddf2c300 2023-09-10T00:41:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.257+0000 7f52477fe700 1 -- 172.21.15.44:0/3252184300 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f52580a1780 con 0x55adddf2c300 2023-09-10T00:41:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.257+0000 7f52477fe700 1 -- 172.21.15.44:0/3252184300 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f525809d160 con 0x55adddf2c300 2023-09-10T00:41:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.257+0000 7f527aa4bb80 10 client.16077 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.257+0000 7f527aa4bb80 1 -- 172.21.15.44:0/3252184300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f526810cbc0 con 0x55adddf2c300 2023-09-10T00:41:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.258+0000 7f52477fe700 1 -- 172.21.15.44:0/3252184300 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55addddd6490 con 0x55adddf2c300 2023-09-10T00:41:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.258+0000 7f52477fe700 1 -- 172.21.15.44:0/3252184300 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f52580b6020 con 0x55adddf2c300 2023-09-10T00:41:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.258+0000 7f527aa4bb80 10 client.16077 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.258+0000 7f527aa4bb80 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.258+0000 7f527aa4bb80 20 client.16077 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.258+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/3252184300 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55adddf46370 0x55adddf48760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.258+0000 7f527aa4bb80 4 client.16077 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.258+0000 7f527aa4bb80 1 -- 172.21.15.44:0/3252184300 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55addde93840 con 0x55adddf46370 2023-09-10T00:41:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.258+0000 7f52677fe700 1 --2- 172.21.15.44:0/3252184300 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55adddf46370 0x55adddf48760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.258+0000 7f52677fe700 1 --2- 172.21.15.44:0/3252184300 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55adddf46370 0x55adddf48760 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c58df5f03fe64281 server_cookie=be0e9dce6afd1955 in_seq=0 out_seq=0 2023-09-10T00:41:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.258+0000 7f52477fe700 10 client.16077 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.259+0000 7f52477fe700 1 -- 172.21.15.44:0/3252184300 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55addde93840 con 0x55adddf46370 2023-09-10T00:41:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.259+0000 7f52477fe700 10 client.16077 handle_command_reply: tid=0 2023-09-10T00:41:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.259+0000 7f527aa4bb80 1 client.16077 shutdown 2023-09-10T00:41:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.259+0000 7f527aa4bb80 1 -- 172.21.15.44:0/3252184300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2c300 msgr2=0x55adddf38e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.259+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/3252184300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2c300 0x55adddf38e20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5258096180 tx=0x7f5258099c50 comp rx=0 tx=0).stop 2023-09-10T00:41:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.259+0000 7f527aa4bb80 1 -- 172.21.15.44:0/3252184300 shutdown_connections 2023-09-10T00:41:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.259+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/3252184300 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55adddf46370 0x55adddf48760 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.259+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/3252184300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf321a0 0x55adddf36740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.259+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/3252184300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2c300 0x55adddf38e20 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.259+0000 7f52477fe700 0 client.16077 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.259+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/3252184300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2ccc0 0x55adddf36110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.259+0000 7f527aa4bb80 1 -- 172.21.15.44:0/3252184300 >> 172.21.15.44:0/3252184300 conn(0x55adddea1030 msgr2=0x55adddf267a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.259+0000 7f527aa4bb80 1 -- 172.21.15.44:0/3252184300 shutdown_connections 2023-09-10T00:41:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.260+0000 7f527aa4bb80 1 -- 172.21.15.44:0/3252184300 wait complete. 2023-09-10T00:41:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.260+0000 7f527aa4bb80 20 client.16077 trim_cache size 0 max 16384 2023-09-10T00:41:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.275+0000 7f527aa4bb80 1 Processor -- start 2023-09-10T00:41:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.275+0000 7f527aa4bb80 1 -- start start 2023-09-10T00:41:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f527aa4bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2c300 0x55adddf389c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f527aa4bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2ccc0 0x55adddf27090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f527aa4bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf321a0 0x55adddf275d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f527aa4bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adddf27b10 con 0x55adddf2ccc0 2023-09-10T00:41:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f527aa4bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55addde93840 con 0x55adddf321a0 2023-09-10T00:41:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f527aa4bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adddf28390 con 0x55adddf2c300 2023-09-10T00:41:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f52677fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2ccc0 0x55adddf27090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f52677fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2ccc0 0x55adddf27090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49482/0 (socket says 172.21.15.44:49482) 2023-09-10T00:41:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f52677fe700 1 -- 172.21.15.44:0/607899753 learned_addr learned my addr 172.21.15.44:0/607899753 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f526ce22700 1 --2- 172.21.15.44:0/607899753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf321a0 0x55adddf275d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f5267fff700 1 --2- 172.21.15.44:0/607899753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2c300 0x55adddf389c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f52677fe700 1 -- 172.21.15.44:0/607899753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2c300 msgr2=0x55adddf389c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f52677fe700 1 --2- 172.21.15.44:0/607899753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2c300 0x55adddf389c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f52677fe700 1 -- 172.21.15.44:0/607899753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf321a0 msgr2=0x55adddf275d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f52677fe700 1 --2- 172.21.15.44:0/607899753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf321a0 0x55adddf275d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f52677fe700 1 -- 172.21.15.44:0/607899753 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adddc93c00 con 0x55adddf2ccc0 2023-09-10T00:41:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.276+0000 7f52677fe700 1 --2- 172.21.15.44:0/607899753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2ccc0 0x55adddf27090 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f525c003ea0 tx=0x7f525c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55a6b823129a270 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.277+0000 7f524effd700 1 -- 172.21.15.44:0/607899753 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c028040 con 0x55adddf2ccc0 2023-09-10T00:41:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.277+0000 7f524effd700 1 -- 172.21.15.44:0/607899753 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f525c029040 con 0x55adddf2ccc0 2023-09-10T00:41:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.277+0000 7f524effd700 1 -- 172.21.15.44:0/607899753 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c0373d0 con 0x55adddf2ccc0 2023-09-10T00:41:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.277+0000 7f527aa4bb80 1 -- 172.21.15.44:0/607899753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2ccc0 msgr2=0x55adddf27090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.277+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/607899753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2ccc0 0x55adddf27090 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f525c003ea0 tx=0x7f525c009cd0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.278+0000 7f527aa4bb80 1 -- 172.21.15.44:0/607899753 shutdown_connections 2023-09-10T00:41:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.278+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/607899753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf321a0 0x55adddf275d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.278+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/607899753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2c300 0x55adddf389c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.278+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/607899753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf2ccc0 0x55adddf27090 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.278+0000 7f527aa4bb80 1 -- 172.21.15.44:0/607899753 >> 172.21.15.44:0/607899753 conn(0x55adddea1030 msgr2=0x55adddf301f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.278+0000 7f527aa4bb80 1 -- 172.21.15.44:0/607899753 shutdown_connections 2023-09-10T00:41:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.278+0000 7f527aa4bb80 1 -- 172.21.15.44:0/607899753 wait complete. 2023-09-10T00:41:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.278+0000 7f527aa4bb80 1 -- start start 2023-09-10T00:41:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.278+0000 7f527aa4bb80 1 Processor -- start 2023-09-10T00:41:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.278+0000 7f527aa4bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2c300 0x55adddf3d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.278+0000 7f527aa4bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf2ccc0 0x55adddf3b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.278+0000 7f527aa4bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf321a0 0x55adddf3b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.278+0000 7f527aa4bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adddf3be90 con 0x55adddf321a0 2023-09-10T00:41:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.278+0000 7f527aa4bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adddf3c000 con 0x55adddf2ccc0 2023-09-10T00:41:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.278+0000 7f527aa4bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adddf3c1c0 con 0x55adddf2c300 2023-09-10T00:41:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.279+0000 7f527aa4bb80 10 client.0 _ll_register_callbacks cb 0x7f5258099320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.279+0000 7f526ce22700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf321a0 0x55adddf3b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.279+0000 7f526ce22700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf321a0 0x55adddf3b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49490/0 (socket says 172.21.15.44:49490) 2023-09-10T00:41:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.279+0000 7f526ce22700 1 -- 172.21.15.44:0/1991572511 learned_addr learned my addr 172.21.15.44:0/1991572511 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.279+0000 7f5267fff700 1 --2- 172.21.15.44:0/1991572511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2c300 0x55adddf3d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.279+0000 7f52677fe700 1 --2- 172.21.15.44:0/1991572511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf2ccc0 0x55adddf3b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.279+0000 7f526ce22700 1 -- 172.21.15.44:0/1991572511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2c300 msgr2=0x55adddf3d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.279+0000 7f526ce22700 1 --2- 172.21.15.44:0/1991572511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2c300 0x55adddf3d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.279+0000 7f526ce22700 1 -- 172.21.15.44:0/1991572511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf2ccc0 msgr2=0x55adddf3b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.279+0000 7f526ce22700 1 --2- 172.21.15.44:0/1991572511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf2ccc0 0x55adddf3b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.279+0000 7f526ce22700 1 -- 172.21.15.44:0/1991572511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55adddcaad80 con 0x55adddf321a0 2023-09-10T00:41:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.279+0000 7f526ce22700 1 --2- 172.21.15.44:0/1991572511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf321a0 0x55adddf3b950 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5260018ef0 tx=0x7f5260018f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f376de3e341d194 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.279+0000 7f52477fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.279+0000 7f52477fe700 1 -- 172.21.15.44:0/1991572511 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260007dd0 con 0x55adddf321a0 2023-09-10T00:41:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.279+0000 7f52477fe700 1 -- 172.21.15.44:0/1991572511 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f52600203f0 con 0x55adddf321a0 2023-09-10T00:41:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.279+0000 7f527aa4bb80 1 -- 172.21.15.44:0/1991572511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f525c01e040 con 0x55adddf321a0 2023-09-10T00:41:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.280+0000 7f52477fe700 1 -- 172.21.15.44:0/1991572511 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52600205f0 con 0x55adddf321a0 2023-09-10T00:41:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.280+0000 7f527aa4bb80 10 client.16083 fetch_fsmap learned FSMap version 27 2023-09-10T00:41:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.280+0000 7f527aa4bb80 1 -- 172.21.15.44:0/1991572511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55adddcd11f0 con 0x55adddf321a0 2023-09-10T00:41:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.280+0000 7f52477fe700 1 -- 172.21.15.44:0/1991572511 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f525c01e040 con 0x55adddf321a0 2023-09-10T00:41:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.280+0000 7f52477fe700 1 -- 172.21.15.44:0/1991572511 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5260015420 con 0x55adddf321a0 2023-09-10T00:41:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.280+0000 7f527aa4bb80 10 client.16083 fetch_fsmap finished waiting for FSMap version 27 2023-09-10T00:41:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.280+0000 7f527aa4bb80 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.280+0000 7f527aa4bb80 20 client.16083 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.280+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/1991572511 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55adddf9b5b0 0x55adddf9d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.280+0000 7f527aa4bb80 4 client.16083 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:41:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.280+0000 7f527aa4bb80 1 -- 172.21.15.44:0/1991572511 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55addde778c0 con 0x55adddf9b5b0 2023-09-10T00:41:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.280+0000 7f5267fff700 1 --2- 172.21.15.44:0/1991572511 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55adddf9b5b0 0x55adddf9d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.281+0000 7f52477fe700 10 client.16083 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.281+0000 7f5267fff700 1 --2- 172.21.15.44:0/1991572511 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55adddf9b5b0 0x55adddf9d9d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=628dd38a1040713a server_cookie=fd826551818dd43b in_seq=0 out_seq=0 2023-09-10T00:41:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.375+0000 7f52477fe700 1 -- 172.21.15.44:0/1991572511 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55addde778c0 con 0x55adddf9b5b0 2023-09-10T00:41:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.375+0000 7f52477fe700 10 client.16083 handle_command_reply: tid=0 2023-09-10T00:41:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.375+0000 7f527aa4bb80 1 client.16083 shutdown 2023-09-10T00:41:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.375+0000 7f527aa4bb80 1 -- 172.21.15.44:0/1991572511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf321a0 msgr2=0x55adddf3b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.375+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/1991572511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf321a0 0x55adddf3b950 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5260018ef0 tx=0x7f5260018f20 comp rx=0 tx=0).stop 2023-09-10T00:41:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.375+0000 7f527aa4bb80 1 -- 172.21.15.44:0/1991572511 shutdown_connections 2023-09-10T00:41:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.375+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/1991572511 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55adddf9b5b0 0x55adddf9d9d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.375+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/1991572511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adddf321a0 0x55adddf3b950 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.375+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/1991572511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adddf2c300 0x55adddf3d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.375+0000 7f527aa4bb80 1 --2- 172.21.15.44:0/1991572511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adddf2ccc0 0x55adddf3b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.375+0000 7f527aa4bb80 1 -- 172.21.15.44:0/1991572511 >> 172.21.15.44:0/1991572511 conn(0x55adddea1030 msgr2=0x55adddf27090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.375+0000 7f52477fe700 0 client.16083 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.375+0000 7f527aa4bb80 1 -- 172.21.15.44:0/1991572511 shutdown_connections 2023-09-10T00:41:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.375+0000 7f527aa4bb80 1 -- 172.21.15.44:0/1991572511 wait complete. 2023-09-10T00:41:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.375+0000 7f527aa4bb80 20 client.16083 trim_cache size 0 max 16384 2023-09-10T00:41:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:41:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:41:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:41:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:41:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.376+0000 7f526e885700 1 -- 172.21.15.44:0/2936101673 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f52500a08d0 msgr2=0x7f52500a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.376+0000 7f526e885700 1 --2- 172.21.15.44:0/2936101673 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f52500a08d0 0x7f52500a2d90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f525c00b580 tx=0x7f525c005be0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.376+0000 7f526e885700 1 -- 172.21.15.44:0/2936101673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52680ff760 msgr2=0x7f52681975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.376+0000 7f526e885700 1 --2- 172.21.15.44:0/2936101673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52680ff760 0x7f52681975a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f525800e9a0 tx=0x7f525800ecb0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.376+0000 7f526e885700 1 -- 172.21.15.44:0/2936101673 shutdown_connections 2023-09-10T00:41:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.376+0000 7f526e885700 1 --2- 172.21.15.44:0/2936101673 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f52500a08d0 0x7f52500a2d90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.376+0000 7f526e885700 1 --2- 172.21.15.44:0/2936101673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52681055b0 0x7f52681a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.376+0000 7f526e885700 1 --2- 172.21.15.44:0/2936101673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52680ff760 0x7f52681975a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.376+0000 7f526e885700 1 --2- 172.21.15.44:0/2936101673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5268100120 0x7f5268197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.376+0000 7f526e885700 1 -- 172.21.15.44:0/2936101673 >> 172.21.15.44:0/2936101673 conn(0x7f5268074f30 msgr2=0x7f52680fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.376+0000 7f526e885700 1 -- 172.21.15.44:0/2936101673 shutdown_connections 2023-09-10T00:41:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:08.376+0000 7f526e885700 1 -- 172.21.15.44:0/2936101673 wait complete. 2023-09-10T00:41:09.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:08 smithi131 ceph-mon[103713]: pgmap v17: 129 pgs: 129 active+clean; 150 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 16 MiB/s wr, 380 op/s 2023-09-10T00:41:09.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:08 smithi044 ceph-mon[98676]: pgmap v17: 129 pgs: 129 active+clean; 150 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 16 MiB/s wr, 380 op/s 2023-09-10T00:41:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:08 smithi099 ceph-mon[102692]: pgmap v17: 129 pgs: 129 active+clean; 150 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 16 MiB/s wr, 380 op/s 2023-09-10T00:41:09.151 INFO:teuthology.orchestra.run.smithi044.stderr:800000 of 50000000 tuples (1%) done (elapsed 8.79 s, remaining 540.31 s) 2023-09-10T00:41:10.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:09 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:10.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:09 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:10.056 INFO:teuthology.orchestra.run.smithi044.stderr:900000 of 50000000 tuples (1%) done (elapsed 9.69 s, remaining 528.68 s) 2023-09-10T00:41:10.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:09 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:10 smithi131 ceph-mon[103713]: pgmap v18: 129 pgs: 129 active+clean; 150 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 15 MiB/s wr, 234 op/s 2023-09-10T00:41:11.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:10 smithi044 ceph-mon[98676]: pgmap v18: 129 pgs: 129 active+clean; 150 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 15 MiB/s wr, 234 op/s 2023-09-10T00:41:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:10 smithi099 ceph-mon[102692]: pgmap v18: 129 pgs: 129 active+clean; 150 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 15 MiB/s wr, 234 op/s 2023-09-10T00:41:11.349 INFO:teuthology.orchestra.run.smithi044.stderr:1000000 of 50000000 tuples (2%) done (elapsed 10.98 s, remaining 538.18 s) 2023-09-10T00:41:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:1100000 of 50000000 tuples (2%) done (elapsed 12.19 s, remaining 541.90 s) 2023-09-10T00:41:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:12 smithi131 ceph-mon[103713]: pgmap v19: 129 pgs: 129 active+clean; 247 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 33 MiB/s wr, 303 op/s 2023-09-10T00:41:13.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:12 smithi044 ceph-mon[98676]: pgmap v19: 129 pgs: 129 active+clean; 247 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 33 MiB/s wr, 303 op/s 2023-09-10T00:41:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:12 smithi099 ceph-mon[102692]: pgmap v19: 129 pgs: 129 active+clean; 247 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 33 MiB/s wr, 303 op/s 2023-09-10T00:41:13.428 INFO:teuthology.orchestra.run.smithi044.stderr:1200000 of 50000000 tuples (2%) done (elapsed 13.06 s, remaining 531.17 s) 2023-09-10T00:41:14.808 INFO:teuthology.orchestra.run.smithi044.stderr:1300000 of 50000000 tuples (2%) done (elapsed 14.44 s, remaining 541.05 s) 2023-09-10T00:41:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:14 smithi131 ceph-mon[103713]: pgmap v20: 129 pgs: 129 active+clean; 247 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 32 MiB/s wr, 170 op/s 2023-09-10T00:41:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:14 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:15.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:14 smithi044 ceph-mon[98676]: pgmap v20: 129 pgs: 129 active+clean; 247 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 32 MiB/s wr, 170 op/s 2023-09-10T00:41:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:14 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:14 smithi099 ceph-mon[102692]: pgmap v20: 129 pgs: 129 active+clean; 247 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 32 MiB/s wr, 170 op/s 2023-09-10T00:41:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:14 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:16.066 INFO:teuthology.orchestra.run.smithi044.stderr:1400000 of 50000000 tuples (2%) done (elapsed 15.70 s, remaining 545.03 s) 2023-09-10T00:41:16.926 INFO:teuthology.orchestra.run.smithi044.stderr:1500000 of 50000000 tuples (3%) done (elapsed 16.56 s, remaining 535.43 s) 2023-09-10T00:41:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:16 smithi131 ceph-mon[103713]: pgmap v21: 129 pgs: 129 active+clean; 311 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 42 MiB/s wr, 181 op/s 2023-09-10T00:41:17.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:16 smithi044 ceph-mon[98676]: pgmap v21: 129 pgs: 129 active+clean; 311 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 42 MiB/s wr, 181 op/s 2023-09-10T00:41:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:16 smithi099 ceph-mon[102692]: pgmap v21: 129 pgs: 129 active+clean; 311 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 42 MiB/s wr, 181 op/s 2023-09-10T00:41:18.238 INFO:teuthology.orchestra.run.smithi044.stderr:1600000 of 50000000 tuples (3%) done (elapsed 17.87 s, remaining 540.63 s) 2023-09-10T00:41:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:18 smithi131 ceph-mon[103713]: pgmap v22: 129 pgs: 129 active+clean; 347 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 40 MiB/s wr, 117 op/s 2023-09-10T00:41:19.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:18 smithi044 ceph-mon[98676]: pgmap v22: 129 pgs: 129 active+clean; 347 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 40 MiB/s wr, 117 op/s 2023-09-10T00:41:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:18 smithi099 ceph-mon[102692]: pgmap v22: 129 pgs: 129 active+clean; 347 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 40 MiB/s wr, 117 op/s 2023-09-10T00:41:19.152 INFO:teuthology.orchestra.run.smithi044.stderr:1700000 of 50000000 tuples (3%) done (elapsed 18.79 s, remaining 533.76 s) 2023-09-10T00:41:20.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:19 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:20.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:19 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:20.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:19 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:20.318 INFO:teuthology.orchestra.run.smithi044.stderr:1800000 of 50000000 tuples (3%) done (elapsed 19.95 s, remaining 534.29 s) 2023-09-10T00:41:21.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:20 smithi131 ceph-mon[103713]: pgmap v23: 129 pgs: 129 active+clean; 347 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 34 MiB/s wr, 84 op/s 2023-09-10T00:41:21.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:20 smithi044 ceph-mon[98676]: pgmap v23: 129 pgs: 129 active+clean; 347 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 34 MiB/s wr, 84 op/s 2023-09-10T00:41:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:20 smithi099 ceph-mon[102692]: pgmap v23: 129 pgs: 129 active+clean; 347 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 34 MiB/s wr, 84 op/s 2023-09-10T00:41:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.512+0000 7f759d9d9700 1 -- 172.21.15.44:0/1427742522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7598106670 msgr2=0x7f7598106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.512+0000 7f759d9d9700 1 --2- 172.21.15.44:0/1427742522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7598106670 0x7f7598106a50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f75880099c0 tx=0x7f7588009cd0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.512+0000 7f759d9d9700 1 -- 172.21.15.44:0/1427742522 shutdown_connections 2023-09-10T00:41:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.512+0000 7f759d9d9700 1 --2- 172.21.15.44:0/1427742522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7598100f70 0x7f759810d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.512+0000 7f759d9d9700 1 --2- 172.21.15.44:0/1427742522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7598106670 0x7f7598106a50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.512+0000 7f759d9d9700 1 --2- 172.21.15.44:0/1427742522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75981005b0 0x7f7598100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.512+0000 7f759d9d9700 1 -- 172.21.15.44:0/1427742522 >> 172.21.15.44:0/1427742522 conn(0x7f75980fc060 msgr2=0x7f75980fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.512+0000 7f759d9d9700 1 -- 172.21.15.44:0/1427742522 shutdown_connections 2023-09-10T00:41:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.512+0000 7f759d9d9700 1 -- 172.21.15.44:0/1427742522 wait complete. 2023-09-10T00:41:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.513+0000 7f759d9d9700 1 Processor -- start 2023-09-10T00:41:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.513+0000 7f759d9d9700 1 -- start start 2023-09-10T00:41:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.513+0000 7f759d9d9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75981005b0 0x7f759819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.513+0000 7f759d9d9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7598100f70 0x7f759819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.513+0000 7f759d9d9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7598106670 0x7f75981963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.513+0000 7f759d9d9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7598196a30 con 0x7f7598106670 2023-09-10T00:41:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.513+0000 7f759d9d9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7598196ba0 con 0x7f75981005b0 2023-09-10T00:41:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.513+0000 7f759d9d9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7598196d60 con 0x7f7598100f70 2023-09-10T00:41:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.513+0000 7f75977fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7598106670 0x7f75981963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.513+0000 7f75977fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7598106670 0x7f75981963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49504/0 (socket says 172.21.15.44:49504) 2023-09-10T00:41:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.513+0000 7f75977fe700 1 -- 172.21.15.44:0/549431872 learned_addr learned my addr 172.21.15.44:0/549431872 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.513+0000 7f7596ffd700 1 --2- 172.21.15.44:0/549431872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75981005b0 0x7f759819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.513+0000 7f75967fc700 1 --2- 172.21.15.44:0/549431872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7598100f70 0x7f759819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.514+0000 7f75977fe700 1 -- 172.21.15.44:0/549431872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7598100f70 msgr2=0x7f759819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.514+0000 7f75977fe700 1 --2- 172.21.15.44:0/549431872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7598100f70 0x7f759819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.514+0000 7f75977fe700 1 -- 172.21.15.44:0/549431872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75981005b0 msgr2=0x7f759819c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.514+0000 7f75977fe700 1 --2- 172.21.15.44:0/549431872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75981005b0 0x7f759819c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.514+0000 7f75977fe700 1 -- 172.21.15.44:0/549431872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75880095f0 con 0x7f7598106670 2023-09-10T00:41:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.514+0000 7f75977fe700 1 --2- 172.21.15.44:0/549431872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7598106670 0x7f75981963c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7590007ee0 tx=0x7f759000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.514+0000 7f759c9d7700 1 -- 172.21.15.44:0/549431872 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590017070 con 0x7f7598106670 2023-09-10T00:41:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.514+0000 7f759d9d9700 1 -- 172.21.15.44:0/549431872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7598197040 con 0x7f7598106670 2023-09-10T00:41:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.514+0000 7f759c9d7700 1 -- 172.21.15.44:0/549431872 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f759000f040 con 0x7f7598106670 2023-09-10T00:41:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.514+0000 7f759c9d7700 1 -- 172.21.15.44:0/549431872 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590005570 con 0x7f7598106670 2023-09-10T00:41:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.514+0000 7f759d9d9700 1 -- 172.21.15.44:0/549431872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75981a3550 con 0x7f7598106670 2023-09-10T00:41:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.516+0000 7f759c9d7700 1 -- 172.21.15.44:0/549431872 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f759000db20 con 0x7f7598106670 2023-09-10T00:41:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.517+0000 7f759c9d7700 1 --2- 172.21.15.44:0/549431872 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f75800a08d0 0x7f75800a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.517+0000 7f7596ffd700 1 --2- 172.21.15.44:0/549431872 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f75800a08d0 0x7f75800a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.517+0000 7f759c9d7700 1 -- 172.21.15.44:0/549431872 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f75900cbd60 con 0x7f7598106670 2023-09-10T00:41:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.518+0000 7f7596ffd700 1 --2- 172.21.15.44:0/549431872 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f75800a08d0 0x7f75800a2d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f758800b580 tx=0x7f7588005be0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.518+0000 7f75a9b9fb80 1 Processor -- start 2023-09-10T00:41:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.518+0000 7f75a9b9fb80 1 -- start start 2023-09-10T00:41:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.518+0000 7f75a9b9fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c29792a1a0 0x55c29792a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.518+0000 7f75a9b9fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924300 0x55c297924780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.518+0000 7f75a9b9fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924cc0 0x55c297930f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.518+0000 7f75a9b9fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c29787d6d0 con 0x55c297924300 2023-09-10T00:41:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.518+0000 7f75a9b9fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c29787aff0 con 0x55c29792a1a0 2023-09-10T00:41:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.518+0000 7f75a9b9fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2978768e0 con 0x55c297924cc0 2023-09-10T00:41:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.518+0000 7f75967fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924300 0x55c297924780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.518+0000 7f75967fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924300 0x55c297924780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49510/0 (socket says 172.21.15.44:49510) 2023-09-10T00:41:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.518+0000 7f75967fc700 1 -- 172.21.15.44:0/839637351 learned_addr learned my addr 172.21.15.44:0/839637351 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.518+0000 7f7596ffd700 1 --2- 172.21.15.44:0/839637351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c29792a1a0 0x55c29792a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.519+0000 7f75977fe700 1 --2- 172.21.15.44:0/839637351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924cc0 0x55c297930f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.519+0000 7f75967fc700 1 -- 172.21.15.44:0/839637351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924cc0 msgr2=0x55c297930f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.519+0000 7f75967fc700 1 --2- 172.21.15.44:0/839637351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924cc0 0x55c297930f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.519+0000 7f75967fc700 1 -- 172.21.15.44:0/839637351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c29792a1a0 msgr2=0x55c29792a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.519+0000 7f75967fc700 1 --2- 172.21.15.44:0/839637351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c29792a1a0 0x55c29792a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.519+0000 7f75967fc700 1 -- 172.21.15.44:0/839637351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2976a2d80 con 0x55c297924300 2023-09-10T00:41:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.519+0000 7f75977fe700 1 --2- 172.21.15.44:0/839637351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924cc0 0x55c297930f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.519+0000 7f75967fc700 1 --2- 172.21.15.44:0/839637351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924300 0x55c297924780 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f758c009d20 tx=0x7f758c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0119b5b479fd59 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.519+0000 7f757dffb700 1 -- 172.21.15.44:0/839637351 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c019630 con 0x55c297924300 2023-09-10T00:41:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.519+0000 7f757dffb700 1 -- 172.21.15.44:0/839637351 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f758c019c20 con 0x55c297924300 2023-09-10T00:41:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.519+0000 7f757dffb700 1 -- 172.21.15.44:0/839637351 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c00eab0 con 0x55c297924300 2023-09-10T00:41:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.520+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/839637351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924300 msgr2=0x55c297924780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.520+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/839637351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924300 0x55c297924780 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f758c009d20 tx=0x7f758c011720 comp rx=0 tx=0).stop 2023-09-10T00:41:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.520+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/839637351 shutdown_connections 2023-09-10T00:41:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.520+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/839637351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924cc0 0x55c297930f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.520+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/839637351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c29792a1a0 0x55c29792a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.520+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/839637351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924300 0x55c297924780 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.520+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/839637351 >> 172.21.15.44:0/839637351 conn(0x55c297899030 msgr2=0x55c29791f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.520+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/839637351 shutdown_connections 2023-09-10T00:41:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.520+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/839637351 wait complete. 2023-09-10T00:41:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.520+0000 7f75a9b9fb80 1 -- start start 2023-09-10T00:41:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f75a9b9fb80 1 Processor -- start 2023-09-10T00:41:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f75a9b9fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924300 0x55c297930da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f75a9b9fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924cc0 0x55c29792e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f75a9b9fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c29792a1a0 0x55c29792e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f75a9b9fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c297814b10 con 0x55c297924cc0 2023-09-10T00:41:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f75a9b9fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c297814810 con 0x55c29792a1a0 2023-09-10T00:41:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f75a9b9fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2976f88c0 con 0x55c297924300 2023-09-10T00:41:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f75a9b9fb80 10 client.0 _ll_register_callbacks cb 0x55c29775ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f75967fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924cc0 0x55c29792e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f75967fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924cc0 0x55c29792e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49516/0 (socket says 172.21.15.44:49516) 2023-09-10T00:41:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f75967fc700 1 -- 172.21.15.44:0/1279271632 learned_addr learned my addr 172.21.15.44:0/1279271632 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f7596ffd700 1 --2- 172.21.15.44:0/1279271632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924300 0x55c297930da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f75977fe700 1 --2- 172.21.15.44:0/1279271632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c29792a1a0 0x55c29792e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f75967fc700 1 -- 172.21.15.44:0/1279271632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924300 msgr2=0x55c297930da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f75967fc700 1 --2- 172.21.15.44:0/1279271632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924300 0x55c297930da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f75967fc700 1 -- 172.21.15.44:0/1279271632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c29792a1a0 msgr2=0x55c29792e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.521+0000 7f75967fc700 1 --2- 172.21.15.44:0/1279271632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c29792a1a0 0x55c29792e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.522+0000 7f75967fc700 1 -- 172.21.15.44:0/1279271632 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c009980 con 0x55c297924cc0 2023-09-10T00:41:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.522+0000 7f75967fc700 1 --2- 172.21.15.44:0/1279271632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924cc0 0x55c29792e090 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f758c01c040 tx=0x7f758c005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c35416007f3de9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.522+0000 7f7576ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.522+0000 7f7576ffd700 1 -- 172.21.15.44:0/1279271632 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c004230 con 0x55c297924cc0 2023-09-10T00:41:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.522+0000 7f7576ffd700 1 -- 172.21.15.44:0/1279271632 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f758c004390 con 0x55c297924cc0 2023-09-10T00:41:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.522+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/1279271632 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2977ce490 con 0x55c297924cc0 2023-09-10T00:41:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.522+0000 7f7576ffd700 1 -- 172.21.15.44:0/1279271632 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f758c004550 con 0x55c297924cc0 2023-09-10T00:41:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.522+0000 7f75a9b9fb80 10 client.16095 fetch_fsmap learned FSMap version 30 2023-09-10T00:41:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.522+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/1279271632 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c29768bc00 con 0x55c297924cc0 2023-09-10T00:41:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.522+0000 7f7576ffd700 1 -- 172.21.15.44:0/1279271632 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c2977ce490 con 0x55c297924cc0 2023-09-10T00:41:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.523+0000 7f7576ffd700 1 -- 172.21.15.44:0/1279271632 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f758c004bd0 con 0x55c297924cc0 2023-09-10T00:41:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.523+0000 7f75a9b9fb80 10 client.16095 fetch_fsmap finished waiting for FSMap version 30 2023-09-10T00:41:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.523+0000 7f75a9b9fb80 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.523+0000 7f75a9b9fb80 20 client.16095 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.523+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/1279271632 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c29793e390 0x55c2979407b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.523+0000 7f75a9b9fb80 4 client.16095 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.523+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/1279271632 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c29788b840 con 0x55c29793e390 2023-09-10T00:41:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.523+0000 7f7596ffd700 1 --2- 172.21.15.44:0/1279271632 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c29793e390 0x55c2979407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.523+0000 7f7596ffd700 1 --2- 172.21.15.44:0/1279271632 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c29793e390 0x55c2979407b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db8d971eeb45fda8 server_cookie=3b5f646079f12598 in_seq=0 out_seq=0 2023-09-10T00:41:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.523+0000 7f7576ffd700 10 client.16095 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.524+0000 7f7576ffd700 1 -- 172.21.15.44:0/1279271632 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c29788b840 con 0x55c29793e390 2023-09-10T00:41:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.524+0000 7f7576ffd700 10 client.16095 handle_command_reply: tid=0 2023-09-10T00:41:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.524+0000 7f75a9b9fb80 1 client.16095 shutdown 2023-09-10T00:41:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.524+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/1279271632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924cc0 msgr2=0x55c29792e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.524+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/1279271632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924cc0 0x55c29792e090 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f758c01c040 tx=0x7f758c005d60 comp rx=0 tx=0).stop 2023-09-10T00:41:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.524+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/1279271632 shutdown_connections 2023-09-10T00:41:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.524+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/1279271632 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c29793e390 0x55c2979407b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.524+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/1279271632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c29792a1a0 0x55c29792e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.524+0000 7f7576ffd700 0 client.16095 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.524+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/1279271632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924300 0x55c297930da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.524+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/1279271632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924cc0 0x55c29792e090 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.524+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/1279271632 >> 172.21.15.44:0/1279271632 conn(0x55c297899030 msgr2=0x55c29791e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.524+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/1279271632 shutdown_connections 2023-09-10T00:41:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.524+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/1279271632 wait complete. 2023-09-10T00:41:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.525+0000 7f75a9b9fb80 20 client.16095 trim_cache size 0 max 16384 2023-09-10T00:41:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.543+0000 7f75a9b9fb80 1 Processor -- start 2023-09-10T00:41:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75a9b9fb80 1 -- start start 2023-09-10T00:41:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75a9b9fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c297924300 0x55c297928190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75a9b9fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924cc0 0x55c2979286d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75a9b9fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c29792a1a0 0x55c297929110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75a9b9fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2977ce490 con 0x55c297924cc0 2023-09-10T00:41:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75a9b9fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c297920390 con 0x55c297924300 2023-09-10T00:41:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75a9b9fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c29793a390 con 0x55c29792a1a0 2023-09-10T00:41:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75967fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924cc0 0x55c2979286d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75967fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924cc0 0x55c2979286d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49524/0 (socket says 172.21.15.44:49524) 2023-09-10T00:41:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75967fc700 1 -- 172.21.15.44:0/4019827568 learned_addr learned my addr 172.21.15.44:0/4019827568 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f7596ffd700 1 --2- 172.21.15.44:0/4019827568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c297924300 0x55c297928190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75977fe700 1 --2- 172.21.15.44:0/4019827568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c29792a1a0 0x55c297929110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75967fc700 1 -- 172.21.15.44:0/4019827568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c29792a1a0 msgr2=0x55c297929110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75967fc700 1 --2- 172.21.15.44:0/4019827568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c29792a1a0 0x55c297929110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75967fc700 1 -- 172.21.15.44:0/4019827568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c297924300 msgr2=0x55c297928190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75967fc700 1 --2- 172.21.15.44:0/4019827568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c297924300 0x55c297928190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75967fc700 1 -- 172.21.15.44:0/4019827568 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2976c91f0 con 0x55c297924cc0 2023-09-10T00:41:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75977fe700 1 --2- 172.21.15.44:0/4019827568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c29792a1a0 0x55c297929110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f7596ffd700 1 --2- 172.21.15.44:0/4019827568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c297924300 0x55c297928190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.544+0000 7f75967fc700 1 --2- 172.21.15.44:0/4019827568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924cc0 0x55c2979286d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f758c009f50 tx=0x7f758c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a69947ce982a8f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.545+0000 7f757e7fc700 1 -- 172.21.15.44:0/4019827568 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c021e30 con 0x55c297924cc0 2023-09-10T00:41:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.545+0000 7f757e7fc700 1 -- 172.21.15.44:0/4019827568 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f758c005040 con 0x55c297924cc0 2023-09-10T00:41:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.545+0000 7f757e7fc700 1 -- 172.21.15.44:0/4019827568 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c021580 con 0x55c297924cc0 2023-09-10T00:41:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.546+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/4019827568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924cc0 msgr2=0x55c2979286d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.546+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/4019827568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924cc0 0x55c2979286d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f758c009f50 tx=0x7f758c011720 comp rx=0 tx=0).stop 2023-09-10T00:41:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.546+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/4019827568 shutdown_connections 2023-09-10T00:41:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.546+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/4019827568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c29792a1a0 0x55c297929110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.546+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/4019827568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c297924300 0x55c297928190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.546+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/4019827568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c297924cc0 0x55c2979286d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.546+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/4019827568 >> 172.21.15.44:0/4019827568 conn(0x55c297899030 msgr2=0x55c297939c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.546+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/4019827568 shutdown_connections 2023-09-10T00:41:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.546+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/4019827568 wait complete. 2023-09-10T00:41:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.546+0000 7f75a9b9fb80 1 -- start start 2023-09-10T00:41:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75a9b9fb80 1 Processor -- start 2023-09-10T00:41:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75a9b9fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924300 0x55c297931500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75a9b9fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c297924cc0 0x55c297931a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75a9b9fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c29792a1a0 0x55c297931f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75a9b9fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2979911b0 con 0x55c29792a1a0 2023-09-10T00:41:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75a9b9fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2979912f0 con 0x55c297924cc0 2023-09-10T00:41:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75a9b9fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c297991460 con 0x55c297924300 2023-09-10T00:41:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75a9b9fb80 10 client.0 _ll_register_callbacks cb 0x7f758800b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75977fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c29792a1a0 0x55c297931f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75977fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c29792a1a0 0x55c297931f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49532/0 (socket says 172.21.15.44:49532) 2023-09-10T00:41:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75977fe700 1 -- 172.21.15.44:0/1427422840 learned_addr learned my addr 172.21.15.44:0/1427422840 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75967fc700 1 --2- 172.21.15.44:0/1427422840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c297924cc0 0x55c297931a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f7596ffd700 1 --2- 172.21.15.44:0/1427422840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924300 0x55c297931500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75977fe700 1 -- 172.21.15.44:0/1427422840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924300 msgr2=0x55c297931500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75977fe700 1 --2- 172.21.15.44:0/1427422840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924300 0x55c297931500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75977fe700 1 -- 172.21.15.44:0/1427422840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c297924cc0 msgr2=0x55c297931a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75977fe700 1 --2- 172.21.15.44:0/1427422840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c297924cc0 0x55c297931a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f75977fe700 1 -- 172.21.15.44:0/1427422840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759810dd30 con 0x55c29792a1a0 2023-09-10T00:41:21.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.547+0000 7f7596ffd700 1 --2- 172.21.15.44:0/1427422840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924300 0x55c297931500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:41:21.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.548+0000 7f75977fe700 1 --2- 172.21.15.44:0/1427422840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c29792a1a0 0x55c297931f80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f75900c90a0 tx=0x7f7590098580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673eab6b69d741b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:21.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.548+0000 7f7576ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:21.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.548+0000 7f7576ffd700 1 -- 172.21.15.44:0/1427422840 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75900b0070 con 0x55c29792a1a0 2023-09-10T00:41:21.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.548+0000 7f7576ffd700 1 -- 172.21.15.44:0/1427422840 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7590099320 con 0x55c29792a1a0 2023-09-10T00:41:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.548+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/1427422840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f758c019c20 con 0x55c29792a1a0 2023-09-10T00:41:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.548+0000 7f7576ffd700 1 -- 172.21.15.44:0/1427422840 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7590098c20 con 0x55c29792a1a0 2023-09-10T00:41:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.548+0000 7f75a9b9fb80 10 client.16101 fetch_fsmap learned FSMap version 30 2023-09-10T00:41:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.548+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/1427422840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c29768bc00 con 0x55c29792a1a0 2023-09-10T00:41:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.548+0000 7f7576ffd700 1 -- 172.21.15.44:0/1427422840 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7590062070 con 0x55c29792a1a0 2023-09-10T00:41:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.548+0000 7f7576ffd700 1 -- 172.21.15.44:0/1427422840 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f75900b3020 con 0x55c29792a1a0 2023-09-10T00:41:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.549+0000 7f75a9b9fb80 10 client.16101 fetch_fsmap finished waiting for FSMap version 30 2023-09-10T00:41:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.549+0000 7f75a9b9fb80 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.549+0000 7f75a9b9fb80 20 client.16101 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.549+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/1427422840 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c297992b30 0x55c297994f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.549+0000 7f75a9b9fb80 4 client.16101 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:41:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.549+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/1427422840 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c29786f8c0 con 0x55c297992b30 2023-09-10T00:41:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.549+0000 7f7596ffd700 1 --2- 172.21.15.44:0/1427422840 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c297992b30 0x55c297994f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.549+0000 7f7576ffd700 10 client.16101 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.549+0000 7f7596ffd700 1 --2- 172.21.15.44:0/1427422840 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c297992b30 0x55c297994f50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b10c37b3ee0bf8d4 server_cookie=29dcd67c1b982215 in_seq=0 out_seq=0 2023-09-10T00:41:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.563+0000 7f7576ffd700 1 -- 172.21.15.44:0/1427422840 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c29786f8c0 con 0x55c297992b30 2023-09-10T00:41:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.563+0000 7f7576ffd700 10 client.16101 handle_command_reply: tid=0 2023-09-10T00:41:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.563+0000 7f75a9b9fb80 1 client.16101 shutdown 2023-09-10T00:41:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.563+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/1427422840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c29792a1a0 msgr2=0x55c297931f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.563+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/1427422840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c29792a1a0 0x55c297931f80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f75900c90a0 tx=0x7f7590098580 comp rx=0 tx=0).stop 2023-09-10T00:41:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.563+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/1427422840 shutdown_connections 2023-09-10T00:41:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.563+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/1427422840 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c297992b30 0x55c297994f50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.563+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/1427422840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c29792a1a0 0x55c297931f80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.563+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/1427422840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c297924300 0x55c297931500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.563+0000 7f7576ffd700 0 client.16101 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.563+0000 7f75a9b9fb80 1 --2- 172.21.15.44:0/1427422840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c297924cc0 0x55c297931a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.563+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/1427422840 >> 172.21.15.44:0/1427422840 conn(0x55c297899030 msgr2=0x55c297928190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.564+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/1427422840 shutdown_connections 2023-09-10T00:41:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.564+0000 7f75a9b9fb80 1 -- 172.21.15.44:0/1427422840 wait complete. 2023-09-10T00:41:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.564+0000 7f75a9b9fb80 20 client.16101 trim_cache size 0 max 16384 2023-09-10T00:41:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:41:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:41:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:41:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:41:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.564+0000 7f759d9d9700 1 -- 172.21.15.44:0/549431872 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f75800a08d0 msgr2=0x7f75800a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.564+0000 7f759d9d9700 1 --2- 172.21.15.44:0/549431872 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f75800a08d0 0x7f75800a2d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f758800b580 tx=0x7f7588005be0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.564+0000 7f759d9d9700 1 -- 172.21.15.44:0/549431872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7598106670 msgr2=0x7f75981963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.564+0000 7f759d9d9700 1 --2- 172.21.15.44:0/549431872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7598106670 0x7f75981963c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7590007ee0 tx=0x7f759000d3b0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.564+0000 7f759d9d9700 1 -- 172.21.15.44:0/549431872 shutdown_connections 2023-09-10T00:41:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.564+0000 7f759d9d9700 1 --2- 172.21.15.44:0/549431872 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f75800a08d0 0x7f75800a2d90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.564+0000 7f759d9d9700 1 --2- 172.21.15.44:0/549431872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7598106670 0x7f75981963c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.564+0000 7f759d9d9700 1 --2- 172.21.15.44:0/549431872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75981005b0 0x7f759819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.564+0000 7f759d9d9700 1 --2- 172.21.15.44:0/549431872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7598100f70 0x7f759819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.564+0000 7f759d9d9700 1 -- 172.21.15.44:0/549431872 >> 172.21.15.44:0/549431872 conn(0x7f75980fc060 msgr2=0x7f75981042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.565+0000 7f759d9d9700 1 -- 172.21.15.44:0/549431872 shutdown_connections 2023-09-10T00:41:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:21.565+0000 7f759d9d9700 1 -- 172.21.15.44:0/549431872 wait complete. 2023-09-10T00:41:21.601 INFO:teuthology.orchestra.run.smithi044.stderr:1900000 of 50000000 tuples (3%) done (elapsed 21.24 s, remaining 537.59 s) 2023-09-10T00:41:22.511 INFO:teuthology.orchestra.run.smithi044.stderr:2000000 of 50000000 tuples (4%) done (elapsed 22.15 s, remaining 531.49 s) 2023-09-10T00:41:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:22 smithi131 ceph-mon[103713]: pgmap v24: 129 pgs: 129 active+clean; 451 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 51 MiB/s wr, 101 op/s 2023-09-10T00:41:23.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:22 smithi044 ceph-mon[98676]: pgmap v24: 129 pgs: 129 active+clean; 451 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 51 MiB/s wr, 101 op/s 2023-09-10T00:41:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:22 smithi099 ceph-mon[102692]: pgmap v24: 129 pgs: 129 active+clean; 451 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 51 MiB/s wr, 101 op/s 2023-09-10T00:41:23.783 INFO:teuthology.orchestra.run.smithi044.stderr:2100000 of 50000000 tuples (4%) done (elapsed 23.42 s, remaining 534.14 s) 2023-09-10T00:41:24.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:23 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:24.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:23 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:24.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:23 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:24 smithi131 ceph-mon[103713]: pgmap v25: 129 pgs: 129 active+clean; 451 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 34 MiB/s wr, 32 op/s 2023-09-10T00:41:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:24 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:25.046 INFO:teuthology.orchestra.run.smithi044.stderr:2200000 of 50000000 tuples (4%) done (elapsed 24.68 s, remaining 536.24 s) 2023-09-10T00:41:25.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:24 smithi044 ceph-mon[98676]: pgmap v25: 129 pgs: 129 active+clean; 451 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 34 MiB/s wr, 32 op/s 2023-09-10T00:41:25.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:24 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:24 smithi099 ceph-mon[102692]: pgmap v25: 129 pgs: 129 active+clean; 451 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 34 MiB/s wr, 32 op/s 2023-09-10T00:41:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:24 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:25.963 INFO:teuthology.orchestra.run.smithi044.stderr:2300000 of 50000000 tuples (4%) done (elapsed 25.60 s, remaining 530.86 s) 2023-09-10T00:41:26.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:25 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:26.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:25 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:26.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:25 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:26 smithi131 ceph-mon[103713]: pgmap v26: 129 pgs: 129 active+clean; 545 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 49 MiB/s wr, 51 op/s 2023-09-10T00:41:27.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:26 smithi044 ceph-mon[98676]: pgmap v26: 129 pgs: 129 active+clean; 545 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 49 MiB/s wr, 51 op/s 2023-09-10T00:41:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:26 smithi099 ceph-mon[102692]: pgmap v26: 129 pgs: 129 active+clean; 545 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 49 MiB/s wr, 51 op/s 2023-09-10T00:41:27.240 INFO:teuthology.orchestra.run.smithi044.stderr:2400000 of 50000000 tuples (4%) done (elapsed 26.87 s, remaining 533.01 s) 2023-09-10T00:41:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:27 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:28.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:27 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:28.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:27 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:28.439 INFO:teuthology.orchestra.run.smithi044.stderr:2500000 of 50000000 tuples (5%) done (elapsed 28.07 s, remaining 533.39 s) 2023-09-10T00:41:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:28 smithi131 ceph-mon[103713]: pgmap v27: 129 pgs: 129 active+clean; 557 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 40 MiB/s wr, 42 op/s 2023-09-10T00:41:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:28 smithi044 ceph-mon[98676]: pgmap v27: 129 pgs: 129 active+clean; 557 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 40 MiB/s wr, 42 op/s 2023-09-10T00:41:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:28 smithi099 ceph-mon[102692]: pgmap v27: 129 pgs: 129 active+clean; 557 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 40 MiB/s wr, 42 op/s 2023-09-10T00:41:29.313 INFO:teuthology.orchestra.run.smithi044.stderr:2600000 of 50000000 tuples (5%) done (elapsed 28.95 s, remaining 527.74 s) 2023-09-10T00:41:30.589 INFO:teuthology.orchestra.run.smithi044.stderr:2700000 of 50000000 tuples (5%) done (elapsed 30.22 s, remaining 529.47 s) 2023-09-10T00:41:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:30 smithi131 ceph-mon[103713]: pgmap v28: 129 pgs: 129 active+clean; 557 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 34 MiB/s wr, 37 op/s 2023-09-10T00:41:31.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:30 smithi044 ceph-mon[98676]: pgmap v28: 129 pgs: 129 active+clean; 557 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 34 MiB/s wr, 37 op/s 2023-09-10T00:41:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:30 smithi099 ceph-mon[102692]: pgmap v28: 129 pgs: 129 active+clean; 557 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 34 MiB/s wr, 37 op/s 2023-09-10T00:41:31.906 INFO:teuthology.orchestra.run.smithi044.stderr:2800000 of 50000000 tuples (5%) done (elapsed 31.54 s, remaining 531.68 s) 2023-09-10T00:41:32.768 INFO:teuthology.orchestra.run.smithi044.stderr:2900000 of 50000000 tuples (5%) done (elapsed 32.40 s, remaining 526.26 s) 2023-09-10T00:41:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:32 smithi044 ceph-mon[98676]: pgmap v29: 129 pgs: 129 active+clean; 657 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 51 MiB/s wr, 55 op/s 2023-09-10T00:41:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:32 smithi099 ceph-mon[102692]: pgmap v29: 129 pgs: 129 active+clean; 657 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 51 MiB/s wr, 55 op/s 2023-09-10T00:41:33.253 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:41:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:32 smithi131 ceph-mon[103713]: pgmap v29: 129 pgs: 129 active+clean; 657 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 51 MiB/s wr, 55 op/s 2023-09-10T00:41:33.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.404+0000 7f95562d7700 1 -- 172.21.15.44:0/1340972232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95500ff4a0 msgr2=0x7f95500ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:33.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.404+0000 7f95562d7700 1 --2- 172.21.15.44:0/1340972232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95500ff4a0 0x7f95500ff920 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f95440099c0 tx=0x7f9544009cd0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.405+0000 7f95562d7700 1 -- 172.21.15.44:0/1340972232 shutdown_connections 2023-09-10T00:41:33.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.405+0000 7f95562d7700 1 --2- 172.21.15.44:0/1340972232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95500ffe60 0x7f955010c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.405+0000 7f95562d7700 1 --2- 172.21.15.44:0/1340972232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9550105560 0x7f9550105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.405+0000 7f95562d7700 1 --2- 172.21.15.44:0/1340972232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95500ff4a0 0x7f95500ff920 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.405+0000 7f95562d7700 1 -- 172.21.15.44:0/1340972232 >> 172.21.15.44:0/1340972232 conn(0x7f95500faf50 msgr2=0x7f95500fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:33.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.405+0000 7f95562d7700 1 -- 172.21.15.44:0/1340972232 shutdown_connections 2023-09-10T00:41:33.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.405+0000 7f95562d7700 1 -- 172.21.15.44:0/1340972232 wait complete. 2023-09-10T00:41:33.409 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.406+0000 7f95562d7700 1 Processor -- start 2023-09-10T00:41:33.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.406+0000 7f95562d7700 1 -- start start 2023-09-10T00:41:33.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.406+0000 7f95562d7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95500ff4a0 0x7f9550195180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.406+0000 7f95562d7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95500ffe60 0x7f95501956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.406+0000 7f95562d7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9550105560 0x7f9550195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.406+0000 7f95562d7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95501a49c0 con 0x7f95500ff4a0 2023-09-10T00:41:33.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.406+0000 7f95562d7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95501a4b30 con 0x7f95500ffe60 2023-09-10T00:41:33.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.406+0000 7f95562d7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95501a4cf0 con 0x7f9550105560 2023-09-10T00:41:33.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.406+0000 7f954ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95500ff4a0 0x7f9550195180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.406+0000 7f954ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95500ff4a0 0x7f9550195180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49540/0 (socket says 172.21.15.44:49540) 2023-09-10T00:41:33.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.406+0000 7f954ffff700 1 -- 172.21.15.44:0/318121539 learned_addr learned my addr 172.21.15.44:0/318121539 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:33.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.406+0000 7f954f7fe700 1 --2- 172.21.15.44:0/318121539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95500ffe60 0x7f95501956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.406+0000 7f9554874700 1 --2- 172.21.15.44:0/318121539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9550105560 0x7f9550195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.407+0000 7f954ffff700 1 -- 172.21.15.44:0/318121539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9550105560 msgr2=0x7f9550195c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:33.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.407+0000 7f954ffff700 1 --2- 172.21.15.44:0/318121539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9550105560 0x7f9550195c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.407+0000 7f954ffff700 1 -- 172.21.15.44:0/318121539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95500ffe60 msgr2=0x7f95501956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:33.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.407+0000 7f954ffff700 1 --2- 172.21.15.44:0/318121539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95500ffe60 0x7f95501956c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.407+0000 7f954ffff700 1 -- 172.21.15.44:0/318121539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95440095f0 con 0x7f95500ff4a0 2023-09-10T00:41:33.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.407+0000 7f954f7fe700 1 --2- 172.21.15.44:0/318121539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95500ffe60 0x7f95501956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:33.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.407+0000 7f954ffff700 1 --2- 172.21.15.44:0/318121539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95500ff4a0 0x7f9550195180 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f954000e9a0 tx=0x7f954000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:33.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.407+0000 7f954d7fa700 1 -- 172.21.15.44:0/318121539 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540009790 con 0x7f95500ff4a0 2023-09-10T00:41:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.407+0000 7f95562d7700 1 -- 172.21.15.44:0/318121539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95501a4fd0 con 0x7f95500ff4a0 2023-09-10T00:41:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.407+0000 7f954d7fa700 1 -- 172.21.15.44:0/318121539 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9540010430 con 0x7f95500ff4a0 2023-09-10T00:41:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.407+0000 7f954d7fa700 1 -- 172.21.15.44:0/318121539 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954000cea0 con 0x7f95500ff4a0 2023-09-10T00:41:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.407+0000 7f95562d7700 1 -- 172.21.15.44:0/318121539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95501a5400 con 0x7f95500ff4a0 2023-09-10T00:41:33.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.411+0000 7f954d7fa700 1 -- 172.21.15.44:0/318121539 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f95400105a0 con 0x7f95500ff4a0 2023-09-10T00:41:33.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.412+0000 7f956249db80 1 Processor -- start 2023-09-10T00:41:33.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.412+0000 7f956249db80 1 -- start start 2023-09-10T00:41:33.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.412+0000 7f954d7fa700 1 --2- 172.21.15.44:0/318121539 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f95380a08d0 0x7f95380a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.412+0000 7f956249db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fd9710 0x55c216fd9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.412+0000 7f956249db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fda030 0x55c216fe3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.412+0000 7f956249db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fe3e40 0x55c216fe6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.412+0000 7f956249db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c216f32730 con 0x55c216fe3e40 2023-09-10T00:41:33.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.412+0000 7f956249db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c216f30050 con 0x55c216fda030 2023-09-10T00:41:33.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.412+0000 7f956249db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c216f2b940 con 0x55c216fd9710 2023-09-10T00:41:33.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.412+0000 7f954d7fa700 1 -- 172.21.15.44:0/318121539 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f9540014070 con 0x7f95500ff4a0 2023-09-10T00:41:33.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.412+0000 7f9554874700 1 --2- 172.21.15.44:0/318121539 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f95380a08d0 0x7f95380a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.412+0000 7f954f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fda030 0x55c216fe3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.412+0000 7f9554874700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fe3e40 0x55c216fe6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.412+0000 7f954f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fda030 0x55c216fe3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51288/0 (socket says 172.21.15.44:51288) 2023-09-10T00:41:33.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.412+0000 7f954f7fe700 1 -- 172.21.15.44:0/1485612692 learned_addr learned my addr 172.21.15.44:0/1485612692 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:33.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.413+0000 7f954ffff700 1 --2- 172.21.15.44:0/1485612692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fd9710 0x55c216fd9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.413+0000 7f954f7fe700 1 -- 172.21.15.44:0/1485612692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fd9710 msgr2=0x55c216fd9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:33.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.413+0000 7f954f7fe700 1 --2- 172.21.15.44:0/1485612692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fd9710 0x55c216fd9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.413+0000 7f954f7fe700 1 -- 172.21.15.44:0/1485612692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fe3e40 msgr2=0x55c216fe6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.413+0000 7f954f7fe700 1 --2- 172.21.15.44:0/1485612692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fe3e40 0x55c216fe6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.413+0000 7f954f7fe700 1 -- 172.21.15.44:0/1485612692 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c216d57de0 con 0x55c216fda030 2023-09-10T00:41:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.413+0000 7f9554874700 1 --2- 172.21.15.44:0/318121539 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f95380a08d0 0x7f95380a2d90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f954800f870 tx=0x7f954800fc30 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.413+0000 7f954f7fe700 1 --2- 172.21.15.44:0/1485612692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fda030 0x55c216fe3900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f95440099c0 tx=0x7f9544005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd118fc6a3abc43 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.413+0000 7f95367fc700 1 -- 172.21.15.44:0/1485612692 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954400bd40 con 0x55c216fda030 2023-09-10T00:41:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.413+0000 7f95367fc700 1 -- 172.21.15.44:0/1485612692 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f954401e040 con 0x55c216fda030 2023-09-10T00:41:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.413+0000 7f95367fc700 1 -- 172.21.15.44:0/1485612692 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544003ce0 con 0x55c216fda030 2023-09-10T00:41:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.414+0000 7f956249db80 1 -- 172.21.15.44:0/1485612692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fda030 msgr2=0x55c216fe3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.414+0000 7f956249db80 1 --2- 172.21.15.44:0/1485612692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fda030 0x55c216fe3900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f95440099c0 tx=0x7f9544005ec0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.414+0000 7f956249db80 1 -- 172.21.15.44:0/1485612692 shutdown_connections 2023-09-10T00:41:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.414+0000 7f956249db80 1 --2- 172.21.15.44:0/1485612692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fe3e40 0x55c216fe6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.414+0000 7f956249db80 1 --2- 172.21.15.44:0/1485612692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fd9710 0x55c216fd9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.414+0000 7f956249db80 1 --2- 172.21.15.44:0/1485612692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fda030 0x55c216fe3900 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.414+0000 7f956249db80 1 -- 172.21.15.44:0/1485612692 >> 172.21.15.44:0/1485612692 conn(0x55c216f4e090 msgr2=0x55c216fd4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.414+0000 7f956249db80 1 -- 172.21.15.44:0/1485612692 shutdown_connections 2023-09-10T00:41:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.414+0000 7f956249db80 1 -- 172.21.15.44:0/1485612692 wait complete. 2023-09-10T00:41:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.414+0000 7f956249db80 1 -- start start 2023-09-10T00:41:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.415+0000 7f956249db80 1 Processor -- start 2023-09-10T00:41:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.415+0000 7f956249db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fd9710 0x55c216fe3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.415+0000 7f956249db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fda030 0x55c216fe0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.415+0000 7f956249db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fe3e40 0x55c216fe10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.415+0000 7f956249db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c216ec9b70 con 0x55c216fda030 2023-09-10T00:41:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.415+0000 7f956249db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c216ec9870 con 0x55c216fe3e40 2023-09-10T00:41:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.415+0000 7f956249db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c216dad920 con 0x55c216fd9710 2023-09-10T00:41:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.415+0000 7f956249db80 10 client.0 _ll_register_callbacks cb 0x55c216e13a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.415+0000 7f954f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fda030 0x55c216fe0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.415+0000 7f954f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fda030 0x55c216fe0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49556/0 (socket says 172.21.15.44:49556) 2023-09-10T00:41:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.415+0000 7f954f7fe700 1 -- 172.21.15.44:0/2089118219 learned_addr learned my addr 172.21.15.44:0/2089118219 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.415+0000 7f954ffff700 1 --2- 172.21.15.44:0/2089118219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fd9710 0x55c216fe3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.415+0000 7f9554874700 1 --2- 172.21.15.44:0/2089118219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fe3e40 0x55c216fe10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.416+0000 7f954f7fe700 1 -- 172.21.15.44:0/2089118219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fd9710 msgr2=0x55c216fe3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.416+0000 7f954f7fe700 1 --2- 172.21.15.44:0/2089118219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fd9710 0x55c216fe3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.416+0000 7f954f7fe700 1 -- 172.21.15.44:0/2089118219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fe3e40 msgr2=0x55c216fe10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:33.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.416+0000 7f954f7fe700 1 --2- 172.21.15.44:0/2089118219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fe3e40 0x55c216fe10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.416+0000 7f954f7fe700 1 -- 172.21.15.44:0/2089118219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955010cc20 con 0x55c216fda030 2023-09-10T00:41:33.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.416+0000 7f954ffff700 1 --2- 172.21.15.44:0/2089118219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fd9710 0x55c216fe3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:33.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.416+0000 7f954f7fe700 1 --2- 172.21.15.44:0/2089118219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fda030 0x55c216fe0ac0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9544025040 tx=0x7f9544003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7e104efcce54c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:33.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.416+0000 7f952f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:33.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.416+0000 7f952f7fe700 1 -- 172.21.15.44:0/2089118219 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95440052c0 con 0x55c216fda030 2023-09-10T00:41:33.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.416+0000 7f956249db80 1 -- 172.21.15.44:0/2089118219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c216e834f0 con 0x55c216fda030 2023-09-10T00:41:33.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.416+0000 7f952f7fe700 1 -- 172.21.15.44:0/2089118219 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f9544005420 con 0x55c216fda030 2023-09-10T00:41:33.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.416+0000 7f952f7fe700 1 -- 172.21.15.44:0/2089118219 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95440055e0 con 0x55c216fda030 2023-09-10T00:41:33.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.416+0000 7f956249db80 10 client.16113 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:33.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.416+0000 7f956249db80 1 -- 172.21.15.44:0/2089118219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c216d40c60 con 0x55c216fda030 2023-09-10T00:41:33.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.416+0000 7f952f7fe700 1 -- 172.21.15.44:0/2089118219 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c216e834f0 con 0x55c216fda030 2023-09-10T00:41:33.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.417+0000 7f952f7fe700 1 -- 172.21.15.44:0/2089118219 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9544021070 con 0x55c216fda030 2023-09-10T00:41:33.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.417+0000 7f956249db80 10 client.16113 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:33.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.417+0000 7f956249db80 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:33.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.417+0000 7f956249db80 20 client.16113 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:33.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.417+0000 7f956249db80 1 --2- 172.21.15.44:0/2089118219 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c216ff3490 0x55c216ff58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.417+0000 7f956249db80 4 client.16113 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:33.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.417+0000 7f956249db80 1 -- 172.21.15.44:0/2089118219 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c216f408a0 con 0x55c216ff3490 2023-09-10T00:41:33.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.417+0000 7f954ffff700 1 --2- 172.21.15.44:0/2089118219 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c216ff3490 0x55c216ff58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.417+0000 7f952f7fe700 10 client.16113 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:33.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.417+0000 7f954ffff700 1 --2- 172.21.15.44:0/2089118219 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c216ff3490 0x55c216ff58b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82834a4368f611c8 server_cookie=6f4a7f11ca90ab9b in_seq=0 out_seq=0 2023-09-10T00:41:33.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.418+0000 7f952f7fe700 1 -- 172.21.15.44:0/2089118219 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c216f408a0 con 0x55c216ff3490 2023-09-10T00:41:33.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.418+0000 7f952f7fe700 10 client.16113 handle_command_reply: tid=0 2023-09-10T00:41:33.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.418+0000 7f956249db80 1 client.16113 shutdown 2023-09-10T00:41:33.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.418+0000 7f956249db80 1 -- 172.21.15.44:0/2089118219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fda030 msgr2=0x55c216fe0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:33.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.418+0000 7f956249db80 1 --2- 172.21.15.44:0/2089118219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fda030 0x55c216fe0ac0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9544025040 tx=0x7f9544003b10 comp rx=0 tx=0).stop 2023-09-10T00:41:33.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.418+0000 7f956249db80 1 -- 172.21.15.44:0/2089118219 shutdown_connections 2023-09-10T00:41:33.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.418+0000 7f956249db80 1 --2- 172.21.15.44:0/2089118219 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c216ff3490 0x55c216ff58b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.418+0000 7f956249db80 1 --2- 172.21.15.44:0/2089118219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fe3e40 0x55c216fe10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.418+0000 7f956249db80 1 --2- 172.21.15.44:0/2089118219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fd9710 0x55c216fe3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.418+0000 7f952f7fe700 0 client.16113 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:33.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.418+0000 7f956249db80 1 --2- 172.21.15.44:0/2089118219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fda030 0x55c216fe0ac0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.418+0000 7f956249db80 1 -- 172.21.15.44:0/2089118219 >> 172.21.15.44:0/2089118219 conn(0x55c216f4e090 msgr2=0x55c216fd47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:33.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.419+0000 7f956249db80 1 -- 172.21.15.44:0/2089118219 shutdown_connections 2023-09-10T00:41:33.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.419+0000 7f956249db80 1 -- 172.21.15.44:0/2089118219 wait complete. 2023-09-10T00:41:33.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.419+0000 7f956249db80 20 client.16113 trim_cache size 0 max 16384 2023-09-10T00:41:33.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.437+0000 7f956249db80 1 Processor -- start 2023-09-10T00:41:33.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.437+0000 7f956249db80 1 -- start start 2023-09-10T00:41:33.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.437+0000 7f956249db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fd9710 0x55c216fe0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f956249db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fda030 0x55c216fdbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.440 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f956249db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fe3e40 0x55c216fef0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.440 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f956249db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c216fe16d0 con 0x55c216fda030 2023-09-10T00:41:33.440 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f956249db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c216fe08a0 con 0x55c216fe3e40 2023-09-10T00:41:33.440 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f956249db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c216fdf020 con 0x55c216fd9710 2023-09-10T00:41:33.440 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f954f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fda030 0x55c216fdbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.441 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f954f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fda030 0x55c216fdbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49564/0 (socket says 172.21.15.44:49564) 2023-09-10T00:41:33.441 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f954f7fe700 1 -- 172.21.15.44:0/842961372 learned_addr learned my addr 172.21.15.44:0/842961372 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:33.441 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f954ffff700 1 --2- 172.21.15.44:0/842961372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fd9710 0x55c216fe0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.441 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f9554874700 1 --2- 172.21.15.44:0/842961372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fe3e40 0x55c216fef0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f954f7fe700 1 -- 172.21.15.44:0/842961372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fd9710 msgr2=0x55c216fe0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:33.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f954f7fe700 1 --2- 172.21.15.44:0/842961372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fd9710 0x55c216fe0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f954f7fe700 1 -- 172.21.15.44:0/842961372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fe3e40 msgr2=0x55c216fef0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:33.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f954f7fe700 1 --2- 172.21.15.44:0/842961372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fe3e40 0x55c216fef0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f954f7fe700 1 -- 172.21.15.44:0/842961372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c216d7e250 con 0x55c216fda030 2023-09-10T00:41:33.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f9554874700 1 --2- 172.21.15.44:0/842961372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fe3e40 0x55c216fef0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:33.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f954f7fe700 1 --2- 172.21.15.44:0/842961372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fda030 0x55c216fdbe00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f954400a010 tx=0x7f9544005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395e2f76478419e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:33.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f9536ffd700 1 -- 172.21.15.44:0/842961372 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544022a50 con 0x55c216fda030 2023-09-10T00:41:33.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.438+0000 7f9536ffd700 1 -- 172.21.15.44:0/842961372 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f954400b540 con 0x55c216fda030 2023-09-10T00:41:33.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.439+0000 7f9536ffd700 1 -- 172.21.15.44:0/842961372 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544022ea0 con 0x55c216fda030 2023-09-10T00:41:33.449 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.439+0000 7f956249db80 1 -- 172.21.15.44:0/842961372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fda030 msgr2=0x55c216fdbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:33.449 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.439+0000 7f956249db80 1 --2- 172.21.15.44:0/842961372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fda030 0x55c216fdbe00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f954400a010 tx=0x7f9544005040 comp rx=0 tx=0).stop 2023-09-10T00:41:33.449 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.439+0000 7f956249db80 1 -- 172.21.15.44:0/842961372 shutdown_connections 2023-09-10T00:41:33.450 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.439+0000 7f956249db80 1 --2- 172.21.15.44:0/842961372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fe3e40 0x55c216fef0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.450 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.439+0000 7f956249db80 1 --2- 172.21.15.44:0/842961372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fd9710 0x55c216fe0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.450 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.439+0000 7f956249db80 1 --2- 172.21.15.44:0/842961372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fda030 0x55c216fdbe00 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.451 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.439+0000 7f956249db80 1 -- 172.21.15.44:0/842961372 >> 172.21.15.44:0/842961372 conn(0x55c216f4e090 msgr2=0x55c216ffa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:33.451 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.439+0000 7f956249db80 1 -- 172.21.15.44:0/842961372 shutdown_connections 2023-09-10T00:41:33.451 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.440+0000 7f956249db80 1 -- 172.21.15.44:0/842961372 wait complete. 2023-09-10T00:41:33.451 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.440+0000 7f956249db80 1 -- start start 2023-09-10T00:41:33.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.440+0000 7f956249db80 1 Processor -- start 2023-09-10T00:41:33.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.440+0000 7f956249db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fd9710 0x55c216fea420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.440+0000 7f956249db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fda030 0x55c216fea960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.440+0000 7f956249db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fe3e40 0x55c216fe87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.440+0000 7f956249db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c216fe8ef0 con 0x55c216fd9710 2023-09-10T00:41:33.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.440+0000 7f956249db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c216fe9060 con 0x55c216fda030 2023-09-10T00:41:33.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.440+0000 7f956249db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c216fe9220 con 0x55c216fe3e40 2023-09-10T00:41:33.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.440+0000 7f956249db80 10 client.0 _ll_register_callbacks cb 0x7f954801e050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:33.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.440+0000 7f954ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fd9710 0x55c216fea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.441+0000 7f954ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fd9710 0x55c216fea420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49568/0 (socket says 172.21.15.44:49568) 2023-09-10T00:41:33.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.441+0000 7f954ffff700 1 -- 172.21.15.44:0/665721331 learned_addr learned my addr 172.21.15.44:0/665721331 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:33.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.441+0000 7f954f7fe700 1 --2- 172.21.15.44:0/665721331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fda030 0x55c216fea960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.441+0000 7f954ffff700 1 -- 172.21.15.44:0/665721331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fe3e40 msgr2=0x55c216fe87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:33.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.441+0000 7f954ffff700 1 --2- 172.21.15.44:0/665721331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fe3e40 0x55c216fe87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.441+0000 7f954ffff700 1 -- 172.21.15.44:0/665721331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fda030 msgr2=0x55c216fea960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:33.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.441+0000 7f954ffff700 1 --2- 172.21.15.44:0/665721331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fda030 0x55c216fea960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.441+0000 7f954ffff700 1 -- 172.21.15.44:0/665721331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c216d40c60 con 0x55c216fd9710 2023-09-10T00:41:33.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.441+0000 7f954ffff700 1 --2- 172.21.15.44:0/665721331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fd9710 0x55c216fea420 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f954000c2d0 tx=0x7f954000c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b930d48984f4493 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:33.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.441+0000 7f952f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:33.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.441+0000 7f952f7fe700 1 -- 172.21.15.44:0/665721331 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540099630 con 0x55c216fd9710 2023-09-10T00:41:33.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.441+0000 7f952f7fe700 1 -- 172.21.15.44:0/665721331 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f95400b0070 con 0x55c216fd9710 2023-09-10T00:41:33.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.441+0000 7f956249db80 1 -- 172.21.15.44:0/665721331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f954401e040 con 0x55c216fd9710 2023-09-10T00:41:33.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.441+0000 7f952f7fe700 1 -- 172.21.15.44:0/665721331 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f954009d460 con 0x55c216fd9710 2023-09-10T00:41:33.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.441+0000 7f956249db80 10 client.16119 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:33.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.441+0000 7f956249db80 1 -- 172.21.15.44:0/665721331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c216d562b0 con 0x55c216fd9710 2023-09-10T00:41:33.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.442+0000 7f952f7fe700 1 -- 172.21.15.44:0/665721331 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f954401e040 con 0x55c216fd9710 2023-09-10T00:41:33.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.442+0000 7f952f7fe700 1 -- 172.21.15.44:0/665721331 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f95400cc350 con 0x55c216fd9710 2023-09-10T00:41:33.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.442+0000 7f956249db80 10 client.16119 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:33.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.442+0000 7f956249db80 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:33.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.442+0000 7f956249db80 20 client.16119 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:33.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.442+0000 7f956249db80 1 --2- 172.21.15.44:0/665721331 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c217048500 0x55c21704a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.442+0000 7f956249db80 4 client.16119 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:41:33.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.442+0000 7f956249db80 1 -- 172.21.15.44:0/665721331 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c216f24920 con 0x55c217048500 2023-09-10T00:41:33.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.442+0000 7f954f7fe700 1 --2- 172.21.15.44:0/665721331 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c217048500 0x55c21704a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f952f7fe700 10 client.16119 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:33.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f954f7fe700 1 --2- 172.21.15.44:0/665721331 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c217048500 0x55c21704a940 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=127ffc0456f57b6e server_cookie=c4e37d7852fa0722 in_seq=0 out_seq=0 2023-09-10T00:41:33.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f952f7fe700 1 -- 172.21.15.44:0/665721331 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c216f24920 con 0x55c217048500 2023-09-10T00:41:33.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f952f7fe700 10 client.16119 handle_command_reply: tid=0 2023-09-10T00:41:33.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f956249db80 1 client.16119 shutdown 2023-09-10T00:41:33.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f956249db80 1 -- 172.21.15.44:0/665721331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fd9710 msgr2=0x55c216fea420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:33.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f956249db80 1 --2- 172.21.15.44:0/665721331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fd9710 0x55c216fea420 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f954000c2d0 tx=0x7f954000c400 comp rx=0 tx=0).stop 2023-09-10T00:41:33.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f956249db80 1 -- 172.21.15.44:0/665721331 shutdown_connections 2023-09-10T00:41:33.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f956249db80 1 --2- 172.21.15.44:0/665721331 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c217048500 0x55c21704a940 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f956249db80 1 --2- 172.21.15.44:0/665721331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c216fe3e40 0x55c216fe87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f956249db80 1 --2- 172.21.15.44:0/665721331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c216fd9710 0x55c216fea420 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f956249db80 1 --2- 172.21.15.44:0/665721331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c216fda030 0x55c216fea960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f956249db80 1 -- 172.21.15.44:0/665721331 >> 172.21.15.44:0/665721331 conn(0x55c216f4e090 msgr2=0x55c216fe0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:33.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f952f7fe700 0 client.16119 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:33.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f956249db80 1 -- 172.21.15.44:0/665721331 shutdown_connections 2023-09-10T00:41:33.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f956249db80 1 -- 172.21.15.44:0/665721331 wait complete. 2023-09-10T00:41:33.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.443+0000 7f956249db80 20 client.16119 trim_cache size 0 max 16384 2023-09-10T00:41:33.463 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:41:33.463 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2023-09-10T00:41:33.463 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2023-09-10T00:41:33.463 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:41:33.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.444+0000 7f95562d7700 1 -- 172.21.15.44:0/318121539 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f95380a08d0 msgr2=0x7f95380a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:33.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.444+0000 7f95562d7700 1 --2- 172.21.15.44:0/318121539 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f95380a08d0 0x7f95380a2d90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f954800f870 tx=0x7f954800fc30 comp rx=0 tx=0).stop 2023-09-10T00:41:33.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.444+0000 7f95562d7700 1 -- 172.21.15.44:0/318121539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95500ff4a0 msgr2=0x7f9550195180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:33.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.444+0000 7f95562d7700 1 --2- 172.21.15.44:0/318121539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95500ff4a0 0x7f9550195180 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f954000e9a0 tx=0x7f954000ecb0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.444+0000 7f95562d7700 1 -- 172.21.15.44:0/318121539 shutdown_connections 2023-09-10T00:41:33.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.444+0000 7f95562d7700 1 --2- 172.21.15.44:0/318121539 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f95380a08d0 0x7f95380a2d90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.444+0000 7f95562d7700 1 --2- 172.21.15.44:0/318121539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9550105560 0x7f9550195c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.444+0000 7f95562d7700 1 --2- 172.21.15.44:0/318121539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95500ff4a0 0x7f9550195180 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.444+0000 7f95562d7700 1 --2- 172.21.15.44:0/318121539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95500ffe60 0x7f95501956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.444+0000 7f95562d7700 1 -- 172.21.15.44:0/318121539 >> 172.21.15.44:0/318121539 conn(0x7f95500faf50 msgr2=0x7f95500fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:33.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.444+0000 7f95562d7700 1 -- 172.21.15.44:0/318121539 shutdown_connections 2023-09-10T00:41:33.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.444+0000 7f95562d7700 1 -- 172.21.15.44:0/318121539 wait complete. 2023-09-10T00:41:33.468 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-10T00:41:33.468 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-10T00:41:33.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.610+0000 7f0a2c2c1700 1 -- 172.21.15.44:0/1263953433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a240ffe00 msgr2=0x7f0a2410c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:33.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.610+0000 7f0a2c2c1700 1 --2- 172.21.15.44:0/1263953433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a240ffe00 0x7f0a2410c310 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00b0a0 tx=0x7f0a1c00b3b0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.610+0000 7f0a2c2c1700 1 -- 172.21.15.44:0/1263953433 shutdown_connections 2023-09-10T00:41:33.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.610+0000 7f0a2c2c1700 1 --2- 172.21.15.44:0/1263953433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a240ffe00 0x7f0a2410c310 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.610+0000 7f0a2c2c1700 1 --2- 172.21.15.44:0/1263953433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a241054e0 0x7f0a241058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.610+0000 7f0a2c2c1700 1 --2- 172.21.15.44:0/1263953433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a240ff440 0x7f0a240ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.610+0000 7f0a2c2c1700 1 -- 172.21.15.44:0/1263953433 >> 172.21.15.44:0/1263953433 conn(0x7f0a240faf10 msgr2=0x7f0a240fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:33.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.610+0000 7f0a2c2c1700 1 -- 172.21.15.44:0/1263953433 shutdown_connections 2023-09-10T00:41:33.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.610+0000 7f0a2c2c1700 1 -- 172.21.15.44:0/1263953433 wait complete. 2023-09-10T00:41:33.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2c2c1700 1 Processor -- start 2023-09-10T00:41:33.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2c2c1700 1 -- start start 2023-09-10T00:41:33.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2c2c1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a240ff440 0x7f0a241964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2c2c1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a240ffe00 0x7f0a241969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2c2c1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a241054e0 0x7f0a2419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2c2c1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2419b480 con 0x7f0a241054e0 2023-09-10T00:41:33.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2c2c1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2419b5f0 con 0x7f0a240ff440 2023-09-10T00:41:33.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2c2c1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2419b7b0 con 0x7f0a240ffe00 2023-09-10T00:41:33.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2a85e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a241054e0 0x7f0a2419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2985c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a240ffe00 0x7f0a241969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2a85e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a241054e0 0x7f0a2419ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49586/0 (socket says 172.21.15.44:49586) 2023-09-10T00:41:33.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2a85e700 1 -- 172.21.15.44:0/865454883 learned_addr learned my addr 172.21.15.44:0/865454883 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:33.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2a05d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a240ff440 0x7f0a241964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2a85e700 1 -- 172.21.15.44:0/865454883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a240ffe00 msgr2=0x7f0a241969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:33.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2a85e700 1 --2- 172.21.15.44:0/865454883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a240ffe00 0x7f0a241969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2a85e700 1 -- 172.21.15.44:0/865454883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a240ff440 msgr2=0x7f0a241964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:33.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2a85e700 1 --2- 172.21.15.44:0/865454883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a240ff440 0x7f0a241964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.611+0000 7f0a2a85e700 1 -- 172.21.15.44:0/865454883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c009d00 con 0x7f0a241054e0 2023-09-10T00:41:33.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.612+0000 7f0a2a85e700 1 --2- 172.21.15.44:0/865454883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a241054e0 0x7f0a2419ae40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00b780 tx=0x7f0a1c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:33.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.612+0000 7f0a1b7fe700 1 -- 172.21.15.44:0/865454883 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c00e040 con 0x7f0a241054e0 2023-09-10T00:41:33.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.612+0000 7f0a1b7fe700 1 -- 172.21.15.44:0/865454883 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a1c0094c0 con 0x7f0a241054e0 2023-09-10T00:41:33.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.612+0000 7f0a2c2c1700 1 -- 172.21.15.44:0/865454883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a2419ba30 con 0x7f0a241054e0 2023-09-10T00:41:33.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.612+0000 7f0a1b7fe700 1 -- 172.21.15.44:0/865454883 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c0052a0 con 0x7f0a241054e0 2023-09-10T00:41:33.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.612+0000 7f0a2c2c1700 1 -- 172.21.15.44:0/865454883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a241a32d0 con 0x7f0a241054e0 2023-09-10T00:41:33.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.613+0000 7f0a2c2c1700 1 -- 172.21.15.44:0/865454883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a240673f0 con 0x7f0a241054e0 2023-09-10T00:41:33.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.614+0000 7f0a1b7fe700 1 -- 172.21.15.44:0/865454883 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f0a1c01d470 con 0x7f0a241054e0 2023-09-10T00:41:33.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.615+0000 7f0a1b7fe700 1 --2- 172.21.15.44:0/865454883 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0a100a0920 0x7f0a100a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:33.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.615+0000 7f0a2a05d700 1 --2- 172.21.15.44:0/865454883 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0a100a0920 0x7f0a100a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:33.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.615+0000 7f0a1b7fe700 1 -- 172.21.15.44:0/865454883 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f0a1c0ce580 con 0x7f0a241054e0 2023-09-10T00:41:33.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.615+0000 7f0a2a05d700 1 --2- 172.21.15.44:0/865454883 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0a100a0920 0x7f0a100a2de0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009be0 tx=0x7f0a20009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:33.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.619+0000 7f0a1b7fe700 1 -- 172.21.15.44:0/865454883 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0a1c097810 con 0x7f0a241054e0 2023-09-10T00:41:33.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.916+0000 7f0a2c2c1700 1 -- 172.21.15.44:0/865454883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a241977f0 con 0x7f0a241054e0 2023-09-10T00:41:33.917 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.917+0000 7f0a1b7fe700 1 -- 172.21.15.44:0/865454883 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+11380 (secure 0 0 0) 0x7f0a1c09ae30 con 0x7f0a241054e0 2023-09-10T00:41:33.917 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T00:41:33.918 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4035626069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4035626069},{"type":"v1","addr":"172.21.15.44:6835","nonce":4035626069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1919207693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1919207693},{"type":"v1","addr":"172.21.15.44:6837","nonce":1919207693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-10T00:38:39.183533+0000","modified":"2023-09-10T00:41:26.768829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24605,"mds_1":24574,"mds_2":24572,"mds_3":24515,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14751":{"gid":14751,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/3958270510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":3958270510},{"type":"v1","addr":"172.21.15.44:6839","nonce":3958270510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14763":{"gid":14763,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/1241002552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":1241002552},{"type":"v1","addr":"172.21.15.44:6841","nonce":1241002552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.131:6833/3395466517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3395466517},{"type":"v1","addr":"172.21.15.131:6833","nonce":3395466517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.99:6833/1934109590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1934109590},{"type":"v1","addr":"172.21.15.99:6833","nonce":1934109590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2403105238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2403105238},{"type":"v1","addr":"172.21.15.99:6835","nonce":2403105238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1445095724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1445095724},{"type":"v1","addr":"172.21.15.131:6835","nonce":1445095724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24556":{"gid":24556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/2397154745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2397154745},{"type":"v1","addr":"172.21.15.131:6837","nonce":2397154745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/566283638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":566283638},{"type":"v1","addr":"172.21.15.99:6837","nonce":566283638}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/3222652693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":3222652693},{"type":"v1","addr":"172.21.15.131:6839","nonce":3222652693}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24605":{"gid":24605,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/2622966260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2622966260},{"type":"v1","addr":"172.21.15.99:6839","nonce":2622966260}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-10T00:41:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.919+0000 7f0a2c2c1700 1 -- 172.21.15.44:0/865454883 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0a100a0920 msgr2=0x7f0a100a2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.919+0000 7f0a2c2c1700 1 --2- 172.21.15.44:0/865454883 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0a100a0920 0x7f0a100a2de0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009be0 tx=0x7f0a20009260 comp rx=0 tx=0).stop 2023-09-10T00:41:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.919+0000 7f0a2c2c1700 1 -- 172.21.15.44:0/865454883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a241054e0 msgr2=0x7f0a2419ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:33.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.919+0000 7f0a2c2c1700 1 --2- 172.21.15.44:0/865454883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a241054e0 0x7f0a2419ae40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00b780 tx=0x7f0a1c00bd60 comp rx=0 tx=0).stop 2023-09-10T00:41:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.919+0000 7f0a2c2c1700 1 -- 172.21.15.44:0/865454883 shutdown_connections 2023-09-10T00:41:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.919+0000 7f0a2c2c1700 1 --2- 172.21.15.44:0/865454883 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0a100a0920 0x7f0a100a2de0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.919+0000 7f0a2c2c1700 1 --2- 172.21.15.44:0/865454883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a241054e0 0x7f0a2419ae40 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.919+0000 7f0a2c2c1700 1 --2- 172.21.15.44:0/865454883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a240ff440 0x7f0a241964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.919+0000 7f0a2c2c1700 1 --2- 172.21.15.44:0/865454883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a240ffe00 0x7f0a241969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:33.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.919+0000 7f0a2c2c1700 1 -- 172.21.15.44:0/865454883 >> 172.21.15.44:0/865454883 conn(0x7f0a240faf10 msgr2=0x7f0a241031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:33.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.920+0000 7f0a2c2c1700 1 -- 172.21.15.44:0/865454883 shutdown_connections 2023-09-10T00:41:33.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:33.920+0000 7f0a2c2c1700 1 -- 172.21.15.44:0/865454883 wait complete. 2023-09-10T00:41:33.923 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 34 2023-09-10T00:41:33.933 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-10T00:41:34.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.081+0000 7f45d1f08700 1 -- 172.21.15.44:0/1624975270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc1017c0 msgr2=0x7f45cc101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.081+0000 7f45d1f08700 1 --2- 172.21.15.44:0/1624975270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc1017c0 0x7f45cc101ba0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009990 tx=0x7f45bc009ca0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.081+0000 7f45d1f08700 1 -- 172.21.15.44:0/1624975270 shutdown_connections 2023-09-10T00:41:34.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.081+0000 7f45d1f08700 1 --2- 172.21.15.44:0/1624975270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45cc10ab90 0x7f45cc10d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.081+0000 7f45d1f08700 1 --2- 172.21.15.44:0/1624975270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc1017c0 0x7f45cc101ba0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.081+0000 7f45d1f08700 1 --2- 172.21.15.44:0/1624975270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45cc1020e0 0x7f45cc10a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.081+0000 7f45d1f08700 1 -- 172.21.15.44:0/1624975270 >> 172.21.15.44:0/1624975270 conn(0x7f45cc0fafa0 msgr2=0x7f45cc0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.081+0000 7f45d1f08700 1 -- 172.21.15.44:0/1624975270 shutdown_connections 2023-09-10T00:41:34.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.081+0000 7f45d1f08700 1 -- 172.21.15.44:0/1624975270 wait complete. 2023-09-10T00:41:34.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.082+0000 7f45d1f08700 1 Processor -- start 2023-09-10T00:41:34.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.083+0000 7f45d1f08700 1 -- start start 2023-09-10T00:41:34.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.083+0000 7f45d1f08700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc1017c0 0x7f45cc196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.083+0000 7f45d1f08700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45cc1020e0 0x7f45cc196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.083+0000 7f45d1f08700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45cc10ab90 0x7f45cc19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.083+0000 7f45d1f08700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc19b5f0 con 0x7f45cc1017c0 2023-09-10T00:41:34.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.083+0000 7f45d1f08700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc19b760 con 0x7f45cc1020e0 2023-09-10T00:41:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.083+0000 7f45d1f08700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc19b920 con 0x7f45cc10ab90 2023-09-10T00:41:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.083+0000 7f45cb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc1017c0 0x7f45cc196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.083+0000 7f45cb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc1017c0 0x7f45cc196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49596/0 (socket says 172.21.15.44:49596) 2023-09-10T00:41:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.083+0000 7f45cb7fe700 1 -- 172.21.15.44:0/4157744098 learned_addr learned my addr 172.21.15.44:0/4157744098 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.083+0000 7f45cbfff700 1 --2- 172.21.15.44:0/4157744098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45cc10ab90 0x7f45cc19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.083+0000 7f45caffd700 1 --2- 172.21.15.44:0/4157744098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45cc1020e0 0x7f45cc196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.084+0000 7f45cb7fe700 1 -- 172.21.15.44:0/4157744098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45cc10ab90 msgr2=0x7f45cc19afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.084+0000 7f45cb7fe700 1 --2- 172.21.15.44:0/4157744098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45cc10ab90 0x7f45cc19afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.084+0000 7f45cb7fe700 1 -- 172.21.15.44:0/4157744098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45cc1020e0 msgr2=0x7f45cc196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.084+0000 7f45cb7fe700 1 --2- 172.21.15.44:0/4157744098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45cc1020e0 0x7f45cc196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.084+0000 7f45cb7fe700 1 -- 172.21.15.44:0/4157744098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc0095f0 con 0x7f45cc1017c0 2023-09-10T00:41:34.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.084+0000 7f45cb7fe700 1 --2- 172.21.15.44:0/4157744098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc1017c0 0x7f45cc196610 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f45bc005910 tx=0x7f45bc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.084+0000 7f45c8ff9700 1 -- 172.21.15.44:0/4157744098 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc01d070 con 0x7f45cc1017c0 2023-09-10T00:41:34.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.084+0000 7f45c8ff9700 1 -- 172.21.15.44:0/4157744098 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45bc003ec0 con 0x7f45cc1017c0 2023-09-10T00:41:34.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.084+0000 7f45d1f08700 1 -- 172.21.15.44:0/4157744098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45cc19bba0 con 0x7f45cc1017c0 2023-09-10T00:41:34.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.084+0000 7f45c8ff9700 1 -- 172.21.15.44:0/4157744098 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00fb50 con 0x7f45cc1017c0 2023-09-10T00:41:34.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.084+0000 7f45d1f08700 1 -- 172.21.15.44:0/4157744098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45cc1a3440 con 0x7f45cc1017c0 2023-09-10T00:41:34.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.086+0000 7f45c8ff9700 1 -- 172.21.15.44:0/4157744098 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f45bc004030 con 0x7f45cc1017c0 2023-09-10T00:41:34.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.087+0000 7f45c8ff9700 1 --2- 172.21.15.44:0/4157744098 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45b40a08d0 0x7f45b40a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.087+0000 7f45caffd700 1 --2- 172.21.15.44:0/4157744098 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45b40a08d0 0x7f45b40a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.088+0000 7f45c8ff9700 1 -- 172.21.15.44:0/4157744098 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f45bc0cd650 con 0x7f45cc1017c0 2023-09-10T00:41:34.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.088+0000 7f45caffd700 1 --2- 172.21.15.44:0/4157744098 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45b40a08d0 0x7f45b40a2d90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f45c0005fd0 tx=0x7f45c0005f60 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.088+0000 7f45de0ceb80 1 Processor -- start 2023-09-10T00:41:34.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.088+0000 7f45de0ceb80 1 -- start start 2023-09-10T00:41:34.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.088+0000 7f45de0ceb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d8246d710 0x557d8246daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.088+0000 7f45de0ceb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246e030 0x557d82477900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.088+0000 7f45de0ceb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d82477e40 0x557d8247a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.088+0000 7f45de0ceb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d823c6730 con 0x557d8246e030 2023-09-10T00:41:34.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.088+0000 7f45de0ceb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d823c4050 con 0x557d8246d710 2023-09-10T00:41:34.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.088+0000 7f45de0ceb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d823bf940 con 0x557d82477e40 2023-09-10T00:41:34.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.089+0000 7f45caffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246e030 0x557d82477900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.089+0000 7f45caffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246e030 0x557d82477900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49606/0 (socket says 172.21.15.44:49606) 2023-09-10T00:41:34.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.089+0000 7f45caffd700 1 -- 172.21.15.44:0/3810694341 learned_addr learned my addr 172.21.15.44:0/3810694341 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.089+0000 7f45cb7fe700 1 --2- 172.21.15.44:0/3810694341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d8246d710 0x557d8246daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.089+0000 7f45cbfff700 1 --2- 172.21.15.44:0/3810694341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d82477e40 0x557d8247a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.089+0000 7f45caffd700 1 -- 172.21.15.44:0/3810694341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d82477e40 msgr2=0x557d8247a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.089+0000 7f45caffd700 1 --2- 172.21.15.44:0/3810694341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d82477e40 0x557d8247a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.089+0000 7f45caffd700 1 -- 172.21.15.44:0/3810694341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d8246d710 msgr2=0x557d8246daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.089+0000 7f45caffd700 1 --2- 172.21.15.44:0/3810694341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d8246d710 0x557d8246daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.089+0000 7f45caffd700 1 -- 172.21.15.44:0/3810694341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d821ebde0 con 0x557d8246e030 2023-09-10T00:41:34.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.089+0000 7f45caffd700 1 --2- 172.21.15.44:0/3810694341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246e030 0x557d82477900 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f45c000fc60 tx=0x7f45c0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e977877da6fa250 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.089+0000 7f45b1ffb700 1 -- 172.21.15.44:0/3810694341 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0011ad0 con 0x557d8246e030 2023-09-10T00:41:34.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.089+0000 7f45b1ffb700 1 -- 172.21.15.44:0/3810694341 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45c0011c30 con 0x557d8246e030 2023-09-10T00:41:34.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.089+0000 7f45b1ffb700 1 -- 172.21.15.44:0/3810694341 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c001d5d0 con 0x557d8246e030 2023-09-10T00:41:34.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.090+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3810694341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246e030 msgr2=0x557d82477900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.090+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/3810694341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246e030 0x557d82477900 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f45c000fc60 tx=0x7f45c0012000 comp rx=0 tx=0).stop 2023-09-10T00:41:34.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.090+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3810694341 shutdown_connections 2023-09-10T00:41:34.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.090+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/3810694341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d82477e40 0x557d8247a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.090+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/3810694341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d8246d710 0x557d8246daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.090+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/3810694341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246e030 0x557d82477900 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.090+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3810694341 >> 172.21.15.44:0/3810694341 conn(0x557d823e2090 msgr2=0x557d82468bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.090+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3810694341 shutdown_connections 2023-09-10T00:41:34.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.090+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3810694341 wait complete. 2023-09-10T00:41:34.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.090+0000 7f45de0ceb80 1 -- start start 2023-09-10T00:41:34.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45de0ceb80 1 Processor -- start 2023-09-10T00:41:34.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45de0ceb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246d710 0x557d82477880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45de0ceb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d8246e030 0x557d82474b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45de0ceb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d82477e40 0x557d82475240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45de0ceb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d8235db70 con 0x557d8246d710 2023-09-10T00:41:34.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45de0ceb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d8235d870 con 0x557d82477e40 2023-09-10T00:41:34.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45de0ceb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d82241920 con 0x557d8246e030 2023-09-10T00:41:34.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45de0ceb80 10 client.0 _ll_register_callbacks cb 0x557d822a7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:34.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45cb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246d710 0x557d82477880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45cb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246d710 0x557d82477880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49610/0 (socket says 172.21.15.44:49610) 2023-09-10T00:41:34.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45cb7fe700 1 -- 172.21.15.44:0/1753027677 learned_addr learned my addr 172.21.15.44:0/1753027677 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45caffd700 1 --2- 172.21.15.44:0/1753027677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d8246e030 0x557d82474b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45cb7fe700 1 -- 172.21.15.44:0/1753027677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d8246e030 msgr2=0x557d82474b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45cb7fe700 1 --2- 172.21.15.44:0/1753027677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d8246e030 0x557d82474b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45cb7fe700 1 -- 172.21.15.44:0/1753027677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d82477e40 msgr2=0x557d82475240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45cb7fe700 1 --2- 172.21.15.44:0/1753027677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d82477e40 0x557d82475240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.091+0000 7f45cb7fe700 1 -- 172.21.15.44:0/1753027677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c000f940 con 0x557d8246d710 2023-09-10T00:41:34.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.092+0000 7f45cb7fe700 1 --2- 172.21.15.44:0/1753027677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246d710 0x557d82477880 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00c010 tx=0x7f45bc09ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c280dc3c29f5b558 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.092+0000 7f45aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:34.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.092+0000 7f45aaffd700 1 -- 172.21.15.44:0/1753027677 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc09e370 con 0x557d8246d710 2023-09-10T00:41:34.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.092+0000 7f45aaffd700 1 -- 172.21.15.44:0/1753027677 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f45bc0a2520 con 0x557d8246d710 2023-09-10T00:41:34.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.092+0000 7f45de0ceb80 1 -- 172.21.15.44:0/1753027677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d823174f0 con 0x557d8246d710 2023-09-10T00:41:34.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.092+0000 7f45aaffd700 1 -- 172.21.15.44:0/1753027677 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45bc09e030 con 0x557d8246d710 2023-09-10T00:41:34.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.092+0000 7f45de0ceb80 10 client.16137 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:34.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.092+0000 7f45de0ceb80 1 -- 172.21.15.44:0/1753027677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557d821d4c60 con 0x557d8246d710 2023-09-10T00:41:34.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.092+0000 7f45aaffd700 1 -- 172.21.15.44:0/1753027677 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45bc064080 con 0x557d8246d710 2023-09-10T00:41:34.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.092+0000 7f45aaffd700 1 -- 172.21.15.44:0/1753027677 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f45bc066140 con 0x557d8246d710 2023-09-10T00:41:34.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.092+0000 7f45de0ceb80 10 client.16137 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:34.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.092+0000 7f45de0ceb80 10 client.16137 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2023-09-10T00:41:34.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.092+0000 7f45de0ceb80 20 client.16137 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:34.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.092+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/1753027677 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x557d82487610 0x557d82489a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.092+0000 7f45de0ceb80 4 client.16137 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:34.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.092+0000 7f45de0ceb80 1 -- 172.21.15.44:0/1753027677 --> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d823d48a0 con 0x557d82487610 2023-09-10T00:41:34.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.093+0000 7f45caffd700 1 --2- 172.21.15.44:0/1753027677 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x557d82487610 0x557d82489a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.093+0000 7f45aaffd700 10 client.16137 ms_handle_connect on v2:172.21.15.131:6832/3395466517 2023-09-10T00:41:34.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.093+0000 7f45caffd700 1 --2- 172.21.15.44:0/1753027677 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x557d82487610 0x557d82489a30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6ededa5b16296946 server_cookie=be7e12d59417c473 in_seq=0 out_seq=0 2023-09-10T00:41:34.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.094+0000 7f45aaffd700 1 -- 172.21.15.44:0/1753027677 <== mds.4 v2:172.21.15.131:6832/3395466517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557d823d48a0 con 0x557d82487610 2023-09-10T00:41:34.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.094+0000 7f45aaffd700 10 client.16137 handle_command_reply: tid=0 2023-09-10T00:41:34.113 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.094+0000 7f45de0ceb80 1 client.16137 shutdown 2023-09-10T00:41:34.113 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.094+0000 7f45de0ceb80 1 -- 172.21.15.44:0/1753027677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246d710 msgr2=0x557d82477880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.113 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.094+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/1753027677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246d710 0x557d82477880 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00c010 tx=0x7f45bc09ad80 comp rx=0 tx=0).stop 2023-09-10T00:41:34.113 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.094+0000 7f45de0ceb80 1 -- 172.21.15.44:0/1753027677 shutdown_connections 2023-09-10T00:41:34.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.094+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/1753027677 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x557d82487610 0x557d82489a30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.094+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/1753027677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d82477e40 0x557d82475240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.094+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/1753027677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246d710 0x557d82477880 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.094+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/1753027677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d8246e030 0x557d82474b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.094+0000 7f45aaffd700 0 client.16137 ms_handle_reset on v2:172.21.15.131:6832/3395466517 2023-09-10T00:41:34.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.094+0000 7f45de0ceb80 1 -- 172.21.15.44:0/1753027677 >> 172.21.15.44:0/1753027677 conn(0x557d823e2090 msgr2=0x557d824687e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.094+0000 7f45de0ceb80 1 -- 172.21.15.44:0/1753027677 shutdown_connections 2023-09-10T00:41:34.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.094+0000 7f45de0ceb80 1 -- 172.21.15.44:0/1753027677 wait complete. 2023-09-10T00:41:34.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.094+0000 7f45de0ceb80 20 client.16137 trim_cache size 0 max 16384 2023-09-10T00:41:34.121 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.113+0000 7f45de0ceb80 1 Processor -- start 2023-09-10T00:41:34.121 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.113+0000 7f45de0ceb80 1 -- start start 2023-09-10T00:41:34.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.113+0000 7f45de0ceb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246d710 0x557d82477420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.113+0000 7f45de0ceb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d8246e030 0x557d8246fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.113+0000 7f45de0ceb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d82477e40 0x557d82470340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.113+0000 7f45de0ceb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d8246e800 con 0x557d8246d710 2023-09-10T00:41:34.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.113+0000 7f45de0ceb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d82473060 con 0x557d82477e40 2023-09-10T00:41:34.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.113+0000 7f45de0ceb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d824696e0 con 0x557d8246e030 2023-09-10T00:41:34.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.114+0000 7f45cb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246d710 0x557d82477420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.114+0000 7f45cb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246d710 0x557d82477420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49618/0 (socket says 172.21.15.44:49618) 2023-09-10T00:41:34.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.114+0000 7f45cb7fe700 1 -- 172.21.15.44:0/3021849633 learned_addr learned my addr 172.21.15.44:0/3021849633 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.114+0000 7f45caffd700 1 --2- 172.21.15.44:0/3021849633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d8246e030 0x557d8246fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.114+0000 7f45cbfff700 1 --2- 172.21.15.44:0/3021849633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d82477e40 0x557d82470340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.114+0000 7f45cb7fe700 1 -- 172.21.15.44:0/3021849633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d8246e030 msgr2=0x557d8246fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.114+0000 7f45cb7fe700 1 --2- 172.21.15.44:0/3021849633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d8246e030 0x557d8246fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.114+0000 7f45cb7fe700 1 -- 172.21.15.44:0/3021849633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d82477e40 msgr2=0x557d82470340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.114+0000 7f45cb7fe700 1 --2- 172.21.15.44:0/3021849633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d82477e40 0x557d82470340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.114+0000 7f45cb7fe700 1 -- 172.21.15.44:0/3021849633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d82212250 con 0x557d8246d710 2023-09-10T00:41:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.114+0000 7f45cb7fe700 1 --2- 172.21.15.44:0/3021849633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246d710 0x557d82477420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f45bc096a40 tx=0x7f45bc09a390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deeb97a37eb5f4e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.114+0000 7f45b27fc700 1 -- 172.21.15.44:0/3021849633 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc0a2520 con 0x557d8246d710 2023-09-10T00:41:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.114+0000 7f45b27fc700 1 -- 172.21.15.44:0/3021849633 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557d823174f0 con 0x557d8246d710 2023-09-10T00:41:34.126 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.114+0000 7f45b27fc700 1 -- 172.21.15.44:0/3021849633 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc0a2090 con 0x557d8246d710 2023-09-10T00:41:34.126 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.115+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3021849633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246d710 msgr2=0x557d82477420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.126 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.115+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/3021849633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246d710 0x557d82477420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f45bc096a40 tx=0x7f45bc09a390 comp rx=0 tx=0).stop 2023-09-10T00:41:34.126 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.115+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3021849633 shutdown_connections 2023-09-10T00:41:34.126 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.115+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/3021849633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d82477e40 0x557d82470340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.127 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.115+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/3021849633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246d710 0x557d82477420 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.127 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.115+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/3021849633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d8246e030 0x557d8246fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.127 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.115+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3021849633 >> 172.21.15.44:0/3021849633 conn(0x557d823e2090 msgr2=0x557d82483140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.127 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.115+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3021849633 shutdown_connections 2023-09-10T00:41:34.127 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.115+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3021849633 wait complete. 2023-09-10T00:41:34.127 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.116+0000 7f45de0ceb80 1 -- start start 2023-09-10T00:41:34.128 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.116+0000 7f45de0ceb80 1 Processor -- start 2023-09-10T00:41:34.128 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.116+0000 7f45de0ceb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d8246d710 0x557d8247e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.128 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.116+0000 7f45de0ceb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246e030 0x557d8247c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.128 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.116+0000 7f45de0ceb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d82477e40 0x557d8247cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.128 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.116+0000 7f45de0ceb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d8247ef80 con 0x557d8246e030 2023-09-10T00:41:34.129 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.116+0000 7f45de0ceb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d8247d450 con 0x557d8246d710 2023-09-10T00:41:34.129 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.116+0000 7f45de0ceb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d8247d5e0 con 0x557d82477e40 2023-09-10T00:41:34.129 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.116+0000 7f45de0ceb80 10 client.0 _ll_register_callbacks cb 0x7f45bc00f460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:34.129 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.116+0000 7f45caffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246e030 0x557d8247c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.129 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.116+0000 7f45caffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246e030 0x557d8247c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49626/0 (socket says 172.21.15.44:49626) 2023-09-10T00:41:34.129 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.116+0000 7f45caffd700 1 -- 172.21.15.44:0/3300041360 learned_addr learned my addr 172.21.15.44:0/3300041360 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.130 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.116+0000 7f45cb7fe700 1 --2- 172.21.15.44:0/3300041360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d8246d710 0x557d8247e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.130 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.116+0000 7f45cbfff700 1 --2- 172.21.15.44:0/3300041360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d82477e40 0x557d8247cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.130 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.117+0000 7f45caffd700 1 -- 172.21.15.44:0/3300041360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d82477e40 msgr2=0x557d8247cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.130 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.117+0000 7f45caffd700 1 --2- 172.21.15.44:0/3300041360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d82477e40 0x557d8247cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.130 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.117+0000 7f45caffd700 1 -- 172.21.15.44:0/3300041360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d8246d710 msgr2=0x557d8247e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.117+0000 7f45caffd700 1 --2- 172.21.15.44:0/3300041360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d8246d710 0x557d8247e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.117+0000 7f45caffd700 1 -- 172.21.15.44:0/3300041360 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d821d4c60 con 0x557d8246e030 2023-09-10T00:41:34.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.117+0000 7f45cb7fe700 1 --2- 172.21.15.44:0/3300041360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d8246d710 0x557d8247e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:34.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.117+0000 7f45caffd700 1 --2- 172.21.15.44:0/3300041360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246e030 0x557d8247c9d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f45c000fc30 tx=0x7f45c000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b19051d0301e14 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.117+0000 7f45aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:34.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.117+0000 7f45aaffd700 1 -- 172.21.15.44:0/3300041360 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c002d030 con 0x557d8246e030 2023-09-10T00:41:34.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.117+0000 7f45aaffd700 1 -- 172.21.15.44:0/3300041360 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f45c0034980 con 0x557d8246e030 2023-09-10T00:41:34.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.117+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3300041360 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45c0011c30 con 0x557d8246e030 2023-09-10T00:41:34.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.117+0000 7f45aaffd700 1 -- 172.21.15.44:0/3300041360 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45c0011ad0 con 0x557d8246e030 2023-09-10T00:41:34.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.117+0000 7f45de0ceb80 10 client.16143 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:34.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.117+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3300041360 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557d821ea2b0 con 0x557d8246e030 2023-09-10T00:41:34.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.117+0000 7f45aaffd700 1 -- 172.21.15.44:0/3300041360 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45c0011c30 con 0x557d8246e030 2023-09-10T00:41:34.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.118+0000 7f45aaffd700 1 -- 172.21.15.44:0/3300041360 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f45c000fc90 con 0x557d8246e030 2023-09-10T00:41:34.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.118+0000 7f45de0ceb80 10 client.16143 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:34.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.118+0000 7f45de0ceb80 10 client.16143 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2023-09-10T00:41:34.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.118+0000 7f45de0ceb80 20 client.16143 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:34.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.118+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/3300041360 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x557d824e0a70 0x557d824e2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.134 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.118+0000 7f45de0ceb80 4 client.16143 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2023-09-10T00:41:34.134 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.118+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3300041360 --> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d823b8920 con 0x557d824e0a70 2023-09-10T00:41:34.134 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.118+0000 7f45cb7fe700 1 --2- 172.21.15.44:0/3300041360 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x557d824e0a70 0x557d824e2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.134 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.118+0000 7f45aaffd700 10 client.16143 ms_handle_connect on v2:172.21.15.131:6832/3395466517 2023-09-10T00:41:34.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.118+0000 7f45cb7fe700 1 --2- 172.21.15.44:0/3300041360 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x557d824e0a70 0x557d824e2e90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b3f671dca1838506 server_cookie=b74e5a6494ea57c4 in_seq=0 out_seq=0 2023-09-10T00:41:34.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.119+0000 7f45aaffd700 1 -- 172.21.15.44:0/3300041360 <== mds.4 v2:172.21.15.131:6832/3395466517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d823b8920 con 0x557d824e0a70 2023-09-10T00:41:34.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.119+0000 7f45aaffd700 10 client.16143 handle_command_reply: tid=0 2023-09-10T00:41:34.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.119+0000 7f45de0ceb80 1 client.16143 shutdown 2023-09-10T00:41:34.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.119+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3300041360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246e030 msgr2=0x557d8247c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.119+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/3300041360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246e030 0x557d8247c9d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f45c000fc30 tx=0x7f45c000b8e0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.119+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3300041360 shutdown_connections 2023-09-10T00:41:34.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.119+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/3300041360 >> [v2:172.21.15.131:6832/3395466517,v1:172.21.15.131:6833/3395466517] conn(0x557d824e0a70 0x557d824e2e90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.119+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/3300041360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d82477e40 0x557d8247cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.119+0000 7f45aaffd700 0 client.16143 ms_handle_reset on v2:172.21.15.131:6832/3395466517 2023-09-10T00:41:34.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.119+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/3300041360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d8246d710 0x557d8247e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.119+0000 7f45de0ceb80 1 --2- 172.21.15.44:0/3300041360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557d8246e030 0x557d8247c9d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.119+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3300041360 >> 172.21.15.44:0/3300041360 conn(0x557d823e2090 msgr2=0x557d8246fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.119+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3300041360 shutdown_connections 2023-09-10T00:41:34.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.119+0000 7f45de0ceb80 1 -- 172.21.15.44:0/3300041360 wait complete. 2023-09-10T00:41:34.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.119+0000 7f45de0ceb80 20 client.16143 trim_cache size 0 max 16384 2023-09-10T00:41:34.138 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2023-09-10T00:41:34.138 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.120+0000 7f45d1f08700 1 -- 172.21.15.44:0/4157744098 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45b40a08d0 msgr2=0x7f45b40a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.120+0000 7f45d1f08700 1 --2- 172.21.15.44:0/4157744098 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45b40a08d0 0x7f45b40a2d90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f45c0005fd0 tx=0x7f45c0005f60 comp rx=0 tx=0).stop 2023-09-10T00:41:34.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.120+0000 7f45d1f08700 1 -- 172.21.15.44:0/4157744098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc1017c0 msgr2=0x7f45cc196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.120+0000 7f45d1f08700 1 --2- 172.21.15.44:0/4157744098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc1017c0 0x7f45cc196610 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f45bc005910 tx=0x7f45bc009580 comp rx=0 tx=0).stop 2023-09-10T00:41:34.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.120+0000 7f45d1f08700 1 -- 172.21.15.44:0/4157744098 shutdown_connections 2023-09-10T00:41:34.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.120+0000 7f45d1f08700 1 --2- 172.21.15.44:0/4157744098 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45b40a08d0 0x7f45b40a2d90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.120+0000 7f45d1f08700 1 --2- 172.21.15.44:0/4157744098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45cc10ab90 0x7f45cc19afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.120+0000 7f45d1f08700 1 --2- 172.21.15.44:0/4157744098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc1017c0 0x7f45cc196610 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.120+0000 7f45d1f08700 1 --2- 172.21.15.44:0/4157744098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45cc1020e0 0x7f45cc196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.120+0000 7f45d1f08700 1 -- 172.21.15.44:0/4157744098 >> 172.21.15.44:0/4157744098 conn(0x7f45cc0fafa0 msgr2=0x7f45cc0fbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.120+0000 7f45d1f08700 1 -- 172.21.15.44:0/4157744098 shutdown_connections 2023-09-10T00:41:34.141 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.120+0000 7f45d1f08700 1 -- 172.21.15.44:0/4157744098 wait complete. 2023-09-10T00:41:34.141 INFO:teuthology.orchestra.run.smithi044.stderr:3000000 of 50000000 tuples (6%) done (elapsed 33.77 s, remaining 529.02 s) 2023-09-10T00:41:34.142 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-10T00:41:34.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.281+0000 7fe939f26700 1 -- 172.21.15.44:0/1230505444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9340feaa0 msgr2=0x7fe9340fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.281+0000 7fe939f26700 1 --2- 172.21.15.44:0/1230505444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9340feaa0 0x7fe9340fef20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe928009990 tx=0x7fe928009ca0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.281+0000 7fe939f26700 1 -- 172.21.15.44:0/1230505444 shutdown_connections 2023-09-10T00:41:34.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.281+0000 7fe939f26700 1 --2- 172.21.15.44:0/1230505444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9340ff460 0x7fe93410d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.281+0000 7fe939f26700 1 --2- 172.21.15.44:0/1230505444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe934106620 0x7fe934106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.281+0000 7fe939f26700 1 --2- 172.21.15.44:0/1230505444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9340feaa0 0x7fe9340fef20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.281+0000 7fe939f26700 1 -- 172.21.15.44:0/1230505444 >> 172.21.15.44:0/1230505444 conn(0x7fe934076a70 msgr2=0x7fe9340fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.281+0000 7fe939f26700 1 -- 172.21.15.44:0/1230505444 shutdown_connections 2023-09-10T00:41:34.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.281+0000 7fe939f26700 1 -- 172.21.15.44:0/1230505444 wait complete. 2023-09-10T00:41:34.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.282+0000 7fe939f26700 1 Processor -- start 2023-09-10T00:41:34.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.282+0000 7fe939f26700 1 -- start start 2023-09-10T00:41:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe939f26700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9340feaa0 0x7fe9340ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe939f26700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9340ff460 0x7fe9341001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe939f26700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe934106620 0x7fe9341a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe939f26700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9341009f0 con 0x7fe9340feaa0 2023-09-10T00:41:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe939f26700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9341a4cd0 con 0x7fe9340ff460 2023-09-10T00:41:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe939f26700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9341a4e10 con 0x7fe934106620 2023-09-10T00:41:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe9337fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9340feaa0 0x7fe9340ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe9337fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9340feaa0 0x7fe9340ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49638/0 (socket says 172.21.15.44:49638) 2023-09-10T00:41:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe9337fe700 1 -- 172.21.15.44:0/1206399140 learned_addr learned my addr 172.21.15.44:0/1206399140 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe932ffd700 1 --2- 172.21.15.44:0/1206399140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9340ff460 0x7fe9341001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe933fff700 1 --2- 172.21.15.44:0/1206399140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe934106620 0x7fe9341a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe9337fe700 1 -- 172.21.15.44:0/1206399140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe934106620 msgr2=0x7fe9341a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe9337fe700 1 --2- 172.21.15.44:0/1206399140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe934106620 0x7fe9341a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe9337fe700 1 -- 172.21.15.44:0/1206399140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9340ff460 msgr2=0x7fe9341001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe9337fe700 1 --2- 172.21.15.44:0/1206399140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9340ff460 0x7fe9341001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe9337fe700 1 -- 172.21.15.44:0/1206399140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9280095f0 con 0x7fe9340feaa0 2023-09-10T00:41:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.283+0000 7fe9337fe700 1 --2- 172.21.15.44:0/1206399140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9340feaa0 0x7fe9340ffc80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe92400d950 tx=0x7fe92400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.284+0000 7fe930ff9700 1 -- 172.21.15.44:0/1206399140 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92400f890 con 0x7fe9340feaa0 2023-09-10T00:41:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.284+0000 7fe939f26700 1 -- 172.21.15.44:0/1206399140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9341a5070 con 0x7fe9340feaa0 2023-09-10T00:41:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.284+0000 7fe930ff9700 1 -- 172.21.15.44:0/1206399140 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe92400fe80 con 0x7fe9340feaa0 2023-09-10T00:41:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.284+0000 7fe930ff9700 1 -- 172.21.15.44:0/1206399140 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92400e3b0 con 0x7fe9340feaa0 2023-09-10T00:41:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.284+0000 7fe939f26700 1 -- 172.21.15.44:0/1206399140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9341a55a0 con 0x7fe9340feaa0 2023-09-10T00:41:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.286+0000 7fe930ff9700 1 -- 172.21.15.44:0/1206399140 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7fe924010460 con 0x7fe9340feaa0 2023-09-10T00:41:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.287+0000 7fe930ff9700 1 --2- 172.21.15.44:0/1206399140 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe91c0a08d0 0x7fe91c0a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.287+0000 7fe932ffd700 1 --2- 172.21.15.44:0/1206399140 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe91c0a08d0 0x7fe91c0a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.287+0000 7fe930ff9700 1 -- 172.21.15.44:0/1206399140 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe9240cc350 con 0x7fe9340feaa0 2023-09-10T00:41:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.287+0000 7fe932ffd700 1 --2- 172.21.15.44:0/1206399140 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe91c0a08d0 0x7fe91c0a2d90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe934101470 tx=0x7fe928005fb0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.288+0000 7fe9460ecb80 1 Processor -- start 2023-09-10T00:41:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.288+0000 7fe9460ecb80 1 -- start start 2023-09-10T00:41:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.288+0000 7fe9460ecb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9ca710 0x5609fa9caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.288+0000 7fe9460ecb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9cb030 0x5609fa9d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.288+0000 7fe9460ecb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9d4e40 0x5609fa9d7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.288+0000 7fe9460ecb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fa923730 con 0x5609fa9ca710 2023-09-10T00:41:34.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.288+0000 7fe9460ecb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fa921050 con 0x5609fa9d4e40 2023-09-10T00:41:34.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.288+0000 7fe9460ecb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fa91c940 con 0x5609fa9cb030 2023-09-10T00:41:34.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.288+0000 7fe9337fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9ca710 0x5609fa9caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.288+0000 7fe9337fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9ca710 0x5609fa9caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49646/0 (socket says 172.21.15.44:49646) 2023-09-10T00:41:34.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.288+0000 7fe9337fe700 1 -- 172.21.15.44:0/558190331 learned_addr learned my addr 172.21.15.44:0/558190331 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.288+0000 7fe932ffd700 1 --2- 172.21.15.44:0/558190331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9cb030 0x5609fa9d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.289+0000 7fe933fff700 1 --2- 172.21.15.44:0/558190331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9d4e40 0x5609fa9d7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.289+0000 7fe9337fe700 1 -- 172.21.15.44:0/558190331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9cb030 msgr2=0x5609fa9d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.289+0000 7fe9337fe700 1 --2- 172.21.15.44:0/558190331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9cb030 0x5609fa9d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.289+0000 7fe9337fe700 1 -- 172.21.15.44:0/558190331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9d4e40 msgr2=0x5609fa9d7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.289+0000 7fe9337fe700 1 --2- 172.21.15.44:0/558190331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9d4e40 0x5609fa9d7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.289+0000 7fe9337fe700 1 -- 172.21.15.44:0/558190331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609fa748de0 con 0x5609fa9ca710 2023-09-10T00:41:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.289+0000 7fe932ffd700 1 --2- 172.21.15.44:0/558190331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9cb030 0x5609fa9d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:34.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.289+0000 7fe9337fe700 1 --2- 172.21.15.44:0/558190331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9ca710 0x5609fa9caaf0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe92400b2f0 tx=0x7fe924098c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da7746da2fd0667 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.289+0000 7fe919ffb700 1 -- 172.21.15.44:0/558190331 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9240cc170 con 0x5609fa9ca710 2023-09-10T00:41:34.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.289+0000 7fe919ffb700 1 -- 172.21.15.44:0/558190331 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe924099210 con 0x5609fa9ca710 2023-09-10T00:41:34.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.289+0000 7fe919ffb700 1 -- 172.21.15.44:0/558190331 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9240989d0 con 0x5609fa9ca710 2023-09-10T00:41:34.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.290+0000 7fe9460ecb80 1 -- 172.21.15.44:0/558190331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9ca710 msgr2=0x5609fa9caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.290+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/558190331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9ca710 0x5609fa9caaf0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe92400b2f0 tx=0x7fe924098c40 comp rx=0 tx=0).stop 2023-09-10T00:41:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.290+0000 7fe9460ecb80 1 -- 172.21.15.44:0/558190331 shutdown_connections 2023-09-10T00:41:34.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.290+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/558190331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9d4e40 0x5609fa9d7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.290+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/558190331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9ca710 0x5609fa9caaf0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.290+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/558190331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9cb030 0x5609fa9d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.290+0000 7fe9460ecb80 1 -- 172.21.15.44:0/558190331 >> 172.21.15.44:0/558190331 conn(0x5609fa93f090 msgr2=0x5609fa9c5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.290+0000 7fe9460ecb80 1 -- 172.21.15.44:0/558190331 shutdown_connections 2023-09-10T00:41:34.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.290+0000 7fe9460ecb80 1 -- 172.21.15.44:0/558190331 wait complete. 2023-09-10T00:41:34.308 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.290+0000 7fe9460ecb80 1 -- start start 2023-09-10T00:41:34.308 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe9460ecb80 1 Processor -- start 2023-09-10T00:41:34.308 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe9460ecb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9ca710 0x5609fa9d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.308 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe9460ecb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9cb030 0x5609fa9d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.309 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe9460ecb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9d4e40 0x5609fa9d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.309 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe9460ecb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fa8bab70 con 0x5609fa9d4e40 2023-09-10T00:41:34.309 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe9460ecb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fa8ba870 con 0x5609fa9cb030 2023-09-10T00:41:34.309 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe9460ecb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fa79e920 con 0x5609fa9ca710 2023-09-10T00:41:34.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe9460ecb80 10 client.0 _ll_register_callbacks cb 0x5609fa804a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:34.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe933fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9d4e40 0x5609fa9d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe933fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9d4e40 0x5609fa9d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49656/0 (socket says 172.21.15.44:49656) 2023-09-10T00:41:34.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe933fff700 1 -- 172.21.15.44:0/2412441552 learned_addr learned my addr 172.21.15.44:0/2412441552 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe9337fe700 1 --2- 172.21.15.44:0/2412441552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9ca710 0x5609fa9d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe932ffd700 1 --2- 172.21.15.44:0/2412441552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9cb030 0x5609fa9d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe933fff700 1 -- 172.21.15.44:0/2412441552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9ca710 msgr2=0x5609fa9d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe933fff700 1 --2- 172.21.15.44:0/2412441552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9ca710 0x5609fa9d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe933fff700 1 -- 172.21.15.44:0/2412441552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9cb030 msgr2=0x5609fa9d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe933fff700 1 --2- 172.21.15.44:0/2412441552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9cb030 0x5609fa9d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.291+0000 7fe933fff700 1 -- 172.21.15.44:0/2412441552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9341a55a0 con 0x5609fa9d4e40 2023-09-10T00:41:34.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.292+0000 7fe933fff700 1 --2- 172.21.15.44:0/2412441552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9d4e40 0x5609fa9d25e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0042f0 tx=0x7fe92c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10bb3d8aa6d8f78 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.292+0000 7fe912ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:34.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.292+0000 7fe912ffd700 1 -- 172.21.15.44:0/2412441552 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c0078b0 con 0x5609fa9d4e40 2023-09-10T00:41:34.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.292+0000 7fe9460ecb80 1 -- 172.21.15.44:0/2412441552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609fa8744f0 con 0x5609fa9d4e40 2023-09-10T00:41:34.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.292+0000 7fe912ffd700 1 -- 172.21.15.44:0/2412441552 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe92c007a10 con 0x5609fa9d4e40 2023-09-10T00:41:34.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.292+0000 7fe912ffd700 1 -- 172.21.15.44:0/2412441552 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe92c007bd0 con 0x5609fa9d4e40 2023-09-10T00:41:34.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.292+0000 7fe9460ecb80 10 client.16155 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:34.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.292+0000 7fe9460ecb80 1 -- 172.21.15.44:0/2412441552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5609fa731c60 con 0x5609fa9d4e40 2023-09-10T00:41:34.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.292+0000 7fe912ffd700 1 -- 172.21.15.44:0/2412441552 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5609fa8744f0 con 0x5609fa9d4e40 2023-09-10T00:41:34.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.293+0000 7fe912ffd700 1 -- 172.21.15.44:0/2412441552 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe92c014040 con 0x5609fa9d4e40 2023-09-10T00:41:34.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.293+0000 7fe9460ecb80 10 client.16155 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:34.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.293+0000 7fe9460ecb80 10 client.16155 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2023-09-10T00:41:34.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.293+0000 7fe9460ecb80 20 client.16155 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:34.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.293+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/2412441552 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x5609fa9e4820 0x5609fa9e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.293+0000 7fe9460ecb80 4 client.16155 mds_command: new command op to 24515 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:34.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.293+0000 7fe9460ecb80 1 -- 172.21.15.44:0/2412441552 --> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609fa9318a0 con 0x5609fa9e4820 2023-09-10T00:41:34.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.294+0000 7fe9337fe700 1 --2- 172.21.15.44:0/2412441552 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x5609fa9e4820 0x5609fa9e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.294+0000 7fe9337fe700 1 --2- 172.21.15.44:0/2412441552 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x5609fa9e4820 0x5609fa9e6c10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=40b267fffecb58f6 server_cookie=63c12498ab92dcb in_seq=0 out_seq=0 2023-09-10T00:41:34.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.294+0000 7fe912ffd700 10 client.16155 ms_handle_connect on v2:172.21.15.99:6832/1934109590 2023-09-10T00:41:34.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.295+0000 7fe912ffd700 1 -- 172.21.15.44:0/2412441552 <== mds.3 v2:172.21.15.99:6832/1934109590 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5609fa9318a0 con 0x5609fa9e4820 2023-09-10T00:41:34.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.295+0000 7fe912ffd700 10 client.16155 handle_command_reply: tid=0 2023-09-10T00:41:34.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.295+0000 7fe9460ecb80 1 client.16155 shutdown 2023-09-10T00:41:34.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.295+0000 7fe9460ecb80 1 -- 172.21.15.44:0/2412441552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9d4e40 msgr2=0x5609fa9d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.295+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/2412441552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9d4e40 0x5609fa9d25e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0042f0 tx=0x7fe92c004600 comp rx=0 tx=0).stop 2023-09-10T00:41:34.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.295+0000 7fe9460ecb80 1 -- 172.21.15.44:0/2412441552 shutdown_connections 2023-09-10T00:41:34.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.295+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/2412441552 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x5609fa9e4820 0x5609fa9e6c10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.295+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/2412441552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9d4e40 0x5609fa9d25e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.295+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/2412441552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9ca710 0x5609fa9d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.295+0000 7fe912ffd700 0 client.16155 ms_handle_reset on v2:172.21.15.99:6832/1934109590 2023-09-10T00:41:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.295+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/2412441552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9cb030 0x5609fa9d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.295+0000 7fe9460ecb80 1 -- 172.21.15.44:0/2412441552 >> 172.21.15.44:0/2412441552 conn(0x5609fa93f090 msgr2=0x5609fa9c57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.295+0000 7fe9460ecb80 1 -- 172.21.15.44:0/2412441552 shutdown_connections 2023-09-10T00:41:34.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.295+0000 7fe9460ecb80 1 -- 172.21.15.44:0/2412441552 wait complete. 2023-09-10T00:41:34.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.295+0000 7fe9460ecb80 20 client.16155 trim_cache size 0 max 16384 2023-09-10T00:41:34.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.311+0000 7fe9460ecb80 1 Processor -- start 2023-09-10T00:41:34.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.311+0000 7fe9460ecb80 1 -- start start 2023-09-10T00:41:34.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.311+0000 7fe9460ecb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9ca710 0x5609fa9db820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.311+0000 7fe9460ecb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9cb030 0x5609fa9c4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.311+0000 7fe9460ecb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9d4e40 0x5609fa9cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.311+0000 7fe9460ecb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fa9c2440 con 0x5609fa9cb030 2023-09-10T00:41:34.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.311+0000 7fe9460ecb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fa9e4630 con 0x5609fa9ca710 2023-09-10T00:41:34.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.311+0000 7fe9460ecb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fa9e1f10 con 0x5609fa9d4e40 2023-09-10T00:41:34.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.311+0000 7fe932ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9cb030 0x5609fa9c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.311+0000 7fe933fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9d4e40 0x5609fa9cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.311+0000 7fe932ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9cb030 0x5609fa9c4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49662/0 (socket says 172.21.15.44:49662) 2023-09-10T00:41:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.311+0000 7fe933fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9d4e40 0x5609fa9cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56546/0 (socket says 172.21.15.44:56546) 2023-09-10T00:41:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.311+0000 7fe932ffd700 1 -- 172.21.15.44:0/3429899668 learned_addr learned my addr 172.21.15.44:0/3429899668 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.312+0000 7fe9337fe700 1 --2- 172.21.15.44:0/3429899668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9ca710 0x5609fa9db820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.312+0000 7fe933fff700 1 -- 172.21.15.44:0/3429899668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9ca710 msgr2=0x5609fa9db820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.312+0000 7fe933fff700 1 --2- 172.21.15.44:0/3429899668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9ca710 0x5609fa9db820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.312+0000 7fe933fff700 1 -- 172.21.15.44:0/3429899668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9cb030 msgr2=0x5609fa9c4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.312+0000 7fe933fff700 1 --2- 172.21.15.44:0/3429899668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9cb030 0x5609fa9c4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.312+0000 7fe933fff700 1 -- 172.21.15.44:0/3429899668 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609fa76f250 con 0x5609fa9d4e40 2023-09-10T00:41:34.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.312+0000 7fe933fff700 1 --2- 172.21.15.44:0/3429899668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9d4e40 0x5609fa9cce40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe92c003c20 tx=0x7fe92c009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc4b094e5377b11e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.312+0000 7fe91a7fc700 1 -- 172.21.15.44:0/3429899668 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c0164e0 con 0x5609fa9d4e40 2023-09-10T00:41:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.312+0000 7fe91a7fc700 1 -- 172.21.15.44:0/3429899668 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe92c00f040 con 0x5609fa9d4e40 2023-09-10T00:41:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.312+0000 7fe91a7fc700 1 -- 172.21.15.44:0/3429899668 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c0076b0 con 0x5609fa9d4e40 2023-09-10T00:41:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.313+0000 7fe9460ecb80 1 -- 172.21.15.44:0/3429899668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9d4e40 msgr2=0x5609fa9cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.313+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/3429899668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9d4e40 0x5609fa9cce40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe92c003c20 tx=0x7fe92c009570 comp rx=0 tx=0).stop 2023-09-10T00:41:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.314+0000 7fe9460ecb80 1 -- 172.21.15.44:0/3429899668 shutdown_connections 2023-09-10T00:41:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.314+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/3429899668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9d4e40 0x5609fa9cce40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.314+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/3429899668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9ca710 0x5609fa9db820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.314+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/3429899668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9cb030 0x5609fa9c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.314+0000 7fe9460ecb80 1 -- 172.21.15.44:0/3429899668 >> 172.21.15.44:0/3429899668 conn(0x5609fa93f090 msgr2=0x5609fa9eb570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.314+0000 7fe9460ecb80 1 -- 172.21.15.44:0/3429899668 shutdown_connections 2023-09-10T00:41:34.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.314+0000 7fe9460ecb80 1 -- 172.21.15.44:0/3429899668 wait complete. 2023-09-10T00:41:34.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.314+0000 7fe9460ecb80 1 -- start start 2023-09-10T00:41:34.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.314+0000 7fe9460ecb80 1 Processor -- start 2023-09-10T00:41:34.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.315+0000 7fe9460ecb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9ca710 0x5609fa9db020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.315+0000 7fe9460ecb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9cb030 0x5609fa9db560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.333 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.315+0000 7fe9460ecb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9d4e40 0x5609fa9d94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.333 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.315+0000 7fe9460ecb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fa9d9ab0 con 0x5609fa9cb030 2023-09-10T00:41:34.333 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.315+0000 7fe9460ecb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fa9d9c20 con 0x5609fa9ca710 2023-09-10T00:41:34.333 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.315+0000 7fe9460ecb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fa9d9de0 con 0x5609fa9d4e40 2023-09-10T00:41:34.333 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.315+0000 7fe9460ecb80 10 client.0 _ll_register_callbacks cb 0x7fe92c009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:34.333 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.315+0000 7fe932ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9cb030 0x5609fa9db560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.315+0000 7fe932ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9cb030 0x5609fa9db560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49670/0 (socket says 172.21.15.44:49670) 2023-09-10T00:41:34.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.315+0000 7fe932ffd700 1 -- 172.21.15.44:0/482253654 learned_addr learned my addr 172.21.15.44:0/482253654 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.315+0000 7fe9337fe700 1 --2- 172.21.15.44:0/482253654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9ca710 0x5609fa9db020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.315+0000 7fe933fff700 1 --2- 172.21.15.44:0/482253654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9d4e40 0x5609fa9d94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.316+0000 7fe932ffd700 1 -- 172.21.15.44:0/482253654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9d4e40 msgr2=0x5609fa9d94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.316+0000 7fe932ffd700 1 --2- 172.21.15.44:0/482253654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9d4e40 0x5609fa9d94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.316+0000 7fe932ffd700 1 -- 172.21.15.44:0/482253654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9ca710 msgr2=0x5609fa9db020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.316+0000 7fe932ffd700 1 --2- 172.21.15.44:0/482253654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9ca710 0x5609fa9db020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.316+0000 7fe932ffd700 1 -- 172.21.15.44:0/482253654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609fa731c60 con 0x5609fa9cb030 2023-09-10T00:41:34.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.316+0000 7fe932ffd700 1 --2- 172.21.15.44:0/482253654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9cb030 0x5609fa9db560 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fe9280036c0 tx=0x7fe928009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e6b683a419dae1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.316+0000 7fe912ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.316+0000 7fe912ffd700 1 -- 172.21.15.44:0/482253654 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928004b70 con 0x5609fa9cb030 2023-09-10T00:41:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.316+0000 7fe912ffd700 1 -- 172.21.15.44:0/482253654 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe928004cd0 con 0x5609fa9cb030 2023-09-10T00:41:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.316+0000 7fe9460ecb80 1 -- 172.21.15.44:0/482253654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe924099210 con 0x5609fa9cb030 2023-09-10T00:41:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.316+0000 7fe912ffd700 1 -- 172.21.15.44:0/482253654 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe928004e90 con 0x5609fa9cb030 2023-09-10T00:41:34.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.316+0000 7fe9460ecb80 10 client.16161 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:34.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.316+0000 7fe9460ecb80 1 -- 172.21.15.44:0/482253654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5609fa7472b0 con 0x5609fa9cb030 2023-09-10T00:41:34.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.316+0000 7fe912ffd700 1 -- 172.21.15.44:0/482253654 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe924099210 con 0x5609fa9cb030 2023-09-10T00:41:34.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.317+0000 7fe912ffd700 1 -- 172.21.15.44:0/482253654 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe92800f460 con 0x5609fa9cb030 2023-09-10T00:41:34.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.317+0000 7fe9460ecb80 10 client.16161 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:34.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.317+0000 7fe9460ecb80 10 client.16161 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2023-09-10T00:41:34.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.317+0000 7fe9460ecb80 20 client.16161 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:34.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.317+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/482253654 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x5609faa396e0 0x5609faa3bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.317+0000 7fe9460ecb80 4 client.16161 mds_command: new command op to 24515 tid=0[{"prefix": "damage ls"}] 2023-09-10T00:41:34.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.317+0000 7fe9460ecb80 1 -- 172.21.15.44:0/482253654 --> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5609fa915920 con 0x5609faa396e0 2023-09-10T00:41:34.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.317+0000 7fe9337fe700 1 --2- 172.21.15.44:0/482253654 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x5609faa396e0 0x5609faa3bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.317+0000 7fe912ffd700 10 client.16161 ms_handle_connect on v2:172.21.15.99:6832/1934109590 2023-09-10T00:41:34.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.317+0000 7fe9337fe700 1 --2- 172.21.15.44:0/482253654 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x5609faa396e0 0x5609faa3bb00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9860e613a1c125a1 server_cookie=bcd138b9248420fe in_seq=0 out_seq=0 2023-09-10T00:41:34.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.318+0000 7fe912ffd700 1 -- 172.21.15.44:0/482253654 <== mds.3 v2:172.21.15.99:6832/1934109590 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5609fa915920 con 0x5609faa396e0 2023-09-10T00:41:34.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.318+0000 7fe912ffd700 10 client.16161 handle_command_reply: tid=0 2023-09-10T00:41:34.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.318+0000 7fe9460ecb80 1 client.16161 shutdown 2023-09-10T00:41:34.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.318+0000 7fe9460ecb80 1 -- 172.21.15.44:0/482253654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9cb030 msgr2=0x5609fa9db560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.318+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/482253654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9cb030 0x5609fa9db560 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fe9280036c0 tx=0x7fe928009ca0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.318+0000 7fe9460ecb80 1 -- 172.21.15.44:0/482253654 shutdown_connections 2023-09-10T00:41:34.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.318+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/482253654 >> [v2:172.21.15.99:6832/1934109590,v1:172.21.15.99:6833/1934109590] conn(0x5609faa396e0 0x5609faa3bb00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.318+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/482253654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609fa9d4e40 0x5609fa9d94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.318+0000 7fe912ffd700 0 client.16161 ms_handle_reset on v2:172.21.15.99:6832/1934109590 2023-09-10T00:41:34.342 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.318+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/482253654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609fa9ca710 0x5609fa9db020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.342 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.318+0000 7fe9460ecb80 1 --2- 172.21.15.44:0/482253654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609fa9cb030 0x5609fa9db560 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.342 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.318+0000 7fe9460ecb80 1 -- 172.21.15.44:0/482253654 >> 172.21.15.44:0/482253654 conn(0x5609fa93f090 msgr2=0x5609fa9c4320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.342 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.318+0000 7fe9460ecb80 1 -- 172.21.15.44:0/482253654 shutdown_connections 2023-09-10T00:41:34.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.318+0000 7fe9460ecb80 1 -- 172.21.15.44:0/482253654 wait complete. 2023-09-10T00:41:34.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.318+0000 7fe9460ecb80 20 client.16161 trim_cache size 0 max 16384 2023-09-10T00:41:34.343 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2023-09-10T00:41:34.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.319+0000 7fe939f26700 1 -- 172.21.15.44:0/1206399140 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe91c0a08d0 msgr2=0x7fe91c0a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.319+0000 7fe939f26700 1 --2- 172.21.15.44:0/1206399140 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe91c0a08d0 0x7fe91c0a2d90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe934101470 tx=0x7fe928005fb0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.319+0000 7fe939f26700 1 -- 172.21.15.44:0/1206399140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9340feaa0 msgr2=0x7fe9340ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.319+0000 7fe939f26700 1 --2- 172.21.15.44:0/1206399140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9340feaa0 0x7fe9340ffc80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe92400d950 tx=0x7fe92400dc60 comp rx=0 tx=0).stop 2023-09-10T00:41:34.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.319+0000 7fe939f26700 1 -- 172.21.15.44:0/1206399140 shutdown_connections 2023-09-10T00:41:34.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.319+0000 7fe939f26700 1 --2- 172.21.15.44:0/1206399140 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe91c0a08d0 0x7fe91c0a2d90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.319+0000 7fe939f26700 1 --2- 172.21.15.44:0/1206399140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe934106620 0x7fe9341a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.319+0000 7fe939f26700 1 --2- 172.21.15.44:0/1206399140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9340feaa0 0x7fe9340ffc80 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.319+0000 7fe939f26700 1 --2- 172.21.15.44:0/1206399140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9340ff460 0x7fe9341001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.319+0000 7fe939f26700 1 -- 172.21.15.44:0/1206399140 >> 172.21.15.44:0/1206399140 conn(0x7fe934076a70 msgr2=0x7fe9340fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.319+0000 7fe939f26700 1 -- 172.21.15.44:0/1206399140 shutdown_connections 2023-09-10T00:41:34.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.319+0000 7fe939f26700 1 -- 172.21.15.44:0/1206399140 wait complete. 2023-09-10T00:41:34.349 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-10T00:41:34.501 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.498+0000 7f97031e8700 1 -- 172.21.15.44:0/3846032662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96fc0ffe00 msgr2=0x7f96fc10c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.502 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.498+0000 7f97031e8700 1 --2- 172.21.15.44:0/3846032662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96fc0ffe00 0x7f96fc10c310 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f96f400b0a0 tx=0x7f96f400b3b0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.502 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.498+0000 7f97031e8700 1 -- 172.21.15.44:0/3846032662 shutdown_connections 2023-09-10T00:41:34.502 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.498+0000 7f97031e8700 1 --2- 172.21.15.44:0/3846032662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96fc0ffe00 0x7f96fc10c310 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.498+0000 7f97031e8700 1 --2- 172.21.15.44:0/3846032662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96fc1054e0 0x7f96fc1058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.498+0000 7f97031e8700 1 --2- 172.21.15.44:0/3846032662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96fc0ff440 0x7f96fc0ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.498+0000 7f97031e8700 1 -- 172.21.15.44:0/3846032662 >> 172.21.15.44:0/3846032662 conn(0x7f96fc0faf50 msgr2=0x7f96fc0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.498+0000 7f97031e8700 1 -- 172.21.15.44:0/3846032662 shutdown_connections 2023-09-10T00:41:34.504 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.498+0000 7f97031e8700 1 -- 172.21.15.44:0/3846032662 wait complete. 2023-09-10T00:41:34.504 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.499+0000 7f97031e8700 1 Processor -- start 2023-09-10T00:41:34.504 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.499+0000 7f97031e8700 1 -- start start 2023-09-10T00:41:34.504 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.499+0000 7f97031e8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96fc0ff440 0x7f96fc197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.499+0000 7f97031e8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96fc0ffe00 0x7f96fc197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.499+0000 7f97031e8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96fc1054e0 0x7f96fc1a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.499+0000 7f97031e8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc1981b0 con 0x7f96fc0ffe00 2023-09-10T00:41:34.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.499+0000 7f97031e8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc1a28f0 con 0x7f96fc1054e0 2023-09-10T00:41:34.506 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.499+0000 7f97031e8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc1a2ab0 con 0x7f96fc0ff440 2023-09-10T00:41:34.506 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.499+0000 7f96fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96fc0ffe00 0x7f96fc197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.506 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.499+0000 7f96fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96fc0ffe00 0x7f96fc197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49682/0 (socket says 172.21.15.44:49682) 2023-09-10T00:41:34.506 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.499+0000 7f96fbfff700 1 -- 172.21.15.44:0/3157384151 learned_addr learned my addr 172.21.15.44:0/3157384151 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.499+0000 7f9700f84700 1 --2- 172.21.15.44:0/3157384151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96fc0ff440 0x7f96fc197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.499+0000 7f9701785700 1 --2- 172.21.15.44:0/3157384151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96fc1054e0 0x7f96fc1a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.500+0000 7f96fbfff700 1 -- 172.21.15.44:0/3157384151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96fc0ff440 msgr2=0x7f96fc197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.500+0000 7f96fbfff700 1 --2- 172.21.15.44:0/3157384151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96fc0ff440 0x7f96fc197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.500+0000 7f96fbfff700 1 -- 172.21.15.44:0/3157384151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96fc1054e0 msgr2=0x7f96fc1a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.500+0000 7f96fbfff700 1 --2- 172.21.15.44:0/3157384151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96fc1054e0 0x7f96fc1a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.500+0000 7f96fbfff700 1 -- 172.21.15.44:0/3157384151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f4009d00 con 0x7f96fc0ffe00 2023-09-10T00:41:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.500+0000 7f96fbfff700 1 --2- 172.21.15.44:0/3157384151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96fc0ffe00 0x7f96fc197870 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f96e800d980 tx=0x7f96e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.500+0000 7f96f9ffb700 1 -- 172.21.15.44:0/3157384151 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8004220 con 0x7f96fc0ffe00 2023-09-10T00:41:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.500+0000 7f97031e8700 1 -- 172.21.15.44:0/3157384151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96fc1a2d90 con 0x7f96fc0ffe00 2023-09-10T00:41:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.500+0000 7f96f9ffb700 1 -- 172.21.15.44:0/3157384151 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96e8007640 con 0x7f96fc0ffe00 2023-09-10T00:41:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.500+0000 7f96f9ffb700 1 -- 172.21.15.44:0/3157384151 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8005660 con 0x7f96fc0ffe00 2023-09-10T00:41:34.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.500+0000 7f97031e8700 1 -- 172.21.15.44:0/3157384151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96fc1a3260 con 0x7f96fc0ffe00 2023-09-10T00:41:34.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.503+0000 7f96f9ffb700 1 -- 172.21.15.44:0/3157384151 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f96e8004750 con 0x7f96fc0ffe00 2023-09-10T00:41:34.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.503+0000 7f96f9ffb700 1 --2- 172.21.15.44:0/3157384151 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f96ec0a08d0 0x7f96ec0a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.504+0000 7f96f9ffb700 1 -- 172.21.15.44:0/3157384151 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f96e80cc2e0 con 0x7f96fc0ffe00 2023-09-10T00:41:34.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.504+0000 7f9700f84700 1 --2- 172.21.15.44:0/3157384151 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f96ec0a08d0 0x7f96ec0a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.504+0000 7f9700f84700 1 --2- 172.21.15.44:0/3157384151 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f96ec0a08d0 0x7f96ec0a2d90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f96f0007b60 tx=0x7f96f00073d0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.504+0000 7f970f3aeb80 1 Processor -- start 2023-09-10T00:41:34.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.504+0000 7f970f3aeb80 1 -- start start 2023-09-10T00:41:34.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f970f3aeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d490b10 0x56079d490ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f970f3aeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d491430 0x56079d49bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f970f3aeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d49e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f970f3aeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079d3e6730 con 0x56079d491430 2023-09-10T00:41:34.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f970f3aeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079d3e4050 con 0x56079d490b10 2023-09-10T00:41:34.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f970f3aeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079d3df940 con 0x56079d49c260 2023-09-10T00:41:34.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f96fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d491430 0x56079d49bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f96fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d491430 0x56079d49bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49690/0 (socket says 172.21.15.44:49690) 2023-09-10T00:41:34.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f96fbfff700 1 -- 172.21.15.44:0/1394320009 learned_addr learned my addr 172.21.15.44:0/1394320009 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f9701785700 1 --2- 172.21.15.44:0/1394320009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d49e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f9700f84700 1 --2- 172.21.15.44:0/1394320009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d490b10 0x56079d490ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f96fbfff700 1 -- 172.21.15.44:0/1394320009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 msgr2=0x56079d49e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.519 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f96fbfff700 1 --2- 172.21.15.44:0/1394320009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d49e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.519 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f96fbfff700 1 -- 172.21.15.44:0/1394320009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d490b10 msgr2=0x56079d490ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.519 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f96fbfff700 1 --2- 172.21.15.44:0/1394320009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d490b10 0x56079d490ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.519 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f96fbfff700 1 -- 172.21.15.44:0/1394320009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56079d20bde0 con 0x56079d491430 2023-09-10T00:41:34.519 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f96fbfff700 1 --2- 172.21.15.44:0/1394320009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d491430 0x56079d49bd20 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f96e80956a0 tx=0x7f96e8098d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa03886573d4196c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.520 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.505+0000 7f96e6ffd700 1 -- 172.21.15.44:0/1394320009 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8004380 con 0x56079d491430 2023-09-10T00:41:34.520 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.506+0000 7f96e6ffd700 1 -- 172.21.15.44:0/1394320009 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96e80998a0 con 0x56079d491430 2023-09-10T00:41:34.520 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.506+0000 7f96e6ffd700 1 -- 172.21.15.44:0/1394320009 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e80988f0 con 0x56079d491430 2023-09-10T00:41:34.520 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.506+0000 7f970f3aeb80 1 -- 172.21.15.44:0/1394320009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d491430 msgr2=0x56079d49bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.520 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.506+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/1394320009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d491430 0x56079d49bd20 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f96e80956a0 tx=0x7f96e8098d80 comp rx=0 tx=0).stop 2023-09-10T00:41:34.521 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.506+0000 7f970f3aeb80 1 -- 172.21.15.44:0/1394320009 shutdown_connections 2023-09-10T00:41:34.521 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.506+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/1394320009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d49e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.521 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.506+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/1394320009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d490b10 0x56079d490ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.521 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.506+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/1394320009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d491430 0x56079d49bd20 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.521 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.506+0000 7f970f3aeb80 1 -- 172.21.15.44:0/1394320009 >> 172.21.15.44:0/1394320009 conn(0x56079d402090 msgr2=0x56079d488a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.521 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.506+0000 7f970f3aeb80 1 -- 172.21.15.44:0/1394320009 shutdown_connections 2023-09-10T00:41:34.522 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.506+0000 7f970f3aeb80 1 -- 172.21.15.44:0/1394320009 wait complete. 2023-09-10T00:41:34.522 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.507+0000 7f970f3aeb80 1 -- start start 2023-09-10T00:41:34.522 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.507+0000 7f970f3aeb80 1 Processor -- start 2023-09-10T00:41:34.522 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.507+0000 7f970f3aeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d49bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.523 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.507+0000 7f970f3aeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 0x56079d498ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.523 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.507+0000 7f970f3aeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d499730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.523 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.507+0000 7f970f3aeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079d37db70 con 0x56079d490b10 2023-09-10T00:41:34.523 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.507+0000 7f970f3aeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079d37d870 con 0x56079d491430 2023-09-10T00:41:34.524 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.507+0000 7f970f3aeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079d261920 con 0x56079d49c260 2023-09-10T00:41:34.524 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.507+0000 7f970f3aeb80 10 client.0 _ll_register_callbacks cb 0x56079d2c7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:34.524 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.507+0000 7f9700f84700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d49bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.524 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.507+0000 7f9700f84700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d49bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49694/0 (socket says 172.21.15.44:49694) 2023-09-10T00:41:34.524 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f9700f84700 1 -- 172.21.15.44:0/2908099545 learned_addr learned my addr 172.21.15.44:0/2908099545 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.525 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f96fbfff700 1 --2- 172.21.15.44:0/2908099545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 0x56079d498ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.525 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f9701785700 1 --2- 172.21.15.44:0/2908099545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d499730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.525 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f9700f84700 1 -- 172.21.15.44:0/2908099545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 msgr2=0x56079d499730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.525 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f9700f84700 1 --2- 172.21.15.44:0/2908099545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d499730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.526 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f9700f84700 1 -- 172.21.15.44:0/2908099545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 msgr2=0x56079d498ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.526 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f9700f84700 1 --2- 172.21.15.44:0/2908099545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 0x56079d498ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.526 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f9700f84700 1 -- 172.21.15.44:0/2908099545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96fc1a3260 con 0x56079d490b10 2023-09-10T00:41:34.526 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f9701785700 1 --2- 172.21.15.44:0/2908099545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d499730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f96fbfff700 1 --2- 172.21.15.44:0/2908099545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 0x56079d498ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f9700f84700 1 --2- 172.21.15.44:0/2908099545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d49bd00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f96f00114c0 tx=0x7f96f00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd332807de6b59d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f96dbfff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f96dbfff700 1 -- 172.21.15.44:0/2908099545 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0010a80 con 0x56079d490b10 2023-09-10T00:41:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f970f3aeb80 1 -- 172.21.15.44:0/2908099545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56079d3374f0 con 0x56079d490b10 2023-09-10T00:41:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f96dbfff700 1 -- 172.21.15.44:0/2908099545 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f96f0017070 con 0x56079d490b10 2023-09-10T00:41:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f96dbfff700 1 -- 172.21.15.44:0/2908099545 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96f0028670 con 0x56079d490b10 2023-09-10T00:41:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f970f3aeb80 10 client.16173 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:34.528 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.508+0000 7f970f3aeb80 1 -- 172.21.15.44:0/2908099545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56079d1f4c60 con 0x56079d490b10 2023-09-10T00:41:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.509+0000 7f96dbfff700 1 -- 172.21.15.44:0/2908099545 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56079d3374f0 con 0x56079d490b10 2023-09-10T00:41:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.509+0000 7f96dbfff700 1 -- 172.21.15.44:0/2908099545 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f96f001e570 con 0x56079d490b10 2023-09-10T00:41:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.509+0000 7f970f3aeb80 10 client.16173 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:34.529 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.509+0000 7f970f3aeb80 10 client.16173 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-10T00:41:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.509+0000 7f970f3aeb80 20 client.16173 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.509+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/2908099545 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x56079d4abd50 0x56079d4ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.509+0000 7f970f3aeb80 4 client.16173 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:34.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.509+0000 7f970f3aeb80 1 -- 172.21.15.44:0/2908099545 --> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56079d3f48a0 con 0x56079d4abd50 2023-09-10T00:41:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.509+0000 7f96fbfff700 1 --2- 172.21.15.44:0/2908099545 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x56079d4abd50 0x56079d4ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.510+0000 7f96dbfff700 10 client.16173 ms_handle_connect on v2:172.21.15.99:6836/566283638 2023-09-10T00:41:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.510+0000 7f96fbfff700 1 --2- 172.21.15.44:0/2908099545 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x56079d4abd50 0x56079d4ae190 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=769f4a01ccec2c80 server_cookie=18515300a4fbdbe3 in_seq=0 out_seq=0 2023-09-10T00:41:34.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.510+0000 7f96dbfff700 1 -- 172.21.15.44:0/2908099545 <== mds.2 v2:172.21.15.99:6836/566283638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56079d3f48a0 con 0x56079d4abd50 2023-09-10T00:41:34.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.510+0000 7f96dbfff700 10 client.16173 handle_command_reply: tid=0 2023-09-10T00:41:34.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.510+0000 7f970f3aeb80 1 client.16173 shutdown 2023-09-10T00:41:34.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.511+0000 7f970f3aeb80 1 -- 172.21.15.44:0/2908099545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 msgr2=0x56079d49bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.511+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/2908099545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d49bd00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f96f00114c0 tx=0x7f96f00047c0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.511+0000 7f970f3aeb80 1 -- 172.21.15.44:0/2908099545 shutdown_connections 2023-09-10T00:41:34.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.511+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/2908099545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d499730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.511+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/2908099545 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x56079d4abd50 0x56079d4ae190 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.511+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/2908099545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d49bd00 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.511+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/2908099545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 0x56079d498ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.511+0000 7f96dbfff700 0 client.16173 ms_handle_reset on v2:172.21.15.99:6836/566283638 2023-09-10T00:41:34.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.511+0000 7f970f3aeb80 1 -- 172.21.15.44:0/2908099545 >> 172.21.15.44:0/2908099545 conn(0x56079d402090 msgr2=0x56079d4886a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.511+0000 7f970f3aeb80 1 -- 172.21.15.44:0/2908099545 shutdown_connections 2023-09-10T00:41:34.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.511+0000 7f970f3aeb80 1 -- 172.21.15.44:0/2908099545 wait complete. 2023-09-10T00:41:34.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.511+0000 7f970f3aeb80 20 client.16173 trim_cache size 0 max 16384 2023-09-10T00:41:34.541 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.530+0000 7f970f3aeb80 1 Processor -- start 2023-09-10T00:41:34.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.530+0000 7f970f3aeb80 1 -- start start 2023-09-10T00:41:34.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.530+0000 7f970f3aeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d4a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.530+0000 7f970f3aeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 0x56079d487dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.530+0000 7f970f3aeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d4a7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.530+0000 7f970f3aeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079d4a3230 con 0x56079d490b10 2023-09-10T00:41:34.543 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.530+0000 7f970f3aeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079d488300 con 0x56079d491430 2023-09-10T00:41:34.543 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.530+0000 7f970f3aeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079d4a8500 con 0x56079d49c260 2023-09-10T00:41:34.543 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.531+0000 7f9700f84700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d4a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.543 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.531+0000 7f9700f84700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d4a2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49702/0 (socket says 172.21.15.44:49702) 2023-09-10T00:41:34.543 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.531+0000 7f9700f84700 1 -- 172.21.15.44:0/3301972080 learned_addr learned my addr 172.21.15.44:0/3301972080 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.543 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.531+0000 7f96fbfff700 1 --2- 172.21.15.44:0/3301972080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 0x56079d487dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.531+0000 7f9701785700 1 --2- 172.21.15.44:0/3301972080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d4a7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.531+0000 7f9700f84700 1 -- 172.21.15.44:0/3301972080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 msgr2=0x56079d4a7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.531+0000 7f9700f84700 1 --2- 172.21.15.44:0/3301972080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d4a7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.531+0000 7f9700f84700 1 -- 172.21.15.44:0/3301972080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 msgr2=0x56079d487dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.531+0000 7f9700f84700 1 --2- 172.21.15.44:0/3301972080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 0x56079d487dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.531+0000 7f9700f84700 1 -- 172.21.15.44:0/3301972080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56079d232250 con 0x56079d490b10 2023-09-10T00:41:34.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.531+0000 7f96fbfff700 1 --2- 172.21.15.44:0/3301972080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 0x56079d487dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:34.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.531+0000 7f9700f84700 1 --2- 172.21.15.44:0/3301972080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d4a2cf0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f96f0000c00 tx=0x7f96f001e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b98f1127d963cc server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.531+0000 7f96e77fe700 1 -- 172.21.15.44:0/3301972080 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0010a80 con 0x56079d490b10 2023-09-10T00:41:34.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.531+0000 7f96e77fe700 1 -- 172.21.15.44:0/3301972080 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96f0015070 con 0x56079d490b10 2023-09-10T00:41:34.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.531+0000 7f96e77fe700 1 -- 172.21.15.44:0/3301972080 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0028b60 con 0x56079d490b10 2023-09-10T00:41:34.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.532+0000 7f970f3aeb80 1 -- 172.21.15.44:0/3301972080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 msgr2=0x56079d4a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.532+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/3301972080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d4a2cf0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f96f0000c00 tx=0x7f96f001e430 comp rx=0 tx=0).stop 2023-09-10T00:41:34.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.532+0000 7f970f3aeb80 1 -- 172.21.15.44:0/3301972080 shutdown_connections 2023-09-10T00:41:34.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.532+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/3301972080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d4a7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.532+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/3301972080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d4a2cf0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.532+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/3301972080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 0x56079d487dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.532+0000 7f970f3aeb80 1 -- 172.21.15.44:0/3301972080 >> 172.21.15.44:0/3301972080 conn(0x56079d402090 msgr2=0x56079d4b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.532+0000 7f970f3aeb80 1 -- 172.21.15.44:0/3301972080 shutdown_connections 2023-09-10T00:41:34.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.532+0000 7f970f3aeb80 1 -- 172.21.15.44:0/3301972080 wait complete. 2023-09-10T00:41:34.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.533+0000 7f970f3aeb80 1 -- start start 2023-09-10T00:41:34.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.533+0000 7f970f3aeb80 1 Processor -- start 2023-09-10T00:41:34.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.533+0000 7f970f3aeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d4a2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.533+0000 7f970f3aeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 0x56079d4a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.533+0000 7f970f3aeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d4feba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.533+0000 7f970f3aeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079d4ff0e0 con 0x56079d490b10 2023-09-10T00:41:34.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.533+0000 7f970f3aeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079d4ff220 con 0x56079d491430 2023-09-10T00:41:34.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.533+0000 7f970f3aeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079d4ff360 con 0x56079d49c260 2023-09-10T00:41:34.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.533+0000 7f970f3aeb80 10 client.0 _ll_register_callbacks cb 0x7f96f0010c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:34.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.533+0000 7f9700f84700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d4a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.533+0000 7f9700f84700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d4a2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49708/0 (socket says 172.21.15.44:49708) 2023-09-10T00:41:34.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.533+0000 7f9700f84700 1 -- 172.21.15.44:0/2261996504 learned_addr learned my addr 172.21.15.44:0/2261996504 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.533+0000 7f96fbfff700 1 --2- 172.21.15.44:0/2261996504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 0x56079d4a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.533+0000 7f9701785700 1 --2- 172.21.15.44:0/2261996504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d4feba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f9700f84700 1 -- 172.21.15.44:0/2261996504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 msgr2=0x56079d4feba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f9700f84700 1 --2- 172.21.15.44:0/2261996504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d4feba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f9700f84700 1 -- 172.21.15.44:0/2261996504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 msgr2=0x56079d4a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f9700f84700 1 --2- 172.21.15.44:0/2261996504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 0x56079d4a25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f9700f84700 1 -- 172.21.15.44:0/2261996504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56079d20bde0 con 0x56079d490b10 2023-09-10T00:41:34.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f9701785700 1 --2- 172.21.15.44:0/2261996504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d4feba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:41:34.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f96fbfff700 1 --2- 172.21.15.44:0/2261996504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 0x56079d4a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:34.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f9700f84700 1 --2- 172.21.15.44:0/2261996504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d4a2090 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f96f001e8d0 tx=0x7f96f0005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f0880e8f27982 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f96dbfff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:34.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f96dbfff700 1 -- 172.21.15.44:0/2261996504 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0004450 con 0x56079d490b10 2023-09-10T00:41:34.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f96dbfff700 1 -- 172.21.15.44:0/2261996504 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f96f0017070 con 0x56079d490b10 2023-09-10T00:41:34.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f970f3aeb80 1 -- 172.21.15.44:0/2261996504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96e80998a0 con 0x56079d490b10 2023-09-10T00:41:34.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f96dbfff700 1 -- 172.21.15.44:0/2261996504 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96f00045b0 con 0x56079d490b10 2023-09-10T00:41:34.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f970f3aeb80 10 client.16179 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:34.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f970f3aeb80 1 -- 172.21.15.44:0/2261996504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56079d1f4c60 con 0x56079d490b10 2023-09-10T00:41:34.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f96dbfff700 1 -- 172.21.15.44:0/2261996504 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96e80998a0 con 0x56079d490b10 2023-09-10T00:41:34.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.534+0000 7f96dbfff700 1 -- 172.21.15.44:0/2261996504 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f96f0004c10 con 0x56079d490b10 2023-09-10T00:41:34.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.535+0000 7f970f3aeb80 10 client.16179 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:34.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.535+0000 7f970f3aeb80 10 client.16179 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-10T00:41:34.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.535+0000 7f970f3aeb80 20 client.16179 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:34.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.535+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/2261996504 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x56079d500600 0x56079d502a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.535+0000 7f970f3aeb80 4 client.16179 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-09-10T00:41:34.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.535+0000 7f970f3aeb80 1 -- 172.21.15.44:0/2261996504 --> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56079d3d8920 con 0x56079d500600 2023-09-10T00:41:34.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.535+0000 7f96fbfff700 1 --2- 172.21.15.44:0/2261996504 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x56079d500600 0x56079d502a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.535+0000 7f96fbfff700 1 --2- 172.21.15.44:0/2261996504 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x56079d500600 0x56079d502a20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c536c39e08162a9 server_cookie=7926c19e5a49d7d5 in_seq=0 out_seq=0 2023-09-10T00:41:34.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.535+0000 7f96dbfff700 10 client.16179 ms_handle_connect on v2:172.21.15.99:6836/566283638 2023-09-10T00:41:34.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.536+0000 7f96dbfff700 1 -- 172.21.15.44:0/2261996504 <== mds.2 v2:172.21.15.99:6836/566283638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56079d3d8920 con 0x56079d500600 2023-09-10T00:41:34.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.536+0000 7f96dbfff700 10 client.16179 handle_command_reply: tid=0 2023-09-10T00:41:34.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.536+0000 7f970f3aeb80 1 client.16179 shutdown 2023-09-10T00:41:34.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.536+0000 7f970f3aeb80 1 -- 172.21.15.44:0/2261996504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 msgr2=0x56079d4a2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.536+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/2261996504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d4a2090 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f96f001e8d0 tx=0x7f96f0005a00 comp rx=0 tx=0).stop 2023-09-10T00:41:34.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.536+0000 7f970f3aeb80 1 -- 172.21.15.44:0/2261996504 shutdown_connections 2023-09-10T00:41:34.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.536+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/2261996504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56079d49c260 0x56079d4feba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.536+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/2261996504 >> [v2:172.21.15.99:6836/566283638,v1:172.21.15.99:6837/566283638] conn(0x56079d500600 0x56079d502a20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.536+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/2261996504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56079d490b10 0x56079d4a2090 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.536+0000 7f970f3aeb80 1 --2- 172.21.15.44:0/2261996504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56079d491430 0x56079d4a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.536+0000 7f96dbfff700 0 client.16179 ms_handle_reset on v2:172.21.15.99:6836/566283638 2023-09-10T00:41:34.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.536+0000 7f970f3aeb80 1 -- 172.21.15.44:0/2261996504 >> 172.21.15.44:0/2261996504 conn(0x56079d402090 msgr2=0x56079d494360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.536+0000 7f970f3aeb80 1 -- 172.21.15.44:0/2261996504 shutdown_connections 2023-09-10T00:41:34.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.536+0000 7f970f3aeb80 1 -- 172.21.15.44:0/2261996504 wait complete. 2023-09-10T00:41:34.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.536+0000 7f970f3aeb80 20 client.16179 trim_cache size 0 max 16384 2023-09-10T00:41:34.557 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2023-09-10T00:41:34.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.537+0000 7f97031e8700 1 -- 172.21.15.44:0/3157384151 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f96ec0a08d0 msgr2=0x7f96ec0a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.537+0000 7f97031e8700 1 --2- 172.21.15.44:0/3157384151 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f96ec0a08d0 0x7f96ec0a2d90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f96f0007b60 tx=0x7f96f00073d0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.537+0000 7f97031e8700 1 -- 172.21.15.44:0/3157384151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96fc0ffe00 msgr2=0x7f96fc197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.537+0000 7f97031e8700 1 --2- 172.21.15.44:0/3157384151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96fc0ffe00 0x7f96fc197870 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f96e800d980 tx=0x7f96e800dc90 comp rx=0 tx=0).stop 2023-09-10T00:41:34.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.537+0000 7f97031e8700 1 -- 172.21.15.44:0/3157384151 shutdown_connections 2023-09-10T00:41:34.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.537+0000 7f97031e8700 1 --2- 172.21.15.44:0/3157384151 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f96ec0a08d0 0x7f96ec0a2d90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.537+0000 7f97031e8700 1 --2- 172.21.15.44:0/3157384151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96fc1054e0 0x7f96fc1a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.537+0000 7f97031e8700 1 --2- 172.21.15.44:0/3157384151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96fc0ff440 0x7f96fc197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.537+0000 7f97031e8700 1 --2- 172.21.15.44:0/3157384151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96fc0ffe00 0x7f96fc197870 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.537+0000 7f97031e8700 1 -- 172.21.15.44:0/3157384151 >> 172.21.15.44:0/3157384151 conn(0x7f96fc0faf50 msgr2=0x7f96fc0fb630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.537+0000 7f97031e8700 1 -- 172.21.15.44:0/3157384151 shutdown_connections 2023-09-10T00:41:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.537+0000 7f97031e8700 1 -- 172.21.15.44:0/3157384151 wait complete. 2023-09-10T00:41:34.562 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-10T00:41:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.694+0000 7f6ae2386700 1 -- 172.21.15.44:0/1914841027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6adc1055b0 msgr2=0x7f6adc105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.694+0000 7f6ae2386700 1 --2- 172.21.15.44:0/1914841027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6adc1055b0 0x7f6adc105990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0099c0 tx=0x7f6acc009cd0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.694+0000 7f6ae2386700 1 -- 172.21.15.44:0/1914841027 shutdown_connections 2023-09-10T00:41:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.694+0000 7f6ae2386700 1 --2- 172.21.15.44:0/1914841027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6adc100050 0x7f6adc10c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.694+0000 7f6ae2386700 1 --2- 172.21.15.44:0/1914841027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6adc1055b0 0x7f6adc105990 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.694+0000 7f6ae2386700 1 --2- 172.21.15.44:0/1914841027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6adc0ff690 0x7f6adc0ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.694+0000 7f6ae2386700 1 -- 172.21.15.44:0/1914841027 >> 172.21.15.44:0/1914841027 conn(0x7f6adc077430 msgr2=0x7f6adc077840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.694+0000 7f6ae2386700 1 -- 172.21.15.44:0/1914841027 shutdown_connections 2023-09-10T00:41:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.694+0000 7f6ae2386700 1 -- 172.21.15.44:0/1914841027 wait complete. 2023-09-10T00:41:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.695+0000 7f6ae2386700 1 Processor -- start 2023-09-10T00:41:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.695+0000 7f6ae2386700 1 -- start start 2023-09-10T00:41:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.695+0000 7f6ae2386700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6adc0ff690 0x7f6adc19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.695+0000 7f6ae2386700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6adc100050 0x7f6adc19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.695+0000 7f6ae2386700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6adc1055b0 0x7f6adc10b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.695+0000 7f6ae2386700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc10bdb0 con 0x7f6adc0ff690 2023-09-10T00:41:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.695+0000 7f6ae2386700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc10bf20 con 0x7f6adc100050 2023-09-10T00:41:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.695+0000 7f6ae2386700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc10c0e0 con 0x7f6adc1055b0 2023-09-10T00:41:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.695+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6adc0ff690 0x7f6adc19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.695+0000 7f6adb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6adc100050 0x7f6adc19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.695+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6adc0ff690 0x7f6adc19e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49722/0 (socket says 172.21.15.44:49722) 2023-09-10T00:41:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.695+0000 7f6adbfff700 1 -- 172.21.15.44:0/2088935647 learned_addr learned my addr 172.21.15.44:0/2088935647 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.696+0000 7f6ae0923700 1 --2- 172.21.15.44:0/2088935647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6adc1055b0 0x7f6adc10b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.696+0000 7f6adbfff700 1 -- 172.21.15.44:0/2088935647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6adc1055b0 msgr2=0x7f6adc10b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.696+0000 7f6adbfff700 1 --2- 172.21.15.44:0/2088935647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6adc1055b0 0x7f6adc10b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.696+0000 7f6adbfff700 1 -- 172.21.15.44:0/2088935647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6adc100050 msgr2=0x7f6adc19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.696+0000 7f6adbfff700 1 --2- 172.21.15.44:0/2088935647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6adc100050 0x7f6adc19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.696+0000 7f6adbfff700 1 -- 172.21.15.44:0/2088935647 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc0095f0 con 0x7f6adc0ff690 2023-09-10T00:41:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.696+0000 7f6adb7fe700 1 --2- 172.21.15.44:0/2088935647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6adc100050 0x7f6adc19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.696+0000 7f6adbfff700 1 --2- 172.21.15.44:0/2088935647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6adc0ff690 0x7f6adc19e7b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00c010 tx=0x7f6acc005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.696+0000 7f6ad97fa700 1 -- 172.21.15.44:0/2088935647 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc01d070 con 0x7f6adc0ff690 2023-09-10T00:41:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.696+0000 7f6ad97fa700 1 -- 172.21.15.44:0/2088935647 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6acc003f80 con 0x7f6adc0ff690 2023-09-10T00:41:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.696+0000 7f6ae2386700 1 -- 172.21.15.44:0/2088935647 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6adc10c360 con 0x7f6adc0ff690 2023-09-10T00:41:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.696+0000 7f6ad97fa700 1 -- 172.21.15.44:0/2088935647 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc00fb80 con 0x7f6adc0ff690 2023-09-10T00:41:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.696+0000 7f6ae2386700 1 -- 172.21.15.44:0/2088935647 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc1a57a0 con 0x7f6adc0ff690 2023-09-10T00:41:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.699+0000 7f6ad97fa700 1 -- 172.21.15.44:0/2088935647 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f6acc00fce0 con 0x7f6adc0ff690 2023-09-10T00:41:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.700+0000 7f6ad97fa700 1 --2- 172.21.15.44:0/2088935647 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6ac40a09a0 0x7f6ac40a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.700+0000 7f6ad97fa700 1 -- 172.21.15.44:0/2088935647 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f6acc0cd380 con 0x7f6adc0ff690 2023-09-10T00:41:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.700+0000 7f6adb7fe700 1 --2- 172.21.15.44:0/2088935647 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6ac40a09a0 0x7f6ac40a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.700+0000 7f6adb7fe700 1 --2- 172.21.15.44:0/2088935647 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6ac40a09a0 0x7f6ac40a2e60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6adc10c820 tx=0x7f6ad00073d0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.701+0000 7f6aee54cb80 1 Processor -- start 2023-09-10T00:41:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.701+0000 7f6aee54cb80 1 -- start start 2023-09-10T00:41:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.701+0000 7f6aee54cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd31e1a0 0x555ecd31e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.701+0000 7f6aee54cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd318780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.701+0000 7f6aee54cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd318cc0 0x555ecd324f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.701+0000 7f6aee54cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ecd2716d0 con 0x555ecd31e1a0 2023-09-10T00:41:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.701+0000 7f6aee54cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ecd26eff0 con 0x555ecd318300 2023-09-10T00:41:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.701+0000 7f6aee54cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ecd26a8e0 con 0x555ecd318cc0 2023-09-10T00:41:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.701+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd31e1a0 0x555ecd31e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.701+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd31e1a0 0x555ecd31e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49730/0 (socket says 172.21.15.44:49730) 2023-09-10T00:41:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.701+0000 7f6adbfff700 1 -- 172.21.15.44:0/810800100 learned_addr learned my addr 172.21.15.44:0/810800100 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.701+0000 7f6adb7fe700 1 --2- 172.21.15.44:0/810800100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd318780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.701+0000 7f6adbfff700 1 -- 172.21.15.44:0/810800100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd318cc0 msgr2=0x555ecd324f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.702+0000 7f6ae0923700 1 --2- 172.21.15.44:0/810800100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd318cc0 0x555ecd324f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.702+0000 7f6adbfff700 1 --2- 172.21.15.44:0/810800100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd318cc0 0x555ecd324f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.702+0000 7f6adbfff700 1 -- 172.21.15.44:0/810800100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 msgr2=0x555ecd318780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.702+0000 7f6adbfff700 1 --2- 172.21.15.44:0/810800100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd318780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.702+0000 7f6adbfff700 1 -- 172.21.15.44:0/810800100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ecd096d80 con 0x555ecd31e1a0 2023-09-10T00:41:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.702+0000 7f6adb7fe700 1 --2- 172.21.15.44:0/810800100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd318780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.702+0000 7f6ae0923700 1 --2- 172.21.15.44:0/810800100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd318cc0 0x555ecd324f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:41:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.702+0000 7f6adbfff700 1 --2- 172.21.15.44:0/810800100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd31e1a0 0x555ecd31e580 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6acc000c00 tx=0x7f6acc009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed7ce25b8e01ecf server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.702+0000 7f6ac27fc700 1 -- 172.21.15.44:0/810800100 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc09e100 con 0x555ecd31e1a0 2023-09-10T00:41:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.702+0000 7f6ac27fc700 1 -- 172.21.15.44:0/810800100 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6acc0a22b0 con 0x555ecd31e1a0 2023-09-10T00:41:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.702+0000 7f6ac27fc700 1 -- 172.21.15.44:0/810800100 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc09a530 con 0x555ecd31e1a0 2023-09-10T00:41:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.703+0000 7f6aee54cb80 1 -- 172.21.15.44:0/810800100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd31e1a0 msgr2=0x555ecd31e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.703+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/810800100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd31e1a0 0x555ecd31e580 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6acc000c00 tx=0x7f6acc009cd0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.703+0000 7f6aee54cb80 1 -- 172.21.15.44:0/810800100 shutdown_connections 2023-09-10T00:41:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.703+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/810800100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd318cc0 0x555ecd324f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.703+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/810800100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd31e1a0 0x555ecd31e580 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.703+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/810800100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd318780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.703+0000 7f6aee54cb80 1 -- 172.21.15.44:0/810800100 >> 172.21.15.44:0/810800100 conn(0x555ecd28d030 msgr2=0x555ecd313560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.703+0000 7f6aee54cb80 1 -- 172.21.15.44:0/810800100 shutdown_connections 2023-09-10T00:41:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.703+0000 7f6aee54cb80 1 -- 172.21.15.44:0/810800100 wait complete. 2023-09-10T00:41:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.703+0000 7f6aee54cb80 1 -- start start 2023-09-10T00:41:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.704+0000 7f6aee54cb80 1 Processor -- start 2023-09-10T00:41:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.704+0000 7f6aee54cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd327ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.704+0000 7f6aee54cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd318cc0 0x555ecd328010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.704+0000 7f6aee54cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd31e1a0 0x555ecd321d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.704+0000 7f6aee54cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ecd208b10 con 0x555ecd318cc0 2023-09-10T00:41:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.704+0000 7f6aee54cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ecd208810 con 0x555ecd318300 2023-09-10T00:41:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.704+0000 7f6aee54cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ecd0ec8c0 con 0x555ecd31e1a0 2023-09-10T00:41:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.704+0000 7f6aee54cb80 10 client.0 _ll_register_callbacks cb 0x555ecd152a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.704+0000 7f6adb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd318cc0 0x555ecd328010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.704+0000 7f6adb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd318cc0 0x555ecd328010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49736/0 (socket says 172.21.15.44:49736) 2023-09-10T00:41:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.704+0000 7f6adb7fe700 1 -- 172.21.15.44:0/2663334864 learned_addr learned my addr 172.21.15.44:0/2663334864 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.704+0000 7f6adbfff700 1 --2- 172.21.15.44:0/2663334864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd327ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.705+0000 7f6ae0923700 1 --2- 172.21.15.44:0/2663334864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd31e1a0 0x555ecd321d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.705+0000 7f6adb7fe700 1 -- 172.21.15.44:0/2663334864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd31e1a0 msgr2=0x555ecd321d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.705+0000 7f6adb7fe700 1 --2- 172.21.15.44:0/2663334864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd31e1a0 0x555ecd321d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.705+0000 7f6adb7fe700 1 -- 172.21.15.44:0/2663334864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 msgr2=0x555ecd327ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.705+0000 7f6adb7fe700 1 --2- 172.21.15.44:0/2663334864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd327ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.705+0000 7f6adb7fe700 1 -- 172.21.15.44:0/2663334864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc1a57a0 con 0x555ecd318cc0 2023-09-10T00:41:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.705+0000 7f6adb7fe700 1 --2- 172.21.15.44:0/2663334864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd318cc0 0x555ecd328010 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0004430 tx=0x7f6ad00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f256a04c612eeb3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.705+0000 7f6abb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.705+0000 7f6abb7fe700 1 -- 172.21.15.44:0/2663334864 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0011440 con 0x555ecd318cc0 2023-09-10T00:41:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.705+0000 7f6abb7fe700 1 -- 172.21.15.44:0/2663334864 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f6ad0017070 con 0x555ecd318cc0 2023-09-10T00:41:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.705+0000 7f6aee54cb80 1 -- 172.21.15.44:0/2663334864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ecd1c2490 con 0x555ecd318cc0 2023-09-10T00:41:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.705+0000 7f6abb7fe700 1 -- 172.21.15.44:0/2663334864 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ad00115a0 con 0x555ecd318cc0 2023-09-10T00:41:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.705+0000 7f6aee54cb80 10 client.16191 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.705+0000 7f6aee54cb80 1 -- 172.21.15.44:0/2663334864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555ecd07fc00 con 0x555ecd318cc0 2023-09-10T00:41:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.706+0000 7f6abb7fe700 1 -- 172.21.15.44:0/2663334864 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555ecd1c2490 con 0x555ecd318cc0 2023-09-10T00:41:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.706+0000 7f6abb7fe700 1 -- 172.21.15.44:0/2663334864 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6ad0020480 con 0x555ecd318cc0 2023-09-10T00:41:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.706+0000 7f6aee54cb80 10 client.16191 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.706+0000 7f6aee54cb80 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.706+0000 7f6aee54cb80 20 client.16191 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.706+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/2663334864 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555ecd332090 0x555ecd3344b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.706+0000 7f6aee54cb80 4 client.16191 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.706+0000 7f6aee54cb80 1 -- 172.21.15.44:0/2663334864 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ecd27f840 con 0x555ecd332090 2023-09-10T00:41:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.706+0000 7f6adbfff700 1 --2- 172.21.15.44:0/2663334864 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555ecd332090 0x555ecd3344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.707+0000 7f6adbfff700 1 --2- 172.21.15.44:0/2663334864 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555ecd332090 0x555ecd3344b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f439816b05dcf694 server_cookie=bc0c687ab50ff2a3 in_seq=0 out_seq=0 2023-09-10T00:41:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.707+0000 7f6abb7fe700 10 client.16191 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.707+0000 7f6abb7fe700 1 -- 172.21.15.44:0/2663334864 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555ecd27f840 con 0x555ecd332090 2023-09-10T00:41:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.707+0000 7f6abb7fe700 10 client.16191 handle_command_reply: tid=0 2023-09-10T00:41:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.707+0000 7f6aee54cb80 1 client.16191 shutdown 2023-09-10T00:41:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.707+0000 7f6aee54cb80 1 -- 172.21.15.44:0/2663334864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd318cc0 msgr2=0x555ecd328010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.707+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/2663334864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd318cc0 0x555ecd328010 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0004430 tx=0x7f6ad00047c0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.708+0000 7f6aee54cb80 1 -- 172.21.15.44:0/2663334864 shutdown_connections 2023-09-10T00:41:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.708+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/2663334864 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555ecd332090 0x555ecd3344b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.708+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/2663334864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd31e1a0 0x555ecd321d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.708+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/2663334864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd327ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.708+0000 7f6abb7fe700 0 client.16191 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.708+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/2663334864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd318cc0 0x555ecd328010 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.708+0000 7f6aee54cb80 1 -- 172.21.15.44:0/2663334864 >> 172.21.15.44:0/2663334864 conn(0x555ecd28d030 msgr2=0x555ecd3127a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.708+0000 7f6aee54cb80 1 -- 172.21.15.44:0/2663334864 shutdown_connections 2023-09-10T00:41:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.708+0000 7f6aee54cb80 1 -- 172.21.15.44:0/2663334864 wait complete. 2023-09-10T00:41:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.708+0000 7f6aee54cb80 20 client.16191 trim_cache size 0 max 16384 2023-09-10T00:41:34.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.722+0000 7f66ceee4700 1 -- 172.21.15.44:0/3555879485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66c80ff4b0 msgr2=0x7f66c80ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.722+0000 7f66ceee4700 1 --2- 172.21.15.44:0/3555879485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66c80ff4b0 0x7f66c80ff930 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f66b40099c0 tx=0x7f66b4009cd0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.722+0000 7f66ceee4700 1 -- 172.21.15.44:0/3555879485 shutdown_connections 2023-09-10T00:41:34.730 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.722+0000 7f66ceee4700 1 --2- 172.21.15.44:0/3555879485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c80ffe70 0x7f66c810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.730 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.722+0000 7f66ceee4700 1 --2- 172.21.15.44:0/3555879485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66c8105570 0x7f66c8105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.730 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.722+0000 7f66ceee4700 1 --2- 172.21.15.44:0/3555879485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66c80ff4b0 0x7f66c80ff930 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.730 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.722+0000 7f66ceee4700 1 -- 172.21.15.44:0/3555879485 >> 172.21.15.44:0/3555879485 conn(0x7f66c80fafa0 msgr2=0x7f66c80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.730 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.722+0000 7f66ceee4700 1 -- 172.21.15.44:0/3555879485 shutdown_connections 2023-09-10T00:41:34.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.722+0000 7f66ceee4700 1 -- 172.21.15.44:0/3555879485 wait complete. 2023-09-10T00:41:34.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.723+0000 7f66ceee4700 1 Processor -- start 2023-09-10T00:41:34.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.723+0000 7f66ceee4700 1 -- start start 2023-09-10T00:41:34.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.723+0000 7f66ceee4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66c80ff4b0 0x7f66c8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.723+0000 7f66ceee4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c80ffe70 0x7f66c8198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.723+0000 7f66ceee4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66c8105570 0x7f66c819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.723+0000 7f66ceee4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c819c610 con 0x7f66c8105570 2023-09-10T00:41:34.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.723+0000 7f66ceee4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c819c780 con 0x7f66c80ffe70 2023-09-10T00:41:34.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.723+0000 7f66ceee4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c819c940 con 0x7f66c80ff4b0 2023-09-10T00:41:34.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.723+0000 7f66cd481700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66c8105570 0x7f66c819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.723+0000 7f66cd481700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66c8105570 0x7f66c819c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49752/0 (socket says 172.21.15.44:49752) 2023-09-10T00:41:34.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.723+0000 7f66cd481700 1 -- 172.21.15.44:0/1992247507 learned_addr learned my addr 172.21.15.44:0/1992247507 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.723+0000 7f66ccc80700 1 --2- 172.21.15.44:0/1992247507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66c80ff4b0 0x7f66c8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.724+0000 7f66c7fff700 1 --2- 172.21.15.44:0/1992247507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c80ffe70 0x7f66c8198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.724+0000 7f66cd481700 1 -- 172.21.15.44:0/1992247507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66c80ff4b0 msgr2=0x7f66c8198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.724+0000 7f66cd481700 1 --2- 172.21.15.44:0/1992247507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66c80ff4b0 0x7f66c8198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.724+0000 7f66cd481700 1 -- 172.21.15.44:0/1992247507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c80ffe70 msgr2=0x7f66c8198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.734 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.724+0000 7f66cd481700 1 --2- 172.21.15.44:0/1992247507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c80ffe70 0x7f66c8198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.734 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.724+0000 7f66cd481700 1 -- 172.21.15.44:0/1992247507 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b40095f0 con 0x7f66c8105570 2023-09-10T00:41:34.734 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.724+0000 7f66cd481700 1 --2- 172.21.15.44:0/1992247507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66c8105570 0x7f66c819c010 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f66c0007ee0 tx=0x7f66c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.734 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.724+0000 7f66c5ffb700 1 -- 172.21.15.44:0/1992247507 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0017070 con 0x7f66c8105570 2023-09-10T00:41:34.734 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.724+0000 7f66c5ffb700 1 -- 172.21.15.44:0/1992247507 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66c000f040 con 0x7f66c8105570 2023-09-10T00:41:34.735 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.724+0000 7f66c5ffb700 1 -- 172.21.15.44:0/1992247507 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0005710 con 0x7f66c8105570 2023-09-10T00:41:34.735 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.725+0000 7f66ceee4700 1 -- 172.21.15.44:0/1992247507 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c819cc20 con 0x7f66c8105570 2023-09-10T00:41:34.735 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.725+0000 7f66ceee4700 1 -- 172.21.15.44:0/1992247507 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c81a5580 con 0x7f66c8105570 2023-09-10T00:41:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.727+0000 7f6aee54cb80 1 Processor -- start 2023-09-10T00:41:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.727+0000 7f6aee54cb80 1 -- start start 2023-09-10T00:41:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6aee54cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd31d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6aee54cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd318cc0 0x555ecd317a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6aee54cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd31e1a0 0x555ecd312f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6aee54cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ecd319580 con 0x555ecd318cc0 2023-09-10T00:41:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6aee54cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ecd322ea0 con 0x555ecd318300 2023-09-10T00:41:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6aee54cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ecd314d10 con 0x555ecd31e1a0 2023-09-10T00:41:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6ae0923700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd31e1a0 0x555ecd312f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6ae0923700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd31e1a0 0x555ecd312f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56640/0 (socket says 172.21.15.44:56640) 2023-09-10T00:41:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6ae0923700 1 -- 172.21.15.44:0/4254470973 learned_addr learned my addr 172.21.15.44:0/4254470973 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6adb7fe700 1 --2- 172.21.15.44:0/4254470973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd318cc0 0x555ecd317a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6adbfff700 1 --2- 172.21.15.44:0/4254470973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd31d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6adb7fe700 1 -- 172.21.15.44:0/4254470973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd31e1a0 msgr2=0x555ecd312f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6adb7fe700 1 --2- 172.21.15.44:0/4254470973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd31e1a0 0x555ecd312f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6adb7fe700 1 -- 172.21.15.44:0/4254470973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 msgr2=0x555ecd31d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6adb7fe700 1 --2- 172.21.15.44:0/4254470973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd31d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6adb7fe700 1 -- 172.21.15.44:0/4254470973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ecd0bd1f0 con 0x555ecd318cc0 2023-09-10T00:41:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f6adbfff700 1 --2- 172.21.15.44:0/4254470973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd31d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.729+0000 7f6adb7fe700 1 --2- 172.21.15.44:0/4254470973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd318cc0 0x555ecd317a00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0020890 tx=0x7f6ad00043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102fbcb51b5ec5e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.729+0000 7f6ac2ffd700 1 -- 172.21.15.44:0/4254470973 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0038030 con 0x555ecd318cc0 2023-09-10T00:41:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.729+0000 7f6ac2ffd700 1 -- 172.21.15.44:0/4254470973 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ad0003960 con 0x555ecd318cc0 2023-09-10T00:41:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.729+0000 7f6ac2ffd700 1 -- 172.21.15.44:0/4254470973 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0011b30 con 0x555ecd318cc0 2023-09-10T00:41:34.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.728+0000 7f66c5ffb700 1 -- 172.21.15.44:0/1992247507 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f66c0005870 con 0x7f66c8105570 2023-09-10T00:41:34.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.729+0000 7f66c5ffb700 1 --2- 172.21.15.44:0/1992247507 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f66b80a0950 0x7f66b80a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.729+0000 7f66ccc80700 1 --2- 172.21.15.44:0/1992247507 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f66b80a0950 0x7f66b80a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.729+0000 7f66c5ffb700 1 -- 172.21.15.44:0/1992247507 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f66c00cb570 con 0x7f66c8105570 2023-09-10T00:41:34.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.730+0000 7f66db0aab80 1 Processor -- start 2023-09-10T00:41:34.743 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.730+0000 7f66ccc80700 1 --2- 172.21.15.44:0/1992247507 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f66b80a0950 0x7f66b80a2e10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f66bc005950 tx=0x7f66bc0058e0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.730+0000 7f6aee54cb80 1 -- 172.21.15.44:0/4254470973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd318cc0 msgr2=0x555ecd317a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.730+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/4254470973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd318cc0 0x555ecd317a00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0020890 tx=0x7f6ad00043a0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.730+0000 7f6aee54cb80 1 -- 172.21.15.44:0/4254470973 shutdown_connections 2023-09-10T00:41:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.730+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/4254470973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd31e1a0 0x555ecd312f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.730+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/4254470973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd31d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.730+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/4254470973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd318cc0 0x555ecd317a00 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.730+0000 7f6aee54cb80 1 -- 172.21.15.44:0/4254470973 >> 172.21.15.44:0/4254470973 conn(0x555ecd28d030 msgr2=0x555ecd3172f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.730+0000 7f6aee54cb80 1 -- 172.21.15.44:0/4254470973 shutdown_connections 2023-09-10T00:41:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.730+0000 7f6aee54cb80 1 -- 172.21.15.44:0/4254470973 wait complete. 2023-09-10T00:41:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f6aee54cb80 1 -- start start 2023-09-10T00:41:34.746 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.730+0000 7f66db0aab80 1 -- start start 2023-09-10T00:41:34.746 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f66db0aab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83da710 0x55f9e83daaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.746 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f66db0aab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83db030 0x55f9e83ed110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f66db0aab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83ed650 0x55f9e83efa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f66db0aab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9e8333730 con 0x55f9e83db030 2023-09-10T00:41:34.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f66db0aab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9e8331050 con 0x55f9e83da710 2023-09-10T00:41:34.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f66db0aab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9e832c940 con 0x55f9e83ed650 2023-09-10T00:41:34.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f66c7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83db030 0x55f9e83ed110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f66c7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83db030 0x55f9e83ed110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49764/0 (socket says 172.21.15.44:49764) 2023-09-10T00:41:34.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f66c7fff700 1 -- 172.21.15.44:0/510141832 learned_addr learned my addr 172.21.15.44:0/510141832 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f66cd481700 1 --2- 172.21.15.44:0/510141832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83ed650 0x55f9e83efa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f66ccc80700 1 --2- 172.21.15.44:0/510141832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83da710 0x55f9e83daaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f6aee54cb80 1 Processor -- start 2023-09-10T00:41:34.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f66cd481700 1 -- 172.21.15.44:0/510141832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83da710 msgr2=0x55f9e83daaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f66cd481700 1 --2- 172.21.15.44:0/510141832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83da710 0x55f9e83daaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f66cd481700 1 -- 172.21.15.44:0/510141832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83db030 msgr2=0x55f9e83ed110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f66cd481700 1 --2- 172.21.15.44:0/510141832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83db030 0x55f9e83ed110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f66cd481700 1 -- 172.21.15.44:0/510141832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9e8158de0 con 0x55f9e83ed650 2023-09-10T00:41:34.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f66cd481700 1 --2- 172.21.15.44:0/510141832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83ed650 0x55f9e83efa40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66c00c87a0 tx=0x7f66c0094040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc876e608ca4bc17 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f6aee54cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd37f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f6aee54cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd318cc0 0x555ecd37fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f6aee54cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd31e1a0 0x555ecd379e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f6aee54cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ecd37a3e0 con 0x555ecd31e1a0 2023-09-10T00:41:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.731+0000 7f6aee54cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ecd37a550 con 0x555ecd318300 2023-09-10T00:41:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f6aee54cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ecd37a710 con 0x555ecd318cc0 2023-09-10T00:41:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f6aee54cb80 10 client.0 _ll_register_callbacks cb 0x7f6ad0010ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd37f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd37f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51510/0 (socket says 172.21.15.44:51510) 2023-09-10T00:41:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f6adbfff700 1 -- 172.21.15.44:0/4271348591 learned_addr learned my addr 172.21.15.44:0/4271348591 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f6adb7fe700 1 --2- 172.21.15.44:0/4271348591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd318cc0 0x555ecd37fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f6ae0923700 1 --2- 172.21.15.44:0/4271348591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd31e1a0 0x555ecd379e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f66b2ffd700 1 -- 172.21.15.44:0/510141832 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c00830c0 con 0x55f9e83ed650 2023-09-10T00:41:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f6adbfff700 1 -- 172.21.15.44:0/4271348591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd318cc0 msgr2=0x555ecd37fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f6adbfff700 1 --2- 172.21.15.44:0/4271348591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd318cc0 0x555ecd37fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f6adbfff700 1 -- 172.21.15.44:0/4271348591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd31e1a0 msgr2=0x555ecd379e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f6adbfff700 1 --2- 172.21.15.44:0/4271348591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd31e1a0 0x555ecd379e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f6adbfff700 1 -- 172.21.15.44:0/4271348591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ecd096d80 con 0x555ecd318300 2023-09-10T00:41:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f6adbfff700 1 --2- 172.21.15.44:0/4271348591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd37f860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6acc096d40 tx=0x7f6acc099d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4880aa40cc51891c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f6abb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:41:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.732+0000 7f6abb7fe700 1 -- 172.21.15.44:0/4271348591 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc09e100 con 0x555ecd318300 2023-09-10T00:41:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f6abb7fe700 1 -- 172.21.15.44:0/4271348591 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f6acc09ee40 con 0x555ecd318300 2023-09-10T00:41:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f6aee54cb80 1 -- 172.21.15.44:0/4271348591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6acc0a22b0 con 0x555ecd318300 2023-09-10T00:41:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f6abb7fe700 1 -- 172.21.15.44:0/4271348591 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6acc00f460 con 0x555ecd318300 2023-09-10T00:41:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f6aee54cb80 10 client.25883 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f6aee54cb80 1 -- 172.21.15.44:0/4271348591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555ecd07fc00 con 0x555ecd318300 2023-09-10T00:41:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f6abb7fe700 1 -- 172.21.15.44:0/4271348591 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6acc0a22b0 con 0x555ecd318300 2023-09-10T00:41:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f6abb7fe700 1 -- 172.21.15.44:0/4271348591 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6acc0997c0 con 0x555ecd318300 2023-09-10T00:41:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f6aee54cb80 10 client.25883 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f6aee54cb80 10 client.25883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f6aee54cb80 20 client.25883 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/4271348591 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555ecd387310 0x555ecd389730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f6aee54cb80 4 client.25883 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:41:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f6aee54cb80 1 -- 172.21.15.44:0/4271348591 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555ecd2638c0 con 0x555ecd387310 2023-09-10T00:41:34.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f66b2ffd700 1 -- 172.21.15.44:0/510141832 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66c00112b0 con 0x55f9e83ed650 2023-09-10T00:41:34.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f66b2ffd700 1 -- 172.21.15.44:0/510141832 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c00a04a0 con 0x55f9e83ed650 2023-09-10T00:41:34.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f66db0aab80 1 -- 172.21.15.44:0/510141832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83ed650 msgr2=0x55f9e83efa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.733+0000 7f66db0aab80 1 --2- 172.21.15.44:0/510141832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83ed650 0x55f9e83efa40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66c00c87a0 tx=0x7f66c0094040 comp rx=0 tx=0).stop 2023-09-10T00:41:34.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.734+0000 7f66db0aab80 1 -- 172.21.15.44:0/510141832 shutdown_connections 2023-09-10T00:41:34.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.734+0000 7f66db0aab80 1 --2- 172.21.15.44:0/510141832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83ed650 0x55f9e83efa40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.734+0000 7f66db0aab80 1 --2- 172.21.15.44:0/510141832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83da710 0x55f9e83daaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.734+0000 7f66db0aab80 1 --2- 172.21.15.44:0/510141832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83db030 0x55f9e83ed110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.734+0000 7f66db0aab80 1 -- 172.21.15.44:0/510141832 >> 172.21.15.44:0/510141832 conn(0x55f9e834f090 msgr2=0x55f9e83d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.734+0000 7f6adb7fe700 1 --2- 172.21.15.44:0/4271348591 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555ecd387310 0x555ecd389730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.734+0000 7f66db0aab80 1 -- 172.21.15.44:0/510141832 shutdown_connections 2023-09-10T00:41:34.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.734+0000 7f66db0aab80 1 -- 172.21.15.44:0/510141832 wait complete. 2023-09-10T00:41:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.734+0000 7f6abb7fe700 10 client.25883 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.734+0000 7f6adb7fe700 1 --2- 172.21.15.44:0/4271348591 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555ecd387310 0x555ecd389730 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdbdb6e96d732bd0 server_cookie=fd7e7b537faf36cb in_seq=0 out_seq=0 2023-09-10T00:41:34.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.734+0000 7f66db0aab80 1 -- start start 2023-09-10T00:41:34.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.734+0000 7f66db0aab80 1 Processor -- start 2023-09-10T00:41:34.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.735+0000 7f66db0aab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83da710 0x55f9e83e6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.735+0000 7f66db0aab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83db030 0x55f9e83e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.735+0000 7f66db0aab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83ed650 0x55f9e83ec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.735+0000 7f66db0aab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9e82cab70 con 0x55f9e83db030 2023-09-10T00:41:34.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.735+0000 7f66db0aab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9e82ca870 con 0x55f9e83ed650 2023-09-10T00:41:34.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.735+0000 7f66db0aab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9e81ae920 con 0x55f9e83da710 2023-09-10T00:41:34.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.735+0000 7f66db0aab80 10 client.0 _ll_register_callbacks cb 0x55f9e8214a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:34.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.735+0000 7f66cd481700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83ed650 0x55f9e83ec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.735+0000 7f66ccc80700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83da710 0x55f9e83e6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.735+0000 7f66cd481700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83ed650 0x55f9e83ec810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51520/0 (socket says 172.21.15.44:51520) 2023-09-10T00:41:34.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.735+0000 7f66cd481700 1 -- 172.21.15.44:0/3865239895 learned_addr learned my addr 172.21.15.44:0/3865239895 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.735+0000 7f66c7fff700 1 --2- 172.21.15.44:0/3865239895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83db030 0x55f9e83e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.736+0000 7f66cd481700 1 -- 172.21.15.44:0/3865239895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83da710 msgr2=0x55f9e83e6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.736+0000 7f66cd481700 1 --2- 172.21.15.44:0/3865239895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83da710 0x55f9e83e6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.736+0000 7f66cd481700 1 -- 172.21.15.44:0/3865239895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83db030 msgr2=0x55f9e83e6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.736+0000 7f66cd481700 1 --2- 172.21.15.44:0/3865239895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83db030 0x55f9e83e6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.736+0000 7f66cd481700 1 -- 172.21.15.44:0/3865239895 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c81a5580 con 0x55f9e83ed650 2023-09-10T00:41:34.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.736+0000 7f66c7fff700 1 --2- 172.21.15.44:0/3865239895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83db030 0x55f9e83e6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:34.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.736+0000 7f66cd481700 1 --2- 172.21.15.44:0/3865239895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83ed650 0x55f9e83ec810 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66c0096780 tx=0x7f66c0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be7b1db50fcd478b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.736+0000 7f66a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:41:34.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.737+0000 7f66a7fff700 1 -- 172.21.15.44:0/3865239895 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c00a04a0 con 0x55f9e83ed650 2023-09-10T00:41:34.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.737+0000 7f66a7fff700 1 -- 172.21.15.44:0/3865239895 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f66c0098430 con 0x55f9e83ed650 2023-09-10T00:41:34.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.737+0000 7f66db0aab80 1 -- 172.21.15.44:0/3865239895 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9e82844f0 con 0x55f9e83ed650 2023-09-10T00:41:34.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.737+0000 7f66a7fff700 1 -- 172.21.15.44:0/3865239895 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66c009c260 con 0x55f9e83ed650 2023-09-10T00:41:34.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.737+0000 7f66db0aab80 10 client.25886 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:34.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.737+0000 7f66db0aab80 1 -- 172.21.15.44:0/3865239895 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f9e8141c60 con 0x55f9e83ed650 2023-09-10T00:41:34.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.738+0000 7f66a7fff700 1 -- 172.21.15.44:0/3865239895 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f9e82844f0 con 0x55f9e83ed650 2023-09-10T00:41:34.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.738+0000 7f66a7fff700 1 -- 172.21.15.44:0/3865239895 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f66c0080020 con 0x55f9e83ed650 2023-09-10T00:41:34.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.738+0000 7f66db0aab80 10 client.25886 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:34.769 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.738+0000 7f66db0aab80 10 client.25886 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-10T00:41:34.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.738+0000 7f66db0aab80 20 client.25886 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:34.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.738+0000 7f66db0aab80 1 --2- 172.21.15.44:0/3865239895 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55f9e8405620 0x55f9e8407a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.738+0000 7f66db0aab80 4 client.25886 mds_command: new command op to 24574 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:34.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.738+0000 7f66db0aab80 1 -- 172.21.15.44:0/3865239895 --> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9e83418a0 con 0x55f9e8405620 2023-09-10T00:41:34.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.739+0000 7f66ccc80700 1 --2- 172.21.15.44:0/3865239895 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55f9e8405620 0x55f9e8407a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.740+0000 7f66a7fff700 10 client.25886 ms_handle_connect on v2:172.21.15.131:6838/3222652693 2023-09-10T00:41:34.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.740+0000 7f66ccc80700 1 --2- 172.21.15.44:0/3865239895 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55f9e8405620 0x55f9e8407a40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=310fe8fed2d48c56 server_cookie=6cf9c84d51b546ee in_seq=0 out_seq=0 2023-09-10T00:41:34.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.741+0000 7f66a7fff700 1 -- 172.21.15.44:0/3865239895 <== mds.1 v2:172.21.15.131:6838/3222652693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f9e83418a0 con 0x55f9e8405620 2023-09-10T00:41:34.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.741+0000 7f66a7fff700 10 client.25886 handle_command_reply: tid=0 2023-09-10T00:41:34.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.741+0000 7f66db0aab80 1 client.25886 shutdown 2023-09-10T00:41:34.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.742+0000 7f66db0aab80 1 -- 172.21.15.44:0/3865239895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83ed650 msgr2=0x55f9e83ec810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.742+0000 7f66db0aab80 1 --2- 172.21.15.44:0/3865239895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83ed650 0x55f9e83ec810 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66c0096780 tx=0x7f66c0009260 comp rx=0 tx=0).stop 2023-09-10T00:41:34.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.742+0000 7f66db0aab80 1 -- 172.21.15.44:0/3865239895 shutdown_connections 2023-09-10T00:41:34.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.742+0000 7f66db0aab80 1 --2- 172.21.15.44:0/3865239895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83ed650 0x55f9e83ec810 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.742+0000 7f66db0aab80 1 --2- 172.21.15.44:0/3865239895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83da710 0x55f9e83e6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.742+0000 7f66db0aab80 1 --2- 172.21.15.44:0/3865239895 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55f9e8405620 0x55f9e8407a40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.742+0000 7f66db0aab80 1 --2- 172.21.15.44:0/3865239895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83db030 0x55f9e83e6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.742+0000 7f66db0aab80 1 -- 172.21.15.44:0/3865239895 >> 172.21.15.44:0/3865239895 conn(0x55f9e834f090 msgr2=0x55f9e83d57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.742+0000 7f66a7fff700 0 client.25886 ms_handle_reset on v2:172.21.15.131:6838/3222652693 2023-09-10T00:41:34.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.742+0000 7f66db0aab80 1 -- 172.21.15.44:0/3865239895 shutdown_connections 2023-09-10T00:41:34.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.742+0000 7f66db0aab80 1 -- 172.21.15.44:0/3865239895 wait complete. 2023-09-10T00:41:34.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.743+0000 7f66db0aab80 20 client.25886 trim_cache size 0 max 16384 2023-09-10T00:41:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.751+0000 7f6abb7fe700 1 -- 172.21.15.44:0/4271348591 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555ecd2638c0 con 0x555ecd387310 2023-09-10T00:41:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.751+0000 7f6abb7fe700 10 client.25883 handle_command_reply: tid=0 2023-09-10T00:41:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.751+0000 7f6aee54cb80 1 client.25883 shutdown 2023-09-10T00:41:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.751+0000 7f6aee54cb80 1 -- 172.21.15.44:0/4271348591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 msgr2=0x555ecd37f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.751+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/4271348591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd37f860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6acc096d40 tx=0x7f6acc099d60 comp rx=0 tx=0).stop 2023-09-10T00:41:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.752+0000 7f6aee54cb80 1 -- 172.21.15.44:0/4271348591 shutdown_connections 2023-09-10T00:41:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.752+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/4271348591 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555ecd387310 0x555ecd389730 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.752+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/4271348591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecd31e1a0 0x555ecd379e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.752+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/4271348591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecd318300 0x555ecd37f860 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.752+0000 7f6aee54cb80 1 --2- 172.21.15.44:0/4271348591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ecd318cc0 0x555ecd37fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.752+0000 7f6aee54cb80 1 -- 172.21.15.44:0/4271348591 >> 172.21.15.44:0/4271348591 conn(0x555ecd28d030 msgr2=0x555ecd317b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.752+0000 7f6abb7fe700 0 client.25883 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.752+0000 7f6aee54cb80 1 -- 172.21.15.44:0/4271348591 shutdown_connections 2023-09-10T00:41:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.752+0000 7f6aee54cb80 1 -- 172.21.15.44:0/4271348591 wait complete. 2023-09-10T00:41:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.753+0000 7f6aee54cb80 20 client.25883 trim_cache size 0 max 16384 2023-09-10T00:41:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:41:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:41:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:41:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:41:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.754+0000 7f6ae2386700 1 -- 172.21.15.44:0/2088935647 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6ac40a09a0 msgr2=0x7f6ac40a2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.754+0000 7f6ae2386700 1 --2- 172.21.15.44:0/2088935647 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6ac40a09a0 0x7f6ac40a2e60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6adc10c820 tx=0x7f6ad00073d0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.754+0000 7f6ae2386700 1 -- 172.21.15.44:0/2088935647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6adc0ff690 msgr2=0x7f6adc19e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.754+0000 7f6ae2386700 1 --2- 172.21.15.44:0/2088935647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6adc0ff690 0x7f6adc19e7b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00c010 tx=0x7f6acc005b00 comp rx=0 tx=0).stop 2023-09-10T00:41:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.754+0000 7f6ae2386700 1 -- 172.21.15.44:0/2088935647 shutdown_connections 2023-09-10T00:41:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.754+0000 7f6ae2386700 1 --2- 172.21.15.44:0/2088935647 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6ac40a09a0 0x7f6ac40a2e60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.754+0000 7f6ae2386700 1 --2- 172.21.15.44:0/2088935647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6adc1055b0 0x7f6adc10b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.754+0000 7f6ae2386700 1 --2- 172.21.15.44:0/2088935647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6adc0ff690 0x7f6adc19e7b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.754+0000 7f6ae2386700 1 --2- 172.21.15.44:0/2088935647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6adc100050 0x7f6adc19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.754+0000 7f6ae2386700 1 -- 172.21.15.44:0/2088935647 >> 172.21.15.44:0/2088935647 conn(0x7f6adc077430 msgr2=0x7f6adc0fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.754+0000 7f6ae2386700 1 -- 172.21.15.44:0/2088935647 shutdown_connections 2023-09-10T00:41:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:34.754+0000 7f6ae2386700 1 -- 172.21.15.44:0/2088935647 wait complete. 2023-09-10T00:41:34.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.765+0000 7f66db0aab80 1 Processor -- start 2023-09-10T00:41:34.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.765+0000 7f66db0aab80 1 -- start start 2023-09-10T00:41:34.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.766+0000 7f66db0aab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83da710 0x55f9e83e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.766+0000 7f66db0aab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83db030 0x55f9e83ec610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.766+0000 7f66db0aab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83ed650 0x55f9e83e66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.766+0000 7f66db0aab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9e83db410 con 0x55f9e83da710 2023-09-10T00:41:34.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.766+0000 7f66db0aab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9e83ecb50 con 0x55f9e83ed650 2023-09-10T00:41:34.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.766+0000 7f66db0aab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9e84080a0 con 0x55f9e83db030 2023-09-10T00:41:34.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.766+0000 7f66ccc80700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83da710 0x55f9e83e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.766+0000 7f66ccc80700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83da710 0x55f9e83e1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49786/0 (socket says 172.21.15.44:49786) 2023-09-10T00:41:34.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.766+0000 7f66ccc80700 1 -- 172.21.15.44:0/2727684844 learned_addr learned my addr 172.21.15.44:0/2727684844 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.766+0000 7f66c7fff700 1 --2- 172.21.15.44:0/2727684844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83db030 0x55f9e83ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.766+0000 7f66cd481700 1 --2- 172.21.15.44:0/2727684844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83ed650 0x55f9e83e66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.766+0000 7f66ccc80700 1 -- 172.21.15.44:0/2727684844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83db030 msgr2=0x55f9e83ec610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.766+0000 7f66ccc80700 1 --2- 172.21.15.44:0/2727684844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83db030 0x55f9e83ec610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.766+0000 7f66ccc80700 1 -- 172.21.15.44:0/2727684844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83ed650 msgr2=0x55f9e83e66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.766+0000 7f66ccc80700 1 --2- 172.21.15.44:0/2727684844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83ed650 0x55f9e83e66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.766+0000 7f66ccc80700 1 -- 172.21.15.44:0/2727684844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9e817f250 con 0x55f9e83da710 2023-09-10T00:41:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.767+0000 7f66ccc80700 1 --2- 172.21.15.44:0/2727684844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83da710 0x55f9e83e1d20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00b570 tx=0x7f66bc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48a8d6f1672060 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.767+0000 7f66b37fe700 1 -- 172.21.15.44:0/2727684844 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc025040 con 0x55f9e83da710 2023-09-10T00:41:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.767+0000 7f66b37fe700 1 -- 172.21.15.44:0/2727684844 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66bc027ca0 con 0x55f9e83da710 2023-09-10T00:41:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.767+0000 7f66b37fe700 1 -- 172.21.15.44:0/2727684844 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0275a0 con 0x55f9e83da710 2023-09-10T00:41:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.768+0000 7f66db0aab80 1 -- 172.21.15.44:0/2727684844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83da710 msgr2=0x55f9e83e1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.768+0000 7f66db0aab80 1 --2- 172.21.15.44:0/2727684844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83da710 0x55f9e83e1d20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00b570 tx=0x7f66bc015000 comp rx=0 tx=0).stop 2023-09-10T00:41:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.768+0000 7f66db0aab80 1 -- 172.21.15.44:0/2727684844 shutdown_connections 2023-09-10T00:41:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.768+0000 7f66db0aab80 1 --2- 172.21.15.44:0/2727684844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83ed650 0x55f9e83e66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.768+0000 7f66db0aab80 1 --2- 172.21.15.44:0/2727684844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83da710 0x55f9e83e1d20 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.768+0000 7f66db0aab80 1 --2- 172.21.15.44:0/2727684844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83db030 0x55f9e83ec610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.768+0000 7f66db0aab80 1 -- 172.21.15.44:0/2727684844 >> 172.21.15.44:0/2727684844 conn(0x55f9e834f090 msgr2=0x55f9e83e1610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.768+0000 7f66db0aab80 1 -- 172.21.15.44:0/2727684844 shutdown_connections 2023-09-10T00:41:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.768+0000 7f66db0aab80 1 -- 172.21.15.44:0/2727684844 wait complete. 2023-09-10T00:41:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.768+0000 7f66db0aab80 1 -- start start 2023-09-10T00:41:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.769+0000 7f66db0aab80 1 Processor -- start 2023-09-10T00:41:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.769+0000 7f66db0aab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83da710 0x55f9e83ea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.769+0000 7f66db0aab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83db030 0x55f9e83ea620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.769+0000 7f66db0aab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83ed650 0x55f9e8458410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.769+0000 7f66db0aab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9e83eadc0 con 0x55f9e83da710 2023-09-10T00:41:34.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.769+0000 7f66db0aab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9e8458980 con 0x55f9e83db030 2023-09-10T00:41:34.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.769+0000 7f66db0aab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9e8458b40 con 0x55f9e83ed650 2023-09-10T00:41:34.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.769+0000 7f66db0aab80 10 client.0 _ll_register_callbacks cb 0x7f66bc015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:34.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.769+0000 7f66ccc80700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83da710 0x55f9e83ea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.769+0000 7f66ccc80700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83da710 0x55f9e83ea0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49792/0 (socket says 172.21.15.44:49792) 2023-09-10T00:41:34.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.769+0000 7f66ccc80700 1 -- 172.21.15.44:0/183546292 learned_addr learned my addr 172.21.15.44:0/183546292 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.770+0000 7f66cd481700 1 --2- 172.21.15.44:0/183546292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83ed650 0x55f9e8458410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.770+0000 7f66ccc80700 1 -- 172.21.15.44:0/183546292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83ed650 msgr2=0x55f9e8458410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.770+0000 7f66ccc80700 1 --2- 172.21.15.44:0/183546292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83ed650 0x55f9e8458410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.770+0000 7f66ccc80700 1 -- 172.21.15.44:0/183546292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83db030 msgr2=0x55f9e83ea620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.770+0000 7f66ccc80700 1 --2- 172.21.15.44:0/183546292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83db030 0x55f9e83ea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.770+0000 7f66ccc80700 1 -- 172.21.15.44:0/183546292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66bc00fa40 con 0x55f9e83da710 2023-09-10T00:41:34.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.770+0000 7f66ccc80700 1 --2- 172.21.15.44:0/183546292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83da710 0x55f9e83ea0e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00f930 tx=0x7f66bc011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e346b0f4e1cbfa server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.770+0000 7f66a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:34.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.770+0000 7f66a7fff700 1 -- 172.21.15.44:0/183546292 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc005040 con 0x55f9e83da710 2023-09-10T00:41:34.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.770+0000 7f66db0aab80 1 -- 172.21.15.44:0/183546292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66c00112b0 con 0x55f9e83da710 2023-09-10T00:41:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.770+0000 7f66a7fff700 1 -- 172.21.15.44:0/183546292 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f66bc0051a0 con 0x55f9e83da710 2023-09-10T00:41:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.771+0000 7f66a7fff700 1 -- 172.21.15.44:0/183546292 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66bc005360 con 0x55f9e83da710 2023-09-10T00:41:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.771+0000 7f66db0aab80 10 client.16215 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.771+0000 7f66db0aab80 1 -- 172.21.15.44:0/183546292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f9e81572b0 con 0x55f9e83da710 2023-09-10T00:41:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.771+0000 7f66a7fff700 1 -- 172.21.15.44:0/183546292 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66c00112b0 con 0x55f9e83da710 2023-09-10T00:41:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.771+0000 7f66a7fff700 1 -- 172.21.15.44:0/183546292 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f66bc0272b0 con 0x55f9e83da710 2023-09-10T00:41:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.771+0000 7f66db0aab80 10 client.16215 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.771+0000 7f66db0aab80 10 client.16215 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-10T00:41:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.771+0000 7f66db0aab80 20 client.16215 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.771+0000 7f66db0aab80 1 --2- 172.21.15.44:0/183546292 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55f9e845a250 0x55f9e845c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.771+0000 7f66db0aab80 4 client.16215 mds_command: new command op to 24574 tid=0[{"prefix": "damage ls"}] 2023-09-10T00:41:34.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.771+0000 7f66db0aab80 1 -- 172.21.15.44:0/183546292 --> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f9e8325920 con 0x55f9e845a250 2023-09-10T00:41:34.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.771+0000 7f66c7fff700 1 --2- 172.21.15.44:0/183546292 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55f9e845a250 0x55f9e845c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.772+0000 7f66c7fff700 1 --2- 172.21.15.44:0/183546292 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55f9e845a250 0x55f9e845c670 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e5d883b43c14b81 server_cookie=f5d84249ba72d5b4 in_seq=0 out_seq=0 2023-09-10T00:41:34.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.772+0000 7f66a7fff700 10 client.16215 ms_handle_connect on v2:172.21.15.131:6838/3222652693 2023-09-10T00:41:34.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.772+0000 7f66a7fff700 1 -- 172.21.15.44:0/183546292 <== mds.1 v2:172.21.15.131:6838/3222652693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f9e8325920 con 0x55f9e845a250 2023-09-10T00:41:34.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.772+0000 7f66a7fff700 10 client.16215 handle_command_reply: tid=0 2023-09-10T00:41:34.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.772+0000 7f66db0aab80 1 client.16215 shutdown 2023-09-10T00:41:34.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.772+0000 7f66db0aab80 1 -- 172.21.15.44:0/183546292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83da710 msgr2=0x55f9e83ea0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.772+0000 7f66db0aab80 1 --2- 172.21.15.44:0/183546292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83da710 0x55f9e83ea0e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00f930 tx=0x7f66bc011c60 comp rx=0 tx=0).stop 2023-09-10T00:41:34.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.772+0000 7f66db0aab80 1 -- 172.21.15.44:0/183546292 shutdown_connections 2023-09-10T00:41:34.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.772+0000 7f66db0aab80 1 --2- 172.21.15.44:0/183546292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9e83ed650 0x55f9e8458410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.772+0000 7f66db0aab80 1 --2- 172.21.15.44:0/183546292 >> [v2:172.21.15.131:6838/3222652693,v1:172.21.15.131:6839/3222652693] conn(0x55f9e845a250 0x55f9e845c670 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.772+0000 7f66db0aab80 1 --2- 172.21.15.44:0/183546292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9e83da710 0x55f9e83ea0e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.772+0000 7f66db0aab80 1 --2- 172.21.15.44:0/183546292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9e83db030 0x55f9e83ea620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.772+0000 7f66a7fff700 0 client.16215 ms_handle_reset on v2:172.21.15.131:6838/3222652693 2023-09-10T00:41:34.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.772+0000 7f66db0aab80 1 -- 172.21.15.44:0/183546292 >> 172.21.15.44:0/183546292 conn(0x55f9e834f090 msgr2=0x55f9e83e0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.772+0000 7f66db0aab80 1 -- 172.21.15.44:0/183546292 shutdown_connections 2023-09-10T00:41:34.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.773+0000 7f66db0aab80 1 -- 172.21.15.44:0/183546292 wait complete. 2023-09-10T00:41:34.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.773+0000 7f66db0aab80 20 client.16215 trim_cache size 0 max 16384 2023-09-10T00:41:34.801 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2023-09-10T00:41:34.803 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.773+0000 7f66ceee4700 1 -- 172.21.15.44:0/1992247507 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f66b80a0950 msgr2=0x7f66b80a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.803 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.773+0000 7f66ceee4700 1 --2- 172.21.15.44:0/1992247507 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f66b80a0950 0x7f66b80a2e10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f66bc005950 tx=0x7f66bc0058e0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.803 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.773+0000 7f66ceee4700 1 -- 172.21.15.44:0/1992247507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66c8105570 msgr2=0x7f66c819c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.803 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.773+0000 7f66ceee4700 1 --2- 172.21.15.44:0/1992247507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66c8105570 0x7f66c819c010 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f66c0007ee0 tx=0x7f66c000d3b0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.803 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.773+0000 7f66ceee4700 1 -- 172.21.15.44:0/1992247507 shutdown_connections 2023-09-10T00:41:34.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.773+0000 7f66ceee4700 1 --2- 172.21.15.44:0/1992247507 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f66b80a0950 0x7f66b80a2e10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.773+0000 7f66ceee4700 1 --2- 172.21.15.44:0/1992247507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66c8105570 0x7f66c819c010 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.773+0000 7f66ceee4700 1 --2- 172.21.15.44:0/1992247507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66c80ff4b0 0x7f66c8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.773+0000 7f66ceee4700 1 --2- 172.21.15.44:0/1992247507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c80ffe70 0x7f66c8198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.773+0000 7f66ceee4700 1 -- 172.21.15.44:0/1992247507 >> 172.21.15.44:0/1992247507 conn(0x7f66c80fafa0 msgr2=0x7f66c80fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.773+0000 7f66ceee4700 1 -- 172.21.15.44:0/1992247507 shutdown_connections 2023-09-10T00:41:34.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.773+0000 7f66ceee4700 1 -- 172.21.15.44:0/1992247507 wait complete. 2023-09-10T00:41:34.805 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-10T00:41:34.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.956+0000 7fd4ad5db700 1 -- 172.21.15.44:0/3569025470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a80ffe00 msgr2=0x7fd4a810c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.957 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.956+0000 7fd4ad5db700 1 --2- 172.21.15.44:0/3569025470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a80ffe00 0x7fd4a810c310 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000b0a0 tx=0x7fd4a000b3b0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.956+0000 7fd4ad5db700 1 -- 172.21.15.44:0/3569025470 shutdown_connections 2023-09-10T00:41:34.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.956+0000 7fd4ad5db700 1 --2- 172.21.15.44:0/3569025470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a80ffe00 0x7fd4a810c310 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.956+0000 7fd4ad5db700 1 --2- 172.21.15.44:0/3569025470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a81054e0 0x7fd4a81058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.956+0000 7fd4ad5db700 1 --2- 172.21.15.44:0/3569025470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4a80ff440 0x7fd4a80ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.958 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.956+0000 7fd4ad5db700 1 -- 172.21.15.44:0/3569025470 >> 172.21.15.44:0/3569025470 conn(0x7fd4a80faf50 msgr2=0x7fd4a80fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.956+0000 7fd4ad5db700 1 -- 172.21.15.44:0/3569025470 shutdown_connections 2023-09-10T00:41:34.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.956+0000 7fd4ad5db700 1 -- 172.21.15.44:0/3569025470 wait complete. 2023-09-10T00:41:34.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.957+0000 7fd4ad5db700 1 Processor -- start 2023-09-10T00:41:34.959 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.957+0000 7fd4ad5db700 1 -- start start 2023-09-10T00:41:34.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.957+0000 7fd4ad5db700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4a80ff440 0x7fd4a8197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.957+0000 7fd4ad5db700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a80ffe00 0x7fd4a8197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.957+0000 7fd4ad5db700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a81054e0 0x7fd4a81a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.957+0000 7fd4ad5db700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a81a2900 con 0x7fd4a80ffe00 2023-09-10T00:41:34.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.957+0000 7fd4ad5db700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a81a2a40 con 0x7fd4a81054e0 2023-09-10T00:41:34.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.957+0000 7fd4ad5db700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a81a2b80 con 0x7fd4a80ff440 2023-09-10T00:41:34.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.957+0000 7fd4a67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a80ffe00 0x7fd4a8197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.958+0000 7fd4a67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a80ffe00 0x7fd4a8197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49808/0 (socket says 172.21.15.44:49808) 2023-09-10T00:41:34.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.958+0000 7fd4a67fc700 1 -- 172.21.15.44:0/4015732899 learned_addr learned my addr 172.21.15.44:0/4015732899 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.958+0000 7fd4a6ffd700 1 --2- 172.21.15.44:0/4015732899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4a80ff440 0x7fd4a8197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.958+0000 7fd4a77fe700 1 --2- 172.21.15.44:0/4015732899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a81054e0 0x7fd4a81a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.958+0000 7fd4a67fc700 1 -- 172.21.15.44:0/4015732899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4a80ff440 msgr2=0x7fd4a8197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.958+0000 7fd4a67fc700 1 --2- 172.21.15.44:0/4015732899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4a80ff440 0x7fd4a8197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.958+0000 7fd4a67fc700 1 -- 172.21.15.44:0/4015732899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a81054e0 msgr2=0x7fd4a81a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.958+0000 7fd4a67fc700 1 --2- 172.21.15.44:0/4015732899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a81054e0 0x7fd4a81a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.958+0000 7fd4a67fc700 1 -- 172.21.15.44:0/4015732899 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a0009d00 con 0x7fd4a80ffe00 2023-09-10T00:41:34.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.958+0000 7fd4a67fc700 1 --2- 172.21.15.44:0/4015732899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a80ffe00 0x7fd4a8197880 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00d950 tx=0x7fd49c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.958+0000 7fd48ffff700 1 -- 172.21.15.44:0/4015732899 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c00f890 con 0x7fd4a80ffe00 2023-09-10T00:41:34.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.958+0000 7fd48ffff700 1 -- 172.21.15.44:0/4015732899 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd49c00fe80 con 0x7fd4a80ffe00 2023-09-10T00:41:34.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.958+0000 7fd4ad5db700 1 -- 172.21.15.44:0/4015732899 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a81a2de0 con 0x7fd4a80ffe00 2023-09-10T00:41:34.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.958+0000 7fd48ffff700 1 -- 172.21.15.44:0/4015732899 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c00e3b0 con 0x7fd4a80ffe00 2023-09-10T00:41:34.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.958+0000 7fd4ad5db700 1 -- 172.21.15.44:0/4015732899 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a81a3260 con 0x7fd4a80ffe00 2023-09-10T00:41:34.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.961+0000 7fd48ffff700 1 -- 172.21.15.44:0/4015732899 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7fd49c010460 con 0x7fd4a80ffe00 2023-09-10T00:41:34.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.962+0000 7fd48ffff700 1 --2- 172.21.15.44:0/4015732899 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd4900a08d0 0x7fd4900a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.962+0000 7fd48ffff700 1 -- 172.21.15.44:0/4015732899 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fd49c0cc760 con 0x7fd4a80ffe00 2023-09-10T00:41:34.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.962+0000 7fd4a6ffd700 1 --2- 172.21.15.44:0/4015732899 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd4900a08d0 0x7fd4900a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.963+0000 7fd4a6ffd700 1 --2- 172.21.15.44:0/4015732899 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd4900a08d0 0x7fd4900a2d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd498005950 tx=0x7fd49800b410 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.964+0000 7fd4b97a1b80 1 Processor -- start 2023-09-10T00:41:34.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.964+0000 7fd4b97a1b80 1 -- start start 2023-09-10T00:41:34.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.964+0000 7fd4b97a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 0x563a0dcbcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.964+0000 7fd4b97a1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcbd030 0x563a0dcc6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.964+0000 7fd4b97a1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcc6e40 0x563a0dcc9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.964+0000 7fd4b97a1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0dc15730 con 0x563a0dcbd030 2023-09-10T00:41:34.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.964+0000 7fd4b97a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0dc13050 con 0x563a0dcbc710 2023-09-10T00:41:34.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.964+0000 7fd4b97a1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0dc0e940 con 0x563a0dcc6e40 2023-09-10T00:41:34.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.964+0000 7fd4a67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcbd030 0x563a0dcc6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.964+0000 7fd4a6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 0x563a0dcbcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.964+0000 7fd4a67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcbd030 0x563a0dcc6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49816/0 (socket says 172.21.15.44:49816) 2023-09-10T00:41:34.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.964+0000 7fd4a67fc700 1 -- 172.21.15.44:0/2187901228 learned_addr learned my addr 172.21.15.44:0/2187901228 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.964+0000 7fd4a77fe700 1 --2- 172.21.15.44:0/2187901228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcc6e40 0x563a0dcc9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.964+0000 7fd4a67fc700 1 -- 172.21.15.44:0/2187901228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcc6e40 msgr2=0x563a0dcc9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.964+0000 7fd4a67fc700 1 --2- 172.21.15.44:0/2187901228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcc6e40 0x563a0dcc9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.965+0000 7fd4a67fc700 1 -- 172.21.15.44:0/2187901228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 msgr2=0x563a0dcbcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.965+0000 7fd4a67fc700 1 --2- 172.21.15.44:0/2187901228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 0x563a0dcbcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.965+0000 7fd4a67fc700 1 -- 172.21.15.44:0/2187901228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a0da3ade0 con 0x563a0dcbd030 2023-09-10T00:41:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.965+0000 7fd4a6ffd700 1 --2- 172.21.15.44:0/2187901228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 0x563a0dcbcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.965+0000 7fd4a67fc700 1 --2- 172.21.15.44:0/2187901228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcbd030 0x563a0dcc6900 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd49c096090 tx=0x7fd49c099000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e25a5f9bb05ba0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.965+0000 7fd48d7fa700 1 -- 172.21.15.44:0/2187901228 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c099620 con 0x563a0dcbd030 2023-09-10T00:41:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.965+0000 7fd48d7fa700 1 -- 172.21.15.44:0/2187901228 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd49c09d450 con 0x563a0dcbd030 2023-09-10T00:41:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.965+0000 7fd48d7fa700 1 -- 172.21.15.44:0/2187901228 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c0842f0 con 0x563a0dcbd030 2023-09-10T00:41:34.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.966+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2187901228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcbd030 msgr2=0x563a0dcc6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.966+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/2187901228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcbd030 0x563a0dcc6900 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd49c096090 tx=0x7fd49c099000 comp rx=0 tx=0).stop 2023-09-10T00:41:34.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.966+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2187901228 shutdown_connections 2023-09-10T00:41:34.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.966+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/2187901228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcc6e40 0x563a0dcc9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.966+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/2187901228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 0x563a0dcbcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.966+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/2187901228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcbd030 0x563a0dcc6900 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.966+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2187901228 >> 172.21.15.44:0/2187901228 conn(0x563a0dc31090 msgr2=0x563a0dcb7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.966+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2187901228 shutdown_connections 2023-09-10T00:41:34.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.966+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2187901228 wait complete. 2023-09-10T00:41:34.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.966+0000 7fd4b97a1b80 1 -- start start 2023-09-10T00:41:34.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.966+0000 7fd4b97a1b80 1 Processor -- start 2023-09-10T00:41:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.966+0000 7fd4b97a1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcbc710 0x563a0dcc6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4b97a1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcbd030 0x563a0dcc3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4b97a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcc6e40 0x563a0dcc4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4b97a1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0dbacb70 con 0x563a0dcbc710 2023-09-10T00:41:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4b97a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0dbac870 con 0x563a0dcc6e40 2023-09-10T00:41:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4b97a1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0da90920 con 0x563a0dcbd030 2023-09-10T00:41:34.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4b97a1b80 10 client.0 _ll_register_callbacks cb 0x563a0daf6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:34.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4a6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcbc710 0x563a0dcc6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4a6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcbc710 0x563a0dcc6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49820/0 (socket says 172.21.15.44:49820) 2023-09-10T00:41:34.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4a6ffd700 1 -- 172.21.15.44:0/2200336653 learned_addr learned my addr 172.21.15.44:0/2200336653 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.980 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4a67fc700 1 --2- 172.21.15.44:0/2200336653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcbd030 0x563a0dcc3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4a77fe700 1 --2- 172.21.15.44:0/2200336653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcc6e40 0x563a0dcc4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4a6ffd700 1 -- 172.21.15.44:0/2200336653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcbd030 msgr2=0x563a0dcc3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4a6ffd700 1 --2- 172.21.15.44:0/2200336653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcbd030 0x563a0dcc3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4a6ffd700 1 -- 172.21.15.44:0/2200336653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcc6e40 msgr2=0x563a0dcc4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.981 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4a6ffd700 1 --2- 172.21.15.44:0/2200336653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcc6e40 0x563a0dcc4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4a6ffd700 1 -- 172.21.15.44:0/2200336653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd498010a50 con 0x563a0dcbc710 2023-09-10T00:41:34.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4a77fe700 1 --2- 172.21.15.44:0/2200336653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcc6e40 0x563a0dcc4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:41:34.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4a6ffd700 1 --2- 172.21.15.44:0/2200336653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcbc710 0x563a0dcc6810 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd498011790 tx=0x7fd4980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74073060b7d75e4d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd48aff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:34.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd48aff5700 1 -- 172.21.15.44:0/2200336653 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd498004a30 con 0x563a0dcbc710 2023-09-10T00:41:34.982 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd48aff5700 1 -- 172.21.15.44:0/2200336653 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fd498017070 con 0x563a0dcbc710 2023-09-10T00:41:34.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.967+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2200336653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a0db664f0 con 0x563a0dcbc710 2023-09-10T00:41:34.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.968+0000 7fd48aff5700 1 -- 172.21.15.44:0/2200336653 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd498004b90 con 0x563a0dcbc710 2023-09-10T00:41:34.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.968+0000 7fd4b97a1b80 10 client.16227 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:34.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.968+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2200336653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fd4a81a3260 con 0x563a0dcbc710 2023-09-10T00:41:34.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.968+0000 7fd48aff5700 1 -- 172.21.15.44:0/2200336653 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563a0db664f0 con 0x563a0dcbc710 2023-09-10T00:41:34.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.968+0000 7fd48aff5700 1 -- 172.21.15.44:0/2200336653 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd49801f430 con 0x563a0dcbc710 2023-09-10T00:41:34.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.968+0000 7fd4b97a1b80 10 client.16227 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:34.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.968+0000 7fd4b97a1b80 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:34.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.968+0000 7fd4b97a1b80 20 client.16227 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:34.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.968+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/2200336653 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a0dcd6940 0x563a0dcd8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.968+0000 7fd4b97a1b80 4 client.16227 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:34.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.968+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2200336653 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a0dc238a0 con 0x563a0dcd6940 2023-09-10T00:41:34.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.968+0000 7fd4a67fc700 1 --2- 172.21.15.44:0/2200336653 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a0dcd6940 0x563a0dcd8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.969+0000 7fd48aff5700 10 client.16227 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:34.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.969+0000 7fd4a67fc700 1 --2- 172.21.15.44:0/2200336653 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a0dcd6940 0x563a0dcd8d60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae5b777eb499061a server_cookie=a73ff8141472aa87 in_seq=0 out_seq=0 2023-09-10T00:41:34.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.969+0000 7fd48aff5700 1 -- 172.21.15.44:0/2200336653 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563a0dc238a0 con 0x563a0dcd6940 2023-09-10T00:41:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.969+0000 7fd48aff5700 10 client.16227 handle_command_reply: tid=0 2023-09-10T00:41:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.969+0000 7fd4b97a1b80 1 client.16227 shutdown 2023-09-10T00:41:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.970+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2200336653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcbc710 msgr2=0x563a0dcc6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.970+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/2200336653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcbc710 0x563a0dcc6810 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd498011790 tx=0x7fd4980047c0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.986 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.970+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2200336653 shutdown_connections 2023-09-10T00:41:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.970+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/2200336653 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a0dcd6940 0x563a0dcd8d60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.970+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/2200336653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcc6e40 0x563a0dcc4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.970+0000 7fd48aff5700 0 client.16227 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.970+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/2200336653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcbc710 0x563a0dcc6810 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.987 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.970+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/2200336653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcbd030 0x563a0dcc3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.970+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2200336653 >> 172.21.15.44:0/2200336653 conn(0x563a0dc31090 msgr2=0x563a0dcb77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.970+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2200336653 shutdown_connections 2023-09-10T00:41:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.970+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2200336653 wait complete. 2023-09-10T00:41:34.988 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.970+0000 7fd4b97a1b80 20 client.16227 trim_cache size 0 max 16384 2023-09-10T00:41:34.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.986+0000 7fd4b97a1b80 1 Processor -- start 2023-09-10T00:41:34.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.986+0000 7fd4b97a1b80 1 -- start start 2023-09-10T00:41:34.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.986+0000 7fd4b97a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 0x563a0dcc9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.986+0000 7fd4b97a1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcbd030 0x563a0dcc6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.986+0000 7fd4b97a1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcc6e40 0x563a0dcbed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:34.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.986+0000 7fd4b97a1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0dcd3ef0 con 0x563a0dcc6e40 2023-09-10T00:41:34.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.986+0000 7fd4b97a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0dcb43d0 con 0x563a0dcbc710 2023-09-10T00:41:34.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.986+0000 7fd4b97a1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0dcbe890 con 0x563a0dcbd030 2023-09-10T00:41:34.996 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.986+0000 7fd4a77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcc6e40 0x563a0dcbed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.986+0000 7fd4a77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcc6e40 0x563a0dcbed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49832/0 (socket says 172.21.15.44:49832) 2023-09-10T00:41:34.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.986+0000 7fd4a77fe700 1 -- 172.21.15.44:0/2397440832 learned_addr learned my addr 172.21.15.44:0/2397440832 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:34.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.986+0000 7fd4a6ffd700 1 --2- 172.21.15.44:0/2397440832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 0x563a0dcc9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.987+0000 7fd4a67fc700 1 --2- 172.21.15.44:0/2397440832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcbd030 0x563a0dcc6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:34.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.987+0000 7fd4a77fe700 1 -- 172.21.15.44:0/2397440832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcbd030 msgr2=0x563a0dcc6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.987+0000 7fd4a77fe700 1 --2- 172.21.15.44:0/2397440832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcbd030 0x563a0dcc6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.987+0000 7fd4a77fe700 1 -- 172.21.15.44:0/2397440832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 msgr2=0x563a0dcc9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:34.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.987+0000 7fd4a77fe700 1 --2- 172.21.15.44:0/2397440832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 0x563a0dcc9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:34.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.987+0000 7fd4a77fe700 1 -- 172.21.15.44:0/2397440832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a0da23c60 con 0x563a0dcc6e40 2023-09-10T00:41:34.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.987+0000 7fd4a67fc700 1 --2- 172.21.15.44:0/2397440832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcbd030 0x563a0dcc6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:34.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.987+0000 7fd4a77fe700 1 --2- 172.21.15.44:0/2397440832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcc6e40 0x563a0dcbed60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0015000 tx=0x7fd4a00092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef82e3f73683e46 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:34.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.987+0000 7fd48dffb700 1 -- 172.21.15.44:0/2397440832 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a000e040 con 0x563a0dcc6e40 2023-09-10T00:41:34.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.987+0000 7fd48dffb700 1 -- 172.21.15.44:0/2397440832 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4a0007970 con 0x563a0dcc6e40 2023-09-10T00:41:35.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.987+0000 7fd48dffb700 1 -- 172.21.15.44:0/2397440832 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a00055f0 con 0x563a0dcc6e40 2023-09-10T00:41:35.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.988+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2397440832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcc6e40 msgr2=0x563a0dcbed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:35.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.988+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/2397440832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcc6e40 0x563a0dcbed60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0015000 tx=0x7fd4a00092e0 comp rx=0 tx=0).stop 2023-09-10T00:41:35.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.988+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2397440832 shutdown_connections 2023-09-10T00:41:35.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.988+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/2397440832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcc6e40 0x563a0dcbed60 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:35.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.988+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/2397440832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 0x563a0dcc9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:35.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.988+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/2397440832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcbd030 0x563a0dcc6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:35.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.988+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2397440832 >> 172.21.15.44:0/2397440832 conn(0x563a0dc31090 msgr2=0x563a0dcd2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:35.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.988+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2397440832 shutdown_connections 2023-09-10T00:41:35.001 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.988+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/2397440832 wait complete. 2023-09-10T00:41:35.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.988+0000 7fd4b97a1b80 1 -- start start 2023-09-10T00:41:35.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.988+0000 7fd4b97a1b80 1 Processor -- start 2023-09-10T00:41:35.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4b97a1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 0x563a0dccbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:35.002 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4b97a1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcbd030 0x563a0dccc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4b97a1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcc6e40 0x563a0dd29750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4b97a1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0dd29c90 con 0x563a0dcc6e40 2023-09-10T00:41:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4b97a1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0dd29e00 con 0x563a0dcbc710 2023-09-10T00:41:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4b97a1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0dd29fc0 con 0x563a0dcbd030 2023-09-10T00:41:35.003 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4b97a1b80 10 client.0 _ll_register_callbacks cb 0x7fd498010ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4a6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 0x563a0dccbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4a77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcc6e40 0x563a0dd29750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4a6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 0x563a0dccbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51574/0 (socket says 172.21.15.44:51574) 2023-09-10T00:41:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4a6ffd700 1 -- 172.21.15.44:0/3759890188 learned_addr learned my addr 172.21.15.44:0/3759890188 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4a67fc700 1 --2- 172.21.15.44:0/3759890188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcbd030 0x563a0dccc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:35.004 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4a6ffd700 1 -- 172.21.15.44:0/3759890188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcbd030 msgr2=0x563a0dccc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4a6ffd700 1 --2- 172.21.15.44:0/3759890188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcbd030 0x563a0dccc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4a6ffd700 1 -- 172.21.15.44:0/3759890188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcc6e40 msgr2=0x563a0dd29750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4a6ffd700 1 --2- 172.21.15.44:0/3759890188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcc6e40 0x563a0dd29750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.989+0000 7fd4a6ffd700 1 -- 172.21.15.44:0/3759890188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a810cc10 con 0x563a0dcbc710 2023-09-10T00:41:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.990+0000 7fd4a6ffd700 1 --2- 172.21.15.44:0/3759890188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 0x563a0dccbba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd498005cf0 tx=0x7fd498013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c98c0b226c313eb3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:35.005 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.990+0000 7fd48aff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:41:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.990+0000 7fd48aff5700 1 -- 172.21.15.44:0/3759890188 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd498034830 con 0x563a0dcbc710 2023-09-10T00:41:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.990+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/3759890188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd49c09d450 con 0x563a0dcbc710 2023-09-10T00:41:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.990+0000 7fd48aff5700 1 -- 172.21.15.44:0/3759890188 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fd498017040 con 0x563a0dcbc710 2023-09-10T00:41:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.990+0000 7fd48aff5700 1 -- 172.21.15.44:0/3759890188 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd498034990 con 0x563a0dcbc710 2023-09-10T00:41:35.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.990+0000 7fd4b97a1b80 10 client.25901 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.990+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/3759890188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563a0da61250 con 0x563a0dcbc710 2023-09-10T00:41:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.990+0000 7fd48aff5700 1 -- 172.21.15.44:0/3759890188 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd49c09d450 con 0x563a0dcbc710 2023-09-10T00:41:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.990+0000 7fd48aff5700 1 -- 172.21.15.44:0/3759890188 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd49803da60 con 0x563a0dcbc710 2023-09-10T00:41:35.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.990+0000 7fd4b97a1b80 10 client.25901 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:35.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.990+0000 7fd4b97a1b80 10 client.25901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:35.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.990+0000 7fd4b97a1b80 20 client.25901 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:35.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.990+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/3759890188 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a0dd2b7a0 0x563a0dd2dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:35.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.990+0000 7fd4b97a1b80 4 client.25901 mds_command: new command op to 24605 tid=0[{"prefix": "damage ls"}] 2023-09-10T00:41:35.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.990+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/3759890188 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563a0dc07920 con 0x563a0dd2b7a0 2023-09-10T00:41:35.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.991+0000 7fd4a67fc700 1 --2- 172.21.15.44:0/3759890188 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a0dd2b7a0 0x563a0dd2dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:35.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.991+0000 7fd48aff5700 10 client.25901 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:35.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.991+0000 7fd4a67fc700 1 --2- 172.21.15.44:0/3759890188 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a0dd2b7a0 0x563a0dd2dbc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5476d763ba93cc25 server_cookie=d77165b24dd52a94 in_seq=0 out_seq=0 2023-09-10T00:41:35.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.991+0000 7fd48aff5700 1 -- 172.21.15.44:0/3759890188 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563a0dc07920 con 0x563a0dd2b7a0 2023-09-10T00:41:35.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.991+0000 7fd48aff5700 10 client.25901 handle_command_reply: tid=0 2023-09-10T00:41:35.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.991+0000 7fd4b97a1b80 1 client.25901 shutdown 2023-09-10T00:41:35.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.992+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/3759890188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 msgr2=0x563a0dccbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:35.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.992+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/3759890188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 0x563a0dccbba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd498005cf0 tx=0x7fd498013000 comp rx=0 tx=0).stop 2023-09-10T00:41:35.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.992+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/3759890188 shutdown_connections 2023-09-10T00:41:35.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.992+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/3759890188 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a0dd2b7a0 0x563a0dd2dbc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:35.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.992+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/3759890188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a0dcc6e40 0x563a0dd29750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:35.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.992+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/3759890188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a0dcbc710 0x563a0dccbba0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:35.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.992+0000 7fd4b97a1b80 1 --2- 172.21.15.44:0/3759890188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a0dcbd030 0x563a0dccc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:35.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.992+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/3759890188 >> 172.21.15.44:0/3759890188 conn(0x563a0dc31090 msgr2=0x563a0dcbed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:35.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.992+0000 7fd48aff5700 0 client.25901 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:35.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.992+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/3759890188 shutdown_connections 2023-09-10T00:41:35.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.992+0000 7fd4b97a1b80 1 -- 172.21.15.44:0/3759890188 wait complete. 2023-09-10T00:41:35.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.992+0000 7fd4b97a1b80 20 client.25901 trim_cache size 0 max 16384 2023-09-10T00:41:35.013 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2023-09-10T00:41:35.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.992+0000 7fd4ad5db700 1 -- 172.21.15.44:0/4015732899 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd4900a08d0 msgr2=0x7fd4900a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:35.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.992+0000 7fd4ad5db700 1 --2- 172.21.15.44:0/4015732899 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd4900a08d0 0x7fd4900a2d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd498005950 tx=0x7fd49800b410 comp rx=0 tx=0).stop 2023-09-10T00:41:35.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.992+0000 7fd4ad5db700 1 -- 172.21.15.44:0/4015732899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a80ffe00 msgr2=0x7fd4a8197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:35.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.992+0000 7fd4ad5db700 1 --2- 172.21.15.44:0/4015732899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a80ffe00 0x7fd4a8197880 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00d950 tx=0x7fd49c00dc60 comp rx=0 tx=0).stop 2023-09-10T00:41:35.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.993+0000 7fd4ad5db700 1 -- 172.21.15.44:0/4015732899 shutdown_connections 2023-09-10T00:41:35.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.993+0000 7fd4ad5db700 1 --2- 172.21.15.44:0/4015732899 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd4900a08d0 0x7fd4900a2d90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.993+0000 7fd4ad5db700 1 --2- 172.21.15.44:0/4015732899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a81054e0 0x7fd4a81a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.993+0000 7fd4ad5db700 1 --2- 172.21.15.44:0/4015732899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4a80ff440 0x7fd4a8197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.993+0000 7fd4ad5db700 1 --2- 172.21.15.44:0/4015732899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a80ffe00 0x7fd4a8197880 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.993+0000 7fd4ad5db700 1 -- 172.21.15.44:0/4015732899 >> 172.21.15.44:0/4015732899 conn(0x7fd4a80faf50 msgr2=0x7fd4a80fb530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.993+0000 7fd4ad5db700 1 -- 172.21.15.44:0/4015732899 shutdown_connections 2023-09-10T00:41:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:34.993+0000 7fd4ad5db700 1 -- 172.21.15.44:0/4015732899 wait complete. 2023-09-10T00:41:35.018 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:34 smithi044 ceph-mon[98676]: pgmap v30: 129 pgs: 129 active+clean; 657 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 34 MiB/s wr, 38 op/s 2023-09-10T00:41:35.018 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:34 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/865454883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:41:35.068 INFO:teuthology.orchestra.run.smithi044.stderr:3100000 of 50000000 tuples (6%) done (elapsed 34.70 s, remaining 525.01 s) 2023-09-10T00:41:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:34 smithi099 ceph-mon[102692]: pgmap v30: 129 pgs: 129 active+clean; 657 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 34 MiB/s wr, 38 op/s 2023-09-10T00:41:35.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:34 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/865454883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:41:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:34 smithi131 ceph-mon[103713]: pgmap v30: 129 pgs: 129 active+clean; 657 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 34 MiB/s wr, 38 op/s 2023-09-10T00:41:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:34 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/865454883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-10T00:41:36.020 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-10T00:41:36.020 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-10T00:41:36.164 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.164+0000 7fa455fe5700 1 -- 172.21.15.44:0/703054223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4500ff4b0 msgr2=0x7fa4500ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.164 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.164+0000 7fa455fe5700 1 --2- 172.21.15.44:0/703054223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4500ff4b0 0x7fa4500ff930 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa4440099c0 tx=0x7fa444009cd0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.164+0000 7fa455fe5700 1 -- 172.21.15.44:0/703054223 shutdown_connections 2023-09-10T00:41:36.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.164+0000 7fa455fe5700 1 --2- 172.21.15.44:0/703054223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4500ffe70 0x7fa45010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.164+0000 7fa455fe5700 1 --2- 172.21.15.44:0/703054223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa450105570 0x7fa450105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.164+0000 7fa455fe5700 1 --2- 172.21.15.44:0/703054223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4500ff4b0 0x7fa4500ff930 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.164+0000 7fa455fe5700 1 -- 172.21.15.44:0/703054223 >> 172.21.15.44:0/703054223 conn(0x7fa4500fafa0 msgr2=0x7fa4500fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:36.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.164+0000 7fa455fe5700 1 -- 172.21.15.44:0/703054223 shutdown_connections 2023-09-10T00:41:36.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.164+0000 7fa455fe5700 1 -- 172.21.15.44:0/703054223 wait complete. 2023-09-10T00:41:36.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.165+0000 7fa455fe5700 1 Processor -- start 2023-09-10T00:41:36.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.165+0000 7fa455fe5700 1 -- start start 2023-09-10T00:41:36.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.165+0000 7fa455fe5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4500ff4b0 0x7fa4501965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.165+0000 7fa455fe5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4500ffe70 0x7fa450196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.165+0000 7fa455fe5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa450105570 0x7fa45019af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.165+0000 7fa455fe5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45019b590 con 0x7fa4500ffe70 2023-09-10T00:41:36.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.165+0000 7fa455fe5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45019b700 con 0x7fa4500ff4b0 2023-09-10T00:41:36.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.165+0000 7fa455fe5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45019b8c0 con 0x7fa450105570 2023-09-10T00:41:36.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.165+0000 7fa44effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4500ffe70 0x7fa450196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.165+0000 7fa44effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4500ffe70 0x7fa450196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49846/0 (socket says 172.21.15.44:49846) 2023-09-10T00:41:36.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.165+0000 7fa44effd700 1 -- 172.21.15.44:0/1863209539 learned_addr learned my addr 172.21.15.44:0/1863209539 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:36.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.165+0000 7fa44ffff700 1 --2- 172.21.15.44:0/1863209539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa450105570 0x7fa45019af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.166+0000 7fa44f7fe700 1 --2- 172.21.15.44:0/1863209539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4500ff4b0 0x7fa4501965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.166+0000 7fa44effd700 1 -- 172.21.15.44:0/1863209539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa450105570 msgr2=0x7fa45019af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.166+0000 7fa44effd700 1 --2- 172.21.15.44:0/1863209539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa450105570 0x7fa45019af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.166+0000 7fa44effd700 1 -- 172.21.15.44:0/1863209539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4500ff4b0 msgr2=0x7fa4501965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.166+0000 7fa44effd700 1 --2- 172.21.15.44:0/1863209539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4500ff4b0 0x7fa4501965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.166+0000 7fa44effd700 1 -- 172.21.15.44:0/1863209539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4440095f0 con 0x7fa4500ffe70 2023-09-10T00:41:36.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.166+0000 7fa44ffff700 1 --2- 172.21.15.44:0/1863209539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa450105570 0x7fa45019af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:36.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.166+0000 7fa44effd700 1 --2- 172.21.15.44:0/1863209539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4500ffe70 0x7fa450196af0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa444006010 tx=0x7fa44400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:36.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.166+0000 7fa44cff9700 1 -- 172.21.15.44:0/1863209539 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444003aa0 con 0x7fa4500ffe70 2023-09-10T00:41:36.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.166+0000 7fa44cff9700 1 -- 172.21.15.44:0/1863209539 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa444023070 con 0x7fa4500ffe70 2023-09-10T00:41:36.173 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.166+0000 7fa455fe5700 1 -- 172.21.15.44:0/1863209539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45019bb40 con 0x7fa4500ffe70 2023-09-10T00:41:36.173 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.166+0000 7fa44cff9700 1 -- 172.21.15.44:0/1863209539 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44401fce0 con 0x7fa4500ffe70 2023-09-10T00:41:36.173 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.167+0000 7fa455fe5700 1 -- 172.21.15.44:0/1863209539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa450101230 con 0x7fa4500ffe70 2023-09-10T00:41:36.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.170+0000 7fa44cff9700 1 -- 172.21.15.44:0/1863209539 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7fa444025030 con 0x7fa4500ffe70 2023-09-10T00:41:36.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.170+0000 7fa44cff9700 1 --2- 172.21.15.44:0/1863209539 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4380a0950 0x7fa4380a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.171+0000 7fa44cff9700 1 -- 172.21.15.44:0/1863209539 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa4440d1690 con 0x7fa4500ffe70 2023-09-10T00:41:36.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.171+0000 7fa44f7fe700 1 --2- 172.21.15.44:0/1863209539 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4380a0950 0x7fa4380a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.172+0000 7fa44f7fe700 1 --2- 172.21.15.44:0/1863209539 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4380a0950 0x7fa4380a2e10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa440007b60 tx=0x7fa4400073d0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:36.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.172+0000 7fa4621abb80 1 Processor -- start 2023-09-10T00:41:36.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.172+0000 7fa4621abb80 1 -- start start 2023-09-10T00:41:36.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.172+0000 7fa4621abb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360dab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.172+0000 7fa4621abb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360db0b0 0x55f8360e4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.172+0000 7fa4621abb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360e4ec0 0x55f8360e72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.172+0000 7fa4621abb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8360337b0 con 0x55f8360da790 2023-09-10T00:41:36.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.172+0000 7fa4621abb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8360310d0 con 0x55f8360db0b0 2023-09-10T00:41:36.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.172+0000 7fa4621abb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83602c9c0 con 0x55f8360e4ec0 2023-09-10T00:41:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.172+0000 7fa44f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360dab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.172+0000 7fa44f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360dab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49852/0 (socket says 172.21.15.44:49852) 2023-09-10T00:41:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.173+0000 7fa44f7fe700 1 -- 172.21.15.44:0/2892866286 learned_addr learned my addr 172.21.15.44:0/2892866286 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.173+0000 7fa44ffff700 1 --2- 172.21.15.44:0/2892866286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360e4ec0 0x55f8360e72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.173+0000 7fa44effd700 1 --2- 172.21.15.44:0/2892866286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360db0b0 0x55f8360e4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.173+0000 7fa44f7fe700 1 -- 172.21.15.44:0/2892866286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360e4ec0 msgr2=0x55f8360e72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:36.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.173+0000 7fa44f7fe700 1 --2- 172.21.15.44:0/2892866286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360e4ec0 0x55f8360e72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.173+0000 7fa44f7fe700 1 -- 172.21.15.44:0/2892866286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360db0b0 msgr2=0x55f8360e4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:36.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.173+0000 7fa44f7fe700 1 --2- 172.21.15.44:0/2892866286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360db0b0 0x55f8360e4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.173+0000 7fa44f7fe700 1 -- 172.21.15.44:0/2892866286 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f835e58e60 con 0x55f8360da790 2023-09-10T00:41:36.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.173+0000 7fa44f7fe700 1 --2- 172.21.15.44:0/2892866286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360dab70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa440010d90 tx=0x7fa440013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b0d16898455581 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:36.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.173+0000 7fa435ffb700 1 -- 172.21.15.44:0/2892866286 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44001e420 con 0x55f8360da790 2023-09-10T00:41:36.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.173+0000 7fa435ffb700 1 -- 172.21.15.44:0/2892866286 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa440015070 con 0x55f8360da790 2023-09-10T00:41:36.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.173+0000 7fa435ffb700 1 -- 172.21.15.44:0/2892866286 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44001c750 con 0x55f8360da790 2023-09-10T00:41:36.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.174+0000 7fa4621abb80 1 -- 172.21.15.44:0/2892866286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 msgr2=0x55f8360dab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.174+0000 7fa4621abb80 1 --2- 172.21.15.44:0/2892866286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360dab70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa440010d90 tx=0x7fa440013000 comp rx=0 tx=0).stop 2023-09-10T00:41:36.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.174+0000 7fa4621abb80 1 -- 172.21.15.44:0/2892866286 shutdown_connections 2023-09-10T00:41:36.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.174+0000 7fa4621abb80 1 --2- 172.21.15.44:0/2892866286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360e4ec0 0x55f8360e72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.174+0000 7fa4621abb80 1 --2- 172.21.15.44:0/2892866286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360dab70 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.174+0000 7fa4621abb80 1 --2- 172.21.15.44:0/2892866286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360db0b0 0x55f8360e4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.174+0000 7fa4621abb80 1 -- 172.21.15.44:0/2892866286 >> 172.21.15.44:0/2892866286 conn(0x55f83604f110 msgr2=0x55f8360d5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:36.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.174+0000 7fa4621abb80 1 -- 172.21.15.44:0/2892866286 shutdown_connections 2023-09-10T00:41:36.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.174+0000 7fa4621abb80 1 -- 172.21.15.44:0/2892866286 wait complete. 2023-09-10T00:41:36.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.175+0000 7fa4621abb80 1 -- start start 2023-09-10T00:41:36.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.175+0000 7fa4621abb80 1 Processor -- start 2023-09-10T00:41:36.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa4621abb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360e48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.186 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa4621abb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360db0b0 0x55f8360e1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.186 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa4621abb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360e4ec0 0x55f8360e2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.186 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa4621abb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f835fcabf0 con 0x55f8360da790 2023-09-10T00:41:36.186 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa4621abb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f835fca8f0 con 0x55f8360db0b0 2023-09-10T00:41:36.186 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa4621abb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f835eae9a0 con 0x55f8360e4ec0 2023-09-10T00:41:36.187 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa4621abb80 10 client.0 _ll_register_callbacks cb 0x55f835f14b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:36.187 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa44f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360e48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.187 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa44f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360e48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49858/0 (socket says 172.21.15.44:49858) 2023-09-10T00:41:36.187 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa44f7fe700 1 -- 172.21.15.44:0/3315985160 learned_addr learned my addr 172.21.15.44:0/3315985160 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:36.187 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa44ffff700 1 --2- 172.21.15.44:0/3315985160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360e4ec0 0x55f8360e2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.187 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa44effd700 1 --2- 172.21.15.44:0/3315985160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360db0b0 0x55f8360e1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa44f7fe700 1 -- 172.21.15.44:0/3315985160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360e4ec0 msgr2=0x55f8360e2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa44f7fe700 1 --2- 172.21.15.44:0/3315985160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360e4ec0 0x55f8360e2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa44f7fe700 1 -- 172.21.15.44:0/3315985160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360db0b0 msgr2=0x55f8360e1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa44f7fe700 1 --2- 172.21.15.44:0/3315985160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360db0b0 0x55f8360e1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa44f7fe700 1 -- 172.21.15.44:0/3315985160 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa440010a40 con 0x55f8360da790 2023-09-10T00:41:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa44f7fe700 1 --2- 172.21.15.44:0/3315985160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360e48f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa440010d60 tx=0x7fa4400050f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccca458eb4368b2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.176+0000 7fa42effd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:41:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.177+0000 7fa42effd700 1 -- 172.21.15.44:0/3315985160 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44002d070 con 0x55f8360da790 2023-09-10T00:41:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.177+0000 7fa42effd700 1 -- 172.21.15.44:0/3315985160 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa440017070 con 0x55f8360da790 2023-09-10T00:41:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.177+0000 7fa4621abb80 1 -- 172.21.15.44:0/3315985160 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f835f84570 con 0x55f8360da790 2023-09-10T00:41:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.177+0000 7fa42effd700 1 -- 172.21.15.44:0/3315985160 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa440005610 con 0x55f8360da790 2023-09-10T00:41:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.177+0000 7fa4621abb80 10 client.16245 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.177+0000 7fa4621abb80 1 -- 172.21.15.44:0/3315985160 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f835e41ce0 con 0x55f8360da790 2023-09-10T00:41:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.177+0000 7fa42effd700 1 -- 172.21.15.44:0/3315985160 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f835f84570 con 0x55f8360da790 2023-09-10T00:41:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.177+0000 7fa42effd700 1 -- 172.21.15.44:0/3315985160 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa4400039e0 con 0x55f8360da790 2023-09-10T00:41:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.177+0000 7fa4621abb80 10 client.16245 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.177+0000 7fa4621abb80 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.177+0000 7fa4621abb80 20 client.16245 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.177+0000 7fa4621abb80 1 --2- 172.21.15.44:0/3315985160 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f8360f4840 0x55f8360f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.177+0000 7fa4621abb80 4 client.16245 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.177+0000 7fa4621abb80 1 -- 172.21.15.44:0/3315985160 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f836041920 con 0x55f8360f4840 2023-09-10T00:41:36.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.178+0000 7fa44effd700 1 --2- 172.21.15.44:0/3315985160 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f8360f4840 0x55f8360f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.178+0000 7fa42effd700 10 client.16245 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:36.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.178+0000 7fa44effd700 1 --2- 172.21.15.44:0/3315985160 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f8360f4840 0x55f8360f6c60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=513d74fbeea21ec6 server_cookie=b86d35ddb224f68a in_seq=0 out_seq=0 2023-09-10T00:41:36.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.179+0000 7fa42effd700 1 -- 172.21.15.44:0/3315985160 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f836041920 con 0x55f8360f4840 2023-09-10T00:41:36.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.179+0000 7fa42effd700 10 client.16245 handle_command_reply: tid=0 2023-09-10T00:41:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.179+0000 7fa4621abb80 1 client.16245 shutdown 2023-09-10T00:41:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.179+0000 7fa4621abb80 1 -- 172.21.15.44:0/3315985160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 msgr2=0x55f8360e48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.179+0000 7fa4621abb80 1 --2- 172.21.15.44:0/3315985160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360e48f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa440010d60 tx=0x7fa4400050f0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.179+0000 7fa4621abb80 1 -- 172.21.15.44:0/3315985160 shutdown_connections 2023-09-10T00:41:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.179+0000 7fa4621abb80 1 --2- 172.21.15.44:0/3315985160 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f8360f4840 0x55f8360f6c60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.179+0000 7fa4621abb80 1 --2- 172.21.15.44:0/3315985160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360e4ec0 0x55f8360e2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.179+0000 7fa4621abb80 1 --2- 172.21.15.44:0/3315985160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360e48f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.179+0000 7fa42effd700 0 client.16245 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:36.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.179+0000 7fa4621abb80 1 --2- 172.21.15.44:0/3315985160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360db0b0 0x55f8360e1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.195 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.179+0000 7fa4621abb80 1 -- 172.21.15.44:0/3315985160 >> 172.21.15.44:0/3315985160 conn(0x55f83604f110 msgr2=0x55f8360d5860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:36.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.179+0000 7fa4621abb80 1 -- 172.21.15.44:0/3315985160 shutdown_connections 2023-09-10T00:41:36.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.179+0000 7fa4621abb80 1 -- 172.21.15.44:0/3315985160 wait complete. 2023-09-10T00:41:36.196 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.179+0000 7fa4621abb80 20 client.16245 trim_cache size 0 max 16384 2023-09-10T00:41:36.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.198+0000 7fa4621abb80 1 Processor -- start 2023-09-10T00:41:36.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.198+0000 7fa4621abb80 1 -- start start 2023-09-10T00:41:36.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa4621abb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360eba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa4621abb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360db0b0 0x55f8360e4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa4621abb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360e4ec0 0x55f8360dce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa4621abb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8360cfdf0 con 0x55f8360da790 2023-09-10T00:41:36.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa4621abb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8360ebfb0 con 0x55f8360db0b0 2023-09-10T00:41:36.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa4621abb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8360cd480 con 0x55f8360e4ec0 2023-09-10T00:41:36.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa44f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360eba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa44f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360eba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49866/0 (socket says 172.21.15.44:49866) 2023-09-10T00:41:36.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa44f7fe700 1 -- 172.21.15.44:0/2994984272 learned_addr learned my addr 172.21.15.44:0/2994984272 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:36.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa44ffff700 1 --2- 172.21.15.44:0/2994984272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360e4ec0 0x55f8360dce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa44effd700 1 --2- 172.21.15.44:0/2994984272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360db0b0 0x55f8360e4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa44f7fe700 1 -- 172.21.15.44:0/2994984272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360e4ec0 msgr2=0x55f8360dce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa44f7fe700 1 --2- 172.21.15.44:0/2994984272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360e4ec0 0x55f8360dce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa44f7fe700 1 -- 172.21.15.44:0/2994984272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360db0b0 msgr2=0x55f8360e4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa44f7fe700 1 --2- 172.21.15.44:0/2994984272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360db0b0 0x55f8360e4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa44f7fe700 1 -- 172.21.15.44:0/2994984272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f835e7f2d0 con 0x55f8360da790 2023-09-10T00:41:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa44f7fe700 1 --2- 172.21.15.44:0/2994984272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360eba70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa440010d30 tx=0x7fa4400071c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad67b74032e75e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.199+0000 7fa4367fc700 1 -- 172.21.15.44:0/2994984272 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44002d070 con 0x55f8360da790 2023-09-10T00:41:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.200+0000 7fa4367fc700 1 -- 172.21.15.44:0/2994984272 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa440003b10 con 0x55f8360da790 2023-09-10T00:41:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.200+0000 7fa4367fc700 1 -- 172.21.15.44:0/2994984272 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44001e9d0 con 0x55f8360da790 2023-09-10T00:41:36.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.200+0000 7fa4621abb80 1 -- 172.21.15.44:0/2994984272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 msgr2=0x55f8360eba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.200+0000 7fa4621abb80 1 --2- 172.21.15.44:0/2994984272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360eba70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa440010d30 tx=0x7fa4400071c0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.201+0000 7fa4621abb80 1 -- 172.21.15.44:0/2994984272 shutdown_connections 2023-09-10T00:41:36.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.201+0000 7fa4621abb80 1 --2- 172.21.15.44:0/2994984272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360e4ec0 0x55f8360dce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.201+0000 7fa4621abb80 1 --2- 172.21.15.44:0/2994984272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360da790 0x55f8360eba70 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.201+0000 7fa4621abb80 1 --2- 172.21.15.44:0/2994984272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360db0b0 0x55f8360e4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.201+0000 7fa4621abb80 1 -- 172.21.15.44:0/2994984272 >> 172.21.15.44:0/2994984272 conn(0x55f83604f110 msgr2=0x55f8360f01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:36.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.201+0000 7fa4621abb80 1 -- 172.21.15.44:0/2994984272 shutdown_connections 2023-09-10T00:41:36.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.201+0000 7fa4621abb80 1 -- 172.21.15.44:0/2994984272 wait complete. 2023-09-10T00:41:36.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.201+0000 7fa4621abb80 1 -- start start 2023-09-10T00:41:36.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.201+0000 7fa4621abb80 1 Processor -- start 2023-09-10T00:41:36.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.201+0000 7fa4621abb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360da790 0x55f8360ea220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa4621abb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360db0b0 0x55f8360ea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa4621abb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360e4ec0 0x55f8360eace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa4621abb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f836148830 con 0x55f8360db0b0 2023-09-10T00:41:36.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa4621abb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8361489a0 con 0x55f8360e4ec0 2023-09-10T00:41:36.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa4621abb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f836148b60 con 0x55f8360da790 2023-09-10T00:41:36.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa4621abb80 10 client.0 _ll_register_callbacks cb 0x7fa440010c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:36.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa44f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360da790 0x55f8360ea220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa44f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360da790 0x55f8360ea220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56754/0 (socket says 172.21.15.44:56754) 2023-09-10T00:41:36.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa44effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360db0b0 0x55f8360ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa44f7fe700 1 -- 172.21.15.44:0/3750264868 learned_addr learned my addr 172.21.15.44:0/3750264868 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa44ffff700 1 --2- 172.21.15.44:0/3750264868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360e4ec0 0x55f8360eace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa44f7fe700 1 -- 172.21.15.44:0/3750264868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360e4ec0 msgr2=0x55f8360eace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa44f7fe700 1 --2- 172.21.15.44:0/3750264868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360e4ec0 0x55f8360eace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa44f7fe700 1 -- 172.21.15.44:0/3750264868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360db0b0 msgr2=0x55f8360ea760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa44f7fe700 1 --2- 172.21.15.44:0/3750264868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360db0b0 0x55f8360ea760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa44f7fe700 1 -- 172.21.15.44:0/3750264868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f835e41ce0 con 0x55f8360da790 2023-09-10T00:41:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa44ffff700 1 --2- 172.21.15.44:0/3750264868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360e4ec0 0x55f8360eace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa44effd700 1 --2- 172.21.15.44:0/3750264868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360db0b0 0x55f8360ea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa44f7fe700 1 --2- 172.21.15.44:0/3750264868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360da790 0x55f8360ea220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa440021040 tx=0x7fa440010fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5686e0de083c151 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.202+0000 7fa42effd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:41:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.203+0000 7fa42effd700 1 -- 172.21.15.44:0/3750264868 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44002d070 con 0x55f8360da790 2023-09-10T00:41:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.203+0000 7fa4621abb80 1 -- 172.21.15.44:0/3750264868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa440015070 con 0x55f8360da790 2023-09-10T00:41:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.203+0000 7fa42effd700 1 -- 172.21.15.44:0/3750264868 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa440017070 con 0x55f8360da790 2023-09-10T00:41:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.203+0000 7fa42effd700 1 -- 172.21.15.44:0/3750264868 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa44001e730 con 0x55f8360da790 2023-09-10T00:41:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.203+0000 7fa4621abb80 10 client.25855 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:36.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.203+0000 7fa4621abb80 1 -- 172.21.15.44:0/3750264868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f835e57330 con 0x55f8360da790 2023-09-10T00:41:36.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.203+0000 7fa42effd700 1 -- 172.21.15.44:0/3750264868 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa440015070 con 0x55f8360da790 2023-09-10T00:41:36.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.203+0000 7fa42effd700 1 -- 172.21.15.44:0/3750264868 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa440032a70 con 0x55f8360da790 2023-09-10T00:41:36.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.203+0000 7fa4621abb80 10 client.25855 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:36.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.203+0000 7fa4621abb80 10 client.25855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.203+0000 7fa4621abb80 20 client.25855 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.203+0000 7fa4621abb80 1 --2- 172.21.15.44:0/3750264868 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f836149f10 0x55f83614c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.203+0000 7fa4621abb80 4 client.25855 mds_command: new command op to 24605 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-10T00:41:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.204+0000 7fa4621abb80 1 -- 172.21.15.44:0/3750264868 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f8360259a0 con 0x55f836149f10 2023-09-10T00:41:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.204+0000 7fa44effd700 1 --2- 172.21.15.44:0/3750264868 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f836149f10 0x55f83614c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.204+0000 7fa42effd700 10 client.25855 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:36.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.204+0000 7fa44effd700 1 --2- 172.21.15.44:0/3750264868 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f836149f10 0x55f83614c330 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef9dce300423f1cc server_cookie=84af122670e74646 in_seq=0 out_seq=0 2023-09-10T00:41:36.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.205+0000 7fa42effd700 1 -- 172.21.15.44:0/3750264868 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f8360259a0 con 0x55f836149f10 2023-09-10T00:41:36.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.205+0000 7fa42effd700 10 client.25855 handle_command_reply: tid=0 2023-09-10T00:41:36.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.205+0000 7fa4621abb80 1 client.25855 shutdown 2023-09-10T00:41:36.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.205+0000 7fa4621abb80 1 -- 172.21.15.44:0/3750264868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360da790 msgr2=0x55f8360ea220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.205+0000 7fa4621abb80 1 --2- 172.21.15.44:0/3750264868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360da790 0x55f8360ea220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa440021040 tx=0x7fa440010fb0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.205+0000 7fa4621abb80 1 -- 172.21.15.44:0/3750264868 shutdown_connections 2023-09-10T00:41:36.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.205+0000 7fa4621abb80 1 --2- 172.21.15.44:0/3750264868 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f836149f10 0x55f83614c330 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.205+0000 7fa4621abb80 1 --2- 172.21.15.44:0/3750264868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8360e4ec0 0x55f8360eace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.205+0000 7fa4621abb80 1 --2- 172.21.15.44:0/3750264868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8360da790 0x55f8360ea220 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.205+0000 7fa42effd700 0 client.25855 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:36.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.205+0000 7fa4621abb80 1 --2- 172.21.15.44:0/3750264868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8360db0b0 0x55f8360ea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.205+0000 7fa4621abb80 1 -- 172.21.15.44:0/3750264868 >> 172.21.15.44:0/3750264868 conn(0x55f83604f110 msgr2=0x55f8360dce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:36.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.206+0000 7fa4621abb80 1 -- 172.21.15.44:0/3750264868 shutdown_connections 2023-09-10T00:41:36.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.206+0000 7fa4621abb80 1 -- 172.21.15.44:0/3750264868 wait complete. 2023-09-10T00:41:36.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.206+0000 7fa4621abb80 20 client.25855 trim_cache size 0 max 16384 2023-09-10T00:41:36.221 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:41:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2023-09-10T00:41:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:41:36.222 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2023-09-10T00:41:36.222 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:41:36.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.206+0000 7fa455fe5700 1 -- 172.21.15.44:0/1863209539 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4380a0950 msgr2=0x7fa4380a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.206+0000 7fa455fe5700 1 --2- 172.21.15.44:0/1863209539 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4380a0950 0x7fa4380a2e10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa440007b60 tx=0x7fa4400073d0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.206+0000 7fa455fe5700 1 -- 172.21.15.44:0/1863209539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4500ffe70 msgr2=0x7fa450196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.206+0000 7fa455fe5700 1 --2- 172.21.15.44:0/1863209539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4500ffe70 0x7fa450196af0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa444006010 tx=0x7fa44400beb0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.206+0000 7fa455fe5700 1 -- 172.21.15.44:0/1863209539 shutdown_connections 2023-09-10T00:41:36.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.206+0000 7fa455fe5700 1 --2- 172.21.15.44:0/1863209539 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4380a0950 0x7fa4380a2e10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.206+0000 7fa455fe5700 1 --2- 172.21.15.44:0/1863209539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa450105570 0x7fa45019af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.206+0000 7fa455fe5700 1 --2- 172.21.15.44:0/1863209539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4500ff4b0 0x7fa4501965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.206+0000 7fa455fe5700 1 --2- 172.21.15.44:0/1863209539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4500ffe70 0x7fa450196af0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.206+0000 7fa455fe5700 1 -- 172.21.15.44:0/1863209539 >> 172.21.15.44:0/1863209539 conn(0x7fa4500fafa0 msgr2=0x7fa4500fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:36.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.206+0000 7fa455fe5700 1 -- 172.21.15.44:0/1863209539 shutdown_connections 2023-09-10T00:41:36.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.206+0000 7fa455fe5700 1 -- 172.21.15.44:0/1863209539 wait complete. 2023-09-10T00:41:36.228 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:41:36.369 INFO:teuthology.orchestra.run.smithi044.stderr:3200000 of 50000000 tuples (6%) done (elapsed 36.00 s, remaining 526.55 s) 2023-09-10T00:41:36.380 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.379+0000 7fde827f4700 1 -- 172.21.15.44:0/1893976388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7c100f50 msgr2=0x7fde7c10d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.380 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.379+0000 7fde827f4700 1 --2- 172.21.15.44:0/1893976388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7c100f50 0x7fde7c10d410 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fde7400b0d0 tx=0x7fde7400b3e0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.379+0000 7fde827f4700 1 -- 172.21.15.44:0/1893976388 shutdown_connections 2023-09-10T00:41:36.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.379+0000 7fde827f4700 1 --2- 172.21.15.44:0/1893976388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7c100f50 0x7fde7c10d410 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.379+0000 7fde827f4700 1 --2- 172.21.15.44:0/1893976388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde7c106650 0x7fde7c106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.379+0000 7fde827f4700 1 --2- 172.21.15.44:0/1893976388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde7c100590 0x7fde7c100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.379+0000 7fde827f4700 1 -- 172.21.15.44:0/1893976388 >> 172.21.15.44:0/1893976388 conn(0x7fde7c0fc060 msgr2=0x7fde7c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:36.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.379+0000 7fde827f4700 1 -- 172.21.15.44:0/1893976388 shutdown_connections 2023-09-10T00:41:36.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.379+0000 7fde827f4700 1 -- 172.21.15.44:0/1893976388 wait complete. 2023-09-10T00:41:36.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.380+0000 7fde827f4700 1 Processor -- start 2023-09-10T00:41:36.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.380+0000 7fde827f4700 1 -- start start 2023-09-10T00:41:36.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.380+0000 7fde827f4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde7c100590 0x7fde7c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.380+0000 7fde827f4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7c100f50 0x7fde7c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.380+0000 7fde827f4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde7c106650 0x7fde7c1a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.380+0000 7fde827f4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c073670 con 0x7fde7c100f50 2023-09-10T00:41:36.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.380+0000 7fde827f4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c1a4b60 con 0x7fde7c106650 2023-09-10T00:41:36.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.380+0000 7fde827f4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c1a4ca0 con 0x7fde7c100590 2023-09-10T00:41:36.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.381+0000 7fde7b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7c100f50 0x7fde7c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.381+0000 7fde7bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde7c100590 0x7fde7c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.381+0000 7fde7b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7c100f50 0x7fde7c072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49888/0 (socket says 172.21.15.44:49888) 2023-09-10T00:41:36.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.381+0000 7fde7b7fe700 1 -- 172.21.15.44:0/2804722640 learned_addr learned my addr 172.21.15.44:0/2804722640 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:36.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.381+0000 7fde80d91700 1 --2- 172.21.15.44:0/2804722640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde7c106650 0x7fde7c1a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.381+0000 7fde7bfff700 1 -- 172.21.15.44:0/2804722640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde7c106650 msgr2=0x7fde7c1a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.381+0000 7fde7bfff700 1 --2- 172.21.15.44:0/2804722640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde7c106650 0x7fde7c1a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.381+0000 7fde7bfff700 1 -- 172.21.15.44:0/2804722640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7c100f50 msgr2=0x7fde7c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.381+0000 7fde7bfff700 1 --2- 172.21.15.44:0/2804722640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7c100f50 0x7fde7c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.381+0000 7fde7bfff700 1 -- 172.21.15.44:0/2804722640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde74009d00 con 0x7fde7c100590 2023-09-10T00:41:36.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.381+0000 7fde7bfff700 1 --2- 172.21.15.44:0/2804722640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde7c100590 0x7fde7c072900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00c970 tx=0x7fde6c00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:36.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.381+0000 7fde797fa700 1 -- 172.21.15.44:0/2804722640 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c004440 con 0x7fde7c100590 2023-09-10T00:41:36.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.381+0000 7fde797fa700 1 -- 172.21.15.44:0/2804722640 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fde6c003b10 con 0x7fde7c100590 2023-09-10T00:41:36.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.381+0000 7fde797fa700 1 -- 172.21.15.44:0/2804722640 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c007aa0 con 0x7fde7c100590 2023-09-10T00:41:36.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.381+0000 7fde827f4700 1 -- 172.21.15.44:0/2804722640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7c1a4f00 con 0x7fde7c100590 2023-09-10T00:41:36.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.381+0000 7fde827f4700 1 -- 172.21.15.44:0/2804722640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7c1a5380 con 0x7fde7c100590 2023-09-10T00:41:36.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.384+0000 7fde797fa700 1 -- 172.21.15.44:0/2804722640 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7fde6c003c80 con 0x7fde7c100590 2023-09-10T00:41:36.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.385+0000 7fde797fa700 1 --2- 172.21.15.44:0/2804722640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fde640a0950 0x7fde640a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.385+0000 7fde7b7fe700 1 --2- 172.21.15.44:0/2804722640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fde640a0950 0x7fde640a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.385+0000 7fde797fa700 1 -- 172.21.15.44:0/2804722640 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fde6c0cc100 con 0x7fde7c100590 2023-09-10T00:41:36.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.385+0000 7fde7b7fe700 1 --2- 172.21.15.44:0/2804722640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fde640a0950 0x7fde640a2e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0740f0 tx=0x7fde70009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:36.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.385+0000 7fde8e9bab80 1 Processor -- start 2023-09-10T00:41:36.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.385+0000 7fde8e9bab80 1 -- start start 2023-09-10T00:41:36.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde8e9bab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c7710 0x56199b7c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde8e9bab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c8030 0x56199b7d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde8e9bab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7d1e40 0x56199b7d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde8e9bab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199b720730 con 0x56199b7c8030 2023-09-10T00:41:36.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde8e9bab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199b71e050 con 0x56199b7d1e40 2023-09-10T00:41:36.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde8e9bab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199b719940 con 0x56199b7c7710 2023-09-10T00:41:36.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde7bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c7710 0x56199b7c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde7b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c8030 0x56199b7d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde7bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c7710 0x56199b7c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56776/0 (socket says 172.21.15.44:56776) 2023-09-10T00:41:36.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde7b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c8030 0x56199b7d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49896/0 (socket says 172.21.15.44:49896) 2023-09-10T00:41:36.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde7bfff700 1 -- 172.21.15.44:0/2736775407 learned_addr learned my addr 172.21.15.44:0/2736775407 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:36.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde80d91700 1 --2- 172.21.15.44:0/2736775407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7d1e40 0x56199b7d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde7bfff700 1 -- 172.21.15.44:0/2736775407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7d1e40 msgr2=0x56199b7d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:36.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde7bfff700 1 --2- 172.21.15.44:0/2736775407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7d1e40 0x56199b7d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde7bfff700 1 -- 172.21.15.44:0/2736775407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c8030 msgr2=0x56199b7d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:36.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde7bfff700 1 --2- 172.21.15.44:0/2736775407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c8030 0x56199b7d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde7bfff700 1 -- 172.21.15.44:0/2736775407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56199b545de0 con 0x56199b7c7710 2023-09-10T00:41:36.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde7b7fe700 1 --2- 172.21.15.44:0/2736775407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c8030 0x56199b7d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:36.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.386+0000 7fde7bfff700 1 --2- 172.21.15.44:0/2736775407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c7710 0x56199b7c7af0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fde6c095a30 tx=0x7fde6c098920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5c550e66372d24e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:36.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.387+0000 7fde627fc700 1 -- 172.21.15.44:0/2736775407 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c098fc0 con 0x56199b7c7710 2023-09-10T00:41:36.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.387+0000 7fde627fc700 1 -- 172.21.15.44:0/2736775407 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fde6c09cdf0 con 0x56199b7c7710 2023-09-10T00:41:36.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.387+0000 7fde627fc700 1 -- 172.21.15.44:0/2736775407 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c099420 con 0x56199b7c7710 2023-09-10T00:41:36.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.387+0000 7fde8e9bab80 1 -- 172.21.15.44:0/2736775407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c7710 msgr2=0x56199b7c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.387+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/2736775407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c7710 0x56199b7c7af0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fde6c095a30 tx=0x7fde6c098920 comp rx=0 tx=0).stop 2023-09-10T00:41:36.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.387+0000 7fde8e9bab80 1 -- 172.21.15.44:0/2736775407 shutdown_connections 2023-09-10T00:41:36.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.387+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/2736775407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7d1e40 0x56199b7d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.387+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/2736775407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c7710 0x56199b7c7af0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.388+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/2736775407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c8030 0x56199b7d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.388+0000 7fde8e9bab80 1 -- 172.21.15.44:0/2736775407 >> 172.21.15.44:0/2736775407 conn(0x56199b73c090 msgr2=0x56199b7c2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:36.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.388+0000 7fde8e9bab80 1 -- 172.21.15.44:0/2736775407 shutdown_connections 2023-09-10T00:41:36.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.388+0000 7fde8e9bab80 1 -- 172.21.15.44:0/2736775407 wait complete. 2023-09-10T00:41:36.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.388+0000 7fde8e9bab80 1 -- start start 2023-09-10T00:41:36.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.388+0000 7fde8e9bab80 1 Processor -- start 2023-09-10T00:41:36.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.388+0000 7fde8e9bab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c7710 0x56199b7d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.388+0000 7fde8e9bab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 0x56199b7ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.388+0000 7fde8e9bab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7d1e40 0x56199b7cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.388+0000 7fde8e9bab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199b6b7b70 con 0x56199b7c7710 2023-09-10T00:41:36.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde8e9bab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199b6b7870 con 0x56199b7d1e40 2023-09-10T00:41:36.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde8e9bab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199b59b920 con 0x56199b7c8030 2023-09-10T00:41:36.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde8e9bab80 10 client.0 _ll_register_callbacks cb 0x56199b601a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:36.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde7bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c7710 0x56199b7d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde7bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c7710 0x56199b7d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49900/0 (socket says 172.21.15.44:49900) 2023-09-10T00:41:36.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde7bfff700 1 -- 172.21.15.44:0/1295370059 learned_addr learned my addr 172.21.15.44:0/1295370059 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:36.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde80d91700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7d1e40 0x56199b7cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde7b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 0x56199b7ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde7b7fe700 1 -- 172.21.15.44:0/1295370059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7d1e40 msgr2=0x56199b7cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:36.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde7b7fe700 1 --2- 172.21.15.44:0/1295370059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7d1e40 0x56199b7cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde7b7fe700 1 -- 172.21.15.44:0/1295370059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c7710 msgr2=0x56199b7d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:36.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde7b7fe700 1 --2- 172.21.15.44:0/1295370059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c7710 0x56199b7d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde7b7fe700 1 -- 172.21.15.44:0/1295370059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde70017070 con 0x56199b7c8030 2023-09-10T00:41:36.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde7bfff700 1 --2- 172.21.15.44:0/1295370059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c7710 0x56199b7d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:36.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde7b7fe700 1 --2- 172.21.15.44:0/1295370059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 0x56199b7ce9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x56199b7bd280 tx=0x7fde700047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b92f0e7a1c05aa0e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:36.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde5b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:41:36.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde5b7fe700 1 -- 172.21.15.44:0/1295370059 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70022070 con 0x56199b7c8030 2023-09-10T00:41:36.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde5b7fe700 1 -- 172.21.15.44:0/1295370059 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fde7001c9f0 con 0x56199b7c8030 2023-09-10T00:41:36.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.389+0000 7fde8e9bab80 1 -- 172.21.15.44:0/1295370059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56199b6714f0 con 0x56199b7c8030 2023-09-10T00:41:36.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.390+0000 7fde5b7fe700 1 -- 172.21.15.44:0/1295370059 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde7001cbb0 con 0x56199b7c8030 2023-09-10T00:41:36.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.390+0000 7fde8e9bab80 10 client.25867 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:36.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.390+0000 7fde8e9bab80 1 -- 172.21.15.44:0/1295370059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fde7c1a5380 con 0x56199b7c8030 2023-09-10T00:41:36.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.390+0000 7fde5b7fe700 1 -- 172.21.15.44:0/1295370059 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56199b6714f0 con 0x56199b7c8030 2023-09-10T00:41:36.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.390+0000 7fde5b7fe700 1 -- 172.21.15.44:0/1295370059 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fde7000e3b0 con 0x56199b7c8030 2023-09-10T00:41:36.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.390+0000 7fde8e9bab80 10 client.25867 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:36.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.390+0000 7fde8e9bab80 10 client.25867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:36.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.390+0000 7fde8e9bab80 20 client.25867 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:36.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.390+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/1295370059 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56199b7e1900 0x56199b7e3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.390+0000 7fde8e9bab80 4 client.25867 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:36.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.390+0000 7fde8e9bab80 1 -- 172.21.15.44:0/1295370059 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56199b72e8a0 con 0x56199b7e1900 2023-09-10T00:41:36.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.390+0000 7fde7bfff700 1 --2- 172.21.15.44:0/1295370059 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56199b7e1900 0x56199b7e3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.408 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.391+0000 7fde5b7fe700 10 client.25867 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:36.408 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.391+0000 7fde7bfff700 1 --2- 172.21.15.44:0/1295370059 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56199b7e1900 0x56199b7e3d20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97bee2958b2266e9 server_cookie=6f9dca3582178eb3 in_seq=0 out_seq=0 2023-09-10T00:41:36.408 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.391+0000 7fde5b7fe700 1 -- 172.21.15.44:0/1295370059 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56199b72e8a0 con 0x56199b7e1900 2023-09-10T00:41:36.408 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.391+0000 7fde5b7fe700 10 client.25867 handle_command_reply: tid=0 2023-09-10T00:41:36.408 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.392+0000 7fde8e9bab80 1 client.25867 shutdown 2023-09-10T00:41:36.409 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.392+0000 7fde8e9bab80 1 -- 172.21.15.44:0/1295370059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 msgr2=0x56199b7ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.409 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.392+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/1295370059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 0x56199b7ce9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x56199b7bd280 tx=0x7fde700047c0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.409 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.392+0000 7fde8e9bab80 1 -- 172.21.15.44:0/1295370059 shutdown_connections 2023-09-10T00:41:36.409 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.392+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/1295370059 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56199b7e1900 0x56199b7e3d20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.409 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.392+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/1295370059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7d1e40 0x56199b7cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.409 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.392+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/1295370059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c7710 0x56199b7d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.392+0000 7fde5b7fe700 0 client.25867 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:36.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.392+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/1295370059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 0x56199b7ce9a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.392+0000 7fde8e9bab80 1 -- 172.21.15.44:0/1295370059 >> 172.21.15.44:0/1295370059 conn(0x56199b73c090 msgr2=0x56199b7c2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:36.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.392+0000 7fde8e9bab80 1 -- 172.21.15.44:0/1295370059 shutdown_connections 2023-09-10T00:41:36.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.392+0000 7fde8e9bab80 1 -- 172.21.15.44:0/1295370059 wait complete. 2023-09-10T00:41:36.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.392+0000 7fde8e9bab80 20 client.25867 trim_cache size 0 max 16384 2023-09-10T00:41:36.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde8e9bab80 1 Processor -- start 2023-09-10T00:41:36.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde8e9bab80 1 -- start start 2023-09-10T00:41:36.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde8e9bab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7c7710 0x56199b7d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde8e9bab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 0x56199b7c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde8e9bab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7d1e40 0x56199b7ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde8e9bab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199b7dd850 con 0x56199b7d1e40 2023-09-10T00:41:36.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde8e9bab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199b7dd990 con 0x56199b7c7710 2023-09-10T00:41:36.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde8e9bab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199b7e3cf0 con 0x56199b7c8030 2023-09-10T00:41:36.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde80d91700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7d1e40 0x56199b7ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde7bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7c7710 0x56199b7d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde7bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7c7710 0x56199b7d12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51648/0 (socket says 172.21.15.44:51648) 2023-09-10T00:41:36.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde7b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 0x56199b7c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde7bfff700 1 -- 172.21.15.44:0/459603798 learned_addr learned my addr 172.21.15.44:0/459603798 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:36.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde80d91700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7d1e40 0x56199b7ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49910/0 (socket says 172.21.15.44:49910) 2023-09-10T00:41:36.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde7bfff700 1 -- 172.21.15.44:0/459603798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 msgr2=0x56199b7c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:36.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde7bfff700 1 --2- 172.21.15.44:0/459603798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 0x56199b7c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde7bfff700 1 -- 172.21.15.44:0/459603798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7d1e40 msgr2=0x56199b7ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:36.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde7bfff700 1 --2- 172.21.15.44:0/459603798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7d1e40 0x56199b7ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde7bfff700 1 -- 172.21.15.44:0/459603798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56199b52ec60 con 0x56199b7c7710 2023-09-10T00:41:36.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde80d91700 1 --2- 172.21.15.44:0/459603798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7d1e40 0x56199b7ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:36.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.417+0000 7fde7b7fe700 1 --2- 172.21.15.44:0/459603798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 0x56199b7c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:36.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.418+0000 7fde7bfff700 1 --2- 172.21.15.44:0/459603798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7c7710 0x56199b7d12c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fde6c098f50 tx=0x7fde6c0cb040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46e733684e246363 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:36.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.418+0000 7fde62ffd700 1 -- 172.21.15.44:0/459603798 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c099040 con 0x56199b7c7710 2023-09-10T00:41:36.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.418+0000 7fde62ffd700 1 -- 172.21.15.44:0/459603798 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fde6c077e90 con 0x56199b7c7710 2023-09-10T00:41:36.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.418+0000 7fde62ffd700 1 -- 172.21.15.44:0/459603798 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c099220 con 0x56199b7c7710 2023-09-10T00:41:36.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.419+0000 7fde8e9bab80 1 -- 172.21.15.44:0/459603798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7c7710 msgr2=0x56199b7d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.419+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/459603798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7c7710 0x56199b7d12c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fde6c098f50 tx=0x7fde6c0cb040 comp rx=0 tx=0).stop 2023-09-10T00:41:36.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.419+0000 7fde8e9bab80 1 -- 172.21.15.44:0/459603798 shutdown_connections 2023-09-10T00:41:36.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.419+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/459603798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7d1e40 0x56199b7ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.419+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/459603798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7c7710 0x56199b7d12c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.419+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/459603798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 0x56199b7c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.419+0000 7fde8e9bab80 1 -- 172.21.15.44:0/459603798 >> 172.21.15.44:0/459603798 conn(0x56199b73c090 msgr2=0x56199b7dd140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:36.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.419+0000 7fde8e9bab80 1 -- 172.21.15.44:0/459603798 shutdown_connections 2023-09-10T00:41:36.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.419+0000 7fde8e9bab80 1 -- 172.21.15.44:0/459603798 wait complete. 2023-09-10T00:41:36.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.419+0000 7fde8e9bab80 1 -- start start 2023-09-10T00:41:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.419+0000 7fde8e9bab80 1 Processor -- start 2023-09-10T00:41:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde8e9bab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c7710 0x56199b7d8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde8e9bab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 0x56199b7d6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde8e9bab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7d1e40 0x56199b7d7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde8e9bab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199b7d75f0 con 0x56199b7c7710 2023-09-10T00:41:36.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde8e9bab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199b7d7730 con 0x56199b7d1e40 2023-09-10T00:41:36.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde8e9bab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199b7d78c0 con 0x56199b7c8030 2023-09-10T00:41:36.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde8e9bab80 10 client.0 _ll_register_callbacks cb 0x7fde7001d7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:36.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde7bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c7710 0x56199b7d8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde7b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 0x56199b7d6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde7b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 0x56199b7d6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56798/0 (socket says 172.21.15.44:56798) 2023-09-10T00:41:36.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde7b7fe700 1 -- 172.21.15.44:0/3948420515 learned_addr learned my addr 172.21.15.44:0/3948420515 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:36.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde7bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c7710 0x56199b7d8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49914/0 (socket says 172.21.15.44:49914) 2023-09-10T00:41:36.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde80d91700 1 --2- 172.21.15.44:0/3948420515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7d1e40 0x56199b7d7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde7b7fe700 1 -- 172.21.15.44:0/3948420515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7d1e40 msgr2=0x56199b7d7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:36.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde7b7fe700 1 --2- 172.21.15.44:0/3948420515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7d1e40 0x56199b7d7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde7b7fe700 1 -- 172.21.15.44:0/3948420515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c7710 msgr2=0x56199b7d8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:36.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde7b7fe700 1 --2- 172.21.15.44:0/3948420515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c7710 0x56199b7d8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde7b7fe700 1 -- 172.21.15.44:0/3948420515 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7c1a5380 con 0x56199b7c8030 2023-09-10T00:41:36.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde7b7fe700 1 --2- 172.21.15.44:0/3948420515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 0x56199b7d6b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x56199b7cff50 tx=0x7fde7000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d8f60c7ea0e7df server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.420+0000 7fde5b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:41:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.421+0000 7fde5b7fe700 1 -- 172.21.15.44:0/3948420515 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70022070 con 0x56199b7c8030 2023-09-10T00:41:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.421+0000 7fde5b7fe700 1 -- 172.21.15.44:0/3948420515 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fde7000e930 con 0x56199b7c8030 2023-09-10T00:41:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.421+0000 7fde8e9bab80 1 -- 172.21.15.44:0/3948420515 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde6c09cdf0 con 0x56199b7c8030 2023-09-10T00:41:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.421+0000 7fde5b7fe700 1 -- 172.21.15.44:0/3948420515 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde7000eaf0 con 0x56199b7c8030 2023-09-10T00:41:36.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.421+0000 7fde8e9bab80 10 client.25873 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.421+0000 7fde8e9bab80 1 -- 172.21.15.44:0/3948420515 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fde7c10dd10 con 0x56199b7c8030 2023-09-10T00:41:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.421+0000 7fde5b7fe700 1 -- 172.21.15.44:0/3948420515 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fde6c09cdf0 con 0x56199b7c8030 2023-09-10T00:41:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.421+0000 7fde5b7fe700 1 -- 172.21.15.44:0/3948420515 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fde7003f060 con 0x56199b7c8030 2023-09-10T00:41:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.421+0000 7fde8e9bab80 10 client.25873 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:36.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.421+0000 7fde8e9bab80 10 client.25873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:36.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.421+0000 7fde8e9bab80 20 client.25873 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:36.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.421+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/3948420515 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56199b836cd0 0x56199b8390f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:36.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.421+0000 7fde8e9bab80 4 client.25873 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:41:36.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.421+0000 7fde8e9bab80 1 -- 172.21.15.44:0/3948420515 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56199b712920 con 0x56199b836cd0 2023-09-10T00:41:36.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.421+0000 7fde7bfff700 1 --2- 172.21.15.44:0/3948420515 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56199b836cd0 0x56199b8390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:36.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.422+0000 7fde5b7fe700 10 client.25873 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:36.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.422+0000 7fde7bfff700 1 --2- 172.21.15.44:0/3948420515 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56199b836cd0 0x56199b8390f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e26ace0f1a54f7b8 server_cookie=c2fc8b6c985c2b5c in_seq=0 out_seq=0 2023-09-10T00:41:36.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.422+0000 7fde5b7fe700 1 -- 172.21.15.44:0/3948420515 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56199b712920 con 0x56199b836cd0 2023-09-10T00:41:36.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.422+0000 7fde5b7fe700 10 client.25873 handle_command_reply: tid=0 2023-09-10T00:41:36.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.422+0000 7fde8e9bab80 1 client.25873 shutdown 2023-09-10T00:41:36.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.422+0000 7fde8e9bab80 1 -- 172.21.15.44:0/3948420515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 msgr2=0x56199b7d6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.422+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/3948420515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 0x56199b7d6b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x56199b7cff50 tx=0x7fde7000ef90 comp rx=0 tx=0).stop 2023-09-10T00:41:36.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.422+0000 7fde8e9bab80 1 -- 172.21.15.44:0/3948420515 shutdown_connections 2023-09-10T00:41:36.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.422+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/3948420515 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56199b836cd0 0x56199b8390f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.422+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/3948420515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56199b7d1e40 0x56199b7d7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.422+0000 7fde5b7fe700 0 client.25873 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:36.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.422+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/3948420515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56199b7c7710 0x56199b7d8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.422+0000 7fde8e9bab80 1 --2- 172.21.15.44:0/3948420515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56199b7c8030 0x56199b7d6b30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.422+0000 7fde8e9bab80 1 -- 172.21.15.44:0/3948420515 >> 172.21.15.44:0/3948420515 conn(0x56199b73c090 msgr2=0x56199b7c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:36.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.422+0000 7fde8e9bab80 1 -- 172.21.15.44:0/3948420515 shutdown_connections 2023-09-10T00:41:36.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.422+0000 7fde8e9bab80 1 -- 172.21.15.44:0/3948420515 wait complete. 2023-09-10T00:41:36.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.423+0000 7fde8e9bab80 20 client.25873 trim_cache size 0 max 16384 2023-09-10T00:41:36.439 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:41:36.439 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "scrub active (3 inodes in the stack)", 2023-09-10T00:41:36.439 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:41:36.440 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:41:36.440 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:41:36.440 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:41:36.440 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:41:36.440 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:41:36.441 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:41:36.441 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:41:36.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.423+0000 7fde827f4700 1 -- 172.21.15.44:0/2804722640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fde640a0950 msgr2=0x7fde640a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.423+0000 7fde827f4700 1 --2- 172.21.15.44:0/2804722640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fde640a0950 0x7fde640a2e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0740f0 tx=0x7fde70009260 comp rx=0 tx=0).stop 2023-09-10T00:41:36.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.423+0000 7fde827f4700 1 -- 172.21.15.44:0/2804722640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde7c100590 msgr2=0x7fde7c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:36.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.423+0000 7fde827f4700 1 --2- 172.21.15.44:0/2804722640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde7c100590 0x7fde7c072900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00c970 tx=0x7fde6c00cc80 comp rx=0 tx=0).stop 2023-09-10T00:41:36.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.423+0000 7fde827f4700 1 -- 172.21.15.44:0/2804722640 shutdown_connections 2023-09-10T00:41:36.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.423+0000 7fde827f4700 1 --2- 172.21.15.44:0/2804722640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fde640a0950 0x7fde640a2e10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.423+0000 7fde827f4700 1 --2- 172.21.15.44:0/2804722640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde7c106650 0x7fde7c1a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.423+0000 7fde827f4700 1 --2- 172.21.15.44:0/2804722640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde7c100590 0x7fde7c072900 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.423+0000 7fde827f4700 1 --2- 172.21.15.44:0/2804722640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7c100f50 0x7fde7c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:36.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.423+0000 7fde827f4700 1 -- 172.21.15.44:0/2804722640 >> 172.21.15.44:0/2804722640 conn(0x7fde7c0fc060 msgr2=0x7fde7c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:36.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.423+0000 7fde827f4700 1 -- 172.21.15.44:0/2804722640 shutdown_connections 2023-09-10T00:41:36.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:41:36.423+0000 7fde827f4700 1 -- 172.21.15.44:0/2804722640 wait complete. 2023-09-10T00:41:36.445 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:41:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:36 smithi044 ceph-mon[98676]: pgmap v31: 129 pgs: 129 active+clean; 728 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 46 MiB/s wr, 53 op/s 2023-09-10T00:41:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:36 smithi044 ceph-mon[98676]: scrub queued for path: / 2023-09-10T00:41:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:36 smithi044 ceph-mon[98676]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:36 smithi044 ceph-mon[98676]: scrub summary: active paths [/] 2023-09-10T00:41:37.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:36 smithi044 ceph-mon[98676]: scrub summary: active paths [/] 2023-09-10T00:41:37.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:36 smithi044 ceph-mon[98676]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:37.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:36 smithi044 ceph-mon[98676]: scrub summary: active paths [/] 2023-09-10T00:41:37.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:36 smithi044 ceph-mon[98676]: scrub summary: active paths [/] 2023-09-10T00:41:37.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:36 smithi044 ceph-mon[98676]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:37.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:36 smithi044 ceph-mon[98676]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:36 smithi099 ceph-mon[102692]: pgmap v31: 129 pgs: 129 active+clean; 728 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 46 MiB/s wr, 53 op/s 2023-09-10T00:41:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:36 smithi099 ceph-mon[102692]: scrub queued for path: / 2023-09-10T00:41:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:36 smithi099 ceph-mon[102692]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:37.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:36 smithi099 ceph-mon[102692]: scrub summary: active paths [/] 2023-09-10T00:41:37.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:36 smithi099 ceph-mon[102692]: scrub summary: active paths [/] 2023-09-10T00:41:37.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:36 smithi099 ceph-mon[102692]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:37.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:36 smithi099 ceph-mon[102692]: scrub summary: active paths [/] 2023-09-10T00:41:37.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:36 smithi099 ceph-mon[102692]: scrub summary: active paths [/] 2023-09-10T00:41:37.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:36 smithi099 ceph-mon[102692]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:37.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:36 smithi099 ceph-mon[102692]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:37.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:36 smithi131 ceph-mon[103713]: pgmap v31: 129 pgs: 129 active+clean; 728 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 46 MiB/s wr, 53 op/s 2023-09-10T00:41:37.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:36 smithi131 ceph-mon[103713]: scrub queued for path: / 2023-09-10T00:41:37.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:36 smithi131 ceph-mon[103713]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:37.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:36 smithi131 ceph-mon[103713]: scrub summary: active paths [/] 2023-09-10T00:41:37.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:36 smithi131 ceph-mon[103713]: scrub summary: active paths [/] 2023-09-10T00:41:37.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:36 smithi131 ceph-mon[103713]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:37.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:36 smithi131 ceph-mon[103713]: scrub summary: active paths [/] 2023-09-10T00:41:37.299 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:36 smithi131 ceph-mon[103713]: scrub summary: active paths [/] 2023-09-10T00:41:37.299 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:36 smithi131 ceph-mon[103713]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:37.299 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:36 smithi131 ceph-mon[103713]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:37.609 INFO:teuthology.orchestra.run.smithi044.stderr:3300000 of 50000000 tuples (6%) done (elapsed 37.24 s, remaining 527.05 s) 2023-09-10T00:41:38.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:37 smithi099 ceph-mon[102692]: scrub summary: active paths [/] 2023-09-10T00:41:38.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:37 smithi099 ceph-mon[102692]: scrub summary: active paths [/] 2023-09-10T00:41:38.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:37 smithi099 ceph-mon[102692]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:38.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:37 smithi099 ceph-mon[102692]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:38.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:37 smithi131 ceph-mon[103713]: scrub summary: active paths [/] 2023-09-10T00:41:38.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:37 smithi131 ceph-mon[103713]: scrub summary: active paths [/] 2023-09-10T00:41:38.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:37 smithi131 ceph-mon[103713]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:38.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:37 smithi131 ceph-mon[103713]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:38.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:37 smithi044 ceph-mon[98676]: scrub summary: active paths [/] 2023-09-10T00:41:38.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:37 smithi044 ceph-mon[98676]: scrub summary: active paths [/] 2023-09-10T00:41:38.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:37 smithi044 ceph-mon[98676]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:38.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:37 smithi044 ceph-mon[98676]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:38.525 INFO:teuthology.orchestra.run.smithi044.stderr:3400000 of 50000000 tuples (6%) done (elapsed 38.16 s, remaining 523.00 s) 2023-09-10T00:41:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:38 smithi099 ceph-mon[102692]: pgmap v32: 129 pgs: 129 active+clean; 752 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 35 MiB/s wr, 36 op/s 2023-09-10T00:41:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:38 smithi131 ceph-mon[103713]: pgmap v32: 129 pgs: 129 active+clean; 752 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 35 MiB/s wr, 36 op/s 2023-09-10T00:41:39.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:38 smithi044 ceph-mon[98676]: pgmap v32: 129 pgs: 129 active+clean; 752 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 35 MiB/s wr, 36 op/s 2023-09-10T00:41:39.808 INFO:teuthology.orchestra.run.smithi044.stderr:3500000 of 50000000 tuples (7%) done (elapsed 39.44 s, remaining 524.03 s) 2023-09-10T00:41:41.059 INFO:teuthology.orchestra.run.smithi044.stderr:3600000 of 50000000 tuples (7%) done (elapsed 40.69 s, remaining 524.49 s) 2023-09-10T00:41:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:40 smithi099 ceph-mon[102692]: pgmap v33: 129 pgs: 129 active+clean; 752 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 34 MiB/s wr, 34 op/s 2023-09-10T00:41:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:40 smithi131 ceph-mon[103713]: pgmap v33: 129 pgs: 129 active+clean; 752 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 34 MiB/s wr, 34 op/s 2023-09-10T00:41:41.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:40 smithi044 ceph-mon[98676]: pgmap v33: 129 pgs: 129 active+clean; 752 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 34 MiB/s wr, 34 op/s 2023-09-10T00:41:41.971 INFO:teuthology.orchestra.run.smithi044.stderr:3700000 of 50000000 tuples (7%) done (elapsed 41.61 s, remaining 520.63 s) 2023-09-10T00:41:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:42 smithi099 ceph-mon[102692]: pgmap v34: 129 pgs: 129 active+clean; 856 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 50 MiB/s wr, 105 op/s 2023-09-10T00:41:43.234 INFO:teuthology.orchestra.run.smithi044.stderr:3800000 of 50000000 tuples (7%) done (elapsed 42.87 s, remaining 521.19 s) 2023-09-10T00:41:43.294 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:42 smithi131 ceph-mon[103713]: pgmap v34: 129 pgs: 129 active+clean; 856 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 50 MiB/s wr, 105 op/s 2023-09-10T00:41:43.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:42 smithi044 ceph-mon[98676]: pgmap v34: 129 pgs: 129 active+clean; 856 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 50 MiB/s wr, 105 op/s 2023-09-10T00:41:44.469 INFO:teuthology.orchestra.run.smithi044.stderr:3900000 of 50000000 tuples (7%) done (elapsed 44.10 s, remaining 521.31 s) 2023-09-10T00:41:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:44 smithi099 ceph-mon[102692]: pgmap v35: 129 pgs: 129 active+clean; 856 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 33 MiB/s wr, 87 op/s 2023-09-10T00:41:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:44 smithi099 ceph-mon[102692]: scrub summary: active paths [/] 2023-09-10T00:41:45.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:44 smithi099 ceph-mon[102692]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:44 smithi131 ceph-mon[103713]: pgmap v35: 129 pgs: 129 active+clean; 856 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 33 MiB/s wr, 87 op/s 2023-09-10T00:41:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:44 smithi131 ceph-mon[103713]: scrub summary: active paths [/] 2023-09-10T00:41:45.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:44 smithi131 ceph-mon[103713]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:44 smithi044 ceph-mon[98676]: pgmap v35: 129 pgs: 129 active+clean; 856 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 33 MiB/s wr, 87 op/s 2023-09-10T00:41:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:44 smithi044 ceph-mon[98676]: scrub summary: active paths [/] 2023-09-10T00:41:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:44 smithi044 ceph-mon[98676]: scrub summary: idle+waiting paths [/] 2023-09-10T00:41:45.422 INFO:teuthology.orchestra.run.smithi044.stderr:4000000 of 50000000 tuples (8%) done (elapsed 45.06 s, remaining 518.15 s) 2023-09-10T00:41:46.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:45 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:41:46.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:45 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:41:46.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:45 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:41:46.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:45 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:41:46.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:45 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:41:46.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:45 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:41:46.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:45 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:41:46.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:45 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:41:46.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:45 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:41:46.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:45 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:41:46.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:45 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:41:46.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:45 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:41:46.725 INFO:teuthology.orchestra.run.smithi044.stderr:4100000 of 50000000 tuples (8%) done (elapsed 46.36 s, remaining 519.00 s) 2023-09-10T00:41:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:46 smithi099 ceph-mon[102692]: pgmap v36: 129 pgs: 129 active+clean; 945 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 48 MiB/s wr, 114 op/s 2023-09-10T00:41:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:46 smithi131 ceph-mon[103713]: pgmap v36: 129 pgs: 129 active+clean; 945 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 48 MiB/s wr, 114 op/s 2023-09-10T00:41:47.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:46 smithi044 ceph-mon[98676]: pgmap v36: 129 pgs: 129 active+clean; 945 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 48 MiB/s wr, 114 op/s 2023-09-10T00:41:47.618 INFO:teuthology.orchestra.run.smithi044.stderr:4200000 of 50000000 tuples (8%) done (elapsed 47.25 s, remaining 515.27 s) 2023-09-10T00:41:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.898+0000 7fcdd5d72700 1 -- 172.21.15.44:0/4290031393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdd0102100 msgr2=0x7fcdd010a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.898+0000 7fcdd5d72700 1 --2- 172.21.15.44:0/4290031393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdd0102100 0x7fcdd010a5e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009990 tx=0x7fcdc4009ca0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.898+0000 7fcdd5d72700 1 -- 172.21.15.44:0/4290031393 shutdown_connections 2023-09-10T00:41:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.898+0000 7fcdd5d72700 1 --2- 172.21.15.44:0/4290031393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdd010abb0 0x7fcdd010d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.898+0000 7fcdd5d72700 1 --2- 172.21.15.44:0/4290031393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdd01017e0 0x7fcdd0101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.898+0000 7fcdd5d72700 1 --2- 172.21.15.44:0/4290031393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdd0102100 0x7fcdd010a5e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.898+0000 7fcdd5d72700 1 -- 172.21.15.44:0/4290031393 >> 172.21.15.44:0/4290031393 conn(0x7fcdd00fafc0 msgr2=0x7fcdd00fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.898+0000 7fcdd5d72700 1 -- 172.21.15.44:0/4290031393 shutdown_connections 2023-09-10T00:41:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.898+0000 7fcdd5d72700 1 -- 172.21.15.44:0/4290031393 wait complete. 2023-09-10T00:41:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.899+0000 7fcdd5d72700 1 Processor -- start 2023-09-10T00:41:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.899+0000 7fcdd5d72700 1 -- start start 2023-09-10T00:41:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.899+0000 7fcdd5d72700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdd01017e0 0x7fcdd019c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.899+0000 7fcdd5d72700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdd0102100 0x7fcdd019cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.899+0000 7fcdd5d72700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdd010abb0 0x7fcdd0196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.899+0000 7fcdd5d72700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd0196aa0 con 0x7fcdd0102100 2023-09-10T00:41:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.899+0000 7fcdd5d72700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd0196c10 con 0x7fcdd010abb0 2023-09-10T00:41:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.899+0000 7fcdd5d72700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd0196dd0 con 0x7fcdd01017e0 2023-09-10T00:41:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.899+0000 7fcdceffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdd0102100 0x7fcdd019cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.899+0000 7fcdcf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdd01017e0 0x7fcdd019c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.899+0000 7fcdcf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdd01017e0 0x7fcdd019c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56810/0 (socket says 172.21.15.44:56810) 2023-09-10T00:41:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.899+0000 7fcdceffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdd0102100 0x7fcdd019cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49930/0 (socket says 172.21.15.44:49930) 2023-09-10T00:41:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.899+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/4058163497 learned_addr learned my addr 172.21.15.44:0/4058163497 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.899+0000 7fcdcffff700 1 --2- 172.21.15.44:0/4058163497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdd010abb0 0x7fcdd0196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.900+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/4058163497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdd010abb0 msgr2=0x7fcdd0196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.900+0000 7fcdcf7fe700 1 --2- 172.21.15.44:0/4058163497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdd010abb0 0x7fcdd0196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.900+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/4058163497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdd0102100 msgr2=0x7fcdd019cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.900+0000 7fcdcf7fe700 1 --2- 172.21.15.44:0/4058163497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdd0102100 0x7fcdd019cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.900+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/4058163497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc8009c40 con 0x7fcdd01017e0 2023-09-10T00:41:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.900+0000 7fcdceffd700 1 --2- 172.21.15.44:0/4058163497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdd0102100 0x7fcdd019cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.900+0000 7fcdcf7fe700 1 --2- 172.21.15.44:0/4058163497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdd01017e0 0x7fcdd019c5d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000e9a0 tx=0x7fcdc000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.900+0000 7fcdccff9700 1 -- 172.21.15.44:0/4058163497 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0009790 con 0x7fcdd01017e0 2023-09-10T00:41:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.900+0000 7fcdccff9700 1 -- 172.21.15.44:0/4058163497 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcdc000ce50 con 0x7fcdd01017e0 2023-09-10T00:41:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.900+0000 7fcdd5d72700 1 -- 172.21.15.44:0/4058163497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc40095f0 con 0x7fcdd01017e0 2023-09-10T00:41:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.900+0000 7fcdccff9700 1 -- 172.21.15.44:0/4058163497 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0018420 con 0x7fcdd01017e0 2023-09-10T00:41:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.900+0000 7fcdd5d72700 1 -- 172.21.15.44:0/4058163497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd01a33e0 con 0x7fcdd01017e0 2023-09-10T00:41:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.903+0000 7fcdccff9700 1 -- 172.21.15.44:0/4058163497 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7fcdc0010430 con 0x7fcdd01017e0 2023-09-10T00:41:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.904+0000 7fcdccff9700 1 --2- 172.21.15.44:0/4058163497 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcdb80a08d0 0x7fcdb80a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.904+0000 7fcdceffd700 1 --2- 172.21.15.44:0/4058163497 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcdb80a08d0 0x7fcdb80a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.904+0000 7fcdccff9700 1 -- 172.21.15.44:0/4058163497 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fcdc0014070 con 0x7fcdd01017e0 2023-09-10T00:41:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.904+0000 7fcde1f38b80 1 Processor -- start 2023-09-10T00:41:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.904+0000 7fcde1f38b80 1 -- start start 2023-09-10T00:41:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcde1f38b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e71a0 0x555f926e7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcde1f38b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1300 0x555f926e1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcde1f38b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e1cc0 0x555f926edf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcde1f38b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f9263a6d0 con 0x555f926e1cc0 2023-09-10T00:41:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcde1f38b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f92637ff0 con 0x555f926e71a0 2023-09-10T00:41:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcde1f38b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f926338e0 con 0x555f926e1300 2023-09-10T00:41:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcdceffd700 1 --2- 172.21.15.44:0/4058163497 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcdb80a08d0 0x7fcdb80a2d90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4006010 tx=0x7fcdc4005960 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcdcf7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e71a0 0x555f926e7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcdcf7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e71a0 0x555f926e7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51676/0 (socket says 172.21.15.44:51676) 2023-09-10T00:41:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/1440596690 learned_addr learned my addr 172.21.15.44:0/1440596690 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcdcffff700 1 --2- 172.21.15.44:0/1440596690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e1cc0 0x555f926edf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcdceffd700 1 --2- 172.21.15.44:0/1440596690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1300 0x555f926e1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/1440596690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1300 msgr2=0x555f926e1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcdcf7fe700 1 --2- 172.21.15.44:0/1440596690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1300 0x555f926e1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/1440596690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e1cc0 msgr2=0x555f926edf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcdcf7fe700 1 --2- 172.21.15.44:0/1440596690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e1cc0 0x555f926edf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/1440596690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f9245fd80 con 0x555f926e71a0 2023-09-10T00:41:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcdcffff700 1 --2- 172.21.15.44:0/1440596690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e1cc0 0x555f926edf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.905+0000 7fcdcf7fe700 1 --2- 172.21.15.44:0/1440596690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e71a0 0x555f926e7580 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00ca120 tx=0x7fcdc0099090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72aecbc898d03f56 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.906+0000 7fcdb5ffb700 1 -- 172.21.15.44:0/1440596690 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0099630 con 0x555f926e71a0 2023-09-10T00:41:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.906+0000 7fcdb5ffb700 1 -- 172.21.15.44:0/1440596690 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcdc009d460 con 0x555f926e71a0 2023-09-10T00:41:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.906+0000 7fcdb5ffb700 1 -- 172.21.15.44:0/1440596690 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc00cc390 con 0x555f926e71a0 2023-09-10T00:41:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.906+0000 7fcde1f38b80 1 -- 172.21.15.44:0/1440596690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e71a0 msgr2=0x555f926e7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.906+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/1440596690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e71a0 0x555f926e7580 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00ca120 tx=0x7fcdc0099090 comp rx=0 tx=0).stop 2023-09-10T00:41:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.906+0000 7fcde1f38b80 1 -- 172.21.15.44:0/1440596690 shutdown_connections 2023-09-10T00:41:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.906+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/1440596690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e1cc0 0x555f926edf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.906+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/1440596690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e71a0 0x555f926e7580 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.906+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/1440596690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1300 0x555f926e1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.906+0000 7fcde1f38b80 1 -- 172.21.15.44:0/1440596690 >> 172.21.15.44:0/1440596690 conn(0x555f92656030 msgr2=0x555f926dc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.906+0000 7fcde1f38b80 1 -- 172.21.15.44:0/1440596690 shutdown_connections 2023-09-10T00:41:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.906+0000 7fcde1f38b80 1 -- 172.21.15.44:0/1440596690 wait complete. 2023-09-10T00:41:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.907+0000 7fcde1f38b80 1 -- start start 2023-09-10T00:41:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.907+0000 7fcde1f38b80 1 Processor -- start 2023-09-10T00:41:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.907+0000 7fcde1f38b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1300 0x555f926edd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.907+0000 7fcde1f38b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e1cc0 0x555f926eb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.907+0000 7fcde1f38b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e71a0 0x555f926eb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.907+0000 7fcde1f38b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f925d1b10 con 0x555f926e71a0 2023-09-10T00:41:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.907+0000 7fcde1f38b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f925d1810 con 0x555f926e1cc0 2023-09-10T00:41:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.907+0000 7fcde1f38b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f924b58c0 con 0x555f926e1300 2023-09-10T00:41:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.907+0000 7fcde1f38b80 10 client.0 _ll_register_callbacks cb 0x555f9251ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.907+0000 7fcdcf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1300 0x555f926edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.907+0000 7fcdcffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e71a0 0x555f926eb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcdcf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1300 0x555f926edd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56824/0 (socket says 172.21.15.44:56824) 2023-09-10T00:41:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/153752820 learned_addr learned my addr 172.21.15.44:0/153752820 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcdceffd700 1 --2- 172.21.15.44:0/153752820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e1cc0 0x555f926eb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/153752820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e1cc0 msgr2=0x555f926eb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcdcf7fe700 1 --2- 172.21.15.44:0/153752820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e1cc0 0x555f926eb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/153752820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e71a0 msgr2=0x555f926eb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcdcf7fe700 1 --2- 172.21.15.44:0/153752820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e71a0 0x555f926eb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/153752820 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc800e330 con 0x555f926e1300 2023-09-10T00:41:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcdceffd700 1 --2- 172.21.15.44:0/153752820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e1cc0 0x555f926eb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcdcffff700 1 --2- 172.21.15.44:0/153752820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e71a0 0x555f926eb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcdcf7fe700 1 --2- 172.21.15.44:0/153752820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1300 0x555f926edd80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00a1170 tx=0x7fcdc009d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f8755edde39dc7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcdaeffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:41:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcdaeffd700 1 -- 172.21.15.44:0/153752820 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc00a16a0 con 0x555f926e1300 2023-09-10T00:41:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcdaeffd700 1 -- 172.21.15.44:0/153752820 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fcdc00842d0 con 0x555f926e1300 2023-09-10T00:41:47.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcde1f38b80 1 -- 172.21.15.44:0/153752820 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f9258b490 con 0x555f926e1300 2023-09-10T00:41:47.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcdaeffd700 1 -- 172.21.15.44:0/153752820 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdc0099630 con 0x555f926e1300 2023-09-10T00:41:47.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcde1f38b80 10 client.25885 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:47.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.908+0000 7fcde1f38b80 1 -- 172.21.15.44:0/153752820 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fcdd01a33e0 con 0x555f926e1300 2023-09-10T00:41:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.909+0000 7fcdaeffd700 1 -- 172.21.15.44:0/153752820 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555f9258b490 con 0x555f926e1300 2023-09-10T00:41:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.909+0000 7fcdaeffd700 1 -- 172.21.15.44:0/153752820 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fcdc0098bb0 con 0x555f926e1300 2023-09-10T00:41:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.909+0000 7fcde1f38b80 10 client.25885 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.909+0000 7fcde1f38b80 10 client.25885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.909+0000 7fcde1f38b80 20 client.25885 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.909+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/153752820 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f926fb030 0x555f926fd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.909+0000 7fcde1f38b80 4 client.25885 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:41:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.909+0000 7fcde1f38b80 1 -- 172.21.15.44:0/153752820 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f92648840 con 0x555f926fb030 2023-09-10T00:41:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.909+0000 7fcdceffd700 1 --2- 172.21.15.44:0/153752820 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f926fb030 0x555f926fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.910+0000 7fcdaeffd700 10 client.25885 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.910+0000 7fcdceffd700 1 --2- 172.21.15.44:0/153752820 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f926fb030 0x555f926fd450 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e3303fb75c98be3 server_cookie=73a2a28da633245 in_seq=0 out_seq=0 2023-09-10T00:41:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.910+0000 7fcdaeffd700 1 -- 172.21.15.44:0/153752820 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555f92648840 con 0x555f926fb030 2023-09-10T00:41:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.910+0000 7fcdaeffd700 10 client.25885 handle_command_reply: tid=0 2023-09-10T00:41:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.910+0000 7fcde1f38b80 1 client.25885 shutdown 2023-09-10T00:41:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.910+0000 7fcde1f38b80 1 -- 172.21.15.44:0/153752820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1300 msgr2=0x555f926edd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.910+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/153752820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1300 0x555f926edd80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00a1170 tx=0x7fcdc009d330 comp rx=0 tx=0).stop 2023-09-10T00:41:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.910+0000 7fcde1f38b80 1 -- 172.21.15.44:0/153752820 shutdown_connections 2023-09-10T00:41:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.911+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/153752820 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f926fb030 0x555f926fd450 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.911+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/153752820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e71a0 0x555f926eb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.911+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/153752820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1300 0x555f926edd80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.911+0000 7fcdaeffd700 0 client.25885 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.911+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/153752820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e1cc0 0x555f926eb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.911+0000 7fcde1f38b80 1 -- 172.21.15.44:0/153752820 >> 172.21.15.44:0/153752820 conn(0x555f92656030 msgr2=0x555f926db7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.911+0000 7fcde1f38b80 1 -- 172.21.15.44:0/153752820 shutdown_connections 2023-09-10T00:41:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.911+0000 7fcde1f38b80 1 -- 172.21.15.44:0/153752820 wait complete. 2023-09-10T00:41:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.911+0000 7fcde1f38b80 20 client.25885 trim_cache size 0 max 16384 2023-09-10T00:41:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.934+0000 7fcde1f38b80 1 Processor -- start 2023-09-10T00:41:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.934+0000 7fcde1f38b80 1 -- start start 2023-09-10T00:41:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcde1f38b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1cc0 0x555f926f2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcde1f38b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e71a0 0x555f926ed920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcde1f38b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926ede60 0x555f926dbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcde1f38b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f926ec620 con 0x555f926e71a0 2023-09-10T00:41:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcde1f38b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f926ddd10 con 0x555f926ede60 2023-09-10T00:41:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcde1f38b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f926e2180 con 0x555f926e1cc0 2023-09-10T00:41:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcdceffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e71a0 0x555f926ed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcdcf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1cc0 0x555f926f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcdcf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1cc0 0x555f926f2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56832/0 (socket says 172.21.15.44:56832) 2023-09-10T00:41:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/1496772912 learned_addr learned my addr 172.21.15.44:0/1496772912 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcdceffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e71a0 0x555f926ed920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49952/0 (socket says 172.21.15.44:49952) 2023-09-10T00:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcdcffff700 1 --2- 172.21.15.44:0/1496772912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926ede60 0x555f926dbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/1496772912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926ede60 msgr2=0x555f926dbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcdcf7fe700 1 --2- 172.21.15.44:0/1496772912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926ede60 0x555f926dbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/1496772912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e71a0 msgr2=0x555f926ed920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcdcf7fe700 1 --2- 172.21.15.44:0/1496772912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e71a0 0x555f926ed920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/1496772912 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f92448c00 con 0x555f926e1cc0 2023-09-10T00:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcdcffff700 1 --2- 172.21.15.44:0/1496772912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926ede60 0x555f926dbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcdceffd700 1 --2- 172.21.15.44:0/1496772912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e71a0 0x555f926ed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.935+0000 7fcdcf7fe700 1 --2- 172.21.15.44:0/1496772912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1cc0 0x555f926f2490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0063040 tx=0x7fcdc009f820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab917e6d8bc35154 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.936+0000 7fcdb67fc700 1 -- 172.21.15.44:0/1496772912 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0095970 con 0x555f926e1cc0 2023-09-10T00:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.936+0000 7fcdb67fc700 1 -- 172.21.15.44:0/1496772912 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcdc00c98e0 con 0x555f926e1cc0 2023-09-10T00:41:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.936+0000 7fcdb67fc700 1 -- 172.21.15.44:0/1496772912 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc00a16a0 con 0x555f926e1cc0 2023-09-10T00:41:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.936+0000 7fcde1f38b80 1 -- 172.21.15.44:0/1496772912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1cc0 msgr2=0x555f926f2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.936+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/1496772912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1cc0 0x555f926f2490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0063040 tx=0x7fcdc009f820 comp rx=0 tx=0).stop 2023-09-10T00:41:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.936+0000 7fcde1f38b80 1 -- 172.21.15.44:0/1496772912 shutdown_connections 2023-09-10T00:41:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.936+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/1496772912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926ede60 0x555f926dbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.936+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/1496772912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1cc0 0x555f926f2490 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.936+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/1496772912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926e71a0 0x555f926ed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.936+0000 7fcde1f38b80 1 -- 172.21.15.44:0/1496772912 >> 172.21.15.44:0/1496772912 conn(0x555f92656030 msgr2=0x555f926e68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.936+0000 7fcde1f38b80 1 -- 172.21.15.44:0/1496772912 shutdown_connections 2023-09-10T00:41:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.937+0000 7fcde1f38b80 1 -- 172.21.15.44:0/1496772912 wait complete. 2023-09-10T00:41:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.937+0000 7fcde1f38b80 1 -- start start 2023-09-10T00:41:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.937+0000 7fcde1f38b80 1 Processor -- start 2023-09-10T00:41:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.937+0000 7fcde1f38b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1cc0 0x555f926f2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.937+0000 7fcde1f38b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e71a0 0x555f926f0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.937+0000 7fcde1f38b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926ede60 0x555f926f09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.937+0000 7fcde1f38b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f926f0fa0 con 0x555f926ede60 2023-09-10T00:41:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.937+0000 7fcde1f38b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f926f10e0 con 0x555f926e71a0 2023-09-10T00:41:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.937+0000 7fcde1f38b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f926f12a0 con 0x555f926e1cc0 2023-09-10T00:41:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.937+0000 7fcde1f38b80 10 client.0 _ll_register_callbacks cb 0x7fcdc800e280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:41:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcdcf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1cc0 0x555f926f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcdcf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1cc0 0x555f926f2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56838/0 (socket says 172.21.15.44:56838) 2023-09-10T00:41:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcdcffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926ede60 0x555f926f09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcdceffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e71a0 0x555f926f0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/242449986 learned_addr learned my addr 172.21.15.44:0/242449986 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:41:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/242449986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e71a0 msgr2=0x555f926f0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcdcf7fe700 1 --2- 172.21.15.44:0/242449986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e71a0 0x555f926f0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/242449986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926ede60 msgr2=0x555f926f09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:41:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcdcf7fe700 1 --2- 172.21.15.44:0/242449986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926ede60 0x555f926f09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcdcf7fe700 1 -- 172.21.15.44:0/242449986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd010dd10 con 0x555f926e1cc0 2023-09-10T00:41:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcdceffd700 1 --2- 172.21.15.44:0/242449986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e71a0 0x555f926f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:41:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcdcf7fe700 1 --2- 172.21.15.44:0/242449986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1cc0 0x555f926f2170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00c9b40 tx=0x7fcdc0065450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=759173636a19061a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:41:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcdaeffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:41:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcdaeffd700 1 -- 172.21.15.44:0/242449986 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0095970 con 0x555f926e1cc0 2023-09-10T00:41:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcdaeffd700 1 -- 172.21.15.44:0/242449986 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fcdc00842d0 con 0x555f926e1cc0 2023-09-10T00:41:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcde1f38b80 1 -- 172.21.15.44:0/242449986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdc009d460 con 0x555f926e1cc0 2023-09-10T00:41:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcdaeffd700 1 -- 172.21.15.44:0/242449986 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdc00a16a0 con 0x555f926e1cc0 2023-09-10T00:41:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcde1f38b80 10 client.25891 fetch_fsmap learned FSMap version 34 2023-09-10T00:41:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.938+0000 7fcde1f38b80 1 -- 172.21.15.44:0/242449986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fcdd01a33e0 con 0x555f926e1cc0 2023-09-10T00:41:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.939+0000 7fcdaeffd700 1 -- 172.21.15.44:0/242449986 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcdc009d460 con 0x555f926e1cc0 2023-09-10T00:41:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.939+0000 7fcdaeffd700 1 -- 172.21.15.44:0/242449986 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fcdc0081070 con 0x555f926e1cc0 2023-09-10T00:41:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.939+0000 7fcde1f38b80 10 client.25891 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:41:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.939+0000 7fcde1f38b80 10 client.25891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:41:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.939+0000 7fcde1f38b80 20 client.25891 populate_metadata read hostname 'smithi044' 2023-09-10T00:41:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.939+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/242449986 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f92750010 0x555f92752430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:41:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.939+0000 7fcde1f38b80 4 client.25891 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:41:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.939+0000 7fcde1f38b80 1 -- 172.21.15.44:0/242449986 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555f9262c8c0 con 0x555f92750010 2023-09-10T00:41:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.939+0000 7fcdceffd700 1 --2- 172.21.15.44:0/242449986 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f92750010 0x555f92752430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:41:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.939+0000 7fcdceffd700 1 --2- 172.21.15.44:0/242449986 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f92750010 0x555f92752430 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b024f326531dbe80 server_cookie=3d664748960bb862 in_seq=0 out_seq=0 2023-09-10T00:41:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.940+0000 7fcdaeffd700 10 client.25891 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.953+0000 7fcdaeffd700 1 -- 172.21.15.44:0/242449986 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555f9262c8c0 con 0x555f92750010 2023-09-10T00:41:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.953+0000 7fcdaeffd700 10 client.25891 handle_command_reply: tid=0 2023-09-10T00:41:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.953+0000 7fcde1f38b80 1 client.25891 shutdown 2023-09-10T00:41:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.953+0000 7fcde1f38b80 1 -- 172.21.15.44:0/242449986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1cc0 msgr2=0x555f926f2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.953+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/242449986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1cc0 0x555f926f2170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00c9b40 tx=0x7fcdc0065450 comp rx=0 tx=0).stop 2023-09-10T00:41:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.953+0000 7fcde1f38b80 1 -- 172.21.15.44:0/242449986 shutdown_connections 2023-09-10T00:41:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.953+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/242449986 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f92750010 0x555f92752430 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.953+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/242449986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f926ede60 0x555f926f09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.953+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/242449986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f926e1cc0 0x555f926f2170 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.953+0000 7fcde1f38b80 1 --2- 172.21.15.44:0/242449986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f926e71a0 0x555f926f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.953+0000 7fcdaeffd700 0 client.25891 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:41:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.953+0000 7fcde1f38b80 1 -- 172.21.15.44:0/242449986 >> 172.21.15.44:0/242449986 conn(0x555f92656030 msgr2=0x555f926e02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.953+0000 7fcde1f38b80 1 -- 172.21.15.44:0/242449986 shutdown_connections 2023-09-10T00:41:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.953+0000 7fcde1f38b80 1 -- 172.21.15.44:0/242449986 wait complete. 2023-09-10T00:41:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.953+0000 7fcde1f38b80 20 client.25891 trim_cache size 0 max 16384 2023-09-10T00:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.954+0000 7fcdd5d72700 1 -- 172.21.15.44:0/4058163497 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcdb80a08d0 msgr2=0x7fcdb80a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.954+0000 7fcdd5d72700 1 --2- 172.21.15.44:0/4058163497 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcdb80a08d0 0x7fcdb80a2d90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4006010 tx=0x7fcdc4005960 comp rx=0 tx=0).stop 2023-09-10T00:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.954+0000 7fcdd5d72700 1 -- 172.21.15.44:0/4058163497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdd01017e0 msgr2=0x7fcdd019c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:41:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.954+0000 7fcdd5d72700 1 --2- 172.21.15.44:0/4058163497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdd01017e0 0x7fcdd019c5d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000e9a0 tx=0x7fcdc000ecb0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.954+0000 7fcdd5d72700 1 -- 172.21.15.44:0/4058163497 shutdown_connections 2023-09-10T00:41:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.954+0000 7fcdd5d72700 1 --2- 172.21.15.44:0/4058163497 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcdb80a08d0 0x7fcdb80a2d90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.954+0000 7fcdd5d72700 1 --2- 172.21.15.44:0/4058163497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdd010abb0 0x7fcdd0196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.954+0000 7fcdd5d72700 1 --2- 172.21.15.44:0/4058163497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdd01017e0 0x7fcdd019c5d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.954+0000 7fcdd5d72700 1 --2- 172.21.15.44:0/4058163497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdd0102100 0x7fcdd019cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:41:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.954+0000 7fcdd5d72700 1 -- 172.21.15.44:0/4058163497 >> 172.21.15.44:0/4058163497 conn(0x7fcdd00fafc0 msgr2=0x7fcdd00fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:41:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.954+0000 7fcdd5d72700 1 -- 172.21.15.44:0/4058163497 shutdown_connections 2023-09-10T00:41:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:41:47.954+0000 7fcdd5d72700 1 -- 172.21.15.44:0/4058163497 wait complete. 2023-09-10T00:41:48.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:47 smithi099 ceph-mon[102692]: pgmap v37: 129 pgs: 129 active+clean; 957 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 38 MiB/s wr, 104 op/s 2023-09-10T00:41:48.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:47 smithi131 ceph-mon[103713]: pgmap v37: 129 pgs: 129 active+clean; 957 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 38 MiB/s wr, 104 op/s 2023-09-10T00:41:48.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:47 smithi044 ceph-mon[98676]: pgmap v37: 129 pgs: 129 active+clean; 957 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 38 MiB/s wr, 104 op/s 2023-09-10T00:41:48.879 INFO:teuthology.orchestra.run.smithi044.stderr:4300000 of 50000000 tuples (8%) done (elapsed 48.51 s, remaining 515.59 s) 2023-09-10T00:41:50.187 INFO:teuthology.orchestra.run.smithi044.stderr:4400000 of 50000000 tuples (8%) done (elapsed 49.82 s, remaining 516.33 s) 2023-09-10T00:41:50.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:50 smithi099 ceph-mon[102692]: pgmap v38: 129 pgs: 129 active+clean; 957 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 34 MiB/s wr, 102 op/s 2023-09-10T00:41:50.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:50 smithi131 ceph-mon[103713]: pgmap v38: 129 pgs: 129 active+clean; 957 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 34 MiB/s wr, 102 op/s 2023-09-10T00:41:50.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:50 smithi044 ceph-mon[98676]: pgmap v38: 129 pgs: 129 active+clean; 957 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 34 MiB/s wr, 102 op/s 2023-09-10T00:41:51.083 INFO:teuthology.orchestra.run.smithi044.stderr:4500000 of 50000000 tuples (9%) done (elapsed 50.72 s, remaining 512.81 s) 2023-09-10T00:41:52.303 INFO:teuthology.orchestra.run.smithi044.stderr:4600000 of 50000000 tuples (9%) done (elapsed 51.94 s, remaining 512.59 s) 2023-09-10T00:41:52.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:52 smithi131 ceph-mon[103713]: pgmap v39: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 51 MiB/s wr, 120 op/s 2023-09-10T00:41:52.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:52 smithi044 ceph-mon[98676]: pgmap v39: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 51 MiB/s wr, 120 op/s 2023-09-10T00:41:52.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:52 smithi099 ceph-mon[102692]: pgmap v39: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 51 MiB/s wr, 120 op/s 2023-09-10T00:41:53.541 INFO:teuthology.orchestra.run.smithi044.stderr:4700000 of 50000000 tuples (9%) done (elapsed 53.18 s, remaining 512.52 s) 2023-09-10T00:41:54.468 INFO:teuthology.orchestra.run.smithi044.stderr:4800000 of 50000000 tuples (9%) done (elapsed 54.10 s, remaining 509.46 s) 2023-09-10T00:41:54.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:54 smithi131 ceph-mon[103713]: pgmap v40: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 34 MiB/s wr, 50 op/s 2023-09-10T00:41:54.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:54 smithi044 ceph-mon[98676]: pgmap v40: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 34 MiB/s wr, 50 op/s 2023-09-10T00:41:54.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:54 smithi099 ceph-mon[102692]: pgmap v40: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 34 MiB/s wr, 50 op/s 2023-09-10T00:41:55.737 INFO:teuthology.orchestra.run.smithi044.stderr:4900000 of 50000000 tuples (9%) done (elapsed 55.37 s, remaining 509.64 s) 2023-09-10T00:41:56.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:56 smithi131 ceph-mon[103713]: pgmap v41: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 49 MiB/s wr, 64 op/s 2023-09-10T00:41:56.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:56 smithi044 ceph-mon[98676]: pgmap v41: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 49 MiB/s wr, 64 op/s 2023-09-10T00:41:56.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:56 smithi099 ceph-mon[102692]: pgmap v41: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 49 MiB/s wr, 64 op/s 2023-09-10T00:41:57.011 INFO:teuthology.orchestra.run.smithi044.stderr:5000000 of 50000000 tuples (10%) done (elapsed 56.65 s, remaining 509.81 s) 2023-09-10T00:41:57.889 INFO:teuthology.orchestra.run.smithi044.stderr:5100000 of 50000000 tuples (10%) done (elapsed 57.52 s, remaining 506.43 s) 2023-09-10T00:41:58.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:41:58 smithi131 ceph-mon[103713]: pgmap v42: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 MiB/s wr, 38 op/s 2023-09-10T00:41:58.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:41:58 smithi044 ceph-mon[98676]: pgmap v42: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 MiB/s wr, 38 op/s 2023-09-10T00:41:58.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:41:58 smithi099 ceph-mon[102692]: pgmap v42: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 MiB/s wr, 38 op/s 2023-09-10T00:41:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:5200000 of 50000000 tuples (10%) done (elapsed 58.82 s, remaining 506.73 s) 2023-09-10T00:42:00.013 INFO:teuthology.orchestra.run.smithi044.stderr:5300000 of 50000000 tuples (10%) done (elapsed 59.65 s, remaining 503.06 s) 2023-09-10T00:42:00.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:00 smithi131 ceph-mon[103713]: pgmap v43: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 MiB/s wr, 33 op/s 2023-09-10T00:42:00.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:00 smithi044 ceph-mon[98676]: pgmap v43: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 MiB/s wr, 33 op/s 2023-09-10T00:42:00.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:00 smithi099 ceph-mon[102692]: pgmap v43: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 MiB/s wr, 33 op/s 2023-09-10T00:42:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.089+0000 7f7cadfb5700 1 -- 172.21.15.44:0/3707865328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ca80ffe70 msgr2=0x7f7ca810c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.089+0000 7f7cadfb5700 1 --2- 172.21.15.44:0/3707865328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ca80ffe70 0x7f7ca810c310 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000b0a0 tx=0x7f7ca000b3b0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.090+0000 7f7cadfb5700 1 -- 172.21.15.44:0/3707865328 shutdown_connections 2023-09-10T00:42:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.090+0000 7f7cadfb5700 1 --2- 172.21.15.44:0/3707865328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ca80ffe70 0x7f7ca810c310 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.090+0000 7f7cadfb5700 1 --2- 172.21.15.44:0/3707865328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ca8104420 0x7f7ca8104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.090+0000 7f7cadfb5700 1 --2- 172.21.15.44:0/3707865328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ca80ff4b0 0x7f7ca80ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.090+0000 7f7cadfb5700 1 -- 172.21.15.44:0/3707865328 >> 172.21.15.44:0/3707865328 conn(0x7f7ca8076600 msgr2=0x7f7ca8076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.090+0000 7f7cadfb5700 1 -- 172.21.15.44:0/3707865328 shutdown_connections 2023-09-10T00:42:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.090+0000 7f7cadfb5700 1 -- 172.21.15.44:0/3707865328 wait complete. 2023-09-10T00:42:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.091+0000 7f7cadfb5700 1 Processor -- start 2023-09-10T00:42:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.091+0000 7f7cadfb5700 1 -- start start 2023-09-10T00:42:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.091+0000 7f7cadfb5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ca80ff4b0 0x7f7ca8197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.091+0000 7f7cadfb5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ca80ffe70 0x7f7ca8197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.091+0000 7f7cadfb5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ca8104420 0x7f7ca81a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.091+0000 7f7cadfb5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca81a2c00 con 0x7f7ca8104420 2023-09-10T00:42:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.091+0000 7f7cadfb5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca81a2d70 con 0x7f7ca80ffe70 2023-09-10T00:42:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.091+0000 7f7cadfb5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca81a2f30 con 0x7f7ca80ff4b0 2023-09-10T00:42:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.091+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ca80ffe70 0x7f7ca8197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.091+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ca80ffe70 0x7f7ca8197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51710/0 (socket says 172.21.15.44:51710) 2023-09-10T00:42:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.091+0000 7f7ca6ffd700 1 -- 172.21.15.44:0/1662989824 learned_addr learned my addr 172.21.15.44:0/1662989824 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.091+0000 7f7ca77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ca80ff4b0 0x7f7ca8197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.091+0000 7f7ca7fff700 1 --2- 172.21.15.44:0/1662989824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ca8104420 0x7f7ca81a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.092+0000 7f7ca6ffd700 1 -- 172.21.15.44:0/1662989824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ca80ff4b0 msgr2=0x7f7ca8197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.092+0000 7f7ca6ffd700 1 --2- 172.21.15.44:0/1662989824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ca80ff4b0 0x7f7ca8197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.092+0000 7f7ca6ffd700 1 -- 172.21.15.44:0/1662989824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ca8104420 msgr2=0x7f7ca81a2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.092+0000 7f7ca6ffd700 1 --2- 172.21.15.44:0/1662989824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ca8104420 0x7f7ca81a2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.092+0000 7f7ca6ffd700 1 -- 172.21.15.44:0/1662989824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca0009d00 con 0x7f7ca80ffe70 2023-09-10T00:42:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.092+0000 7f7ca6ffd700 1 --2- 172.21.15.44:0/1662989824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ca80ffe70 0x7f7ca8197ca0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e9a0 tx=0x7f7c9c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.092+0000 7f7ca4ff9700 1 -- 172.21.15.44:0/1662989824 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c009790 con 0x7f7ca80ffe70 2023-09-10T00:42:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.092+0000 7f7cadfb5700 1 -- 172.21.15.44:0/1662989824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca81a3210 con 0x7f7ca80ffe70 2023-09-10T00:42:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.092+0000 7f7ca4ff9700 1 -- 172.21.15.44:0/1662989824 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c9c010430 con 0x7f7ca80ffe70 2023-09-10T00:42:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.092+0000 7f7ca4ff9700 1 -- 172.21.15.44:0/1662989824 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c00cea0 con 0x7f7ca80ffe70 2023-09-10T00:42:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.092+0000 7f7cadfb5700 1 -- 172.21.15.44:0/1662989824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca81a3720 con 0x7f7ca80ffe70 2023-09-10T00:42:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.095+0000 7f7ca4ff9700 1 -- 172.21.15.44:0/1662989824 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f7c9c0105a0 con 0x7f7ca80ffe70 2023-09-10T00:42:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.095+0000 7f7ca4ff9700 1 --2- 172.21.15.44:0/1662989824 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7c900a08d0 0x7f7c900a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.096+0000 7f7ca77fe700 1 --2- 172.21.15.44:0/1662989824 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7c900a08d0 0x7f7c900a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.096+0000 7f7ca4ff9700 1 -- 172.21.15.44:0/1662989824 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7c9c014070 con 0x7f7ca80ffe70 2023-09-10T00:42:01.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.096+0000 7f7cba17bb80 1 Processor -- start 2023-09-10T00:42:01.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.096+0000 7f7ca77fe700 1 --2- 172.21.15.44:0/1662989824 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7c900a08d0 0x7f7c900a2d90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7c98009be0 tx=0x7f7c98009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:01.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.096+0000 7f7cba17bb80 1 -- start start 2023-09-10T00:42:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.096+0000 7f7cba17bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558582291a0 0x555858229580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.096+0000 7f7cba17bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555858223300 0x555858223780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.096+0000 7f7cba17bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223cc0 0x55585822ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7cba17bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585817c6d0 con 0x555858223300 2023-09-10T00:42:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7cba17bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555858179ff0 con 0x5558582291a0 2023-09-10T00:42:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7cba17bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558581758e0 con 0x555858223cc0 2023-09-10T00:42:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7ca77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558582291a0 0x555858229580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555858223300 0x555858223780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7ca77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558582291a0 0x555858229580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51716/0 (socket says 172.21.15.44:51716) 2023-09-10T00:42:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555858223300 0x555858223780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49978/0 (socket says 172.21.15.44:49978) 2023-09-10T00:42:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7ca77fe700 1 -- 172.21.15.44:0/2617214440 learned_addr learned my addr 172.21.15.44:0/2617214440 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:01.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7ca7fff700 1 --2- 172.21.15.44:0/2617214440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223cc0 0x55585822ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7ca77fe700 1 -- 172.21.15.44:0/2617214440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223cc0 msgr2=0x55585822ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:01.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7ca77fe700 1 --2- 172.21.15.44:0/2617214440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223cc0 0x55585822ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7ca77fe700 1 -- 172.21.15.44:0/2617214440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555858223300 msgr2=0x555858223780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7ca77fe700 1 --2- 172.21.15.44:0/2617214440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555858223300 0x555858223780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7ca77fe700 1 -- 172.21.15.44:0/2617214440 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555857fa1d80 con 0x5558582291a0 2023-09-10T00:42:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7ca6ffd700 1 --2- 172.21.15.44:0/2617214440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555858223300 0x555858223780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7ca77fe700 1 --2- 172.21.15.44:0/2617214440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558582291a0 0x555858229580 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c98011d70 tx=0x7f7c98014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f02a7d85aa876193 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7c8dffb700 1 -- 172.21.15.44:0/2617214440 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98011da0 con 0x5558582291a0 2023-09-10T00:42:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7c8dffb700 1 -- 172.21.15.44:0/2617214440 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c98004020 con 0x5558582291a0 2023-09-10T00:42:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.097+0000 7f7c8dffb700 1 -- 172.21.15.44:0/2617214440 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98003ba0 con 0x5558582291a0 2023-09-10T00:42:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.098+0000 7f7cba17bb80 1 -- 172.21.15.44:0/2617214440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558582291a0 msgr2=0x555858229580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.098+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/2617214440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558582291a0 0x555858229580 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c98011d70 tx=0x7f7c98014000 comp rx=0 tx=0).stop 2023-09-10T00:42:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.098+0000 7f7cba17bb80 1 -- 172.21.15.44:0/2617214440 shutdown_connections 2023-09-10T00:42:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.098+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/2617214440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223cc0 0x55585822ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.098+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/2617214440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558582291a0 0x555858229580 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.098+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/2617214440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555858223300 0x555858223780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.098+0000 7f7cba17bb80 1 -- 172.21.15.44:0/2617214440 >> 172.21.15.44:0/2617214440 conn(0x555858198030 msgr2=0x55585821e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:01.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.098+0000 7f7cba17bb80 1 -- 172.21.15.44:0/2617214440 shutdown_connections 2023-09-10T00:42:01.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.098+0000 7f7cba17bb80 1 -- 172.21.15.44:0/2617214440 wait complete. 2023-09-10T00:42:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.099+0000 7f7cba17bb80 1 -- start start 2023-09-10T00:42:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.099+0000 7f7cba17bb80 1 Processor -- start 2023-09-10T00:42:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.099+0000 7f7cba17bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223300 0x55585822fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.099+0000 7f7cba17bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555858223cc0 0x55585822d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.099+0000 7f7cba17bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558582291a0 0x55585822d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.099+0000 7f7cba17bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555858113b10 con 0x555858223cc0 2023-09-10T00:42:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.099+0000 7f7cba17bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555858113810 con 0x555858223300 2023-09-10T00:42:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.099+0000 7f7cba17bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555857ff78c0 con 0x5558582291a0 2023-09-10T00:42:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.099+0000 7f7cba17bb80 10 client.0 _ll_register_callbacks cb 0x55585805da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:42:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.099+0000 7f7ca77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223300 0x55585822fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.099+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555858223cc0 0x55585822d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.099+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555858223cc0 0x55585822d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49984/0 (socket says 172.21.15.44:49984) 2023-09-10T00:42:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.099+0000 7f7ca77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223300 0x55585822fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51722/0 (socket says 172.21.15.44:51722) 2023-09-10T00:42:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.099+0000 7f7ca6ffd700 1 -- 172.21.15.44:0/1837416272 learned_addr learned my addr 172.21.15.44:0/1837416272 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.099+0000 7f7ca7fff700 1 --2- 172.21.15.44:0/1837416272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558582291a0 0x55585822d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.100+0000 7f7ca6ffd700 1 -- 172.21.15.44:0/1837416272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558582291a0 msgr2=0x55585822d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.100+0000 7f7ca6ffd700 1 --2- 172.21.15.44:0/1837416272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558582291a0 0x55585822d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.100+0000 7f7ca6ffd700 1 -- 172.21.15.44:0/1837416272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223300 msgr2=0x55585822fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.100+0000 7f7ca6ffd700 1 --2- 172.21.15.44:0/1837416272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223300 0x55585822fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.100+0000 7f7ca6ffd700 1 -- 172.21.15.44:0/1837416272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca81a3720 con 0x555858223cc0 2023-09-10T00:42:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.100+0000 7f7ca77fe700 1 --2- 172.21.15.44:0/1837416272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223300 0x55585822fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.100+0000 7f7ca6ffd700 1 --2- 172.21.15.44:0/1837416272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555858223cc0 0x55585822d080 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e970 tx=0x7f7c9c010880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9db30498314f14a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.100+0000 7f7c86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:42:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.100+0000 7f7c86ffd700 1 -- 172.21.15.44:0/1837416272 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c099ac0 con 0x555858223cc0 2023-09-10T00:42:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.100+0000 7f7c86ffd700 1 -- 172.21.15.44:0/1837416272 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7c9c099660 con 0x555858223cc0 2023-09-10T00:42:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.100+0000 7f7cba17bb80 1 -- 172.21.15.44:0/1837416272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558580cd490 con 0x555858223cc0 2023-09-10T00:42:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.100+0000 7f7c86ffd700 1 -- 172.21.15.44:0/1837416272 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c9c09d490 con 0x555858223cc0 2023-09-10T00:42:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.100+0000 7f7cba17bb80 10 client.16299 fetch_fsmap learned FSMap version 34 2023-09-10T00:42:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.100+0000 7f7cba17bb80 1 -- 172.21.15.44:0/1837416272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f7c98017070 con 0x555858223cc0 2023-09-10T00:42:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.100+0000 7f7c86ffd700 1 -- 172.21.15.44:0/1837416272 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5558580cd490 con 0x555858223cc0 2023-09-10T00:42:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.101+0000 7f7c86ffd700 1 -- 172.21.15.44:0/1837416272 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7c9c098b50 con 0x555858223cc0 2023-09-10T00:42:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.101+0000 7f7cba17bb80 10 client.16299 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:42:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.101+0000 7f7cba17bb80 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:42:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.101+0000 7f7cba17bb80 20 client.16299 populate_metadata read hostname 'smithi044' 2023-09-10T00:42:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.101+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/1837416272 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55585823d440 0x55585823f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.101+0000 7f7cba17bb80 4 client.16299 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:42:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.101+0000 7f7cba17bb80 1 -- 172.21.15.44:0/1837416272 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55585818a840 con 0x55585823d440 2023-09-10T00:42:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.101+0000 7f7ca77fe700 1 --2- 172.21.15.44:0/1837416272 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55585823d440 0x55585823f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.101+0000 7f7c86ffd700 10 client.16299 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.101+0000 7f7ca77fe700 1 --2- 172.21.15.44:0/1837416272 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55585823d440 0x55585823f860 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1452e03e48ec04e server_cookie=b8c6489be07b9bd1 in_seq=0 out_seq=0 2023-09-10T00:42:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.102+0000 7f7c86ffd700 1 -- 172.21.15.44:0/1837416272 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55585818a840 con 0x55585823d440 2023-09-10T00:42:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.102+0000 7f7c86ffd700 10 client.16299 handle_command_reply: tid=0 2023-09-10T00:42:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.102+0000 7f7cba17bb80 1 client.16299 shutdown 2023-09-10T00:42:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.103+0000 7f7cba17bb80 1 -- 172.21.15.44:0/1837416272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555858223cc0 msgr2=0x55585822d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.103+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/1837416272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555858223cc0 0x55585822d080 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e970 tx=0x7f7c9c010880 comp rx=0 tx=0).stop 2023-09-10T00:42:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.103+0000 7f7cba17bb80 1 -- 172.21.15.44:0/1837416272 shutdown_connections 2023-09-10T00:42:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.103+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/1837416272 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55585823d440 0x55585823f860 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.103+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/1837416272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558582291a0 0x55585822d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.103+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/1837416272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223300 0x55585822fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.103+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/1837416272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555858223cc0 0x55585822d080 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.103+0000 7f7cba17bb80 1 -- 172.21.15.44:0/1837416272 >> 172.21.15.44:0/1837416272 conn(0x555858198030 msgr2=0x55585821d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.103+0000 7f7c86ffd700 0 client.16299 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.103+0000 7f7cba17bb80 1 -- 172.21.15.44:0/1837416272 shutdown_connections 2023-09-10T00:42:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.103+0000 7f7cba17bb80 1 -- 172.21.15.44:0/1837416272 wait complete. 2023-09-10T00:42:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.103+0000 7f7cba17bb80 20 client.16299 trim_cache size 0 max 16384 2023-09-10T00:42:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.122+0000 7f7cba17bb80 1 Processor -- start 2023-09-10T00:42:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.122+0000 7f7cba17bb80 1 -- start start 2023-09-10T00:42:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.122+0000 7f7cba17bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223300 0x555858234520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.122+0000 7f7cba17bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223cc0 0x55585822f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7cba17bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5558582291a0 0x55585822fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7cba17bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555858227ae0 con 0x5558582291a0 2023-09-10T00:42:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7cba17bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555858224580 con 0x555858223300 2023-09-10T00:42:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7cba17bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585822dfa0 con 0x555858223cc0 2023-09-10T00:42:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7ca77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223300 0x555858234520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223cc0 0x55585822f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7ca7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5558582291a0 0x55585822fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223cc0 0x55585822f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56874/0 (socket says 172.21.15.44:56874) 2023-09-10T00:42:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7ca77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223300 0x555858234520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51730/0 (socket says 172.21.15.44:51730) 2023-09-10T00:42:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7ca6ffd700 1 -- 172.21.15.44:0/3460251549 learned_addr learned my addr 172.21.15.44:0/3460251549 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7ca7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5558582291a0 0x55585822fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49994/0 (socket says 172.21.15.44:49994) 2023-09-10T00:42:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7ca6ffd700 1 -- 172.21.15.44:0/3460251549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223300 msgr2=0x555858234520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7ca6ffd700 1 --2- 172.21.15.44:0/3460251549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223300 0x555858234520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7ca6ffd700 1 -- 172.21.15.44:0/3460251549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5558582291a0 msgr2=0x55585822fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7ca6ffd700 1 --2- 172.21.15.44:0/3460251549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5558582291a0 0x55585822fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7ca6ffd700 1 -- 172.21.15.44:0/3460251549 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555857f8ac00 con 0x555858223cc0 2023-09-10T00:42:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7ca7fff700 1 --2- 172.21.15.44:0/3460251549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5558582291a0 0x55585822fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7ca6ffd700 1 --2- 172.21.15.44:0/3460251549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223cc0 0x55585822f890 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00c2d0 tx=0x7f7c9c095ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50253022b262ea47 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.123+0000 7f7c8e7fc700 1 -- 172.21.15.44:0/3460251549 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c0a12e0 con 0x555858223cc0 2023-09-10T00:42:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.124+0000 7f7c8e7fc700 1 -- 172.21.15.44:0/3460251549 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c9c0c9910 con 0x555858223cc0 2023-09-10T00:42:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.124+0000 7f7c8e7fc700 1 -- 172.21.15.44:0/3460251549 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c0653e0 con 0x555858223cc0 2023-09-10T00:42:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.124+0000 7f7cba17bb80 1 -- 172.21.15.44:0/3460251549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223cc0 msgr2=0x55585822f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.124+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/3460251549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223cc0 0x55585822f890 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00c2d0 tx=0x7f7c9c095ad0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.124+0000 7f7cba17bb80 1 -- 172.21.15.44:0/3460251549 shutdown_connections 2023-09-10T00:42:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.124+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/3460251549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5558582291a0 0x55585822fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.124+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/3460251549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223300 0x555858234520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.124+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/3460251549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223cc0 0x55585822f890 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.124+0000 7f7cba17bb80 1 -- 172.21.15.44:0/3460251549 >> 172.21.15.44:0/3460251549 conn(0x555858198030 msgr2=0x555858222b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.124+0000 7f7cba17bb80 1 -- 172.21.15.44:0/3460251549 shutdown_connections 2023-09-10T00:42:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.124+0000 7f7cba17bb80 1 -- 172.21.15.44:0/3460251549 wait complete. 2023-09-10T00:42:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.125+0000 7f7cba17bb80 1 -- start start 2023-09-10T00:42:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.125+0000 7f7cba17bb80 1 Processor -- start 2023-09-10T00:42:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.125+0000 7f7cba17bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223300 0x5558582318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.125+0000 7f7cba17bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223cc0 0x555858231de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.125+0000 7f7cba17bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5558582291a0 0x555858232320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.125+0000 7f7cba17bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555858234900 con 0x5558582291a0 2023-09-10T00:42:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.125+0000 7f7cba17bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555858290250 con 0x555858223cc0 2023-09-10T00:42:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.125+0000 7f7cba17bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558582903e0 con 0x555858223300 2023-09-10T00:42:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.125+0000 7f7cba17bb80 10 client.0 _ll_register_callbacks cb 0x7f7c9c098fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:42:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.126+0000 7f7ca77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223300 0x5558582318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.126+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223cc0 0x555858231de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.126+0000 7f7ca77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223300 0x5558582318a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56878/0 (socket says 172.21.15.44:56878) 2023-09-10T00:42:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.126+0000 7f7ca77fe700 1 -- 172.21.15.44:0/480881658 learned_addr learned my addr 172.21.15.44:0/480881658 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.126+0000 7f7ca7fff700 1 --2- 172.21.15.44:0/480881658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5558582291a0 0x555858232320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.126+0000 7f7ca77fe700 1 -- 172.21.15.44:0/480881658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223cc0 msgr2=0x555858231de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.126+0000 7f7ca77fe700 1 --2- 172.21.15.44:0/480881658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223cc0 0x555858231de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.126+0000 7f7ca77fe700 1 -- 172.21.15.44:0/480881658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5558582291a0 msgr2=0x555858232320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.126+0000 7f7ca77fe700 1 --2- 172.21.15.44:0/480881658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5558582291a0 0x555858232320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.126+0000 7f7ca77fe700 1 -- 172.21.15.44:0/480881658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca810cc00 con 0x555858223300 2023-09-10T00:42:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.126+0000 7f7ca77fe700 1 --2- 172.21.15.44:0/480881658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223300 0x5558582318a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7c98005950 tx=0x7f7c98005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f41c4e0f0c3d6d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.126+0000 7f7c86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:42:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.126+0000 7f7c86ffd700 1 -- 172.21.15.44:0/480881658 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98039030 con 0x555858223300 2023-09-10T00:42:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.126+0000 7f7cba17bb80 1 -- 172.21.15.44:0/480881658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c98004020 con 0x555858223300 2023-09-10T00:42:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.126+0000 7f7c86ffd700 1 -- 172.21.15.44:0/480881658 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7c9803a070 con 0x555858223300 2023-09-10T00:42:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.126+0000 7f7c86ffd700 1 -- 172.21.15.44:0/480881658 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c9802b040 con 0x555858223300 2023-09-10T00:42:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.127+0000 7f7cba17bb80 10 client.25909 fetch_fsmap learned FSMap version 34 2023-09-10T00:42:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.127+0000 7f7cba17bb80 1 -- 172.21.15.44:0/480881658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f7c98017070 con 0x555858223300 2023-09-10T00:42:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.127+0000 7f7c86ffd700 1 -- 172.21.15.44:0/480881658 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c98004020 con 0x555858223300 2023-09-10T00:42:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.127+0000 7f7c86ffd700 1 -- 172.21.15.44:0/480881658 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7c98024030 con 0x555858223300 2023-09-10T00:42:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.127+0000 7f7cba17bb80 10 client.25909 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:42:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.127+0000 7f7cba17bb80 10 client.25909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:42:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.127+0000 7f7cba17bb80 20 client.25909 populate_metadata read hostname 'smithi044' 2023-09-10T00:42:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.127+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/480881658 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555858291bd0 0x555858293ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.127+0000 7f7cba17bb80 4 client.25909 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:42:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.127+0000 7f7cba17bb80 1 -- 172.21.15.44:0/480881658 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55585816e8c0 con 0x555858291bd0 2023-09-10T00:42:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.127+0000 7f7ca6ffd700 1 --2- 172.21.15.44:0/480881658 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555858291bd0 0x555858293ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.127+0000 7f7c86ffd700 10 client.25909 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.127+0000 7f7ca6ffd700 1 --2- 172.21.15.44:0/480881658 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555858291bd0 0x555858293ff0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8741ec13d50b2d4 server_cookie=cedceb11d8298de2 in_seq=0 out_seq=0 2023-09-10T00:42:01.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.143+0000 7f7c86ffd700 1 -- 172.21.15.44:0/480881658 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55585816e8c0 con 0x555858291bd0 2023-09-10T00:42:01.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.143+0000 7f7c86ffd700 10 client.25909 handle_command_reply: tid=0 2023-09-10T00:42:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.143+0000 7f7cba17bb80 1 client.25909 shutdown 2023-09-10T00:42:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.143+0000 7f7cba17bb80 1 -- 172.21.15.44:0/480881658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223300 msgr2=0x5558582318a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.143+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/480881658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223300 0x5558582318a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7c98005950 tx=0x7f7c98005d30 comp rx=0 tx=0).stop 2023-09-10T00:42:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.143+0000 7f7cba17bb80 1 -- 172.21.15.44:0/480881658 shutdown_connections 2023-09-10T00:42:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.143+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/480881658 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555858291bd0 0x555858293ff0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.143+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/480881658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5558582291a0 0x555858232320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.143+0000 7f7c86ffd700 0 client.25909 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.143+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/480881658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555858223300 0x5558582318a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.143+0000 7f7cba17bb80 1 --2- 172.21.15.44:0/480881658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555858223cc0 0x555858231de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.143+0000 7f7cba17bb80 1 -- 172.21.15.44:0/480881658 >> 172.21.15.44:0/480881658 conn(0x555858198030 msgr2=0x555858238c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.143+0000 7f7cba17bb80 1 -- 172.21.15.44:0/480881658 shutdown_connections 2023-09-10T00:42:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.143+0000 7f7cba17bb80 1 -- 172.21.15.44:0/480881658 wait complete. 2023-09-10T00:42:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.143+0000 7f7cba17bb80 20 client.25909 trim_cache size 0 max 16384 2023-09-10T00:42:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:42:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:42:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:42:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:42:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.144+0000 7f7cadfb5700 1 -- 172.21.15.44:0/1662989824 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7c900a08d0 msgr2=0x7f7c900a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.144+0000 7f7cadfb5700 1 --2- 172.21.15.44:0/1662989824 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7c900a08d0 0x7f7c900a2d90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7c98009be0 tx=0x7f7c98009260 comp rx=0 tx=0).stop 2023-09-10T00:42:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.144+0000 7f7cadfb5700 1 -- 172.21.15.44:0/1662989824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ca80ffe70 msgr2=0x7f7ca8197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.144+0000 7f7cadfb5700 1 --2- 172.21.15.44:0/1662989824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ca80ffe70 0x7f7ca8197ca0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e9a0 tx=0x7f7c9c00ecb0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.144+0000 7f7cadfb5700 1 -- 172.21.15.44:0/1662989824 shutdown_connections 2023-09-10T00:42:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.144+0000 7f7cadfb5700 1 --2- 172.21.15.44:0/1662989824 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7c900a08d0 0x7f7c900a2d90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.144+0000 7f7cadfb5700 1 --2- 172.21.15.44:0/1662989824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ca8104420 0x7f7ca81a2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.144+0000 7f7cadfb5700 1 --2- 172.21.15.44:0/1662989824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ca80ff4b0 0x7f7ca8197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.144+0000 7f7cadfb5700 1 --2- 172.21.15.44:0/1662989824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ca80ffe70 0x7f7ca8197ca0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.144+0000 7f7cadfb5700 1 -- 172.21.15.44:0/1662989824 >> 172.21.15.44:0/1662989824 conn(0x7f7ca8076600 msgr2=0x7f7ca80fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.144+0000 7f7cadfb5700 1 -- 172.21.15.44:0/1662989824 shutdown_connections 2023-09-10T00:42:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:01.144+0000 7f7cadfb5700 1 -- 172.21.15.44:0/1662989824 wait complete. 2023-09-10T00:42:01.306 INFO:teuthology.orchestra.run.smithi044.stderr:5400000 of 50000000 tuples (10%) done (elapsed 60.94 s, remaining 503.32 s) 2023-09-10T00:42:02.594 INFO:teuthology.orchestra.run.smithi044.stderr:5500000 of 50000000 tuples (11%) done (elapsed 62.23 s, remaining 503.48 s) 2023-09-10T00:42:02.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:02 smithi131 ceph-mon[103713]: pgmap v44: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 51 MiB/s wr, 47 op/s 2023-09-10T00:42:02.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:02 smithi044 ceph-mon[98676]: pgmap v44: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 51 MiB/s wr, 47 op/s 2023-09-10T00:42:02.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:02 smithi099 ceph-mon[102692]: pgmap v44: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 51 MiB/s wr, 47 op/s 2023-09-10T00:42:03.469 INFO:teuthology.orchestra.run.smithi044.stderr:5600000 of 50000000 tuples (11%) done (elapsed 63.10 s, remaining 500.32 s) 2023-09-10T00:42:04.751 INFO:teuthology.orchestra.run.smithi044.stderr:5700000 of 50000000 tuples (11%) done (elapsed 64.39 s, remaining 500.40 s) 2023-09-10T00:42:04.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:04 smithi131 ceph-mon[103713]: pgmap v45: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 29 op/s 2023-09-10T00:42:04.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:04 smithi044 ceph-mon[98676]: pgmap v45: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 29 op/s 2023-09-10T00:42:04.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:04 smithi099 ceph-mon[102692]: pgmap v45: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 29 op/s 2023-09-10T00:42:06.012 INFO:teuthology.orchestra.run.smithi044.stderr:5800000 of 50000000 tuples (11%) done (elapsed 65.65 s, remaining 500.27 s) 2023-09-10T00:42:06.446 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:42:06.588 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.586+0000 7f02bb609700 1 -- 172.21.15.44:0/1536041454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02b41054e0 msgr2=0x7f02b41058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:06.588 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.586+0000 7f02bb609700 1 --2- 172.21.15.44:0/1536041454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02b41054e0 0x7f02b41058c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009990 tx=0x7f02b0009ca0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.586+0000 7f02bb609700 1 -- 172.21.15.44:0/1536041454 shutdown_connections 2023-09-10T00:42:06.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.586+0000 7f02bb609700 1 --2- 172.21.15.44:0/1536041454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b40ffe00 0x7f02b410c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.586+0000 7f02bb609700 1 --2- 172.21.15.44:0/1536041454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02b41054e0 0x7f02b41058c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.586+0000 7f02bb609700 1 --2- 172.21.15.44:0/1536041454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02b40ff440 0x7f02b40ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.586+0000 7f02bb609700 1 -- 172.21.15.44:0/1536041454 >> 172.21.15.44:0/1536041454 conn(0x7f02b40faf50 msgr2=0x7f02b40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:06.590 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.587+0000 7f02bb609700 1 -- 172.21.15.44:0/1536041454 shutdown_connections 2023-09-10T00:42:06.590 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.587+0000 7f02bb609700 1 -- 172.21.15.44:0/1536041454 wait complete. 2023-09-10T00:42:06.592 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.588+0000 7f02bb609700 1 Processor -- start 2023-09-10T00:42:06.592 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.588+0000 7f02bb609700 1 -- start start 2023-09-10T00:42:06.592 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.588+0000 7f02bb609700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02b40ff440 0x7f02b419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.592 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.588+0000 7f02bb609700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02b40ffe00 0x7f02b419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.592 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.588+0000 7f02bb609700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b41054e0 0x7f02b4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.588+0000 7f02bb609700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b4198c70 con 0x7f02b40ff440 2023-09-10T00:42:06.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.588+0000 7f02bb609700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b4198de0 con 0x7f02b41054e0 2023-09-10T00:42:06.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.588+0000 7f02bb609700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b4198fa0 con 0x7f02b40ffe00 2023-09-10T00:42:06.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.588+0000 7f02b93a5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02b40ff440 0x7f02b419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.588+0000 7f02b93a5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02b40ff440 0x7f02b419e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50010/0 (socket says 172.21.15.44:50010) 2023-09-10T00:42:06.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.588+0000 7f02b93a5700 1 -- 172.21.15.44:0/4148354558 learned_addr learned my addr 172.21.15.44:0/4148354558 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:06.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.589+0000 7f02b9ba6700 1 --2- 172.21.15.44:0/4148354558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b41054e0 0x7f02b4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.589+0000 7f02b93a5700 1 -- 172.21.15.44:0/4148354558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02b40ffe00 msgr2=0x7f02b419ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:06.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.589+0000 7f02b8ba4700 1 --2- 172.21.15.44:0/4148354558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02b40ffe00 0x7f02b419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.589+0000 7f02b93a5700 1 --2- 172.21.15.44:0/4148354558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02b40ffe00 0x7f02b419ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.589+0000 7f02b93a5700 1 -- 172.21.15.44:0/4148354558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b41054e0 msgr2=0x7f02b4198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:06.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.589+0000 7f02b93a5700 1 --2- 172.21.15.44:0/4148354558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b41054e0 0x7f02b4198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.589+0000 7f02b93a5700 1 -- 172.21.15.44:0/4148354558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b00095f0 con 0x7f02b40ff440 2023-09-10T00:42:06.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.589+0000 7f02b9ba6700 1 --2- 172.21.15.44:0/4148354558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b41054e0 0x7f02b4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:42:06.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.589+0000 7f02b8ba4700 1 --2- 172.21.15.44:0/4148354558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02b40ffe00 0x7f02b419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:42:06.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.589+0000 7f02b93a5700 1 --2- 172.21.15.44:0/4148354558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02b40ff440 0x7f02b419e590 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f02b0005910 tx=0x7f02b00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:06.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.589+0000 7f02aa7fc700 1 -- 172.21.15.44:0/4148354558 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b001d070 con 0x7f02b40ff440 2023-09-10T00:42:06.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.590+0000 7f02bb609700 1 -- 172.21.15.44:0/4148354558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b4199280 con 0x7f02b40ff440 2023-09-10T00:42:06.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.590+0000 7f02aa7fc700 1 -- 172.21.15.44:0/4148354558 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02b0021410 con 0x7f02b40ff440 2023-09-10T00:42:06.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.590+0000 7f02aa7fc700 1 -- 172.21.15.44:0/4148354558 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0017650 con 0x7f02b40ff440 2023-09-10T00:42:06.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.590+0000 7f02bb609700 1 -- 172.21.15.44:0/4148354558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b41a5580 con 0x7f02b40ff440 2023-09-10T00:42:06.601 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.592+0000 7f02aa7fc700 1 -- 172.21.15.44:0/4148354558 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f02b000fb00 con 0x7f02b40ff440 2023-09-10T00:42:06.601 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.593+0000 7f02aa7fc700 1 --2- 172.21.15.44:0/4148354558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f02a00a0950 0x7f02a00a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.593+0000 7f02b8ba4700 1 --2- 172.21.15.44:0/4148354558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f02a00a0950 0x7f02a00a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.593+0000 7f02aa7fc700 1 -- 172.21.15.44:0/4148354558 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f02b00cd9e0 con 0x7f02b40ff440 2023-09-10T00:42:06.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.594+0000 7f02b8ba4700 1 --2- 172.21.15.44:0/4148354558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f02a00a0950 0x7f02a00a2e10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f02b40fd900 tx=0x7f02a4009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:06.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.594+0000 7f02c77cfb80 1 Processor -- start 2023-09-10T00:42:06.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.594+0000 7f02c77cfb80 1 -- start start 2023-09-10T00:42:06.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02c77cfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c6710 0x5586997c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02c77cfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 0x5586997d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02c77cfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 0x5586997d3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02c77cfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869971f730 con 0x5586997c6710 2023-09-10T00:42:06.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02c77cfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869971d050 con 0x5586997c7030 2023-09-10T00:42:06.605 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02c77cfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558699718940 con 0x5586997d0e40 2023-09-10T00:42:06.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02b93a5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c6710 0x5586997c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02b93a5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c6710 0x5586997c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50018/0 (socket says 172.21.15.44:50018) 2023-09-10T00:42:06.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02b9ba6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 0x5586997d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02b93a5700 1 -- 172.21.15.44:0/3028865701 learned_addr learned my addr 172.21.15.44:0/3028865701 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:06.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02b8ba4700 1 --2- 172.21.15.44:0/3028865701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 0x5586997d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02b9ba6700 1 -- 172.21.15.44:0/3028865701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 msgr2=0x5586997d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:06.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02b9ba6700 1 --2- 172.21.15.44:0/3028865701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 0x5586997d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02b9ba6700 1 -- 172.21.15.44:0/3028865701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c6710 msgr2=0x5586997c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:06.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02b9ba6700 1 --2- 172.21.15.44:0/3028865701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c6710 0x5586997c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02b9ba6700 1 -- 172.21.15.44:0/3028865701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558699544de0 con 0x5586997d0e40 2023-09-10T00:42:06.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.595+0000 7f02b93a5700 1 --2- 172.21.15.44:0/3028865701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c6710 0x5586997c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:06.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.596+0000 7f02b9ba6700 1 --2- 172.21.15.44:0/3028865701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 0x5586997d3230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009370 tx=0x7f02ac009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9e169619418268a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:06.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.596+0000 7f029f7fe700 1 -- 172.21.15.44:0/3028865701 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac00e070 con 0x5586997d0e40 2023-09-10T00:42:06.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.596+0000 7f029f7fe700 1 -- 172.21.15.44:0/3028865701 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02ac00f040 con 0x5586997d0e40 2023-09-10T00:42:06.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.596+0000 7f029f7fe700 1 -- 172.21.15.44:0/3028865701 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac004420 con 0x5586997d0e40 2023-09-10T00:42:06.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.597+0000 7f02c77cfb80 1 -- 172.21.15.44:0/3028865701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 msgr2=0x5586997d3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:06.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.597+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/3028865701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 0x5586997d3230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009370 tx=0x7f02ac009920 comp rx=0 tx=0).stop 2023-09-10T00:42:06.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.597+0000 7f02c77cfb80 1 -- 172.21.15.44:0/3028865701 shutdown_connections 2023-09-10T00:42:06.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.597+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/3028865701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 0x5586997d3230 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.597+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/3028865701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c6710 0x5586997c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.597+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/3028865701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 0x5586997d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.597+0000 7f02c77cfb80 1 -- 172.21.15.44:0/3028865701 >> 172.21.15.44:0/3028865701 conn(0x55869973b090 msgr2=0x5586997c1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:06.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.597+0000 7f02c77cfb80 1 -- 172.21.15.44:0/3028865701 shutdown_connections 2023-09-10T00:42:06.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.597+0000 7f02c77cfb80 1 -- 172.21.15.44:0/3028865701 wait complete. 2023-09-10T00:42:06.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.597+0000 7f02c77cfb80 1 -- start start 2023-09-10T00:42:06.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.597+0000 7f02c77cfb80 1 Processor -- start 2023-09-10T00:42:06.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02c77cfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c6710 0x5586997d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02c77cfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c7030 0x5586997cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02c77cfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 0x5586997cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02c77cfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586996b6b70 con 0x5586997c7030 2023-09-10T00:42:06.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02c77cfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586996b6870 con 0x5586997c6710 2023-09-10T00:42:06.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02c77cfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869959a920 con 0x5586997d0e40 2023-09-10T00:42:06.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02c77cfb80 10 client.0 _ll_register_callbacks cb 0x558699600a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:42:06.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02b9ba6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 0x5586997cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02b93a5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c6710 0x5586997d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02b9ba6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 0x5586997cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56908/0 (socket says 172.21.15.44:56908) 2023-09-10T00:42:06.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02b9ba6700 1 -- 172.21.15.44:0/2911898156 learned_addr learned my addr 172.21.15.44:0/2911898156 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:06.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02b93a5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c6710 0x5586997d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51764/0 (socket says 172.21.15.44:51764) 2023-09-10T00:42:06.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02b8ba4700 1 --2- 172.21.15.44:0/2911898156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c7030 0x5586997cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02b9ba6700 1 -- 172.21.15.44:0/2911898156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c6710 msgr2=0x5586997d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:06.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02b9ba6700 1 --2- 172.21.15.44:0/2911898156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c6710 0x5586997d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02b9ba6700 1 -- 172.21.15.44:0/2911898156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c7030 msgr2=0x5586997cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:06.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02b9ba6700 1 --2- 172.21.15.44:0/2911898156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c7030 0x5586997cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02b9ba6700 1 -- 172.21.15.44:0/2911898156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b41a5580 con 0x5586997d0e40 2023-09-10T00:42:06.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f02b9ba6700 1 --2- 172.21.15.44:0/2911898156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 0x5586997cdf80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009690 tx=0x7f02ac0096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26ef09a093069521 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:06.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.598+0000 7f029cff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:42:06.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.599+0000 7f029cff9700 1 -- 172.21.15.44:0/2911898156 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac00e030 con 0x5586997d0e40 2023-09-10T00:42:06.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.599+0000 7f029cff9700 1 -- 172.21.15.44:0/2911898156 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f02ac0042d0 con 0x5586997d0e40 2023-09-10T00:42:06.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.599+0000 7f02c77cfb80 1 -- 172.21.15.44:0/2911898156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586996704f0 con 0x5586997d0e40 2023-09-10T00:42:06.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.599+0000 7f029cff9700 1 -- 172.21.15.44:0/2911898156 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02ac004490 con 0x5586997d0e40 2023-09-10T00:42:06.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.599+0000 7f02c77cfb80 10 client.25918 fetch_fsmap learned FSMap version 34 2023-09-10T00:42:06.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.599+0000 7f02c77cfb80 1 -- 172.21.15.44:0/2911898156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f02ac009050 con 0x5586997d0e40 2023-09-10T00:42:06.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.599+0000 7f029cff9700 1 -- 172.21.15.44:0/2911898156 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5586996704f0 con 0x5586997d0e40 2023-09-10T00:42:06.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.599+0000 7f029cff9700 1 -- 172.21.15.44:0/2911898156 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f02ac027b30 con 0x5586997d0e40 2023-09-10T00:42:06.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.599+0000 7f02c77cfb80 10 client.25918 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:42:06.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.599+0000 7f02c77cfb80 10 client.25918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:42:06.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.599+0000 7f02c77cfb80 20 client.25918 populate_metadata read hostname 'smithi044' 2023-09-10T00:42:06.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.599+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/2911898156 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5586997e08a0 0x5586997e2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.599+0000 7f02c77cfb80 4 client.25918 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:42:06.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.599+0000 7f02c77cfb80 1 -- 172.21.15.44:0/2911898156 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55869972d8a0 con 0x5586997e08a0 2023-09-10T00:42:06.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.599+0000 7f02b93a5700 1 --2- 172.21.15.44:0/2911898156 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5586997e08a0 0x5586997e2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.600+0000 7f02b93a5700 1 --2- 172.21.15.44:0/2911898156 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5586997e08a0 0x5586997e2cc0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1550149623e10e6 server_cookie=90f641fc6af5a8ba in_seq=0 out_seq=0 2023-09-10T00:42:06.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.600+0000 7f029cff9700 10 client.25918 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:06.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.601+0000 7f029cff9700 1 -- 172.21.15.44:0/2911898156 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55869972d8a0 con 0x5586997e08a0 2023-09-10T00:42:06.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.601+0000 7f029cff9700 10 client.25918 handle_command_reply: tid=0 2023-09-10T00:42:06.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.601+0000 7f02c77cfb80 1 client.25918 shutdown 2023-09-10T00:42:06.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.601+0000 7f02c77cfb80 1 -- 172.21.15.44:0/2911898156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 msgr2=0x5586997cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:06.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.601+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/2911898156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 0x5586997cdf80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009690 tx=0x7f02ac0096c0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.601+0000 7f02c77cfb80 1 -- 172.21.15.44:0/2911898156 shutdown_connections 2023-09-10T00:42:06.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.601+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/2911898156 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5586997e08a0 0x5586997e2cc0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.601+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/2911898156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 0x5586997cdf80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.601+0000 7f029cff9700 0 client.25918 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:06.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.601+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/2911898156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c6710 0x5586997d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.601+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/2911898156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c7030 0x5586997cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.601+0000 7f02c77cfb80 1 -- 172.21.15.44:0/2911898156 >> 172.21.15.44:0/2911898156 conn(0x55869973b090 msgr2=0x5586997c17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:06.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.601+0000 7f02c77cfb80 1 -- 172.21.15.44:0/2911898156 shutdown_connections 2023-09-10T00:42:06.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.601+0000 7f02c77cfb80 1 -- 172.21.15.44:0/2911898156 wait complete. 2023-09-10T00:42:06.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.601+0000 7f02c77cfb80 20 client.25918 trim_cache size 0 max 16384 2023-09-10T00:42:06.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.618+0000 7f02c77cfb80 1 Processor -- start 2023-09-10T00:42:06.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.618+0000 7f02c77cfb80 1 -- start start 2023-09-10T00:42:06.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02c77cfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997c6710 0x5586997d7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02c77cfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 0x5586997c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02c77cfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997d0e40 0x5586997c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02c77cfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586997c2170 con 0x5586997d0e40 2023-09-10T00:42:06.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02c77cfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586997d7fd0 con 0x5586997c7030 2023-09-10T00:42:06.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02c77cfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586997ce560 con 0x5586997c6710 2023-09-10T00:42:06.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02b9ba6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997d0e40 0x5586997c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02b8ba4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 0x5586997c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02b93a5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997c6710 0x5586997d7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02b8ba4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 0x5586997c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51774/0 (socket says 172.21.15.44:51774) 2023-09-10T00:42:06.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02b9ba6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997d0e40 0x5586997c9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50036/0 (socket says 172.21.15.44:50036) 2023-09-10T00:42:06.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02b8ba4700 1 -- 172.21.15.44:0/2306968402 learned_addr learned my addr 172.21.15.44:0/2306968402 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:06.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02b93a5700 1 -- 172.21.15.44:0/2306968402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 msgr2=0x5586997c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:06.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02b93a5700 1 --2- 172.21.15.44:0/2306968402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 0x5586997c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02b93a5700 1 -- 172.21.15.44:0/2306968402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997d0e40 msgr2=0x5586997c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:06.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02b93a5700 1 --2- 172.21.15.44:0/2306968402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997d0e40 0x5586997c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02b93a5700 1 -- 172.21.15.44:0/2306968402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55869952dc60 con 0x5586997c6710 2023-09-10T00:42:06.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f02b93a5700 1 --2- 172.21.15.44:0/2306968402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997c6710 0x5586997d7a90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f02b009a8c0 tx=0x7f02b00ba040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67ae63a53567ef4a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:06.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.619+0000 7f029ffff700 1 -- 172.21.15.44:0/2306968402 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b00b2ea0 con 0x5586997c6710 2023-09-10T00:42:06.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.620+0000 7f029ffff700 1 -- 172.21.15.44:0/2306968402 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02b009e760 con 0x5586997c6710 2023-09-10T00:42:06.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.620+0000 7f029ffff700 1 -- 172.21.15.44:0/2306968402 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0085040 con 0x5586997c6710 2023-09-10T00:42:06.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.620+0000 7f02c77cfb80 1 -- 172.21.15.44:0/2306968402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997c6710 msgr2=0x5586997d7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:06.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.620+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/2306968402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997c6710 0x5586997d7a90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f02b009a8c0 tx=0x7f02b00ba040 comp rx=0 tx=0).stop 2023-09-10T00:42:06.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.620+0000 7f02c77cfb80 1 -- 172.21.15.44:0/2306968402 shutdown_connections 2023-09-10T00:42:06.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.620+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/2306968402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997d0e40 0x5586997c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.620+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/2306968402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997c6710 0x5586997d7a90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.620+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/2306968402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 0x5586997c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.620+0000 7f02c77cfb80 1 -- 172.21.15.44:0/2306968402 >> 172.21.15.44:0/2306968402 conn(0x55869973b090 msgr2=0x5586997dc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:06.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.620+0000 7f02c77cfb80 1 -- 172.21.15.44:0/2306968402 shutdown_connections 2023-09-10T00:42:06.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.620+0000 7f02c77cfb80 1 -- 172.21.15.44:0/2306968402 wait complete. 2023-09-10T00:42:06.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.621+0000 7f02c77cfb80 1 -- start start 2023-09-10T00:42:06.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.621+0000 7f02c77cfb80 1 Processor -- start 2023-09-10T00:42:06.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.621+0000 7f02c77cfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c6710 0x558699828850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.621+0000 7f02c77cfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 0x558699828d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.621+0000 7f02c77cfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 0x55869982c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.652 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.621+0000 7f02c77cfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869982cc50 con 0x5586997c6710 2023-09-10T00:42:06.652 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.621+0000 7f02c77cfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869982cdc0 con 0x5586997c7030 2023-09-10T00:42:06.652 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.621+0000 7f02c77cfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869982cf80 con 0x5586997d0e40 2023-09-10T00:42:06.652 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.621+0000 7f02c77cfb80 10 client.0 _ll_register_callbacks cb 0x7f02b0005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:42:06.653 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.621+0000 7f02b93a5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c6710 0x558699828850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.653 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.621+0000 7f02b8ba4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 0x558699828d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.653 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.621+0000 7f02b93a5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c6710 0x558699828850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50038/0 (socket says 172.21.15.44:50038) 2023-09-10T00:42:06.654 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.621+0000 7f02b93a5700 1 -- 172.21.15.44:0/805977230 learned_addr learned my addr 172.21.15.44:0/805977230 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:06.654 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.621+0000 7f02b9ba6700 1 --2- 172.21.15.44:0/805977230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 0x55869982c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.654 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.622+0000 7f02b8ba4700 1 -- 172.21.15.44:0/805977230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 msgr2=0x55869982c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:06.654 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.622+0000 7f02b8ba4700 1 --2- 172.21.15.44:0/805977230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 0x55869982c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.655 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.622+0000 7f02b8ba4700 1 -- 172.21.15.44:0/805977230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c6710 msgr2=0x558699828850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:06.655 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.622+0000 7f02b8ba4700 1 --2- 172.21.15.44:0/805977230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c6710 0x558699828850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.655 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.622+0000 7f02b8ba4700 1 -- 172.21.15.44:0/805977230 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b4199280 con 0x5586997c7030 2023-09-10T00:42:06.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.622+0000 7f02b9ba6700 1 --2- 172.21.15.44:0/805977230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 0x55869982c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:06.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.622+0000 7f02b93a5700 1 --2- 172.21.15.44:0/805977230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c6710 0x558699828850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:06.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.622+0000 7f02b8ba4700 1 --2- 172.21.15.44:0/805977230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 0x558699828d90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x5586997ceaa0 tx=0x7f02a4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f729189b7991865 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:06.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.622+0000 7f029cff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:42:06.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.622+0000 7f029cff9700 1 -- 172.21.15.44:0/805977230 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4025070 con 0x5586997c7030 2023-09-10T00:42:06.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.622+0000 7f02c77cfb80 1 -- 172.21.15.44:0/805977230 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02ac00f040 con 0x5586997c7030 2023-09-10T00:42:06.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.622+0000 7f029cff9700 1 -- 172.21.15.44:0/805977230 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f02a4003c90 con 0x5586997c7030 2023-09-10T00:42:06.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.622+0000 7f029cff9700 1 -- 172.21.15.44:0/805977230 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02a4010ed0 con 0x5586997c7030 2023-09-10T00:42:06.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.622+0000 7f02c77cfb80 10 client.25988 fetch_fsmap learned FSMap version 34 2023-09-10T00:42:06.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.622+0000 7f02c77cfb80 1 -- 172.21.15.44:0/805977230 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55869956b250 con 0x5586997c7030 2023-09-10T00:42:06.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.623+0000 7f029cff9700 1 -- 172.21.15.44:0/805977230 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02ac00f040 con 0x5586997c7030 2023-09-10T00:42:06.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.623+0000 7f029cff9700 1 -- 172.21.15.44:0/805977230 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f02a402a420 con 0x5586997c7030 2023-09-10T00:42:06.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.623+0000 7f02c77cfb80 10 client.25988 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:42:06.659 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.623+0000 7f02c77cfb80 10 client.25988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:42:06.659 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.623+0000 7f02c77cfb80 20 client.25988 populate_metadata read hostname 'smithi044' 2023-09-10T00:42:06.659 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.623+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/805977230 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558699835ad0 0x558699837ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:06.659 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.623+0000 7f02c77cfb80 4 client.25988 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:42:06.659 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.623+0000 7f02c77cfb80 1 -- 172.21.15.44:0/805977230 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558699711920 con 0x558699835ad0 2023-09-10T00:42:06.660 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.623+0000 7f02b93a5700 1 --2- 172.21.15.44:0/805977230 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558699835ad0 0x558699837ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:06.660 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.623+0000 7f029cff9700 10 client.25988 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:06.660 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.623+0000 7f02b93a5700 1 --2- 172.21.15.44:0/805977230 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558699835ad0 0x558699837ef0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fbb3108b63592de server_cookie=309ea752bd7c4a84 in_seq=0 out_seq=0 2023-09-10T00:42:06.660 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.624+0000 7f029cff9700 1 -- 172.21.15.44:0/805977230 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558699711920 con 0x558699835ad0 2023-09-10T00:42:06.660 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.624+0000 7f029cff9700 10 client.25988 handle_command_reply: tid=0 2023-09-10T00:42:06.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.624+0000 7f02c77cfb80 1 client.25988 shutdown 2023-09-10T00:42:06.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.624+0000 7f02c77cfb80 1 -- 172.21.15.44:0/805977230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 msgr2=0x558699828d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:06.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.624+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/805977230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 0x558699828d90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x5586997ceaa0 tx=0x7f02a4004750 comp rx=0 tx=0).stop 2023-09-10T00:42:06.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.624+0000 7f02c77cfb80 1 -- 172.21.15.44:0/805977230 shutdown_connections 2023-09-10T00:42:06.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.624+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/805977230 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558699835ad0 0x558699837ef0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.624+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/805977230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586997d0e40 0x55869982c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.662 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.624+0000 7f029cff9700 0 client.25988 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:06.662 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.624+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/805977230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586997c6710 0x558699828850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.662 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.624+0000 7f02c77cfb80 1 --2- 172.21.15.44:0/805977230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586997c7030 0x558699828d90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.662 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.624+0000 7f02c77cfb80 1 -- 172.21.15.44:0/805977230 >> 172.21.15.44:0/805977230 conn(0x55869973b090 msgr2=0x5586997c8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:06.662 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.624+0000 7f02c77cfb80 1 -- 172.21.15.44:0/805977230 shutdown_connections 2023-09-10T00:42:06.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.624+0000 7f02c77cfb80 1 -- 172.21.15.44:0/805977230 wait complete. 2023-09-10T00:42:06.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.624+0000 7f02c77cfb80 20 client.25988 trim_cache size 0 max 16384 2023-09-10T00:42:06.663 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:42:06.664 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:42:06.664 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:42:06.664 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:42:06.664 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:42:06.664 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:42:06.665 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:42:06.665 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:42:06.665 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:42:06.665 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:42:06.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.625+0000 7f02bb609700 1 -- 172.21.15.44:0/4148354558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f02a00a0950 msgr2=0x7f02a00a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:06.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.625+0000 7f02bb609700 1 --2- 172.21.15.44:0/4148354558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f02a00a0950 0x7f02a00a2e10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f02b40fd900 tx=0x7f02a4009260 comp rx=0 tx=0).stop 2023-09-10T00:42:06.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.625+0000 7f02bb609700 1 -- 172.21.15.44:0/4148354558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02b40ff440 msgr2=0x7f02b419e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:06.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.625+0000 7f02bb609700 1 --2- 172.21.15.44:0/4148354558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02b40ff440 0x7f02b419e590 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f02b0005910 tx=0x7f02b00047b0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.625+0000 7f02bb609700 1 -- 172.21.15.44:0/4148354558 shutdown_connections 2023-09-10T00:42:06.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.625+0000 7f02bb609700 1 --2- 172.21.15.44:0/4148354558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f02a00a0950 0x7f02a00a2e10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.625+0000 7f02bb609700 1 --2- 172.21.15.44:0/4148354558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b41054e0 0x7f02b4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.625+0000 7f02bb609700 1 --2- 172.21.15.44:0/4148354558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02b40ff440 0x7f02b419e590 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.625+0000 7f02bb609700 1 --2- 172.21.15.44:0/4148354558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02b40ffe00 0x7f02b419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:06.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.625+0000 7f02bb609700 1 -- 172.21.15.44:0/4148354558 >> 172.21.15.44:0/4148354558 conn(0x7f02b40faf50 msgr2=0x7f02b40fb680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:06.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.625+0000 7f02bb609700 1 -- 172.21.15.44:0/4148354558 shutdown_connections 2023-09-10T00:42:06.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:06.625+0000 7f02bb609700 1 -- 172.21.15.44:0/4148354558 wait complete. 2023-09-10T00:42:06.672 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:42:06.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:06 smithi131 ceph-mon[103713]: pgmap v46: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 47 MiB/s wr, 62 op/s 2023-09-10T00:42:06.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:06 smithi044 ceph-mon[98676]: pgmap v46: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 47 MiB/s wr, 62 op/s 2023-09-10T00:42:06.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:06 smithi099 ceph-mon[102692]: pgmap v46: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 47 MiB/s wr, 62 op/s 2023-09-10T00:42:06.928 INFO:teuthology.orchestra.run.smithi044.stderr:5900000 of 50000000 tuples (11%) done (elapsed 66.56 s, remaining 497.53 s) 2023-09-10T00:42:08.183 INFO:teuthology.orchestra.run.smithi044.stderr:6000000 of 50000000 tuples (12%) done (elapsed 67.82 s, remaining 497.33 s) 2023-09-10T00:42:08.794 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:08 smithi131 ceph-mon[103713]: pgmap v47: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 36 MiB/s wr, 50 op/s 2023-09-10T00:42:08.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:08 smithi044 ceph-mon[98676]: pgmap v47: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 36 MiB/s wr, 50 op/s 2023-09-10T00:42:08.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:08 smithi099 ceph-mon[102692]: pgmap v47: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 36 MiB/s wr, 50 op/s 2023-09-10T00:42:09.448 INFO:teuthology.orchestra.run.smithi044.stderr:6100000 of 50000000 tuples (12%) done (elapsed 69.08 s, remaining 497.17 s) 2023-09-10T00:42:10.337 INFO:teuthology.orchestra.run.smithi044.stderr:6200000 of 50000000 tuples (12%) done (elapsed 69.97 s, remaining 494.32 s) 2023-09-10T00:42:10.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:10 smithi131 ceph-mon[103713]: pgmap v48: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 34 MiB/s wr, 49 op/s 2023-09-10T00:42:10.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:10 smithi044 ceph-mon[98676]: pgmap v48: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 34 MiB/s wr, 49 op/s 2023-09-10T00:42:10.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:10 smithi099 ceph-mon[102692]: pgmap v48: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 34 MiB/s wr, 49 op/s 2023-09-10T00:42:11.648 INFO:teuthology.orchestra.run.smithi044.stderr:6300000 of 50000000 tuples (12%) done (elapsed 71.28 s, remaining 494.45 s) 2023-09-10T00:42:11.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:11 smithi131 ceph-mon[103713]: mgrmap e26: x(active, since 91s), standbys: z, y 2023-09-10T00:42:11.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:11 smithi044 ceph-mon[98676]: mgrmap e26: x(active, since 91s), standbys: z, y 2023-09-10T00:42:11.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:11 smithi099 ceph-mon[102692]: mgrmap e26: x(active, since 91s), standbys: z, y 2023-09-10T00:42:12.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:12 smithi131 ceph-mon[103713]: pgmap v49: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 51 MiB/s wr, 67 op/s 2023-09-10T00:42:12.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:12 smithi044 ceph-mon[98676]: pgmap v49: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 51 MiB/s wr, 67 op/s 2023-09-10T00:42:12.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:12 smithi099 ceph-mon[102692]: pgmap v49: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 51 MiB/s wr, 67 op/s 2023-09-10T00:42:12.911 INFO:teuthology.orchestra.run.smithi044.stderr:6400000 of 50000000 tuples (12%) done (elapsed 72.55 s, remaining 494.21 s) 2023-09-10T00:42:13.792 INFO:teuthology.orchestra.run.smithi044.stderr:6500000 of 50000000 tuples (13%) done (elapsed 73.43 s, remaining 491.39 s) 2023-09-10T00:42:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.278+0000 7fea2f204700 1 -- 172.21.15.44:0/1925691525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea281005d0 msgr2=0x7fea28100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.278+0000 7fea2f204700 1 --2- 172.21.15.44:0/1925691525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea281005d0 0x7fea28100a50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fea140099c0 tx=0x7fea14009cd0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.278+0000 7fea2f204700 1 -- 172.21.15.44:0/1925691525 shutdown_connections 2023-09-10T00:42:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.278+0000 7fea2f204700 1 --2- 172.21.15.44:0/1925691525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea28100f90 0x7fea2810d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.278+0000 7fea2f204700 1 --2- 172.21.15.44:0/1925691525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea28106690 0x7fea28106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.278+0000 7fea2f204700 1 --2- 172.21.15.44:0/1925691525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea281005d0 0x7fea28100a50 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.278+0000 7fea2f204700 1 -- 172.21.15.44:0/1925691525 >> 172.21.15.44:0/1925691525 conn(0x7fea280fc0c0 msgr2=0x7fea280fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.278+0000 7fea2f204700 1 -- 172.21.15.44:0/1925691525 shutdown_connections 2023-09-10T00:42:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.278+0000 7fea2f204700 1 -- 172.21.15.44:0/1925691525 wait complete. 2023-09-10T00:42:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.278+0000 7fea2f204700 1 Processor -- start 2023-09-10T00:42:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.278+0000 7fea2f204700 1 -- start start 2023-09-10T00:42:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2f204700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea281005d0 0x7fea2819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2f204700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea28100f90 0x7fea2819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2f204700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea28106690 0x7fea28196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2f204700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea281969f0 con 0x7fea28106690 2023-09-10T00:42:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2f204700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea28196b60 con 0x7fea281005d0 2023-09-10T00:42:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2f204700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea28196d20 con 0x7fea28100f90 2023-09-10T00:42:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2d7a1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea28106690 0x7fea28196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2d7a1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea28106690 0x7fea28196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50052/0 (socket says 172.21.15.44:50052) 2023-09-10T00:42:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2cfa0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea281005d0 0x7fea2819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2d7a1700 1 -- 172.21.15.44:0/801012893 learned_addr learned my addr 172.21.15.44:0/801012893 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea27fff700 1 --2- 172.21.15.44:0/801012893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea28100f90 0x7fea2819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2d7a1700 1 -- 172.21.15.44:0/801012893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea28100f90 msgr2=0x7fea2819ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2d7a1700 1 --2- 172.21.15.44:0/801012893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea28100f90 0x7fea2819ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2d7a1700 1 -- 172.21.15.44:0/801012893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea281005d0 msgr2=0x7fea2819c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2d7a1700 1 --2- 172.21.15.44:0/801012893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea281005d0 0x7fea2819c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2d7a1700 1 -- 172.21.15.44:0/801012893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea140095f0 con 0x7fea28106690 2023-09-10T00:42:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea27fff700 1 --2- 172.21.15.44:0/801012893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea28100f90 0x7fea2819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2cfa0700 1 --2- 172.21.15.44:0/801012893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea281005d0 0x7fea2819c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.279+0000 7fea2d7a1700 1 --2- 172.21.15.44:0/801012893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea28106690 0x7fea28196380 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fea2000ef30 tx=0x7fea2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.280+0000 7fea25ffb700 1 -- 172.21.15.44:0/801012893 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea20018070 con 0x7fea28106690 2023-09-10T00:42:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.280+0000 7fea25ffb700 1 -- 172.21.15.44:0/801012893 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea2000f040 con 0x7fea28106690 2023-09-10T00:42:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.280+0000 7fea2f204700 1 -- 172.21.15.44:0/801012893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea28196fa0 con 0x7fea28106690 2023-09-10T00:42:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.280+0000 7fea25ffb700 1 -- 172.21.15.44:0/801012893 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2000cbc0 con 0x7fea28106690 2023-09-10T00:42:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.280+0000 7fea2f204700 1 -- 172.21.15.44:0/801012893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea281a3510 con 0x7fea28106690 2023-09-10T00:42:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.282+0000 7fea25ffb700 1 -- 172.21.15.44:0/801012893 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fea2000cd80 con 0x7fea28106690 2023-09-10T00:42:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.283+0000 7fea25ffb700 1 --2- 172.21.15.44:0/801012893 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fea180a0830 0x7fea180a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.283+0000 7fea2cfa0700 1 --2- 172.21.15.44:0/801012893 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fea180a0830 0x7fea180a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.283+0000 7fea25ffb700 1 -- 172.21.15.44:0/801012893 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fea200cbd90 con 0x7fea28106690 2023-09-10T00:42:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.284+0000 7fea2cfa0700 1 --2- 172.21.15.44:0/801012893 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fea180a0830 0x7fea180a2cf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fea1c011490 tx=0x7fea1c009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.284+0000 7fea3b3cab80 1 Processor -- start 2023-09-10T00:42:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.284+0000 7fea3b3cab80 1 -- start start 2023-09-10T00:42:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.284+0000 7fea3b3cab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f2a5b0 0x5631f2f2a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.284+0000 7fea3b3cab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f24710 0x5631f2f24b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.285+0000 7fea3b3cab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f250d0 0x5631f2f31340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.285+0000 7fea3b3cab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f2e796d0 con 0x5631f2f24710 2023-09-10T00:42:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.285+0000 7fea3b3cab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f2e76ff0 con 0x5631f2f2a5b0 2023-09-10T00:42:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.285+0000 7fea3b3cab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f2e728e0 con 0x5631f2f250d0 2023-09-10T00:42:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.285+0000 7fea2cfa0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f2a5b0 0x5631f2f2a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.285+0000 7fea2cfa0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f2a5b0 0x5631f2f2a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51798/0 (socket says 172.21.15.44:51798) 2023-09-10T00:42:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.285+0000 7fea2cfa0700 1 -- 172.21.15.44:0/2797806150 learned_addr learned my addr 172.21.15.44:0/2797806150 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.285+0000 7fea27fff700 1 --2- 172.21.15.44:0/2797806150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f24710 0x5631f2f24b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.285+0000 7fea2d7a1700 1 --2- 172.21.15.44:0/2797806150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f250d0 0x5631f2f31340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.285+0000 7fea2cfa0700 1 -- 172.21.15.44:0/2797806150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f250d0 msgr2=0x5631f2f31340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.285+0000 7fea2cfa0700 1 --2- 172.21.15.44:0/2797806150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f250d0 0x5631f2f31340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.285+0000 7fea2cfa0700 1 -- 172.21.15.44:0/2797806150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f24710 msgr2=0x5631f2f24b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.285+0000 7fea2cfa0700 1 --2- 172.21.15.44:0/2797806150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f24710 0x5631f2f24b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.285+0000 7fea2cfa0700 1 -- 172.21.15.44:0/2797806150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631f2c9ed80 con 0x5631f2f2a5b0 2023-09-10T00:42:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.285+0000 7fea2cfa0700 1 --2- 172.21.15.44:0/2797806150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f2a5b0 0x5631f2f2a990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0043c0 tx=0x7fea1c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc04e4555a448f44 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.285+0000 7fea12ffd700 1 -- 172.21.15.44:0/2797806150 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c003d10 con 0x5631f2f2a5b0 2023-09-10T00:42:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.286+0000 7fea12ffd700 1 -- 172.21.15.44:0/2797806150 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea1c003e70 con 0x5631f2f2a5b0 2023-09-10T00:42:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.286+0000 7fea12ffd700 1 -- 172.21.15.44:0/2797806150 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c01e710 con 0x5631f2f2a5b0 2023-09-10T00:42:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.286+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2797806150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f2a5b0 msgr2=0x5631f2f2a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.286+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/2797806150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f2a5b0 0x5631f2f2a990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0043c0 tx=0x7fea1c014000 comp rx=0 tx=0).stop 2023-09-10T00:42:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.288+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2797806150 shutdown_connections 2023-09-10T00:42:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.288+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/2797806150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f250d0 0x5631f2f31340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.288+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/2797806150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f2a5b0 0x5631f2f2a990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.288+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/2797806150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f24710 0x5631f2f24b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.288+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2797806150 >> 172.21.15.44:0/2797806150 conn(0x5631f2e95030 msgr2=0x5631f2f1b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.288+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2797806150 shutdown_connections 2023-09-10T00:42:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.288+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2797806150 wait complete. 2023-09-10T00:42:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.289+0000 7fea3b3cab80 1 -- start start 2023-09-10T00:42:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.289+0000 7fea3b3cab80 1 Processor -- start 2023-09-10T00:42:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.289+0000 7fea3b3cab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f24710 0x5631f2f31200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.289+0000 7fea3b3cab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f250d0 0x5631f2f2e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.289+0000 7fea3b3cab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f2a5b0 0x5631f2f2ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.289+0000 7fea3b3cab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f2e10b10 con 0x5631f2f2a5b0 2023-09-10T00:42:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.289+0000 7fea3b3cab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f2e10810 con 0x5631f2f250d0 2023-09-10T00:42:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.289+0000 7fea3b3cab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f2cf48c0 con 0x5631f2f24710 2023-09-10T00:42:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.289+0000 7fea3b3cab80 10 client.0 _ll_register_callbacks cb 0x5631f2d5aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:42:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.289+0000 7fea2cfa0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f24710 0x5631f2f31200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.290+0000 7fea2cfa0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f24710 0x5631f2f31200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56946/0 (socket says 172.21.15.44:56946) 2023-09-10T00:42:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.290+0000 7fea2cfa0700 1 -- 172.21.15.44:0/2917933296 learned_addr learned my addr 172.21.15.44:0/2917933296 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.290+0000 7fea27fff700 1 --2- 172.21.15.44:0/2917933296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f250d0 0x5631f2f2e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.290+0000 7fea2cfa0700 1 -- 172.21.15.44:0/2917933296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f250d0 msgr2=0x5631f2f2e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.290+0000 7fea2cfa0700 1 --2- 172.21.15.44:0/2917933296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f250d0 0x5631f2f2e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.290+0000 7fea2cfa0700 1 -- 172.21.15.44:0/2917933296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f2a5b0 msgr2=0x5631f2f2ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.290+0000 7fea2d7a1700 1 --2- 172.21.15.44:0/2917933296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f2a5b0 0x5631f2f2ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.290+0000 7fea2cfa0700 1 --2- 172.21.15.44:0/2917933296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f2a5b0 0x5631f2f2ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.290+0000 7fea2cfa0700 1 -- 172.21.15.44:0/2917933296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1c017070 con 0x5631f2f24710 2023-09-10T00:42:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.290+0000 7fea2d7a1700 1 --2- 172.21.15.44:0/2917933296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f2a5b0 0x5631f2f2ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:42:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.290+0000 7fea2cfa0700 1 --2- 172.21.15.44:0/2917933296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f24710 0x5631f2f31200 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fea1c005920 tx=0x7fea1c026630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b8ac075f699689b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.290+0000 7fea07fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.290+0000 7fea07fff700 1 -- 172.21.15.44:0/2917933296 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c02b070 con 0x5631f2f24710 2023-09-10T00:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.290+0000 7fea07fff700 1 -- 172.21.15.44:0/2917933296 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fea1c026ae0 con 0x5631f2f24710 2023-09-10T00:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.290+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2917933296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631f2dca490 con 0x5631f2f24710 2023-09-10T00:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.291+0000 7fea07fff700 1 -- 172.21.15.44:0/2917933296 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea1c026db0 con 0x5631f2f24710 2023-09-10T00:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.291+0000 7fea3b3cab80 10 client.25936 fetch_fsmap learned FSMap version 34 2023-09-10T00:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.291+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2917933296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5631f2c87c00 con 0x5631f2f24710 2023-09-10T00:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.291+0000 7fea07fff700 1 -- 172.21.15.44:0/2917933296 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5631f2dca490 con 0x5631f2f24710 2023-09-10T00:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.291+0000 7fea07fff700 1 -- 172.21.15.44:0/2917933296 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fea1c024020 con 0x5631f2f24710 2023-09-10T00:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.291+0000 7fea3b3cab80 10 client.25936 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:42:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.291+0000 7fea3b3cab80 10 client.25936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:42:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.291+0000 7fea3b3cab80 20 client.25936 populate_metadata read hostname 'smithi044' 2023-09-10T00:42:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.291+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/2917933296 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5631f2f1fbc0 0x5631f2f1ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.291+0000 7fea3b3cab80 4 client.25936 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:42:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.291+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2917933296 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631f2e87840 con 0x5631f2f1fbc0 2023-09-10T00:42:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.291+0000 7fea27fff700 1 --2- 172.21.15.44:0/2917933296 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5631f2f1fbc0 0x5631f2f1ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.292+0000 7fea07fff700 10 client.25936 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.292+0000 7fea27fff700 1 --2- 172.21.15.44:0/2917933296 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5631f2f1fbc0 0x5631f2f1ffd0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81004f84f2ca19aa server_cookie=605531be711645fa in_seq=0 out_seq=0 2023-09-10T00:42:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.292+0000 7fea07fff700 1 -- 172.21.15.44:0/2917933296 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5631f2e87840 con 0x5631f2f1fbc0 2023-09-10T00:42:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.292+0000 7fea07fff700 10 client.25936 handle_command_reply: tid=0 2023-09-10T00:42:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.292+0000 7fea3b3cab80 1 client.25936 shutdown 2023-09-10T00:42:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.293+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2917933296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f24710 msgr2=0x5631f2f31200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.293+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/2917933296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f24710 0x5631f2f31200 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fea1c005920 tx=0x7fea1c026630 comp rx=0 tx=0).stop 2023-09-10T00:42:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.293+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2917933296 shutdown_connections 2023-09-10T00:42:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.293+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/2917933296 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5631f2f1fbc0 0x5631f2f1ffd0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.293+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/2917933296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f2a5b0 0x5631f2f2ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.293+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/2917933296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f24710 0x5631f2f31200 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.293+0000 7fea07fff700 0 client.25936 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.293+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/2917933296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f250d0 0x5631f2f2e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.293+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2917933296 >> 172.21.15.44:0/2917933296 conn(0x5631f2e95030 msgr2=0x5631f2f1a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.293+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2917933296 shutdown_connections 2023-09-10T00:42:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.293+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2917933296 wait complete. 2023-09-10T00:42:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.293+0000 7fea3b3cab80 20 client.25936 trim_cache size 0 max 16384 2023-09-10T00:42:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea3b3cab80 1 Processor -- start 2023-09-10T00:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea3b3cab80 1 -- start start 2023-09-10T00:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea3b3cab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f24710 0x5631f2f24af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea3b3cab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f250d0 0x5631f2f3a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea3b3cab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f2a5b0 0x5631f2f30cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea3b3cab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f2f29e90 con 0x5631f2f2a5b0 2023-09-10T00:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea3b3cab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f2f2cab0 con 0x5631f2f24710 2023-09-10T00:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea3b3cab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f2f290e0 con 0x5631f2f250d0 2023-09-10T00:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea2cfa0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f24710 0x5631f2f24af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea27fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f250d0 0x5631f2f3a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea2d7a1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f2a5b0 0x5631f2f30cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea2cfa0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f24710 0x5631f2f24af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51812/0 (socket says 172.21.15.44:51812) 2023-09-10T00:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea27fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f250d0 0x5631f2f3a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56956/0 (socket says 172.21.15.44:56956) 2023-09-10T00:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea2cfa0700 1 -- 172.21.15.44:0/1480090290 learned_addr learned my addr 172.21.15.44:0/1480090290 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea27fff700 1 -- 172.21.15.44:0/1480090290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f24710 msgr2=0x5631f2f24af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea27fff700 1 --2- 172.21.15.44:0/1480090290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f24710 0x5631f2f24af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea27fff700 1 -- 172.21.15.44:0/1480090290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f2a5b0 msgr2=0x5631f2f30cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea27fff700 1 --2- 172.21.15.44:0/1480090290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f2a5b0 0x5631f2f30cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.311+0000 7fea27fff700 1 -- 172.21.15.44:0/1480090290 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631f2cc51f0 con 0x5631f2f250d0 2023-09-10T00:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.312+0000 7fea2cfa0700 1 --2- 172.21.15.44:0/1480090290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f24710 0x5631f2f24af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.312+0000 7fea27fff700 1 --2- 172.21.15.44:0/1480090290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f250d0 0x5631f2f3a060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fea14009990 tx=0x7fea14005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6566ec247a98a04f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.312+0000 7fea137fe700 1 -- 172.21.15.44:0/1480090290 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14028da0 con 0x5631f2f250d0 2023-09-10T00:42:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.312+0000 7fea137fe700 1 -- 172.21.15.44:0/1480090290 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea1401e040 con 0x5631f2f250d0 2023-09-10T00:42:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.312+0000 7fea137fe700 1 -- 172.21.15.44:0/1480090290 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14005660 con 0x5631f2f250d0 2023-09-10T00:42:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.313+0000 7fea3b3cab80 1 -- 172.21.15.44:0/1480090290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f250d0 msgr2=0x5631f2f3a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.313+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/1480090290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f250d0 0x5631f2f3a060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fea14009990 tx=0x7fea14005a90 comp rx=0 tx=0).stop 2023-09-10T00:42:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.313+0000 7fea3b3cab80 1 -- 172.21.15.44:0/1480090290 shutdown_connections 2023-09-10T00:42:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.313+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/1480090290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f2a5b0 0x5631f2f30cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.313+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/1480090290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f24710 0x5631f2f24af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.313+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/1480090290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f250d0 0x5631f2f3a060 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.313+0000 7fea3b3cab80 1 -- 172.21.15.44:0/1480090290 >> 172.21.15.44:0/1480090290 conn(0x5631f2e95030 msgr2=0x5631f2f39950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.313+0000 7fea3b3cab80 1 -- 172.21.15.44:0/1480090290 shutdown_connections 2023-09-10T00:42:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.313+0000 7fea3b3cab80 1 -- 172.21.15.44:0/1480090290 wait complete. 2023-09-10T00:42:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.313+0000 7fea3b3cab80 1 -- start start 2023-09-10T00:42:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.313+0000 7fea3b3cab80 1 Processor -- start 2023-09-10T00:42:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea3b3cab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f24710 0x5631f2f46650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea3b3cab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f250d0 0x5631f2f46b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea3b3cab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f2a5b0 0x5631f2f470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea3b3cab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f2f96890 con 0x5631f2f24710 2023-09-10T00:42:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea3b3cab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f2f969d0 con 0x5631f2f250d0 2023-09-10T00:42:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea3b3cab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f2f96b90 con 0x5631f2f2a5b0 2023-09-10T00:42:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea3b3cab80 10 client.0 _ll_register_callbacks cb 0x7fea200097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:42:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea2cfa0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f24710 0x5631f2f46650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea27fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f250d0 0x5631f2f46b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea2d7a1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f2a5b0 0x5631f2f470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea27fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f250d0 0x5631f2f46b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51820/0 (socket says 172.21.15.44:51820) 2023-09-10T00:42:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea2cfa0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f24710 0x5631f2f46650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50078/0 (socket says 172.21.15.44:50078) 2023-09-10T00:42:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea27fff700 1 -- 172.21.15.44:0/2547719195 learned_addr learned my addr 172.21.15.44:0/2547719195 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea2d7a1700 1 -- 172.21.15.44:0/2547719195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f250d0 msgr2=0x5631f2f46b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea2d7a1700 1 --2- 172.21.15.44:0/2547719195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f250d0 0x5631f2f46b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea2d7a1700 1 -- 172.21.15.44:0/2547719195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f24710 msgr2=0x5631f2f46650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea2d7a1700 1 --2- 172.21.15.44:0/2547719195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f24710 0x5631f2f46650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea2d7a1700 1 -- 172.21.15.44:0/2547719195 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631f2c87c00 con 0x5631f2f2a5b0 2023-09-10T00:42:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.314+0000 7fea27fff700 1 --2- 172.21.15.44:0/2547719195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f250d0 0x5631f2f46b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea2d7a1700 1 --2- 172.21.15.44:0/2547719195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f2a5b0 0x5631f2f470d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fea200c9140 tx=0x7fea2000c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=480114bbdded4127 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea07fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:42:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea07fff700 1 -- 172.21.15.44:0/2547719195 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea200b12b0 con 0x5631f2f2a5b0 2023-09-10T00:42:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea07fff700 1 -- 172.21.15.44:0/2547719195 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fea20099630 con 0x5631f2f2a5b0 2023-09-10T00:42:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2547719195 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea1c003e70 con 0x5631f2f2a5b0 2023-09-10T00:42:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea07fff700 1 -- 172.21.15.44:0/2547719195 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea20098f30 con 0x5631f2f2a5b0 2023-09-10T00:42:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea3b3cab80 10 client.25942 fetch_fsmap learned FSMap version 34 2023-09-10T00:42:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2547719195 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fea2810dd50 con 0x5631f2f2a5b0 2023-09-10T00:42:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea07fff700 1 -- 172.21.15.44:0/2547719195 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea1c003e70 con 0x5631f2f2a5b0 2023-09-10T00:42:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea07fff700 1 -- 172.21.15.44:0/2547719195 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fea200a0a80 con 0x5631f2f2a5b0 2023-09-10T00:42:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea3b3cab80 10 client.25942 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:42:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea3b3cab80 10 client.25942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:42:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea3b3cab80 20 client.25942 populate_metadata read hostname 'smithi044' 2023-09-10T00:42:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/2547719195 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5631f2f98300 0x5631f2f9a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea3b3cab80 4 client.25942 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:42:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2547719195 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5631f2e6b8c0 con 0x5631f2f98300 2023-09-10T00:42:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.315+0000 7fea2cfa0700 1 --2- 172.21.15.44:0/2547719195 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5631f2f98300 0x5631f2f9a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.316+0000 7fea2cfa0700 1 --2- 172.21.15.44:0/2547719195 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5631f2f98300 0x5631f2f9a720 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eebd8be80165f016 server_cookie=ef0a2358c9088a0f in_seq=0 out_seq=0 2023-09-10T00:42:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.316+0000 7fea07fff700 10 client.25942 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.331+0000 7fea07fff700 1 -- 172.21.15.44:0/2547719195 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5631f2e6b8c0 con 0x5631f2f98300 2023-09-10T00:42:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.331+0000 7fea07fff700 10 client.25942 handle_command_reply: tid=0 2023-09-10T00:42:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.331+0000 7fea3b3cab80 1 client.25942 shutdown 2023-09-10T00:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.331+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2547719195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f2a5b0 msgr2=0x5631f2f470d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.331+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/2547719195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f2a5b0 0x5631f2f470d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fea200c9140 tx=0x7fea2000c7a0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.331+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2547719195 shutdown_connections 2023-09-10T00:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.331+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/2547719195 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5631f2f98300 0x5631f2f9a720 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.331+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/2547719195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5631f2f2a5b0 0x5631f2f470d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.331+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/2547719195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631f2f24710 0x5631f2f46650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.331+0000 7fea07fff700 0 client.25942 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.331+0000 7fea3b3cab80 1 --2- 172.21.15.44:0/2547719195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631f2f250d0 0x5631f2f46b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.331+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2547719195 >> 172.21.15.44:0/2547719195 conn(0x5631f2e95030 msgr2=0x5631f2f2b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.332+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2547719195 shutdown_connections 2023-09-10T00:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.332+0000 7fea3b3cab80 1 -- 172.21.15.44:0/2547719195 wait complete. 2023-09-10T00:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.332+0000 7fea3b3cab80 20 client.25942 trim_cache size 0 max 16384 2023-09-10T00:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:42:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:42:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.332+0000 7fea2f204700 1 -- 172.21.15.44:0/801012893 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fea180a0830 msgr2=0x7fea180a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.332+0000 7fea2f204700 1 --2- 172.21.15.44:0/801012893 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fea180a0830 0x7fea180a2cf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fea1c011490 tx=0x7fea1c009260 comp rx=0 tx=0).stop 2023-09-10T00:42:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.332+0000 7fea2f204700 1 -- 172.21.15.44:0/801012893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea28106690 msgr2=0x7fea28196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.332+0000 7fea2f204700 1 --2- 172.21.15.44:0/801012893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea28106690 0x7fea28196380 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fea2000ef30 tx=0x7fea2000c3a0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.332+0000 7fea2f204700 1 -- 172.21.15.44:0/801012893 shutdown_connections 2023-09-10T00:42:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.332+0000 7fea2f204700 1 --2- 172.21.15.44:0/801012893 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fea180a0830 0x7fea180a2cf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.332+0000 7fea2f204700 1 --2- 172.21.15.44:0/801012893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea28106690 0x7fea28196380 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.332+0000 7fea2f204700 1 --2- 172.21.15.44:0/801012893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea281005d0 0x7fea2819c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.332+0000 7fea2f204700 1 --2- 172.21.15.44:0/801012893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea28100f90 0x7fea2819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.332+0000 7fea2f204700 1 -- 172.21.15.44:0/801012893 >> 172.21.15.44:0/801012893 conn(0x7fea280fc0c0 msgr2=0x7fea280fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.333+0000 7fea2f204700 1 -- 172.21.15.44:0/801012893 shutdown_connections 2023-09-10T00:42:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:14.333+0000 7fea2f204700 1 -- 172.21.15.44:0/801012893 wait complete. 2023-09-10T00:42:14.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:14 smithi044 ceph-mon[98676]: pgmap v50: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 34 MiB/s wr, 53 op/s 2023-09-10T00:42:14.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:14 smithi099 ceph-mon[102692]: pgmap v50: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 34 MiB/s wr, 53 op/s 2023-09-10T00:42:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:14 smithi131 ceph-mon[103713]: pgmap v50: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 34 MiB/s wr, 53 op/s 2023-09-10T00:42:15.118 INFO:teuthology.orchestra.run.smithi044.stderr:6600000 of 50000000 tuples (13%) done (elapsed 74.75 s, remaining 491.55 s) 2023-09-10T00:42:16.398 INFO:teuthology.orchestra.run.smithi044.stderr:6700000 of 50000000 tuples (13%) done (elapsed 76.03 s, remaining 491.37 s) 2023-09-10T00:42:16.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:16 smithi044 ceph-mon[98676]: pgmap v51: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 46 MiB/s wr, 69 op/s 2023-09-10T00:42:16.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:16 smithi099 ceph-mon[102692]: pgmap v51: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 46 MiB/s wr, 69 op/s 2023-09-10T00:42:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:16 smithi131 ceph-mon[103713]: pgmap v51: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 46 MiB/s wr, 69 op/s 2023-09-10T00:42:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:6800000 of 50000000 tuples (13%) done (elapsed 76.82 s, remaining 488.00 s) 2023-09-10T00:42:18.466 INFO:teuthology.orchestra.run.smithi044.stderr:6900000 of 50000000 tuples (13%) done (elapsed 78.10 s, remaining 487.84 s) 2023-09-10T00:42:18.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:18 smithi044 ceph-mon[98676]: pgmap v52: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 MiB/s wr, 39 op/s 2023-09-10T00:42:18.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:18 smithi099 ceph-mon[102692]: pgmap v52: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 MiB/s wr, 39 op/s 2023-09-10T00:42:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:18 smithi131 ceph-mon[103713]: pgmap v52: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 MiB/s wr, 39 op/s 2023-09-10T00:42:19.383 INFO:teuthology.orchestra.run.smithi044.stderr:7000000 of 50000000 tuples (14%) done (elapsed 79.02 s, remaining 485.39 s) 2023-09-10T00:42:20.667 INFO:teuthology.orchestra.run.smithi044.stderr:7100000 of 50000000 tuples (14%) done (elapsed 80.30 s, remaining 485.19 s) 2023-09-10T00:42:20.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:20 smithi044 ceph-mon[98676]: pgmap v53: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 MiB/s wr, 37 op/s 2023-09-10T00:42:20.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:20 smithi099 ceph-mon[102692]: pgmap v53: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 MiB/s wr, 37 op/s 2023-09-10T00:42:21.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:20 smithi131 ceph-mon[103713]: pgmap v53: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 MiB/s wr, 37 op/s 2023-09-10T00:42:21.926 INFO:teuthology.orchestra.run.smithi044.stderr:7200000 of 50000000 tuples (14%) done (elapsed 81.56 s, remaining 484.83 s) 2023-09-10T00:42:22.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:22 smithi044 ceph-mon[98676]: pgmap v54: 129 pgs: 129 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 MiB/s wr, 52 op/s 2023-09-10T00:42:22.845 INFO:teuthology.orchestra.run.smithi044.stderr:7300000 of 50000000 tuples (14%) done (elapsed 82.48 s, remaining 482.45 s) 2023-09-10T00:42:22.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:22 smithi099 ceph-mon[102692]: pgmap v54: 129 pgs: 129 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 MiB/s wr, 52 op/s 2023-09-10T00:42:23.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:22 smithi131 ceph-mon[103713]: pgmap v54: 129 pgs: 129 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 MiB/s wr, 52 op/s 2023-09-10T00:42:24.126 INFO:teuthology.orchestra.run.smithi044.stderr:7400000 of 50000000 tuples (14%) done (elapsed 83.76 s, remaining 482.19 s) 2023-09-10T00:42:24.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:24 smithi099 ceph-mon[102692]: pgmap v55: 129 pgs: 129 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 MiB/s wr, 33 op/s 2023-09-10T00:42:25.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:24 smithi131 ceph-mon[103713]: pgmap v55: 129 pgs: 129 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 MiB/s wr, 33 op/s 2023-09-10T00:42:25.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:24 smithi044 ceph-mon[98676]: pgmap v55: 129 pgs: 129 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 MiB/s wr, 33 op/s 2023-09-10T00:42:25.324 INFO:teuthology.orchestra.run.smithi044.stderr:7500000 of 50000000 tuples (15%) done (elapsed 84.96 s, remaining 481.43 s) 2023-09-10T00:42:26.249 INFO:teuthology.orchestra.run.smithi044.stderr:7600000 of 50000000 tuples (15%) done (elapsed 85.88 s, remaining 479.14 s) 2023-09-10T00:42:26.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:26 smithi099 ceph-mon[102692]: pgmap v56: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 MiB/s wr, 46 op/s 2023-09-10T00:42:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:26 smithi131 ceph-mon[103713]: pgmap v56: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 MiB/s wr, 46 op/s 2023-09-10T00:42:27.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:26 smithi044 ceph-mon[98676]: pgmap v56: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 MiB/s wr, 46 op/s 2023-09-10T00:42:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.462+0000 7ffb4d63d700 1 -- 172.21.15.44:0/3856115565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb481087d0 msgr2=0x7ffb48108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.462+0000 7ffb4d63d700 1 --2- 172.21.15.44:0/3856115565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb481087d0 0x7ffb48108bb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffb380098f0 tx=0x7ffb38009c00 comp rx=0 tx=0).stop 2023-09-10T00:42:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.462+0000 7ffb4d63d700 1 -- 172.21.15.44:0/3856115565 shutdown_connections 2023-09-10T00:42:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.462+0000 7ffb4d63d700 1 --2- 172.21.15.44:0/3856115565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb48103180 0x7ffb4810f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.462+0000 7ffb4d63d700 1 --2- 172.21.15.44:0/3856115565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb481087d0 0x7ffb48108bb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.462+0000 7ffb4d63d700 1 --2- 172.21.15.44:0/3856115565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb481027c0 0x7ffb48102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.462+0000 7ffb4d63d700 1 -- 172.21.15.44:0/3856115565 >> 172.21.15.44:0/3856115565 conn(0x7ffb480fe2f0 msgr2=0x7ffb48100710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.462+0000 7ffb4d63d700 1 -- 172.21.15.44:0/3856115565 shutdown_connections 2023-09-10T00:42:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.462+0000 7ffb4d63d700 1 -- 172.21.15.44:0/3856115565 wait complete. 2023-09-10T00:42:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.462+0000 7ffb4d63d700 1 Processor -- start 2023-09-10T00:42:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb4d63d700 1 -- start start 2023-09-10T00:42:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb4d63d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb481027c0 0x7ffb4819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb4d63d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb48103180 0x7ffb4819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb4d63d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb481087d0 0x7ffb48198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb4d63d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb48198c80 con 0x7ffb481027c0 2023-09-10T00:42:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb4d63d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb48198df0 con 0x7ffb481087d0 2023-09-10T00:42:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb4d63d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb48198fb0 con 0x7ffb48103180 2023-09-10T00:42:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb46ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb481027c0 0x7ffb4819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb46ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb481027c0 0x7ffb4819e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50092/0 (socket says 172.21.15.44:50092) 2023-09-10T00:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb46ffd700 1 -- 172.21.15.44:0/3010550779 learned_addr learned my addr 172.21.15.44:0/3010550779 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb467fc700 1 --2- 172.21.15.44:0/3010550779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb48103180 0x7ffb4819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb477fe700 1 --2- 172.21.15.44:0/3010550779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb481087d0 0x7ffb48198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb46ffd700 1 -- 172.21.15.44:0/3010550779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb48103180 msgr2=0x7ffb4819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb46ffd700 1 --2- 172.21.15.44:0/3010550779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb48103180 0x7ffb4819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb46ffd700 1 -- 172.21.15.44:0/3010550779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb481087d0 msgr2=0x7ffb48198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb46ffd700 1 --2- 172.21.15.44:0/3010550779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb481087d0 0x7ffb48198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb46ffd700 1 -- 172.21.15.44:0/3010550779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb40009d00 con 0x7ffb481027c0 2023-09-10T00:42:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.463+0000 7ffb467fc700 1 --2- 172.21.15.44:0/3010550779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb48103180 0x7ffb4819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.464+0000 7ffb46ffd700 1 --2- 172.21.15.44:0/3010550779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb481027c0 0x7ffb4819e7b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009230 tx=0x7ffb3800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.464+0000 7ffb2ffff700 1 -- 172.21.15.44:0/3010550779 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3801c070 con 0x7ffb481027c0 2023-09-10T00:42:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.464+0000 7ffb4d63d700 1 -- 172.21.15.44:0/3010550779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb38009520 con 0x7ffb481027c0 2023-09-10T00:42:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.464+0000 7ffb2ffff700 1 -- 172.21.15.44:0/3010550779 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb380058e0 con 0x7ffb481027c0 2023-09-10T00:42:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.464+0000 7ffb2ffff700 1 -- 172.21.15.44:0/3010550779 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38003680 con 0x7ffb481027c0 2023-09-10T00:42:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.464+0000 7ffb4d63d700 1 -- 172.21.15.44:0/3010550779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb48199480 con 0x7ffb481027c0 2023-09-10T00:42:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.467+0000 7ffb2ffff700 1 -- 172.21.15.44:0/3010550779 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ffb38020030 con 0x7ffb481027c0 2023-09-10T00:42:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.467+0000 7ffb2ffff700 1 --2- 172.21.15.44:0/3010550779 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffb300a07b0 0x7ffb300a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.468+0000 7ffb2ffff700 1 -- 172.21.15.44:0/3010550779 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ffb380ccf90 con 0x7ffb481027c0 2023-09-10T00:42:27.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.468+0000 7ffb467fc700 1 --2- 172.21.15.44:0/3010550779 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffb300a07b0 0x7ffb300a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.468+0000 7ffb467fc700 1 --2- 172.21.15.44:0/3010550779 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffb300a07b0 0x7ffb300a2c70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffb48107e80 tx=0x7ffb3c009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:27.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.469+0000 7ffb59803b80 1 Processor -- start 2023-09-10T00:42:27.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.469+0000 7ffb59803b80 1 -- start start 2023-09-10T00:42:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.469+0000 7ffb59803b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559930a011a0 0x559930a01580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.469+0000 7ffb59803b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fb300 0x5599309fb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.469+0000 7ffb59803b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599309fbcc0 0x559930a07f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.469+0000 7ffb59803b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599309546d0 con 0x5599309fbcc0 2023-09-10T00:42:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.469+0000 7ffb59803b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559930951ff0 con 0x559930a011a0 2023-09-10T00:42:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.469+0000 7ffb59803b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993094d8e0 con 0x5599309fb300 2023-09-10T00:42:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.469+0000 7ffb477fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599309fbcc0 0x559930a07f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.469+0000 7ffb46ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559930a011a0 0x559930a01580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.469+0000 7ffb467fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fb300 0x5599309fb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.469+0000 7ffb46ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559930a011a0 0x559930a01580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51840/0 (socket says 172.21.15.44:51840) 2023-09-10T00:42:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.469+0000 7ffb46ffd700 1 -- 172.21.15.44:0/160994008 learned_addr learned my addr 172.21.15.44:0/160994008 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.470+0000 7ffb46ffd700 1 -- 172.21.15.44:0/160994008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fb300 msgr2=0x5599309fb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.470+0000 7ffb46ffd700 1 --2- 172.21.15.44:0/160994008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fb300 0x5599309fb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.470+0000 7ffb46ffd700 1 -- 172.21.15.44:0/160994008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599309fbcc0 msgr2=0x559930a07f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.470+0000 7ffb46ffd700 1 --2- 172.21.15.44:0/160994008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599309fbcc0 0x559930a07f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.470+0000 7ffb46ffd700 1 -- 172.21.15.44:0/160994008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559930779d80 con 0x559930a011a0 2023-09-10T00:42:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.470+0000 7ffb467fc700 1 --2- 172.21.15.44:0/160994008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fb300 0x5599309fb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.470+0000 7ffb477fe700 1 --2- 172.21.15.44:0/160994008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599309fbcc0 0x559930a07f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.470+0000 7ffb46ffd700 1 --2- 172.21.15.44:0/160994008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559930a011a0 0x559930a01580 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb38096630 tx=0x7ffb38009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38c5cc0d2098a1f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.470+0000 7ffb2d7fa700 1 -- 172.21.15.44:0/160994008 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3809a830 con 0x559930a011a0 2023-09-10T00:42:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.470+0000 7ffb2d7fa700 1 -- 172.21.15.44:0/160994008 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb3809c050 con 0x559930a011a0 2023-09-10T00:42:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.470+0000 7ffb2d7fa700 1 -- 172.21.15.44:0/160994008 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38005160 con 0x559930a011a0 2023-09-10T00:42:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.471+0000 7ffb59803b80 1 -- 172.21.15.44:0/160994008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559930a011a0 msgr2=0x559930a01580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.471+0000 7ffb59803b80 1 --2- 172.21.15.44:0/160994008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559930a011a0 0x559930a01580 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb38096630 tx=0x7ffb38009c00 comp rx=0 tx=0).stop 2023-09-10T00:42:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.471+0000 7ffb59803b80 1 -- 172.21.15.44:0/160994008 shutdown_connections 2023-09-10T00:42:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.471+0000 7ffb59803b80 1 --2- 172.21.15.44:0/160994008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599309fbcc0 0x559930a07f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.471+0000 7ffb59803b80 1 --2- 172.21.15.44:0/160994008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559930a011a0 0x559930a01580 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.471+0000 7ffb59803b80 1 --2- 172.21.15.44:0/160994008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fb300 0x5599309fb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.471+0000 7ffb59803b80 1 -- 172.21.15.44:0/160994008 >> 172.21.15.44:0/160994008 conn(0x559930970030 msgr2=0x5599309f6560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.471+0000 7ffb59803b80 1 -- 172.21.15.44:0/160994008 shutdown_connections 2023-09-10T00:42:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.471+0000 7ffb59803b80 1 -- 172.21.15.44:0/160994008 wait complete. 2023-09-10T00:42:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.471+0000 7ffb59803b80 1 -- start start 2023-09-10T00:42:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.471+0000 7ffb59803b80 1 Processor -- start 2023-09-10T00:42:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb59803b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fb300 0x559930a0ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb59803b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fbcc0 0x559930a0b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb59803b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559930a011a0 0x559930a04db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb59803b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599308ebb10 con 0x559930a011a0 2023-09-10T00:42:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb59803b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599308eb810 con 0x5599309fbcc0 2023-09-10T00:42:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb59803b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599307cf8c0 con 0x5599309fb300 2023-09-10T00:42:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb59803b80 10 client.0 _ll_register_callbacks cb 0x559930835a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:42:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb46ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fb300 0x559930a0ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb477fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559930a011a0 0x559930a04db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb46ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fb300 0x559930a0ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:56988/0 (socket says 172.21.15.44:56988) 2023-09-10T00:42:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb46ffd700 1 -- 172.21.15.44:0/3667548278 learned_addr learned my addr 172.21.15.44:0/3667548278 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb467fc700 1 --2- 172.21.15.44:0/3667548278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fbcc0 0x559930a0b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb46ffd700 1 -- 172.21.15.44:0/3667548278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fbcc0 msgr2=0x559930a0b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:27.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb46ffd700 1 --2- 172.21.15.44:0/3667548278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fbcc0 0x559930a0b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb46ffd700 1 -- 172.21.15.44:0/3667548278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559930a011a0 msgr2=0x559930a04db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:27.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb46ffd700 1 --2- 172.21.15.44:0/3667548278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559930a011a0 0x559930a04db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb46ffd700 1 -- 172.21.15.44:0/3667548278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3c017070 con 0x5599309fb300 2023-09-10T00:42:27.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb477fe700 1 --2- 172.21.15.44:0/3667548278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559930a011a0 0x559930a04db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:27.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb46ffd700 1 --2- 172.21.15.44:0/3667548278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fb300 0x559930a0ab20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffb380ca3f0 tx=0x7ffb380986e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3511134513cc9aad server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:27.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.472+0000 7ffb2aff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:42:27.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.473+0000 7ffb2aff5700 1 -- 172.21.15.44:0/3667548278 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3809a830 con 0x5599309fb300 2023-09-10T00:42:27.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.473+0000 7ffb2aff5700 1 -- 172.21.15.44:0/3667548278 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ffb38096840 con 0x5599309fb300 2023-09-10T00:42:27.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.473+0000 7ffb59803b80 1 -- 172.21.15.44:0/3667548278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599308a5490 con 0x5599309fb300 2023-09-10T00:42:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.473+0000 7ffb2aff5700 1 -- 172.21.15.44:0/3667548278 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb38096a00 con 0x5599309fb300 2023-09-10T00:42:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.473+0000 7ffb59803b80 10 client.25954 fetch_fsmap learned FSMap version 34 2023-09-10T00:42:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.473+0000 7ffb59803b80 1 -- 172.21.15.44:0/3667548278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7ffb40009160 con 0x5599309fb300 2023-09-10T00:42:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.473+0000 7ffb2aff5700 1 -- 172.21.15.44:0/3667548278 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5599308a5490 con 0x5599309fb300 2023-09-10T00:42:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.473+0000 7ffb2aff5700 1 -- 172.21.15.44:0/3667548278 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ffb3809a390 con 0x5599309fb300 2023-09-10T00:42:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.473+0000 7ffb59803b80 10 client.25954 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:42:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.473+0000 7ffb59803b80 10 client.25954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:42:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.473+0000 7ffb59803b80 20 client.25954 populate_metadata read hostname 'smithi044' 2023-09-10T00:42:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.473+0000 7ffb59803b80 1 --2- 172.21.15.44:0/3667548278 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559930a14db0 0x559930a171d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.473+0000 7ffb59803b80 4 client.25954 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:42:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.473+0000 7ffb59803b80 1 -- 172.21.15.44:0/3667548278 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559930962840 con 0x559930a14db0 2023-09-10T00:42:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.474+0000 7ffb467fc700 1 --2- 172.21.15.44:0/3667548278 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559930a14db0 0x559930a171d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.474+0000 7ffb467fc700 1 --2- 172.21.15.44:0/3667548278 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559930a14db0 0x559930a171d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66ad9f1b069c805f server_cookie=dc8796e5216c818a in_seq=0 out_seq=0 2023-09-10T00:42:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.474+0000 7ffb2aff5700 10 client.25954 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.475+0000 7ffb2aff5700 1 -- 172.21.15.44:0/3667548278 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559930962840 con 0x559930a14db0 2023-09-10T00:42:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.475+0000 7ffb2aff5700 10 client.25954 handle_command_reply: tid=0 2023-09-10T00:42:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.475+0000 7ffb59803b80 1 client.25954 shutdown 2023-09-10T00:42:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.475+0000 7ffb59803b80 1 -- 172.21.15.44:0/3667548278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fb300 msgr2=0x559930a0ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.475+0000 7ffb59803b80 1 --2- 172.21.15.44:0/3667548278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fb300 0x559930a0ab20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffb380ca3f0 tx=0x7ffb380986e0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.475+0000 7ffb59803b80 1 -- 172.21.15.44:0/3667548278 shutdown_connections 2023-09-10T00:42:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.475+0000 7ffb59803b80 1 --2- 172.21.15.44:0/3667548278 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559930a14db0 0x559930a171d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.475+0000 7ffb59803b80 1 --2- 172.21.15.44:0/3667548278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559930a011a0 0x559930a04db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.475+0000 7ffb59803b80 1 --2- 172.21.15.44:0/3667548278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fb300 0x559930a0ab20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.475+0000 7ffb59803b80 1 --2- 172.21.15.44:0/3667548278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fbcc0 0x559930a0b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.475+0000 7ffb2aff5700 0 client.25954 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.475+0000 7ffb59803b80 1 -- 172.21.15.44:0/3667548278 >> 172.21.15.44:0/3667548278 conn(0x559930970030 msgr2=0x5599309f57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.475+0000 7ffb59803b80 1 -- 172.21.15.44:0/3667548278 shutdown_connections 2023-09-10T00:42:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.475+0000 7ffb59803b80 1 -- 172.21.15.44:0/3667548278 wait complete. 2023-09-10T00:42:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.475+0000 7ffb59803b80 20 client.25954 trim_cache size 0 max 16384 2023-09-10T00:42:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.497+0000 7ffb59803b80 1 Processor -- start 2023-09-10T00:42:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.497+0000 7ffb59803b80 1 -- start start 2023-09-10T00:42:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.497+0000 7ffb59803b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599309fb300 0x559930a00700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.497+0000 7ffb59803b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fbcc0 0x559930a0a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.497+0000 7ffb59803b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559930a011a0 0x559930a0ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.497+0000 7ffb59803b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559930962840 con 0x5599309fb300 2023-09-10T00:42:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.497+0000 7ffb59803b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599308a5490 con 0x5599309fbcc0 2023-09-10T00:42:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.497+0000 7ffb59803b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559930a01970 con 0x559930a011a0 2023-09-10T00:42:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.498+0000 7ffb46ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599309fb300 0x559930a00700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.498+0000 7ffb467fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fbcc0 0x559930a0a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.498+0000 7ffb467fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fbcc0 0x559930a0a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51856/0 (socket says 172.21.15.44:51856) 2023-09-10T00:42:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.498+0000 7ffb467fc700 1 -- 172.21.15.44:0/4144534920 learned_addr learned my addr 172.21.15.44:0/4144534920 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.498+0000 7ffb477fe700 1 --2- 172.21.15.44:0/4144534920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559930a011a0 0x559930a0ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.498+0000 7ffb467fc700 1 -- 172.21.15.44:0/4144534920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559930a011a0 msgr2=0x559930a0ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.498+0000 7ffb467fc700 1 --2- 172.21.15.44:0/4144534920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559930a011a0 0x559930a0ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.498+0000 7ffb467fc700 1 -- 172.21.15.44:0/4144534920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599309fb300 msgr2=0x559930a00700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.498+0000 7ffb467fc700 1 --2- 172.21.15.44:0/4144534920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599309fb300 0x559930a00700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.498+0000 7ffb467fc700 1 -- 172.21.15.44:0/4144534920 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb38009520 con 0x5599309fbcc0 2023-09-10T00:42:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.498+0000 7ffb46ffd700 1 --2- 172.21.15.44:0/4144534920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599309fb300 0x559930a00700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.498+0000 7ffb467fc700 1 --2- 172.21.15.44:0/4144534920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fbcc0 0x559930a0a6f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c013010 tx=0x7ffb3c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c4fe87d1311cb79 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.498+0000 7ffb2dffb700 1 -- 172.21.15.44:0/4144534920 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c024040 con 0x5599309fbcc0 2023-09-10T00:42:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.498+0000 7ffb2dffb700 1 -- 172.21.15.44:0/4144534920 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb3c003db0 con 0x5599309fbcc0 2023-09-10T00:42:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.498+0000 7ffb2dffb700 1 -- 172.21.15.44:0/4144534920 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c01e770 con 0x5599309fbcc0 2023-09-10T00:42:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.499+0000 7ffb59803b80 1 -- 172.21.15.44:0/4144534920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fbcc0 msgr2=0x559930a0a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.499+0000 7ffb59803b80 1 --2- 172.21.15.44:0/4144534920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fbcc0 0x559930a0a6f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c013010 tx=0x7ffb3c014000 comp rx=0 tx=0).stop 2023-09-10T00:42:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.499+0000 7ffb59803b80 1 -- 172.21.15.44:0/4144534920 shutdown_connections 2023-09-10T00:42:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.499+0000 7ffb59803b80 1 --2- 172.21.15.44:0/4144534920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559930a011a0 0x559930a0ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.499+0000 7ffb59803b80 1 --2- 172.21.15.44:0/4144534920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599309fb300 0x559930a00700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.499+0000 7ffb59803b80 1 --2- 172.21.15.44:0/4144534920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fbcc0 0x559930a0a6f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.499+0000 7ffb59803b80 1 -- 172.21.15.44:0/4144534920 >> 172.21.15.44:0/4144534920 conn(0x559930970030 msgr2=0x5599309ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.499+0000 7ffb59803b80 1 -- 172.21.15.44:0/4144534920 shutdown_connections 2023-09-10T00:42:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.499+0000 7ffb59803b80 1 -- 172.21.15.44:0/4144534920 wait complete. 2023-09-10T00:42:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.499+0000 7ffb59803b80 1 -- start start 2023-09-10T00:42:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.500+0000 7ffb59803b80 1 Processor -- start 2023-09-10T00:42:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.500+0000 7ffb59803b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fb300 0x559930a61c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.500+0000 7ffb59803b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fbcc0 0x559930a62140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.500+0000 7ffb59803b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559930a011a0 0x559930a5bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.500+0000 7ffb59803b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559930a5c540 con 0x559930a011a0 2023-09-10T00:42:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.500+0000 7ffb59803b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559930a5c6b0 con 0x5599309fb300 2023-09-10T00:42:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.500+0000 7ffb59803b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559930a5c870 con 0x5599309fbcc0 2023-09-10T00:42:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.500+0000 7ffb59803b80 10 client.0 _ll_register_callbacks cb 0x7ffb40009050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:42:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.500+0000 7ffb46ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fb300 0x559930a61c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.500+0000 7ffb467fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fbcc0 0x559930a62140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.500+0000 7ffb477fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559930a011a0 0x559930a5bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.500+0000 7ffb46ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fb300 0x559930a61c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51860/0 (socket says 172.21.15.44:51860) 2023-09-10T00:42:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.500+0000 7ffb46ffd700 1 -- 172.21.15.44:0/3190046390 learned_addr learned my addr 172.21.15.44:0/3190046390 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.500+0000 7ffb467fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fbcc0 0x559930a62140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57004/0 (socket says 172.21.15.44:57004) 2023-09-10T00:42:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.500+0000 7ffb46ffd700 1 -- 172.21.15.44:0/3190046390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fbcc0 msgr2=0x559930a62140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.501+0000 7ffb46ffd700 1 --2- 172.21.15.44:0/3190046390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fbcc0 0x559930a62140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.501+0000 7ffb46ffd700 1 -- 172.21.15.44:0/3190046390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559930a011a0 msgr2=0x559930a5bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.501+0000 7ffb46ffd700 1 --2- 172.21.15.44:0/3190046390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559930a011a0 0x559930a5bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.501+0000 7ffb46ffd700 1 -- 172.21.15.44:0/3190046390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb40009160 con 0x5599309fb300 2023-09-10T00:42:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.501+0000 7ffb467fc700 1 --2- 172.21.15.44:0/3190046390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fbcc0 0x559930a62140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.501+0000 7ffb477fe700 1 --2- 172.21.15.44:0/3190046390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559930a011a0 0x559930a5bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.501+0000 7ffb46ffd700 1 --2- 172.21.15.44:0/3190046390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fb300 0x559930a61c00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffb380098c0 tx=0x7ffb38005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b980304237c2c631 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.501+0000 7ffb2aff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:42:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.501+0000 7ffb2aff5700 1 -- 172.21.15.44:0/3190046390 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3809a590 con 0x5599309fb300 2023-09-10T00:42:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.501+0000 7ffb59803b80 1 -- 172.21.15.44:0/3190046390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb3809c050 con 0x5599309fb300 2023-09-10T00:42:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.501+0000 7ffb2aff5700 1 -- 172.21.15.44:0/3190046390 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ffb3809a830 con 0x5599309fb300 2023-09-10T00:42:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.501+0000 7ffb2aff5700 1 -- 172.21.15.44:0/3190046390 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb38096470 con 0x5599309fb300 2023-09-10T00:42:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.501+0000 7ffb59803b80 10 client.26024 fetch_fsmap learned FSMap version 34 2023-09-10T00:42:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.501+0000 7ffb59803b80 1 -- 172.21.15.44:0/3190046390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7ffb3c0042b0 con 0x5599309fb300 2023-09-10T00:42:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.501+0000 7ffb2aff5700 1 -- 172.21.15.44:0/3190046390 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb3809c050 con 0x5599309fb300 2023-09-10T00:42:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.501+0000 7ffb2aff5700 1 -- 172.21.15.44:0/3190046390 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ffb380b2220 con 0x5599309fb300 2023-09-10T00:42:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.502+0000 7ffb59803b80 10 client.26024 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:42:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.502+0000 7ffb59803b80 10 client.26024 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:42:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.502+0000 7ffb59803b80 20 client.26024 populate_metadata read hostname 'smithi044' 2023-09-10T00:42:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.502+0000 7ffb59803b80 1 --2- 172.21.15.44:0/3190046390 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559930a69670 0x559930a6ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.502+0000 7ffb59803b80 4 client.26024 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:42:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.502+0000 7ffb59803b80 1 -- 172.21.15.44:0/3190046390 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5599309468c0 con 0x559930a69670 2023-09-10T00:42:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.502+0000 7ffb467fc700 1 --2- 172.21.15.44:0/3190046390 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559930a69670 0x559930a6ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.502+0000 7ffb2aff5700 10 client.26024 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.502+0000 7ffb467fc700 1 --2- 172.21.15.44:0/3190046390 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559930a69670 0x559930a6ba90 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d97759c6f30525 server_cookie=981f9d0426bca5 in_seq=0 out_seq=0 2023-09-10T00:42:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.515+0000 7ffb2aff5700 1 -- 172.21.15.44:0/3190046390 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5599309468c0 con 0x559930a69670 2023-09-10T00:42:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.515+0000 7ffb2aff5700 10 client.26024 handle_command_reply: tid=0 2023-09-10T00:42:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.515+0000 7ffb59803b80 1 client.26024 shutdown 2023-09-10T00:42:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.515+0000 7ffb59803b80 1 -- 172.21.15.44:0/3190046390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fb300 msgr2=0x559930a61c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.515+0000 7ffb59803b80 1 --2- 172.21.15.44:0/3190046390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fb300 0x559930a61c00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffb380098c0 tx=0x7ffb38005850 comp rx=0 tx=0).stop 2023-09-10T00:42:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.515+0000 7ffb59803b80 1 -- 172.21.15.44:0/3190046390 shutdown_connections 2023-09-10T00:42:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.515+0000 7ffb59803b80 1 --2- 172.21.15.44:0/3190046390 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559930a69670 0x559930a6ba90 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.515+0000 7ffb59803b80 1 --2- 172.21.15.44:0/3190046390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559930a011a0 0x559930a5bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.515+0000 7ffb2aff5700 0 client.26024 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.515+0000 7ffb59803b80 1 --2- 172.21.15.44:0/3190046390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599309fb300 0x559930a61c00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.515+0000 7ffb59803b80 1 --2- 172.21.15.44:0/3190046390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599309fbcc0 0x559930a62140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.515+0000 7ffb59803b80 1 -- 172.21.15.44:0/3190046390 >> 172.21.15.44:0/3190046390 conn(0x559930970030 msgr2=0x559930a01e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.515+0000 7ffb59803b80 1 -- 172.21.15.44:0/3190046390 shutdown_connections 2023-09-10T00:42:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.515+0000 7ffb59803b80 1 -- 172.21.15.44:0/3190046390 wait complete. 2023-09-10T00:42:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.516+0000 7ffb59803b80 20 client.26024 trim_cache size 0 max 16384 2023-09-10T00:42:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:42:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:42:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:42:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:42:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.516+0000 7ffb4d63d700 1 -- 172.21.15.44:0/3010550779 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffb300a07b0 msgr2=0x7ffb300a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.516+0000 7ffb4d63d700 1 --2- 172.21.15.44:0/3010550779 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffb300a07b0 0x7ffb300a2c70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffb48107e80 tx=0x7ffb3c009260 comp rx=0 tx=0).stop 2023-09-10T00:42:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.516+0000 7ffb4d63d700 1 -- 172.21.15.44:0/3010550779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb481027c0 msgr2=0x7ffb4819e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.516+0000 7ffb4d63d700 1 --2- 172.21.15.44:0/3010550779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb481027c0 0x7ffb4819e7b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009230 tx=0x7ffb3800fa00 comp rx=0 tx=0).stop 2023-09-10T00:42:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.516+0000 7ffb4d63d700 1 -- 172.21.15.44:0/3010550779 shutdown_connections 2023-09-10T00:42:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.516+0000 7ffb4d63d700 1 --2- 172.21.15.44:0/3010550779 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffb300a07b0 0x7ffb300a2c70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.516+0000 7ffb4d63d700 1 --2- 172.21.15.44:0/3010550779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb481087d0 0x7ffb48198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.516+0000 7ffb4d63d700 1 --2- 172.21.15.44:0/3010550779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb481027c0 0x7ffb4819e7b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.516+0000 7ffb4d63d700 1 --2- 172.21.15.44:0/3010550779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb48103180 0x7ffb4819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.516+0000 7ffb4d63d700 1 -- 172.21.15.44:0/3010550779 >> 172.21.15.44:0/3010550779 conn(0x7ffb480fe2f0 msgr2=0x7ffb480fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.516+0000 7ffb4d63d700 1 -- 172.21.15.44:0/3010550779 shutdown_connections 2023-09-10T00:42:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:27.516+0000 7ffb4d63d700 1 -- 172.21.15.44:0/3010550779 wait complete. 2023-09-10T00:42:27.546 INFO:teuthology.orchestra.run.smithi044.stderr:7700000 of 50000000 tuples (15%) done (elapsed 87.18 s, remaining 478.92 s) 2023-09-10T00:42:28.814 INFO:teuthology.orchestra.run.smithi044.stderr:7800000 of 50000000 tuples (15%) done (elapsed 88.45 s, remaining 478.53 s) 2023-09-10T00:42:28.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:28 smithi099 ceph-mon[102692]: pgmap v57: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 39 MiB/s wr, 31 op/s 2023-09-10T00:42:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:28 smithi131 ceph-mon[103713]: pgmap v57: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 39 MiB/s wr, 31 op/s 2023-09-10T00:42:29.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:28 smithi044 ceph-mon[98676]: pgmap v57: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 39 MiB/s wr, 31 op/s 2023-09-10T00:42:29.665 INFO:teuthology.orchestra.run.smithi044.stderr:7900000 of 50000000 tuples (15%) done (elapsed 89.30 s, remaining 475.89 s) 2023-09-10T00:42:30.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:30 smithi099 ceph-mon[102692]: pgmap v58: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 29 op/s 2023-09-10T00:42:30.989 INFO:teuthology.orchestra.run.smithi044.stderr:8000000 of 50000000 tuples (16%) done (elapsed 90.62 s, remaining 475.77 s) 2023-09-10T00:42:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:30 smithi131 ceph-mon[103713]: pgmap v58: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 29 op/s 2023-09-10T00:42:31.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:30 smithi044 ceph-mon[98676]: pgmap v58: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 29 op/s 2023-09-10T00:42:31.874 INFO:teuthology.orchestra.run.smithi044.stderr:8100000 of 50000000 tuples (16%) done (elapsed 91.51 s, remaining 473.36 s) 2023-09-10T00:42:32.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:32 smithi099 ceph-mon[102692]: pgmap v59: 129 pgs: 129 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 MiB/s wr, 46 op/s 2023-09-10T00:42:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:32 smithi131 ceph-mon[103713]: pgmap v59: 129 pgs: 129 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 MiB/s wr, 46 op/s 2023-09-10T00:42:33.052 INFO:teuthology.orchestra.run.smithi044.stderr:8200000 of 50000000 tuples (16%) done (elapsed 92.69 s, remaining 472.48 s) 2023-09-10T00:42:33.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:32 smithi044 ceph-mon[98676]: pgmap v59: 129 pgs: 129 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 MiB/s wr, 46 op/s 2023-09-10T00:42:34.310 INFO:teuthology.orchestra.run.smithi044.stderr:8300000 of 50000000 tuples (16%) done (elapsed 93.94 s, remaining 471.99 s) 2023-09-10T00:42:34.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:34 smithi099 ceph-mon[102692]: pgmap v60: 129 pgs: 129 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 MiB/s wr, 31 op/s 2023-09-10T00:42:35.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:34 smithi131 ceph-mon[103713]: pgmap v60: 129 pgs: 129 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 MiB/s wr, 31 op/s 2023-09-10T00:42:35.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:34 smithi044 ceph-mon[98676]: pgmap v60: 129 pgs: 129 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 MiB/s wr, 31 op/s 2023-09-10T00:42:35.217 INFO:teuthology.orchestra.run.smithi044.stderr:8400000 of 50000000 tuples (16%) done (elapsed 94.85 s, remaining 469.74 s) 2023-09-10T00:42:36.496 INFO:teuthology.orchestra.run.smithi044.stderr:8500000 of 50000000 tuples (17%) done (elapsed 96.13 s, remaining 469.34 s) 2023-09-10T00:42:36.673 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:42:36.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.819+0000 7f73d060e700 1 -- 172.21.15.44:0/229401801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c80ff440 msgr2=0x7f73c80ff8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:36.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.819+0000 7f73d060e700 1 --2- 172.21.15.44:0/229401801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c80ff440 0x7f73c80ff8c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009990 tx=0x7f73b8009ca0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.819+0000 7f73d060e700 1 -- 172.21.15.44:0/229401801 shutdown_connections 2023-09-10T00:42:36.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.819+0000 7f73d060e700 1 --2- 172.21.15.44:0/229401801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73c80ffe00 0x7f73c810c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.819+0000 7f73d060e700 1 --2- 172.21.15.44:0/229401801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c81054e0 0x7f73c81058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.819+0000 7f73d060e700 1 --2- 172.21.15.44:0/229401801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c80ff440 0x7f73c80ff8c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.819+0000 7f73d060e700 1 -- 172.21.15.44:0/229401801 >> 172.21.15.44:0/229401801 conn(0x7f73c80faf50 msgr2=0x7f73c80fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:36.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.819+0000 7f73d060e700 1 -- 172.21.15.44:0/229401801 shutdown_connections 2023-09-10T00:42:36.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.819+0000 7f73d060e700 1 -- 172.21.15.44:0/229401801 wait complete. 2023-09-10T00:42:36.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73d060e700 1 Processor -- start 2023-09-10T00:42:36.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73d060e700 1 -- start start 2023-09-10T00:42:36.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73d060e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c80ff440 0x7f73c81951e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73d060e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73c80ffe00 0x7f73c8195720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73d060e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c81054e0 0x7f73c8195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73d060e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c81a49d0 con 0x7f73c80ff440 2023-09-10T00:42:36.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73d060e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c81a4b40 con 0x7f73c81054e0 2023-09-10T00:42:36.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73d060e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c81a4d00 con 0x7f73c80ffe00 2023-09-10T00:42:36.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73ce3aa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c80ff440 0x7f73c81951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73cdba9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73c80ffe00 0x7f73c8195720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73ce3aa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c80ff440 0x7f73c81951e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50134/0 (socket says 172.21.15.44:50134) 2023-09-10T00:42:36.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73ce3aa700 1 -- 172.21.15.44:0/2670135988 learned_addr learned my addr 172.21.15.44:0/2670135988 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:36.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73cebab700 1 --2- 172.21.15.44:0/2670135988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c81054e0 0x7f73c8195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73cdba9700 1 -- 172.21.15.44:0/2670135988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c81054e0 msgr2=0x7f73c8195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:36.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73cdba9700 1 --2- 172.21.15.44:0/2670135988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c81054e0 0x7f73c8195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73cdba9700 1 -- 172.21.15.44:0/2670135988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c80ff440 msgr2=0x7f73c81951e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:36.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73cdba9700 1 --2- 172.21.15.44:0/2670135988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c80ff440 0x7f73c81951e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73cdba9700 1 -- 172.21.15.44:0/2670135988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b80095f0 con 0x7f73c80ffe00 2023-09-10T00:42:36.832 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.820+0000 7f73ce3aa700 1 --2- 172.21.15.44:0/2670135988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c80ff440 0x7f73c81951e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:36.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.821+0000 7f73cdba9700 1 --2- 172.21.15.44:0/2670135988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73c80ffe00 0x7f73c8195720 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f73b8005910 tx=0x7f73b8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:36.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.821+0000 7f73bf7fe700 1 -- 172.21.15.44:0/2670135988 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b801d070 con 0x7f73c80ffe00 2023-09-10T00:42:36.833 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.821+0000 7f73bf7fe700 1 -- 172.21.15.44:0/2670135988 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f73b8022470 con 0x7f73c80ffe00 2023-09-10T00:42:36.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.821+0000 7f73d060e700 1 -- 172.21.15.44:0/2670135988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c81a4f80 con 0x7f73c80ffe00 2023-09-10T00:42:36.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.821+0000 7f73bf7fe700 1 -- 172.21.15.44:0/2670135988 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b8017650 con 0x7f73c80ffe00 2023-09-10T00:42:36.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.821+0000 7f73d060e700 1 -- 172.21.15.44:0/2670135988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c81a5400 con 0x7f73c80ffe00 2023-09-10T00:42:36.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.824+0000 7f73bf7fe700 1 -- 172.21.15.44:0/2670135988 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f73b80225e0 con 0x7f73c80ffe00 2023-09-10T00:42:36.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.824+0000 7f73bf7fe700 1 --2- 172.21.15.44:0/2670135988 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f73b40a07b0 0x7f73b40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.825+0000 7f73ce3aa700 1 --2- 172.21.15.44:0/2670135988 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f73b40a07b0 0x7f73b40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.825+0000 7f73bf7fe700 1 -- 172.21.15.44:0/2670135988 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f73b80cd1d0 con 0x7f73c80ffe00 2023-09-10T00:42:36.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.825+0000 7f73dc7d4b80 1 Processor -- start 2023-09-10T00:42:36.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.825+0000 7f73ce3aa700 1 --2- 172.21.15.44:0/2670135988 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f73b40a07b0 0x7f73b40a2c70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f73c4011550 tx=0x7f73c4009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:36.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.825+0000 7f73dc7d4b80 1 -- start start 2023-09-10T00:42:36.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.825+0000 7f73dc7d4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 0x55f6ee26caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73dc7d4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26d030 0x55f6ee276900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73dc7d4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 0x55f6ee279230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73dc7d4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee1c5730 con 0x55f6ee26c710 2023-09-10T00:42:36.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73dc7d4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee1c3050 con 0x55f6ee276e40 2023-09-10T00:42:36.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73dc7d4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee1be940 con 0x55f6ee26d030 2023-09-10T00:42:36.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73ce3aa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 0x55f6ee26caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73cebab700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 0x55f6ee279230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73cdba9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26d030 0x55f6ee276900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73cebab700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 0x55f6ee279230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51886/0 (socket says 172.21.15.44:51886) 2023-09-10T00:42:36.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73cebab700 1 -- 172.21.15.44:0/3481260554 learned_addr learned my addr 172.21.15.44:0/3481260554 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:36.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73cebab700 1 -- 172.21.15.44:0/3481260554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26d030 msgr2=0x55f6ee276900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:36.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73cebab700 1 --2- 172.21.15.44:0/3481260554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26d030 0x55f6ee276900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73cebab700 1 -- 172.21.15.44:0/3481260554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 msgr2=0x55f6ee26caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:36.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73cebab700 1 --2- 172.21.15.44:0/3481260554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 0x55f6ee26caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73cebab700 1 -- 172.21.15.44:0/3481260554 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6edfeade0 con 0x55f6ee276e40 2023-09-10T00:42:36.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73cdba9700 1 --2- 172.21.15.44:0/3481260554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26d030 0x55f6ee276900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:36.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73cebab700 1 --2- 172.21.15.44:0/3481260554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 0x55f6ee279230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f73c0011680 tx=0x7f73c0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=641ab5409a7a1512 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:36.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.826+0000 7f73bcff9700 1 -- 172.21.15.44:0/3481260554 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c000e040 con 0x55f6ee276e40 2023-09-10T00:42:36.846 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.827+0000 7f73bcff9700 1 -- 172.21.15.44:0/3481260554 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f73c000f040 con 0x55f6ee276e40 2023-09-10T00:42:36.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.827+0000 7f73bcff9700 1 -- 172.21.15.44:0/3481260554 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c0004730 con 0x55f6ee276e40 2023-09-10T00:42:36.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.827+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/3481260554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 msgr2=0x55f6ee279230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:36.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.827+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/3481260554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 0x55f6ee279230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f73c0011680 tx=0x7f73c0009920 comp rx=0 tx=0).stop 2023-09-10T00:42:36.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.827+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/3481260554 shutdown_connections 2023-09-10T00:42:36.847 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.827+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/3481260554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 0x55f6ee279230 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.827+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/3481260554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 0x55f6ee26caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.827+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/3481260554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26d030 0x55f6ee276900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.827+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/3481260554 >> 172.21.15.44:0/3481260554 conn(0x55f6ee1e1090 msgr2=0x55f6ee267bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:36.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.827+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/3481260554 shutdown_connections 2023-09-10T00:42:36.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.828+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/3481260554 wait complete. 2023-09-10T00:42:36.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.828+0000 7f73dc7d4b80 1 -- start start 2023-09-10T00:42:36.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.828+0000 7f73dc7d4b80 1 Processor -- start 2023-09-10T00:42:36.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.828+0000 7f73dc7d4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26c710 0x55f6ee276700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.828+0000 7f73dc7d4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26d030 0x55f6ee273980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.828+0000 7f73dc7d4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 0x55f6ee273ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.828+0000 7f73dc7d4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee15cb70 con 0x55f6ee26d030 2023-09-10T00:42:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.828+0000 7f73dc7d4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee15c870 con 0x55f6ee276e40 2023-09-10T00:42:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.828+0000 7f73dc7d4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee040920 con 0x55f6ee26c710 2023-09-10T00:42:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.828+0000 7f73cdba9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26d030 0x55f6ee273980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.828+0000 7f73ce3aa700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26c710 0x55f6ee276700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.828+0000 7f73dc7d4b80 10 client.0 _ll_register_callbacks cb 0x55f6ee0a6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:42:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.828+0000 7f73cdba9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26d030 0x55f6ee273980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50150/0 (socket says 172.21.15.44:50150) 2023-09-10T00:42:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.828+0000 7f73cdba9700 1 -- 172.21.15.44:0/2707531526 learned_addr learned my addr 172.21.15.44:0/2707531526 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:36.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.829+0000 7f73cebab700 1 --2- 172.21.15.44:0/2707531526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 0x55f6ee273ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.829+0000 7f73ce3aa700 1 -- 172.21.15.44:0/2707531526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 msgr2=0x55f6ee273ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:36.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.829+0000 7f73ce3aa700 1 --2- 172.21.15.44:0/2707531526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 0x55f6ee273ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.829+0000 7f73ce3aa700 1 -- 172.21.15.44:0/2707531526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26d030 msgr2=0x55f6ee273980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:36.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.829+0000 7f73ce3aa700 1 --2- 172.21.15.44:0/2707531526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26d030 0x55f6ee273980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.829+0000 7f73ce3aa700 1 -- 172.21.15.44:0/2707531526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c81a5400 con 0x55f6ee26c710 2023-09-10T00:42:36.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.829+0000 7f73cdba9700 1 --2- 172.21.15.44:0/2707531526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26d030 0x55f6ee273980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:36.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.829+0000 7f73ce3aa700 1 --2- 172.21.15.44:0/2707531526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26c710 0x55f6ee276700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f73c4011ca0 tx=0x7f73c40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce5bea8f2f9c5e7f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:36.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.829+0000 7f73b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:42:36.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.829+0000 7f73b1ffb700 1 -- 172.21.15.44:0/2707531526 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c400ee10 con 0x55f6ee26c710 2023-09-10T00:42:36.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.829+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/2707531526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6ee1164f0 con 0x55f6ee26c710 2023-09-10T00:42:36.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.829+0000 7f73b1ffb700 1 -- 172.21.15.44:0/2707531526 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f73c401fdb0 con 0x55f6ee26c710 2023-09-10T00:42:36.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.829+0000 7f73b1ffb700 1 -- 172.21.15.44:0/2707531526 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73c4028850 con 0x55f6ee26c710 2023-09-10T00:42:36.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.829+0000 7f73dc7d4b80 10 client.25972 fetch_fsmap learned FSMap version 34 2023-09-10T00:42:36.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.829+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/2707531526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f73c0011330 con 0x55f6ee26c710 2023-09-10T00:42:36.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.830+0000 7f73b1ffb700 1 -- 172.21.15.44:0/2707531526 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f6ee1164f0 con 0x55f6ee26c710 2023-09-10T00:42:36.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.830+0000 7f73b1ffb700 1 -- 172.21.15.44:0/2707531526 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f73c4030970 con 0x55f6ee26c710 2023-09-10T00:42:36.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.830+0000 7f73dc7d4b80 10 client.25972 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:42:36.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.830+0000 7f73dc7d4b80 10 client.25972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:42:36.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.830+0000 7f73dc7d4b80 20 client.25972 populate_metadata read hostname 'smithi044' 2023-09-10T00:42:36.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.830+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/2707531526 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f6ee2869e0 0x55f6ee288e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.830+0000 7f73dc7d4b80 4 client.25972 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:42:36.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.830+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/2707531526 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6ee1d38a0 con 0x55f6ee2869e0 2023-09-10T00:42:36.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.830+0000 7f73cdba9700 1 --2- 172.21.15.44:0/2707531526 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f6ee2869e0 0x55f6ee288e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.831+0000 7f73b1ffb700 10 client.25972 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:36.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.831+0000 7f73cdba9700 1 --2- 172.21.15.44:0/2707531526 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f6ee2869e0 0x55f6ee288e00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fbb0e01a43dd740 server_cookie=203a1081073b6ca5 in_seq=0 out_seq=0 2023-09-10T00:42:36.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.831+0000 7f73b1ffb700 1 -- 172.21.15.44:0/2707531526 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f6ee1d38a0 con 0x55f6ee2869e0 2023-09-10T00:42:36.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.831+0000 7f73b1ffb700 10 client.25972 handle_command_reply: tid=0 2023-09-10T00:42:36.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.831+0000 7f73dc7d4b80 1 client.25972 shutdown 2023-09-10T00:42:36.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.831+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/2707531526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26c710 msgr2=0x55f6ee276700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:36.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.831+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/2707531526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26c710 0x55f6ee276700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f73c4011ca0 tx=0x7f73c40047c0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.831+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/2707531526 shutdown_connections 2023-09-10T00:42:36.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.831+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/2707531526 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f6ee2869e0 0x55f6ee288e00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.831+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/2707531526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 0x55f6ee273ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.831+0000 7f73b1ffb700 0 client.25972 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:36.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.831+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/2707531526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26c710 0x55f6ee276700 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.832+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/2707531526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26d030 0x55f6ee273980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.832+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/2707531526 >> 172.21.15.44:0/2707531526 conn(0x55f6ee1e1090 msgr2=0x55f6ee267780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:36.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.832+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/2707531526 shutdown_connections 2023-09-10T00:42:36.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.832+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/2707531526 wait complete. 2023-09-10T00:42:36.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.832+0000 7f73dc7d4b80 20 client.25972 trim_cache size 0 max 16384 2023-09-10T00:42:36.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:36 smithi099 ceph-mon[102692]: pgmap v61: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 45 MiB/s wr, 45 op/s 2023-09-10T00:42:36.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.854+0000 7f73dc7d4b80 1 Processor -- start 2023-09-10T00:42:36.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.854+0000 7f73dc7d4b80 1 -- start start 2023-09-10T00:42:36.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.854+0000 7f73dc7d4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 0x55f6ee27da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.854+0000 7f73dc7d4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26d030 0x55f6ee2762a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.854+0000 7f73dc7d4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 0x55f6ee270c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.854+0000 7f73dc7d4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee26e800 con 0x55f6ee26c710 2023-09-10T00:42:36.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.854+0000 7f73dc7d4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee27df70 con 0x55f6ee276e40 2023-09-10T00:42:36.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.854+0000 7f73dc7d4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee271ee0 con 0x55f6ee26d030 2023-09-10T00:42:36.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.854+0000 7f73ce3aa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 0x55f6ee27da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.854+0000 7f73ce3aa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 0x55f6ee27da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50156/0 (socket says 172.21.15.44:50156) 2023-09-10T00:42:36.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.854+0000 7f73cebab700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 0x55f6ee270c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.854+0000 7f73ce3aa700 1 -- 172.21.15.44:0/429453031 learned_addr learned my addr 172.21.15.44:0/429453031 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:36.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.854+0000 7f73cdba9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26d030 0x55f6ee2762a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.855+0000 7f73cdba9700 1 -- 172.21.15.44:0/429453031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 msgr2=0x55f6ee270c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:36.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.855+0000 7f73cdba9700 1 --2- 172.21.15.44:0/429453031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 0x55f6ee270c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.855+0000 7f73cdba9700 1 -- 172.21.15.44:0/429453031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 msgr2=0x55f6ee27da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:36.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.855+0000 7f73cdba9700 1 --2- 172.21.15.44:0/429453031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 0x55f6ee27da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.855+0000 7f73cdba9700 1 -- 172.21.15.44:0/429453031 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6edfd3c60 con 0x55f6ee26d030 2023-09-10T00:42:36.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.855+0000 7f73cebab700 1 --2- 172.21.15.44:0/429453031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 0x55f6ee270c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:36.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.855+0000 7f73ce3aa700 1 --2- 172.21.15.44:0/429453031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 0x55f6ee27da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:36.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.855+0000 7f73cdba9700 1 --2- 172.21.15.44:0/429453031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26d030 0x55f6ee2762a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f73b80ca5e0 tx=0x7f73b8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca17d5151cf0b6db server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:36.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.855+0000 7f73bd7fa700 1 -- 172.21.15.44:0/429453031 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b809aa70 con 0x55f6ee26d030 2023-09-10T00:42:36.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.855+0000 7f73bd7fa700 1 -- 172.21.15.44:0/429453031 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f73b809a370 con 0x55f6ee26d030 2023-09-10T00:42:36.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.855+0000 7f73bd7fa700 1 -- 172.21.15.44:0/429453031 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b809a5d0 con 0x55f6ee26d030 2023-09-10T00:42:36.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.856+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/429453031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26d030 msgr2=0x55f6ee2762a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:36.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.856+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/429453031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26d030 0x55f6ee2762a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f73b80ca5e0 tx=0x7f73b8009ca0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.856+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/429453031 shutdown_connections 2023-09-10T00:42:36.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.856+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/429453031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee276e40 0x55f6ee270c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.856+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/429453031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 0x55f6ee27da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.856+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/429453031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee26d030 0x55f6ee2762a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.856+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/429453031 >> 172.21.15.44:0/429453031 conn(0x55f6ee1e1090 msgr2=0x55f6ee26ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:36.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.856+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/429453031 shutdown_connections 2023-09-10T00:42:36.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.856+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/429453031 wait complete. 2023-09-10T00:42:36.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.856+0000 7f73dc7d4b80 1 -- start start 2023-09-10T00:42:36.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.856+0000 7f73dc7d4b80 1 Processor -- start 2023-09-10T00:42:36.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73dc7d4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 0x55f6ee2ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73dc7d4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee26d030 0x55f6ee2ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73dc7d4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee276e40 0x55f6ee2d3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73dc7d4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee2d3dc0 con 0x55f6ee26c710 2023-09-10T00:42:36.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73dc7d4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee2d3f00 con 0x55f6ee26d030 2023-09-10T00:42:36.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73dc7d4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee2d40c0 con 0x55f6ee276e40 2023-09-10T00:42:36.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73dc7d4b80 10 client.0 _ll_register_callbacks cb 0x7f73c400e9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:42:36.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73ce3aa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 0x55f6ee2ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73cdba9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee26d030 0x55f6ee2ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73cdba9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee26d030 0x55f6ee2ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51904/0 (socket says 172.21.15.44:51904) 2023-09-10T00:42:36.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73ce3aa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 0x55f6ee2ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50162/0 (socket says 172.21.15.44:50162) 2023-09-10T00:42:36.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73cdba9700 1 -- 172.21.15.44:0/741264004 learned_addr learned my addr 172.21.15.44:0/741264004 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:36.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73cebab700 1 --2- 172.21.15.44:0/741264004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee276e40 0x55f6ee2d3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73ce3aa700 1 -- 172.21.15.44:0/741264004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee276e40 msgr2=0x55f6ee2d3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:36.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73ce3aa700 1 --2- 172.21.15.44:0/741264004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee276e40 0x55f6ee2d3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73ce3aa700 1 -- 172.21.15.44:0/741264004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee26d030 msgr2=0x55f6ee2ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:36.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73ce3aa700 1 --2- 172.21.15.44:0/741264004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee26d030 0x55f6ee2ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73ce3aa700 1 -- 172.21.15.44:0/741264004 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c0011330 con 0x55f6ee26c710 2023-09-10T00:42:36.881 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73cdba9700 1 --2- 172.21.15.44:0/741264004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee26d030 0x55f6ee2ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:36.881 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73cebab700 1 --2- 172.21.15.44:0/741264004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee276e40 0x55f6ee2d3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:36.881 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73ce3aa700 1 --2- 172.21.15.44:0/741264004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 0x55f6ee2ce9b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f73c4030d50 tx=0x7f73c4009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c96106e6ddfa7d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:36.881 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.857+0000 7f73b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:42:36.881 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.858+0000 7f73b1ffb700 1 -- 172.21.15.44:0/741264004 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c400ee10 con 0x55f6ee26c710 2023-09-10T00:42:36.882 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.858+0000 7f73b1ffb700 1 -- 172.21.15.44:0/741264004 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f73c4028a60 con 0x55f6ee26c710 2023-09-10T00:42:36.882 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.858+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/741264004 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73c000f040 con 0x55f6ee26c710 2023-09-10T00:42:36.882 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.858+0000 7f73b1ffb700 1 -- 172.21.15.44:0/741264004 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73c4028c20 con 0x55f6ee26c710 2023-09-10T00:42:36.882 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.858+0000 7f73dc7d4b80 10 client.16374 fetch_fsmap learned FSMap version 34 2023-09-10T00:42:36.882 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.858+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/741264004 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f6edfeade0 con 0x55f6ee26c710 2023-09-10T00:42:36.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.858+0000 7f73b1ffb700 1 -- 172.21.15.44:0/741264004 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f73c000f040 con 0x55f6ee26c710 2023-09-10T00:42:36.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.858+0000 7f73b1ffb700 1 -- 172.21.15.44:0/741264004 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f73c4005030 con 0x55f6ee26c710 2023-09-10T00:42:36.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.858+0000 7f73dc7d4b80 10 client.16374 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:42:36.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.858+0000 7f73dc7d4b80 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:42:36.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.858+0000 7f73dc7d4b80 20 client.16374 populate_metadata read hostname 'smithi044' 2023-09-10T00:42:36.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.858+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/741264004 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f6ee2869e0 0x55f6ee2dd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.858+0000 7f73dc7d4b80 4 client.16374 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:42:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.858+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/741264004 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f6ee1b7920 con 0x55f6ee2869e0 2023-09-10T00:42:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.859+0000 7f73cdba9700 1 --2- 172.21.15.44:0/741264004 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f6ee2869e0 0x55f6ee2dd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.859+0000 7f73b1ffb700 10 client.16374 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.859+0000 7f73cdba9700 1 --2- 172.21.15.44:0/741264004 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f6ee2869e0 0x55f6ee2dd8f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f2a976d2dd96e8 server_cookie=f99721147ba0df94 in_seq=0 out_seq=0 2023-09-10T00:42:36.885 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.859+0000 7f73b1ffb700 1 -- 172.21.15.44:0/741264004 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f6ee1b7920 con 0x55f6ee2869e0 2023-09-10T00:42:36.885 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.859+0000 7f73b1ffb700 10 client.16374 handle_command_reply: tid=0 2023-09-10T00:42:36.885 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.859+0000 7f73dc7d4b80 1 client.16374 shutdown 2023-09-10T00:42:36.885 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.859+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/741264004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 msgr2=0x55f6ee2ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:36.885 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.859+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/741264004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 0x55f6ee2ce9b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f73c4030d50 tx=0x7f73c4009340 comp rx=0 tx=0).stop 2023-09-10T00:42:36.885 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.859+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/741264004 shutdown_connections 2023-09-10T00:42:36.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.859+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/741264004 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f6ee2869e0 0x55f6ee2dd8f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.859+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/741264004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6ee276e40 0x55f6ee2d3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.859+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/741264004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6ee26c710 0x55f6ee2ce9b0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.859+0000 7f73dc7d4b80 1 --2- 172.21.15.44:0/741264004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6ee26d030 0x55f6ee2ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.859+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/741264004 >> 172.21.15.44:0/741264004 conn(0x55f6ee1e1090 msgr2=0x55f6ee26b700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:36.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.859+0000 7f73b1ffb700 0 client.16374 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:36.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.860+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/741264004 shutdown_connections 2023-09-10T00:42:36.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.860+0000 7f73dc7d4b80 1 -- 172.21.15.44:0/741264004 wait complete. 2023-09-10T00:42:36.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.860+0000 7f73dc7d4b80 20 client.16374 trim_cache size 0 max 16384 2023-09-10T00:42:36.888 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:42:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:42:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:42:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:42:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:42:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:42:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:42:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:42:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:42:36.889 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:42:36.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.860+0000 7f73d060e700 1 -- 172.21.15.44:0/2670135988 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f73b40a07b0 msgr2=0x7f73b40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:36.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.860+0000 7f73d060e700 1 --2- 172.21.15.44:0/2670135988 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f73b40a07b0 0x7f73b40a2c70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f73c4011550 tx=0x7f73c4009260 comp rx=0 tx=0).stop 2023-09-10T00:42:36.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.860+0000 7f73d060e700 1 -- 172.21.15.44:0/2670135988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73c80ffe00 msgr2=0x7f73c8195720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:36.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.860+0000 7f73d060e700 1 --2- 172.21.15.44:0/2670135988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73c80ffe00 0x7f73c8195720 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f73b8005910 tx=0x7f73b8004730 comp rx=0 tx=0).stop 2023-09-10T00:42:36.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.860+0000 7f73d060e700 1 -- 172.21.15.44:0/2670135988 shutdown_connections 2023-09-10T00:42:36.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.860+0000 7f73d060e700 1 --2- 172.21.15.44:0/2670135988 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f73b40a07b0 0x7f73b40a2c70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.860+0000 7f73d060e700 1 --2- 172.21.15.44:0/2670135988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c81054e0 0x7f73c8195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.860+0000 7f73d060e700 1 --2- 172.21.15.44:0/2670135988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c80ff440 0x7f73c81951e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.860+0000 7f73d060e700 1 --2- 172.21.15.44:0/2670135988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73c80ffe00 0x7f73c8195720 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:36.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.860+0000 7f73d060e700 1 -- 172.21.15.44:0/2670135988 >> 172.21.15.44:0/2670135988 conn(0x7f73c80faf50 msgr2=0x7f73c80fb810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:36.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.860+0000 7f73d060e700 1 -- 172.21.15.44:0/2670135988 shutdown_connections 2023-09-10T00:42:36.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:42:36.860+0000 7f73d060e700 1 -- 172.21.15.44:0/2670135988 wait complete. 2023-09-10T00:42:36.893 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:36 smithi044 ceph-mon[98676]: pgmap v61: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 45 MiB/s wr, 45 op/s 2023-09-10T00:42:36.894 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:42:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:36 smithi131 ceph-mon[103713]: pgmap v61: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 45 MiB/s wr, 45 op/s 2023-09-10T00:42:37.759 INFO:teuthology.orchestra.run.smithi044.stderr:8600000 of 50000000 tuples (17%) done (elapsed 97.39 s, remaining 468.85 s) 2023-09-10T00:42:38.677 INFO:teuthology.orchestra.run.smithi044.stderr:8700000 of 50000000 tuples (17%) done (elapsed 98.31 s, remaining 466.69 s) 2023-09-10T00:42:38.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:38 smithi099 ceph-mon[102692]: pgmap v62: 129 pgs: 129 active+clean; 2.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 MiB/s wr, 34 op/s 2023-09-10T00:42:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:38 smithi131 ceph-mon[103713]: pgmap v62: 129 pgs: 129 active+clean; 2.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 MiB/s wr, 34 op/s 2023-09-10T00:42:39.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:38 smithi044 ceph-mon[98676]: pgmap v62: 129 pgs: 129 active+clean; 2.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 MiB/s wr, 34 op/s 2023-09-10T00:42:39.945 INFO:teuthology.orchestra.run.smithi044.stderr:8800000 of 50000000 tuples (17%) done (elapsed 99.58 s, remaining 466.21 s) 2023-09-10T00:42:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.639+0000 7faf2929b700 1 -- 172.21.15.44:0/2900102504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf240feaa0 msgr2=0x7faf240fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.640+0000 7faf2929b700 1 --2- 172.21.15.44:0/2900102504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf240feaa0 0x7faf240fef20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7faf18009990 tx=0x7faf18009ca0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.640+0000 7faf2929b700 1 -- 172.21.15.44:0/2900102504 shutdown_connections 2023-09-10T00:42:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.640+0000 7faf2929b700 1 --2- 172.21.15.44:0/2900102504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf240ff460 0x7faf2410d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.640+0000 7faf2929b700 1 --2- 172.21.15.44:0/2900102504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf24106620 0x7faf24106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.640+0000 7faf2929b700 1 --2- 172.21.15.44:0/2900102504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf240feaa0 0x7faf240fef20 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.640+0000 7faf2929b700 1 -- 172.21.15.44:0/2900102504 >> 172.21.15.44:0/2900102504 conn(0x7faf240769d0 msgr2=0x7faf240fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.640+0000 7faf2929b700 1 -- 172.21.15.44:0/2900102504 shutdown_connections 2023-09-10T00:42:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.640+0000 7faf2929b700 1 -- 172.21.15.44:0/2900102504 wait complete. 2023-09-10T00:42:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.640+0000 7faf2929b700 1 Processor -- start 2023-09-10T00:42:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf2929b700 1 -- start start 2023-09-10T00:42:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf2929b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf240feaa0 0x7faf2419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf2929b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf240ff460 0x7faf2419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf2929b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf24106620 0x7faf241963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf2929b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf24196a30 con 0x7faf240feaa0 2023-09-10T00:42:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf2929b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf24196ba0 con 0x7faf24106620 2023-09-10T00:42:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf2929b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf24196d60 con 0x7faf240ff460 2023-09-10T00:42:40.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf22ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf240feaa0 0x7faf2419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf22ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf240feaa0 0x7faf2419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50176/0 (socket says 172.21.15.44:50176) 2023-09-10T00:42:40.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf22ffd700 1 -- 172.21.15.44:0/153740529 learned_addr learned my addr 172.21.15.44:0/153740529 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:40.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf237fe700 1 --2- 172.21.15.44:0/153740529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf24106620 0x7faf241963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf227fc700 1 --2- 172.21.15.44:0/153740529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf240ff460 0x7faf2419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf22ffd700 1 -- 172.21.15.44:0/153740529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf240ff460 msgr2=0x7faf2419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf22ffd700 1 --2- 172.21.15.44:0/153740529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf240ff460 0x7faf2419caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf22ffd700 1 -- 172.21.15.44:0/153740529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf24106620 msgr2=0x7faf241963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf22ffd700 1 --2- 172.21.15.44:0/153740529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf24106620 0x7faf241963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf22ffd700 1 -- 172.21.15.44:0/153740529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf180095f0 con 0x7faf240feaa0 2023-09-10T00:42:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.641+0000 7faf237fe700 1 --2- 172.21.15.44:0/153740529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf24106620 0x7faf241963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.642+0000 7faf22ffd700 1 --2- 172.21.15.44:0/153740529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf240feaa0 0x7faf2419c560 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7faf1400e9a0 tx=0x7faf1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.642+0000 7faf0bfff700 1 -- 172.21.15.44:0/153740529 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf14009790 con 0x7faf240feaa0 2023-09-10T00:42:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.642+0000 7faf0bfff700 1 -- 172.21.15.44:0/153740529 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf14010430 con 0x7faf240feaa0 2023-09-10T00:42:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.642+0000 7faf0bfff700 1 -- 172.21.15.44:0/153740529 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1400cea0 con 0x7faf240feaa0 2023-09-10T00:42:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.642+0000 7faf2929b700 1 -- 172.21.15.44:0/153740529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf24197040 con 0x7faf240feaa0 2023-09-10T00:42:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.642+0000 7faf2929b700 1 -- 172.21.15.44:0/153740529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf240ffd90 con 0x7faf240feaa0 2023-09-10T00:42:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.644+0000 7faf0bfff700 1 -- 172.21.15.44:0/153740529 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7faf140105a0 con 0x7faf240feaa0 2023-09-10T00:42:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.645+0000 7faf0bfff700 1 --2- 172.21.15.44:0/153740529 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faf0c0a0830 0x7faf0c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.645+0000 7faf227fc700 1 --2- 172.21.15.44:0/153740529 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faf0c0a0830 0x7faf0c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.645+0000 7faf0bfff700 1 -- 172.21.15.44:0/153740529 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7faf14014070 con 0x7faf240feaa0 2023-09-10T00:42:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.645+0000 7faf227fc700 1 --2- 172.21.15.44:0/153740529 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faf0c0a0830 0x7faf0c0a2cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faf240fcaf0 tx=0x7faf1800b260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.646+0000 7faf35461b80 1 Processor -- start 2023-09-10T00:42:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.646+0000 7faf35461b80 1 -- start start 2023-09-10T00:42:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.646+0000 7faf35461b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0efa1a0 0x55baf0efa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.646+0000 7faf35461b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 0x55baf0ef4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.646+0000 7faf35461b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0ef4cc0 0x55baf0f00f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.646+0000 7faf35461b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf0e4d6d0 con 0x55baf0efa1a0 2023-09-10T00:42:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.646+0000 7faf35461b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf0e4aff0 con 0x55baf0ef4300 2023-09-10T00:42:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.646+0000 7faf35461b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf0e468e0 con 0x55baf0ef4cc0 2023-09-10T00:42:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.646+0000 7faf22ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0efa1a0 0x55baf0efa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.646+0000 7faf227fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 0x55baf0ef4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.647+0000 7faf22ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0efa1a0 0x55baf0efa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50184/0 (socket says 172.21.15.44:50184) 2023-09-10T00:42:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.647+0000 7faf22ffd700 1 -- 172.21.15.44:0/3846814330 learned_addr learned my addr 172.21.15.44:0/3846814330 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.647+0000 7faf237fe700 1 --2- 172.21.15.44:0/3846814330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0ef4cc0 0x55baf0f00f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.647+0000 7faf22ffd700 1 -- 172.21.15.44:0/3846814330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0ef4cc0 msgr2=0x55baf0f00f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.647+0000 7faf22ffd700 1 --2- 172.21.15.44:0/3846814330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0ef4cc0 0x55baf0f00f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.647+0000 7faf22ffd700 1 -- 172.21.15.44:0/3846814330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 msgr2=0x55baf0ef4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.647+0000 7faf22ffd700 1 --2- 172.21.15.44:0/3846814330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 0x55baf0ef4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.647+0000 7faf22ffd700 1 -- 172.21.15.44:0/3846814330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55baf0c72d80 con 0x55baf0efa1a0 2023-09-10T00:42:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.647+0000 7faf227fc700 1 --2- 172.21.15.44:0/3846814330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 0x55baf0ef4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.647+0000 7faf22ffd700 1 --2- 172.21.15.44:0/3846814330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0efa1a0 0x55baf0efa580 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7faf140a10b0 tx=0x7faf14098ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47200ed1f6db0e42 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.647+0000 7faf097fa700 1 -- 172.21.15.44:0/3846814330 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1409d170 con 0x55baf0efa1a0 2023-09-10T00:42:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.647+0000 7faf097fa700 1 -- 172.21.15.44:0/3846814330 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf14099570 con 0x55baf0efa1a0 2023-09-10T00:42:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.647+0000 7faf097fa700 1 -- 172.21.15.44:0/3846814330 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf140997d0 con 0x55baf0efa1a0 2023-09-10T00:42:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.648+0000 7faf35461b80 1 -- 172.21.15.44:0/3846814330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0efa1a0 msgr2=0x55baf0efa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.648+0000 7faf35461b80 1 --2- 172.21.15.44:0/3846814330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0efa1a0 0x55baf0efa580 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7faf140a10b0 tx=0x7faf14098ed0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.648+0000 7faf35461b80 1 -- 172.21.15.44:0/3846814330 shutdown_connections 2023-09-10T00:42:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.648+0000 7faf35461b80 1 --2- 172.21.15.44:0/3846814330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0ef4cc0 0x55baf0f00f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.648+0000 7faf35461b80 1 --2- 172.21.15.44:0/3846814330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0efa1a0 0x55baf0efa580 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.648+0000 7faf35461b80 1 --2- 172.21.15.44:0/3846814330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 0x55baf0ef4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.648+0000 7faf35461b80 1 -- 172.21.15.44:0/3846814330 >> 172.21.15.44:0/3846814330 conn(0x55baf0e69030 msgr2=0x55baf0eef560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.648+0000 7faf35461b80 1 -- 172.21.15.44:0/3846814330 shutdown_connections 2023-09-10T00:42:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.648+0000 7faf35461b80 1 -- 172.21.15.44:0/3846814330 wait complete. 2023-09-10T00:42:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf35461b80 1 -- start start 2023-09-10T00:42:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf35461b80 1 Processor -- start 2023-09-10T00:42:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf35461b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 0x55baf0f00f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf35461b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0ef4cc0 0x55baf0efe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf35461b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0efa1a0 0x55baf0efe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf35461b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf0de4b10 con 0x55baf0ef4cc0 2023-09-10T00:42:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf35461b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf0de4810 con 0x55baf0ef4300 2023-09-10T00:42:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf35461b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf0cc88c0 con 0x55baf0efa1a0 2023-09-10T00:42:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf35461b80 10 client.0 _ll_register_callbacks cb 0x55baf0d2ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:42:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf227fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0ef4cc0 0x55baf0efe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf227fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0ef4cc0 0x55baf0efe190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50192/0 (socket says 172.21.15.44:50192) 2023-09-10T00:42:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf227fc700 1 -- 172.21.15.44:0/2098746312 learned_addr learned my addr 172.21.15.44:0/2098746312 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf22ffd700 1 --2- 172.21.15.44:0/2098746312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 0x55baf0f00f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf237fe700 1 --2- 172.21.15.44:0/2098746312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0efa1a0 0x55baf0efe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf22ffd700 1 -- 172.21.15.44:0/2098746312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0efa1a0 msgr2=0x55baf0efe8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf22ffd700 1 --2- 172.21.15.44:0/2098746312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0efa1a0 0x55baf0efe8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf22ffd700 1 -- 172.21.15.44:0/2098746312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0ef4cc0 msgr2=0x55baf0efe190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf22ffd700 1 --2- 172.21.15.44:0/2098746312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0ef4cc0 0x55baf0efe190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.649+0000 7faf22ffd700 1 -- 172.21.15.44:0/2098746312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2410dd30 con 0x55baf0ef4300 2023-09-10T00:42:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.650+0000 7faf227fc700 1 --2- 172.21.15.44:0/2098746312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0ef4cc0 0x55baf0efe190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.650+0000 7faf22ffd700 1 --2- 172.21.15.44:0/2098746312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 0x55baf0f00f10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf14099500 tx=0x7faf14010bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7636efed48a9a4a8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.650+0000 7faf06ff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:42:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.650+0000 7faf06ff5700 1 -- 172.21.15.44:0/2098746312 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf140aeea0 con 0x55baf0ef4300 2023-09-10T00:42:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.650+0000 7faf06ff5700 1 -- 172.21.15.44:0/2098746312 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7faf1409d170 con 0x55baf0ef4300 2023-09-10T00:42:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.650+0000 7faf35461b80 1 -- 172.21.15.44:0/2098746312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55baf0d9e490 con 0x55baf0ef4300 2023-09-10T00:42:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.650+0000 7faf06ff5700 1 -- 172.21.15.44:0/2098746312 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf1409d3a0 con 0x55baf0ef4300 2023-09-10T00:42:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.650+0000 7faf35461b80 10 client.26054 fetch_fsmap learned FSMap version 34 2023-09-10T00:42:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.650+0000 7faf35461b80 1 -- 172.21.15.44:0/2098746312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7faf240ffd90 con 0x55baf0ef4300 2023-09-10T00:42:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.650+0000 7faf06ff5700 1 -- 172.21.15.44:0/2098746312 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55baf0d9e490 con 0x55baf0ef4300 2023-09-10T00:42:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.651+0000 7faf06ff5700 1 -- 172.21.15.44:0/2098746312 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7faf140a1150 con 0x55baf0ef4300 2023-09-10T00:42:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.651+0000 7faf35461b80 10 client.26054 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:42:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.651+0000 7faf35461b80 10 client.26054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:42:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.651+0000 7faf35461b80 20 client.26054 populate_metadata read hostname 'smithi044' 2023-09-10T00:42:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.651+0000 7faf35461b80 1 --2- 172.21.15.44:0/2098746312 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55baf0f0dfd0 0x55baf0f103f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.651+0000 7faf35461b80 4 client.26054 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:42:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.651+0000 7faf35461b80 1 -- 172.21.15.44:0/2098746312 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55baf0e5b840 con 0x55baf0f0dfd0 2023-09-10T00:42:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.651+0000 7faf227fc700 1 --2- 172.21.15.44:0/2098746312 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55baf0f0dfd0 0x55baf0f103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.651+0000 7faf06ff5700 10 client.26054 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.651+0000 7faf227fc700 1 --2- 172.21.15.44:0/2098746312 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55baf0f0dfd0 0x55baf0f103f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55fd9d0f22af7223 server_cookie=818606f606503583 in_seq=0 out_seq=0 2023-09-10T00:42:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.652+0000 7faf06ff5700 1 -- 172.21.15.44:0/2098746312 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55baf0e5b840 con 0x55baf0f0dfd0 2023-09-10T00:42:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.652+0000 7faf06ff5700 10 client.26054 handle_command_reply: tid=0 2023-09-10T00:42:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.652+0000 7faf35461b80 1 client.26054 shutdown 2023-09-10T00:42:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.652+0000 7faf35461b80 1 -- 172.21.15.44:0/2098746312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 msgr2=0x55baf0f00f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.652+0000 7faf35461b80 1 --2- 172.21.15.44:0/2098746312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 0x55baf0f00f10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf14099500 tx=0x7faf14010bf0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.652+0000 7faf35461b80 1 -- 172.21.15.44:0/2098746312 shutdown_connections 2023-09-10T00:42:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.652+0000 7faf35461b80 1 --2- 172.21.15.44:0/2098746312 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55baf0f0dfd0 0x55baf0f103f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.652+0000 7faf35461b80 1 --2- 172.21.15.44:0/2098746312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0efa1a0 0x55baf0efe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.652+0000 7faf35461b80 1 --2- 172.21.15.44:0/2098746312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 0x55baf0f00f10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.652+0000 7faf35461b80 1 --2- 172.21.15.44:0/2098746312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0ef4cc0 0x55baf0efe190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.652+0000 7faf06ff5700 0 client.26054 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.652+0000 7faf35461b80 1 -- 172.21.15.44:0/2098746312 >> 172.21.15.44:0/2098746312 conn(0x55baf0e69030 msgr2=0x55baf0eee7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.652+0000 7faf35461b80 1 -- 172.21.15.44:0/2098746312 shutdown_connections 2023-09-10T00:42:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.652+0000 7faf35461b80 1 -- 172.21.15.44:0/2098746312 wait complete. 2023-09-10T00:42:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.652+0000 7faf35461b80 20 client.26054 trim_cache size 0 max 16384 2023-09-10T00:42:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf35461b80 1 Processor -- start 2023-09-10T00:42:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf35461b80 1 -- start start 2023-09-10T00:42:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf35461b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0ef4300 0x55baf0ef46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf35461b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4cc0 0x55baf0f03ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf35461b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0efa1a0 0x55baf0efd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf35461b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf0f04050 con 0x55baf0efa1a0 2023-09-10T00:42:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf35461b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf0efaa70 con 0x55baf0ef4cc0 2023-09-10T00:42:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf35461b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf0f10a50 con 0x55baf0ef4300 2023-09-10T00:42:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf237fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0efa1a0 0x55baf0efd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf22ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0ef4300 0x55baf0ef46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf237fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0efa1a0 0x55baf0efd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50202/0 (socket says 172.21.15.44:50202) 2023-09-10T00:42:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf22ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0ef4300 0x55baf0ef46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57080/0 (socket says 172.21.15.44:57080) 2023-09-10T00:42:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf237fe700 1 -- 172.21.15.44:0/4284574505 learned_addr learned my addr 172.21.15.44:0/4284574505 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf227fc700 1 --2- 172.21.15.44:0/4284574505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4cc0 0x55baf0f03ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf237fe700 1 -- 172.21.15.44:0/4284574505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0ef4300 msgr2=0x55baf0ef46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf237fe700 1 --2- 172.21.15.44:0/4284574505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0ef4300 0x55baf0ef46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf237fe700 1 -- 172.21.15.44:0/4284574505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4cc0 msgr2=0x55baf0f03ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf237fe700 1 --2- 172.21.15.44:0/4284574505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4cc0 0x55baf0f03ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf237fe700 1 -- 172.21.15.44:0/4284574505 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55baf0c5bc00 con 0x55baf0efa1a0 2023-09-10T00:42:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf22ffd700 1 --2- 172.21.15.44:0/4284574505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0ef4300 0x55baf0ef46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.670+0000 7faf227fc700 1 --2- 172.21.15.44:0/4284574505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4cc0 0x55baf0f03ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.671+0000 7faf237fe700 1 --2- 172.21.15.44:0/4284574505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0efa1a0 0x55baf0efd990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00e9f0 tx=0x7faf1c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a4eb773a2a90f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.671+0000 7faf09ffb700 1 -- 172.21.15.44:0/4284574505 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c017aa0 con 0x55baf0efa1a0 2023-09-10T00:42:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.671+0000 7faf09ffb700 1 -- 172.21.15.44:0/4284574505 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf1c00f040 con 0x55baf0efa1a0 2023-09-10T00:42:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.671+0000 7faf09ffb700 1 -- 172.21.15.44:0/4284574505 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c0073e0 con 0x55baf0efa1a0 2023-09-10T00:42:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.671+0000 7faf35461b80 1 -- 172.21.15.44:0/4284574505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0efa1a0 msgr2=0x55baf0efd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.671+0000 7faf35461b80 1 --2- 172.21.15.44:0/4284574505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0efa1a0 0x55baf0efd990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00e9f0 tx=0x7faf1c009920 comp rx=0 tx=0).stop 2023-09-10T00:42:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.671+0000 7faf35461b80 1 -- 172.21.15.44:0/4284574505 shutdown_connections 2023-09-10T00:42:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.671+0000 7faf35461b80 1 --2- 172.21.15.44:0/4284574505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0efa1a0 0x55baf0efd990 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.671+0000 7faf35461b80 1 --2- 172.21.15.44:0/4284574505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0ef4300 0x55baf0ef46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.671+0000 7faf35461b80 1 --2- 172.21.15.44:0/4284574505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4cc0 0x55baf0f03ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.671+0000 7faf35461b80 1 -- 172.21.15.44:0/4284574505 >> 172.21.15.44:0/4284574505 conn(0x55baf0e69030 msgr2=0x55baf0ef81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.672+0000 7faf35461b80 1 -- 172.21.15.44:0/4284574505 shutdown_connections 2023-09-10T00:42:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.672+0000 7faf35461b80 1 -- 172.21.15.44:0/4284574505 wait complete. 2023-09-10T00:42:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.672+0000 7faf35461b80 1 -- start start 2023-09-10T00:42:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.672+0000 7faf35461b80 1 Processor -- start 2023-09-10T00:42:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.672+0000 7faf35461b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 0x55baf0f55d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.672+0000 7faf35461b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0ef4cc0 0x55baf0f56290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.672+0000 7faf35461b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0efa1a0 0x55baf0f567f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.672+0000 7faf35461b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf0f60f80 con 0x55baf0ef4cc0 2023-09-10T00:42:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.672+0000 7faf35461b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf0f610f0 con 0x55baf0ef4300 2023-09-10T00:42:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.672+0000 7faf35461b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf0f612b0 con 0x55baf0efa1a0 2023-09-10T00:42:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.672+0000 7faf35461b80 10 client.0 _ll_register_callbacks cb 0x7faf180055c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:42:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.672+0000 7faf227fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0ef4cc0 0x55baf0f56290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.672+0000 7faf22ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 0x55baf0f55d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf22ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 0x55baf0f55d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51944/0 (socket says 172.21.15.44:51944) 2023-09-10T00:42:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.672+0000 7faf237fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0efa1a0 0x55baf0f567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf22ffd700 1 -- 172.21.15.44:0/262541847 learned_addr learned my addr 172.21.15.44:0/262541847 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf227fc700 1 -- 172.21.15.44:0/262541847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0efa1a0 msgr2=0x55baf0f567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf227fc700 1 --2- 172.21.15.44:0/262541847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0efa1a0 0x55baf0f567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf227fc700 1 -- 172.21.15.44:0/262541847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 msgr2=0x55baf0f55d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf227fc700 1 --2- 172.21.15.44:0/262541847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 0x55baf0f55d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf227fc700 1 -- 172.21.15.44:0/262541847 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf240ffd90 con 0x55baf0ef4cc0 2023-09-10T00:42:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf22ffd700 1 --2- 172.21.15.44:0/262541847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 0x55baf0f55d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf237fe700 1 --2- 172.21.15.44:0/262541847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0efa1a0 0x55baf0f567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf227fc700 1 --2- 172.21.15.44:0/262541847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0ef4cc0 0x55baf0f56290 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x55baf0f10c90 tx=0x7faf18009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5ea412e153f4e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf06ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:42:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf06ff5700 1 -- 172.21.15.44:0/262541847 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18036d10 con 0x55baf0ef4cc0 2023-09-10T00:42:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf06ff5700 1 -- 172.21.15.44:0/262541847 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7faf1800f460 con 0x55baf0ef4cc0 2023-09-10T00:42:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf35461b80 1 -- 172.21.15.44:0/262541847 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf14099570 con 0x55baf0ef4cc0 2023-09-10T00:42:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf06ff5700 1 -- 172.21.15.44:0/262541847 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf18036e70 con 0x55baf0ef4cc0 2023-09-10T00:42:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf35461b80 10 client.16392 fetch_fsmap learned FSMap version 34 2023-09-10T00:42:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.673+0000 7faf35461b80 1 -- 172.21.15.44:0/262541847 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7faf18005670 con 0x55baf0ef4cc0 2023-09-10T00:42:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.674+0000 7faf06ff5700 1 -- 172.21.15.44:0/262541847 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf14099570 con 0x55baf0ef4cc0 2023-09-10T00:42:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.674+0000 7faf06ff5700 1 -- 172.21.15.44:0/262541847 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7faf18024510 con 0x55baf0ef4cc0 2023-09-10T00:42:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.674+0000 7faf35461b80 10 client.16392 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:42:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.674+0000 7faf35461b80 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:42:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.674+0000 7faf35461b80 20 client.16392 populate_metadata read hostname 'smithi044' 2023-09-10T00:42:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.674+0000 7faf35461b80 1 --2- 172.21.15.44:0/262541847 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55baf0f62a50 0x55baf0f64e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.674+0000 7faf35461b80 4 client.16392 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:42:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.674+0000 7faf35461b80 1 -- 172.21.15.44:0/262541847 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55baf0e3f8c0 con 0x55baf0f62a50 2023-09-10T00:42:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.674+0000 7faf22ffd700 1 --2- 172.21.15.44:0/262541847 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55baf0f62a50 0x55baf0f64e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.675+0000 7faf22ffd700 1 --2- 172.21.15.44:0/262541847 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55baf0f62a50 0x55baf0f64e70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b0adcb4c01683d7 server_cookie=5846d3edc2ed6e0 in_seq=0 out_seq=0 2023-09-10T00:42:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.675+0000 7faf06ff5700 10 client.16392 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.687+0000 7faf06ff5700 1 -- 172.21.15.44:0/262541847 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55baf0e3f8c0 con 0x55baf0f62a50 2023-09-10T00:42:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.687+0000 7faf06ff5700 10 client.16392 handle_command_reply: tid=0 2023-09-10T00:42:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.687+0000 7faf35461b80 1 client.16392 shutdown 2023-09-10T00:42:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.687+0000 7faf35461b80 1 -- 172.21.15.44:0/262541847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0ef4cc0 msgr2=0x55baf0f56290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.687+0000 7faf35461b80 1 --2- 172.21.15.44:0/262541847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0ef4cc0 0x55baf0f56290 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x55baf0f10c90 tx=0x7faf18009ca0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.688+0000 7faf35461b80 1 -- 172.21.15.44:0/262541847 shutdown_connections 2023-09-10T00:42:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.688+0000 7faf35461b80 1 --2- 172.21.15.44:0/262541847 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55baf0f62a50 0x55baf0f64e70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.688+0000 7faf35461b80 1 --2- 172.21.15.44:0/262541847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55baf0efa1a0 0x55baf0f567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.688+0000 7faf06ff5700 0 client.16392 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.688+0000 7faf35461b80 1 --2- 172.21.15.44:0/262541847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baf0ef4300 0x55baf0f55d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.688+0000 7faf35461b80 1 --2- 172.21.15.44:0/262541847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baf0ef4cc0 0x55baf0f56290 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.688+0000 7faf35461b80 1 -- 172.21.15.44:0/262541847 >> 172.21.15.44:0/262541847 conn(0x55baf0e69030 msgr2=0x55baf0ef32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.688+0000 7faf35461b80 1 -- 172.21.15.44:0/262541847 shutdown_connections 2023-09-10T00:42:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.688+0000 7faf35461b80 1 -- 172.21.15.44:0/262541847 wait complete. 2023-09-10T00:42:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.688+0000 7faf35461b80 20 client.16392 trim_cache size 0 max 16384 2023-09-10T00:42:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:42:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:42:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:42:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:42:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.688+0000 7faf2929b700 1 -- 172.21.15.44:0/153740529 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faf0c0a0830 msgr2=0x7faf0c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.688+0000 7faf2929b700 1 --2- 172.21.15.44:0/153740529 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faf0c0a0830 0x7faf0c0a2cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faf240fcaf0 tx=0x7faf1800b260 comp rx=0 tx=0).stop 2023-09-10T00:42:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.688+0000 7faf2929b700 1 -- 172.21.15.44:0/153740529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf240feaa0 msgr2=0x7faf2419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.688+0000 7faf2929b700 1 --2- 172.21.15.44:0/153740529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf240feaa0 0x7faf2419c560 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7faf1400e9a0 tx=0x7faf1400ecb0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.689+0000 7faf2929b700 1 -- 172.21.15.44:0/153740529 shutdown_connections 2023-09-10T00:42:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.689+0000 7faf2929b700 1 --2- 172.21.15.44:0/153740529 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faf0c0a0830 0x7faf0c0a2cf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.689+0000 7faf2929b700 1 --2- 172.21.15.44:0/153740529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf24106620 0x7faf241963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.689+0000 7faf2929b700 1 --2- 172.21.15.44:0/153740529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf240feaa0 0x7faf2419c560 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.689+0000 7faf2929b700 1 --2- 172.21.15.44:0/153740529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf240ff460 0x7faf2419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.689+0000 7faf2929b700 1 -- 172.21.15.44:0/153740529 >> 172.21.15.44:0/153740529 conn(0x7faf240769d0 msgr2=0x7faf240fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.689+0000 7faf2929b700 1 -- 172.21.15.44:0/153740529 shutdown_connections 2023-09-10T00:42:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:40.689+0000 7faf2929b700 1 -- 172.21.15.44:0/153740529 wait complete. 2023-09-10T00:42:40.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:40 smithi099 ceph-mon[102692]: pgmap v63: 129 pgs: 129 active+clean; 2.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 33 op/s 2023-09-10T00:42:41.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:40 smithi131 ceph-mon[103713]: pgmap v63: 129 pgs: 129 active+clean; 2.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 33 op/s 2023-09-10T00:42:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:40 smithi044 ceph-mon[98676]: pgmap v63: 129 pgs: 129 active+clean; 2.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 33 op/s 2023-09-10T00:42:41.187 INFO:teuthology.orchestra.run.smithi044.stderr:8900000 of 50000000 tuples (17%) done (elapsed 100.82 s, remaining 465.59 s) 2023-09-10T00:42:42.040 INFO:teuthology.orchestra.run.smithi044.stderr:9000000 of 50000000 tuples (18%) done (elapsed 101.67 s, remaining 463.18 s) 2023-09-10T00:42:42.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:42 smithi099 ceph-mon[102692]: pgmap v64: 129 pgs: 129 active+clean; 2.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 MiB/s wr, 49 op/s 2023-09-10T00:42:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:42 smithi131 ceph-mon[103713]: pgmap v64: 129 pgs: 129 active+clean; 2.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 MiB/s wr, 49 op/s 2023-09-10T00:42:43.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:42 smithi044 ceph-mon[98676]: pgmap v64: 129 pgs: 129 active+clean; 2.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 MiB/s wr, 49 op/s 2023-09-10T00:42:43.352 INFO:teuthology.orchestra.run.smithi044.stderr:9100000 of 50000000 tuples (18%) done (elapsed 102.99 s, remaining 462.87 s) 2023-09-10T00:42:44.258 INFO:teuthology.orchestra.run.smithi044.stderr:9200000 of 50000000 tuples (18%) done (elapsed 103.89 s, remaining 460.74 s) 2023-09-10T00:42:44.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:44 smithi099 ceph-mon[102692]: pgmap v65: 129 pgs: 129 active+clean; 2.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 31 op/s 2023-09-10T00:42:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:44 smithi131 ceph-mon[103713]: pgmap v65: 129 pgs: 129 active+clean; 2.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 31 op/s 2023-09-10T00:42:45.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:44 smithi044 ceph-mon[98676]: pgmap v65: 129 pgs: 129 active+clean; 2.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 31 op/s 2023-09-10T00:42:45.591 INFO:teuthology.orchestra.run.smithi044.stderr:9300000 of 50000000 tuples (18%) done (elapsed 105.23 s, remaining 460.50 s) 2023-09-10T00:42:46.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:46 smithi099 ceph-mon[102692]: pgmap v66: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 MiB/s wr, 47 op/s 2023-09-10T00:42:46.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:46 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:42:46.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:46 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:46.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:46 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:46.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:46 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:46.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:46 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:46.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:46 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:46.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:46 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:46.926 INFO:teuthology.orchestra.run.smithi044.stderr:9400000 of 50000000 tuples (18%) done (elapsed 106.56 s, remaining 460.24 s) 2023-09-10T00:42:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:46 smithi131 ceph-mon[103713]: pgmap v66: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 MiB/s wr, 47 op/s 2023-09-10T00:42:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:46 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:42:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:46 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:46 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:47.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:46 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:47.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:46 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:47.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:46 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:47.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:46 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:47.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:46 smithi044 ceph-mon[98676]: pgmap v66: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 MiB/s wr, 47 op/s 2023-09-10T00:42:47.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:46 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:42:47.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:46 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:47.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:46 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:47.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:46 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:47.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:46 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:47.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:46 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:47.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:46 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:47.827 INFO:teuthology.orchestra.run.smithi044.stderr:9500000 of 50000000 tuples (19%) done (elapsed 107.46 s, remaining 458.13 s) 2023-09-10T00:42:47.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:47 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:42:47.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:47 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:42:47.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:47 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:48.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:47 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:42:48.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:47 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:42:48.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:47 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:48.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:47 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:42:48.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:47 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:42:48.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:47 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:42:48.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:48 smithi099 ceph-mon[102692]: pgmap v67: 129 pgs: 129 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 MiB/s wr, 35 op/s 2023-09-10T00:42:49.012 INFO:teuthology.orchestra.run.smithi044.stderr:9600000 of 50000000 tuples (19%) done (elapsed 108.65 s, remaining 457.22 s) 2023-09-10T00:42:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:48 smithi131 ceph-mon[103713]: pgmap v67: 129 pgs: 129 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 MiB/s wr, 35 op/s 2023-09-10T00:42:49.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:48 smithi044 ceph-mon[98676]: pgmap v67: 129 pgs: 129 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 MiB/s wr, 35 op/s 2023-09-10T00:42:50.285 INFO:teuthology.orchestra.run.smithi044.stderr:9700000 of 50000000 tuples (19%) done (elapsed 109.92 s, remaining 456.68 s) 2023-09-10T00:42:50.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:50 smithi099 ceph-mon[102692]: pgmap v68: 129 pgs: 129 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 MiB/s wr, 33 op/s 2023-09-10T00:42:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:50 smithi131 ceph-mon[103713]: pgmap v68: 129 pgs: 129 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 MiB/s wr, 33 op/s 2023-09-10T00:42:51.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:50 smithi044 ceph-mon[98676]: pgmap v68: 129 pgs: 129 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 MiB/s wr, 33 op/s 2023-09-10T00:42:51.283 INFO:teuthology.orchestra.run.smithi044.stderr:9800000 of 50000000 tuples (19%) done (elapsed 110.92 s, remaining 454.99 s) 2023-09-10T00:42:52.294 INFO:teuthology.orchestra.run.smithi044.stderr:9900000 of 50000000 tuples (19%) done (elapsed 111.93 s, remaining 453.36 s) 2023-09-10T00:42:52.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:52 smithi099 ceph-mon[102692]: pgmap v69: 129 pgs: 129 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 MiB/s wr, 47 op/s 2023-09-10T00:42:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:52 smithi131 ceph-mon[103713]: pgmap v69: 129 pgs: 129 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 MiB/s wr, 47 op/s 2023-09-10T00:42:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:52 smithi044 ceph-mon[98676]: pgmap v69: 129 pgs: 129 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 MiB/s wr, 47 op/s 2023-09-10T00:42:53.159 INFO:teuthology.orchestra.run.smithi044.stderr:10000000 of 50000000 tuples (20%) done (elapsed 112.79 s, remaining 451.17 s) 2023-09-10T00:42:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.813+0000 7fe4d255c700 1 -- 172.21.15.44:0/1791470678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4cc106660 msgr2=0x7fe4cc106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.813+0000 7fe4d255c700 1 --2- 172.21.15.44:0/1791470678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4cc106660 0x7fe4cc106a40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0098f0 tx=0x7fe4bc009c00 comp rx=0 tx=0).stop 2023-09-10T00:42:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.813+0000 7fe4d255c700 1 -- 172.21.15.44:0/1791470678 shutdown_connections 2023-09-10T00:42:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.813+0000 7fe4d255c700 1 --2- 172.21.15.44:0/1791470678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc100f80 0x7fe4cc10d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.813+0000 7fe4d255c700 1 --2- 172.21.15.44:0/1791470678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4cc106660 0x7fe4cc106a40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.813+0000 7fe4d255c700 1 --2- 172.21.15.44:0/1791470678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4cc1005c0 0x7fe4cc100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.813+0000 7fe4d255c700 1 -- 172.21.15.44:0/1791470678 >> 172.21.15.44:0/1791470678 conn(0x7fe4cc0fc0d0 msgr2=0x7fe4cc0fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.813+0000 7fe4d255c700 1 -- 172.21.15.44:0/1791470678 shutdown_connections 2023-09-10T00:42:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.814+0000 7fe4d255c700 1 -- 172.21.15.44:0/1791470678 wait complete. 2023-09-10T00:42:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.814+0000 7fe4d255c700 1 Processor -- start 2023-09-10T00:42:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.814+0000 7fe4d255c700 1 -- start start 2023-09-10T00:42:53.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4d255c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4cc1005c0 0x7fe4cc19c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4d255c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4cc100f80 0x7fe4cc19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4d255c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc106660 0x7fe4cc196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4d255c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc196b10 con 0x7fe4cc100f80 2023-09-10T00:42:53.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4d255c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc196c80 con 0x7fe4cc106660 2023-09-10T00:42:53.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4d255c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc196e40 con 0x7fe4cc1005c0 2023-09-10T00:42:53.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4cbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4cc1005c0 0x7fe4cc19c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4cbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4cc1005c0 0x7fe4cc19c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57098/0 (socket says 172.21.15.44:57098) 2023-09-10T00:42:53.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4cbfff700 1 -- 172.21.15.44:0/2881781278 learned_addr learned my addr 172.21.15.44:0/2881781278 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:53.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4d0af9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc106660 0x7fe4cc196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4cb7fe700 1 --2- 172.21.15.44:0/2881781278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4cc100f80 0x7fe4cc19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4cbfff700 1 -- 172.21.15.44:0/2881781278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc106660 msgr2=0x7fe4cc196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:53.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4cbfff700 1 --2- 172.21.15.44:0/2881781278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc106660 0x7fe4cc196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4cbfff700 1 -- 172.21.15.44:0/2881781278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4cc100f80 msgr2=0x7fe4cc19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4cbfff700 1 --2- 172.21.15.44:0/2881781278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4cc100f80 0x7fe4cc19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4cbfff700 1 -- 172.21.15.44:0/2881781278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c00095f0 con 0x7fe4cc1005c0 2023-09-10T00:42:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4d0af9700 1 --2- 172.21.15.44:0/2881781278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc106660 0x7fe4cc196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.815+0000 7fe4cbfff700 1 --2- 172.21.15.44:0/2881781278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4cc1005c0 0x7fe4cc19c570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009230 tx=0x7fe4bc00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.816+0000 7fe4c97fa700 1 -- 172.21.15.44:0/2881781278 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc01c070 con 0x7fe4cc1005c0 2023-09-10T00:42:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.816+0000 7fe4c97fa700 1 -- 172.21.15.44:0/2881781278 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4bc00ba70 con 0x7fe4cc1005c0 2023-09-10T00:42:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.816+0000 7fe4d255c700 1 -- 172.21.15.44:0/2881781278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4bc009520 con 0x7fe4cc1005c0 2023-09-10T00:42:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.816+0000 7fe4c97fa700 1 -- 172.21.15.44:0/2881781278 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc004930 con 0x7fe4cc1005c0 2023-09-10T00:42:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.816+0000 7fe4d255c700 1 -- 172.21.15.44:0/2881781278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4cc06a200 con 0x7fe4cc1005c0 2023-09-10T00:42:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.819+0000 7fe4c97fa700 1 -- 172.21.15.44:0/2881781278 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fe4bc020070 con 0x7fe4cc1005c0 2023-09-10T00:42:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.819+0000 7fe4c97fa700 1 --2- 172.21.15.44:0/2881781278 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4b40a0830 0x7fe4b40a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4cb7fe700 1 --2- 172.21.15.44:0/2881781278 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4b40a0830 0x7fe4b40a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4c97fa700 1 -- 172.21.15.44:0/2881781278 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe4bc0ccee0 con 0x7fe4cc1005c0 2023-09-10T00:42:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4de722b80 1 Processor -- start 2023-09-10T00:42:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4de722b80 1 -- start start 2023-09-10T00:42:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4cb7fe700 1 --2- 172.21.15.44:0/2881781278 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4b40a0830 0x7fe4b40a2cf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc197df0 tx=0x7fe4c0009310 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4de722b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 0x55e43be01580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4de722b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfb300 0x55e43bdfb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4de722b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfbcc0 0x55e43be07f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4de722b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43bd546d0 con 0x55e43bdfbcc0 2023-09-10T00:42:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4de722b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43bd51ff0 con 0x55e43bdfb300 2023-09-10T00:42:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4de722b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43bd4d8e0 con 0x55e43be011a0 2023-09-10T00:42:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4d0af9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfbcc0 0x55e43be07f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4cb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfb300 0x55e43bdfb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4cbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 0x55e43be01580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4d0af9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfbcc0 0x55e43be07f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50228/0 (socket says 172.21.15.44:50228) 2023-09-10T00:42:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4cbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 0x55e43be01580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57106/0 (socket says 172.21.15.44:57106) 2023-09-10T00:42:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.820+0000 7fe4d0af9700 1 -- 172.21.15.44:0/3104808368 learned_addr learned my addr 172.21.15.44:0/3104808368 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.821+0000 7fe4cb7fe700 1 -- 172.21.15.44:0/3104808368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 msgr2=0x55e43be01580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.821+0000 7fe4cb7fe700 1 --2- 172.21.15.44:0/3104808368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 0x55e43be01580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.821+0000 7fe4cb7fe700 1 -- 172.21.15.44:0/3104808368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfbcc0 msgr2=0x55e43be07f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.821+0000 7fe4cb7fe700 1 --2- 172.21.15.44:0/3104808368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfbcc0 0x55e43be07f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.821+0000 7fe4cb7fe700 1 -- 172.21.15.44:0/3104808368 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e43bb79d80 con 0x55e43bdfb300 2023-09-10T00:42:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.821+0000 7fe4d0af9700 1 --2- 172.21.15.44:0/3104808368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfbcc0 0x55e43be07f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.821+0000 7fe4cbfff700 1 --2- 172.21.15.44:0/3104808368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 0x55e43be01580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.821+0000 7fe4cb7fe700 1 --2- 172.21.15.44:0/3104808368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfb300 0x55e43bdfb780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0005fd0 tx=0x7fe4c0025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73ceddc055475a2e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:53.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.821+0000 7fe4b27fc700 1 -- 172.21.15.44:0/3104808368 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0037430 con 0x55e43bdfb300 2023-09-10T00:42:53.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.821+0000 7fe4b27fc700 1 -- 172.21.15.44:0/3104808368 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4c0032070 con 0x55e43bdfb300 2023-09-10T00:42:53.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.821+0000 7fe4b27fc700 1 -- 172.21.15.44:0/3104808368 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c002e670 con 0x55e43bdfb300 2023-09-10T00:42:53.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.822+0000 7fe4de722b80 1 -- 172.21.15.44:0/3104808368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfb300 msgr2=0x55e43bdfb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:53.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.822+0000 7fe4de722b80 1 --2- 172.21.15.44:0/3104808368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfb300 0x55e43bdfb780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0005fd0 tx=0x7fe4c0025000 comp rx=0 tx=0).stop 2023-09-10T00:42:53.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.822+0000 7fe4de722b80 1 -- 172.21.15.44:0/3104808368 shutdown_connections 2023-09-10T00:42:53.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.822+0000 7fe4de722b80 1 --2- 172.21.15.44:0/3104808368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfbcc0 0x55e43be07f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.822+0000 7fe4de722b80 1 --2- 172.21.15.44:0/3104808368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 0x55e43be01580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.822+0000 7fe4de722b80 1 --2- 172.21.15.44:0/3104808368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfb300 0x55e43bdfb780 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.822+0000 7fe4de722b80 1 -- 172.21.15.44:0/3104808368 >> 172.21.15.44:0/3104808368 conn(0x55e43bd70030 msgr2=0x55e43bdf6560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.822+0000 7fe4de722b80 1 -- 172.21.15.44:0/3104808368 shutdown_connections 2023-09-10T00:42:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.822+0000 7fe4de722b80 1 -- 172.21.15.44:0/3104808368 wait complete. 2023-09-10T00:42:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.822+0000 7fe4de722b80 1 -- start start 2023-09-10T00:42:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.823+0000 7fe4de722b80 1 Processor -- start 2023-09-10T00:42:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.823+0000 7fe4de722b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfb300 0x55e43be07d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.823+0000 7fe4de722b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfbcc0 0x55e43be05070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.823+0000 7fe4de722b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 0x55e43be05610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.823+0000 7fe4de722b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43bcebb10 con 0x55e43bdfb300 2023-09-10T00:42:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.823+0000 7fe4de722b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43bceb810 con 0x55e43bdfbcc0 2023-09-10T00:42:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.823+0000 7fe4de722b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43bbcf8c0 con 0x55e43be011a0 2023-09-10T00:42:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.823+0000 7fe4de722b80 10 client.0 _ll_register_callbacks cb 0x55e43bc35a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:42:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.823+0000 7fe4cbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfb300 0x55e43be07d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.823+0000 7fe4cbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfb300 0x55e43be07d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50230/0 (socket says 172.21.15.44:50230) 2023-09-10T00:42:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.823+0000 7fe4cbfff700 1 -- 172.21.15.44:0/4176462165 learned_addr learned my addr 172.21.15.44:0/4176462165 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.823+0000 7fe4cb7fe700 1 --2- 172.21.15.44:0/4176462165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfbcc0 0x55e43be05070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.823+0000 7fe4d0af9700 1 --2- 172.21.15.44:0/4176462165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 0x55e43be05610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.823+0000 7fe4cbfff700 1 -- 172.21.15.44:0/4176462165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 msgr2=0x55e43be05610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.823+0000 7fe4cbfff700 1 --2- 172.21.15.44:0/4176462165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 0x55e43be05610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.824+0000 7fe4cbfff700 1 -- 172.21.15.44:0/4176462165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfbcc0 msgr2=0x55e43be05070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.824+0000 7fe4cbfff700 1 --2- 172.21.15.44:0/4176462165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfbcc0 0x55e43be05070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.824+0000 7fe4cbfff700 1 -- 172.21.15.44:0/4176462165 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c4003b20 con 0x55e43bdfb300 2023-09-10T00:42:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.824+0000 7fe4d0af9700 1 --2- 172.21.15.44:0/4176462165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 0x55e43be05610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.824+0000 7fe4cb7fe700 1 --2- 172.21.15.44:0/4176462165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfbcc0 0x55e43be05070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.824+0000 7fe4cbfff700 1 --2- 172.21.15.44:0/4176462165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfb300 0x55e43be07d80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc029000 tx=0x7fe4bc099a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6878401bbbc4910 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.824+0000 7fe4ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:42:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.824+0000 7fe4ab7fe700 1 -- 172.21.15.44:0/4176462165 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc005350 con 0x55e43bdfb300 2023-09-10T00:42:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.824+0000 7fe4ab7fe700 1 -- 172.21.15.44:0/4176462165 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe4bc09a080 con 0x55e43bdfb300 2023-09-10T00:42:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.824+0000 7fe4de722b80 1 -- 172.21.15.44:0/4176462165 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e43bca5490 con 0x55e43bdfb300 2023-09-10T00:42:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.824+0000 7fe4ab7fe700 1 -- 172.21.15.44:0/4176462165 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4bc0b2420 con 0x55e43bdfb300 2023-09-10T00:42:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.824+0000 7fe4de722b80 10 client.16404 fetch_fsmap learned FSMap version 34 2023-09-10T00:42:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.824+0000 7fe4de722b80 1 -- 172.21.15.44:0/4176462165 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fe4cc06a200 con 0x55e43bdfb300 2023-09-10T00:42:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.824+0000 7fe4ab7fe700 1 -- 172.21.15.44:0/4176462165 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e43bca5490 con 0x55e43bdfb300 2023-09-10T00:42:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.825+0000 7fe4ab7fe700 1 -- 172.21.15.44:0/4176462165 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe4bc099570 con 0x55e43bdfb300 2023-09-10T00:42:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.825+0000 7fe4de722b80 10 client.16404 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:42:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.825+0000 7fe4de722b80 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:42:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.825+0000 7fe4de722b80 20 client.16404 populate_metadata read hostname 'smithi044' 2023-09-10T00:42:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.825+0000 7fe4de722b80 1 --2- 172.21.15.44:0/4176462165 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e43be150f0 0x55e43be17510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.825+0000 7fe4de722b80 4 client.16404 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:42:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.825+0000 7fe4de722b80 1 -- 172.21.15.44:0/4176462165 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e43bd62840 con 0x55e43be150f0 2023-09-10T00:42:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.825+0000 7fe4cb7fe700 1 --2- 172.21.15.44:0/4176462165 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e43be150f0 0x55e43be17510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.825+0000 7fe4ab7fe700 10 client.16404 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.825+0000 7fe4cb7fe700 1 --2- 172.21.15.44:0/4176462165 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e43be150f0 0x55e43be17510 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c262c2783a12f691 server_cookie=9f57019d14fc7f04 in_seq=0 out_seq=0 2023-09-10T00:42:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.826+0000 7fe4ab7fe700 1 -- 172.21.15.44:0/4176462165 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e43bd62840 con 0x55e43be150f0 2023-09-10T00:42:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.826+0000 7fe4ab7fe700 10 client.16404 handle_command_reply: tid=0 2023-09-10T00:42:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.826+0000 7fe4de722b80 1 client.16404 shutdown 2023-09-10T00:42:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.826+0000 7fe4de722b80 1 -- 172.21.15.44:0/4176462165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfb300 msgr2=0x55e43be07d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.826+0000 7fe4de722b80 1 --2- 172.21.15.44:0/4176462165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfb300 0x55e43be07d80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc029000 tx=0x7fe4bc099a60 comp rx=0 tx=0).stop 2023-09-10T00:42:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.826+0000 7fe4de722b80 1 -- 172.21.15.44:0/4176462165 shutdown_connections 2023-09-10T00:42:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.826+0000 7fe4de722b80 1 --2- 172.21.15.44:0/4176462165 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e43be150f0 0x55e43be17510 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.826+0000 7fe4de722b80 1 --2- 172.21.15.44:0/4176462165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 0x55e43be05610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.826+0000 7fe4de722b80 1 --2- 172.21.15.44:0/4176462165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfb300 0x55e43be07d80 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.826+0000 7fe4ab7fe700 0 client.16404 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.826+0000 7fe4de722b80 1 --2- 172.21.15.44:0/4176462165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfbcc0 0x55e43be05070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.826+0000 7fe4de722b80 1 -- 172.21.15.44:0/4176462165 >> 172.21.15.44:0/4176462165 conn(0x55e43bd70030 msgr2=0x55e43bdf57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.826+0000 7fe4de722b80 1 -- 172.21.15.44:0/4176462165 shutdown_connections 2023-09-10T00:42:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.826+0000 7fe4de722b80 1 -- 172.21.15.44:0/4176462165 wait complete. 2023-09-10T00:42:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.827+0000 7fe4de722b80 20 client.16404 trim_cache size 0 max 16384 2023-09-10T00:42:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.846+0000 7fe4de722b80 1 Processor -- start 2023-09-10T00:42:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.846+0000 7fe4de722b80 1 -- start start 2023-09-10T00:42:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4de722b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43bdfb300 0x55e43be07920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4de722b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfbcc0 0x55e43be04650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4de722b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43be011a0 0x55e43be04b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4de722b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43be065e0 con 0x55e43be011a0 2023-09-10T00:42:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4de722b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43bdf7d10 con 0x55e43bdfbcc0 2023-09-10T00:42:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4de722b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43bdffd60 con 0x55e43bdfb300 2023-09-10T00:42:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4cbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43bdfb300 0x55e43be07920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4d0af9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43be011a0 0x55e43be04b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4cb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfbcc0 0x55e43be04650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4cbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43bdfb300 0x55e43be07920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57120/0 (socket says 172.21.15.44:57120) 2023-09-10T00:42:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4d0af9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43be011a0 0x55e43be04b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50242/0 (socket says 172.21.15.44:50242) 2023-09-10T00:42:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4cbfff700 1 -- 172.21.15.44:0/3335847544 learned_addr learned my addr 172.21.15.44:0/3335847544 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4cb7fe700 1 -- 172.21.15.44:0/3335847544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43bdfb300 msgr2=0x55e43be07920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4cb7fe700 1 --2- 172.21.15.44:0/3335847544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43bdfb300 0x55e43be07920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4cb7fe700 1 -- 172.21.15.44:0/3335847544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43be011a0 msgr2=0x55e43be04b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4cb7fe700 1 --2- 172.21.15.44:0/3335847544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43be011a0 0x55e43be04b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4cb7fe700 1 -- 172.21.15.44:0/3335847544 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c0028070 con 0x55e43bdfbcc0 2023-09-10T00:42:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4cbfff700 1 --2- 172.21.15.44:0/3335847544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43bdfb300 0x55e43be07920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.847+0000 7fe4cb7fe700 1 --2- 172.21.15.44:0/3335847544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfbcc0 0x55e43be04650 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55e43be065b0 tx=0x7fe4c0005af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c44670fac856676 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.848+0000 7fe4b2ffd700 1 -- 172.21.15.44:0/3335847544 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0047040 con 0x55e43bdfbcc0 2023-09-10T00:42:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.848+0000 7fe4b2ffd700 1 -- 172.21.15.44:0/3335847544 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4c001fe60 con 0x55e43bdfbcc0 2023-09-10T00:42:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.848+0000 7fe4b2ffd700 1 -- 172.21.15.44:0/3335847544 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0005040 con 0x55e43bdfbcc0 2023-09-10T00:42:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.848+0000 7fe4de722b80 1 -- 172.21.15.44:0/3335847544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfbcc0 msgr2=0x55e43be04650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.848+0000 7fe4de722b80 1 --2- 172.21.15.44:0/3335847544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfbcc0 0x55e43be04650 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55e43be065b0 tx=0x7fe4c0005af0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.848+0000 7fe4de722b80 1 -- 172.21.15.44:0/3335847544 shutdown_connections 2023-09-10T00:42:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.848+0000 7fe4de722b80 1 --2- 172.21.15.44:0/3335847544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43be011a0 0x55e43be04b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.848+0000 7fe4de722b80 1 --2- 172.21.15.44:0/3335847544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43bdfb300 0x55e43be07920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.848+0000 7fe4de722b80 1 --2- 172.21.15.44:0/3335847544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfbcc0 0x55e43be04650 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.849+0000 7fe4de722b80 1 -- 172.21.15.44:0/3335847544 >> 172.21.15.44:0/3335847544 conn(0x55e43bd70030 msgr2=0x55e43bdf6090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.849+0000 7fe4de722b80 1 -- 172.21.15.44:0/3335847544 shutdown_connections 2023-09-10T00:42:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.849+0000 7fe4de722b80 1 -- 172.21.15.44:0/3335847544 wait complete. 2023-09-10T00:42:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.849+0000 7fe4de722b80 1 -- start start 2023-09-10T00:42:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.849+0000 7fe4de722b80 1 Processor -- start 2023-09-10T00:42:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.849+0000 7fe4de722b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfb300 0x55e43be0c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.849+0000 7fe4de722b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfbcc0 0x55e43be0a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4de722b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 0x55e43be0aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4de722b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43be0b5b0 con 0x55e43bdfbcc0 2023-09-10T00:42:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4de722b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43be0b720 con 0x55e43bdfb300 2023-09-10T00:42:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4de722b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43be68ef0 con 0x55e43be011a0 2023-09-10T00:42:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4de722b80 10 client.0 _ll_register_callbacks cb 0x55e43bc35a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:42:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4cbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfb300 0x55e43be0c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4cb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfbcc0 0x55e43be0a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4cbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfb300 0x55e43be0c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51984/0 (socket says 172.21.15.44:51984) 2023-09-10T00:42:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4d0af9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 0x55e43be0aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4cbfff700 1 -- 172.21.15.44:0/1549574934 learned_addr learned my addr 172.21.15.44:0/1549574934 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:42:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4cbfff700 1 -- 172.21.15.44:0/1549574934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 msgr2=0x55e43be0aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4cbfff700 1 --2- 172.21.15.44:0/1549574934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 0x55e43be0aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4cbfff700 1 -- 172.21.15.44:0/1549574934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfbcc0 msgr2=0x55e43be0a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:42:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4cbfff700 1 --2- 172.21.15.44:0/1549574934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfbcc0 0x55e43be0a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4cbfff700 1 -- 172.21.15.44:0/1549574934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4cc06a200 con 0x55e43bdfb300 2023-09-10T00:42:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4cb7fe700 1 --2- 172.21.15.44:0/1549574934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfbcc0 0x55e43be0a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4d0af9700 1 --2- 172.21.15.44:0/1549574934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 0x55e43be0aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:42:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4cbfff700 1 --2- 172.21.15.44:0/1549574934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfb300 0x55e43be0c520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0ca890 tx=0x7fe4bc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa0c2c57023928c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:42:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:42:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4ab7fe700 1 -- 172.21.15.44:0/1549574934 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc0b2430 con 0x55e43bdfb300 2023-09-10T00:42:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4ab7fe700 1 -- 172.21.15.44:0/1549574934 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe4bc083070 con 0x55e43bdfb300 2023-09-10T00:42:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.850+0000 7fe4de722b80 1 -- 172.21.15.44:0/1549574934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4c0032070 con 0x55e43bdfb300 2023-09-10T00:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.851+0000 7fe4ab7fe700 1 -- 172.21.15.44:0/1549574934 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4bc005350 con 0x55e43bdfb300 2023-09-10T00:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.851+0000 7fe4de722b80 10 client.26078 fetch_fsmap learned FSMap version 34 2023-09-10T00:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.851+0000 7fe4de722b80 1 -- 172.21.15.44:0/1549574934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e43bb79d80 con 0x55e43bdfb300 2023-09-10T00:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.851+0000 7fe4ab7fe700 1 -- 172.21.15.44:0/1549574934 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4bc0ad020 con 0x55e43bdfb300 2023-09-10T00:42:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.851+0000 7fe4ab7fe700 1 -- 172.21.15.44:0/1549574934 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe4bc0a3070 con 0x55e43bdfb300 2023-09-10T00:42:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.851+0000 7fe4de722b80 10 client.26078 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:42:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.851+0000 7fe4de722b80 10 client.26078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:42:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.851+0000 7fe4de722b80 20 client.26078 populate_metadata read hostname 'smithi044' 2023-09-10T00:42:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.851+0000 7fe4de722b80 1 --2- 172.21.15.44:0/1549574934 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e43be6a2c0 0x55e43be6c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:42:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.851+0000 7fe4de722b80 4 client.26078 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:42:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.851+0000 7fe4de722b80 1 -- 172.21.15.44:0/1549574934 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e43bd468c0 con 0x55e43be6a2c0 2023-09-10T00:42:53.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.851+0000 7fe4cb7fe700 1 --2- 172.21.15.44:0/1549574934 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e43be6a2c0 0x55e43be6c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:42:53.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.852+0000 7fe4ab7fe700 10 client.26078 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:53.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.852+0000 7fe4cb7fe700 1 --2- 172.21.15.44:0/1549574934 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e43be6a2c0 0x55e43be6c6e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f0508ad0ae5ac20 server_cookie=850e075aa3d3c3af in_seq=0 out_seq=0 2023-09-10T00:42:53.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.866+0000 7fe4ab7fe700 1 -- 172.21.15.44:0/1549574934 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e43bd468c0 con 0x55e43be6a2c0 2023-09-10T00:42:53.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.866+0000 7fe4ab7fe700 10 client.26078 handle_command_reply: tid=0 2023-09-10T00:42:53.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.866+0000 7fe4de722b80 1 client.26078 shutdown 2023-09-10T00:42:53.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.866+0000 7fe4de722b80 1 -- 172.21.15.44:0/1549574934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfb300 msgr2=0x55e43be0c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:53.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.866+0000 7fe4de722b80 1 --2- 172.21.15.44:0/1549574934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfb300 0x55e43be0c520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0ca890 tx=0x7fe4bc009c00 comp rx=0 tx=0).stop 2023-09-10T00:42:53.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.866+0000 7fe4de722b80 1 -- 172.21.15.44:0/1549574934 shutdown_connections 2023-09-10T00:42:53.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.866+0000 7fe4de722b80 1 --2- 172.21.15.44:0/1549574934 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e43be6a2c0 0x55e43be6c6e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.866+0000 7fe4de722b80 1 --2- 172.21.15.44:0/1549574934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e43be011a0 0x55e43be0aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.867+0000 7fe4de722b80 1 --2- 172.21.15.44:0/1549574934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e43bdfb300 0x55e43be0c520 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.866+0000 7fe4ab7fe700 0 client.26078 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:42:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.867+0000 7fe4de722b80 1 --2- 172.21.15.44:0/1549574934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e43bdfbcc0 0x55e43be0a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.867+0000 7fe4de722b80 1 -- 172.21.15.44:0/1549574934 >> 172.21.15.44:0/1549574934 conn(0x55e43bd70030 msgr2=0x55e43bdfa2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.867+0000 7fe4de722b80 1 -- 172.21.15.44:0/1549574934 shutdown_connections 2023-09-10T00:42:53.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.867+0000 7fe4de722b80 1 -- 172.21.15.44:0/1549574934 wait complete. 2023-09-10T00:42:53.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.867+0000 7fe4de722b80 20 client.26078 trim_cache size 0 max 16384 2023-09-10T00:42:53.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:42:53.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:42:53.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:42:53.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:42:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.867+0000 7fe4d255c700 1 -- 172.21.15.44:0/2881781278 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4b40a0830 msgr2=0x7fe4b40a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.867+0000 7fe4d255c700 1 --2- 172.21.15.44:0/2881781278 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4b40a0830 0x7fe4b40a2cf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc197df0 tx=0x7fe4c0009310 comp rx=0 tx=0).stop 2023-09-10T00:42:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.867+0000 7fe4d255c700 1 -- 172.21.15.44:0/2881781278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4cc1005c0 msgr2=0x7fe4cc19c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:42:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.867+0000 7fe4d255c700 1 --2- 172.21.15.44:0/2881781278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4cc1005c0 0x7fe4cc19c570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009230 tx=0x7fe4bc00fa00 comp rx=0 tx=0).stop 2023-09-10T00:42:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.867+0000 7fe4d255c700 1 -- 172.21.15.44:0/2881781278 shutdown_connections 2023-09-10T00:42:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.867+0000 7fe4d255c700 1 --2- 172.21.15.44:0/2881781278 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4b40a0830 0x7fe4b40a2cf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.867+0000 7fe4d255c700 1 --2- 172.21.15.44:0/2881781278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc106660 0x7fe4cc196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.867+0000 7fe4d255c700 1 --2- 172.21.15.44:0/2881781278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4cc1005c0 0x7fe4cc19c570 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.867+0000 7fe4d255c700 1 --2- 172.21.15.44:0/2881781278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4cc100f80 0x7fe4cc19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:42:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.867+0000 7fe4d255c700 1 -- 172.21.15.44:0/2881781278 >> 172.21.15.44:0/2881781278 conn(0x7fe4cc0fc0d0 msgr2=0x7fe4cc0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:42:53.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.868+0000 7fe4d255c700 1 -- 172.21.15.44:0/2881781278 shutdown_connections 2023-09-10T00:42:53.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:42:53.868+0000 7fe4d255c700 1 -- 172.21.15.44:0/2881781278 wait complete. 2023-09-10T00:42:54.070 INFO:teuthology.orchestra.run.smithi044.stderr:10100000 of 50000000 tuples (20%) done (elapsed 113.70 s, remaining 449.19 s) 2023-09-10T00:42:54.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:54 smithi099 ceph-mon[102692]: pgmap v70: 129 pgs: 129 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 MiB/s wr, 32 op/s 2023-09-10T00:42:54.912 INFO:teuthology.orchestra.run.smithi044.stderr:10200000 of 50000000 tuples (20%) done (elapsed 114.55 s, remaining 446.96 s) 2023-09-10T00:42:55.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:54 smithi131 ceph-mon[103713]: pgmap v70: 129 pgs: 129 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 MiB/s wr, 32 op/s 2023-09-10T00:42:55.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:54 smithi044 ceph-mon[98676]: pgmap v70: 129 pgs: 129 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 MiB/s wr, 32 op/s 2023-09-10T00:42:55.761 INFO:teuthology.orchestra.run.smithi044.stderr:10300000 of 50000000 tuples (20%) done (elapsed 115.40 s, remaining 444.78 s) 2023-09-10T00:42:56.578 INFO:teuthology.orchestra.run.smithi044.stderr:10400000 of 50000000 tuples (20%) done (elapsed 116.21 s, remaining 442.50 s) 2023-09-10T00:42:56.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:56 smithi099 ceph-mon[102692]: pgmap v71: 129 pgs: 129 active+clean; 2.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 47 MiB/s wr, 64 op/s 2023-09-10T00:42:57.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:56 smithi131 ceph-mon[103713]: pgmap v71: 129 pgs: 129 active+clean; 2.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 47 MiB/s wr, 64 op/s 2023-09-10T00:42:57.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:56 smithi044 ceph-mon[98676]: pgmap v71: 129 pgs: 129 active+clean; 2.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 47 MiB/s wr, 64 op/s 2023-09-10T00:42:57.480 INFO:teuthology.orchestra.run.smithi044.stderr:10500000 of 50000000 tuples (21%) done (elapsed 117.11 s, remaining 440.57 s) 2023-09-10T00:42:58.317 INFO:teuthology.orchestra.run.smithi044.stderr:10600000 of 50000000 tuples (21%) done (elapsed 117.95 s, remaining 438.42 s) 2023-09-10T00:42:58.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:42:58 smithi099 ceph-mon[102692]: pgmap v72: 129 pgs: 129 active+clean; 2.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 37 MiB/s wr, 51 op/s 2023-09-10T00:42:59.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:42:58 smithi131 ceph-mon[103713]: pgmap v72: 129 pgs: 129 active+clean; 2.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 37 MiB/s wr, 51 op/s 2023-09-10T00:42:59.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:42:58 smithi044 ceph-mon[98676]: pgmap v72: 129 pgs: 129 active+clean; 2.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 37 MiB/s wr, 51 op/s 2023-09-10T00:42:59.146 INFO:teuthology.orchestra.run.smithi044.stderr:10700000 of 50000000 tuples (21%) done (elapsed 118.78 s, remaining 436.27 s) 2023-09-10T00:43:00.061 INFO:teuthology.orchestra.run.smithi044.stderr:10800000 of 50000000 tuples (21%) done (elapsed 119.69 s, remaining 434.45 s) 2023-09-10T00:43:00.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:00 smithi099 ceph-mon[102692]: pgmap v73: 129 pgs: 129 active+clean; 2.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 33 MiB/s wr, 48 op/s 2023-09-10T00:43:00.955 INFO:teuthology.orchestra.run.smithi044.stderr:10900000 of 50000000 tuples (21%) done (elapsed 120.59 s, remaining 432.57 s) 2023-09-10T00:43:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:00 smithi131 ceph-mon[103713]: pgmap v73: 129 pgs: 129 active+clean; 2.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 33 MiB/s wr, 48 op/s 2023-09-10T00:43:01.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:00 smithi044 ceph-mon[98676]: pgmap v73: 129 pgs: 129 active+clean; 2.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 33 MiB/s wr, 48 op/s 2023-09-10T00:43:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:11000000 of 50000000 tuples (22%) done (elapsed 121.46 s, remaining 430.65 s) 2023-09-10T00:43:02.721 INFO:teuthology.orchestra.run.smithi044.stderr:11100000 of 50000000 tuples (22%) done (elapsed 122.36 s, remaining 428.80 s) 2023-09-10T00:43:02.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:02 smithi099 ceph-mon[102692]: pgmap v74: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 51 MiB/s wr, 61 op/s 2023-09-10T00:43:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:02 smithi131 ceph-mon[103713]: pgmap v74: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 51 MiB/s wr, 61 op/s 2023-09-10T00:43:03.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:02 smithi044 ceph-mon[98676]: pgmap v74: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 51 MiB/s wr, 61 op/s 2023-09-10T00:43:03.591 INFO:teuthology.orchestra.run.smithi044.stderr:11200000 of 50000000 tuples (22%) done (elapsed 123.23 s, remaining 426.89 s) 2023-09-10T00:43:04.435 INFO:teuthology.orchestra.run.smithi044.stderr:11300000 of 50000000 tuples (22%) done (elapsed 124.07 s, remaining 424.91 s) 2023-09-10T00:43:04.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:04 smithi099 ceph-mon[102692]: pgmap v75: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 33 MiB/s wr, 46 op/s 2023-09-10T00:43:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:04 smithi131 ceph-mon[103713]: pgmap v75: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 33 MiB/s wr, 46 op/s 2023-09-10T00:43:05.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:04 smithi044 ceph-mon[98676]: pgmap v75: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 33 MiB/s wr, 46 op/s 2023-09-10T00:43:05.310 INFO:teuthology.orchestra.run.smithi044.stderr:11400000 of 50000000 tuples (22%) done (elapsed 124.94 s, remaining 423.06 s) 2023-09-10T00:43:06.194 INFO:teuthology.orchestra.run.smithi044.stderr:11500000 of 50000000 tuples (23%) done (elapsed 125.83 s, remaining 421.25 s) 2023-09-10T00:43:06.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:06 smithi099 ceph-mon[102692]: pgmap v76: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 50 MiB/s wr, 74 op/s 2023-09-10T00:43:06.896 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:43:06.926 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:06 smithi044 ceph-mon[98676]: pgmap v76: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 50 MiB/s wr, 74 op/s 2023-09-10T00:43:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.005+0000 7fdd8ada2700 1 -- 172.21.15.44:0/1218640736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd840ff4a0 msgr2=0x7fdd840ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.005+0000 7fdd8ada2700 1 --2- 172.21.15.44:0/1218640736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd840ff4a0 0x7fdd840ff920 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdd78009990 tx=0x7fdd78009ca0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.005+0000 7fdd8ada2700 1 -- 172.21.15.44:0/1218640736 shutdown_connections 2023-09-10T00:43:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.005+0000 7fdd8ada2700 1 --2- 172.21.15.44:0/1218640736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd840ffe60 0x7fdd84068550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.005+0000 7fdd8ada2700 1 --2- 172.21.15.44:0/1218640736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd84105560 0x7fdd84105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.005+0000 7fdd8ada2700 1 --2- 172.21.15.44:0/1218640736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd840ff4a0 0x7fdd840ff920 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.005+0000 7fdd8ada2700 1 -- 172.21.15.44:0/1218640736 >> 172.21.15.44:0/1218640736 conn(0x7fdd840faf30 msgr2=0x7fdd840fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.005+0000 7fdd8ada2700 1 -- 172.21.15.44:0/1218640736 shutdown_connections 2023-09-10T00:43:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.005+0000 7fdd8ada2700 1 -- 172.21.15.44:0/1218640736 wait complete. 2023-09-10T00:43:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.006+0000 7fdd8ada2700 1 Processor -- start 2023-09-10T00:43:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.006+0000 7fdd8ada2700 1 -- start start 2023-09-10T00:43:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.006+0000 7fdd8ada2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd840ff4a0 0x7fdd8419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.006+0000 7fdd8ada2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd840ffe60 0x7fdd8419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.006+0000 7fdd8ada2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd84105560 0x7fdd841986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.006+0000 7fdd8ada2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd84198d20 con 0x7fdd84105560 2023-09-10T00:43:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.006+0000 7fdd8ada2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd84198e90 con 0x7fdd840ffe60 2023-09-10T00:43:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.006+0000 7fdd8ada2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd84199050 con 0x7fdd840ff4a0 2023-09-10T00:43:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.006+0000 7fdd83fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd840ffe60 0x7fdd8419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.006+0000 7fdd88b3e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd840ff4a0 0x7fdd8419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.007+0000 7fdd83fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd840ffe60 0x7fdd8419ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51998/0 (socket says 172.21.15.44:51998) 2023-09-10T00:43:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.007+0000 7fdd88b3e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd840ff4a0 0x7fdd8419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57138/0 (socket says 172.21.15.44:57138) 2023-09-10T00:43:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.007+0000 7fdd83fff700 1 -- 172.21.15.44:0/2453292276 learned_addr learned my addr 172.21.15.44:0/2453292276 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.007+0000 7fdd8933f700 1 --2- 172.21.15.44:0/2453292276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd84105560 0x7fdd841986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.007+0000 7fdd83fff700 1 -- 172.21.15.44:0/2453292276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd840ff4a0 msgr2=0x7fdd8419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.007+0000 7fdd83fff700 1 --2- 172.21.15.44:0/2453292276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd840ff4a0 0x7fdd8419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.007+0000 7fdd83fff700 1 -- 172.21.15.44:0/2453292276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd84105560 msgr2=0x7fdd841986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.007+0000 7fdd83fff700 1 --2- 172.21.15.44:0/2453292276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd84105560 0x7fdd841986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.007+0000 7fdd83fff700 1 -- 172.21.15.44:0/2453292276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd780095f0 con 0x7fdd840ffe60 2023-09-10T00:43:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.007+0000 7fdd88b3e700 1 --2- 172.21.15.44:0/2453292276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd840ff4a0 0x7fdd8419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.007+0000 7fdd83fff700 1 --2- 172.21.15.44:0/2453292276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd840ffe60 0x7fdd8419ed90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdd78004800 tx=0x7fdd78003920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.007+0000 7fdd81ffb700 1 -- 172.21.15.44:0/2453292276 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7801d070 con 0x7fdd840ffe60 2023-09-10T00:43:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.007+0000 7fdd8ada2700 1 -- 172.21.15.44:0/2453292276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd841992d0 con 0x7fdd840ffe60 2023-09-10T00:43:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.007+0000 7fdd81ffb700 1 -- 172.21.15.44:0/2453292276 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd7800bdb0 con 0x7fdd840ffe60 2023-09-10T00:43:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.007+0000 7fdd81ffb700 1 -- 172.21.15.44:0/2453292276 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78005280 con 0x7fdd840ffe60 2023-09-10T00:43:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.008+0000 7fdd8ada2700 1 -- 172.21.15.44:0/2453292276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd841a5840 con 0x7fdd840ffe60 2023-09-10T00:43:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.010+0000 7fdd81ffb700 1 -- 172.21.15.44:0/2453292276 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fdd7800b920 con 0x7fdd840ffe60 2023-09-10T00:43:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.011+0000 7fdd81ffb700 1 --2- 172.21.15.44:0/2453292276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdd6c0a07b0 0x7fdd6c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.011+0000 7fdd88b3e700 1 --2- 172.21.15.44:0/2453292276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdd6c0a07b0 0x7fdd6c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.011+0000 7fdd81ffb700 1 -- 172.21.15.44:0/2453292276 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fdd780cd490 con 0x7fdd840ffe60 2023-09-10T00:43:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.012+0000 7fdd96f68b80 1 Processor -- start 2023-09-10T00:43:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.012+0000 7fdd96f68b80 1 -- start start 2023-09-10T00:43:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.012+0000 7fdd88b3e700 1 --2- 172.21.15.44:0/2453292276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdd6c0a07b0 0x7fdd6c0a2c70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdd74005950 tx=0x7fdd7400b410 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.012+0000 7fdd96f68b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac1fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.012+0000 7fdd96f68b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac19b10 0x563f8ac19f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.012+0000 7fdd96f68b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac1a4d0 0x563f8ac26740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.012+0000 7fdd96f68b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8ab6a6d0 con 0x563f8ac19b10 2023-09-10T00:43:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.012+0000 7fdd96f68b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8ab67ff0 con 0x563f8ac1f9b0 2023-09-10T00:43:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.012+0000 7fdd96f68b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8ab638e0 con 0x563f8ac1a4d0 2023-09-10T00:43:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.012+0000 7fdd83fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac19b10 0x563f8ac19f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.012+0000 7fdd88b3e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac1fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.012+0000 7fdd83fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac19b10 0x563f8ac19f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50266/0 (socket says 172.21.15.44:50266) 2023-09-10T00:43:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.012+0000 7fdd88b3e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac1fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52004/0 (socket says 172.21.15.44:52004) 2023-09-10T00:43:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.012+0000 7fdd83fff700 1 -- 172.21.15.44:0/3420702033 learned_addr learned my addr 172.21.15.44:0/3420702033 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.012+0000 7fdd8933f700 1 --2- 172.21.15.44:0/3420702033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac1a4d0 0x563f8ac26740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.013+0000 7fdd8933f700 1 -- 172.21.15.44:0/3420702033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 msgr2=0x563f8ac1fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.013+0000 7fdd8933f700 1 --2- 172.21.15.44:0/3420702033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac1fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.013+0000 7fdd8933f700 1 -- 172.21.15.44:0/3420702033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac19b10 msgr2=0x563f8ac19f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.013+0000 7fdd8933f700 1 --2- 172.21.15.44:0/3420702033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac19b10 0x563f8ac19f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.013+0000 7fdd8933f700 1 -- 172.21.15.44:0/3420702033 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f8a98fd80 con 0x563f8ac1a4d0 2023-09-10T00:43:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.013+0000 7fdd8933f700 1 --2- 172.21.15.44:0/3420702033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac1a4d0 0x563f8ac26740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c011550 tx=0x7fdd7c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e441cf2ea80e6b36 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.013+0000 7fdd6affd700 1 -- 172.21.15.44:0/3420702033 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c00e040 con 0x563f8ac1a4d0 2023-09-10T00:43:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.013+0000 7fdd6affd700 1 -- 172.21.15.44:0/3420702033 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd7c00f040 con 0x563f8ac1a4d0 2023-09-10T00:43:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.013+0000 7fdd6affd700 1 -- 172.21.15.44:0/3420702033 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c004270 con 0x563f8ac1a4d0 2023-09-10T00:43:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.014+0000 7fdd96f68b80 1 -- 172.21.15.44:0/3420702033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac1a4d0 msgr2=0x563f8ac26740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.014+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/3420702033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac1a4d0 0x563f8ac26740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c011550 tx=0x7fdd7c009920 comp rx=0 tx=0).stop 2023-09-10T00:43:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.014+0000 7fdd96f68b80 1 -- 172.21.15.44:0/3420702033 shutdown_connections 2023-09-10T00:43:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.014+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/3420702033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac1a4d0 0x563f8ac26740 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.014+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/3420702033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac1fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.014+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/3420702033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac19b10 0x563f8ac19f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.014+0000 7fdd96f68b80 1 -- 172.21.15.44:0/3420702033 >> 172.21.15.44:0/3420702033 conn(0x563f8ab86030 msgr2=0x563f8ac0c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.014+0000 7fdd96f68b80 1 -- 172.21.15.44:0/3420702033 shutdown_connections 2023-09-10T00:43:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.014+0000 7fdd96f68b80 1 -- 172.21.15.44:0/3420702033 wait complete. 2023-09-10T00:43:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.015+0000 7fdd96f68b80 1 -- start start 2023-09-10T00:43:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.015+0000 7fdd96f68b80 1 Processor -- start 2023-09-10T00:43:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.015+0000 7fdd96f68b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac19b10 0x563f8ac26520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.015+0000 7fdd96f68b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac1a4d0 0x563f8ac23840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.015+0000 7fdd96f68b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac23d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.015+0000 7fdd96f68b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8ab01b10 con 0x563f8ac19b10 2023-09-10T00:43:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.015+0000 7fdd96f68b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8ab01810 con 0x563f8ac1f9b0 2023-09-10T00:43:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.015+0000 7fdd96f68b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8a9e58c0 con 0x563f8ac1a4d0 2023-09-10T00:43:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.015+0000 7fdd96f68b80 10 client.0 _ll_register_callbacks cb 0x563f8aa4ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:43:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.015+0000 7fdd83fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac1a4d0 0x563f8ac23840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.015+0000 7fdd88b3e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac19b10 0x563f8ac26520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.015+0000 7fdd83fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac1a4d0 0x563f8ac23840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57152/0 (socket says 172.21.15.44:57152) 2023-09-10T00:43:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.015+0000 7fdd83fff700 1 -- 172.21.15.44:0/863782898 learned_addr learned my addr 172.21.15.44:0/863782898 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.015+0000 7fdd8933f700 1 --2- 172.21.15.44:0/863782898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac23d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.016+0000 7fdd8933f700 1 -- 172.21.15.44:0/863782898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac1a4d0 msgr2=0x563f8ac23840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.016+0000 7fdd8933f700 1 --2- 172.21.15.44:0/863782898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac1a4d0 0x563f8ac23840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.016+0000 7fdd8933f700 1 -- 172.21.15.44:0/863782898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac19b10 msgr2=0x563f8ac26520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.016+0000 7fdd8933f700 1 --2- 172.21.15.44:0/863782898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac19b10 0x563f8ac26520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.016+0000 7fdd8933f700 1 -- 172.21.15.44:0/863782898 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7c011230 con 0x563f8ac1f9b0 2023-09-10T00:43:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.016+0000 7fdd83fff700 1 --2- 172.21.15.44:0/863782898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac1a4d0 0x563f8ac23840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.016+0000 7fdd88b3e700 1 --2- 172.21.15.44:0/863782898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac19b10 0x563f8ac26520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.016+0000 7fdd8933f700 1 --2- 172.21.15.44:0/863782898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac23d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c000f80 tx=0x7fdd7c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db0ec7276232391b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.016+0000 7fdd63fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:43:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.016+0000 7fdd63fff700 1 -- 172.21.15.44:0/863782898 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c00e040 con 0x563f8ac1f9b0 2023-09-10T00:43:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.016+0000 7fdd63fff700 1 -- 172.21.15.44:0/863782898 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fdd7c004040 con 0x563f8ac1f9b0 2023-09-10T00:43:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.016+0000 7fdd96f68b80 1 -- 172.21.15.44:0/863782898 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f8aabb490 con 0x563f8ac1f9b0 2023-09-10T00:43:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.016+0000 7fdd63fff700 1 -- 172.21.15.44:0/863782898 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd7c003e90 con 0x563f8ac1f9b0 2023-09-10T00:43:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.016+0000 7fdd96f68b80 10 client.26090 fetch_fsmap learned FSMap version 34 2023-09-10T00:43:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.016+0000 7fdd96f68b80 1 -- 172.21.15.44:0/863782898 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563f8a978c00 con 0x563f8ac1f9b0 2023-09-10T00:43:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.017+0000 7fdd63fff700 1 -- 172.21.15.44:0/863782898 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563f8aabb490 con 0x563f8ac1f9b0 2023-09-10T00:43:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.017+0000 7fdd63fff700 1 -- 172.21.15.44:0/863782898 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fdd7c01b020 con 0x563f8ac1f9b0 2023-09-10T00:43:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.017+0000 7fdd96f68b80 10 client.26090 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:43:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.017+0000 7fdd96f68b80 10 client.26090 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:43:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.017+0000 7fdd96f68b80 20 client.26090 populate_metadata read hostname 'smithi044' 2023-09-10T00:43:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.017+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/863782898 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563f8ac33ad0 0x563f8ac35ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.017+0000 7fdd96f68b80 4 client.26090 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:43:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.017+0000 7fdd96f68b80 1 -- 172.21.15.44:0/863782898 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f8ab78840 con 0x563f8ac33ad0 2023-09-10T00:43:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.017+0000 7fdd88b3e700 1 --2- 172.21.15.44:0/863782898 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563f8ac33ad0 0x563f8ac35ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.018+0000 7fdd88b3e700 1 --2- 172.21.15.44:0/863782898 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563f8ac33ad0 0x563f8ac35ef0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=537b52b4bcf9e38d server_cookie=1e92d9ef7bf8eebe in_seq=0 out_seq=0 2023-09-10T00:43:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.018+0000 7fdd63fff700 10 client.26090 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.018+0000 7fdd63fff700 1 -- 172.21.15.44:0/863782898 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563f8ab78840 con 0x563f8ac33ad0 2023-09-10T00:43:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.018+0000 7fdd63fff700 10 client.26090 handle_command_reply: tid=0 2023-09-10T00:43:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.018+0000 7fdd96f68b80 1 client.26090 shutdown 2023-09-10T00:43:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.018+0000 7fdd96f68b80 1 -- 172.21.15.44:0/863782898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 msgr2=0x563f8ac23d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.018+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/863782898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac23d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c000f80 tx=0x7fdd7c009510 comp rx=0 tx=0).stop 2023-09-10T00:43:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.019+0000 7fdd96f68b80 1 -- 172.21.15.44:0/863782898 shutdown_connections 2023-09-10T00:43:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.019+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/863782898 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563f8ac33ad0 0x563f8ac35ef0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.019+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/863782898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac23d80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.019+0000 7fdd63fff700 0 client.26090 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.019+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/863782898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac19b10 0x563f8ac26520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.019+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/863782898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac1a4d0 0x563f8ac23840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.019+0000 7fdd96f68b80 1 -- 172.21.15.44:0/863782898 >> 172.21.15.44:0/863782898 conn(0x563f8ab86030 msgr2=0x563f8ac0b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.019+0000 7fdd96f68b80 1 -- 172.21.15.44:0/863782898 shutdown_connections 2023-09-10T00:43:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.019+0000 7fdd96f68b80 1 -- 172.21.15.44:0/863782898 wait complete. 2023-09-10T00:43:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.019+0000 7fdd96f68b80 20 client.26090 trim_cache size 0 max 16384 2023-09-10T00:43:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.038+0000 7fdd96f68b80 1 Processor -- start 2023-09-10T00:43:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.038+0000 7fdd96f68b80 1 -- start start 2023-09-10T00:43:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.038+0000 7fdd96f68b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac19b10 0x563f8ac23000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.038+0000 7fdd96f68b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac1a4d0 0x563f8ac19210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.038+0000 7fdd96f68b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac0bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.038+0000 7fdd96f68b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8ac1f610 con 0x563f8ac1a4d0 2023-09-10T00:43:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.038+0000 7fdd96f68b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8ac24e20 con 0x563f8ac1f9b0 2023-09-10T00:43:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.038+0000 7fdd96f68b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8ac24950 con 0x563f8ac19b10 2023-09-10T00:43:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.038+0000 7fdd88b3e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac19b10 0x563f8ac23000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.038+0000 7fdd83fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac1a4d0 0x563f8ac19210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.038+0000 7fdd88b3e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac19b10 0x563f8ac23000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57158/0 (socket says 172.21.15.44:57158) 2023-09-10T00:43:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.038+0000 7fdd88b3e700 1 -- 172.21.15.44:0/1787986783 learned_addr learned my addr 172.21.15.44:0/1787986783 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.039+0000 7fdd8933f700 1 --2- 172.21.15.44:0/1787986783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac0bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.039+0000 7fdd88b3e700 1 -- 172.21.15.44:0/1787986783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 msgr2=0x563f8ac0bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.039+0000 7fdd88b3e700 1 --2- 172.21.15.44:0/1787986783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac0bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.039+0000 7fdd88b3e700 1 -- 172.21.15.44:0/1787986783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac1a4d0 msgr2=0x563f8ac19210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.039+0000 7fdd88b3e700 1 --2- 172.21.15.44:0/1787986783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac1a4d0 0x563f8ac19210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.039+0000 7fdd88b3e700 1 -- 172.21.15.44:0/1787986783 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f8a9b61f0 con 0x563f8ac19b10 2023-09-10T00:43:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.039+0000 7fdd83fff700 1 --2- 172.21.15.44:0/1787986783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac1a4d0 0x563f8ac19210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.039+0000 7fdd8933f700 1 --2- 172.21.15.44:0/1787986783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac0bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.039+0000 7fdd88b3e700 1 --2- 172.21.15.44:0/1787986783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac19b10 0x563f8ac23000 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdd740119b0 tx=0x7fdd74013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c359e781f977ca73 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.039+0000 7fdd6b7fe700 1 -- 172.21.15.44:0/1787986783 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd74024040 con 0x563f8ac19b10 2023-09-10T00:43:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.039+0000 7fdd6b7fe700 1 -- 172.21.15.44:0/1787986783 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd74015070 con 0x563f8ac19b10 2023-09-10T00:43:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.039+0000 7fdd6b7fe700 1 -- 172.21.15.44:0/1787986783 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd74003680 con 0x563f8ac19b10 2023-09-10T00:43:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.040+0000 7fdd96f68b80 1 -- 172.21.15.44:0/1787986783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac19b10 msgr2=0x563f8ac23000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.040+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/1787986783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac19b10 0x563f8ac23000 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdd740119b0 tx=0x7fdd74013000 comp rx=0 tx=0).stop 2023-09-10T00:43:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.040+0000 7fdd96f68b80 1 -- 172.21.15.44:0/1787986783 shutdown_connections 2023-09-10T00:43:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.040+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/1787986783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac0bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.040+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/1787986783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac19b10 0x563f8ac23000 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.040+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/1787986783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac1a4d0 0x563f8ac19210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.040+0000 7fdd96f68b80 1 -- 172.21.15.44:0/1787986783 >> 172.21.15.44:0/1787986783 conn(0x563f8ab86030 msgr2=0x563f8ac18b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.040+0000 7fdd96f68b80 1 -- 172.21.15.44:0/1787986783 shutdown_connections 2023-09-10T00:43:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.040+0000 7fdd96f68b80 1 -- 172.21.15.44:0/1787986783 wait complete. 2023-09-10T00:43:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.041+0000 7fdd96f68b80 1 -- start start 2023-09-10T00:43:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.041+0000 7fdd96f68b80 1 Processor -- start 2023-09-10T00:43:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.041+0000 7fdd96f68b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac19b10 0x563f8ac28f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.041+0000 7fdd96f68b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac1a4d0 0x563f8ac29450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.041+0000 7fdd96f68b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac80990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.041+0000 7fdd96f68b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8ac80ed0 con 0x563f8ac1a4d0 2023-09-10T00:43:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.041+0000 7fdd96f68b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8ac81010 con 0x563f8ac1f9b0 2023-09-10T00:43:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.041+0000 7fdd96f68b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8ac811d0 con 0x563f8ac19b10 2023-09-10T00:43:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.041+0000 7fdd96f68b80 10 client.0 _ll_register_callbacks cb 0x7fdd78003e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:43:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.041+0000 7fdd83fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac1a4d0 0x563f8ac29450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.041+0000 7fdd88b3e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac19b10 0x563f8ac28f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.041+0000 7fdd88b3e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac19b10 0x563f8ac28f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57164/0 (socket says 172.21.15.44:57164) 2023-09-10T00:43:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.041+0000 7fdd88b3e700 1 -- 172.21.15.44:0/1850432638 learned_addr learned my addr 172.21.15.44:0/1850432638 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.042+0000 7fdd8933f700 1 --2- 172.21.15.44:0/1850432638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac80990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.042+0000 7fdd88b3e700 1 -- 172.21.15.44:0/1850432638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 msgr2=0x563f8ac80990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.042+0000 7fdd88b3e700 1 --2- 172.21.15.44:0/1850432638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac80990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.042+0000 7fdd88b3e700 1 -- 172.21.15.44:0/1850432638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac1a4d0 msgr2=0x563f8ac29450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.042+0000 7fdd88b3e700 1 --2- 172.21.15.44:0/1850432638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac1a4d0 0x563f8ac29450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.042+0000 7fdd88b3e700 1 -- 172.21.15.44:0/1850432638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd841a5840 con 0x563f8ac19b10 2023-09-10T00:43:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.042+0000 7fdd88b3e700 1 --2- 172.21.15.44:0/1850432638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac19b10 0x563f8ac28f10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdd74011980 tx=0x7fdd74029900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4955f838b0dffba7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.042+0000 7fdd63fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:43:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.042+0000 7fdd63fff700 1 -- 172.21.15.44:0/1850432638 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd74024040 con 0x563f8ac19b10 2023-09-10T00:43:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.042+0000 7fdd63fff700 1 -- 172.21.15.44:0/1850432638 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fdd74017070 con 0x563f8ac19b10 2023-09-10T00:43:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.042+0000 7fdd96f68b80 1 -- 172.21.15.44:0/1850432638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd7c00f040 con 0x563f8ac19b10 2023-09-10T00:43:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.042+0000 7fdd63fff700 1 -- 172.21.15.44:0/1850432638 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd74029cf0 con 0x563f8ac19b10 2023-09-10T00:43:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.042+0000 7fdd96f68b80 10 client.26032 fetch_fsmap learned FSMap version 34 2023-09-10T00:43:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.042+0000 7fdd96f68b80 1 -- 172.21.15.44:0/1850432638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563f8a978c00 con 0x563f8ac19b10 2023-09-10T00:43:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.043+0000 7fdd63fff700 1 -- 172.21.15.44:0/1850432638 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd7c00f040 con 0x563f8ac19b10 2023-09-10T00:43:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.043+0000 7fdd63fff700 1 -- 172.21.15.44:0/1850432638 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fdd74031400 con 0x563f8ac19b10 2023-09-10T00:43:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.043+0000 7fdd96f68b80 10 client.26032 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:43:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.043+0000 7fdd96f68b80 10 client.26032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:43:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.043+0000 7fdd96f68b80 20 client.26032 populate_metadata read hostname 'smithi044' 2023-09-10T00:43:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.043+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/1850432638 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563f8ac33ad0 0x563f8ac8aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.043+0000 7fdd96f68b80 4 client.26032 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:43:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.043+0000 7fdd96f68b80 1 -- 172.21.15.44:0/1850432638 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563f8ab5c8c0 con 0x563f8ac33ad0 2023-09-10T00:43:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.043+0000 7fdd83fff700 1 --2- 172.21.15.44:0/1850432638 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563f8ac33ad0 0x563f8ac8aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.043+0000 7fdd63fff700 10 client.26032 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.043+0000 7fdd83fff700 1 --2- 172.21.15.44:0/1850432638 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563f8ac33ad0 0x563f8ac8aa20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a503727276a8b28 server_cookie=3691ff1f7bc198e1 in_seq=0 out_seq=0 2023-09-10T00:43:07.058 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:06 smithi131 ceph-mon[103713]: pgmap v76: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 50 MiB/s wr, 74 op/s 2023-09-10T00:43:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.058+0000 7fdd63fff700 1 -- 172.21.15.44:0/1850432638 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563f8ab5c8c0 con 0x563f8ac33ad0 2023-09-10T00:43:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.058+0000 7fdd63fff700 10 client.26032 handle_command_reply: tid=0 2023-09-10T00:43:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.058+0000 7fdd96f68b80 1 client.26032 shutdown 2023-09-10T00:43:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.059+0000 7fdd96f68b80 1 -- 172.21.15.44:0/1850432638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac19b10 msgr2=0x563f8ac28f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.059+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/1850432638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac19b10 0x563f8ac28f10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdd74011980 tx=0x7fdd74029900 comp rx=0 tx=0).stop 2023-09-10T00:43:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.059+0000 7fdd96f68b80 1 -- 172.21.15.44:0/1850432638 shutdown_connections 2023-09-10T00:43:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.059+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/1850432638 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563f8ac33ad0 0x563f8ac8aa20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.059+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/1850432638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f8ac1f9b0 0x563f8ac80990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.059+0000 7fdd63fff700 0 client.26032 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.059+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/1850432638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f8ac19b10 0x563f8ac28f10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.059+0000 7fdd96f68b80 1 --2- 172.21.15.44:0/1850432638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f8ac1a4d0 0x563f8ac29450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.059+0000 7fdd96f68b80 1 -- 172.21.15.44:0/1850432638 >> 172.21.15.44:0/1850432638 conn(0x563f8ab86030 msgr2=0x563f8ac19390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.059+0000 7fdd96f68b80 1 -- 172.21.15.44:0/1850432638 shutdown_connections 2023-09-10T00:43:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.059+0000 7fdd96f68b80 1 -- 172.21.15.44:0/1850432638 wait complete. 2023-09-10T00:43:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.059+0000 7fdd96f68b80 20 client.26032 trim_cache size 0 max 16384 2023-09-10T00:43:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:43:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:43:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:43:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:43:07.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.059+0000 7f0b88543700 1 -- 172.21.15.44:0/3290437745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b80106630 msgr2=0x7f0b80106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.059+0000 7f0b88543700 1 --2- 172.21.15.44:0/3290437745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b80106630 0x7f0b80106a10 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0099c0 tx=0x7f0b7c009cd0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.059+0000 7f0b88543700 1 -- 172.21.15.44:0/3290437745 shutdown_connections 2023-09-10T00:43:07.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.059+0000 7f0b88543700 1 --2- 172.21.15.44:0/3290437745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80100f30 0x7f0b8010d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.059+0000 7f0b88543700 1 --2- 172.21.15.44:0/3290437745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b80106630 0x7f0b80106a10 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.059+0000 7f0b88543700 1 --2- 172.21.15.44:0/3290437745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b80100570 0x7f0b801009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.059+0000 7f0b88543700 1 -- 172.21.15.44:0/3290437745 >> 172.21.15.44:0/3290437745 conn(0x7f0b800fc080 msgr2=0x7f0b800fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:07.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.059+0000 7f0b88543700 1 -- 172.21.15.44:0/3290437745 shutdown_connections 2023-09-10T00:43:07.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.059+0000 7f0b88543700 1 -- 172.21.15.44:0/3290437745 wait complete. 2023-09-10T00:43:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.060+0000 7fdd8ada2700 1 -- 172.21.15.44:0/2453292276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdd6c0a07b0 msgr2=0x7fdd6c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.060+0000 7fdd8ada2700 1 --2- 172.21.15.44:0/2453292276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdd6c0a07b0 0x7fdd6c0a2c70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdd74005950 tx=0x7fdd7400b410 comp rx=0 tx=0).stop 2023-09-10T00:43:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.060+0000 7fdd8ada2700 1 -- 172.21.15.44:0/2453292276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd840ffe60 msgr2=0x7fdd8419ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.060+0000 7fdd8ada2700 1 --2- 172.21.15.44:0/2453292276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd840ffe60 0x7fdd8419ed90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdd78004800 tx=0x7fdd78003920 comp rx=0 tx=0).stop 2023-09-10T00:43:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.060+0000 7fdd8ada2700 1 -- 172.21.15.44:0/2453292276 shutdown_connections 2023-09-10T00:43:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.060+0000 7fdd8ada2700 1 --2- 172.21.15.44:0/2453292276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdd6c0a07b0 0x7fdd6c0a2c70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.060+0000 7fdd8ada2700 1 --2- 172.21.15.44:0/2453292276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd84105560 0x7fdd841986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.060+0000 7fdd8ada2700 1 --2- 172.21.15.44:0/2453292276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd840ff4a0 0x7fdd8419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.060+0000 7fdd8ada2700 1 --2- 172.21.15.44:0/2453292276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd840ffe60 0x7fdd8419ed90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.060+0000 7fdd8ada2700 1 -- 172.21.15.44:0/2453292276 >> 172.21.15.44:0/2453292276 conn(0x7fdd840faf30 msgr2=0x7fdd84101040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.060+0000 7fdd8ada2700 1 -- 172.21.15.44:0/2453292276 shutdown_connections 2023-09-10T00:43:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:07.060+0000 7fdd8ada2700 1 -- 172.21.15.44:0/2453292276 wait complete. 2023-09-10T00:43:07.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.060+0000 7f0b88543700 1 Processor -- start 2023-09-10T00:43:07.074 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.060+0000 7f0b88543700 1 -- start start 2023-09-10T00:43:07.074 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.060+0000 7f0b88543700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80100570 0x7f0b80072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.074 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.060+0000 7f0b88543700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b80100f30 0x7f0b80072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.074 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.060+0000 7f0b88543700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b80106630 0x7f0b801a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.074 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.060+0000 7f0b88543700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b80073670 con 0x7f0b80100f30 2023-09-10T00:43:07.075 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.060+0000 7f0b88543700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b801a4c20 con 0x7f0b80106630 2023-09-10T00:43:07.075 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.060+0000 7f0b88543700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b801a4d60 con 0x7f0b80100570 2023-09-10T00:43:07.075 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.061+0000 7f0b85ade700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b80100f30 0x7f0b80072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.075 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.061+0000 7f0b85ade700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b80100f30 0x7f0b80072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50298/0 (socket says 172.21.15.44:50298) 2023-09-10T00:43:07.075 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.061+0000 7f0b862df700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80100570 0x7f0b80072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.061+0000 7f0b85ade700 1 -- 172.21.15.44:0/461372382 learned_addr learned my addr 172.21.15.44:0/461372382 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.061+0000 7f0b86ae0700 1 --2- 172.21.15.44:0/461372382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b80106630 0x7f0b801a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.061+0000 7f0b85ade700 1 -- 172.21.15.44:0/461372382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80100570 msgr2=0x7f0b80072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.061+0000 7f0b85ade700 1 --2- 172.21.15.44:0/461372382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80100570 0x7f0b80072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.061+0000 7f0b85ade700 1 -- 172.21.15.44:0/461372382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b80106630 msgr2=0x7f0b801a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.061+0000 7f0b85ade700 1 --2- 172.21.15.44:0/461372382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b80106630 0x7f0b801a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.061+0000 7f0b85ade700 1 -- 172.21.15.44:0/461372382 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c0095f0 con 0x7f0b80100f30 2023-09-10T00:43:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.061+0000 7f0b85ade700 1 --2- 172.21.15.44:0/461372382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b80100f30 0x7f0b80072e40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000d950 tx=0x7f0b7000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.061+0000 7f0b777fe700 1 -- 172.21.15.44:0/461372382 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000f890 con 0x7f0b80100f30 2023-09-10T00:43:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.061+0000 7f0b777fe700 1 -- 172.21.15.44:0/461372382 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b7000fe80 con 0x7f0b80100f30 2023-09-10T00:43:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.061+0000 7f0b88543700 1 -- 172.21.15.44:0/461372382 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b801a4fc0 con 0x7f0b80100f30 2023-09-10T00:43:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.061+0000 7f0b777fe700 1 -- 172.21.15.44:0/461372382 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000e3b0 con 0x7f0b80100f30 2023-09-10T00:43:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.062+0000 7f0b88543700 1 -- 172.21.15.44:0/461372382 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b801a5440 con 0x7f0b80100f30 2023-09-10T00:43:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.064+0000 7f0b777fe700 1 -- 172.21.15.44:0/461372382 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f0b70010460 con 0x7f0b80100f30 2023-09-10T00:43:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.065+0000 7f0b777fe700 1 --2- 172.21.15.44:0/461372382 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0b6c0a07b0 0x7f0b6c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.065+0000 7f0b777fe700 1 -- 172.21.15.44:0/461372382 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f0b700cbb30 con 0x7f0b80100f30 2023-09-10T00:43:07.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.065+0000 7f0b862df700 1 --2- 172.21.15.44:0/461372382 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0b6c0a07b0 0x7f0b6c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.066+0000 7f0b94709b80 1 Processor -- start 2023-09-10T00:43:07.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.066+0000 7f0b94709b80 1 -- start start 2023-09-10T00:43:07.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.066+0000 7f0b862df700 1 --2- 172.21.15.44:0/461372382 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0b6c0a07b0 0x7f0b6c0a2c70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00b580 tx=0x7f0b7c009f10 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:07.079 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.066+0000 7f0b94709b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948fe7710 0x55b948fe7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.066+0000 7f0b94709b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 0x55b948ffa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.066+0000 7f0b94709b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948ffa650 0x55b948ffca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.066+0000 7f0b94709b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b948f40730 con 0x55b948fe8030 2023-09-10T00:43:07.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.066+0000 7f0b94709b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b948f3e050 con 0x55b948fe7710 2023-09-10T00:43:07.080 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.066+0000 7f0b94709b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b948f39940 con 0x55b948ffa650 2023-09-10T00:43:07.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.066+0000 7f0b862df700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948fe7710 0x55b948fe7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.066+0000 7f0b85ade700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 0x55b948ffa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.066+0000 7f0b86ae0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948ffa650 0x55b948ffca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.066+0000 7f0b85ade700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 0x55b948ffa110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50306/0 (socket says 172.21.15.44:50306) 2023-09-10T00:43:07.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.066+0000 7f0b862df700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948fe7710 0x55b948fe7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52044/0 (socket says 172.21.15.44:52044) 2023-09-10T00:43:07.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.066+0000 7f0b85ade700 1 -- 172.21.15.44:0/1676373963 learned_addr learned my addr 172.21.15.44:0/1676373963 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.067+0000 7f0b86ae0700 1 -- 172.21.15.44:0/1676373963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948fe7710 msgr2=0x55b948fe7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.067+0000 7f0b86ae0700 1 --2- 172.21.15.44:0/1676373963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948fe7710 0x55b948fe7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.067+0000 7f0b86ae0700 1 -- 172.21.15.44:0/1676373963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 msgr2=0x55b948ffa110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.067+0000 7f0b86ae0700 1 --2- 172.21.15.44:0/1676373963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 0x55b948ffa110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.067+0000 7f0b86ae0700 1 -- 172.21.15.44:0/1676373963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b948d65de0 con 0x55b948ffa650 2023-09-10T00:43:07.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.067+0000 7f0b862df700 1 --2- 172.21.15.44:0/1676373963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948fe7710 0x55b948fe7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:07.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.067+0000 7f0b85ade700 1 --2- 172.21.15.44:0/1676373963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 0x55b948ffa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:07.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.067+0000 7f0b86ae0700 1 --2- 172.21.15.44:0/1676373963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948ffa650 0x55b948ffca40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0b78003ef0 tx=0x7f0b78009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f7955cf4ffcb07a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:07.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.067+0000 7f0b74ff9700 1 -- 172.21.15.44:0/1676373963 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b78019070 con 0x55b948ffa650 2023-09-10T00:43:07.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.067+0000 7f0b74ff9700 1 -- 172.21.15.44:0/1676373963 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b7800f040 con 0x55b948ffa650 2023-09-10T00:43:07.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.067+0000 7f0b74ff9700 1 -- 172.21.15.44:0/1676373963 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b78007900 con 0x55b948ffa650 2023-09-10T00:43:07.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.068+0000 7f0b94709b80 1 -- 172.21.15.44:0/1676373963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948ffa650 msgr2=0x55b948ffca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.068+0000 7f0b94709b80 1 --2- 172.21.15.44:0/1676373963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948ffa650 0x55b948ffca40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0b78003ef0 tx=0x7f0b78009ba0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.068+0000 7f0b94709b80 1 -- 172.21.15.44:0/1676373963 shutdown_connections 2023-09-10T00:43:07.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.068+0000 7f0b94709b80 1 --2- 172.21.15.44:0/1676373963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948ffa650 0x55b948ffca40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.068+0000 7f0b94709b80 1 --2- 172.21.15.44:0/1676373963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948fe7710 0x55b948fe7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.068+0000 7f0b94709b80 1 --2- 172.21.15.44:0/1676373963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 0x55b948ffa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.068+0000 7f0b94709b80 1 -- 172.21.15.44:0/1676373963 >> 172.21.15.44:0/1676373963 conn(0x55b948f5c090 msgr2=0x55b948fe2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:07.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.068+0000 7f0b94709b80 1 -- 172.21.15.44:0/1676373963 shutdown_connections 2023-09-10T00:43:07.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.068+0000 7f0b94709b80 1 -- 172.21.15.44:0/1676373963 wait complete. 2023-09-10T00:43:07.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.069+0000 7f0b94709b80 1 -- start start 2023-09-10T00:43:07.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.069+0000 7f0b94709b80 1 Processor -- start 2023-09-10T00:43:07.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.069+0000 7f0b94709b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948fe7710 0x55b948ff39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.069+0000 7f0b94709b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 0x55b948ff3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.069+0000 7f0b94709b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948ffa650 0x55b948ff9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.087 INFO:teuthology.orchestra.run.smithi044.stderr:11600000 of 50000000 tuples (23%) done (elapsed 126.70 s, remaining 419.44 s) 2023-09-10T00:43:07.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.069+0000 7f0b94709b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b948ed7b70 con 0x55b948fe8030 2023-09-10T00:43:07.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.069+0000 7f0b94709b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b948ed7870 con 0x55b948ffa650 2023-09-10T00:43:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.069+0000 7f0b94709b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b948dbb920 con 0x55b948fe7710 2023-09-10T00:43:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.069+0000 7f0b94709b80 10 client.0 _ll_register_callbacks cb 0x55b948e21a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:43:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.069+0000 7f0b862df700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948fe7710 0x55b948ff39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.069+0000 7f0b86ae0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948ffa650 0x55b948ff9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.069+0000 7f0b862df700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948fe7710 0x55b948ff39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57192/0 (socket says 172.21.15.44:57192) 2023-09-10T00:43:07.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.069+0000 7f0b862df700 1 -- 172.21.15.44:0/4028521459 learned_addr learned my addr 172.21.15.44:0/4028521459 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:07.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.069+0000 7f0b85ade700 1 --2- 172.21.15.44:0/4028521459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 0x55b948ff3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.070+0000 7f0b862df700 1 -- 172.21.15.44:0/4028521459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948ffa650 msgr2=0x55b948ff9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:07.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.070+0000 7f0b862df700 1 --2- 172.21.15.44:0/4028521459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948ffa650 0x55b948ff9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.070+0000 7f0b862df700 1 -- 172.21.15.44:0/4028521459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 msgr2=0x55b948ff3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:07.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.070+0000 7f0b862df700 1 --2- 172.21.15.44:0/4028521459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 0x55b948ff3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.070+0000 7f0b862df700 1 -- 172.21.15.44:0/4028521459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8010dcf0 con 0x55b948fe7710 2023-09-10T00:43:07.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.070+0000 7f0b862df700 1 --2- 172.21.15.44:0/4028521459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948fe7710 0x55b948ff39f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0099c0 tx=0x7f0b7c009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c73285966f13e45 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:07.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.070+0000 7f0b69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:43:07.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.070+0000 7f0b69ffb700 1 -- 172.21.15.44:0/4028521459 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c004b70 con 0x55b948fe7710 2023-09-10T00:43:07.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.070+0000 7f0b94709b80 1 -- 172.21.15.44:0/4028521459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b948e914f0 con 0x55b948fe7710 2023-09-10T00:43:07.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.070+0000 7f0b69ffb700 1 -- 172.21.15.44:0/4028521459 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f0b7c004cd0 con 0x55b948fe7710 2023-09-10T00:43:07.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.070+0000 7f0b69ffb700 1 -- 172.21.15.44:0/4028521459 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b7c004e90 con 0x55b948fe7710 2023-09-10T00:43:07.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.070+0000 7f0b94709b80 10 client.26044 fetch_fsmap learned FSMap version 34 2023-09-10T00:43:07.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.070+0000 7f0b94709b80 1 -- 172.21.15.44:0/4028521459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f0b801a5440 con 0x55b948fe7710 2023-09-10T00:43:07.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.070+0000 7f0b69ffb700 1 -- 172.21.15.44:0/4028521459 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b948e914f0 con 0x55b948fe7710 2023-09-10T00:43:07.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.071+0000 7f0b69ffb700 1 -- 172.21.15.44:0/4028521459 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0b7c02f070 con 0x55b948fe7710 2023-09-10T00:43:07.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.071+0000 7f0b94709b80 10 client.26044 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:43:07.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.071+0000 7f0b94709b80 10 client.26044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:43:07.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.071+0000 7f0b94709b80 20 client.26044 populate_metadata read hostname 'smithi044' 2023-09-10T00:43:07.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.071+0000 7f0b94709b80 1 --2- 172.21.15.44:0/4028521459 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b949012a10 0x55b949014e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.071+0000 7f0b94709b80 4 client.26044 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:43:07.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.071+0000 7f0b94709b80 1 -- 172.21.15.44:0/4028521459 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b948f4e8a0 con 0x55b949012a10 2023-09-10T00:43:07.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.071+0000 7f0b85ade700 1 --2- 172.21.15.44:0/4028521459 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b949012a10 0x55b949014e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.071+0000 7f0b69ffb700 10 client.26044 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:07.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.071+0000 7f0b85ade700 1 --2- 172.21.15.44:0/4028521459 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b949012a10 0x55b949014e30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92be51c24edc8bca server_cookie=12a2664ffd4cd7e4 in_seq=0 out_seq=0 2023-09-10T00:43:07.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.072+0000 7f0b69ffb700 1 -- 172.21.15.44:0/4028521459 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b948f4e8a0 con 0x55b949012a10 2023-09-10T00:43:07.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.072+0000 7f0b69ffb700 10 client.26044 handle_command_reply: tid=0 2023-09-10T00:43:07.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.072+0000 7f0b94709b80 1 client.26044 shutdown 2023-09-10T00:43:07.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.072+0000 7f0b94709b80 1 -- 172.21.15.44:0/4028521459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948fe7710 msgr2=0x55b948ff39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.072+0000 7f0b94709b80 1 --2- 172.21.15.44:0/4028521459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948fe7710 0x55b948ff39f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0099c0 tx=0x7f0b7c009cd0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.072+0000 7f0b94709b80 1 -- 172.21.15.44:0/4028521459 shutdown_connections 2023-09-10T00:43:07.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.072+0000 7f0b94709b80 1 --2- 172.21.15.44:0/4028521459 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b949012a10 0x55b949014e30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.072+0000 7f0b94709b80 1 --2- 172.21.15.44:0/4028521459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948ffa650 0x55b948ff9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.072+0000 7f0b69ffb700 0 client.26044 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:07.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.072+0000 7f0b94709b80 1 --2- 172.21.15.44:0/4028521459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948fe7710 0x55b948ff39f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.072+0000 7f0b94709b80 1 --2- 172.21.15.44:0/4028521459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 0x55b948ff3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.072+0000 7f0b94709b80 1 -- 172.21.15.44:0/4028521459 >> 172.21.15.44:0/4028521459 conn(0x55b948f5c090 msgr2=0x55b948fe2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:07.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.073+0000 7f0b94709b80 1 -- 172.21.15.44:0/4028521459 shutdown_connections 2023-09-10T00:43:07.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.073+0000 7f0b94709b80 1 -- 172.21.15.44:0/4028521459 wait complete. 2023-09-10T00:43:07.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.073+0000 7f0b94709b80 20 client.26044 trim_cache size 0 max 16384 2023-09-10T00:43:07.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.094+0000 7f0b94709b80 1 Processor -- start 2023-09-10T00:43:07.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.094+0000 7f0b94709b80 1 -- start start 2023-09-10T00:43:07.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.094+0000 7f0b94709b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe7710 0x55b948fee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.094+0000 7f0b94709b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948fe8030 0x55b948feeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.094+0000 7f0b94709b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948ffa650 0x55b948ff35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.094+0000 7f0b94709b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b948fe2fc0 con 0x55b948fe7710 2023-09-10T00:43:07.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.094+0000 7f0b94709b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b948fe89f0 con 0x55b948fe8030 2023-09-10T00:43:07.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.094+0000 7f0b94709b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b948fe87e0 con 0x55b948ffa650 2023-09-10T00:43:07.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.094+0000 7f0b862df700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe7710 0x55b948fee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.094+0000 7f0b85ade700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948fe8030 0x55b948feeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.104 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.094+0000 7f0b862df700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe7710 0x55b948fee580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50318/0 (socket says 172.21.15.44:50318) 2023-09-10T00:43:07.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.094+0000 7f0b862df700 1 -- 172.21.15.44:0/3716186903 learned_addr learned my addr 172.21.15.44:0/3716186903 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:07.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.094+0000 7f0b86ae0700 1 --2- 172.21.15.44:0/3716186903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948ffa650 0x55b948ff35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.095+0000 7f0b85ade700 1 -- 172.21.15.44:0/3716186903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948ffa650 msgr2=0x55b948ff35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:07.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.095+0000 7f0b85ade700 1 --2- 172.21.15.44:0/3716186903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948ffa650 0x55b948ff35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.095+0000 7f0b85ade700 1 -- 172.21.15.44:0/3716186903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe7710 msgr2=0x55b948fee580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:07.105 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.095+0000 7f0b85ade700 1 --2- 172.21.15.44:0/3716186903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe7710 0x55b948fee580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.095+0000 7f0b85ade700 1 -- 172.21.15.44:0/3716186903 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b78003b50 con 0x55b948fe8030 2023-09-10T00:43:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.095+0000 7f0b86ae0700 1 --2- 172.21.15.44:0/3716186903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948ffa650 0x55b948ff35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.095+0000 7f0b862df700 1 --2- 172.21.15.44:0/3716186903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe7710 0x55b948fee580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.095+0000 7f0b85ade700 1 --2- 172.21.15.44:0/3716186903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948fe8030 0x55b948feeac0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0b70098bd0 tx=0x7f0b700b8040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a917fc4ba70c4af6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.095+0000 7f0b757fa700 1 -- 172.21.15.44:0/3716186903 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700b0070 con 0x55b948fe8030 2023-09-10T00:43:07.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.095+0000 7f0b757fa700 1 -- 172.21.15.44:0/3716186903 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b700b6e90 con 0x55b948fe8030 2023-09-10T00:43:07.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.095+0000 7f0b757fa700 1 -- 172.21.15.44:0/3716186903 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70099100 con 0x55b948fe8030 2023-09-10T00:43:07.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.096+0000 7f0b94709b80 1 -- 172.21.15.44:0/3716186903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948fe8030 msgr2=0x55b948feeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.096+0000 7f0b94709b80 1 --2- 172.21.15.44:0/3716186903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948fe8030 0x55b948feeac0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0b70098bd0 tx=0x7f0b700b8040 comp rx=0 tx=0).stop 2023-09-10T00:43:07.107 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.096+0000 7f0b94709b80 1 -- 172.21.15.44:0/3716186903 shutdown_connections 2023-09-10T00:43:07.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.096+0000 7f0b94709b80 1 --2- 172.21.15.44:0/3716186903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948ffa650 0x55b948ff35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.096+0000 7f0b94709b80 1 --2- 172.21.15.44:0/3716186903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe7710 0x55b948fee580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.096+0000 7f0b94709b80 1 --2- 172.21.15.44:0/3716186903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948fe8030 0x55b948feeac0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.096+0000 7f0b94709b80 1 -- 172.21.15.44:0/3716186903 >> 172.21.15.44:0/3716186903 conn(0x55b948f5c090 msgr2=0x55b948fea030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:07.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.096+0000 7f0b94709b80 1 -- 172.21.15.44:0/3716186903 shutdown_connections 2023-09-10T00:43:07.108 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.096+0000 7f0b94709b80 1 -- 172.21.15.44:0/3716186903 wait complete. 2023-09-10T00:43:07.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.096+0000 7f0b94709b80 1 -- start start 2023-09-10T00:43:07.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.096+0000 7f0b94709b80 1 Processor -- start 2023-09-10T00:43:07.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b94709b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948fe7710 0x55b948ff7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b94709b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 0x55b948ff7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.109 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b94709b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948ffa650 0x55b948ff80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b94709b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b949066830 con 0x55b948fe8030 2023-09-10T00:43:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b94709b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b949066970 con 0x55b948ffa650 2023-09-10T00:43:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b94709b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b949066ae0 con 0x55b948fe7710 2023-09-10T00:43:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b94709b80 10 client.0 _ll_register_callbacks cb 0x55b948e21a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:43:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b85ade700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 0x55b948ff7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b85ade700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 0x55b948ff7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50326/0 (socket says 172.21.15.44:50326) 2023-09-10T00:43:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b862df700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948fe7710 0x55b948ff7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b85ade700 1 -- 172.21.15.44:0/238400383 learned_addr learned my addr 172.21.15.44:0/238400383 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b86ae0700 1 --2- 172.21.15.44:0/238400383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948ffa650 0x55b948ff80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b85ade700 1 -- 172.21.15.44:0/238400383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948fe7710 msgr2=0x55b948ff7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b85ade700 1 --2- 172.21.15.44:0/238400383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948fe7710 0x55b948ff7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b85ade700 1 -- 172.21.15.44:0/238400383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948ffa650 msgr2=0x55b948ff80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b85ade700 1 --2- 172.21.15.44:0/238400383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948ffa650 0x55b948ff80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b85ade700 1 -- 172.21.15.44:0/238400383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b801a5440 con 0x55b948fe8030 2023-09-10T00:43:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b85ade700 1 --2- 172.21.15.44:0/238400383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 0x55b948ff7b80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0b70098940 tx=0x7f0b700a0d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c1e71f7756d4f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.097+0000 7f0b69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:43:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.098+0000 7f0b69ffb700 1 -- 172.21.15.44:0/238400383 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700b0070 con 0x55b948fe8030 2023-09-10T00:43:07.113 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.098+0000 7f0b69ffb700 1 -- 172.21.15.44:0/238400383 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f0b7009ca70 con 0x55b948fe8030 2023-09-10T00:43:07.113 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.098+0000 7f0b94709b80 1 -- 172.21.15.44:0/238400383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b7800f040 con 0x55b948fe8030 2023-09-10T00:43:07.113 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.098+0000 7f0b69ffb700 1 -- 172.21.15.44:0/238400383 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b70098c40 con 0x55b948fe8030 2023-09-10T00:43:07.113 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.098+0000 7f0b94709b80 10 client.16443 fetch_fsmap learned FSMap version 34 2023-09-10T00:43:07.113 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.098+0000 7f0b94709b80 1 -- 172.21.15.44:0/238400383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b948d65de0 con 0x55b948fe8030 2023-09-10T00:43:07.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.098+0000 7f0b69ffb700 1 -- 172.21.15.44:0/238400383 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b7800f040 con 0x55b948fe8030 2023-09-10T00:43:07.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.098+0000 7f0b69ffb700 1 -- 172.21.15.44:0/238400383 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0b700a0840 con 0x55b948fe8030 2023-09-10T00:43:07.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.098+0000 7f0b94709b80 10 client.16443 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:43:07.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.098+0000 7f0b94709b80 10 client.16443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:43:07.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.098+0000 7f0b94709b80 20 client.16443 populate_metadata read hostname 'smithi044' 2023-09-10T00:43:07.114 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.098+0000 7f0b94709b80 1 --2- 172.21.15.44:0/238400383 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b949068070 0x55b94906a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:07.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.098+0000 7f0b94709b80 4 client.16443 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:43:07.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.098+0000 7f0b94709b80 1 -- 172.21.15.44:0/238400383 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b948f32920 con 0x55b949068070 2023-09-10T00:43:07.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.098+0000 7f0b862df700 1 --2- 172.21.15.44:0/238400383 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b949068070 0x55b94906a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:07.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b69ffb700 10 client.16443 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:07.115 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b862df700 1 --2- 172.21.15.44:0/238400383 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b949068070 0x55b94906a490 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=826a43c0136526d1 server_cookie=ca06da4a67b298c1 in_seq=0 out_seq=0 2023-09-10T00:43:07.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b69ffb700 1 -- 172.21.15.44:0/238400383 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b948f32920 con 0x55b949068070 2023-09-10T00:43:07.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b69ffb700 10 client.16443 handle_command_reply: tid=0 2023-09-10T00:43:07.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b94709b80 1 client.16443 shutdown 2023-09-10T00:43:07.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b94709b80 1 -- 172.21.15.44:0/238400383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 msgr2=0x55b948ff7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b94709b80 1 --2- 172.21.15.44:0/238400383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 0x55b948ff7b80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0b70098940 tx=0x7f0b700a0d20 comp rx=0 tx=0).stop 2023-09-10T00:43:07.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b94709b80 1 -- 172.21.15.44:0/238400383 shutdown_connections 2023-09-10T00:43:07.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b94709b80 1 --2- 172.21.15.44:0/238400383 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b949068070 0x55b94906a490 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b94709b80 1 --2- 172.21.15.44:0/238400383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b948ffa650 0x55b948ff80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b94709b80 1 --2- 172.21.15.44:0/238400383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b948fe7710 0x55b948ff7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b69ffb700 0 client.16443 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:07.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b94709b80 1 --2- 172.21.15.44:0/238400383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b948fe8030 0x55b948ff7b80 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.118 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b94709b80 1 -- 172.21.15.44:0/238400383 >> 172.21.15.44:0/238400383 conn(0x55b948f5c090 msgr2=0x55b948fe6700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:07.118 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b94709b80 1 -- 172.21.15.44:0/238400383 shutdown_connections 2023-09-10T00:43:07.118 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b94709b80 1 -- 172.21.15.44:0/238400383 wait complete. 2023-09-10T00:43:07.118 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.099+0000 7f0b94709b80 20 client.16443 trim_cache size 0 max 16384 2023-09-10T00:43:07.119 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:43:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:43:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:43:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:43:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:43:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:43:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:43:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:43:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:43:07.120 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:43:07.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.100+0000 7f0b88543700 1 -- 172.21.15.44:0/461372382 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0b6c0a07b0 msgr2=0x7f0b6c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.100+0000 7f0b88543700 1 --2- 172.21.15.44:0/461372382 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0b6c0a07b0 0x7f0b6c0a2c70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00b580 tx=0x7f0b7c009f10 comp rx=0 tx=0).stop 2023-09-10T00:43:07.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.100+0000 7f0b88543700 1 -- 172.21.15.44:0/461372382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b80100f30 msgr2=0x7f0b80072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:07.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.100+0000 7f0b88543700 1 --2- 172.21.15.44:0/461372382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b80100f30 0x7f0b80072e40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000d950 tx=0x7f0b7000dc60 comp rx=0 tx=0).stop 2023-09-10T00:43:07.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.100+0000 7f0b88543700 1 -- 172.21.15.44:0/461372382 shutdown_connections 2023-09-10T00:43:07.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.100+0000 7f0b88543700 1 --2- 172.21.15.44:0/461372382 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0b6c0a07b0 0x7f0b6c0a2c70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.100+0000 7f0b88543700 1 --2- 172.21.15.44:0/461372382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b80106630 0x7f0b801a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.100+0000 7f0b88543700 1 --2- 172.21.15.44:0/461372382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80100570 0x7f0b80072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.100+0000 7f0b88543700 1 --2- 172.21.15.44:0/461372382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b80100f30 0x7f0b80072e40 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:07.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.100+0000 7f0b88543700 1 -- 172.21.15.44:0/461372382 >> 172.21.15.44:0/461372382 conn(0x7f0b800fc080 msgr2=0x7f0b800fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:07.124 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.100+0000 7f0b88543700 1 -- 172.21.15.44:0/461372382 shutdown_connections 2023-09-10T00:43:07.124 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:07.100+0000 7f0b88543700 1 -- 172.21.15.44:0/461372382 wait complete. 2023-09-10T00:43:07.124 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:43:07.915 INFO:teuthology.orchestra.run.smithi044.stderr:11700000 of 50000000 tuples (23%) done (elapsed 127.55 s, remaining 417.53 s) 2023-09-10T00:43:08.801 INFO:teuthology.orchestra.run.smithi044.stderr:11800000 of 50000000 tuples (23%) done (elapsed 128.44 s, remaining 415.78 s) 2023-09-10T00:43:08.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:08 smithi099 ceph-mon[102692]: pgmap v77: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 42 op/s 2023-09-10T00:43:09.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:08 smithi131 ceph-mon[103713]: pgmap v77: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 42 op/s 2023-09-10T00:43:09.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:08 smithi044 ceph-mon[98676]: pgmap v77: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 42 op/s 2023-09-10T00:43:09.657 INFO:teuthology.orchestra.run.smithi044.stderr:11900000 of 50000000 tuples (23%) done (elapsed 129.29 s, remaining 413.95 s) 2023-09-10T00:43:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:12000000 of 50000000 tuples (24%) done (elapsed 130.11 s, remaining 412.03 s) 2023-09-10T00:43:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:10 smithi131 ceph-mon[103713]: pgmap v78: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 40 op/s 2023-09-10T00:43:11.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:10 smithi044 ceph-mon[98676]: pgmap v78: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 40 op/s 2023-09-10T00:43:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:10 smithi099 ceph-mon[102692]: pgmap v78: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 40 op/s 2023-09-10T00:43:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:12100000 of 50000000 tuples (24%) done (elapsed 131.04 s, remaining 410.45 s) 2023-09-10T00:43:12.253 INFO:teuthology.orchestra.run.smithi044.stderr:12200000 of 50000000 tuples (24%) done (elapsed 131.89 s, remaining 408.64 s) 2023-09-10T00:43:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:12 smithi131 ceph-mon[103713]: pgmap v79: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 52 MiB/s wr, 54 op/s 2023-09-10T00:43:13.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:12 smithi044 ceph-mon[98676]: pgmap v79: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 52 MiB/s wr, 54 op/s 2023-09-10T00:43:13.091 INFO:teuthology.orchestra.run.smithi044.stderr:12300000 of 50000000 tuples (24%) done (elapsed 132.73 s, remaining 406.81 s) 2023-09-10T00:43:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:12 smithi099 ceph-mon[102692]: pgmap v79: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 52 MiB/s wr, 54 op/s 2023-09-10T00:43:13.934 INFO:teuthology.orchestra.run.smithi044.stderr:12400000 of 50000000 tuples (24%) done (elapsed 133.57 s, remaining 405.01 s) 2023-09-10T00:43:14.837 INFO:teuthology.orchestra.run.smithi044.stderr:12500000 of 50000000 tuples (25%) done (elapsed 134.47 s, remaining 403.41 s) 2023-09-10T00:43:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:14 smithi131 ceph-mon[103713]: pgmap v80: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 MiB/s wr, 42 op/s 2023-09-10T00:43:15.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:14 smithi044 ceph-mon[98676]: pgmap v80: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 MiB/s wr, 42 op/s 2023-09-10T00:43:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:14 smithi099 ceph-mon[102692]: pgmap v80: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 MiB/s wr, 42 op/s 2023-09-10T00:43:15.670 INFO:teuthology.orchestra.run.smithi044.stderr:12600000 of 50000000 tuples (25%) done (elapsed 135.30 s, remaining 401.62 s) 2023-09-10T00:43:16.536 INFO:teuthology.orchestra.run.smithi044.stderr:12700000 of 50000000 tuples (25%) done (elapsed 136.17 s, remaining 399.93 s) 2023-09-10T00:43:17.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:16 smithi131 ceph-mon[103713]: pgmap v81: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 MiB/s wr, 53 op/s 2023-09-10T00:43:17.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:16 smithi044 ceph-mon[98676]: pgmap v81: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 MiB/s wr, 53 op/s 2023-09-10T00:43:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:16 smithi099 ceph-mon[102692]: pgmap v81: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 MiB/s wr, 53 op/s 2023-09-10T00:43:17.361 INFO:teuthology.orchestra.run.smithi044.stderr:12800000 of 50000000 tuples (25%) done (elapsed 137.00 s, remaining 398.14 s) 2023-09-10T00:43:18.246 INFO:teuthology.orchestra.run.smithi044.stderr:12900000 of 50000000 tuples (25%) done (elapsed 137.88 s, remaining 396.54 s) 2023-09-10T00:43:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:18 smithi131 ceph-mon[103713]: pgmap v82: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 34 MiB/s wr, 25 op/s 2023-09-10T00:43:19.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:18 smithi044 ceph-mon[98676]: pgmap v82: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 34 MiB/s wr, 25 op/s 2023-09-10T00:43:19.098 INFO:teuthology.orchestra.run.smithi044.stderr:13000000 of 50000000 tuples (26%) done (elapsed 138.73 s, remaining 394.85 s) 2023-09-10T00:43:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:18 smithi099 ceph-mon[102692]: pgmap v82: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 34 MiB/s wr, 25 op/s 2023-09-10T00:43:19.930 INFO:teuthology.orchestra.run.smithi044.stderr:13100000 of 50000000 tuples (26%) done (elapsed 139.56 s, remaining 393.12 s) 2023-09-10T00:43:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.181+0000 7f5e7bed8700 1 -- 172.21.15.44:0/750149084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e741005d0 msgr2=0x7f5e74100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.181+0000 7f5e7bed8700 1 --2- 172.21.15.44:0/750149084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e741005d0 0x7f5e74100a50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5e640099c0 tx=0x7f5e64009cd0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.181+0000 7f5e7bed8700 1 -- 172.21.15.44:0/750149084 shutdown_connections 2023-09-10T00:43:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.181+0000 7f5e7bed8700 1 --2- 172.21.15.44:0/750149084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e74100f90 0x7f5e7410d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.181+0000 7f5e7bed8700 1 --2- 172.21.15.44:0/750149084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e74106690 0x7f5e74106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.181+0000 7f5e7bed8700 1 --2- 172.21.15.44:0/750149084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e741005d0 0x7f5e74100a50 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.181+0000 7f5e7bed8700 1 -- 172.21.15.44:0/750149084 >> 172.21.15.44:0/750149084 conn(0x7f5e740fc0c0 msgr2=0x7f5e740fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.182+0000 7f5e7bed8700 1 -- 172.21.15.44:0/750149084 shutdown_connections 2023-09-10T00:43:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.182+0000 7f5e7bed8700 1 -- 172.21.15.44:0/750149084 wait complete. 2023-09-10T00:43:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.182+0000 7f5e7bed8700 1 Processor -- start 2023-09-10T00:43:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.182+0000 7f5e7bed8700 1 -- start start 2023-09-10T00:43:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e7bed8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e741005d0 0x7f5e7419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e7bed8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e74100f90 0x7f5e7419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e7bed8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e74106690 0x7f5e741963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e7bed8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e74196a10 con 0x7f5e74106690 2023-09-10T00:43:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e7bed8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e74196b80 con 0x7f5e74100f90 2023-09-10T00:43:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e7bed8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e74196d40 con 0x7f5e741005d0 2023-09-10T00:43:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e7a475700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e74106690 0x7f5e741963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e79473700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e74100f90 0x7f5e7419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e7a475700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e74106690 0x7f5e741963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50342/0 (socket says 172.21.15.44:50342) 2023-09-10T00:43:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e7a475700 1 -- 172.21.15.44:0/1000970232 learned_addr learned my addr 172.21.15.44:0/1000970232 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e79c74700 1 --2- 172.21.15.44:0/1000970232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e741005d0 0x7f5e7419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e7a475700 1 -- 172.21.15.44:0/1000970232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e741005d0 msgr2=0x7f5e7419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e7a475700 1 --2- 172.21.15.44:0/1000970232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e741005d0 0x7f5e7419c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e7a475700 1 -- 172.21.15.44:0/1000970232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e74100f90 msgr2=0x7f5e7419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e7a475700 1 --2- 172.21.15.44:0/1000970232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e74100f90 0x7f5e7419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e7a475700 1 -- 172.21.15.44:0/1000970232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e640095f0 con 0x7f5e74106690 2023-09-10T00:43:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e79c74700 1 --2- 172.21.15.44:0/1000970232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e741005d0 0x7f5e7419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e7a475700 1 --2- 172.21.15.44:0/1000970232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e74106690 0x7f5e741963a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00ef30 tx=0x7f5e6c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e6affd700 1 -- 172.21.15.44:0/1000970232 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c018070 con 0x7f5e74106690 2023-09-10T00:43:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e6affd700 1 -- 172.21.15.44:0/1000970232 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e6c00f040 con 0x7f5e74106690 2023-09-10T00:43:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e7bed8700 1 -- 172.21.15.44:0/1000970232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e74196fc0 con 0x7f5e74106690 2023-09-10T00:43:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.183+0000 7f5e6affd700 1 -- 172.21.15.44:0/1000970232 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c00cbc0 con 0x7f5e74106690 2023-09-10T00:43:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.184+0000 7f5e7bed8700 1 -- 172.21.15.44:0/1000970232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e741a3530 con 0x7f5e74106690 2023-09-10T00:43:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.187+0000 7f5e6affd700 1 -- 172.21.15.44:0/1000970232 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f5e6c00cd80 con 0x7f5e74106690 2023-09-10T00:43:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.187+0000 7f5e6affd700 1 --2- 172.21.15.44:0/1000970232 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5e600a07b0 0x7f5e600a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.187+0000 7f5e6affd700 1 -- 172.21.15.44:0/1000970232 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f5e6c0cbe30 con 0x7f5e74106690 2023-09-10T00:43:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.187+0000 7f5e79c74700 1 --2- 172.21.15.44:0/1000970232 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5e600a07b0 0x7f5e600a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.187+0000 7f5e8809eb80 1 Processor -- start 2023-09-10T00:43:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.188+0000 7f5e79c74700 1 --2- 172.21.15.44:0/1000970232 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5e600a07b0 0x7f5e600a2c70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e70011490 tx=0x7f5e70009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.188+0000 7f5e8809eb80 1 -- start start 2023-09-10T00:43:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.188+0000 7f5e8809eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c25e5b0 0x55f65c25e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.188+0000 7f5e8809eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c258710 0x55f65c258b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.188+0000 7f5e8809eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c2590d0 0x55f65c265340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.188+0000 7f5e8809eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65c1ad6d0 con 0x55f65c25e5b0 2023-09-10T00:43:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.188+0000 7f5e8809eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65c1aaff0 con 0x55f65c2590d0 2023-09-10T00:43:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.188+0000 7f5e8809eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65c1a68e0 con 0x55f65c258710 2023-09-10T00:43:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.188+0000 7f5e79c74700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c25e5b0 0x55f65c25e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.188+0000 7f5e79c74700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c25e5b0 0x55f65c25e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50346/0 (socket says 172.21.15.44:50346) 2023-09-10T00:43:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.188+0000 7f5e79c74700 1 -- 172.21.15.44:0/33738269 learned_addr learned my addr 172.21.15.44:0/33738269 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.188+0000 7f5e79473700 1 --2- 172.21.15.44:0/33738269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c258710 0x55f65c258b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.188+0000 7f5e7a475700 1 --2- 172.21.15.44:0/33738269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c2590d0 0x55f65c265340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.188+0000 7f5e79c74700 1 -- 172.21.15.44:0/33738269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c258710 msgr2=0x55f65c258b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.188+0000 7f5e79c74700 1 --2- 172.21.15.44:0/33738269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c258710 0x55f65c258b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.188+0000 7f5e79c74700 1 -- 172.21.15.44:0/33738269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c2590d0 msgr2=0x55f65c265340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.189+0000 7f5e79c74700 1 --2- 172.21.15.44:0/33738269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c2590d0 0x55f65c265340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.189+0000 7f5e79c74700 1 -- 172.21.15.44:0/33738269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f65bfd2d80 con 0x55f65c25e5b0 2023-09-10T00:43:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.189+0000 7f5e7a475700 1 --2- 172.21.15.44:0/33738269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c2590d0 0x55f65c265340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:43:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.189+0000 7f5e79c74700 1 --2- 172.21.15.44:0/33738269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c25e5b0 0x55f65c25e990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5e700043c0 tx=0x7f5e70014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e485b6387d1f8b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.189+0000 7f5e5ffff700 1 -- 172.21.15.44:0/33738269 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70003d10 con 0x55f65c25e5b0 2023-09-10T00:43:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.189+0000 7f5e5ffff700 1 -- 172.21.15.44:0/33738269 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e70003e70 con 0x55f65c25e5b0 2023-09-10T00:43:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.189+0000 7f5e5ffff700 1 -- 172.21.15.44:0/33738269 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000e730 con 0x55f65c25e5b0 2023-09-10T00:43:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.190+0000 7f5e8809eb80 1 -- 172.21.15.44:0/33738269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c25e5b0 msgr2=0x55f65c25e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.190+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/33738269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c25e5b0 0x55f65c25e990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5e700043c0 tx=0x7f5e70014000 comp rx=0 tx=0).stop 2023-09-10T00:43:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.190+0000 7f5e8809eb80 1 -- 172.21.15.44:0/33738269 shutdown_connections 2023-09-10T00:43:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.190+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/33738269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c2590d0 0x55f65c265340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.190+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/33738269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c25e5b0 0x55f65c25e990 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.190+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/33738269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c258710 0x55f65c258b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.190+0000 7f5e8809eb80 1 -- 172.21.15.44:0/33738269 >> 172.21.15.44:0/33738269 conn(0x55f65c1c9030 msgr2=0x55f65c24f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.190+0000 7f5e8809eb80 1 -- 172.21.15.44:0/33738269 shutdown_connections 2023-09-10T00:43:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.190+0000 7f5e8809eb80 1 -- 172.21.15.44:0/33738269 wait complete. 2023-09-10T00:43:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.190+0000 7f5e8809eb80 1 -- start start 2023-09-10T00:43:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.190+0000 7f5e8809eb80 1 Processor -- start 2023-09-10T00:43:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e8809eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c258710 0x55f65c2651e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e8809eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c2590d0 0x55f65c262500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e8809eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c25e5b0 0x55f65c262ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e8809eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65c144b10 con 0x55f65c258710 2023-09-10T00:43:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e8809eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65c144810 con 0x55f65c25e5b0 2023-09-10T00:43:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e8809eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65c0288c0 con 0x55f65c2590d0 2023-09-10T00:43:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e8809eb80 10 client.0 _ll_register_callbacks cb 0x55f65c08ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:43:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e79c74700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c258710 0x55f65c2651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e7a475700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c25e5b0 0x55f65c262ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e79473700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c2590d0 0x55f65c262500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e79c74700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c258710 0x55f65c2651e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50352/0 (socket says 172.21.15.44:50352) 2023-09-10T00:43:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e79c74700 1 -- 172.21.15.44:0/181281495 learned_addr learned my addr 172.21.15.44:0/181281495 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e79c74700 1 -- 172.21.15.44:0/181281495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c2590d0 msgr2=0x55f65c262500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e79c74700 1 --2- 172.21.15.44:0/181281495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c2590d0 0x55f65c262500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e79c74700 1 -- 172.21.15.44:0/181281495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c25e5b0 msgr2=0x55f65c262ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e79c74700 1 --2- 172.21.15.44:0/181281495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c25e5b0 0x55f65c262ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e79c74700 1 -- 172.21.15.44:0/181281495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e70017070 con 0x55f65c258710 2023-09-10T00:43:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e7a475700 1 --2- 172.21.15.44:0/181281495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c25e5b0 0x55f65c262ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.191+0000 7f5e79473700 1 --2- 172.21.15.44:0/181281495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c2590d0 0x55f65c262500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.192+0000 7f5e79c74700 1 --2- 172.21.15.44:0/181281495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c258710 0x55f65c2651e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5e700041d0 tx=0x7f5e70005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82b42813f8dc19 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.192+0000 7f5e5d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:43:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.192+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/181281495 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7002c070 con 0x55f65c258710 2023-09-10T00:43:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.192+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/181281495 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f5e70004710 con 0x55f65c258710 2023-09-10T00:43:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.192+0000 7f5e8809eb80 1 -- 172.21.15.44:0/181281495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f65c0fe490 con 0x55f65c258710 2023-09-10T00:43:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.192+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/181281495 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e700048d0 con 0x55f65c258710 2023-09-10T00:43:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.192+0000 7f5e8809eb80 10 client.16455 fetch_fsmap learned FSMap version 34 2023-09-10T00:43:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.192+0000 7f5e8809eb80 1 -- 172.21.15.44:0/181281495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f65bfbbc00 con 0x55f65c258710 2023-09-10T00:43:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.192+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/181281495 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f65c0fe490 con 0x55f65c258710 2023-09-10T00:43:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.192+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/181281495 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5e70024020 con 0x55f65c258710 2023-09-10T00:43:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.193+0000 7f5e8809eb80 10 client.16455 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:43:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.193+0000 7f5e8809eb80 10 client.16455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:43:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.193+0000 7f5e8809eb80 20 client.16455 populate_metadata read hostname 'smithi044' 2023-09-10T00:43:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.193+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/181281495 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f65c2726d0 0x55f65c274af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.193+0000 7f5e8809eb80 4 client.16455 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:43:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.193+0000 7f5e8809eb80 1 -- 172.21.15.44:0/181281495 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f65c1bb840 con 0x55f65c2726d0 2023-09-10T00:43:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.193+0000 7f5e79473700 1 --2- 172.21.15.44:0/181281495 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f65c2726d0 0x55f65c274af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.193+0000 7f5e5d7fa700 10 client.16455 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.193+0000 7f5e79473700 1 --2- 172.21.15.44:0/181281495 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f65c2726d0 0x55f65c274af0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f753375b9365a2c7 server_cookie=51f801b99a93c6cc in_seq=0 out_seq=0 2023-09-10T00:43:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.194+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/181281495 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f65c1bb840 con 0x55f65c2726d0 2023-09-10T00:43:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.194+0000 7f5e5d7fa700 10 client.16455 handle_command_reply: tid=0 2023-09-10T00:43:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.194+0000 7f5e8809eb80 1 client.16455 shutdown 2023-09-10T00:43:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.194+0000 7f5e8809eb80 1 -- 172.21.15.44:0/181281495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c258710 msgr2=0x55f65c2651e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.194+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/181281495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c258710 0x55f65c2651e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5e700041d0 tx=0x7f5e70005270 comp rx=0 tx=0).stop 2023-09-10T00:43:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.194+0000 7f5e8809eb80 1 -- 172.21.15.44:0/181281495 shutdown_connections 2023-09-10T00:43:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.194+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/181281495 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f65c2726d0 0x55f65c274af0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.194+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/181281495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c25e5b0 0x55f65c262ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.194+0000 7f5e5d7fa700 0 client.16455 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.194+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/181281495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c258710 0x55f65c2651e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.194+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/181281495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c2590d0 0x55f65c262500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.194+0000 7f5e8809eb80 1 -- 172.21.15.44:0/181281495 >> 172.21.15.44:0/181281495 conn(0x55f65c1c9030 msgr2=0x55f65c24e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.194+0000 7f5e8809eb80 1 -- 172.21.15.44:0/181281495 shutdown_connections 2023-09-10T00:43:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.194+0000 7f5e8809eb80 1 -- 172.21.15.44:0/181281495 wait complete. 2023-09-10T00:43:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.194+0000 7f5e8809eb80 20 client.16455 trim_cache size 0 max 16384 2023-09-10T00:43:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.212+0000 7f5e8809eb80 1 Processor -- start 2023-09-10T00:43:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.212+0000 7f5e8809eb80 1 -- start start 2023-09-10T00:43:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.212+0000 7f5e8809eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c258710 0x55f65c269940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e8809eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c2590d0 0x55f65c25c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e8809eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c25e5b0 0x55f65c25cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e8809eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65c25e210 con 0x55f65c25e5b0 2023-09-10T00:43:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e8809eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65c263ab0 con 0x55f65c2590d0 2023-09-10T00:43:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e8809eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65c250b80 con 0x55f65c258710 2023-09-10T00:43:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e79c74700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c258710 0x55f65c269940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e7a475700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c25e5b0 0x55f65c25cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e79473700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c2590d0 0x55f65c25c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e79c74700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c258710 0x55f65c269940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57242/0 (socket says 172.21.15.44:57242) 2023-09-10T00:43:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e7a475700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c25e5b0 0x55f65c25cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50364/0 (socket says 172.21.15.44:50364) 2023-09-10T00:43:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e79c74700 1 -- 172.21.15.44:0/3467333792 learned_addr learned my addr 172.21.15.44:0/3467333792 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e79c74700 1 -- 172.21.15.44:0/3467333792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c2590d0 msgr2=0x55f65c25c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e79c74700 1 --2- 172.21.15.44:0/3467333792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c2590d0 0x55f65c25c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e79c74700 1 -- 172.21.15.44:0/3467333792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c25e5b0 msgr2=0x55f65c25cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e79c74700 1 --2- 172.21.15.44:0/3467333792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c25e5b0 0x55f65c25cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e79c74700 1 -- 172.21.15.44:0/3467333792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f65bff91f0 con 0x55f65c258710 2023-09-10T00:43:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e7a475700 1 --2- 172.21.15.44:0/3467333792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c25e5b0 0x55f65c25cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e79c74700 1 --2- 172.21.15.44:0/3467333792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c258710 0x55f65c269940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5e70026420 tx=0x7f5e7001ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4da79af836c109f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.213+0000 7f5e68ff9700 1 -- 172.21.15.44:0/3467333792 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7002a070 con 0x55f65c258710 2023-09-10T00:43:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.214+0000 7f5e68ff9700 1 -- 172.21.15.44:0/3467333792 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e700052e0 con 0x55f65c258710 2023-09-10T00:43:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.214+0000 7f5e68ff9700 1 -- 172.21.15.44:0/3467333792 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7002c040 con 0x55f65c258710 2023-09-10T00:43:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.214+0000 7f5e8809eb80 1 -- 172.21.15.44:0/3467333792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c258710 msgr2=0x55f65c269940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.214+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/3467333792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c258710 0x55f65c269940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5e70026420 tx=0x7f5e7001ecf0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.214+0000 7f5e8809eb80 1 -- 172.21.15.44:0/3467333792 shutdown_connections 2023-09-10T00:43:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.214+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/3467333792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c25e5b0 0x55f65c25cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.214+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/3467333792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c258710 0x55f65c269940 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.214+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/3467333792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c2590d0 0x55f65c25c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.214+0000 7f5e8809eb80 1 -- 172.21.15.44:0/3467333792 >> 172.21.15.44:0/3467333792 conn(0x55f65c1c9030 msgr2=0x55f65c25d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.214+0000 7f5e8809eb80 1 -- 172.21.15.44:0/3467333792 shutdown_connections 2023-09-10T00:43:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.215+0000 7f5e8809eb80 1 -- 172.21.15.44:0/3467333792 wait complete. 2023-09-10T00:43:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.215+0000 7f5e8809eb80 1 -- start start 2023-09-10T00:43:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.215+0000 7f5e8809eb80 1 Processor -- start 2023-09-10T00:43:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.215+0000 7f5e8809eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c258710 0x55f65c269a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.215+0000 7f5e8809eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c2590d0 0x55f65c268000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.215+0000 7f5e8809eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c25e5b0 0x55f65c268540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.215+0000 7f5e8809eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65c268c20 con 0x55f65c258710 2023-09-10T00:43:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.215+0000 7f5e8809eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65c268d90 con 0x55f65c2590d0 2023-09-10T00:43:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.215+0000 7f5e8809eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65c257330 con 0x55f65c25e5b0 2023-09-10T00:43:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.215+0000 7f5e8809eb80 10 client.0 _ll_register_callbacks cb 0x7f5e6c0097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:43:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.215+0000 7f5e79c74700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c258710 0x55f65c269a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.215+0000 7f5e79473700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c2590d0 0x55f65c268000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.215+0000 7f5e79c74700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c258710 0x55f65c269a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50366/0 (socket says 172.21.15.44:50366) 2023-09-10T00:43:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.215+0000 7f5e79c74700 1 -- 172.21.15.44:0/4190598011 learned_addr learned my addr 172.21.15.44:0/4190598011 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.215+0000 7f5e7a475700 1 --2- 172.21.15.44:0/4190598011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c25e5b0 0x55f65c268540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.216+0000 7f5e7a475700 1 -- 172.21.15.44:0/4190598011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c2590d0 msgr2=0x55f65c268000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.216+0000 7f5e7a475700 1 --2- 172.21.15.44:0/4190598011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c2590d0 0x55f65c268000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.216+0000 7f5e7a475700 1 -- 172.21.15.44:0/4190598011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c258710 msgr2=0x55f65c269a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.216+0000 7f5e7a475700 1 --2- 172.21.15.44:0/4190598011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c258710 0x55f65c269a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.216+0000 7f5e7a475700 1 -- 172.21.15.44:0/4190598011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e741a3530 con 0x55f65c25e5b0 2023-09-10T00:43:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.216+0000 7f5e79c74700 1 --2- 172.21.15.44:0/4190598011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c258710 0x55f65c269a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.216+0000 7f5e7a475700 1 --2- 172.21.15.44:0/4190598011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c25e5b0 0x55f65c268540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0c91e0 tx=0x7f5e6c093ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19db02ab8bace43f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.216+0000 7f5e5d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:43:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.216+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/4190598011 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0986c0 con 0x55f65c25e5b0 2023-09-10T00:43:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.216+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/4190598011 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f5e6c0b8040 con 0x55f65c25e5b0 2023-09-10T00:43:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.216+0000 7f5e8809eb80 1 -- 172.21.15.44:0/4190598011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e70003e70 con 0x55f65c25e5b0 2023-09-10T00:43:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.216+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/4190598011 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e6c098fd0 con 0x55f65c25e5b0 2023-09-10T00:43:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.216+0000 7f5e8809eb80 10 client.26068 fetch_fsmap learned FSMap version 34 2023-09-10T00:43:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.216+0000 7f5e8809eb80 1 -- 172.21.15.44:0/4190598011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f65bfbbc00 con 0x55f65c25e5b0 2023-09-10T00:43:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.217+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/4190598011 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e70003e70 con 0x55f65c25e5b0 2023-09-10T00:43:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.217+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/4190598011 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5e6c0b10a0 con 0x55f65c25e5b0 2023-09-10T00:43:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.217+0000 7f5e8809eb80 10 client.26068 fetch_fsmap finished waiting for FSMap version 34 2023-09-10T00:43:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.217+0000 7f5e8809eb80 10 client.26068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:43:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.217+0000 7f5e8809eb80 20 client.26068 populate_metadata read hostname 'smithi044' 2023-09-10T00:43:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.217+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/4190598011 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f65c2cb450 0x55f65c2cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.217+0000 7f5e8809eb80 4 client.26068 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:43:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.217+0000 7f5e8809eb80 1 -- 172.21.15.44:0/4190598011 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f65c19f8c0 con 0x55f65c2cb450 2023-09-10T00:43:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.217+0000 7f5e79c74700 1 --2- 172.21.15.44:0/4190598011 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f65c2cb450 0x55f65c2cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.217+0000 7f5e5d7fa700 10 client.26068 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.217+0000 7f5e79c74700 1 --2- 172.21.15.44:0/4190598011 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f65c2cb450 0x55f65c2cd870 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83181e133c5ebc81 server_cookie=675b927c3c6f7e1b in_seq=0 out_seq=0 2023-09-10T00:43:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.230+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/4190598011 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f65c19f8c0 con 0x55f65c2cb450 2023-09-10T00:43:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.230+0000 7f5e5d7fa700 10 client.26068 handle_command_reply: tid=0 2023-09-10T00:43:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.230+0000 7f5e8809eb80 1 client.26068 shutdown 2023-09-10T00:43:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.230+0000 7f5e8809eb80 1 -- 172.21.15.44:0/4190598011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c25e5b0 msgr2=0x55f65c268540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.230+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/4190598011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c25e5b0 0x55f65c268540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0c91e0 tx=0x7f5e6c093ec0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.230+0000 7f5e8809eb80 1 -- 172.21.15.44:0/4190598011 shutdown_connections 2023-09-10T00:43:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.230+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/4190598011 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f65c2cb450 0x55f65c2cd870 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.230+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/4190598011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f65c25e5b0 0x55f65c268540 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.230+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/4190598011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65c258710 0x55f65c269a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.230+0000 7f5e5d7fa700 0 client.26068 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.230+0000 7f5e8809eb80 1 --2- 172.21.15.44:0/4190598011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f65c2590d0 0x55f65c268000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.230+0000 7f5e8809eb80 1 -- 172.21.15.44:0/4190598011 >> 172.21.15.44:0/4190598011 conn(0x55f65c1c9030 msgr2=0x55f65c25d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.230+0000 7f5e8809eb80 1 -- 172.21.15.44:0/4190598011 shutdown_connections 2023-09-10T00:43:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.230+0000 7f5e8809eb80 1 -- 172.21.15.44:0/4190598011 wait complete. 2023-09-10T00:43:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.230+0000 7f5e8809eb80 20 client.26068 trim_cache size 0 max 16384 2023-09-10T00:43:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:43:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:43:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:43:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:43:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.231+0000 7f5e7bed8700 1 -- 172.21.15.44:0/1000970232 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5e600a07b0 msgr2=0x7f5e600a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.231+0000 7f5e7bed8700 1 --2- 172.21.15.44:0/1000970232 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5e600a07b0 0x7f5e600a2c70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e70011490 tx=0x7f5e70009260 comp rx=0 tx=0).stop 2023-09-10T00:43:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.231+0000 7f5e7bed8700 1 -- 172.21.15.44:0/1000970232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e74106690 msgr2=0x7f5e741963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.231+0000 7f5e7bed8700 1 --2- 172.21.15.44:0/1000970232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e74106690 0x7f5e741963a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00ef30 tx=0x7f5e6c00c3a0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.231+0000 7f5e7bed8700 1 -- 172.21.15.44:0/1000970232 shutdown_connections 2023-09-10T00:43:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.231+0000 7f5e7bed8700 1 --2- 172.21.15.44:0/1000970232 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5e600a07b0 0x7f5e600a2c70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.231+0000 7f5e7bed8700 1 --2- 172.21.15.44:0/1000970232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e74106690 0x7f5e741963a0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.231+0000 7f5e7bed8700 1 --2- 172.21.15.44:0/1000970232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e741005d0 0x7f5e7419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.231+0000 7f5e7bed8700 1 --2- 172.21.15.44:0/1000970232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e74100f90 0x7f5e7419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.231+0000 7f5e7bed8700 1 -- 172.21.15.44:0/1000970232 >> 172.21.15.44:0/1000970232 conn(0x7f5e740fc0c0 msgr2=0x7f5e740fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.231+0000 7f5e7bed8700 1 -- 172.21.15.44:0/1000970232 shutdown_connections 2023-09-10T00:43:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:20.231+0000 7f5e7bed8700 1 -- 172.21.15.44:0/1000970232 wait complete. 2023-09-10T00:43:20.846 INFO:teuthology.orchestra.run.smithi044.stderr:13200000 of 50000000 tuples (26%) done (elapsed 140.48 s, remaining 391.64 s) 2023-09-10T00:43:21.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:20 smithi131 ceph-mon[103713]: pgmap v83: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 34 MiB/s wr, 25 op/s 2023-09-10T00:43:21.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:20 smithi044 ceph-mon[98676]: pgmap v83: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 34 MiB/s wr, 25 op/s 2023-09-10T00:43:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:20 smithi099 ceph-mon[102692]: pgmap v83: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 34 MiB/s wr, 25 op/s 2023-09-10T00:43:21.692 INFO:teuthology.orchestra.run.smithi044.stderr:13300000 of 50000000 tuples (26%) done (elapsed 141.33 s, remaining 389.97 s) 2023-09-10T00:43:22.551 INFO:teuthology.orchestra.run.smithi044.stderr:13400000 of 50000000 tuples (26%) done (elapsed 142.19 s, remaining 388.36 s) 2023-09-10T00:43:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:22 smithi131 ceph-mon[103713]: pgmap v84: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 52 MiB/s wr, 36 op/s 2023-09-10T00:43:23.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:22 smithi044 ceph-mon[98676]: pgmap v84: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 52 MiB/s wr, 36 op/s 2023-09-10T00:43:23.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:22 smithi099 ceph-mon[102692]: pgmap v84: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 52 MiB/s wr, 36 op/s 2023-09-10T00:43:23.430 INFO:teuthology.orchestra.run.smithi044.stderr:13500000 of 50000000 tuples (27%) done (elapsed 143.06 s, remaining 386.80 s) 2023-09-10T00:43:24.271 INFO:teuthology.orchestra.run.smithi044.stderr:13600000 of 50000000 tuples (27%) done (elapsed 143.91 s, remaining 385.16 s) 2023-09-10T00:43:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:24 smithi131 ceph-mon[103713]: pgmap v85: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 35 MiB/s wr, 22 op/s 2023-09-10T00:43:25.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:24 smithi044 ceph-mon[98676]: pgmap v85: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 35 MiB/s wr, 22 op/s 2023-09-10T00:43:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:24 smithi099 ceph-mon[102692]: pgmap v85: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 35 MiB/s wr, 22 op/s 2023-09-10T00:43:25.187 INFO:teuthology.orchestra.run.smithi044.stderr:13700000 of 50000000 tuples (27%) done (elapsed 144.82 s, remaining 383.72 s) 2023-09-10T00:43:26.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:25 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:26.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:25 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:26.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:25 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:26.176 INFO:teuthology.orchestra.run.smithi044.stderr:13800000 of 50000000 tuples (27%) done (elapsed 145.81 s, remaining 382.49 s) 2023-09-10T00:43:27.043 INFO:teuthology.orchestra.run.smithi044.stderr:13900000 of 50000000 tuples (27%) done (elapsed 146.68 s, remaining 380.94 s) 2023-09-10T00:43:27.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:26 smithi131 ceph-mon[103713]: pgmap v86: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 52 MiB/s wr, 45 op/s 2023-09-10T00:43:27.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:26 smithi044 ceph-mon[98676]: pgmap v86: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 52 MiB/s wr, 45 op/s 2023-09-10T00:43:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:26 smithi099 ceph-mon[102692]: pgmap v86: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 52 MiB/s wr, 45 op/s 2023-09-10T00:43:27.877 INFO:teuthology.orchestra.run.smithi044.stderr:14000000 of 50000000 tuples (28%) done (elapsed 147.51 s, remaining 379.32 s) 2023-09-10T00:43:28.706 INFO:teuthology.orchestra.run.smithi044.stderr:14100000 of 50000000 tuples (28%) done (elapsed 148.34 s, remaining 377.69 s) 2023-09-10T00:43:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:28 smithi131 ceph-mon[103713]: pgmap v87: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 MiB/s wr, 33 op/s 2023-09-10T00:43:29.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:28 smithi044 ceph-mon[98676]: pgmap v87: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 MiB/s wr, 33 op/s 2023-09-10T00:43:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:28 smithi099 ceph-mon[102692]: pgmap v87: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 MiB/s wr, 33 op/s 2023-09-10T00:43:29.617 INFO:teuthology.orchestra.run.smithi044.stderr:14200000 of 50000000 tuples (28%) done (elapsed 149.25 s, remaining 376.28 s) 2023-09-10T00:43:30.459 INFO:teuthology.orchestra.run.smithi044.stderr:14300000 of 50000000 tuples (28%) done (elapsed 150.09 s, remaining 374.71 s) 2023-09-10T00:43:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:30 smithi131 ceph-mon[103713]: pgmap v88: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 MiB/s wr, 33 op/s 2023-09-10T00:43:31.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:30 smithi044 ceph-mon[98676]: pgmap v88: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 MiB/s wr, 33 op/s 2023-09-10T00:43:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:30 smithi099 ceph-mon[102692]: pgmap v88: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 MiB/s wr, 33 op/s 2023-09-10T00:43:31.318 INFO:teuthology.orchestra.run.smithi044.stderr:14400000 of 50000000 tuples (28%) done (elapsed 150.95 s, remaining 373.19 s) 2023-09-10T00:43:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:14500000 of 50000000 tuples (29%) done (elapsed 151.79 s, remaining 371.63 s) 2023-09-10T00:43:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:32 smithi131 ceph-mon[103713]: pgmap v89: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 52 MiB/s wr, 59 op/s 2023-09-10T00:43:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:32 smithi044 ceph-mon[98676]: pgmap v89: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 52 MiB/s wr, 59 op/s 2023-09-10T00:43:33.095 INFO:teuthology.orchestra.run.smithi044.stderr:14600000 of 50000000 tuples (29%) done (elapsed 152.73 s, remaining 370.32 s) 2023-09-10T00:43:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:32 smithi099 ceph-mon[102692]: pgmap v89: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 52 MiB/s wr, 59 op/s 2023-09-10T00:43:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.352+0000 7f1f15e1e700 1 -- 172.21.15.44:0/4249858266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f101055b0 msgr2=0x7f1f10105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.352+0000 7f1f15e1e700 1 --2- 172.21.15.44:0/4249858266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f101055b0 0x7f1f10105990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1f000099c0 tx=0x7f1f00009cd0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.352+0000 7f1f15e1e700 1 -- 172.21.15.44:0/4249858266 shutdown_connections 2023-09-10T00:43:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.352+0000 7f1f15e1e700 1 --2- 172.21.15.44:0/4249858266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f10100120 0x7f1f1010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.352+0000 7f1f15e1e700 1 --2- 172.21.15.44:0/4249858266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f101055b0 0x7f1f10105990 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.352+0000 7f1f15e1e700 1 --2- 172.21.15.44:0/4249858266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f100ff760 0x7f1f100ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.352+0000 7f1f15e1e700 1 -- 172.21.15.44:0/4249858266 >> 172.21.15.44:0/4249858266 conn(0x7f1f10074f30 msgr2=0x7f1f10075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.352+0000 7f1f15e1e700 1 -- 172.21.15.44:0/4249858266 shutdown_connections 2023-09-10T00:43:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.352+0000 7f1f15e1e700 1 -- 172.21.15.44:0/4249858266 wait complete. 2023-09-10T00:43:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.353+0000 7f1f15e1e700 1 Processor -- start 2023-09-10T00:43:33.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.353+0000 7f1f15e1e700 1 -- start start 2023-09-10T00:43:33.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.353+0000 7f1f15e1e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f100ff760 0x7f1f10197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.353+0000 7f1f15e1e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f10100120 0x7f1f10197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.353+0000 7f1f15e1e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f101055b0 0x7f1f101a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.353+0000 7f1f15e1e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f101a2b00 con 0x7f1f100ff760 2023-09-10T00:43:33.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.353+0000 7f1f15e1e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f101a2c70 con 0x7f1f10100120 2023-09-10T00:43:33.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.353+0000 7f1f15e1e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f101a2e30 con 0x7f1f101055b0 2023-09-10T00:43:33.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.353+0000 7f1f0f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f100ff760 0x7f1f10197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.353+0000 7f1f0f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f100ff760 0x7f1f10197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50380/0 (socket says 172.21.15.44:50380) 2023-09-10T00:43:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.353+0000 7f1f0f7fe700 1 -- 172.21.15.44:0/685718689 learned_addr learned my addr 172.21.15.44:0/685718689 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.353+0000 7f1f0ffff700 1 --2- 172.21.15.44:0/685718689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f101055b0 0x7f1f101a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.353+0000 7f1f0effd700 1 --2- 172.21.15.44:0/685718689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f10100120 0x7f1f10197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.354+0000 7f1f0f7fe700 1 -- 172.21.15.44:0/685718689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f101055b0 msgr2=0x7f1f101a2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.354+0000 7f1f0f7fe700 1 --2- 172.21.15.44:0/685718689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f101055b0 0x7f1f101a2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.354+0000 7f1f0f7fe700 1 -- 172.21.15.44:0/685718689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f10100120 msgr2=0x7f1f10197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.354+0000 7f1f0f7fe700 1 --2- 172.21.15.44:0/685718689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f10100120 0x7f1f10197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.354+0000 7f1f0f7fe700 1 -- 172.21.15.44:0/685718689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f000095f0 con 0x7f1f100ff760 2023-09-10T00:43:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.354+0000 7f1f0ffff700 1 --2- 172.21.15.44:0/685718689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f101055b0 0x7f1f101a2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.354+0000 7f1f0f7fe700 1 --2- 172.21.15.44:0/685718689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f100ff760 0x7f1f10197690 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1f00006010 tx=0x7f1f0000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.354+0000 7f1f0cff9700 1 -- 172.21.15.44:0/685718689 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0001d070 con 0x7f1f100ff760 2023-09-10T00:43:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.354+0000 7f1f0cff9700 1 -- 172.21.15.44:0/685718689 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f00005030 con 0x7f1f100ff760 2023-09-10T00:43:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.354+0000 7f1f15e1e700 1 -- 172.21.15.44:0/685718689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f101a30b0 con 0x7f1f100ff760 2023-09-10T00:43:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.354+0000 7f1f0cff9700 1 -- 172.21.15.44:0/685718689 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f000049c0 con 0x7f1f100ff760 2023-09-10T00:43:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.354+0000 7f1f15e1e700 1 -- 172.21.15.44:0/685718689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f101a34d0 con 0x7f1f100ff760 2023-09-10T00:43:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.356+0000 7f1f0cff9700 1 -- 172.21.15.44:0/685718689 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1f000051a0 con 0x7f1f100ff760 2023-09-10T00:43:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.357+0000 7f1f0cff9700 1 --2- 172.21.15.44:0/685718689 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1ef80a07b0 0x7f1ef80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.357+0000 7f1f0effd700 1 --2- 172.21.15.44:0/685718689 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1ef80a07b0 0x7f1ef80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.357+0000 7f1f0cff9700 1 -- 172.21.15.44:0/685718689 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f1f000cd010 con 0x7f1f100ff760 2023-09-10T00:43:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.358+0000 7f1f0effd700 1 --2- 172.21.15.44:0/685718689 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1ef80a07b0 0x7f1ef80a2c70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f10068ad0 tx=0x7f1f04009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.359+0000 7f1f21fe4b80 1 Processor -- start 2023-09-10T00:43:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.359+0000 7f1f21fe4b80 1 -- start start 2023-09-10T00:43:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.359+0000 7f1f21fe4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad81a0 0x558bbaad8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.359+0000 7f1f21fe4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 0x558bbaad2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.359+0000 7f1f21fe4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad2cc0 0x558bbaadef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.359+0000 7f1f21fe4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbaa2b6d0 con 0x558bbaad2300 2023-09-10T00:43:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.359+0000 7f1f21fe4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbaa28ff0 con 0x558bbaad81a0 2023-09-10T00:43:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.359+0000 7f1f21fe4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbaa248e0 con 0x558bbaad2cc0 2023-09-10T00:43:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.359+0000 7f1f0f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad81a0 0x558bbaad8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.359+0000 7f1f0ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad2cc0 0x558bbaadef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.359+0000 7f1f0f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad81a0 0x558bbaad8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52128/0 (socket says 172.21.15.44:52128) 2023-09-10T00:43:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.359+0000 7f1f0f7fe700 1 -- 172.21.15.44:0/2040664460 learned_addr learned my addr 172.21.15.44:0/2040664460 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.360+0000 7f1f0effd700 1 --2- 172.21.15.44:0/2040664460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 0x558bbaad2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.360+0000 7f1f0f7fe700 1 -- 172.21.15.44:0/2040664460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad2cc0 msgr2=0x558bbaadef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.360+0000 7f1f0f7fe700 1 --2- 172.21.15.44:0/2040664460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad2cc0 0x558bbaadef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.360+0000 7f1f0f7fe700 1 -- 172.21.15.44:0/2040664460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 msgr2=0x558bbaad2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.360+0000 7f1f0f7fe700 1 --2- 172.21.15.44:0/2040664460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 0x558bbaad2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.360+0000 7f1f0f7fe700 1 -- 172.21.15.44:0/2040664460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bba850d80 con 0x558bbaad81a0 2023-09-10T00:43:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.360+0000 7f1f0effd700 1 --2- 172.21.15.44:0/2040664460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 0x558bbaad2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:43:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.360+0000 7f1f0ffff700 1 --2- 172.21.15.44:0/2040664460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad2cc0 0x558bbaadef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.360+0000 7f1f0f7fe700 1 --2- 172.21.15.44:0/2040664460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad81a0 0x558bbaad8580 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f000966b0 tx=0x7f1f0009a9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2890c801d06d781 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.360+0000 7f1ef5ffb700 1 -- 172.21.15.44:0/2040664460 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f000a2190 con 0x558bbaad81a0 2023-09-10T00:43:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.360+0000 7f1ef5ffb700 1 -- 172.21.15.44:0/2040664460 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f00098550 con 0x558bbaad81a0 2023-09-10T00:43:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.360+0000 7f1ef5ffb700 1 -- 172.21.15.44:0/2040664460 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f000996a0 con 0x558bbaad81a0 2023-09-10T00:43:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.361+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/2040664460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad81a0 msgr2=0x558bbaad8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.361+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/2040664460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad81a0 0x558bbaad8580 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f000966b0 tx=0x7f1f0009a9a0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.361+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/2040664460 shutdown_connections 2023-09-10T00:43:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.361+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/2040664460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad2cc0 0x558bbaadef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.361+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/2040664460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad81a0 0x558bbaad8580 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.361+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/2040664460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 0x558bbaad2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.361+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/2040664460 >> 172.21.15.44:0/2040664460 conn(0x558bbaa47030 msgr2=0x558bbaacd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.361+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/2040664460 shutdown_connections 2023-09-10T00:43:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.361+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/2040664460 wait complete. 2023-09-10T00:43:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.361+0000 7f1f21fe4b80 1 -- start start 2023-09-10T00:43:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f21fe4b80 1 Processor -- start 2023-09-10T00:43:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f21fe4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 0x558bbaae1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f21fe4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 0x558bbaae2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f21fe4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad81a0 0x558bbaadbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f21fe4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bba9c2b10 con 0x558bbaad2300 2023-09-10T00:43:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f21fe4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bba9c2810 con 0x558bbaad2cc0 2023-09-10T00:43:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f21fe4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bba8a68c0 con 0x558bbaad81a0 2023-09-10T00:43:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f21fe4b80 10 client.0 _ll_register_callbacks cb 0x558bba90ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:43:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f0effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 0x558bbaae2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f0f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 0x558bbaae1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f0effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 0x558bbaae2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52136/0 (socket says 172.21.15.44:52136) 2023-09-10T00:43:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f0effd700 1 -- 172.21.15.44:0/551090613 learned_addr learned my addr 172.21.15.44:0/551090613 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f0ffff700 1 --2- 172.21.15.44:0/551090613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad81a0 0x558bbaadbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f0ffff700 1 -- 172.21.15.44:0/551090613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 msgr2=0x558bbaae2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f0ffff700 1 --2- 172.21.15.44:0/551090613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 0x558bbaae2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f0ffff700 1 -- 172.21.15.44:0/551090613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 msgr2=0x558bbaae1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f0ffff700 1 --2- 172.21.15.44:0/551090613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 0x558bbaae1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f0ffff700 1 -- 172.21.15.44:0/551090613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f08003a60 con 0x558bbaad81a0 2023-09-10T00:43:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.362+0000 7f1f0f7fe700 1 --2- 172.21.15.44:0/551090613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 0x558bbaae1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.363+0000 7f1f0ffff700 1 --2- 172.21.15.44:0/551090613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad81a0 0x558bbaadbe80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1f08003db0 tx=0x7f1f0800e7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfd86f49358599b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.363+0000 7f1eeeffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:43:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.363+0000 7f1eeeffd700 1 -- 172.21.15.44:0/551090613 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08004db0 con 0x558bbaad81a0 2023-09-10T00:43:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.363+0000 7f1eeeffd700 1 -- 172.21.15.44:0/551090613 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f1f080090d0 con 0x558bbaad81a0 2023-09-10T00:43:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.363+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/551090613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bba97c490 con 0x558bbaad81a0 2023-09-10T00:43:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.363+0000 7f1eeeffd700 1 -- 172.21.15.44:0/551090613 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f0800eec0 con 0x558bbaad81a0 2023-09-10T00:43:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.363+0000 7f1f21fe4b80 10 client.26080 fetch_fsmap learned FSMap version 35 2023-09-10T00:43:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.363+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/551090613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f1f101a34d0 con 0x558bbaad81a0 2023-09-10T00:43:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.363+0000 7f1eeeffd700 1 -- 172.21.15.44:0/551090613 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558bba97c490 con 0x558bbaad81a0 2023-09-10T00:43:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.363+0000 7f1eeeffd700 1 -- 172.21.15.44:0/551090613 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f1f0801c030 con 0x558bbaad81a0 2023-09-10T00:43:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.363+0000 7f1f21fe4b80 10 client.26080 fetch_fsmap finished waiting for FSMap version 35 2023-09-10T00:43:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.363+0000 7f1f21fe4b80 10 client.26080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:43:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.364+0000 7f1f21fe4b80 20 client.26080 populate_metadata read hostname 'smithi044' 2023-09-10T00:43:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.364+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/551090613 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558bbaaec220 0x558bbaaee640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.364+0000 7f1f21fe4b80 4 client.26080 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:43:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.364+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/551090613 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bbaa39840 con 0x558bbaaec220 2023-09-10T00:43:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.364+0000 7f1f0f7fe700 1 --2- 172.21.15.44:0/551090613 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558bbaaec220 0x558bbaaee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.364+0000 7f1eeeffd700 10 client.26080 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.364+0000 7f1f0f7fe700 1 --2- 172.21.15.44:0/551090613 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558bbaaec220 0x558bbaaee640 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ba00d3ba30b4e66 server_cookie=cd801cc5f5c50684 in_seq=0 out_seq=0 2023-09-10T00:43:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.365+0000 7f1eeeffd700 1 -- 172.21.15.44:0/551090613 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558bbaa39840 con 0x558bbaaec220 2023-09-10T00:43:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.365+0000 7f1eeeffd700 10 client.26080 handle_command_reply: tid=0 2023-09-10T00:43:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.365+0000 7f1f21fe4b80 1 client.26080 shutdown 2023-09-10T00:43:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.365+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/551090613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad81a0 msgr2=0x558bbaadbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.365+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/551090613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad81a0 0x558bbaadbe80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1f08003db0 tx=0x7f1f0800e7d0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.365+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/551090613 shutdown_connections 2023-09-10T00:43:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.365+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/551090613 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558bbaaec220 0x558bbaaee640 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.365+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/551090613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad81a0 0x558bbaadbe80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.365+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/551090613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 0x558bbaae1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.365+0000 7f1eeeffd700 0 client.26080 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.365+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/551090613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 0x558bbaae2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.365+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/551090613 >> 172.21.15.44:0/551090613 conn(0x558bbaa47030 msgr2=0x558bbaacc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.365+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/551090613 shutdown_connections 2023-09-10T00:43:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.365+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/551090613 wait complete. 2023-09-10T00:43:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.366+0000 7f1f21fe4b80 20 client.26080 trim_cache size 0 max 16384 2023-09-10T00:43:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.383+0000 7f1f21fe4b80 1 Processor -- start 2023-09-10T00:43:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.383+0000 7f1f21fe4b80 1 -- start start 2023-09-10T00:43:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.384+0000 7f1f21fe4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 0x558bbaaccf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.384+0000 7f1f21fe4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 0x558bbaacd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.384+0000 7f1f21fe4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad81a0 0x558bbaad70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.384+0000 7f1f21fe4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbaace310 con 0x558bbaad2300 2023-09-10T00:43:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.384+0000 7f1f21fe4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbaad6cc0 con 0x558bbaad2cc0 2023-09-10T00:43:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.384+0000 7f1f21fe4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbaae89b0 con 0x558bbaad81a0 2023-09-10T00:43:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.384+0000 7f1f0f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 0x558bbaaccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.384+0000 7f1f0effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 0x558bbaacd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.384+0000 7f1f0f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 0x558bbaaccf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50402/0 (socket says 172.21.15.44:50402) 2023-09-10T00:43:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.384+0000 7f1f0f7fe700 1 -- 172.21.15.44:0/3041903625 learned_addr learned my addr 172.21.15.44:0/3041903625 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.384+0000 7f1f0ffff700 1 --2- 172.21.15.44:0/3041903625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad81a0 0x558bbaad70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.384+0000 7f1f0effd700 1 -- 172.21.15.44:0/3041903625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad81a0 msgr2=0x558bbaad70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.384+0000 7f1f0effd700 1 --2- 172.21.15.44:0/3041903625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad81a0 0x558bbaad70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.384+0000 7f1f0effd700 1 -- 172.21.15.44:0/3041903625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 msgr2=0x558bbaaccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.384+0000 7f1f0effd700 1 --2- 172.21.15.44:0/3041903625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 0x558bbaaccf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.384+0000 7f1f0effd700 1 -- 172.21.15.44:0/3041903625 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bba839c00 con 0x558bbaad2cc0 2023-09-10T00:43:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.385+0000 7f1f0f7fe700 1 --2- 172.21.15.44:0/3041903625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 0x558bbaaccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.385+0000 7f1f0effd700 1 --2- 172.21.15.44:0/3041903625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 0x558bbaacd460 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1f04011770 tx=0x7f1f04014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e44cf587602729 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.385+0000 7f1ef67fc700 1 -- 172.21.15.44:0/3041903625 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04012d90 con 0x558bbaad2cc0 2023-09-10T00:43:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.385+0000 7f1ef67fc700 1 -- 172.21.15.44:0/3041903625 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f04016070 con 0x558bbaad2cc0 2023-09-10T00:43:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.385+0000 7f1ef67fc700 1 -- 172.21.15.44:0/3041903625 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04003dd0 con 0x558bbaad2cc0 2023-09-10T00:43:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.386+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/3041903625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 msgr2=0x558bbaacd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.386+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/3041903625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 0x558bbaacd460 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1f04011770 tx=0x7f1f04014000 comp rx=0 tx=0).stop 2023-09-10T00:43:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.386+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/3041903625 shutdown_connections 2023-09-10T00:43:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.386+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/3041903625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad81a0 0x558bbaad70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.386+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/3041903625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad2300 0x558bbaaccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.386+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/3041903625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 0x558bbaacd460 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.386+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/3041903625 >> 172.21.15.44:0/3041903625 conn(0x558bbaa47030 msgr2=0x558bbaadc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.386+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/3041903625 shutdown_connections 2023-09-10T00:43:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.386+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/3041903625 wait complete. 2023-09-10T00:43:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.387+0000 7f1f21fe4b80 1 -- start start 2023-09-10T00:43:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.387+0000 7f1f21fe4b80 1 Processor -- start 2023-09-10T00:43:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.387+0000 7f1f21fe4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad2300 0x558bbaae2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.387+0000 7f1f21fe4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 0x558bbaae3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.387+0000 7f1f21fe4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad81a0 0x558bbaae79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.387+0000 7f1f21fe4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbaae7ee0 con 0x558bbaad81a0 2023-09-10T00:43:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.387+0000 7f1f21fe4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbaae8050 con 0x558bbaad2cc0 2023-09-10T00:43:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.387+0000 7f1f21fe4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbaae8210 con 0x558bbaad2300 2023-09-10T00:43:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.387+0000 7f1f21fe4b80 10 client.0 _ll_register_callbacks cb 0x7f1f080043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:43:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.387+0000 7f1f0ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad81a0 0x558bbaae79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.387+0000 7f1f0f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad2300 0x558bbaae2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.387+0000 7f1f0f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad2300 0x558bbaae2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57290/0 (socket says 172.21.15.44:57290) 2023-09-10T00:43:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.387+0000 7f1f0ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad81a0 0x558bbaae79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50412/0 (socket says 172.21.15.44:50412) 2023-09-10T00:43:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.387+0000 7f1f0f7fe700 1 -- 172.21.15.44:0/2406754933 learned_addr learned my addr 172.21.15.44:0/2406754933 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.388+0000 7f1f0effd700 1 --2- 172.21.15.44:0/2406754933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 0x558bbaae3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.388+0000 7f1f0f7fe700 1 -- 172.21.15.44:0/2406754933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 msgr2=0x558bbaae3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.388+0000 7f1f0f7fe700 1 --2- 172.21.15.44:0/2406754933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 0x558bbaae3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.388+0000 7f1f0f7fe700 1 -- 172.21.15.44:0/2406754933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad81a0 msgr2=0x558bbaae79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.388+0000 7f1f0f7fe700 1 --2- 172.21.15.44:0/2406754933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad81a0 0x558bbaae79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.388+0000 7f1f0f7fe700 1 -- 172.21.15.44:0/2406754933 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bba850d80 con 0x558bbaad2300 2023-09-10T00:43:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.388+0000 7f1f0ffff700 1 --2- 172.21.15.44:0/2406754933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad81a0 0x558bbaae79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.388+0000 7f1f0f7fe700 1 --2- 172.21.15.44:0/2406754933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad2300 0x558bbaae2ad0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1f000966b0 tx=0x7f1f00009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f82d1109ccc2db98 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.388+0000 7f1eeeffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:43:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.388+0000 7f1eeeffd700 1 -- 172.21.15.44:0/2406754933 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0009a410 con 0x558bbaad2300 2023-09-10T00:43:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.388+0000 7f1eeeffd700 1 -- 172.21.15.44:0/2406754933 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f1f000b2040 con 0x558bbaad2300 2023-09-10T00:43:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.388+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/2406754933 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f00098550 con 0x558bbaad2300 2023-09-10T00:43:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.388+0000 7f1eeeffd700 1 -- 172.21.15.44:0/2406754933 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f0009a570 con 0x558bbaad2300 2023-09-10T00:43:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.388+0000 7f1f21fe4b80 10 client.26086 fetch_fsmap learned FSMap version 35 2023-09-10T00:43:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.388+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/2406754933 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f1f04011bf0 con 0x558bbaad2300 2023-09-10T00:43:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.389+0000 7f1eeeffd700 1 -- 172.21.15.44:0/2406754933 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f0007f080 con 0x558bbaad2300 2023-09-10T00:43:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.389+0000 7f1eeeffd700 1 -- 172.21.15.44:0/2406754933 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f1f0009ed20 con 0x558bbaad2300 2023-09-10T00:43:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.389+0000 7f1f21fe4b80 10 client.26086 fetch_fsmap finished waiting for FSMap version 35 2023-09-10T00:43:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.389+0000 7f1f21fe4b80 10 client.26086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:43:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.389+0000 7f1f21fe4b80 20 client.26086 populate_metadata read hostname 'smithi044' 2023-09-10T00:43:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.389+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/2406754933 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558bbab45140 0x558bbab47560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.389+0000 7f1f21fe4b80 4 client.26086 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:43:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.389+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/2406754933 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558bbaa1d8c0 con 0x558bbab45140 2023-09-10T00:43:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.390+0000 7f1f0effd700 1 --2- 172.21.15.44:0/2406754933 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558bbab45140 0x558bbab47560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.390+0000 7f1eeeffd700 10 client.26086 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.390+0000 7f1f0effd700 1 --2- 172.21.15.44:0/2406754933 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558bbab45140 0x558bbab47560 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f46f0abd7ac04a4 server_cookie=5554c4538af8d8c3 in_seq=0 out_seq=0 2023-09-10T00:43:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.402+0000 7f1eeeffd700 1 -- 172.21.15.44:0/2406754933 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558bbaa1d8c0 con 0x558bbab45140 2023-09-10T00:43:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.402+0000 7f1eeeffd700 10 client.26086 handle_command_reply: tid=0 2023-09-10T00:43:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.402+0000 7f1f21fe4b80 1 client.26086 shutdown 2023-09-10T00:43:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.402+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/2406754933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad2300 msgr2=0x558bbaae2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.402+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/2406754933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad2300 0x558bbaae2ad0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1f000966b0 tx=0x7f1f00009310 comp rx=0 tx=0).stop 2023-09-10T00:43:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.402+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/2406754933 shutdown_connections 2023-09-10T00:43:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.402+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/2406754933 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558bbab45140 0x558bbab47560 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.402+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/2406754933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558bbaad81a0 0x558bbaae79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.402+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/2406754933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558bbaad2300 0x558bbaae2ad0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.402+0000 7f1f21fe4b80 1 --2- 172.21.15.44:0/2406754933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bbaad2cc0 0x558bbaae3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.402+0000 7f1eeeffd700 0 client.26086 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.402+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/2406754933 >> 172.21.15.44:0/2406754933 conn(0x558bbaa47030 msgr2=0x558bbaad70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.402+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/2406754933 shutdown_connections 2023-09-10T00:43:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.402+0000 7f1f21fe4b80 1 -- 172.21.15.44:0/2406754933 wait complete. 2023-09-10T00:43:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.402+0000 7f1f21fe4b80 20 client.26086 trim_cache size 0 max 16384 2023-09-10T00:43:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:43:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:43:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:43:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:43:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.403+0000 7f1f15e1e700 1 -- 172.21.15.44:0/685718689 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1ef80a07b0 msgr2=0x7f1ef80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.403+0000 7f1f15e1e700 1 --2- 172.21.15.44:0/685718689 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1ef80a07b0 0x7f1ef80a2c70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f10068ad0 tx=0x7f1f04009260 comp rx=0 tx=0).stop 2023-09-10T00:43:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.403+0000 7f1f15e1e700 1 -- 172.21.15.44:0/685718689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f100ff760 msgr2=0x7f1f10197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.403+0000 7f1f15e1e700 1 --2- 172.21.15.44:0/685718689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f100ff760 0x7f1f10197690 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1f00006010 tx=0x7f1f0000f670 comp rx=0 tx=0).stop 2023-09-10T00:43:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.403+0000 7f1f15e1e700 1 -- 172.21.15.44:0/685718689 shutdown_connections 2023-09-10T00:43:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.403+0000 7f1f15e1e700 1 --2- 172.21.15.44:0/685718689 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1ef80a07b0 0x7f1ef80a2c70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.403+0000 7f1f15e1e700 1 --2- 172.21.15.44:0/685718689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f101055b0 0x7f1f101a2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.403+0000 7f1f15e1e700 1 --2- 172.21.15.44:0/685718689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f100ff760 0x7f1f10197690 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.403+0000 7f1f15e1e700 1 --2- 172.21.15.44:0/685718689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f10100120 0x7f1f10197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.403+0000 7f1f15e1e700 1 -- 172.21.15.44:0/685718689 >> 172.21.15.44:0/685718689 conn(0x7f1f10074f30 msgr2=0x7f1f100fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.403+0000 7f1f15e1e700 1 -- 172.21.15.44:0/685718689 shutdown_connections 2023-09-10T00:43:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:33.403+0000 7f1f15e1e700 1 -- 172.21.15.44:0/685718689 wait complete. 2023-09-10T00:43:33.953 INFO:teuthology.orchestra.run.smithi044.stderr:14700000 of 50000000 tuples (29%) done (elapsed 153.59 s, remaining 368.82 s) 2023-09-10T00:43:34.826 INFO:teuthology.orchestra.run.smithi044.stderr:14800000 of 50000000 tuples (29%) done (elapsed 154.46 s, remaining 367.37 s) 2023-09-10T00:43:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:34 smithi131 ceph-mon[103713]: pgmap v90: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 34 MiB/s wr, 48 op/s 2023-09-10T00:43:35.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:34 smithi044 ceph-mon[98676]: pgmap v90: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 34 MiB/s wr, 48 op/s 2023-09-10T00:43:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:34 smithi099 ceph-mon[102692]: pgmap v90: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 34 MiB/s wr, 48 op/s 2023-09-10T00:43:35.735 INFO:teuthology.orchestra.run.smithi044.stderr:14900000 of 50000000 tuples (29%) done (elapsed 155.37 s, remaining 366.00 s) 2023-09-10T00:43:36.562 INFO:teuthology.orchestra.run.smithi044.stderr:15000000 of 50000000 tuples (30%) done (elapsed 156.20 s, remaining 364.46 s) 2023-09-10T00:43:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:36 smithi131 ceph-mon[103713]: pgmap v91: 129 pgs: 129 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 51 MiB/s wr, 63 op/s 2023-09-10T00:43:37.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:36 smithi044 ceph-mon[98676]: pgmap v91: 129 pgs: 129 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 51 MiB/s wr, 63 op/s 2023-09-10T00:43:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:36 smithi099 ceph-mon[102692]: pgmap v91: 129 pgs: 129 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 51 MiB/s wr, 63 op/s 2023-09-10T00:43:37.126 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:43:37.271 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.271+0000 7fa18819d700 1 -- 172.21.15.44:0/3228041206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1800ff440 msgr2=0x7fa1800ff8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:37.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.271+0000 7fa18819d700 1 --2- 172.21.15.44:0/3228041206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1800ff440 0x7fa1800ff8c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa170009990 tx=0x7fa170009ca0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.271+0000 7fa18819d700 1 -- 172.21.15.44:0/3228041206 shutdown_connections 2023-09-10T00:43:37.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.271+0000 7fa18819d700 1 --2- 172.21.15.44:0/3228041206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1800ffe00 0x7fa18010c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.272 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.271+0000 7fa18819d700 1 --2- 172.21.15.44:0/3228041206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1801054e0 0x7fa1801058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.271+0000 7fa18819d700 1 --2- 172.21.15.44:0/3228041206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1800ff440 0x7fa1800ff8c0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.271+0000 7fa18819d700 1 -- 172.21.15.44:0/3228041206 >> 172.21.15.44:0/3228041206 conn(0x7fa1800faf50 msgr2=0x7fa1800fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:37.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.271+0000 7fa18819d700 1 -- 172.21.15.44:0/3228041206 shutdown_connections 2023-09-10T00:43:37.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.271+0000 7fa18819d700 1 -- 172.21.15.44:0/3228041206 wait complete. 2023-09-10T00:43:37.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.272+0000 7fa18819d700 1 Processor -- start 2023-09-10T00:43:37.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.272+0000 7fa18819d700 1 -- start start 2023-09-10T00:43:37.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.272+0000 7fa18819d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1800ff440 0x7fa180197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.272+0000 7fa18819d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1800ffe00 0x7fa1801979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.272+0000 7fa18819d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1801054e0 0x7fa1801a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.272+0000 7fa18819d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801982f0 con 0x7fa1800ff440 2023-09-10T00:43:37.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.272+0000 7fa18819d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801a2a30 con 0x7fa1800ffe00 2023-09-10T00:43:37.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.272+0000 7fa18819d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801a2bf0 con 0x7fa1801054e0 2023-09-10T00:43:37.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.272+0000 7fa185f39700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1800ff440 0x7fa180197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.272+0000 7fa18673a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1801054e0 0x7fa1801a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.272+0000 7fa18673a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1801054e0 0x7fa1801a24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57306/0 (socket says 172.21.15.44:57306) 2023-09-10T00:43:37.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.272+0000 7fa18673a700 1 -- 172.21.15.44:0/2815628416 learned_addr learned my addr 172.21.15.44:0/2815628416 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:37.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.272+0000 7fa185738700 1 --2- 172.21.15.44:0/2815628416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1800ffe00 0x7fa1801979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.281 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.273+0000 7fa185f39700 1 -- 172.21.15.44:0/2815628416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1801054e0 msgr2=0x7fa1801a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:37.281 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.273+0000 7fa185f39700 1 --2- 172.21.15.44:0/2815628416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1801054e0 0x7fa1801a24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.281 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.273+0000 7fa185f39700 1 -- 172.21.15.44:0/2815628416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1800ffe00 msgr2=0x7fa1801979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:37.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.273+0000 7fa185f39700 1 --2- 172.21.15.44:0/2815628416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1800ffe00 0x7fa1801979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.273+0000 7fa185f39700 1 -- 172.21.15.44:0/2815628416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1700095f0 con 0x7fa1800ff440 2023-09-10T00:43:37.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.273+0000 7fa185738700 1 --2- 172.21.15.44:0/2815628416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1800ffe00 0x7fa1801979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:43:37.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.273+0000 7fa18673a700 1 --2- 172.21.15.44:0/2815628416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1801054e0 0x7fa1801a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:37.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.273+0000 7fa185f39700 1 --2- 172.21.15.44:0/2815628416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1800ff440 0x7fa180197470 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00e9f0 tx=0x7fa17c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:37.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.273+0000 7fa176ffd700 1 -- 172.21.15.44:0/2815628416 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c00eee0 con 0x7fa1800ff440 2023-09-10T00:43:37.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.273+0000 7fa176ffd700 1 -- 172.21.15.44:0/2815628416 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa17c00cbd0 con 0x7fa1800ff440 2023-09-10T00:43:37.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.273+0000 7fa18819d700 1 -- 172.21.15.44:0/2815628416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1801a2ed0 con 0x7fa1800ff440 2023-09-10T00:43:37.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.273+0000 7fa176ffd700 1 -- 172.21.15.44:0/2815628416 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c018680 con 0x7fa1800ff440 2023-09-10T00:43:37.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.273+0000 7fa18819d700 1 -- 172.21.15.44:0/2815628416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1801a3350 con 0x7fa1800ff440 2023-09-10T00:43:37.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.276+0000 7fa176ffd700 1 -- 172.21.15.44:0/2815628416 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa17c00cd40 con 0x7fa1800ff440 2023-09-10T00:43:37.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.276+0000 7fa176ffd700 1 --2- 172.21.15.44:0/2815628416 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa16c0a07b0 0x7fa16c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.277+0000 7fa176ffd700 1 -- 172.21.15.44:0/2815628416 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa17c014070 con 0x7fa1800ff440 2023-09-10T00:43:37.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.277+0000 7fa185738700 1 --2- 172.21.15.44:0/2815628416 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa16c0a07b0 0x7fa16c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.277+0000 7fa194363b80 1 Processor -- start 2023-09-10T00:43:37.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.277+0000 7fa185738700 1 --2- 172.21.15.44:0/2815628416 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa16c0a07b0 0x7fa16c0a2c70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa180068030 tx=0x7fa170005a90 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:37.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.277+0000 7fa194363b80 1 -- start start 2023-09-10T00:43:37.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa194363b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723d710 0x561d3723daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa194363b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723e030 0x561d3724bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa194363b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3724c250 0x561d3724e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa194363b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d37196730 con 0x561d3723d710 2023-09-10T00:43:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa194363b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d37194050 con 0x561d3723e030 2023-09-10T00:43:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa194363b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d3718f940 con 0x561d3724c250 2023-09-10T00:43:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa185f39700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723d710 0x561d3723daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa185f39700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723d710 0x561d3723daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50430/0 (socket says 172.21.15.44:50430) 2023-09-10T00:43:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa185f39700 1 -- 172.21.15.44:0/4097910184 learned_addr learned my addr 172.21.15.44:0/4097910184 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa185738700 1 --2- 172.21.15.44:0/4097910184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723e030 0x561d3724bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.294 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa18673a700 1 --2- 172.21.15.44:0/4097910184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3724c250 0x561d3724e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.294 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa185f39700 1 -- 172.21.15.44:0/4097910184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3724c250 msgr2=0x561d3724e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:37.294 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa185f39700 1 --2- 172.21.15.44:0/4097910184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3724c250 0x561d3724e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.294 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa185f39700 1 -- 172.21.15.44:0/4097910184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723e030 msgr2=0x561d3724bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:37.294 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa185f39700 1 --2- 172.21.15.44:0/4097910184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723e030 0x561d3724bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.295 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa185f39700 1 -- 172.21.15.44:0/4097910184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d36fbbde0 con 0x561d3723d710 2023-09-10T00:43:37.295 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.278+0000 7fa185f39700 1 --2- 172.21.15.44:0/4097910184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723d710 0x561d3723daf0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0995c0 tx=0x7fa17c099010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd2e34b6947b358 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:37.295 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.279+0000 7fa16bfff700 1 -- 172.21.15.44:0/4097910184 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c09d460 con 0x561d3723d710 2023-09-10T00:43:37.295 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.279+0000 7fa16bfff700 1 -- 172.21.15.44:0/4097910184 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa17c0a16a0 con 0x561d3723d710 2023-09-10T00:43:37.295 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.279+0000 7fa16bfff700 1 -- 172.21.15.44:0/4097910184 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0840b0 con 0x561d3723d710 2023-09-10T00:43:37.296 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.279+0000 7fa194363b80 1 -- 172.21.15.44:0/4097910184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723d710 msgr2=0x561d3723daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:37.296 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.279+0000 7fa194363b80 1 --2- 172.21.15.44:0/4097910184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723d710 0x561d3723daf0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0995c0 tx=0x7fa17c099010 comp rx=0 tx=0).stop 2023-09-10T00:43:37.296 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.279+0000 7fa194363b80 1 -- 172.21.15.44:0/4097910184 shutdown_connections 2023-09-10T00:43:37.296 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.279+0000 7fa194363b80 1 --2- 172.21.15.44:0/4097910184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3724c250 0x561d3724e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.296 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.279+0000 7fa194363b80 1 --2- 172.21.15.44:0/4097910184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723d710 0x561d3723daf0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.296 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.279+0000 7fa194363b80 1 --2- 172.21.15.44:0/4097910184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723e030 0x561d3724bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.280+0000 7fa194363b80 1 -- 172.21.15.44:0/4097910184 >> 172.21.15.44:0/4097910184 conn(0x561d371b2090 msgr2=0x561d37238bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.280+0000 7fa194363b80 1 -- 172.21.15.44:0/4097910184 shutdown_connections 2023-09-10T00:43:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.280+0000 7fa194363b80 1 -- 172.21.15.44:0/4097910184 wait complete. 2023-09-10T00:43:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.280+0000 7fa194363b80 1 -- start start 2023-09-10T00:43:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.280+0000 7fa194363b80 1 Processor -- start 2023-09-10T00:43:37.298 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa194363b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3723d710 0x561d3724bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.298 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa194363b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723e030 0x561d37248f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.298 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa194363b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3724c250 0x561d37249650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.298 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa194363b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d3712db70 con 0x561d3723e030 2023-09-10T00:43:37.298 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa194363b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d3712d870 con 0x561d3724c250 2023-09-10T00:43:37.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa194363b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d37011920 con 0x561d3723d710 2023-09-10T00:43:37.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa194363b80 10 client.0 _ll_register_callbacks cb 0x561d37077a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:43:37.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa185f39700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3723d710 0x561d3724bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa185738700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723e030 0x561d37248f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa185f39700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3723d710 0x561d3724bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57318/0 (socket says 172.21.15.44:57318) 2023-09-10T00:43:37.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa185f39700 1 -- 172.21.15.44:0/1231685091 learned_addr learned my addr 172.21.15.44:0/1231685091 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:37.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa18673a700 1 --2- 172.21.15.44:0/1231685091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3724c250 0x561d37249650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa185f39700 1 -- 172.21.15.44:0/1231685091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3724c250 msgr2=0x561d37249650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:37.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa185f39700 1 --2- 172.21.15.44:0/1231685091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3724c250 0x561d37249650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa185f39700 1 -- 172.21.15.44:0/1231685091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723e030 msgr2=0x561d37248f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:37.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa185f39700 1 --2- 172.21.15.44:0/1231685091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723e030 0x561d37248f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa185f39700 1 -- 172.21.15.44:0/1231685091 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1801a3350 con 0x561d3723d710 2023-09-10T00:43:37.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa185738700 1 --2- 172.21.15.44:0/1231685091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723e030 0x561d37248f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:37.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa185f39700 1 --2- 172.21.15.44:0/1231685091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3723d710 0x561d3724bd00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa17c099100 tx=0x7fa17c0cc3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=793a95b9858ef8b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:37.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa1697fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:43:37.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa1697fa700 1 -- 172.21.15.44:0/1231685091 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c09d460 con 0x561d3723d710 2023-09-10T00:43:37.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.281+0000 7fa1697fa700 1 -- 172.21.15.44:0/1231685091 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa17c098b20 con 0x561d3723d710 2023-09-10T00:43:37.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.282+0000 7fa194363b80 1 -- 172.21.15.44:0/1231685091 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d370e74f0 con 0x561d3723d710 2023-09-10T00:43:37.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.282+0000 7fa1697fa700 1 -- 172.21.15.44:0/1231685091 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa17c098ce0 con 0x561d3723d710 2023-09-10T00:43:37.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.282+0000 7fa194363b80 10 client.26095 fetch_fsmap learned FSMap version 35 2023-09-10T00:43:37.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.282+0000 7fa194363b80 1 -- 172.21.15.44:0/1231685091 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x561d36fa4c60 con 0x561d3723d710 2023-09-10T00:43:37.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.282+0000 7fa1697fa700 1 -- 172.21.15.44:0/1231685091 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561d370e74f0 con 0x561d3723d710 2023-09-10T00:43:37.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.282+0000 7fa1697fa700 1 -- 172.21.15.44:0/1231685091 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fa17c0a1210 con 0x561d3723d710 2023-09-10T00:43:37.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.282+0000 7fa194363b80 10 client.26095 fetch_fsmap finished waiting for FSMap version 35 2023-09-10T00:43:37.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.282+0000 7fa194363b80 10 client.26095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:43:37.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.282+0000 7fa194363b80 20 client.26095 populate_metadata read hostname 'smithi044' 2023-09-10T00:43:37.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.282+0000 7fa194363b80 1 --2- 172.21.15.44:0/1231685091 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561d3725b8e0 0x561d3725dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.282+0000 7fa194363b80 4 client.26095 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:43:37.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.282+0000 7fa194363b80 1 -- 172.21.15.44:0/1231685091 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d371a48a0 con 0x561d3725b8e0 2023-09-10T00:43:37.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.282+0000 7fa185738700 1 --2- 172.21.15.44:0/1231685091 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561d3725b8e0 0x561d3725dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.283+0000 7fa1697fa700 10 client.26095 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:37.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.283+0000 7fa185738700 1 --2- 172.21.15.44:0/1231685091 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561d3725b8e0 0x561d3725dcd0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a39fc1dc0c79de server_cookie=325c6423d5f4155d in_seq=0 out_seq=0 2023-09-10T00:43:37.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.284+0000 7fa1697fa700 1 -- 172.21.15.44:0/1231685091 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561d371a48a0 con 0x561d3725b8e0 2023-09-10T00:43:37.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.284+0000 7fa1697fa700 10 client.26095 handle_command_reply: tid=0 2023-09-10T00:43:37.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.284+0000 7fa194363b80 1 client.26095 shutdown 2023-09-10T00:43:37.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.284+0000 7fa194363b80 1 -- 172.21.15.44:0/1231685091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3723d710 msgr2=0x561d3724bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:37.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.284+0000 7fa194363b80 1 --2- 172.21.15.44:0/1231685091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3723d710 0x561d3724bd00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa17c099100 tx=0x7fa17c0cc3e0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.284+0000 7fa194363b80 1 -- 172.21.15.44:0/1231685091 shutdown_connections 2023-09-10T00:43:37.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.284+0000 7fa194363b80 1 --2- 172.21.15.44:0/1231685091 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561d3725b8e0 0x561d3725dcd0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.284+0000 7fa194363b80 1 --2- 172.21.15.44:0/1231685091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3724c250 0x561d37249650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.284+0000 7fa1697fa700 0 client.26095 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:37.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.284+0000 7fa194363b80 1 --2- 172.21.15.44:0/1231685091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3723d710 0x561d3724bd00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.284+0000 7fa194363b80 1 --2- 172.21.15.44:0/1231685091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723e030 0x561d37248f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.284+0000 7fa194363b80 1 -- 172.21.15.44:0/1231685091 >> 172.21.15.44:0/1231685091 conn(0x561d371b2090 msgr2=0x561d372387e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:37.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.284+0000 7fa194363b80 1 -- 172.21.15.44:0/1231685091 shutdown_connections 2023-09-10T00:43:37.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.284+0000 7fa194363b80 1 -- 172.21.15.44:0/1231685091 wait complete. 2023-09-10T00:43:37.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.284+0000 7fa194363b80 20 client.26095 trim_cache size 0 max 16384 2023-09-10T00:43:37.309 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.308+0000 7fa194363b80 1 Processor -- start 2023-09-10T00:43:37.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.308+0000 7fa194363b80 1 -- start start 2023-09-10T00:43:37.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.308+0000 7fa194363b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723d710 0x561d3723daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.308+0000 7fa194363b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723e030 0x561d37252da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.308+0000 7fa194363b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3724c250 0x561d37244210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.308+0000 7fa194363b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d372532e0 con 0x561d3723e030 2023-09-10T00:43:37.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.308+0000 7fa194363b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d37230400 con 0x561d3723d710 2023-09-10T00:43:37.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.308+0000 7fa194363b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d372586d0 con 0x561d3724c250 2023-09-10T00:43:37.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.308+0000 7fa185738700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723e030 0x561d37252da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.308+0000 7fa185738700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723e030 0x561d37252da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50444/0 (socket says 172.21.15.44:50444) 2023-09-10T00:43:37.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.308+0000 7fa185f39700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723d710 0x561d3723daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.308+0000 7fa185738700 1 -- 172.21.15.44:0/536542802 learned_addr learned my addr 172.21.15.44:0/536542802 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:37.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.308+0000 7fa18673a700 1 --2- 172.21.15.44:0/536542802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3724c250 0x561d37244210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.309+0000 7fa185f39700 1 -- 172.21.15.44:0/536542802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3724c250 msgr2=0x561d37244210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:37.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.309+0000 7fa185f39700 1 --2- 172.21.15.44:0/536542802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3724c250 0x561d37244210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.309+0000 7fa185f39700 1 -- 172.21.15.44:0/536542802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723e030 msgr2=0x561d37252da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:37.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.309+0000 7fa185f39700 1 --2- 172.21.15.44:0/536542802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723e030 0x561d37252da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.309+0000 7fa185f39700 1 -- 172.21.15.44:0/536542802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d36fe2250 con 0x561d3723d710 2023-09-10T00:43:37.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.309+0000 7fa185738700 1 --2- 172.21.15.44:0/536542802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723e030 0x561d37252da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:37.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.309+0000 7fa185f39700 1 --2- 172.21.15.44:0/536542802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723d710 0x561d3723daf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00c5b0 tx=0x7fa17c0843c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63a61952ccfbc1c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:37.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.309+0000 7fa174ff9700 1 -- 172.21.15.44:0/536542802 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c084160 con 0x561d3723d710 2023-09-10T00:43:37.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.309+0000 7fa174ff9700 1 -- 172.21.15.44:0/536542802 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa17c095970 con 0x561d3723d710 2023-09-10T00:43:37.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.309+0000 7fa174ff9700 1 -- 172.21.15.44:0/536542802 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c098b20 con 0x561d3723d710 2023-09-10T00:43:37.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.310+0000 7fa194363b80 1 -- 172.21.15.44:0/536542802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723d710 msgr2=0x561d3723daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:37.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.310+0000 7fa194363b80 1 --2- 172.21.15.44:0/536542802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723d710 0x561d3723daf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00c5b0 tx=0x7fa17c0843c0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.310+0000 7fa194363b80 1 -- 172.21.15.44:0/536542802 shutdown_connections 2023-09-10T00:43:37.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.310+0000 7fa194363b80 1 --2- 172.21.15.44:0/536542802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3724c250 0x561d37244210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.310+0000 7fa194363b80 1 --2- 172.21.15.44:0/536542802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723d710 0x561d3723daf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.310+0000 7fa194363b80 1 --2- 172.21.15.44:0/536542802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723e030 0x561d37252da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.310+0000 7fa194363b80 1 -- 172.21.15.44:0/536542802 >> 172.21.15.44:0/536542802 conn(0x561d371b2090 msgr2=0x561d372574f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:37.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.310+0000 7fa194363b80 1 -- 172.21.15.44:0/536542802 shutdown_connections 2023-09-10T00:43:37.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.310+0000 7fa194363b80 1 -- 172.21.15.44:0/536542802 wait complete. 2023-09-10T00:43:37.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.310+0000 7fa194363b80 1 -- start start 2023-09-10T00:43:37.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.310+0000 7fa194363b80 1 Processor -- start 2023-09-10T00:43:37.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa194363b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723d710 0x561d372a3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa194363b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723e030 0x561d372a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa194363b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3724c250 0x561d372ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa194363b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d372a44c0 con 0x561d3723d710 2023-09-10T00:43:37.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa194363b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d372aec30 con 0x561d3723e030 2023-09-10T00:43:37.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa194363b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d372aed70 con 0x561d3724c250 2023-09-10T00:43:37.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa194363b80 10 client.0 _ll_register_callbacks cb 0x7fa170004620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:43:37.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa185f39700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723d710 0x561d372a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa185738700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723e030 0x561d372a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa185f39700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723d710 0x561d372a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50450/0 (socket says 172.21.15.44:50450) 2023-09-10T00:43:37.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa185f39700 1 -- 172.21.15.44:0/521847085 learned_addr learned my addr 172.21.15.44:0/521847085 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:37.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa18673a700 1 --2- 172.21.15.44:0/521847085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3724c250 0x561d372ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa185f39700 1 -- 172.21.15.44:0/521847085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3724c250 msgr2=0x561d372ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:37.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa185f39700 1 --2- 172.21.15.44:0/521847085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3724c250 0x561d372ae6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa185f39700 1 -- 172.21.15.44:0/521847085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723e030 msgr2=0x561d372a3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:37.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa185f39700 1 --2- 172.21.15.44:0/521847085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723e030 0x561d372a3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa185f39700 1 -- 172.21.15.44:0/521847085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18010cc10 con 0x561d3723d710 2023-09-10T00:43:37.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa18673a700 1 --2- 172.21.15.44:0/521847085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3724c250 0x561d372ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:37.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.311+0000 7fa185738700 1 --2- 172.21.15.44:0/521847085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723e030 0x561d372a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:37.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.312+0000 7fa185f39700 1 --2- 172.21.15.44:0/521847085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723d710 0x561d372a3670 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa17c099330 tx=0x7fa17c0a1440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3993a684b2e4efce server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:37.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.312+0000 7fa1697fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:43:37.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.312+0000 7fa1697fa700 1 -- 172.21.15.44:0/521847085 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c084160 con 0x561d3723d710 2023-09-10T00:43:37.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.312+0000 7fa1697fa700 1 -- 172.21.15.44:0/521847085 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa17c09d460 con 0x561d3723d710 2023-09-10T00:43:37.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.312+0000 7fa194363b80 1 -- 172.21.15.44:0/521847085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa17c0a16a0 con 0x561d3723d710 2023-09-10T00:43:37.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.312+0000 7fa1697fa700 1 -- 172.21.15.44:0/521847085 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa17c098d30 con 0x561d3723d710 2023-09-10T00:43:37.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.312+0000 7fa194363b80 10 client.16494 fetch_fsmap learned FSMap version 35 2023-09-10T00:43:37.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.312+0000 7fa194363b80 1 -- 172.21.15.44:0/521847085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x561d36fa4c60 con 0x561d3723d710 2023-09-10T00:43:37.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.312+0000 7fa1697fa700 1 -- 172.21.15.44:0/521847085 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa17c0a16a0 con 0x561d3723d710 2023-09-10T00:43:37.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.312+0000 7fa1697fa700 1 -- 172.21.15.44:0/521847085 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fa17c09e1a0 con 0x561d3723d710 2023-09-10T00:43:37.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.312+0000 7fa194363b80 10 client.16494 fetch_fsmap finished waiting for FSMap version 35 2023-09-10T00:43:37.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.312+0000 7fa194363b80 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:43:37.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.312+0000 7fa194363b80 20 client.16494 populate_metadata read hostname 'smithi044' 2023-09-10T00:43:37.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.312+0000 7fa194363b80 1 --2- 172.21.15.44:0/521847085 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561d372aff90 0x561d372b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:37.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.312+0000 7fa194363b80 4 client.16494 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:43:37.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.312+0000 7fa194363b80 1 -- 172.21.15.44:0/521847085 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561d37188920 con 0x561d372aff90 2023-09-10T00:43:37.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.313+0000 7fa185738700 1 --2- 172.21.15.44:0/521847085 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561d372aff90 0x561d372b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:37.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.313+0000 7fa1697fa700 10 client.16494 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:37.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.313+0000 7fa185738700 1 --2- 172.21.15.44:0/521847085 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561d372aff90 0x561d372b23b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ed7ad2fc7f8012a server_cookie=a01a4e8cbdcf4b87 in_seq=0 out_seq=0 2023-09-10T00:43:37.329 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.313+0000 7fa1697fa700 1 -- 172.21.15.44:0/521847085 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561d37188920 con 0x561d372aff90 2023-09-10T00:43:37.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.313+0000 7fa1697fa700 10 client.16494 handle_command_reply: tid=0 2023-09-10T00:43:37.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.313+0000 7fa194363b80 1 client.16494 shutdown 2023-09-10T00:43:37.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.313+0000 7fa194363b80 1 -- 172.21.15.44:0/521847085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723d710 msgr2=0x561d372a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:37.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.313+0000 7fa194363b80 1 --2- 172.21.15.44:0/521847085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723d710 0x561d372a3670 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa17c099330 tx=0x7fa17c0a1440 comp rx=0 tx=0).stop 2023-09-10T00:43:37.330 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.313+0000 7fa194363b80 1 -- 172.21.15.44:0/521847085 shutdown_connections 2023-09-10T00:43:37.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.313+0000 7fa194363b80 1 --2- 172.21.15.44:0/521847085 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561d372aff90 0x561d372b23b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.313+0000 7fa194363b80 1 --2- 172.21.15.44:0/521847085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561d3724c250 0x561d372ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.313+0000 7fa194363b80 1 --2- 172.21.15.44:0/521847085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d3723d710 0x561d372a3670 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.313+0000 7fa1697fa700 0 client.16494 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:37.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.313+0000 7fa194363b80 1 --2- 172.21.15.44:0/521847085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d3723e030 0x561d372a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.331 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.313+0000 7fa194363b80 1 -- 172.21.15.44:0/521847085 >> 172.21.15.44:0/521847085 conn(0x561d371b2090 msgr2=0x561d37244210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:37.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.314+0000 7fa194363b80 1 -- 172.21.15.44:0/521847085 shutdown_connections 2023-09-10T00:43:37.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.314+0000 7fa194363b80 1 -- 172.21.15.44:0/521847085 wait complete. 2023-09-10T00:43:37.332 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.314+0000 7fa194363b80 20 client.16494 trim_cache size 0 max 16384 2023-09-10T00:43:37.332 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:43:37.333 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:43:37.333 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:43:37.333 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:43:37.333 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:43:37.334 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:43:37.334 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:43:37.334 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:43:37.334 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:43:37.334 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:43:37.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.314+0000 7fa18819d700 1 -- 172.21.15.44:0/2815628416 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa16c0a07b0 msgr2=0x7fa16c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:37.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.314+0000 7fa18819d700 1 --2- 172.21.15.44:0/2815628416 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa16c0a07b0 0x7fa16c0a2c70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa180068030 tx=0x7fa170005a90 comp rx=0 tx=0).stop 2023-09-10T00:43:37.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.314+0000 7fa18819d700 1 -- 172.21.15.44:0/2815628416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1800ff440 msgr2=0x7fa180197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:37.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.314+0000 7fa18819d700 1 --2- 172.21.15.44:0/2815628416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1800ff440 0x7fa180197470 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00e9f0 tx=0x7fa17c00ed00 comp rx=0 tx=0).stop 2023-09-10T00:43:37.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.314+0000 7fa18819d700 1 -- 172.21.15.44:0/2815628416 shutdown_connections 2023-09-10T00:43:37.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.314+0000 7fa18819d700 1 --2- 172.21.15.44:0/2815628416 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa16c0a07b0 0x7fa16c0a2c70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.314+0000 7fa18819d700 1 --2- 172.21.15.44:0/2815628416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1801054e0 0x7fa1801a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.314+0000 7fa18819d700 1 --2- 172.21.15.44:0/2815628416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1800ff440 0x7fa180197470 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.314+0000 7fa18819d700 1 --2- 172.21.15.44:0/2815628416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1800ffe00 0x7fa1801979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:37.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.314+0000 7fa18819d700 1 -- 172.21.15.44:0/2815628416 >> 172.21.15.44:0/2815628416 conn(0x7fa1800faf50 msgr2=0x7fa180102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:37.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.314+0000 7fa18819d700 1 -- 172.21.15.44:0/2815628416 shutdown_connections 2023-09-10T00:43:37.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:43:37.314+0000 7fa18819d700 1 -- 172.21.15.44:0/2815628416 wait complete. 2023-09-10T00:43:37.338 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:43:37.461 INFO:teuthology.orchestra.run.smithi044.stderr:15100000 of 50000000 tuples (30%) done (elapsed 157.10 s, remaining 363.09 s) 2023-09-10T00:43:38.294 INFO:teuthology.orchestra.run.smithi044.stderr:15200000 of 50000000 tuples (30%) done (elapsed 157.93 s, remaining 361.57 s) 2023-09-10T00:43:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:38 smithi131 ceph-mon[103713]: pgmap v92: 129 pgs: 129 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 34 MiB/s wr, 40 op/s 2023-09-10T00:43:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:38 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:39.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:38 smithi044 ceph-mon[98676]: pgmap v92: 129 pgs: 129 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 34 MiB/s wr, 40 op/s 2023-09-10T00:43:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:38 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:38 smithi099 ceph-mon[102692]: pgmap v92: 129 pgs: 129 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 34 MiB/s wr, 40 op/s 2023-09-10T00:43:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:38 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:39.194 INFO:teuthology.orchestra.run.smithi044.stderr:15300000 of 50000000 tuples (30%) done (elapsed 158.83 s, remaining 360.22 s) 2023-09-10T00:43:40.023 INFO:teuthology.orchestra.run.smithi044.stderr:15400000 of 50000000 tuples (30%) done (elapsed 159.66 s, remaining 358.71 s) 2023-09-10T00:43:40.881 INFO:teuthology.orchestra.run.smithi044.stderr:15500000 of 50000000 tuples (31%) done (elapsed 160.52 s, remaining 357.28 s) 2023-09-10T00:43:41.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:40 smithi131 ceph-mon[103713]: pgmap v93: 129 pgs: 129 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 34 MiB/s wr, 40 op/s 2023-09-10T00:43:41.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:40 smithi044 ceph-mon[98676]: pgmap v93: 129 pgs: 129 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 34 MiB/s wr, 40 op/s 2023-09-10T00:43:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:40 smithi099 ceph-mon[102692]: pgmap v93: 129 pgs: 129 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 34 MiB/s wr, 40 op/s 2023-09-10T00:43:41.780 INFO:teuthology.orchestra.run.smithi044.stderr:15600000 of 50000000 tuples (31%) done (elapsed 161.41 s, remaining 355.94 s) 2023-09-10T00:43:42.608 INFO:teuthology.orchestra.run.smithi044.stderr:15700000 of 50000000 tuples (31%) done (elapsed 162.24 s, remaining 354.45 s) 2023-09-10T00:43:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:42 smithi131 ceph-mon[103713]: pgmap v94: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 51 MiB/s wr, 52 op/s 2023-09-10T00:43:43.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:42 smithi044 ceph-mon[98676]: pgmap v94: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 51 MiB/s wr, 52 op/s 2023-09-10T00:43:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:42 smithi099 ceph-mon[102692]: pgmap v94: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 51 MiB/s wr, 52 op/s 2023-09-10T00:43:43.476 INFO:teuthology.orchestra.run.smithi044.stderr:15800000 of 50000000 tuples (31%) done (elapsed 163.11 s, remaining 353.06 s) 2023-09-10T00:43:44.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:43 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:44.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:43 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:44.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:43 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:44.363 INFO:teuthology.orchestra.run.smithi044.stderr:15900000 of 50000000 tuples (31%) done (elapsed 164.00 s, remaining 351.72 s) 2023-09-10T00:43:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:44 smithi131 ceph-mon[103713]: pgmap v95: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 34 MiB/s wr, 27 op/s 2023-09-10T00:43:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:44 smithi044 ceph-mon[98676]: pgmap v95: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 34 MiB/s wr, 27 op/s 2023-09-10T00:43:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:44 smithi099 ceph-mon[102692]: pgmap v95: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 34 MiB/s wr, 27 op/s 2023-09-10T00:43:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.527+0000 7f0f4a2ee700 1 -- 172.21.15.44:0/2136383893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f441055b0 msgr2=0x7f0f44105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.527+0000 7f0f4a2ee700 1 --2- 172.21.15.44:0/2136383893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f441055b0 0x7f0f44105990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0f340099c0 tx=0x7f0f34009cd0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.527+0000 7f0f4a2ee700 1 -- 172.21.15.44:0/2136383893 shutdown_connections 2023-09-10T00:43:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.527+0000 7f0f4a2ee700 1 --2- 172.21.15.44:0/2136383893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f44100120 0x7f0f4410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.527+0000 7f0f4a2ee700 1 --2- 172.21.15.44:0/2136383893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f441055b0 0x7f0f44105990 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.527+0000 7f0f4a2ee700 1 --2- 172.21.15.44:0/2136383893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f440ff760 0x7f0f440ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.527+0000 7f0f4a2ee700 1 -- 172.21.15.44:0/2136383893 >> 172.21.15.44:0/2136383893 conn(0x7f0f44074f30 msgr2=0x7f0f44075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.527+0000 7f0f4a2ee700 1 -- 172.21.15.44:0/2136383893 shutdown_connections 2023-09-10T00:43:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.527+0000 7f0f4a2ee700 1 -- 172.21.15.44:0/2136383893 wait complete. 2023-09-10T00:43:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.528+0000 7f0f4a2ee700 1 Processor -- start 2023-09-10T00:43:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.528+0000 7f0f4a2ee700 1 -- start start 2023-09-10T00:43:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.528+0000 7f0f4a2ee700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f440ff760 0x7f0f44195430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.528+0000 7f0f4a2ee700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f44100120 0x7f0f44195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.528+0000 7f0f4a2ee700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f441055b0 0x7f0f44199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.528+0000 7f0f4a2ee700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4419a5b0 con 0x7f0f440ff760 2023-09-10T00:43:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.528+0000 7f0f4a2ee700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4419a720 con 0x7f0f441055b0 2023-09-10T00:43:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.528+0000 7f0f4a2ee700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4419a8e0 con 0x7f0f44100120 2023-09-10T00:43:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.528+0000 7f0f43fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f440ff760 0x7f0f44195430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.528+0000 7f0f43fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f440ff760 0x7f0f44195430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50464/0 (socket says 172.21.15.44:50464) 2023-09-10T00:43:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.528+0000 7f0f43fff700 1 -- 172.21.15.44:0/625634459 learned_addr learned my addr 172.21.15.44:0/625634459 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.528+0000 7f0f437fe700 1 --2- 172.21.15.44:0/625634459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f44100120 0x7f0f44195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.528+0000 7f0f4888b700 1 --2- 172.21.15.44:0/625634459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f441055b0 0x7f0f44199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.529+0000 7f0f43fff700 1 -- 172.21.15.44:0/625634459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f44100120 msgr2=0x7f0f44195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.529+0000 7f0f43fff700 1 --2- 172.21.15.44:0/625634459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f44100120 0x7f0f44195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.529+0000 7f0f43fff700 1 -- 172.21.15.44:0/625634459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f441055b0 msgr2=0x7f0f44199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.529+0000 7f0f43fff700 1 --2- 172.21.15.44:0/625634459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f441055b0 0x7f0f44199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.529+0000 7f0f43fff700 1 -- 172.21.15.44:0/625634459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f340095f0 con 0x7f0f440ff760 2023-09-10T00:43:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.529+0000 7f0f43fff700 1 --2- 172.21.15.44:0/625634459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f440ff760 0x7f0f44195430 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0f34005910 tx=0x7f0f34009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.529+0000 7f0f417fa700 1 -- 172.21.15.44:0/625634459 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3401d070 con 0x7f0f440ff760 2023-09-10T00:43:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.529+0000 7f0f4a2ee700 1 -- 172.21.15.44:0/625634459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4419ab60 con 0x7f0f440ff760 2023-09-10T00:43:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.529+0000 7f0f417fa700 1 -- 172.21.15.44:0/625634459 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f34022470 con 0x7f0f440ff760 2023-09-10T00:43:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.529+0000 7f0f417fa700 1 -- 172.21.15.44:0/625634459 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f34017650 con 0x7f0f440ff760 2023-09-10T00:43:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.529+0000 7f0f4a2ee700 1 -- 172.21.15.44:0/625634459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f441a2400 con 0x7f0f440ff760 2023-09-10T00:43:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.531+0000 7f0f417fa700 1 -- 172.21.15.44:0/625634459 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f0f340225e0 con 0x7f0f440ff760 2023-09-10T00:43:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.532+0000 7f0f417fa700 1 --2- 172.21.15.44:0/625634459 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f2c0a07b0 0x7f0f2c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.532+0000 7f0f417fa700 1 -- 172.21.15.44:0/625634459 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f0f340cd3f0 con 0x7f0f440ff760 2023-09-10T00:43:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.532+0000 7f0f437fe700 1 --2- 172.21.15.44:0/625634459 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f2c0a07b0 0x7f0f2c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.533+0000 7f0f437fe700 1 --2- 172.21.15.44:0/625634459 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f2c0a07b0 0x7f0f2c0a2c70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0f44196be0 tx=0x7f0f38009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.533+0000 7f0f564b4b80 1 Processor -- start 2023-09-10T00:43:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.533+0000 7f0f564b4b80 1 -- start start 2023-09-10T00:43:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.533+0000 7f0f564b4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831ab31a0 0x562831ab3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.533+0000 7f0f564b4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 0x562831aad780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.533+0000 7f0f564b4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 0x562831ab9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.533+0000 7f0f564b4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562831a066d0 con 0x562831aadcc0 2023-09-10T00:43:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.533+0000 7f0f564b4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562831a03ff0 con 0x562831ab31a0 2023-09-10T00:43:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.533+0000 7f0f564b4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628319ff8e0 con 0x562831aad300 2023-09-10T00:43:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.533+0000 7f0f43fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831ab31a0 0x562831ab3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.533+0000 7f0f437fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 0x562831aad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.533+0000 7f0f4888b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 0x562831ab9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.533+0000 7f0f43fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831ab31a0 0x562831ab3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52212/0 (socket says 172.21.15.44:52212) 2023-09-10T00:43:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.533+0000 7f0f437fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 0x562831aad780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57356/0 (socket says 172.21.15.44:57356) 2023-09-10T00:43:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.533+0000 7f0f43fff700 1 -- 172.21.15.44:0/1362349754 learned_addr learned my addr 172.21.15.44:0/1362349754 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.534+0000 7f0f43fff700 1 -- 172.21.15.44:0/1362349754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 msgr2=0x562831aad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.534+0000 7f0f43fff700 1 --2- 172.21.15.44:0/1362349754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 0x562831aad780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.534+0000 7f0f43fff700 1 -- 172.21.15.44:0/1362349754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 msgr2=0x562831ab9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.534+0000 7f0f43fff700 1 --2- 172.21.15.44:0/1362349754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 0x562831ab9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.534+0000 7f0f43fff700 1 -- 172.21.15.44:0/1362349754 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56283182bd80 con 0x562831ab31a0 2023-09-10T00:43:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.534+0000 7f0f4888b700 1 --2- 172.21.15.44:0/1362349754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 0x562831ab9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.534+0000 7f0f43fff700 1 --2- 172.21.15.44:0/1362349754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831ab31a0 0x562831ab3580 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0f34096fa0 tx=0x7f0f3409a070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47a639f1c5b4a466 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.534+0000 7f0f2a7fc700 1 -- 172.21.15.44:0/1362349754 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3409ac70 con 0x562831ab31a0 2023-09-10T00:43:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.534+0000 7f0f2a7fc700 1 -- 172.21.15.44:0/1362349754 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f340cae90 con 0x562831ab31a0 2023-09-10T00:43:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.534+0000 7f0f2a7fc700 1 -- 172.21.15.44:0/1362349754 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f340228c0 con 0x562831ab31a0 2023-09-10T00:43:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.535+0000 7f0f564b4b80 1 -- 172.21.15.44:0/1362349754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831ab31a0 msgr2=0x562831ab3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.535+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/1362349754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831ab31a0 0x562831ab3580 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0f34096fa0 tx=0x7f0f3409a070 comp rx=0 tx=0).stop 2023-09-10T00:43:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.535+0000 7f0f564b4b80 1 -- 172.21.15.44:0/1362349754 shutdown_connections 2023-09-10T00:43:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.535+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/1362349754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 0x562831ab9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.535+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/1362349754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831ab31a0 0x562831ab3580 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.535+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/1362349754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 0x562831aad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.535+0000 7f0f564b4b80 1 -- 172.21.15.44:0/1362349754 >> 172.21.15.44:0/1362349754 conn(0x562831a22030 msgr2=0x562831aa8560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.535+0000 7f0f564b4b80 1 -- 172.21.15.44:0/1362349754 shutdown_connections 2023-09-10T00:43:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.535+0000 7f0f564b4b80 1 -- 172.21.15.44:0/1362349754 wait complete. 2023-09-10T00:43:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.535+0000 7f0f564b4b80 1 -- start start 2023-09-10T00:43:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.535+0000 7f0f564b4b80 1 Processor -- start 2023-09-10T00:43:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f564b4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831aad300 0x562831ab6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f564b4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 0x562831ab72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f564b4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831ab31a0 0x562831ab77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f564b4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56283199db10 con 0x562831aadcc0 2023-09-10T00:43:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f564b4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56283199d810 con 0x562831aad300 2023-09-10T00:43:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f564b4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628318818c0 con 0x562831ab31a0 2023-09-10T00:43:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f564b4b80 10 client.0 _ll_register_callbacks cb 0x5628318e7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:43:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f437fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 0x562831ab72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f437fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 0x562831ab72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50480/0 (socket says 172.21.15.44:50480) 2023-09-10T00:43:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f4888b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831ab31a0 0x562831ab77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f437fe700 1 -- 172.21.15.44:0/211221494 learned_addr learned my addr 172.21.15.44:0/211221494 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f43fff700 1 --2- 172.21.15.44:0/211221494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831aad300 0x562831ab6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f437fe700 1 -- 172.21.15.44:0/211221494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831ab31a0 msgr2=0x562831ab77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f437fe700 1 --2- 172.21.15.44:0/211221494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831ab31a0 0x562831ab77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f437fe700 1 -- 172.21.15.44:0/211221494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831aad300 msgr2=0x562831ab6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f437fe700 1 --2- 172.21.15.44:0/211221494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831aad300 0x562831ab6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f437fe700 1 -- 172.21.15.44:0/211221494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3c011230 con 0x562831aadcc0 2023-09-10T00:43:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f437fe700 1 --2- 172.21.15.44:0/211221494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 0x562831ab72b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0f38004580 tx=0x7f0f380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5249699d5a457b08 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.536+0000 7f0f237fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:43:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.537+0000 7f0f237fe700 1 -- 172.21.15.44:0/211221494 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38022070 con 0x562831aadcc0 2023-09-10T00:43:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.537+0000 7f0f237fe700 1 -- 172.21.15.44:0/211221494 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f0f3801d410 con 0x562831aadcc0 2023-09-10T00:43:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.537+0000 7f0f564b4b80 1 -- 172.21.15.44:0/211221494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562831957490 con 0x562831aadcc0 2023-09-10T00:43:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.537+0000 7f0f237fe700 1 -- 172.21.15.44:0/211221494 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f380045b0 con 0x562831aadcc0 2023-09-10T00:43:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.537+0000 7f0f564b4b80 10 client.16506 fetch_fsmap learned FSMap version 37 2023-09-10T00:43:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.537+0000 7f0f564b4b80 1 -- 172.21.15.44:0/211221494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7f0f441a2400 con 0x562831aadcc0 2023-09-10T00:43:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.537+0000 7f0f237fe700 1 -- 172.21.15.44:0/211221494 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562831957490 con 0x562831aadcc0 2023-09-10T00:43:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.537+0000 7f0f237fe700 1 -- 172.21.15.44:0/211221494 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 37) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0f38027680 con 0x562831aadcc0 2023-09-10T00:43:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.537+0000 7f0f564b4b80 10 client.16506 fetch_fsmap finished waiting for FSMap version 37 2023-09-10T00:43:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.537+0000 7f0f564b4b80 10 client.16506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:43:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.537+0000 7f0f564b4b80 20 client.16506 populate_metadata read hostname 'smithi044' 2023-09-10T00:43:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.537+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/211221494 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562831acb650 0x562831acda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.537+0000 7f0f564b4b80 4 client.16506 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:43:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.537+0000 7f0f564b4b80 1 -- 172.21.15.44:0/211221494 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562831a14840 con 0x562831acb650 2023-09-10T00:43:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.537+0000 7f0f43fff700 1 --2- 172.21.15.44:0/211221494 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562831acb650 0x562831acda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.538+0000 7f0f43fff700 1 --2- 172.21.15.44:0/211221494 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562831acb650 0x562831acda70 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=949bd426329d6a9c server_cookie=5c79da3595e41b11 in_seq=0 out_seq=0 2023-09-10T00:43:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.538+0000 7f0f237fe700 10 client.16506 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.538+0000 7f0f237fe700 1 -- 172.21.15.44:0/211221494 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562831a14840 con 0x562831acb650 2023-09-10T00:43:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.538+0000 7f0f237fe700 10 client.16506 handle_command_reply: tid=0 2023-09-10T00:43:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.538+0000 7f0f564b4b80 1 client.16506 shutdown 2023-09-10T00:43:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.539+0000 7f0f564b4b80 1 -- 172.21.15.44:0/211221494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 msgr2=0x562831ab72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.539+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/211221494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 0x562831ab72b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0f38004580 tx=0x7f0f380047c0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.539+0000 7f0f564b4b80 1 -- 172.21.15.44:0/211221494 shutdown_connections 2023-09-10T00:43:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.539+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/211221494 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562831acb650 0x562831acda70 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.539+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/211221494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831ab31a0 0x562831ab77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.539+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/211221494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831aad300 0x562831ab6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.539+0000 7f0f237fe700 0 client.16506 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.539+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/211221494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 0x562831ab72b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.539+0000 7f0f564b4b80 1 -- 172.21.15.44:0/211221494 >> 172.21.15.44:0/211221494 conn(0x562831a22030 msgr2=0x562831aa77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.539+0000 7f0f564b4b80 1 -- 172.21.15.44:0/211221494 shutdown_connections 2023-09-10T00:43:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.539+0000 7f0f564b4b80 1 -- 172.21.15.44:0/211221494 wait complete. 2023-09-10T00:43:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.539+0000 7f0f564b4b80 20 client.16506 trim_cache size 0 max 16384 2023-09-10T00:43:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.554+0000 7f0f564b4b80 1 Processor -- start 2023-09-10T00:43:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.554+0000 7f0f564b4b80 1 -- start start 2023-09-10T00:43:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.554+0000 7f0f564b4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831a22030 0x562831ab7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.554+0000 7f0f564b4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 0x562831aaca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.554+0000 7f0f564b4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831aadcc0 0x562831ac6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.554+0000 7f0f564b4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562831a9fe00 con 0x562831a22030 2023-09-10T00:43:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.554+0000 7f0f564b4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562831aae580 con 0x562831aadcc0 2023-09-10T00:43:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.554+0000 7f0f564b4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562831aa4fc0 con 0x562831aad300 2023-09-10T00:43:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.554+0000 7f0f43fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831a22030 0x562831ab7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.554+0000 7f0f43fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831a22030 0x562831ab7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50486/0 (socket says 172.21.15.44:50486) 2023-09-10T00:43:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.554+0000 7f0f43fff700 1 -- 172.21.15.44:0/585792655 learned_addr learned my addr 172.21.15.44:0/585792655 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.554+0000 7f0f437fe700 1 --2- 172.21.15.44:0/585792655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 0x562831aaca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.554+0000 7f0f4888b700 1 --2- 172.21.15.44:0/585792655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831aadcc0 0x562831ac6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.555+0000 7f0f43fff700 1 -- 172.21.15.44:0/585792655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 msgr2=0x562831aaca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.555+0000 7f0f43fff700 1 --2- 172.21.15.44:0/585792655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 0x562831aaca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.555+0000 7f0f43fff700 1 -- 172.21.15.44:0/585792655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831aadcc0 msgr2=0x562831ac6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.555+0000 7f0f43fff700 1 --2- 172.21.15.44:0/585792655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831aadcc0 0x562831ac6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.555+0000 7f0f43fff700 1 -- 172.21.15.44:0/585792655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562831814c00 con 0x562831a22030 2023-09-10T00:43:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.555+0000 7f0f43fff700 1 --2- 172.21.15.44:0/585792655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831a22030 0x562831ab7660 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0f34098880 tx=0x7f0f34085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdef90817e35e08b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.555+0000 7f0f2affd700 1 -- 172.21.15.44:0/585792655 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3409e360 con 0x562831a22030 2023-09-10T00:43:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.555+0000 7f0f2affd700 1 -- 172.21.15.44:0/585792655 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f3409ac30 con 0x562831a22030 2023-09-10T00:43:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.555+0000 7f0f2affd700 1 -- 172.21.15.44:0/585792655 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f340850b0 con 0x562831a22030 2023-09-10T00:43:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.556+0000 7f0f564b4b80 1 -- 172.21.15.44:0/585792655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831a22030 msgr2=0x562831ab7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.556+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/585792655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831a22030 0x562831ab7660 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0f34098880 tx=0x7f0f34085040 comp rx=0 tx=0).stop 2023-09-10T00:43:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.556+0000 7f0f564b4b80 1 -- 172.21.15.44:0/585792655 shutdown_connections 2023-09-10T00:43:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.556+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/585792655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831aadcc0 0x562831ac6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.556+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/585792655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831a22030 0x562831ab7660 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.556+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/585792655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 0x562831aaca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.556+0000 7f0f564b4b80 1 -- 172.21.15.44:0/585792655 >> 172.21.15.44:0/585792655 conn(0x562831ab17d0 msgr2=0x562831aac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.556+0000 7f0f564b4b80 1 -- 172.21.15.44:0/585792655 shutdown_connections 2023-09-10T00:43:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.556+0000 7f0f564b4b80 1 -- 172.21.15.44:0/585792655 wait complete. 2023-09-10T00:43:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.556+0000 7f0f564b4b80 1 -- start start 2023-09-10T00:43:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.556+0000 7f0f564b4b80 1 Processor -- start 2023-09-10T00:43:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.556+0000 7f0f564b4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831a22030 0x562831abf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.556+0000 7f0f564b4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 0x562831abf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f564b4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 0x562831b1e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f564b4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562831b1e980 con 0x562831aadcc0 2023-09-10T00:43:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f564b4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562831b1eaf0 con 0x562831a22030 2023-09-10T00:43:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f564b4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562831b1ecb0 con 0x562831aad300 2023-09-10T00:43:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f564b4b80 10 client.0 _ll_register_callbacks cb 0x7f0f380043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:43:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f43fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831a22030 0x562831abf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f437fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 0x562831abf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f43fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831a22030 0x562831abf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52232/0 (socket says 172.21.15.44:52232) 2023-09-10T00:43:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f437fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 0x562831abf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57376/0 (socket says 172.21.15.44:57376) 2023-09-10T00:43:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f43fff700 1 -- 172.21.15.44:0/520301868 learned_addr learned my addr 172.21.15.44:0/520301868 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f4888b700 1 --2- 172.21.15.44:0/520301868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 0x562831b1e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f43fff700 1 -- 172.21.15.44:0/520301868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 msgr2=0x562831abf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f43fff700 1 --2- 172.21.15.44:0/520301868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 0x562831abf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f43fff700 1 -- 172.21.15.44:0/520301868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 msgr2=0x562831b1e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f43fff700 1 --2- 172.21.15.44:0/520301868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 0x562831b1e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f43fff700 1 -- 172.21.15.44:0/520301868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56283182bd80 con 0x562831a22030 2023-09-10T00:43:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f437fe700 1 --2- 172.21.15.44:0/520301868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 0x562831abf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f43fff700 1 --2- 172.21.15.44:0/520301868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831a22030 0x562831abf490 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0f34000c00 tx=0x7f0f34005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeec7c72140e162a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f237fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:43:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f237fe700 1 -- 172.21.15.44:0/520301868 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f340a2510 con 0x562831a22030 2023-09-10T00:43:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f564b4b80 1 -- 172.21.15.44:0/520301868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562831ab89e0 con 0x562831a22030 2023-09-10T00:43:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.557+0000 7f0f237fe700 1 -- 172.21.15.44:0/520301868 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f0f3409e120 con 0x562831a22030 2023-09-10T00:43:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.558+0000 7f0f237fe700 1 -- 172.21.15.44:0/520301868 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f340850b0 con 0x562831a22030 2023-09-10T00:43:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.558+0000 7f0f564b4b80 10 client.26183 fetch_fsmap learned FSMap version 37 2023-09-10T00:43:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.558+0000 7f0f564b4b80 1 -- 172.21.15.44:0/520301868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5628318521f0 con 0x562831a22030 2023-09-10T00:43:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.558+0000 7f0f237fe700 1 -- 172.21.15.44:0/520301868 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562831ab89e0 con 0x562831a22030 2023-09-10T00:43:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.558+0000 7f0f237fe700 1 -- 172.21.15.44:0/520301868 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 37) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0f340228c0 con 0x562831a22030 2023-09-10T00:43:46.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.558+0000 7f0f564b4b80 10 client.26183 fetch_fsmap finished waiting for FSMap version 37 2023-09-10T00:43:46.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.558+0000 7f0f564b4b80 10 client.26183 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:43:46.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.558+0000 7f0f564b4b80 20 client.26183 populate_metadata read hostname 'smithi044' 2023-09-10T00:43:46.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.558+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/520301868 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562831ab31a0 0x562831b22170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:46.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.558+0000 7f0f564b4b80 4 client.26183 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:43:46.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.558+0000 7f0f564b4b80 1 -- 172.21.15.44:0/520301868 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f0f340cae90 con 0x562831ab31a0 2023-09-10T00:43:46.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.558+0000 7f0f437fe700 1 --2- 172.21.15.44:0/520301868 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562831ab31a0 0x562831b22170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:46.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.559+0000 7f0f237fe700 10 client.26183 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:46.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.559+0000 7f0f437fe700 1 --2- 172.21.15.44:0/520301868 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562831ab31a0 0x562831b22170 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12e715f5b7099027 server_cookie=85cfb0347536217a in_seq=0 out_seq=0 2023-09-10T00:43:46.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.568+0000 7f0f237fe700 1 -- 172.21.15.44:0/520301868 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f0f340cae90 con 0x562831ab31a0 2023-09-10T00:43:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.568+0000 7f0f237fe700 10 client.26183 handle_command_reply: tid=0 2023-09-10T00:43:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.568+0000 7f0f564b4b80 1 client.26183 shutdown 2023-09-10T00:43:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.568+0000 7f0f564b4b80 1 -- 172.21.15.44:0/520301868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831a22030 msgr2=0x562831abf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.568+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/520301868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831a22030 0x562831abf490 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0f34000c00 tx=0x7f0f34005c80 comp rx=0 tx=0).stop 2023-09-10T00:43:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.568+0000 7f0f564b4b80 1 -- 172.21.15.44:0/520301868 shutdown_connections 2023-09-10T00:43:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.568+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/520301868 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562831ab31a0 0x562831b22170 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.568+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/520301868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562831aadcc0 0x562831b1e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.568+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/520301868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562831a22030 0x562831abf490 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.568+0000 7f0f237fe700 0 client.26183 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.568+0000 7f0f564b4b80 1 --2- 172.21.15.44:0/520301868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562831aad300 0x562831abf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.568+0000 7f0f564b4b80 1 -- 172.21.15.44:0/520301868 >> 172.21.15.44:0/520301868 conn(0x562831ab17d0 msgr2=0x562831aacb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.568+0000 7f0f564b4b80 1 -- 172.21.15.44:0/520301868 shutdown_connections 2023-09-10T00:43:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.568+0000 7f0f564b4b80 1 -- 172.21.15.44:0/520301868 wait complete. 2023-09-10T00:43:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.568+0000 7f0f564b4b80 20 client.26183 trim_cache size 0 max 16384 2023-09-10T00:43:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:43:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:43:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:43:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:43:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.569+0000 7f0f4a2ee700 1 -- 172.21.15.44:0/625634459 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f2c0a07b0 msgr2=0x7f0f2c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.569+0000 7f0f4a2ee700 1 --2- 172.21.15.44:0/625634459 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f2c0a07b0 0x7f0f2c0a2c70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0f44196be0 tx=0x7f0f38009260 comp rx=0 tx=0).stop 2023-09-10T00:43:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.569+0000 7f0f4a2ee700 1 -- 172.21.15.44:0/625634459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f440ff760 msgr2=0x7f0f44195430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.569+0000 7f0f4a2ee700 1 --2- 172.21.15.44:0/625634459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f440ff760 0x7f0f44195430 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0f34005910 tx=0x7f0f34009580 comp rx=0 tx=0).stop 2023-09-10T00:43:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.569+0000 7f0f4a2ee700 1 -- 172.21.15.44:0/625634459 shutdown_connections 2023-09-10T00:43:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.569+0000 7f0f4a2ee700 1 --2- 172.21.15.44:0/625634459 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f2c0a07b0 0x7f0f2c0a2c70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.569+0000 7f0f4a2ee700 1 --2- 172.21.15.44:0/625634459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f441055b0 0x7f0f44199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.569+0000 7f0f4a2ee700 1 --2- 172.21.15.44:0/625634459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f440ff760 0x7f0f44195430 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.569+0000 7f0f4a2ee700 1 --2- 172.21.15.44:0/625634459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f44100120 0x7f0f44195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.569+0000 7f0f4a2ee700 1 -- 172.21.15.44:0/625634459 >> 172.21.15.44:0/625634459 conn(0x7f0f44074f30 msgr2=0x7f0f440fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.569+0000 7f0f4a2ee700 1 -- 172.21.15.44:0/625634459 shutdown_connections 2023-09-10T00:43:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:46.569+0000 7f0f4a2ee700 1 -- 172.21.15.44:0/625634459 wait complete. 2023-09-10T00:43:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:46 smithi131 ceph-mon[103713]: pgmap v96: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 49 MiB/s wr, 38 op/s 2023-09-10T00:43:47.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:46 smithi044 ceph-mon[98676]: pgmap v96: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 49 MiB/s wr, 38 op/s 2023-09-10T00:43:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:46 smithi099 ceph-mon[102692]: pgmap v96: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 49 MiB/s wr, 38 op/s 2023-09-10T00:43:47.562 INFO:teuthology.orchestra.run.smithi044.stderr:16000000 of 50000000 tuples (32%) done (elapsed 167.20 s, remaining 355.29 s) 2023-09-10T00:43:48.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:47 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:43:48.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:47 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:43:48.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:47 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:43:48.443 INFO:teuthology.orchestra.run.smithi044.stderr:16100000 of 50000000 tuples (32%) done (elapsed 168.08 s, remaining 353.90 s) 2023-09-10T00:43:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:48 smithi131 ceph-mon[103713]: pgmap v97: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 32 MiB/s wr, 22 op/s 2023-09-10T00:43:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:48 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:43:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:48 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:43:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:48 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:43:49.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:48 smithi044 ceph-mon[98676]: pgmap v97: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 32 MiB/s wr, 22 op/s 2023-09-10T00:43:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:48 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:43:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:48 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:43:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:48 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:43:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:48 smithi099 ceph-mon[102692]: pgmap v97: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 32 MiB/s wr, 22 op/s 2023-09-10T00:43:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:48 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:43:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:48 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:43:49.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:48 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:43:49.294 INFO:teuthology.orchestra.run.smithi044.stderr:16200000 of 50000000 tuples (32%) done (elapsed 168.93 s, remaining 352.46 s) 2023-09-10T00:43:50.185 INFO:teuthology.orchestra.run.smithi044.stderr:16300000 of 50000000 tuples (32%) done (elapsed 169.82 s, remaining 351.10 s) 2023-09-10T00:43:50.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:49 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:50.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:49 smithi131 ceph-mon[103713]: pgmap v98: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 32 MiB/s wr, 22 op/s 2023-09-10T00:43:50.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:49 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:50.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:49 smithi044 ceph-mon[98676]: pgmap v98: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 32 MiB/s wr, 22 op/s 2023-09-10T00:43:50.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:49 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:50.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:49 smithi099 ceph-mon[102692]: pgmap v98: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 32 MiB/s wr, 22 op/s 2023-09-10T00:43:51.032 INFO:teuthology.orchestra.run.smithi044.stderr:16400000 of 50000000 tuples (32%) done (elapsed 170.67 s, remaining 349.66 s) 2023-09-10T00:43:51.878 INFO:teuthology.orchestra.run.smithi044.stderr:16500000 of 50000000 tuples (33%) done (elapsed 171.51 s, remaining 348.22 s) 2023-09-10T00:43:52.763 INFO:teuthology.orchestra.run.smithi044.stderr:16600000 of 50000000 tuples (33%) done (elapsed 172.40 s, remaining 346.87 s) 2023-09-10T00:43:52.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:52 smithi131 ceph-mon[103713]: pgmap v99: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 MiB/s wr, 35 op/s 2023-09-10T00:43:52.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:52 smithi044 ceph-mon[98676]: pgmap v99: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 MiB/s wr, 35 op/s 2023-09-10T00:43:52.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:52 smithi099 ceph-mon[102692]: pgmap v99: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 MiB/s wr, 35 op/s 2023-09-10T00:43:53.672 INFO:teuthology.orchestra.run.smithi044.stderr:16700000 of 50000000 tuples (33%) done (elapsed 173.31 s, remaining 345.58 s) 2023-09-10T00:43:54.482 INFO:teuthology.orchestra.run.smithi044.stderr:16800000 of 50000000 tuples (33%) done (elapsed 174.12 s, remaining 344.09 s) 2023-09-10T00:43:54.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:54 smithi131 ceph-mon[103713]: pgmap v100: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 22 op/s 2023-09-10T00:43:54.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:54 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:54.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:54 smithi044 ceph-mon[98676]: pgmap v100: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 22 op/s 2023-09-10T00:43:54.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:54 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:54.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:54 smithi099 ceph-mon[102692]: pgmap v100: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 22 op/s 2023-09-10T00:43:54.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:54 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:55.332 INFO:teuthology.orchestra.run.smithi044.stderr:16900000 of 50000000 tuples (33%) done (elapsed 174.97 s, remaining 342.69 s) 2023-09-10T00:43:56.239 INFO:teuthology.orchestra.run.smithi044.stderr:17000000 of 50000000 tuples (34%) done (elapsed 175.87 s, remaining 341.40 s) 2023-09-10T00:43:56.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:56 smithi131 ceph-mon[103713]: pgmap v101: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 44 MiB/s wr, 35 op/s 2023-09-10T00:43:56.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:56 smithi044 ceph-mon[98676]: pgmap v101: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 44 MiB/s wr, 35 op/s 2023-09-10T00:43:56.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:56 smithi099 ceph-mon[102692]: pgmap v101: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 44 MiB/s wr, 35 op/s 2023-09-10T00:43:57.119 INFO:teuthology.orchestra.run.smithi044.stderr:17100000 of 50000000 tuples (34%) done (elapsed 176.75 s, remaining 340.07 s) 2023-09-10T00:43:57.951 INFO:teuthology.orchestra.run.smithi044.stderr:17200000 of 50000000 tuples (34%) done (elapsed 177.59 s, remaining 338.65 s) 2023-09-10T00:43:58.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:43:58 smithi131 ceph-mon[103713]: pgmap v102: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 MiB/s wr, 24 op/s 2023-09-10T00:43:58.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:43:58 smithi044 ceph-mon[98676]: pgmap v102: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 MiB/s wr, 24 op/s 2023-09-10T00:43:58.815 INFO:teuthology.orchestra.run.smithi044.stderr:17300000 of 50000000 tuples (34%) done (elapsed 178.45 s, remaining 337.30 s) 2023-09-10T00:43:58.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:43:58 smithi099 ceph-mon[102692]: pgmap v102: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 MiB/s wr, 24 op/s 2023-09-10T00:43:59.676 INFO:teuthology.orchestra.run.smithi044.stderr:17400000 of 50000000 tuples (34%) done (elapsed 179.31 s, remaining 335.95 s) 2023-09-10T00:43:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.701+0000 7fa5af2c6700 1 -- 172.21.15.44:0/2197520711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a80679f0 msgr2=0x7fa5a8067e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.701+0000 7fa5af2c6700 1 --2- 172.21.15.44:0/2197520711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a80679f0 0x7fa5a8067e70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa5980099c0 tx=0x7fa598009cd0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.701+0000 7fa5af2c6700 1 -- 172.21.15.44:0/2197520711 shutdown_connections 2023-09-10T00:43:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.701+0000 7fa5af2c6700 1 --2- 172.21.15.44:0/2197520711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5a80683b0 0x7fa5a810d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.701+0000 7fa5af2c6700 1 --2- 172.21.15.44:0/2197520711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5a8106650 0x7fa5a8106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.701+0000 7fa5af2c6700 1 --2- 172.21.15.44:0/2197520711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a80679f0 0x7fa5a8067e70 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.701+0000 7fa5af2c6700 1 -- 172.21.15.44:0/2197520711 >> 172.21.15.44:0/2197520711 conn(0x7fa5a8074f30 msgr2=0x7fa5a8075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.701+0000 7fa5af2c6700 1 -- 172.21.15.44:0/2197520711 shutdown_connections 2023-09-10T00:43:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.701+0000 7fa5af2c6700 1 -- 172.21.15.44:0/2197520711 wait complete. 2023-09-10T00:43:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5af2c6700 1 Processor -- start 2023-09-10T00:43:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5af2c6700 1 -- start start 2023-09-10T00:43:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5af2c6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a80679f0 0x7fa5a8196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5af2c6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5a80683b0 0x7fa5a8196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5af2c6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5a8106650 0x7fa5a819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5af2c6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a819a600 con 0x7fa5a80679f0 2023-09-10T00:43:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5af2c6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a819a770 con 0x7fa5a80683b0 2023-09-10T00:43:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5af2c6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a819a930 con 0x7fa5a8106650 2023-09-10T00:43:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5ad062700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a80679f0 0x7fa5a8196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5ad062700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a80679f0 0x7fa5a8196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50506/0 (socket says 172.21.15.44:50506) 2023-09-10T00:43:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5ad062700 1 -- 172.21.15.44:0/2908128491 learned_addr learned my addr 172.21.15.44:0/2908128491 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5ac861700 1 --2- 172.21.15.44:0/2908128491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5a80683b0 0x7fa5a8196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5ad863700 1 --2- 172.21.15.44:0/2908128491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5a8106650 0x7fa5a819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5ad062700 1 -- 172.21.15.44:0/2908128491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5a8106650 msgr2=0x7fa5a819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5ad062700 1 --2- 172.21.15.44:0/2908128491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5a8106650 0x7fa5a819a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5ad062700 1 -- 172.21.15.44:0/2908128491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5a80683b0 msgr2=0x7fa5a8196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5ad062700 1 --2- 172.21.15.44:0/2908128491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5a80683b0 0x7fa5a8196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.702+0000 7fa5ad062700 1 -- 172.21.15.44:0/2908128491 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5980095f0 con 0x7fa5a80679f0 2023-09-10T00:43:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.703+0000 7fa5ad062700 1 --2- 172.21.15.44:0/2908128491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a80679f0 0x7fa5a8196500 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400e9a0 tx=0x7fa5a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.703+0000 7fa59e7fc700 1 -- 172.21.15.44:0/2908128491 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4009790 con 0x7fa5a80679f0 2023-09-10T00:43:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.703+0000 7fa5af2c6700 1 -- 172.21.15.44:0/2908128491 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a819ac10 con 0x7fa5a80679f0 2023-09-10T00:43:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.703+0000 7fa59e7fc700 1 -- 172.21.15.44:0/2908128491 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa5a4010430 con 0x7fa5a80679f0 2023-09-10T00:43:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.703+0000 7fa59e7fc700 1 -- 172.21.15.44:0/2908128491 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a400cea0 con 0x7fa5a80679f0 2023-09-10T00:43:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.703+0000 7fa5af2c6700 1 -- 172.21.15.44:0/2908128491 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a81a3570 con 0x7fa5a80679f0 2023-09-10T00:43:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.705+0000 7fa59e7fc700 1 -- 172.21.15.44:0/2908128491 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa5a40105a0 con 0x7fa5a80679f0 2023-09-10T00:43:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.706+0000 7fa59e7fc700 1 --2- 172.21.15.44:0/2908128491 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa5940a07b0 0x7fa5940a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.706+0000 7fa5ac861700 1 --2- 172.21.15.44:0/2908128491 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa5940a07b0 0x7fa5940a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.706+0000 7fa59e7fc700 1 -- 172.21.15.44:0/2908128491 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa5a4014070 con 0x7fa5a80679f0 2023-09-10T00:43:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.707+0000 7fa5ac861700 1 --2- 172.21.15.44:0/2908128491 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa5940a07b0 0x7fa5940a2c70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa59800b580 tx=0x7fa598005c00 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.707+0000 7fa5bb48cb80 1 Processor -- start 2023-09-10T00:43:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.707+0000 7fa5bb48cb80 1 -- start start 2023-09-10T00:43:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5bb48cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 0x55ee46eda580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5bb48cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4300 0x55ee46ed4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5bb48cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46ed4cc0 0x55ee46ee0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5bb48cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee46e2d6d0 con 0x55ee46eda1a0 2023-09-10T00:43:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5bb48cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee46e2aff0 con 0x55ee46ed4cc0 2023-09-10T00:43:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5bb48cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee46e268e0 con 0x55ee46ed4300 2023-09-10T00:43:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5ad062700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 0x55ee46eda580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5ac861700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4300 0x55ee46ed4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5ad062700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 0x55ee46eda580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50514/0 (socket says 172.21.15.44:50514) 2023-09-10T00:43:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5ad062700 1 -- 172.21.15.44:0/3741309784 learned_addr learned my addr 172.21.15.44:0/3741309784 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5ad863700 1 --2- 172.21.15.44:0/3741309784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46ed4cc0 0x55ee46ee0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5ad062700 1 -- 172.21.15.44:0/3741309784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4300 msgr2=0x55ee46ed4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5ad062700 1 --2- 172.21.15.44:0/3741309784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4300 0x55ee46ed4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5ad062700 1 -- 172.21.15.44:0/3741309784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46ed4cc0 msgr2=0x55ee46ee0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5ad062700 1 --2- 172.21.15.44:0/3741309784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46ed4cc0 0x55ee46ee0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5ad062700 1 -- 172.21.15.44:0/3741309784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee46c52d80 con 0x55ee46eda1a0 2023-09-10T00:43:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5ad062700 1 --2- 172.21.15.44:0/3741309784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 0x55ee46eda580 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40a1130 tx=0x7fa5a4098f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc5d9ed85b647c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.708+0000 7fa5937fe700 1 -- 172.21.15.44:0/3741309784 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40995f0 con 0x55ee46eda1a0 2023-09-10T00:43:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.709+0000 7fa5937fe700 1 -- 172.21.15.44:0/3741309784 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa5a409d420 con 0x55ee46eda1a0 2023-09-10T00:43:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.709+0000 7fa5937fe700 1 -- 172.21.15.44:0/3741309784 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40840b0 con 0x55ee46eda1a0 2023-09-10T00:43:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.709+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3741309784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 msgr2=0x55ee46eda580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.709+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/3741309784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 0x55ee46eda580 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40a1130 tx=0x7fa5a4098f50 comp rx=0 tx=0).stop 2023-09-10T00:43:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.710+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3741309784 shutdown_connections 2023-09-10T00:43:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.710+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/3741309784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46ed4cc0 0x55ee46ee0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.710+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/3741309784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 0x55ee46eda580 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.710+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/3741309784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4300 0x55ee46ed4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.710+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3741309784 >> 172.21.15.44:0/3741309784 conn(0x55ee46e49030 msgr2=0x55ee46ecf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.710+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3741309784 shutdown_connections 2023-09-10T00:43:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.710+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3741309784 wait complete. 2023-09-10T00:43:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.710+0000 7fa5bb48cb80 1 -- start start 2023-09-10T00:43:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.710+0000 7fa5bb48cb80 1 Processor -- start 2023-09-10T00:43:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5bb48cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46ed4300 0x55ee46ede1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5bb48cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4cc0 0x55ee46ede700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5bb48cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46eda1a0 0x55ee46edec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5bb48cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee46dc4b10 con 0x55ee46ed4300 2023-09-10T00:43:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5bb48cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee46dc4810 con 0x55ee46eda1a0 2023-09-10T00:43:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5bb48cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee46ca88c0 con 0x55ee46ed4cc0 2023-09-10T00:43:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5bb48cb80 10 client.0 _ll_register_callbacks cb 0x55ee46d0ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:43:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5ad062700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46ed4300 0x55ee46ede1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5ad863700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46eda1a0 0x55ee46edec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5ac861700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4cc0 0x55ee46ede700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5ad062700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46ed4300 0x55ee46ede1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50520/0 (socket says 172.21.15.44:50520) 2023-09-10T00:43:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5ad062700 1 -- 172.21.15.44:0/3181524767 learned_addr learned my addr 172.21.15.44:0/3181524767 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5ad863700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46eda1a0 0x55ee46edec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52264/0 (socket says 172.21.15.44:52264) 2023-09-10T00:43:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5ac861700 1 -- 172.21.15.44:0/3181524767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46eda1a0 msgr2=0x55ee46edec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5ac861700 1 --2- 172.21.15.44:0/3181524767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46eda1a0 0x55ee46edec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5ac861700 1 -- 172.21.15.44:0/3181524767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46ed4300 msgr2=0x55ee46ede1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5ac861700 1 --2- 172.21.15.44:0/3181524767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46ed4300 0x55ee46ede1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5ac861700 1 -- 172.21.15.44:0/3181524767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a81a3570 con 0x55ee46ed4cc0 2023-09-10T00:43:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5ad863700 1 --2- 172.21.15.44:0/3181524767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46eda1a0 0x55ee46edec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5ad062700 1 --2- 172.21.15.44:0/3181524767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46ed4300 0x55ee46ede1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa5ac861700 1 --2- 172.21.15.44:0/3181524767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4cc0 0x55ee46ede700 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa598000c00 tx=0x7fa598009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daf66b91e44a3f4b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.711+0000 7fa590ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:43:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.712+0000 7fa590ff9700 1 -- 172.21.15.44:0/3181524767 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59802e040 con 0x55ee46ed4cc0 2023-09-10T00:43:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.712+0000 7fa590ff9700 1 -- 172.21.15.44:0/3181524767 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa598003960 con 0x55ee46ed4cc0 2023-09-10T00:43:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.712+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3181524767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee46d7e490 con 0x55ee46ed4cc0 2023-09-10T00:43:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.712+0000 7fa590ff9700 1 -- 172.21.15.44:0/3181524767 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa598003b60 con 0x55ee46ed4cc0 2023-09-10T00:43:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.712+0000 7fa5bb48cb80 10 client.26131 fetch_fsmap learned FSMap version 39 2023-09-10T00:43:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.712+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3181524767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ee46c3bc00 con 0x55ee46ed4cc0 2023-09-10T00:43:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.712+0000 7fa590ff9700 1 -- 172.21.15.44:0/3181524767 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ee46d7e490 con 0x55ee46ed4cc0 2023-09-10T00:43:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.712+0000 7fa590ff9700 1 -- 172.21.15.44:0/3181524767 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 39) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fa59800fb20 con 0x55ee46ed4cc0 2023-09-10T00:43:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.712+0000 7fa5bb48cb80 10 client.26131 fetch_fsmap finished waiting for FSMap version 39 2023-09-10T00:43:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.712+0000 7fa5bb48cb80 10 client.26131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:43:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.712+0000 7fa5bb48cb80 20 client.26131 populate_metadata read hostname 'smithi044' 2023-09-10T00:43:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.712+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/3181524767 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ee46eee470 0x55ee46ef0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.712+0000 7fa5bb48cb80 4 client.26131 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:43:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.712+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3181524767 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee46e3b840 con 0x55ee46eee470 2023-09-10T00:43:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.712+0000 7fa5ad062700 1 --2- 172.21.15.44:0/3181524767 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ee46eee470 0x55ee46ef0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.713+0000 7fa590ff9700 10 client.26131 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.713+0000 7fa5ad062700 1 --2- 172.21.15.44:0/3181524767 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ee46eee470 0x55ee46ef0890 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ce400304601a147 server_cookie=d495c73c1842b5e4 in_seq=0 out_seq=0 2023-09-10T00:43:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.714+0000 7fa590ff9700 1 -- 172.21.15.44:0/3181524767 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ee46e3b840 con 0x55ee46eee470 2023-09-10T00:43:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.714+0000 7fa590ff9700 10 client.26131 handle_command_reply: tid=0 2023-09-10T00:43:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.714+0000 7fa5bb48cb80 1 client.26131 shutdown 2023-09-10T00:43:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.714+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3181524767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4cc0 msgr2=0x55ee46ede700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.714+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/3181524767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4cc0 0x55ee46ede700 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa598000c00 tx=0x7fa598009cd0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.714+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3181524767 shutdown_connections 2023-09-10T00:43:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.714+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/3181524767 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ee46eee470 0x55ee46ef0890 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.714+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/3181524767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46eda1a0 0x55ee46edec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.714+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/3181524767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46ed4300 0x55ee46ede1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.714+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/3181524767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4cc0 0x55ee46ede700 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.714+0000 7fa590ff9700 0 client.26131 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.714+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3181524767 >> 172.21.15.44:0/3181524767 conn(0x55ee46e49030 msgr2=0x55ee46ece7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.714+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3181524767 shutdown_connections 2023-09-10T00:43:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.714+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3181524767 wait complete. 2023-09-10T00:43:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.714+0000 7fa5bb48cb80 20 client.26131 trim_cache size 0 max 16384 2023-09-10T00:43:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.735+0000 7fa5bb48cb80 1 Processor -- start 2023-09-10T00:43:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.735+0000 7fa5bb48cb80 1 -- start start 2023-09-10T00:43:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.735+0000 7fa5bb48cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46ed4300 0x55ee46edbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5bb48cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4cc0 0x55ee46ee5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5bb48cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 0x55ee46ecef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5bb48cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee46ecbfc0 con 0x55ee46eda1a0 2023-09-10T00:43:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5bb48cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee46ed50a0 con 0x55ee46ed4300 2023-09-10T00:43:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5bb48cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee46eebb30 con 0x55ee46ed4cc0 2023-09-10T00:43:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5ac861700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4cc0 0x55ee46ee5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5ad062700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46ed4300 0x55ee46edbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5ac861700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4cc0 0x55ee46ee5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57412/0 (socket says 172.21.15.44:57412) 2023-09-10T00:43:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5ad863700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 0x55ee46ecef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5ac861700 1 -- 172.21.15.44:0/3378526831 learned_addr learned my addr 172.21.15.44:0/3378526831 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5ad062700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46ed4300 0x55ee46edbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52268/0 (socket says 172.21.15.44:52268) 2023-09-10T00:43:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5ac861700 1 -- 172.21.15.44:0/3378526831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46ed4300 msgr2=0x55ee46edbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5ac861700 1 --2- 172.21.15.44:0/3378526831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46ed4300 0x55ee46edbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5ac861700 1 -- 172.21.15.44:0/3378526831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 msgr2=0x55ee46ecef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5ac861700 1 --2- 172.21.15.44:0/3378526831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 0x55ee46ecef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5ac861700 1 -- 172.21.15.44:0/3378526831 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee46c791f0 con 0x55ee46ed4cc0 2023-09-10T00:43:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa5ac861700 1 --2- 172.21.15.44:0/3378526831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4cc0 0x55ee46ee5420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa59800ff30 tx=0x7fa59800bb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6ff6d0b9aa530b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa593fff700 1 -- 172.21.15.44:0/3378526831 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59802e050 con 0x55ee46ed4cc0 2023-09-10T00:43:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa593fff700 1 -- 172.21.15.44:0/3378526831 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa59801e040 con 0x55ee46ed4cc0 2023-09-10T00:43:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.736+0000 7fa593fff700 1 -- 172.21.15.44:0/3378526831 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598003ae0 con 0x55ee46ed4cc0 2023-09-10T00:43:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.737+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3378526831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4cc0 msgr2=0x55ee46ee5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.737+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/3378526831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4cc0 0x55ee46ee5420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa59800ff30 tx=0x7fa59800bb30 comp rx=0 tx=0).stop 2023-09-10T00:43:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.737+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3378526831 shutdown_connections 2023-09-10T00:43:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.737+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/3378526831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 0x55ee46ecef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.737+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/3378526831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46ed4300 0x55ee46edbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.737+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/3378526831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4cc0 0x55ee46ee5420 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.737+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3378526831 >> 172.21.15.44:0/3378526831 conn(0x55ee46e49030 msgr2=0x55ee46ed82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.737+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3378526831 shutdown_connections 2023-09-10T00:43:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.737+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/3378526831 wait complete. 2023-09-10T00:43:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.738+0000 7fa5bb48cb80 1 -- start start 2023-09-10T00:43:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.738+0000 7fa5bb48cb80 1 Processor -- start 2023-09-10T00:43:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.738+0000 7fa5bb48cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4300 0x55ee46f36420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.738+0000 7fa5bb48cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46ed4cc0 0x55ee46f36960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.738+0000 7fa5bb48cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 0x55ee46f412b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.738+0000 7fa5bb48cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee46f37030 con 0x55ee46eda1a0 2023-09-10T00:43:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.738+0000 7fa5bb48cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee46f417f0 con 0x55ee46ed4cc0 2023-09-10T00:43:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.738+0000 7fa5bb48cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee46f41930 con 0x55ee46ed4300 2023-09-10T00:43:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.738+0000 7fa5bb48cb80 10 client.0 _ll_register_callbacks cb 0x7fa598026b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:43:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.738+0000 7fa5ad062700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4300 0x55ee46f36420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa5ad863700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 0x55ee46f412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa5ad062700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4300 0x55ee46f36420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57416/0 (socket says 172.21.15.44:57416) 2023-09-10T00:43:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa5ad062700 1 -- 172.21.15.44:0/1368971438 learned_addr learned my addr 172.21.15.44:0/1368971438 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:43:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa5ac861700 1 --2- 172.21.15.44:0/1368971438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46ed4cc0 0x55ee46f36960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa5ad062700 1 -- 172.21.15.44:0/1368971438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46ed4cc0 msgr2=0x55ee46f36960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa5ad062700 1 --2- 172.21.15.44:0/1368971438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46ed4cc0 0x55ee46f36960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa5ad062700 1 -- 172.21.15.44:0/1368971438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 msgr2=0x55ee46f412b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:43:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa5ad062700 1 --2- 172.21.15.44:0/1368971438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 0x55ee46f412b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa5ad062700 1 -- 172.21.15.44:0/1368971438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee46c3bc00 con 0x55ee46ed4300 2023-09-10T00:43:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa5ad863700 1 --2- 172.21.15.44:0/1368971438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 0x55ee46f412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:43:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa5ad062700 1 --2- 172.21.15.44:0/1368971438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4300 0x55ee46f36420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4096060 tx=0x7fa5a40994e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98482cb173be278e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:43:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa590ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:43:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa590ff9700 1 -- 172.21.15.44:0/1368971438 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40995f0 con 0x55ee46ed4300 2023-09-10T00:43:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa590ff9700 1 -- 172.21.15.44:0/1368971438 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa5a40a1660 con 0x55ee46ed4300 2023-09-10T00:43:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/1368971438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5a409d420 con 0x55ee46ed4300 2023-09-10T00:43:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa590ff9700 1 -- 172.21.15.44:0/1368971438 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5a4097990 con 0x55ee46ed4300 2023-09-10T00:43:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa5bb48cb80 10 client.26137 fetch_fsmap learned FSMap version 39 2023-09-10T00:43:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.739+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/1368971438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ee46c51250 con 0x55ee46ed4300 2023-09-10T00:43:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.740+0000 7fa590ff9700 1 -- 172.21.15.44:0/1368971438 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa5a409d420 con 0x55ee46ed4300 2023-09-10T00:43:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.740+0000 7fa590ff9700 1 -- 172.21.15.44:0/1368971438 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 39) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fa5a4099850 con 0x55ee46ed4300 2023-09-10T00:43:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.740+0000 7fa5bb48cb80 10 client.26137 fetch_fsmap finished waiting for FSMap version 39 2023-09-10T00:43:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.740+0000 7fa5bb48cb80 10 client.26137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:43:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.740+0000 7fa5bb48cb80 20 client.26137 populate_metadata read hostname 'smithi044' 2023-09-10T00:43:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.740+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/1368971438 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ee46f42ce0 0x55ee46f45100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:43:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.740+0000 7fa5bb48cb80 4 client.26137 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:43:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.740+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/1368971438 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ee46e1f8c0 con 0x55ee46f42ce0 2023-09-10T00:43:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.740+0000 7fa5ac861700 1 --2- 172.21.15.44:0/1368971438 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ee46f42ce0 0x55ee46f45100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:43:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.740+0000 7fa590ff9700 10 client.26137 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.740+0000 7fa5ac861700 1 --2- 172.21.15.44:0/1368971438 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ee46f42ce0 0x55ee46f45100 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2db13b4ae6bc3ea server_cookie=4edf4bf3106c2eb in_seq=0 out_seq=0 2023-09-10T00:43:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.752+0000 7fa590ff9700 1 -- 172.21.15.44:0/1368971438 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ee46e1f8c0 con 0x55ee46f42ce0 2023-09-10T00:43:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.752+0000 7fa590ff9700 10 client.26137 handle_command_reply: tid=0 2023-09-10T00:43:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.753+0000 7fa5bb48cb80 1 client.26137 shutdown 2023-09-10T00:43:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.753+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/1368971438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4300 msgr2=0x55ee46f36420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.753+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/1368971438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4300 0x55ee46f36420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4096060 tx=0x7fa5a40994e0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.753+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/1368971438 shutdown_connections 2023-09-10T00:43:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.753+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/1368971438 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ee46f42ce0 0x55ee46f45100 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.753+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/1368971438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee46eda1a0 0x55ee46f412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.753+0000 7fa590ff9700 0 client.26137 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:43:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.753+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/1368971438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee46ed4300 0x55ee46f36420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.753+0000 7fa5bb48cb80 1 --2- 172.21.15.44:0/1368971438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee46ed4cc0 0x55ee46f36960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.753+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/1368971438 >> 172.21.15.44:0/1368971438 conn(0x55ee46e49030 msgr2=0x55ee46ed32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.753+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/1368971438 shutdown_connections 2023-09-10T00:43:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.753+0000 7fa5bb48cb80 1 -- 172.21.15.44:0/1368971438 wait complete. 2023-09-10T00:43:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.753+0000 7fa5bb48cb80 20 client.26137 trim_cache size 0 max 16384 2023-09-10T00:43:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:43:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:43:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:43:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:43:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.754+0000 7fa5af2c6700 1 -- 172.21.15.44:0/2908128491 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa5940a07b0 msgr2=0x7fa5940a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.754+0000 7fa5af2c6700 1 --2- 172.21.15.44:0/2908128491 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa5940a07b0 0x7fa5940a2c70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa59800b580 tx=0x7fa598005c00 comp rx=0 tx=0).stop 2023-09-10T00:43:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.754+0000 7fa5af2c6700 1 -- 172.21.15.44:0/2908128491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a80679f0 msgr2=0x7fa5a8196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:43:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.754+0000 7fa5af2c6700 1 --2- 172.21.15.44:0/2908128491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a80679f0 0x7fa5a8196500 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400e9a0 tx=0x7fa5a400ecb0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.754+0000 7fa5af2c6700 1 -- 172.21.15.44:0/2908128491 shutdown_connections 2023-09-10T00:43:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.754+0000 7fa5af2c6700 1 --2- 172.21.15.44:0/2908128491 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa5940a07b0 0x7fa5940a2c70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.754+0000 7fa5af2c6700 1 --2- 172.21.15.44:0/2908128491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5a8106650 0x7fa5a819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.754+0000 7fa5af2c6700 1 --2- 172.21.15.44:0/2908128491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5a80679f0 0x7fa5a8196500 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.754+0000 7fa5af2c6700 1 --2- 172.21.15.44:0/2908128491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5a80683b0 0x7fa5a8196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:43:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.754+0000 7fa5af2c6700 1 -- 172.21.15.44:0/2908128491 >> 172.21.15.44:0/2908128491 conn(0x7fa5a8074f30 msgr2=0x7fa5a80fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:43:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.754+0000 7fa5af2c6700 1 -- 172.21.15.44:0/2908128491 shutdown_connections 2023-09-10T00:43:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:43:59.754+0000 7fa5af2c6700 1 -- 172.21.15.44:0/2908128491 wait complete. 2023-09-10T00:44:00.585 INFO:teuthology.orchestra.run.smithi044.stderr:17500000 of 50000000 tuples (35%) done (elapsed 180.22 s, remaining 334.69 s) 2023-09-10T00:44:00.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:00 smithi131 ceph-mon[103713]: pgmap v103: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 MiB/s wr, 24 op/s 2023-09-10T00:44:00.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:00 smithi044 ceph-mon[98676]: pgmap v103: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 MiB/s wr, 24 op/s 2023-09-10T00:44:00.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:00 smithi099 ceph-mon[102692]: pgmap v103: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 MiB/s wr, 24 op/s 2023-09-10T00:44:01.486 INFO:teuthology.orchestra.run.smithi044.stderr:17600000 of 50000000 tuples (35%) done (elapsed 181.12 s, remaining 333.43 s) 2023-09-10T00:44:02.421 INFO:teuthology.orchestra.run.smithi044.stderr:17700000 of 50000000 tuples (35%) done (elapsed 182.05 s, remaining 332.22 s) 2023-09-10T00:44:02.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:02 smithi044 ceph-mon[98676]: pgmap v104: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 46 MiB/s wr, 39 op/s 2023-09-10T00:44:02.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:02 smithi099 ceph-mon[102692]: pgmap v104: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 46 MiB/s wr, 39 op/s 2023-09-10T00:44:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:02 smithi131 ceph-mon[103713]: pgmap v104: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 46 MiB/s wr, 39 op/s 2023-09-10T00:44:04.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:04 smithi099 ceph-mon[102692]: pgmap v105: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 35 MiB/s wr, 27 op/s 2023-09-10T00:44:05.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:04 smithi131 ceph-mon[103713]: pgmap v105: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 35 MiB/s wr, 27 op/s 2023-09-10T00:44:05.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:04 smithi044 ceph-mon[98676]: pgmap v105: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 35 MiB/s wr, 27 op/s 2023-09-10T00:44:06.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:06 smithi099 ceph-mon[102692]: pgmap v106: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 45 MiB/s wr, 67 op/s 2023-09-10T00:44:06.938 INFO:teuthology.orchestra.run.smithi044.stderr:17800000 of 50000000 tuples (35%) done (elapsed 186.57 s, remaining 337.51 s) 2023-09-10T00:44:07.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:06 smithi131 ceph-mon[103713]: pgmap v106: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 45 MiB/s wr, 67 op/s 2023-09-10T00:44:07.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:06 smithi044 ceph-mon[98676]: pgmap v106: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 45 MiB/s wr, 67 op/s 2023-09-10T00:44:07.340 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:44:07.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.485+0000 7ff11e667700 1 -- 172.21.15.44:0/451406919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff118105550 msgr2=0x7ff118105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:07.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.485+0000 7ff11e667700 1 --2- 172.21.15.44:0/451406919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff118105550 0x7ff118105930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff1080098f0 tx=0x7ff108009c00 comp rx=0 tx=0).stop 2023-09-10T00:44:07.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.486+0000 7ff11e667700 1 -- 172.21.15.44:0/451406919 shutdown_connections 2023-09-10T00:44:07.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.486+0000 7ff11e667700 1 --2- 172.21.15.44:0/451406919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1180ffe50 0x7ff11810c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.486+0000 7ff11e667700 1 --2- 172.21.15.44:0/451406919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff118105550 0x7ff118105930 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.486+0000 7ff11e667700 1 --2- 172.21.15.44:0/451406919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1180ff490 0x7ff1180ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.486+0000 7ff11e667700 1 -- 172.21.15.44:0/451406919 >> 172.21.15.44:0/451406919 conn(0x7ff1180faf20 msgr2=0x7ff1180fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:07.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.486+0000 7ff11e667700 1 -- 172.21.15.44:0/451406919 shutdown_connections 2023-09-10T00:44:07.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.486+0000 7ff11e667700 1 -- 172.21.15.44:0/451406919 wait complete. 2023-09-10T00:44:07.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.486+0000 7ff11e667700 1 Processor -- start 2023-09-10T00:44:07.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.486+0000 7ff11e667700 1 -- start start 2023-09-10T00:44:07.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff11e667700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1180ff490 0x7ff118197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff11e667700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1180ffe50 0x7ff1181978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff11e667700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff118105550 0x7ff1181a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff11e667700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1181981d0 con 0x7ff1180ff490 2023-09-10T00:44:07.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff11e667700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1181a2940 con 0x7ff1180ffe50 2023-09-10T00:44:07.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff11e667700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1181a2ab0 con 0x7ff118105550 2023-09-10T00:44:07.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff117fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1180ff490 0x7ff118197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff117fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1180ff490 0x7ff118197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50548/0 (socket says 172.21.15.44:50548) 2023-09-10T00:44:07.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff117fff700 1 -- 172.21.15.44:0/1967114449 learned_addr learned my addr 172.21.15.44:0/1967114449 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:07.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff11cc04700 1 --2- 172.21.15.44:0/1967114449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff118105550 0x7ff1181a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff1177fe700 1 --2- 172.21.15.44:0/1967114449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1180ffe50 0x7ff1181978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff117fff700 1 -- 172.21.15.44:0/1967114449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff118105550 msgr2=0x7ff1181a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:07.493 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff117fff700 1 --2- 172.21.15.44:0/1967114449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff118105550 0x7ff1181a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff117fff700 1 -- 172.21.15.44:0/1967114449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1180ffe50 msgr2=0x7ff1181978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:07.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff117fff700 1 --2- 172.21.15.44:0/1967114449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1180ffe50 0x7ff1181978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff117fff700 1 -- 172.21.15.44:0/1967114449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c0095f0 con 0x7ff1180ff490 2023-09-10T00:44:07.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff11cc04700 1 --2- 172.21.15.44:0/1967114449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff118105550 0x7ff1181a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:07.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.487+0000 7ff117fff700 1 --2- 172.21.15.44:0/1967114449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1180ff490 0x7ff118197380 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff108009810 tx=0x7ff10800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:07.494 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.488+0000 7ff1157fa700 1 -- 172.21.15.44:0/1967114449 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10801c070 con 0x7ff1180ff490 2023-09-10T00:44:07.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.488+0000 7ff1157fa700 1 -- 172.21.15.44:0/1967114449 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1080058e0 con 0x7ff1180ff490 2023-09-10T00:44:07.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.488+0000 7ff11e667700 1 -- 172.21.15.44:0/1967114449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff108009520 con 0x7ff1180ff490 2023-09-10T00:44:07.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.488+0000 7ff1157fa700 1 -- 172.21.15.44:0/1967114449 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1080036c0 con 0x7ff1180ff490 2023-09-10T00:44:07.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.488+0000 7ff11e667700 1 -- 172.21.15.44:0/1967114449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1181a3050 con 0x7ff1180ff490 2023-09-10T00:44:07.500 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.491+0000 7ff1157fa700 1 -- 172.21.15.44:0/1967114449 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ff108020030 con 0x7ff1180ff490 2023-09-10T00:44:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.491+0000 7ff1157fa700 1 --2- 172.21.15.44:0/1967114449 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1000a07b0 0x7ff1000a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff1177fe700 1 --2- 172.21.15.44:0/1967114449 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1000a07b0 0x7ff1000a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.491+0000 7ff1157fa700 1 -- 172.21.15.44:0/1967114449 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ff1080cd210 con 0x7ff1180ff490 2023-09-10T00:44:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff12a82db80 1 Processor -- start 2023-09-10T00:44:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff12a82db80 1 -- start start 2023-09-10T00:44:07.502 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff12a82db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579c710 0x5624c579caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.502 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff12a82db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579d030 0x5624c57a6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.502 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff12a82db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c57a6e40 0x5624c57a9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.502 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff12a82db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624c56f5730 con 0x5624c57a6e40 2023-09-10T00:44:07.502 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff12a82db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624c56f3050 con 0x5624c579d030 2023-09-10T00:44:07.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff12a82db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624c56ee940 con 0x5624c579c710 2023-09-10T00:44:07.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff1177fe700 1 --2- 172.21.15.44:0/1967114449 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1000a07b0 0x7ff1000a2c70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff118069000 tx=0x7ff10c009310 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:07.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff1177fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579d030 0x5624c57a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff11cc04700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c57a6e40 0x5624c57a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff117fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579c710 0x5624c579caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff1177fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579d030 0x5624c57a6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52298/0 (socket says 172.21.15.44:52298) 2023-09-10T00:44:07.504 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff11cc04700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c57a6e40 0x5624c57a9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50560/0 (socket says 172.21.15.44:50560) 2023-09-10T00:44:07.504 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff1177fe700 1 -- 172.21.15.44:0/2833318015 learned_addr learned my addr 172.21.15.44:0/2833318015 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:07.504 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff117fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579c710 0x5624c579caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57438/0 (socket says 172.21.15.44:57438) 2023-09-10T00:44:07.504 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff117fff700 1 -- 172.21.15.44:0/2833318015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579d030 msgr2=0x5624c57a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:07.504 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff117fff700 1 --2- 172.21.15.44:0/2833318015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579d030 0x5624c57a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff117fff700 1 -- 172.21.15.44:0/2833318015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c57a6e40 msgr2=0x5624c57a9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:07.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.492+0000 7ff117fff700 1 --2- 172.21.15.44:0/2833318015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c57a6e40 0x5624c57a9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.493+0000 7ff117fff700 1 -- 172.21.15.44:0/2833318015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624c551ade0 con 0x5624c579c710 2023-09-10T00:44:07.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.493+0000 7ff117fff700 1 --2- 172.21.15.44:0/2833318015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579c710 0x5624c579caf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff1080ca5c0 tx=0x7ff108099d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38e1cb83fc6ba7c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:07.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.493+0000 7ff0fe7fc700 1 -- 172.21.15.44:0/2833318015 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10809aab0 con 0x5624c579c710 2023-09-10T00:44:07.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.493+0000 7ff0fe7fc700 1 -- 172.21.15.44:0/2833318015 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff10809a3b0 con 0x5624c579c710 2023-09-10T00:44:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.493+0000 7ff0fe7fc700 1 -- 172.21.15.44:0/2833318015 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108096ac0 con 0x5624c579c710 2023-09-10T00:44:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.494+0000 7ff12a82db80 1 -- 172.21.15.44:0/2833318015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579c710 msgr2=0x5624c579caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.494+0000 7ff12a82db80 1 --2- 172.21.15.44:0/2833318015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579c710 0x5624c579caf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff1080ca5c0 tx=0x7ff108099d10 comp rx=0 tx=0).stop 2023-09-10T00:44:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.494+0000 7ff12a82db80 1 -- 172.21.15.44:0/2833318015 shutdown_connections 2023-09-10T00:44:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.494+0000 7ff12a82db80 1 --2- 172.21.15.44:0/2833318015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c57a6e40 0x5624c57a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.494+0000 7ff12a82db80 1 --2- 172.21.15.44:0/2833318015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579c710 0x5624c579caf0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.494+0000 7ff12a82db80 1 --2- 172.21.15.44:0/2833318015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579d030 0x5624c57a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.494+0000 7ff12a82db80 1 -- 172.21.15.44:0/2833318015 >> 172.21.15.44:0/2833318015 conn(0x5624c5711090 msgr2=0x5624c5797bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.494+0000 7ff12a82db80 1 -- 172.21.15.44:0/2833318015 shutdown_connections 2023-09-10T00:44:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.494+0000 7ff12a82db80 1 -- 172.21.15.44:0/2833318015 wait complete. 2023-09-10T00:44:07.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.494+0000 7ff12a82db80 1 -- start start 2023-09-10T00:44:07.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.494+0000 7ff12a82db80 1 Processor -- start 2023-09-10T00:44:07.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff12a82db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579c710 0x5624c57a6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff12a82db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579d030 0x5624c57a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff12a82db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c57a6e40 0x5624c57a4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff12a82db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624c568cb70 con 0x5624c57a6e40 2023-09-10T00:44:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff12a82db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624c568c870 con 0x5624c579c710 2023-09-10T00:44:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff12a82db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624c5570920 con 0x5624c579d030 2023-09-10T00:44:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff12a82db80 10 client.0 _ll_register_callbacks cb 0x5624c55d6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:44:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff117fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579c710 0x5624c57a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff11cc04700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c57a6e40 0x5624c57a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.510 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff117fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579c710 0x5624c57a6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52302/0 (socket says 172.21.15.44:52302) 2023-09-10T00:44:07.510 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff11cc04700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c57a6e40 0x5624c57a4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50564/0 (socket says 172.21.15.44:50564) 2023-09-10T00:44:07.510 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff117fff700 1 -- 172.21.15.44:0/1505904500 learned_addr learned my addr 172.21.15.44:0/1505904500 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:07.510 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff1177fe700 1 --2- 172.21.15.44:0/1505904500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579d030 0x5624c57a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.511 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff117fff700 1 -- 172.21.15.44:0/1505904500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579d030 msgr2=0x5624c57a3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:07.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff117fff700 1 --2- 172.21.15.44:0/1505904500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579d030 0x5624c57a3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff117fff700 1 -- 172.21.15.44:0/1505904500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c57a6e40 msgr2=0x5624c57a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:07.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff117fff700 1 --2- 172.21.15.44:0/1505904500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c57a6e40 0x5624c57a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff117fff700 1 -- 172.21.15.44:0/1505904500 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1181a3050 con 0x5624c579c710 2023-09-10T00:44:07.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff117fff700 1 --2- 172.21.15.44:0/1505904500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579c710 0x5624c57a6820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff1080ca620 tx=0x7ff108005140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e695d42252e0767c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:07.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.495+0000 7ff0f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:44:07.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.496+0000 7ff0f77fe700 1 -- 172.21.15.44:0/1505904500 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10809aab0 con 0x5624c579c710 2023-09-10T00:44:07.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.496+0000 7ff12a82db80 1 -- 172.21.15.44:0/1505904500 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624c56464f0 con 0x5624c579c710 2023-09-10T00:44:07.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.496+0000 7ff0f77fe700 1 -- 172.21.15.44:0/1505904500 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ff10809e1e0 con 0x5624c579c710 2023-09-10T00:44:07.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.496+0000 7ff0f77fe700 1 -- 172.21.15.44:0/1505904500 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff108066120 con 0x5624c579c710 2023-09-10T00:44:07.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.496+0000 7ff12a82db80 10 client.26213 fetch_fsmap learned FSMap version 39 2023-09-10T00:44:07.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.496+0000 7ff12a82db80 1 -- 172.21.15.44:0/1505904500 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5624c5503c60 con 0x5624c579c710 2023-09-10T00:44:07.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.496+0000 7ff0f77fe700 1 -- 172.21.15.44:0/1505904500 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5624c56464f0 con 0x5624c579c710 2023-09-10T00:44:07.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.496+0000 7ff0f77fe700 1 -- 172.21.15.44:0/1505904500 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 39) v1 ==== 7910+0+0 (secure 0 0 0) 0x7ff10809a610 con 0x5624c579c710 2023-09-10T00:44:07.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.496+0000 7ff12a82db80 10 client.26213 fetch_fsmap finished waiting for FSMap version 39 2023-09-10T00:44:07.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.496+0000 7ff12a82db80 10 client.26213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:44:07.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.496+0000 7ff12a82db80 20 client.26213 populate_metadata read hostname 'smithi044' 2023-09-10T00:44:07.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.496+0000 7ff12a82db80 1 --2- 172.21.15.44:0/1505904500 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5624c57c70a0 0x5624c57c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.496+0000 7ff12a82db80 4 client.26213 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:44:07.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.496+0000 7ff12a82db80 1 -- 172.21.15.44:0/1505904500 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624c57038a0 con 0x5624c57c70a0 2023-09-10T00:44:07.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.497+0000 7ff1177fe700 1 --2- 172.21.15.44:0/1505904500 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5624c57c70a0 0x5624c57c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.497+0000 7ff1177fe700 1 --2- 172.21.15.44:0/1505904500 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5624c57c70a0 0x5624c57c9490 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a31586b32829c09 server_cookie=96c32aba734a2f99 in_seq=0 out_seq=0 2023-09-10T00:44:07.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.497+0000 7ff0f77fe700 10 client.26213 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:07.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.498+0000 7ff0f77fe700 1 -- 172.21.15.44:0/1505904500 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5624c57038a0 con 0x5624c57c70a0 2023-09-10T00:44:07.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.498+0000 7ff0f77fe700 10 client.26213 handle_command_reply: tid=0 2023-09-10T00:44:07.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.498+0000 7ff12a82db80 1 client.26213 shutdown 2023-09-10T00:44:07.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.498+0000 7ff12a82db80 1 -- 172.21.15.44:0/1505904500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579c710 msgr2=0x5624c57a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:07.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.498+0000 7ff12a82db80 1 --2- 172.21.15.44:0/1505904500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579c710 0x5624c57a6820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff1080ca620 tx=0x7ff108005140 comp rx=0 tx=0).stop 2023-09-10T00:44:07.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.498+0000 7ff12a82db80 1 -- 172.21.15.44:0/1505904500 shutdown_connections 2023-09-10T00:44:07.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.498+0000 7ff12a82db80 1 --2- 172.21.15.44:0/1505904500 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5624c57c70a0 0x5624c57c9490 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.517 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.498+0000 7ff12a82db80 1 --2- 172.21.15.44:0/1505904500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c57a6e40 0x5624c57a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.498+0000 7ff12a82db80 1 --2- 172.21.15.44:0/1505904500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579c710 0x5624c57a6820 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.498+0000 7ff0f77fe700 0 client.26213 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:07.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.498+0000 7ff12a82db80 1 --2- 172.21.15.44:0/1505904500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579d030 0x5624c57a3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.498+0000 7ff12a82db80 1 -- 172.21.15.44:0/1505904500 >> 172.21.15.44:0/1505904500 conn(0x5624c5711090 msgr2=0x5624c57977e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:07.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.498+0000 7ff12a82db80 1 -- 172.21.15.44:0/1505904500 shutdown_connections 2023-09-10T00:44:07.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.498+0000 7ff12a82db80 1 -- 172.21.15.44:0/1505904500 wait complete. 2023-09-10T00:44:07.519 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.498+0000 7ff12a82db80 20 client.26213 trim_cache size 0 max 16384 2023-09-10T00:44:07.524 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.515+0000 7ff12a82db80 1 Processor -- start 2023-09-10T00:44:07.524 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.515+0000 7ff12a82db80 1 -- start start 2023-09-10T00:44:07.525 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.515+0000 7ff12a82db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c579c710 0x5624c57a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.525 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.515+0000 7ff12a82db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579d030 0x5624c57a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.525 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.515+0000 7ff12a82db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c57a6e40 0x5624c57a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.525 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.515+0000 7ff12a82db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624c57986e0 con 0x5624c579c710 2023-09-10T00:44:07.525 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.515+0000 7ff12a82db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624c57943d0 con 0x5624c57a6e40 2023-09-10T00:44:07.526 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.515+0000 7ff12a82db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624c57a4600 con 0x5624c579d030 2023-09-10T00:44:07.526 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff117fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c579c710 0x5624c57a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.526 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff11cc04700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c57a6e40 0x5624c57a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.526 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff117fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c579c710 0x5624c57a6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50570/0 (socket says 172.21.15.44:50570) 2023-09-10T00:44:07.526 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff117fff700 1 -- 172.21.15.44:0/1252778786 learned_addr learned my addr 172.21.15.44:0/1252778786 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:07.527 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff11cc04700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c57a6e40 0x5624c57a11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52314/0 (socket says 172.21.15.44:52314) 2023-09-10T00:44:07.527 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff1177fe700 1 --2- 172.21.15.44:0/1252778786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579d030 0x5624c57a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.527 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff11cc04700 1 -- 172.21.15.44:0/1252778786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579d030 msgr2=0x5624c57a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:07.527 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff11cc04700 1 --2- 172.21.15.44:0/1252778786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579d030 0x5624c57a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.527 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff11cc04700 1 -- 172.21.15.44:0/1252778786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c579c710 msgr2=0x5624c57a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:07.527 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff11cc04700 1 --2- 172.21.15.44:0/1252778786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c579c710 0x5624c57a6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.528 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff11cc04700 1 -- 172.21.15.44:0/1252778786 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624c5541250 con 0x5624c57a6e40 2023-09-10T00:44:07.528 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff1177fe700 1 --2- 172.21.15.44:0/1252778786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579d030 0x5624c57a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:07.528 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff117fff700 1 --2- 172.21.15.44:0/1252778786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c579c710 0x5624c57a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:07.528 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff11cc04700 1 --2- 172.21.15.44:0/1252778786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c57a6e40 0x5624c57a11e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff11000e950 tx=0x7ff110009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fb8d4ee00649ed server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:07.528 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff0feffd700 1 -- 172.21.15.44:0/1252778786 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110004db0 con 0x5624c57a6e40 2023-09-10T00:44:07.529 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff0feffd700 1 -- 172.21.15.44:0/1252778786 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff11000f040 con 0x5624c57a6e40 2023-09-10T00:44:07.529 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.516+0000 7ff0feffd700 1 -- 172.21.15.44:0/1252778786 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110007880 con 0x5624c57a6e40 2023-09-10T00:44:07.529 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.517+0000 7ff12a82db80 1 -- 172.21.15.44:0/1252778786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c57a6e40 msgr2=0x5624c57a11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:07.529 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.517+0000 7ff12a82db80 1 --2- 172.21.15.44:0/1252778786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c57a6e40 0x5624c57a11e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff11000e950 tx=0x7ff110009920 comp rx=0 tx=0).stop 2023-09-10T00:44:07.529 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.517+0000 7ff12a82db80 1 -- 172.21.15.44:0/1252778786 shutdown_connections 2023-09-10T00:44:07.529 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.517+0000 7ff12a82db80 1 --2- 172.21.15.44:0/1252778786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c57a6e40 0x5624c57a11e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.517+0000 7ff12a82db80 1 --2- 172.21.15.44:0/1252778786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c579c710 0x5624c57a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.517+0000 7ff12a82db80 1 --2- 172.21.15.44:0/1252778786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c579d030 0x5624c57a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.517+0000 7ff12a82db80 1 -- 172.21.15.44:0/1252778786 >> 172.21.15.44:0/1252778786 conn(0x5624c5711090 msgr2=0x5624c579ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:07.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.517+0000 7ff12a82db80 1 -- 172.21.15.44:0/1252778786 shutdown_connections 2023-09-10T00:44:07.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.517+0000 7ff12a82db80 1 -- 172.21.15.44:0/1252778786 wait complete. 2023-09-10T00:44:07.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.518+0000 7ff12a82db80 1 -- start start 2023-09-10T00:44:07.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.518+0000 7ff12a82db80 1 Processor -- start 2023-09-10T00:44:07.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.518+0000 7ff12a82db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579c710 0x5624c57bd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.518+0000 7ff12a82db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c579d030 0x5624c57bd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.518+0000 7ff12a82db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c57a6e40 0x5624c581ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.518+0000 7ff12a82db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624c57bde30 con 0x5624c579d030 2023-09-10T00:44:07.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.518+0000 7ff12a82db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624c57bdf70 con 0x5624c579c710 2023-09-10T00:44:07.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.518+0000 7ff12a82db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624c581b350 con 0x5624c57a6e40 2023-09-10T00:44:07.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.518+0000 7ff12a82db80 10 client.0 _ll_register_callbacks cb 0x7ff110009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:44:07.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.518+0000 7ff1177fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c579d030 0x5624c57bd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.532 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.518+0000 7ff117fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579c710 0x5624c57bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.518+0000 7ff117fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579c710 0x5624c57bd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52316/0 (socket says 172.21.15.44:52316) 2023-09-10T00:44:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.518+0000 7ff1177fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c579d030 0x5624c57bd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50578/0 (socket says 172.21.15.44:50578) 2023-09-10T00:44:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.519+0000 7ff117fff700 1 -- 172.21.15.44:0/1553536782 learned_addr learned my addr 172.21.15.44:0/1553536782 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.519+0000 7ff11cc04700 1 --2- 172.21.15.44:0/1553536782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c57a6e40 0x5624c581ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.519+0000 7ff1177fe700 1 -- 172.21.15.44:0/1553536782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c57a6e40 msgr2=0x5624c581ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.519+0000 7ff1177fe700 1 --2- 172.21.15.44:0/1553536782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c57a6e40 0x5624c581ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.519+0000 7ff1177fe700 1 -- 172.21.15.44:0/1553536782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579c710 msgr2=0x5624c57bd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.519+0000 7ff1177fe700 1 --2- 172.21.15.44:0/1553536782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579c710 0x5624c57bd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.519+0000 7ff1177fe700 1 -- 172.21.15.44:0/1553536782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1181a3050 con 0x5624c579d030 2023-09-10T00:44:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.519+0000 7ff117fff700 1 --2- 172.21.15.44:0/1553536782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579c710 0x5624c57bd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.519+0000 7ff1177fe700 1 --2- 172.21.15.44:0/1553536782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c579d030 0x5624c57bd660 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff10c005ce0 tx=0x7ff10c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b9b0560ce9b544 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:07.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.519+0000 7ff0f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:44:07.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.519+0000 7ff0f77fe700 1 -- 172.21.15.44:0/1553536782 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c022040 con 0x5624c579d030 2023-09-10T00:44:07.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.519+0000 7ff12a82db80 1 -- 172.21.15.44:0/1553536782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff10809a3b0 con 0x5624c579d030 2023-09-10T00:44:07.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.519+0000 7ff0f77fe700 1 -- 172.21.15.44:0/1553536782 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ff10c00edd0 con 0x5624c579d030 2023-09-10T00:44:07.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.519+0000 7ff0f77fe700 1 -- 172.21.15.44:0/1553536782 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff10c02b0b0 con 0x5624c579d030 2023-09-10T00:44:07.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.519+0000 7ff12a82db80 10 client.16548 fetch_fsmap learned FSMap version 39 2023-09-10T00:44:07.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.520+0000 7ff12a82db80 1 -- 172.21.15.44:0/1553536782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7ff10c017070 con 0x5624c579d030 2023-09-10T00:44:07.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.520+0000 7ff0f77fe700 1 -- 172.21.15.44:0/1553536782 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff10809a3b0 con 0x5624c579d030 2023-09-10T00:44:07.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.520+0000 7ff0f77fe700 1 -- 172.21.15.44:0/1553536782 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) v1 ==== 7910+0+0 (secure 0 0 0) 0x7ff10c034b20 con 0x5624c579d030 2023-09-10T00:44:07.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.520+0000 7ff12a82db80 10 client.16548 fetch_fsmap finished waiting for FSMap version 39 2023-09-10T00:44:07.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.520+0000 7ff12a82db80 10 client.16548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:44:07.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.520+0000 7ff12a82db80 20 client.16548 populate_metadata read hostname 'smithi044' 2023-09-10T00:44:07.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.520+0000 7ff12a82db80 1 --2- 172.21.15.44:0/1553536782 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5624c581cbc0 0x5624c581efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:07.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.520+0000 7ff12a82db80 4 client.16548 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:44:07.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.520+0000 7ff12a82db80 1 -- 172.21.15.44:0/1553536782 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5624c56e7920 con 0x5624c581cbc0 2023-09-10T00:44:07.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.520+0000 7ff117fff700 1 --2- 172.21.15.44:0/1553536782 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5624c581cbc0 0x5624c581efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff117fff700 1 --2- 172.21.15.44:0/1553536782 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5624c581cbc0 0x5624c581efe0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fc0b05627f05343 server_cookie=6be399b72774cacf in_seq=0 out_seq=0 2023-09-10T00:44:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff0f77fe700 10 client.16548 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff0f77fe700 1 -- 172.21.15.44:0/1553536782 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5624c56e7920 con 0x5624c581cbc0 2023-09-10T00:44:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff0f77fe700 10 client.16548 handle_command_reply: tid=0 2023-09-10T00:44:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff12a82db80 1 client.16548 shutdown 2023-09-10T00:44:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff12a82db80 1 -- 172.21.15.44:0/1553536782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c579d030 msgr2=0x5624c57bd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff12a82db80 1 --2- 172.21.15.44:0/1553536782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c579d030 0x5624c57bd660 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff10c005ce0 tx=0x7ff10c0047c0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff12a82db80 1 -- 172.21.15.44:0/1553536782 shutdown_connections 2023-09-10T00:44:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff12a82db80 1 --2- 172.21.15.44:0/1553536782 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5624c581cbc0 0x5624c581efe0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff12a82db80 1 --2- 172.21.15.44:0/1553536782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624c57a6e40 0x5624c581ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff0f77fe700 0 client.16548 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:07.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff12a82db80 1 --2- 172.21.15.44:0/1553536782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624c579c710 0x5624c57bd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff12a82db80 1 --2- 172.21.15.44:0/1553536782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624c579d030 0x5624c57bd660 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff12a82db80 1 -- 172.21.15.44:0/1553536782 >> 172.21.15.44:0/1553536782 conn(0x5624c5711090 msgr2=0x5624c57a21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:07.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff12a82db80 1 -- 172.21.15.44:0/1553536782 shutdown_connections 2023-09-10T00:44:07.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff12a82db80 1 -- 172.21.15.44:0/1553536782 wait complete. 2023-09-10T00:44:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.521+0000 7ff12a82db80 20 client.16548 trim_cache size 0 max 16384 2023-09-10T00:44:07.541 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:44:07.541 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:44:07.542 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:44:07.542 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:44:07.542 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:44:07.542 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:44:07.542 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:44:07.543 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:44:07.543 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:44:07.543 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:44:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.522+0000 7ff11e667700 1 -- 172.21.15.44:0/1967114449 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1000a07b0 msgr2=0x7ff1000a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.522+0000 7ff11e667700 1 --2- 172.21.15.44:0/1967114449 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1000a07b0 0x7ff1000a2c70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff118069000 tx=0x7ff10c009310 comp rx=0 tx=0).stop 2023-09-10T00:44:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.522+0000 7ff11e667700 1 -- 172.21.15.44:0/1967114449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1180ff490 msgr2=0x7ff118197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.522+0000 7ff11e667700 1 --2- 172.21.15.44:0/1967114449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1180ff490 0x7ff118197380 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff108009810 tx=0x7ff10800bf50 comp rx=0 tx=0).stop 2023-09-10T00:44:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.522+0000 7ff11e667700 1 -- 172.21.15.44:0/1967114449 shutdown_connections 2023-09-10T00:44:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.522+0000 7ff11e667700 1 --2- 172.21.15.44:0/1967114449 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1000a07b0 0x7ff1000a2c70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.522+0000 7ff11e667700 1 --2- 172.21.15.44:0/1967114449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff118105550 0x7ff1181a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.522+0000 7ff11e667700 1 --2- 172.21.15.44:0/1967114449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1180ff490 0x7ff118197380 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.522+0000 7ff11e667700 1 --2- 172.21.15.44:0/1967114449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1180ffe50 0x7ff1181978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.522+0000 7ff11e667700 1 -- 172.21.15.44:0/1967114449 >> 172.21.15.44:0/1967114449 conn(0x7ff1180faf20 msgr2=0x7ff1180fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.522+0000 7ff11e667700 1 -- 172.21.15.44:0/1967114449 shutdown_connections 2023-09-10T00:44:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:07.522+0000 7ff11e667700 1 -- 172.21.15.44:0/1967114449 wait complete. 2023-09-10T00:44:07.547 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:44:07.938 INFO:teuthology.orchestra.run.smithi044.stderr:17900000 of 50000000 tuples (35%) done (elapsed 187.57 s, remaining 336.37 s) 2023-09-10T00:44:08.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:08 smithi099 ceph-mon[102692]: pgmap v107: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 28 MiB/s wr, 55 op/s 2023-09-10T00:44:09.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:08 smithi131 ceph-mon[103713]: pgmap v107: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 28 MiB/s wr, 55 op/s 2023-09-10T00:44:09.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:08 smithi044 ceph-mon[98676]: pgmap v107: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 28 MiB/s wr, 55 op/s 2023-09-10T00:44:10.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:10 smithi099 ceph-mon[102692]: pgmap v108: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 28 MiB/s wr, 55 op/s 2023-09-10T00:44:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:10 smithi131 ceph-mon[103713]: pgmap v108: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 28 MiB/s wr, 55 op/s 2023-09-10T00:44:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:10 smithi044 ceph-mon[98676]: pgmap v108: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 28 MiB/s wr, 55 op/s 2023-09-10T00:44:12.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:12 smithi099 ceph-mon[102692]: pgmap v109: 129 pgs: 129 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 34 MiB/s wr, 80 op/s 2023-09-10T00:44:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.881+0000 7f4c2c006700 1 -- 172.21.15.44:0/3459466076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c240ff690 msgr2=0x7f4c240ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.881+0000 7f4c2c006700 1 --2- 172.21.15.44:0/3459466076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c240ff690 0x7f4c240ffb10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009990 tx=0x7f4c14009ca0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.881+0000 7f4c2c006700 1 -- 172.21.15.44:0/3459466076 shutdown_connections 2023-09-10T00:44:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.881+0000 7f4c2c006700 1 --2- 172.21.15.44:0/3459466076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c24100050 0x7f4c2410c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.881+0000 7f4c2c006700 1 --2- 172.21.15.44:0/3459466076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c24105560 0x7f4c24105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.881+0000 7f4c2c006700 1 --2- 172.21.15.44:0/3459466076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c240ff690 0x7f4c240ffb10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.881+0000 7f4c2c006700 1 -- 172.21.15.44:0/3459466076 >> 172.21.15.44:0/3459466076 conn(0x7f4c240773d0 msgr2=0x7f4c240777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.881+0000 7f4c2c006700 1 -- 172.21.15.44:0/3459466076 shutdown_connections 2023-09-10T00:44:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.881+0000 7f4c2c006700 1 -- 172.21.15.44:0/3459466076 wait complete. 2023-09-10T00:44:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.882+0000 7f4c2c006700 1 Processor -- start 2023-09-10T00:44:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.882+0000 7f4c2c006700 1 -- start start 2023-09-10T00:44:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.882+0000 7f4c2c006700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c240ff690 0x7f4c241014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.882+0000 7f4c2c006700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c24100050 0x7f4c24101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.882+0000 7f4c2c006700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c24105560 0x7f4c241a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.882+0000 7f4c2c006700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c241a4c70 con 0x7f4c24105560 2023-09-10T00:44:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.882+0000 7f4c2c006700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c241a4db0 con 0x7f4c240ff690 2023-09-10T00:44:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.882+0000 7f4c2c006700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c241a4f20 con 0x7f4c24100050 2023-09-10T00:44:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.882+0000 7f4c2a5a3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c24105560 0x7f4c241a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.882+0000 7f4c29da2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c240ff690 0x7f4c241014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.882+0000 7f4c295a1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c24100050 0x7f4c24101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.882+0000 7f4c29da2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c240ff690 0x7f4c241014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52330/0 (socket says 172.21.15.44:52330) 2023-09-10T00:44:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.882+0000 7f4c29da2700 1 -- 172.21.15.44:0/1909289384 learned_addr learned my addr 172.21.15.44:0/1909289384 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.883+0000 7f4c295a1700 1 -- 172.21.15.44:0/1909289384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c240ff690 msgr2=0x7f4c241014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.883+0000 7f4c295a1700 1 --2- 172.21.15.44:0/1909289384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c240ff690 0x7f4c241014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.883+0000 7f4c295a1700 1 -- 172.21.15.44:0/1909289384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c24105560 msgr2=0x7f4c241a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.883+0000 7f4c295a1700 1 --2- 172.21.15.44:0/1909289384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c24105560 0x7f4c241a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.883+0000 7f4c295a1700 1 -- 172.21.15.44:0/1909289384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c009c40 con 0x7f4c24100050 2023-09-10T00:44:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.883+0000 7f4c295a1700 1 --2- 172.21.15.44:0/1909289384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c24100050 0x7f4c24101a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c140098e0 tx=0x7f4c14004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.883+0000 7f4c1affd700 1 -- 172.21.15.44:0/1909289384 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1401d070 con 0x7f4c24100050 2023-09-10T00:44:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.883+0000 7f4c1affd700 1 -- 172.21.15.44:0/1909289384 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c14021410 con 0x7f4c24100050 2023-09-10T00:44:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.883+0000 7f4c2c006700 1 -- 172.21.15.44:0/1909289384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c140095f0 con 0x7f4c24100050 2023-09-10T00:44:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.883+0000 7f4c1affd700 1 -- 172.21.15.44:0/1909289384 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c140173f0 con 0x7f4c24100050 2023-09-10T00:44:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.883+0000 7f4c2c006700 1 -- 172.21.15.44:0/1909289384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c241a5430 con 0x7f4c24100050 2023-09-10T00:44:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.885+0000 7f4c1affd700 1 -- 172.21.15.44:0/1909289384 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f4c1400fa90 con 0x7f4c24100050 2023-09-10T00:44:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.886+0000 7f4c1affd700 1 --2- 172.21.15.44:0/1909289384 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4c100a07b0 0x7f4c100a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.886+0000 7f4c29da2700 1 --2- 172.21.15.44:0/1909289384 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4c100a07b0 0x7f4c100a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.886+0000 7f4c1affd700 1 -- 172.21.15.44:0/1909289384 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f4c140cd400 con 0x7f4c24100050 2023-09-10T00:44:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.887+0000 7f4c29da2700 1 --2- 172.21.15.44:0/1909289384 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4c100a07b0 0x7f4c100a2c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009d20 tx=0x7f4c20009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.887+0000 7f4c381ccb80 1 Processor -- start 2023-09-10T00:44:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.887+0000 7f4c381ccb80 1 -- start start 2023-09-10T00:44:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.887+0000 7f4c381ccb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdba1a0 0x5588cbdba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.887+0000 7f4c381ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 0x5588cbdb4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.887+0000 7f4c381ccb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdb4cc0 0x5588cbdc0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.887+0000 7f4c381ccb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cbd0d6d0 con 0x5588cbdba1a0 2023-09-10T00:44:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.887+0000 7f4c381ccb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cbd0aff0 con 0x5588cbdb4cc0 2023-09-10T00:44:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.887+0000 7f4c381ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cbd068e0 con 0x5588cbdb4300 2023-09-10T00:44:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.888+0000 7f4c29da2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdba1a0 0x5588cbdba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.888+0000 7f4c2a5a3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdb4cc0 0x5588cbdc0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.888+0000 7f4c29da2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdba1a0 0x5588cbdba580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50598/0 (socket says 172.21.15.44:50598) 2023-09-10T00:44:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.888+0000 7f4c29da2700 1 -- 172.21.15.44:0/1205612163 learned_addr learned my addr 172.21.15.44:0/1205612163 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.888+0000 7f4c295a1700 1 --2- 172.21.15.44:0/1205612163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 0x5588cbdb4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.888+0000 7f4c29da2700 1 -- 172.21.15.44:0/1205612163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 msgr2=0x5588cbdb4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.888+0000 7f4c29da2700 1 --2- 172.21.15.44:0/1205612163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 0x5588cbdb4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.888+0000 7f4c29da2700 1 -- 172.21.15.44:0/1205612163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdb4cc0 msgr2=0x5588cbdc0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.888+0000 7f4c29da2700 1 --2- 172.21.15.44:0/1205612163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdb4cc0 0x5588cbdc0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.888+0000 7f4c29da2700 1 -- 172.21.15.44:0/1205612163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588cbb32d80 con 0x5588cbdba1a0 2023-09-10T00:44:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.888+0000 7f4c2a5a3700 1 --2- 172.21.15.44:0/1205612163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdb4cc0 0x5588cbdc0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.888+0000 7f4c29da2700 1 --2- 172.21.15.44:0/1205612163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdba1a0 0x5588cbdba580 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000e540 tx=0x7f4c20014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1a217b08b1671f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.888+0000 7f4c0ffff700 1 -- 172.21.15.44:0/1205612163 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20026430 con 0x5588cbdba1a0 2023-09-10T00:44:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.888+0000 7f4c0ffff700 1 -- 172.21.15.44:0/1205612163 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c20021070 con 0x5588cbdba1a0 2023-09-10T00:44:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.888+0000 7f4c0ffff700 1 -- 172.21.15.44:0/1205612163 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2001d670 con 0x5588cbdba1a0 2023-09-10T00:44:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.889+0000 7f4c381ccb80 1 -- 172.21.15.44:0/1205612163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdba1a0 msgr2=0x5588cbdba580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.889+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/1205612163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdba1a0 0x5588cbdba580 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000e540 tx=0x7f4c20014000 comp rx=0 tx=0).stop 2023-09-10T00:44:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.889+0000 7f4c381ccb80 1 -- 172.21.15.44:0/1205612163 shutdown_connections 2023-09-10T00:44:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.889+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/1205612163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdb4cc0 0x5588cbdc0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.889+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/1205612163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdba1a0 0x5588cbdba580 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.889+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/1205612163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 0x5588cbdb4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.889+0000 7f4c381ccb80 1 -- 172.21.15.44:0/1205612163 >> 172.21.15.44:0/1205612163 conn(0x5588cbd29030 msgr2=0x5588cbdaf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.889+0000 7f4c381ccb80 1 -- 172.21.15.44:0/1205612163 shutdown_connections 2023-09-10T00:44:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.889+0000 7f4c381ccb80 1 -- 172.21.15.44:0/1205612163 wait complete. 2023-09-10T00:44:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.889+0000 7f4c381ccb80 1 -- start start 2023-09-10T00:44:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.890+0000 7f4c381ccb80 1 Processor -- start 2023-09-10T00:44:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.890+0000 7f4c381ccb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdb4300 0x5588cbdc0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.890+0000 7f4c381ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4cc0 0x5588cbdbe130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.890+0000 7f4c381ccb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdba1a0 0x5588cbdbe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.890+0000 7f4c381ccb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cbca4b10 con 0x5588cbdb4300 2023-09-10T00:44:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.890+0000 7f4c381ccb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cbca4810 con 0x5588cbdba1a0 2023-09-10T00:44:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.890+0000 7f4c381ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cbb888c0 con 0x5588cbdb4cc0 2023-09-10T00:44:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.890+0000 7f4c381ccb80 10 client.0 _ll_register_callbacks cb 0x5588cbbeea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:44:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.890+0000 7f4c29da2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdb4300 0x5588cbdc0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.890+0000 7f4c29da2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdb4300 0x5588cbdc0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50604/0 (socket says 172.21.15.44:50604) 2023-09-10T00:44:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.890+0000 7f4c2a5a3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdba1a0 0x5588cbdbe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.890+0000 7f4c295a1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4cc0 0x5588cbdbe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.890+0000 7f4c29da2700 1 -- 172.21.15.44:0/188633478 learned_addr learned my addr 172.21.15.44:0/188633478 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.891+0000 7f4c2a5a3700 1 -- 172.21.15.44:0/188633478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4cc0 msgr2=0x5588cbdbe130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.891+0000 7f4c2a5a3700 1 --2- 172.21.15.44:0/188633478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4cc0 0x5588cbdbe130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.891+0000 7f4c2a5a3700 1 -- 172.21.15.44:0/188633478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdb4300 msgr2=0x5588cbdc0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.891+0000 7f4c2a5a3700 1 --2- 172.21.15.44:0/188633478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdb4300 0x5588cbdc0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.891+0000 7f4c2a5a3700 1 -- 172.21.15.44:0/188633478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c00e330 con 0x5588cbdba1a0 2023-09-10T00:44:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.891+0000 7f4c29da2700 1 --2- 172.21.15.44:0/188633478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdb4300 0x5588cbdc0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.891+0000 7f4c295a1700 1 --2- 172.21.15.44:0/188633478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4cc0 0x5588cbdbe130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.891+0000 7f4c2a5a3700 1 --2- 172.21.15.44:0/188633478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdba1a0 0x5588cbdbe760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00f010 tx=0x7f4c1c014650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b9c031950af4fd8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.891+0000 7f4c0d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:44:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.891+0000 7f4c0d7fa700 1 -- 172.21.15.44:0/188633478 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c00e040 con 0x5588cbdba1a0 2023-09-10T00:44:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.891+0000 7f4c381ccb80 1 -- 172.21.15.44:0/188633478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588cbc5e490 con 0x5588cbdba1a0 2023-09-10T00:44:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.891+0000 7f4c0d7fa700 1 -- 172.21.15.44:0/188633478 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f4c1c004db0 con 0x5588cbdba1a0 2023-09-10T00:44:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.891+0000 7f4c0d7fa700 1 -- 172.21.15.44:0/188633478 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c1c0090d0 con 0x5588cbdba1a0 2023-09-10T00:44:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.891+0000 7f4c381ccb80 10 client.26231 fetch_fsmap learned FSMap version 40 2023-09-10T00:44:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.891+0000 7f4c381ccb80 1 -- 172.21.15.44:0/188633478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f4c20017070 con 0x5588cbdba1a0 2023-09-10T00:44:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.891+0000 7f4c0d7fa700 1 -- 172.21.15.44:0/188633478 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5588cbc5e490 con 0x5588cbdba1a0 2023-09-10T00:44:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.892+0000 7f4c0d7fa700 1 -- 172.21.15.44:0/188633478 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f4c1c01c030 con 0x5588cbdba1a0 2023-09-10T00:44:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.892+0000 7f4c381ccb80 10 client.26231 fetch_fsmap finished waiting for FSMap version 40 2023-09-10T00:44:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.892+0000 7f4c381ccb80 10 client.26231 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:44:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.892+0000 7f4c381ccb80 20 client.26231 populate_metadata read hostname 'smithi044' 2023-09-10T00:44:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.892+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/188633478 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5588cbdce540 0x5588cbdd0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.892+0000 7f4c381ccb80 4 client.26231 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:44:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.892+0000 7f4c381ccb80 1 -- 172.21.15.44:0/188633478 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588cbd1b840 con 0x5588cbdce540 2023-09-10T00:44:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.892+0000 7f4c29da2700 1 --2- 172.21.15.44:0/188633478 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5588cbdce540 0x5588cbdd0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.892+0000 7f4c29da2700 1 --2- 172.21.15.44:0/188633478 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5588cbdce540 0x5588cbdd0960 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe4b8d022b02e134 server_cookie=2455f49e7d0679e3 in_seq=0 out_seq=0 2023-09-10T00:44:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.892+0000 7f4c0d7fa700 10 client.26231 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.893+0000 7f4c0d7fa700 1 -- 172.21.15.44:0/188633478 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5588cbd1b840 con 0x5588cbdce540 2023-09-10T00:44:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.893+0000 7f4c0d7fa700 10 client.26231 handle_command_reply: tid=0 2023-09-10T00:44:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.893+0000 7f4c381ccb80 1 client.26231 shutdown 2023-09-10T00:44:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.893+0000 7f4c381ccb80 1 -- 172.21.15.44:0/188633478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdba1a0 msgr2=0x5588cbdbe760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.893+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/188633478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdba1a0 0x5588cbdbe760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00f010 tx=0x7f4c1c014650 comp rx=0 tx=0).stop 2023-09-10T00:44:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.893+0000 7f4c381ccb80 1 -- 172.21.15.44:0/188633478 shutdown_connections 2023-09-10T00:44:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.893+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/188633478 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5588cbdce540 0x5588cbdd0960 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.893+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/188633478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdba1a0 0x5588cbdbe760 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.893+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/188633478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdb4300 0x5588cbdc0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.893+0000 7f4c0d7fa700 0 client.26231 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.893+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/188633478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4cc0 0x5588cbdbe130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.893+0000 7f4c381ccb80 1 -- 172.21.15.44:0/188633478 >> 172.21.15.44:0/188633478 conn(0x5588cbd29030 msgr2=0x5588cbdae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.893+0000 7f4c381ccb80 1 -- 172.21.15.44:0/188633478 shutdown_connections 2023-09-10T00:44:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.893+0000 7f4c381ccb80 1 -- 172.21.15.44:0/188633478 wait complete. 2023-09-10T00:44:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.893+0000 7f4c381ccb80 20 client.26231 trim_cache size 0 max 16384 2023-09-10T00:44:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.908+0000 7f4c381ccb80 1 Processor -- start 2023-09-10T00:44:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.908+0000 7f4c381ccb80 1 -- start start 2023-09-10T00:44:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.908+0000 7f4c381ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 0x5588cbdc5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.908+0000 7f4c381ccb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdb4cc0 0x5588cbdaef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.908+0000 7f4c381ccb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdba1a0 0x5588cbdaf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.908+0000 7f4c381ccb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cbdb55c0 con 0x5588cbdba1a0 2023-09-10T00:44:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.908+0000 7f4c381ccb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cbdb9e00 con 0x5588cbdb4cc0 2023-09-10T00:44:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.908+0000 7f4c381ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cbdabff0 con 0x5588cbdb4300 2023-09-10T00:44:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.909+0000 7f4c29da2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 0x5588cbdc5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.909+0000 7f4c2a5a3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdba1a0 0x5588cbdaf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.909+0000 7f4c295a1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdb4cc0 0x5588cbdaef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.909+0000 7f4c29da2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 0x5588cbdc5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57494/0 (socket says 172.21.15.44:57494) 2023-09-10T00:44:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.909+0000 7f4c29da2700 1 -- 172.21.15.44:0/2454430158 learned_addr learned my addr 172.21.15.44:0/2454430158 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.909+0000 7f4c295a1700 1 --2- 172.21.15.44:0/2454430158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdb4cc0 0x5588cbdaef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52354/0 (socket says 172.21.15.44:52354) 2023-09-10T00:44:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.909+0000 7f4c29da2700 1 -- 172.21.15.44:0/2454430158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdb4cc0 msgr2=0x5588cbdaef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.909+0000 7f4c29da2700 1 --2- 172.21.15.44:0/2454430158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdb4cc0 0x5588cbdaef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.909+0000 7f4c29da2700 1 -- 172.21.15.44:0/2454430158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdba1a0 msgr2=0x5588cbdaf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.909+0000 7f4c29da2700 1 --2- 172.21.15.44:0/2454430158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdba1a0 0x5588cbdaf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.909+0000 7f4c29da2700 1 -- 172.21.15.44:0/2454430158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588cbb1bc00 con 0x5588cbdb4300 2023-09-10T00:44:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.909+0000 7f4c2a5a3700 1 --2- 172.21.15.44:0/2454430158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdba1a0 0x5588cbdaf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.909+0000 7f4c29da2700 1 --2- 172.21.15.44:0/2454430158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 0x5588cbdc5530 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009be0 tx=0x7f4c2001cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26fd2349ce3b7b09 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.909+0000 7f4c18ff9700 1 -- 172.21.15.44:0/2454430158 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20009ea0 con 0x5588cbdb4300 2023-09-10T00:44:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.909+0000 7f4c18ff9700 1 -- 172.21.15.44:0/2454430158 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c2001cca0 con 0x5588cbdb4300 2023-09-10T00:44:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.909+0000 7f4c18ff9700 1 -- 172.21.15.44:0/2454430158 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2001c670 con 0x5588cbdb4300 2023-09-10T00:44:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.910+0000 7f4c381ccb80 1 -- 172.21.15.44:0/2454430158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 msgr2=0x5588cbdc5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.910+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/2454430158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 0x5588cbdc5530 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009be0 tx=0x7f4c2001cb90 comp rx=0 tx=0).stop 2023-09-10T00:44:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.910+0000 7f4c381ccb80 1 -- 172.21.15.44:0/2454430158 shutdown_connections 2023-09-10T00:44:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.910+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/2454430158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdba1a0 0x5588cbdaf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.910+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/2454430158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 0x5588cbdc5530 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.910+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/2454430158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdb4cc0 0x5588cbdaef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.910+0000 7f4c381ccb80 1 -- 172.21.15.44:0/2454430158 >> 172.21.15.44:0/2454430158 conn(0x5588cbd29030 msgr2=0x5588cbdb81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.910+0000 7f4c381ccb80 1 -- 172.21.15.44:0/2454430158 shutdown_connections 2023-09-10T00:44:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.910+0000 7f4c381ccb80 1 -- 172.21.15.44:0/2454430158 wait complete. 2023-09-10T00:44:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.910+0000 7f4c381ccb80 1 -- start start 2023-09-10T00:44:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.910+0000 7f4c381ccb80 1 Processor -- start 2023-09-10T00:44:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c381ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 0x5588cbe1a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c381ccb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdb4cc0 0x5588cbe1ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c381ccb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdba1a0 0x5588cbe1b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c381ccb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cbe1b960 con 0x5588cbdb4cc0 2023-09-10T00:44:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c381ccb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cbe1baa0 con 0x5588cbdba1a0 2023-09-10T00:44:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c381ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cbe1bc60 con 0x5588cbdb4300 2023-09-10T00:44:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c381ccb80 10 client.0 _ll_register_callbacks cb 0x5588cbbeea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:44:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c295a1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdb4cc0 0x5588cbe1ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c29da2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 0x5588cbe1a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c295a1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdb4cc0 0x5588cbe1ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50620/0 (socket says 172.21.15.44:50620) 2023-09-10T00:44:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c295a1700 1 -- 172.21.15.44:0/1637073601 learned_addr learned my addr 172.21.15.44:0/1637073601 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c2a5a3700 1 --2- 172.21.15.44:0/1637073601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdba1a0 0x5588cbe1b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c29da2700 1 -- 172.21.15.44:0/1637073601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdba1a0 msgr2=0x5588cbe1b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c29da2700 1 --2- 172.21.15.44:0/1637073601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdba1a0 0x5588cbe1b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c29da2700 1 -- 172.21.15.44:0/1637073601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdb4cc0 msgr2=0x5588cbe1ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c29da2700 1 --2- 172.21.15.44:0/1637073601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdb4cc0 0x5588cbe1ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c29da2700 1 -- 172.21.15.44:0/1637073601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c20017070 con 0x5588cbdb4300 2023-09-10T00:44:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.911+0000 7f4c295a1700 1 --2- 172.21.15.44:0/1637073601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdb4cc0 0x5588cbe1ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.912+0000 7f4c29da2700 1 --2- 172.21.15.44:0/1637073601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 0x5588cbe1a5f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4c2001cc70 tx=0x7f4c2000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=882fa0cfbd13924 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.912+0000 7f4c0d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:44:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.912+0000 7f4c0d7fa700 1 -- 172.21.15.44:0/1637073601 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20033b50 con 0x5588cbdb4300 2023-09-10T00:44:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.912+0000 7f4c381ccb80 1 -- 172.21.15.44:0/1637073601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c20021070 con 0x5588cbdb4300 2023-09-10T00:44:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.912+0000 7f4c0d7fa700 1 -- 172.21.15.44:0/1637073601 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f4c20033cb0 con 0x5588cbdb4300 2023-09-10T00:44:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.912+0000 7f4c0d7fa700 1 -- 172.21.15.44:0/1637073601 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c20033e70 con 0x5588cbdb4300 2023-09-10T00:44:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.912+0000 7f4c381ccb80 10 client.26173 fetch_fsmap learned FSMap version 40 2023-09-10T00:44:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.912+0000 7f4c381ccb80 1 -- 172.21.15.44:0/1637073601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f4c2002a040 con 0x5588cbdb4300 2023-09-10T00:44:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.912+0000 7f4c0d7fa700 1 -- 172.21.15.44:0/1637073601 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c20021070 con 0x5588cbdb4300 2023-09-10T00:44:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.912+0000 7f4c0d7fa700 1 -- 172.21.15.44:0/1637073601 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 40) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f4c2003a020 con 0x5588cbdb4300 2023-09-10T00:44:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.912+0000 7f4c381ccb80 10 client.26173 fetch_fsmap finished waiting for FSMap version 40 2023-09-10T00:44:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.912+0000 7f4c381ccb80 10 client.26173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:44:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.912+0000 7f4c381ccb80 20 client.26173 populate_metadata read hostname 'smithi044' 2023-09-10T00:44:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.912+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/1637073601 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5588cbe230c0 0x5588cbe254e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.912+0000 7f4c381ccb80 4 client.26173 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:44:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.912+0000 7f4c381ccb80 1 -- 172.21.15.44:0/1637073601 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5588cbcff8c0 con 0x5588cbe230c0 2023-09-10T00:44:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.913+0000 7f4c295a1700 1 --2- 172.21.15.44:0/1637073601 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5588cbe230c0 0x5588cbe254e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.913+0000 7f4c295a1700 1 --2- 172.21.15.44:0/1637073601 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5588cbe230c0 0x5588cbe254e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18e9f364eb2d6567 server_cookie=d6b0a09971406594 in_seq=0 out_seq=0 2023-09-10T00:44:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.913+0000 7f4c0d7fa700 10 client.26173 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.926+0000 7f4c0d7fa700 1 -- 172.21.15.44:0/1637073601 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5588cbcff8c0 con 0x5588cbe230c0 2023-09-10T00:44:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.926+0000 7f4c0d7fa700 10 client.26173 handle_command_reply: tid=0 2023-09-10T00:44:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.926+0000 7f4c381ccb80 1 client.26173 shutdown 2023-09-10T00:44:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.926+0000 7f4c381ccb80 1 -- 172.21.15.44:0/1637073601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 msgr2=0x5588cbe1a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.926+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/1637073601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 0x5588cbe1a5f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4c2001cc70 tx=0x7f4c2000edf0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.926+0000 7f4c381ccb80 1 -- 172.21.15.44:0/1637073601 shutdown_connections 2023-09-10T00:44:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.926+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/1637073601 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5588cbe230c0 0x5588cbe254e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.926+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/1637073601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588cbdba1a0 0x5588cbe1b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.926+0000 7f4c0d7fa700 0 client.26173 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.926+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/1637073601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588cbdb4300 0x5588cbe1a5f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.926+0000 7f4c381ccb80 1 --2- 172.21.15.44:0/1637073601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588cbdb4cc0 0x5588cbe1ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.926+0000 7f4c381ccb80 1 -- 172.21.15.44:0/1637073601 >> 172.21.15.44:0/1637073601 conn(0x5588cbd29030 msgr2=0x5588cbdb32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.926+0000 7f4c381ccb80 1 -- 172.21.15.44:0/1637073601 shutdown_connections 2023-09-10T00:44:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.926+0000 7f4c381ccb80 1 -- 172.21.15.44:0/1637073601 wait complete. 2023-09-10T00:44:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.927+0000 7f4c381ccb80 20 client.26173 trim_cache size 0 max 16384 2023-09-10T00:44:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:44:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:44:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:44:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:44:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.927+0000 7f4c2c006700 1 -- 172.21.15.44:0/1909289384 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4c100a07b0 msgr2=0x7f4c100a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.927+0000 7f4c2c006700 1 --2- 172.21.15.44:0/1909289384 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4c100a07b0 0x7f4c100a2c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009d20 tx=0x7f4c20009260 comp rx=0 tx=0).stop 2023-09-10T00:44:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.927+0000 7f4c2c006700 1 -- 172.21.15.44:0/1909289384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c24100050 msgr2=0x7f4c24101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.927+0000 7f4c2c006700 1 --2- 172.21.15.44:0/1909289384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c24100050 0x7f4c24101a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c140098e0 tx=0x7f4c14004730 comp rx=0 tx=0).stop 2023-09-10T00:44:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.927+0000 7f4c2c006700 1 -- 172.21.15.44:0/1909289384 shutdown_connections 2023-09-10T00:44:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.927+0000 7f4c2c006700 1 --2- 172.21.15.44:0/1909289384 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4c100a07b0 0x7f4c100a2c70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.927+0000 7f4c2c006700 1 --2- 172.21.15.44:0/1909289384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c24105560 0x7f4c241a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.927+0000 7f4c2c006700 1 --2- 172.21.15.44:0/1909289384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c240ff690 0x7f4c241014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.927+0000 7f4c2c006700 1 --2- 172.21.15.44:0/1909289384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c24100050 0x7f4c24101a10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.927+0000 7f4c2c006700 1 -- 172.21.15.44:0/1909289384 >> 172.21.15.44:0/1909289384 conn(0x7f4c240773d0 msgr2=0x7f4c24103490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.927+0000 7f4c2c006700 1 -- 172.21.15.44:0/1909289384 shutdown_connections 2023-09-10T00:44:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:12.927+0000 7f4c2c006700 1 -- 172.21.15.44:0/1909289384 wait complete. 2023-09-10T00:44:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:12 smithi131 ceph-mon[103713]: pgmap v109: 129 pgs: 129 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 34 MiB/s wr, 80 op/s 2023-09-10T00:44:13.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:12 smithi044 ceph-mon[98676]: pgmap v109: 129 pgs: 129 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 34 MiB/s wr, 80 op/s 2023-09-10T00:44:13.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:13 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:44:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:13 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:44:14.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:13 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:44:14.135 INFO:teuthology.orchestra.run.smithi044.stderr:18000000 of 50000000 tuples (36%) done (elapsed 193.77 s, remaining 344.48 s) 2023-09-10T00:44:14.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:14 smithi099 ceph-mon[102692]: pgmap v110: 129 pgs: 129 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 16 MiB/s wr, 66 op/s 2023-09-10T00:44:15.014 INFO:teuthology.orchestra.run.smithi044.stderr:18100000 of 50000000 tuples (36%) done (elapsed 194.65 s, remaining 343.05 s) 2023-09-10T00:44:15.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:14 smithi131 ceph-mon[103713]: pgmap v110: 129 pgs: 129 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 16 MiB/s wr, 66 op/s 2023-09-10T00:44:15.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:14 smithi044 ceph-mon[98676]: pgmap v110: 129 pgs: 129 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 16 MiB/s wr, 66 op/s 2023-09-10T00:44:16.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:16 smithi099 ceph-mon[102692]: pgmap v111: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 20 MiB/s wr, 79 op/s 2023-09-10T00:44:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:16 smithi131 ceph-mon[103713]: pgmap v111: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 20 MiB/s wr, 79 op/s 2023-09-10T00:44:17.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:16 smithi044 ceph-mon[98676]: pgmap v111: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 20 MiB/s wr, 79 op/s 2023-09-10T00:44:17.733 INFO:teuthology.orchestra.run.smithi044.stderr:18200000 of 50000000 tuples (36%) done (elapsed 197.37 s, remaining 344.85 s) 2023-09-10T00:44:18.598 INFO:teuthology.orchestra.run.smithi044.stderr:18300000 of 50000000 tuples (36%) done (elapsed 198.23 s, remaining 343.39 s) 2023-09-10T00:44:18.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:18 smithi099 ceph-mon[102692]: pgmap v112: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 9.4 MiB/s wr, 39 op/s 2023-09-10T00:44:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:18 smithi131 ceph-mon[103713]: pgmap v112: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 9.4 MiB/s wr, 39 op/s 2023-09-10T00:44:19.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:18 smithi044 ceph-mon[98676]: pgmap v112: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 9.4 MiB/s wr, 39 op/s 2023-09-10T00:44:19.541 INFO:teuthology.orchestra.run.smithi044.stderr:18400000 of 50000000 tuples (36%) done (elapsed 199.18 s, remaining 342.06 s) 2023-09-10T00:44:19.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:19 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:44:20.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:19 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:44:20.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:19 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:44:20.412 INFO:teuthology.orchestra.run.smithi044.stderr:18500000 of 50000000 tuples (37%) done (elapsed 200.05 s, remaining 340.62 s) 2023-09-10T00:44:20.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:20 smithi099 ceph-mon[102692]: pgmap v113: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 9.4 MiB/s wr, 39 op/s 2023-09-10T00:44:21.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:20 smithi131 ceph-mon[103713]: pgmap v113: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 9.4 MiB/s wr, 39 op/s 2023-09-10T00:44:21.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:20 smithi044 ceph-mon[98676]: pgmap v113: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 9.4 MiB/s wr, 39 op/s 2023-09-10T00:44:21.297 INFO:teuthology.orchestra.run.smithi044.stderr:18600000 of 50000000 tuples (37%) done (elapsed 200.93 s, remaining 339.21 s) 2023-09-10T00:44:22.280 INFO:teuthology.orchestra.run.smithi044.stderr:18700000 of 50000000 tuples (37%) done (elapsed 201.91 s, remaining 337.96 s) 2023-09-10T00:44:22.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:22 smithi099 ceph-mon[102692]: pgmap v114: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 20 MiB/s wr, 59 op/s 2023-09-10T00:44:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:22 smithi131 ceph-mon[103713]: pgmap v114: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 20 MiB/s wr, 59 op/s 2023-09-10T00:44:23.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:22 smithi044 ceph-mon[98676]: pgmap v114: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 20 MiB/s wr, 59 op/s 2023-09-10T00:44:23.143 INFO:teuthology.orchestra.run.smithi044.stderr:18800000 of 50000000 tuples (37%) done (elapsed 202.78 s, remaining 336.52 s) 2023-09-10T00:44:24.050 INFO:teuthology.orchestra.run.smithi044.stderr:18900000 of 50000000 tuples (37%) done (elapsed 203.68 s, remaining 335.16 s) 2023-09-10T00:44:25.000 INFO:teuthology.orchestra.run.smithi044.stderr:19000000 of 50000000 tuples (38%) done (elapsed 204.63 s, remaining 333.88 s) 2023-09-10T00:44:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:24 smithi131 ceph-mon[103713]: pgmap v115: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 14 MiB/s wr, 33 op/s 2023-09-10T00:44:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:24 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:44:25.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:24 smithi044 ceph-mon[98676]: pgmap v115: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 14 MiB/s wr, 33 op/s 2023-09-10T00:44:25.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:24 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:44:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:24 smithi099 ceph-mon[102692]: pgmap v115: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 14 MiB/s wr, 33 op/s 2023-09-10T00:44:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:24 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:44:26.001 INFO:teuthology.orchestra.run.smithi044.stderr:19100000 of 50000000 tuples (38%) done (elapsed 205.64 s, remaining 332.68 s) 2023-09-10T00:44:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.052+0000 7fe51382b700 1 -- 172.21.15.44:0/3632634489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe50c1055b0 msgr2=0x7fe50c105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.052+0000 7fe51382b700 1 --2- 172.21.15.44:0/3632634489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe50c1055b0 0x7fe50c105990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe5080099c0 tx=0x7fe508009cd0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.052+0000 7fe51382b700 1 -- 172.21.15.44:0/3632634489 shutdown_connections 2023-09-10T00:44:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.052+0000 7fe51382b700 1 --2- 172.21.15.44:0/3632634489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe50c100120 0x7fe50c10c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.052+0000 7fe51382b700 1 --2- 172.21.15.44:0/3632634489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe50c1055b0 0x7fe50c105990 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.052+0000 7fe51382b700 1 --2- 172.21.15.44:0/3632634489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe50c0ff760 0x7fe50c0ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.052+0000 7fe51382b700 1 -- 172.21.15.44:0/3632634489 >> 172.21.15.44:0/3632634489 conn(0x7fe50c074f30 msgr2=0x7fe50c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.053+0000 7fe51382b700 1 -- 172.21.15.44:0/3632634489 shutdown_connections 2023-09-10T00:44:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.053+0000 7fe51382b700 1 -- 172.21.15.44:0/3632634489 wait complete. 2023-09-10T00:44:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.053+0000 7fe51382b700 1 Processor -- start 2023-09-10T00:44:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.053+0000 7fe51382b700 1 -- start start 2023-09-10T00:44:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.053+0000 7fe51382b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe50c0ff760 0x7fe50c197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.053+0000 7fe51382b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe50c100120 0x7fe50c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.053+0000 7fe51382b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe50c1055b0 0x7fe50c1a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.053+0000 7fe51382b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c1a29c0 con 0x7fe50c1055b0 2023-09-10T00:44:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.053+0000 7fe51382b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c1a2b30 con 0x7fe50c100120 2023-09-10T00:44:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.053+0000 7fe51382b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c1a2cf0 con 0x7fe50c0ff760 2023-09-10T00:44:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.054+0000 7fe5115c7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe50c0ff760 0x7fe50c197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.054+0000 7fe510dc6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe50c100120 0x7fe50c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.054+0000 7fe511dc8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe50c1055b0 0x7fe50c1a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.054+0000 7fe5115c7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe50c0ff760 0x7fe50c197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57514/0 (socket says 172.21.15.44:57514) 2023-09-10T00:44:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.054+0000 7fe510dc6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe50c100120 0x7fe50c197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52374/0 (socket says 172.21.15.44:52374) 2023-09-10T00:44:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.054+0000 7fe5115c7700 1 -- 172.21.15.44:0/929414414 learned_addr learned my addr 172.21.15.44:0/929414414 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.054+0000 7fe5115c7700 1 -- 172.21.15.44:0/929414414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe50c100120 msgr2=0x7fe50c197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.054+0000 7fe5115c7700 1 --2- 172.21.15.44:0/929414414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe50c100120 0x7fe50c197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.054+0000 7fe5115c7700 1 -- 172.21.15.44:0/929414414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe50c1055b0 msgr2=0x7fe50c1a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.054+0000 7fe5115c7700 1 --2- 172.21.15.44:0/929414414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe50c1055b0 0x7fe50c1a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.054+0000 7fe5115c7700 1 -- 172.21.15.44:0/929414414 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5080095f0 con 0x7fe50c0ff760 2023-09-10T00:44:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.054+0000 7fe5115c7700 1 --2- 172.21.15.44:0/929414414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe50c0ff760 0x7fe50c197550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe508005910 tx=0x7fe508009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.054+0000 7fe5027fc700 1 -- 172.21.15.44:0/929414414 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50801d070 con 0x7fe50c0ff760 2023-09-10T00:44:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.054+0000 7fe51382b700 1 -- 172.21.15.44:0/929414414 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe50c195220 con 0x7fe50c0ff760 2023-09-10T00:44:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.054+0000 7fe5027fc700 1 -- 172.21.15.44:0/929414414 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe508022470 con 0x7fe50c0ff760 2023-09-10T00:44:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.054+0000 7fe5027fc700 1 -- 172.21.15.44:0/929414414 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508017650 con 0x7fe50c0ff760 2023-09-10T00:44:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.055+0000 7fe51382b700 1 -- 172.21.15.44:0/929414414 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe50c1956d0 con 0x7fe50c0ff760 2023-09-10T00:44:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.057+0000 7fe5027fc700 1 -- 172.21.15.44:0/929414414 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fe5080225e0 con 0x7fe50c0ff760 2023-09-10T00:44:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.058+0000 7fe5027fc700 1 --2- 172.21.15.44:0/929414414 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4f80a07b0 0x7fe4f80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.058+0000 7fe510dc6700 1 --2- 172.21.15.44:0/929414414 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4f80a07b0 0x7fe4f80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.058+0000 7fe5027fc700 1 -- 172.21.15.44:0/929414414 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe5080cd390 con 0x7fe50c0ff760 2023-09-10T00:44:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.058+0000 7fe51f9f1b80 1 Processor -- start 2023-09-10T00:44:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.058+0000 7fe51f9f1b80 1 -- start start 2023-09-10T00:44:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.058+0000 7fe510dc6700 1 --2- 172.21.15.44:0/929414414 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4f80a07b0 0x7fe4f80a2c70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe50c068ad0 tx=0x7fe4fc009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe51f9f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde3d1a0 0x5641fde3d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe51f9f1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde37300 0x5641fde37780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe51f9f1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde43f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe51f9f1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641fdd906d0 con 0x5641fde37cc0 2023-09-10T00:44:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe51f9f1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641fdd8dff0 con 0x5641fde37300 2023-09-10T00:44:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe51f9f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641fdd898e0 con 0x5641fde3d1a0 2023-09-10T00:44:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe5115c7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde3d1a0 0x5641fde3d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe511dc8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde43f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe510dc6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde37300 0x5641fde37780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe5115c7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde3d1a0 0x5641fde3d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57522/0 (socket says 172.21.15.44:57522) 2023-09-10T00:44:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe5115c7700 1 -- 172.21.15.44:0/3682552805 learned_addr learned my addr 172.21.15.44:0/3682552805 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe511dc8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde43f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50644/0 (socket says 172.21.15.44:50644) 2023-09-10T00:44:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe510dc6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde37300 0x5641fde37780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52382/0 (socket says 172.21.15.44:52382) 2023-09-10T00:44:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe5115c7700 1 -- 172.21.15.44:0/3682552805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde37300 msgr2=0x5641fde37780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe5115c7700 1 --2- 172.21.15.44:0/3682552805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde37300 0x5641fde37780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe5115c7700 1 -- 172.21.15.44:0/3682552805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 msgr2=0x5641fde43f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe5115c7700 1 --2- 172.21.15.44:0/3682552805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde43f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe5115c7700 1 -- 172.21.15.44:0/3682552805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641fdbb5d80 con 0x5641fde3d1a0 2023-09-10T00:44:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe511dc8700 1 --2- 172.21.15.44:0/3682552805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde43f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.059+0000 7fe5115c7700 1 --2- 172.21.15.44:0/3682552805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde3d1a0 0x5641fde3d580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe5080a1fe0 tx=0x7fe50809a070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca28b40b43e90c08 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.060+0000 7fe4f77fe700 1 -- 172.21.15.44:0/3682552805 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50809a530 con 0x5641fde3d1a0 2023-09-10T00:44:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.060+0000 7fe4f77fe700 1 -- 172.21.15.44:0/3682552805 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5080cae90 con 0x5641fde3d1a0 2023-09-10T00:44:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.060+0000 7fe4f77fe700 1 -- 172.21.15.44:0/3682552805 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5080228c0 con 0x5641fde3d1a0 2023-09-10T00:44:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.060+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/3682552805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde3d1a0 msgr2=0x5641fde3d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.060+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/3682552805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde3d1a0 0x5641fde3d580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe5080a1fe0 tx=0x7fe50809a070 comp rx=0 tx=0).stop 2023-09-10T00:44:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.060+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/3682552805 shutdown_connections 2023-09-10T00:44:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.060+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/3682552805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde43f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.060+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/3682552805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde3d1a0 0x5641fde3d580 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.060+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/3682552805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde37300 0x5641fde37780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.060+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/3682552805 >> 172.21.15.44:0/3682552805 conn(0x5641fddac030 msgr2=0x5641fde32560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.060+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/3682552805 shutdown_connections 2023-09-10T00:44:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.060+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/3682552805 wait complete. 2023-09-10T00:44:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.061+0000 7fe51f9f1b80 1 -- start start 2023-09-10T00:44:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.061+0000 7fe51f9f1b80 1 Processor -- start 2023-09-10T00:44:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe51f9f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde37300 0x5641fde46b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe51f9f1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde47090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe51f9f1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde3d1a0 0x5641fde40de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe51f9f1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641fdd27b10 con 0x5641fde37cc0 2023-09-10T00:44:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe51f9f1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641fdd27810 con 0x5641fde3d1a0 2023-09-10T00:44:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe51f9f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641fdc0b8c0 con 0x5641fde37300 2023-09-10T00:44:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe51f9f1b80 10 client.0 _ll_register_callbacks cb 0x5641fdc71a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:44:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe5115c7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde37300 0x5641fde46b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe5115c7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde37300 0x5641fde46b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57528/0 (socket says 172.21.15.44:57528) 2023-09-10T00:44:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe5115c7700 1 -- 172.21.15.44:0/978033636 learned_addr learned my addr 172.21.15.44:0/978033636 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe511dc8700 1 --2- 172.21.15.44:0/978033636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde3d1a0 0x5641fde40de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe510dc6700 1 --2- 172.21.15.44:0/978033636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde47090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe5115c7700 1 -- 172.21.15.44:0/978033636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde3d1a0 msgr2=0x5641fde40de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe5115c7700 1 --2- 172.21.15.44:0/978033636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde3d1a0 0x5641fde40de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe5115c7700 1 -- 172.21.15.44:0/978033636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 msgr2=0x5641fde47090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe5115c7700 1 --2- 172.21.15.44:0/978033636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde47090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe5115c7700 1 -- 172.21.15.44:0/978033636 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe504011230 con 0x5641fde37300 2023-09-10T00:44:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe510dc6700 1 --2- 172.21.15.44:0/978033636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde47090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:44:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe5115c7700 1 --2- 172.21.15.44:0/978033636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde37300 0x5641fde46b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe508096a30 tx=0x7fe5080092b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdc6074fb0fedcfa server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe4f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:44:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe4f4ff9700 1 -- 172.21.15.44:0/978033636 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508022ac0 con 0x5641fde37300 2023-09-10T00:44:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe4f4ff9700 1 -- 172.21.15.44:0/978033636 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe50809a530 con 0x5641fde37300 2023-09-10T00:44:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.062+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/978033636 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641fdce1490 con 0x5641fde37300 2023-09-10T00:44:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.063+0000 7fe4f4ff9700 1 -- 172.21.15.44:0/978033636 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe50809e170 con 0x5641fde37300 2023-09-10T00:44:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.063+0000 7fe51f9f1b80 10 client.26185 fetch_fsmap learned FSMap version 42 2023-09-10T00:44:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.063+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/978033636 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fe50c1956d0 con 0x5641fde37300 2023-09-10T00:44:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.063+0000 7fe4f4ff9700 1 -- 172.21.15.44:0/978033636 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5641fdce1490 con 0x5641fde37300 2023-09-10T00:44:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.063+0000 7fe4f4ff9700 1 -- 172.21.15.44:0/978033636 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe508099ab0 con 0x5641fde37300 2023-09-10T00:44:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.063+0000 7fe51f9f1b80 10 client.26185 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:44:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.063+0000 7fe51f9f1b80 10 client.26185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:44:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.063+0000 7fe51f9f1b80 20 client.26185 populate_metadata read hostname 'smithi044' 2023-09-10T00:44:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.063+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/978033636 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5641fde50b10 0x5641fde52f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.063+0000 7fe51f9f1b80 4 client.26185 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:44:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.063+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/978033636 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641fdd9e840 con 0x5641fde50b10 2023-09-10T00:44:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.063+0000 7fe510dc6700 1 --2- 172.21.15.44:0/978033636 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5641fde50b10 0x5641fde52f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.064+0000 7fe4f4ff9700 10 client.26185 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.064+0000 7fe510dc6700 1 --2- 172.21.15.44:0/978033636 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5641fde50b10 0x5641fde52f00 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14f9a97cd1e64c4d server_cookie=cb28af3b0b7138ef in_seq=0 out_seq=0 2023-09-10T00:44:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.064+0000 7fe4f4ff9700 1 -- 172.21.15.44:0/978033636 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5641fdd9e840 con 0x5641fde50b10 2023-09-10T00:44:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.064+0000 7fe4f4ff9700 10 client.26185 handle_command_reply: tid=0 2023-09-10T00:44:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.064+0000 7fe51f9f1b80 1 client.26185 shutdown 2023-09-10T00:44:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.064+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/978033636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde37300 msgr2=0x5641fde46b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.064+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/978033636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde37300 0x5641fde46b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe508096a30 tx=0x7fe5080092b0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.065+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/978033636 shutdown_connections 2023-09-10T00:44:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.065+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/978033636 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5641fde50b10 0x5641fde52f00 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.065+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/978033636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde3d1a0 0x5641fde40de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.065+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/978033636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde37300 0x5641fde46b50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.065+0000 7fe4f4ff9700 0 client.26185 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.065+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/978033636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde47090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.065+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/978033636 >> 172.21.15.44:0/978033636 conn(0x5641fddac030 msgr2=0x5641fde317a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.065+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/978033636 shutdown_connections 2023-09-10T00:44:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.065+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/978033636 wait complete. 2023-09-10T00:44:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.065+0000 7fe51f9f1b80 20 client.26185 trim_cache size 0 max 16384 2023-09-10T00:44:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.081+0000 7fe51f9f1b80 1 Processor -- start 2023-09-10T00:44:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.081+0000 7fe51f9f1b80 1 -- start start 2023-09-10T00:44:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.081+0000 7fe51f9f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde37300 0x5641fde32710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.081+0000 7fe51f9f1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde3c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.081+0000 7fe51f9f1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde3d1a0 0x5641fde46720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.081+0000 7fe51f9f1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641fde330d0 con 0x5641fde37cc0 2023-09-10T00:44:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.081+0000 7fe51f9f1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641fde3d970 con 0x5641fde3d1a0 2023-09-10T00:44:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.081+0000 7fe51f9f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641fde3d580 con 0x5641fde37300 2023-09-10T00:44:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.082+0000 7fe510dc6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde3c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.082+0000 7fe5115c7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde37300 0x5641fde32710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.082+0000 7fe510dc6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde3c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50656/0 (socket says 172.21.15.44:50656) 2023-09-10T00:44:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.082+0000 7fe510dc6700 1 -- 172.21.15.44:0/2439861846 learned_addr learned my addr 172.21.15.44:0/2439861846 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.082+0000 7fe511dc8700 1 --2- 172.21.15.44:0/2439861846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde3d1a0 0x5641fde46720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.082+0000 7fe5115c7700 1 -- 172.21.15.44:0/2439861846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde3d1a0 msgr2=0x5641fde46720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.082+0000 7fe5115c7700 1 --2- 172.21.15.44:0/2439861846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde3d1a0 0x5641fde46720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.082+0000 7fe5115c7700 1 -- 172.21.15.44:0/2439861846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 msgr2=0x5641fde3c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.082+0000 7fe5115c7700 1 --2- 172.21.15.44:0/2439861846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde3c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.082+0000 7fe5115c7700 1 -- 172.21.15.44:0/2439861846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641fdb9ec00 con 0x5641fde37300 2023-09-10T00:44:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.082+0000 7fe5115c7700 1 --2- 172.21.15.44:0/2439861846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde37300 0x5641fde32710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe508098880 tx=0x7fe508099f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f16e977b618463b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.082+0000 7fe4f7fff700 1 -- 172.21.15.44:0/2439861846 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508022ac0 con 0x5641fde37300 2023-09-10T00:44:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.082+0000 7fe4f7fff700 1 -- 172.21.15.44:0/2439861846 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe508096050 con 0x5641fde37300 2023-09-10T00:44:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.082+0000 7fe4f7fff700 1 -- 172.21.15.44:0/2439861846 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50809a9f0 con 0x5641fde37300 2023-09-10T00:44:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.083+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/2439861846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde37300 msgr2=0x5641fde32710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.083+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/2439861846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde37300 0x5641fde32710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe508098880 tx=0x7fe508099f70 comp rx=0 tx=0).stop 2023-09-10T00:44:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.083+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/2439861846 shutdown_connections 2023-09-10T00:44:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.083+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/2439861846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde3d1a0 0x5641fde46720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.083+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/2439861846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde37300 0x5641fde32710 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.083+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/2439861846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde3c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.083+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/2439861846 >> 172.21.15.44:0/2439861846 conn(0x5641fddac030 msgr2=0x5641fde3c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.083+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/2439861846 shutdown_connections 2023-09-10T00:44:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.083+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/2439861846 wait complete. 2023-09-10T00:44:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.083+0000 7fe51f9f1b80 1 -- start start 2023-09-10T00:44:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe51f9f1b80 1 Processor -- start 2023-09-10T00:44:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe51f9f1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde37300 0x5641fde9de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe51f9f1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde9e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe51f9f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde3d1a0 0x5641fde98830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe51f9f1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641fde98f40 con 0x5641fde37cc0 2023-09-10T00:44:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe51f9f1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641fde990b0 con 0x5641fde37300 2023-09-10T00:44:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe51f9f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641fde99270 con 0x5641fde3d1a0 2023-09-10T00:44:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe51f9f1b80 10 client.0 _ll_register_callbacks cb 0x7fe504009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:44:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe510dc6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde9e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe5115c7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde37300 0x5641fde9de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe511dc8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde3d1a0 0x5641fde98830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe5115c7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde37300 0x5641fde9de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52400/0 (socket says 172.21.15.44:52400) 2023-09-10T00:44:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe5115c7700 1 -- 172.21.15.44:0/3655329255 learned_addr learned my addr 172.21.15.44:0/3655329255 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe510dc6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde9e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50662/0 (socket says 172.21.15.44:50662) 2023-09-10T00:44:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe511dc8700 1 -- 172.21.15.44:0/3655329255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde37300 msgr2=0x5641fde9de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe511dc8700 1 --2- 172.21.15.44:0/3655329255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde37300 0x5641fde9de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.084+0000 7fe511dc8700 1 -- 172.21.15.44:0/3655329255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 msgr2=0x5641fde9e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe511dc8700 1 --2- 172.21.15.44:0/3655329255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde9e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe511dc8700 1 -- 172.21.15.44:0/3655329255 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe50c1956d0 con 0x5641fde3d1a0 2023-09-10T00:44:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe510dc6700 1 --2- 172.21.15.44:0/3655329255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde9e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe511dc8700 1 --2- 172.21.15.44:0/3655329255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde3d1a0 0x5641fde98830 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x5641fde3dc40 tx=0x7fe504004910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca68618dec1dcdb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe4f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:44:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe4f4ff9700 1 -- 172.21.15.44:0/3655329255 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50401c070 con 0x5641fde3d1a0 2023-09-10T00:44:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/3655329255 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5080cae90 con 0x5641fde3d1a0 2023-09-10T00:44:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe4f4ff9700 1 -- 172.21.15.44:0/3655329255 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe504004bb0 con 0x5641fde3d1a0 2023-09-10T00:44:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe4f4ff9700 1 -- 172.21.15.44:0/3655329255 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe50400e040 con 0x5641fde3d1a0 2023-09-10T00:44:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe51f9f1b80 10 client.26191 fetch_fsmap learned FSMap version 42 2023-09-10T00:44:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/3655329255 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5641fdbdc1f0 con 0x5641fde3d1a0 2023-09-10T00:44:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe4f4ff9700 1 -- 172.21.15.44:0/3655329255 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5080cae90 con 0x5641fde3d1a0 2023-09-10T00:44:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe4f4ff9700 1 -- 172.21.15.44:0/3655329255 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe5040206b0 con 0x5641fde3d1a0 2023-09-10T00:44:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe51f9f1b80 10 client.26191 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:44:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe51f9f1b80 10 client.26191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:44:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe51f9f1b80 20 client.26191 populate_metadata read hostname 'smithi044' 2023-09-10T00:44:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/3655329255 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5641fdea5cb0 0x5641fdea80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe51f9f1b80 4 client.26191 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:44:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.085+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/3655329255 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5641fdd828c0 con 0x5641fdea5cb0 2023-09-10T00:44:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.086+0000 7fe5115c7700 1 --2- 172.21.15.44:0/3655329255 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5641fdea5cb0 0x5641fdea80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.086+0000 7fe4f4ff9700 10 client.26191 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.086+0000 7fe5115c7700 1 --2- 172.21.15.44:0/3655329255 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5641fdea5cb0 0x5641fdea80d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa58b0653feac0c5 server_cookie=6b7d972f56acb8bc in_seq=0 out_seq=0 2023-09-10T00:44:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.097+0000 7fe4f4ff9700 1 -- 172.21.15.44:0/3655329255 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5641fdd828c0 con 0x5641fdea5cb0 2023-09-10T00:44:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.097+0000 7fe4f4ff9700 10 client.26191 handle_command_reply: tid=0 2023-09-10T00:44:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.097+0000 7fe51f9f1b80 1 client.26191 shutdown 2023-09-10T00:44:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.097+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/3655329255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde3d1a0 msgr2=0x5641fde98830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.097+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/3655329255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde3d1a0 0x5641fde98830 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x5641fde3dc40 tx=0x7fe504004910 comp rx=0 tx=0).stop 2023-09-10T00:44:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.097+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/3655329255 shutdown_connections 2023-09-10T00:44:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/3655329255 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5641fdea5cb0 0x5641fdea80d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/3655329255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641fde3d1a0 0x5641fde98830 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe4f4ff9700 0 client.26191 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/3655329255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641fde37300 0x5641fde9de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51f9f1b80 1 --2- 172.21.15.44:0/3655329255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641fde37cc0 0x5641fde9e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/3655329255 >> 172.21.15.44:0/3655329255 conn(0x5641fddac030 msgr2=0x5641fde40d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/3655329255 shutdown_connections 2023-09-10T00:44:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51f9f1b80 1 -- 172.21.15.44:0/3655329255 wait complete. 2023-09-10T00:44:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51f9f1b80 20 client.26191 trim_cache size 0 max 16384 2023-09-10T00:44:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:44:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:44:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:44:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:44:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51382b700 1 -- 172.21.15.44:0/929414414 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4f80a07b0 msgr2=0x7fe4f80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51382b700 1 --2- 172.21.15.44:0/929414414 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4f80a07b0 0x7fe4f80a2c70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe50c068ad0 tx=0x7fe4fc009260 comp rx=0 tx=0).stop 2023-09-10T00:44:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51382b700 1 -- 172.21.15.44:0/929414414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe50c0ff760 msgr2=0x7fe50c197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51382b700 1 --2- 172.21.15.44:0/929414414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe50c0ff760 0x7fe50c197550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe508005910 tx=0x7fe508009580 comp rx=0 tx=0).stop 2023-09-10T00:44:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51382b700 1 -- 172.21.15.44:0/929414414 shutdown_connections 2023-09-10T00:44:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51382b700 1 --2- 172.21.15.44:0/929414414 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4f80a07b0 0x7fe4f80a2c70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51382b700 1 --2- 172.21.15.44:0/929414414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe50c1055b0 0x7fe50c1a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51382b700 1 --2- 172.21.15.44:0/929414414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe50c0ff760 0x7fe50c197550 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51382b700 1 --2- 172.21.15.44:0/929414414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe50c100120 0x7fe50c197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.098+0000 7fe51382b700 1 -- 172.21.15.44:0/929414414 >> 172.21.15.44:0/929414414 conn(0x7fe50c074f30 msgr2=0x7fe50c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.099+0000 7fe51382b700 1 -- 172.21.15.44:0/929414414 shutdown_connections 2023-09-10T00:44:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:26.099+0000 7fe51382b700 1 -- 172.21.15.44:0/929414414 wait complete. 2023-09-10T00:44:26.966 INFO:teuthology.orchestra.run.smithi044.stderr:19200000 of 50000000 tuples (38%) done (elapsed 206.60 s, remaining 331.42 s) 2023-09-10T00:44:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:26 smithi131 ceph-mon[103713]: pgmap v116: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 30 MiB/s wr, 67 op/s 2023-09-10T00:44:27.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:26 smithi044 ceph-mon[98676]: pgmap v116: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 30 MiB/s wr, 67 op/s 2023-09-10T00:44:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:26 smithi099 ceph-mon[102692]: pgmap v116: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 30 MiB/s wr, 67 op/s 2023-09-10T00:44:27.880 INFO:teuthology.orchestra.run.smithi044.stderr:19300000 of 50000000 tuples (38%) done (elapsed 207.51 s, remaining 330.09 s) 2023-09-10T00:44:28.715 INFO:teuthology.orchestra.run.smithi044.stderr:19400000 of 50000000 tuples (38%) done (elapsed 208.35 s, remaining 328.63 s) 2023-09-10T00:44:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:28 smithi131 ceph-mon[103713]: pgmap v117: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 27 MiB/s wr, 54 op/s 2023-09-10T00:44:29.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:28 smithi044 ceph-mon[98676]: pgmap v117: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 27 MiB/s wr, 54 op/s 2023-09-10T00:44:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:28 smithi099 ceph-mon[102692]: pgmap v117: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 27 MiB/s wr, 54 op/s 2023-09-10T00:44:29.612 INFO:teuthology.orchestra.run.smithi044.stderr:19500000 of 50000000 tuples (39%) done (elapsed 209.25 s, remaining 327.28 s) 2023-09-10T00:44:30.488 INFO:teuthology.orchestra.run.smithi044.stderr:19600000 of 50000000 tuples (39%) done (elapsed 210.12 s, remaining 325.90 s) 2023-09-10T00:44:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:30 smithi131 ceph-mon[103713]: pgmap v118: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 27 MiB/s wr, 54 op/s 2023-09-10T00:44:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:30 smithi044 ceph-mon[98676]: pgmap v118: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 27 MiB/s wr, 54 op/s 2023-09-10T00:44:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:30 smithi099 ceph-mon[102692]: pgmap v118: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 27 MiB/s wr, 54 op/s 2023-09-10T00:44:31.349 INFO:teuthology.orchestra.run.smithi044.stderr:19700000 of 50000000 tuples (39%) done (elapsed 210.98 s, remaining 324.51 s) 2023-09-10T00:44:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:19800000 of 50000000 tuples (39%) done (elapsed 212.09 s, remaining 323.49 s) 2023-09-10T00:44:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:32 smithi131 ceph-mon[103713]: pgmap v119: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 43 MiB/s wr, 85 op/s 2023-09-10T00:44:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:32 smithi044 ceph-mon[98676]: pgmap v119: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 43 MiB/s wr, 85 op/s 2023-09-10T00:44:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:32 smithi099 ceph-mon[102692]: pgmap v119: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 43 MiB/s wr, 85 op/s 2023-09-10T00:44:33.387 INFO:teuthology.orchestra.run.smithi044.stderr:19900000 of 50000000 tuples (39%) done (elapsed 213.02 s, remaining 322.21 s) 2023-09-10T00:44:34.318 INFO:teuthology.orchestra.run.smithi044.stderr:20000000 of 50000000 tuples (40%) done (elapsed 213.95 s, remaining 320.93 s) 2023-09-10T00:44:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:34 smithi131 ceph-mon[103713]: pgmap v120: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 65 op/s 2023-09-10T00:44:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:34 smithi044 ceph-mon[98676]: pgmap v120: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 65 op/s 2023-09-10T00:44:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:34 smithi099 ceph-mon[102692]: pgmap v120: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 65 op/s 2023-09-10T00:44:35.388 INFO:teuthology.orchestra.run.smithi044.stderr:20100000 of 50000000 tuples (40%) done (elapsed 215.02 s, remaining 319.86 s) 2023-09-10T00:44:36.318 INFO:teuthology.orchestra.run.smithi044.stderr:20200000 of 50000000 tuples (40%) done (elapsed 215.95 s, remaining 318.58 s) 2023-09-10T00:44:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:36 smithi131 ceph-mon[103713]: pgmap v121: 129 pgs: 129 active+clean; 4.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 48 MiB/s wr, 110 op/s 2023-09-10T00:44:37.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:36 smithi044 ceph-mon[98676]: pgmap v121: 129 pgs: 129 active+clean; 4.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 48 MiB/s wr, 110 op/s 2023-09-10T00:44:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:36 smithi099 ceph-mon[102692]: pgmap v121: 129 pgs: 129 active+clean; 4.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 48 MiB/s wr, 110 op/s 2023-09-10T00:44:37.236 INFO:teuthology.orchestra.run.smithi044.stderr:20300000 of 50000000 tuples (40%) done (elapsed 216.87 s, remaining 317.29 s) 2023-09-10T00:44:37.548 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:44:37.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.700+0000 7f7fe8ad7700 1 -- 172.21.15.44:0/1939853751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe4101770 msgr2=0x7f7fe4101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:37.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.700+0000 7f7fe8ad7700 1 --2- 172.21.15.44:0/1939853751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe4101770 0x7f7fe4101b50 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009990 tx=0x7f7fd4009ca0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.700+0000 7f7fe8ad7700 1 -- 172.21.15.44:0/1939853751 shutdown_connections 2023-09-10T00:44:37.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.700+0000 7f7fe8ad7700 1 --2- 172.21.15.44:0/1939853751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe410ab40 0x7f7fe410cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.700+0000 7f7fe8ad7700 1 --2- 172.21.15.44:0/1939853751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe4101770 0x7f7fe4101b50 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.700+0000 7f7fe8ad7700 1 --2- 172.21.15.44:0/1939853751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fe4102090 0x7f7fe410a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.700+0000 7f7fe8ad7700 1 -- 172.21.15.44:0/1939853751 >> 172.21.15.44:0/1939853751 conn(0x7f7fe40faf50 msgr2=0x7f7fe40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:37.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.700+0000 7f7fe8ad7700 1 -- 172.21.15.44:0/1939853751 shutdown_connections 2023-09-10T00:44:37.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.700+0000 7f7fe8ad7700 1 -- 172.21.15.44:0/1939853751 wait complete. 2023-09-10T00:44:37.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.701+0000 7f7fe8ad7700 1 Processor -- start 2023-09-10T00:44:37.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.701+0000 7f7fe8ad7700 1 -- start start 2023-09-10T00:44:37.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.701+0000 7f7fe8ad7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe4101770 0x7f7fe4196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.701+0000 7f7fe8ad7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe4102090 0x7f7fe4196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.701+0000 7f7fe8ad7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fe410ab40 0x7f7fe419afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.701+0000 7f7fe8ad7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe419b5e0 con 0x7f7fe4102090 2023-09-10T00:44:37.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.701+0000 7f7fe8ad7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe419b750 con 0x7f7fe4101770 2023-09-10T00:44:37.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.701+0000 7f7fe8ad7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe419b910 con 0x7f7fe410ab40 2023-09-10T00:44:37.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.701+0000 7f7fe1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe4102090 0x7f7fe4196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.702+0000 7f7fe259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe4101770 0x7f7fe4196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.702+0000 7f7fe1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe4102090 0x7f7fe4196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50676/0 (socket says 172.21.15.44:50676) 2023-09-10T00:44:37.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.702+0000 7f7fe1d9b700 1 -- 172.21.15.44:0/52373087 learned_addr learned my addr 172.21.15.44:0/52373087 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:37.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.702+0000 7f7fe2d9d700 1 --2- 172.21.15.44:0/52373087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fe410ab40 0x7f7fe419afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.702+0000 7f7fe259c700 1 -- 172.21.15.44:0/52373087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fe410ab40 msgr2=0x7f7fe419afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:37.708 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.702+0000 7f7fe259c700 1 --2- 172.21.15.44:0/52373087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fe410ab40 0x7f7fe419afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.708 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.702+0000 7f7fe259c700 1 -- 172.21.15.44:0/52373087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe4102090 msgr2=0x7f7fe4196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:37.708 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.702+0000 7f7fe259c700 1 --2- 172.21.15.44:0/52373087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe4102090 0x7f7fe4196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.708 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.702+0000 7f7fe259c700 1 -- 172.21.15.44:0/52373087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd40095f0 con 0x7f7fe4101770 2023-09-10T00:44:37.708 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.702+0000 7f7fe2d9d700 1 --2- 172.21.15.44:0/52373087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fe410ab40 0x7f7fe419afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:37.708 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.702+0000 7f7fe259c700 1 --2- 172.21.15.44:0/52373087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe4101770 0x7f7fe4196600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40098e0 tx=0x7f7fd4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:37.709 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.702+0000 7f7fd37fe700 1 -- 172.21.15.44:0/52373087 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd401d070 con 0x7f7fe4101770 2023-09-10T00:44:37.709 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.702+0000 7f7fd37fe700 1 -- 172.21.15.44:0/52373087 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7fd4022470 con 0x7f7fe4101770 2023-09-10T00:44:37.709 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.702+0000 7f7fe8ad7700 1 -- 172.21.15.44:0/52373087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe419bb90 con 0x7f7fe4101770 2023-09-10T00:44:37.709 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.702+0000 7f7fd37fe700 1 -- 172.21.15.44:0/52373087 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4017650 con 0x7f7fe4101770 2023-09-10T00:44:37.710 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.702+0000 7f7fe8ad7700 1 -- 172.21.15.44:0/52373087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe40ff4d0 con 0x7f7fe4101770 2023-09-10T00:44:37.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.705+0000 7f7fd37fe700 1 -- 172.21.15.44:0/52373087 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f7fd40225e0 con 0x7f7fe4101770 2023-09-10T00:44:37.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.706+0000 7f7fd37fe700 1 --2- 172.21.15.44:0/52373087 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7fcc0a07b0 0x7f7fcc0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.706+0000 7f7fd37fe700 1 -- 172.21.15.44:0/52373087 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7fd40cd8c0 con 0x7f7fe4101770 2023-09-10T00:44:37.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.706+0000 7f7fe1d9b700 1 --2- 172.21.15.44:0/52373087 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7fcc0a07b0 0x7f7fcc0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.706+0000 7f7ff4c9db80 1 Processor -- start 2023-09-10T00:44:37.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.706+0000 7f7ff4c9db80 1 -- start start 2023-09-10T00:44:37.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.706+0000 7f7fe1d9b700 1 --2- 172.21.15.44:0/52373087 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7fcc0a07b0 0x7f7fcc0a2c70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4197c10 tx=0x7f7fd8009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:37.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7ff4c9db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 0x5563081fcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7ff4c9db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x556308206900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7ff4c9db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 0x556308209230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7ff4c9db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556308155730 con 0x5563081fc710 2023-09-10T00:44:37.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7ff4c9db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556308153050 con 0x556308206e40 2023-09-10T00:44:37.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7ff4c9db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630814e940 con 0x5563081fd030 2023-09-10T00:44:37.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7fe1d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x556308206900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7fe259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 0x5563081fcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7fe1d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x556308206900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57566/0 (socket says 172.21.15.44:57566) 2023-09-10T00:44:37.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7fe259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 0x5563081fcaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50682/0 (socket says 172.21.15.44:50682) 2023-09-10T00:44:37.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7fe1d9b700 1 -- 172.21.15.44:0/936769948 learned_addr learned my addr 172.21.15.44:0/936769948 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:37.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7fe2d9d700 1 --2- 172.21.15.44:0/936769948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 0x556308209230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7fe1d9b700 1 -- 172.21.15.44:0/936769948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 msgr2=0x556308209230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:37.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7fe1d9b700 1 --2- 172.21.15.44:0/936769948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 0x556308209230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7fe1d9b700 1 -- 172.21.15.44:0/936769948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 msgr2=0x5563081fcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:37.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7fe1d9b700 1 --2- 172.21.15.44:0/936769948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 0x5563081fcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7fe1d9b700 1 -- 172.21.15.44:0/936769948 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556307f7ade0 con 0x5563081fd030 2023-09-10T00:44:37.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.707+0000 7f7fe259c700 1 --2- 172.21.15.44:0/936769948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 0x5563081fcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:37.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.708+0000 7f7fe1d9b700 1 --2- 172.21.15.44:0/936769948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x556308206900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800e570 tx=0x7f7fd8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ee1c50df51b8e90 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:37.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.708+0000 7f7fd0ff9700 1 -- 172.21.15.44:0/936769948 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8026430 con 0x5563081fd030 2023-09-10T00:44:37.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.708+0000 7f7fd0ff9700 1 -- 172.21.15.44:0/936769948 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7fd8021070 con 0x5563081fd030 2023-09-10T00:44:37.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.708+0000 7f7fd0ff9700 1 -- 172.21.15.44:0/936769948 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd801d670 con 0x5563081fd030 2023-09-10T00:44:37.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.709+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/936769948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 msgr2=0x556308206900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:37.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.709+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/936769948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x556308206900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800e570 tx=0x7f7fd8014000 comp rx=0 tx=0).stop 2023-09-10T00:44:37.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.709+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/936769948 shutdown_connections 2023-09-10T00:44:37.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.709+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/936769948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 0x556308209230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.709+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/936769948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 0x5563081fcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.709+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/936769948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x556308206900 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.709+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/936769948 >> 172.21.15.44:0/936769948 conn(0x556308171090 msgr2=0x5563081f7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:37.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.709+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/936769948 shutdown_connections 2023-09-10T00:44:37.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.709+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/936769948 wait complete. 2023-09-10T00:44:37.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.709+0000 7f7ff4c9db80 1 -- start start 2023-09-10T00:44:37.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7ff4c9db80 1 Processor -- start 2023-09-10T00:44:37.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7ff4c9db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 0x5563082068f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7ff4c9db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x556308203b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7ff4c9db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 0x556308204240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7ff4c9db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563080ecb70 con 0x5563081fc710 2023-09-10T00:44:37.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7ff4c9db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563080ec870 con 0x556308206e40 2023-09-10T00:44:37.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7ff4c9db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556307fd0920 con 0x5563081fd030 2023-09-10T00:44:37.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7ff4c9db80 10 client.0 _ll_register_callbacks cb 0x556308036a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:44:37.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7fe259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 0x5563082068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7fe2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 0x556308204240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7fe2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 0x556308204240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52432/0 (socket says 172.21.15.44:52432) 2023-09-10T00:44:37.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7fe259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 0x5563082068f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50688/0 (socket says 172.21.15.44:50688) 2023-09-10T00:44:37.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7fe2d9d700 1 -- 172.21.15.44:0/193483975 learned_addr learned my addr 172.21.15.44:0/193483975 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:37.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7fe1d9b700 1 --2- 172.21.15.44:0/193483975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x556308203b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7fe2d9d700 1 -- 172.21.15.44:0/193483975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 msgr2=0x556308203b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:37.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7fe2d9d700 1 --2- 172.21.15.44:0/193483975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x556308203b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7fe2d9d700 1 -- 172.21.15.44:0/193483975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 msgr2=0x5563082068f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:37.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7fe2d9d700 1 --2- 172.21.15.44:0/193483975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 0x5563082068f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.710+0000 7f7fe2d9d700 1 -- 172.21.15.44:0/193483975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe40ff4d0 con 0x556308206e40 2023-09-10T00:44:37.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.711+0000 7f7fe1d9b700 1 --2- 172.21.15.44:0/193483975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x556308203b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:37.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.711+0000 7f7fe2d9d700 1 --2- 172.21.15.44:0/193483975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 0x556308204240 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc003d50 tx=0x7f7fdc00e7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fab04d42aea9240 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:37.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.711+0000 7f7fc9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:44:37.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.711+0000 7f7fc9ffb700 1 -- 172.21.15.44:0/193483975 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc0090d0 con 0x556308206e40 2023-09-10T00:44:37.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.711+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/193483975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563080a64f0 con 0x556308206e40 2023-09-10T00:44:37.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.711+0000 7f7fc9ffb700 1 -- 172.21.15.44:0/193483975 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7fdc009230 con 0x556308206e40 2023-09-10T00:44:37.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.711+0000 7f7fc9ffb700 1 -- 172.21.15.44:0/193483975 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fdc007500 con 0x556308206e40 2023-09-10T00:44:37.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.711+0000 7f7ff4c9db80 10 client.26267 fetch_fsmap learned FSMap version 42 2023-09-10T00:44:37.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.711+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/193483975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f7fd8017070 con 0x556308206e40 2023-09-10T00:44:37.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.711+0000 7f7fc9ffb700 1 -- 172.21.15.44:0/193483975 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5563080a64f0 con 0x556308206e40 2023-09-10T00:44:37.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.711+0000 7f7fc9ffb700 1 -- 172.21.15.44:0/193483975 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f7fdc01c020 con 0x556308206e40 2023-09-10T00:44:37.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.712+0000 7f7ff4c9db80 10 client.26267 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:44:37.730 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.712+0000 7f7ff4c9db80 10 client.26267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:44:37.730 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.712+0000 7f7ff4c9db80 20 client.26267 populate_metadata read hostname 'smithi044' 2023-09-10T00:44:37.730 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.712+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/193483975 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556308216a70 0x556308218e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.730 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.712+0000 7f7ff4c9db80 4 client.26267 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:44:37.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.712+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/193483975 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563081638a0 con 0x556308216a70 2023-09-10T00:44:37.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.712+0000 7f7fe259c700 1 --2- 172.21.15.44:0/193483975 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556308216a70 0x556308218e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.712+0000 7f7fc9ffb700 10 client.26267 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:37.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.712+0000 7f7fe259c700 1 --2- 172.21.15.44:0/193483975 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556308216a70 0x556308218e90 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e6c6bf5a6c8691 server_cookie=ceb1fdded45fe3f8 in_seq=0 out_seq=0 2023-09-10T00:44:37.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.713+0000 7f7fc9ffb700 1 -- 172.21.15.44:0/193483975 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5563081638a0 con 0x556308216a70 2023-09-10T00:44:37.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.713+0000 7f7fc9ffb700 10 client.26267 handle_command_reply: tid=0 2023-09-10T00:44:37.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.713+0000 7f7ff4c9db80 1 client.26267 shutdown 2023-09-10T00:44:37.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.713+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/193483975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 msgr2=0x556308204240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:37.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.713+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/193483975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 0x556308204240 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc003d50 tx=0x7f7fdc00e7d0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.713+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/193483975 shutdown_connections 2023-09-10T00:44:37.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.713+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/193483975 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556308216a70 0x556308218e90 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.713+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/193483975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 0x556308204240 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.713+0000 7f7fc9ffb700 0 client.26267 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:37.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.713+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/193483975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 0x5563082068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.713+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/193483975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x556308203b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.713+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/193483975 >> 172.21.15.44:0/193483975 conn(0x556308171090 msgr2=0x5563081f77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:37.734 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.713+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/193483975 shutdown_connections 2023-09-10T00:44:37.734 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.713+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/193483975 wait complete. 2023-09-10T00:44:37.734 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.713+0000 7f7ff4c9db80 20 client.26267 trim_cache size 0 max 16384 2023-09-10T00:44:37.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7ff4c9db80 1 Processor -- start 2023-09-10T00:44:37.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7ff4c9db80 1 -- start start 2023-09-10T00:44:37.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7ff4c9db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 0x5563082033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7ff4c9db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x5563081f6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.740 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7ff4c9db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 0x556308206420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7ff4c9db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563081fd7a0 con 0x5563081fc710 2023-09-10T00:44:37.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7ff4c9db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556308209230 con 0x556308206e40 2023-09-10T00:44:37.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7ff4c9db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563081f6860 con 0x5563081fd030 2023-09-10T00:44:37.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7fe259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 0x5563082033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.741 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7fe2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 0x556308206420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7fe1d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x5563081f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7fe2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 0x556308206420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52440/0 (socket says 172.21.15.44:52440) 2023-09-10T00:44:37.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7fe1d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x5563081f6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57580/0 (socket says 172.21.15.44:57580) 2023-09-10T00:44:37.742 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7fe2d9d700 1 -- 172.21.15.44:0/3106510267 learned_addr learned my addr 172.21.15.44:0/3106510267 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:37.743 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7fe2d9d700 1 -- 172.21.15.44:0/3106510267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 msgr2=0x5563081f6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:37.743 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7fe2d9d700 1 --2- 172.21.15.44:0/3106510267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x5563081f6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.743 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7fe2d9d700 1 -- 172.21.15.44:0/3106510267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 msgr2=0x5563082033a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:37.743 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7fe2d9d700 1 --2- 172.21.15.44:0/3106510267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 0x5563082033a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.743 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7fe2d9d700 1 -- 172.21.15.44:0/3106510267 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556307f63c60 con 0x556308206e40 2023-09-10T00:44:37.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.733+0000 7f7fe1d9b700 1 --2- 172.21.15.44:0/3106510267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x5563081f6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:37.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.734+0000 7f7fe2d9d700 1 --2- 172.21.15.44:0/3106510267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 0x556308206420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc000f80 tx=0x7f7fdc006160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7701ff086c52d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:37.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.734+0000 7f7fd17fa700 1 -- 172.21.15.44:0/3106510267 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc007500 con 0x556308206e40 2023-09-10T00:44:37.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.734+0000 7f7fd17fa700 1 -- 172.21.15.44:0/3106510267 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7fdc00f040 con 0x556308206e40 2023-09-10T00:44:37.744 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.734+0000 7f7fd17fa700 1 -- 172.21.15.44:0/3106510267 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc007870 con 0x556308206e40 2023-09-10T00:44:37.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/3106510267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 msgr2=0x556308206420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:37.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/3106510267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 0x556308206420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc000f80 tx=0x7f7fdc006160 comp rx=0 tx=0).stop 2023-09-10T00:44:37.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/3106510267 shutdown_connections 2023-09-10T00:44:37.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/3106510267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556308206e40 0x556308206420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/3106510267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fc710 0x5563082033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.745 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/3106510267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5563081fd030 0x5563081f6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.746 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/3106510267 >> 172.21.15.44:0/3106510267 conn(0x556308171090 msgr2=0x55630821d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:37.746 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/3106510267 shutdown_connections 2023-09-10T00:44:37.746 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/3106510267 wait complete. 2023-09-10T00:44:37.746 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 -- start start 2023-09-10T00:44:37.746 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 Processor -- start 2023-09-10T00:44:37.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563081fc710 0x55630820d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fd030 0x55630820bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556308206e40 0x55630820c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630820c590 con 0x5563081fd030 2023-09-10T00:44:37.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630820c700 con 0x5563081fc710 2023-09-10T00:44:37.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630820c8c0 con 0x556308206e40 2023-09-10T00:44:37.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7ff4c9db80 10 client.0 _ll_register_callbacks cb 0x7f7fdc0043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:44:37.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7fe1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fd030 0x55630820bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.735+0000 7f7fe259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563081fc710 0x55630820d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.748 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7fe259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563081fc710 0x55630820d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52442/0 (socket says 172.21.15.44:52442) 2023-09-10T00:44:37.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7fe259c700 1 -- 172.21.15.44:0/4052420096 learned_addr learned my addr 172.21.15.44:0/4052420096 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:37.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7fe1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fd030 0x55630820bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50704/0 (socket says 172.21.15.44:50704) 2023-09-10T00:44:37.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7fe2d9d700 1 --2- 172.21.15.44:0/4052420096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556308206e40 0x55630820c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7fe259c700 1 -- 172.21.15.44:0/4052420096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556308206e40 msgr2=0x55630820c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:37.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7fe259c700 1 --2- 172.21.15.44:0/4052420096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556308206e40 0x55630820c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7fe259c700 1 -- 172.21.15.44:0/4052420096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fd030 msgr2=0x55630820bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:37.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7fe259c700 1 --2- 172.21.15.44:0/4052420096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fd030 0x55630820bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7fe259c700 1 -- 172.21.15.44:0/4052420096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd802a040 con 0x5563081fc710 2023-09-10T00:44:37.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7fe259c700 1 --2- 172.21.15.44:0/4052420096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563081fc710 0x55630820d850 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7fd409a880 tx=0x7f7fd401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1f98c2884def723 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:37.750 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7fc9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:44:37.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7fc9ffb700 1 -- 172.21.15.44:0/4052420096 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4022ac0 con 0x5563081fc710 2023-09-10T00:44:37.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7fc9ffb700 1 -- 172.21.15.44:0/4052420096 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7fd40b5050 con 0x5563081fc710 2023-09-10T00:44:37.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/4052420096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fd8021070 con 0x5563081fc710 2023-09-10T00:44:37.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7fc9ffb700 1 -- 172.21.15.44:0/4052420096 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fd409ab80 con 0x5563081fc710 2023-09-10T00:44:37.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7ff4c9db80 10 client.26273 fetch_fsmap learned FSMap version 42 2023-09-10T00:44:37.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.736+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/4052420096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f7fd8017070 con 0x5563081fc710 2023-09-10T00:44:37.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.737+0000 7f7fc9ffb700 1 -- 172.21.15.44:0/4052420096 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7fd8021070 con 0x5563081fc710 2023-09-10T00:44:37.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.737+0000 7f7fc9ffb700 1 -- 172.21.15.44:0/4052420096 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f7fd40b7030 con 0x5563081fc710 2023-09-10T00:44:37.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.737+0000 7f7ff4c9db80 10 client.26273 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:44:37.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.737+0000 7f7ff4c9db80 10 client.26273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:44:37.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.737+0000 7f7ff4c9db80 20 client.26273 populate_metadata read hostname 'smithi044' 2023-09-10T00:44:37.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.737+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/4052420096 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55630826b9c0 0x55630826ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:37.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.737+0000 7f7ff4c9db80 4 client.26273 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:44:37.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.737+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/4052420096 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556308147920 con 0x55630826b9c0 2023-09-10T00:44:37.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.737+0000 7f7fe1d9b700 1 --2- 172.21.15.44:0/4052420096 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55630826b9c0 0x55630826ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:37.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.737+0000 7f7fc9ffb700 10 client.26273 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:37.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.737+0000 7f7fe1d9b700 1 --2- 172.21.15.44:0/4052420096 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55630826b9c0 0x55630826ddb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f52727ffe6cd5d9a server_cookie=259f7e457db8ba7b in_seq=0 out_seq=0 2023-09-10T00:44:37.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7fc9ffb700 1 -- 172.21.15.44:0/4052420096 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556308147920 con 0x55630826b9c0 2023-09-10T00:44:37.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7fc9ffb700 10 client.26273 handle_command_reply: tid=0 2023-09-10T00:44:37.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7ff4c9db80 1 client.26273 shutdown 2023-09-10T00:44:37.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/4052420096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563081fc710 msgr2=0x55630820d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:37.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/4052420096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563081fc710 0x55630820d850 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7fd409a880 tx=0x7f7fd401a040 comp rx=0 tx=0).stop 2023-09-10T00:44:37.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/4052420096 shutdown_connections 2023-09-10T00:44:37.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/4052420096 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55630826b9c0 0x55630826ddb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/4052420096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556308206e40 0x55630820c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7fc9ffb700 0 client.26273 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:37.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/4052420096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563081fc710 0x55630820d850 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7ff4c9db80 1 --2- 172.21.15.44:0/4052420096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563081fd030 0x55630820bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/4052420096 >> 172.21.15.44:0/4052420096 conn(0x556308171090 msgr2=0x5563081fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:37.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/4052420096 shutdown_connections 2023-09-10T00:44:37.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7ff4c9db80 1 -- 172.21.15.44:0/4052420096 wait complete. 2023-09-10T00:44:37.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7ff4c9db80 20 client.26273 trim_cache size 0 max 16384 2023-09-10T00:44:37.760 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:44:37.760 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:44:37.760 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:44:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:44:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:44:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:44:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:44:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:44:37.762 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:44:37.762 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:44:37.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7fe8ad7700 1 -- 172.21.15.44:0/52373087 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7fcc0a07b0 msgr2=0x7f7fcc0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:37.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7fe8ad7700 1 --2- 172.21.15.44:0/52373087 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7fcc0a07b0 0x7f7fcc0a2c70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4197c10 tx=0x7f7fd8009260 comp rx=0 tx=0).stop 2023-09-10T00:44:37.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7fe8ad7700 1 -- 172.21.15.44:0/52373087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe4101770 msgr2=0x7f7fe4196600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:37.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.738+0000 7f7fe8ad7700 1 --2- 172.21.15.44:0/52373087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe4101770 0x7f7fe4196600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40098e0 tx=0x7f7fd4004730 comp rx=0 tx=0).stop 2023-09-10T00:44:37.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.739+0000 7f7fe8ad7700 1 -- 172.21.15.44:0/52373087 shutdown_connections 2023-09-10T00:44:37.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.739+0000 7f7fe8ad7700 1 --2- 172.21.15.44:0/52373087 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7fcc0a07b0 0x7f7fcc0a2c70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.739+0000 7f7fe8ad7700 1 --2- 172.21.15.44:0/52373087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fe410ab40 0x7f7fe419afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.739+0000 7f7fe8ad7700 1 --2- 172.21.15.44:0/52373087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe4101770 0x7f7fe4196600 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.739+0000 7f7fe8ad7700 1 --2- 172.21.15.44:0/52373087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fe4102090 0x7f7fe4196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:37.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.739+0000 7f7fe8ad7700 1 -- 172.21.15.44:0/52373087 >> 172.21.15.44:0/52373087 conn(0x7f7fe40faf50 msgr2=0x7f7fe40fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:37.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.739+0000 7f7fe8ad7700 1 -- 172.21.15.44:0/52373087 shutdown_connections 2023-09-10T00:44:37.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:44:37.739+0000 7f7fe8ad7700 1 -- 172.21.15.44:0/52373087 wait complete. 2023-09-10T00:44:37.766 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:44:38.122 INFO:teuthology.orchestra.run.smithi044.stderr:20400000 of 50000000 tuples (40%) done (elapsed 217.76 s, remaining 315.96 s) 2023-09-10T00:44:38.994 INFO:teuthology.orchestra.run.smithi044.stderr:20500000 of 50000000 tuples (41%) done (elapsed 218.63 s, remaining 314.61 s) 2023-09-10T00:44:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:38 smithi131 ceph-mon[103713]: pgmap v122: 129 pgs: 129 active+clean; 4.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 76 op/s 2023-09-10T00:44:39.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:38 smithi044 ceph-mon[98676]: pgmap v122: 129 pgs: 129 active+clean; 4.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 76 op/s 2023-09-10T00:44:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:38 smithi099 ceph-mon[102692]: pgmap v122: 129 pgs: 129 active+clean; 4.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 76 op/s 2023-09-10T00:44:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.230+0000 7f2626601700 1 -- 172.21.15.44:0/1728328927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2620104420 msgr2=0x7f2620104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.230+0000 7f2626601700 1 --2- 172.21.15.44:0/1728328927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2620104420 0x7f2620104800 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2610009990 tx=0x7f2610009ca0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.230+0000 7f2626601700 1 -- 172.21.15.44:0/1728328927 shutdown_connections 2023-09-10T00:44:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.230+0000 7f2626601700 1 --2- 172.21.15.44:0/1728328927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26200ffe70 0x7f262010c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.230+0000 7f2626601700 1 --2- 172.21.15.44:0/1728328927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2620104420 0x7f2620104800 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.230+0000 7f2626601700 1 --2- 172.21.15.44:0/1728328927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26200ff4b0 0x7f26200ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.230+0000 7f2626601700 1 -- 172.21.15.44:0/1728328927 >> 172.21.15.44:0/1728328927 conn(0x7f2620076600 msgr2=0x7f2620076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.230+0000 7f2626601700 1 -- 172.21.15.44:0/1728328927 shutdown_connections 2023-09-10T00:44:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.230+0000 7f2626601700 1 -- 172.21.15.44:0/1728328927 wait complete. 2023-09-10T00:44:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.231+0000 7f2626601700 1 Processor -- start 2023-09-10T00:44:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.231+0000 7f2626601700 1 -- start start 2023-09-10T00:44:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f2626601700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26200ff4b0 0x7f2620194460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f2626601700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26200ffe70 0x7f26201949a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f2626601700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2620104420 0x7f2620199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f2626601700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2620199650 con 0x7f26200ffe70 2023-09-10T00:44:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f2626601700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26201997c0 con 0x7f2620104420 2023-09-10T00:44:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f2626601700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2620199980 con 0x7f26200ff4b0 2023-09-10T00:44:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f2617fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26200ffe70 0x7f26201949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f2617fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26200ffe70 0x7f26201949a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50718/0 (socket says 172.21.15.44:50718) 2023-09-10T00:44:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f2617fff700 1 -- 172.21.15.44:0/2504731627 learned_addr learned my addr 172.21.15.44:0/2504731627 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f261ffff700 1 --2- 172.21.15.44:0/2504731627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26200ff4b0 0x7f2620194460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f2617fff700 1 -- 172.21.15.44:0/2504731627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26200ff4b0 msgr2=0x7f2620194460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f2617fff700 1 --2- 172.21.15.44:0/2504731627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26200ff4b0 0x7f2620194460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f2617fff700 1 -- 172.21.15.44:0/2504731627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2620104420 msgr2=0x7f2620199010 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-10T00:44:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f2617fff700 1 --2- 172.21.15.44:0/2504731627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2620104420 0x7f2620199010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f2617fff700 1 -- 172.21.15.44:0/2504731627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26100095f0 con 0x7f26200ffe70 2023-09-10T00:44:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f261ffff700 1 --2- 172.21.15.44:0/2504731627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26200ff4b0 0x7f2620194460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:44:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.232+0000 7f2617fff700 1 --2- 172.21.15.44:0/2504731627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26200ffe70 0x7f26201949a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f260800e9a0 tx=0x7f260800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.233+0000 7f261dffb700 1 -- 172.21.15.44:0/2504731627 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608009790 con 0x7f26200ffe70 2023-09-10T00:44:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.233+0000 7f261dffb700 1 -- 172.21.15.44:0/2504731627 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2608010430 con 0x7f26200ffe70 2023-09-10T00:44:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.233+0000 7f261dffb700 1 -- 172.21.15.44:0/2504731627 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608018680 con 0x7f26200ffe70 2023-09-10T00:44:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.233+0000 7f2626601700 1 -- 172.21.15.44:0/2504731627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2620199c60 con 0x7f26200ffe70 2023-09-10T00:44:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.233+0000 7f2626601700 1 -- 172.21.15.44:0/2504731627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26201a14a0 con 0x7f26200ffe70 2023-09-10T00:44:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.235+0000 7f261dffb700 1 -- 172.21.15.44:0/2504731627 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f260800cb50 con 0x7f26200ffe70 2023-09-10T00:44:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.236+0000 7f261dffb700 1 --2- 172.21.15.44:0/2504731627 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f260c0a07b0 0x7f260c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.236+0000 7f261ffff700 1 --2- 172.21.15.44:0/2504731627 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f260c0a07b0 0x7f260c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.236+0000 7f261dffb700 1 -- 172.21.15.44:0/2504731627 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f2608014070 con 0x7f26200ffe70 2023-09-10T00:44:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.237+0000 7f261ffff700 1 --2- 172.21.15.44:0/2504731627 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f260c0a07b0 0x7f260c0a2c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2610006010 tx=0x7f2610005940 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.237+0000 7f26327c7b80 1 Processor -- start 2023-09-10T00:44:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.237+0000 7f26327c7b80 1 -- start start 2023-09-10T00:44:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.237+0000 7f26327c7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3b8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.237+0000 7f26327c7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b2300 0x55e4fd3b2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.237+0000 7f26327c7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3bef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.237+0000 7f26327c7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4fd30b6d0 con 0x55e4fd3b81a0 2023-09-10T00:44:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.237+0000 7f26327c7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4fd308ff0 con 0x55e4fd3b2cc0 2023-09-10T00:44:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.237+0000 7f26327c7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4fd3048e0 con 0x55e4fd3b2300 2023-09-10T00:44:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.237+0000 7f261ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.237+0000 7f261ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3b8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50724/0 (socket says 172.21.15.44:50724) 2023-09-10T00:44:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.237+0000 7f261ffff700 1 -- 172.21.15.44:0/1306501803 learned_addr learned my addr 172.21.15.44:0/1306501803 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.237+0000 7f2617fff700 1 --2- 172.21.15.44:0/1306501803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b2300 0x55e4fd3b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.238+0000 7f261ffff700 1 -- 172.21.15.44:0/1306501803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b2300 msgr2=0x55e4fd3b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.238+0000 7f261ffff700 1 --2- 172.21.15.44:0/1306501803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b2300 0x55e4fd3b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.238+0000 7f261ffff700 1 -- 172.21.15.44:0/1306501803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2cc0 msgr2=0x55e4fd3bef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.238+0000 7f261ffff700 1 --2- 172.21.15.44:0/1306501803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3bef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.238+0000 7f261ffff700 1 -- 172.21.15.44:0/1306501803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4fd130d80 con 0x55e4fd3b81a0 2023-09-10T00:44:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.238+0000 7f261ffff700 1 --2- 172.21.15.44:0/1306501803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3b8580 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2610009990 tx=0x7f261001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4e29d0d9ffcf1d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.238+0000 7f26167fc700 1 -- 172.21.15.44:0/1306501803 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261002b070 con 0x55e4fd3b81a0 2023-09-10T00:44:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.238+0000 7f26167fc700 1 -- 172.21.15.44:0/1306501803 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f261001e040 con 0x55e4fd3b81a0 2023-09-10T00:44:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.238+0000 7f26167fc700 1 -- 172.21.15.44:0/1306501803 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2610004e20 con 0x55e4fd3b81a0 2023-09-10T00:44:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.239+0000 7f26327c7b80 1 -- 172.21.15.44:0/1306501803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 msgr2=0x55e4fd3b8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.239+0000 7f26327c7b80 1 --2- 172.21.15.44:0/1306501803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3b8580 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2610009990 tx=0x7f261001f000 comp rx=0 tx=0).stop 2023-09-10T00:44:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.242+0000 7f26327c7b80 1 -- 172.21.15.44:0/1306501803 shutdown_connections 2023-09-10T00:44:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.242+0000 7f26327c7b80 1 --2- 172.21.15.44:0/1306501803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3bef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.242+0000 7f26327c7b80 1 --2- 172.21.15.44:0/1306501803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3b8580 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.242+0000 7f26327c7b80 1 --2- 172.21.15.44:0/1306501803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b2300 0x55e4fd3b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.242+0000 7f26327c7b80 1 -- 172.21.15.44:0/1306501803 >> 172.21.15.44:0/1306501803 conn(0x55e4fd327030 msgr2=0x55e4fd3ad560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.242+0000 7f26327c7b80 1 -- 172.21.15.44:0/1306501803 shutdown_connections 2023-09-10T00:44:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.242+0000 7f26327c7b80 1 -- 172.21.15.44:0/1306501803 wait complete. 2023-09-10T00:44:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.242+0000 7f26327c7b80 1 -- start start 2023-09-10T00:44:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f26327c7b80 1 Processor -- start 2023-09-10T00:44:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f26327c7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b2300 0x55e4fd3beed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f26327c7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3bc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f26327c7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3bc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f26327c7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4fd2a2b10 con 0x55e4fd3b81a0 2023-09-10T00:44:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f26327c7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4fd2a2810 con 0x55e4fd3b2cc0 2023-09-10T00:44:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f26327c7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4fd1868c0 con 0x55e4fd3b2300 2023-09-10T00:44:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f26327c7b80 10 client.0 _ll_register_callbacks cb 0x55e4fd1eca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:44:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f2624b9e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3bc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f2624b9e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3bc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50732/0 (socket says 172.21.15.44:50732) 2023-09-10T00:44:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f2617fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3bc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f2624b9e700 1 -- 172.21.15.44:0/1105888740 learned_addr learned my addr 172.21.15.44:0/1105888740 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f261ffff700 1 --2- 172.21.15.44:0/1105888740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b2300 0x55e4fd3beed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f2624b9e700 1 -- 172.21.15.44:0/1105888740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b2300 msgr2=0x55e4fd3beed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f2624b9e700 1 --2- 172.21.15.44:0/1105888740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b2300 0x55e4fd3beed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f2624b9e700 1 -- 172.21.15.44:0/1105888740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2cc0 msgr2=0x55e4fd3bc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f2624b9e700 1 --2- 172.21.15.44:0/1105888740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3bc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.243+0000 7f2624b9e700 1 -- 172.21.15.44:0/1105888740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262010cc00 con 0x55e4fd3b81a0 2023-09-10T00:44:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.244+0000 7f2624b9e700 1 --2- 172.21.15.44:0/1105888740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3bc890 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2618010bd0 tx=0x7f2618010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96d03c36273f2ff server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.244+0000 7f25ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:44:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.244+0000 7f25ff7fe700 1 -- 172.21.15.44:0/1105888740 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618012040 con 0x55e4fd3b81a0 2023-09-10T00:44:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.244+0000 7f25ff7fe700 1 -- 172.21.15.44:0/1105888740 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f2618003e30 con 0x55e4fd3b81a0 2023-09-10T00:44:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.244+0000 7f26327c7b80 1 -- 172.21.15.44:0/1105888740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4fd25c490 con 0x55e4fd3b81a0 2023-09-10T00:44:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.244+0000 7f25ff7fe700 1 -- 172.21.15.44:0/1105888740 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f261800b110 con 0x55e4fd3b81a0 2023-09-10T00:44:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.244+0000 7f26327c7b80 10 client.16614 fetch_fsmap learned FSMap version 42 2023-09-10T00:44:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.244+0000 7f26327c7b80 1 -- 172.21.15.44:0/1105888740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55e4fd119c00 con 0x55e4fd3b81a0 2023-09-10T00:44:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.244+0000 7f25ff7fe700 1 -- 172.21.15.44:0/1105888740 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e4fd25c490 con 0x55e4fd3b81a0 2023-09-10T00:44:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.244+0000 7f25ff7fe700 1 -- 172.21.15.44:0/1105888740 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f261800d5e0 con 0x55e4fd3b81a0 2023-09-10T00:44:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.244+0000 7f26327c7b80 10 client.16614 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:44:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.244+0000 7f26327c7b80 10 client.16614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:44:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.244+0000 7f26327c7b80 20 client.16614 populate_metadata read hostname 'smithi044' 2023-09-10T00:44:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.244+0000 7f26327c7b80 1 --2- 172.21.15.44:0/1105888740 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e4fd3cc4e0 0x55e4fd3ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.244+0000 7f26327c7b80 4 client.16614 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:44:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.244+0000 7f26327c7b80 1 -- 172.21.15.44:0/1105888740 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4fd319840 con 0x55e4fd3cc4e0 2023-09-10T00:44:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.245+0000 7f261ffff700 1 --2- 172.21.15.44:0/1105888740 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e4fd3cc4e0 0x55e4fd3ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.245+0000 7f25ff7fe700 10 client.16614 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.245+0000 7f261ffff700 1 --2- 172.21.15.44:0/1105888740 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e4fd3cc4e0 0x55e4fd3ce900 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d46b6e11e0647195 server_cookie=1ddc834b78edf69b in_seq=0 out_seq=0 2023-09-10T00:44:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.246+0000 7f25ff7fe700 1 -- 172.21.15.44:0/1105888740 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e4fd319840 con 0x55e4fd3cc4e0 2023-09-10T00:44:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.246+0000 7f25ff7fe700 10 client.16614 handle_command_reply: tid=0 2023-09-10T00:44:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.246+0000 7f26327c7b80 1 client.16614 shutdown 2023-09-10T00:44:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.246+0000 7f26327c7b80 1 -- 172.21.15.44:0/1105888740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 msgr2=0x55e4fd3bc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.246+0000 7f26327c7b80 1 --2- 172.21.15.44:0/1105888740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3bc890 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2618010bd0 tx=0x7f2618010ee0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.246+0000 7f26327c7b80 1 -- 172.21.15.44:0/1105888740 shutdown_connections 2023-09-10T00:44:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.246+0000 7f26327c7b80 1 --2- 172.21.15.44:0/1105888740 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e4fd3cc4e0 0x55e4fd3ce900 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.246+0000 7f26327c7b80 1 --2- 172.21.15.44:0/1105888740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3bc890 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.246+0000 7f26327c7b80 1 --2- 172.21.15.44:0/1105888740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b2300 0x55e4fd3beed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.246+0000 7f26327c7b80 1 --2- 172.21.15.44:0/1105888740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3bc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.246+0000 7f25ff7fe700 0 client.16614 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.246+0000 7f26327c7b80 1 -- 172.21.15.44:0/1105888740 >> 172.21.15.44:0/1105888740 conn(0x55e4fd327030 msgr2=0x55e4fd3ac7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.246+0000 7f26327c7b80 1 -- 172.21.15.44:0/1105888740 shutdown_connections 2023-09-10T00:44:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.246+0000 7f26327c7b80 1 -- 172.21.15.44:0/1105888740 wait complete. 2023-09-10T00:44:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.246+0000 7f26327c7b80 20 client.16614 trim_cache size 0 max 16384 2023-09-10T00:44:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f26327c7b80 1 Processor -- start 2023-09-10T00:44:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f26327c7b80 1 -- start start 2023-09-10T00:44:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f26327c7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b2300 0x55e4fd3c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f26327c7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3bb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f26327c7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3bea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f26327c7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4fd3b79d0 con 0x55e4fd3b2300 2023-09-10T00:44:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f26327c7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4fd3b8580 con 0x55e4fd3b2cc0 2023-09-10T00:44:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f26327c7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4fd3b26e0 con 0x55e4fd3b81a0 2023-09-10T00:44:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f261ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b2300 0x55e4fd3c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f2617fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3bb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f2617fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3bb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52478/0 (socket says 172.21.15.44:52478) 2023-09-10T00:44:39.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f2617fff700 1 -- 172.21.15.44:0/1781316654 learned_addr learned my addr 172.21.15.44:0/1781316654 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:39.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f2624b9e700 1 --2- 172.21.15.44:0/1781316654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3bea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:39.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f2617fff700 1 -- 172.21.15.44:0/1781316654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b81a0 msgr2=0x55e4fd3bea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f2617fff700 1 --2- 172.21.15.44:0/1781316654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3bea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f2617fff700 1 -- 172.21.15.44:0/1781316654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b2300 msgr2=0x55e4fd3c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:39.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f2617fff700 1 --2- 172.21.15.44:0/1781316654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b2300 0x55e4fd3c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f2617fff700 1 -- 172.21.15.44:0/1781316654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4fd1571f0 con 0x55e4fd3b2cc0 2023-09-10T00:44:39.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.265+0000 7f2624b9e700 1 --2- 172.21.15.44:0/1781316654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3bea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:39.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.266+0000 7f2617fff700 1 --2- 172.21.15.44:0/1781316654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3bb6e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f26080c9a60 tx=0x7f2608084040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=629efbd75cd57e8d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.266+0000 7f2616ffd700 1 -- 172.21.15.44:0/1781316654 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608099710 con 0x55e4fd3b2cc0 2023-09-10T00:44:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.266+0000 7f2616ffd700 1 -- 172.21.15.44:0/1781316654 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f260809d540 con 0x55e4fd3b2cc0 2023-09-10T00:44:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.266+0000 7f2616ffd700 1 -- 172.21.15.44:0/1781316654 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260809d380 con 0x55e4fd3b2cc0 2023-09-10T00:44:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.266+0000 7f26327c7b80 1 -- 172.21.15.44:0/1781316654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2cc0 msgr2=0x55e4fd3bb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.267+0000 7f26327c7b80 1 --2- 172.21.15.44:0/1781316654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3bb6e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f26080c9a60 tx=0x7f2608084040 comp rx=0 tx=0).stop 2023-09-10T00:44:39.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.267+0000 7f26327c7b80 1 -- 172.21.15.44:0/1781316654 shutdown_connections 2023-09-10T00:44:39.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.267+0000 7f26327c7b80 1 --2- 172.21.15.44:0/1781316654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3bea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.267+0000 7f26327c7b80 1 --2- 172.21.15.44:0/1781316654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b2300 0x55e4fd3c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.267+0000 7f26327c7b80 1 --2- 172.21.15.44:0/1781316654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3bb6e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.267+0000 7f26327c7b80 1 -- 172.21.15.44:0/1781316654 >> 172.21.15.44:0/1781316654 conn(0x55e4fd327030 msgr2=0x55e4fd3d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.267+0000 7f26327c7b80 1 -- 172.21.15.44:0/1781316654 shutdown_connections 2023-09-10T00:44:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.267+0000 7f26327c7b80 1 -- 172.21.15.44:0/1781316654 wait complete. 2023-09-10T00:44:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.267+0000 7f26327c7b80 1 -- start start 2023-09-10T00:44:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.267+0000 7f26327c7b80 1 Processor -- start 2023-09-10T00:44:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f26327c7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2300 0x55e4fd3c2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f26327c7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3c3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f26327c7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3c1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f26327c7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4fd3c1ba0 con 0x55e4fd3b81a0 2023-09-10T00:44:39.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f26327c7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4fd3c1d10 con 0x55e4fd3b2300 2023-09-10T00:44:39.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f26327c7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4fd3c1ed0 con 0x55e4fd3b2cc0 2023-09-10T00:44:39.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f26327c7b80 10 client.0 _ll_register_callbacks cb 0x7f2618010500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:44:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f261ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2300 0x55e4fd3c2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f2617fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f2624b9e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3c1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f261ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2300 0x55e4fd3c2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52482/0 (socket says 172.21.15.44:52482) 2023-09-10T00:44:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f261ffff700 1 -- 172.21.15.44:0/21974263 learned_addr learned my addr 172.21.15.44:0/21974263 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f2617fff700 1 -- 172.21.15.44:0/21974263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2300 msgr2=0x55e4fd3c2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f2617fff700 1 --2- 172.21.15.44:0/21974263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2300 0x55e4fd3c2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f2617fff700 1 -- 172.21.15.44:0/21974263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 msgr2=0x55e4fd3c1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f2617fff700 1 --2- 172.21.15.44:0/21974263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3c1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f2617fff700 1 -- 172.21.15.44:0/21974263 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4fd119c00 con 0x55e4fd3b2cc0 2023-09-10T00:44:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f2617fff700 1 --2- 172.21.15.44:0/21974263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3c3400 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2608084540 tx=0x7f2608010b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f55d08010800a11b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f25ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:44:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f25ff7fe700 1 -- 172.21.15.44:0/21974263 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608099710 con 0x55e4fd3b2cc0 2023-09-10T00:44:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f26327c7b80 1 -- 172.21.15.44:0/21974263 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f261001e040 con 0x55e4fd3b2cc0 2023-09-10T00:44:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.268+0000 7f25ff7fe700 1 -- 172.21.15.44:0/21974263 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f26080a1780 con 0x55e4fd3b2cc0 2023-09-10T00:44:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.269+0000 7f25ff7fe700 1 -- 172.21.15.44:0/21974263 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26080cc1e0 con 0x55e4fd3b2cc0 2023-09-10T00:44:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.269+0000 7f26327c7b80 10 client.26227 fetch_fsmap learned FSMap version 42 2023-09-10T00:44:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.269+0000 7f26327c7b80 1 -- 172.21.15.44:0/21974263 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f26201a14a0 con 0x55e4fd3b2cc0 2023-09-10T00:44:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.269+0000 7f25ff7fe700 1 -- 172.21.15.44:0/21974263 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f261001e040 con 0x55e4fd3b2cc0 2023-09-10T00:44:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.269+0000 7f25ff7fe700 1 -- 172.21.15.44:0/21974263 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f2608099970 con 0x55e4fd3b2cc0 2023-09-10T00:44:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.269+0000 7f26327c7b80 10 client.26227 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:44:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.269+0000 7f26327c7b80 10 client.26227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:44:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.269+0000 7f26327c7b80 20 client.26227 populate_metadata read hostname 'smithi044' 2023-09-10T00:44:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.269+0000 7f26327c7b80 1 --2- 172.21.15.44:0/21974263 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e4fd425680 0x55e4fd427aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.269+0000 7f26327c7b80 4 client.26227 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:44:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.269+0000 7f26327c7b80 1 -- 172.21.15.44:0/21974263 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e4fd2fd8c0 con 0x55e4fd425680 2023-09-10T00:44:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.269+0000 7f261ffff700 1 --2- 172.21.15.44:0/21974263 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e4fd425680 0x55e4fd427aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.270+0000 7f25ff7fe700 10 client.26227 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.270+0000 7f261ffff700 1 --2- 172.21.15.44:0/21974263 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e4fd425680 0x55e4fd427aa0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b74c95c32cdeea6 server_cookie=6a069778856c2755 in_seq=0 out_seq=0 2023-09-10T00:44:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.280+0000 7f25ff7fe700 1 -- 172.21.15.44:0/21974263 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e4fd2fd8c0 con 0x55e4fd425680 2023-09-10T00:44:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.280+0000 7f25ff7fe700 10 client.26227 handle_command_reply: tid=0 2023-09-10T00:44:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.280+0000 7f26327c7b80 1 client.26227 shutdown 2023-09-10T00:44:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f26327c7b80 1 -- 172.21.15.44:0/21974263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b2cc0 msgr2=0x55e4fd3c3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f26327c7b80 1 --2- 172.21.15.44:0/21974263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3c3400 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2608084540 tx=0x7f2608010b60 comp rx=0 tx=0).stop 2023-09-10T00:44:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f26327c7b80 1 -- 172.21.15.44:0/21974263 shutdown_connections 2023-09-10T00:44:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f26327c7b80 1 --2- 172.21.15.44:0/21974263 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e4fd425680 0x55e4fd427aa0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f26327c7b80 1 --2- 172.21.15.44:0/21974263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e4fd3b81a0 0x55e4fd3c1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f26327c7b80 1 --2- 172.21.15.44:0/21974263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4fd3b2300 0x55e4fd3c2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f26327c7b80 1 --2- 172.21.15.44:0/21974263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4fd3b2cc0 0x55e4fd3c3400 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f25ff7fe700 0 client.26227 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f26327c7b80 1 -- 172.21.15.44:0/21974263 >> 172.21.15.44:0/21974263 conn(0x55e4fd327030 msgr2=0x55e4fd3c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f26327c7b80 1 -- 172.21.15.44:0/21974263 shutdown_connections 2023-09-10T00:44:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f26327c7b80 1 -- 172.21.15.44:0/21974263 wait complete. 2023-09-10T00:44:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f26327c7b80 20 client.26227 trim_cache size 0 max 16384 2023-09-10T00:44:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:44:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:44:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:44:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:44:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f2626601700 1 -- 172.21.15.44:0/2504731627 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f260c0a07b0 msgr2=0x7f260c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f2626601700 1 --2- 172.21.15.44:0/2504731627 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f260c0a07b0 0x7f260c0a2c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2610006010 tx=0x7f2610005940 comp rx=0 tx=0).stop 2023-09-10T00:44:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f2626601700 1 -- 172.21.15.44:0/2504731627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26200ffe70 msgr2=0x7f26201949a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f2626601700 1 --2- 172.21.15.44:0/2504731627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26200ffe70 0x7f26201949a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f260800e9a0 tx=0x7f260800ed60 comp rx=0 tx=0).stop 2023-09-10T00:44:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f2626601700 1 -- 172.21.15.44:0/2504731627 shutdown_connections 2023-09-10T00:44:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f2626601700 1 --2- 172.21.15.44:0/2504731627 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f260c0a07b0 0x7f260c0a2c70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f2626601700 1 --2- 172.21.15.44:0/2504731627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2620104420 0x7f2620199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f2626601700 1 --2- 172.21.15.44:0/2504731627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26200ff4b0 0x7f2620194460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f2626601700 1 --2- 172.21.15.44:0/2504731627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26200ffe70 0x7f26201949a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.281+0000 7f2626601700 1 -- 172.21.15.44:0/2504731627 >> 172.21.15.44:0/2504731627 conn(0x7f2620076600 msgr2=0x7f26200fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.282+0000 7f2626601700 1 -- 172.21.15.44:0/2504731627 shutdown_connections 2023-09-10T00:44:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:39.282+0000 7f2626601700 1 -- 172.21.15.44:0/2504731627 wait complete. 2023-09-10T00:44:39.855 INFO:teuthology.orchestra.run.smithi044.stderr:20600000 of 50000000 tuples (41%) done (elapsed 219.49 s, remaining 313.25 s) 2023-09-10T00:44:40.829 INFO:teuthology.orchestra.run.smithi044.stderr:20700000 of 50000000 tuples (41%) done (elapsed 220.46 s, remaining 312.06 s) 2023-09-10T00:44:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:40 smithi131 ceph-mon[103713]: pgmap v123: 129 pgs: 129 active+clean; 4.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 76 op/s 2023-09-10T00:44:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:40 smithi044 ceph-mon[98676]: pgmap v123: 129 pgs: 129 active+clean; 4.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 76 op/s 2023-09-10T00:44:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:40 smithi099 ceph-mon[102692]: pgmap v123: 129 pgs: 129 active+clean; 4.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 76 op/s 2023-09-10T00:44:41.821 INFO:teuthology.orchestra.run.smithi044.stderr:20800000 of 50000000 tuples (41%) done (elapsed 221.46 s, remaining 310.89 s) 2023-09-10T00:44:42.631 INFO:teuthology.orchestra.run.smithi044.stderr:20900000 of 50000000 tuples (41%) done (elapsed 222.27 s, remaining 309.47 s) 2023-09-10T00:44:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:42 smithi131 ceph-mon[103713]: pgmap v124: 129 pgs: 129 active+clean; 4.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 48 MiB/s wr, 110 op/s 2023-09-10T00:44:43.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:42 smithi044 ceph-mon[98676]: pgmap v124: 129 pgs: 129 active+clean; 4.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 48 MiB/s wr, 110 op/s 2023-09-10T00:44:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:42 smithi099 ceph-mon[102692]: pgmap v124: 129 pgs: 129 active+clean; 4.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 48 MiB/s wr, 110 op/s 2023-09-10T00:44:43.721 INFO:teuthology.orchestra.run.smithi044.stderr:21000000 of 50000000 tuples (42%) done (elapsed 223.36 s, remaining 308.44 s) 2023-09-10T00:44:44.581 INFO:teuthology.orchestra.run.smithi044.stderr:21100000 of 50000000 tuples (42%) done (elapsed 224.22 s, remaining 307.10 s) 2023-09-10T00:44:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:44 smithi131 ceph-mon[103713]: pgmap v125: 129 pgs: 129 active+clean; 4.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 79 op/s 2023-09-10T00:44:45.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:44 smithi044 ceph-mon[98676]: pgmap v125: 129 pgs: 129 active+clean; 4.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 79 op/s 2023-09-10T00:44:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:44 smithi099 ceph-mon[102692]: pgmap v125: 129 pgs: 129 active+clean; 4.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 79 op/s 2023-09-10T00:44:45.517 INFO:teuthology.orchestra.run.smithi044.stderr:21200000 of 50000000 tuples (42%) done (elapsed 225.15 s, remaining 305.87 s) 2023-09-10T00:44:46.434 INFO:teuthology.orchestra.run.smithi044.stderr:21300000 of 50000000 tuples (42%) done (elapsed 226.07 s, remaining 304.61 s) 2023-09-10T00:44:47.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:46 smithi131 ceph-mon[103713]: pgmap v126: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 48 MiB/s wr, 114 op/s 2023-09-10T00:44:47.504 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:46 smithi099 ceph-mon[102692]: pgmap v126: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 48 MiB/s wr, 114 op/s 2023-09-10T00:44:47.504 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:46 smithi044 ceph-mon[98676]: pgmap v126: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 48 MiB/s wr, 114 op/s 2023-09-10T00:44:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:21400000 of 50000000 tuples (42%) done (elapsed 226.96 s, remaining 303.31 s) 2023-09-10T00:44:48.205 INFO:teuthology.orchestra.run.smithi044.stderr:21500000 of 50000000 tuples (43%) done (elapsed 227.84 s, remaining 302.02 s) 2023-09-10T00:44:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:48 smithi131 ceph-mon[103713]: pgmap v127: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 32 MiB/s wr, 69 op/s 2023-09-10T00:44:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:48 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:44:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:48 smithi044 ceph-mon[98676]: pgmap v127: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 32 MiB/s wr, 69 op/s 2023-09-10T00:44:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:48 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:44:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:48 smithi099 ceph-mon[102692]: pgmap v127: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 32 MiB/s wr, 69 op/s 2023-09-10T00:44:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:48 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:44:49.141 INFO:teuthology.orchestra.run.smithi044.stderr:21600000 of 50000000 tuples (43%) done (elapsed 228.78 s, remaining 300.80 s) 2023-09-10T00:44:50.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:49 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:44:50.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:49 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:44:50.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:49 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:44:50.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:49 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:44:50.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:49 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:44:50.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:49 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:44:50.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:49 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:44:50.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:49 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:44:50.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:49 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:44:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:50 smithi099 ceph-mon[102692]: pgmap v128: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 32 MiB/s wr, 69 op/s 2023-09-10T00:44:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:50 smithi131 ceph-mon[103713]: pgmap v128: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 32 MiB/s wr, 69 op/s 2023-09-10T00:44:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:50 smithi044 ceph-mon[98676]: pgmap v128: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 32 MiB/s wr, 69 op/s 2023-09-10T00:44:51.838 INFO:teuthology.orchestra.run.smithi044.stderr:21700000 of 50000000 tuples (43%) done (elapsed 231.47 s, remaining 301.87 s) 2023-09-10T00:44:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.402+0000 7fc56a319700 1 -- 172.21.15.44:0/1548091892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5640ff760 msgr2=0x7fc5640ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.402+0000 7fc56a319700 1 --2- 172.21.15.44:0/1548091892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5640ff760 0x7fc5640ffbe0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0099c0 tx=0x7fc54c009cd0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.402+0000 7fc56a319700 1 -- 172.21.15.44:0/1548091892 shutdown_connections 2023-09-10T00:44:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.402+0000 7fc56a319700 1 --2- 172.21.15.44:0/1548091892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc564100120 0x7fc56410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.402+0000 7fc56a319700 1 --2- 172.21.15.44:0/1548091892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5641055b0 0x7fc564105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.402+0000 7fc56a319700 1 --2- 172.21.15.44:0/1548091892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5640ff760 0x7fc5640ffbe0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.402+0000 7fc56a319700 1 -- 172.21.15.44:0/1548091892 >> 172.21.15.44:0/1548091892 conn(0x7fc564074f30 msgr2=0x7fc564075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.402+0000 7fc56a319700 1 -- 172.21.15.44:0/1548091892 shutdown_connections 2023-09-10T00:44:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.402+0000 7fc56a319700 1 -- 172.21.15.44:0/1548091892 wait complete. 2023-09-10T00:44:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.402+0000 7fc56a319700 1 Processor -- start 2023-09-10T00:44:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.402+0000 7fc56a319700 1 -- start start 2023-09-10T00:44:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc56a319700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5640ff760 0x7fc564197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc56a319700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc564100120 0x7fc564197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc56a319700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5641055b0 0x7fc5641a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc56a319700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5641a29d0 con 0x7fc5640ff760 2023-09-10T00:44:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc56a319700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5641a2b40 con 0x7fc564100120 2023-09-10T00:44:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc56a319700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5641a2d00 con 0x7fc5641055b0 2023-09-10T00:44:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc563fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5640ff760 0x7fc564197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc563fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5640ff760 0x7fc564197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50756/0 (socket says 172.21.15.44:50756) 2023-09-10T00:44:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc563fff700 1 -- 172.21.15.44:0/1091133769 learned_addr learned my addr 172.21.15.44:0/1091133769 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc5688b6700 1 --2- 172.21.15.44:0/1091133769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5641055b0 0x7fc5641a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc5637fe700 1 --2- 172.21.15.44:0/1091133769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc564100120 0x7fc564197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc563fff700 1 -- 172.21.15.44:0/1091133769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5641055b0 msgr2=0x7fc5641a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc563fff700 1 --2- 172.21.15.44:0/1091133769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5641055b0 0x7fc5641a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc563fff700 1 -- 172.21.15.44:0/1091133769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc564100120 msgr2=0x7fc564197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc563fff700 1 --2- 172.21.15.44:0/1091133769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc564100120 0x7fc564197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc563fff700 1 -- 172.21.15.44:0/1091133769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c0095f0 con 0x7fc5640ff760 2023-09-10T00:44:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc5688b6700 1 --2- 172.21.15.44:0/1091133769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5641055b0 0x7fc5641a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.403+0000 7fc563fff700 1 --2- 172.21.15.44:0/1091133769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5640ff760 0x7fc564197560 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc55400e9a0 tx=0x7fc55400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.404+0000 7fc5617fa700 1 -- 172.21.15.44:0/1091133769 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc554009790 con 0x7fc5640ff760 2023-09-10T00:44:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.404+0000 7fc5617fa700 1 -- 172.21.15.44:0/1091133769 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc554010430 con 0x7fc5640ff760 2023-09-10T00:44:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.404+0000 7fc56a319700 1 -- 172.21.15.44:0/1091133769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5641a2fe0 con 0x7fc5640ff760 2023-09-10T00:44:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.404+0000 7fc5617fa700 1 -- 172.21.15.44:0/1091133769 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55400cea0 con 0x7fc5640ff760 2023-09-10T00:44:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.404+0000 7fc56a319700 1 -- 172.21.15.44:0/1091133769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5641a3410 con 0x7fc5640ff760 2023-09-10T00:44:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.406+0000 7fc5617fa700 1 -- 172.21.15.44:0/1091133769 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc5540105a0 con 0x7fc5640ff760 2023-09-10T00:44:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.407+0000 7fc5617fa700 1 --2- 172.21.15.44:0/1091133769 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc5500a0830 0x7fc5500a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.407+0000 7fc5637fe700 1 --2- 172.21.15.44:0/1091133769 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc5500a0830 0x7fc5500a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.407+0000 7fc5617fa700 1 -- 172.21.15.44:0/1091133769 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc554014070 con 0x7fc5640ff760 2023-09-10T00:44:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.407+0000 7fc5637fe700 1 --2- 172.21.15.44:0/1091133769 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc5500a0830 0x7fc5500a2cf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc54c00b580 tx=0x7fc54c005c00 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.407+0000 7fc5764dfb80 1 Processor -- start 2023-09-10T00:44:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc5764dfb80 1 -- start start 2023-09-10T00:44:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc5764dfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 0x55b1261c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc5764dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3300 0x55b1261c3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc5764dfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3cc0 0x55b1261cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc5764dfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b12611c6d0 con 0x55b1261c91a0 2023-09-10T00:44:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc5764dfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b126119ff0 con 0x55b1261c3cc0 2023-09-10T00:44:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc5764dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1261158e0 con 0x55b1261c3300 2023-09-10T00:44:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc563fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 0x55b1261c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc563fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 0x55b1261c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50764/0 (socket says 172.21.15.44:50764) 2023-09-10T00:44:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc563fff700 1 -- 172.21.15.44:0/1443948616 learned_addr learned my addr 172.21.15.44:0/1443948616 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc5637fe700 1 --2- 172.21.15.44:0/1443948616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3300 0x55b1261c3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc5688b6700 1 --2- 172.21.15.44:0/1443948616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3cc0 0x55b1261cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc563fff700 1 -- 172.21.15.44:0/1443948616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3300 msgr2=0x55b1261c3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc563fff700 1 --2- 172.21.15.44:0/1443948616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3300 0x55b1261c3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc563fff700 1 -- 172.21.15.44:0/1443948616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3cc0 msgr2=0x55b1261cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc563fff700 1 --2- 172.21.15.44:0/1443948616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3cc0 0x55b1261cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc563fff700 1 -- 172.21.15.44:0/1443948616 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b125f41d80 con 0x55b1261c91a0 2023-09-10T00:44:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc563fff700 1 --2- 172.21.15.44:0/1443948616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 0x55b1261c9580 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc5540a1010 tx=0x7fc554098e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3940f697caf7cdf7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc55a7fc700 1 -- 172.21.15.44:0/1443948616 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5540994d0 con 0x55b1261c91a0 2023-09-10T00:44:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc55a7fc700 1 -- 172.21.15.44:0/1443948616 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc55409d300 con 0x55b1261c91a0 2023-09-10T00:44:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.408+0000 7fc55a7fc700 1 -- 172.21.15.44:0/1443948616 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc554099990 con 0x55b1261c91a0 2023-09-10T00:44:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.409+0000 7fc5764dfb80 1 -- 172.21.15.44:0/1443948616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 msgr2=0x55b1261c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.409+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/1443948616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 0x55b1261c9580 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc5540a1010 tx=0x7fc554098e30 comp rx=0 tx=0).stop 2023-09-10T00:44:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.409+0000 7fc5764dfb80 1 -- 172.21.15.44:0/1443948616 shutdown_connections 2023-09-10T00:44:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.409+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/1443948616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3cc0 0x55b1261cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.409+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/1443948616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 0x55b1261c9580 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.409+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/1443948616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3300 0x55b1261c3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.409+0000 7fc5764dfb80 1 -- 172.21.15.44:0/1443948616 >> 172.21.15.44:0/1443948616 conn(0x55b126138030 msgr2=0x55b1261be560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.409+0000 7fc5764dfb80 1 -- 172.21.15.44:0/1443948616 shutdown_connections 2023-09-10T00:44:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.409+0000 7fc5764dfb80 1 -- 172.21.15.44:0/1443948616 wait complete. 2023-09-10T00:44:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.409+0000 7fc5764dfb80 1 -- start start 2023-09-10T00:44:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc5764dfb80 1 Processor -- start 2023-09-10T00:44:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc5764dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3300 0x55b1261d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc5764dfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3cc0 0x55b1261d3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc5764dfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 0x55b1261ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc5764dfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1260b3b10 con 0x55b1261c91a0 2023-09-10T00:44:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc5764dfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1260b3810 con 0x55b1261c3cc0 2023-09-10T00:44:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc5764dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b125f978c0 con 0x55b1261c3300 2023-09-10T00:44:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc5764dfb80 10 client.0 _ll_register_callbacks cb 0x55b125ffda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:44:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc5688b6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 0x55b1261ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc5688b6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 0x55b1261ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50774/0 (socket says 172.21.15.44:50774) 2023-09-10T00:44:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc563fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3300 0x55b1261d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc5688b6700 1 -- 172.21.15.44:0/3633800850 learned_addr learned my addr 172.21.15.44:0/3633800850 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc5637fe700 1 --2- 172.21.15.44:0/3633800850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3cc0 0x55b1261d3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc5688b6700 1 -- 172.21.15.44:0/3633800850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3300 msgr2=0x55b1261d2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc5688b6700 1 --2- 172.21.15.44:0/3633800850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3300 0x55b1261d2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc5688b6700 1 -- 172.21.15.44:0/3633800850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3cc0 msgr2=0x55b1261d3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.410+0000 7fc5688b6700 1 --2- 172.21.15.44:0/3633800850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3cc0 0x55b1261d3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5688b6700 1 -- 172.21.15.44:0/3633800850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5641a3410 con 0x55b1261c91a0 2023-09-10T00:44:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc563fff700 1 --2- 172.21.15.44:0/3633800850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3300 0x55b1261d2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:44:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5688b6700 1 --2- 172.21.15.44:0/3633800850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 0x55b1261ccfa0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0046f0 tx=0x7fc55c004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0f32c52f62dbb2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5437fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:44:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5437fe700 1 -- 172.21.15.44:0/3633800850 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c00ee90 con 0x55b1261c91a0 2023-09-10T00:44:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5764dfb80 1 -- 172.21.15.44:0/3633800850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b12606d490 con 0x55b1261c91a0 2023-09-10T00:44:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5437fe700 1 -- 172.21.15.44:0/3633800850 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc55c009030 con 0x55b1261c91a0 2023-09-10T00:44:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5437fe700 1 -- 172.21.15.44:0/3633800850 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc55c0091f0 con 0x55b1261c91a0 2023-09-10T00:44:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5764dfb80 10 client.16632 fetch_fsmap learned FSMap version 42 2023-09-10T00:44:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5764dfb80 1 -- 172.21.15.44:0/3633800850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55b125f2ac00 con 0x55b1261c91a0 2023-09-10T00:44:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5437fe700 1 -- 172.21.15.44:0/3633800850 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b12606d490 con 0x55b1261c91a0 2023-09-10T00:44:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5437fe700 1 -- 172.21.15.44:0/3633800850 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc55c008460 con 0x55b1261c91a0 2023-09-10T00:44:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5764dfb80 10 client.16632 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:44:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5764dfb80 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:44:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5764dfb80 20 client.16632 populate_metadata read hostname 'smithi044' 2023-09-10T00:44:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/3633800850 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b1261dd260 0x55b1261df680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5764dfb80 4 client.16632 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:44:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.411+0000 7fc5764dfb80 1 -- 172.21.15.44:0/3633800850 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b12612a840 con 0x55b1261dd260 2023-09-10T00:44:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.412+0000 7fc563fff700 1 --2- 172.21.15.44:0/3633800850 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b1261dd260 0x55b1261df680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.412+0000 7fc5437fe700 10 client.16632 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.412+0000 7fc563fff700 1 --2- 172.21.15.44:0/3633800850 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b1261dd260 0x55b1261df680 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3adb252ca75f85ae server_cookie=c66721729ff09e99 in_seq=0 out_seq=0 2023-09-10T00:44:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.413+0000 7fc5437fe700 1 -- 172.21.15.44:0/3633800850 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b12612a840 con 0x55b1261dd260 2023-09-10T00:44:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.413+0000 7fc5437fe700 10 client.16632 handle_command_reply: tid=0 2023-09-10T00:44:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.413+0000 7fc5764dfb80 1 client.16632 shutdown 2023-09-10T00:44:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.413+0000 7fc5764dfb80 1 -- 172.21.15.44:0/3633800850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 msgr2=0x55b1261ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.413+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/3633800850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 0x55b1261ccfa0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0046f0 tx=0x7fc55c004a00 comp rx=0 tx=0).stop 2023-09-10T00:44:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.413+0000 7fc5764dfb80 1 -- 172.21.15.44:0/3633800850 shutdown_connections 2023-09-10T00:44:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.413+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/3633800850 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b1261dd260 0x55b1261df680 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.413+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/3633800850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 0x55b1261ccfa0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.413+0000 7fc5437fe700 0 client.16632 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.413+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/3633800850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3300 0x55b1261d2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.413+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/3633800850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3cc0 0x55b1261d3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.413+0000 7fc5764dfb80 1 -- 172.21.15.44:0/3633800850 >> 172.21.15.44:0/3633800850 conn(0x55b126138030 msgr2=0x55b1261bd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.413+0000 7fc5764dfb80 1 -- 172.21.15.44:0/3633800850 shutdown_connections 2023-09-10T00:44:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.413+0000 7fc5764dfb80 1 -- 172.21.15.44:0/3633800850 wait complete. 2023-09-10T00:44:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.413+0000 7fc5764dfb80 20 client.16632 trim_cache size 0 max 16384 2023-09-10T00:44:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc5764dfb80 1 Processor -- start 2023-09-10T00:44:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc5764dfb80 1 -- start start 2023-09-10T00:44:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc5764dfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c3300 0x55b1261be710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc5764dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3cc0 0x55b1261d3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc5764dfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c91a0 0x55b1261cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc5764dfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1261bfd60 con 0x55b1261c3300 2023-09-10T00:44:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc5764dfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1261c7b00 con 0x55b1261c91a0 2023-09-10T00:44:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc5764dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1261c6db0 con 0x55b1261c3cc0 2023-09-10T00:44:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc563fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c3300 0x55b1261be710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc563fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c3300 0x55b1261be710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50778/0 (socket says 172.21.15.44:50778) 2023-09-10T00:44:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc563fff700 1 -- 172.21.15.44:0/2791841927 learned_addr learned my addr 172.21.15.44:0/2791841927 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc5637fe700 1 --2- 172.21.15.44:0/2791841927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3cc0 0x55b1261d3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc5688b6700 1 --2- 172.21.15.44:0/2791841927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c91a0 0x55b1261cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc563fff700 1 -- 172.21.15.44:0/2791841927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3cc0 msgr2=0x55b1261d3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc563fff700 1 --2- 172.21.15.44:0/2791841927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3cc0 0x55b1261d3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc563fff700 1 -- 172.21.15.44:0/2791841927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c91a0 msgr2=0x55b1261cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc563fff700 1 --2- 172.21.15.44:0/2791841927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c91a0 0x55b1261cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.428+0000 7fc563fff700 1 -- 172.21.15.44:0/2791841927 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b125f681f0 con 0x55b1261c3300 2023-09-10T00:44:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.429+0000 7fc563fff700 1 --2- 172.21.15.44:0/2791841927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c3300 0x55b1261be710 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc5540959d0 tx=0x7fc554095810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd54ef05d0e31d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.429+0000 7fc55affd700 1 -- 172.21.15.44:0/2791841927 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5540994d0 con 0x55b1261c3300 2023-09-10T00:44:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.429+0000 7fc55affd700 1 -- 172.21.15.44:0/2791841927 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5540a1540 con 0x55b1261c3300 2023-09-10T00:44:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.429+0000 7fc55affd700 1 -- 172.21.15.44:0/2791841927 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc554099730 con 0x55b1261c3300 2023-09-10T00:44:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.429+0000 7fc5764dfb80 1 -- 172.21.15.44:0/2791841927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c3300 msgr2=0x55b1261be710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.429+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/2791841927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c3300 0x55b1261be710 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc5540959d0 tx=0x7fc554095810 comp rx=0 tx=0).stop 2023-09-10T00:44:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.429+0000 7fc5764dfb80 1 -- 172.21.15.44:0/2791841927 shutdown_connections 2023-09-10T00:44:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.429+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/2791841927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c91a0 0x55b1261cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.429+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/2791841927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c3300 0x55b1261be710 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.429+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/2791841927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3cc0 0x55b1261d3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.429+0000 7fc5764dfb80 1 -- 172.21.15.44:0/2791841927 >> 172.21.15.44:0/2791841927 conn(0x55b126138030 msgr2=0x55b1261d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.429+0000 7fc5764dfb80 1 -- 172.21.15.44:0/2791841927 shutdown_connections 2023-09-10T00:44:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.429+0000 7fc5764dfb80 1 -- 172.21.15.44:0/2791841927 wait complete. 2023-09-10T00:44:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.430+0000 7fc5764dfb80 1 -- start start 2023-09-10T00:44:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.430+0000 7fc5764dfb80 1 Processor -- start 2023-09-10T00:44:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.430+0000 7fc5764dfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3300 0x55b1261d6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.430+0000 7fc5764dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3cc0 0x55b1261d7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.430+0000 7fc5764dfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 0x55b1261d7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.430+0000 7fc5764dfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b126230100 con 0x55b1261c91a0 2023-09-10T00:44:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.430+0000 7fc5764dfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b126230270 con 0x55b1261c3300 2023-09-10T00:44:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.430+0000 7fc5764dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b126230430 con 0x55b1261c3cc0 2023-09-10T00:44:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.430+0000 7fc5764dfb80 10 client.0 _ll_register_callbacks cb 0x55b125ffda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:44:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.430+0000 7fc563fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3300 0x55b1261d6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.430+0000 7fc5688b6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 0x55b1261d7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.430+0000 7fc563fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3300 0x55b1261d6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52524/0 (socket says 172.21.15.44:52524) 2023-09-10T00:44:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.430+0000 7fc563fff700 1 -- 172.21.15.44:0/1553908129 learned_addr learned my addr 172.21.15.44:0/1553908129 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:44:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.430+0000 7fc5637fe700 1 --2- 172.21.15.44:0/1553908129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3cc0 0x55b1261d7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.431+0000 7fc563fff700 1 -- 172.21.15.44:0/1553908129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3cc0 msgr2=0x55b1261d7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.431+0000 7fc563fff700 1 --2- 172.21.15.44:0/1553908129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3cc0 0x55b1261d7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.431+0000 7fc563fff700 1 -- 172.21.15.44:0/1553908129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 msgr2=0x55b1261d7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:44:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.431+0000 7fc563fff700 1 --2- 172.21.15.44:0/1553908129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 0x55b1261d7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.431+0000 7fc563fff700 1 -- 172.21.15.44:0/1553908129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b125f41d80 con 0x55b1261c3300 2023-09-10T00:44:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.431+0000 7fc563fff700 1 --2- 172.21.15.44:0/1553908129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3300 0x55b1261d6ee0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc55400c2d0 tx=0x7fc554098e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b613c506c0e683 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:44:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.431+0000 7fc5437fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:44:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.431+0000 7fc5437fe700 1 -- 172.21.15.44:0/1553908129 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5540994d0 con 0x55b1261c3300 2023-09-10T00:44:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.431+0000 7fc5437fe700 1 -- 172.21.15.44:0/1553908129 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc554097610 con 0x55b1261c3300 2023-09-10T00:44:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.431+0000 7fc5764dfb80 1 -- 172.21.15.44:0/1553908129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc55409d300 con 0x55b1261c3300 2023-09-10T00:44:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.431+0000 7fc5437fe700 1 -- 172.21.15.44:0/1553908129 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5540840b0 con 0x55b1261c3300 2023-09-10T00:44:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.431+0000 7fc5764dfb80 10 client.26303 fetch_fsmap learned FSMap version 42 2023-09-10T00:44:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.431+0000 7fc5764dfb80 1 -- 172.21.15.44:0/1553908129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55b125f40250 con 0x55b1261c3300 2023-09-10T00:44:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.432+0000 7fc5437fe700 1 -- 172.21.15.44:0/1553908129 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc55409d300 con 0x55b1261c3300 2023-09-10T00:44:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.432+0000 7fc5437fe700 1 -- 172.21.15.44:0/1553908129 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc5540997c0 con 0x55b1261c3300 2023-09-10T00:44:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.432+0000 7fc5764dfb80 10 client.26303 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:44:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.432+0000 7fc5764dfb80 10 client.26303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:44:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.432+0000 7fc5764dfb80 20 client.26303 populate_metadata read hostname 'smithi044' 2023-09-10T00:44:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.432+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/1553908129 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b1262316c0 0x55b126233ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:44:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.432+0000 7fc5764dfb80 4 client.26303 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:44:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.432+0000 7fc5764dfb80 1 -- 172.21.15.44:0/1553908129 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b12610e8c0 con 0x55b1262316c0 2023-09-10T00:44:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.432+0000 7fc5637fe700 1 --2- 172.21.15.44:0/1553908129 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b1262316c0 0x55b126233ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:44:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.432+0000 7fc5437fe700 10 client.26303 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.433+0000 7fc5637fe700 1 --2- 172.21.15.44:0/1553908129 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b1262316c0 0x55b126233ab0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2756add0ba009b6b server_cookie=ca3e34bccafb13ed in_seq=0 out_seq=0 2023-09-10T00:44:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.445+0000 7fc5437fe700 1 -- 172.21.15.44:0/1553908129 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b12610e8c0 con 0x55b1262316c0 2023-09-10T00:44:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.445+0000 7fc5437fe700 10 client.26303 handle_command_reply: tid=0 2023-09-10T00:44:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.445+0000 7fc5764dfb80 1 client.26303 shutdown 2023-09-10T00:44:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.445+0000 7fc5764dfb80 1 -- 172.21.15.44:0/1553908129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3300 msgr2=0x55b1261d6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.445+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/1553908129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3300 0x55b1261d6ee0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc55400c2d0 tx=0x7fc554098e50 comp rx=0 tx=0).stop 2023-09-10T00:44:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc5764dfb80 1 -- 172.21.15.44:0/1553908129 shutdown_connections 2023-09-10T00:44:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/1553908129 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b1262316c0 0x55b126233ab0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/1553908129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1261c91a0 0x55b1261d7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/1553908129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1261c3300 0x55b1261d6ee0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc5764dfb80 1 --2- 172.21.15.44:0/1553908129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1261c3cc0 0x55b1261d7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc5437fe700 0 client.26303 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:44:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc5764dfb80 1 -- 172.21.15.44:0/1553908129 >> 172.21.15.44:0/1553908129 conn(0x55b126138030 msgr2=0x55b1261c22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc5764dfb80 1 -- 172.21.15.44:0/1553908129 shutdown_connections 2023-09-10T00:44:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc5764dfb80 1 -- 172.21.15.44:0/1553908129 wait complete. 2023-09-10T00:44:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc5764dfb80 20 client.26303 trim_cache size 0 max 16384 2023-09-10T00:44:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:44:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:44:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:44:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:44:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc56a319700 1 -- 172.21.15.44:0/1091133769 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc5500a0830 msgr2=0x7fc5500a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc56a319700 1 --2- 172.21.15.44:0/1091133769 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc5500a0830 0x7fc5500a2cf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc54c00b580 tx=0x7fc54c005c00 comp rx=0 tx=0).stop 2023-09-10T00:44:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc56a319700 1 -- 172.21.15.44:0/1091133769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5640ff760 msgr2=0x7fc564197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:44:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc56a319700 1 --2- 172.21.15.44:0/1091133769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5640ff760 0x7fc564197560 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc55400e9a0 tx=0x7fc55400ecb0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc56a319700 1 -- 172.21.15.44:0/1091133769 shutdown_connections 2023-09-10T00:44:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc56a319700 1 --2- 172.21.15.44:0/1091133769 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc5500a0830 0x7fc5500a2cf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc56a319700 1 --2- 172.21.15.44:0/1091133769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5641055b0 0x7fc5641a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc56a319700 1 --2- 172.21.15.44:0/1091133769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5640ff760 0x7fc564197560 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc56a319700 1 --2- 172.21.15.44:0/1091133769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc564100120 0x7fc564197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:44:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.446+0000 7fc56a319700 1 -- 172.21.15.44:0/1091133769 >> 172.21.15.44:0/1091133769 conn(0x7fc564074f30 msgr2=0x7fc5640fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:44:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.447+0000 7fc56a319700 1 -- 172.21.15.44:0/1091133769 shutdown_connections 2023-09-10T00:44:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:44:52.447+0000 7fc56a319700 1 -- 172.21.15.44:0/1091133769 wait complete. 2023-09-10T00:44:52.764 INFO:teuthology.orchestra.run.smithi044.stderr:21800000 of 50000000 tuples (43%) done (elapsed 232.40 s, remaining 300.63 s) 2023-09-10T00:44:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:52 smithi099 ceph-mon[102692]: pgmap v129: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 48 MiB/s wr, 87 op/s 2023-09-10T00:44:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:52 smithi131 ceph-mon[103713]: pgmap v129: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 48 MiB/s wr, 87 op/s 2023-09-10T00:44:53.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:52 smithi044 ceph-mon[98676]: pgmap v129: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 48 MiB/s wr, 87 op/s 2023-09-10T00:44:53.775 INFO:teuthology.orchestra.run.smithi044.stderr:21900000 of 50000000 tuples (43%) done (elapsed 233.41 s, remaining 299.49 s) 2023-09-10T00:44:54.727 INFO:teuthology.orchestra.run.smithi044.stderr:22000000 of 50000000 tuples (44%) done (elapsed 234.36 s, remaining 298.28 s) 2023-09-10T00:44:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:54 smithi099 ceph-mon[102692]: pgmap v130: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 32 MiB/s wr, 53 op/s 2023-09-10T00:44:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:54 smithi131 ceph-mon[103713]: pgmap v130: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 32 MiB/s wr, 53 op/s 2023-09-10T00:44:55.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:54 smithi044 ceph-mon[98676]: pgmap v130: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 32 MiB/s wr, 53 op/s 2023-09-10T00:44:56.933 INFO:teuthology.orchestra.run.smithi044.stderr:22100000 of 50000000 tuples (44%) done (elapsed 236.57 s, remaining 298.65 s) 2023-09-10T00:44:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:56 smithi099 ceph-mon[102692]: pgmap v131: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 41 MiB/s wr, 83 op/s 2023-09-10T00:44:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:56 smithi131 ceph-mon[103713]: pgmap v131: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 41 MiB/s wr, 83 op/s 2023-09-10T00:44:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:56 smithi044 ceph-mon[98676]: pgmap v131: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 41 MiB/s wr, 83 op/s 2023-09-10T00:44:57.881 INFO:teuthology.orchestra.run.smithi044.stderr:22200000 of 50000000 tuples (44%) done (elapsed 237.52 s, remaining 297.43 s) 2023-09-10T00:44:58.739 INFO:teuthology.orchestra.run.smithi044.stderr:22300000 of 50000000 tuples (44%) done (elapsed 238.37 s, remaining 296.10 s) 2023-09-10T00:44:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:44:58 smithi099 ceph-mon[102692]: pgmap v132: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 26 MiB/s wr, 48 op/s 2023-09-10T00:44:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:44:58 smithi131 ceph-mon[103713]: pgmap v132: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 26 MiB/s wr, 48 op/s 2023-09-10T00:44:59.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:44:58 smithi044 ceph-mon[98676]: pgmap v132: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 26 MiB/s wr, 48 op/s 2023-09-10T00:44:59.687 INFO:teuthology.orchestra.run.smithi044.stderr:22400000 of 50000000 tuples (44%) done (elapsed 239.32 s, remaining 294.88 s) 2023-09-10T00:45:00.658 INFO:teuthology.orchestra.run.smithi044.stderr:22500000 of 50000000 tuples (45%) done (elapsed 240.29 s, remaining 293.69 s) 2023-09-10T00:45:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:00 smithi099 ceph-mon[102692]: pgmap v133: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 26 MiB/s wr, 48 op/s 2023-09-10T00:45:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:00 smithi131 ceph-mon[103713]: pgmap v133: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 26 MiB/s wr, 48 op/s 2023-09-10T00:45:01.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:00 smithi044 ceph-mon[98676]: pgmap v133: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 26 MiB/s wr, 48 op/s 2023-09-10T00:45:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:02 smithi099 ceph-mon[102692]: pgmap v134: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 39 MiB/s wr, 74 op/s 2023-09-10T00:45:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:02 smithi131 ceph-mon[103713]: pgmap v134: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 39 MiB/s wr, 74 op/s 2023-09-10T00:45:03.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:02 smithi044 ceph-mon[98676]: pgmap v134: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 39 MiB/s wr, 74 op/s 2023-09-10T00:45:04.044 INFO:teuthology.orchestra.run.smithi044.stderr:22600000 of 50000000 tuples (45%) done (elapsed 243.68 s, remaining 295.43 s) 2023-09-10T00:45:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:04 smithi099 ceph-mon[102692]: pgmap v135: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 56 op/s 2023-09-10T00:45:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:04 smithi131 ceph-mon[103713]: pgmap v135: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 56 op/s 2023-09-10T00:45:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:04 smithi044 ceph-mon[98676]: pgmap v135: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 56 op/s 2023-09-10T00:45:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.574+0000 7f8bcd621700 1 -- 172.21.15.44:0/3511523916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bc8100320 msgr2=0x7f8bc81040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.574+0000 7f8bcd621700 1 --2- 172.21.15.44:0/3511523916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bc8100320 0x7f8bc81040c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009990 tx=0x7f8bbc009ca0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.574+0000 7f8bcd621700 1 -- 172.21.15.44:0/3511523916 shutdown_connections 2023-09-10T00:45:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.574+0000 7f8bcd621700 1 --2- 172.21.15.44:0/3511523916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bc81047f0 0x7f8bc810c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.574+0000 7f8bcd621700 1 --2- 172.21.15.44:0/3511523916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc80ff970 0x7f8bc80ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.574+0000 7f8bcd621700 1 --2- 172.21.15.44:0/3511523916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bc8100320 0x7f8bc81040c0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.574+0000 7f8bcd621700 1 -- 172.21.15.44:0/3511523916 >> 172.21.15.44:0/3511523916 conn(0x7f8bc80773b0 msgr2=0x7f8bc80777c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.574+0000 7f8bcd621700 1 -- 172.21.15.44:0/3511523916 shutdown_connections 2023-09-10T00:45:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.574+0000 7f8bcd621700 1 -- 172.21.15.44:0/3511523916 wait complete. 2023-09-10T00:45:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.575+0000 7f8bcd621700 1 Processor -- start 2023-09-10T00:45:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.575+0000 7f8bcd621700 1 -- start start 2023-09-10T00:45:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.575+0000 7f8bcd621700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bc80ff970 0x7f8bc8101340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.575+0000 7f8bcd621700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bc8100320 0x7f8bc8101880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.575+0000 7f8bcd621700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc81047f0 0x7f8bc81a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.575+0000 7f8bcd621700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a4d90 con 0x7f8bc8100320 2023-09-10T00:45:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.575+0000 7f8bcd621700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a4ed0 con 0x7f8bc80ff970 2023-09-10T00:45:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.575+0000 7f8bcd621700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a5040 con 0x7f8bc81047f0 2023-09-10T00:45:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.575+0000 7f8bc67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bc8100320 0x7f8bc8101880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.575+0000 7f8bc6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bc80ff970 0x7f8bc8101340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8bc77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc81047f0 0x7f8bc81a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8bc67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bc8100320 0x7f8bc8101880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50800/0 (socket says 172.21.15.44:50800) 2023-09-10T00:45:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8bc6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bc80ff970 0x7f8bc8101340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52538/0 (socket says 172.21.15.44:52538) 2023-09-10T00:45:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8bc67fc700 1 -- 172.21.15.44:0/3458876676 learned_addr learned my addr 172.21.15.44:0/3458876676 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8bc67fc700 1 -- 172.21.15.44:0/3458876676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc81047f0 msgr2=0x7f8bc81a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8bc67fc700 1 --2- 172.21.15.44:0/3458876676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc81047f0 0x7f8bc81a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8bc67fc700 1 -- 172.21.15.44:0/3458876676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bc80ff970 msgr2=0x7f8bc8101340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8bc67fc700 1 --2- 172.21.15.44:0/3458876676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bc80ff970 0x7f8bc8101340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8bc67fc700 1 -- 172.21.15.44:0/3458876676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc0095f0 con 0x7f8bc8100320 2023-09-10T00:45:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8bc77fe700 1 --2- 172.21.15.44:0/3458876676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc81047f0 0x7f8bc81a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8bc6ffd700 1 --2- 172.21.15.44:0/3458876676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bc80ff970 0x7f8bc8101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8bc67fc700 1 --2- 172.21.15.44:0/3458876676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bc8100320 0x7f8bc8101880 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc000c00 tx=0x7f8bbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8baffff700 1 -- 172.21.15.44:0/3458876676 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc01d070 con 0x7f8bc8100320 2023-09-10T00:45:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8bcd621700 1 -- 172.21.15.44:0/3458876676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc81a52c0 con 0x7f8bc8100320 2023-09-10T00:45:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8baffff700 1 -- 172.21.15.44:0/3458876676 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bbc022470 con 0x7f8bc8100320 2023-09-10T00:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8baffff700 1 -- 172.21.15.44:0/3458876676 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc017650 con 0x7f8bc8100320 2023-09-10T00:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.576+0000 7f8bcd621700 1 -- 172.21.15.44:0/3458876676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc81a56e0 con 0x7f8bc8100320 2023-09-10T00:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.579+0000 7f8baffff700 1 -- 172.21.15.44:0/3458876676 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f8bbc0225e0 con 0x7f8bc8100320 2023-09-10T00:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.579+0000 7f8baffff700 1 --2- 172.21.15.44:0/3458876676 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8bb00a07b0 0x7f8bb00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.579+0000 7f8baffff700 1 -- 172.21.15.44:0/3458876676 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f8bbc0cd820 con 0x7f8bc8100320 2023-09-10T00:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.579+0000 7f8bc6ffd700 1 --2- 172.21.15.44:0/3458876676 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8bb00a07b0 0x7f8bb00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.580+0000 7f8bc6ffd700 1 --2- 172.21.15.44:0/3458876676 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8bb00a07b0 0x7f8bb00a2c70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8011490 tx=0x7f8bb8009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.580+0000 7f8bd97e7b80 1 Processor -- start 2023-09-10T00:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.580+0000 7f8bd97e7b80 1 -- start start 2023-09-10T00:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.580+0000 7f8bd97e7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d94a41a0 0x55a1d94a4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.580+0000 7f8bd97e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949e300 0x55a1d949e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.580+0000 7f8bd97e7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949ecc0 0x55a1d94aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.580+0000 7f8bd97e7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d93f76d0 con 0x55a1d949ecc0 2023-09-10T00:45:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.580+0000 7f8bd97e7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d93f4ff0 con 0x55a1d94a41a0 2023-09-10T00:45:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.580+0000 7f8bd97e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d93f08e0 con 0x55a1d949e300 2023-09-10T00:45:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.580+0000 7f8bc6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d94a41a0 0x55a1d94a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.580+0000 7f8bc6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d94a41a0 0x55a1d94a4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52546/0 (socket says 172.21.15.44:52546) 2023-09-10T00:45:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.580+0000 7f8bc6ffd700 1 -- 172.21.15.44:0/3163523223 learned_addr learned my addr 172.21.15.44:0/3163523223 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.580+0000 7f8bc77fe700 1 --2- 172.21.15.44:0/3163523223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949ecc0 0x55a1d94aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.580+0000 7f8bc67fc700 1 --2- 172.21.15.44:0/3163523223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949e300 0x55a1d949e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.581+0000 7f8bc6ffd700 1 -- 172.21.15.44:0/3163523223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949e300 msgr2=0x55a1d949e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.581+0000 7f8bc6ffd700 1 --2- 172.21.15.44:0/3163523223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949e300 0x55a1d949e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.581+0000 7f8bc6ffd700 1 -- 172.21.15.44:0/3163523223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949ecc0 msgr2=0x55a1d94aaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.581+0000 7f8bc6ffd700 1 --2- 172.21.15.44:0/3163523223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949ecc0 0x55a1d94aaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.581+0000 7f8bc6ffd700 1 -- 172.21.15.44:0/3163523223 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1d921cd80 con 0x55a1d94a41a0 2023-09-10T00:45:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.581+0000 7f8bc77fe700 1 --2- 172.21.15.44:0/3163523223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949ecc0 0x55a1d94aaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.581+0000 7f8bc6ffd700 1 --2- 172.21.15.44:0/3163523223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d94a41a0 0x55a1d94a4580 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80043c0 tx=0x7f8bb8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6c969a187498994 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.581+0000 7f8bad7fa700 1 -- 172.21.15.44:0/3163523223 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8003d10 con 0x55a1d94a41a0 2023-09-10T00:45:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.581+0000 7f8bad7fa700 1 -- 172.21.15.44:0/3163523223 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bb8003e70 con 0x55a1d94a41a0 2023-09-10T00:45:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.581+0000 7f8bad7fa700 1 -- 172.21.15.44:0/3163523223 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb801e710 con 0x55a1d94a41a0 2023-09-10T00:45:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.582+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/3163523223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d94a41a0 msgr2=0x55a1d94a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.582+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/3163523223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d94a41a0 0x55a1d94a4580 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80043c0 tx=0x7f8bb8014000 comp rx=0 tx=0).stop 2023-09-10T00:45:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.582+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/3163523223 shutdown_connections 2023-09-10T00:45:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.582+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/3163523223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949ecc0 0x55a1d94aaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.582+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/3163523223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d94a41a0 0x55a1d94a4580 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.582+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/3163523223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949e300 0x55a1d949e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.582+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/3163523223 >> 172.21.15.44:0/3163523223 conn(0x55a1d9413030 msgr2=0x55a1d9499560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.582+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/3163523223 shutdown_connections 2023-09-10T00:45:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.582+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/3163523223 wait complete. 2023-09-10T00:45:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.582+0000 7f8bd97e7b80 1 -- start start 2023-09-10T00:45:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.582+0000 7f8bd97e7b80 1 Processor -- start 2023-09-10T00:45:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bd97e7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d949e300 0x55a1d94aae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bd97e7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949ecc0 0x55a1d94a80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bd97e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d94a41a0 0x55a1d94a8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bd97e7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d938eb10 con 0x55a1d949ecc0 2023-09-10T00:45:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bd97e7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d938e810 con 0x55a1d949e300 2023-09-10T00:45:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bd97e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d92728c0 con 0x55a1d94a41a0 2023-09-10T00:45:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bd97e7b80 10 client.0 _ll_register_callbacks cb 0x55a1d92d8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:45:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bc67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949ecc0 0x55a1d94a80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bc6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d949e300 0x55a1d94aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bc67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949ecc0 0x55a1d94a80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50814/0 (socket says 172.21.15.44:50814) 2023-09-10T00:45:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bc67fc700 1 -- 172.21.15.44:0/1401776434 learned_addr learned my addr 172.21.15.44:0/1401776434 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bc77fe700 1 --2- 172.21.15.44:0/1401776434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d94a41a0 0x55a1d94a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bc67fc700 1 -- 172.21.15.44:0/1401776434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d94a41a0 msgr2=0x55a1d94a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bc67fc700 1 --2- 172.21.15.44:0/1401776434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d94a41a0 0x55a1d94a8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bc67fc700 1 -- 172.21.15.44:0/1401776434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d949e300 msgr2=0x55a1d94aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bc67fc700 1 --2- 172.21.15.44:0/1401776434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d949e300 0x55a1d94aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bc67fc700 1 -- 172.21.15.44:0/1401776434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc0003a60 con 0x55a1d949ecc0 2023-09-10T00:45:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bc6ffd700 1 --2- 172.21.15.44:0/1401776434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d949e300 0x55a1d94aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bc67fc700 1 --2- 172.21.15.44:0/1401776434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949ecc0 0x55a1d94a80c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc09a7e0 tx=0x7f8bbc09a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470ee7e02af7f63c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8baaff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:45:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8baaff5700 1 -- 172.21.15.44:0/1401776434 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc09e910 con 0x55a1d949ecc0 2023-09-10T00:45:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8baaff5700 1 -- 172.21.15.44:0/1401776434 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f8bbc0a2ac0 con 0x55a1d949ecc0 2023-09-10T00:45:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.583+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/1401776434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1d9348490 con 0x55a1d949ecc0 2023-09-10T00:45:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.584+0000 7f8baaff5700 1 -- 172.21.15.44:0/1401776434 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bbc099fd0 con 0x55a1d949ecc0 2023-09-10T00:45:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.584+0000 7f8bd97e7b80 10 client.16650 fetch_fsmap learned FSMap version 42 2023-09-10T00:45:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.584+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/1401776434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f8bb8017070 con 0x55a1d949ecc0 2023-09-10T00:45:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.584+0000 7f8baaff5700 1 -- 172.21.15.44:0/1401776434 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a1d9348490 con 0x55a1d949ecc0 2023-09-10T00:45:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.584+0000 7f8baaff5700 1 -- 172.21.15.44:0/1401776434 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8bbc0cd430 con 0x55a1d949ecc0 2023-09-10T00:45:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.584+0000 7f8bd97e7b80 10 client.16650 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:45:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.584+0000 7f8bd97e7b80 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:45:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.584+0000 7f8bd97e7b80 20 client.16650 populate_metadata read hostname 'smithi044' 2023-09-10T00:45:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.584+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/1401776434 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a1d94b8000 0x55a1d94ba3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.584+0000 7f8bd97e7b80 4 client.16650 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:45:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.584+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/1401776434 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1d9405840 con 0x55a1d94b8000 2023-09-10T00:45:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.584+0000 7f8bc6ffd700 1 --2- 172.21.15.44:0/1401776434 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a1d94b8000 0x55a1d94ba3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.585+0000 7f8bc6ffd700 1 --2- 172.21.15.44:0/1401776434 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a1d94b8000 0x55a1d94ba3f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc00766334c06af server_cookie=195017a0cd3e9328 in_seq=0 out_seq=0 2023-09-10T00:45:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.585+0000 7f8baaff5700 10 client.16650 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.586+0000 7f8baaff5700 1 -- 172.21.15.44:0/1401776434 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a1d9405840 con 0x55a1d94b8000 2023-09-10T00:45:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.586+0000 7f8baaff5700 10 client.16650 handle_command_reply: tid=0 2023-09-10T00:45:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.586+0000 7f8bd97e7b80 1 client.16650 shutdown 2023-09-10T00:45:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.586+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/1401776434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949ecc0 msgr2=0x55a1d94a80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.586+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/1401776434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949ecc0 0x55a1d94a80c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc09a7e0 tx=0x7f8bbc09a550 comp rx=0 tx=0).stop 2023-09-10T00:45:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.586+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/1401776434 shutdown_connections 2023-09-10T00:45:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.586+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/1401776434 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a1d94b8000 0x55a1d94ba3f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.586+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/1401776434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d94a41a0 0x55a1d94a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.586+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/1401776434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d949e300 0x55a1d94aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.586+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/1401776434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949ecc0 0x55a1d94a80c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.586+0000 7f8baaff5700 0 client.16650 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.586+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/1401776434 >> 172.21.15.44:0/1401776434 conn(0x55a1d9413030 msgr2=0x55a1d94987a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.586+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/1401776434 shutdown_connections 2023-09-10T00:45:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.586+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/1401776434 wait complete. 2023-09-10T00:45:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.586+0000 7f8bd97e7b80 20 client.16650 trim_cache size 0 max 16384 2023-09-10T00:45:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bd97e7b80 1 Processor -- start 2023-09-10T00:45:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bd97e7b80 1 -- start start 2023-09-10T00:45:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bd97e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949e300 0x55a1d94a8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bd97e7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d949ecc0 0x55a1d94aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bd97e7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d94a41a0 0x55a1d94aae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bd97e7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d949f580 con 0x55a1d94a41a0 2023-09-10T00:45:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bd97e7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d9495fc0 con 0x55a1d949ecc0 2023-09-10T00:45:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bd97e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d949a930 con 0x55a1d949e300 2023-09-10T00:45:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bc77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d94a41a0 0x55a1d94aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bc67fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d949ecc0 0x55a1d94aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bc6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949e300 0x55a1d94a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bc77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d94a41a0 0x55a1d94aae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50824/0 (socket says 172.21.15.44:50824) 2023-09-10T00:45:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bc6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949e300 0x55a1d94a8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57702/0 (socket says 172.21.15.44:57702) 2023-09-10T00:45:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bc67fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d949ecc0 0x55a1d94aa8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52562/0 (socket says 172.21.15.44:52562) 2023-09-10T00:45:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bc77fe700 1 -- 172.21.15.44:0/3273501232 learned_addr learned my addr 172.21.15.44:0/3273501232 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bc77fe700 1 -- 172.21.15.44:0/3273501232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949e300 msgr2=0x55a1d94a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bc77fe700 1 --2- 172.21.15.44:0/3273501232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949e300 0x55a1d94a8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bc77fe700 1 -- 172.21.15.44:0/3273501232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d949ecc0 msgr2=0x55a1d94aa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bc77fe700 1 --2- 172.21.15.44:0/3273501232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d949ecc0 0x55a1d94aa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bc77fe700 1 -- 172.21.15.44:0/3273501232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1d9205c00 con 0x55a1d94a41a0 2023-09-10T00:45:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.601+0000 7f8bc6ffd700 1 --2- 172.21.15.44:0/3273501232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949e300 0x55a1d94a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.602+0000 7f8bc77fe700 1 --2- 172.21.15.44:0/3273501232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d94a41a0 0x55a1d94aae10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000ea60 tx=0x7f8bc0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbca0643e99e0d98 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.602+0000 7f8badffb700 1 -- 172.21.15.44:0/3273501232 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc000ee90 con 0x55a1d94a41a0 2023-09-10T00:45:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.602+0000 7f8badffb700 1 -- 172.21.15.44:0/3273501232 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bc000f040 con 0x55a1d94a41a0 2023-09-10T00:45:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.602+0000 7f8badffb700 1 -- 172.21.15.44:0/3273501232 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0007bf0 con 0x55a1d94a41a0 2023-09-10T00:45:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.602+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/3273501232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d94a41a0 msgr2=0x55a1d94aae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.602+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/3273501232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d94a41a0 0x55a1d94aae10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000ea60 tx=0x7f8bc0009920 comp rx=0 tx=0).stop 2023-09-10T00:45:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.602+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/3273501232 shutdown_connections 2023-09-10T00:45:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.602+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/3273501232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d94a41a0 0x55a1d94aae10 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.602+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/3273501232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949e300 0x55a1d94a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.602+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/3273501232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d949ecc0 0x55a1d94aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.602+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/3273501232 >> 172.21.15.44:0/3273501232 conn(0x55a1d9413030 msgr2=0x55a1d94a7960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.602+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/3273501232 shutdown_connections 2023-09-10T00:45:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.602+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/3273501232 wait complete. 2023-09-10T00:45:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bd97e7b80 1 -- start start 2023-09-10T00:45:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bd97e7b80 1 Processor -- start 2023-09-10T00:45:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bd97e7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949e300 0x55a1d94af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bd97e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949ecc0 0x55a1d94ad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bd97e7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d94a41a0 0x55a1d94adf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bd97e7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d94ae610 con 0x55a1d949e300 2023-09-10T00:45:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bd97e7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d94ae780 con 0x55a1d94a41a0 2023-09-10T00:45:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bd97e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d950be10 con 0x55a1d949ecc0 2023-09-10T00:45:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bd97e7b80 10 client.0 _ll_register_callbacks cb 0x7f8bbc09a460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:45:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bc6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949e300 0x55a1d94af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bc6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949e300 0x55a1d94af5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50826/0 (socket says 172.21.15.44:50826) 2023-09-10T00:45:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bc6ffd700 1 -- 172.21.15.44:0/2338130651 learned_addr learned my addr 172.21.15.44:0/2338130651 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bc77fe700 1 --2- 172.21.15.44:0/2338130651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d94a41a0 0x55a1d94adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bc67fc700 1 --2- 172.21.15.44:0/2338130651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949ecc0 0x55a1d94ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bc6ffd700 1 -- 172.21.15.44:0/2338130651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949ecc0 msgr2=0x55a1d94ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bc6ffd700 1 --2- 172.21.15.44:0/2338130651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949ecc0 0x55a1d94ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bc6ffd700 1 -- 172.21.15.44:0/2338130651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d94a41a0 msgr2=0x55a1d94adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bc6ffd700 1 --2- 172.21.15.44:0/2338130651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d94a41a0 0x55a1d94adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.603+0000 7f8bc6ffd700 1 -- 172.21.15.44:0/2338130651 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1d921cd80 con 0x55a1d949e300 2023-09-10T00:45:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.604+0000 7f8bc6ffd700 1 --2- 172.21.15.44:0/2338130651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949e300 0x55a1d94af5e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005950 tx=0x7f8bb80058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7523da733528004a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.604+0000 7f8baaff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:45:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.604+0000 7f8baaff5700 1 -- 172.21.15.44:0/2338130651 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb802b070 con 0x55a1d949e300 2023-09-10T00:45:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.604+0000 7f8baaff5700 1 -- 172.21.15.44:0/2338130651 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f8bb8032b80 con 0x55a1d949e300 2023-09-10T00:45:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.604+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/2338130651 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bb8003e70 con 0x55a1d949e300 2023-09-10T00:45:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.604+0000 7f8baaff5700 1 -- 172.21.15.44:0/2338130651 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bb80092d0 con 0x55a1d949e300 2023-09-10T00:45:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.604+0000 7f8bd97e7b80 10 client.16656 fetch_fsmap learned FSMap version 42 2023-09-10T00:45:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.604+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/2338130651 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f8bc000e6e0 con 0x55a1d949e300 2023-09-10T00:45:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.604+0000 7f8baaff5700 1 -- 172.21.15.44:0/2338130651 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bb8003e70 con 0x55a1d949e300 2023-09-10T00:45:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.604+0000 7f8baaff5700 1 -- 172.21.15.44:0/2338130651 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8bb8024020 con 0x55a1d949e300 2023-09-10T00:45:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.604+0000 7f8bd97e7b80 10 client.16656 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:45:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.604+0000 7f8bd97e7b80 10 client.16656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:45:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.604+0000 7f8bd97e7b80 20 client.16656 populate_metadata read hostname 'smithi044' 2023-09-10T00:45:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.604+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/2338130651 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a1d950d420 0x55a1d950f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.604+0000 7f8bd97e7b80 4 client.16656 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:45:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.604+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/2338130651 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a1d93e98c0 con 0x55a1d950d420 2023-09-10T00:45:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.605+0000 7f8bc67fc700 1 --2- 172.21.15.44:0/2338130651 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a1d950d420 0x55a1d950f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.605+0000 7f8baaff5700 10 client.16656 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.605+0000 7f8bc67fc700 1 --2- 172.21.15.44:0/2338130651 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a1d950d420 0x55a1d950f840 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d443278bb8b962f server_cookie=4c576e4c4eb85c85 in_seq=0 out_seq=0 2023-09-10T00:45:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.617+0000 7f8baaff5700 1 -- 172.21.15.44:0/2338130651 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a1d93e98c0 con 0x55a1d950d420 2023-09-10T00:45:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.617+0000 7f8baaff5700 10 client.16656 handle_command_reply: tid=0 2023-09-10T00:45:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.617+0000 7f8bd97e7b80 1 client.16656 shutdown 2023-09-10T00:45:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.617+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/2338130651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949e300 msgr2=0x55a1d94af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.617+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/2338130651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949e300 0x55a1d94af5e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005950 tx=0x7f8bb80058e0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.617+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/2338130651 shutdown_connections 2023-09-10T00:45:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.617+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/2338130651 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a1d950d420 0x55a1d950f840 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.617+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/2338130651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1d94a41a0 0x55a1d94adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.617+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/2338130651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1d949e300 0x55a1d94af5e0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.617+0000 7f8bd97e7b80 1 --2- 172.21.15.44:0/2338130651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1d949ecc0 0x55a1d94ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.617+0000 7f8baaff5700 0 client.16656 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.617+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/2338130651 >> 172.21.15.44:0/2338130651 conn(0x55a1d9413030 msgr2=0x55a1d94a6890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.617+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/2338130651 shutdown_connections 2023-09-10T00:45:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.617+0000 7f8bd97e7b80 1 -- 172.21.15.44:0/2338130651 wait complete. 2023-09-10T00:45:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.617+0000 7f8bd97e7b80 20 client.16656 trim_cache size 0 max 16384 2023-09-10T00:45:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:45:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:45:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:45:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:45:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.618+0000 7f8bcd621700 1 -- 172.21.15.44:0/3458876676 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8bb00a07b0 msgr2=0x7f8bb00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.618+0000 7f8bcd621700 1 --2- 172.21.15.44:0/3458876676 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8bb00a07b0 0x7f8bb00a2c70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8011490 tx=0x7f8bb8009260 comp rx=0 tx=0).stop 2023-09-10T00:45:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.618+0000 7f8bcd621700 1 -- 172.21.15.44:0/3458876676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bc8100320 msgr2=0x7f8bc8101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.618+0000 7f8bcd621700 1 --2- 172.21.15.44:0/3458876676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bc8100320 0x7f8bc8101880 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc000c00 tx=0x7f8bbc004730 comp rx=0 tx=0).stop 2023-09-10T00:45:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.618+0000 7f8bcd621700 1 -- 172.21.15.44:0/3458876676 shutdown_connections 2023-09-10T00:45:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.618+0000 7f8bcd621700 1 --2- 172.21.15.44:0/3458876676 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8bb00a07b0 0x7f8bb00a2c70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.618+0000 7f8bcd621700 1 --2- 172.21.15.44:0/3458876676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc81047f0 0x7f8bc81a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.618+0000 7f8bcd621700 1 --2- 172.21.15.44:0/3458876676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bc80ff970 0x7f8bc8101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.618+0000 7f8bcd621700 1 --2- 172.21.15.44:0/3458876676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bc8100320 0x7f8bc8101880 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.618+0000 7f8bcd621700 1 -- 172.21.15.44:0/3458876676 >> 172.21.15.44:0/3458876676 conn(0x7f8bc80773b0 msgr2=0x7f8bc8068aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.618+0000 7f8bcd621700 1 -- 172.21.15.44:0/3458876676 shutdown_connections 2023-09-10T00:45:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:05.618+0000 7f8bcd621700 1 -- 172.21.15.44:0/3458876676 wait complete. 2023-09-10T00:45:07.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:06 smithi099 ceph-mon[102692]: pgmap v136: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 28 MiB/s wr, 85 op/s 2023-09-10T00:45:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:06 smithi131 ceph-mon[103713]: pgmap v136: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 28 MiB/s wr, 85 op/s 2023-09-10T00:45:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:06 smithi044 ceph-mon[98676]: pgmap v136: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 28 MiB/s wr, 85 op/s 2023-09-10T00:45:07.767 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:45:07.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.912+0000 7fd719def700 1 -- 172.21.15.44:0/60235092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd714100580 msgr2=0x7fd714100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:07.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.912+0000 7fd719def700 1 --2- 172.21.15.44:0/60235092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd714100580 0x7fd714100a00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd708009920 tx=0x7fd708009c30 comp rx=0 tx=0).stop 2023-09-10T00:45:07.916 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.912+0000 7fd719def700 1 -- 172.21.15.44:0/60235092 shutdown_connections 2023-09-10T00:45:07.917 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.912+0000 7fd719def700 1 --2- 172.21.15.44:0/60235092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd714100f40 0x7fd71410d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.917 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.912+0000 7fd719def700 1 --2- 172.21.15.44:0/60235092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd714106640 0x7fd714106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.917 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.912+0000 7fd719def700 1 --2- 172.21.15.44:0/60235092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd714100580 0x7fd714100a00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.917 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.912+0000 7fd719def700 1 -- 172.21.15.44:0/60235092 >> 172.21.15.44:0/60235092 conn(0x7fd7140fc070 msgr2=0x7fd7140fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:07.917 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.912+0000 7fd719def700 1 -- 172.21.15.44:0/60235092 shutdown_connections 2023-09-10T00:45:07.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.912+0000 7fd719def700 1 -- 172.21.15.44:0/60235092 wait complete. 2023-09-10T00:45:07.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.913+0000 7fd719def700 1 Processor -- start 2023-09-10T00:45:07.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.913+0000 7fd719def700 1 -- start start 2023-09-10T00:45:07.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.913+0000 7fd719def700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd714100580 0x7fd7141965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.913+0000 7fd719def700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd714100f40 0x7fd714196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.913+0000 7fd719def700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd714106640 0x7fd71419af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.913+0000 7fd719def700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71419b590 con 0x7fd714106640 2023-09-10T00:45:07.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.913+0000 7fd719def700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71419b700 con 0x7fd714100f40 2023-09-10T00:45:07.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.913+0000 7fd719def700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71419b8c0 con 0x7fd714100580 2023-09-10T00:45:07.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.913+0000 7fd713fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd714106640 0x7fd71419af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.913+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd714100580 0x7fd7141965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.913+0000 7fd712ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd714100f40 0x7fd714196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.913+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd714100580 0x7fd7141965b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57720/0 (socket says 172.21.15.44:57720) 2023-09-10T00:45:07.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.913+0000 7fd7137fe700 1 -- 172.21.15.44:0/2834852064 learned_addr learned my addr 172.21.15.44:0/2834852064 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:07.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.914+0000 7fd713fff700 1 -- 172.21.15.44:0/2834852064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd714100580 msgr2=0x7fd7141965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:07.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.914+0000 7fd713fff700 1 --2- 172.21.15.44:0/2834852064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd714100580 0x7fd7141965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.914+0000 7fd713fff700 1 -- 172.21.15.44:0/2834852064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd714100f40 msgr2=0x7fd714196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:07.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.914+0000 7fd713fff700 1 --2- 172.21.15.44:0/2834852064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd714100f40 0x7fd714196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.914+0000 7fd713fff700 1 -- 172.21.15.44:0/2834852064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd708009520 con 0x7fd714106640 2023-09-10T00:45:07.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.914+0000 7fd7137fe700 1 --2- 172.21.15.44:0/2834852064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd714100580 0x7fd7141965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:07.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.914+0000 7fd712ffd700 1 --2- 172.21.15.44:0/2834852064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd714100f40 0x7fd714196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:07.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.914+0000 7fd713fff700 1 --2- 172.21.15.44:0/2834852064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd714106640 0x7fd71419af50 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00efc0 tx=0x7fd70c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:07.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.914+0000 7fd710ff9700 1 -- 172.21.15.44:0/2834852064 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c018070 con 0x7fd714106640 2023-09-10T00:45:07.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.914+0000 7fd710ff9700 1 -- 172.21.15.44:0/2834852064 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd70c00f040 con 0x7fd714106640 2023-09-10T00:45:07.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.914+0000 7fd710ff9700 1 -- 172.21.15.44:0/2834852064 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c00cba0 con 0x7fd714106640 2023-09-10T00:45:07.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.914+0000 7fd719def700 1 -- 172.21.15.44:0/2834852064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71419bb40 con 0x7fd714106640 2023-09-10T00:45:07.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.914+0000 7fd719def700 1 -- 172.21.15.44:0/2834852064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7141a33e0 con 0x7fd714106640 2023-09-10T00:45:07.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.917+0000 7fd710ff9700 1 -- 172.21.15.44:0/2834852064 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fd70c003a00 con 0x7fd714106640 2023-09-10T00:45:07.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.917+0000 7fd710ff9700 1 --2- 172.21.15.44:0/2834852064 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd6fc0a0830 0x7fd6fc0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.917+0000 7fd710ff9700 1 -- 172.21.15.44:0/2834852064 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fd70c0cbd20 con 0x7fd714106640 2023-09-10T00:45:07.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.917+0000 7fd7137fe700 1 --2- 172.21.15.44:0/2834852064 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd6fc0a0830 0x7fd6fc0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.918+0000 7fd7137fe700 1 --2- 172.21.15.44:0/2834852064 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd6fc0a0830 0x7fd6fc0a2cf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd704011550 tx=0x7fd704009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:07.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.918+0000 7fd725fb5b80 1 Processor -- start 2023-09-10T00:45:07.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.918+0000 7fd725fb5b80 1 -- start start 2023-09-10T00:45:07.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.918+0000 7fd725fb5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d635710 0x55e19d635af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.918+0000 7fd725fb5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d636030 0x55e19d643d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.918+0000 7fd725fb5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d644250 0x55e19d646640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.918+0000 7fd725fb5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19d58e730 con 0x55e19d635710 2023-09-10T00:45:07.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.918+0000 7fd725fb5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19d58c050 con 0x55e19d644250 2023-09-10T00:45:07.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.918+0000 7fd725fb5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19d587940 con 0x55e19d636030 2023-09-10T00:45:07.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.918+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d635710 0x55e19d635af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.918+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d635710 0x55e19d635af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50846/0 (socket says 172.21.15.44:50846) 2023-09-10T00:45:07.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.918+0000 7fd7137fe700 1 -- 172.21.15.44:0/3505702964 learned_addr learned my addr 172.21.15.44:0/3505702964 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:07.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.918+0000 7fd713fff700 1 --2- 172.21.15.44:0/3505702964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d644250 0x55e19d646640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.918+0000 7fd712ffd700 1 --2- 172.21.15.44:0/3505702964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d636030 0x55e19d643d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.919+0000 7fd7137fe700 1 -- 172.21.15.44:0/3505702964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d636030 msgr2=0x55e19d643d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:07.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.919+0000 7fd7137fe700 1 --2- 172.21.15.44:0/3505702964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d636030 0x55e19d643d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.919+0000 7fd7137fe700 1 -- 172.21.15.44:0/3505702964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d644250 msgr2=0x55e19d646640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:07.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.919+0000 7fd7137fe700 1 --2- 172.21.15.44:0/3505702964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d644250 0x55e19d646640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.919+0000 7fd7137fe700 1 -- 172.21.15.44:0/3505702964 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e19d3b3de0 con 0x55e19d635710 2023-09-10T00:45:07.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.919+0000 7fd713fff700 1 --2- 172.21.15.44:0/3505702964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d644250 0x55e19d646640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:07.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.919+0000 7fd7137fe700 1 --2- 172.21.15.44:0/3505702964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d635710 0x55e19d635af0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd704009880 tx=0x7fd704014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162990da2d24070c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.919+0000 7fd6f9ffb700 1 -- 172.21.15.44:0/3505702964 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704026630 con 0x55e19d635710 2023-09-10T00:45:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.919+0000 7fd6f9ffb700 1 -- 172.21.15.44:0/3505702964 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd704021070 con 0x55e19d635710 2023-09-10T00:45:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.919+0000 7fd6f9ffb700 1 -- 172.21.15.44:0/3505702964 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70401d670 con 0x55e19d635710 2023-09-10T00:45:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.920+0000 7fd725fb5b80 1 -- 172.21.15.44:0/3505702964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d635710 msgr2=0x55e19d635af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:07.940 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.920+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/3505702964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d635710 0x55e19d635af0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd704009880 tx=0x7fd704014000 comp rx=0 tx=0).stop 2023-09-10T00:45:07.940 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.920+0000 7fd725fb5b80 1 -- 172.21.15.44:0/3505702964 shutdown_connections 2023-09-10T00:45:07.940 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.920+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/3505702964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d644250 0x55e19d646640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.940 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.920+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/3505702964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d635710 0x55e19d635af0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.941 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.920+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/3505702964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d636030 0x55e19d643d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.941 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.920+0000 7fd725fb5b80 1 -- 172.21.15.44:0/3505702964 >> 172.21.15.44:0/3505702964 conn(0x55e19d5aa090 msgr2=0x55e19d630bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:07.941 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.920+0000 7fd725fb5b80 1 -- 172.21.15.44:0/3505702964 shutdown_connections 2023-09-10T00:45:07.941 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.920+0000 7fd725fb5b80 1 -- 172.21.15.44:0/3505702964 wait complete. 2023-09-10T00:45:07.941 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.920+0000 7fd725fb5b80 1 -- start start 2023-09-10T00:45:07.942 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd725fb5b80 1 Processor -- start 2023-09-10T00:45:07.942 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd725fb5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d635710 0x55e19d643b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.942 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd725fb5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d636030 0x55e19d640e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd725fb5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d644250 0x55e19d641430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd725fb5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19d525b70 con 0x55e19d644250 2023-09-10T00:45:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd725fb5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19d525870 con 0x55e19d636030 2023-09-10T00:45:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd725fb5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19d409920 con 0x55e19d635710 2023-09-10T00:45:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd725fb5b80 10 client.0 _ll_register_callbacks cb 0x55e19d46fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:45:07.944 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd712ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d636030 0x55e19d640e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.944 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd713fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d644250 0x55e19d641430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.944 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d635710 0x55e19d643b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.944 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd712ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d636030 0x55e19d640e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52594/0 (socket says 172.21.15.44:52594) 2023-09-10T00:45:07.945 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd713fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d644250 0x55e19d641430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50856/0 (socket says 172.21.15.44:50856) 2023-09-10T00:45:07.945 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d635710 0x55e19d643b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57734/0 (socket says 172.21.15.44:57734) 2023-09-10T00:45:07.945 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd712ffd700 1 -- 172.21.15.44:0/1263709082 learned_addr learned my addr 172.21.15.44:0/1263709082 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:07.945 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd713fff700 1 -- 172.21.15.44:0/1263709082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d635710 msgr2=0x55e19d643b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:07.946 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd713fff700 1 --2- 172.21.15.44:0/1263709082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d635710 0x55e19d643b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.946 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd713fff700 1 -- 172.21.15.44:0/1263709082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d636030 msgr2=0x55e19d640e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:07.946 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd713fff700 1 --2- 172.21.15.44:0/1263709082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d636030 0x55e19d640e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.946 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd713fff700 1 -- 172.21.15.44:0/1263709082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7141a33e0 con 0x55e19d644250 2023-09-10T00:45:07.947 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.921+0000 7fd7137fe700 1 --2- 172.21.15.44:0/1263709082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d635710 0x55e19d643b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:07.947 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd712ffd700 1 --2- 172.21.15.44:0/1263709082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d636030 0x55e19d640e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:07.947 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd713fff700 1 --2- 172.21.15.44:0/1263709082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d644250 0x55e19d641430 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0c90d0 tx=0x7fd70c095310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c780703ee4940c57 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:07.947 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd6f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:45:07.948 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd6f2ffd700 1 -- 172.21.15.44:0/1263709082 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0b0070 con 0x55e19d644250 2023-09-10T00:45:07.948 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd6f2ffd700 1 -- 172.21.15.44:0/1263709082 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fd70c098ec0 con 0x55e19d644250 2023-09-10T00:45:07.948 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd725fb5b80 1 -- 172.21.15.44:0/1263709082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e19d4df4f0 con 0x55e19d644250 2023-09-10T00:45:07.948 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd6f2ffd700 1 -- 172.21.15.44:0/1263709082 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd70c0b1260 con 0x55e19d644250 2023-09-10T00:45:07.949 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd725fb5b80 10 client.16668 fetch_fsmap learned FSMap version 42 2023-09-10T00:45:07.949 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd725fb5b80 1 -- 172.21.15.44:0/1263709082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fd704017070 con 0x55e19d644250 2023-09-10T00:45:07.949 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd6f2ffd700 1 -- 172.21.15.44:0/1263709082 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e19d4df4f0 con 0x55e19d644250 2023-09-10T00:45:07.949 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd6f2ffd700 1 -- 172.21.15.44:0/1263709082 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fd70c0a6080 con 0x55e19d644250 2023-09-10T00:45:07.949 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd725fb5b80 10 client.16668 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:45:07.949 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd725fb5b80 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:45:07.950 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd725fb5b80 20 client.16668 populate_metadata read hostname 'smithi044' 2023-09-10T00:45:07.950 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/1263709082 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e19d653b70 0x55e19d655f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.950 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd725fb5b80 4 client.16668 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:45:07.950 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.922+0000 7fd725fb5b80 1 -- 172.21.15.44:0/1263709082 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e19d59c8a0 con 0x55e19d653b70 2023-09-10T00:45:07.950 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.923+0000 7fd7137fe700 1 --2- 172.21.15.44:0/1263709082 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e19d653b70 0x55e19d655f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.951 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.923+0000 7fd7137fe700 1 --2- 172.21.15.44:0/1263709082 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e19d653b70 0x55e19d655f90 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3cff095a23b7578 server_cookie=7805dd58b1f84a16 in_seq=0 out_seq=0 2023-09-10T00:45:07.951 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.923+0000 7fd6f2ffd700 10 client.16668 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:07.951 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.924+0000 7fd6f2ffd700 1 -- 172.21.15.44:0/1263709082 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e19d59c8a0 con 0x55e19d653b70 2023-09-10T00:45:07.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.924+0000 7fd6f2ffd700 10 client.16668 handle_command_reply: tid=0 2023-09-10T00:45:07.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.924+0000 7fd725fb5b80 1 client.16668 shutdown 2023-09-10T00:45:07.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.924+0000 7fd725fb5b80 1 -- 172.21.15.44:0/1263709082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d644250 msgr2=0x55e19d641430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:07.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.924+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/1263709082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d644250 0x55e19d641430 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0c90d0 tx=0x7fd70c095310 comp rx=0 tx=0).stop 2023-09-10T00:45:07.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.924+0000 7fd725fb5b80 1 -- 172.21.15.44:0/1263709082 shutdown_connections 2023-09-10T00:45:07.952 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.924+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/1263709082 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e19d653b70 0x55e19d655f90 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.924+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/1263709082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d644250 0x55e19d641430 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.924+0000 7fd6f2ffd700 0 client.16668 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:07.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.924+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/1263709082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d635710 0x55e19d643b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.953 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.924+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/1263709082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d636030 0x55e19d640e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.924+0000 7fd725fb5b80 1 -- 172.21.15.44:0/1263709082 >> 172.21.15.44:0/1263709082 conn(0x55e19d5aa090 msgr2=0x55e19d6307e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:07.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.924+0000 7fd725fb5b80 1 -- 172.21.15.44:0/1263709082 shutdown_connections 2023-09-10T00:45:07.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.924+0000 7fd725fb5b80 1 -- 172.21.15.44:0/1263709082 wait complete. 2023-09-10T00:45:07.954 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.924+0000 7fd725fb5b80 20 client.16668 trim_cache size 0 max 16384 2023-09-10T00:45:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd725fb5b80 1 Processor -- start 2023-09-10T00:45:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd725fb5b80 1 -- start start 2023-09-10T00:45:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd725fb5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d635710 0x55e19d63c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd725fb5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d636030 0x55e19d63c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd725fb5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d644250 0x55e19d640630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd725fb5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19d646b60 con 0x55e19d636030 2023-09-10T00:45:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd725fb5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19d63bce0 con 0x55e19d635710 2023-09-10T00:45:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd725fb5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19d6369f0 con 0x55e19d644250 2023-09-10T00:45:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d635710 0x55e19d63c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd712ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d636030 0x55e19d63c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d635710 0x55e19d63c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52600/0 (socket says 172.21.15.44:52600) 2023-09-10T00:45:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd712ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d636030 0x55e19d63c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50862/0 (socket says 172.21.15.44:50862) 2023-09-10T00:45:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd7137fe700 1 -- 172.21.15.44:0/1618646744 learned_addr learned my addr 172.21.15.44:0/1618646744 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd713fff700 1 --2- 172.21.15.44:0/1618646744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d644250 0x55e19d640630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd7137fe700 1 -- 172.21.15.44:0/1618646744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d644250 msgr2=0x55e19d640630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd7137fe700 1 --2- 172.21.15.44:0/1618646744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d644250 0x55e19d640630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd7137fe700 1 -- 172.21.15.44:0/1618646744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d636030 msgr2=0x55e19d63c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd7137fe700 1 --2- 172.21.15.44:0/1618646744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d636030 0x55e19d63c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd7137fe700 1 -- 172.21.15.44:0/1618646744 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e19d39cc60 con 0x55e19d635710 2023-09-10T00:45:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd713fff700 1 --2- 172.21.15.44:0/1618646744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d644250 0x55e19d640630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.941+0000 7fd712ffd700 1 --2- 172.21.15.44:0/1618646744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d636030 0x55e19d63c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.942+0000 7fd7137fe700 1 --2- 172.21.15.44:0/1618646744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d635710 0x55e19d63c130 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd704009cf0 tx=0x7fd704011cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8c55ffa492e6d2a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.942+0000 7fd6fa7fc700 1 -- 172.21.15.44:0/1618646744 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70401ce60 con 0x55e19d635710 2023-09-10T00:45:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.942+0000 7fd6fa7fc700 1 -- 172.21.15.44:0/1618646744 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd704033170 con 0x55e19d635710 2023-09-10T00:45:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.942+0000 7fd6fa7fc700 1 -- 172.21.15.44:0/1618646744 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704005460 con 0x55e19d635710 2023-09-10T00:45:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.942+0000 7fd725fb5b80 1 -- 172.21.15.44:0/1618646744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d635710 msgr2=0x55e19d63c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.942+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/1618646744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d635710 0x55e19d63c130 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd704009cf0 tx=0x7fd704011cc0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.942+0000 7fd725fb5b80 1 -- 172.21.15.44:0/1618646744 shutdown_connections 2023-09-10T00:45:07.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.942+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/1618646744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d644250 0x55e19d640630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.942+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/1618646744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d635710 0x55e19d63c130 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.966 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.942+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/1618646744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d636030 0x55e19d63c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.942+0000 7fd725fb5b80 1 -- 172.21.15.44:0/1618646744 >> 172.21.15.44:0/1618646744 conn(0x55e19d5aa090 msgr2=0x55e19d64f550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:07.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.942+0000 7fd725fb5b80 1 -- 172.21.15.44:0/1618646744 shutdown_connections 2023-09-10T00:45:07.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.943+0000 7fd725fb5b80 1 -- 172.21.15.44:0/1618646744 wait complete. 2023-09-10T00:45:07.967 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.943+0000 7fd725fb5b80 1 -- start start 2023-09-10T00:45:07.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.943+0000 7fd725fb5b80 1 Processor -- start 2023-09-10T00:45:07.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.943+0000 7fd725fb5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d635710 0x55e19d64abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.943+0000 7fd725fb5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d636030 0x55e19d648f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.943+0000 7fd725fb5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d644250 0x55e19d649440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.968 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.943+0000 7fd725fb5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19d649980 con 0x55e19d635710 2023-09-10T00:45:07.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.943+0000 7fd725fb5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19d649af0 con 0x55e19d644250 2023-09-10T00:45:07.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.943+0000 7fd725fb5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19d649cb0 con 0x55e19d636030 2023-09-10T00:45:07.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.943+0000 7fd725fb5b80 10 client.0 _ll_register_callbacks cb 0x7fd70c098c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:45:07.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.943+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d635710 0x55e19d64abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.969 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.943+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d635710 0x55e19d64abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50866/0 (socket says 172.21.15.44:50866) 2023-09-10T00:45:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.943+0000 7fd7137fe700 1 -- 172.21.15.44:0/17577621 learned_addr learned my addr 172.21.15.44:0/17577621 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.943+0000 7fd712ffd700 1 --2- 172.21.15.44:0/17577621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d636030 0x55e19d648f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.943+0000 7fd713fff700 1 --2- 172.21.15.44:0/17577621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d644250 0x55e19d649440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.944+0000 7fd7137fe700 1 -- 172.21.15.44:0/17577621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d636030 msgr2=0x55e19d648f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:07.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.944+0000 7fd7137fe700 1 --2- 172.21.15.44:0/17577621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d636030 0x55e19d648f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.944+0000 7fd7137fe700 1 -- 172.21.15.44:0/17577621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d644250 msgr2=0x55e19d649440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:07.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.944+0000 7fd7137fe700 1 --2- 172.21.15.44:0/17577621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d644250 0x55e19d649440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.971 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.944+0000 7fd7137fe700 1 -- 172.21.15.44:0/17577621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71410dd00 con 0x55e19d635710 2023-09-10T00:45:07.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.944+0000 7fd7137fe700 1 --2- 172.21.15.44:0/17577621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d635710 0x55e19d64abc0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd70400e6a0 tx=0x7fd70401cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e736b7c7d64e3b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:07.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.944+0000 7fd6f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:45:07.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.944+0000 7fd6f2ffd700 1 -- 172.21.15.44:0/17577621 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704033460 con 0x55e19d635710 2023-09-10T00:45:07.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.944+0000 7fd6f2ffd700 1 -- 172.21.15.44:0/17577621 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fd704033a90 con 0x55e19d635710 2023-09-10T00:45:07.972 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.944+0000 7fd725fb5b80 1 -- 172.21.15.44:0/17577621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd704021070 con 0x55e19d635710 2023-09-10T00:45:07.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.944+0000 7fd6f2ffd700 1 -- 172.21.15.44:0/17577621 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7040335c0 con 0x55e19d635710 2023-09-10T00:45:07.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.944+0000 7fd725fb5b80 10 client.16674 fetch_fsmap learned FSMap version 42 2023-09-10T00:45:07.973 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.944+0000 7fd725fb5b80 1 -- 172.21.15.44:0/17577621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fd704017070 con 0x55e19d635710 2023-09-10T00:45:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.944+0000 7fd6f2ffd700 1 -- 172.21.15.44:0/17577621 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd704021070 con 0x55e19d635710 2023-09-10T00:45:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.945+0000 7fd6f2ffd700 1 -- 172.21.15.44:0/17577621 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fd70403a070 con 0x55e19d635710 2023-09-10T00:45:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.945+0000 7fd725fb5b80 10 client.16674 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:45:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.945+0000 7fd725fb5b80 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:45:07.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.945+0000 7fd725fb5b80 20 client.16674 populate_metadata read hostname 'smithi044' 2023-09-10T00:45:07.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.945+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/17577621 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e19d6a8b40 0x55e19d6aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:07.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.945+0000 7fd725fb5b80 4 client.16674 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:45:07.975 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.945+0000 7fd725fb5b80 1 -- 172.21.15.44:0/17577621 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e19d580920 con 0x55e19d6a8b40 2023-09-10T00:45:07.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.945+0000 7fd712ffd700 1 --2- 172.21.15.44:0/17577621 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e19d6a8b40 0x55e19d6aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:07.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.945+0000 7fd6f2ffd700 10 client.16674 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:07.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.945+0000 7fd712ffd700 1 --2- 172.21.15.44:0/17577621 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e19d6a8b40 0x55e19d6aaf30 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1c3581eb891c76f server_cookie=94cb3d0602fc19e7 in_seq=0 out_seq=0 2023-09-10T00:45:07.976 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.945+0000 7fd6f2ffd700 1 -- 172.21.15.44:0/17577621 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e19d580920 con 0x55e19d6a8b40 2023-09-10T00:45:07.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.945+0000 7fd6f2ffd700 10 client.16674 handle_command_reply: tid=0 2023-09-10T00:45:07.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.945+0000 7fd725fb5b80 1 client.16674 shutdown 2023-09-10T00:45:07.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd725fb5b80 1 -- 172.21.15.44:0/17577621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d635710 msgr2=0x55e19d64abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:07.977 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/17577621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d635710 0x55e19d64abc0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd70400e6a0 tx=0x7fd70401cfa0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd725fb5b80 1 -- 172.21.15.44:0/17577621 shutdown_connections 2023-09-10T00:45:07.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/17577621 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e19d6a8b40 0x55e19d6aaf30 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/17577621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e19d644250 0x55e19d649440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.978 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd6f2ffd700 0 client.16674 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:07.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/17577621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19d635710 0x55e19d64abc0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd725fb5b80 1 --2- 172.21.15.44:0/17577621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e19d636030 0x55e19d648f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd725fb5b80 1 -- 172.21.15.44:0/17577621 >> 172.21.15.44:0/17577621 conn(0x55e19d5aa090 msgr2=0x55e19d63c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:07.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd725fb5b80 1 -- 172.21.15.44:0/17577621 shutdown_connections 2023-09-10T00:45:07.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd725fb5b80 1 -- 172.21.15.44:0/17577621 wait complete. 2023-09-10T00:45:07.979 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd725fb5b80 20 client.16674 trim_cache size 0 max 16384 2023-09-10T00:45:07.980 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:45:07.980 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:45:07.980 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:45:07.981 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:45:07.981 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:45:07.981 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:45:07.981 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:45:07.981 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:45:07.981 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:45:07.982 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:45:07.983 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd719def700 1 -- 172.21.15.44:0/2834852064 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd6fc0a0830 msgr2=0x7fd6fc0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:07.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd719def700 1 --2- 172.21.15.44:0/2834852064 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd6fc0a0830 0x7fd6fc0a2cf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd704011550 tx=0x7fd704009260 comp rx=0 tx=0).stop 2023-09-10T00:45:07.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd719def700 1 -- 172.21.15.44:0/2834852064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd714106640 msgr2=0x7fd71419af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:07.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd719def700 1 --2- 172.21.15.44:0/2834852064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd714106640 0x7fd71419af50 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00efc0 tx=0x7fd70c00c3a0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd719def700 1 -- 172.21.15.44:0/2834852064 shutdown_connections 2023-09-10T00:45:07.984 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd719def700 1 --2- 172.21.15.44:0/2834852064 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd6fc0a0830 0x7fd6fc0a2cf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd719def700 1 --2- 172.21.15.44:0/2834852064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd714106640 0x7fd71419af50 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd719def700 1 --2- 172.21.15.44:0/2834852064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd714100580 0x7fd7141965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd719def700 1 --2- 172.21.15.44:0/2834852064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd714100f40 0x7fd714196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:07.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd719def700 1 -- 172.21.15.44:0/2834852064 >> 172.21.15.44:0/2834852064 conn(0x7fd7140fc070 msgr2=0x7fd7140fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:07.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd719def700 1 -- 172.21.15.44:0/2834852064 shutdown_connections 2023-09-10T00:45:07.985 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:07.946+0000 7fd719def700 1 -- 172.21.15.44:0/2834852064 wait complete. 2023-09-10T00:45:07.986 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:45:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:08 smithi099 ceph-mon[102692]: pgmap v137: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 19 MiB/s wr, 55 op/s 2023-09-10T00:45:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:08 smithi131 ceph-mon[103713]: pgmap v137: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 19 MiB/s wr, 55 op/s 2023-09-10T00:45:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:08 smithi044 ceph-mon[98676]: pgmap v137: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 19 MiB/s wr, 55 op/s 2023-09-10T00:45:09.594 INFO:teuthology.orchestra.run.smithi044.stderr:22700000 of 50000000 tuples (45%) done (elapsed 249.23 s, remaining 299.73 s) 2023-09-10T00:45:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:10 smithi099 ceph-mon[102692]: pgmap v138: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 19 MiB/s wr, 55 op/s 2023-09-10T00:45:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:10 smithi131 ceph-mon[103713]: pgmap v138: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 19 MiB/s wr, 55 op/s 2023-09-10T00:45:11.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:10 smithi044 ceph-mon[98676]: pgmap v138: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 19 MiB/s wr, 55 op/s 2023-09-10T00:45:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:12 smithi099 ceph-mon[102692]: pgmap v139: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 22 MiB/s wr, 68 op/s 2023-09-10T00:45:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:12 smithi131 ceph-mon[103713]: pgmap v139: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 22 MiB/s wr, 68 op/s 2023-09-10T00:45:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:12 smithi044 ceph-mon[98676]: pgmap v139: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 22 MiB/s wr, 68 op/s 2023-09-10T00:45:14.132 INFO:teuthology.orchestra.run.smithi044.stderr:22800000 of 50000000 tuples (45%) done (elapsed 253.77 s, remaining 302.74 s) 2023-09-10T00:45:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:14 smithi099 ceph-mon[102692]: pgmap v140: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 9.6 MiB/s wr, 42 op/s 2023-09-10T00:45:15.177 INFO:teuthology.orchestra.run.smithi044.stderr:22900000 of 50000000 tuples (45%) done (elapsed 254.81 s, remaining 301.55 s) 2023-09-10T00:45:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:14 smithi131 ceph-mon[103713]: pgmap v140: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 9.6 MiB/s wr, 42 op/s 2023-09-10T00:45:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:14 smithi044 ceph-mon[98676]: pgmap v140: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 9.6 MiB/s wr, 42 op/s 2023-09-10T00:45:16.192 INFO:teuthology.orchestra.run.smithi044.stderr:23000000 of 50000000 tuples (46%) done (elapsed 255.83 s, remaining 300.32 s) 2023-09-10T00:45:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:16 smithi099 ceph-mon[102692]: pgmap v141: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 14 MiB/s wr, 62 op/s 2023-09-10T00:45:17.157 INFO:teuthology.orchestra.run.smithi044.stderr:23100000 of 50000000 tuples (46%) done (elapsed 256.79 s, remaining 299.03 s) 2023-09-10T00:45:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:16 smithi131 ceph-mon[103713]: pgmap v141: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 14 MiB/s wr, 62 op/s 2023-09-10T00:45:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:16 smithi044 ceph-mon[98676]: pgmap v141: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 14 MiB/s wr, 62 op/s 2023-09-10T00:45:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.740+0000 7febae40a700 1 -- 172.21.15.44:0/2675371206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feba81055b0 msgr2=0x7feba8105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.740+0000 7febae40a700 1 --2- 172.21.15.44:0/2675371206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feba81055b0 0x7feba8105990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feb98009920 tx=0x7feb98009c30 comp rx=0 tx=0).stop 2023-09-10T00:45:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.741+0000 7febae40a700 1 -- 172.21.15.44:0/2675371206 shutdown_connections 2023-09-10T00:45:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.741+0000 7febae40a700 1 --2- 172.21.15.44:0/2675371206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba8100120 0x7feba810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.741+0000 7febae40a700 1 --2- 172.21.15.44:0/2675371206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feba81055b0 0x7feba8105990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.741+0000 7febae40a700 1 --2- 172.21.15.44:0/2675371206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feba80ff760 0x7feba80ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.741+0000 7febae40a700 1 -- 172.21.15.44:0/2675371206 >> 172.21.15.44:0/2675371206 conn(0x7feba8074f30 msgr2=0x7feba8075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.741+0000 7febae40a700 1 -- 172.21.15.44:0/2675371206 shutdown_connections 2023-09-10T00:45:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.741+0000 7febae40a700 1 -- 172.21.15.44:0/2675371206 wait complete. 2023-09-10T00:45:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.741+0000 7febae40a700 1 Processor -- start 2023-09-10T00:45:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.742+0000 7febae40a700 1 -- start start 2023-09-10T00:45:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.742+0000 7febae40a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ff760 0x7feba8197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.742+0000 7febae40a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feba8100120 0x7feba8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.742+0000 7febae40a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feba81055b0 0x7feba81a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.742+0000 7febae40a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba81a2ae0 con 0x7feba80ff760 2023-09-10T00:45:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.742+0000 7febae40a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba81a2c50 con 0x7feba81055b0 2023-09-10T00:45:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.742+0000 7febae40a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba81a2e10 con 0x7feba8100120 2023-09-10T00:45:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.742+0000 7feba7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ff760 0x7feba8197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.742+0000 7feba7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ff760 0x7feba8197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50880/0 (socket says 172.21.15.44:50880) 2023-09-10T00:45:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.742+0000 7febac9a7700 1 --2- 172.21.15.44:0/1121996009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feba81055b0 0x7feba81a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.742+0000 7feba77fe700 1 --2- 172.21.15.44:0/1121996009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feba8100120 0x7feba8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.742+0000 7feba7fff700 1 -- 172.21.15.44:0/1121996009 learned_addr learned my addr 172.21.15.44:0/1121996009 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.743+0000 7feba7fff700 1 -- 172.21.15.44:0/1121996009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feba8100120 msgr2=0x7feba8197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.743+0000 7feba7fff700 1 --2- 172.21.15.44:0/1121996009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feba8100120 0x7feba8197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.743+0000 7feba7fff700 1 -- 172.21.15.44:0/1121996009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feba81055b0 msgr2=0x7feba81a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.743+0000 7feba7fff700 1 --2- 172.21.15.44:0/1121996009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feba81055b0 0x7feba81a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.743+0000 7feba7fff700 1 -- 172.21.15.44:0/1121996009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb98009520 con 0x7feba80ff760 2023-09-10T00:45:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.743+0000 7feba77fe700 1 --2- 172.21.15.44:0/1121996009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feba8100120 0x7feba8197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.743+0000 7feba7fff700 1 --2- 172.21.15.44:0/1121996009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ff760 0x7feba8197640 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7feb98009840 tx=0x7feb9800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.743+0000 7feba57fa700 1 -- 172.21.15.44:0/1121996009 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9801c070 con 0x7feba80ff760 2023-09-10T00:45:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.743+0000 7febae40a700 1 -- 172.21.15.44:0/1121996009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba81a3090 con 0x7feba80ff760 2023-09-10T00:45:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.743+0000 7feba57fa700 1 -- 172.21.15.44:0/1121996009 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb9800ba70 con 0x7feba80ff760 2023-09-10T00:45:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.743+0000 7feba57fa700 1 -- 172.21.15.44:0/1121996009 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb98003680 con 0x7feba80ff760 2023-09-10T00:45:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.743+0000 7febae40a700 1 -- 172.21.15.44:0/1121996009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba81a34b0 con 0x7feba80ff760 2023-09-10T00:45:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.747+0000 7febba5d0b80 1 Processor -- start 2023-09-10T00:45:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.747+0000 7febba5d0b80 1 -- start start 2023-09-10T00:45:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.747+0000 7febba5d0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 0x55c5d82c6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.747+0000 7febba5d0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0300 0x55c5d82c0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.747+0000 7febba5d0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0cc0 0x55c5d82ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.747+0000 7febba5d0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d82196d0 con 0x55c5d82c61a0 2023-09-10T00:45:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.747+0000 7febba5d0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d8216ff0 con 0x55c5d82c0300 2023-09-10T00:45:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.747+0000 7febba5d0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d82128e0 con 0x55c5d82c0cc0 2023-09-10T00:45:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.748+0000 7febac9a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 0x55c5d82c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.748+0000 7feba7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0300 0x55c5d82c0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.748+0000 7feba7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0300 0x55c5d82c0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52628/0 (socket says 172.21.15.44:52628) 2023-09-10T00:45:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.748+0000 7feba7fff700 1 -- 172.21.15.44:0/848704114 learned_addr learned my addr 172.21.15.44:0/848704114 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.748+0000 7feba77fe700 1 --2- 172.21.15.44:0/848704114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0cc0 0x55c5d82ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.748+0000 7febac9a7700 1 -- 172.21.15.44:0/848704114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0cc0 msgr2=0x55c5d82ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.748+0000 7febac9a7700 1 --2- 172.21.15.44:0/848704114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0cc0 0x55c5d82ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.748+0000 7febac9a7700 1 -- 172.21.15.44:0/848704114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0300 msgr2=0x55c5d82c0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.748+0000 7febac9a7700 1 --2- 172.21.15.44:0/848704114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0300 0x55c5d82c0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.748+0000 7febac9a7700 1 -- 172.21.15.44:0/848704114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5d803ed80 con 0x55c5d82c61a0 2023-09-10T00:45:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.748+0000 7feba7fff700 1 --2- 172.21.15.44:0/848704114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0300 0x55c5d82c0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.748+0000 7feba77fe700 1 --2- 172.21.15.44:0/848704114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0cc0 0x55c5d82ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.748+0000 7febac9a7700 1 --2- 172.21.15.44:0/848704114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 0x55c5d82c6580 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7feba0011580 tx=0x7feba00099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9bc2b52da99a6d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.748+0000 7feb8e7fc700 1 -- 172.21.15.44:0/848704114 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba000e040 con 0x55c5d82c61a0 2023-09-10T00:45:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.748+0000 7feb8e7fc700 1 -- 172.21.15.44:0/848704114 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feba000f040 con 0x55c5d82c61a0 2023-09-10T00:45:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.748+0000 7feb8e7fc700 1 -- 172.21.15.44:0/848704114 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0004290 con 0x55c5d82c61a0 2023-09-10T00:45:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.749+0000 7febba5d0b80 1 -- 172.21.15.44:0/848704114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 msgr2=0x55c5d82c6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.749+0000 7febba5d0b80 1 --2- 172.21.15.44:0/848704114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 0x55c5d82c6580 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7feba0011580 tx=0x7feba00099e0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.749+0000 7febba5d0b80 1 -- 172.21.15.44:0/848704114 shutdown_connections 2023-09-10T00:45:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.749+0000 7febba5d0b80 1 --2- 172.21.15.44:0/848704114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0cc0 0x55c5d82ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.749+0000 7febba5d0b80 1 --2- 172.21.15.44:0/848704114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 0x55c5d82c6580 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.749+0000 7febba5d0b80 1 --2- 172.21.15.44:0/848704114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0300 0x55c5d82c0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.749+0000 7febba5d0b80 1 -- 172.21.15.44:0/848704114 >> 172.21.15.44:0/848704114 conn(0x55c5d8235030 msgr2=0x55c5d82bb560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.749+0000 7feba57fa700 1 -- 172.21.15.44:0/1121996009 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7feb98020030 con 0x7feba80ff760 2023-09-10T00:45:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.749+0000 7febba5d0b80 1 -- 172.21.15.44:0/848704114 shutdown_connections 2023-09-10T00:45:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.749+0000 7febba5d0b80 1 -- 172.21.15.44:0/848704114 wait complete. 2023-09-10T00:45:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7febba5d0b80 1 -- start start 2023-09-10T00:45:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7febba5d0b80 1 Processor -- start 2023-09-10T00:45:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7feba57fa700 1 --2- 172.21.15.44:0/1121996009 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb900a06c0 0x7feb900a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7febba5d0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0300 0x55c5d82ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7febba5d0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0cc0 0x55c5d82ca000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7febac9a7700 1 --2- 172.21.15.44:0/1121996009 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb900a06c0 0x7feb900a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7febba5d0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 0x55c5d82ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7febba5d0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d81b0b10 con 0x55c5d82c61a0 2023-09-10T00:45:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7febba5d0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d81b0810 con 0x55c5d82c0cc0 2023-09-10T00:45:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7febba5d0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d80948c0 con 0x55c5d82c0300 2023-09-10T00:45:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7feba57fa700 1 -- 172.21.15.44:0/1121996009 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7feb980cd470 con 0x7feba80ff760 2023-09-10T00:45:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7febba5d0b80 10 client.0 _ll_register_callbacks cb 0x55c5d80faa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:45:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7febac9a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 0x55c5d82ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7feba7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0300 0x55c5d82ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7feba7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0300 0x55c5d82ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57778/0 (socket says 172.21.15.44:57778) 2023-09-10T00:45:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7feba7fff700 1 -- 172.21.15.44:0/280436146 learned_addr learned my addr 172.21.15.44:0/280436146 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7febac9a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 0x55c5d82ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50894/0 (socket says 172.21.15.44:50894) 2023-09-10T00:45:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.750+0000 7feba77fe700 1 --2- 172.21.15.44:0/280436146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0cc0 0x55c5d82ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.751+0000 7feba7fff700 1 -- 172.21.15.44:0/280436146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0cc0 msgr2=0x55c5d82ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.751+0000 7feba7fff700 1 --2- 172.21.15.44:0/280436146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0cc0 0x55c5d82ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.751+0000 7feba7fff700 1 -- 172.21.15.44:0/280436146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 msgr2=0x55c5d82ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.751+0000 7feba7fff700 1 --2- 172.21.15.44:0/280436146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 0x55c5d82ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.751+0000 7feba7fff700 1 -- 172.21.15.44:0/280436146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba81a34b0 con 0x55c5d82c0300 2023-09-10T00:45:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.751+0000 7febac9a7700 1 --2- 172.21.15.44:0/1121996009 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb900a06c0 0x7feb900a2b80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feba00062a0 tx=0x7feba0003ae0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.751+0000 7feba7fff700 1 --2- 172.21.15.44:0/280436146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0300 0x55c5d82ccdf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb98005e90 tx=0x7feb9809a2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75d4149695d8b81c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.751+0000 7feb877fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:45:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.751+0000 7feb877fe700 1 -- 172.21.15.44:0/280436146 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb98031040 con 0x55c5d82c0300 2023-09-10T00:45:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.751+0000 7feb877fe700 1 -- 172.21.15.44:0/280436146 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7feb9809ae30 con 0x55c5d82c0300 2023-09-10T00:45:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.751+0000 7febba5d0b80 1 -- 172.21.15.44:0/280436146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5d816a490 con 0x55c5d82c0300 2023-09-10T00:45:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.751+0000 7feb877fe700 1 -- 172.21.15.44:0/280436146 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb9809a730 con 0x55c5d82c0300 2023-09-10T00:45:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.751+0000 7febba5d0b80 10 client.26293 fetch_fsmap learned FSMap version 42 2023-09-10T00:45:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.751+0000 7febba5d0b80 1 -- 172.21.15.44:0/280436146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7feb9c009c90 con 0x55c5d82c0300 2023-09-10T00:45:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.751+0000 7feb877fe700 1 -- 172.21.15.44:0/280436146 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c5d816a490 con 0x55c5d82c0300 2023-09-10T00:45:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.752+0000 7feb877fe700 1 -- 172.21.15.44:0/280436146 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7feb98096e40 con 0x55c5d82c0300 2023-09-10T00:45:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.752+0000 7febba5d0b80 10 client.26293 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:45:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.752+0000 7febba5d0b80 10 client.26293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:45:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.752+0000 7febba5d0b80 20 client.26293 populate_metadata read hostname 'smithi044' 2023-09-10T00:45:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.752+0000 7febba5d0b80 1 --2- 172.21.15.44:0/280436146 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c5d82d9f80 0x55c5d82dc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.752+0000 7febba5d0b80 4 client.26293 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:45:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.752+0000 7febba5d0b80 1 -- 172.21.15.44:0/280436146 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5d8227840 con 0x55c5d82d9f80 2023-09-10T00:45:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.752+0000 7feba77fe700 1 --2- 172.21.15.44:0/280436146 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c5d82d9f80 0x55c5d82dc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.752+0000 7feb877fe700 10 client.26293 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.752+0000 7feba77fe700 1 --2- 172.21.15.44:0/280436146 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c5d82d9f80 0x55c5d82dc370 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d77a72bff239fa9 server_cookie=4f819051484ec436 in_seq=0 out_seq=0 2023-09-10T00:45:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.753+0000 7feb877fe700 1 -- 172.21.15.44:0/280436146 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c5d8227840 con 0x55c5d82d9f80 2023-09-10T00:45:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.753+0000 7feb877fe700 10 client.26293 handle_command_reply: tid=0 2023-09-10T00:45:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.753+0000 7febba5d0b80 1 client.26293 shutdown 2023-09-10T00:45:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.754+0000 7febba5d0b80 1 -- 172.21.15.44:0/280436146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0300 msgr2=0x55c5d82ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.754+0000 7febba5d0b80 1 --2- 172.21.15.44:0/280436146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0300 0x55c5d82ccdf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb98005e90 tx=0x7feb9809a2f0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.754+0000 7febba5d0b80 1 -- 172.21.15.44:0/280436146 shutdown_connections 2023-09-10T00:45:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.754+0000 7febba5d0b80 1 --2- 172.21.15.44:0/280436146 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c5d82d9f80 0x55c5d82dc370 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.754+0000 7febba5d0b80 1 --2- 172.21.15.44:0/280436146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 0x55c5d82ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.754+0000 7febba5d0b80 1 --2- 172.21.15.44:0/280436146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0300 0x55c5d82ccdf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.754+0000 7feb877fe700 0 client.26293 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.754+0000 7febba5d0b80 1 --2- 172.21.15.44:0/280436146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0cc0 0x55c5d82ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.754+0000 7febba5d0b80 1 -- 172.21.15.44:0/280436146 >> 172.21.15.44:0/280436146 conn(0x55c5d8235030 msgr2=0x55c5d82ba7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.754+0000 7febba5d0b80 1 -- 172.21.15.44:0/280436146 shutdown_connections 2023-09-10T00:45:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.754+0000 7febba5d0b80 1 -- 172.21.15.44:0/280436146 wait complete. 2023-09-10T00:45:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.754+0000 7febba5d0b80 20 client.26293 trim_cache size 0 max 16384 2023-09-10T00:45:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.769+0000 7febba5d0b80 1 Processor -- start 2023-09-10T00:45:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.769+0000 7febba5d0b80 1 -- start start 2023-09-10T00:45:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7febba5d0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c0300 0x55c5d82d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7febba5d0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0cc0 0x55c5d82bfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7febba5d0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c61a0 0x55c5d82cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7febba5d0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d82b7fc0 con 0x55c5d82c0300 2023-09-10T00:45:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7febba5d0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d82bbad0 con 0x55c5d82c0cc0 2023-09-10T00:45:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7febba5d0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d82c6580 con 0x55c5d82c61a0 2023-09-10T00:45:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7feba7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c0300 0x55c5d82d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7febac9a7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c61a0 0x55c5d82cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7feba7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c0300 0x55c5d82d1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50902/0 (socket says 172.21.15.44:50902) 2023-09-10T00:45:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7feba7fff700 1 -- 172.21.15.44:0/1498638771 learned_addr learned my addr 172.21.15.44:0/1498638771 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7feba77fe700 1 --2- 172.21.15.44:0/1498638771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0cc0 0x55c5d82bfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7feba7fff700 1 -- 172.21.15.44:0/1498638771 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c61a0 msgr2=0x55c5d82cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7feba7fff700 1 --2- 172.21.15.44:0/1498638771 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c61a0 0x55c5d82cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7feba7fff700 1 -- 172.21.15.44:0/1498638771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0cc0 msgr2=0x55c5d82bfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7feba7fff700 1 --2- 172.21.15.44:0/1498638771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0cc0 0x55c5d82bfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7feba7fff700 1 -- 172.21.15.44:0/1498638771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba0011230 con 0x55c5d82c0300 2023-09-10T00:45:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7feba7fff700 1 --2- 172.21.15.44:0/1498638771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c0300 0x55c5d82d1530 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7feb9809e120 tx=0x7feb98019f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389d155e3f584630 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.770+0000 7feb8effd700 1 -- 172.21.15.44:0/1498638771 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb980cab40 con 0x55c5d82c0300 2023-09-10T00:45:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.771+0000 7feb8effd700 1 -- 172.21.15.44:0/1498638771 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb9809a090 con 0x55c5d82c0300 2023-09-10T00:45:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.771+0000 7feb8effd700 1 -- 172.21.15.44:0/1498638771 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9809a7a0 con 0x55c5d82c0300 2023-09-10T00:45:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.771+0000 7febba5d0b80 1 -- 172.21.15.44:0/1498638771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c0300 msgr2=0x55c5d82d1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.771+0000 7febba5d0b80 1 --2- 172.21.15.44:0/1498638771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c0300 0x55c5d82d1530 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7feb9809e120 tx=0x7feb98019f40 comp rx=0 tx=0).stop 2023-09-10T00:45:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.771+0000 7febba5d0b80 1 -- 172.21.15.44:0/1498638771 shutdown_connections 2023-09-10T00:45:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.771+0000 7febba5d0b80 1 --2- 172.21.15.44:0/1498638771 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c61a0 0x55c5d82cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.771+0000 7febba5d0b80 1 --2- 172.21.15.44:0/1498638771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c0300 0x55c5d82d1530 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.771+0000 7febba5d0b80 1 --2- 172.21.15.44:0/1498638771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0cc0 0x55c5d82bfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.771+0000 7febba5d0b80 1 -- 172.21.15.44:0/1498638771 >> 172.21.15.44:0/1498638771 conn(0x55c5d8235030 msgr2=0x55c5d82bf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.771+0000 7febba5d0b80 1 -- 172.21.15.44:0/1498638771 shutdown_connections 2023-09-10T00:45:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.771+0000 7febba5d0b80 1 -- 172.21.15.44:0/1498638771 wait complete. 2023-09-10T00:45:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.772+0000 7febba5d0b80 1 -- start start 2023-09-10T00:45:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.772+0000 7febba5d0b80 1 Processor -- start 2023-09-10T00:45:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.772+0000 7febba5d0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0300 0x55c5d8321e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.772+0000 7febba5d0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0cc0 0x55c5d83223a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.772+0000 7febba5d0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 0x55c5d832ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.772+0000 7febba5d0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d8322b30 con 0x55c5d82c61a0 2023-09-10T00:45:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.772+0000 7febba5d0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d832d260 con 0x55c5d82c0cc0 2023-09-10T00:45:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.772+0000 7febba5d0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d832d420 con 0x55c5d82c0300 2023-09-10T00:45:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.772+0000 7febba5d0b80 10 client.0 _ll_register_callbacks cb 0x7feb98009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:45:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7febac9a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 0x55c5d832ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7feba7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0300 0x55c5d8321e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7feba77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0cc0 0x55c5d83223a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7febac9a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 0x55c5d832ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50910/0 (socket says 172.21.15.44:50910) 2023-09-10T00:45:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7feba7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0300 0x55c5d8321e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57790/0 (socket says 172.21.15.44:57790) 2023-09-10T00:45:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7febac9a7700 1 -- 172.21.15.44:0/270983159 learned_addr learned my addr 172.21.15.44:0/270983159 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7febac9a7700 1 -- 172.21.15.44:0/270983159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0300 msgr2=0x55c5d8321e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7febac9a7700 1 --2- 172.21.15.44:0/270983159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0300 0x55c5d8321e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7febac9a7700 1 -- 172.21.15.44:0/270983159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0cc0 msgr2=0x55c5d83223a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7febac9a7700 1 --2- 172.21.15.44:0/270983159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0cc0 0x55c5d83223a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7febac9a7700 1 -- 172.21.15.44:0/270983159 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb9c009c90 con 0x55c5d82c61a0 2023-09-10T00:45:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7feba77fe700 1 --2- 172.21.15.44:0/270983159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0cc0 0x55c5d83223a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7feba7fff700 1 --2- 172.21.15.44:0/270983159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0300 0x55c5d8321e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7febac9a7700 1 --2- 172.21.15.44:0/270983159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 0x55c5d832ccf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7feba0011b70 tx=0x7feba0029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a765bf79748959d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7feb877fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:45:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7feb877fe700 1 -- 172.21.15.44:0/270983159 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0007e90 con 0x55c5d82c61a0 2023-09-10T00:45:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7feb877fe700 1 -- 172.21.15.44:0/270983159 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7feba0004040 con 0x55c5d82c61a0 2023-09-10T00:45:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.773+0000 7febba5d0b80 1 -- 172.21.15.44:0/270983159 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feba000f040 con 0x55c5d82c61a0 2023-09-10T00:45:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.774+0000 7feb877fe700 1 -- 172.21.15.44:0/270983159 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7feba0004200 con 0x55c5d82c61a0 2023-09-10T00:45:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.774+0000 7febba5d0b80 10 client.16692 fetch_fsmap learned FSMap version 42 2023-09-10T00:45:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.774+0000 7febba5d0b80 1 -- 172.21.15.44:0/270983159 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55c5d8027c00 con 0x55c5d82c61a0 2023-09-10T00:45:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.774+0000 7feb877fe700 1 -- 172.21.15.44:0/270983159 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feba000f040 con 0x55c5d82c61a0 2023-09-10T00:45:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.774+0000 7feb877fe700 1 -- 172.21.15.44:0/270983159 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7feba0033020 con 0x55c5d82c61a0 2023-09-10T00:45:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.774+0000 7febba5d0b80 10 client.16692 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:45:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.774+0000 7febba5d0b80 10 client.16692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:45:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.774+0000 7febba5d0b80 20 client.16692 populate_metadata read hostname 'smithi044' 2023-09-10T00:45:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.774+0000 7febba5d0b80 1 --2- 172.21.15.44:0/270983159 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c5d832ea00 0x55c5d8330e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.774+0000 7febba5d0b80 4 client.16692 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:45:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.774+0000 7febba5d0b80 1 -- 172.21.15.44:0/270983159 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c5d820b8c0 con 0x55c5d832ea00 2023-09-10T00:45:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.774+0000 7feba7fff700 1 --2- 172.21.15.44:0/270983159 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c5d832ea00 0x55c5d8330e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.775+0000 7feb877fe700 10 client.16692 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.775+0000 7feba7fff700 1 --2- 172.21.15.44:0/270983159 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c5d832ea00 0x55c5d8330e20 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd8e6a642b9efe2e server_cookie=a98f339acacfb546 in_seq=0 out_seq=0 2023-09-10T00:45:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.786+0000 7feb877fe700 1 -- 172.21.15.44:0/270983159 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c5d820b8c0 con 0x55c5d832ea00 2023-09-10T00:45:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.786+0000 7feb877fe700 10 client.16692 handle_command_reply: tid=0 2023-09-10T00:45:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.786+0000 7febba5d0b80 1 client.16692 shutdown 2023-09-10T00:45:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.786+0000 7febba5d0b80 1 -- 172.21.15.44:0/270983159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 msgr2=0x55c5d832ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.786+0000 7febba5d0b80 1 --2- 172.21.15.44:0/270983159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 0x55c5d832ccf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7feba0011b70 tx=0x7feba0029000 comp rx=0 tx=0).stop 2023-09-10T00:45:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.786+0000 7febba5d0b80 1 -- 172.21.15.44:0/270983159 shutdown_connections 2023-09-10T00:45:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.786+0000 7febba5d0b80 1 --2- 172.21.15.44:0/270983159 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c5d832ea00 0x55c5d8330e20 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.786+0000 7febba5d0b80 1 --2- 172.21.15.44:0/270983159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d82c61a0 0x55c5d832ccf0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.786+0000 7febba5d0b80 1 --2- 172.21.15.44:0/270983159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5d82c0300 0x55c5d8321e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.786+0000 7febba5d0b80 1 --2- 172.21.15.44:0/270983159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d82c0cc0 0x55c5d83223a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.786+0000 7feb877fe700 0 client.16692 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.786+0000 7febba5d0b80 1 -- 172.21.15.44:0/270983159 >> 172.21.15.44:0/270983159 conn(0x55c5d8235030 msgr2=0x55c5d82bfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.786+0000 7febba5d0b80 1 -- 172.21.15.44:0/270983159 shutdown_connections 2023-09-10T00:45:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.786+0000 7febba5d0b80 1 -- 172.21.15.44:0/270983159 wait complete. 2023-09-10T00:45:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.786+0000 7febba5d0b80 20 client.16692 trim_cache size 0 max 16384 2023-09-10T00:45:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:45:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:45:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:45:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:45:18.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.787+0000 7febae40a700 1 -- 172.21.15.44:0/1121996009 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb900a06c0 msgr2=0x7feb900a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:18.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.787+0000 7febae40a700 1 --2- 172.21.15.44:0/1121996009 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb900a06c0 0x7feb900a2b80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feba00062a0 tx=0x7feba0003ae0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.787+0000 7febae40a700 1 -- 172.21.15.44:0/1121996009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ff760 msgr2=0x7feba8197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.787+0000 7febae40a700 1 --2- 172.21.15.44:0/1121996009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ff760 0x7feba8197640 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7feb98009840 tx=0x7feb9800fa00 comp rx=0 tx=0).stop 2023-09-10T00:45:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.787+0000 7febae40a700 1 -- 172.21.15.44:0/1121996009 shutdown_connections 2023-09-10T00:45:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.787+0000 7febae40a700 1 --2- 172.21.15.44:0/1121996009 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb900a06c0 0x7feb900a2b80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.787+0000 7febae40a700 1 --2- 172.21.15.44:0/1121996009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feba81055b0 0x7feba81a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.787+0000 7febae40a700 1 --2- 172.21.15.44:0/1121996009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ff760 0x7feba8197640 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.787+0000 7febae40a700 1 --2- 172.21.15.44:0/1121996009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feba8100120 0x7feba8197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.787+0000 7febae40a700 1 -- 172.21.15.44:0/1121996009 >> 172.21.15.44:0/1121996009 conn(0x7feba8074f30 msgr2=0x7feba80fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.787+0000 7febae40a700 1 -- 172.21.15.44:0/1121996009 shutdown_connections 2023-09-10T00:45:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:18.787+0000 7febae40a700 1 -- 172.21.15.44:0/1121996009 wait complete. 2023-09-10T00:45:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:18 smithi099 ceph-mon[102692]: pgmap v142: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.0 MiB/s wr, 32 op/s 2023-09-10T00:45:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:18 smithi131 ceph-mon[103713]: pgmap v142: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.0 MiB/s wr, 32 op/s 2023-09-10T00:45:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:18 smithi044 ceph-mon[98676]: pgmap v142: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.0 MiB/s wr, 32 op/s 2023-09-10T00:45:19.984 INFO:teuthology.orchestra.run.smithi044.stderr:23200000 of 50000000 tuples (46%) done (elapsed 259.62 s, remaining 299.90 s) 2023-09-10T00:45:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:20 smithi099 ceph-mon[102692]: pgmap v143: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.0 MiB/s wr, 32 op/s 2023-09-10T00:45:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:20 smithi131 ceph-mon[103713]: pgmap v143: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.0 MiB/s wr, 32 op/s 2023-09-10T00:45:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:20 smithi044 ceph-mon[98676]: pgmap v143: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.0 MiB/s wr, 32 op/s 2023-09-10T00:45:22.261 INFO:teuthology.orchestra.run.smithi044.stderr:23300000 of 50000000 tuples (46%) done (elapsed 261.90 s, remaining 300.11 s) 2023-09-10T00:45:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:22 smithi099 ceph-mon[102692]: pgmap v144: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 17 MiB/s wr, 74 op/s 2023-09-10T00:45:23.190 INFO:teuthology.orchestra.run.smithi044.stderr:23400000 of 50000000 tuples (46%) done (elapsed 262.82 s, remaining 298.77 s) 2023-09-10T00:45:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:22 smithi131 ceph-mon[103713]: pgmap v144: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 17 MiB/s wr, 74 op/s 2023-09-10T00:45:23.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:22 smithi044 ceph-mon[98676]: pgmap v144: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 17 MiB/s wr, 74 op/s 2023-09-10T00:45:24.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:23 smithi099 ceph-mon[102692]: pgmap v145: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 61 op/s 2023-09-10T00:45:24.194 INFO:teuthology.orchestra.run.smithi044.stderr:23500000 of 50000000 tuples (47%) done (elapsed 263.83 s, remaining 297.51 s) 2023-09-10T00:45:24.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:23 smithi131 ceph-mon[103713]: pgmap v145: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 61 op/s 2023-09-10T00:45:24.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:23 smithi044 ceph-mon[98676]: pgmap v145: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 61 op/s 2023-09-10T00:45:25.181 INFO:teuthology.orchestra.run.smithi044.stderr:23600000 of 50000000 tuples (47%) done (elapsed 264.82 s, remaining 296.23 s) 2023-09-10T00:45:26.089 INFO:teuthology.orchestra.run.smithi044.stderr:23700000 of 50000000 tuples (47%) done (elapsed 265.72 s, remaining 294.87 s) 2023-09-10T00:45:26.804 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:26 smithi044 ceph-mon[98676]: pgmap v146: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 25 MiB/s wr, 104 op/s 2023-09-10T00:45:26.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:26 smithi099 ceph-mon[102692]: pgmap v146: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 25 MiB/s wr, 104 op/s 2023-09-10T00:45:26.935 INFO:teuthology.orchestra.run.smithi044.stderr:23800000 of 50000000 tuples (47%) done (elapsed 266.57 s, remaining 293.45 s) 2023-09-10T00:45:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:26 smithi131 ceph-mon[103713]: pgmap v146: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 25 MiB/s wr, 104 op/s 2023-09-10T00:45:27.828 INFO:teuthology.orchestra.run.smithi044.stderr:23900000 of 50000000 tuples (47%) done (elapsed 267.46 s, remaining 292.08 s) 2023-09-10T00:45:28.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:28 smithi044 ceph-mon[98676]: pgmap v147: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 21 MiB/s wr, 85 op/s 2023-09-10T00:45:28.831 INFO:teuthology.orchestra.run.smithi044.stderr:24000000 of 50000000 tuples (48%) done (elapsed 268.46 s, remaining 290.84 s) 2023-09-10T00:45:28.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:28 smithi099 ceph-mon[102692]: pgmap v147: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 21 MiB/s wr, 85 op/s 2023-09-10T00:45:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:28 smithi131 ceph-mon[103713]: pgmap v147: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 21 MiB/s wr, 85 op/s 2023-09-10T00:45:29.747 INFO:teuthology.orchestra.run.smithi044.stderr:24100000 of 50000000 tuples (48%) done (elapsed 269.38 s, remaining 289.50 s) 2023-09-10T00:45:30.765 INFO:teuthology.orchestra.run.smithi044.stderr:24200000 of 50000000 tuples (48%) done (elapsed 270.40 s, remaining 288.28 s) 2023-09-10T00:45:30.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:30 smithi099 ceph-mon[102692]: pgmap v148: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 21 MiB/s wr, 85 op/s 2023-09-10T00:45:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:30 smithi131 ceph-mon[103713]: pgmap v148: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 21 MiB/s wr, 85 op/s 2023-09-10T00:45:31.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:30 smithi044 ceph-mon[98676]: pgmap v148: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 21 MiB/s wr, 85 op/s 2023-09-10T00:45:31.735 INFO:teuthology.orchestra.run.smithi044.stderr:24300000 of 50000000 tuples (48%) done (elapsed 271.37 s, remaining 287.00 s) 2023-09-10T00:45:31.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.920+0000 7fbae0064700 1 -- 172.21.15.44:0/4217218831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad8100590 msgr2=0x7fbad8100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:31.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.920+0000 7fbae0064700 1 --2- 172.21.15.44:0/4217218831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad8100590 0x7fbad8100a10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbac80099c0 tx=0x7fbac8009cd0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.920+0000 7fbae0064700 1 -- 172.21.15.44:0/4217218831 shutdown_connections 2023-09-10T00:45:31.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.920+0000 7fbae0064700 1 --2- 172.21.15.44:0/4217218831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8100f50 0x7fbad810d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.920+0000 7fbae0064700 1 --2- 172.21.15.44:0/4217218831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbad8106650 0x7fbad8106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.920+0000 7fbae0064700 1 --2- 172.21.15.44:0/4217218831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad8100590 0x7fbad8100a10 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.920+0000 7fbae0064700 1 -- 172.21.15.44:0/4217218831 >> 172.21.15.44:0/4217218831 conn(0x7fbad80fc060 msgr2=0x7fbad80fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.920+0000 7fbae0064700 1 -- 172.21.15.44:0/4217218831 shutdown_connections 2023-09-10T00:45:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.920+0000 7fbae0064700 1 -- 172.21.15.44:0/4217218831 wait complete. 2023-09-10T00:45:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.921+0000 7fbae0064700 1 Processor -- start 2023-09-10T00:45:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.921+0000 7fbae0064700 1 -- start start 2023-09-10T00:45:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.921+0000 7fbae0064700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbad8100590 0x7fbad819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.921+0000 7fbae0064700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad8100f50 0x7fbad819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.921+0000 7fbae0064700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8106650 0x7fbad81963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.921+0000 7fbae0064700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8196a10 con 0x7fbad8100f50 2023-09-10T00:45:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.921+0000 7fbae0064700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8196b80 con 0x7fbad8106650 2023-09-10T00:45:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.921+0000 7fbae0064700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8196d40 con 0x7fbad8100590 2023-09-10T00:45:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.921+0000 7fbadd5ff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad8100f50 0x7fbad819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.921+0000 7fbadde00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbad8100590 0x7fbad819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.921+0000 7fbadd5ff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad8100f50 0x7fbad819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50922/0 (socket says 172.21.15.44:50922) 2023-09-10T00:45:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.921+0000 7fbadd5ff700 1 -- 172.21.15.44:0/1068271154 learned_addr learned my addr 172.21.15.44:0/1068271154 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.922+0000 7fbade601700 1 --2- 172.21.15.44:0/1068271154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8106650 0x7fbad81963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.922+0000 7fbadde00700 1 -- 172.21.15.44:0/1068271154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8106650 msgr2=0x7fbad81963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.922+0000 7fbadde00700 1 --2- 172.21.15.44:0/1068271154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8106650 0x7fbad81963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.922+0000 7fbadde00700 1 -- 172.21.15.44:0/1068271154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad8100f50 msgr2=0x7fbad819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.922+0000 7fbadde00700 1 --2- 172.21.15.44:0/1068271154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad8100f50 0x7fbad819ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.922+0000 7fbadde00700 1 -- 172.21.15.44:0/1068271154 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac80095f0 con 0x7fbad8100590 2023-09-10T00:45:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.922+0000 7fbadd5ff700 1 --2- 172.21.15.44:0/1068271154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad8100f50 0x7fbad819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.922+0000 7fbadde00700 1 --2- 172.21.15.44:0/1068271154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbad8100590 0x7fbad819c540 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbad400e9f0 tx=0x7fbad400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.922+0000 7fbaceffd700 1 -- 172.21.15.44:0/1068271154 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad400c9c0 con 0x7fbad8100590 2023-09-10T00:45:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.922+0000 7fbae0064700 1 -- 172.21.15.44:0/1068271154 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad8197020 con 0x7fbad8100590 2023-09-10T00:45:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.922+0000 7fbaceffd700 1 -- 172.21.15.44:0/1068271154 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbad4010430 con 0x7fbad8100590 2023-09-10T00:45:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.922+0000 7fbaceffd700 1 -- 172.21.15.44:0/1068271154 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad4018680 con 0x7fbad8100590 2023-09-10T00:45:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.922+0000 7fbae0064700 1 -- 172.21.15.44:0/1068271154 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad81a3530 con 0x7fbad8100590 2023-09-10T00:45:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.925+0000 7fbaceffd700 1 -- 172.21.15.44:0/1068271154 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fbad40105a0 con 0x7fbad8100590 2023-09-10T00:45:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.926+0000 7fbaceffd700 1 --2- 172.21.15.44:0/1068271154 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbac40a07b0 0x7fbac40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.926+0000 7fbaceffd700 1 -- 172.21.15.44:0/1068271154 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fbad4014070 con 0x7fbad8100590 2023-09-10T00:45:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.926+0000 7fbadd5ff700 1 --2- 172.21.15.44:0/1068271154 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbac40a07b0 0x7fbac40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.926+0000 7fbaec22ab80 1 Processor -- start 2023-09-10T00:45:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.927+0000 7fbadd5ff700 1 --2- 172.21.15.44:0/1068271154 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbac40a07b0 0x7fbac40a2c70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbac8005b20 tx=0x7fbac801a040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.927+0000 7fbaec22ab80 1 -- start start 2023-09-10T00:45:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.927+0000 7fbaec22ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb572881a0 0x55fb57288580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.927+0000 7fbaec22ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb57282780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.927+0000 7fbaec22ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb57282cc0 0x55fb5728ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.927+0000 7fbaec22ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb571db6d0 con 0x55fb57282300 2023-09-10T00:45:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.927+0000 7fbaec22ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb571d8ff0 con 0x55fb572881a0 2023-09-10T00:45:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.927+0000 7fbaec22ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb571d48e0 con 0x55fb57282cc0 2023-09-10T00:45:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.927+0000 7fbadde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb572881a0 0x55fb57288580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.927+0000 7fbadd5ff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb57282780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.927+0000 7fbade601700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb57282cc0 0x55fb5728ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.927+0000 7fbadde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb572881a0 0x55fb57288580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52668/0 (socket says 172.21.15.44:52668) 2023-09-10T00:45:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.927+0000 7fbadde00700 1 -- 172.21.15.44:0/3672149097 learned_addr learned my addr 172.21.15.44:0/3672149097 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.927+0000 7fbadd5ff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb57282780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50930/0 (socket says 172.21.15.44:50930) 2023-09-10T00:45:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.928+0000 7fbadde00700 1 -- 172.21.15.44:0/3672149097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb57282cc0 msgr2=0x55fb5728ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.928+0000 7fbadde00700 1 --2- 172.21.15.44:0/3672149097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb57282cc0 0x55fb5728ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.928+0000 7fbadde00700 1 -- 172.21.15.44:0/3672149097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 msgr2=0x55fb57282780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.928+0000 7fbadde00700 1 --2- 172.21.15.44:0/3672149097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb57282780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.928+0000 7fbadde00700 1 -- 172.21.15.44:0/3672149097 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb57000d80 con 0x55fb572881a0 2023-09-10T00:45:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.928+0000 7fbadd5ff700 1 --2- 172.21.15.44:0/3672149097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb57282780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.928+0000 7fbade601700 1 --2- 172.21.15.44:0/3672149097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb57282cc0 0x55fb5728ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.928+0000 7fbadde00700 1 --2- 172.21.15.44:0/3672149097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb572881a0 0x55fb57288580 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbad4099270 tx=0x7fbad4099130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2bd8d2f6a0d86e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.928+0000 7fbac3fff700 1 -- 172.21.15.44:0/3672149097 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad4099c70 con 0x55fb572881a0 2023-09-10T00:45:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.928+0000 7fbac3fff700 1 -- 172.21.15.44:0/3672149097 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbad40ace90 con 0x55fb572881a0 2023-09-10T00:45:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.928+0000 7fbac3fff700 1 -- 172.21.15.44:0/3672149097 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad40cc1b0 con 0x55fb572881a0 2023-09-10T00:45:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.929+0000 7fbaec22ab80 1 -- 172.21.15.44:0/3672149097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb572881a0 msgr2=0x55fb57288580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.929+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/3672149097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb572881a0 0x55fb57288580 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbad4099270 tx=0x7fbad4099130 comp rx=0 tx=0).stop 2023-09-10T00:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.929+0000 7fbaec22ab80 1 -- 172.21.15.44:0/3672149097 shutdown_connections 2023-09-10T00:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.929+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/3672149097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb57282cc0 0x55fb5728ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.929+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/3672149097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb572881a0 0x55fb57288580 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.929+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/3672149097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb57282780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.929+0000 7fbaec22ab80 1 -- 172.21.15.44:0/3672149097 >> 172.21.15.44:0/3672149097 conn(0x55fb571f7030 msgr2=0x55fb5727d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.929+0000 7fbaec22ab80 1 -- 172.21.15.44:0/3672149097 shutdown_connections 2023-09-10T00:45:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.929+0000 7fbaec22ab80 1 -- 172.21.15.44:0/3672149097 wait complete. 2023-09-10T00:45:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.929+0000 7fbaec22ab80 1 -- start start 2023-09-10T00:45:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.929+0000 7fbaec22ab80 1 Processor -- start 2023-09-10T00:45:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbaec22ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb5728ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbaec22ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282cc0 0x55fb5728c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbaec22ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb572881a0 0x55fb5728c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbaec22ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb57172b10 con 0x55fb57282300 2023-09-10T00:45:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbaec22ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb57172810 con 0x55fb57282cc0 2023-09-10T00:45:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbaec22ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb570568c0 con 0x55fb572881a0 2023-09-10T00:45:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbaec22ab80 10 client.0 _ll_register_callbacks cb 0x55fb570bca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:45:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbadde00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb5728ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbadde00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb5728ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50934/0 (socket says 172.21.15.44:50934) 2023-09-10T00:45:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbadde00700 1 -- 172.21.15.44:0/591045009 learned_addr learned my addr 172.21.15.44:0/591045009 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbadd5ff700 1 --2- 172.21.15.44:0/591045009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282cc0 0x55fb5728c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbade601700 1 --2- 172.21.15.44:0/591045009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb572881a0 0x55fb5728c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbadde00700 1 -- 172.21.15.44:0/591045009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb572881a0 msgr2=0x55fb5728c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbadde00700 1 --2- 172.21.15.44:0/591045009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb572881a0 0x55fb5728c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbadde00700 1 -- 172.21.15.44:0/591045009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282cc0 msgr2=0x55fb5728c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbadde00700 1 --2- 172.21.15.44:0/591045009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282cc0 0x55fb5728c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbadde00700 1 -- 172.21.15.44:0/591045009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad810dd10 con 0x55fb57282300 2023-09-10T00:45:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbade601700 1 --2- 172.21.15.44:0/591045009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb572881a0 0x55fb5728c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:45:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbadde00700 1 --2- 172.21.15.44:0/591045009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb5728ee10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbad40c9e10 tx=0x7fbad40c9820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b4c5f84f928113 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.930+0000 7fbac17fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:45:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.931+0000 7fbac17fa700 1 -- 172.21.15.44:0/591045009 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad40b5070 con 0x55fb57282300 2023-09-10T00:45:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.931+0000 7fbac17fa700 1 -- 172.21.15.44:0/591045009 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fbad409d3a0 con 0x55fb57282300 2023-09-10T00:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.931+0000 7fbaec22ab80 1 -- 172.21.15.44:0/591045009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb5712c490 con 0x55fb57282300 2023-09-10T00:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.931+0000 7fbac17fa700 1 -- 172.21.15.44:0/591045009 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbad40a15e0 con 0x55fb57282300 2023-09-10T00:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.931+0000 7fbaec22ab80 10 client.16704 fetch_fsmap learned FSMap version 42 2023-09-10T00:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.931+0000 7fbaec22ab80 1 -- 172.21.15.44:0/591045009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fbad0011230 con 0x55fb57282300 2023-09-10T00:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.931+0000 7fbac17fa700 1 -- 172.21.15.44:0/591045009 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fb5712c490 con 0x55fb57282300 2023-09-10T00:45:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.931+0000 7fbac17fa700 1 -- 172.21.15.44:0/591045009 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fbad40997d0 con 0x55fb57282300 2023-09-10T00:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.931+0000 7fbaec22ab80 10 client.16704 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.931+0000 7fbaec22ab80 10 client.16704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.931+0000 7fbaec22ab80 20 client.16704 populate_metadata read hostname 'smithi044' 2023-09-10T00:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.931+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/591045009 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fb5729bed0 0x55fb5729e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.931+0000 7fbaec22ab80 4 client.16704 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:45:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.931+0000 7fbaec22ab80 1 -- 172.21.15.44:0/591045009 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb571e9840 con 0x55fb5729bed0 2023-09-10T00:45:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.931+0000 7fbadd5ff700 1 --2- 172.21.15.44:0/591045009 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fb5729bed0 0x55fb5729e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.932+0000 7fbadd5ff700 1 --2- 172.21.15.44:0/591045009 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fb5729bed0 0x55fb5729e2c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c1760b5354bd4a server_cookie=80880bf445739fea in_seq=0 out_seq=0 2023-09-10T00:45:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.932+0000 7fbac17fa700 10 client.16704 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.933+0000 7fbac17fa700 1 -- 172.21.15.44:0/591045009 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fb571e9840 con 0x55fb5729bed0 2023-09-10T00:45:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.933+0000 7fbac17fa700 10 client.16704 handle_command_reply: tid=0 2023-09-10T00:45:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.933+0000 7fbaec22ab80 1 client.16704 shutdown 2023-09-10T00:45:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.933+0000 7fbaec22ab80 1 -- 172.21.15.44:0/591045009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 msgr2=0x55fb5728ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.933+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/591045009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb5728ee10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbad40c9e10 tx=0x7fbad40c9820 comp rx=0 tx=0).stop 2023-09-10T00:45:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.933+0000 7fbaec22ab80 1 -- 172.21.15.44:0/591045009 shutdown_connections 2023-09-10T00:45:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.933+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/591045009 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fb5729bed0 0x55fb5729e2c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.933+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/591045009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb572881a0 0x55fb5728c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.933+0000 7fbac17fa700 0 client.16704 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.933+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/591045009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb5728ee10 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.933+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/591045009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282cc0 0x55fb5728c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.933+0000 7fbaec22ab80 1 -- 172.21.15.44:0/591045009 >> 172.21.15.44:0/591045009 conn(0x55fb571f7030 msgr2=0x55fb5727c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.933+0000 7fbaec22ab80 1 -- 172.21.15.44:0/591045009 shutdown_connections 2023-09-10T00:45:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.933+0000 7fbaec22ab80 1 -- 172.21.15.44:0/591045009 wait complete. 2023-09-10T00:45:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.933+0000 7fbaec22ab80 20 client.16704 trim_cache size 0 max 16384 2023-09-10T00:45:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.950+0000 7fbaec22ab80 1 Processor -- start 2023-09-10T00:45:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.950+0000 7fbaec22ab80 1 -- start start 2023-09-10T00:45:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.950+0000 7fbaec22ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282300 0x55fb5728e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.950+0000 7fbaec22ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282cc0 0x55fb5727d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.950+0000 7fbaec22ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb572881a0 0x55fb5727d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.950+0000 7fbaec22ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb57279fc0 con 0x55fb57282cc0 2023-09-10T00:45:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.950+0000 7fbaec22ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5728d670 con 0x55fb57282300 2023-09-10T00:45:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.950+0000 7fbaec22ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb57288580 con 0x55fb572881a0 2023-09-10T00:45:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.951+0000 7fbadd5ff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282cc0 0x55fb5727d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.951+0000 7fbadde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282300 0x55fb5728e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.951+0000 7fbadde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282300 0x55fb5728e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52682/0 (socket says 172.21.15.44:52682) 2023-09-10T00:45:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.951+0000 7fbadd5ff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282cc0 0x55fb5727d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50944/0 (socket says 172.21.15.44:50944) 2023-09-10T00:45:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.951+0000 7fbadde00700 1 -- 172.21.15.44:0/4179004629 learned_addr learned my addr 172.21.15.44:0/4179004629 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.951+0000 7fbade601700 1 --2- 172.21.15.44:0/4179004629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb572881a0 0x55fb5727d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.951+0000 7fbadde00700 1 -- 172.21.15.44:0/4179004629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb572881a0 msgr2=0x55fb5727d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.951+0000 7fbadde00700 1 --2- 172.21.15.44:0/4179004629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb572881a0 0x55fb5727d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.951+0000 7fbadde00700 1 -- 172.21.15.44:0/4179004629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282cc0 msgr2=0x55fb5727d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.951+0000 7fbadde00700 1 --2- 172.21.15.44:0/4179004629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282cc0 0x55fb5727d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.951+0000 7fbadde00700 1 -- 172.21.15.44:0/4179004629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad81a3530 con 0x55fb57282300 2023-09-10T00:45:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.951+0000 7fbadd5ff700 1 --2- 172.21.15.44:0/4179004629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282cc0 0x55fb5727d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.951+0000 7fbadde00700 1 --2- 172.21.15.44:0/4179004629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282300 0x55fb5728e9b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbad400c5b0 tx=0x7fbad4099000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1ec782d8fdd5d0f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.951+0000 7fbaccff9700 1 -- 172.21.15.44:0/4179004629 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad40a15e0 con 0x55fb57282300 2023-09-10T00:45:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.951+0000 7fbaccff9700 1 -- 172.21.15.44:0/4179004629 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbad409d3a0 con 0x55fb57282300 2023-09-10T00:45:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.951+0000 7fbaccff9700 1 -- 172.21.15.44:0/4179004629 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad40997d0 con 0x55fb57282300 2023-09-10T00:45:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.952+0000 7fbaec22ab80 1 -- 172.21.15.44:0/4179004629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282300 msgr2=0x55fb5728e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.952+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/4179004629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282300 0x55fb5728e9b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbad400c5b0 tx=0x7fbad4099000 comp rx=0 tx=0).stop 2023-09-10T00:45:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.952+0000 7fbaec22ab80 1 -- 172.21.15.44:0/4179004629 shutdown_connections 2023-09-10T00:45:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.952+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/4179004629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb572881a0 0x55fb5727d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.952+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/4179004629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282300 0x55fb5728e9b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.952+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/4179004629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282cc0 0x55fb5727d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.952+0000 7fbaec22ab80 1 -- 172.21.15.44:0/4179004629 >> 172.21.15.44:0/4179004629 conn(0x55fb571f7030 msgr2=0x55fb572860f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.952+0000 7fbaec22ab80 1 -- 172.21.15.44:0/4179004629 shutdown_connections 2023-09-10T00:45:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.952+0000 7fbaec22ab80 1 -- 172.21.15.44:0/4179004629 wait complete. 2023-09-10T00:45:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.953+0000 7fbaec22ab80 1 -- start start 2023-09-10T00:45:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.953+0000 7fbaec22ab80 1 Processor -- start 2023-09-10T00:45:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.953+0000 7fbaec22ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb57291400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.953+0000 7fbaec22ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282cc0 0x55fb57291940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.953+0000 7fbaec22ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb572881a0 0x55fb57291e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.953+0000 7fbaec22ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb572eed80 con 0x55fb57282300 2023-09-10T00:45:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.953+0000 7fbaec22ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb572eeec0 con 0x55fb57282cc0 2023-09-10T00:45:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.953+0000 7fbaec22ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb572ef080 con 0x55fb572881a0 2023-09-10T00:45:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.953+0000 7fbaec22ab80 10 client.0 _ll_register_callbacks cb 0x7fbac8005be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:45:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.953+0000 7fbadde00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb57291400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.953+0000 7fbadd5ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282cc0 0x55fb57291940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.953+0000 7fbadde00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb57291400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50948/0 (socket says 172.21.15.44:50948) 2023-09-10T00:45:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.953+0000 7fbadde00700 1 -- 172.21.15.44:0/3899831312 learned_addr learned my addr 172.21.15.44:0/3899831312 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.954+0000 7fbade601700 1 --2- 172.21.15.44:0/3899831312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb572881a0 0x55fb57291e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.954+0000 7fbadd5ff700 1 -- 172.21.15.44:0/3899831312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb572881a0 msgr2=0x55fb57291e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.954+0000 7fbadd5ff700 1 --2- 172.21.15.44:0/3899831312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb572881a0 0x55fb57291e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.954+0000 7fbadd5ff700 1 -- 172.21.15.44:0/3899831312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 msgr2=0x55fb57291400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.954+0000 7fbadd5ff700 1 --2- 172.21.15.44:0/3899831312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb57291400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.954+0000 7fbadd5ff700 1 -- 172.21.15.44:0/3899831312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac8005130 con 0x55fb57282cc0 2023-09-10T00:45:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.954+0000 7fbadde00700 1 --2- 172.21.15.44:0/3899831312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb57291400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.954+0000 7fbadd5ff700 1 --2- 172.21.15.44:0/3899831312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282cc0 0x55fb57291940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbad80ffad0 tx=0x7fbac8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f5db39043faa429 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.954+0000 7fbac17fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:45:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.954+0000 7fbac17fa700 1 -- 172.21.15.44:0/3899831312 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac802c040 con 0x55fb57282cc0 2023-09-10T00:45:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.954+0000 7fbac17fa700 1 -- 172.21.15.44:0/3899831312 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fbac8033d90 con 0x55fb57282cc0 2023-09-10T00:45:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.954+0000 7fbaec22ab80 1 -- 172.21.15.44:0/3899831312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbad40ace90 con 0x55fb57282cc0 2023-09-10T00:45:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.954+0000 7fbac17fa700 1 -- 172.21.15.44:0/3899831312 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbac8003760 con 0x55fb57282cc0 2023-09-10T00:45:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.954+0000 7fbaec22ab80 10 client.26375 fetch_fsmap learned FSMap version 42 2023-09-10T00:45:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.954+0000 7fbaec22ab80 1 -- 172.21.15.44:0/3899831312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fbad0011230 con 0x55fb57282cc0 2023-09-10T00:45:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.955+0000 7fbac17fa700 1 -- 172.21.15.44:0/3899831312 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbad40ace90 con 0x55fb57282cc0 2023-09-10T00:45:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.955+0000 7fbac17fa700 1 -- 172.21.15.44:0/3899831312 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fbac8026870 con 0x55fb57282cc0 2023-09-10T00:45:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.955+0000 7fbaec22ab80 10 client.26375 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:45:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.955+0000 7fbaec22ab80 10 client.26375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:45:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.955+0000 7fbaec22ab80 20 client.26375 populate_metadata read hostname 'smithi044' 2023-09-10T00:45:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.955+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/3899831312 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fb572f07d0 0x55fb572f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.955+0000 7fbaec22ab80 4 client.26375 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:45:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.955+0000 7fbaec22ab80 1 -- 172.21.15.44:0/3899831312 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fb571cd8c0 con 0x55fb572f07d0 2023-09-10T00:45:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.955+0000 7fbadde00700 1 --2- 172.21.15.44:0/3899831312 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fb572f07d0 0x55fb572f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.955+0000 7fbac17fa700 10 client.26375 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.955+0000 7fbadde00700 1 --2- 172.21.15.44:0/3899831312 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fb572f07d0 0x55fb572f2bf0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60be0547b3a2c8a5 server_cookie=9a6767d30c0765ea in_seq=0 out_seq=0 2023-09-10T00:45:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.969+0000 7fbac17fa700 1 -- 172.21.15.44:0/3899831312 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fb571cd8c0 con 0x55fb572f07d0 2023-09-10T00:45:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.969+0000 7fbac17fa700 10 client.26375 handle_command_reply: tid=0 2023-09-10T00:45:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.970+0000 7fbaec22ab80 1 client.26375 shutdown 2023-09-10T00:45:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.970+0000 7fbaec22ab80 1 -- 172.21.15.44:0/3899831312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282cc0 msgr2=0x55fb57291940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.970+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/3899831312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282cc0 0x55fb57291940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbad80ffad0 tx=0x7fbac8009310 comp rx=0 tx=0).stop 2023-09-10T00:45:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.970+0000 7fbaec22ab80 1 -- 172.21.15.44:0/3899831312 shutdown_connections 2023-09-10T00:45:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.970+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/3899831312 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fb572f07d0 0x55fb572f2bf0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.970+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/3899831312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb572881a0 0x55fb57291e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.970+0000 7fbac17fa700 0 client.26375 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.970+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/3899831312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb57282300 0x55fb57291400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.970+0000 7fbaec22ab80 1 --2- 172.21.15.44:0/3899831312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb57282cc0 0x55fb57291940 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.970+0000 7fbaec22ab80 1 -- 172.21.15.44:0/3899831312 >> 172.21.15.44:0/3899831312 conn(0x55fb571f7030 msgr2=0x55fb5727d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.970+0000 7fbaec22ab80 1 -- 172.21.15.44:0/3899831312 shutdown_connections 2023-09-10T00:45:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.970+0000 7fbaec22ab80 1 -- 172.21.15.44:0/3899831312 wait complete. 2023-09-10T00:45:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.970+0000 7fbaec22ab80 20 client.26375 trim_cache size 0 max 16384 2023-09-10T00:45:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:45:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:45:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:45:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:45:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.971+0000 7fbae0064700 1 -- 172.21.15.44:0/1068271154 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbac40a07b0 msgr2=0x7fbac40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.971+0000 7fbae0064700 1 --2- 172.21.15.44:0/1068271154 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbac40a07b0 0x7fbac40a2c70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbac8005b20 tx=0x7fbac801a040 comp rx=0 tx=0).stop 2023-09-10T00:45:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.971+0000 7fbae0064700 1 -- 172.21.15.44:0/1068271154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbad8100590 msgr2=0x7fbad819c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.971+0000 7fbae0064700 1 --2- 172.21.15.44:0/1068271154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbad8100590 0x7fbad819c540 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbad400e9f0 tx=0x7fbad400ed00 comp rx=0 tx=0).stop 2023-09-10T00:45:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.971+0000 7fbae0064700 1 -- 172.21.15.44:0/1068271154 shutdown_connections 2023-09-10T00:45:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.971+0000 7fbae0064700 1 --2- 172.21.15.44:0/1068271154 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbac40a07b0 0x7fbac40a2c70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.971+0000 7fbae0064700 1 --2- 172.21.15.44:0/1068271154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8106650 0x7fbad81963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.971+0000 7fbae0064700 1 --2- 172.21.15.44:0/1068271154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbad8100590 0x7fbad819c540 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.971+0000 7fbae0064700 1 --2- 172.21.15.44:0/1068271154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad8100f50 0x7fbad819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.971+0000 7fbae0064700 1 -- 172.21.15.44:0/1068271154 >> 172.21.15.44:0/1068271154 conn(0x7fbad80fc060 msgr2=0x7fbad80fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.971+0000 7fbae0064700 1 -- 172.21.15.44:0/1068271154 shutdown_connections 2023-09-10T00:45:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:31.971+0000 7fbae0064700 1 -- 172.21.15.44:0/1068271154 wait complete. 2023-09-10T00:45:32.699 INFO:teuthology.orchestra.run.smithi044.stderr:24400000 of 50000000 tuples (48%) done (elapsed 272.33 s, remaining 285.73 s) 2023-09-10T00:45:32.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:32 smithi099 ceph-mon[102692]: pgmap v149: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 37 MiB/s wr, 125 op/s 2023-09-10T00:45:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:32 smithi131 ceph-mon[103713]: pgmap v149: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 37 MiB/s wr, 125 op/s 2023-09-10T00:45:33.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:32 smithi044 ceph-mon[98676]: pgmap v149: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 37 MiB/s wr, 125 op/s 2023-09-10T00:45:33.643 INFO:teuthology.orchestra.run.smithi044.stderr:24500000 of 50000000 tuples (49%) done (elapsed 273.28 s, remaining 284.43 s) 2023-09-10T00:45:34.514 INFO:teuthology.orchestra.run.smithi044.stderr:24600000 of 50000000 tuples (49%) done (elapsed 274.15 s, remaining 283.06 s) 2023-09-10T00:45:34.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:34 smithi099 ceph-mon[102692]: pgmap v150: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 28 MiB/s wr, 84 op/s 2023-09-10T00:45:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:34 smithi131 ceph-mon[103713]: pgmap v150: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 28 MiB/s wr, 84 op/s 2023-09-10T00:45:35.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:34 smithi044 ceph-mon[98676]: pgmap v150: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 28 MiB/s wr, 84 op/s 2023-09-10T00:45:35.480 INFO:teuthology.orchestra.run.smithi044.stderr:24700000 of 50000000 tuples (49%) done (elapsed 275.11 s, remaining 281.80 s) 2023-09-10T00:45:36.422 INFO:teuthology.orchestra.run.smithi044.stderr:24800000 of 50000000 tuples (49%) done (elapsed 276.06 s, remaining 280.51 s) 2023-09-10T00:45:36.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:36 smithi099 ceph-mon[102692]: pgmap v151: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 43 MiB/s wr, 124 op/s 2023-09-10T00:45:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:36 smithi131 ceph-mon[103713]: pgmap v151: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 43 MiB/s wr, 124 op/s 2023-09-10T00:45:37.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:36 smithi044 ceph-mon[98676]: pgmap v151: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 43 MiB/s wr, 124 op/s 2023-09-10T00:45:37.260 INFO:teuthology.orchestra.run.smithi044.stderr:24900000 of 50000000 tuples (49%) done (elapsed 276.89 s, remaining 279.12 s) 2023-09-10T00:45:37.987 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:45:38.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.131+0000 7fede4c80700 1 -- 172.21.15.44:0/2680648119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede00ffe00 msgr2=0x7fede010c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:38.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.131+0000 7fede4c80700 1 --2- 172.21.15.44:0/2680648119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede00ffe00 0x7fede010c310 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fedd800b0a0 tx=0x7fedd800b3b0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.131+0000 7fede4c80700 1 -- 172.21.15.44:0/2680648119 shutdown_connections 2023-09-10T00:45:38.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.131+0000 7fede4c80700 1 --2- 172.21.15.44:0/2680648119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede00ffe00 0x7fede010c310 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.131+0000 7fede4c80700 1 --2- 172.21.15.44:0/2680648119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fede01054e0 0x7fede01058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.131+0000 7fede4c80700 1 --2- 172.21.15.44:0/2680648119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fede00ff440 0x7fede00ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.131+0000 7fede4c80700 1 -- 172.21.15.44:0/2680648119 >> 172.21.15.44:0/2680648119 conn(0x7fede00faf50 msgr2=0x7fede00fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:38.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.131+0000 7fede4c80700 1 -- 172.21.15.44:0/2680648119 shutdown_connections 2023-09-10T00:45:38.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.131+0000 7fede4c80700 1 -- 172.21.15.44:0/2680648119 wait complete. 2023-09-10T00:45:38.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.132+0000 7fede4c80700 1 Processor -- start 2023-09-10T00:45:38.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.132+0000 7fede4c80700 1 -- start start 2023-09-10T00:45:38.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.132+0000 7fede4c80700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede00ff440 0x7fede00fb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.132+0000 7fede4c80700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fede00ffe00 0x7fede00fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.132+0000 7fede4c80700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fede01054e0 0x7fede01a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.132+0000 7fede4c80700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede00fc560 con 0x7fede00ff440 2023-09-10T00:45:38.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.132+0000 7fede4c80700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede01a4c80 con 0x7fede00ffe00 2023-09-10T00:45:38.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.132+0000 7fede4c80700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede01a4dc0 con 0x7fede01054e0 2023-09-10T00:45:38.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.132+0000 7fedde59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede00ff440 0x7fede00fb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.132+0000 7fedded9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fede01054e0 0x7fede01a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.132+0000 7fedddd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fede00ffe00 0x7fede00fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.132+0000 7fedde59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede00ff440 0x7fede00fb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50962/0 (socket says 172.21.15.44:50962) 2023-09-10T00:45:38.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.132+0000 7fedde59c700 1 -- 172.21.15.44:0/2822706072 learned_addr learned my addr 172.21.15.44:0/2822706072 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:38.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.133+0000 7fedde59c700 1 -- 172.21.15.44:0/2822706072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fede01054e0 msgr2=0x7fede01a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:38.138 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.133+0000 7fedde59c700 1 --2- 172.21.15.44:0/2822706072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fede01054e0 0x7fede01a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.138 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.133+0000 7fedde59c700 1 -- 172.21.15.44:0/2822706072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fede00ffe00 msgr2=0x7fede00fbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:38.138 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.133+0000 7fedde59c700 1 --2- 172.21.15.44:0/2822706072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fede00ffe00 0x7fede00fbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.138 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.133+0000 7fedde59c700 1 -- 172.21.15.44:0/2822706072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd8009d00 con 0x7fede00ff440 2023-09-10T00:45:38.138 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.133+0000 7fedddd9b700 1 --2- 172.21.15.44:0/2822706072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fede00ffe00 0x7fede00fbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:38.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.133+0000 7fedded9d700 1 --2- 172.21.15.44:0/2822706072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fede01054e0 0x7fede01a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:38.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.133+0000 7fedde59c700 1 --2- 172.21.15.44:0/2822706072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede00ff440 0x7fede00fb780 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fedd000d980 tx=0x7fedd000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:38.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.133+0000 7fedcf7fe700 1 -- 172.21.15.44:0/2822706072 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd00098d0 con 0x7fede00ff440 2023-09-10T00:45:38.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.133+0000 7fedcf7fe700 1 -- 172.21.15.44:0/2822706072 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fedd0010460 con 0x7fede00ff440 2023-09-10T00:45:38.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.133+0000 7fede4c80700 1 -- 172.21.15.44:0/2822706072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede01a5070 con 0x7fede00ff440 2023-09-10T00:45:38.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.133+0000 7fedcf7fe700 1 -- 172.21.15.44:0/2822706072 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd000f620 con 0x7fede00ff440 2023-09-10T00:45:38.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.133+0000 7fede4c80700 1 -- 172.21.15.44:0/2822706072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede01a5580 con 0x7fede00ff440 2023-09-10T00:45:38.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.136+0000 7fedcf7fe700 1 -- 172.21.15.44:0/2822706072 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fedd0009a30 con 0x7fede00ff440 2023-09-10T00:45:38.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.136+0000 7fedcf7fe700 1 --2- 172.21.15.44:0/2822706072 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fedc80a07b0 0x7fedc80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.137+0000 7fedddd9b700 1 --2- 172.21.15.44:0/2822706072 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fedc80a07b0 0x7fedc80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.137+0000 7fedcf7fe700 1 -- 172.21.15.44:0/2822706072 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fedd00cc090 con 0x7fede00ff440 2023-09-10T00:45:38.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.137+0000 7fedddd9b700 1 --2- 172.21.15.44:0/2822706072 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fedc80a07b0 0x7fedc80a2c70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fede00fcf70 tx=0x7fedd4009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:38.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.137+0000 7fedf0e46b80 1 Processor -- start 2023-09-10T00:45:38.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedf0e46b80 1 -- start start 2023-09-10T00:45:38.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedf0e46b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05ef710 0x564be05efaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedf0e46b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f0030 0x564be05f9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedf0e46b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05f9e40 0x564be05fc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedf0e46b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be0548730 con 0x564be05f9e40 2023-09-10T00:45:38.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedf0e46b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be0546050 con 0x564be05f0030 2023-09-10T00:45:38.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedf0e46b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be0541940 con 0x564be05ef710 2023-09-10T00:45:38.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedde59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05ef710 0x564be05efaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedded9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05f9e40 0x564be05fc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedddd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f0030 0x564be05f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedde59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05ef710 0x564be05efaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57852/0 (socket says 172.21.15.44:57852) 2023-09-10T00:45:38.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedde59c700 1 -- 172.21.15.44:0/3521975820 learned_addr learned my addr 172.21.15.44:0/3521975820 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:38.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedde59c700 1 -- 172.21.15.44:0/3521975820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f0030 msgr2=0x564be05f9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:38.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedde59c700 1 --2- 172.21.15.44:0/3521975820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f0030 0x564be05f9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedde59c700 1 -- 172.21.15.44:0/3521975820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05f9e40 msgr2=0x564be05fc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:38.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedde59c700 1 --2- 172.21.15.44:0/3521975820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05f9e40 0x564be05fc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedde59c700 1 -- 172.21.15.44:0/3521975820 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564be036dde0 con 0x564be05ef710 2023-09-10T00:45:38.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.138+0000 7fedddd9b700 1 --2- 172.21.15.44:0/3521975820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f0030 0x564be05f9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:38.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.139+0000 7fedde59c700 1 --2- 172.21.15.44:0/3521975820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05ef710 0x564be05efaf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fedd0099130 tx=0x7fedd0095610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da0702df1c82a33d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:38.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.139+0000 7fedccff9700 1 -- 172.21.15.44:0/3521975820 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd00998a0 con 0x564be05ef710 2023-09-10T00:45:38.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.139+0000 7fedccff9700 1 -- 172.21.15.44:0/3521975820 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fedd00991a0 con 0x564be05ef710 2023-09-10T00:45:38.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.139+0000 7fedccff9700 1 -- 172.21.15.44:0/3521975820 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd0099400 con 0x564be05ef710 2023-09-10T00:45:38.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.140+0000 7fedf0e46b80 1 -- 172.21.15.44:0/3521975820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05ef710 msgr2=0x564be05efaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:38.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.140+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/3521975820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05ef710 0x564be05efaf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fedd0099130 tx=0x7fedd0095610 comp rx=0 tx=0).stop 2023-09-10T00:45:38.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.140+0000 7fedf0e46b80 1 -- 172.21.15.44:0/3521975820 shutdown_connections 2023-09-10T00:45:38.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.140+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/3521975820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05f9e40 0x564be05fc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.140+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/3521975820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05ef710 0x564be05efaf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.140+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/3521975820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f0030 0x564be05f9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.140+0000 7fedf0e46b80 1 -- 172.21.15.44:0/3521975820 >> 172.21.15.44:0/3521975820 conn(0x564be0564090 msgr2=0x564be05eabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.140+0000 7fedf0e46b80 1 -- 172.21.15.44:0/3521975820 shutdown_connections 2023-09-10T00:45:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.140+0000 7fedf0e46b80 1 -- 172.21.15.44:0/3521975820 wait complete. 2023-09-10T00:45:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.140+0000 7fedf0e46b80 1 -- start start 2023-09-10T00:45:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.140+0000 7fedf0e46b80 1 Processor -- start 2023-09-10T00:45:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedf0e46b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05ef710 0x564be05f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedf0e46b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedf0e46b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05f9e40 0x564be05f7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedf0e46b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be04dfb70 con 0x564be05f9e40 2023-09-10T00:45:38.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedf0e46b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be04df870 con 0x564be05ef710 2023-09-10T00:45:38.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedf0e46b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be03c3920 con 0x564be05f0030 2023-09-10T00:45:38.154 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedf0e46b80 10 client.0 _ll_register_callbacks cb 0x564be0429a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:45:38.154 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedde59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05ef710 0x564be05f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.154 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedddd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.154 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedded9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05f9e40 0x564be05f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.154 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedde59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05ef710 0x564be05f98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52716/0 (socket says 172.21.15.44:52716) 2023-09-10T00:45:38.154 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedde59c700 1 -- 172.21.15.44:0/358527739 learned_addr learned my addr 172.21.15.44:0/358527739 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedddd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05f6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57860/0 (socket says 172.21.15.44:57860) 2023-09-10T00:45:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedde59c700 1 -- 172.21.15.44:0/358527739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 msgr2=0x564be05f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedde59c700 1 --2- 172.21.15.44:0/358527739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedde59c700 1 -- 172.21.15.44:0/358527739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05f9e40 msgr2=0x564be05f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedde59c700 1 --2- 172.21.15.44:0/358527739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05f9e40 0x564be05f7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedde59c700 1 -- 172.21.15.44:0/358527739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd4017070 con 0x564be05ef710 2023-09-10T00:45:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedddd9b700 1 --2- 172.21.15.44:0/358527739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedde59c700 1 --2- 172.21.15.44:0/358527739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05ef710 0x564be05f98f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fedd00c9a40 tx=0x7fedd0009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd090d1c6656c28b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.141+0000 7fedc5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:45:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.142+0000 7fedc5ffb700 1 -- 172.21.15.44:0/358527739 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd00b7070 con 0x564be05ef710 2023-09-10T00:45:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.142+0000 7fedc5ffb700 1 -- 172.21.15.44:0/358527739 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fedd009cfd0 con 0x564be05ef710 2023-09-10T00:45:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.142+0000 7fedf0e46b80 1 -- 172.21.15.44:0/358527739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564be04994f0 con 0x564be05ef710 2023-09-10T00:45:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.142+0000 7fedc5ffb700 1 -- 172.21.15.44:0/358527739 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedd00998a0 con 0x564be05ef710 2023-09-10T00:45:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.142+0000 7fedf0e46b80 10 client.26387 fetch_fsmap learned FSMap version 42 2023-09-10T00:45:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.142+0000 7fedf0e46b80 1 -- 172.21.15.44:0/358527739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fede01a5580 con 0x564be05ef710 2023-09-10T00:45:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.142+0000 7fedc5ffb700 1 -- 172.21.15.44:0/358527739 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fedd007b020 con 0x564be05ef710 2023-09-10T00:45:38.158 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.142+0000 7fedc5ffb700 1 -- 172.21.15.44:0/358527739 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fedd00a0d80 con 0x564be05ef710 2023-09-10T00:45:38.158 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.142+0000 7fedf0e46b80 10 client.26387 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:45:38.158 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.142+0000 7fedf0e46b80 10 client.26387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:45:38.158 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.142+0000 7fedf0e46b80 20 client.26387 populate_metadata read hostname 'smithi044' 2023-09-10T00:45:38.158 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.142+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/358527739 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564be06094e0 0x564be060b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.158 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.142+0000 7fedf0e46b80 4 client.26387 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:45:38.159 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.142+0000 7fedf0e46b80 1 -- 172.21.15.44:0/358527739 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564be05568a0 con 0x564be06094e0 2023-09-10T00:45:38.159 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.143+0000 7fedddd9b700 1 --2- 172.21.15.44:0/358527739 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564be06094e0 0x564be060b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.159 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.143+0000 7fedddd9b700 1 --2- 172.21.15.44:0/358527739 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564be06094e0 0x564be060b900 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96a5865abf9bcd60 server_cookie=7a6bc10806eeaa21 in_seq=0 out_seq=0 2023-09-10T00:45:38.159 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.143+0000 7fedc5ffb700 10 client.26387 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:38.159 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.144+0000 7fedc5ffb700 1 -- 172.21.15.44:0/358527739 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564be05568a0 con 0x564be06094e0 2023-09-10T00:45:38.160 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.144+0000 7fedc5ffb700 10 client.26387 handle_command_reply: tid=0 2023-09-10T00:45:38.160 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.144+0000 7fedf0e46b80 1 client.26387 shutdown 2023-09-10T00:45:38.160 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.144+0000 7fedf0e46b80 1 -- 172.21.15.44:0/358527739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05ef710 msgr2=0x564be05f98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:38.160 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.144+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/358527739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05ef710 0x564be05f98f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fedd00c9a40 tx=0x7fedd0009330 comp rx=0 tx=0).stop 2023-09-10T00:45:38.160 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.144+0000 7fedf0e46b80 1 -- 172.21.15.44:0/358527739 shutdown_connections 2023-09-10T00:45:38.160 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.144+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/358527739 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564be06094e0 0x564be060b900 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.161 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.144+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/358527739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05f9e40 0x564be05f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.161 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.144+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/358527739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05ef710 0x564be05f98f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.161 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.144+0000 7fedc5ffb700 0 client.26387 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:38.161 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.144+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/358527739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.161 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.144+0000 7fedf0e46b80 1 -- 172.21.15.44:0/358527739 >> 172.21.15.44:0/358527739 conn(0x564be0564090 msgr2=0x564be05ea7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:38.162 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.144+0000 7fedf0e46b80 1 -- 172.21.15.44:0/358527739 shutdown_connections 2023-09-10T00:45:38.162 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.144+0000 7fedf0e46b80 1 -- 172.21.15.44:0/358527739 wait complete. 2023-09-10T00:45:38.162 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.144+0000 7fedf0e46b80 20 client.26387 trim_cache size 0 max 16384 2023-09-10T00:45:38.167 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.166+0000 7fedf0e46b80 1 Processor -- start 2023-09-10T00:45:38.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.166+0000 7fedf0e46b80 1 -- start start 2023-09-10T00:45:38.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.166+0000 7fedf0e46b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05ef710 0x564be06009f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.166+0000 7fedf0e46b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05f9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.166+0000 7fedf0e46b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f9e40 0x564be05ea6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.166+0000 7fedf0e46b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be05eb1a0 con 0x564be05ef710 2023-09-10T00:45:38.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.166+0000 7fedf0e46b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be05f12b0 con 0x564be05f9e40 2023-09-10T00:45:38.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.166+0000 7fedf0e46b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be0600f60 con 0x564be05f0030 2023-09-10T00:45:38.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.166+0000 7fedde59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05ef710 0x564be06009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.166+0000 7fedded9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f9e40 0x564be05ea6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.166+0000 7fedddd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05f9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.166+0000 7fedde59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05ef710 0x564be06009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50984/0 (socket says 172.21.15.44:50984) 2023-09-10T00:45:38.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.166+0000 7fedddd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05f9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57866/0 (socket says 172.21.15.44:57866) 2023-09-10T00:45:38.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.166+0000 7fedded9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f9e40 0x564be05ea6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52728/0 (socket says 172.21.15.44:52728) 2023-09-10T00:45:38.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.166+0000 7fedde59c700 1 -- 172.21.15.44:0/401361607 learned_addr learned my addr 172.21.15.44:0/401361607 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:38.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.167+0000 7fedddd9b700 1 -- 172.21.15.44:0/401361607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f9e40 msgr2=0x564be05ea6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:38.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.167+0000 7fedddd9b700 1 --2- 172.21.15.44:0/401361607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f9e40 0x564be05ea6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.167+0000 7fedddd9b700 1 -- 172.21.15.44:0/401361607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05ef710 msgr2=0x564be06009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:38.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.167+0000 7fedddd9b700 1 --2- 172.21.15.44:0/401361607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05ef710 0x564be06009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.167+0000 7fedddd9b700 1 -- 172.21.15.44:0/401361607 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564be0356c60 con 0x564be05f0030 2023-09-10T00:45:38.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.167+0000 7fedde59c700 1 --2- 172.21.15.44:0/401361607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05ef710 0x564be06009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:38.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.167+0000 7fedded9d700 1 --2- 172.21.15.44:0/401361607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f9e40 0x564be05ea6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:45:38.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.167+0000 7fedddd9b700 1 --2- 172.21.15.44:0/401361607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05f9420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x564be060c230 tx=0x7fedd4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21c1ea6efd053703 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:38.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.167+0000 7fedcd7fa700 1 -- 172.21.15.44:0/401361607 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4024040 con 0x564be05f0030 2023-09-10T00:45:38.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.167+0000 7fedcd7fa700 1 -- 172.21.15.44:0/401361607 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fedd4029460 con 0x564be05f0030 2023-09-10T00:45:38.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.167+0000 7fedcd7fa700 1 -- 172.21.15.44:0/401361607 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd401eae0 con 0x564be05f0030 2023-09-10T00:45:38.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.168+0000 7fedf0e46b80 1 -- 172.21.15.44:0/401361607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 msgr2=0x564be05f9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:38.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.168+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/401361607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05f9420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x564be060c230 tx=0x7fedd4014000 comp rx=0 tx=0).stop 2023-09-10T00:45:38.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.168+0000 7fedf0e46b80 1 -- 172.21.15.44:0/401361607 shutdown_connections 2023-09-10T00:45:38.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.168+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/401361607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f9e40 0x564be05ea6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.168+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/401361607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05ef710 0x564be06009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.168+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/401361607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05f9420 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.168+0000 7fedf0e46b80 1 -- 172.21.15.44:0/401361607 >> 172.21.15.44:0/401361607 conn(0x564be0564090 msgr2=0x564be0605140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:38.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.168+0000 7fedf0e46b80 1 -- 172.21.15.44:0/401361607 shutdown_connections 2023-09-10T00:45:38.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.168+0000 7fedf0e46b80 1 -- 172.21.15.44:0/401361607 wait complete. 2023-09-10T00:45:38.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.168+0000 7fedf0e46b80 1 -- start start 2023-09-10T00:45:38.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.168+0000 7fedf0e46b80 1 Processor -- start 2023-09-10T00:45:38.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedf0e46b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05ef710 0x564be05fefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedf0e46b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05ff510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedf0e46b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f9e40 0x564be05ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedf0e46b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be065d340 con 0x564be05ef710 2023-09-10T00:45:38.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedf0e46b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be065d480 con 0x564be05f9e40 2023-09-10T00:45:38.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedf0e46b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be065d5c0 con 0x564be05f0030 2023-09-10T00:45:38.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedf0e46b80 10 client.0 _ll_register_callbacks cb 0x7fedd4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:45:38.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedddd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedde59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05ef710 0x564be05fefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedddd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05ff510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57874/0 (socket says 172.21.15.44:57874) 2023-09-10T00:45:38.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedddd9b700 1 -- 172.21.15.44:0/1756422761 learned_addr learned my addr 172.21.15.44:0/1756422761 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:38.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedded9d700 1 --2- 172.21.15.44:0/1756422761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f9e40 0x564be05ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedddd9b700 1 -- 172.21.15.44:0/1756422761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f9e40 msgr2=0x564be05ffa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:38.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedddd9b700 1 --2- 172.21.15.44:0/1756422761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f9e40 0x564be05ffa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedddd9b700 1 -- 172.21.15.44:0/1756422761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05ef710 msgr2=0x564be05fefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:38.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedddd9b700 1 --2- 172.21.15.44:0/1756422761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05ef710 0x564be05fefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedddd9b700 1 -- 172.21.15.44:0/1756422761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede01a5580 con 0x564be05f0030 2023-09-10T00:45:38.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedde59c700 1 --2- 172.21.15.44:0/1756422761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05ef710 0x564be05fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:38.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedddd9b700 1 --2- 172.21.15.44:0/1756422761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05ff510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fedd40057f0 tx=0x7fedd4005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cbf3c31eb15be10 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:38.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.169+0000 7fedc5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:45:38.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.170+0000 7fedc5ffb700 1 -- 172.21.15.44:0/1756422761 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4024050 con 0x564be05f0030 2023-09-10T00:45:38.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.170+0000 7fedc5ffb700 1 -- 172.21.15.44:0/1756422761 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fedd4003d80 con 0x564be05f0030 2023-09-10T00:45:38.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.170+0000 7fedf0e46b80 1 -- 172.21.15.44:0/1756422761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedd00991a0 con 0x564be05f0030 2023-09-10T00:45:38.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.170+0000 7fedc5ffb700 1 -- 172.21.15.44:0/1756422761 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedd4022070 con 0x564be05f0030 2023-09-10T00:45:38.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.170+0000 7fedf0e46b80 10 client.26335 fetch_fsmap learned FSMap version 42 2023-09-10T00:45:38.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.170+0000 7fedf0e46b80 1 -- 172.21.15.44:0/1756422761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fedd4029900 con 0x564be05f0030 2023-09-10T00:45:38.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.170+0000 7fedc5ffb700 1 -- 172.21.15.44:0/1756422761 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fedd00991a0 con 0x564be05f0030 2023-09-10T00:45:38.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.170+0000 7fedc5ffb700 1 -- 172.21.15.44:0/1756422761 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fedd400e3b0 con 0x564be05f0030 2023-09-10T00:45:38.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.170+0000 7fedf0e46b80 10 client.26335 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:45:38.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.170+0000 7fedf0e46b80 10 client.26335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:45:38.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.170+0000 7fedf0e46b80 20 client.26335 populate_metadata read hostname 'smithi044' 2023-09-10T00:45:38.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.170+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/1756422761 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564be065ed90 0x564be06611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:38.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.170+0000 7fedf0e46b80 4 client.26335 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:45:38.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.170+0000 7fedf0e46b80 1 -- 172.21.15.44:0/1756422761 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564be053a920 con 0x564be065ed90 2023-09-10T00:45:38.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.170+0000 7fedde59c700 1 --2- 172.21.15.44:0/1756422761 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564be065ed90 0x564be06611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:38.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedc5ffb700 10 client.26335 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:38.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedde59c700 1 --2- 172.21.15.44:0/1756422761 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564be065ed90 0x564be06611b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=362e58e660e46101 server_cookie=360bac0c186a9a34 in_seq=0 out_seq=0 2023-09-10T00:45:38.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedc5ffb700 1 -- 172.21.15.44:0/1756422761 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564be053a920 con 0x564be065ed90 2023-09-10T00:45:38.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedc5ffb700 10 client.26335 handle_command_reply: tid=0 2023-09-10T00:45:38.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedf0e46b80 1 client.26335 shutdown 2023-09-10T00:45:38.185 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedf0e46b80 1 -- 172.21.15.44:0/1756422761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 msgr2=0x564be05ff510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:38.186 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/1756422761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05ff510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fedd40057f0 tx=0x7fedd4005b00 comp rx=0 tx=0).stop 2023-09-10T00:45:38.186 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedf0e46b80 1 -- 172.21.15.44:0/1756422761 shutdown_connections 2023-09-10T00:45:38.186 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/1756422761 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564be065ed90 0x564be06611b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.186 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/1756422761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564be05f9e40 0x564be05ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.186 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedc5ffb700 0 client.26335 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/1756422761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564be05ef710 0x564be05fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedf0e46b80 1 --2- 172.21.15.44:0/1756422761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564be05f0030 0x564be05ff510 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedf0e46b80 1 -- 172.21.15.44:0/1756422761 >> 172.21.15.44:0/1756422761 conn(0x564be0564090 msgr2=0x564be05ee700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedf0e46b80 1 -- 172.21.15.44:0/1756422761 shutdown_connections 2023-09-10T00:45:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedf0e46b80 1 -- 172.21.15.44:0/1756422761 wait complete. 2023-09-10T00:45:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.171+0000 7fedf0e46b80 20 client.26335 trim_cache size 0 max 16384 2023-09-10T00:45:38.188 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:45:38.188 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:45:38.188 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:45:38.189 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:45:38.189 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:45:38.189 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:45:38.189 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:45:38.189 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:45:38.189 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:45:38.190 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:45:38.190 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.172+0000 7fede4c80700 1 -- 172.21.15.44:0/2822706072 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fedc80a07b0 msgr2=0x7fedc80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:38.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.172+0000 7fede4c80700 1 --2- 172.21.15.44:0/2822706072 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fedc80a07b0 0x7fedc80a2c70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fede00fcf70 tx=0x7fedd4009260 comp rx=0 tx=0).stop 2023-09-10T00:45:38.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.172+0000 7fede4c80700 1 -- 172.21.15.44:0/2822706072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede00ff440 msgr2=0x7fede00fb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:38.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.172+0000 7fede4c80700 1 --2- 172.21.15.44:0/2822706072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede00ff440 0x7fede00fb780 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fedd000d980 tx=0x7fedd000dc90 comp rx=0 tx=0).stop 2023-09-10T00:45:38.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.172+0000 7fede4c80700 1 -- 172.21.15.44:0/2822706072 shutdown_connections 2023-09-10T00:45:38.191 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.172+0000 7fede4c80700 1 --2- 172.21.15.44:0/2822706072 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fedc80a07b0 0x7fedc80a2c70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.172+0000 7fede4c80700 1 --2- 172.21.15.44:0/2822706072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fede01054e0 0x7fede01a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.172+0000 7fede4c80700 1 --2- 172.21.15.44:0/2822706072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede00ff440 0x7fede00fb780 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.172+0000 7fede4c80700 1 --2- 172.21.15.44:0/2822706072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fede00ffe00 0x7fede00fbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:38.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.172+0000 7fede4c80700 1 -- 172.21.15.44:0/2822706072 >> 172.21.15.44:0/2822706072 conn(0x7fede00faf50 msgr2=0x7fede0067900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:38.192 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.172+0000 7fede4c80700 1 -- 172.21.15.44:0/2822706072 shutdown_connections 2023-09-10T00:45:38.193 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:45:38.172+0000 7fede4c80700 1 -- 172.21.15.44:0/2822706072 wait complete. 2023-09-10T00:45:38.193 INFO:teuthology.orchestra.run.smithi044.stderr:25000000 of 50000000 tuples (50%) done (elapsed 277.81 s, remaining 277.81 s) 2023-09-10T00:45:38.194 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:45:38.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:38 smithi099 ceph-mon[102692]: pgmap v152: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 81 op/s 2023-09-10T00:45:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:38 smithi131 ceph-mon[103713]: pgmap v152: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 81 op/s 2023-09-10T00:45:39.045 INFO:teuthology.orchestra.run.smithi044.stderr:25100000 of 50000000 tuples (50%) done (elapsed 278.68 s, remaining 276.46 s) 2023-09-10T00:45:39.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:38 smithi044 ceph-mon[98676]: pgmap v152: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 81 op/s 2023-09-10T00:45:39.988 INFO:teuthology.orchestra.run.smithi044.stderr:25200000 of 50000000 tuples (50%) done (elapsed 279.62 s, remaining 275.18 s) 2023-09-10T00:45:40.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:40 smithi099 ceph-mon[102692]: pgmap v153: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 81 op/s 2023-09-10T00:45:40.977 INFO:teuthology.orchestra.run.smithi044.stderr:25300000 of 50000000 tuples (50%) done (elapsed 280.61 s, remaining 273.96 s) 2023-09-10T00:45:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:40 smithi131 ceph-mon[103713]: pgmap v153: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 81 op/s 2023-09-10T00:45:41.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:40 smithi044 ceph-mon[98676]: pgmap v153: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 81 op/s 2023-09-10T00:45:41.855 INFO:teuthology.orchestra.run.smithi044.stderr:25400000 of 50000000 tuples (50%) done (elapsed 281.49 s, remaining 272.62 s) 2023-09-10T00:45:42.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:42 smithi099 ceph-mon[102692]: pgmap v154: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 48 MiB/s wr, 112 op/s 2023-09-10T00:45:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:42 smithi131 ceph-mon[103713]: pgmap v154: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 48 MiB/s wr, 112 op/s 2023-09-10T00:45:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:42 smithi044 ceph-mon[98676]: pgmap v154: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 48 MiB/s wr, 112 op/s 2023-09-10T00:45:44.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:44 smithi099 ceph-mon[102692]: pgmap v155: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 71 op/s 2023-09-10T00:45:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:44 smithi131 ceph-mon[103713]: pgmap v155: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 71 op/s 2023-09-10T00:45:45.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:44 smithi044 ceph-mon[98676]: pgmap v155: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 71 op/s 2023-09-10T00:45:45.104 INFO:teuthology.orchestra.run.smithi044.stderr:25500000 of 50000000 tuples (51%) done (elapsed 284.74 s, remaining 273.57 s) 2023-09-10T00:45:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.105+0000 7ff31aac1700 1 -- 172.21.15.44:0/34393524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3140fee80 msgr2=0x7ff3140ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.105+0000 7ff31aac1700 1 --2- 172.21.15.44:0/34393524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3140fee80 0x7ff3140ff260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff308009990 tx=0x7ff308009ca0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.105+0000 7ff31aac1700 1 -- 172.21.15.44:0/34393524 shutdown_connections 2023-09-10T00:45:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.105+0000 7ff31aac1700 1 --2- 172.21.15.44:0/34393524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff314103fa0 0x7ff31410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.105+0000 7ff31aac1700 1 --2- 172.21.15.44:0/34393524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3140fee80 0x7ff3140ff260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.105+0000 7ff31aac1700 1 --2- 172.21.15.44:0/34393524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3140ff830 0x7ff314103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.105+0000 7ff31aac1700 1 -- 172.21.15.44:0/34393524 >> 172.21.15.44:0/34393524 conn(0x7ff3140769d0 msgr2=0x7ff3140fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.106+0000 7ff31aac1700 1 -- 172.21.15.44:0/34393524 shutdown_connections 2023-09-10T00:45:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.106+0000 7ff31aac1700 1 -- 172.21.15.44:0/34393524 wait complete. 2023-09-10T00:45:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.106+0000 7ff31aac1700 1 Processor -- start 2023-09-10T00:45:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.106+0000 7ff31aac1700 1 -- start start 2023-09-10T00:45:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31aac1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3140fee80 0x7ff31419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31aac1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff314103fa0 0x7ff31419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31aac1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff314196420 0x7ff3141968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31aac1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314196ea0 con 0x7ff314196420 2023-09-10T00:45:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31aac1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314196fe0 con 0x7ff3140fee80 2023-09-10T00:45:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31aac1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3141971a0 con 0x7ff314103fa0 2023-09-10T00:45:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31905e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff314196420 0x7ff3141968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31885d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3140fee80 0x7ff31419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31905e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff314196420 0x7ff3141968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51008/0 (socket says 172.21.15.44:51008) 2023-09-10T00:45:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31905e700 1 -- 172.21.15.44:0/1153560056 learned_addr learned my addr 172.21.15.44:0/1153560056 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff313fff700 1 --2- 172.21.15.44:0/1153560056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff314103fa0 0x7ff31419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31905e700 1 -- 172.21.15.44:0/1153560056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff314103fa0 msgr2=0x7ff31419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31905e700 1 --2- 172.21.15.44:0/1153560056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff314103fa0 0x7ff31419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31905e700 1 -- 172.21.15.44:0/1153560056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3140fee80 msgr2=0x7ff31419c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31905e700 1 --2- 172.21.15.44:0/1153560056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3140fee80 0x7ff31419c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31905e700 1 -- 172.21.15.44:0/1153560056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff300009520 con 0x7ff314196420 2023-09-10T00:45:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31905e700 1 --2- 172.21.15.44:0/1153560056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff314196420 0x7ff3141968d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff30c007690 tx=0x7ff30c0079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff311ffb700 1 -- 172.21.15.44:0/1153560056 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c018070 con 0x7ff314196420 2023-09-10T00:45:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.107+0000 7ff31aac1700 1 -- 172.21.15.44:0/1153560056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30c00ad50 con 0x7ff314196420 2023-09-10T00:45:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.108+0000 7ff311ffb700 1 -- 172.21.15.44:0/1153560056 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff30c010040 con 0x7ff314196420 2023-09-10T00:45:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.108+0000 7ff311ffb700 1 -- 172.21.15.44:0/1153560056 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c004710 con 0x7ff314196420 2023-09-10T00:45:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.108+0000 7ff31aac1700 1 -- 172.21.15.44:0/1153560056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3080095f0 con 0x7ff314196420 2023-09-10T00:45:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.110+0000 7ff311ffb700 1 -- 172.21.15.44:0/1153560056 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ff30c003a00 con 0x7ff314196420 2023-09-10T00:45:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.111+0000 7ff311ffb700 1 --2- 172.21.15.44:0/1153560056 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff3040a07b0 0x7ff3040a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.111+0000 7ff31885d700 1 --2- 172.21.15.44:0/1153560056 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff3040a07b0 0x7ff3040a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.111+0000 7ff311ffb700 1 -- 172.21.15.44:0/1153560056 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ff30c0cbf00 con 0x7ff314196420 2023-09-10T00:45:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.111+0000 7ff31885d700 1 --2- 172.21.15.44:0/1153560056 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff3040a07b0 0x7ff3040a2c70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff30800b2e0 tx=0x7ff308005040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff326c87b80 1 Processor -- start 2023-09-10T00:45:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff326c87b80 1 -- start start 2023-09-10T00:45:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff326c87b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a473171a0 0x563a47317580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff326c87b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a47311300 0x563a47311780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff326c87b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311cc0 0x563a4731df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff326c87b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4726a6d0 con 0x563a47311cc0 2023-09-10T00:45:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff326c87b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a47267ff0 con 0x563a473171a0 2023-09-10T00:45:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff326c87b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a472638e0 con 0x563a47311300 2023-09-10T00:45:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff31885d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a473171a0 0x563a47317580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff313fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a47311300 0x563a47311780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff31905e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311cc0 0x563a4731df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff31885d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a473171a0 0x563a47317580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52752/0 (socket says 172.21.15.44:52752) 2023-09-10T00:45:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff31885d700 1 -- 172.21.15.44:0/1968894857 learned_addr learned my addr 172.21.15.44:0/1968894857 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff313fff700 1 -- 172.21.15.44:0/1968894857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a473171a0 msgr2=0x563a47317580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff313fff700 1 --2- 172.21.15.44:0/1968894857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a473171a0 0x563a47317580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff313fff700 1 -- 172.21.15.44:0/1968894857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311cc0 msgr2=0x563a4731df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff313fff700 1 --2- 172.21.15.44:0/1968894857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311cc0 0x563a4731df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.112+0000 7ff313fff700 1 -- 172.21.15.44:0/1968894857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a4708fd80 con 0x563a47311300 2023-09-10T00:45:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.113+0000 7ff313fff700 1 --2- 172.21.15.44:0/1968894857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a47311300 0x563a47311780 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff314072cf0 tx=0x7ff300011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c5f15897b82470f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.113+0000 7ff2feffd700 1 -- 172.21.15.44:0/1968894857 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300019630 con 0x563a47311300 2023-09-10T00:45:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.113+0000 7ff2feffd700 1 -- 172.21.15.44:0/1968894857 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff300019c20 con 0x563a47311300 2023-09-10T00:45:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.113+0000 7ff2feffd700 1 -- 172.21.15.44:0/1968894857 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30000ebd0 con 0x563a47311300 2023-09-10T00:45:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.113+0000 7ff326c87b80 1 -- 172.21.15.44:0/1968894857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a47311300 msgr2=0x563a47311780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.113+0000 7ff326c87b80 1 --2- 172.21.15.44:0/1968894857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a47311300 0x563a47311780 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff314072cf0 tx=0x7ff300011720 comp rx=0 tx=0).stop 2023-09-10T00:45:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.114+0000 7ff326c87b80 1 -- 172.21.15.44:0/1968894857 shutdown_connections 2023-09-10T00:45:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.114+0000 7ff326c87b80 1 --2- 172.21.15.44:0/1968894857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311cc0 0x563a4731df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.114+0000 7ff326c87b80 1 --2- 172.21.15.44:0/1968894857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a473171a0 0x563a47317580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.114+0000 7ff326c87b80 1 --2- 172.21.15.44:0/1968894857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a47311300 0x563a47311780 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.114+0000 7ff326c87b80 1 -- 172.21.15.44:0/1968894857 >> 172.21.15.44:0/1968894857 conn(0x563a47286030 msgr2=0x563a4730c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.114+0000 7ff326c87b80 1 -- 172.21.15.44:0/1968894857 shutdown_connections 2023-09-10T00:45:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.114+0000 7ff326c87b80 1 -- 172.21.15.44:0/1968894857 wait complete. 2023-09-10T00:45:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.114+0000 7ff326c87b80 1 -- start start 2023-09-10T00:45:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.114+0000 7ff326c87b80 1 Processor -- start 2023-09-10T00:45:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff326c87b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311300 0x563a4731ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff326c87b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a47311cc0 0x563a4731b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff326c87b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a473171a0 0x563a4731b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff326c87b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a47201b10 con 0x563a47311300 2023-09-10T00:45:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff326c87b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a47201810 con 0x563a47311cc0 2023-09-10T00:45:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff326c87b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a470e58c0 con 0x563a473171a0 2023-09-10T00:45:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff326c87b80 10 client.0 _ll_register_callbacks cb 0x563a4714ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:45:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff31885d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311300 0x563a4731ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff313fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a47311cc0 0x563a4731b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff31905e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a473171a0 0x563a4731b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff31885d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311300 0x563a4731ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51018/0 (socket says 172.21.15.44:51018) 2023-09-10T00:45:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff31885d700 1 -- 172.21.15.44:0/2605229164 learned_addr learned my addr 172.21.15.44:0/2605229164 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff31905e700 1 -- 172.21.15.44:0/2605229164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a47311cc0 msgr2=0x563a4731b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff31905e700 1 --2- 172.21.15.44:0/2605229164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a47311cc0 0x563a4731b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff31905e700 1 -- 172.21.15.44:0/2605229164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311300 msgr2=0x563a4731ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff31905e700 1 --2- 172.21.15.44:0/2605229164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311300 0x563a4731ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff31905e700 1 -- 172.21.15.44:0/2605229164 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff300009a70 con 0x563a473171a0 2023-09-10T00:45:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff31885d700 1 --2- 172.21.15.44:0/2605229164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311300 0x563a4731ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff31905e700 1 --2- 172.21.15.44:0/2605229164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a473171a0 0x563a4731b640 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff30c0c92b0 tx=0x7ff30c007e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cb78ec0db1dbe8d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff2f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:45:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.115+0000 7ff2f3fff700 1 -- 172.21.15.44:0/2605229164 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c099470 con 0x563a473171a0 2023-09-10T00:45:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.116+0000 7ff2f3fff700 1 -- 172.21.15.44:0/2605229164 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ff30c099010 con 0x563a473171a0 2023-09-10T00:45:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.116+0000 7ff326c87b80 1 -- 172.21.15.44:0/2605229164 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a471bb490 con 0x563a473171a0 2023-09-10T00:45:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.116+0000 7ff2f3fff700 1 -- 172.21.15.44:0/2605229164 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff30c09ce40 con 0x563a473171a0 2023-09-10T00:45:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.116+0000 7ff326c87b80 10 client.26347 fetch_fsmap learned FSMap version 42 2023-09-10T00:45:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.116+0000 7ff326c87b80 1 -- 172.21.15.44:0/2605229164 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x563a47078c00 con 0x563a473171a0 2023-09-10T00:45:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.116+0000 7ff2f3fff700 1 -- 172.21.15.44:0/2605229164 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563a471bb490 con 0x563a473171a0 2023-09-10T00:45:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.116+0000 7ff2f3fff700 1 -- 172.21.15.44:0/2605229164 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff30c098500 con 0x563a473171a0 2023-09-10T00:45:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.116+0000 7ff326c87b80 10 client.26347 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:45:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.116+0000 7ff326c87b80 10 client.26347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:45:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.116+0000 7ff326c87b80 20 client.26347 populate_metadata read hostname 'smithi044' 2023-09-10T00:45:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.116+0000 7ff326c87b80 1 --2- 172.21.15.44:0/2605229164 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a4732b360 0x563a4732d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.116+0000 7ff326c87b80 4 client.26347 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:45:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.116+0000 7ff326c87b80 1 -- 172.21.15.44:0/2605229164 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a47278840 con 0x563a4732b360 2023-09-10T00:45:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.116+0000 7ff31885d700 1 --2- 172.21.15.44:0/2605229164 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a4732b360 0x563a4732d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.117+0000 7ff2f3fff700 10 client.26347 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.117+0000 7ff31885d700 1 --2- 172.21.15.44:0/2605229164 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a4732b360 0x563a4732d780 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed8b9a2c9a431c29 server_cookie=ed9ee1b2fd6f9c9e in_seq=0 out_seq=0 2023-09-10T00:45:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.117+0000 7ff2f3fff700 1 -- 172.21.15.44:0/2605229164 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563a47278840 con 0x563a4732b360 2023-09-10T00:45:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.117+0000 7ff2f3fff700 10 client.26347 handle_command_reply: tid=0 2023-09-10T00:45:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.118+0000 7ff326c87b80 1 client.26347 shutdown 2023-09-10T00:45:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.118+0000 7ff326c87b80 1 -- 172.21.15.44:0/2605229164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a473171a0 msgr2=0x563a4731b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.118+0000 7ff326c87b80 1 --2- 172.21.15.44:0/2605229164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a473171a0 0x563a4731b640 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff30c0c92b0 tx=0x7ff30c007e20 comp rx=0 tx=0).stop 2023-09-10T00:45:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.118+0000 7ff326c87b80 1 -- 172.21.15.44:0/2605229164 shutdown_connections 2023-09-10T00:45:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.118+0000 7ff326c87b80 1 --2- 172.21.15.44:0/2605229164 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a4732b360 0x563a4732d780 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.118+0000 7ff326c87b80 1 --2- 172.21.15.44:0/2605229164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a473171a0 0x563a4731b640 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.118+0000 7ff326c87b80 1 --2- 172.21.15.44:0/2605229164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311300 0x563a4731ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.118+0000 7ff2f3fff700 0 client.26347 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.118+0000 7ff326c87b80 1 --2- 172.21.15.44:0/2605229164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a47311cc0 0x563a4731b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.118+0000 7ff326c87b80 1 -- 172.21.15.44:0/2605229164 >> 172.21.15.44:0/2605229164 conn(0x563a47286030 msgr2=0x563a4730b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.118+0000 7ff326c87b80 1 -- 172.21.15.44:0/2605229164 shutdown_connections 2023-09-10T00:45:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.118+0000 7ff326c87b80 1 -- 172.21.15.44:0/2605229164 wait complete. 2023-09-10T00:45:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.118+0000 7ff326c87b80 20 client.26347 trim_cache size 0 max 16384 2023-09-10T00:45:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff326c87b80 1 Processor -- start 2023-09-10T00:45:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff326c87b80 1 -- start start 2023-09-10T00:45:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff326c87b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a47311300 0x563a47322520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff326c87b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311cc0 0x563a4731d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff326c87b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a473171a0 0x563a4731ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff326c87b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4730d930 con 0x563a47311cc0 2023-09-10T00:45:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff326c87b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a47328aa0 con 0x563a47311300 2023-09-10T00:45:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff326c87b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a47322a60 con 0x563a473171a0 2023-09-10T00:45:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff31885d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a47311300 0x563a47322520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff31885d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a47311300 0x563a47322520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52766/0 (socket says 172.21.15.44:52766) 2023-09-10T00:45:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff313fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311cc0 0x563a4731d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff31885d700 1 -- 172.21.15.44:0/3841532990 learned_addr learned my addr 172.21.15.44:0/3841532990 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff31905e700 1 --2- 172.21.15.44:0/3841532990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a473171a0 0x563a4731ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff31885d700 1 -- 172.21.15.44:0/3841532990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a473171a0 msgr2=0x563a4731ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff31885d700 1 --2- 172.21.15.44:0/3841532990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a473171a0 0x563a4731ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff31885d700 1 -- 172.21.15.44:0/3841532990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311cc0 msgr2=0x563a4731d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff31885d700 1 --2- 172.21.15.44:0/3841532990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311cc0 0x563a4731d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff31885d700 1 -- 172.21.15.44:0/3841532990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a470b61f0 con 0x563a47311300 2023-09-10T00:45:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.136+0000 7ff31905e700 1 --2- 172.21.15.44:0/3841532990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a473171a0 0x563a4731ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.137+0000 7ff31885d700 1 --2- 172.21.15.44:0/3841532990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a47311300 0x563a47322520 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff308005cd0 tx=0x7ff308020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74eb99805ba8e0d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.137+0000 7ff2ff7fe700 1 -- 172.21.15.44:0/3841532990 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308005610 con 0x563a47311300 2023-09-10T00:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.137+0000 7ff2ff7fe700 1 -- 172.21.15.44:0/3841532990 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff30801e040 con 0x563a47311300 2023-09-10T00:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.137+0000 7ff2ff7fe700 1 -- 172.21.15.44:0/3841532990 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308004940 con 0x563a47311300 2023-09-10T00:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.138+0000 7ff326c87b80 1 -- 172.21.15.44:0/3841532990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a47311300 msgr2=0x563a47322520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.138+0000 7ff326c87b80 1 --2- 172.21.15.44:0/3841532990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a47311300 0x563a47322520 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff308005cd0 tx=0x7ff308020000 comp rx=0 tx=0).stop 2023-09-10T00:45:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.138+0000 7ff326c87b80 1 -- 172.21.15.44:0/3841532990 shutdown_connections 2023-09-10T00:45:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.138+0000 7ff326c87b80 1 --2- 172.21.15.44:0/3841532990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a473171a0 0x563a4731ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.138+0000 7ff326c87b80 1 --2- 172.21.15.44:0/3841532990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a47311300 0x563a47322520 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.138+0000 7ff326c87b80 1 --2- 172.21.15.44:0/3841532990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311cc0 0x563a4731d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.138+0000 7ff326c87b80 1 -- 172.21.15.44:0/3841532990 >> 172.21.15.44:0/3841532990 conn(0x563a47286030 msgr2=0x563a47316110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.138+0000 7ff326c87b80 1 -- 172.21.15.44:0/3841532990 shutdown_connections 2023-09-10T00:45:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.138+0000 7ff326c87b80 1 -- 172.21.15.44:0/3841532990 wait complete. 2023-09-10T00:45:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.138+0000 7ff326c87b80 1 -- start start 2023-09-10T00:45:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.138+0000 7ff326c87b80 1 Processor -- start 2023-09-10T00:45:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.138+0000 7ff326c87b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a47311300 0x563a47312b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.138+0000 7ff326c87b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311cc0 0x563a47313090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.138+0000 7ff326c87b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a473171a0 0x563a473135d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.138+0000 7ff326c87b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a47313d10 con 0x563a47311cc0 2023-09-10T00:45:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff326c87b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a47313e80 con 0x563a473171a0 2023-09-10T00:45:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff326c87b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a47314040 con 0x563a47311300 2023-09-10T00:45:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff326c87b80 10 client.0 _ll_register_callbacks cb 0x7ff30c098970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:45:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff31885d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a47311300 0x563a47312b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff31905e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a473171a0 0x563a473135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff31885d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a47311300 0x563a47312b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57914/0 (socket says 172.21.15.44:57914) 2023-09-10T00:45:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff31885d700 1 -- 172.21.15.44:0/2834980821 learned_addr learned my addr 172.21.15.44:0/2834980821 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff313fff700 1 --2- 172.21.15.44:0/2834980821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311cc0 0x563a47313090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff31905e700 1 -- 172.21.15.44:0/2834980821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a47311300 msgr2=0x563a47312b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff31905e700 1 --2- 172.21.15.44:0/2834980821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a47311300 0x563a47312b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff31905e700 1 -- 172.21.15.44:0/2834980821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311cc0 msgr2=0x563a47313090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff31905e700 1 --2- 172.21.15.44:0/2834980821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311cc0 0x563a47313090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff31905e700 1 -- 172.21.15.44:0/2834980821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a47078c00 con 0x563a473171a0 2023-09-10T00:45:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff31885d700 1 --2- 172.21.15.44:0/2834980821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a47311300 0x563a47312b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:45.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff31905e700 1 --2- 172.21.15.44:0/2834980821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a473171a0 0x563a473135d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff30c098d10 tx=0x7ff30c098730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63bf4ddfba278faa server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:45.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff2f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:45:45.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff2f3fff700 1 -- 172.21.15.44:0/2834980821 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c099010 con 0x563a473171a0 2023-09-10T00:45:45.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff2f3fff700 1 -- 172.21.15.44:0/2834980821 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ff30c09ce40 con 0x563a473171a0 2023-09-10T00:45:45.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.139+0000 7ff326c87b80 1 -- 172.21.15.44:0/2834980821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff300019c20 con 0x563a473171a0 2023-09-10T00:45:45.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.140+0000 7ff2f3fff700 1 -- 172.21.15.44:0/2834980821 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff30c0a1080 con 0x563a473171a0 2023-09-10T00:45:45.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.140+0000 7ff326c87b80 10 client.26411 fetch_fsmap learned FSMap version 42 2023-09-10T00:45:45.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.140+0000 7ff326c87b80 1 -- 172.21.15.44:0/2834980821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7ff31410c390 con 0x563a473171a0 2023-09-10T00:45:45.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.140+0000 7ff2f3fff700 1 -- 172.21.15.44:0/2834980821 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff300019c20 con 0x563a473171a0 2023-09-10T00:45:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.140+0000 7ff2f3fff700 1 -- 172.21.15.44:0/2834980821 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff30c094e00 con 0x563a473171a0 2023-09-10T00:45:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.140+0000 7ff326c87b80 10 client.26411 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:45:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.140+0000 7ff326c87b80 10 client.26411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:45:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.140+0000 7ff326c87b80 20 client.26411 populate_metadata read hostname 'smithi044' 2023-09-10T00:45:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.140+0000 7ff326c87b80 1 --2- 172.21.15.44:0/2834980821 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a4737fe70 0x563a47382290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.140+0000 7ff326c87b80 4 client.26411 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:45:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.140+0000 7ff326c87b80 1 -- 172.21.15.44:0/2834980821 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563a4725c8c0 con 0x563a4737fe70 2023-09-10T00:45:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.140+0000 7ff31885d700 1 --2- 172.21.15.44:0/2834980821 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a4737fe70 0x563a47382290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.141+0000 7ff2f3fff700 10 client.26411 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.141+0000 7ff31885d700 1 --2- 172.21.15.44:0/2834980821 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a4737fe70 0x563a47382290 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edb66cb890619ccd server_cookie=29c4eeb5d914d0c1 in_seq=0 out_seq=0 2023-09-10T00:45:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.153+0000 7ff2f3fff700 1 -- 172.21.15.44:0/2834980821 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563a4725c8c0 con 0x563a4737fe70 2023-09-10T00:45:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.153+0000 7ff2f3fff700 10 client.26411 handle_command_reply: tid=0 2023-09-10T00:45:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.153+0000 7ff326c87b80 1 client.26411 shutdown 2023-09-10T00:45:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.153+0000 7ff326c87b80 1 -- 172.21.15.44:0/2834980821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a473171a0 msgr2=0x563a473135d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.153+0000 7ff326c87b80 1 --2- 172.21.15.44:0/2834980821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a473171a0 0x563a473135d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff30c098d10 tx=0x7ff30c098730 comp rx=0 tx=0).stop 2023-09-10T00:45:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.153+0000 7ff326c87b80 1 -- 172.21.15.44:0/2834980821 shutdown_connections 2023-09-10T00:45:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.153+0000 7ff326c87b80 1 --2- 172.21.15.44:0/2834980821 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a4737fe70 0x563a47382290 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.153+0000 7ff326c87b80 1 --2- 172.21.15.44:0/2834980821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a473171a0 0x563a473135d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.153+0000 7ff2f3fff700 0 client.26411 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.153+0000 7ff326c87b80 1 --2- 172.21.15.44:0/2834980821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a47311300 0x563a47312b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.153+0000 7ff326c87b80 1 --2- 172.21.15.44:0/2834980821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a47311cc0 0x563a47313090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.153+0000 7ff326c87b80 1 -- 172.21.15.44:0/2834980821 >> 172.21.15.44:0/2834980821 conn(0x563a47286030 msgr2=0x563a47316110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.154+0000 7ff326c87b80 1 -- 172.21.15.44:0/2834980821 shutdown_connections 2023-09-10T00:45:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.154+0000 7ff326c87b80 1 -- 172.21.15.44:0/2834980821 wait complete. 2023-09-10T00:45:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.154+0000 7ff326c87b80 20 client.26411 trim_cache size 0 max 16384 2023-09-10T00:45:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:45:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:45:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:45:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:45:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.154+0000 7ff31aac1700 1 -- 172.21.15.44:0/1153560056 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff3040a07b0 msgr2=0x7ff3040a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.154+0000 7ff31aac1700 1 --2- 172.21.15.44:0/1153560056 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff3040a07b0 0x7ff3040a2c70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff30800b2e0 tx=0x7ff308005040 comp rx=0 tx=0).stop 2023-09-10T00:45:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.154+0000 7ff31aac1700 1 -- 172.21.15.44:0/1153560056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff314196420 msgr2=0x7ff3141968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.154+0000 7ff31aac1700 1 --2- 172.21.15.44:0/1153560056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff314196420 0x7ff3141968d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff30c007690 tx=0x7ff30c0079a0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.154+0000 7ff31aac1700 1 -- 172.21.15.44:0/1153560056 shutdown_connections 2023-09-10T00:45:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.154+0000 7ff31aac1700 1 --2- 172.21.15.44:0/1153560056 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff3040a07b0 0x7ff3040a2c70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.154+0000 7ff31aac1700 1 --2- 172.21.15.44:0/1153560056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff314196420 0x7ff3141968d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.154+0000 7ff31aac1700 1 --2- 172.21.15.44:0/1153560056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3140fee80 0x7ff31419c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.154+0000 7ff31aac1700 1 --2- 172.21.15.44:0/1153560056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff314103fa0 0x7ff31419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.154+0000 7ff31aac1700 1 -- 172.21.15.44:0/1153560056 >> 172.21.15.44:0/1153560056 conn(0x7ff3140769d0 msgr2=0x7ff3140fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.154+0000 7ff31aac1700 1 -- 172.21.15.44:0/1153560056 shutdown_connections 2023-09-10T00:45:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:45.155+0000 7ff31aac1700 1 -- 172.21.15.44:0/1153560056 wait complete. 2023-09-10T00:45:45.963 INFO:teuthology.orchestra.run.smithi044.stderr:25600000 of 50000000 tuples (51%) done (elapsed 285.60 s, remaining 272.21 s) 2023-09-10T00:45:46.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:46 smithi099 ceph-mon[102692]: pgmap v156: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 41 MiB/s wr, 98 op/s 2023-09-10T00:45:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:46 smithi131 ceph-mon[103713]: pgmap v156: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 41 MiB/s wr, 98 op/s 2023-09-10T00:45:47.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:46 smithi044 ceph-mon[98676]: pgmap v156: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 41 MiB/s wr, 98 op/s 2023-09-10T00:45:47.724 INFO:teuthology.orchestra.run.smithi044.stderr:25700000 of 50000000 tuples (51%) done (elapsed 287.36 s, remaining 271.70 s) 2023-09-10T00:45:48.575 INFO:teuthology.orchestra.run.smithi044.stderr:25800000 of 50000000 tuples (51%) done (elapsed 288.21 s, remaining 270.34 s) 2023-09-10T00:45:48.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:48 smithi099 ceph-mon[102692]: pgmap v157: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 25 MiB/s wr, 57 op/s 2023-09-10T00:45:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:48 smithi131 ceph-mon[103713]: pgmap v157: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 25 MiB/s wr, 57 op/s 2023-09-10T00:45:49.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:48 smithi044 ceph-mon[98676]: pgmap v157: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 25 MiB/s wr, 57 op/s 2023-09-10T00:45:49.497 INFO:teuthology.orchestra.run.smithi044.stderr:25900000 of 50000000 tuples (51%) done (elapsed 289.13 s, remaining 269.04 s) 2023-09-10T00:45:49.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:49 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:45:50.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:49 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:45:50.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:49 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:45:50.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:50 smithi099 ceph-mon[102692]: pgmap v158: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 25 MiB/s wr, 57 op/s 2023-09-10T00:45:50.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:50 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:45:50.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:50 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:45:50.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:50 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:45:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:50 smithi131 ceph-mon[103713]: pgmap v158: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 25 MiB/s wr, 57 op/s 2023-09-10T00:45:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:50 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:45:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:50 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:45:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:50 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:45:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:50 smithi044 ceph-mon[98676]: pgmap v158: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 25 MiB/s wr, 57 op/s 2023-09-10T00:45:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:50 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:45:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:50 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:45:51.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:50 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:45:52.044 INFO:teuthology.orchestra.run.smithi044.stderr:26000000 of 50000000 tuples (52%) done (elapsed 291.68 s, remaining 269.24 s) 2023-09-10T00:45:52.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:52 smithi099 ceph-mon[102692]: pgmap v159: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 38 MiB/s wr, 78 op/s 2023-09-10T00:45:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:52 smithi131 ceph-mon[103713]: pgmap v159: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 38 MiB/s wr, 78 op/s 2023-09-10T00:45:53.045 INFO:teuthology.orchestra.run.smithi044.stderr:26100000 of 50000000 tuples (52%) done (elapsed 292.68 s, remaining 268.01 s) 2023-09-10T00:45:53.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:52 smithi044 ceph-mon[98676]: pgmap v159: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 38 MiB/s wr, 78 op/s 2023-09-10T00:45:54.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:54 smithi099 ceph-mon[102692]: pgmap v160: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 21 MiB/s wr, 47 op/s 2023-09-10T00:45:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:26200000 of 50000000 tuples (52%) done (elapsed 294.61 s, remaining 267.62 s) 2023-09-10T00:45:55.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:54 smithi131 ceph-mon[103713]: pgmap v160: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 21 MiB/s wr, 47 op/s 2023-09-10T00:45:55.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:54 smithi044 ceph-mon[98676]: pgmap v160: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 21 MiB/s wr, 47 op/s 2023-09-10T00:45:56.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:56 smithi099 ceph-mon[102692]: pgmap v161: 129 pgs: 129 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 30 MiB/s wr, 90 op/s 2023-09-10T00:45:57.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:56 smithi131 ceph-mon[103713]: pgmap v161: 129 pgs: 129 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 30 MiB/s wr, 90 op/s 2023-09-10T00:45:57.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:56 smithi044 ceph-mon[98676]: pgmap v161: 129 pgs: 129 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 30 MiB/s wr, 90 op/s 2023-09-10T00:45:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.277+0000 7fe7c3a12700 1 -- 172.21.15.44:0/1928573816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7bc101470 msgr2=0x7fe7bc0679f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.277+0000 7fe7c3a12700 1 --2- 172.21.15.44:0/1928573816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7bc101470 0x7fe7bc0679f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009990 tx=0x7fe7b8009ca0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.277+0000 7fe7c3a12700 1 -- 172.21.15.44:0/1928573816 shutdown_connections 2023-09-10T00:45:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.277+0000 7fe7c3a12700 1 --2- 172.21.15.44:0/1928573816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7bc10cdd0 0x7fe7bc10f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.277+0000 7fe7c3a12700 1 --2- 172.21.15.44:0/1928573816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7bc101470 0x7fe7bc0679f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.277+0000 7fe7c3a12700 1 --2- 172.21.15.44:0/1928573816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7bc067f30 0x7fe7bc0683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.277+0000 7fe7c3a12700 1 -- 172.21.15.44:0/1928573816 >> 172.21.15.44:0/1928573816 conn(0x7fe7bc074f30 msgr2=0x7fe7bc075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.277+0000 7fe7c3a12700 1 -- 172.21.15.44:0/1928573816 shutdown_connections 2023-09-10T00:45:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.277+0000 7fe7c3a12700 1 -- 172.21.15.44:0/1928573816 wait complete. 2023-09-10T00:45:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.278+0000 7fe7c3a12700 1 Processor -- start 2023-09-10T00:45:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.278+0000 7fe7c3a12700 1 -- start start 2023-09-10T00:45:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.278+0000 7fe7c3a12700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7bc067f30 0x7fe7bc198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.278+0000 7fe7c3a12700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7bc101470 0x7fe7bc198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.278+0000 7fe7c3a12700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7bc10cdd0 0x7fe7bc19d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.278+0000 7fe7c3a12700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc19d940 con 0x7fe7bc067f30 2023-09-10T00:45:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.278+0000 7fe7c3a12700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc19dab0 con 0x7fe7bc10cdd0 2023-09-10T00:45:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.278+0000 7fe7c3a12700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc19dc70 con 0x7fe7bc101470 2023-09-10T00:45:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.278+0000 7fe7c17ae700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7bc067f30 0x7fe7bc198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.278+0000 7fe7c0fad700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7bc101470 0x7fe7bc198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.278+0000 7fe7c0fad700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7bc101470 0x7fe7bc198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57930/0 (socket says 172.21.15.44:57930) 2023-09-10T00:45:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.278+0000 7fe7c1faf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7bc10cdd0 0x7fe7bc19d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.278+0000 7fe7c0fad700 1 -- 172.21.15.44:0/652676511 learned_addr learned my addr 172.21.15.44:0/652676511 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.279+0000 7fe7c0fad700 1 -- 172.21.15.44:0/652676511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7bc10cdd0 msgr2=0x7fe7bc19d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.279+0000 7fe7c0fad700 1 --2- 172.21.15.44:0/652676511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7bc10cdd0 0x7fe7bc19d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.279+0000 7fe7c0fad700 1 -- 172.21.15.44:0/652676511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7bc067f30 msgr2=0x7fe7bc198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.279+0000 7fe7c0fad700 1 --2- 172.21.15.44:0/652676511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7bc067f30 0x7fe7bc198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.279+0000 7fe7c0fad700 1 -- 172.21.15.44:0/652676511 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b80095f0 con 0x7fe7bc101470 2023-09-10T00:45:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.279+0000 7fe7c17ae700 1 --2- 172.21.15.44:0/652676511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7bc067f30 0x7fe7bc198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.279+0000 7fe7c0fad700 1 --2- 172.21.15.44:0/652676511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7bc101470 0x7fe7bc198cd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000e9a0 tx=0x7fe7b000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.279+0000 7fe7ae7fc700 1 -- 172.21.15.44:0/652676511 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0009790 con 0x7fe7bc101470 2023-09-10T00:45:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.279+0000 7fe7c3a12700 1 -- 172.21.15.44:0/652676511 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7bc19df50 con 0x7fe7bc101470 2023-09-10T00:45:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.279+0000 7fe7ae7fc700 1 -- 172.21.15.44:0/652676511 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7b0010430 con 0x7fe7bc101470 2023-09-10T00:45:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.279+0000 7fe7ae7fc700 1 -- 172.21.15.44:0/652676511 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b000cea0 con 0x7fe7bc101470 2023-09-10T00:45:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.279+0000 7fe7c3a12700 1 -- 172.21.15.44:0/652676511 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc1a5800 con 0x7fe7bc101470 2023-09-10T00:45:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.282+0000 7fe7ae7fc700 1 -- 172.21.15.44:0/652676511 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fe7b00105a0 con 0x7fe7bc101470 2023-09-10T00:45:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.282+0000 7fe7ae7fc700 1 --2- 172.21.15.44:0/652676511 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe7a80a07b0 0x7fe7a80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.283+0000 7fe7c17ae700 1 --2- 172.21.15.44:0/652676511 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe7a80a07b0 0x7fe7a80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.283+0000 7fe7ae7fc700 1 -- 172.21.15.44:0/652676511 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe7b0014070 con 0x7fe7bc101470 2023-09-10T00:45:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.283+0000 7fe7c17ae700 1 --2- 172.21.15.44:0/652676511 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe7a80a07b0 0x7fe7a80a2c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80059d0 tx=0x7fe7b8005960 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.283+0000 7fe7cfbd8b80 1 Processor -- start 2023-09-10T00:45:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.283+0000 7fe7cfbd8b80 1 -- start start 2023-09-10T00:45:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7cfbd8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b1c1a0 0x55ff65b1c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7cfbd8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b16780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7cfbd8b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 0x55ff65b22f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7cfbd8b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff65a6f6d0 con 0x55ff65b16cc0 2023-09-10T00:45:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7cfbd8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff65a6cff0 con 0x55ff65b16300 2023-09-10T00:45:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7cfbd8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff65a688e0 con 0x55ff65b1c1a0 2023-09-10T00:45:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7c17ae700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b1c1a0 0x55ff65b1c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7c0fad700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b16780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7c1faf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 0x55ff65b22f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7c17ae700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b1c1a0 0x55ff65b1c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57936/0 (socket says 172.21.15.44:57936) 2023-09-10T00:45:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7c17ae700 1 -- 172.21.15.44:0/497349403 learned_addr learned my addr 172.21.15.44:0/497349403 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7c0fad700 1 --2- 172.21.15.44:0/497349403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b16780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52796/0 (socket says 172.21.15.44:52796) 2023-09-10T00:45:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7c17ae700 1 -- 172.21.15.44:0/497349403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 msgr2=0x55ff65b16780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7c17ae700 1 --2- 172.21.15.44:0/497349403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b16780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7c17ae700 1 -- 172.21.15.44:0/497349403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 msgr2=0x55ff65b22f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7c17ae700 1 --2- 172.21.15.44:0/497349403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 0x55ff65b22f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7c17ae700 1 -- 172.21.15.44:0/497349403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff65894d80 con 0x55ff65b1c1a0 2023-09-10T00:45:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7c1faf700 1 --2- 172.21.15.44:0/497349403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 0x55ff65b22f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7c0fad700 1 --2- 172.21.15.44:0/497349403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b16780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.284+0000 7fe7c17ae700 1 --2- 172.21.15.44:0/497349403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b1c1a0 0x55ff65b1c580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8015000 tx=0x7fe7b8020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e40f6dfa51c07455 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.285+0000 7fe7a77fe700 1 -- 172.21.15.44:0/497349403 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b802b070 con 0x55ff65b1c1a0 2023-09-10T00:45:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.285+0000 7fe7a77fe700 1 -- 172.21.15.44:0/497349403 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7b801e040 con 0x55ff65b1c1a0 2023-09-10T00:45:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.285+0000 7fe7a77fe700 1 -- 172.21.15.44:0/497349403 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8004a30 con 0x55ff65b1c1a0 2023-09-10T00:45:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.285+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/497349403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b1c1a0 msgr2=0x55ff65b1c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.285+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/497349403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b1c1a0 0x55ff65b1c580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8015000 tx=0x7fe7b8020000 comp rx=0 tx=0).stop 2023-09-10T00:45:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.285+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/497349403 shutdown_connections 2023-09-10T00:45:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.285+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/497349403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 0x55ff65b22f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.285+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/497349403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b1c1a0 0x55ff65b1c580 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.285+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/497349403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b16780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.285+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/497349403 >> 172.21.15.44:0/497349403 conn(0x55ff65a8b030 msgr2=0x55ff65b11560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.285+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/497349403 shutdown_connections 2023-09-10T00:45:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.285+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/497349403 wait complete. 2023-09-10T00:45:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.286+0000 7fe7cfbd8b80 1 -- start start 2023-09-10T00:45:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.286+0000 7fe7cfbd8b80 1 Processor -- start 2023-09-10T00:45:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.286+0000 7fe7cfbd8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b22e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.286+0000 7fe7cfbd8b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 0x55ff65b20080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.286+0000 7fe7cfbd8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b1c1a0 0x55ff65b20690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.286+0000 7fe7cfbd8b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff65a06b10 con 0x55ff65b16cc0 2023-09-10T00:45:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.286+0000 7fe7cfbd8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff65a06810 con 0x55ff65b16300 2023-09-10T00:45:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.286+0000 7fe7cfbd8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff658ea8c0 con 0x55ff65b1c1a0 2023-09-10T00:45:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.286+0000 7fe7cfbd8b80 10 client.0 _ll_register_callbacks cb 0x55ff65950a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:45:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.286+0000 7fe7c0fad700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 0x55ff65b20080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.286+0000 7fe7c17ae700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b22e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.287+0000 7fe7c17ae700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b22e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52800/0 (socket says 172.21.15.44:52800) 2023-09-10T00:45:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.287+0000 7fe7c17ae700 1 -- 172.21.15.44:0/829168736 learned_addr learned my addr 172.21.15.44:0/829168736 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.287+0000 7fe7c1faf700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b1c1a0 0x55ff65b20690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.287+0000 7fe7c17ae700 1 -- 172.21.15.44:0/829168736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b1c1a0 msgr2=0x55ff65b20690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.287+0000 7fe7c17ae700 1 --2- 172.21.15.44:0/829168736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b1c1a0 0x55ff65b20690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.287+0000 7fe7c17ae700 1 -- 172.21.15.44:0/829168736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 msgr2=0x55ff65b20080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.287+0000 7fe7c17ae700 1 --2- 172.21.15.44:0/829168736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 0x55ff65b20080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.287+0000 7fe7c17ae700 1 -- 172.21.15.44:0/829168736 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc1a5800 con 0x55ff65b16300 2023-09-10T00:45:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.287+0000 7fe7c17ae700 1 --2- 172.21.15.44:0/829168736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b22e70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009ce0 tx=0x7fe7b8004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7b4a787ebf5be87 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.287+0000 7fe7a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:45:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.287+0000 7fe7a4ff9700 1 -- 172.21.15.44:0/829168736 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b802b040 con 0x55ff65b16300 2023-09-10T00:45:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.287+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/829168736 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff659c0490 con 0x55ff65b16300 2023-09-10T00:45:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.287+0000 7fe7a4ff9700 1 -- 172.21.15.44:0/829168736 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe7b8005050 con 0x55ff65b16300 2023-09-10T00:45:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.287+0000 7fe7a4ff9700 1 -- 172.21.15.44:0/829168736 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7b8005210 con 0x55ff65b16300 2023-09-10T00:45:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.287+0000 7fe7cfbd8b80 10 client.26423 fetch_fsmap learned FSMap version 42 2023-09-10T00:45:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.287+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/829168736 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fe7b4011230 con 0x55ff65b16300 2023-09-10T00:45:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.288+0000 7fe7a4ff9700 1 -- 172.21.15.44:0/829168736 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ff659c0490 con 0x55ff65b16300 2023-09-10T00:45:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.288+0000 7fe7a4ff9700 1 -- 172.21.15.44:0/829168736 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe7b8030b70 con 0x55ff65b16300 2023-09-10T00:45:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.288+0000 7fe7cfbd8b80 10 client.26423 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:45:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.288+0000 7fe7cfbd8b80 10 client.26423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:45:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.288+0000 7fe7cfbd8b80 20 client.26423 populate_metadata read hostname 'smithi044' 2023-09-10T00:45:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.288+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/829168736 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff65b2fe90 0x55ff65b32280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.288+0000 7fe7cfbd8b80 4 client.26423 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:45:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.288+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/829168736 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff65a7d840 con 0x55ff65b2fe90 2023-09-10T00:45:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.288+0000 7fe7c0fad700 1 --2- 172.21.15.44:0/829168736 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff65b2fe90 0x55ff65b32280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.289+0000 7fe7a4ff9700 10 client.26423 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.289+0000 7fe7c0fad700 1 --2- 172.21.15.44:0/829168736 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff65b2fe90 0x55ff65b32280 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e60f39e64b1eeb9 server_cookie=dcb1f016594e5cdf in_seq=0 out_seq=0 2023-09-10T00:45:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.289+0000 7fe7a4ff9700 1 -- 172.21.15.44:0/829168736 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ff65a7d840 con 0x55ff65b2fe90 2023-09-10T00:45:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.289+0000 7fe7a4ff9700 10 client.26423 handle_command_reply: tid=0 2023-09-10T00:45:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.289+0000 7fe7cfbd8b80 1 client.26423 shutdown 2023-09-10T00:45:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.289+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/829168736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 msgr2=0x55ff65b22e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.289+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/829168736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b22e70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009ce0 tx=0x7fe7b8004a90 comp rx=0 tx=0).stop 2023-09-10T00:45:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.289+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/829168736 shutdown_connections 2023-09-10T00:45:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.289+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/829168736 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff65b2fe90 0x55ff65b32280 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.289+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/829168736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b1c1a0 0x55ff65b20690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.289+0000 7fe7a4ff9700 0 client.26423 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.289+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/829168736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b22e70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.289+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/829168736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 0x55ff65b20080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.289+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/829168736 >> 172.21.15.44:0/829168736 conn(0x55ff65a8b030 msgr2=0x55ff65b107a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.289+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/829168736 shutdown_connections 2023-09-10T00:45:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.290+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/829168736 wait complete. 2023-09-10T00:45:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.290+0000 7fe7cfbd8b80 20 client.26423 trim_cache size 0 max 16384 2023-09-10T00:45:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.304+0000 7fe7cfbd8b80 1 Processor -- start 2023-09-10T00:45:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7cfbd8b80 1 -- start start 2023-09-10T00:45:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7cfbd8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b16300 0x55ff65b229a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7cfbd8b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 0x55ff65b1b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7cfbd8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b1c1a0 0x55ff65b1b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7cfbd8b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff65b170a0 con 0x55ff65b16cc0 2023-09-10T00:45:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7cfbd8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff65b1c580 con 0x55ff65b1c1a0 2023-09-10T00:45:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7cfbd8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff65b21090 con 0x55ff65b16300 2023-09-10T00:45:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7c0fad700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 0x55ff65b1b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7c1faf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b1c1a0 0x55ff65b1b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7c0fad700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 0x55ff65b1b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51070/0 (socket says 172.21.15.44:51070) 2023-09-10T00:45:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7c0fad700 1 -- 172.21.15.44:0/351592738 learned_addr learned my addr 172.21.15.44:0/351592738 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7c1faf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b1c1a0 0x55ff65b1b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52810/0 (socket says 172.21.15.44:52810) 2023-09-10T00:45:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7c17ae700 1 --2- 172.21.15.44:0/351592738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b16300 0x55ff65b229a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7c0fad700 1 -- 172.21.15.44:0/351592738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b16300 msgr2=0x55ff65b229a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7c0fad700 1 --2- 172.21.15.44:0/351592738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b16300 0x55ff65b229a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7c0fad700 1 -- 172.21.15.44:0/351592738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b1c1a0 msgr2=0x55ff65b1b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7c0fad700 1 --2- 172.21.15.44:0/351592738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b1c1a0 0x55ff65b1b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7c0fad700 1 -- 172.21.15.44:0/351592738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc10fec0 con 0x55ff65b16cc0 2023-09-10T00:45:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7c17ae700 1 --2- 172.21.15.44:0/351592738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b16300 0x55ff65b229a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7c1faf700 1 --2- 172.21.15.44:0/351592738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b1c1a0 0x55ff65b1b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.305+0000 7fe7c0fad700 1 --2- 172.21.15.44:0/351592738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 0x55ff65b1b070 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0099580 tx=0x7fe7b000c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a24584567e5fc9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.306+0000 7fe7a7fff700 1 -- 172.21.15.44:0/351592738 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b00995f0 con 0x55ff65b16cc0 2023-09-10T00:45:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.306+0000 7fe7a7fff700 1 -- 172.21.15.44:0/351592738 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7b009d420 con 0x55ff65b16cc0 2023-09-10T00:45:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.306+0000 7fe7a7fff700 1 -- 172.21.15.44:0/351592738 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b00840b0 con 0x55ff65b16cc0 2023-09-10T00:45:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.306+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/351592738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 msgr2=0x55ff65b1b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.306+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/351592738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 0x55ff65b1b070 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0099580 tx=0x7fe7b000c480 comp rx=0 tx=0).stop 2023-09-10T00:45:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.306+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/351592738 shutdown_connections 2023-09-10T00:45:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.306+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/351592738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b1c1a0 0x55ff65b1b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.306+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/351592738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b16300 0x55ff65b229a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.306+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/351592738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b16cc0 0x55ff65b1b070 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.306+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/351592738 >> 172.21.15.44:0/351592738 conn(0x55ff65a8b030 msgr2=0x55ff65b1a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.306+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/351592738 shutdown_connections 2023-09-10T00:45:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.306+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/351592738 wait complete. 2023-09-10T00:45:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.307+0000 7fe7cfbd8b80 1 -- start start 2023-09-10T00:45:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.307+0000 7fe7cfbd8b80 1 Processor -- start 2023-09-10T00:45:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.307+0000 7fe7cfbd8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b27190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.307+0000 7fe7cfbd8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b16cc0 0x55ff65b254a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.307+0000 7fe7cfbd8b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b1c1a0 0x55ff65b259e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.307+0000 7fe7cfbd8b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff65b27a60 con 0x55ff65b1c1a0 2023-09-10T00:45:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.307+0000 7fe7cfbd8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff65b25f20 con 0x55ff65b16300 2023-09-10T00:45:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.307+0000 7fe7cfbd8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff65b26090 con 0x55ff65b16cc0 2023-09-10T00:45:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.307+0000 7fe7cfbd8b80 10 client.0 _ll_register_callbacks cb 0x7fe7b4009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:45:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.307+0000 7fe7c1faf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b1c1a0 0x55ff65b259e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.307+0000 7fe7c17ae700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b27190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.307+0000 7fe7c17ae700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b27190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52814/0 (socket says 172.21.15.44:52814) 2023-09-10T00:45:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.307+0000 7fe7c1faf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b1c1a0 0x55ff65b259e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51078/0 (socket says 172.21.15.44:51078) 2023-09-10T00:45:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.307+0000 7fe7c17ae700 1 -- 172.21.15.44:0/3316341765 learned_addr learned my addr 172.21.15.44:0/3316341765 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:45:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.307+0000 7fe7c0fad700 1 --2- 172.21.15.44:0/3316341765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b16cc0 0x55ff65b254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.308+0000 7fe7c17ae700 1 -- 172.21.15.44:0/3316341765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b16cc0 msgr2=0x55ff65b254a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.308+0000 7fe7c17ae700 1 --2- 172.21.15.44:0/3316341765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b16cc0 0x55ff65b254a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.308+0000 7fe7c17ae700 1 -- 172.21.15.44:0/3316341765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b1c1a0 msgr2=0x55ff65b259e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:45:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.308+0000 7fe7c17ae700 1 --2- 172.21.15.44:0/3316341765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b1c1a0 0x55ff65b259e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.308+0000 7fe7c17ae700 1 -- 172.21.15.44:0/3316341765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b4011230 con 0x55ff65b16300 2023-09-10T00:45:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.308+0000 7fe7c1faf700 1 --2- 172.21.15.44:0/3316341765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b1c1a0 0x55ff65b259e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:45:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.308+0000 7fe7c17ae700 1 --2- 172.21.15.44:0/3316341765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b27190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80098e0 tx=0x7fe7b80047f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea942c5221868875 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:45:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.308+0000 7fe7a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:45:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.308+0000 7fe7a4ff9700 1 -- 172.21.15.44:0/3316341765 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b802b040 con 0x55ff65b16300 2023-09-10T00:45:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.308+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/3316341765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7b801e040 con 0x55ff65b16300 2023-09-10T00:45:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.308+0000 7fe7a4ff9700 1 -- 172.21.15.44:0/3316341765 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe7b8004b00 con 0x55ff65b16300 2023-09-10T00:45:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.308+0000 7fe7a4ff9700 1 -- 172.21.15.44:0/3316341765 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7b800fb30 con 0x55ff65b16300 2023-09-10T00:45:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.308+0000 7fe7cfbd8b80 10 client.26429 fetch_fsmap learned FSMap version 42 2023-09-10T00:45:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.308+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/3316341765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fe7b40091b0 con 0x55ff65b16300 2023-09-10T00:45:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.308+0000 7fe7a4ff9700 1 -- 172.21.15.44:0/3316341765 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7b801e040 con 0x55ff65b16300 2023-09-10T00:45:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.309+0000 7fe7a4ff9700 1 -- 172.21.15.44:0/3316341765 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe7b8030460 con 0x55ff65b16300 2023-09-10T00:45:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.309+0000 7fe7cfbd8b80 10 client.26429 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:45:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.309+0000 7fe7cfbd8b80 10 client.26429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:45:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.309+0000 7fe7cfbd8b80 20 client.26429 populate_metadata read hostname 'smithi044' 2023-09-10T00:45:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.309+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/3316341765 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff65b84cb0 0x55ff65b870d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:45:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.309+0000 7fe7cfbd8b80 4 client.26429 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:45:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.309+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/3316341765 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ff65a618c0 con 0x55ff65b84cb0 2023-09-10T00:45:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.309+0000 7fe7c0fad700 1 --2- 172.21.15.44:0/3316341765 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff65b84cb0 0x55ff65b870d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:45:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.309+0000 7fe7a4ff9700 10 client.26429 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.309+0000 7fe7c0fad700 1 --2- 172.21.15.44:0/3316341765 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff65b84cb0 0x55ff65b870d0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e30e1a4ad98b7412 server_cookie=ba91a5ca75f35899 in_seq=0 out_seq=0 2023-09-10T00:45:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.322+0000 7fe7a4ff9700 1 -- 172.21.15.44:0/3316341765 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ff65a618c0 con 0x55ff65b84cb0 2023-09-10T00:45:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.322+0000 7fe7a4ff9700 10 client.26429 handle_command_reply: tid=0 2023-09-10T00:45:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.322+0000 7fe7cfbd8b80 1 client.26429 shutdown 2023-09-10T00:45:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.323+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/3316341765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 msgr2=0x55ff65b27190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.323+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/3316341765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b27190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80098e0 tx=0x7fe7b80047f0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.323+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/3316341765 shutdown_connections 2023-09-10T00:45:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.323+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/3316341765 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff65b84cb0 0x55ff65b870d0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.323+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/3316341765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff65b1c1a0 0x55ff65b259e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.323+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/3316341765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff65b16300 0x55ff65b27190 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.323+0000 7fe7cfbd8b80 1 --2- 172.21.15.44:0/3316341765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff65b16cc0 0x55ff65b254a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.323+0000 7fe7a4ff9700 0 client.26429 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:45:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.323+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/3316341765 >> 172.21.15.44:0/3316341765 conn(0x55ff65a8b030 msgr2=0x55ff65b1b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.323+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/3316341765 shutdown_connections 2023-09-10T00:45:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.323+0000 7fe7cfbd8b80 1 -- 172.21.15.44:0/3316341765 wait complete. 2023-09-10T00:45:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.323+0000 7fe7cfbd8b80 20 client.26429 trim_cache size 0 max 16384 2023-09-10T00:45:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:45:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:45:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:45:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:45:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.323+0000 7fe7c3a12700 1 -- 172.21.15.44:0/652676511 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe7a80a07b0 msgr2=0x7fe7a80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.323+0000 7fe7c3a12700 1 --2- 172.21.15.44:0/652676511 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe7a80a07b0 0x7fe7a80a2c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80059d0 tx=0x7fe7b8005960 comp rx=0 tx=0).stop 2023-09-10T00:45:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.323+0000 7fe7c3a12700 1 -- 172.21.15.44:0/652676511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7bc101470 msgr2=0x7fe7bc198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:45:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.323+0000 7fe7c3a12700 1 --2- 172.21.15.44:0/652676511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7bc101470 0x7fe7bc198cd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000e9a0 tx=0x7fe7b000ecb0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.324+0000 7fe7c3a12700 1 -- 172.21.15.44:0/652676511 shutdown_connections 2023-09-10T00:45:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.324+0000 7fe7c3a12700 1 --2- 172.21.15.44:0/652676511 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe7a80a07b0 0x7fe7a80a2c70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.324+0000 7fe7c3a12700 1 --2- 172.21.15.44:0/652676511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7bc10cdd0 0x7fe7bc19d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.324+0000 7fe7c3a12700 1 --2- 172.21.15.44:0/652676511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7bc067f30 0x7fe7bc198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.324+0000 7fe7c3a12700 1 --2- 172.21.15.44:0/652676511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7bc101470 0x7fe7bc198cd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:45:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.324+0000 7fe7c3a12700 1 -- 172.21.15.44:0/652676511 >> 172.21.15.44:0/652676511 conn(0x7fe7bc074f30 msgr2=0x7fe7bc0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:45:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.324+0000 7fe7c3a12700 1 -- 172.21.15.44:0/652676511 shutdown_connections 2023-09-10T00:45:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:45:58.324+0000 7fe7c3a12700 1 -- 172.21.15.44:0/652676511 wait complete. 2023-09-10T00:45:58.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:45:58 smithi099 ceph-mon[102692]: pgmap v162: 129 pgs: 129 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 22 MiB/s wr, 63 op/s 2023-09-10T00:45:59.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:45:58 smithi131 ceph-mon[103713]: pgmap v162: 129 pgs: 129 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 22 MiB/s wr, 63 op/s 2023-09-10T00:45:59.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:45:58 smithi044 ceph-mon[98676]: pgmap v162: 129 pgs: 129 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 22 MiB/s wr, 63 op/s 2023-09-10T00:45:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:26300000 of 50000000 tuples (52%) done (elapsed 299.29 s, remaining 269.70 s) 2023-09-10T00:46:00.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:00 smithi099 ceph-mon[102692]: pgmap v163: 129 pgs: 129 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 22 MiB/s wr, 63 op/s 2023-09-10T00:46:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:00 smithi131 ceph-mon[103713]: pgmap v163: 129 pgs: 129 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 22 MiB/s wr, 63 op/s 2023-09-10T00:46:01.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:00 smithi044 ceph-mon[98676]: pgmap v163: 129 pgs: 129 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 22 MiB/s wr, 63 op/s 2023-09-10T00:46:01.386 INFO:teuthology.orchestra.run.smithi044.stderr:26400000 of 50000000 tuples (52%) done (elapsed 301.02 s, remaining 269.09 s) 2023-09-10T00:46:02.693 INFO:teuthology.orchestra.run.smithi044.stderr:26500000 of 50000000 tuples (53%) done (elapsed 302.33 s, remaining 268.10 s) 2023-09-10T00:46:02.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:02 smithi099 ceph-mon[102692]: pgmap v164: 129 pgs: 129 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 26 MiB/s wr, 76 op/s 2023-09-10T00:46:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:02 smithi131 ceph-mon[103713]: pgmap v164: 129 pgs: 129 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 26 MiB/s wr, 76 op/s 2023-09-10T00:46:03.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:02 smithi044 ceph-mon[98676]: pgmap v164: 129 pgs: 129 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 26 MiB/s wr, 76 op/s 2023-09-10T00:46:03.633 INFO:teuthology.orchestra.run.smithi044.stderr:26600000 of 50000000 tuples (53%) done (elapsed 303.27 s, remaining 266.78 s) 2023-09-10T00:46:04.745 INFO:teuthology.orchestra.run.smithi044.stderr:26700000 of 50000000 tuples (53%) done (elapsed 304.38 s, remaining 265.62 s) 2023-09-10T00:46:04.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:04 smithi099 ceph-mon[102692]: pgmap v165: 129 pgs: 129 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 56 op/s 2023-09-10T00:46:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:04 smithi131 ceph-mon[103713]: pgmap v165: 129 pgs: 129 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 56 op/s 2023-09-10T00:46:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:04 smithi044 ceph-mon[98676]: pgmap v165: 129 pgs: 129 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 56 op/s 2023-09-10T00:46:05.675 INFO:teuthology.orchestra.run.smithi044.stderr:26800000 of 50000000 tuples (53%) done (elapsed 305.31 s, remaining 264.30 s) 2023-09-10T00:46:06.692 INFO:teuthology.orchestra.run.smithi044.stderr:26900000 of 50000000 tuples (53%) done (elapsed 306.33 s, remaining 263.05 s) 2023-09-10T00:46:06.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:06 smithi099 ceph-mon[102692]: pgmap v166: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 24 MiB/s wr, 115 op/s 2023-09-10T00:46:07.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:06 smithi131 ceph-mon[103713]: pgmap v166: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 24 MiB/s wr, 115 op/s 2023-09-10T00:46:07.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:06 smithi044 ceph-mon[98676]: pgmap v166: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 24 MiB/s wr, 115 op/s 2023-09-10T00:46:07.680 INFO:teuthology.orchestra.run.smithi044.stderr:27000000 of 50000000 tuples (54%) done (elapsed 307.31 s, remaining 261.79 s) 2023-09-10T00:46:08.195 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:46:08.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.350+0000 7f8975023700 1 -- 172.21.15.44:0/186864833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89701054e0 msgr2=0x7f89701058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:08.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.350+0000 7f8975023700 1 --2- 172.21.15.44:0/186864833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89701054e0 0x7f89701058c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8960009990 tx=0x7f8960009ca0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.350+0000 7f8975023700 1 -- 172.21.15.44:0/186864833 shutdown_connections 2023-09-10T00:46:08.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.350+0000 7f8975023700 1 --2- 172.21.15.44:0/186864833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89700ffe00 0x7f897010c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.350+0000 7f8975023700 1 --2- 172.21.15.44:0/186864833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89701054e0 0x7f89701058c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.356 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.350+0000 7f8975023700 1 --2- 172.21.15.44:0/186864833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89700ff440 0x7f89700ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.350+0000 7f8975023700 1 -- 172.21.15.44:0/186864833 >> 172.21.15.44:0/186864833 conn(0x7f89700faf50 msgr2=0x7f89700fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:08.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.350+0000 7f8975023700 1 -- 172.21.15.44:0/186864833 shutdown_connections 2023-09-10T00:46:08.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.350+0000 7f8975023700 1 -- 172.21.15.44:0/186864833 wait complete. 2023-09-10T00:46:08.357 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.351+0000 7f8975023700 1 Processor -- start 2023-09-10T00:46:08.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.351+0000 7f8975023700 1 -- start start 2023-09-10T00:46:08.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.351+0000 7f8975023700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89700ff440 0x7f89700fb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.351+0000 7f8975023700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89700ffe00 0x7f89700fbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.351+0000 7f8975023700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89701054e0 0x7f89701a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.351+0000 7f8975023700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89700fc4b0 con 0x7f89700ff440 2023-09-10T00:46:08.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.351+0000 7f8975023700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89701a4c40 con 0x7f89700ffe00 2023-09-10T00:46:08.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.351+0000 7f8975023700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89701a4d80 con 0x7f89701054e0 2023-09-10T00:46:08.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.351+0000 7f896ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89700ff440 0x7f89700fb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.351+0000 7f896e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89700ffe00 0x7f89700fbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.352+0000 7f896f59e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89701054e0 0x7f89701a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.352+0000 7f896ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89700ff440 0x7f89700fb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51088/0 (socket says 172.21.15.44:51088) 2023-09-10T00:46:08.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.352+0000 7f896ed9d700 1 -- 172.21.15.44:0/1267618274 learned_addr learned my addr 172.21.15.44:0/1267618274 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:08.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.352+0000 7f896e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89700ffe00 0x7f89700fbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52830/0 (socket says 172.21.15.44:52830) 2023-09-10T00:46:08.361 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.352+0000 7f896ed9d700 1 -- 172.21.15.44:0/1267618274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89701054e0 msgr2=0x7f89701a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:08.362 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.352+0000 7f896ed9d700 1 --2- 172.21.15.44:0/1267618274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89701054e0 0x7f89701a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.362 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.352+0000 7f896ed9d700 1 -- 172.21.15.44:0/1267618274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89700ffe00 msgr2=0x7f89700fbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:08.362 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.352+0000 7f896ed9d700 1 --2- 172.21.15.44:0/1267618274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89700ffe00 0x7f89700fbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.352+0000 7f896ed9d700 1 -- 172.21.15.44:0/1267618274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8958009520 con 0x7f89700ff440 2023-09-10T00:46:08.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.352+0000 7f896e59c700 1 --2- 172.21.15.44:0/1267618274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89700ffe00 0x7f89700fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:08.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.352+0000 7f896ed9d700 1 --2- 172.21.15.44:0/1267618274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89700ff440 0x7f89700fb6d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f89600047b0 tx=0x7f896000bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:08.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.352+0000 7f8967fff700 1 -- 172.21.15.44:0/1267618274 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896001d070 con 0x7f89700ff440 2023-09-10T00:46:08.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.352+0000 7f8967fff700 1 -- 172.21.15.44:0/1267618274 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8960021410 con 0x7f89700ff440 2023-09-10T00:46:08.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.352+0000 7f8975023700 1 -- 172.21.15.44:0/1267618274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89600095f0 con 0x7f89700ff440 2023-09-10T00:46:08.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.352+0000 7f8967fff700 1 -- 172.21.15.44:0/1267618274 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8960017650 con 0x7f89700ff440 2023-09-10T00:46:08.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.352+0000 7f8975023700 1 -- 172.21.15.44:0/1267618274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89701a5320 con 0x7f89700ff440 2023-09-10T00:46:08.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.354+0000 7f8967fff700 1 -- 172.21.15.44:0/1267618274 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f8960021580 con 0x7f89700ff440 2023-09-10T00:46:08.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.355+0000 7f8967fff700 1 --2- 172.21.15.44:0/1267618274 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f895c0a07b0 0x7f895c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.355+0000 7f896e59c700 1 --2- 172.21.15.44:0/1267618274 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f895c0a07b0 0x7f895c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.355+0000 7f8967fff700 1 -- 172.21.15.44:0/1267618274 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f89600cd640 con 0x7f89700ff440 2023-09-10T00:46:08.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.356+0000 7f896e59c700 1 --2- 172.21.15.44:0/1267618274 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f895c0a07b0 0x7f895c0a2c70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8958009fd0 tx=0x7f8958009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:08.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.356+0000 7f89811e9b80 1 Processor -- start 2023-09-10T00:46:08.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.356+0000 7f89811e9b80 1 -- start start 2023-09-10T00:46:08.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f89811e9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fd5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f89811e9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fd6030 0x5575b6fdf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f89811e9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fdfe40 0x5575b6fe2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f89811e9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b6f2e730 con 0x5575b6fdfe40 2023-09-10T00:46:08.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f89811e9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b6f2c050 con 0x5575b6fd6030 2023-09-10T00:46:08.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f89811e9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b6f27940 con 0x5575b6fd5710 2023-09-10T00:46:08.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f896ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fd5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.380 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f896f59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fdfe40 0x5575b6fe2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.380 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f896ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fd5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57978/0 (socket says 172.21.15.44:57978) 2023-09-10T00:46:08.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f896ed9d700 1 -- 172.21.15.44:0/1963187909 learned_addr learned my addr 172.21.15.44:0/1963187909 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:08.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f896e59c700 1 --2- 172.21.15.44:0/1963187909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fd6030 0x5575b6fdf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f896ed9d700 1 -- 172.21.15.44:0/1963187909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fd6030 msgr2=0x5575b6fdf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:08.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f896ed9d700 1 --2- 172.21.15.44:0/1963187909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fd6030 0x5575b6fdf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f896ed9d700 1 -- 172.21.15.44:0/1963187909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fdfe40 msgr2=0x5575b6fe2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:08.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f896ed9d700 1 --2- 172.21.15.44:0/1963187909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fdfe40 0x5575b6fe2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f896ed9d700 1 -- 172.21.15.44:0/1963187909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575b6d53de0 con 0x5575b6fd5710 2023-09-10T00:46:08.383 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f896f59e700 1 --2- 172.21.15.44:0/1963187909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fdfe40 0x5575b6fe2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:08.383 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.357+0000 7f896ed9d700 1 --2- 172.21.15.44:0/1963187909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fd5af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f896009a770 tx=0x7f896009a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6354c3bb16e298f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:08.383 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.358+0000 7f89657fa700 1 -- 172.21.15.44:0/1963187909 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896009a7e0 con 0x5575b6fd5710 2023-09-10T00:46:08.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.358+0000 7f89657fa700 1 -- 172.21.15.44:0/1963187909 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89600cc070 con 0x5575b6fd5710 2023-09-10T00:46:08.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.358+0000 7f89657fa700 1 -- 172.21.15.44:0/1963187909 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8960098c80 con 0x5575b6fd5710 2023-09-10T00:46:08.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.358+0000 7f89811e9b80 1 -- 172.21.15.44:0/1963187909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 msgr2=0x5575b6fd5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:08.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.358+0000 7f89811e9b80 1 --2- 172.21.15.44:0/1963187909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fd5af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f896009a770 tx=0x7f896009a630 comp rx=0 tx=0).stop 2023-09-10T00:46:08.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.358+0000 7f89811e9b80 1 -- 172.21.15.44:0/1963187909 shutdown_connections 2023-09-10T00:46:08.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.358+0000 7f89811e9b80 1 --2- 172.21.15.44:0/1963187909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fdfe40 0x5575b6fe2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.358+0000 7f89811e9b80 1 --2- 172.21.15.44:0/1963187909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fd5af0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.358+0000 7f89811e9b80 1 --2- 172.21.15.44:0/1963187909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fd6030 0x5575b6fdf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.358+0000 7f89811e9b80 1 -- 172.21.15.44:0/1963187909 >> 172.21.15.44:0/1963187909 conn(0x5575b6f4a090 msgr2=0x5575b6fd0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:08.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.359+0000 7f89811e9b80 1 -- 172.21.15.44:0/1963187909 shutdown_connections 2023-09-10T00:46:08.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.359+0000 7f89811e9b80 1 -- 172.21.15.44:0/1963187909 wait complete. 2023-09-10T00:46:08.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.359+0000 7f89811e9b80 1 -- start start 2023-09-10T00:46:08.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.359+0000 7f89811e9b80 1 Processor -- start 2023-09-10T00:46:08.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.359+0000 7f89811e9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fe0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.359+0000 7f89811e9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fd6030 0x5575b6fe0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.359+0000 7f89811e9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fdfe40 0x5575b6feb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.359+0000 7f89811e9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b6ec5b70 con 0x5575b6fd6030 2023-09-10T00:46:08.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.359+0000 7f89811e9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b6ec5870 con 0x5575b6fdfe40 2023-09-10T00:46:08.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.359+0000 7f89811e9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b6da9920 con 0x5575b6fd5710 2023-09-10T00:46:08.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f89811e9b80 10 client.0 _ll_register_callbacks cb 0x5575b6e0fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:46:08.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f896ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fe0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f896e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fd6030 0x5575b6fe0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f896e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fd6030 0x5575b6fe0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51104/0 (socket says 172.21.15.44:51104) 2023-09-10T00:46:08.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f896ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fe0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57984/0 (socket says 172.21.15.44:57984) 2023-09-10T00:46:08.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f896e59c700 1 -- 172.21.15.44:0/2047164710 learned_addr learned my addr 172.21.15.44:0/2047164710 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:08.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f896f59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fdfe40 0x5575b6feb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f896ed9d700 1 -- 172.21.15.44:0/2047164710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fdfe40 msgr2=0x5575b6feb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:08.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f896ed9d700 1 --2- 172.21.15.44:0/2047164710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fdfe40 0x5575b6feb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f896ed9d700 1 -- 172.21.15.44:0/2047164710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fd6030 msgr2=0x5575b6fe0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:08.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f896ed9d700 1 --2- 172.21.15.44:0/2047164710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fd6030 0x5575b6fe0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f896ed9d700 1 -- 172.21.15.44:0/2047164710 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8968011230 con 0x5575b6fd5710 2023-09-10T00:46:08.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f896f59e700 1 --2- 172.21.15.44:0/2047164710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fdfe40 0x5575b6feb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:08.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f896ed9d700 1 --2- 172.21.15.44:0/2047164710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fe0220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8960096ce0 tx=0x7f8960005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1000e10f13a2ac9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f89567fc700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:46:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f89567fc700 1 -- 172.21.15.44:0/2047164710 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896009a7e0 con 0x5575b6fd5710 2023-09-10T00:46:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f89811e9b80 1 -- 172.21.15.44:0/2047164710 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575b6e7f4f0 con 0x5575b6fd5710 2023-09-10T00:46:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f89567fc700 1 -- 172.21.15.44:0/2047164710 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f89600a27c0 con 0x5575b6fd5710 2023-09-10T00:46:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f89567fc700 1 -- 172.21.15.44:0/2047164710 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f896009e610 con 0x5575b6fd5710 2023-09-10T00:46:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.360+0000 7f89811e9b80 10 client.26383 fetch_fsmap learned FSMap version 42 2023-09-10T00:46:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.361+0000 7f89811e9b80 1 -- 172.21.15.44:0/2047164710 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f89701a5320 con 0x5575b6fd5710 2023-09-10T00:46:08.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.361+0000 7f89567fc700 1 -- 172.21.15.44:0/2047164710 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5575b6e7f4f0 con 0x5575b6fd5710 2023-09-10T00:46:08.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.361+0000 7f89567fc700 1 -- 172.21.15.44:0/2047164710 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f89600a2330 con 0x5575b6fd5710 2023-09-10T00:46:08.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.361+0000 7f89811e9b80 10 client.26383 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:46:08.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.361+0000 7f89811e9b80 10 client.26383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:46:08.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.361+0000 7f89811e9b80 20 client.26383 populate_metadata read hostname 'smithi044' 2023-09-10T00:46:08.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.361+0000 7f89811e9b80 1 --2- 172.21.15.44:0/2047164710 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5575b6fef160 0x5575b6ff1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.361+0000 7f89811e9b80 4 client.26383 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:46:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.361+0000 7f89811e9b80 1 -- 172.21.15.44:0/2047164710 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575b6f3c8a0 con 0x5575b6fef160 2023-09-10T00:46:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.361+0000 7f896e59c700 1 --2- 172.21.15.44:0/2047164710 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5575b6fef160 0x5575b6ff1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.362+0000 7f896e59c700 1 --2- 172.21.15.44:0/2047164710 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5575b6fef160 0x5575b6ff1550 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be651bf5c68bc799 server_cookie=156927adffb3d9e9 in_seq=0 out_seq=0 2023-09-10T00:46:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.362+0000 7f89567fc700 10 client.26383 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.362+0000 7f89567fc700 1 -- 172.21.15.44:0/2047164710 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5575b6f3c8a0 con 0x5575b6fef160 2023-09-10T00:46:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.362+0000 7f89567fc700 10 client.26383 handle_command_reply: tid=0 2023-09-10T00:46:08.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.362+0000 7f89811e9b80 1 client.26383 shutdown 2023-09-10T00:46:08.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.362+0000 7f89811e9b80 1 -- 172.21.15.44:0/2047164710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 msgr2=0x5575b6fe0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:08.400 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.362+0000 7f89811e9b80 1 --2- 172.21.15.44:0/2047164710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fe0220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8960096ce0 tx=0x7f8960005fb0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.362+0000 7f89811e9b80 1 -- 172.21.15.44:0/2047164710 shutdown_connections 2023-09-10T00:46:08.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.362+0000 7f89811e9b80 1 --2- 172.21.15.44:0/2047164710 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5575b6fef160 0x5575b6ff1550 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.401 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.362+0000 7f89811e9b80 1 --2- 172.21.15.44:0/2047164710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fdfe40 0x5575b6feb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.362+0000 7f89567fc700 0 client.26383 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:08.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.362+0000 7f89811e9b80 1 --2- 172.21.15.44:0/2047164710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fe0220 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.402 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.362+0000 7f89811e9b80 1 --2- 172.21.15.44:0/2047164710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fd6030 0x5575b6fe0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.363+0000 7f89811e9b80 1 -- 172.21.15.44:0/2047164710 >> 172.21.15.44:0/2047164710 conn(0x5575b6f4a090 msgr2=0x5575b6fd07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:08.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.363+0000 7f89811e9b80 1 -- 172.21.15.44:0/2047164710 shutdown_connections 2023-09-10T00:46:08.403 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.363+0000 7f89811e9b80 1 -- 172.21.15.44:0/2047164710 wait complete. 2023-09-10T00:46:08.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.363+0000 7f89811e9b80 20 client.26383 trim_cache size 0 max 16384 2023-09-10T00:46:08.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.380+0000 7f89811e9b80 1 Processor -- start 2023-09-10T00:46:08.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.380+0000 7f89811e9b80 1 -- start start 2023-09-10T00:46:08.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.380+0000 7f89811e9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fd5710 0x5575b6fe5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.380+0000 7f89811e9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd6030 0x5575b6fe0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.380+0000 7f89811e9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fdfe40 0x5575b6fe0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.380+0000 7f89811e9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b6fd9e80 con 0x5575b6fdfe40 2023-09-10T00:46:08.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.380+0000 7f89811e9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b6fe5760 con 0x5575b6fd5710 2023-09-10T00:46:08.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.380+0000 7f89811e9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b6fdd520 con 0x5575b6fd6030 2023-09-10T00:46:08.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.380+0000 7f896f59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fdfe40 0x5575b6fe0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.380+0000 7f896e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd6030 0x5575b6fe0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.380+0000 7f896ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fd5710 0x5575b6fe5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.380+0000 7f896e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd6030 0x5575b6fe0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:57994/0 (socket says 172.21.15.44:57994) 2023-09-10T00:46:08.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.380+0000 7f896e59c700 1 -- 172.21.15.44:0/2606926916 learned_addr learned my addr 172.21.15.44:0/2606926916 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:08.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.380+0000 7f896f59e700 1 --2- 172.21.15.44:0/2606926916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fdfe40 0x5575b6fe0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51114/0 (socket says 172.21.15.44:51114) 2023-09-10T00:46:08.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.381+0000 7f896e59c700 1 -- 172.21.15.44:0/2606926916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fd5710 msgr2=0x5575b6fe5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:08.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.381+0000 7f896e59c700 1 --2- 172.21.15.44:0/2606926916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fd5710 0x5575b6fe5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.381+0000 7f896e59c700 1 -- 172.21.15.44:0/2606926916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fdfe40 msgr2=0x5575b6fe0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:08.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.381+0000 7f896e59c700 1 --2- 172.21.15.44:0/2606926916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fdfe40 0x5575b6fe0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.381+0000 7f896e59c700 1 -- 172.21.15.44:0/2606926916 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575b6d3cc60 con 0x5575b6fd6030 2023-09-10T00:46:08.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.381+0000 7f896e59c700 1 --2- 172.21.15.44:0/2606926916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd6030 0x5575b6fe0220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x5575b6fdf610 tx=0x7f8958018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=825d0f95cde252fa server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:08.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.381+0000 7f8965ffb700 1 -- 172.21.15.44:0/2606926916 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958023040 con 0x5575b6fd6030 2023-09-10T00:46:08.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.381+0000 7f8965ffb700 1 -- 172.21.15.44:0/2606926916 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8958028d90 con 0x5575b6fd6030 2023-09-10T00:46:08.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.381+0000 7f8965ffb700 1 -- 172.21.15.44:0/2606926916 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895801d850 con 0x5575b6fd6030 2023-09-10T00:46:08.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 -- 172.21.15.44:0/2606926916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd6030 msgr2=0x5575b6fe0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:08.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 --2- 172.21.15.44:0/2606926916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd6030 0x5575b6fe0220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x5575b6fdf610 tx=0x7f8958018000 comp rx=0 tx=0).stop 2023-09-10T00:46:08.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 -- 172.21.15.44:0/2606926916 shutdown_connections 2023-09-10T00:46:08.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 --2- 172.21.15.44:0/2606926916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fdfe40 0x5575b6fe0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 --2- 172.21.15.44:0/2606926916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fd5710 0x5575b6fe5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 --2- 172.21.15.44:0/2606926916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd6030 0x5575b6fe0220 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 -- 172.21.15.44:0/2606926916 >> 172.21.15.44:0/2606926916 conn(0x5575b6f4a090 msgr2=0x5575b6fda360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:08.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 -- 172.21.15.44:0/2606926916 shutdown_connections 2023-09-10T00:46:08.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 -- 172.21.15.44:0/2606926916 wait complete. 2023-09-10T00:46:08.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 -- start start 2023-09-10T00:46:08.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 Processor -- start 2023-09-10T00:46:08.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fe51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fd6030 0x5575b6fe1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fdfe40 0x5575b6fe1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b6fe2480 con 0x5575b6fd6030 2023-09-10T00:46:08.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7042f20 con 0x5575b6fdfe40 2023-09-10T00:46:08.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.382+0000 7f89811e9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7043060 con 0x5575b6fd5710 2023-09-10T00:46:08.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f89811e9b80 10 client.0 _ll_register_callbacks cb 0x7f8968009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:46:08.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f896e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fd6030 0x5575b6fe1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f896ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fe51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f896e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fd6030 0x5575b6fe1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51118/0 (socket says 172.21.15.44:51118) 2023-09-10T00:46:08.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f896e59c700 1 -- 172.21.15.44:0/1652386427 learned_addr learned my addr 172.21.15.44:0/1652386427 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:08.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f896f59e700 1 --2- 172.21.15.44:0/1652386427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fdfe40 0x5575b6fe1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.424 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f896e59c700 1 -- 172.21.15.44:0/1652386427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 msgr2=0x5575b6fe51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:08.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f896e59c700 1 --2- 172.21.15.44:0/1652386427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fe51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f896e59c700 1 -- 172.21.15.44:0/1652386427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fdfe40 msgr2=0x5575b6fe1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:08.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f896e59c700 1 --2- 172.21.15.44:0/1652386427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fdfe40 0x5575b6fe1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f896e59c700 1 -- 172.21.15.44:0/1652386427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8958017050 con 0x5575b6fd6030 2023-09-10T00:46:08.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f896ed9d700 1 --2- 172.21.15.44:0/1652386427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fe51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:08.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f896e59c700 1 --2- 172.21.15.44:0/1652386427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fd6030 0x5575b6fe1650 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f895802f000 tx=0x7f8958029fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103ea5f3c10826d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:08.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f89567fc700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:46:08.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f89567fc700 1 -- 172.21.15.44:0/1652386427 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958023040 con 0x5575b6fd6030 2023-09-10T00:46:08.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f89811e9b80 1 -- 172.21.15.44:0/1652386427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89600cc070 con 0x5575b6fd6030 2023-09-10T00:46:08.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f89567fc700 1 -- 172.21.15.44:0/1652386427 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f8958028420 con 0x5575b6fd6030 2023-09-10T00:46:08.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f89567fc700 1 -- 172.21.15.44:0/1652386427 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8958028900 con 0x5575b6fd6030 2023-09-10T00:46:08.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.383+0000 7f89811e9b80 10 client.16782 fetch_fsmap learned FSMap version 42 2023-09-10T00:46:08.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.384+0000 7f89811e9b80 1 -- 172.21.15.44:0/1652386427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5575b6d7a250 con 0x5575b6fd6030 2023-09-10T00:46:08.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.384+0000 7f89567fc700 1 -- 172.21.15.44:0/1652386427 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89600cc070 con 0x5575b6fd6030 2023-09-10T00:46:08.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.384+0000 7f89567fc700 1 -- 172.21.15.44:0/1652386427 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f89580047e0 con 0x5575b6fd6030 2023-09-10T00:46:08.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.384+0000 7f89811e9b80 10 client.16782 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:46:08.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.384+0000 7f89811e9b80 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:46:08.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.384+0000 7f89811e9b80 20 client.16782 populate_metadata read hostname 'smithi044' 2023-09-10T00:46:08.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.384+0000 7f89811e9b80 1 --2- 172.21.15.44:0/1652386427 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5575b70447e0 0x5575b7046c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:08.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.384+0000 7f89811e9b80 4 client.16782 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:46:08.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.384+0000 7f89811e9b80 1 -- 172.21.15.44:0/1652386427 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5575b6f20920 con 0x5575b70447e0 2023-09-10T00:46:08.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.384+0000 7f896ed9d700 1 --2- 172.21.15.44:0/1652386427 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5575b70447e0 0x5575b7046c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:08.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.384+0000 7f89567fc700 10 client.16782 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:08.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.384+0000 7f896ed9d700 1 --2- 172.21.15.44:0/1652386427 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5575b70447e0 0x5575b7046c00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c674d679a78987ad server_cookie=6ab6f7ea79c25efe in_seq=0 out_seq=0 2023-09-10T00:46:08.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f89567fc700 1 -- 172.21.15.44:0/1652386427 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5575b6f20920 con 0x5575b70447e0 2023-09-10T00:46:08.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f89567fc700 10 client.16782 handle_command_reply: tid=0 2023-09-10T00:46:08.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f89811e9b80 1 client.16782 shutdown 2023-09-10T00:46:08.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f89811e9b80 1 -- 172.21.15.44:0/1652386427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fd6030 msgr2=0x5575b6fe1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:08.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f89811e9b80 1 --2- 172.21.15.44:0/1652386427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fd6030 0x5575b6fe1650 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f895802f000 tx=0x7f8958029fa0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f89811e9b80 1 -- 172.21.15.44:0/1652386427 shutdown_connections 2023-09-10T00:46:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f89811e9b80 1 --2- 172.21.15.44:0/1652386427 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5575b70447e0 0x5575b7046c00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f89811e9b80 1 --2- 172.21.15.44:0/1652386427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5575b6fdfe40 0x5575b6fe1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f89567fc700 0 client.16782 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f89811e9b80 1 --2- 172.21.15.44:0/1652386427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5575b6fd5710 0x5575b6fe51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f89811e9b80 1 --2- 172.21.15.44:0/1652386427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575b6fd6030 0x5575b6fe1650 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f89811e9b80 1 -- 172.21.15.44:0/1652386427 >> 172.21.15.44:0/1652386427 conn(0x5575b6f4a090 msgr2=0x5575b6fe0220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:08.434 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f89811e9b80 1 -- 172.21.15.44:0/1652386427 shutdown_connections 2023-09-10T00:46:08.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f89811e9b80 1 -- 172.21.15.44:0/1652386427 wait complete. 2023-09-10T00:46:08.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f89811e9b80 20 client.16782 trim_cache size 0 max 16384 2023-09-10T00:46:08.436 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:46:08.436 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:46:08.436 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:46:08.437 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:46:08.437 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:46:08.437 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:46:08.438 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:46:08.438 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:46:08.438 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:46:08.439 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:46:08.440 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f8975023700 1 -- 172.21.15.44:0/1267618274 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f895c0a07b0 msgr2=0x7f895c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:08.441 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f8975023700 1 --2- 172.21.15.44:0/1267618274 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f895c0a07b0 0x7f895c0a2c70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8958009fd0 tx=0x7f8958009260 comp rx=0 tx=0).stop 2023-09-10T00:46:08.441 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f8975023700 1 -- 172.21.15.44:0/1267618274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89700ff440 msgr2=0x7f89700fb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:08.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f8975023700 1 --2- 172.21.15.44:0/1267618274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89700ff440 0x7f89700fb6d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f89600047b0 tx=0x7f896000bf50 comp rx=0 tx=0).stop 2023-09-10T00:46:08.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f8975023700 1 -- 172.21.15.44:0/1267618274 shutdown_connections 2023-09-10T00:46:08.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f8975023700 1 --2- 172.21.15.44:0/1267618274 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f895c0a07b0 0x7f895c0a2c70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f8975023700 1 --2- 172.21.15.44:0/1267618274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89701054e0 0x7f89701a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.385+0000 7f8975023700 1 --2- 172.21.15.44:0/1267618274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89700ff440 0x7f89700fb6d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.386+0000 7f8975023700 1 --2- 172.21.15.44:0/1267618274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89700ffe00 0x7f89700fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:08.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.386+0000 7f8975023700 1 -- 172.21.15.44:0/1267618274 >> 172.21.15.44:0/1267618274 conn(0x7f89700faf50 msgr2=0x7f89700678c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:08.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.386+0000 7f8975023700 1 -- 172.21.15.44:0/1267618274 shutdown_connections 2023-09-10T00:46:08.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:08.386+0000 7f8975023700 1 -- 172.21.15.44:0/1267618274 wait complete. 2023-09-10T00:46:08.445 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:46:08.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:08 smithi099 ceph-mon[102692]: pgmap v167: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 16 MiB/s wr, 71 op/s 2023-09-10T00:46:09.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:08 smithi131 ceph-mon[103713]: pgmap v167: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 16 MiB/s wr, 71 op/s 2023-09-10T00:46:09.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:08 smithi044 ceph-mon[98676]: pgmap v167: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 16 MiB/s wr, 71 op/s 2023-09-10T00:46:10.449 INFO:teuthology.orchestra.run.smithi044.stderr:27100000 of 50000000 tuples (54%) done (elapsed 310.08 s, remaining 262.03 s) 2023-09-10T00:46:10.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:10 smithi099 ceph-mon[102692]: pgmap v168: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 16 MiB/s wr, 71 op/s 2023-09-10T00:46:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:10 smithi131 ceph-mon[103713]: pgmap v168: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 16 MiB/s wr, 71 op/s 2023-09-10T00:46:11.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:10 smithi044 ceph-mon[98676]: pgmap v168: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 16 MiB/s wr, 71 op/s 2023-09-10T00:46:11.312 INFO:teuthology.orchestra.run.smithi044.stderr:27200000 of 50000000 tuples (54%) done (elapsed 310.95 s, remaining 260.65 s) 2023-09-10T00:46:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.459+0000 7f0b25719700 1 -- 172.21.15.44:0/1566037831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b20106650 msgr2=0x7f0b20106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.459+0000 7f0b25719700 1 --2- 172.21.15.44:0/1566037831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b20106650 0x7f0b20106a30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0b100099c0 tx=0x7f0b10009cd0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.459+0000 7f0b25719700 1 -- 172.21.15.44:0/1566037831 shutdown_connections 2023-09-10T00:46:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.459+0000 7f0b25719700 1 --2- 172.21.15.44:0/1566037831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b20100f50 0x7f0b2010d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.459+0000 7f0b25719700 1 --2- 172.21.15.44:0/1566037831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b20106650 0x7f0b20106a30 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.459+0000 7f0b25719700 1 --2- 172.21.15.44:0/1566037831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b20100590 0x7f0b20100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.459+0000 7f0b25719700 1 -- 172.21.15.44:0/1566037831 >> 172.21.15.44:0/1566037831 conn(0x7f0b200fc060 msgr2=0x7f0b200fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.459+0000 7f0b25719700 1 -- 172.21.15.44:0/1566037831 shutdown_connections 2023-09-10T00:46:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.459+0000 7f0b25719700 1 -- 172.21.15.44:0/1566037831 wait complete. 2023-09-10T00:46:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.460+0000 7f0b25719700 1 Processor -- start 2023-09-10T00:46:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.460+0000 7f0b25719700 1 -- start start 2023-09-10T00:46:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.460+0000 7f0b25719700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b20100590 0x7f0b2019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.460+0000 7f0b25719700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b20100f50 0x7f0b2019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b25719700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b20106650 0x7f0b201963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b25719700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b20196a10 con 0x7f0b20100590 2023-09-10T00:46:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b25719700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b20196b80 con 0x7f0b20106650 2023-09-10T00:46:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b25719700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b20196d40 con 0x7f0b20100f50 2023-09-10T00:46:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b1effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b20100590 0x7f0b2019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b1effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b20100590 0x7f0b2019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51130/0 (socket says 172.21.15.44:51130) 2023-09-10T00:46:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b1effd700 1 -- 172.21.15.44:0/705951025 learned_addr learned my addr 172.21.15.44:0/705951025 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b1e7fc700 1 --2- 172.21.15.44:0/705951025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b20100f50 0x7f0b2019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b1f7fe700 1 --2- 172.21.15.44:0/705951025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b20106650 0x7f0b201963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b1e7fc700 1 -- 172.21.15.44:0/705951025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b20106650 msgr2=0x7f0b201963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b1e7fc700 1 --2- 172.21.15.44:0/705951025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b20106650 0x7f0b201963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b1e7fc700 1 -- 172.21.15.44:0/705951025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b20100590 msgr2=0x7f0b2019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b1e7fc700 1 --2- 172.21.15.44:0/705951025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b20100590 0x7f0b2019c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b1e7fc700 1 -- 172.21.15.44:0/705951025 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b100095f0 con 0x7f0b20100f50 2023-09-10T00:46:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b1effd700 1 --2- 172.21.15.44:0/705951025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b20100590 0x7f0b2019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b1f7fe700 1 --2- 172.21.15.44:0/705951025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b20106650 0x7f0b201963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.461+0000 7f0b1e7fc700 1 --2- 172.21.15.44:0/705951025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b20100f50 0x7f0b2019ca80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400e9f0 tx=0x7f0b1400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.462+0000 7f0b07fff700 1 -- 172.21.15.44:0/705951025 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1400c9c0 con 0x7f0b20100f50 2023-09-10T00:46:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.462+0000 7f0b25719700 1 -- 172.21.15.44:0/705951025 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b20197020 con 0x7f0b20100f50 2023-09-10T00:46:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.462+0000 7f0b07fff700 1 -- 172.21.15.44:0/705951025 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b14010430 con 0x7f0b20100f50 2023-09-10T00:46:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.462+0000 7f0b07fff700 1 -- 172.21.15.44:0/705951025 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14018680 con 0x7f0b20100f50 2023-09-10T00:46:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.462+0000 7f0b25719700 1 -- 172.21.15.44:0/705951025 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b20072a10 con 0x7f0b20100f50 2023-09-10T00:46:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.464+0000 7f0b07fff700 1 -- 172.21.15.44:0/705951025 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f0b140105a0 con 0x7f0b20100f50 2023-09-10T00:46:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.465+0000 7f0b07fff700 1 --2- 172.21.15.44:0/705951025 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0b080a07b0 0x7f0b080a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.466+0000 7f0b07fff700 1 -- 172.21.15.44:0/705951025 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f0b14014070 con 0x7f0b20100f50 2023-09-10T00:46:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.466+0000 7f0b1effd700 1 --2- 172.21.15.44:0/705951025 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0b080a07b0 0x7f0b080a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.466+0000 7f0b1effd700 1 --2- 172.21.15.44:0/705951025 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0b080a07b0 0x7f0b080a2c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0b10000c00 tx=0x7f0b1001a040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.466+0000 7f0b318dfb80 1 Processor -- start 2023-09-10T00:46:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.466+0000 7f0b318dfb80 1 -- start start 2023-09-10T00:46:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b318dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56476976d1a0 0x56476976d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b318dfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564769767300 0x564769767780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b318dfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767cc0 0x564769773f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b318dfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647696c06d0 con 0x564769767cc0 2023-09-10T00:46:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b318dfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647696bdff0 con 0x564769767300 2023-09-10T00:46:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b318dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647696b98e0 con 0x56476976d1a0 2023-09-10T00:46:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b1effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56476976d1a0 0x56476976d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b1effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56476976d1a0 0x56476976d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58020/0 (socket says 172.21.15.44:58020) 2023-09-10T00:46:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b1effd700 1 -- 172.21.15.44:0/1343003803 learned_addr learned my addr 172.21.15.44:0/1343003803 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b1e7fc700 1 --2- 172.21.15.44:0/1343003803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564769767300 0x564769767780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b1f7fe700 1 --2- 172.21.15.44:0/1343003803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767cc0 0x564769773f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b1effd700 1 -- 172.21.15.44:0/1343003803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564769767300 msgr2=0x564769767780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b1effd700 1 --2- 172.21.15.44:0/1343003803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564769767300 0x564769767780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b1effd700 1 -- 172.21.15.44:0/1343003803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767cc0 msgr2=0x564769773f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b1effd700 1 --2- 172.21.15.44:0/1343003803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767cc0 0x564769773f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b1effd700 1 -- 172.21.15.44:0/1343003803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647694e5d80 con 0x56476976d1a0 2023-09-10T00:46:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.467+0000 7f0b1f7fe700 1 --2- 172.21.15.44:0/1343003803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767cc0 0x564769773f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.468+0000 7f0b1effd700 1 --2- 172.21.15.44:0/1343003803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56476976d1a0 0x56476976d580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0b10005090 tx=0x7f0b1001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28d9bbd03a0c6753 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.468+0000 7f0b057fa700 1 -- 172.21.15.44:0/1343003803 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10003760 con 0x56476976d1a0 2023-09-10T00:46:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.468+0000 7f0b057fa700 1 -- 172.21.15.44:0/1343003803 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b1001e040 con 0x56476976d1a0 2023-09-10T00:46:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.468+0000 7f0b057fa700 1 -- 172.21.15.44:0/1343003803 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1000fb40 con 0x56476976d1a0 2023-09-10T00:46:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.469+0000 7f0b318dfb80 1 -- 172.21.15.44:0/1343003803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56476976d1a0 msgr2=0x56476976d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.469+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/1343003803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56476976d1a0 0x56476976d580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0b10005090 tx=0x7f0b1001f000 comp rx=0 tx=0).stop 2023-09-10T00:46:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.469+0000 7f0b318dfb80 1 -- 172.21.15.44:0/1343003803 shutdown_connections 2023-09-10T00:46:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.469+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/1343003803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767cc0 0x564769773f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.469+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/1343003803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56476976d1a0 0x56476976d580 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.469+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/1343003803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564769767300 0x564769767780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.469+0000 7f0b318dfb80 1 -- 172.21.15.44:0/1343003803 >> 172.21.15.44:0/1343003803 conn(0x5647696dc030 msgr2=0x564769762560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.469+0000 7f0b318dfb80 1 -- 172.21.15.44:0/1343003803 shutdown_connections 2023-09-10T00:46:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.469+0000 7f0b318dfb80 1 -- 172.21.15.44:0/1343003803 wait complete. 2023-09-10T00:46:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.469+0000 7f0b318dfb80 1 -- start start 2023-09-10T00:46:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.469+0000 7f0b318dfb80 1 Processor -- start 2023-09-10T00:46:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b318dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564769767300 0x564769773e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b318dfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564769767cc0 0x564769771060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b318dfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56476976d1a0 0x564769771650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b318dfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564769657b10 con 0x56476976d1a0 2023-09-10T00:46:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b318dfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564769657810 con 0x564769767cc0 2023-09-10T00:46:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b318dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56476953b8c0 con 0x564769767300 2023-09-10T00:46:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b318dfb80 10 client.0 _ll_register_callbacks cb 0x5647695a1a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:46:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b1effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564769767300 0x564769773e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b1effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564769767300 0x564769773e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58026/0 (socket says 172.21.15.44:58026) 2023-09-10T00:46:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b1effd700 1 -- 172.21.15.44:0/1640160591 learned_addr learned my addr 172.21.15.44:0/1640160591 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b1e7fc700 1 --2- 172.21.15.44:0/1640160591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564769767cc0 0x564769771060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b1f7fe700 1 --2- 172.21.15.44:0/1640160591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56476976d1a0 0x564769771650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b1effd700 1 -- 172.21.15.44:0/1640160591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564769767cc0 msgr2=0x564769771060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b1effd700 1 --2- 172.21.15.44:0/1640160591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564769767cc0 0x564769771060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b1effd700 1 -- 172.21.15.44:0/1640160591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56476976d1a0 msgr2=0x564769771650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b1effd700 1 --2- 172.21.15.44:0/1640160591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56476976d1a0 0x564769771650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b1effd700 1 -- 172.21.15.44:0/1640160591 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b18003a80 con 0x564769767300 2023-09-10T00:46:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b1f7fe700 1 --2- 172.21.15.44:0/1640160591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56476976d1a0 0x564769771650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:46:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b1effd700 1 --2- 172.21.15.44:0/1640160591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564769767300 0x564769773e50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000f9f0 tx=0x7f0b1000fa20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfe2865e2240a8f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.470+0000 7f0b02ff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:46:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.471+0000 7f0b02ff5700 1 -- 172.21.15.44:0/1640160591 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10003970 con 0x564769767300 2023-09-10T00:46:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.471+0000 7f0b02ff5700 1 -- 172.21.15.44:0/1640160591 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f0b10003ad0 con 0x564769767300 2023-09-10T00:46:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.471+0000 7f0b318dfb80 1 -- 172.21.15.44:0/1640160591 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564769611490 con 0x564769767300 2023-09-10T00:46:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.471+0000 7f0b02ff5700 1 -- 172.21.15.44:0/1640160591 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b10003c90 con 0x564769767300 2023-09-10T00:46:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.471+0000 7f0b318dfb80 10 client.26401 fetch_fsmap learned FSMap version 42 2023-09-10T00:46:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.471+0000 7f0b318dfb80 1 -- 172.21.15.44:0/1640160591 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f0b2010dd10 con 0x564769767300 2023-09-10T00:46:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.471+0000 7f0b02ff5700 1 -- 172.21.15.44:0/1640160591 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564769611490 con 0x564769767300 2023-09-10T00:46:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.471+0000 7f0b02ff5700 1 -- 172.21.15.44:0/1640160591 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0b10024440 con 0x564769767300 2023-09-10T00:46:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.471+0000 7f0b318dfb80 10 client.26401 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:46:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.471+0000 7f0b318dfb80 10 client.26401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:46:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.471+0000 7f0b318dfb80 20 client.26401 populate_metadata read hostname 'smithi044' 2023-09-10T00:46:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.471+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/1640160591 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564769780ef0 0x564769783310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.471+0000 7f0b318dfb80 4 client.26401 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:46:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.471+0000 7f0b318dfb80 1 -- 172.21.15.44:0/1640160591 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647696ce840 con 0x564769780ef0 2023-09-10T00:46:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.471+0000 7f0b1e7fc700 1 --2- 172.21.15.44:0/1640160591 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564769780ef0 0x564769783310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.472+0000 7f0b02ff5700 10 client.26401 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.472+0000 7f0b1e7fc700 1 --2- 172.21.15.44:0/1640160591 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564769780ef0 0x564769783310 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b745d43bb5ccd687 server_cookie=eb8bf87cb6a4611 in_seq=0 out_seq=0 2023-09-10T00:46:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.473+0000 7f0b02ff5700 1 -- 172.21.15.44:0/1640160591 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5647696ce840 con 0x564769780ef0 2023-09-10T00:46:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.473+0000 7f0b02ff5700 10 client.26401 handle_command_reply: tid=0 2023-09-10T00:46:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.473+0000 7f0b318dfb80 1 client.26401 shutdown 2023-09-10T00:46:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.473+0000 7f0b318dfb80 1 -- 172.21.15.44:0/1640160591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564769767300 msgr2=0x564769773e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.473+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/1640160591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564769767300 0x564769773e50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000f9f0 tx=0x7f0b1000fa20 comp rx=0 tx=0).stop 2023-09-10T00:46:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.473+0000 7f0b318dfb80 1 -- 172.21.15.44:0/1640160591 shutdown_connections 2023-09-10T00:46:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.473+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/1640160591 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564769780ef0 0x564769783310 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.473+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/1640160591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56476976d1a0 0x564769771650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.473+0000 7f0b02ff5700 0 client.26401 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.473+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/1640160591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564769767300 0x564769773e50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.473+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/1640160591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564769767cc0 0x564769771060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.473+0000 7f0b318dfb80 1 -- 172.21.15.44:0/1640160591 >> 172.21.15.44:0/1640160591 conn(0x5647696dc030 msgr2=0x5647697617a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.473+0000 7f0b318dfb80 1 -- 172.21.15.44:0/1640160591 shutdown_connections 2023-09-10T00:46:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.473+0000 7f0b318dfb80 1 -- 172.21.15.44:0/1640160591 wait complete. 2023-09-10T00:46:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.473+0000 7f0b318dfb80 20 client.26401 trim_cache size 0 max 16384 2023-09-10T00:46:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.491+0000 7f0b318dfb80 1 Processor -- start 2023-09-10T00:46:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.491+0000 7f0b318dfb80 1 -- start start 2023-09-10T00:46:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b318dfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767300 0x564769773980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b318dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564769767cc0 0x56476976c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b318dfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56476976d1a0 0x56476976c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b318dfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56476976bc40 con 0x564769767300 2023-09-10T00:46:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b318dfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564769771c50 con 0x56476976d1a0 2023-09-10T00:46:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b318dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564769772050 con 0x564769767cc0 2023-09-10T00:46:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b1e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564769767cc0 0x56476976c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b1effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767300 0x564769773980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b1e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564769767cc0 0x56476976c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58036/0 (socket says 172.21.15.44:58036) 2023-09-10T00:46:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b1effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767300 0x564769773980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51152/0 (socket says 172.21.15.44:51152) 2023-09-10T00:46:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b1e7fc700 1 -- 172.21.15.44:0/3226578240 learned_addr learned my addr 172.21.15.44:0/3226578240 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b1f7fe700 1 --2- 172.21.15.44:0/3226578240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56476976d1a0 0x56476976c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b1e7fc700 1 -- 172.21.15.44:0/3226578240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56476976d1a0 msgr2=0x56476976c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b1e7fc700 1 --2- 172.21.15.44:0/3226578240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56476976d1a0 0x56476976c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b1e7fc700 1 -- 172.21.15.44:0/3226578240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767300 msgr2=0x564769773980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b1e7fc700 1 --2- 172.21.15.44:0/3226578240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767300 0x564769773980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b1e7fc700 1 -- 172.21.15.44:0/3226578240 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647694cec00 con 0x564769767cc0 2023-09-10T00:46:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b1e7fc700 1 --2- 172.21.15.44:0/3226578240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564769767cc0 0x56476976c050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0b14095fe0 tx=0x7f0b140cb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2569b357675405d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.492+0000 7f0b05ffb700 1 -- 172.21.15.44:0/3226578240 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14010880 con 0x564769767cc0 2023-09-10T00:46:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.493+0000 7f0b05ffb700 1 -- 172.21.15.44:0/3226578240 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b140ace90 con 0x564769767cc0 2023-09-10T00:46:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.493+0000 7f0b05ffb700 1 -- 172.21.15.44:0/3226578240 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b140a1670 con 0x564769767cc0 2023-09-10T00:46:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.493+0000 7f0b318dfb80 1 -- 172.21.15.44:0/3226578240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564769767cc0 msgr2=0x56476976c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.493+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/3226578240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564769767cc0 0x56476976c050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0b14095fe0 tx=0x7f0b140cb040 comp rx=0 tx=0).stop 2023-09-10T00:46:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.493+0000 7f0b318dfb80 1 -- 172.21.15.44:0/3226578240 shutdown_connections 2023-09-10T00:46:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.493+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/3226578240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56476976d1a0 0x56476976c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.493+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/3226578240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767300 0x564769773980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.493+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/3226578240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564769767cc0 0x56476976c050 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.493+0000 7f0b318dfb80 1 -- 172.21.15.44:0/3226578240 >> 172.21.15.44:0/3226578240 conn(0x5647696dc030 msgr2=0x56476976b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.493+0000 7f0b318dfb80 1 -- 172.21.15.44:0/3226578240 shutdown_connections 2023-09-10T00:46:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.494+0000 7f0b318dfb80 1 -- 172.21.15.44:0/3226578240 wait complete. 2023-09-10T00:46:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.494+0000 7f0b318dfb80 1 -- start start 2023-09-10T00:46:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.494+0000 7f0b318dfb80 1 Processor -- start 2023-09-10T00:46:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.494+0000 7f0b318dfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564769767300 0x5647697cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.494+0000 7f0b318dfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767cc0 0x5647697ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.494+0000 7f0b318dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56476976d1a0 0x564769776440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.494+0000 7f0b318dfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564769776a10 con 0x564769767cc0 2023-09-10T00:46:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.494+0000 7f0b318dfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564769776b80 con 0x564769767300 2023-09-10T00:46:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.494+0000 7f0b318dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564769776d40 con 0x56476976d1a0 2023-09-10T00:46:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.494+0000 7f0b318dfb80 10 client.0 _ll_register_callbacks cb 0x7f0b18009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:46:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.494+0000 7f0b1e7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767cc0 0x5647697ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.494+0000 7f0b1effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564769767300 0x5647697cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.494+0000 7f0b1e7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767cc0 0x5647697ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51160/0 (socket says 172.21.15.44:51160) 2023-09-10T00:46:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.494+0000 7f0b1e7fc700 1 -- 172.21.15.44:0/2852260903 learned_addr learned my addr 172.21.15.44:0/2852260903 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.494+0000 7f0b1f7fe700 1 --2- 172.21.15.44:0/2852260903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56476976d1a0 0x564769776440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.495+0000 7f0b1e7fc700 1 -- 172.21.15.44:0/2852260903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56476976d1a0 msgr2=0x564769776440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.495+0000 7f0b1e7fc700 1 --2- 172.21.15.44:0/2852260903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56476976d1a0 0x564769776440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.495+0000 7f0b1e7fc700 1 -- 172.21.15.44:0/2852260903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564769767300 msgr2=0x5647697cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.495+0000 7f0b1e7fc700 1 --2- 172.21.15.44:0/2852260903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564769767300 0x5647697cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.495+0000 7f0b1e7fc700 1 -- 172.21.15.44:0/2852260903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b20072a10 con 0x564769767cc0 2023-09-10T00:46:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.495+0000 7f0b1e7fc700 1 --2- 172.21.15.44:0/2852260903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767cc0 0x5647697ce520 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0b14000c00 tx=0x7f0b140109a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598e2cdf2410b991 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.495+0000 7f0b02ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:46:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.495+0000 7f0b02ff5700 1 -- 172.21.15.44:0/2852260903 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1409d170 con 0x564769767cc0 2023-09-10T00:46:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.495+0000 7f0b02ff5700 1 -- 172.21.15.44:0/2852260903 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f0b14099c70 con 0x564769767cc0 2023-09-10T00:46:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.495+0000 7f0b318dfb80 1 -- 172.21.15.44:0/2852260903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b1001e040 con 0x564769767cc0 2023-09-10T00:46:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.495+0000 7f0b02ff5700 1 -- 172.21.15.44:0/2852260903 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b140652d0 con 0x564769767cc0 2023-09-10T00:46:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.495+0000 7f0b318dfb80 10 client.16800 fetch_fsmap learned FSMap version 42 2023-09-10T00:46:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.495+0000 7f0b318dfb80 1 -- 172.21.15.44:0/2852260903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x56476950c1f0 con 0x564769767cc0 2023-09-10T00:46:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.495+0000 7f0b02ff5700 1 -- 172.21.15.44:0/2852260903 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b1001e040 con 0x564769767cc0 2023-09-10T00:46:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.496+0000 7f0b02ff5700 1 -- 172.21.15.44:0/2852260903 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0b140997d0 con 0x564769767cc0 2023-09-10T00:46:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.496+0000 7f0b318dfb80 10 client.16800 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:46:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.496+0000 7f0b318dfb80 10 client.16800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:46:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.496+0000 7f0b318dfb80 20 client.16800 populate_metadata read hostname 'smithi044' 2023-09-10T00:46:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.496+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/2852260903 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5647697d5a80 0x5647697d7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.496+0000 7f0b318dfb80 4 client.16800 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:46:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.496+0000 7f0b318dfb80 1 -- 172.21.15.44:0/2852260903 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5647696b28c0 con 0x5647697d5a80 2023-09-10T00:46:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.496+0000 7f0b1effd700 1 --2- 172.21.15.44:0/2852260903 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5647697d5a80 0x5647697d7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.496+0000 7f0b02ff5700 10 client.16800 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.496+0000 7f0b1effd700 1 --2- 172.21.15.44:0/2852260903 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5647697d5a80 0x5647697d7ea0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c341c5bd193a52d server_cookie=80fd4059e124b639 in_seq=0 out_seq=0 2023-09-10T00:46:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.508+0000 7f0b02ff5700 1 -- 172.21.15.44:0/2852260903 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5647696b28c0 con 0x5647697d5a80 2023-09-10T00:46:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.508+0000 7f0b02ff5700 10 client.16800 handle_command_reply: tid=0 2023-09-10T00:46:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.508+0000 7f0b318dfb80 1 client.16800 shutdown 2023-09-10T00:46:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.508+0000 7f0b318dfb80 1 -- 172.21.15.44:0/2852260903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767cc0 msgr2=0x5647697ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.508+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/2852260903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767cc0 0x5647697ce520 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0b14000c00 tx=0x7f0b140109a0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.508+0000 7f0b318dfb80 1 -- 172.21.15.44:0/2852260903 shutdown_connections 2023-09-10T00:46:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.508+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/2852260903 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5647697d5a80 0x5647697d7ea0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.508+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/2852260903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56476976d1a0 0x564769776440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.508+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/2852260903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564769767300 0x5647697cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.508+0000 7f0b318dfb80 1 --2- 172.21.15.44:0/2852260903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564769767cc0 0x5647697ce520 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.508+0000 7f0b318dfb80 1 -- 172.21.15.44:0/2852260903 >> 172.21.15.44:0/2852260903 conn(0x5647696dc030 msgr2=0x56476976b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.508+0000 7f0b02ff5700 0 client.16800 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.508+0000 7f0b318dfb80 1 -- 172.21.15.44:0/2852260903 shutdown_connections 2023-09-10T00:46:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.508+0000 7f0b318dfb80 1 -- 172.21.15.44:0/2852260903 wait complete. 2023-09-10T00:46:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.508+0000 7f0b318dfb80 20 client.16800 trim_cache size 0 max 16384 2023-09-10T00:46:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:46:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:46:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:46:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:46:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.509+0000 7f0b25719700 1 -- 172.21.15.44:0/705951025 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0b080a07b0 msgr2=0x7f0b080a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.509+0000 7f0b25719700 1 --2- 172.21.15.44:0/705951025 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0b080a07b0 0x7f0b080a2c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0b10000c00 tx=0x7f0b1001a040 comp rx=0 tx=0).stop 2023-09-10T00:46:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.509+0000 7f0b25719700 1 -- 172.21.15.44:0/705951025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b20100f50 msgr2=0x7f0b2019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.509+0000 7f0b25719700 1 --2- 172.21.15.44:0/705951025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b20100f50 0x7f0b2019ca80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400e9f0 tx=0x7f0b1400ed00 comp rx=0 tx=0).stop 2023-09-10T00:46:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.509+0000 7f0b25719700 1 -- 172.21.15.44:0/705951025 shutdown_connections 2023-09-10T00:46:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.509+0000 7f0b25719700 1 --2- 172.21.15.44:0/705951025 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0b080a07b0 0x7f0b080a2c70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.509+0000 7f0b25719700 1 --2- 172.21.15.44:0/705951025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b20106650 0x7f0b201963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.509+0000 7f0b25719700 1 --2- 172.21.15.44:0/705951025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b20100590 0x7f0b2019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.509+0000 7f0b25719700 1 --2- 172.21.15.44:0/705951025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b20100f50 0x7f0b2019ca80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.509+0000 7f0b25719700 1 -- 172.21.15.44:0/705951025 >> 172.21.15.44:0/705951025 conn(0x7f0b200fc060 msgr2=0x7f0b20104290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.509+0000 7f0b25719700 1 -- 172.21.15.44:0/705951025 shutdown_connections 2023-09-10T00:46:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:11.509+0000 7f0b25719700 1 -- 172.21.15.44:0/705951025 wait complete. 2023-09-10T00:46:12.363 INFO:teuthology.orchestra.run.smithi044.stderr:27300000 of 50000000 tuples (54%) done (elapsed 312.00 s, remaining 259.43 s) 2023-09-10T00:46:12.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:12 smithi099 ceph-mon[102692]: pgmap v169: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 25 MiB/s wr, 105 op/s 2023-09-10T00:46:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:12 smithi131 ceph-mon[103713]: pgmap v169: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 25 MiB/s wr, 105 op/s 2023-09-10T00:46:13.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:12 smithi044 ceph-mon[98676]: pgmap v169: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 25 MiB/s wr, 105 op/s 2023-09-10T00:46:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:27400000 of 50000000 tuples (54%) done (elapsed 312.98 s, remaining 258.15 s) 2023-09-10T00:46:14.266 INFO:teuthology.orchestra.run.smithi044.stderr:27500000 of 50000000 tuples (55%) done (elapsed 313.90 s, remaining 256.83 s) 2023-09-10T00:46:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:14 smithi131 ceph-mon[103713]: pgmap v170: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 21 MiB/s wr, 91 op/s 2023-09-10T00:46:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:14 smithi044 ceph-mon[98676]: pgmap v170: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 21 MiB/s wr, 91 op/s 2023-09-10T00:46:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:14 smithi099 ceph-mon[102692]: pgmap v170: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 21 MiB/s wr, 91 op/s 2023-09-10T00:46:15.274 INFO:teuthology.orchestra.run.smithi044.stderr:27600000 of 50000000 tuples (55%) done (elapsed 314.91 s, remaining 255.58 s) 2023-09-10T00:46:16.222 INFO:teuthology.orchestra.run.smithi044.stderr:27700000 of 50000000 tuples (55%) done (elapsed 315.86 s, remaining 254.28 s) 2023-09-10T00:46:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:16 smithi131 ceph-mon[103713]: pgmap v171: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 37 MiB/s wr, 140 op/s 2023-09-10T00:46:17.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:16 smithi044 ceph-mon[98676]: pgmap v171: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 37 MiB/s wr, 140 op/s 2023-09-10T00:46:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:16 smithi099 ceph-mon[102692]: pgmap v171: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 37 MiB/s wr, 140 op/s 2023-09-10T00:46:17.214 INFO:teuthology.orchestra.run.smithi044.stderr:27800000 of 50000000 tuples (55%) done (elapsed 316.85 s, remaining 253.02 s) 2023-09-10T00:46:18.068 INFO:teuthology.orchestra.run.smithi044.stderr:27900000 of 50000000 tuples (55%) done (elapsed 317.70 s, remaining 251.66 s) 2023-09-10T00:46:19.020 INFO:teuthology.orchestra.run.smithi044.stderr:28000000 of 50000000 tuples (56%) done (elapsed 318.65 s, remaining 250.37 s) 2023-09-10T00:46:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:18 smithi131 ceph-mon[103713]: pgmap v172: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 25 MiB/s wr, 81 op/s 2023-09-10T00:46:19.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:18 smithi044 ceph-mon[98676]: pgmap v172: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 25 MiB/s wr, 81 op/s 2023-09-10T00:46:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:18 smithi099 ceph-mon[102692]: pgmap v172: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 25 MiB/s wr, 81 op/s 2023-09-10T00:46:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:28100000 of 50000000 tuples (56%) done (elapsed 319.59 s, remaining 249.07 s) 2023-09-10T00:46:21.011 INFO:teuthology.orchestra.run.smithi044.stderr:28200000 of 50000000 tuples (56%) done (elapsed 320.64 s, remaining 247.87 s) 2023-09-10T00:46:21.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:20 smithi131 ceph-mon[103713]: pgmap v173: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 25 MiB/s wr, 81 op/s 2023-09-10T00:46:21.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:20 smithi044 ceph-mon[98676]: pgmap v173: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 25 MiB/s wr, 81 op/s 2023-09-10T00:46:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:20 smithi099 ceph-mon[102692]: pgmap v173: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 25 MiB/s wr, 81 op/s 2023-09-10T00:46:21.889 INFO:teuthology.orchestra.run.smithi044.stderr:28300000 of 50000000 tuples (56%) done (elapsed 321.52 s, remaining 246.54 s) 2023-09-10T00:46:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:28400000 of 50000000 tuples (56%) done (elapsed 322.39 s, remaining 245.20 s) 2023-09-10T00:46:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:22 smithi131 ceph-mon[103713]: pgmap v174: 129 pgs: 129 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 41 MiB/s wr, 122 op/s 2023-09-10T00:46:23.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:22 smithi044 ceph-mon[98676]: pgmap v174: 129 pgs: 129 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 41 MiB/s wr, 122 op/s 2023-09-10T00:46:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:22 smithi099 ceph-mon[102692]: pgmap v174: 129 pgs: 129 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 41 MiB/s wr, 122 op/s 2023-09-10T00:46:23.787 INFO:teuthology.orchestra.run.smithi044.stderr:28500000 of 50000000 tuples (57%) done (elapsed 323.42 s, remaining 243.98 s) 2023-09-10T00:46:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.640+0000 7f44c5bac700 1 -- 172.21.15.44:0/2102520636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44c0106680 msgr2=0x7f44c0106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.640+0000 7f44c5bac700 1 --2- 172.21.15.44:0/2102520636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44c0106680 0x7f44c0106a60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f44b00098f0 tx=0x7f44b0009c00 comp rx=0 tx=0).stop 2023-09-10T00:46:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.640+0000 7f44c5bac700 1 -- 172.21.15.44:0/2102520636 shutdown_connections 2023-09-10T00:46:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.640+0000 7f44c5bac700 1 --2- 172.21.15.44:0/2102520636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c0100f80 0x7f44c010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.640+0000 7f44c5bac700 1 --2- 172.21.15.44:0/2102520636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44c0106680 0x7f44c0106a60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.640+0000 7f44c5bac700 1 --2- 172.21.15.44:0/2102520636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44c01005c0 0x7f44c0100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.640+0000 7f44c5bac700 1 -- 172.21.15.44:0/2102520636 >> 172.21.15.44:0/2102520636 conn(0x7f44c00fc0b0 msgr2=0x7f44c00fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.640+0000 7f44c5bac700 1 -- 172.21.15.44:0/2102520636 shutdown_connections 2023-09-10T00:46:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.640+0000 7f44c5bac700 1 -- 172.21.15.44:0/2102520636 wait complete. 2023-09-10T00:46:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.641+0000 7f44c5bac700 1 Processor -- start 2023-09-10T00:46:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.641+0000 7f44c5bac700 1 -- start start 2023-09-10T00:46:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.641+0000 7f44c5bac700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44c01005c0 0x7f44c019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.641+0000 7f44c5bac700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c0100f80 0x7f44c019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.641+0000 7f44c5bac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44c0106680 0x7f44c01963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.641+0000 7f44c5bac700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c0196a60 con 0x7f44c0100f80 2023-09-10T00:46:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.641+0000 7f44c5bac700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c0196bd0 con 0x7f44c0106680 2023-09-10T00:46:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.641+0000 7f44c5bac700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c0196d90 con 0x7f44c01005c0 2023-09-10T00:46:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.641+0000 7f44beffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c0100f80 0x7f44c019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.641+0000 7f44bf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44c01005c0 0x7f44c019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.641+0000 7f44beffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c0100f80 0x7f44c019cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51174/0 (socket says 172.21.15.44:51174) 2023-09-10T00:46:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.641+0000 7f44beffd700 1 -- 172.21.15.44:0/638337207 learned_addr learned my addr 172.21.15.44:0/638337207 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.641+0000 7f44bffff700 1 --2- 172.21.15.44:0/638337207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44c0106680 0x7f44c01963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.641+0000 7f44bf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44c01005c0 0x7f44c019c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58054/0 (socket says 172.21.15.44:58054) 2023-09-10T00:46:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.642+0000 7f44bffff700 1 -- 172.21.15.44:0/638337207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44c01005c0 msgr2=0x7f44c019c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.642+0000 7f44bffff700 1 --2- 172.21.15.44:0/638337207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44c01005c0 0x7f44c019c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.642+0000 7f44bffff700 1 -- 172.21.15.44:0/638337207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c0100f80 msgr2=0x7f44c019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.642+0000 7f44bffff700 1 --2- 172.21.15.44:0/638337207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c0100f80 0x7f44c019cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.642+0000 7f44bffff700 1 -- 172.21.15.44:0/638337207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b0009520 con 0x7f44c0106680 2023-09-10T00:46:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.642+0000 7f44bf7fe700 1 --2- 172.21.15.44:0/638337207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44c01005c0 0x7f44c019c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:46:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.642+0000 7f44beffd700 1 --2- 172.21.15.44:0/638337207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c0100f80 0x7f44c019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.642+0000 7f44bffff700 1 --2- 172.21.15.44:0/638337207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44c0106680 0x7f44c01963f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f44b800efc0 tx=0x7f44b800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.642+0000 7f44bcff9700 1 -- 172.21.15.44:0/638337207 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8018070 con 0x7f44c0106680 2023-09-10T00:46:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.642+0000 7f44c5bac700 1 -- 172.21.15.44:0/638337207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c0197010 con 0x7f44c0106680 2023-09-10T00:46:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.642+0000 7f44bcff9700 1 -- 172.21.15.44:0/638337207 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f44b800f040 con 0x7f44c0106680 2023-09-10T00:46:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.642+0000 7f44c5bac700 1 -- 172.21.15.44:0/638337207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c01a3580 con 0x7f44c0106680 2023-09-10T00:46:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.642+0000 7f44bcff9700 1 -- 172.21.15.44:0/638337207 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b800cc50 con 0x7f44c0106680 2023-09-10T00:46:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.645+0000 7f44bcff9700 1 -- 172.21.15.44:0/638337207 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f44b8003a00 con 0x7f44c0106680 2023-09-10T00:46:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.645+0000 7f44bcff9700 1 --2- 172.21.15.44:0/638337207 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44a80a07b0 0x7f44a80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.646+0000 7f44bf7fe700 1 --2- 172.21.15.44:0/638337207 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44a80a07b0 0x7f44a80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.646+0000 7f44bcff9700 1 -- 172.21.15.44:0/638337207 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f44b80cbe60 con 0x7f44c0106680 2023-09-10T00:46:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.646+0000 7f44bf7fe700 1 --2- 172.21.15.44:0/638337207 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44a80a07b0 0x7f44a80a2c70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f44b00098c0 tx=0x7f44b0019040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.646+0000 7f44d1d72b80 1 Processor -- start 2023-09-10T00:46:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.646+0000 7f44d1d72b80 1 -- start start 2023-09-10T00:46:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44d1d72b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e515b0 0x55ff84e51990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44d1d72b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e4bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44d1d72b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4c0d0 0x55ff84e58340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44d1d72b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff84da06d0 con 0x55ff84e4c0d0 2023-09-10T00:46:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44d1d72b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff84d9dff0 con 0x55ff84e515b0 2023-09-10T00:46:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44d1d72b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff84d998e0 con 0x55ff84e4b710 2023-09-10T00:46:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44beffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e4bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44bf7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e515b0 0x55ff84e51990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44bffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4c0d0 0x55ff84e58340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44beffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e4bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58064/0 (socket says 172.21.15.44:58064) 2023-09-10T00:46:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44bf7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e515b0 0x55ff84e51990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52920/0 (socket says 172.21.15.44:52920) 2023-09-10T00:46:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44beffd700 1 -- 172.21.15.44:0/3263025655 learned_addr learned my addr 172.21.15.44:0/3263025655 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44beffd700 1 -- 172.21.15.44:0/3263025655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e515b0 msgr2=0x55ff84e51990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44beffd700 1 --2- 172.21.15.44:0/3263025655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e515b0 0x55ff84e51990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44beffd700 1 -- 172.21.15.44:0/3263025655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4c0d0 msgr2=0x55ff84e58340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44beffd700 1 --2- 172.21.15.44:0/3263025655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4c0d0 0x55ff84e58340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44beffd700 1 -- 172.21.15.44:0/3263025655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff84bc5d80 con 0x55ff84e4b710 2023-09-10T00:46:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44bffff700 1 --2- 172.21.15.44:0/3263025655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4c0d0 0x55ff84e58340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.647+0000 7f44beffd700 1 --2- 172.21.15.44:0/3263025655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e4bb90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f44b4011680 tx=0x7f44b4011990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19ca1d2661577cbc server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.648+0000 7f44a5ffb700 1 -- 172.21.15.44:0/3263025655 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4004030 con 0x55ff84e4b710 2023-09-10T00:46:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.648+0000 7f44a5ffb700 1 -- 172.21.15.44:0/3263025655 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f44b4003680 con 0x55ff84e4b710 2023-09-10T00:46:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.648+0000 7f44a5ffb700 1 -- 172.21.15.44:0/3263025655 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b40196b0 con 0x55ff84e4b710 2023-09-10T00:46:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.648+0000 7f44d1d72b80 1 -- 172.21.15.44:0/3263025655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 msgr2=0x55ff84e4bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.648+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/3263025655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e4bb90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f44b4011680 tx=0x7f44b4011990 comp rx=0 tx=0).stop 2023-09-10T00:46:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.648+0000 7f44d1d72b80 1 -- 172.21.15.44:0/3263025655 shutdown_connections 2023-09-10T00:46:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.648+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/3263025655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4c0d0 0x55ff84e58340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.648+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/3263025655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e515b0 0x55ff84e51990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.648+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/3263025655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e4bb90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.648+0000 7f44d1d72b80 1 -- 172.21.15.44:0/3263025655 >> 172.21.15.44:0/3263025655 conn(0x55ff84dbc030 msgr2=0x55ff84e42560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.648+0000 7f44d1d72b80 1 -- 172.21.15.44:0/3263025655 shutdown_connections 2023-09-10T00:46:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.648+0000 7f44d1d72b80 1 -- 172.21.15.44:0/3263025655 wait complete. 2023-09-10T00:46:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.649+0000 7f44d1d72b80 1 -- start start 2023-09-10T00:46:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.649+0000 7f44d1d72b80 1 Processor -- start 2023-09-10T00:46:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.649+0000 7f44d1d72b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e58200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.649+0000 7f44d1d72b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e4c0d0 0x55ff84e554b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.649+0000 7f44d1d72b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e515b0 0x55ff84e55a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.649+0000 7f44d1d72b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff84d37b10 con 0x55ff84e515b0 2023-09-10T00:46:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.649+0000 7f44d1d72b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff84d37810 con 0x55ff84e4c0d0 2023-09-10T00:46:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.649+0000 7f44d1d72b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff84c1b8c0 con 0x55ff84e4b710 2023-09-10T00:46:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.649+0000 7f44d1d72b80 10 client.0 _ll_register_callbacks cb 0x55ff84c81a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:46:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f44bf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e58200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f44beffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e4c0d0 0x55ff84e554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f44bffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e515b0 0x55ff84e55a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f44bf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e58200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58068/0 (socket says 172.21.15.44:58068) 2023-09-10T00:46:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f44bf7fe700 1 -- 172.21.15.44:0/2335811700 learned_addr learned my addr 172.21.15.44:0/2335811700 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f44beffd700 1 -- 172.21.15.44:0/2335811700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 msgr2=0x55ff84e58200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f44beffd700 1 --2- 172.21.15.44:0/2335811700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e58200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f44beffd700 1 -- 172.21.15.44:0/2335811700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e515b0 msgr2=0x55ff84e55a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f44beffd700 1 --2- 172.21.15.44:0/2335811700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e515b0 0x55ff84e55a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f44beffd700 1 -- 172.21.15.44:0/2335811700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c01a3580 con 0x55ff84e4c0d0 2023-09-10T00:46:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f44bf7fe700 1 --2- 172.21.15.44:0/2335811700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e58200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f44bffff700 1 --2- 172.21.15.44:0/2335811700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e515b0 0x55ff84e55a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f44beffd700 1 --2- 172.21.15.44:0/2335811700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e4c0d0 0x55ff84e554b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f44b4000c00 tx=0x7f44b40041a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b499462ca355453 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f449effd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:46:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f449effd700 1 -- 172.21.15.44:0/2335811700 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4003dd0 con 0x55ff84e4c0d0 2023-09-10T00:46:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f449effd700 1 -- 172.21.15.44:0/2335811700 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f44b4005ce0 con 0x55ff84e4c0d0 2023-09-10T00:46:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.650+0000 7f44d1d72b80 1 -- 172.21.15.44:0/2335811700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff84cf1490 con 0x55ff84e4c0d0 2023-09-10T00:46:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.651+0000 7f449effd700 1 -- 172.21.15.44:0/2335811700 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44b40045a0 con 0x55ff84e4c0d0 2023-09-10T00:46:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.651+0000 7f44d1d72b80 10 client.26477 fetch_fsmap learned FSMap version 42 2023-09-10T00:46:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.651+0000 7f44d1d72b80 1 -- 172.21.15.44:0/2335811700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f44b4009e10 con 0x55ff84e4c0d0 2023-09-10T00:46:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.651+0000 7f449effd700 1 -- 172.21.15.44:0/2335811700 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ff84cf1490 con 0x55ff84e4c0d0 2023-09-10T00:46:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.651+0000 7f449effd700 1 -- 172.21.15.44:0/2335811700 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f44b4004c60 con 0x55ff84e4c0d0 2023-09-10T00:46:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.651+0000 7f44d1d72b80 10 client.26477 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:46:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.651+0000 7f44d1d72b80 10 client.26477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:46:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.651+0000 7f44d1d72b80 20 client.26477 populate_metadata read hostname 'smithi044' 2023-09-10T00:46:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.651+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/2335811700 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff84e656d0 0x55ff84e67ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.651+0000 7f44d1d72b80 4 client.26477 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:46:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.651+0000 7f44d1d72b80 1 -- 172.21.15.44:0/2335811700 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff84dae840 con 0x55ff84e656d0 2023-09-10T00:46:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.651+0000 7f44bf7fe700 1 --2- 172.21.15.44:0/2335811700 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff84e656d0 0x55ff84e67ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.652+0000 7f449effd700 10 client.26477 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.652+0000 7f44bf7fe700 1 --2- 172.21.15.44:0/2335811700 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff84e656d0 0x55ff84e67ac0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52113d0d8b3ae92e server_cookie=5422ce4921a05900 in_seq=0 out_seq=0 2023-09-10T00:46:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.652+0000 7f449effd700 1 -- 172.21.15.44:0/2335811700 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ff84dae840 con 0x55ff84e656d0 2023-09-10T00:46:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.652+0000 7f449effd700 10 client.26477 handle_command_reply: tid=0 2023-09-10T00:46:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.652+0000 7f44d1d72b80 1 client.26477 shutdown 2023-09-10T00:46:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.653+0000 7f44d1d72b80 1 -- 172.21.15.44:0/2335811700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e4c0d0 msgr2=0x55ff84e554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.653+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/2335811700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e4c0d0 0x55ff84e554b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f44b4000c00 tx=0x7f44b40041a0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.653+0000 7f44d1d72b80 1 -- 172.21.15.44:0/2335811700 shutdown_connections 2023-09-10T00:46:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.653+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/2335811700 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff84e656d0 0x55ff84e67ac0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.653+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/2335811700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e515b0 0x55ff84e55a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.653+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/2335811700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e58200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.653+0000 7f449effd700 0 client.26477 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.653+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/2335811700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e4c0d0 0x55ff84e554b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.653+0000 7f44d1d72b80 1 -- 172.21.15.44:0/2335811700 >> 172.21.15.44:0/2335811700 conn(0x55ff84dbc030 msgr2=0x55ff84e417a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.653+0000 7f44d1d72b80 1 -- 172.21.15.44:0/2335811700 shutdown_connections 2023-09-10T00:46:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.653+0000 7f44d1d72b80 1 -- 172.21.15.44:0/2335811700 wait complete. 2023-09-10T00:46:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.653+0000 7f44d1d72b80 20 client.26477 trim_cache size 0 max 16384 2023-09-10T00:46:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.668+0000 7f44d1d72b80 1 Processor -- start 2023-09-10T00:46:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.668+0000 7f44d1d72b80 1 -- start start 2023-09-10T00:46:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44d1d72b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4b710 0x55ff84e55470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44d1d72b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e4c0d0 0x55ff84e54d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44d1d72b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e515b0 0x55ff84e61ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44d1d72b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff84e53a30 con 0x55ff84e4b710 2023-09-10T00:46:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44d1d72b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff84e50110 con 0x55ff84e4c0d0 2023-09-10T00:46:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44d1d72b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff84e4beb0 con 0x55ff84e515b0 2023-09-10T00:46:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44bf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4b710 0x55ff84e55470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44bffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e515b0 0x55ff84e61ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44beffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e4c0d0 0x55ff84e54d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44bf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4b710 0x55ff84e55470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51194/0 (socket says 172.21.15.44:51194) 2023-09-10T00:46:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44bffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e515b0 0x55ff84e61ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58080/0 (socket says 172.21.15.44:58080) 2023-09-10T00:46:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44beffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e4c0d0 0x55ff84e54d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52936/0 (socket says 172.21.15.44:52936) 2023-09-10T00:46:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44bf7fe700 1 -- 172.21.15.44:0/1833456582 learned_addr learned my addr 172.21.15.44:0/1833456582 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44bffff700 1 -- 172.21.15.44:0/1833456582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e4c0d0 msgr2=0x55ff84e54d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44bffff700 1 --2- 172.21.15.44:0/1833456582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e4c0d0 0x55ff84e54d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44bffff700 1 -- 172.21.15.44:0/1833456582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4b710 msgr2=0x55ff84e55470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44bffff700 1 --2- 172.21.15.44:0/1833456582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4b710 0x55ff84e55470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44bffff700 1 -- 172.21.15.44:0/1833456582 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff84baec00 con 0x55ff84e515b0 2023-09-10T00:46:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44bf7fe700 1 --2- 172.21.15.44:0/1833456582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4b710 0x55ff84e55470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.669+0000 7f44beffd700 1 --2- 172.21.15.44:0/1833456582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e4c0d0 0x55ff84e54d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.670+0000 7f44bffff700 1 --2- 172.21.15.44:0/1833456582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e515b0 0x55ff84e61ab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f44b80c9210 tx=0x7f44b809f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=533bb37e9c1cfa0a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.670+0000 7f44a67fc700 1 -- 172.21.15.44:0/1833456582 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8099700 con 0x55ff84e515b0 2023-09-10T00:46:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.670+0000 7f44a67fc700 1 -- 172.21.15.44:0/1833456582 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f44b8099000 con 0x55ff84e515b0 2023-09-10T00:46:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.670+0000 7f44a67fc700 1 -- 172.21.15.44:0/1833456582 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8099260 con 0x55ff84e515b0 2023-09-10T00:46:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.671+0000 7f44d1d72b80 1 -- 172.21.15.44:0/1833456582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e515b0 msgr2=0x55ff84e61ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.671+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/1833456582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e515b0 0x55ff84e61ab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f44b80c9210 tx=0x7f44b809f040 comp rx=0 tx=0).stop 2023-09-10T00:46:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.671+0000 7f44d1d72b80 1 -- 172.21.15.44:0/1833456582 shutdown_connections 2023-09-10T00:46:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.671+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/1833456582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e515b0 0x55ff84e61ab0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.671+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/1833456582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4b710 0x55ff84e55470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.671+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/1833456582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e4c0d0 0x55ff84e54d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.671+0000 7f44d1d72b80 1 -- 172.21.15.44:0/1833456582 >> 172.21.15.44:0/1833456582 conn(0x55ff84dbc030 msgr2=0x55ff84e6c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.671+0000 7f44d1d72b80 1 -- 172.21.15.44:0/1833456582 shutdown_connections 2023-09-10T00:46:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.671+0000 7f44d1d72b80 1 -- 172.21.15.44:0/1833456582 wait complete. 2023-09-10T00:46:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.671+0000 7f44d1d72b80 1 -- start start 2023-09-10T00:46:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.671+0000 7f44d1d72b80 1 Processor -- start 2023-09-10T00:46:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44d1d72b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e58910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44d1d72b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4c0d0 0x55ff84e58e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44d1d72b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e515b0 0x55ff84e59390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44d1d72b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff84e5ced0 con 0x55ff84e4c0d0 2023-09-10T00:46:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44d1d72b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff84eb84c0 con 0x55ff84e515b0 2023-09-10T00:46:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44d1d72b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff84eb8650 con 0x55ff84e4b710 2023-09-10T00:46:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44d1d72b80 10 client.0 _ll_register_callbacks cb 0x7f44b8004ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:46:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44beffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4c0d0 0x55ff84e58e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44bf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e58910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44beffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4c0d0 0x55ff84e58e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51202/0 (socket says 172.21.15.44:51202) 2023-09-10T00:46:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44beffd700 1 -- 172.21.15.44:0/3949399437 learned_addr learned my addr 172.21.15.44:0/3949399437 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44bffff700 1 --2- 172.21.15.44:0/3949399437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e515b0 0x55ff84e59390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44bf7fe700 1 -- 172.21.15.44:0/3949399437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e515b0 msgr2=0x55ff84e59390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44bf7fe700 1 --2- 172.21.15.44:0/3949399437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e515b0 0x55ff84e59390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44bf7fe700 1 -- 172.21.15.44:0/3949399437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4c0d0 msgr2=0x55ff84e58e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44bf7fe700 1 --2- 172.21.15.44:0/3949399437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4c0d0 0x55ff84e58e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44bf7fe700 1 -- 172.21.15.44:0/3949399437 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c010dd40 con 0x55ff84e4b710 2023-09-10T00:46:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44beffd700 1 --2- 172.21.15.44:0/3949399437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4c0d0 0x55ff84e58e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f44bf7fe700 1 --2- 172.21.15.44:0/3949399437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e58910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f44b00094f0 tx=0x7f44b0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ccb8b5ddb28b88b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.672+0000 7f449effd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:46:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.673+0000 7f449effd700 1 -- 172.21.15.44:0/3949399437 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0027040 con 0x55ff84e4b710 2023-09-10T00:46:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.673+0000 7f449effd700 1 -- 172.21.15.44:0/3949399437 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f44b000b600 con 0x55ff84e4b710 2023-09-10T00:46:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.673+0000 7f44d1d72b80 1 -- 172.21.15.44:0/3949399437 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44b4003680 con 0x55ff84e4b710 2023-09-10T00:46:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.673+0000 7f449effd700 1 -- 172.21.15.44:0/3949399437 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44b001c040 con 0x55ff84e4b710 2023-09-10T00:46:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.673+0000 7f44d1d72b80 10 client.26425 fetch_fsmap learned FSMap version 42 2023-09-10T00:46:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.673+0000 7f44d1d72b80 1 -- 172.21.15.44:0/3949399437 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f44b4009e10 con 0x55ff84e4b710 2023-09-10T00:46:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.673+0000 7f449effd700 1 -- 172.21.15.44:0/3949399437 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f44b4003680 con 0x55ff84e4b710 2023-09-10T00:46:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.673+0000 7f449effd700 1 -- 172.21.15.44:0/3949399437 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f44b0005570 con 0x55ff84e4b710 2023-09-10T00:46:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.673+0000 7f44d1d72b80 10 client.26425 fetch_fsmap finished waiting for FSMap version 42 2023-09-10T00:46:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.673+0000 7f44d1d72b80 10 client.26425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:46:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.673+0000 7f44d1d72b80 20 client.26425 populate_metadata read hostname 'smithi044' 2023-09-10T00:46:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.673+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/3949399437 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff84e656d0 0x55ff84ebbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.673+0000 7f44d1d72b80 4 client.26425 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:46:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.673+0000 7f44d1d72b80 1 -- 172.21.15.44:0/3949399437 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ff84d928c0 con 0x55ff84e656d0 2023-09-10T00:46:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.673+0000 7f44beffd700 1 --2- 172.21.15.44:0/3949399437 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff84e656d0 0x55ff84ebbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.674+0000 7f449effd700 10 client.26425 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.674+0000 7f44beffd700 1 --2- 172.21.15.44:0/3949399437 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff84e656d0 0x55ff84ebbe30 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f98a3f81e08a5ee server_cookie=b4b17abb0d1e74f in_seq=0 out_seq=0 2023-09-10T00:46:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.683+0000 7f449effd700 1 -- 172.21.15.44:0/3949399437 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ff84d928c0 con 0x55ff84e656d0 2023-09-10T00:46:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.683+0000 7f449effd700 10 client.26425 handle_command_reply: tid=0 2023-09-10T00:46:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.683+0000 7f44d1d72b80 1 client.26425 shutdown 2023-09-10T00:46:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.684+0000 7f44d1d72b80 1 -- 172.21.15.44:0/3949399437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 msgr2=0x55ff84e58910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.684+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/3949399437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e58910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f44b00094f0 tx=0x7f44b0009c00 comp rx=0 tx=0).stop 2023-09-10T00:46:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.684+0000 7f44d1d72b80 1 -- 172.21.15.44:0/3949399437 shutdown_connections 2023-09-10T00:46:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.684+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/3949399437 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff84e656d0 0x55ff84ebbe30 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.684+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/3949399437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff84e515b0 0x55ff84e59390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.684+0000 7f449effd700 0 client.26425 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.684+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/3949399437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff84e4b710 0x55ff84e58910 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.684+0000 7f44d1d72b80 1 --2- 172.21.15.44:0/3949399437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff84e4c0d0 0x55ff84e58e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.684+0000 7f44d1d72b80 1 -- 172.21.15.44:0/3949399437 >> 172.21.15.44:0/3949399437 conn(0x55ff84dbc030 msgr2=0x55ff84e61ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.684+0000 7f44d1d72b80 1 -- 172.21.15.44:0/3949399437 shutdown_connections 2023-09-10T00:46:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.684+0000 7f44d1d72b80 1 -- 172.21.15.44:0/3949399437 wait complete. 2023-09-10T00:46:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.684+0000 7f44d1d72b80 20 client.26425 trim_cache size 0 max 16384 2023-09-10T00:46:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:46:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:46:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:46:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:46:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.684+0000 7f44c5bac700 1 -- 172.21.15.44:0/638337207 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44a80a07b0 msgr2=0x7f44a80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.684+0000 7f44c5bac700 1 --2- 172.21.15.44:0/638337207 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44a80a07b0 0x7f44a80a2c70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f44b00098c0 tx=0x7f44b0019040 comp rx=0 tx=0).stop 2023-09-10T00:46:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.684+0000 7f44c5bac700 1 -- 172.21.15.44:0/638337207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44c0106680 msgr2=0x7f44c01963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.684+0000 7f44c5bac700 1 --2- 172.21.15.44:0/638337207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44c0106680 0x7f44c01963f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f44b800efc0 tx=0x7f44b800c3a0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.685+0000 7f44c5bac700 1 -- 172.21.15.44:0/638337207 shutdown_connections 2023-09-10T00:46:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.685+0000 7f44c5bac700 1 --2- 172.21.15.44:0/638337207 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44a80a07b0 0x7f44a80a2c70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.685+0000 7f44c5bac700 1 --2- 172.21.15.44:0/638337207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44c0106680 0x7f44c01963f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.685+0000 7f44c5bac700 1 --2- 172.21.15.44:0/638337207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44c01005c0 0x7f44c019c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.685+0000 7f44c5bac700 1 --2- 172.21.15.44:0/638337207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c0100f80 0x7f44c019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.685+0000 7f44c5bac700 1 -- 172.21.15.44:0/638337207 >> 172.21.15.44:0/638337207 conn(0x7f44c00fc0b0 msgr2=0x7f44c00fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.685+0000 7f44c5bac700 1 -- 172.21.15.44:0/638337207 shutdown_connections 2023-09-10T00:46:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:24.685+0000 7f44c5bac700 1 -- 172.21.15.44:0/638337207 wait complete. 2023-09-10T00:46:24.737 INFO:teuthology.orchestra.run.smithi044.stderr:28600000 of 50000000 tuples (57%) done (elapsed 324.37 s, remaining 242.71 s) 2023-09-10T00:46:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:24 smithi131 ceph-mon[103713]: pgmap v175: 129 pgs: 129 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 MiB/s wr, 89 op/s 2023-09-10T00:46:25.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:24 smithi044 ceph-mon[98676]: pgmap v175: 129 pgs: 129 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 MiB/s wr, 89 op/s 2023-09-10T00:46:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:24 smithi099 ceph-mon[102692]: pgmap v175: 129 pgs: 129 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 MiB/s wr, 89 op/s 2023-09-10T00:46:25.742 INFO:teuthology.orchestra.run.smithi044.stderr:28700000 of 50000000 tuples (57%) done (elapsed 325.38 s, remaining 241.48 s) 2023-09-10T00:46:26.734 INFO:teuthology.orchestra.run.smithi044.stderr:28800000 of 50000000 tuples (57%) done (elapsed 326.37 s, remaining 240.24 s) 2023-09-10T00:46:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:26 smithi131 ceph-mon[103713]: pgmap v176: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 MiB/s wr, 132 op/s 2023-09-10T00:46:27.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:26 smithi044 ceph-mon[98676]: pgmap v176: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 MiB/s wr, 132 op/s 2023-09-10T00:46:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:26 smithi099 ceph-mon[102692]: pgmap v176: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 MiB/s wr, 132 op/s 2023-09-10T00:46:27.665 INFO:teuthology.orchestra.run.smithi044.stderr:28900000 of 50000000 tuples (57%) done (elapsed 327.30 s, remaining 238.96 s) 2023-09-10T00:46:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:29000000 of 50000000 tuples (58%) done (elapsed 328.21 s, remaining 237.67 s) 2023-09-10T00:46:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:28 smithi131 ceph-mon[103713]: pgmap v177: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 84 op/s 2023-09-10T00:46:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:28 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:28 smithi044 ceph-mon[98676]: pgmap v177: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 84 op/s 2023-09-10T00:46:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:28 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:28 smithi099 ceph-mon[102692]: pgmap v177: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 84 op/s 2023-09-10T00:46:29.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:28 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:29.695 INFO:teuthology.orchestra.run.smithi044.stderr:29100000 of 50000000 tuples (58%) done (elapsed 329.33 s, remaining 236.53 s) 2023-09-10T00:46:30.607 INFO:teuthology.orchestra.run.smithi044.stderr:29200000 of 50000000 tuples (58%) done (elapsed 330.24 s, remaining 235.24 s) 2023-09-10T00:46:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:30 smithi131 ceph-mon[103713]: pgmap v178: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 84 op/s 2023-09-10T00:46:31.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:30 smithi044 ceph-mon[98676]: pgmap v178: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 84 op/s 2023-09-10T00:46:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:30 smithi099 ceph-mon[102692]: pgmap v178: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 84 op/s 2023-09-10T00:46:31.576 INFO:teuthology.orchestra.run.smithi044.stderr:29300000 of 50000000 tuples (58%) done (elapsed 331.21 s, remaining 233.99 s) 2023-09-10T00:46:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:32 smithi131 ceph-mon[103713]: pgmap v179: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 MiB/s wr, 132 op/s 2023-09-10T00:46:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:32 smithi044 ceph-mon[98676]: pgmap v179: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 MiB/s wr, 132 op/s 2023-09-10T00:46:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:32 smithi099 ceph-mon[102692]: pgmap v179: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 MiB/s wr, 132 op/s 2023-09-10T00:46:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:34 smithi131 ceph-mon[103713]: pgmap v180: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 31 MiB/s wr, 91 op/s 2023-09-10T00:46:35.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:34 smithi044 ceph-mon[98676]: pgmap v180: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 31 MiB/s wr, 91 op/s 2023-09-10T00:46:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:34 smithi099 ceph-mon[102692]: pgmap v180: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 31 MiB/s wr, 91 op/s 2023-09-10T00:46:35.192 INFO:teuthology.orchestra.run.smithi044.stderr:29400000 of 50000000 tuples (58%) done (elapsed 334.83 s, remaining 234.61 s) 2023-09-10T00:46:36.199 INFO:teuthology.orchestra.run.smithi044.stderr:29500000 of 50000000 tuples (59%) done (elapsed 335.83 s, remaining 233.38 s) 2023-09-10T00:46:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:36 smithi131 ceph-mon[103713]: pgmap v181: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 38 MiB/s wr, 123 op/s 2023-09-10T00:46:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:36 smithi044 ceph-mon[98676]: pgmap v181: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 38 MiB/s wr, 123 op/s 2023-09-10T00:46:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:36 smithi099 ceph-mon[102692]: pgmap v181: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 38 MiB/s wr, 123 op/s 2023-09-10T00:46:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.808+0000 7fc8e5af8700 1 -- 172.21.15.44:0/158539807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8e01055b0 msgr2=0x7fc8e0105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.808+0000 7fc8e5af8700 1 --2- 172.21.15.44:0/158539807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8e01055b0 0x7fc8e0105990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009920 tx=0x7fc8d4009c30 comp rx=0 tx=0).stop 2023-09-10T00:46:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.808+0000 7fc8e5af8700 1 -- 172.21.15.44:0/158539807 shutdown_connections 2023-09-10T00:46:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.808+0000 7fc8e5af8700 1 --2- 172.21.15.44:0/158539807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8e0100120 0x7fc8e010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.808+0000 7fc8e5af8700 1 --2- 172.21.15.44:0/158539807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8e01055b0 0x7fc8e0105990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.808+0000 7fc8e5af8700 1 --2- 172.21.15.44:0/158539807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8e00ff760 0x7fc8e00ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.808+0000 7fc8e5af8700 1 -- 172.21.15.44:0/158539807 >> 172.21.15.44:0/158539807 conn(0x7fc8e0074f30 msgr2=0x7fc8e0075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.808+0000 7fc8e5af8700 1 -- 172.21.15.44:0/158539807 shutdown_connections 2023-09-10T00:46:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.808+0000 7fc8e5af8700 1 -- 172.21.15.44:0/158539807 wait complete. 2023-09-10T00:46:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.809+0000 7fc8e5af8700 1 Processor -- start 2023-09-10T00:46:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.809+0000 7fc8e5af8700 1 -- start start 2023-09-10T00:46:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.809+0000 7fc8e5af8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8e00ff760 0x7fc8e01975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.809+0000 7fc8e5af8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8e0100120 0x7fc8e0197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.809+0000 7fc8e5af8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8e01055b0 0x7fc8e01a2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.809+0000 7fc8e5af8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e01a2a40 con 0x7fc8e01055b0 2023-09-10T00:46:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.809+0000 7fc8e5af8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e01a2bb0 con 0x7fc8e00ff760 2023-09-10T00:46:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.809+0000 7fc8e5af8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e01a2d70 con 0x7fc8e0100120 2023-09-10T00:46:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.809+0000 7fc8df7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8e00ff760 0x7fc8e01975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.809+0000 7fc8dffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8e01055b0 0x7fc8e01a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.809+0000 7fc8df7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8e00ff760 0x7fc8e01975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52954/0 (socket says 172.21.15.44:52954) 2023-09-10T00:46:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.809+0000 7fc8df7fe700 1 -- 172.21.15.44:0/1433608798 learned_addr learned my addr 172.21.15.44:0/1433608798 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.809+0000 7fc8deffd700 1 --2- 172.21.15.44:0/1433608798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8e0100120 0x7fc8e0197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.810+0000 7fc8dffff700 1 -- 172.21.15.44:0/1433608798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8e0100120 msgr2=0x7fc8e0197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.810+0000 7fc8dffff700 1 --2- 172.21.15.44:0/1433608798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8e0100120 0x7fc8e0197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.810+0000 7fc8dffff700 1 -- 172.21.15.44:0/1433608798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8e00ff760 msgr2=0x7fc8e01975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.810+0000 7fc8dffff700 1 --2- 172.21.15.44:0/1433608798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8e00ff760 0x7fc8e01975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.810+0000 7fc8dffff700 1 -- 172.21.15.44:0/1433608798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d00095f0 con 0x7fc8e01055b0 2023-09-10T00:46:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.810+0000 7fc8dffff700 1 --2- 172.21.15.44:0/1433608798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8e01055b0 0x7fc8e01a2440 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800efc0 tx=0x7fc8d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.810+0000 7fc8dcff9700 1 -- 172.21.15.44:0/1433608798 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8018070 con 0x7fc8e01055b0 2023-09-10T00:46:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.810+0000 7fc8dcff9700 1 -- 172.21.15.44:0/1433608798 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8d800f040 con 0x7fc8e01055b0 2023-09-10T00:46:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.810+0000 7fc8e5af8700 1 -- 172.21.15.44:0/1433608798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d4009520 con 0x7fc8e01055b0 2023-09-10T00:46:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.810+0000 7fc8dcff9700 1 -- 172.21.15.44:0/1433608798 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800cba0 con 0x7fc8e01055b0 2023-09-10T00:46:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.810+0000 7fc8e5af8700 1 -- 172.21.15.44:0/1433608798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e01a3340 con 0x7fc8e01055b0 2023-09-10T00:46:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.812+0000 7fc8dcff9700 1 -- 172.21.15.44:0/1433608798 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc8d8003a00 con 0x7fc8e01055b0 2023-09-10T00:46:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.813+0000 7fc8dcff9700 1 --2- 172.21.15.44:0/1433608798 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc8c80a07b0 0x7fc8c80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.813+0000 7fc8df7fe700 1 --2- 172.21.15.44:0/1433608798 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc8c80a07b0 0x7fc8c80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.813+0000 7fc8dcff9700 1 -- 172.21.15.44:0/1433608798 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc8d80cbe10 con 0x7fc8e01055b0 2023-09-10T00:46:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.814+0000 7fc8df7fe700 1 --2- 172.21.15.44:0/1433608798 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc8c80a07b0 0x7fc8c80a2c70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4005950 tx=0x7fc8d4019040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.814+0000 7fc8f1cbeb80 1 Processor -- start 2023-09-10T00:46:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.814+0000 7fc8f1cbeb80 1 -- start start 2023-09-10T00:46:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.814+0000 7fc8f1cbeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f1561a0 0x564d6f156580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.814+0000 7fc8f1cbeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150300 0x564d6f150780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.814+0000 7fc8f1cbeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150cc0 0x564d6f15cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.814+0000 7fc8f1cbeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d6f0a96d0 con 0x564d6f1561a0 2023-09-10T00:46:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.814+0000 7fc8f1cbeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d6f0a6ff0 con 0x564d6f150300 2023-09-10T00:46:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.814+0000 7fc8f1cbeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d6f0a28e0 con 0x564d6f150cc0 2023-09-10T00:46:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.814+0000 7fc8df7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f1561a0 0x564d6f156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.815+0000 7fc8df7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f1561a0 0x564d6f156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51222/0 (socket says 172.21.15.44:51222) 2023-09-10T00:46:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.815+0000 7fc8dffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150cc0 0x564d6f15cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.815+0000 7fc8deffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150300 0x564d6f150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.815+0000 7fc8df7fe700 1 -- 172.21.15.44:0/3431838178 learned_addr learned my addr 172.21.15.44:0/3431838178 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.815+0000 7fc8df7fe700 1 -- 172.21.15.44:0/3431838178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150cc0 msgr2=0x564d6f15cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.815+0000 7fc8df7fe700 1 --2- 172.21.15.44:0/3431838178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150cc0 0x564d6f15cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.815+0000 7fc8df7fe700 1 -- 172.21.15.44:0/3431838178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150300 msgr2=0x564d6f150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.815+0000 7fc8df7fe700 1 --2- 172.21.15.44:0/3431838178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150300 0x564d6f150780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.815+0000 7fc8df7fe700 1 -- 172.21.15.44:0/3431838178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d6eeced80 con 0x564d6f1561a0 2023-09-10T00:46:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.815+0000 7fc8deffd700 1 --2- 172.21.15.44:0/3431838178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150300 0x564d6f150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.815+0000 7fc8df7fe700 1 --2- 172.21.15.44:0/3431838178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f1561a0 0x564d6f156580 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400ff60 tx=0x7fc8d401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa28ec4940fec4d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.815+0000 7fc8c5ffb700 1 -- 172.21.15.44:0/3431838178 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d402a070 con 0x564d6f1561a0 2023-09-10T00:46:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.815+0000 7fc8c5ffb700 1 -- 172.21.15.44:0/3431838178 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8d401d040 con 0x564d6f1561a0 2023-09-10T00:46:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.815+0000 7fc8c5ffb700 1 -- 172.21.15.44:0/3431838178 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40052a0 con 0x564d6f1561a0 2023-09-10T00:46:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.816+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/3431838178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f1561a0 msgr2=0x564d6f156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.816+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/3431838178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f1561a0 0x564d6f156580 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400ff60 tx=0x7fc8d401e000 comp rx=0 tx=0).stop 2023-09-10T00:46:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.816+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/3431838178 shutdown_connections 2023-09-10T00:46:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.816+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/3431838178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150cc0 0x564d6f15cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.816+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/3431838178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f1561a0 0x564d6f156580 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.816+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/3431838178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150300 0x564d6f150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.816+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/3431838178 >> 172.21.15.44:0/3431838178 conn(0x564d6f0c5030 msgr2=0x564d6f14b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.816+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/3431838178 shutdown_connections 2023-09-10T00:46:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.816+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/3431838178 wait complete. 2023-09-10T00:46:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.816+0000 7fc8f1cbeb80 1 -- start start 2023-09-10T00:46:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8f1cbeb80 1 Processor -- start 2023-09-10T00:46:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8f1cbeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f150300 0x564d6f15fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8f1cbeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150cc0 0x564d6f1602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8f1cbeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f1561a0 0x564d6f15a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8f1cbeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d6f040b10 con 0x564d6f150300 2023-09-10T00:46:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8f1cbeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d6f040810 con 0x564d6f1561a0 2023-09-10T00:46:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8f1cbeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d6ef248c0 con 0x564d6f150cc0 2023-09-10T00:46:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8f1cbeb80 10 client.0 _ll_register_callbacks cb 0x564d6ef8aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:46:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8df7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f150300 0x564d6f15fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8df7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f150300 0x564d6f15fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51228/0 (socket says 172.21.15.44:51228) 2023-09-10T00:46:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8df7fe700 1 -- 172.21.15.44:0/304173727 learned_addr learned my addr 172.21.15.44:0/304173727 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8deffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150cc0 0x564d6f1602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8dffff700 1 --2- 172.21.15.44:0/304173727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f1561a0 0x564d6f15a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8df7fe700 1 -- 172.21.15.44:0/304173727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150cc0 msgr2=0x564d6f1602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8df7fe700 1 --2- 172.21.15.44:0/304173727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150cc0 0x564d6f1602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8df7fe700 1 -- 172.21.15.44:0/304173727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f1561a0 msgr2=0x564d6f15a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8df7fe700 1 --2- 172.21.15.44:0/304173727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f1561a0 0x564d6f15a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.817+0000 7fc8df7fe700 1 -- 172.21.15.44:0/304173727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d0009b80 con 0x564d6f150300 2023-09-10T00:46:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.818+0000 7fc8df7fe700 1 --2- 172.21.15.44:0/304173727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f150300 0x564d6f15fd70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009c70 tx=0x7fc8d4005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db503ef8f56f106d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.818+0000 7fc8beffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:46:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.818+0000 7fc8beffd700 1 -- 172.21.15.44:0/304173727 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d402a040 con 0x564d6f150300 2023-09-10T00:46:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.818+0000 7fc8beffd700 1 -- 172.21.15.44:0/304173727 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc8d400b560 con 0x564d6f150300 2023-09-10T00:46:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.818+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/304173727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d6effa490 con 0x564d6f150300 2023-09-10T00:46:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.818+0000 7fc8beffd700 1 -- 172.21.15.44:0/304173727 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8d400b720 con 0x564d6f150300 2023-09-10T00:46:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.818+0000 7fc8f1cbeb80 10 client.16830 fetch_fsmap learned FSMap version 45 2023-09-10T00:46:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.818+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/304173727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7fc8e010cbc0 con 0x564d6f150300 2023-09-10T00:46:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.818+0000 7fc8beffd700 1 -- 172.21.15.44:0/304173727 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564d6effa490 con 0x564d6f150300 2023-09-10T00:46:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.818+0000 7fc8beffd700 1 -- 172.21.15.44:0/304173727 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 45) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc8d4023480 con 0x564d6f150300 2023-09-10T00:46:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.818+0000 7fc8f1cbeb80 10 client.16830 fetch_fsmap finished waiting for FSMap version 45 2023-09-10T00:46:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.818+0000 7fc8f1cbeb80 10 client.16830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:46:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.818+0000 7fc8f1cbeb80 20 client.16830 populate_metadata read hostname 'smithi044' 2023-09-10T00:46:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.818+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/304173727 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564d6f169dd0 0x564d6f16c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.818+0000 7fc8f1cbeb80 4 client.16830 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:46:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.818+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/304173727 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d6f0b7840 con 0x564d6f169dd0 2023-09-10T00:46:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.819+0000 7fc8deffd700 1 --2- 172.21.15.44:0/304173727 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564d6f169dd0 0x564d6f16c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.819+0000 7fc8deffd700 1 --2- 172.21.15.44:0/304173727 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564d6f169dd0 0x564d6f16c1c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2f25140b9a28489 server_cookie=fb0b2721464347d4 in_seq=0 out_seq=0 2023-09-10T00:46:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.819+0000 7fc8beffd700 10 client.16830 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.820+0000 7fc8beffd700 1 -- 172.21.15.44:0/304173727 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564d6f0b7840 con 0x564d6f169dd0 2023-09-10T00:46:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.820+0000 7fc8beffd700 10 client.16830 handle_command_reply: tid=0 2023-09-10T00:46:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.820+0000 7fc8f1cbeb80 1 client.16830 shutdown 2023-09-10T00:46:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.820+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/304173727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f150300 msgr2=0x564d6f15fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.820+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/304173727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f150300 0x564d6f15fd70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009c70 tx=0x7fc8d4005270 comp rx=0 tx=0).stop 2023-09-10T00:46:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.820+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/304173727 shutdown_connections 2023-09-10T00:46:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.820+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/304173727 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564d6f169dd0 0x564d6f16c1c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.820+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/304173727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f1561a0 0x564d6f15a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.820+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/304173727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f150300 0x564d6f15fd70 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.820+0000 7fc8beffd700 0 client.16830 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.820+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/304173727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150cc0 0x564d6f1602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.820+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/304173727 >> 172.21.15.44:0/304173727 conn(0x564d6f0c5030 msgr2=0x564d6f14a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.820+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/304173727 shutdown_connections 2023-09-10T00:46:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.820+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/304173727 wait complete. 2023-09-10T00:46:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.820+0000 7fc8f1cbeb80 20 client.16830 trim_cache size 0 max 16384 2023-09-10T00:46:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.835+0000 7fc8f1cbeb80 1 Processor -- start 2023-09-10T00:46:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.835+0000 7fc8f1cbeb80 1 -- start start 2023-09-10T00:46:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.835+0000 7fc8f1cbeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f150300 0x564d6f154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.835+0000 7fc8f1cbeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150cc0 0x564d6f155530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.835+0000 7fc8f1cbeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f1561a0 0x564d6f1598a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.835+0000 7fc8f1cbeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d6f1565b0 con 0x564d6f150300 2023-09-10T00:46:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.835+0000 7fc8f1cbeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d6f154be0 con 0x564d6f150cc0 2023-09-10T00:46:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.835+0000 7fc8f1cbeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d6f154960 con 0x564d6f1561a0 2023-09-10T00:46:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.835+0000 7fc8df7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f150300 0x564d6f154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.835+0000 7fc8df7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f150300 0x564d6f154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51236/0 (socket says 172.21.15.44:51236) 2023-09-10T00:46:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.835+0000 7fc8df7fe700 1 -- 172.21.15.44:0/3086032482 learned_addr learned my addr 172.21.15.44:0/3086032482 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.835+0000 7fc8deffd700 1 --2- 172.21.15.44:0/3086032482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150cc0 0x564d6f155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.835+0000 7fc8dffff700 1 --2- 172.21.15.44:0/3086032482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f1561a0 0x564d6f1598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.836+0000 7fc8df7fe700 1 -- 172.21.15.44:0/3086032482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f1561a0 msgr2=0x564d6f1598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.836+0000 7fc8df7fe700 1 --2- 172.21.15.44:0/3086032482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f1561a0 0x564d6f1598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.836+0000 7fc8df7fe700 1 -- 172.21.15.44:0/3086032482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150cc0 msgr2=0x564d6f155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.836+0000 7fc8df7fe700 1 --2- 172.21.15.44:0/3086032482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150cc0 0x564d6f155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.836+0000 7fc8df7fe700 1 -- 172.21.15.44:0/3086032482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d6eeb7c00 con 0x564d6f150300 2023-09-10T00:46:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.836+0000 7fc8dffff700 1 --2- 172.21.15.44:0/3086032482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f1561a0 0x564d6f1598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.836+0000 7fc8deffd700 1 --2- 172.21.15.44:0/3086032482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150cc0 0x564d6f155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.836+0000 7fc8df7fe700 1 --2- 172.21.15.44:0/3086032482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f150300 0x564d6f154ff0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4006010 tx=0x7fc8d400bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dedc705f3944c25 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.836+0000 7fc8c67fc700 1 -- 172.21.15.44:0/3086032482 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d402a070 con 0x564d6f150300 2023-09-10T00:46:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.836+0000 7fc8c67fc700 1 -- 172.21.15.44:0/3086032482 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8d400f7b0 con 0x564d6f150300 2023-09-10T00:46:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.836+0000 7fc8c67fc700 1 -- 172.21.15.44:0/3086032482 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4003680 con 0x564d6f150300 2023-09-10T00:46:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/3086032482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f150300 msgr2=0x564d6f154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/3086032482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f150300 0x564d6f154ff0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4006010 tx=0x7fc8d400bc50 comp rx=0 tx=0).stop 2023-09-10T00:46:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/3086032482 shutdown_connections 2023-09-10T00:46:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/3086032482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f1561a0 0x564d6f1598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/3086032482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f150300 0x564d6f154ff0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/3086032482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150cc0 0x564d6f155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/3086032482 >> 172.21.15.44:0/3086032482 conn(0x564d6f0c5030 msgr2=0x564d6f15a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/3086032482 shutdown_connections 2023-09-10T00:46:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/3086032482 wait complete. 2023-09-10T00:46:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 -- start start 2023-09-10T00:46:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 Processor -- start 2023-09-10T00:46:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150300 0x564d6f1b6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150cc0 0x564d6f1b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f1561a0 0x564d6f1c0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d6f1b6e10 con 0x564d6f1561a0 2023-09-10T00:46:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d6f1c1530 con 0x564d6f150cc0 2023-09-10T00:46:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.837+0000 7fc8f1cbeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d6f1c16f0 con 0x564d6f150300 2023-09-10T00:46:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8f1cbeb80 10 client.0 _ll_register_callbacks cb 0x7fc8d0009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:46:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8df7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150300 0x564d6f1b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8deffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150cc0 0x564d6f1b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8dffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f1561a0 0x564d6f1c0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8deffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150cc0 0x564d6f1b66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52984/0 (socket says 172.21.15.44:52984) 2023-09-10T00:46:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8df7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150300 0x564d6f1b6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58124/0 (socket says 172.21.15.44:58124) 2023-09-10T00:46:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8deffd700 1 -- 172.21.15.44:0/579797351 learned_addr learned my addr 172.21.15.44:0/579797351 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8df7fe700 1 -- 172.21.15.44:0/579797351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150cc0 msgr2=0x564d6f1b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8df7fe700 1 --2- 172.21.15.44:0/579797351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150cc0 0x564d6f1b66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8df7fe700 1 -- 172.21.15.44:0/579797351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f1561a0 msgr2=0x564d6f1c0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8df7fe700 1 --2- 172.21.15.44:0/579797351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f1561a0 0x564d6f1c0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8df7fe700 1 -- 172.21.15.44:0/579797351 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d0012170 con 0x564d6f150300 2023-09-10T00:46:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8deffd700 1 --2- 172.21.15.44:0/579797351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150cc0 0x564d6f1b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8dffff700 1 --2- 172.21.15.44:0/579797351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f1561a0 0x564d6f1c0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8df7fe700 1 --2- 172.21.15.44:0/579797351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150300 0x564d6f1b6160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009c70 tx=0x7fc8d4023c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5577eca512c434f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8beffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:46:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8beffd700 1 -- 172.21.15.44:0/579797351 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d402a040 con 0x564d6f150300 2023-09-10T00:46:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/579797351 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8d401d040 con 0x564d6f150300 2023-09-10T00:46:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8beffd700 1 -- 172.21.15.44:0/579797351 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc8d4005390 con 0x564d6f150300 2023-09-10T00:46:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8beffd700 1 -- 172.21.15.44:0/579797351 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8d4023620 con 0x564d6f150300 2023-09-10T00:46:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.838+0000 7fc8f1cbeb80 10 client.26443 fetch_fsmap learned FSMap version 45 2023-09-10T00:46:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.839+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/579797351 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7fc8e01a3340 con 0x564d6f150300 2023-09-10T00:46:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.839+0000 7fc8beffd700 1 -- 172.21.15.44:0/579797351 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8d401d040 con 0x564d6f150300 2023-09-10T00:46:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.839+0000 7fc8beffd700 1 -- 172.21.15.44:0/579797351 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 45) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc8d404e030 con 0x564d6f150300 2023-09-10T00:46:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.839+0000 7fc8f1cbeb80 10 client.26443 fetch_fsmap finished waiting for FSMap version 45 2023-09-10T00:46:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.839+0000 7fc8f1cbeb80 10 client.26443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:46:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.839+0000 7fc8f1cbeb80 20 client.26443 populate_metadata read hostname 'smithi044' 2023-09-10T00:46:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.839+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/579797351 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564d6f1c29f0 0x564d6f1c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.839+0000 7fc8f1cbeb80 4 client.26443 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:46:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.839+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/579797351 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564d6f09b8c0 con 0x564d6f1c29f0 2023-09-10T00:46:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.839+0000 7fc8deffd700 1 --2- 172.21.15.44:0/579797351 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564d6f1c29f0 0x564d6f1c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.839+0000 7fc8beffd700 10 client.26443 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.839+0000 7fc8deffd700 1 --2- 172.21.15.44:0/579797351 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564d6f1c29f0 0x564d6f1c4de0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5af93089d0145a2 server_cookie=e553fe594572ce18 in_seq=0 out_seq=0 2023-09-10T00:46:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.856+0000 7fc8beffd700 1 -- 172.21.15.44:0/579797351 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564d6f09b8c0 con 0x564d6f1c29f0 2023-09-10T00:46:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.856+0000 7fc8beffd700 10 client.26443 handle_command_reply: tid=0 2023-09-10T00:46:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.856+0000 7fc8f1cbeb80 1 client.26443 shutdown 2023-09-10T00:46:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.856+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/579797351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150300 msgr2=0x564d6f1b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.856+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/579797351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150300 0x564d6f1b6160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009c70 tx=0x7fc8d4023c00 comp rx=0 tx=0).stop 2023-09-10T00:46:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.856+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/579797351 shutdown_connections 2023-09-10T00:46:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.856+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/579797351 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564d6f1c29f0 0x564d6f1c4de0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.856+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/579797351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d6f1561a0 0x564d6f1c0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.856+0000 7fc8beffd700 0 client.26443 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.856+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/579797351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d6f150300 0x564d6f1b6160 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.856+0000 7fc8f1cbeb80 1 --2- 172.21.15.44:0/579797351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d6f150cc0 0x564d6f1b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.857+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/579797351 >> 172.21.15.44:0/579797351 conn(0x564d6f0c5030 msgr2=0x564d6f14b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.857+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/579797351 shutdown_connections 2023-09-10T00:46:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.857+0000 7fc8f1cbeb80 1 -- 172.21.15.44:0/579797351 wait complete. 2023-09-10T00:46:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.857+0000 7fc8f1cbeb80 20 client.26443 trim_cache size 0 max 16384 2023-09-10T00:46:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:46:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:46:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:46:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:46:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.857+0000 7fc8e5af8700 1 -- 172.21.15.44:0/1433608798 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc8c80a07b0 msgr2=0x7fc8c80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.857+0000 7fc8e5af8700 1 --2- 172.21.15.44:0/1433608798 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc8c80a07b0 0x7fc8c80a2c70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4005950 tx=0x7fc8d4019040 comp rx=0 tx=0).stop 2023-09-10T00:46:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.857+0000 7fc8e5af8700 1 -- 172.21.15.44:0/1433608798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8e01055b0 msgr2=0x7fc8e01a2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.857+0000 7fc8e5af8700 1 --2- 172.21.15.44:0/1433608798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8e01055b0 0x7fc8e01a2440 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800efc0 tx=0x7fc8d800c3a0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.857+0000 7fc8e5af8700 1 -- 172.21.15.44:0/1433608798 shutdown_connections 2023-09-10T00:46:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.857+0000 7fc8e5af8700 1 --2- 172.21.15.44:0/1433608798 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc8c80a07b0 0x7fc8c80a2c70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.857+0000 7fc8e5af8700 1 --2- 172.21.15.44:0/1433608798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8e01055b0 0x7fc8e01a2440 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.857+0000 7fc8e5af8700 1 --2- 172.21.15.44:0/1433608798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8e00ff760 0x7fc8e01975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.857+0000 7fc8e5af8700 1 --2- 172.21.15.44:0/1433608798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8e0100120 0x7fc8e0197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.857+0000 7fc8e5af8700 1 -- 172.21.15.44:0/1433608798 >> 172.21.15.44:0/1433608798 conn(0x7fc8e0074f30 msgr2=0x7fc8e00fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.857+0000 7fc8e5af8700 1 -- 172.21.15.44:0/1433608798 shutdown_connections 2023-09-10T00:46:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:37.857+0000 7fc8e5af8700 1 -- 172.21.15.44:0/1433608798 wait complete. 2023-09-10T00:46:38.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:37 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:38.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:37 smithi131 ceph-mon[103713]: scrub summary: active paths [/] 2023-09-10T00:46:38.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:37 smithi131 ceph-mon[103713]: scrub summary: idle+waiting paths [/] 2023-09-10T00:46:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:37 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:37 smithi044 ceph-mon[98676]: scrub summary: active paths [/] 2023-09-10T00:46:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:37 smithi044 ceph-mon[98676]: scrub summary: idle+waiting paths [/] 2023-09-10T00:46:38.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:37 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:38.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:37 smithi099 ceph-mon[102692]: scrub summary: active paths [/] 2023-09-10T00:46:38.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:37 smithi099 ceph-mon[102692]: scrub summary: idle+waiting paths [/] 2023-09-10T00:46:38.447 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:46:38.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.591+0000 7f86bf065700 1 -- 172.21.15.44:0/488871106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b80ff4a0 msgr2=0x7f86b80ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:38.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.591+0000 7f86bf065700 1 --2- 172.21.15.44:0/488871106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b80ff4a0 0x7f86b80ff920 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f86a40099c0 tx=0x7f86a4009cd0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.591+0000 7f86bf065700 1 -- 172.21.15.44:0/488871106 shutdown_connections 2023-09-10T00:46:38.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.591+0000 7f86bf065700 1 --2- 172.21.15.44:0/488871106 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86b80ffe60 0x7f86b810c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.591+0000 7f86bf065700 1 --2- 172.21.15.44:0/488871106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b8105560 0x7f86b8105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.591+0000 7f86bf065700 1 --2- 172.21.15.44:0/488871106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b80ff4a0 0x7f86b80ff920 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.591+0000 7f86bf065700 1 -- 172.21.15.44:0/488871106 >> 172.21.15.44:0/488871106 conn(0x7f86b80faf50 msgr2=0x7f86b80fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:38.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.591+0000 7f86bf065700 1 -- 172.21.15.44:0/488871106 shutdown_connections 2023-09-10T00:46:38.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.591+0000 7f86bf065700 1 -- 172.21.15.44:0/488871106 wait complete. 2023-09-10T00:46:38.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86bf065700 1 Processor -- start 2023-09-10T00:46:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86bf065700 1 -- start start 2023-09-10T00:46:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86bf065700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86b80ff4a0 0x7f86b8196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86bf065700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b80ffe60 0x7f86b8196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86bf065700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b8105560 0x7f86b819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86bf065700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b819b4e0 con 0x7f86b80ffe60 2023-09-10T00:46:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86bf065700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b819b650 con 0x7f86b8105560 2023-09-10T00:46:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86bf065700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b819b810 con 0x7f86b80ff4a0 2023-09-10T00:46:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86b7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b80ffe60 0x7f86b8196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86bce01700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86b80ff4a0 0x7f86b8196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86b7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b80ffe60 0x7f86b8196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51258/0 (socket says 172.21.15.44:51258) 2023-09-10T00:46:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86b7fff700 1 -- 172.21.15.44:0/1585064246 learned_addr learned my addr 172.21.15.44:0/1585064246 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86bd602700 1 --2- 172.21.15.44:0/1585064246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b8105560 0x7f86b819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86b7fff700 1 -- 172.21.15.44:0/1585064246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86b80ff4a0 msgr2=0x7f86b8196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86b7fff700 1 --2- 172.21.15.44:0/1585064246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86b80ff4a0 0x7f86b8196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86b7fff700 1 -- 172.21.15.44:0/1585064246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b8105560 msgr2=0x7f86b819aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86b7fff700 1 --2- 172.21.15.44:0/1585064246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b8105560 0x7f86b819aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.592+0000 7f86b7fff700 1 -- 172.21.15.44:0/1585064246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a40095f0 con 0x7f86b80ffe60 2023-09-10T00:46:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.593+0000 7f86b7fff700 1 --2- 172.21.15.44:0/1585064246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b80ffe60 0x7f86b8196a40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009fd0 tx=0x7f86a400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.593+0000 7f86b5ffb700 1 -- 172.21.15.44:0/1585064246 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a401d070 con 0x7f86b80ffe60 2023-09-10T00:46:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.593+0000 7f86b5ffb700 1 -- 172.21.15.44:0/1585064246 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86a4005030 con 0x7f86b80ffe60 2023-09-10T00:46:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.593+0000 7f86b5ffb700 1 -- 172.21.15.44:0/1585064246 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4004990 con 0x7f86b80ffe60 2023-09-10T00:46:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.593+0000 7f86bf065700 1 -- 172.21.15.44:0/1585064246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b819ba90 con 0x7f86b80ffe60 2023-09-10T00:46:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.593+0000 7f86bf065700 1 -- 172.21.15.44:0/1585064246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b819bd10 con 0x7f86b80ffe60 2023-09-10T00:46:38.606 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.595+0000 7f86b5ffb700 1 -- 172.21.15.44:0/1585064246 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f86a40051a0 con 0x7f86b80ffe60 2023-09-10T00:46:38.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.596+0000 7f86b5ffb700 1 --2- 172.21.15.44:0/1585064246 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f86a80a07b0 0x7f86a80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.596+0000 7f86b5ffb700 1 -- 172.21.15.44:0/1585064246 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f86a40ccf00 con 0x7f86b80ffe60 2023-09-10T00:46:38.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.596+0000 7f86bce01700 1 --2- 172.21.15.44:0/1585064246 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f86a80a07b0 0x7f86a80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.607 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.597+0000 7f86bce01700 1 --2- 172.21.15.44:0/1585064246 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f86a80a07b0 0x7f86a80a2c70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009c60 tx=0x7f86ac009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.597+0000 7f86cb22bb80 1 Processor -- start 2023-09-10T00:46:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.597+0000 7f86cb22bb80 1 -- start start 2023-09-10T00:46:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.597+0000 7f86cb22bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0234710 0x55dfd0234af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.597+0000 7f86cb22bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd0235030 0x55dfd023e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.597+0000 7f86cb22bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd023ee40 0x55dfd0241230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.597+0000 7f86cb22bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd018d730 con 0x55dfd023ee40 2023-09-10T00:46:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.597+0000 7f86cb22bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd018b050 con 0x55dfd0235030 2023-09-10T00:46:38.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.597+0000 7f86cb22bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd0186940 con 0x55dfd0234710 2023-09-10T00:46:38.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.597+0000 7f86bce01700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0234710 0x55dfd0234af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.597+0000 7f86bd602700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd023ee40 0x55dfd0241230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.597+0000 7f86b7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd0235030 0x55dfd023e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.597+0000 7f86bce01700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0234710 0x55dfd0234af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58146/0 (socket says 172.21.15.44:58146) 2023-09-10T00:46:38.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.597+0000 7f86bce01700 1 -- 172.21.15.44:0/1778308191 learned_addr learned my addr 172.21.15.44:0/1778308191 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:38.611 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.598+0000 7f86b7fff700 1 -- 172.21.15.44:0/1778308191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0234710 msgr2=0x55dfd0234af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:38.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.598+0000 7f86b7fff700 1 --2- 172.21.15.44:0/1778308191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0234710 0x55dfd0234af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.598+0000 7f86b7fff700 1 -- 172.21.15.44:0/1778308191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd023ee40 msgr2=0x55dfd0241230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:38.612 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.598+0000 7f86b7fff700 1 --2- 172.21.15.44:0/1778308191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd023ee40 0x55dfd0241230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.598+0000 7f86b7fff700 1 -- 172.21.15.44:0/1778308191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfcffb2de0 con 0x55dfd0235030 2023-09-10T00:46:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.598+0000 7f86bce01700 1 --2- 172.21.15.44:0/1778308191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0234710 0x55dfd0234af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.598+0000 7f86b7fff700 1 --2- 172.21.15.44:0/1778308191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd0235030 0x55dfd023e900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f86a40ca2b0 tx=0x7f86a409a870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6faa01c04635490 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.598+0000 7f86a2ffd700 1 -- 172.21.15.44:0/1778308191 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a40a2080 con 0x55dfd0235030 2023-09-10T00:46:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.598+0000 7f86a2ffd700 1 -- 172.21.15.44:0/1778308191 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86a40967b0 con 0x55dfd0235030 2023-09-10T00:46:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.598+0000 7f86a2ffd700 1 -- 172.21.15.44:0/1778308191 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a40997f0 con 0x55dfd0235030 2023-09-10T00:46:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.599+0000 7f86cb22bb80 1 -- 172.21.15.44:0/1778308191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd0235030 msgr2=0x55dfd023e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.599+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/1778308191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd0235030 0x55dfd023e900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f86a40ca2b0 tx=0x7f86a409a870 comp rx=0 tx=0).stop 2023-09-10T00:46:38.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.599+0000 7f86cb22bb80 1 -- 172.21.15.44:0/1778308191 shutdown_connections 2023-09-10T00:46:38.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.599+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/1778308191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd023ee40 0x55dfd0241230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.599+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/1778308191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0234710 0x55dfd0234af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.615 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.599+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/1778308191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd0235030 0x55dfd023e900 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.599+0000 7f86cb22bb80 1 -- 172.21.15.44:0/1778308191 >> 172.21.15.44:0/1778308191 conn(0x55dfd01a9090 msgr2=0x55dfd022fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:38.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.599+0000 7f86cb22bb80 1 -- 172.21.15.44:0/1778308191 shutdown_connections 2023-09-10T00:46:38.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.599+0000 7f86cb22bb80 1 -- 172.21.15.44:0/1778308191 wait complete. 2023-09-10T00:46:38.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.599+0000 7f86cb22bb80 1 -- start start 2023-09-10T00:46:38.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.599+0000 7f86cb22bb80 1 Processor -- start 2023-09-10T00:46:38.616 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86cb22bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0234710 0x55dfd023f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86cb22bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0235030 0x55dfd023f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86cb22bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd023ee40 0x55dfd024a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86cb22bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd0124b70 con 0x55dfd0234710 2023-09-10T00:46:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86cb22bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd0124870 con 0x55dfd023ee40 2023-09-10T00:46:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86cb22bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd0008920 con 0x55dfd0235030 2023-09-10T00:46:38.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86cb22bb80 10 client.0 _ll_register_callbacks cb 0x55dfd006ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:46:38.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86bce01700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0234710 0x55dfd023f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.618 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86b7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0235030 0x55dfd023f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86bd602700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd023ee40 0x55dfd024a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86b7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0235030 0x55dfd023f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58154/0 (socket says 172.21.15.44:58154) 2023-09-10T00:46:38.619 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86bce01700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0234710 0x55dfd023f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51270/0 (socket says 172.21.15.44:51270) 2023-09-10T00:46:38.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86b7fff700 1 -- 172.21.15.44:0/142954704 learned_addr learned my addr 172.21.15.44:0/142954704 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:38.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86b7fff700 1 -- 172.21.15.44:0/142954704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd023ee40 msgr2=0x55dfd024a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:38.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86b7fff700 1 --2- 172.21.15.44:0/142954704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd023ee40 0x55dfd024a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.620 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86b7fff700 1 -- 172.21.15.44:0/142954704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0234710 msgr2=0x55dfd023f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:38.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86b7fff700 1 --2- 172.21.15.44:0/142954704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0234710 0x55dfd023f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86b7fff700 1 -- 172.21.15.44:0/142954704 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ac017070 con 0x55dfd0235030 2023-09-10T00:46:38.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f86b7fff700 1 --2- 172.21.15.44:0/142954704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0235030 0x55dfd023f760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f86a40ca8b0 tx=0x7f86a409a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5781d280ba7409b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:38.621 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f8697fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:46:38.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.600+0000 7f8697fff700 1 -- 172.21.15.44:0/142954704 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a40a2080 con 0x55dfd0235030 2023-09-10T00:46:38.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.601+0000 7f86cb22bb80 1 -- 172.21.15.44:0/142954704 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfd00de4f0 con 0x55dfd0235030 2023-09-10T00:46:38.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.601+0000 7f8697fff700 1 -- 172.21.15.44:0/142954704 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f86a40a1e00 con 0x55dfd0235030 2023-09-10T00:46:38.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.601+0000 7f8697fff700 1 -- 172.21.15.44:0/142954704 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86a40b2270 con 0x55dfd0235030 2023-09-10T00:46:38.622 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.601+0000 7f86cb22bb80 10 client.26455 fetch_fsmap learned FSMap version 45 2023-09-10T00:46:38.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.601+0000 7f86cb22bb80 1 -- 172.21.15.44:0/142954704 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f86b819ba90 con 0x55dfd0235030 2023-09-10T00:46:38.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.601+0000 7f8697fff700 1 -- 172.21.15.44:0/142954704 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dfd00de4f0 con 0x55dfd0235030 2023-09-10T00:46:38.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.601+0000 7f8697fff700 1 -- 172.21.15.44:0/142954704 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 45) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f86a4004bf0 con 0x55dfd0235030 2023-09-10T00:46:38.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.601+0000 7f86cb22bb80 10 client.26455 fetch_fsmap finished waiting for FSMap version 45 2023-09-10T00:46:38.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.601+0000 7f86cb22bb80 10 client.26455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:46:38.623 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.601+0000 7f86cb22bb80 20 client.26455 populate_metadata read hostname 'smithi044' 2023-09-10T00:46:38.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.601+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/142954704 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55dfd024e0a0 0x55dfd0250490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.601+0000 7f86cb22bb80 4 client.26455 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:46:38.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.601+0000 7f86cb22bb80 1 -- 172.21.15.44:0/142954704 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfd019b8a0 con 0x55dfd024e0a0 2023-09-10T00:46:38.624 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.601+0000 7f86bce01700 1 --2- 172.21.15.44:0/142954704 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55dfd024e0a0 0x55dfd0250490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.602+0000 7f8697fff700 10 client.26455 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:38.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.602+0000 7f86bce01700 1 --2- 172.21.15.44:0/142954704 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55dfd024e0a0 0x55dfd0250490 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38e8acd783b8d34 server_cookie=45efaea3dda77418 in_seq=0 out_seq=0 2023-09-10T00:46:38.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.602+0000 7f8697fff700 1 -- 172.21.15.44:0/142954704 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dfd019b8a0 con 0x55dfd024e0a0 2023-09-10T00:46:38.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.602+0000 7f8697fff700 10 client.26455 handle_command_reply: tid=0 2023-09-10T00:46:38.625 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.602+0000 7f86cb22bb80 1 client.26455 shutdown 2023-09-10T00:46:38.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.602+0000 7f86cb22bb80 1 -- 172.21.15.44:0/142954704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0235030 msgr2=0x55dfd023f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:38.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.602+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/142954704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0235030 0x55dfd023f760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f86a40ca8b0 tx=0x7f86a409a5b0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.602+0000 7f86cb22bb80 1 -- 172.21.15.44:0/142954704 shutdown_connections 2023-09-10T00:46:38.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.602+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/142954704 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55dfd024e0a0 0x55dfd0250490 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.626 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.603+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/142954704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd023ee40 0x55dfd024a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.603+0000 7f8697fff700 0 client.26455 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:38.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.603+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/142954704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0234710 0x55dfd023f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.603+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/142954704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0235030 0x55dfd023f760 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.603+0000 7f86cb22bb80 1 -- 172.21.15.44:0/142954704 >> 172.21.15.44:0/142954704 conn(0x55dfd01a9090 msgr2=0x55dfd022f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:38.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.603+0000 7f86cb22bb80 1 -- 172.21.15.44:0/142954704 shutdown_connections 2023-09-10T00:46:38.627 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.603+0000 7f86cb22bb80 1 -- 172.21.15.44:0/142954704 wait complete. 2023-09-10T00:46:38.628 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.603+0000 7f86cb22bb80 20 client.26455 trim_cache size 0 max 16384 2023-09-10T00:46:38.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86cb22bb80 1 Processor -- start 2023-09-10T00:46:38.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86cb22bb80 1 -- start start 2023-09-10T00:46:38.633 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86cb22bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0234710 0x55dfd02392a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86cb22bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0235030 0x55dfd02397e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86cb22bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd023ee40 0x55dfd02499a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86cb22bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd023e990 con 0x55dfd0235030 2023-09-10T00:46:38.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86cb22bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd023e610 con 0x55dfd023ee40 2023-09-10T00:46:38.634 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86cb22bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd023eb90 con 0x55dfd0234710 2023-09-10T00:46:38.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86bce01700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0234710 0x55dfd02392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86b7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0235030 0x55dfd02397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86bce01700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0234710 0x55dfd02392a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58160/0 (socket says 172.21.15.44:58160) 2023-09-10T00:46:38.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86bce01700 1 -- 172.21.15.44:0/2904141822 learned_addr learned my addr 172.21.15.44:0/2904141822 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:38.635 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86bd602700 1 --2- 172.21.15.44:0/2904141822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd023ee40 0x55dfd02499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86b7fff700 1 -- 172.21.15.44:0/2904141822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0234710 msgr2=0x55dfd02392a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:38.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86b7fff700 1 --2- 172.21.15.44:0/2904141822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0234710 0x55dfd02392a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86b7fff700 1 -- 172.21.15.44:0/2904141822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd023ee40 msgr2=0x55dfd02499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:38.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86b7fff700 1 --2- 172.21.15.44:0/2904141822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd023ee40 0x55dfd02499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.636 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86b7fff700 1 -- 172.21.15.44:0/2904141822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfcff9bc60 con 0x55dfd0235030 2023-09-10T00:46:38.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.620+0000 7f86b7fff700 1 --2- 172.21.15.44:0/2904141822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0235030 0x55dfd02397e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f86a4096b10 tx=0x7f86a4096510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f9f6adf97db60f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:38.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.621+0000 7f86a37fe700 1 -- 172.21.15.44:0/2904141822 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a409a120 con 0x55dfd0235030 2023-09-10T00:46:38.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.621+0000 7f86a37fe700 1 -- 172.21.15.44:0/2904141822 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86a40a2080 con 0x55dfd0235030 2023-09-10T00:46:38.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.621+0000 7f86a37fe700 1 -- 172.21.15.44:0/2904141822 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4098510 con 0x55dfd0235030 2023-09-10T00:46:38.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.621+0000 7f86cb22bb80 1 -- 172.21.15.44:0/2904141822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0235030 msgr2=0x55dfd02397e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:38.637 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.621+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/2904141822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0235030 0x55dfd02397e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f86a4096b10 tx=0x7f86a4096510 comp rx=0 tx=0).stop 2023-09-10T00:46:38.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.621+0000 7f86cb22bb80 1 -- 172.21.15.44:0/2904141822 shutdown_connections 2023-09-10T00:46:38.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.621+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/2904141822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd023ee40 0x55dfd02499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.621+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/2904141822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0234710 0x55dfd02392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.621+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/2904141822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0235030 0x55dfd02397e0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.638 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.621+0000 7f86cb22bb80 1 -- 172.21.15.44:0/2904141822 >> 172.21.15.44:0/2904141822 conn(0x55dfd01a9090 msgr2=0x55dfd02386a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:38.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.621+0000 7f86cb22bb80 1 -- 172.21.15.44:0/2904141822 shutdown_connections 2023-09-10T00:46:38.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.621+0000 7f86cb22bb80 1 -- 172.21.15.44:0/2904141822 wait complete. 2023-09-10T00:46:38.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.621+0000 7f86cb22bb80 1 -- start start 2023-09-10T00:46:38.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86cb22bb80 1 Processor -- start 2023-09-10T00:46:38.639 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86cb22bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0234710 0x55dfd0243f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86cb22bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0235030 0x55dfd0242280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86cb22bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd023ee40 0x55dfd02427c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86cb22bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd0242d00 con 0x55dfd0234710 2023-09-10T00:46:38.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86cb22bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd0242e70 con 0x55dfd023ee40 2023-09-10T00:46:38.640 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86cb22bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd0243030 con 0x55dfd0235030 2023-09-10T00:46:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86cb22bb80 10 client.0 _ll_register_callbacks cb 0x7f86b0003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:46:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86bce01700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0234710 0x55dfd0243f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86bce01700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0234710 0x55dfd0243f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51284/0 (socket says 172.21.15.44:51284) 2023-09-10T00:46:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86bce01700 1 -- 172.21.15.44:0/1767687384 learned_addr learned my addr 172.21.15.44:0/1767687384 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86b7fff700 1 --2- 172.21.15.44:0/1767687384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0235030 0x55dfd0242280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86bd602700 1 --2- 172.21.15.44:0/1767687384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd023ee40 0x55dfd02427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86bce01700 1 -- 172.21.15.44:0/1767687384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0235030 msgr2=0x55dfd0242280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:38.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86bce01700 1 --2- 172.21.15.44:0/1767687384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0235030 0x55dfd0242280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.642 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86bce01700 1 -- 172.21.15.44:0/1767687384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd023ee40 msgr2=0x55dfd02427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:38.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86bce01700 1 --2- 172.21.15.44:0/1767687384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd023ee40 0x55dfd02427c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.622+0000 7f86bce01700 1 -- 172.21.15.44:0/1767687384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b819ba90 con 0x55dfd0234710 2023-09-10T00:46:38.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.623+0000 7f86bce01700 1 --2- 172.21.15.44:0/1767687384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0234710 0x55dfd0243f50 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f86ac011ab0 tx=0x7f86ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38105a7e9346679 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:38.643 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.623+0000 7f8697fff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:46:38.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.623+0000 7f8697fff700 1 -- 172.21.15.44:0/1767687384 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac022040 con 0x55dfd0234710 2023-09-10T00:46:38.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.623+0000 7f8697fff700 1 -- 172.21.15.44:0/1767687384 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f86ac003680 con 0x55dfd0234710 2023-09-10T00:46:38.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.623+0000 7f86cb22bb80 1 -- 172.21.15.44:0/1767687384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86a40967b0 con 0x55dfd0234710 2023-09-10T00:46:38.644 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.623+0000 7f8697fff700 1 -- 172.21.15.44:0/1767687384 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86ac003840 con 0x55dfd0234710 2023-09-10T00:46:38.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.623+0000 7f86cb22bb80 10 client.16854 fetch_fsmap learned FSMap version 45 2023-09-10T00:46:38.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.623+0000 7f86cb22bb80 1 -- 172.21.15.44:0/1767687384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55dfcffd9250 con 0x55dfd0234710 2023-09-10T00:46:38.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.623+0000 7f8697fff700 1 -- 172.21.15.44:0/1767687384 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86a40967b0 con 0x55dfd0234710 2023-09-10T00:46:38.645 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.623+0000 7f8697fff700 1 -- 172.21.15.44:0/1767687384 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 45) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f86ac028860 con 0x55dfd0234710 2023-09-10T00:46:38.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.623+0000 7f86cb22bb80 10 client.16854 fetch_fsmap finished waiting for FSMap version 45 2023-09-10T00:46:38.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.623+0000 7f86cb22bb80 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:46:38.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.623+0000 7f86cb22bb80 20 client.16854 populate_metadata read hostname 'smithi044' 2023-09-10T00:46:38.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.623+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/1767687384 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55dfd02a3240 0x55dfd02a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:38.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.623+0000 7f86cb22bb80 4 client.16854 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:46:38.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.623+0000 7f86cb22bb80 1 -- 172.21.15.44:0/1767687384 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dfd017f920 con 0x55dfd02a3240 2023-09-10T00:46:38.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f86b7fff700 1 --2- 172.21.15.44:0/1767687384 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55dfd02a3240 0x55dfd02a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:38.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f86b7fff700 1 --2- 172.21.15.44:0/1767687384 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55dfd02a3240 0x55dfd02a5660 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ecef2314dc2bea8 server_cookie=2ea8da60e48f7633 in_seq=0 out_seq=0 2023-09-10T00:46:38.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f8697fff700 10 client.16854 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:38.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f8697fff700 1 -- 172.21.15.44:0/1767687384 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55dfd017f920 con 0x55dfd02a3240 2023-09-10T00:46:38.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f8697fff700 10 client.16854 handle_command_reply: tid=0 2023-09-10T00:46:38.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f86cb22bb80 1 client.16854 shutdown 2023-09-10T00:46:38.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f86cb22bb80 1 -- 172.21.15.44:0/1767687384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0234710 msgr2=0x55dfd0243f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:38.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/1767687384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0234710 0x55dfd0243f50 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f86ac011ab0 tx=0x7f86ac0047c0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f86cb22bb80 1 -- 172.21.15.44:0/1767687384 shutdown_connections 2023-09-10T00:46:38.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/1767687384 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55dfd02a3240 0x55dfd02a5660 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/1767687384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfd023ee40 0x55dfd02427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/1767687384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfd0234710 0x55dfd0243f50 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f8697fff700 0 client.16854 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f86cb22bb80 1 --2- 172.21.15.44:0/1767687384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dfd0235030 0x55dfd0242280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f86cb22bb80 1 -- 172.21.15.44:0/1767687384 >> 172.21.15.44:0/1767687384 conn(0x55dfd01a9090 msgr2=0x55dfd02392a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f86cb22bb80 1 -- 172.21.15.44:0/1767687384 shutdown_connections 2023-09-10T00:46:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.624+0000 7f86cb22bb80 1 -- 172.21.15.44:0/1767687384 wait complete. 2023-09-10T00:46:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.625+0000 7f86cb22bb80 20 client.16854 trim_cache size 0 max 16384 2023-09-10T00:46:38.652 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:46:38.653 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "scrub active (2 inodes in the stack)", 2023-09-10T00:46:38.653 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:46:38.653 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:46:38.653 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:46:38.654 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:46:38.654 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:46:38.654 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:46:38.654 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:46:38.655 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:46:38.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.625+0000 7f86bf065700 1 -- 172.21.15.44:0/1585064246 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f86a80a07b0 msgr2=0x7f86a80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:38.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.625+0000 7f86bf065700 1 --2- 172.21.15.44:0/1585064246 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f86a80a07b0 0x7f86a80a2c70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009c60 tx=0x7f86ac009260 comp rx=0 tx=0).stop 2023-09-10T00:46:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.625+0000 7f86bf065700 1 -- 172.21.15.44:0/1585064246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b80ffe60 msgr2=0x7f86b8196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.625+0000 7f86bf065700 1 --2- 172.21.15.44:0/1585064246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b80ffe60 0x7f86b8196a40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009fd0 tx=0x7f86a400f690 comp rx=0 tx=0).stop 2023-09-10T00:46:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.625+0000 7f86bf065700 1 -- 172.21.15.44:0/1585064246 shutdown_connections 2023-09-10T00:46:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.625+0000 7f86bf065700 1 --2- 172.21.15.44:0/1585064246 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f86a80a07b0 0x7f86a80a2c70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.625+0000 7f86bf065700 1 --2- 172.21.15.44:0/1585064246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b8105560 0x7f86b819aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.625+0000 7f86bf065700 1 --2- 172.21.15.44:0/1585064246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86b80ff4a0 0x7f86b8196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.625+0000 7f86bf065700 1 --2- 172.21.15.44:0/1585064246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b80ffe60 0x7f86b8196a40 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.625+0000 7f86bf065700 1 -- 172.21.15.44:0/1585064246 >> 172.21.15.44:0/1585064246 conn(0x7f86b80faf50 msgr2=0x7f86b80fc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:38.659 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.625+0000 7f86bf065700 1 -- 172.21.15.44:0/1585064246 shutdown_connections 2023-09-10T00:46:38.659 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:46:38.625+0000 7f86bf065700 1 -- 172.21.15.44:0/1585064246 wait complete. 2023-09-10T00:46:38.659 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:46:38.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:38 smithi044 ceph-mon[98676]: pgmap v182: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 79 op/s 2023-09-10T00:46:38.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:38 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:38 smithi131 ceph-mon[103713]: pgmap v182: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 79 op/s 2023-09-10T00:46:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:38 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:38 smithi099 ceph-mon[102692]: pgmap v182: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 79 op/s 2023-09-10T00:46:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:38 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:40.286 INFO:teuthology.orchestra.run.smithi044.stderr:29600000 of 50000000 tuples (59%) done (elapsed 339.92 s, remaining 234.27 s) 2023-09-10T00:46:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:40 smithi131 ceph-mon[103713]: pgmap v183: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 79 op/s 2023-09-10T00:46:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:40 smithi044 ceph-mon[98676]: pgmap v183: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 79 op/s 2023-09-10T00:46:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:40 smithi099 ceph-mon[102692]: pgmap v183: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 79 op/s 2023-09-10T00:46:41.166 INFO:teuthology.orchestra.run.smithi044.stderr:29700000 of 50000000 tuples (59%) done (elapsed 340.80 s, remaining 232.94 s) 2023-09-10T00:46:42.112 INFO:teuthology.orchestra.run.smithi044.stderr:29800000 of 50000000 tuples (59%) done (elapsed 341.75 s, remaining 231.65 s) 2023-09-10T00:46:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:42 smithi131 ceph-mon[103713]: pgmap v184: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 28 MiB/s wr, 114 op/s 2023-09-10T00:46:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:42 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:42 smithi044 ceph-mon[98676]: pgmap v184: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 28 MiB/s wr, 114 op/s 2023-09-10T00:46:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:42 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:42 smithi099 ceph-mon[102692]: pgmap v184: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 28 MiB/s wr, 114 op/s 2023-09-10T00:46:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:42 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:44 smithi131 ceph-mon[103713]: pgmap v185: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 13 MiB/s wr, 66 op/s 2023-09-10T00:46:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:44 smithi044 ceph-mon[98676]: pgmap v185: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 13 MiB/s wr, 66 op/s 2023-09-10T00:46:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:44 smithi099 ceph-mon[102692]: pgmap v185: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 13 MiB/s wr, 66 op/s 2023-09-10T00:46:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:46 smithi131 ceph-mon[103713]: pgmap v186: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 20 MiB/s wr, 86 op/s 2023-09-10T00:46:47.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:46 smithi044 ceph-mon[98676]: pgmap v186: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 20 MiB/s wr, 86 op/s 2023-09-10T00:46:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:46 smithi099 ceph-mon[102692]: pgmap v186: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 20 MiB/s wr, 86 op/s 2023-09-10T00:46:47.678 INFO:teuthology.orchestra.run.smithi044.stderr:29900000 of 50000000 tuples (59%) done (elapsed 347.31 s, remaining 233.48 s) 2023-09-10T00:46:48.622 INFO:teuthology.orchestra.run.smithi044.stderr:30000000 of 50000000 tuples (60%) done (elapsed 348.26 s, remaining 232.17 s) 2023-09-10T00:46:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:48 smithi131 ceph-mon[103713]: pgmap v187: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 13 MiB/s wr, 55 op/s 2023-09-10T00:46:49.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:48 smithi044 ceph-mon[98676]: pgmap v187: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 13 MiB/s wr, 55 op/s 2023-09-10T00:46:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:48 smithi099 ceph-mon[102692]: pgmap v187: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 13 MiB/s wr, 55 op/s 2023-09-10T00:46:49.699 INFO:teuthology.orchestra.run.smithi044.stderr:30100000 of 50000000 tuples (60%) done (elapsed 349.33 s, remaining 230.95 s) 2023-09-10T00:46:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.968+0000 7f2628a18700 1 -- 172.21.15.44:0/3366105111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2624100120 msgr2=0x7f262410c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.968+0000 7f2628a18700 1 --2- 172.21.15.44:0/3366105111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2624100120 0x7f262410c330 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f261c00b0d0 tx=0x7f261c00b3e0 comp rx=0 tx=0).stop 2023-09-10T00:46:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.968+0000 7f2628a18700 1 -- 172.21.15.44:0/3366105111 shutdown_connections 2023-09-10T00:46:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.968+0000 7f2628a18700 1 --2- 172.21.15.44:0/3366105111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2624100120 0x7f262410c330 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.968+0000 7f2628a18700 1 --2- 172.21.15.44:0/3366105111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26241055b0 0x7f2624105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.968+0000 7f2628a18700 1 --2- 172.21.15.44:0/3366105111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26240ff760 0x7f26240ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.968+0000 7f2628a18700 1 -- 172.21.15.44:0/3366105111 >> 172.21.15.44:0/3366105111 conn(0x7f2624074f30 msgr2=0x7f2624075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.968+0000 7f2628a18700 1 -- 172.21.15.44:0/3366105111 shutdown_connections 2023-09-10T00:46:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.968+0000 7f2628a18700 1 -- 172.21.15.44:0/3366105111 wait complete. 2023-09-10T00:46:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.969+0000 7f2628a18700 1 Processor -- start 2023-09-10T00:46:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.969+0000 7f2628a18700 1 -- start start 2023-09-10T00:46:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.969+0000 7f2628a18700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26240ff760 0x7f26241012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.969+0000 7f2628a18700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2624100120 0x7f26241017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.969+0000 7f2628a18700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26241055b0 0x7f26241a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.969+0000 7f2628a18700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26241a4db0 con 0x7f26240ff760 2023-09-10T00:46:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.969+0000 7f2628a18700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26241a4ef0 con 0x7f26241055b0 2023-09-10T00:46:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.969+0000 7f2628a18700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26241a5030 con 0x7f2624100120 2023-09-10T00:46:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.969+0000 7f262259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26240ff760 0x7f26241012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.969+0000 7f262259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26240ff760 0x7f26241012a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51298/0 (socket says 172.21.15.44:51298) 2023-09-10T00:46:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.969+0000 7f262259c700 1 -- 172.21.15.44:0/4020603116 learned_addr learned my addr 172.21.15.44:0/4020603116 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.969+0000 7f2621d9b700 1 --2- 172.21.15.44:0/4020603116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2624100120 0x7f26241017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.969+0000 7f2622d9d700 1 --2- 172.21.15.44:0/4020603116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26241055b0 0x7f26241a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.970+0000 7f262259c700 1 -- 172.21.15.44:0/4020603116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2624100120 msgr2=0x7f26241017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.970+0000 7f262259c700 1 --2- 172.21.15.44:0/4020603116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2624100120 0x7f26241017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.970+0000 7f262259c700 1 -- 172.21.15.44:0/4020603116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26241055b0 msgr2=0x7f26241a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.970+0000 7f262259c700 1 --2- 172.21.15.44:0/4020603116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26241055b0 0x7f26241a4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.970+0000 7f262259c700 1 -- 172.21.15.44:0/4020603116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c009d00 con 0x7f26240ff760 2023-09-10T00:46:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.970+0000 7f262259c700 1 --2- 172.21.15.44:0/4020603116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26240ff760 0x7f26241012a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f261400e9a0 tx=0x7f261400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.970+0000 7f26137fe700 1 -- 172.21.15.44:0/4020603116 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614009790 con 0x7f26240ff760 2023-09-10T00:46:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.970+0000 7f2628a18700 1 -- 172.21.15.44:0/4020603116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26241a52e0 con 0x7f26240ff760 2023-09-10T00:46:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.970+0000 7f26137fe700 1 -- 172.21.15.44:0/4020603116 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2614010430 con 0x7f26240ff760 2023-09-10T00:46:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.970+0000 7f26137fe700 1 -- 172.21.15.44:0/4020603116 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261400cea0 con 0x7f26240ff760 2023-09-10T00:46:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.970+0000 7f2628a18700 1 -- 172.21.15.44:0/4020603116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26241a5840 con 0x7f26240ff760 2023-09-10T00:46:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.972+0000 7f26137fe700 1 -- 172.21.15.44:0/4020603116 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f26140105a0 con 0x7f26240ff760 2023-09-10T00:46:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.973+0000 7f26137fe700 1 --2- 172.21.15.44:0/4020603116 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f260c0a07b0 0x7f260c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.973+0000 7f26137fe700 1 -- 172.21.15.44:0/4020603116 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f2614014070 con 0x7f26240ff760 2023-09-10T00:46:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.973+0000 7f2621d9b700 1 --2- 172.21.15.44:0/4020603116 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f260c0a07b0 0x7f260c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.974+0000 7f2634bdeb80 1 Processor -- start 2023-09-10T00:46:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.974+0000 7f2634bdeb80 1 -- start start 2023-09-10T00:46:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.974+0000 7f2634bdeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e691a0 0x5599b4e69580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.974+0000 7f2634bdeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e63780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.974+0000 7f2634bdeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e63cc0 0x5599b4e6ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.974+0000 7f2634bdeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b4dbc6d0 con 0x5599b4e63300 2023-09-10T00:46:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.974+0000 7f2634bdeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b4db9ff0 con 0x5599b4e63cc0 2023-09-10T00:46:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.974+0000 7f2634bdeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b4db58e0 con 0x5599b4e691a0 2023-09-10T00:46:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.974+0000 7f262259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e691a0 0x5599b4e69580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.974+0000 7f2622d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e63cc0 0x5599b4e6ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.974+0000 7f262259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e691a0 0x5599b4e69580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58188/0 (socket says 172.21.15.44:58188) 2023-09-10T00:46:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.974+0000 7f262259c700 1 -- 172.21.15.44:0/2809788060 learned_addr learned my addr 172.21.15.44:0/2809788060 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.974+0000 7f2621d9b700 1 --2- 172.21.15.44:0/4020603116 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f260c0a07b0 0x7f260c0a2c70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2624102a70 tx=0x7f2618009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.974+0000 7f2621d9b700 1 --2- 172.21.15.44:0/2809788060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e63780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.975+0000 7f2622d9d700 1 -- 172.21.15.44:0/2809788060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e691a0 msgr2=0x5599b4e69580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.975+0000 7f2622d9d700 1 --2- 172.21.15.44:0/2809788060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e691a0 0x5599b4e69580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.975+0000 7f2622d9d700 1 -- 172.21.15.44:0/2809788060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 msgr2=0x5599b4e63780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.975+0000 7f2622d9d700 1 --2- 172.21.15.44:0/2809788060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e63780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.975+0000 7f2622d9d700 1 -- 172.21.15.44:0/2809788060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599b4be1d80 con 0x5599b4e63cc0 2023-09-10T00:46:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.975+0000 7f2621d9b700 1 --2- 172.21.15.44:0/2809788060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e63780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:46:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.975+0000 7f262259c700 1 --2- 172.21.15.44:0/2809788060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e691a0 0x5599b4e69580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.975+0000 7f2622d9d700 1 --2- 172.21.15.44:0/2809788060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e63cc0 0x5599b4e6ff30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f261c0062a0 tx=0x7f261c006210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4883e3c073e0b6d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.975+0000 7f2610ff9700 1 -- 172.21.15.44:0/2809788060 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c00e040 con 0x5599b4e63cc0 2023-09-10T00:46:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.975+0000 7f2610ff9700 1 -- 172.21.15.44:0/2809788060 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f261c007b90 con 0x5599b4e63cc0 2023-09-10T00:46:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.975+0000 7f2610ff9700 1 -- 172.21.15.44:0/2809788060 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c003c60 con 0x5599b4e63cc0 2023-09-10T00:46:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.976+0000 7f2634bdeb80 1 -- 172.21.15.44:0/2809788060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e63cc0 msgr2=0x5599b4e6ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.976+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/2809788060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e63cc0 0x5599b4e6ff30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f261c0062a0 tx=0x7f261c006210 comp rx=0 tx=0).stop 2023-09-10T00:46:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.976+0000 7f2634bdeb80 1 -- 172.21.15.44:0/2809788060 shutdown_connections 2023-09-10T00:46:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.976+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/2809788060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e63cc0 0x5599b4e6ff30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.976+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/2809788060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e691a0 0x5599b4e69580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.976+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/2809788060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e63780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.976+0000 7f2634bdeb80 1 -- 172.21.15.44:0/2809788060 >> 172.21.15.44:0/2809788060 conn(0x5599b4dd8030 msgr2=0x5599b4e5e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.976+0000 7f2634bdeb80 1 -- 172.21.15.44:0/2809788060 shutdown_connections 2023-09-10T00:46:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.976+0000 7f2634bdeb80 1 -- 172.21.15.44:0/2809788060 wait complete. 2023-09-10T00:46:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.976+0000 7f2634bdeb80 1 -- start start 2023-09-10T00:46:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.976+0000 7f2634bdeb80 1 Processor -- start 2023-09-10T00:46:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2634bdeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e6fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2634bdeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e63cc0 0x5599b4e6cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2634bdeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e691a0 0x5599b4e6d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2634bdeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b4d53b10 con 0x5599b4e63300 2023-09-10T00:46:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2634bdeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b4d53810 con 0x5599b4e691a0 2023-09-10T00:46:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2634bdeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b4c378c0 con 0x5599b4e63cc0 2023-09-10T00:46:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2634bdeb80 10 client.0 _ll_register_callbacks cb 0x5599b4c9da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:46:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f262259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e6fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f262259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e6fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51312/0 (socket says 172.21.15.44:51312) 2023-09-10T00:46:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f262259c700 1 -- 172.21.15.44:0/3908429119 learned_addr learned my addr 172.21.15.44:0/3908429119 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2621d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e63cc0 0x5599b4e6cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2622d9d700 1 --2- 172.21.15.44:0/3908429119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e691a0 0x5599b4e6d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2621d9b700 1 -- 172.21.15.44:0/3908429119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e691a0 msgr2=0x5599b4e6d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2621d9b700 1 --2- 172.21.15.44:0/3908429119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e691a0 0x5599b4e6d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2621d9b700 1 -- 172.21.15.44:0/3908429119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 msgr2=0x5599b4e6fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2621d9b700 1 --2- 172.21.15.44:0/3908429119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e6fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2621d9b700 1 -- 172.21.15.44:0/3908429119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26241a5840 con 0x5599b4e63cc0 2023-09-10T00:46:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2622d9d700 1 --2- 172.21.15.44:0/3908429119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e691a0 0x5599b4e6d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f262259c700 1 --2- 172.21.15.44:0/3908429119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e6fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:46:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2621d9b700 1 --2- 172.21.15.44:0/3908429119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e63cc0 0x5599b4e6cfc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x5599b4e5e060 tx=0x7f26180047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d190140c4deb9890 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.977+0000 7f2609ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:46:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.978+0000 7f2609ffb700 1 -- 172.21.15.44:0/3908429119 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261801dec0 con 0x5599b4e63cc0 2023-09-10T00:46:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.978+0000 7f2634bdeb80 1 -- 172.21.15.44:0/3908429119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599b4d0d490 con 0x5599b4e63cc0 2023-09-10T00:46:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.978+0000 7f2609ffb700 1 -- 172.21.15.44:0/3908429119 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f2618004020 con 0x5599b4e63cc0 2023-09-10T00:46:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.978+0000 7f2609ffb700 1 -- 172.21.15.44:0/3908429119 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26180041e0 con 0x5599b4e63cc0 2023-09-10T00:46:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.978+0000 7f2634bdeb80 10 client.26473 fetch_fsmap learned FSMap version 46 2023-09-10T00:46:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.978+0000 7f2634bdeb80 1 -- 172.21.15.44:0/3908429119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f262410cbc0 con 0x5599b4e63cc0 2023-09-10T00:46:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.978+0000 7f2609ffb700 1 -- 172.21.15.44:0/3908429119 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5599b4d0d490 con 0x5599b4e63cc0 2023-09-10T00:46:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.978+0000 7f2609ffb700 1 -- 172.21.15.44:0/3908429119 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 46) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f261801e6c0 con 0x5599b4e63cc0 2023-09-10T00:46:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.978+0000 7f2634bdeb80 10 client.26473 fetch_fsmap finished waiting for FSMap version 46 2023-09-10T00:46:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.978+0000 7f2634bdeb80 10 client.26473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:46:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.978+0000 7f2634bdeb80 20 client.26473 populate_metadata read hostname 'smithi044' 2023-09-10T00:46:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.978+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/3908429119 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5599b4e7d2f0 0x5599b4e7f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.978+0000 7f2634bdeb80 4 client.26473 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:46:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.978+0000 7f2634bdeb80 1 -- 172.21.15.44:0/3908429119 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599b4dca840 con 0x5599b4e7d2f0 2023-09-10T00:46:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.978+0000 7f262259c700 1 --2- 172.21.15.44:0/3908429119 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5599b4e7d2f0 0x5599b4e7f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.979+0000 7f2609ffb700 10 client.26473 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.979+0000 7f262259c700 1 --2- 172.21.15.44:0/3908429119 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5599b4e7d2f0 0x5599b4e7f710 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=133c6acefa1f2238 server_cookie=bf2a8d400897a76d in_seq=0 out_seq=0 2023-09-10T00:46:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.979+0000 7f2609ffb700 1 -- 172.21.15.44:0/3908429119 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5599b4dca840 con 0x5599b4e7d2f0 2023-09-10T00:46:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.979+0000 7f2609ffb700 10 client.26473 handle_command_reply: tid=0 2023-09-10T00:46:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.979+0000 7f2634bdeb80 1 client.26473 shutdown 2023-09-10T00:46:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.980+0000 7f2634bdeb80 1 -- 172.21.15.44:0/3908429119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e63cc0 msgr2=0x5599b4e6cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.980+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/3908429119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e63cc0 0x5599b4e6cfc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x5599b4e5e060 tx=0x7f26180047c0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.980+0000 7f2634bdeb80 1 -- 172.21.15.44:0/3908429119 shutdown_connections 2023-09-10T00:46:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.980+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/3908429119 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5599b4e7d2f0 0x5599b4e7f710 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.980+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/3908429119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e691a0 0x5599b4e6d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.980+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/3908429119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e6fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.980+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/3908429119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e63cc0 0x5599b4e6cfc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.980+0000 7f2609ffb700 0 client.26473 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.980+0000 7f2634bdeb80 1 -- 172.21.15.44:0/3908429119 >> 172.21.15.44:0/3908429119 conn(0x5599b4dd8030 msgr2=0x5599b4e5d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.980+0000 7f2634bdeb80 1 -- 172.21.15.44:0/3908429119 shutdown_connections 2023-09-10T00:46:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.980+0000 7f2634bdeb80 1 -- 172.21.15.44:0/3908429119 wait complete. 2023-09-10T00:46:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.980+0000 7f2634bdeb80 20 client.26473 trim_cache size 0 max 16384 2023-09-10T00:46:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.994+0000 7f2634bdeb80 1 Processor -- start 2023-09-10T00:46:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.994+0000 7f2634bdeb80 1 -- start start 2023-09-10T00:46:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f2634bdeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e68760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f2634bdeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e63cc0 0x5599b4e6f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f2634bdeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e691a0 0x5599b4e6c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f2634bdeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b4e55f50 con 0x5599b4e63300 2023-09-10T00:46:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f2634bdeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b4e64380 con 0x5599b4e691a0 2023-09-10T00:46:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f2634bdeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b4e588a0 con 0x5599b4e63cc0 2023-09-10T00:46:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f262259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e68760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f262259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e68760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51320/0 (socket says 172.21.15.44:51320) 2023-09-10T00:46:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f262259c700 1 -- 172.21.15.44:0/2539166726 learned_addr learned my addr 172.21.15.44:0/2539166726 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f2621d9b700 1 --2- 172.21.15.44:0/2539166726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e63cc0 0x5599b4e6f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f2622d9d700 1 --2- 172.21.15.44:0/2539166726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e691a0 0x5599b4e6c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f262259c700 1 -- 172.21.15.44:0/2539166726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e63cc0 msgr2=0x5599b4e6f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f262259c700 1 --2- 172.21.15.44:0/2539166726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e63cc0 0x5599b4e6f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f262259c700 1 -- 172.21.15.44:0/2539166726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e691a0 msgr2=0x5599b4e6c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f262259c700 1 --2- 172.21.15.44:0/2539166726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e691a0 0x5599b4e6c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f262259c700 1 -- 172.21.15.44:0/2539166726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599b4bcac00 con 0x5599b4e63300 2023-09-10T00:46:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f262259c700 1 --2- 172.21.15.44:0/2539166726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e68760 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f26140994e0 tx=0x7f261400c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713b642c1f6f351c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f26117fa700 1 -- 172.21.15.44:0/2539166726 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261409d150 con 0x5599b4e63300 2023-09-10T00:46:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f26117fa700 1 -- 172.21.15.44:0/2539166726 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2614099c50 con 0x5599b4e63300 2023-09-10T00:46:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.995+0000 7f26117fa700 1 -- 172.21.15.44:0/2539166726 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614095c60 con 0x5599b4e63300 2023-09-10T00:46:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.996+0000 7f2634bdeb80 1 -- 172.21.15.44:0/2539166726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 msgr2=0x5599b4e68760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.996+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/2539166726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e68760 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f26140994e0 tx=0x7f261400c480 comp rx=0 tx=0).stop 2023-09-10T00:46:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.996+0000 7f2634bdeb80 1 -- 172.21.15.44:0/2539166726 shutdown_connections 2023-09-10T00:46:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.996+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/2539166726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e691a0 0x5599b4e6c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.996+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/2539166726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63300 0x5599b4e68760 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.996+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/2539166726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e63cc0 0x5599b4e6f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.996+0000 7f2634bdeb80 1 -- 172.21.15.44:0/2539166726 >> 172.21.15.44:0/2539166726 conn(0x5599b4dd8030 msgr2=0x5599b4e62b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.996+0000 7f2634bdeb80 1 -- 172.21.15.44:0/2539166726 shutdown_connections 2023-09-10T00:46:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.996+0000 7f2634bdeb80 1 -- 172.21.15.44:0/2539166726 wait complete. 2023-09-10T00:46:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.996+0000 7f2634bdeb80 1 -- start start 2023-09-10T00:46:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.996+0000 7f2634bdeb80 1 Processor -- start 2023-09-10T00:46:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2634bdeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e63300 0x5599b4e745e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2634bdeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63cc0 0x5599b4e728d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2634bdeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e691a0 0x5599b4e72f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2634bdeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b4e73640 con 0x5599b4e63cc0 2023-09-10T00:46:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2634bdeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b4ed1170 con 0x5599b4e691a0 2023-09-10T00:46:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2634bdeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b4ed12b0 con 0x5599b4e63300 2023-09-10T00:46:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2634bdeb80 10 client.0 _ll_register_callbacks cb 0x7f261801d3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:46:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2621d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63cc0 0x5599b4e728d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2621d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63cc0 0x5599b4e728d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51328/0 (socket says 172.21.15.44:51328) 2023-09-10T00:46:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2621d9b700 1 -- 172.21.15.44:0/1133147980 learned_addr learned my addr 172.21.15.44:0/1133147980 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:46:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f262259c700 1 --2- 172.21.15.44:0/1133147980 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e63300 0x5599b4e745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2622d9d700 1 --2- 172.21.15.44:0/1133147980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e691a0 0x5599b4e72f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2621d9b700 1 -- 172.21.15.44:0/1133147980 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e63300 msgr2=0x5599b4e745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2621d9b700 1 --2- 172.21.15.44:0/1133147980 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e63300 0x5599b4e745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2621d9b700 1 -- 172.21.15.44:0/1133147980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e691a0 msgr2=0x5599b4e72f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:46:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2621d9b700 1 --2- 172.21.15.44:0/1133147980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e691a0 0x5599b4e72f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2621d9b700 1 -- 172.21.15.44:0/1133147980 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26241a52e0 con 0x5599b4e63cc0 2023-09-10T00:46:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2621d9b700 1 --2- 172.21.15.44:0/1133147980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63cc0 0x5599b4e728d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2618000c00 tx=0x7f26180040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885be9737309ed4d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:46:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2609ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:46:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2609ffb700 1 -- 172.21.15.44:0/1133147980 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618004230 con 0x5599b4e63cc0 2023-09-10T00:46:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2609ffb700 1 -- 172.21.15.44:0/1133147980 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f2618004390 con 0x5599b4e63cc0 2023-09-10T00:46:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2634bdeb80 1 -- 172.21.15.44:0/1133147980 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f261c007b90 con 0x5599b4e63cc0 2023-09-10T00:46:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2609ffb700 1 -- 172.21.15.44:0/1133147980 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2618004550 con 0x5599b4e63cc0 2023-09-10T00:46:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2634bdeb80 10 client.16869 fetch_fsmap learned FSMap version 46 2023-09-10T00:46:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.997+0000 7f2634bdeb80 1 -- 172.21.15.44:0/1133147980 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5599b4c081f0 con 0x5599b4e63cc0 2023-09-10T00:46:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.998+0000 7f2609ffb700 1 -- 172.21.15.44:0/1133147980 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f261c007b90 con 0x5599b4e63cc0 2023-09-10T00:46:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.998+0000 7f2609ffb700 1 -- 172.21.15.44:0/1133147980 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 46) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2618005520 con 0x5599b4e63cc0 2023-09-10T00:46:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.998+0000 7f2634bdeb80 10 client.16869 fetch_fsmap finished waiting for FSMap version 46 2023-09-10T00:46:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.998+0000 7f2634bdeb80 10 client.16869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:46:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.998+0000 7f2634bdeb80 20 client.16869 populate_metadata read hostname 'smithi044' 2023-09-10T00:46:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.998+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/1133147980 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5599b4ed29d0 0x5599b4ed4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:46:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.998+0000 7f2634bdeb80 4 client.16869 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:46:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.998+0000 7f2634bdeb80 1 -- 172.21.15.44:0/1133147980 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5599b4dae8c0 con 0x5599b4ed29d0 2023-09-10T00:46:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.998+0000 7f262259c700 1 --2- 172.21.15.44:0/1133147980 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5599b4ed29d0 0x5599b4ed4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:46:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.998+0000 7f262259c700 1 --2- 172.21.15.44:0/1133147980 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5599b4ed29d0 0x5599b4ed4df0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78ea0d0738545ef1 server_cookie=5e657ffd39d71f34 in_seq=0 out_seq=0 2023-09-10T00:46:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:50.998+0000 7f2609ffb700 10 client.16869 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.009+0000 7f2609ffb700 1 -- 172.21.15.44:0/1133147980 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5599b4dae8c0 con 0x5599b4ed29d0 2023-09-10T00:46:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.009+0000 7f2609ffb700 10 client.16869 handle_command_reply: tid=0 2023-09-10T00:46:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.009+0000 7f2634bdeb80 1 client.16869 shutdown 2023-09-10T00:46:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.009+0000 7f2634bdeb80 1 -- 172.21.15.44:0/1133147980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63cc0 msgr2=0x5599b4e728d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.009+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/1133147980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63cc0 0x5599b4e728d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2618000c00 tx=0x7f26180040c0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.009+0000 7f2634bdeb80 1 -- 172.21.15.44:0/1133147980 shutdown_connections 2023-09-10T00:46:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.009+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/1133147980 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5599b4ed29d0 0x5599b4ed4df0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.009+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/1133147980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599b4e691a0 0x5599b4e72f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.009+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/1133147980 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599b4e63300 0x5599b4e745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.009+0000 7f2634bdeb80 1 --2- 172.21.15.44:0/1133147980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599b4e63cc0 0x5599b4e728d0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.009+0000 7f2609ffb700 0 client.16869 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:46:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.010+0000 7f2634bdeb80 1 -- 172.21.15.44:0/1133147980 >> 172.21.15.44:0/1133147980 conn(0x5599b4dd8030 msgr2=0x5599b4e62b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.010+0000 7f2634bdeb80 1 -- 172.21.15.44:0/1133147980 shutdown_connections 2023-09-10T00:46:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.010+0000 7f2634bdeb80 1 -- 172.21.15.44:0/1133147980 wait complete. 2023-09-10T00:46:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.010+0000 7f2634bdeb80 20 client.16869 trim_cache size 0 max 16384 2023-09-10T00:46:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:46:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:46:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:46:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:46:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.010+0000 7f2628a18700 1 -- 172.21.15.44:0/4020603116 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f260c0a07b0 msgr2=0x7f260c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.010+0000 7f2628a18700 1 --2- 172.21.15.44:0/4020603116 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f260c0a07b0 0x7f260c0a2c70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2624102a70 tx=0x7f2618009260 comp rx=0 tx=0).stop 2023-09-10T00:46:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.010+0000 7f2628a18700 1 -- 172.21.15.44:0/4020603116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26240ff760 msgr2=0x7f26241012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:46:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.010+0000 7f2628a18700 1 --2- 172.21.15.44:0/4020603116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26240ff760 0x7f26241012a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f261400e9a0 tx=0x7f261400ecb0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.010+0000 7f2628a18700 1 -- 172.21.15.44:0/4020603116 shutdown_connections 2023-09-10T00:46:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.010+0000 7f2628a18700 1 --2- 172.21.15.44:0/4020603116 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f260c0a07b0 0x7f260c0a2c70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.010+0000 7f2628a18700 1 --2- 172.21.15.44:0/4020603116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26241055b0 0x7f26241a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.010+0000 7f2628a18700 1 --2- 172.21.15.44:0/4020603116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26240ff760 0x7f26241012a0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.010+0000 7f2628a18700 1 --2- 172.21.15.44:0/4020603116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2624100120 0x7f26241017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:46:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.010+0000 7f2628a18700 1 -- 172.21.15.44:0/4020603116 >> 172.21.15.44:0/4020603116 conn(0x7f2624074f30 msgr2=0x7f2624103200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:46:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.010+0000 7f2628a18700 1 -- 172.21.15.44:0/4020603116 shutdown_connections 2023-09-10T00:46:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:46:51.010+0000 7f2628a18700 1 -- 172.21.15.44:0/4020603116 wait complete. 2023-09-10T00:46:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:50 smithi131 ceph-mon[103713]: pgmap v188: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 13 MiB/s wr, 55 op/s 2023-09-10T00:46:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:50 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:46:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:50 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:46:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:50 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:46:51.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:50 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:46:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:50 smithi044 ceph-mon[98676]: pgmap v188: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 13 MiB/s wr, 55 op/s 2023-09-10T00:46:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:50 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:46:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:50 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:46:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:50 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:46:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:50 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:46:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:50 smithi099 ceph-mon[102692]: pgmap v188: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 13 MiB/s wr, 55 op/s 2023-09-10T00:46:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:50 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:46:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:50 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:46:51.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:50 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:46:51.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:50 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:46:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:52 smithi131 ceph-mon[103713]: pgmap v189: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 22 MiB/s wr, 93 op/s 2023-09-10T00:46:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:52 smithi044 ceph-mon[98676]: pgmap v189: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 22 MiB/s wr, 93 op/s 2023-09-10T00:46:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:52 smithi099 ceph-mon[102692]: pgmap v189: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 22 MiB/s wr, 93 op/s 2023-09-10T00:46:54.255 INFO:teuthology.orchestra.run.smithi044.stderr:30200000 of 50000000 tuples (60%) done (elapsed 353.89 s, remaining 232.02 s) 2023-09-10T00:46:55.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:54 smithi131 ceph-mon[103713]: pgmap v190: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 17 MiB/s wr, 58 op/s 2023-09-10T00:46:55.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:54 smithi044 ceph-mon[98676]: pgmap v190: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 17 MiB/s wr, 58 op/s 2023-09-10T00:46:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:54 smithi099 ceph-mon[102692]: pgmap v190: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 17 MiB/s wr, 58 op/s 2023-09-10T00:46:57.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:56 smithi131 ceph-mon[103713]: pgmap v191: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 21 MiB/s wr, 75 op/s 2023-09-10T00:46:57.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:56 smithi044 ceph-mon[98676]: pgmap v191: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 21 MiB/s wr, 75 op/s 2023-09-10T00:46:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:56 smithi099 ceph-mon[102692]: pgmap v191: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 21 MiB/s wr, 75 op/s 2023-09-10T00:46:59.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:46:58 smithi131 ceph-mon[103713]: pgmap v192: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 14 MiB/s wr, 55 op/s 2023-09-10T00:46:59.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:46:58 smithi044 ceph-mon[98676]: pgmap v192: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 14 MiB/s wr, 55 op/s 2023-09-10T00:46:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:46:58 smithi099 ceph-mon[102692]: pgmap v192: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 14 MiB/s wr, 55 op/s 2023-09-10T00:46:59.112 INFO:teuthology.orchestra.run.smithi044.stderr:30300000 of 50000000 tuples (60%) done (elapsed 358.75 s, remaining 233.24 s) 2023-09-10T00:46:59.975 INFO:teuthology.orchestra.run.smithi044.stderr:30400000 of 50000000 tuples (60%) done (elapsed 359.61 s, remaining 231.85 s) 2023-09-10T00:47:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:00 smithi131 ceph-mon[103713]: pgmap v193: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 14 MiB/s wr, 55 op/s 2023-09-10T00:47:01.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:00 smithi044 ceph-mon[98676]: pgmap v193: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 14 MiB/s wr, 55 op/s 2023-09-10T00:47:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:00 smithi099 ceph-mon[102692]: pgmap v193: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 14 MiB/s wr, 55 op/s 2023-09-10T00:47:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:02 smithi131 ceph-mon[103713]: pgmap v194: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 17 MiB/s wr, 64 op/s 2023-09-10T00:47:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:02 smithi044 ceph-mon[98676]: pgmap v194: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 17 MiB/s wr, 64 op/s 2023-09-10T00:47:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:02 smithi099 ceph-mon[102692]: pgmap v194: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 17 MiB/s wr, 64 op/s 2023-09-10T00:47:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.155+0000 7fd0d3853700 1 -- 172.21.15.44:0/1472660900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc106650 msgr2=0x7fd0cc106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.155+0000 7fd0d3853700 1 --2- 172.21.15.44:0/1472660900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc106650 0x7fd0cc106a30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80099c0 tx=0x7fd0c8009cd0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.155+0000 7fd0d3853700 1 -- 172.21.15.44:0/1472660900 shutdown_connections 2023-09-10T00:47:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.155+0000 7fd0d3853700 1 --2- 172.21.15.44:0/1472660900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0cc0683b0 0x7fd0cc10d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.155+0000 7fd0d3853700 1 --2- 172.21.15.44:0/1472660900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc106650 0x7fd0cc106a30 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.155+0000 7fd0d3853700 1 --2- 172.21.15.44:0/1472660900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0cc0679f0 0x7fd0cc067e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.155+0000 7fd0d3853700 1 -- 172.21.15.44:0/1472660900 >> 172.21.15.44:0/1472660900 conn(0x7fd0cc074f30 msgr2=0x7fd0cc075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.156+0000 7fd0d3853700 1 -- 172.21.15.44:0/1472660900 shutdown_connections 2023-09-10T00:47:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.156+0000 7fd0d3853700 1 -- 172.21.15.44:0/1472660900 wait complete. 2023-09-10T00:47:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.156+0000 7fd0d3853700 1 Processor -- start 2023-09-10T00:47:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.156+0000 7fd0d3853700 1 -- start start 2023-09-10T00:47:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.156+0000 7fd0d3853700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0cc0679f0 0x7fd0cc1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d3853700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0cc0683b0 0x7fd0cc199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d3853700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc106650 0x7fd0cc19d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d3853700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc19d9a0 con 0x7fd0cc106650 2023-09-10T00:47:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d3853700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc19db10 con 0x7fd0cc0683b0 2023-09-10T00:47:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d3853700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc19dcd0 con 0x7fd0cc0679f0 2023-09-10T00:47:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d1df0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc106650 0x7fd0cc19d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d0dee700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0cc0683b0 0x7fd0cc199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d15ef700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0cc0679f0 0x7fd0cc1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d1df0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc106650 0x7fd0cc19d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51344/0 (socket says 172.21.15.44:51344) 2023-09-10T00:47:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d0dee700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0cc0683b0 0x7fd0cc199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53082/0 (socket says 172.21.15.44:53082) 2023-09-10T00:47:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d1df0700 1 -- 172.21.15.44:0/3353750226 learned_addr learned my addr 172.21.15.44:0/3353750226 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d15ef700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0cc0679f0 0x7fd0cc1991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58222/0 (socket says 172.21.15.44:58222) 2023-09-10T00:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d1df0700 1 -- 172.21.15.44:0/3353750226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0cc0679f0 msgr2=0x7fd0cc1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d1df0700 1 --2- 172.21.15.44:0/3353750226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0cc0679f0 0x7fd0cc1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d1df0700 1 -- 172.21.15.44:0/3353750226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0cc0683b0 msgr2=0x7fd0cc199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d1df0700 1 --2- 172.21.15.44:0/3353750226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0cc0683b0 0x7fd0cc199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d1df0700 1 -- 172.21.15.44:0/3353750226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c80095f0 con 0x7fd0cc106650 2023-09-10T00:47:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d0dee700 1 --2- 172.21.15.44:0/3353750226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0cc0683b0 0x7fd0cc199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0d1df0700 1 --2- 172.21.15.44:0/3353750226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc106650 0x7fd0cc19d360 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400ef30 tx=0x7fd0c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.157+0000 7fd0c27fc700 1 -- 172.21.15.44:0/3353750226 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4018070 con 0x7fd0cc106650 2023-09-10T00:47:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.158+0000 7fd0d3853700 1 -- 172.21.15.44:0/3353750226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0cc19df50 con 0x7fd0cc106650 2023-09-10T00:47:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.158+0000 7fd0c27fc700 1 -- 172.21.15.44:0/3353750226 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0c400f040 con 0x7fd0cc106650 2023-09-10T00:47:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.158+0000 7fd0c27fc700 1 -- 172.21.15.44:0/3353750226 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c400cbc0 con 0x7fd0cc106650 2023-09-10T00:47:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.158+0000 7fd0d3853700 1 -- 172.21.15.44:0/3353750226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc1a57f0 con 0x7fd0cc106650 2023-09-10T00:47:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.161+0000 7fd0c27fc700 1 -- 172.21.15.44:0/3353750226 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fd0c4007500 con 0x7fd0cc106650 2023-09-10T00:47:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.161+0000 7fd0c27fc700 1 --2- 172.21.15.44:0/3353750226 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd0b80a07b0 0x7fd0b80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.161+0000 7fd0c27fc700 1 -- 172.21.15.44:0/3353750226 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fd0c40cbe40 con 0x7fd0cc106650 2023-09-10T00:47:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.161+0000 7fd0d15ef700 1 --2- 172.21.15.44:0/3353750226 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd0b80a07b0 0x7fd0b80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.162+0000 7fd0d15ef700 1 --2- 172.21.15.44:0/3353750226 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd0b80a07b0 0x7fd0b80a2c70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800b580 tx=0x7fd0c8005c00 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.162+0000 7fd0dfa19b80 1 Processor -- start 2023-09-10T00:47:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.162+0000 7fd0dfa19b80 1 -- start start 2023-09-10T00:47:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.162+0000 7fd0dfa19b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f6c1a0 0x560c97f6c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.162+0000 7fd0dfa19b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66300 0x560c97f66780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.162+0000 7fd0dfa19b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66cc0 0x560c97f72f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.162+0000 7fd0dfa19b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c97ebf6d0 con 0x560c97f6c1a0 2023-09-10T00:47:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.162+0000 7fd0dfa19b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c97ebcff0 con 0x560c97f66cc0 2023-09-10T00:47:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.162+0000 7fd0dfa19b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c97eb88e0 con 0x560c97f66300 2023-09-10T00:47:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.163+0000 7fd0d15ef700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f6c1a0 0x560c97f6c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.163+0000 7fd0d0dee700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66300 0x560c97f66780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.163+0000 7fd0d0dee700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66300 0x560c97f66780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58232/0 (socket says 172.21.15.44:58232) 2023-09-10T00:47:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.163+0000 7fd0d0dee700 1 -- 172.21.15.44:0/3434239658 learned_addr learned my addr 172.21.15.44:0/3434239658 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.163+0000 7fd0d1df0700 1 --2- 172.21.15.44:0/3434239658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66cc0 0x560c97f72f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.163+0000 7fd0d0dee700 1 -- 172.21.15.44:0/3434239658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66cc0 msgr2=0x560c97f72f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.163+0000 7fd0d0dee700 1 --2- 172.21.15.44:0/3434239658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66cc0 0x560c97f72f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.163+0000 7fd0d0dee700 1 -- 172.21.15.44:0/3434239658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f6c1a0 msgr2=0x560c97f6c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.163+0000 7fd0d0dee700 1 --2- 172.21.15.44:0/3434239658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f6c1a0 0x560c97f6c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.163+0000 7fd0d0dee700 1 -- 172.21.15.44:0/3434239658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c97ce4d80 con 0x560c97f66300 2023-09-10T00:47:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.163+0000 7fd0d0dee700 1 --2- 172.21.15.44:0/3434239658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66300 0x560c97f66780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc010580 tx=0x7fd0bc010890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8985bae52f4d8dcd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.163+0000 7fd0b77fe700 1 -- 172.21.15.44:0/3434239658 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc01c680 con 0x560c97f66300 2023-09-10T00:47:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.163+0000 7fd0b77fe700 1 -- 172.21.15.44:0/3434239658 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0bc015070 con 0x560c97f66300 2023-09-10T00:47:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.163+0000 7fd0b77fe700 1 -- 172.21.15.44:0/3434239658 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc011990 con 0x560c97f66300 2023-09-10T00:47:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.164+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/3434239658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66300 msgr2=0x560c97f66780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.164+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/3434239658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66300 0x560c97f66780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc010580 tx=0x7fd0bc010890 comp rx=0 tx=0).stop 2023-09-10T00:47:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.164+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/3434239658 shutdown_connections 2023-09-10T00:47:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.164+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/3434239658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66cc0 0x560c97f72f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.164+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/3434239658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f6c1a0 0x560c97f6c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.164+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/3434239658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66300 0x560c97f66780 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.164+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/3434239658 >> 172.21.15.44:0/3434239658 conn(0x560c97edb030 msgr2=0x560c97f61560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.164+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/3434239658 shutdown_connections 2023-09-10T00:47:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.164+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/3434239658 wait complete. 2023-09-10T00:47:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.165+0000 7fd0dfa19b80 1 -- start start 2023-09-10T00:47:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.165+0000 7fd0dfa19b80 1 Processor -- start 2023-09-10T00:47:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.165+0000 7fd0dfa19b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f66300 0x560c97f72e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.165+0000 7fd0dfa19b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66cc0 0x560c97f70130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.165+0000 7fd0dfa19b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f6c1a0 0x560c97f70760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.165+0000 7fd0dfa19b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c97e56b10 con 0x560c97f66300 2023-09-10T00:47:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.165+0000 7fd0dfa19b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c97e56810 con 0x560c97f66cc0 2023-09-10T00:47:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.165+0000 7fd0dfa19b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c97d3a8c0 con 0x560c97f6c1a0 2023-09-10T00:47:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.165+0000 7fd0dfa19b80 10 client.0 _ll_register_callbacks cb 0x560c97da0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:47:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.166+0000 7fd0d0dee700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66cc0 0x560c97f70130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.166+0000 7fd0d1df0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f6c1a0 0x560c97f70760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.166+0000 7fd0d1df0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f6c1a0 0x560c97f70760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58240/0 (socket says 172.21.15.44:58240) 2023-09-10T00:47:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.166+0000 7fd0d0dee700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66cc0 0x560c97f70130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53096/0 (socket says 172.21.15.44:53096) 2023-09-10T00:47:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.166+0000 7fd0d15ef700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f66300 0x560c97f72e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.166+0000 7fd0d1df0700 1 -- 172.21.15.44:0/2885215624 learned_addr learned my addr 172.21.15.44:0/2885215624 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.166+0000 7fd0d0dee700 1 -- 172.21.15.44:0/2885215624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f6c1a0 msgr2=0x560c97f70760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.166+0000 7fd0d0dee700 1 --2- 172.21.15.44:0/2885215624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f6c1a0 0x560c97f70760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.166+0000 7fd0d0dee700 1 -- 172.21.15.44:0/2885215624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f66300 msgr2=0x560c97f72e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.166+0000 7fd0d0dee700 1 --2- 172.21.15.44:0/2885215624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f66300 0x560c97f72e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.166+0000 7fd0d0dee700 1 -- 172.21.15.44:0/2885215624 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0bc00bd50 con 0x560c97f66cc0 2023-09-10T00:47:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.166+0000 7fd0d1df0700 1 --2- 172.21.15.44:0/2885215624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f6c1a0 0x560c97f70760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.166+0000 7fd0d0dee700 1 --2- 172.21.15.44:0/2885215624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66cc0 0x560c97f70130 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc010550 tx=0x7fd0bc01c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f33a0d8b03c4ddb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.166+0000 7fd0b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:47:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.166+0000 7fd0b4ff9700 1 -- 172.21.15.44:0/2885215624 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc01cdc0 con 0x560c97f66cc0 2023-09-10T00:47:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.167+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/2885215624 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c97e10490 con 0x560c97f66cc0 2023-09-10T00:47:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.167+0000 7fd0b4ff9700 1 -- 172.21.15.44:0/2885215624 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fd0bc022070 con 0x560c97f66cc0 2023-09-10T00:47:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.167+0000 7fd0b4ff9700 1 -- 172.21.15.44:0/2885215624 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0bc010eb0 con 0x560c97f66cc0 2023-09-10T00:47:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.167+0000 7fd0dfa19b80 10 client.26549 fetch_fsmap learned FSMap version 47 2023-09-10T00:47:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.167+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/2885215624 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x560c97ccdc00 con 0x560c97f66cc0 2023-09-10T00:47:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.167+0000 7fd0b4ff9700 1 -- 172.21.15.44:0/2885215624 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560c97e10490 con 0x560c97f66cc0 2023-09-10T00:47:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.167+0000 7fd0b4ff9700 1 -- 172.21.15.44:0/2885215624 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 47) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fd0bc026450 con 0x560c97f66cc0 2023-09-10T00:47:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.167+0000 7fd0dfa19b80 10 client.26549 fetch_fsmap finished waiting for FSMap version 47 2023-09-10T00:47:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.167+0000 7fd0dfa19b80 10 client.26549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:47:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.167+0000 7fd0dfa19b80 20 client.26549 populate_metadata read hostname 'smithi044' 2023-09-10T00:47:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.167+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/2885215624 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c97f80470 0x560c97f82890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.167+0000 7fd0dfa19b80 4 client.26549 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:47:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.167+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/2885215624 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c97ecd840 con 0x560c97f80470 2023-09-10T00:47:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.167+0000 7fd0d15ef700 1 --2- 172.21.15.44:0/2885215624 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c97f80470 0x560c97f82890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.168+0000 7fd0b4ff9700 10 client.26549 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.168+0000 7fd0d15ef700 1 --2- 172.21.15.44:0/2885215624 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c97f80470 0x560c97f82890 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ee2838036c5da83 server_cookie=8d013e24652b003b in_seq=0 out_seq=0 2023-09-10T00:47:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.168+0000 7fd0b4ff9700 1 -- 172.21.15.44:0/2885215624 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560c97ecd840 con 0x560c97f80470 2023-09-10T00:47:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.168+0000 7fd0b4ff9700 10 client.26549 handle_command_reply: tid=0 2023-09-10T00:47:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.168+0000 7fd0dfa19b80 1 client.26549 shutdown 2023-09-10T00:47:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.169+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/2885215624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66cc0 msgr2=0x560c97f70130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.169+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/2885215624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66cc0 0x560c97f70130 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc010550 tx=0x7fd0bc01c710 comp rx=0 tx=0).stop 2023-09-10T00:47:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.169+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/2885215624 shutdown_connections 2023-09-10T00:47:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.169+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/2885215624 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c97f80470 0x560c97f82890 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.169+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/2885215624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f6c1a0 0x560c97f70760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.169+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/2885215624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f66300 0x560c97f72e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.169+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/2885215624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66cc0 0x560c97f70130 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.169+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/2885215624 >> 172.21.15.44:0/2885215624 conn(0x560c97edb030 msgr2=0x560c97f607a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.169+0000 7fd0b4ff9700 0 client.26549 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.169+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/2885215624 shutdown_connections 2023-09-10T00:47:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.169+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/2885215624 wait complete. 2023-09-10T00:47:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.169+0000 7fd0dfa19b80 20 client.26549 trim_cache size 0 max 16384 2023-09-10T00:47:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.189+0000 7fd0dfa19b80 1 Processor -- start 2023-09-10T00:47:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.189+0000 7fd0dfa19b80 1 -- start start 2023-09-10T00:47:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.189+0000 7fd0dfa19b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66300 0x560c97f77520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.189+0000 7fd0dfa19b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66cc0 0x560c97f6b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.189+0000 7fd0dfa19b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f6c1a0 0x560c97f6a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.189+0000 7fd0dfa19b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c97f5b8a0 con 0x560c97f6c1a0 2023-09-10T00:47:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.189+0000 7fd0dfa19b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c97f6cb50 con 0x560c97f66300 2023-09-10T00:47:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.189+0000 7fd0dfa19b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c97f6be00 con 0x560c97f66cc0 2023-09-10T00:47:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.189+0000 7fd0d0dee700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66cc0 0x560c97f6b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.189+0000 7fd0d15ef700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66300 0x560c97f77520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.189+0000 7fd0d1df0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f6c1a0 0x560c97f6a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.189+0000 7fd0d0dee700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66cc0 0x560c97f6b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58246/0 (socket says 172.21.15.44:58246) 2023-09-10T00:47:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.189+0000 7fd0d0dee700 1 -- 172.21.15.44:0/1328456200 learned_addr learned my addr 172.21.15.44:0/1328456200 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.190+0000 7fd0d0dee700 1 -- 172.21.15.44:0/1328456200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66300 msgr2=0x560c97f77520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.190+0000 7fd0d0dee700 1 --2- 172.21.15.44:0/1328456200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66300 0x560c97f77520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.190+0000 7fd0d0dee700 1 -- 172.21.15.44:0/1328456200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f6c1a0 msgr2=0x560c97f6a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.190+0000 7fd0d0dee700 1 --2- 172.21.15.44:0/1328456200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f6c1a0 0x560c97f6a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.190+0000 7fd0d0dee700 1 -- 172.21.15.44:0/1328456200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c97d0b1f0 con 0x560c97f66cc0 2023-09-10T00:47:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.190+0000 7fd0d0dee700 1 --2- 172.21.15.44:0/1328456200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66cc0 0x560c97f6b1e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00c010 tx=0x7fd0bc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4241fbefeaffac4a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.190+0000 7fd0b7fff700 1 -- 172.21.15.44:0/1328456200 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc005950 con 0x560c97f66cc0 2023-09-10T00:47:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.190+0000 7fd0b7fff700 1 -- 172.21.15.44:0/1328456200 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0bc01c830 con 0x560c97f66cc0 2023-09-10T00:47:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.190+0000 7fd0b7fff700 1 -- 172.21.15.44:0/1328456200 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc005030 con 0x560c97f66cc0 2023-09-10T00:47:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.191+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/1328456200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66cc0 msgr2=0x560c97f6b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.191+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/1328456200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66cc0 0x560c97f6b1e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00c010 tx=0x7fd0bc000bc0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.191+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/1328456200 shutdown_connections 2023-09-10T00:47:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.191+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/1328456200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f6c1a0 0x560c97f6a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.191+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/1328456200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66300 0x560c97f77520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.191+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/1328456200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66cc0 0x560c97f6b1e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.191+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/1328456200 >> 172.21.15.44:0/1328456200 conn(0x560c97edb030 msgr2=0x560c97f7bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.191+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/1328456200 shutdown_connections 2023-09-10T00:47:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.191+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/1328456200 wait complete. 2023-09-10T00:47:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.191+0000 7fd0dfa19b80 1 -- start start 2023-09-10T00:47:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.192+0000 7fd0dfa19b80 1 Processor -- start 2023-09-10T00:47:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.192+0000 7fd0dfa19b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66300 0x560c97f77530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.192+0000 7fd0dfa19b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66cc0 0x560c97f60ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.192+0000 7fd0dfa19b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f6c1a0 0x560c97f611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.192+0000 7fd0dfa19b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c97f61820 con 0x560c97f6c1a0 2023-09-10T00:47:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.192+0000 7fd0dfa19b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c97f61990 con 0x560c97f66300 2023-09-10T00:47:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.192+0000 7fd0dfa19b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c97f6e960 con 0x560c97f66cc0 2023-09-10T00:47:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.192+0000 7fd0dfa19b80 10 client.0 _ll_register_callbacks cb 0x7fd0c800b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:47:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.192+0000 7fd0d0dee700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66cc0 0x560c97f60ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.192+0000 7fd0d0dee700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66cc0 0x560c97f60ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58250/0 (socket says 172.21.15.44:58250) 2023-09-10T00:47:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.192+0000 7fd0d0dee700 1 -- 172.21.15.44:0/1937380342 learned_addr learned my addr 172.21.15.44:0/1937380342 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.193+0000 7fd0d1df0700 1 --2- 172.21.15.44:0/1937380342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f6c1a0 0x560c97f611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.193+0000 7fd0d15ef700 1 --2- 172.21.15.44:0/1937380342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66300 0x560c97f77530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.193+0000 7fd0d0dee700 1 -- 172.21.15.44:0/1937380342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66300 msgr2=0x560c97f77530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.193+0000 7fd0d0dee700 1 --2- 172.21.15.44:0/1937380342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66300 0x560c97f77530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.193+0000 7fd0d0dee700 1 -- 172.21.15.44:0/1937380342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f6c1a0 msgr2=0x560c97f611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.193+0000 7fd0d0dee700 1 --2- 172.21.15.44:0/1937380342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f6c1a0 0x560c97f611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.193+0000 7fd0d0dee700 1 -- 172.21.15.44:0/1937380342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c97ccdc00 con 0x560c97f66cc0 2023-09-10T00:47:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.193+0000 7fd0d15ef700 1 --2- 172.21.15.44:0/1937380342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66300 0x560c97f77530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:47:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.193+0000 7fd0d0dee700 1 --2- 172.21.15.44:0/1937380342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66cc0 0x560c97f60ca0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc01c750 tx=0x7fd0bc00bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5443771de17625b4 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.193+0000 7fd0b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:47:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.193+0000 7fd0b4ff9700 1 -- 172.21.15.44:0/1937380342 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc03a070 con 0x560c97f66cc0 2023-09-10T00:47:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.193+0000 7fd0b4ff9700 1 -- 172.21.15.44:0/1937380342 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fd0bc022040 con 0x560c97f66cc0 2023-09-10T00:47:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.193+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/1937380342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c97eb18c0 con 0x560c97f66cc0 2023-09-10T00:47:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.193+0000 7fd0b4ff9700 1 -- 172.21.15.44:0/1937380342 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0bc01c830 con 0x560c97f66cc0 2023-09-10T00:47:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.193+0000 7fd0dfa19b80 10 client.26497 fetch_fsmap learned FSMap version 47 2023-09-10T00:47:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.193+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/1937380342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x560c97ce3250 con 0x560c97f66cc0 2023-09-10T00:47:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.194+0000 7fd0b4ff9700 1 -- 172.21.15.44:0/1937380342 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0bc048080 con 0x560c97f66cc0 2023-09-10T00:47:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.194+0000 7fd0b4ff9700 1 -- 172.21.15.44:0/1937380342 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 47) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fd0bc004a20 con 0x560c97f66cc0 2023-09-10T00:47:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.194+0000 7fd0dfa19b80 10 client.26497 fetch_fsmap finished waiting for FSMap version 47 2023-09-10T00:47:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.194+0000 7fd0dfa19b80 10 client.26497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:47:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.194+0000 7fd0dfa19b80 20 client.26497 populate_metadata read hostname 'smithi044' 2023-09-10T00:47:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.194+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/1937380342 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c97fd4480 0x560c97fd6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.194+0000 7fd0dfa19b80 4 client.26497 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:47:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.194+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/1937380342 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c97e72050 con 0x560c97fd4480 2023-09-10T00:47:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.194+0000 7fd0d15ef700 1 --2- 172.21.15.44:0/1937380342 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c97fd4480 0x560c97fd6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.194+0000 7fd0d15ef700 1 --2- 172.21.15.44:0/1937380342 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c97fd4480 0x560c97fd6870 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=634537c49d8d5a5f server_cookie=74f3d4b31454d4f3 in_seq=0 out_seq=0 2023-09-10T00:47:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.194+0000 7fd0b4ff9700 10 client.26497 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.207+0000 7fd0b4ff9700 1 -- 172.21.15.44:0/1937380342 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c97e72050 con 0x560c97fd4480 2023-09-10T00:47:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.207+0000 7fd0b4ff9700 10 client.26497 handle_command_reply: tid=0 2023-09-10T00:47:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.207+0000 7fd0dfa19b80 1 client.26497 shutdown 2023-09-10T00:47:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.207+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/1937380342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66cc0 msgr2=0x560c97f60ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.207+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/1937380342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66cc0 0x560c97f60ca0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc01c750 tx=0x7fd0bc00bcb0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.207+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/1937380342 shutdown_connections 2023-09-10T00:47:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.207+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/1937380342 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c97fd4480 0x560c97fd6870 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.207+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/1937380342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c97f6c1a0 0x560c97f611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.207+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/1937380342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c97f66300 0x560c97f77530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.207+0000 7fd0b4ff9700 0 client.26497 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.207+0000 7fd0dfa19b80 1 --2- 172.21.15.44:0/1937380342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c97f66cc0 0x560c97f60ca0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.207+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/1937380342 >> 172.21.15.44:0/1937380342 conn(0x560c97edb030 msgr2=0x560c97f6b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.208+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/1937380342 shutdown_connections 2023-09-10T00:47:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.208+0000 7fd0dfa19b80 1 -- 172.21.15.44:0/1937380342 wait complete. 2023-09-10T00:47:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.208+0000 7fd0dfa19b80 20 client.26497 trim_cache size 0 max 16384 2023-09-10T00:47:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:47:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:47:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:47:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:47:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.208+0000 7fd0d3853700 1 -- 172.21.15.44:0/3353750226 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd0b80a07b0 msgr2=0x7fd0b80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.208+0000 7fd0d3853700 1 --2- 172.21.15.44:0/3353750226 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd0b80a07b0 0x7fd0b80a2c70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800b580 tx=0x7fd0c8005c00 comp rx=0 tx=0).stop 2023-09-10T00:47:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.208+0000 7fd0d3853700 1 -- 172.21.15.44:0/3353750226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc106650 msgr2=0x7fd0cc19d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.208+0000 7fd0d3853700 1 --2- 172.21.15.44:0/3353750226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc106650 0x7fd0cc19d360 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400ef30 tx=0x7fd0c400c3a0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.209+0000 7fd0d3853700 1 -- 172.21.15.44:0/3353750226 shutdown_connections 2023-09-10T00:47:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.209+0000 7fd0d3853700 1 --2- 172.21.15.44:0/3353750226 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd0b80a07b0 0x7fd0b80a2c70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.209+0000 7fd0d3853700 1 --2- 172.21.15.44:0/3353750226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc106650 0x7fd0cc19d360 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.209+0000 7fd0d3853700 1 --2- 172.21.15.44:0/3353750226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0cc0679f0 0x7fd0cc1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.209+0000 7fd0d3853700 1 --2- 172.21.15.44:0/3353750226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0cc0683b0 0x7fd0cc199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.209+0000 7fd0d3853700 1 -- 172.21.15.44:0/3353750226 >> 172.21.15.44:0/3353750226 conn(0x7fd0cc074f30 msgr2=0x7fd0cc0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.209+0000 7fd0d3853700 1 -- 172.21.15.44:0/3353750226 shutdown_connections 2023-09-10T00:47:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:04.209+0000 7fd0d3853700 1 -- 172.21.15.44:0/3353750226 wait complete. 2023-09-10T00:47:04.550 INFO:teuthology.orchestra.run.smithi044.stderr:30500000 of 50000000 tuples (61%) done (elapsed 364.18 s, remaining 232.84 s) 2023-09-10T00:47:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:04 smithi131 ceph-mon[103713]: pgmap v195: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 8.1 MiB/s wr, 26 op/s 2023-09-10T00:47:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:04 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:05.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:04 smithi131 ceph-mon[103713]: scrub summary: active paths [/] 2023-09-10T00:47:05.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:04 smithi131 ceph-mon[103713]: scrub summary: idle+waiting paths [/] 2023-09-10T00:47:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:04 smithi044 ceph-mon[98676]: pgmap v195: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 8.1 MiB/s wr, 26 op/s 2023-09-10T00:47:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:04 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:04 smithi044 ceph-mon[98676]: scrub summary: active paths [/] 2023-09-10T00:47:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:04 smithi044 ceph-mon[98676]: scrub summary: idle+waiting paths [/] 2023-09-10T00:47:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:04 smithi099 ceph-mon[102692]: pgmap v195: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 8.1 MiB/s wr, 26 op/s 2023-09-10T00:47:05.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:04 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:05.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:04 smithi099 ceph-mon[102692]: scrub summary: active paths [/] 2023-09-10T00:47:05.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:04 smithi099 ceph-mon[102692]: scrub summary: idle+waiting paths [/] 2023-09-10T00:47:07.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:06 smithi131 ceph-mon[103713]: pgmap v196: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 12 MiB/s wr, 63 op/s 2023-09-10T00:47:07.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:06 smithi044 ceph-mon[98676]: pgmap v196: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 12 MiB/s wr, 63 op/s 2023-09-10T00:47:07.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:06 smithi099 ceph-mon[102692]: pgmap v196: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 12 MiB/s wr, 63 op/s 2023-09-10T00:47:07.551 INFO:teuthology.orchestra.run.smithi044.stderr:30600000 of 50000000 tuples (61%) done (elapsed 367.19 s, remaining 232.79 s) 2023-09-10T00:47:08.540 INFO:teuthology.orchestra.run.smithi044.stderr:30700000 of 50000000 tuples (61%) done (elapsed 368.17 s, remaining 231.46 s) 2023-09-10T00:47:08.660 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:47:08.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.833+0000 7f70acab3700 1 -- 172.21.15.44:0/4203335017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a8100f70 msgr2=0x7f70a810d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:08.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.833+0000 7f70acab3700 1 --2- 172.21.15.44:0/4203335017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a8100f70 0x7f70a810d430 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f70a000b0d0 tx=0x7f70a000b3e0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.833+0000 7f70acab3700 1 -- 172.21.15.44:0/4203335017 shutdown_connections 2023-09-10T00:47:08.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.833+0000 7f70acab3700 1 --2- 172.21.15.44:0/4203335017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a8100f70 0x7f70a810d430 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.833+0000 7f70acab3700 1 --2- 172.21.15.44:0/4203335017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70a8106670 0x7f70a8106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.833+0000 7f70acab3700 1 --2- 172.21.15.44:0/4203335017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70a81005b0 0x7f70a8100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.833+0000 7f70acab3700 1 -- 172.21.15.44:0/4203335017 >> 172.21.15.44:0/4203335017 conn(0x7f70a80fc0a0 msgr2=0x7f70a80fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:08.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.833+0000 7f70acab3700 1 -- 172.21.15.44:0/4203335017 shutdown_connections 2023-09-10T00:47:08.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.833+0000 7f70acab3700 1 -- 172.21.15.44:0/4203335017 wait complete. 2023-09-10T00:47:08.837 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.834+0000 7f70acab3700 1 Processor -- start 2023-09-10T00:47:08.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.834+0000 7f70acab3700 1 -- start start 2023-09-10T00:47:08.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.834+0000 7f70acab3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a81005b0 0x7f70a819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.834+0000 7f70acab3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70a8100f70 0x7f70a819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.834+0000 7f70acab3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70a8106670 0x7f70a8198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.838 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.834+0000 7f70acab3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a8198cd0 con 0x7f70a81005b0 2023-09-10T00:47:08.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.834+0000 7f70acab3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a8198e40 con 0x7f70a8100f70 2023-09-10T00:47:08.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.834+0000 7f70acab3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a8199000 con 0x7f70a8106670 2023-09-10T00:47:08.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.835+0000 7f70a659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a81005b0 0x7f70a819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.835+0000 7f70a5d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70a8100f70 0x7f70a819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.835+0000 7f70a6d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70a8106670 0x7f70a8198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.835+0000 7f70a659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a81005b0 0x7f70a819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51382/0 (socket says 172.21.15.44:51382) 2023-09-10T00:47:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.835+0000 7f70a659c700 1 -- 172.21.15.44:0/1651646630 learned_addr learned my addr 172.21.15.44:0/1651646630 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.835+0000 7f70a6d9d700 1 -- 172.21.15.44:0/1651646630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70a8100f70 msgr2=0x7f70a819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.835+0000 7f70a6d9d700 1 --2- 172.21.15.44:0/1651646630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70a8100f70 0x7f70a819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.835+0000 7f70a6d9d700 1 -- 172.21.15.44:0/1651646630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a81005b0 msgr2=0x7f70a819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.835+0000 7f70a6d9d700 1 --2- 172.21.15.44:0/1651646630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a81005b0 0x7f70a819e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.835+0000 7f70a6d9d700 1 -- 172.21.15.44:0/1651646630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7098009520 con 0x7f70a8106670 2023-09-10T00:47:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.835+0000 7f70a659c700 1 --2- 172.21.15.44:0/1651646630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a81005b0 0x7f70a819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.835+0000 7f70a5d9b700 1 --2- 172.21.15.44:0/1651646630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70a8100f70 0x7f70a819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.835+0000 7f70a6d9d700 1 --2- 172.21.15.44:0/1651646630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70a8106670 0x7f70a8198660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f70a000b770 tx=0x7f70a0008e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.835+0000 7f70977fe700 1 -- 172.21.15.44:0/1651646630 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a000e040 con 0x7f70a8106670 2023-09-10T00:47:08.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.835+0000 7f70977fe700 1 -- 172.21.15.44:0/1651646630 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70a001d900 con 0x7f70a8106670 2023-09-10T00:47:08.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.835+0000 7f70acab3700 1 -- 172.21.15.44:0/1651646630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a0009d00 con 0x7f70a8106670 2023-09-10T00:47:08.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.836+0000 7f70977fe700 1 -- 172.21.15.44:0/1651646630 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a001b7e0 con 0x7f70a8106670 2023-09-10T00:47:08.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.836+0000 7f70acab3700 1 -- 172.21.15.44:0/1651646630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a81a5400 con 0x7f70a8106670 2023-09-10T00:47:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.839+0000 7f70977fe700 1 -- 172.21.15.44:0/1651646630 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f70a001da70 con 0x7f70a8106670 2023-09-10T00:47:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.839+0000 7f70977fe700 1 --2- 172.21.15.44:0/1651646630 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f70900a07b0 0x7f70900a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.839+0000 7f70977fe700 1 -- 172.21.15.44:0/1651646630 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f70a00cd660 con 0x7f70a8106670 2023-09-10T00:47:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.839+0000 7f70a659c700 1 --2- 172.21.15.44:0/1651646630 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f70900a07b0 0x7f70900a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.840+0000 7f70b8c79b80 1 Processor -- start 2023-09-10T00:47:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.840+0000 7f70a659c700 1 --2- 172.21.15.44:0/1651646630 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f70900a07b0 0x7f70900a2c70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7098009fd0 tx=0x7f7098009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.840+0000 7f70b8c79b80 1 -- start start 2023-09-10T00:47:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.840+0000 7f70b8c79b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84bd8710 0x557c84bd8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.840+0000 7f70b8c79b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.840+0000 7f70b8c79b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84be2e40 0x557c84be5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.840+0000 7f70b8c79b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c84b31730 con 0x557c84bd8710 2023-09-10T00:47:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.840+0000 7f70b8c79b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c84b2f050 con 0x557c84be2e40 2023-09-10T00:47:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.840+0000 7f70b8c79b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c84b2a940 con 0x557c84bd9030 2023-09-10T00:47:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.840+0000 7f70a659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84bd8710 0x557c84bd8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.841+0000 7f70a5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.841+0000 7f70a6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84be2e40 0x557c84be5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.841+0000 7f70a5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58274/0 (socket says 172.21.15.44:58274) 2023-09-10T00:47:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.841+0000 7f70a5d9b700 1 -- 172.21.15.44:0/2191673309 learned_addr learned my addr 172.21.15.44:0/2191673309 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.841+0000 7f70a6d9d700 1 -- 172.21.15.44:0/2191673309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 msgr2=0x557c84be2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.841+0000 7f70a6d9d700 1 --2- 172.21.15.44:0/2191673309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.841+0000 7f70a6d9d700 1 -- 172.21.15.44:0/2191673309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84bd8710 msgr2=0x557c84bd8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.841+0000 7f70a6d9d700 1 --2- 172.21.15.44:0/2191673309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84bd8710 0x557c84bd8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.841+0000 7f70a6d9d700 1 -- 172.21.15.44:0/2191673309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c84956de0 con 0x557c84be2e40 2023-09-10T00:47:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.841+0000 7f70a6d9d700 1 --2- 172.21.15.44:0/2191673309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84be2e40 0x557c84be5230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f70a00ca960 tx=0x7f70a009a310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21be4d4c2290e1e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.841+0000 7f7094ff9700 1 -- 172.21.15.44:0/2191673309 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a009a750 con 0x557c84be2e40 2023-09-10T00:47:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.841+0000 7f7094ff9700 1 -- 172.21.15.44:0/2191673309 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70a009e580 con 0x557c84be2e40 2023-09-10T00:47:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.841+0000 7f7094ff9700 1 -- 172.21.15.44:0/2191673309 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a009e360 con 0x557c84be2e40 2023-09-10T00:47:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.842+0000 7f70b8c79b80 1 -- 172.21.15.44:0/2191673309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84be2e40 msgr2=0x557c84be5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.842+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/2191673309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84be2e40 0x557c84be5230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f70a00ca960 tx=0x7f70a009a310 comp rx=0 tx=0).stop 2023-09-10T00:47:08.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.842+0000 7f70b8c79b80 1 -- 172.21.15.44:0/2191673309 shutdown_connections 2023-09-10T00:47:08.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.842+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/2191673309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84be2e40 0x557c84be5230 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.842+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/2191673309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84bd8710 0x557c84bd8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.842+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/2191673309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.842+0000 7f70b8c79b80 1 -- 172.21.15.44:0/2191673309 >> 172.21.15.44:0/2191673309 conn(0x557c84b4d090 msgr2=0x557c84bd3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:08.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.842+0000 7f70b8c79b80 1 -- 172.21.15.44:0/2191673309 shutdown_connections 2023-09-10T00:47:08.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.842+0000 7f70b8c79b80 1 -- 172.21.15.44:0/2191673309 wait complete. 2023-09-10T00:47:08.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.842+0000 7f70b8c79b80 1 -- start start 2023-09-10T00:47:08.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.843+0000 7f70b8c79b80 1 Processor -- start 2023-09-10T00:47:08.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.843+0000 7f70b8c79b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd8710 0x557c84bdfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.843+0000 7f70b8c79b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.843+0000 7f70b8c79b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 0x557c84be05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.843+0000 7f70b8c79b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c84ac8b70 con 0x557c84be2e40 2023-09-10T00:47:08.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.843+0000 7f70b8c79b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c84ac8870 con 0x557c84bd8710 2023-09-10T00:47:08.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.843+0000 7f70b8c79b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c849ac920 con 0x557c84bd9030 2023-09-10T00:47:08.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.843+0000 7f70b8c79b80 10 client.0 _ll_register_callbacks cb 0x557c84a12a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:47:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.843+0000 7f70a5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.843+0000 7f70a659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd8710 0x557c84bdfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.843+0000 7f70a5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58280/0 (socket says 172.21.15.44:58280) 2023-09-10T00:47:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.843+0000 7f70a659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd8710 0x557c84bdfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53136/0 (socket says 172.21.15.44:53136) 2023-09-10T00:47:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.843+0000 7f70a6d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 0x557c84be05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.843+0000 7f70a5d9b700 1 -- 172.21.15.44:0/3277883930 learned_addr learned my addr 172.21.15.44:0/3277883930 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:08.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.844+0000 7f70a5d9b700 1 -- 172.21.15.44:0/3277883930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd8710 msgr2=0x557c84bdfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:08.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.844+0000 7f70a5d9b700 1 --2- 172.21.15.44:0/3277883930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd8710 0x557c84bdfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.844+0000 7f70a5d9b700 1 -- 172.21.15.44:0/3277883930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 msgr2=0x557c84be05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:08.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.844+0000 7f70a5d9b700 1 --2- 172.21.15.44:0/3277883930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 0x557c84be05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.844+0000 7f70a5d9b700 1 -- 172.21.15.44:0/3277883930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a81a5400 con 0x557c84bd9030 2023-09-10T00:47:08.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.844+0000 7f70a6d9d700 1 --2- 172.21.15.44:0/3277883930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 0x557c84be05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:08.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.844+0000 7f70a659c700 1 --2- 172.21.15.44:0/3277883930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd8710 0x557c84bdfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:08.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.844+0000 7f70a5d9b700 1 --2- 172.21.15.44:0/3277883930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be0080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f709c012d20 tx=0x7f709c00f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9efef1aeeb60d838 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:08.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.844+0000 7f708dffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:47:08.859 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.844+0000 7f708dffb700 1 -- 172.21.15.44:0/3277883930 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c017070 con 0x557c84bd9030 2023-09-10T00:47:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.844+0000 7f70b8c79b80 1 -- 172.21.15.44:0/3277883930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c84a824f0 con 0x557c84bd9030 2023-09-10T00:47:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.844+0000 7f708dffb700 1 -- 172.21.15.44:0/3277883930 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f709c00fb80 con 0x557c84bd9030 2023-09-10T00:47:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.844+0000 7f708dffb700 1 -- 172.21.15.44:0/3277883930 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f709c00fd80 con 0x557c84bd9030 2023-09-10T00:47:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.844+0000 7f70b8c79b80 10 client.26509 fetch_fsmap learned FSMap version 48 2023-09-10T00:47:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.844+0000 7f70b8c79b80 1 -- 172.21.15.44:0/3277883930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x557c8493fc60 con 0x557c84bd9030 2023-09-10T00:47:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.844+0000 7f708dffb700 1 -- 172.21.15.44:0/3277883930 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557c84a824f0 con 0x557c84bd9030 2023-09-10T00:47:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.845+0000 7f708dffb700 1 -- 172.21.15.44:0/3277883930 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 48) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f709c010430 con 0x557c84bd9030 2023-09-10T00:47:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.845+0000 7f70b8c79b80 10 client.26509 fetch_fsmap finished waiting for FSMap version 48 2023-09-10T00:47:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.845+0000 7f70b8c79b80 10 client.26509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:47:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.845+0000 7f70b8c79b80 20 client.26509 populate_metadata read hostname 'smithi044' 2023-09-10T00:47:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.845+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/3277883930 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557c84bf2880 0x557c84bf4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.845+0000 7f70b8c79b80 4 client.26509 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:47:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.845+0000 7f70b8c79b80 1 -- 172.21.15.44:0/3277883930 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c84b3f8a0 con 0x557c84bf2880 2023-09-10T00:47:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.845+0000 7f70a659c700 1 --2- 172.21.15.44:0/3277883930 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557c84bf2880 0x557c84bf4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.845+0000 7f70a659c700 1 --2- 172.21.15.44:0/3277883930 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557c84bf2880 0x557c84bf4ca0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2df8b6b18f625643 server_cookie=6bb671e6ff07d881 in_seq=0 out_seq=0 2023-09-10T00:47:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.845+0000 7f708dffb700 10 client.26509 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:08.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.846+0000 7f708dffb700 1 -- 172.21.15.44:0/3277883930 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557c84b3f8a0 con 0x557c84bf2880 2023-09-10T00:47:08.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.846+0000 7f708dffb700 10 client.26509 handle_command_reply: tid=0 2023-09-10T00:47:08.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.846+0000 7f70b8c79b80 1 client.26509 shutdown 2023-09-10T00:47:08.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.846+0000 7f70b8c79b80 1 -- 172.21.15.44:0/3277883930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 msgr2=0x557c84be0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:08.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.846+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/3277883930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be0080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f709c012d20 tx=0x7f709c00f6a0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.863 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.846+0000 7f70b8c79b80 1 -- 172.21.15.44:0/3277883930 shutdown_connections 2023-09-10T00:47:08.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.846+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/3277883930 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557c84bf2880 0x557c84bf4ca0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.846+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/3277883930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 0x557c84be05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.846+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/3277883930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd8710 0x557c84bdfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.846+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/3277883930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be0080 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.864 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.846+0000 7f708dffb700 0 client.26509 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:08.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.846+0000 7f70b8c79b80 1 -- 172.21.15.44:0/3277883930 >> 172.21.15.44:0/3277883930 conn(0x557c84b4d090 msgr2=0x557c84bd37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:08.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.846+0000 7f70b8c79b80 1 -- 172.21.15.44:0/3277883930 shutdown_connections 2023-09-10T00:47:08.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.846+0000 7f70b8c79b80 1 -- 172.21.15.44:0/3277883930 wait complete. 2023-09-10T00:47:08.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.846+0000 7f70b8c79b80 20 client.26509 trim_cache size 0 max 16384 2023-09-10T00:47:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.865+0000 7f70b8c79b80 1 Processor -- start 2023-09-10T00:47:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.865+0000 7f70b8c79b80 1 -- start start 2023-09-10T00:47:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.865+0000 7f70b8c79b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd8710 0x557c84bdf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.865+0000 7f70b8c79b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd9030 0x557c84bd7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.865+0000 7f70b8c79b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 0x557c84bdcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.865+0000 7f70b8c79b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c84bd29a0 con 0x557c84be2e40 2023-09-10T00:47:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.865+0000 7f70b8c79b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c84bf5300 con 0x557c84bd9030 2023-09-10T00:47:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.865+0000 7f70b8c79b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c84bde0a0 con 0x557c84bd8710 2023-09-10T00:47:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.865+0000 7f70a6d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 0x557c84bdcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.865+0000 7f70a5d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd9030 0x557c84bd7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.866+0000 7f70a5d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd9030 0x557c84bd7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53146/0 (socket says 172.21.15.44:53146) 2023-09-10T00:47:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.866+0000 7f70a5d9b700 1 -- 172.21.15.44:0/1672365296 learned_addr learned my addr 172.21.15.44:0/1672365296 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.866+0000 7f70a659c700 1 --2- 172.21.15.44:0/1672365296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd8710 0x557c84bdf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.866+0000 7f70a5d9b700 1 -- 172.21.15.44:0/1672365296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd8710 msgr2=0x557c84bdf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.866+0000 7f70a5d9b700 1 --2- 172.21.15.44:0/1672365296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd8710 0x557c84bdf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.866+0000 7f70a5d9b700 1 -- 172.21.15.44:0/1672365296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 msgr2=0x557c84bdcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.866+0000 7f70a5d9b700 1 --2- 172.21.15.44:0/1672365296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 0x557c84bdcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.866+0000 7f70a5d9b700 1 -- 172.21.15.44:0/1672365296 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c8497d250 con 0x557c84bd9030 2023-09-10T00:47:08.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.866+0000 7f70a5d9b700 1 --2- 172.21.15.44:0/1672365296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd9030 0x557c84bd7e10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f709c009340 tx=0x7f709c009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0f608538d6468b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:08.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.866+0000 7f70957fa700 1 -- 172.21.15.44:0/1672365296 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c017030 con 0x557c84bd9030 2023-09-10T00:47:08.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.866+0000 7f70957fa700 1 -- 172.21.15.44:0/1672365296 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f709c018040 con 0x557c84bd9030 2023-09-10T00:47:08.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.866+0000 7f70957fa700 1 -- 172.21.15.44:0/1672365296 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c005620 con 0x557c84bd9030 2023-09-10T00:47:08.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.867+0000 7f70b8c79b80 1 -- 172.21.15.44:0/1672365296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd9030 msgr2=0x557c84bd7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.867+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/1672365296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd9030 0x557c84bd7e10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f709c009340 tx=0x7f709c009ea0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.867+0000 7f70b8c79b80 1 -- 172.21.15.44:0/1672365296 shutdown_connections 2023-09-10T00:47:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.867+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/1672365296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 0x557c84bdcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.867+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/1672365296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd8710 0x557c84bdf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.867+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/1672365296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd9030 0x557c84bd7e10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.867+0000 7f70b8c79b80 1 -- 172.21.15.44:0/1672365296 >> 172.21.15.44:0/1672365296 conn(0x557c84b4d090 msgr2=0x557c84bd7700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:08.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.867+0000 7f70b8c79b80 1 -- 172.21.15.44:0/1672365296 shutdown_connections 2023-09-10T00:47:08.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.867+0000 7f70b8c79b80 1 -- 172.21.15.44:0/1672365296 wait complete. 2023-09-10T00:47:08.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.867+0000 7f70b8c79b80 1 -- start start 2023-09-10T00:47:08.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.867+0000 7f70b8c79b80 1 Processor -- start 2023-09-10T00:47:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70b8c79b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd8710 0x557c84be9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70b8c79b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70b8c79b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 0x557c84be7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70b8c79b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c84be83e0 con 0x557c84be2e40 2023-09-10T00:47:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70b8c79b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c84be8550 con 0x557c84bd8710 2023-09-10T00:47:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70b8c79b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c84be8710 con 0x557c84bd9030 2023-09-10T00:47:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70b8c79b80 10 client.0 _ll_register_callbacks cb 0x7f709c0127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:47:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70a5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70a659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd8710 0x557c84be9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70a6d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 0x557c84be7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70a659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd8710 0x557c84be9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53150/0 (socket says 172.21.15.44:53150) 2023-09-10T00:47:08.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70a5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58294/0 (socket says 172.21.15.44:58294) 2023-09-10T00:47:08.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70a659c700 1 -- 172.21.15.44:0/3293908143 learned_addr learned my addr 172.21.15.44:0/3293908143 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:08.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70a659c700 1 -- 172.21.15.44:0/3293908143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 msgr2=0x557c84be7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:08.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70a659c700 1 --2- 172.21.15.44:0/3293908143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70a659c700 1 -- 172.21.15.44:0/3293908143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 msgr2=0x557c84be7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:08.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70a659c700 1 --2- 172.21.15.44:0/3293908143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 0x557c84be7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70a659c700 1 -- 172.21.15.44:0/3293908143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c8493fc60 con 0x557c84bd8710 2023-09-10T00:47:08.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70a6d9d700 1 --2- 172.21.15.44:0/3293908143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 0x557c84be7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:08.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70a5d9b700 1 --2- 172.21.15.44:0/3293908143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:08.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f70a659c700 1 --2- 172.21.15.44:0/3293908143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd8710 0x557c84be9630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7098013010 tx=0x7f7098013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b34c88167dc59f1c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:08.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.868+0000 7f708dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:47:08.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.869+0000 7f708dffb700 1 -- 172.21.15.44:0/3293908143 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098021040 con 0x557c84bd8710 2023-09-10T00:47:08.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.869+0000 7f70b8c79b80 1 -- 172.21.15.44:0/3293908143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c84bef170 con 0x557c84bd8710 2023-09-10T00:47:08.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.869+0000 7f708dffb700 1 -- 172.21.15.44:0/3293908143 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f709802b990 con 0x557c84bd8710 2023-09-10T00:47:08.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.869+0000 7f708dffb700 1 -- 172.21.15.44:0/3293908143 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f709802bb50 con 0x557c84bd8710 2023-09-10T00:47:08.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.869+0000 7f70b8c79b80 10 client.26570 fetch_fsmap learned FSMap version 48 2023-09-10T00:47:08.881 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.869+0000 7f70b8c79b80 1 -- 172.21.15.44:0/3293908143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x557c849552b0 con 0x557c84bd8710 2023-09-10T00:47:08.882 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.869+0000 7f708dffb700 1 -- 172.21.15.44:0/3293908143 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557c84bef170 con 0x557c84bd8710 2023-09-10T00:47:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.869+0000 7f708dffb700 1 -- 172.21.15.44:0/3293908143 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 48) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f709801b400 con 0x557c84bd8710 2023-09-10T00:47:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.869+0000 7f70b8c79b80 10 client.26570 fetch_fsmap finished waiting for FSMap version 48 2023-09-10T00:47:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.869+0000 7f70b8c79b80 10 client.26570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:47:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.869+0000 7f70b8c79b80 20 client.26570 populate_metadata read hostname 'smithi044' 2023-09-10T00:47:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.869+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/3293908143 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557c84c47c00 0x557c84c4a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.869+0000 7f70b8c79b80 4 client.26570 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:47:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.869+0000 7f70b8c79b80 1 -- 172.21.15.44:0/3293908143 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f70a009e580 con 0x557c84c47c00 2023-09-10T00:47:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.870+0000 7f70a5d9b700 1 --2- 172.21.15.44:0/3293908143 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557c84c47c00 0x557c84c4a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.871+0000 7f708dffb700 10 client.26570 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.871+0000 7f70a5d9b700 1 --2- 172.21.15.44:0/3293908143 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557c84c47c00 0x557c84c4a020 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bca242d98f1fafb3 server_cookie=b17d56a27192f5b9 in_seq=0 out_seq=0 2023-09-10T00:47:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.871+0000 7f708dffb700 1 -- 172.21.15.44:0/3293908143 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f70a009e580 con 0x557c84c47c00 2023-09-10T00:47:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.871+0000 7f708dffb700 10 client.26570 handle_command_reply: tid=0 2023-09-10T00:47:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.872+0000 7f70b8c79b80 1 client.26570 shutdown 2023-09-10T00:47:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.872+0000 7f70b8c79b80 1 -- 172.21.15.44:0/3293908143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd8710 msgr2=0x557c84be9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.872+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/3293908143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd8710 0x557c84be9630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7098013010 tx=0x7f7098013040 comp rx=0 tx=0).stop 2023-09-10T00:47:08.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.872+0000 7f70b8c79b80 1 -- 172.21.15.44:0/3293908143 shutdown_connections 2023-09-10T00:47:08.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.872+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/3293908143 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557c84c47c00 0x557c84c4a020 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.872+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/3293908143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c84be2e40 0x557c84be7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.872+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/3293908143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c84bd8710 0x557c84be9630 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.872+0000 7f70b8c79b80 1 --2- 172.21.15.44:0/3293908143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c84bd9030 0x557c84be7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.872+0000 7f708dffb700 0 client.26570 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.872+0000 7f70b8c79b80 1 -- 172.21.15.44:0/3293908143 >> 172.21.15.44:0/3293908143 conn(0x557c84b4d090 msgr2=0x557c84bd7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.872+0000 7f70b8c79b80 1 -- 172.21.15.44:0/3293908143 shutdown_connections 2023-09-10T00:47:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.872+0000 7f70b8c79b80 1 -- 172.21.15.44:0/3293908143 wait complete. 2023-09-10T00:47:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.872+0000 7f70b8c79b80 20 client.26570 trim_cache size 0 max 16384 2023-09-10T00:47:08.888 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:47:08.888 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:47:08.888 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:47:08.889 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:47:08.889 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:47:08.889 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:47:08.889 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:47:08.889 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:47:08.890 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:47:08.890 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:47:08.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.873+0000 7f70acab3700 1 -- 172.21.15.44:0/1651646630 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f70900a07b0 msgr2=0x7f70900a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:08.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.873+0000 7f70acab3700 1 --2- 172.21.15.44:0/1651646630 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f70900a07b0 0x7f70900a2c70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7098009fd0 tx=0x7f7098009260 comp rx=0 tx=0).stop 2023-09-10T00:47:08.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.873+0000 7f70acab3700 1 -- 172.21.15.44:0/1651646630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70a8106670 msgr2=0x7f70a8198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:08.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.873+0000 7f70acab3700 1 --2- 172.21.15.44:0/1651646630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70a8106670 0x7f70a8198660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f70a000b770 tx=0x7f70a0008e50 comp rx=0 tx=0).stop 2023-09-10T00:47:08.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.873+0000 7f70acab3700 1 -- 172.21.15.44:0/1651646630 shutdown_connections 2023-09-10T00:47:08.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.873+0000 7f70acab3700 1 --2- 172.21.15.44:0/1651646630 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f70900a07b0 0x7f70900a2c70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.873+0000 7f70acab3700 1 --2- 172.21.15.44:0/1651646630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70a8106670 0x7f70a8198660 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.873+0000 7f70acab3700 1 --2- 172.21.15.44:0/1651646630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a81005b0 0x7f70a819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.873+0000 7f70acab3700 1 --2- 172.21.15.44:0/1651646630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70a8100f70 0x7f70a819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:08.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.873+0000 7f70acab3700 1 -- 172.21.15.44:0/1651646630 >> 172.21.15.44:0/1651646630 conn(0x7f70a80fc0a0 msgr2=0x7f70a80fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:08.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.873+0000 7f70acab3700 1 -- 172.21.15.44:0/1651646630 shutdown_connections 2023-09-10T00:47:08.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:08.873+0000 7f70acab3700 1 -- 172.21.15.44:0/1651646630 wait complete. 2023-09-10T00:47:08.894 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:47:09.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:08 smithi131 ceph-mon[103713]: pgmap v197: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.1 MiB/s wr, 46 op/s 2023-09-10T00:47:09.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:08 smithi044 ceph-mon[98676]: pgmap v197: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.1 MiB/s wr, 46 op/s 2023-09-10T00:47:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:08 smithi099 ceph-mon[102692]: pgmap v197: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.1 MiB/s wr, 46 op/s 2023-09-10T00:47:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:30800000 of 50000000 tuples (61%) done (elapsed 369.25 s, remaining 230.18 s) 2023-09-10T00:47:10.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:09 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:10.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:09 smithi131 ceph-mon[103713]: osdmap e93: 12 total, 12 up, 12 in 2023-09-10T00:47:10.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:09 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:10.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:09 smithi044 ceph-mon[98676]: osdmap e93: 12 total, 12 up, 12 in 2023-09-10T00:47:10.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:09 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:10.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:09 smithi099 ceph-mon[102692]: osdmap e93: 12 total, 12 up, 12 in 2023-09-10T00:47:10.656 INFO:teuthology.orchestra.run.smithi044.stderr:30900000 of 50000000 tuples (61%) done (elapsed 370.29 s, remaining 228.88 s) 2023-09-10T00:47:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:10 smithi131 ceph-mon[103713]: pgmap v199: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 8.6 MiB/s wr, 56 op/s 2023-09-10T00:47:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:10 smithi044 ceph-mon[98676]: pgmap v199: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 8.6 MiB/s wr, 56 op/s 2023-09-10T00:47:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:10 smithi099 ceph-mon[102692]: pgmap v199: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 8.6 MiB/s wr, 56 op/s 2023-09-10T00:47:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:12 smithi131 ceph-mon[103713]: pgmap v200: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 16 MiB/s wr, 89 op/s 2023-09-10T00:47:13.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:12 smithi044 ceph-mon[98676]: pgmap v200: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 16 MiB/s wr, 89 op/s 2023-09-10T00:47:13.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:12 smithi099 ceph-mon[102692]: pgmap v200: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 16 MiB/s wr, 89 op/s 2023-09-10T00:47:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:14 smithi131 ceph-mon[103713]: pgmap v201: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 16 MiB/s wr, 89 op/s 2023-09-10T00:47:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:14 smithi044 ceph-mon[98676]: pgmap v201: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 16 MiB/s wr, 89 op/s 2023-09-10T00:47:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:14 smithi099 ceph-mon[102692]: pgmap v201: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 16 MiB/s wr, 89 op/s 2023-09-10T00:47:15.410 INFO:teuthology.orchestra.run.smithi044.stderr:31000000 of 50000000 tuples (62%) done (elapsed 375.04 s, remaining 229.87 s) 2023-09-10T00:47:16.270 INFO:teuthology.orchestra.run.smithi044.stderr:31100000 of 50000000 tuples (62%) done (elapsed 375.90 s, remaining 228.44 s) 2023-09-10T00:47:16.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:16 smithi099 ceph-mon[102692]: pgmap v202: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 15 MiB/s wr, 61 op/s 2023-09-10T00:47:16.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:16 smithi099 ceph-mon[102692]: osdmap e94: 12 total, 12 up, 12 in 2023-09-10T00:47:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:16 smithi131 ceph-mon[103713]: pgmap v202: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 15 MiB/s wr, 61 op/s 2023-09-10T00:47:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:16 smithi131 ceph-mon[103713]: osdmap e94: 12 total, 12 up, 12 in 2023-09-10T00:47:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:16 smithi044 ceph-mon[98676]: pgmap v202: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 15 MiB/s wr, 61 op/s 2023-09-10T00:47:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:16 smithi044 ceph-mon[98676]: osdmap e94: 12 total, 12 up, 12 in 2023-09-10T00:47:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.343+0000 7f43d8f37700 1 -- 172.21.15.44:0/1139657641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d4100f60 msgr2=0x7f43d410d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.343+0000 7f43d8f37700 1 --2- 172.21.15.44:0/1139657641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d4100f60 0x7f43d410d420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00b0d0 tx=0x7f43cc00b3e0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.344+0000 7f43d8f37700 1 -- 172.21.15.44:0/1139657641 shutdown_connections 2023-09-10T00:47:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.344+0000 7f43d8f37700 1 --2- 172.21.15.44:0/1139657641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d4100f60 0x7f43d410d420 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.344+0000 7f43d8f37700 1 --2- 172.21.15.44:0/1139657641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43d4106660 0x7f43d4106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.344+0000 7f43d8f37700 1 --2- 172.21.15.44:0/1139657641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d41005a0 0x7f43d4100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.344+0000 7f43d8f37700 1 -- 172.21.15.44:0/1139657641 >> 172.21.15.44:0/1139657641 conn(0x7f43d40fc0b0 msgr2=0x7f43d40fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.344+0000 7f43d8f37700 1 -- 172.21.15.44:0/1139657641 shutdown_connections 2023-09-10T00:47:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.344+0000 7f43d8f37700 1 -- 172.21.15.44:0/1139657641 wait complete. 2023-09-10T00:47:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.345+0000 7f43d8f37700 1 Processor -- start 2023-09-10T00:47:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.345+0000 7f43d8f37700 1 -- start start 2023-09-10T00:47:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.345+0000 7f43d8f37700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d41005a0 0x7f43d4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.345+0000 7f43d8f37700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4100f60 0x7f43d4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.345+0000 7f43d8f37700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43d4106660 0x7f43d41a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.345+0000 7f43d8f37700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d41a4dc0 con 0x7f43d41005a0 2023-09-10T00:47:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.345+0000 7f43d8f37700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d41a4f00 con 0x7f43d4106660 2023-09-10T00:47:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.345+0000 7f43d8f37700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d41a5040 con 0x7f43d4100f60 2023-09-10T00:47:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.345+0000 7f43d2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43d4106660 0x7f43d41a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.345+0000 7f43d259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d41005a0 0x7f43d4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.345+0000 7f43d2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43d4106660 0x7f43d41a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53166/0 (socket says 172.21.15.44:53166) 2023-09-10T00:47:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.345+0000 7f43d259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d41005a0 0x7f43d4072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51424/0 (socket says 172.21.15.44:51424) 2023-09-10T00:47:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.345+0000 7f43d2d9d700 1 -- 172.21.15.44:0/1092265734 learned_addr learned my addr 172.21.15.44:0/1092265734 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.346+0000 7f43d259c700 1 -- 172.21.15.44:0/1092265734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4100f60 msgr2=0x7f43d4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.346+0000 7f43d259c700 1 --2- 172.21.15.44:0/1092265734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4100f60 0x7f43d4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.346+0000 7f43d259c700 1 -- 172.21.15.44:0/1092265734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43d4106660 msgr2=0x7f43d41a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.346+0000 7f43d259c700 1 --2- 172.21.15.44:0/1092265734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43d4106660 0x7f43d41a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.346+0000 7f43d259c700 1 -- 172.21.15.44:0/1092265734 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc009d00 con 0x7f43d41005a0 2023-09-10T00:47:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.346+0000 7f43d259c700 1 --2- 172.21.15.44:0/1092265734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d41005a0 0x7f43d4072900 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f43c400c970 tx=0x7f43c400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.347+0000 7f43c37fe700 1 -- 172.21.15.44:0/1092265734 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4004440 con 0x7f43d41005a0 2023-09-10T00:47:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.347+0000 7f43c37fe700 1 -- 172.21.15.44:0/1092265734 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43c400ce60 con 0x7f43d41005a0 2023-09-10T00:47:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.347+0000 7f43d8f37700 1 -- 172.21.15.44:0/1092265734 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d41a52d0 con 0x7f43d41005a0 2023-09-10T00:47:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.347+0000 7f43c37fe700 1 -- 172.21.15.44:0/1092265734 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4003b60 con 0x7f43d41005a0 2023-09-10T00:47:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.347+0000 7f43d8f37700 1 -- 172.21.15.44:0/1092265734 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d41a57e0 con 0x7f43d41005a0 2023-09-10T00:47:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.349+0000 7f43c37fe700 1 -- 172.21.15.44:0/1092265734 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f43c4007c80 con 0x7f43d41005a0 2023-09-10T00:47:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.350+0000 7f43c37fe700 1 --2- 172.21.15.44:0/1092265734 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f43bc0a0900 0x7f43bc0a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.350+0000 7f43d1d9b700 1 --2- 172.21.15.44:0/1092265734 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f43bc0a0900 0x7f43bc0a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.350+0000 7f43c37fe700 1 -- 172.21.15.44:0/1092265734 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f43c40cbe80 con 0x7f43d41005a0 2023-09-10T00:47:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.350+0000 7f43d1d9b700 1 --2- 172.21.15.44:0/1092265734 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f43bc0a0900 0x7f43bc0a2dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f43c8005950 tx=0x7f43c80058e0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.350+0000 7f43e50fdb80 1 Processor -- start 2023-09-10T00:47:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43e50fdb80 1 -- start start 2023-09-10T00:47:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43e50fdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286df1a0 0x5602286df580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43e50fdb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9300 0x5602286d9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43e50fdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9cc0 0x5602286e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43e50fdb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602286326d0 con 0x5602286d9300 2023-09-10T00:47:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43e50fdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022862fff0 con 0x5602286d9cc0 2023-09-10T00:47:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43e50fdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022862b8e0 con 0x5602286df1a0 2023-09-10T00:47:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43d1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9300 0x5602286d9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43d1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9300 0x5602286d9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51434/0 (socket says 172.21.15.44:51434) 2023-09-10T00:47:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43d1d9b700 1 -- 172.21.15.44:0/2899984792 learned_addr learned my addr 172.21.15.44:0/2899984792 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43d2d9d700 1 --2- 172.21.15.44:0/2899984792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9cc0 0x5602286e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43d1d9b700 1 -- 172.21.15.44:0/2899984792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286df1a0 msgr2=0x5602286df580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T00:47:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43d1d9b700 1 --2- 172.21.15.44:0/2899984792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286df1a0 0x5602286df580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43d1d9b700 1 -- 172.21.15.44:0/2899984792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9cc0 msgr2=0x5602286e5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43d1d9b700 1 --2- 172.21.15.44:0/2899984792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9cc0 0x5602286e5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43d1d9b700 1 -- 172.21.15.44:0/2899984792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560228457d80 con 0x5602286d9300 2023-09-10T00:47:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43d2d9d700 1 --2- 172.21.15.44:0/2899984792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9cc0 0x5602286e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.351+0000 7f43d1d9b700 1 --2- 172.21.15.44:0/2899984792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9300 0x5602286d9780 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f43c800ebb0 tx=0x7f43c8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fe5242df9d1d1d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.352+0000 7f43c0ff9700 1 -- 172.21.15.44:0/2899984792 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8010ac0 con 0x5602286d9300 2023-09-10T00:47:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.352+0000 7f43c0ff9700 1 -- 172.21.15.44:0/2899984792 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43c8009820 con 0x5602286d9300 2023-09-10T00:47:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.352+0000 7f43c0ff9700 1 -- 172.21.15.44:0/2899984792 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c800f3c0 con 0x5602286d9300 2023-09-10T00:47:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.352+0000 7f43e50fdb80 1 -- 172.21.15.44:0/2899984792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9300 msgr2=0x5602286d9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.352+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/2899984792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9300 0x5602286d9780 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f43c800ebb0 tx=0x7f43c8011000 comp rx=0 tx=0).stop 2023-09-10T00:47:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.352+0000 7f43e50fdb80 1 -- 172.21.15.44:0/2899984792 shutdown_connections 2023-09-10T00:47:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.352+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/2899984792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9cc0 0x5602286e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.352+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/2899984792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286df1a0 0x5602286df580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.352+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/2899984792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9300 0x5602286d9780 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.352+0000 7f43e50fdb80 1 -- 172.21.15.44:0/2899984792 >> 172.21.15.44:0/2899984792 conn(0x56022864e030 msgr2=0x5602286d4560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.353+0000 7f43e50fdb80 1 -- 172.21.15.44:0/2899984792 shutdown_connections 2023-09-10T00:47:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.353+0000 7f43e50fdb80 1 -- 172.21.15.44:0/2899984792 wait complete. 2023-09-10T00:47:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.353+0000 7f43e50fdb80 1 -- start start 2023-09-10T00:47:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.353+0000 7f43e50fdb80 1 Processor -- start 2023-09-10T00:47:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.354+0000 7f43e50fdb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9300 0x5602286e5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.354+0000 7f43e50fdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9cc0 0x5602286e3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.354+0000 7f43e50fdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286df1a0 0x5602286e3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.354+0000 7f43e50fdb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602285c9b10 con 0x5602286d9300 2023-09-10T00:47:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.354+0000 7f43e50fdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602285c9810 con 0x5602286d9cc0 2023-09-10T00:47:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43e50fdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602284ad8c0 con 0x5602286df1a0 2023-09-10T00:47:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43e50fdb80 10 client.0 _ll_register_callbacks cb 0x560228513a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:47:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43d259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9300 0x5602286e5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43d1d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9cc0 0x5602286e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43d2d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286df1a0 0x5602286e3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43d1d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9cc0 0x5602286e3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53178/0 (socket says 172.21.15.44:53178) 2023-09-10T00:47:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43d2d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286df1a0 0x5602286e3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58322/0 (socket says 172.21.15.44:58322) 2023-09-10T00:47:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43d1d9b700 1 -- 172.21.15.44:0/948701402 learned_addr learned my addr 172.21.15.44:0/948701402 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43d259c700 1 -- 172.21.15.44:0/948701402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286df1a0 msgr2=0x5602286e3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43d259c700 1 --2- 172.21.15.44:0/948701402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286df1a0 0x5602286e3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43d259c700 1 -- 172.21.15.44:0/948701402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9cc0 msgr2=0x5602286e3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43d259c700 1 --2- 172.21.15.44:0/948701402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9cc0 0x5602286e3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43d259c700 1 -- 172.21.15.44:0/948701402 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d410dd20 con 0x5602286d9300 2023-09-10T00:47:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43d1d9b700 1 --2- 172.21.15.44:0/948701402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9cc0 0x5602286e3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43d259c700 1 --2- 172.21.15.44:0/948701402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9300 0x5602286e5e90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f43c40c91d0 tx=0x7f43c4098b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cadba45a889b323b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:47:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43b9ffb700 1 -- 172.21.15.44:0/948701402 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4098f30 con 0x5602286d9300 2023-09-10T00:47:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43b9ffb700 1 -- 172.21.15.44:0/948701402 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f43c409cd60 con 0x5602286d9300 2023-09-10T00:47:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.355+0000 7f43e50fdb80 1 -- 172.21.15.44:0/948701402 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560228640840 con 0x5602286d9300 2023-09-10T00:47:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.356+0000 7f43b9ffb700 1 -- 172.21.15.44:0/948701402 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43c40a0fa0 con 0x5602286d9300 2023-09-10T00:47:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.356+0000 7f43e50fdb80 10 client.16917 fetch_fsmap learned FSMap version 48 2023-09-10T00:47:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.356+0000 7f43e50fdb80 1 -- 172.21.15.44:0/948701402 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x7f43c800e890 con 0x5602286d9300 2023-09-10T00:47:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.356+0000 7f43b9ffb700 1 -- 172.21.15.44:0/948701402 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560228640840 con 0x5602286d9300 2023-09-10T00:47:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.356+0000 7f43b9ffb700 1 -- 172.21.15.44:0/948701402 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 48) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f43c4098420 con 0x5602286d9300 2023-09-10T00:47:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.356+0000 7f43e50fdb80 10 client.16917 fetch_fsmap finished waiting for FSMap version 48 2023-09-10T00:47:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.356+0000 7f43e50fdb80 10 client.16917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:47:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.356+0000 7f43e50fdb80 20 client.16917 populate_metadata read hostname 'smithi044' 2023-09-10T00:47:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.356+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/948701402 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5602286f3390 0x5602286f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.356+0000 7f43e50fdb80 4 client.16917 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:47:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.356+0000 7f43e50fdb80 1 -- 172.21.15.44:0/948701402 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602286248c0 con 0x5602286f3390 2023-09-10T00:47:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.356+0000 7f43d1d9b700 1 --2- 172.21.15.44:0/948701402 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5602286f3390 0x5602286f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.357+0000 7f43b9ffb700 10 client.16917 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.357+0000 7f43d1d9b700 1 --2- 172.21.15.44:0/948701402 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5602286f3390 0x5602286f57b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbacfc09c4fdcd2f server_cookie=d09882f8d71ead6f in_seq=0 out_seq=0 2023-09-10T00:47:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.357+0000 7f43b9ffb700 1 -- 172.21.15.44:0/948701402 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5602286248c0 con 0x5602286f3390 2023-09-10T00:47:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.357+0000 7f43b9ffb700 10 client.16917 handle_command_reply: tid=0 2023-09-10T00:47:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.357+0000 7f43e50fdb80 1 client.16917 shutdown 2023-09-10T00:47:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.357+0000 7f43e50fdb80 1 -- 172.21.15.44:0/948701402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9300 msgr2=0x5602286e5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.357+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/948701402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9300 0x5602286e5e90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f43c40c91d0 tx=0x7f43c4098b60 comp rx=0 tx=0).stop 2023-09-10T00:47:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.358+0000 7f43e50fdb80 1 -- 172.21.15.44:0/948701402 shutdown_connections 2023-09-10T00:47:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.358+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/948701402 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5602286f3390 0x5602286f57b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.358+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/948701402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286df1a0 0x5602286e3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.358+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/948701402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9300 0x5602286e5e90 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.358+0000 7f43b9ffb700 0 client.16917 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.358+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/948701402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9cc0 0x5602286e3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.358+0000 7f43e50fdb80 1 -- 172.21.15.44:0/948701402 >> 172.21.15.44:0/948701402 conn(0x56022864e030 msgr2=0x5602286d37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.358+0000 7f43e50fdb80 1 -- 172.21.15.44:0/948701402 shutdown_connections 2023-09-10T00:47:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.358+0000 7f43e50fdb80 1 -- 172.21.15.44:0/948701402 wait complete. 2023-09-10T00:47:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.358+0000 7f43e50fdb80 20 client.16917 trim_cache size 0 max 16384 2023-09-10T00:47:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.376+0000 7f43e50fdb80 1 Processor -- start 2023-09-10T00:47:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43e50fdb80 1 -- start start 2023-09-10T00:47:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43e50fdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9300 0x5602286e5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43e50fdb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9cc0 0x5602286d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43e50fdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286df1a0 0x5602286d4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43e50fdb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602286ce8a0 con 0x5602286d9cc0 2023-09-10T00:47:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43e50fdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602286e4130 con 0x5602286d9300 2023-09-10T00:47:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43e50fdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602286dea00 con 0x5602286df1a0 2023-09-10T00:47:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43d1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9cc0 0x5602286d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43d259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9300 0x5602286e5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43d1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9cc0 0x5602286d3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51446/0 (socket says 172.21.15.44:51446) 2023-09-10T00:47:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43d1d9b700 1 -- 172.21.15.44:0/165285866 learned_addr learned my addr 172.21.15.44:0/165285866 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43d2d9d700 1 --2- 172.21.15.44:0/165285866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286df1a0 0x5602286d4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43d1d9b700 1 -- 172.21.15.44:0/165285866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286df1a0 msgr2=0x5602286d4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43d1d9b700 1 --2- 172.21.15.44:0/165285866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286df1a0 0x5602286d4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43d1d9b700 1 -- 172.21.15.44:0/165285866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9300 msgr2=0x5602286e5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43d1d9b700 1 --2- 172.21.15.44:0/165285866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9300 0x5602286e5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.377+0000 7f43d1d9b700 1 -- 172.21.15.44:0/165285866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560228440c00 con 0x5602286d9cc0 2023-09-10T00:47:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.378+0000 7f43d1d9b700 1 --2- 172.21.15.44:0/165285866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9cc0 0x5602286d3f20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f43c800e860 tx=0x7f43c8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f427083c8c9dcf0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.378+0000 7f43c17fa700 1 -- 172.21.15.44:0/165285866 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c802b050 con 0x5602286d9cc0 2023-09-10T00:47:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.378+0000 7f43c17fa700 1 -- 172.21.15.44:0/165285866 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43c8036070 con 0x5602286d9cc0 2023-09-10T00:47:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.378+0000 7f43c17fa700 1 -- 172.21.15.44:0/165285866 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8032290 con 0x5602286d9cc0 2023-09-10T00:47:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.379+0000 7f43e50fdb80 1 -- 172.21.15.44:0/165285866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9cc0 msgr2=0x5602286d3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.379+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/165285866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9cc0 0x5602286d3f20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f43c800e860 tx=0x7f43c8015000 comp rx=0 tx=0).stop 2023-09-10T00:47:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.379+0000 7f43e50fdb80 1 -- 172.21.15.44:0/165285866 shutdown_connections 2023-09-10T00:47:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.379+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/165285866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286df1a0 0x5602286d4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.379+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/165285866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9300 0x5602286e5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.379+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/165285866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286d9cc0 0x5602286d3f20 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.379+0000 7f43e50fdb80 1 -- 172.21.15.44:0/165285866 >> 172.21.15.44:0/165285866 conn(0x56022864e030 msgr2=0x5602286dd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.379+0000 7f43e50fdb80 1 -- 172.21.15.44:0/165285866 shutdown_connections 2023-09-10T00:47:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.379+0000 7f43e50fdb80 1 -- 172.21.15.44:0/165285866 wait complete. 2023-09-10T00:47:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.379+0000 7f43e50fdb80 1 -- start start 2023-09-10T00:47:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.379+0000 7f43e50fdb80 1 Processor -- start 2023-09-10T00:47:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43e50fdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9300 0x5602286e8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43e50fdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286d9cc0 0x5602286e8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43e50fdb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286df1a0 0x5602287401c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43e50fdb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560228740700 con 0x5602286df1a0 2023-09-10T00:47:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43e50fdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560228740840 con 0x5602286d9300 2023-09-10T00:47:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43e50fdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560228740a00 con 0x5602286d9cc0 2023-09-10T00:47:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43e50fdb80 10 client.0 _ll_register_callbacks cb 0x7f43c4095360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:47:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43d2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286df1a0 0x5602287401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43d259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9300 0x5602286e8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43d1d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286d9cc0 0x5602286e8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43d2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286df1a0 0x5602287401c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51454/0 (socket says 172.21.15.44:51454) 2023-09-10T00:47:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43d259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9300 0x5602286e8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53190/0 (socket says 172.21.15.44:53190) 2023-09-10T00:47:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43d2d9d700 1 -- 172.21.15.44:0/1116839622 learned_addr learned my addr 172.21.15.44:0/1116839622 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43d2d9d700 1 -- 172.21.15.44:0/1116839622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286d9cc0 msgr2=0x5602286e8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43d2d9d700 1 --2- 172.21.15.44:0/1116839622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286d9cc0 0x5602286e8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43d2d9d700 1 -- 172.21.15.44:0/1116839622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9300 msgr2=0x5602286e8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43d2d9d700 1 --2- 172.21.15.44:0/1116839622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9300 0x5602286e8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43d2d9d700 1 -- 172.21.15.44:0/1116839622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560228457d80 con 0x5602286df1a0 2023-09-10T00:47:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43d259c700 1 --2- 172.21.15.44:0/1116839622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9300 0x5602286e8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43d1d9b700 1 --2- 172.21.15.44:0/1116839622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286d9cc0 0x5602286e8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43d2d9d700 1 --2- 172.21.15.44:0/1116839622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286df1a0 0x5602287401c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00bf70 tx=0x7f43cc0074e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db0bc55a577e9f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:47:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.380+0000 7f43b9ffb700 1 -- 172.21.15.44:0/1116839622 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc023070 con 0x5602286df1a0 2023-09-10T00:47:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.381+0000 7f43b9ffb700 1 -- 172.21.15.44:0/1116839622 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f43cc007810 con 0x5602286df1a0 2023-09-10T00:47:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.381+0000 7f43e50fdb80 1 -- 172.21.15.44:0/1116839622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560228583490 con 0x5602286df1a0 2023-09-10T00:47:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.381+0000 7f43b9ffb700 1 -- 172.21.15.44:0/1116839622 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43cc00e040 con 0x5602286df1a0 2023-09-10T00:47:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.381+0000 7f43e50fdb80 10 client.16923 fetch_fsmap learned FSMap version 48 2023-09-10T00:47:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.381+0000 7f43e50fdb80 1 -- 172.21.15.44:0/1116839622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x56022847e1f0 con 0x5602286df1a0 2023-09-10T00:47:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.381+0000 7f43b9ffb700 1 -- 172.21.15.44:0/1116839622 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560228583490 con 0x5602286df1a0 2023-09-10T00:47:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.381+0000 7f43b9ffb700 1 -- 172.21.15.44:0/1116839622 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 48) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f43cc00b100 con 0x5602286df1a0 2023-09-10T00:47:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.381+0000 7f43e50fdb80 10 client.16923 fetch_fsmap finished waiting for FSMap version 48 2023-09-10T00:47:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.381+0000 7f43e50fdb80 10 client.16923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:47:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.381+0000 7f43e50fdb80 20 client.16923 populate_metadata read hostname 'smithi044' 2023-09-10T00:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.381+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/1116839622 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560228748300 0x56022874a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.381+0000 7f43e50fdb80 4 client.16923 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.381+0000 7f43e50fdb80 1 -- 172.21.15.44:0/1116839622 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5602285e5050 con 0x560228748300 2023-09-10T00:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.381+0000 7f43d259c700 1 --2- 172.21.15.44:0/1116839622 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560228748300 0x56022874a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.382+0000 7f43d259c700 1 --2- 172.21.15.44:0/1116839622 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560228748300 0x56022874a720 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fbfde7e3c5222d2 server_cookie=ca79674126f17bf7 in_seq=0 out_seq=0 2023-09-10T00:47:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.382+0000 7f43b9ffb700 10 client.16923 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.395+0000 7f43b9ffb700 1 -- 172.21.15.44:0/1116839622 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5602285e5050 con 0x560228748300 2023-09-10T00:47:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.395+0000 7f43b9ffb700 10 client.16923 handle_command_reply: tid=0 2023-09-10T00:47:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.395+0000 7f43e50fdb80 1 client.16923 shutdown 2023-09-10T00:47:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.395+0000 7f43e50fdb80 1 -- 172.21.15.44:0/1116839622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286df1a0 msgr2=0x5602287401c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.395+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/1116839622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286df1a0 0x5602287401c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00bf70 tx=0x7f43cc0074e0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.395+0000 7f43e50fdb80 1 -- 172.21.15.44:0/1116839622 shutdown_connections 2023-09-10T00:47:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.395+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/1116839622 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560228748300 0x56022874a720 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.395+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/1116839622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5602286df1a0 0x5602287401c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.395+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/1116839622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602286d9300 0x5602286e8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.395+0000 7f43e50fdb80 1 --2- 172.21.15.44:0/1116839622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602286d9cc0 0x5602286e8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.395+0000 7f43b9ffb700 0 client.16923 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.395+0000 7f43e50fdb80 1 -- 172.21.15.44:0/1116839622 >> 172.21.15.44:0/1116839622 conn(0x56022864e030 msgr2=0x5602286d82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.395+0000 7f43e50fdb80 1 -- 172.21.15.44:0/1116839622 shutdown_connections 2023-09-10T00:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.395+0000 7f43e50fdb80 1 -- 172.21.15.44:0/1116839622 wait complete. 2023-09-10T00:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.395+0000 7f43e50fdb80 20 client.16923 trim_cache size 0 max 16384 2023-09-10T00:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:47:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:47:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.396+0000 7f43d8f37700 1 -- 172.21.15.44:0/1092265734 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f43bc0a0900 msgr2=0x7f43bc0a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.396+0000 7f43d8f37700 1 --2- 172.21.15.44:0/1092265734 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f43bc0a0900 0x7f43bc0a2dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f43c8005950 tx=0x7f43c80058e0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.396+0000 7f43d8f37700 1 -- 172.21.15.44:0/1092265734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d41005a0 msgr2=0x7f43d4072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.396+0000 7f43d8f37700 1 --2- 172.21.15.44:0/1092265734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d41005a0 0x7f43d4072900 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f43c400c970 tx=0x7f43c400cc80 comp rx=0 tx=0).stop 2023-09-10T00:47:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.396+0000 7f43d8f37700 1 -- 172.21.15.44:0/1092265734 shutdown_connections 2023-09-10T00:47:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.396+0000 7f43d8f37700 1 --2- 172.21.15.44:0/1092265734 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f43bc0a0900 0x7f43bc0a2dc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.396+0000 7f43d8f37700 1 --2- 172.21.15.44:0/1092265734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43d4106660 0x7f43d41a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.396+0000 7f43d8f37700 1 --2- 172.21.15.44:0/1092265734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d41005a0 0x7f43d4072900 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.396+0000 7f43d8f37700 1 --2- 172.21.15.44:0/1092265734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4100f60 0x7f43d4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.396+0000 7f43d8f37700 1 -- 172.21.15.44:0/1092265734 >> 172.21.15.44:0/1092265734 conn(0x7f43d40fc0b0 msgr2=0x7f43d40fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.396+0000 7f43d8f37700 1 -- 172.21.15.44:0/1092265734 shutdown_connections 2023-09-10T00:47:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:17.396+0000 7f43d8f37700 1 -- 172.21.15.44:0/1092265734 wait complete. 2023-09-10T00:47:18.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:18 smithi099 ceph-mon[102692]: pgmap v204: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 18 MiB/s wr, 72 op/s 2023-09-10T00:47:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:18 smithi131 ceph-mon[103713]: pgmap v204: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 18 MiB/s wr, 72 op/s 2023-09-10T00:47:19.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:18 smithi044 ceph-mon[98676]: pgmap v204: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 18 MiB/s wr, 72 op/s 2023-09-10T00:47:19.223 INFO:teuthology.orchestra.run.smithi044.stderr:31200000 of 50000000 tuples (62%) done (elapsed 378.86 s, remaining 228.29 s) 2023-09-10T00:47:20.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:20 smithi099 ceph-mon[102692]: pgmap v205: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 15 MiB/s wr, 61 op/s 2023-09-10T00:47:21.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:20 smithi131 ceph-mon[103713]: pgmap v205: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 15 MiB/s wr, 61 op/s 2023-09-10T00:47:21.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:20 smithi044 ceph-mon[98676]: pgmap v205: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 15 MiB/s wr, 61 op/s 2023-09-10T00:47:22.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:22 smithi099 ceph-mon[102692]: pgmap v206: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 14 MiB/s wr, 152 op/s 2023-09-10T00:47:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:22 smithi131 ceph-mon[103713]: pgmap v206: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 14 MiB/s wr, 152 op/s 2023-09-10T00:47:23.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:22 smithi044 ceph-mon[98676]: pgmap v206: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 14 MiB/s wr, 152 op/s 2023-09-10T00:47:24.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:24 smithi099 ceph-mon[102692]: pgmap v207: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 14 MiB/s wr, 152 op/s 2023-09-10T00:47:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:24 smithi131 ceph-mon[103713]: pgmap v207: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 14 MiB/s wr, 152 op/s 2023-09-10T00:47:25.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:24 smithi044 ceph-mon[98676]: pgmap v207: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 14 MiB/s wr, 152 op/s 2023-09-10T00:47:26.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:26 smithi099 ceph-mon[102692]: pgmap v208: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 12 MiB/s wr, 336 op/s 2023-09-10T00:47:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:26 smithi131 ceph-mon[103713]: pgmap v208: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 12 MiB/s wr, 336 op/s 2023-09-10T00:47:27.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:26 smithi044 ceph-mon[98676]: pgmap v208: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 12 MiB/s wr, 336 op/s 2023-09-10T00:47:28.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:28 smithi099 ceph-mon[102692]: pgmap v209: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 11 MiB/s wr, 292 op/s 2023-09-10T00:47:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:28 smithi131 ceph-mon[103713]: pgmap v209: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 11 MiB/s wr, 292 op/s 2023-09-10T00:47:29.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:28 smithi044 ceph-mon[98676]: pgmap v209: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 11 MiB/s wr, 292 op/s 2023-09-10T00:47:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.531+0000 7fcaeb58b700 1 -- 172.21.15.44:0/1015573939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcae41055b0 msgr2=0x7fcae4105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.531+0000 7fcaeb58b700 1 --2- 172.21.15.44:0/1015573939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcae41055b0 0x7fcae4105990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fcae00099c0 tx=0x7fcae0009cd0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.531+0000 7fcaeb58b700 1 -- 172.21.15.44:0/1015573939 shutdown_connections 2023-09-10T00:47:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.531+0000 7fcaeb58b700 1 --2- 172.21.15.44:0/1015573939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcae4100120 0x7fcae410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.531+0000 7fcaeb58b700 1 --2- 172.21.15.44:0/1015573939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcae41055b0 0x7fcae4105990 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.531+0000 7fcaeb58b700 1 --2- 172.21.15.44:0/1015573939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcae40ff760 0x7fcae40ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.531+0000 7fcaeb58b700 1 -- 172.21.15.44:0/1015573939 >> 172.21.15.44:0/1015573939 conn(0x7fcae4074f30 msgr2=0x7fcae4075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.531+0000 7fcaeb58b700 1 -- 172.21.15.44:0/1015573939 shutdown_connections 2023-09-10T00:47:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.531+0000 7fcaeb58b700 1 -- 172.21.15.44:0/1015573939 wait complete. 2023-09-10T00:47:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.532+0000 7fcaeb58b700 1 Processor -- start 2023-09-10T00:47:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.532+0000 7fcaeb58b700 1 -- start start 2023-09-10T00:47:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.532+0000 7fcaeb58b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcae40ff760 0x7fcae4195480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.532+0000 7fcaeb58b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcae4100120 0x7fcae41959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.532+0000 7fcaeb58b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcae41055b0 0x7fcae4199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.532+0000 7fcaeb58b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae419a5d0 con 0x7fcae41055b0 2023-09-10T00:47:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.532+0000 7fcaeb58b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae419a740 con 0x7fcae4100120 2023-09-10T00:47:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.532+0000 7fcaeb58b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae419a900 con 0x7fcae40ff760 2023-09-10T00:47:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.532+0000 7fcae9b28700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcae41055b0 0x7fcae4199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.532+0000 7fcae9327700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcae40ff760 0x7fcae4195480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.532+0000 7fcae9b28700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcae41055b0 0x7fcae4199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51468/0 (socket says 172.21.15.44:51468) 2023-09-10T00:47:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.532+0000 7fcae8b26700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcae4100120 0x7fcae41959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.532+0000 7fcae9b28700 1 -- 172.21.15.44:0/1316294553 learned_addr learned my addr 172.21.15.44:0/1316294553 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.533+0000 7fcae9b28700 1 -- 172.21.15.44:0/1316294553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcae40ff760 msgr2=0x7fcae4195480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.533+0000 7fcae9b28700 1 --2- 172.21.15.44:0/1316294553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcae40ff760 0x7fcae4195480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.533+0000 7fcae9b28700 1 -- 172.21.15.44:0/1316294553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcae4100120 msgr2=0x7fcae41959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.533+0000 7fcae9b28700 1 --2- 172.21.15.44:0/1316294553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcae4100120 0x7fcae41959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.533+0000 7fcae9b28700 1 -- 172.21.15.44:0/1316294553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae00095f0 con 0x7fcae41055b0 2023-09-10T00:47:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.533+0000 7fcae8b26700 1 --2- 172.21.15.44:0/1316294553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcae4100120 0x7fcae41959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.533+0000 7fcae9b28700 1 --2- 172.21.15.44:0/1316294553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcae41055b0 0x7fcae4199f90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00ef30 tx=0x7fcadc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.533+0000 7fcada7fc700 1 -- 172.21.15.44:0/1316294553 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc018070 con 0x7fcae41055b0 2023-09-10T00:47:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.533+0000 7fcada7fc700 1 -- 172.21.15.44:0/1316294553 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcadc00f040 con 0x7fcae41055b0 2023-09-10T00:47:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.533+0000 7fcada7fc700 1 -- 172.21.15.44:0/1316294553 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc00cbc0 con 0x7fcae41055b0 2023-09-10T00:47:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.533+0000 7fcaeb58b700 1 -- 172.21.15.44:0/1316294553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae419ab80 con 0x7fcae41055b0 2023-09-10T00:47:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.533+0000 7fcaeb58b700 1 -- 172.21.15.44:0/1316294553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae41a2420 con 0x7fcae41055b0 2023-09-10T00:47:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.535+0000 7fcada7fc700 1 -- 172.21.15.44:0/1316294553 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fcadc007500 con 0x7fcae41055b0 2023-09-10T00:47:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.536+0000 7fcada7fc700 1 --2- 172.21.15.44:0/1316294553 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcad00a07b0 0x7fcad00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.536+0000 7fcada7fc700 1 -- 172.21.15.44:0/1316294553 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fcadc0cbeb0 con 0x7fcae41055b0 2023-09-10T00:47:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.536+0000 7fcae9327700 1 --2- 172.21.15.44:0/1316294553 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcad00a07b0 0x7fcad00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.537+0000 7fcae9327700 1 --2- 172.21.15.44:0/1316294553 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcad00a07b0 0x7fcad00a2c70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcae000b580 tx=0x7fcae0005c00 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcaf7751b80 1 Processor -- start 2023-09-10T00:47:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcaf7751b80 1 -- start start 2023-09-10T00:47:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcaf7751b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f9561a0 0x558f7f956580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcaf7751b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950300 0x558f7f950780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcaf7751b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f950cc0 0x558f7f95cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcaf7751b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7f8a96d0 con 0x558f7f9561a0 2023-09-10T00:47:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcaf7751b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7f8a6ff0 con 0x558f7f950300 2023-09-10T00:47:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcaf7751b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7f8a28e0 con 0x558f7f950cc0 2023-09-10T00:47:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcae9327700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f9561a0 0x558f7f956580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcae8b26700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950300 0x558f7f950780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcae9327700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f9561a0 0x558f7f956580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51472/0 (socket says 172.21.15.44:51472) 2023-09-10T00:47:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcae9327700 1 -- 172.21.15.44:0/764206989 learned_addr learned my addr 172.21.15.44:0/764206989 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcae9b28700 1 --2- 172.21.15.44:0/764206989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f950cc0 0x558f7f95cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcae9327700 1 -- 172.21.15.44:0/764206989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f950cc0 msgr2=0x558f7f95cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcae9327700 1 --2- 172.21.15.44:0/764206989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f950cc0 0x558f7f95cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcae9327700 1 -- 172.21.15.44:0/764206989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950300 msgr2=0x558f7f950780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcae9327700 1 --2- 172.21.15.44:0/764206989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950300 0x558f7f950780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcae9327700 1 -- 172.21.15.44:0/764206989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f7f6ced80 con 0x558f7f9561a0 2023-09-10T00:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcae9b28700 1 --2- 172.21.15.44:0/764206989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f950cc0 0x558f7f95cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.538+0000 7fcae8b26700 1 --2- 172.21.15.44:0/764206989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950300 0x558f7f950780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.539+0000 7fcae9327700 1 --2- 172.21.15.44:0/764206989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f9561a0 0x558f7f956580 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fcae0015000 tx=0x7fcae001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51e77a9120d6c89 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.539+0000 7fcacf7fe700 1 -- 172.21.15.44:0/764206989 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae002a070 con 0x558f7f9561a0 2023-09-10T00:47:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.539+0000 7fcacf7fe700 1 -- 172.21.15.44:0/764206989 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcae001e040 con 0x558f7f9561a0 2023-09-10T00:47:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.539+0000 7fcacf7fe700 1 -- 172.21.15.44:0/764206989 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0024880 con 0x558f7f9561a0 2023-09-10T00:47:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.540+0000 7fcaf7751b80 1 -- 172.21.15.44:0/764206989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f9561a0 msgr2=0x558f7f956580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.540+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/764206989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f9561a0 0x558f7f956580 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fcae0015000 tx=0x7fcae001f000 comp rx=0 tx=0).stop 2023-09-10T00:47:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.540+0000 7fcaf7751b80 1 -- 172.21.15.44:0/764206989 shutdown_connections 2023-09-10T00:47:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.540+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/764206989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f950cc0 0x558f7f95cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.540+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/764206989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f9561a0 0x558f7f956580 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.540+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/764206989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950300 0x558f7f950780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.540+0000 7fcaf7751b80 1 -- 172.21.15.44:0/764206989 >> 172.21.15.44:0/764206989 conn(0x558f7f8c5030 msgr2=0x558f7f94b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.540+0000 7fcaf7751b80 1 -- 172.21.15.44:0/764206989 shutdown_connections 2023-09-10T00:47:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.540+0000 7fcaf7751b80 1 -- 172.21.15.44:0/764206989 wait complete. 2023-09-10T00:47:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.540+0000 7fcaf7751b80 1 -- start start 2023-09-10T00:47:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.540+0000 7fcaf7751b80 1 Processor -- start 2023-09-10T00:47:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcaf7751b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950300 0x558f7f95cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcaf7751b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950cc0 0x558f7f95a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcaf7751b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f9561a0 0x558f7f95a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcaf7751b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7f840b10 con 0x558f7f950cc0 2023-09-10T00:47:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcaf7751b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7f840810 con 0x558f7f950300 2023-09-10T00:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcaf7751b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7f7248c0 con 0x558f7f9561a0 2023-09-10T00:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcaf7751b80 10 client.0 _ll_register_callbacks cb 0x558f7f78aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcae8b26700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950cc0 0x558f7f95a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcae9327700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950300 0x558f7f95cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcae9b28700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f9561a0 0x558f7f95a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcae8b26700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950cc0 0x558f7f95a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51480/0 (socket says 172.21.15.44:51480) 2023-09-10T00:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcae9b28700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f9561a0 0x558f7f95a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58364/0 (socket says 172.21.15.44:58364) 2023-09-10T00:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcae8b26700 1 -- 172.21.15.44:0/2843637775 learned_addr learned my addr 172.21.15.44:0/2843637775 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcae9b28700 1 -- 172.21.15.44:0/2843637775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950300 msgr2=0x558f7f95cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcae9b28700 1 --2- 172.21.15.44:0/2843637775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950300 0x558f7f95cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcae9b28700 1 -- 172.21.15.44:0/2843637775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950cc0 msgr2=0x558f7f95a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcae9b28700 1 --2- 172.21.15.44:0/2843637775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950cc0 0x558f7f95a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcae9b28700 1 -- 172.21.15.44:0/2843637775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad4009c90 con 0x558f7f9561a0 2023-09-10T00:47:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcae9327700 1 --2- 172.21.15.44:0/2843637775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950300 0x558f7f95cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcae8b26700 1 --2- 172.21.15.44:0/2843637775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950cc0 0x558f7f95a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcae9b28700 1 --2- 172.21.15.44:0/2843637775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f9561a0 0x558f7f95a8e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0c9200 tx=0x7fcadc0989c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=547b7f099db5f280 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.541+0000 7fcaccff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:47:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.542+0000 7fcaccff9700 1 -- 172.21.15.44:0/2843637775 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc0996f0 con 0x558f7f9561a0 2023-09-10T00:47:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.542+0000 7fcaccff9700 1 -- 172.21.15.44:0/2843637775 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fcadc098ff0 con 0x558f7f9561a0 2023-09-10T00:47:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.542+0000 7fcaf7751b80 1 -- 172.21.15.44:0/2843637775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f7f7fa490 con 0x558f7f9561a0 2023-09-10T00:47:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.542+0000 7fcaccff9700 1 -- 172.21.15.44:0/2843637775 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcadc09ce20 con 0x558f7f9561a0 2023-09-10T00:47:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.542+0000 7fcaf7751b80 10 client.26539 fetch_fsmap learned FSMap version 49 2023-09-10T00:47:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.542+0000 7fcaf7751b80 1 -- 172.21.15.44:0/2843637775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7fcae41a2420 con 0x558f7f9561a0 2023-09-10T00:47:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.542+0000 7fcaccff9700 1 -- 172.21.15.44:0/2843637775 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558f7f7fa490 con 0x558f7f9561a0 2023-09-10T00:47:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.542+0000 7fcaccff9700 1 -- 172.21.15.44:0/2843637775 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 49) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fcadc0a0b40 con 0x558f7f9561a0 2023-09-10T00:47:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.542+0000 7fcaf7751b80 10 client.26539 fetch_fsmap finished waiting for FSMap version 49 2023-09-10T00:47:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.542+0000 7fcaf7751b80 10 client.26539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:47:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.542+0000 7fcaf7751b80 20 client.26539 populate_metadata read hostname 'smithi044' 2023-09-10T00:47:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.542+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/2843637775 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f7f96a4c0 0x558f7f96c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.542+0000 7fcaf7751b80 4 client.26539 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:47:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.542+0000 7fcaf7751b80 1 -- 172.21.15.44:0/2843637775 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f7f8b7840 con 0x558f7f96a4c0 2023-09-10T00:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.542+0000 7fcae9327700 1 --2- 172.21.15.44:0/2843637775 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f7f96a4c0 0x558f7f96c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.543+0000 7fcaccff9700 10 client.26539 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.543+0000 7fcae9327700 1 --2- 172.21.15.44:0/2843637775 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f7f96a4c0 0x558f7f96c8e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee70cebca6a4769 server_cookie=de532ac0d2ee9354 in_seq=0 out_seq=0 2023-09-10T00:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.544+0000 7fcaccff9700 1 -- 172.21.15.44:0/2843637775 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558f7f8b7840 con 0x558f7f96a4c0 2023-09-10T00:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.544+0000 7fcaccff9700 10 client.26539 handle_command_reply: tid=0 2023-09-10T00:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.544+0000 7fcaf7751b80 1 client.26539 shutdown 2023-09-10T00:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.544+0000 7fcaf7751b80 1 -- 172.21.15.44:0/2843637775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f9561a0 msgr2=0x558f7f95a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.544+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/2843637775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f9561a0 0x558f7f95a8e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0c9200 tx=0x7fcadc0989c0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.544+0000 7fcaf7751b80 1 -- 172.21.15.44:0/2843637775 shutdown_connections 2023-09-10T00:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.544+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/2843637775 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f7f96a4c0 0x558f7f96c8e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.544+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/2843637775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f9561a0 0x558f7f95a8e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.544+0000 7fcaccff9700 0 client.26539 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.544+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/2843637775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950300 0x558f7f95cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.544+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/2843637775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950cc0 0x558f7f95a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.544+0000 7fcaf7751b80 1 -- 172.21.15.44:0/2843637775 >> 172.21.15.44:0/2843637775 conn(0x558f7f8c5030 msgr2=0x558f7f94a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.544+0000 7fcaf7751b80 1 -- 172.21.15.44:0/2843637775 shutdown_connections 2023-09-10T00:47:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.544+0000 7fcaf7751b80 1 -- 172.21.15.44:0/2843637775 wait complete. 2023-09-10T00:47:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.544+0000 7fcaf7751b80 20 client.26539 trim_cache size 0 max 16384 2023-09-10T00:47:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.563+0000 7fcaf7751b80 1 Processor -- start 2023-09-10T00:47:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.563+0000 7fcaf7751b80 1 -- start start 2023-09-10T00:47:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.563+0000 7fcaf7751b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950300 0x558f7f959a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.563+0000 7fcaf7751b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f950cc0 0x558f7f95fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.563+0000 7fcaf7751b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f9561a0 0x558f7f9541f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.563+0000 7fcaf7751b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7f951580 con 0x558f7f950300 2023-09-10T00:47:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.563+0000 7fcaf7751b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7f95aee0 con 0x558f7f9561a0 2023-09-10T00:47:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.563+0000 7fcaf7751b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7f959f80 con 0x558f7f950cc0 2023-09-10T00:47:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.563+0000 7fcae9327700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950300 0x558f7f959a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.563+0000 7fcae8b26700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f950cc0 0x558f7f95fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.563+0000 7fcae9327700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950300 0x558f7f959a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51486/0 (socket says 172.21.15.44:51486) 2023-09-10T00:47:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.563+0000 7fcae8b26700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f950cc0 0x558f7f95fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58370/0 (socket says 172.21.15.44:58370) 2023-09-10T00:47:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.563+0000 7fcae9327700 1 -- 172.21.15.44:0/1398463840 learned_addr learned my addr 172.21.15.44:0/1398463840 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.563+0000 7fcae9b28700 1 --2- 172.21.15.44:0/1398463840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f9561a0 0x558f7f9541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.563+0000 7fcae8b26700 1 -- 172.21.15.44:0/1398463840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f9561a0 msgr2=0x558f7f9541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.563+0000 7fcae8b26700 1 --2- 172.21.15.44:0/1398463840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f9561a0 0x558f7f9541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.564+0000 7fcae8b26700 1 -- 172.21.15.44:0/1398463840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950300 msgr2=0x558f7f959a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.564+0000 7fcae8b26700 1 --2- 172.21.15.44:0/1398463840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950300 0x558f7f959a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.564+0000 7fcae8b26700 1 -- 172.21.15.44:0/1398463840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae410cbc0 con 0x558f7f950cc0 2023-09-10T00:47:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.564+0000 7fcae9327700 1 --2- 172.21.15.44:0/1398463840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950300 0x558f7f959a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.564+0000 7fcae9b28700 1 --2- 172.21.15.44:0/1398463840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f9561a0 0x558f7f9541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:47:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.564+0000 7fcae8b26700 1 --2- 172.21.15.44:0/1398463840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f950cc0 0x558f7f95fa70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x558f7f95b7e0 tx=0x7fcad4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad297847c68c19fa server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.564+0000 7fcacffff700 1 -- 172.21.15.44:0/1398463840 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4017050 con 0x558f7f950cc0 2023-09-10T00:47:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.564+0000 7fcacffff700 1 -- 172.21.15.44:0/1398463840 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcad4018040 con 0x558f7f950cc0 2023-09-10T00:47:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.564+0000 7fcacffff700 1 -- 172.21.15.44:0/1398463840 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4003680 con 0x558f7f950cc0 2023-09-10T00:47:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.565+0000 7fcaf7751b80 1 -- 172.21.15.44:0/1398463840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f950cc0 msgr2=0x558f7f95fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.565+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/1398463840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f950cc0 0x558f7f95fa70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x558f7f95b7e0 tx=0x7fcad4009260 comp rx=0 tx=0).stop 2023-09-10T00:47:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.565+0000 7fcaf7751b80 1 -- 172.21.15.44:0/1398463840 shutdown_connections 2023-09-10T00:47:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.565+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/1398463840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f9561a0 0x558f7f9541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.565+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/1398463840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950300 0x558f7f959a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.565+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/1398463840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f950cc0 0x558f7f95fa70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.565+0000 7fcaf7751b80 1 -- 172.21.15.44:0/1398463840 >> 172.21.15.44:0/1398463840 conn(0x558f7f8c5030 msgr2=0x558f7f971240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.565+0000 7fcaf7751b80 1 -- 172.21.15.44:0/1398463840 shutdown_connections 2023-09-10T00:47:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.565+0000 7fcaf7751b80 1 -- 172.21.15.44:0/1398463840 wait complete. 2023-09-10T00:47:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.565+0000 7fcaf7751b80 1 -- start start 2023-09-10T00:47:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.565+0000 7fcaf7751b80 1 Processor -- start 2023-09-10T00:47:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.566+0000 7fcaf7751b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950300 0x558f7f9bab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.566+0000 7fcaf7751b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950cc0 0x558f7f9bb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.566+0000 7fcaf7751b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f9561a0 0x558f7f9b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.566+0000 7fcaf7751b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7f9b6da0 con 0x558f7f950300 2023-09-10T00:47:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.566+0000 7fcaf7751b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7f9b6f10 con 0x558f7f950cc0 2023-09-10T00:47:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.566+0000 7fcaf7751b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7f9b70d0 con 0x558f7f9561a0 2023-09-10T00:47:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.566+0000 7fcaf7751b80 10 client.0 _ll_register_callbacks cb 0x7fcadc098e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:47:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.566+0000 7fcae9327700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950300 0x558f7f9bab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.566+0000 7fcae8b26700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950cc0 0x558f7f9bb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.566+0000 7fcae8b26700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950cc0 0x558f7f9bb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53234/0 (socket says 172.21.15.44:53234) 2023-09-10T00:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.566+0000 7fcae9327700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950300 0x558f7f9bab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51492/0 (socket says 172.21.15.44:51492) 2023-09-10T00:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.566+0000 7fcae8b26700 1 -- 172.21.15.44:0/672858645 learned_addr learned my addr 172.21.15.44:0/672858645 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.566+0000 7fcae9b28700 1 --2- 172.21.15.44:0/672858645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f9561a0 0x558f7f9b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.567+0000 7fcae9b28700 1 -- 172.21.15.44:0/672858645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950cc0 msgr2=0x558f7f9bb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.567+0000 7fcae9b28700 1 --2- 172.21.15.44:0/672858645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950cc0 0x558f7f9bb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.567+0000 7fcae9b28700 1 -- 172.21.15.44:0/672858645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950300 msgr2=0x558f7f9bab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.567+0000 7fcae9b28700 1 --2- 172.21.15.44:0/672858645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950300 0x558f7f9bab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.567+0000 7fcae9b28700 1 -- 172.21.15.44:0/672858645 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f7f6ced80 con 0x558f7f9561a0 2023-09-10T00:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.567+0000 7fcae9b28700 1 --2- 172.21.15.44:0/672858645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f9561a0 0x558f7f9b6690 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcadc01b000 tx=0x7fcadc0b1480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=447ab3f0958d61dc server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.567+0000 7fcaccff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.567+0000 7fcaccff9700 1 -- 172.21.15.44:0/672858645 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc0984e0 con 0x558f7f9561a0 2023-09-10T00:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.567+0000 7fcaccff9700 1 -- 172.21.15.44:0/672858645 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fcadc0996f0 con 0x558f7f9561a0 2023-09-10T00:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.567+0000 7fcaf7751b80 1 -- 172.21.15.44:0/672858645 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcae001e040 con 0x558f7f9561a0 2023-09-10T00:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.567+0000 7fcaccff9700 1 -- 172.21.15.44:0/672858645 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcadc098ff0 con 0x558f7f9561a0 2023-09-10T00:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.567+0000 7fcaf7751b80 10 client.26545 fetch_fsmap learned FSMap version 49 2023-09-10T00:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.567+0000 7fcaf7751b80 1 -- 172.21.15.44:0/672858645 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7fcad4012850 con 0x558f7f9561a0 2023-09-10T00:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.567+0000 7fcaccff9700 1 -- 172.21.15.44:0/672858645 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcae001e040 con 0x558f7f9561a0 2023-09-10T00:47:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.567+0000 7fcaccff9700 1 -- 172.21.15.44:0/672858645 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 49) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fcadc097f50 con 0x558f7f9561a0 2023-09-10T00:47:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.568+0000 7fcaf7751b80 10 client.26545 fetch_fsmap finished waiting for FSMap version 49 2023-09-10T00:47:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.568+0000 7fcaf7751b80 10 client.26545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:47:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.568+0000 7fcaf7751b80 20 client.26545 populate_metadata read hostname 'smithi044' 2023-09-10T00:47:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.568+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/672858645 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f7f9c36f0 0x558f7f9c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.568+0000 7fcaf7751b80 4 client.26545 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:47:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.568+0000 7fcaf7751b80 1 -- 172.21.15.44:0/672858645 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558f7f89b8c0 con 0x558f7f9c36f0 2023-09-10T00:47:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.568+0000 7fcae9327700 1 --2- 172.21.15.44:0/672858645 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f7f9c36f0 0x558f7f9c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.568+0000 7fcaccff9700 10 client.26545 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.568+0000 7fcae9327700 1 --2- 172.21.15.44:0/672858645 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f7f9c36f0 0x558f7f9c5b10 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=babeb2d810a7072e server_cookie=90c3b106fc2a210f in_seq=0 out_seq=0 2023-09-10T00:47:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.591+0000 7fcaccff9700 1 -- 172.21.15.44:0/672858645 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558f7f89b8c0 con 0x558f7f9c36f0 2023-09-10T00:47:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.591+0000 7fcaccff9700 10 client.26545 handle_command_reply: tid=0 2023-09-10T00:47:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.591+0000 7fcaf7751b80 1 client.26545 shutdown 2023-09-10T00:47:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.591+0000 7fcaf7751b80 1 -- 172.21.15.44:0/672858645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f9561a0 msgr2=0x558f7f9b6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.591+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/672858645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f9561a0 0x558f7f9b6690 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcadc01b000 tx=0x7fcadc0b1480 comp rx=0 tx=0).stop 2023-09-10T00:47:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.591+0000 7fcaf7751b80 1 -- 172.21.15.44:0/672858645 shutdown_connections 2023-09-10T00:47:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.591+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/672858645 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f7f9c36f0 0x558f7f9c5b10 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.591+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/672858645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f7f9561a0 0x558f7f9b6690 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.591+0000 7fcaccff9700 0 client.26545 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.591+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/672858645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f7f950300 0x558f7f9bab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.591+0000 7fcaf7751b80 1 --2- 172.21.15.44:0/672858645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f7f950cc0 0x558f7f9bb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.591+0000 7fcaf7751b80 1 -- 172.21.15.44:0/672858645 >> 172.21.15.44:0/672858645 conn(0x558f7f8c5030 msgr2=0x558f7f9541f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.591+0000 7fcaf7751b80 1 -- 172.21.15.44:0/672858645 shutdown_connections 2023-09-10T00:47:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.591+0000 7fcaf7751b80 1 -- 172.21.15.44:0/672858645 wait complete. 2023-09-10T00:47:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.591+0000 7fcaf7751b80 20 client.26545 trim_cache size 0 max 16384 2023-09-10T00:47:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:47:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:47:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:47:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:47:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.592+0000 7fcaeb58b700 1 -- 172.21.15.44:0/1316294553 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcad00a07b0 msgr2=0x7fcad00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.592+0000 7fcaeb58b700 1 --2- 172.21.15.44:0/1316294553 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcad00a07b0 0x7fcad00a2c70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcae000b580 tx=0x7fcae0005c00 comp rx=0 tx=0).stop 2023-09-10T00:47:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.592+0000 7fcaeb58b700 1 -- 172.21.15.44:0/1316294553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcae41055b0 msgr2=0x7fcae4199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.592+0000 7fcaeb58b700 1 --2- 172.21.15.44:0/1316294553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcae41055b0 0x7fcae4199f90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00ef30 tx=0x7fcadc00c3a0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.592+0000 7fcaeb58b700 1 -- 172.21.15.44:0/1316294553 shutdown_connections 2023-09-10T00:47:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.592+0000 7fcaeb58b700 1 --2- 172.21.15.44:0/1316294553 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcad00a07b0 0x7fcad00a2c70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.592+0000 7fcaeb58b700 1 --2- 172.21.15.44:0/1316294553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcae41055b0 0x7fcae4199f90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.592+0000 7fcaeb58b700 1 --2- 172.21.15.44:0/1316294553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcae40ff760 0x7fcae4195480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.592+0000 7fcaeb58b700 1 --2- 172.21.15.44:0/1316294553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcae4100120 0x7fcae41959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.592+0000 7fcaeb58b700 1 -- 172.21.15.44:0/1316294553 >> 172.21.15.44:0/1316294553 conn(0x7fcae4074f30 msgr2=0x7fcae40fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.592+0000 7fcaeb58b700 1 -- 172.21.15.44:0/1316294553 shutdown_connections 2023-09-10T00:47:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:30.592+0000 7fcaeb58b700 1 -- 172.21.15.44:0/1316294553 wait complete. 2023-09-10T00:47:30.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:30 smithi099 ceph-mon[102692]: pgmap v210: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 10 MiB/s wr, 280 op/s 2023-09-10T00:47:30.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:30 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:30 smithi131 ceph-mon[103713]: pgmap v210: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 10 MiB/s wr, 280 op/s 2023-09-10T00:47:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:30 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:30 smithi044 ceph-mon[98676]: pgmap v210: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 10 MiB/s wr, 280 op/s 2023-09-10T00:47:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:30 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:32.302 INFO:teuthology.orchestra.run.smithi044.stderr:31300000 of 50000000 tuples (62%) done (elapsed 391.94 s, remaining 234.16 s) 2023-09-10T00:47:32.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:32 smithi099 ceph-mon[102692]: pgmap v211: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 12 MiB/s wr, 446 op/s 2023-09-10T00:47:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:32 smithi131 ceph-mon[103713]: pgmap v211: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 12 MiB/s wr, 446 op/s 2023-09-10T00:47:33.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:32 smithi044 ceph-mon[98676]: pgmap v211: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 12 MiB/s wr, 446 op/s 2023-09-10T00:47:34.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:34 smithi099 ceph-mon[102692]: pgmap v212: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 2.9 MiB/s wr, 333 op/s 2023-09-10T00:47:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:34 smithi131 ceph-mon[103713]: pgmap v212: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 2.9 MiB/s wr, 333 op/s 2023-09-10T00:47:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:34 smithi044 ceph-mon[98676]: pgmap v212: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 2.9 MiB/s wr, 333 op/s 2023-09-10T00:47:36.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:35 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:36.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:35 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:36.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:35 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:36 smithi131 ceph-mon[103713]: pgmap v213: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 4.3 MiB/s wr, 502 op/s 2023-09-10T00:47:37.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:36 smithi044 ceph-mon[98676]: pgmap v213: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 4.3 MiB/s wr, 502 op/s 2023-09-10T00:47:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:36 smithi099 ceph-mon[102692]: pgmap v213: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 4.3 MiB/s wr, 502 op/s 2023-09-10T00:47:38.897 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:47:38.923 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:38 smithi044 ceph-mon[98676]: pgmap v214: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 2.9 MiB/s wr, 334 op/s 2023-09-10T00:47:39.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.041+0000 7ff0e5fa2700 1 -- 172.21.15.44:0/12573955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e01065a0 msgr2=0x7ff0e0106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:39.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.041+0000 7ff0e5fa2700 1 --2- 172.21.15.44:0/12573955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e01065a0 0x7ff0e0106980 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009990 tx=0x7ff0d4009ca0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.041+0000 7ff0e5fa2700 1 -- 172.21.15.44:0/12573955 shutdown_connections 2023-09-10T00:47:39.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.041+0000 7ff0e5fa2700 1 --2- 172.21.15.44:0/12573955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e00ff670 0x7ff0e010d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.042+0000 7ff0e5fa2700 1 --2- 172.21.15.44:0/12573955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e01065a0 0x7ff0e0106980 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.042+0000 7ff0e5fa2700 1 --2- 172.21.15.44:0/12573955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e00fecb0 0x7ff0e00ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.042+0000 7ff0e5fa2700 1 -- 172.21.15.44:0/12573955 >> 172.21.15.44:0/12573955 conn(0x7ff0e0074590 msgr2=0x7ff0e00749a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:39.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.042+0000 7ff0e5fa2700 1 -- 172.21.15.44:0/12573955 shutdown_connections 2023-09-10T00:47:39.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.042+0000 7ff0e5fa2700 1 -- 172.21.15.44:0/12573955 wait complete. 2023-09-10T00:47:39.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.042+0000 7ff0e5fa2700 1 Processor -- start 2023-09-10T00:47:39.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.042+0000 7ff0e5fa2700 1 -- start start 2023-09-10T00:47:39.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.042+0000 7ff0e5fa2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e00fecb0 0x7ff0e01962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.042+0000 7ff0e5fa2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e00ff670 0x7ff0e0196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.046 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.042+0000 7ff0e5fa2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e01065a0 0x7ff0e019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.042+0000 7ff0e5fa2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e019b2c0 con 0x7ff0e01065a0 2023-09-10T00:47:39.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.042+0000 7ff0e5fa2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e019b430 con 0x7ff0e00fecb0 2023-09-10T00:47:39.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0e5fa2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e019b5f0 con 0x7ff0e00ff670 2023-09-10T00:47:39.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0e57a1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e01065a0 0x7ff0e019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0e4fa0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e00fecb0 0x7ff0e01962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.047 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0dffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e00ff670 0x7ff0e0196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.048 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0e57a1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e01065a0 0x7ff0e019ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51510/0 (socket says 172.21.15.44:51510) 2023-09-10T00:47:39.048 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0e57a1700 1 -- 172.21.15.44:0/3882285808 learned_addr learned my addr 172.21.15.44:0/3882285808 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:39.048 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0e4fa0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e00fecb0 0x7ff0e01962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53246/0 (socket says 172.21.15.44:53246) 2023-09-10T00:47:39.048 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0dffff700 1 -- 172.21.15.44:0/3882285808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e00fecb0 msgr2=0x7ff0e01962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:39.048 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0dffff700 1 --2- 172.21.15.44:0/3882285808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e00fecb0 0x7ff0e01962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.049 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0dffff700 1 -- 172.21.15.44:0/3882285808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e01065a0 msgr2=0x7ff0e019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:39.049 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0dffff700 1 --2- 172.21.15.44:0/3882285808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e01065a0 0x7ff0e019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.049 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0dffff700 1 -- 172.21.15.44:0/3882285808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d0009520 con 0x7ff0e00ff670 2023-09-10T00:47:39.049 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0dffff700 1 --2- 172.21.15.44:0/3882285808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e00ff670 0x7ff0e0196820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000e9f0 tx=0x7ff0d000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:39.049 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0ddffb700 1 -- 172.21.15.44:0/3882285808 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d000cb70 con 0x7ff0e00ff670 2023-09-10T00:47:39.049 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0e5fa2700 1 -- 172.21.15.44:0/3882285808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d40095f0 con 0x7ff0e00ff670 2023-09-10T00:47:39.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0ddffb700 1 -- 172.21.15.44:0/3882285808 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0d000ee90 con 0x7ff0e00ff670 2023-09-10T00:47:39.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.043+0000 7ff0ddffb700 1 -- 172.21.15.44:0/3882285808 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0003b60 con 0x7ff0e00ff670 2023-09-10T00:47:39.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.044+0000 7ff0e5fa2700 1 -- 172.21.15.44:0/3882285808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e019bac0 con 0x7ff0e00ff670 2023-09-10T00:47:39.051 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:38 smithi131 ceph-mon[103713]: pgmap v214: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 2.9 MiB/s wr, 334 op/s 2023-09-10T00:47:39.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.047+0000 7ff0ddffb700 1 -- 172.21.15.44:0/3882285808 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ff0d0003d40 con 0x7ff0e00ff670 2023-09-10T00:47:39.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.047+0000 7ff0ddffb700 1 --2- 172.21.15.44:0/3882285808 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff0c80a0850 0x7ff0c80a2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.048+0000 7ff0e4fa0700 1 --2- 172.21.15.44:0/3882285808 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff0c80a0850 0x7ff0c80a2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.048+0000 7ff0ddffb700 1 -- 172.21.15.44:0/3882285808 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff0d0014070 con 0x7ff0e00ff670 2023-09-10T00:47:39.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.048+0000 7ff0f33cab80 1 Processor -- start 2023-09-10T00:47:39.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.048+0000 7ff0f33cab80 1 -- start start 2023-09-10T00:47:39.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.048+0000 7ff0e4fa0700 1 --2- 172.21.15.44:0/3882285808 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff0c80a0850 0x7ff0c80a2d10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4006010 tx=0x7ff0d400b260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:39.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.048+0000 7ff0f33cab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176dd860 0x5614176ddc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.048+0000 7ff0f33cab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176de180 0x5614176e8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.048+0000 7ff0f33cab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176e9030 0x5614176eb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.048+0000 7ff0f33cab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561417637730 con 0x5614176e9030 2023-09-10T00:47:39.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.048+0000 7ff0f33cab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561417635050 con 0x5614176dd860 2023-09-10T00:47:39.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.048+0000 7ff0f33cab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561417630940 con 0x5614176de180 2023-09-10T00:47:39.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.048+0000 7ff0e4fa0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176dd860 0x5614176ddc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.048+0000 7ff0e57a1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176e9030 0x5614176eb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.049+0000 7ff0e4fa0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176dd860 0x5614176ddc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53254/0 (socket says 172.21.15.44:53254) 2023-09-10T00:47:39.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.049+0000 7ff0e4fa0700 1 -- 172.21.15.44:0/3816346432 learned_addr learned my addr 172.21.15.44:0/3816346432 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:39.059 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.049+0000 7ff0dffff700 1 --2- 172.21.15.44:0/3816346432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176de180 0x5614176e8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.049+0000 7ff0dffff700 1 -- 172.21.15.44:0/3816346432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176dd860 msgr2=0x5614176ddc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:39.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.049+0000 7ff0dffff700 1 --2- 172.21.15.44:0/3816346432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176dd860 0x5614176ddc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.049+0000 7ff0dffff700 1 -- 172.21.15.44:0/3816346432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176e9030 msgr2=0x5614176eb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:39.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.049+0000 7ff0dffff700 1 --2- 172.21.15.44:0/3816346432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176e9030 0x5614176eb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.049+0000 7ff0dffff700 1 -- 172.21.15.44:0/3816346432 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141745cde0 con 0x5614176de180 2023-09-10T00:47:39.060 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.049+0000 7ff0e4fa0700 1 --2- 172.21.15.44:0/3816346432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176dd860 0x5614176ddc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:39.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.049+0000 7ff0dffff700 1 --2- 172.21.15.44:0/3816346432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176de180 0x5614176e8af0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00c9950 tx=0x7ff0d0099300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d34dc666e1057802 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:39.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.049+0000 7ff0c6ffd700 1 -- 172.21.15.44:0/3816346432 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d00a1720 con 0x5614176de180 2023-09-10T00:47:39.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.049+0000 7ff0c6ffd700 1 -- 172.21.15.44:0/3816346432 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0d00cc070 con 0x5614176de180 2023-09-10T00:47:39.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.049+0000 7ff0c6ffd700 1 -- 172.21.15.44:0/3816346432 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d00cc200 con 0x5614176de180 2023-09-10T00:47:39.061 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.050+0000 7ff0f33cab80 1 -- 172.21.15.44:0/3816346432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176de180 msgr2=0x5614176e8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:39.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.050+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/3816346432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176de180 0x5614176e8af0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00c9950 tx=0x7ff0d0099300 comp rx=0 tx=0).stop 2023-09-10T00:47:39.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.050+0000 7ff0f33cab80 1 -- 172.21.15.44:0/3816346432 shutdown_connections 2023-09-10T00:47:39.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.050+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/3816346432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176e9030 0x5614176eb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.050+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/3816346432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176dd860 0x5614176ddc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.050+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/3816346432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176de180 0x5614176e8af0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.062 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.050+0000 7ff0f33cab80 1 -- 172.21.15.44:0/3816346432 >> 172.21.15.44:0/3816346432 conn(0x561417653090 msgr2=0x5614176d9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:39.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.050+0000 7ff0f33cab80 1 -- 172.21.15.44:0/3816346432 shutdown_connections 2023-09-10T00:47:39.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.050+0000 7ff0f33cab80 1 -- 172.21.15.44:0/3816346432 wait complete. 2023-09-10T00:47:39.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.050+0000 7ff0f33cab80 1 -- start start 2023-09-10T00:47:39.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.050+0000 7ff0f33cab80 1 Processor -- start 2023-09-10T00:47:39.063 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0f33cab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176dd860 0x5614176e5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0f33cab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176de180 0x5614176e6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0f33cab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176e9030 0x5614176e67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0f33cab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614175ceb70 con 0x5614176dd860 2023-09-10T00:47:39.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0f33cab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614175ce870 con 0x5614176de180 2023-09-10T00:47:39.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0f33cab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614174b2920 con 0x5614176e9030 2023-09-10T00:47:39.064 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0f33cab80 10 client.0 _ll_register_callbacks cb 0x561417518a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:47:39.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0e4fa0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176dd860 0x5614176e5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0e4fa0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176dd860 0x5614176e5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51520/0 (socket says 172.21.15.44:51520) 2023-09-10T00:47:39.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0dffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176de180 0x5614176e6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0e4fa0700 1 -- 172.21.15.44:0/2822615759 learned_addr learned my addr 172.21.15.44:0/2822615759 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:39.065 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0e57a1700 1 --2- 172.21.15.44:0/2822615759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176e9030 0x5614176e67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0e57a1700 1 -- 172.21.15.44:0/2822615759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176de180 msgr2=0x5614176e6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:39.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0e57a1700 1 --2- 172.21.15.44:0/2822615759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176de180 0x5614176e6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0e57a1700 1 -- 172.21.15.44:0/2822615759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176dd860 msgr2=0x5614176e5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:39.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0e57a1700 1 --2- 172.21.15.44:0/2822615759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176dd860 0x5614176e5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0e57a1700 1 -- 172.21.15.44:0/2822615759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e010db20 con 0x5614176e9030 2023-09-10T00:47:39.066 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.051+0000 7ff0e4fa0700 1 --2- 172.21.15.44:0/2822615759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176dd860 0x5614176e5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:39.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.052+0000 7ff0e57a1700 1 --2- 172.21.15.44:0/2822615759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176e9030 0x5614176e67d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009c10 tx=0x7ff0d800e610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d015627417933776 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:39.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.052+0000 7ff0bffff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:47:39.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.052+0000 7ff0bffff700 1 -- 172.21.15.44:0/2822615759 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d800e040 con 0x5614176e9030 2023-09-10T00:47:39.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.052+0000 7ff0f33cab80 1 -- 172.21.15.44:0/2822615759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614175884f0 con 0x5614176e9030 2023-09-10T00:47:39.067 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.052+0000 7ff0bffff700 1 -- 172.21.15.44:0/2822615759 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff0d801a070 con 0x5614176e9030 2023-09-10T00:47:39.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.052+0000 7ff0bffff700 1 -- 172.21.15.44:0/2822615759 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0d8004bb0 con 0x5614176e9030 2023-09-10T00:47:39.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.052+0000 7ff0f33cab80 10 client.26557 fetch_fsmap learned FSMap version 50 2023-09-10T00:47:39.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.052+0000 7ff0f33cab80 1 -- 172.21.15.44:0/2822615759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7ff0e019bac0 con 0x5614176e9030 2023-09-10T00:47:39.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.052+0000 7ff0bffff700 1 -- 172.21.15.44:0/2822615759 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5614175884f0 con 0x5614176e9030 2023-09-10T00:47:39.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.052+0000 7ff0bffff700 1 -- 172.21.15.44:0/2822615759 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 50) v1 ==== 7930+0+0 (secure 0 0 0) 0x7ff0d80079f0 con 0x5614176e9030 2023-09-10T00:47:39.068 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.052+0000 7ff0f33cab80 10 client.26557 fetch_fsmap finished waiting for FSMap version 50 2023-09-10T00:47:39.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.052+0000 7ff0f33cab80 10 client.26557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:47:39.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.052+0000 7ff0f33cab80 20 client.26557 populate_metadata read hostname 'smithi044' 2023-09-10T00:47:39.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.052+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/2822615759 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5614176f8bd0 0x5614176faff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.052+0000 7ff0f33cab80 4 client.26557 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:47:39.069 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.052+0000 7ff0f33cab80 1 -- 172.21.15.44:0/2822615759 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614176458a0 con 0x5614176f8bd0 2023-09-10T00:47:39.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.053+0000 7ff0e4fa0700 1 --2- 172.21.15.44:0/2822615759 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5614176f8bd0 0x5614176faff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.053+0000 7ff0bffff700 10 client.26557 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:39.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.053+0000 7ff0e4fa0700 1 --2- 172.21.15.44:0/2822615759 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5614176f8bd0 0x5614176faff0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=684e18e89d53dff1 server_cookie=5cb81ff005927114 in_seq=0 out_seq=0 2023-09-10T00:47:39.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.054+0000 7ff0bffff700 1 -- 172.21.15.44:0/2822615759 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5614176458a0 con 0x5614176f8bd0 2023-09-10T00:47:39.070 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.054+0000 7ff0bffff700 10 client.26557 handle_command_reply: tid=0 2023-09-10T00:47:39.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.054+0000 7ff0f33cab80 1 client.26557 shutdown 2023-09-10T00:47:39.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.055+0000 7ff0f33cab80 1 -- 172.21.15.44:0/2822615759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176e9030 msgr2=0x5614176e67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:39.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.055+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/2822615759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176e9030 0x5614176e67d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009c10 tx=0x7ff0d800e610 comp rx=0 tx=0).stop 2023-09-10T00:47:39.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.055+0000 7ff0f33cab80 1 -- 172.21.15.44:0/2822615759 shutdown_connections 2023-09-10T00:47:39.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.055+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/2822615759 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5614176f8bd0 0x5614176faff0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.071 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.055+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/2822615759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176e9030 0x5614176e67d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.055+0000 7ff0bffff700 0 client.26557 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:39.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.055+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/2822615759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176dd860 0x5614176e5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.055+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/2822615759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176de180 0x5614176e6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.055+0000 7ff0f33cab80 1 -- 172.21.15.44:0/2822615759 >> 172.21.15.44:0/2822615759 conn(0x561417653090 msgr2=0x5614176d98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:39.072 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.055+0000 7ff0f33cab80 1 -- 172.21.15.44:0/2822615759 shutdown_connections 2023-09-10T00:47:39.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.055+0000 7ff0f33cab80 1 -- 172.21.15.44:0/2822615759 wait complete. 2023-09-10T00:47:39.073 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.055+0000 7ff0f33cab80 20 client.26557 trim_cache size 0 max 16384 2023-09-10T00:47:39.075 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.075+0000 7ff0f33cab80 1 Processor -- start 2023-09-10T00:47:39.076 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.075+0000 7ff0f33cab80 1 -- start start 2023-09-10T00:47:39.076 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.075+0000 7ff0f33cab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176dd860 0x5614176e0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.076 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.075+0000 7ff0f33cab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176de180 0x5614176e6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.076 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.075+0000 7ff0f33cab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176e9030 0x5614176e2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.076 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.075+0000 7ff0f33cab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614176db040 con 0x5614176e9030 2023-09-10T00:47:39.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.075+0000 7ff0f33cab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614176f61b0 con 0x5614176de180 2023-09-10T00:47:39.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.075+0000 7ff0f33cab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614176db200 con 0x5614176dd860 2023-09-10T00:47:39.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.075+0000 7ff0e4fa0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176dd860 0x5614176e0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.075+0000 7ff0e57a1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176e9030 0x5614176e2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.077 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.075+0000 7ff0dffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176de180 0x5614176e6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.078 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.075+0000 7ff0e4fa0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176dd860 0x5614176e0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58410/0 (socket says 172.21.15.44:58410) 2023-09-10T00:47:39.078 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.075+0000 7ff0e4fa0700 1 -- 172.21.15.44:0/1288879988 learned_addr learned my addr 172.21.15.44:0/1288879988 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:39.081 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.076+0000 7ff0e4fa0700 1 -- 172.21.15.44:0/1288879988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176de180 msgr2=0x5614176e6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:39.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.076+0000 7ff0e4fa0700 1 --2- 172.21.15.44:0/1288879988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176de180 0x5614176e6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.076+0000 7ff0e4fa0700 1 -- 172.21.15.44:0/1288879988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176e9030 msgr2=0x5614176e2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:39.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.076+0000 7ff0e4fa0700 1 --2- 172.21.15.44:0/1288879988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176e9030 0x5614176e2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.076+0000 7ff0e4fa0700 1 -- 172.21.15.44:0/1288879988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614176ffbe0 con 0x5614176dd860 2023-09-10T00:47:39.082 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.076+0000 7ff0e4fa0700 1 --2- 172.21.15.44:0/1288879988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176dd860 0x5614176e0700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400b650 tx=0x7ff0d4023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f6318c1885238fe server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:39.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.076+0000 7ff0c77fe700 1 -- 172.21.15.44:0/1288879988 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4036de0 con 0x5614176dd860 2023-09-10T00:47:39.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.076+0000 7ff0c77fe700 1 -- 172.21.15.44:0/1288879988 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0d401e040 con 0x5614176dd860 2023-09-10T00:47:39.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.076+0000 7ff0c77fe700 1 -- 172.21.15.44:0/1288879988 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400f460 con 0x5614176dd860 2023-09-10T00:47:39.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.077+0000 7ff0f33cab80 1 -- 172.21.15.44:0/1288879988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176dd860 msgr2=0x5614176e0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:39.083 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.077+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/1288879988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176dd860 0x5614176e0700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400b650 tx=0x7ff0d4023000 comp rx=0 tx=0).stop 2023-09-10T00:47:39.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.077+0000 7ff0f33cab80 1 -- 172.21.15.44:0/1288879988 shutdown_connections 2023-09-10T00:47:39.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.077+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/1288879988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176e9030 0x5614176e2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.077+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/1288879988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176dd860 0x5614176e0700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.077+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/1288879988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176de180 0x5614176e6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.077+0000 7ff0f33cab80 1 -- 172.21.15.44:0/1288879988 >> 172.21.15.44:0/1288879988 conn(0x561417653090 msgr2=0x5614176d9820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:39.084 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.077+0000 7ff0f33cab80 1 -- 172.21.15.44:0/1288879988 shutdown_connections 2023-09-10T00:47:39.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.077+0000 7ff0f33cab80 1 -- 172.21.15.44:0/1288879988 wait complete. 2023-09-10T00:47:39.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.077+0000 7ff0f33cab80 1 -- start start 2023-09-10T00:47:39.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.077+0000 7ff0f33cab80 1 Processor -- start 2023-09-10T00:47:39.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0f33cab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176dd860 0x5614176efb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0f33cab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176de180 0x5614176ede90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.085 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0f33cab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176e9030 0x5614176ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0f33cab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614176eeae0 con 0x5614176dd860 2023-09-10T00:47:39.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0f33cab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614176eec50 con 0x5614176e9030 2023-09-10T00:47:39.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0f33cab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561417750de0 con 0x5614176de180 2023-09-10T00:47:39.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0f33cab80 10 client.0 _ll_register_callbacks cb 0x7ff0d8014700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:47:39.086 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0e4fa0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176dd860 0x5614176efb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0e4fa0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176dd860 0x5614176efb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51534/0 (socket says 172.21.15.44:51534) 2023-09-10T00:47:39.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0dffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176de180 0x5614176ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0e4fa0700 1 -- 172.21.15.44:0/53678611 learned_addr learned my addr 172.21.15.44:0/53678611 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:39.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0e57a1700 1 --2- 172.21.15.44:0/53678611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176e9030 0x5614176ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.087 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0e4fa0700 1 -- 172.21.15.44:0/53678611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176de180 msgr2=0x5614176ede90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:39.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0e4fa0700 1 --2- 172.21.15.44:0/53678611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176de180 0x5614176ede90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0e4fa0700 1 -- 172.21.15.44:0/53678611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176e9030 msgr2=0x5614176ee3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:39.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0e4fa0700 1 --2- 172.21.15.44:0/53678611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176e9030 0x5614176ee3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0e4fa0700 1 -- 172.21.15.44:0/53678611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d40054d0 con 0x5614176dd860 2023-09-10T00:47:39.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0e4fa0700 1 --2- 172.21.15.44:0/53678611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176dd860 0x5614176efb60 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4039040 tx=0x7ff0d400bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61728154c3494c20 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:39.088 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.078+0000 7ff0bffff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:47:39.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.079+0000 7ff0bffff700 1 -- 172.21.15.44:0/53678611 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4004860 con 0x5614176dd860 2023-09-10T00:47:39.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.079+0000 7ff0bffff700 1 -- 172.21.15.44:0/53678611 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff0d40049c0 con 0x5614176dd860 2023-09-10T00:47:39.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.079+0000 7ff0f33cab80 1 -- 172.21.15.44:0/53678611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0d00cc070 con 0x5614176dd860 2023-09-10T00:47:39.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.079+0000 7ff0bffff700 1 -- 172.21.15.44:0/53678611 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0d4004b80 con 0x5614176dd860 2023-09-10T00:47:39.089 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.079+0000 7ff0f33cab80 10 client.16959 fetch_fsmap learned FSMap version 50 2023-09-10T00:47:39.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.079+0000 7ff0f33cab80 1 -- 172.21.15.44:0/53678611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x561417445c60 con 0x5614176dd860 2023-09-10T00:47:39.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.079+0000 7ff0bffff700 1 -- 172.21.15.44:0/53678611 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0d00cc070 con 0x5614176dd860 2023-09-10T00:47:39.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.079+0000 7ff0bffff700 1 -- 172.21.15.44:0/53678611 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 50) v1 ==== 7930+0+0 (secure 0 0 0) 0x7ff0d402f030 con 0x5614176dd860 2023-09-10T00:47:39.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.079+0000 7ff0f33cab80 10 client.16959 fetch_fsmap finished waiting for FSMap version 50 2023-09-10T00:47:39.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.079+0000 7ff0f33cab80 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:47:39.090 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.079+0000 7ff0f33cab80 20 client.16959 populate_metadata read hostname 'smithi044' 2023-09-10T00:47:39.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.079+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/53678611 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561417752050 0x561417754470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:39.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.079+0000 7ff0f33cab80 4 client.16959 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:47:39.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.079+0000 7ff0f33cab80 1 -- 172.21.15.44:0/53678611 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561417629920 con 0x561417752050 2023-09-10T00:47:39.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.079+0000 7ff0dffff700 1 --2- 172.21.15.44:0/53678611 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561417752050 0x561417754470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:39.091 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.080+0000 7ff0bffff700 10 client.16959 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:39.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.080+0000 7ff0dffff700 1 --2- 172.21.15.44:0/53678611 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561417752050 0x561417754470 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45b226be4f748f server_cookie=a996d085324f1097 in_seq=0 out_seq=0 2023-09-10T00:47:39.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.080+0000 7ff0bffff700 1 -- 172.21.15.44:0/53678611 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561417629920 con 0x561417752050 2023-09-10T00:47:39.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.080+0000 7ff0bffff700 10 client.16959 handle_command_reply: tid=0 2023-09-10T00:47:39.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.080+0000 7ff0f33cab80 1 client.16959 shutdown 2023-09-10T00:47:39.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.080+0000 7ff0f33cab80 1 -- 172.21.15.44:0/53678611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176dd860 msgr2=0x5614176efb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:39.092 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.080+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/53678611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176dd860 0x5614176efb60 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4039040 tx=0x7ff0d400bd10 comp rx=0 tx=0).stop 2023-09-10T00:47:39.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.080+0000 7ff0f33cab80 1 -- 172.21.15.44:0/53678611 shutdown_connections 2023-09-10T00:47:39.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.080+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/53678611 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561417752050 0x561417754470 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.080+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/53678611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614176e9030 0x5614176ee3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.080+0000 7ff0bffff700 0 client.16959 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:39.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.080+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/53678611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614176dd860 0x5614176efb60 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.080+0000 7ff0f33cab80 1 --2- 172.21.15.44:0/53678611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614176de180 0x5614176ede90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.080+0000 7ff0f33cab80 1 -- 172.21.15.44:0/53678611 >> 172.21.15.44:0/53678611 conn(0x561417653090 msgr2=0x5614176e7660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:39.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.080+0000 7ff0f33cab80 1 -- 172.21.15.44:0/53678611 shutdown_connections 2023-09-10T00:47:39.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.080+0000 7ff0f33cab80 1 -- 172.21.15.44:0/53678611 wait complete. 2023-09-10T00:47:39.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.081+0000 7ff0f33cab80 20 client.16959 trim_cache size 0 max 16384 2023-09-10T00:47:39.095 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:47:39.095 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:47:39.095 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:47:39.095 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:47:39.095 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:47:39.096 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:47:39.096 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:47:39.096 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:47:39.096 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:47:39.096 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:47:39.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.081+0000 7ff0e5fa2700 1 -- 172.21.15.44:0/3882285808 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff0c80a0850 msgr2=0x7ff0c80a2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:39.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.081+0000 7ff0e5fa2700 1 --2- 172.21.15.44:0/3882285808 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff0c80a0850 0x7ff0c80a2d10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4006010 tx=0x7ff0d400b260 comp rx=0 tx=0).stop 2023-09-10T00:47:39.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.081+0000 7ff0e5fa2700 1 -- 172.21.15.44:0/3882285808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e00ff670 msgr2=0x7ff0e0196820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:39.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.081+0000 7ff0e5fa2700 1 --2- 172.21.15.44:0/3882285808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e00ff670 0x7ff0e0196820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000e9f0 tx=0x7ff0d000ed00 comp rx=0 tx=0).stop 2023-09-10T00:47:39.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.081+0000 7ff0e5fa2700 1 -- 172.21.15.44:0/3882285808 shutdown_connections 2023-09-10T00:47:39.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.081+0000 7ff0e5fa2700 1 --2- 172.21.15.44:0/3882285808 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff0c80a0850 0x7ff0c80a2d10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.081+0000 7ff0e5fa2700 1 --2- 172.21.15.44:0/3882285808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e01065a0 0x7ff0e019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.081+0000 7ff0e5fa2700 1 --2- 172.21.15.44:0/3882285808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e00fecb0 0x7ff0e01962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.081+0000 7ff0e5fa2700 1 --2- 172.21.15.44:0/3882285808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e00ff670 0x7ff0e0196820 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:39.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.081+0000 7ff0e5fa2700 1 -- 172.21.15.44:0/3882285808 >> 172.21.15.44:0/3882285808 conn(0x7ff0e0074590 msgr2=0x7ff0e00fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:39.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.081+0000 7ff0e5fa2700 1 -- 172.21.15.44:0/3882285808 shutdown_connections 2023-09-10T00:47:39.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:47:39.081+0000 7ff0e5fa2700 1 -- 172.21.15.44:0/3882285808 wait complete. 2023-09-10T00:47:39.100 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:47:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:38 smithi099 ceph-mon[102692]: pgmap v214: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 2.9 MiB/s wr, 334 op/s 2023-09-10T00:47:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:40 smithi131 ceph-mon[103713]: pgmap v215: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 2.9 MiB/s wr, 334 op/s 2023-09-10T00:47:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:40 smithi044 ceph-mon[98676]: pgmap v215: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 2.9 MiB/s wr, 334 op/s 2023-09-10T00:47:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:40 smithi099 ceph-mon[102692]: pgmap v215: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 2.9 MiB/s wr, 334 op/s 2023-09-10T00:47:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:42 smithi131 ceph-mon[103713]: pgmap v216: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 4.2 MiB/s wr, 493 op/s 2023-09-10T00:47:43.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:42 smithi044 ceph-mon[98676]: pgmap v216: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 4.2 MiB/s wr, 493 op/s 2023-09-10T00:47:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:42 smithi099 ceph-mon[102692]: pgmap v216: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 4.2 MiB/s wr, 493 op/s 2023-09-10T00:47:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.734+0000 7f80da5d9700 1 -- 172.21.15.44:0/1246445835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d4105590 msgr2=0x7f80d4105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.734+0000 7f80da5d9700 1 --2- 172.21.15.44:0/1246445835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d4105590 0x7f80d4105970 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f80c40099c0 tx=0x7f80c4009cd0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.734+0000 7f80da5d9700 1 -- 172.21.15.44:0/1246445835 shutdown_connections 2023-09-10T00:47:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.734+0000 7f80da5d9700 1 --2- 172.21.15.44:0/1246445835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80d40ffe90 0x7f80d410c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.734+0000 7f80da5d9700 1 --2- 172.21.15.44:0/1246445835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d4105590 0x7f80d4105970 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.734+0000 7f80da5d9700 1 --2- 172.21.15.44:0/1246445835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80d40ff4d0 0x7f80d40ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.734+0000 7f80da5d9700 1 -- 172.21.15.44:0/1246445835 >> 172.21.15.44:0/1246445835 conn(0x7f80d40faf40 msgr2=0x7f80d40fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.734+0000 7f80da5d9700 1 -- 172.21.15.44:0/1246445835 shutdown_connections 2023-09-10T00:47:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.734+0000 7f80da5d9700 1 -- 172.21.15.44:0/1246445835 wait complete. 2023-09-10T00:47:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.735+0000 7f80da5d9700 1 Processor -- start 2023-09-10T00:47:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.735+0000 7f80da5d9700 1 -- start start 2023-09-10T00:47:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.736+0000 7f80da5d9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80d40ff4d0 0x7f80d4195300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.736+0000 7f80da5d9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d40ffe90 0x7f80d4195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.736+0000 7f80da5d9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80d4105590 0x7f80d419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.736+0000 7f80da5d9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d419a770 con 0x7f80d40ffe90 2023-09-10T00:47:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.736+0000 7f80da5d9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d419a8e0 con 0x7f80d40ff4d0 2023-09-10T00:47:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.736+0000 7f80da5d9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d419aaa0 con 0x7f80d4105590 2023-09-10T00:47:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.736+0000 7f80d3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80d40ff4d0 0x7f80d4195300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.736+0000 7f80d8b76700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80d4105590 0x7f80d419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.736+0000 7f80d3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80d40ff4d0 0x7f80d4195300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53288/0 (socket says 172.21.15.44:53288) 2023-09-10T00:47:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.736+0000 7f80d3fff700 1 -- 172.21.15.44:0/424727182 learned_addr learned my addr 172.21.15.44:0/424727182 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.736+0000 7f80d37fe700 1 --2- 172.21.15.44:0/424727182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d40ffe90 0x7f80d4195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.736+0000 7f80d8b76700 1 -- 172.21.15.44:0/424727182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80d40ff4d0 msgr2=0x7f80d4195300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.736+0000 7f80d8b76700 1 --2- 172.21.15.44:0/424727182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80d40ff4d0 0x7f80d4195300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.736+0000 7f80d8b76700 1 -- 172.21.15.44:0/424727182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d40ffe90 msgr2=0x7f80d4195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.736+0000 7f80d8b76700 1 --2- 172.21.15.44:0/424727182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d40ffe90 0x7f80d4195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.736+0000 7f80d8b76700 1 -- 172.21.15.44:0/424727182 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c40095f0 con 0x7f80d4105590 2023-09-10T00:47:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.737+0000 7f80d37fe700 1 --2- 172.21.15.44:0/424727182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d40ffe90 0x7f80d4195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.737+0000 7f80d8b76700 1 --2- 172.21.15.44:0/424727182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80d4105590 0x7f80d419a170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00efc0 tx=0x7f80cc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.737+0000 7f80d17fa700 1 -- 172.21.15.44:0/424727182 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc018070 con 0x7f80d4105590 2023-09-10T00:47:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.737+0000 7f80da5d9700 1 -- 172.21.15.44:0/424727182 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d419ad20 con 0x7f80d4105590 2023-09-10T00:47:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.737+0000 7f80d17fa700 1 -- 172.21.15.44:0/424727182 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80cc00f040 con 0x7f80d4105590 2023-09-10T00:47:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.737+0000 7f80d17fa700 1 -- 172.21.15.44:0/424727182 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc00cba0 con 0x7f80d4105590 2023-09-10T00:47:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.737+0000 7f80da5d9700 1 -- 172.21.15.44:0/424727182 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d419b230 con 0x7f80d4105590 2023-09-10T00:47:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.740+0000 7f80d17fa700 1 -- 172.21.15.44:0/424727182 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f80cc003a00 con 0x7f80d4105590 2023-09-10T00:47:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.740+0000 7f80d17fa700 1 --2- 172.21.15.44:0/424727182 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80bc0a07b0 0x7f80bc0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.741+0000 7f80d17fa700 1 -- 172.21.15.44:0/424727182 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f80cc0cbe60 con 0x7f80d4105590 2023-09-10T00:47:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.741+0000 7f80d3fff700 1 --2- 172.21.15.44:0/424727182 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80bc0a07b0 0x7f80bc0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.741+0000 7f80d3fff700 1 --2- 172.21.15.44:0/424727182 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80bc0a07b0 0x7f80bc0a2c70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f80c4000c00 tx=0x7f80c4005fb0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.742+0000 7f80e679fb80 1 Processor -- start 2023-09-10T00:47:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.742+0000 7f80e679fb80 1 -- start start 2023-09-10T00:47:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.742+0000 7f80e679fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f3a01a0 0x55c48f3a0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.742+0000 7f80e679fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39a300 0x55c48f39a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.742+0000 7f80e679fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f39acc0 0x55c48f3a6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.742+0000 7f80e679fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48f2f36d0 con 0x55c48f39a300 2023-09-10T00:47:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.742+0000 7f80e679fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48f2f0ff0 con 0x55c48f3a01a0 2023-09-10T00:47:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.742+0000 7f80e679fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48f2ec8e0 con 0x55c48f39acc0 2023-09-10T00:47:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.742+0000 7f80d37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39a300 0x55c48f39a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.742+0000 7f80d3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f3a01a0 0x55c48f3a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.742+0000 7f80d37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39a300 0x55c48f39a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51558/0 (socket says 172.21.15.44:51558) 2023-09-10T00:47:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.742+0000 7f80d3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f3a01a0 0x55c48f3a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53296/0 (socket says 172.21.15.44:53296) 2023-09-10T00:47:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.742+0000 7f80d8b76700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f39acc0 0x55c48f3a6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.742+0000 7f80d37fe700 1 -- 172.21.15.44:0/1439724077 learned_addr learned my addr 172.21.15.44:0/1439724077 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.743+0000 7f80d8b76700 1 -- 172.21.15.44:0/1439724077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f3a01a0 msgr2=0x55c48f3a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.743+0000 7f80d8b76700 1 --2- 172.21.15.44:0/1439724077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f3a01a0 0x55c48f3a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.743+0000 7f80d8b76700 1 -- 172.21.15.44:0/1439724077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39a300 msgr2=0x55c48f39a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.743+0000 7f80d8b76700 1 --2- 172.21.15.44:0/1439724077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39a300 0x55c48f39a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.743+0000 7f80d8b76700 1 -- 172.21.15.44:0/1439724077 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c48f118d80 con 0x55c48f39acc0 2023-09-10T00:47:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.743+0000 7f80d37fe700 1 --2- 172.21.15.44:0/1439724077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39a300 0x55c48f39a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.743+0000 7f80d8b76700 1 --2- 172.21.15.44:0/1439724077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f39acc0 0x55c48f3a6f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0c91b0 tx=0x7f80cc098900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc3eac877c6b0a8b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.743+0000 7f80ba7fc700 1 -- 172.21.15.44:0/1439724077 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc0996a0 con 0x55c48f39acc0 2023-09-10T00:47:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.743+0000 7f80ba7fc700 1 -- 172.21.15.44:0/1439724077 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80cc098fa0 con 0x55c48f39acc0 2023-09-10T00:47:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.743+0000 7f80ba7fc700 1 -- 172.21.15.44:0/1439724077 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc099200 con 0x55c48f39acc0 2023-09-10T00:47:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.744+0000 7f80e679fb80 1 -- 172.21.15.44:0/1439724077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f39acc0 msgr2=0x55c48f3a6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.744+0000 7f80e679fb80 1 --2- 172.21.15.44:0/1439724077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f39acc0 0x55c48f3a6f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0c91b0 tx=0x7f80cc098900 comp rx=0 tx=0).stop 2023-09-10T00:47:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.744+0000 7f80e679fb80 1 -- 172.21.15.44:0/1439724077 shutdown_connections 2023-09-10T00:47:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.744+0000 7f80e679fb80 1 --2- 172.21.15.44:0/1439724077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f39acc0 0x55c48f3a6f30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.744+0000 7f80e679fb80 1 --2- 172.21.15.44:0/1439724077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f3a01a0 0x55c48f3a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.744+0000 7f80e679fb80 1 --2- 172.21.15.44:0/1439724077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39a300 0x55c48f39a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.744+0000 7f80e679fb80 1 -- 172.21.15.44:0/1439724077 >> 172.21.15.44:0/1439724077 conn(0x55c48f30f030 msgr2=0x55c48f395560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.744+0000 7f80e679fb80 1 -- 172.21.15.44:0/1439724077 shutdown_connections 2023-09-10T00:47:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.744+0000 7f80e679fb80 1 -- 172.21.15.44:0/1439724077 wait complete. 2023-09-10T00:47:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.744+0000 7f80e679fb80 1 -- start start 2023-09-10T00:47:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.745+0000 7f80e679fb80 1 Processor -- start 2023-09-10T00:47:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.745+0000 7f80e679fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39a300 0x55c48f3a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.745+0000 7f80e679fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f39acc0 0x55c48f3aa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.745+0000 7f80e679fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f3a01a0 0x55c48f3a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.745+0000 7f80e679fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48f28ab10 con 0x55c48f39a300 2023-09-10T00:47:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.745+0000 7f80e679fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48f28a810 con 0x55c48f39acc0 2023-09-10T00:47:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.745+0000 7f80e679fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48f16e8c0 con 0x55c48f3a01a0 2023-09-10T00:47:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.745+0000 7f80e679fb80 10 client.0 _ll_register_callbacks cb 0x55c48f1d4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:47:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.745+0000 7f80d3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39a300 0x55c48f3a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.745+0000 7f80d3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39a300 0x55c48f3a9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51562/0 (socket says 172.21.15.44:51562) 2023-09-10T00:47:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.745+0000 7f80d3fff700 1 -- 172.21.15.44:0/1732409120 learned_addr learned my addr 172.21.15.44:0/1732409120 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.745+0000 7f80d8b76700 1 --2- 172.21.15.44:0/1732409120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f3a01a0 0x55c48f3a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.745+0000 7f80d37fe700 1 --2- 172.21.15.44:0/1732409120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f39acc0 0x55c48f3aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80d3fff700 1 -- 172.21.15.44:0/1732409120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f3a01a0 msgr2=0x55c48f3a40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80d3fff700 1 --2- 172.21.15.44:0/1732409120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f3a01a0 0x55c48f3a40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80d3fff700 1 -- 172.21.15.44:0/1732409120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f39acc0 msgr2=0x55c48f3aa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80d3fff700 1 --2- 172.21.15.44:0/1732409120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f39acc0 0x55c48f3aa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80d3fff700 1 -- 172.21.15.44:0/1732409120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c8009d10 con 0x55c48f39a300 2023-09-10T00:47:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80d3fff700 1 --2- 172.21.15.44:0/1732409120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39a300 0x55c48f3a9e50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f80c40057a0 tx=0x7f80c401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315886af10833180 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:47:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80b37fe700 1 -- 172.21.15.44:0/1732409120 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c402b040 con 0x55c48f39a300 2023-09-10T00:47:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80b37fe700 1 -- 172.21.15.44:0/1732409120 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f80c4003e40 con 0x55c48f39a300 2023-09-10T00:47:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80e679fb80 1 -- 172.21.15.44:0/1732409120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c48f244490 con 0x55c48f39a300 2023-09-10T00:47:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80b37fe700 1 -- 172.21.15.44:0/1732409120 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80c400f660 con 0x55c48f39a300 2023-09-10T00:47:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80e679fb80 10 client.16971 fetch_fsmap learned FSMap version 50 2023-09-10T00:47:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80e679fb80 1 -- 172.21.15.44:0/1732409120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f80d419b230 con 0x55c48f39a300 2023-09-10T00:47:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80b37fe700 1 -- 172.21.15.44:0/1732409120 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c48f244490 con 0x55c48f39a300 2023-09-10T00:47:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80b37fe700 1 -- 172.21.15.44:0/1732409120 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 50) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f80c402f770 con 0x55c48f39a300 2023-09-10T00:47:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80e679fb80 10 client.16971 fetch_fsmap finished waiting for FSMap version 50 2023-09-10T00:47:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.746+0000 7f80e679fb80 10 client.16971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:47:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.747+0000 7f80e679fb80 20 client.16971 populate_metadata read hostname 'smithi044' 2023-09-10T00:47:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.747+0000 7f80e679fb80 1 --2- 172.21.15.44:0/1732409120 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c48f3b4410 0x55c48f3b6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.747+0000 7f80e679fb80 4 client.16971 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:47:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.747+0000 7f80e679fb80 1 -- 172.21.15.44:0/1732409120 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c48f301840 con 0x55c48f3b4410 2023-09-10T00:47:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.747+0000 7f80d37fe700 1 --2- 172.21.15.44:0/1732409120 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c48f3b4410 0x55c48f3b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.747+0000 7f80b37fe700 10 client.16971 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.747+0000 7f80d37fe700 1 --2- 172.21.15.44:0/1732409120 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c48f3b4410 0x55c48f3b6830 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71aa708292d90a0f server_cookie=7064728be3a9410 in_seq=0 out_seq=0 2023-09-10T00:47:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.748+0000 7f80b37fe700 1 -- 172.21.15.44:0/1732409120 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c48f301840 con 0x55c48f3b4410 2023-09-10T00:47:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.748+0000 7f80b37fe700 10 client.16971 handle_command_reply: tid=0 2023-09-10T00:47:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.748+0000 7f80e679fb80 1 client.16971 shutdown 2023-09-10T00:47:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.748+0000 7f80e679fb80 1 -- 172.21.15.44:0/1732409120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39a300 msgr2=0x55c48f3a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.748+0000 7f80e679fb80 1 --2- 172.21.15.44:0/1732409120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39a300 0x55c48f3a9e50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f80c40057a0 tx=0x7f80c401f000 comp rx=0 tx=0).stop 2023-09-10T00:47:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.748+0000 7f80e679fb80 1 -- 172.21.15.44:0/1732409120 shutdown_connections 2023-09-10T00:47:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.748+0000 7f80e679fb80 1 --2- 172.21.15.44:0/1732409120 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c48f3b4410 0x55c48f3b6830 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.748+0000 7f80e679fb80 1 --2- 172.21.15.44:0/1732409120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f3a01a0 0x55c48f3a40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.748+0000 7f80e679fb80 1 --2- 172.21.15.44:0/1732409120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39a300 0x55c48f3a9e50 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.748+0000 7f80e679fb80 1 --2- 172.21.15.44:0/1732409120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f39acc0 0x55c48f3aa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.748+0000 7f80e679fb80 1 -- 172.21.15.44:0/1732409120 >> 172.21.15.44:0/1732409120 conn(0x55c48f30f030 msgr2=0x55c48f3947a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.748+0000 7f80b37fe700 0 client.16971 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.748+0000 7f80e679fb80 1 -- 172.21.15.44:0/1732409120 shutdown_connections 2023-09-10T00:47:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.748+0000 7f80e679fb80 1 -- 172.21.15.44:0/1732409120 wait complete. 2023-09-10T00:47:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.748+0000 7f80e679fb80 20 client.16971 trim_cache size 0 max 16384 2023-09-10T00:47:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.763+0000 7f80e679fb80 1 Processor -- start 2023-09-10T00:47:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.763+0000 7f80e679fb80 1 -- start start 2023-09-10T00:47:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.763+0000 7f80e679fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f39a300 0x55c48f3a3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.763+0000 7f80e679fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39acc0 0x55c48f3a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.763+0000 7f80e679fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f3a01a0 0x55c48f3a9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.763+0000 7f80e679fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48f3a5540 con 0x55c48f39acc0 2023-09-10T00:47:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.763+0000 7f80e679fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48f3a2660 con 0x55c48f39a300 2023-09-10T00:47:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.763+0000 7f80e679fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48f39ed30 con 0x55c48f3a01a0 2023-09-10T00:47:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.763+0000 7f80d37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39acc0 0x55c48f3a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.763+0000 7f80d3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f39a300 0x55c48f3a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.763+0000 7f80d37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39acc0 0x55c48f3a3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51572/0 (socket says 172.21.15.44:51572) 2023-09-10T00:47:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.763+0000 7f80d37fe700 1 -- 172.21.15.44:0/3822528074 learned_addr learned my addr 172.21.15.44:0/3822528074 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.763+0000 7f80d8b76700 1 --2- 172.21.15.44:0/3822528074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f3a01a0 0x55c48f3a9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.764+0000 7f80d8b76700 1 -- 172.21.15.44:0/3822528074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f39a300 msgr2=0x55c48f3a3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.764+0000 7f80d8b76700 1 --2- 172.21.15.44:0/3822528074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f39a300 0x55c48f3a3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.764+0000 7f80d8b76700 1 -- 172.21.15.44:0/3822528074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39acc0 msgr2=0x55c48f3a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.764+0000 7f80d8b76700 1 --2- 172.21.15.44:0/3822528074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39acc0 0x55c48f3a3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.764+0000 7f80d8b76700 1 -- 172.21.15.44:0/3822528074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c48f101c00 con 0x55c48f3a01a0 2023-09-10T00:47:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.764+0000 7f80d37fe700 1 --2- 172.21.15.44:0/3822528074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39acc0 0x55c48f3a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.764+0000 7f80d3fff700 1 --2- 172.21.15.44:0/3822528074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f39a300 0x55c48f3a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.764+0000 7f80d8b76700 1 --2- 172.21.15.44:0/3822528074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f3a01a0 0x55c48f3a9a20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0c97b0 tx=0x7f80cc0094d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de7017f528ad021f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.764+0000 7f80baffd700 1 -- 172.21.15.44:0/3822528074 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc0996a0 con 0x55c48f3a01a0 2023-09-10T00:47:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.764+0000 7f80baffd700 1 -- 172.21.15.44:0/3822528074 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80cc09cdd0 con 0x55c48f3a01a0 2023-09-10T00:47:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.764+0000 7f80baffd700 1 -- 172.21.15.44:0/3822528074 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc098490 con 0x55c48f3a01a0 2023-09-10T00:47:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.764+0000 7f80e679fb80 1 -- 172.21.15.44:0/3822528074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f3a01a0 msgr2=0x55c48f3a9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.764+0000 7f80e679fb80 1 --2- 172.21.15.44:0/3822528074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f3a01a0 0x55c48f3a9a20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0c97b0 tx=0x7f80cc0094d0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80e679fb80 1 -- 172.21.15.44:0/3822528074 shutdown_connections 2023-09-10T00:47:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80e679fb80 1 --2- 172.21.15.44:0/3822528074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f3a01a0 0x55c48f3a9a20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80e679fb80 1 --2- 172.21.15.44:0/3822528074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f39a300 0x55c48f3a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80e679fb80 1 --2- 172.21.15.44:0/3822528074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f39acc0 0x55c48f3a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80e679fb80 1 -- 172.21.15.44:0/3822528074 >> 172.21.15.44:0/3822528074 conn(0x55c48f30f030 msgr2=0x55c48f395090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80e679fb80 1 -- 172.21.15.44:0/3822528074 shutdown_connections 2023-09-10T00:47:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80e679fb80 1 -- 172.21.15.44:0/3822528074 wait complete. 2023-09-10T00:47:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80e679fb80 1 -- start start 2023-09-10T00:47:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80e679fb80 1 Processor -- start 2023-09-10T00:47:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80e679fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f39a300 0x55c48f3fc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80e679fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f39acc0 0x55c48f3fc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80e679fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f3a01a0 0x55c48f407250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80e679fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48f407790 con 0x55c48f3a01a0 2023-09-10T00:47:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80e679fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48f4078d0 con 0x55c48f39a300 2023-09-10T00:47:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80e679fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48f407a90 con 0x55c48f39acc0 2023-09-10T00:47:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80e679fb80 10 client.0 _ll_register_callbacks cb 0x7f80c4005380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:47:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.765+0000 7f80d8b76700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f3a01a0 0x55c48f407250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80d37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f39acc0 0x55c48f3fc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80d8b76700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f3a01a0 0x55c48f407250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51580/0 (socket says 172.21.15.44:51580) 2023-09-10T00:47:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80d8b76700 1 -- 172.21.15.44:0/840018638 learned_addr learned my addr 172.21.15.44:0/840018638 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80d3fff700 1 --2- 172.21.15.44:0/840018638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f39a300 0x55c48f3fc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80d37fe700 1 -- 172.21.15.44:0/840018638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f39a300 msgr2=0x55c48f3fc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80d37fe700 1 --2- 172.21.15.44:0/840018638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f39a300 0x55c48f3fc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80d37fe700 1 -- 172.21.15.44:0/840018638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f3a01a0 msgr2=0x55c48f407250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80d37fe700 1 --2- 172.21.15.44:0/840018638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f3a01a0 0x55c48f407250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80d37fe700 1 -- 172.21.15.44:0/840018638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d419b230 con 0x55c48f39acc0 2023-09-10T00:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80d8b76700 1 --2- 172.21.15.44:0/840018638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f3a01a0 0x55c48f407250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80d37fe700 1 --2- 172.21.15.44:0/840018638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f39acc0 0x55c48f3fc900 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80c8004790 tx=0x7f80c8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3b954ec0a0ab9e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:47:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80b37fe700 1 -- 172.21.15.44:0/840018638 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8017040 con 0x55c48f39acc0 2023-09-10T00:47:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80e679fb80 1 -- 172.21.15.44:0/840018638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80cc098fa0 con 0x55c48f39acc0 2023-09-10T00:47:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80b37fe700 1 -- 172.21.15.44:0/840018638 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f80c8021c00 con 0x55c48f39acc0 2023-09-10T00:47:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80b37fe700 1 -- 172.21.15.44:0/840018638 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80c8021dc0 con 0x55c48f39acc0 2023-09-10T00:47:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80e679fb80 10 client.26581 fetch_fsmap learned FSMap version 50 2023-09-10T00:47:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.766+0000 7f80e679fb80 1 -- 172.21.15.44:0/840018638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f80c8012050 con 0x55c48f39acc0 2023-09-10T00:47:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.767+0000 7f80b37fe700 1 -- 172.21.15.44:0/840018638 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80cc098fa0 con 0x55c48f39acc0 2023-09-10T00:47:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.767+0000 7f80b37fe700 1 -- 172.21.15.44:0/840018638 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 50) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f80c8027030 con 0x55c48f39acc0 2023-09-10T00:47:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.767+0000 7f80e679fb80 10 client.26581 fetch_fsmap finished waiting for FSMap version 50 2023-09-10T00:47:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.767+0000 7f80e679fb80 10 client.26581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:47:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.767+0000 7f80e679fb80 20 client.26581 populate_metadata read hostname 'smithi044' 2023-09-10T00:47:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.767+0000 7f80e679fb80 1 --2- 172.21.15.44:0/840018638 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c48f409350 0x55c48f40b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.767+0000 7f80e679fb80 4 client.26581 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:47:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.767+0000 7f80e679fb80 1 -- 172.21.15.44:0/840018638 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c48f2e58c0 con 0x55c48f409350 2023-09-10T00:47:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.767+0000 7f80d3fff700 1 --2- 172.21.15.44:0/840018638 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c48f409350 0x55c48f40b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.767+0000 7f80b37fe700 10 client.26581 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.767+0000 7f80d3fff700 1 --2- 172.21.15.44:0/840018638 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c48f409350 0x55c48f40b770 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32951218e75abd69 server_cookie=6fbfa792602822b8 in_seq=0 out_seq=0 2023-09-10T00:47:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.858+0000 7f80b37fe700 1 -- 172.21.15.44:0/840018638 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c48f2e58c0 con 0x55c48f409350 2023-09-10T00:47:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.858+0000 7f80b37fe700 10 client.26581 handle_command_reply: tid=0 2023-09-10T00:47:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.858+0000 7f80e679fb80 1 client.26581 shutdown 2023-09-10T00:47:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.859+0000 7f80e679fb80 1 -- 172.21.15.44:0/840018638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f39acc0 msgr2=0x55c48f3fc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.859+0000 7f80e679fb80 1 --2- 172.21.15.44:0/840018638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f39acc0 0x55c48f3fc900 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80c8004790 tx=0x7f80c8004040 comp rx=0 tx=0).stop 2023-09-10T00:47:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.859+0000 7f80e679fb80 1 -- 172.21.15.44:0/840018638 shutdown_connections 2023-09-10T00:47:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.859+0000 7f80e679fb80 1 --2- 172.21.15.44:0/840018638 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c48f409350 0x55c48f40b770 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.859+0000 7f80e679fb80 1 --2- 172.21.15.44:0/840018638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c48f3a01a0 0x55c48f407250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.859+0000 7f80b37fe700 0 client.26581 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.859+0000 7f80e679fb80 1 --2- 172.21.15.44:0/840018638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c48f39a300 0x55c48f3fc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.859+0000 7f80e679fb80 1 --2- 172.21.15.44:0/840018638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c48f39acc0 0x55c48f3fc900 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.859+0000 7f80e679fb80 1 -- 172.21.15.44:0/840018638 >> 172.21.15.44:0/840018638 conn(0x55c48f30f030 msgr2=0x55c48f3992f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.859+0000 7f80e679fb80 1 -- 172.21.15.44:0/840018638 shutdown_connections 2023-09-10T00:47:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.859+0000 7f80e679fb80 1 -- 172.21.15.44:0/840018638 wait complete. 2023-09-10T00:47:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.859+0000 7f80e679fb80 20 client.26581 trim_cache size 0 max 16384 2023-09-10T00:47:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:47:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:47:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:47:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:47:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.859+0000 7f80da5d9700 1 -- 172.21.15.44:0/424727182 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80bc0a07b0 msgr2=0x7f80bc0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.859+0000 7f80da5d9700 1 --2- 172.21.15.44:0/424727182 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80bc0a07b0 0x7f80bc0a2c70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f80c4000c00 tx=0x7f80c4005fb0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.859+0000 7f80da5d9700 1 -- 172.21.15.44:0/424727182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80d4105590 msgr2=0x7f80d419a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.859+0000 7f80da5d9700 1 --2- 172.21.15.44:0/424727182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80d4105590 0x7f80d419a170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00efc0 tx=0x7f80cc00c3a0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.860+0000 7f80da5d9700 1 -- 172.21.15.44:0/424727182 shutdown_connections 2023-09-10T00:47:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.860+0000 7f80da5d9700 1 --2- 172.21.15.44:0/424727182 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80bc0a07b0 0x7f80bc0a2c70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.860+0000 7f80da5d9700 1 --2- 172.21.15.44:0/424727182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80d4105590 0x7f80d419a170 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.860+0000 7f80da5d9700 1 --2- 172.21.15.44:0/424727182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80d40ff4d0 0x7f80d4195300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.860+0000 7f80da5d9700 1 --2- 172.21.15.44:0/424727182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d40ffe90 0x7f80d4195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.860+0000 7f80da5d9700 1 -- 172.21.15.44:0/424727182 >> 172.21.15.44:0/424727182 conn(0x7f80d40faf40 msgr2=0x7f80d41031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.860+0000 7f80da5d9700 1 -- 172.21.15.44:0/424727182 shutdown_connections 2023-09-10T00:47:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:43.860+0000 7f80da5d9700 1 -- 172.21.15.44:0/424727182 wait complete. 2023-09-10T00:47:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:44 smithi131 ceph-mon[103713]: pgmap v217: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.8 MiB/s wr, 327 op/s 2023-09-10T00:47:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:44 smithi044 ceph-mon[98676]: pgmap v217: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.8 MiB/s wr, 327 op/s 2023-09-10T00:47:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:44 smithi099 ceph-mon[102692]: pgmap v217: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.8 MiB/s wr, 327 op/s 2023-09-10T00:47:45.823 INFO:teuthology.orchestra.run.smithi044.stderr:31400000 of 50000000 tuples (62%) done (elapsed 405.46 s, remaining 240.18 s) 2023-09-10T00:47:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:46 smithi131 ceph-mon[103713]: pgmap v218: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 4.1 MiB/s wr, 484 op/s 2023-09-10T00:47:47.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:46 smithi044 ceph-mon[98676]: pgmap v218: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 4.1 MiB/s wr, 484 op/s 2023-09-10T00:47:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:46 smithi099 ceph-mon[102692]: pgmap v218: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 4.1 MiB/s wr, 484 op/s 2023-09-10T00:47:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:48 smithi131 ceph-mon[103713]: pgmap v219: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 2.7 MiB/s wr, 315 op/s 2023-09-10T00:47:49.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:48 smithi044 ceph-mon[98676]: pgmap v219: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 2.7 MiB/s wr, 315 op/s 2023-09-10T00:47:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:48 smithi099 ceph-mon[102692]: pgmap v219: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 2.7 MiB/s wr, 315 op/s 2023-09-10T00:47:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:50 smithi131 ceph-mon[103713]: pgmap v220: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 2.7 MiB/s wr, 315 op/s 2023-09-10T00:47:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:50 smithi044 ceph-mon[98676]: pgmap v220: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 2.7 MiB/s wr, 315 op/s 2023-09-10T00:47:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:50 smithi099 ceph-mon[102692]: pgmap v220: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 2.7 MiB/s wr, 315 op/s 2023-09-10T00:47:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:51 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:47:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:51 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:47:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:51 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:47:52.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:51 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:47:52.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:51 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:47:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:51 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:47:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:51 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:47:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:51 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:47:52.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:51 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:47:52.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:51 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:47:52.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:51 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:47:52.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:51 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:47:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:52 smithi131 ceph-mon[103713]: pgmap v221: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 4.1 MiB/s wr, 478 op/s 2023-09-10T00:47:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:52 smithi044 ceph-mon[98676]: pgmap v221: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 4.1 MiB/s wr, 478 op/s 2023-09-10T00:47:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:52 smithi099 ceph-mon[102692]: pgmap v221: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 4.1 MiB/s wr, 478 op/s 2023-09-10T00:47:55.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:54 smithi131 ceph-mon[103713]: pgmap v222: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 2.8 MiB/s wr, 319 op/s 2023-09-10T00:47:55.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:54 smithi044 ceph-mon[98676]: pgmap v222: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 2.8 MiB/s wr, 319 op/s 2023-09-10T00:47:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:54 smithi099 ceph-mon[102692]: pgmap v222: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 2.8 MiB/s wr, 319 op/s 2023-09-10T00:47:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.001+0000 7f13de1b2700 1 -- 172.21.15.44:0/3593852678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d8104420 msgr2=0x7f13d8104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.001+0000 7f13de1b2700 1 --2- 172.21.15.44:0/3593852678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d8104420 0x7f13d8104800 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f13c80098f0 tx=0x7f13c8009c00 comp rx=0 tx=0).stop 2023-09-10T00:47:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.001+0000 7f13de1b2700 1 -- 172.21.15.44:0/3593852678 shutdown_connections 2023-09-10T00:47:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.001+0000 7f13de1b2700 1 --2- 172.21.15.44:0/3593852678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13d80ffe70 0x7f13d810c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.001+0000 7f13de1b2700 1 --2- 172.21.15.44:0/3593852678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d8104420 0x7f13d8104800 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.001+0000 7f13de1b2700 1 --2- 172.21.15.44:0/3593852678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13d80ff4b0 0x7f13d80ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.001+0000 7f13de1b2700 1 -- 172.21.15.44:0/3593852678 >> 172.21.15.44:0/3593852678 conn(0x7f13d8076600 msgr2=0x7f13d8076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.001+0000 7f13de1b2700 1 -- 172.21.15.44:0/3593852678 shutdown_connections 2023-09-10T00:47:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.001+0000 7f13de1b2700 1 -- 172.21.15.44:0/3593852678 wait complete. 2023-09-10T00:47:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.002+0000 7f13de1b2700 1 Processor -- start 2023-09-10T00:47:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.002+0000 7f13de1b2700 1 -- start start 2023-09-10T00:47:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.002+0000 7f13de1b2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d80ff4b0 0x7f13d819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13de1b2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13d80ffe70 0x7f13d819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13de1b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13d8104420 0x7f13d8196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13de1b2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8196b00 con 0x7f13d8104420 2023-09-10T00:47:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13de1b2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8196c70 con 0x7f13d80ff4b0 2023-09-10T00:47:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13de1b2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8196e30 con 0x7f13d80ffe70 2023-09-10T00:47:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13d7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13d8104420 0x7f13d8196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13d77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d80ff4b0 0x7f13d819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13d77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d80ff4b0 0x7f13d819c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53328/0 (socket says 172.21.15.44:53328) 2023-09-10T00:47:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13d77fe700 1 -- 172.21.15.44:0/2942083647 learned_addr learned my addr 172.21.15.44:0/2942083647 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13d7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13d8104420 0x7f13d8196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51590/0 (socket says 172.21.15.44:51590) 2023-09-10T00:47:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13d6ffd700 1 --2- 172.21.15.44:0/2942083647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13d80ffe70 0x7f13d819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13d77fe700 1 -- 172.21.15.44:0/2942083647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13d80ffe70 msgr2=0x7f13d819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13d77fe700 1 --2- 172.21.15.44:0/2942083647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13d80ffe70 0x7f13d819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13d77fe700 1 -- 172.21.15.44:0/2942083647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13d8104420 msgr2=0x7f13d8196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13d77fe700 1 --2- 172.21.15.44:0/2942083647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13d8104420 0x7f13d8196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13d77fe700 1 -- 172.21.15.44:0/2942083647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d0009c40 con 0x7f13d80ff4b0 2023-09-10T00:47:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13d7fff700 1 --2- 172.21.15.44:0/2942083647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13d8104420 0x7f13d8196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13d77fe700 1 --2- 172.21.15.44:0/2942083647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d80ff4b0 0x7f13d819c630 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009230 tx=0x7f13c800b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.003+0000 7f13d4ff9700 1 -- 172.21.15.44:0/2942083647 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c801c070 con 0x7f13d80ff4b0 2023-09-10T00:47:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.004+0000 7f13d4ff9700 1 -- 172.21.15.44:0/2942083647 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f13c8005040 con 0x7f13d80ff4b0 2023-09-10T00:47:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.004+0000 7f13d4ff9700 1 -- 172.21.15.44:0/2942083647 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c80038e0 con 0x7f13d80ff4b0 2023-09-10T00:47:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.004+0000 7f13de1b2700 1 -- 172.21.15.44:0/2942083647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c8009520 con 0x7f13d80ff4b0 2023-09-10T00:47:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.004+0000 7f13de1b2700 1 -- 172.21.15.44:0/2942083647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d81a3440 con 0x7f13d80ff4b0 2023-09-10T00:47:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.007+0000 7f13d4ff9700 1 -- 172.21.15.44:0/2942083647 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f13c8020030 con 0x7f13d80ff4b0 2023-09-10T00:47:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.008+0000 7f13d4ff9700 1 --2- 172.21.15.44:0/2942083647 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f13c00a0830 0x7f13c00a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.008+0000 7f13d4ff9700 1 -- 172.21.15.44:0/2942083647 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f13c80cd490 con 0x7f13d80ff4b0 2023-09-10T00:47:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.009+0000 7f13ea378b80 1 Processor -- start 2023-09-10T00:47:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.009+0000 7f13d6ffd700 1 --2- 172.21.15.44:0/2942083647 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f13c00a0830 0x7f13c00a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.009+0000 7f13ea378b80 1 -- start start 2023-09-10T00:47:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.009+0000 7f13ea378b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 0x55aa1d93b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.009+0000 7f13ea378b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935300 0x55aa1d935780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.009+0000 7f13ea378b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935cc0 0x55aa1d941f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.009+0000 7f13ea378b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa1d88e6d0 con 0x55aa1d935300 2023-09-10T00:47:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.009+0000 7f13ea378b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa1d88bff0 con 0x55aa1d93b1a0 2023-09-10T00:47:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.009+0000 7f13ea378b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa1d8878e0 con 0x55aa1d935cc0 2023-09-10T00:47:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.009+0000 7f13d77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 0x55aa1d93b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.009+0000 7f13d77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 0x55aa1d93b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53336/0 (socket says 172.21.15.44:53336) 2023-09-10T00:47:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.009+0000 7f13d77fe700 1 -- 172.21.15.44:0/2833369509 learned_addr learned my addr 172.21.15.44:0/2833369509 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.009+0000 7f13d7fff700 1 --2- 172.21.15.44:0/2833369509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935cc0 0x55aa1d941f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.010+0000 7f13d6ffd700 1 --2- 172.21.15.44:0/2942083647 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f13c00a0830 0x7f13c00a2cf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f13d8076b40 tx=0x7f13cc00b410 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.010+0000 7f13d6ffd700 1 --2- 172.21.15.44:0/2833369509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935300 0x55aa1d935780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.010+0000 7f13d7fff700 1 -- 172.21.15.44:0/2833369509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 msgr2=0x55aa1d93b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.010+0000 7f13d7fff700 1 --2- 172.21.15.44:0/2833369509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 0x55aa1d93b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.010+0000 7f13d7fff700 1 -- 172.21.15.44:0/2833369509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935300 msgr2=0x55aa1d935780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.010+0000 7f13d7fff700 1 --2- 172.21.15.44:0/2833369509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935300 0x55aa1d935780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.010+0000 7f13d7fff700 1 -- 172.21.15.44:0/2833369509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa1d6b3d80 con 0x55aa1d935cc0 2023-09-10T00:47:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.010+0000 7f13d77fe700 1 --2- 172.21.15.44:0/2833369509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 0x55aa1d93b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.010+0000 7f13d7fff700 1 --2- 172.21.15.44:0/2833369509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935cc0 0x55aa1d941f30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009fd0 tx=0x7f13d0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32ac2e4d4177c22e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.010+0000 7f13bdffb700 1 -- 172.21.15.44:0/2833369509 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0018070 con 0x55aa1d935cc0 2023-09-10T00:47:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.010+0000 7f13bdffb700 1 -- 172.21.15.44:0/2833369509 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f13d000f040 con 0x55aa1d935cc0 2023-09-10T00:47:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.010+0000 7f13bdffb700 1 -- 172.21.15.44:0/2833369509 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d000e4f0 con 0x55aa1d935cc0 2023-09-10T00:47:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.011+0000 7f13ea378b80 1 -- 172.21.15.44:0/2833369509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935cc0 msgr2=0x55aa1d941f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.011+0000 7f13ea378b80 1 --2- 172.21.15.44:0/2833369509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935cc0 0x55aa1d941f30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009fd0 tx=0x7f13d0009920 comp rx=0 tx=0).stop 2023-09-10T00:47:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.011+0000 7f13ea378b80 1 -- 172.21.15.44:0/2833369509 shutdown_connections 2023-09-10T00:47:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.011+0000 7f13ea378b80 1 --2- 172.21.15.44:0/2833369509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935cc0 0x55aa1d941f30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.011+0000 7f13ea378b80 1 --2- 172.21.15.44:0/2833369509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 0x55aa1d93b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.011+0000 7f13ea378b80 1 --2- 172.21.15.44:0/2833369509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935300 0x55aa1d935780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.011+0000 7f13ea378b80 1 -- 172.21.15.44:0/2833369509 >> 172.21.15.44:0/2833369509 conn(0x55aa1d8aa030 msgr2=0x55aa1d930560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.011+0000 7f13ea378b80 1 -- 172.21.15.44:0/2833369509 shutdown_connections 2023-09-10T00:47:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.011+0000 7f13ea378b80 1 -- 172.21.15.44:0/2833369509 wait complete. 2023-09-10T00:47:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.012+0000 7f13ea378b80 1 -- start start 2023-09-10T00:47:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.012+0000 7f13ea378b80 1 Processor -- start 2023-09-10T00:47:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.012+0000 7f13ea378b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935300 0x55aa1d941d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.012+0000 7f13ea378b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935cc0 0x55aa1d93f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.012+0000 7f13ea378b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 0x55aa1d93f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.012+0000 7f13ea378b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa1d825b10 con 0x55aa1d935cc0 2023-09-10T00:47:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.012+0000 7f13ea378b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa1d825810 con 0x55aa1d93b1a0 2023-09-10T00:47:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.012+0000 7f13ea378b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa1d7098c0 con 0x55aa1d935300 2023-09-10T00:47:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.012+0000 7f13ea378b80 10 client.0 _ll_register_callbacks cb 0x55aa1d76fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:47:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.012+0000 7f13d77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935300 0x55aa1d941d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.012+0000 7f13d6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935cc0 0x55aa1d93f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.012+0000 7f13d77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935300 0x55aa1d941d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58484/0 (socket says 172.21.15.44:58484) 2023-09-10T00:47:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.012+0000 7f13d77fe700 1 -- 172.21.15.44:0/3555895647 learned_addr learned my addr 172.21.15.44:0/3555895647 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.012+0000 7f13d7fff700 1 --2- 172.21.15.44:0/3555895647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 0x55aa1d93f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.013+0000 7f13d7fff700 1 -- 172.21.15.44:0/3555895647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935300 msgr2=0x55aa1d941d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.013+0000 7f13d7fff700 1 --2- 172.21.15.44:0/3555895647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935300 0x55aa1d941d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.013+0000 7f13d7fff700 1 -- 172.21.15.44:0/3555895647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935cc0 msgr2=0x55aa1d93f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.013+0000 7f13d7fff700 1 --2- 172.21.15.44:0/3555895647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935cc0 0x55aa1d93f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.013+0000 7f13d7fff700 1 -- 172.21.15.44:0/3555895647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d81a3440 con 0x55aa1d93b1a0 2023-09-10T00:47:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.013+0000 7f13d7fff700 1 --2- 172.21.15.44:0/3555895647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 0x55aa1d93f570 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f13d0000f80 tx=0x7f13d0004df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ace2671e543928d8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.013+0000 7f13b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:47:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.013+0000 7f13b6ffd700 1 -- 172.21.15.44:0/3555895647 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0018070 con 0x55aa1d93b1a0 2023-09-10T00:47:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.013+0000 7f13ea378b80 1 -- 172.21.15.44:0/3555895647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa1d7df490 con 0x55aa1d93b1a0 2023-09-10T00:47:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.013+0000 7f13b6ffd700 1 -- 172.21.15.44:0/3555895647 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f13d000e4f0 con 0x55aa1d93b1a0 2023-09-10T00:47:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.013+0000 7f13b6ffd700 1 -- 172.21.15.44:0/3555895647 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13d000e6b0 con 0x55aa1d93b1a0 2023-09-10T00:47:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.013+0000 7f13ea378b80 10 client.26651 fetch_fsmap learned FSMap version 50 2023-09-10T00:47:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.013+0000 7f13ea378b80 1 -- 172.21.15.44:0/3555895647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f13d00091a0 con 0x55aa1d93b1a0 2023-09-10T00:47:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.014+0000 7f13b6ffd700 1 -- 172.21.15.44:0/3555895647 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aa1d7df490 con 0x55aa1d93b1a0 2023-09-10T00:47:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.014+0000 7f13b6ffd700 1 -- 172.21.15.44:0/3555895647 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 50) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f13d00053b0 con 0x55aa1d93b1a0 2023-09-10T00:47:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.014+0000 7f13ea378b80 10 client.26651 fetch_fsmap finished waiting for FSMap version 50 2023-09-10T00:47:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.014+0000 7f13ea378b80 10 client.26651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:47:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.014+0000 7f13ea378b80 20 client.26651 populate_metadata read hostname 'smithi044' 2023-09-10T00:47:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.014+0000 7f13ea378b80 1 --2- 172.21.15.44:0/3555895647 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aa1d94f3c0 0x55aa1d9517e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.014+0000 7f13ea378b80 4 client.26651 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:47:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.014+0000 7f13ea378b80 1 -- 172.21.15.44:0/3555895647 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa1d89c840 con 0x55aa1d94f3c0 2023-09-10T00:47:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.014+0000 7f13d77fe700 1 --2- 172.21.15.44:0/3555895647 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aa1d94f3c0 0x55aa1d9517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.014+0000 7f13b6ffd700 10 client.26651 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.015+0000 7f13d77fe700 1 --2- 172.21.15.44:0/3555895647 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aa1d94f3c0 0x55aa1d9517e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ff1d20e7a658cb4 server_cookie=7e0041e20dcbbc77 in_seq=0 out_seq=0 2023-09-10T00:47:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.015+0000 7f13b6ffd700 1 -- 172.21.15.44:0/3555895647 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aa1d89c840 con 0x55aa1d94f3c0 2023-09-10T00:47:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.015+0000 7f13b6ffd700 10 client.26651 handle_command_reply: tid=0 2023-09-10T00:47:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.015+0000 7f13ea378b80 1 client.26651 shutdown 2023-09-10T00:47:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.015+0000 7f13ea378b80 1 -- 172.21.15.44:0/3555895647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 msgr2=0x55aa1d93f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.015+0000 7f13ea378b80 1 --2- 172.21.15.44:0/3555895647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 0x55aa1d93f570 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f13d0000f80 tx=0x7f13d0004df0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.015+0000 7f13ea378b80 1 -- 172.21.15.44:0/3555895647 shutdown_connections 2023-09-10T00:47:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.015+0000 7f13ea378b80 1 --2- 172.21.15.44:0/3555895647 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aa1d94f3c0 0x55aa1d9517e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.015+0000 7f13ea378b80 1 --2- 172.21.15.44:0/3555895647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 0x55aa1d93f570 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.015+0000 7f13b6ffd700 0 client.26651 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.015+0000 7f13ea378b80 1 --2- 172.21.15.44:0/3555895647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935300 0x55aa1d941d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.015+0000 7f13ea378b80 1 --2- 172.21.15.44:0/3555895647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935cc0 0x55aa1d93f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.015+0000 7f13ea378b80 1 -- 172.21.15.44:0/3555895647 >> 172.21.15.44:0/3555895647 conn(0x55aa1d8aa030 msgr2=0x55aa1d92f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.016+0000 7f13ea378b80 1 -- 172.21.15.44:0/3555895647 shutdown_connections 2023-09-10T00:47:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.016+0000 7f13ea378b80 1 -- 172.21.15.44:0/3555895647 wait complete. 2023-09-10T00:47:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.016+0000 7f13ea378b80 20 client.26651 trim_cache size 0 max 16384 2023-09-10T00:47:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.034+0000 7f13ea378b80 1 Processor -- start 2023-09-10T00:47:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.034+0000 7f13ea378b80 1 -- start start 2023-09-10T00:47:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.034+0000 7f13ea378b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935300 0x55aa1d930710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.034+0000 7f13ea378b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935cc0 0x55aa1d93a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.034+0000 7f13ea378b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 0x55aa1d946530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.034+0000 7f13ea378b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa1d89c840 con 0x55aa1d935cc0 2023-09-10T00:47:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.034+0000 7f13ea378b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa1d7df490 con 0x55aa1d93b1a0 2023-09-10T00:47:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.034+0000 7f13ea378b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa1d927f50 con 0x55aa1d935300 2023-09-10T00:47:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.034+0000 7f13d7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 0x55aa1d946530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.035+0000 7f13d7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 0x55aa1d946530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53350/0 (socket says 172.21.15.44:53350) 2023-09-10T00:47:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.035+0000 7f13d7fff700 1 -- 172.21.15.44:0/3766930883 learned_addr learned my addr 172.21.15.44:0/3766930883 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.035+0000 7f13d77fe700 1 --2- 172.21.15.44:0/3766930883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935300 0x55aa1d930710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.035+0000 7f13d6ffd700 1 --2- 172.21.15.44:0/3766930883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935cc0 0x55aa1d93a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.035+0000 7f13d77fe700 1 -- 172.21.15.44:0/3766930883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 msgr2=0x55aa1d946530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.035+0000 7f13d77fe700 1 --2- 172.21.15.44:0/3766930883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 0x55aa1d946530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.035+0000 7f13d77fe700 1 -- 172.21.15.44:0/3766930883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935cc0 msgr2=0x55aa1d93a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.035+0000 7f13d77fe700 1 --2- 172.21.15.44:0/3766930883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935cc0 0x55aa1d93a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.035+0000 7f13d77fe700 1 -- 172.21.15.44:0/3766930883 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa1d69cc00 con 0x55aa1d935300 2023-09-10T00:47:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.035+0000 7f13d6ffd700 1 --2- 172.21.15.44:0/3766930883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935cc0 0x55aa1d93a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:47:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.035+0000 7f13d77fe700 1 --2- 172.21.15.44:0/3766930883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935300 0x55aa1d930710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f13c80059d0 tx=0x7f13c8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1333325ebfbfdf0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.035+0000 7f13be7fc700 1 -- 172.21.15.44:0/3766930883 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c809ac70 con 0x55aa1d935300 2023-09-10T00:47:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.036+0000 7f13be7fc700 1 -- 172.21.15.44:0/3766930883 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f13c809a570 con 0x55aa1d935300 2023-09-10T00:47:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.036+0000 7f13be7fc700 1 -- 172.21.15.44:0/3766930883 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8098910 con 0x55aa1d935300 2023-09-10T00:47:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.036+0000 7f13ea378b80 1 -- 172.21.15.44:0/3766930883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935300 msgr2=0x55aa1d930710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.036+0000 7f13ea378b80 1 --2- 172.21.15.44:0/3766930883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935300 0x55aa1d930710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f13c80059d0 tx=0x7f13c8009c00 comp rx=0 tx=0).stop 2023-09-10T00:47:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.036+0000 7f13ea378b80 1 -- 172.21.15.44:0/3766930883 shutdown_connections 2023-09-10T00:47:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.036+0000 7f13ea378b80 1 --2- 172.21.15.44:0/3766930883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d93b1a0 0x55aa1d946530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.036+0000 7f13ea378b80 1 --2- 172.21.15.44:0/3766930883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d935300 0x55aa1d930710 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.036+0000 7f13ea378b80 1 --2- 172.21.15.44:0/3766930883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935cc0 0x55aa1d93a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.036+0000 7f13ea378b80 1 -- 172.21.15.44:0/3766930883 >> 172.21.15.44:0/3766930883 conn(0x55aa1d8aa030 msgr2=0x55aa1d9342f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:57.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.036+0000 7f13ea378b80 1 -- 172.21.15.44:0/3766930883 shutdown_connections 2023-09-10T00:47:57.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.036+0000 7f13ea378b80 1 -- 172.21.15.44:0/3766930883 wait complete. 2023-09-10T00:47:57.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.037+0000 7f13ea378b80 1 -- start start 2023-09-10T00:47:57.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.037+0000 7f13ea378b80 1 Processor -- start 2023-09-10T00:47:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.037+0000 7f13ea378b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d935300 0x55aa1d99ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.037+0000 7f13ea378b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935cc0 0x55aa1d99cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.037+0000 7f13ea378b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d93b1a0 0x55aa1d997140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.037+0000 7f13ea378b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa1d997720 con 0x55aa1d935cc0 2023-09-10T00:47:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.037+0000 7f13ea378b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa1d997890 con 0x55aa1d935300 2023-09-10T00:47:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.037+0000 7f13ea378b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa1d997a50 con 0x55aa1d93b1a0 2023-09-10T00:47:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.037+0000 7f13ea378b80 10 client.0 _ll_register_callbacks cb 0x7f13cc004300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:47:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.038+0000 7f13d7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d93b1a0 0x55aa1d997140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.038+0000 7f13d7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d93b1a0 0x55aa1d997140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58498/0 (socket says 172.21.15.44:58498) 2023-09-10T00:47:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.038+0000 7f13d7fff700 1 -- 172.21.15.44:0/193699282 learned_addr learned my addr 172.21.15.44:0/193699282 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:47:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.038+0000 7f13d6ffd700 1 --2- 172.21.15.44:0/193699282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935cc0 0x55aa1d99cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.038+0000 7f13d77fe700 1 --2- 172.21.15.44:0/193699282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d935300 0x55aa1d99ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.038+0000 7f13d7fff700 1 -- 172.21.15.44:0/193699282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d935300 msgr2=0x55aa1d99ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.038+0000 7f13d7fff700 1 --2- 172.21.15.44:0/193699282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d935300 0x55aa1d99ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.038+0000 7f13d7fff700 1 -- 172.21.15.44:0/193699282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935cc0 msgr2=0x55aa1d99cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:47:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.038+0000 7f13d7fff700 1 --2- 172.21.15.44:0/193699282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935cc0 0x55aa1d99cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.038+0000 7f13d7fff700 1 -- 172.21.15.44:0/193699282 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13cc003910 con 0x55aa1d93b1a0 2023-09-10T00:47:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.038+0000 7f13d77fe700 1 --2- 172.21.15.44:0/193699282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d935300 0x55aa1d99ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:47:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.038+0000 7f13d7fff700 1 --2- 172.21.15.44:0/193699282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d93b1a0 0x55aa1d997140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009fd0 tx=0x7f13d00060b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2606678844d8c6c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:47:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.038+0000 7f13b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:47:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.038+0000 7f13b6ffd700 1 -- 172.21.15.44:0/193699282 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0018040 con 0x55aa1d93b1a0 2023-09-10T00:47:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.039+0000 7f13b6ffd700 1 -- 172.21.15.44:0/193699282 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f13d0003c90 con 0x55aa1d93b1a0 2023-09-10T00:47:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.039+0000 7f13ea378b80 1 -- 172.21.15.44:0/193699282 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13d000f040 con 0x55aa1d93b1a0 2023-09-10T00:47:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.039+0000 7f13b6ffd700 1 -- 172.21.15.44:0/193699282 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13d0003e70 con 0x55aa1d93b1a0 2023-09-10T00:47:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.039+0000 7f13ea378b80 10 client.26599 fetch_fsmap learned FSMap version 50 2023-09-10T00:47:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.039+0000 7f13ea378b80 1 -- 172.21.15.44:0/193699282 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f13c8009520 con 0x55aa1d93b1a0 2023-09-10T00:47:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.039+0000 7f13b6ffd700 1 -- 172.21.15.44:0/193699282 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f13d000f040 con 0x55aa1d93b1a0 2023-09-10T00:47:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.039+0000 7f13b6ffd700 1 -- 172.21.15.44:0/193699282 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 50) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f13d0026940 con 0x55aa1d93b1a0 2023-09-10T00:47:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.039+0000 7f13ea378b80 10 client.26599 fetch_fsmap finished waiting for FSMap version 50 2023-09-10T00:47:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.039+0000 7f13ea378b80 10 client.26599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:47:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.039+0000 7f13ea378b80 20 client.26599 populate_metadata read hostname 'smithi044' 2023-09-10T00:47:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.039+0000 7f13ea378b80 1 --2- 172.21.15.44:0/193699282 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aa1d9a4530 0x55aa1d9a6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:47:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.039+0000 7f13ea378b80 4 client.26599 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:47:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.039+0000 7f13ea378b80 1 -- 172.21.15.44:0/193699282 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aa1d8808c0 con 0x55aa1d9a4530 2023-09-10T00:47:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.039+0000 7f13d77fe700 1 --2- 172.21.15.44:0/193699282 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aa1d9a4530 0x55aa1d9a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:47:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.040+0000 7f13d77fe700 1 --2- 172.21.15.44:0/193699282 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aa1d9a4530 0x55aa1d9a6950 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3b1436fa6c358d5 server_cookie=9465b708f212e75d in_seq=0 out_seq=0 2023-09-10T00:47:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.040+0000 7f13b6ffd700 10 client.26599 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:57.057 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:56 smithi131 ceph-mon[103713]: pgmap v223: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 4.1 MiB/s wr, 472 op/s 2023-09-10T00:47:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.049+0000 7f13b6ffd700 1 -- 172.21.15.44:0/193699282 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aa1d8808c0 con 0x55aa1d9a4530 2023-09-10T00:47:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.049+0000 7f13b6ffd700 10 client.26599 handle_command_reply: tid=0 2023-09-10T00:47:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.049+0000 7f13ea378b80 1 client.26599 shutdown 2023-09-10T00:47:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.049+0000 7f13ea378b80 1 -- 172.21.15.44:0/193699282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d93b1a0 msgr2=0x55aa1d997140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.049+0000 7f13ea378b80 1 --2- 172.21.15.44:0/193699282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d93b1a0 0x55aa1d997140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009fd0 tx=0x7f13d00060b0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.049+0000 7f13ea378b80 1 -- 172.21.15.44:0/193699282 shutdown_connections 2023-09-10T00:47:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.049+0000 7f13ea378b80 1 --2- 172.21.15.44:0/193699282 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aa1d9a4530 0x55aa1d9a6950 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.049+0000 7f13ea378b80 1 --2- 172.21.15.44:0/193699282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa1d93b1a0 0x55aa1d997140 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.049+0000 7f13ea378b80 1 --2- 172.21.15.44:0/193699282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa1d935300 0x55aa1d99ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.049+0000 7f13ea378b80 1 --2- 172.21.15.44:0/193699282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa1d935cc0 0x55aa1d99cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.049+0000 7f13ea378b80 1 -- 172.21.15.44:0/193699282 >> 172.21.15.44:0/193699282 conn(0x55aa1d8aa030 msgr2=0x55aa1d94b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.049+0000 7f13b6ffd700 0 client.26599 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:47:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.049+0000 7f13ea378b80 1 -- 172.21.15.44:0/193699282 shutdown_connections 2023-09-10T00:47:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.049+0000 7f13ea378b80 1 -- 172.21.15.44:0/193699282 wait complete. 2023-09-10T00:47:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.050+0000 7f13ea378b80 20 client.26599 trim_cache size 0 max 16384 2023-09-10T00:47:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:47:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:47:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:47:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:47:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.050+0000 7f13de1b2700 1 -- 172.21.15.44:0/2942083647 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f13c00a0830 msgr2=0x7f13c00a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.050+0000 7f13de1b2700 1 --2- 172.21.15.44:0/2942083647 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f13c00a0830 0x7f13c00a2cf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f13d8076b40 tx=0x7f13cc00b410 comp rx=0 tx=0).stop 2023-09-10T00:47:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.050+0000 7f13de1b2700 1 -- 172.21.15.44:0/2942083647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d80ff4b0 msgr2=0x7f13d819c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:47:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.050+0000 7f13de1b2700 1 --2- 172.21.15.44:0/2942083647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d80ff4b0 0x7f13d819c630 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009230 tx=0x7f13c800b560 comp rx=0 tx=0).stop 2023-09-10T00:47:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.050+0000 7f13de1b2700 1 -- 172.21.15.44:0/2942083647 shutdown_connections 2023-09-10T00:47:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.050+0000 7f13de1b2700 1 --2- 172.21.15.44:0/2942083647 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f13c00a0830 0x7f13c00a2cf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.050+0000 7f13de1b2700 1 --2- 172.21.15.44:0/2942083647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13d8104420 0x7f13d8196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.050+0000 7f13de1b2700 1 --2- 172.21.15.44:0/2942083647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d80ff4b0 0x7f13d819c630 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.050+0000 7f13de1b2700 1 --2- 172.21.15.44:0/2942083647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13d80ffe70 0x7f13d819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:47:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.050+0000 7f13de1b2700 1 -- 172.21.15.44:0/2942083647 >> 172.21.15.44:0/2942083647 conn(0x7f13d8076600 msgr2=0x7f13d80fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:47:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.050+0000 7f13de1b2700 1 -- 172.21.15.44:0/2942083647 shutdown_connections 2023-09-10T00:47:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:47:57.050+0000 7f13de1b2700 1 -- 172.21.15.44:0/2942083647 wait complete. 2023-09-10T00:47:57.065 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:56 smithi044 ceph-mon[98676]: pgmap v223: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 4.1 MiB/s wr, 472 op/s 2023-09-10T00:47:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:56 smithi099 ceph-mon[102692]: pgmap v223: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 4.1 MiB/s wr, 472 op/s 2023-09-10T00:47:59.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:47:58 smithi131 ceph-mon[103713]: pgmap v224: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.7 MiB/s wr, 315 op/s 2023-09-10T00:47:59.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:47:58 smithi044 ceph-mon[98676]: pgmap v224: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.7 MiB/s wr, 315 op/s 2023-09-10T00:47:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:47:58 smithi099 ceph-mon[102692]: pgmap v224: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.7 MiB/s wr, 315 op/s 2023-09-10T00:47:59.350 INFO:teuthology.orchestra.run.smithi044.stderr:31500000 of 50000000 tuples (63%) done (elapsed 418.98 s, remaining 246.07 s) 2023-09-10T00:48:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:00 smithi131 ceph-mon[103713]: pgmap v225: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.7 MiB/s wr, 315 op/s 2023-09-10T00:48:01.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:00 smithi044 ceph-mon[98676]: pgmap v225: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.7 MiB/s wr, 315 op/s 2023-09-10T00:48:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:00 smithi099 ceph-mon[102692]: pgmap v225: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.7 MiB/s wr, 315 op/s 2023-09-10T00:48:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:02 smithi131 ceph-mon[103713]: pgmap v226: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 4.1 MiB/s wr, 476 op/s 2023-09-10T00:48:03.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:02 smithi044 ceph-mon[98676]: pgmap v226: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 4.1 MiB/s wr, 476 op/s 2023-09-10T00:48:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:02 smithi099 ceph-mon[102692]: pgmap v226: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 4.1 MiB/s wr, 476 op/s 2023-09-10T00:48:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:04 smithi131 ceph-mon[103713]: pgmap v227: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 2.7 MiB/s wr, 313 op/s 2023-09-10T00:48:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:04 smithi131 ceph-mon[103713]: osdmap e95: 12 total, 12 up, 12 in 2023-09-10T00:48:05.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:04 smithi044 ceph-mon[98676]: pgmap v227: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 2.7 MiB/s wr, 313 op/s 2023-09-10T00:48:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:04 smithi044 ceph-mon[98676]: osdmap e95: 12 total, 12 up, 12 in 2023-09-10T00:48:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:04 smithi099 ceph-mon[102692]: pgmap v227: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 2.7 MiB/s wr, 313 op/s 2023-09-10T00:48:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:04 smithi099 ceph-mon[102692]: osdmap e95: 12 total, 12 up, 12 in 2023-09-10T00:48:07.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:06 smithi131 ceph-mon[103713]: pgmap v229: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 3.4 MiB/s wr, 397 op/s 2023-09-10T00:48:07.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:06 smithi044 ceph-mon[98676]: pgmap v229: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 3.4 MiB/s wr, 397 op/s 2023-09-10T00:48:07.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:06 smithi099 ceph-mon[102692]: pgmap v229: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 3.4 MiB/s wr, 397 op/s 2023-09-10T00:48:09.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:08 smithi131 ceph-mon[103713]: pgmap v230: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 3.4 MiB/s wr, 397 op/s 2023-09-10T00:48:09.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:08 smithi044 ceph-mon[98676]: pgmap v230: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 3.4 MiB/s wr, 397 op/s 2023-09-10T00:48:09.101 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:48:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:08 smithi099 ceph-mon[102692]: pgmap v230: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 3.4 MiB/s wr, 397 op/s 2023-09-10T00:48:09.262 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.262+0000 7fc03813b700 1 -- 172.21.15.44:0/679389103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0301054e0 msgr2=0x7fc0301058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:09.263 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.262+0000 7fc03813b700 1 --2- 172.21.15.44:0/679389103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0301054e0 0x7fc0301058c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009990 tx=0x7fc02c009ca0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.264 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.262+0000 7fc03813b700 1 -- 172.21.15.44:0/679389103 shutdown_connections 2023-09-10T00:48:09.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.262+0000 7fc03813b700 1 --2- 172.21.15.44:0/679389103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0300ffe00 0x7fc03010c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.262+0000 7fc03813b700 1 --2- 172.21.15.44:0/679389103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0301054e0 0x7fc0301058c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.262+0000 7fc03813b700 1 --2- 172.21.15.44:0/679389103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0300ff440 0x7fc0300ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.262+0000 7fc03813b700 1 -- 172.21.15.44:0/679389103 >> 172.21.15.44:0/679389103 conn(0x7fc0300faf50 msgr2=0x7fc0300fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:09.265 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.262+0000 7fc03813b700 1 -- 172.21.15.44:0/679389103 shutdown_connections 2023-09-10T00:48:09.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.262+0000 7fc03813b700 1 -- 172.21.15.44:0/679389103 wait complete. 2023-09-10T00:48:09.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.263+0000 7fc03813b700 1 Processor -- start 2023-09-10T00:48:09.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.263+0000 7fc03813b700 1 -- start start 2023-09-10T00:48:09.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.263+0000 7fc03813b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0300ff440 0x7fc030196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.263+0000 7fc03813b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0300ffe00 0x7fc030196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.266 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.263+0000 7fc03813b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0301054e0 0x7fc03019afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.263+0000 7fc03813b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03019b5e0 con 0x7fc0301054e0 2023-09-10T00:48:09.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.263+0000 7fc03813b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03019b750 con 0x7fc0300ffe00 2023-09-10T00:48:09.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.263+0000 7fc03813b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03019b910 con 0x7fc0300ff440 2023-09-10T00:48:09.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.263+0000 7fc0366d8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0301054e0 0x7fc03019afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:09.267 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.263+0000 7fc035ed7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0300ff440 0x7fc030196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:09.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.263+0000 7fc0366d8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0301054e0 0x7fc03019afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51632/0 (socket says 172.21.15.44:51632) 2023-09-10T00:48:09.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.263+0000 7fc0356d6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0300ffe00 0x7fc030196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:09.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.263+0000 7fc0366d8700 1 -- 172.21.15.44:0/1147782084 learned_addr learned my addr 172.21.15.44:0/1147782084 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:09.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.264+0000 7fc0356d6700 1 -- 172.21.15.44:0/1147782084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0300ff440 msgr2=0x7fc030196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:09.268 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.264+0000 7fc0356d6700 1 --2- 172.21.15.44:0/1147782084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0300ff440 0x7fc030196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.264+0000 7fc0356d6700 1 -- 172.21.15.44:0/1147782084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0301054e0 msgr2=0x7fc03019afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:09.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.264+0000 7fc0356d6700 1 --2- 172.21.15.44:0/1147782084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0301054e0 0x7fc03019afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.264+0000 7fc0356d6700 1 -- 172.21.15.44:0/1147782084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc028009c40 con 0x7fc0300ffe00 2023-09-10T00:48:09.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.264+0000 7fc0366d8700 1 --2- 172.21.15.44:0/1147782084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0301054e0 0x7fc03019afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:09.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.264+0000 7fc0356d6700 1 --2- 172.21.15.44:0/1147782084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0300ffe00 0x7fc030196b40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc02000e9f0 tx=0x7fc02000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:09.269 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.264+0000 7fc026ffd700 1 -- 172.21.15.44:0/1147782084 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02000c9c0 con 0x7fc0300ffe00 2023-09-10T00:48:09.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.264+0000 7fc03813b700 1 -- 172.21.15.44:0/1147782084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02c0095f0 con 0x7fc0300ffe00 2023-09-10T00:48:09.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.264+0000 7fc026ffd700 1 -- 172.21.15.44:0/1147782084 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc020010430 con 0x7fc0300ffe00 2023-09-10T00:48:09.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.264+0000 7fc026ffd700 1 -- 172.21.15.44:0/1147782084 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc020018680 con 0x7fc0300ffe00 2023-09-10T00:48:09.270 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.264+0000 7fc03813b700 1 -- 172.21.15.44:0/1147782084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03019bde0 con 0x7fc0300ffe00 2023-09-10T00:48:09.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.267+0000 7fc026ffd700 1 -- 172.21.15.44:0/1147782084 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc0200105a0 con 0x7fc0300ffe00 2023-09-10T00:48:09.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.267+0000 7fc026ffd700 1 --2- 172.21.15.44:0/1147782084 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc01c0a07b0 0x7fc01c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.267+0000 7fc026ffd700 1 -- 172.21.15.44:0/1147782084 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fc020014070 con 0x7fc0300ffe00 2023-09-10T00:48:09.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.268+0000 7fc044301b80 1 Processor -- start 2023-09-10T00:48:09.273 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.268+0000 7fc035ed7700 1 --2- 172.21.15.44:0/1147782084 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc01c0a07b0 0x7fc01c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:09.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.269+0000 7fc044301b80 1 -- start start 2023-09-10T00:48:09.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.269+0000 7fc044301b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56231507f710 0x56231507faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.269+0000 7fc044301b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562315080030 0x56231508dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.269+0000 7fc044301b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231508e250 0x562315090640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.274 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.269+0000 7fc044301b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562314fd8730 con 0x56231507f710 2023-09-10T00:48:09.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.269+0000 7fc044301b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562314fd6050 con 0x56231508e250 2023-09-10T00:48:09.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.269+0000 7fc044301b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562314fd1940 con 0x562315080030 2023-09-10T00:48:09.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.269+0000 7fc035ed7700 1 --2- 172.21.15.44:0/1147782084 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc01c0a07b0 0x7fc01c0a2c70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc02c006010 tx=0x7fc02c00b260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:09.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.269+0000 7fc035ed7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56231507f710 0x56231507faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:09.275 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.269+0000 7fc035ed7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56231507f710 0x56231507faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51638/0 (socket says 172.21.15.44:51638) 2023-09-10T00:48:09.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.269+0000 7fc035ed7700 1 -- 172.21.15.44:0/4234388546 learned_addr learned my addr 172.21.15.44:0/4234388546 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:09.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.269+0000 7fc035ed7700 1 -- 172.21.15.44:0/4234388546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562315080030 msgr2=0x56231508dd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T00:48:09.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.269+0000 7fc035ed7700 1 --2- 172.21.15.44:0/4234388546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562315080030 0x56231508dd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.269+0000 7fc035ed7700 1 -- 172.21.15.44:0/4234388546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231508e250 msgr2=0x562315090640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T00:48:09.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.269+0000 7fc035ed7700 1 --2- 172.21.15.44:0/4234388546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231508e250 0x562315090640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.276 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.269+0000 7fc035ed7700 1 -- 172.21.15.44:0/4234388546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562314dfdde0 con 0x56231507f710 2023-09-10T00:48:09.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.270+0000 7fc035ed7700 1 --2- 172.21.15.44:0/4234388546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56231507f710 0x56231507faf0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc02c005760 tx=0x7fc02c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b635314163978986 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:09.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.270+0000 7fc01bfff700 1 -- 172.21.15.44:0/4234388546 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c004700 con 0x56231507f710 2023-09-10T00:48:09.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.270+0000 7fc01bfff700 1 -- 172.21.15.44:0/4234388546 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc02c01e040 con 0x56231507f710 2023-09-10T00:48:09.277 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.270+0000 7fc01bfff700 1 -- 172.21.15.44:0/4234388546 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c00fa30 con 0x56231507f710 2023-09-10T00:48:09.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.271+0000 7fc044301b80 1 -- 172.21.15.44:0/4234388546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56231507f710 msgr2=0x56231507faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:09.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.271+0000 7fc044301b80 1 --2- 172.21.15.44:0/4234388546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56231507f710 0x56231507faf0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc02c005760 tx=0x7fc02c020000 comp rx=0 tx=0).stop 2023-09-10T00:48:09.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.271+0000 7fc044301b80 1 -- 172.21.15.44:0/4234388546 shutdown_connections 2023-09-10T00:48:09.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.271+0000 7fc044301b80 1 --2- 172.21.15.44:0/4234388546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231508e250 0x562315090640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.271+0000 7fc044301b80 1 --2- 172.21.15.44:0/4234388546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56231507f710 0x56231507faf0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.271+0000 7fc044301b80 1 --2- 172.21.15.44:0/4234388546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562315080030 0x56231508dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.271+0000 7fc044301b80 1 -- 172.21.15.44:0/4234388546 >> 172.21.15.44:0/4234388546 conn(0x562314ff4090 msgr2=0x56231507abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:09.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.272+0000 7fc044301b80 1 -- 172.21.15.44:0/4234388546 shutdown_connections 2023-09-10T00:48:09.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.272+0000 7fc044301b80 1 -- 172.21.15.44:0/4234388546 wait complete. 2023-09-10T00:48:09.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.272+0000 7fc044301b80 1 -- start start 2023-09-10T00:48:09.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.272+0000 7fc044301b80 1 Processor -- start 2023-09-10T00:48:09.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc044301b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231507f710 0x56231508dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc044301b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562315080030 0x56231508af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc044301b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56231508e250 0x56231508b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc044301b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562314f6fb70 con 0x562315080030 2023-09-10T00:48:09.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc044301b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562314f6f870 con 0x56231507f710 2023-09-10T00:48:09.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc044301b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562314e53920 con 0x56231508e250 2023-09-10T00:48:09.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc044301b80 10 client.0 _ll_register_callbacks cb 0x562314eb9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:48:09.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc0356d6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562315080030 0x56231508af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:09.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc035ed7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231507f710 0x56231508dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:09.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc035ed7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231507f710 0x56231508dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53380/0 (socket says 172.21.15.44:53380) 2023-09-10T00:48:09.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc0356d6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562315080030 0x56231508af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51642/0 (socket says 172.21.15.44:51642) 2023-09-10T00:48:09.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc035ed7700 1 -- 172.21.15.44:0/2765163798 learned_addr learned my addr 172.21.15.44:0/2765163798 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:09.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc0366d8700 1 --2- 172.21.15.44:0/2765163798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56231508e250 0x56231508b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:09.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc035ed7700 1 -- 172.21.15.44:0/2765163798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56231508e250 msgr2=0x56231508b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:09.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc035ed7700 1 --2- 172.21.15.44:0/2765163798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56231508e250 0x56231508b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc035ed7700 1 -- 172.21.15.44:0/2765163798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562315080030 msgr2=0x56231508af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:09.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc035ed7700 1 --2- 172.21.15.44:0/2765163798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562315080030 0x56231508af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc035ed7700 1 -- 172.21.15.44:0/2765163798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc030067ae0 con 0x56231507f710 2023-09-10T00:48:09.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc035ed7700 1 --2- 172.21.15.44:0/2765163798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231507f710 0x56231508dcc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009990 tx=0x7fc02c00f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f5f9aef5dedb657 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:09.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.273+0000 7fc0197fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:48:09.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.274+0000 7fc0197fa700 1 -- 172.21.15.44:0/2765163798 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c004500 con 0x56231507f710 2023-09-10T00:48:09.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.274+0000 7fc044301b80 1 -- 172.21.15.44:0/2765163798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562314f294f0 con 0x56231507f710 2023-09-10T00:48:09.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.274+0000 7fc0197fa700 1 -- 172.21.15.44:0/2765163798 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fc02c004660 con 0x56231507f710 2023-09-10T00:48:09.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.274+0000 7fc0197fa700 1 -- 172.21.15.44:0/2765163798 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc02c004820 con 0x56231507f710 2023-09-10T00:48:09.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.274+0000 7fc044301b80 10 client.26666 fetch_fsmap learned FSMap version 50 2023-09-10T00:48:09.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.274+0000 7fc044301b80 1 -- 172.21.15.44:0/2765163798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x562314de6c60 con 0x56231507f710 2023-09-10T00:48:09.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.274+0000 7fc0197fa700 1 -- 172.21.15.44:0/2765163798 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562314f294f0 con 0x56231507f710 2023-09-10T00:48:09.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.274+0000 7fc0197fa700 1 -- 172.21.15.44:0/2765163798 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 50) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fc02c030a60 con 0x56231507f710 2023-09-10T00:48:09.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.274+0000 7fc044301b80 10 client.26666 fetch_fsmap finished waiting for FSMap version 50 2023-09-10T00:48:09.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.274+0000 7fc044301b80 10 client.26666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:48:09.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.274+0000 7fc044301b80 20 client.26666 populate_metadata read hostname 'smithi044' 2023-09-10T00:48:09.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.274+0000 7fc044301b80 1 --2- 172.21.15.44:0/2765163798 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623150ae880 0x5623150b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.274+0000 7fc044301b80 4 client.26666 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:48:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.274+0000 7fc044301b80 1 -- 172.21.15.44:0/2765163798 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562314fe68a0 con 0x5623150ae880 2023-09-10T00:48:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.275+0000 7fc0356d6700 1 --2- 172.21.15.44:0/2765163798 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623150ae880 0x5623150b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.275+0000 7fc0197fa700 10 client.26666 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.275+0000 7fc0356d6700 1 --2- 172.21.15.44:0/2765163798 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623150ae880 0x5623150b0c70 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3647b78c63d612e server_cookie=9b454aadd7410b1c in_seq=0 out_seq=0 2023-09-10T00:48:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.276+0000 7fc0197fa700 1 -- 172.21.15.44:0/2765163798 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562314fe68a0 con 0x5623150ae880 2023-09-10T00:48:09.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.276+0000 7fc0197fa700 10 client.26666 handle_command_reply: tid=0 2023-09-10T00:48:09.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.276+0000 7fc044301b80 1 client.26666 shutdown 2023-09-10T00:48:09.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.276+0000 7fc044301b80 1 -- 172.21.15.44:0/2765163798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231507f710 msgr2=0x56231508dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:09.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.276+0000 7fc044301b80 1 --2- 172.21.15.44:0/2765163798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231507f710 0x56231508dcc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009990 tx=0x7fc02c00f710 comp rx=0 tx=0).stop 2023-09-10T00:48:09.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.276+0000 7fc044301b80 1 -- 172.21.15.44:0/2765163798 shutdown_connections 2023-09-10T00:48:09.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.276+0000 7fc044301b80 1 --2- 172.21.15.44:0/2765163798 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623150ae880 0x5623150b0c70 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.276+0000 7fc044301b80 1 --2- 172.21.15.44:0/2765163798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56231508e250 0x56231508b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.276+0000 7fc044301b80 1 --2- 172.21.15.44:0/2765163798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231507f710 0x56231508dcc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.276+0000 7fc0197fa700 0 client.26666 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:09.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.276+0000 7fc044301b80 1 --2- 172.21.15.44:0/2765163798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562315080030 0x56231508af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.276+0000 7fc044301b80 1 -- 172.21.15.44:0/2765163798 >> 172.21.15.44:0/2765163798 conn(0x562314ff4090 msgr2=0x56231507a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:09.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.276+0000 7fc044301b80 1 -- 172.21.15.44:0/2765163798 shutdown_connections 2023-09-10T00:48:09.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.276+0000 7fc044301b80 1 -- 172.21.15.44:0/2765163798 wait complete. 2023-09-10T00:48:09.292 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.276+0000 7fc044301b80 20 client.26666 trim_cache size 0 max 16384 2023-09-10T00:48:09.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.298+0000 7fc044301b80 1 Processor -- start 2023-09-10T00:48:09.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.298+0000 7fc044301b80 1 -- start start 2023-09-10T00:48:09.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.298+0000 7fc044301b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56231507f710 0x562315094de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.298+0000 7fc044301b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562315080030 0x5623150862f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.298+0000 7fc044301b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56231508e250 0x562315086830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.298+0000 7fc044301b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56231507b780 con 0x56231507f710 2023-09-10T00:48:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.298+0000 7fc044301b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562315074d70 con 0x562315080030 2023-09-10T00:48:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.298+0000 7fc044301b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623150ab7f0 con 0x56231508e250 2023-09-10T00:48:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.298+0000 7fc035ed7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56231507f710 0x562315094de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.298+0000 7fc0356d6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562315080030 0x5623150862f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:09.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.298+0000 7fc035ed7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56231507f710 0x562315094de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51648/0 (socket says 172.21.15.44:51648) 2023-09-10T00:48:09.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.298+0000 7fc035ed7700 1 -- 172.21.15.44:0/473723127 learned_addr learned my addr 172.21.15.44:0/473723127 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:09.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.298+0000 7fc0366d8700 1 --2- 172.21.15.44:0/473723127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56231508e250 0x562315086830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:09.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.299+0000 7fc0356d6700 1 -- 172.21.15.44:0/473723127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56231508e250 msgr2=0x562315086830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:09.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.299+0000 7fc0356d6700 1 --2- 172.21.15.44:0/473723127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56231508e250 0x562315086830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.299+0000 7fc0356d6700 1 -- 172.21.15.44:0/473723127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56231507f710 msgr2=0x562315094de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:09.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.299+0000 7fc0356d6700 1 --2- 172.21.15.44:0/473723127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56231507f710 0x562315094de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.299+0000 7fc0356d6700 1 -- 172.21.15.44:0/473723127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562314e24250 con 0x562315080030 2023-09-10T00:48:09.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.299+0000 7fc0366d8700 1 --2- 172.21.15.44:0/473723127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56231508e250 0x562315086830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:48:09.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.299+0000 7fc0356d6700 1 --2- 172.21.15.44:0/473723127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562315080030 0x5623150862f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc0200c9820 tx=0x7fc020009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afde81fbb2d806c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:09.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.299+0000 7fc024ff9700 1 -- 172.21.15.44:0/473723127 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0200acea0 con 0x562315080030 2023-09-10T00:48:09.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.299+0000 7fc024ff9700 1 -- 172.21.15.44:0/473723127 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc020010880 con 0x562315080030 2023-09-10T00:48:09.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.299+0000 7fc024ff9700 1 -- 172.21.15.44:0/473723127 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc020095c90 con 0x562315080030 2023-09-10T00:48:09.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.300+0000 7fc044301b80 1 -- 172.21.15.44:0/473723127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562315080030 msgr2=0x5623150862f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:09.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.300+0000 7fc044301b80 1 --2- 172.21.15.44:0/473723127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562315080030 0x5623150862f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc0200c9820 tx=0x7fc020009790 comp rx=0 tx=0).stop 2023-09-10T00:48:09.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.300+0000 7fc044301b80 1 -- 172.21.15.44:0/473723127 shutdown_connections 2023-09-10T00:48:09.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.300+0000 7fc044301b80 1 --2- 172.21.15.44:0/473723127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56231508e250 0x562315086830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.300+0000 7fc044301b80 1 --2- 172.21.15.44:0/473723127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56231507f710 0x562315094de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.300+0000 7fc044301b80 1 --2- 172.21.15.44:0/473723127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562315080030 0x5623150862f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.300+0000 7fc044301b80 1 -- 172.21.15.44:0/473723127 >> 172.21.15.44:0/473723127 conn(0x562314ff4090 msgr2=0x5623150ab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:09.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.300+0000 7fc044301b80 1 -- 172.21.15.44:0/473723127 shutdown_connections 2023-09-10T00:48:09.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.300+0000 7fc044301b80 1 -- 172.21.15.44:0/473723127 wait complete. 2023-09-10T00:48:09.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.300+0000 7fc044301b80 1 -- start start 2023-09-10T00:48:09.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.300+0000 7fc044301b80 1 Processor -- start 2023-09-10T00:48:09.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc044301b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56231507f710 0x562315094aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.308 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc044301b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562315080030 0x562315092dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.308 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc044301b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231508e250 0x562315093300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.308 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc044301b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562315093880 con 0x562315080030 2023-09-10T00:48:09.308 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc044301b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623150939c0 con 0x56231508e250 2023-09-10T00:48:09.308 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc044301b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562315093b80 con 0x56231507f710 2023-09-10T00:48:09.309 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc044301b80 10 client.0 _ll_register_callbacks cb 0x7fc02c005660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:48:09.309 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc0366d8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231508e250 0x562315093300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:09.309 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc0366d8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231508e250 0x562315093300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53396/0 (socket says 172.21.15.44:53396) 2023-09-10T00:48:09.309 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc0366d8700 1 -- 172.21.15.44:0/3351164181 learned_addr learned my addr 172.21.15.44:0/3351164181 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:09.309 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc035ed7700 1 --2- 172.21.15.44:0/3351164181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56231507f710 0x562315094aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:09.309 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc0356d6700 1 --2- 172.21.15.44:0/3351164181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562315080030 0x562315092dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:09.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc0366d8700 1 -- 172.21.15.44:0/3351164181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56231507f710 msgr2=0x562315094aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:09.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc0366d8700 1 --2- 172.21.15.44:0/3351164181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56231507f710 0x562315094aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc0366d8700 1 -- 172.21.15.44:0/3351164181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562315080030 msgr2=0x562315092dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:09.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc0366d8700 1 --2- 172.21.15.44:0/3351164181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562315080030 0x562315092dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.301+0000 7fc0366d8700 1 -- 172.21.15.44:0/3351164181 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03019bde0 con 0x56231508e250 2023-09-10T00:48:09.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.302+0000 7fc0366d8700 1 --2- 172.21.15.44:0/3351164181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231508e250 0x562315093300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc02800ef20 tx=0x7fc028007700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57c40b70b087824a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:09.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.302+0000 7fc0197fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:48:09.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.302+0000 7fc0197fa700 1 -- 172.21.15.44:0/3351164181 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028021680 con 0x56231508e250 2023-09-10T00:48:09.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.302+0000 7fc0197fa700 1 -- 172.21.15.44:0/3351164181 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fc02801c070 con 0x56231508e250 2023-09-10T00:48:09.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.302+0000 7fc044301b80 1 -- 172.21.15.44:0/3351164181 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc02c01e040 con 0x56231508e250 2023-09-10T00:48:09.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.302+0000 7fc0197fa700 1 -- 172.21.15.44:0/3351164181 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0280217e0 con 0x56231508e250 2023-09-10T00:48:09.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.302+0000 7fc044301b80 10 client.26672 fetch_fsmap learned FSMap version 50 2023-09-10T00:48:09.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.302+0000 7fc044301b80 1 -- 172.21.15.44:0/3351164181 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x562314dfc2b0 con 0x56231508e250 2023-09-10T00:48:09.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.302+0000 7fc0197fa700 1 -- 172.21.15.44:0/3351164181 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc02c01e040 con 0x56231508e250 2023-09-10T00:48:09.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.302+0000 7fc0197fa700 1 -- 172.21.15.44:0/3351164181 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 50) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fc028016920 con 0x56231508e250 2023-09-10T00:48:09.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.303+0000 7fc044301b80 10 client.26672 fetch_fsmap finished waiting for FSMap version 50 2023-09-10T00:48:09.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.303+0000 7fc044301b80 10 client.26672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:48:09.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.303+0000 7fc044301b80 20 client.26672 populate_metadata read hostname 'smithi044' 2023-09-10T00:48:09.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.303+0000 7fc044301b80 1 --2- 172.21.15.44:0/3351164181 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562315103ce0 0x562315106100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:09.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.303+0000 7fc044301b80 4 client.26672 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:48:09.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.303+0000 7fc044301b80 1 -- 172.21.15.44:0/3351164181 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562314fca920 con 0x562315103ce0 2023-09-10T00:48:09.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.303+0000 7fc035ed7700 1 --2- 172.21.15.44:0/3351164181 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562315103ce0 0x562315106100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.303+0000 7fc0197fa700 10 client.26672 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.303+0000 7fc035ed7700 1 --2- 172.21.15.44:0/3351164181 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562315103ce0 0x562315106100 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afa230ee43eef1ca server_cookie=26476f721eae3784 in_seq=0 out_seq=0 2023-09-10T00:48:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.303+0000 7fc0197fa700 1 -- 172.21.15.44:0/3351164181 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562314fca920 con 0x562315103ce0 2023-09-10T00:48:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.303+0000 7fc0197fa700 10 client.26672 handle_command_reply: tid=0 2023-09-10T00:48:09.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.304+0000 7fc044301b80 1 client.26672 shutdown 2023-09-10T00:48:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.304+0000 7fc044301b80 1 -- 172.21.15.44:0/3351164181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231508e250 msgr2=0x562315093300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.304+0000 7fc044301b80 1 --2- 172.21.15.44:0/3351164181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231508e250 0x562315093300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc02800ef20 tx=0x7fc028007700 comp rx=0 tx=0).stop 2023-09-10T00:48:09.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.304+0000 7fc044301b80 1 -- 172.21.15.44:0/3351164181 shutdown_connections 2023-09-10T00:48:09.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.304+0000 7fc044301b80 1 --2- 172.21.15.44:0/3351164181 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562315103ce0 0x562315106100 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.304+0000 7fc044301b80 1 --2- 172.21.15.44:0/3351164181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56231508e250 0x562315093300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.304+0000 7fc044301b80 1 --2- 172.21.15.44:0/3351164181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56231507f710 0x562315094aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.304+0000 7fc0197fa700 0 client.26672 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:09.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.304+0000 7fc044301b80 1 --2- 172.21.15.44:0/3351164181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562315080030 0x562315092dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.304+0000 7fc044301b80 1 -- 172.21.15.44:0/3351164181 >> 172.21.15.44:0/3351164181 conn(0x562314ff4090 msgr2=0x5623150862f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:09.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.304+0000 7fc044301b80 1 -- 172.21.15.44:0/3351164181 shutdown_connections 2023-09-10T00:48:09.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.304+0000 7fc044301b80 1 -- 172.21.15.44:0/3351164181 wait complete. 2023-09-10T00:48:09.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.304+0000 7fc044301b80 20 client.26672 trim_cache size 0 max 16384 2023-09-10T00:48:09.318 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:48:09.318 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:48:09.318 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:48:09.318 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:48:09.319 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:48:09.319 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:48:09.319 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:48:09.319 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:48:09.319 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:48:09.320 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:48:09.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.305+0000 7fc03813b700 1 -- 172.21.15.44:0/1147782084 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc01c0a07b0 msgr2=0x7fc01c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:09.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.305+0000 7fc03813b700 1 --2- 172.21.15.44:0/1147782084 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc01c0a07b0 0x7fc01c0a2c70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc02c006010 tx=0x7fc02c00b260 comp rx=0 tx=0).stop 2023-09-10T00:48:09.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.305+0000 7fc03813b700 1 -- 172.21.15.44:0/1147782084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0300ffe00 msgr2=0x7fc030196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:09.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.305+0000 7fc03813b700 1 --2- 172.21.15.44:0/1147782084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0300ffe00 0x7fc030196b40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc02000e9f0 tx=0x7fc02000ed00 comp rx=0 tx=0).stop 2023-09-10T00:48:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.305+0000 7fc03813b700 1 -- 172.21.15.44:0/1147782084 shutdown_connections 2023-09-10T00:48:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.305+0000 7fc03813b700 1 --2- 172.21.15.44:0/1147782084 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc01c0a07b0 0x7fc01c0a2c70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.305+0000 7fc03813b700 1 --2- 172.21.15.44:0/1147782084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0301054e0 0x7fc03019afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.305+0000 7fc03813b700 1 --2- 172.21.15.44:0/1147782084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0300ff440 0x7fc030196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.305+0000 7fc03813b700 1 --2- 172.21.15.44:0/1147782084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0300ffe00 0x7fc030196b40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:09.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.305+0000 7fc03813b700 1 -- 172.21.15.44:0/1147782084 >> 172.21.15.44:0/1147782084 conn(0x7fc0300faf50 msgr2=0x7fc0300fb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:09.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.305+0000 7fc03813b700 1 -- 172.21.15.44:0/1147782084 shutdown_connections 2023-09-10T00:48:09.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:09.305+0000 7fc03813b700 1 -- 172.21.15.44:0/1147782084 wait complete. 2023-09-10T00:48:09.323 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:48:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.191+0000 7f193ae3b700 1 -- 172.21.15.44:0/1856012668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1934106650 msgr2=0x7f1934106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.191+0000 7f193ae3b700 1 --2- 172.21.15.44:0/1856012668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1934106650 0x7f1934106a30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1928009920 tx=0x7f1928009c30 comp rx=0 tx=0).stop 2023-09-10T00:48:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.191+0000 7f193ae3b700 1 -- 172.21.15.44:0/1856012668 shutdown_connections 2023-09-10T00:48:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.191+0000 7f193ae3b700 1 --2- 172.21.15.44:0/1856012668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1934100f50 0x7f193410d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.191+0000 7f193ae3b700 1 --2- 172.21.15.44:0/1856012668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1934106650 0x7f1934106a30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.191+0000 7f193ae3b700 1 --2- 172.21.15.44:0/1856012668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1934100590 0x7f1934100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.191+0000 7f193ae3b700 1 -- 172.21.15.44:0/1856012668 >> 172.21.15.44:0/1856012668 conn(0x7f19340fc060 msgr2=0x7f19340fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.191+0000 7f193ae3b700 1 -- 172.21.15.44:0/1856012668 shutdown_connections 2023-09-10T00:48:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.191+0000 7f193ae3b700 1 -- 172.21.15.44:0/1856012668 wait complete. 2023-09-10T00:48:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f193ae3b700 1 Processor -- start 2023-09-10T00:48:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f193ae3b700 1 -- start start 2023-09-10T00:48:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f193ae3b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1934100590 0x7f193419c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f193ae3b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1934100f50 0x7f193419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f193ae3b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1934106650 0x7f1934196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f193ae3b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1934196ab0 con 0x7f1934106650 2023-09-10T00:48:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f193ae3b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1934196c20 con 0x7f1934100f50 2023-09-10T00:48:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f193ae3b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1934196de0 con 0x7f1934100590 2023-09-10T00:48:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f1938bd7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1934100590 0x7f193419c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f19393d8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1934106650 0x7f1934196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f1938bd7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1934100590 0x7f193419c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58550/0 (socket says 172.21.15.44:58550) 2023-09-10T00:48:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f1938bd7700 1 -- 172.21.15.44:0/958099870 learned_addr learned my addr 172.21.15.44:0/958099870 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f1933fff700 1 --2- 172.21.15.44:0/958099870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1934100f50 0x7f193419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f1938bd7700 1 -- 172.21.15.44:0/958099870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1934100f50 msgr2=0x7f193419cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f1938bd7700 1 --2- 172.21.15.44:0/958099870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1934100f50 0x7f193419cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f1938bd7700 1 -- 172.21.15.44:0/958099870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1934106650 msgr2=0x7f1934196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f1938bd7700 1 --2- 172.21.15.44:0/958099870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1934106650 0x7f1934196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.192+0000 7f1938bd7700 1 -- 172.21.15.44:0/958099870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192c009d00 con 0x7f1934100590 2023-09-10T00:48:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.193+0000 7f1938bd7700 1 --2- 172.21.15.44:0/958099870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1934100590 0x7f193419c5e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1928009840 tx=0x7f192800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.193+0000 7f1931ffb700 1 -- 172.21.15.44:0/958099870 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192801c070 con 0x7f1934100590 2023-09-10T00:48:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.193+0000 7f193ae3b700 1 -- 172.21.15.44:0/958099870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1928009520 con 0x7f1934100590 2023-09-10T00:48:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.193+0000 7f1931ffb700 1 -- 172.21.15.44:0/958099870 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f192800ba70 con 0x7f1934100590 2023-09-10T00:48:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.193+0000 7f1931ffb700 1 -- 172.21.15.44:0/958099870 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1928003680 con 0x7f1934100590 2023-09-10T00:48:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.193+0000 7f193ae3b700 1 -- 172.21.15.44:0/958099870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19341a33f0 con 0x7f1934100590 2023-09-10T00:48:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.195+0000 7f1931ffb700 1 -- 172.21.15.44:0/958099870 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1928020030 con 0x7f1934100590 2023-09-10T00:48:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.196+0000 7f1931ffb700 1 --2- 172.21.15.44:0/958099870 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f19240a07b0 0x7f19240a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.196+0000 7f1933fff700 1 --2- 172.21.15.44:0/958099870 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f19240a07b0 0x7f19240a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.196+0000 7f1931ffb700 1 -- 172.21.15.44:0/958099870 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f19280cd220 con 0x7f1934100590 2023-09-10T00:48:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.197+0000 7f1933fff700 1 --2- 172.21.15.44:0/958099870 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f19240a07b0 0x7f19240a2c70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1934105f80 tx=0x7f1920009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.197+0000 7f1947001b80 1 Processor -- start 2023-09-10T00:48:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.197+0000 7f1947001b80 1 -- start start 2023-09-10T00:48:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f1947001b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef915185b0 0x55ef91518990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f1947001b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef9150e300 0x55ef91510790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f1947001b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef91510cd0 0x55ef91513140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f1947001b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef914676d0 con 0x55ef915185b0 2023-09-10T00:48:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f1947001b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef91464ff0 con 0x55ef9150e300 2023-09-10T00:48:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f1947001b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef914608e0 con 0x55ef91510cd0 2023-09-10T00:48:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f1938bd7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef915185b0 0x55ef91518990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f1933fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef9150e300 0x55ef91510790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f1938bd7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef915185b0 0x55ef91518990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51676/0 (socket says 172.21.15.44:51676) 2023-09-10T00:48:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f1938bd7700 1 -- 172.21.15.44:0/317633346 learned_addr learned my addr 172.21.15.44:0/317633346 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f19393d8700 1 --2- 172.21.15.44:0/317633346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef91510cd0 0x55ef91513140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f1933fff700 1 -- 172.21.15.44:0/317633346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef91510cd0 msgr2=0x55ef91513140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f1933fff700 1 --2- 172.21.15.44:0/317633346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef91510cd0 0x55ef91513140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f1933fff700 1 -- 172.21.15.44:0/317633346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef915185b0 msgr2=0x55ef91518990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f1933fff700 1 --2- 172.21.15.44:0/317633346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef915185b0 0x55ef91518990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f1933fff700 1 -- 172.21.15.44:0/317633346 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef9128cd80 con 0x55ef9150e300 2023-09-10T00:48:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f1933fff700 1 --2- 172.21.15.44:0/317633346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef9150e300 0x55ef91510790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1920011c90 tx=0x7f1920014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2af94b65fdcd2ac server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.198+0000 7f191effd700 1 -- 172.21.15.44:0/317633346 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920004230 con 0x55ef9150e300 2023-09-10T00:48:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.199+0000 7f191effd700 1 -- 172.21.15.44:0/317633346 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1920021070 con 0x55ef9150e300 2023-09-10T00:48:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.199+0000 7f191effd700 1 -- 172.21.15.44:0/317633346 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920003b20 con 0x55ef9150e300 2023-09-10T00:48:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.199+0000 7f1947001b80 1 -- 172.21.15.44:0/317633346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef9150e300 msgr2=0x55ef91510790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.199+0000 7f1947001b80 1 --2- 172.21.15.44:0/317633346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef9150e300 0x55ef91510790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1920011c90 tx=0x7f1920014000 comp rx=0 tx=0).stop 2023-09-10T00:48:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.199+0000 7f1947001b80 1 -- 172.21.15.44:0/317633346 shutdown_connections 2023-09-10T00:48:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.199+0000 7f1947001b80 1 --2- 172.21.15.44:0/317633346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef91510cd0 0x55ef91513140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.199+0000 7f1947001b80 1 --2- 172.21.15.44:0/317633346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef915185b0 0x55ef91518990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.199+0000 7f1947001b80 1 --2- 172.21.15.44:0/317633346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef9150e300 0x55ef91510790 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.199+0000 7f1947001b80 1 -- 172.21.15.44:0/317633346 >> 172.21.15.44:0/317633346 conn(0x55ef91483030 msgr2=0x55ef91509560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.199+0000 7f1947001b80 1 -- 172.21.15.44:0/317633346 shutdown_connections 2023-09-10T00:48:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.199+0000 7f1947001b80 1 -- 172.21.15.44:0/317633346 wait complete. 2023-09-10T00:48:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.200+0000 7f1947001b80 1 -- start start 2023-09-10T00:48:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.200+0000 7f1947001b80 1 Processor -- start 2023-09-10T00:48:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.200+0000 7f1947001b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef9150e300 0x55ef91514e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.200+0000 7f1947001b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef91510cd0 0x55ef91515380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.200+0000 7f1947001b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef915185b0 0x55ef915158c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.200+0000 7f1947001b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef913feb10 con 0x55ef9150e300 2023-09-10T00:48:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.200+0000 7f1947001b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef913fe810 con 0x55ef915185b0 2023-09-10T00:48:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.200+0000 7f1947001b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef912e28c0 con 0x55ef91510cd0 2023-09-10T00:48:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1947001b80 10 client.0 _ll_register_callbacks cb 0x55ef91348a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:48:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1938bd7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef9150e300 0x55ef91514e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1938bd7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef9150e300 0x55ef91514e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51682/0 (socket says 172.21.15.44:51682) 2023-09-10T00:48:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1938bd7700 1 -- 172.21.15.44:0/2123026285 learned_addr learned my addr 172.21.15.44:0/2123026285 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1933fff700 1 --2- 172.21.15.44:0/2123026285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef91510cd0 0x55ef91515380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f19393d8700 1 --2- 172.21.15.44:0/2123026285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef915185b0 0x55ef915158c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1938bd7700 1 -- 172.21.15.44:0/2123026285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef91510cd0 msgr2=0x55ef91515380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1938bd7700 1 --2- 172.21.15.44:0/2123026285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef91510cd0 0x55ef91515380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1938bd7700 1 -- 172.21.15.44:0/2123026285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef915185b0 msgr2=0x55ef915158c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1938bd7700 1 --2- 172.21.15.44:0/2123026285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef915185b0 0x55ef915158c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1938bd7700 1 -- 172.21.15.44:0/2123026285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1920017070 con 0x55ef9150e300 2023-09-10T00:48:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f19393d8700 1 --2- 172.21.15.44:0/2123026285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef915185b0 0x55ef915158c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1933fff700 1 --2- 172.21.15.44:0/2123026285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef91510cd0 0x55ef91515380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1938bd7700 1 --2- 172.21.15.44:0/2123026285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef9150e300 0x55ef91514e40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f19280cab20 tx=0x7f1928099c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92c2a5e80755589 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1913fff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:48:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1913fff700 1 -- 172.21.15.44:0/2123026285 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192809e140 con 0x55ef9150e300 2023-09-10T00:48:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1913fff700 1 -- 172.21.15.44:0/2123026285 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f19280a22f0 con 0x55ef9150e300 2023-09-10T00:48:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1947001b80 1 -- 172.21.15.44:0/2123026285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef913b8490 con 0x55ef9150e300 2023-09-10T00:48:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1913fff700 1 -- 172.21.15.44:0/2123026285 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19280b8170 con 0x55ef9150e300 2023-09-10T00:48:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1947001b80 10 client.17025 fetch_fsmap learned FSMap version 50 2023-09-10T00:48:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.201+0000 7f1947001b80 1 -- 172.21.15.44:0/2123026285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55ef91275c00 con 0x55ef9150e300 2023-09-10T00:48:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.202+0000 7f1913fff700 1 -- 172.21.15.44:0/2123026285 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1928064080 con 0x55ef9150e300 2023-09-10T00:48:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.202+0000 7f1913fff700 1 -- 172.21.15.44:0/2123026285 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 50) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f1928096a20 con 0x55ef9150e300 2023-09-10T00:48:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.202+0000 7f1947001b80 10 client.17025 fetch_fsmap finished waiting for FSMap version 50 2023-09-10T00:48:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.202+0000 7f1947001b80 10 client.17025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:48:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.202+0000 7f1947001b80 20 client.17025 populate_metadata read hostname 'smithi044' 2023-09-10T00:48:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.202+0000 7f1947001b80 1 --2- 172.21.15.44:0/2123026285 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ef91530a40 0x55ef91532e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.202+0000 7f1947001b80 4 client.17025 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:48:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.202+0000 7f1947001b80 1 -- 172.21.15.44:0/2123026285 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef91475840 con 0x55ef91530a40 2023-09-10T00:48:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.202+0000 7f1933fff700 1 --2- 172.21.15.44:0/2123026285 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ef91530a40 0x55ef91532e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.202+0000 7f1913fff700 10 client.17025 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.202+0000 7f1933fff700 1 --2- 172.21.15.44:0/2123026285 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ef91530a40 0x55ef91532e60 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c25a569fe8740073 server_cookie=59903de0b35d6882 in_seq=0 out_seq=0 2023-09-10T00:48:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.203+0000 7f1913fff700 1 -- 172.21.15.44:0/2123026285 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ef91475840 con 0x55ef91530a40 2023-09-10T00:48:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.203+0000 7f1913fff700 10 client.17025 handle_command_reply: tid=0 2023-09-10T00:48:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.203+0000 7f1947001b80 1 client.17025 shutdown 2023-09-10T00:48:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.203+0000 7f1947001b80 1 -- 172.21.15.44:0/2123026285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef9150e300 msgr2=0x55ef91514e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.203+0000 7f1947001b80 1 --2- 172.21.15.44:0/2123026285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef9150e300 0x55ef91514e40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f19280cab20 tx=0x7f1928099c70 comp rx=0 tx=0).stop 2023-09-10T00:48:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.203+0000 7f1947001b80 1 -- 172.21.15.44:0/2123026285 shutdown_connections 2023-09-10T00:48:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.203+0000 7f1947001b80 1 --2- 172.21.15.44:0/2123026285 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ef91530a40 0x55ef91532e60 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.203+0000 7f1947001b80 1 --2- 172.21.15.44:0/2123026285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef915185b0 0x55ef915158c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.203+0000 7f1913fff700 0 client.17025 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.203+0000 7f1947001b80 1 --2- 172.21.15.44:0/2123026285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef9150e300 0x55ef91514e40 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.203+0000 7f1947001b80 1 --2- 172.21.15.44:0/2123026285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef91510cd0 0x55ef91515380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.203+0000 7f1947001b80 1 -- 172.21.15.44:0/2123026285 >> 172.21.15.44:0/2123026285 conn(0x55ef91483030 msgr2=0x55ef915087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.203+0000 7f1947001b80 1 -- 172.21.15.44:0/2123026285 shutdown_connections 2023-09-10T00:48:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.203+0000 7f1947001b80 1 -- 172.21.15.44:0/2123026285 wait complete. 2023-09-10T00:48:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.203+0000 7f1947001b80 20 client.17025 trim_cache size 0 max 16384 2023-09-10T00:48:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.222+0000 7f1947001b80 1 Processor -- start 2023-09-10T00:48:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.223+0000 7f1947001b80 1 -- start start 2023-09-10T00:48:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.223+0000 7f1947001b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef9150e300 0x55ef91527cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.223+0000 7f1947001b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef91510cd0 0x55ef9150d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.223+0000 7f1947001b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef915185b0 0x55ef9150dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.223+0000 7f1947001b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9150a960 con 0x55ef9150e300 2023-09-10T00:48:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.223+0000 7f1947001b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef91513720 con 0x55ef91510cd0 2023-09-10T00:48:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.223+0000 7f1947001b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef91528210 con 0x55ef915185b0 2023-09-10T00:48:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.223+0000 7f1938bd7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef9150e300 0x55ef91527cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.223+0000 7f1938bd7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef9150e300 0x55ef91527cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51690/0 (socket says 172.21.15.44:51690) 2023-09-10T00:48:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.223+0000 7f1938bd7700 1 -- 172.21.15.44:0/3805061013 learned_addr learned my addr 172.21.15.44:0/3805061013 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.223+0000 7f19393d8700 1 --2- 172.21.15.44:0/3805061013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef915185b0 0x55ef9150dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.223+0000 7f1933fff700 1 --2- 172.21.15.44:0/3805061013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef91510cd0 0x55ef9150d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.223+0000 7f1933fff700 1 -- 172.21.15.44:0/3805061013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef915185b0 msgr2=0x55ef9150dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.224+0000 7f1933fff700 1 --2- 172.21.15.44:0/3805061013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef915185b0 0x55ef9150dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.224+0000 7f1933fff700 1 -- 172.21.15.44:0/3805061013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef9150e300 msgr2=0x55ef91527cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.224+0000 7f1933fff700 1 --2- 172.21.15.44:0/3805061013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef9150e300 0x55ef91527cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.224+0000 7f1933fff700 1 -- 172.21.15.44:0/3805061013 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef912b31f0 con 0x55ef91510cd0 2023-09-10T00:48:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.224+0000 7f1938bd7700 1 --2- 172.21.15.44:0/3805061013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef9150e300 0x55ef91527cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.224+0000 7f19393d8700 1 --2- 172.21.15.44:0/3805061013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef915185b0 0x55ef9150dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.224+0000 7f1933fff700 1 --2- 172.21.15.44:0/3805061013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef91510cd0 0x55ef9150d740 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1920005920 tx=0x7f1920005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39ee8a69cfc8ff5d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.224+0000 7f191f7fe700 1 -- 172.21.15.44:0/3805061013 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920037070 con 0x55ef91510cd0 2023-09-10T00:48:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.224+0000 7f191f7fe700 1 -- 172.21.15.44:0/3805061013 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1920033e80 con 0x55ef91510cd0 2023-09-10T00:48:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.224+0000 7f191f7fe700 1 -- 172.21.15.44:0/3805061013 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192002a040 con 0x55ef91510cd0 2023-09-10T00:48:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.225+0000 7f1947001b80 1 -- 172.21.15.44:0/3805061013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef91510cd0 msgr2=0x55ef9150d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.225+0000 7f1947001b80 1 --2- 172.21.15.44:0/3805061013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef91510cd0 0x55ef9150d740 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1920005920 tx=0x7f1920005e40 comp rx=0 tx=0).stop 2023-09-10T00:48:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.225+0000 7f1947001b80 1 -- 172.21.15.44:0/3805061013 shutdown_connections 2023-09-10T00:48:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.225+0000 7f1947001b80 1 --2- 172.21.15.44:0/3805061013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef915185b0 0x55ef9150dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.225+0000 7f1947001b80 1 --2- 172.21.15.44:0/3805061013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef9150e300 0x55ef91527cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.225+0000 7f1947001b80 1 --2- 172.21.15.44:0/3805061013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef91510cd0 0x55ef9150d740 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.225+0000 7f1947001b80 1 -- 172.21.15.44:0/3805061013 >> 172.21.15.44:0/3805061013 conn(0x55ef91483030 msgr2=0x55ef91512940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.225+0000 7f1947001b80 1 -- 172.21.15.44:0/3805061013 shutdown_connections 2023-09-10T00:48:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.225+0000 7f1947001b80 1 -- 172.21.15.44:0/3805061013 wait complete. 2023-09-10T00:48:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.225+0000 7f1947001b80 1 -- start start 2023-09-10T00:48:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.225+0000 7f1947001b80 1 Processor -- start 2023-09-10T00:48:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f1947001b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef9150e300 0x55ef91578b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f1947001b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef91510cd0 0x55ef91579040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f1947001b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef915185b0 0x55ef9157db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f1947001b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9157e0f0 con 0x55ef915185b0 2023-09-10T00:48:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f1947001b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9157e260 con 0x55ef9150e300 2023-09-10T00:48:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f1947001b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9157e420 con 0x55ef91510cd0 2023-09-10T00:48:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f1947001b80 10 client.0 _ll_register_callbacks cb 0x7f192809a0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:48:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f1938bd7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef9150e300 0x55ef91578b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f19393d8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef915185b0 0x55ef9157db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f1938bd7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef9150e300 0x55ef91578b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53436/0 (socket says 172.21.15.44:53436) 2023-09-10T00:48:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f1938bd7700 1 -- 172.21.15.44:0/1357984166 learned_addr learned my addr 172.21.15.44:0/1357984166 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f19393d8700 1 --2- 172.21.15.44:0/1357984166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef915185b0 0x55ef9157db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51700/0 (socket says 172.21.15.44:51700) 2023-09-10T00:48:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f1933fff700 1 --2- 172.21.15.44:0/1357984166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef91510cd0 0x55ef91579040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f1938bd7700 1 -- 172.21.15.44:0/1357984166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef91510cd0 msgr2=0x55ef91579040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f1938bd7700 1 --2- 172.21.15.44:0/1357984166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef91510cd0 0x55ef91579040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f1938bd7700 1 -- 172.21.15.44:0/1357984166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef915185b0 msgr2=0x55ef9157db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f1938bd7700 1 --2- 172.21.15.44:0/1357984166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef915185b0 0x55ef9157db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f1938bd7700 1 -- 172.21.15.44:0/1357984166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef91275c00 con 0x55ef9150e300 2023-09-10T00:48:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.226+0000 7f19393d8700 1 --2- 172.21.15.44:0/1357984166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef915185b0 0x55ef9157db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.227+0000 7f1938bd7700 1 --2- 172.21.15.44:0/1357984166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef9150e300 0x55ef91578b00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f19280cab20 tx=0x7f1928009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7c1c20ca4513661 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.227+0000 7f1913fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:48:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.227+0000 7f1913fff700 1 -- 172.21.15.44:0/1357984166 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192809a570 con 0x55ef9150e300 2023-09-10T00:48:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.227+0000 7f1913fff700 1 -- 172.21.15.44:0/1357984166 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f19280a22f0 con 0x55ef9150e300 2023-09-10T00:48:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.227+0000 7f1947001b80 1 -- 172.21.15.44:0/1357984166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef914598c0 con 0x55ef9150e300 2023-09-10T00:48:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.227+0000 7f1913fff700 1 -- 172.21.15.44:0/1357984166 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1928096650 con 0x55ef9150e300 2023-09-10T00:48:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.227+0000 7f1947001b80 10 client.26690 fetch_fsmap learned FSMap version 50 2023-09-10T00:48:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.227+0000 7f1947001b80 1 -- 172.21.15.44:0/1357984166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f192c009290 con 0x55ef9150e300 2023-09-10T00:48:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.227+0000 7f1913fff700 1 -- 172.21.15.44:0/1357984166 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ef914598c0 con 0x55ef9150e300 2023-09-10T00:48:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.227+0000 7f1913fff700 1 -- 172.21.15.44:0/1357984166 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 50) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f19280a1e60 con 0x55ef9150e300 2023-09-10T00:48:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.227+0000 7f1947001b80 10 client.26690 fetch_fsmap finished waiting for FSMap version 50 2023-09-10T00:48:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.227+0000 7f1947001b80 10 client.26690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:48:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.227+0000 7f1947001b80 20 client.26690 populate_metadata read hostname 'smithi044' 2023-09-10T00:48:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.227+0000 7f1947001b80 1 --2- 172.21.15.44:0/1357984166 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ef915859c0 0x55ef91587de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.227+0000 7f1947001b80 4 client.26690 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:48:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.227+0000 7f1947001b80 1 -- 172.21.15.44:0/1357984166 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ef9141a050 con 0x55ef915859c0 2023-09-10T00:48:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.228+0000 7f1933fff700 1 --2- 172.21.15.44:0/1357984166 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ef915859c0 0x55ef91587de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.228+0000 7f1933fff700 1 --2- 172.21.15.44:0/1357984166 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ef915859c0 0x55ef91587de0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c896076d412e4abf server_cookie=6ba71cc2a2078e64 in_seq=0 out_seq=0 2023-09-10T00:48:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.228+0000 7f1913fff700 10 client.26690 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.244+0000 7f1913fff700 1 -- 172.21.15.44:0/1357984166 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ef9141a050 con 0x55ef915859c0 2023-09-10T00:48:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.244+0000 7f1913fff700 10 client.26690 handle_command_reply: tid=0 2023-09-10T00:48:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.244+0000 7f1947001b80 1 client.26690 shutdown 2023-09-10T00:48:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.244+0000 7f1947001b80 1 -- 172.21.15.44:0/1357984166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef9150e300 msgr2=0x55ef91578b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.244+0000 7f1947001b80 1 --2- 172.21.15.44:0/1357984166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef9150e300 0x55ef91578b00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f19280cab20 tx=0x7f1928009c30 comp rx=0 tx=0).stop 2023-09-10T00:48:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.244+0000 7f1947001b80 1 -- 172.21.15.44:0/1357984166 shutdown_connections 2023-09-10T00:48:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.244+0000 7f1947001b80 1 --2- 172.21.15.44:0/1357984166 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ef915859c0 0x55ef91587de0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.244+0000 7f1947001b80 1 --2- 172.21.15.44:0/1357984166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef915185b0 0x55ef9157db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.244+0000 7f1947001b80 1 --2- 172.21.15.44:0/1357984166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef9150e300 0x55ef91578b00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.244+0000 7f1913fff700 0 client.26690 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.244+0000 7f1947001b80 1 --2- 172.21.15.44:0/1357984166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef91510cd0 0x55ef91579040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.244+0000 7f1947001b80 1 -- 172.21.15.44:0/1357984166 >> 172.21.15.44:0/1357984166 conn(0x55ef91483030 msgr2=0x55ef91512940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.244+0000 7f1947001b80 1 -- 172.21.15.44:0/1357984166 shutdown_connections 2023-09-10T00:48:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.244+0000 7f1947001b80 1 -- 172.21.15.44:0/1357984166 wait complete. 2023-09-10T00:48:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.245+0000 7f1947001b80 20 client.26690 trim_cache size 0 max 16384 2023-09-10T00:48:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:48:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:48:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:48:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:48:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.245+0000 7f193ae3b700 1 -- 172.21.15.44:0/958099870 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f19240a07b0 msgr2=0x7f19240a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.245+0000 7f193ae3b700 1 --2- 172.21.15.44:0/958099870 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f19240a07b0 0x7f19240a2c70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1934105f80 tx=0x7f1920009260 comp rx=0 tx=0).stop 2023-09-10T00:48:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.245+0000 7f193ae3b700 1 -- 172.21.15.44:0/958099870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1934100590 msgr2=0x7f193419c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.245+0000 7f193ae3b700 1 --2- 172.21.15.44:0/958099870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1934100590 0x7f193419c5e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1928009840 tx=0x7f192800fa00 comp rx=0 tx=0).stop 2023-09-10T00:48:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.246+0000 7f193ae3b700 1 -- 172.21.15.44:0/958099870 shutdown_connections 2023-09-10T00:48:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.246+0000 7f193ae3b700 1 --2- 172.21.15.44:0/958099870 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f19240a07b0 0x7f19240a2c70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.246+0000 7f193ae3b700 1 --2- 172.21.15.44:0/958099870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1934106650 0x7f1934196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.246+0000 7f193ae3b700 1 --2- 172.21.15.44:0/958099870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1934100590 0x7f193419c5e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.246+0000 7f193ae3b700 1 --2- 172.21.15.44:0/958099870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1934100f50 0x7f193419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.246+0000 7f193ae3b700 1 -- 172.21.15.44:0/958099870 >> 172.21.15.44:0/958099870 conn(0x7f19340fc060 msgr2=0x7f19340fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.246+0000 7f193ae3b700 1 -- 172.21.15.44:0/958099870 shutdown_connections 2023-09-10T00:48:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:10.246+0000 7f193ae3b700 1 -- 172.21.15.44:0/958099870 wait complete. 2023-09-10T00:48:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:10 smithi131 ceph-mon[103713]: pgmap v231: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 3.4 MiB/s wr, 397 op/s 2023-09-10T00:48:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:10 smithi131 ceph-mon[103713]: osdmap e96: 12 total, 12 up, 12 in 2023-09-10T00:48:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:10 smithi044 ceph-mon[98676]: pgmap v231: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 3.4 MiB/s wr, 397 op/s 2023-09-10T00:48:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:10 smithi044 ceph-mon[98676]: osdmap e96: 12 total, 12 up, 12 in 2023-09-10T00:48:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:10 smithi099 ceph-mon[102692]: pgmap v231: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 3.4 MiB/s wr, 397 op/s 2023-09-10T00:48:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:10 smithi099 ceph-mon[102692]: osdmap e96: 12 total, 12 up, 12 in 2023-09-10T00:48:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:12 smithi131 ceph-mon[103713]: pgmap v233: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 4.3 MiB/s wr, 490 op/s 2023-09-10T00:48:13.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:12 smithi044 ceph-mon[98676]: pgmap v233: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 4.3 MiB/s wr, 490 op/s 2023-09-10T00:48:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:12 smithi099 ceph-mon[102692]: pgmap v233: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 4.3 MiB/s wr, 490 op/s 2023-09-10T00:48:14.278 INFO:teuthology.orchestra.run.smithi044.stderr:31600000 of 50000000 tuples (63%) done (elapsed 433.91 s, remaining 252.66 s) 2023-09-10T00:48:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:14 smithi131 ceph-mon[103713]: pgmap v234: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 3.6 MiB/s wr, 414 op/s 2023-09-10T00:48:15.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:14 smithi044 ceph-mon[98676]: pgmap v234: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 3.6 MiB/s wr, 414 op/s 2023-09-10T00:48:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:14 smithi099 ceph-mon[102692]: pgmap v234: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 3.6 MiB/s wr, 414 op/s 2023-09-10T00:48:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:16 smithi131 ceph-mon[103713]: pgmap v235: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 3.3 MiB/s wr, 377 op/s 2023-09-10T00:48:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:16 smithi044 ceph-mon[98676]: pgmap v235: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 3.3 MiB/s wr, 377 op/s 2023-09-10T00:48:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:16 smithi099 ceph-mon[102692]: pgmap v235: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 3.3 MiB/s wr, 377 op/s 2023-09-10T00:48:19.050 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:18 smithi131 ceph-mon[103713]: pgmap v236: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 3.3 MiB/s wr, 377 op/s 2023-09-10T00:48:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:18 smithi099 ceph-mon[102692]: pgmap v236: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 3.3 MiB/s wr, 377 op/s 2023-09-10T00:48:19.263 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:18 smithi044 ceph-mon[98676]: pgmap v236: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 3.3 MiB/s wr, 377 op/s 2023-09-10T00:48:21.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:20 smithi131 ceph-mon[103713]: pgmap v237: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 3.3 MiB/s wr, 377 op/s 2023-09-10T00:48:21.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:20 smithi044 ceph-mon[98676]: pgmap v237: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 3.3 MiB/s wr, 377 op/s 2023-09-10T00:48:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:20 smithi099 ceph-mon[102692]: pgmap v237: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 3.3 MiB/s wr, 377 op/s 2023-09-10T00:48:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:22 smithi131 ceph-mon[103713]: pgmap v238: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 3.1 MiB/s wr, 354 op/s 2023-09-10T00:48:23.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:22 smithi044 ceph-mon[98676]: pgmap v238: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 3.1 MiB/s wr, 354 op/s 2023-09-10T00:48:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:22 smithi099 ceph-mon[102692]: pgmap v238: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 3.1 MiB/s wr, 354 op/s 2023-09-10T00:48:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.373+0000 7f3484cdd700 1 -- 172.21.15.44:0/3695890906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3480100120 msgr2=0x7f348010c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.373+0000 7f3484cdd700 1 --2- 172.21.15.44:0/3695890906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3480100120 0x7f348010c330 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f347800b0d0 tx=0x7f347800b3e0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.373+0000 7f3484cdd700 1 -- 172.21.15.44:0/3695890906 shutdown_connections 2023-09-10T00:48:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.373+0000 7f3484cdd700 1 --2- 172.21.15.44:0/3695890906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3480100120 0x7f348010c330 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.373+0000 7f3484cdd700 1 --2- 172.21.15.44:0/3695890906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34801055b0 0x7f3480105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.373+0000 7f3484cdd700 1 --2- 172.21.15.44:0/3695890906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34800ff760 0x7f34800ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.373+0000 7f3484cdd700 1 -- 172.21.15.44:0/3695890906 >> 172.21.15.44:0/3695890906 conn(0x7f3480074f30 msgr2=0x7f3480075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.373+0000 7f3484cdd700 1 -- 172.21.15.44:0/3695890906 shutdown_connections 2023-09-10T00:48:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.373+0000 7f3484cdd700 1 -- 172.21.15.44:0/3695890906 wait complete. 2023-09-10T00:48:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f3484cdd700 1 Processor -- start 2023-09-10T00:48:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f3484cdd700 1 -- start start 2023-09-10T00:48:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f3484cdd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34800ff760 0x7f348019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f3484cdd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3480100120 0x7f348019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f3484cdd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34801055b0 0x7f3480196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f3484cdd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3480196b90 con 0x7f34800ff760 2023-09-10T00:48:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f3484cdd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3480196d00 con 0x7f34801055b0 2023-09-10T00:48:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f3484cdd700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3480196ec0 con 0x7f3480100120 2023-09-10T00:48:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f347e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34800ff760 0x7f348019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f347e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34800ff760 0x7f348019c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51708/0 (socket says 172.21.15.44:51708) 2023-09-10T00:48:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f347e59c700 1 -- 172.21.15.44:0/3728328198 learned_addr learned my addr 172.21.15.44:0/3728328198 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f347ed9d700 1 --2- 172.21.15.44:0/3728328198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34801055b0 0x7f3480196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f347dd9b700 1 --2- 172.21.15.44:0/3728328198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3480100120 0x7f348019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f347e59c700 1 -- 172.21.15.44:0/3728328198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3480100120 msgr2=0x7f348019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f347e59c700 1 --2- 172.21.15.44:0/3728328198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3480100120 0x7f348019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f347e59c700 1 -- 172.21.15.44:0/3728328198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34801055b0 msgr2=0x7f3480196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f347e59c700 1 --2- 172.21.15.44:0/3728328198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34801055b0 0x7f3480196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.374+0000 7f347e59c700 1 -- 172.21.15.44:0/3728328198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3478009d00 con 0x7f34800ff760 2023-09-10T00:48:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.375+0000 7f347e59c700 1 --2- 172.21.15.44:0/3728328198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34800ff760 0x7f348019c620 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f347000e9a0 tx=0x7f347000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.375+0000 7f346f7fe700 1 -- 172.21.15.44:0/3728328198 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470009790 con 0x7f34800ff760 2023-09-10T00:48:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.375+0000 7f346f7fe700 1 -- 172.21.15.44:0/3728328198 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f347000cbd0 con 0x7f34800ff760 2023-09-10T00:48:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.375+0000 7f3484cdd700 1 -- 172.21.15.44:0/3728328198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34801971a0 con 0x7f34800ff760 2023-09-10T00:48:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.375+0000 7f346f7fe700 1 -- 172.21.15.44:0/3728328198 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470018680 con 0x7f34800ff760 2023-09-10T00:48:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.375+0000 7f3484cdd700 1 -- 172.21.15.44:0/3728328198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34801a3610 con 0x7f34800ff760 2023-09-10T00:48:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.377+0000 7f346f7fe700 1 -- 172.21.15.44:0/3728328198 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f347000cd40 con 0x7f34800ff760 2023-09-10T00:48:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.377+0000 7f346f7fe700 1 --2- 172.21.15.44:0/3728328198 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f34680a07b0 0x7f34680a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.378+0000 7f346f7fe700 1 -- 172.21.15.44:0/3728328198 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f3470014070 con 0x7f34800ff760 2023-09-10T00:48:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.378+0000 7f347dd9b700 1 --2- 172.21.15.44:0/3728328198 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f34680a07b0 0x7f34680a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.378+0000 7f347dd9b700 1 --2- 172.21.15.44:0/3728328198 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f34680a07b0 0x7f34680a2c70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3480197990 tx=0x7f3474009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.378+0000 7f3490ea3b80 1 Processor -- start 2023-09-10T00:48:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f3490ea3b80 1 -- start start 2023-09-10T00:48:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f3490ea3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e971661a0 0x555e97166580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f3490ea3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e97160300 0x555e97160780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f3490ea3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e97160cc0 0x555e9716cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f3490ea3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e970b96d0 con 0x555e97160300 2023-09-10T00:48:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f3490ea3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e970b6ff0 con 0x555e97160cc0 2023-09-10T00:48:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f3490ea3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e970b28e0 con 0x555e971661a0 2023-09-10T00:48:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f347dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e97160300 0x555e97160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f347e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e971661a0 0x555e97166580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f347dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e97160300 0x555e97160780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51718/0 (socket says 172.21.15.44:51718) 2023-09-10T00:48:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f347dd9b700 1 -- 172.21.15.44:0/2785965162 learned_addr learned my addr 172.21.15.44:0/2785965162 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f347ed9d700 1 --2- 172.21.15.44:0/2785965162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e97160cc0 0x555e9716cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f347e59c700 1 -- 172.21.15.44:0/2785965162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e97160cc0 msgr2=0x555e9716cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f347e59c700 1 --2- 172.21.15.44:0/2785965162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e97160cc0 0x555e9716cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f347e59c700 1 -- 172.21.15.44:0/2785965162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e97160300 msgr2=0x555e97160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f347e59c700 1 --2- 172.21.15.44:0/2785965162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e97160300 0x555e97160780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.379+0000 7f347e59c700 1 -- 172.21.15.44:0/2785965162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e96eded80 con 0x555e971661a0 2023-09-10T00:48:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.380+0000 7f347e59c700 1 --2- 172.21.15.44:0/2785965162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e971661a0 0x555e97166580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f34700c99a0 tx=0x7f3470099060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f308168ce3e4772f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.380+0000 7f346cff9700 1 -- 172.21.15.44:0/2785965162 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470099700 con 0x555e971661a0 2023-09-10T00:48:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.380+0000 7f346cff9700 1 -- 172.21.15.44:0/2785965162 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f347009d530 con 0x555e971661a0 2023-09-10T00:48:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.380+0000 7f346cff9700 1 -- 172.21.15.44:0/2785965162 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470099960 con 0x555e971661a0 2023-09-10T00:48:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.381+0000 7f3490ea3b80 1 -- 172.21.15.44:0/2785965162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e971661a0 msgr2=0x555e97166580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.381+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/2785965162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e971661a0 0x555e97166580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f34700c99a0 tx=0x7f3470099060 comp rx=0 tx=0).stop 2023-09-10T00:48:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.381+0000 7f3490ea3b80 1 -- 172.21.15.44:0/2785965162 shutdown_connections 2023-09-10T00:48:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.381+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/2785965162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e97160cc0 0x555e9716cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.381+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/2785965162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e971661a0 0x555e97166580 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.381+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/2785965162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e97160300 0x555e97160780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.381+0000 7f3490ea3b80 1 -- 172.21.15.44:0/2785965162 >> 172.21.15.44:0/2785965162 conn(0x555e970d5030 msgr2=0x555e9715b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.381+0000 7f3490ea3b80 1 -- 172.21.15.44:0/2785965162 shutdown_connections 2023-09-10T00:48:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.381+0000 7f3490ea3b80 1 -- 172.21.15.44:0/2785965162 wait complete. 2023-09-10T00:48:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.381+0000 7f3490ea3b80 1 -- start start 2023-09-10T00:48:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.381+0000 7f3490ea3b80 1 Processor -- start 2023-09-10T00:48:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f3490ea3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e9716ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f3490ea3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e97160cc0 0x555e9716a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f3490ea3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e971661a0 0x555e9716a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f3490ea3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e97050b10 con 0x555e971661a0 2023-09-10T00:48:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f3490ea3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e97050810 con 0x555e97160cc0 2023-09-10T00:48:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f3490ea3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e96f348c0 con 0x555e97160300 2023-09-10T00:48:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f3490ea3b80 10 client.0 _ll_register_callbacks cb 0x555e96f9aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:48:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f347ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e971661a0 0x555e9716a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f347e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e9716ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f347dd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e97160cc0 0x555e9716a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f347ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e971661a0 0x555e9716a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51726/0 (socket says 172.21.15.44:51726) 2023-09-10T00:48:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f347ed9d700 1 -- 172.21.15.44:0/646595994 learned_addr learned my addr 172.21.15.44:0/646595994 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f347e59c700 1 --2- 172.21.15.44:0/646595994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e9716ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58604/0 (socket says 172.21.15.44:58604) 2023-09-10T00:48:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f347ed9d700 1 -- 172.21.15.44:0/646595994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 msgr2=0x555e9716ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f347ed9d700 1 --2- 172.21.15.44:0/646595994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e9716ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f347ed9d700 1 -- 172.21.15.44:0/646595994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e97160cc0 msgr2=0x555e9716a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f347ed9d700 1 --2- 172.21.15.44:0/646595994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e97160cc0 0x555e9716a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f347ed9d700 1 -- 172.21.15.44:0/646595994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34801a3610 con 0x555e971661a0 2023-09-10T00:48:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f347e59c700 1 --2- 172.21.15.44:0/646595994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e9716ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f347ed9d700 1 --2- 172.21.15.44:0/646595994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e971661a0 0x555e9716a760 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f3478008f10 tx=0x7f3478008f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b91262fbe95825 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f3465ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:48:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f3465ffb700 1 -- 172.21.15.44:0/646595994 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347800e040 con 0x555e971661a0 2023-09-10T00:48:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f3490ea3b80 1 -- 172.21.15.44:0/646595994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e9700a490 con 0x555e971661a0 2023-09-10T00:48:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.382+0000 7f3465ffb700 1 -- 172.21.15.44:0/646595994 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f34780074e0 con 0x555e971661a0 2023-09-10T00:48:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.383+0000 7f3465ffb700 1 -- 172.21.15.44:0/646595994 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3478006080 con 0x555e971661a0 2023-09-10T00:48:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.383+0000 7f3490ea3b80 10 client.17043 fetch_fsmap learned FSMap version 50 2023-09-10T00:48:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.383+0000 7f3490ea3b80 1 -- 172.21.15.44:0/646595994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x555e96ec7c00 con 0x555e971661a0 2023-09-10T00:48:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.383+0000 7f3465ffb700 1 -- 172.21.15.44:0/646595994 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555e9700a490 con 0x555e971661a0 2023-09-10T00:48:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.383+0000 7f3465ffb700 1 -- 172.21.15.44:0/646595994 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 50) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f347800b100 con 0x555e971661a0 2023-09-10T00:48:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.383+0000 7f3490ea3b80 10 client.17043 fetch_fsmap finished waiting for FSMap version 50 2023-09-10T00:48:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.383+0000 7f3490ea3b80 10 client.17043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:48:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.383+0000 7f3490ea3b80 20 client.17043 populate_metadata read hostname 'smithi044' 2023-09-10T00:48:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.383+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/646595994 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555e9717a480 0x555e9717c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.383+0000 7f3490ea3b80 4 client.17043 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:48:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.383+0000 7f3490ea3b80 1 -- 172.21.15.44:0/646595994 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e970c7840 con 0x555e9717a480 2023-09-10T00:48:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.383+0000 7f347e59c700 1 --2- 172.21.15.44:0/646595994 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555e9717a480 0x555e9717c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.384+0000 7f347e59c700 1 --2- 172.21.15.44:0/646595994 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555e9717a480 0x555e9717c8a0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4302c24dc86c9c7 server_cookie=8a268ed72a3fa866 in_seq=0 out_seq=0 2023-09-10T00:48:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.384+0000 7f3465ffb700 10 client.17043 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.384+0000 7f3465ffb700 1 -- 172.21.15.44:0/646595994 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555e970c7840 con 0x555e9717a480 2023-09-10T00:48:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.384+0000 7f3465ffb700 10 client.17043 handle_command_reply: tid=0 2023-09-10T00:48:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.384+0000 7f3490ea3b80 1 client.17043 shutdown 2023-09-10T00:48:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.384+0000 7f3490ea3b80 1 -- 172.21.15.44:0/646595994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e971661a0 msgr2=0x555e9716a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.384+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/646595994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e971661a0 0x555e9716a760 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f3478008f10 tx=0x7f3478008f40 comp rx=0 tx=0).stop 2023-09-10T00:48:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.385+0000 7f3490ea3b80 1 -- 172.21.15.44:0/646595994 shutdown_connections 2023-09-10T00:48:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.385+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/646595994 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555e9717a480 0x555e9717c8a0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.385+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/646595994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e971661a0 0x555e9716a760 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.385+0000 7f3465ffb700 0 client.17043 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.385+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/646595994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e9716ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.385+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/646595994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e97160cc0 0x555e9716a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.385+0000 7f3490ea3b80 1 -- 172.21.15.44:0/646595994 >> 172.21.15.44:0/646595994 conn(0x555e970d5030 msgr2=0x555e9715a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.385+0000 7f3490ea3b80 1 -- 172.21.15.44:0/646595994 shutdown_connections 2023-09-10T00:48:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.385+0000 7f3490ea3b80 1 -- 172.21.15.44:0/646595994 wait complete. 2023-09-10T00:48:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.385+0000 7f3490ea3b80 20 client.17043 trim_cache size 0 max 16384 2023-09-10T00:48:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.400+0000 7f3490ea3b80 1 Processor -- start 2023-09-10T00:48:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.400+0000 7f3490ea3b80 1 -- start start 2023-09-10T00:48:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.400+0000 7f3490ea3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e971714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.400+0000 7f3490ea3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e97160cc0 0x555e9715b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.400+0000 7f3490ea3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e971661a0 0x555e9715b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.400+0000 7f3490ea3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9700a490 con 0x555e971661a0 2023-09-10T00:48:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.400+0000 7f3490ea3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e971610a0 con 0x555e97160cc0 2023-09-10T00:48:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.400+0000 7f3490ea3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e97165e00 con 0x555e97160300 2023-09-10T00:48:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.401+0000 7f347ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e971661a0 0x555e9715b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.401+0000 7f347e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e971714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.401+0000 7f347e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e971714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58612/0 (socket says 172.21.15.44:58612) 2023-09-10T00:48:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.401+0000 7f347ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e971661a0 0x555e9715b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51734/0 (socket says 172.21.15.44:51734) 2023-09-10T00:48:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.401+0000 7f347e59c700 1 -- 172.21.15.44:0/1761655036 learned_addr learned my addr 172.21.15.44:0/1761655036 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.401+0000 7f347dd9b700 1 --2- 172.21.15.44:0/1761655036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e97160cc0 0x555e9715b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.401+0000 7f347e59c700 1 -- 172.21.15.44:0/1761655036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e97160cc0 msgr2=0x555e9715b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.401+0000 7f347e59c700 1 --2- 172.21.15.44:0/1761655036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e97160cc0 0x555e9715b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.401+0000 7f347e59c700 1 -- 172.21.15.44:0/1761655036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e971661a0 msgr2=0x555e9715b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.401+0000 7f347e59c700 1 --2- 172.21.15.44:0/1761655036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e971661a0 0x555e9715b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.401+0000 7f347e59c700 1 -- 172.21.15.44:0/1761655036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e96f051f0 con 0x555e97160300 2023-09-10T00:48:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.401+0000 7f347ed9d700 1 --2- 172.21.15.44:0/1761655036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e971661a0 0x555e9715b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.401+0000 7f347e59c700 1 --2- 172.21.15.44:0/1761655036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e971714a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3470099690 tx=0x7f3470095a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90a831d4e220c67f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.401+0000 7f346d7fa700 1 -- 172.21.15.44:0/1761655036 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34700a1770 con 0x555e97160300 2023-09-10T00:48:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.401+0000 7f346d7fa700 1 -- 172.21.15.44:0/1761655036 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3470098bf0 con 0x555e97160300 2023-09-10T00:48:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.401+0000 7f346d7fa700 1 -- 172.21.15.44:0/1761655036 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34700a1540 con 0x555e97160300 2023-09-10T00:48:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.402+0000 7f3490ea3b80 1 -- 172.21.15.44:0/1761655036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 msgr2=0x555e971714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.402+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/1761655036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e971714a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3470099690 tx=0x7f3470095a40 comp rx=0 tx=0).stop 2023-09-10T00:48:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.402+0000 7f3490ea3b80 1 -- 172.21.15.44:0/1761655036 shutdown_connections 2023-09-10T00:48:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.402+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/1761655036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e971661a0 0x555e9715b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.402+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/1761655036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e971714a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.402+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/1761655036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e97160cc0 0x555e9715b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.402+0000 7f3490ea3b80 1 -- 172.21.15.44:0/1761655036 >> 172.21.15.44:0/1761655036 conn(0x555e970d5030 msgr2=0x555e971651e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.402+0000 7f3490ea3b80 1 -- 172.21.15.44:0/1761655036 shutdown_connections 2023-09-10T00:48:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.402+0000 7f3490ea3b80 1 -- 172.21.15.44:0/1761655036 wait complete. 2023-09-10T00:48:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.402+0000 7f3490ea3b80 1 -- start start 2023-09-10T00:48:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f3490ea3b80 1 Processor -- start 2023-09-10T00:48:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f3490ea3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e971c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f3490ea3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e97160cc0 0x555e971c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f3490ea3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e971661a0 0x555e971cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f3490ea3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e971c3070 con 0x555e97160cc0 2023-09-10T00:48:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f3490ea3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e971cd7e0 con 0x555e971661a0 2023-09-10T00:48:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f3490ea3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e971cd920 con 0x555e97160300 2023-09-10T00:48:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f3490ea3b80 10 client.0 _ll_register_callbacks cb 0x7f3478005a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:48:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f347e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e971c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f347dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e97160cc0 0x555e971c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f347e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e971c2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58618/0 (socket says 172.21.15.44:58618) 2023-09-10T00:48:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f347ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e971661a0 0x555e971cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f347e59c700 1 -- 172.21.15.44:0/3306375200 learned_addr learned my addr 172.21.15.44:0/3306375200 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f347e59c700 1 -- 172.21.15.44:0/3306375200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e971661a0 msgr2=0x555e971cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f347e59c700 1 --2- 172.21.15.44:0/3306375200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e971661a0 0x555e971cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f347e59c700 1 -- 172.21.15.44:0/3306375200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e97160cc0 msgr2=0x555e971c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f347e59c700 1 --2- 172.21.15.44:0/3306375200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e97160cc0 0x555e971c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f347e59c700 1 -- 172.21.15.44:0/3306375200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e96ec7c00 con 0x555e97160300 2023-09-10T00:48:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.403+0000 7f347dd9b700 1 --2- 172.21.15.44:0/3306375200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e97160cc0 0x555e971c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.404+0000 7f347e59c700 1 --2- 172.21.15.44:0/3306375200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e971c2220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f34700998c0 tx=0x7f3470099780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3698d82cfece3f4f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.404+0000 7f3465ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:48:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.404+0000 7f3465ffb700 1 -- 172.21.15.44:0/3306375200 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470099b60 con 0x555e97160300 2023-09-10T00:48:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.404+0000 7f3490ea3b80 1 -- 172.21.15.44:0/3306375200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f347009d530 con 0x555e97160300 2023-09-10T00:48:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.404+0000 7f3465ffb700 1 -- 172.21.15.44:0/3306375200 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f34700cc070 con 0x555e97160300 2023-09-10T00:48:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.404+0000 7f3465ffb700 1 -- 172.21.15.44:0/3306375200 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3470097b60 con 0x555e97160300 2023-09-10T00:48:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.404+0000 7f3490ea3b80 10 client.26650 fetch_fsmap learned FSMap version 50 2023-09-10T00:48:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.404+0000 7f3490ea3b80 1 -- 172.21.15.44:0/3306375200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x555e96eded80 con 0x555e97160300 2023-09-10T00:48:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.404+0000 7f3465ffb700 1 -- 172.21.15.44:0/3306375200 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f347009d530 con 0x555e97160300 2023-09-10T00:48:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.404+0000 7f3465ffb700 1 -- 172.21.15.44:0/3306375200 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 50) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f34700840b0 con 0x555e97160300 2023-09-10T00:48:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.404+0000 7f3490ea3b80 10 client.26650 fetch_fsmap finished waiting for FSMap version 50 2023-09-10T00:48:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.404+0000 7f3490ea3b80 10 client.26650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:48:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.404+0000 7f3490ea3b80 20 client.26650 populate_metadata read hostname 'smithi044' 2023-09-10T00:48:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.405+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/3306375200 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555e971ceb40 0x555e971d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.405+0000 7f3490ea3b80 4 client.26650 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:48:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.405+0000 7f3490ea3b80 1 -- 172.21.15.44:0/3306375200 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555e970ab8c0 con 0x555e971ceb40 2023-09-10T00:48:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.405+0000 7f347dd9b700 1 --2- 172.21.15.44:0/3306375200 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555e971ceb40 0x555e971d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.405+0000 7f3465ffb700 10 client.26650 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.405+0000 7f347dd9b700 1 --2- 172.21.15.44:0/3306375200 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555e971ceb40 0x555e971d0f60 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3669fc25f6d7aef server_cookie=f93e8e020ec3e460 in_seq=0 out_seq=0 2023-09-10T00:48:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.417+0000 7f3465ffb700 1 -- 172.21.15.44:0/3306375200 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555e970ab8c0 con 0x555e971ceb40 2023-09-10T00:48:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.417+0000 7f3465ffb700 10 client.26650 handle_command_reply: tid=0 2023-09-10T00:48:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.417+0000 7f3490ea3b80 1 client.26650 shutdown 2023-09-10T00:48:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.417+0000 7f3490ea3b80 1 -- 172.21.15.44:0/3306375200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 msgr2=0x555e971c2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.417+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/3306375200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e971c2220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f34700998c0 tx=0x7f3470099780 comp rx=0 tx=0).stop 2023-09-10T00:48:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.417+0000 7f3490ea3b80 1 -- 172.21.15.44:0/3306375200 shutdown_connections 2023-09-10T00:48:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.417+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/3306375200 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555e971ceb40 0x555e971d0f60 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.417+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/3306375200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e971661a0 0x555e971cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.417+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/3306375200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e97160300 0x555e971c2220 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.417+0000 7f3490ea3b80 1 --2- 172.21.15.44:0/3306375200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e97160cc0 0x555e971c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.417+0000 7f3490ea3b80 1 -- 172.21.15.44:0/3306375200 >> 172.21.15.44:0/3306375200 conn(0x555e970d5030 msgr2=0x555e971651e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.417+0000 7f3465ffb700 0 client.26650 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.418+0000 7f3490ea3b80 1 -- 172.21.15.44:0/3306375200 shutdown_connections 2023-09-10T00:48:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.418+0000 7f3490ea3b80 1 -- 172.21.15.44:0/3306375200 wait complete. 2023-09-10T00:48:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.418+0000 7f3490ea3b80 20 client.26650 trim_cache size 0 max 16384 2023-09-10T00:48:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:48:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:48:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:48:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:48:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.418+0000 7f3484cdd700 1 -- 172.21.15.44:0/3728328198 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f34680a07b0 msgr2=0x7f34680a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.418+0000 7f3484cdd700 1 --2- 172.21.15.44:0/3728328198 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f34680a07b0 0x7f34680a2c70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3480197990 tx=0x7f3474009260 comp rx=0 tx=0).stop 2023-09-10T00:48:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.418+0000 7f3484cdd700 1 -- 172.21.15.44:0/3728328198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34800ff760 msgr2=0x7f348019c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.418+0000 7f3484cdd700 1 --2- 172.21.15.44:0/3728328198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34800ff760 0x7f348019c620 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f347000e9a0 tx=0x7f347000ecb0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.418+0000 7f3484cdd700 1 -- 172.21.15.44:0/3728328198 shutdown_connections 2023-09-10T00:48:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.418+0000 7f3484cdd700 1 --2- 172.21.15.44:0/3728328198 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f34680a07b0 0x7f34680a2c70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.418+0000 7f3484cdd700 1 --2- 172.21.15.44:0/3728328198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34801055b0 0x7f3480196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.418+0000 7f3484cdd700 1 --2- 172.21.15.44:0/3728328198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34800ff760 0x7f348019c620 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.418+0000 7f3484cdd700 1 --2- 172.21.15.44:0/3728328198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3480100120 0x7f348019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.418+0000 7f3484cdd700 1 -- 172.21.15.44:0/3728328198 >> 172.21.15.44:0/3728328198 conn(0x7f3480074f30 msgr2=0x7f34800fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.418+0000 7f3484cdd700 1 -- 172.21.15.44:0/3728328198 shutdown_connections 2023-09-10T00:48:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:23.418+0000 7f3484cdd700 1 -- 172.21.15.44:0/3728328198 wait complete. 2023-09-10T00:48:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:24 smithi131 ceph-mon[103713]: pgmap v239: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.8 MiB/s wr, 310 op/s 2023-09-10T00:48:25.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:24 smithi044 ceph-mon[98676]: pgmap v239: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.8 MiB/s wr, 310 op/s 2023-09-10T00:48:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:24 smithi099 ceph-mon[102692]: pgmap v239: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.8 MiB/s wr, 310 op/s 2023-09-10T00:48:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:26 smithi131 ceph-mon[103713]: pgmap v240: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 4.2 MiB/s wr, 469 op/s 2023-09-10T00:48:27.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:26 smithi044 ceph-mon[98676]: pgmap v240: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 4.2 MiB/s wr, 469 op/s 2023-09-10T00:48:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:26 smithi099 ceph-mon[102692]: pgmap v240: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 4.2 MiB/s wr, 469 op/s 2023-09-10T00:48:28.446 INFO:teuthology.orchestra.run.smithi044.stderr:31700000 of 50000000 tuples (63%) done (elapsed 448.08 s, remaining 258.67 s) 2023-09-10T00:48:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:28 smithi131 ceph-mon[103713]: pgmap v241: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 2.8 MiB/s wr, 311 op/s 2023-09-10T00:48:29.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:28 smithi044 ceph-mon[98676]: pgmap v241: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 2.8 MiB/s wr, 311 op/s 2023-09-10T00:48:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:28 smithi099 ceph-mon[102692]: pgmap v241: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 2.8 MiB/s wr, 311 op/s 2023-09-10T00:48:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:30 smithi131 ceph-mon[103713]: pgmap v242: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 2.8 MiB/s wr, 311 op/s 2023-09-10T00:48:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:30 smithi044 ceph-mon[98676]: pgmap v242: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 2.8 MiB/s wr, 311 op/s 2023-09-10T00:48:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:30 smithi099 ceph-mon[102692]: pgmap v242: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 2.8 MiB/s wr, 311 op/s 2023-09-10T00:48:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:32 smithi131 ceph-mon[103713]: pgmap v243: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 4.0 MiB/s wr, 451 op/s 2023-09-10T00:48:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:32 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:48:33.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:32 smithi044 ceph-mon[98676]: pgmap v243: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 4.0 MiB/s wr, 451 op/s 2023-09-10T00:48:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:32 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:48:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:32 smithi099 ceph-mon[102692]: pgmap v243: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 4.0 MiB/s wr, 451 op/s 2023-09-10T00:48:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:32 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:48:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:34 smithi131 ceph-mon[103713]: pgmap v244: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.7 MiB/s wr, 298 op/s 2023-09-10T00:48:35.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:34 smithi044 ceph-mon[98676]: pgmap v244: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.7 MiB/s wr, 298 op/s 2023-09-10T00:48:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:34 smithi099 ceph-mon[102692]: pgmap v244: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.7 MiB/s wr, 298 op/s 2023-09-10T00:48:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.538+0000 7f4e4269f700 1 -- 172.21.15.44:0/4203963491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e3c1055b0 msgr2=0x7f4e3c105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.538+0000 7f4e4269f700 1 --2- 172.21.15.44:0/4203963491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e3c1055b0 0x7f4e3c105990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009920 tx=0x7f4e2c009c30 comp rx=0 tx=0).stop 2023-09-10T00:48:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.538+0000 7f4e4269f700 1 -- 172.21.15.44:0/4203963491 shutdown_connections 2023-09-10T00:48:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.538+0000 7f4e4269f700 1 --2- 172.21.15.44:0/4203963491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e3c100120 0x7f4e3c10c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.538+0000 7f4e4269f700 1 --2- 172.21.15.44:0/4203963491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e3c1055b0 0x7f4e3c105990 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.538+0000 7f4e4269f700 1 --2- 172.21.15.44:0/4203963491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c0ff760 0x7f4e3c0ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.538+0000 7f4e4269f700 1 -- 172.21.15.44:0/4203963491 >> 172.21.15.44:0/4203963491 conn(0x7f4e3c074f30 msgr2=0x7f4e3c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.538+0000 7f4e4269f700 1 -- 172.21.15.44:0/4203963491 shutdown_connections 2023-09-10T00:48:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.538+0000 7f4e4269f700 1 -- 172.21.15.44:0/4203963491 wait complete. 2023-09-10T00:48:36.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.538+0000 7f4e4269f700 1 Processor -- start 2023-09-10T00:48:36.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.539+0000 7f4e4269f700 1 -- start start 2023-09-10T00:48:36.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.539+0000 7f4e4269f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e3c0ff760 0x7f4e3c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.539+0000 7f4e4269f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e3c100120 0x7f4e3c198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.539+0000 7f4e4269f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c1055b0 0x7f4e3c19d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.539+0000 7f4e4269f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c19d930 con 0x7f4e3c100120 2023-09-10T00:48:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.539+0000 7f4e4269f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c19daa0 con 0x7f4e3c1055b0 2023-09-10T00:48:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.539+0000 7f4e4269f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c19dc60 con 0x7f4e3c0ff760 2023-09-10T00:48:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.539+0000 7f4e3b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e3c100120 0x7f4e3c198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.539+0000 7f4e3bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e3c0ff760 0x7f4e3c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.539+0000 7f4e3b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e3c100120 0x7f4e3c198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51752/0 (socket says 172.21.15.44:51752) 2023-09-10T00:48:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.539+0000 7f4e3b7fe700 1 -- 172.21.15.44:0/3965104580 learned_addr learned my addr 172.21.15.44:0/3965104580 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.539+0000 7f4e40c3c700 1 --2- 172.21.15.44:0/3965104580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c1055b0 0x7f4e3c19d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.539+0000 7f4e3bfff700 1 -- 172.21.15.44:0/3965104580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c1055b0 msgr2=0x7f4e3c19d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.539+0000 7f4e3bfff700 1 --2- 172.21.15.44:0/3965104580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c1055b0 0x7f4e3c19d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.539+0000 7f4e3bfff700 1 -- 172.21.15.44:0/3965104580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e3c100120 msgr2=0x7f4e3c198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.539+0000 7f4e3bfff700 1 --2- 172.21.15.44:0/3965104580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e3c100120 0x7f4e3c198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.540+0000 7f4e3bfff700 1 -- 172.21.15.44:0/3965104580 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2c009520 con 0x7f4e3c0ff760 2023-09-10T00:48:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.540+0000 7f4e3b7fe700 1 --2- 172.21.15.44:0/3965104580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e3c100120 0x7f4e3c198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.540+0000 7f4e3bfff700 1 --2- 172.21.15.44:0/3965104580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e3c0ff760 0x7f4e3c198780 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009230 tx=0x7f4e2c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.540+0000 7f4e397fa700 1 -- 172.21.15.44:0/3965104580 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c01c070 con 0x7f4e3c0ff760 2023-09-10T00:48:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.540+0000 7f4e4269f700 1 -- 172.21.15.44:0/3965104580 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3c19dee0 con 0x7f4e3c0ff760 2023-09-10T00:48:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.540+0000 7f4e397fa700 1 -- 172.21.15.44:0/3965104580 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e2c00bab0 con 0x7f4e3c0ff760 2023-09-10T00:48:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.540+0000 7f4e397fa700 1 -- 172.21.15.44:0/3965104580 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c0036e0 con 0x7f4e3c0ff760 2023-09-10T00:48:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.540+0000 7f4e4269f700 1 -- 172.21.15.44:0/3965104580 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3c1a57f0 con 0x7f4e3c0ff760 2023-09-10T00:48:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.543+0000 7f4e397fa700 1 -- 172.21.15.44:0/3965104580 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f4e2c020030 con 0x7f4e3c0ff760 2023-09-10T00:48:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.544+0000 7f4e397fa700 1 --2- 172.21.15.44:0/3965104580 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e240a07b0 0x7f4e240a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.544+0000 7f4e3b7fe700 1 --2- 172.21.15.44:0/3965104580 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e240a07b0 0x7f4e240a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.544+0000 7f4e397fa700 1 -- 172.21.15.44:0/3965104580 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f4e2c0cd200 con 0x7f4e3c0ff760 2023-09-10T00:48:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.544+0000 7f4e3b7fe700 1 --2- 172.21.15.44:0/3965104580 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e240a07b0 0x7f4e240a2c70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c199fa0 tx=0x7f4e30009310 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e4e865b80 1 Processor -- start 2023-09-10T00:48:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e4e865b80 1 -- start start 2023-09-10T00:48:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e4e865b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af471a0 0x55e93af47580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e4e865b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af41300 0x55e93af41780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e4e865b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41cc0 0x55e93af4df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e4e865b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93ae9a6d0 con 0x55e93af41300 2023-09-10T00:48:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e4e865b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93ae97ff0 con 0x55e93af471a0 2023-09-10T00:48:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e4e865b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93ae938e0 con 0x55e93af41cc0 2023-09-10T00:48:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e3b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af41300 0x55e93af41780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e3b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af41300 0x55e93af41780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51758/0 (socket says 172.21.15.44:51758) 2023-09-10T00:48:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e3b7fe700 1 -- 172.21.15.44:0/3195468645 learned_addr learned my addr 172.21.15.44:0/3195468645 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e3bfff700 1 --2- 172.21.15.44:0/3195468645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af471a0 0x55e93af47580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e40c3c700 1 --2- 172.21.15.44:0/3195468645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41cc0 0x55e93af4df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e3b7fe700 1 -- 172.21.15.44:0/3195468645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41cc0 msgr2=0x55e93af4df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e3b7fe700 1 --2- 172.21.15.44:0/3195468645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41cc0 0x55e93af4df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e3b7fe700 1 -- 172.21.15.44:0/3195468645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af471a0 msgr2=0x55e93af47580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e3b7fe700 1 --2- 172.21.15.44:0/3195468645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af471a0 0x55e93af47580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e3b7fe700 1 -- 172.21.15.44:0/3195468645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e93acbfd80 con 0x55e93af41300 2023-09-10T00:48:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e3bfff700 1 --2- 172.21.15.44:0/3195468645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af471a0 0x55e93af47580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.545+0000 7f4e40c3c700 1 --2- 172.21.15.44:0/3195468645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41cc0 0x55e93af4df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:48:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.546+0000 7f4e3b7fe700 1 --2- 172.21.15.44:0/3195468645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af41300 0x55e93af41780 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000e6b0 tx=0x7f4e30014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b56eb5b61cab08 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.546+0000 7f4e227fc700 1 -- 172.21.15.44:0/3195468645 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e30009df0 con 0x55e93af41300 2023-09-10T00:48:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.546+0000 7f4e227fc700 1 -- 172.21.15.44:0/3195468645 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e30021070 con 0x55e93af41300 2023-09-10T00:48:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.546+0000 7f4e227fc700 1 -- 172.21.15.44:0/3195468645 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3001d720 con 0x55e93af41300 2023-09-10T00:48:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.547+0000 7f4e4e865b80 1 -- 172.21.15.44:0/3195468645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af41300 msgr2=0x55e93af41780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.547+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/3195468645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af41300 0x55e93af41780 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000e6b0 tx=0x7f4e30014000 comp rx=0 tx=0).stop 2023-09-10T00:48:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.547+0000 7f4e4e865b80 1 -- 172.21.15.44:0/3195468645 shutdown_connections 2023-09-10T00:48:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.547+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/3195468645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41cc0 0x55e93af4df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.547+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/3195468645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af471a0 0x55e93af47580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.547+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/3195468645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af41300 0x55e93af41780 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.547+0000 7f4e4e865b80 1 -- 172.21.15.44:0/3195468645 >> 172.21.15.44:0/3195468645 conn(0x55e93aeb6030 msgr2=0x55e93af3c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.547+0000 7f4e4e865b80 1 -- 172.21.15.44:0/3195468645 shutdown_connections 2023-09-10T00:48:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.547+0000 7f4e4e865b80 1 -- 172.21.15.44:0/3195468645 wait complete. 2023-09-10T00:48:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.547+0000 7f4e4e865b80 1 -- start start 2023-09-10T00:48:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.547+0000 7f4e4e865b80 1 Processor -- start 2023-09-10T00:48:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e4e865b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41300 0x55e93af4dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e4e865b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af41cc0 0x55e93af4b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e4e865b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af471a0 0x55e93af4b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e4e865b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93ae31b10 con 0x55e93af41cc0 2023-09-10T00:48:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e4e865b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93ae31810 con 0x55e93af41300 2023-09-10T00:48:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e4e865b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93ad158c0 con 0x55e93af471a0 2023-09-10T00:48:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e4e865b80 10 client.0 _ll_register_callbacks cb 0x55e93ad7ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:48:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e3b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af41cc0 0x55e93af4b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e3b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af41cc0 0x55e93af4b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51766/0 (socket says 172.21.15.44:51766) 2023-09-10T00:48:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e3b7fe700 1 -- 172.21.15.44:0/100781342 learned_addr learned my addr 172.21.15.44:0/100781342 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e3bfff700 1 --2- 172.21.15.44:0/100781342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41300 0x55e93af4dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e40c3c700 1 --2- 172.21.15.44:0/100781342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af471a0 0x55e93af4b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e3b7fe700 1 -- 172.21.15.44:0/100781342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af471a0 msgr2=0x55e93af4b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e3b7fe700 1 --2- 172.21.15.44:0/100781342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af471a0 0x55e93af4b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e3b7fe700 1 -- 172.21.15.44:0/100781342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41300 msgr2=0x55e93af4dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e3b7fe700 1 --2- 172.21.15.44:0/100781342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41300 0x55e93af4dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e3b7fe700 1 -- 172.21.15.44:0/100781342 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3c1a57f0 con 0x55e93af41cc0 2023-09-10T00:48:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e40c3c700 1 --2- 172.21.15.44:0/100781342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af471a0 0x55e93af4b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:48:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e3bfff700 1 --2- 172.21.15.44:0/100781342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41300 0x55e93af4dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e3b7fe700 1 --2- 172.21.15.44:0/100781342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af41cc0 0x55e93af4b0c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4e3001cc20 tx=0x7f4e30009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575861f279acc7ca server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.548+0000 7f4e1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:48:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.549+0000 7f4e1b7fe700 1 -- 172.21.15.44:0/100781342 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3001c6e0 con 0x55e93af41cc0 2023-09-10T00:48:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.549+0000 7f4e4e865b80 1 -- 172.21.15.44:0/100781342 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e93adeb490 con 0x55e93af41cc0 2023-09-10T00:48:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.549+0000 7f4e1b7fe700 1 -- 172.21.15.44:0/100781342 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f4e3001c840 con 0x55e93af41cc0 2023-09-10T00:48:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.549+0000 7f4e1b7fe700 1 -- 172.21.15.44:0/100781342 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e3001ca00 con 0x55e93af41cc0 2023-09-10T00:48:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.549+0000 7f4e4e865b80 10 client.17061 fetch_fsmap learned FSMap version 51 2023-09-10T00:48:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.549+0000 7f4e4e865b80 1 -- 172.21.15.44:0/100781342 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f4e30017070 con 0x55e93af41cc0 2023-09-10T00:48:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.549+0000 7f4e1b7fe700 1 -- 172.21.15.44:0/100781342 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e93adeb490 con 0x55e93af41cc0 2023-09-10T00:48:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.549+0000 7f4e1b7fe700 1 -- 172.21.15.44:0/100781342 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 51) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f4e30004af0 con 0x55e93af41cc0 2023-09-10T00:48:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.549+0000 7f4e4e865b80 10 client.17061 fetch_fsmap finished waiting for FSMap version 51 2023-09-10T00:48:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.549+0000 7f4e4e865b80 10 client.17061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:48:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.549+0000 7f4e4e865b80 20 client.17061 populate_metadata read hostname 'smithi044' 2023-09-10T00:48:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.549+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/100781342 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e93af5b3e0 0x55e93af5d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.549+0000 7f4e4e865b80 4 client.17061 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:48:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.549+0000 7f4e4e865b80 1 -- 172.21.15.44:0/100781342 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e93aea8840 con 0x55e93af5b3e0 2023-09-10T00:48:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.550+0000 7f4e3bfff700 1 --2- 172.21.15.44:0/100781342 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e93af5b3e0 0x55e93af5d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.550+0000 7f4e1b7fe700 10 client.17061 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.550+0000 7f4e3bfff700 1 --2- 172.21.15.44:0/100781342 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e93af5b3e0 0x55e93af5d800 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69a527b98d131cbb server_cookie=a12480df42820ff1 in_seq=0 out_seq=0 2023-09-10T00:48:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.551+0000 7f4e1b7fe700 1 -- 172.21.15.44:0/100781342 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e93aea8840 con 0x55e93af5b3e0 2023-09-10T00:48:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.551+0000 7f4e1b7fe700 10 client.17061 handle_command_reply: tid=0 2023-09-10T00:48:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.551+0000 7f4e4e865b80 1 client.17061 shutdown 2023-09-10T00:48:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.551+0000 7f4e4e865b80 1 -- 172.21.15.44:0/100781342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af41cc0 msgr2=0x55e93af4b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:36.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.551+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/100781342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af41cc0 0x55e93af4b0c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4e3001cc20 tx=0x7f4e30009ed0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.551+0000 7f4e4e865b80 1 -- 172.21.15.44:0/100781342 shutdown_connections 2023-09-10T00:48:36.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.551+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/100781342 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e93af5b3e0 0x55e93af5d800 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.551+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/100781342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af471a0 0x55e93af4b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.551+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/100781342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41300 0x55e93af4dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.551+0000 7f4e1b7fe700 0 client.17061 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.551+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/100781342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af41cc0 0x55e93af4b0c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.551+0000 7f4e4e865b80 1 -- 172.21.15.44:0/100781342 >> 172.21.15.44:0/100781342 conn(0x55e93aeb6030 msgr2=0x55e93af3b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.551+0000 7f4e4e865b80 1 -- 172.21.15.44:0/100781342 shutdown_connections 2023-09-10T00:48:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.551+0000 7f4e4e865b80 1 -- 172.21.15.44:0/100781342 wait complete. 2023-09-10T00:48:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.551+0000 7f4e4e865b80 20 client.17061 trim_cache size 0 max 16384 2023-09-10T00:48:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e4e865b80 1 Processor -- start 2023-09-10T00:48:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e4e865b80 1 -- start start 2023-09-10T00:48:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e4e865b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41300 0x55e93af52530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e4e865b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41cc0 0x55e93af4d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e4e865b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af471a0 0x55e93af45230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e4e865b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93aea8840 con 0x55e93af471a0 2023-09-10T00:48:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e4e865b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93adeb490 con 0x55e93af41300 2023-09-10T00:48:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e4e865b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93af57db0 con 0x55e93af41cc0 2023-09-10T00:48:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e40c3c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af471a0 0x55e93af45230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e3bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41300 0x55e93af52530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e3b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41cc0 0x55e93af4d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e3bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41300 0x55e93af52530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53512/0 (socket says 172.21.15.44:53512) 2023-09-10T00:48:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e3b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41cc0 0x55e93af4d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58654/0 (socket says 172.21.15.44:58654) 2023-09-10T00:48:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e3bfff700 1 -- 172.21.15.44:0/1224685543 learned_addr learned my addr 172.21.15.44:0/1224685543 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e40c3c700 1 -- 172.21.15.44:0/1224685543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41cc0 msgr2=0x55e93af4d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e40c3c700 1 --2- 172.21.15.44:0/1224685543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41cc0 0x55e93af4d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e40c3c700 1 -- 172.21.15.44:0/1224685543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41300 msgr2=0x55e93af52530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e40c3c700 1 --2- 172.21.15.44:0/1224685543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41300 0x55e93af52530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e40c3c700 1 -- 172.21.15.44:0/1224685543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e93aca8c00 con 0x55e93af471a0 2023-09-10T00:48:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.568+0000 7f4e3b7fe700 1 --2- 172.21.15.44:0/1224685543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41cc0 0x55e93af4d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.569+0000 7f4e40c3c700 1 --2- 172.21.15.44:0/1224685543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af471a0 0x55e93af45230 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4e340094b0 tx=0x7f4e3400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b29041716f4fbdf server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.569+0000 7f4e22ffd700 1 -- 172.21.15.44:0/1224685543 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3400e040 con 0x55e93af471a0 2023-09-10T00:48:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.569+0000 7f4e22ffd700 1 -- 172.21.15.44:0/1224685543 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e3400f040 con 0x55e93af471a0 2023-09-10T00:48:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.569+0000 7f4e22ffd700 1 -- 172.21.15.44:0/1224685543 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34017670 con 0x55e93af471a0 2023-09-10T00:48:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.569+0000 7f4e4e865b80 1 -- 172.21.15.44:0/1224685543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af471a0 msgr2=0x55e93af45230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.569+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/1224685543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af471a0 0x55e93af45230 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4e340094b0 tx=0x7f4e3400b040 comp rx=0 tx=0).stop 2023-09-10T00:48:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.570+0000 7f4e4e865b80 1 -- 172.21.15.44:0/1224685543 shutdown_connections 2023-09-10T00:48:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.570+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/1224685543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af471a0 0x55e93af45230 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.570+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/1224685543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41300 0x55e93af52530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.570+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/1224685543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41cc0 0x55e93af4d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.570+0000 7f4e4e865b80 1 -- 172.21.15.44:0/1224685543 >> 172.21.15.44:0/1224685543 conn(0x55e93aeb6030 msgr2=0x55e93af576a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.570+0000 7f4e4e865b80 1 -- 172.21.15.44:0/1224685543 shutdown_connections 2023-09-10T00:48:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.570+0000 7f4e4e865b80 1 -- 172.21.15.44:0/1224685543 wait complete. 2023-09-10T00:48:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.570+0000 7f4e4e865b80 1 -- start start 2023-09-10T00:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.570+0000 7f4e4e865b80 1 Processor -- start 2023-09-10T00:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e4e865b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41300 0x55e93afa3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e4e865b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41cc0 0x55e93afa3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e4e865b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af471a0 0x55e93afae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e4e865b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93afae720 con 0x55e93af471a0 2023-09-10T00:48:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e4e865b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93afae860 con 0x55e93af41cc0 2023-09-10T00:48:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e4e865b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93afae9a0 con 0x55e93af41300 2023-09-10T00:48:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e4e865b80 10 client.0 _ll_register_callbacks cb 0x55e93ad7ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:48:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e40c3c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af471a0 0x55e93afae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e3bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41300 0x55e93afa3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e3bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41300 0x55e93afa3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58660/0 (socket says 172.21.15.44:58660) 2023-09-10T00:48:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e3bfff700 1 -- 172.21.15.44:0/3854526087 learned_addr learned my addr 172.21.15.44:0/3854526087 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e3b7fe700 1 --2- 172.21.15.44:0/3854526087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41cc0 0x55e93afa3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e3bfff700 1 -- 172.21.15.44:0/3854526087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41cc0 msgr2=0x55e93afa3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e3bfff700 1 --2- 172.21.15.44:0/3854526087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41cc0 0x55e93afa3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e3bfff700 1 -- 172.21.15.44:0/3854526087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af471a0 msgr2=0x55e93afae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e3bfff700 1 --2- 172.21.15.44:0/3854526087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af471a0 0x55e93afae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e3bfff700 1 -- 172.21.15.44:0/3854526087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e30017070 con 0x55e93af41300 2023-09-10T00:48:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e3b7fe700 1 --2- 172.21.15.44:0/3854526087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41cc0 0x55e93afa3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:48:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e3bfff700 1 --2- 172.21.15.44:0/3854526087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41300 0x55e93afa3350 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0967e0 tx=0x7f4e2c0a0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=585317280cdf8ac2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.571+0000 7f4e1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:48:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.572+0000 7f4e1b7fe700 1 -- 172.21.15.44:0/3854526087 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c09e110 con 0x55e93af41300 2023-09-10T00:48:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.572+0000 7f4e4e865b80 1 -- 172.21.15.44:0/3854526087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e30021070 con 0x55e93af41300 2023-09-10T00:48:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.572+0000 7f4e1b7fe700 1 -- 172.21.15.44:0/3854526087 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f4e2c0a22c0 con 0x55e93af41300 2023-09-10T00:48:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.572+0000 7f4e1b7fe700 1 -- 172.21.15.44:0/3854526087 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e2c0b8250 con 0x55e93af41300 2023-09-10T00:48:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.572+0000 7f4e4e865b80 10 client.26668 fetch_fsmap learned FSMap version 51 2023-09-10T00:48:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.572+0000 7f4e4e865b80 1 -- 172.21.15.44:0/3854526087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f4e34009190 con 0x55e93af41300 2023-09-10T00:48:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.572+0000 7f4e1b7fe700 1 -- 172.21.15.44:0/3854526087 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e30021070 con 0x55e93af41300 2023-09-10T00:48:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.572+0000 7f4e1b7fe700 1 -- 172.21.15.44:0/3854526087 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 51) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f4e2c0ca7e0 con 0x55e93af41300 2023-09-10T00:48:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.572+0000 7f4e4e865b80 10 client.26668 fetch_fsmap finished waiting for FSMap version 51 2023-09-10T00:48:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.572+0000 7f4e4e865b80 10 client.26668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:48:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.572+0000 7f4e4e865b80 20 client.26668 populate_metadata read hostname 'smithi044' 2023-09-10T00:48:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.572+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/3854526087 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e93afafd70 0x55e93afb2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.572+0000 7f4e4e865b80 4 client.26668 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:48:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.572+0000 7f4e4e865b80 1 -- 172.21.15.44:0/3854526087 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e93ae8c8c0 con 0x55e93afafd70 2023-09-10T00:48:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.572+0000 7f4e3b7fe700 1 --2- 172.21.15.44:0/3854526087 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e93afafd70 0x55e93afb2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.573+0000 7f4e1b7fe700 10 client.26668 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.573+0000 7f4e3b7fe700 1 --2- 172.21.15.44:0/3854526087 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e93afafd70 0x55e93afb2190 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8324f5d4527cdd58 server_cookie=a72715c9d1752164 in_seq=0 out_seq=0 2023-09-10T00:48:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.584+0000 7f4e1b7fe700 1 -- 172.21.15.44:0/3854526087 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e93ae8c8c0 con 0x55e93afafd70 2023-09-10T00:48:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.584+0000 7f4e1b7fe700 10 client.26668 handle_command_reply: tid=0 2023-09-10T00:48:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.584+0000 7f4e4e865b80 1 client.26668 shutdown 2023-09-10T00:48:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.584+0000 7f4e4e865b80 1 -- 172.21.15.44:0/3854526087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41300 msgr2=0x55e93afa3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.584+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/3854526087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41300 0x55e93afa3350 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0967e0 tx=0x7f4e2c0a0040 comp rx=0 tx=0).stop 2023-09-10T00:48:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.584+0000 7f4e4e865b80 1 -- 172.21.15.44:0/3854526087 shutdown_connections 2023-09-10T00:48:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.584+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/3854526087 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e93afafd70 0x55e93afb2190 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.584+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/3854526087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e93af471a0 0x55e93afae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.584+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/3854526087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93af41300 0x55e93afa3350 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.584+0000 7f4e1b7fe700 0 client.26668 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.584+0000 7f4e4e865b80 1 --2- 172.21.15.44:0/3854526087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93af41cc0 0x55e93afa3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.584+0000 7f4e4e865b80 1 -- 172.21.15.44:0/3854526087 >> 172.21.15.44:0/3854526087 conn(0x55e93aeb6030 msgr2=0x55e93af461b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.584+0000 7f4e4e865b80 1 -- 172.21.15.44:0/3854526087 shutdown_connections 2023-09-10T00:48:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.584+0000 7f4e4e865b80 1 -- 172.21.15.44:0/3854526087 wait complete. 2023-09-10T00:48:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.584+0000 7f4e4e865b80 20 client.26668 trim_cache size 0 max 16384 2023-09-10T00:48:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:48:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:48:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:48:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:48:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.585+0000 7f4e4269f700 1 -- 172.21.15.44:0/3965104580 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e240a07b0 msgr2=0x7f4e240a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.585+0000 7f4e4269f700 1 --2- 172.21.15.44:0/3965104580 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e240a07b0 0x7f4e240a2c70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c199fa0 tx=0x7f4e30009310 comp rx=0 tx=0).stop 2023-09-10T00:48:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.585+0000 7f4e4269f700 1 -- 172.21.15.44:0/3965104580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e3c0ff760 msgr2=0x7f4e3c198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.585+0000 7f4e4269f700 1 --2- 172.21.15.44:0/3965104580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e3c0ff760 0x7f4e3c198780 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009230 tx=0x7f4e2c00fa00 comp rx=0 tx=0).stop 2023-09-10T00:48:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.585+0000 7f4e4269f700 1 -- 172.21.15.44:0/3965104580 shutdown_connections 2023-09-10T00:48:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.585+0000 7f4e4269f700 1 --2- 172.21.15.44:0/3965104580 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e240a07b0 0x7f4e240a2c70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.585+0000 7f4e4269f700 1 --2- 172.21.15.44:0/3965104580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c1055b0 0x7f4e3c19d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.585+0000 7f4e4269f700 1 --2- 172.21.15.44:0/3965104580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e3c0ff760 0x7f4e3c198780 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.585+0000 7f4e4269f700 1 --2- 172.21.15.44:0/3965104580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e3c100120 0x7f4e3c198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.585+0000 7f4e4269f700 1 -- 172.21.15.44:0/3965104580 >> 172.21.15.44:0/3965104580 conn(0x7f4e3c074f30 msgr2=0x7f4e3c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.585+0000 7f4e4269f700 1 -- 172.21.15.44:0/3965104580 shutdown_connections 2023-09-10T00:48:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:36.585+0000 7f4e4269f700 1 -- 172.21.15.44:0/3965104580 wait complete. 2023-09-10T00:48:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:36 smithi131 ceph-mon[103713]: pgmap v245: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 4.0 MiB/s wr, 451 op/s 2023-09-10T00:48:37.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:36 smithi044 ceph-mon[98676]: pgmap v245: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 4.0 MiB/s wr, 451 op/s 2023-09-10T00:48:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:36 smithi099 ceph-mon[102692]: pgmap v245: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 4.0 MiB/s wr, 451 op/s 2023-09-10T00:48:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:38 smithi131 ceph-mon[103713]: pgmap v246: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 2.6 MiB/s wr, 293 op/s 2023-09-10T00:48:39.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:38 smithi044 ceph-mon[98676]: pgmap v246: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 2.6 MiB/s wr, 293 op/s 2023-09-10T00:48:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:38 smithi099 ceph-mon[102692]: pgmap v246: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 2.6 MiB/s wr, 293 op/s 2023-09-10T00:48:39.324 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:48:39.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.461+0000 7f4368761700 1 -- 172.21.15.44:0/2499617974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f436010abf0 msgr2=0x7f436010d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:39.462 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.461+0000 7f4368761700 1 --2- 172.21.15.44:0/2499617974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f436010abf0 0x7f436010d060 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f435800b0a0 tx=0x7f435800b3b0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.461+0000 7f4368761700 1 -- 172.21.15.44:0/2499617974 shutdown_connections 2023-09-10T00:48:39.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.461+0000 7f4368761700 1 --2- 172.21.15.44:0/2499617974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f436010abf0 0x7f436010d060 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.461+0000 7f4368761700 1 --2- 172.21.15.44:0/2499617974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4360101820 0x7f4360101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.461+0000 7f4368761700 1 --2- 172.21.15.44:0/2499617974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4360102140 0x7f436010a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.461+0000 7f4368761700 1 -- 172.21.15.44:0/2499617974 >> 172.21.15.44:0/2499617974 conn(0x7f43600fafa0 msgr2=0x7f43600fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:39.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.461+0000 7f4368761700 1 -- 172.21.15.44:0/2499617974 shutdown_connections 2023-09-10T00:48:39.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.461+0000 7f4368761700 1 -- 172.21.15.44:0/2499617974 wait complete. 2023-09-10T00:48:39.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.462+0000 7f4368761700 1 Processor -- start 2023-09-10T00:48:39.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.462+0000 7f4368761700 1 -- start start 2023-09-10T00:48:39.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f4368761700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4360101820 0x7f4360196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f4368761700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4360102140 0x7f4360196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f4368761700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f436010abf0 0x7f4360199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f4368761700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436019a5f0 con 0x7f4360101820 2023-09-10T00:48:39.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f4368761700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436019a760 con 0x7f4360102140 2023-09-10T00:48:39.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f4368761700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436019a920 con 0x7f436010abf0 2023-09-10T00:48:39.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f43664fd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4360101820 0x7f4360196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f4365cfc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4360102140 0x7f4360196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f4365cfc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4360102140 0x7f4360196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53534/0 (socket says 172.21.15.44:53534) 2023-09-10T00:48:39.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f4366cfe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f436010abf0 0x7f4360199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f4365cfc700 1 -- 172.21.15.44:0/65050896 learned_addr learned my addr 172.21.15.44:0/65050896 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:39.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f43664fd700 1 -- 172.21.15.44:0/65050896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f436010abf0 msgr2=0x7f4360199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:39.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f43664fd700 1 --2- 172.21.15.44:0/65050896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f436010abf0 0x7f4360199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f43664fd700 1 -- 172.21.15.44:0/65050896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4360102140 msgr2=0x7f4360196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:39.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f43664fd700 1 --2- 172.21.15.44:0/65050896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4360102140 0x7f4360196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f43664fd700 1 -- 172.21.15.44:0/65050896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4358009d00 con 0x7f4360101820 2023-09-10T00:48:39.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.463+0000 7f43664fd700 1 --2- 172.21.15.44:0/65050896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4360101820 0x7f4360196700 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f435c009c50 tx=0x7f435c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:39.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.464+0000 7f43577fe700 1 -- 172.21.15.44:0/65050896 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c015070 con 0x7f4360101820 2023-09-10T00:48:39.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.464+0000 7f43577fe700 1 -- 172.21.15.44:0/65050896 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f435c019410 con 0x7f4360101820 2023-09-10T00:48:39.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.464+0000 7f4368761700 1 -- 172.21.15.44:0/65050896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436019ac00 con 0x7f4360101820 2023-09-10T00:48:39.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.464+0000 7f43577fe700 1 -- 172.21.15.44:0/65050896 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c018610 con 0x7f4360101820 2023-09-10T00:48:39.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.464+0000 7f4368761700 1 -- 172.21.15.44:0/65050896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43600ff580 con 0x7f4360101820 2023-09-10T00:48:39.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.466+0000 7f43577fe700 1 -- 172.21.15.44:0/65050896 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f435c019580 con 0x7f4360101820 2023-09-10T00:48:39.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.467+0000 7f43577fe700 1 --2- 172.21.15.44:0/65050896 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f434c0a07b0 0x7f434c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.467+0000 7f43577fe700 1 -- 172.21.15.44:0/65050896 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f435c0cfa60 con 0x7f4360101820 2023-09-10T00:48:39.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.467+0000 7f4365cfc700 1 --2- 172.21.15.44:0/65050896 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f434c0a07b0 0x7f434c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.468+0000 7f4365cfc700 1 --2- 172.21.15.44:0/65050896 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f434c0a07b0 0x7f434c0a2c70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4360197d10 tx=0x7f4350009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:39.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.468+0000 7f4374927b80 1 Processor -- start 2023-09-10T00:48:39.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.468+0000 7f4374927b80 1 -- start start 2023-09-10T00:48:39.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.468+0000 7f4374927b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18011710 0x55ba18011af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.468+0000 7f4374927b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18012030 0x55ba1801b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.468+0000 7f4374927b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba1801be40 0x55ba1801e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.468+0000 7f4374927b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba17f6a730 con 0x55ba1801be40 2023-09-10T00:48:39.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.468+0000 7f4374927b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba17f68050 con 0x55ba18011710 2023-09-10T00:48:39.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.468+0000 7f4374927b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba17f63940 con 0x55ba18012030 2023-09-10T00:48:39.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.469+0000 7f4366cfe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba1801be40 0x55ba1801e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.469+0000 7f43664fd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18011710 0x55ba18011af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.469+0000 7f43664fd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18011710 0x55ba18011af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53540/0 (socket says 172.21.15.44:53540) 2023-09-10T00:48:39.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.469+0000 7f43664fd700 1 -- 172.21.15.44:0/2067288914 learned_addr learned my addr 172.21.15.44:0/2067288914 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:39.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.469+0000 7f4365cfc700 1 --2- 172.21.15.44:0/2067288914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18012030 0x55ba1801b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.469+0000 7f4366cfe700 1 -- 172.21.15.44:0/2067288914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18012030 msgr2=0x55ba1801b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:39.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.469+0000 7f4366cfe700 1 --2- 172.21.15.44:0/2067288914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18012030 0x55ba1801b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.469+0000 7f4366cfe700 1 -- 172.21.15.44:0/2067288914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18011710 msgr2=0x55ba18011af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:39.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.469+0000 7f4366cfe700 1 --2- 172.21.15.44:0/2067288914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18011710 0x55ba18011af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.469+0000 7f4366cfe700 1 -- 172.21.15.44:0/2067288914 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba17d8fde0 con 0x55ba1801be40 2023-09-10T00:48:39.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.469+0000 7f4366cfe700 1 --2- 172.21.15.44:0/2067288914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba1801be40 0x55ba1801e230 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4358009fd0 tx=0x7f4358009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183899a6f6fd39c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:39.480 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.469+0000 7f4354ff9700 1 -- 172.21.15.44:0/2067288914 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435800e040 con 0x55ba1801be40 2023-09-10T00:48:39.480 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.469+0000 7f4354ff9700 1 -- 172.21.15.44:0/2067288914 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43580094c0 con 0x55ba1801be40 2023-09-10T00:48:39.480 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.469+0000 7f4354ff9700 1 -- 172.21.15.44:0/2067288914 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358003c60 con 0x55ba1801be40 2023-09-10T00:48:39.480 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.470+0000 7f4374927b80 1 -- 172.21.15.44:0/2067288914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba1801be40 msgr2=0x55ba1801e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:39.480 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.470+0000 7f4374927b80 1 --2- 172.21.15.44:0/2067288914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba1801be40 0x55ba1801e230 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4358009fd0 tx=0x7f4358009c40 comp rx=0 tx=0).stop 2023-09-10T00:48:39.481 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.470+0000 7f4374927b80 1 -- 172.21.15.44:0/2067288914 shutdown_connections 2023-09-10T00:48:39.481 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.470+0000 7f4374927b80 1 --2- 172.21.15.44:0/2067288914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba1801be40 0x55ba1801e230 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.481 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.470+0000 7f4374927b80 1 --2- 172.21.15.44:0/2067288914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18011710 0x55ba18011af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.481 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.470+0000 7f4374927b80 1 --2- 172.21.15.44:0/2067288914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18012030 0x55ba1801b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.481 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.470+0000 7f4374927b80 1 -- 172.21.15.44:0/2067288914 >> 172.21.15.44:0/2067288914 conn(0x55ba17f86090 msgr2=0x55ba1800cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:39.481 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.470+0000 7f4374927b80 1 -- 172.21.15.44:0/2067288914 shutdown_connections 2023-09-10T00:48:39.482 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.470+0000 7f4374927b80 1 -- 172.21.15.44:0/2067288914 wait complete. 2023-09-10T00:48:39.482 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.471+0000 7f4374927b80 1 -- start start 2023-09-10T00:48:39.482 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.471+0000 7f4374927b80 1 Processor -- start 2023-09-10T00:48:39.482 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.471+0000 7f4374927b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba18011710 0x55ba1801b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.482 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.471+0000 7f4374927b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18012030 0x55ba18018960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.471+0000 7f4374927b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba1801be40 0x55ba18018ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.471+0000 7f4374927b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba17f01b70 con 0x55ba18011710 2023-09-10T00:48:39.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.471+0000 7f4374927b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba17f01870 con 0x55ba1801be40 2023-09-10T00:48:39.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.471+0000 7f4374927b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba17de5920 con 0x55ba18012030 2023-09-10T00:48:39.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.471+0000 7f4374927b80 10 client.0 _ll_register_callbacks cb 0x55ba17e4ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:48:39.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.471+0000 7f43664fd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba18011710 0x55ba1801b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.471+0000 7f43664fd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba18011710 0x55ba1801b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51806/0 (socket says 172.21.15.44:51806) 2023-09-10T00:48:39.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.471+0000 7f43664fd700 1 -- 172.21.15.44:0/2909230092 learned_addr learned my addr 172.21.15.44:0/2909230092 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:39.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.471+0000 7f4365cfc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18012030 0x55ba18018960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.485 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.471+0000 7f4366cfe700 1 --2- 172.21.15.44:0/2909230092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba1801be40 0x55ba18018ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.485 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.472+0000 7f43664fd700 1 -- 172.21.15.44:0/2909230092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18012030 msgr2=0x55ba18018960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:39.485 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.472+0000 7f43664fd700 1 --2- 172.21.15.44:0/2909230092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18012030 0x55ba18018960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.485 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.472+0000 7f43664fd700 1 -- 172.21.15.44:0/2909230092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba1801be40 msgr2=0x55ba18018ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:39.485 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.472+0000 7f43664fd700 1 --2- 172.21.15.44:0/2909230092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba1801be40 0x55ba18018ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.485 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.472+0000 7f43664fd700 1 -- 172.21.15.44:0/2909230092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436010dd20 con 0x55ba18011710 2023-09-10T00:48:39.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.472+0000 7f4365cfc700 1 --2- 172.21.15.44:0/2909230092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18012030 0x55ba18018960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:39.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.472+0000 7f43664fd700 1 --2- 172.21.15.44:0/2909230092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba18011710 0x55ba1801b6e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f435c0cd3b0 tx=0x7f435c09c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e3ab7e0c92569b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:39.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.472+0000 7f4349ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:48:39.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.472+0000 7f4349ffb700 1 -- 172.21.15.44:0/2909230092 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c09d210 con 0x55ba18011710 2023-09-10T00:48:39.486 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.472+0000 7f4349ffb700 1 -- 172.21.15.44:0/2909230092 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f435c09cb10 con 0x55ba18011710 2023-09-10T00:48:39.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.472+0000 7f4374927b80 1 -- 172.21.15.44:0/2909230092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba17f788a0 con 0x55ba18011710 2023-09-10T00:48:39.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.472+0000 7f4349ffb700 1 -- 172.21.15.44:0/2909230092 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f435c0a0940 con 0x55ba18011710 2023-09-10T00:48:39.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.472+0000 7f4374927b80 10 client.17079 fetch_fsmap learned FSMap version 51 2023-09-10T00:48:39.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.472+0000 7f4374927b80 1 -- 172.21.15.44:0/2909230092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55ba17d78c60 con 0x55ba18011710 2023-09-10T00:48:39.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.472+0000 7f4349ffb700 1 -- 172.21.15.44:0/2909230092 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f435c07f020 con 0x55ba18011710 2023-09-10T00:48:39.487 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.472+0000 7f4349ffb700 1 -- 172.21.15.44:0/2909230092 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 51) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f435c0a46f0 con 0x55ba18011710 2023-09-10T00:48:39.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.473+0000 7f4374927b80 10 client.17079 fetch_fsmap finished waiting for FSMap version 51 2023-09-10T00:48:39.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.473+0000 7f4374927b80 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:48:39.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.473+0000 7f4374927b80 20 client.17079 populate_metadata read hostname 'smithi044' 2023-09-10T00:48:39.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.473+0000 7f4374927b80 1 --2- 172.21.15.44:0/2909230092 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ba1802b880 0x55ba1802dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.473+0000 7f4374927b80 4 client.17079 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:48:39.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.473+0000 7f4374927b80 1 -- 172.21.15.44:0/2909230092 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba17f5c920 con 0x55ba1802b880 2023-09-10T00:48:39.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.473+0000 7f4365cfc700 1 --2- 172.21.15.44:0/2909230092 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ba1802b880 0x55ba1802dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.473+0000 7f4349ffb700 10 client.17079 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:39.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.473+0000 7f4365cfc700 1 --2- 172.21.15.44:0/2909230092 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ba1802b880 0x55ba1802dca0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbfbddc0594afcdf server_cookie=96fb3920b2dcccf4 in_seq=0 out_seq=0 2023-09-10T00:48:39.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.474+0000 7f4349ffb700 1 -- 172.21.15.44:0/2909230092 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ba17f5c920 con 0x55ba1802b880 2023-09-10T00:48:39.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.474+0000 7f4349ffb700 10 client.17079 handle_command_reply: tid=0 2023-09-10T00:48:39.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.474+0000 7f4374927b80 1 client.17079 shutdown 2023-09-10T00:48:39.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.474+0000 7f4374927b80 1 -- 172.21.15.44:0/2909230092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba18011710 msgr2=0x55ba1801b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:39.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.474+0000 7f4374927b80 1 --2- 172.21.15.44:0/2909230092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba18011710 0x55ba1801b6e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f435c0cd3b0 tx=0x7f435c09c470 comp rx=0 tx=0).stop 2023-09-10T00:48:39.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.474+0000 7f4374927b80 1 -- 172.21.15.44:0/2909230092 shutdown_connections 2023-09-10T00:48:39.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.474+0000 7f4374927b80 1 --2- 172.21.15.44:0/2909230092 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ba1802b880 0x55ba1802dca0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.474+0000 7f4374927b80 1 --2- 172.21.15.44:0/2909230092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba1801be40 0x55ba18018ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.474+0000 7f4349ffb700 0 client.17079 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:39.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.474+0000 7f4374927b80 1 --2- 172.21.15.44:0/2909230092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba18011710 0x55ba1801b6e0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.474+0000 7f4374927b80 1 --2- 172.21.15.44:0/2909230092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18012030 0x55ba18018960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.491 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.474+0000 7f4374927b80 1 -- 172.21.15.44:0/2909230092 >> 172.21.15.44:0/2909230092 conn(0x55ba17f86090 msgr2=0x55ba1800c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:39.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.474+0000 7f4374927b80 1 -- 172.21.15.44:0/2909230092 shutdown_connections 2023-09-10T00:48:39.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.474+0000 7f4374927b80 1 -- 172.21.15.44:0/2909230092 wait complete. 2023-09-10T00:48:39.492 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.474+0000 7f4374927b80 20 client.17079 trim_cache size 0 max 16384 2023-09-10T00:48:39.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.493+0000 7f4374927b80 1 Processor -- start 2023-09-10T00:48:39.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.493+0000 7f4374927b80 1 -- start start 2023-09-10T00:48:39.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f4374927b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18011710 0x55ba18022990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.495 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f4374927b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18012030 0x55ba18010e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f4374927b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba1801be40 0x55ba18015bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f4374927b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba180145b0 con 0x55ba1801be40 2023-09-10T00:48:39.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f4374927b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1800d130 con 0x55ba18011710 2023-09-10T00:48:39.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f4374927b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba18012790 con 0x55ba18012030 2023-09-10T00:48:39.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f43664fd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18011710 0x55ba18022990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.496 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f4366cfe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba1801be40 0x55ba18015bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f4365cfc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18012030 0x55ba18010e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f43664fd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18011710 0x55ba18022990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53554/0 (socket says 172.21.15.44:53554) 2023-09-10T00:48:39.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f43664fd700 1 -- 172.21.15.44:0/964419744 learned_addr learned my addr 172.21.15.44:0/964419744 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:39.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f4366cfe700 1 -- 172.21.15.44:0/964419744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18012030 msgr2=0x55ba18010e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:39.497 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f4366cfe700 1 --2- 172.21.15.44:0/964419744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18012030 0x55ba18010e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.498 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f4366cfe700 1 -- 172.21.15.44:0/964419744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18011710 msgr2=0x55ba18022990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:39.498 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f4366cfe700 1 --2- 172.21.15.44:0/964419744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18011710 0x55ba18022990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.498 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f4366cfe700 1 -- 172.21.15.44:0/964419744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba17db6250 con 0x55ba1801be40 2023-09-10T00:48:39.498 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f43664fd700 1 --2- 172.21.15.44:0/964419744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18011710 0x55ba18022990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:39.498 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.494+0000 7f4366cfe700 1 --2- 172.21.15.44:0/964419744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba1801be40 0x55ba18015bd0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f435800d010 tx=0x7f4358009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a05a0de46f36cb2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:39.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.495+0000 7f43557fa700 1 -- 172.21.15.44:0/964419744 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435800e040 con 0x55ba1801be40 2023-09-10T00:48:39.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.495+0000 7f43557fa700 1 -- 172.21.15.44:0/964419744 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f435800bcb0 con 0x55ba1801be40 2023-09-10T00:48:39.499 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.495+0000 7f43557fa700 1 -- 172.21.15.44:0/964419744 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358005310 con 0x55ba1801be40 2023-09-10T00:48:39.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.495+0000 7f4374927b80 1 -- 172.21.15.44:0/964419744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba1801be40 msgr2=0x55ba18015bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:39.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.495+0000 7f4374927b80 1 --2- 172.21.15.44:0/964419744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba1801be40 0x55ba18015bd0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f435800d010 tx=0x7f4358009780 comp rx=0 tx=0).stop 2023-09-10T00:48:39.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.495+0000 7f4374927b80 1 -- 172.21.15.44:0/964419744 shutdown_connections 2023-09-10T00:48:39.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.495+0000 7f4374927b80 1 --2- 172.21.15.44:0/964419744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba1801be40 0x55ba18015bd0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.503 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.495+0000 7f4374927b80 1 --2- 172.21.15.44:0/964419744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18011710 0x55ba18022990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.504 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.495+0000 7f4374927b80 1 --2- 172.21.15.44:0/964419744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18012030 0x55ba18010e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.504 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.495+0000 7f4374927b80 1 -- 172.21.15.44:0/964419744 >> 172.21.15.44:0/964419744 conn(0x55ba17f86090 msgr2=0x55ba18010700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:39.504 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.495+0000 7f4374927b80 1 -- 172.21.15.44:0/964419744 shutdown_connections 2023-09-10T00:48:39.504 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.495+0000 7f4374927b80 1 -- 172.21.15.44:0/964419744 wait complete. 2023-09-10T00:48:39.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.496+0000 7f4374927b80 1 -- start start 2023-09-10T00:48:39.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.496+0000 7f4374927b80 1 Processor -- start 2023-09-10T00:48:39.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.496+0000 7f4374927b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18011710 0x55ba18077a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.496+0000 7f4374927b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18012030 0x55ba18077f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.496+0000 7f4374927b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba1801be40 0x55ba18073640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.505 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.496+0000 7f4374927b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba18073c10 con 0x55ba1801be40 2023-09-10T00:48:39.506 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.496+0000 7f4374927b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba18073d80 con 0x55ba18012030 2023-09-10T00:48:39.506 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.496+0000 7f4374927b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba18073f40 con 0x55ba18011710 2023-09-10T00:48:39.506 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.496+0000 7f4374927b80 10 client.0 _ll_register_callbacks cb 0x7f435c09c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:48:39.506 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.496+0000 7f43664fd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18011710 0x55ba18077a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.506 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.496+0000 7f43664fd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18011710 0x55ba18077a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58702/0 (socket says 172.21.15.44:58702) 2023-09-10T00:48:39.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.496+0000 7f43664fd700 1 -- 172.21.15.44:0/3427219756 learned_addr learned my addr 172.21.15.44:0/3427219756 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:39.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.496+0000 7f4365cfc700 1 --2- 172.21.15.44:0/3427219756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18012030 0x55ba18077f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.497+0000 7f43664fd700 1 -- 172.21.15.44:0/3427219756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18012030 msgr2=0x55ba18077f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:39.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.497+0000 7f43664fd700 1 --2- 172.21.15.44:0/3427219756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18012030 0x55ba18077f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.507 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.497+0000 7f43664fd700 1 -- 172.21.15.44:0/3427219756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba1801be40 msgr2=0x55ba18073640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T00:48:39.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.497+0000 7f43664fd700 1 --2- 172.21.15.44:0/3427219756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba1801be40 0x55ba18073640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.497+0000 7f43664fd700 1 -- 172.21.15.44:0/3427219756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba17d78c60 con 0x55ba18011710 2023-09-10T00:48:39.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.497+0000 7f43664fd700 1 --2- 172.21.15.44:0/3427219756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18011710 0x55ba18077a00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f435c0cd330 tx=0x7f435c019ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ced13cf65ab12763 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:39.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.497+0000 7f4349ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:48:39.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.497+0000 7f4349ffb700 1 -- 172.21.15.44:0/3427219756 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c09d290 con 0x55ba18011710 2023-09-10T00:48:39.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.497+0000 7f4349ffb700 1 -- 172.21.15.44:0/3427219756 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f435c080080 con 0x55ba18011710 2023-09-10T00:48:39.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.497+0000 7f4374927b80 1 -- 172.21.15.44:0/3427219756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba17ebb4f0 con 0x55ba18011710 2023-09-10T00:48:39.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.497+0000 7f4349ffb700 1 -- 172.21.15.44:0/3427219756 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f435c09cb10 con 0x55ba18011710 2023-09-10T00:48:39.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.497+0000 7f4374927b80 10 client.26686 fetch_fsmap learned FSMap version 51 2023-09-10T00:48:39.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.497+0000 7f4374927b80 1 -- 172.21.15.44:0/3427219756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55ba17d8fde0 con 0x55ba18011710 2023-09-10T00:48:39.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.498+0000 7f4349ffb700 1 -- 172.21.15.44:0/3427219756 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ba17ebb4f0 con 0x55ba18011710 2023-09-10T00:48:39.510 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.498+0000 7f4349ffb700 1 -- 172.21.15.44:0/3427219756 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 51) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f435c09ba70 con 0x55ba18011710 2023-09-10T00:48:39.510 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.498+0000 7f4374927b80 10 client.26686 fetch_fsmap finished waiting for FSMap version 51 2023-09-10T00:48:39.510 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.498+0000 7f4374927b80 10 client.26686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:48:39.510 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.498+0000 7f4374927b80 20 client.26686 populate_metadata read hostname 'smithi044' 2023-09-10T00:48:39.510 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.498+0000 7f4374927b80 1 --2- 172.21.15.44:0/3427219756 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ba18080610 0x55ba18082a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:39.511 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.498+0000 7f4374927b80 4 client.26686 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:48:39.511 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.498+0000 7f4374927b80 1 -- 172.21.15.44:0/3427219756 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba17f1d0b0 con 0x55ba18080610 2023-09-10T00:48:39.511 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.498+0000 7f4365cfc700 1 --2- 172.21.15.44:0/3427219756 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ba18080610 0x55ba18082a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:39.511 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.498+0000 7f4349ffb700 10 client.26686 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:39.511 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.498+0000 7f4365cfc700 1 --2- 172.21.15.44:0/3427219756 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ba18080610 0x55ba18082a30 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ec5e9ab6c013ad server_cookie=c08487d3f7d3d399 in_seq=0 out_seq=0 2023-09-10T00:48:39.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4349ffb700 1 -- 172.21.15.44:0/3427219756 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ba17f1d0b0 con 0x55ba18080610 2023-09-10T00:48:39.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4349ffb700 10 client.26686 handle_command_reply: tid=0 2023-09-10T00:48:39.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4374927b80 1 client.26686 shutdown 2023-09-10T00:48:39.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4374927b80 1 -- 172.21.15.44:0/3427219756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18011710 msgr2=0x55ba18077a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:39.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4374927b80 1 --2- 172.21.15.44:0/3427219756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18011710 0x55ba18077a00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f435c0cd330 tx=0x7f435c019ae0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4374927b80 1 -- 172.21.15.44:0/3427219756 shutdown_connections 2023-09-10T00:48:39.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4374927b80 1 --2- 172.21.15.44:0/3427219756 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ba18080610 0x55ba18082a30 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4374927b80 1 --2- 172.21.15.44:0/3427219756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba1801be40 0x55ba18073640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4349ffb700 0 client.26686 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:39.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4374927b80 1 --2- 172.21.15.44:0/3427219756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba18011710 0x55ba18077a00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4374927b80 1 --2- 172.21.15.44:0/3427219756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba18012030 0x55ba18077f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4374927b80 1 -- 172.21.15.44:0/3427219756 >> 172.21.15.44:0/3427219756 conn(0x55ba17f86090 msgr2=0x55ba1800bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:39.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4374927b80 1 -- 172.21.15.44:0/3427219756 shutdown_connections 2023-09-10T00:48:39.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4374927b80 1 -- 172.21.15.44:0/3427219756 wait complete. 2023-09-10T00:48:39.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4374927b80 20 client.26686 trim_cache size 0 max 16384 2023-09-10T00:48:39.515 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:48:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:48:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:48:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:48:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:48:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:48:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:48:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:48:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:48:39.516 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:48:39.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4368761700 1 -- 172.21.15.44:0/65050896 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f434c0a07b0 msgr2=0x7f434c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:39.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4368761700 1 --2- 172.21.15.44:0/65050896 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f434c0a07b0 0x7f434c0a2c70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4360197d10 tx=0x7f4350009260 comp rx=0 tx=0).stop 2023-09-10T00:48:39.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4368761700 1 -- 172.21.15.44:0/65050896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4360101820 msgr2=0x7f4360196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:39.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.499+0000 7f4368761700 1 --2- 172.21.15.44:0/65050896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4360101820 0x7f4360196700 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f435c009c50 tx=0x7f435c009f60 comp rx=0 tx=0).stop 2023-09-10T00:48:39.518 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.500+0000 7f4368761700 1 -- 172.21.15.44:0/65050896 shutdown_connections 2023-09-10T00:48:39.519 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.500+0000 7f4368761700 1 --2- 172.21.15.44:0/65050896 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f434c0a07b0 0x7f434c0a2c70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.519 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.500+0000 7f4368761700 1 --2- 172.21.15.44:0/65050896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f436010abf0 0x7f4360199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.519 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.500+0000 7f4368761700 1 --2- 172.21.15.44:0/65050896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4360101820 0x7f4360196700 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.519 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.500+0000 7f4368761700 1 --2- 172.21.15.44:0/65050896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4360102140 0x7f4360196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:39.519 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.500+0000 7f4368761700 1 -- 172.21.15.44:0/65050896 >> 172.21.15.44:0/65050896 conn(0x7f43600fafa0 msgr2=0x7f436010ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:39.519 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.500+0000 7f4368761700 1 -- 172.21.15.44:0/65050896 shutdown_connections 2023-09-10T00:48:39.520 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:48:39.500+0000 7f4368761700 1 -- 172.21.15.44:0/65050896 wait complete. 2023-09-10T00:48:39.520 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:48:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:40 smithi131 ceph-mon[103713]: pgmap v247: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 2.6 MiB/s wr, 293 op/s 2023-09-10T00:48:41.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:40 smithi044 ceph-mon[98676]: pgmap v247: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 2.6 MiB/s wr, 293 op/s 2023-09-10T00:48:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:40 smithi099 ceph-mon[102692]: pgmap v247: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 2.6 MiB/s wr, 293 op/s 2023-09-10T00:48:42.820 INFO:teuthology.orchestra.run.smithi044.stderr:31800000 of 50000000 tuples (63%) done (elapsed 462.45 s, remaining 264.67 s) 2023-09-10T00:48:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:42 smithi131 ceph-mon[103713]: pgmap v248: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 4.0 MiB/s wr, 446 op/s 2023-09-10T00:48:43.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:42 smithi044 ceph-mon[98676]: pgmap v248: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 4.0 MiB/s wr, 446 op/s 2023-09-10T00:48:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:42 smithi099 ceph-mon[102692]: pgmap v248: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 4.0 MiB/s wr, 446 op/s 2023-09-10T00:48:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:44 smithi131 ceph-mon[103713]: pgmap v249: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.7 MiB/s wr, 306 op/s 2023-09-10T00:48:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:44 smithi044 ceph-mon[98676]: pgmap v249: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.7 MiB/s wr, 306 op/s 2023-09-10T00:48:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:44 smithi099 ceph-mon[102692]: pgmap v249: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.7 MiB/s wr, 306 op/s 2023-09-10T00:48:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:46 smithi131 ceph-mon[103713]: pgmap v250: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 4.0 MiB/s wr, 447 op/s 2023-09-10T00:48:47.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:46 smithi044 ceph-mon[98676]: pgmap v250: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 4.0 MiB/s wr, 447 op/s 2023-09-10T00:48:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:46 smithi099 ceph-mon[102692]: pgmap v250: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 4.0 MiB/s wr, 447 op/s 2023-09-10T00:48:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:48 smithi131 ceph-mon[103713]: pgmap v251: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 2.6 MiB/s wr, 294 op/s 2023-09-10T00:48:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:48 smithi044 ceph-mon[98676]: pgmap v251: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 2.6 MiB/s wr, 294 op/s 2023-09-10T00:48:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:48 smithi099 ceph-mon[102692]: pgmap v251: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 2.6 MiB/s wr, 294 op/s 2023-09-10T00:48:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.716+0000 7f55afc82700 1 -- 172.21.15.44:0/4207307232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55a80ff760 msgr2=0x7f55a80ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.716+0000 7f55afc82700 1 --2- 172.21.15.44:0/4207307232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55a80ff760 0x7f55a80ffbe0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f55980099c0 tx=0x7f5598009cd0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.717+0000 7f55afc82700 1 -- 172.21.15.44:0/4207307232 shutdown_connections 2023-09-10T00:48:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.717+0000 7f55afc82700 1 --2- 172.21.15.44:0/4207307232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a8100120 0x7f55a810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.717+0000 7f55afc82700 1 --2- 172.21.15.44:0/4207307232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a81055b0 0x7f55a8105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.717+0000 7f55afc82700 1 --2- 172.21.15.44:0/4207307232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55a80ff760 0x7f55a80ffbe0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.717+0000 7f55afc82700 1 -- 172.21.15.44:0/4207307232 >> 172.21.15.44:0/4207307232 conn(0x7f55a8074f30 msgr2=0x7f55a8075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.717+0000 7f55afc82700 1 -- 172.21.15.44:0/4207307232 shutdown_connections 2023-09-10T00:48:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.717+0000 7f55afc82700 1 -- 172.21.15.44:0/4207307232 wait complete. 2023-09-10T00:48:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.717+0000 7f55afc82700 1 Processor -- start 2023-09-10T00:48:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.717+0000 7f55afc82700 1 -- start start 2023-09-10T00:48:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55afc82700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55a80ff760 0x7f55a8194340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55afc82700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a8100120 0x7f55a8194880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55afc82700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a81055b0 0x7f55a8198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55afc82700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8199530 con 0x7f55a80ff760 2023-09-10T00:48:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55afc82700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a81996a0 con 0x7f55a8100120 2023-09-10T00:48:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55afc82700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8199860 con 0x7f55a81055b0 2023-09-10T00:48:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55ada1e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55a80ff760 0x7f55a8194340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55ada1e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55a80ff760 0x7f55a8194340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51832/0 (socket says 172.21.15.44:51832) 2023-09-10T00:48:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55ada1e700 1 -- 172.21.15.44:0/3358687697 learned_addr learned my addr 172.21.15.44:0/3358687697 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55ad21d700 1 --2- 172.21.15.44:0/3358687697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a8100120 0x7f55a8194880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55ae21f700 1 --2- 172.21.15.44:0/3358687697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a81055b0 0x7f55a8198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55ada1e700 1 -- 172.21.15.44:0/3358687697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a81055b0 msgr2=0x7f55a8198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55ada1e700 1 --2- 172.21.15.44:0/3358687697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a81055b0 0x7f55a8198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55ada1e700 1 -- 172.21.15.44:0/3358687697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a8100120 msgr2=0x7f55a8194880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55ada1e700 1 --2- 172.21.15.44:0/3358687697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a8100120 0x7f55a8194880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55ada1e700 1 -- 172.21.15.44:0/3358687697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55980095f0 con 0x7f55a80ff760 2023-09-10T00:48:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55ad21d700 1 --2- 172.21.15.44:0/3358687697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a8100120 0x7f55a8194880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55ada1e700 1 --2- 172.21.15.44:0/3358687697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55a80ff760 0x7f55a8194340 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f55a400e9a0 tx=0x7f55a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f559effd700 1 -- 172.21.15.44:0/3358687697 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a4009790 con 0x7f55a80ff760 2023-09-10T00:48:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f559effd700 1 -- 172.21.15.44:0/3358687697 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55a4010430 con 0x7f55a80ff760 2023-09-10T00:48:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f559effd700 1 -- 172.21.15.44:0/3358687697 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a400cea0 con 0x7f55a80ff760 2023-09-10T00:48:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.718+0000 7f55afc82700 1 -- 172.21.15.44:0/3358687697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a8199b40 con 0x7f55a80ff760 2023-09-10T00:48:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.719+0000 7f55afc82700 1 -- 172.21.15.44:0/3358687697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a81a1380 con 0x7f55a80ff760 2023-09-10T00:48:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.721+0000 7f559effd700 1 -- 172.21.15.44:0/3358687697 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f55a40105a0 con 0x7f55a80ff760 2023-09-10T00:48:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.722+0000 7f559effd700 1 --2- 172.21.15.44:0/3358687697 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55940a0830 0x7f55940a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.722+0000 7f559effd700 1 -- 172.21.15.44:0/3358687697 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f55a4014070 con 0x7f55a80ff760 2023-09-10T00:48:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.722+0000 7f55ad21d700 1 --2- 172.21.15.44:0/3358687697 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55940a0830 0x7f55940a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.722+0000 7f55bbe48b80 1 Processor -- start 2023-09-10T00:48:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.723+0000 7f55bbe48b80 1 -- start start 2023-09-10T00:48:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.723+0000 7f55ad21d700 1 --2- 172.21.15.44:0/3358687697 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55940a0830 0x7f55940a2cf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f559800b580 tx=0x7f5598012040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.723+0000 7f55bbe48b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea52471a0 0x559ea5247580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.723+0000 7f55bbe48b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241300 0x559ea5241780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.723+0000 7f55bbe48b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea5241cc0 0x559ea524df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.723+0000 7f55bbe48b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea519a6d0 con 0x559ea5241cc0 2023-09-10T00:48:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.723+0000 7f55bbe48b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea5197ff0 con 0x559ea52471a0 2023-09-10T00:48:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.723+0000 7f55bbe48b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea51938e0 con 0x559ea5241300 2023-09-10T00:48:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.723+0000 7f55ada1e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea52471a0 0x559ea5247580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.723+0000 7f55ada1e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea52471a0 0x559ea5247580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53580/0 (socket says 172.21.15.44:53580) 2023-09-10T00:48:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.723+0000 7f55ad21d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241300 0x559ea5241780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.723+0000 7f55ada1e700 1 -- 172.21.15.44:0/389539508 learned_addr learned my addr 172.21.15.44:0/389539508 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.723+0000 7f55ae21f700 1 --2- 172.21.15.44:0/389539508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea5241cc0 0x559ea524df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.724+0000 7f55ad21d700 1 -- 172.21.15.44:0/389539508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea52471a0 msgr2=0x559ea5247580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.724+0000 7f55ad21d700 1 --2- 172.21.15.44:0/389539508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea52471a0 0x559ea5247580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.724+0000 7f55ad21d700 1 -- 172.21.15.44:0/389539508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea5241cc0 msgr2=0x559ea524df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.724+0000 7f55ad21d700 1 --2- 172.21.15.44:0/389539508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea5241cc0 0x559ea524df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.724+0000 7f55ad21d700 1 -- 172.21.15.44:0/389539508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ea4fbfd80 con 0x559ea5241300 2023-09-10T00:48:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.724+0000 7f55ad21d700 1 --2- 172.21.15.44:0/389539508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241300 0x559ea5241780 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5598009fd0 tx=0x7f5598023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b98ac0c8a2c5609d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.725+0000 7f5593fff700 1 -- 172.21.15.44:0/389539508 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559800b9d0 con 0x559ea5241300 2023-09-10T00:48:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.725+0000 7f5593fff700 1 -- 172.21.15.44:0/389539508 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f559800bb30 con 0x559ea5241300 2023-09-10T00:48:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.725+0000 7f5593fff700 1 -- 172.21.15.44:0/389539508 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598030660 con 0x559ea5241300 2023-09-10T00:48:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.726+0000 7f55bbe48b80 1 -- 172.21.15.44:0/389539508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241300 msgr2=0x559ea5241780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.726+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/389539508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241300 0x559ea5241780 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5598009fd0 tx=0x7f5598023000 comp rx=0 tx=0).stop 2023-09-10T00:48:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.726+0000 7f55bbe48b80 1 -- 172.21.15.44:0/389539508 shutdown_connections 2023-09-10T00:48:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.726+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/389539508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea5241cc0 0x559ea524df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.726+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/389539508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea52471a0 0x559ea5247580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.726+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/389539508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241300 0x559ea5241780 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.726+0000 7f55bbe48b80 1 -- 172.21.15.44:0/389539508 >> 172.21.15.44:0/389539508 conn(0x559ea51b6030 msgr2=0x559ea523c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.726+0000 7f55bbe48b80 1 -- 172.21.15.44:0/389539508 shutdown_connections 2023-09-10T00:48:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.727+0000 7f55bbe48b80 1 -- 172.21.15.44:0/389539508 wait complete. 2023-09-10T00:48:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.728+0000 7f55bbe48b80 1 -- start start 2023-09-10T00:48:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.728+0000 7f55bbe48b80 1 Processor -- start 2023-09-10T00:48:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.728+0000 7f55bbe48b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea5241300 0x559ea524de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.728+0000 7f55bbe48b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241cc0 0x559ea524b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.728+0000 7f55bbe48b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea52471a0 0x559ea524b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.728+0000 7f55bbe48b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea5131b10 con 0x559ea5241300 2023-09-10T00:48:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.728+0000 7f55bbe48b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea5131810 con 0x559ea52471a0 2023-09-10T00:48:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.728+0000 7f55bbe48b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea50158c0 con 0x559ea5241cc0 2023-09-10T00:48:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.728+0000 7f55bbe48b80 10 client.0 _ll_register_callbacks cb 0x559ea507ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:48:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.728+0000 7f55ada1e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea5241300 0x559ea524de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.729+0000 7f55ae21f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea52471a0 0x559ea524b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.729+0000 7f55ada1e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea5241300 0x559ea524de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51846/0 (socket says 172.21.15.44:51846) 2023-09-10T00:48:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.729+0000 7f55ada1e700 1 -- 172.21.15.44:0/3542460735 learned_addr learned my addr 172.21.15.44:0/3542460735 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.729+0000 7f55ad21d700 1 --2- 172.21.15.44:0/3542460735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241cc0 0x559ea524b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.729+0000 7f55ae21f700 1 -- 172.21.15.44:0/3542460735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241cc0 msgr2=0x559ea524b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.729+0000 7f55ae21f700 1 --2- 172.21.15.44:0/3542460735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241cc0 0x559ea524b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.729+0000 7f55ae21f700 1 -- 172.21.15.44:0/3542460735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea5241300 msgr2=0x559ea524de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.729+0000 7f55ae21f700 1 --2- 172.21.15.44:0/3542460735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea5241300 0x559ea524de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.729+0000 7f55ae21f700 1 -- 172.21.15.44:0/3542460735 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a8067bb0 con 0x559ea52471a0 2023-09-10T00:48:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.729+0000 7f55ada1e700 1 --2- 172.21.15.44:0/3542460735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea5241300 0x559ea524de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.729+0000 7f55ae21f700 1 --2- 172.21.15.44:0/3542460735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea52471a0 0x559ea524b5b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f55a0011e90 tx=0x7f55a0014600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d05c7013883657c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.729+0000 7f55917fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:48:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.729+0000 7f55917fa700 1 -- 172.21.15.44:0/3542460735 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a000e040 con 0x559ea52471a0 2023-09-10T00:48:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.729+0000 7f55917fa700 1 -- 172.21.15.44:0/3542460735 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f55a00093a0 con 0x559ea52471a0 2023-09-10T00:48:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.729+0000 7f55bbe48b80 1 -- 172.21.15.44:0/3542460735 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ea50eb490 con 0x559ea52471a0 2023-09-10T00:48:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.729+0000 7f55917fa700 1 -- 172.21.15.44:0/3542460735 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55a0004db0 con 0x559ea52471a0 2023-09-10T00:48:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.730+0000 7f55bbe48b80 10 client.26753 fetch_fsmap learned FSMap version 51 2023-09-10T00:48:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.730+0000 7f55bbe48b80 1 -- 172.21.15.44:0/3542460735 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x559ea4fa8c00 con 0x559ea52471a0 2023-09-10T00:48:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.730+0000 7f55917fa700 1 -- 172.21.15.44:0/3542460735 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559ea50eb490 con 0x559ea52471a0 2023-09-10T00:48:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.730+0000 7f55917fa700 1 -- 172.21.15.44:0/3542460735 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 51) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f55a001c030 con 0x559ea52471a0 2023-09-10T00:48:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.730+0000 7f55bbe48b80 10 client.26753 fetch_fsmap finished waiting for FSMap version 51 2023-09-10T00:48:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.730+0000 7f55bbe48b80 10 client.26753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:48:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.730+0000 7f55bbe48b80 20 client.26753 populate_metadata read hostname 'smithi044' 2023-09-10T00:48:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.730+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/3542460735 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559ea525b400 0x559ea525d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.730+0000 7f55bbe48b80 4 client.26753 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:48:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.730+0000 7f55bbe48b80 1 -- 172.21.15.44:0/3542460735 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ea51a8840 con 0x559ea525b400 2023-09-10T00:48:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.730+0000 7f55ada1e700 1 --2- 172.21.15.44:0/3542460735 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559ea525b400 0x559ea525d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.731+0000 7f55917fa700 10 client.26753 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.731+0000 7f55ada1e700 1 --2- 172.21.15.44:0/3542460735 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559ea525b400 0x559ea525d820 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46a07f240b757b73 server_cookie=3675c80e036bb935 in_seq=0 out_seq=0 2023-09-10T00:48:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.731+0000 7f55917fa700 1 -- 172.21.15.44:0/3542460735 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559ea51a8840 con 0x559ea525b400 2023-09-10T00:48:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.731+0000 7f55917fa700 10 client.26753 handle_command_reply: tid=0 2023-09-10T00:48:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.731+0000 7f55bbe48b80 1 client.26753 shutdown 2023-09-10T00:48:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.731+0000 7f55bbe48b80 1 -- 172.21.15.44:0/3542460735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea52471a0 msgr2=0x559ea524b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.731+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/3542460735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea52471a0 0x559ea524b5b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f55a0011e90 tx=0x7f55a0014600 comp rx=0 tx=0).stop 2023-09-10T00:48:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.731+0000 7f55bbe48b80 1 -- 172.21.15.44:0/3542460735 shutdown_connections 2023-09-10T00:48:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.731+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/3542460735 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559ea525b400 0x559ea525d820 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.731+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/3542460735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea52471a0 0x559ea524b5b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.731+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/3542460735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea5241300 0x559ea524de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.731+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/3542460735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241cc0 0x559ea524b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.731+0000 7f55917fa700 0 client.26753 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.732+0000 7f55bbe48b80 1 -- 172.21.15.44:0/3542460735 >> 172.21.15.44:0/3542460735 conn(0x559ea51b6030 msgr2=0x559ea523b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.732+0000 7f55bbe48b80 1 -- 172.21.15.44:0/3542460735 shutdown_connections 2023-09-10T00:48:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.732+0000 7f55bbe48b80 1 -- 172.21.15.44:0/3542460735 wait complete. 2023-09-10T00:48:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.732+0000 7f55bbe48b80 20 client.26753 trim_cache size 0 max 16384 2023-09-10T00:48:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.750+0000 7f55bbe48b80 1 Processor -- start 2023-09-10T00:48:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.750+0000 7f55bbe48b80 1 -- start start 2023-09-10T00:48:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.750+0000 7f55bbe48b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea5241300 0x559ea5252420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.750+0000 7f55bbe48b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241cc0 0x559ea524d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.750+0000 7f55bbe48b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea52471a0 0x559ea52402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.750+0000 7f55bbe48b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea5258950 con 0x559ea52471a0 2023-09-10T00:48:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.750+0000 7f55bbe48b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea5252960 con 0x559ea5241300 2023-09-10T00:48:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.750+0000 7f55bbe48b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea5249570 con 0x559ea5241cc0 2023-09-10T00:48:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.751+0000 7f55ae21f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea52471a0 0x559ea52402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.751+0000 7f55ada1e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea5241300 0x559ea5252420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.751+0000 7f55ad21d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241cc0 0x559ea524d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.751+0000 7f55ae21f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea52471a0 0x559ea52402f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51858/0 (socket says 172.21.15.44:51858) 2023-09-10T00:48:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.751+0000 7f55ae21f700 1 -- 172.21.15.44:0/3744549985 learned_addr learned my addr 172.21.15.44:0/3744549985 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.751+0000 7f55ae21f700 1 -- 172.21.15.44:0/3744549985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241cc0 msgr2=0x559ea524d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.751+0000 7f55ae21f700 1 --2- 172.21.15.44:0/3744549985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241cc0 0x559ea524d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.751+0000 7f55ae21f700 1 -- 172.21.15.44:0/3744549985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea5241300 msgr2=0x559ea5252420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.751+0000 7f55ae21f700 1 --2- 172.21.15.44:0/3744549985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea5241300 0x559ea5252420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.751+0000 7f55ae21f700 1 -- 172.21.15.44:0/3744549985 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ea4fe61f0 con 0x559ea52471a0 2023-09-10T00:48:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.751+0000 7f55ada1e700 1 --2- 172.21.15.44:0/3744549985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea5241300 0x559ea5252420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:48:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.751+0000 7f55ae21f700 1 --2- 172.21.15.44:0/3744549985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea52471a0 0x559ea52402f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f55a000b010 tx=0x7f55a00093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef92159f0161a08 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.751+0000 7f559cd89700 1 -- 172.21.15.44:0/3744549985 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a002e070 con 0x559ea52471a0 2023-09-10T00:48:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.751+0000 7f559cd89700 1 -- 172.21.15.44:0/3744549985 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55a000f040 con 0x559ea52471a0 2023-09-10T00:48:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.751+0000 7f559cd89700 1 -- 172.21.15.44:0/3744549985 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a000e070 con 0x559ea52471a0 2023-09-10T00:48:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.752+0000 7f55bbe48b80 1 -- 172.21.15.44:0/3744549985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea52471a0 msgr2=0x559ea52402f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.752+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/3744549985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea52471a0 0x559ea52402f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f55a000b010 tx=0x7f55a00093a0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.752+0000 7f55bbe48b80 1 -- 172.21.15.44:0/3744549985 shutdown_connections 2023-09-10T00:48:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.752+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/3744549985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea52471a0 0x559ea52402f0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.752+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/3744549985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea5241300 0x559ea5252420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.752+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/3744549985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241cc0 0x559ea524d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.752+0000 7f55bbe48b80 1 -- 172.21.15.44:0/3744549985 >> 172.21.15.44:0/3744549985 conn(0x559ea51b6030 msgr2=0x559ea523c090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.752+0000 7f55bbe48b80 1 -- 172.21.15.44:0/3744549985 shutdown_connections 2023-09-10T00:48:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.752+0000 7f55bbe48b80 1 -- 172.21.15.44:0/3744549985 wait complete. 2023-09-10T00:48:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.752+0000 7f55bbe48b80 1 -- start start 2023-09-10T00:48:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.753+0000 7f55bbe48b80 1 Processor -- start 2023-09-10T00:48:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.753+0000 7f55bbe48b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241300 0x559ea5251fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.753+0000 7f55bbe48b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea5241cc0 0x559ea52503f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.753+0000 7f55bbe48b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea52471a0 0x559ea5250930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.753+0000 7f55bbe48b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea5250e70 con 0x559ea52471a0 2023-09-10T00:48:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.753+0000 7f55bbe48b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea5250fe0 con 0x559ea5241cc0 2023-09-10T00:48:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.753+0000 7f55bbe48b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea52511a0 con 0x559ea5241300 2023-09-10T00:48:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.753+0000 7f55bbe48b80 10 client.0 _ll_register_callbacks cb 0x7f55a00117c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:48:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.753+0000 7f55ada1e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241300 0x559ea5251fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.753+0000 7f55ad21d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea5241cc0 0x559ea52503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.753+0000 7f55ada1e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241300 0x559ea5251fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58742/0 (socket says 172.21.15.44:58742) 2023-09-10T00:48:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.753+0000 7f55ad21d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea5241cc0 0x559ea52503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53602/0 (socket says 172.21.15.44:53602) 2023-09-10T00:48:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.753+0000 7f55ada1e700 1 -- 172.21.15.44:0/1292988276 learned_addr learned my addr 172.21.15.44:0/1292988276 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:48:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.754+0000 7f55ae21f700 1 --2- 172.21.15.44:0/1292988276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea52471a0 0x559ea5250930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.754+0000 7f55ada1e700 1 -- 172.21.15.44:0/1292988276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea5241cc0 msgr2=0x559ea52503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.754+0000 7f55ada1e700 1 --2- 172.21.15.44:0/1292988276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea5241cc0 0x559ea52503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.754+0000 7f55ada1e700 1 -- 172.21.15.44:0/1292988276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea52471a0 msgr2=0x559ea5250930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:48:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.754+0000 7f55ada1e700 1 --2- 172.21.15.44:0/1292988276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea52471a0 0x559ea5250930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.754+0000 7f55ada1e700 1 -- 172.21.15.44:0/1292988276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a81a1380 con 0x559ea5241300 2023-09-10T00:48:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.754+0000 7f55ae21f700 1 --2- 172.21.15.44:0/1292988276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea52471a0 0x559ea5250930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:48:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.754+0000 7f55ada1e700 1 --2- 172.21.15.44:0/1292988276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241300 0x559ea5251fa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f55a4095a70 tx=0x7f55a40cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d865451e5fa804c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:48:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.754+0000 7f55917fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:48:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.754+0000 7f55917fa700 1 -- 172.21.15.44:0/1292988276 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a4099570 con 0x559ea5241300 2023-09-10T00:48:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.754+0000 7f55bbe48b80 1 -- 172.21.15.44:0/1292988276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f559800bb30 con 0x559ea5241300 2023-09-10T00:48:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.754+0000 7f55917fa700 1 -- 172.21.15.44:0/1292988276 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f55a409d3a0 con 0x559ea5241300 2023-09-10T00:48:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.754+0000 7f55917fa700 1 -- 172.21.15.44:0/1292988276 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55a40a15e0 con 0x559ea5241300 2023-09-10T00:48:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.754+0000 7f55bbe48b80 10 client.26704 fetch_fsmap learned FSMap version 51 2023-09-10T00:48:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.754+0000 7f55bbe48b80 1 -- 172.21.15.44:0/1292988276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x559ea4fa8c00 con 0x559ea5241300 2023-09-10T00:48:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.755+0000 7f55917fa700 1 -- 172.21.15.44:0/1292988276 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f559800bb30 con 0x559ea5241300 2023-09-10T00:48:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.755+0000 7f55917fa700 1 -- 172.21.15.44:0/1292988276 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 51) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f55a4098a60 con 0x559ea5241300 2023-09-10T00:48:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.755+0000 7f55bbe48b80 10 client.26704 fetch_fsmap finished waiting for FSMap version 51 2023-09-10T00:48:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.755+0000 7f55bbe48b80 10 client.26704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:48:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.755+0000 7f55bbe48b80 20 client.26704 populate_metadata read hostname 'smithi044' 2023-09-10T00:48:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.755+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/1292988276 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559ea52b06b0 0x559ea52b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:48:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.755+0000 7f55bbe48b80 4 client.26704 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:48:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.755+0000 7f55bbe48b80 1 -- 172.21.15.44:0/1292988276 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559ea518c8c0 con 0x559ea52b06b0 2023-09-10T00:48:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.755+0000 7f55ad21d700 1 --2- 172.21.15.44:0/1292988276 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559ea52b06b0 0x559ea52b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:48:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.755+0000 7f55917fa700 10 client.26704 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.755+0000 7f55ad21d700 1 --2- 172.21.15.44:0/1292988276 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559ea52b06b0 0x559ea52b2ad0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdedb15488769195 server_cookie=d82024aa6c1146f7 in_seq=0 out_seq=0 2023-09-10T00:48:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.765+0000 7f55917fa700 1 -- 172.21.15.44:0/1292988276 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559ea518c8c0 con 0x559ea52b06b0 2023-09-10T00:48:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.765+0000 7f55917fa700 10 client.26704 handle_command_reply: tid=0 2023-09-10T00:48:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.765+0000 7f55bbe48b80 1 client.26704 shutdown 2023-09-10T00:48:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.765+0000 7f55bbe48b80 1 -- 172.21.15.44:0/1292988276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241300 msgr2=0x559ea5251fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.765+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/1292988276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241300 0x559ea5251fa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f55a4095a70 tx=0x7f55a40cc040 comp rx=0 tx=0).stop 2023-09-10T00:48:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.765+0000 7f55bbe48b80 1 -- 172.21.15.44:0/1292988276 shutdown_connections 2023-09-10T00:48:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.765+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/1292988276 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559ea52b06b0 0x559ea52b2ad0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.765+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/1292988276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ea52471a0 0x559ea5250930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.765+0000 7f55917fa700 0 client.26704 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:48:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.765+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/1292988276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ea5241300 0x559ea5251fa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.765+0000 7f55bbe48b80 1 --2- 172.21.15.44:0/1292988276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ea5241cc0 0x559ea52503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.765+0000 7f55bbe48b80 1 -- 172.21.15.44:0/1292988276 >> 172.21.15.44:0/1292988276 conn(0x559ea51b6030 msgr2=0x559ea523af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.765+0000 7f55bbe48b80 1 -- 172.21.15.44:0/1292988276 shutdown_connections 2023-09-10T00:48:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.765+0000 7f55bbe48b80 1 -- 172.21.15.44:0/1292988276 wait complete. 2023-09-10T00:48:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.765+0000 7f55bbe48b80 20 client.26704 trim_cache size 0 max 16384 2023-09-10T00:48:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:48:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:48:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:48:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:48:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.766+0000 7f55afc82700 1 -- 172.21.15.44:0/3358687697 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55940a0830 msgr2=0x7f55940a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.766+0000 7f55afc82700 1 --2- 172.21.15.44:0/3358687697 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55940a0830 0x7f55940a2cf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f559800b580 tx=0x7f5598012040 comp rx=0 tx=0).stop 2023-09-10T00:48:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.766+0000 7f55afc82700 1 -- 172.21.15.44:0/3358687697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55a80ff760 msgr2=0x7f55a8194340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:48:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.766+0000 7f55afc82700 1 --2- 172.21.15.44:0/3358687697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55a80ff760 0x7f55a8194340 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f55a400e9a0 tx=0x7f55a400ecb0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.766+0000 7f55afc82700 1 -- 172.21.15.44:0/3358687697 shutdown_connections 2023-09-10T00:48:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.766+0000 7f55afc82700 1 --2- 172.21.15.44:0/3358687697 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55940a0830 0x7f55940a2cf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.766+0000 7f55afc82700 1 --2- 172.21.15.44:0/3358687697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55a81055b0 0x7f55a8198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.766+0000 7f55afc82700 1 --2- 172.21.15.44:0/3358687697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55a80ff760 0x7f55a8194340 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.766+0000 7f55afc82700 1 --2- 172.21.15.44:0/3358687697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a8100120 0x7f55a8194880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:48:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.766+0000 7f55afc82700 1 -- 172.21.15.44:0/3358687697 >> 172.21.15.44:0/3358687697 conn(0x7f55a8074f30 msgr2=0x7f55a80fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:48:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.766+0000 7f55afc82700 1 -- 172.21.15.44:0/3358687697 shutdown_connections 2023-09-10T00:48:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:48:49.766+0000 7f55afc82700 1 -- 172.21.15.44:0/3358687697 wait complete. 2023-09-10T00:48:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:50 smithi131 ceph-mon[103713]: pgmap v252: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 2.6 MiB/s wr, 294 op/s 2023-09-10T00:48:51.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:50 smithi044 ceph-mon[98676]: pgmap v252: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 2.6 MiB/s wr, 294 op/s 2023-09-10T00:48:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:50 smithi099 ceph-mon[102692]: pgmap v252: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 2.6 MiB/s wr, 294 op/s 2023-09-10T00:48:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:51 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:48:52.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:51 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:48:52.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:51 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:48:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:52 smithi131 ceph-mon[103713]: pgmap v253: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 3.8 MiB/s wr, 431 op/s 2023-09-10T00:48:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:52 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:48:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:52 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:48:53.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:52 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:48:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:52 smithi044 ceph-mon[98676]: pgmap v253: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 3.8 MiB/s wr, 431 op/s 2023-09-10T00:48:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:52 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:48:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:52 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:48:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:52 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:48:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:52 smithi099 ceph-mon[102692]: pgmap v253: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 3.8 MiB/s wr, 431 op/s 2023-09-10T00:48:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:52 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:48:53.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:52 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:48:53.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:52 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:48:55.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:54 smithi131 ceph-mon[103713]: pgmap v254: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 2.5 MiB/s wr, 278 op/s 2023-09-10T00:48:55.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:54 smithi044 ceph-mon[98676]: pgmap v254: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 2.5 MiB/s wr, 278 op/s 2023-09-10T00:48:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:54 smithi099 ceph-mon[102692]: pgmap v254: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 2.5 MiB/s wr, 278 op/s 2023-09-10T00:48:57.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:56 smithi131 ceph-mon[103713]: pgmap v255: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 3.8 MiB/s wr, 422 op/s 2023-09-10T00:48:57.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:56 smithi044 ceph-mon[98676]: pgmap v255: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 3.8 MiB/s wr, 422 op/s 2023-09-10T00:48:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:56 smithi099 ceph-mon[102692]: pgmap v255: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 3.8 MiB/s wr, 422 op/s 2023-09-10T00:48:59.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:58 smithi131 ceph-mon[103713]: pgmap v256: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 2.5 MiB/s wr, 281 op/s 2023-09-10T00:48:59.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:58 smithi044 ceph-mon[98676]: pgmap v256: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 2.5 MiB/s wr, 281 op/s 2023-09-10T00:48:59.067 INFO:teuthology.orchestra.run.smithi044.stderr:31900000 of 50000000 tuples (63%) done (elapsed 478.70 s, remaining 271.61 s) 2023-09-10T00:48:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:58 smithi099 ceph-mon[102692]: pgmap v256: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 2.5 MiB/s wr, 281 op/s 2023-09-10T00:49:00.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:48:59 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:00.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:48:59 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:00.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:48:59 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:01.051 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:00 smithi131 ceph-mon[103713]: pgmap v257: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 2.5 MiB/s wr, 281 op/s 2023-09-10T00:49:01.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:00 smithi044 ceph-mon[98676]: pgmap v257: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 2.5 MiB/s wr, 281 op/s 2023-09-10T00:49:01.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:00 smithi099 ceph-mon[102692]: pgmap v257: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 2.5 MiB/s wr, 281 op/s 2023-09-10T00:49:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.885+0000 7fa49511b700 1 -- 172.21.15.44:0/2991835610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4901055b0 msgr2=0x7fa490105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.885+0000 7fa49511b700 1 --2- 172.21.15.44:0/2991835610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4901055b0 0x7fa490105990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa4800099c0 tx=0x7fa480009cd0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.885+0000 7fa49511b700 1 -- 172.21.15.44:0/2991835610 shutdown_connections 2023-09-10T00:49:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.885+0000 7fa49511b700 1 --2- 172.21.15.44:0/2991835610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa490100120 0x7fa49010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.885+0000 7fa49511b700 1 --2- 172.21.15.44:0/2991835610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4901055b0 0x7fa490105990 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.885+0000 7fa49511b700 1 --2- 172.21.15.44:0/2991835610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4900ff760 0x7fa4900ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.885+0000 7fa49511b700 1 -- 172.21.15.44:0/2991835610 >> 172.21.15.44:0/2991835610 conn(0x7fa490074f30 msgr2=0x7fa490075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.885+0000 7fa49511b700 1 -- 172.21.15.44:0/2991835610 shutdown_connections 2023-09-10T00:49:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.885+0000 7fa49511b700 1 -- 172.21.15.44:0/2991835610 wait complete. 2023-09-10T00:49:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.886+0000 7fa49511b700 1 Processor -- start 2023-09-10T00:49:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.886+0000 7fa49511b700 1 -- start start 2023-09-10T00:49:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.886+0000 7fa49511b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4900ff760 0x7fa49019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.886+0000 7fa49511b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa490100120 0x7fa49019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.886+0000 7fa49511b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4901055b0 0x7fa4901986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.886+0000 7fa49511b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa490198e90 con 0x7fa4900ff760 2023-09-10T00:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.886+0000 7fa49511b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa490199000 con 0x7fa4901055b0 2023-09-10T00:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.886+0000 7fa49511b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4901991c0 con 0x7fa490100120 2023-09-10T00:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.886+0000 7fa48ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4900ff760 0x7fa49019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.886+0000 7fa48ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4900ff760 0x7fa49019e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51874/0 (socket says 172.21.15.44:51874) 2023-09-10T00:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.886+0000 7fa48ed9d700 1 -- 172.21.15.44:0/3730104438 learned_addr learned my addr 172.21.15.44:0/3730104438 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.886+0000 7fa48e59c700 1 --2- 172.21.15.44:0/3730104438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa490100120 0x7fa49019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.886+0000 7fa48f59e700 1 --2- 172.21.15.44:0/3730104438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4901055b0 0x7fa4901986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.887+0000 7fa48e59c700 1 -- 172.21.15.44:0/3730104438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4901055b0 msgr2=0x7fa4901986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.887+0000 7fa48e59c700 1 --2- 172.21.15.44:0/3730104438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4901055b0 0x7fa4901986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.887+0000 7fa48e59c700 1 -- 172.21.15.44:0/3730104438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4900ff760 msgr2=0x7fa49019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.887+0000 7fa48e59c700 1 --2- 172.21.15.44:0/3730104438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4900ff760 0x7fa49019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.887+0000 7fa48e59c700 1 -- 172.21.15.44:0/3730104438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4800095f0 con 0x7fa490100120 2023-09-10T00:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.887+0000 7fa48ed9d700 1 --2- 172.21.15.44:0/3730104438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4900ff760 0x7fa49019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.887+0000 7fa48e59c700 1 --2- 172.21.15.44:0/3730104438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa490100120 0x7fa49019ee30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa48400e9f0 tx=0x7fa48400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.887+0000 7fa477fff700 1 -- 172.21.15.44:0/3730104438 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48400c9c0 con 0x7fa490100120 2023-09-10T00:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.887+0000 7fa49511b700 1 -- 172.21.15.44:0/3730104438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4901994a0 con 0x7fa490100120 2023-09-10T00:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.887+0000 7fa477fff700 1 -- 172.21.15.44:0/3730104438 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa484010430 con 0x7fa490100120 2023-09-10T00:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.887+0000 7fa477fff700 1 -- 172.21.15.44:0/3730104438 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa484018680 con 0x7fa490100120 2023-09-10T00:49:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.887+0000 7fa49511b700 1 -- 172.21.15.44:0/3730104438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4901a5870 con 0x7fa490100120 2023-09-10T00:49:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.889+0000 7fa477fff700 1 -- 172.21.15.44:0/3730104438 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa4840105a0 con 0x7fa490100120 2023-09-10T00:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.890+0000 7fa477fff700 1 --2- 172.21.15.44:0/3730104438 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4780a07b0 0x7fa4780a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.890+0000 7fa48ed9d700 1 --2- 172.21.15.44:0/3730104438 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4780a07b0 0x7fa4780a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.890+0000 7fa477fff700 1 -- 172.21.15.44:0/3730104438 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa484014070 con 0x7fa490100120 2023-09-10T00:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.891+0000 7fa48ed9d700 1 --2- 172.21.15.44:0/3730104438 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4780a07b0 0x7fa4780a2c70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa480000c00 tx=0x7fa48001a040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.891+0000 7fa4a12e1b80 1 Processor -- start 2023-09-10T00:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.891+0000 7fa4a12e1b80 1 -- start start 2023-09-10T00:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.891+0000 7fa4a12e1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 0x55e5476db580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.891+0000 7fa4a12e1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5300 0x55e5476d5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.891+0000 7fa4a12e1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5cc0 0x55e5476e1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.891+0000 7fa4a12e1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54762e6d0 con 0x55e5476d5300 2023-09-10T00:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.891+0000 7fa4a12e1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54762bff0 con 0x55e5476d5cc0 2023-09-10T00:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.891+0000 7fa4a12e1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5476278e0 con 0x55e5476db1a0 2023-09-10T00:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.891+0000 7fa48e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5300 0x55e5476d5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.891+0000 7fa48e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5300 0x55e5476d5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51884/0 (socket says 172.21.15.44:51884) 2023-09-10T00:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.891+0000 7fa48e59c700 1 -- 172.21.15.44:0/1862253782 learned_addr learned my addr 172.21.15.44:0/1862253782 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.891+0000 7fa48ed9d700 1 --2- 172.21.15.44:0/1862253782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 0x55e5476db580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.891+0000 7fa48f59e700 1 --2- 172.21.15.44:0/1862253782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5cc0 0x55e5476e1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.892+0000 7fa48e59c700 1 -- 172.21.15.44:0/1862253782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 msgr2=0x55e5476db580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.892+0000 7fa48e59c700 1 --2- 172.21.15.44:0/1862253782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 0x55e5476db580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.892+0000 7fa48e59c700 1 -- 172.21.15.44:0/1862253782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5cc0 msgr2=0x55e5476e1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.892+0000 7fa48e59c700 1 --2- 172.21.15.44:0/1862253782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5cc0 0x55e5476e1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.892+0000 7fa48e59c700 1 -- 172.21.15.44:0/1862253782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e547453d80 con 0x55e5476d5300 2023-09-10T00:49:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.892+0000 7fa48e59c700 1 --2- 172.21.15.44:0/1862253782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5300 0x55e5476d5780 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa484095a80 tx=0x7fa4840990c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40628d288ab9cfd9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.892+0000 7fa4757fa700 1 -- 172.21.15.44:0/1862253782 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4840acea0 con 0x55e5476d5300 2023-09-10T00:49:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.892+0000 7fa4757fa700 1 -- 172.21.15.44:0/1862253782 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa484099c80 con 0x55e5476d5300 2023-09-10T00:49:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.892+0000 7fa4757fa700 1 -- 172.21.15.44:0/1862253782 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4840cc1b0 con 0x55e5476d5300 2023-09-10T00:49:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.893+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/1862253782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5300 msgr2=0x55e5476d5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.893+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/1862253782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5300 0x55e5476d5780 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa484095a80 tx=0x7fa4840990c0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.893+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/1862253782 shutdown_connections 2023-09-10T00:49:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.893+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/1862253782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5cc0 0x55e5476e1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.893+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/1862253782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 0x55e5476db580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.893+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/1862253782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5300 0x55e5476d5780 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.893+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/1862253782 >> 172.21.15.44:0/1862253782 conn(0x55e54764a030 msgr2=0x55e5476d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.893+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/1862253782 shutdown_connections 2023-09-10T00:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.893+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/1862253782 wait complete. 2023-09-10T00:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.893+0000 7fa4a12e1b80 1 -- start start 2023-09-10T00:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.893+0000 7fa4a12e1b80 1 Processor -- start 2023-09-10T00:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa4a12e1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5300 0x55e5476e1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa4a12e1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476df090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa4a12e1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 0x55e5476df6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa4a12e1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5475c5b10 con 0x55e5476d5cc0 2023-09-10T00:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa4a12e1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5475c5810 con 0x55e5476d5300 2023-09-10T00:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa4a12e1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5474a98c0 con 0x55e5476db1a0 2023-09-10T00:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa4a12e1b80 10 client.0 _ll_register_callbacks cb 0x55e54750fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa48e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476df090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa48e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476df090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51890/0 (socket says 172.21.15.44:51890) 2023-09-10T00:49:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa48ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5300 0x55e5476e1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa48e59c700 1 -- 172.21.15.44:0/812939948 learned_addr learned my addr 172.21.15.44:0/812939948 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa48f59e700 1 --2- 172.21.15.44:0/812939948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 0x55e5476df6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa48e59c700 1 -- 172.21.15.44:0/812939948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 msgr2=0x55e5476df6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa48e59c700 1 --2- 172.21.15.44:0/812939948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 0x55e5476df6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa48e59c700 1 -- 172.21.15.44:0/812939948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5300 msgr2=0x55e5476e1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa48e59c700 1 --2- 172.21.15.44:0/812939948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5300 0x55e5476e1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa48e59c700 1 -- 172.21.15.44:0/812939948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4901a5870 con 0x55e5476d5cc0 2023-09-10T00:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa48e59c700 1 --2- 172.21.15.44:0/812939948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476df090 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa484095ff0 tx=0x7fa4840c97c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4c7bbc916f4361 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa472ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:49:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa472ff5700 1 -- 172.21.15.44:0/812939948 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4840b5070 con 0x55e5476d5cc0 2023-09-10T00:49:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa472ff5700 1 -- 172.21.15.44:0/812939948 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa484095040 con 0x55e5476d5cc0 2023-09-10T00:49:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.894+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/812939948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e54757f490 con 0x55e5476d5cc0 2023-09-10T00:49:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.895+0000 7fa472ff5700 1 -- 172.21.15.44:0/812939948 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4840acea0 con 0x55e5476d5cc0 2023-09-10T00:49:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.895+0000 7fa4a12e1b80 10 client.17112 fetch_fsmap learned FSMap version 52 2023-09-10T00:49:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.895+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/812939948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55e54743cc00 con 0x55e5476d5cc0 2023-09-10T00:49:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.895+0000 7fa472ff5700 1 -- 172.21.15.44:0/812939948 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e54757f490 con 0x55e5476d5cc0 2023-09-10T00:49:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.895+0000 7fa472ff5700 1 -- 172.21.15.44:0/812939948 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 52) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa4840651a0 con 0x55e5476d5cc0 2023-09-10T00:49:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.895+0000 7fa4a12e1b80 10 client.17112 fetch_fsmap finished waiting for FSMap version 52 2023-09-10T00:49:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.895+0000 7fa4a12e1b80 10 client.17112 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:49:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.895+0000 7fa4a12e1b80 20 client.17112 populate_metadata read hostname 'smithi044' 2023-09-10T00:49:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.895+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/812939948 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e5476eef30 0x55e5476f1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.895+0000 7fa4a12e1b80 4 client.17112 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:49:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.895+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/812939948 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e54763c840 con 0x55e5476eef30 2023-09-10T00:49:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.895+0000 7fa48ed9d700 1 --2- 172.21.15.44:0/812939948 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e5476eef30 0x55e5476f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.896+0000 7fa472ff5700 10 client.17112 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.896+0000 7fa48ed9d700 1 --2- 172.21.15.44:0/812939948 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e5476eef30 0x55e5476f1350 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6ad90ea5f574ffb server_cookie=51489b422671010d in_seq=0 out_seq=0 2023-09-10T00:49:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.896+0000 7fa472ff5700 1 -- 172.21.15.44:0/812939948 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e54763c840 con 0x55e5476eef30 2023-09-10T00:49:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.896+0000 7fa472ff5700 10 client.17112 handle_command_reply: tid=0 2023-09-10T00:49:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.896+0000 7fa4a12e1b80 1 client.17112 shutdown 2023-09-10T00:49:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.896+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/812939948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 msgr2=0x55e5476df090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.896+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/812939948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476df090 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa484095ff0 tx=0x7fa4840c97c0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.896+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/812939948 shutdown_connections 2023-09-10T00:49:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.897+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/812939948 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e5476eef30 0x55e5476f1350 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.897+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/812939948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 0x55e5476df6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.897+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/812939948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5300 0x55e5476e1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.897+0000 7fa472ff5700 0 client.17112 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.897+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/812939948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476df090 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.897+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/812939948 >> 172.21.15.44:0/812939948 conn(0x55e54764a030 msgr2=0x55e5476cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.897+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/812939948 shutdown_connections 2023-09-10T00:49:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.897+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/812939948 wait complete. 2023-09-10T00:49:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.897+0000 7fa4a12e1b80 20 client.17112 trim_cache size 0 max 16384 2023-09-10T00:49:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.913+0000 7fa4a12e1b80 1 Processor -- start 2023-09-10T00:49:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.913+0000 7fa4a12e1b80 1 -- start start 2023-09-10T00:49:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.913+0000 7fa4a12e1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5300 0x55e5476d9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.913+0000 7fa4a12e1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476d9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.913+0000 7fa4a12e1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 0x55e5476cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.913+0000 7fa4a12e1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5476ca8a0 con 0x55e5476d5cc0 2023-09-10T00:49:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.913+0000 7fa4a12e1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5476ccfc0 con 0x55e5476d5300 2023-09-10T00:49:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.913+0000 7fa4a12e1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5476dade0 con 0x55e5476db1a0 2023-09-10T00:49:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.913+0000 7fa48e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476d9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.913+0000 7fa48e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476d9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51898/0 (socket says 172.21.15.44:51898) 2023-09-10T00:49:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.913+0000 7fa48e59c700 1 -- 172.21.15.44:0/3817582416 learned_addr learned my addr 172.21.15.44:0/3817582416 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.913+0000 7fa48ed9d700 1 --2- 172.21.15.44:0/3817582416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5300 0x55e5476d9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.913+0000 7fa48f59e700 1 --2- 172.21.15.44:0/3817582416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 0x55e5476cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.914+0000 7fa48e59c700 1 -- 172.21.15.44:0/3817582416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 msgr2=0x55e5476cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.914+0000 7fa48e59c700 1 --2- 172.21.15.44:0/3817582416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 0x55e5476cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.914+0000 7fa48e59c700 1 -- 172.21.15.44:0/3817582416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5300 msgr2=0x55e5476d9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.914+0000 7fa48e59c700 1 --2- 172.21.15.44:0/3817582416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5300 0x55e5476d9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.914+0000 7fa48e59c700 1 -- 172.21.15.44:0/3817582416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e54747a1f0 con 0x55e5476d5cc0 2023-09-10T00:49:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.914+0000 7fa48e59c700 1 --2- 172.21.15.44:0/3817582416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476d9690 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa484095a80 tx=0x7fa484010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d623b3322dad52 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.914+0000 7fa475ffb700 1 -- 172.21.15.44:0/3817582416 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4840acea0 con 0x55e5476d5cc0 2023-09-10T00:49:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.914+0000 7fa475ffb700 1 -- 172.21.15.44:0/3817582416 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa484084250 con 0x55e5476d5cc0 2023-09-10T00:49:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.914+0000 7fa475ffb700 1 -- 172.21.15.44:0/3817582416 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa484098b10 con 0x55e5476d5cc0 2023-09-10T00:49:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/3817582416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 msgr2=0x55e5476d9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/3817582416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476d9690 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa484095a80 tx=0x7fa484010b80 comp rx=0 tx=0).stop 2023-09-10T00:49:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/3817582416 shutdown_connections 2023-09-10T00:49:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/3817582416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 0x55e5476cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/3817582416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5300 0x55e5476d9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/3817582416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476d9690 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/3817582416 >> 172.21.15.44:0/3817582416 conn(0x55e54764a030 msgr2=0x55e5476da0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/3817582416 shutdown_connections 2023-09-10T00:49:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/3817582416 wait complete. 2023-09-10T00:49:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 -- start start 2023-09-10T00:49:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 Processor -- start 2023-09-10T00:49:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5300 0x55e5476e4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476e4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 0x55e547741d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5476e5250 con 0x55e5476d5cc0 2023-09-10T00:49:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5477422a0 con 0x55e5476d5300 2023-09-10T00:49:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5477423e0 con 0x55e5476db1a0 2023-09-10T00:49:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.915+0000 7fa4a12e1b80 10 client.0 _ll_register_callbacks cb 0x7fa488009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:49:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa48e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476e4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa48ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5300 0x55e5476e4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa48f59e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 0x55e547741d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa48e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476e4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51904/0 (socket says 172.21.15.44:51904) 2023-09-10T00:49:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa48e59c700 1 -- 172.21.15.44:0/4026911653 learned_addr learned my addr 172.21.15.44:0/4026911653 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa48e59c700 1 -- 172.21.15.44:0/4026911653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 msgr2=0x55e547741d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa48e59c700 1 --2- 172.21.15.44:0/4026911653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 0x55e547741d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa48e59c700 1 -- 172.21.15.44:0/4026911653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5300 msgr2=0x55e5476e4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa48e59c700 1 --2- 172.21.15.44:0/4026911653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5300 0x55e5476e4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa48e59c700 1 -- 172.21.15.44:0/4026911653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e54743cc00 con 0x55e5476d5cc0 2023-09-10T00:49:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa48e59c700 1 --2- 172.21.15.44:0/4026911653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476e4940 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa484095ff0 tx=0x7fa4840653d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b80d0c77e48356 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa472ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:49:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa472ff5700 1 -- 172.21.15.44:0/4026911653 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4840b5020 con 0x55e5476d5cc0 2023-09-10T00:49:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa472ff5700 1 -- 172.21.15.44:0/4026911653 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa484095040 con 0x55e5476d5cc0 2023-09-10T00:49:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/4026911653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5476d65c0 con 0x55e5476d5cc0 2023-09-10T00:49:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa472ff5700 1 -- 172.21.15.44:0/4026911653 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4840acea0 con 0x55e5476d5cc0 2023-09-10T00:49:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa4a12e1b80 10 client.17118 fetch_fsmap learned FSMap version 52 2023-09-10T00:49:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.916+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/4026911653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55e547452250 con 0x55e5476d5cc0 2023-09-10T00:49:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.917+0000 7fa472ff5700 1 -- 172.21.15.44:0/4026911653 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e5476d65c0 con 0x55e5476d5cc0 2023-09-10T00:49:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.917+0000 7fa472ff5700 1 -- 172.21.15.44:0/4026911653 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 52) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa4840b1b80 con 0x55e5476d5cc0 2023-09-10T00:49:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.917+0000 7fa4a12e1b80 10 client.17118 fetch_fsmap finished waiting for FSMap version 52 2023-09-10T00:49:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.917+0000 7fa4a12e1b80 10 client.17118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:49:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.917+0000 7fa4a12e1b80 20 client.17118 populate_metadata read hostname 'smithi044' 2023-09-10T00:49:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.917+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/4026911653 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e5477435b0 0x55e5477459d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.917+0000 7fa4a12e1b80 4 client.17118 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:49:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.917+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/4026911653 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fa484099c80 con 0x55e5477435b0 2023-09-10T00:49:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.917+0000 7fa48ed9d700 1 --2- 172.21.15.44:0/4026911653 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e5477435b0 0x55e5477459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.918+0000 7fa472ff5700 10 client.17118 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.918+0000 7fa48ed9d700 1 --2- 172.21.15.44:0/4026911653 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e5477435b0 0x55e5477459d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2733e154e14e0b6f server_cookie=69e464a57b877d34 in_seq=0 out_seq=0 2023-09-10T00:49:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.944+0000 7fa472ff5700 1 -- 172.21.15.44:0/4026911653 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fa484099c80 con 0x55e5477435b0 2023-09-10T00:49:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.944+0000 7fa472ff5700 10 client.17118 handle_command_reply: tid=0 2023-09-10T00:49:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.944+0000 7fa4a12e1b80 1 client.17118 shutdown 2023-09-10T00:49:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.944+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/4026911653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 msgr2=0x55e5476e4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.944+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/4026911653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476e4940 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa484095ff0 tx=0x7fa4840653d0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.944+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/4026911653 shutdown_connections 2023-09-10T00:49:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.944+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/4026911653 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e5477435b0 0x55e5477459d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.944+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/4026911653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5476db1a0 0x55e547741d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.944+0000 7fa472ff5700 0 client.17118 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.944+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/4026911653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5476d5300 0x55e5476e4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.944+0000 7fa4a12e1b80 1 --2- 172.21.15.44:0/4026911653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5476d5cc0 0x55e5476e4940 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.944+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/4026911653 >> 172.21.15.44:0/4026911653 conn(0x55e54764a030 msgr2=0x55e5476da0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.944+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/4026911653 shutdown_connections 2023-09-10T00:49:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.944+0000 7fa4a12e1b80 1 -- 172.21.15.44:0/4026911653 wait complete. 2023-09-10T00:49:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.944+0000 7fa4a12e1b80 20 client.17118 trim_cache size 0 max 16384 2023-09-10T00:49:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:49:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:49:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:49:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:49:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.945+0000 7fa49511b700 1 -- 172.21.15.44:0/3730104438 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4780a07b0 msgr2=0x7fa4780a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.945+0000 7fa49511b700 1 --2- 172.21.15.44:0/3730104438 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4780a07b0 0x7fa4780a2c70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa480000c00 tx=0x7fa48001a040 comp rx=0 tx=0).stop 2023-09-10T00:49:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.945+0000 7fa49511b700 1 -- 172.21.15.44:0/3730104438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa490100120 msgr2=0x7fa49019ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.945+0000 7fa49511b700 1 --2- 172.21.15.44:0/3730104438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa490100120 0x7fa49019ee30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa48400e9f0 tx=0x7fa48400ed00 comp rx=0 tx=0).stop 2023-09-10T00:49:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.945+0000 7fa49511b700 1 -- 172.21.15.44:0/3730104438 shutdown_connections 2023-09-10T00:49:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.945+0000 7fa49511b700 1 --2- 172.21.15.44:0/3730104438 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4780a07b0 0x7fa4780a2c70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.945+0000 7fa49511b700 1 --2- 172.21.15.44:0/3730104438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4901055b0 0x7fa4901986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.945+0000 7fa49511b700 1 --2- 172.21.15.44:0/3730104438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4900ff760 0x7fa49019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.945+0000 7fa49511b700 1 --2- 172.21.15.44:0/3730104438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa490100120 0x7fa49019ee30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.945+0000 7fa49511b700 1 -- 172.21.15.44:0/3730104438 >> 172.21.15.44:0/3730104438 conn(0x7fa490074f30 msgr2=0x7fa4900679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.945+0000 7fa49511b700 1 -- 172.21.15.44:0/3730104438 shutdown_connections 2023-09-10T00:49:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:02.945+0000 7fa49511b700 1 -- 172.21.15.44:0/3730104438 wait complete. 2023-09-10T00:49:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:02 smithi131 ceph-mon[103713]: pgmap v258: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2023-09-10T00:49:03.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:02 smithi044 ceph-mon[98676]: pgmap v258: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2023-09-10T00:49:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:02 smithi099 ceph-mon[102692]: pgmap v258: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2023-09-10T00:49:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:04 smithi131 ceph-mon[103713]: pgmap v259: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.6 MiB/s wr, 290 op/s 2023-09-10T00:49:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:04 smithi131 ceph-mon[103713]: osdmap e97: 12 total, 12 up, 12 in 2023-09-10T00:49:05.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:04 smithi044 ceph-mon[98676]: pgmap v259: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.6 MiB/s wr, 290 op/s 2023-09-10T00:49:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:04 smithi044 ceph-mon[98676]: osdmap e97: 12 total, 12 up, 12 in 2023-09-10T00:49:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:04 smithi099 ceph-mon[102692]: pgmap v259: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.6 MiB/s wr, 290 op/s 2023-09-10T00:49:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:04 smithi099 ceph-mon[102692]: osdmap e97: 12 total, 12 up, 12 in 2023-09-10T00:49:07.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:06 smithi131 ceph-mon[103713]: pgmap v261: 129 pgs: 129 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.1 MiB/s wr, 362 op/s 2023-09-10T00:49:07.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:06 smithi044 ceph-mon[98676]: pgmap v261: 129 pgs: 129 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.1 MiB/s wr, 362 op/s 2023-09-10T00:49:07.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:06 smithi099 ceph-mon[102692]: pgmap v261: 129 pgs: 129 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.1 MiB/s wr, 362 op/s 2023-09-10T00:49:09.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:08 smithi131 ceph-mon[103713]: pgmap v262: 129 pgs: 129 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.1 MiB/s wr, 362 op/s 2023-09-10T00:49:09.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:08 smithi044 ceph-mon[98676]: pgmap v262: 129 pgs: 129 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.1 MiB/s wr, 362 op/s 2023-09-10T00:49:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:08 smithi099 ceph-mon[102692]: pgmap v262: 129 pgs: 129 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.1 MiB/s wr, 362 op/s 2023-09-10T00:49:09.521 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:49:09.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.669+0000 7fd795b9f700 1 -- 172.21.15.44:0/1169607307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7900ffeb0 msgr2=0x7fd790103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:09.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.669+0000 7fd795b9f700 1 --2- 172.21.15.44:0/1169607307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7900ffeb0 0x7fd790103ef0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009990 tx=0x7fd77c009ca0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.669+0000 7fd795b9f700 1 -- 172.21.15.44:0/1169607307 shutdown_connections 2023-09-10T00:49:09.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.669+0000 7fd795b9f700 1 --2- 172.21.15.44:0/1169607307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd790104620 0x7fd79010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.669+0000 7fd795b9f700 1 --2- 172.21.15.44:0/1169607307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7900ff500 0x7fd7900ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.669+0000 7fd795b9f700 1 --2- 172.21.15.44:0/1169607307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7900ffeb0 0x7fd790103ef0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.669+0000 7fd795b9f700 1 -- 172.21.15.44:0/1169607307 >> 172.21.15.44:0/1169607307 conn(0x7fd7900fad80 msgr2=0x7fd7900fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:09.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.669+0000 7fd795b9f700 1 -- 172.21.15.44:0/1169607307 shutdown_connections 2023-09-10T00:49:09.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.669+0000 7fd795b9f700 1 -- 172.21.15.44:0/1169607307 wait complete. 2023-09-10T00:49:09.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.670+0000 7fd795b9f700 1 Processor -- start 2023-09-10T00:49:09.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.670+0000 7fd795b9f700 1 -- start start 2023-09-10T00:49:09.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.670+0000 7fd795b9f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7900ff500 0x7fd790195240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.670+0000 7fd795b9f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd790104620 0x7fd790195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.670+0000 7fd795b9f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd790199be0 0x7fd79019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.670+0000 7fd795b9f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79019a660 con 0x7fd790199be0 2023-09-10T00:49:09.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.670+0000 7fd795b9f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79019a7d0 con 0x7fd790104620 2023-09-10T00:49:09.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.671+0000 7fd795b9f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79019a990 con 0x7fd7900ff500 2023-09-10T00:49:09.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.671+0000 7fd794b9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7900ff500 0x7fd790195240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.671+0000 7fd79539e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd790199be0 0x7fd79019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.671+0000 7fd794b9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7900ff500 0x7fd790195240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58798/0 (socket says 172.21.15.44:58798) 2023-09-10T00:49:09.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.671+0000 7fd794b9d700 1 -- 172.21.15.44:0/578475262 learned_addr learned my addr 172.21.15.44:0/578475262 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:09.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.671+0000 7fd78ffff700 1 --2- 172.21.15.44:0/578475262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd790104620 0x7fd790195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.671+0000 7fd794b9d700 1 -- 172.21.15.44:0/578475262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd790104620 msgr2=0x7fd790195780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:09.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.671+0000 7fd794b9d700 1 --2- 172.21.15.44:0/578475262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd790104620 0x7fd790195780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.671+0000 7fd794b9d700 1 -- 172.21.15.44:0/578475262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd790199be0 msgr2=0x7fd79019a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:09.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.671+0000 7fd794b9d700 1 --2- 172.21.15.44:0/578475262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd790199be0 0x7fd79019a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.671+0000 7fd794b9d700 1 -- 172.21.15.44:0/578475262 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd784009520 con 0x7fd7900ff500 2023-09-10T00:49:09.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.671+0000 7fd79539e700 1 --2- 172.21.15.44:0/578475262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd790199be0 0x7fd79019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:09.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.671+0000 7fd794b9d700 1 --2- 172.21.15.44:0/578475262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7900ff500 0x7fd790195240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd78400eb30 tx=0x7fd78400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:09.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.671+0000 7fd78dffb700 1 -- 172.21.15.44:0/578475262 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78400dbe0 con 0x7fd7900ff500 2023-09-10T00:49:09.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.672+0000 7fd78dffb700 1 -- 172.21.15.44:0/578475262 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7840044b0 con 0x7fd7900ff500 2023-09-10T00:49:09.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.672+0000 7fd795b9f700 1 -- 172.21.15.44:0/578475262 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77c0095f0 con 0x7fd7900ff500 2023-09-10T00:49:09.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.672+0000 7fd78dffb700 1 -- 172.21.15.44:0/578475262 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784003d50 con 0x7fd7900ff500 2023-09-10T00:49:09.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.672+0000 7fd795b9f700 1 -- 172.21.15.44:0/578475262 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd790068cc0 con 0x7fd7900ff500 2023-09-10T00:49:09.684 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.674+0000 7fd78dffb700 1 -- 172.21.15.44:0/578475262 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fd784003eb0 con 0x7fd7900ff500 2023-09-10T00:49:09.685 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.675+0000 7fd78dffb700 1 --2- 172.21.15.44:0/578475262 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7800a08d0 0x7fd7800a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.685 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.675+0000 7fd78dffb700 1 -- 172.21.15.44:0/578475262 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fd7840cbec0 con 0x7fd7900ff500 2023-09-10T00:49:09.685 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.675+0000 7fd78ffff700 1 --2- 172.21.15.44:0/578475262 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7800a08d0 0x7fd7800a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.685 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd7a2fc7b80 1 Processor -- start 2023-09-10T00:49:09.685 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd7a2fc7b80 1 -- start start 2023-09-10T00:49:09.686 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd78ffff700 1 --2- 172.21.15.44:0/578475262 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7800a08d0 0x7fd7800a2d90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00b2e0 tx=0x7fd77c005040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:09.686 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd7a2fc7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874d5860 0x55ae874d5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.686 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd7a2fc7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d6180 0x55ae874e0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.686 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd7a2fc7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874e1030 0x55ae874e3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.686 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd7a2fc7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8742f730 con 0x55ae874d5860 2023-09-10T00:49:09.687 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd7a2fc7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8742d050 con 0x55ae874d6180 2023-09-10T00:49:09.687 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd7a2fc7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae87428940 con 0x55ae874e1030 2023-09-10T00:49:09.687 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd794b9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874d5860 0x55ae874d5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.687 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd794b9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874d5860 0x55ae874d5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51924/0 (socket says 172.21.15.44:51924) 2023-09-10T00:49:09.687 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd78ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d6180 0x55ae874e0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.687 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd794b9d700 1 -- 172.21.15.44:0/3852374879 learned_addr learned my addr 172.21.15.44:0/3852374879 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:09.688 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd79539e700 1 --2- 172.21.15.44:0/3852374879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874e1030 0x55ae874e3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.688 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd794b9d700 1 -- 172.21.15.44:0/3852374879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874e1030 msgr2=0x55ae874e3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:09.688 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd794b9d700 1 --2- 172.21.15.44:0/3852374879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874e1030 0x55ae874e3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.688 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd794b9d700 1 -- 172.21.15.44:0/3852374879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d6180 msgr2=0x55ae874e0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:09.688 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd794b9d700 1 --2- 172.21.15.44:0/3852374879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d6180 0x55ae874e0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd794b9d700 1 -- 172.21.15.44:0/3852374879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae87254de0 con 0x55ae874d5860 2023-09-10T00:49:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd78ffff700 1 --2- 172.21.15.44:0/3852374879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d6180 0x55ae874e0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.676+0000 7fd79539e700 1 --2- 172.21.15.44:0/3852374879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874e1030 0x55ae874e3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:49:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.677+0000 7fd794b9d700 1 --2- 172.21.15.44:0/3852374879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874d5860 0x55ae874d5c40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd7840954b0 tx=0x7fd7840989e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6156311bbe0cf089 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.677+0000 7fd77affd700 1 -- 172.21.15.44:0/3852374879 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7840996b0 con 0x55ae874d5860 2023-09-10T00:49:09.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.677+0000 7fd77affd700 1 -- 172.21.15.44:0/3852374879 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd784098fb0 con 0x55ae874d5860 2023-09-10T00:49:09.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.677+0000 7fd77affd700 1 -- 172.21.15.44:0/3852374879 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784099470 con 0x55ae874d5860 2023-09-10T00:49:09.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.678+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3852374879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874d5860 msgr2=0x55ae874d5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:09.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.678+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/3852374879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874d5860 0x55ae874d5c40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd7840954b0 tx=0x7fd7840989e0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.678+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3852374879 shutdown_connections 2023-09-10T00:49:09.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.678+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/3852374879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874e1030 0x55ae874e3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.678+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/3852374879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874d5860 0x55ae874d5c40 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.678+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/3852374879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d6180 0x55ae874e0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.678+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3852374879 >> 172.21.15.44:0/3852374879 conn(0x55ae8744b090 msgr2=0x55ae874d1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:09.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.678+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3852374879 shutdown_connections 2023-09-10T00:49:09.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.678+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3852374879 wait complete. 2023-09-10T00:49:09.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.678+0000 7fd7a2fc7b80 1 -- start start 2023-09-10T00:49:09.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.678+0000 7fd7a2fc7b80 1 Processor -- start 2023-09-10T00:49:09.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd7a2fc7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d5860 0x55ae874e0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd7a2fc7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874d6180 0x55ae874ddd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd7a2fc7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874e1030 0x55ae874de4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd7a2fc7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae873c6b70 con 0x55ae874d6180 2023-09-10T00:49:09.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd7a2fc7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae873c6870 con 0x55ae874e1030 2023-09-10T00:49:09.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd7a2fc7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae872aa920 con 0x55ae874d5860 2023-09-10T00:49:09.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd7a2fc7b80 10 client.0 _ll_register_callbacks cb 0x55ae87310a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:49:09.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd78ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874d6180 0x55ae874ddd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd794b9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d5860 0x55ae874e0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd78ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874d6180 0x55ae874ddd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51932/0 (socket says 172.21.15.44:51932) 2023-09-10T00:49:09.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd78ffff700 1 -- 172.21.15.44:0/3499970415 learned_addr learned my addr 172.21.15.44:0/3499970415 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:09.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd79539e700 1 --2- 172.21.15.44:0/3499970415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874e1030 0x55ae874de4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd78ffff700 1 -- 172.21.15.44:0/3499970415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d5860 msgr2=0x55ae874e0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:09.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd78ffff700 1 --2- 172.21.15.44:0/3499970415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d5860 0x55ae874e0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd78ffff700 1 -- 172.21.15.44:0/3499970415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874e1030 msgr2=0x55ae874de4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:09.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd78ffff700 1 --2- 172.21.15.44:0/3499970415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874e1030 0x55ae874de4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd78ffff700 1 -- 172.21.15.44:0/3499970415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79010ca10 con 0x55ae874d6180 2023-09-10T00:49:09.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd78ffff700 1 --2- 172.21.15.44:0/3499970415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874d6180 0x55ae874ddd80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd790069bc0 tx=0x7fd77c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbaba8df170bfccd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:09.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.679+0000 7fd76ffff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:49:09.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.680+0000 7fd76ffff700 1 -- 172.21.15.44:0/3499970415 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c004c40 con 0x55ae874d6180 2023-09-10T00:49:09.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.680+0000 7fd76ffff700 1 -- 172.21.15.44:0/3499970415 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fd77c004da0 con 0x55ae874d6180 2023-09-10T00:49:09.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.680+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3499970415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae873804f0 con 0x55ae874d6180 2023-09-10T00:49:09.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.680+0000 7fd76ffff700 1 -- 172.21.15.44:0/3499970415 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd77c00f460 con 0x55ae874d6180 2023-09-10T00:49:09.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.680+0000 7fd7a2fc7b80 10 client.17130 fetch_fsmap learned FSMap version 52 2023-09-10T00:49:09.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.680+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3499970415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7fd790068cc0 con 0x55ae874d6180 2023-09-10T00:49:09.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.680+0000 7fd76ffff700 1 -- 172.21.15.44:0/3499970415 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ae873804f0 con 0x55ae874d6180 2023-09-10T00:49:09.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.681+0000 7fd76ffff700 1 -- 172.21.15.44:0/3499970415 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 52) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd77c02f070 con 0x55ae874d6180 2023-09-10T00:49:09.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.681+0000 7fd7a2fc7b80 10 client.17130 fetch_fsmap finished waiting for FSMap version 52 2023-09-10T00:49:09.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.681+0000 7fd7a2fc7b80 10 client.17130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:49:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.681+0000 7fd7a2fc7b80 20 client.17130 populate_metadata read hostname 'smithi044' 2023-09-10T00:49:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.681+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/3499970415 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ae874f0b20 0x55ae874f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.681+0000 7fd7a2fc7b80 4 client.17130 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:49:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.681+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3499970415 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae8743d8a0 con 0x55ae874f0b20 2023-09-10T00:49:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.681+0000 7fd794b9d700 1 --2- 172.21.15.44:0/3499970415 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ae874f0b20 0x55ae874f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.681+0000 7fd76ffff700 10 client.17130 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:09.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.681+0000 7fd794b9d700 1 --2- 172.21.15.44:0/3499970415 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ae874f0b20 0x55ae874f2f40 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b52dfc92678cc0e2 server_cookie=ba55716795bd8abd in_seq=0 out_seq=0 2023-09-10T00:49:09.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.682+0000 7fd76ffff700 1 -- 172.21.15.44:0/3499970415 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ae8743d8a0 con 0x55ae874f0b20 2023-09-10T00:49:09.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.682+0000 7fd76ffff700 10 client.17130 handle_command_reply: tid=0 2023-09-10T00:49:09.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.682+0000 7fd7a2fc7b80 1 client.17130 shutdown 2023-09-10T00:49:09.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.682+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3499970415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874d6180 msgr2=0x55ae874ddd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:09.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.682+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/3499970415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874d6180 0x55ae874ddd80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd790069bc0 tx=0x7fd77c009ca0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.683+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3499970415 shutdown_connections 2023-09-10T00:49:09.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.683+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/3499970415 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ae874f0b20 0x55ae874f2f40 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.683+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/3499970415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874e1030 0x55ae874de4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.683+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/3499970415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d5860 0x55ae874e0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.683+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/3499970415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874d6180 0x55ae874ddd80 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.683+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3499970415 >> 172.21.15.44:0/3499970415 conn(0x55ae8744b090 msgr2=0x55ae874d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:09.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.683+0000 7fd76ffff700 0 client.17130 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:09.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.683+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3499970415 shutdown_connections 2023-09-10T00:49:09.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.683+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3499970415 wait complete. 2023-09-10T00:49:09.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.683+0000 7fd7a2fc7b80 20 client.17130 trim_cache size 0 max 16384 2023-09-10T00:49:09.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.706+0000 7fd7a2fc7b80 1 Processor -- start 2023-09-10T00:49:09.707 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.706+0000 7fd7a2fc7b80 1 -- start start 2023-09-10T00:49:09.708 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.706+0000 7fd7a2fc7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d5860 0x55ae874dd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.708 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.706+0000 7fd7a2fc7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d6180 0x55ae874ddaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.708 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.706+0000 7fd7a2fc7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874e1030 0x55ae874db030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.708 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.706+0000 7fd7a2fc7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae874d6560 con 0x55ae874e1030 2023-09-10T00:49:09.708 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.706+0000 7fd7a2fc7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae874d76f0 con 0x55ae874d5860 2023-09-10T00:49:09.708 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.706+0000 7fd7a2fc7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae874ee110 con 0x55ae874d6180 2023-09-10T00:49:09.709 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.706+0000 7fd79539e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874e1030 0x55ae874db030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.709 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.706+0000 7fd78ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d6180 0x55ae874ddaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.709 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.706+0000 7fd794b9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d5860 0x55ae874dd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.709 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.706+0000 7fd78ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d6180 0x55ae874ddaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58822/0 (socket says 172.21.15.44:58822) 2023-09-10T00:49:09.709 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.706+0000 7fd79539e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874e1030 0x55ae874db030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51942/0 (socket says 172.21.15.44:51942) 2023-09-10T00:49:09.710 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.706+0000 7fd794b9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d5860 0x55ae874dd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53678/0 (socket says 172.21.15.44:53678) 2023-09-10T00:49:09.710 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.706+0000 7fd78ffff700 1 -- 172.21.15.44:0/1691144694 learned_addr learned my addr 172.21.15.44:0/1691144694 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:09.710 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.707+0000 7fd79539e700 1 -- 172.21.15.44:0/1691144694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d6180 msgr2=0x55ae874ddaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:09.710 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.707+0000 7fd79539e700 1 --2- 172.21.15.44:0/1691144694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d6180 0x55ae874ddaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.710 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.707+0000 7fd79539e700 1 -- 172.21.15.44:0/1691144694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d5860 msgr2=0x55ae874dd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:09.711 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.707+0000 7fd79539e700 1 --2- 172.21.15.44:0/1691144694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d5860 0x55ae874dd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.711 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.707+0000 7fd79539e700 1 -- 172.21.15.44:0/1691144694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae8723dc60 con 0x55ae874e1030 2023-09-10T00:49:09.711 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.707+0000 7fd79539e700 1 --2- 172.21.15.44:0/1691144694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874e1030 0x55ae874db030 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd788010e40 tx=0x7fd78800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2547fe921e14f5be server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:09.711 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.707+0000 7fd77b7fe700 1 -- 172.21.15.44:0/1691144694 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788015040 con 0x55ae874e1030 2023-09-10T00:49:09.711 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.707+0000 7fd77b7fe700 1 -- 172.21.15.44:0/1691144694 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd788003e90 con 0x55ae874e1030 2023-09-10T00:49:09.712 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.707+0000 7fd77b7fe700 1 -- 172.21.15.44:0/1691144694 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788019ee0 con 0x55ae874e1030 2023-09-10T00:49:09.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.708+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/1691144694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874e1030 msgr2=0x55ae874db030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:09.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.708+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/1691144694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874e1030 0x55ae874db030 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd788010e40 tx=0x7fd78800b040 comp rx=0 tx=0).stop 2023-09-10T00:49:09.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.708+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/1691144694 shutdown_connections 2023-09-10T00:49:09.716 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.708+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/1691144694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874e1030 0x55ae874db030 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.708+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/1691144694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d5860 0x55ae874dd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.708+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/1691144694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d6180 0x55ae874ddaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.708+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/1691144694 >> 172.21.15.44:0/1691144694 conn(0x55ae8744b090 msgr2=0x55ae874d90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:09.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.708+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/1691144694 shutdown_connections 2023-09-10T00:49:09.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.708+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/1691144694 wait complete. 2023-09-10T00:49:09.717 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.708+0000 7fd7a2fc7b80 1 -- start start 2023-09-10T00:49:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.708+0000 7fd7a2fc7b80 1 Processor -- start 2023-09-10T00:49:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd7a2fc7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d5860 0x55ae874e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd7a2fc7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d6180 0x55ae874e6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd7a2fc7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874e1030 0x55ae874e6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd7a2fc7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae874e8100 con 0x55ae874e1030 2023-09-10T00:49:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd7a2fc7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae87544960 con 0x55ae874d5860 2023-09-10T00:49:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd7a2fc7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae87544ad0 con 0x55ae874d6180 2023-09-10T00:49:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd7a2fc7b80 10 client.0 _ll_register_callbacks cb 0x55ae87310a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:49:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd794b9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d5860 0x55ae874e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd794b9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d5860 0x55ae874e6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53684/0 (socket says 172.21.15.44:53684) 2023-09-10T00:49:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd794b9d700 1 -- 172.21.15.44:0/3934700401 learned_addr learned my addr 172.21.15.44:0/3934700401 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd78ffff700 1 --2- 172.21.15.44:0/3934700401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d6180 0x55ae874e6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd79539e700 1 --2- 172.21.15.44:0/3934700401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874e1030 0x55ae874e6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd794b9d700 1 -- 172.21.15.44:0/3934700401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d6180 msgr2=0x55ae874e6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd794b9d700 1 --2- 172.21.15.44:0/3934700401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d6180 0x55ae874e6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd794b9d700 1 -- 172.21.15.44:0/3934700401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874e1030 msgr2=0x55ae874e6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:09.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd794b9d700 1 --2- 172.21.15.44:0/3934700401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874e1030 0x55ae874e6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd794b9d700 1 -- 172.21.15.44:0/3934700401 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd788010ac0 con 0x55ae874d5860 2023-09-10T00:49:09.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd79539e700 1 --2- 172.21.15.44:0/3934700401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874e1030 0x55ae874e6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:49:09.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd78ffff700 1 --2- 172.21.15.44:0/3934700401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d6180 0x55ae874e6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:09.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd794b9d700 1 --2- 172.21.15.44:0/3934700401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d5860 0x55ae874e6100 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd784098f40 tx=0x7fd784004620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9575ac7a6b70f0f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:09.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.709+0000 7fd76ffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:49:09.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.710+0000 7fd76ffff700 1 -- 172.21.15.44:0/3934700401 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784004160 con 0x55ae874d5860 2023-09-10T00:49:09.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.710+0000 7fd76ffff700 1 -- 172.21.15.44:0/3934700401 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fd7840a0f90 con 0x55ae874d5860 2023-09-10T00:49:09.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.710+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3934700401 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd784098fb0 con 0x55ae874d5860 2023-09-10T00:49:09.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.710+0000 7fd76ffff700 1 -- 172.21.15.44:0/3934700401 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7840984a0 con 0x55ae874d5860 2023-09-10T00:49:09.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.710+0000 7fd7a2fc7b80 10 client.26795 fetch_fsmap learned FSMap version 52 2023-09-10T00:49:09.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.710+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3934700401 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55ae8727b250 con 0x55ae874d5860 2023-09-10T00:49:09.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.710+0000 7fd76ffff700 1 -- 172.21.15.44:0/3934700401 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd784098fb0 con 0x55ae874d5860 2023-09-10T00:49:09.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.710+0000 7fd76ffff700 1 -- 172.21.15.44:0/3934700401 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 52) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd7840a0b00 con 0x55ae874d5860 2023-09-10T00:49:09.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.710+0000 7fd7a2fc7b80 10 client.26795 fetch_fsmap finished waiting for FSMap version 52 2023-09-10T00:49:09.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.710+0000 7fd7a2fc7b80 10 client.26795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:49:09.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.710+0000 7fd7a2fc7b80 20 client.26795 populate_metadata read hostname 'smithi044' 2023-09-10T00:49:09.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.710+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/3934700401 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ae875461d0 0x55ae875485f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:09.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.710+0000 7fd7a2fc7b80 4 client.26795 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:49:09.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.710+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3934700401 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ae87421920 con 0x55ae875461d0 2023-09-10T00:49:09.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.710+0000 7fd78ffff700 1 --2- 172.21.15.44:0/3934700401 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ae875461d0 0x55ae875485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:09.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.711+0000 7fd76ffff700 10 client.26795 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:09.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.711+0000 7fd78ffff700 1 --2- 172.21.15.44:0/3934700401 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ae875461d0 0x55ae875485f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=313932f5ff9d52e6 server_cookie=cb4554f1bc85834d in_seq=0 out_seq=0 2023-09-10T00:49:09.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.711+0000 7fd76ffff700 1 -- 172.21.15.44:0/3934700401 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ae87421920 con 0x55ae875461d0 2023-09-10T00:49:09.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.711+0000 7fd76ffff700 10 client.26795 handle_command_reply: tid=0 2023-09-10T00:49:09.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.711+0000 7fd7a2fc7b80 1 client.26795 shutdown 2023-09-10T00:49:09.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.711+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3934700401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d5860 msgr2=0x55ae874e6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:09.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.711+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/3934700401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d5860 0x55ae874e6100 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd784098f40 tx=0x7fd784004620 comp rx=0 tx=0).stop 2023-09-10T00:49:09.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.711+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3934700401 shutdown_connections 2023-09-10T00:49:09.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.711+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/3934700401 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ae875461d0 0x55ae875485f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.711+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/3934700401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae874e1030 0x55ae874e6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.711+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/3934700401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae874d5860 0x55ae874e6100 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.711+0000 7fd7a2fc7b80 1 --2- 172.21.15.44:0/3934700401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae874d6180 0x55ae874e6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.711+0000 7fd76ffff700 0 client.26795 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:09.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.711+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3934700401 >> 172.21.15.44:0/3934700401 conn(0x55ae8744b090 msgr2=0x55ae874d8150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:09.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.711+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3934700401 shutdown_connections 2023-09-10T00:49:09.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.711+0000 7fd7a2fc7b80 1 -- 172.21.15.44:0/3934700401 wait complete. 2023-09-10T00:49:09.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.712+0000 7fd7a2fc7b80 20 client.26795 trim_cache size 0 max 16384 2023-09-10T00:49:09.728 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:49:09.728 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:49:09.728 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:49:09.729 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:49:09.729 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:49:09.729 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:49:09.729 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:49:09.729 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:49:09.730 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:49:09.730 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:49:09.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.712+0000 7fd795b9f700 1 -- 172.21.15.44:0/578475262 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7800a08d0 msgr2=0x7fd7800a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:09.731 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.712+0000 7fd795b9f700 1 --2- 172.21.15.44:0/578475262 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7800a08d0 0x7fd7800a2d90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00b2e0 tx=0x7fd77c005040 comp rx=0 tx=0).stop 2023-09-10T00:49:09.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.712+0000 7fd795b9f700 1 -- 172.21.15.44:0/578475262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7900ff500 msgr2=0x7fd790195240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:09.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.712+0000 7fd795b9f700 1 --2- 172.21.15.44:0/578475262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7900ff500 0x7fd790195240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd78400eb30 tx=0x7fd78400ee40 comp rx=0 tx=0).stop 2023-09-10T00:49:09.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.712+0000 7fd795b9f700 1 -- 172.21.15.44:0/578475262 shutdown_connections 2023-09-10T00:49:09.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.712+0000 7fd795b9f700 1 --2- 172.21.15.44:0/578475262 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7800a08d0 0x7fd7800a2d90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.732 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.712+0000 7fd795b9f700 1 --2- 172.21.15.44:0/578475262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd790199be0 0x7fd79019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.712+0000 7fd795b9f700 1 --2- 172.21.15.44:0/578475262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7900ff500 0x7fd790195240 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.712+0000 7fd795b9f700 1 --2- 172.21.15.44:0/578475262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd790104620 0x7fd790195780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:09.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.712+0000 7fd795b9f700 1 -- 172.21.15.44:0/578475262 >> 172.21.15.44:0/578475262 conn(0x7fd7900fad80 msgr2=0x7fd7900fb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:09.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.712+0000 7fd795b9f700 1 -- 172.21.15.44:0/578475262 shutdown_connections 2023-09-10T00:49:09.733 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:09.712+0000 7fd795b9f700 1 -- 172.21.15.44:0/578475262 wait complete. 2023-09-10T00:49:09.734 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:49:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:10 smithi131 ceph-mon[103713]: pgmap v263: 129 pgs: 129 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.1 MiB/s wr, 362 op/s 2023-09-10T00:49:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:10 smithi131 ceph-mon[103713]: osdmap e98: 12 total, 12 up, 12 in 2023-09-10T00:49:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:10 smithi044 ceph-mon[98676]: pgmap v263: 129 pgs: 129 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.1 MiB/s wr, 362 op/s 2023-09-10T00:49:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:10 smithi044 ceph-mon[98676]: osdmap e98: 12 total, 12 up, 12 in 2023-09-10T00:49:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:10 smithi099 ceph-mon[102692]: pgmap v263: 129 pgs: 129 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.1 MiB/s wr, 362 op/s 2023-09-10T00:49:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:10 smithi099 ceph-mon[102692]: osdmap e98: 12 total, 12 up, 12 in 2023-09-10T00:49:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:12 smithi131 ceph-mon[103713]: pgmap v265: 129 pgs: 129 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 4.0 MiB/s wr, 453 op/s 2023-09-10T00:49:13.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:12 smithi044 ceph-mon[98676]: pgmap v265: 129 pgs: 129 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 4.0 MiB/s wr, 453 op/s 2023-09-10T00:49:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:12 smithi099 ceph-mon[102692]: pgmap v265: 129 pgs: 129 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 4.0 MiB/s wr, 453 op/s 2023-09-10T00:49:13.284 INFO:teuthology.orchestra.run.smithi044.stderr:32000000 of 50000000 tuples (64%) done (elapsed 492.92 s, remaining 277.27 s) 2023-09-10T00:49:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:14 smithi131 ceph-mon[103713]: pgmap v266: 129 pgs: 129 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 3.4 MiB/s wr, 386 op/s 2023-09-10T00:49:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:14 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:15.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:14 smithi044 ceph-mon[98676]: pgmap v266: 129 pgs: 129 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 3.4 MiB/s wr, 386 op/s 2023-09-10T00:49:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:14 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:14 smithi099 ceph-mon[102692]: pgmap v266: 129 pgs: 129 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 3.4 MiB/s wr, 386 op/s 2023-09-10T00:49:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:14 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.097+0000 7fc1c45a7700 1 -- 172.21.15.44:0/2259518990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1bc1055b0 msgr2=0x7fc1bc105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.097+0000 7fc1c45a7700 1 --2- 172.21.15.44:0/2259518990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1bc1055b0 0x7fc1bc105990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80098f0 tx=0x7fc1b8009c00 comp rx=0 tx=0).stop 2023-09-10T00:49:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.097+0000 7fc1c45a7700 1 -- 172.21.15.44:0/2259518990 shutdown_connections 2023-09-10T00:49:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.097+0000 7fc1c45a7700 1 --2- 172.21.15.44:0/2259518990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1bc100120 0x7fc1bc0679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.097+0000 7fc1c45a7700 1 --2- 172.21.15.44:0/2259518990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1bc1055b0 0x7fc1bc105990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.097+0000 7fc1c45a7700 1 --2- 172.21.15.44:0/2259518990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc0ff760 0x7fc1bc0ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.097+0000 7fc1c45a7700 1 -- 172.21.15.44:0/2259518990 >> 172.21.15.44:0/2259518990 conn(0x7fc1bc074f30 msgr2=0x7fc1bc075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.098+0000 7fc1c45a7700 1 -- 172.21.15.44:0/2259518990 shutdown_connections 2023-09-10T00:49:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.098+0000 7fc1c45a7700 1 -- 172.21.15.44:0/2259518990 wait complete. 2023-09-10T00:49:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.098+0000 7fc1c45a7700 1 Processor -- start 2023-09-10T00:49:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c45a7700 1 -- start start 2023-09-10T00:49:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c45a7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc0ff760 0x7fc1bc19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c45a7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1bc100120 0x7fc1bc19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c45a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1bc1055b0 0x7fc1bc198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c45a7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc198e30 con 0x7fc1bc1055b0 2023-09-10T00:49:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c45a7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc198fa0 con 0x7fc1bc0ff760 2023-09-10T00:49:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c45a7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc199160 con 0x7fc1bc100120 2023-09-10T00:49:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c1b42700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1bc100120 0x7fc1bc19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c2b44700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1bc1055b0 0x7fc1bc198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c2343700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc0ff760 0x7fc1bc19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c1b42700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1bc100120 0x7fc1bc19edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58842/0 (socket says 172.21.15.44:58842) 2023-09-10T00:49:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c2b44700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1bc1055b0 0x7fc1bc198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51962/0 (socket says 172.21.15.44:51962) 2023-09-10T00:49:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c1b42700 1 -- 172.21.15.44:0/4055936995 learned_addr learned my addr 172.21.15.44:0/4055936995 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c2343700 1 -- 172.21.15.44:0/4055936995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1bc100120 msgr2=0x7fc1bc19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c2343700 1 --2- 172.21.15.44:0/4055936995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1bc100120 0x7fc1bc19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c2343700 1 -- 172.21.15.44:0/4055936995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1bc1055b0 msgr2=0x7fc1bc198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c2343700 1 --2- 172.21.15.44:0/4055936995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1bc1055b0 0x7fc1bc198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c2343700 1 -- 172.21.15.44:0/4055936995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b8009520 con 0x7fc1bc0ff760 2023-09-10T00:49:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.099+0000 7fc1c1b42700 1 --2- 172.21.15.44:0/4055936995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1bc100120 0x7fc1bc19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.100+0000 7fc1c2343700 1 --2- 172.21.15.44:0/4055936995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc0ff760 0x7fc1bc19e890 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009810 tx=0x7fc1b800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.100+0000 7fc1b37fe700 1 -- 172.21.15.44:0/4055936995 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b801c070 con 0x7fc1bc0ff760 2023-09-10T00:49:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.100+0000 7fc1b37fe700 1 -- 172.21.15.44:0/4055936995 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1b800ba70 con 0x7fc1bc0ff760 2023-09-10T00:49:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.100+0000 7fc1c45a7700 1 -- 172.21.15.44:0/4055936995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1bc1993e0 con 0x7fc1bc0ff760 2023-09-10T00:49:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.100+0000 7fc1b37fe700 1 -- 172.21.15.44:0/4055936995 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80038e0 con 0x7fc1bc0ff760 2023-09-10T00:49:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.100+0000 7fc1c45a7700 1 -- 172.21.15.44:0/4055936995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc1a5810 con 0x7fc1bc0ff760 2023-09-10T00:49:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.103+0000 7fc1b37fe700 1 -- 172.21.15.44:0/4055936995 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc1b8020030 con 0x7fc1bc0ff760 2023-09-10T00:49:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.103+0000 7fc1b37fe700 1 --2- 172.21.15.44:0/4055936995 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1a80a0830 0x7fc1a80a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.103+0000 7fc1b37fe700 1 -- 172.21.15.44:0/4055936995 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fc1b80cd000 con 0x7fc1bc0ff760 2023-09-10T00:49:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.104+0000 7fc1c1b42700 1 --2- 172.21.15.44:0/4055936995 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1a80a0830 0x7fc1a80a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.104+0000 7fc1c1b42700 1 --2- 172.21.15.44:0/4055936995 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1a80a0830 0x7fc1a80a2cf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc19a0a0 tx=0x7fc1ac009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:16.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.104+0000 7fc1d076db80 1 Processor -- start 2023-09-10T00:49:16.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.105+0000 7fc1d076db80 1 -- start start 2023-09-10T00:49:16.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.105+0000 7fc1d076db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc77b1a0 0x55c7dc77b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.105+0000 7fc1d076db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775300 0x55c7dc775780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.105+0000 7fc1d076db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc775cc0 0x55c7dc781f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.105+0000 7fc1d076db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dc6ce6d0 con 0x55c7dc775300 2023-09-10T00:49:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.105+0000 7fc1d076db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dc6cbff0 con 0x55c7dc775cc0 2023-09-10T00:49:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.105+0000 7fc1d076db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dc6c78e0 con 0x55c7dc77b1a0 2023-09-10T00:49:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.105+0000 7fc1c1b42700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775300 0x55c7dc775780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.105+0000 7fc1c2b44700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc775cc0 0x55c7dc781f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.105+0000 7fc1c1b42700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775300 0x55c7dc775780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51968/0 (socket says 172.21.15.44:51968) 2023-09-10T00:49:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.105+0000 7fc1c1b42700 1 -- 172.21.15.44:0/943606915 learned_addr learned my addr 172.21.15.44:0/943606915 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.105+0000 7fc1c2b44700 1 -- 172.21.15.44:0/943606915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc77b1a0 msgr2=0x55c7dc77b580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T00:49:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.105+0000 7fc1c2b44700 1 --2- 172.21.15.44:0/943606915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc77b1a0 0x55c7dc77b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.105+0000 7fc1c2b44700 1 -- 172.21.15.44:0/943606915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775300 msgr2=0x55c7dc775780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.105+0000 7fc1c2b44700 1 --2- 172.21.15.44:0/943606915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775300 0x55c7dc775780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.105+0000 7fc1c2b44700 1 -- 172.21.15.44:0/943606915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7dc4f3d80 con 0x55c7dc775cc0 2023-09-10T00:49:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.106+0000 7fc1c2b44700 1 --2- 172.21.15.44:0/943606915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc775cc0 0x55c7dc781f30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4011550 tx=0x7fc1b40099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2a1b69683312323 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.106+0000 7fc1b0ff9700 1 -- 172.21.15.44:0/943606915 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400e040 con 0x55c7dc775cc0 2023-09-10T00:49:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.106+0000 7fc1b0ff9700 1 -- 172.21.15.44:0/943606915 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1b400f040 con 0x55c7dc775cc0 2023-09-10T00:49:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.106+0000 7fc1b0ff9700 1 -- 172.21.15.44:0/943606915 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4004290 con 0x55c7dc775cc0 2023-09-10T00:49:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.107+0000 7fc1d076db80 1 -- 172.21.15.44:0/943606915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc775cc0 msgr2=0x55c7dc781f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.107+0000 7fc1d076db80 1 --2- 172.21.15.44:0/943606915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc775cc0 0x55c7dc781f30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4011550 tx=0x7fc1b40099e0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.107+0000 7fc1d076db80 1 -- 172.21.15.44:0/943606915 shutdown_connections 2023-09-10T00:49:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.107+0000 7fc1d076db80 1 --2- 172.21.15.44:0/943606915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc775cc0 0x55c7dc781f30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.107+0000 7fc1d076db80 1 --2- 172.21.15.44:0/943606915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc77b1a0 0x55c7dc77b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.107+0000 7fc1d076db80 1 --2- 172.21.15.44:0/943606915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775300 0x55c7dc775780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.107+0000 7fc1d076db80 1 -- 172.21.15.44:0/943606915 >> 172.21.15.44:0/943606915 conn(0x55c7dc6ea030 msgr2=0x55c7dc770560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.107+0000 7fc1d076db80 1 -- 172.21.15.44:0/943606915 shutdown_connections 2023-09-10T00:49:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.107+0000 7fc1d076db80 1 -- 172.21.15.44:0/943606915 wait complete. 2023-09-10T00:49:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.108+0000 7fc1d076db80 1 -- start start 2023-09-10T00:49:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.108+0000 7fc1d076db80 1 Processor -- start 2023-09-10T00:49:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.108+0000 7fc1d076db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775300 0x55c7dc781dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.108+0000 7fc1d076db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc775cc0 0x55c7dc77efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.108+0000 7fc1d076db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc77b1a0 0x55c7dc77f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.108+0000 7fc1d076db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dc665b10 con 0x55c7dc77b1a0 2023-09-10T00:49:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.108+0000 7fc1d076db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dc665810 con 0x55c7dc775cc0 2023-09-10T00:49:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.108+0000 7fc1d076db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dc5498c0 con 0x55c7dc775300 2023-09-10T00:49:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.108+0000 7fc1d076db80 10 client.0 _ll_register_callbacks cb 0x55c7dc5afa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:49:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.108+0000 7fc1c2343700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775300 0x55c7dc781dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.108+0000 7fc1c2b44700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc77b1a0 0x55c7dc77f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.108+0000 7fc1c1b42700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc775cc0 0x55c7dc77efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.108+0000 7fc1c2343700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775300 0x55c7dc781dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58854/0 (socket says 172.21.15.44:58854) 2023-09-10T00:49:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.108+0000 7fc1c2343700 1 -- 172.21.15.44:0/1503145150 learned_addr learned my addr 172.21.15.44:0/1503145150 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.109+0000 7fc1c2343700 1 -- 172.21.15.44:0/1503145150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc775cc0 msgr2=0x55c7dc77efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.109+0000 7fc1c2343700 1 --2- 172.21.15.44:0/1503145150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc775cc0 0x55c7dc77efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.109+0000 7fc1c2343700 1 -- 172.21.15.44:0/1503145150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc77b1a0 msgr2=0x55c7dc77f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.109+0000 7fc1c2343700 1 --2- 172.21.15.44:0/1503145150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc77b1a0 0x55c7dc77f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.109+0000 7fc1c2343700 1 -- 172.21.15.44:0/1503145150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b4011230 con 0x55c7dc775300 2023-09-10T00:49:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.109+0000 7fc1c2b44700 1 --2- 172.21.15.44:0/1503145150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc77b1a0 0x55c7dc77f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.109+0000 7fc1c1b42700 1 --2- 172.21.15.44:0/1503145150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc775cc0 0x55c7dc77efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.109+0000 7fc1c2343700 1 --2- 172.21.15.44:0/1503145150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775300 0x55c7dc781dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8099e40 tx=0x7fc1b8099c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88b1039a1806789 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.109+0000 7fc1a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:49:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.109+0000 7fc1a5ffb700 1 -- 172.21.15.44:0/1503145150 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80213e0 con 0x55c7dc775300 2023-09-10T00:49:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.109+0000 7fc1a5ffb700 1 -- 172.21.15.44:0/1503145150 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fc1b809a840 con 0x55c7dc775300 2023-09-10T00:49:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.109+0000 7fc1d076db80 1 -- 172.21.15.44:0/1503145150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7dc61f490 con 0x55c7dc775300 2023-09-10T00:49:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.109+0000 7fc1a5ffb700 1 -- 172.21.15.44:0/1503145150 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1b809a140 con 0x55c7dc775300 2023-09-10T00:49:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.109+0000 7fc1d076db80 10 client.26749 fetch_fsmap learned FSMap version 53 2023-09-10T00:49:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.109+0000 7fc1d076db80 1 -- 172.21.15.44:0/1503145150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55c7dc4dcc00 con 0x55c7dc775300 2023-09-10T00:49:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.110+0000 7fc1a5ffb700 1 -- 172.21.15.44:0/1503145150 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c7dc61f490 con 0x55c7dc775300 2023-09-10T00:49:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.110+0000 7fc1a5ffb700 1 -- 172.21.15.44:0/1503145150 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 53) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fc1b8005040 con 0x55c7dc775300 2023-09-10T00:49:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.110+0000 7fc1d076db80 10 client.26749 fetch_fsmap finished waiting for FSMap version 53 2023-09-10T00:49:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.110+0000 7fc1d076db80 10 client.26749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:49:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.110+0000 7fc1d076db80 20 client.26749 populate_metadata read hostname 'smithi044' 2023-09-10T00:49:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.110+0000 7fc1d076db80 1 --2- 172.21.15.44:0/1503145150 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c7dc78f0b0 0x55c7dc7914d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.110+0000 7fc1d076db80 4 client.26749 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:49:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.110+0000 7fc1d076db80 1 -- 172.21.15.44:0/1503145150 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7dc6dc840 con 0x55c7dc78f0b0 2023-09-10T00:49:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.110+0000 7fc1c1b42700 1 --2- 172.21.15.44:0/1503145150 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c7dc78f0b0 0x55c7dc7914d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.110+0000 7fc1a5ffb700 10 client.26749 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.111+0000 7fc1c1b42700 1 --2- 172.21.15.44:0/1503145150 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c7dc78f0b0 0x55c7dc7914d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60a5385efb9c5231 server_cookie=a8a7d244a11e8f7d in_seq=0 out_seq=0 2023-09-10T00:49:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.111+0000 7fc1a5ffb700 1 -- 172.21.15.44:0/1503145150 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c7dc6dc840 con 0x55c7dc78f0b0 2023-09-10T00:49:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.111+0000 7fc1a5ffb700 10 client.26749 handle_command_reply: tid=0 2023-09-10T00:49:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.111+0000 7fc1d076db80 1 client.26749 shutdown 2023-09-10T00:49:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.111+0000 7fc1d076db80 1 -- 172.21.15.44:0/1503145150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775300 msgr2=0x55c7dc781dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.111+0000 7fc1d076db80 1 --2- 172.21.15.44:0/1503145150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775300 0x55c7dc781dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8099e40 tx=0x7fc1b8099c80 comp rx=0 tx=0).stop 2023-09-10T00:49:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.111+0000 7fc1d076db80 1 -- 172.21.15.44:0/1503145150 shutdown_connections 2023-09-10T00:49:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.111+0000 7fc1d076db80 1 --2- 172.21.15.44:0/1503145150 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c7dc78f0b0 0x55c7dc7914d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.111+0000 7fc1d076db80 1 --2- 172.21.15.44:0/1503145150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc77b1a0 0x55c7dc77f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.111+0000 7fc1a5ffb700 0 client.26749 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.111+0000 7fc1d076db80 1 --2- 172.21.15.44:0/1503145150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775300 0x55c7dc781dc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.111+0000 7fc1d076db80 1 --2- 172.21.15.44:0/1503145150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc775cc0 0x55c7dc77efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.111+0000 7fc1d076db80 1 -- 172.21.15.44:0/1503145150 >> 172.21.15.44:0/1503145150 conn(0x55c7dc6ea030 msgr2=0x55c7dc76f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.112+0000 7fc1d076db80 1 -- 172.21.15.44:0/1503145150 shutdown_connections 2023-09-10T00:49:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.112+0000 7fc1d076db80 1 -- 172.21.15.44:0/1503145150 wait complete. 2023-09-10T00:49:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.112+0000 7fc1d076db80 20 client.26749 trim_cache size 0 max 16384 2023-09-10T00:49:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.134+0000 7fc1d076db80 1 Processor -- start 2023-09-10T00:49:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.134+0000 7fc1d076db80 1 -- start start 2023-09-10T00:49:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.134+0000 7fc1d076db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775300 0x55c7dc77efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.134+0000 7fc1d076db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775cc0 0x55c7dc786520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.134+0000 7fc1d076db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc77b1a0 0x55c7dc7818f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.134+0000 7fc1d076db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dc770ad0 con 0x55c7dc775cc0 2023-09-10T00:49:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.134+0000 7fc1d076db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dc76cfc0 con 0x55c7dc77b1a0 2023-09-10T00:49:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.134+0000 7fc1d076db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dc779a00 con 0x55c7dc775300 2023-09-10T00:49:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.134+0000 7fc1c2343700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775300 0x55c7dc77efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.134+0000 7fc1c2343700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775300 0x55c7dc77efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58862/0 (socket says 172.21.15.44:58862) 2023-09-10T00:49:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.134+0000 7fc1c2b44700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc77b1a0 0x55c7dc7818f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.134+0000 7fc1c2343700 1 -- 172.21.15.44:0/2228168304 learned_addr learned my addr 172.21.15.44:0/2228168304 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.135+0000 7fc1c1b42700 1 --2- 172.21.15.44:0/2228168304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775cc0 0x55c7dc786520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.135+0000 7fc1c2343700 1 -- 172.21.15.44:0/2228168304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc77b1a0 msgr2=0x55c7dc7818f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.135+0000 7fc1c2343700 1 --2- 172.21.15.44:0/2228168304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc77b1a0 0x55c7dc7818f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.135+0000 7fc1c2343700 1 -- 172.21.15.44:0/2228168304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775cc0 msgr2=0x55c7dc786520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.135+0000 7fc1c2343700 1 --2- 172.21.15.44:0/2228168304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775cc0 0x55c7dc786520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.135+0000 7fc1c2343700 1 -- 172.21.15.44:0/2228168304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7dc51a1f0 con 0x55c7dc775300 2023-09-10T00:49:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.135+0000 7fc1c1b42700 1 --2- 172.21.15.44:0/2228168304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775cc0 0x55c7dc786520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:49:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.135+0000 7fc1c2343700 1 --2- 172.21.15.44:0/2228168304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775300 0x55c7dc77efd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80ca400 tx=0x7fc1b80986f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=233adbe7cd8baeda server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.135+0000 7fc1b17fa700 1 -- 172.21.15.44:0/2228168304 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b800bbe0 con 0x55c7dc775300 2023-09-10T00:49:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.135+0000 7fc1b17fa700 1 -- 172.21.15.44:0/2228168304 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1b80213e0 con 0x55c7dc775300 2023-09-10T00:49:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.135+0000 7fc1b17fa700 1 -- 172.21.15.44:0/2228168304 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b809dfe0 con 0x55c7dc775300 2023-09-10T00:49:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.137+0000 7fc1d076db80 1 -- 172.21.15.44:0/2228168304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775300 msgr2=0x55c7dc77efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.137+0000 7fc1d076db80 1 --2- 172.21.15.44:0/2228168304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775300 0x55c7dc77efd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80ca400 tx=0x7fc1b80986f0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.137+0000 7fc1d076db80 1 -- 172.21.15.44:0/2228168304 shutdown_connections 2023-09-10T00:49:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.137+0000 7fc1d076db80 1 --2- 172.21.15.44:0/2228168304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc77b1a0 0x55c7dc7818f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.137+0000 7fc1d076db80 1 --2- 172.21.15.44:0/2228168304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775300 0x55c7dc77efd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.137+0000 7fc1d076db80 1 --2- 172.21.15.44:0/2228168304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775cc0 0x55c7dc786520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.137+0000 7fc1d076db80 1 -- 172.21.15.44:0/2228168304 >> 172.21.15.44:0/2228168304 conn(0x55c7dc6ea030 msgr2=0x55c7dc77e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.137+0000 7fc1d076db80 1 -- 172.21.15.44:0/2228168304 shutdown_connections 2023-09-10T00:49:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.137+0000 7fc1d076db80 1 -- 172.21.15.44:0/2228168304 wait complete. 2023-09-10T00:49:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.137+0000 7fc1d076db80 1 -- start start 2023-09-10T00:49:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.137+0000 7fc1d076db80 1 Processor -- start 2023-09-10T00:49:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1d076db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775300 0x55c7dc786380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1d076db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775cc0 0x55c7dc7846b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1d076db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc77b1a0 0x55c7dc784bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1d076db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dc785170 con 0x55c7dc775300 2023-09-10T00:49:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1d076db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dc7852e0 con 0x55c7dc77b1a0 2023-09-10T00:49:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1d076db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dc7854a0 con 0x55c7dc775cc0 2023-09-10T00:49:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1d076db80 10 client.0 _ll_register_callbacks cb 0x7fc1b8096480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:49:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1c1b42700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775cc0 0x55c7dc7846b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1c2b44700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc77b1a0 0x55c7dc784bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1c1b42700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775cc0 0x55c7dc7846b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58870/0 (socket says 172.21.15.44:58870) 2023-09-10T00:49:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1c1b42700 1 -- 172.21.15.44:0/1487406286 learned_addr learned my addr 172.21.15.44:0/1487406286 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1c2343700 1 --2- 172.21.15.44:0/1487406286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775300 0x55c7dc786380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1c1b42700 1 -- 172.21.15.44:0/1487406286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc77b1a0 msgr2=0x55c7dc784bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1c1b42700 1 --2- 172.21.15.44:0/1487406286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc77b1a0 0x55c7dc784bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1c1b42700 1 -- 172.21.15.44:0/1487406286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775300 msgr2=0x55c7dc786380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1c1b42700 1 --2- 172.21.15.44:0/1487406286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775300 0x55c7dc786380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1c1b42700 1 -- 172.21.15.44:0/1487406286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7dc4dcc00 con 0x55c7dc775cc0 2023-09-10T00:49:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1c1b42700 1 --2- 172.21.15.44:0/1487406286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775cc0 0x55c7dc7846b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac013010 tx=0x7fc1ac0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f685f704d024c574 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.138+0000 7fc1a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:49:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.139+0000 7fc1a5ffb700 1 -- 172.21.15.44:0/1487406286 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac022040 con 0x55c7dc775cc0 2023-09-10T00:49:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.139+0000 7fc1d076db80 1 -- 172.21.15.44:0/1487406286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7dc6c08c0 con 0x55c7dc775cc0 2023-09-10T00:49:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.139+0000 7fc1a5ffb700 1 -- 172.21.15.44:0/1487406286 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fc1ac02bd40 con 0x55c7dc775cc0 2023-09-10T00:49:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.139+0000 7fc1a5ffb700 1 -- 172.21.15.44:0/1487406286 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1ac02b040 con 0x55c7dc775cc0 2023-09-10T00:49:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.139+0000 7fc1d076db80 10 client.26755 fetch_fsmap learned FSMap version 53 2023-09-10T00:49:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.139+0000 7fc1d076db80 1 -- 172.21.15.44:0/1487406286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55c7dc4f2250 con 0x55c7dc775cc0 2023-09-10T00:49:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.139+0000 7fc1a5ffb700 1 -- 172.21.15.44:0/1487406286 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c7dc6c08c0 con 0x55c7dc775cc0 2023-09-10T00:49:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.139+0000 7fc1a5ffb700 1 -- 172.21.15.44:0/1487406286 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 53) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fc1ac004830 con 0x55c7dc775cc0 2023-09-10T00:49:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.139+0000 7fc1d076db80 10 client.26755 fetch_fsmap finished waiting for FSMap version 53 2023-09-10T00:49:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.139+0000 7fc1d076db80 10 client.26755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:49:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.139+0000 7fc1d076db80 20 client.26755 populate_metadata read hostname 'smithi044' 2023-09-10T00:49:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.139+0000 7fc1d076db80 1 --2- 172.21.15.44:0/1487406286 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c7dc78f0b0 0x55c7dc7e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.139+0000 7fc1d076db80 4 client.26755 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:49:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.139+0000 7fc1d076db80 1 -- 172.21.15.44:0/1487406286 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c7dc681050 con 0x55c7dc78f0b0 2023-09-10T00:49:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.139+0000 7fc1c2343700 1 --2- 172.21.15.44:0/1487406286 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c7dc78f0b0 0x55c7dc7e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.140+0000 7fc1a5ffb700 10 client.26755 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.140+0000 7fc1c2343700 1 --2- 172.21.15.44:0/1487406286 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c7dc78f0b0 0x55c7dc7e64e0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82698db436086358 server_cookie=f8cd56d27ecca1b1 in_seq=0 out_seq=0 2023-09-10T00:49:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.156+0000 7fc1a5ffb700 1 -- 172.21.15.44:0/1487406286 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c7dc681050 con 0x55c7dc78f0b0 2023-09-10T00:49:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.156+0000 7fc1a5ffb700 10 client.26755 handle_command_reply: tid=0 2023-09-10T00:49:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.156+0000 7fc1d076db80 1 client.26755 shutdown 2023-09-10T00:49:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.156+0000 7fc1d076db80 1 -- 172.21.15.44:0/1487406286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775cc0 msgr2=0x55c7dc7846b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.156+0000 7fc1d076db80 1 --2- 172.21.15.44:0/1487406286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775cc0 0x55c7dc7846b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac013010 tx=0x7fc1ac0047c0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.156+0000 7fc1d076db80 1 -- 172.21.15.44:0/1487406286 shutdown_connections 2023-09-10T00:49:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.156+0000 7fc1d076db80 1 --2- 172.21.15.44:0/1487406286 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c7dc78f0b0 0x55c7dc7e64e0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.156+0000 7fc1d076db80 1 --2- 172.21.15.44:0/1487406286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7dc77b1a0 0x55c7dc784bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.156+0000 7fc1d076db80 1 --2- 172.21.15.44:0/1487406286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c7dc775300 0x55c7dc786380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.156+0000 7fc1d076db80 1 --2- 172.21.15.44:0/1487406286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7dc775cc0 0x55c7dc7846b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.156+0000 7fc1d076db80 1 -- 172.21.15.44:0/1487406286 >> 172.21.15.44:0/1487406286 conn(0x55c7dc6ea030 msgr2=0x55c7dc76ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.156+0000 7fc1a5ffb700 0 client.26755 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.157+0000 7fc1d076db80 1 -- 172.21.15.44:0/1487406286 shutdown_connections 2023-09-10T00:49:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.157+0000 7fc1d076db80 1 -- 172.21.15.44:0/1487406286 wait complete. 2023-09-10T00:49:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.157+0000 7fc1d076db80 20 client.26755 trim_cache size 0 max 16384 2023-09-10T00:49:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:49:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:49:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:49:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.157+0000 7fc1c45a7700 1 -- 172.21.15.44:0/4055936995 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1a80a0830 msgr2=0x7fc1a80a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.157+0000 7fc1c45a7700 1 --2- 172.21.15.44:0/4055936995 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1a80a0830 0x7fc1a80a2cf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc19a0a0 tx=0x7fc1ac009260 comp rx=0 tx=0).stop 2023-09-10T00:49:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.157+0000 7fc1c45a7700 1 -- 172.21.15.44:0/4055936995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc0ff760 msgr2=0x7fc1bc19e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.157+0000 7fc1c45a7700 1 --2- 172.21.15.44:0/4055936995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc0ff760 0x7fc1bc19e890 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009810 tx=0x7fc1b800fa00 comp rx=0 tx=0).stop 2023-09-10T00:49:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.157+0000 7fc1c45a7700 1 -- 172.21.15.44:0/4055936995 shutdown_connections 2023-09-10T00:49:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.157+0000 7fc1c45a7700 1 --2- 172.21.15.44:0/4055936995 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1a80a0830 0x7fc1a80a2cf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.157+0000 7fc1c45a7700 1 --2- 172.21.15.44:0/4055936995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1bc1055b0 0x7fc1bc198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.157+0000 7fc1c45a7700 1 --2- 172.21.15.44:0/4055936995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc0ff760 0x7fc1bc19e890 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.157+0000 7fc1c45a7700 1 --2- 172.21.15.44:0/4055936995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1bc100120 0x7fc1bc19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.157+0000 7fc1c45a7700 1 -- 172.21.15.44:0/4055936995 >> 172.21.15.44:0/4055936995 conn(0x7fc1bc074f30 msgr2=0x7fc1bc0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.158+0000 7fc1c45a7700 1 -- 172.21.15.44:0/4055936995 shutdown_connections 2023-09-10T00:49:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:16.158+0000 7fc1c45a7700 1 -- 172.21.15.44:0/4055936995 wait complete. 2023-09-10T00:49:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:16 smithi131 ceph-mon[103713]: pgmap v267: 129 pgs: 129 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 3.6 MiB/s wr, 358 op/s 2023-09-10T00:49:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:16 smithi044 ceph-mon[98676]: pgmap v267: 129 pgs: 129 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 3.6 MiB/s wr, 358 op/s 2023-09-10T00:49:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:16 smithi099 ceph-mon[102692]: pgmap v267: 129 pgs: 129 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 3.6 MiB/s wr, 358 op/s 2023-09-10T00:49:17.586 INFO:teuthology.orchestra.run.smithi044.stderr:32100000 of 50000000 tuples (64%) done (elapsed 497.22 s, remaining 277.27 s) 2023-09-10T00:49:18.958 INFO:teuthology.orchestra.run.smithi044.stderr:32200000 of 50000000 tuples (64%) done (elapsed 498.59 s, remaining 275.62 s) 2023-09-10T00:49:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:18 smithi131 ceph-mon[103713]: pgmap v268: 129 pgs: 129 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 3.6 MiB/s wr, 358 op/s 2023-09-10T00:49:19.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:18 smithi044 ceph-mon[98676]: pgmap v268: 129 pgs: 129 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 3.6 MiB/s wr, 358 op/s 2023-09-10T00:49:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:18 smithi099 ceph-mon[102692]: pgmap v268: 129 pgs: 129 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 3.6 MiB/s wr, 358 op/s 2023-09-10T00:49:20.109 INFO:teuthology.orchestra.run.smithi044.stderr:32300000 of 50000000 tuples (64%) done (elapsed 499.74 s, remaining 273.85 s) 2023-09-10T00:49:21.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:20 smithi131 ceph-mon[103713]: pgmap v269: 129 pgs: 129 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 3.6 MiB/s wr, 358 op/s 2023-09-10T00:49:21.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:20 smithi044 ceph-mon[98676]: pgmap v269: 129 pgs: 129 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 3.6 MiB/s wr, 358 op/s 2023-09-10T00:49:21.061 INFO:teuthology.orchestra.run.smithi044.stderr:32400000 of 50000000 tuples (64%) done (elapsed 500.70 s, remaining 271.98 s) 2023-09-10T00:49:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:20 smithi099 ceph-mon[102692]: pgmap v269: 129 pgs: 129 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 3.6 MiB/s wr, 358 op/s 2023-09-10T00:49:22.018 INFO:teuthology.orchestra.run.smithi044.stderr:32500000 of 50000000 tuples (65%) done (elapsed 501.64 s, remaining 270.12 s) 2023-09-10T00:49:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:22 smithi131 ceph-mon[103713]: pgmap v270: 129 pgs: 129 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 10 MiB/s wr, 326 op/s 2023-09-10T00:49:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:22 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:23.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:22 smithi044 ceph-mon[98676]: pgmap v270: 129 pgs: 129 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 10 MiB/s wr, 326 op/s 2023-09-10T00:49:23.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:22 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:32600000 of 50000000 tuples (65%) done (elapsed 502.71 s, remaining 268.32 s) 2023-09-10T00:49:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:22 smithi099 ceph-mon[102692]: pgmap v270: 129 pgs: 129 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 10 MiB/s wr, 326 op/s 2023-09-10T00:49:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:22 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:23.974 INFO:teuthology.orchestra.run.smithi044.stderr:32700000 of 50000000 tuples (65%) done (elapsed 503.61 s, remaining 266.44 s) 2023-09-10T00:49:24.967 INFO:teuthology.orchestra.run.smithi044.stderr:32800000 of 50000000 tuples (65%) done (elapsed 504.60 s, remaining 264.61 s) 2023-09-10T00:49:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:24 smithi131 ceph-mon[103713]: pgmap v271: 129 pgs: 129 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 9.2 MiB/s wr, 286 op/s 2023-09-10T00:49:25.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:24 smithi044 ceph-mon[98676]: pgmap v271: 129 pgs: 129 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 9.2 MiB/s wr, 286 op/s 2023-09-10T00:49:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:24 smithi099 ceph-mon[102692]: pgmap v271: 129 pgs: 129 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 9.2 MiB/s wr, 286 op/s 2023-09-10T00:49:25.890 INFO:teuthology.orchestra.run.smithi044.stderr:32900000 of 50000000 tuples (65%) done (elapsed 505.52 s, remaining 262.75 s) 2023-09-10T00:49:26.786 INFO:teuthology.orchestra.run.smithi044.stderr:33000000 of 50000000 tuples (66%) done (elapsed 506.42 s, remaining 260.88 s) 2023-09-10T00:49:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:26 smithi131 ceph-mon[103713]: pgmap v272: 129 pgs: 129 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 25 MiB/s wr, 326 op/s 2023-09-10T00:49:27.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:26 smithi044 ceph-mon[98676]: pgmap v272: 129 pgs: 129 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 25 MiB/s wr, 326 op/s 2023-09-10T00:49:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:26 smithi099 ceph-mon[102692]: pgmap v272: 129 pgs: 129 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 25 MiB/s wr, 326 op/s 2023-09-10T00:49:27.618 INFO:teuthology.orchestra.run.smithi044.stderr:33100000 of 50000000 tuples (66%) done (elapsed 507.25 s, remaining 258.99 s) 2023-09-10T00:49:28.490 INFO:teuthology.orchestra.run.smithi044.stderr:33200000 of 50000000 tuples (66%) done (elapsed 508.12 s, remaining 257.12 s) 2023-09-10T00:49:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:28 smithi131 ceph-mon[103713]: pgmap v273: 129 pgs: 129 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 174 op/s 2023-09-10T00:49:29.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:28 smithi044 ceph-mon[98676]: pgmap v273: 129 pgs: 129 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 174 op/s 2023-09-10T00:49:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:28 smithi099 ceph-mon[102692]: pgmap v273: 129 pgs: 129 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 174 op/s 2023-09-10T00:49:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.294+0000 7fd73e4d3700 1 -- 172.21.15.44:0/1230229825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd738068960 msgr2=0x7fd738068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.294+0000 7fd73e4d3700 1 --2- 172.21.15.44:0/1230229825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd738068960 0x7fd738068de0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009920 tx=0x7fd72c009c30 comp rx=0 tx=0).stop 2023-09-10T00:49:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.294+0000 7fd73e4d3700 1 -- 172.21.15.44:0/1230229825 shutdown_connections 2023-09-10T00:49:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.294+0000 7fd73e4d3700 1 --2- 172.21.15.44:0/1230229825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd738069320 0x7fd73810d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.294+0000 7fd73e4d3700 1 --2- 172.21.15.44:0/1230229825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd738106670 0x7fd738106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.294+0000 7fd73e4d3700 1 --2- 172.21.15.44:0/1230229825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd738068960 0x7fd738068de0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.294+0000 7fd73e4d3700 1 -- 172.21.15.44:0/1230229825 >> 172.21.15.44:0/1230229825 conn(0x7fd738076600 msgr2=0x7fd738076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.294+0000 7fd73e4d3700 1 -- 172.21.15.44:0/1230229825 shutdown_connections 2023-09-10T00:49:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.294+0000 7fd73e4d3700 1 -- 172.21.15.44:0/1230229825 wait complete. 2023-09-10T00:49:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.295+0000 7fd73e4d3700 1 Processor -- start 2023-09-10T00:49:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.295+0000 7fd73e4d3700 1 -- start start 2023-09-10T00:49:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.295+0000 7fd73e4d3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd738068960 0x7fd73819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.295+0000 7fd73e4d3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd738069320 0x7fd73819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.295+0000 7fd73e4d3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd738106670 0x7fd738196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.295+0000 7fd73e4d3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd738196a90 con 0x7fd738106670 2023-09-10T00:49:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.295+0000 7fd73e4d3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd738196c00 con 0x7fd738068960 2023-09-10T00:49:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.295+0000 7fd73e4d3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd738196dc0 con 0x7fd738069320 2023-09-10T00:49:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.295+0000 7fd7377fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd738069320 0x7fd73819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.295+0000 7fd73ca70700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd738106670 0x7fd738196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.295+0000 7fd737fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd738068960 0x7fd73819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.295+0000 7fd7377fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd738069320 0x7fd73819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58884/0 (socket says 172.21.15.44:58884) 2023-09-10T00:49:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.295+0000 7fd7377fe700 1 -- 172.21.15.44:0/4255036555 learned_addr learned my addr 172.21.15.44:0/4255036555 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.296+0000 7fd7377fe700 1 -- 172.21.15.44:0/4255036555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd738068960 msgr2=0x7fd73819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.296+0000 7fd7377fe700 1 --2- 172.21.15.44:0/4255036555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd738068960 0x7fd73819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.296+0000 7fd7377fe700 1 -- 172.21.15.44:0/4255036555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd738106670 msgr2=0x7fd738196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.296+0000 7fd7377fe700 1 --2- 172.21.15.44:0/4255036555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd738106670 0x7fd738196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.296+0000 7fd7377fe700 1 -- 172.21.15.44:0/4255036555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd730009c40 con 0x7fd738069320 2023-09-10T00:49:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.296+0000 7fd737fff700 1 --2- 172.21.15.44:0/4255036555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd738068960 0x7fd73819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.296+0000 7fd73ca70700 1 --2- 172.21.15.44:0/4255036555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd738106670 0x7fd738196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.296+0000 7fd7377fe700 1 --2- 172.21.15.44:0/4255036555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd738069320 0x7fd73819cb00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009230 tx=0x7fd72c00f9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.296+0000 7fd7357fa700 1 -- 172.21.15.44:0/4255036555 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c01c070 con 0x7fd738069320 2023-09-10T00:49:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.296+0000 7fd7357fa700 1 -- 172.21.15.44:0/4255036555 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd72c00bbd0 con 0x7fd738069320 2023-09-10T00:49:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.296+0000 7fd73e4d3700 1 -- 172.21.15.44:0/4255036555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72c009520 con 0x7fd738069320 2023-09-10T00:49:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.296+0000 7fd7357fa700 1 -- 172.21.15.44:0/4255036555 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c0038e0 con 0x7fd738069320 2023-09-10T00:49:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.296+0000 7fd73e4d3700 1 -- 172.21.15.44:0/4255036555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7381a33d0 con 0x7fd738069320 2023-09-10T00:49:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.299+0000 7fd7357fa700 1 -- 172.21.15.44:0/4255036555 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fd72c020030 con 0x7fd738069320 2023-09-10T00:49:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.299+0000 7fd7357fa700 1 --2- 172.21.15.44:0/4255036555 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7200a07b0 0x7fd7200a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.300+0000 7fd737fff700 1 --2- 172.21.15.44:0/4255036555 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7200a07b0 0x7fd7200a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.300+0000 7fd7357fa700 1 -- 172.21.15.44:0/4255036555 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fd72c0cd2f0 con 0x7fd738069320 2023-09-10T00:49:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.300+0000 7fd737fff700 1 --2- 172.21.15.44:0/4255036555 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7200a07b0 0x7fd7200a2c70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd728005fd0 tx=0x7fd728009310 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd74a699b80 1 Processor -- start 2023-09-10T00:49:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd74a699b80 1 -- start start 2023-09-10T00:49:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd74a699b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453ea5b0 0x55fd453ea990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd74a699b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453e4710 0x55fd453e4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd74a699b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e50d0 0x55fd453f1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd74a699b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd453396d0 con 0x55fd453ea5b0 2023-09-10T00:49:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd74a699b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd45336ff0 con 0x55fd453e50d0 2023-09-10T00:49:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd74a699b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd453328e0 con 0x55fd453e4710 2023-09-10T00:49:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd737fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453ea5b0 0x55fd453ea990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd7377fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453e4710 0x55fd453e4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd737fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453ea5b0 0x55fd453ea990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52008/0 (socket says 172.21.15.44:52008) 2023-09-10T00:49:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd7377fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453e4710 0x55fd453e4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58892/0 (socket says 172.21.15.44:58892) 2023-09-10T00:49:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd737fff700 1 -- 172.21.15.44:0/1954255357 learned_addr learned my addr 172.21.15.44:0/1954255357 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd73ca70700 1 --2- 172.21.15.44:0/1954255357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e50d0 0x55fd453f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd7377fe700 1 -- 172.21.15.44:0/1954255357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e50d0 msgr2=0x55fd453f1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd7377fe700 1 --2- 172.21.15.44:0/1954255357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e50d0 0x55fd453f1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd7377fe700 1 -- 172.21.15.44:0/1954255357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453ea5b0 msgr2=0x55fd453ea990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd7377fe700 1 --2- 172.21.15.44:0/1954255357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453ea5b0 0x55fd453ea990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd7377fe700 1 -- 172.21.15.44:0/1954255357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd4515ed80 con 0x55fd453e4710 2023-09-10T00:49:29.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd73ca70700 1 --2- 172.21.15.44:0/1954255357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e50d0 0x55fd453f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:29.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.301+0000 7fd737fff700 1 --2- 172.21.15.44:0/1954255357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453ea5b0 0x55fd453ea990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:29.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.302+0000 7fd7377fe700 1 --2- 172.21.15.44:0/1954255357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453e4710 0x55fd453e4b90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0ca5e0 tx=0x7fd72c09a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92d446509159e6ec server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:29.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.302+0000 7fd71e7fc700 1 -- 172.21.15.44:0/1954255357 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c0cd120 con 0x55fd453e4710 2023-09-10T00:49:29.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.302+0000 7fd71e7fc700 1 -- 172.21.15.44:0/1954255357 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd72c021380 con 0x55fd453e4710 2023-09-10T00:49:29.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.302+0000 7fd71e7fc700 1 -- 172.21.15.44:0/1954255357 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c0a23b0 con 0x55fd453e4710 2023-09-10T00:49:29.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.302+0000 7fd74a699b80 1 -- 172.21.15.44:0/1954255357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453e4710 msgr2=0x55fd453e4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:29.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.302+0000 7fd74a699b80 1 --2- 172.21.15.44:0/1954255357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453e4710 0x55fd453e4b90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0ca5e0 tx=0x7fd72c09a270 comp rx=0 tx=0).stop 2023-09-10T00:49:29.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.303+0000 7fd74a699b80 1 -- 172.21.15.44:0/1954255357 shutdown_connections 2023-09-10T00:49:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.303+0000 7fd74a699b80 1 --2- 172.21.15.44:0/1954255357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e50d0 0x55fd453f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.303+0000 7fd74a699b80 1 --2- 172.21.15.44:0/1954255357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453ea5b0 0x55fd453ea990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.303+0000 7fd74a699b80 1 --2- 172.21.15.44:0/1954255357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453e4710 0x55fd453e4b90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.303+0000 7fd74a699b80 1 -- 172.21.15.44:0/1954255357 >> 172.21.15.44:0/1954255357 conn(0x55fd45355030 msgr2=0x55fd453db560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.303+0000 7fd74a699b80 1 -- 172.21.15.44:0/1954255357 shutdown_connections 2023-09-10T00:49:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.303+0000 7fd74a699b80 1 -- 172.21.15.44:0/1954255357 wait complete. 2023-09-10T00:49:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.303+0000 7fd74a699b80 1 -- start start 2023-09-10T00:49:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.303+0000 7fd74a699b80 1 Processor -- start 2023-09-10T00:49:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd74a699b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e4710 0x55fd453f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd74a699b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453e50d0 0x55fd453f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd74a699b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453ea5b0 0x55fd453ee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd74a699b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd452d0b10 con 0x55fd453ea5b0 2023-09-10T00:49:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd74a699b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd452d0810 con 0x55fd453e4710 2023-09-10T00:49:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd74a699b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd451b48c0 con 0x55fd453e50d0 2023-09-10T00:49:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd74a699b80 10 client.0 _ll_register_callbacks cb 0x55fd4521aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:49:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd7377fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453e50d0 0x55fd453f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd7377fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453e50d0 0x55fd453f44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58898/0 (socket says 172.21.15.44:58898) 2023-09-10T00:49:29.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd737fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e4710 0x55fd453f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd7377fe700 1 -- 172.21.15.44:0/260492905 learned_addr learned my addr 172.21.15.44:0/260492905 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:29.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd73ca70700 1 --2- 172.21.15.44:0/260492905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453ea5b0 0x55fd453ee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd7377fe700 1 -- 172.21.15.44:0/260492905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e4710 msgr2=0x55fd453f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:29.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd7377fe700 1 --2- 172.21.15.44:0/260492905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e4710 0x55fd453f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd7377fe700 1 -- 172.21.15.44:0/260492905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453ea5b0 msgr2=0x55fd453ee1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:29.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd7377fe700 1 --2- 172.21.15.44:0/260492905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453ea5b0 0x55fd453ee1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd7377fe700 1 -- 172.21.15.44:0/260492905 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7300091a0 con 0x55fd453e50d0 2023-09-10T00:49:29.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd7377fe700 1 --2- 172.21.15.44:0/260492905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453e50d0 0x55fd453f44c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0098f0 tx=0x7fd72c005400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afd8c5e741f16cfc server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:29.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.304+0000 7fd7177fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:49:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.305+0000 7fd7177fe700 1 -- 172.21.15.44:0/260492905 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c0cd120 con 0x55fd453e50d0 2023-09-10T00:49:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.305+0000 7fd7177fe700 1 -- 172.21.15.44:0/260492905 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fd72c09aad0 con 0x55fd453e50d0 2023-09-10T00:49:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.305+0000 7fd74a699b80 1 -- 172.21.15.44:0/260492905 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd4528a490 con 0x55fd453e50d0 2023-09-10T00:49:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.305+0000 7fd7177fe700 1 -- 172.21.15.44:0/260492905 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd72c0850b0 con 0x55fd453e50d0 2023-09-10T00:49:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.305+0000 7fd74a699b80 10 client.26767 fetch_fsmap learned FSMap version 54 2023-09-10T00:49:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.305+0000 7fd74a699b80 1 -- 172.21.15.44:0/260492905 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x7fd728017070 con 0x55fd453e50d0 2023-09-10T00:49:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.305+0000 7fd7177fe700 1 -- 172.21.15.44:0/260492905 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fd4528a490 con 0x55fd453e50d0 2023-09-10T00:49:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.305+0000 7fd7177fe700 1 -- 172.21.15.44:0/260492905 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 54) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fd72c0a00c0 con 0x55fd453e50d0 2023-09-10T00:49:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.305+0000 7fd74a699b80 10 client.26767 fetch_fsmap finished waiting for FSMap version 54 2023-09-10T00:49:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.305+0000 7fd74a699b80 10 client.26767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:49:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.305+0000 7fd74a699b80 20 client.26767 populate_metadata read hostname 'smithi044' 2023-09-10T00:49:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.305+0000 7fd74a699b80 1 --2- 172.21.15.44:0/260492905 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fd453fe080 0x55fd454004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.305+0000 7fd74a699b80 4 client.26767 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:49:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.305+0000 7fd74a699b80 1 -- 172.21.15.44:0/260492905 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd45347840 con 0x55fd453fe080 2023-09-10T00:49:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.305+0000 7fd737fff700 1 --2- 172.21.15.44:0/260492905 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fd453fe080 0x55fd454004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.306+0000 7fd7177fe700 10 client.26767 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.306+0000 7fd737fff700 1 --2- 172.21.15.44:0/260492905 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fd453fe080 0x55fd454004a0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef76ffe21d5a6518 server_cookie=a15983fa726bc6f5 in_seq=0 out_seq=0 2023-09-10T00:49:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.306+0000 7fd7177fe700 1 -- 172.21.15.44:0/260492905 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fd45347840 con 0x55fd453fe080 2023-09-10T00:49:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.306+0000 7fd7177fe700 10 client.26767 handle_command_reply: tid=0 2023-09-10T00:49:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.306+0000 7fd74a699b80 1 client.26767 shutdown 2023-09-10T00:49:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.307+0000 7fd74a699b80 1 -- 172.21.15.44:0/260492905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453e50d0 msgr2=0x55fd453f44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.307+0000 7fd74a699b80 1 --2- 172.21.15.44:0/260492905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453e50d0 0x55fd453f44c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0098f0 tx=0x7fd72c005400 comp rx=0 tx=0).stop 2023-09-10T00:49:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.307+0000 7fd74a699b80 1 -- 172.21.15.44:0/260492905 shutdown_connections 2023-09-10T00:49:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.307+0000 7fd74a699b80 1 --2- 172.21.15.44:0/260492905 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fd453fe080 0x55fd454004a0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.307+0000 7fd74a699b80 1 --2- 172.21.15.44:0/260492905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453ea5b0 0x55fd453ee1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.307+0000 7fd74a699b80 1 --2- 172.21.15.44:0/260492905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e4710 0x55fd453f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.307+0000 7fd74a699b80 1 --2- 172.21.15.44:0/260492905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453e50d0 0x55fd453f44c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.307+0000 7fd7177fe700 0 client.26767 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:29.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.307+0000 7fd74a699b80 1 -- 172.21.15.44:0/260492905 >> 172.21.15.44:0/260492905 conn(0x55fd45355030 msgr2=0x55fd453da7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:29.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.307+0000 7fd74a699b80 1 -- 172.21.15.44:0/260492905 shutdown_connections 2023-09-10T00:49:29.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.307+0000 7fd74a699b80 1 -- 172.21.15.44:0/260492905 wait complete. 2023-09-10T00:49:29.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.307+0000 7fd74a699b80 20 client.26767 trim_cache size 0 max 16384 2023-09-10T00:49:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.322+0000 7fd74a699b80 1 Processor -- start 2023-09-10T00:49:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.322+0000 7fd74a699b80 1 -- start start 2023-09-10T00:49:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.322+0000 7fd74a699b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e4710 0x55fd453e55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.322+0000 7fd74a699b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453e50d0 0x55fd453daf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.322+0000 7fd74a699b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453ea5b0 0x55fd453db460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.322+0000 7fd74a699b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd453d7fc0 con 0x55fd453e50d0 2023-09-10T00:49:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.322+0000 7fd74a699b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd453dcd10 con 0x55fd453e4710 2023-09-10T00:49:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.322+0000 7fd74a699b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd45400b00 con 0x55fd453ea5b0 2023-09-10T00:49:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.322+0000 7fd737fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e4710 0x55fd453e55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.322+0000 7fd7377fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453e50d0 0x55fd453daf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.322+0000 7fd737fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e4710 0x55fd453e55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53762/0 (socket says 172.21.15.44:53762) 2023-09-10T00:49:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.322+0000 7fd737fff700 1 -- 172.21.15.44:0/1409693869 learned_addr learned my addr 172.21.15.44:0/1409693869 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.322+0000 7fd73ca70700 1 --2- 172.21.15.44:0/1409693869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453ea5b0 0x55fd453db460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.323+0000 7fd737fff700 1 -- 172.21.15.44:0/1409693869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453ea5b0 msgr2=0x55fd453db460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.323+0000 7fd737fff700 1 --2- 172.21.15.44:0/1409693869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453ea5b0 0x55fd453db460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.323+0000 7fd737fff700 1 -- 172.21.15.44:0/1409693869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453e50d0 msgr2=0x55fd453daf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.323+0000 7fd737fff700 1 --2- 172.21.15.44:0/1409693869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453e50d0 0x55fd453daf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.323+0000 7fd737fff700 1 -- 172.21.15.44:0/1409693869 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7381a33d0 con 0x55fd453e4710 2023-09-10T00:49:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.323+0000 7fd7377fe700 1 --2- 172.21.15.44:0/1409693869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453e50d0 0x55fd453daf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.323+0000 7fd73ca70700 1 --2- 172.21.15.44:0/1409693869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453ea5b0 0x55fd453db460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.323+0000 7fd737fff700 1 --2- 172.21.15.44:0/1409693869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e4710 0x55fd453e55c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55fd45400d40 tx=0x7fd728014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0beeab2494bf196 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.323+0000 7fd71effd700 1 -- 172.21.15.44:0/1409693869 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728024040 con 0x55fd453e4710 2023-09-10T00:49:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.323+0000 7fd71effd700 1 -- 172.21.15.44:0/1409693869 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7280295b0 con 0x55fd453e4710 2023-09-10T00:49:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.323+0000 7fd71effd700 1 -- 172.21.15.44:0/1409693869 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72801e830 con 0x55fd453e4710 2023-09-10T00:49:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.324+0000 7fd74a699b80 1 -- 172.21.15.44:0/1409693869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e4710 msgr2=0x55fd453e55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.324+0000 7fd74a699b80 1 --2- 172.21.15.44:0/1409693869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e4710 0x55fd453e55c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55fd45400d40 tx=0x7fd728014000 comp rx=0 tx=0).stop 2023-09-10T00:49:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.324+0000 7fd74a699b80 1 -- 172.21.15.44:0/1409693869 shutdown_connections 2023-09-10T00:49:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.324+0000 7fd74a699b80 1 --2- 172.21.15.44:0/1409693869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453ea5b0 0x55fd453db460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.324+0000 7fd74a699b80 1 --2- 172.21.15.44:0/1409693869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e4710 0x55fd453e55c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.324+0000 7fd74a699b80 1 --2- 172.21.15.44:0/1409693869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453e50d0 0x55fd453daf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.324+0000 7fd74a699b80 1 -- 172.21.15.44:0/1409693869 >> 172.21.15.44:0/1409693869 conn(0x55fd45355030 msgr2=0x55fd453eeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.324+0000 7fd74a699b80 1 -- 172.21.15.44:0/1409693869 shutdown_connections 2023-09-10T00:49:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.324+0000 7fd74a699b80 1 -- 172.21.15.44:0/1409693869 wait complete. 2023-09-10T00:49:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.324+0000 7fd74a699b80 1 -- start start 2023-09-10T00:49:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.325+0000 7fd74a699b80 1 Processor -- start 2023-09-10T00:49:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.325+0000 7fd74a699b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453e4710 0x55fd453fa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.325+0000 7fd74a699b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e50d0 0x55fd453faae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.325+0000 7fd74a699b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453ea5b0 0x55fd453f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.325+0000 7fd74a699b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd453f90d0 con 0x55fd453e4710 2023-09-10T00:49:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.325+0000 7fd74a699b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd453f9240 con 0x55fd453e50d0 2023-09-10T00:49:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.325+0000 7fd74a699b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd453f9400 con 0x55fd453ea5b0 2023-09-10T00:49:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.325+0000 7fd74a699b80 10 client.0 _ll_register_callbacks cb 0x7fd728004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:49:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.325+0000 7fd737fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453e4710 0x55fd453fa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.325+0000 7fd7377fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e50d0 0x55fd453faae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.325+0000 7fd737fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453e4710 0x55fd453fa5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52028/0 (socket says 172.21.15.44:52028) 2023-09-10T00:49:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.325+0000 7fd737fff700 1 -- 172.21.15.44:0/2800493384 learned_addr learned my addr 172.21.15.44:0/2800493384 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.325+0000 7fd73ca70700 1 --2- 172.21.15.44:0/2800493384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453ea5b0 0x55fd453f89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.325+0000 7fd737fff700 1 -- 172.21.15.44:0/2800493384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453ea5b0 msgr2=0x55fd453f89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.326+0000 7fd737fff700 1 --2- 172.21.15.44:0/2800493384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453ea5b0 0x55fd453f89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.326+0000 7fd737fff700 1 -- 172.21.15.44:0/2800493384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e50d0 msgr2=0x55fd453faae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.326+0000 7fd737fff700 1 --2- 172.21.15.44:0/2800493384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e50d0 0x55fd453faae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.326+0000 7fd737fff700 1 -- 172.21.15.44:0/2800493384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd728017070 con 0x55fd453e4710 2023-09-10T00:49:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.326+0000 7fd7377fe700 1 --2- 172.21.15.44:0/2800493384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e50d0 0x55fd453faae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.326+0000 7fd73ca70700 1 --2- 172.21.15.44:0/2800493384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453ea5b0 0x55fd453f89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.326+0000 7fd737fff700 1 --2- 172.21.15.44:0/2800493384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453e4710 0x55fd453fa5a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd7280096f0 tx=0x7fd728004240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfea549c030baeda server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.326+0000 7fd7177fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:49:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.326+0000 7fd7177fe700 1 -- 172.21.15.44:0/2800493384 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728024070 con 0x55fd453e4710 2023-09-10T00:49:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.326+0000 7fd7177fe700 1 -- 172.21.15.44:0/2800493384 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fd728029b00 con 0x55fd453e4710 2023-09-10T00:49:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.326+0000 7fd74a699b80 1 -- 172.21.15.44:0/2800493384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd4532b8c0 con 0x55fd453e4710 2023-09-10T00:49:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.326+0000 7fd7177fe700 1 -- 172.21.15.44:0/2800493384 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd728029cc0 con 0x55fd453e4710 2023-09-10T00:49:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.326+0000 7fd74a699b80 10 client.17169 fetch_fsmap learned FSMap version 54 2023-09-10T00:49:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.326+0000 7fd74a699b80 1 -- 172.21.15.44:0/2800493384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x7fd730003a60 con 0x55fd453e4710 2023-09-10T00:49:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.326+0000 7fd7177fe700 1 -- 172.21.15.44:0/2800493384 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fd4532b8c0 con 0x55fd453e4710 2023-09-10T00:49:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.327+0000 7fd7177fe700 1 -- 172.21.15.44:0/2800493384 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 54) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fd7280049a0 con 0x55fd453e4710 2023-09-10T00:49:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.327+0000 7fd74a699b80 10 client.17169 fetch_fsmap finished waiting for FSMap version 54 2023-09-10T00:49:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.327+0000 7fd74a699b80 10 client.17169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:49:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.327+0000 7fd74a699b80 20 client.17169 populate_metadata read hostname 'smithi044' 2023-09-10T00:49:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.327+0000 7fd74a699b80 1 --2- 172.21.15.44:0/2800493384 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fd454530e0 0x55fd45455500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.327+0000 7fd74a699b80 4 client.17169 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:49:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.327+0000 7fd74a699b80 1 -- 172.21.15.44:0/2800493384 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fd452ec050 con 0x55fd454530e0 2023-09-10T00:49:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.327+0000 7fd7377fe700 1 --2- 172.21.15.44:0/2800493384 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fd454530e0 0x55fd45455500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.327+0000 7fd7177fe700 10 client.17169 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.327+0000 7fd7377fe700 1 --2- 172.21.15.44:0/2800493384 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fd454530e0 0x55fd45455500 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7665e7565f13e5f server_cookie=7e63d2ae1dabfbd5 in_seq=0 out_seq=0 2023-09-10T00:49:29.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.347+0000 7fd7177fe700 1 -- 172.21.15.44:0/2800493384 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fd452ec050 con 0x55fd454530e0 2023-09-10T00:49:29.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.347+0000 7fd7177fe700 10 client.17169 handle_command_reply: tid=0 2023-09-10T00:49:29.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.347+0000 7fd74a699b80 1 client.17169 shutdown 2023-09-10T00:49:29.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.347+0000 7fd74a699b80 1 -- 172.21.15.44:0/2800493384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453e4710 msgr2=0x55fd453fa5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:29.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.347+0000 7fd74a699b80 1 --2- 172.21.15.44:0/2800493384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453e4710 0x55fd453fa5a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd7280096f0 tx=0x7fd728004240 comp rx=0 tx=0).stop 2023-09-10T00:49:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.347+0000 7fd74a699b80 1 -- 172.21.15.44:0/2800493384 shutdown_connections 2023-09-10T00:49:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.347+0000 7fd74a699b80 1 --2- 172.21.15.44:0/2800493384 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fd454530e0 0x55fd45455500 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.347+0000 7fd74a699b80 1 --2- 172.21.15.44:0/2800493384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd453ea5b0 0x55fd453f89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.347+0000 7fd74a699b80 1 --2- 172.21.15.44:0/2800493384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fd453e4710 0x55fd453fa5a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.347+0000 7fd74a699b80 1 --2- 172.21.15.44:0/2800493384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fd453e50d0 0x55fd453faae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.347+0000 7fd74a699b80 1 -- 172.21.15.44:0/2800493384 >> 172.21.15.44:0/2800493384 conn(0x55fd45355030 msgr2=0x55fd453e92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.347+0000 7fd7177fe700 0 client.17169 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.347+0000 7fd74a699b80 1 -- 172.21.15.44:0/2800493384 shutdown_connections 2023-09-10T00:49:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.347+0000 7fd74a699b80 1 -- 172.21.15.44:0/2800493384 wait complete. 2023-09-10T00:49:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.347+0000 7fd74a699b80 20 client.17169 trim_cache size 0 max 16384 2023-09-10T00:49:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:49:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:49:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:49:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:49:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.348+0000 7fd73e4d3700 1 -- 172.21.15.44:0/4255036555 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7200a07b0 msgr2=0x7fd7200a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.348+0000 7fd73e4d3700 1 --2- 172.21.15.44:0/4255036555 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7200a07b0 0x7fd7200a2c70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd728005fd0 tx=0x7fd728009310 comp rx=0 tx=0).stop 2023-09-10T00:49:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.348+0000 7fd73e4d3700 1 -- 172.21.15.44:0/4255036555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd738069320 msgr2=0x7fd73819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.348+0000 7fd73e4d3700 1 --2- 172.21.15.44:0/4255036555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd738069320 0x7fd73819cb00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009230 tx=0x7fd72c00f9e0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.348+0000 7fd73e4d3700 1 -- 172.21.15.44:0/4255036555 shutdown_connections 2023-09-10T00:49:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.348+0000 7fd73e4d3700 1 --2- 172.21.15.44:0/4255036555 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7200a07b0 0x7fd7200a2c70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.348+0000 7fd73e4d3700 1 --2- 172.21.15.44:0/4255036555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd738106670 0x7fd738196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.348+0000 7fd73e4d3700 1 --2- 172.21.15.44:0/4255036555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd738068960 0x7fd73819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.348+0000 7fd73e4d3700 1 --2- 172.21.15.44:0/4255036555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd738069320 0x7fd73819cb00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.348+0000 7fd73e4d3700 1 -- 172.21.15.44:0/4255036555 >> 172.21.15.44:0/4255036555 conn(0x7fd738076600 msgr2=0x7fd7380fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.348+0000 7fd73e4d3700 1 -- 172.21.15.44:0/4255036555 shutdown_connections 2023-09-10T00:49:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:29.348+0000 7fd73e4d3700 1 -- 172.21.15.44:0/4255036555 wait complete. 2023-09-10T00:49:29.609 INFO:teuthology.orchestra.run.smithi044.stderr:33300000 of 50000000 tuples (66%) done (elapsed 509.24 s, remaining 255.39 s) 2023-09-10T00:49:30.592 INFO:teuthology.orchestra.run.smithi044.stderr:33400000 of 50000000 tuples (66%) done (elapsed 510.23 s, remaining 253.59 s) 2023-09-10T00:49:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:30 smithi131 ceph-mon[103713]: pgmap v274: 129 pgs: 129 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 174 op/s 2023-09-10T00:49:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:30 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:30 smithi044 ceph-mon[98676]: pgmap v274: 129 pgs: 129 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 174 op/s 2023-09-10T00:49:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:30 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:30 smithi099 ceph-mon[102692]: pgmap v274: 129 pgs: 129 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 174 op/s 2023-09-10T00:49:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:30 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:32.895 INFO:teuthology.orchestra.run.smithi044.stderr:33500000 of 50000000 tuples (67%) done (elapsed 512.53 s, remaining 252.44 s) 2023-09-10T00:49:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:32 smithi131 ceph-mon[103713]: pgmap v275: 129 pgs: 129 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 39 MiB/s wr, 217 op/s 2023-09-10T00:49:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:32 smithi044 ceph-mon[98676]: pgmap v275: 129 pgs: 129 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 39 MiB/s wr, 217 op/s 2023-09-10T00:49:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:32 smithi099 ceph-mon[102692]: pgmap v275: 129 pgs: 129 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 39 MiB/s wr, 217 op/s 2023-09-10T00:49:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:34 smithi131 ceph-mon[103713]: pgmap v276: 129 pgs: 129 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 32 MiB/s wr, 82 op/s 2023-09-10T00:49:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:34 smithi044 ceph-mon[98676]: pgmap v276: 129 pgs: 129 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 32 MiB/s wr, 82 op/s 2023-09-10T00:49:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:34 smithi099 ceph-mon[102692]: pgmap v276: 129 pgs: 129 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 32 MiB/s wr, 82 op/s 2023-09-10T00:49:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:36 smithi131 ceph-mon[103713]: pgmap v277: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 38 MiB/s wr, 113 op/s 2023-09-10T00:49:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:36 smithi044 ceph-mon[98676]: pgmap v277: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 38 MiB/s wr, 113 op/s 2023-09-10T00:49:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:36 smithi099 ceph-mon[102692]: pgmap v277: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 38 MiB/s wr, 113 op/s 2023-09-10T00:49:37.143 INFO:teuthology.orchestra.run.smithi044.stderr:33600000 of 50000000 tuples (67%) done (elapsed 516.78 s, remaining 252.24 s) 2023-09-10T00:49:38.291 INFO:teuthology.orchestra.run.smithi044.stderr:33700000 of 50000000 tuples (67%) done (elapsed 517.93 s, remaining 250.51 s) 2023-09-10T00:49:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:38 smithi131 ceph-mon[103713]: pgmap v278: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 23 MiB/s wr, 72 op/s 2023-09-10T00:49:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:38 smithi044 ceph-mon[98676]: pgmap v278: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 23 MiB/s wr, 72 op/s 2023-09-10T00:49:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:38 smithi099 ceph-mon[102692]: pgmap v278: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 23 MiB/s wr, 72 op/s 2023-09-10T00:49:39.737 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:49:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.880+0000 7fb54454b700 1 -- 172.21.15.44:0/127675513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb53c0ffe00 msgr2=0x7fb53c10c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.880+0000 7fb54454b700 1 --2- 172.21.15.44:0/127675513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb53c0ffe00 0x7fb53c10c310 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb53400b0a0 tx=0x7fb53400b3b0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.880+0000 7fb54454b700 1 -- 172.21.15.44:0/127675513 shutdown_connections 2023-09-10T00:49:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.880+0000 7fb54454b700 1 --2- 172.21.15.44:0/127675513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb53c0ffe00 0x7fb53c10c310 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.880+0000 7fb54454b700 1 --2- 172.21.15.44:0/127675513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53c1054e0 0x7fb53c1058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.880+0000 7fb54454b700 1 --2- 172.21.15.44:0/127675513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb53c0ff440 0x7fb53c0ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.880+0000 7fb54454b700 1 -- 172.21.15.44:0/127675513 >> 172.21.15.44:0/127675513 conn(0x7fb53c0faf50 msgr2=0x7fb53c0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.880+0000 7fb54454b700 1 -- 172.21.15.44:0/127675513 shutdown_connections 2023-09-10T00:49:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.880+0000 7fb54454b700 1 -- 172.21.15.44:0/127675513 wait complete. 2023-09-10T00:49:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb54454b700 1 Processor -- start 2023-09-10T00:49:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb54454b700 1 -- start start 2023-09-10T00:49:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb54454b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53c0ff440 0x7fb53c197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb54454b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb53c0ffe00 0x7fb53c1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb54454b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb53c1054e0 0x7fb53c1a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb54454b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c1981f0 con 0x7fb53c0ffe00 2023-09-10T00:49:39.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb54454b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c1a2930 con 0x7fb53c0ff440 2023-09-10T00:49:39.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb54454b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c1a2af0 con 0x7fb53c1054e0 2023-09-10T00:49:39.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb541ae6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb53c0ffe00 0x7fb53c1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb5422e7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53c0ff440 0x7fb53c197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.887 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb5422e7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53c0ff440 0x7fb53c197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53782/0 (socket says 172.21.15.44:53782) 2023-09-10T00:49:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb5422e7700 1 -- 172.21.15.44:0/2821604469 learned_addr learned my addr 172.21.15.44:0/2821604469 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb541ae6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb53c0ffe00 0x7fb53c1978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52044/0 (socket says 172.21.15.44:52044) 2023-09-10T00:49:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb542ae8700 1 --2- 172.21.15.44:0/2821604469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb53c1054e0 0x7fb53c1a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb5422e7700 1 -- 172.21.15.44:0/2821604469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb53c1054e0 msgr2=0x7fb53c1a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb5422e7700 1 --2- 172.21.15.44:0/2821604469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb53c1054e0 0x7fb53c1a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb5422e7700 1 -- 172.21.15.44:0/2821604469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb53c0ffe00 msgr2=0x7fb53c1978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb5422e7700 1 --2- 172.21.15.44:0/2821604469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb53c0ffe00 0x7fb53c1978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb5422e7700 1 -- 172.21.15.44:0/2821604469 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb534009d00 con 0x7fb53c0ff440 2023-09-10T00:49:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.881+0000 7fb541ae6700 1 --2- 172.21.15.44:0/2821604469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb53c0ffe00 0x7fb53c1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.882+0000 7fb5422e7700 1 --2- 172.21.15.44:0/2821604469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53c0ff440 0x7fb53c197370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb53000e9a0 tx=0x7fb53000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.882+0000 7fb52f7fe700 1 -- 172.21.15.44:0/2821604469 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530009790 con 0x7fb53c0ff440 2023-09-10T00:49:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.882+0000 7fb52f7fe700 1 -- 172.21.15.44:0/2821604469 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb530010430 con 0x7fb53c0ff440 2023-09-10T00:49:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.882+0000 7fb54454b700 1 -- 172.21.15.44:0/2821604469 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53c1a2dd0 con 0x7fb53c0ff440 2023-09-10T00:49:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.882+0000 7fb52f7fe700 1 -- 172.21.15.44:0/2821604469 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53000cea0 con 0x7fb53c0ff440 2023-09-10T00:49:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.882+0000 7fb54454b700 1 -- 172.21.15.44:0/2821604469 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53c1a3260 con 0x7fb53c0ff440 2023-09-10T00:49:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.884+0000 7fb52f7fe700 1 -- 172.21.15.44:0/2821604469 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fb5300105a0 con 0x7fb53c0ff440 2023-09-10T00:49:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.885+0000 7fb52f7fe700 1 --2- 172.21.15.44:0/2821604469 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5280a0830 0x7fb5280a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.885+0000 7fb541ae6700 1 --2- 172.21.15.44:0/2821604469 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5280a0830 0x7fb5280a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.885+0000 7fb52f7fe700 1 -- 172.21.15.44:0/2821604469 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb530014070 con 0x7fb53c0ff440 2023-09-10T00:49:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.885+0000 7fb541ae6700 1 --2- 172.21.15.44:0/2821604469 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5280a0830 0x7fb5280a2cf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb53c068030 tx=0x7fb538009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.886+0000 7fb550711b80 1 Processor -- start 2023-09-10T00:49:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.886+0000 7fb550711b80 1 -- start start 2023-09-10T00:49:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.886+0000 7fb550711b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa1710 0x55fe8bfa1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.886+0000 7fb550711b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfa2030 0x55fe8bfafd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.886+0000 7fb550711b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfb0250 0x55fe8bfb2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.886+0000 7fb550711b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8befa730 con 0x55fe8bfb0250 2023-09-10T00:49:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.886+0000 7fb550711b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8bef8050 con 0x55fe8bfa2030 2023-09-10T00:49:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.886+0000 7fb550711b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8bef3940 con 0x55fe8bfa1710 2023-09-10T00:49:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.886+0000 7fb5422e7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa1710 0x55fe8bfa1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.886+0000 7fb542ae8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfb0250 0x55fe8bfb2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.886+0000 7fb541ae6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfa2030 0x55fe8bfafd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.886+0000 7fb5422e7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa1710 0x55fe8bfa1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58932/0 (socket says 172.21.15.44:58932) 2023-09-10T00:49:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.886+0000 7fb5422e7700 1 -- 172.21.15.44:0/2277746580 learned_addr learned my addr 172.21.15.44:0/2277746580 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.887+0000 7fb5422e7700 1 -- 172.21.15.44:0/2277746580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfa2030 msgr2=0x55fe8bfafd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.887+0000 7fb5422e7700 1 --2- 172.21.15.44:0/2277746580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfa2030 0x55fe8bfafd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.887+0000 7fb5422e7700 1 -- 172.21.15.44:0/2277746580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfb0250 msgr2=0x55fe8bfb2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.887+0000 7fb5422e7700 1 --2- 172.21.15.44:0/2277746580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfb0250 0x55fe8bfb2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.887+0000 7fb5422e7700 1 -- 172.21.15.44:0/2277746580 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe8bd1fde0 con 0x55fe8bfa1710 2023-09-10T00:49:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.887+0000 7fb541ae6700 1 --2- 172.21.15.44:0/2277746580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfa2030 0x55fe8bfafd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.887+0000 7fb5422e7700 1 --2- 172.21.15.44:0/2277746580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa1710 0x55fe8bfa1af0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb5300a0ff0 tx=0x7fb530098e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef918af3ba308cc2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.887+0000 7fb52cff9700 1 -- 172.21.15.44:0/2277746580 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300c9ea0 con 0x55fe8bfa1710 2023-09-10T00:49:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.887+0000 7fb52cff9700 1 -- 172.21.15.44:0/2277746580 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5300994b0 con 0x55fe8bfa1710 2023-09-10T00:49:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.887+0000 7fb52cff9700 1 -- 172.21.15.44:0/2277746580 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530099710 con 0x55fe8bfa1710 2023-09-10T00:49:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.888+0000 7fb550711b80 1 -- 172.21.15.44:0/2277746580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa1710 msgr2=0x55fe8bfa1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.888+0000 7fb550711b80 1 --2- 172.21.15.44:0/2277746580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa1710 0x55fe8bfa1af0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb5300a0ff0 tx=0x7fb530098e90 comp rx=0 tx=0).stop 2023-09-10T00:49:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.888+0000 7fb550711b80 1 -- 172.21.15.44:0/2277746580 shutdown_connections 2023-09-10T00:49:39.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.888+0000 7fb550711b80 1 --2- 172.21.15.44:0/2277746580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfb0250 0x55fe8bfb2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.888+0000 7fb550711b80 1 --2- 172.21.15.44:0/2277746580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa1710 0x55fe8bfa1af0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.888+0000 7fb550711b80 1 --2- 172.21.15.44:0/2277746580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfa2030 0x55fe8bfafd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.888+0000 7fb550711b80 1 -- 172.21.15.44:0/2277746580 >> 172.21.15.44:0/2277746580 conn(0x55fe8bf16090 msgr2=0x55fe8bf9cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:39.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.888+0000 7fb550711b80 1 -- 172.21.15.44:0/2277746580 shutdown_connections 2023-09-10T00:49:39.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.888+0000 7fb550711b80 1 -- 172.21.15.44:0/2277746580 wait complete. 2023-09-10T00:49:39.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.888+0000 7fb550711b80 1 -- start start 2023-09-10T00:49:39.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.888+0000 7fb550711b80 1 Processor -- start 2023-09-10T00:49:39.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb550711b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa1710 0x55fe8bfacf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb550711b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfa2030 0x55fe8bfad490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb550711b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfb0250 0x55fe8bfad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb550711b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8be91b70 con 0x55fe8bfa2030 2023-09-10T00:49:39.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb550711b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8be91870 con 0x55fe8bfb0250 2023-09-10T00:49:39.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb550711b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8bd75920 con 0x55fe8bfa1710 2023-09-10T00:49:39.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb550711b80 10 client.0 _ll_register_callbacks cb 0x55fe8bddba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:49:39.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb5422e7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa1710 0x55fe8bfacf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb541ae6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfa2030 0x55fe8bfad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb5422e7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa1710 0x55fe8bfacf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58938/0 (socket says 172.21.15.44:58938) 2023-09-10T00:49:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb5422e7700 1 -- 172.21.15.44:0/1940212406 learned_addr learned my addr 172.21.15.44:0/1940212406 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb542ae8700 1 --2- 172.21.15.44:0/1940212406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfb0250 0x55fe8bfad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb5422e7700 1 -- 172.21.15.44:0/1940212406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfb0250 msgr2=0x55fe8bfad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb5422e7700 1 --2- 172.21.15.44:0/1940212406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfb0250 0x55fe8bfad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb5422e7700 1 -- 172.21.15.44:0/1940212406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfa2030 msgr2=0x55fe8bfad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb5422e7700 1 --2- 172.21.15.44:0/1940212406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfa2030 0x55fe8bfad490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb5422e7700 1 -- 172.21.15.44:0/1940212406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb538017070 con 0x55fe8bfa1710 2023-09-10T00:49:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb5422e7700 1 --2- 172.21.15.44:0/1940212406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa1710 0x55fe8bfacf50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb5300991b0 tx=0x7fb53009d170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc5747353aae1669 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb525ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:49:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb525ffb700 1 -- 172.21.15.44:0/1940212406 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300c9ea0 con 0x55fe8bfa1710 2023-09-10T00:49:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb525ffb700 1 -- 172.21.15.44:0/1940212406 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb53009d2e0 con 0x55fe8bfa1710 2023-09-10T00:49:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.889+0000 7fb550711b80 1 -- 172.21.15.44:0/1940212406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe8be4b4f0 con 0x55fe8bfa1710 2023-09-10T00:49:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.890+0000 7fb525ffb700 1 -- 172.21.15.44:0/1940212406 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb530099710 con 0x55fe8bfa1710 2023-09-10T00:49:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.890+0000 7fb550711b80 10 client.26785 fetch_fsmap learned FSMap version 55 2023-09-10T00:49:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.890+0000 7fb550711b80 1 -- 172.21.15.44:0/1940212406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7fb53c1a3260 con 0x55fe8bfa1710 2023-09-10T00:49:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.890+0000 7fb525ffb700 1 -- 172.21.15.44:0/1940212406 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fe8be4b4f0 con 0x55fe8bfa1710 2023-09-10T00:49:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.890+0000 7fb525ffb700 1 -- 172.21.15.44:0/1940212406 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 55) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb5300953c0 con 0x55fe8bfa1710 2023-09-10T00:49:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.890+0000 7fb550711b80 10 client.26785 fetch_fsmap finished waiting for FSMap version 55 2023-09-10T00:49:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.890+0000 7fb550711b80 10 client.26785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:49:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.890+0000 7fb550711b80 20 client.26785 populate_metadata read hostname 'smithi044' 2023-09-10T00:49:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.890+0000 7fb550711b80 1 --2- 172.21.15.44:0/1940212406 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe8bfbf9a0 0x55fe8bfc1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.890+0000 7fb550711b80 4 client.26785 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:49:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.890+0000 7fb550711b80 1 -- 172.21.15.44:0/1940212406 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe8bf088a0 con 0x55fe8bfbf9a0 2023-09-10T00:49:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.890+0000 7fb541ae6700 1 --2- 172.21.15.44:0/1940212406 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe8bfbf9a0 0x55fe8bfc1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb525ffb700 10 client.26785 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb541ae6700 1 --2- 172.21.15.44:0/1940212406 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe8bfbf9a0 0x55fe8bfc1dc0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=994eb2c8163e6e9c server_cookie=9033bf6da9ecbd9c in_seq=0 out_seq=0 2023-09-10T00:49:39.910 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb525ffb700 1 -- 172.21.15.44:0/1940212406 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fe8bf088a0 con 0x55fe8bfbf9a0 2023-09-10T00:49:39.910 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb525ffb700 10 client.26785 handle_command_reply: tid=0 2023-09-10T00:49:39.910 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb550711b80 1 client.26785 shutdown 2023-09-10T00:49:39.910 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb550711b80 1 -- 172.21.15.44:0/1940212406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa1710 msgr2=0x55fe8bfacf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:39.910 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb550711b80 1 --2- 172.21.15.44:0/1940212406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa1710 0x55fe8bfacf50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb5300991b0 tx=0x7fb53009d170 comp rx=0 tx=0).stop 2023-09-10T00:49:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb550711b80 1 -- 172.21.15.44:0/1940212406 shutdown_connections 2023-09-10T00:49:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb550711b80 1 --2- 172.21.15.44:0/1940212406 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe8bfbf9a0 0x55fe8bfc1dc0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb550711b80 1 --2- 172.21.15.44:0/1940212406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfb0250 0x55fe8bfad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb525ffb700 0 client.26785 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb550711b80 1 --2- 172.21.15.44:0/1940212406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa1710 0x55fe8bfacf50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb550711b80 1 --2- 172.21.15.44:0/1940212406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfa2030 0x55fe8bfad490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb550711b80 1 -- 172.21.15.44:0/1940212406 >> 172.21.15.44:0/1940212406 conn(0x55fe8bf16090 msgr2=0x55fe8bf9c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:39.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb550711b80 1 -- 172.21.15.44:0/1940212406 shutdown_connections 2023-09-10T00:49:39.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb550711b80 1 -- 172.21.15.44:0/1940212406 wait complete. 2023-09-10T00:49:39.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.891+0000 7fb550711b80 20 client.26785 trim_cache size 0 max 16384 2023-09-10T00:49:39.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb550711b80 1 Processor -- start 2023-09-10T00:49:39.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb550711b80 1 -- start start 2023-09-10T00:49:39.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb550711b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfa1710 0x55fe8bfad800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.918 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb550711b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa2030 0x55fe8bfbb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb550711b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfb0250 0x55fe8bf9c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb550711b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8bf9b9a0 con 0x55fe8bfb0250 2023-09-10T00:49:39.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb550711b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8bf9d6e0 con 0x55fe8bfa1710 2023-09-10T00:49:39.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb550711b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8bfc6ac0 con 0x55fe8bfa2030 2023-09-10T00:49:39.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb542ae8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfb0250 0x55fe8bf9c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.919 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb5422e7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfa1710 0x55fe8bfad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb541ae6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa2030 0x55fe8bfbb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb5422e7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfa1710 0x55fe8bfad800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53804/0 (socket says 172.21.15.44:53804) 2023-09-10T00:49:39.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb5422e7700 1 -- 172.21.15.44:0/1350334905 learned_addr learned my addr 172.21.15.44:0/1350334905 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:39.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb542ae8700 1 -- 172.21.15.44:0/1350334905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa2030 msgr2=0x55fe8bfbb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:39.920 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb542ae8700 1 --2- 172.21.15.44:0/1350334905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa2030 0x55fe8bfbb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb542ae8700 1 -- 172.21.15.44:0/1350334905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfa1710 msgr2=0x55fe8bfad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:39.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb542ae8700 1 --2- 172.21.15.44:0/1350334905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfa1710 0x55fe8bfad800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb542ae8700 1 -- 172.21.15.44:0/1350334905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe8bd08c60 con 0x55fe8bfb0250 2023-09-10T00:49:39.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.909+0000 7fb5422e7700 1 --2- 172.21.15.44:0/1350334905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfa1710 0x55fe8bfad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:39.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.910+0000 7fb542ae8700 1 --2- 172.21.15.44:0/1350334905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfb0250 0x55fe8bf9c6f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb53400b070 tx=0x7fb53400bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725e3a7b472bd7c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:39.921 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.910+0000 7fb52d7fa700 1 -- 172.21.15.44:0/1350334905 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53400e040 con 0x55fe8bfb0250 2023-09-10T00:49:39.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.910+0000 7fb52d7fa700 1 -- 172.21.15.44:0/1350334905 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb534003a00 con 0x55fe8bfb0250 2023-09-10T00:49:39.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.910+0000 7fb52d7fa700 1 -- 172.21.15.44:0/1350334905 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340053b0 con 0x55fe8bfb0250 2023-09-10T00:49:39.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.910+0000 7fb550711b80 1 -- 172.21.15.44:0/1350334905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfb0250 msgr2=0x55fe8bf9c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:39.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.910+0000 7fb550711b80 1 --2- 172.21.15.44:0/1350334905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfb0250 0x55fe8bf9c6f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb53400b070 tx=0x7fb53400bb60 comp rx=0 tx=0).stop 2023-09-10T00:49:39.922 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.910+0000 7fb550711b80 1 -- 172.21.15.44:0/1350334905 shutdown_connections 2023-09-10T00:49:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.910+0000 7fb550711b80 1 --2- 172.21.15.44:0/1350334905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfb0250 0x55fe8bf9c6f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.910+0000 7fb550711b80 1 --2- 172.21.15.44:0/1350334905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfa1710 0x55fe8bfad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.910+0000 7fb550711b80 1 --2- 172.21.15.44:0/1350334905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa2030 0x55fe8bfbb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.910+0000 7fb550711b80 1 -- 172.21.15.44:0/1350334905 >> 172.21.15.44:0/1350334905 conn(0x55fe8bf16090 msgr2=0x55fe8bfbc050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.910+0000 7fb550711b80 1 -- 172.21.15.44:0/1350334905 shutdown_connections 2023-09-10T00:49:39.923 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.910+0000 7fb550711b80 1 -- 172.21.15.44:0/1350334905 wait complete. 2023-09-10T00:49:39.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb550711b80 1 -- start start 2023-09-10T00:49:39.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb550711b80 1 Processor -- start 2023-09-10T00:49:39.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb550711b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfa1710 0x55fe8bfb5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb550711b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa2030 0x55fe8bfb5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.924 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb550711b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfb0250 0x55fe8bfb5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb550711b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8c013850 con 0x55fe8bfa1710 2023-09-10T00:49:39.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb550711b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8c013990 con 0x55fe8bfb0250 2023-09-10T00:49:39.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb550711b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8c013b50 con 0x55fe8bfa2030 2023-09-10T00:49:39.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb550711b80 10 client.0 _ll_register_callbacks cb 0x7fb530005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:49:39.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb5422e7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfa1710 0x55fe8bfb5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.925 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb542ae8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfb0250 0x55fe8bfb5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb5422e7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfa1710 0x55fe8bfb5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52070/0 (socket says 172.21.15.44:52070) 2023-09-10T00:49:39.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb542ae8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfb0250 0x55fe8bfb5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53814/0 (socket says 172.21.15.44:53814) 2023-09-10T00:49:39.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb5422e7700 1 -- 172.21.15.44:0/875467590 learned_addr learned my addr 172.21.15.44:0/875467590 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:39.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb541ae6700 1 --2- 172.21.15.44:0/875467590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa2030 0x55fe8bfb5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.926 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb542ae8700 1 -- 172.21.15.44:0/875467590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa2030 msgr2=0x55fe8bfb5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb542ae8700 1 --2- 172.21.15.44:0/875467590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa2030 0x55fe8bfb5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.911+0000 7fb542ae8700 1 -- 172.21.15.44:0/875467590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfa1710 msgr2=0x55fe8bfb5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.912+0000 7fb542ae8700 1 --2- 172.21.15.44:0/875467590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfa1710 0x55fe8bfb5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.912+0000 7fb542ae8700 1 -- 172.21.15.44:0/875467590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53c1a3260 con 0x55fe8bfb0250 2023-09-10T00:49:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.912+0000 7fb5422e7700 1 --2- 172.21.15.44:0/875467590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfa1710 0x55fe8bfb5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:39.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.912+0000 7fb541ae6700 1 --2- 172.21.15.44:0/875467590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa2030 0x55fe8bfb5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:39.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.912+0000 7fb542ae8700 1 --2- 172.21.15.44:0/875467590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfb0250 0x55fe8bfb5fb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb534009fd0 tx=0x7fb534003e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=576bbb1ee5e6543b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:39.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.912+0000 7fb525ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:49:39.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.912+0000 7fb525ffb700 1 -- 172.21.15.44:0/875467590 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53400e070 con 0x55fe8bfb0250 2023-09-10T00:49:39.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.912+0000 7fb525ffb700 1 -- 172.21.15.44:0/875467590 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb53402e460 con 0x55fe8bfb0250 2023-09-10T00:49:39.928 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.912+0000 7fb550711b80 1 -- 172.21.15.44:0/875467590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5300994b0 con 0x55fe8bfb0250 2023-09-10T00:49:39.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.912+0000 7fb525ffb700 1 -- 172.21.15.44:0/875467590 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb534007930 con 0x55fe8bfb0250 2023-09-10T00:49:39.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.912+0000 7fb550711b80 10 client.26849 fetch_fsmap learned FSMap version 55 2023-09-10T00:49:39.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.912+0000 7fb550711b80 1 -- 172.21.15.44:0/875467590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7fb53c10cc10 con 0x55fe8bfb0250 2023-09-10T00:49:39.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.912+0000 7fb525ffb700 1 -- 172.21.15.44:0/875467590 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5300994b0 con 0x55fe8bfb0250 2023-09-10T00:49:39.929 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.912+0000 7fb525ffb700 1 -- 172.21.15.44:0/875467590 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 55) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb53400b0d0 con 0x55fe8bfb0250 2023-09-10T00:49:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb550711b80 10 client.26849 fetch_fsmap finished waiting for FSMap version 55 2023-09-10T00:49:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb550711b80 10 client.26849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:49:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb550711b80 20 client.26849 populate_metadata read hostname 'smithi044' 2023-09-10T00:49:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb550711b80 1 --2- 172.21.15.44:0/875467590 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe8c014f20 0x55fe8c017310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb550711b80 4 client.26849 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:49:39.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb550711b80 1 -- 172.21.15.44:0/875467590 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fe8beec920 con 0x55fe8c014f20 2023-09-10T00:49:39.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb5422e7700 1 --2- 172.21.15.44:0/875467590 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe8c014f20 0x55fe8c017310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:39.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb5422e7700 1 --2- 172.21.15.44:0/875467590 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe8c014f20 0x55fe8c017310 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c8272b43e80206 server_cookie=a739ef0efc16800f in_seq=0 out_seq=0 2023-09-10T00:49:39.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb525ffb700 10 client.26849 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:39.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb525ffb700 1 -- 172.21.15.44:0/875467590 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fe8beec920 con 0x55fe8c014f20 2023-09-10T00:49:39.931 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb525ffb700 10 client.26849 handle_command_reply: tid=0 2023-09-10T00:49:39.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb550711b80 1 client.26849 shutdown 2023-09-10T00:49:39.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb550711b80 1 -- 172.21.15.44:0/875467590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfb0250 msgr2=0x55fe8bfb5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:39.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb550711b80 1 --2- 172.21.15.44:0/875467590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfb0250 0x55fe8bfb5fb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb534009fd0 tx=0x7fb534003e20 comp rx=0 tx=0).stop 2023-09-10T00:49:39.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb550711b80 1 -- 172.21.15.44:0/875467590 shutdown_connections 2023-09-10T00:49:39.932 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb550711b80 1 --2- 172.21.15.44:0/875467590 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe8c014f20 0x55fe8c017310 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb550711b80 1 --2- 172.21.15.44:0/875467590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe8bfb0250 0x55fe8bfb5fb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb525ffb700 0 client.26849 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:39.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb550711b80 1 --2- 172.21.15.44:0/875467590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe8bfa1710 0x55fe8bfb5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb550711b80 1 --2- 172.21.15.44:0/875467590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe8bfa2030 0x55fe8bfb5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb550711b80 1 -- 172.21.15.44:0/875467590 >> 172.21.15.44:0/875467590 conn(0x55fe8bf16090 msgr2=0x55fe8bfbb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:39.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.913+0000 7fb550711b80 1 -- 172.21.15.44:0/875467590 shutdown_connections 2023-09-10T00:49:39.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.914+0000 7fb550711b80 1 -- 172.21.15.44:0/875467590 wait complete. 2023-09-10T00:49:39.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.914+0000 7fb550711b80 20 client.26849 trim_cache size 0 max 16384 2023-09-10T00:49:39.934 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:49:39.934 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:49:39.935 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:49:39.935 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:49:39.935 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:49:39.935 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:49:39.935 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:49:39.936 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:49:39.936 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:49:39.936 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:49:39.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.914+0000 7fb54454b700 1 -- 172.21.15.44:0/2821604469 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5280a0830 msgr2=0x7fb5280a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:39.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.914+0000 7fb54454b700 1 --2- 172.21.15.44:0/2821604469 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5280a0830 0x7fb5280a2cf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb53c068030 tx=0x7fb538009260 comp rx=0 tx=0).stop 2023-09-10T00:49:39.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.914+0000 7fb54454b700 1 -- 172.21.15.44:0/2821604469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53c0ff440 msgr2=0x7fb53c197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:39.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.914+0000 7fb54454b700 1 --2- 172.21.15.44:0/2821604469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53c0ff440 0x7fb53c197370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb53000e9a0 tx=0x7fb53000ecb0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.914+0000 7fb54454b700 1 -- 172.21.15.44:0/2821604469 shutdown_connections 2023-09-10T00:49:39.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.914+0000 7fb54454b700 1 --2- 172.21.15.44:0/2821604469 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5280a0830 0x7fb5280a2cf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.914+0000 7fb54454b700 1 --2- 172.21.15.44:0/2821604469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb53c1054e0 0x7fb53c1a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.914+0000 7fb54454b700 1 --2- 172.21.15.44:0/2821604469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53c0ff440 0x7fb53c197370 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.914+0000 7fb54454b700 1 --2- 172.21.15.44:0/2821604469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb53c0ffe00 0x7fb53c1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:39.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.914+0000 7fb54454b700 1 -- 172.21.15.44:0/2821604469 >> 172.21.15.44:0/2821604469 conn(0x7fb53c0faf50 msgr2=0x7fb53c0fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:39.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.914+0000 7fb54454b700 1 -- 172.21.15.44:0/2821604469 shutdown_connections 2023-09-10T00:49:39.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:49:39.914+0000 7fb54454b700 1 -- 172.21.15.44:0/2821604469 wait complete. 2023-09-10T00:49:39.942 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:49:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:40 smithi131 ceph-mon[103713]: pgmap v279: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 23 MiB/s wr, 72 op/s 2023-09-10T00:49:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:40 smithi044 ceph-mon[98676]: pgmap v279: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 23 MiB/s wr, 72 op/s 2023-09-10T00:49:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:40 smithi099 ceph-mon[102692]: pgmap v279: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 23 MiB/s wr, 72 op/s 2023-09-10T00:49:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.477+0000 7f1596b1e700 1 -- 172.21.15.44:0/2747295115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1590100f60 msgr2=0x7f159010d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.477+0000 7f1596b1e700 1 --2- 172.21.15.44:0/2747295115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1590100f60 0x7f159010d420 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1588009fc0 tx=0x7f158800b330 comp rx=0 tx=0).stop 2023-09-10T00:49:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.477+0000 7f1596b1e700 1 -- 172.21.15.44:0/2747295115 shutdown_connections 2023-09-10T00:49:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.477+0000 7f1596b1e700 1 --2- 172.21.15.44:0/2747295115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1590100f60 0x7f159010d420 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.477+0000 7f1596b1e700 1 --2- 172.21.15.44:0/2747295115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1590106660 0x7f1590106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.477+0000 7f1596b1e700 1 --2- 172.21.15.44:0/2747295115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15901005a0 0x7f1590100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.477+0000 7f1596b1e700 1 -- 172.21.15.44:0/2747295115 >> 172.21.15.44:0/2747295115 conn(0x7f15900fc0b0 msgr2=0x7f15900fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.477+0000 7f1596b1e700 1 -- 172.21.15.44:0/2747295115 shutdown_connections 2023-09-10T00:49:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.477+0000 7f1596b1e700 1 -- 172.21.15.44:0/2747295115 wait complete. 2023-09-10T00:49:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.478+0000 7f1596b1e700 1 Processor -- start 2023-09-10T00:49:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.478+0000 7f1596b1e700 1 -- start start 2023-09-10T00:49:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.478+0000 7f1596b1e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15901005a0 0x7f1590072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.478+0000 7f1596b1e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1590100f60 0x7f1590073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.478+0000 7f1596b1e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1590106660 0x7f15901a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.478+0000 7f1596b1e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15901a4dc0 con 0x7f1590106660 2023-09-10T00:49:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.478+0000 7f1596b1e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15901a4f00 con 0x7f15901005a0 2023-09-10T00:49:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.478+0000 7f1596b1e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15901a5070 con 0x7f1590100f60 2023-09-10T00:49:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.478+0000 7f15948ba700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15901005a0 0x7f1590072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.478+0000 7f15950bb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1590106660 0x7f15901a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.478+0000 7f158ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1590100f60 0x7f1590073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.478+0000 7f15948ba700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15901005a0 0x7f1590072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53824/0 (socket says 172.21.15.44:53824) 2023-09-10T00:49:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.478+0000 7f15948ba700 1 -- 172.21.15.44:0/3388502257 learned_addr learned my addr 172.21.15.44:0/3388502257 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.478+0000 7f15950bb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1590106660 0x7f15901a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52088/0 (socket says 172.21.15.44:52088) 2023-09-10T00:49:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.479+0000 7f15948ba700 1 -- 172.21.15.44:0/3388502257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1590100f60 msgr2=0x7f1590073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.479+0000 7f15948ba700 1 --2- 172.21.15.44:0/3388502257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1590100f60 0x7f1590073020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.479+0000 7f15948ba700 1 -- 172.21.15.44:0/3388502257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1590106660 msgr2=0x7f15901a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.479+0000 7f15948ba700 1 --2- 172.21.15.44:0/3388502257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1590106660 0x7f15901a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.479+0000 7f15948ba700 1 -- 172.21.15.44:0/3388502257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1588009c40 con 0x7f15901005a0 2023-09-10T00:49:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.479+0000 7f15950bb700 1 --2- 172.21.15.44:0/3388502257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1590106660 0x7f15901a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.479+0000 7f15948ba700 1 --2- 172.21.15.44:0/3388502257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15901005a0 0x7f1590072ae0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f158400e9c0 tx=0x7f158400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.479+0000 7f158dffb700 1 -- 172.21.15.44:0/3388502257 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1584004a30 con 0x7f15901005a0 2023-09-10T00:49:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.479+0000 7f158dffb700 1 -- 172.21.15.44:0/3388502257 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f158400cc50 con 0x7f15901005a0 2023-09-10T00:49:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.479+0000 7f1596b1e700 1 -- 172.21.15.44:0/3388502257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15901a5350 con 0x7f15901005a0 2023-09-10T00:49:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.479+0000 7f158dffb700 1 -- 172.21.15.44:0/3388502257 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1584005410 con 0x7f15901005a0 2023-09-10T00:49:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.479+0000 7f1596b1e700 1 -- 172.21.15.44:0/3388502257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15901a58b0 con 0x7f15901005a0 2023-09-10T00:49:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.481+0000 7f158dffb700 1 -- 172.21.15.44:0/3388502257 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1584005570 con 0x7f15901005a0 2023-09-10T00:49:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.482+0000 7f158dffb700 1 --2- 172.21.15.44:0/3388502257 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f15800a0830 0x7f15800a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.482+0000 7f158dffb700 1 -- 172.21.15.44:0/3388502257 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1584014070 con 0x7f15901005a0 2023-09-10T00:49:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.482+0000 7f158ffff700 1 --2- 172.21.15.44:0/3388502257 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f15800a0830 0x7f15800a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.483+0000 7f158ffff700 1 --2- 172.21.15.44:0/3388502257 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f15800a0830 0x7f15800a2cf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1590074300 tx=0x7f157c009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.483+0000 7f15a2ce4b80 1 Processor -- start 2023-09-10T00:49:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.483+0000 7f15a2ce4b80 1 -- start start 2023-09-10T00:49:42.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15a2ce4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee901a0 0x5569aee90580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15a2ce4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 0x5569aee8a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15a2ce4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee8acc0 0x5569aee96f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15a2ce4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569aede36d0 con 0x5569aee8a300 2023-09-10T00:49:42.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15a2ce4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569aede0ff0 con 0x5569aee901a0 2023-09-10T00:49:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15a2ce4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569aeddc8e0 con 0x5569aee8acc0 2023-09-10T00:49:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15948ba700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee901a0 0x5569aee90580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15950bb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee8acc0 0x5569aee96f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15950bb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee8acc0 0x5569aee96f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:58976/0 (socket says 172.21.15.44:58976) 2023-09-10T00:49:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15950bb700 1 -- 172.21.15.44:0/2094108368 learned_addr learned my addr 172.21.15.44:0/2094108368 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15948ba700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee901a0 0x5569aee90580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53832/0 (socket says 172.21.15.44:53832) 2023-09-10T00:49:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15950bb700 1 -- 172.21.15.44:0/2094108368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee901a0 msgr2=0x5569aee90580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15950bb700 1 --2- 172.21.15.44:0/2094108368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee901a0 0x5569aee90580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f158ffff700 1 --2- 172.21.15.44:0/2094108368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 0x5569aee8a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15950bb700 1 -- 172.21.15.44:0/2094108368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 msgr2=0x5569aee8a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15950bb700 1 --2- 172.21.15.44:0/2094108368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 0x5569aee8a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15950bb700 1 -- 172.21.15.44:0/2094108368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569aec08d80 con 0x5569aee8acc0 2023-09-10T00:49:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f15948ba700 1 --2- 172.21.15.44:0/2094108368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee901a0 0x5569aee90580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.484+0000 7f158ffff700 1 --2- 172.21.15.44:0/2094108368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 0x5569aee8a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:49:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.485+0000 7f15950bb700 1 --2- 172.21.15.44:0/2094108368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee8acc0 0x5569aee96f30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1588009b70 tx=0x7f1588009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2df4df479736ad8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.485+0000 7f157affd700 1 -- 172.21.15.44:0/2094108368 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800e040 con 0x5569aee8acc0 2023-09-10T00:49:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.485+0000 7f157affd700 1 -- 172.21.15.44:0/2094108368 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1588003e90 con 0x5569aee8acc0 2023-09-10T00:49:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.485+0000 7f157affd700 1 -- 172.21.15.44:0/2094108368 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588005040 con 0x5569aee8acc0 2023-09-10T00:49:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.485+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/2094108368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee8acc0 msgr2=0x5569aee96f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.485+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/2094108368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee8acc0 0x5569aee96f30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1588009b70 tx=0x7f1588009ba0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.486+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/2094108368 shutdown_connections 2023-09-10T00:49:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.486+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/2094108368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee8acc0 0x5569aee96f30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.486+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/2094108368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee901a0 0x5569aee90580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.486+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/2094108368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 0x5569aee8a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.486+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/2094108368 >> 172.21.15.44:0/2094108368 conn(0x5569aedff030 msgr2=0x5569aee85560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.486+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/2094108368 shutdown_connections 2023-09-10T00:49:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.486+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/2094108368 wait complete. 2023-09-10T00:49:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.486+0000 7f15a2ce4b80 1 -- start start 2023-09-10T00:49:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.486+0000 7f15a2ce4b80 1 Processor -- start 2023-09-10T00:49:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.486+0000 7f15a2ce4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee8a300 0x5569aee96d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.486+0000 7f15a2ce4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8acc0 0x5569aee93ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.486+0000 7f15a2ce4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee901a0 0x5569aee94570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.486+0000 7f15a2ce4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569aed7ab10 con 0x5569aee8acc0 2023-09-10T00:49:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.486+0000 7f15a2ce4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569aed7a810 con 0x5569aee8a300 2023-09-10T00:49:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.486+0000 7f15a2ce4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569aec5e8c0 con 0x5569aee901a0 2023-09-10T00:49:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.486+0000 7f15a2ce4b80 10 client.0 _ll_register_callbacks cb 0x5569aecc4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:49:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.487+0000 7f158ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8acc0 0x5569aee93ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.487+0000 7f15948ba700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee8a300 0x5569aee96d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.487+0000 7f158ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8acc0 0x5569aee93ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52100/0 (socket says 172.21.15.44:52100) 2023-09-10T00:49:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.487+0000 7f158ffff700 1 -- 172.21.15.44:0/1722918983 learned_addr learned my addr 172.21.15.44:0/1722918983 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.487+0000 7f15950bb700 1 --2- 172.21.15.44:0/1722918983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee901a0 0x5569aee94570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.487+0000 7f158ffff700 1 -- 172.21.15.44:0/1722918983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee901a0 msgr2=0x5569aee94570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.487+0000 7f158ffff700 1 --2- 172.21.15.44:0/1722918983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee901a0 0x5569aee94570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.487+0000 7f158ffff700 1 -- 172.21.15.44:0/1722918983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee8a300 msgr2=0x5569aee96d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.487+0000 7f158ffff700 1 --2- 172.21.15.44:0/1722918983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee8a300 0x5569aee96d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.487+0000 7f158ffff700 1 -- 172.21.15.44:0/1722918983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15901a58b0 con 0x5569aee8acc0 2023-09-10T00:49:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.487+0000 7f15948ba700 1 --2- 172.21.15.44:0/1722918983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee8a300 0x5569aee96d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.487+0000 7f158ffff700 1 --2- 172.21.15.44:0/1722918983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8acc0 0x5569aee93ff0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f157c009b20 tx=0x7f157c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48aafb207fef8a08 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.487+0000 7f156ffff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:49:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.487+0000 7f156ffff700 1 -- 172.21.15.44:0/1722918983 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c004030 con 0x5569aee8acc0 2023-09-10T00:49:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.487+0000 7f156ffff700 1 -- 172.21.15.44:0/1722918983 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f157c004190 con 0x5569aee8acc0 2023-09-10T00:49:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.487+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/1722918983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569aed34490 con 0x5569aee8acc0 2023-09-10T00:49:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.488+0000 7f156ffff700 1 -- 172.21.15.44:0/1722918983 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f157c004350 con 0x5569aee8acc0 2023-09-10T00:49:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.488+0000 7f15a2ce4b80 10 client.17196 fetch_fsmap learned FSMap version 55 2023-09-10T00:49:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.488+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/1722918983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f159010dd20 con 0x5569aee8acc0 2023-09-10T00:49:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.488+0000 7f156ffff700 1 -- 172.21.15.44:0/1722918983 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5569aed34490 con 0x5569aee8acc0 2023-09-10T00:49:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.488+0000 7f156ffff700 1 -- 172.21.15.44:0/1722918983 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f157c01e650 con 0x5569aee8acc0 2023-09-10T00:49:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.488+0000 7f15a2ce4b80 10 client.17196 fetch_fsmap finished waiting for FSMap version 55 2023-09-10T00:49:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.488+0000 7f15a2ce4b80 10 client.17196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:49:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.488+0000 7f15a2ce4b80 20 client.17196 populate_metadata read hostname 'smithi044' 2023-09-10T00:49:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.488+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/1722918983 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5569aeea4310 0x5569aeea6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.488+0000 7f15a2ce4b80 4 client.17196 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:49:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.488+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/1722918983 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569aedf1840 con 0x5569aeea4310 2023-09-10T00:49:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.488+0000 7f15948ba700 1 --2- 172.21.15.44:0/1722918983 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5569aeea4310 0x5569aeea6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.489+0000 7f156ffff700 10 client.17196 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.489+0000 7f15948ba700 1 --2- 172.21.15.44:0/1722918983 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5569aeea4310 0x5569aeea6730 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=813d2632a63b3235 server_cookie=f1acd507ccfc7659 in_seq=0 out_seq=0 2023-09-10T00:49:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.489+0000 7f156ffff700 1 -- 172.21.15.44:0/1722918983 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5569aedf1840 con 0x5569aeea4310 2023-09-10T00:49:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.489+0000 7f156ffff700 10 client.17196 handle_command_reply: tid=0 2023-09-10T00:49:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.489+0000 7f15a2ce4b80 1 client.17196 shutdown 2023-09-10T00:49:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.489+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/1722918983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8acc0 msgr2=0x5569aee93ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.489+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/1722918983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8acc0 0x5569aee93ff0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f157c009b20 tx=0x7f157c0047c0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.490+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/1722918983 shutdown_connections 2023-09-10T00:49:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.490+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/1722918983 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5569aeea4310 0x5569aeea6730 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.490+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/1722918983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee901a0 0x5569aee94570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.490+0000 7f156ffff700 0 client.17196 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.490+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/1722918983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee8a300 0x5569aee96d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.490+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/1722918983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8acc0 0x5569aee93ff0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.490+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/1722918983 >> 172.21.15.44:0/1722918983 conn(0x5569aedff030 msgr2=0x5569aee847a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.490+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/1722918983 shutdown_connections 2023-09-10T00:49:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.490+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/1722918983 wait complete. 2023-09-10T00:49:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.490+0000 7f15a2ce4b80 20 client.17196 trim_cache size 0 max 16384 2023-09-10T00:49:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.507+0000 7f15a2ce4b80 1 Processor -- start 2023-09-10T00:49:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.507+0000 7f15a2ce4b80 1 -- start start 2023-09-10T00:49:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.507+0000 7f15a2ce4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 0x5569aee9b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.507+0000 7f15a2ce4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee8acc0 0x5569aeea0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.507+0000 7f15a2ce4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee901a0 0x5569aeea0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.507+0000 7f15a2ce4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569aed34490 con 0x5569aee8a300 2023-09-10T00:49:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.508+0000 7f15a2ce4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569aee7cf50 con 0x5569aee901a0 2023-09-10T00:49:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.508+0000 7f15a2ce4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569aee7f8a0 con 0x5569aee8acc0 2023-09-10T00:49:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.508+0000 7f15948ba700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 0x5569aee9b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.508+0000 7f158ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee8acc0 0x5569aeea0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.508+0000 7f15948ba700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 0x5569aee9b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52106/0 (socket says 172.21.15.44:52106) 2023-09-10T00:49:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.508+0000 7f15948ba700 1 -- 172.21.15.44:0/4148023895 learned_addr learned my addr 172.21.15.44:0/4148023895 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.508+0000 7f15950bb700 1 --2- 172.21.15.44:0/4148023895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee901a0 0x5569aeea0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.508+0000 7f158ffff700 1 -- 172.21.15.44:0/4148023895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee901a0 msgr2=0x5569aeea0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.508+0000 7f158ffff700 1 --2- 172.21.15.44:0/4148023895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee901a0 0x5569aeea0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.508+0000 7f158ffff700 1 -- 172.21.15.44:0/4148023895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 msgr2=0x5569aee9b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.508+0000 7f158ffff700 1 --2- 172.21.15.44:0/4148023895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 0x5569aee9b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.508+0000 7f158ffff700 1 -- 172.21.15.44:0/4148023895 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569aebf1c00 con 0x5569aee8acc0 2023-09-10T00:49:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.508+0000 7f158ffff700 1 --2- 172.21.15.44:0/4148023895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee8acc0 0x5569aeea0300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f157c01ea50 tx=0x7f157c011a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1827df1b62b1eae1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.508+0000 7f157b7fe700 1 -- 172.21.15.44:0/4148023895 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c01d0d0 con 0x5569aee8acc0 2023-09-10T00:49:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.508+0000 7f157b7fe700 1 -- 172.21.15.44:0/4148023895 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f157c01d230 con 0x5569aee8acc0 2023-09-10T00:49:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.508+0000 7f157b7fe700 1 -- 172.21.15.44:0/4148023895 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c003680 con 0x5569aee8acc0 2023-09-10T00:49:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.509+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/4148023895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee8acc0 msgr2=0x5569aeea0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.509+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/4148023895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee8acc0 0x5569aeea0300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f157c01ea50 tx=0x7f157c011a40 comp rx=0 tx=0).stop 2023-09-10T00:49:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.509+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/4148023895 shutdown_connections 2023-09-10T00:49:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.509+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/4148023895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee901a0 0x5569aeea0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.509+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/4148023895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 0x5569aee9b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.509+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/4148023895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee8acc0 0x5569aeea0300 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.509+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/4148023895 >> 172.21.15.44:0/4148023895 conn(0x5569aedff030 msgr2=0x5569aee9fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.509+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/4148023895 shutdown_connections 2023-09-10T00:49:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.509+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/4148023895 wait complete. 2023-09-10T00:49:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.510+0000 7f15a2ce4b80 1 -- start start 2023-09-10T00:49:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.510+0000 7f15a2ce4b80 1 Processor -- start 2023-09-10T00:49:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.510+0000 7f15a2ce4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 0x5569aee99c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.510+0000 7f15a2ce4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee8acc0 0x5569aee9a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.510+0000 7f15a2ce4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee901a0 0x5569aee9a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.510+0000 7f15a2ce4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569aeef81f0 con 0x5569aee8a300 2023-09-10T00:49:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.510+0000 7f15a2ce4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569aeef8330 con 0x5569aee8acc0 2023-09-10T00:49:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.510+0000 7f15a2ce4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569aeef84f0 con 0x5569aee901a0 2023-09-10T00:49:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.510+0000 7f15a2ce4b80 10 client.0 _ll_register_callbacks cb 0x7f157c01d3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:49:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.510+0000 7f15948ba700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 0x5569aee99c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.510+0000 7f15948ba700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 0x5569aee99c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52112/0 (socket says 172.21.15.44:52112) 2023-09-10T00:49:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f15948ba700 1 -- 172.21.15.44:0/2883025005 learned_addr learned my addr 172.21.15.44:0/2883025005 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f158ffff700 1 --2- 172.21.15.44:0/2883025005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee8acc0 0x5569aee9a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f15950bb700 1 --2- 172.21.15.44:0/2883025005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee901a0 0x5569aee9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f15948ba700 1 -- 172.21.15.44:0/2883025005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee901a0 msgr2=0x5569aee9a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f15948ba700 1 --2- 172.21.15.44:0/2883025005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee901a0 0x5569aee9a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f15948ba700 1 -- 172.21.15.44:0/2883025005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee8acc0 msgr2=0x5569aee9a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f15948ba700 1 --2- 172.21.15.44:0/2883025005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee8acc0 0x5569aee9a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f15948ba700 1 -- 172.21.15.44:0/2883025005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159010dd20 con 0x5569aee8a300 2023-09-10T00:49:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f158ffff700 1 --2- 172.21.15.44:0/2883025005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee8acc0 0x5569aee9a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f15948ba700 1 --2- 172.21.15.44:0/2883025005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 0x5569aee99c00 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1584005950 tx=0x7f1584084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11899d0a7d6e77 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f156ffff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:49:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f156ffff700 1 -- 172.21.15.44:0/2883025005 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15840994b0 con 0x5569aee8a300 2023-09-10T00:49:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/2883025005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569aedd58c0 con 0x5569aee8a300 2023-09-10T00:49:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f156ffff700 1 -- 172.21.15.44:0/2883025005 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f158409d2e0 con 0x5569aee8a300 2023-09-10T00:49:42.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f156ffff700 1 -- 172.21.15.44:0/2883025005 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15840a1520 con 0x5569aee8a300 2023-09-10T00:49:42.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f15a2ce4b80 10 client.17202 fetch_fsmap learned FSMap version 55 2023-09-10T00:49:42.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.511+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/2883025005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x5569aec2f1f0 con 0x5569aee8a300 2023-09-10T00:49:42.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.512+0000 7f156ffff700 1 -- 172.21.15.44:0/2883025005 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5569aedd58c0 con 0x5569aee8a300 2023-09-10T00:49:42.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.512+0000 7f156ffff700 1 -- 172.21.15.44:0/2883025005 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1584004b90 con 0x5569aee8a300 2023-09-10T00:49:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.512+0000 7f15a2ce4b80 10 client.17202 fetch_fsmap finished waiting for FSMap version 55 2023-09-10T00:49:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.512+0000 7f15a2ce4b80 10 client.17202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:49:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.512+0000 7f15a2ce4b80 20 client.17202 populate_metadata read hostname 'smithi044' 2023-09-10T00:49:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.512+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/2883025005 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5569aeef9c80 0x5569aeefc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.512+0000 7f15a2ce4b80 4 client.17202 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:49:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.512+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/2883025005 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5569aed96050 con 0x5569aeef9c80 2023-09-10T00:49:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.512+0000 7f158ffff700 1 --2- 172.21.15.44:0/2883025005 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5569aeef9c80 0x5569aeefc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.513+0000 7f156ffff700 10 client.17202 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.513+0000 7f158ffff700 1 --2- 172.21.15.44:0/2883025005 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5569aeef9c80 0x5569aeefc0a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ad95ce59d5efa17 server_cookie=a468072a41bf45f8 in_seq=0 out_seq=0 2023-09-10T00:49:42.529 INFO:teuthology.orchestra.run.smithi044.stderr:33800000 of 50000000 tuples (67%) done (elapsed 522.16 s, remaining 250.27 s) 2023-09-10T00:49:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.529+0000 7f156ffff700 1 -- 172.21.15.44:0/2883025005 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5569aed96050 con 0x5569aeef9c80 2023-09-10T00:49:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.529+0000 7f156ffff700 10 client.17202 handle_command_reply: tid=0 2023-09-10T00:49:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.529+0000 7f15a2ce4b80 1 client.17202 shutdown 2023-09-10T00:49:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.529+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/2883025005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 msgr2=0x5569aee99c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.529+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/2883025005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 0x5569aee99c00 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1584005950 tx=0x7f1584084040 comp rx=0 tx=0).stop 2023-09-10T00:49:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.529+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/2883025005 shutdown_connections 2023-09-10T00:49:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.529+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/2883025005 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5569aeef9c80 0x5569aeefc0a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.529+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/2883025005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5569aee901a0 0x5569aee9a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.529+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/2883025005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5569aee8a300 0x5569aee99c00 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.529+0000 7f15a2ce4b80 1 --2- 172.21.15.44:0/2883025005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569aee8acc0 0x5569aee9a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.529+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/2883025005 >> 172.21.15.44:0/2883025005 conn(0x5569aedff030 msgr2=0x5569aeea0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.530+0000 7f156ffff700 0 client.17202 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.530+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/2883025005 shutdown_connections 2023-09-10T00:49:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.530+0000 7f15a2ce4b80 1 -- 172.21.15.44:0/2883025005 wait complete. 2023-09-10T00:49:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.530+0000 7f15a2ce4b80 20 client.17202 trim_cache size 0 max 16384 2023-09-10T00:49:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:49:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:49:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:49:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:49:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.530+0000 7f1596b1e700 1 -- 172.21.15.44:0/3388502257 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f15800a0830 msgr2=0x7f15800a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.530+0000 7f1596b1e700 1 --2- 172.21.15.44:0/3388502257 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f15800a0830 0x7f15800a2cf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1590074300 tx=0x7f157c009260 comp rx=0 tx=0).stop 2023-09-10T00:49:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.530+0000 7f1596b1e700 1 -- 172.21.15.44:0/3388502257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15901005a0 msgr2=0x7f1590072ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.530+0000 7f1596b1e700 1 --2- 172.21.15.44:0/3388502257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15901005a0 0x7f1590072ae0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f158400e9c0 tx=0x7f158400ecd0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.531+0000 7f1596b1e700 1 -- 172.21.15.44:0/3388502257 shutdown_connections 2023-09-10T00:49:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.531+0000 7f1596b1e700 1 --2- 172.21.15.44:0/3388502257 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f15800a0830 0x7f15800a2cf0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.531+0000 7f1596b1e700 1 --2- 172.21.15.44:0/3388502257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1590106660 0x7f15901a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.531+0000 7f1596b1e700 1 --2- 172.21.15.44:0/3388502257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15901005a0 0x7f1590072ae0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.531+0000 7f1596b1e700 1 --2- 172.21.15.44:0/3388502257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1590100f60 0x7f1590073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.531+0000 7f1596b1e700 1 -- 172.21.15.44:0/3388502257 >> 172.21.15.44:0/3388502257 conn(0x7f15900fc0b0 msgr2=0x7f15900fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.531+0000 7f1596b1e700 1 -- 172.21.15.44:0/3388502257 shutdown_connections 2023-09-10T00:49:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:42.531+0000 7f1596b1e700 1 -- 172.21.15.44:0/3388502257 wait complete. 2023-09-10T00:49:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:42 smithi131 ceph-mon[103713]: pgmap v280: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 27 MiB/s wr, 95 op/s 2023-09-10T00:49:43.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:42 smithi044 ceph-mon[98676]: pgmap v280: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 27 MiB/s wr, 95 op/s 2023-09-10T00:49:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:42 smithi099 ceph-mon[102692]: pgmap v280: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 27 MiB/s wr, 95 op/s 2023-09-10T00:49:43.513 INFO:teuthology.orchestra.run.smithi044.stderr:33900000 of 50000000 tuples (67%) done (elapsed 523.15 s, remaining 248.46 s) 2023-09-10T00:49:44.525 INFO:teuthology.orchestra.run.smithi044.stderr:34000000 of 50000000 tuples (68%) done (elapsed 524.16 s, remaining 246.66 s) 2023-09-10T00:49:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:44 smithi131 ceph-mon[103713]: pgmap v281: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 53 op/s 2023-09-10T00:49:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:44 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:44 smithi044 ceph-mon[98676]: pgmap v281: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 53 op/s 2023-09-10T00:49:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:44 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:44 smithi099 ceph-mon[102692]: pgmap v281: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 53 op/s 2023-09-10T00:49:45.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:44 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:46.940 INFO:teuthology.orchestra.run.smithi044.stderr:34100000 of 50000000 tuples (68%) done (elapsed 526.57 s, remaining 245.53 s) 2023-09-10T00:49:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:46 smithi131 ceph-mon[103713]: pgmap v282: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 20 MiB/s wr, 85 op/s 2023-09-10T00:49:47.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:46 smithi044 ceph-mon[98676]: pgmap v282: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 20 MiB/s wr, 85 op/s 2023-09-10T00:49:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:46 smithi099 ceph-mon[102692]: pgmap v282: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 20 MiB/s wr, 85 op/s 2023-09-10T00:49:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:48 smithi044 ceph-mon[98676]: pgmap v283: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 14 MiB/s wr, 55 op/s 2023-09-10T00:49:49.080 INFO:teuthology.orchestra.run.smithi044.stderr:34200000 of 50000000 tuples (68%) done (elapsed 528.71 s, remaining 244.26 s) 2023-09-10T00:49:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:48 smithi099 ceph-mon[102692]: pgmap v283: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 14 MiB/s wr, 55 op/s 2023-09-10T00:49:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:48 smithi131 ceph-mon[103713]: pgmap v283: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 14 MiB/s wr, 55 op/s 2023-09-10T00:49:50.115 INFO:teuthology.orchestra.run.smithi044.stderr:34300000 of 50000000 tuples (68%) done (elapsed 529.75 s, remaining 242.48 s) 2023-09-10T00:49:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:50 smithi044 ceph-mon[98676]: pgmap v284: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 14 MiB/s wr, 55 op/s 2023-09-10T00:49:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:50 smithi099 ceph-mon[102692]: pgmap v284: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 14 MiB/s wr, 55 op/s 2023-09-10T00:49:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:50 smithi131 ceph-mon[103713]: pgmap v284: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 14 MiB/s wr, 55 op/s 2023-09-10T00:49:51.633 INFO:teuthology.orchestra.run.smithi044.stderr:34400000 of 50000000 tuples (68%) done (elapsed 531.27 s, remaining 240.92 s) 2023-09-10T00:49:52.487 INFO:teuthology.orchestra.run.smithi044.stderr:34500000 of 50000000 tuples (69%) done (elapsed 532.12 s, remaining 239.07 s) 2023-09-10T00:49:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:52 smithi044 ceph-mon[98676]: pgmap v285: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 22 MiB/s wr, 94 op/s 2023-09-10T00:49:53.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:52 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:49:53.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:52 smithi099 ceph-mon[102692]: pgmap v285: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 22 MiB/s wr, 94 op/s 2023-09-10T00:49:53.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:52 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:49:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:52 smithi131 ceph-mon[103713]: pgmap v285: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 22 MiB/s wr, 94 op/s 2023-09-10T00:49:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:52 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:49:53.379 INFO:teuthology.orchestra.run.smithi044.stderr:34600000 of 50000000 tuples (69%) done (elapsed 533.01 s, remaining 237.24 s) 2023-09-10T00:49:54.366 INFO:teuthology.orchestra.run.smithi044.stderr:34700000 of 50000000 tuples (69%) done (elapsed 534.00 s, remaining 235.45 s) 2023-09-10T00:49:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:54 smithi131 ceph-mon[103713]: pgmap v286: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 17 MiB/s wr, 71 op/s 2023-09-10T00:49:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:54 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:55.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:54 smithi044 ceph-mon[98676]: pgmap v286: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 17 MiB/s wr, 71 op/s 2023-09-10T00:49:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:54 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:55.517 INFO:teuthology.orchestra.run.smithi044.stderr:34800000 of 50000000 tuples (69%) done (elapsed 535.15 s, remaining 233.74 s) 2023-09-10T00:49:55.596 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:54 smithi099 ceph-mon[102692]: pgmap v286: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 17 MiB/s wr, 71 op/s 2023-09-10T00:49:55.596 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:54 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.657+0000 7f1cec1b6700 1 -- 172.21.15.44:0/1314663397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ce40683b0 msgr2=0x7f1ce410d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.657+0000 7f1cec1b6700 1 --2- 172.21.15.44:0/1314663397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ce40683b0 0x7f1ce410d3d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00b0d0 tx=0x7f1cdc00b3e0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.657+0000 7f1cec1b6700 1 -- 172.21.15.44:0/1314663397 shutdown_connections 2023-09-10T00:49:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.657+0000 7f1cec1b6700 1 --2- 172.21.15.44:0/1314663397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ce40683b0 0x7f1ce410d3d0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.657+0000 7f1cec1b6700 1 --2- 172.21.15.44:0/1314663397 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ce4106650 0x7f1ce4106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.657+0000 7f1cec1b6700 1 --2- 172.21.15.44:0/1314663397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ce40679f0 0x7f1ce4067e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.657+0000 7f1cec1b6700 1 -- 172.21.15.44:0/1314663397 >> 172.21.15.44:0/1314663397 conn(0x7f1ce4074f30 msgr2=0x7f1ce4075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.657+0000 7f1cec1b6700 1 -- 172.21.15.44:0/1314663397 shutdown_connections 2023-09-10T00:49:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.657+0000 7f1cec1b6700 1 -- 172.21.15.44:0/1314663397 wait complete. 2023-09-10T00:49:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.658+0000 7f1cec1b6700 1 Processor -- start 2023-09-10T00:49:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.658+0000 7f1cec1b6700 1 -- start start 2023-09-10T00:49:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.658+0000 7f1cec1b6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ce40679f0 0x7f1ce40ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.658+0000 7f1cec1b6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ce40683b0 0x7f1ce40fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.658+0000 7f1cec1b6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ce4106650 0x7f1ce41a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.658+0000 7f1cec1b6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce41a4dd0 con 0x7f1ce40679f0 2023-09-10T00:49:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.658+0000 7f1cec1b6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce41a4f10 con 0x7f1ce40683b0 2023-09-10T00:49:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.658+0000 7f1cec1b6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce41a5080 con 0x7f1ce4106650 2023-09-10T00:49:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.658+0000 7f1ce9f52700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ce40679f0 0x7f1ce40ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.658+0000 7f1ce9f52700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ce40679f0 0x7f1ce40ffa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52124/0 (socket says 172.21.15.44:52124) 2023-09-10T00:49:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.658+0000 7f1ce9751700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ce40683b0 0x7f1ce40fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.658+0000 7f1ce9f52700 1 -- 172.21.15.44:0/1057960933 learned_addr learned my addr 172.21.15.44:0/1057960933 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.658+0000 7f1cea753700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ce4106650 0x7f1ce41a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.659+0000 7f1cea753700 1 -- 172.21.15.44:0/1057960933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ce40683b0 msgr2=0x7f1ce40fff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.659+0000 7f1cea753700 1 --2- 172.21.15.44:0/1057960933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ce40683b0 0x7f1ce40fff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.659+0000 7f1cea753700 1 -- 172.21.15.44:0/1057960933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ce40679f0 msgr2=0x7f1ce40ffa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.659+0000 7f1cea753700 1 --2- 172.21.15.44:0/1057960933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ce40679f0 0x7f1ce40ffa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.659+0000 7f1cea753700 1 -- 172.21.15.44:0/1057960933 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce0009520 con 0x7f1ce4106650 2023-09-10T00:49:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.659+0000 7f1ce9f52700 1 --2- 172.21.15.44:0/1057960933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ce40679f0 0x7f1ce40ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.659+0000 7f1cea753700 1 --2- 172.21.15.44:0/1057960933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ce4106650 0x7f1ce41a4890 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009fd0 tx=0x7f1cdc007910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.659+0000 7f1cdaffd700 1 -- 172.21.15.44:0/1057960933 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc00e040 con 0x7f1ce4106650 2023-09-10T00:49:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.659+0000 7f1cec1b6700 1 -- 172.21.15.44:0/1057960933 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cdc009d00 con 0x7f1ce4106650 2023-09-10T00:49:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.659+0000 7f1cdaffd700 1 -- 172.21.15.44:0/1057960933 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1cdc01d480 con 0x7f1ce4106650 2023-09-10T00:49:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.659+0000 7f1cdaffd700 1 -- 172.21.15.44:0/1057960933 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc01b6b0 con 0x7f1ce4106650 2023-09-10T00:49:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.659+0000 7f1cec1b6700 1 -- 172.21.15.44:0/1057960933 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce41a55f0 con 0x7f1ce4106650 2023-09-10T00:49:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.662+0000 7f1cdaffd700 1 -- 172.21.15.44:0/1057960933 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1cdc01d5f0 con 0x7f1ce4106650 2023-09-10T00:49:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.662+0000 7f1cdaffd700 1 --2- 172.21.15.44:0/1057960933 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1cd00a07b0 0x7f1cd00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.663+0000 7f1ce9f52700 1 --2- 172.21.15.44:0/1057960933 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1cd00a07b0 0x7f1cd00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.663+0000 7f1cdaffd700 1 -- 172.21.15.44:0/1057960933 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1cdc0cdad0 con 0x7f1ce4106650 2023-09-10T00:49:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.663+0000 7f1ce9f52700 1 --2- 172.21.15.44:0/1057960933 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1cd00a07b0 0x7f1cd00a2c70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00116b0 tx=0x7f1ce0009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.663+0000 7f1cf837cb80 1 Processor -- start 2023-09-10T00:49:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.663+0000 7f1cf837cb80 1 -- start start 2023-09-10T00:49:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.663+0000 7f1cf837cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 0x5610d7e25580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.663+0000 7f1cf837cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1f300 0x5610d7e1f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.663+0000 7f1cf837cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1fcc0 0x5610d7e2bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.663+0000 7f1cf837cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d7d786d0 con 0x5610d7e1f300 2023-09-10T00:49:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.663+0000 7f1cf837cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d7d75ff0 con 0x5610d7e1fcc0 2023-09-10T00:49:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1cf837cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d7d718e0 con 0x5610d7e251a0 2023-09-10T00:49:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1ce9751700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1f300 0x5610d7e1f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1ce9f52700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 0x5610d7e25580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1cea753700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1fcc0 0x5610d7e2bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1ce9f52700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 0x5610d7e25580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59014/0 (socket says 172.21.15.44:59014) 2023-09-10T00:49:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1ce9751700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1f300 0x5610d7e1f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52134/0 (socket says 172.21.15.44:52134) 2023-09-10T00:49:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1ce9f52700 1 -- 172.21.15.44:0/2526786 learned_addr learned my addr 172.21.15.44:0/2526786 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1cea753700 1 -- 172.21.15.44:0/2526786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 msgr2=0x5610d7e25580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1cea753700 1 --2- 172.21.15.44:0/2526786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 0x5610d7e25580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1cea753700 1 -- 172.21.15.44:0/2526786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1f300 msgr2=0x5610d7e1f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1cea753700 1 --2- 172.21.15.44:0/2526786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1f300 0x5610d7e1f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1cea753700 1 -- 172.21.15.44:0/2526786 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610d7b9dd80 con 0x5610d7e1fcc0 2023-09-10T00:49:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1ce9f52700 1 --2- 172.21.15.44:0/2526786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 0x5610d7e25580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1ce9751700 1 --2- 172.21.15.44:0/2526786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1f300 0x5610d7e1f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1cea753700 1 --2- 172.21.15.44:0/2526786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1fcc0 0x5610d7e2bf30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0cadc0 tx=0x7f1cdc019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec9dc21d9ff2f9df server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1ccffff700 1 -- 172.21.15.44:0/2526786 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc09abb0 con 0x5610d7e1fcc0 2023-09-10T00:49:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1ccffff700 1 -- 172.21.15.44:0/2526786 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1cdc09e9e0 con 0x5610d7e1fcc0 2023-09-10T00:49:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.664+0000 7f1ccffff700 1 -- 172.21.15.44:0/2526786 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc0ba220 con 0x5610d7e1fcc0 2023-09-10T00:49:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.665+0000 7f1cf837cb80 1 -- 172.21.15.44:0/2526786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1fcc0 msgr2=0x5610d7e2bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.665+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/2526786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1fcc0 0x5610d7e2bf30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0cadc0 tx=0x7f1cdc019000 comp rx=0 tx=0).stop 2023-09-10T00:49:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.665+0000 7f1cf837cb80 1 -- 172.21.15.44:0/2526786 shutdown_connections 2023-09-10T00:49:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.665+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/2526786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1fcc0 0x5610d7e2bf30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.665+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/2526786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 0x5610d7e25580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.665+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/2526786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1f300 0x5610d7e1f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.665+0000 7f1cf837cb80 1 -- 172.21.15.44:0/2526786 >> 172.21.15.44:0/2526786 conn(0x5610d7d94030 msgr2=0x5610d7e1a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.665+0000 7f1cf837cb80 1 -- 172.21.15.44:0/2526786 shutdown_connections 2023-09-10T00:49:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.665+0000 7f1cf837cb80 1 -- 172.21.15.44:0/2526786 wait complete. 2023-09-10T00:49:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.665+0000 7f1cf837cb80 1 -- start start 2023-09-10T00:49:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1cf837cb80 1 Processor -- start 2023-09-10T00:49:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1cf837cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1f300 0x5610d7e2ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1cf837cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1fcc0 0x5610d7e2f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1cf837cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 0x5610d7e28f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1cf837cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d7d0fb10 con 0x5610d7e1fcc0 2023-09-10T00:49:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1cf837cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d7d0f810 con 0x5610d7e1f300 2023-09-10T00:49:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1cf837cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d7bf38c0 con 0x5610d7e251a0 2023-09-10T00:49:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1cf837cb80 10 client.0 _ll_register_callbacks cb 0x5610d7c59a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:49:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1ce9751700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1fcc0 0x5610d7e2f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1ce9f52700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1f300 0x5610d7e2ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1ce9751700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1fcc0 0x5610d7e2f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52140/0 (socket says 172.21.15.44:52140) 2023-09-10T00:49:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1ce9f52700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1f300 0x5610d7e2ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53878/0 (socket says 172.21.15.44:53878) 2023-09-10T00:49:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1ce9751700 1 -- 172.21.15.44:0/3040871138 learned_addr learned my addr 172.21.15.44:0/3040871138 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1cea753700 1 --2- 172.21.15.44:0/3040871138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 0x5610d7e28f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1ce9751700 1 -- 172.21.15.44:0/3040871138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 msgr2=0x5610d7e28f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1ce9751700 1 --2- 172.21.15.44:0/3040871138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 0x5610d7e28f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1ce9751700 1 -- 172.21.15.44:0/3040871138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1f300 msgr2=0x5610d7e2ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1ce9751700 1 --2- 172.21.15.44:0/3040871138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1f300 0x5610d7e2ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1ce9751700 1 -- 172.21.15.44:0/3040871138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd40098e0 con 0x5610d7e1fcc0 2023-09-10T00:49:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.666+0000 7f1ce9f52700 1 --2- 172.21.15.44:0/3040871138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1f300 0x5610d7e2ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.667+0000 7f1ce9751700 1 --2- 172.21.15.44:0/3040871138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1fcc0 0x5610d7e2f220 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4012980 tx=0x7f1cd4012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b04da604b28ebf3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.667+0000 7f1ccd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:49:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.667+0000 7f1ccd7fa700 1 -- 172.21.15.44:0/3040871138 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4017070 con 0x5610d7e1fcc0 2023-09-10T00:49:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.667+0000 7f1cf837cb80 1 -- 172.21.15.44:0/3040871138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610d7cc9490 con 0x5610d7e1fcc0 2023-09-10T00:49:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.667+0000 7f1ccd7fa700 1 -- 172.21.15.44:0/3040871138 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1cd4004030 con 0x5610d7e1fcc0 2023-09-10T00:49:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.667+0000 7f1ccd7fa700 1 -- 172.21.15.44:0/3040871138 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cd40041f0 con 0x5610d7e1fcc0 2023-09-10T00:49:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.667+0000 7f1cf837cb80 10 client.17214 fetch_fsmap learned FSMap version 57 2023-09-10T00:49:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.667+0000 7f1cf837cb80 1 -- 172.21.15.44:0/3040871138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x7f1ce0017050 con 0x5610d7e1fcc0 2023-09-10T00:49:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.667+0000 7f1ccd7fa700 1 -- 172.21.15.44:0/3040871138 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5610d7cc9490 con 0x5610d7e1fcc0 2023-09-10T00:49:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.667+0000 7f1ccd7fa700 1 -- 172.21.15.44:0/3040871138 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1cd40036f0 con 0x5610d7e1fcc0 2023-09-10T00:49:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.667+0000 7f1cf837cb80 10 client.17214 fetch_fsmap finished waiting for FSMap version 57 2023-09-10T00:49:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.667+0000 7f1cf837cb80 10 client.17214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:49:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.667+0000 7f1cf837cb80 20 client.17214 populate_metadata read hostname 'smithi044' 2023-09-10T00:49:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.667+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/3040871138 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5610d7e39280 0x5610d7e3b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.667+0000 7f1cf837cb80 4 client.17214 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:49:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.667+0000 7f1cf837cb80 1 -- 172.21.15.44:0/3040871138 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610d7d86840 con 0x5610d7e39280 2023-09-10T00:49:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.668+0000 7f1ce9f52700 1 --2- 172.21.15.44:0/3040871138 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5610d7e39280 0x5610d7e3b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.668+0000 7f1ccd7fa700 10 client.17214 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.668+0000 7f1ce9f52700 1 --2- 172.21.15.44:0/3040871138 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5610d7e39280 0x5610d7e3b6a0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f2bb8cf22365e94 server_cookie=5f1cee1b4c364e9c in_seq=0 out_seq=0 2023-09-10T00:49:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.669+0000 7f1ccd7fa700 1 -- 172.21.15.44:0/3040871138 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5610d7d86840 con 0x5610d7e39280 2023-09-10T00:49:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.669+0000 7f1ccd7fa700 10 client.17214 handle_command_reply: tid=0 2023-09-10T00:49:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.669+0000 7f1cf837cb80 1 client.17214 shutdown 2023-09-10T00:49:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.669+0000 7f1cf837cb80 1 -- 172.21.15.44:0/3040871138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1fcc0 msgr2=0x5610d7e2f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.669+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/3040871138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1fcc0 0x5610d7e2f220 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4012980 tx=0x7f1cd4012c90 comp rx=0 tx=0).stop 2023-09-10T00:49:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.669+0000 7f1cf837cb80 1 -- 172.21.15.44:0/3040871138 shutdown_connections 2023-09-10T00:49:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.669+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/3040871138 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5610d7e39280 0x5610d7e3b6a0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.669+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/3040871138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 0x5610d7e28f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.669+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/3040871138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1f300 0x5610d7e2ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.669+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/3040871138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1fcc0 0x5610d7e2f220 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.669+0000 7f1ccd7fa700 0 client.17214 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.669+0000 7f1cf837cb80 1 -- 172.21.15.44:0/3040871138 >> 172.21.15.44:0/3040871138 conn(0x5610d7d94030 msgr2=0x5610d7e197a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.669+0000 7f1cf837cb80 1 -- 172.21.15.44:0/3040871138 shutdown_connections 2023-09-10T00:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.669+0000 7f1cf837cb80 1 -- 172.21.15.44:0/3040871138 wait complete. 2023-09-10T00:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.669+0000 7f1cf837cb80 20 client.17214 trim_cache size 0 max 16384 2023-09-10T00:49:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1cf837cb80 1 Processor -- start 2023-09-10T00:49:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1cf837cb80 1 -- start start 2023-09-10T00:49:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1cf837cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1f300 0x5610d7e290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1cf837cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e1fcc0 0x5610d7e1ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1cf837cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e251a0 0x5610d7e19f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1cf837cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d7e16ff0 con 0x5610d7e1f300 2023-09-10T00:49:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1cf837cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d7e2a080 con 0x5610d7e251a0 2023-09-10T00:49:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1cf837cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d7e168b0 con 0x5610d7e1fcc0 2023-09-10T00:49:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1ce9f52700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1f300 0x5610d7e290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1ce9f52700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1f300 0x5610d7e290a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52146/0 (socket says 172.21.15.44:52146) 2023-09-10T00:49:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1ce9f52700 1 -- 172.21.15.44:0/3842467866 learned_addr learned my addr 172.21.15.44:0/3842467866 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1ce9751700 1 --2- 172.21.15.44:0/3842467866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e1fcc0 0x5610d7e1ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1cea753700 1 --2- 172.21.15.44:0/3842467866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e251a0 0x5610d7e19f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1ce9f52700 1 -- 172.21.15.44:0/3842467866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e1fcc0 msgr2=0x5610d7e1ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1ce9f52700 1 --2- 172.21.15.44:0/3842467866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e1fcc0 0x5610d7e1ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1ce9f52700 1 -- 172.21.15.44:0/3842467866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e251a0 msgr2=0x5610d7e19f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1ce9f52700 1 --2- 172.21.15.44:0/3842467866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e251a0 0x5610d7e19f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1ce9f52700 1 -- 172.21.15.44:0/3842467866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610d7e40290 con 0x5610d7e1f300 2023-09-10T00:49:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.684+0000 7f1ce9f52700 1 --2- 172.21.15.44:0/3842467866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1f300 0x5610d7e290a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0011e00 tx=0x7f1ce0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f459caee94ea261 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.685+0000 7f1cd8ff9700 1 -- 172.21.15.44:0/3842467866 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0022070 con 0x5610d7e1f300 2023-09-10T00:49:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.685+0000 7f1cd8ff9700 1 -- 172.21.15.44:0/3842467866 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ce0023040 con 0x5610d7e1f300 2023-09-10T00:49:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.685+0000 7f1cd8ff9700 1 -- 172.21.15.44:0/3842467866 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce001de00 con 0x5610d7e1f300 2023-09-10T00:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.685+0000 7f1cf837cb80 1 -- 172.21.15.44:0/3842467866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1f300 msgr2=0x5610d7e290a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.685+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/3842467866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1f300 0x5610d7e290a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0011e00 tx=0x7f1ce0014000 comp rx=0 tx=0).stop 2023-09-10T00:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.685+0000 7f1cf837cb80 1 -- 172.21.15.44:0/3842467866 shutdown_connections 2023-09-10T00:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.685+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/3842467866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e251a0 0x5610d7e19f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.685+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/3842467866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1f300 0x5610d7e290a0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.685+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/3842467866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e1fcc0 0x5610d7e1ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.685+0000 7f1cf837cb80 1 -- 172.21.15.44:0/3842467866 >> 172.21.15.44:0/3842467866 conn(0x5610d7d94030 msgr2=0x5610d7e1e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.685+0000 7f1cf837cb80 1 -- 172.21.15.44:0/3842467866 shutdown_connections 2023-09-10T00:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.685+0000 7f1cf837cb80 1 -- 172.21.15.44:0/3842467866 wait complete. 2023-09-10T00:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1cf837cb80 1 -- start start 2023-09-10T00:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1cf837cb80 1 Processor -- start 2023-09-10T00:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1cf837cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1f300 0x5610d7e86500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1cf837cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1fcc0 0x5610d7e86a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1cf837cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 0x5610d7e85270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1cf837cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d7e85820 con 0x5610d7e1fcc0 2023-09-10T00:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1cf837cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d7e85990 con 0x5610d7e1f300 2023-09-10T00:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1cf837cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d7e85b50 con 0x5610d7e251a0 2023-09-10T00:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1cf837cb80 10 client.0 _ll_register_callbacks cb 0x7f1cd4012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:49:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1ce9751700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1fcc0 0x5610d7e86a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1ce9f52700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1f300 0x5610d7e86500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1cea753700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 0x5610d7e85270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1ce9f52700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1f300 0x5610d7e86500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53892/0 (socket says 172.21.15.44:53892) 2023-09-10T00:49:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1ce9f52700 1 -- 172.21.15.44:0/814669032 learned_addr learned my addr 172.21.15.44:0/814669032 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1ce9751700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1fcc0 0x5610d7e86a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52154/0 (socket says 172.21.15.44:52154) 2023-09-10T00:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1ce9f52700 1 -- 172.21.15.44:0/814669032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 msgr2=0x5610d7e85270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1ce9f52700 1 --2- 172.21.15.44:0/814669032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 0x5610d7e85270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1ce9f52700 1 -- 172.21.15.44:0/814669032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1fcc0 msgr2=0x5610d7e86a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1ce9f52700 1 --2- 172.21.15.44:0/814669032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1fcc0 0x5610d7e86a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1ce9f52700 1 -- 172.21.15.44:0/814669032 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce000fd20 con 0x5610d7e1f300 2023-09-10T00:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.686+0000 7f1ce9751700 1 --2- 172.21.15.44:0/814669032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1fcc0 0x5610d7e86a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.687+0000 7f1ce9f52700 1 --2- 172.21.15.44:0/814669032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1f300 0x5610d7e86500 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000f450 tx=0x7f1ce000fa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=398eb48729a24402 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.687+0000 7f1ccd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.687+0000 7f1ccd7fa700 1 -- 172.21.15.44:0/814669032 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0022040 con 0x5610d7e1f300 2023-09-10T00:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.687+0000 7f1cf837cb80 1 -- 172.21.15.44:0/814669032 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1cdc09e9e0 con 0x5610d7e1f300 2023-09-10T00:49:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.687+0000 7f1ccd7fa700 1 -- 172.21.15.44:0/814669032 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1ce001d9e0 con 0x5610d7e1f300 2023-09-10T00:49:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.687+0000 7f1ccd7fa700 1 -- 172.21.15.44:0/814669032 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ce001dba0 con 0x5610d7e1f300 2023-09-10T00:49:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.687+0000 7f1cf837cb80 10 client.26885 fetch_fsmap learned FSMap version 57 2023-09-10T00:49:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.687+0000 7f1cf837cb80 1 -- 172.21.15.44:0/814669032 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x7f1ce41a55f0 con 0x5610d7e1f300 2023-09-10T00:49:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.687+0000 7f1ccd7fa700 1 -- 172.21.15.44:0/814669032 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1cdc09e9e0 con 0x5610d7e1f300 2023-09-10T00:49:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.687+0000 7f1ccd7fa700 1 -- 172.21.15.44:0/814669032 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1ce00356e0 con 0x5610d7e1f300 2023-09-10T00:49:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.687+0000 7f1cf837cb80 10 client.26885 fetch_fsmap finished waiting for FSMap version 57 2023-09-10T00:49:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.687+0000 7f1cf837cb80 10 client.26885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:49:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.687+0000 7f1cf837cb80 20 client.26885 populate_metadata read hostname 'smithi044' 2023-09-10T00:49:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.687+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/814669032 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5610d7e8e560 0x5610d7e90980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:49:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.687+0000 7f1cf837cb80 4 client.26885 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:49:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.687+0000 7f1cf837cb80 1 -- 172.21.15.44:0/814669032 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5610d7d6a8c0 con 0x5610d7e8e560 2023-09-10T00:49:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.688+0000 7f1ce9751700 1 --2- 172.21.15.44:0/814669032 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5610d7e8e560 0x5610d7e90980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:49:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.688+0000 7f1ce9751700 1 --2- 172.21.15.44:0/814669032 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5610d7e8e560 0x5610d7e90980 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44974b593f4b9fae server_cookie=1e765763dcedcfce in_seq=0 out_seq=0 2023-09-10T00:49:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.688+0000 7f1ccd7fa700 10 client.26885 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.706+0000 7f1ccd7fa700 1 -- 172.21.15.44:0/814669032 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5610d7d6a8c0 con 0x5610d7e8e560 2023-09-10T00:49:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.706+0000 7f1ccd7fa700 10 client.26885 handle_command_reply: tid=0 2023-09-10T00:49:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.706+0000 7f1cf837cb80 1 client.26885 shutdown 2023-09-10T00:49:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.706+0000 7f1cf837cb80 1 -- 172.21.15.44:0/814669032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1f300 msgr2=0x5610d7e86500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.706+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/814669032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1f300 0x5610d7e86500 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000f450 tx=0x7f1ce000fa30 comp rx=0 tx=0).stop 2023-09-10T00:49:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.706+0000 7f1cf837cb80 1 -- 172.21.15.44:0/814669032 shutdown_connections 2023-09-10T00:49:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.706+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/814669032 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5610d7e8e560 0x5610d7e90980 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.706+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/814669032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610d7e251a0 0x5610d7e85270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.706+0000 7f1ccd7fa700 0 client.26885 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:49:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.706+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/814669032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610d7e1f300 0x5610d7e86500 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.706+0000 7f1cf837cb80 1 --2- 172.21.15.44:0/814669032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610d7e1fcc0 0x5610d7e86a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.706+0000 7f1cf837cb80 1 -- 172.21.15.44:0/814669032 >> 172.21.15.44:0/814669032 conn(0x5610d7d94030 msgr2=0x5610d7e25c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.706+0000 7f1cf837cb80 1 -- 172.21.15.44:0/814669032 shutdown_connections 2023-09-10T00:49:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.706+0000 7f1cf837cb80 1 -- 172.21.15.44:0/814669032 wait complete. 2023-09-10T00:49:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.707+0000 7f1cf837cb80 20 client.26885 trim_cache size 0 max 16384 2023-09-10T00:49:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:49:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:49:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:49:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:49:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.707+0000 7f1cec1b6700 1 -- 172.21.15.44:0/1057960933 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1cd00a07b0 msgr2=0x7f1cd00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.707+0000 7f1cec1b6700 1 --2- 172.21.15.44:0/1057960933 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1cd00a07b0 0x7f1cd00a2c70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00116b0 tx=0x7f1ce0009260 comp rx=0 tx=0).stop 2023-09-10T00:49:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.707+0000 7f1cec1b6700 1 -- 172.21.15.44:0/1057960933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ce4106650 msgr2=0x7f1ce41a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:49:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.707+0000 7f1cec1b6700 1 --2- 172.21.15.44:0/1057960933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ce4106650 0x7f1ce41a4890 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009fd0 tx=0x7f1cdc007910 comp rx=0 tx=0).stop 2023-09-10T00:49:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.707+0000 7f1cec1b6700 1 -- 172.21.15.44:0/1057960933 shutdown_connections 2023-09-10T00:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.707+0000 7f1cec1b6700 1 --2- 172.21.15.44:0/1057960933 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1cd00a07b0 0x7f1cd00a2c70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.707+0000 7f1cec1b6700 1 --2- 172.21.15.44:0/1057960933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ce4106650 0x7f1ce41a4890 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.707+0000 7f1cec1b6700 1 --2- 172.21.15.44:0/1057960933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ce40679f0 0x7f1ce40ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.707+0000 7f1cec1b6700 1 --2- 172.21.15.44:0/1057960933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ce40683b0 0x7f1ce40fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.707+0000 7f1cec1b6700 1 -- 172.21.15.44:0/1057960933 >> 172.21.15.44:0/1057960933 conn(0x7f1ce4074f30 msgr2=0x7f1ce4104250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:49:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.707+0000 7f1cec1b6700 1 -- 172.21.15.44:0/1057960933 shutdown_connections 2023-09-10T00:49:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:49:55.707+0000 7f1cec1b6700 1 -- 172.21.15.44:0/1057960933 wait complete. 2023-09-10T00:49:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:56 smithi099 ceph-mon[102692]: pgmap v287: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 31 MiB/s wr, 101 op/s 2023-09-10T00:49:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:56 smithi131 ceph-mon[103713]: pgmap v287: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 31 MiB/s wr, 101 op/s 2023-09-10T00:49:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:56 smithi044 ceph-mon[98676]: pgmap v287: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 31 MiB/s wr, 101 op/s 2023-09-10T00:49:58.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:57 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:49:58.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:57 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:49:58.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:57 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:49:58.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:57 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:49:58.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:57 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:49:58.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:57 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:49:58.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:57 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:49:58.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:57 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:49:58.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:57 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:49:59.117 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:49:58 smithi099 ceph-mon[102692]: pgmap v288: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 22 MiB/s wr, 69 op/s 2023-09-10T00:49:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:49:58 smithi131 ceph-mon[103713]: pgmap v288: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 22 MiB/s wr, 69 op/s 2023-09-10T00:49:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:49:58 smithi044 ceph-mon[98676]: pgmap v288: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 22 MiB/s wr, 69 op/s 2023-09-10T00:49:59.888 INFO:teuthology.orchestra.run.smithi044.stderr:34900000 of 50000000 tuples (69%) done (elapsed 539.52 s, remaining 233.43 s) 2023-09-10T00:50:00.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:00 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:50:00.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:00 smithi131 ceph-mon[103713]: pgmap v289: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 22 MiB/s wr, 69 op/s 2023-09-10T00:50:00.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:00 smithi131 ceph-mon[103713]: overall HEALTH_OK 2023-09-10T00:50:00.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:00 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:50:00.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:00 smithi044 ceph-mon[98676]: pgmap v289: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 22 MiB/s wr, 69 op/s 2023-09-10T00:50:00.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:00 smithi044 ceph-mon[98676]: overall HEALTH_OK 2023-09-10T00:50:00.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:00 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:50:00.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:00 smithi099 ceph-mon[102692]: pgmap v289: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 22 MiB/s wr, 69 op/s 2023-09-10T00:50:00.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:00 smithi099 ceph-mon[102692]: overall HEALTH_OK 2023-09-10T00:50:02.157 INFO:teuthology.orchestra.run.smithi044.stderr:35000000 of 50000000 tuples (70%) done (elapsed 541.79 s, remaining 232.20 s) 2023-09-10T00:50:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:02 smithi131 ceph-mon[103713]: pgmap v290: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 26 MiB/s wr, 96 op/s 2023-09-10T00:50:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:02 smithi044 ceph-mon[98676]: pgmap v290: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 26 MiB/s wr, 96 op/s 2023-09-10T00:50:03.240 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:02 smithi099 ceph-mon[102692]: pgmap v290: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 26 MiB/s wr, 96 op/s 2023-09-10T00:50:03.355 INFO:teuthology.orchestra.run.smithi044.stderr:35100000 of 50000000 tuples (70%) done (elapsed 542.98 s, remaining 230.50 s) 2023-09-10T00:50:04.580 INFO:teuthology.orchestra.run.smithi044.stderr:35200000 of 50000000 tuples (70%) done (elapsed 544.21 s, remaining 228.82 s) 2023-09-10T00:50:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:04 smithi131 ceph-mon[103713]: pgmap v291: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 19 MiB/s wr, 56 op/s 2023-09-10T00:50:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:04 smithi131 ceph-mon[103713]: osdmap e99: 12 total, 12 up, 12 in 2023-09-10T00:50:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:04 smithi044 ceph-mon[98676]: pgmap v291: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 19 MiB/s wr, 56 op/s 2023-09-10T00:50:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:04 smithi044 ceph-mon[98676]: osdmap e99: 12 total, 12 up, 12 in 2023-09-10T00:50:05.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:04 smithi099 ceph-mon[102692]: pgmap v291: 129 pgs: 129 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 19 MiB/s wr, 56 op/s 2023-09-10T00:50:05.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:04 smithi099 ceph-mon[102692]: osdmap e99: 12 total, 12 up, 12 in 2023-09-10T00:50:05.874 INFO:teuthology.orchestra.run.smithi044.stderr:35300000 of 50000000 tuples (70%) done (elapsed 545.51 s, remaining 227.17 s) 2023-09-10T00:50:06.902 INFO:teuthology.orchestra.run.smithi044.stderr:35400000 of 50000000 tuples (70%) done (elapsed 546.54 s, remaining 225.41 s) 2023-09-10T00:50:07.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:06 smithi131 ceph-mon[103713]: pgmap v293: 129 pgs: 129 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 17 MiB/s wr, 104 op/s 2023-09-10T00:50:07.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:06 smithi044 ceph-mon[98676]: pgmap v293: 129 pgs: 129 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 17 MiB/s wr, 104 op/s 2023-09-10T00:50:07.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:06 smithi099 ceph-mon[102692]: pgmap v293: 129 pgs: 129 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 17 MiB/s wr, 104 op/s 2023-09-10T00:50:07.919 INFO:teuthology.orchestra.run.smithi044.stderr:35500000 of 50000000 tuples (71%) done (elapsed 547.55 s, remaining 223.65 s) 2023-09-10T00:50:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.842+0000 7f58e3034700 1 -- 172.21.15.44:0/1017333595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58dc103180 msgr2=0x7f58dc10f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.842+0000 7f58e3034700 1 --2- 172.21.15.44:0/1017333595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58dc103180 0x7f58dc10f690 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f58d400b0a0 tx=0x7f58d400b3b0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.842+0000 7f58e3034700 1 -- 172.21.15.44:0/1017333595 shutdown_connections 2023-09-10T00:50:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.842+0000 7f58e3034700 1 --2- 172.21.15.44:0/1017333595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58dc103180 0x7f58dc10f690 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.842+0000 7f58e3034700 1 --2- 172.21.15.44:0/1017333595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58dc1087d0 0x7f58dc108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.842+0000 7f58e3034700 1 --2- 172.21.15.44:0/1017333595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58dc1027c0 0x7f58dc102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.842+0000 7f58e3034700 1 -- 172.21.15.44:0/1017333595 >> 172.21.15.44:0/1017333595 conn(0x7f58dc0fe290 msgr2=0x7f58dc1006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.842+0000 7f58e3034700 1 -- 172.21.15.44:0/1017333595 shutdown_connections 2023-09-10T00:50:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.842+0000 7f58e3034700 1 -- 172.21.15.44:0/1017333595 wait complete. 2023-09-10T00:50:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.842+0000 7f58e3034700 1 Processor -- start 2023-09-10T00:50:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58e3034700 1 -- start start 2023-09-10T00:50:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58e3034700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58dc1027c0 0x7f58dc19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58e3034700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58dc103180 0x7f58dc19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58e3034700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58dc1087d0 0x7f58dc198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58e3034700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc198bd0 con 0x7f58dc1027c0 2023-09-10T00:50:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58e3034700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc198d40 con 0x7f58dc1087d0 2023-09-10T00:50:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58e3034700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc198f00 con 0x7f58dc103180 2023-09-10T00:50:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58e0dd0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58dc1027c0 0x7f58dc19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58dbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58dc103180 0x7f58dc19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58e15d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58dc1087d0 0x7f58dc198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58dbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58dc103180 0x7f58dc19ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59050/0 (socket says 172.21.15.44:59050) 2023-09-10T00:50:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58e15d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58dc1087d0 0x7f58dc198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53910/0 (socket says 172.21.15.44:53910) 2023-09-10T00:50:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58dbfff700 1 -- 172.21.15.44:0/684179283 learned_addr learned my addr 172.21.15.44:0/684179283 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58e15d1700 1 -- 172.21.15.44:0/684179283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58dc103180 msgr2=0x7f58dc19ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58e15d1700 1 --2- 172.21.15.44:0/684179283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58dc103180 0x7f58dc19ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58e15d1700 1 -- 172.21.15.44:0/684179283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58dc1027c0 msgr2=0x7f58dc19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58e15d1700 1 --2- 172.21.15.44:0/684179283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58dc1027c0 0x7f58dc19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.843+0000 7f58e15d1700 1 -- 172.21.15.44:0/684179283 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d0009520 con 0x7f58dc1087d0 2023-09-10T00:50:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.844+0000 7f58e15d1700 1 --2- 172.21.15.44:0/684179283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58dc1087d0 0x7f58dc198560 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f58d400b0a0 tx=0x7f58d40094d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.844+0000 7f58d9ffb700 1 -- 172.21.15.44:0/684179283 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d400e040 con 0x7f58dc1087d0 2023-09-10T00:50:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.844+0000 7f58d9ffb700 1 -- 172.21.15.44:0/684179283 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58d401d470 con 0x7f58dc1087d0 2023-09-10T00:50:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.844+0000 7f58d9ffb700 1 -- 172.21.15.44:0/684179283 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d40052a0 con 0x7f58dc1087d0 2023-09-10T00:50:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.844+0000 7f58e3034700 1 -- 172.21.15.44:0/684179283 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d4009d00 con 0x7f58dc1087d0 2023-09-10T00:50:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.845+0000 7f58e3034700 1 -- 172.21.15.44:0/684179283 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc1a5510 con 0x7f58dc1087d0 2023-09-10T00:50:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.848+0000 7f58d9ffb700 1 -- 172.21.15.44:0/684179283 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f58d401d5e0 con 0x7f58dc1087d0 2023-09-10T00:50:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.849+0000 7f58d9ffb700 1 --2- 172.21.15.44:0/684179283 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f58cc0a0830 0x7f58cc0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.849+0000 7f58d9ffb700 1 -- 172.21.15.44:0/684179283 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f58d40cd940 con 0x7f58dc1087d0 2023-09-10T00:50:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.849+0000 7f58e0dd0700 1 --2- 172.21.15.44:0/684179283 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f58cc0a0830 0x7f58cc0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.849+0000 7f58ef1fab80 1 Processor -- start 2023-09-10T00:50:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.849+0000 7f58ef1fab80 1 -- start start 2023-09-10T00:50:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.849+0000 7f58ef1fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738f8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.849+0000 7f58ef1fab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d7738f2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.849+0000 7f58ef1fab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2cc0 0x55d7738fef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.849+0000 7f58ef1fab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d77384b6d0 con 0x55d7738f2cc0 2023-09-10T00:50:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.849+0000 7f58ef1fab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773848ff0 con 0x55d7738f2300 2023-09-10T00:50:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.849+0000 7f58ef1fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7738448e0 con 0x55d7738f81a0 2023-09-10T00:50:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.850+0000 7f58e0dd0700 1 --2- 172.21.15.44:0/684179283 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f58cc0a0830 0x7f58cc0a2cf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009d70 tx=0x7f58d0009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.850+0000 7f58e15d1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2cc0 0x55d7738fef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.850+0000 7f58e0dd0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.850+0000 7f58dbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d7738f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.850+0000 7f58e0dd0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738f8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59056/0 (socket says 172.21.15.44:59056) 2023-09-10T00:50:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.850+0000 7f58e15d1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2cc0 0x55d7738fef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52178/0 (socket says 172.21.15.44:52178) 2023-09-10T00:50:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.850+0000 7f58e0dd0700 1 -- 172.21.15.44:0/2325247232 learned_addr learned my addr 172.21.15.44:0/2325247232 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.850+0000 7f58dbfff700 1 -- 172.21.15.44:0/2325247232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 msgr2=0x55d7738f8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.850+0000 7f58dbfff700 1 --2- 172.21.15.44:0/2325247232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738f8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.850+0000 7f58dbfff700 1 -- 172.21.15.44:0/2325247232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2cc0 msgr2=0x55d7738fef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.850+0000 7f58dbfff700 1 --2- 172.21.15.44:0/2325247232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2cc0 0x55d7738fef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.850+0000 7f58dbfff700 1 -- 172.21.15.44:0/2325247232 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d773670d80 con 0x55d7738f2300 2023-09-10T00:50:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.850+0000 7f58e15d1700 1 --2- 172.21.15.44:0/2325247232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2cc0 0x55d7738fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.850+0000 7f58dbfff700 1 --2- 172.21.15.44:0/2325247232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d7738f2780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f58dc1999d0 tx=0x7f58c8011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d612957de3e21c7f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.850+0000 7f58c6ffd700 1 -- 172.21.15.44:0/2325247232 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8011d80 con 0x55d7738f2300 2023-09-10T00:50:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.850+0000 7f58c6ffd700 1 -- 172.21.15.44:0/2325247232 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58c80044e0 con 0x55d7738f2300 2023-09-10T00:50:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.851+0000 7f58c6ffd700 1 -- 172.21.15.44:0/2325247232 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8005280 con 0x55d7738f2300 2023-09-10T00:50:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.851+0000 7f58ef1fab80 1 -- 172.21.15.44:0/2325247232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 msgr2=0x55d7738f2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.851+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/2325247232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d7738f2780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f58dc1999d0 tx=0x7f58c8011440 comp rx=0 tx=0).stop 2023-09-10T00:50:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.851+0000 7f58ef1fab80 1 -- 172.21.15.44:0/2325247232 shutdown_connections 2023-09-10T00:50:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.851+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/2325247232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2cc0 0x55d7738fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.851+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/2325247232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.851+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/2325247232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d7738f2780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.851+0000 7f58ef1fab80 1 -- 172.21.15.44:0/2325247232 >> 172.21.15.44:0/2325247232 conn(0x55d773867030 msgr2=0x55d7738ed560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.852+0000 7f58ef1fab80 1 -- 172.21.15.44:0/2325247232 shutdown_connections 2023-09-10T00:50:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.852+0000 7f58ef1fab80 1 -- 172.21.15.44:0/2325247232 wait complete. 2023-09-10T00:50:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.852+0000 7f58ef1fab80 1 -- start start 2023-09-10T00:50:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.852+0000 7f58ef1fab80 1 Processor -- start 2023-09-10T00:50:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58ef1fab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2300 0x55d7738fed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58ef1fab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2cc0 0x55d7738fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58ef1fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738fc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58ef1fab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7737e2b10 con 0x55d7738f2300 2023-09-10T00:50:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58ef1fab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7737e2810 con 0x55d7738f2cc0 2023-09-10T00:50:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58ef1fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7736c68c0 con 0x55d7738f81a0 2023-09-10T00:50:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58ef1fab80 10 client.0 _ll_register_callbacks cb 0x55d77372ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:50:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58dbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2cc0 0x55d7738fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58e0dd0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2300 0x55d7738fed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58dbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2cc0 0x55d7738fc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53922/0 (socket says 172.21.15.44:53922) 2023-09-10T00:50:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58dbfff700 1 -- 172.21.15.44:0/1157809262 learned_addr learned my addr 172.21.15.44:0/1157809262 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58e15d1700 1 --2- 172.21.15.44:0/1157809262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738fc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58dbfff700 1 -- 172.21.15.44:0/1157809262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 msgr2=0x55d7738fc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58dbfff700 1 --2- 172.21.15.44:0/1157809262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738fc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58dbfff700 1 -- 172.21.15.44:0/1157809262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2300 msgr2=0x55d7738fed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58dbfff700 1 --2- 172.21.15.44:0/1157809262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2300 0x55d7738fed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58dbfff700 1 -- 172.21.15.44:0/1157809262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc1a5510 con 0x55d7738f2cc0 2023-09-10T00:50:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58e15d1700 1 --2- 172.21.15.44:0/1157809262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738fc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:50:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58dbfff700 1 --2- 172.21.15.44:0/1157809262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2cc0 0x55d7738fc0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f58c8000c00 tx=0x7f58c8005d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95c7e3cd955c6715 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.853+0000 7f58bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:50:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.854+0000 7f58bbfff700 1 -- 172.21.15.44:0/1157809262 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8003680 con 0x55d7738f2cc0 2023-09-10T00:50:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.854+0000 7f58bbfff700 1 -- 172.21.15.44:0/1157809262 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f58c80037e0 con 0x55d7738f2cc0 2023-09-10T00:50:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.854+0000 7f58ef1fab80 1 -- 172.21.15.44:0/1157809262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d77379c490 con 0x55d7738f2cc0 2023-09-10T00:50:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.854+0000 7f58bbfff700 1 -- 172.21.15.44:0/1157809262 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58c80039a0 con 0x55d7738f2cc0 2023-09-10T00:50:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.854+0000 7f58ef1fab80 10 client.26897 fetch_fsmap learned FSMap version 59 2023-09-10T00:50:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.854+0000 7f58ef1fab80 1 -- 172.21.15.44:0/1157809262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7f58c8009970 con 0x55d7738f2cc0 2023-09-10T00:50:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.854+0000 7f58bbfff700 1 -- 172.21.15.44:0/1157809262 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d77379c490 con 0x55d7738f2cc0 2023-09-10T00:50:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.854+0000 7f58bbfff700 1 -- 172.21.15.44:0/1157809262 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 59) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f58c8022b70 con 0x55d7738f2cc0 2023-09-10T00:50:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.854+0000 7f58ef1fab80 10 client.26897 fetch_fsmap finished waiting for FSMap version 59 2023-09-10T00:50:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.854+0000 7f58ef1fab80 10 client.26897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:50:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.854+0000 7f58ef1fab80 20 client.26897 populate_metadata read hostname 'smithi044' 2023-09-10T00:50:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.854+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/1157809262 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d77390c480 0x55d77390e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.854+0000 7f58ef1fab80 4 client.26897 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:50:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.854+0000 7f58ef1fab80 1 -- 172.21.15.44:0/1157809262 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d773859840 con 0x55d77390c480 2023-09-10T00:50:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.854+0000 7f58e0dd0700 1 --2- 172.21.15.44:0/1157809262 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d77390c480 0x55d77390e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.855+0000 7f58e0dd0700 1 --2- 172.21.15.44:0/1157809262 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d77390c480 0x55d77390e8a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61d7d251b93d9df server_cookie=54e3920cfd863c6e in_seq=0 out_seq=0 2023-09-10T00:50:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.855+0000 7f58bbfff700 10 client.26897 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.855+0000 7f58bbfff700 1 -- 172.21.15.44:0/1157809262 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d773859840 con 0x55d77390c480 2023-09-10T00:50:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.855+0000 7f58bbfff700 10 client.26897 handle_command_reply: tid=0 2023-09-10T00:50:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.855+0000 7f58ef1fab80 1 client.26897 shutdown 2023-09-10T00:50:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.856+0000 7f58ef1fab80 1 -- 172.21.15.44:0/1157809262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2cc0 msgr2=0x55d7738fc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.856+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/1157809262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2cc0 0x55d7738fc0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f58c8000c00 tx=0x7f58c8005d60 comp rx=0 tx=0).stop 2023-09-10T00:50:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.856+0000 7f58ef1fab80 1 -- 172.21.15.44:0/1157809262 shutdown_connections 2023-09-10T00:50:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.856+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/1157809262 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d77390c480 0x55d77390e8a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.856+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/1157809262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738fc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.856+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/1157809262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2300 0x55d7738fed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.856+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/1157809262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2cc0 0x55d7738fc0a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.856+0000 7f58bbfff700 0 client.26897 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.856+0000 7f58ef1fab80 1 -- 172.21.15.44:0/1157809262 >> 172.21.15.44:0/1157809262 conn(0x55d773867030 msgr2=0x55d7738ec7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.856+0000 7f58ef1fab80 1 -- 172.21.15.44:0/1157809262 shutdown_connections 2023-09-10T00:50:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.856+0000 7f58ef1fab80 1 -- 172.21.15.44:0/1157809262 wait complete. 2023-09-10T00:50:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.856+0000 7f58ef1fab80 20 client.26897 trim_cache size 0 max 16384 2023-09-10T00:50:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.870+0000 7f58ef1fab80 1 Processor -- start 2023-09-10T00:50:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58ef1fab80 1 -- start start 2023-09-10T00:50:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58ef1fab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d7738f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58ef1fab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2cc0 0x55d7738f7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58ef1fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738f6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58ef1fab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7738e9fc0 con 0x55d7738f2cc0 2023-09-10T00:50:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58ef1fab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7738f7de0 con 0x55d7738f2300 2023-09-10T00:50:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58ef1fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773908390 con 0x55d7738f81a0 2023-09-10T00:50:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58e0dd0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d7738f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58e15d1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58e0dd0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d7738f70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53928/0 (socket says 172.21.15.44:53928) 2023-09-10T00:50:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58e15d1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738f6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59072/0 (socket says 172.21.15.44:59072) 2023-09-10T00:50:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58e0dd0700 1 -- 172.21.15.44:0/401767734 learned_addr learned my addr 172.21.15.44:0/401767734 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58e0dd0700 1 -- 172.21.15.44:0/401767734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 msgr2=0x55d7738f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58e0dd0700 1 --2- 172.21.15.44:0/401767734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58e0dd0700 1 -- 172.21.15.44:0/401767734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2cc0 msgr2=0x55d7738f7620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T00:50:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58e0dd0700 1 --2- 172.21.15.44:0/401767734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2cc0 0x55d7738f7620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58e0dd0700 1 -- 172.21.15.44:0/401767734 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d773659c00 con 0x55d7738f2300 2023-09-10T00:50:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58e15d1700 1 --2- 172.21.15.44:0/401767734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:08.879 INFO:teuthology.orchestra.run.smithi044.stderr:35600000 of 50000000 tuples (71%) done (elapsed 548.51 s, remaining 221.87 s) 2023-09-10T00:50:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.871+0000 7f58e0dd0700 1 --2- 172.21.15.44:0/401767734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d7738f70e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55d77390f1b0 tx=0x7f58d0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74a397a7ede68e73 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.872+0000 7f58c77fe700 1 -- 172.21.15.44:0/401767734 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0024040 con 0x55d7738f2300 2023-09-10T00:50:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.872+0000 7f58c77fe700 1 -- 172.21.15.44:0/401767734 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58d00044c0 con 0x55d7738f2300 2023-09-10T00:50:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.872+0000 7f58c77fe700 1 -- 172.21.15.44:0/401767734 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0003680 con 0x55d7738f2300 2023-09-10T00:50:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.872+0000 7f58ef1fab80 1 -- 172.21.15.44:0/401767734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 msgr2=0x55d7738f70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.872+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/401767734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d7738f70e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55d77390f1b0 tx=0x7f58d0014000 comp rx=0 tx=0).stop 2023-09-10T00:50:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.873+0000 7f58ef1fab80 1 -- 172.21.15.44:0/401767734 shutdown_connections 2023-09-10T00:50:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.873+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/401767734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.873+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/401767734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d7738f70e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.873+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/401767734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2cc0 0x55d7738f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.873+0000 7f58ef1fab80 1 -- 172.21.15.44:0/401767734 >> 172.21.15.44:0/401767734 conn(0x55d773867030 msgr2=0x55d773907c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.873+0000 7f58ef1fab80 1 -- 172.21.15.44:0/401767734 shutdown_connections 2023-09-10T00:50:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.873+0000 7f58ef1fab80 1 -- 172.21.15.44:0/401767734 wait complete. 2023-09-10T00:50:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.873+0000 7f58ef1fab80 1 -- start start 2023-09-10T00:50:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.873+0000 7f58ef1fab80 1 Processor -- start 2023-09-10T00:50:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58ef1fab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d773903200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58ef1fab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2cc0 0x55d7738ff500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58ef1fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58ef1fab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773903b00 con 0x55d7738f2cc0 2023-09-10T00:50:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58ef1fab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7738fffb0 con 0x55d7738f2300 2023-09-10T00:50:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58ef1fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773900170 con 0x55d7738f81a0 2023-09-10T00:50:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58ef1fab80 10 client.0 _ll_register_callbacks cb 0x7f58d401df50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:50:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58e0dd0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d773903200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58e0dd0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d773903200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53932/0 (socket says 172.21.15.44:53932) 2023-09-10T00:50:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58dbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2cc0 0x55d7738ff500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58e15d1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58e0dd0700 1 -- 172.21.15.44:0/587914923 learned_addr learned my addr 172.21.15.44:0/587914923 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58e0dd0700 1 -- 172.21.15.44:0/587914923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 msgr2=0x55d7738ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58e0dd0700 1 --2- 172.21.15.44:0/587914923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58e0dd0700 1 -- 172.21.15.44:0/587914923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2cc0 msgr2=0x55d7738ff500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58e0dd0700 1 --2- 172.21.15.44:0/587914923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2cc0 0x55d7738ff500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58e0dd0700 1 -- 172.21.15.44:0/587914923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d4009d00 con 0x55d7738f2300 2023-09-10T00:50:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58e15d1700 1 --2- 172.21.15.44:0/587914923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58e0dd0700 1 --2- 172.21.15.44:0/587914923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d773903200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f58d000e740 tx=0x7f58d0029a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc589954c8159ff5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:50:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.874+0000 7f58bbfff700 1 -- 172.21.15.44:0/587914923 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0024050 con 0x55d7738f2300 2023-09-10T00:50:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.875+0000 7f58ef1fab80 1 -- 172.21.15.44:0/587914923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58c80044e0 con 0x55d7738f2300 2023-09-10T00:50:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.875+0000 7f58bbfff700 1 -- 172.21.15.44:0/587914923 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f58d0029d80 con 0x55d7738f2300 2023-09-10T00:50:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.875+0000 7f58bbfff700 1 -- 172.21.15.44:0/587914923 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58d003a480 con 0x55d7738f2300 2023-09-10T00:50:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.875+0000 7f58ef1fab80 10 client.26903 fetch_fsmap learned FSMap version 59 2023-09-10T00:50:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.875+0000 7f58ef1fab80 1 -- 172.21.15.44:0/587914923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7f58d0017070 con 0x55d7738f2300 2023-09-10T00:50:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.875+0000 7f58bbfff700 1 -- 172.21.15.44:0/587914923 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58c80044e0 con 0x55d7738f2300 2023-09-10T00:50:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.875+0000 7f58bbfff700 1 -- 172.21.15.44:0/587914923 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 59) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f58d001e5e0 con 0x55d7738f2300 2023-09-10T00:50:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.875+0000 7f58ef1fab80 10 client.26903 fetch_fsmap finished waiting for FSMap version 59 2023-09-10T00:50:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.875+0000 7f58ef1fab80 10 client.26903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:50:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.875+0000 7f58ef1fab80 20 client.26903 populate_metadata read hostname 'smithi044' 2023-09-10T00:50:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.875+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/587914923 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d773961690 0x55d773963ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.875+0000 7f58ef1fab80 4 client.26903 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:50:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.875+0000 7f58ef1fab80 1 -- 172.21.15.44:0/587914923 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d77383d8c0 con 0x55d773961690 2023-09-10T00:50:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.875+0000 7f58dbfff700 1 --2- 172.21.15.44:0/587914923 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d773961690 0x55d773963ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.876+0000 7f58bbfff700 10 client.26903 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.876+0000 7f58dbfff700 1 --2- 172.21.15.44:0/587914923 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d773961690 0x55d773963ab0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fdbb47a82df3a04 server_cookie=63e6603c43d8340f in_seq=0 out_seq=0 2023-09-10T00:50:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.891+0000 7f58bbfff700 1 -- 172.21.15.44:0/587914923 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d77383d8c0 con 0x55d773961690 2023-09-10T00:50:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.891+0000 7f58bbfff700 10 client.26903 handle_command_reply: tid=0 2023-09-10T00:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.891+0000 7f58ef1fab80 1 client.26903 shutdown 2023-09-10T00:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.892+0000 7f58ef1fab80 1 -- 172.21.15.44:0/587914923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 msgr2=0x55d773903200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.892+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/587914923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d773903200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f58d000e740 tx=0x7f58d0029a60 comp rx=0 tx=0).stop 2023-09-10T00:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.892+0000 7f58ef1fab80 1 -- 172.21.15.44:0/587914923 shutdown_connections 2023-09-10T00:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.892+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/587914923 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d773961690 0x55d773963ab0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.892+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/587914923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7738f81a0 0x55d7738ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.892+0000 7f58bbfff700 0 client.26903 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.892+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/587914923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7738f2300 0x55d773903200 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.892+0000 7f58ef1fab80 1 --2- 172.21.15.44:0/587914923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7738f2cc0 0x55d7738ff500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.892+0000 7f58ef1fab80 1 -- 172.21.15.44:0/587914923 >> 172.21.15.44:0/587914923 conn(0x55d773867030 msgr2=0x55d7738f70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.892+0000 7f58ef1fab80 1 -- 172.21.15.44:0/587914923 shutdown_connections 2023-09-10T00:50:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.892+0000 7f58ef1fab80 1 -- 172.21.15.44:0/587914923 wait complete. 2023-09-10T00:50:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.892+0000 7f58ef1fab80 20 client.26903 trim_cache size 0 max 16384 2023-09-10T00:50:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:50:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:50:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:50:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:50:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.893+0000 7f58e3034700 1 -- 172.21.15.44:0/684179283 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f58cc0a0830 msgr2=0x7f58cc0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.893+0000 7f58e3034700 1 --2- 172.21.15.44:0/684179283 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f58cc0a0830 0x7f58cc0a2cf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009d70 tx=0x7f58d0009260 comp rx=0 tx=0).stop 2023-09-10T00:50:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.893+0000 7f58e3034700 1 -- 172.21.15.44:0/684179283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58dc1087d0 msgr2=0x7f58dc198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.893+0000 7f58e3034700 1 --2- 172.21.15.44:0/684179283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58dc1087d0 0x7f58dc198560 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f58d400b0a0 tx=0x7f58d40094d0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.893+0000 7f58e3034700 1 -- 172.21.15.44:0/684179283 shutdown_connections 2023-09-10T00:50:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.893+0000 7f58e3034700 1 --2- 172.21.15.44:0/684179283 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f58cc0a0830 0x7f58cc0a2cf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.893+0000 7f58e3034700 1 --2- 172.21.15.44:0/684179283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58dc1087d0 0x7f58dc198560 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.893+0000 7f58e3034700 1 --2- 172.21.15.44:0/684179283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58dc1027c0 0x7f58dc19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.893+0000 7f58e3034700 1 --2- 172.21.15.44:0/684179283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58dc103180 0x7f58dc19ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.893+0000 7f58e3034700 1 -- 172.21.15.44:0/684179283 >> 172.21.15.44:0/684179283 conn(0x7f58dc0fe290 msgr2=0x7f58dc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.893+0000 7f58e3034700 1 -- 172.21.15.44:0/684179283 shutdown_connections 2023-09-10T00:50:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:08.893+0000 7f58e3034700 1 -- 172.21.15.44:0/684179283 wait complete. 2023-09-10T00:50:09.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:08 smithi131 ceph-mon[103713]: pgmap v294: 129 pgs: 129 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 17 MiB/s wr, 104 op/s 2023-09-10T00:50:09.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:08 smithi044 ceph-mon[98676]: pgmap v294: 129 pgs: 129 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 17 MiB/s wr, 104 op/s 2023-09-10T00:50:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:08 smithi099 ceph-mon[102692]: pgmap v294: 129 pgs: 129 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 17 MiB/s wr, 104 op/s 2023-09-10T00:50:09.808 INFO:teuthology.orchestra.run.smithi044.stderr:35700000 of 50000000 tuples (71%) done (elapsed 549.44 s, remaining 220.08 s) 2023-09-10T00:50:09.943 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:50:09.974 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:09 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:50:10.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:09 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:50:10.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:09 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:50:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.115+0000 7fc62e9f4700 1 -- 172.21.15.44:0/687054679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc628105570 msgr2=0x7fc628105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.115+0000 7fc62e9f4700 1 --2- 172.21.15.44:0/687054679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc628105570 0x7fc628105950 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc6180099c0 tx=0x7fc618009cd0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.115+0000 7fc62e9f4700 1 -- 172.21.15.44:0/687054679 shutdown_connections 2023-09-10T00:50:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.115+0000 7fc62e9f4700 1 --2- 172.21.15.44:0/687054679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6280ffe70 0x7fc62810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.115+0000 7fc62e9f4700 1 --2- 172.21.15.44:0/687054679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc628105570 0x7fc628105950 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.115+0000 7fc62e9f4700 1 --2- 172.21.15.44:0/687054679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6280ff4b0 0x7fc6280ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.115+0000 7fc62e9f4700 1 -- 172.21.15.44:0/687054679 >> 172.21.15.44:0/687054679 conn(0x7fc6280fafa0 msgr2=0x7fc6280fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.115+0000 7fc62e9f4700 1 -- 172.21.15.44:0/687054679 shutdown_connections 2023-09-10T00:50:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.115+0000 7fc62e9f4700 1 -- 172.21.15.44:0/687054679 wait complete. 2023-09-10T00:50:10.119 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.116+0000 7fc62e9f4700 1 Processor -- start 2023-09-10T00:50:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.116+0000 7fc62e9f4700 1 -- start start 2023-09-10T00:50:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc62e9f4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6280ff4b0 0x7fc628197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc62e9f4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6280ffe70 0x7fc6281978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc62e9f4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc628105570 0x7fc6281a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc62e9f4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc628198210 con 0x7fc6280ffe70 2023-09-10T00:50:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc62e9f4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6281a2950 con 0x7fc628105570 2023-09-10T00:50:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc62e9f4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6281a2b10 con 0x7fc6280ff4b0 2023-09-10T00:50:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc62cf91700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc628105570 0x7fc6281a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc627fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6280ff4b0 0x7fc628197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc62cf91700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc628105570 0x7fc6281a2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53948/0 (socket says 172.21.15.44:53948) 2023-09-10T00:50:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc627fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6280ff4b0 0x7fc628197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59088/0 (socket says 172.21.15.44:59088) 2023-09-10T00:50:10.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc62cf91700 1 -- 172.21.15.44:0/2330731199 learned_addr learned my addr 172.21.15.44:0/2330731199 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:10.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc6277fe700 1 --2- 172.21.15.44:0/2330731199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6280ffe70 0x7fc6281978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc62cf91700 1 -- 172.21.15.44:0/2330731199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6280ff4b0 msgr2=0x7fc628197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:10.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc62cf91700 1 --2- 172.21.15.44:0/2330731199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6280ff4b0 0x7fc628197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.122 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc62cf91700 1 -- 172.21.15.44:0/2330731199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6280ffe70 msgr2=0x7fc6281978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:10.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc62cf91700 1 --2- 172.21.15.44:0/2330731199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6280ffe70 0x7fc6281978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc62cf91700 1 -- 172.21.15.44:0/2330731199 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6180095f0 con 0x7fc628105570 2023-09-10T00:50:10.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.117+0000 7fc627fff700 1 --2- 172.21.15.44:0/2330731199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6280ff4b0 0x7fc628197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:10.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.118+0000 7fc62cf91700 1 --2- 172.21.15.44:0/2330731199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc628105570 0x7fc6281a2410 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc62000ef30 tx=0x7fc62000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:10.123 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.118+0000 7fc6257fa700 1 -- 172.21.15.44:0/2330731199 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc620018070 con 0x7fc628105570 2023-09-10T00:50:10.124 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.118+0000 7fc6257fa700 1 -- 172.21.15.44:0/2330731199 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc62000f040 con 0x7fc628105570 2023-09-10T00:50:10.124 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.118+0000 7fc62e9f4700 1 -- 172.21.15.44:0/2330731199 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6281a2d90 con 0x7fc628105570 2023-09-10T00:50:10.124 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.118+0000 7fc6257fa700 1 -- 172.21.15.44:0/2330731199 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62000cbc0 con 0x7fc628105570 2023-09-10T00:50:10.124 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.118+0000 7fc62e9f4700 1 -- 172.21.15.44:0/2330731199 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6281a32c0 con 0x7fc628105570 2023-09-10T00:50:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.121+0000 7fc6257fa700 1 -- 172.21.15.44:0/2330731199 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc620007500 con 0x7fc628105570 2023-09-10T00:50:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.122+0000 7fc6257fa700 1 --2- 172.21.15.44:0/2330731199 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc6100a0830 0x7fc6100a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.122+0000 7fc6257fa700 1 -- 172.21.15.44:0/2330731199 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fc6200cbd90 con 0x7fc628105570 2023-09-10T00:50:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.122+0000 7fc627fff700 1 --2- 172.21.15.44:0/2330731199 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc6100a0830 0x7fc6100a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.123+0000 7fc627fff700 1 --2- 172.21.15.44:0/2330731199 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc6100a0830 0x7fc6100a2cf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc61800b580 tx=0x7fc61801a040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.123+0000 7fc63abbab80 1 Processor -- start 2023-09-10T00:50:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.123+0000 7fc63abbab80 1 -- start start 2023-09-10T00:50:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.123+0000 7fc63abbab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0b710 0x5581fee0baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc63abbab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 0x5581fee15900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc63abbab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 0x5581fee18230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc63abbab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fed64730 con 0x5581fee15e40 2023-09-10T00:50:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc63abbab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fed62050 con 0x5581fee0c030 2023-09-10T00:50:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc63abbab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fed5d940 con 0x5581fee0b710 2023-09-10T00:50:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc627fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0b710 0x5581fee0baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc627fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0b710 0x5581fee0baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59096/0 (socket says 172.21.15.44:59096) 2023-09-10T00:50:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc62cf91700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 0x5581fee18230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc627fff700 1 -- 172.21.15.44:0/3725697303 learned_addr learned my addr 172.21.15.44:0/3725697303 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc6277fe700 1 --2- 172.21.15.44:0/3725697303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 0x5581fee15900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc627fff700 1 -- 172.21.15.44:0/3725697303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 msgr2=0x5581fee15900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc627fff700 1 --2- 172.21.15.44:0/3725697303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 0x5581fee15900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc627fff700 1 -- 172.21.15.44:0/3725697303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 msgr2=0x5581fee18230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc627fff700 1 --2- 172.21.15.44:0/3725697303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 0x5581fee18230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc627fff700 1 -- 172.21.15.44:0/3725697303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581feb89de0 con 0x5581fee0b710 2023-09-10T00:50:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc627fff700 1 --2- 172.21.15.44:0/3725697303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0b710 0x5581fee0baf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc6180099c0 tx=0x7fc61801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a10fc66aa11a8caf server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.124+0000 7fc60e7fc700 1 -- 172.21.15.44:0/3725697303 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6180056a0 con 0x5581fee0b710 2023-09-10T00:50:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.125+0000 7fc60e7fc700 1 -- 172.21.15.44:0/3725697303 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc61801e040 con 0x5581fee0b710 2023-09-10T00:50:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.125+0000 7fc60e7fc700 1 -- 172.21.15.44:0/3725697303 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61800f920 con 0x5581fee0b710 2023-09-10T00:50:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.125+0000 7fc63abbab80 1 -- 172.21.15.44:0/3725697303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0b710 msgr2=0x5581fee0baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.125+0000 7fc63abbab80 1 --2- 172.21.15.44:0/3725697303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0b710 0x5581fee0baf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc6180099c0 tx=0x7fc61801f000 comp rx=0 tx=0).stop 2023-09-10T00:50:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.125+0000 7fc63abbab80 1 -- 172.21.15.44:0/3725697303 shutdown_connections 2023-09-10T00:50:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.126+0000 7fc63abbab80 1 --2- 172.21.15.44:0/3725697303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 0x5581fee18230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.126+0000 7fc63abbab80 1 --2- 172.21.15.44:0/3725697303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0b710 0x5581fee0baf0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.126+0000 7fc63abbab80 1 --2- 172.21.15.44:0/3725697303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 0x5581fee15900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.126+0000 7fc63abbab80 1 -- 172.21.15.44:0/3725697303 >> 172.21.15.44:0/3725697303 conn(0x5581fed80090 msgr2=0x5581fee06bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.126+0000 7fc63abbab80 1 -- 172.21.15.44:0/3725697303 shutdown_connections 2023-09-10T00:50:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.126+0000 7fc63abbab80 1 -- 172.21.15.44:0/3725697303 wait complete. 2023-09-10T00:50:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.126+0000 7fc63abbab80 1 -- start start 2023-09-10T00:50:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.126+0000 7fc63abbab80 1 Processor -- start 2023-09-10T00:50:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc63abbab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0b710 0x5581fee157c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc63abbab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 0x5581fee12a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc63abbab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 0x5581fee13070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc63abbab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fecfbb70 con 0x5581fee15e40 2023-09-10T00:50:10.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc63abbab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fecfb870 con 0x5581fee0c030 2023-09-10T00:50:10.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc63abbab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581febdf920 con 0x5581fee0b710 2023-09-10T00:50:10.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc63abbab80 10 client.0 _ll_register_callbacks cb 0x5581fec45a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:50:10.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc627fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0b710 0x5581fee157c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc6277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 0x5581fee12a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.138 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc6277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 0x5581fee12a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53962/0 (socket says 172.21.15.44:53962) 2023-09-10T00:50:10.138 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc627fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0b710 0x5581fee157c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59102/0 (socket says 172.21.15.44:59102) 2023-09-10T00:50:10.138 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc6277fe700 1 -- 172.21.15.44:0/3747883126 learned_addr learned my addr 172.21.15.44:0/3747883126 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:10.138 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc62cf91700 1 --2- 172.21.15.44:0/3747883126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 0x5581fee13070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.138 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc627fff700 1 -- 172.21.15.44:0/3747883126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 msgr2=0x5581fee12a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc627fff700 1 --2- 172.21.15.44:0/3747883126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 0x5581fee12a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc627fff700 1 -- 172.21.15.44:0/3747883126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 msgr2=0x5581fee13070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc627fff700 1 --2- 172.21.15.44:0/3747883126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 0x5581fee13070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc627fff700 1 -- 172.21.15.44:0/3747883126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62810cc30 con 0x5581fee0b710 2023-09-10T00:50:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.127+0000 7fc6277fe700 1 --2- 172.21.15.44:0/3747883126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 0x5581fee12a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.128+0000 7fc627fff700 1 --2- 172.21.15.44:0/3747883126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0b710 0x5581fee157c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc618005d10 tx=0x7fc618005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8d0103cc4766156 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.128+0000 7fc6077fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:50:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.128+0000 7fc6077fe700 1 -- 172.21.15.44:0/3747883126 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61800f7e0 con 0x5581fee0b710 2023-09-10T00:50:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.128+0000 7fc6077fe700 1 -- 172.21.15.44:0/3747883126 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fc61800f940 con 0x5581fee0b710 2023-09-10T00:50:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.128+0000 7fc63abbab80 1 -- 172.21.15.44:0/3747883126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581fecb54f0 con 0x5581fee0b710 2023-09-10T00:50:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.128+0000 7fc6077fe700 1 -- 172.21.15.44:0/3747883126 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc61800feb0 con 0x5581fee0b710 2023-09-10T00:50:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.128+0000 7fc63abbab80 10 client.26857 fetch_fsmap learned FSMap version 59 2023-09-10T00:50:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.128+0000 7fc63abbab80 1 -- 172.21.15.44:0/3747883126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x5581feb72c60 con 0x5581fee0b710 2023-09-10T00:50:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.128+0000 7fc6077fe700 1 -- 172.21.15.44:0/3747883126 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5581fecb54f0 con 0x5581fee0b710 2023-09-10T00:50:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.128+0000 7fc6077fe700 1 -- 172.21.15.44:0/3747883126 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 59) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fc618038930 con 0x5581fee0b710 2023-09-10T00:50:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.128+0000 7fc63abbab80 10 client.26857 fetch_fsmap finished waiting for FSMap version 59 2023-09-10T00:50:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.128+0000 7fc63abbab80 10 client.26857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:50:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.128+0000 7fc63abbab80 20 client.26857 populate_metadata read hostname 'smithi044' 2023-09-10T00:50:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.129+0000 7fc63abbab80 1 --2- 172.21.15.44:0/3747883126 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5581fee29860 0x5581fee2bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.129+0000 7fc63abbab80 4 client.26857 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:50:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.129+0000 7fc63abbab80 1 -- 172.21.15.44:0/3747883126 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581fed728a0 con 0x5581fee29860 2023-09-10T00:50:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.129+0000 7fc6277fe700 1 --2- 172.21.15.44:0/3747883126 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5581fee29860 0x5581fee2bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.129+0000 7fc6077fe700 10 client.26857 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:10.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.129+0000 7fc6277fe700 1 --2- 172.21.15.44:0/3747883126 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5581fee29860 0x5581fee2bc80 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6b7bebc37bb1523 server_cookie=ff1b3566aa34fc96 in_seq=0 out_seq=0 2023-09-10T00:50:10.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.130+0000 7fc6077fe700 1 -- 172.21.15.44:0/3747883126 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5581fed728a0 con 0x5581fee29860 2023-09-10T00:50:10.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.130+0000 7fc6077fe700 10 client.26857 handle_command_reply: tid=0 2023-09-10T00:50:10.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.130+0000 7fc63abbab80 1 client.26857 shutdown 2023-09-10T00:50:10.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.130+0000 7fc63abbab80 1 -- 172.21.15.44:0/3747883126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0b710 msgr2=0x5581fee157c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:10.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.130+0000 7fc63abbab80 1 --2- 172.21.15.44:0/3747883126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0b710 0x5581fee157c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc618005d10 tx=0x7fc618005750 comp rx=0 tx=0).stop 2023-09-10T00:50:10.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.130+0000 7fc63abbab80 1 -- 172.21.15.44:0/3747883126 shutdown_connections 2023-09-10T00:50:10.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.130+0000 7fc63abbab80 1 --2- 172.21.15.44:0/3747883126 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5581fee29860 0x5581fee2bc80 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.130+0000 7fc63abbab80 1 --2- 172.21.15.44:0/3747883126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 0x5581fee13070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.130+0000 7fc6077fe700 0 client.26857 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:10.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.130+0000 7fc63abbab80 1 --2- 172.21.15.44:0/3747883126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0b710 0x5581fee157c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.130+0000 7fc63abbab80 1 --2- 172.21.15.44:0/3747883126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 0x5581fee12a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.130+0000 7fc63abbab80 1 -- 172.21.15.44:0/3747883126 >> 172.21.15.44:0/3747883126 conn(0x5581fed80090 msgr2=0x5581fee06780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:10.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.130+0000 7fc63abbab80 1 -- 172.21.15.44:0/3747883126 shutdown_connections 2023-09-10T00:50:10.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.130+0000 7fc63abbab80 1 -- 172.21.15.44:0/3747883126 wait complete. 2023-09-10T00:50:10.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.130+0000 7fc63abbab80 20 client.26857 trim_cache size 0 max 16384 2023-09-10T00:50:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc63abbab80 1 Processor -- start 2023-09-10T00:50:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc63abbab80 1 -- start start 2023-09-10T00:50:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc63abbab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0b710 0x5581fee20e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc63abbab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0c030 0x5581fee0dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc63abbab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 0x5581fee0e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc63abbab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fee14020 con 0x5581fee15e40 2023-09-10T00:50:10.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc63abbab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fee0c9e0 con 0x5581fee0b710 2023-09-10T00:50:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc63abbab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fee213a0 con 0x5581fee0c030 2023-09-10T00:50:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc62cf91700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 0x5581fee0e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc6277fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0c030 0x5581fee0dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc627fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0b710 0x5581fee20e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc62cf91700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 0x5581fee0e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52232/0 (socket says 172.21.15.44:52232) 2023-09-10T00:50:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc6277fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0c030 0x5581fee0dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59112/0 (socket says 172.21.15.44:59112) 2023-09-10T00:50:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc627fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0b710 0x5581fee20e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53968/0 (socket says 172.21.15.44:53968) 2023-09-10T00:50:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc62cf91700 1 -- 172.21.15.44:0/938292259 learned_addr learned my addr 172.21.15.44:0/938292259 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc627fff700 1 -- 172.21.15.44:0/938292259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0c030 msgr2=0x5581fee0dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc627fff700 1 --2- 172.21.15.44:0/938292259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0c030 0x5581fee0dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc627fff700 1 -- 172.21.15.44:0/938292259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 msgr2=0x5581fee0e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc627fff700 1 --2- 172.21.15.44:0/938292259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 0x5581fee0e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc627fff700 1 -- 172.21.15.44:0/938292259 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581febb0250 con 0x5581fee0b710 2023-09-10T00:50:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.150+0000 7fc62cf91700 1 --2- 172.21.15.44:0/938292259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 0x5581fee0e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.151+0000 7fc627fff700 1 --2- 172.21.15.44:0/938292259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0b710 0x5581fee20e60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc618009350 tx=0x7fc618009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64dd2d3a56157078 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.151+0000 7fc60effd700 1 -- 172.21.15.44:0/938292259 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61800f780 con 0x5581fee0b710 2023-09-10T00:50:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.151+0000 7fc60effd700 1 -- 172.21.15.44:0/938292259 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc618005320 con 0x5581fee0b710 2023-09-10T00:50:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.151+0000 7fc60effd700 1 -- 172.21.15.44:0/938292259 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618038690 con 0x5581fee0b710 2023-09-10T00:50:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.152+0000 7fc63abbab80 1 -- 172.21.15.44:0/938292259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0b710 msgr2=0x5581fee20e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.152+0000 7fc63abbab80 1 --2- 172.21.15.44:0/938292259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0b710 0x5581fee20e60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc618009350 tx=0x7fc618009fd0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.152+0000 7fc63abbab80 1 -- 172.21.15.44:0/938292259 shutdown_connections 2023-09-10T00:50:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.152+0000 7fc63abbab80 1 --2- 172.21.15.44:0/938292259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee15e40 0x5581fee0e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.152+0000 7fc63abbab80 1 --2- 172.21.15.44:0/938292259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0b710 0x5581fee20e60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.152+0000 7fc63abbab80 1 --2- 172.21.15.44:0/938292259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee0c030 0x5581fee0dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.152+0000 7fc63abbab80 1 -- 172.21.15.44:0/938292259 >> 172.21.15.44:0/938292259 conn(0x5581fed80090 msgr2=0x5581fee26010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.152+0000 7fc63abbab80 1 -- 172.21.15.44:0/938292259 shutdown_connections 2023-09-10T00:50:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.152+0000 7fc63abbab80 1 -- 172.21.15.44:0/938292259 wait complete. 2023-09-10T00:50:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.152+0000 7fc63abbab80 1 -- start start 2023-09-10T00:50:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc63abbab80 1 Processor -- start 2023-09-10T00:50:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc63abbab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee0b710 0x5581fee75950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc63abbab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 0x5581fee75e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.163 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc63abbab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee15e40 0x5581fee766e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.163 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc63abbab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fee76cc0 con 0x5581fee0b710 2023-09-10T00:50:10.163 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc63abbab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fee76e00 con 0x5581fee0c030 2023-09-10T00:50:10.163 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc63abbab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fee76fc0 con 0x5581fee15e40 2023-09-10T00:50:10.163 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc63abbab80 10 client.0 _ll_register_callbacks cb 0x7fc620094aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:50:10.163 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc6277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 0x5581fee75e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc62cf91700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee15e40 0x5581fee766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc6277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 0x5581fee75e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53976/0 (socket says 172.21.15.44:53976) 2023-09-10T00:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc62cf91700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee15e40 0x5581fee766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59120/0 (socket says 172.21.15.44:59120) 2023-09-10T00:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc6277fe700 1 -- 172.21.15.44:0/876140260 learned_addr learned my addr 172.21.15.44:0/876140260 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc627fff700 1 --2- 172.21.15.44:0/876140260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee0b710 0x5581fee75950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc62cf91700 1 -- 172.21.15.44:0/876140260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 msgr2=0x5581fee75e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc62cf91700 1 --2- 172.21.15.44:0/876140260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 0x5581fee75e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc62cf91700 1 -- 172.21.15.44:0/876140260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee0b710 msgr2=0x5581fee75950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc62cf91700 1 --2- 172.21.15.44:0/876140260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee0b710 0x5581fee75950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc62cf91700 1 -- 172.21.15.44:0/876140260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6281a32c0 con 0x5581fee15e40 2023-09-10T00:50:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.153+0000 7fc6277fe700 1 --2- 172.21.15.44:0/876140260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 0x5581fee75e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:10.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.154+0000 7fc62cf91700 1 --2- 172.21.15.44:0/876140260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee15e40 0x5581fee766e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc6200c9690 tx=0x7fc620098850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3451f05c8daf8934 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:10.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.154+0000 7fc6077fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:50:10.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.154+0000 7fc6077fe700 1 -- 172.21.15.44:0/876140260 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc620099580 con 0x5581fee15e40 2023-09-10T00:50:10.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.154+0000 7fc6077fe700 1 -- 172.21.15.44:0/876140260 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fc620098e80 con 0x5581fee15e40 2023-09-10T00:50:10.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.154+0000 7fc63abbab80 1 -- 172.21.15.44:0/876140260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581fed56920 con 0x5581fee15e40 2023-09-10T00:50:10.166 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.154+0000 7fc6077fe700 1 -- 172.21.15.44:0/876140260 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc62009ccb0 con 0x5581fee15e40 2023-09-10T00:50:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.154+0000 7fc63abbab80 10 client.26863 fetch_fsmap learned FSMap version 59 2023-09-10T00:50:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.154+0000 7fc63abbab80 1 -- 172.21.15.44:0/876140260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x5581feb72c60 con 0x5581fee15e40 2023-09-10T00:50:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.154+0000 7fc6077fe700 1 -- 172.21.15.44:0/876140260 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc62007b080 con 0x5581fee15e40 2023-09-10T00:50:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.154+0000 7fc6077fe700 1 -- 172.21.15.44:0/876140260 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 59) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fc6200a09d0 con 0x5581fee15e40 2023-09-10T00:50:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.154+0000 7fc63abbab80 10 client.26863 fetch_fsmap finished waiting for FSMap version 59 2023-09-10T00:50:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.154+0000 7fc63abbab80 10 client.26863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:50:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.154+0000 7fc63abbab80 20 client.26863 populate_metadata read hostname 'smithi044' 2023-09-10T00:50:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.154+0000 7fc63abbab80 1 --2- 172.21.15.44:0/876140260 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5581fee7e940 0x5581fee80d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.154+0000 7fc63abbab80 4 client.26863 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:50:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.154+0000 7fc63abbab80 1 -- 172.21.15.44:0/876140260 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5581fed170b0 con 0x5581fee7e940 2023-09-10T00:50:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.155+0000 7fc627fff700 1 --2- 172.21.15.44:0/876140260 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5581fee7e940 0x5581fee80d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:10.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.155+0000 7fc6077fe700 10 client.26863 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:10.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.155+0000 7fc627fff700 1 --2- 172.21.15.44:0/876140260 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5581fee7e940 0x5581fee80d60 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a198796fbef9c9 server_cookie=a7472869c44263c5 in_seq=0 out_seq=0 2023-09-10T00:50:10.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.155+0000 7fc6077fe700 1 -- 172.21.15.44:0/876140260 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5581fed170b0 con 0x5581fee7e940 2023-09-10T00:50:10.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.155+0000 7fc6077fe700 10 client.26863 handle_command_reply: tid=0 2023-09-10T00:50:10.169 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.155+0000 7fc63abbab80 1 client.26863 shutdown 2023-09-10T00:50:10.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.155+0000 7fc63abbab80 1 -- 172.21.15.44:0/876140260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee15e40 msgr2=0x5581fee766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:10.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.155+0000 7fc63abbab80 1 --2- 172.21.15.44:0/876140260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee15e40 0x5581fee766e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc6200c9690 tx=0x7fc620098850 comp rx=0 tx=0).stop 2023-09-10T00:50:10.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.155+0000 7fc63abbab80 1 -- 172.21.15.44:0/876140260 shutdown_connections 2023-09-10T00:50:10.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.155+0000 7fc63abbab80 1 --2- 172.21.15.44:0/876140260 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5581fee7e940 0x5581fee80d60 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.170 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.155+0000 7fc63abbab80 1 --2- 172.21.15.44:0/876140260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5581fee15e40 0x5581fee766e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.155+0000 7fc63abbab80 1 --2- 172.21.15.44:0/876140260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581fee0b710 0x5581fee75950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.155+0000 7fc6077fe700 0 client.26863 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.155+0000 7fc63abbab80 1 --2- 172.21.15.44:0/876140260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581fee0c030 0x5581fee75e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.155+0000 7fc63abbab80 1 -- 172.21.15.44:0/876140260 >> 172.21.15.44:0/876140260 conn(0x5581fed80090 msgr2=0x5581fee0a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.155+0000 7fc63abbab80 1 -- 172.21.15.44:0/876140260 shutdown_connections 2023-09-10T00:50:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.156+0000 7fc63abbab80 1 -- 172.21.15.44:0/876140260 wait complete. 2023-09-10T00:50:10.172 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.156+0000 7fc63abbab80 20 client.26863 trim_cache size 0 max 16384 2023-09-10T00:50:10.172 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:50:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:50:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:50:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:50:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:50:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:50:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:50:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:50:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:50:10.174 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:50:10.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.156+0000 7fc62e9f4700 1 -- 172.21.15.44:0/2330731199 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc6100a0830 msgr2=0x7fc6100a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:10.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.156+0000 7fc62e9f4700 1 --2- 172.21.15.44:0/2330731199 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc6100a0830 0x7fc6100a2cf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc61800b580 tx=0x7fc61801a040 comp rx=0 tx=0).stop 2023-09-10T00:50:10.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.156+0000 7fc62e9f4700 1 -- 172.21.15.44:0/2330731199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc628105570 msgr2=0x7fc6281a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:10.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.156+0000 7fc62e9f4700 1 --2- 172.21.15.44:0/2330731199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc628105570 0x7fc6281a2410 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc62000ef30 tx=0x7fc62000c3a0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.156+0000 7fc62e9f4700 1 -- 172.21.15.44:0/2330731199 shutdown_connections 2023-09-10T00:50:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.156+0000 7fc62e9f4700 1 --2- 172.21.15.44:0/2330731199 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc6100a0830 0x7fc6100a2cf0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.156+0000 7fc62e9f4700 1 --2- 172.21.15.44:0/2330731199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc628105570 0x7fc6281a2410 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.156+0000 7fc62e9f4700 1 --2- 172.21.15.44:0/2330731199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6280ff4b0 0x7fc628197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.156+0000 7fc62e9f4700 1 --2- 172.21.15.44:0/2330731199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6280ffe70 0x7fc6281978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.156+0000 7fc62e9f4700 1 -- 172.21.15.44:0/2330731199 >> 172.21.15.44:0/2330731199 conn(0x7fc6280fafa0 msgr2=0x7fc6281031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:10.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.156+0000 7fc62e9f4700 1 -- 172.21.15.44:0/2330731199 shutdown_connections 2023-09-10T00:50:10.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:10.156+0000 7fc62e9f4700 1 -- 172.21.15.44:0/2330731199 wait complete. 2023-09-10T00:50:10.177 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:50:10.923 INFO:teuthology.orchestra.run.smithi044.stderr:35800000 of 50000000 tuples (71%) done (elapsed 550.56 s, remaining 218.38 s) 2023-09-10T00:50:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:10 smithi131 ceph-mon[103713]: pgmap v295: 129 pgs: 129 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 17 MiB/s wr, 104 op/s 2023-09-10T00:50:11.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:10 smithi131 ceph-mon[103713]: osdmap e100: 12 total, 12 up, 12 in 2023-09-10T00:50:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:10 smithi044 ceph-mon[98676]: pgmap v295: 129 pgs: 129 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 17 MiB/s wr, 104 op/s 2023-09-10T00:50:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:10 smithi044 ceph-mon[98676]: osdmap e100: 12 total, 12 up, 12 in 2023-09-10T00:50:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:10 smithi099 ceph-mon[102692]: pgmap v295: 129 pgs: 129 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 17 MiB/s wr, 104 op/s 2023-09-10T00:50:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:10 smithi099 ceph-mon[102692]: osdmap e100: 12 total, 12 up, 12 in 2023-09-10T00:50:11.879 INFO:teuthology.orchestra.run.smithi044.stderr:35900000 of 50000000 tuples (71%) done (elapsed 551.51 s, remaining 216.61 s) 2023-09-10T00:50:12.829 INFO:teuthology.orchestra.run.smithi044.stderr:36000000 of 50000000 tuples (72%) done (elapsed 552.46 s, remaining 214.85 s) 2023-09-10T00:50:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:12 smithi131 ceph-mon[103713]: pgmap v297: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 36 MiB/s wr, 170 op/s 2023-09-10T00:50:13.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:12 smithi044 ceph-mon[98676]: pgmap v297: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 36 MiB/s wr, 170 op/s 2023-09-10T00:50:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:12 smithi099 ceph-mon[102692]: pgmap v297: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 36 MiB/s wr, 170 op/s 2023-09-10T00:50:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:36100000 of 50000000 tuples (72%) done (elapsed 553.48 s, remaining 213.11 s) 2023-09-10T00:50:14.893 INFO:teuthology.orchestra.run.smithi044.stderr:36200000 of 50000000 tuples (72%) done (elapsed 554.53 s, remaining 211.39 s) 2023-09-10T00:50:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:14 smithi131 ceph-mon[103713]: pgmap v298: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 30 MiB/s wr, 143 op/s 2023-09-10T00:50:15.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:14 smithi044 ceph-mon[98676]: pgmap v298: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 30 MiB/s wr, 143 op/s 2023-09-10T00:50:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:14 smithi099 ceph-mon[102692]: pgmap v298: 129 pgs: 129 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 30 MiB/s wr, 143 op/s 2023-09-10T00:50:15.717 INFO:teuthology.orchestra.run.smithi044.stderr:36300000 of 50000000 tuples (72%) done (elapsed 555.35 s, remaining 209.60 s) 2023-09-10T00:50:16.672 INFO:teuthology.orchestra.run.smithi044.stderr:36400000 of 50000000 tuples (72%) done (elapsed 556.31 s, remaining 207.85 s) 2023-09-10T00:50:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:16 smithi131 ceph-mon[103713]: pgmap v299: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 MiB/s wr, 123 op/s 2023-09-10T00:50:17.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:16 smithi044 ceph-mon[98676]: pgmap v299: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 MiB/s wr, 123 op/s 2023-09-10T00:50:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:16 smithi099 ceph-mon[102692]: pgmap v299: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 MiB/s wr, 123 op/s 2023-09-10T00:50:17.527 INFO:teuthology.orchestra.run.smithi044.stderr:36500000 of 50000000 tuples (73%) done (elapsed 557.16 s, remaining 206.07 s) 2023-09-10T00:50:18.398 INFO:teuthology.orchestra.run.smithi044.stderr:36600000 of 50000000 tuples (73%) done (elapsed 558.03 s, remaining 204.31 s) 2023-09-10T00:50:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:18 smithi131 ceph-mon[103713]: pgmap v300: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 MiB/s wr, 123 op/s 2023-09-10T00:50:19.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:18 smithi044 ceph-mon[98676]: pgmap v300: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 MiB/s wr, 123 op/s 2023-09-10T00:50:19.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:18 smithi099 ceph-mon[102692]: pgmap v300: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 MiB/s wr, 123 op/s 2023-09-10T00:50:19.254 INFO:teuthology.orchestra.run.smithi044.stderr:36700000 of 50000000 tuples (73%) done (elapsed 558.89 s, remaining 202.54 s) 2023-09-10T00:50:20.175 INFO:teuthology.orchestra.run.smithi044.stderr:36800000 of 50000000 tuples (73%) done (elapsed 559.81 s, remaining 200.80 s) 2023-09-10T00:50:21.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:20 smithi131 ceph-mon[103713]: pgmap v301: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 MiB/s wr, 123 op/s 2023-09-10T00:50:21.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:20 smithi044 ceph-mon[98676]: pgmap v301: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 MiB/s wr, 123 op/s 2023-09-10T00:50:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:20 smithi099 ceph-mon[102692]: pgmap v301: 129 pgs: 129 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 MiB/s wr, 123 op/s 2023-09-10T00:50:21.208 INFO:teuthology.orchestra.run.smithi044.stderr:36900000 of 50000000 tuples (73%) done (elapsed 560.84 s, remaining 199.11 s) 2023-09-10T00:50:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.016+0000 7f8fe3be6700 1 -- 172.21.15.44:0/1357657585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fdc100120 msgr2=0x7f8fdc10c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.016+0000 7f8fe3be6700 1 --2- 172.21.15.44:0/1357657585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fdc100120 0x7f8fdc10c330 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009fc0 tx=0x7f8fd400b330 comp rx=0 tx=0).stop 2023-09-10T00:50:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.016+0000 7f8fe3be6700 1 -- 172.21.15.44:0/1357657585 shutdown_connections 2023-09-10T00:50:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.017+0000 7f8fe3be6700 1 --2- 172.21.15.44:0/1357657585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fdc100120 0x7f8fdc10c330 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.017+0000 7f8fe3be6700 1 --2- 172.21.15.44:0/1357657585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fdc1055b0 0x7f8fdc105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.017+0000 7f8fe3be6700 1 --2- 172.21.15.44:0/1357657585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fdc0ff760 0x7f8fdc0ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.017+0000 7f8fe3be6700 1 -- 172.21.15.44:0/1357657585 >> 172.21.15.44:0/1357657585 conn(0x7f8fdc074f30 msgr2=0x7f8fdc075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.017+0000 7f8fe3be6700 1 -- 172.21.15.44:0/1357657585 shutdown_connections 2023-09-10T00:50:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.017+0000 7f8fe3be6700 1 -- 172.21.15.44:0/1357657585 wait complete. 2023-09-10T00:50:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.017+0000 7f8fe3be6700 1 Processor -- start 2023-09-10T00:50:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.017+0000 7f8fe3be6700 1 -- start start 2023-09-10T00:50:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe3be6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fdc0ff760 0x7f8fdc195460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe3be6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fdc100120 0x7f8fdc1959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe3be6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fdc1055b0 0x7f8fdc1a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe3be6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc1960f0 con 0x7f8fdc0ff760 2023-09-10T00:50:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe3be6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc1a4e40 con 0x7f8fdc1055b0 2023-09-10T00:50:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe3be6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc1a4fd0 con 0x7f8fdc100120 2023-09-10T00:50:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe1982700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fdc0ff760 0x7f8fdc195460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe1982700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fdc0ff760 0x7f8fdc195460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52248/0 (socket says 172.21.15.44:52248) 2023-09-10T00:50:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe1982700 1 -- 172.21.15.44:0/2504887419 learned_addr learned my addr 172.21.15.44:0/2504887419 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe1181700 1 --2- 172.21.15.44:0/2504887419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fdc100120 0x7f8fdc1959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe2183700 1 --2- 172.21.15.44:0/2504887419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fdc1055b0 0x7f8fdc1a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe1982700 1 -- 172.21.15.44:0/2504887419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fdc100120 msgr2=0x7f8fdc1959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe1982700 1 --2- 172.21.15.44:0/2504887419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fdc100120 0x7f8fdc1959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe1982700 1 -- 172.21.15.44:0/2504887419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fdc1055b0 msgr2=0x7f8fdc1a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe1982700 1 --2- 172.21.15.44:0/2504887419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fdc1055b0 0x7f8fdc1a4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe1982700 1 -- 172.21.15.44:0/2504887419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd4009c40 con 0x7f8fdc0ff760 2023-09-10T00:50:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fe1982700 1 --2- 172.21.15.44:0/2504887419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fdc0ff760 0x7f8fdc195460 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800e9c0 tx=0x7f8fd800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.018+0000 7f8fd2ffd700 1 -- 172.21.15.44:0/2504887419 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8004a30 con 0x7f8fdc0ff760 2023-09-10T00:50:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.019+0000 7f8fd2ffd700 1 -- 172.21.15.44:0/2504887419 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fd800cc50 con 0x7f8fdc0ff760 2023-09-10T00:50:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.019+0000 7f8fe3be6700 1 -- 172.21.15.44:0/2504887419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fdc1a52b0 con 0x7f8fdc0ff760 2023-09-10T00:50:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.019+0000 7f8fd2ffd700 1 -- 172.21.15.44:0/2504887419 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8005410 con 0x7f8fdc0ff760 2023-09-10T00:50:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.019+0000 7f8fe3be6700 1 -- 172.21.15.44:0/2504887419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fdc1a57c0 con 0x7f8fdc0ff760 2023-09-10T00:50:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.022+0000 7f8fd2ffd700 1 -- 172.21.15.44:0/2504887419 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f8fd8005570 con 0x7f8fdc0ff760 2023-09-10T00:50:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.022+0000 7f8fd2ffd700 1 --2- 172.21.15.44:0/2504887419 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fc80a07b0 0x7f8fc80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.022+0000 7f8fd2ffd700 1 -- 172.21.15.44:0/2504887419 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8fd8014070 con 0x7f8fdc0ff760 2023-09-10T00:50:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.022+0000 7f8fe1181700 1 --2- 172.21.15.44:0/2504887419 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fc80a07b0 0x7f8fc80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.023+0000 7f8fe1181700 1 --2- 172.21.15.44:0/2504887419 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fc80a07b0 0x7f8fc80a2c70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc068560 tx=0x7f8fcc009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.024+0000 7f8fefdacb80 1 Processor -- start 2023-09-10T00:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.025+0000 7f8fefdacb80 1 -- start start 2023-09-10T00:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.025+0000 7f8fefdacb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb8941a0 0x5613eb894580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.025+0000 7f8fefdacb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb88e300 0x5613eb88e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.025+0000 7f8fefdacb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88ecc0 0x5613eb89af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.025+0000 7f8fefdacb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613eb7e76d0 con 0x5613eb88e300 2023-09-10T00:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.025+0000 7f8fefdacb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613eb7e4ff0 con 0x5613eb88ecc0 2023-09-10T00:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.025+0000 7f8fefdacb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613eb7e08e0 con 0x5613eb8941a0 2023-09-10T00:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.025+0000 7f8fe1181700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb88e300 0x5613eb88e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.025+0000 7f8fe1181700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb88e300 0x5613eb88e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52258/0 (socket says 172.21.15.44:52258) 2023-09-10T00:50:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.025+0000 7f8fe1181700 1 -- 172.21.15.44:0/264431123 learned_addr learned my addr 172.21.15.44:0/264431123 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.026+0000 7f8fe1181700 1 -- 172.21.15.44:0/264431123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb8941a0 msgr2=0x5613eb894580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.026+0000 7f8fe1181700 1 --2- 172.21.15.44:0/264431123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb8941a0 0x5613eb894580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.026+0000 7f8fe1181700 1 -- 172.21.15.44:0/264431123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88ecc0 msgr2=0x5613eb89af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.026+0000 7f8fe1181700 1 --2- 172.21.15.44:0/264431123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88ecc0 0x5613eb89af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.026+0000 7f8fe1181700 1 -- 172.21.15.44:0/264431123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613eb60cd80 con 0x5613eb88e300 2023-09-10T00:50:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.026+0000 7f8fe1181700 1 --2- 172.21.15.44:0/264431123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb88e300 0x5613eb88e780 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc011d20 tx=0x7f8fcc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196454a5ae528ab server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.026+0000 7f8fc7fff700 1 -- 172.21.15.44:0/264431123 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc00ebd0 con 0x5613eb88e300 2023-09-10T00:50:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.026+0000 7f8fc7fff700 1 -- 172.21.15.44:0/264431123 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fcc032070 con 0x5613eb88e300 2023-09-10T00:50:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.026+0000 7f8fc7fff700 1 -- 172.21.15.44:0/264431123 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc01c670 con 0x5613eb88e300 2023-09-10T00:50:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.027+0000 7f8fefdacb80 1 -- 172.21.15.44:0/264431123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb88e300 msgr2=0x5613eb88e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.027+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/264431123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb88e300 0x5613eb88e780 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc011d20 tx=0x7f8fcc014000 comp rx=0 tx=0).stop 2023-09-10T00:50:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.027+0000 7f8fefdacb80 1 -- 172.21.15.44:0/264431123 shutdown_connections 2023-09-10T00:50:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.027+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/264431123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88ecc0 0x5613eb89af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.027+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/264431123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb8941a0 0x5613eb894580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.027+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/264431123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb88e300 0x5613eb88e780 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.027+0000 7f8fefdacb80 1 -- 172.21.15.44:0/264431123 >> 172.21.15.44:0/264431123 conn(0x5613eb803030 msgr2=0x5613eb889560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.027+0000 7f8fefdacb80 1 -- 172.21.15.44:0/264431123 shutdown_connections 2023-09-10T00:50:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.027+0000 7f8fefdacb80 1 -- 172.21.15.44:0/264431123 wait complete. 2023-09-10T00:50:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.027+0000 7f8fefdacb80 1 -- start start 2023-09-10T00:50:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.028+0000 7f8fefdacb80 1 Processor -- start 2023-09-10T00:50:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.028+0000 7f8fefdacb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88e300 0x5613eb89ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.028+0000 7f8fefdacb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb88ecc0 0x5613eb898130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.028+0000 7f8fefdacb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb8941a0 0x5613eb8987b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.028+0000 7f8fefdacb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613eb77eb10 con 0x5613eb88ecc0 2023-09-10T00:50:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.028+0000 7f8fefdacb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613eb77e810 con 0x5613eb8941a0 2023-09-10T00:50:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.028+0000 7f8fefdacb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613eb6628c0 con 0x5613eb88e300 2023-09-10T00:50:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.028+0000 7f8fefdacb80 10 client.0 _ll_register_callbacks cb 0x5613eb6c8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:50:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.028+0000 7f8fe1181700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb88ecc0 0x5613eb898130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.028+0000 7f8fe2183700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb8941a0 0x5613eb8987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.028+0000 7f8fe1982700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88e300 0x5613eb89ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.028+0000 7f8fe1181700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb88ecc0 0x5613eb898130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52264/0 (socket says 172.21.15.44:52264) 2023-09-10T00:50:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.028+0000 7f8fe2183700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb8941a0 0x5613eb8987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54006/0 (socket says 172.21.15.44:54006) 2023-09-10T00:50:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.028+0000 7f8fe1181700 1 -- 172.21.15.44:0/1248012104 learned_addr learned my addr 172.21.15.44:0/1248012104 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.029+0000 7f8fe2183700 1 -- 172.21.15.44:0/1248012104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88e300 msgr2=0x5613eb89ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.029+0000 7f8fe2183700 1 --2- 172.21.15.44:0/1248012104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88e300 0x5613eb89ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.029+0000 7f8fe2183700 1 -- 172.21.15.44:0/1248012104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb88ecc0 msgr2=0x5613eb898130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.029+0000 7f8fe2183700 1 --2- 172.21.15.44:0/1248012104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb88ecc0 0x5613eb898130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.029+0000 7f8fe2183700 1 -- 172.21.15.44:0/1248012104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc017070 con 0x5613eb8941a0 2023-09-10T00:50:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.029+0000 7f8fe1982700 1 --2- 172.21.15.44:0/1248012104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88e300 0x5613eb89ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.029+0000 7f8fe1181700 1 --2- 172.21.15.44:0/1248012104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb88ecc0 0x5613eb898130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.029+0000 7f8fe2183700 1 --2- 172.21.15.44:0/1248012104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb8941a0 0x5613eb8987b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc068ad0 tx=0x7f8fd4009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c1c51ebde1dbd2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.029+0000 7f8fc57fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.029+0000 7f8fc57fa700 1 -- 172.21.15.44:0/1248012104 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd400e040 con 0x5613eb8941a0 2023-09-10T00:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.029+0000 7f8fc57fa700 1 -- 172.21.15.44:0/1248012104 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8fd4003a00 con 0x5613eb8941a0 2023-09-10T00:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.029+0000 7f8fefdacb80 1 -- 172.21.15.44:0/1248012104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613eb738490 con 0x5613eb8941a0 2023-09-10T00:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.029+0000 7f8fc57fa700 1 -- 172.21.15.44:0/1248012104 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fd4003bc0 con 0x5613eb8941a0 2023-09-10T00:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.029+0000 7f8fefdacb80 10 client.26930 fetch_fsmap learned FSMap version 59 2023-09-10T00:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.029+0000 7f8fefdacb80 1 -- 172.21.15.44:0/1248012104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x5613eb5f5c00 con 0x5613eb8941a0 2023-09-10T00:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.030+0000 7f8fc57fa700 1 -- 172.21.15.44:0/1248012104 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5613eb738490 con 0x5613eb8941a0 2023-09-10T00:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.030+0000 7f8fc57fa700 1 -- 172.21.15.44:0/1248012104 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 59) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f8fd400b050 con 0x5613eb8941a0 2023-09-10T00:50:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.030+0000 7f8fefdacb80 10 client.26930 fetch_fsmap finished waiting for FSMap version 59 2023-09-10T00:50:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.030+0000 7f8fefdacb80 10 client.26930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:50:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.030+0000 7f8fefdacb80 20 client.26930 populate_metadata read hostname 'smithi044' 2023-09-10T00:50:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.030+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/1248012104 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5613eb8a8480 0x5613eb8aa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.030+0000 7f8fefdacb80 4 client.26930 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:50:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.030+0000 7f8fefdacb80 1 -- 172.21.15.44:0/1248012104 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613eb7f5840 con 0x5613eb8a8480 2023-09-10T00:50:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.030+0000 7f8fe1982700 1 --2- 172.21.15.44:0/1248012104 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5613eb8a8480 0x5613eb8aa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.030+0000 7f8fe1982700 1 --2- 172.21.15.44:0/1248012104 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5613eb8a8480 0x5613eb8aa8a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30369573de424449 server_cookie=4f26bec134d13e93 in_seq=0 out_seq=0 2023-09-10T00:50:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.030+0000 7f8fc57fa700 10 client.26930 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.031+0000 7f8fc57fa700 1 -- 172.21.15.44:0/1248012104 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5613eb7f5840 con 0x5613eb8a8480 2023-09-10T00:50:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.031+0000 7f8fc57fa700 10 client.26930 handle_command_reply: tid=0 2023-09-10T00:50:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.031+0000 7f8fefdacb80 1 client.26930 shutdown 2023-09-10T00:50:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.031+0000 7f8fefdacb80 1 -- 172.21.15.44:0/1248012104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb8941a0 msgr2=0x5613eb8987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.031+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/1248012104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb8941a0 0x5613eb8987b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc068ad0 tx=0x7f8fd4009510 comp rx=0 tx=0).stop 2023-09-10T00:50:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.031+0000 7f8fefdacb80 1 -- 172.21.15.44:0/1248012104 shutdown_connections 2023-09-10T00:50:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.031+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/1248012104 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5613eb8a8480 0x5613eb8aa8a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.031+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/1248012104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb8941a0 0x5613eb8987b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.031+0000 7f8fc57fa700 0 client.26930 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.031+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/1248012104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88e300 0x5613eb89ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.031+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/1248012104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb88ecc0 0x5613eb898130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.031+0000 7f8fefdacb80 1 -- 172.21.15.44:0/1248012104 >> 172.21.15.44:0/1248012104 conn(0x5613eb803030 msgr2=0x5613eb8887a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.031+0000 7f8fefdacb80 1 -- 172.21.15.44:0/1248012104 shutdown_connections 2023-09-10T00:50:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.032+0000 7f8fefdacb80 1 -- 172.21.15.44:0/1248012104 wait complete. 2023-09-10T00:50:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.032+0000 7f8fefdacb80 20 client.26930 trim_cache size 0 max 16384 2023-09-10T00:50:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.050+0000 7f8fefdacb80 1 Processor -- start 2023-09-10T00:50:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.050+0000 7f8fefdacb80 1 -- start start 2023-09-10T00:50:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.050+0000 7f8fefdacb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88e300 0x5613eb89f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.050+0000 7f8fefdacb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88ecc0 0x5613eb888f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.050+0000 7f8fefdacb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb8941a0 0x5613eb889460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.050+0000 7f8fefdacb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613eb898b70 con 0x5613eb8941a0 2023-09-10T00:50:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.050+0000 7f8fefdacb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613eb89f9b0 con 0x5613eb88e300 2023-09-10T00:50:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.050+0000 7f8fefdacb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613eb893db0 con 0x5613eb88ecc0 2023-09-10T00:50:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.050+0000 7f8fe1982700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88e300 0x5613eb89f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.050+0000 7f8fe2183700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb8941a0 0x5613eb889460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.050+0000 7f8fe1982700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88e300 0x5613eb89f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54010/0 (socket says 172.21.15.44:54010) 2023-09-10T00:50:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.050+0000 7f8fe1982700 1 -- 172.21.15.44:0/264896678 learned_addr learned my addr 172.21.15.44:0/264896678 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.050+0000 7f8fe1181700 1 --2- 172.21.15.44:0/264896678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88ecc0 0x5613eb888f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.050+0000 7f8fe1982700 1 -- 172.21.15.44:0/264896678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88ecc0 msgr2=0x5613eb888f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.050+0000 7f8fe1982700 1 --2- 172.21.15.44:0/264896678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88ecc0 0x5613eb888f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.050+0000 7f8fe1982700 1 -- 172.21.15.44:0/264896678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb8941a0 msgr2=0x5613eb889460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.051+0000 7f8fe1982700 1 --2- 172.21.15.44:0/264896678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb8941a0 0x5613eb889460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.051+0000 7f8fe1982700 1 -- 172.21.15.44:0/264896678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613eb6331f0 con 0x5613eb88e300 2023-09-10T00:50:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.051+0000 7f8fe1982700 1 --2- 172.21.15.44:0/264896678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88e300 0x5613eb89f470 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8005950 tx=0x7f8fd8009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d36eb0f4c29a312 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.051+0000 7f8fd0d89700 1 -- 172.21.15.44:0/264896678 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd809d150 con 0x5613eb88e300 2023-09-10T00:50:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.051+0000 7f8fd0d89700 1 -- 172.21.15.44:0/264896678 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fd8099c50 con 0x5613eb88e300 2023-09-10T00:50:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.051+0000 7f8fd0d89700 1 -- 172.21.15.44:0/264896678 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8016ea0 con 0x5613eb88e300 2023-09-10T00:50:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.052+0000 7f8fefdacb80 1 -- 172.21.15.44:0/264896678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88e300 msgr2=0x5613eb89f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.052+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/264896678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88e300 0x5613eb89f470 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8005950 tx=0x7f8fd8009550 comp rx=0 tx=0).stop 2023-09-10T00:50:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.052+0000 7f8fefdacb80 1 -- 172.21.15.44:0/264896678 shutdown_connections 2023-09-10T00:50:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.052+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/264896678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb8941a0 0x5613eb889460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.052+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/264896678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88e300 0x5613eb89f470 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.052+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/264896678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88ecc0 0x5613eb888f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.052+0000 7f8fefdacb80 1 -- 172.21.15.44:0/264896678 >> 172.21.15.44:0/264896678 conn(0x5613eb803030 msgr2=0x5613eb8922f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.052+0000 7f8fefdacb80 1 -- 172.21.15.44:0/264896678 shutdown_connections 2023-09-10T00:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.052+0000 7f8fefdacb80 1 -- 172.21.15.44:0/264896678 wait complete. 2023-09-10T00:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.052+0000 7f8fefdacb80 1 -- start start 2023-09-10T00:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.052+0000 7f8fefdacb80 1 Processor -- start 2023-09-10T00:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fefdacb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88e300 0x5613eb8f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fefdacb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88ecc0 0x5613eb8f0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fefdacb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb8941a0 0x5613eb8fb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fefdacb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613eb8fb7c0 con 0x5613eb8941a0 2023-09-10T00:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fefdacb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613eb8fb900 con 0x5613eb88e300 2023-09-10T00:50:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fefdacb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613eb8fba70 con 0x5613eb88ecc0 2023-09-10T00:50:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fefdacb80 10 client.0 _ll_register_callbacks cb 0x7f8fd4009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:50:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fe1181700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88ecc0 0x5613eb8f0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fe1982700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88e300 0x5613eb8f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fe2183700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb8941a0 0x5613eb8fb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fe1982700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88e300 0x5613eb8f03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54016/0 (socket says 172.21.15.44:54016) 2023-09-10T00:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fe1181700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88ecc0 0x5613eb8f0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59160/0 (socket says 172.21.15.44:59160) 2023-09-10T00:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fe1982700 1 -- 172.21.15.44:0/2089455741 learned_addr learned my addr 172.21.15.44:0/2089455741 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fe1982700 1 -- 172.21.15.44:0/2089455741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88ecc0 msgr2=0x5613eb8f0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fe1982700 1 --2- 172.21.15.44:0/2089455741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88ecc0 0x5613eb8f0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fe1982700 1 -- 172.21.15.44:0/2089455741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb8941a0 msgr2=0x5613eb8fb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fe1982700 1 --2- 172.21.15.44:0/2089455741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb8941a0 0x5613eb8fb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fe1982700 1 -- 172.21.15.44:0/2089455741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd8095040 con 0x5613eb88e300 2023-09-10T00:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fe2183700 1 --2- 172.21.15.44:0/2089455741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb8941a0 0x5613eb8fb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fe1982700 1 --2- 172.21.15.44:0/2089455741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88e300 0x5613eb8f03f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80c98a0 tx=0x7f8fd800cf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74ddfc8060627ec4 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.053+0000 7f8fc57fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:50:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.054+0000 7f8fc57fa700 1 -- 172.21.15.44:0/2089455741 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd809d150 con 0x5613eb88e300 2023-09-10T00:50:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.054+0000 7f8fc57fa700 1 -- 172.21.15.44:0/2089455741 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8fd8099550 con 0x5613eb88e300 2023-09-10T00:50:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.054+0000 7f8fefdacb80 1 -- 172.21.15.44:0/2089455741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fcc032070 con 0x5613eb88e300 2023-09-10T00:50:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.054+0000 7f8fc57fa700 1 -- 172.21.15.44:0/2089455741 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fd8065250 con 0x5613eb88e300 2023-09-10T00:50:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.054+0000 7f8fefdacb80 10 client.26936 fetch_fsmap learned FSMap version 59 2023-09-10T00:50:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.054+0000 7f8fefdacb80 1 -- 172.21.15.44:0/2089455741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x5613eb60b250 con 0x5613eb88e300 2023-09-10T00:50:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.054+0000 7f8fc57fa700 1 -- 172.21.15.44:0/2089455741 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fd807e080 con 0x5613eb88e300 2023-09-10T00:50:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.054+0000 7f8fc57fa700 1 -- 172.21.15.44:0/2089455741 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 59) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f8fd80cc0b0 con 0x5613eb88e300 2023-09-10T00:50:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.054+0000 7f8fefdacb80 10 client.26936 fetch_fsmap finished waiting for FSMap version 59 2023-09-10T00:50:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.054+0000 7f8fefdacb80 10 client.26936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:50:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.054+0000 7f8fefdacb80 20 client.26936 populate_metadata read hostname 'smithi044' 2023-09-10T00:50:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.054+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/2089455741 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5613eb8fcd70 0x5613eb8ff160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.054+0000 7f8fefdacb80 4 client.26936 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:50:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.054+0000 7f8fefdacb80 1 -- 172.21.15.44:0/2089455741 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5613eb7d98c0 con 0x5613eb8fcd70 2023-09-10T00:50:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.054+0000 7f8fe1181700 1 --2- 172.21.15.44:0/2089455741 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5613eb8fcd70 0x5613eb8ff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.055+0000 7f8fe1181700 1 --2- 172.21.15.44:0/2089455741 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5613eb8fcd70 0x5613eb8ff160 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=850b7dc556b07df5 server_cookie=d6095f4a2a7c1713 in_seq=0 out_seq=0 2023-09-10T00:50:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.055+0000 7f8fc57fa700 10 client.26936 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.070+0000 7f8fc57fa700 1 -- 172.21.15.44:0/2089455741 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5613eb7d98c0 con 0x5613eb8fcd70 2023-09-10T00:50:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.070+0000 7f8fc57fa700 10 client.26936 handle_command_reply: tid=0 2023-09-10T00:50:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.070+0000 7f8fefdacb80 1 client.26936 shutdown 2023-09-10T00:50:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.070+0000 7f8fefdacb80 1 -- 172.21.15.44:0/2089455741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88e300 msgr2=0x5613eb8f03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.070+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/2089455741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88e300 0x5613eb8f03f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80c98a0 tx=0x7f8fd800cf00 comp rx=0 tx=0).stop 2023-09-10T00:50:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.070+0000 7f8fefdacb80 1 -- 172.21.15.44:0/2089455741 shutdown_connections 2023-09-10T00:50:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.070+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/2089455741 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5613eb8fcd70 0x5613eb8ff160 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.070+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/2089455741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613eb8941a0 0x5613eb8fb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.070+0000 7f8fc57fa700 0 client.26936 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.070+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/2089455741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613eb88e300 0x5613eb8f03f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.070+0000 7f8fefdacb80 1 --2- 172.21.15.44:0/2089455741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613eb88ecc0 0x5613eb8f0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.070+0000 7f8fefdacb80 1 -- 172.21.15.44:0/2089455741 >> 172.21.15.44:0/2089455741 conn(0x5613eb803030 msgr2=0x5613eb88d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.070+0000 7f8fefdacb80 1 -- 172.21.15.44:0/2089455741 shutdown_connections 2023-09-10T00:50:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.070+0000 7f8fefdacb80 1 -- 172.21.15.44:0/2089455741 wait complete. 2023-09-10T00:50:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.070+0000 7f8fefdacb80 20 client.26936 trim_cache size 0 max 16384 2023-09-10T00:50:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:50:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:50:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:50:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:50:22.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.071+0000 7f8fe3be6700 1 -- 172.21.15.44:0/2504887419 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fc80a07b0 msgr2=0x7f8fc80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:22.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.071+0000 7f8fe3be6700 1 --2- 172.21.15.44:0/2504887419 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fc80a07b0 0x7f8fc80a2c70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc068560 tx=0x7f8fcc009260 comp rx=0 tx=0).stop 2023-09-10T00:50:22.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.071+0000 7f8fe3be6700 1 -- 172.21.15.44:0/2504887419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fdc0ff760 msgr2=0x7f8fdc195460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:22.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.071+0000 7f8fe3be6700 1 --2- 172.21.15.44:0/2504887419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fdc0ff760 0x7f8fdc195460 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800e9c0 tx=0x7f8fd800ecd0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.071+0000 7f8fe3be6700 1 -- 172.21.15.44:0/2504887419 shutdown_connections 2023-09-10T00:50:22.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.071+0000 7f8fe3be6700 1 --2- 172.21.15.44:0/2504887419 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fc80a07b0 0x7f8fc80a2c70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.071+0000 7f8fe3be6700 1 --2- 172.21.15.44:0/2504887419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fdc1055b0 0x7f8fdc1a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.071+0000 7f8fe3be6700 1 --2- 172.21.15.44:0/2504887419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fdc0ff760 0x7f8fdc195460 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.071+0000 7f8fe3be6700 1 --2- 172.21.15.44:0/2504887419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fdc100120 0x7f8fdc1959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.071+0000 7f8fe3be6700 1 -- 172.21.15.44:0/2504887419 >> 172.21.15.44:0/2504887419 conn(0x7f8fdc074f30 msgr2=0x7f8fdc0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.071+0000 7f8fe3be6700 1 -- 172.21.15.44:0/2504887419 shutdown_connections 2023-09-10T00:50:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:22.071+0000 7f8fe3be6700 1 -- 172.21.15.44:0/2504887419 wait complete. 2023-09-10T00:50:22.141 INFO:teuthology.orchestra.run.smithi044.stderr:37000000 of 50000000 tuples (74%) done (elapsed 561.78 s, remaining 197.38 s) 2023-09-10T00:50:23.020 INFO:teuthology.orchestra.run.smithi044.stderr:37100000 of 50000000 tuples (74%) done (elapsed 562.65 s, remaining 195.64 s) 2023-09-10T00:50:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:22 smithi131 ceph-mon[103713]: pgmap v302: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 MiB/s wr, 70 op/s 2023-09-10T00:50:23.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:22 smithi044 ceph-mon[98676]: pgmap v302: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 MiB/s wr, 70 op/s 2023-09-10T00:50:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:22 smithi099 ceph-mon[102692]: pgmap v302: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 MiB/s wr, 70 op/s 2023-09-10T00:50:24.153 INFO:teuthology.orchestra.run.smithi044.stderr:37200000 of 50000000 tuples (74%) done (elapsed 563.79 s, remaining 193.99 s) 2023-09-10T00:50:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:24 smithi131 ceph-mon[103713]: pgmap v303: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 61 op/s 2023-09-10T00:50:25.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:24 smithi044 ceph-mon[98676]: pgmap v303: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 61 op/s 2023-09-10T00:50:25.076 INFO:teuthology.orchestra.run.smithi044.stderr:37300000 of 50000000 tuples (74%) done (elapsed 564.71 s, remaining 192.27 s) 2023-09-10T00:50:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:24 smithi099 ceph-mon[102692]: pgmap v303: 129 pgs: 129 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 61 op/s 2023-09-10T00:50:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:26 smithi131 ceph-mon[103713]: pgmap v304: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 47 MiB/s wr, 110 op/s 2023-09-10T00:50:27.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:26 smithi044 ceph-mon[98676]: pgmap v304: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 47 MiB/s wr, 110 op/s 2023-09-10T00:50:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:26 smithi099 ceph-mon[102692]: pgmap v304: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 47 MiB/s wr, 110 op/s 2023-09-10T00:50:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:28 smithi131 ceph-mon[103713]: pgmap v305: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 32 MiB/s wr, 60 op/s 2023-09-10T00:50:29.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:28 smithi044 ceph-mon[98676]: pgmap v305: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 32 MiB/s wr, 60 op/s 2023-09-10T00:50:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:28 smithi099 ceph-mon[102692]: pgmap v305: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 32 MiB/s wr, 60 op/s 2023-09-10T00:50:29.344 INFO:teuthology.orchestra.run.smithi044.stderr:37400000 of 50000000 tuples (74%) done (elapsed 568.98 s, remaining 191.69 s) 2023-09-10T00:50:30.427 INFO:teuthology.orchestra.run.smithi044.stderr:37500000 of 50000000 tuples (75%) done (elapsed 570.06 s, remaining 190.02 s) 2023-09-10T00:50:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:30 smithi131 ceph-mon[103713]: pgmap v306: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 32 MiB/s wr, 60 op/s 2023-09-10T00:50:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:30 smithi044 ceph-mon[98676]: pgmap v306: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 32 MiB/s wr, 60 op/s 2023-09-10T00:50:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:30 smithi099 ceph-mon[102692]: pgmap v306: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 32 MiB/s wr, 60 op/s 2023-09-10T00:50:31.413 INFO:teuthology.orchestra.run.smithi044.stderr:37600000 of 50000000 tuples (75%) done (elapsed 571.05 s, remaining 188.32 s) 2023-09-10T00:50:32.325 INFO:teuthology.orchestra.run.smithi044.stderr:37700000 of 50000000 tuples (75%) done (elapsed 571.96 s, remaining 186.61 s) 2023-09-10T00:50:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:32 smithi131 ceph-mon[103713]: pgmap v307: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 38 MiB/s wr, 74 op/s 2023-09-10T00:50:33.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:32 smithi044 ceph-mon[98676]: pgmap v307: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 38 MiB/s wr, 74 op/s 2023-09-10T00:50:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:32 smithi099 ceph-mon[102692]: pgmap v307: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 38 MiB/s wr, 74 op/s 2023-09-10T00:50:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:34 smithi131 ceph-mon[103713]: pgmap v308: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 21 MiB/s wr, 61 op/s 2023-09-10T00:50:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:34 smithi044 ceph-mon[98676]: pgmap v308: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 21 MiB/s wr, 61 op/s 2023-09-10T00:50:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:34 smithi099 ceph-mon[102692]: pgmap v308: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 21 MiB/s wr, 61 op/s 2023-09-10T00:50:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.194+0000 7f2ac655c700 1 -- 172.21.15.44:0/58578551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac00679f0 msgr2=0x7f2ac0067e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.194+0000 7f2ac655c700 1 --2- 172.21.15.44:0/58578551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac00679f0 0x7f2ac0067e70 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40099c0 tx=0x7f2ab4009cd0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.194+0000 7f2ac655c700 1 -- 172.21.15.44:0/58578551 shutdown_connections 2023-09-10T00:50:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.194+0000 7f2ac655c700 1 --2- 172.21.15.44:0/58578551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ac00683b0 0x7f2ac010d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.194+0000 7f2ac655c700 1 --2- 172.21.15.44:0/58578551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac0106650 0x7f2ac0106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.194+0000 7f2ac655c700 1 --2- 172.21.15.44:0/58578551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac00679f0 0x7f2ac0067e70 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.194+0000 7f2ac655c700 1 -- 172.21.15.44:0/58578551 >> 172.21.15.44:0/58578551 conn(0x7f2ac0074f30 msgr2=0x7f2ac0075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.194+0000 7f2ac655c700 1 -- 172.21.15.44:0/58578551 shutdown_connections 2023-09-10T00:50:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.194+0000 7f2ac655c700 1 -- 172.21.15.44:0/58578551 wait complete. 2023-09-10T00:50:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.194+0000 7f2ac655c700 1 Processor -- start 2023-09-10T00:50:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2ac655c700 1 -- start start 2023-09-10T00:50:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2ac655c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac00679f0 0x7f2ac019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2ac655c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ac00683b0 0x7f2ac019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2ac655c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac0106650 0x7f2ac0196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2ac655c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac0196a70 con 0x7f2ac0106650 2023-09-10T00:50:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2ac655c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac0196be0 con 0x7f2ac00679f0 2023-09-10T00:50:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2ac655c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac0196da0 con 0x7f2ac00683b0 2023-09-10T00:50:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2abffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac00679f0 0x7f2ac019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2ac4af9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac0106650 0x7f2ac0196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2abffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac00679f0 0x7f2ac019c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54026/0 (socket says 172.21.15.44:54026) 2023-09-10T00:50:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2abffff700 1 -- 172.21.15.44:0/2365722289 learned_addr learned my addr 172.21.15.44:0/2365722289 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2ac4af9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac0106650 0x7f2ac0196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52290/0 (socket says 172.21.15.44:52290) 2023-09-10T00:50:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2abf7fe700 1 --2- 172.21.15.44:0/2365722289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ac00683b0 0x7f2ac019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2abffff700 1 -- 172.21.15.44:0/2365722289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ac00683b0 msgr2=0x7f2ac019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2abffff700 1 --2- 172.21.15.44:0/2365722289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ac00683b0 0x7f2ac019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2abffff700 1 -- 172.21.15.44:0/2365722289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac0106650 msgr2=0x7f2ac0196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2abffff700 1 --2- 172.21.15.44:0/2365722289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac0106650 0x7f2ac0196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2abffff700 1 -- 172.21.15.44:0/2365722289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab40095f0 con 0x7f2ac00679f0 2023-09-10T00:50:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.195+0000 7f2ac4af9700 1 --2- 172.21.15.44:0/2365722289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac0106650 0x7f2ac0196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.196+0000 7f2abffff700 1 --2- 172.21.15.44:0/2365722289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac00679f0 0x7f2ac019c5a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000e9a0 tx=0x7f2ab000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.196+0000 7f2abd7fa700 1 -- 172.21.15.44:0/2365722289 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0009790 con 0x7f2ac00679f0 2023-09-10T00:50:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.196+0000 7f2ac655c700 1 -- 172.21.15.44:0/2365722289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac0197080 con 0x7f2ac00679f0 2023-09-10T00:50:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.196+0000 7f2abd7fa700 1 -- 172.21.15.44:0/2365722289 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ab0010430 con 0x7f2ac00679f0 2023-09-10T00:50:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.196+0000 7f2abd7fa700 1 -- 172.21.15.44:0/2365722289 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab000cea0 con 0x7f2ac00679f0 2023-09-10T00:50:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.196+0000 7f2ac655c700 1 -- 172.21.15.44:0/2365722289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac01a3590 con 0x7f2ac00679f0 2023-09-10T00:50:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.198+0000 7f2abd7fa700 1 -- 172.21.15.44:0/2365722289 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f2ab00105a0 con 0x7f2ac00679f0 2023-09-10T00:50:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.199+0000 7f2abd7fa700 1 --2- 172.21.15.44:0/2365722289 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2aa80a07b0 0x7f2aa80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.199+0000 7f2abf7fe700 1 --2- 172.21.15.44:0/2365722289 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2aa80a07b0 0x7f2aa80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.199+0000 7f2abd7fa700 1 -- 172.21.15.44:0/2365722289 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2ab0014070 con 0x7f2ac00679f0 2023-09-10T00:50:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.199+0000 7f2abf7fe700 1 --2- 172.21.15.44:0/2365722289 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2aa80a07b0 0x7f2aa80a2c70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400b580 tx=0x7f2ab4005c00 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2ad2722b80 1 Processor -- start 2023-09-10T00:50:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2ad2722b80 1 -- start start 2023-09-10T00:50:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2ad2722b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ad41a0 0x55df13ad4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2ad2722b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ace300 0x55df13ace780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2ad2722b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13acecc0 0x55df13adaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2ad2722b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df13a276d0 con 0x55df13ad41a0 2023-09-10T00:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2ad2722b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df13a24ff0 con 0x55df13ace300 2023-09-10T00:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2ad2722b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df13a208e0 con 0x55df13acecc0 2023-09-10T00:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2abffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ad41a0 0x55df13ad4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2abffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ad41a0 0x55df13ad4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52294/0 (socket says 172.21.15.44:52294) 2023-09-10T00:50:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2abffff700 1 -- 172.21.15.44:0/138491187 learned_addr learned my addr 172.21.15.44:0/138491187 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2abf7fe700 1 --2- 172.21.15.44:0/138491187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ace300 0x55df13ace780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2ac4af9700 1 --2- 172.21.15.44:0/138491187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13acecc0 0x55df13adaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2abffff700 1 -- 172.21.15.44:0/138491187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13acecc0 msgr2=0x55df13adaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2abffff700 1 --2- 172.21.15.44:0/138491187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13acecc0 0x55df13adaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2abffff700 1 -- 172.21.15.44:0/138491187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ace300 msgr2=0x55df13ace780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2abffff700 1 --2- 172.21.15.44:0/138491187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ace300 0x55df13ace780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2abffff700 1 -- 172.21.15.44:0/138491187 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df1384cd80 con 0x55df13ad41a0 2023-09-10T00:50:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.200+0000 7f2abf7fe700 1 --2- 172.21.15.44:0/138491187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ace300 0x55df13ace780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.201+0000 7f2abffff700 1 --2- 172.21.15.44:0/138491187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ad41a0 0x55df13ad4580 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0095fc0 tx=0x7f2ab0098f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea44045885e64d84 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.201+0000 7f2aa67fc700 1 -- 172.21.15.44:0/138491187 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab009d170 con 0x55df13ad41a0 2023-09-10T00:50:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.201+0000 7f2aa67fc700 1 -- 172.21.15.44:0/138491187 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ab0099550 con 0x55df13ad41a0 2023-09-10T00:50:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.201+0000 7f2aa67fc700 1 -- 172.21.15.44:0/138491187 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab00997b0 con 0x55df13ad41a0 2023-09-10T00:50:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.201+0000 7f2ad2722b80 1 -- 172.21.15.44:0/138491187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ad41a0 msgr2=0x55df13ad4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.201+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/138491187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ad41a0 0x55df13ad4580 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0095fc0 tx=0x7f2ab0098f30 comp rx=0 tx=0).stop 2023-09-10T00:50:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.201+0000 7f2ad2722b80 1 -- 172.21.15.44:0/138491187 shutdown_connections 2023-09-10T00:50:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.201+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/138491187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13acecc0 0x55df13adaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.201+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/138491187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ad41a0 0x55df13ad4580 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.201+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/138491187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ace300 0x55df13ace780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.201+0000 7f2ad2722b80 1 -- 172.21.15.44:0/138491187 >> 172.21.15.44:0/138491187 conn(0x55df13a43030 msgr2=0x55df13ac9560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.201+0000 7f2ad2722b80 1 -- 172.21.15.44:0/138491187 shutdown_connections 2023-09-10T00:50:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.201+0000 7f2ad2722b80 1 -- 172.21.15.44:0/138491187 wait complete. 2023-09-10T00:50:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.202+0000 7f2ad2722b80 1 -- start start 2023-09-10T00:50:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.202+0000 7f2ad2722b80 1 Processor -- start 2023-09-10T00:50:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.202+0000 7f2ad2722b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ace300 0x55df13adde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.202+0000 7f2ad2722b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13acecc0 0x55df13ade3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.202+0000 7f2ad2722b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ad41a0 0x55df13ad8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.202+0000 7f2ad2722b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df139beb10 con 0x55df13ad41a0 2023-09-10T00:50:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.202+0000 7f2ad2722b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df139be810 con 0x55df13ace300 2023-09-10T00:50:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.202+0000 7f2ad2722b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df138a28c0 con 0x55df13acecc0 2023-09-10T00:50:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.202+0000 7f2ad2722b80 10 client.0 _ll_register_callbacks cb 0x55df13908a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:50:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.202+0000 7f2ac4af9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ad41a0 0x55df13ad8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.202+0000 7f2abffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ace300 0x55df13adde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.203+0000 7f2abffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ace300 0x55df13adde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54040/0 (socket says 172.21.15.44:54040) 2023-09-10T00:50:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.203+0000 7f2abf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13acecc0 0x55df13ade3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.203+0000 7f2abffff700 1 -- 172.21.15.44:0/2568993858 learned_addr learned my addr 172.21.15.44:0/2568993858 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.203+0000 7f2abffff700 1 -- 172.21.15.44:0/2568993858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13acecc0 msgr2=0x55df13ade3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.203+0000 7f2abffff700 1 --2- 172.21.15.44:0/2568993858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13acecc0 0x55df13ade3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.203+0000 7f2abffff700 1 -- 172.21.15.44:0/2568993858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ad41a0 msgr2=0x55df13ad8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.203+0000 7f2abffff700 1 --2- 172.21.15.44:0/2568993858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ad41a0 0x55df13ad8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.203+0000 7f2abffff700 1 -- 172.21.15.44:0/2568993858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac010dcd0 con 0x55df13ace300 2023-09-10T00:50:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.203+0000 7f2abffff700 1 --2- 172.21.15.44:0/2568993858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ace300 0x55df13adde80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00c9d70 tx=0x7f2ab0098c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=865ae1fad7ead744 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.203+0000 7f2a9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:50:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.203+0000 7f2a9f7fe700 1 -- 172.21.15.44:0/2568993858 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab009d170 con 0x55df13ace300 2023-09-10T00:50:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.203+0000 7f2a9f7fe700 1 -- 172.21.15.44:0/2568993858 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2ab009d380 con 0x55df13ace300 2023-09-10T00:50:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.203+0000 7f2ad2722b80 1 -- 172.21.15.44:0/2568993858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df13978490 con 0x55df13ace300 2023-09-10T00:50:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.203+0000 7f2a9f7fe700 1 -- 172.21.15.44:0/2568993858 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ab00a15c0 con 0x55df13ace300 2023-09-10T00:50:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.203+0000 7f2ad2722b80 10 client.26948 fetch_fsmap learned FSMap version 59 2023-09-10T00:50:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.203+0000 7f2ad2722b80 1 -- 172.21.15.44:0/2568993858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7f2ac01a3590 con 0x55df13ace300 2023-09-10T00:50:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.204+0000 7f2a9f7fe700 1 -- 172.21.15.44:0/2568993858 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55df13978490 con 0x55df13ace300 2023-09-10T00:50:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.204+0000 7f2a9f7fe700 1 -- 172.21.15.44:0/2568993858 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 59) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f2ab00b7220 con 0x55df13ace300 2023-09-10T00:50:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.204+0000 7f2ad2722b80 10 client.26948 fetch_fsmap finished waiting for FSMap version 59 2023-09-10T00:50:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.204+0000 7f2ad2722b80 10 client.26948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:50:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.204+0000 7f2ad2722b80 20 client.26948 populate_metadata read hostname 'smithi044' 2023-09-10T00:50:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.204+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/2568993858 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55df13ae8170 0x55df13aea590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.204+0000 7f2ad2722b80 4 client.26948 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:50:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.204+0000 7f2ad2722b80 1 -- 172.21.15.44:0/2568993858 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df13a35840 con 0x55df13ae8170 2023-09-10T00:50:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.204+0000 7f2abf7fe700 1 --2- 172.21.15.44:0/2568993858 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55df13ae8170 0x55df13aea590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.204+0000 7f2a9f7fe700 10 client.26948 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.204+0000 7f2abf7fe700 1 --2- 172.21.15.44:0/2568993858 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55df13ae8170 0x55df13aea590 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d178e9296c4df66 server_cookie=8d89de49769c3a79 in_seq=0 out_seq=0 2023-09-10T00:50:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.205+0000 7f2a9f7fe700 1 -- 172.21.15.44:0/2568993858 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55df13a35840 con 0x55df13ae8170 2023-09-10T00:50:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.205+0000 7f2a9f7fe700 10 client.26948 handle_command_reply: tid=0 2023-09-10T00:50:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.205+0000 7f2ad2722b80 1 client.26948 shutdown 2023-09-10T00:50:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.205+0000 7f2ad2722b80 1 -- 172.21.15.44:0/2568993858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ace300 msgr2=0x55df13adde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.205+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/2568993858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ace300 0x55df13adde80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00c9d70 tx=0x7f2ab0098c70 comp rx=0 tx=0).stop 2023-09-10T00:50:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.205+0000 7f2ad2722b80 1 -- 172.21.15.44:0/2568993858 shutdown_connections 2023-09-10T00:50:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.205+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/2568993858 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55df13ae8170 0x55df13aea590 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.205+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/2568993858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ad41a0 0x55df13ad8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.205+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/2568993858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ace300 0x55df13adde80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.205+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/2568993858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13acecc0 0x55df13ade3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.205+0000 7f2a9f7fe700 0 client.26948 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.205+0000 7f2ad2722b80 1 -- 172.21.15.44:0/2568993858 >> 172.21.15.44:0/2568993858 conn(0x55df13a43030 msgr2=0x55df13ac87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.205+0000 7f2ad2722b80 1 -- 172.21.15.44:0/2568993858 shutdown_connections 2023-09-10T00:50:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.205+0000 7f2ad2722b80 1 -- 172.21.15.44:0/2568993858 wait complete. 2023-09-10T00:50:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.205+0000 7f2ad2722b80 20 client.26948 trim_cache size 0 max 16384 2023-09-10T00:50:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.220+0000 7f2ad2722b80 1 Processor -- start 2023-09-10T00:50:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.220+0000 7f2ad2722b80 1 -- start start 2023-09-10T00:50:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.220+0000 7f2ad2722b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13ace300 0x55df13ace6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.220+0000 7f2ad2722b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13acecc0 0x55df13ad7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.220+0000 7f2ad2722b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ad41a0 0x55df13ac8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.220+0000 7f2ad2722b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df13ad7870 con 0x55df13acecc0 2023-09-10T00:50:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.220+0000 7f2ad2722b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df13aea560 con 0x55df13ad41a0 2023-09-10T00:50:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.220+0000 7f2ad2722b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df13ac9460 con 0x55df13ace300 2023-09-10T00:50:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.220+0000 7f2abf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13acecc0 0x55df13ad7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.220+0000 7f2abffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13ace300 0x55df13ace6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.220+0000 7f2abf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13acecc0 0x55df13ad7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52310/0 (socket says 172.21.15.44:52310) 2023-09-10T00:50:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.220+0000 7f2abffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13ace300 0x55df13ace6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59190/0 (socket says 172.21.15.44:59190) 2023-09-10T00:50:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.220+0000 7f2ac4af9700 1 --2- 172.21.15.44:0/3273485510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ad41a0 0x55df13ac8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.220+0000 7f2abf7fe700 1 -- 172.21.15.44:0/3273485510 learned_addr learned my addr 172.21.15.44:0/3273485510 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.221+0000 7f2abf7fe700 1 -- 172.21.15.44:0/3273485510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13ace300 msgr2=0x55df13ace6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.221+0000 7f2abf7fe700 1 --2- 172.21.15.44:0/3273485510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13ace300 0x55df13ace6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.221+0000 7f2abf7fe700 1 -- 172.21.15.44:0/3273485510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ad41a0 msgr2=0x55df13ac8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.221+0000 7f2abf7fe700 1 --2- 172.21.15.44:0/3273485510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ad41a0 0x55df13ac8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.221+0000 7f2abf7fe700 1 -- 172.21.15.44:0/3273485510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df13835c00 con 0x55df13acecc0 2023-09-10T00:50:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.221+0000 7f2ac4af9700 1 --2- 172.21.15.44:0/3273485510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ad41a0 0x55df13ac8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.221+0000 7f2abf7fe700 1 --2- 172.21.15.44:0/3273485510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13acecc0 0x55df13ad7330 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40098e0 tx=0x7f2ab4027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f0a94a28342359 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.221+0000 7f2aa6ffd700 1 -- 172.21.15.44:0/3273485510 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab402a070 con 0x55df13acecc0 2023-09-10T00:50:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.221+0000 7f2aa6ffd700 1 -- 172.21.15.44:0/3273485510 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ab402b040 con 0x55df13acecc0 2023-09-10T00:50:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.221+0000 7f2aa6ffd700 1 -- 172.21.15.44:0/3273485510 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4003de0 con 0x55df13acecc0 2023-09-10T00:50:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.221+0000 7f2ad2722b80 1 -- 172.21.15.44:0/3273485510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13acecc0 msgr2=0x55df13ad7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.221+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/3273485510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13acecc0 0x55df13ad7330 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40098e0 tx=0x7f2ab4027040 comp rx=0 tx=0).stop 2023-09-10T00:50:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.222+0000 7f2ad2722b80 1 -- 172.21.15.44:0/3273485510 shutdown_connections 2023-09-10T00:50:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.222+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/3273485510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13ad41a0 0x55df13ac8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.222+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/3273485510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13ace300 0x55df13ace6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.222+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/3273485510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13acecc0 0x55df13ad7330 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.222+0000 7f2ad2722b80 1 -- 172.21.15.44:0/3273485510 >> 172.21.15.44:0/3273485510 conn(0x55df13a43030 msgr2=0x55df13ad8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.222+0000 7f2ad2722b80 1 -- 172.21.15.44:0/3273485510 shutdown_connections 2023-09-10T00:50:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.222+0000 7f2ad2722b80 1 -- 172.21.15.44:0/3273485510 wait complete. 2023-09-10T00:50:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.222+0000 7f2ad2722b80 1 -- start start 2023-09-10T00:50:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.222+0000 7f2ad2722b80 1 Processor -- start 2023-09-10T00:50:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.222+0000 7f2ad2722b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ace300 0x55df13b344f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.222+0000 7f2ad2722b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13acecc0 0x55df13b34a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.222+0000 7f2ad2722b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13ad41a0 0x55df13b39350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.222+0000 7f2ad2722b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df13b39890 con 0x55df13ace300 2023-09-10T00:50:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.222+0000 7f2ad2722b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df13b399d0 con 0x55df13acecc0 2023-09-10T00:50:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.222+0000 7f2ad2722b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df13b39b90 con 0x55df13ad41a0 2023-09-10T00:50:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2ad2722b80 10 client.0 _ll_register_callbacks cb 0x7f2ab400b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:50:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2abffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ace300 0x55df13b344f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2abf7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13acecc0 0x55df13b34a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2ac4af9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13ad41a0 0x55df13b39350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2abf7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13acecc0 0x55df13b34a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54056/0 (socket says 172.21.15.44:54056) 2023-09-10T00:50:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2abffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ace300 0x55df13b344f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52314/0 (socket says 172.21.15.44:52314) 2023-09-10T00:50:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2abf7fe700 1 -- 172.21.15.44:0/23678657 learned_addr learned my addr 172.21.15.44:0/23678657 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2abf7fe700 1 -- 172.21.15.44:0/23678657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13ad41a0 msgr2=0x55df13b39350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2abf7fe700 1 --2- 172.21.15.44:0/23678657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13ad41a0 0x55df13b39350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2abf7fe700 1 -- 172.21.15.44:0/23678657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ace300 msgr2=0x55df13b344f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2abf7fe700 1 --2- 172.21.15.44:0/23678657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ace300 0x55df13b344f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2abf7fe700 1 -- 172.21.15.44:0/23678657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab800e610 con 0x55df13acecc0 2023-09-10T00:50:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2abffff700 1 --2- 172.21.15.44:0/23678657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ace300 0x55df13b344f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2abf7fe700 1 --2- 172.21.15.44:0/23678657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13acecc0 0x55df13b34a30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4034040 tx=0x7f2ab40314b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=534ec895656bd7c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2a9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:50:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2a9f7fe700 1 -- 172.21.15.44:0/23678657 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab402a040 con 0x55df13acecc0 2023-09-10T00:50:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2ad2722b80 1 -- 172.21.15.44:0/23678657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ab0099550 con 0x55df13acecc0 2023-09-10T00:50:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2a9f7fe700 1 -- 172.21.15.44:0/23678657 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2ab4031d90 con 0x55df13acecc0 2023-09-10T00:50:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2a9f7fe700 1 -- 172.21.15.44:0/23678657 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ab40037e0 con 0x55df13acecc0 2023-09-10T00:50:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2ad2722b80 10 client.26954 fetch_fsmap learned FSMap version 59 2023-09-10T00:50:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.223+0000 7f2ad2722b80 1 -- 172.21.15.44:0/23678657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55df138731f0 con 0x55df13acecc0 2023-09-10T00:50:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.224+0000 7f2a9f7fe700 1 -- 172.21.15.44:0/23678657 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ab0099550 con 0x55df13acecc0 2023-09-10T00:50:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.224+0000 7f2a9f7fe700 1 -- 172.21.15.44:0/23678657 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 59) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f2ab4038030 con 0x55df13acecc0 2023-09-10T00:50:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.224+0000 7f2ad2722b80 10 client.26954 fetch_fsmap finished waiting for FSMap version 59 2023-09-10T00:50:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.224+0000 7f2ad2722b80 10 client.26954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.224+0000 7f2ad2722b80 20 client.26954 populate_metadata read hostname 'smithi044' 2023-09-10T00:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.224+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/23678657 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55df13b41040 0x55df13b43460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.224+0000 7f2ad2722b80 4 client.26954 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.224+0000 7f2ad2722b80 1 -- 172.21.15.44:0/23678657 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55df13a198c0 con 0x55df13b41040 2023-09-10T00:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.224+0000 7f2abffff700 1 --2- 172.21.15.44:0/23678657 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55df13b41040 0x55df13b43460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.224+0000 7f2a9f7fe700 10 client.26954 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.224+0000 7f2abffff700 1 --2- 172.21.15.44:0/23678657 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55df13b41040 0x55df13b43460 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ca51973a3c0cb1a server_cookie=cf36ac35dff0cea6 in_seq=0 out_seq=0 2023-09-10T00:50:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.241+0000 7f2a9f7fe700 1 -- 172.21.15.44:0/23678657 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55df13a198c0 con 0x55df13b41040 2023-09-10T00:50:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.241+0000 7f2a9f7fe700 10 client.26954 handle_command_reply: tid=0 2023-09-10T00:50:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.241+0000 7f2ad2722b80 1 client.26954 shutdown 2023-09-10T00:50:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.241+0000 7f2ad2722b80 1 -- 172.21.15.44:0/23678657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13acecc0 msgr2=0x55df13b34a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.241+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/23678657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13acecc0 0x55df13b34a30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4034040 tx=0x7f2ab40314b0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.241+0000 7f2ad2722b80 1 -- 172.21.15.44:0/23678657 shutdown_connections 2023-09-10T00:50:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.241+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/23678657 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55df13b41040 0x55df13b43460 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.241+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/23678657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df13ad41a0 0x55df13b39350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.241+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/23678657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df13ace300 0x55df13b344f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.241+0000 7f2a9f7fe700 0 client.26954 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.241+0000 7f2ad2722b80 1 --2- 172.21.15.44:0/23678657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df13acecc0 0x55df13b34a30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.241+0000 7f2ad2722b80 1 -- 172.21.15.44:0/23678657 >> 172.21.15.44:0/23678657 conn(0x55df13a43030 msgr2=0x55df13ad79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.241+0000 7f2ad2722b80 1 -- 172.21.15.44:0/23678657 shutdown_connections 2023-09-10T00:50:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.241+0000 7f2ad2722b80 1 -- 172.21.15.44:0/23678657 wait complete. 2023-09-10T00:50:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.241+0000 7f2ad2722b80 20 client.26954 trim_cache size 0 max 16384 2023-09-10T00:50:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:50:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:50:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:50:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:50:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.242+0000 7f2ac655c700 1 -- 172.21.15.44:0/2365722289 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2aa80a07b0 msgr2=0x7f2aa80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.242+0000 7f2ac655c700 1 --2- 172.21.15.44:0/2365722289 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2aa80a07b0 0x7f2aa80a2c70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400b580 tx=0x7f2ab4005c00 comp rx=0 tx=0).stop 2023-09-10T00:50:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.242+0000 7f2ac655c700 1 -- 172.21.15.44:0/2365722289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac00679f0 msgr2=0x7f2ac019c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.242+0000 7f2ac655c700 1 --2- 172.21.15.44:0/2365722289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac00679f0 0x7f2ac019c5a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000e9a0 tx=0x7f2ab000ecb0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.242+0000 7f2ac655c700 1 -- 172.21.15.44:0/2365722289 shutdown_connections 2023-09-10T00:50:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.242+0000 7f2ac655c700 1 --2- 172.21.15.44:0/2365722289 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2aa80a07b0 0x7f2aa80a2c70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.242+0000 7f2ac655c700 1 --2- 172.21.15.44:0/2365722289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac0106650 0x7f2ac0196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.242+0000 7f2ac655c700 1 --2- 172.21.15.44:0/2365722289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac00679f0 0x7f2ac019c5a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.242+0000 7f2ac655c700 1 --2- 172.21.15.44:0/2365722289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ac00683b0 0x7f2ac019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.242+0000 7f2ac655c700 1 -- 172.21.15.44:0/2365722289 >> 172.21.15.44:0/2365722289 conn(0x7f2ac0074f30 msgr2=0x7f2ac00fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.242+0000 7f2ac655c700 1 -- 172.21.15.44:0/2365722289 shutdown_connections 2023-09-10T00:50:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:35.242+0000 7f2ac655c700 1 -- 172.21.15.44:0/2365722289 wait complete. 2023-09-10T00:50:36.375 INFO:teuthology.orchestra.run.smithi044.stderr:37800000 of 50000000 tuples (75%) done (elapsed 576.01 s, remaining 185.91 s) 2023-09-10T00:50:37.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:36 smithi131 ceph-mon[103713]: pgmap v309: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 30 MiB/s wr, 88 op/s 2023-09-10T00:50:37.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:36 smithi044 ceph-mon[98676]: pgmap v309: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 30 MiB/s wr, 88 op/s 2023-09-10T00:50:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:36 smithi099 ceph-mon[102692]: pgmap v309: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 30 MiB/s wr, 88 op/s 2023-09-10T00:50:37.322 INFO:teuthology.orchestra.run.smithi044.stderr:37900000 of 50000000 tuples (75%) done (elapsed 576.96 s, remaining 184.20 s) 2023-09-10T00:50:38.302 INFO:teuthology.orchestra.run.smithi044.stderr:38000000 of 50000000 tuples (76%) done (elapsed 577.94 s, remaining 182.51 s) 2023-09-10T00:50:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:38 smithi131 ceph-mon[103713]: pgmap v310: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 15 MiB/s wr, 39 op/s 2023-09-10T00:50:39.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:38 smithi044 ceph-mon[98676]: pgmap v310: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 15 MiB/s wr, 39 op/s 2023-09-10T00:50:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:38 smithi099 ceph-mon[102692]: pgmap v310: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 15 MiB/s wr, 39 op/s 2023-09-10T00:50:39.253 INFO:teuthology.orchestra.run.smithi044.stderr:38100000 of 50000000 tuples (76%) done (elapsed 578.89 s, remaining 180.81 s) 2023-09-10T00:50:40.178 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:50:40.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.318+0000 7faf3284e700 1 -- 172.21.15.44:0/3864936945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf2c100550 msgr2=0x7faf2c10c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.318+0000 7faf3284e700 1 --2- 172.21.15.44:0/3864936945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf2c100550 0x7faf2c10c3b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7faf24009fc0 tx=0x7faf2400b330 comp rx=0 tx=0).stop 2023-09-10T00:50:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.318+0000 7faf3284e700 1 -- 172.21.15.44:0/3864936945 shutdown_connections 2023-09-10T00:50:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.318+0000 7faf3284e700 1 --2- 172.21.15.44:0/3864936945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf2c100550 0x7faf2c10c3b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.318+0000 7faf3284e700 1 --2- 172.21.15.44:0/3864936945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c1044c0 0x7faf2c1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.318+0000 7faf3284e700 1 --2- 172.21.15.44:0/3864936945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0ffb90 0x7faf2c100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.318+0000 7faf3284e700 1 -- 172.21.15.44:0/3864936945 >> 172.21.15.44:0/3864936945 conn(0x7faf2c076a70 msgr2=0x7faf2c0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.318+0000 7faf3284e700 1 -- 172.21.15.44:0/3864936945 shutdown_connections 2023-09-10T00:50:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.318+0000 7faf3284e700 1 -- 172.21.15.44:0/3864936945 wait complete. 2023-09-10T00:50:40.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.319+0000 7faf3284e700 1 Processor -- start 2023-09-10T00:50:40.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.319+0000 7faf3284e700 1 -- start start 2023-09-10T00:50:40.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.319+0000 7faf3284e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0ffb90 0x7faf2c195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.319+0000 7faf3284e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf2c100550 0x7faf2c195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.319+0000 7faf3284e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c1044c0 0x7faf2c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.324 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.319+0000 7faf3284e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c19a600 con 0x7faf2c100550 2023-09-10T00:50:40.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.319+0000 7faf3284e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c19a770 con 0x7faf2c1044c0 2023-09-10T00:50:40.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.319+0000 7faf3284e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c19a930 con 0x7faf2c0ffb90 2023-09-10T00:50:40.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.319+0000 7faf2b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf2c100550 0x7faf2c195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.319+0000 7faf2bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0ffb90 0x7faf2c195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.325 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.319+0000 7faf2b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf2c100550 0x7faf2c195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52330/0 (socket says 172.21.15.44:52330) 2023-09-10T00:50:40.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.319+0000 7faf2bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0ffb90 0x7faf2c195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59210/0 (socket says 172.21.15.44:59210) 2023-09-10T00:50:40.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.319+0000 7faf2b7fe700 1 -- 172.21.15.44:0/2697524193 learned_addr learned my addr 172.21.15.44:0/2697524193 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:40.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.320+0000 7faf30deb700 1 --2- 172.21.15.44:0/2697524193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c1044c0 0x7faf2c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.320+0000 7faf2b7fe700 1 -- 172.21.15.44:0/2697524193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0ffb90 msgr2=0x7faf2c195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:40.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.320+0000 7faf2b7fe700 1 --2- 172.21.15.44:0/2697524193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0ffb90 0x7faf2c195620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.326 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.320+0000 7faf2b7fe700 1 -- 172.21.15.44:0/2697524193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c1044c0 msgr2=0x7faf2c199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:40.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.320+0000 7faf2b7fe700 1 --2- 172.21.15.44:0/2697524193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c1044c0 0x7faf2c199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.320+0000 7faf2b7fe700 1 -- 172.21.15.44:0/2697524193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf20009520 con 0x7faf2c100550 2023-09-10T00:50:40.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.320+0000 7faf2bfff700 1 --2- 172.21.15.44:0/2697524193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0ffb90 0x7faf2c195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:40.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.320+0000 7faf30deb700 1 --2- 172.21.15.44:0/2697524193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c1044c0 0x7faf2c199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:50:40.327 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.320+0000 7faf2b7fe700 1 --2- 172.21.15.44:0/2697524193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf2c100550 0x7faf2c195b60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7faf2000eb30 tx=0x7faf2000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:40.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.320+0000 7faf297fa700 1 -- 172.21.15.44:0/2697524193 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20009880 con 0x7faf2c100550 2023-09-10T00:50:40.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.320+0000 7faf3284e700 1 -- 172.21.15.44:0/2697524193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf1c0095f0 con 0x7faf2c100550 2023-09-10T00:50:40.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.320+0000 7faf297fa700 1 -- 172.21.15.44:0/2697524193 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf200044b0 con 0x7faf2c100550 2023-09-10T00:50:40.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.320+0000 7faf297fa700 1 -- 172.21.15.44:0/2697524193 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20003df0 con 0x7faf2c100550 2023-09-10T00:50:40.328 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.320+0000 7faf3284e700 1 -- 172.21.15.44:0/2697524193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf24009c40 con 0x7faf2c100550 2023-09-10T00:50:40.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.323+0000 7faf297fa700 1 -- 172.21.15.44:0/2697524193 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7faf20003f50 con 0x7faf2c100550 2023-09-10T00:50:40.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.323+0000 7faf297fa700 1 --2- 172.21.15.44:0/2697524193 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faf140a07b0 0x7faf140a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.334 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.324+0000 7faf2bfff700 1 --2- 172.21.15.44:0/2697524193 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faf140a07b0 0x7faf140a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.324+0000 7faf297fa700 1 -- 172.21.15.44:0/2697524193 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9540+0+0 (secure 0 0 0) 0x7faf20014070 con 0x7faf2c100550 2023-09-10T00:50:40.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.324+0000 7faf2bfff700 1 --2- 172.21.15.44:0/2697524193 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faf140a07b0 0x7faf140a2c70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faf1c005fd0 tx=0x7faf1c009310 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:40.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.324+0000 7faf3ea14b80 1 Processor -- start 2023-09-10T00:50:40.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.324+0000 7faf3ea14b80 1 -- start start 2023-09-10T00:50:40.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.324+0000 7faf3ea14b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822146710 0x564822146af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.324+0000 7faf3ea14b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822147030 0x564822150900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.324+0000 7faf3ea14b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822150e40 0x564822153230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf3ea14b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482209f730 con 0x564822147030 2023-09-10T00:50:40.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf3ea14b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482209d050 con 0x564822150e40 2023-09-10T00:50:40.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf3ea14b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564822098940 con 0x564822146710 2023-09-10T00:50:40.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf2b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822147030 0x564822150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf2bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822146710 0x564822146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf2b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822147030 0x564822150900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52338/0 (socket says 172.21.15.44:52338) 2023-09-10T00:50:40.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf2b7fe700 1 -- 172.21.15.44:0/3059146179 learned_addr learned my addr 172.21.15.44:0/3059146179 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:40.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf30deb700 1 --2- 172.21.15.44:0/3059146179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822150e40 0x564822153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf2bfff700 1 -- 172.21.15.44:0/3059146179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822150e40 msgr2=0x564822153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:40.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf2bfff700 1 --2- 172.21.15.44:0/3059146179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822150e40 0x564822153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf2bfff700 1 -- 172.21.15.44:0/3059146179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822147030 msgr2=0x564822150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:40.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf2bfff700 1 --2- 172.21.15.44:0/3059146179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822147030 0x564822150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf2bfff700 1 -- 172.21.15.44:0/3059146179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564821ec4de0 con 0x564822146710 2023-09-10T00:50:40.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf2b7fe700 1 --2- 172.21.15.44:0/3059146179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822147030 0x564822150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:40.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf2bfff700 1 --2- 172.21.15.44:0/3059146179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822146710 0x564822146af0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00f660 tx=0x7faf1c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb299ceae55770b2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:40.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf127fc700 1 -- 172.21.15.44:0/3059146179 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c004220 con 0x564822146710 2023-09-10T00:50:40.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.325+0000 7faf127fc700 1 -- 172.21.15.44:0/3059146179 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf1c00fec0 con 0x564822146710 2023-09-10T00:50:40.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.326+0000 7faf127fc700 1 -- 172.21.15.44:0/3059146179 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c01b620 con 0x564822146710 2023-09-10T00:50:40.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.326+0000 7faf3ea14b80 1 -- 172.21.15.44:0/3059146179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822146710 msgr2=0x564822146af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:40.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.326+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/3059146179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822146710 0x564822146af0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00f660 tx=0x7faf1c014000 comp rx=0 tx=0).stop 2023-09-10T00:50:40.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.326+0000 7faf3ea14b80 1 -- 172.21.15.44:0/3059146179 shutdown_connections 2023-09-10T00:50:40.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.326+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/3059146179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822150e40 0x564822153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.326+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/3059146179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822146710 0x564822146af0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.326+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/3059146179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822147030 0x564822150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.326+0000 7faf3ea14b80 1 -- 172.21.15.44:0/3059146179 >> 172.21.15.44:0/3059146179 conn(0x5648220bb090 msgr2=0x564822141bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:40.340 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.326+0000 7faf3ea14b80 1 -- 172.21.15.44:0/3059146179 shutdown_connections 2023-09-10T00:50:40.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.326+0000 7faf3ea14b80 1 -- 172.21.15.44:0/3059146179 wait complete. 2023-09-10T00:50:40.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.327+0000 7faf3ea14b80 1 -- start start 2023-09-10T00:50:40.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.327+0000 7faf3ea14b80 1 Processor -- start 2023-09-10T00:50:40.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.327+0000 7faf3ea14b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822146710 0x564822150830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.327+0000 7faf3ea14b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822147030 0x56482214db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.341 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.327+0000 7faf3ea14b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822150e40 0x56482214e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.342 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.327+0000 7faf3ea14b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564822036b70 con 0x564822150e40 2023-09-10T00:50:40.342 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.327+0000 7faf3ea14b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564822036870 con 0x564822147030 2023-09-10T00:50:40.342 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.327+0000 7faf3ea14b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564821f1a920 con 0x564822146710 2023-09-10T00:50:40.342 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.327+0000 7faf3ea14b80 10 client.0 _ll_register_callbacks cb 0x564821f80a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:50:40.342 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf2bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822146710 0x564822150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf2bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822146710 0x564822150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59224/0 (socket says 172.21.15.44:59224) 2023-09-10T00:50:40.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf2bfff700 1 -- 172.21.15.44:0/4157018394 learned_addr learned my addr 172.21.15.44:0/4157018394 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:40.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf2b7fe700 1 --2- 172.21.15.44:0/4157018394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822147030 0x56482214db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf30deb700 1 --2- 172.21.15.44:0/4157018394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822150e40 0x56482214e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf2bfff700 1 -- 172.21.15.44:0/4157018394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822147030 msgr2=0x56482214db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:40.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf2bfff700 1 --2- 172.21.15.44:0/4157018394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822147030 0x56482214db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf2bfff700 1 -- 172.21.15.44:0/4157018394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822150e40 msgr2=0x56482214e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:40.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf2bfff700 1 --2- 172.21.15.44:0/4157018394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822150e40 0x56482214e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf2bfff700 1 -- 172.21.15.44:0/4157018394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf24009c40 con 0x564822146710 2023-09-10T00:50:40.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf2bfff700 1 --2- 172.21.15.44:0/4157018394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822146710 0x564822150830 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009860 tx=0x7faf1c003e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=724c7bfe2d908ee8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:40.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:50:40.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf0b7fe700 1 -- 172.21.15.44:0/4157018394 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c0044f0 con 0x564822146710 2023-09-10T00:50:40.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf3ea14b80 1 -- 172.21.15.44:0/4157018394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648220ad8a0 con 0x564822146710 2023-09-10T00:50:40.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf0b7fe700 1 -- 172.21.15.44:0/4157018394 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9540+0+0 (secure 0 0 0) 0x7faf1c004650 con 0x564822146710 2023-09-10T00:50:40.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf0b7fe700 1 -- 172.21.15.44:0/4157018394 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf1c030440 con 0x564822146710 2023-09-10T00:50:40.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf3ea14b80 10 client.26908 fetch_fsmap learned FSMap version 59 2023-09-10T00:50:40.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.328+0000 7faf3ea14b80 1 -- 172.21.15.44:0/4157018394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7faf1c017050 con 0x564822146710 2023-09-10T00:50:40.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.329+0000 7faf0b7fe700 1 -- 172.21.15.44:0/4157018394 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf1c039080 con 0x564822146710 2023-09-10T00:50:40.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.329+0000 7faf0b7fe700 1 -- 172.21.15.44:0/4157018394 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 59) v1 ==== 7914+0+0 (secure 0 0 0) 0x7faf1c0266a0 con 0x564822146710 2023-09-10T00:50:40.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.329+0000 7faf3ea14b80 10 client.26908 fetch_fsmap finished waiting for FSMap version 59 2023-09-10T00:50:40.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.329+0000 7faf3ea14b80 10 client.26908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:50:40.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.329+0000 7faf3ea14b80 20 client.26908 populate_metadata read hostname 'smithi044' 2023-09-10T00:50:40.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.329+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/4157018394 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564822160930 0x564822162d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.329+0000 7faf3ea14b80 4 client.26908 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:50:40.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.329+0000 7faf3ea14b80 1 -- 172.21.15.44:0/4157018394 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564822091920 con 0x564822160930 2023-09-10T00:50:40.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.329+0000 7faf2b7fe700 1 --2- 172.21.15.44:0/4157018394 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564822160930 0x564822162d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.329+0000 7faf0b7fe700 10 client.26908 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:40.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.329+0000 7faf2b7fe700 1 --2- 172.21.15.44:0/4157018394 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564822160930 0x564822162d50 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7613562406ad13c4 server_cookie=60922a73b5fb0b30 in_seq=0 out_seq=0 2023-09-10T00:50:40.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.330+0000 7faf0b7fe700 1 -- 172.21.15.44:0/4157018394 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564822091920 con 0x564822160930 2023-09-10T00:50:40.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.330+0000 7faf0b7fe700 10 client.26908 handle_command_reply: tid=0 2023-09-10T00:50:40.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.330+0000 7faf3ea14b80 1 client.26908 shutdown 2023-09-10T00:50:40.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.330+0000 7faf3ea14b80 1 -- 172.21.15.44:0/4157018394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822146710 msgr2=0x564822150830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:40.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.330+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/4157018394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822146710 0x564822150830 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009860 tx=0x7faf1c003e10 comp rx=0 tx=0).stop 2023-09-10T00:50:40.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.330+0000 7faf3ea14b80 1 -- 172.21.15.44:0/4157018394 shutdown_connections 2023-09-10T00:50:40.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.330+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/4157018394 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564822160930 0x564822162d50 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.330+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/4157018394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822150e40 0x56482214e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.330+0000 7faf0b7fe700 0 client.26908 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:40.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.330+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/4157018394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822146710 0x564822150830 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.330+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/4157018394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822147030 0x56482214db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.330+0000 7faf3ea14b80 1 -- 172.21.15.44:0/4157018394 >> 172.21.15.44:0/4157018394 conn(0x5648220bb090 msgr2=0x5648221417e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:40.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.330+0000 7faf3ea14b80 1 -- 172.21.15.44:0/4157018394 shutdown_connections 2023-09-10T00:50:40.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.330+0000 7faf3ea14b80 1 -- 172.21.15.44:0/4157018394 wait complete. 2023-09-10T00:50:40.351 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.331+0000 7faf3ea14b80 20 client.26908 trim_cache size 0 max 16384 2023-09-10T00:50:40.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.356+0000 7faf3ea14b80 1 Processor -- start 2023-09-10T00:50:40.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.356+0000 7faf3ea14b80 1 -- start start 2023-09-10T00:50:40.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.356+0000 7faf3ea14b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822146710 0x564822153230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.356+0000 7faf3ea14b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822147030 0x5648221503d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.356+0000 7faf3ea14b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822150e40 0x564822148e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.356+0000 7faf3ea14b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482214c130 con 0x564822150e40 2023-09-10T00:50:40.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.356+0000 7faf3ea14b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648221426e0 con 0x564822146710 2023-09-10T00:50:40.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.356+0000 7faf3ea14b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564822162f70 con 0x564822147030 2023-09-10T00:50:40.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.356+0000 7faf30deb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822150e40 0x564822148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.356+0000 7faf2bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822146710 0x564822153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.356+0000 7faf2b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822147030 0x5648221503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.356+0000 7faf2bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822146710 0x564822153230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54090/0 (socket says 172.21.15.44:54090) 2023-09-10T00:50:40.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.356+0000 7faf30deb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822150e40 0x564822148e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52352/0 (socket says 172.21.15.44:52352) 2023-09-10T00:50:40.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.356+0000 7faf2bfff700 1 -- 172.21.15.44:0/3607369506 learned_addr learned my addr 172.21.15.44:0/3607369506 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:40.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.357+0000 7faf2b7fe700 1 -- 172.21.15.44:0/3607369506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822146710 msgr2=0x564822153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:40.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.357+0000 7faf2b7fe700 1 --2- 172.21.15.44:0/3607369506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822146710 0x564822153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.357+0000 7faf2b7fe700 1 -- 172.21.15.44:0/3607369506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822150e40 msgr2=0x564822148e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:40.363 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.357+0000 7faf2b7fe700 1 --2- 172.21.15.44:0/3607369506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822150e40 0x564822148e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.357+0000 7faf2b7fe700 1 -- 172.21.15.44:0/3607369506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564821eadc60 con 0x564822147030 2023-09-10T00:50:40.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.357+0000 7faf2b7fe700 1 --2- 172.21.15.44:0/3607369506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822147030 0x5648221503d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faf200962a0 tx=0x7faf20004620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21973396951b92c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:40.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.357+0000 7faf12ffd700 1 -- 172.21.15.44:0/3607369506 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20099830 con 0x564822147030 2023-09-10T00:50:40.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.357+0000 7faf12ffd700 1 -- 172.21.15.44:0/3607369506 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf2009d660 con 0x564822147030 2023-09-10T00:50:40.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.357+0000 7faf12ffd700 1 -- 172.21.15.44:0/3607369506 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20004200 con 0x564822147030 2023-09-10T00:50:40.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.358+0000 7faf3ea14b80 1 -- 172.21.15.44:0/3607369506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822147030 msgr2=0x5648221503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:40.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.358+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/3607369506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822147030 0x5648221503d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faf200962a0 tx=0x7faf20004620 comp rx=0 tx=0).stop 2023-09-10T00:50:40.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.358+0000 7faf3ea14b80 1 -- 172.21.15.44:0/3607369506 shutdown_connections 2023-09-10T00:50:40.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.358+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/3607369506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822150e40 0x564822148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.358+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/3607369506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822146710 0x564822153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.365 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.358+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/3607369506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822147030 0x5648221503d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.358+0000 7faf3ea14b80 1 -- 172.21.15.44:0/3607369506 >> 172.21.15.44:0/3607369506 conn(0x5648220bb090 msgr2=0x56482215ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:40.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.359+0000 7faf3ea14b80 1 -- 172.21.15.44:0/3607369506 shutdown_connections 2023-09-10T00:50:40.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.359+0000 7faf3ea14b80 1 -- 172.21.15.44:0/3607369506 wait complete. 2023-09-10T00:50:40.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.360+0000 7faf3ea14b80 1 -- start start 2023-09-10T00:50:40.366 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.360+0000 7faf3ea14b80 1 Processor -- start 2023-09-10T00:50:40.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.360+0000 7faf3ea14b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822146710 0x5648221578e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.360+0000 7faf3ea14b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822147030 0x564822155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.360+0000 7faf3ea14b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822150e40 0x5648221560e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.360+0000 7faf3ea14b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648221b47c0 con 0x564822146710 2023-09-10T00:50:40.367 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.360+0000 7faf3ea14b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648221b4900 con 0x564822147030 2023-09-10T00:50:40.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.360+0000 7faf3ea14b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564822156620 con 0x564822150e40 2023-09-10T00:50:40.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.360+0000 7faf3ea14b80 10 client.0 _ll_register_callbacks cb 0x7faf20095450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:50:40.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.360+0000 7faf30deb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822150e40 0x5648221560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.360+0000 7faf30deb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822150e40 0x5648221560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59240/0 (socket says 172.21.15.44:59240) 2023-09-10T00:50:40.368 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.360+0000 7faf30deb700 1 -- 172.21.15.44:0/1220695409 learned_addr learned my addr 172.21.15.44:0/1220695409 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:40.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.361+0000 7faf2b7fe700 1 --2- 172.21.15.44:0/1220695409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822147030 0x564822155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.361+0000 7faf2bfff700 1 --2- 172.21.15.44:0/1220695409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822146710 0x5648221578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.361+0000 7faf30deb700 1 -- 172.21.15.44:0/1220695409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822147030 msgr2=0x564822155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:40.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.361+0000 7faf30deb700 1 --2- 172.21.15.44:0/1220695409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822147030 0x564822155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.369 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.361+0000 7faf30deb700 1 -- 172.21.15.44:0/1220695409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822146710 msgr2=0x5648221578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:40.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.361+0000 7faf30deb700 1 --2- 172.21.15.44:0/1220695409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822146710 0x5648221578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.361+0000 7faf30deb700 1 -- 172.21.15.44:0/1220695409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1c0095f0 con 0x564822150e40 2023-09-10T00:50:40.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.361+0000 7faf30deb700 1 --2- 172.21.15.44:0/1220695409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822150e40 0x5648221560e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faf24007cc0 tx=0x7faf240078c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8969cd26dccdd79 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:40.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.361+0000 7faf0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:50:40.370 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.361+0000 7faf0b7fe700 1 -- 172.21.15.44:0/1220695409 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf24020040 con 0x564822150e40 2023-09-10T00:50:40.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.361+0000 7faf0b7fe700 1 -- 172.21.15.44:0/1220695409 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9540+0+0 (secure 0 0 0) 0x7faf240093c0 con 0x564822150e40 2023-09-10T00:50:40.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.361+0000 7faf3ea14b80 1 -- 172.21.15.44:0/1220695409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564821ff04f0 con 0x564822150e40 2023-09-10T00:50:40.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.362+0000 7faf0b7fe700 1 -- 172.21.15.44:0/1220695409 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf2400e040 con 0x564822150e40 2023-09-10T00:50:40.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.362+0000 7faf3ea14b80 10 client.26914 fetch_fsmap learned FSMap version 59 2023-09-10T00:50:40.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.362+0000 7faf3ea14b80 1 -- 172.21.15.44:0/1220695409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x564821eeb250 con 0x564822150e40 2023-09-10T00:50:40.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.362+0000 7faf0b7fe700 1 -- 172.21.15.44:0/1220695409 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564821ff04f0 con 0x564822150e40 2023-09-10T00:50:40.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.362+0000 7faf0b7fe700 1 -- 172.21.15.44:0/1220695409 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 59) v1 ==== 7914+0+0 (secure 0 0 0) 0x7faf2400b050 con 0x564822150e40 2023-09-10T00:50:40.373 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.362+0000 7faf3ea14b80 10 client.26914 fetch_fsmap finished waiting for FSMap version 59 2023-09-10T00:50:40.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.362+0000 7faf3ea14b80 10 client.26914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:50:40.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.362+0000 7faf3ea14b80 20 client.26914 populate_metadata read hostname 'smithi044' 2023-09-10T00:50:40.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.362+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/1220695409 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5648221be630 0x5648221c0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:40.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.362+0000 7faf3ea14b80 4 client.26914 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:50:40.374 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.362+0000 7faf3ea14b80 1 -- 172.21.15.44:0/1220695409 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5648220520b0 con 0x5648221be630 2023-09-10T00:50:40.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.362+0000 7faf2bfff700 1 --2- 172.21.15.44:0/1220695409 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5648221be630 0x5648221c0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:40.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.363+0000 7faf0b7fe700 10 client.26914 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:40.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.363+0000 7faf2bfff700 1 --2- 172.21.15.44:0/1220695409 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5648221be630 0x5648221c0a50 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0de5317e4cd1625 server_cookie=f3bf79387064bffe in_seq=0 out_seq=0 2023-09-10T00:50:40.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.363+0000 7faf0b7fe700 1 -- 172.21.15.44:0/1220695409 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5648220520b0 con 0x5648221be630 2023-09-10T00:50:40.375 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.363+0000 7faf0b7fe700 10 client.26914 handle_command_reply: tid=0 2023-09-10T00:50:40.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.363+0000 7faf3ea14b80 1 client.26914 shutdown 2023-09-10T00:50:40.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.363+0000 7faf3ea14b80 1 -- 172.21.15.44:0/1220695409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822150e40 msgr2=0x5648221560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:40.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.363+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/1220695409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822150e40 0x5648221560e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faf24007cc0 tx=0x7faf240078c0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.363+0000 7faf3ea14b80 1 -- 172.21.15.44:0/1220695409 shutdown_connections 2023-09-10T00:50:40.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.364+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/1220695409 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5648221be630 0x5648221c0a50 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.376 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.364+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/1220695409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564822150e40 0x5648221560e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.364+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/1220695409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564822146710 0x5648221578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.364+0000 7faf0b7fe700 0 client.26914 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:40.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.364+0000 7faf3ea14b80 1 --2- 172.21.15.44:0/1220695409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564822147030 0x564822155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.377 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.364+0000 7faf3ea14b80 1 -- 172.21.15.44:0/1220695409 >> 172.21.15.44:0/1220695409 conn(0x5648220bb090 msgr2=0x564822148e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:40.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.364+0000 7faf3ea14b80 1 -- 172.21.15.44:0/1220695409 shutdown_connections 2023-09-10T00:50:40.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.364+0000 7faf3ea14b80 1 -- 172.21.15.44:0/1220695409 wait complete. 2023-09-10T00:50:40.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.365+0000 7faf3ea14b80 20 client.26914 trim_cache size 0 max 16384 2023-09-10T00:50:40.378 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:50:40.379 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:50:40.379 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:50:40.379 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:50:40.379 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:50:40.379 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:50:40.380 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:50:40.380 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:50:40.380 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:50:40.380 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:50:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.365+0000 7faf3284e700 1 -- 172.21.15.44:0/2697524193 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faf140a07b0 msgr2=0x7faf140a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.365+0000 7faf3284e700 1 --2- 172.21.15.44:0/2697524193 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faf140a07b0 0x7faf140a2c70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faf1c005fd0 tx=0x7faf1c009310 comp rx=0 tx=0).stop 2023-09-10T00:50:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.365+0000 7faf3284e700 1 -- 172.21.15.44:0/2697524193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf2c100550 msgr2=0x7faf2c195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:40.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.365+0000 7faf3284e700 1 --2- 172.21.15.44:0/2697524193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf2c100550 0x7faf2c195b60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7faf2000eb30 tx=0x7faf2000ee40 comp rx=0 tx=0).stop 2023-09-10T00:50:40.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.365+0000 7faf3284e700 1 -- 172.21.15.44:0/2697524193 shutdown_connections 2023-09-10T00:50:40.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.365+0000 7faf3284e700 1 --2- 172.21.15.44:0/2697524193 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faf140a07b0 0x7faf140a2c70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.365+0000 7faf3284e700 1 --2- 172.21.15.44:0/2697524193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c1044c0 0x7faf2c199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.365+0000 7faf3284e700 1 --2- 172.21.15.44:0/2697524193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0ffb90 0x7faf2c195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.383 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.365+0000 7faf3284e700 1 --2- 172.21.15.44:0/2697524193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf2c100550 0x7faf2c195b60 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:40.383 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.365+0000 7faf3284e700 1 -- 172.21.15.44:0/2697524193 >> 172.21.15.44:0/2697524193 conn(0x7faf2c076a70 msgr2=0x7faf2c102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:40.383 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.365+0000 7faf3284e700 1 -- 172.21.15.44:0/2697524193 shutdown_connections 2023-09-10T00:50:40.383 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:50:40.365+0000 7faf3284e700 1 -- 172.21.15.44:0/2697524193 wait complete. 2023-09-10T00:50:40.384 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:50:40.384 INFO:teuthology.orchestra.run.smithi044.stderr:38200000 of 50000000 tuples (76%) done (elapsed 580.02 s, remaining 179.17 s) 2023-09-10T00:50:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:40 smithi131 ceph-mon[103713]: pgmap v311: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 15 MiB/s wr, 39 op/s 2023-09-10T00:50:41.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:40 smithi044 ceph-mon[98676]: pgmap v311: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 15 MiB/s wr, 39 op/s 2023-09-10T00:50:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:40 smithi099 ceph-mon[102692]: pgmap v311: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 15 MiB/s wr, 39 op/s 2023-09-10T00:50:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:42 smithi131 ceph-mon[103713]: pgmap v312: 129 pgs: 129 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 26 MiB/s wr, 84 op/s 2023-09-10T00:50:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:42 smithi044 ceph-mon[98676]: pgmap v312: 129 pgs: 129 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 26 MiB/s wr, 84 op/s 2023-09-10T00:50:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:42 smithi099 ceph-mon[102692]: pgmap v312: 129 pgs: 129 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 26 MiB/s wr, 84 op/s 2023-09-10T00:50:44.111 INFO:teuthology.orchestra.run.smithi044.stderr:38300000 of 50000000 tuples (76%) done (elapsed 583.74 s, remaining 178.32 s) 2023-09-10T00:50:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:44 smithi131 ceph-mon[103713]: pgmap v313: 129 pgs: 129 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 21 MiB/s wr, 70 op/s 2023-09-10T00:50:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:44 smithi044 ceph-mon[98676]: pgmap v313: 129 pgs: 129 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 21 MiB/s wr, 70 op/s 2023-09-10T00:50:45.097 INFO:teuthology.orchestra.run.smithi044.stderr:38400000 of 50000000 tuples (76%) done (elapsed 584.73 s, remaining 176.64 s) 2023-09-10T00:50:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:44 smithi099 ceph-mon[102692]: pgmap v313: 129 pgs: 129 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 21 MiB/s wr, 70 op/s 2023-09-10T00:50:46.229 INFO:teuthology.orchestra.run.smithi044.stderr:38500000 of 50000000 tuples (77%) done (elapsed 585.86 s, remaining 175.00 s) 2023-09-10T00:50:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:46 smithi131 ceph-mon[103713]: pgmap v314: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 28 MiB/s wr, 104 op/s 2023-09-10T00:50:47.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:46 smithi044 ceph-mon[98676]: pgmap v314: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 28 MiB/s wr, 104 op/s 2023-09-10T00:50:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:46 smithi099 ceph-mon[102692]: pgmap v314: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 28 MiB/s wr, 104 op/s 2023-09-10T00:50:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.364+0000 7fbd9bfd7700 1 -- 172.21.15.44:0/379979219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd941055b0 msgr2=0x7fbd94105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.364+0000 7fbd9bfd7700 1 --2- 172.21.15.44:0/379979219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd941055b0 0x7fbd94105990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009990 tx=0x7fbd90009ca0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.364+0000 7fbd9bfd7700 1 -- 172.21.15.44:0/379979219 shutdown_connections 2023-09-10T00:50:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.364+0000 7fbd9bfd7700 1 --2- 172.21.15.44:0/379979219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd94100120 0x7fbd9410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.364+0000 7fbd9bfd7700 1 --2- 172.21.15.44:0/379979219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd941055b0 0x7fbd94105990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.364+0000 7fbd9bfd7700 1 --2- 172.21.15.44:0/379979219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd940ff760 0x7fbd940ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.364+0000 7fbd9bfd7700 1 -- 172.21.15.44:0/379979219 >> 172.21.15.44:0/379979219 conn(0x7fbd94074f30 msgr2=0x7fbd94075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.365+0000 7fbd9bfd7700 1 -- 172.21.15.44:0/379979219 shutdown_connections 2023-09-10T00:50:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.365+0000 7fbd9bfd7700 1 -- 172.21.15.44:0/379979219 wait complete. 2023-09-10T00:50:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.365+0000 7fbd9bfd7700 1 Processor -- start 2023-09-10T00:50:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.365+0000 7fbd9bfd7700 1 -- start start 2023-09-10T00:50:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.365+0000 7fbd9bfd7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd940ff760 0x7fbd941975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.365+0000 7fbd9bfd7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd94100120 0x7fbd94197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.365+0000 7fbd9bfd7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd941055b0 0x7fbd941a2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.365+0000 7fbd9bfd7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd941a2a40 con 0x7fbd94100120 2023-09-10T00:50:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.365+0000 7fbd9bfd7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd941a2bb0 con 0x7fbd940ff760 2023-09-10T00:50:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.365+0000 7fbd9bfd7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd941a2d70 con 0x7fbd941055b0 2023-09-10T00:50:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd99d73700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd940ff760 0x7fbd941975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd99572700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd94100120 0x7fbd94197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd99d73700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd940ff760 0x7fbd941975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54108/0 (socket says 172.21.15.44:54108) 2023-09-10T00:50:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd99d73700 1 -- 172.21.15.44:0/1382676399 learned_addr learned my addr 172.21.15.44:0/1382676399 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd9a574700 1 --2- 172.21.15.44:0/1382676399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd941055b0 0x7fbd941a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd99d73700 1 -- 172.21.15.44:0/1382676399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd941055b0 msgr2=0x7fbd941a2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd99d73700 1 --2- 172.21.15.44:0/1382676399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd941055b0 0x7fbd941a2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd99d73700 1 -- 172.21.15.44:0/1382676399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd94100120 msgr2=0x7fbd94197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd99d73700 1 --2- 172.21.15.44:0/1382676399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd94100120 0x7fbd94197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd99d73700 1 -- 172.21.15.44:0/1382676399 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c009c40 con 0x7fbd940ff760 2023-09-10T00:50:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd99572700 1 --2- 172.21.15.44:0/1382676399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd94100120 0x7fbd94197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd99d73700 1 --2- 172.21.15.44:0/1382676399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd940ff760 0x7fbd941975d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbd90000c00 tx=0x7fbd9000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd86ffd700 1 -- 172.21.15.44:0/1382676399 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9001d070 con 0x7fbd940ff760 2023-09-10T00:50:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd9bfd7700 1 -- 172.21.15.44:0/1382676399 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd900095f0 con 0x7fbd940ff760 2023-09-10T00:50:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd86ffd700 1 -- 172.21.15.44:0/1382676399 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd90021410 con 0x7fbd940ff760 2023-09-10T00:50:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd86ffd700 1 -- 172.21.15.44:0/1382676399 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90017650 con 0x7fbd940ff760 2023-09-10T00:50:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.366+0000 7fbd9bfd7700 1 -- 172.21.15.44:0/1382676399 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd941a3230 con 0x7fbd940ff760 2023-09-10T00:50:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.369+0000 7fbd86ffd700 1 -- 172.21.15.44:0/1382676399 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fbd90021580 con 0x7fbd940ff760 2023-09-10T00:50:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.369+0000 7fbd86ffd700 1 --2- 172.21.15.44:0/1382676399 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbd800a07b0 0x7fbd800a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.369+0000 7fbd99572700 1 --2- 172.21.15.44:0/1382676399 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbd800a07b0 0x7fbd800a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.369+0000 7fbd86ffd700 1 -- 172.21.15.44:0/1382676399 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fbd900cd740 con 0x7fbd940ff760 2023-09-10T00:50:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.370+0000 7fbda819db80 1 Processor -- start 2023-09-10T00:50:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.370+0000 7fbd99572700 1 --2- 172.21.15.44:0/1382676399 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbd800a07b0 0x7fbd800a2c70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbd940684b0 tx=0x7fbd88009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.370+0000 7fbda819db80 1 -- start start 2023-09-10T00:50:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.370+0000 7fbda819db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc03c1a0 0x5645fc03c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.370+0000 7fbda819db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc036300 0x5645fc036780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.370+0000 7fbda819db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc042f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.370+0000 7fbda819db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fbf8f6d0 con 0x5645fc036cc0 2023-09-10T00:50:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.370+0000 7fbda819db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fbf8cff0 con 0x5645fc03c1a0 2023-09-10T00:50:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.370+0000 7fbda819db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fbf888e0 con 0x5645fc036300 2023-09-10T00:50:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.371+0000 7fbd99d73700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc03c1a0 0x5645fc03c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.371+0000 7fbd99572700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc036300 0x5645fc036780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.371+0000 7fbd9a574700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc042f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.371+0000 7fbd99d73700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc03c1a0 0x5645fc03c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54116/0 (socket says 172.21.15.44:54116) 2023-09-10T00:50:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.371+0000 7fbd99d73700 1 -- 172.21.15.44:0/224918951 learned_addr learned my addr 172.21.15.44:0/224918951 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.371+0000 7fbd99572700 1 -- 172.21.15.44:0/224918951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc03c1a0 msgr2=0x5645fc03c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.371+0000 7fbd99572700 1 --2- 172.21.15.44:0/224918951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc03c1a0 0x5645fc03c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.371+0000 7fbd99572700 1 -- 172.21.15.44:0/224918951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 msgr2=0x5645fc042f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.371+0000 7fbd99572700 1 --2- 172.21.15.44:0/224918951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc042f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.371+0000 7fbd99572700 1 -- 172.21.15.44:0/224918951 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645fbdb4d80 con 0x5645fc036300 2023-09-10T00:50:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.371+0000 7fbd9a574700 1 --2- 172.21.15.44:0/224918951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc042f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.371+0000 7fbd99d73700 1 --2- 172.21.15.44:0/224918951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc03c1a0 0x5645fc03c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.371+0000 7fbd99572700 1 --2- 172.21.15.44:0/224918951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc036300 0x5645fc036780 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbd880042f0 tx=0x7fbd88014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63bca73bde1d783f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.371+0000 7fbd7ffff700 1 -- 172.21.15.44:0/224918951 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd88003d10 con 0x5645fc036300 2023-09-10T00:50:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.371+0000 7fbd7ffff700 1 -- 172.21.15.44:0/224918951 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd88003e70 con 0x5645fc036300 2023-09-10T00:50:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.371+0000 7fbd7ffff700 1 -- 172.21.15.44:0/224918951 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8801e650 con 0x5645fc036300 2023-09-10T00:50:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.372+0000 7fbda819db80 1 -- 172.21.15.44:0/224918951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc036300 msgr2=0x5645fc036780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.372+0000 7fbda819db80 1 --2- 172.21.15.44:0/224918951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc036300 0x5645fc036780 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbd880042f0 tx=0x7fbd88014000 comp rx=0 tx=0).stop 2023-09-10T00:50:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.372+0000 7fbda819db80 1 -- 172.21.15.44:0/224918951 shutdown_connections 2023-09-10T00:50:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.372+0000 7fbda819db80 1 --2- 172.21.15.44:0/224918951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc042f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.372+0000 7fbda819db80 1 --2- 172.21.15.44:0/224918951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc03c1a0 0x5645fc03c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.372+0000 7fbda819db80 1 --2- 172.21.15.44:0/224918951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc036300 0x5645fc036780 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.372+0000 7fbda819db80 1 -- 172.21.15.44:0/224918951 >> 172.21.15.44:0/224918951 conn(0x5645fbfab030 msgr2=0x5645fc031560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.372+0000 7fbda819db80 1 -- 172.21.15.44:0/224918951 shutdown_connections 2023-09-10T00:50:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.372+0000 7fbda819db80 1 -- 172.21.15.44:0/224918951 wait complete. 2023-09-10T00:50:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.372+0000 7fbda819db80 1 -- start start 2023-09-10T00:50:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.373+0000 7fbda819db80 1 Processor -- start 2023-09-10T00:50:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.373+0000 7fbda819db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036300 0x5645fc042ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.373+0000 7fbda819db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc036cc0 0x5645fc040190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.373+0000 7fbda819db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc03c1a0 0x5645fc0408d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.373+0000 7fbda819db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fbf26b10 con 0x5645fc036300 2023-09-10T00:50:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.373+0000 7fbda819db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fbf26810 con 0x5645fc03c1a0 2023-09-10T00:50:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.373+0000 7fbda819db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fbe0a8c0 con 0x5645fc036cc0 2023-09-10T00:50:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.373+0000 7fbda819db80 10 client.0 _ll_register_callbacks cb 0x5645fbe70a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:50:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.373+0000 7fbd99d73700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036300 0x5645fc042ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.373+0000 7fbd99d73700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036300 0x5645fc042ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52382/0 (socket says 172.21.15.44:52382) 2023-09-10T00:50:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.373+0000 7fbd99d73700 1 -- 172.21.15.44:0/1894232351 learned_addr learned my addr 172.21.15.44:0/1894232351 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.373+0000 7fbd9a574700 1 --2- 172.21.15.44:0/1894232351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc03c1a0 0x5645fc0408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbd99572700 1 --2- 172.21.15.44:0/1894232351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc036cc0 0x5645fc040190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbd99d73700 1 -- 172.21.15.44:0/1894232351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc036cc0 msgr2=0x5645fc040190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbd99d73700 1 --2- 172.21.15.44:0/1894232351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc036cc0 0x5645fc040190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbd99d73700 1 -- 172.21.15.44:0/1894232351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc03c1a0 msgr2=0x5645fc0408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbd99d73700 1 --2- 172.21.15.44:0/1894232351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc03c1a0 0x5645fc0408d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbd99d73700 1 -- 172.21.15.44:0/1894232351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8c00e330 con 0x5645fc036300 2023-09-10T00:50:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbd99572700 1 --2- 172.21.15.44:0/1894232351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc036cc0 0x5645fc040190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:50:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbd9a574700 1 --2- 172.21.15.44:0/1894232351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc03c1a0 0x5645fc0408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbd99d73700 1 --2- 172.21.15.44:0/1894232351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036300 0x5645fc042ea0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fbd90005910 tx=0x7fbd9009a260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e502c8be9f93746 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbd7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:50:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbd7d7fa700 1 -- 172.21.15.44:0/1894232351 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9009ace0 con 0x5645fc036300 2023-09-10T00:50:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbd7d7fa700 1 -- 172.21.15.44:0/1894232351 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fbd9009a880 con 0x5645fc036300 2023-09-10T00:50:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbda819db80 1 -- 172.21.15.44:0/1894232351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645fbee0490 con 0x5645fc036300 2023-09-10T00:50:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbd7d7fa700 1 -- 172.21.15.44:0/1894232351 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd9009e6b0 con 0x5645fc036300 2023-09-10T00:50:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbda819db80 10 client.17319 fetch_fsmap learned FSMap version 59 2023-09-10T00:50:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbda819db80 1 -- 172.21.15.44:0/1894232351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7fbd941a3230 con 0x5645fc036300 2023-09-10T00:50:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.374+0000 7fbd7d7fa700 1 -- 172.21.15.44:0/1894232351 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd90064020 con 0x5645fc036300 2023-09-10T00:50:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.375+0000 7fbd7d7fa700 1 -- 172.21.15.44:0/1894232351 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 59) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fbd900a23d0 con 0x5645fc036300 2023-09-10T00:50:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.375+0000 7fbda819db80 10 client.17319 fetch_fsmap finished waiting for FSMap version 59 2023-09-10T00:50:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.375+0000 7fbda819db80 10 client.17319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:50:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.375+0000 7fbda819db80 20 client.17319 populate_metadata read hostname 'smithi044' 2023-09-10T00:50:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.375+0000 7fbda819db80 1 --2- 172.21.15.44:0/1894232351 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5645fc0501d0 0x5645fc0525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.375+0000 7fbda819db80 4 client.17319 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:50:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.375+0000 7fbda819db80 1 -- 172.21.15.44:0/1894232351 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645fbf9d840 con 0x5645fc0501d0 2023-09-10T00:50:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.375+0000 7fbd99572700 1 --2- 172.21.15.44:0/1894232351 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5645fc0501d0 0x5645fc0525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.375+0000 7fbd7d7fa700 10 client.17319 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.375+0000 7fbd99572700 1 --2- 172.21.15.44:0/1894232351 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5645fc0501d0 0x5645fc0525f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3f6075dcae5aee6 server_cookie=e642cd5cbe697090 in_seq=0 out_seq=0 2023-09-10T00:50:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.376+0000 7fbd7d7fa700 1 -- 172.21.15.44:0/1894232351 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5645fbf9d840 con 0x5645fc0501d0 2023-09-10T00:50:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.376+0000 7fbd7d7fa700 10 client.17319 handle_command_reply: tid=0 2023-09-10T00:50:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.376+0000 7fbda819db80 1 client.17319 shutdown 2023-09-10T00:50:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.376+0000 7fbda819db80 1 -- 172.21.15.44:0/1894232351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036300 msgr2=0x5645fc042ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.376+0000 7fbda819db80 1 --2- 172.21.15.44:0/1894232351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036300 0x5645fc042ea0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fbd90005910 tx=0x7fbd9009a260 comp rx=0 tx=0).stop 2023-09-10T00:50:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.376+0000 7fbda819db80 1 -- 172.21.15.44:0/1894232351 shutdown_connections 2023-09-10T00:50:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.376+0000 7fbda819db80 1 --2- 172.21.15.44:0/1894232351 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5645fc0501d0 0x5645fc0525f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.376+0000 7fbda819db80 1 --2- 172.21.15.44:0/1894232351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc03c1a0 0x5645fc0408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.376+0000 7fbda819db80 1 --2- 172.21.15.44:0/1894232351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036300 0x5645fc042ea0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.376+0000 7fbd7d7fa700 0 client.17319 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.376+0000 7fbda819db80 1 --2- 172.21.15.44:0/1894232351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc036cc0 0x5645fc040190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.376+0000 7fbda819db80 1 -- 172.21.15.44:0/1894232351 >> 172.21.15.44:0/1894232351 conn(0x5645fbfab030 msgr2=0x5645fc0307a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.376+0000 7fbda819db80 1 -- 172.21.15.44:0/1894232351 shutdown_connections 2023-09-10T00:50:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.376+0000 7fbda819db80 1 -- 172.21.15.44:0/1894232351 wait complete. 2023-09-10T00:50:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.377+0000 7fbda819db80 20 client.17319 trim_cache size 0 max 16384 2023-09-10T00:50:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbda819db80 1 Processor -- start 2023-09-10T00:50:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbda819db80 1 -- start start 2023-09-10T00:50:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbda819db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc036300 0x5645fc03dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbda819db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc032930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbda819db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc03c1a0 0x5645fc042a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbda819db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fc041140 con 0x5645fc036cc0 2023-09-10T00:50:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbda819db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fc0416d0 con 0x5645fc036300 2023-09-10T00:50:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbda819db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fc03aeb0 con 0x5645fc03c1a0 2023-09-10T00:50:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbd99572700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc032930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbd99d73700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc036300 0x5645fc03dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbd99d73700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc036300 0x5645fc03dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54130/0 (socket says 172.21.15.44:54130) 2023-09-10T00:50:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbd99572700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc032930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52392/0 (socket says 172.21.15.44:52392) 2023-09-10T00:50:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbd99d73700 1 -- 172.21.15.44:0/2815642614 learned_addr learned my addr 172.21.15.44:0/2815642614 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbd9a574700 1 --2- 172.21.15.44:0/2815642614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc03c1a0 0x5645fc042a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbd99d73700 1 -- 172.21.15.44:0/2815642614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc03c1a0 msgr2=0x5645fc042a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbd99d73700 1 --2- 172.21.15.44:0/2815642614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc03c1a0 0x5645fc042a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbd99d73700 1 -- 172.21.15.44:0/2815642614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 msgr2=0x5645fc032930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbd99d73700 1 --2- 172.21.15.44:0/2815642614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc032930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbd99d73700 1 -- 172.21.15.44:0/2815642614 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd88017070 con 0x5645fc036300 2023-09-10T00:50:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.392+0000 7fbd99572700 1 --2- 172.21.15.44:0/2815642614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc032930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.393+0000 7fbd99d73700 1 --2- 172.21.15.44:0/2815642614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc036300 0x5645fc03dd80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbd900cab40 tx=0x7fbd900092b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f604e4ca9e819086 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.393+0000 7fbd84ff9700 1 -- 172.21.15.44:0/2815642614 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90096bc0 con 0x5645fc036300 2023-09-10T00:50:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.393+0000 7fbd84ff9700 1 -- 172.21.15.44:0/2815642614 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd90064070 con 0x5645fc036300 2023-09-10T00:50:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.393+0000 7fbd84ff9700 1 -- 172.21.15.44:0/2815642614 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90099f70 con 0x5645fc036300 2023-09-10T00:50:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.393+0000 7fbda819db80 1 -- 172.21.15.44:0/2815642614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc036300 msgr2=0x5645fc03dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.393+0000 7fbda819db80 1 --2- 172.21.15.44:0/2815642614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc036300 0x5645fc03dd80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbd900cab40 tx=0x7fbd900092b0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.393+0000 7fbda819db80 1 -- 172.21.15.44:0/2815642614 shutdown_connections 2023-09-10T00:50:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.393+0000 7fbda819db80 1 --2- 172.21.15.44:0/2815642614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc03c1a0 0x5645fc042a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.393+0000 7fbda819db80 1 --2- 172.21.15.44:0/2815642614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc036300 0x5645fc03dd80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.393+0000 7fbda819db80 1 --2- 172.21.15.44:0/2815642614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc032930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.393+0000 7fbda819db80 1 -- 172.21.15.44:0/2815642614 >> 172.21.15.44:0/2815642614 conn(0x5645fbfab030 msgr2=0x5645fc04c760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.393+0000 7fbda819db80 1 -- 172.21.15.44:0/2815642614 shutdown_connections 2023-09-10T00:50:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.393+0000 7fbda819db80 1 -- 172.21.15.44:0/2815642614 wait complete. 2023-09-10T00:50:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.394+0000 7fbda819db80 1 -- start start 2023-09-10T00:50:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.394+0000 7fbda819db80 1 Processor -- start 2023-09-10T00:50:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.394+0000 7fbda819db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc036300 0x5645fc047720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.394+0000 7fbda819db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc045510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.394+0000 7fbda819db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc03c1a0 0x5645fc045b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.394+0000 7fbda819db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fc0461e0 con 0x5645fc036cc0 2023-09-10T00:50:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.394+0000 7fbda819db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fc046350 con 0x5645fc036300 2023-09-10T00:50:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.394+0000 7fbda819db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fc0a73d0 con 0x5645fc03c1a0 2023-09-10T00:50:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.394+0000 7fbda819db80 10 client.0 _ll_register_callbacks cb 0x7fbd9009a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:50:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.394+0000 7fbd99572700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc045510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.394+0000 7fbd99d73700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc036300 0x5645fc047720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.394+0000 7fbd99572700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc045510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52398/0 (socket says 172.21.15.44:52398) 2023-09-10T00:50:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.394+0000 7fbd99572700 1 -- 172.21.15.44:0/485067228 learned_addr learned my addr 172.21.15.44:0/485067228 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:50:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.394+0000 7fbd9a574700 1 --2- 172.21.15.44:0/485067228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc03c1a0 0x5645fc045b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.395+0000 7fbd99d73700 1 -- 172.21.15.44:0/485067228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc03c1a0 msgr2=0x5645fc045b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.395+0000 7fbd99d73700 1 --2- 172.21.15.44:0/485067228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc03c1a0 0x5645fc045b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.395+0000 7fbd99d73700 1 -- 172.21.15.44:0/485067228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 msgr2=0x5645fc045510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:50:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.395+0000 7fbd99d73700 1 --2- 172.21.15.44:0/485067228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc045510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.395+0000 7fbd99d73700 1 -- 172.21.15.44:0/485067228 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645fbdb4d80 con 0x5645fc036300 2023-09-10T00:50:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.395+0000 7fbd99572700 1 --2- 172.21.15.44:0/485067228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc045510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:50:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.395+0000 7fbd99d73700 1 --2- 172.21.15.44:0/485067228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc036300 0x5645fc047720 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbd90005910 tx=0x7fbd9009a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f9d80b1bca2860 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:50:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.395+0000 7fbd7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:50:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.395+0000 7fbd7d7fa700 1 -- 172.21.15.44:0/485067228 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9009ace0 con 0x5645fc036300 2023-09-10T00:50:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.395+0000 7fbd7d7fa700 1 -- 172.21.15.44:0/485067228 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fbd90065080 con 0x5645fc036300 2023-09-10T00:50:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.395+0000 7fbda819db80 1 -- 172.21.15.44:0/485067228 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd88003e70 con 0x5645fc036300 2023-09-10T00:50:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.395+0000 7fbd7d7fa700 1 -- 172.21.15.44:0/485067228 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd9009a880 con 0x5645fc036300 2023-09-10T00:50:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.395+0000 7fbda819db80 10 client.26990 fetch_fsmap learned FSMap version 59 2023-09-10T00:50:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.395+0000 7fbda819db80 1 -- 172.21.15.44:0/485067228 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7fbd941a3230 con 0x5645fc036300 2023-09-10T00:50:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.395+0000 7fbd7d7fa700 1 -- 172.21.15.44:0/485067228 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd88003e70 con 0x5645fc036300 2023-09-10T00:50:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.395+0000 7fbd7d7fa700 1 -- 172.21.15.44:0/485067228 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 59) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fbd90066430 con 0x5645fc036300 2023-09-10T00:50:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.396+0000 7fbda819db80 10 client.26990 fetch_fsmap finished waiting for FSMap version 59 2023-09-10T00:50:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.396+0000 7fbda819db80 10 client.26990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:50:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.396+0000 7fbda819db80 20 client.26990 populate_metadata read hostname 'smithi044' 2023-09-10T00:50:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.396+0000 7fbda819db80 1 --2- 172.21.15.44:0/485067228 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5645fc0a87a0 0x5645fc0aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:50:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.396+0000 7fbda819db80 4 client.26990 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:50:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.396+0000 7fbda819db80 1 -- 172.21.15.44:0/485067228 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5645fbf818c0 con 0x5645fc0a87a0 2023-09-10T00:50:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.396+0000 7fbd99572700 1 --2- 172.21.15.44:0/485067228 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5645fc0a87a0 0x5645fc0aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:50:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.396+0000 7fbd99572700 1 --2- 172.21.15.44:0/485067228 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5645fc0a87a0 0x5645fc0aabc0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86ac239283f89588 server_cookie=4fa8748c3a56acc6 in_seq=0 out_seq=0 2023-09-10T00:50:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.396+0000 7fbd7d7fa700 10 client.26990 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.412+0000 7fbd7d7fa700 1 -- 172.21.15.44:0/485067228 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5645fbf818c0 con 0x5645fc0a87a0 2023-09-10T00:50:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.412+0000 7fbd7d7fa700 10 client.26990 handle_command_reply: tid=0 2023-09-10T00:50:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.412+0000 7fbda819db80 1 client.26990 shutdown 2023-09-10T00:50:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.413+0000 7fbda819db80 1 -- 172.21.15.44:0/485067228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc036300 msgr2=0x5645fc047720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.413+0000 7fbda819db80 1 --2- 172.21.15.44:0/485067228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc036300 0x5645fc047720 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbd90005910 tx=0x7fbd9009a100 comp rx=0 tx=0).stop 2023-09-10T00:50:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.413+0000 7fbda819db80 1 -- 172.21.15.44:0/485067228 shutdown_connections 2023-09-10T00:50:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.413+0000 7fbda819db80 1 --2- 172.21.15.44:0/485067228 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5645fc0a87a0 0x5645fc0aabc0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.413+0000 7fbda819db80 1 --2- 172.21.15.44:0/485067228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5645fc03c1a0 0x5645fc045b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.413+0000 7fbda819db80 1 --2- 172.21.15.44:0/485067228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5645fc036300 0x5645fc047720 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.413+0000 7fbd7d7fa700 0 client.26990 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:50:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.413+0000 7fbda819db80 1 --2- 172.21.15.44:0/485067228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645fc036cc0 0x5645fc045510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.413+0000 7fbda819db80 1 -- 172.21.15.44:0/485067228 >> 172.21.15.44:0/485067228 conn(0x5645fbfab030 msgr2=0x5645fc03a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.413+0000 7fbda819db80 1 -- 172.21.15.44:0/485067228 shutdown_connections 2023-09-10T00:50:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.413+0000 7fbda819db80 1 -- 172.21.15.44:0/485067228 wait complete. 2023-09-10T00:50:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.413+0000 7fbda819db80 20 client.26990 trim_cache size 0 max 16384 2023-09-10T00:50:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:50:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:50:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:50:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:50:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.413+0000 7fbd9bfd7700 1 -- 172.21.15.44:0/1382676399 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbd800a07b0 msgr2=0x7fbd800a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.413+0000 7fbd9bfd7700 1 --2- 172.21.15.44:0/1382676399 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbd800a07b0 0x7fbd800a2c70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbd940684b0 tx=0x7fbd88009260 comp rx=0 tx=0).stop 2023-09-10T00:50:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.413+0000 7fbd9bfd7700 1 -- 172.21.15.44:0/1382676399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd940ff760 msgr2=0x7fbd941975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:50:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.413+0000 7fbd9bfd7700 1 --2- 172.21.15.44:0/1382676399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd940ff760 0x7fbd941975d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbd90000c00 tx=0x7fbd9000bfd0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.414+0000 7fbd9bfd7700 1 -- 172.21.15.44:0/1382676399 shutdown_connections 2023-09-10T00:50:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.414+0000 7fbd9bfd7700 1 --2- 172.21.15.44:0/1382676399 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbd800a07b0 0x7fbd800a2c70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.414+0000 7fbd9bfd7700 1 --2- 172.21.15.44:0/1382676399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd941055b0 0x7fbd941a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.414+0000 7fbd9bfd7700 1 --2- 172.21.15.44:0/1382676399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd940ff760 0x7fbd941975d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.414+0000 7fbd9bfd7700 1 --2- 172.21.15.44:0/1382676399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd94100120 0x7fbd94197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:50:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.414+0000 7fbd9bfd7700 1 -- 172.21.15.44:0/1382676399 >> 172.21.15.44:0/1382676399 conn(0x7fbd94074f30 msgr2=0x7fbd940fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:50:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.414+0000 7fbd9bfd7700 1 -- 172.21.15.44:0/1382676399 shutdown_connections 2023-09-10T00:50:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:50:48.414+0000 7fbd9bfd7700 1 -- 172.21.15.44:0/1382676399 wait complete. 2023-09-10T00:50:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:48 smithi131 ceph-mon[103713]: pgmap v315: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 77 op/s 2023-09-10T00:50:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:48 smithi044 ceph-mon[98676]: pgmap v315: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 77 op/s 2023-09-10T00:50:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:48 smithi099 ceph-mon[102692]: pgmap v315: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 77 op/s 2023-09-10T00:50:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:50 smithi131 ceph-mon[103713]: pgmap v316: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 77 op/s 2023-09-10T00:50:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:50 smithi044 ceph-mon[98676]: pgmap v316: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 77 op/s 2023-09-10T00:50:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:50 smithi099 ceph-mon[102692]: pgmap v316: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 77 op/s 2023-09-10T00:50:52.054 INFO:teuthology.orchestra.run.smithi044.stderr:38600000 of 50000000 tuples (77%) done (elapsed 591.69 s, remaining 174.75 s) 2023-09-10T00:50:52.905 INFO:teuthology.orchestra.run.smithi044.stderr:38700000 of 50000000 tuples (77%) done (elapsed 592.54 s, remaining 173.02 s) 2023-09-10T00:50:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:52 smithi131 ceph-mon[103713]: pgmap v317: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 23 MiB/s wr, 103 op/s 2023-09-10T00:50:53.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:52 smithi044 ceph-mon[98676]: pgmap v317: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 23 MiB/s wr, 103 op/s 2023-09-10T00:50:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:52 smithi099 ceph-mon[102692]: pgmap v317: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 23 MiB/s wr, 103 op/s 2023-09-10T00:50:55.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:54 smithi131 ceph-mon[103713]: pgmap v318: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 12 MiB/s wr, 59 op/s 2023-09-10T00:50:55.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:54 smithi044 ceph-mon[98676]: pgmap v318: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 12 MiB/s wr, 59 op/s 2023-09-10T00:50:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:54 smithi099 ceph-mon[102692]: pgmap v318: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 12 MiB/s wr, 59 op/s 2023-09-10T00:50:56.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:55 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:50:56.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:55 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:50:56.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:55 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:50:57.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:56 smithi131 ceph-mon[103713]: pgmap v319: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 17 MiB/s wr, 68 op/s 2023-09-10T00:50:57.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:56 smithi044 ceph-mon[98676]: pgmap v319: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 17 MiB/s wr, 68 op/s 2023-09-10T00:50:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:56 smithi099 ceph-mon[102692]: pgmap v319: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 17 MiB/s wr, 68 op/s 2023-09-10T00:50:57.592 INFO:teuthology.orchestra.run.smithi044.stderr:38800000 of 50000000 tuples (77%) done (elapsed 597.23 s, remaining 172.40 s) 2023-09-10T00:50:58.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:57 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:50:58.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:57 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:50:58.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:57 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:50:59.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:50:58 smithi131 ceph-mon[103713]: pgmap v320: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.7 MiB/s wr, 34 op/s 2023-09-10T00:50:59.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:50:58 smithi044 ceph-mon[98676]: pgmap v320: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.7 MiB/s wr, 34 op/s 2023-09-10T00:50:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:50:58 smithi099 ceph-mon[102692]: pgmap v320: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.7 MiB/s wr, 34 op/s 2023-09-10T00:51:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:00 smithi131 ceph-mon[103713]: pgmap v321: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.7 MiB/s wr, 34 op/s 2023-09-10T00:51:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:00 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:01.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:00 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:01.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:00 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:01.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:00 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:01.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:00 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:00 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:00 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:00 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:00 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:00 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:00 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:00 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:00 smithi044 ceph-mon[98676]: pgmap v321: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.7 MiB/s wr, 34 op/s 2023-09-10T00:51:01.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:00 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:01.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:00 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:01.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:00 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:01.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:00 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:01.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:00 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:00 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:00 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:00 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:00 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:00 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:00 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:00 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:00 smithi099 ceph-mon[102692]: pgmap v321: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.7 MiB/s wr, 34 op/s 2023-09-10T00:51:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:00 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:01.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:00 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:01.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:00 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:01.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:00 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:01.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:00 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:00 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:00 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:00 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:00 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:00 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:00 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:00 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.525+0000 7f9005e2d700 1 -- 172.21.15.44:0/2589845274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9000102100 msgr2=0x7f900010a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.525+0000 7f9005e2d700 1 --2- 172.21.15.44:0/2589845274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9000102100 0x7f900010a5e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009990 tx=0x7f8fe8009ca0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.525+0000 7f9005e2d700 1 -- 172.21.15.44:0/2589845274 shutdown_connections 2023-09-10T00:51:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.525+0000 7f9005e2d700 1 --2- 172.21.15.44:0/2589845274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f900010abb0 0x7f900010d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.525+0000 7f9005e2d700 1 --2- 172.21.15.44:0/2589845274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90001017e0 0x7f9000101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.525+0000 7f9005e2d700 1 --2- 172.21.15.44:0/2589845274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9000102100 0x7f900010a5e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.525+0000 7f9005e2d700 1 -- 172.21.15.44:0/2589845274 >> 172.21.15.44:0/2589845274 conn(0x7f90000faf40 msgr2=0x7f90000fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.525+0000 7f9005e2d700 1 -- 172.21.15.44:0/2589845274 shutdown_connections 2023-09-10T00:51:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.525+0000 7f9005e2d700 1 -- 172.21.15.44:0/2589845274 wait complete. 2023-09-10T00:51:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.525+0000 7f9005e2d700 1 Processor -- start 2023-09-10T00:51:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f9005e2d700 1 -- start start 2023-09-10T00:51:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f9005e2d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90001017e0 0x7f900019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f9005e2d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9000102100 0x7f900019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f9005e2d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f900010abb0 0x7f9000196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f9005e2d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9000196a70 con 0x7f90001017e0 2023-09-10T00:51:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f9005e2d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9000196be0 con 0x7f9000102100 2023-09-10T00:51:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f9005e2d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9000196da0 con 0x7f900010abb0 2023-09-10T00:51:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90001017e0 0x7f900019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f8ffeffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9000102100 0x7f900019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90001017e0 0x7f900019c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52410/0 (socket says 172.21.15.44:52410) 2023-09-10T00:51:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f8fff7fe700 1 -- 172.21.15.44:0/921781903 learned_addr learned my addr 172.21.15.44:0/921781903 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f8ffffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f900010abb0 0x7f9000196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f8fff7fe700 1 -- 172.21.15.44:0/921781903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f900010abb0 msgr2=0x7f9000196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f8fff7fe700 1 --2- 172.21.15.44:0/921781903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f900010abb0 0x7f9000196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f8fff7fe700 1 -- 172.21.15.44:0/921781903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9000102100 msgr2=0x7f900019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f8fff7fe700 1 --2- 172.21.15.44:0/921781903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9000102100 0x7f900019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f8fff7fe700 1 -- 172.21.15.44:0/921781903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe80095f0 con 0x7f90001017e0 2023-09-10T00:51:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.526+0000 7f8fff7fe700 1 --2- 172.21.15.44:0/921781903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90001017e0 0x7f900019c5a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000e9a0 tx=0x7f8ff000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.527+0000 7f8ffcff9700 1 -- 172.21.15.44:0/921781903 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0009790 con 0x7f90001017e0 2023-09-10T00:51:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.527+0000 7f9005e2d700 1 -- 172.21.15.44:0/921781903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9000197080 con 0x7f90001017e0 2023-09-10T00:51:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.527+0000 7f8ffcff9700 1 -- 172.21.15.44:0/921781903 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ff0010430 con 0x7f90001017e0 2023-09-10T00:51:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.527+0000 7f8ffcff9700 1 -- 172.21.15.44:0/921781903 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff000cea0 con 0x7f90001017e0 2023-09-10T00:51:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.527+0000 7f9005e2d700 1 -- 172.21.15.44:0/921781903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90001a3590 con 0x7f90001017e0 2023-09-10T00:51:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.529+0000 7f8ffcff9700 1 -- 172.21.15.44:0/921781903 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f8ff00105a0 con 0x7f90001017e0 2023-09-10T00:51:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.530+0000 7f8ffcff9700 1 --2- 172.21.15.44:0/921781903 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fec0a07b0 0x7f8fec0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.530+0000 7f8ffcff9700 1 -- 172.21.15.44:0/921781903 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8ff0014070 con 0x7f90001017e0 2023-09-10T00:51:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.530+0000 7f8ffeffd700 1 --2- 172.21.15.44:0/921781903 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fec0a07b0 0x7f8fec0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.530+0000 7f8ffeffd700 1 --2- 172.21.15.44:0/921781903 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fec0a07b0 0x7f8fec0a2c70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80059d0 tx=0x7f8fe8005960 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f9011ff3b80 1 Processor -- start 2023-09-10T00:51:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f9011ff3b80 1 -- start start 2023-09-10T00:51:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f9011ff3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca3551a0 0x561cca355580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f9011ff3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca34f300 0x561cca34f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f9011ff3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca35bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f9011ff3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cca2a86d0 con 0x561cca34fcc0 2023-09-10T00:51:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f9011ff3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cca2a5ff0 con 0x561cca3551a0 2023-09-10T00:51:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f9011ff3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cca2a18e0 con 0x561cca34f300 2023-09-10T00:51:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca3551a0 0x561cca355580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f8ffffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca35bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca3551a0 0x561cca355580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54158/0 (socket says 172.21.15.44:54158) 2023-09-10T00:51:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f8fff7fe700 1 -- 172.21.15.44:0/3664510304 learned_addr learned my addr 172.21.15.44:0/3664510304 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f8ffffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca35bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52422/0 (socket says 172.21.15.44:52422) 2023-09-10T00:51:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f8ffeffd700 1 --2- 172.21.15.44:0/3664510304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca34f300 0x561cca34f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f8ffffff700 1 -- 172.21.15.44:0/3664510304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca34f300 msgr2=0x561cca34f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f8ffffff700 1 --2- 172.21.15.44:0/3664510304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca34f300 0x561cca34f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f8ffffff700 1 -- 172.21.15.44:0/3664510304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca3551a0 msgr2=0x561cca355580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f8ffffff700 1 --2- 172.21.15.44:0/3664510304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca3551a0 0x561cca355580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f8ffffff700 1 -- 172.21.15.44:0/3664510304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cca0cdd80 con 0x561cca34fcc0 2023-09-10T00:51:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.531+0000 7f8fff7fe700 1 --2- 172.21.15.44:0/3664510304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca3551a0 0x561cca355580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.532+0000 7f8ffffff700 1 --2- 172.21.15.44:0/3664510304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca35bf30 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8011550 tx=0x7f8ff8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578426d9ec162962 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.532+0000 7f8ff5ffb700 1 -- 172.21.15.44:0/3664510304 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff800e040 con 0x561cca34fcc0 2023-09-10T00:51:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.532+0000 7f8ff5ffb700 1 -- 172.21.15.44:0/3664510304 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ff800f040 con 0x561cca34fcc0 2023-09-10T00:51:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.532+0000 7f8ff5ffb700 1 -- 172.21.15.44:0/3664510304 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8004320 con 0x561cca34fcc0 2023-09-10T00:51:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.532+0000 7f9011ff3b80 1 -- 172.21.15.44:0/3664510304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 msgr2=0x561cca35bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.532+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/3664510304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca35bf30 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8011550 tx=0x7f8ff8009920 comp rx=0 tx=0).stop 2023-09-10T00:51:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.532+0000 7f9011ff3b80 1 -- 172.21.15.44:0/3664510304 shutdown_connections 2023-09-10T00:51:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.532+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/3664510304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca35bf30 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.532+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/3664510304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca3551a0 0x561cca355580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.532+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/3664510304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca34f300 0x561cca34f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.532+0000 7f9011ff3b80 1 -- 172.21.15.44:0/3664510304 >> 172.21.15.44:0/3664510304 conn(0x561cca2c4030 msgr2=0x561cca34a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.532+0000 7f9011ff3b80 1 -- 172.21.15.44:0/3664510304 shutdown_connections 2023-09-10T00:51:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.533+0000 7f9011ff3b80 1 -- 172.21.15.44:0/3664510304 wait complete. 2023-09-10T00:51:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.533+0000 7f9011ff3b80 1 -- start start 2023-09-10T00:51:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.533+0000 7f9011ff3b80 1 Processor -- start 2023-09-10T00:51:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.533+0000 7f9011ff3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca34f300 0x561cca35be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.533+0000 7f9011ff3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34fcc0 0x561cca359090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.533+0000 7f9011ff3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca3551a0 0x561cca359650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.533+0000 7f9011ff3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cca23fb10 con 0x561cca3551a0 2023-09-10T00:51:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.533+0000 7f9011ff3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cca23f810 con 0x561cca34fcc0 2023-09-10T00:51:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.533+0000 7f9011ff3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cca1238c0 con 0x561cca34f300 2023-09-10T00:51:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.533+0000 7f9011ff3b80 10 client.0 _ll_register_callbacks cb 0x561cca189a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:51:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.533+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca34f300 0x561cca35be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.533+0000 7f8ffffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca3551a0 0x561cca359650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.533+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca34f300 0x561cca35be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59306/0 (socket says 172.21.15.44:59306) 2023-09-10T00:51:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.533+0000 7f8fff7fe700 1 -- 172.21.15.44:0/1351369842 learned_addr learned my addr 172.21.15.44:0/1351369842 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.534+0000 7f8ffeffd700 1 --2- 172.21.15.44:0/1351369842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34fcc0 0x561cca359090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.534+0000 7f8fff7fe700 1 -- 172.21.15.44:0/1351369842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34fcc0 msgr2=0x561cca359090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.534+0000 7f8fff7fe700 1 --2- 172.21.15.44:0/1351369842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34fcc0 0x561cca359090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.534+0000 7f8fff7fe700 1 -- 172.21.15.44:0/1351369842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca3551a0 msgr2=0x561cca359650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.534+0000 7f8fff7fe700 1 --2- 172.21.15.44:0/1351369842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca3551a0 0x561cca359650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.534+0000 7f8fff7fe700 1 -- 172.21.15.44:0/1351369842 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90001a3590 con 0x561cca34f300 2023-09-10T00:51:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.534+0000 7f8fff7fe700 1 --2- 172.21.15.44:0/1351369842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca34f300 0x561cca35be10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000e970 tx=0x7f8ff0099430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=512eeafa79cf75c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.534+0000 7f8fdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:51:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.534+0000 7f8fdeffd700 1 -- 172.21.15.44:0/1351369842 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff00cc360 con 0x561cca34f300 2023-09-10T00:51:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.534+0000 7f9011ff3b80 1 -- 172.21.15.44:0/1351369842 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cca1f9490 con 0x561cca34f300 2023-09-10T00:51:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.534+0000 7f8fdeffd700 1 -- 172.21.15.44:0/1351369842 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8ff00995e0 con 0x561cca34f300 2023-09-10T00:51:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.534+0000 7f8fdeffd700 1 -- 172.21.15.44:0/1351369842 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ff009d410 con 0x561cca34f300 2023-09-10T00:51:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.534+0000 7f9011ff3b80 10 client.26944 fetch_fsmap learned FSMap version 60 2023-09-10T00:51:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.534+0000 7f9011ff3b80 1 -- 172.21.15.44:0/1351369842 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x7f8ff8011230 con 0x561cca34f300 2023-09-10T00:51:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.535+0000 7f8fdeffd700 1 -- 172.21.15.44:0/1351369842 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561cca1f9490 con 0x561cca34f300 2023-09-10T00:51:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.535+0000 7f8fdeffd700 1 -- 172.21.15.44:0/1351369842 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 60) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f8ff00650f0 con 0x561cca34f300 2023-09-10T00:51:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.535+0000 7f9011ff3b80 10 client.26944 fetch_fsmap finished waiting for FSMap version 60 2023-09-10T00:51:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.535+0000 7f9011ff3b80 10 client.26944 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:51:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.535+0000 7f9011ff3b80 20 client.26944 populate_metadata read hostname 'smithi044' 2023-09-10T00:51:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.535+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/1351369842 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561cca369410 0x561cca36b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.535+0000 7f9011ff3b80 4 client.26944 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:51:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.535+0000 7f9011ff3b80 1 -- 172.21.15.44:0/1351369842 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cca2b6840 con 0x561cca369410 2023-09-10T00:51:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.535+0000 7f8ffeffd700 1 --2- 172.21.15.44:0/1351369842 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561cca369410 0x561cca36b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.535+0000 7f8fdeffd700 10 client.26944 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.535+0000 7f8ffeffd700 1 --2- 172.21.15.44:0/1351369842 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561cca369410 0x561cca36b830 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b483d2f1a30f54a9 server_cookie=1ccda30d8de2704 in_seq=0 out_seq=0 2023-09-10T00:51:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.536+0000 7f8fdeffd700 1 -- 172.21.15.44:0/1351369842 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561cca2b6840 con 0x561cca369410 2023-09-10T00:51:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.536+0000 7f8fdeffd700 10 client.26944 handle_command_reply: tid=0 2023-09-10T00:51:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.536+0000 7f9011ff3b80 1 client.26944 shutdown 2023-09-10T00:51:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.536+0000 7f9011ff3b80 1 -- 172.21.15.44:0/1351369842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca34f300 msgr2=0x561cca35be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.536+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/1351369842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca34f300 0x561cca35be10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000e970 tx=0x7f8ff0099430 comp rx=0 tx=0).stop 2023-09-10T00:51:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.536+0000 7f9011ff3b80 1 -- 172.21.15.44:0/1351369842 shutdown_connections 2023-09-10T00:51:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.536+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/1351369842 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561cca369410 0x561cca36b830 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.536+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/1351369842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca3551a0 0x561cca359650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.536+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/1351369842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca34f300 0x561cca35be10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.536+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/1351369842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34fcc0 0x561cca359090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.536+0000 7f8fdeffd700 0 client.26944 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.536+0000 7f9011ff3b80 1 -- 172.21.15.44:0/1351369842 >> 172.21.15.44:0/1351369842 conn(0x561cca2c4030 msgr2=0x561cca3497a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.536+0000 7f9011ff3b80 1 -- 172.21.15.44:0/1351369842 shutdown_connections 2023-09-10T00:51:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.536+0000 7f9011ff3b80 1 -- 172.21.15.44:0/1351369842 wait complete. 2023-09-10T00:51:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.536+0000 7f9011ff3b80 20 client.26944 trim_cache size 0 max 16384 2023-09-10T00:51:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.551+0000 7f9011ff3b80 1 Processor -- start 2023-09-10T00:51:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.551+0000 7f9011ff3b80 1 -- start start 2023-09-10T00:51:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.551+0000 7f9011ff3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34f300 0x561cca35b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.551+0000 7f9011ff3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca358950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.551+0000 7f9011ff3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca3551a0 0x561cca358e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.551+0000 7f9011ff3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cca359c30 con 0x561cca34fcc0 2023-09-10T00:51:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.551+0000 7f9011ff3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cca352db0 con 0x561cca34f300 2023-09-10T00:51:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.551+0000 7f9011ff3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cca353b20 con 0x561cca3551a0 2023-09-10T00:51:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.551+0000 7f8ffeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca358950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.551+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34f300 0x561cca35b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.552+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34f300 0x561cca35b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54172/0 (socket says 172.21.15.44:54172) 2023-09-10T00:51:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.552+0000 7f8ffeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca358950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52434/0 (socket says 172.21.15.44:52434) 2023-09-10T00:51:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.552+0000 7f8fff7fe700 1 -- 172.21.15.44:0/2246680122 learned_addr learned my addr 172.21.15.44:0/2246680122 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.552+0000 7f8ffeffd700 1 -- 172.21.15.44:0/2246680122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca3551a0 msgr2=0x561cca358e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.552+0000 7f8ffeffd700 1 --2- 172.21.15.44:0/2246680122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca3551a0 0x561cca358e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.552+0000 7f8ffeffd700 1 -- 172.21.15.44:0/2246680122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34f300 msgr2=0x561cca35b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.552+0000 7f8ffeffd700 1 --2- 172.21.15.44:0/2246680122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34f300 0x561cca35b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.552+0000 7f8ffeffd700 1 -- 172.21.15.44:0/2246680122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cca0b6c00 con 0x561cca34fcc0 2023-09-10T00:51:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.552+0000 7f8fff7fe700 1 --2- 172.21.15.44:0/2246680122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34f300 0x561cca35b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.552+0000 7f8ffeffd700 1 --2- 172.21.15.44:0/2246680122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca358950 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80098e0 tx=0x7f8fe8023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe95fa4bd69e2a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.552+0000 7f8ff67fc700 1 -- 172.21.15.44:0/2246680122 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8036de0 con 0x561cca34fcc0 2023-09-10T00:51:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.552+0000 7f8ff67fc700 1 -- 172.21.15.44:0/2246680122 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fe801e040 con 0x561cca34fcc0 2023-09-10T00:51:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.552+0000 7f8ff67fc700 1 -- 172.21.15.44:0/2246680122 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8004be0 con 0x561cca34fcc0 2023-09-10T00:51:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.553+0000 7f9011ff3b80 1 -- 172.21.15.44:0/2246680122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 msgr2=0x561cca358950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.553+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/2246680122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca358950 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80098e0 tx=0x7f8fe8023000 comp rx=0 tx=0).stop 2023-09-10T00:51:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.553+0000 7f9011ff3b80 1 -- 172.21.15.44:0/2246680122 shutdown_connections 2023-09-10T00:51:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.553+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/2246680122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca3551a0 0x561cca358e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.553+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/2246680122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34f300 0x561cca35b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.553+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/2246680122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca358950 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.553+0000 7f9011ff3b80 1 -- 172.21.15.44:0/2246680122 >> 172.21.15.44:0/2246680122 conn(0x561cca2c4030 msgr2=0x561cca365660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.553+0000 7f9011ff3b80 1 -- 172.21.15.44:0/2246680122 shutdown_connections 2023-09-10T00:51:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.553+0000 7f9011ff3b80 1 -- 172.21.15.44:0/2246680122 wait complete. 2023-09-10T00:51:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.553+0000 7f9011ff3b80 1 -- start start 2023-09-10T00:51:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.553+0000 7f9011ff3b80 1 Processor -- start 2023-09-10T00:51:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f9011ff3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34f300 0x561cca35f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f9011ff3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca35fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f9011ff3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca3551a0 0x561cca3bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f9011ff3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cca3605a0 con 0x561cca34fcc0 2023-09-10T00:51:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f9011ff3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cca3bc760 con 0x561cca34f300 2023-09-10T00:51:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f9011ff3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cca3bc8a0 con 0x561cca3551a0 2023-09-10T00:51:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f9011ff3b80 10 client.0 _ll_register_callbacks cb 0x7f8ff0099210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:51:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34f300 0x561cca35f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f8ffeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca35fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34f300 0x561cca35f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54178/0 (socket says 172.21.15.44:54178) 2023-09-10T00:51:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f8fff7fe700 1 -- 172.21.15.44:0/2895825011 learned_addr learned my addr 172.21.15.44:0/2895825011 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f8ffffff700 1 --2- 172.21.15.44:0/2895825011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca3551a0 0x561cca3bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f8ffeffd700 1 -- 172.21.15.44:0/2895825011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca3551a0 msgr2=0x561cca3bc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f8ffeffd700 1 --2- 172.21.15.44:0/2895825011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca3551a0 0x561cca3bc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f8ffeffd700 1 -- 172.21.15.44:0/2895825011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34f300 msgr2=0x561cca35f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f8ffeffd700 1 --2- 172.21.15.44:0/2895825011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34f300 0x561cca35f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f8ffeffd700 1 -- 172.21.15.44:0/2895825011 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff8011230 con 0x561cca34fcc0 2023-09-10T00:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f8fff7fe700 1 --2- 172.21.15.44:0/2895825011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34f300 0x561cca35f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f8ffeffd700 1 --2- 172.21.15.44:0/2895825011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca35fc90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8006010 tx=0x7f8fe800b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce3c31186aa0f7f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f8fdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:51:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f8fdeffd700 1 -- 172.21.15.44:0/2895825011 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8036570 con 0x561cca34fcc0 2023-09-10T00:51:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f8fdeffd700 1 -- 172.21.15.44:0/2895825011 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8fe8004500 con 0x561cca34fcc0 2023-09-10T00:51:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f9011ff3b80 1 -- 172.21.15.44:0/2895825011 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ff800f040 con 0x561cca34fcc0 2023-09-10T00:51:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.554+0000 7f8fdeffd700 1 -- 172.21.15.44:0/2895825011 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fe80046c0 con 0x561cca34fcc0 2023-09-10T00:51:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.555+0000 7f9011ff3b80 10 client.17343 fetch_fsmap learned FSMap version 60 2023-09-10T00:51:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.555+0000 7f9011ff3b80 1 -- 172.21.15.44:0/2895825011 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x7f900010dd10 con 0x561cca34fcc0 2023-09-10T00:51:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.555+0000 7f8fdeffd700 1 -- 172.21.15.44:0/2895825011 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ff800f040 con 0x561cca34fcc0 2023-09-10T00:51:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.555+0000 7f8fdeffd700 1 -- 172.21.15.44:0/2895825011 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 60) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f8fe802f030 con 0x561cca34fcc0 2023-09-10T00:51:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.555+0000 7f9011ff3b80 10 client.17343 fetch_fsmap finished waiting for FSMap version 60 2023-09-10T00:51:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.555+0000 7f9011ff3b80 10 client.17343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:51:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.555+0000 7f9011ff3b80 20 client.17343 populate_metadata read hostname 'smithi044' 2023-09-10T00:51:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.555+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/2895825011 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561cca3bdca0 0x561cca3c00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.555+0000 7f9011ff3b80 4 client.17343 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:51:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.555+0000 7f9011ff3b80 1 -- 172.21.15.44:0/2895825011 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561cca29a8c0 con 0x561cca3bdca0 2023-09-10T00:51:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.555+0000 7f8fff7fe700 1 --2- 172.21.15.44:0/2895825011 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561cca3bdca0 0x561cca3c00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.556+0000 7f8fdeffd700 10 client.17343 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.556+0000 7f8fff7fe700 1 --2- 172.21.15.44:0/2895825011 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561cca3bdca0 0x561cca3c00c0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4b6bcb3bf656e97 server_cookie=43a118797c188b97 in_seq=0 out_seq=0 2023-09-10T00:51:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.572+0000 7f8fdeffd700 1 -- 172.21.15.44:0/2895825011 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561cca29a8c0 con 0x561cca3bdca0 2023-09-10T00:51:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.572+0000 7f8fdeffd700 10 client.17343 handle_command_reply: tid=0 2023-09-10T00:51:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.572+0000 7f9011ff3b80 1 client.17343 shutdown 2023-09-10T00:51:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.572+0000 7f9011ff3b80 1 -- 172.21.15.44:0/2895825011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 msgr2=0x561cca35fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.572+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/2895825011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca35fc90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8006010 tx=0x7f8fe800b710 comp rx=0 tx=0).stop 2023-09-10T00:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.572+0000 7f9011ff3b80 1 -- 172.21.15.44:0/2895825011 shutdown_connections 2023-09-10T00:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.572+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/2895825011 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561cca3bdca0 0x561cca3c00c0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.572+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/2895825011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cca3551a0 0x561cca3bc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.572+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/2895825011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cca34f300 0x561cca35f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.572+0000 7f9011ff3b80 1 --2- 172.21.15.44:0/2895825011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cca34fcc0 0x561cca35fc90 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.572+0000 7f8fdeffd700 0 client.17343 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.572+0000 7f9011ff3b80 1 -- 172.21.15.44:0/2895825011 >> 172.21.15.44:0/2895825011 conn(0x561cca2c4030 msgr2=0x561cca354150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.572+0000 7f9011ff3b80 1 -- 172.21.15.44:0/2895825011 shutdown_connections 2023-09-10T00:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.572+0000 7f9011ff3b80 1 -- 172.21.15.44:0/2895825011 wait complete. 2023-09-10T00:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.572+0000 7f9011ff3b80 20 client.17343 trim_cache size 0 max 16384 2023-09-10T00:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.573+0000 7f9005e2d700 1 -- 172.21.15.44:0/921781903 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fec0a07b0 msgr2=0x7f8fec0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.573+0000 7f9005e2d700 1 --2- 172.21.15.44:0/921781903 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fec0a07b0 0x7f8fec0a2c70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80059d0 tx=0x7f8fe8005960 comp rx=0 tx=0).stop 2023-09-10T00:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.573+0000 7f9005e2d700 1 -- 172.21.15.44:0/921781903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90001017e0 msgr2=0x7f900019c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:01.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.573+0000 7f9005e2d700 1 --2- 172.21.15.44:0/921781903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90001017e0 0x7f900019c5a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000e9a0 tx=0x7f8ff000ecb0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.573+0000 7f9005e2d700 1 -- 172.21.15.44:0/921781903 shutdown_connections 2023-09-10T00:51:01.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.573+0000 7f9005e2d700 1 --2- 172.21.15.44:0/921781903 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fec0a07b0 0x7f8fec0a2c70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.573+0000 7f9005e2d700 1 --2- 172.21.15.44:0/921781903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f900010abb0 0x7f9000196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.573+0000 7f9005e2d700 1 --2- 172.21.15.44:0/921781903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90001017e0 0x7f900019c5a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.573+0000 7f9005e2d700 1 --2- 172.21.15.44:0/921781903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9000102100 0x7f900019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.573+0000 7f9005e2d700 1 -- 172.21.15.44:0/921781903 >> 172.21.15.44:0/921781903 conn(0x7f90000faf40 msgr2=0x7f900010ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.573+0000 7f9005e2d700 1 -- 172.21.15.44:0/921781903 shutdown_connections 2023-09-10T00:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:01.573+0000 7f9005e2d700 1 -- 172.21.15.44:0/921781903 wait complete. 2023-09-10T00:51:02.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T00:51:02.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T00:51:02.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:02.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:02.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:51:02.050 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:51:02.050 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:01 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:02.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T00:51:02.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T00:51:02.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:02.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:02.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:51:02.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:51:02.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:01 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:02.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T00:51:02.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T00:51:02.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:02.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:02.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:51:02.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:51:02.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:51:02.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:01 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:51:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:02 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T00:51:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:02 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:51:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:02 smithi131 ceph-mon[103713]: pgmap v322: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 12 MiB/s wr, 47 op/s 2023-09-10T00:51:03.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:02 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T00:51:03.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:02 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T00:51:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:02 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T00:51:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:02 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:51:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:02 smithi044 ceph-mon[98676]: pgmap v322: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 12 MiB/s wr, 47 op/s 2023-09-10T00:51:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:02 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T00:51:03.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:02 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T00:51:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:02 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T00:51:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:02 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:51:03.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:02 smithi099 ceph-mon[102692]: pgmap v322: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 12 MiB/s wr, 47 op/s 2023-09-10T00:51:03.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:02 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T00:51:03.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:02 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T00:51:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:04 smithi131 ceph-mon[103713]: pgmap v323: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 21 op/s 2023-09-10T00:51:05.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:04 smithi131 ceph-mon[103713]: osdmap e101: 12 total, 12 up, 12 in 2023-09-10T00:51:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:04 smithi044 ceph-mon[98676]: pgmap v323: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 21 op/s 2023-09-10T00:51:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:04 smithi044 ceph-mon[98676]: osdmap e101: 12 total, 12 up, 12 in 2023-09-10T00:51:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:04 smithi099 ceph-mon[102692]: pgmap v323: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 21 op/s 2023-09-10T00:51:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:04 smithi099 ceph-mon[102692]: osdmap e101: 12 total, 12 up, 12 in 2023-09-10T00:51:07.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:06 smithi131 ceph-mon[103713]: pgmap v325: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 5.7 MiB/s wr, 61 op/s 2023-09-10T00:51:07.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:06 smithi044 ceph-mon[98676]: pgmap v325: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 5.7 MiB/s wr, 61 op/s 2023-09-10T00:51:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:38900000 of 50000000 tuples (77%) done (elapsed 606.71 s, remaining 173.12 s) 2023-09-10T00:51:07.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:06 smithi099 ceph-mon[102692]: pgmap v325: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 5.7 MiB/s wr, 61 op/s 2023-09-10T00:51:08.221 INFO:teuthology.orchestra.run.smithi044.stderr:39000000 of 50000000 tuples (78%) done (elapsed 607.85 s, remaining 171.45 s) 2023-09-10T00:51:09.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:08 smithi131 ceph-mon[103713]: pgmap v326: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 5.7 MiB/s wr, 61 op/s 2023-09-10T00:51:09.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:08 smithi044 ceph-mon[98676]: pgmap v326: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 5.7 MiB/s wr, 61 op/s 2023-09-10T00:51:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:08 smithi099 ceph-mon[102692]: pgmap v326: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 5.7 MiB/s wr, 61 op/s 2023-09-10T00:51:09.165 INFO:teuthology.orchestra.run.smithi044.stderr:39100000 of 50000000 tuples (78%) done (elapsed 608.80 s, remaining 169.72 s) 2023-09-10T00:51:10.109 INFO:teuthology.orchestra.run.smithi044.stderr:39200000 of 50000000 tuples (78%) done (elapsed 609.74 s, remaining 167.99 s) 2023-09-10T00:51:10.386 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:51:10.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.539+0000 7f9cc6bdd700 1 -- 172.21.15.44:0/3586689149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc0105570 msgr2=0x7f9cc0105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:10.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.539+0000 7f9cc6bdd700 1 --2- 172.21.15.44:0/3586689149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc0105570 0x7f9cc0105950 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009920 tx=0x7f9cb4009c30 comp rx=0 tx=0).stop 2023-09-10T00:51:10.541 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.539+0000 7f9cc6bdd700 1 -- 172.21.15.44:0/3586689149 shutdown_connections 2023-09-10T00:51:10.541 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.539+0000 7f9cc6bdd700 1 --2- 172.21.15.44:0/3586689149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc00ffe70 0x7f9cc010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.541 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.539+0000 7f9cc6bdd700 1 --2- 172.21.15.44:0/3586689149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc0105570 0x7f9cc0105950 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.541 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.539+0000 7f9cc6bdd700 1 --2- 172.21.15.44:0/3586689149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc00ff4b0 0x7f9cc00ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.541 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.539+0000 7f9cc6bdd700 1 -- 172.21.15.44:0/3586689149 >> 172.21.15.44:0/3586689149 conn(0x7f9cc00fafa0 msgr2=0x7f9cc00fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:10.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.539+0000 7f9cc6bdd700 1 -- 172.21.15.44:0/3586689149 shutdown_connections 2023-09-10T00:51:10.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.539+0000 7f9cc6bdd700 1 -- 172.21.15.44:0/3586689149 wait complete. 2023-09-10T00:51:10.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.540+0000 7f9cc6bdd700 1 Processor -- start 2023-09-10T00:51:10.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.540+0000 7f9cc6bdd700 1 -- start start 2023-09-10T00:51:10.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.541+0000 7f9cc6bdd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc00ff4b0 0x7f9cc0197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.541+0000 7f9cc6bdd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc00ffe70 0x7f9cc0197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.541+0000 7f9cc6bdd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0105570 0x7f9cc01a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.541+0000 7f9cc6bdd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc0198260 con 0x7f9cc0105570 2023-09-10T00:51:10.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.541+0000 7f9cc6bdd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc01a29d0 con 0x7f9cc00ff4b0 2023-09-10T00:51:10.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.541+0000 7f9cc6bdd700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc01a2b40 con 0x7f9cc00ffe70 2023-09-10T00:51:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.541+0000 7f9cc4979700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc00ff4b0 0x7f9cc0197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.541+0000 7f9cc4979700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc00ff4b0 0x7f9cc0197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54192/0 (socket says 172.21.15.44:54192) 2023-09-10T00:51:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.541+0000 7f9cc4979700 1 -- 172.21.15.44:0/3242577067 learned_addr learned my addr 172.21.15.44:0/3242577067 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.541+0000 7f9cc517a700 1 --2- 172.21.15.44:0/3242577067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0105570 0x7f9cc01a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.541+0000 7f9cbffff700 1 --2- 172.21.15.44:0/3242577067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc00ffe70 0x7f9cc0197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.541+0000 7f9cc4979700 1 -- 172.21.15.44:0/3242577067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc00ffe70 msgr2=0x7f9cc0197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.541+0000 7f9cc4979700 1 --2- 172.21.15.44:0/3242577067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc00ffe70 0x7f9cc0197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.541+0000 7f9cc4979700 1 -- 172.21.15.44:0/3242577067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0105570 msgr2=0x7f9cc01a2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.541+0000 7f9cc4979700 1 --2- 172.21.15.44:0/3242577067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0105570 0x7f9cc01a2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.541+0000 7f9cc4979700 1 -- 172.21.15.44:0/3242577067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb4009520 con 0x7f9cc00ff4b0 2023-09-10T00:51:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.542+0000 7f9cc4979700 1 --2- 172.21.15.44:0/3242577067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc00ff4b0 0x7f9cc0197410 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009840 tx=0x7f9cb400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.542+0000 7f9cbdffb700 1 -- 172.21.15.44:0/3242577067 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb401c070 con 0x7f9cc00ff4b0 2023-09-10T00:51:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.542+0000 7f9cbdffb700 1 -- 172.21.15.44:0/3242577067 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cb400bb70 con 0x7f9cc00ff4b0 2023-09-10T00:51:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.542+0000 7f9cc6bdd700 1 -- 172.21.15.44:0/3242577067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc01a2dc0 con 0x7f9cc00ff4b0 2023-09-10T00:51:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.542+0000 7f9cbdffb700 1 -- 172.21.15.44:0/3242577067 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb40038e0 con 0x7f9cc00ff4b0 2023-09-10T00:51:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.542+0000 7f9cc6bdd700 1 -- 172.21.15.44:0/3242577067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc01a3190 con 0x7f9cc00ff4b0 2023-09-10T00:51:10.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.545+0000 7f9cbdffb700 1 -- 172.21.15.44:0/3242577067 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f9cb4020030 con 0x7f9cc00ff4b0 2023-09-10T00:51:10.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.546+0000 7f9cbdffb700 1 --2- 172.21.15.44:0/3242577067 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9cb00a0830 0x7f9cb00a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.546+0000 7f9cbdffb700 1 -- 172.21.15.44:0/3242577067 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f9cb40cd1e0 con 0x7f9cc00ff4b0 2023-09-10T00:51:10.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.547+0000 7f9cbffff700 1 --2- 172.21.15.44:0/3242577067 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9cb00a0830 0x7f9cb00a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.547+0000 7f9cbffff700 1 --2- 172.21.15.44:0/3242577067 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9cb00a0830 0x7f9cb00a2cf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0068f80 tx=0x7f9cac009310 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:10.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.547+0000 7f9cd2da3b80 1 Processor -- start 2023-09-10T00:51:10.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cd2da3b80 1 -- start start 2023-09-10T00:51:10.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cd2da3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceb7f40 0x56252ceb8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cd2da3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 0x56252cea9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cd2da3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 0x56252cebec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cd2da3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252ce02730 con 0x56252cea9710 2023-09-10T00:51:10.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cd2da3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252ce00050 con 0x56252ceb7f40 2023-09-10T00:51:10.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cd2da3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252cdfb940 con 0x56252ceaa0d0 2023-09-10T00:51:10.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cc517a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 0x56252cebec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cc4979700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceb7f40 0x56252ceb8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cbffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 0x56252cea9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cc4979700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceb7f40 0x56252ceb8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54200/0 (socket says 172.21.15.44:54200) 2023-09-10T00:51:10.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cc4979700 1 -- 172.21.15.44:0/4203911511 learned_addr learned my addr 172.21.15.44:0/4203911511 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:10.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cc4979700 1 -- 172.21.15.44:0/4203911511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 msgr2=0x56252cebec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:10.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cc4979700 1 --2- 172.21.15.44:0/4203911511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 0x56252cebec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cc4979700 1 -- 172.21.15.44:0/4203911511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 msgr2=0x56252cea9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:10.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cc4979700 1 --2- 172.21.15.44:0/4203911511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 0x56252cea9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cc4979700 1 -- 172.21.15.44:0/4203911511 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56252cc27de0 con 0x56252ceb7f40 2023-09-10T00:51:10.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cbffff700 1 --2- 172.21.15.44:0/4203911511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 0x56252cea9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:51:10.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.548+0000 7f9cc517a700 1 --2- 172.21.15.44:0/4203911511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 0x56252cebec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:51:10.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.549+0000 7f9cc4979700 1 --2- 172.21.15.44:0/4203911511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceb7f40 0x56252ceb8320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4096d40 tx=0x7f9cb409a170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f444a00db9c389a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:10.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.549+0000 7f9caaffd700 1 -- 172.21.15.44:0/4203911511 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb40cd030 con 0x56252ceb7f40 2023-09-10T00:51:10.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.549+0000 7f9caaffd700 1 -- 172.21.15.44:0/4203911511 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cb409a9d0 con 0x56252ceb7f40 2023-09-10T00:51:10.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.549+0000 7f9caaffd700 1 -- 172.21.15.44:0/4203911511 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb40a0250 con 0x56252ceb7f40 2023-09-10T00:51:10.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.550+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/4203911511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceb7f40 msgr2=0x56252ceb8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:10.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.550+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/4203911511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceb7f40 0x56252ceb8320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4096d40 tx=0x7f9cb409a170 comp rx=0 tx=0).stop 2023-09-10T00:51:10.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.550+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/4203911511 shutdown_connections 2023-09-10T00:51:10.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.550+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/4203911511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 0x56252cebec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.550+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/4203911511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceb7f40 0x56252ceb8320 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.550+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/4203911511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 0x56252cea9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.550+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/4203911511 >> 172.21.15.44:0/4203911511 conn(0x56252ce1e090 msgr2=0x56252cea4820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:10.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.550+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/4203911511 shutdown_connections 2023-09-10T00:51:10.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.550+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/4203911511 wait complete. 2023-09-10T00:51:10.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.550+0000 7f9cd2da3b80 1 -- start start 2023-09-10T00:51:10.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.550+0000 7f9cd2da3b80 1 Processor -- start 2023-09-10T00:51:10.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cd2da3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252cea9710 0x56252cec17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cd2da3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 0x56252cec1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cd2da3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252ceb7f40 0x56252cebbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cd2da3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252cd99b70 con 0x56252ceb7f40 2023-09-10T00:51:10.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cd2da3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252cd99870 con 0x56252cea9710 2023-09-10T00:51:10.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cd2da3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252cc7d920 con 0x56252ceaa0d0 2023-09-10T00:51:10.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cd2da3b80 10 client.0 _ll_register_callbacks cb 0x56252cce3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:51:10.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cbffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 0x56252cec1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cbffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 0x56252cec1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59350/0 (socket says 172.21.15.44:59350) 2023-09-10T00:51:10.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cbffff700 1 -- 172.21.15.44:0/2512747988 learned_addr learned my addr 172.21.15.44:0/2512747988 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:10.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cc517a700 1 --2- 172.21.15.44:0/2512747988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252ceb7f40 0x56252cebbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cc4979700 1 --2- 172.21.15.44:0/2512747988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252cea9710 0x56252cec17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cbffff700 1 -- 172.21.15.44:0/2512747988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252cea9710 msgr2=0x56252cec17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:10.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cbffff700 1 --2- 172.21.15.44:0/2512747988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252cea9710 0x56252cec17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cbffff700 1 -- 172.21.15.44:0/2512747988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252ceb7f40 msgr2=0x56252cebbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:10.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cbffff700 1 --2- 172.21.15.44:0/2512747988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252ceb7f40 0x56252cebbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.551+0000 7f9cbffff700 1 -- 172.21.15.44:0/2512747988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc01a3190 con 0x56252ceaa0d0 2023-09-10T00:51:10.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.552+0000 7f9cc517a700 1 --2- 172.21.15.44:0/2512747988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252ceb7f40 0x56252cebbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:51:10.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.552+0000 7f9cbffff700 1 --2- 172.21.15.44:0/2512747988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 0x56252cec1d10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9cac003900 tx=0x7f9cac0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b5f7274c12a5925 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:10.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.552+0000 7f9c9ffff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:51:10.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.552+0000 7f9c9ffff700 1 -- 172.21.15.44:0/2512747988 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac01da40 con 0x56252ceaa0d0 2023-09-10T00:51:10.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.552+0000 7f9c9ffff700 1 -- 172.21.15.44:0/2512747988 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f9cac01dba0 con 0x56252ceaa0d0 2023-09-10T00:51:10.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.552+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/2512747988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56252cd534f0 con 0x56252ceaa0d0 2023-09-10T00:51:10.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.552+0000 7f9c9ffff700 1 -- 172.21.15.44:0/2512747988 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cac01dd60 con 0x56252ceaa0d0 2023-09-10T00:51:10.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.552+0000 7f9cd2da3b80 10 client.26956 fetch_fsmap learned FSMap version 60 2023-09-10T00:51:10.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.552+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/2512747988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x56252cc10c60 con 0x56252ceaa0d0 2023-09-10T00:51:10.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.553+0000 7f9c9ffff700 1 -- 172.21.15.44:0/2512747988 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56252cd534f0 con 0x56252ceaa0d0 2023-09-10T00:51:10.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.553+0000 7f9c9ffff700 1 -- 172.21.15.44:0/2512747988 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 60) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f9cac01b410 con 0x56252ceaa0d0 2023-09-10T00:51:10.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.554+0000 7f9cd2da3b80 10 client.26956 fetch_fsmap finished waiting for FSMap version 60 2023-09-10T00:51:10.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.554+0000 7f9cd2da3b80 10 client.26956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:51:10.569 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.554+0000 7f9cd2da3b80 20 client.26956 populate_metadata read hostname 'smithi044' 2023-09-10T00:51:10.569 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.554+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/2512747988 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56252cecbde0 0x56252cece200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.569 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.554+0000 7f9cd2da3b80 4 client.26956 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:51:10.569 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.554+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/2512747988 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56252ce108a0 con 0x56252cecbde0 2023-09-10T00:51:10.569 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.554+0000 7f9cc4979700 1 --2- 172.21.15.44:0/2512747988 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56252cecbde0 0x56252cece200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.554+0000 7f9c9ffff700 10 client.26956 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.554+0000 7f9cc4979700 1 --2- 172.21.15.44:0/2512747988 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56252cecbde0 0x56252cece200 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55375d03769b0b85 server_cookie=936318abd0478872 in_seq=0 out_seq=0 2023-09-10T00:51:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.555+0000 7f9c9ffff700 1 -- 172.21.15.44:0/2512747988 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56252ce108a0 con 0x56252cecbde0 2023-09-10T00:51:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.555+0000 7f9c9ffff700 10 client.26956 handle_command_reply: tid=0 2023-09-10T00:51:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.555+0000 7f9cd2da3b80 1 client.26956 shutdown 2023-09-10T00:51:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.555+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/2512747988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 msgr2=0x56252cec1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.555+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/2512747988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 0x56252cec1d10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9cac003900 tx=0x7f9cac0047c0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.555+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/2512747988 shutdown_connections 2023-09-10T00:51:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.555+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/2512747988 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56252cecbde0 0x56252cece200 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.555+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/2512747988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252ceb7f40 0x56252cebbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.555+0000 7f9c9ffff700 0 client.26956 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.555+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/2512747988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252cea9710 0x56252cec17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.555+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/2512747988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 0x56252cec1d10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.555+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/2512747988 >> 172.21.15.44:0/2512747988 conn(0x56252ce1e090 msgr2=0x56252cea4520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.555+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/2512747988 shutdown_connections 2023-09-10T00:51:10.573 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.555+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/2512747988 wait complete. 2023-09-10T00:51:10.573 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.555+0000 7f9cd2da3b80 20 client.26956 trim_cache size 0 max 16384 2023-09-10T00:51:10.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.575+0000 7f9cd2da3b80 1 Processor -- start 2023-09-10T00:51:10.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.575+0000 7f9cd2da3b80 1 -- start start 2023-09-10T00:51:10.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.575+0000 7f9cd2da3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 0x56252cec1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.575+0000 7f9cd2da3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceaa0d0 0x56252ceb9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.575+0000 7f9cd2da3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceb7f40 0x56252cebb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.575+0000 7f9cd2da3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252cec20e0 con 0x56252cea9710 2023-09-10T00:51:10.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.575+0000 7f9cd2da3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252cea13d0 con 0x56252ceaa0d0 2023-09-10T00:51:10.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.575+0000 7f9cd2da3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252cec92c0 con 0x56252ceb7f40 2023-09-10T00:51:10.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.576+0000 7f9cbffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceaa0d0 0x56252ceb9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.576+0000 7f9cc517a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceb7f40 0x56252cebb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.576+0000 7f9cc4979700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 0x56252cec1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.576+0000 7f9cbffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceaa0d0 0x56252ceb9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54216/0 (socket says 172.21.15.44:54216) 2023-09-10T00:51:10.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.576+0000 7f9cc4979700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 0x56252cec1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52474/0 (socket says 172.21.15.44:52474) 2023-09-10T00:51:10.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.576+0000 7f9cbffff700 1 -- 172.21.15.44:0/3819136965 learned_addr learned my addr 172.21.15.44:0/3819136965 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:10.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.576+0000 7f9cc517a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceb7f40 0x56252cebb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59360/0 (socket says 172.21.15.44:59360) 2023-09-10T00:51:10.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.576+0000 7f9cbffff700 1 -- 172.21.15.44:0/3819136965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceb7f40 msgr2=0x56252cebb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:10.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.576+0000 7f9cbffff700 1 --2- 172.21.15.44:0/3819136965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceb7f40 0x56252cebb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.576+0000 7f9cbffff700 1 -- 172.21.15.44:0/3819136965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 msgr2=0x56252cec1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:10.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.576+0000 7f9cbffff700 1 --2- 172.21.15.44:0/3819136965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 0x56252cec1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.576+0000 7f9cbffff700 1 -- 172.21.15.44:0/3819136965 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56252cc4e250 con 0x56252ceaa0d0 2023-09-10T00:51:10.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.576+0000 7f9cbffff700 1 --2- 172.21.15.44:0/3819136965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceaa0d0 0x56252ceb9330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9cac01b8c0 tx=0x7f9cac011b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70d688a565f1ec54 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:10.581 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.576+0000 7f9cab7fe700 1 -- 172.21.15.44:0/3819136965 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac0045a0 con 0x56252ceaa0d0 2023-09-10T00:51:10.581 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.576+0000 7f9cab7fe700 1 -- 172.21.15.44:0/3819136965 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cac01d0c0 con 0x56252ceaa0d0 2023-09-10T00:51:10.581 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.576+0000 7f9cab7fe700 1 -- 172.21.15.44:0/3819136965 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac01ba30 con 0x56252ceaa0d0 2023-09-10T00:51:10.588 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.577+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/3819136965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceaa0d0 msgr2=0x56252ceb9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:10.588 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.577+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/3819136965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceaa0d0 0x56252ceb9330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9cac01b8c0 tx=0x7f9cac011b30 comp rx=0 tx=0).stop 2023-09-10T00:51:10.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.577+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/3819136965 shutdown_connections 2023-09-10T00:51:10.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.577+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/3819136965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceb7f40 0x56252cebb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.589 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.577+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/3819136965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 0x56252cec1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.590 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.577+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/3819136965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceaa0d0 0x56252ceb9330 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.590 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.577+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/3819136965 >> 172.21.15.44:0/3819136965 conn(0x56252ce1e090 msgr2=0x56252cec1490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:10.591 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.577+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/3819136965 shutdown_connections 2023-09-10T00:51:10.591 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.577+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/3819136965 wait complete. 2023-09-10T00:51:10.591 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.578+0000 7f9cd2da3b80 1 -- start start 2023-09-10T00:51:10.592 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.578+0000 7f9cd2da3b80 1 Processor -- start 2023-09-10T00:51:10.592 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.578+0000 7f9cd2da3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 0x56252cec69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.592 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.578+0000 7f9cd2da3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 0x56252cec6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.592 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.578+0000 7f9cd2da3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceb7f40 0x56252cec7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.578+0000 7f9cd2da3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252cf1fcc0 con 0x56252cea9710 2023-09-10T00:51:10.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.578+0000 7f9cd2da3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252cf1fe00 con 0x56252ceb7f40 2023-09-10T00:51:10.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.578+0000 7f9cd2da3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252cf1ff90 con 0x56252ceaa0d0 2023-09-10T00:51:10.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cd2da3b80 10 client.0 _ll_register_callbacks cb 0x7f9cac004400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:51:10.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cc4979700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 0x56252cec69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.593 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cbffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 0x56252cec6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cc4979700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 0x56252cec69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52480/0 (socket says 172.21.15.44:52480) 2023-09-10T00:51:10.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cc517a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceb7f40 0x56252cec7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cc4979700 1 -- 172.21.15.44:0/2936279803 learned_addr learned my addr 172.21.15.44:0/2936279803 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:10.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cc4979700 1 -- 172.21.15.44:0/2936279803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 msgr2=0x56252cec6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:10.594 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cc4979700 1 --2- 172.21.15.44:0/2936279803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 0x56252cec6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cc4979700 1 -- 172.21.15.44:0/2936279803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceb7f40 msgr2=0x56252cec7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:10.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cc4979700 1 --2- 172.21.15.44:0/2936279803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceb7f40 0x56252cec7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cc4979700 1 -- 172.21.15.44:0/2936279803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56252cc10c60 con 0x56252cea9710 2023-09-10T00:51:10.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cbffff700 1 --2- 172.21.15.44:0/2936279803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 0x56252cec6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:10.595 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cc517a700 1 --2- 172.21.15.44:0/2936279803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceb7f40 0x56252cec7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:10.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cc4979700 1 --2- 172.21.15.44:0/2936279803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 0x56252cec69d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40caae0 tx=0x7f9cb4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16acf4018edba5d8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:10.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9c9ffff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:51:10.596 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9c9ffff700 1 -- 172.21.15.44:0/2936279803 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb40a22b0 con 0x56252cea9710 2023-09-10T00:51:10.597 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/2936279803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cb409a9d0 con 0x56252cea9710 2023-09-10T00:51:10.597 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9c9ffff700 1 -- 172.21.15.44:0/2936279803 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f9cb409ee40 con 0x56252cea9710 2023-09-10T00:51:10.597 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9c9ffff700 1 -- 172.21.15.44:0/2936279803 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cb409e100 con 0x56252cea9710 2023-09-10T00:51:10.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cd2da3b80 10 client.17358 fetch_fsmap learned FSMap version 60 2023-09-10T00:51:10.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.579+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/2936279803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x56252cc262b0 con 0x56252cea9710 2023-09-10T00:51:10.598 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.580+0000 7f9c9ffff700 1 -- 172.21.15.44:0/2936279803 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cb409a9d0 con 0x56252cea9710 2023-09-10T00:51:10.599 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.580+0000 7f9c9ffff700 1 -- 172.21.15.44:0/2936279803 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 60) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f9cb40969e0 con 0x56252cea9710 2023-09-10T00:51:10.599 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.580+0000 7f9cd2da3b80 10 client.17358 fetch_fsmap finished waiting for FSMap version 60 2023-09-10T00:51:10.599 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.580+0000 7f9cd2da3b80 10 client.17358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:51:10.600 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.580+0000 7f9cd2da3b80 20 client.17358 populate_metadata read hostname 'smithi044' 2023-09-10T00:51:10.600 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.580+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/2936279803 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56252cf21340 0x56252cf23730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:10.600 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.580+0000 7f9cd2da3b80 4 client.17358 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:51:10.600 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.580+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/2936279803 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56252cdf4920 con 0x56252cf21340 2023-09-10T00:51:10.601 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.580+0000 7f9cbffff700 1 --2- 172.21.15.44:0/2936279803 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56252cf21340 0x56252cf23730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:10.601 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.580+0000 7f9cbffff700 1 --2- 172.21.15.44:0/2936279803 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56252cf21340 0x56252cf23730 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c49ecc0a9853b4b5 server_cookie=a86827f6fa91ac55 in_seq=0 out_seq=0 2023-09-10T00:51:10.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.580+0000 7f9c9ffff700 10 client.17358 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:10.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.581+0000 7f9c9ffff700 1 -- 172.21.15.44:0/2936279803 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56252cdf4920 con 0x56252cf21340 2023-09-10T00:51:10.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.581+0000 7f9c9ffff700 10 client.17358 handle_command_reply: tid=0 2023-09-10T00:51:10.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.581+0000 7f9cd2da3b80 1 client.17358 shutdown 2023-09-10T00:51:10.602 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.581+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/2936279803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 msgr2=0x56252cec69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:10.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.581+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/2936279803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 0x56252cec69d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40caae0 tx=0x7f9cb4009c30 comp rx=0 tx=0).stop 2023-09-10T00:51:10.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.581+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/2936279803 shutdown_connections 2023-09-10T00:51:10.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.581+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/2936279803 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56252cf21340 0x56252cf23730 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.581+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/2936279803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56252ceb7f40 0x56252cec7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.603 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.581+0000 7f9c9ffff700 0 client.17358 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:10.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.581+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/2936279803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56252cea9710 0x56252cec69d0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.581+0000 7f9cd2da3b80 1 --2- 172.21.15.44:0/2936279803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56252ceaa0d0 0x56252cec6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.581+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/2936279803 >> 172.21.15.44:0/2936279803 conn(0x56252ce1e090 msgr2=0x56252cea8700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:10.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.581+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/2936279803 shutdown_connections 2023-09-10T00:51:10.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.581+0000 7f9cd2da3b80 1 -- 172.21.15.44:0/2936279803 wait complete. 2023-09-10T00:51:10.604 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.581+0000 7f9cd2da3b80 20 client.17358 trim_cache size 0 max 16384 2023-09-10T00:51:10.605 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:51:10.605 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:51:10.605 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:51:10.606 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:51:10.606 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:51:10.606 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:51:10.606 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:51:10.606 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:51:10.607 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:51:10.607 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:51:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.582+0000 7f9cc6bdd700 1 -- 172.21.15.44:0/3242577067 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9cb00a0830 msgr2=0x7f9cb00a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.582+0000 7f9cc6bdd700 1 --2- 172.21.15.44:0/3242577067 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9cb00a0830 0x7f9cb00a2cf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0068f80 tx=0x7f9cac009310 comp rx=0 tx=0).stop 2023-09-10T00:51:10.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.582+0000 7f9cc6bdd700 1 -- 172.21.15.44:0/3242577067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc00ff4b0 msgr2=0x7f9cc0197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:10.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.582+0000 7f9cc6bdd700 1 --2- 172.21.15.44:0/3242577067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc00ff4b0 0x7f9cc0197410 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009840 tx=0x7f9cb400fa00 comp rx=0 tx=0).stop 2023-09-10T00:51:10.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.582+0000 7f9cc6bdd700 1 -- 172.21.15.44:0/3242577067 shutdown_connections 2023-09-10T00:51:10.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.582+0000 7f9cc6bdd700 1 --2- 172.21.15.44:0/3242577067 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9cb00a0830 0x7f9cb00a2cf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.609 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.582+0000 7f9cc6bdd700 1 --2- 172.21.15.44:0/3242577067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0105570 0x7f9cc01a2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.582+0000 7f9cc6bdd700 1 --2- 172.21.15.44:0/3242577067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cc00ff4b0 0x7f9cc0197410 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.582+0000 7f9cc6bdd700 1 --2- 172.21.15.44:0/3242577067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc00ffe70 0x7f9cc0197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:10.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.582+0000 7f9cc6bdd700 1 -- 172.21.15.44:0/3242577067 >> 172.21.15.44:0/3242577067 conn(0x7f9cc00fafa0 msgr2=0x7f9cc00fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:10.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.582+0000 7f9cc6bdd700 1 -- 172.21.15.44:0/3242577067 shutdown_connections 2023-09-10T00:51:10.610 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:10.582+0000 7f9cc6bdd700 1 -- 172.21.15.44:0/3242577067 wait complete. 2023-09-10T00:51:10.611 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:51:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:10 smithi131 ceph-mon[103713]: pgmap v327: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 5.7 MiB/s wr, 61 op/s 2023-09-10T00:51:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:10 smithi131 ceph-mon[103713]: osdmap e102: 12 total, 12 up, 12 in 2023-09-10T00:51:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:10 smithi044 ceph-mon[98676]: pgmap v327: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 5.7 MiB/s wr, 61 op/s 2023-09-10T00:51:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:10 smithi044 ceph-mon[98676]: osdmap e102: 12 total, 12 up, 12 in 2023-09-10T00:51:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:10 smithi099 ceph-mon[102692]: pgmap v327: 129 pgs: 129 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 5.7 MiB/s wr, 61 op/s 2023-09-10T00:51:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:10 smithi099 ceph-mon[102692]: osdmap e102: 12 total, 12 up, 12 in 2023-09-10T00:51:11.142 INFO:teuthology.orchestra.run.smithi044.stderr:39300000 of 50000000 tuples (78%) done (elapsed 610.78 s, remaining 166.29 s) 2023-09-10T00:51:12.181 INFO:teuthology.orchestra.run.smithi044.stderr:39400000 of 50000000 tuples (78%) done (elapsed 611.82 s, remaining 164.60 s) 2023-09-10T00:51:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:12 smithi131 ceph-mon[103713]: pgmap v329: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 18 MiB/s wr, 120 op/s 2023-09-10T00:51:13.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:12 smithi044 ceph-mon[98676]: pgmap v329: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 18 MiB/s wr, 120 op/s 2023-09-10T00:51:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:12 smithi099 ceph-mon[102692]: pgmap v329: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 18 MiB/s wr, 120 op/s 2023-09-10T00:51:13.116 INFO:teuthology.orchestra.run.smithi044.stderr:39500000 of 50000000 tuples (79%) done (elapsed 612.75 s, remaining 162.88 s) 2023-09-10T00:51:14.154 INFO:teuthology.orchestra.run.smithi044.stderr:39600000 of 50000000 tuples (79%) done (elapsed 613.79 s, remaining 161.20 s) 2023-09-10T00:51:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.701+0000 7f9b226b0700 1 -- 172.21.15.44:0/2476539313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b1c104410 msgr2=0x7f9b1c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.701+0000 7f9b226b0700 1 --2- 172.21.15.44:0/2476539313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b1c104410 0x7f9b1c1047f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009990 tx=0x7f9b0c009ca0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.701+0000 7f9b226b0700 1 -- 172.21.15.44:0/2476539313 shutdown_connections 2023-09-10T00:51:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.701+0000 7f9b226b0700 1 --2- 172.21.15.44:0/2476539313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b1c0ffe70 0x7f9b1c10c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.701+0000 7f9b226b0700 1 --2- 172.21.15.44:0/2476539313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b1c104410 0x7f9b1c1047f0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.701+0000 7f9b226b0700 1 --2- 172.21.15.44:0/2476539313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b1c0ff4b0 0x7f9b1c0ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.701+0000 7f9b226b0700 1 -- 172.21.15.44:0/2476539313 >> 172.21.15.44:0/2476539313 conn(0x7f9b1c0765b0 msgr2=0x7f9b1c0769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.701+0000 7f9b226b0700 1 -- 172.21.15.44:0/2476539313 shutdown_connections 2023-09-10T00:51:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.701+0000 7f9b226b0700 1 -- 172.21.15.44:0/2476539313 wait complete. 2023-09-10T00:51:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.701+0000 7f9b226b0700 1 Processor -- start 2023-09-10T00:51:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.702+0000 7f9b226b0700 1 -- start start 2023-09-10T00:51:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.702+0000 7f9b226b0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b1c0ff4b0 0x7f9b1c101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.702+0000 7f9b226b0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b1c0ffe70 0x7f9b1c1017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.702+0000 7f9b226b0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b1c104410 0x7f9b1c1a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.702+0000 7f9b226b0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c1a4d00 con 0x7f9b1c104410 2023-09-10T00:51:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.702+0000 7f9b226b0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c1a4e40 con 0x7f9b1c0ffe70 2023-09-10T00:51:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.702+0000 7f9b226b0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c1a4fb0 con 0x7f9b1c0ff4b0 2023-09-10T00:51:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.702+0000 7f9b1bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b1c0ff4b0 0x7f9b1c101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.702+0000 7f9b1bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b1c0ff4b0 0x7f9b1c101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59376/0 (socket says 172.21.15.44:59376) 2023-09-10T00:51:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.702+0000 7f9b1bfff700 1 -- 172.21.15.44:0/296638153 learned_addr learned my addr 172.21.15.44:0/296638153 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.702+0000 7f9b1b7fe700 1 --2- 172.21.15.44:0/296638153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b1c0ffe70 0x7f9b1c1017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.702+0000 7f9b20c4d700 1 --2- 172.21.15.44:0/296638153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b1c104410 0x7f9b1c1a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.703+0000 7f9b1bfff700 1 -- 172.21.15.44:0/296638153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b1c0ffe70 msgr2=0x7f9b1c1017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.703+0000 7f9b1bfff700 1 --2- 172.21.15.44:0/296638153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b1c0ffe70 0x7f9b1c1017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.703+0000 7f9b1bfff700 1 -- 172.21.15.44:0/296638153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b1c104410 msgr2=0x7f9b1c1a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.703+0000 7f9b1bfff700 1 --2- 172.21.15.44:0/296638153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b1c104410 0x7f9b1c1a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.703+0000 7f9b1bfff700 1 -- 172.21.15.44:0/296638153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c0095f0 con 0x7f9b1c0ff4b0 2023-09-10T00:51:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.703+0000 7f9b1bfff700 1 --2- 172.21.15.44:0/296638153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b1c0ff4b0 0x7f9b1c101290 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c005910 tx=0x7f9b0c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.703+0000 7f9b197fa700 1 -- 172.21.15.44:0/296638153 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c01d070 con 0x7f9b1c0ff4b0 2023-09-10T00:51:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.703+0000 7f9b197fa700 1 -- 172.21.15.44:0/296638153 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b0c021410 con 0x7f9b1c0ff4b0 2023-09-10T00:51:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.703+0000 7f9b226b0700 1 -- 172.21.15.44:0/296638153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1c1a5230 con 0x7f9b1c0ff4b0 2023-09-10T00:51:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.703+0000 7f9b197fa700 1 -- 172.21.15.44:0/296638153 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c017650 con 0x7f9b1c0ff4b0 2023-09-10T00:51:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.703+0000 7f9b226b0700 1 -- 172.21.15.44:0/296638153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1c1a56e0 con 0x7f9b1c0ff4b0 2023-09-10T00:51:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.706+0000 7f9b197fa700 1 -- 172.21.15.44:0/296638153 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f9b0c021580 con 0x7f9b1c0ff4b0 2023-09-10T00:51:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.706+0000 7f9b197fa700 1 --2- 172.21.15.44:0/296638153 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9b040a07b0 0x7f9b040a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.707+0000 7f9b197fa700 1 -- 172.21.15.44:0/296638153 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f9b0c0cd4e0 con 0x7f9b1c0ff4b0 2023-09-10T00:51:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.707+0000 7f9b1b7fe700 1 --2- 172.21.15.44:0/296638153 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9b040a07b0 0x7f9b040a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.707+0000 7f9b2e876b80 1 Processor -- start 2023-09-10T00:51:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.707+0000 7f9b2e876b80 1 -- start start 2023-09-10T00:51:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.707+0000 7f9b1b7fe700 1 --2- 172.21.15.44:0/296638153 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9b040a07b0 0x7f9b040a2c70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c102ab0 tx=0x7f9b10009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.707+0000 7f9b2e876b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf61a0 0x556aedcf6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.707+0000 7f9b2e876b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0300 0x556aedcf0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.707+0000 7f9b2e876b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf0cc0 0x556aedcfcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.707+0000 7f9b2e876b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aedc496d0 con 0x556aedcf0cc0 2023-09-10T00:51:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.707+0000 7f9b2e876b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aedc46ff0 con 0x556aedcf0300 2023-09-10T00:51:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.707+0000 7f9b2e876b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aedc428e0 con 0x556aedcf61a0 2023-09-10T00:51:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.707+0000 7f9b1b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0300 0x556aedcf0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.707+0000 7f9b1b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0300 0x556aedcf0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54244/0 (socket says 172.21.15.44:54244) 2023-09-10T00:51:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.707+0000 7f9b1bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf61a0 0x556aedcf6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.707+0000 7f9b1b7fe700 1 -- 172.21.15.44:0/1876707112 learned_addr learned my addr 172.21.15.44:0/1876707112 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.708+0000 7f9b20c4d700 1 --2- 172.21.15.44:0/1876707112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf0cc0 0x556aedcfcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.708+0000 7f9b1b7fe700 1 -- 172.21.15.44:0/1876707112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf61a0 msgr2=0x556aedcf6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.708+0000 7f9b1b7fe700 1 --2- 172.21.15.44:0/1876707112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf61a0 0x556aedcf6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.708+0000 7f9b1b7fe700 1 -- 172.21.15.44:0/1876707112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf0cc0 msgr2=0x556aedcfcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.708+0000 7f9b1b7fe700 1 --2- 172.21.15.44:0/1876707112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf0cc0 0x556aedcfcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.708+0000 7f9b1b7fe700 1 -- 172.21.15.44:0/1876707112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556aeda6ed80 con 0x556aedcf0300 2023-09-10T00:51:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.708+0000 7f9b1b7fe700 1 --2- 172.21.15.44:0/1876707112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0300 0x556aedcf0780 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9b10011d50 tx=0x7f9b10014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a8545ef5d502296 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.708+0000 7f9b027fc700 1 -- 172.21.15.44:0/1876707112 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1000eb20 con 0x556aedcf0300 2023-09-10T00:51:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.708+0000 7f9b027fc700 1 -- 172.21.15.44:0/1876707112 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b10021070 con 0x556aedcf0300 2023-09-10T00:51:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.708+0000 7f9b027fc700 1 -- 172.21.15.44:0/1876707112 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1001d410 con 0x556aedcf0300 2023-09-10T00:51:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.709+0000 7f9b2e876b80 1 -- 172.21.15.44:0/1876707112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0300 msgr2=0x556aedcf0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.709+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/1876707112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0300 0x556aedcf0780 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9b10011d50 tx=0x7f9b10014000 comp rx=0 tx=0).stop 2023-09-10T00:51:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.709+0000 7f9b2e876b80 1 -- 172.21.15.44:0/1876707112 shutdown_connections 2023-09-10T00:51:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.709+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/1876707112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf0cc0 0x556aedcfcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.709+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/1876707112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf61a0 0x556aedcf6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.709+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/1876707112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0300 0x556aedcf0780 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.709+0000 7f9b2e876b80 1 -- 172.21.15.44:0/1876707112 >> 172.21.15.44:0/1876707112 conn(0x556aedc65030 msgr2=0x556aedceb560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.709+0000 7f9b2e876b80 1 -- 172.21.15.44:0/1876707112 shutdown_connections 2023-09-10T00:51:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.709+0000 7f9b2e876b80 1 -- 172.21.15.44:0/1876707112 wait complete. 2023-09-10T00:51:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.709+0000 7f9b2e876b80 1 -- start start 2023-09-10T00:51:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.710+0000 7f9b2e876b80 1 Processor -- start 2023-09-10T00:51:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.710+0000 7f9b2e876b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 0x556aedcfce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.710+0000 7f9b2e876b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0cc0 0x556aedcfa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.710+0000 7f9b2e876b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf61a0 0x556aedcfa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.710+0000 7f9b2e876b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aedbe0b10 con 0x556aedcf61a0 2023-09-10T00:51:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.710+0000 7f9b2e876b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aedbe0810 con 0x556aedcf0cc0 2023-09-10T00:51:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.710+0000 7f9b2e876b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aedac48c0 con 0x556aedcf0300 2023-09-10T00:51:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.710+0000 7f9b2e876b80 10 client.0 _ll_register_callbacks cb 0x556aedb2aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:51:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.710+0000 7f9b20c4d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf61a0 0x556aedcfa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.710+0000 7f9b1b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0cc0 0x556aedcfa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.710+0000 7f9b1bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 0x556aedcfce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.710+0000 7f9b1bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 0x556aedcfce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59390/0 (socket says 172.21.15.44:59390) 2023-09-10T00:51:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.710+0000 7f9b1bfff700 1 -- 172.21.15.44:0/1014507982 learned_addr learned my addr 172.21.15.44:0/1014507982 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.710+0000 7f9b1b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0cc0 0x556aedcfa0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54250/0 (socket says 172.21.15.44:54250) 2023-09-10T00:51:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.710+0000 7f9b20c4d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf61a0 0x556aedcfa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52512/0 (socket says 172.21.15.44:52512) 2023-09-10T00:51:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.711+0000 7f9b1bfff700 1 -- 172.21.15.44:0/1014507982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0cc0 msgr2=0x556aedcfa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.711+0000 7f9b1bfff700 1 --2- 172.21.15.44:0/1014507982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0cc0 0x556aedcfa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.711+0000 7f9b1bfff700 1 -- 172.21.15.44:0/1014507982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf61a0 msgr2=0x556aedcfa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.711+0000 7f9b1bfff700 1 --2- 172.21.15.44:0/1014507982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf61a0 0x556aedcfa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.711+0000 7f9b1bfff700 1 -- 172.21.15.44:0/1014507982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b10017070 con 0x556aedcf0300 2023-09-10T00:51:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.711+0000 7f9b20c4d700 1 --2- 172.21.15.44:0/1014507982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf61a0 0x556aedcfa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.711+0000 7f9b1b7fe700 1 --2- 172.21.15.44:0/1014507982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0cc0 0x556aedcfa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.711+0000 7f9b1bfff700 1 --2- 172.21.15.44:0/1014507982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 0x556aedcfce00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c005cd0 tx=0x7f9b0c099ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87c61b9f1f68afc6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.711+0000 7f9afb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:51:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.711+0000 7f9afb7fe700 1 -- 172.21.15.44:0/1014507982 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0852d0 con 0x556aedcf0300 2023-09-10T00:51:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.711+0000 7f9afb7fe700 1 -- 172.21.15.44:0/1014507982 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f9b0c0a0040 con 0x556aedcf0300 2023-09-10T00:51:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.711+0000 7f9b2e876b80 1 -- 172.21.15.44:0/1014507982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556aedb9a490 con 0x556aedcf0300 2023-09-10T00:51:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.712+0000 7f9afb7fe700 1 -- 172.21.15.44:0/1014507982 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b0c09e450 con 0x556aedcf0300 2023-09-10T00:51:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.712+0000 7f9b2e876b80 10 client.26974 fetch_fsmap learned FSMap version 60 2023-09-10T00:51:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.712+0000 7f9b2e876b80 1 -- 172.21.15.44:0/1014507982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x556aeda57c00 con 0x556aedcf0300 2023-09-10T00:51:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.712+0000 7f9afb7fe700 1 -- 172.21.15.44:0/1014507982 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556aedb9a490 con 0x556aedcf0300 2023-09-10T00:51:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.712+0000 7f9afb7fe700 1 -- 172.21.15.44:0/1014507982 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 60) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f9b0c099b10 con 0x556aedcf0300 2023-09-10T00:51:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.712+0000 7f9b2e876b80 10 client.26974 fetch_fsmap finished waiting for FSMap version 60 2023-09-10T00:51:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.712+0000 7f9b2e876b80 10 client.26974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:51:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.712+0000 7f9b2e876b80 20 client.26974 populate_metadata read hostname 'smithi044' 2023-09-10T00:51:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.712+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/1014507982 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556aedd0a100 0x556aedd0c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.712+0000 7f9b2e876b80 4 client.26974 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:51:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.712+0000 7f9b2e876b80 1 -- 172.21.15.44:0/1014507982 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556aedc57840 con 0x556aedd0a100 2023-09-10T00:51:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.712+0000 7f9b1b7fe700 1 --2- 172.21.15.44:0/1014507982 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556aedd0a100 0x556aedd0c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.713+0000 7f9afb7fe700 10 client.26974 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:14.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.713+0000 7f9b1b7fe700 1 --2- 172.21.15.44:0/1014507982 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556aedd0a100 0x556aedd0c520 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df1b96591becb7b7 server_cookie=6ed34e0faa0838e7 in_seq=0 out_seq=0 2023-09-10T00:51:14.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.713+0000 7f9afb7fe700 1 -- 172.21.15.44:0/1014507982 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556aedc57840 con 0x556aedd0a100 2023-09-10T00:51:14.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.713+0000 7f9afb7fe700 10 client.26974 handle_command_reply: tid=0 2023-09-10T00:51:14.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.713+0000 7f9b2e876b80 1 client.26974 shutdown 2023-09-10T00:51:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.714+0000 7f9b2e876b80 1 -- 172.21.15.44:0/1014507982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 msgr2=0x556aedcfce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.714+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/1014507982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 0x556aedcfce00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c005cd0 tx=0x7f9b0c099ff0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.714+0000 7f9b2e876b80 1 -- 172.21.15.44:0/1014507982 shutdown_connections 2023-09-10T00:51:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.714+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/1014507982 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556aedd0a100 0x556aedd0c520 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.714+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/1014507982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf61a0 0x556aedcfa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.714+0000 7f9afb7fe700 0 client.26974 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.714+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/1014507982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 0x556aedcfce00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.714+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/1014507982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0cc0 0x556aedcfa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.714+0000 7f9b2e876b80 1 -- 172.21.15.44:0/1014507982 >> 172.21.15.44:0/1014507982 conn(0x556aedc65030 msgr2=0x556aedcea7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.714+0000 7f9b2e876b80 1 -- 172.21.15.44:0/1014507982 shutdown_connections 2023-09-10T00:51:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.714+0000 7f9b2e876b80 1 -- 172.21.15.44:0/1014507982 wait complete. 2023-09-10T00:51:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.714+0000 7f9b2e876b80 20 client.26974 trim_cache size 0 max 16384 2023-09-10T00:51:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b2e876b80 1 Processor -- start 2023-09-10T00:51:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b2e876b80 1 -- start start 2023-09-10T00:51:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b2e876b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 0x556aedcfa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b2e876b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf0cc0 0x556aedd01530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b2e876b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf61a0 0x556aedcfc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b2e876b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aedcfb6b0 con 0x556aedcf0cc0 2023-09-10T00:51:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b2e876b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aedcfb100 con 0x556aedcf61a0 2023-09-10T00:51:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b2e876b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aedd01aa0 con 0x556aedcf0300 2023-09-10T00:51:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b1b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf0cc0 0x556aedd01530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b1b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf0cc0 0x556aedd01530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52518/0 (socket says 172.21.15.44:52518) 2023-09-10T00:51:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b1b7fe700 1 -- 172.21.15.44:0/2212055173 learned_addr learned my addr 172.21.15.44:0/2212055173 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b20c4d700 1 --2- 172.21.15.44:0/2212055173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf61a0 0x556aedcfc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b1bfff700 1 --2- 172.21.15.44:0/2212055173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 0x556aedcfa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b1b7fe700 1 -- 172.21.15.44:0/2212055173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 msgr2=0x556aedcfa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b1b7fe700 1 --2- 172.21.15.44:0/2212055173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 0x556aedcfa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b1b7fe700 1 -- 172.21.15.44:0/2212055173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf61a0 msgr2=0x556aedcfc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b1b7fe700 1 --2- 172.21.15.44:0/2212055173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf61a0 0x556aedcfc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b1b7fe700 1 -- 172.21.15.44:0/2212055173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556aeda951f0 con 0x556aedcf0cc0 2023-09-10T00:51:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.734+0000 7f9b20c4d700 1 --2- 172.21.15.44:0/2212055173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf61a0 0x556aedcfc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.735+0000 7f9b1b7fe700 1 --2- 172.21.15.44:0/2212055173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf0cc0 0x556aedd01530 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x556aedceb060 tx=0x7f9b10011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98aaee571961d24a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.735+0000 7f9b02ffd700 1 -- 172.21.15.44:0/2212055173 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b10011df0 con 0x556aedcf0cc0 2023-09-10T00:51:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.735+0000 7f9b02ffd700 1 -- 172.21.15.44:0/2212055173 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b1001cce0 con 0x556aedcf0cc0 2023-09-10T00:51:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.735+0000 7f9b02ffd700 1 -- 172.21.15.44:0/2212055173 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b100320d0 con 0x556aedcf0cc0 2023-09-10T00:51:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.736+0000 7f9b2e876b80 1 -- 172.21.15.44:0/2212055173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf0cc0 msgr2=0x556aedd01530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.736+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/2212055173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf0cc0 0x556aedd01530 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x556aedceb060 tx=0x7f9b10011d80 comp rx=0 tx=0).stop 2023-09-10T00:51:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.736+0000 7f9b2e876b80 1 -- 172.21.15.44:0/2212055173 shutdown_connections 2023-09-10T00:51:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.736+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/2212055173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf61a0 0x556aedcfc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.736+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/2212055173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 0x556aedcfa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.736+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/2212055173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf0cc0 0x556aedd01530 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.736+0000 7f9b2e876b80 1 -- 172.21.15.44:0/2212055173 >> 172.21.15.44:0/2212055173 conn(0x556aedc65030 msgr2=0x556aedcf41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.736+0000 7f9b2e876b80 1 -- 172.21.15.44:0/2212055173 shutdown_connections 2023-09-10T00:51:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.736+0000 7f9b2e876b80 1 -- 172.21.15.44:0/2212055173 wait complete. 2023-09-10T00:51:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.736+0000 7f9b2e876b80 1 -- start start 2023-09-10T00:51:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.736+0000 7f9b2e876b80 1 Processor -- start 2023-09-10T00:51:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b2e876b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 0x556aedd51e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b2e876b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0cc0 0x556aedd523c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b2e876b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf61a0 0x556aedd52900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b2e876b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aedd5cfe0 con 0x556aedcf61a0 2023-09-10T00:51:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b2e876b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aedd5d120 con 0x556aedcf0cc0 2023-09-10T00:51:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b2e876b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aedd5d2e0 con 0x556aedcf0300 2023-09-10T00:51:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b2e876b80 10 client.0 _ll_register_callbacks cb 0x7f9b0c09a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:51:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b20c4d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf61a0 0x556aedd52900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b1b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0cc0 0x556aedd523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b20c4d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf61a0 0x556aedd52900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52526/0 (socket says 172.21.15.44:52526) 2023-09-10T00:51:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b1bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 0x556aedd51e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b20c4d700 1 -- 172.21.15.44:0/422290477 learned_addr learned my addr 172.21.15.44:0/422290477 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b1b7fe700 1 -- 172.21.15.44:0/422290477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 msgr2=0x556aedd51e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b1b7fe700 1 --2- 172.21.15.44:0/422290477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 0x556aedd51e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b1b7fe700 1 -- 172.21.15.44:0/422290477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf61a0 msgr2=0x556aedd52900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b1b7fe700 1 --2- 172.21.15.44:0/422290477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf61a0 0x556aedd52900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b1b7fe700 1 -- 172.21.15.44:0/422290477 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b140118c0 con 0x556aedcf0cc0 2023-09-10T00:51:14.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b20c4d700 1 --2- 172.21.15.44:0/422290477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf61a0 0x556aedd52900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:14.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9b1b7fe700 1 --2- 172.21.15.44:0/422290477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0cc0 0x556aedd523c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x556aedbe0bb0 tx=0x7f9b10005cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f20fd8c594858405 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:14.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.737+0000 7f9afb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:51:14.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.738+0000 7f9afb7fe700 1 -- 172.21.15.44:0/422290477 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b10032c20 con 0x556aedcf0cc0 2023-09-10T00:51:14.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.738+0000 7f9b2e876b80 1 -- 172.21.15.44:0/422290477 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b10021070 con 0x556aedcf0cc0 2023-09-10T00:51:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.738+0000 7f9afb7fe700 1 -- 172.21.15.44:0/422290477 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f9b10032d80 con 0x556aedcf0cc0 2023-09-10T00:51:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.738+0000 7f9afb7fe700 1 -- 172.21.15.44:0/422290477 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b1000ea20 con 0x556aedcf0cc0 2023-09-10T00:51:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.738+0000 7f9b2e876b80 10 client.27044 fetch_fsmap learned FSMap version 60 2023-09-10T00:51:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.738+0000 7f9b2e876b80 1 -- 172.21.15.44:0/422290477 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x7f9b1002a040 con 0x556aedcf0cc0 2023-09-10T00:51:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.738+0000 7f9afb7fe700 1 -- 172.21.15.44:0/422290477 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b10021070 con 0x556aedcf0cc0 2023-09-10T00:51:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.738+0000 7f9afb7fe700 1 -- 172.21.15.44:0/422290477 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 60) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f9b10043070 con 0x556aedcf0cc0 2023-09-10T00:51:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.738+0000 7f9b2e876b80 10 client.27044 fetch_fsmap finished waiting for FSMap version 60 2023-09-10T00:51:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.738+0000 7f9b2e876b80 10 client.27044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:51:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.738+0000 7f9b2e876b80 20 client.27044 populate_metadata read hostname 'smithi044' 2023-09-10T00:51:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.738+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/422290477 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556aedd5e7d0 0x556aedd60bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.738+0000 7f9b2e876b80 4 client.27044 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:51:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.738+0000 7f9b2e876b80 1 -- 172.21.15.44:0/422290477 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556aedc3b8c0 con 0x556aedd5e7d0 2023-09-10T00:51:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.738+0000 7f9b1bfff700 1 --2- 172.21.15.44:0/422290477 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556aedd5e7d0 0x556aedd60bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.739+0000 7f9afb7fe700 10 client.27044 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.739+0000 7f9b1bfff700 1 --2- 172.21.15.44:0/422290477 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556aedd5e7d0 0x556aedd60bf0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56669fd0fd7384a3 server_cookie=8c2cc78cef113116 in_seq=0 out_seq=0 2023-09-10T00:51:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.754+0000 7f9afb7fe700 1 -- 172.21.15.44:0/422290477 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556aedc3b8c0 con 0x556aedd5e7d0 2023-09-10T00:51:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.754+0000 7f9afb7fe700 10 client.27044 handle_command_reply: tid=0 2023-09-10T00:51:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.754+0000 7f9b2e876b80 1 client.27044 shutdown 2023-09-10T00:51:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.754+0000 7f9b2e876b80 1 -- 172.21.15.44:0/422290477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0cc0 msgr2=0x556aedd523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.754+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/422290477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0cc0 0x556aedd523c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x556aedbe0bb0 tx=0x7f9b10005cb0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.754+0000 7f9b2e876b80 1 -- 172.21.15.44:0/422290477 shutdown_connections 2023-09-10T00:51:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/422290477 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556aedd5e7d0 0x556aedd60bf0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/422290477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556aedcf61a0 0x556aedd52900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9afb7fe700 0 client.27044 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/422290477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556aedcf0300 0x556aedd51e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b2e876b80 1 --2- 172.21.15.44:0/422290477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556aedcf0cc0 0x556aedd523c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b2e876b80 1 -- 172.21.15.44:0/422290477 >> 172.21.15.44:0/422290477 conn(0x556aedc65030 msgr2=0x556aedcf50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b2e876b80 1 -- 172.21.15.44:0/422290477 shutdown_connections 2023-09-10T00:51:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b2e876b80 1 -- 172.21.15.44:0/422290477 wait complete. 2023-09-10T00:51:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b2e876b80 20 client.27044 trim_cache size 0 max 16384 2023-09-10T00:51:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:51:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:51:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:51:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:51:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b226b0700 1 -- 172.21.15.44:0/296638153 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9b040a07b0 msgr2=0x7f9b040a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b226b0700 1 --2- 172.21.15.44:0/296638153 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9b040a07b0 0x7f9b040a2c70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c102ab0 tx=0x7f9b10009260 comp rx=0 tx=0).stop 2023-09-10T00:51:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b226b0700 1 -- 172.21.15.44:0/296638153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b1c0ff4b0 msgr2=0x7f9b1c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b226b0700 1 --2- 172.21.15.44:0/296638153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b1c0ff4b0 0x7f9b1c101290 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c005910 tx=0x7f9b0c0047b0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b226b0700 1 -- 172.21.15.44:0/296638153 shutdown_connections 2023-09-10T00:51:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b226b0700 1 --2- 172.21.15.44:0/296638153 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9b040a07b0 0x7f9b040a2c70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b226b0700 1 --2- 172.21.15.44:0/296638153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b1c104410 0x7f9b1c1a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b226b0700 1 --2- 172.21.15.44:0/296638153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b1c0ff4b0 0x7f9b1c101290 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b226b0700 1 --2- 172.21.15.44:0/296638153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b1c0ffe70 0x7f9b1c1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.755+0000 7f9b226b0700 1 -- 172.21.15.44:0/296638153 >> 172.21.15.44:0/296638153 conn(0x7f9b1c0765b0 msgr2=0x7f9b1c068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.756+0000 7f9b226b0700 1 -- 172.21.15.44:0/296638153 shutdown_connections 2023-09-10T00:51:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:14.756+0000 7f9b226b0700 1 -- 172.21.15.44:0/296638153 wait complete. 2023-09-10T00:51:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:14 smithi131 ceph-mon[103713]: pgmap v330: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 15 MiB/s wr, 102 op/s 2023-09-10T00:51:15.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:14 smithi044 ceph-mon[98676]: pgmap v330: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 15 MiB/s wr, 102 op/s 2023-09-10T00:51:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:14 smithi099 ceph-mon[102692]: pgmap v330: 129 pgs: 129 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 15 MiB/s wr, 102 op/s 2023-09-10T00:51:15.249 INFO:teuthology.orchestra.run.smithi044.stderr:39700000 of 50000000 tuples (79%) done (elapsed 614.88 s, remaining 159.53 s) 2023-09-10T00:51:16.220 INFO:teuthology.orchestra.run.smithi044.stderr:39800000 of 50000000 tuples (79%) done (elapsed 615.85 s, remaining 157.83 s) 2023-09-10T00:51:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:16 smithi131 ceph-mon[103713]: pgmap v331: 129 pgs: 129 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 112 op/s 2023-09-10T00:51:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:16 smithi044 ceph-mon[98676]: pgmap v331: 129 pgs: 129 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 112 op/s 2023-09-10T00:51:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:16 smithi099 ceph-mon[102692]: pgmap v331: 129 pgs: 129 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 112 op/s 2023-09-10T00:51:17.142 INFO:teuthology.orchestra.run.smithi044.stderr:39900000 of 50000000 tuples (79%) done (elapsed 616.78 s, remaining 156.13 s) 2023-09-10T00:51:18.019 INFO:teuthology.orchestra.run.smithi044.stderr:40000000 of 50000000 tuples (80%) done (elapsed 617.65 s, remaining 154.41 s) 2023-09-10T00:51:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:40100000 of 50000000 tuples (80%) done (elapsed 618.50 s, remaining 152.70 s) 2023-09-10T00:51:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:18 smithi131 ceph-mon[103713]: pgmap v332: 129 pgs: 129 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 112 op/s 2023-09-10T00:51:19.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:18 smithi044 ceph-mon[98676]: pgmap v332: 129 pgs: 129 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 112 op/s 2023-09-10T00:51:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:18 smithi099 ceph-mon[102692]: pgmap v332: 129 pgs: 129 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 112 op/s 2023-09-10T00:51:19.904 INFO:teuthology.orchestra.run.smithi044.stderr:40200000 of 50000000 tuples (80%) done (elapsed 619.54 s, remaining 151.03 s) 2023-09-10T00:51:20.836 INFO:teuthology.orchestra.run.smithi044.stderr:40300000 of 50000000 tuples (80%) done (elapsed 620.47 s, remaining 149.34 s) 2023-09-10T00:51:21.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:20 smithi131 ceph-mon[103713]: pgmap v333: 129 pgs: 129 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 112 op/s 2023-09-10T00:51:21.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:20 smithi044 ceph-mon[98676]: pgmap v333: 129 pgs: 129 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 112 op/s 2023-09-10T00:51:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:20 smithi099 ceph-mon[102692]: pgmap v333: 129 pgs: 129 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 112 op/s 2023-09-10T00:51:21.770 INFO:teuthology.orchestra.run.smithi044.stderr:40400000 of 50000000 tuples (80%) done (elapsed 621.40 s, remaining 147.66 s) 2023-09-10T00:51:22.749 INFO:teuthology.orchestra.run.smithi044.stderr:40500000 of 50000000 tuples (81%) done (elapsed 622.38 s, remaining 145.99 s) 2023-09-10T00:51:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:22 smithi131 ceph-mon[103713]: pgmap v334: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 MiB/s wr, 94 op/s 2023-09-10T00:51:23.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:22 smithi044 ceph-mon[98676]: pgmap v334: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 MiB/s wr, 94 op/s 2023-09-10T00:51:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:22 smithi099 ceph-mon[102692]: pgmap v334: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 MiB/s wr, 94 op/s 2023-09-10T00:51:23.664 INFO:teuthology.orchestra.run.smithi044.stderr:40600000 of 50000000 tuples (81%) done (elapsed 623.30 s, remaining 144.31 s) 2023-09-10T00:51:24.520 INFO:teuthology.orchestra.run.smithi044.stderr:40700000 of 50000000 tuples (81%) done (elapsed 624.15 s, remaining 142.62 s) 2023-09-10T00:51:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:24 smithi131 ceph-mon[103713]: pgmap v335: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 83 op/s 2023-09-10T00:51:25.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:24 smithi044 ceph-mon[98676]: pgmap v335: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 83 op/s 2023-09-10T00:51:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:24 smithi099 ceph-mon[102692]: pgmap v335: 129 pgs: 129 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 83 op/s 2023-09-10T00:51:25.626 INFO:teuthology.orchestra.run.smithi044.stderr:40800000 of 50000000 tuples (81%) done (elapsed 625.26 s, remaining 140.99 s) 2023-09-10T00:51:26.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:25 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:51:26.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:25 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:51:26.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:25 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:51:26.644 INFO:teuthology.orchestra.run.smithi044.stderr:40900000 of 50000000 tuples (81%) done (elapsed 626.28 s, remaining 139.34 s) 2023-09-10T00:51:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:26 smithi131 ceph-mon[103713]: pgmap v336: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 MiB/s wr, 112 op/s 2023-09-10T00:51:27.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:26 smithi044 ceph-mon[98676]: pgmap v336: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 MiB/s wr, 112 op/s 2023-09-10T00:51:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:26 smithi099 ceph-mon[102692]: pgmap v336: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 MiB/s wr, 112 op/s 2023-09-10T00:51:27.634 INFO:teuthology.orchestra.run.smithi044.stderr:41000000 of 50000000 tuples (82%) done (elapsed 627.27 s, remaining 137.69 s) 2023-09-10T00:51:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.913+0000 7fb602db2700 1 -- 172.21.15.44:0/615545759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5fc103f90 msgr2=0x7fb5fc10bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.913+0000 7fb602db2700 1 --2- 172.21.15.44:0/615545759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5fc103f90 0x7fb5fc10bb10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009f90 tx=0x7fb5f400b330 comp rx=0 tx=0).stop 2023-09-10T00:51:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.913+0000 7fb602db2700 1 -- 172.21.15.44:0/615545759 shutdown_connections 2023-09-10T00:51:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.913+0000 7fb602db2700 1 --2- 172.21.15.44:0/615545759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5fc103f90 0x7fb5fc10bb10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.913+0000 7fb602db2700 1 --2- 172.21.15.44:0/615545759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5fc0ff080 0x7fb5fc0ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.913+0000 7fb602db2700 1 --2- 172.21.15.44:0/615545759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5fc0ffa30 0x7fb5fc103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.913+0000 7fb602db2700 1 -- 172.21.15.44:0/615545759 >> 172.21.15.44:0/615545759 conn(0x7fb5fc075710 msgr2=0x7fb5fc075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.913+0000 7fb602db2700 1 -- 172.21.15.44:0/615545759 shutdown_connections 2023-09-10T00:51:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.913+0000 7fb602db2700 1 -- 172.21.15.44:0/615545759 wait complete. 2023-09-10T00:51:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb602db2700 1 Processor -- start 2023-09-10T00:51:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb602db2700 1 -- start start 2023-09-10T00:51:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb602db2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5fc0ff080 0x7fb5fc1954a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb602db2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5fc103f90 0x7fb5fc1959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb602db2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5fc1a2930 0x7fb5fc1a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb602db2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc196180 con 0x7fb5fc103f90 2023-09-10T00:51:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb602db2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc1a5260 con 0x7fb5fc0ff080 2023-09-10T00:51:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb602db2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc1a53f0 con 0x7fb5fc1a2930 2023-09-10T00:51:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb600b4e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5fc0ff080 0x7fb5fc1954a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb5fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5fc103f90 0x7fb5fc1959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb600b4e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5fc0ff080 0x7fb5fc1954a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54276/0 (socket says 172.21.15.44:54276) 2023-09-10T00:51:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb600b4e700 1 -- 172.21.15.44:0/4244515959 learned_addr learned my addr 172.21.15.44:0/4244515959 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb60134f700 1 --2- 172.21.15.44:0/4244515959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5fc1a2930 0x7fb5fc1a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb600b4e700 1 -- 172.21.15.44:0/4244515959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5fc1a2930 msgr2=0x7fb5fc1a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb600b4e700 1 --2- 172.21.15.44:0/4244515959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5fc1a2930 0x7fb5fc1a4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb600b4e700 1 -- 172.21.15.44:0/4244515959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5fc103f90 msgr2=0x7fb5fc1959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.914+0000 7fb600b4e700 1 --2- 172.21.15.44:0/4244515959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5fc103f90 0x7fb5fc1959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.915+0000 7fb600b4e700 1 -- 172.21.15.44:0/4244515959 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f4009c40 con 0x7fb5fc0ff080 2023-09-10T00:51:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.915+0000 7fb5fbfff700 1 --2- 172.21.15.44:0/4244515959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5fc103f90 0x7fb5fc1959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.915+0000 7fb600b4e700 1 --2- 172.21.15.44:0/4244515959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5fc0ff080 0x7fb5fc1954a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000e9a0 tx=0x7fb5f000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.915+0000 7fb5f9ffb700 1 -- 172.21.15.44:0/4244515959 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0009790 con 0x7fb5fc0ff080 2023-09-10T00:51:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.915+0000 7fb5f9ffb700 1 -- 172.21.15.44:0/4244515959 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5f000ce50 con 0x7fb5fc0ff080 2023-09-10T00:51:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.915+0000 7fb602db2700 1 -- 172.21.15.44:0/4244515959 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc1a56d0 con 0x7fb5fc0ff080 2023-09-10T00:51:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.915+0000 7fb5f9ffb700 1 -- 172.21.15.44:0/4244515959 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0018420 con 0x7fb5fc0ff080 2023-09-10T00:51:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.915+0000 7fb602db2700 1 -- 172.21.15.44:0/4244515959 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc1a5be0 con 0x7fb5fc0ff080 2023-09-10T00:51:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.917+0000 7fb5f9ffb700 1 -- 172.21.15.44:0/4244515959 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fb5f0010430 con 0x7fb5fc0ff080 2023-09-10T00:51:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.918+0000 7fb5f9ffb700 1 --2- 172.21.15.44:0/4244515959 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5ec0a07b0 0x7fb5ec0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.918+0000 7fb5f9ffb700 1 -- 172.21.15.44:0/4244515959 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb5f0014070 con 0x7fb5fc0ff080 2023-09-10T00:51:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.918+0000 7fb5fbfff700 1 --2- 172.21.15.44:0/4244515959 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5ec0a07b0 0x7fb5ec0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.919+0000 7fb5fbfff700 1 --2- 172.21.15.44:0/4244515959 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5ec0a07b0 0x7fb5ec0a2c70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc10cef0 tx=0x7fb5e8009310 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.919+0000 7fb60ef78b80 1 Processor -- start 2023-09-10T00:51:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.919+0000 7fb60ef78b80 1 -- start start 2023-09-10T00:51:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb60ef78b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d31c1a0 0x55cb8d31c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb60ef78b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316300 0x55cb8d316780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb60ef78b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d316cc0 0x55cb8d322f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb60ef78b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8d26f6d0 con 0x55cb8d316300 2023-09-10T00:51:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb60ef78b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8d26cff0 con 0x55cb8d316cc0 2023-09-10T00:51:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb60ef78b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8d2688e0 con 0x55cb8d31c1a0 2023-09-10T00:51:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb5fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316300 0x55cb8d316780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb5fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316300 0x55cb8d316780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52546/0 (socket says 172.21.15.44:52546) 2023-09-10T00:51:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb600b4e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d31c1a0 0x55cb8d31c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb5fbfff700 1 -- 172.21.15.44:0/3236844688 learned_addr learned my addr 172.21.15.44:0/3236844688 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb60134f700 1 --2- 172.21.15.44:0/3236844688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d316cc0 0x55cb8d322f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb600b4e700 1 -- 172.21.15.44:0/3236844688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d316cc0 msgr2=0x55cb8d322f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb600b4e700 1 --2- 172.21.15.44:0/3236844688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d316cc0 0x55cb8d322f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb600b4e700 1 -- 172.21.15.44:0/3236844688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316300 msgr2=0x55cb8d316780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb600b4e700 1 --2- 172.21.15.44:0/3236844688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316300 0x55cb8d316780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb600b4e700 1 -- 172.21.15.44:0/3236844688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb8d094d80 con 0x55cb8d31c1a0 2023-09-10T00:51:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb5fbfff700 1 --2- 172.21.15.44:0/3236844688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316300 0x55cb8d316780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.920+0000 7fb600b4e700 1 --2- 172.21.15.44:0/3236844688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d31c1a0 0x55cb8d31c580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00c98c0 tx=0x7fb5f0099070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d92c06164ddab542 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.921+0000 7fb5e6ffd700 1 -- 172.21.15.44:0/3236844688 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0099620 con 0x55cb8d31c1a0 2023-09-10T00:51:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.921+0000 7fb5e6ffd700 1 -- 172.21.15.44:0/3236844688 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5f009d450 con 0x55cb8d31c1a0 2023-09-10T00:51:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.921+0000 7fb5e6ffd700 1 -- 172.21.15.44:0/3236844688 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0084310 con 0x55cb8d31c1a0 2023-09-10T00:51:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.921+0000 7fb60ef78b80 1 -- 172.21.15.44:0/3236844688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d31c1a0 msgr2=0x55cb8d31c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.921+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/3236844688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d31c1a0 0x55cb8d31c580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00c98c0 tx=0x7fb5f0099070 comp rx=0 tx=0).stop 2023-09-10T00:51:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.921+0000 7fb60ef78b80 1 -- 172.21.15.44:0/3236844688 shutdown_connections 2023-09-10T00:51:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.921+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/3236844688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d316cc0 0x55cb8d322f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.921+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/3236844688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d31c1a0 0x55cb8d31c580 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.921+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/3236844688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316300 0x55cb8d316780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.921+0000 7fb60ef78b80 1 -- 172.21.15.44:0/3236844688 >> 172.21.15.44:0/3236844688 conn(0x55cb8d28b030 msgr2=0x55cb8d311560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.921+0000 7fb60ef78b80 1 -- 172.21.15.44:0/3236844688 shutdown_connections 2023-09-10T00:51:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.921+0000 7fb60ef78b80 1 -- 172.21.15.44:0/3236844688 wait complete. 2023-09-10T00:51:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.922+0000 7fb60ef78b80 1 -- start start 2023-09-10T00:51:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.922+0000 7fb60ef78b80 1 Processor -- start 2023-09-10T00:51:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.922+0000 7fb60ef78b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316300 0x55cb8d322e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.922+0000 7fb60ef78b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316cc0 0x55cb8d3200f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.922+0000 7fb60ef78b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 0x55cb8d320770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.922+0000 7fb60ef78b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8d206b10 con 0x55cb8d316cc0 2023-09-10T00:51:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.922+0000 7fb60ef78b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8d206810 con 0x55cb8d31c1a0 2023-09-10T00:51:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.922+0000 7fb60ef78b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8d0ea8c0 con 0x55cb8d316300 2023-09-10T00:51:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.922+0000 7fb60ef78b80 10 client.0 _ll_register_callbacks cb 0x55cb8d150a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:51:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.922+0000 7fb60134f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 0x55cb8d320770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.922+0000 7fb600b4e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316300 0x55cb8d322e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.922+0000 7fb5fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316cc0 0x55cb8d3200f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.922+0000 7fb60134f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 0x55cb8d320770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54294/0 (socket says 172.21.15.44:54294) 2023-09-10T00:51:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.922+0000 7fb60134f700 1 -- 172.21.15.44:0/3216018257 learned_addr learned my addr 172.21.15.44:0/3216018257 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.923+0000 7fb60134f700 1 -- 172.21.15.44:0/3216018257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316300 msgr2=0x55cb8d322e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.923+0000 7fb60134f700 1 --2- 172.21.15.44:0/3216018257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316300 0x55cb8d322e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.923+0000 7fb60134f700 1 -- 172.21.15.44:0/3216018257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316cc0 msgr2=0x55cb8d3200f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.923+0000 7fb60134f700 1 --2- 172.21.15.44:0/3216018257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316cc0 0x55cb8d3200f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.923+0000 7fb60134f700 1 -- 172.21.15.44:0/3216018257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e8017070 con 0x55cb8d31c1a0 2023-09-10T00:51:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.923+0000 7fb5fbfff700 1 --2- 172.21.15.44:0/3216018257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316cc0 0x55cb8d3200f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.923+0000 7fb60134f700 1 --2- 172.21.15.44:0/3216018257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 0x55cb8d320770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4015000 tx=0x7fb5f4007650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df8a90a48618c85 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.923+0000 7fb5dbfff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:51:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.923+0000 7fb5dbfff700 1 -- 172.21.15.44:0/3216018257 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400e040 con 0x55cb8d31c1a0 2023-09-10T00:51:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.923+0000 7fb5dbfff700 1 -- 172.21.15.44:0/3216018257 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb5f400bbf0 con 0x55cb8d31c1a0 2023-09-10T00:51:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.923+0000 7fb60ef78b80 1 -- 172.21.15.44:0/3216018257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb8d1c0490 con 0x55cb8d31c1a0 2023-09-10T00:51:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.923+0000 7fb5dbfff700 1 -- 172.21.15.44:0/3216018257 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5f4007870 con 0x55cb8d31c1a0 2023-09-10T00:51:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.923+0000 7fb60ef78b80 10 client.27056 fetch_fsmap learned FSMap version 61 2023-09-10T00:51:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.924+0000 7fb60ef78b80 1 -- 172.21.15.44:0/3216018257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7fb5fc1a5be0 con 0x55cb8d31c1a0 2023-09-10T00:51:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.924+0000 7fb5dbfff700 1 -- 172.21.15.44:0/3216018257 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cb8d1c0490 con 0x55cb8d31c1a0 2023-09-10T00:51:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.924+0000 7fb5dbfff700 1 -- 172.21.15.44:0/3216018257 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fb5f4023030 con 0x55cb8d31c1a0 2023-09-10T00:51:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.924+0000 7fb60ef78b80 10 client.27056 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:51:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.924+0000 7fb60ef78b80 10 client.27056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:51:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.924+0000 7fb60ef78b80 20 client.27056 populate_metadata read hostname 'smithi044' 2023-09-10T00:51:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.924+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/3216018257 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb8d330580 0x55cb8d3329a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.924+0000 7fb60ef78b80 4 client.27056 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:51:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.924+0000 7fb60ef78b80 1 -- 172.21.15.44:0/3216018257 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb8d27d840 con 0x55cb8d330580 2023-09-10T00:51:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.924+0000 7fb600b4e700 1 --2- 172.21.15.44:0/3216018257 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb8d330580 0x55cb8d3329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.925+0000 7fb600b4e700 1 --2- 172.21.15.44:0/3216018257 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb8d330580 0x55cb8d3329a0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdca730d756900dd server_cookie=2a407a77b54f3bae in_seq=0 out_seq=0 2023-09-10T00:51:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.925+0000 7fb5dbfff700 10 client.27056 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.925+0000 7fb5dbfff700 1 -- 172.21.15.44:0/3216018257 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cb8d27d840 con 0x55cb8d330580 2023-09-10T00:51:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.925+0000 7fb5dbfff700 10 client.27056 handle_command_reply: tid=0 2023-09-10T00:51:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.925+0000 7fb60ef78b80 1 client.27056 shutdown 2023-09-10T00:51:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.925+0000 7fb60ef78b80 1 -- 172.21.15.44:0/3216018257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 msgr2=0x55cb8d320770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.925+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/3216018257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 0x55cb8d320770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4015000 tx=0x7fb5f4007650 comp rx=0 tx=0).stop 2023-09-10T00:51:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.925+0000 7fb60ef78b80 1 -- 172.21.15.44:0/3216018257 shutdown_connections 2023-09-10T00:51:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.925+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/3216018257 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb8d330580 0x55cb8d3329a0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.925+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/3216018257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 0x55cb8d320770 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.925+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/3216018257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316300 0x55cb8d322e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.925+0000 7fb5dbfff700 0 client.27056 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.925+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/3216018257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316cc0 0x55cb8d3200f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.925+0000 7fb60ef78b80 1 -- 172.21.15.44:0/3216018257 >> 172.21.15.44:0/3216018257 conn(0x55cb8d28b030 msgr2=0x55cb8d3107a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.926+0000 7fb60ef78b80 1 -- 172.21.15.44:0/3216018257 shutdown_connections 2023-09-10T00:51:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.926+0000 7fb60ef78b80 1 -- 172.21.15.44:0/3216018257 wait complete. 2023-09-10T00:51:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.926+0000 7fb60ef78b80 20 client.27056 trim_cache size 0 max 16384 2023-09-10T00:51:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.942+0000 7fb60ef78b80 1 Processor -- start 2023-09-10T00:51:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.942+0000 7fb60ef78b80 1 -- start start 2023-09-10T00:51:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.943+0000 7fb60ef78b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316300 0x55cb8d3274a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.943+0000 7fb60ef78b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316cc0 0x55cb8d311090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.943+0000 7fb60ef78b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 0x55cb8d3115d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.943+0000 7fb60ef78b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8d31be00 con 0x55cb8d316cc0 2023-09-10T00:51:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.943+0000 7fb60ef78b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8d319db0 con 0x55cb8d31c1a0 2023-09-10T00:51:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.943+0000 7fb60ef78b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8d3279e0 con 0x55cb8d316300 2023-09-10T00:51:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.943+0000 7fb5fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316cc0 0x55cb8d311090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.943+0000 7fb5fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316cc0 0x55cb8d311090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52560/0 (socket says 172.21.15.44:52560) 2023-09-10T00:51:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.943+0000 7fb5fbfff700 1 -- 172.21.15.44:0/425911101 learned_addr learned my addr 172.21.15.44:0/425911101 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.943+0000 7fb60134f700 1 --2- 172.21.15.44:0/425911101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 0x55cb8d3115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.943+0000 7fb600b4e700 1 --2- 172.21.15.44:0/425911101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316300 0x55cb8d3274a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.943+0000 7fb5fbfff700 1 -- 172.21.15.44:0/425911101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316300 msgr2=0x55cb8d3274a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.943+0000 7fb5fbfff700 1 --2- 172.21.15.44:0/425911101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316300 0x55cb8d3274a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.943+0000 7fb5fbfff700 1 -- 172.21.15.44:0/425911101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 msgr2=0x55cb8d3115d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.943+0000 7fb5fbfff700 1 --2- 172.21.15.44:0/425911101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 0x55cb8d3115d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.943+0000 7fb5fbfff700 1 -- 172.21.15.44:0/425911101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb8d07dc00 con 0x55cb8d316cc0 2023-09-10T00:51:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.944+0000 7fb5fbfff700 1 --2- 172.21.15.44:0/425911101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316cc0 0x55cb8d311090 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80039d0 tx=0x7fb5e8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17bb5e37bd9347 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.944+0000 7fb5e77fe700 1 -- 172.21.15.44:0/425911101 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8024070 con 0x55cb8d316cc0 2023-09-10T00:51:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.944+0000 7fb5e77fe700 1 -- 172.21.15.44:0/425911101 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5e801e940 con 0x55cb8d316cc0 2023-09-10T00:51:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.944+0000 7fb5e77fe700 1 -- 172.21.15.44:0/425911101 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8028a20 con 0x55cb8d316cc0 2023-09-10T00:51:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.945+0000 7fb60ef78b80 1 -- 172.21.15.44:0/425911101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316cc0 msgr2=0x55cb8d311090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.945+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/425911101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316cc0 0x55cb8d311090 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80039d0 tx=0x7fb5e8014000 comp rx=0 tx=0).stop 2023-09-10T00:51:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.945+0000 7fb60ef78b80 1 -- 172.21.15.44:0/425911101 shutdown_connections 2023-09-10T00:51:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.945+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/425911101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 0x55cb8d3115d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.945+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/425911101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316300 0x55cb8d3274a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.945+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/425911101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316cc0 0x55cb8d311090 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.945+0000 7fb60ef78b80 1 -- 172.21.15.44:0/425911101 >> 172.21.15.44:0/425911101 conn(0x55cb8d28b030 msgr2=0x55cb8d31b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.945+0000 7fb60ef78b80 1 -- 172.21.15.44:0/425911101 shutdown_connections 2023-09-10T00:51:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.945+0000 7fb60ef78b80 1 -- 172.21.15.44:0/425911101 wait complete. 2023-09-10T00:51:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.945+0000 7fb60ef78b80 1 -- start start 2023-09-10T00:51:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.945+0000 7fb60ef78b80 1 Processor -- start 2023-09-10T00:51:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb60ef78b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316300 0x55cb8d378530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb60ef78b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316cc0 0x55cb8d378a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb60ef78b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 0x55cb8d37d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb60ef78b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8d37d940 con 0x55cb8d316300 2023-09-10T00:51:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb60ef78b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8d37dab0 con 0x55cb8d31c1a0 2023-09-10T00:51:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb60ef78b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8d37dc70 con 0x55cb8d316cc0 2023-09-10T00:51:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb60ef78b80 10 client.0 _ll_register_callbacks cb 0x7fb5f4007e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:51:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb5fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316cc0 0x55cb8d378a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb600b4e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316300 0x55cb8d378530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb5fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316cc0 0x55cb8d378a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59446/0 (socket says 172.21.15.44:59446) 2023-09-10T00:51:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb600b4e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316300 0x55cb8d378530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52564/0 (socket says 172.21.15.44:52564) 2023-09-10T00:51:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb5fbfff700 1 -- 172.21.15.44:0/303854167 learned_addr learned my addr 172.21.15.44:0/303854167 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb60134f700 1 --2- 172.21.15.44:0/303854167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 0x55cb8d37d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb5fbfff700 1 -- 172.21.15.44:0/303854167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 msgr2=0x55cb8d37d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb5fbfff700 1 --2- 172.21.15.44:0/303854167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 0x55cb8d37d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb5fbfff700 1 -- 172.21.15.44:0/303854167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316300 msgr2=0x55cb8d378530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb5fbfff700 1 --2- 172.21.15.44:0/303854167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316300 0x55cb8d378530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb5fbfff700 1 -- 172.21.15.44:0/303854167 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e8003680 con 0x55cb8d316cc0 2023-09-10T00:51:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb5fbfff700 1 --2- 172.21.15.44:0/303854167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316cc0 0x55cb8d378a70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8029000 tx=0x7fb5e8028560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b565ccfe87564453 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.946+0000 7fb5dbfff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:51:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.947+0000 7fb5dbfff700 1 -- 172.21.15.44:0/303854167 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8024040 con 0x55cb8d316cc0 2023-09-10T00:51:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.947+0000 7fb60ef78b80 1 -- 172.21.15.44:0/303854167 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5f009d450 con 0x55cb8d316cc0 2023-09-10T00:51:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.947+0000 7fb5dbfff700 1 -- 172.21.15.44:0/303854167 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb5e801e6b0 con 0x55cb8d316cc0 2023-09-10T00:51:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.947+0000 7fb5dbfff700 1 -- 172.21.15.44:0/303854167 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5e8005050 con 0x55cb8d316cc0 2023-09-10T00:51:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.947+0000 7fb60ef78b80 10 client.26998 fetch_fsmap learned FSMap version 61 2023-09-10T00:51:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.947+0000 7fb60ef78b80 1 -- 172.21.15.44:0/303854167 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55cb8d0bb1f0 con 0x55cb8d316cc0 2023-09-10T00:51:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.948+0000 7fb5dbfff700 1 -- 172.21.15.44:0/303854167 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5e803a020 con 0x55cb8d316cc0 2023-09-10T00:51:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.948+0000 7fb5dbfff700 1 -- 172.21.15.44:0/303854167 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fb5e8004c10 con 0x55cb8d316cc0 2023-09-10T00:51:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.948+0000 7fb60ef78b80 10 client.26998 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:51:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.948+0000 7fb60ef78b80 10 client.26998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:51:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.948+0000 7fb60ef78b80 20 client.26998 populate_metadata read hostname 'smithi044' 2023-09-10T00:51:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.948+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/303854167 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb8d3855f0 0x55cb8d387a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.948+0000 7fb60ef78b80 4 client.26998 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:51:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.948+0000 7fb60ef78b80 1 -- 172.21.15.44:0/303854167 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cb8d2618c0 con 0x55cb8d3855f0 2023-09-10T00:51:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.948+0000 7fb600b4e700 1 --2- 172.21.15.44:0/303854167 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb8d3855f0 0x55cb8d387a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.950+0000 7fb5dbfff700 10 client.26998 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.950+0000 7fb600b4e700 1 --2- 172.21.15.44:0/303854167 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb8d3855f0 0x55cb8d387a10 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e08bcc4ff1f975f1 server_cookie=7f014338dbc563ff in_seq=0 out_seq=0 2023-09-10T00:51:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.964+0000 7fb5dbfff700 1 -- 172.21.15.44:0/303854167 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cb8d2618c0 con 0x55cb8d3855f0 2023-09-10T00:51:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.964+0000 7fb5dbfff700 10 client.26998 handle_command_reply: tid=0 2023-09-10T00:51:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.964+0000 7fb60ef78b80 1 client.26998 shutdown 2023-09-10T00:51:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.964+0000 7fb60ef78b80 1 -- 172.21.15.44:0/303854167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316cc0 msgr2=0x55cb8d378a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.964+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/303854167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316cc0 0x55cb8d378a70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8029000 tx=0x7fb5e8028560 comp rx=0 tx=0).stop 2023-09-10T00:51:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.964+0000 7fb60ef78b80 1 -- 172.21.15.44:0/303854167 shutdown_connections 2023-09-10T00:51:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.964+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/303854167 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb8d3855f0 0x55cb8d387a10 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.964+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/303854167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb8d31c1a0 0x55cb8d37d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.964+0000 7fb5dbfff700 0 client.26998 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.964+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/303854167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb8d316300 0x55cb8d378530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.964+0000 7fb60ef78b80 1 --2- 172.21.15.44:0/303854167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb8d316cc0 0x55cb8d378a70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.964+0000 7fb60ef78b80 1 -- 172.21.15.44:0/303854167 >> 172.21.15.44:0/303854167 conn(0x55cb8d28b030 msgr2=0x55cb8d31b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.965+0000 7fb60ef78b80 1 -- 172.21.15.44:0/303854167 shutdown_connections 2023-09-10T00:51:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.965+0000 7fb60ef78b80 1 -- 172.21.15.44:0/303854167 wait complete. 2023-09-10T00:51:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.965+0000 7fb60ef78b80 20 client.26998 trim_cache size 0 max 16384 2023-09-10T00:51:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:51:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:51:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:51:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:51:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.965+0000 7fb602db2700 1 -- 172.21.15.44:0/4244515959 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5ec0a07b0 msgr2=0x7fb5ec0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.965+0000 7fb602db2700 1 --2- 172.21.15.44:0/4244515959 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5ec0a07b0 0x7fb5ec0a2c70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc10cef0 tx=0x7fb5e8009310 comp rx=0 tx=0).stop 2023-09-10T00:51:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.965+0000 7fb602db2700 1 -- 172.21.15.44:0/4244515959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5fc0ff080 msgr2=0x7fb5fc1954a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.965+0000 7fb602db2700 1 --2- 172.21.15.44:0/4244515959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5fc0ff080 0x7fb5fc1954a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000e9a0 tx=0x7fb5f000ecb0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.965+0000 7fb602db2700 1 -- 172.21.15.44:0/4244515959 shutdown_connections 2023-09-10T00:51:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.965+0000 7fb602db2700 1 --2- 172.21.15.44:0/4244515959 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5ec0a07b0 0x7fb5ec0a2c70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.965+0000 7fb602db2700 1 --2- 172.21.15.44:0/4244515959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5fc1a2930 0x7fb5fc1a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.965+0000 7fb602db2700 1 --2- 172.21.15.44:0/4244515959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5fc0ff080 0x7fb5fc1954a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.965+0000 7fb602db2700 1 --2- 172.21.15.44:0/4244515959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5fc103f90 0x7fb5fc1959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.965+0000 7fb602db2700 1 -- 172.21.15.44:0/4244515959 >> 172.21.15.44:0/4244515959 conn(0x7fb5fc075710 msgr2=0x7fb5fc0fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.965+0000 7fb602db2700 1 -- 172.21.15.44:0/4244515959 shutdown_connections 2023-09-10T00:51:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:27.965+0000 7fb602db2700 1 -- 172.21.15.44:0/4244515959 wait complete. 2023-09-10T00:51:28.671 INFO:teuthology.orchestra.run.smithi044.stderr:41100000 of 50000000 tuples (82%) done (elapsed 628.31 s, remaining 136.06 s) 2023-09-10T00:51:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:28 smithi131 ceph-mon[103713]: pgmap v337: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 MiB/s wr, 60 op/s 2023-09-10T00:51:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:28 smithi044 ceph-mon[98676]: pgmap v337: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 MiB/s wr, 60 op/s 2023-09-10T00:51:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:28 smithi099 ceph-mon[102692]: pgmap v337: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 MiB/s wr, 60 op/s 2023-09-10T00:51:29.629 INFO:teuthology.orchestra.run.smithi044.stderr:41200000 of 50000000 tuples (82%) done (elapsed 629.26 s, remaining 134.41 s) 2023-09-10T00:51:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:41300000 of 50000000 tuples (82%) done (elapsed 630.12 s, remaining 132.74 s) 2023-09-10T00:51:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:30 smithi131 ceph-mon[103713]: pgmap v338: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 MiB/s wr, 60 op/s 2023-09-10T00:51:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:30 smithi044 ceph-mon[98676]: pgmap v338: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 MiB/s wr, 60 op/s 2023-09-10T00:51:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:30 smithi099 ceph-mon[102692]: pgmap v338: 129 pgs: 129 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 MiB/s wr, 60 op/s 2023-09-10T00:51:31.421 INFO:teuthology.orchestra.run.smithi044.stderr:41400000 of 50000000 tuples (82%) done (elapsed 631.06 s, remaining 131.09 s) 2023-09-10T00:51:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:32 smithi044 ceph-mon[98676]: pgmap v339: 129 pgs: 129 active+clean; 7.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 46 MiB/s wr, 121 op/s 2023-09-10T00:51:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:32 smithi099 ceph-mon[102692]: pgmap v339: 129 pgs: 129 active+clean; 7.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 46 MiB/s wr, 121 op/s 2023-09-10T00:51:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:32 smithi131 ceph-mon[103713]: pgmap v339: 129 pgs: 129 active+clean; 7.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 46 MiB/s wr, 121 op/s 2023-09-10T00:51:34.760 INFO:teuthology.orchestra.run.smithi044.stderr:41500000 of 50000000 tuples (83%) done (elapsed 634.39 s, remaining 129.94 s) 2023-09-10T00:51:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:34 smithi044 ceph-mon[98676]: pgmap v340: 129 pgs: 129 active+clean; 7.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 30 MiB/s wr, 89 op/s 2023-09-10T00:51:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:34 smithi099 ceph-mon[102692]: pgmap v340: 129 pgs: 129 active+clean; 7.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 30 MiB/s wr, 89 op/s 2023-09-10T00:51:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:34 smithi131 ceph-mon[103713]: pgmap v340: 129 pgs: 129 active+clean; 7.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 30 MiB/s wr, 89 op/s 2023-09-10T00:51:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:36 smithi044 ceph-mon[98676]: pgmap v341: 129 pgs: 129 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 39 MiB/s wr, 108 op/s 2023-09-10T00:51:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:36 smithi099 ceph-mon[102692]: pgmap v341: 129 pgs: 129 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 39 MiB/s wr, 108 op/s 2023-09-10T00:51:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:36 smithi131 ceph-mon[103713]: pgmap v341: 129 pgs: 129 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 39 MiB/s wr, 108 op/s 2023-09-10T00:51:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:38 smithi044 ceph-mon[98676]: pgmap v342: 129 pgs: 129 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 23 MiB/s wr, 79 op/s 2023-09-10T00:51:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:38 smithi099 ceph-mon[102692]: pgmap v342: 129 pgs: 129 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 23 MiB/s wr, 79 op/s 2023-09-10T00:51:39.282 INFO:teuthology.orchestra.run.smithi044.stderr:41600000 of 50000000 tuples (83%) done (elapsed 638.92 s, remaining 129.01 s) 2023-09-10T00:51:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:38 smithi131 ceph-mon[103713]: pgmap v342: 129 pgs: 129 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 23 MiB/s wr, 79 op/s 2023-09-10T00:51:40.612 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:51:40.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.753+0000 7fe502178700 1 -- 172.21.15.44:0/3264539206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4fc0ffe00 msgr2=0x7fe4fc10c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.753+0000 7fe502178700 1 --2- 172.21.15.44:0/3264539206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4fc0ffe00 0x7fe4fc10c310 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400b0a0 tx=0x7fe4f400b3b0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.753+0000 7fe502178700 1 -- 172.21.15.44:0/3264539206 shutdown_connections 2023-09-10T00:51:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.753+0000 7fe502178700 1 --2- 172.21.15.44:0/3264539206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4fc0ffe00 0x7fe4fc10c310 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.753+0000 7fe502178700 1 --2- 172.21.15.44:0/3264539206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4fc1054e0 0x7fe4fc1058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.753+0000 7fe502178700 1 --2- 172.21.15.44:0/3264539206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4fc0ff440 0x7fe4fc0ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.753+0000 7fe502178700 1 -- 172.21.15.44:0/3264539206 >> 172.21.15.44:0/3264539206 conn(0x7fe4fc0faf50 msgr2=0x7fe4fc0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.753+0000 7fe502178700 1 -- 172.21.15.44:0/3264539206 shutdown_connections 2023-09-10T00:51:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.754+0000 7fe502178700 1 -- 172.21.15.44:0/3264539206 wait complete. 2023-09-10T00:51:40.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.754+0000 7fe502178700 1 Processor -- start 2023-09-10T00:51:40.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.754+0000 7fe502178700 1 -- start start 2023-09-10T00:51:40.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe502178700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4fc0ff440 0x7fe4fc1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe502178700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4fc0ffe00 0x7fe4fc1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe502178700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4fc1054e0 0x7fe4fc19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe502178700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc19b490 con 0x7fe4fc1054e0 2023-09-10T00:51:40.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe502178700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc19b600 con 0x7fe4fc0ff440 2023-09-10T00:51:40.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe502178700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc19b7c0 con 0x7fe4fc0ffe00 2023-09-10T00:51:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe4fb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4fc0ff440 0x7fe4fc1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe4fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4fc1054e0 0x7fe4fc19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe4f2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4fc0ffe00 0x7fe4fc1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe4fb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4fc0ff440 0x7fe4fc1964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54318/0 (socket says 172.21.15.44:54318) 2023-09-10T00:51:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe4fb7fe700 1 -- 172.21.15.44:0/4128063624 learned_addr learned my addr 172.21.15.44:0/4128063624 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe4fb7fe700 1 -- 172.21.15.44:0/4128063624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4fc0ffe00 msgr2=0x7fe4fc1969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe4fb7fe700 1 --2- 172.21.15.44:0/4128063624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4fc0ffe00 0x7fe4fc1969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe4fb7fe700 1 -- 172.21.15.44:0/4128063624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4fc1054e0 msgr2=0x7fe4fc19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe4fb7fe700 1 --2- 172.21.15.44:0/4128063624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4fc1054e0 0x7fe4fc19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe4fb7fe700 1 -- 172.21.15.44:0/4128063624 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f4009d00 con 0x7fe4fc0ff440 2023-09-10T00:51:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe4f2ffd700 1 --2- 172.21.15.44:0/4128063624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4fc0ffe00 0x7fe4fc1969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.755+0000 7fe4fbfff700 1 --2- 172.21.15.44:0/4128063624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4fc1054e0 0x7fe4fc19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.756+0000 7fe4fb7fe700 1 --2- 172.21.15.44:0/4128063624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4fc0ff440 0x7fe4fc1964b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00e9a0 tx=0x7fe4ec00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.756+0000 7fe4f97fa700 1 -- 172.21.15.44:0/4128063624 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec009790 con 0x7fe4fc0ff440 2023-09-10T00:51:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.756+0000 7fe502178700 1 -- 172.21.15.44:0/4128063624 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4fc19baa0 con 0x7fe4fc0ff440 2023-09-10T00:51:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.756+0000 7fe4f97fa700 1 -- 172.21.15.44:0/4128063624 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4ec010430 con 0x7fe4fc0ff440 2023-09-10T00:51:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.756+0000 7fe4f97fa700 1 -- 172.21.15.44:0/4128063624 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec00cea0 con 0x7fe4fc0ff440 2023-09-10T00:51:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.756+0000 7fe502178700 1 -- 172.21.15.44:0/4128063624 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc1a32e0 con 0x7fe4fc0ff440 2023-09-10T00:51:40.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.758+0000 7fe4f97fa700 1 -- 172.21.15.44:0/4128063624 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fe4ec0105a0 con 0x7fe4fc0ff440 2023-09-10T00:51:40.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.759+0000 7fe4f97fa700 1 --2- 172.21.15.44:0/4128063624 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4e80a07b0 0x7fe4e80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.770 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.759+0000 7fe4f97fa700 1 -- 172.21.15.44:0/4128063624 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fe4ec014070 con 0x7fe4fc0ff440 2023-09-10T00:51:40.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.759+0000 7fe4f2ffd700 1 --2- 172.21.15.44:0/4128063624 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4e80a07b0 0x7fe4e80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.759+0000 7fe4f2ffd700 1 --2- 172.21.15.44:0/4128063624 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4e80a07b0 0x7fe4e80a2c70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc197ac0 tx=0x7fe4e4009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:40.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.759+0000 7fe50e33eb80 1 Processor -- start 2023-09-10T00:51:40.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe50e33eb80 1 -- start start 2023-09-10T00:51:40.771 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe50e33eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65bc710 0x5579e65bcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe50e33eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 0x5579e65c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe50e33eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65c6e40 0x5579e65c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe50e33eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e6515730 con 0x5579e65bd030 2023-09-10T00:51:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe50e33eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e6513050 con 0x5579e65c6e40 2023-09-10T00:51:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe50e33eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e650e940 con 0x5579e65bc710 2023-09-10T00:51:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe4fb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65bc710 0x5579e65bcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe4f2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 0x5579e65c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe4fb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65bc710 0x5579e65bcaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59468/0 (socket says 172.21.15.44:59468) 2023-09-10T00:51:40.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe4fb7fe700 1 -- 172.21.15.44:0/1182010830 learned_addr learned my addr 172.21.15.44:0/1182010830 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:40.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe4fbfff700 1 --2- 172.21.15.44:0/1182010830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65c6e40 0x5579e65c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe4fb7fe700 1 -- 172.21.15.44:0/1182010830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65c6e40 msgr2=0x5579e65c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:40.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe4fb7fe700 1 --2- 172.21.15.44:0/1182010830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65c6e40 0x5579e65c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe4fb7fe700 1 -- 172.21.15.44:0/1182010830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 msgr2=0x5579e65c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:40.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe4fb7fe700 1 --2- 172.21.15.44:0/1182010830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 0x5579e65c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe4fb7fe700 1 -- 172.21.15.44:0/1182010830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579e633ade0 con 0x5579e65bc710 2023-09-10T00:51:40.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.760+0000 7fe4fb7fe700 1 --2- 172.21.15.44:0/1182010830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65bc710 0x5579e65bcaf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec095fc0 tx=0x7fe4ec098f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d328050e842b4ac6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.761+0000 7fe4f1ffb700 1 -- 172.21.15.44:0/1182010830 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec09d170 con 0x5579e65bc710 2023-09-10T00:51:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.761+0000 7fe4f1ffb700 1 -- 172.21.15.44:0/1182010830 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4ec099550 con 0x5579e65bc710 2023-09-10T00:51:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.761+0000 7fe4f1ffb700 1 -- 172.21.15.44:0/1182010830 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0997b0 con 0x5579e65bc710 2023-09-10T00:51:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.761+0000 7fe50e33eb80 1 -- 172.21.15.44:0/1182010830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65bc710 msgr2=0x5579e65bcaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.761+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/1182010830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65bc710 0x5579e65bcaf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec095fc0 tx=0x7fe4ec098f30 comp rx=0 tx=0).stop 2023-09-10T00:51:40.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.761+0000 7fe50e33eb80 1 -- 172.21.15.44:0/1182010830 shutdown_connections 2023-09-10T00:51:40.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.761+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/1182010830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65c6e40 0x5579e65c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.761+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/1182010830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65bc710 0x5579e65bcaf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.761+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/1182010830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 0x5579e65c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.761+0000 7fe50e33eb80 1 -- 172.21.15.44:0/1182010830 >> 172.21.15.44:0/1182010830 conn(0x5579e6531090 msgr2=0x5579e65b7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:40.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.761+0000 7fe50e33eb80 1 -- 172.21.15.44:0/1182010830 shutdown_connections 2023-09-10T00:51:40.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.761+0000 7fe50e33eb80 1 -- 172.21.15.44:0/1182010830 wait complete. 2023-09-10T00:51:40.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.762+0000 7fe50e33eb80 1 -- start start 2023-09-10T00:51:40.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.762+0000 7fe50e33eb80 1 Processor -- start 2023-09-10T00:51:40.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.762+0000 7fe50e33eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65bc710 0x5579e65c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.762+0000 7fe50e33eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 0x5579e65c40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.762+0000 7fe50e33eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65c6e40 0x5579e65c4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.762+0000 7fe50e33eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e64acb70 con 0x5579e65bd030 2023-09-10T00:51:40.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.762+0000 7fe50e33eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e64ac870 con 0x5579e65bc710 2023-09-10T00:51:40.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.762+0000 7fe50e33eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e6390920 con 0x5579e65c6e40 2023-09-10T00:51:40.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe50e33eb80 10 client.0 _ll_register_callbacks cb 0x5579e63f6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:51:40.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4fb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65bc710 0x5579e65c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4f2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 0x5579e65c40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4fb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65bc710 0x5579e65c3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54332/0 (socket says 172.21.15.44:54332) 2023-09-10T00:51:40.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4fb7fe700 1 -- 172.21.15.44:0/3283412322 learned_addr learned my addr 172.21.15.44:0/3283412322 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:40.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4fbfff700 1 --2- 172.21.15.44:0/3283412322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65c6e40 0x5579e65c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4f2ffd700 1 -- 172.21.15.44:0/3283412322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65c6e40 msgr2=0x5579e65c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:40.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4f2ffd700 1 --2- 172.21.15.44:0/3283412322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65c6e40 0x5579e65c4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4f2ffd700 1 -- 172.21.15.44:0/3283412322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65bc710 msgr2=0x5579e65c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:40.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4f2ffd700 1 --2- 172.21.15.44:0/3283412322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65bc710 0x5579e65c3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4f2ffd700 1 -- 172.21.15.44:0/3283412322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc1a32e0 con 0x5579e65bd030 2023-09-10T00:51:40.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4fb7fe700 1 --2- 172.21.15.44:0/3283412322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65bc710 0x5579e65c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:40.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4fbfff700 1 --2- 172.21.15.44:0/3283412322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65c6e40 0x5579e65c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:40.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4f2ffd700 1 --2- 172.21.15.44:0/3283412322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 0x5579e65c40e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4011bb0 tx=0x7fe4e40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbf9666d75af03d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:40.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4daffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:51:40.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4daffd700 1 -- 172.21.15.44:0/3283412322 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e401fa90 con 0x5579e65bd030 2023-09-10T00:51:40.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe50e33eb80 1 -- 172.21.15.44:0/3283412322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579e64664f0 con 0x5579e65bd030 2023-09-10T00:51:40.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4daffd700 1 -- 172.21.15.44:0/3283412322 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fe4e401fbf0 con 0x5579e65bd030 2023-09-10T00:51:40.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe4daffd700 1 -- 172.21.15.44:0/3283412322 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4e401fdb0 con 0x5579e65bd030 2023-09-10T00:51:40.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe50e33eb80 10 client.17406 fetch_fsmap learned FSMap version 61 2023-09-10T00:51:40.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.763+0000 7fe50e33eb80 1 -- 172.21.15.44:0/3283412322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x5579e6323c60 con 0x5579e65bd030 2023-09-10T00:51:40.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.764+0000 7fe4daffd700 1 -- 172.21.15.44:0/3283412322 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5579e64664f0 con 0x5579e65bd030 2023-09-10T00:51:40.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.764+0000 7fe4daffd700 1 -- 172.21.15.44:0/3283412322 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe4e40273e0 con 0x5579e65bd030 2023-09-10T00:51:40.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.764+0000 7fe50e33eb80 10 client.17406 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:51:40.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.764+0000 7fe50e33eb80 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:51:40.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.764+0000 7fe50e33eb80 20 client.17406 populate_metadata read hostname 'smithi044' 2023-09-10T00:51:40.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.764+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/3283412322 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5579e65d69c0 0x5579e65d8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.764+0000 7fe50e33eb80 4 client.17406 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:51:40.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.764+0000 7fe50e33eb80 1 -- 172.21.15.44:0/3283412322 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579e65238a0 con 0x5579e65d69c0 2023-09-10T00:51:40.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.764+0000 7fe4fb7fe700 1 --2- 172.21.15.44:0/3283412322 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5579e65d69c0 0x5579e65d8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.764+0000 7fe4daffd700 10 client.17406 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:40.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.764+0000 7fe4fb7fe700 1 --2- 172.21.15.44:0/3283412322 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5579e65d69c0 0x5579e65d8de0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa47a21b80acc14f server_cookie=40bda929c4646b80 in_seq=0 out_seq=0 2023-09-10T00:51:40.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.765+0000 7fe4daffd700 1 -- 172.21.15.44:0/3283412322 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5579e65238a0 con 0x5579e65d69c0 2023-09-10T00:51:40.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.765+0000 7fe4daffd700 10 client.17406 handle_command_reply: tid=0 2023-09-10T00:51:40.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.765+0000 7fe50e33eb80 1 client.17406 shutdown 2023-09-10T00:51:40.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.765+0000 7fe50e33eb80 1 -- 172.21.15.44:0/3283412322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 msgr2=0x5579e65c40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:40.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.765+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/3283412322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 0x5579e65c40e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4011bb0 tx=0x7fe4e40047c0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.765+0000 7fe50e33eb80 1 -- 172.21.15.44:0/3283412322 shutdown_connections 2023-09-10T00:51:40.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.765+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/3283412322 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5579e65d69c0 0x5579e65d8de0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.765+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/3283412322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65c6e40 0x5579e65c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.765+0000 7fe4daffd700 0 client.17406 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:40.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.765+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/3283412322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65bc710 0x5579e65c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.765+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/3283412322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 0x5579e65c40e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.765+0000 7fe50e33eb80 1 -- 172.21.15.44:0/3283412322 >> 172.21.15.44:0/3283412322 conn(0x5579e6531090 msgr2=0x5579e65b77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:40.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.765+0000 7fe50e33eb80 1 -- 172.21.15.44:0/3283412322 shutdown_connections 2023-09-10T00:51:40.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.765+0000 7fe50e33eb80 1 -- 172.21.15.44:0/3283412322 wait complete. 2023-09-10T00:51:40.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.765+0000 7fe50e33eb80 20 client.17406 trim_cache size 0 max 16384 2023-09-10T00:51:40.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.782+0000 7fe50e33eb80 1 Processor -- start 2023-09-10T00:51:40.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.782+0000 7fe50e33eb80 1 -- start start 2023-09-10T00:51:40.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe50e33eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65bc710 0x5579e65c4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe50e33eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 0x5579e65beea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe50e33eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65c6e40 0x5579e65bf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.794 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe50e33eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e65d4020 con 0x5579e65bd030 2023-09-10T00:51:40.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe50e33eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e65bd560 con 0x5579e65c6e40 2023-09-10T00:51:40.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe50e33eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e65d3320 con 0x5579e65bc710 2023-09-10T00:51:40.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe4f2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 0x5579e65beea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe4fb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65bc710 0x5579e65c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe4f2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 0x5579e65beea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52602/0 (socket says 172.21.15.44:52602) 2023-09-10T00:51:40.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe4f2ffd700 1 -- 172.21.15.44:0/2426382403 learned_addr learned my addr 172.21.15.44:0/2426382403 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:40.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe4fbfff700 1 --2- 172.21.15.44:0/2426382403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65c6e40 0x5579e65bf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe4f2ffd700 1 -- 172.21.15.44:0/2426382403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65bc710 msgr2=0x5579e65c4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:40.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe4f2ffd700 1 --2- 172.21.15.44:0/2426382403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65bc710 0x5579e65c4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe4f2ffd700 1 -- 172.21.15.44:0/2426382403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65c6e40 msgr2=0x5579e65bf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:40.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe4f2ffd700 1 --2- 172.21.15.44:0/2426382403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65c6e40 0x5579e65bf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe4f2ffd700 1 -- 172.21.15.44:0/2426382403 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579e6361250 con 0x5579e65bd030 2023-09-10T00:51:40.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe4fbfff700 1 --2- 172.21.15.44:0/2426382403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65c6e40 0x5579e65bf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:51:40.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe4f2ffd700 1 --2- 172.21.15.44:0/2426382403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 0x5579e65beea0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40097c0 tx=0x7fe4e4009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c1f3debfdbf268 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:40.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe4f27fc700 1 -- 172.21.15.44:0/2426382403 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4027a00 con 0x5579e65bd030 2023-09-10T00:51:40.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe4f27fc700 1 -- 172.21.15.44:0/2426382403 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4e401f6e0 con 0x5579e65bd030 2023-09-10T00:51:40.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.783+0000 7fe4f27fc700 1 -- 172.21.15.44:0/2426382403 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e40286a0 con 0x5579e65bd030 2023-09-10T00:51:40.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.784+0000 7fe50e33eb80 1 -- 172.21.15.44:0/2426382403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 msgr2=0x5579e65beea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:40.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.784+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/2426382403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 0x5579e65beea0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40097c0 tx=0x7fe4e4009390 comp rx=0 tx=0).stop 2023-09-10T00:51:40.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.784+0000 7fe50e33eb80 1 -- 172.21.15.44:0/2426382403 shutdown_connections 2023-09-10T00:51:40.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.784+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/2426382403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65c6e40 0x5579e65bf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.784+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/2426382403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65bc710 0x5579e65c4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.784+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/2426382403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bd030 0x5579e65beea0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.784+0000 7fe50e33eb80 1 -- 172.21.15.44:0/2426382403 >> 172.21.15.44:0/2426382403 conn(0x5579e6531090 msgr2=0x5579e65b76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:40.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.784+0000 7fe50e33eb80 1 -- 172.21.15.44:0/2426382403 shutdown_connections 2023-09-10T00:51:40.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.784+0000 7fe50e33eb80 1 -- 172.21.15.44:0/2426382403 wait complete. 2023-09-10T00:51:40.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.784+0000 7fe50e33eb80 1 -- start start 2023-09-10T00:51:40.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.784+0000 7fe50e33eb80 1 Processor -- start 2023-09-10T00:51:40.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe50e33eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bc710 0x5579e65cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe50e33eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65bd030 0x5579e65cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe50e33eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65c6e40 0x5579e65cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe50e33eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e65cdf10 con 0x5579e65bc710 2023-09-10T00:51:40.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe50e33eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e65cc340 con 0x5579e65bd030 2023-09-10T00:51:40.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe50e33eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e65cc4b0 con 0x5579e65c6e40 2023-09-10T00:51:40.802 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe50e33eb80 10 client.0 _ll_register_callbacks cb 0x7fe4e4003680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:51:40.802 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe4fb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bc710 0x5579e65cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.802 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe4fb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bc710 0x5579e65cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52606/0 (socket says 172.21.15.44:52606) 2023-09-10T00:51:40.802 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe4f2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65bd030 0x5579e65cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.802 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe4fb7fe700 1 -- 172.21.15.44:0/2405950464 learned_addr learned my addr 172.21.15.44:0/2405950464 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:40.803 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe4fbfff700 1 --2- 172.21.15.44:0/2405950464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65c6e40 0x5579e65cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.803 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe4f2ffd700 1 -- 172.21.15.44:0/2405950464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65c6e40 msgr2=0x5579e65cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:40.803 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe4f2ffd700 1 --2- 172.21.15.44:0/2405950464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65c6e40 0x5579e65cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.803 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe4f2ffd700 1 -- 172.21.15.44:0/2405950464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bc710 msgr2=0x5579e65cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:40.803 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe4f2ffd700 1 --2- 172.21.15.44:0/2405950464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bc710 0x5579e65cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe4f2ffd700 1 -- 172.21.15.44:0/2405950464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579e6323c60 con 0x5579e65bd030 2023-09-10T00:51:40.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe4fb7fe700 1 --2- 172.21.15.44:0/2405950464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bc710 0x5579e65cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:40.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe4f2ffd700 1 --2- 172.21.15.44:0/2405950464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65bd030 0x5579e65cb8c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009f10 tx=0x7fe4e40276d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efb63867898e966 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:40.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe4daffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:51:40.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe4daffd700 1 -- 172.21.15.44:0/2405950464 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4047070 con 0x5579e65bd030 2023-09-10T00:51:40.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.785+0000 7fe50e33eb80 1 -- 172.21.15.44:0/2405950464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4ec099550 con 0x5579e65bd030 2023-09-10T00:51:40.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.786+0000 7fe4daffd700 1 -- 172.21.15.44:0/2405950464 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fe4e401fa60 con 0x5579e65bd030 2023-09-10T00:51:40.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.786+0000 7fe4daffd700 1 -- 172.21.15.44:0/2405950464 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4e401fc20 con 0x5579e65bd030 2023-09-10T00:51:40.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.786+0000 7fe50e33eb80 10 client.27080 fetch_fsmap learned FSMap version 61 2023-09-10T00:51:40.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.786+0000 7fe50e33eb80 1 -- 172.21.15.44:0/2405950464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x5579e63392b0 con 0x5579e65bd030 2023-09-10T00:51:40.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.786+0000 7fe4daffd700 1 -- 172.21.15.44:0/2405950464 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4ec099550 con 0x5579e65bd030 2023-09-10T00:51:40.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.786+0000 7fe4daffd700 1 -- 172.21.15.44:0/2405950464 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe4e404a030 con 0x5579e65bd030 2023-09-10T00:51:40.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.786+0000 7fe50e33eb80 10 client.27080 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:51:40.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.786+0000 7fe50e33eb80 10 client.27080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:51:40.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.786+0000 7fe50e33eb80 20 client.27080 populate_metadata read hostname 'smithi044' 2023-09-10T00:51:40.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.786+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/2405950464 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5579e662fdd0 0x5579e66321f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:40.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.786+0000 7fe50e33eb80 4 client.27080 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:51:40.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.786+0000 7fe50e33eb80 1 -- 172.21.15.44:0/2405950464 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5579e6507920 con 0x5579e662fdd0 2023-09-10T00:51:40.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.786+0000 7fe4fb7fe700 1 --2- 172.21.15.44:0/2405950464 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5579e662fdd0 0x5579e66321f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:40.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe4daffd700 10 client.27080 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:40.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe4fb7fe700 1 --2- 172.21.15.44:0/2405950464 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5579e662fdd0 0x5579e66321f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cda1ef4d4e6add8 server_cookie=7d714551393886fa in_seq=0 out_seq=0 2023-09-10T00:51:40.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe4daffd700 1 -- 172.21.15.44:0/2405950464 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5579e6507920 con 0x5579e662fdd0 2023-09-10T00:51:40.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe4daffd700 10 client.27080 handle_command_reply: tid=0 2023-09-10T00:51:40.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe50e33eb80 1 client.27080 shutdown 2023-09-10T00:51:40.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe50e33eb80 1 -- 172.21.15.44:0/2405950464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65bd030 msgr2=0x5579e65cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:40.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/2405950464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65bd030 0x5579e65cb8c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009f10 tx=0x7fe4e40276d0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe50e33eb80 1 -- 172.21.15.44:0/2405950464 shutdown_connections 2023-09-10T00:51:40.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/2405950464 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5579e662fdd0 0x5579e66321f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/2405950464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579e65c6e40 0x5579e65cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe4daffd700 0 client.27080 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:40.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/2405950464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579e65bc710 0x5579e65cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe50e33eb80 1 --2- 172.21.15.44:0/2405950464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e65bd030 0x5579e65cb8c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe50e33eb80 1 -- 172.21.15.44:0/2405950464 >> 172.21.15.44:0/2405950464 conn(0x5579e6531090 msgr2=0x5579e65b7670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:40.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe50e33eb80 1 -- 172.21.15.44:0/2405950464 shutdown_connections 2023-09-10T00:51:40.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe50e33eb80 1 -- 172.21.15.44:0/2405950464 wait complete. 2023-09-10T00:51:40.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.787+0000 7fe50e33eb80 20 client.27080 trim_cache size 0 max 16384 2023-09-10T00:51:40.811 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:51:40.811 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:51:40.811 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:51:40.811 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:51:40.811 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:51:40.812 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:51:40.812 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:51:40.812 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:51:40.812 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:51:40.812 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:51:40.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.788+0000 7fe502178700 1 -- 172.21.15.44:0/4128063624 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4e80a07b0 msgr2=0x7fe4e80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:40.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.788+0000 7fe502178700 1 --2- 172.21.15.44:0/4128063624 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4e80a07b0 0x7fe4e80a2c70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc197ac0 tx=0x7fe4e4009260 comp rx=0 tx=0).stop 2023-09-10T00:51:40.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.788+0000 7fe502178700 1 -- 172.21.15.44:0/4128063624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4fc0ff440 msgr2=0x7fe4fc1964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:40.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.788+0000 7fe502178700 1 --2- 172.21.15.44:0/4128063624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4fc0ff440 0x7fe4fc1964b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00e9a0 tx=0x7fe4ec00ecb0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.788+0000 7fe502178700 1 -- 172.21.15.44:0/4128063624 shutdown_connections 2023-09-10T00:51:40.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.788+0000 7fe502178700 1 --2- 172.21.15.44:0/4128063624 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4e80a07b0 0x7fe4e80a2c70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.788+0000 7fe502178700 1 --2- 172.21.15.44:0/4128063624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4fc1054e0 0x7fe4fc19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.788+0000 7fe502178700 1 --2- 172.21.15.44:0/4128063624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4fc0ff440 0x7fe4fc1964b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.788+0000 7fe502178700 1 --2- 172.21.15.44:0/4128063624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4fc0ffe00 0x7fe4fc1969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:40.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.788+0000 7fe502178700 1 -- 172.21.15.44:0/4128063624 >> 172.21.15.44:0/4128063624 conn(0x7fe4fc0faf50 msgr2=0x7fe4fc067820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:40.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.788+0000 7fe502178700 1 -- 172.21.15.44:0/4128063624 shutdown_connections 2023-09-10T00:51:40.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:51:40.788+0000 7fe502178700 1 -- 172.21.15.44:0/4128063624 wait complete. 2023-09-10T00:51:40.816 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:51:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:40 smithi044 ceph-mon[98676]: pgmap v343: 129 pgs: 129 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 23 MiB/s wr, 79 op/s 2023-09-10T00:51:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.075+0000 7f6521c6c700 1 -- 172.21.15.44:0/63044965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f651c1055b0 msgr2=0x7f651c105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.075+0000 7f6521c6c700 1 --2- 172.21.15.44:0/63044965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f651c1055b0 0x7f651c105990 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f650c0099c0 tx=0x7f650c009cd0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.075+0000 7f6521c6c700 1 -- 172.21.15.44:0/63044965 shutdown_connections 2023-09-10T00:51:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.075+0000 7f6521c6c700 1 --2- 172.21.15.44:0/63044965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f651c100120 0x7f651c10c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.075+0000 7f6521c6c700 1 --2- 172.21.15.44:0/63044965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f651c1055b0 0x7f651c105990 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.075+0000 7f6521c6c700 1 --2- 172.21.15.44:0/63044965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0ff760 0x7f651c0ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.075+0000 7f6521c6c700 1 -- 172.21.15.44:0/63044965 >> 172.21.15.44:0/63044965 conn(0x7f651c074f30 msgr2=0x7f651c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.075+0000 7f6521c6c700 1 -- 172.21.15.44:0/63044965 shutdown_connections 2023-09-10T00:51:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.075+0000 7f6521c6c700 1 -- 172.21.15.44:0/63044965 wait complete. 2023-09-10T00:51:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.076+0000 7f6521c6c700 1 Processor -- start 2023-09-10T00:51:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.076+0000 7f6521c6c700 1 -- start start 2023-09-10T00:51:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.076+0000 7f6521c6c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0ff760 0x7f651c19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.076+0000 7f6521c6c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f651c100120 0x7f651c19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.076+0000 7f6521c6c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f651c1055b0 0x7f651c1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.076+0000 7f6521c6c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c196a20 con 0x7f651c100120 2023-09-10T00:51:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.076+0000 7f6521c6c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c196b90 con 0x7f651c0ff760 2023-09-10T00:51:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.076+0000 7f6521c6c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c196d50 con 0x7f651c1055b0 2023-09-10T00:51:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.076+0000 7f651b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0ff760 0x7f651c19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.076+0000 7f651affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f651c100120 0x7f651c19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.076+0000 7f651b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0ff760 0x7f651c19c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54360/0 (socket says 172.21.15.44:54360) 2023-09-10T00:51:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.076+0000 7f651b7fe700 1 -- 172.21.15.44:0/2798234897 learned_addr learned my addr 172.21.15.44:0/2798234897 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.076+0000 7f651bfff700 1 --2- 172.21.15.44:0/2798234897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f651c1055b0 0x7f651c1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.077+0000 7f651b7fe700 1 -- 172.21.15.44:0/2798234897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f651c1055b0 msgr2=0x7f651c1963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.077+0000 7f651b7fe700 1 --2- 172.21.15.44:0/2798234897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f651c1055b0 0x7f651c1963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.077+0000 7f651b7fe700 1 -- 172.21.15.44:0/2798234897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f651c100120 msgr2=0x7f651c19ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.077+0000 7f651b7fe700 1 --2- 172.21.15.44:0/2798234897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f651c100120 0x7f651c19ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.077+0000 7f651b7fe700 1 -- 172.21.15.44:0/2798234897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650c0095f0 con 0x7f651c0ff760 2023-09-10T00:51:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.077+0000 7f651affd700 1 --2- 172.21.15.44:0/2798234897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f651c100120 0x7f651c19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.077+0000 7f651b7fe700 1 --2- 172.21.15.44:0/2798234897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0ff760 0x7f651c19c550 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f650c005910 tx=0x7f650c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.077+0000 7f6518ff9700 1 -- 172.21.15.44:0/2798234897 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c01d070 con 0x7f651c0ff760 2023-09-10T00:51:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.077+0000 7f6521c6c700 1 -- 172.21.15.44:0/2798234897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c196fd0 con 0x7f651c0ff760 2023-09-10T00:51:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.077+0000 7f6518ff9700 1 -- 172.21.15.44:0/2798234897 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f650c022470 con 0x7f651c0ff760 2023-09-10T00:51:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.077+0000 7f6518ff9700 1 -- 172.21.15.44:0/2798234897 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c017650 con 0x7f651c0ff760 2023-09-10T00:51:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.077+0000 7f6521c6c700 1 -- 172.21.15.44:0/2798234897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c1a3540 con 0x7f651c0ff760 2023-09-10T00:51:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.079+0000 7f6518ff9700 1 -- 172.21.15.44:0/2798234897 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f650c0225e0 con 0x7f651c0ff760 2023-09-10T00:51:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.080+0000 7f6518ff9700 1 --2- 172.21.15.44:0/2798234897 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f65080a07b0 0x7f65080a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.080+0000 7f6518ff9700 1 -- 172.21.15.44:0/2798234897 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f650c0cd3f0 con 0x7f651c0ff760 2023-09-10T00:51:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.080+0000 7f651affd700 1 --2- 172.21.15.44:0/2798234897 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f65080a07b0 0x7f65080a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.080+0000 7f651affd700 1 --2- 172.21.15.44:0/2798234897 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f65080a07b0 0x7f65080a2c70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f651c104a80 tx=0x7f6504009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f652de32b80 1 Processor -- start 2023-09-10T00:51:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f652de32b80 1 -- start start 2023-09-10T00:51:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f652de32b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 0x55a71ff18990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f652de32b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 0x55a71ff12b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f652de32b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff130d0 0x55a71ff1f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f652de32b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71fe676d0 con 0x55a71ff130d0 2023-09-10T00:51:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f652de32b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71fe64ff0 con 0x55a71ff12710 2023-09-10T00:51:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f652de32b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71fe608e0 con 0x55a71ff185b0 2023-09-10T00:51:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f651b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 0x55a71ff18990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f651bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff130d0 0x55a71ff1f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f651affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 0x55a71ff12b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f651b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 0x55a71ff18990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59510/0 (socket says 172.21.15.44:59510) 2023-09-10T00:51:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f651affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 0x55a71ff12b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54370/0 (socket says 172.21.15.44:54370) 2023-09-10T00:51:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f651bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff130d0 0x55a71ff1f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52632/0 (socket says 172.21.15.44:52632) 2023-09-10T00:51:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f651b7fe700 1 -- 172.21.15.44:0/578976549 learned_addr learned my addr 172.21.15.44:0/578976549 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f651bfff700 1 -- 172.21.15.44:0/578976549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 msgr2=0x55a71ff18990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f651bfff700 1 --2- 172.21.15.44:0/578976549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 0x55a71ff18990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f651bfff700 1 -- 172.21.15.44:0/578976549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 msgr2=0x55a71ff12b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f651bfff700 1 --2- 172.21.15.44:0/578976549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 0x55a71ff12b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f651bfff700 1 -- 172.21.15.44:0/578976549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a71fc8cd80 con 0x55a71ff130d0 2023-09-10T00:51:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f651b7fe700 1 --2- 172.21.15.44:0/578976549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 0x55a71ff18990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f651affd700 1 --2- 172.21.15.44:0/578976549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 0x55a71ff12b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.081+0000 7f651bfff700 1 --2- 172.21.15.44:0/578976549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff130d0 0x55a71ff1f340 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6514011550 tx=0x7f6514009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0ecad62e12c3f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.082+0000 7f6511ffb700 1 -- 172.21.15.44:0/578976549 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400e040 con 0x55a71ff130d0 2023-09-10T00:51:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.082+0000 7f6511ffb700 1 -- 172.21.15.44:0/578976549 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f651400f040 con 0x55a71ff130d0 2023-09-10T00:51:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.082+0000 7f6511ffb700 1 -- 172.21.15.44:0/578976549 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514004270 con 0x55a71ff130d0 2023-09-10T00:51:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.082+0000 7f652de32b80 1 -- 172.21.15.44:0/578976549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff130d0 msgr2=0x55a71ff1f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.082+0000 7f652de32b80 1 --2- 172.21.15.44:0/578976549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff130d0 0x55a71ff1f340 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6514011550 tx=0x7f6514009920 comp rx=0 tx=0).stop 2023-09-10T00:51:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.082+0000 7f652de32b80 1 -- 172.21.15.44:0/578976549 shutdown_connections 2023-09-10T00:51:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.082+0000 7f652de32b80 1 --2- 172.21.15.44:0/578976549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff130d0 0x55a71ff1f340 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.082+0000 7f652de32b80 1 --2- 172.21.15.44:0/578976549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 0x55a71ff18990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.082+0000 7f652de32b80 1 --2- 172.21.15.44:0/578976549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 0x55a71ff12b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.082+0000 7f652de32b80 1 -- 172.21.15.44:0/578976549 >> 172.21.15.44:0/578976549 conn(0x55a71fe83030 msgr2=0x55a71ff09560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.082+0000 7f652de32b80 1 -- 172.21.15.44:0/578976549 shutdown_connections 2023-09-10T00:51:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.082+0000 7f652de32b80 1 -- 172.21.15.44:0/578976549 wait complete. 2023-09-10T00:51:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.083+0000 7f652de32b80 1 -- start start 2023-09-10T00:51:41.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:40 smithi099 ceph-mon[102692]: pgmap v343: 129 pgs: 129 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 23 MiB/s wr, 79 op/s 2023-09-10T00:51:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.083+0000 7f652de32b80 1 Processor -- start 2023-09-10T00:51:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.083+0000 7f652de32b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff12710 0x55a71ff1f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.083+0000 7f652de32b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff130d0 0x55a71ff1c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.083+0000 7f652de32b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 0x55a71ff1c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.083+0000 7f652de32b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71fdfeb10 con 0x55a71ff12710 2023-09-10T00:51:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.083+0000 7f652de32b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71fdfe810 con 0x55a71ff130d0 2023-09-10T00:51:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.083+0000 7f652de32b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71fce28c0 con 0x55a71ff185b0 2023-09-10T00:51:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.083+0000 7f652de32b80 10 client.0 _ll_register_callbacks cb 0x55a71fd48a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:51:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.083+0000 7f651b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff12710 0x55a71ff1f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.083+0000 7f651b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff12710 0x55a71ff1f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52634/0 (socket says 172.21.15.44:52634) 2023-09-10T00:51:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.083+0000 7f651b7fe700 1 -- 172.21.15.44:0/2597357975 learned_addr learned my addr 172.21.15.44:0/2597357975 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.083+0000 7f651bfff700 1 --2- 172.21.15.44:0/2597357975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 0x55a71ff1c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.083+0000 7f651affd700 1 --2- 172.21.15.44:0/2597357975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff130d0 0x55a71ff1c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f651b7fe700 1 -- 172.21.15.44:0/2597357975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 msgr2=0x55a71ff1c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f651b7fe700 1 --2- 172.21.15.44:0/2597357975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 0x55a71ff1c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f651b7fe700 1 -- 172.21.15.44:0/2597357975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff130d0 msgr2=0x55a71ff1c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f651b7fe700 1 --2- 172.21.15.44:0/2597357975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff130d0 0x55a71ff1c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f651b7fe700 1 -- 172.21.15.44:0/2597357975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6504017070 con 0x55a71ff12710 2023-09-10T00:51:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f651b7fe700 1 --2- 172.21.15.44:0/2597357975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff12710 0x55a71ff1f180 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f650c005cd0 tx=0x7f650c09a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcc92495801a9c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f64faffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:51:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f64faffd700 1 -- 172.21.15.44:0/2597357975 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c0cd230 con 0x55a71ff12710 2023-09-10T00:51:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f64faffd700 1 -- 172.21.15.44:0/2597357975 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f650c09ac30 con 0x55a71ff12710 2023-09-10T00:51:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f652de32b80 1 -- 172.21.15.44:0/2597357975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a71fdb8490 con 0x55a71ff12710 2023-09-10T00:51:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f64faffd700 1 -- 172.21.15.44:0/2597357975 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f650c022ad0 con 0x55a71ff12710 2023-09-10T00:51:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f652de32b80 10 client.17424 fetch_fsmap learned FSMap version 61 2023-09-10T00:51:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f652de32b80 1 -- 172.21.15.44:0/2597357975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f651c1a3540 con 0x55a71ff12710 2023-09-10T00:51:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f64faffd700 1 -- 172.21.15.44:0/2597357975 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f650c0b6020 con 0x55a71ff12710 2023-09-10T00:51:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f64faffd700 1 -- 172.21.15.44:0/2597357975 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f650c09a790 con 0x55a71ff12710 2023-09-10T00:51:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f652de32b80 10 client.17424 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:51:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f652de32b80 10 client.17424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:51:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.084+0000 7f652de32b80 20 client.17424 populate_metadata read hostname 'smithi044' 2023-09-10T00:51:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.085+0000 7f652de32b80 1 --2- 172.21.15.44:0/2597357975 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a71ff2c500 0x55a71ff2e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.085+0000 7f652de32b80 4 client.17424 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:51:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.085+0000 7f652de32b80 1 -- 172.21.15.44:0/2597357975 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a71fe75840 con 0x55a71ff2c500 2023-09-10T00:51:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.085+0000 7f651affd700 1 --2- 172.21.15.44:0/2597357975 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a71ff2c500 0x55a71ff2e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.085+0000 7f64faffd700 10 client.17424 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.085+0000 7f651affd700 1 --2- 172.21.15.44:0/2597357975 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a71ff2c500 0x55a71ff2e920 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a13d4162742cbee8 server_cookie=1d6d2988ddcf7c70 in_seq=0 out_seq=0 2023-09-10T00:51:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.086+0000 7f64faffd700 1 -- 172.21.15.44:0/2597357975 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a71fe75840 con 0x55a71ff2c500 2023-09-10T00:51:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.086+0000 7f64faffd700 10 client.17424 handle_command_reply: tid=0 2023-09-10T00:51:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.086+0000 7f652de32b80 1 client.17424 shutdown 2023-09-10T00:51:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.086+0000 7f652de32b80 1 -- 172.21.15.44:0/2597357975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff12710 msgr2=0x55a71ff1f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.086+0000 7f652de32b80 1 --2- 172.21.15.44:0/2597357975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff12710 0x55a71ff1f180 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f650c005cd0 tx=0x7f650c09a180 comp rx=0 tx=0).stop 2023-09-10T00:51:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.086+0000 7f652de32b80 1 -- 172.21.15.44:0/2597357975 shutdown_connections 2023-09-10T00:51:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.086+0000 7f652de32b80 1 --2- 172.21.15.44:0/2597357975 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a71ff2c500 0x55a71ff2e920 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.086+0000 7f652de32b80 1 --2- 172.21.15.44:0/2597357975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 0x55a71ff1c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.086+0000 7f64faffd700 0 client.17424 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.086+0000 7f652de32b80 1 --2- 172.21.15.44:0/2597357975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff12710 0x55a71ff1f180 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.086+0000 7f652de32b80 1 --2- 172.21.15.44:0/2597357975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff130d0 0x55a71ff1c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.086+0000 7f652de32b80 1 -- 172.21.15.44:0/2597357975 >> 172.21.15.44:0/2597357975 conn(0x55a71fe83030 msgr2=0x55a71ff087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.086+0000 7f652de32b80 1 -- 172.21.15.44:0/2597357975 shutdown_connections 2023-09-10T00:51:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.086+0000 7f652de32b80 1 -- 172.21.15.44:0/2597357975 wait complete. 2023-09-10T00:51:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.086+0000 7f652de32b80 20 client.17424 trim_cache size 0 max 16384 2023-09-10T00:51:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.101+0000 7f652de32b80 1 Processor -- start 2023-09-10T00:51:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.101+0000 7f652de32b80 1 -- start start 2023-09-10T00:51:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.101+0000 7f652de32b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 0x55a71ff17de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.101+0000 7f652de32b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff130d0 0x55a71ff0da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.101+0000 7f652de32b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 0x55a71ff1ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.101+0000 7f652de32b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71ff09ad0 con 0x55a71ff130d0 2023-09-10T00:51:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.101+0000 7f652de32b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71ff13990 con 0x55a71ff12710 2023-09-10T00:51:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.101+0000 7f652de32b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71ff05fc0 con 0x55a71ff185b0 2023-09-10T00:51:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.101+0000 7f651affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff130d0 0x55a71ff0da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.101+0000 7f651b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 0x55a71ff17de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.101+0000 7f651affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff130d0 0x55a71ff0da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52644/0 (socket says 172.21.15.44:52644) 2023-09-10T00:51:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.101+0000 7f651bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 0x55a71ff1ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.101+0000 7f651affd700 1 -- 172.21.15.44:0/2623439535 learned_addr learned my addr 172.21.15.44:0/2623439535 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.102+0000 7f651bfff700 1 -- 172.21.15.44:0/2623439535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 msgr2=0x55a71ff17de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.102+0000 7f651bfff700 1 --2- 172.21.15.44:0/2623439535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 0x55a71ff17de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.102+0000 7f651bfff700 1 -- 172.21.15.44:0/2623439535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff130d0 msgr2=0x55a71ff0da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.102+0000 7f651bfff700 1 --2- 172.21.15.44:0/2623439535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff130d0 0x55a71ff0da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.102+0000 7f651bfff700 1 -- 172.21.15.44:0/2623439535 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6514011230 con 0x55a71ff185b0 2023-09-10T00:51:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.102+0000 7f651bfff700 1 --2- 172.21.15.44:0/2623439535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 0x55a71ff1ed20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f651400b010 tx=0x7f6514009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a640cdf9b824ee58 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.102+0000 7f65127fc700 1 -- 172.21.15.44:0/2623439535 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400e040 con 0x55a71ff185b0 2023-09-10T00:51:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.102+0000 7f65127fc700 1 -- 172.21.15.44:0/2623439535 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6514003e90 con 0x55a71ff185b0 2023-09-10T00:51:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.102+0000 7f65127fc700 1 -- 172.21.15.44:0/2623439535 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514007d40 con 0x55a71ff185b0 2023-09-10T00:51:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.103+0000 7f652de32b80 1 -- 172.21.15.44:0/2623439535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 msgr2=0x55a71ff1ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.103+0000 7f652de32b80 1 --2- 172.21.15.44:0/2623439535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 0x55a71ff1ed20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f651400b010 tx=0x7f6514009b30 comp rx=0 tx=0).stop 2023-09-10T00:51:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.103+0000 7f652de32b80 1 -- 172.21.15.44:0/2623439535 shutdown_connections 2023-09-10T00:51:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.103+0000 7f652de32b80 1 --2- 172.21.15.44:0/2623439535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff185b0 0x55a71ff1ed20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.103+0000 7f652de32b80 1 --2- 172.21.15.44:0/2623439535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 0x55a71ff17de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.103+0000 7f652de32b80 1 --2- 172.21.15.44:0/2623439535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff130d0 0x55a71ff0da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.103+0000 7f652de32b80 1 -- 172.21.15.44:0/2623439535 >> 172.21.15.44:0/2623439535 conn(0x55a71fe83030 msgr2=0x55a71ff0d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.103+0000 7f652de32b80 1 -- 172.21.15.44:0/2623439535 shutdown_connections 2023-09-10T00:51:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.103+0000 7f652de32b80 1 -- 172.21.15.44:0/2623439535 wait complete. 2023-09-10T00:51:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.103+0000 7f652de32b80 1 -- start start 2023-09-10T00:51:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.103+0000 7f652de32b80 1 Processor -- start 2023-09-10T00:51:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f652de32b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 0x55a71ff23370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f652de32b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff130d0 0x55a71ff238b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f652de32b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff185b0 0x55a71ff21920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f652de32b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71ff22030 con 0x55a71ff185b0 2023-09-10T00:51:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f652de32b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71ff221a0 con 0x55a71ff12710 2023-09-10T00:51:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f652de32b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71ff22360 con 0x55a71ff130d0 2023-09-10T00:51:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f652de32b80 10 client.0 _ll_register_callbacks cb 0x7f650c09a320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:51:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f651bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff185b0 0x55a71ff21920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f651affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff130d0 0x55a71ff238b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f651b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 0x55a71ff23370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f651bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff185b0 0x55a71ff21920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52652/0 (socket says 172.21.15.44:52652) 2023-09-10T00:51:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f651bfff700 1 -- 172.21.15.44:0/1126116039 learned_addr learned my addr 172.21.15.44:0/1126116039 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f651bfff700 1 -- 172.21.15.44:0/1126116039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff130d0 msgr2=0x55a71ff238b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f651bfff700 1 --2- 172.21.15.44:0/1126116039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff130d0 0x55a71ff238b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f651bfff700 1 -- 172.21.15.44:0/1126116039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 msgr2=0x55a71ff23370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f651bfff700 1 --2- 172.21.15.44:0/1126116039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 0x55a71ff23370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f651bfff700 1 -- 172.21.15.44:0/1126116039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a71fc8cd80 con 0x55a71ff185b0 2023-09-10T00:51:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f651b7fe700 1 --2- 172.21.15.44:0/1126116039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 0x55a71ff23370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f651bfff700 1 --2- 172.21.15.44:0/1126116039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff185b0 0x55a71ff21920 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6514011520 tx=0x7f6514009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8462e14b5de0d404 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.104+0000 7f64faffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:51:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.105+0000 7f64faffd700 1 -- 172.21.15.44:0/1126116039 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400e070 con 0x55a71ff185b0 2023-09-10T00:51:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.105+0000 7f652de32b80 1 -- 172.21.15.44:0/1126116039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f651400f040 con 0x55a71ff185b0 2023-09-10T00:51:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.105+0000 7f64faffd700 1 -- 172.21.15.44:0/1126116039 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f6514007810 con 0x55a71ff185b0 2023-09-10T00:51:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.105+0000 7f64faffd700 1 -- 172.21.15.44:0/1126116039 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65140079d0 con 0x55a71ff185b0 2023-09-10T00:51:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.105+0000 7f652de32b80 10 client.17430 fetch_fsmap learned FSMap version 61 2023-09-10T00:51:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.105+0000 7f652de32b80 1 -- 172.21.15.44:0/1126116039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55a71fc75c00 con 0x55a71ff185b0 2023-09-10T00:51:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.105+0000 7f64faffd700 1 -- 172.21.15.44:0/1126116039 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f651400f040 con 0x55a71ff185b0 2023-09-10T00:51:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.105+0000 7f64faffd700 1 -- 172.21.15.44:0/1126116039 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f651401b020 con 0x55a71ff185b0 2023-09-10T00:51:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.105+0000 7f652de32b80 10 client.17430 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:51:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.105+0000 7f652de32b80 10 client.17430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:51:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.105+0000 7f652de32b80 20 client.17430 populate_metadata read hostname 'smithi044' 2023-09-10T00:51:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.105+0000 7f652de32b80 1 --2- 172.21.15.44:0/1126116039 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a71ff81130 0x55a71ff83550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.105+0000 7f652de32b80 4 client.17430 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:51:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.105+0000 7f652de32b80 1 -- 172.21.15.44:0/1126116039 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a71fe598c0 con 0x55a71ff81130 2023-09-10T00:51:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.105+0000 7f651b7fe700 1 --2- 172.21.15.44:0/1126116039 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a71ff81130 0x55a71ff83550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.106+0000 7f64faffd700 10 client.17430 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.106+0000 7f651b7fe700 1 --2- 172.21.15.44:0/1126116039 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a71ff81130 0x55a71ff83550 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48db728d5b4054f server_cookie=20f730818546ef0c in_seq=0 out_seq=0 2023-09-10T00:51:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f64faffd700 1 -- 172.21.15.44:0/1126116039 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a71fe598c0 con 0x55a71ff81130 2023-09-10T00:51:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f64faffd700 10 client.17430 handle_command_reply: tid=0 2023-09-10T00:51:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f652de32b80 1 client.17430 shutdown 2023-09-10T00:51:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f652de32b80 1 -- 172.21.15.44:0/1126116039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff185b0 msgr2=0x55a71ff21920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f652de32b80 1 --2- 172.21.15.44:0/1126116039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff185b0 0x55a71ff21920 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6514011520 tx=0x7f6514009510 comp rx=0 tx=0).stop 2023-09-10T00:51:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f652de32b80 1 -- 172.21.15.44:0/1126116039 shutdown_connections 2023-09-10T00:51:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f652de32b80 1 --2- 172.21.15.44:0/1126116039 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a71ff81130 0x55a71ff83550 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f652de32b80 1 --2- 172.21.15.44:0/1126116039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a71ff185b0 0x55a71ff21920 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f652de32b80 1 --2- 172.21.15.44:0/1126116039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a71ff12710 0x55a71ff23370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f652de32b80 1 --2- 172.21.15.44:0/1126116039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a71ff130d0 0x55a71ff238b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f64faffd700 0 client.17430 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f652de32b80 1 -- 172.21.15.44:0/1126116039 >> 172.21.15.44:0/1126116039 conn(0x55a71fe83030 msgr2=0x55a71ff17c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f652de32b80 1 -- 172.21.15.44:0/1126116039 shutdown_connections 2023-09-10T00:51:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f652de32b80 1 -- 172.21.15.44:0/1126116039 wait complete. 2023-09-10T00:51:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f652de32b80 20 client.17430 trim_cache size 0 max 16384 2023-09-10T00:51:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:51:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:51:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:51:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:51:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f6521c6c700 1 -- 172.21.15.44:0/2798234897 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f65080a07b0 msgr2=0x7f65080a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f6521c6c700 1 --2- 172.21.15.44:0/2798234897 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f65080a07b0 0x7f65080a2c70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f651c104a80 tx=0x7f6504009260 comp rx=0 tx=0).stop 2023-09-10T00:51:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f6521c6c700 1 -- 172.21.15.44:0/2798234897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0ff760 msgr2=0x7f651c19c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f6521c6c700 1 --2- 172.21.15.44:0/2798234897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0ff760 0x7f651c19c550 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f650c005910 tx=0x7f650c009580 comp rx=0 tx=0).stop 2023-09-10T00:51:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f6521c6c700 1 -- 172.21.15.44:0/2798234897 shutdown_connections 2023-09-10T00:51:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f6521c6c700 1 --2- 172.21.15.44:0/2798234897 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f65080a07b0 0x7f65080a2c70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f6521c6c700 1 --2- 172.21.15.44:0/2798234897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f651c1055b0 0x7f651c1963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f6521c6c700 1 --2- 172.21.15.44:0/2798234897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0ff760 0x7f651c19c550 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f6521c6c700 1 --2- 172.21.15.44:0/2798234897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f651c100120 0x7f651c19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.121+0000 7f6521c6c700 1 -- 172.21.15.44:0/2798234897 >> 172.21.15.44:0/2798234897 conn(0x7f651c074f30 msgr2=0x7f651c103200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.122+0000 7f6521c6c700 1 -- 172.21.15.44:0/2798234897 shutdown_connections 2023-09-10T00:51:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:41.122+0000 7f6521c6c700 1 -- 172.21.15.44:0/2798234897 wait complete. 2023-09-10T00:51:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:40 smithi131 ceph-mon[103713]: pgmap v343: 129 pgs: 129 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 23 MiB/s wr, 79 op/s 2023-09-10T00:51:42.006 INFO:teuthology.orchestra.run.smithi044.stderr:41700000 of 50000000 tuples (83%) done (elapsed 641.64 s, remaining 127.71 s) 2023-09-10T00:51:42.980 INFO:teuthology.orchestra.run.smithi044.stderr:41800000 of 50000000 tuples (83%) done (elapsed 642.61 s, remaining 126.06 s) 2023-09-10T00:51:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:42 smithi099 ceph-mon[102692]: pgmap v344: 129 pgs: 129 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 28 MiB/s wr, 104 op/s 2023-09-10T00:51:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:42 smithi131 ceph-mon[103713]: pgmap v344: 129 pgs: 129 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 28 MiB/s wr, 104 op/s 2023-09-10T00:51:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:42 smithi044 ceph-mon[98676]: pgmap v344: 129 pgs: 129 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 28 MiB/s wr, 104 op/s 2023-09-10T00:51:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:44 smithi099 ceph-mon[102692]: pgmap v345: 129 pgs: 129 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 13 MiB/s wr, 44 op/s 2023-09-10T00:51:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:44 smithi131 ceph-mon[103713]: pgmap v345: 129 pgs: 129 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 13 MiB/s wr, 44 op/s 2023-09-10T00:51:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:44 smithi044 ceph-mon[98676]: pgmap v345: 129 pgs: 129 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 13 MiB/s wr, 44 op/s 2023-09-10T00:51:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:46 smithi099 ceph-mon[102692]: pgmap v346: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 18 MiB/s wr, 68 op/s 2023-09-10T00:51:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:46 smithi131 ceph-mon[103713]: pgmap v346: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 18 MiB/s wr, 68 op/s 2023-09-10T00:51:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:46 smithi044 ceph-mon[98676]: pgmap v346: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 18 MiB/s wr, 68 op/s 2023-09-10T00:51:47.476 INFO:teuthology.orchestra.run.smithi044.stderr:41900000 of 50000000 tuples (83%) done (elapsed 647.11 s, remaining 125.10 s) 2023-09-10T00:51:48.474 INFO:teuthology.orchestra.run.smithi044.stderr:42000000 of 50000000 tuples (84%) done (elapsed 648.11 s, remaining 123.45 s) 2023-09-10T00:51:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:48 smithi099 ceph-mon[102692]: pgmap v347: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.5 MiB/s wr, 50 op/s 2023-09-10T00:51:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:48 smithi131 ceph-mon[103713]: pgmap v347: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.5 MiB/s wr, 50 op/s 2023-09-10T00:51:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:48 smithi044 ceph-mon[98676]: pgmap v347: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.5 MiB/s wr, 50 op/s 2023-09-10T00:51:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:50 smithi099 ceph-mon[102692]: pgmap v348: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.5 MiB/s wr, 50 op/s 2023-09-10T00:51:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:50 smithi131 ceph-mon[103713]: pgmap v348: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.5 MiB/s wr, 50 op/s 2023-09-10T00:51:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:50 smithi044 ceph-mon[98676]: pgmap v348: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.5 MiB/s wr, 50 op/s 2023-09-10T00:51:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:52 smithi099 ceph-mon[102692]: pgmap v349: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 14 MiB/s wr, 75 op/s 2023-09-10T00:51:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:52 smithi131 ceph-mon[103713]: pgmap v349: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 14 MiB/s wr, 75 op/s 2023-09-10T00:51:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:52 smithi044 ceph-mon[98676]: pgmap v349: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 14 MiB/s wr, 75 op/s 2023-09-10T00:51:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.244+0000 7fee57b91700 1 -- 172.21.15.44:0/3704761536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee500ff4a0 msgr2=0x7fee500ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.244+0000 7fee57b91700 1 --2- 172.21.15.44:0/3704761536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee500ff4a0 0x7fee500ff920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fee40009990 tx=0x7fee40009ca0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.244+0000 7fee57b91700 1 -- 172.21.15.44:0/3704761536 shutdown_connections 2023-09-10T00:51:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.244+0000 7fee57b91700 1 --2- 172.21.15.44:0/3704761536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee500ffe60 0x7fee5010c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.244+0000 7fee57b91700 1 --2- 172.21.15.44:0/3704761536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee50105560 0x7fee50105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.244+0000 7fee57b91700 1 --2- 172.21.15.44:0/3704761536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee500ff4a0 0x7fee500ff920 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.244+0000 7fee57b91700 1 -- 172.21.15.44:0/3704761536 >> 172.21.15.44:0/3704761536 conn(0x7fee500faf30 msgr2=0x7fee500fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.244+0000 7fee57b91700 1 -- 172.21.15.44:0/3704761536 shutdown_connections 2023-09-10T00:51:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.244+0000 7fee57b91700 1 -- 172.21.15.44:0/3704761536 wait complete. 2023-09-10T00:51:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee57b91700 1 Processor -- start 2023-09-10T00:51:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee57b91700 1 -- start start 2023-09-10T00:51:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee57b91700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee500ff4a0 0x7fee50196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee57b91700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee500ffe60 0x7fee50196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee57b91700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee50105560 0x7fee5019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee57b91700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5019a600 con 0x7fee500ffe60 2023-09-10T00:51:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee57b91700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5019a770 con 0x7fee50105560 2023-09-10T00:51:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee57b91700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5019a930 con 0x7fee500ff4a0 2023-09-10T00:51:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee5512c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee500ffe60 0x7fee50196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee5592d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee500ff4a0 0x7fee50196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee5512c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee500ffe60 0x7fee50196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52664/0 (socket says 172.21.15.44:52664) 2023-09-10T00:51:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee5512c700 1 -- 172.21.15.44:0/604106286 learned_addr learned my addr 172.21.15.44:0/604106286 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee5612e700 1 --2- 172.21.15.44:0/604106286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee50105560 0x7fee5019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee5592d700 1 -- 172.21.15.44:0/604106286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee50105560 msgr2=0x7fee5019a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee5592d700 1 --2- 172.21.15.44:0/604106286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee50105560 0x7fee5019a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee5592d700 1 -- 172.21.15.44:0/604106286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee500ffe60 msgr2=0x7fee50196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee5592d700 1 --2- 172.21.15.44:0/604106286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee500ffe60 0x7fee50196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee5592d700 1 -- 172.21.15.44:0/604106286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee48009c40 con 0x7fee500ff4a0 2023-09-10T00:51:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.245+0000 7fee5592d700 1 --2- 172.21.15.44:0/604106286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee500ff4a0 0x7fee50196500 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00e9f0 tx=0x7fee4c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.246+0000 7fee46ffd700 1 -- 172.21.15.44:0/604106286 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c00c9c0 con 0x7fee500ff4a0 2023-09-10T00:51:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.246+0000 7fee46ffd700 1 -- 172.21.15.44:0/604106286 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee4c010430 con 0x7fee500ff4a0 2023-09-10T00:51:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.246+0000 7fee57b91700 1 -- 172.21.15.44:0/604106286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee400095f0 con 0x7fee500ff4a0 2023-09-10T00:51:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.246+0000 7fee46ffd700 1 -- 172.21.15.44:0/604106286 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c018680 con 0x7fee500ff4a0 2023-09-10T00:51:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.246+0000 7fee57b91700 1 -- 172.21.15.44:0/604106286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee501a3390 con 0x7fee500ff4a0 2023-09-10T00:51:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.248+0000 7fee46ffd700 1 -- 172.21.15.44:0/604106286 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fee4c0105a0 con 0x7fee500ff4a0 2023-09-10T00:51:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.249+0000 7fee46ffd700 1 --2- 172.21.15.44:0/604106286 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fee3c0a07b0 0x7fee3c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.249+0000 7fee46ffd700 1 -- 172.21.15.44:0/604106286 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fee4c014070 con 0x7fee500ff4a0 2023-09-10T00:51:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.249+0000 7fee5512c700 1 --2- 172.21.15.44:0/604106286 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fee3c0a07b0 0x7fee3c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.249+0000 7fee63d57b80 1 Processor -- start 2023-09-10T00:51:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.249+0000 7fee5512c700 1 --2- 172.21.15.44:0/604106286 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fee3c0a07b0 0x7fee3c0a2c70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee50197e20 tx=0x7fee40005960 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.249+0000 7fee63d57b80 1 -- start start 2023-09-10T00:51:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee63d57b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d331a0 0x55e087d33580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee63d57b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d2d300 0x55e087d2d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee63d57b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 0x55e087d39f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee63d57b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e087c866d0 con 0x55e087d331a0 2023-09-10T00:51:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee63d57b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e087c83ff0 con 0x55e087d2dcc0 2023-09-10T00:51:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee63d57b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e087c7f8e0 con 0x55e087d2d300 2023-09-10T00:51:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee5592d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d331a0 0x55e087d33580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee5592d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d331a0 0x55e087d33580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52670/0 (socket says 172.21.15.44:52670) 2023-09-10T00:51:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee5512c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d2d300 0x55e087d2d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee5592d700 1 -- 172.21.15.44:0/58296458 learned_addr learned my addr 172.21.15.44:0/58296458 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee5612e700 1 --2- 172.21.15.44:0/58296458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 0x55e087d39f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee5592d700 1 -- 172.21.15.44:0/58296458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d2d300 msgr2=0x55e087d2d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee5592d700 1 --2- 172.21.15.44:0/58296458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d2d300 0x55e087d2d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee5592d700 1 -- 172.21.15.44:0/58296458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 msgr2=0x55e087d39f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee5592d700 1 --2- 172.21.15.44:0/58296458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 0x55e087d39f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee5592d700 1 -- 172.21.15.44:0/58296458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e087aabd80 con 0x55e087d331a0 2023-09-10T00:51:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.250+0000 7fee5592d700 1 --2- 172.21.15.44:0/58296458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d331a0 0x55e087d33580 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fee4c099490 tx=0x7fee4c095490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a915b105f9ddf85d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.251+0000 7fee3bfff700 1 -- 172.21.15.44:0/58296458 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c010880 con 0x55e087d331a0 2023-09-10T00:51:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.251+0000 7fee3bfff700 1 -- 172.21.15.44:0/58296458 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee4c099e90 con 0x55e087d331a0 2023-09-10T00:51:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.251+0000 7fee3bfff700 1 -- 172.21.15.44:0/58296458 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c098c80 con 0x55e087d331a0 2023-09-10T00:51:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.251+0000 7fee63d57b80 1 -- 172.21.15.44:0/58296458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d331a0 msgr2=0x55e087d33580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.251+0000 7fee63d57b80 1 --2- 172.21.15.44:0/58296458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d331a0 0x55e087d33580 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fee4c099490 tx=0x7fee4c095490 comp rx=0 tx=0).stop 2023-09-10T00:51:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.251+0000 7fee63d57b80 1 -- 172.21.15.44:0/58296458 shutdown_connections 2023-09-10T00:51:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.251+0000 7fee63d57b80 1 --2- 172.21.15.44:0/58296458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 0x55e087d39f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.251+0000 7fee63d57b80 1 --2- 172.21.15.44:0/58296458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d331a0 0x55e087d33580 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.251+0000 7fee63d57b80 1 --2- 172.21.15.44:0/58296458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d2d300 0x55e087d2d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.251+0000 7fee63d57b80 1 -- 172.21.15.44:0/58296458 >> 172.21.15.44:0/58296458 conn(0x55e087ca2030 msgr2=0x55e087d28560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.251+0000 7fee63d57b80 1 -- 172.21.15.44:0/58296458 shutdown_connections 2023-09-10T00:51:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.251+0000 7fee63d57b80 1 -- 172.21.15.44:0/58296458 wait complete. 2023-09-10T00:51:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.252+0000 7fee63d57b80 1 -- start start 2023-09-10T00:51:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.252+0000 7fee63d57b80 1 Processor -- start 2023-09-10T00:51:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.252+0000 7fee63d57b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2d300 0x55e087d39ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.252+0000 7fee63d57b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d2dcc0 0x55e087d37140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.252+0000 7fee63d57b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d331a0 0x55e087d37880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.252+0000 7fee63d57b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e087c1db10 con 0x55e087d2dcc0 2023-09-10T00:51:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.252+0000 7fee63d57b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e087c1d810 con 0x55e087d2d300 2023-09-10T00:51:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.252+0000 7fee63d57b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e087b018c0 con 0x55e087d331a0 2023-09-10T00:51:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.252+0000 7fee63d57b80 10 client.0 _ll_register_callbacks cb 0x55e087b67a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:51:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee5512c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d2dcc0 0x55e087d37140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee5592d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2d300 0x55e087d39ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee5512c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d2dcc0 0x55e087d37140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52678/0 (socket says 172.21.15.44:52678) 2023-09-10T00:51:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee5512c700 1 -- 172.21.15.44:0/2998504135 learned_addr learned my addr 172.21.15.44:0/2998504135 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee5612e700 1 --2- 172.21.15.44:0/2998504135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d331a0 0x55e087d37880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee5592d700 1 -- 172.21.15.44:0/2998504135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d331a0 msgr2=0x55e087d37880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee5592d700 1 --2- 172.21.15.44:0/2998504135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d331a0 0x55e087d37880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee5592d700 1 -- 172.21.15.44:0/2998504135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d2dcc0 msgr2=0x55e087d37140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee5592d700 1 --2- 172.21.15.44:0/2998504135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d2dcc0 0x55e087d37140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee5592d700 1 -- 172.21.15.44:0/2998504135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee501a3390 con 0x55e087d2d300 2023-09-10T00:51:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee5512c700 1 --2- 172.21.15.44:0/2998504135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d2dcc0 0x55e087d37140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee5592d700 1 --2- 172.21.15.44:0/2998504135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2d300 0x55e087d39ec0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee4c099490 tx=0x7fee4c0c9a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34f6a08e83740e51 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee397fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:51:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee397fa700 1 -- 172.21.15.44:0/2998504135 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0b7070 con 0x55e087d2d300 2023-09-10T00:51:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee397fa700 1 -- 172.21.15.44:0/2998504135 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fee4c010880 con 0x55e087d2d300 2023-09-10T00:51:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee63d57b80 1 -- 172.21.15.44:0/2998504135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e087bd7490 con 0x55e087d2d300 2023-09-10T00:51:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee397fa700 1 -- 172.21.15.44:0/2998504135 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee4c099790 con 0x55e087d2d300 2023-09-10T00:51:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee63d57b80 10 client.27110 fetch_fsmap learned FSMap version 61 2023-09-10T00:51:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.253+0000 7fee63d57b80 1 -- 172.21.15.44:0/2998504135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55e087a94c00 con 0x55e087d2d300 2023-09-10T00:51:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.254+0000 7fee397fa700 1 -- 172.21.15.44:0/2998504135 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee4c07c020 con 0x55e087d2d300 2023-09-10T00:51:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.254+0000 7fee397fa700 1 -- 172.21.15.44:0/2998504135 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fee4c0cc2a0 con 0x55e087d2d300 2023-09-10T00:51:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.254+0000 7fee63d57b80 10 client.27110 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:51:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.254+0000 7fee63d57b80 10 client.27110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:51:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.254+0000 7fee63d57b80 20 client.27110 populate_metadata read hostname 'smithi044' 2023-09-10T00:51:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.254+0000 7fee63d57b80 1 --2- 172.21.15.44:0/2998504135 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e087d470c0 0x55e087d494e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.254+0000 7fee63d57b80 4 client.27110 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:51:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.254+0000 7fee63d57b80 1 -- 172.21.15.44:0/2998504135 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e087c94840 con 0x55e087d470c0 2023-09-10T00:51:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.254+0000 7fee5512c700 1 --2- 172.21.15.44:0/2998504135 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e087d470c0 0x55e087d494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.254+0000 7fee5512c700 1 --2- 172.21.15.44:0/2998504135 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e087d470c0 0x55e087d494e0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcf4a07eccda30b9 server_cookie=3c51feea65267075 in_seq=0 out_seq=0 2023-09-10T00:51:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.254+0000 7fee397fa700 10 client.27110 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.255+0000 7fee397fa700 1 -- 172.21.15.44:0/2998504135 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e087c94840 con 0x55e087d470c0 2023-09-10T00:51:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.255+0000 7fee397fa700 10 client.27110 handle_command_reply: tid=0 2023-09-10T00:51:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.255+0000 7fee63d57b80 1 client.27110 shutdown 2023-09-10T00:51:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.255+0000 7fee63d57b80 1 -- 172.21.15.44:0/2998504135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2d300 msgr2=0x55e087d39ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.255+0000 7fee63d57b80 1 --2- 172.21.15.44:0/2998504135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2d300 0x55e087d39ec0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee4c099490 tx=0x7fee4c0c9a00 comp rx=0 tx=0).stop 2023-09-10T00:51:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.255+0000 7fee63d57b80 1 -- 172.21.15.44:0/2998504135 shutdown_connections 2023-09-10T00:51:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.255+0000 7fee63d57b80 1 --2- 172.21.15.44:0/2998504135 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e087d470c0 0x55e087d494e0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.255+0000 7fee63d57b80 1 --2- 172.21.15.44:0/2998504135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d331a0 0x55e087d37880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.255+0000 7fee63d57b80 1 --2- 172.21.15.44:0/2998504135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2d300 0x55e087d39ec0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.255+0000 7fee63d57b80 1 --2- 172.21.15.44:0/2998504135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d2dcc0 0x55e087d37140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.255+0000 7fee397fa700 0 client.27110 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.255+0000 7fee63d57b80 1 -- 172.21.15.44:0/2998504135 >> 172.21.15.44:0/2998504135 conn(0x55e087ca2030 msgr2=0x55e087d277a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.255+0000 7fee63d57b80 1 -- 172.21.15.44:0/2998504135 shutdown_connections 2023-09-10T00:51:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.255+0000 7fee63d57b80 1 -- 172.21.15.44:0/2998504135 wait complete. 2023-09-10T00:51:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.256+0000 7fee63d57b80 20 client.27110 trim_cache size 0 max 16384 2023-09-10T00:51:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee63d57b80 1 Processor -- start 2023-09-10T00:51:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee63d57b80 1 -- start start 2023-09-10T00:51:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee63d57b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d2d300 0x55e087d2d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee63d57b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 0x55e087d39a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee63d57b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d331a0 0x55e087d31230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee63d57b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e087d32e00 con 0x55e087d2d300 2023-09-10T00:51:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee63d57b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e087d38720 con 0x55e087d2dcc0 2023-09-10T00:51:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee63d57b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e087d33ab0 con 0x55e087d331a0 2023-09-10T00:51:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee5592d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d2d300 0x55e087d2d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee5592d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d2d300 0x55e087d2d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52684/0 (socket says 172.21.15.44:52684) 2023-09-10T00:51:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee5592d700 1 -- 172.21.15.44:0/3641562286 learned_addr learned my addr 172.21.15.44:0/3641562286 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee5512c700 1 --2- 172.21.15.44:0/3641562286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 0x55e087d39a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee5612e700 1 --2- 172.21.15.44:0/3641562286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d331a0 0x55e087d31230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee5592d700 1 -- 172.21.15.44:0/3641562286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d331a0 msgr2=0x55e087d31230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee5592d700 1 --2- 172.21.15.44:0/3641562286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d331a0 0x55e087d31230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee5592d700 1 -- 172.21.15.44:0/3641562286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 msgr2=0x55e087d39a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee5592d700 1 --2- 172.21.15.44:0/3641562286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 0x55e087d39a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee5592d700 1 -- 172.21.15.44:0/3641562286 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e087ad21f0 con 0x55e087d2d300 2023-09-10T00:51:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee5592d700 1 --2- 172.21.15.44:0/3641562286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d2d300 0x55e087d2d6e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fee4c096200 tx=0x7fee4c09d4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9fb0304953d79c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee44d89700 1 -- 172.21.15.44:0/3641562286 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0b7070 con 0x55e087d2d300 2023-09-10T00:51:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee44d89700 1 -- 172.21.15.44:0/3641562286 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee4c07c020 con 0x55e087d2d300 2023-09-10T00:51:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.271+0000 7fee44d89700 1 -- 172.21.15.44:0/3641562286 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0a1800 con 0x55e087d2d300 2023-09-10T00:51:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.272+0000 7fee63d57b80 1 -- 172.21.15.44:0/3641562286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d2d300 msgr2=0x55e087d2d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.272+0000 7fee63d57b80 1 --2- 172.21.15.44:0/3641562286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d2d300 0x55e087d2d6e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fee4c096200 tx=0x7fee4c09d4f0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.272+0000 7fee63d57b80 1 -- 172.21.15.44:0/3641562286 shutdown_connections 2023-09-10T00:51:54.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.272+0000 7fee63d57b80 1 --2- 172.21.15.44:0/3641562286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d331a0 0x55e087d31230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.272+0000 7fee63d57b80 1 --2- 172.21.15.44:0/3641562286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d2d300 0x55e087d2d6e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.272+0000 7fee63d57b80 1 --2- 172.21.15.44:0/3641562286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 0x55e087d39a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.272+0000 7fee63d57b80 1 -- 172.21.15.44:0/3641562286 >> 172.21.15.44:0/3641562286 conn(0x55e087ca2030 msgr2=0x55e087d321b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:54.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.272+0000 7fee63d57b80 1 -- 172.21.15.44:0/3641562286 shutdown_connections 2023-09-10T00:51:54.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.272+0000 7fee63d57b80 1 -- 172.21.15.44:0/3641562286 wait complete. 2023-09-10T00:51:54.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.272+0000 7fee63d57b80 1 -- start start 2023-09-10T00:51:54.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.272+0000 7fee63d57b80 1 Processor -- start 2023-09-10T00:51:54.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee63d57b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d2d300 0x55e087d3e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee63d57b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 0x55e087d3b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee63d57b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d331a0 0x55e087d3c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee63d57b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e087d3c760 con 0x55e087d331a0 2023-09-10T00:51:54.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee63d57b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e087d99f10 con 0x55e087d2dcc0 2023-09-10T00:51:54.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee63d57b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e087d9a050 con 0x55e087d2d300 2023-09-10T00:51:54.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee63d57b80 10 client.0 _ll_register_callbacks cb 0x7fee4800e280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:51:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee5612e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d331a0 0x55e087d3c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee5512c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 0x55e087d3b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee5592d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d2d300 0x55e087d3e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee5512c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 0x55e087d3b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54432/0 (socket says 172.21.15.44:54432) 2023-09-10T00:51:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee5612e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d331a0 0x55e087d3c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52694/0 (socket says 172.21.15.44:52694) 2023-09-10T00:51:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee5512c700 1 -- 172.21.15.44:0/679950408 learned_addr learned my addr 172.21.15.44:0/679950408 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:51:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee5612e700 1 -- 172.21.15.44:0/679950408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d2d300 msgr2=0x55e087d3e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee5612e700 1 --2- 172.21.15.44:0/679950408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d2d300 0x55e087d3e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee5612e700 1 -- 172.21.15.44:0/679950408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 msgr2=0x55e087d3b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:51:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee5612e700 1 --2- 172.21.15.44:0/679950408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 0x55e087d3b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee5612e700 1 -- 172.21.15.44:0/679950408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e087a94c00 con 0x55e087d331a0 2023-09-10T00:51:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee5512c700 1 --2- 172.21.15.44:0/679950408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 0x55e087d3b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:51:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee5612e700 1 --2- 172.21.15.44:0/679950408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d331a0 0x55e087d3c050 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fee48009fd0 tx=0x7fee4800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6403e53ee770ee66 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:51:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.273+0000 7fee397fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:51:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.274+0000 7fee397fa700 1 -- 172.21.15.44:0/679950408 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4801c070 con 0x55e087d331a0 2023-09-10T00:51:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.274+0000 7fee63d57b80 1 -- 172.21.15.44:0/679950408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee4c099e90 con 0x55e087d331a0 2023-09-10T00:51:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.274+0000 7fee397fa700 1 -- 172.21.15.44:0/679950408 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fee48007bc0 con 0x55e087d331a0 2023-09-10T00:51:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.274+0000 7fee397fa700 1 -- 172.21.15.44:0/679950408 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee4800e040 con 0x55e087d331a0 2023-09-10T00:51:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.274+0000 7fee63d57b80 10 client.17448 fetch_fsmap learned FSMap version 61 2023-09-10T00:51:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.274+0000 7fee63d57b80 1 -- 172.21.15.44:0/679950408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55e087aaa250 con 0x55e087d331a0 2023-09-10T00:51:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.274+0000 7fee397fa700 1 -- 172.21.15.44:0/679950408 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee4c099e90 con 0x55e087d331a0 2023-09-10T00:51:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.274+0000 7fee397fa700 1 -- 172.21.15.44:0/679950408 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fee48016400 con 0x55e087d331a0 2023-09-10T00:51:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.274+0000 7fee63d57b80 10 client.17448 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:51:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.274+0000 7fee63d57b80 10 client.17448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:51:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.274+0000 7fee63d57b80 20 client.17448 populate_metadata read hostname 'smithi044' 2023-09-10T00:51:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.274+0000 7fee63d57b80 1 --2- 172.21.15.44:0/679950408 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e087d9b8e0 0x55e087d9dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:51:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.274+0000 7fee63d57b80 4 client.17448 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:51:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.274+0000 7fee63d57b80 1 -- 172.21.15.44:0/679950408 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e087c788c0 con 0x55e087d9b8e0 2023-09-10T00:51:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.274+0000 7fee5592d700 1 --2- 172.21.15.44:0/679950408 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e087d9b8e0 0x55e087d9dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:51:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.275+0000 7fee397fa700 10 client.17448 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.275+0000 7fee5592d700 1 --2- 172.21.15.44:0/679950408 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e087d9b8e0 0x55e087d9dd00 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d8e49019a80739 server_cookie=f814495b176a95e8 in_seq=0 out_seq=0 2023-09-10T00:51:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee397fa700 1 -- 172.21.15.44:0/679950408 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e087c788c0 con 0x55e087d9b8e0 2023-09-10T00:51:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee397fa700 10 client.17448 handle_command_reply: tid=0 2023-09-10T00:51:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee63d57b80 1 client.17448 shutdown 2023-09-10T00:51:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee63d57b80 1 -- 172.21.15.44:0/679950408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d331a0 msgr2=0x55e087d3c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee63d57b80 1 --2- 172.21.15.44:0/679950408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d331a0 0x55e087d3c050 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fee48009fd0 tx=0x7fee4800ef30 comp rx=0 tx=0).stop 2023-09-10T00:51:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee63d57b80 1 -- 172.21.15.44:0/679950408 shutdown_connections 2023-09-10T00:51:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee63d57b80 1 --2- 172.21.15.44:0/679950408 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e087d9b8e0 0x55e087d9dd00 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee63d57b80 1 --2- 172.21.15.44:0/679950408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e087d331a0 0x55e087d3c050 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee63d57b80 1 --2- 172.21.15.44:0/679950408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e087d2d300 0x55e087d3e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee63d57b80 1 --2- 172.21.15.44:0/679950408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e087d2dcc0 0x55e087d3b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee397fa700 0 client.17448 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:51:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee63d57b80 1 -- 172.21.15.44:0/679950408 >> 172.21.15.44:0/679950408 conn(0x55e087ca2030 msgr2=0x55e087d31230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee63d57b80 1 -- 172.21.15.44:0/679950408 shutdown_connections 2023-09-10T00:51:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee63d57b80 1 -- 172.21.15.44:0/679950408 wait complete. 2023-09-10T00:51:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee63d57b80 20 client.17448 trim_cache size 0 max 16384 2023-09-10T00:51:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:51:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:51:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:51:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:51:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee57b91700 1 -- 172.21.15.44:0/604106286 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fee3c0a07b0 msgr2=0x7fee3c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee57b91700 1 --2- 172.21.15.44:0/604106286 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fee3c0a07b0 0x7fee3c0a2c70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee50197e20 tx=0x7fee40005960 comp rx=0 tx=0).stop 2023-09-10T00:51:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee57b91700 1 -- 172.21.15.44:0/604106286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee500ff4a0 msgr2=0x7fee50196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:51:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee57b91700 1 --2- 172.21.15.44:0/604106286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee500ff4a0 0x7fee50196500 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00e9f0 tx=0x7fee4c00ed00 comp rx=0 tx=0).stop 2023-09-10T00:51:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.291+0000 7fee57b91700 1 -- 172.21.15.44:0/604106286 shutdown_connections 2023-09-10T00:51:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.292+0000 7fee57b91700 1 --2- 172.21.15.44:0/604106286 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fee3c0a07b0 0x7fee3c0a2c70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.292+0000 7fee57b91700 1 --2- 172.21.15.44:0/604106286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee50105560 0x7fee5019a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.292+0000 7fee57b91700 1 --2- 172.21.15.44:0/604106286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee500ff4a0 0x7fee50196500 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.292+0000 7fee57b91700 1 --2- 172.21.15.44:0/604106286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee500ffe60 0x7fee50196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:51:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.292+0000 7fee57b91700 1 -- 172.21.15.44:0/604106286 >> 172.21.15.44:0/604106286 conn(0x7fee500faf30 msgr2=0x7fee500fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:51:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.292+0000 7fee57b91700 1 -- 172.21.15.44:0/604106286 shutdown_connections 2023-09-10T00:51:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:51:54.292+0000 7fee57b91700 1 -- 172.21.15.44:0/604106286 wait complete. 2023-09-10T00:51:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:54 smithi099 ceph-mon[102692]: pgmap v350: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.7 MiB/s wr, 50 op/s 2023-09-10T00:51:55.178 INFO:teuthology.orchestra.run.smithi044.stderr:42100000 of 50000000 tuples (84%) done (elapsed 654.81 s, remaining 122.87 s) 2023-09-10T00:51:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:54 smithi131 ceph-mon[103713]: pgmap v350: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.7 MiB/s wr, 50 op/s 2023-09-10T00:51:55.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:54 smithi044 ceph-mon[98676]: pgmap v350: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.7 MiB/s wr, 50 op/s 2023-09-10T00:51:56.037 INFO:teuthology.orchestra.run.smithi044.stderr:42200000 of 50000000 tuples (84%) done (elapsed 655.67 s, remaining 121.19 s) 2023-09-10T00:51:56.962 INFO:teuthology.orchestra.run.smithi044.stderr:42300000 of 50000000 tuples (84%) done (elapsed 656.60 s, remaining 119.52 s) 2023-09-10T00:51:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:56 smithi099 ceph-mon[102692]: pgmap v351: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 13 MiB/s wr, 64 op/s 2023-09-10T00:51:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:56 smithi131 ceph-mon[103713]: pgmap v351: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 13 MiB/s wr, 64 op/s 2023-09-10T00:51:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:56 smithi044 ceph-mon[98676]: pgmap v351: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 13 MiB/s wr, 64 op/s 2023-09-10T00:51:57.819 INFO:teuthology.orchestra.run.smithi044.stderr:42400000 of 50000000 tuples (84%) done (elapsed 657.45 s, remaining 117.85 s) 2023-09-10T00:51:58.674 INFO:teuthology.orchestra.run.smithi044.stderr:42500000 of 50000000 tuples (85%) done (elapsed 658.31 s, remaining 116.17 s) 2023-09-10T00:51:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:51:58 smithi099 ceph-mon[102692]: pgmap v352: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.7 MiB/s wr, 40 op/s 2023-09-10T00:51:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:51:58 smithi131 ceph-mon[103713]: pgmap v352: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.7 MiB/s wr, 40 op/s 2023-09-10T00:51:59.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:51:58 smithi044 ceph-mon[98676]: pgmap v352: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.7 MiB/s wr, 40 op/s 2023-09-10T00:51:59.727 INFO:teuthology.orchestra.run.smithi044.stderr:42600000 of 50000000 tuples (85%) done (elapsed 659.36 s, remaining 114.54 s) 2023-09-10T00:52:00.731 INFO:teuthology.orchestra.run.smithi044.stderr:42700000 of 50000000 tuples (85%) done (elapsed 660.37 s, remaining 112.90 s) 2023-09-10T00:52:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:00 smithi099 ceph-mon[102692]: pgmap v353: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.7 MiB/s wr, 40 op/s 2023-09-10T00:52:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:00 smithi131 ceph-mon[103713]: pgmap v353: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.7 MiB/s wr, 40 op/s 2023-09-10T00:52:01.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:00 smithi044 ceph-mon[98676]: pgmap v353: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.7 MiB/s wr, 40 op/s 2023-09-10T00:52:01.816 INFO:teuthology.orchestra.run.smithi044.stderr:42800000 of 50000000 tuples (85%) done (elapsed 661.45 s, remaining 111.27 s) 2023-09-10T00:52:02.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:01 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:52:02.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:01 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:52:02.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:01 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:52:02.776 INFO:teuthology.orchestra.run.smithi044.stderr:42900000 of 50000000 tuples (85%) done (elapsed 662.41 s, remaining 109.63 s) 2023-09-10T00:52:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:02 smithi099 ceph-mon[102692]: pgmap v354: 129 pgs: 129 active+clean; 8.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 24 MiB/s wr, 68 op/s 2023-09-10T00:52:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:02 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:52:03.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:02 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:52:03.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:02 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:52:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:02 smithi131 ceph-mon[103713]: pgmap v354: 129 pgs: 129 active+clean; 8.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 24 MiB/s wr, 68 op/s 2023-09-10T00:52:03.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:02 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:52:03.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:02 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:52:03.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:02 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:52:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:02 smithi044 ceph-mon[98676]: pgmap v354: 129 pgs: 129 active+clean; 8.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 24 MiB/s wr, 68 op/s 2023-09-10T00:52:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:02 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:52:03.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:02 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:52:03.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:02 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:52:03.755 INFO:teuthology.orchestra.run.smithi044.stderr:43000000 of 50000000 tuples (86%) done (elapsed 663.39 s, remaining 107.99 s) 2023-09-10T00:52:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:43100000 of 50000000 tuples (86%) done (elapsed 664.50 s, remaining 106.38 s) 2023-09-10T00:52:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:04 smithi099 ceph-mon[102692]: pgmap v355: 129 pgs: 129 active+clean; 8.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 42 op/s 2023-09-10T00:52:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:04 smithi131 ceph-mon[103713]: pgmap v355: 129 pgs: 129 active+clean; 8.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 42 op/s 2023-09-10T00:52:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:04 smithi044 ceph-mon[98676]: pgmap v355: 129 pgs: 129 active+clean; 8.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 42 op/s 2023-09-10T00:52:05.795 INFO:teuthology.orchestra.run.smithi044.stderr:43200000 of 50000000 tuples (86%) done (elapsed 665.43 s, remaining 104.74 s) 2023-09-10T00:52:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:43300000 of 50000000 tuples (86%) done (elapsed 666.42 s, remaining 103.12 s) 2023-09-10T00:52:07.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:06 smithi099 ceph-mon[102692]: pgmap v356: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 34 MiB/s wr, 111 op/s 2023-09-10T00:52:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:06 smithi131 ceph-mon[103713]: pgmap v356: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 34 MiB/s wr, 111 op/s 2023-09-10T00:52:07.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:06 smithi044 ceph-mon[98676]: pgmap v356: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 34 MiB/s wr, 111 op/s 2023-09-10T00:52:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.418+0000 7fb0e31f4700 1 -- 172.21.15.44:0/1090645830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0dc100f90 msgr2=0x7fb0dc10d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.418+0000 7fb0e31f4700 1 --2- 172.21.15.44:0/1090645830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0dc100f90 0x7fb0dc10d450 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400b0d0 tx=0x7fb0d400b3e0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.418+0000 7fb0e31f4700 1 -- 172.21.15.44:0/1090645830 shutdown_connections 2023-09-10T00:52:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.418+0000 7fb0e31f4700 1 --2- 172.21.15.44:0/1090645830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0dc100f90 0x7fb0dc10d450 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.418+0000 7fb0e31f4700 1 --2- 172.21.15.44:0/1090645830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc106690 0x7fb0dc106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.418+0000 7fb0e31f4700 1 --2- 172.21.15.44:0/1090645830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0dc1005d0 0x7fb0dc100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.418+0000 7fb0e31f4700 1 -- 172.21.15.44:0/1090645830 >> 172.21.15.44:0/1090645830 conn(0x7fb0dc0fc0c0 msgr2=0x7fb0dc0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.418+0000 7fb0e31f4700 1 -- 172.21.15.44:0/1090645830 shutdown_connections 2023-09-10T00:52:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.419+0000 7fb0e31f4700 1 -- 172.21.15.44:0/1090645830 wait complete. 2023-09-10T00:52:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.419+0000 7fb0e31f4700 1 Processor -- start 2023-09-10T00:52:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.419+0000 7fb0e31f4700 1 -- start start 2023-09-10T00:52:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.419+0000 7fb0e31f4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc1005d0 0x7fb0dc196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.419+0000 7fb0e31f4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0dc100f90 0x7fb0dc196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.419+0000 7fb0e31f4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0dc106690 0x7fb0dc19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.419+0000 7fb0e31f4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc19b900 con 0x7fb0dc100f90 2023-09-10T00:52:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.419+0000 7fb0e31f4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc19ba70 con 0x7fb0dc1005d0 2023-09-10T00:52:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.419+0000 7fb0e31f4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc19bc30 con 0x7fb0dc106690 2023-09-10T00:52:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.420+0000 7fb0e0f90700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc1005d0 0x7fb0dc196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.420+0000 7fb0dbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0dc100f90 0x7fb0dc196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.420+0000 7fb0e0f90700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc1005d0 0x7fb0dc196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54444/0 (socket says 172.21.15.44:54444) 2023-09-10T00:52:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.420+0000 7fb0e0f90700 1 -- 172.21.15.44:0/2253348626 learned_addr learned my addr 172.21.15.44:0/2253348626 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.420+0000 7fb0e1791700 1 --2- 172.21.15.44:0/2253348626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0dc106690 0x7fb0dc19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.420+0000 7fb0e0f90700 1 -- 172.21.15.44:0/2253348626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0dc106690 msgr2=0x7fb0dc19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.420+0000 7fb0e0f90700 1 --2- 172.21.15.44:0/2253348626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0dc106690 0x7fb0dc19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.420+0000 7fb0e0f90700 1 -- 172.21.15.44:0/2253348626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0dc100f90 msgr2=0x7fb0dc196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.420+0000 7fb0e0f90700 1 --2- 172.21.15.44:0/2253348626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0dc100f90 0x7fb0dc196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.420+0000 7fb0e0f90700 1 -- 172.21.15.44:0/2253348626 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d4009d00 con 0x7fb0dc1005d0 2023-09-10T00:52:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.420+0000 7fb0e0f90700 1 --2- 172.21.15.44:0/2253348626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc1005d0 0x7fb0dc196750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000e9a0 tx=0x7fb0d000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.420+0000 7fb0d9ffb700 1 -- 172.21.15.44:0/2253348626 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0009790 con 0x7fb0dc1005d0 2023-09-10T00:52:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.420+0000 7fb0e31f4700 1 -- 172.21.15.44:0/2253348626 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0dc19bf10 con 0x7fb0dc1005d0 2023-09-10T00:52:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.420+0000 7fb0d9ffb700 1 -- 172.21.15.44:0/2253348626 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0d0010430 con 0x7fb0dc1005d0 2023-09-10T00:52:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.420+0000 7fb0d9ffb700 1 -- 172.21.15.44:0/2253348626 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d000cea0 con 0x7fb0dc1005d0 2023-09-10T00:52:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.421+0000 7fb0e31f4700 1 -- 172.21.15.44:0/2253348626 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc1a37c0 con 0x7fb0dc1005d0 2023-09-10T00:52:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.423+0000 7fb0d9ffb700 1 -- 172.21.15.44:0/2253348626 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fb0d00105a0 con 0x7fb0dc1005d0 2023-09-10T00:52:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.424+0000 7fb0d9ffb700 1 --2- 172.21.15.44:0/2253348626 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb0cc0a07b0 0x7fb0cc0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.424+0000 7fb0d9ffb700 1 -- 172.21.15.44:0/2253348626 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb0d0014070 con 0x7fb0dc1005d0 2023-09-10T00:52:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.424+0000 7fb0ef3bab80 1 Processor -- start 2023-09-10T00:52:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.424+0000 7fb0dbfff700 1 --2- 172.21.15.44:0/2253348626 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb0cc0a07b0 0x7fb0cc0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.424+0000 7fb0ef3bab80 1 -- start start 2023-09-10T00:52:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0ef3bab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e0c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0ef3bab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 0x55f1d1e06780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0ef3bab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e12f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0ef3bab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1d5f6d0 con 0x55f1d1e0c1a0 2023-09-10T00:52:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0ef3bab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1d5cff0 con 0x55f1d1e06cc0 2023-09-10T00:52:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0ef3bab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1d588e0 con 0x55f1d1e06300 2023-09-10T00:52:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0e0f90700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e0c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0e0f90700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e0c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52712/0 (socket says 172.21.15.44:52712) 2023-09-10T00:52:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0e0f90700 1 -- 172.21.15.44:0/3690605257 learned_addr learned my addr 172.21.15.44:0/3690605257 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0dbfff700 1 --2- 172.21.15.44:0/2253348626 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb0cc0a07b0 0x7fb0cc0a2c70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc197f70 tx=0x7fb0c8009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0e1791700 1 --2- 172.21.15.44:0/3690605257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e12f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0dbfff700 1 --2- 172.21.15.44:0/3690605257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 0x55f1d1e06780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0e1791700 1 -- 172.21.15.44:0/3690605257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 msgr2=0x55f1d1e06780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0e1791700 1 --2- 172.21.15.44:0/3690605257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 0x55f1d1e06780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0e1791700 1 -- 172.21.15.44:0/3690605257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 msgr2=0x55f1d1e0c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0e1791700 1 --2- 172.21.15.44:0/3690605257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e0c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0e1791700 1 -- 172.21.15.44:0/3690605257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1d1b84d80 con 0x55f1d1e06cc0 2023-09-10T00:52:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.425+0000 7fb0e1791700 1 --2- 172.21.15.44:0/3690605257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e12f30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400b770 tx=0x7fb0d4009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=695a13c08c26eb74 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.426+0000 7fb0c6ffd700 1 -- 172.21.15.44:0/3690605257 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d400e040 con 0x55f1d1e06cc0 2023-09-10T00:52:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.426+0000 7fb0c6ffd700 1 -- 172.21.15.44:0/3690605257 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0d4007b90 con 0x55f1d1e06cc0 2023-09-10T00:52:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.426+0000 7fb0c6ffd700 1 -- 172.21.15.44:0/3690605257 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4003a00 con 0x55f1d1e06cc0 2023-09-10T00:52:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.426+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/3690605257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06cc0 msgr2=0x55f1d1e12f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.426+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/3690605257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e12f30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400b770 tx=0x7fb0d4009c60 comp rx=0 tx=0).stop 2023-09-10T00:52:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.426+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/3690605257 shutdown_connections 2023-09-10T00:52:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.426+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/3690605257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e12f30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.426+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/3690605257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e0c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.426+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/3690605257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 0x55f1d1e06780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.426+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/3690605257 >> 172.21.15.44:0/3690605257 conn(0x55f1d1d7b030 msgr2=0x55f1d1e01560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.427+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/3690605257 shutdown_connections 2023-09-10T00:52:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.427+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/3690605257 wait complete. 2023-09-10T00:52:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.427+0000 7fb0ef3bab80 1 -- start start 2023-09-10T00:52:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.427+0000 7fb0ef3bab80 1 Processor -- start 2023-09-10T00:52:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.427+0000 7fb0ef3bab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 0x55f1d1e12dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.427+0000 7fb0ef3bab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e100b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.427+0000 7fb0ef3bab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e10690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.427+0000 7fb0ef3bab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1cf6b10 con 0x55f1d1e06cc0 2023-09-10T00:52:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.427+0000 7fb0ef3bab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1cf6810 con 0x55f1d1e0c1a0 2023-09-10T00:52:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.427+0000 7fb0ef3bab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1bda8c0 con 0x55f1d1e06300 2023-09-10T00:52:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0ef3bab80 10 client.0 _ll_register_callbacks cb 0x55f1d1c40a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:52:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0dbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0e0f90700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 0x55f1d1e12dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0e1791700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e10690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0e1791700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e10690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54462/0 (socket says 172.21.15.44:54462) 2023-09-10T00:52:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0e1791700 1 -- 172.21.15.44:0/617932243 learned_addr learned my addr 172.21.15.44:0/617932243 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0e0f90700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 0x55f1d1e12dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59600/0 (socket says 172.21.15.44:59600) 2023-09-10T00:52:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0e1791700 1 -- 172.21.15.44:0/617932243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 msgr2=0x55f1d1e12dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0e1791700 1 --2- 172.21.15.44:0/617932243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 0x55f1d1e12dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0e1791700 1 -- 172.21.15.44:0/617932243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e06cc0 msgr2=0x55f1d1e100b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0e1791700 1 --2- 172.21.15.44:0/617932243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e100b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0e1791700 1 -- 172.21.15.44:0/617932243 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc10dd50 con 0x55f1d1e0c1a0 2023-09-10T00:52:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0e1791700 1 --2- 172.21.15.44:0/617932243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e10690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4007590 tx=0x7fb0d40060c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4d6d6ee9619e94f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:52:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0bbfff700 1 -- 172.21.15.44:0/617932243 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d400e040 con 0x55f1d1e0c1a0 2023-09-10T00:52:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0bbfff700 1 -- 172.21.15.44:0/617932243 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb0d4003a00 con 0x55f1d1e0c1a0 2023-09-10T00:52:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.428+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/617932243 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1d1cb0490 con 0x55f1d1e0c1a0 2023-09-10T00:52:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.429+0000 7fb0bbfff700 1 -- 172.21.15.44:0/617932243 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0d4007e80 con 0x55f1d1e0c1a0 2023-09-10T00:52:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.429+0000 7fb0ef3bab80 10 client.27128 fetch_fsmap learned FSMap version 61 2023-09-10T00:52:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.429+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/617932243 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55f1d1b6dc00 con 0x55f1d1e0c1a0 2023-09-10T00:52:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.429+0000 7fb0bbfff700 1 -- 172.21.15.44:0/617932243 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f1d1cb0490 con 0x55f1d1e0c1a0 2023-09-10T00:52:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.429+0000 7fb0bbfff700 1 -- 172.21.15.44:0/617932243 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fb0d400b100 con 0x55f1d1e0c1a0 2023-09-10T00:52:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.429+0000 7fb0ef3bab80 10 client.27128 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:52:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.429+0000 7fb0ef3bab80 10 client.27128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:52:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.429+0000 7fb0ef3bab80 20 client.27128 populate_metadata read hostname 'smithi044' 2023-09-10T00:52:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.429+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/617932243 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f1d1e1ffa0 0x55f1d1e223c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.429+0000 7fb0ef3bab80 4 client.27128 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:52:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.429+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/617932243 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1d1d6d840 con 0x55f1d1e1ffa0 2023-09-10T00:52:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.429+0000 7fb0e0f90700 1 --2- 172.21.15.44:0/617932243 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f1d1e1ffa0 0x55f1d1e223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.430+0000 7fb0bbfff700 10 client.27128 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.430+0000 7fb0e0f90700 1 --2- 172.21.15.44:0/617932243 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f1d1e1ffa0 0x55f1d1e223c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee855de6319d9d48 server_cookie=89edbbd285d75b5f in_seq=0 out_seq=0 2023-09-10T00:52:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.430+0000 7fb0bbfff700 1 -- 172.21.15.44:0/617932243 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f1d1d6d840 con 0x55f1d1e1ffa0 2023-09-10T00:52:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.430+0000 7fb0bbfff700 10 client.27128 handle_command_reply: tid=0 2023-09-10T00:52:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.430+0000 7fb0ef3bab80 1 client.27128 shutdown 2023-09-10T00:52:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.430+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/617932243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e0c1a0 msgr2=0x55f1d1e10690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.430+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/617932243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e10690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4007590 tx=0x7fb0d40060c0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.430+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/617932243 shutdown_connections 2023-09-10T00:52:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.430+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/617932243 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f1d1e1ffa0 0x55f1d1e223c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.430+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/617932243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e10690 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.430+0000 7fb0bbfff700 0 client.27128 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.430+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/617932243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 0x55f1d1e12dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.430+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/617932243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.430+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/617932243 >> 172.21.15.44:0/617932243 conn(0x55f1d1d7b030 msgr2=0x55f1d1e007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.431+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/617932243 shutdown_connections 2023-09-10T00:52:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.431+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/617932243 wait complete. 2023-09-10T00:52:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.431+0000 7fb0ef3bab80 20 client.27128 trim_cache size 0 max 16384 2023-09-10T00:52:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.450+0000 7fb0ef3bab80 1 Processor -- start 2023-09-10T00:52:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.450+0000 7fb0ef3bab80 1 -- start start 2023-09-10T00:52:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.450+0000 7fb0ef3bab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 0x55f1d1e173f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.450+0000 7fb0ef3bab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e0a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.450+0000 7fb0ef3bab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e0a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.450+0000 7fb0ef3bab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1d6d840 con 0x55f1d1e0c1a0 2023-09-10T00:52:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.450+0000 7fb0ef3bab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1cb0490 con 0x55f1d1e06cc0 2023-09-10T00:52:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.450+0000 7fb0ef3bab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1e021c0 con 0x55f1d1e06300 2023-09-10T00:52:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.450+0000 7fb0dbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e0a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.451+0000 7fb0dbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e0a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54468/0 (socket says 172.21.15.44:54468) 2023-09-10T00:52:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.451+0000 7fb0e1791700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e0a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.451+0000 7fb0dbfff700 1 -- 172.21.15.44:0/547728636 learned_addr learned my addr 172.21.15.44:0/547728636 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.451+0000 7fb0e0f90700 1 --2- 172.21.15.44:0/547728636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 0x55f1d1e173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.451+0000 7fb0dbfff700 1 -- 172.21.15.44:0/547728636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 msgr2=0x55f1d1e173f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.451+0000 7fb0dbfff700 1 --2- 172.21.15.44:0/547728636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 0x55f1d1e173f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.451+0000 7fb0dbfff700 1 -- 172.21.15.44:0/547728636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 msgr2=0x55f1d1e0a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.451+0000 7fb0dbfff700 1 --2- 172.21.15.44:0/547728636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e0a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.451+0000 7fb0dbfff700 1 -- 172.21.15.44:0/547728636 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1d1bab1f0 con 0x55f1d1e06cc0 2023-09-10T00:52:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.451+0000 7fb0e0f90700 1 --2- 172.21.15.44:0/547728636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 0x55f1d1e173f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.451+0000 7fb0dbfff700 1 --2- 172.21.15.44:0/547728636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e0a1f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55f1d1e01060 tx=0x7fb0c8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=988029bd2a6a7140 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.451+0000 7fb0c77fe700 1 -- 172.21.15.44:0/547728636 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c802a030 con 0x55f1d1e06cc0 2023-09-10T00:52:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.451+0000 7fb0c77fe700 1 -- 172.21.15.44:0/547728636 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0c802b040 con 0x55f1d1e06cc0 2023-09-10T00:52:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.451+0000 7fb0c77fe700 1 -- 172.21.15.44:0/547728636 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c80038e0 con 0x55f1d1e06cc0 2023-09-10T00:52:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.452+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/547728636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06cc0 msgr2=0x55f1d1e0a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.452+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/547728636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e0a1f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55f1d1e01060 tx=0x7fb0c8014000 comp rx=0 tx=0).stop 2023-09-10T00:52:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.452+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/547728636 shutdown_connections 2023-09-10T00:52:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.452+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/547728636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e0a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.452+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/547728636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06300 0x55f1d1e173f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.452+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/547728636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e0a1f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.452+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/547728636 >> 172.21.15.44:0/547728636 conn(0x55f1d1d7b030 msgr2=0x55f1d1e1c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.452+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/547728636 shutdown_connections 2023-09-10T00:52:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.452+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/547728636 wait complete. 2023-09-10T00:52:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.452+0000 7fb0ef3bab80 1 -- start start 2023-09-10T00:52:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.452+0000 7fb0ef3bab80 1 Processor -- start 2023-09-10T00:52:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.453+0000 7fb0ef3bab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06300 0x55f1d1e67f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.453+0000 7fb0ef3bab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e68490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.453+0000 7fb0ef3bab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e72de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.453+0000 7fb0ef3bab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1e73320 con 0x55f1d1e0c1a0 2023-09-10T00:52:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.453+0000 7fb0ef3bab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1e73460 con 0x55f1d1e06300 2023-09-10T00:52:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.453+0000 7fb0ef3bab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1e735a0 con 0x55f1d1e06cc0 2023-09-10T00:52:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.453+0000 7fb0ef3bab80 10 client.0 _ll_register_callbacks cb 0x7fb0d0099320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:52:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.453+0000 7fb0e1791700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e72de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.453+0000 7fb0dbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e68490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.453+0000 7fb0e1791700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e72de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52736/0 (socket says 172.21.15.44:52736) 2023-09-10T00:52:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.453+0000 7fb0e1791700 1 -- 172.21.15.44:0/3703594880 learned_addr learned my addr 172.21.15.44:0/3703594880 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.453+0000 7fb0e0f90700 1 --2- 172.21.15.44:0/3703594880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06300 0x55f1d1e67f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0e0f90700 1 -- 172.21.15.44:0/3703594880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06cc0 msgr2=0x55f1d1e68490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0e0f90700 1 --2- 172.21.15.44:0/3703594880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e68490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0e0f90700 1 -- 172.21.15.44:0/3703594880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 msgr2=0x55f1d1e72de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0e0f90700 1 --2- 172.21.15.44:0/3703594880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e72de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0e0f90700 1 -- 172.21.15.44:0/3703594880 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc1a37c0 con 0x55f1d1e06300 2023-09-10T00:52:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0dbfff700 1 --2- 172.21.15.44:0/3703594880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e68490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0e1791700 1 --2- 172.21.15.44:0/3703594880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e72de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0e0f90700 1 --2- 172.21.15.44:0/3703594880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06300 0x55f1d1e67f50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00a1090 tx=0x7fb0d00cb040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5029c10c09dc4a4b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:52:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0bbfff700 1 -- 172.21.15.44:0/3703594880 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0099550 con 0x55f1d1e06300 2023-09-10T00:52:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/3703594880 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0d4007b90 con 0x55f1d1e06300 2023-09-10T00:52:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0bbfff700 1 -- 172.21.15.44:0/3703594880 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb0d009d380 con 0x55f1d1e06300 2023-09-10T00:52:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0bbfff700 1 -- 172.21.15.44:0/3703594880 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0d00a15c0 con 0x55f1d1e06300 2023-09-10T00:52:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0ef3bab80 10 client.27134 fetch_fsmap learned FSMap version 61 2023-09-10T00:52:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/3703594880 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7fb0c8017070 con 0x55f1d1e06300 2023-09-10T00:52:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0bbfff700 1 -- 172.21.15.44:0/3703594880 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0d4007b90 con 0x55f1d1e06300 2023-09-10T00:52:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.454+0000 7fb0bbfff700 1 -- 172.21.15.44:0/3703594880 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fb0d00997b0 con 0x55f1d1e06300 2023-09-10T00:52:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.455+0000 7fb0ef3bab80 10 client.27134 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:52:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.455+0000 7fb0ef3bab80 10 client.27134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:52:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.455+0000 7fb0ef3bab80 20 client.27134 populate_metadata read hostname 'smithi044' 2023-09-10T00:52:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.455+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/3703594880 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f1d1e74dc0 0x55f1d1e771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.455+0000 7fb0ef3bab80 4 client.27134 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:52:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.455+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/3703594880 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f1d1d518c0 con 0x55f1d1e74dc0 2023-09-10T00:52:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.455+0000 7fb0dbfff700 1 --2- 172.21.15.44:0/3703594880 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f1d1e74dc0 0x55f1d1e771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.455+0000 7fb0bbfff700 10 client.27134 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.455+0000 7fb0dbfff700 1 --2- 172.21.15.44:0/3703594880 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f1d1e74dc0 0x55f1d1e771e0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf992c8f441721b4 server_cookie=d2ea396453166d63 in_seq=0 out_seq=0 2023-09-10T00:52:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.470+0000 7fb0bbfff700 1 -- 172.21.15.44:0/3703594880 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f1d1d518c0 con 0x55f1d1e74dc0 2023-09-10T00:52:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.470+0000 7fb0bbfff700 10 client.27134 handle_command_reply: tid=0 2023-09-10T00:52:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.470+0000 7fb0ef3bab80 1 client.27134 shutdown 2023-09-10T00:52:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.470+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/3703594880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06300 msgr2=0x55f1d1e67f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.470+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/3703594880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06300 0x55f1d1e67f50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00a1090 tx=0x7fb0d00cb040 comp rx=0 tx=0).stop 2023-09-10T00:52:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.470+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/3703594880 shutdown_connections 2023-09-10T00:52:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.470+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/3703594880 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f1d1e74dc0 0x55f1d1e771e0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.470+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/3703594880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f1d1e0c1a0 0x55f1d1e72de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.470+0000 7fb0bbfff700 0 client.27134 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.470+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/3703594880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d1e06300 0x55f1d1e67f50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.470+0000 7fb0ef3bab80 1 --2- 172.21.15.44:0/3703594880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1d1e06cc0 0x55f1d1e68490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.470+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/3703594880 >> 172.21.15.44:0/3703594880 conn(0x55f1d1d7b030 msgr2=0x55f1d1e0a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.470+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/3703594880 shutdown_connections 2023-09-10T00:52:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.470+0000 7fb0ef3bab80 1 -- 172.21.15.44:0/3703594880 wait complete. 2023-09-10T00:52:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.470+0000 7fb0ef3bab80 20 client.27134 trim_cache size 0 max 16384 2023-09-10T00:52:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:52:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:52:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:52:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:52:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.471+0000 7fb0e31f4700 1 -- 172.21.15.44:0/2253348626 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb0cc0a07b0 msgr2=0x7fb0cc0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.471+0000 7fb0e31f4700 1 --2- 172.21.15.44:0/2253348626 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb0cc0a07b0 0x7fb0cc0a2c70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc197f70 tx=0x7fb0c8009260 comp rx=0 tx=0).stop 2023-09-10T00:52:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.471+0000 7fb0e31f4700 1 -- 172.21.15.44:0/2253348626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc1005d0 msgr2=0x7fb0dc196750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.471+0000 7fb0e31f4700 1 --2- 172.21.15.44:0/2253348626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc1005d0 0x7fb0dc196750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000e9a0 tx=0x7fb0d000ecb0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.471+0000 7fb0e31f4700 1 -- 172.21.15.44:0/2253348626 shutdown_connections 2023-09-10T00:52:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.471+0000 7fb0e31f4700 1 --2- 172.21.15.44:0/2253348626 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb0cc0a07b0 0x7fb0cc0a2c70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.471+0000 7fb0e31f4700 1 --2- 172.21.15.44:0/2253348626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0dc106690 0x7fb0dc19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.471+0000 7fb0e31f4700 1 --2- 172.21.15.44:0/2253348626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc1005d0 0x7fb0dc196750 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.471+0000 7fb0e31f4700 1 --2- 172.21.15.44:0/2253348626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0dc100f90 0x7fb0dc196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.471+0000 7fb0e31f4700 1 -- 172.21.15.44:0/2253348626 >> 172.21.15.44:0/2253348626 conn(0x7fb0dc0fc0c0 msgr2=0x7fb0dc0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.471+0000 7fb0e31f4700 1 -- 172.21.15.44:0/2253348626 shutdown_connections 2023-09-10T00:52:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:07.471+0000 7fb0e31f4700 1 -- 172.21.15.44:0/2253348626 wait complete. 2023-09-10T00:52:07.769 INFO:teuthology.orchestra.run.smithi044.stderr:43400000 of 50000000 tuples (86%) done (elapsed 667.40 s, remaining 101.49 s) 2023-09-10T00:52:08.924 INFO:teuthology.orchestra.run.smithi044.stderr:43500000 of 50000000 tuples (87%) done (elapsed 668.56 s, remaining 99.90 s) 2023-09-10T00:52:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:08 smithi099 ceph-mon[102692]: pgmap v357: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 31 MiB/s wr, 97 op/s 2023-09-10T00:52:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:08 smithi131 ceph-mon[103713]: pgmap v357: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 31 MiB/s wr, 97 op/s 2023-09-10T00:52:09.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:08 smithi044 ceph-mon[98676]: pgmap v357: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 31 MiB/s wr, 97 op/s 2023-09-10T00:52:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:43600000 of 50000000 tuples (87%) done (elapsed 669.57 s, remaining 98.29 s) 2023-09-10T00:52:10.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:09 smithi131 ceph-mon[103713]: osdmap e103: 12 total, 12 up, 12 in 2023-09-10T00:52:10.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:09 smithi044 ceph-mon[98676]: osdmap e103: 12 total, 12 up, 12 in 2023-09-10T00:52:10.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:09 smithi099 ceph-mon[102692]: osdmap e103: 12 total, 12 up, 12 in 2023-09-10T00:52:10.817 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:52:10.990 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.990+0000 7ff1e5e61700 1 -- 172.21.15.44:0/2601843135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1e0105560 msgr2=0x7ff1e0105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:10.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.990+0000 7ff1e5e61700 1 --2- 172.21.15.44:0/2601843135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1e0105560 0x7ff1e0105940 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009990 tx=0x7ff1d0009ca0 comp rx=0 tx=0).stop 2023-09-10T00:52:10.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.990+0000 7ff1e5e61700 1 -- 172.21.15.44:0/2601843135 shutdown_connections 2023-09-10T00:52:10.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.990+0000 7ff1e5e61700 1 --2- 172.21.15.44:0/2601843135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1e00ffe60 0x7ff1e010c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:10.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.990+0000 7ff1e5e61700 1 --2- 172.21.15.44:0/2601843135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1e0105560 0x7ff1e0105940 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:10.991 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.990+0000 7ff1e5e61700 1 --2- 172.21.15.44:0/2601843135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1e00ff4a0 0x7ff1e00ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:10.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.990+0000 7ff1e5e61700 1 -- 172.21.15.44:0/2601843135 >> 172.21.15.44:0/2601843135 conn(0x7ff1e00faf50 msgr2=0x7ff1e00fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:10.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.990+0000 7ff1e5e61700 1 -- 172.21.15.44:0/2601843135 shutdown_connections 2023-09-10T00:52:10.992 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.990+0000 7ff1e5e61700 1 -- 172.21.15.44:0/2601843135 wait complete. 2023-09-10T00:52:10.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.990+0000 7ff1e5e61700 1 Processor -- start 2023-09-10T00:52:10.994 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.991+0000 7ff1e5e61700 1 -- start start 2023-09-10T00:52:10.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.991+0000 7ff1e5e61700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1e00ff4a0 0x7ff1e01953c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:10.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.991+0000 7ff1e5e61700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1e00ffe60 0x7ff1e0195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:10.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.991+0000 7ff1e5e61700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1e0105560 0x7ff1e0199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:10.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.991+0000 7ff1e5e61700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e019a3a0 con 0x7ff1e00ffe60 2023-09-10T00:52:10.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.991+0000 7ff1e5e61700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e019a510 con 0x7ff1e00ff4a0 2023-09-10T00:52:10.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.991+0000 7ff1e5e61700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e019a6d0 con 0x7ff1e0105560 2023-09-10T00:52:10.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.991+0000 7ff1df7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1e00ff4a0 0x7ff1e01953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:10.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.991+0000 7ff1df7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1e00ff4a0 0x7ff1e01953c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54486/0 (socket says 172.21.15.44:54486) 2023-09-10T00:52:10.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.991+0000 7ff1df7fe700 1 -- 172.21.15.44:0/1742467566 learned_addr learned my addr 172.21.15.44:0/1742467566 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:10.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.991+0000 7ff1dffff700 1 --2- 172.21.15.44:0/1742467566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1e0105560 0x7ff1e0199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:10.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.991+0000 7ff1deffd700 1 --2- 172.21.15.44:0/1742467566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1e00ffe60 0x7ff1e0195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:10.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.991+0000 7ff1df7fe700 1 -- 172.21.15.44:0/1742467566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1e0105560 msgr2=0x7ff1e0199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:10.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.991+0000 7ff1df7fe700 1 --2- 172.21.15.44:0/1742467566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1e0105560 0x7ff1e0199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:10.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.991+0000 7ff1df7fe700 1 -- 172.21.15.44:0/1742467566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1e00ffe60 msgr2=0x7ff1e0195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:10.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.991+0000 7ff1df7fe700 1 --2- 172.21.15.44:0/1742467566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1e00ffe60 0x7ff1e0195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:10.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.992+0000 7ff1df7fe700 1 -- 172.21.15.44:0/1742467566 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d4009520 con 0x7ff1e00ff4a0 2023-09-10T00:52:10.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.992+0000 7ff1dffff700 1 --2- 172.21.15.44:0/1742467566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1e0105560 0x7ff1e0199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:10.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.992+0000 7ff1deffd700 1 --2- 172.21.15.44:0/1742467566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1e00ffe60 0x7ff1e0195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:10.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.992+0000 7ff1df7fe700 1 --2- 172.21.15.44:0/1742467566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1e00ff4a0 0x7ff1e01953c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0006010 tx=0x7ff1d0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:11.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.992+0000 7ff1dcff9700 1 -- 172.21.15.44:0/1742467566 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d001f070 con 0x7ff1e00ff4a0 2023-09-10T00:52:11.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.992+0000 7ff1e5e61700 1 -- 172.21.15.44:0/1742467566 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d00095f0 con 0x7ff1e00ff4a0 2023-09-10T00:52:11.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.992+0000 7ff1dcff9700 1 -- 172.21.15.44:0/1742467566 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1d000f460 con 0x7ff1e00ff4a0 2023-09-10T00:52:11.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.992+0000 7ff1dcff9700 1 -- 172.21.15.44:0/1742467566 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0025680 con 0x7ff1e00ff4a0 2023-09-10T00:52:11.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.992+0000 7ff1e5e61700 1 -- 172.21.15.44:0/1742467566 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e0068710 con 0x7ff1e00ff4a0 2023-09-10T00:52:11.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.994+0000 7ff1dcff9700 1 -- 172.21.15.44:0/1742467566 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ff1d0023070 con 0x7ff1e00ff4a0 2023-09-10T00:52:11.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.995+0000 7ff1dcff9700 1 --2- 172.21.15.44:0/1742467566 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1c80a07b0 0x7ff1c80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:11.006 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.995+0000 7ff1deffd700 1 --2- 172.21.15.44:0/1742467566 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1c80a07b0 0x7ff1c80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:11.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.995+0000 7ff1dcff9700 1 -- 172.21.15.44:0/1742467566 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9628+0+0 (secure 0 0 0) 0x7ff1d00cf660 con 0x7ff1e00ff4a0 2023-09-10T00:52:11.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.996+0000 7ff1deffd700 1 --2- 172.21.15.44:0/1742467566 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1c80a07b0 0x7ff1c80a2c70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009fd0 tx=0x7ff1d4009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:11.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.996+0000 7ff1f2027b80 1 Processor -- start 2023-09-10T00:52:11.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.996+0000 7ff1f2027b80 1 -- start start 2023-09-10T00:52:11.007 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1f2027b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b083af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:11.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1f2027b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b084030 0x55bc5b08d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:11.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1f2027b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b08de40 0x55bc5b090230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:11.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1f2027b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5afdc730 con 0x55bc5b08de40 2023-09-10T00:52:11.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1f2027b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5afda050 con 0x55bc5b084030 2023-09-10T00:52:11.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1f2027b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5afd5940 con 0x55bc5b083710 2023-09-10T00:52:11.008 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1df7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b083af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:11.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1df7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b083af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59636/0 (socket says 172.21.15.44:59636) 2023-09-10T00:52:11.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1deffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b084030 0x55bc5b08d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:11.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1df7fe700 1 -- 172.21.15.44:0/1534301706 learned_addr learned my addr 172.21.15.44:0/1534301706 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:11.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1dffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b08de40 0x55bc5b090230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:11.009 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1df7fe700 1 -- 172.21.15.44:0/1534301706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b084030 msgr2=0x55bc5b08d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:11.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1df7fe700 1 --2- 172.21.15.44:0/1534301706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b084030 0x55bc5b08d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1df7fe700 1 -- 172.21.15.44:0/1534301706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b08de40 msgr2=0x55bc5b090230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:11.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1df7fe700 1 --2- 172.21.15.44:0/1534301706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b08de40 0x55bc5b090230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1df7fe700 1 -- 172.21.15.44:0/1534301706 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc5ae01de0 con 0x55bc5b083710 2023-09-10T00:52:11.010 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1df7fe700 1 --2- 172.21.15.44:0/1534301706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b083af0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00991c0 tx=0x7ff1d009c5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a22f96013b8e445 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:11.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.997+0000 7ff1c5ffb700 1 -- 172.21.15.44:0/1534301706 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d009c750 con 0x55bc5b083710 2023-09-10T00:52:11.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.998+0000 7ff1c5ffb700 1 -- 172.21.15.44:0/1534301706 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1d00a0580 con 0x55bc5b083710 2023-09-10T00:52:11.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.998+0000 7ff1c5ffb700 1 -- 172.21.15.44:0/1534301706 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00a0120 con 0x55bc5b083710 2023-09-10T00:52:11.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.998+0000 7ff1f2027b80 1 -- 172.21.15.44:0/1534301706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 msgr2=0x55bc5b083af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:11.011 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.998+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/1534301706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b083af0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00991c0 tx=0x7ff1d009c5f0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.998+0000 7ff1f2027b80 1 -- 172.21.15.44:0/1534301706 shutdown_connections 2023-09-10T00:52:11.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.998+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/1534301706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b08de40 0x55bc5b090230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.998+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/1534301706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b083af0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.998+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/1534301706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b084030 0x55bc5b08d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.998+0000 7ff1f2027b80 1 -- 172.21.15.44:0/1534301706 >> 172.21.15.44:0/1534301706 conn(0x55bc5aff8090 msgr2=0x55bc5b07ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:11.012 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.998+0000 7ff1f2027b80 1 -- 172.21.15.44:0/1534301706 shutdown_connections 2023-09-10T00:52:11.013 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.998+0000 7ff1f2027b80 1 -- 172.21.15.44:0/1534301706 wait complete. 2023-09-10T00:52:11.013 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.999+0000 7ff1f2027b80 1 -- start start 2023-09-10T00:52:11.013 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.999+0000 7ff1f2027b80 1 Processor -- start 2023-09-10T00:52:11.013 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.999+0000 7ff1f2027b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b08e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:11.013 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.999+0000 7ff1f2027b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b084030 0x55bc5b08e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:11.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.999+0000 7ff1f2027b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b08de40 0x55bc5b0992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:11.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.999+0000 7ff1f2027b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5af73b70 con 0x55bc5b084030 2023-09-10T00:52:11.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.999+0000 7ff1f2027b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5af73870 con 0x55bc5b08de40 2023-09-10T00:52:11.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.999+0000 7ff1f2027b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5ae57920 con 0x55bc5b083710 2023-09-10T00:52:11.014 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.999+0000 7ff1f2027b80 10 client.0 _ll_register_callbacks cb 0x55bc5aebda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:52:11.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.999+0000 7ff1deffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b084030 0x55bc5b08e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:11.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.999+0000 7ff1df7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b08e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:11.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.999+0000 7ff1df7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b08e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59642/0 (socket says 172.21.15.44:59642) 2023-09-10T00:52:11.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.999+0000 7ff1deffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b084030 0x55bc5b08e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52762/0 (socket says 172.21.15.44:52762) 2023-09-10T00:52:11.015 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.999+0000 7ff1df7fe700 1 -- 172.21.15.44:0/704470499 learned_addr learned my addr 172.21.15.44:0/704470499 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:11.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:10.999+0000 7ff1dffff700 1 --2- 172.21.15.44:0/704470499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b08de40 0x55bc5b0992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:11.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.000+0000 7ff1df7fe700 1 -- 172.21.15.44:0/704470499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b08de40 msgr2=0x55bc5b0992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:11.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.000+0000 7ff1df7fe700 1 --2- 172.21.15.44:0/704470499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b08de40 0x55bc5b0992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.000+0000 7ff1df7fe700 1 -- 172.21.15.44:0/704470499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b084030 msgr2=0x55bc5b08e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:11.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.000+0000 7ff1df7fe700 1 --2- 172.21.15.44:0/704470499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b084030 0x55bc5b08e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.016 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.000+0000 7ff1df7fe700 1 -- 172.21.15.44:0/704470499 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e0068710 con 0x55bc5b083710 2023-09-10T00:52:11.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.000+0000 7ff1deffd700 1 --2- 172.21.15.44:0/704470499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b084030 0x55bc5b08e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:11.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.000+0000 7ff1df7fe700 1 --2- 172.21.15.44:0/704470499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b08e220 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0098c50 tx=0x7ff1d000fba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fd7bca2f9e64f35 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:11.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.000+0000 7ff1beffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:52:11.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.000+0000 7ff1beffd700 1 -- 172.21.15.44:0/704470499 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00ba070 con 0x55bc5b083710 2023-09-10T00:52:11.017 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.000+0000 7ff1beffd700 1 -- 172.21.15.44:0/704470499 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9628+0+0 (secure 0 0 0) 0x7ff1d00a4730 con 0x55bc5b083710 2023-09-10T00:52:11.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.000+0000 7ff1f2027b80 1 -- 172.21.15.44:0/704470499 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc5af2d4f0 con 0x55bc5b083710 2023-09-10T00:52:11.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.000+0000 7ff1beffd700 1 -- 172.21.15.44:0/704470499 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1d009c750 con 0x55bc5b083710 2023-09-10T00:52:11.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.000+0000 7ff1f2027b80 10 client.27082 fetch_fsmap learned FSMap version 61 2023-09-10T00:52:11.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.000+0000 7ff1f2027b80 1 -- 172.21.15.44:0/704470499 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55bc5adeac60 con 0x55bc5b083710 2023-09-10T00:52:11.018 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.000+0000 7ff1beffd700 1 -- 172.21.15.44:0/704470499 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bc5af2d4f0 con 0x55bc5b083710 2023-09-10T00:52:11.019 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.001+0000 7ff1beffd700 1 -- 172.21.15.44:0/704470499 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff1d009bcd0 con 0x55bc5b083710 2023-09-10T00:52:11.019 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.001+0000 7ff1f2027b80 10 client.27082 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:52:11.019 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.001+0000 7ff1f2027b80 10 client.27082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:52:11.019 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.001+0000 7ff1f2027b80 20 client.27082 populate_metadata read hostname 'smithi044' 2023-09-10T00:52:11.019 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.001+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/704470499 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc5b09d110 0x55bc5b09f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:11.019 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.001+0000 7ff1f2027b80 4 client.27082 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:52:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.001+0000 7ff1f2027b80 1 -- 172.21.15.44:0/704470499 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc5afea8a0 con 0x55bc5b09d110 2023-09-10T00:52:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.001+0000 7ff1deffd700 1 --2- 172.21.15.44:0/704470499 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc5b09d110 0x55bc5b09f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.001+0000 7ff1beffd700 10 client.27082 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.001+0000 7ff1deffd700 1 --2- 172.21.15.44:0/704470499 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc5b09d110 0x55bc5b09f530 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2296bf3c236fa21f server_cookie=60e65f29654b287d in_seq=0 out_seq=0 2023-09-10T00:52:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.002+0000 7ff1beffd700 1 -- 172.21.15.44:0/704470499 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bc5afea8a0 con 0x55bc5b09d110 2023-09-10T00:52:11.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.002+0000 7ff1beffd700 10 client.27082 handle_command_reply: tid=0 2023-09-10T00:52:11.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.002+0000 7ff1f2027b80 1 client.27082 shutdown 2023-09-10T00:52:11.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.002+0000 7ff1f2027b80 1 -- 172.21.15.44:0/704470499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 msgr2=0x55bc5b08e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:11.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.002+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/704470499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b08e220 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0098c50 tx=0x7ff1d000fba0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.021 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.002+0000 7ff1f2027b80 1 -- 172.21.15.44:0/704470499 shutdown_connections 2023-09-10T00:52:11.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.002+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/704470499 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc5b09d110 0x55bc5b09f530 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.002+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/704470499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b08de40 0x55bc5b0992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.002+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/704470499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b08e220 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.002+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/704470499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b084030 0x55bc5b08e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.002+0000 7ff1beffd700 0 client.27082 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:11.022 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.002+0000 7ff1f2027b80 1 -- 172.21.15.44:0/704470499 >> 172.21.15.44:0/704470499 conn(0x55bc5aff8090 msgr2=0x55bc5b07e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:11.023 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.002+0000 7ff1f2027b80 1 -- 172.21.15.44:0/704470499 shutdown_connections 2023-09-10T00:52:11.023 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.002+0000 7ff1f2027b80 1 -- 172.21.15.44:0/704470499 wait complete. 2023-09-10T00:52:11.023 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.003+0000 7ff1f2027b80 20 client.27082 trim_cache size 0 max 16384 2023-09-10T00:52:11.028 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.022+0000 7ff1f2027b80 1 Processor -- start 2023-09-10T00:52:11.028 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.022+0000 7ff1f2027b80 1 -- start start 2023-09-10T00:52:11.029 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.022+0000 7ff1f2027b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b093230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:11.029 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.022+0000 7ff1f2027b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b084030 0x55bc5b08e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:11.029 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.022+0000 7ff1f2027b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b08de40 0x55bc5b08e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:11.029 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.022+0000 7ff1f2027b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b093770 con 0x55bc5b08de40 2023-09-10T00:52:11.029 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.022+0000 7ff1f2027b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b080190 con 0x55bc5b084030 2023-09-10T00:52:11.030 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.022+0000 7ff1f2027b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b07f6e0 con 0x55bc5b083710 2023-09-10T00:52:11.030 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.022+0000 7ff1df7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b093230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:11.030 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.022+0000 7ff1dffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b08de40 0x55bc5b08e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:11.030 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.022+0000 7ff1deffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b084030 0x55bc5b08e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:11.030 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.022+0000 7ff1df7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b093230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59650/0 (socket says 172.21.15.44:59650) 2023-09-10T00:52:11.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.022+0000 7ff1deffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b084030 0x55bc5b08e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54510/0 (socket says 172.21.15.44:54510) 2023-09-10T00:52:11.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.022+0000 7ff1df7fe700 1 -- 172.21.15.44:0/2280911724 learned_addr learned my addr 172.21.15.44:0/2280911724 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:11.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.022+0000 7ff1dffff700 1 -- 172.21.15.44:0/2280911724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 msgr2=0x55bc5b093230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:11.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.023+0000 7ff1dffff700 1 --2- 172.21.15.44:0/2280911724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b093230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.023+0000 7ff1dffff700 1 -- 172.21.15.44:0/2280911724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b084030 msgr2=0x55bc5b08e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:11.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.023+0000 7ff1dffff700 1 --2- 172.21.15.44:0/2280911724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b084030 0x55bc5b08e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.023+0000 7ff1dffff700 1 -- 172.21.15.44:0/2280911724 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc5ae28250 con 0x55bc5b08de40 2023-09-10T00:52:11.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.023+0000 7ff1df7fe700 1 --2- 172.21.15.44:0/2280911724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b093230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:11.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.023+0000 7ff1deffd700 1 --2- 172.21.15.44:0/2280911724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b084030 0x55bc5b08e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:11.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.023+0000 7ff1dffff700 1 --2- 172.21.15.44:0/2280911724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b08de40 0x55bc5b08e760 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800ea30 tx=0x7ff1d8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d8f9559852a187 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:11.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.023+0000 7ff1c67fc700 1 -- 172.21.15.44:0/2280911724 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8004db0 con 0x55bc5b08de40 2023-09-10T00:52:11.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.023+0000 7ff1c67fc700 1 -- 172.21.15.44:0/2280911724 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1d800f040 con 0x55bc5b08de40 2023-09-10T00:52:11.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.023+0000 7ff1c67fc700 1 -- 172.21.15.44:0/2280911724 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8007880 con 0x55bc5b08de40 2023-09-10T00:52:11.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.024+0000 7ff1f2027b80 1 -- 172.21.15.44:0/2280911724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b08de40 msgr2=0x55bc5b08e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:11.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.024+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/2280911724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b08de40 0x55bc5b08e760 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800ea30 tx=0x7ff1d8009920 comp rx=0 tx=0).stop 2023-09-10T00:52:11.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.024+0000 7ff1f2027b80 1 -- 172.21.15.44:0/2280911724 shutdown_connections 2023-09-10T00:52:11.034 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.024+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/2280911724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b08de40 0x55bc5b08e760 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.034 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.024+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/2280911724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b083710 0x55bc5b093230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.034 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.024+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/2280911724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b084030 0x55bc5b08e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.034 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.024+0000 7ff1f2027b80 1 -- 172.21.15.44:0/2280911724 >> 172.21.15.44:0/2280911724 conn(0x55bc5aff8090 msgr2=0x55bc5b0883b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:11.034 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.024+0000 7ff1f2027b80 1 -- 172.21.15.44:0/2280911724 shutdown_connections 2023-09-10T00:52:11.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.024+0000 7ff1f2027b80 1 -- 172.21.15.44:0/2280911724 wait complete. 2023-09-10T00:52:11.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.024+0000 7ff1f2027b80 1 -- start start 2023-09-10T00:52:11.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.024+0000 7ff1f2027b80 1 Processor -- start 2023-09-10T00:52:11.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1f2027b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b083710 0x55bc5b0ee900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:11.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1f2027b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b084030 0x55bc5b0eee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:11.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1f2027b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b08de40 0x55bc5b0e8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:11.036 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1f2027b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b0e91d0 con 0x55bc5b08de40 2023-09-10T00:52:11.036 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1f2027b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b0e9340 con 0x55bc5b083710 2023-09-10T00:52:11.036 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1f2027b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b0e9500 con 0x55bc5b084030 2023-09-10T00:52:11.036 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1f2027b80 10 client.0 _ll_register_callbacks cb 0x7ff1d8009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:52:11.036 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1deffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b084030 0x55bc5b0eee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:11.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1deffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b084030 0x55bc5b0eee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59658/0 (socket says 172.21.15.44:59658) 2023-09-10T00:52:11.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1df7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b083710 0x55bc5b0ee900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:11.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1deffd700 1 -- 172.21.15.44:0/3961318637 learned_addr learned my addr 172.21.15.44:0/3961318637 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:11.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1dffff700 1 --2- 172.21.15.44:0/3961318637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b08de40 0x55bc5b0e8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:11.037 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1deffd700 1 -- 172.21.15.44:0/3961318637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b083710 msgr2=0x55bc5b0ee900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:11.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1deffd700 1 --2- 172.21.15.44:0/3961318637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b083710 0x55bc5b0ee900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1deffd700 1 -- 172.21.15.44:0/3961318637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b08de40 msgr2=0x55bc5b0e8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:11.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1deffd700 1 --2- 172.21.15.44:0/3961318637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b08de40 0x55bc5b0e8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1deffd700 1 -- 172.21.15.44:0/3961318637 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc5adeac60 con 0x55bc5b084030 2023-09-10T00:52:11.038 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1df7fe700 1 --2- 172.21.15.44:0/3961318637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b083710 0x55bc5b0ee900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:11.039 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1deffd700 1 --2- 172.21.15.44:0/3961318637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b084030 0x55bc5b0eee40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4005d90 tx=0x7ff1d40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91da9f4888b6859c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:11.039 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.025+0000 7ff1beffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:52:11.039 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.026+0000 7ff1beffd700 1 -- 172.21.15.44:0/3961318637 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4023040 con 0x55bc5b084030 2023-09-10T00:52:11.039 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.026+0000 7ff1beffd700 1 -- 172.21.15.44:0/3961318637 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9628+0+0 (secure 0 0 0) 0x7ff1d4028be0 con 0x55bc5b084030 2023-09-10T00:52:11.039 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.026+0000 7ff1f2027b80 1 -- 172.21.15.44:0/3961318637 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1d00a0580 con 0x55bc5b084030 2023-09-10T00:52:11.039 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.026+0000 7ff1beffd700 1 -- 172.21.15.44:0/3961318637 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1d4028580 con 0x55bc5b084030 2023-09-10T00:52:11.040 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.026+0000 7ff1f2027b80 10 client.27088 fetch_fsmap learned FSMap version 61 2023-09-10T00:52:11.040 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.026+0000 7ff1f2027b80 1 -- 172.21.15.44:0/3961318637 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7ff1d4017050 con 0x55bc5b084030 2023-09-10T00:52:11.040 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.026+0000 7ff1beffd700 1 -- 172.21.15.44:0/3961318637 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1d00a0580 con 0x55bc5b084030 2023-09-10T00:52:11.040 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.026+0000 7ff1beffd700 1 -- 172.21.15.44:0/3961318637 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff1d401d3f0 con 0x55bc5b084030 2023-09-10T00:52:11.040 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.026+0000 7ff1f2027b80 10 client.27088 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:52:11.041 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.026+0000 7ff1f2027b80 10 client.27088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:52:11.041 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.026+0000 7ff1f2027b80 20 client.27088 populate_metadata read hostname 'smithi044' 2023-09-10T00:52:11.041 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.026+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/3961318637 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc5b0f68d0 0x55bc5b0f8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:11.041 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.026+0000 7ff1f2027b80 4 client.27088 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:52:11.041 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.026+0000 7ff1f2027b80 1 -- 172.21.15.44:0/3961318637 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bc5afce920 con 0x55bc5b0f68d0 2023-09-10T00:52:11.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.026+0000 7ff1df7fe700 1 --2- 172.21.15.44:0/3961318637 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc5b0f68d0 0x55bc5b0f8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:11.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.027+0000 7ff1beffd700 10 client.27088 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:11.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.027+0000 7ff1df7fe700 1 --2- 172.21.15.44:0/3961318637 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc5b0f68d0 0x55bc5b0f8cf0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc13d99d261ba034 server_cookie=1bf7f241f6a42b6b in_seq=0 out_seq=0 2023-09-10T00:52:11.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.027+0000 7ff1beffd700 1 -- 172.21.15.44:0/3961318637 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bc5afce920 con 0x55bc5b0f68d0 2023-09-10T00:52:11.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.027+0000 7ff1beffd700 10 client.27088 handle_command_reply: tid=0 2023-09-10T00:52:11.042 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.027+0000 7ff1f2027b80 1 client.27088 shutdown 2023-09-10T00:52:11.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.027+0000 7ff1f2027b80 1 -- 172.21.15.44:0/3961318637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b084030 msgr2=0x55bc5b0eee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:11.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.027+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/3961318637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b084030 0x55bc5b0eee40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4005d90 tx=0x7ff1d40047c0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.027+0000 7ff1f2027b80 1 -- 172.21.15.44:0/3961318637 shutdown_connections 2023-09-10T00:52:11.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.027+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/3961318637 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc5b0f68d0 0x55bc5b0f8cf0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.043 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.027+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/3961318637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc5b08de40 0x55bc5b0e8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.027+0000 7ff1beffd700 0 client.27088 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:11.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.027+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/3961318637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc5b083710 0x55bc5b0ee900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.027+0000 7ff1f2027b80 1 --2- 172.21.15.44:0/3961318637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5b084030 0x55bc5b0eee40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.027+0000 7ff1f2027b80 1 -- 172.21.15.44:0/3961318637 >> 172.21.15.44:0/3961318637 conn(0x55bc5aff8090 msgr2=0x55bc5b08e220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:11.044 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.027+0000 7ff1f2027b80 1 -- 172.21.15.44:0/3961318637 shutdown_connections 2023-09-10T00:52:11.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.027+0000 7ff1f2027b80 1 -- 172.21.15.44:0/3961318637 wait complete. 2023-09-10T00:52:11.045 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.028+0000 7ff1f2027b80 20 client.27088 trim_cache size 0 max 16384 2023-09-10T00:52:11.045 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:52:11.045 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:52:11.046 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:52:11.046 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:52:11.046 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:52:11.046 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:52:11.046 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:52:11.047 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:52:11.047 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:52:11.047 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:52:11.048 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.028+0000 7ff1e5e61700 1 -- 172.21.15.44:0/1742467566 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1c80a07b0 msgr2=0x7ff1c80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:11.049 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.028+0000 7ff1e5e61700 1 --2- 172.21.15.44:0/1742467566 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1c80a07b0 0x7ff1c80a2c70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009fd0 tx=0x7ff1d4009260 comp rx=0 tx=0).stop 2023-09-10T00:52:11.049 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.028+0000 7ff1e5e61700 1 -- 172.21.15.44:0/1742467566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1e00ff4a0 msgr2=0x7ff1e01953c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:11.049 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.028+0000 7ff1e5e61700 1 --2- 172.21.15.44:0/1742467566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1e00ff4a0 0x7ff1e01953c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0006010 tx=0x7ff1d0005740 comp rx=0 tx=0).stop 2023-09-10T00:52:11.049 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.028+0000 7ff1e5e61700 1 -- 172.21.15.44:0/1742467566 shutdown_connections 2023-09-10T00:52:11.049 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.028+0000 7ff1e5e61700 1 --2- 172.21.15.44:0/1742467566 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1c80a07b0 0x7ff1c80a2c70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.028+0000 7ff1e5e61700 1 --2- 172.21.15.44:0/1742467566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1e0105560 0x7ff1e0199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.028+0000 7ff1e5e61700 1 --2- 172.21.15.44:0/1742467566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1e00ff4a0 0x7ff1e01953c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.028+0000 7ff1e5e61700 1 --2- 172.21.15.44:0/1742467566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1e00ffe60 0x7ff1e0195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:11.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.028+0000 7ff1e5e61700 1 -- 172.21.15.44:0/1742467566 >> 172.21.15.44:0/1742467566 conn(0x7ff1e00faf50 msgr2=0x7ff1e00fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:11.050 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.028+0000 7ff1e5e61700 1 -- 172.21.15.44:0/1742467566 shutdown_connections 2023-09-10T00:52:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:11.028+0000 7ff1e5e61700 1 -- 172.21.15.44:0/1742467566 wait complete. 2023-09-10T00:52:11.051 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:52:11.060 INFO:teuthology.orchestra.run.smithi044.stderr:43700000 of 50000000 tuples (87%) done (elapsed 670.69 s, remaining 96.69 s) 2023-09-10T00:52:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:10 smithi131 ceph-mon[103713]: pgmap v359: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 37 MiB/s wr, 116 op/s 2023-09-10T00:52:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:10 smithi044 ceph-mon[98676]: pgmap v359: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 37 MiB/s wr, 116 op/s 2023-09-10T00:52:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:10 smithi099 ceph-mon[102692]: pgmap v359: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 37 MiB/s wr, 116 op/s 2023-09-10T00:52:12.145 INFO:teuthology.orchestra.run.smithi044.stderr:43800000 of 50000000 tuples (87%) done (elapsed 671.78 s, remaining 95.09 s) 2023-09-10T00:52:13.079 INFO:teuthology.orchestra.run.smithi044.stderr:43900000 of 50000000 tuples (87%) done (elapsed 672.71 s, remaining 93.47 s) 2023-09-10T00:52:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:12 smithi131 ceph-mon[103713]: pgmap v360: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 35 MiB/s wr, 146 op/s 2023-09-10T00:52:13.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:12 smithi044 ceph-mon[98676]: pgmap v360: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 35 MiB/s wr, 146 op/s 2023-09-10T00:52:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:12 smithi099 ceph-mon[102692]: pgmap v360: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 35 MiB/s wr, 146 op/s 2023-09-10T00:52:14.147 INFO:teuthology.orchestra.run.smithi044.stderr:44000000 of 50000000 tuples (88%) done (elapsed 673.78 s, remaining 91.88 s) 2023-09-10T00:52:15.170 INFO:teuthology.orchestra.run.smithi044.stderr:44100000 of 50000000 tuples (88%) done (elapsed 674.80 s, remaining 90.28 s) 2023-09-10T00:52:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:14 smithi131 ceph-mon[103713]: pgmap v361: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 35 MiB/s wr, 146 op/s 2023-09-10T00:52:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:14 smithi044 ceph-mon[98676]: pgmap v361: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 35 MiB/s wr, 146 op/s 2023-09-10T00:52:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:14 smithi099 ceph-mon[102692]: pgmap v361: 129 pgs: 129 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 35 MiB/s wr, 146 op/s 2023-09-10T00:52:16.186 INFO:teuthology.orchestra.run.smithi044.stderr:44200000 of 50000000 tuples (88%) done (elapsed 675.82 s, remaining 88.68 s) 2023-09-10T00:52:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:16 smithi131 ceph-mon[103713]: pgmap v362: 129 pgs: 129 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 125 op/s 2023-09-10T00:52:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:16 smithi131 ceph-mon[103713]: osdmap e104: 12 total, 12 up, 12 in 2023-09-10T00:52:17.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:16 smithi044 ceph-mon[98676]: pgmap v362: 129 pgs: 129 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 125 op/s 2023-09-10T00:52:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:16 smithi044 ceph-mon[98676]: osdmap e104: 12 total, 12 up, 12 in 2023-09-10T00:52:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:16 smithi099 ceph-mon[102692]: pgmap v362: 129 pgs: 129 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 125 op/s 2023-09-10T00:52:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:16 smithi099 ceph-mon[102692]: osdmap e104: 12 total, 12 up, 12 in 2023-09-10T00:52:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:44300000 of 50000000 tuples (88%) done (elapsed 676.75 s, remaining 87.08 s) 2023-09-10T00:52:18.096 INFO:teuthology.orchestra.run.smithi044.stderr:44400000 of 50000000 tuples (88%) done (elapsed 677.73 s, remaining 85.48 s) 2023-09-10T00:52:18.955 INFO:teuthology.orchestra.run.smithi044.stderr:44500000 of 50000000 tuples (89%) done (elapsed 678.59 s, remaining 83.87 s) 2023-09-10T00:52:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:18 smithi131 ceph-mon[103713]: pgmap v364: 129 pgs: 129 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 MiB/s wr, 150 op/s 2023-09-10T00:52:19.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:18 smithi044 ceph-mon[98676]: pgmap v364: 129 pgs: 129 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 MiB/s wr, 150 op/s 2023-09-10T00:52:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:18 smithi099 ceph-mon[102692]: pgmap v364: 129 pgs: 129 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 MiB/s wr, 150 op/s 2023-09-10T00:52:19.833 INFO:teuthology.orchestra.run.smithi044.stderr:44600000 of 50000000 tuples (89%) done (elapsed 679.47 s, remaining 82.27 s) 2023-09-10T00:52:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.595+0000 7f27c6338700 1 -- 172.21.15.44:0/3750062215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c0119de0 msgr2=0x7f27c011c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.595+0000 7f27c6338700 1 --2- 172.21.15.44:0/3750062215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c0119de0 0x7f27c011c1d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009f90 tx=0x7f27b800b330 comp rx=0 tx=0).stop 2023-09-10T00:52:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.595+0000 7f27c6338700 1 -- 172.21.15.44:0/3750062215 shutdown_connections 2023-09-10T00:52:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.595+0000 7f27c6338700 1 --2- 172.21.15.44:0/3750062215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c0119de0 0x7f27c011c1d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.595+0000 7f27c6338700 1 --2- 172.21.15.44:0/3750062215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c010ed50 0x7f27c0107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.595+0000 7f27c6338700 1 --2- 172.21.15.44:0/3750062215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27c010f1c0 0x7f27c0107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.595+0000 7f27c6338700 1 -- 172.21.15.44:0/3750062215 >> 172.21.15.44:0/3750062215 conn(0x7f27c006c6c0 msgr2=0x7f27c006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.595+0000 7f27c6338700 1 -- 172.21.15.44:0/3750062215 shutdown_connections 2023-09-10T00:52:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.595+0000 7f27c6338700 1 -- 172.21.15.44:0/3750062215 wait complete. 2023-09-10T00:52:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.596+0000 7f27c6338700 1 Processor -- start 2023-09-10T00:52:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.596+0000 7f27c6338700 1 -- start start 2023-09-10T00:52:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.596+0000 7f27c6338700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c010ed50 0x7f27c01a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.596+0000 7f27c6338700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c010f1c0 0x7f27c01a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.596+0000 7f27c6338700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27c0119de0 0x7f27c01a8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.596+0000 7f27c6338700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c01a9580 con 0x7f27c010f1c0 2023-09-10T00:52:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.596+0000 7f27c6338700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c01a96f0 con 0x7f27c010ed50 2023-09-10T00:52:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.596+0000 7f27c6338700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c01a98b0 con 0x7f27c0119de0 2023-09-10T00:52:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.596+0000 7f27bf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c010f1c0 0x7f27c01a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.596+0000 7f27bffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c010ed50 0x7f27c01a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.596+0000 7f27bf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c010f1c0 0x7f27c01a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52788/0 (socket says 172.21.15.44:52788) 2023-09-10T00:52:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.596+0000 7f27bf7fe700 1 -- 172.21.15.44:0/3044705795 learned_addr learned my addr 172.21.15.44:0/3044705795 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.596+0000 7f27c48d5700 1 --2- 172.21.15.44:0/3044705795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27c0119de0 0x7f27c01a8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.597+0000 7f27bffff700 1 -- 172.21.15.44:0/3044705795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27c0119de0 msgr2=0x7f27c01a8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.597+0000 7f27bffff700 1 --2- 172.21.15.44:0/3044705795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27c0119de0 0x7f27c01a8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.597+0000 7f27bffff700 1 -- 172.21.15.44:0/3044705795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c010f1c0 msgr2=0x7f27c01a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.597+0000 7f27bffff700 1 --2- 172.21.15.44:0/3044705795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c010f1c0 0x7f27c01a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.597+0000 7f27bffff700 1 -- 172.21.15.44:0/3044705795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8009c40 con 0x7f27c010ed50 2023-09-10T00:52:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.597+0000 7f27bf7fe700 1 --2- 172.21.15.44:0/3044705795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c010f1c0 0x7f27c01a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.597+0000 7f27bffff700 1 --2- 172.21.15.44:0/3044705795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c010ed50 0x7f27c01a54d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f27b000e9f0 tx=0x7f27b000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.597+0000 7f27bd7fa700 1 -- 172.21.15.44:0/3044705795 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b000c9c0 con 0x7f27c010ed50 2023-09-10T00:52:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.597+0000 7f27c6338700 1 -- 172.21.15.44:0/3044705795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c01a9b90 con 0x7f27c010ed50 2023-09-10T00:52:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.597+0000 7f27bd7fa700 1 -- 172.21.15.44:0/3044705795 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27b0010430 con 0x7f27c010ed50 2023-09-10T00:52:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.597+0000 7f27bd7fa700 1 -- 172.21.15.44:0/3044705795 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0018680 con 0x7f27c010ed50 2023-09-10T00:52:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.597+0000 7f27c6338700 1 -- 172.21.15.44:0/3044705795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c01b24f0 con 0x7f27c010ed50 2023-09-10T00:52:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.599+0000 7f27bd7fa700 1 -- 172.21.15.44:0/3044705795 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f27b00105a0 con 0x7f27c010ed50 2023-09-10T00:52:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.600+0000 7f27bd7fa700 1 --2- 172.21.15.44:0/3044705795 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f27a80a07b0 0x7f27a80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.600+0000 7f27bd7fa700 1 -- 172.21.15.44:0/3044705795 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f27b0014070 con 0x7f27c010ed50 2023-09-10T00:52:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.600+0000 7f27bf7fe700 1 --2- 172.21.15.44:0/3044705795 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f27a80a07b0 0x7f27a80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.601+0000 7f27bf7fe700 1 --2- 172.21.15.44:0/3044705795 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f27a80a07b0 0x7f27a80a2c70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f27c01a6ca0 tx=0x7f27b400b480 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27d24feb80 1 Processor -- start 2023-09-10T00:52:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27d24feb80 1 -- start start 2023-09-10T00:52:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27d24feb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638da1a0 0x55f6638da580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27d24feb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638d4300 0x55f6638d4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27d24feb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4cc0 0x55f6638e0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27d24feb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66382d6d0 con 0x55f6638da1a0 2023-09-10T00:52:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27d24feb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66382aff0 con 0x55f6638d4cc0 2023-09-10T00:52:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27d24feb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6638268e0 con 0x55f6638d4300 2023-09-10T00:52:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27bffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638da1a0 0x55f6638da580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27bffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638da1a0 0x55f6638da580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52794/0 (socket says 172.21.15.44:52794) 2023-09-10T00:52:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27bf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638d4300 0x55f6638d4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27bffff700 1 -- 172.21.15.44:0/1560947979 learned_addr learned my addr 172.21.15.44:0/1560947979 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27c48d5700 1 --2- 172.21.15.44:0/1560947979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4cc0 0x55f6638e0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27bf7fe700 1 -- 172.21.15.44:0/1560947979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4cc0 msgr2=0x55f6638e0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27bf7fe700 1 --2- 172.21.15.44:0/1560947979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4cc0 0x55f6638e0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27bf7fe700 1 -- 172.21.15.44:0/1560947979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638da1a0 msgr2=0x55f6638da580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27bf7fe700 1 --2- 172.21.15.44:0/1560947979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638da1a0 0x55f6638da580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27bf7fe700 1 -- 172.21.15.44:0/1560947979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f663652d80 con 0x55f6638d4300 2023-09-10T00:52:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.602+0000 7f27bffff700 1 --2- 172.21.15.44:0/1560947979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638da1a0 0x55f6638da580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.603+0000 7f27bf7fe700 1 --2- 172.21.15.44:0/1560947979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638d4300 0x55f6638d4780 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f27b4011710 tx=0x7f27b4013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a83cd28a415433a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.603+0000 7f27a67fc700 1 -- 172.21.15.44:0/1560947979 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b401e620 con 0x55f6638d4300 2023-09-10T00:52:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.603+0000 7f27a67fc700 1 -- 172.21.15.44:0/1560947979 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27b4015070 con 0x55f6638d4300 2023-09-10T00:52:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.603+0000 7f27a67fc700 1 -- 172.21.15.44:0/1560947979 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b401caa0 con 0x55f6638d4300 2023-09-10T00:52:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.604+0000 7f27d24feb80 1 -- 172.21.15.44:0/1560947979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638d4300 msgr2=0x55f6638d4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.604+0000 7f27d24feb80 1 --2- 172.21.15.44:0/1560947979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638d4300 0x55f6638d4780 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f27b4011710 tx=0x7f27b4013000 comp rx=0 tx=0).stop 2023-09-10T00:52:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.604+0000 7f27d24feb80 1 -- 172.21.15.44:0/1560947979 shutdown_connections 2023-09-10T00:52:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.604+0000 7f27d24feb80 1 --2- 172.21.15.44:0/1560947979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4cc0 0x55f6638e0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.604+0000 7f27d24feb80 1 --2- 172.21.15.44:0/1560947979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638da1a0 0x55f6638da580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.604+0000 7f27d24feb80 1 --2- 172.21.15.44:0/1560947979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638d4300 0x55f6638d4780 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.604+0000 7f27d24feb80 1 -- 172.21.15.44:0/1560947979 >> 172.21.15.44:0/1560947979 conn(0x55f663849030 msgr2=0x55f6638cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.604+0000 7f27d24feb80 1 -- 172.21.15.44:0/1560947979 shutdown_connections 2023-09-10T00:52:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.604+0000 7f27d24feb80 1 -- 172.21.15.44:0/1560947979 wait complete. 2023-09-10T00:52:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.604+0000 7f27d24feb80 1 -- start start 2023-09-10T00:52:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.604+0000 7f27d24feb80 1 Processor -- start 2023-09-10T00:52:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27d24feb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4300 0x55f6638e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27d24feb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638de110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27d24feb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638da1a0 0x55f6638de6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27d24feb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6637c4b10 con 0x55f6638d4cc0 2023-09-10T00:52:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27d24feb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6637c4810 con 0x55f6638d4300 2023-09-10T00:52:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27d24feb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6636a88c0 con 0x55f6638da1a0 2023-09-10T00:52:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27d24feb80 10 client.0 _ll_register_callbacks cb 0x55f66370ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:52:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27bf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638de110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27bf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638de110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52802/0 (socket says 172.21.15.44:52802) 2023-09-10T00:52:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27bffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4300 0x55f6638e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27bf7fe700 1 -- 172.21.15.44:0/50479725 learned_addr learned my addr 172.21.15.44:0/50479725 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27c48d5700 1 --2- 172.21.15.44:0/50479725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638da1a0 0x55f6638de6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27bf7fe700 1 -- 172.21.15.44:0/50479725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638da1a0 msgr2=0x55f6638de6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27bf7fe700 1 --2- 172.21.15.44:0/50479725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638da1a0 0x55f6638de6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27bf7fe700 1 -- 172.21.15.44:0/50479725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4300 msgr2=0x55f6638e0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27bf7fe700 1 --2- 172.21.15.44:0/50479725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4300 0x55f6638e0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27bf7fe700 1 -- 172.21.15.44:0/50479725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c01b24f0 con 0x55f6638d4cc0 2023-09-10T00:52:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27c48d5700 1 --2- 172.21.15.44:0/50479725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638da1a0 0x55f6638de6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27bffff700 1 --2- 172.21.15.44:0/50479725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4300 0x55f6638e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f27bf7fe700 1 --2- 172.21.15.44:0/50479725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638de110 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f27b401c870 tx=0x7f27b401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fbbb66f5aa3a00 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.605+0000 7f279f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:52:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.606+0000 7f279f7fe700 1 -- 172.21.15.44:0/50479725 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b402c070 con 0x55f6638d4cc0 2023-09-10T00:52:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.606+0000 7f27d24feb80 1 -- 172.21.15.44:0/50479725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f66377e490 con 0x55f6638d4cc0 2023-09-10T00:52:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.606+0000 7f279f7fe700 1 -- 172.21.15.44:0/50479725 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f27b4017070 con 0x55f6638d4cc0 2023-09-10T00:52:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.606+0000 7f279f7fe700 1 -- 172.21.15.44:0/50479725 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27b401cdc0 con 0x55f6638d4cc0 2023-09-10T00:52:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.606+0000 7f27d24feb80 10 client.17496 fetch_fsmap learned FSMap version 61 2023-09-10T00:52:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.606+0000 7f27d24feb80 1 -- 172.21.15.44:0/50479725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f27b4011440 con 0x55f6638d4cc0 2023-09-10T00:52:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.606+0000 7f279f7fe700 1 -- 172.21.15.44:0/50479725 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f66377e490 con 0x55f6638d4cc0 2023-09-10T00:52:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.606+0000 7f279f7fe700 1 -- 172.21.15.44:0/50479725 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f27b40263f0 con 0x55f6638d4cc0 2023-09-10T00:52:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.606+0000 7f27d24feb80 10 client.17496 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:52:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.606+0000 7f27d24feb80 10 client.17496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:52:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.606+0000 7f27d24feb80 20 client.17496 populate_metadata read hostname 'smithi044' 2023-09-10T00:52:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.606+0000 7f27d24feb80 1 --2- 172.21.15.44:0/50479725 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f6638ee390 0x55f6638f07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.606+0000 7f27d24feb80 4 client.17496 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:52:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.606+0000 7f27d24feb80 1 -- 172.21.15.44:0/50479725 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f66383b840 con 0x55f6638ee390 2023-09-10T00:52:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.607+0000 7f27bffff700 1 --2- 172.21.15.44:0/50479725 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f6638ee390 0x55f6638f07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.607+0000 7f27bffff700 1 --2- 172.21.15.44:0/50479725 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f6638ee390 0x55f6638f07b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53fd72fbf91b8d98 server_cookie=fe9ca049bf500ab2 in_seq=0 out_seq=0 2023-09-10T00:52:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.607+0000 7f279f7fe700 10 client.17496 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.607+0000 7f279f7fe700 1 -- 172.21.15.44:0/50479725 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f66383b840 con 0x55f6638ee390 2023-09-10T00:52:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.608+0000 7f279f7fe700 10 client.17496 handle_command_reply: tid=0 2023-09-10T00:52:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.608+0000 7f27d24feb80 1 client.17496 shutdown 2023-09-10T00:52:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.608+0000 7f27d24feb80 1 -- 172.21.15.44:0/50479725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 msgr2=0x55f6638de110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.608+0000 7f27d24feb80 1 --2- 172.21.15.44:0/50479725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638de110 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f27b401c870 tx=0x7f27b401c8a0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.608+0000 7f27d24feb80 1 -- 172.21.15.44:0/50479725 shutdown_connections 2023-09-10T00:52:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.608+0000 7f27d24feb80 1 --2- 172.21.15.44:0/50479725 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f6638ee390 0x55f6638f07b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.608+0000 7f27d24feb80 1 --2- 172.21.15.44:0/50479725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638da1a0 0x55f6638de6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.608+0000 7f279f7fe700 0 client.17496 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.608+0000 7f27d24feb80 1 --2- 172.21.15.44:0/50479725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4300 0x55f6638e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.608+0000 7f27d24feb80 1 --2- 172.21.15.44:0/50479725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638de110 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.608+0000 7f27d24feb80 1 -- 172.21.15.44:0/50479725 >> 172.21.15.44:0/50479725 conn(0x55f663849030 msgr2=0x55f6638ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.608+0000 7f27d24feb80 1 -- 172.21.15.44:0/50479725 shutdown_connections 2023-09-10T00:52:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.608+0000 7f27d24feb80 1 -- 172.21.15.44:0/50479725 wait complete. 2023-09-10T00:52:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.608+0000 7f27d24feb80 20 client.17496 trim_cache size 0 max 16384 2023-09-10T00:52:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.628+0000 7f27d24feb80 1 Processor -- start 2023-09-10T00:52:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.628+0000 7f27d24feb80 1 -- start start 2023-09-10T00:52:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.628+0000 7f27d24feb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638d4300 0x55f6638f0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.628+0000 7f27d24feb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638e08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.628+0000 7f27d24feb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638da1a0 0x55f6638d81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.628+0000 7f27d24feb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66377e4c0 con 0x55f6638d4cc0 2023-09-10T00:52:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.628+0000 7f27d24feb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6638dc6a0 con 0x55f6638da1a0 2023-09-10T00:52:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.628+0000 7f27d24feb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6638df0e0 con 0x55f6638d4300 2023-09-10T00:52:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.628+0000 7f27bf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638e08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.628+0000 7f27bffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638d4300 0x55f6638f0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.628+0000 7f27bffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638d4300 0x55f6638f0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59690/0 (socket says 172.21.15.44:59690) 2023-09-10T00:52:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.628+0000 7f27bf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638e08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52810/0 (socket says 172.21.15.44:52810) 2023-09-10T00:52:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.628+0000 7f27bffff700 1 -- 172.21.15.44:0/2827996422 learned_addr learned my addr 172.21.15.44:0/2827996422 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.628+0000 7f27c48d5700 1 --2- 172.21.15.44:0/2827996422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638da1a0 0x55f6638d81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.629+0000 7f27bffff700 1 -- 172.21.15.44:0/2827996422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638da1a0 msgr2=0x55f6638d81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.629+0000 7f27bffff700 1 --2- 172.21.15.44:0/2827996422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638da1a0 0x55f6638d81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.629+0000 7f27bffff700 1 -- 172.21.15.44:0/2827996422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 msgr2=0x55f6638e08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.629+0000 7f27bffff700 1 --2- 172.21.15.44:0/2827996422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638e08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.629+0000 7f27bffff700 1 -- 172.21.15.44:0/2827996422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f66363bc00 con 0x55f6638d4300 2023-09-10T00:52:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.629+0000 7f27bffff700 1 --2- 172.21.15.44:0/2827996422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638d4300 0x55f6638f0780 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f27b0000c00 tx=0x7f27b0095290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8bd993e4c3cc6ca server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.629+0000 7f27a6ffd700 1 -- 172.21.15.44:0/2827996422 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0099c90 con 0x55f6638d4300 2023-09-10T00:52:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.629+0000 7f27a6ffd700 1 -- 172.21.15.44:0/2827996422 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27b0099590 con 0x55f6638d4300 2023-09-10T00:52:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.629+0000 7f27a6ffd700 1 -- 172.21.15.44:0/2827996422 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0099a50 con 0x55f6638d4300 2023-09-10T00:52:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.630+0000 7f27d24feb80 1 -- 172.21.15.44:0/2827996422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638d4300 msgr2=0x55f6638f0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.630+0000 7f27d24feb80 1 --2- 172.21.15.44:0/2827996422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638d4300 0x55f6638f0780 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f27b0000c00 tx=0x7f27b0095290 comp rx=0 tx=0).stop 2023-09-10T00:52:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.630+0000 7f27d24feb80 1 -- 172.21.15.44:0/2827996422 shutdown_connections 2023-09-10T00:52:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.630+0000 7f27d24feb80 1 --2- 172.21.15.44:0/2827996422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638da1a0 0x55f6638d81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.630+0000 7f27d24feb80 1 --2- 172.21.15.44:0/2827996422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638d4300 0x55f6638f0780 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.630+0000 7f27d24feb80 1 --2- 172.21.15.44:0/2827996422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638e08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.630+0000 7f27d24feb80 1 -- 172.21.15.44:0/2827996422 >> 172.21.15.44:0/2827996422 conn(0x55f663849030 msgr2=0x55f6638d9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.630+0000 7f27d24feb80 1 -- 172.21.15.44:0/2827996422 shutdown_connections 2023-09-10T00:52:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.630+0000 7f27d24feb80 1 -- 172.21.15.44:0/2827996422 wait complete. 2023-09-10T00:52:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.630+0000 7f27d24feb80 1 -- start start 2023-09-10T00:52:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27d24feb80 1 Processor -- start 2023-09-10T00:52:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27d24feb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4300 0x55f6638e3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27d24feb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638e3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27d24feb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638da1a0 0x55f6638e3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27d24feb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f663941360 con 0x55f6638d4cc0 2023-09-10T00:52:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27d24feb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6639414d0 con 0x55f6638d4300 2023-09-10T00:52:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27d24feb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f663941690 con 0x55f6638da1a0 2023-09-10T00:52:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27d24feb80 10 client.0 _ll_register_callbacks cb 0x7f27b0009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:52:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27bf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638e3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27bf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638e3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52816/0 (socket says 172.21.15.44:52816) 2023-09-10T00:52:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27bf7fe700 1 -- 172.21.15.44:0/925712422 learned_addr learned my addr 172.21.15.44:0/925712422 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27c48d5700 1 --2- 172.21.15.44:0/925712422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638da1a0 0x55f6638e3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27bffff700 1 --2- 172.21.15.44:0/925712422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4300 0x55f6638e3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27c48d5700 1 -- 172.21.15.44:0/925712422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4300 msgr2=0x55f6638e3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27c48d5700 1 --2- 172.21.15.44:0/925712422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4300 0x55f6638e3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27c48d5700 1 -- 172.21.15.44:0/925712422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 msgr2=0x55f6638e3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27c48d5700 1 --2- 172.21.15.44:0/925712422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638e3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27c48d5700 1 -- 172.21.15.44:0/925712422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c01a9b90 con 0x55f6638da1a0 2023-09-10T00:52:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27bf7fe700 1 --2- 172.21.15.44:0/925712422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638e3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.631+0000 7f27bffff700 1 --2- 172.21.15.44:0/925712422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4300 0x55f6638e3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.632+0000 7f27c48d5700 1 --2- 172.21.15.44:0/925712422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638da1a0 0x55f6638e3fb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009790 tx=0x7f27b8007c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5229fc4f3c6b486 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.632+0000 7f279f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:52:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.632+0000 7f279f7fe700 1 -- 172.21.15.44:0/925712422 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8022070 con 0x55f6638da1a0 2023-09-10T00:52:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.632+0000 7f27d24feb80 1 -- 172.21.15.44:0/925712422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27b4015070 con 0x55f6638da1a0 2023-09-10T00:52:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.632+0000 7f279f7fe700 1 -- 172.21.15.44:0/925712422 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f27b8007e40 con 0x55f6638da1a0 2023-09-10T00:52:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.632+0000 7f279f7fe700 1 -- 172.21.15.44:0/925712422 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27b800e040 con 0x55f6638da1a0 2023-09-10T00:52:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.632+0000 7f27d24feb80 10 client.27106 fetch_fsmap learned FSMap version 61 2023-09-10T00:52:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.632+0000 7f27d24feb80 1 -- 172.21.15.44:0/925712422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55f6636791f0 con 0x55f6638da1a0 2023-09-10T00:52:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.632+0000 7f279f7fe700 1 -- 172.21.15.44:0/925712422 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27b802f020 con 0x55f6638da1a0 2023-09-10T00:52:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.632+0000 7f279f7fe700 1 -- 172.21.15.44:0/925712422 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f27b800b050 con 0x55f6638da1a0 2023-09-10T00:52:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.632+0000 7f27d24feb80 10 client.27106 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:52:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.632+0000 7f27d24feb80 10 client.27106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:52:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.632+0000 7f27d24feb80 20 client.27106 populate_metadata read hostname 'smithi044' 2023-09-10T00:52:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.632+0000 7f27d24feb80 1 --2- 172.21.15.44:0/925712422 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f663942f00 0x55f663945320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.632+0000 7f27d24feb80 4 client.27106 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:52:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.632+0000 7f27d24feb80 1 -- 172.21.15.44:0/925712422 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f66381f8c0 con 0x55f663942f00 2023-09-10T00:52:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.633+0000 7f27bffff700 1 --2- 172.21.15.44:0/925712422 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f663942f00 0x55f663945320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.633+0000 7f279f7fe700 10 client.27106 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.633+0000 7f27bffff700 1 --2- 172.21.15.44:0/925712422 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f663942f00 0x55f663945320 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fae2c8cb925200ce server_cookie=37b0a15e7ba2bd3d in_seq=0 out_seq=0 2023-09-10T00:52:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.646+0000 7f279f7fe700 1 -- 172.21.15.44:0/925712422 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f66381f8c0 con 0x55f663942f00 2023-09-10T00:52:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.646+0000 7f279f7fe700 10 client.27106 handle_command_reply: tid=0 2023-09-10T00:52:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.647+0000 7f27d24feb80 1 client.27106 shutdown 2023-09-10T00:52:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.647+0000 7f27d24feb80 1 -- 172.21.15.44:0/925712422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638da1a0 msgr2=0x55f6638e3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.647+0000 7f27d24feb80 1 --2- 172.21.15.44:0/925712422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638da1a0 0x55f6638e3fb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009790 tx=0x7f27b8007c90 comp rx=0 tx=0).stop 2023-09-10T00:52:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.647+0000 7f27d24feb80 1 -- 172.21.15.44:0/925712422 shutdown_connections 2023-09-10T00:52:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.647+0000 7f27d24feb80 1 --2- 172.21.15.44:0/925712422 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f663942f00 0x55f663945320 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.647+0000 7f27d24feb80 1 --2- 172.21.15.44:0/925712422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f6638da1a0 0x55f6638e3fb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.647+0000 7f279f7fe700 0 client.27106 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.647+0000 7f27d24feb80 1 --2- 172.21.15.44:0/925712422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6638d4300 0x55f6638e3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.647+0000 7f27d24feb80 1 --2- 172.21.15.44:0/925712422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f6638d4cc0 0x55f6638e3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.647+0000 7f27d24feb80 1 -- 172.21.15.44:0/925712422 >> 172.21.15.44:0/925712422 conn(0x55f663849030 msgr2=0x55f6638d81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.647+0000 7f27d24feb80 1 -- 172.21.15.44:0/925712422 shutdown_connections 2023-09-10T00:52:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.647+0000 7f27d24feb80 1 -- 172.21.15.44:0/925712422 wait complete. 2023-09-10T00:52:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.647+0000 7f27d24feb80 20 client.27106 trim_cache size 0 max 16384 2023-09-10T00:52:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:52:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:52:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:52:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:52:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.648+0000 7f27c6338700 1 -- 172.21.15.44:0/3044705795 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f27a80a07b0 msgr2=0x7f27a80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.648+0000 7f27c6338700 1 --2- 172.21.15.44:0/3044705795 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f27a80a07b0 0x7f27a80a2c70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f27c01a6ca0 tx=0x7f27b400b480 comp rx=0 tx=0).stop 2023-09-10T00:52:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.648+0000 7f27c6338700 1 -- 172.21.15.44:0/3044705795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c010ed50 msgr2=0x7f27c01a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.648+0000 7f27c6338700 1 --2- 172.21.15.44:0/3044705795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c010ed50 0x7f27c01a54d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f27b000e9f0 tx=0x7f27b000ed00 comp rx=0 tx=0).stop 2023-09-10T00:52:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.649+0000 7f27c6338700 1 -- 172.21.15.44:0/3044705795 shutdown_connections 2023-09-10T00:52:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.649+0000 7f27c6338700 1 --2- 172.21.15.44:0/3044705795 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f27a80a07b0 0x7f27a80a2c70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.649+0000 7f27c6338700 1 --2- 172.21.15.44:0/3044705795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27c0119de0 0x7f27c01a8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.649+0000 7f27c6338700 1 --2- 172.21.15.44:0/3044705795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c010ed50 0x7f27c01a54d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.649+0000 7f27c6338700 1 --2- 172.21.15.44:0/3044705795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c010f1c0 0x7f27c01a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:20.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.649+0000 7f27c6338700 1 -- 172.21.15.44:0/3044705795 >> 172.21.15.44:0/3044705795 conn(0x7f27c006c6c0 msgr2=0x7f27c0117dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:20.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.649+0000 7f27c6338700 1 -- 172.21.15.44:0/3044705795 shutdown_connections 2023-09-10T00:52:20.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:20.649+0000 7f27c6338700 1 -- 172.21.15.44:0/3044705795 wait complete. 2023-09-10T00:52:20.754 INFO:teuthology.orchestra.run.smithi044.stderr:44700000 of 50000000 tuples (89%) done (elapsed 680.39 s, remaining 80.67 s) 2023-09-10T00:52:21.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:20 smithi131 ceph-mon[103713]: pgmap v365: 129 pgs: 129 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 125 op/s 2023-09-10T00:52:21.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:20 smithi044 ceph-mon[98676]: pgmap v365: 129 pgs: 129 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 125 op/s 2023-09-10T00:52:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:20 smithi099 ceph-mon[102692]: pgmap v365: 129 pgs: 129 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 125 op/s 2023-09-10T00:52:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:44800000 of 50000000 tuples (89%) done (elapsed 681.50 s, remaining 79.10 s) 2023-09-10T00:52:22.734 INFO:teuthology.orchestra.run.smithi044.stderr:44900000 of 50000000 tuples (89%) done (elapsed 682.37 s, remaining 77.51 s) 2023-09-10T00:52:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:22 smithi131 ceph-mon[103713]: pgmap v366: 129 pgs: 129 active+clean; 8.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 37 MiB/s wr, 95 op/s 2023-09-10T00:52:23.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:22 smithi044 ceph-mon[98676]: pgmap v366: 129 pgs: 129 active+clean; 8.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 37 MiB/s wr, 95 op/s 2023-09-10T00:52:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:22 smithi099 ceph-mon[102692]: pgmap v366: 129 pgs: 129 active+clean; 8.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 37 MiB/s wr, 95 op/s 2023-09-10T00:52:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:24 smithi131 ceph-mon[103713]: pgmap v367: 129 pgs: 129 active+clean; 8.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 37 MiB/s wr, 95 op/s 2023-09-10T00:52:25.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:24 smithi044 ceph-mon[98676]: pgmap v367: 129 pgs: 129 active+clean; 8.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 37 MiB/s wr, 95 op/s 2023-09-10T00:52:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:24 smithi099 ceph-mon[102692]: pgmap v367: 129 pgs: 129 active+clean; 8.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 37 MiB/s wr, 95 op/s 2023-09-10T00:52:26.488 INFO:teuthology.orchestra.run.smithi044.stderr:45000000 of 50000000 tuples (90%) done (elapsed 686.12 s, remaining 76.24 s) 2023-09-10T00:52:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:26 smithi131 ceph-mon[103713]: pgmap v368: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 MiB/s wr, 66 op/s 2023-09-10T00:52:27.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:26 smithi044 ceph-mon[98676]: pgmap v368: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 MiB/s wr, 66 op/s 2023-09-10T00:52:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:26 smithi099 ceph-mon[102692]: pgmap v368: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 MiB/s wr, 66 op/s 2023-09-10T00:52:27.640 INFO:teuthology.orchestra.run.smithi044.stderr:45100000 of 50000000 tuples (90%) done (elapsed 687.27 s, remaining 74.67 s) 2023-09-10T00:52:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:28 smithi131 ceph-mon[103713]: pgmap v369: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 MiB/s wr, 57 op/s 2023-09-10T00:52:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:28 smithi044 ceph-mon[98676]: pgmap v369: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 MiB/s wr, 57 op/s 2023-09-10T00:52:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:28 smithi099 ceph-mon[102692]: pgmap v369: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 MiB/s wr, 57 op/s 2023-09-10T00:52:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:30 smithi131 ceph-mon[103713]: pgmap v370: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 55 op/s 2023-09-10T00:52:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:30 smithi044 ceph-mon[98676]: pgmap v370: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 55 op/s 2023-09-10T00:52:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:30 smithi099 ceph-mon[102692]: pgmap v370: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 55 op/s 2023-09-10T00:52:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:45200000 of 50000000 tuples (90%) done (elapsed 691.83 s, remaining 73.47 s) 2023-09-10T00:52:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:32 smithi131 ceph-mon[103713]: pgmap v371: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 32 MiB/s wr, 84 op/s 2023-09-10T00:52:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:32 smithi044 ceph-mon[98676]: pgmap v371: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 32 MiB/s wr, 84 op/s 2023-09-10T00:52:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:32 smithi099 ceph-mon[102692]: pgmap v371: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 32 MiB/s wr, 84 op/s 2023-09-10T00:52:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.777+0000 7f2e30a73700 1 -- 172.21.15.44:0/453907514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e2c1055b0 msgr2=0x7f2e2c105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.777+0000 7f2e30a73700 1 --2- 172.21.15.44:0/453907514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e2c1055b0 0x7f2e2c105990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009990 tx=0x7f2e1c009ca0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.777+0000 7f2e30a73700 1 -- 172.21.15.44:0/453907514 shutdown_connections 2023-09-10T00:52:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.777+0000 7f2e30a73700 1 --2- 172.21.15.44:0/453907514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e2c100120 0x7f2e2c10c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.777+0000 7f2e30a73700 1 --2- 172.21.15.44:0/453907514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e2c1055b0 0x7f2e2c105990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.777+0000 7f2e30a73700 1 --2- 172.21.15.44:0/453907514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e2c0ff760 0x7f2e2c0ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.777+0000 7f2e30a73700 1 -- 172.21.15.44:0/453907514 >> 172.21.15.44:0/453907514 conn(0x7f2e2c074f30 msgr2=0x7f2e2c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.777+0000 7f2e30a73700 1 -- 172.21.15.44:0/453907514 shutdown_connections 2023-09-10T00:52:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.777+0000 7f2e30a73700 1 -- 172.21.15.44:0/453907514 wait complete. 2023-09-10T00:52:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.778+0000 7f2e30a73700 1 Processor -- start 2023-09-10T00:52:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.778+0000 7f2e30a73700 1 -- start start 2023-09-10T00:52:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.778+0000 7f2e30a73700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e2c0ff760 0x7f2e2c197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.778+0000 7f2e30a73700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e2c100120 0x7f2e2c197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.778+0000 7f2e30a73700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e2c1055b0 0x7f2e2c1a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.778+0000 7f2e30a73700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c1a29b0 con 0x7f2e2c100120 2023-09-10T00:52:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.778+0000 7f2e30a73700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c1a2b20 con 0x7f2e2c0ff760 2023-09-10T00:52:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.778+0000 7f2e30a73700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c1a2ce0 con 0x7f2e2c1055b0 2023-09-10T00:52:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.779+0000 7f2e2a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e2c0ff760 0x7f2e2c197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.779+0000 7f2e2a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e2c0ff760 0x7f2e2c197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54568/0 (socket says 172.21.15.44:54568) 2023-09-10T00:52:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.779+0000 7f2e2a59c700 1 -- 172.21.15.44:0/3879779967 learned_addr learned my addr 172.21.15.44:0/3879779967 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.779+0000 7f2e29d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e2c100120 0x7f2e2c197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.779+0000 7f2e2ad9d700 1 --2- 172.21.15.44:0/3879779967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e2c1055b0 0x7f2e2c1a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.779+0000 7f2e2a59c700 1 -- 172.21.15.44:0/3879779967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e2c1055b0 msgr2=0x7f2e2c1a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.779+0000 7f2e2a59c700 1 --2- 172.21.15.44:0/3879779967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e2c1055b0 0x7f2e2c1a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.779+0000 7f2e2a59c700 1 -- 172.21.15.44:0/3879779967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e2c100120 msgr2=0x7f2e2c197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.779+0000 7f2e2a59c700 1 --2- 172.21.15.44:0/3879779967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e2c100120 0x7f2e2c197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.779+0000 7f2e2a59c700 1 -- 172.21.15.44:0/3879779967 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c0095f0 con 0x7f2e2c0ff760 2023-09-10T00:52:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.779+0000 7f2e2a59c700 1 --2- 172.21.15.44:0/3879779967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e2c0ff760 0x7f2e2c197540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c006010 tx=0x7f2e1c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.779+0000 7f2e1b7fe700 1 -- 172.21.15.44:0/3879779967 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c01d070 con 0x7f2e2c0ff760 2023-09-10T00:52:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.779+0000 7f2e1b7fe700 1 -- 172.21.15.44:0/3879779967 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e1c022470 con 0x7f2e2c0ff760 2023-09-10T00:52:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.779+0000 7f2e30a73700 1 -- 172.21.15.44:0/3879779967 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2c1a2f60 con 0x7f2e2c0ff760 2023-09-10T00:52:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.779+0000 7f2e1b7fe700 1 -- 172.21.15.44:0/3879779967 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c017650 con 0x7f2e2c0ff760 2023-09-10T00:52:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.779+0000 7f2e30a73700 1 -- 172.21.15.44:0/3879779967 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2c1a3320 con 0x7f2e2c0ff760 2023-09-10T00:52:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.782+0000 7f2e1b7fe700 1 -- 172.21.15.44:0/3879779967 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f2e1c0225e0 con 0x7f2e2c0ff760 2023-09-10T00:52:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.782+0000 7f2e1b7fe700 1 --2- 172.21.15.44:0/3879779967 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2e140a07b0 0x7f2e140a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.782+0000 7f2e1b7fe700 1 -- 172.21.15.44:0/3879779967 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2e1c0cd340 con 0x7f2e2c0ff760 2023-09-10T00:52:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.782+0000 7f2e29d9b700 1 --2- 172.21.15.44:0/3879779967 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2e140a07b0 0x7f2e140a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.783+0000 7f2e29d9b700 1 --2- 172.21.15.44:0/3879779967 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2e140a07b0 0x7f2e140a2c70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c068ad0 tx=0x7f2e20009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.783+0000 7f2e3cc39b80 1 Processor -- start 2023-09-10T00:52:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.783+0000 7f2e3cc39b80 1 -- start start 2023-09-10T00:52:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.783+0000 7f2e3cc39b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 0x560851701990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.783+0000 7f2e3cc39b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fb710 0x5608516fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.783+0000 7f2e3cc39b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fc0d0 0x560851708340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.783+0000 7f2e3cc39b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608516506d0 con 0x5608516fb710 2023-09-10T00:52:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.783+0000 7f2e3cc39b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085164dff0 con 0x5608517015b0 2023-09-10T00:52:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.783+0000 7f2e3cc39b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608516498e0 con 0x5608516fc0d0 2023-09-10T00:52:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.784+0000 7f2e29d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fb710 0x5608516fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.784+0000 7f2e2a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 0x560851701990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.784+0000 7f2e2ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fc0d0 0x560851708340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.784+0000 7f2e2a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 0x560851701990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54576/0 (socket says 172.21.15.44:54576) 2023-09-10T00:52:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.784+0000 7f2e29d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fb710 0x5608516fbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52838/0 (socket says 172.21.15.44:52838) 2023-09-10T00:52:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.784+0000 7f2e2ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fc0d0 0x560851708340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59722/0 (socket says 172.21.15.44:59722) 2023-09-10T00:52:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.784+0000 7f2e2a59c700 1 -- 172.21.15.44:0/3511991567 learned_addr learned my addr 172.21.15.44:0/3511991567 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.784+0000 7f2e2a59c700 1 -- 172.21.15.44:0/3511991567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fc0d0 msgr2=0x560851708340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.784+0000 7f2e2a59c700 1 --2- 172.21.15.44:0/3511991567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fc0d0 0x560851708340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.784+0000 7f2e2a59c700 1 -- 172.21.15.44:0/3511991567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fb710 msgr2=0x5608516fbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.784+0000 7f2e2a59c700 1 --2- 172.21.15.44:0/3511991567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fb710 0x5608516fbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.784+0000 7f2e2a59c700 1 -- 172.21.15.44:0/3511991567 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560851475d80 con 0x5608517015b0 2023-09-10T00:52:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.784+0000 7f2e2a59c700 1 --2- 172.21.15.44:0/3511991567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 0x560851701990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0a1f30 tx=0x7f2e1c09a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2830446c512a7e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.784+0000 7f2e18ff9700 1 -- 172.21.15.44:0/3511991567 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c09abe0 con 0x5608517015b0 2023-09-10T00:52:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.784+0000 7f2e18ff9700 1 -- 172.21.15.44:0/3511991567 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e1c09a480 con 0x5608517015b0 2023-09-10T00:52:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.784+0000 7f2e18ff9700 1 -- 172.21.15.44:0/3511991567 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c09a940 con 0x5608517015b0 2023-09-10T00:52:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.785+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/3511991567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 msgr2=0x560851701990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.785+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/3511991567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 0x560851701990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0a1f30 tx=0x7f2e1c09a370 comp rx=0 tx=0).stop 2023-09-10T00:52:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.785+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/3511991567 shutdown_connections 2023-09-10T00:52:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.785+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/3511991567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fc0d0 0x560851708340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.785+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/3511991567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 0x560851701990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.785+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/3511991567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fb710 0x5608516fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.785+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/3511991567 >> 172.21.15.44:0/3511991567 conn(0x56085166c030 msgr2=0x5608516f2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.785+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/3511991567 shutdown_connections 2023-09-10T00:52:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.785+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/3511991567 wait complete. 2023-09-10T00:52:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.786+0000 7f2e3cc39b80 1 -- start start 2023-09-10T00:52:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.786+0000 7f2e3cc39b80 1 Processor -- start 2023-09-10T00:52:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.786+0000 7f2e3cc39b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 0x56085170aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.786+0000 7f2e3cc39b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608516fc0d0 0x56085170b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.786+0000 7f2e3cc39b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608517015b0 0x560851705170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.786+0000 7f2e3cc39b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608515e7b10 con 0x5608517015b0 2023-09-10T00:52:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.786+0000 7f2e3cc39b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608515e7810 con 0x5608516fc0d0 2023-09-10T00:52:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.786+0000 7f2e3cc39b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608514cb8c0 con 0x5608516fb710 2023-09-10T00:52:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.786+0000 7f2e2ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608517015b0 0x560851705170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.786+0000 7f2e29d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608516fc0d0 0x56085170b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.786+0000 7f2e2a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 0x56085170aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.786+0000 7f2e3cc39b80 10 client.0 _ll_register_callbacks cb 0x560851531a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:52:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.786+0000 7f2e29d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608516fc0d0 0x56085170b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54584/0 (socket says 172.21.15.44:54584) 2023-09-10T00:52:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.786+0000 7f2e2a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 0x56085170aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59724/0 (socket says 172.21.15.44:59724) 2023-09-10T00:52:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.786+0000 7f2e29d9b700 1 -- 172.21.15.44:0/563625682 learned_addr learned my addr 172.21.15.44:0/563625682 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.787+0000 7f2e29d9b700 1 -- 172.21.15.44:0/563625682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 msgr2=0x56085170aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.787+0000 7f2e29d9b700 1 --2- 172.21.15.44:0/563625682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 0x56085170aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.787+0000 7f2e29d9b700 1 -- 172.21.15.44:0/563625682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608517015b0 msgr2=0x560851705170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.787+0000 7f2e29d9b700 1 --2- 172.21.15.44:0/563625682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608517015b0 0x560851705170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.787+0000 7f2e29d9b700 1 -- 172.21.15.44:0/563625682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2c1a3320 con 0x5608516fc0d0 2023-09-10T00:52:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.787+0000 7f2e2a59c700 1 --2- 172.21.15.44:0/563625682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 0x56085170aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.787+0000 7f2e29d9b700 1 --2- 172.21.15.44:0/563625682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608516fc0d0 0x56085170b420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2e20005910 tx=0x7f2e200047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa28a4cc1c181b5a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.787+0000 7f2e11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:52:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.787+0000 7f2e11ffb700 1 -- 172.21.15.44:0/563625682 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20022070 con 0x5608516fc0d0 2023-09-10T00:52:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.787+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/563625682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608515a1490 con 0x5608516fc0d0 2023-09-10T00:52:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.787+0000 7f2e11ffb700 1 -- 172.21.15.44:0/563625682 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2e20004510 con 0x5608516fc0d0 2023-09-10T00:52:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.787+0000 7f2e11ffb700 1 -- 172.21.15.44:0/563625682 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e20027420 con 0x5608516fc0d0 2023-09-10T00:52:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.787+0000 7f2e3cc39b80 10 client.27182 fetch_fsmap learned FSMap version 61 2023-09-10T00:52:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.787+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/563625682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x56085145ec00 con 0x5608516fc0d0 2023-09-10T00:52:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.787+0000 7f2e11ffb700 1 -- 172.21.15.44:0/563625682 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e20030020 con 0x5608516fc0d0 2023-09-10T00:52:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.787+0000 7f2e11ffb700 1 -- 172.21.15.44:0/563625682 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f2e2001d510 con 0x5608516fc0d0 2023-09-10T00:52:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.788+0000 7f2e3cc39b80 10 client.27182 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:52:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.788+0000 7f2e3cc39b80 10 client.27182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:52:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.788+0000 7f2e3cc39b80 20 client.27182 populate_metadata read hostname 'smithi044' 2023-09-10T00:52:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.788+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/563625682 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560851715590 0x5608517179b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.788+0000 7f2e3cc39b80 4 client.27182 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:52:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.788+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/563625682 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56085165e840 con 0x560851715590 2023-09-10T00:52:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.788+0000 7f2e2a59c700 1 --2- 172.21.15.44:0/563625682 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560851715590 0x5608517179b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.788+0000 7f2e11ffb700 10 client.27182 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.788+0000 7f2e2a59c700 1 --2- 172.21.15.44:0/563625682 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560851715590 0x5608517179b0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1ba7edb0ed623ec server_cookie=767dd6b2a53f28e2 in_seq=0 out_seq=0 2023-09-10T00:52:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.789+0000 7f2e11ffb700 1 -- 172.21.15.44:0/563625682 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56085165e840 con 0x560851715590 2023-09-10T00:52:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.789+0000 7f2e11ffb700 10 client.27182 handle_command_reply: tid=0 2023-09-10T00:52:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.789+0000 7f2e3cc39b80 1 client.27182 shutdown 2023-09-10T00:52:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.789+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/563625682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608516fc0d0 msgr2=0x56085170b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.789+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/563625682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608516fc0d0 0x56085170b420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2e20005910 tx=0x7f2e200047c0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.789+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/563625682 shutdown_connections 2023-09-10T00:52:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.789+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/563625682 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560851715590 0x5608517179b0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.789+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/563625682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608517015b0 0x560851705170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.789+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/563625682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 0x56085170aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.789+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/563625682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608516fc0d0 0x56085170b420 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.789+0000 7f2e11ffb700 0 client.27182 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.789+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/563625682 >> 172.21.15.44:0/563625682 conn(0x56085166c030 msgr2=0x5608516f17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.789+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/563625682 shutdown_connections 2023-09-10T00:52:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.789+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/563625682 wait complete. 2023-09-10T00:52:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.789+0000 7f2e3cc39b80 20 client.27182 trim_cache size 0 max 16384 2023-09-10T00:52:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.804+0000 7f2e3cc39b80 1 Processor -- start 2023-09-10T00:52:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.804+0000 7f2e3cc39b80 1 -- start start 2023-09-10T00:52:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e3cc39b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 0x5608516f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e3cc39b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fc0d0 0x56085170aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e3cc39b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 0x56085170aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e3cc39b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608516fc790 con 0x5608516fc0d0 2023-09-10T00:52:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e3cc39b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560851701d80 con 0x5608517015b0 2023-09-10T00:52:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e3cc39b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560851712a60 con 0x5608516fb710 2023-09-10T00:52:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e29d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fc0d0 0x56085170aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e2a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 0x5608516f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e29d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fc0d0 0x56085170aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52852/0 (socket says 172.21.15.44:52852) 2023-09-10T00:52:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e2a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 0x5608516f6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59732/0 (socket says 172.21.15.44:59732) 2023-09-10T00:52:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e29d9b700 1 -- 172.21.15.44:0/2672666753 learned_addr learned my addr 172.21.15.44:0/2672666753 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e2ad9d700 1 --2- 172.21.15.44:0/2672666753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 0x56085170aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e29d9b700 1 -- 172.21.15.44:0/2672666753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 msgr2=0x5608516f6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e29d9b700 1 --2- 172.21.15.44:0/2672666753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 0x5608516f6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e29d9b700 1 -- 172.21.15.44:0/2672666753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 msgr2=0x56085170aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e29d9b700 1 --2- 172.21.15.44:0/2672666753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 0x56085170aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e29d9b700 1 -- 172.21.15.44:0/2672666753 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56085171c5c0 con 0x5608516fc0d0 2023-09-10T00:52:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e2a59c700 1 --2- 172.21.15.44:0/2672666753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 0x5608516f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e2ad9d700 1 --2- 172.21.15.44:0/2672666753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 0x56085170aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.805+0000 7f2e29d9b700 1 --2- 172.21.15.44:0/2672666753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fc0d0 0x56085170aab0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f2e20016040 tx=0x7f2e20011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cbd9561125251a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.806+0000 7f2e197fa700 1 -- 172.21.15.44:0/2672666753 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20022040 con 0x5608516fc0d0 2023-09-10T00:52:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.806+0000 7f2e197fa700 1 -- 172.21.15.44:0/2672666753 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e20030030 con 0x5608516fc0d0 2023-09-10T00:52:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.806+0000 7f2e197fa700 1 -- 172.21.15.44:0/2672666753 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20035a80 con 0x5608516fc0d0 2023-09-10T00:52:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.806+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/2672666753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fc0d0 msgr2=0x56085170aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.806+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/2672666753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fc0d0 0x56085170aab0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f2e20016040 tx=0x7f2e20011810 comp rx=0 tx=0).stop 2023-09-10T00:52:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.806+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/2672666753 shutdown_connections 2023-09-10T00:52:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.806+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/2672666753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 0x56085170aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.806+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/2672666753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 0x5608516f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.806+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/2672666753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fc0d0 0x56085170aab0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.806+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/2672666753 >> 172.21.15.44:0/2672666753 conn(0x56085166c030 msgr2=0x5608516f62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.806+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/2672666753 shutdown_connections 2023-09-10T00:52:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.806+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/2672666753 wait complete. 2023-09-10T00:52:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e3cc39b80 1 -- start start 2023-09-10T00:52:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e3cc39b80 1 Processor -- start 2023-09-10T00:52:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e3cc39b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 0x56085175d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e3cc39b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fc0d0 0x56085175da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e3cc39b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 0x5608517623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e3cc39b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560851762930 con 0x5608516fc0d0 2023-09-10T00:52:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e3cc39b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560851762aa0 con 0x5608517015b0 2023-09-10T00:52:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e3cc39b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560851762c60 con 0x5608516fb710 2023-09-10T00:52:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e3cc39b80 10 client.0 _ll_register_callbacks cb 0x7f2e200040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:52:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e29d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fc0d0 0x56085175da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e29d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fc0d0 0x56085175da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52858/0 (socket says 172.21.15.44:52858) 2023-09-10T00:52:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e29d9b700 1 -- 172.21.15.44:0/3766047061 learned_addr learned my addr 172.21.15.44:0/3766047061 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e2a59c700 1 --2- 172.21.15.44:0/3766047061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 0x56085175d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e2ad9d700 1 --2- 172.21.15.44:0/3766047061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 0x5608517623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e29d9b700 1 -- 172.21.15.44:0/3766047061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 msgr2=0x56085175d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e29d9b700 1 --2- 172.21.15.44:0/3766047061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 0x56085175d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e29d9b700 1 -- 172.21.15.44:0/3766047061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 msgr2=0x5608517623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e29d9b700 1 --2- 172.21.15.44:0/3766047061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 0x5608517623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e29d9b700 1 -- 172.21.15.44:0/3766047061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560851475d80 con 0x5608516fc0d0 2023-09-10T00:52:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.807+0000 7f2e2ad9d700 1 --2- 172.21.15.44:0/3766047061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 0x5608517623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.808+0000 7f2e29d9b700 1 --2- 172.21.15.44:0/3766047061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fc0d0 0x56085175da60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2e200094f0 tx=0x7f2e2001d640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f31431ed996f7ab server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.808+0000 7f2e11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:52:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.808+0000 7f2e11ffb700 1 -- 172.21.15.44:0/3766047061 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20022040 con 0x5608516fc0d0 2023-09-10T00:52:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.808+0000 7f2e11ffb700 1 -- 172.21.15.44:0/3766047061 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2e20031080 con 0x5608516fc0d0 2023-09-10T00:52:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.808+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/3766047061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e1c09a480 con 0x5608516fc0d0 2023-09-10T00:52:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.808+0000 7f2e11ffb700 1 -- 172.21.15.44:0/3766047061 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e2001dcb0 con 0x5608516fc0d0 2023-09-10T00:52:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.808+0000 7f2e3cc39b80 10 client.17520 fetch_fsmap learned FSMap version 61 2023-09-10T00:52:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.808+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/3766047061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f2e240115b0 con 0x5608516fc0d0 2023-09-10T00:52:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.808+0000 7f2e11ffb700 1 -- 172.21.15.44:0/3766047061 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e1c09a480 con 0x5608516fc0d0 2023-09-10T00:52:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.808+0000 7f2e11ffb700 1 -- 172.21.15.44:0/3766047061 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f2e200049d0 con 0x5608516fc0d0 2023-09-10T00:52:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.808+0000 7f2e3cc39b80 10 client.17520 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:52:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.808+0000 7f2e3cc39b80 10 client.17520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:52:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.808+0000 7f2e3cc39b80 20 client.17520 populate_metadata read hostname 'smithi044' 2023-09-10T00:52:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.808+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/3766047061 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56085176a170 0x56085176c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.808+0000 7f2e3cc39b80 4 client.17520 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:52:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.808+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/3766047061 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5608516428c0 con 0x56085176a170 2023-09-10T00:52:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.809+0000 7f2e2a59c700 1 --2- 172.21.15.44:0/3766047061 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56085176a170 0x56085176c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.809+0000 7f2e11ffb700 10 client.17520 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.809+0000 7f2e2a59c700 1 --2- 172.21.15.44:0/3766047061 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56085176a170 0x56085176c590 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de321755dc6f3544 server_cookie=f1d80458f2ae96ef in_seq=0 out_seq=0 2023-09-10T00:52:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.824+0000 7f2e11ffb700 1 -- 172.21.15.44:0/3766047061 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5608516428c0 con 0x56085176a170 2023-09-10T00:52:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.824+0000 7f2e11ffb700 10 client.17520 handle_command_reply: tid=0 2023-09-10T00:52:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.824+0000 7f2e3cc39b80 1 client.17520 shutdown 2023-09-10T00:52:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.824+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/3766047061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fc0d0 msgr2=0x56085175da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.824+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/3766047061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fc0d0 0x56085175da60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2e200094f0 tx=0x7f2e2001d640 comp rx=0 tx=0).stop 2023-09-10T00:52:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.824+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/3766047061 shutdown_connections 2023-09-10T00:52:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.824+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/3766047061 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56085176a170 0x56085176c590 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.824+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/3766047061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608517015b0 0x5608517623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.824+0000 7f2e11ffb700 0 client.17520 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.824+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/3766047061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608516fb710 0x56085175d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.824+0000 7f2e3cc39b80 1 --2- 172.21.15.44:0/3766047061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608516fc0d0 0x56085175da60 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.824+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/3766047061 >> 172.21.15.44:0/3766047061 conn(0x56085166c030 msgr2=0x5608516f6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.824+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/3766047061 shutdown_connections 2023-09-10T00:52:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.824+0000 7f2e3cc39b80 1 -- 172.21.15.44:0/3766047061 wait complete. 2023-09-10T00:52:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.824+0000 7f2e3cc39b80 20 client.17520 trim_cache size 0 max 16384 2023-09-10T00:52:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:52:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:52:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:52:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:52:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.825+0000 7f2e30a73700 1 -- 172.21.15.44:0/3879779967 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2e140a07b0 msgr2=0x7f2e140a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.825+0000 7f2e30a73700 1 --2- 172.21.15.44:0/3879779967 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2e140a07b0 0x7f2e140a2c70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c068ad0 tx=0x7f2e20009260 comp rx=0 tx=0).stop 2023-09-10T00:52:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.825+0000 7f2e30a73700 1 -- 172.21.15.44:0/3879779967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e2c0ff760 msgr2=0x7f2e2c197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.825+0000 7f2e30a73700 1 --2- 172.21.15.44:0/3879779967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e2c0ff760 0x7f2e2c197540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c006010 tx=0x7f2e1c004730 comp rx=0 tx=0).stop 2023-09-10T00:52:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.825+0000 7f2e30a73700 1 -- 172.21.15.44:0/3879779967 shutdown_connections 2023-09-10T00:52:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.825+0000 7f2e30a73700 1 --2- 172.21.15.44:0/3879779967 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2e140a07b0 0x7f2e140a2c70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.825+0000 7f2e30a73700 1 --2- 172.21.15.44:0/3879779967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e2c1055b0 0x7f2e2c1a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.825+0000 7f2e30a73700 1 --2- 172.21.15.44:0/3879779967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e2c0ff760 0x7f2e2c197540 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.825+0000 7f2e30a73700 1 --2- 172.21.15.44:0/3879779967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e2c100120 0x7f2e2c197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.825+0000 7f2e30a73700 1 -- 172.21.15.44:0/3879779967 >> 172.21.15.44:0/3879779967 conn(0x7f2e2c074f30 msgr2=0x7f2e2c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.825+0000 7f2e30a73700 1 -- 172.21.15.44:0/3879779967 shutdown_connections 2023-09-10T00:52:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:33.825+0000 7f2e30a73700 1 -- 172.21.15.44:0/3879779967 wait complete. 2023-09-10T00:52:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:34 smithi131 ceph-mon[103713]: pgmap v372: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 16 MiB/s wr, 56 op/s 2023-09-10T00:52:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:34 smithi044 ceph-mon[98676]: pgmap v372: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 16 MiB/s wr, 56 op/s 2023-09-10T00:52:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:34 smithi099 ceph-mon[102692]: pgmap v372: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 16 MiB/s wr, 56 op/s 2023-09-10T00:52:36.955 INFO:teuthology.orchestra.run.smithi044.stderr:45300000 of 50000000 tuples (90%) done (elapsed 696.59 s, remaining 72.27 s) 2023-09-10T00:52:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:36 smithi131 ceph-mon[103713]: pgmap v373: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 20 MiB/s wr, 78 op/s 2023-09-10T00:52:37.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:36 smithi044 ceph-mon[98676]: pgmap v373: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 20 MiB/s wr, 78 op/s 2023-09-10T00:52:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:36 smithi099 ceph-mon[102692]: pgmap v373: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 20 MiB/s wr, 78 op/s 2023-09-10T00:52:37.829 INFO:teuthology.orchestra.run.smithi044.stderr:45400000 of 50000000 tuples (90%) done (elapsed 697.46 s, remaining 70.67 s) 2023-09-10T00:52:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:38 smithi131 ceph-mon[103713]: pgmap v374: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 9.4 MiB/s wr, 51 op/s 2023-09-10T00:52:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:38 smithi044 ceph-mon[98676]: pgmap v374: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 9.4 MiB/s wr, 51 op/s 2023-09-10T00:52:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:38 smithi099 ceph-mon[102692]: pgmap v374: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 9.4 MiB/s wr, 51 op/s 2023-09-10T00:52:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:40 smithi131 ceph-mon[103713]: pgmap v375: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 9.4 MiB/s wr, 51 op/s 2023-09-10T00:52:41.052 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:52:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:40 smithi044 ceph-mon[98676]: pgmap v375: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 9.4 MiB/s wr, 51 op/s 2023-09-10T00:52:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:40 smithi099 ceph-mon[102692]: pgmap v375: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 9.4 MiB/s wr, 51 op/s 2023-09-10T00:52:41.197 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.193+0000 7efee79b2700 1 -- 172.21.15.44:0/3879874067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee0102030 msgr2=0x7efee010a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:41.197 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.193+0000 7efee79b2700 1 --2- 172.21.15.44:0/3879874067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee0102030 0x7efee010a510 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efed0009990 tx=0x7efed0009ca0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.197 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.194+0000 7efee79b2700 1 -- 172.21.15.44:0/3879874067 shutdown_connections 2023-09-10T00:52:41.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.194+0000 7efee79b2700 1 --2- 172.21.15.44:0/3879874067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee010ab30 0x7efee010cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.194+0000 7efee79b2700 1 --2- 172.21.15.44:0/3879874067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee0101710 0x7efee0101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.194+0000 7efee79b2700 1 --2- 172.21.15.44:0/3879874067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee0102030 0x7efee010a510 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.198 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.194+0000 7efee79b2700 1 -- 172.21.15.44:0/3879874067 >> 172.21.15.44:0/3879874067 conn(0x7efee00faf50 msgr2=0x7efee00fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:41.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.194+0000 7efee79b2700 1 -- 172.21.15.44:0/3879874067 shutdown_connections 2023-09-10T00:52:41.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.194+0000 7efee79b2700 1 -- 172.21.15.44:0/3879874067 wait complete. 2023-09-10T00:52:41.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.194+0000 7efee79b2700 1 Processor -- start 2023-09-10T00:52:41.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.194+0000 7efee79b2700 1 -- start start 2023-09-10T00:52:41.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.194+0000 7efee79b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee0101710 0x7efee00ff360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.194+0000 7efee79b2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee0102030 0x7efee00ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.194+0000 7efee79b2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee010ab30 0x7efee01a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efee79b2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee01000d0 con 0x7efee0101710 2023-09-10T00:52:41.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efee79b2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee01a4d00 con 0x7efee0102030 2023-09-10T00:52:41.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efee79b2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee01a4e40 con 0x7efee010ab30 2023-09-10T00:52:41.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efee574e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee0101710 0x7efee00ff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efee4f4d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee0102030 0x7efee00ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efee5f4f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee010ab30 0x7efee01a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efee574e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee0101710 0x7efee00ff360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52870/0 (socket says 172.21.15.44:52870) 2023-09-10T00:52:41.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efee574e700 1 -- 172.21.15.44:0/2491146299 learned_addr learned my addr 172.21.15.44:0/2491146299 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:41.201 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efee5f4f700 1 -- 172.21.15.44:0/2491146299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee0102030 msgr2=0x7efee00ff8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:41.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efee5f4f700 1 --2- 172.21.15.44:0/2491146299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee0102030 0x7efee00ff8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efee5f4f700 1 -- 172.21.15.44:0/2491146299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee0101710 msgr2=0x7efee00ff360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:41.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efee5f4f700 1 --2- 172.21.15.44:0/2491146299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee0101710 0x7efee00ff360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efee5f4f700 1 -- 172.21.15.44:0/2491146299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed00095f0 con 0x7efee010ab30 2023-09-10T00:52:41.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efee4f4d700 1 --2- 172.21.15.44:0/2491146299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee0102030 0x7efee00ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:41.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efee5f4f700 1 --2- 172.21.15.44:0/2491146299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee010ab30 0x7efee01a47c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efed800efc0 tx=0x7efed800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:41.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efed67fc700 1 -- 172.21.15.44:0/2491146299 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8018070 con 0x7efee010ab30 2023-09-10T00:52:41.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efed67fc700 1 -- 172.21.15.44:0/2491146299 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efed800f040 con 0x7efee010ab30 2023-09-10T00:52:41.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efee79b2700 1 -- 172.21.15.44:0/2491146299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee01a5040 con 0x7efee010ab30 2023-09-10T00:52:41.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.195+0000 7efed67fc700 1 -- 172.21.15.44:0/2491146299 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800cba0 con 0x7efee010ab30 2023-09-10T00:52:41.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.196+0000 7efee79b2700 1 -- 172.21.15.44:0/2491146299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee01a5520 con 0x7efee010ab30 2023-09-10T00:52:41.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.198+0000 7efed67fc700 1 -- 172.21.15.44:0/2491146299 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7efed8003a00 con 0x7efee010ab30 2023-09-10T00:52:41.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.199+0000 7efed67fc700 1 --2- 172.21.15.44:0/2491146299 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efecc0a0830 0x7efecc0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.199+0000 7efee574e700 1 --2- 172.21.15.44:0/2491146299 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efecc0a0830 0x7efecc0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.199+0000 7efed67fc700 1 -- 172.21.15.44:0/2491146299 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9540+0+0 (secure 0 0 0) 0x7efed80cbe10 con 0x7efee010ab30 2023-09-10T00:52:41.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.199+0000 7efee574e700 1 --2- 172.21.15.44:0/2491146299 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efecc0a0830 0x7efecc0a2cf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efedc009d20 tx=0x7efedc009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:41.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efef3b78b80 1 Processor -- start 2023-09-10T00:52:41.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efef3b78b80 1 -- start start 2023-09-10T00:52:41.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efef3b78b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73251710 0x55cf73251af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efef3b78b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73252030 0x55cf7325b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efef3b78b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf7325be40 0x55cf7325e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efef3b78b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf731aa730 con 0x55cf73251710 2023-09-10T00:52:41.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efef3b78b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf731a8050 con 0x55cf73252030 2023-09-10T00:52:41.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efef3b78b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf731a3940 con 0x55cf7325be40 2023-09-10T00:52:41.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efee574e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73251710 0x55cf73251af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efee574e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73251710 0x55cf73251af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52876/0 (socket says 172.21.15.44:52876) 2023-09-10T00:52:41.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efee574e700 1 -- 172.21.15.44:0/3274983598 learned_addr learned my addr 172.21.15.44:0/3274983598 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:41.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efee4f4d700 1 --2- 172.21.15.44:0/3274983598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73252030 0x55cf7325b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efee5f4f700 1 --2- 172.21.15.44:0/3274983598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf7325be40 0x55cf7325e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efee574e700 1 -- 172.21.15.44:0/3274983598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf7325be40 msgr2=0x55cf7325e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:41.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efee574e700 1 --2- 172.21.15.44:0/3274983598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf7325be40 0x55cf7325e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efee574e700 1 -- 172.21.15.44:0/3274983598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73252030 msgr2=0x55cf7325b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:41.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efee574e700 1 --2- 172.21.15.44:0/3274983598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73252030 0x55cf7325b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efee574e700 1 -- 172.21.15.44:0/3274983598 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf72fcfde0 con 0x55cf73251710 2023-09-10T00:52:41.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.200+0000 7efee574e700 1 --2- 172.21.15.44:0/3274983598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73251710 0x55cf73251af0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7efedc00e540 tx=0x7efedc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a550aeb412c17a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:41.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.201+0000 7efecb7fe700 1 -- 172.21.15.44:0/3274983598 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc026430 con 0x55cf73251710 2023-09-10T00:52:41.214 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.201+0000 7efecb7fe700 1 -- 172.21.15.44:0/3274983598 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efedc021070 con 0x55cf73251710 2023-09-10T00:52:41.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.201+0000 7efecb7fe700 1 -- 172.21.15.44:0/3274983598 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc01d670 con 0x55cf73251710 2023-09-10T00:52:41.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.201+0000 7efef3b78b80 1 -- 172.21.15.44:0/3274983598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73251710 msgr2=0x55cf73251af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:41.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.201+0000 7efef3b78b80 1 --2- 172.21.15.44:0/3274983598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73251710 0x55cf73251af0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7efedc00e540 tx=0x7efedc014000 comp rx=0 tx=0).stop 2023-09-10T00:52:41.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.201+0000 7efef3b78b80 1 -- 172.21.15.44:0/3274983598 shutdown_connections 2023-09-10T00:52:41.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.201+0000 7efef3b78b80 1 --2- 172.21.15.44:0/3274983598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf7325be40 0x55cf7325e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.201+0000 7efef3b78b80 1 --2- 172.21.15.44:0/3274983598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73251710 0x55cf73251af0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.201+0000 7efef3b78b80 1 --2- 172.21.15.44:0/3274983598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73252030 0x55cf7325b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.201+0000 7efef3b78b80 1 -- 172.21.15.44:0/3274983598 >> 172.21.15.44:0/3274983598 conn(0x55cf731c6090 msgr2=0x55cf7324cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:41.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.201+0000 7efef3b78b80 1 -- 172.21.15.44:0/3274983598 shutdown_connections 2023-09-10T00:52:41.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.201+0000 7efef3b78b80 1 -- 172.21.15.44:0/3274983598 wait complete. 2023-09-10T00:52:41.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.202+0000 7efef3b78b80 1 -- start start 2023-09-10T00:52:41.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.202+0000 7efef3b78b80 1 Processor -- start 2023-09-10T00:52:41.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.202+0000 7efef3b78b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf73251710 0x55cf7325b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.202+0000 7efef3b78b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73252030 0x55cf73258a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.202+0000 7efef3b78b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf7325be40 0x55cf73258fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.202+0000 7efef3b78b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf73141b70 con 0x55cf73252030 2023-09-10T00:52:41.217 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.202+0000 7efef3b78b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf73141870 con 0x55cf7325be40 2023-09-10T00:52:41.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.202+0000 7efef3b78b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf73025920 con 0x55cf73251710 2023-09-10T00:52:41.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.202+0000 7efef3b78b80 10 client.0 _ll_register_callbacks cb 0x55cf7308ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:52:41.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.202+0000 7efee4f4d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73252030 0x55cf73258a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.202+0000 7efee574e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf73251710 0x55cf7325b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.218 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.202+0000 7efee4f4d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73252030 0x55cf73258a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52884/0 (socket says 172.21.15.44:52884) 2023-09-10T00:52:41.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.202+0000 7efee4f4d700 1 -- 172.21.15.44:0/513309872 learned_addr learned my addr 172.21.15.44:0/513309872 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:41.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.202+0000 7efee5f4f700 1 --2- 172.21.15.44:0/513309872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf7325be40 0x55cf73258fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.203+0000 7efee5f4f700 1 -- 172.21.15.44:0/513309872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf73251710 msgr2=0x55cf7325b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:41.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.203+0000 7efee5f4f700 1 --2- 172.21.15.44:0/513309872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf73251710 0x55cf7325b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.203+0000 7efee5f4f700 1 -- 172.21.15.44:0/513309872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73252030 msgr2=0x55cf73258a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:41.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.203+0000 7efee5f4f700 1 --2- 172.21.15.44:0/513309872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73252030 0x55cf73258a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.203+0000 7efee5f4f700 1 -- 172.21.15.44:0/513309872 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efedc017070 con 0x55cf7325be40 2023-09-10T00:52:41.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.203+0000 7efee4f4d700 1 --2- 172.21.15.44:0/513309872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73252030 0x55cf73258a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:41.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.203+0000 7efee5f4f700 1 --2- 172.21.15.44:0/513309872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf7325be40 0x55cf73258fe0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efed80c9100 tx=0x7efed80093a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7017ebb8a4c53ca server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:41.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.203+0000 7efec8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:52:41.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.203+0000 7efec8ff9700 1 -- 172.21.15.44:0/513309872 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8099350 con 0x55cf7325be40 2023-09-10T00:52:41.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.203+0000 7efec8ff9700 1 -- 172.21.15.44:0/513309872 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9540+0+0 (secure 0 0 0) 0x7efed8098ef0 con 0x55cf7325be40 2023-09-10T00:52:41.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.203+0000 7efef3b78b80 1 -- 172.21.15.44:0/513309872 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf730fb4f0 con 0x55cf7325be40 2023-09-10T00:52:41.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.203+0000 7efec8ff9700 1 -- 172.21.15.44:0/513309872 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7efed80b1250 con 0x55cf7325be40 2023-09-10T00:52:41.221 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.203+0000 7efef3b78b80 10 client.27200 fetch_fsmap learned FSMap version 61 2023-09-10T00:52:41.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.203+0000 7efef3b78b80 1 -- 172.21.15.44:0/513309872 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55cf72fb8c60 con 0x55cf7325be40 2023-09-10T00:52:41.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.203+0000 7efec8ff9700 1 -- 172.21.15.44:0/513309872 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cf730fb4f0 con 0x55cf7325be40 2023-09-10T00:52:41.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.204+0000 7efec8ff9700 1 -- 172.21.15.44:0/513309872 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7efed80a0a40 con 0x55cf7325be40 2023-09-10T00:52:41.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.204+0000 7efef3b78b80 10 client.27200 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:52:41.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.204+0000 7efef3b78b80 10 client.27200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:52:41.222 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.204+0000 7efef3b78b80 20 client.27200 populate_metadata read hostname 'smithi044' 2023-09-10T00:52:41.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.204+0000 7efef3b78b80 1 --2- 172.21.15.44:0/513309872 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cf7326b990 0x55cf7326ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.204+0000 7efef3b78b80 4 client.27200 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:52:41.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.204+0000 7efef3b78b80 1 -- 172.21.15.44:0/513309872 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf731b88a0 con 0x55cf7326b990 2023-09-10T00:52:41.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.204+0000 7efee574e700 1 --2- 172.21.15.44:0/513309872 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cf7326b990 0x55cf7326ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.223 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.204+0000 7efee574e700 1 --2- 172.21.15.44:0/513309872 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cf7326b990 0x55cf7326ddb0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d88a5f1466aa6808 server_cookie=2955becc536c38aa in_seq=0 out_seq=0 2023-09-10T00:52:41.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.204+0000 7efec8ff9700 10 client.27200 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:41.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.205+0000 7efec8ff9700 1 -- 172.21.15.44:0/513309872 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cf731b88a0 con 0x55cf7326b990 2023-09-10T00:52:41.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.205+0000 7efec8ff9700 10 client.27200 handle_command_reply: tid=0 2023-09-10T00:52:41.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.205+0000 7efef3b78b80 1 client.27200 shutdown 2023-09-10T00:52:41.224 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.205+0000 7efef3b78b80 1 -- 172.21.15.44:0/513309872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf7325be40 msgr2=0x55cf73258fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:41.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.205+0000 7efef3b78b80 1 --2- 172.21.15.44:0/513309872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf7325be40 0x55cf73258fe0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efed80c9100 tx=0x7efed80093a0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.205+0000 7efef3b78b80 1 -- 172.21.15.44:0/513309872 shutdown_connections 2023-09-10T00:52:41.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.205+0000 7efef3b78b80 1 --2- 172.21.15.44:0/513309872 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cf7326b990 0x55cf7326ddb0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.205+0000 7efef3b78b80 1 --2- 172.21.15.44:0/513309872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf7325be40 0x55cf73258fe0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.205+0000 7efec8ff9700 0 client.27200 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:41.225 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.205+0000 7efef3b78b80 1 --2- 172.21.15.44:0/513309872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf73251710 0x55cf7325b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.205+0000 7efef3b78b80 1 --2- 172.21.15.44:0/513309872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73252030 0x55cf73258a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.205+0000 7efef3b78b80 1 -- 172.21.15.44:0/513309872 >> 172.21.15.44:0/513309872 conn(0x55cf731c6090 msgr2=0x55cf7324c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:41.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.205+0000 7efef3b78b80 1 -- 172.21.15.44:0/513309872 shutdown_connections 2023-09-10T00:52:41.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.205+0000 7efef3b78b80 1 -- 172.21.15.44:0/513309872 wait complete. 2023-09-10T00:52:41.226 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.205+0000 7efef3b78b80 20 client.27200 trim_cache size 0 max 16384 2023-09-10T00:52:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.222+0000 7efef3b78b80 1 Processor -- start 2023-09-10T00:52:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.222+0000 7efef3b78b80 1 -- start start 2023-09-10T00:52:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.222+0000 7efef3b78b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73251710 0x55cf732629f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.222+0000 7efef3b78b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73252030 0x55cf73255cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.222+0000 7efef3b78b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf7325be40 0x55cf732561f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.222+0000 7efef3b78b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf73268f20 con 0x55cf73251710 2023-09-10T00:52:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.222+0000 7efef3b78b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf732529f0 con 0x55cf73252030 2023-09-10T00:52:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.222+0000 7efef3b78b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf732526d0 con 0x55cf7325be40 2023-09-10T00:52:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.223+0000 7efee574e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73251710 0x55cf732629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.223+0000 7efee5f4f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf7325be40 0x55cf732561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.223+0000 7efee574e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73251710 0x55cf732629f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52890/0 (socket says 172.21.15.44:52890) 2023-09-10T00:52:41.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.223+0000 7efee4f4d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73252030 0x55cf73255cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.223+0000 7efee574e700 1 -- 172.21.15.44:0/3045139045 learned_addr learned my addr 172.21.15.44:0/3045139045 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.223+0000 7efee574e700 1 -- 172.21.15.44:0/3045139045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf7325be40 msgr2=0x55cf732561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.223+0000 7efee574e700 1 --2- 172.21.15.44:0/3045139045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf7325be40 0x55cf732561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.223+0000 7efee574e700 1 -- 172.21.15.44:0/3045139045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73252030 msgr2=0x55cf73255cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.223+0000 7efee574e700 1 --2- 172.21.15.44:0/3045139045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73252030 0x55cf73255cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.223+0000 7efee574e700 1 -- 172.21.15.44:0/3045139045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf72ff6250 con 0x55cf73251710 2023-09-10T00:52:41.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.223+0000 7efee574e700 1 --2- 172.21.15.44:0/3045139045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73251710 0x55cf732629f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7efedc00e540 tx=0x7efedc0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7518e05113fc0ac5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:41.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.223+0000 7efecbfff700 1 -- 172.21.15.44:0/3045139045 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc00ece0 con 0x55cf73251710 2023-09-10T00:52:41.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.223+0000 7efecbfff700 1 -- 172.21.15.44:0/3045139045 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efedc00ee40 con 0x55cf73251710 2023-09-10T00:52:41.236 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.223+0000 7efecbfff700 1 -- 172.21.15.44:0/3045139045 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc005280 con 0x55cf73251710 2023-09-10T00:52:41.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.224+0000 7efef3b78b80 1 -- 172.21.15.44:0/3045139045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73251710 msgr2=0x55cf732629f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:41.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.224+0000 7efef3b78b80 1 --2- 172.21.15.44:0/3045139045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73251710 0x55cf732629f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7efedc00e540 tx=0x7efedc0092d0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.224+0000 7efef3b78b80 1 -- 172.21.15.44:0/3045139045 shutdown_connections 2023-09-10T00:52:41.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.224+0000 7efef3b78b80 1 --2- 172.21.15.44:0/3045139045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf7325be40 0x55cf732561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.237 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.224+0000 7efef3b78b80 1 --2- 172.21.15.44:0/3045139045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf73251710 0x55cf732629f0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.224+0000 7efef3b78b80 1 --2- 172.21.15.44:0/3045139045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73252030 0x55cf73255cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.224+0000 7efef3b78b80 1 -- 172.21.15.44:0/3045139045 >> 172.21.15.44:0/3045139045 conn(0x55cf731c6090 msgr2=0x55cf73253d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:41.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.224+0000 7efef3b78b80 1 -- 172.21.15.44:0/3045139045 shutdown_connections 2023-09-10T00:52:41.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.224+0000 7efef3b78b80 1 -- 172.21.15.44:0/3045139045 wait complete. 2023-09-10T00:52:41.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.224+0000 7efef3b78b80 1 -- start start 2023-09-10T00:52:41.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.224+0000 7efef3b78b80 1 Processor -- start 2023-09-10T00:52:41.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efef3b78b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73251710 0x55cf732b8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efef3b78b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf73252030 0x55cf732b9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efef3b78b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf7325be40 0x55cf732b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efef3b78b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf732b3e60 con 0x55cf7325be40 2023-09-10T00:52:41.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efef3b78b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf732b3fd0 con 0x55cf73251710 2023-09-10T00:52:41.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efef3b78b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf732b4190 con 0x55cf73252030 2023-09-10T00:52:41.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efef3b78b80 10 client.0 _ll_register_callbacks cb 0x7efed8098b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:52:41.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efee574e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73251710 0x55cf732b8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efee5f4f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf7325be40 0x55cf732b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efee574e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73251710 0x55cf732b8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54636/0 (socket says 172.21.15.44:54636) 2023-09-10T00:52:41.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efee574e700 1 -- 172.21.15.44:0/1742061479 learned_addr learned my addr 172.21.15.44:0/1742061479 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:41.241 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efee4f4d700 1 --2- 172.21.15.44:0/1742061479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf73252030 0x55cf732b9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.242 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efee5f4f700 1 -- 172.21.15.44:0/1742061479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf73252030 msgr2=0x55cf732b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:41.242 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efee5f4f700 1 --2- 172.21.15.44:0/1742061479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf73252030 0x55cf732b9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.242 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efee5f4f700 1 -- 172.21.15.44:0/1742061479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73251710 msgr2=0x55cf732b8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:41.242 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efee5f4f700 1 --2- 172.21.15.44:0/1742061479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73251710 0x55cf732b8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efee5f4f700 1 -- 172.21.15.44:0/1742061479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efedc02a040 con 0x55cf7325be40 2023-09-10T00:52:41.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efee574e700 1 --2- 172.21.15.44:0/1742061479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73251710 0x55cf732b8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:41.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efee5f4f700 1 --2- 172.21.15.44:0/1742061479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf7325be40 0x55cf732b3750 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7efed8098e80 tx=0x7efed8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc87ecc08d917ef server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:41.243 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efec8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:52:41.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efec8ff9700 1 -- 172.21.15.44:0/1742061479 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed80b1250 con 0x55cf7325be40 2023-09-10T00:52:41.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efec8ff9700 1 -- 172.21.15.44:0/1742061479 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9540+0+0 (secure 0 0 0) 0x7efed8098ef0 con 0x55cf7325be40 2023-09-10T00:52:41.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.225+0000 7efef3b78b80 1 -- 172.21.15.44:0/1742061479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efedc021070 con 0x55cf7325be40 2023-09-10T00:52:41.244 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.226+0000 7efec8ff9700 1 -- 172.21.15.44:0/1742061479 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7efed809cd20 con 0x55cf7325be40 2023-09-10T00:52:41.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.226+0000 7efef3b78b80 10 client.17538 fetch_fsmap learned FSMap version 61 2023-09-10T00:52:41.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.226+0000 7efef3b78b80 1 -- 172.21.15.44:0/1742061479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55cf72fce2b0 con 0x55cf7325be40 2023-09-10T00:52:41.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.226+0000 7efec8ff9700 1 -- 172.21.15.44:0/1742061479 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efed8082020 con 0x55cf7325be40 2023-09-10T00:52:41.245 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.226+0000 7efec8ff9700 1 -- 172.21.15.44:0/1742061479 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7efed8093cc0 con 0x55cf7325be40 2023-09-10T00:52:41.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.226+0000 7efef3b78b80 10 client.17538 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:52:41.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.226+0000 7efef3b78b80 10 client.17538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:52:41.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.226+0000 7efef3b78b80 20 client.17538 populate_metadata read hostname 'smithi044' 2023-09-10T00:52:41.246 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.226+0000 7efef3b78b80 1 --2- 172.21.15.44:0/1742061479 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cf7326b990 0x55cf732c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:41.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.226+0000 7efef3b78b80 4 client.17538 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:52:41.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.226+0000 7efef3b78b80 1 -- 172.21.15.44:0/1742061479 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cf7319c920 con 0x55cf7326b990 2023-09-10T00:52:41.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.226+0000 7efee574e700 1 --2- 172.21.15.44:0/1742061479 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cf7326b990 0x55cf732c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:41.247 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efec8ff9700 10 client.17538 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:41.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efee574e700 1 --2- 172.21.15.44:0/1742061479 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cf7326b990 0x55cf732c2920 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a80f12994ae264cd server_cookie=c9a71740cb36197a in_seq=0 out_seq=0 2023-09-10T00:52:41.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efec8ff9700 1 -- 172.21.15.44:0/1742061479 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cf7319c920 con 0x55cf7326b990 2023-09-10T00:52:41.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efec8ff9700 10 client.17538 handle_command_reply: tid=0 2023-09-10T00:52:41.248 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efef3b78b80 1 client.17538 shutdown 2023-09-10T00:52:41.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efef3b78b80 1 -- 172.21.15.44:0/1742061479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf7325be40 msgr2=0x55cf732b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:41.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efef3b78b80 1 --2- 172.21.15.44:0/1742061479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf7325be40 0x55cf732b3750 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7efed8098e80 tx=0x7efed8009f90 comp rx=0 tx=0).stop 2023-09-10T00:52:41.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efef3b78b80 1 -- 172.21.15.44:0/1742061479 shutdown_connections 2023-09-10T00:52:41.249 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efef3b78b80 1 --2- 172.21.15.44:0/1742061479 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cf7326b990 0x55cf732c2920 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efef3b78b80 1 --2- 172.21.15.44:0/1742061479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf7325be40 0x55cf732b3750 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efef3b78b80 1 --2- 172.21.15.44:0/1742061479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cf73251710 0x55cf732b8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efec8ff9700 0 client.17538 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efef3b78b80 1 --2- 172.21.15.44:0/1742061479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf73252030 0x55cf732b9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efef3b78b80 1 -- 172.21.15.44:0/1742061479 >> 172.21.15.44:0/1742061479 conn(0x55cf731c6090 msgr2=0x55cf7325a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:41.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efef3b78b80 1 -- 172.21.15.44:0/1742061479 shutdown_connections 2023-09-10T00:52:41.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efef3b78b80 1 -- 172.21.15.44:0/1742061479 wait complete. 2023-09-10T00:52:41.251 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.227+0000 7efef3b78b80 20 client.17538 trim_cache size 0 max 16384 2023-09-10T00:52:41.251 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:52:41.252 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:52:41.252 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:52:41.252 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:52:41.252 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:52:41.252 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:52:41.253 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:52:41.253 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:52:41.253 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:52:41.253 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:52:41.254 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.228+0000 7efee79b2700 1 -- 172.21.15.44:0/2491146299 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efecc0a0830 msgr2=0x7efecc0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:41.255 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.228+0000 7efee79b2700 1 --2- 172.21.15.44:0/2491146299 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efecc0a0830 0x7efecc0a2cf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efedc009d20 tx=0x7efedc009260 comp rx=0 tx=0).stop 2023-09-10T00:52:41.255 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.228+0000 7efee79b2700 1 -- 172.21.15.44:0/2491146299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee010ab30 msgr2=0x7efee01a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:41.255 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.228+0000 7efee79b2700 1 --2- 172.21.15.44:0/2491146299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee010ab30 0x7efee01a47c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efed800efc0 tx=0x7efed800c3a0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.255 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.228+0000 7efee79b2700 1 -- 172.21.15.44:0/2491146299 shutdown_connections 2023-09-10T00:52:41.256 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.228+0000 7efee79b2700 1 --2- 172.21.15.44:0/2491146299 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efecc0a0830 0x7efecc0a2cf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.256 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.228+0000 7efee79b2700 1 --2- 172.21.15.44:0/2491146299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee010ab30 0x7efee01a47c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.256 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.228+0000 7efee79b2700 1 --2- 172.21.15.44:0/2491146299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee0101710 0x7efee00ff360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.256 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.228+0000 7efee79b2700 1 --2- 172.21.15.44:0/2491146299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee0102030 0x7efee00ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:41.256 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.228+0000 7efee79b2700 1 -- 172.21.15.44:0/2491146299 >> 172.21.15.44:0/2491146299 conn(0x7efee00faf50 msgr2=0x7efee00fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:41.257 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.228+0000 7efee79b2700 1 -- 172.21.15.44:0/2491146299 shutdown_connections 2023-09-10T00:52:41.257 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:52:41.228+0000 7efee79b2700 1 -- 172.21.15.44:0/2491146299 wait complete. 2023-09-10T00:52:41.257 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:52:42.033 INFO:teuthology.orchestra.run.smithi044.stderr:45500000 of 50000000 tuples (91%) done (elapsed 701.67 s, remaining 69.40 s) 2023-09-10T00:52:42.953 INFO:teuthology.orchestra.run.smithi044.stderr:45600000 of 50000000 tuples (91%) done (elapsed 702.59 s, remaining 67.79 s) 2023-09-10T00:52:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:42 smithi131 ceph-mon[103713]: pgmap v376: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 15 MiB/s wr, 62 op/s 2023-09-10T00:52:43.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:42 smithi044 ceph-mon[98676]: pgmap v376: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 15 MiB/s wr, 62 op/s 2023-09-10T00:52:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:42 smithi099 ceph-mon[102692]: pgmap v376: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 15 MiB/s wr, 62 op/s 2023-09-10T00:52:43.788 INFO:teuthology.orchestra.run.smithi044.stderr:45700000 of 50000000 tuples (91%) done (elapsed 703.42 s, remaining 66.19 s) 2023-09-10T00:52:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:44 smithi131 ceph-mon[103713]: pgmap v377: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.4 MiB/s wr, 33 op/s 2023-09-10T00:52:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:44 smithi044 ceph-mon[98676]: pgmap v377: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.4 MiB/s wr, 33 op/s 2023-09-10T00:52:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:44 smithi099 ceph-mon[102692]: pgmap v377: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.4 MiB/s wr, 33 op/s 2023-09-10T00:52:46.806 INFO:teuthology.orchestra.run.smithi044.stderr:45800000 of 50000000 tuples (91%) done (elapsed 706.44 s, remaining 64.78 s) 2023-09-10T00:52:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.955+0000 7f0f733ca700 1 -- 172.21.15.44:0/2808154390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f6c0679f0 msgr2=0x7f0f6c067e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.955+0000 7f0f733ca700 1 --2- 172.21.15.44:0/2808154390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f6c0679f0 0x7f0f6c067e70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0099c0 tx=0x7f0f5c009cd0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.955+0000 7f0f733ca700 1 -- 172.21.15.44:0/2808154390 shutdown_connections 2023-09-10T00:52:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.955+0000 7f0f733ca700 1 --2- 172.21.15.44:0/2808154390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f6c0683b0 0x7f0f6c10d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.955+0000 7f0f733ca700 1 --2- 172.21.15.44:0/2808154390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f6c106650 0x7f0f6c106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.955+0000 7f0f733ca700 1 --2- 172.21.15.44:0/2808154390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f6c0679f0 0x7f0f6c067e70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.955+0000 7f0f733ca700 1 -- 172.21.15.44:0/2808154390 >> 172.21.15.44:0/2808154390 conn(0x7f0f6c074f30 msgr2=0x7f0f6c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.955+0000 7f0f733ca700 1 -- 172.21.15.44:0/2808154390 shutdown_connections 2023-09-10T00:52:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.955+0000 7f0f733ca700 1 -- 172.21.15.44:0/2808154390 wait complete. 2023-09-10T00:52:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f733ca700 1 Processor -- start 2023-09-10T00:52:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f733ca700 1 -- start start 2023-09-10T00:52:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f733ca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f6c0679f0 0x7f0f6c0ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f733ca700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f6c0683b0 0x7f0f6c0fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f733ca700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f6c106650 0x7f0f6c1a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f733ca700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c1a4e30 con 0x7f0f6c0683b0 2023-09-10T00:52:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f733ca700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c1a4f70 con 0x7f0f6c0679f0 2023-09-10T00:52:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f733ca700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c1a50e0 con 0x7f0f6c106650 2023-09-10T00:52:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f70965700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f6c0683b0 0x7f0f6c0fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f70965700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f6c0683b0 0x7f0f6c0fff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52912/0 (socket says 172.21.15.44:52912) 2023-09-10T00:52:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f70965700 1 -- 172.21.15.44:0/135032573 learned_addr learned my addr 172.21.15.44:0/135032573 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f71967700 1 --2- 172.21.15.44:0/135032573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f6c106650 0x7f0f6c1a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f71166700 1 --2- 172.21.15.44:0/135032573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f6c0679f0 0x7f0f6c0ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f70965700 1 -- 172.21.15.44:0/135032573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f6c106650 msgr2=0x7f0f6c1a48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f70965700 1 --2- 172.21.15.44:0/135032573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f6c106650 0x7f0f6c1a48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f70965700 1 -- 172.21.15.44:0/135032573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f6c0679f0 msgr2=0x7f0f6c0ffa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f70965700 1 --2- 172.21.15.44:0/135032573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f6c0679f0 0x7f0f6c0ffa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f70965700 1 -- 172.21.15.44:0/135032573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f64009c40 con 0x7f0f6c0683b0 2023-09-10T00:52:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.956+0000 7f0f71166700 1 --2- 172.21.15.44:0/135032573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f6c0679f0 0x7f0f6c0ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.957+0000 7f0f70965700 1 --2- 172.21.15.44:0/135032573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f6c0683b0 0x7f0f6c0fff50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c005910 tx=0x7f0f5c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.957+0000 7f0f627fc700 1 -- 172.21.15.44:0/135032573 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c01d070 con 0x7f0f6c0683b0 2023-09-10T00:52:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.957+0000 7f0f627fc700 1 -- 172.21.15.44:0/135032573 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f5c0054e0 con 0x7f0f6c0683b0 2023-09-10T00:52:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.957+0000 7f0f627fc700 1 -- 172.21.15.44:0/135032573 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c004970 con 0x7f0f6c0683b0 2023-09-10T00:52:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.957+0000 7f0f733ca700 1 -- 172.21.15.44:0/135032573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c0095f0 con 0x7f0f6c0683b0 2023-09-10T00:52:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.957+0000 7f0f733ca700 1 -- 172.21.15.44:0/135032573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6c1a55f0 con 0x7f0f6c0683b0 2023-09-10T00:52:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.959+0000 7f0f627fc700 1 -- 172.21.15.44:0/135032573 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f0f5c005050 con 0x7f0f6c0683b0 2023-09-10T00:52:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.960+0000 7f0f627fc700 1 --2- 172.21.15.44:0/135032573 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f580a0830 0x7f0f580a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.960+0000 7f0f627fc700 1 -- 172.21.15.44:0/135032573 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f0f5c0ccfe0 con 0x7f0f6c0683b0 2023-09-10T00:52:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.960+0000 7f0f71166700 1 --2- 172.21.15.44:0/135032573 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f580a0830 0x7f0f580a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.961+0000 7f0f71166700 1 --2- 172.21.15.44:0/135032573 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f580a0830 0x7f0f580a2cf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009fd0 tx=0x7f0f68009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.961+0000 7f0f7f590b80 1 Processor -- start 2023-09-10T00:52:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f7f590b80 1 -- start start 2023-09-10T00:52:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f7f590b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff555a21a0 0x55ff555a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f7f590b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff5559c300 0x55ff5559c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f7f590b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 0x55ff555a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f7f590b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff554f56d0 con 0x55ff5559ccc0 2023-09-10T00:52:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f7f590b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff554f2ff0 con 0x55ff555a21a0 2023-09-10T00:52:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f7f590b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff554ee8e0 con 0x55ff5559c300 2023-09-10T00:52:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f71166700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff555a21a0 0x55ff555a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f71967700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 0x55ff555a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f71166700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff555a21a0 0x55ff555a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54658/0 (socket says 172.21.15.44:54658) 2023-09-10T00:52:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f71166700 1 -- 172.21.15.44:0/928722258 learned_addr learned my addr 172.21.15.44:0/928722258 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f70965700 1 --2- 172.21.15.44:0/928722258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff5559c300 0x55ff5559c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f71166700 1 -- 172.21.15.44:0/928722258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff5559c300 msgr2=0x55ff5559c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f71166700 1 --2- 172.21.15.44:0/928722258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff5559c300 0x55ff5559c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f71166700 1 -- 172.21.15.44:0/928722258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 msgr2=0x55ff555a8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f71166700 1 --2- 172.21.15.44:0/928722258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 0x55ff555a8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f71166700 1 -- 172.21.15.44:0/928722258 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff5531ad80 con 0x55ff555a21a0 2023-09-10T00:52:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.962+0000 7f0f71166700 1 --2- 172.21.15.44:0/928722258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff555a21a0 0x55ff555a2580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800e720 tx=0x7f0f68014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0774eac0fc28a83 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.963+0000 7f0f577fe700 1 -- 172.21.15.44:0/928722258 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68026600 con 0x55ff555a21a0 2023-09-10T00:52:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.963+0000 7f0f577fe700 1 -- 172.21.15.44:0/928722258 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f68026bf0 con 0x55ff555a21a0 2023-09-10T00:52:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.963+0000 7f0f577fe700 1 -- 172.21.15.44:0/928722258 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6801eb50 con 0x55ff555a21a0 2023-09-10T00:52:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.963+0000 7f0f7f590b80 1 -- 172.21.15.44:0/928722258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff555a21a0 msgr2=0x55ff555a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.963+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/928722258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff555a21a0 0x55ff555a2580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800e720 tx=0x7f0f68014000 comp rx=0 tx=0).stop 2023-09-10T00:52:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.963+0000 7f0f7f590b80 1 -- 172.21.15.44:0/928722258 shutdown_connections 2023-09-10T00:52:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.963+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/928722258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 0x55ff555a8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.963+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/928722258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff555a21a0 0x55ff555a2580 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.963+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/928722258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff5559c300 0x55ff5559c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.963+0000 7f0f7f590b80 1 -- 172.21.15.44:0/928722258 >> 172.21.15.44:0/928722258 conn(0x55ff55511030 msgr2=0x55ff55597560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.963+0000 7f0f7f590b80 1 -- 172.21.15.44:0/928722258 shutdown_connections 2023-09-10T00:52:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.964+0000 7f0f7f590b80 1 -- 172.21.15.44:0/928722258 wait complete. 2023-09-10T00:52:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.964+0000 7f0f7f590b80 1 -- start start 2023-09-10T00:52:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.964+0000 7f0f7f590b80 1 Processor -- start 2023-09-10T00:52:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.964+0000 7f0f7f590b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559c300 0x55ff555a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.964+0000 7f0f7f590b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 0x55ff555a5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.964+0000 7f0f7f590b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff555a21a0 0x55ff555a6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.964+0000 7f0f7f590b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff5548cb10 con 0x55ff5559ccc0 2023-09-10T00:52:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.964+0000 7f0f7f590b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff5548c810 con 0x55ff5559c300 2023-09-10T00:52:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.964+0000 7f0f7f590b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff553708c0 con 0x55ff555a21a0 2023-09-10T00:52:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.964+0000 7f0f7f590b80 10 client.0 _ll_register_callbacks cb 0x55ff553d6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:52:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f70965700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 0x55ff555a5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f71166700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559c300 0x55ff555a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f70965700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 0x55ff555a5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52926/0 (socket says 172.21.15.44:52926) 2023-09-10T00:52:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f71166700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559c300 0x55ff555a8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54664/0 (socket says 172.21.15.44:54664) 2023-09-10T00:52:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f70965700 1 -- 172.21.15.44:0/1543982351 learned_addr learned my addr 172.21.15.44:0/1543982351 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f70965700 1 -- 172.21.15.44:0/1543982351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff555a21a0 msgr2=0x55ff555a6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f70965700 1 --2- 172.21.15.44:0/1543982351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff555a21a0 0x55ff555a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f70965700 1 -- 172.21.15.44:0/1543982351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559c300 msgr2=0x55ff555a8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f70965700 1 --2- 172.21.15.44:0/1543982351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559c300 0x55ff555a8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f70965700 1 -- 172.21.15.44:0/1543982351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f68017070 con 0x55ff5559ccc0 2023-09-10T00:52:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f70965700 1 --2- 172.21.15.44:0/1543982351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 0x55ff555a5ff0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c005cd0 tx=0x7f0f5c0a00d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5189a87f035178 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:52:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f54ff9700 1 -- 172.21.15.44:0/1543982351 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c0b1070 con 0x55ff5559ccc0 2023-09-10T00:52:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f54ff9700 1 -- 172.21.15.44:0/1543982351 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f0f5c09a120 con 0x55ff5559ccc0 2023-09-10T00:52:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f7f590b80 1 -- 172.21.15.44:0/1543982351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff55446490 con 0x55ff5559ccc0 2023-09-10T00:52:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f54ff9700 1 -- 172.21.15.44:0/1543982351 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f5c09c070 con 0x55ff5559ccc0 2023-09-10T00:52:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f7f590b80 10 client.17550 fetch_fsmap learned FSMap version 61 2023-09-10T00:52:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.965+0000 7f0f7f590b80 1 -- 172.21.15.44:0/1543982351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55ff55303c00 con 0x55ff5559ccc0 2023-09-10T00:52:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.966+0000 7f0f54ff9700 1 -- 172.21.15.44:0/1543982351 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ff55446490 con 0x55ff5559ccc0 2023-09-10T00:52:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.966+0000 7f0f54ff9700 1 -- 172.21.15.44:0/1543982351 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0f5c09a380 con 0x55ff5559ccc0 2023-09-10T00:52:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.966+0000 7f0f7f590b80 10 client.17550 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:52:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.966+0000 7f0f7f590b80 10 client.17550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:52:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.966+0000 7f0f7f590b80 20 client.17550 populate_metadata read hostname 'smithi044' 2023-09-10T00:52:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.966+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/1543982351 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff555b6060 0x55ff555b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.966+0000 7f0f7f590b80 4 client.17550 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:52:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.966+0000 7f0f7f590b80 1 -- 172.21.15.44:0/1543982351 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff55503840 con 0x55ff555b6060 2023-09-10T00:52:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.966+0000 7f0f71166700 1 --2- 172.21.15.44:0/1543982351 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff555b6060 0x55ff555b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.966+0000 7f0f71166700 1 --2- 172.21.15.44:0/1543982351 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff555b6060 0x55ff555b8480 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad089b786c0e5903 server_cookie=7d529a8cfef89c8d in_seq=0 out_seq=0 2023-09-10T00:52:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.966+0000 7f0f54ff9700 10 client.17550 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.967+0000 7f0f54ff9700 1 -- 172.21.15.44:0/1543982351 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ff55503840 con 0x55ff555b6060 2023-09-10T00:52:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.967+0000 7f0f54ff9700 10 client.17550 handle_command_reply: tid=0 2023-09-10T00:52:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.967+0000 7f0f7f590b80 1 client.17550 shutdown 2023-09-10T00:52:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.967+0000 7f0f7f590b80 1 -- 172.21.15.44:0/1543982351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 msgr2=0x55ff555a5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.967+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/1543982351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 0x55ff555a5ff0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c005cd0 tx=0x7f0f5c0a00d0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.967+0000 7f0f7f590b80 1 -- 172.21.15.44:0/1543982351 shutdown_connections 2023-09-10T00:52:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.967+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/1543982351 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff555b6060 0x55ff555b8480 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.967+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/1543982351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff555a21a0 0x55ff555a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.967+0000 7f0f54ff9700 0 client.17550 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.967+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/1543982351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559c300 0x55ff555a8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.967+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/1543982351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 0x55ff555a5ff0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.967+0000 7f0f7f590b80 1 -- 172.21.15.44:0/1543982351 >> 172.21.15.44:0/1543982351 conn(0x55ff55511030 msgr2=0x55ff555967a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.967+0000 7f0f7f590b80 1 -- 172.21.15.44:0/1543982351 shutdown_connections 2023-09-10T00:52:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.967+0000 7f0f7f590b80 1 -- 172.21.15.44:0/1543982351 wait complete. 2023-09-10T00:52:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.968+0000 7f0f7f590b80 20 client.17550 trim_cache size 0 max 16384 2023-09-10T00:52:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.988+0000 7f0f7f590b80 1 Processor -- start 2023-09-10T00:52:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.988+0000 7f0f7f590b80 1 -- start start 2023-09-10T00:52:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.988+0000 7f0f7f590b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559c300 0x55ff555a8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.988+0000 7f0f7f590b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 0x55ff555a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.988+0000 7f0f7f590b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff555a21a0 0x55ff555a0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.988+0000 7f0f7f590b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff555a2580 con 0x55ff5559ccc0 2023-09-10T00:52:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.988+0000 7f0f7f590b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff555a0bc0 con 0x55ff5559c300 2023-09-10T00:52:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.988+0000 7f0f7f590b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff5559d0a0 con 0x55ff555a21a0 2023-09-10T00:52:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.989+0000 7f0f70965700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 0x55ff555a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.989+0000 7f0f71166700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559c300 0x55ff555a8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.989+0000 7f0f71166700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559c300 0x55ff555a8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54672/0 (socket says 172.21.15.44:54672) 2023-09-10T00:52:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.989+0000 7f0f71166700 1 -- 172.21.15.44:0/600413326 learned_addr learned my addr 172.21.15.44:0/600413326 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.989+0000 7f0f70965700 1 --2- 172.21.15.44:0/600413326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 0x55ff555a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52934/0 (socket says 172.21.15.44:52934) 2023-09-10T00:52:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.989+0000 7f0f71967700 1 --2- 172.21.15.44:0/600413326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff555a21a0 0x55ff555a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.989+0000 7f0f71166700 1 -- 172.21.15.44:0/600413326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff555a21a0 msgr2=0x55ff555a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.989+0000 7f0f71166700 1 --2- 172.21.15.44:0/600413326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff555a21a0 0x55ff555a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.989+0000 7f0f71166700 1 -- 172.21.15.44:0/600413326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 msgr2=0x55ff555a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.989+0000 7f0f71166700 1 --2- 172.21.15.44:0/600413326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 0x55ff555a00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.989+0000 7f0f71166700 1 -- 172.21.15.44:0/600413326 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff553411f0 con 0x55ff5559c300 2023-09-10T00:52:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.989+0000 7f0f70965700 1 --2- 172.21.15.44:0/600413326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 0x55ff555a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.989+0000 7f0f71166700 1 --2- 172.21.15.44:0/600413326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559c300 0x55ff555a8910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800ed10 tx=0x7f0f68005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ad78d670faaf1f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.989+0000 7f0f57fff700 1 -- 172.21.15.44:0/600413326 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6801e3f0 con 0x55ff5559c300 2023-09-10T00:52:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.989+0000 7f0f57fff700 1 -- 172.21.15.44:0/600413326 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f6802b040 con 0x55ff5559c300 2023-09-10T00:52:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.989+0000 7f0f57fff700 1 -- 172.21.15.44:0/600413326 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68005040 con 0x55ff5559c300 2023-09-10T00:52:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.990+0000 7f0f7f590b80 1 -- 172.21.15.44:0/600413326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559c300 msgr2=0x55ff555a8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.990+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/600413326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559c300 0x55ff555a8910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800ed10 tx=0x7f0f68005ce0 comp rx=0 tx=0).stop 2023-09-10T00:52:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.990+0000 7f0f7f590b80 1 -- 172.21.15.44:0/600413326 shutdown_connections 2023-09-10T00:52:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.990+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/600413326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff555a21a0 0x55ff555a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.990+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/600413326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559c300 0x55ff555a8910 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.990+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/600413326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff5559ccc0 0x55ff555a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.990+0000 7f0f7f590b80 1 -- 172.21.15.44:0/600413326 >> 172.21.15.44:0/600413326 conn(0x55ff55511030 msgr2=0x55ff555bcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.990+0000 7f0f7f590b80 1 -- 172.21.15.44:0/600413326 shutdown_connections 2023-09-10T00:52:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.990+0000 7f0f7f590b80 1 -- 172.21.15.44:0/600413326 wait complete. 2023-09-10T00:52:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.991+0000 7f0f7f590b80 1 -- start start 2023-09-10T00:52:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.991+0000 7f0f7f590b80 1 Processor -- start 2023-09-10T00:52:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.991+0000 7f0f7f590b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff5559c300 0x55ff555ab700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.991+0000 7f0f7f590b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559ccc0 0x55ff555abc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.991+0000 7f0f7f590b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff555a21a0 0x55ff55608e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.991+0000 7f0f7f590b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff556093d0 con 0x55ff555a21a0 2023-09-10T00:52:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.991+0000 7f0f7f590b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff55609510 con 0x55ff5559ccc0 2023-09-10T00:52:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.991+0000 7f0f7f590b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff55609650 con 0x55ff5559c300 2023-09-10T00:52:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.991+0000 7f0f7f590b80 10 client.0 _ll_register_callbacks cb 0x7f0f5c099d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:52:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.992+0000 7f0f71166700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff5559c300 0x55ff555ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.992+0000 7f0f71967700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff555a21a0 0x55ff55608e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.992+0000 7f0f71166700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff5559c300 0x55ff555ab700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59820/0 (socket says 172.21.15.44:59820) 2023-09-10T00:52:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.992+0000 7f0f71166700 1 -- 172.21.15.44:0/1808124593 learned_addr learned my addr 172.21.15.44:0/1808124593 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:52:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.992+0000 7f0f70965700 1 --2- 172.21.15.44:0/1808124593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559ccc0 0x55ff555abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.992+0000 7f0f71166700 1 -- 172.21.15.44:0/1808124593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559ccc0 msgr2=0x55ff555abc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.992+0000 7f0f71166700 1 --2- 172.21.15.44:0/1808124593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559ccc0 0x55ff555abc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.992+0000 7f0f71166700 1 -- 172.21.15.44:0/1808124593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff555a21a0 msgr2=0x55ff55608e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:52:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.992+0000 7f0f71166700 1 --2- 172.21.15.44:0/1808124593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff555a21a0 0x55ff55608e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.992+0000 7f0f71166700 1 -- 172.21.15.44:0/1808124593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff55303c00 con 0x55ff5559c300 2023-09-10T00:52:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.992+0000 7f0f71967700 1 --2- 172.21.15.44:0/1808124593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff555a21a0 0x55ff55608e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:52:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.992+0000 7f0f71166700 1 --2- 172.21.15.44:0/1808124593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff5559c300 0x55ff555ab700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009790 tx=0x7f0f68032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f1e32c312b62f97 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:52:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.992+0000 7f0f54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:52:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.993+0000 7f0f54ff9700 1 -- 172.21.15.44:0/1808124593 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68032b90 con 0x55ff5559c300 2023-09-10T00:52:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.993+0000 7f0f54ff9700 1 -- 172.21.15.44:0/1808124593 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f0f68032cf0 con 0x55ff5559c300 2023-09-10T00:52:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.993+0000 7f0f7f590b80 1 -- 172.21.15.44:0/1808124593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f68026bf0 con 0x55ff5559c300 2023-09-10T00:52:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.993+0000 7f0f54ff9700 1 -- 172.21.15.44:0/1808124593 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f680325a0 con 0x55ff5559c300 2023-09-10T00:52:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.993+0000 7f0f7f590b80 10 client.27157 fetch_fsmap learned FSMap version 61 2023-09-10T00:52:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.993+0000 7f0f7f590b80 1 -- 172.21.15.44:0/1808124593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55ff5531ad80 con 0x55ff5559c300 2023-09-10T00:52:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.993+0000 7f0f54ff9700 1 -- 172.21.15.44:0/1808124593 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f68026bf0 con 0x55ff5559c300 2023-09-10T00:52:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.993+0000 7f0f54ff9700 1 -- 172.21.15.44:0/1808124593 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0f6800e750 con 0x55ff5559c300 2023-09-10T00:52:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.993+0000 7f0f7f590b80 10 client.27157 fetch_fsmap finished waiting for FSMap version 61 2023-09-10T00:52:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.993+0000 7f0f7f590b80 10 client.27157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:52:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.993+0000 7f0f7f590b80 20 client.27157 populate_metadata read hostname 'smithi044' 2023-09-10T00:52:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.993+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/1808124593 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff5560a8b0 0x55ff5560ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:52:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.993+0000 7f0f7f590b80 4 client.27157 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:52:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.993+0000 7f0f7f590b80 1 -- 172.21.15.44:0/1808124593 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ff554e78c0 con 0x55ff5560a8b0 2023-09-10T00:52:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.994+0000 7f0f70965700 1 --2- 172.21.15.44:0/1808124593 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff5560a8b0 0x55ff5560ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:52:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.994+0000 7f0f54ff9700 10 client.27157 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:46.994+0000 7f0f70965700 1 --2- 172.21.15.44:0/1808124593 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff5560a8b0 0x55ff5560ccd0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27b857b0d029568f server_cookie=e623b6dd4ce9b115 in_seq=0 out_seq=0 2023-09-10T00:52:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.010+0000 7f0f54ff9700 1 -- 172.21.15.44:0/1808124593 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ff554e78c0 con 0x55ff5560a8b0 2023-09-10T00:52:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.010+0000 7f0f54ff9700 10 client.27157 handle_command_reply: tid=0 2023-09-10T00:52:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.010+0000 7f0f7f590b80 1 client.27157 shutdown 2023-09-10T00:52:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.010+0000 7f0f7f590b80 1 -- 172.21.15.44:0/1808124593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff5559c300 msgr2=0x55ff555ab700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.010+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/1808124593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff5559c300 0x55ff555ab700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009790 tx=0x7f0f68032430 comp rx=0 tx=0).stop 2023-09-10T00:52:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.010+0000 7f0f7f590b80 1 -- 172.21.15.44:0/1808124593 shutdown_connections 2023-09-10T00:52:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.010+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/1808124593 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ff5560a8b0 0x55ff5560ccd0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.010+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/1808124593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ff555a21a0 0x55ff55608e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.010+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/1808124593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff5559c300 0x55ff555ab700 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.010+0000 7f0f7f590b80 1 --2- 172.21.15.44:0/1808124593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff5559ccc0 0x55ff555abc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.010+0000 7f0f54ff9700 0 client.27157 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:52:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.010+0000 7f0f7f590b80 1 -- 172.21.15.44:0/1808124593 >> 172.21.15.44:0/1808124593 conn(0x55ff55511030 msgr2=0x55ff555a00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.010+0000 7f0f7f590b80 1 -- 172.21.15.44:0/1808124593 shutdown_connections 2023-09-10T00:52:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.011+0000 7f0f7f590b80 1 -- 172.21.15.44:0/1808124593 wait complete. 2023-09-10T00:52:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.011+0000 7f0f7f590b80 20 client.27157 trim_cache size 0 max 16384 2023-09-10T00:52:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:52:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:52:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:52:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:52:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.011+0000 7f0f733ca700 1 -- 172.21.15.44:0/135032573 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f580a0830 msgr2=0x7f0f580a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.011+0000 7f0f733ca700 1 --2- 172.21.15.44:0/135032573 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f580a0830 0x7f0f580a2cf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009fd0 tx=0x7f0f68009260 comp rx=0 tx=0).stop 2023-09-10T00:52:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.011+0000 7f0f733ca700 1 -- 172.21.15.44:0/135032573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f6c0683b0 msgr2=0x7f0f6c0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:52:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.011+0000 7f0f733ca700 1 --2- 172.21.15.44:0/135032573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f6c0683b0 0x7f0f6c0fff50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c005910 tx=0x7f0f5c00f690 comp rx=0 tx=0).stop 2023-09-10T00:52:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.011+0000 7f0f733ca700 1 -- 172.21.15.44:0/135032573 shutdown_connections 2023-09-10T00:52:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.011+0000 7f0f733ca700 1 --2- 172.21.15.44:0/135032573 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f580a0830 0x7f0f580a2cf0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.011+0000 7f0f733ca700 1 --2- 172.21.15.44:0/135032573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f6c106650 0x7f0f6c1a48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.011+0000 7f0f733ca700 1 --2- 172.21.15.44:0/135032573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f6c0679f0 0x7f0f6c0ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.011+0000 7f0f733ca700 1 --2- 172.21.15.44:0/135032573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f6c0683b0 0x7f0f6c0fff50 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:52:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.011+0000 7f0f733ca700 1 -- 172.21.15.44:0/135032573 >> 172.21.15.44:0/135032573 conn(0x7f0f6c074f30 msgr2=0x7f0f6c0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:52:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.012+0000 7f0f733ca700 1 -- 172.21.15.44:0/135032573 shutdown_connections 2023-09-10T00:52:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:52:47.012+0000 7f0f733ca700 1 -- 172.21.15.44:0/135032573 wait complete. 2023-09-10T00:52:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:46 smithi131 ceph-mon[103713]: pgmap v378: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 18 MiB/s wr, 48 op/s 2023-09-10T00:52:47.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:46 smithi044 ceph-mon[98676]: pgmap v378: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 18 MiB/s wr, 48 op/s 2023-09-10T00:52:47.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:46 smithi099 ceph-mon[102692]: pgmap v378: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 18 MiB/s wr, 48 op/s 2023-09-10T00:52:47.817 INFO:teuthology.orchestra.run.smithi044.stderr:45900000 of 50000000 tuples (91%) done (elapsed 707.45 s, remaining 63.19 s) 2023-09-10T00:52:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:48 smithi131 ceph-mon[103713]: pgmap v379: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 25 op/s 2023-09-10T00:52:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:48 smithi044 ceph-mon[98676]: pgmap v379: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 25 op/s 2023-09-10T00:52:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:48 smithi099 ceph-mon[102692]: pgmap v379: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 25 op/s 2023-09-10T00:52:49.822 INFO:teuthology.orchestra.run.smithi044.stderr:46000000 of 50000000 tuples (92%) done (elapsed 709.46 s, remaining 61.69 s) 2023-09-10T00:52:50.705 INFO:teuthology.orchestra.run.smithi044.stderr:46100000 of 50000000 tuples (92%) done (elapsed 710.33 s, remaining 60.09 s) 2023-09-10T00:52:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:50 smithi131 ceph-mon[103713]: pgmap v380: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 25 op/s 2023-09-10T00:52:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:50 smithi044 ceph-mon[98676]: pgmap v380: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 25 op/s 2023-09-10T00:52:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:50 smithi099 ceph-mon[102692]: pgmap v380: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 25 op/s 2023-09-10T00:52:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:46200000 of 50000000 tuples (92%) done (elapsed 711.48 s, remaining 58.52 s) 2023-09-10T00:52:52.796 INFO:teuthology.orchestra.run.smithi044.stderr:46300000 of 50000000 tuples (92%) done (elapsed 712.43 s, remaining 56.93 s) 2023-09-10T00:52:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:52 smithi131 ceph-mon[103713]: pgmap v381: 129 pgs: 129 active+clean; 8.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 23 MiB/s wr, 60 op/s 2023-09-10T00:52:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:52 smithi044 ceph-mon[98676]: pgmap v381: 129 pgs: 129 active+clean; 8.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 23 MiB/s wr, 60 op/s 2023-09-10T00:52:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:52 smithi099 ceph-mon[102692]: pgmap v381: 129 pgs: 129 active+clean; 8.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 23 MiB/s wr, 60 op/s 2023-09-10T00:52:53.819 INFO:teuthology.orchestra.run.smithi044.stderr:46400000 of 50000000 tuples (92%) done (elapsed 713.45 s, remaining 55.35 s) 2023-09-10T00:52:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:46500000 of 50000000 tuples (93%) done (elapsed 714.60 s, remaining 53.79 s) 2023-09-10T00:52:55.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:54 smithi131 ceph-mon[103713]: pgmap v382: 129 pgs: 129 active+clean; 8.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 17 MiB/s wr, 48 op/s 2023-09-10T00:52:55.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:54 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:52:55.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:54 smithi044 ceph-mon[98676]: pgmap v382: 129 pgs: 129 active+clean; 8.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 17 MiB/s wr, 48 op/s 2023-09-10T00:52:55.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:54 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:52:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:54 smithi099 ceph-mon[102692]: pgmap v382: 129 pgs: 129 active+clean; 8.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 17 MiB/s wr, 48 op/s 2023-09-10T00:52:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:54 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:52:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:46600000 of 50000000 tuples (93%) done (elapsed 715.56 s, remaining 52.21 s) 2023-09-10T00:52:57.043 INFO:teuthology.orchestra.run.smithi044.stderr:46700000 of 50000000 tuples (93%) done (elapsed 716.68 s, remaining 50.64 s) 2023-09-10T00:52:57.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:56 smithi131 ceph-mon[103713]: pgmap v383: 129 pgs: 129 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 31 MiB/s wr, 102 op/s 2023-09-10T00:52:57.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:56 smithi044 ceph-mon[98676]: pgmap v383: 129 pgs: 129 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 31 MiB/s wr, 102 op/s 2023-09-10T00:52:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:56 smithi099 ceph-mon[102692]: pgmap v383: 129 pgs: 129 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 31 MiB/s wr, 102 op/s 2023-09-10T00:52:58.028 INFO:teuthology.orchestra.run.smithi044.stderr:46800000 of 50000000 tuples (93%) done (elapsed 717.66 s, remaining 49.07 s) 2023-09-10T00:52:59.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:52:58 smithi131 ceph-mon[103713]: pgmap v384: 129 pgs: 129 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 23 MiB/s wr, 88 op/s 2023-09-10T00:52:59.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:52:58 smithi044 ceph-mon[98676]: pgmap v384: 129 pgs: 129 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 23 MiB/s wr, 88 op/s 2023-09-10T00:52:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:52:58 smithi099 ceph-mon[102692]: pgmap v384: 129 pgs: 129 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 23 MiB/s wr, 88 op/s 2023-09-10T00:52:59.142 INFO:teuthology.orchestra.run.smithi044.stderr:46900000 of 50000000 tuples (93%) done (elapsed 718.78 s, remaining 47.51 s) 2023-09-10T00:53:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.135+0000 7f4dee1e9700 1 -- 172.21.15.44:0/4151507598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4de80ff760 msgr2=0x7f4de80ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.135+0000 7f4dee1e9700 1 --2- 172.21.15.44:0/4151507598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4de80ff760 0x7f4de80ffbe0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0099c0 tx=0x7f4ddc009cd0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.135+0000 7f4dee1e9700 1 -- 172.21.15.44:0/4151507598 shutdown_connections 2023-09-10T00:53:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.135+0000 7f4dee1e9700 1 --2- 172.21.15.44:0/4151507598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4de8100120 0x7f4de810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.135+0000 7f4dee1e9700 1 --2- 172.21.15.44:0/4151507598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4de81055b0 0x7f4de8105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.135+0000 7f4dee1e9700 1 --2- 172.21.15.44:0/4151507598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4de80ff760 0x7f4de80ffbe0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.136+0000 7f4dee1e9700 1 -- 172.21.15.44:0/4151507598 >> 172.21.15.44:0/4151507598 conn(0x7f4de8074f30 msgr2=0x7f4de8075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.136+0000 7f4dee1e9700 1 -- 172.21.15.44:0/4151507598 shutdown_connections 2023-09-10T00:53:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.136+0000 7f4dee1e9700 1 -- 172.21.15.44:0/4151507598 wait complete. 2023-09-10T00:53:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.136+0000 7f4dee1e9700 1 Processor -- start 2023-09-10T00:53:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.136+0000 7f4dee1e9700 1 -- start start 2023-09-10T00:53:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4dee1e9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4de80ff760 0x7f4de8194420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4dee1e9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4de8100120 0x7f4de8194960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4dee1e9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4de81055b0 0x7f4de8198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4dee1e9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de8199610 con 0x7f4de80ff760 2023-09-10T00:53:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4dee1e9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de8199780 con 0x7f4de8100120 2023-09-10T00:53:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4dee1e9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de8199940 con 0x7f4de81055b0 2023-09-10T00:53:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4de80ff760 0x7f4de8194420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4de6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4de8100120 0x7f4de8194960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4de80ff760 0x7f4de8194420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52954/0 (socket says 172.21.15.44:52954) 2023-09-10T00:53:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4de77fe700 1 -- 172.21.15.44:0/4016352640 learned_addr learned my addr 172.21.15.44:0/4016352640 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4de7fff700 1 --2- 172.21.15.44:0/4016352640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4de81055b0 0x7f4de8198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4de77fe700 1 -- 172.21.15.44:0/4016352640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4de81055b0 msgr2=0x7f4de8198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4de77fe700 1 --2- 172.21.15.44:0/4016352640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4de81055b0 0x7f4de8198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4de77fe700 1 -- 172.21.15.44:0/4016352640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4de8100120 msgr2=0x7f4de8194960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4de77fe700 1 --2- 172.21.15.44:0/4016352640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4de8100120 0x7f4de8194960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4de77fe700 1 -- 172.21.15.44:0/4016352640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc0095f0 con 0x7f4de80ff760 2023-09-10T00:53:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4de6ffd700 1 --2- 172.21.15.44:0/4016352640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4de8100120 0x7f4de8194960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4de77fe700 1 --2- 172.21.15.44:0/4016352640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4de80ff760 0x7f4de8194420 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009a20 tx=0x7f4dd8009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.137+0000 7f4de4ff9700 1 -- 172.21.15.44:0/4016352640 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8012070 con 0x7f4de80ff760 2023-09-10T00:53:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.138+0000 7f4dee1e9700 1 -- 172.21.15.44:0/4016352640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de8199c20 con 0x7f4de80ff760 2023-09-10T00:53:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.138+0000 7f4de4ff9700 1 -- 172.21.15.44:0/4016352640 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dd800d650 con 0x7f4de80ff760 2023-09-10T00:53:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.138+0000 7f4de4ff9700 1 -- 172.21.15.44:0/4016352640 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8016c30 con 0x7f4de80ff760 2023-09-10T00:53:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.138+0000 7f4dee1e9700 1 -- 172.21.15.44:0/4016352640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de81a1460 con 0x7f4de80ff760 2023-09-10T00:53:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.140+0000 7f4de4ff9700 1 -- 172.21.15.44:0/4016352640 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f4dd800c760 con 0x7f4de80ff760 2023-09-10T00:53:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.140+0000 7f4de4ff9700 1 --2- 172.21.15.44:0/4016352640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4dd00a07b0 0x7f4dd00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.141+0000 7f4de6ffd700 1 --2- 172.21.15.44:0/4016352640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4dd00a07b0 0x7f4dd00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.141+0000 7f4de4ff9700 1 -- 172.21.15.44:0/4016352640 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f4dd80cb6a0 con 0x7f4de80ff760 2023-09-10T00:53:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.141+0000 7f4de6ffd700 1 --2- 172.21.15.44:0/4016352640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4dd00a07b0 0x7f4dd00a2c70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00b580 tx=0x7f4ddc012040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.142+0000 7f4dfa3afb80 1 Processor -- start 2023-09-10T00:53:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.142+0000 7f4dfa3afb80 1 -- start start 2023-09-10T00:53:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.142+0000 7f4dfa3afb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe161a0 0x55f57fe16580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.142+0000 7f4dfa3afb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe10300 0x55f57fe10780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.142+0000 7f4dfa3afb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10cc0 0x55f57fe1cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.142+0000 7f4dfa3afb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57fd696d0 con 0x55f57fe10cc0 2023-09-10T00:53:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.142+0000 7f4dfa3afb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57fd66ff0 con 0x55f57fe10300 2023-09-10T00:53:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.142+0000 7f4dfa3afb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57fd628e0 con 0x55f57fe161a0 2023-09-10T00:53:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.142+0000 7f4de6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe10300 0x55f57fe10780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.142+0000 7f4de7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10cc0 0x55f57fe1cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.142+0000 7f4de6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe10300 0x55f57fe10780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54704/0 (socket says 172.21.15.44:54704) 2023-09-10T00:53:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.142+0000 7f4de7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10cc0 0x55f57fe1cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52966/0 (socket says 172.21.15.44:52966) 2023-09-10T00:53:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.142+0000 7f4de6ffd700 1 -- 172.21.15.44:0/452254865 learned_addr learned my addr 172.21.15.44:0/452254865 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.142+0000 7f4de77fe700 1 --2- 172.21.15.44:0/452254865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe161a0 0x55f57fe16580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.143+0000 7f4de6ffd700 1 -- 172.21.15.44:0/452254865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe161a0 msgr2=0x55f57fe16580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.143+0000 7f4de6ffd700 1 --2- 172.21.15.44:0/452254865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe161a0 0x55f57fe16580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.143+0000 7f4de6ffd700 1 -- 172.21.15.44:0/452254865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10cc0 msgr2=0x55f57fe1cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.143+0000 7f4de6ffd700 1 --2- 172.21.15.44:0/452254865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10cc0 0x55f57fe1cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.143+0000 7f4de6ffd700 1 -- 172.21.15.44:0/452254865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f57fb8ed80 con 0x55f57fe10300 2023-09-10T00:53:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.143+0000 7f4de7fff700 1 --2- 172.21.15.44:0/452254865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10cc0 0x55f57fe1cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.143+0000 7f4de6ffd700 1 --2- 172.21.15.44:0/452254865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe10300 0x55f57fe10780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0099c0 tx=0x7f4ddc023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9092c981678de3c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.143+0000 7f4dcdffb700 1 -- 172.21.15.44:0/452254865 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc00b9d0 con 0x55f57fe10300 2023-09-10T00:53:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.143+0000 7f4dcdffb700 1 -- 172.21.15.44:0/452254865 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ddc00bb30 con 0x55f57fe10300 2023-09-10T00:53:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.143+0000 7f4dcdffb700 1 -- 172.21.15.44:0/452254865 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc0306a0 con 0x55f57fe10300 2023-09-10T00:53:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.144+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/452254865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe10300 msgr2=0x55f57fe10780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.144+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/452254865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe10300 0x55f57fe10780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0099c0 tx=0x7f4ddc023000 comp rx=0 tx=0).stop 2023-09-10T00:53:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.144+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/452254865 shutdown_connections 2023-09-10T00:53:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.144+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/452254865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10cc0 0x55f57fe1cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.144+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/452254865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe161a0 0x55f57fe16580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.144+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/452254865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe10300 0x55f57fe10780 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.144+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/452254865 >> 172.21.15.44:0/452254865 conn(0x55f57fd85030 msgr2=0x55f57fe0b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.144+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/452254865 shutdown_connections 2023-09-10T00:53:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.144+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/452254865 wait complete. 2023-09-10T00:53:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.145+0000 7f4dfa3afb80 1 -- start start 2023-09-10T00:53:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.145+0000 7f4dfa3afb80 1 Processor -- start 2023-09-10T00:53:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.145+0000 7f4dfa3afb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe1cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.145+0000 7f4dfa3afb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 0x55f57fe1a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.145+0000 7f4dfa3afb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 0x55f57fe1a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.145+0000 7f4dfa3afb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57fd00b10 con 0x55f57fe10300 2023-09-10T00:53:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.145+0000 7f4dfa3afb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57fd00810 con 0x55f57fe161a0 2023-09-10T00:53:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.145+0000 7f4dfa3afb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57fbe48c0 con 0x55f57fe10cc0 2023-09-10T00:53:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.145+0000 7f4dfa3afb80 10 client.0 _ll_register_callbacks cb 0x55f57fc4aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:53:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.145+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe1cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.145+0000 7f4de6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 0x55f57fe1a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.145+0000 7f4de6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 0x55f57fe1a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59852/0 (socket says 172.21.15.44:59852) 2023-09-10T00:53:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.145+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe1cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52968/0 (socket says 172.21.15.44:52968) 2023-09-10T00:53:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.145+0000 7f4de6ffd700 1 -- 172.21.15.44:0/1012292906 learned_addr learned my addr 172.21.15.44:0/1012292906 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.146+0000 7f4de7fff700 1 --2- 172.21.15.44:0/1012292906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 0x55f57fe1a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.146+0000 7f4de77fe700 1 -- 172.21.15.44:0/1012292906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 msgr2=0x55f57fe1a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.146+0000 7f4de77fe700 1 --2- 172.21.15.44:0/1012292906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 0x55f57fe1a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.146+0000 7f4de77fe700 1 -- 172.21.15.44:0/1012292906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 msgr2=0x55f57fe1a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.146+0000 7f4de77fe700 1 --2- 172.21.15.44:0/1012292906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 0x55f57fe1a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.146+0000 7f4de77fe700 1 -- 172.21.15.44:0/1012292906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de0011230 con 0x55f57fe10300 2023-09-10T00:53:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.146+0000 7f4de7fff700 1 --2- 172.21.15.44:0/1012292906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 0x55f57fe1a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:53:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.146+0000 7f4de6ffd700 1 --2- 172.21.15.44:0/1012292906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 0x55f57fe1a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.146+0000 7f4de77fe700 1 --2- 172.21.15.44:0/1012292906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe1cdd0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80c89f0 tx=0x7f4dd8098380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74915b5e13c742fc server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.146+0000 7f4dc6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:53:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.146+0000 7f4dc6ffd700 1 -- 172.21.15.44:0/1012292906 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8097e40 con 0x55f57fe10300 2023-09-10T00:53:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.146+0000 7f4dc6ffd700 1 -- 172.21.15.44:0/1012292906 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f4dd8098750 con 0x55f57fe10300 2023-09-10T00:53:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.146+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/1012292906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f57fcba490 con 0x55f57fe10300 2023-09-10T00:53:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.146+0000 7f4dc6ffd700 1 -- 172.21.15.44:0/1012292906 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dd809c380 con 0x55f57fe10300 2023-09-10T00:53:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.146+0000 7f4dfa3afb80 10 client.17568 fetch_fsmap learned FSMap version 62 2023-09-10T00:53:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.146+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/1012292906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x7f4de810cbc0 con 0x55f57fe10300 2023-09-10T00:53:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.147+0000 7f4dc6ffd700 1 -- 172.21.15.44:0/1012292906 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dd807b020 con 0x55f57fe10300 2023-09-10T00:53:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.147+0000 7f4dc6ffd700 1 -- 172.21.15.44:0/1012292906 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 62) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f4dd80cb220 con 0x55f57fe10300 2023-09-10T00:53:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.147+0000 7f4dfa3afb80 10 client.17568 fetch_fsmap finished waiting for FSMap version 62 2023-09-10T00:53:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.147+0000 7f4dfa3afb80 10 client.17568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:53:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.147+0000 7f4dfa3afb80 20 client.17568 populate_metadata read hostname 'smithi044' 2023-09-10T00:53:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.147+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/1012292906 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f57fe2a3f0 0x55f57fe2c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.147+0000 7f4dfa3afb80 4 client.17568 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:53:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.147+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/1012292906 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f57fd77840 con 0x55f57fe2a3f0 2023-09-10T00:53:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.147+0000 7f4de6ffd700 1 --2- 172.21.15.44:0/1012292906 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f57fe2a3f0 0x55f57fe2c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.147+0000 7f4de6ffd700 1 --2- 172.21.15.44:0/1012292906 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f57fe2a3f0 0x55f57fe2c810 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baa269440ed30649 server_cookie=ecf4baf83a6eb8e1 in_seq=0 out_seq=0 2023-09-10T00:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.147+0000 7f4dc6ffd700 10 client.17568 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.148+0000 7f4dc6ffd700 1 -- 172.21.15.44:0/1012292906 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f57fd77840 con 0x55f57fe2a3f0 2023-09-10T00:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.148+0000 7f4dc6ffd700 10 client.17568 handle_command_reply: tid=0 2023-09-10T00:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.148+0000 7f4dfa3afb80 1 client.17568 shutdown 2023-09-10T00:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.148+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/1012292906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 msgr2=0x55f57fe1cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.148+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/1012292906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe1cdd0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80c89f0 tx=0x7f4dd8098380 comp rx=0 tx=0).stop 2023-09-10T00:53:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.148+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/1012292906 shutdown_connections 2023-09-10T00:53:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.148+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/1012292906 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f57fe2a3f0 0x55f57fe2c810 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.148+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/1012292906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 0x55f57fe1a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.148+0000 7f4dc6ffd700 0 client.17568 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.148+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/1012292906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe1cdd0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.148+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/1012292906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 0x55f57fe1a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.148+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/1012292906 >> 172.21.15.44:0/1012292906 conn(0x55f57fd85030 msgr2=0x55f57fe0a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.148+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/1012292906 shutdown_connections 2023-09-10T00:53:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.149+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/1012292906 wait complete. 2023-09-10T00:53:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.149+0000 7f4dfa3afb80 20 client.17568 trim_cache size 0 max 16384 2023-09-10T00:53:00.171 INFO:teuthology.orchestra.run.smithi044.stderr:47000000 of 50000000 tuples (94%) done (elapsed 719.79 s, remaining 45.94 s) 2023-09-10T00:53:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4dfa3afb80 1 Processor -- start 2023-09-10T00:53:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4dfa3afb80 1 -- start start 2023-09-10T00:53:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4dfa3afb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe1c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4dfa3afb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 0x55f57fe19850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4dfa3afb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 0x55f57fe19d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4dfa3afb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57fe27990 con 0x55f57fe10300 2023-09-10T00:53:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4dfa3afb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57fe11580 con 0x55f57fe161a0 2023-09-10T00:53:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4dfa3afb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57fe0cd10 con 0x55f57fe10cc0 2023-09-10T00:53:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe1c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4de7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 0x55f57fe19d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4de6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 0x55f57fe19850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe1c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52976/0 (socket says 172.21.15.44:52976) 2023-09-10T00:53:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4de77fe700 1 -- 172.21.15.44:0/1751062406 learned_addr learned my addr 172.21.15.44:0/1751062406 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4de7fff700 1 -- 172.21.15.44:0/1751062406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 msgr2=0x55f57fe19850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4de7fff700 1 --2- 172.21.15.44:0/1751062406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 0x55f57fe19850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4de7fff700 1 -- 172.21.15.44:0/1751062406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 msgr2=0x55f57fe1c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4de7fff700 1 --2- 172.21.15.44:0/1751062406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe1c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4de7fff700 1 -- 172.21.15.44:0/1751062406 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f57fb77c00 con 0x55f57fe161a0 2023-09-10T00:53:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.165+0000 7f4de77fe700 1 --2- 172.21.15.44:0/1751062406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe1c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.166+0000 7f4de7fff700 1 --2- 172.21.15.44:0/1751062406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 0x55f57fe19d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009490 tx=0x7f4de000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5a60bcdda0d4f99 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.166+0000 7f4dce7fc700 1 -- 172.21.15.44:0/1751062406 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de000e040 con 0x55f57fe161a0 2023-09-10T00:53:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.166+0000 7f4dce7fc700 1 -- 172.21.15.44:0/1751062406 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4de001b070 con 0x55f57fe161a0 2023-09-10T00:53:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.166+0000 7f4dce7fc700 1 -- 172.21.15.44:0/1751062406 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0004880 con 0x55f57fe161a0 2023-09-10T00:53:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.167+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/1751062406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 msgr2=0x55f57fe19d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.167+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/1751062406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 0x55f57fe19d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009490 tx=0x7f4de000b040 comp rx=0 tx=0).stop 2023-09-10T00:53:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.167+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/1751062406 shutdown_connections 2023-09-10T00:53:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.167+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/1751062406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 0x55f57fe19d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.167+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/1751062406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe1c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.167+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/1751062406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 0x55f57fe19850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.167+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/1751062406 >> 172.21.15.44:0/1751062406 conn(0x55f57fd85030 msgr2=0x55f57fe14150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.167+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/1751062406 shutdown_connections 2023-09-10T00:53:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.167+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/1751062406 wait complete. 2023-09-10T00:53:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.167+0000 7f4dfa3afb80 1 -- start start 2023-09-10T00:53:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.167+0000 7f4dfa3afb80 1 Processor -- start 2023-09-10T00:53:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4dfa3afb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe21340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4dfa3afb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 0x55f57fe1f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4dfa3afb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 0x55f57fe1fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4dfa3afb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57fe201d0 con 0x55f57fe10300 2023-09-10T00:53:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4dfa3afb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57fe20340 con 0x55f57fe161a0 2023-09-10T00:53:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4dfa3afb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57fe7e230 con 0x55f57fe10cc0 2023-09-10T00:53:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4dfa3afb80 10 client.0 _ll_register_callbacks cb 0x7f4dd8094b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:53:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe21340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe21340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52982/0 (socket says 172.21.15.44:52982) 2023-09-10T00:53:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4de77fe700 1 -- 172.21.15.44:0/2057352561 learned_addr learned my addr 172.21.15.44:0/2057352561 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4de6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 0x55f57fe1f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4de7fff700 1 --2- 172.21.15.44:0/2057352561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 0x55f57fe1fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4de77fe700 1 -- 172.21.15.44:0/2057352561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 msgr2=0x55f57fe1f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4de77fe700 1 --2- 172.21.15.44:0/2057352561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 0x55f57fe1f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4de77fe700 1 -- 172.21.15.44:0/2057352561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 msgr2=0x55f57fe1fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4de77fe700 1 --2- 172.21.15.44:0/2057352561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 0x55f57fe1fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4de77fe700 1 -- 172.21.15.44:0/2057352561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de810cbc0 con 0x55f57fe10300 2023-09-10T00:53:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4de77fe700 1 --2- 172.21.15.44:0/2057352561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe21340 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80986e0 tx=0x7f4dd8016560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40ec633bbb4fefb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4dc6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:53:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4dc6ffd700 1 -- 172.21.15.44:0/2057352561 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd809c380 con 0x55f57fe10300 2023-09-10T00:53:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4dc6ffd700 1 -- 172.21.15.44:0/2057352561 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f4dd807c080 con 0x55f57fe10300 2023-09-10T00:53:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.168+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/2057352561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ddc00bb30 con 0x55f57fe10300 2023-09-10T00:53:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.169+0000 7f4dc6ffd700 1 -- 172.21.15.44:0/2057352561 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dd8098750 con 0x55f57fe10300 2023-09-10T00:53:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.169+0000 7f4dfa3afb80 10 client.17574 fetch_fsmap learned FSMap version 62 2023-09-10T00:53:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.169+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/2057352561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x7f4de0009170 con 0x55f57fe10300 2023-09-10T00:53:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.169+0000 7f4dc6ffd700 1 -- 172.21.15.44:0/2057352561 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ddc00bb30 con 0x55f57fe10300 2023-09-10T00:53:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.169+0000 7f4dc6ffd700 1 -- 172.21.15.44:0/2057352561 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 62) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f4dd8094540 con 0x55f57fe10300 2023-09-10T00:53:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.169+0000 7f4dfa3afb80 10 client.17574 fetch_fsmap finished waiting for FSMap version 62 2023-09-10T00:53:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.169+0000 7f4dfa3afb80 10 client.17574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:53:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.169+0000 7f4dfa3afb80 20 client.17574 populate_metadata read hostname 'smithi044' 2023-09-10T00:53:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.169+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/2057352561 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f57fe7f420 0x55f57fe81840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.169+0000 7f4dfa3afb80 4 client.17574 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:53:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.169+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/2057352561 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f57fd5b8c0 con 0x55f57fe7f420 2023-09-10T00:53:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.169+0000 7f4de6ffd700 1 --2- 172.21.15.44:0/2057352561 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f57fe7f420 0x55f57fe81840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.170+0000 7f4de6ffd700 1 --2- 172.21.15.44:0/2057352561 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f57fe7f420 0x55f57fe81840 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab461760603eb27 server_cookie=3756e4fd764ca5d7 in_seq=0 out_seq=0 2023-09-10T00:53:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.170+0000 7f4dc6ffd700 10 client.17574 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.185+0000 7f4dc6ffd700 1 -- 172.21.15.44:0/2057352561 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f57fd5b8c0 con 0x55f57fe7f420 2023-09-10T00:53:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.185+0000 7f4dc6ffd700 10 client.17574 handle_command_reply: tid=0 2023-09-10T00:53:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.185+0000 7f4dfa3afb80 1 client.17574 shutdown 2023-09-10T00:53:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.185+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/2057352561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 msgr2=0x55f57fe21340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.185+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/2057352561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe21340 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80986e0 tx=0x7f4dd8016560 comp rx=0 tx=0).stop 2023-09-10T00:53:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.185+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/2057352561 shutdown_connections 2023-09-10T00:53:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.185+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/2057352561 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f57fe7f420 0x55f57fe81840 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.185+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/2057352561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f57fe161a0 0x55f57fe1fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.185+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/2057352561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f57fe10300 0x55f57fe21340 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.185+0000 7f4dc6ffd700 0 client.17574 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.185+0000 7f4dfa3afb80 1 --2- 172.21.15.44:0/2057352561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f57fe10cc0 0x55f57fe1f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.185+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/2057352561 >> 172.21.15.44:0/2057352561 conn(0x55f57fd85030 msgr2=0x55f57fe0f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.185+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/2057352561 shutdown_connections 2023-09-10T00:53:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.185+0000 7f4dfa3afb80 1 -- 172.21.15.44:0/2057352561 wait complete. 2023-09-10T00:53:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.186+0000 7f4dfa3afb80 20 client.17574 trim_cache size 0 max 16384 2023-09-10T00:53:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:53:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:53:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:53:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:53:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.186+0000 7f4dee1e9700 1 -- 172.21.15.44:0/4016352640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4dd00a07b0 msgr2=0x7f4dd00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.186+0000 7f4dee1e9700 1 --2- 172.21.15.44:0/4016352640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4dd00a07b0 0x7f4dd00a2c70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00b580 tx=0x7f4ddc012040 comp rx=0 tx=0).stop 2023-09-10T00:53:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.186+0000 7f4dee1e9700 1 -- 172.21.15.44:0/4016352640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4de80ff760 msgr2=0x7f4de8194420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.186+0000 7f4dee1e9700 1 --2- 172.21.15.44:0/4016352640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4de80ff760 0x7f4de8194420 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009a20 tx=0x7f4dd8009d30 comp rx=0 tx=0).stop 2023-09-10T00:53:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.186+0000 7f4dee1e9700 1 -- 172.21.15.44:0/4016352640 shutdown_connections 2023-09-10T00:53:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.186+0000 7f4dee1e9700 1 --2- 172.21.15.44:0/4016352640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4dd00a07b0 0x7f4dd00a2c70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.186+0000 7f4dee1e9700 1 --2- 172.21.15.44:0/4016352640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4de81055b0 0x7f4de8198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.186+0000 7f4dee1e9700 1 --2- 172.21.15.44:0/4016352640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4de80ff760 0x7f4de8194420 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.186+0000 7f4dee1e9700 1 --2- 172.21.15.44:0/4016352640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4de8100120 0x7f4de8194960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.186+0000 7f4dee1e9700 1 -- 172.21.15.44:0/4016352640 >> 172.21.15.44:0/4016352640 conn(0x7f4de8074f30 msgr2=0x7f4de80fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.186+0000 7f4dee1e9700 1 -- 172.21.15.44:0/4016352640 shutdown_connections 2023-09-10T00:53:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:00.186+0000 7f4dee1e9700 1 -- 172.21.15.44:0/4016352640 wait complete. 2023-09-10T00:53:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:00 smithi131 ceph-mon[103713]: pgmap v385: 129 pgs: 129 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 23 MiB/s wr, 88 op/s 2023-09-10T00:53:01.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:00 smithi044 ceph-mon[98676]: pgmap v385: 129 pgs: 129 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 23 MiB/s wr, 88 op/s 2023-09-10T00:53:01.085 INFO:teuthology.orchestra.run.smithi044.stderr:47100000 of 50000000 tuples (94%) done (elapsed 720.72 s, remaining 44.38 s) 2023-09-10T00:53:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:00 smithi099 ceph-mon[102692]: pgmap v385: 129 pgs: 129 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 23 MiB/s wr, 88 op/s 2023-09-10T00:53:02.171 INFO:teuthology.orchestra.run.smithi044.stderr:47200000 of 50000000 tuples (94%) done (elapsed 721.81 s, remaining 42.82 s) 2023-09-10T00:53:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:02 smithi131 ceph-mon[103713]: pgmap v386: 129 pgs: 129 active+clean; 8.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 37 MiB/s wr, 140 op/s 2023-09-10T00:53:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:02 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:53:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:02 smithi044 ceph-mon[98676]: pgmap v386: 129 pgs: 129 active+clean; 8.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 37 MiB/s wr, 140 op/s 2023-09-10T00:53:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:02 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:53:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:02 smithi099 ceph-mon[102692]: pgmap v386: 129 pgs: 129 active+clean; 8.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 37 MiB/s wr, 140 op/s 2023-09-10T00:53:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:02 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:53:03.259 INFO:teuthology.orchestra.run.smithi044.stderr:47300000 of 50000000 tuples (94%) done (elapsed 722.89 s, remaining 41.26 s) 2023-09-10T00:53:04.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:03 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:03 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:03 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:03 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:03 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:03 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:03 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:53:04.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:03 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:53:04.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:03 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:03 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:03 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:03 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:03 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:03 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:03 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:03 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:53:04.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:03 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:53:04.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:03 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:03 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:03 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:03 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:03 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:03 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:03 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:03 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:53:04.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:03 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:53:04.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:03 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:53:04.456 INFO:teuthology.orchestra.run.smithi044.stderr:47400000 of 50000000 tuples (94%) done (elapsed 724.09 s, remaining 39.72 s) 2023-09-10T00:53:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:04 smithi099 ceph-mon[102692]: pgmap v387: 129 pgs: 129 active+clean; 8.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 28 MiB/s wr, 105 op/s 2023-09-10T00:53:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:04 smithi099 ceph-mon[102692]: osdmap e105: 12 total, 12 up, 12 in 2023-09-10T00:53:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:04 smithi131 ceph-mon[103713]: pgmap v387: 129 pgs: 129 active+clean; 8.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 28 MiB/s wr, 105 op/s 2023-09-10T00:53:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:04 smithi131 ceph-mon[103713]: osdmap e105: 12 total, 12 up, 12 in 2023-09-10T00:53:05.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:04 smithi044 ceph-mon[98676]: pgmap v387: 129 pgs: 129 active+clean; 8.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 28 MiB/s wr, 105 op/s 2023-09-10T00:53:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:04 smithi044 ceph-mon[98676]: osdmap e105: 12 total, 12 up, 12 in 2023-09-10T00:53:05.514 INFO:teuthology.orchestra.run.smithi044.stderr:47500000 of 50000000 tuples (95%) done (elapsed 725.15 s, remaining 38.17 s) 2023-09-10T00:53:06.636 INFO:teuthology.orchestra.run.smithi044.stderr:47600000 of 50000000 tuples (95%) done (elapsed 726.27 s, remaining 36.62 s) 2023-09-10T00:53:07.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:06 smithi099 ceph-mon[102692]: pgmap v389: 129 pgs: 129 active+clean; 8.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 33 MiB/s wr, 134 op/s 2023-09-10T00:53:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:06 smithi131 ceph-mon[103713]: pgmap v389: 129 pgs: 129 active+clean; 8.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 33 MiB/s wr, 134 op/s 2023-09-10T00:53:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:06 smithi044 ceph-mon[98676]: pgmap v389: 129 pgs: 129 active+clean; 8.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 33 MiB/s wr, 134 op/s 2023-09-10T00:53:07.608 INFO:teuthology.orchestra.run.smithi044.stderr:47700000 of 50000000 tuples (95%) done (elapsed 727.24 s, remaining 35.07 s) 2023-09-10T00:53:08.587 INFO:teuthology.orchestra.run.smithi044.stderr:47800000 of 50000000 tuples (95%) done (elapsed 728.22 s, remaining 33.52 s) 2023-09-10T00:53:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:08 smithi099 ceph-mon[102692]: pgmap v390: 129 pgs: 129 active+clean; 8.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 33 MiB/s wr, 134 op/s 2023-09-10T00:53:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:08 smithi131 ceph-mon[103713]: pgmap v390: 129 pgs: 129 active+clean; 8.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 33 MiB/s wr, 134 op/s 2023-09-10T00:53:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:08 smithi044 ceph-mon[98676]: pgmap v390: 129 pgs: 129 active+clean; 8.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 33 MiB/s wr, 134 op/s 2023-09-10T00:53:09.562 INFO:teuthology.orchestra.run.smithi044.stderr:47900000 of 50000000 tuples (95%) done (elapsed 729.20 s, remaining 31.97 s) 2023-09-10T00:53:10.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:09 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:10.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:09 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:09 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:10.740 INFO:teuthology.orchestra.run.smithi044.stderr:48000000 of 50000000 tuples (96%) done (elapsed 730.37 s, remaining 30.43 s) 2023-09-10T00:53:11.259 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:53:11.291 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:10 smithi044 ceph-mon[98676]: pgmap v391: 129 pgs: 129 active+clean; 8.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 33 MiB/s wr, 134 op/s 2023-09-10T00:53:11.292 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:10 smithi044 ceph-mon[98676]: osdmap e106: 12 total, 12 up, 12 in 2023-09-10T00:53:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:10 smithi131 ceph-mon[103713]: pgmap v391: 129 pgs: 129 active+clean; 8.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 33 MiB/s wr, 134 op/s 2023-09-10T00:53:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:10 smithi131 ceph-mon[103713]: osdmap e106: 12 total, 12 up, 12 in 2023-09-10T00:53:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:10 smithi099 ceph-mon[102692]: pgmap v391: 129 pgs: 129 active+clean; 8.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 33 MiB/s wr, 134 op/s 2023-09-10T00:53:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:10 smithi099 ceph-mon[102692]: osdmap e106: 12 total, 12 up, 12 in 2023-09-10T00:53:11.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.420+0000 7f189c89b700 1 -- 172.21.15.44:0/2203391170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1898102070 msgr2=0x7f189810a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.420+0000 7f189c89b700 1 --2- 172.21.15.44:0/2203391170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1898102070 0x7f189810a550 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f188c009990 tx=0x7f188c009ca0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.421+0000 7f189c89b700 1 -- 172.21.15.44:0/2203391170 shutdown_connections 2023-09-10T00:53:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.421+0000 7f189c89b700 1 --2- 172.21.15.44:0/2203391170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f189810ab70 0x7f189810d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.421+0000 7f189c89b700 1 --2- 172.21.15.44:0/2203391170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1898101750 0x7f1898101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.421+0000 7f189c89b700 1 --2- 172.21.15.44:0/2203391170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1898102070 0x7f189810a550 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.421+0000 7f189c89b700 1 -- 172.21.15.44:0/2203391170 >> 172.21.15.44:0/2203391170 conn(0x7f18980faf50 msgr2=0x7f18980fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:11.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.421+0000 7f189c89b700 1 -- 172.21.15.44:0/2203391170 shutdown_connections 2023-09-10T00:53:11.423 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.421+0000 7f189c89b700 1 -- 172.21.15.44:0/2203391170 wait complete. 2023-09-10T00:53:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.421+0000 7f189c89b700 1 Processor -- start 2023-09-10T00:53:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.421+0000 7f189c89b700 1 -- start start 2023-09-10T00:53:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f189c89b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1898101750 0x7f18980ff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f189c89b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1898102070 0x7f18980ff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f189c89b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f189810ab70 0x7f18981a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f189c89b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1898100110 con 0x7f1898101750 2023-09-10T00:53:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f189c89b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18981a4da0 con 0x7f189810ab70 2023-09-10T00:53:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f189c89b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18981a4ee0 con 0x7f1898102070 2023-09-10T00:53:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f189659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1898101750 0x7f18980ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f189659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1898101750 0x7f18980ff3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52996/0 (socket says 172.21.15.44:52996) 2023-09-10T00:53:11.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f189659c700 1 -- 172.21.15.44:0/2914885420 learned_addr learned my addr 172.21.15.44:0/2914885420 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:11.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f1896d9d700 1 --2- 172.21.15.44:0/2914885420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f189810ab70 0x7f18981a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f1895d9b700 1 --2- 172.21.15.44:0/2914885420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1898102070 0x7f18980ff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f1896d9d700 1 -- 172.21.15.44:0/2914885420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1898102070 msgr2=0x7f18980ff8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:11.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f1896d9d700 1 --2- 172.21.15.44:0/2914885420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1898102070 0x7f18980ff8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f1896d9d700 1 -- 172.21.15.44:0/2914885420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1898101750 msgr2=0x7f18980ff3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:11.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f1896d9d700 1 --2- 172.21.15.44:0/2914885420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1898101750 0x7f18980ff3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f1896d9d700 1 -- 172.21.15.44:0/2914885420 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c0095f0 con 0x7f189810ab70 2023-09-10T00:53:11.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f189659c700 1 --2- 172.21.15.44:0/2914885420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1898101750 0x7f18980ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:11.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.422+0000 7f1895d9b700 1 --2- 172.21.15.44:0/2914885420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1898102070 0x7f18980ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:11.430 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.423+0000 7f1896d9d700 1 --2- 172.21.15.44:0/2914885420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f189810ab70 0x7f18981a4860 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f189000efc0 tx=0x7f189000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.423+0000 7f18877fe700 1 -- 172.21.15.44:0/2914885420 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890018070 con 0x7f189810ab70 2023-09-10T00:53:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.423+0000 7f18877fe700 1 -- 172.21.15.44:0/2914885420 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f189000f040 con 0x7f189810ab70 2023-09-10T00:53:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.423+0000 7f189c89b700 1 -- 172.21.15.44:0/2914885420 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18981a50e0 con 0x7f189810ab70 2023-09-10T00:53:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.423+0000 7f18877fe700 1 -- 172.21.15.44:0/2914885420 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189000cba0 con 0x7f189810ab70 2023-09-10T00:53:11.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.423+0000 7f189c89b700 1 -- 172.21.15.44:0/2914885420 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18981a5570 con 0x7f189810ab70 2023-09-10T00:53:11.432 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.425+0000 7f18877fe700 1 -- 172.21.15.44:0/2914885420 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1890003a00 con 0x7f189810ab70 2023-09-10T00:53:11.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.426+0000 7f18877fe700 1 --2- 172.21.15.44:0/2914885420 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f18800a07b0 0x7f18800a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.426+0000 7f189659c700 1 --2- 172.21.15.44:0/2914885420 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f18800a07b0 0x7f18800a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.426+0000 7f18877fe700 1 -- 172.21.15.44:0/2914885420 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f18900cbe40 con 0x7f189810ab70 2023-09-10T00:53:11.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.427+0000 7f18a8a61b80 1 Processor -- start 2023-09-10T00:53:11.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.427+0000 7f18a8a61b80 1 -- start start 2023-09-10T00:53:11.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.427+0000 7f18a8a61b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 0x55d35e76daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.427+0000 7f18a8a61b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 0x55d35e777900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.427+0000 7f18a8a61b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 0x55d35e77a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.427+0000 7f18a8a61b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35e6c6730 con 0x55d35e76e030 2023-09-10T00:53:11.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.427+0000 7f18a8a61b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35e6c4050 con 0x55d35e777e40 2023-09-10T00:53:11.445 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.427+0000 7f18a8a61b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35e6bf940 con 0x55d35e76d710 2023-09-10T00:53:11.445 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.427+0000 7f189659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 0x55d35e76daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.445 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.427+0000 7f1895d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 0x55d35e777900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.445 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.427+0000 7f1896d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 0x55d35e77a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.445 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.427+0000 7f189659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 0x55d35e76daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59886/0 (socket says 172.21.15.44:59886) 2023-09-10T00:53:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.427+0000 7f189659c700 1 -- 172.21.15.44:0/2213056094 learned_addr learned my addr 172.21.15.44:0/2213056094 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.428+0000 7f1896d9d700 1 -- 172.21.15.44:0/2213056094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 msgr2=0x55d35e76daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.428+0000 7f1896d9d700 1 --2- 172.21.15.44:0/2213056094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 0x55d35e76daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.428+0000 7f1896d9d700 1 -- 172.21.15.44:0/2213056094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 msgr2=0x55d35e777900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.428+0000 7f1896d9d700 1 --2- 172.21.15.44:0/2213056094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 0x55d35e777900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.447 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.428+0000 7f1896d9d700 1 -- 172.21.15.44:0/2213056094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d35e4ebde0 con 0x55d35e777e40 2023-09-10T00:53:11.447 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.428+0000 7f1895d9b700 1 --2- 172.21.15.44:0/2213056094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 0x55d35e777900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:11.447 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.428+0000 7f189659c700 1 --2- 172.21.15.44:0/2914885420 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f18800a07b0 0x7f18800a2c70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1888013f60 tx=0x7f1888013560 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:11.447 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.428+0000 7f1896d9d700 1 --2- 172.21.15.44:0/2213056094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 0x55d35e77a230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f18900c9190 tx=0x7f1890098b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d341cfb9be5b474 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:11.447 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.428+0000 7f1884ff9700 1 -- 172.21.15.44:0/2213056094 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189001b320 con 0x55d35e777e40 2023-09-10T00:53:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.428+0000 7f1884ff9700 1 -- 172.21.15.44:0/2213056094 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18900b8040 con 0x55d35e777e40 2023-09-10T00:53:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.428+0000 7f1884ff9700 1 -- 172.21.15.44:0/2213056094 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18900a0f60 con 0x55d35e777e40 2023-09-10T00:53:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.429+0000 7f18a8a61b80 1 -- 172.21.15.44:0/2213056094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 msgr2=0x55d35e77a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.429+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/2213056094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 0x55d35e77a230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f18900c9190 tx=0x7f1890098b40 comp rx=0 tx=0).stop 2023-09-10T00:53:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.429+0000 7f18a8a61b80 1 -- 172.21.15.44:0/2213056094 shutdown_connections 2023-09-10T00:53:11.449 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.429+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/2213056094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 0x55d35e77a230 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.449 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.429+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/2213056094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 0x55d35e76daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.449 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.429+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/2213056094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 0x55d35e777900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.449 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.429+0000 7f18a8a61b80 1 -- 172.21.15.44:0/2213056094 >> 172.21.15.44:0/2213056094 conn(0x55d35e6e2090 msgr2=0x55d35e768bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:11.449 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.429+0000 7f18a8a61b80 1 -- 172.21.15.44:0/2213056094 shutdown_connections 2023-09-10T00:53:11.450 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.429+0000 7f18a8a61b80 1 -- 172.21.15.44:0/2213056094 wait complete. 2023-09-10T00:53:11.450 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.429+0000 7f18a8a61b80 1 -- start start 2023-09-10T00:53:11.450 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f18a8a61b80 1 Processor -- start 2023-09-10T00:53:11.450 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f18a8a61b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 0x55d35e777880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.450 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f18a8a61b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 0x55d35e774b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f18a8a61b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 0x55d35e775240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f18a8a61b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35e65db70 con 0x55d35e76e030 2023-09-10T00:53:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f18a8a61b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35e65d870 con 0x55d35e777e40 2023-09-10T00:53:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f18a8a61b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35e541920 con 0x55d35e76d710 2023-09-10T00:53:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f18a8a61b80 10 client.0 _ll_register_callbacks cb 0x55d35e5a7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:53:11.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f189659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 0x55d35e777880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f1895d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 0x55d35e774b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f189659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 0x55d35e777880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59892/0 (socket says 172.21.15.44:59892) 2023-09-10T00:53:11.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f1895d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 0x55d35e774b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53012/0 (socket says 172.21.15.44:53012) 2023-09-10T00:53:11.452 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f189659c700 1 -- 172.21.15.44:0/829999249 learned_addr learned my addr 172.21.15.44:0/829999249 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:11.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f1896d9d700 1 --2- 172.21.15.44:0/829999249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 0x55d35e775240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f189659c700 1 -- 172.21.15.44:0/829999249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 msgr2=0x55d35e775240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:11.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f189659c700 1 --2- 172.21.15.44:0/829999249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 0x55d35e775240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f189659c700 1 -- 172.21.15.44:0/829999249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 msgr2=0x55d35e774b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:11.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f189659c700 1 --2- 172.21.15.44:0/829999249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 0x55d35e774b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f189659c700 1 -- 172.21.15.44:0/829999249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189810dcf0 con 0x55d35e76d710 2023-09-10T00:53:11.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.430+0000 7f1896d9d700 1 --2- 172.21.15.44:0/829999249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 0x55d35e775240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:11.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.431+0000 7f189659c700 1 --2- 172.21.15.44:0/829999249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 0x55d35e777880 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1888009cf0 tx=0x7f18880047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=529a31bcc807f65 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:11.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.431+0000 7f187dffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:53:11.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.431+0000 7f187dffb700 1 -- 172.21.15.44:0/829999249 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18880132c0 con 0x55d35e76d710 2023-09-10T00:53:11.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.431+0000 7f18a8a61b80 1 -- 172.21.15.44:0/829999249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d35e6174f0 con 0x55d35e76d710 2023-09-10T00:53:11.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.431+0000 7f187dffb700 1 -- 172.21.15.44:0/829999249 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1888011ab0 con 0x55d35e76d710 2023-09-10T00:53:11.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.431+0000 7f187dffb700 1 -- 172.21.15.44:0/829999249 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1888011c70 con 0x55d35e76d710 2023-09-10T00:53:11.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.431+0000 7f18a8a61b80 10 client.27187 fetch_fsmap learned FSMap version 63 2023-09-10T00:53:11.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.431+0000 7f18a8a61b80 1 -- 172.21.15.44:0/829999249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x7f18981a5570 con 0x55d35e76d710 2023-09-10T00:53:11.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.431+0000 7f187dffb700 1 -- 172.21.15.44:0/829999249 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d35e6174f0 con 0x55d35e76d710 2023-09-10T00:53:11.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.431+0000 7f187dffb700 1 -- 172.21.15.44:0/829999249 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 63) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1888021410 con 0x55d35e76d710 2023-09-10T00:53:11.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.431+0000 7f18a8a61b80 10 client.27187 fetch_fsmap finished waiting for FSMap version 63 2023-09-10T00:53:11.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.431+0000 7f18a8a61b80 10 client.27187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:53:11.456 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.431+0000 7f18a8a61b80 20 client.27187 populate_metadata read hostname 'smithi044' 2023-09-10T00:53:11.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.431+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/829999249 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d35e7879e0 0x55d35e789e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.431+0000 7f18a8a61b80 4 client.27187 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:53:11.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.431+0000 7f18a8a61b80 1 -- 172.21.15.44:0/829999249 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d35e6d48a0 con 0x55d35e7879e0 2023-09-10T00:53:11.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.432+0000 7f1895d9b700 1 --2- 172.21.15.44:0/829999249 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d35e7879e0 0x55d35e789e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.457 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.432+0000 7f187dffb700 10 client.27187 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:11.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.432+0000 7f1895d9b700 1 --2- 172.21.15.44:0/829999249 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d35e7879e0 0x55d35e789e00 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4ecf34acd011fa server_cookie=96412e6ac99a9199 in_seq=0 out_seq=0 2023-09-10T00:53:11.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.433+0000 7f187dffb700 1 -- 172.21.15.44:0/829999249 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d35e6d48a0 con 0x55d35e7879e0 2023-09-10T00:53:11.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.433+0000 7f187dffb700 10 client.27187 handle_command_reply: tid=0 2023-09-10T00:53:11.458 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.433+0000 7f18a8a61b80 1 client.27187 shutdown 2023-09-10T00:53:11.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.433+0000 7f18a8a61b80 1 -- 172.21.15.44:0/829999249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 msgr2=0x55d35e777880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:11.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.433+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/829999249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 0x55d35e777880 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1888009cf0 tx=0x7f18880047c0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.433+0000 7f18a8a61b80 1 -- 172.21.15.44:0/829999249 shutdown_connections 2023-09-10T00:53:11.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.433+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/829999249 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d35e7879e0 0x55d35e789e00 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.433+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/829999249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 0x55d35e775240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.459 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.433+0000 7f187dffb700 0 client.27187 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:11.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.433+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/829999249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 0x55d35e777880 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.433+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/829999249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 0x55d35e774b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.433+0000 7f18a8a61b80 1 -- 172.21.15.44:0/829999249 >> 172.21.15.44:0/829999249 conn(0x55d35e6e2090 msgr2=0x55d35e768780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:11.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.433+0000 7f18a8a61b80 1 -- 172.21.15.44:0/829999249 shutdown_connections 2023-09-10T00:53:11.460 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.433+0000 7f18a8a61b80 1 -- 172.21.15.44:0/829999249 wait complete. 2023-09-10T00:53:11.461 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.433+0000 7f18a8a61b80 20 client.27187 trim_cache size 0 max 16384 2023-09-10T00:53:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.455+0000 7f18a8a61b80 1 Processor -- start 2023-09-10T00:53:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.455+0000 7f18a8a61b80 1 -- start start 2023-09-10T00:53:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.455+0000 7f18a8a61b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 0x55d35e77e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.455+0000 7f18a8a61b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 0x55d35e7686f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.455+0000 7f18a8a61b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 0x55d35e768c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.455+0000 7f18a8a61b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35e76a0b0 con 0x55d35e76e030 2023-09-10T00:53:11.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.455+0000 7f18a8a61b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35e785040 con 0x55d35e777e40 2023-09-10T00:53:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.455+0000 7f18a8a61b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35e7760e0 con 0x55d35e76d710 2023-09-10T00:53:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.455+0000 7f1895d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 0x55d35e7686f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.455+0000 7f189659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 0x55d35e77e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.455+0000 7f1895d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 0x55d35e7686f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53020/0 (socket says 172.21.15.44:53020) 2023-09-10T00:53:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.455+0000 7f1895d9b700 1 -- 172.21.15.44:0/3116149024 learned_addr learned my addr 172.21.15.44:0/3116149024 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.455+0000 7f1896d9d700 1 --2- 172.21.15.44:0/3116149024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 0x55d35e768c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.456+0000 7f1896d9d700 1 -- 172.21.15.44:0/3116149024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 msgr2=0x55d35e77e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.456+0000 7f1896d9d700 1 --2- 172.21.15.44:0/3116149024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 0x55d35e77e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.456+0000 7f1896d9d700 1 -- 172.21.15.44:0/3116149024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 msgr2=0x55d35e7686f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.456+0000 7f1896d9d700 1 --2- 172.21.15.44:0/3116149024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 0x55d35e7686f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.456+0000 7f1896d9d700 1 -- 172.21.15.44:0/3116149024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d35e4d4c60 con 0x55d35e777e40 2023-09-10T00:53:11.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.456+0000 7f189659c700 1 --2- 172.21.15.44:0/3116149024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 0x55d35e77e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:11.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.456+0000 7f1896d9d700 1 --2- 172.21.15.44:0/3116149024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 0x55d35e768c30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f18900959f0 tx=0x7f1890098e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a079f0c1c9754e3b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:11.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.456+0000 7f18857fa700 1 -- 172.21.15.44:0/3116149024 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189001b320 con 0x55d35e777e40 2023-09-10T00:53:11.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.456+0000 7f18857fa700 1 -- 172.21.15.44:0/3116149024 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1890099680 con 0x55d35e777e40 2023-09-10T00:53:11.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.456+0000 7f18857fa700 1 -- 172.21.15.44:0/3116149024 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189000ce90 con 0x55d35e777e40 2023-09-10T00:53:11.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.457+0000 7f18a8a61b80 1 -- 172.21.15.44:0/3116149024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 msgr2=0x55d35e768c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:11.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.457+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/3116149024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 0x55d35e768c30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f18900959f0 tx=0x7f1890098e40 comp rx=0 tx=0).stop 2023-09-10T00:53:11.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.457+0000 7f18a8a61b80 1 -- 172.21.15.44:0/3116149024 shutdown_connections 2023-09-10T00:53:11.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.457+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/3116149024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e777e40 0x55d35e768c30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.457+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/3116149024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76d710 0x55d35e77e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.457+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/3116149024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e76e030 0x55d35e7686f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.457+0000 7f18a8a61b80 1 -- 172.21.15.44:0/3116149024 >> 172.21.15.44:0/3116149024 conn(0x55d35e6e2090 msgr2=0x55d35e783140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:11.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.457+0000 7f18a8a61b80 1 -- 172.21.15.44:0/3116149024 shutdown_connections 2023-09-10T00:53:11.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.457+0000 7f18a8a61b80 1 -- 172.21.15.44:0/3116149024 wait complete. 2023-09-10T00:53:11.473 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.457+0000 7f18a8a61b80 1 -- start start 2023-09-10T00:53:11.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.457+0000 7f18a8a61b80 1 Processor -- start 2023-09-10T00:53:11.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f18a8a61b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e76d710 0x55d35e77c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f18a8a61b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76e030 0x55d35e77cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f18a8a61b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e777e40 0x55d35e7da810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.474 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f18a8a61b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35e7dad50 con 0x55d35e777e40 2023-09-10T00:53:11.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f18a8a61b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35e7dae90 con 0x55d35e76d710 2023-09-10T00:53:11.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f18a8a61b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35e7db020 con 0x55d35e76e030 2023-09-10T00:53:11.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f18a8a61b80 10 client.0 _ll_register_callbacks cb 0x7f1888009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:53:11.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f189659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e76d710 0x55d35e77c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f1895d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76e030 0x55d35e77cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.475 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f189659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e76d710 0x55d35e77c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54764/0 (socket says 172.21.15.44:54764) 2023-09-10T00:53:11.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f189659c700 1 -- 172.21.15.44:0/3814033360 learned_addr learned my addr 172.21.15.44:0/3814033360 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:11.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f1895d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76e030 0x55d35e77cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59908/0 (socket says 172.21.15.44:59908) 2023-09-10T00:53:11.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f1896d9d700 1 --2- 172.21.15.44:0/3814033360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e777e40 0x55d35e7da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f189659c700 1 -- 172.21.15.44:0/3814033360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76e030 msgr2=0x55d35e77cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:11.476 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f189659c700 1 --2- 172.21.15.44:0/3814033360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76e030 0x55d35e77cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f189659c700 1 -- 172.21.15.44:0/3814033360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e777e40 msgr2=0x55d35e7da810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:11.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f189659c700 1 --2- 172.21.15.44:0/3814033360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e777e40 0x55d35e7da810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f189659c700 1 -- 172.21.15.44:0/3814033360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18981a5570 con 0x55d35e76d710 2023-09-10T00:53:11.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f1896d9d700 1 --2- 172.21.15.44:0/3814033360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e777e40 0x55d35e7da810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:53:11.477 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f189659c700 1 --2- 172.21.15.44:0/3814033360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e76d710 0x55d35e77c760 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1888009cf0 tx=0x7f1888021b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fded9cd6fdf531c0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:11.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f187dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:53:11.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.458+0000 7f187dffb700 1 -- 172.21.15.44:0/3814033360 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1888013c60 con 0x55d35e76d710 2023-09-10T00:53:11.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.459+0000 7f187dffb700 1 -- 172.21.15.44:0/3814033360 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f18880132c0 con 0x55d35e76d710 2023-09-10T00:53:11.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.459+0000 7f18a8a61b80 1 -- 172.21.15.44:0/3814033360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18900b8040 con 0x55d35e76d710 2023-09-10T00:53:11.478 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.459+0000 7f187dffb700 1 -- 172.21.15.44:0/3814033360 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1888011c10 con 0x55d35e76d710 2023-09-10T00:53:11.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.459+0000 7f18a8a61b80 10 client.27260 fetch_fsmap learned FSMap version 63 2023-09-10T00:53:11.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.459+0000 7f18a8a61b80 1 -- 172.21.15.44:0/3814033360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x55d35e4ebde0 con 0x55d35e76d710 2023-09-10T00:53:11.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.459+0000 7f187dffb700 1 -- 172.21.15.44:0/3814033360 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18900b8040 con 0x55d35e76d710 2023-09-10T00:53:11.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.459+0000 7f187dffb700 1 -- 172.21.15.44:0/3814033360 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 63) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1888005020 con 0x55d35e76d710 2023-09-10T00:53:11.479 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.459+0000 7f18a8a61b80 10 client.27260 fetch_fsmap finished waiting for FSMap version 63 2023-09-10T00:53:11.480 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.459+0000 7f18a8a61b80 10 client.27260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:53:11.480 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.459+0000 7f18a8a61b80 20 client.27260 populate_metadata read hostname 'smithi044' 2023-09-10T00:53:11.480 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.459+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/3814033360 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d35e7879e0 0x55d35e7de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:11.480 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.459+0000 7f18a8a61b80 4 client.27260 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:53:11.480 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.459+0000 7f18a8a61b80 1 -- 172.21.15.44:0/3814033360 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d35e6b8920 con 0x55d35e7879e0 2023-09-10T00:53:11.480 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.459+0000 7f1895d9b700 1 --2- 172.21.15.44:0/3814033360 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d35e7879e0 0x55d35e7de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:11.481 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f1895d9b700 1 --2- 172.21.15.44:0/3814033360 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d35e7879e0 0x55d35e7de760 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa0431966a9a2b98 server_cookie=3af8989341f054ed in_seq=0 out_seq=0 2023-09-10T00:53:11.481 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f187dffb700 10 client.27260 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:11.481 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f187dffb700 1 -- 172.21.15.44:0/3814033360 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d35e6b8920 con 0x55d35e7879e0 2023-09-10T00:53:11.481 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f187dffb700 10 client.27260 handle_command_reply: tid=0 2023-09-10T00:53:11.481 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f18a8a61b80 1 client.27260 shutdown 2023-09-10T00:53:11.482 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f18a8a61b80 1 -- 172.21.15.44:0/3814033360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e76d710 msgr2=0x55d35e77c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:11.482 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/3814033360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e76d710 0x55d35e77c760 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1888009cf0 tx=0x7f1888021b90 comp rx=0 tx=0).stop 2023-09-10T00:53:11.482 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f18a8a61b80 1 -- 172.21.15.44:0/3814033360 shutdown_connections 2023-09-10T00:53:11.482 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/3814033360 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d35e7879e0 0x55d35e7de760 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.482 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/3814033360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d35e777e40 0x55d35e7da810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f187dffb700 0 client.27260 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:11.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/3814033360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d35e76d710 0x55d35e77c760 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f18a8a61b80 1 --2- 172.21.15.44:0/3814033360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d35e76e030 0x55d35e77cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.483 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f18a8a61b80 1 -- 172.21.15.44:0/3814033360 >> 172.21.15.44:0/3814033360 conn(0x55d35e6e2090 msgr2=0x55d35e76fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:11.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f18a8a61b80 1 -- 172.21.15.44:0/3814033360 shutdown_connections 2023-09-10T00:53:11.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f18a8a61b80 1 -- 172.21.15.44:0/3814033360 wait complete. 2023-09-10T00:53:11.484 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.460+0000 7f18a8a61b80 20 client.27260 trim_cache size 0 max 16384 2023-09-10T00:53:11.484 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:53:11.485 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:53:11.485 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:53:11.485 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:53:11.485 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:53:11.485 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:53:11.486 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:53:11.486 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:53:11.486 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:53:11.486 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:53:11.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.461+0000 7f189c89b700 1 -- 172.21.15.44:0/2914885420 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f18800a07b0 msgr2=0x7f18800a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:11.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.461+0000 7f189c89b700 1 --2- 172.21.15.44:0/2914885420 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f18800a07b0 0x7f18800a2c70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1888013f60 tx=0x7f1888013560 comp rx=0 tx=0).stop 2023-09-10T00:53:11.488 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.461+0000 7f189c89b700 1 -- 172.21.15.44:0/2914885420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f189810ab70 msgr2=0x7f18981a4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:11.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.461+0000 7f189c89b700 1 --2- 172.21.15.44:0/2914885420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f189810ab70 0x7f18981a4860 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f189000efc0 tx=0x7f189000c3a0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.461+0000 7f189c89b700 1 -- 172.21.15.44:0/2914885420 shutdown_connections 2023-09-10T00:53:11.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.461+0000 7f189c89b700 1 --2- 172.21.15.44:0/2914885420 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f18800a07b0 0x7f18800a2c70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.461+0000 7f189c89b700 1 --2- 172.21.15.44:0/2914885420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f189810ab70 0x7f18981a4860 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.489 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.461+0000 7f189c89b700 1 --2- 172.21.15.44:0/2914885420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1898101750 0x7f18980ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.461+0000 7f189c89b700 1 --2- 172.21.15.44:0/2914885420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1898102070 0x7f18980ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:11.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.461+0000 7f189c89b700 1 -- 172.21.15.44:0/2914885420 >> 172.21.15.44:0/2914885420 conn(0x7f18980faf50 msgr2=0x7f189810ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:11.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.461+0000 7f189c89b700 1 -- 172.21.15.44:0/2914885420 shutdown_connections 2023-09-10T00:53:11.490 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:11.461+0000 7f189c89b700 1 -- 172.21.15.44:0/2914885420 wait complete. 2023-09-10T00:53:11.491 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:53:11.699 INFO:teuthology.orchestra.run.smithi044.stderr:48100000 of 50000000 tuples (96%) done (elapsed 731.33 s, remaining 28.89 s) 2023-09-10T00:53:12.679 INFO:teuthology.orchestra.run.smithi044.stderr:48200000 of 50000000 tuples (96%) done (elapsed 732.31 s, remaining 27.35 s) 2023-09-10T00:53:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:12 smithi131 ceph-mon[103713]: pgmap v393: 129 pgs: 129 active+clean; 8.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 43 MiB/s wr, 173 op/s 2023-09-10T00:53:13.310 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:12 smithi044 ceph-mon[98676]: pgmap v393: 129 pgs: 129 active+clean; 8.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 43 MiB/s wr, 173 op/s 2023-09-10T00:53:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.333+0000 7f8caab60700 1 -- 172.21.15.44:0/1238131708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca41055b0 msgr2=0x7f8ca4105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.333+0000 7f8caab60700 1 --2- 172.21.15.44:0/1238131708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca41055b0 0x7f8ca4105990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8c900099c0 tx=0x7f8c90009cd0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.333+0000 7f8caab60700 1 -- 172.21.15.44:0/1238131708 shutdown_connections 2023-09-10T00:53:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.333+0000 7f8caab60700 1 --2- 172.21.15.44:0/1238131708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca4100120 0x7f8ca410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.333+0000 7f8caab60700 1 --2- 172.21.15.44:0/1238131708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca41055b0 0x7f8ca4105990 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.333+0000 7f8caab60700 1 --2- 172.21.15.44:0/1238131708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ca40ff760 0x7f8ca40ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.333+0000 7f8caab60700 1 -- 172.21.15.44:0/1238131708 >> 172.21.15.44:0/1238131708 conn(0x7f8ca4074f30 msgr2=0x7f8ca4075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.333+0000 7f8caab60700 1 -- 172.21.15.44:0/1238131708 shutdown_connections 2023-09-10T00:53:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.333+0000 7f8caab60700 1 -- 172.21.15.44:0/1238131708 wait complete. 2023-09-10T00:53:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.335+0000 7f8caab60700 1 Processor -- start 2023-09-10T00:53:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.335+0000 7f8caab60700 1 -- start start 2023-09-10T00:53:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.335+0000 7f8caab60700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ca40ff760 0x7f8ca410cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.335+0000 7f8caab60700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca4100120 0x7f8ca410d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.335+0000 7f8caab60700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca41055b0 0x7f8ca410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.335+0000 7f8caab60700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca41a8f90 con 0x7f8ca4100120 2023-09-10T00:53:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.335+0000 7f8caab60700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca419a6b0 con 0x7f8ca41055b0 2023-09-10T00:53:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.335+0000 7f8caab60700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca419a870 con 0x7f8ca40ff760 2023-09-10T00:53:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.336+0000 7f8ca88fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ca40ff760 0x7f8ca410cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.336+0000 7f8ca90fd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca41055b0 0x7f8ca410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.336+0000 7f8ca88fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ca40ff760 0x7f8ca410cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59920/0 (socket says 172.21.15.44:59920) 2023-09-10T00:53:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.336+0000 7f8ca88fc700 1 -- 172.21.15.44:0/4208497562 learned_addr learned my addr 172.21.15.44:0/4208497562 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.336+0000 7f8ca3fff700 1 --2- 172.21.15.44:0/4208497562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca4100120 0x7f8ca410d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.336+0000 7f8ca88fc700 1 -- 172.21.15.44:0/4208497562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca41055b0 msgr2=0x7f8ca410d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.336+0000 7f8ca88fc700 1 --2- 172.21.15.44:0/4208497562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca41055b0 0x7f8ca410d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.336+0000 7f8ca88fc700 1 -- 172.21.15.44:0/4208497562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca4100120 msgr2=0x7f8ca410d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:13.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.336+0000 7f8ca88fc700 1 --2- 172.21.15.44:0/4208497562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca4100120 0x7f8ca410d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.336+0000 7f8ca88fc700 1 -- 172.21.15.44:0/4208497562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c900095f0 con 0x7f8ca40ff760 2023-09-10T00:53:13.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.336+0000 7f8ca88fc700 1 --2- 172.21.15.44:0/4208497562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ca40ff760 0x7f8ca410cee0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8c90005910 tx=0x7f8c9000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.336+0000 7f8ca1ffb700 1 -- 172.21.15.44:0/4208497562 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9001d070 con 0x7f8ca40ff760 2023-09-10T00:53:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.337+0000 7f8caab60700 1 -- 172.21.15.44:0/4208497562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca419aaf0 con 0x7f8ca40ff760 2023-09-10T00:53:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.337+0000 7f8ca1ffb700 1 -- 172.21.15.44:0/4208497562 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c90005050 con 0x7f8ca40ff760 2023-09-10T00:53:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.337+0000 7f8ca1ffb700 1 -- 172.21.15.44:0/4208497562 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c900049b0 con 0x7f8ca40ff760 2023-09-10T00:53:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.337+0000 7f8caab60700 1 -- 172.21.15.44:0/4208497562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca419aff0 con 0x7f8ca40ff760 2023-09-10T00:53:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.341+0000 7f8ca1ffb700 1 -- 172.21.15.44:0/4208497562 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f8c900054c0 con 0x7f8ca40ff760 2023-09-10T00:53:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.341+0000 7f8cb6d26b80 1 Processor -- start 2023-09-10T00:53:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.341+0000 7f8cb6d26b80 1 -- start start 2023-09-10T00:53:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.341+0000 7f8cb6d26b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60db31a0 0x560c60db3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.341+0000 7f8cb6d26b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60dad300 0x560c60dad780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.341+0000 7f8cb6d26b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dadcc0 0x560c60db9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.341+0000 7f8cb6d26b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c60d066d0 con 0x560c60db31a0 2023-09-10T00:53:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.341+0000 7f8cb6d26b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c60d03ff0 con 0x560c60dad300 2023-09-10T00:53:13.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.341+0000 7f8cb6d26b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c60cff8e0 con 0x560c60dadcc0 2023-09-10T00:53:13.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8ca90fd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60db31a0 0x560c60db3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8ca88fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60dad300 0x560c60dad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8ca90fd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60db31a0 0x560c60db3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53044/0 (socket says 172.21.15.44:53044) 2023-09-10T00:53:13.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8ca90fd700 1 -- 172.21.15.44:0/678726675 learned_addr learned my addr 172.21.15.44:0/678726675 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8ca1ffb700 1 --2- 172.21.15.44:0/4208497562 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8c940a07b0 0x7f8c940a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8ca3fff700 1 --2- 172.21.15.44:0/678726675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dadcc0 0x560c60db9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8ca90fd700 1 --2- 172.21.15.44:0/4208497562 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8c940a07b0 0x7f8c940a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8ca1ffb700 1 -- 172.21.15.44:0/4208497562 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8c900cd0a0 con 0x7f8ca40ff760 2023-09-10T00:53:13.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8ca88fc700 1 -- 172.21.15.44:0/678726675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dadcc0 msgr2=0x560c60db9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:13.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8ca88fc700 1 --2- 172.21.15.44:0/678726675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dadcc0 0x560c60db9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8ca88fc700 1 -- 172.21.15.44:0/678726675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60db31a0 msgr2=0x560c60db3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:13.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8ca88fc700 1 --2- 172.21.15.44:0/678726675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60db31a0 0x560c60db3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8ca88fc700 1 -- 172.21.15.44:0/678726675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c60b2bd80 con 0x560c60dad300 2023-09-10T00:53:13.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8ca90fd700 1 --2- 172.21.15.44:0/678726675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60db31a0 0x560c60db3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:13.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8ca88fc700 1 --2- 172.21.15.44:0/678726675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60dad300 0x560c60dad780 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8c900ca3f0 tx=0x7f8c90009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd484b7bb58a1bc9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:13.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8ca90fd700 1 --2- 172.21.15.44:0/4208497562 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8c940a07b0 0x7f8c940a2c70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c003fd0 tx=0x7f8c9c004b40 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:13.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8c8effd700 1 -- 172.21.15.44:0/678726675 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90099b40 con 0x560c60dad300 2023-09-10T00:53:13.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8c8effd700 1 -- 172.21.15.44:0/678726675 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c900a0110 con 0x560c60dad300 2023-09-10T00:53:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.342+0000 7f8c8effd700 1 -- 172.21.15.44:0/678726675 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9009a6a0 con 0x560c60dad300 2023-09-10T00:53:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.343+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/678726675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60dad300 msgr2=0x560c60dad780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.343+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/678726675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60dad300 0x560c60dad780 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8c900ca3f0 tx=0x7f8c90009310 comp rx=0 tx=0).stop 2023-09-10T00:53:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.343+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/678726675 shutdown_connections 2023-09-10T00:53:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.343+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/678726675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dadcc0 0x560c60db9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.343+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/678726675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60db31a0 0x560c60db3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.343+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/678726675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60dad300 0x560c60dad780 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.343+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/678726675 >> 172.21.15.44:0/678726675 conn(0x560c60d22030 msgr2=0x560c60da8560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.343+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/678726675 shutdown_connections 2023-09-10T00:53:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.343+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/678726675 wait complete. 2023-09-10T00:53:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.344+0000 7f8cb6d26b80 1 -- start start 2023-09-10T00:53:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.344+0000 7f8cb6d26b80 1 Processor -- start 2023-09-10T00:53:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.344+0000 7f8cb6d26b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60dad300 0x560c60dbcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.344+0000 7f8cb6d26b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60dadcc0 0x560c60dbd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.344+0000 7f8cb6d26b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60db31a0 0x560c60db6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.344+0000 7f8cb6d26b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c60c9db10 con 0x560c60dadcc0 2023-09-10T00:53:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.344+0000 7f8cb6d26b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c60c9d810 con 0x560c60dad300 2023-09-10T00:53:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.344+0000 7f8cb6d26b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c60b818c0 con 0x560c60db31a0 2023-09-10T00:53:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.344+0000 7f8cb6d26b80 10 client.0 _ll_register_callbacks cb 0x560c60be7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:53:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.344+0000 7f8ca3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60dadcc0 0x560c60dbd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8ca88fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60dad300 0x560c60dbcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8ca3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60dadcc0 0x560c60dbd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53054/0 (socket says 172.21.15.44:53054) 2023-09-10T00:53:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8ca3fff700 1 -- 172.21.15.44:0/1816384669 learned_addr learned my addr 172.21.15.44:0/1816384669 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8ca90fd700 1 --2- 172.21.15.44:0/1816384669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60db31a0 0x560c60db6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8ca3fff700 1 -- 172.21.15.44:0/1816384669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60db31a0 msgr2=0x560c60db6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8ca3fff700 1 --2- 172.21.15.44:0/1816384669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60db31a0 0x560c60db6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8ca3fff700 1 -- 172.21.15.44:0/1816384669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60dad300 msgr2=0x560c60dbcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8ca3fff700 1 --2- 172.21.15.44:0/1816384669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60dad300 0x560c60dbcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8ca3fff700 1 -- 172.21.15.44:0/1816384669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9c009c00 con 0x560c60dadcc0 2023-09-10T00:53:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8ca88fc700 1 --2- 172.21.15.44:0/1816384669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60dad300 0x560c60dbcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8ca90fd700 1 --2- 172.21.15.44:0/1816384669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60db31a0 0x560c60db6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8ca3fff700 1 --2- 172.21.15.44:0/1816384669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60dadcc0 0x560c60dbd000 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8c98011730 tx=0x7f8c98011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf389505dac3ee5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8c83fff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:53:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8c83fff700 1 -- 172.21.15.44:0/1816384669 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98016070 con 0x560c60dadcc0 2023-09-10T00:53:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/1816384669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c60c57490 con 0x560c60dadcc0 2023-09-10T00:53:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8c83fff700 1 -- 172.21.15.44:0/1816384669 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8c9800f690 con 0x560c60dadcc0 2023-09-10T00:53:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8c83fff700 1 -- 172.21.15.44:0/1816384669 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c9800f850 con 0x560c60dadcc0 2023-09-10T00:53:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.345+0000 7f8cb6d26b80 10 client.17604 fetch_fsmap learned FSMap version 63 2023-09-10T00:53:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.346+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/1816384669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x7f8ca419aff0 con 0x560c60dadcc0 2023-09-10T00:53:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.346+0000 7f8c83fff700 1 -- 172.21.15.44:0/1816384669 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560c60c57490 con 0x560c60dadcc0 2023-09-10T00:53:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.346+0000 7f8c83fff700 1 -- 172.21.15.44:0/1816384669 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 63) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8c9801a9d0 con 0x560c60dadcc0 2023-09-10T00:53:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.346+0000 7f8cb6d26b80 10 client.17604 fetch_fsmap finished waiting for FSMap version 63 2023-09-10T00:53:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.346+0000 7f8cb6d26b80 10 client.17604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:53:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.346+0000 7f8cb6d26b80 20 client.17604 populate_metadata read hostname 'smithi044' 2023-09-10T00:53:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.346+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/1816384669 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c60dc7110 0x560c60dc9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.346+0000 7f8cb6d26b80 4 client.17604 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:53:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.346+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/1816384669 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c60d14840 con 0x560c60dc7110 2023-09-10T00:53:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.346+0000 7f8ca88fc700 1 --2- 172.21.15.44:0/1816384669 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c60dc7110 0x560c60dc9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.347+0000 7f8c83fff700 10 client.17604 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.347+0000 7f8ca88fc700 1 --2- 172.21.15.44:0/1816384669 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c60dc7110 0x560c60dc9530 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d745a4f85d9046a3 server_cookie=f1f3ad55222663e3 in_seq=0 out_seq=0 2023-09-10T00:53:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.347+0000 7f8c83fff700 1 -- 172.21.15.44:0/1816384669 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560c60d14840 con 0x560c60dc7110 2023-09-10T00:53:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.347+0000 7f8c83fff700 10 client.17604 handle_command_reply: tid=0 2023-09-10T00:53:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.347+0000 7f8cb6d26b80 1 client.17604 shutdown 2023-09-10T00:53:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.347+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/1816384669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60dadcc0 msgr2=0x560c60dbd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.347+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/1816384669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60dadcc0 0x560c60dbd000 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8c98011730 tx=0x7f8c98011a40 comp rx=0 tx=0).stop 2023-09-10T00:53:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.347+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/1816384669 shutdown_connections 2023-09-10T00:53:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.347+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/1816384669 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c60dc7110 0x560c60dc9530 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.347+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/1816384669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60db31a0 0x560c60db6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.347+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/1816384669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60dad300 0x560c60dbcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.347+0000 7f8c83fff700 0 client.17604 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.347+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/1816384669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60dadcc0 0x560c60dbd000 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.347+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/1816384669 >> 172.21.15.44:0/1816384669 conn(0x560c60d22030 msgr2=0x560c60da77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.348+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/1816384669 shutdown_connections 2023-09-10T00:53:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.348+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/1816384669 wait complete. 2023-09-10T00:53:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.348+0000 7f8cb6d26b80 20 client.17604 trim_cache size 0 max 16384 2023-09-10T00:53:13.372 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:12 smithi099 ceph-mon[102692]: pgmap v393: 129 pgs: 129 active+clean; 8.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 43 MiB/s wr, 173 op/s 2023-09-10T00:53:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.364+0000 7f8cb6d26b80 1 Processor -- start 2023-09-10T00:53:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.364+0000 7f8cb6d26b80 1 -- start start 2023-09-10T00:53:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.364+0000 7f8cb6d26b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dad300 0x560c60db2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.364+0000 7f8cb6d26b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60dadcc0 0x560c60dbc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.364+0000 7f8cb6d26b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60db31a0 0x560c60dbcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.364+0000 7f8cb6d26b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c60da28a0 con 0x560c60db31a0 2023-09-10T00:53:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.364+0000 7f8cb6d26b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c60da4fc0 con 0x560c60dadcc0 2023-09-10T00:53:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.364+0000 7f8cb6d26b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c60db7d80 con 0x560c60dad300 2023-09-10T00:53:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.365+0000 7f8ca90fd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60db31a0 0x560c60dbcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.365+0000 7f8ca88fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dad300 0x560c60db2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.365+0000 7f8ca88fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dad300 0x560c60db2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59942/0 (socket says 172.21.15.44:59942) 2023-09-10T00:53:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.365+0000 7f8ca90fd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60db31a0 0x560c60dbcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53062/0 (socket says 172.21.15.44:53062) 2023-09-10T00:53:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.365+0000 7f8ca88fc700 1 -- 172.21.15.44:0/1156748806 learned_addr learned my addr 172.21.15.44:0/1156748806 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.365+0000 7f8ca88fc700 1 -- 172.21.15.44:0/1156748806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60dadcc0 msgr2=0x560c60dbc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.365+0000 7f8ca88fc700 1 --2- 172.21.15.44:0/1156748806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60dadcc0 0x560c60dbc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.365+0000 7f8ca88fc700 1 -- 172.21.15.44:0/1156748806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60db31a0 msgr2=0x560c60dbcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.365+0000 7f8ca88fc700 1 --2- 172.21.15.44:0/1156748806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60db31a0 0x560c60dbcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.365+0000 7f8ca88fc700 1 -- 172.21.15.44:0/1156748806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c60b14c00 con 0x560c60dad300 2023-09-10T00:53:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.365+0000 7f8ca88fc700 1 --2- 172.21.15.44:0/1156748806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dad300 0x560c60db2540 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8c90098530 tx=0x7f8c90096650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2660c2869bb5374 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.365+0000 7f8c8f7fe700 1 -- 172.21.15.44:0/1156748806 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c900051c0 con 0x560c60dad300 2023-09-10T00:53:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.366+0000 7f8c8f7fe700 1 -- 172.21.15.44:0/1156748806 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c9009e010 con 0x560c60dad300 2023-09-10T00:53:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.366+0000 7f8c8f7fe700 1 -- 172.21.15.44:0/1156748806 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9009a6a0 con 0x560c60dad300 2023-09-10T00:53:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.366+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/1156748806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dad300 msgr2=0x560c60db2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.366+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/1156748806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dad300 0x560c60db2540 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8c90098530 tx=0x7f8c90096650 comp rx=0 tx=0).stop 2023-09-10T00:53:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.367+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/1156748806 shutdown_connections 2023-09-10T00:53:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.367+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/1156748806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60db31a0 0x560c60dbcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.367+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/1156748806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dad300 0x560c60db2540 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.367+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/1156748806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60dadcc0 0x560c60dbc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.367+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/1156748806 >> 172.21.15.44:0/1156748806 conn(0x560c60d22030 msgr2=0x560c60db1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.367+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/1156748806 shutdown_connections 2023-09-10T00:53:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.367+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/1156748806 wait complete. 2023-09-10T00:53:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.367+0000 7f8cb6d26b80 1 -- start start 2023-09-10T00:53:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.367+0000 7f8cb6d26b80 1 Processor -- start 2023-09-10T00:53:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8cb6d26b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dad300 0x560c60e0f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8cb6d26b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60dadcc0 0x560c60e0f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8cb6d26b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60db31a0 0x560c60e19f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8cb6d26b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c60e1a490 con 0x560c60dadcc0 2023-09-10T00:53:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8cb6d26b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c60e1a5d0 con 0x560c60db31a0 2023-09-10T00:53:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8cb6d26b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c60e1a740 con 0x560c60dad300 2023-09-10T00:53:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8cb6d26b80 10 client.0 _ll_register_callbacks cb 0x7f8c98009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:53:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8ca3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60dadcc0 0x560c60e0f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8ca88fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dad300 0x560c60e0f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8ca3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60dadcc0 0x560c60e0f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53068/0 (socket says 172.21.15.44:53068) 2023-09-10T00:53:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8ca88fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dad300 0x560c60e0f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:59948/0 (socket says 172.21.15.44:59948) 2023-09-10T00:53:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8ca3fff700 1 -- 172.21.15.44:0/3747658198 learned_addr learned my addr 172.21.15.44:0/3747658198 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8ca90fd700 1 --2- 172.21.15.44:0/3747658198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60db31a0 0x560c60e19f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8ca88fc700 1 -- 172.21.15.44:0/3747658198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60db31a0 msgr2=0x560c60e19f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8ca88fc700 1 --2- 172.21.15.44:0/3747658198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60db31a0 0x560c60e19f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8ca88fc700 1 -- 172.21.15.44:0/3747658198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60dadcc0 msgr2=0x560c60e0f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8ca88fc700 1 --2- 172.21.15.44:0/3747658198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60dadcc0 0x560c60e0f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8ca88fc700 1 -- 172.21.15.44:0/3747658198 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c60b2bd80 con 0x560c60dad300 2023-09-10T00:53:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.368+0000 7f8ca3fff700 1 --2- 172.21.15.44:0/3747658198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60dadcc0 0x560c60e0f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.369+0000 7f8ca88fc700 1 --2- 172.21.15.44:0/3747658198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dad300 0x560c60e0f0c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009fd0 tx=0x7f8c900cd3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e6a562074d150cf server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.369+0000 7f8c83fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:53:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.369+0000 7f8c83fff700 1 -- 172.21.15.44:0/3747658198 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c900a21c0 con 0x560c60dad300 2023-09-10T00:53:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.369+0000 7f8c83fff700 1 -- 172.21.15.44:0/3747658198 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8c900051c0 con 0x560c60dad300 2023-09-10T00:53:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.369+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/3747658198 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c900a0110 con 0x560c60dad300 2023-09-10T00:53:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.369+0000 7f8c83fff700 1 -- 172.21.15.44:0/3747658198 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c9009a440 con 0x560c60dad300 2023-09-10T00:53:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.369+0000 7f8cb6d26b80 10 client.27211 fetch_fsmap learned FSMap version 63 2023-09-10T00:53:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.369+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/3747658198 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x560c60b521f0 con 0x560c60dad300 2023-09-10T00:53:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.369+0000 7f8c83fff700 1 -- 172.21.15.44:0/3747658198 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c900a0110 con 0x560c60dad300 2023-09-10T00:53:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.369+0000 7f8c83fff700 1 -- 172.21.15.44:0/3747658198 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 63) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8c900cab40 con 0x560c60dad300 2023-09-10T00:53:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.369+0000 7f8cb6d26b80 10 client.27211 fetch_fsmap finished waiting for FSMap version 63 2023-09-10T00:53:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.369+0000 7f8cb6d26b80 10 client.27211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:53:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.369+0000 7f8cb6d26b80 20 client.27211 populate_metadata read hostname 'smithi044' 2023-09-10T00:53:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.369+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/3747658198 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c60e1b9a0 0x560c60e1dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.370+0000 7f8cb6d26b80 4 client.27211 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:53:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.370+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/3747658198 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c60cf88c0 con 0x560c60e1b9a0 2023-09-10T00:53:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.370+0000 7f8ca3fff700 1 --2- 172.21.15.44:0/3747658198 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c60e1b9a0 0x560c60e1dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.370+0000 7f8ca3fff700 1 --2- 172.21.15.44:0/3747658198 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c60e1b9a0 0x560c60e1dd90 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d94a0c4ab3b55e0e server_cookie=e28a32d033f6daa2 in_seq=0 out_seq=0 2023-09-10T00:53:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.370+0000 7f8c83fff700 10 client.27211 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.386+0000 7f8c83fff700 1 -- 172.21.15.44:0/3747658198 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c60cf88c0 con 0x560c60e1b9a0 2023-09-10T00:53:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.386+0000 7f8c83fff700 10 client.27211 handle_command_reply: tid=0 2023-09-10T00:53:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.386+0000 7f8cb6d26b80 1 client.27211 shutdown 2023-09-10T00:53:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.386+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/3747658198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dad300 msgr2=0x560c60e0f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.386+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/3747658198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dad300 0x560c60e0f0c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009fd0 tx=0x7f8c900cd3e0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.386+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/3747658198 shutdown_connections 2023-09-10T00:53:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.386+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/3747658198 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c60e1b9a0 0x560c60e1dd90 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.386+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/3747658198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c60db31a0 0x560c60e19f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.386+0000 7f8c83fff700 0 client.27211 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.386+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/3747658198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c60dad300 0x560c60e0f0c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.386+0000 7f8cb6d26b80 1 --2- 172.21.15.44:0/3747658198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c60dadcc0 0x560c60e0f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.386+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/3747658198 >> 172.21.15.44:0/3747658198 conn(0x560c60d22030 msgr2=0x560c60dacb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.386+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/3747658198 shutdown_connections 2023-09-10T00:53:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.386+0000 7f8cb6d26b80 1 -- 172.21.15.44:0/3747658198 wait complete. 2023-09-10T00:53:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.386+0000 7f8cb6d26b80 20 client.27211 trim_cache size 0 max 16384 2023-09-10T00:53:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:53:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:53:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:53:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:53:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.387+0000 7f8caab60700 1 -- 172.21.15.44:0/4208497562 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8c940a07b0 msgr2=0x7f8c940a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.387+0000 7f8caab60700 1 --2- 172.21.15.44:0/4208497562 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8c940a07b0 0x7f8c940a2c70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c003fd0 tx=0x7f8c9c004b40 comp rx=0 tx=0).stop 2023-09-10T00:53:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.387+0000 7f8caab60700 1 -- 172.21.15.44:0/4208497562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ca40ff760 msgr2=0x7f8ca410cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.387+0000 7f8caab60700 1 --2- 172.21.15.44:0/4208497562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ca40ff760 0x7f8ca410cee0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8c90005910 tx=0x7f8c9000f690 comp rx=0 tx=0).stop 2023-09-10T00:53:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.387+0000 7f8caab60700 1 -- 172.21.15.44:0/4208497562 shutdown_connections 2023-09-10T00:53:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.387+0000 7f8caab60700 1 --2- 172.21.15.44:0/4208497562 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8c940a07b0 0x7f8c940a2c70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.387+0000 7f8caab60700 1 --2- 172.21.15.44:0/4208497562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca41055b0 0x7f8ca410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.387+0000 7f8caab60700 1 --2- 172.21.15.44:0/4208497562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ca40ff760 0x7f8ca410cee0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.387+0000 7f8caab60700 1 --2- 172.21.15.44:0/4208497562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca4100120 0x7f8ca410d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.387+0000 7f8caab60700 1 -- 172.21.15.44:0/4208497562 >> 172.21.15.44:0/4208497562 conn(0x7f8ca4074f30 msgr2=0x7f8ca40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.387+0000 7f8caab60700 1 -- 172.21.15.44:0/4208497562 shutdown_connections 2023-09-10T00:53:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:13.387+0000 7f8caab60700 1 -- 172.21.15.44:0/4208497562 wait complete. 2023-09-10T00:53:13.947 INFO:teuthology.orchestra.run.smithi044.stderr:48300000 of 50000000 tuples (96%) done (elapsed 733.58 s, remaining 25.82 s) 2023-09-10T00:53:14.868 INFO:teuthology.orchestra.run.smithi044.stderr:48400000 of 50000000 tuples (96%) done (elapsed 734.50 s, remaining 24.28 s) 2023-09-10T00:53:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:14 smithi131 ceph-mon[103713]: pgmap v394: 129 pgs: 129 active+clean; 8.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 37 MiB/s wr, 149 op/s 2023-09-10T00:53:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:14 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:14 smithi044 ceph-mon[98676]: pgmap v394: 129 pgs: 129 active+clean; 8.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 37 MiB/s wr, 149 op/s 2023-09-10T00:53:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:14 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:14 smithi099 ceph-mon[102692]: pgmap v394: 129 pgs: 129 active+clean; 8.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 37 MiB/s wr, 149 op/s 2023-09-10T00:53:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:14 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:15.818 INFO:teuthology.orchestra.run.smithi044.stderr:48500000 of 50000000 tuples (97%) done (elapsed 735.45 s, remaining 22.75 s) 2023-09-10T00:53:16.838 INFO:teuthology.orchestra.run.smithi044.stderr:48600000 of 50000000 tuples (97%) done (elapsed 736.47 s, remaining 21.22 s) 2023-09-10T00:53:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:16 smithi131 ceph-mon[103713]: pgmap v395: 129 pgs: 129 active+clean; 8.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 127 op/s 2023-09-10T00:53:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:16 smithi044 ceph-mon[98676]: pgmap v395: 129 pgs: 129 active+clean; 8.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 127 op/s 2023-09-10T00:53:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:16 smithi099 ceph-mon[102692]: pgmap v395: 129 pgs: 129 active+clean; 8.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 127 op/s 2023-09-10T00:53:17.881 INFO:teuthology.orchestra.run.smithi044.stderr:48700000 of 50000000 tuples (97%) done (elapsed 737.52 s, remaining 19.69 s) 2023-09-10T00:53:18.859 INFO:teuthology.orchestra.run.smithi044.stderr:48800000 of 50000000 tuples (97%) done (elapsed 738.49 s, remaining 18.16 s) 2023-09-10T00:53:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:18 smithi131 ceph-mon[103713]: pgmap v396: 129 pgs: 129 active+clean; 8.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 127 op/s 2023-09-10T00:53:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:18 smithi044 ceph-mon[98676]: pgmap v396: 129 pgs: 129 active+clean; 8.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 127 op/s 2023-09-10T00:53:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:18 smithi099 ceph-mon[102692]: pgmap v396: 129 pgs: 129 active+clean; 8.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 127 op/s 2023-09-10T00:53:20.282 INFO:teuthology.orchestra.run.smithi044.stderr:48900000 of 50000000 tuples (97%) done (elapsed 739.92 s, remaining 16.64 s) 2023-09-10T00:53:20.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:19 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:20.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:19 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:20.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:19 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:20 smithi131 ceph-mon[103713]: pgmap v397: 129 pgs: 129 active+clean; 8.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 127 op/s 2023-09-10T00:53:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:20 smithi044 ceph-mon[98676]: pgmap v397: 129 pgs: 129 active+clean; 8.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 127 op/s 2023-09-10T00:53:21.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:20 smithi099 ceph-mon[102692]: pgmap v397: 129 pgs: 129 active+clean; 8.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 127 op/s 2023-09-10T00:53:22.458 INFO:teuthology.orchestra.run.smithi044.stderr:49000000 of 50000000 tuples (98%) done (elapsed 742.09 s, remaining 15.14 s) 2023-09-10T00:53:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:22 smithi131 ceph-mon[103713]: pgmap v398: 129 pgs: 129 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 34 MiB/s wr, 117 op/s 2023-09-10T00:53:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:22 smithi044 ceph-mon[98676]: pgmap v398: 129 pgs: 129 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 34 MiB/s wr, 117 op/s 2023-09-10T00:53:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:22 smithi099 ceph-mon[102692]: pgmap v398: 129 pgs: 129 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 34 MiB/s wr, 117 op/s 2023-09-10T00:53:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:24 smithi131 ceph-mon[103713]: pgmap v399: 129 pgs: 129 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 30 MiB/s wr, 102 op/s 2023-09-10T00:53:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:24 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:24 smithi044 ceph-mon[98676]: pgmap v399: 129 pgs: 129 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 30 MiB/s wr, 102 op/s 2023-09-10T00:53:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:24 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:24 smithi099 ceph-mon[102692]: pgmap v399: 129 pgs: 129 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 30 MiB/s wr, 102 op/s 2023-09-10T00:53:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:24 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.514+0000 7faa5d4ea700 1 -- 172.21.15.44:0/2379740319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa580ff4b0 msgr2=0x7faa580ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.514+0000 7faa5d4ea700 1 --2- 172.21.15.44:0/2379740319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa580ff4b0 0x7faa580ff930 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0098f0 tx=0x7faa4c009c00 comp rx=0 tx=0).stop 2023-09-10T00:53:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.514+0000 7faa5d4ea700 1 -- 172.21.15.44:0/2379740319 shutdown_connections 2023-09-10T00:53:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.514+0000 7faa5d4ea700 1 --2- 172.21.15.44:0/2379740319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa580ffe70 0x7faa5810c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.514+0000 7faa5d4ea700 1 --2- 172.21.15.44:0/2379740319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa58104410 0x7faa581047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.514+0000 7faa5d4ea700 1 --2- 172.21.15.44:0/2379740319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa580ff4b0 0x7faa580ff930 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.514+0000 7faa5d4ea700 1 -- 172.21.15.44:0/2379740319 >> 172.21.15.44:0/2379740319 conn(0x7faa580765b0 msgr2=0x7faa580769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.514+0000 7faa5d4ea700 1 -- 172.21.15.44:0/2379740319 shutdown_connections 2023-09-10T00:53:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.514+0000 7faa5d4ea700 1 -- 172.21.15.44:0/2379740319 wait complete. 2023-09-10T00:53:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.515+0000 7faa5d4ea700 1 Processor -- start 2023-09-10T00:53:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.515+0000 7faa5d4ea700 1 -- start start 2023-09-10T00:53:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.515+0000 7faa5d4ea700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa580ff4b0 0x7faa5819b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.515+0000 7faa5d4ea700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa580ffe70 0x7faa5819ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.515+0000 7faa5d4ea700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa58104410 0x7faa58195360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.515+0000 7faa5d4ea700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa581959d0 con 0x7faa58104410 2023-09-10T00:53:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.515+0000 7faa5d4ea700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa58195b40 con 0x7faa580ffe70 2023-09-10T00:53:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.515+0000 7faa5d4ea700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa58195d00 con 0x7faa580ff4b0 2023-09-10T00:53:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.515+0000 7faa577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa58104410 0x7faa58195360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.515+0000 7faa56ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa580ff4b0 0x7faa5819b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.515+0000 7faa567fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa580ffe70 0x7faa5819ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.515+0000 7faa577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa58104410 0x7faa58195360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53082/0 (socket says 172.21.15.44:53082) 2023-09-10T00:53:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.515+0000 7faa577fe700 1 -- 172.21.15.44:0/4224734939 learned_addr learned my addr 172.21.15.44:0/4224734939 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.516+0000 7faa577fe700 1 -- 172.21.15.44:0/4224734939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa580ff4b0 msgr2=0x7faa5819b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.516+0000 7faa577fe700 1 --2- 172.21.15.44:0/4224734939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa580ff4b0 0x7faa5819b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.516+0000 7faa577fe700 1 -- 172.21.15.44:0/4224734939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa580ffe70 msgr2=0x7faa5819ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.516+0000 7faa577fe700 1 --2- 172.21.15.44:0/4224734939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa580ffe70 0x7faa5819ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.516+0000 7faa577fe700 1 -- 172.21.15.44:0/4224734939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4c009520 con 0x7faa58104410 2023-09-10T00:53:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.516+0000 7faa56ffd700 1 --2- 172.21.15.44:0/4224734939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa580ff4b0 0x7faa5819b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.516+0000 7faa567fc700 1 --2- 172.21.15.44:0/4224734939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa580ffe70 0x7faa5819ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.516+0000 7faa577fe700 1 --2- 172.21.15.44:0/4224734939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa58104410 0x7faa58195360 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7faa5000ef30 tx=0x7faa5000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.516+0000 7faa3ffff700 1 -- 172.21.15.44:0/4224734939 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa50018070 con 0x7faa58104410 2023-09-10T00:53:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.516+0000 7faa3ffff700 1 -- 172.21.15.44:0/4224734939 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa5000f040 con 0x7faa58104410 2023-09-10T00:53:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.516+0000 7faa5d4ea700 1 -- 172.21.15.44:0/4224734939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa58195f80 con 0x7faa58104410 2023-09-10T00:53:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.516+0000 7faa3ffff700 1 -- 172.21.15.44:0/4224734939 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5000cbb0 con 0x7faa58104410 2023-09-10T00:53:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.516+0000 7faa5d4ea700 1 -- 172.21.15.44:0/4224734939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa581a24f0 con 0x7faa58104410 2023-09-10T00:53:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.518+0000 7faa3ffff700 1 -- 172.21.15.44:0/4224734939 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7faa5000cd70 con 0x7faa58104410 2023-09-10T00:53:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.519+0000 7faa3ffff700 1 --2- 172.21.15.44:0/4224734939 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faa400a07b0 0x7faa400a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.519+0000 7faa3ffff700 1 -- 172.21.15.44:0/4224734939 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7faa500cbee0 con 0x7faa58104410 2023-09-10T00:53:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.519+0000 7faa56ffd700 1 --2- 172.21.15.44:0/4224734939 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faa400a07b0 0x7faa400a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.519+0000 7faa56ffd700 1 --2- 172.21.15.44:0/4224734939 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faa400a07b0 0x7faa400a2c70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faa48005fd0 tx=0x7faa48009310 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa696b0b80 1 Processor -- start 2023-09-10T00:53:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa696b0b80 1 -- start start 2023-09-10T00:53:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa696b0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 0x559462e47580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa696b0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41300 0x559462e41780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa696b0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e4df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa696b0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559462d9a6d0 con 0x559462e41cc0 2023-09-10T00:53:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa696b0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559462d97ff0 con 0x559462e41300 2023-09-10T00:53:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa696b0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559462d938e0 con 0x559462e471a0 2023-09-10T00:53:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e4df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa56ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 0x559462e47580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e4df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53090/0 (socket says 172.21.15.44:53090) 2023-09-10T00:53:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa577fe700 1 -- 172.21.15.44:0/229108454 learned_addr learned my addr 172.21.15.44:0/229108454 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa567fc700 1 --2- 172.21.15.44:0/229108454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41300 0x559462e41780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa577fe700 1 -- 172.21.15.44:0/229108454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 msgr2=0x559462e47580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa577fe700 1 --2- 172.21.15.44:0/229108454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 0x559462e47580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa577fe700 1 -- 172.21.15.44:0/229108454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41300 msgr2=0x559462e41780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa577fe700 1 --2- 172.21.15.44:0/229108454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41300 0x559462e41780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa577fe700 1 -- 172.21.15.44:0/229108454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559462bbfd80 con 0x559462e41cc0 2023-09-10T00:53:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa577fe700 1 --2- 172.21.15.44:0/229108454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e4df30 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7faa50009f40 tx=0x7faa50098920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a12516068f14a62 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.520+0000 7faa3d7fa700 1 -- 172.21.15.44:0/229108454 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa500996c0 con 0x559462e41cc0 2023-09-10T00:53:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.521+0000 7faa3d7fa700 1 -- 172.21.15.44:0/229108454 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa50098fc0 con 0x559462e41cc0 2023-09-10T00:53:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.521+0000 7faa3d7fa700 1 -- 172.21.15.44:0/229108454 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa50099480 con 0x559462e41cc0 2023-09-10T00:53:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.521+0000 7faa696b0b80 1 -- 172.21.15.44:0/229108454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 msgr2=0x559462e4df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.521+0000 7faa696b0b80 1 --2- 172.21.15.44:0/229108454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e4df30 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7faa50009f40 tx=0x7faa50098920 comp rx=0 tx=0).stop 2023-09-10T00:53:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.521+0000 7faa696b0b80 1 -- 172.21.15.44:0/229108454 shutdown_connections 2023-09-10T00:53:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.521+0000 7faa696b0b80 1 --2- 172.21.15.44:0/229108454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e4df30 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.521+0000 7faa696b0b80 1 --2- 172.21.15.44:0/229108454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 0x559462e47580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.521+0000 7faa696b0b80 1 --2- 172.21.15.44:0/229108454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41300 0x559462e41780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.521+0000 7faa696b0b80 1 -- 172.21.15.44:0/229108454 >> 172.21.15.44:0/229108454 conn(0x559462db6030 msgr2=0x559462e3c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.521+0000 7faa696b0b80 1 -- 172.21.15.44:0/229108454 shutdown_connections 2023-09-10T00:53:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.521+0000 7faa696b0b80 1 -- 172.21.15.44:0/229108454 wait complete. 2023-09-10T00:53:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa696b0b80 1 -- start start 2023-09-10T00:53:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa696b0b80 1 Processor -- start 2023-09-10T00:53:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa696b0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41300 0x559462e4dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa696b0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e4b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa696b0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 0x559462e4b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa696b0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559462d31b10 con 0x559462e41cc0 2023-09-10T00:53:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa696b0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559462d31810 con 0x559462e41300 2023-09-10T00:53:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa696b0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559462c158c0 con 0x559462e471a0 2023-09-10T00:53:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa696b0b80 10 client.0 _ll_register_callbacks cb 0x559462c7ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:53:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa567fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e4b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa567fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e4b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53094/0 (socket says 172.21.15.44:53094) 2023-09-10T00:53:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa567fc700 1 -- 172.21.15.44:0/3817540191 learned_addr learned my addr 172.21.15.44:0/3817540191 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa56ffd700 1 --2- 172.21.15.44:0/3817540191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41300 0x559462e4dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa577fe700 1 --2- 172.21.15.44:0/3817540191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 0x559462e4b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa567fc700 1 -- 172.21.15.44:0/3817540191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 msgr2=0x559462e4b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa567fc700 1 --2- 172.21.15.44:0/3817540191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 0x559462e4b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa567fc700 1 -- 172.21.15.44:0/3817540191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41300 msgr2=0x559462e4dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa567fc700 1 --2- 172.21.15.44:0/3817540191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41300 0x559462e4dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa567fc700 1 -- 172.21.15.44:0/3817540191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa581a24f0 con 0x559462e41cc0 2023-09-10T00:53:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa567fc700 1 --2- 172.21.15.44:0/3817540191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e4b1e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0098f0 tx=0x7faa4c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d834bc437bbe16a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.522+0000 7faa3aff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:53:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.523+0000 7faa3aff5700 1 -- 172.21.15.44:0/3817540191 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c01c040 con 0x559462e41cc0 2023-09-10T00:53:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.523+0000 7faa3aff5700 1 -- 172.21.15.44:0/3817540191 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7faa4c005040 con 0x559462e41cc0 2023-09-10T00:53:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.523+0000 7faa696b0b80 1 -- 172.21.15.44:0/3817540191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559462ceb490 con 0x559462e41cc0 2023-09-10T00:53:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.523+0000 7faa3aff5700 1 -- 172.21.15.44:0/3817540191 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa4c00bec0 con 0x559462e41cc0 2023-09-10T00:53:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.523+0000 7faa696b0b80 10 client.17622 fetch_fsmap learned FSMap version 66 2023-09-10T00:53:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.523+0000 7faa696b0b80 1 -- 172.21.15.44:0/3817540191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x559462ba8c00 con 0x559462e41cc0 2023-09-10T00:53:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.523+0000 7faa3aff5700 1 -- 172.21.15.44:0/3817540191 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559462ceb490 con 0x559462e41cc0 2023-09-10T00:53:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.523+0000 7faa3aff5700 1 -- 172.21.15.44:0/3817540191 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 66) v1 ==== 7906+0+0 (secure 0 0 0) 0x7faa4c015020 con 0x559462e41cc0 2023-09-10T00:53:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.523+0000 7faa696b0b80 10 client.17622 fetch_fsmap finished waiting for FSMap version 66 2023-09-10T00:53:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.523+0000 7faa696b0b80 10 client.17622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:53:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.523+0000 7faa696b0b80 20 client.17622 populate_metadata read hostname 'smithi044' 2023-09-10T00:53:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.523+0000 7faa696b0b80 1 --2- 172.21.15.44:0/3817540191 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559462e5b5e0 0x559462e5da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.523+0000 7faa696b0b80 4 client.17622 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:53:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.523+0000 7faa696b0b80 1 -- 172.21.15.44:0/3817540191 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559462da8840 con 0x559462e5b5e0 2023-09-10T00:53:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.523+0000 7faa56ffd700 1 --2- 172.21.15.44:0/3817540191 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559462e5b5e0 0x559462e5da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.524+0000 7faa56ffd700 1 --2- 172.21.15.44:0/3817540191 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559462e5b5e0 0x559462e5da00 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75388a425119bf35 server_cookie=3317c297ed72bfa5 in_seq=0 out_seq=0 2023-09-10T00:53:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.524+0000 7faa3aff5700 10 client.17622 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.524+0000 7faa3aff5700 1 -- 172.21.15.44:0/3817540191 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559462da8840 con 0x559462e5b5e0 2023-09-10T00:53:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.524+0000 7faa3aff5700 10 client.17622 handle_command_reply: tid=0 2023-09-10T00:53:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.524+0000 7faa696b0b80 1 client.17622 shutdown 2023-09-10T00:53:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.525+0000 7faa696b0b80 1 -- 172.21.15.44:0/3817540191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 msgr2=0x559462e4b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.525+0000 7faa696b0b80 1 --2- 172.21.15.44:0/3817540191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e4b1e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0098f0 tx=0x7faa4c00b790 comp rx=0 tx=0).stop 2023-09-10T00:53:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.525+0000 7faa696b0b80 1 -- 172.21.15.44:0/3817540191 shutdown_connections 2023-09-10T00:53:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.525+0000 7faa696b0b80 1 --2- 172.21.15.44:0/3817540191 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559462e5b5e0 0x559462e5da00 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.525+0000 7faa696b0b80 1 --2- 172.21.15.44:0/3817540191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 0x559462e4b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.525+0000 7faa696b0b80 1 --2- 172.21.15.44:0/3817540191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41300 0x559462e4dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.525+0000 7faa696b0b80 1 --2- 172.21.15.44:0/3817540191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e4b1e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.525+0000 7faa3aff5700 0 client.17622 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.525+0000 7faa696b0b80 1 -- 172.21.15.44:0/3817540191 >> 172.21.15.44:0/3817540191 conn(0x559462db6030 msgr2=0x559462e3b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.525+0000 7faa696b0b80 1 -- 172.21.15.44:0/3817540191 shutdown_connections 2023-09-10T00:53:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.525+0000 7faa696b0b80 1 -- 172.21.15.44:0/3817540191 wait complete. 2023-09-10T00:53:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.525+0000 7faa696b0b80 20 client.17622 trim_cache size 0 max 16384 2023-09-10T00:53:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.539+0000 7faa696b0b80 1 Processor -- start 2023-09-10T00:53:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.539+0000 7faa696b0b80 1 -- start start 2023-09-10T00:53:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa696b0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41300 0x559462e52530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa696b0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41cc0 0x559462e40a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa696b0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 0x559462e3bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa696b0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559462e33f50 con 0x559462e41300 2023-09-10T00:53:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa696b0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559462e368a0 con 0x559462e41cc0 2023-09-10T00:53:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa696b0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559462e3d380 con 0x559462e471a0 2023-09-10T00:53:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa56ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41300 0x559462e52530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa56ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41300 0x559462e52530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53100/0 (socket says 172.21.15.44:53100) 2023-09-10T00:53:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa56ffd700 1 -- 172.21.15.44:0/2079196060 learned_addr learned my addr 172.21.15.44:0/2079196060 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa567fc700 1 --2- 172.21.15.44:0/2079196060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41cc0 0x559462e40a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa577fe700 1 --2- 172.21.15.44:0/2079196060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 0x559462e3bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa56ffd700 1 -- 172.21.15.44:0/2079196060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 msgr2=0x559462e3bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa56ffd700 1 --2- 172.21.15.44:0/2079196060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 0x559462e3bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa56ffd700 1 -- 172.21.15.44:0/2079196060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41cc0 msgr2=0x559462e40a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa56ffd700 1 --2- 172.21.15.44:0/2079196060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41cc0 0x559462e40a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa56ffd700 1 -- 172.21.15.44:0/2079196060 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559462be61f0 con 0x559462e41300 2023-09-10T00:53:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa56ffd700 1 --2- 172.21.15.44:0/2079196060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41300 0x559462e52530 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x559462e5e2a0 tx=0x7faa48014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8425aa53793ae89 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa3dffb700 1 -- 172.21.15.44:0/2079196060 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa48024040 con 0x559462e41300 2023-09-10T00:53:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa3dffb700 1 -- 172.21.15.44:0/2079196060 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa48003b10 con 0x559462e41300 2023-09-10T00:53:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.540+0000 7faa3dffb700 1 -- 172.21.15.44:0/2079196060 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4801ea30 con 0x559462e41300 2023-09-10T00:53:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 -- 172.21.15.44:0/2079196060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41300 msgr2=0x559462e52530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 --2- 172.21.15.44:0/2079196060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41300 0x559462e52530 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x559462e5e2a0 tx=0x7faa48014000 comp rx=0 tx=0).stop 2023-09-10T00:53:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 -- 172.21.15.44:0/2079196060 shutdown_connections 2023-09-10T00:53:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 --2- 172.21.15.44:0/2079196060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 0x559462e3bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 --2- 172.21.15.44:0/2079196060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41300 0x559462e52530 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 --2- 172.21.15.44:0/2079196060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41cc0 0x559462e40a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 -- 172.21.15.44:0/2079196060 >> 172.21.15.44:0/2079196060 conn(0x559462db6030 msgr2=0x559462e402f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 -- 172.21.15.44:0/2079196060 shutdown_connections 2023-09-10T00:53:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 -- 172.21.15.44:0/2079196060 wait complete. 2023-09-10T00:53:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 -- start start 2023-09-10T00:53:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 Processor -- start 2023-09-10T00:53:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41300 0x559462e523d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e50690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 0x559462e50bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559462e51150 con 0x559462e41cc0 2023-09-10T00:53:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559462e51290 con 0x559462e41300 2023-09-10T00:53:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.541+0000 7faa696b0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559462e51450 con 0x559462e471a0 2023-09-10T00:53:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa696b0b80 10 client.0 _ll_register_callbacks cb 0x7faa4c00f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:53:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa567fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e50690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa567fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e50690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53108/0 (socket says 172.21.15.44:53108) 2023-09-10T00:53:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa56ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41300 0x559462e523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa567fc700 1 -- 172.21.15.44:0/2672477109 learned_addr learned my addr 172.21.15.44:0/2672477109 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa577fe700 1 --2- 172.21.15.44:0/2672477109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 0x559462e50bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa567fc700 1 -- 172.21.15.44:0/2672477109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 msgr2=0x559462e50bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa567fc700 1 --2- 172.21.15.44:0/2672477109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 0x559462e50bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa567fc700 1 -- 172.21.15.44:0/2672477109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41300 msgr2=0x559462e523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa567fc700 1 --2- 172.21.15.44:0/2672477109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41300 0x559462e523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa567fc700 1 -- 172.21.15.44:0/2672477109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa48017070 con 0x559462e41cc0 2023-09-10T00:53:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa567fc700 1 --2- 172.21.15.44:0/2672477109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e50690 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009810 tx=0x7faa4c009fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bd01ac7e7fb6c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa3aff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:53:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa3aff5700 1 -- 172.21.15.44:0/2672477109 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c01c040 con 0x559462e41cc0 2023-09-10T00:53:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa3aff5700 1 -- 172.21.15.44:0/2672477109 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7faa4c000e40 con 0x559462e41cc0 2023-09-10T00:53:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa696b0b80 1 -- 172.21.15.44:0/2672477109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa50098fc0 con 0x559462e41cc0 2023-09-10T00:53:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa3aff5700 1 -- 172.21.15.44:0/2672477109 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa4c00bec0 con 0x559462e41cc0 2023-09-10T00:53:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa696b0b80 10 client.17628 fetch_fsmap learned FSMap version 66 2023-09-10T00:53:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.542+0000 7faa696b0b80 1 -- 172.21.15.44:0/2672477109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x559462bbe250 con 0x559462e41cc0 2023-09-10T00:53:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.543+0000 7faa3aff5700 1 -- 172.21.15.44:0/2672477109 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa50098fc0 con 0x559462e41cc0 2023-09-10T00:53:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.543+0000 7faa3aff5700 1 -- 172.21.15.44:0/2672477109 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 66) v1 ==== 7906+0+0 (secure 0 0 0) 0x7faa4c015030 con 0x559462e41cc0 2023-09-10T00:53:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.543+0000 7faa696b0b80 10 client.17628 fetch_fsmap finished waiting for FSMap version 66 2023-09-10T00:53:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.543+0000 7faa696b0b80 10 client.17628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:53:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.543+0000 7faa696b0b80 20 client.17628 populate_metadata read hostname 'smithi044' 2023-09-10T00:53:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.543+0000 7faa696b0b80 1 --2- 172.21.15.44:0/2672477109 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559462eb0680 0x559462eb2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.543+0000 7faa696b0b80 4 client.17628 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:53:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.543+0000 7faa696b0b80 1 -- 172.21.15.44:0/2672477109 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559462d8c8c0 con 0x559462eb0680 2023-09-10T00:53:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.543+0000 7faa56ffd700 1 --2- 172.21.15.44:0/2672477109 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559462eb0680 0x559462eb2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.543+0000 7faa3aff5700 10 client.17628 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.543+0000 7faa56ffd700 1 --2- 172.21.15.44:0/2672477109 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559462eb0680 0x559462eb2aa0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ebbb3ad0f3530d6 server_cookie=6cada868c761df6d in_seq=0 out_seq=0 2023-09-10T00:53:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.556+0000 7faa3aff5700 1 -- 172.21.15.44:0/2672477109 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559462d8c8c0 con 0x559462eb0680 2023-09-10T00:53:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.556+0000 7faa3aff5700 10 client.17628 handle_command_reply: tid=0 2023-09-10T00:53:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.556+0000 7faa696b0b80 1 client.17628 shutdown 2023-09-10T00:53:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.556+0000 7faa696b0b80 1 -- 172.21.15.44:0/2672477109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 msgr2=0x559462e50690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.556+0000 7faa696b0b80 1 --2- 172.21.15.44:0/2672477109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e50690 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009810 tx=0x7faa4c009fc0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.556+0000 7faa696b0b80 1 -- 172.21.15.44:0/2672477109 shutdown_connections 2023-09-10T00:53:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.556+0000 7faa696b0b80 1 --2- 172.21.15.44:0/2672477109 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559462eb0680 0x559462eb2aa0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.556+0000 7faa696b0b80 1 --2- 172.21.15.44:0/2672477109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559462e471a0 0x559462e50bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.556+0000 7faa696b0b80 1 --2- 172.21.15.44:0/2672477109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559462e41300 0x559462e523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.556+0000 7faa696b0b80 1 --2- 172.21.15.44:0/2672477109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559462e41cc0 0x559462e50690 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.557+0000 7faa696b0b80 1 -- 172.21.15.44:0/2672477109 >> 172.21.15.44:0/2672477109 conn(0x559462db6030 msgr2=0x559462e40b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.556+0000 7faa3aff5700 0 client.17628 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.557+0000 7faa696b0b80 1 -- 172.21.15.44:0/2672477109 shutdown_connections 2023-09-10T00:53:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.557+0000 7faa696b0b80 1 -- 172.21.15.44:0/2672477109 wait complete. 2023-09-10T00:53:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.557+0000 7faa696b0b80 20 client.17628 trim_cache size 0 max 16384 2023-09-10T00:53:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:53:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:53:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:53:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:53:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.557+0000 7faa5d4ea700 1 -- 172.21.15.44:0/4224734939 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faa400a07b0 msgr2=0x7faa400a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.557+0000 7faa5d4ea700 1 --2- 172.21.15.44:0/4224734939 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faa400a07b0 0x7faa400a2c70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faa48005fd0 tx=0x7faa48009310 comp rx=0 tx=0).stop 2023-09-10T00:53:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.557+0000 7faa5d4ea700 1 -- 172.21.15.44:0/4224734939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa58104410 msgr2=0x7faa58195360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.557+0000 7faa5d4ea700 1 --2- 172.21.15.44:0/4224734939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa58104410 0x7faa58195360 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7faa5000ef30 tx=0x7faa5000c3a0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.557+0000 7faa5d4ea700 1 -- 172.21.15.44:0/4224734939 shutdown_connections 2023-09-10T00:53:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.557+0000 7faa5d4ea700 1 --2- 172.21.15.44:0/4224734939 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faa400a07b0 0x7faa400a2c70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.557+0000 7faa5d4ea700 1 --2- 172.21.15.44:0/4224734939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa58104410 0x7faa58195360 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.557+0000 7faa5d4ea700 1 --2- 172.21.15.44:0/4224734939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa580ff4b0 0x7faa5819b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.557+0000 7faa5d4ea700 1 --2- 172.21.15.44:0/4224734939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa580ffe70 0x7faa5819ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.557+0000 7faa5d4ea700 1 -- 172.21.15.44:0/4224734939 >> 172.21.15.44:0/4224734939 conn(0x7faa580765b0 msgr2=0x7faa580fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.557+0000 7faa5d4ea700 1 -- 172.21.15.44:0/4224734939 shutdown_connections 2023-09-10T00:53:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:26.557+0000 7faa5d4ea700 1 -- 172.21.15.44:0/4224734939 wait complete. 2023-09-10T00:53:26.840 INFO:teuthology.orchestra.run.smithi044.stderr:49100000 of 50000000 tuples (98%) done (elapsed 746.47 s, remaining 13.68 s) 2023-09-10T00:53:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:26 smithi131 ceph-mon[103713]: pgmap v400: 129 pgs: 129 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 34 MiB/s wr, 127 op/s 2023-09-10T00:53:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:26 smithi044 ceph-mon[98676]: pgmap v400: 129 pgs: 129 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 34 MiB/s wr, 127 op/s 2023-09-10T00:53:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:26 smithi099 ceph-mon[102692]: pgmap v400: 129 pgs: 129 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 34 MiB/s wr, 127 op/s 2023-09-10T00:53:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:28 smithi131 ceph-mon[103713]: pgmap v401: 129 pgs: 129 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 20 MiB/s wr, 75 op/s 2023-09-10T00:53:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:28 smithi044 ceph-mon[98676]: pgmap v401: 129 pgs: 129 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 20 MiB/s wr, 75 op/s 2023-09-10T00:53:29.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:28 smithi099 ceph-mon[102692]: pgmap v401: 129 pgs: 129 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 20 MiB/s wr, 75 op/s 2023-09-10T00:53:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:30 smithi131 ceph-mon[103713]: pgmap v402: 129 pgs: 129 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 20 MiB/s wr, 75 op/s 2023-09-10T00:53:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:30 smithi044 ceph-mon[98676]: pgmap v402: 129 pgs: 129 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 20 MiB/s wr, 75 op/s 2023-09-10T00:53:31.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:30 smithi099 ceph-mon[102692]: pgmap v402: 129 pgs: 129 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 20 MiB/s wr, 75 op/s 2023-09-10T00:53:32.134 INFO:teuthology.orchestra.run.smithi044.stderr:49200000 of 50000000 tuples (98%) done (elapsed 751.77 s, remaining 12.22 s) 2023-09-10T00:53:33.062 INFO:teuthology.orchestra.run.smithi044.stderr:49300000 of 50000000 tuples (98%) done (elapsed 752.70 s, remaining 10.69 s) 2023-09-10T00:53:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:32 smithi131 ceph-mon[103713]: pgmap v403: 129 pgs: 129 active+clean; 9.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 22 MiB/s wr, 92 op/s 2023-09-10T00:53:33.304 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:32 smithi044 ceph-mon[98676]: pgmap v403: 129 pgs: 129 active+clean; 9.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 22 MiB/s wr, 92 op/s 2023-09-10T00:53:33.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:32 smithi099 ceph-mon[102692]: pgmap v403: 129 pgs: 129 active+clean; 9.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 22 MiB/s wr, 92 op/s 2023-09-10T00:53:33.914 INFO:teuthology.orchestra.run.smithi044.stderr:49400000 of 50000000 tuples (98%) done (elapsed 753.55 s, remaining 9.15 s) 2023-09-10T00:53:34.801 INFO:teuthology.orchestra.run.smithi044.stderr:49500000 of 50000000 tuples (99%) done (elapsed 754.43 s, remaining 7.62 s) 2023-09-10T00:53:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:34 smithi131 ceph-mon[103713]: pgmap v404: 129 pgs: 129 active+clean; 9.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.3 MiB/s wr, 41 op/s 2023-09-10T00:53:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:34 smithi044 ceph-mon[98676]: pgmap v404: 129 pgs: 129 active+clean; 9.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.3 MiB/s wr, 41 op/s 2023-09-10T00:53:35.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:34 smithi099 ceph-mon[102692]: pgmap v404: 129 pgs: 129 active+clean; 9.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.3 MiB/s wr, 41 op/s 2023-09-10T00:53:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:36 smithi131 ceph-mon[103713]: pgmap v405: 129 pgs: 129 active+clean; 9.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 18 MiB/s wr, 62 op/s 2023-09-10T00:53:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:36 smithi044 ceph-mon[98676]: pgmap v405: 129 pgs: 129 active+clean; 9.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 18 MiB/s wr, 62 op/s 2023-09-10T00:53:37.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:36 smithi099 ceph-mon[102692]: pgmap v405: 129 pgs: 129 active+clean; 9.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 18 MiB/s wr, 62 op/s 2023-09-10T00:53:38.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:37 smithi131 ceph-mon[103713]: pgmap v406: 129 pgs: 129 active+clean; 9.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-09-10T00:53:38.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:37 smithi044 ceph-mon[98676]: pgmap v406: 129 pgs: 129 active+clean; 9.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-09-10T00:53:38.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:37 smithi099 ceph-mon[102692]: pgmap v406: 129 pgs: 129 active+clean; 9.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-09-10T00:53:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.695+0000 7ffbaa3ac700 1 -- 172.21.15.44:0/1833299117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba40ff4b0 msgr2=0x7ffba40ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.695+0000 7ffbaa3ac700 1 --2- 172.21.15.44:0/1833299117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba40ff4b0 0x7ffba40ff930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009990 tx=0x7ffb98009ca0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.695+0000 7ffbaa3ac700 1 -- 172.21.15.44:0/1833299117 shutdown_connections 2023-09-10T00:53:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.695+0000 7ffbaa3ac700 1 --2- 172.21.15.44:0/1833299117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba40ffe70 0x7ffba410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.695+0000 7ffbaa3ac700 1 --2- 172.21.15.44:0/1833299117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffba4105570 0x7ffba4105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.695+0000 7ffbaa3ac700 1 --2- 172.21.15.44:0/1833299117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba40ff4b0 0x7ffba40ff930 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.695+0000 7ffbaa3ac700 1 -- 172.21.15.44:0/1833299117 >> 172.21.15.44:0/1833299117 conn(0x7ffba40faf60 msgr2=0x7ffba40fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.695+0000 7ffbaa3ac700 1 -- 172.21.15.44:0/1833299117 shutdown_connections 2023-09-10T00:53:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.695+0000 7ffbaa3ac700 1 -- 172.21.15.44:0/1833299117 wait complete. 2023-09-10T00:53:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.696+0000 7ffbaa3ac700 1 Processor -- start 2023-09-10T00:53:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.696+0000 7ffbaa3ac700 1 -- start start 2023-09-10T00:53:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.696+0000 7ffbaa3ac700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba40ff4b0 0x7ffba4197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.696+0000 7ffbaa3ac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffba40ffe70 0x7ffba4197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.696+0000 7ffbaa3ac700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba4105570 0x7ffba41a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.696+0000 7ffbaa3ac700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba41a29b0 con 0x7ffba4105570 2023-09-10T00:53:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.696+0000 7ffbaa3ac700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba41a2b20 con 0x7ffba40ffe70 2023-09-10T00:53:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.696+0000 7ffbaa3ac700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba41a2ce0 con 0x7ffba40ff4b0 2023-09-10T00:53:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.696+0000 7ffba3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba40ff4b0 0x7ffba4197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.696+0000 7ffba8949700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba4105570 0x7ffba41a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.697+0000 7ffba3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba40ff4b0 0x7ffba4197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60002/0 (socket says 172.21.15.44:60002) 2023-09-10T00:53:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.697+0000 7ffba3fff700 1 -- 172.21.15.44:0/529268628 learned_addr learned my addr 172.21.15.44:0/529268628 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.697+0000 7ffba37fe700 1 --2- 172.21.15.44:0/529268628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffba40ffe70 0x7ffba4197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.697+0000 7ffba3fff700 1 -- 172.21.15.44:0/529268628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffba40ffe70 msgr2=0x7ffba4197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.697+0000 7ffba3fff700 1 --2- 172.21.15.44:0/529268628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffba40ffe70 0x7ffba4197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.697+0000 7ffba3fff700 1 -- 172.21.15.44:0/529268628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba4105570 msgr2=0x7ffba41a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.697+0000 7ffba3fff700 1 --2- 172.21.15.44:0/529268628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba4105570 0x7ffba41a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.697+0000 7ffba3fff700 1 -- 172.21.15.44:0/529268628 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c009c40 con 0x7ffba40ff4b0 2023-09-10T00:53:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.697+0000 7ffba8949700 1 --2- 172.21.15.44:0/529268628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba4105570 0x7ffba41a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.697+0000 7ffba3fff700 1 --2- 172.21.15.44:0/529268628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba40ff4b0 0x7ffba4197540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400e9a0 tx=0x7ffb9400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.697+0000 7ffba17fa700 1 -- 172.21.15.44:0/529268628 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94009790 con 0x7ffba40ff4b0 2023-09-10T00:53:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.697+0000 7ffba17fa700 1 -- 172.21.15.44:0/529268628 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb9400ce50 con 0x7ffba40ff4b0 2023-09-10T00:53:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.697+0000 7ffba17fa700 1 -- 172.21.15.44:0/529268628 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94018420 con 0x7ffba40ff4b0 2023-09-10T00:53:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.697+0000 7ffbaa3ac700 1 -- 172.21.15.44:0/529268628 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb980095f0 con 0x7ffba40ff4b0 2023-09-10T00:53:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.697+0000 7ffbaa3ac700 1 -- 172.21.15.44:0/529268628 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba41a3260 con 0x7ffba40ff4b0 2023-09-10T00:53:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.700+0000 7ffba17fa700 1 -- 172.21.15.44:0/529268628 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ffb94010430 con 0x7ffba40ff4b0 2023-09-10T00:53:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.700+0000 7ffba17fa700 1 --2- 172.21.15.44:0/529268628 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffb8c0a07b0 0x7ffb8c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.701+0000 7ffba17fa700 1 -- 172.21.15.44:0/529268628 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ffb94014070 con 0x7ffba40ff4b0 2023-09-10T00:53:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.701+0000 7ffba37fe700 1 --2- 172.21.15.44:0/529268628 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffb8c0a07b0 0x7ffb8c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.701+0000 7ffba37fe700 1 --2- 172.21.15.44:0/529268628 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffb8c0a07b0 0x7ffb8c0a2c70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffba4069610 tx=0x7ffb98005fb0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.701+0000 7ffbb6572b80 1 Processor -- start 2023-09-10T00:53:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.701+0000 7ffbb6572b80 1 -- start start 2023-09-10T00:53:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffbb6572b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b61a0 0x558f505b6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffbb6572b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0300 0x558f505b0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffbb6572b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 0x558f505bcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffbb6572b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f505096d0 con 0x558f505b0cc0 2023-09-10T00:53:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffbb6572b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f50506ff0 con 0x558f505b61a0 2023-09-10T00:53:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffbb6572b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f505028e0 con 0x558f505b0300 2023-09-10T00:53:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffba3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b61a0 0x558f505b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffba8949700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 0x558f505bcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffba37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0300 0x558f505b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffba3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b61a0 0x558f505b6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54868/0 (socket says 172.21.15.44:54868) 2023-09-10T00:53:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffba3fff700 1 -- 172.21.15.44:0/423529068 learned_addr learned my addr 172.21.15.44:0/423529068 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffba8949700 1 -- 172.21.15.44:0/423529068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0300 msgr2=0x558f505b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffba8949700 1 --2- 172.21.15.44:0/423529068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0300 0x558f505b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffba8949700 1 -- 172.21.15.44:0/423529068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b61a0 msgr2=0x558f505b6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffba8949700 1 --2- 172.21.15.44:0/423529068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b61a0 0x558f505b6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffba8949700 1 -- 172.21.15.44:0/423529068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f5032ed80 con 0x558f505b0cc0 2023-09-10T00:53:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffba37fe700 1 --2- 172.21.15.44:0/423529068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0300 0x558f505b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:39.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.702+0000 7ffba8949700 1 --2- 172.21.15.44:0/423529068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 0x558f505bcf30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0094d0 tx=0x7ffb9c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274edee0b8dd4f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:39.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.703+0000 7ffb8a7fc700 1 -- 172.21.15.44:0/423529068 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c004db0 con 0x558f505b0cc0 2023-09-10T00:53:39.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.703+0000 7ffb8a7fc700 1 -- 172.21.15.44:0/423529068 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb9c00f040 con 0x558f505b0cc0 2023-09-10T00:53:39.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.703+0000 7ffb8a7fc700 1 -- 172.21.15.44:0/423529068 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c005420 con 0x558f505b0cc0 2023-09-10T00:53:39.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.703+0000 7ffbb6572b80 1 -- 172.21.15.44:0/423529068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 msgr2=0x558f505bcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.703+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/423529068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 0x558f505bcf30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0094d0 tx=0x7ffb9c009920 comp rx=0 tx=0).stop 2023-09-10T00:53:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.704+0000 7ffbb6572b80 1 -- 172.21.15.44:0/423529068 shutdown_connections 2023-09-10T00:53:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.704+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/423529068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 0x558f505bcf30 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.704+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/423529068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b61a0 0x558f505b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.704+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/423529068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0300 0x558f505b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.704+0000 7ffbb6572b80 1 -- 172.21.15.44:0/423529068 >> 172.21.15.44:0/423529068 conn(0x558f50525030 msgr2=0x558f505ab560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.704+0000 7ffbb6572b80 1 -- 172.21.15.44:0/423529068 shutdown_connections 2023-09-10T00:53:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.704+0000 7ffbb6572b80 1 -- 172.21.15.44:0/423529068 wait complete. 2023-09-10T00:53:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.704+0000 7ffbb6572b80 1 -- start start 2023-09-10T00:53:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.704+0000 7ffbb6572b80 1 Processor -- start 2023-09-10T00:53:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffbb6572b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0300 0x558f505bcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffbb6572b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 0x558f505b9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffbb6572b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b61a0 0x558f505ba550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffbb6572b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f504a0b10 con 0x558f505b0cc0 2023-09-10T00:53:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffbb6572b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f504a0810 con 0x558f505b61a0 2023-09-10T00:53:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffbb6572b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f503848c0 con 0x558f505b0300 2023-09-10T00:53:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffbb6572b80 10 client.0 _ll_register_callbacks cb 0x558f503eaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:53:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffba37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 0x558f505b9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffba37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 0x558f505b9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53136/0 (socket says 172.21.15.44:53136) 2023-09-10T00:53:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffba37fe700 1 -- 172.21.15.44:0/3475243103 learned_addr learned my addr 172.21.15.44:0/3475243103 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffba3fff700 1 --2- 172.21.15.44:0/3475243103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0300 0x558f505bcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffba8949700 1 --2- 172.21.15.44:0/3475243103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b61a0 0x558f505ba550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffba3fff700 1 -- 172.21.15.44:0/3475243103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b61a0 msgr2=0x558f505ba550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffba3fff700 1 --2- 172.21.15.44:0/3475243103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b61a0 0x558f505ba550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffba3fff700 1 -- 172.21.15.44:0/3475243103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 msgr2=0x558f505b9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffba3fff700 1 --2- 172.21.15.44:0/3475243103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 0x558f505b9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffba3fff700 1 -- 172.21.15.44:0/3475243103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba410cc30 con 0x558f505b0300 2023-09-10T00:53:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffba37fe700 1 --2- 172.21.15.44:0/3475243103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 0x558f505b9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:53:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffba3fff700 1 --2- 172.21.15.44:0/3475243103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0300 0x558f505bcdc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffb94096010 tx=0x7ffb94010790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=334d5c48ea96c006 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.705+0000 7ffb837fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:53:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.706+0000 7ffb837fe700 1 -- 172.21.15.44:0/3475243103 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9409d3d0 con 0x558f505b0300 2023-09-10T00:53:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.706+0000 7ffb837fe700 1 -- 172.21.15.44:0/3475243103 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ffb940a1610 con 0x558f505b0300 2023-09-10T00:53:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.706+0000 7ffbb6572b80 1 -- 172.21.15.44:0/3475243103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f5045a490 con 0x558f505b0300 2023-09-10T00:53:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.706+0000 7ffb837fe700 1 -- 172.21.15.44:0/3475243103 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb94099a10 con 0x558f505b0300 2023-09-10T00:53:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.706+0000 7ffbb6572b80 10 client.27241 fetch_fsmap learned FSMap version 66 2023-09-10T00:53:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.706+0000 7ffbb6572b80 1 -- 172.21.15.44:0/3475243103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x7ffb9c0091b0 con 0x558f505b0300 2023-09-10T00:53:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.706+0000 7ffb837fe700 1 -- 172.21.15.44:0/3475243103 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558f5045a490 con 0x558f505b0300 2023-09-10T00:53:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.706+0000 7ffb837fe700 1 -- 172.21.15.44:0/3475243103 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 66) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ffb940650f0 con 0x558f505b0300 2023-09-10T00:53:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.706+0000 7ffbb6572b80 10 client.27241 fetch_fsmap finished waiting for FSMap version 66 2023-09-10T00:53:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.706+0000 7ffbb6572b80 10 client.27241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:53:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.706+0000 7ffbb6572b80 20 client.27241 populate_metadata read hostname 'smithi044' 2023-09-10T00:53:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.706+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/3475243103 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f505ca420 0x558f505cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.706+0000 7ffbb6572b80 4 client.27241 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:53:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.706+0000 7ffbb6572b80 1 -- 172.21.15.44:0/3475243103 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f50517840 con 0x558f505ca420 2023-09-10T00:53:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.706+0000 7ffba37fe700 1 --2- 172.21.15.44:0/3475243103 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f505ca420 0x558f505cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.707+0000 7ffba37fe700 1 --2- 172.21.15.44:0/3475243103 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f505ca420 0x558f505cc840 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=568b9f5e76eacc30 server_cookie=e2f6d8056be8bb63 in_seq=0 out_seq=0 2023-09-10T00:53:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.707+0000 7ffb837fe700 10 client.27241 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.707+0000 7ffb837fe700 1 -- 172.21.15.44:0/3475243103 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558f50517840 con 0x558f505ca420 2023-09-10T00:53:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.707+0000 7ffb837fe700 10 client.27241 handle_command_reply: tid=0 2023-09-10T00:53:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.707+0000 7ffbb6572b80 1 client.27241 shutdown 2023-09-10T00:53:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.708+0000 7ffbb6572b80 1 -- 172.21.15.44:0/3475243103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0300 msgr2=0x558f505bcdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.708+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/3475243103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0300 0x558f505bcdc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffb94096010 tx=0x7ffb94010790 comp rx=0 tx=0).stop 2023-09-10T00:53:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.708+0000 7ffbb6572b80 1 -- 172.21.15.44:0/3475243103 shutdown_connections 2023-09-10T00:53:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.708+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/3475243103 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f505ca420 0x558f505cc840 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.708+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/3475243103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b61a0 0x558f505ba550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.708+0000 7ffb837fe700 0 client.27241 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.708+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/3475243103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0300 0x558f505bcdc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.708+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/3475243103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 0x558f505b9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.708+0000 7ffbb6572b80 1 -- 172.21.15.44:0/3475243103 >> 172.21.15.44:0/3475243103 conn(0x558f50525030 msgr2=0x558f505aa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.708+0000 7ffbb6572b80 1 -- 172.21.15.44:0/3475243103 shutdown_connections 2023-09-10T00:53:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.708+0000 7ffbb6572b80 1 -- 172.21.15.44:0/3475243103 wait complete. 2023-09-10T00:53:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.708+0000 7ffbb6572b80 20 client.27241 trim_cache size 0 max 16384 2023-09-10T00:53:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.725+0000 7ffbb6572b80 1 Processor -- start 2023-09-10T00:53:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.725+0000 7ffbb6572b80 1 -- start start 2023-09-10T00:53:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffbb6572b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b0300 0x558f505b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffbb6572b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 0x558f505bc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffbb6572b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b61a0 0x558f505aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffbb6572b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f505a58a0 con 0x558f505b0cc0 2023-09-10T00:53:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffbb6572b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f505a7fc0 con 0x558f505b0300 2023-09-10T00:53:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffbb6572b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f505bab30 con 0x558f505b61a0 2023-09-10T00:53:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffba8949700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b61a0 0x558f505aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffba3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b0300 0x558f505b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffba8949700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b61a0 0x558f505aaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60026/0 (socket says 172.21.15.44:60026) 2023-09-10T00:53:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffba8949700 1 -- 172.21.15.44:0/3132527726 learned_addr learned my addr 172.21.15.44:0/3132527726 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffba3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b0300 0x558f505b57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54882/0 (socket says 172.21.15.44:54882) 2023-09-10T00:53:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffba37fe700 1 --2- 172.21.15.44:0/3132527726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 0x558f505bc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffba8949700 1 -- 172.21.15.44:0/3132527726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b0300 msgr2=0x558f505b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffba8949700 1 --2- 172.21.15.44:0/3132527726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b0300 0x558f505b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffba8949700 1 -- 172.21.15.44:0/3132527726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 msgr2=0x558f505bc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffba8949700 1 --2- 172.21.15.44:0/3132527726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 0x558f505bc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffba8949700 1 -- 172.21.15.44:0/3132527726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f505ba380 con 0x558f505b61a0 2023-09-10T00:53:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffba3fff700 1 --2- 172.21.15.44:0/3132527726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b0300 0x558f505b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffba8949700 1 --2- 172.21.15.44:0/3132527726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b61a0 0x558f505aaf20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009180 tx=0x7ffb9c007420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bca313b27a3ad7c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.726+0000 7ffb8affd700 1 -- 172.21.15.44:0/3132527726 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c003d40 con 0x558f505b61a0 2023-09-10T00:53:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.727+0000 7ffb8affd700 1 -- 172.21.15.44:0/3132527726 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb9c023040 con 0x558f505b61a0 2023-09-10T00:53:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.727+0000 7ffb8affd700 1 -- 172.21.15.44:0/3132527726 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c0286a0 con 0x558f505b61a0 2023-09-10T00:53:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.727+0000 7ffbb6572b80 1 -- 172.21.15.44:0/3132527726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b61a0 msgr2=0x558f505aaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.727+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/3132527726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b61a0 0x558f505aaf20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009180 tx=0x7ffb9c007420 comp rx=0 tx=0).stop 2023-09-10T00:53:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.727+0000 7ffbb6572b80 1 -- 172.21.15.44:0/3132527726 shutdown_connections 2023-09-10T00:53:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.727+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/3132527726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b61a0 0x558f505aaf20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.727+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/3132527726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b0300 0x558f505b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.727+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/3132527726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b0cc0 0x558f505bc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.727+0000 7ffbb6572b80 1 -- 172.21.15.44:0/3132527726 >> 172.21.15.44:0/3132527726 conn(0x558f50525030 msgr2=0x558f505afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.727+0000 7ffbb6572b80 1 -- 172.21.15.44:0/3132527726 shutdown_connections 2023-09-10T00:53:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.727+0000 7ffbb6572b80 1 -- 172.21.15.44:0/3132527726 wait complete. 2023-09-10T00:53:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.728+0000 7ffbb6572b80 1 -- start start 2023-09-10T00:53:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.728+0000 7ffbb6572b80 1 Processor -- start 2023-09-10T00:53:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.728+0000 7ffbb6572b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b0300 0x558f505c01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.728+0000 7ffbb6572b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0cc0 0x558f505c0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.728+0000 7ffbb6572b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b61a0 0x558f5061e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.728+0000 7ffbb6572b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f505c0e40 con 0x558f505b61a0 2023-09-10T00:53:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.728+0000 7ffbb6572b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5061e780 con 0x558f505b0300 2023-09-10T00:53:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.728+0000 7ffbb6572b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5061e8f0 con 0x558f505b0cc0 2023-09-10T00:53:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.728+0000 7ffbb6572b80 10 client.0 _ll_register_callbacks cb 0x558f503eaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:53:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.728+0000 7ffba3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b0300 0x558f505c01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.728+0000 7ffba8949700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b61a0 0x558f5061e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.728+0000 7ffba37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0cc0 0x558f505c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.728+0000 7ffba3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b0300 0x558f505c01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54888/0 (socket says 172.21.15.44:54888) 2023-09-10T00:53:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.728+0000 7ffba37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0cc0 0x558f505c0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60032/0 (socket says 172.21.15.44:60032) 2023-09-10T00:53:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.729+0000 7ffba3fff700 1 -- 172.21.15.44:0/1335393169 learned_addr learned my addr 172.21.15.44:0/1335393169 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.729+0000 7ffba8949700 1 -- 172.21.15.44:0/1335393169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0cc0 msgr2=0x558f505c0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.729+0000 7ffba8949700 1 --2- 172.21.15.44:0/1335393169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0cc0 0x558f505c0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.729+0000 7ffba8949700 1 -- 172.21.15.44:0/1335393169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b0300 msgr2=0x558f505c01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.729+0000 7ffba8949700 1 --2- 172.21.15.44:0/1335393169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b0300 0x558f505c01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.729+0000 7ffba8949700 1 -- 172.21.15.44:0/1335393169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9c0091b0 con 0x558f505b61a0 2023-09-10T00:53:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.729+0000 7ffba3fff700 1 --2- 172.21.15.44:0/1335393169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b0300 0x558f505c01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.729+0000 7ffba8949700 1 --2- 172.21.15.44:0/1335393169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b61a0 0x558f5061e240 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c007c20 tx=0x7ffb9c0060b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c595f4d2da0dc1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.729+0000 7ffb837fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:53:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.729+0000 7ffb837fe700 1 -- 172.21.15.44:0/1335393169 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c00e9c0 con 0x558f505b61a0 2023-09-10T00:53:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.729+0000 7ffb837fe700 1 -- 172.21.15.44:0/1335393169 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ffb9c028440 con 0x558f505b61a0 2023-09-10T00:53:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.729+0000 7ffbb6572b80 1 -- 172.21.15.44:0/1335393169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb9c00f040 con 0x558f505b61a0 2023-09-10T00:53:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.729+0000 7ffb837fe700 1 -- 172.21.15.44:0/1335393169 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb9c028600 con 0x558f505b61a0 2023-09-10T00:53:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.729+0000 7ffbb6572b80 10 client.17646 fetch_fsmap learned FSMap version 66 2023-09-10T00:53:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.729+0000 7ffbb6572b80 1 -- 172.21.15.44:0/1335393169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x558f5032ed80 con 0x558f505b61a0 2023-09-10T00:53:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.730+0000 7ffb837fe700 1 -- 172.21.15.44:0/1335393169 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb9c00f040 con 0x558f505b61a0 2023-09-10T00:53:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.730+0000 7ffb837fe700 1 -- 172.21.15.44:0/1335393169 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 66) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ffb9c0157e0 con 0x558f505b61a0 2023-09-10T00:53:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.730+0000 7ffbb6572b80 10 client.17646 fetch_fsmap finished waiting for FSMap version 66 2023-09-10T00:53:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.730+0000 7ffbb6572b80 10 client.17646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:53:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.730+0000 7ffbb6572b80 20 client.17646 populate_metadata read hostname 'smithi044' 2023-09-10T00:53:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.730+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/1335393169 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f5061fc20 0x558f50622040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.730+0000 7ffbb6572b80 4 client.17646 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:53:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.730+0000 7ffbb6572b80 1 -- 172.21.15.44:0/1335393169 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558f504fb8c0 con 0x558f5061fc20 2023-09-10T00:53:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.730+0000 7ffba3fff700 1 --2- 172.21.15.44:0/1335393169 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f5061fc20 0x558f50622040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.731+0000 7ffb837fe700 10 client.17646 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.731+0000 7ffba3fff700 1 --2- 172.21.15.44:0/1335393169 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f5061fc20 0x558f50622040 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa482f26c9026923 server_cookie=52b7017b0ea9b9ff in_seq=0 out_seq=0 2023-09-10T00:53:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.741+0000 7ffb837fe700 1 -- 172.21.15.44:0/1335393169 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558f504fb8c0 con 0x558f5061fc20 2023-09-10T00:53:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.741+0000 7ffb837fe700 10 client.17646 handle_command_reply: tid=0 2023-09-10T00:53:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.741+0000 7ffbb6572b80 1 client.17646 shutdown 2023-09-10T00:53:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.742+0000 7ffbb6572b80 1 -- 172.21.15.44:0/1335393169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b61a0 msgr2=0x558f5061e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.742+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/1335393169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b61a0 0x558f5061e240 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c007c20 tx=0x7ffb9c0060b0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.742+0000 7ffbb6572b80 1 -- 172.21.15.44:0/1335393169 shutdown_connections 2023-09-10T00:53:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.742+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/1335393169 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558f5061fc20 0x558f50622040 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.742+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/1335393169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f505b61a0 0x558f5061e240 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.742+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/1335393169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f505b0300 0x558f505c01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.742+0000 7ffb837fe700 0 client.17646 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.742+0000 7ffbb6572b80 1 --2- 172.21.15.44:0/1335393169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558f505b0cc0 0x558f505c0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.742+0000 7ffbb6572b80 1 -- 172.21.15.44:0/1335393169 >> 172.21.15.44:0/1335393169 conn(0x558f50525030 msgr2=0x558f505afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.742+0000 7ffbb6572b80 1 -- 172.21.15.44:0/1335393169 shutdown_connections 2023-09-10T00:53:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.742+0000 7ffbb6572b80 1 -- 172.21.15.44:0/1335393169 wait complete. 2023-09-10T00:53:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.742+0000 7ffbb6572b80 20 client.17646 trim_cache size 0 max 16384 2023-09-10T00:53:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:53:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:53:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:53:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:53:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.742+0000 7ffbaa3ac700 1 -- 172.21.15.44:0/529268628 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffb8c0a07b0 msgr2=0x7ffb8c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.742+0000 7ffbaa3ac700 1 --2- 172.21.15.44:0/529268628 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffb8c0a07b0 0x7ffb8c0a2c70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffba4069610 tx=0x7ffb98005fb0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.742+0000 7ffbaa3ac700 1 -- 172.21.15.44:0/529268628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba40ff4b0 msgr2=0x7ffba4197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.742+0000 7ffbaa3ac700 1 --2- 172.21.15.44:0/529268628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba40ff4b0 0x7ffba4197540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400e9a0 tx=0x7ffb9400ecb0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.743+0000 7ffbaa3ac700 1 -- 172.21.15.44:0/529268628 shutdown_connections 2023-09-10T00:53:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.743+0000 7ffbaa3ac700 1 --2- 172.21.15.44:0/529268628 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffb8c0a07b0 0x7ffb8c0a2c70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.743+0000 7ffbaa3ac700 1 --2- 172.21.15.44:0/529268628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba4105570 0x7ffba41a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.743+0000 7ffbaa3ac700 1 --2- 172.21.15.44:0/529268628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba40ff4b0 0x7ffba4197540 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.743+0000 7ffbaa3ac700 1 --2- 172.21.15.44:0/529268628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffba40ffe70 0x7ffba4197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.743+0000 7ffbaa3ac700 1 -- 172.21.15.44:0/529268628 >> 172.21.15.44:0/529268628 conn(0x7ffba40faf60 msgr2=0x7ffba40fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.743+0000 7ffbaa3ac700 1 -- 172.21.15.44:0/529268628 shutdown_connections 2023-09-10T00:53:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:39.743+0000 7ffbaa3ac700 1 -- 172.21.15.44:0/529268628 wait complete. 2023-09-10T00:53:39.937 INFO:teuthology.orchestra.run.smithi044.stderr:49600000 of 50000000 tuples (99%) done (elapsed 759.57 s, remaining 6.13 s) 2023-09-10T00:53:40.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:40 smithi131 ceph-mon[103713]: pgmap v407: 129 pgs: 129 active+clean; 9.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-09-10T00:53:40.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:40 smithi044 ceph-mon[98676]: pgmap v407: 129 pgs: 129 active+clean; 9.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-09-10T00:53:40.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:40 smithi099 ceph-mon[102692]: pgmap v407: 129 pgs: 129 active+clean; 9.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-09-10T00:53:40.925 INFO:teuthology.orchestra.run.smithi044.stderr:49700000 of 50000000 tuples (99%) done (elapsed 760.56 s, remaining 4.59 s) 2023-09-10T00:53:41.491 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:53:41.646 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.645+0000 7fabb4011700 1 -- 172.21.15.44:0/262320068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabac0ffe00 msgr2=0x7fabac10c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:41.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.645+0000 7fabb4011700 1 --2- 172.21.15.44:0/262320068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabac0ffe00 0x7fabac10c310 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7faba400b0a0 tx=0x7faba400b3b0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.645+0000 7fabb4011700 1 -- 172.21.15.44:0/262320068 shutdown_connections 2023-09-10T00:53:41.647 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.645+0000 7fabb4011700 1 --2- 172.21.15.44:0/262320068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabac0ffe00 0x7fabac10c310 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.645+0000 7fabb4011700 1 --2- 172.21.15.44:0/262320068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabac1054e0 0x7fabac1058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.645+0000 7fabb4011700 1 --2- 172.21.15.44:0/262320068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabac0ff440 0x7fabac0ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.645+0000 7fabb4011700 1 -- 172.21.15.44:0/262320068 >> 172.21.15.44:0/262320068 conn(0x7fabac0faf50 msgr2=0x7fabac0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:41.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.645+0000 7fabb4011700 1 -- 172.21.15.44:0/262320068 shutdown_connections 2023-09-10T00:53:41.648 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.645+0000 7fabb4011700 1 -- 172.21.15.44:0/262320068 wait complete. 2023-09-10T00:53:41.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.646+0000 7fabb4011700 1 Processor -- start 2023-09-10T00:53:41.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.646+0000 7fabb4011700 1 -- start start 2023-09-10T00:53:41.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.646+0000 7fabb4011700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabac0ff440 0x7fabac195420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.646+0000 7fabb4011700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabac0ffe00 0x7fabac195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.649 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.646+0000 7fabb4011700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabac1054e0 0x7fabac199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.646+0000 7fabb4011700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac19a400 con 0x7fabac0ff440 2023-09-10T00:53:41.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.646+0000 7fabb4011700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac19a570 con 0x7fabac1054e0 2023-09-10T00:53:41.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.646+0000 7fabb4011700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac19a730 con 0x7fabac0ffe00 2023-09-10T00:53:41.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.646+0000 7fabb1dad700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabac0ff440 0x7fabac195420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.650 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.646+0000 7fabb15ac700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabac0ffe00 0x7fabac195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.646+0000 7fabb1dad700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabac0ff440 0x7fabac195420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53162/0 (socket says 172.21.15.44:53162) 2023-09-10T00:53:41.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.646+0000 7fabb1dad700 1 -- 172.21.15.44:0/3600243106 learned_addr learned my addr 172.21.15.44:0/3600243106 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:41.651 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.646+0000 7fabb25ae700 1 --2- 172.21.15.44:0/3600243106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabac1054e0 0x7fabac199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.647+0000 7fabb1dad700 1 -- 172.21.15.44:0/3600243106 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabac0ffe00 msgr2=0x7fabac195960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:41.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.647+0000 7fabb1dad700 1 --2- 172.21.15.44:0/3600243106 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabac0ffe00 0x7fabac195960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.647+0000 7fabb1dad700 1 -- 172.21.15.44:0/3600243106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabac1054e0 msgr2=0x7fabac199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:41.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.647+0000 7fabb1dad700 1 --2- 172.21.15.44:0/3600243106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabac1054e0 0x7fabac199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.665 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.647+0000 7fabb1dad700 1 -- 172.21.15.44:0/3600243106 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba4009d00 con 0x7fabac0ff440 2023-09-10T00:53:41.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.647+0000 7fabb1dad700 1 --2- 172.21.15.44:0/3600243106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabac0ff440 0x7fabac195420 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7faba800e9a0 tx=0x7faba800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:41.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.647+0000 7faba2ffd700 1 -- 172.21.15.44:0/3600243106 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8009790 con 0x7fabac0ff440 2023-09-10T00:53:41.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.647+0000 7faba2ffd700 1 -- 172.21.15.44:0/3600243106 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faba800dcb0 con 0x7fabac0ff440 2023-09-10T00:53:41.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.647+0000 7fabb4011700 1 -- 172.21.15.44:0/3600243106 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabac19aa10 con 0x7fabac0ff440 2023-09-10T00:53:41.666 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.647+0000 7faba2ffd700 1 -- 172.21.15.44:0/3600243106 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8016dd0 con 0x7fabac0ff440 2023-09-10T00:53:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.647+0000 7fabb4011700 1 -- 172.21.15.44:0/3600243106 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabac067940 con 0x7fabac0ff440 2023-09-10T00:53:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.650+0000 7faba2ffd700 1 -- 172.21.15.44:0/3600243106 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7faba800d820 con 0x7fabac0ff440 2023-09-10T00:53:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.650+0000 7faba2ffd700 1 --2- 172.21.15.44:0/3600243106 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fab980a0830 0x7fab980a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.651+0000 7fabb15ac700 1 --2- 172.21.15.44:0/3600243106 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fab980a0830 0x7fab980a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.667 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.651+0000 7faba2ffd700 1 -- 172.21.15.44:0/3600243106 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7faba80cbac0 con 0x7fabac0ff440 2023-09-10T00:53:41.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.651+0000 7fabc01d7b80 1 Processor -- start 2023-09-10T00:53:41.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.651+0000 7fabb15ac700 1 --2- 172.21.15.44:0/3600243106 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fab980a0830 0x7fab980a2cf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fabac196a30 tx=0x7fab9c012040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:41.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.651+0000 7fabc01d7b80 1 -- start start 2023-09-10T00:53:41.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabc01d7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e1710 0x55d8617e1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.668 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabc01d7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617e2030 0x55d8617eb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabc01d7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617ebe40 0x55d8617ee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabc01d7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86173a730 con 0x55d8617e2030 2023-09-10T00:53:41.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabc01d7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d861738050 con 0x55d8617e1710 2023-09-10T00:53:41.669 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabc01d7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d861733940 con 0x55d8617ebe40 2023-09-10T00:53:41.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabb15ac700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617e2030 0x55d8617eb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabb15ac700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617e2030 0x55d8617eb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53172/0 (socket says 172.21.15.44:53172) 2023-09-10T00:53:41.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabb1dad700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e1710 0x55d8617e1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabb15ac700 1 -- 172.21.15.44:0/2108638390 learned_addr learned my addr 172.21.15.44:0/2108638390 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:41.670 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabb25ae700 1 --2- 172.21.15.44:0/2108638390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617ebe40 0x55d8617ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabb1dad700 1 -- 172.21.15.44:0/2108638390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617ebe40 msgr2=0x55d8617ee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:41.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabb1dad700 1 --2- 172.21.15.44:0/2108638390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617ebe40 0x55d8617ee230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabb1dad700 1 -- 172.21.15.44:0/2108638390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617e2030 msgr2=0x55d8617eb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:41.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabb1dad700 1 --2- 172.21.15.44:0/2108638390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617e2030 0x55d8617eb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.671 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabb1dad700 1 -- 172.21.15.44:0/2108638390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d86155fde0 con 0x55d8617e1710 2023-09-10T00:53:41.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabb15ac700 1 --2- 172.21.15.44:0/2108638390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617e2030 0x55d8617eb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:41.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabb25ae700 1 --2- 172.21.15.44:0/2108638390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617ebe40 0x55d8617ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:41.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.652+0000 7fabb1dad700 1 --2- 172.21.15.44:0/2108638390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e1710 0x55d8617e1af0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faba80c8db0 tx=0x7faba8098a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31653ee316730c95 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:41.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.653+0000 7fab97fff700 1 -- 172.21.15.44:0/2108638390 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8099210 con 0x55d8617e1710 2023-09-10T00:53:41.672 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.653+0000 7fab97fff700 1 -- 172.21.15.44:0/2108638390 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faba800de20 con 0x55d8617e1710 2023-09-10T00:53:41.673 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.653+0000 7fab97fff700 1 -- 172.21.15.44:0/2108638390 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8098d70 con 0x55d8617e1710 2023-09-10T00:53:41.673 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.653+0000 7fabc01d7b80 1 -- 172.21.15.44:0/2108638390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e1710 msgr2=0x55d8617e1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:41.673 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.653+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/2108638390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e1710 0x55d8617e1af0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faba80c8db0 tx=0x7faba8098a10 comp rx=0 tx=0).stop 2023-09-10T00:53:41.673 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.653+0000 7fabc01d7b80 1 -- 172.21.15.44:0/2108638390 shutdown_connections 2023-09-10T00:53:41.673 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.653+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/2108638390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617ebe40 0x55d8617ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.653+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/2108638390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e1710 0x55d8617e1af0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.653+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/2108638390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617e2030 0x55d8617eb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.653+0000 7fabc01d7b80 1 -- 172.21.15.44:0/2108638390 >> 172.21.15.44:0/2108638390 conn(0x55d861756090 msgr2=0x55d8617dcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:41.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.654+0000 7fabc01d7b80 1 -- 172.21.15.44:0/2108638390 shutdown_connections 2023-09-10T00:53:41.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.654+0000 7fabc01d7b80 1 -- 172.21.15.44:0/2108638390 wait complete. 2023-09-10T00:53:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.654+0000 7fabc01d7b80 1 -- start start 2023-09-10T00:53:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.654+0000 7fabc01d7b80 1 Processor -- start 2023-09-10T00:53:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.654+0000 7fabc01d7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617e1710 0x55d8617e8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.654+0000 7fabc01d7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e2030 0x55d8617e9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.654+0000 7fabc01d7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617ebe40 0x55d8617e96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabc01d7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8616d1b70 con 0x55d8617ebe40 2023-09-10T00:53:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabc01d7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8616d1870 con 0x55d8617e2030 2023-09-10T00:53:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabc01d7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8615b5920 con 0x55d8617e1710 2023-09-10T00:53:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabc01d7b80 10 client.0 _ll_register_callbacks cb 0x55d86161ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:53:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabb15ac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e2030 0x55d8617e9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabb1dad700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617e1710 0x55d8617e8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabb15ac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e2030 0x55d8617e9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54918/0 (socket says 172.21.15.44:54918) 2023-09-10T00:53:41.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabb15ac700 1 -- 172.21.15.44:0/3869658520 learned_addr learned my addr 172.21.15.44:0/3869658520 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:41.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabb25ae700 1 --2- 172.21.15.44:0/3869658520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617ebe40 0x55d8617e96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabb15ac700 1 -- 172.21.15.44:0/3869658520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617e1710 msgr2=0x55d8617e8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:41.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabb15ac700 1 --2- 172.21.15.44:0/3869658520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617e1710 0x55d8617e8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabb15ac700 1 -- 172.21.15.44:0/3869658520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617ebe40 msgr2=0x55d8617e96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:41.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabb15ac700 1 --2- 172.21.15.44:0/3869658520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617ebe40 0x55d8617e96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabb15ac700 1 -- 172.21.15.44:0/3869658520 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c009dc0 con 0x55d8617e2030 2023-09-10T00:53:41.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabb25ae700 1 --2- 172.21.15.44:0/3869658520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617ebe40 0x55d8617e96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:41.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabb15ac700 1 --2- 172.21.15.44:0/3869658520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e2030 0x55d8617e9160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab9c003910 tx=0x7fab9c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a1999b8a5117d7f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:41.679 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fab957fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:53:41.679 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fab957fa700 1 -- 172.21.15.44:0/3869658520 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c01c6f0 con 0x55d8617e2030 2023-09-10T00:53:41.679 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fab957fa700 1 -- 172.21.15.44:0/3869658520 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fab9c01c850 con 0x55d8617e2030 2023-09-10T00:53:41.679 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.655+0000 7fabc01d7b80 1 -- 172.21.15.44:0/3869658520 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d86168b4f0 con 0x55d8617e2030 2023-09-10T00:53:41.679 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.656+0000 7fab957fa700 1 -- 172.21.15.44:0/3869658520 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab9c01ca10 con 0x55d8617e2030 2023-09-10T00:53:41.679 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.656+0000 7fabc01d7b80 10 client.27323 fetch_fsmap learned FSMap version 66 2023-09-10T00:53:41.680 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.656+0000 7fabc01d7b80 1 -- 172.21.15.44:0/3869658520 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x7fabac10cc10 con 0x55d8617e2030 2023-09-10T00:53:41.680 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.656+0000 7fab957fa700 1 -- 172.21.15.44:0/3869658520 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d86168b4f0 con 0x55d8617e2030 2023-09-10T00:53:41.680 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.656+0000 7fab957fa700 1 -- 172.21.15.44:0/3869658520 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 66) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fab9c0303b0 con 0x55d8617e2030 2023-09-10T00:53:41.680 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.656+0000 7fabc01d7b80 10 client.27323 fetch_fsmap finished waiting for FSMap version 66 2023-09-10T00:53:41.680 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.656+0000 7fabc01d7b80 10 client.27323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:53:41.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.656+0000 7fabc01d7b80 20 client.27323 populate_metadata read hostname 'smithi044' 2023-09-10T00:53:41.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.656+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/3869658520 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d8617fbb90 0x55d8617fdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.656+0000 7fabc01d7b80 4 client.27323 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:53:41.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.656+0000 7fabc01d7b80 1 -- 172.21.15.44:0/3869658520 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8617488a0 con 0x55d8617fbb90 2023-09-10T00:53:41.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.656+0000 7fabb1dad700 1 --2- 172.21.15.44:0/3869658520 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d8617fbb90 0x55d8617fdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.682 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.657+0000 7fab957fa700 10 client.27323 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:41.682 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.657+0000 7fabb1dad700 1 --2- 172.21.15.44:0/3869658520 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d8617fbb90 0x55d8617fdfb0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d015084aec84a1 server_cookie=c0fb316352e5460e in_seq=0 out_seq=0 2023-09-10T00:53:41.682 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.657+0000 7fab957fa700 1 -- 172.21.15.44:0/3869658520 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d8617488a0 con 0x55d8617fbb90 2023-09-10T00:53:41.682 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.657+0000 7fab957fa700 10 client.27323 handle_command_reply: tid=0 2023-09-10T00:53:41.682 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.657+0000 7fabc01d7b80 1 client.27323 shutdown 2023-09-10T00:53:41.682 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.657+0000 7fabc01d7b80 1 -- 172.21.15.44:0/3869658520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e2030 msgr2=0x55d8617e9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:41.683 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.657+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/3869658520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e2030 0x55d8617e9160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab9c003910 tx=0x7fab9c004750 comp rx=0 tx=0).stop 2023-09-10T00:53:41.683 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.658+0000 7fabc01d7b80 1 -- 172.21.15.44:0/3869658520 shutdown_connections 2023-09-10T00:53:41.683 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.658+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/3869658520 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d8617fbb90 0x55d8617fdfb0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.683 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.658+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/3869658520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617ebe40 0x55d8617e96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.683 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.658+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/3869658520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617e1710 0x55d8617e8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.684 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.658+0000 7fab957fa700 0 client.27323 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:41.684 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.658+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/3869658520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e2030 0x55d8617e9160 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.684 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.658+0000 7fabc01d7b80 1 -- 172.21.15.44:0/3869658520 >> 172.21.15.44:0/3869658520 conn(0x55d861756090 msgr2=0x55d8617dc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:41.684 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.658+0000 7fabc01d7b80 1 -- 172.21.15.44:0/3869658520 shutdown_connections 2023-09-10T00:53:41.684 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.658+0000 7fabc01d7b80 1 -- 172.21.15.44:0/3869658520 wait complete. 2023-09-10T00:53:41.684 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.658+0000 7fabc01d7b80 20 client.27323 trim_cache size 0 max 16384 2023-09-10T00:53:41.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.679+0000 7fabc01d7b80 1 Processor -- start 2023-09-10T00:53:41.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.679+0000 7fabc01d7b80 1 -- start start 2023-09-10T00:53:41.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.679+0000 7fabc01d7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617e1710 0x55d8617f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.679+0000 7fabc01d7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e2030 0x55d8617e3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.690 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.679+0000 7fabc01d7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617ebe40 0x55d8617e43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.679+0000 7fabc01d7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8617f8470 con 0x55d8617ebe40 2023-09-10T00:53:41.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.679+0000 7fabc01d7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8617de1e0 con 0x55d8617e2030 2023-09-10T00:53:41.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.679+0000 7fabc01d7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8617f2f30 con 0x55d8617e1710 2023-09-10T00:53:41.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.680+0000 7fabb15ac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e2030 0x55d8617e3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.691 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.680+0000 7fabb15ac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e2030 0x55d8617e3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54926/0 (socket says 172.21.15.44:54926) 2023-09-10T00:53:41.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.680+0000 7fabb15ac700 1 -- 172.21.15.44:0/1220680749 learned_addr learned my addr 172.21.15.44:0/1220680749 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:41.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.680+0000 7fabb25ae700 1 --2- 172.21.15.44:0/1220680749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617ebe40 0x55d8617e43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.680+0000 7fabb1dad700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617e1710 0x55d8617f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.680+0000 7fabb15ac700 1 -- 172.21.15.44:0/1220680749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617e1710 msgr2=0x55d8617f29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:41.692 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.680+0000 7fabb15ac700 1 --2- 172.21.15.44:0/1220680749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617e1710 0x55d8617f29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.680+0000 7fabb15ac700 1 -- 172.21.15.44:0/1220680749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617ebe40 msgr2=0x55d8617e43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:41.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.680+0000 7fabb15ac700 1 --2- 172.21.15.44:0/1220680749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617ebe40 0x55d8617e43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.680+0000 7fabb15ac700 1 -- 172.21.15.44:0/1220680749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac067940 con 0x55d8617e2030 2023-09-10T00:53:41.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.680+0000 7fabb25ae700 1 --2- 172.21.15.44:0/1220680749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617ebe40 0x55d8617e43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:41.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.680+0000 7fabb15ac700 1 --2- 172.21.15.44:0/1220680749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e2030 0x55d8617e3e60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fab9c000c00 tx=0x7fab9c009350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5305c4e9f3700566 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:41.693 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.680+0000 7faba0ff9700 1 -- 172.21.15.44:0/1220680749 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c010ab0 con 0x55d8617e2030 2023-09-10T00:53:41.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.680+0000 7faba0ff9700 1 -- 172.21.15.44:0/1220680749 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab9c016040 con 0x55d8617e2030 2023-09-10T00:53:41.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.680+0000 7faba0ff9700 1 -- 172.21.15.44:0/1220680749 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c039b20 con 0x55d8617e2030 2023-09-10T00:53:41.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.681+0000 7fabc01d7b80 1 -- 172.21.15.44:0/1220680749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e2030 msgr2=0x55d8617e3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:41.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.681+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/1220680749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e2030 0x55d8617e3e60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fab9c000c00 tx=0x7fab9c009350 comp rx=0 tx=0).stop 2023-09-10T00:53:41.694 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.681+0000 7fabc01d7b80 1 -- 172.21.15.44:0/1220680749 shutdown_connections 2023-09-10T00:53:41.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.681+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/1220680749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617ebe40 0x55d8617e43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.681+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/1220680749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617e1710 0x55d8617f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.681+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/1220680749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e2030 0x55d8617e3e60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.681+0000 7fabc01d7b80 1 -- 172.21.15.44:0/1220680749 >> 172.21.15.44:0/1220680749 conn(0x55d861756090 msgr2=0x55d8617db320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:41.695 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.681+0000 7fabc01d7b80 1 -- 172.21.15.44:0/1220680749 shutdown_connections 2023-09-10T00:53:41.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.681+0000 7fabc01d7b80 1 -- 172.21.15.44:0/1220680749 wait complete. 2023-09-10T00:53:41.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.682+0000 7fabc01d7b80 1 -- start start 2023-09-10T00:53:41.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.682+0000 7fabc01d7b80 1 Processor -- start 2023-09-10T00:53:41.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.682+0000 7fabc01d7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e1710 0x55d861843b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.682+0000 7fabc01d7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617e2030 0x55d861844080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.696 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.682+0000 7fabc01d7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617ebe40 0x55d861848a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.682+0000 7fabc01d7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d861848f50 con 0x55d8617e2030 2023-09-10T00:53:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.682+0000 7fabc01d7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8618490c0 con 0x55d8617e1710 2023-09-10T00:53:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.682+0000 7fabc01d7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d861849280 con 0x55d8617ebe40 2023-09-10T00:53:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.682+0000 7fabc01d7b80 10 client.0 _ll_register_callbacks cb 0x7fab9c004620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:53:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.682+0000 7fabb1dad700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e1710 0x55d861843b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.682+0000 7fabb15ac700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617e2030 0x55d861844080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.682+0000 7fabb15ac700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617e2030 0x55d861844080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53192/0 (socket says 172.21.15.44:53192) 2023-09-10T00:53:41.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.682+0000 7fabb1dad700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e1710 0x55d861843b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54930/0 (socket says 172.21.15.44:54930) 2023-09-10T00:53:41.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.682+0000 7fabb25ae700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617ebe40 0x55d861848a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.698 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.682+0000 7fabb15ac700 1 -- 172.21.15.44:0/2339396236 learned_addr learned my addr 172.21.15.44:0/2339396236 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:41.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.683+0000 7fabb15ac700 1 -- 172.21.15.44:0/2339396236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617ebe40 msgr2=0x55d861848a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:41.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.683+0000 7fabb15ac700 1 --2- 172.21.15.44:0/2339396236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617ebe40 0x55d861848a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.683+0000 7fabb15ac700 1 -- 172.21.15.44:0/2339396236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e1710 msgr2=0x55d861843b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:41.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.683+0000 7fabb15ac700 1 --2- 172.21.15.44:0/2339396236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e1710 0x55d861843b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.683+0000 7fabb15ac700 1 -- 172.21.15.44:0/2339396236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba400bdc0 con 0x55d8617e2030 2023-09-10T00:53:41.699 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.683+0000 7fabb25ae700 1 --2- 172.21.15.44:0/2339396236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617ebe40 0x55d861848a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.683+0000 7fabb1dad700 1 --2- 172.21.15.44:0/2339396236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e1710 0x55d861843b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.683+0000 7fabb15ac700 1 --2- 172.21.15.44:0/2339396236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617e2030 0x55d861844080 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fab9c030830 tx=0x7fab9c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e8859e3197c7fa server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.683+0000 7fab957fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:53:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.683+0000 7fab957fa700 1 -- 172.21.15.44:0/2339396236 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c030b00 con 0x55d8617e2030 2023-09-10T00:53:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.683+0000 7fab957fa700 1 -- 172.21.15.44:0/2339396236 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fab9c030c60 con 0x55d8617e2030 2023-09-10T00:53:41.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.683+0000 7fabc01d7b80 1 -- 172.21.15.44:0/2339396236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faba800de20 con 0x55d8617e2030 2023-09-10T00:53:41.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.683+0000 7fab957fa700 1 -- 172.21.15.44:0/2339396236 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab9c030e20 con 0x55d8617e2030 2023-09-10T00:53:41.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.683+0000 7fabc01d7b80 10 client.17664 fetch_fsmap learned FSMap version 66 2023-09-10T00:53:41.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.683+0000 7fabc01d7b80 1 -- 172.21.15.44:0/2339396236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x7fabac10cc10 con 0x55d8617e2030 2023-09-10T00:53:41.701 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.683+0000 7fab957fa700 1 -- 172.21.15.44:0/2339396236 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faba800de20 con 0x55d8617e2030 2023-09-10T00:53:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.684+0000 7fab957fa700 1 -- 172.21.15.44:0/2339396236 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 66) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fab9c0495d0 con 0x55d8617e2030 2023-09-10T00:53:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.684+0000 7fabc01d7b80 10 client.17664 fetch_fsmap finished waiting for FSMap version 66 2023-09-10T00:53:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.684+0000 7fabc01d7b80 10 client.17664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:53:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.684+0000 7fabc01d7b80 20 client.17664 populate_metadata read hostname 'smithi044' 2023-09-10T00:53:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.684+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/2339396236 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d8618507e0 0x55d861852c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.684+0000 7fabc01d7b80 4 client.17664 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:53:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.684+0000 7fabc01d7b80 1 -- 172.21.15.44:0/2339396236 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d86172c920 con 0x55d8618507e0 2023-09-10T00:53:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.684+0000 7fabb1dad700 1 --2- 172.21.15.44:0/2339396236 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d8618507e0 0x55d861852c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.684+0000 7fab957fa700 10 client.17664 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.684+0000 7fabb1dad700 1 --2- 172.21.15.44:0/2339396236 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d8618507e0 0x55d861852c00 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dffd28074abf140 server_cookie=b98d5e38fa18bcf in_seq=0 out_seq=0 2023-09-10T00:53:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.685+0000 7fab957fa700 1 -- 172.21.15.44:0/2339396236 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d86172c920 con 0x55d8618507e0 2023-09-10T00:53:41.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.685+0000 7fab957fa700 10 client.17664 handle_command_reply: tid=0 2023-09-10T00:53:41.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.685+0000 7fabc01d7b80 1 client.17664 shutdown 2023-09-10T00:53:41.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.685+0000 7fabc01d7b80 1 -- 172.21.15.44:0/2339396236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617e2030 msgr2=0x55d861844080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:41.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.685+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/2339396236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617e2030 0x55d861844080 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fab9c030830 tx=0x7fab9c0057b0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.704 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.685+0000 7fabc01d7b80 1 -- 172.21.15.44:0/2339396236 shutdown_connections 2023-09-10T00:53:41.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.685+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/2339396236 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d8618507e0 0x55d861852c00 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.685+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/2339396236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d8617ebe40 0x55d861848a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.685+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/2339396236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8617e1710 0x55d861843b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.685+0000 7fab957fa700 0 client.17664 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:41.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.685+0000 7fabc01d7b80 1 --2- 172.21.15.44:0/2339396236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d8617e2030 0x55d861844080 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.705 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.685+0000 7fabc01d7b80 1 -- 172.21.15.44:0/2339396236 >> 172.21.15.44:0/2339396236 conn(0x55d861756090 msgr2=0x55d8617e3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:41.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.685+0000 7fabc01d7b80 1 -- 172.21.15.44:0/2339396236 shutdown_connections 2023-09-10T00:53:41.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.685+0000 7fabc01d7b80 1 -- 172.21.15.44:0/2339396236 wait complete. 2023-09-10T00:53:41.706 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.685+0000 7fabc01d7b80 20 client.17664 trim_cache size 0 max 16384 2023-09-10T00:53:41.706 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:53:41.707 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:53:41.707 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:53:41.707 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:53:41.707 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:53:41.708 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:53:41.708 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:53:41.708 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:53:41.708 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:53:41.708 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:53:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.686+0000 7fabb4011700 1 -- 172.21.15.44:0/3600243106 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fab980a0830 msgr2=0x7fab980a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.686+0000 7fabb4011700 1 --2- 172.21.15.44:0/3600243106 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fab980a0830 0x7fab980a2cf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fabac196a30 tx=0x7fab9c012040 comp rx=0 tx=0).stop 2023-09-10T00:53:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.686+0000 7fabb4011700 1 -- 172.21.15.44:0/3600243106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabac0ff440 msgr2=0x7fabac195420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.686+0000 7fabb4011700 1 --2- 172.21.15.44:0/3600243106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabac0ff440 0x7fabac195420 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7faba800e9a0 tx=0x7faba800ecb0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.686+0000 7fabb4011700 1 -- 172.21.15.44:0/3600243106 shutdown_connections 2023-09-10T00:53:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.686+0000 7fabb4011700 1 --2- 172.21.15.44:0/3600243106 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fab980a0830 0x7fab980a2cf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.686+0000 7fabb4011700 1 --2- 172.21.15.44:0/3600243106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabac1054e0 0x7fabac199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.686+0000 7fabb4011700 1 --2- 172.21.15.44:0/3600243106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabac0ff440 0x7fabac195420 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.686+0000 7fabb4011700 1 --2- 172.21.15.44:0/3600243106 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabac0ffe00 0x7fabac195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.686+0000 7fabb4011700 1 -- 172.21.15.44:0/3600243106 >> 172.21.15.44:0/3600243106 conn(0x7fabac0faf50 msgr2=0x7fabac0fb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.686+0000 7fabb4011700 1 -- 172.21.15.44:0/3600243106 shutdown_connections 2023-09-10T00:53:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:53:41.686+0000 7fabb4011700 1 -- 172.21.15.44:0/3600243106 wait complete. 2023-09-10T00:53:41.712 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:53:41.929 INFO:teuthology.orchestra.run.smithi044.stderr:49800000 of 50000000 tuples (99%) done (elapsed 761.56 s, remaining 3.06 s) 2023-09-10T00:53:43.004 INFO:teuthology.orchestra.run.smithi044.stderr:49900000 of 50000000 tuples (99%) done (elapsed 762.64 s, remaining 1.53 s) 2023-09-10T00:53:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:42 smithi131 ceph-mon[103713]: pgmap v408: 129 pgs: 129 active+clean; 9.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 18 MiB/s wr, 49 op/s 2023-09-10T00:53:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:42 smithi044 ceph-mon[98676]: pgmap v408: 129 pgs: 129 active+clean; 9.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 18 MiB/s wr, 49 op/s 2023-09-10T00:53:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:42 smithi099 ceph-mon[102692]: pgmap v408: 129 pgs: 129 active+clean; 9.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 18 MiB/s wr, 49 op/s 2023-09-10T00:53:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:50000000 of 50000000 tuples (100%) done (elapsed 764.01 s, remaining 0.00 s) 2023-09-10T00:53:44.490 INFO:teuthology.orchestra.run.smithi044.stderr:vacuum... 2023-09-10T00:53:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:44 smithi131 ceph-mon[103713]: pgmap v409: 129 pgs: 129 active+clean; 9.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-09-10T00:53:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:44 smithi044 ceph-mon[98676]: pgmap v409: 129 pgs: 129 active+clean; 9.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-09-10T00:53:45.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:44 smithi099 ceph-mon[102692]: pgmap v409: 129 pgs: 129 active+clean; 9.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-09-10T00:53:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:46 smithi131 ceph-mon[103713]: pgmap v410: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 27 MiB/s wr, 97 op/s 2023-09-10T00:53:47.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:46 smithi044 ceph-mon[98676]: pgmap v410: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 27 MiB/s wr, 97 op/s 2023-09-10T00:53:47.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:46 smithi099 ceph-mon[102692]: pgmap v410: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 27 MiB/s wr, 97 op/s 2023-09-10T00:53:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:48 smithi131 ceph-mon[103713]: pgmap v411: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 16 MiB/s wr, 76 op/s 2023-09-10T00:53:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:48 smithi044 ceph-mon[98676]: pgmap v411: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 16 MiB/s wr, 76 op/s 2023-09-10T00:53:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:48 smithi099 ceph-mon[102692]: pgmap v411: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 16 MiB/s wr, 76 op/s 2023-09-10T00:53:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:50 smithi131 ceph-mon[103713]: pgmap v412: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 16 MiB/s wr, 76 op/s 2023-09-10T00:53:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:50 smithi044 ceph-mon[98676]: pgmap v412: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 16 MiB/s wr, 76 op/s 2023-09-10T00:53:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:50 smithi099 ceph-mon[102692]: pgmap v412: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 16 MiB/s wr, 76 op/s 2023-09-10T00:53:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.857+0000 7f22e09d1700 1 -- 172.21.15.44:0/2522428948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22dc0ff4a0 msgr2=0x7f22dc0ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.857+0000 7f22e09d1700 1 --2- 172.21.15.44:0/2522428948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22dc0ff4a0 0x7f22dc0ff920 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f22d00099c0 tx=0x7f22d0009cd0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.857+0000 7f22e09d1700 1 -- 172.21.15.44:0/2522428948 shutdown_connections 2023-09-10T00:53:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.857+0000 7f22e09d1700 1 --2- 172.21.15.44:0/2522428948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22dc0ffe60 0x7f22dc10c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.857+0000 7f22e09d1700 1 --2- 172.21.15.44:0/2522428948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22dc105560 0x7f22dc105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.857+0000 7f22e09d1700 1 --2- 172.21.15.44:0/2522428948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22dc0ff4a0 0x7f22dc0ff920 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.857+0000 7f22e09d1700 1 -- 172.21.15.44:0/2522428948 >> 172.21.15.44:0/2522428948 conn(0x7f22dc0faf30 msgr2=0x7f22dc0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.857+0000 7f22e09d1700 1 -- 172.21.15.44:0/2522428948 shutdown_connections 2023-09-10T00:53:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.857+0000 7f22e09d1700 1 -- 172.21.15.44:0/2522428948 wait complete. 2023-09-10T00:53:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22e09d1700 1 Processor -- start 2023-09-10T00:53:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22e09d1700 1 -- start start 2023-09-10T00:53:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22e09d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22dc0ff4a0 0x7f22dc1974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22e09d1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22dc0ffe60 0x7f22dc197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22e09d1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22dc105560 0x7f22dc1a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22e09d1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc1a2940 con 0x7f22dc0ffe60 2023-09-10T00:53:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22e09d1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc1a2ab0 con 0x7f22dc0ff4a0 2023-09-10T00:53:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22e09d1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc1a2c70 con 0x7f22dc105560 2023-09-10T00:53:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22da59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22dc0ff4a0 0x7f22dc1974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22d9d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22dc0ffe60 0x7f22dc197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22da59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22dc0ff4a0 0x7f22dc1974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54944/0 (socket says 172.21.15.44:54944) 2023-09-10T00:53:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22da59c700 1 -- 172.21.15.44:0/1456813104 learned_addr learned my addr 172.21.15.44:0/1456813104 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22dad9d700 1 --2- 172.21.15.44:0/1456813104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22dc105560 0x7f22dc1a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22d9d9b700 1 -- 172.21.15.44:0/1456813104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22dc105560 msgr2=0x7f22dc1a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22d9d9b700 1 --2- 172.21.15.44:0/1456813104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22dc105560 0x7f22dc1a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22d9d9b700 1 -- 172.21.15.44:0/1456813104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22dc0ff4a0 msgr2=0x7f22dc1974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22d9d9b700 1 --2- 172.21.15.44:0/1456813104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22dc0ff4a0 0x7f22dc1974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22d9d9b700 1 -- 172.21.15.44:0/1456813104 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d00095f0 con 0x7f22dc0ffe60 2023-09-10T00:53:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22da59c700 1 --2- 172.21.15.44:0/1456813104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22dc0ff4a0 0x7f22dc1974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.858+0000 7f22d9d9b700 1 --2- 172.21.15.44:0/1456813104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22dc0ffe60 0x7f22dc197a10 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009fd0 tx=0x7f22d0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.859+0000 7f22cb7fe700 1 -- 172.21.15.44:0/1456813104 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d001d070 con 0x7f22dc0ffe60 2023-09-10T00:53:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.859+0000 7f22e09d1700 1 -- 172.21.15.44:0/1456813104 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22dc1a2ef0 con 0x7f22dc0ffe60 2023-09-10T00:53:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.859+0000 7f22cb7fe700 1 -- 172.21.15.44:0/1456813104 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22d00059a0 con 0x7f22dc0ffe60 2023-09-10T00:53:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.859+0000 7f22cb7fe700 1 -- 172.21.15.44:0/1456813104 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0022cb0 con 0x7f22dc0ffe60 2023-09-10T00:53:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.859+0000 7f22e09d1700 1 -- 172.21.15.44:0/1456813104 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22dc1a3320 con 0x7f22dc0ffe60 2023-09-10T00:53:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.861+0000 7f22cb7fe700 1 -- 172.21.15.44:0/1456813104 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f22d0022470 con 0x7f22dc0ffe60 2023-09-10T00:53:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.862+0000 7f22cb7fe700 1 --2- 172.21.15.44:0/1456813104 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f22c40a07b0 0x7f22c40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.862+0000 7f22da59c700 1 --2- 172.21.15.44:0/1456813104 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f22c40a07b0 0x7f22c40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.862+0000 7f22cb7fe700 1 -- 172.21.15.44:0/1456813104 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f22d00cd470 con 0x7f22dc0ffe60 2023-09-10T00:53:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.862+0000 7f22da59c700 1 --2- 172.21.15.44:0/1456813104 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f22c40a07b0 0x7f22c40a2c70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f22cc011550 tx=0x7f22cc009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.862+0000 7f22ecb97b80 1 Processor -- start 2023-09-10T00:53:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22ecb97b80 1 -- start start 2023-09-10T00:53:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22ecb97b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af803325b0 0x55af80332990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22ecb97b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af8032c710 0x55af8032cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22ecb97b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032d0d0 0x55af80339340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22ecb97b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af802816d0 con 0x55af8032c710 2023-09-10T00:53:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22ecb97b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af8027eff0 con 0x55af8032d0d0 2023-09-10T00:53:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22ecb97b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af8027a8e0 con 0x55af803325b0 2023-09-10T00:53:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22d9d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af8032c710 0x55af8032cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22d9d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af8032c710 0x55af8032cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53214/0 (socket says 172.21.15.44:53214) 2023-09-10T00:53:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22d9d9b700 1 -- 172.21.15.44:0/1498570942 learned_addr learned my addr 172.21.15.44:0/1498570942 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22da59c700 1 --2- 172.21.15.44:0/1498570942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af803325b0 0x55af80332990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22dad9d700 1 --2- 172.21.15.44:0/1498570942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032d0d0 0x55af80339340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22d9d9b700 1 -- 172.21.15.44:0/1498570942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af803325b0 msgr2=0x55af80332990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22d9d9b700 1 --2- 172.21.15.44:0/1498570942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af803325b0 0x55af80332990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22d9d9b700 1 -- 172.21.15.44:0/1498570942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032d0d0 msgr2=0x55af80339340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22d9d9b700 1 --2- 172.21.15.44:0/1498570942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032d0d0 0x55af80339340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22d9d9b700 1 -- 172.21.15.44:0/1498570942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af800a6d80 con 0x55af8032c710 2023-09-10T00:53:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.863+0000 7f22d9d9b700 1 --2- 172.21.15.44:0/1498570942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af8032c710 0x55af8032cb90 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f22d0096a50 tx=0x7f22d009a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21f0720bda06461 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.864+0000 7f22c8ff9700 1 -- 172.21.15.44:0/1498570942 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d009e170 con 0x55af8032c710 2023-09-10T00:53:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.864+0000 7f22c8ff9700 1 -- 172.21.15.44:0/1498570942 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22d009a550 con 0x55af8032c710 2023-09-10T00:53:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.864+0000 7f22c8ff9700 1 -- 172.21.15.44:0/1498570942 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d009aa10 con 0x55af8032c710 2023-09-10T00:53:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.864+0000 7f22ecb97b80 1 -- 172.21.15.44:0/1498570942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af8032c710 msgr2=0x55af8032cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.864+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/1498570942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af8032c710 0x55af8032cb90 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f22d0096a50 tx=0x7f22d009a110 comp rx=0 tx=0).stop 2023-09-10T00:53:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.864+0000 7f22ecb97b80 1 -- 172.21.15.44:0/1498570942 shutdown_connections 2023-09-10T00:53:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.864+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/1498570942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032d0d0 0x55af80339340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.864+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/1498570942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af803325b0 0x55af80332990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.864+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/1498570942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af8032c710 0x55af8032cb90 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.864+0000 7f22ecb97b80 1 -- 172.21.15.44:0/1498570942 >> 172.21.15.44:0/1498570942 conn(0x55af8029d030 msgr2=0x55af80323560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.864+0000 7f22ecb97b80 1 -- 172.21.15.44:0/1498570942 shutdown_connections 2023-09-10T00:53:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.864+0000 7f22ecb97b80 1 -- 172.21.15.44:0/1498570942 wait complete. 2023-09-10T00:53:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.865+0000 7f22ecb97b80 1 -- start start 2023-09-10T00:53:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.865+0000 7f22ecb97b80 1 Processor -- start 2023-09-10T00:53:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.865+0000 7f22ecb97b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af8032c710 0x55af8033be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.865+0000 7f22ecb97b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af8032d0d0 0x55af8033c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.865+0000 7f22ecb97b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af803325b0 0x55af803360c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.865+0000 7f22ecb97b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af80218b10 con 0x55af8032d0d0 2023-09-10T00:53:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.865+0000 7f22ecb97b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af80218810 con 0x55af803325b0 2023-09-10T00:53:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.865+0000 7f22ecb97b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af800fc8c0 con 0x55af8032c710 2023-09-10T00:53:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.865+0000 7f22ecb97b80 10 client.0 _ll_register_callbacks cb 0x55af80162a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:53:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.865+0000 7f22d9d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af8032d0d0 0x55af8033c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.865+0000 7f22d9d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af8032d0d0 0x55af8033c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53220/0 (socket says 172.21.15.44:53220) 2023-09-10T00:53:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.865+0000 7f22da59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af8032c710 0x55af8033be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.865+0000 7f22d9d9b700 1 -- 172.21.15.44:0/1420724826 learned_addr learned my addr 172.21.15.44:0/1420724826 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.865+0000 7f22dad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af803325b0 0x55af803360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.866+0000 7f22d9d9b700 1 -- 172.21.15.44:0/1420724826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af8032c710 msgr2=0x55af8033be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.866+0000 7f22d9d9b700 1 --2- 172.21.15.44:0/1420724826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af8032c710 0x55af8033be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.866+0000 7f22d9d9b700 1 -- 172.21.15.44:0/1420724826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af803325b0 msgr2=0x55af803360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.866+0000 7f22d9d9b700 1 --2- 172.21.15.44:0/1420724826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af803325b0 0x55af803360c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.866+0000 7f22d9d9b700 1 -- 172.21.15.44:0/1420724826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22dc1a3320 con 0x55af8032d0d0 2023-09-10T00:53:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.866+0000 7f22dad9d700 1 --2- 172.21.15.44:0/1420724826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af803325b0 0x55af803360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.866+0000 7f22d9d9b700 1 --2- 172.21.15.44:0/1420724826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af8032d0d0 0x55af8033c3b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f22d00a2000 tx=0x7f22d0099e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6becfae976e9c83 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.866+0000 7f22c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:53:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.866+0000 7f22c1ffb700 1 -- 172.21.15.44:0/1420724826 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d009e380 con 0x55af8032d0d0 2023-09-10T00:53:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.866+0000 7f22c1ffb700 1 -- 172.21.15.44:0/1420724826 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f22d00a2530 con 0x55af8032d0d0 2023-09-10T00:53:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.866+0000 7f22ecb97b80 1 -- 172.21.15.44:0/1420724826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af801d2490 con 0x55af8032d0d0 2023-09-10T00:53:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.866+0000 7f22c1ffb700 1 -- 172.21.15.44:0/1420724826 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d009a9c0 con 0x55af8032d0d0 2023-09-10T00:53:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.866+0000 7f22ecb97b80 10 client.17676 fetch_fsmap learned FSMap version 66 2023-09-10T00:53:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.866+0000 7f22ecb97b80 1 -- 172.21.15.44:0/1420724826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x55af8008fc00 con 0x55af8032d0d0 2023-09-10T00:53:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.866+0000 7f22c1ffb700 1 -- 172.21.15.44:0/1420724826 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55af801d2490 con 0x55af8032d0d0 2023-09-10T00:53:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.866+0000 7f22c1ffb700 1 -- 172.21.15.44:0/1420724826 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 66) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f22d0096440 con 0x55af8032d0d0 2023-09-10T00:53:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.867+0000 7f22ecb97b80 10 client.17676 fetch_fsmap finished waiting for FSMap version 66 2023-09-10T00:53:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.867+0000 7f22ecb97b80 10 client.17676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:53:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.867+0000 7f22ecb97b80 20 client.17676 populate_metadata read hostname 'smithi044' 2023-09-10T00:53:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.867+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/1420724826 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af80345f70 0x55af80348390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.867+0000 7f22ecb97b80 4 client.17676 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:53:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.867+0000 7f22ecb97b80 1 -- 172.21.15.44:0/1420724826 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af8028f840 con 0x55af80345f70 2023-09-10T00:53:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.867+0000 7f22da59c700 1 --2- 172.21.15.44:0/1420724826 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af80345f70 0x55af80348390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.867+0000 7f22c1ffb700 10 client.17676 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.867+0000 7f22da59c700 1 --2- 172.21.15.44:0/1420724826 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af80345f70 0x55af80348390 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d45207d9d9be3d4b server_cookie=7301c2d6a537d3b7 in_seq=0 out_seq=0 2023-09-10T00:53:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.868+0000 7f22c1ffb700 1 -- 172.21.15.44:0/1420724826 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55af8028f840 con 0x55af80345f70 2023-09-10T00:53:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.868+0000 7f22c1ffb700 10 client.17676 handle_command_reply: tid=0 2023-09-10T00:53:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.868+0000 7f22ecb97b80 1 client.17676 shutdown 2023-09-10T00:53:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.868+0000 7f22ecb97b80 1 -- 172.21.15.44:0/1420724826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af8032d0d0 msgr2=0x55af8033c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.868+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/1420724826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af8032d0d0 0x55af8033c3b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f22d00a2000 tx=0x7f22d0099e00 comp rx=0 tx=0).stop 2023-09-10T00:53:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.868+0000 7f22ecb97b80 1 -- 172.21.15.44:0/1420724826 shutdown_connections 2023-09-10T00:53:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.868+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/1420724826 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af80345f70 0x55af80348390 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.868+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/1420724826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af803325b0 0x55af803360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.868+0000 7f22c1ffb700 0 client.17676 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.868+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/1420724826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af8032c710 0x55af8033be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.868+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/1420724826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af8032d0d0 0x55af8033c3b0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.868+0000 7f22ecb97b80 1 -- 172.21.15.44:0/1420724826 >> 172.21.15.44:0/1420724826 conn(0x55af8029d030 msgr2=0x55af803227a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.868+0000 7f22ecb97b80 1 -- 172.21.15.44:0/1420724826 shutdown_connections 2023-09-10T00:53:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.868+0000 7f22ecb97b80 1 -- 172.21.15.44:0/1420724826 wait complete. 2023-09-10T00:53:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.868+0000 7f22ecb97b80 20 client.17676 trim_cache size 0 max 16384 2023-09-10T00:53:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.883+0000 7f22ecb97b80 1 Processor -- start 2023-09-10T00:53:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22ecb97b80 1 -- start start 2023-09-10T00:53:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22ecb97b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af8032c710 0x55af80331b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22ecb97b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032d0d0 0x55af803339a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22ecb97b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af803325b0 0x55af80322f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22ecb97b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af803489f0 con 0x55af803325b0 2023-09-10T00:53:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22ecb97b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af80334640 con 0x55af8032d0d0 2023-09-10T00:53:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22ecb97b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af80331070 con 0x55af8032c710 2023-09-10T00:53:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22dad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af803325b0 0x55af80322f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22d9d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032d0d0 0x55af803339a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22da59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af8032c710 0x55af80331b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22dad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af803325b0 0x55af80322f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53230/0 (socket says 172.21.15.44:53230) 2023-09-10T00:53:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22d9d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032d0d0 0x55af803339a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54968/0 (socket says 172.21.15.44:54968) 2023-09-10T00:53:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22dad9d700 1 -- 172.21.15.44:0/310541605 learned_addr learned my addr 172.21.15.44:0/310541605 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22d9d9b700 1 -- 172.21.15.44:0/310541605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af8032c710 msgr2=0x55af80331b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22d9d9b700 1 --2- 172.21.15.44:0/310541605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af8032c710 0x55af80331b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22d9d9b700 1 -- 172.21.15.44:0/310541605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af803325b0 msgr2=0x55af80322f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22d9d9b700 1 --2- 172.21.15.44:0/310541605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af803325b0 0x55af80322f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22d9d9b700 1 -- 172.21.15.44:0/310541605 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af800cd1f0 con 0x55af8032d0d0 2023-09-10T00:53:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.884+0000 7f22dad9d700 1 --2- 172.21.15.44:0/310541605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af803325b0 0x55af80322f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.885+0000 7f22d9d9b700 1 --2- 172.21.15.44:0/310541605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032d0d0 0x55af803339a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f22d000b580 tx=0x7f22d00961f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4581609d4bd79246 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.885+0000 7f22c97fa700 1 -- 172.21.15.44:0/310541605 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d00b2be0 con 0x55af8032d0d0 2023-09-10T00:53:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.885+0000 7f22c97fa700 1 -- 172.21.15.44:0/310541605 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22d00a2530 con 0x55af8032d0d0 2023-09-10T00:53:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.885+0000 7f22c97fa700 1 -- 172.21.15.44:0/310541605 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d00a2300 con 0x55af8032d0d0 2023-09-10T00:53:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.885+0000 7f22ecb97b80 1 -- 172.21.15.44:0/310541605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032d0d0 msgr2=0x55af803339a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.885+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/310541605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032d0d0 0x55af803339a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f22d000b580 tx=0x7f22d00961f0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.885+0000 7f22ecb97b80 1 -- 172.21.15.44:0/310541605 shutdown_connections 2023-09-10T00:53:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.885+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/310541605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af803325b0 0x55af80322f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.885+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/310541605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af8032c710 0x55af80331b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.885+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/310541605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032d0d0 0x55af803339a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.885+0000 7f22ecb97b80 1 -- 172.21.15.44:0/310541605 >> 172.21.15.44:0/310541605 conn(0x55af8029d030 msgr2=0x55af80331480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.885+0000 7f22ecb97b80 1 -- 172.21.15.44:0/310541605 shutdown_connections 2023-09-10T00:53:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.885+0000 7f22ecb97b80 1 -- 172.21.15.44:0/310541605 wait complete. 2023-09-10T00:53:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.886+0000 7f22ecb97b80 1 -- start start 2023-09-10T00:53:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.886+0000 7f22ecb97b80 1 Processor -- start 2023-09-10T00:53:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.886+0000 7f22ecb97b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032c710 0x55af80340540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.886+0000 7f22ecb97b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af8032d0d0 0x55af80340a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.886+0000 7f22ecb97b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af803325b0 0x55af80398d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.886+0000 7f22ecb97b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af80341220 con 0x55af803325b0 2023-09-10T00:53:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.886+0000 7f22ecb97b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af803992d0 con 0x55af8032c710 2023-09-10T00:53:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.886+0000 7f22ecb97b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af80399490 con 0x55af8032d0d0 2023-09-10T00:53:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.886+0000 7f22ecb97b80 10 client.0 _ll_register_callbacks cb 0x7f22d4009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:53:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.886+0000 7f22dad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af803325b0 0x55af80398d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.886+0000 7f22da59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032c710 0x55af80340540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.886+0000 7f22d9d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af8032d0d0 0x55af80340a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.886+0000 7f22dad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af803325b0 0x55af80398d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53236/0 (socket says 172.21.15.44:53236) 2023-09-10T00:53:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.886+0000 7f22dad9d700 1 -- 172.21.15.44:0/3603732165 learned_addr learned my addr 172.21.15.44:0/3603732165 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:53:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.886+0000 7f22da59c700 1 --2- 172.21.15.44:0/3603732165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032c710 0x55af80340540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54972/0 (socket says 172.21.15.44:54972) 2023-09-10T00:53:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.887+0000 7f22dad9d700 1 -- 172.21.15.44:0/3603732165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af8032d0d0 msgr2=0x55af80340a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.887+0000 7f22dad9d700 1 --2- 172.21.15.44:0/3603732165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af8032d0d0 0x55af80340a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.887+0000 7f22dad9d700 1 -- 172.21.15.44:0/3603732165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032c710 msgr2=0x55af80340540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:53:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.887+0000 7f22dad9d700 1 --2- 172.21.15.44:0/3603732165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032c710 0x55af80340540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.887+0000 7f22dad9d700 1 -- 172.21.15.44:0/3603732165 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d40118c0 con 0x55af803325b0 2023-09-10T00:53:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.887+0000 7f22da59c700 1 --2- 172.21.15.44:0/3603732165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032c710 0x55af80340540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:53:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.887+0000 7f22dad9d700 1 --2- 172.21.15.44:0/3603732165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af803325b0 0x55af80398d60 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f22d4007580 tx=0x7f22d4007890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a284a948de7703 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:53:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.887+0000 7f22c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:53:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.887+0000 7f22c1ffb700 1 -- 172.21.15.44:0/3603732165 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d401c070 con 0x55af803325b0 2023-09-10T00:53:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.887+0000 7f22c1ffb700 1 -- 172.21.15.44:0/3603732165 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f22d4007e10 con 0x55af803325b0 2023-09-10T00:53:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.887+0000 7f22ecb97b80 1 -- 172.21.15.44:0/3603732165 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22d009a550 con 0x55af803325b0 2023-09-10T00:53:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.887+0000 7f22c1ffb700 1 -- 172.21.15.44:0/3603732165 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d400e040 con 0x55af803325b0 2023-09-10T00:53:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.887+0000 7f22ecb97b80 10 client.17682 fetch_fsmap learned FSMap version 66 2023-09-10T00:53:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.887+0000 7f22ecb97b80 1 -- 172.21.15.44:0/3603732165 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x55af8008fc00 con 0x55af803325b0 2023-09-10T00:53:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.887+0000 7f22c1ffb700 1 -- 172.21.15.44:0/3603732165 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22d009a550 con 0x55af803325b0 2023-09-10T00:53:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.888+0000 7f22c1ffb700 1 -- 172.21.15.44:0/3603732165 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 66) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f22d40044e0 con 0x55af803325b0 2023-09-10T00:53:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.888+0000 7f22ecb97b80 10 client.17682 fetch_fsmap finished waiting for FSMap version 66 2023-09-10T00:53:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.888+0000 7f22ecb97b80 10 client.17682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:53:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.888+0000 7f22ecb97b80 20 client.17682 populate_metadata read hostname 'smithi044' 2023-09-10T00:53:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.888+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/3603732165 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af8039ad30 0x55af8039d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:53:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.888+0000 7f22ecb97b80 4 client.17682 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:53:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.888+0000 7f22ecb97b80 1 -- 172.21.15.44:0/3603732165 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55af802738c0 con 0x55af8039ad30 2023-09-10T00:53:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.888+0000 7f22da59c700 1 --2- 172.21.15.44:0/3603732165 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af8039ad30 0x55af8039d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:53:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.888+0000 7f22da59c700 1 --2- 172.21.15.44:0/3603732165 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af8039ad30 0x55af8039d150 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53d17fc56ef25c2a server_cookie=eb6d4fdefd7bf716 in_seq=0 out_seq=0 2023-09-10T00:53:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.888+0000 7f22c1ffb700 10 client.17682 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.898+0000 7f22c1ffb700 1 -- 172.21.15.44:0/3603732165 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55af802738c0 con 0x55af8039ad30 2023-09-10T00:53:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.898+0000 7f22c1ffb700 10 client.17682 handle_command_reply: tid=0 2023-09-10T00:53:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.898+0000 7f22ecb97b80 1 client.17682 shutdown 2023-09-10T00:53:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.898+0000 7f22ecb97b80 1 -- 172.21.15.44:0/3603732165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af803325b0 msgr2=0x55af80398d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.898+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/3603732165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af803325b0 0x55af80398d60 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f22d4007580 tx=0x7f22d4007890 comp rx=0 tx=0).stop 2023-09-10T00:53:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22ecb97b80 1 -- 172.21.15.44:0/3603732165 shutdown_connections 2023-09-10T00:53:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/3603732165 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af8039ad30 0x55af8039d150 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/3603732165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af803325b0 0x55af80398d60 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22c1ffb700 0 client.17682 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:53:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/3603732165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af8032c710 0x55af80340540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22ecb97b80 1 --2- 172.21.15.44:0/3603732165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af8032d0d0 0x55af80340a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22ecb97b80 1 -- 172.21.15.44:0/3603732165 >> 172.21.15.44:0/3603732165 conn(0x55af8029d030 msgr2=0x55af80334890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22ecb97b80 1 -- 172.21.15.44:0/3603732165 shutdown_connections 2023-09-10T00:53:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22ecb97b80 1 -- 172.21.15.44:0/3603732165 wait complete. 2023-09-10T00:53:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22ecb97b80 20 client.17682 trim_cache size 0 max 16384 2023-09-10T00:53:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:53:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:53:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:53:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:53:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22e09d1700 1 -- 172.21.15.44:0/1456813104 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f22c40a07b0 msgr2=0x7f22c40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22e09d1700 1 --2- 172.21.15.44:0/1456813104 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f22c40a07b0 0x7f22c40a2c70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f22cc011550 tx=0x7f22cc009260 comp rx=0 tx=0).stop 2023-09-10T00:53:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22e09d1700 1 -- 172.21.15.44:0/1456813104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22dc0ffe60 msgr2=0x7f22dc197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:53:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22e09d1700 1 --2- 172.21.15.44:0/1456813104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22dc0ffe60 0x7f22dc197a10 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009fd0 tx=0x7f22d0009580 comp rx=0 tx=0).stop 2023-09-10T00:53:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22e09d1700 1 -- 172.21.15.44:0/1456813104 shutdown_connections 2023-09-10T00:53:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22e09d1700 1 --2- 172.21.15.44:0/1456813104 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f22c40a07b0 0x7f22c40a2c70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22e09d1700 1 --2- 172.21.15.44:0/1456813104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22dc105560 0x7f22dc1a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22e09d1700 1 --2- 172.21.15.44:0/1456813104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22dc0ff4a0 0x7f22dc1974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22e09d1700 1 --2- 172.21.15.44:0/1456813104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22dc0ffe60 0x7f22dc197a10 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:53:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22e09d1700 1 -- 172.21.15.44:0/1456813104 >> 172.21.15.44:0/1456813104 conn(0x7f22dc0faf30 msgr2=0x7f22dc0fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:53:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22e09d1700 1 -- 172.21.15.44:0/1456813104 shutdown_connections 2023-09-10T00:53:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:53:52.899+0000 7f22e09d1700 1 -- 172.21.15.44:0/1456813104 wait complete. 2023-09-10T00:53:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:52 smithi131 ceph-mon[103713]: pgmap v413: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 17 MiB/s wr, 252 op/s 2023-09-10T00:53:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:52 smithi044 ceph-mon[98676]: pgmap v413: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 17 MiB/s wr, 252 op/s 2023-09-10T00:53:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:52 smithi099 ceph-mon[102692]: pgmap v413: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 17 MiB/s wr, 252 op/s 2023-09-10T00:53:55.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:54 smithi131 ceph-mon[103713]: pgmap v414: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 13 MiB/s wr, 240 op/s 2023-09-10T00:53:55.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:54 smithi044 ceph-mon[98676]: pgmap v414: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 13 MiB/s wr, 240 op/s 2023-09-10T00:53:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:54 smithi099 ceph-mon[102692]: pgmap v414: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 13 MiB/s wr, 240 op/s 2023-09-10T00:53:57.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:56 smithi131 ceph-mon[103713]: pgmap v415: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 15 MiB/s wr, 416 op/s 2023-09-10T00:53:57.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:56 smithi044 ceph-mon[98676]: pgmap v415: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 15 MiB/s wr, 416 op/s 2023-09-10T00:53:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:56 smithi099 ceph-mon[102692]: pgmap v415: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 15 MiB/s wr, 416 op/s 2023-09-10T00:53:59.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:53:58 smithi131 ceph-mon[103713]: pgmap v416: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 3.2 MiB/s wr, 353 op/s 2023-09-10T00:53:59.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:53:58 smithi044 ceph-mon[98676]: pgmap v416: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 3.2 MiB/s wr, 353 op/s 2023-09-10T00:53:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:53:58 smithi099 ceph-mon[102692]: pgmap v416: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 3.2 MiB/s wr, 353 op/s 2023-09-10T00:54:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:00 smithi131 ceph-mon[103713]: pgmap v417: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 3.2 MiB/s wr, 353 op/s 2023-09-10T00:54:01.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:00 smithi044 ceph-mon[98676]: pgmap v417: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 3.2 MiB/s wr, 353 op/s 2023-09-10T00:54:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:00 smithi099 ceph-mon[102692]: pgmap v417: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 3.2 MiB/s wr, 353 op/s 2023-09-10T00:54:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:02 smithi131 ceph-mon[103713]: pgmap v418: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 4.4 MiB/s wr, 501 op/s 2023-09-10T00:54:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:02 smithi044 ceph-mon[98676]: pgmap v418: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 4.4 MiB/s wr, 501 op/s 2023-09-10T00:54:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:02 smithi099 ceph-mon[102692]: pgmap v418: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 4.4 MiB/s wr, 501 op/s 2023-09-10T00:54:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:04 smithi131 ceph-mon[103713]: pgmap v419: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.0 MiB/s wr, 325 op/s 2023-09-10T00:54:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:04 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:54:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:04 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:54:05.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:04 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:54:05.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:04 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:54:05.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:04 smithi044 ceph-mon[98676]: pgmap v419: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.0 MiB/s wr, 325 op/s 2023-09-10T00:54:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:04 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:54:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:04 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:54:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:04 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:54:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:04 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:54:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:04 smithi099 ceph-mon[102692]: pgmap v419: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.0 MiB/s wr, 325 op/s 2023-09-10T00:54:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:04 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:54:05.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:04 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:54:05.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:04 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:54:05.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:04 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:54:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.039+0000 7fcbf159a700 1 -- 172.21.15.44:0/2133857278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec1054d0 msgr2=0x7fcbec1058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.039+0000 7fcbf159a700 1 --2- 172.21.15.44:0/2133857278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec1054d0 0x7fcbec1058b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009990 tx=0x7fcbdc009ca0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.039+0000 7fcbf159a700 1 -- 172.21.15.44:0/2133857278 shutdown_connections 2023-09-10T00:54:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.039+0000 7fcbf159a700 1 --2- 172.21.15.44:0/2133857278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbec0ff3c0 0x7fcbec1041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.039+0000 7fcbf159a700 1 --2- 172.21.15.44:0/2133857278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec1054d0 0x7fcbec1058b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.039+0000 7fcbf159a700 1 --2- 172.21.15.44:0/2133857278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbec0fea00 0x7fcbec0fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.039+0000 7fcbf159a700 1 -- 172.21.15.44:0/2133857278 >> 172.21.15.44:0/2133857278 conn(0x7fcbec0769d0 msgr2=0x7fcbec0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.040+0000 7fcbf159a700 1 -- 172.21.15.44:0/2133857278 shutdown_connections 2023-09-10T00:54:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.040+0000 7fcbf159a700 1 -- 172.21.15.44:0/2133857278 wait complete. 2023-09-10T00:54:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.040+0000 7fcbf159a700 1 Processor -- start 2023-09-10T00:54:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.040+0000 7fcbf159a700 1 -- start start 2023-09-10T00:54:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbf159a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec0fea00 0x7fcbec1a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbf159a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbec0ff3c0 0x7fcbec1a3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbf159a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbec1054d0 0x7fcbec19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbf159a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec19d020 con 0x7fcbec0ff3c0 2023-09-10T00:54:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbf159a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec19d190 con 0x7fcbec1054d0 2023-09-10T00:54:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbf159a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec19d350 con 0x7fcbec0fea00 2023-09-10T00:54:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbeaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec0fea00 0x7fcbec1a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbeb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbec1054d0 0x7fcbec19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbeaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec0fea00 0x7fcbec1a2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60136/0 (socket says 172.21.15.44:60136) 2023-09-10T00:54:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbeaffd700 1 -- 172.21.15.44:0/2797184467 learned_addr learned my addr 172.21.15.44:0/2797184467 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbea7fc700 1 --2- 172.21.15.44:0/2797184467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbec0ff3c0 0x7fcbec1a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbeaffd700 1 -- 172.21.15.44:0/2797184467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbec1054d0 msgr2=0x7fcbec19c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbeaffd700 1 --2- 172.21.15.44:0/2797184467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbec1054d0 0x7fcbec19c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbeaffd700 1 -- 172.21.15.44:0/2797184467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbec0ff3c0 msgr2=0x7fcbec1a3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbeaffd700 1 --2- 172.21.15.44:0/2797184467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbec0ff3c0 0x7fcbec1a3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbeaffd700 1 -- 172.21.15.44:0/2797184467 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc0095f0 con 0x7fcbec0fea00 2023-09-10T00:54:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.041+0000 7fcbeb7fe700 1 --2- 172.21.15.44:0/2797184467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbec1054d0 0x7fcbec19c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.042+0000 7fcbeaffd700 1 --2- 172.21.15.44:0/2797184467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec0fea00 0x7fcbec1a2b50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc005910 tx=0x7fcbdc0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.042+0000 7fcbd3fff700 1 -- 172.21.15.44:0/2797184467 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc01d070 con 0x7fcbec0fea00 2023-09-10T00:54:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.042+0000 7fcbf159a700 1 -- 172.21.15.44:0/2797184467 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbec19d5d0 con 0x7fcbec0fea00 2023-09-10T00:54:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.042+0000 7fcbd3fff700 1 -- 172.21.15.44:0/2797184467 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbdc021410 con 0x7fcbec0fea00 2023-09-10T00:54:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.042+0000 7fcbf159a700 1 -- 172.21.15.44:0/2797184467 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbec1a9b40 con 0x7fcbec0fea00 2023-09-10T00:54:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.042+0000 7fcbd3fff700 1 -- 172.21.15.44:0/2797184467 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0175f0 con 0x7fcbec0fea00 2023-09-10T00:54:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.045+0000 7fcbd3fff700 1 -- 172.21.15.44:0/2797184467 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fcbdc021580 con 0x7fcbec0fea00 2023-09-10T00:54:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.045+0000 7fcbd3fff700 1 --2- 172.21.15.44:0/2797184467 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcbd40a07b0 0x7fcbd40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.046+0000 7fcbea7fc700 1 --2- 172.21.15.44:0/2797184467 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcbd40a07b0 0x7fcbd40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.046+0000 7fcbd3fff700 1 -- 172.21.15.44:0/2797184467 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fcbdc0cd520 con 0x7fcbec0fea00 2023-09-10T00:54:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.046+0000 7fcbfd760b80 1 Processor -- start 2023-09-10T00:54:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.046+0000 7fcbea7fc700 1 --2- 172.21.15.44:0/2797184467 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcbd40a07b0 0x7fcbd40a2c70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0fc9f0 tx=0x7fcbe0009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.046+0000 7fcbfd760b80 1 -- start start 2023-09-10T00:54:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.046+0000 7fcbfd760b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 0x55a6c6ae0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.046+0000 7fcbfd760b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6ada780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.046+0000 7fcbfd760b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6adacc0 0x55a6c6ae6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.046+0000 7fcbfd760b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c6a336d0 con 0x55a6c6adacc0 2023-09-10T00:54:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.046+0000 7fcbfd760b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c6a30ff0 con 0x55a6c6ada300 2023-09-10T00:54:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.046+0000 7fcbfd760b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c6a2c8e0 con 0x55a6c6ae01a0 2023-09-10T00:54:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.047+0000 7fcbeaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 0x55a6c6ae0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.047+0000 7fcbea7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6ada780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.047+0000 7fcbeb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6adacc0 0x55a6c6ae6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.047+0000 7fcbeaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 0x55a6c6ae0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60144/0 (socket says 172.21.15.44:60144) 2023-09-10T00:54:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.047+0000 7fcbea7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6ada780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55004/0 (socket says 172.21.15.44:55004) 2023-09-10T00:54:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.047+0000 7fcbeaffd700 1 -- 172.21.15.44:0/2873546084 learned_addr learned my addr 172.21.15.44:0/2873546084 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.047+0000 7fcbeaffd700 1 -- 172.21.15.44:0/2873546084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 msgr2=0x55a6c6ada780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.047+0000 7fcbeaffd700 1 --2- 172.21.15.44:0/2873546084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6ada780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.047+0000 7fcbeaffd700 1 -- 172.21.15.44:0/2873546084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6adacc0 msgr2=0x55a6c6ae6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.047+0000 7fcbeaffd700 1 --2- 172.21.15.44:0/2873546084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6adacc0 0x55a6c6ae6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.047+0000 7fcbeaffd700 1 -- 172.21.15.44:0/2873546084 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6c6858d80 con 0x55a6c6ae01a0 2023-09-10T00:54:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.047+0000 7fcbea7fc700 1 --2- 172.21.15.44:0/2873546084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6ada780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.047+0000 7fcbeaffd700 1 --2- 172.21.15.44:0/2873546084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 0x55a6c6ae0580 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0970d0 tx=0x7fcbdc09a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffe6247cbd392df server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.047+0000 7fcbd17fa700 1 -- 172.21.15.44:0/2873546084 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc09ad60 con 0x55a6c6ae01a0 2023-09-10T00:54:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.047+0000 7fcbd17fa700 1 -- 172.21.15.44:0/2873546084 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbdc09a660 con 0x55a6c6ae01a0 2023-09-10T00:54:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.047+0000 7fcbd17fa700 1 -- 172.21.15.44:0/2873546084 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc021860 con 0x55a6c6ae01a0 2023-09-10T00:54:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.048+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2873546084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 msgr2=0x55a6c6ae0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.048+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/2873546084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 0x55a6c6ae0580 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0970d0 tx=0x7fcbdc09a1f0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.048+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2873546084 shutdown_connections 2023-09-10T00:54:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.048+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/2873546084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6adacc0 0x55a6c6ae6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.048+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/2873546084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 0x55a6c6ae0580 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.048+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/2873546084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6ada780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.048+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2873546084 >> 172.21.15.44:0/2873546084 conn(0x55a6c6a4f030 msgr2=0x55a6c6ad5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.048+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2873546084 shutdown_connections 2023-09-10T00:54:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.048+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2873546084 wait complete. 2023-09-10T00:54:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.049+0000 7fcbfd760b80 1 -- start start 2023-09-10T00:54:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.049+0000 7fcbfd760b80 1 Processor -- start 2023-09-10T00:54:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.049+0000 7fcbfd760b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6ada300 0x55a6c6ae9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.049+0000 7fcbfd760b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6adacc0 0x55a6c6aea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.049+0000 7fcbfd760b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 0x55a6c6ae3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.049+0000 7fcbfd760b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c69cab10 con 0x55a6c6ada300 2023-09-10T00:54:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.049+0000 7fcbfd760b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c69ca810 con 0x55a6c6adacc0 2023-09-10T00:54:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.049+0000 7fcbfd760b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c68ae8c0 con 0x55a6c6ae01a0 2023-09-10T00:54:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.049+0000 7fcbfd760b80 10 client.0 _ll_register_callbacks cb 0x55a6c6914a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:54:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.049+0000 7fcbeaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6ada300 0x55a6c6ae9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.049+0000 7fcbea7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6adacc0 0x55a6c6aea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.049+0000 7fcbeaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6ada300 0x55a6c6ae9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53268/0 (socket says 172.21.15.44:53268) 2023-09-10T00:54:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.049+0000 7fcbeaffd700 1 -- 172.21.15.44:0/2811090857 learned_addr learned my addr 172.21.15.44:0/2811090857 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.050+0000 7fcbeb7fe700 1 --2- 172.21.15.44:0/2811090857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 0x55a6c6ae3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.050+0000 7fcbea7fc700 1 -- 172.21.15.44:0/2811090857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 msgr2=0x55a6c6ae3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.050+0000 7fcbea7fc700 1 --2- 172.21.15.44:0/2811090857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 0x55a6c6ae3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.050+0000 7fcbea7fc700 1 -- 172.21.15.44:0/2811090857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6ada300 msgr2=0x55a6c6ae9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.050+0000 7fcbea7fc700 1 --2- 172.21.15.44:0/2811090857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6ada300 0x55a6c6ae9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.050+0000 7fcbea7fc700 1 -- 172.21.15.44:0/2811090857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe0017070 con 0x55a6c6adacc0 2023-09-10T00:54:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.050+0000 7fcbea7fc700 1 --2- 172.21.15.44:0/2811090857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6adacc0 0x55a6c6aea100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55a6c6ad5060 tx=0x7fcbe00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3b06f709a097d1d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.050+0000 7fcbceff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:54:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.050+0000 7fcbceff5700 1 -- 172.21.15.44:0/2811090857 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0033070 con 0x55a6c6adacc0 2023-09-10T00:54:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.050+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2811090857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6c6984490 con 0x55a6c6adacc0 2023-09-10T00:54:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.050+0000 7fcbceff5700 1 -- 172.21.15.44:0/2811090857 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fcbe0004260 con 0x55a6c6adacc0 2023-09-10T00:54:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.050+0000 7fcbceff5700 1 -- 172.21.15.44:0/2811090857 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbe0004420 con 0x55a6c6adacc0 2023-09-10T00:54:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.050+0000 7fcbfd760b80 10 client.27359 fetch_fsmap learned FSMap version 67 2023-09-10T00:54:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.050+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2811090857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x7fcbec1a9b40 con 0x55a6c6adacc0 2023-09-10T00:54:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.051+0000 7fcbceff5700 1 -- 172.21.15.44:0/2811090857 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a6c6984490 con 0x55a6c6adacc0 2023-09-10T00:54:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.051+0000 7fcbceff5700 1 -- 172.21.15.44:0/2811090857 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 67) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fcbe0038590 con 0x55a6c6adacc0 2023-09-10T00:54:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.051+0000 7fcbfd760b80 10 client.27359 fetch_fsmap finished waiting for FSMap version 67 2023-09-10T00:54:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.051+0000 7fcbfd760b80 10 client.27359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:54:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.051+0000 7fcbfd760b80 20 client.27359 populate_metadata read hostname 'smithi044' 2023-09-10T00:54:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.051+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/2811090857 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a6c6af4170 0x55a6c6af6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.051+0000 7fcbfd760b80 4 client.27359 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:54:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.051+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2811090857 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6c6a41840 con 0x55a6c6af4170 2023-09-10T00:54:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.051+0000 7fcbeaffd700 1 --2- 172.21.15.44:0/2811090857 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a6c6af4170 0x55a6c6af6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.051+0000 7fcbeaffd700 1 --2- 172.21.15.44:0/2811090857 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a6c6af4170 0x55a6c6af6590 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da20663fa1c6b8c server_cookie=7f918ecbdc277611 in_seq=0 out_seq=0 2023-09-10T00:54:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.052+0000 7fcbceff5700 10 client.27359 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.052+0000 7fcbceff5700 1 -- 172.21.15.44:0/2811090857 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a6c6a41840 con 0x55a6c6af4170 2023-09-10T00:54:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.052+0000 7fcbceff5700 10 client.27359 handle_command_reply: tid=0 2023-09-10T00:54:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.052+0000 7fcbfd760b80 1 client.27359 shutdown 2023-09-10T00:54:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.052+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2811090857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6adacc0 msgr2=0x55a6c6aea100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.052+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/2811090857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6adacc0 0x55a6c6aea100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55a6c6ad5060 tx=0x7fcbe00047c0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.052+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2811090857 shutdown_connections 2023-09-10T00:54:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.052+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/2811090857 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a6c6af4170 0x55a6c6af6590 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.052+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/2811090857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 0x55a6c6ae3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.052+0000 7fcbceff5700 0 client.27359 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.052+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/2811090857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6ada300 0x55a6c6ae9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.052+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/2811090857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6adacc0 0x55a6c6aea100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.052+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2811090857 >> 172.21.15.44:0/2811090857 conn(0x55a6c6a4f030 msgr2=0x55a6c6ad47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.052+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2811090857 shutdown_connections 2023-09-10T00:54:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.052+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2811090857 wait complete. 2023-09-10T00:54:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.053+0000 7fcbfd760b80 20 client.27359 trim_cache size 0 max 16384 2023-09-10T00:54:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.072+0000 7fcbfd760b80 1 Processor -- start 2023-09-10T00:54:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.072+0000 7fcbfd760b80 1 -- start start 2023-09-10T00:54:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.072+0000 7fcbfd760b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6adf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.072+0000 7fcbfd760b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6adacc0 0x55a6c6ae3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.072+0000 7fcbfd760b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 0x55a6c6ae3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.072+0000 7fcbfd760b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c6ae2400 con 0x55a6c6adacc0 2023-09-10T00:54:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.072+0000 7fcbfd760b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c6ae0a10 con 0x55a6c6ada300 2023-09-10T00:54:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.072+0000 7fcbfd760b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c6aded00 con 0x55a6c6ae01a0 2023-09-10T00:54:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.072+0000 7fcbeaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6adf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.072+0000 7fcbeb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 0x55a6c6ae3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.072+0000 7fcbeaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6adf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55016/0 (socket says 172.21.15.44:55016) 2023-09-10T00:54:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.072+0000 7fcbeaffd700 1 -- 172.21.15.44:0/2457843503 learned_addr learned my addr 172.21.15.44:0/2457843503 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.072+0000 7fcbea7fc700 1 --2- 172.21.15.44:0/2457843503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6adacc0 0x55a6c6ae3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.073+0000 7fcbeaffd700 1 -- 172.21.15.44:0/2457843503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 msgr2=0x55a6c6ae3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.073+0000 7fcbeaffd700 1 --2- 172.21.15.44:0/2457843503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 0x55a6c6ae3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.073+0000 7fcbeaffd700 1 -- 172.21.15.44:0/2457843503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6adacc0 msgr2=0x55a6c6ae3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.073+0000 7fcbeaffd700 1 --2- 172.21.15.44:0/2457843503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6adacc0 0x55a6c6ae3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.073+0000 7fcbeaffd700 1 -- 172.21.15.44:0/2457843503 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6c6841c00 con 0x55a6c6ada300 2023-09-10T00:54:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.073+0000 7fcbeb7fe700 1 --2- 172.21.15.44:0/2457843503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 0x55a6c6ae3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.073+0000 7fcbeaffd700 1 --2- 172.21.15.44:0/2457843503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6adf820 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc09a130 tx=0x7fcbdc009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82ec50b4c59e0383 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.073+0000 7fcbd1ffb700 1 -- 172.21.15.44:0/2457843503 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc09ade0 con 0x55a6c6ada300 2023-09-10T00:54:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.074+0000 7fcbd1ffb700 1 -- 172.21.15.44:0/2457843503 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbdc09e490 con 0x55a6c6ada300 2023-09-10T00:54:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.074+0000 7fcbd1ffb700 1 -- 172.21.15.44:0/2457843503 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc021950 con 0x55a6c6ada300 2023-09-10T00:54:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.074+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2457843503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 msgr2=0x55a6c6adf820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.074+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/2457843503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6adf820 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc09a130 tx=0x7fcbdc009ca0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.075+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2457843503 shutdown_connections 2023-09-10T00:54:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.075+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/2457843503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6ae01a0 0x55a6c6ae3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.075+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/2457843503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6adf820 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.075+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/2457843503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6adacc0 0x55a6c6ae3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.075+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2457843503 >> 172.21.15.44:0/2457843503 conn(0x55a6c6a4f030 msgr2=0x55a6c6adf110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.075+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2457843503 shutdown_connections 2023-09-10T00:54:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.075+0000 7fcbfd760b80 1 -- 172.21.15.44:0/2457843503 wait complete. 2023-09-10T00:54:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.075+0000 7fcbfd760b80 1 -- start start 2023-09-10T00:54:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.075+0000 7fcbfd760b80 1 Processor -- start 2023-09-10T00:54:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbfd760b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6b3bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbfd760b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6adacc0 0x55a6c6b3c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbfd760b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6ae01a0 0x55a6c6b3c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbfd760b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c6b47060 con 0x55a6c6ae01a0 2023-09-10T00:54:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbfd760b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c6b471a0 con 0x55a6c6ada300 2023-09-10T00:54:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbfd760b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c6b47360 con 0x55a6c6adacc0 2023-09-10T00:54:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbfd760b80 10 client.0 _ll_register_callbacks cb 0x55a6c6914a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:54:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbeaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6b3bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbeaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6b3bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55022/0 (socket says 172.21.15.44:55022) 2023-09-10T00:54:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbeaffd700 1 -- 172.21.15.44:0/3602673486 learned_addr learned my addr 172.21.15.44:0/3602673486 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbea7fc700 1 --2- 172.21.15.44:0/3602673486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6adacc0 0x55a6c6b3c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbeb7fe700 1 --2- 172.21.15.44:0/3602673486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6ae01a0 0x55a6c6b3c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbeaffd700 1 -- 172.21.15.44:0/3602673486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6adacc0 msgr2=0x55a6c6b3c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbeaffd700 1 --2- 172.21.15.44:0/3602673486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6adacc0 0x55a6c6b3c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbeaffd700 1 -- 172.21.15.44:0/3602673486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6ae01a0 msgr2=0x55a6c6b3c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbeaffd700 1 --2- 172.21.15.44:0/3602673486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6ae01a0 0x55a6c6b3c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbeaffd700 1 -- 172.21.15.44:0/3602673486 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe400e6e0 con 0x55a6c6ada300 2023-09-10T00:54:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbeaffd700 1 --2- 172.21.15.44:0/3602673486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6b3bf50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc005cd0 tx=0x7fcbdc0a2760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26567442527c0bb3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.076+0000 7fcbceff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:54:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.077+0000 7fcbceff5700 1 -- 172.21.15.44:0/3602673486 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc09ade0 con 0x55a6c6ada300 2023-09-10T00:54:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.077+0000 7fcbfd760b80 1 -- 172.21.15.44:0/3602673486 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbdc09a660 con 0x55a6c6ada300 2023-09-10T00:54:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.077+0000 7fcbceff5700 1 -- 172.21.15.44:0/3602673486 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fcbdc09e170 con 0x55a6c6ada300 2023-09-10T00:54:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.077+0000 7fcbceff5700 1 -- 172.21.15.44:0/3602673486 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbdc099fc0 con 0x55a6c6ada300 2023-09-10T00:54:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.077+0000 7fcbfd760b80 10 client.27365 fetch_fsmap learned FSMap version 67 2023-09-10T00:54:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.077+0000 7fcbfd760b80 1 -- 172.21.15.44:0/3602673486 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55a6c6858d80 con 0x55a6c6ada300 2023-09-10T00:54:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.077+0000 7fcbceff5700 1 -- 172.21.15.44:0/3602673486 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbdc09a660 con 0x55a6c6ada300 2023-09-10T00:54:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.077+0000 7fcbceff5700 1 -- 172.21.15.44:0/3602673486 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 67) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fcbdc0850b0 con 0x55a6c6ada300 2023-09-10T00:54:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.077+0000 7fcbfd760b80 10 client.27365 fetch_fsmap finished waiting for FSMap version 67 2023-09-10T00:54:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.077+0000 7fcbfd760b80 10 client.27365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:54:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.077+0000 7fcbfd760b80 20 client.27365 populate_metadata read hostname 'smithi044' 2023-09-10T00:54:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.077+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/3602673486 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a6c6b486c0 0x55a6c6b4aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.077+0000 7fcbfd760b80 4 client.27365 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:54:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.077+0000 7fcbfd760b80 1 -- 172.21.15.44:0/3602673486 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a6c6a258c0 con 0x55a6c6b486c0 2023-09-10T00:54:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.077+0000 7fcbea7fc700 1 --2- 172.21.15.44:0/3602673486 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a6c6b486c0 0x55a6c6b4aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.078+0000 7fcbceff5700 10 client.27365 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.078+0000 7fcbea7fc700 1 --2- 172.21.15.44:0/3602673486 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a6c6b486c0 0x55a6c6b4aae0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=996a973fe437e169 server_cookie=d456847da6d1ef12 in_seq=0 out_seq=0 2023-09-10T00:54:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.783+0000 7fcbceff5700 1 -- 172.21.15.44:0/3602673486 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a6c6a258c0 con 0x55a6c6b486c0 2023-09-10T00:54:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.783+0000 7fcbceff5700 10 client.27365 handle_command_reply: tid=0 2023-09-10T00:54:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.783+0000 7fcbfd760b80 1 client.27365 shutdown 2023-09-10T00:54:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.783+0000 7fcbfd760b80 1 -- 172.21.15.44:0/3602673486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 msgr2=0x55a6c6b3bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.783+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/3602673486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6b3bf50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc005cd0 tx=0x7fcbdc0a2760 comp rx=0 tx=0).stop 2023-09-10T00:54:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.783+0000 7fcbfd760b80 1 -- 172.21.15.44:0/3602673486 shutdown_connections 2023-09-10T00:54:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.783+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/3602673486 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a6c6b486c0 0x55a6c6b4aae0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.783+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/3602673486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6c6ae01a0 0x55a6c6b3c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.783+0000 7fcbceff5700 0 client.27365 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.783+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/3602673486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6c6ada300 0x55a6c6b3bf50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.783+0000 7fcbfd760b80 1 --2- 172.21.15.44:0/3602673486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6c6adacc0 0x55a6c6b3c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.783+0000 7fcbfd760b80 1 -- 172.21.15.44:0/3602673486 >> 172.21.15.44:0/3602673486 conn(0x55a6c6a4f030 msgr2=0x55a6c6ad92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.783+0000 7fcbfd760b80 1 -- 172.21.15.44:0/3602673486 shutdown_connections 2023-09-10T00:54:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.783+0000 7fcbfd760b80 1 -- 172.21.15.44:0/3602673486 wait complete. 2023-09-10T00:54:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.784+0000 7fcbfd760b80 20 client.27365 trim_cache size 0 max 16384 2023-09-10T00:54:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:54:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:54:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:54:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:54:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.784+0000 7fcbf159a700 1 -- 172.21.15.44:0/2797184467 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcbd40a07b0 msgr2=0x7fcbd40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.784+0000 7fcbf159a700 1 --2- 172.21.15.44:0/2797184467 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcbd40a07b0 0x7fcbd40a2c70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0fc9f0 tx=0x7fcbe0009260 comp rx=0 tx=0).stop 2023-09-10T00:54:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.784+0000 7fcbf159a700 1 -- 172.21.15.44:0/2797184467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec0fea00 msgr2=0x7fcbec1a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.784+0000 7fcbf159a700 1 --2- 172.21.15.44:0/2797184467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec0fea00 0x7fcbec1a2b50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc005910 tx=0x7fcbdc0047b0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.784+0000 7fcbf159a700 1 -- 172.21.15.44:0/2797184467 shutdown_connections 2023-09-10T00:54:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.784+0000 7fcbf159a700 1 --2- 172.21.15.44:0/2797184467 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcbd40a07b0 0x7fcbd40a2c70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.784+0000 7fcbf159a700 1 --2- 172.21.15.44:0/2797184467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbec1054d0 0x7fcbec19c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.784+0000 7fcbf159a700 1 --2- 172.21.15.44:0/2797184467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec0fea00 0x7fcbec1a2b50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.784+0000 7fcbf159a700 1 --2- 172.21.15.44:0/2797184467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbec0ff3c0 0x7fcbec1a3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.784+0000 7fcbf159a700 1 -- 172.21.15.44:0/2797184467 >> 172.21.15.44:0/2797184467 conn(0x7fcbec0769d0 msgr2=0x7fcbec1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.785+0000 7fcbf159a700 1 -- 172.21.15.44:0/2797184467 shutdown_connections 2023-09-10T00:54:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:06.785+0000 7fcbf159a700 1 -- 172.21.15.44:0/2797184467 wait complete. 2023-09-10T00:54:06.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:06 smithi099 ceph-mon[102692]: pgmap v420: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.3 MiB/s wr, 503 op/s 2023-09-10T00:54:06.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:06 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:54:07.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:06 smithi131 ceph-mon[103713]: pgmap v420: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.3 MiB/s wr, 503 op/s 2023-09-10T00:54:07.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:06 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:54:07.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:06 smithi044 ceph-mon[98676]: pgmap v420: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.3 MiB/s wr, 503 op/s 2023-09-10T00:54:07.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:06 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:54:08.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:08 smithi099 ceph-mon[102692]: pgmap v421: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2.4 MiB/s wr, 326 op/s 2023-09-10T00:54:09.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:08 smithi131 ceph-mon[103713]: pgmap v421: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2.4 MiB/s wr, 326 op/s 2023-09-10T00:54:09.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:08 smithi044 ceph-mon[98676]: pgmap v421: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2.4 MiB/s wr, 326 op/s 2023-09-10T00:54:09.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:09 smithi099 ceph-mon[102692]: osdmap e107: 12 total, 12 up, 12 in 2023-09-10T00:54:10.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:09 smithi131 ceph-mon[103713]: osdmap e107: 12 total, 12 up, 12 in 2023-09-10T00:54:10.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:09 smithi044 ceph-mon[98676]: osdmap e107: 12 total, 12 up, 12 in 2023-09-10T00:54:10.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:10 smithi099 ceph-mon[102692]: pgmap v423: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 2.8 MiB/s wr, 391 op/s 2023-09-10T00:54:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:10 smithi131 ceph-mon[103713]: pgmap v423: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 2.8 MiB/s wr, 391 op/s 2023-09-10T00:54:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:10 smithi044 ceph-mon[98676]: pgmap v423: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 2.8 MiB/s wr, 391 op/s 2023-09-10T00:54:11.714 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:54:11.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.847+0000 7f9302921700 1 -- 172.21.15.44:0/471459221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc0ff440 msgr2=0x7f92fc0ff8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:11.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.847+0000 7f9302921700 1 --2- 172.21.15.44:0/471459221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc0ff440 0x7f92fc0ff8c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009990 tx=0x7f92f0009ca0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.847+0000 7f9302921700 1 -- 172.21.15.44:0/471459221 shutdown_connections 2023-09-10T00:54:11.850 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.847+0000 7f9302921700 1 --2- 172.21.15.44:0/471459221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92fc0ffe00 0x7f92fc10c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.847+0000 7f9302921700 1 --2- 172.21.15.44:0/471459221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92fc1054e0 0x7f92fc1058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.847+0000 7f9302921700 1 --2- 172.21.15.44:0/471459221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc0ff440 0x7f92fc0ff8c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.847+0000 7f9302921700 1 -- 172.21.15.44:0/471459221 >> 172.21.15.44:0/471459221 conn(0x7f92fc0faf50 msgr2=0x7f92fc0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:11.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.847+0000 7f9302921700 1 -- 172.21.15.44:0/471459221 shutdown_connections 2023-09-10T00:54:11.851 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.847+0000 7f9302921700 1 -- 172.21.15.44:0/471459221 wait complete. 2023-09-10T00:54:11.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.848+0000 7f9302921700 1 Processor -- start 2023-09-10T00:54:11.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.848+0000 7f9302921700 1 -- start start 2023-09-10T00:54:11.852 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.848+0000 7f9302921700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92fc0ff440 0x7f92fc19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.848+0000 7f9302921700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92fc0ffe00 0x7f92fc19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.848+0000 7f9302921700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1054e0 0x7f92fc198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.853 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.848+0000 7f9302921700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc198ca0 con 0x7f92fc1054e0 2023-09-10T00:54:11.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.848+0000 7f9302921700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc198e10 con 0x7f92fc0ff440 2023-09-10T00:54:11.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.848+0000 7f9302921700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc198fd0 con 0x7f92fc0ffe00 2023-09-10T00:54:11.854 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.848+0000 7f9300ebe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1054e0 0x7f92fc198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.848+0000 7f9300ebe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1054e0 0x7f92fc198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53300/0 (socket says 172.21.15.44:53300) 2023-09-10T00:54:11.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.848+0000 7f9300ebe700 1 -- 172.21.15.44:0/2258921824 learned_addr learned my addr 172.21.15.44:0/2258921824 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:11.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.848+0000 7f92fb7fe700 1 --2- 172.21.15.44:0/2258921824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92fc0ffe00 0x7f92fc19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.849+0000 7f92fbfff700 1 --2- 172.21.15.44:0/2258921824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92fc0ff440 0x7f92fc19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.855 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.849+0000 7f9300ebe700 1 -- 172.21.15.44:0/2258921824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92fc0ffe00 msgr2=0x7f92fc19eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:11.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.849+0000 7f9300ebe700 1 --2- 172.21.15.44:0/2258921824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92fc0ffe00 0x7f92fc19eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.849+0000 7f9300ebe700 1 -- 172.21.15.44:0/2258921824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92fc0ff440 msgr2=0x7f92fc19e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:11.856 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.849+0000 7f9300ebe700 1 --2- 172.21.15.44:0/2258921824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92fc0ff440 0x7f92fc19e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.849+0000 7f9300ebe700 1 -- 172.21.15.44:0/2258921824 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f00095f0 con 0x7f92fc1054e0 2023-09-10T00:54:11.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.849+0000 7f9300ebe700 1 --2- 172.21.15.44:0/2258921824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1054e0 0x7f92fc198640 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f92f400ef30 tx=0x7f92f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:11.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.849+0000 7f92f97fa700 1 -- 172.21.15.44:0/2258921824 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4018070 con 0x7f92fc1054e0 2023-09-10T00:54:11.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.849+0000 7f9302921700 1 -- 172.21.15.44:0/2258921824 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92fc199250 con 0x7f92fc1054e0 2023-09-10T00:54:11.857 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.849+0000 7f92f97fa700 1 -- 172.21.15.44:0/2258921824 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92f400f040 con 0x7f92fc1054e0 2023-09-10T00:54:11.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.849+0000 7f92f97fa700 1 -- 172.21.15.44:0/2258921824 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f400cbc0 con 0x7f92fc1054e0 2023-09-10T00:54:11.858 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.849+0000 7f9302921700 1 -- 172.21.15.44:0/2258921824 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92fc1a55c0 con 0x7f92fc1054e0 2023-09-10T00:54:11.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:11 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:54:11.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.851+0000 7f92f97fa700 1 -- 172.21.15.44:0/2258921824 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f92f4007500 con 0x7f92fc1054e0 2023-09-10T00:54:11.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.852+0000 7f92f97fa700 1 --2- 172.21.15.44:0/2258921824 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f92e40a07b0 0x7f92e40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.852+0000 7f92fbfff700 1 --2- 172.21.15.44:0/2258921824 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f92e40a07b0 0x7f92e40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.865 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.853+0000 7f92f97fa700 1 -- 172.21.15.44:0/2258921824 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f92f40cbf00 con 0x7f92fc1054e0 2023-09-10T00:54:11.866 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.853+0000 7f92fbfff700 1 --2- 172.21.15.44:0/2258921824 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f92e40a07b0 0x7f92e40a2c70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009be0 tx=0x7f92ec009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:11.866 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.853+0000 7f930eae7b80 1 Processor -- start 2023-09-10T00:54:11.866 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.853+0000 7f930eae7b80 1 -- start start 2023-09-10T00:54:11.866 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.853+0000 7f930eae7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594c710 0x56357594caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.866 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f930eae7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594d030 0x563575956900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.867 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f930eae7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563575956e40 0x563575959230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.867 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f930eae7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635758a5730 con 0x563575956e40 2023-09-10T00:54:11.867 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f930eae7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635758a3050 con 0x56357594d030 2023-09-10T00:54:11.867 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f930eae7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357589e940 con 0x56357594c710 2023-09-10T00:54:11.867 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f92fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594c710 0x56357594caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.868 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f92fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594c710 0x56357594caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60186/0 (socket says 172.21.15.44:60186) 2023-09-10T00:54:11.868 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f9300ebe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563575956e40 0x563575959230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.868 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f92fbfff700 1 -- 172.21.15.44:0/1212782841 learned_addr learned my addr 172.21.15.44:0/1212782841 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:11.868 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f92fb7fe700 1 --2- 172.21.15.44:0/1212782841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594d030 0x563575956900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.868 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f92fbfff700 1 -- 172.21.15.44:0/1212782841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594d030 msgr2=0x563575956900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:11.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f92fbfff700 1 --2- 172.21.15.44:0/1212782841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594d030 0x563575956900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f92fbfff700 1 -- 172.21.15.44:0/1212782841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563575956e40 msgr2=0x563575959230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:11.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f92fbfff700 1 --2- 172.21.15.44:0/1212782841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563575956e40 0x563575959230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f92fbfff700 1 -- 172.21.15.44:0/1212782841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635756cade0 con 0x56357594c710 2023-09-10T00:54:11.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f92fb7fe700 1 --2- 172.21.15.44:0/1212782841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594d030 0x563575956900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:54:11.869 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f92fbfff700 1 --2- 172.21.15.44:0/1212782841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594c710 0x56357594caf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f92ec011d70 tx=0x7f92ec014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=124ab302033218fe server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:11.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f92e27fc700 1 -- 172.21.15.44:0/1212782841 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec011da0 con 0x56357594c710 2023-09-10T00:54:11.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.854+0000 7f92e27fc700 1 -- 172.21.15.44:0/1212782841 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92ec004020 con 0x56357594c710 2023-09-10T00:54:11.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.855+0000 7f92e27fc700 1 -- 172.21.15.44:0/1212782841 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec003c30 con 0x56357594c710 2023-09-10T00:54:11.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.855+0000 7f930eae7b80 1 -- 172.21.15.44:0/1212782841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594c710 msgr2=0x56357594caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:11.870 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.855+0000 7f930eae7b80 1 --2- 172.21.15.44:0/1212782841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594c710 0x56357594caf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f92ec011d70 tx=0x7f92ec014000 comp rx=0 tx=0).stop 2023-09-10T00:54:11.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.855+0000 7f930eae7b80 1 -- 172.21.15.44:0/1212782841 shutdown_connections 2023-09-10T00:54:11.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.855+0000 7f930eae7b80 1 --2- 172.21.15.44:0/1212782841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563575956e40 0x563575959230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.855+0000 7f930eae7b80 1 --2- 172.21.15.44:0/1212782841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594c710 0x56357594caf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.855+0000 7f930eae7b80 1 --2- 172.21.15.44:0/1212782841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594d030 0x563575956900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.871 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.855+0000 7f930eae7b80 1 -- 172.21.15.44:0/1212782841 >> 172.21.15.44:0/1212782841 conn(0x5635758c1090 msgr2=0x563575947bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:11.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.855+0000 7f930eae7b80 1 -- 172.21.15.44:0/1212782841 shutdown_connections 2023-09-10T00:54:11.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.855+0000 7f930eae7b80 1 -- 172.21.15.44:0/1212782841 wait complete. 2023-09-10T00:54:11.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.856+0000 7f930eae7b80 1 -- start start 2023-09-10T00:54:11.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.856+0000 7f930eae7b80 1 Processor -- start 2023-09-10T00:54:11.872 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.856+0000 7f930eae7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594c710 0x5635759567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.856+0000 7f930eae7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594d030 0x563575953a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.856+0000 7f930eae7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563575956e40 0x563575953fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.856+0000 7f930eae7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357583cb70 con 0x563575956e40 2023-09-10T00:54:11.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.856+0000 7f930eae7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357583c870 con 0x56357594c710 2023-09-10T00:54:11.873 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.856+0000 7f930eae7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563575720920 con 0x56357594d030 2023-09-10T00:54:11.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.856+0000 7f930eae7b80 10 client.0 _ll_register_callbacks cb 0x563575786a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:54:11.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.856+0000 7f92fbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594c710 0x5635759567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.856+0000 7f92fbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594c710 0x5635759567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55050/0 (socket says 172.21.15.44:55050) 2023-09-10T00:54:11.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.856+0000 7f92fbfff700 1 -- 172.21.15.44:0/1654334498 learned_addr learned my addr 172.21.15.44:0/1654334498 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:11.874 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.856+0000 7f9300ebe700 1 --2- 172.21.15.44:0/1654334498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563575956e40 0x563575953fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.856+0000 7f92fb7fe700 1 --2- 172.21.15.44:0/1654334498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594d030 0x563575953a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.857+0000 7f9300ebe700 1 -- 172.21.15.44:0/1654334498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594d030 msgr2=0x563575953a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:11.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.857+0000 7f9300ebe700 1 --2- 172.21.15.44:0/1654334498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594d030 0x563575953a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.857+0000 7f9300ebe700 1 -- 172.21.15.44:0/1654334498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594c710 msgr2=0x5635759567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:11.875 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.857+0000 7f9300ebe700 1 --2- 172.21.15.44:0/1654334498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594c710 0x5635759567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.857+0000 7f9300ebe700 1 -- 172.21.15.44:0/1654334498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ec017070 con 0x563575956e40 2023-09-10T00:54:11.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.857+0000 7f92fb7fe700 1 --2- 172.21.15.44:0/1654334498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594d030 0x563575953a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:11.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.857+0000 7f92fbfff700 1 --2- 172.21.15.44:0/1654334498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594c710 0x5635759567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:11.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.857+0000 7f9300ebe700 1 --2- 172.21.15.44:0/1654334498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563575956e40 0x563575953fe0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f92f40c9200 tx=0x7f92f4098c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e12b9be46ad6de server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:11.876 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.857+0000 7f92db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:54:11.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.857+0000 7f92db7fe700 1 -- 172.21.15.44:0/1654334498 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f40996f0 con 0x563575956e40 2023-09-10T00:54:11.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.857+0000 7f930eae7b80 1 -- 172.21.15.44:0/1654334498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635757f64f0 con 0x563575956e40 2023-09-10T00:54:11.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.857+0000 7f92db7fe700 1 -- 172.21.15.44:0/1654334498 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f92f4098ff0 con 0x563575956e40 2023-09-10T00:54:11.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.857+0000 7f92db7fe700 1 -- 172.21.15.44:0/1654334498 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92f409ce20 con 0x563575956e40 2023-09-10T00:54:11.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.857+0000 7f930eae7b80 10 client.17712 fetch_fsmap learned FSMap version 68 2023-09-10T00:54:11.877 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.857+0000 7f930eae7b80 1 -- 172.21.15.44:0/1654334498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x5635756b3c60 con 0x563575956e40 2023-09-10T00:54:11.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.857+0000 7f92db7fe700 1 -- 172.21.15.44:0/1654334498 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5635757f64f0 con 0x563575956e40 2023-09-10T00:54:11.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.858+0000 7f92db7fe700 1 -- 172.21.15.44:0/1654334498 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 68) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f92f40a0b40 con 0x563575956e40 2023-09-10T00:54:11.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.858+0000 7f930eae7b80 10 client.17712 fetch_fsmap finished waiting for FSMap version 68 2023-09-10T00:54:11.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.858+0000 7f930eae7b80 10 client.17712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:54:11.878 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.858+0000 7f930eae7b80 20 client.17712 populate_metadata read hostname 'smithi044' 2023-09-10T00:54:11.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.858+0000 7f930eae7b80 1 --2- 172.21.15.44:0/1654334498 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563575966880 0x563575968ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.858+0000 7f930eae7b80 4 client.17712 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:54:11.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.858+0000 7f930eae7b80 1 -- 172.21.15.44:0/1654334498 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635758b38a0 con 0x563575966880 2023-09-10T00:54:11.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.858+0000 7f92fbfff700 1 --2- 172.21.15.44:0/1654334498 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563575966880 0x563575968ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.858+0000 7f92db7fe700 10 client.17712 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:11.879 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.858+0000 7f92fbfff700 1 --2- 172.21.15.44:0/1654334498 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563575966880 0x563575968ca0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb101e4bf22d06c0 server_cookie=2819166dedbb723a in_seq=0 out_seq=0 2023-09-10T00:54:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.859+0000 7f92db7fe700 1 -- 172.21.15.44:0/1654334498 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5635758b38a0 con 0x563575966880 2023-09-10T00:54:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.859+0000 7f92db7fe700 10 client.17712 handle_command_reply: tid=0 2023-09-10T00:54:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.859+0000 7f930eae7b80 1 client.17712 shutdown 2023-09-10T00:54:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.859+0000 7f930eae7b80 1 -- 172.21.15.44:0/1654334498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563575956e40 msgr2=0x563575953fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.859+0000 7f930eae7b80 1 --2- 172.21.15.44:0/1654334498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563575956e40 0x563575953fe0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f92f40c9200 tx=0x7f92f4098c50 comp rx=0 tx=0).stop 2023-09-10T00:54:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.859+0000 7f930eae7b80 1 -- 172.21.15.44:0/1654334498 shutdown_connections 2023-09-10T00:54:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.859+0000 7f930eae7b80 1 --2- 172.21.15.44:0/1654334498 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563575966880 0x563575968ca0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.859+0000 7f930eae7b80 1 --2- 172.21.15.44:0/1654334498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563575956e40 0x563575953fe0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.859+0000 7f92db7fe700 0 client.17712 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.859+0000 7f930eae7b80 1 --2- 172.21.15.44:0/1654334498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594c710 0x5635759567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.859+0000 7f930eae7b80 1 --2- 172.21.15.44:0/1654334498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594d030 0x563575953a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.882 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.859+0000 7f930eae7b80 1 -- 172.21.15.44:0/1654334498 >> 172.21.15.44:0/1654334498 conn(0x5635758c1090 msgr2=0x5635759477e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:11.882 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.859+0000 7f930eae7b80 1 -- 172.21.15.44:0/1654334498 shutdown_connections 2023-09-10T00:54:11.882 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.859+0000 7f930eae7b80 1 -- 172.21.15.44:0/1654334498 wait complete. 2023-09-10T00:54:11.882 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.859+0000 7f930eae7b80 20 client.17712 trim_cache size 0 max 16384 2023-09-10T00:54:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.877+0000 7f930eae7b80 1 Processor -- start 2023-09-10T00:54:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.877+0000 7f930eae7b80 1 -- start start 2023-09-10T00:54:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.877+0000 7f930eae7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56357594c710 0x563575953250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.877+0000 7f930eae7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594d030 0x5635759562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.877+0000 7f930eae7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563575956e40 0x563575950d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.877+0000 7f930eae7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635759486e0 con 0x56357594c710 2023-09-10T00:54:11.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.877+0000 7f930eae7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357594e210 con 0x56357594d030 2023-09-10T00:54:11.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.877+0000 7f930eae7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357594d790 con 0x563575956e40 2023-09-10T00:54:11.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.877+0000 7f92fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56357594c710 0x563575953250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.889 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.877+0000 7f9300ebe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563575956e40 0x563575950d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.877+0000 7f92fb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594d030 0x5635759562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.877+0000 7f92fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56357594c710 0x563575953250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53318/0 (socket says 172.21.15.44:53318) 2023-09-10T00:54:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.877+0000 7f92fbfff700 1 -- 172.21.15.44:0/2554589244 learned_addr learned my addr 172.21.15.44:0/2554589244 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.877+0000 7f92fb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594d030 0x5635759562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55060/0 (socket says 172.21.15.44:55060) 2023-09-10T00:54:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.878+0000 7f9300ebe700 1 -- 172.21.15.44:0/2554589244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594d030 msgr2=0x5635759562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.878+0000 7f9300ebe700 1 --2- 172.21.15.44:0/2554589244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594d030 0x5635759562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.878+0000 7f9300ebe700 1 -- 172.21.15.44:0/2554589244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56357594c710 msgr2=0x563575953250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.878+0000 7f9300ebe700 1 --2- 172.21.15.44:0/2554589244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56357594c710 0x563575953250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.878+0000 7f9300ebe700 1 -- 172.21.15.44:0/2554589244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635756f1250 con 0x563575956e40 2023-09-10T00:54:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.878+0000 7f9300ebe700 1 --2- 172.21.15.44:0/2554589244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563575956e40 0x563575950d00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f92f40954f0 tx=0x7f92f40c92b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9371c944d8e4dfbb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.878+0000 7f92e2ffd700 1 -- 172.21.15.44:0/2554589244 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4098780 con 0x563575956e40 2023-09-10T00:54:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.878+0000 7f92e2ffd700 1 -- 172.21.15.44:0/2554589244 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92f40996f0 con 0x563575956e40 2023-09-10T00:54:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.878+0000 7f92e2ffd700 1 -- 172.21.15.44:0/2554589244 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f40a0b40 con 0x563575956e40 2023-09-10T00:54:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.879+0000 7f930eae7b80 1 -- 172.21.15.44:0/2554589244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563575956e40 msgr2=0x563575950d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.879+0000 7f930eae7b80 1 --2- 172.21.15.44:0/2554589244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563575956e40 0x563575950d00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f92f40954f0 tx=0x7f92f40c92b0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.879+0000 7f930eae7b80 1 -- 172.21.15.44:0/2554589244 shutdown_connections 2023-09-10T00:54:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.879+0000 7f930eae7b80 1 --2- 172.21.15.44:0/2554589244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563575956e40 0x563575950d00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.879+0000 7f930eae7b80 1 --2- 172.21.15.44:0/2554589244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56357594c710 0x563575953250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.879+0000 7f930eae7b80 1 --2- 172.21.15.44:0/2554589244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56357594d030 0x5635759562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.879+0000 7f930eae7b80 1 -- 172.21.15.44:0/2554589244 >> 172.21.15.44:0/2554589244 conn(0x5635758c1090 msgr2=0x563575953a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.879+0000 7f930eae7b80 1 -- 172.21.15.44:0/2554589244 shutdown_connections 2023-09-10T00:54:11.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.879+0000 7f930eae7b80 1 -- 172.21.15.44:0/2554589244 wait complete. 2023-09-10T00:54:11.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.879+0000 7f930eae7b80 1 -- start start 2023-09-10T00:54:11.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.879+0000 7f930eae7b80 1 Processor -- start 2023-09-10T00:54:11.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f930eae7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594c710 0x56357595d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.894 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f930eae7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56357594d030 0x56357595d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f930eae7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563575956e40 0x56357595b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f930eae7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357595bdc0 con 0x56357594d030 2023-09-10T00:54:11.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f930eae7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357595bf30 con 0x563575956e40 2023-09-10T00:54:11.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f930eae7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357595c0f0 con 0x56357594c710 2023-09-10T00:54:11.895 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f930eae7b80 10 client.0 _ll_register_callbacks cb 0x7f92f4098b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:54:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f92fb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56357594d030 0x56357595d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f92fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594c710 0x56357595d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f92fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594c710 0x56357595d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60206/0 (socket says 172.21.15.44:60206) 2023-09-10T00:54:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f92fbfff700 1 -- 172.21.15.44:0/499361109 learned_addr learned my addr 172.21.15.44:0/499361109 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f9300ebe700 1 --2- 172.21.15.44:0/499361109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563575956e40 0x56357595b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f92fbfff700 1 -- 172.21.15.44:0/499361109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563575956e40 msgr2=0x56357595b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f92fbfff700 1 --2- 172.21.15.44:0/499361109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563575956e40 0x56357595b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f92fbfff700 1 -- 172.21.15.44:0/499361109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56357594d030 msgr2=0x56357595d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f92fbfff700 1 --2- 172.21.15.44:0/499361109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56357594d030 0x56357595d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f92fbfff700 1 -- 172.21.15.44:0/499361109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635756b3c60 con 0x56357594c710 2023-09-10T00:54:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f92fb7fe700 1 --2- 172.21.15.44:0/499361109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56357594d030 0x56357595d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f92fbfff700 1 --2- 172.21.15.44:0/499361109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594c710 0x56357595d330 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009210 tx=0x7f92ec005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e19c6e1f43956b84 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.880+0000 7f92db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:54:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.881+0000 7f92db7fe700 1 -- 172.21.15.44:0/499361109 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec02c040 con 0x56357594c710 2023-09-10T00:54:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.881+0000 7f930eae7b80 1 -- 172.21.15.44:0/499361109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92ec004020 con 0x56357594c710 2023-09-10T00:54:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.881+0000 7f92db7fe700 1 -- 172.21.15.44:0/499361109 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f92ec00eae0 con 0x56357594c710 2023-09-10T00:54:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.881+0000 7f92db7fe700 1 -- 172.21.15.44:0/499361109 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92ec031900 con 0x56357594c710 2023-09-10T00:54:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.881+0000 7f930eae7b80 10 client.27319 fetch_fsmap learned FSMap version 68 2023-09-10T00:54:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.881+0000 7f930eae7b80 1 -- 172.21.15.44:0/499361109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x5635756c92b0 con 0x56357594c710 2023-09-10T00:54:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.881+0000 7f92db7fe700 1 -- 172.21.15.44:0/499361109 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92ec004020 con 0x56357594c710 2023-09-10T00:54:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.881+0000 7f92db7fe700 1 -- 172.21.15.44:0/499361109 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 68) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f92ec024030 con 0x56357594c710 2023-09-10T00:54:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.881+0000 7f930eae7b80 10 client.27319 fetch_fsmap finished waiting for FSMap version 68 2023-09-10T00:54:11.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.881+0000 7f930eae7b80 10 client.27319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:54:11.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.881+0000 7f930eae7b80 20 client.27319 populate_metadata read hostname 'smithi044' 2023-09-10T00:54:11.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.881+0000 7f930eae7b80 1 --2- 172.21.15.44:0/499361109 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5635759bb7c0 0x5635759bdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:11.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.881+0000 7f930eae7b80 4 client.27319 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:54:11.900 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.881+0000 7f930eae7b80 1 -- 172.21.15.44:0/499361109 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563575897920 con 0x5635759bb7c0 2023-09-10T00:54:11.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.882+0000 7f92fb7fe700 1 --2- 172.21.15.44:0/499361109 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5635759bb7c0 0x5635759bdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:11.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.882+0000 7f92fb7fe700 1 --2- 172.21.15.44:0/499361109 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5635759bb7c0 0x5635759bdbe0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40fa600ce787725c server_cookie=9ffbf2e92c2423a5 in_seq=0 out_seq=0 2023-09-10T00:54:11.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.882+0000 7f92db7fe700 10 client.27319 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:11.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.882+0000 7f92db7fe700 1 -- 172.21.15.44:0/499361109 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563575897920 con 0x5635759bb7c0 2023-09-10T00:54:11.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.882+0000 7f92db7fe700 10 client.27319 handle_command_reply: tid=0 2023-09-10T00:54:11.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.882+0000 7f930eae7b80 1 client.27319 shutdown 2023-09-10T00:54:11.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.882+0000 7f930eae7b80 1 -- 172.21.15.44:0/499361109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594c710 msgr2=0x56357595d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:11.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.882+0000 7f930eae7b80 1 --2- 172.21.15.44:0/499361109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594c710 0x56357595d330 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009210 tx=0x7f92ec005ce0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.882+0000 7f930eae7b80 1 -- 172.21.15.44:0/499361109 shutdown_connections 2023-09-10T00:54:11.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.882+0000 7f930eae7b80 1 --2- 172.21.15.44:0/499361109 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5635759bb7c0 0x5635759bdbe0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f930eae7b80 1 --2- 172.21.15.44:0/499361109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563575956e40 0x56357595b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f92db7fe700 0 client.27319 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:11.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f930eae7b80 1 --2- 172.21.15.44:0/499361109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56357594c710 0x56357595d330 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f930eae7b80 1 --2- 172.21.15.44:0/499361109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56357594d030 0x56357595d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f930eae7b80 1 -- 172.21.15.44:0/499361109 >> 172.21.15.44:0/499361109 conn(0x5635758c1090 msgr2=0x56357594b700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:11.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f930eae7b80 1 -- 172.21.15.44:0/499361109 shutdown_connections 2023-09-10T00:54:11.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f930eae7b80 1 -- 172.21.15.44:0/499361109 wait complete. 2023-09-10T00:54:11.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f930eae7b80 20 client.27319 trim_cache size 0 max 16384 2023-09-10T00:54:11.904 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:54:11.904 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:54:11.905 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:54:11.905 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:54:11.905 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:54:11.905 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:54:11.905 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:54:11.906 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:54:11.906 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:54:11.906 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:54:11.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f9302921700 1 -- 172.21.15.44:0/2258921824 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f92e40a07b0 msgr2=0x7f92e40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:11.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f9302921700 1 --2- 172.21.15.44:0/2258921824 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f92e40a07b0 0x7f92e40a2c70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009be0 tx=0x7f92ec009260 comp rx=0 tx=0).stop 2023-09-10T00:54:11.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f9302921700 1 -- 172.21.15.44:0/2258921824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1054e0 msgr2=0x7f92fc198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:11.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f9302921700 1 --2- 172.21.15.44:0/2258921824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1054e0 0x7f92fc198640 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f92f400ef30 tx=0x7f92f400c3a0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f9302921700 1 -- 172.21.15.44:0/2258921824 shutdown_connections 2023-09-10T00:54:11.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f9302921700 1 --2- 172.21.15.44:0/2258921824 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f92e40a07b0 0x7f92e40a2c70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f9302921700 1 --2- 172.21.15.44:0/2258921824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1054e0 0x7f92fc198640 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f9302921700 1 --2- 172.21.15.44:0/2258921824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92fc0ff440 0x7f92fc19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f9302921700 1 --2- 172.21.15.44:0/2258921824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92fc0ffe00 0x7f92fc19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:11.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f9302921700 1 -- 172.21.15.44:0/2258921824 >> 172.21.15.44:0/2258921824 conn(0x7f92fc0faf50 msgr2=0x7f92fc10ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:11.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f9302921700 1 -- 172.21.15.44:0/2258921824 shutdown_connections 2023-09-10T00:54:11.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:11.883+0000 7f9302921700 1 -- 172.21.15.44:0/2258921824 wait complete. 2023-09-10T00:54:11.909 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:11 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:54:11.910 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:54:12.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:11 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:54:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:12 smithi131 ceph-mon[103713]: pgmap v424: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 3.0 MiB/s wr, 410 op/s 2023-09-10T00:54:13.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:12 smithi044 ceph-mon[98676]: pgmap v424: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 3.0 MiB/s wr, 410 op/s 2023-09-10T00:54:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:12 smithi099 ceph-mon[102692]: pgmap v424: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 3.0 MiB/s wr, 410 op/s 2023-09-10T00:54:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:14 smithi131 ceph-mon[103713]: pgmap v425: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 3.0 MiB/s wr, 410 op/s 2023-09-10T00:54:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:14 smithi044 ceph-mon[98676]: pgmap v425: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 3.0 MiB/s wr, 410 op/s 2023-09-10T00:54:15.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:14 smithi099 ceph-mon[102692]: pgmap v425: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 3.0 MiB/s wr, 410 op/s 2023-09-10T00:54:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:16 smithi131 ceph-mon[103713]: pgmap v426: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 3.0 MiB/s wr, 396 op/s 2023-09-10T00:54:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:16 smithi131 ceph-mon[103713]: osdmap e108: 12 total, 12 up, 12 in 2023-09-10T00:54:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:16 smithi044 ceph-mon[98676]: pgmap v426: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 3.0 MiB/s wr, 396 op/s 2023-09-10T00:54:17.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:16 smithi044 ceph-mon[98676]: osdmap e108: 12 total, 12 up, 12 in 2023-09-10T00:54:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:16 smithi099 ceph-mon[102692]: pgmap v426: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 3.0 MiB/s wr, 396 op/s 2023-09-10T00:54:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:16 smithi099 ceph-mon[102692]: osdmap e108: 12 total, 12 up, 12 in 2023-09-10T00:54:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:18 smithi131 ceph-mon[103713]: pgmap v428: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 3.5 MiB/s wr, 458 op/s 2023-09-10T00:54:19.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:18 smithi044 ceph-mon[98676]: pgmap v428: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 3.5 MiB/s wr, 458 op/s 2023-09-10T00:54:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:18 smithi099 ceph-mon[102692]: pgmap v428: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 3.5 MiB/s wr, 458 op/s 2023-09-10T00:54:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.901+0000 7ff6bcad4700 1 -- 172.21.15.44:0/3544450332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b8100120 msgr2=0x7ff6b810c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.901+0000 7ff6bcad4700 1 --2- 172.21.15.44:0/3544450332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b8100120 0x7ff6b810c330 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000b0d0 tx=0x7ff6b000b3e0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.901+0000 7ff6bcad4700 1 -- 172.21.15.44:0/3544450332 shutdown_connections 2023-09-10T00:54:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.901+0000 7ff6bcad4700 1 --2- 172.21.15.44:0/3544450332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b8100120 0x7ff6b810c330 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.901+0000 7ff6bcad4700 1 --2- 172.21.15.44:0/3544450332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6b81055b0 0x7ff6b8105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.901+0000 7ff6bcad4700 1 --2- 172.21.15.44:0/3544450332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6b80ff760 0x7ff6b80ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.901+0000 7ff6bcad4700 1 -- 172.21.15.44:0/3544450332 >> 172.21.15.44:0/3544450332 conn(0x7ff6b8074f30 msgr2=0x7ff6b8075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.901+0000 7ff6bcad4700 1 -- 172.21.15.44:0/3544450332 shutdown_connections 2023-09-10T00:54:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.901+0000 7ff6bcad4700 1 -- 172.21.15.44:0/3544450332 wait complete. 2023-09-10T00:54:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.902+0000 7ff6bcad4700 1 Processor -- start 2023-09-10T00:54:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.902+0000 7ff6bcad4700 1 -- start start 2023-09-10T00:54:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6bcad4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6b80ff760 0x7ff6b819b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6bcad4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6b8100120 0x7ff6b819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6bcad4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b81055b0 0x7ff6b8195380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6bcad4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b81959f0 con 0x7ff6b81055b0 2023-09-10T00:54:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6bcad4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b8195b60 con 0x7ff6b8100120 2023-09-10T00:54:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6bcad4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b8195d20 con 0x7ff6b80ff760 2023-09-10T00:54:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6b6d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b81055b0 0x7ff6b8195380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6b659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6b80ff760 0x7ff6b819b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6b6d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b81055b0 0x7ff6b8195380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53342/0 (socket says 172.21.15.44:53342) 2023-09-10T00:54:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6b6d9d700 1 -- 172.21.15.44:0/698456750 learned_addr learned my addr 172.21.15.44:0/698456750 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6b5d9b700 1 --2- 172.21.15.44:0/698456750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6b8100120 0x7ff6b819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6b6d9d700 1 -- 172.21.15.44:0/698456750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6b80ff760 msgr2=0x7ff6b819b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6b6d9d700 1 --2- 172.21.15.44:0/698456750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6b80ff760 0x7ff6b819b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6b6d9d700 1 -- 172.21.15.44:0/698456750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6b8100120 msgr2=0x7ff6b819ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6b6d9d700 1 --2- 172.21.15.44:0/698456750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6b8100120 0x7ff6b819ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6b6d9d700 1 -- 172.21.15.44:0/698456750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b0009d00 con 0x7ff6b81055b0 2023-09-10T00:54:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.903+0000 7ff6b6d9d700 1 --2- 172.21.15.44:0/698456750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b81055b0 0x7ff6b8195380 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0008e50 tx=0x7ff6b0008e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.904+0000 7ff6af7fe700 1 -- 172.21.15.44:0/698456750 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b000e040 con 0x7ff6b81055b0 2023-09-10T00:54:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.904+0000 7ff6af7fe700 1 -- 172.21.15.44:0/698456750 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff6b0003e90 con 0x7ff6b81055b0 2023-09-10T00:54:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.904+0000 7ff6bcad4700 1 -- 172.21.15.44:0/698456750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b8195fa0 con 0x7ff6b81055b0 2023-09-10T00:54:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.904+0000 7ff6af7fe700 1 -- 172.21.15.44:0/698456750 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b001b460 con 0x7ff6b81055b0 2023-09-10T00:54:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.904+0000 7ff6bcad4700 1 -- 172.21.15.44:0/698456750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b81a2510 con 0x7ff6b81055b0 2023-09-10T00:54:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.906+0000 7ff6af7fe700 1 -- 172.21.15.44:0/698456750 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ff6b0003a00 con 0x7ff6b81055b0 2023-09-10T00:54:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.906+0000 7ff6af7fe700 1 --2- 172.21.15.44:0/698456750 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff6a40a07b0 0x7ff6a40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.907+0000 7ff6af7fe700 1 -- 172.21.15.44:0/698456750 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff6b00cd420 con 0x7ff6b81055b0 2023-09-10T00:54:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.907+0000 7ff6b659c700 1 --2- 172.21.15.44:0/698456750 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff6a40a07b0 0x7ff6a40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.907+0000 7ff6b659c700 1 --2- 172.21.15.44:0/698456750 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff6a40a07b0 0x7ff6a40a2c70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009c60 tx=0x7ff6a0009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.907+0000 7ff6c8c9ab80 1 Processor -- start 2023-09-10T00:54:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6c8c9ab80 1 -- start start 2023-09-10T00:54:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6c8c9ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a8f3b0 0x555e15a8f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6c8c9ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e15a89510 0x555e15a89990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6c8c9ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e15a89ed0 0x555e15a96140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6c8c9ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e159e06d0 con 0x555e15a8f3b0 2023-09-10T00:54:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6c8c9ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e159ddff0 con 0x555e15a89ed0 2023-09-10T00:54:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6c8c9ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e159d98e0 con 0x555e15a89510 2023-09-10T00:54:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6b659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a8f3b0 0x555e15a8f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6b659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a8f3b0 0x555e15a8f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53346/0 (socket says 172.21.15.44:53346) 2023-09-10T00:54:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6b659c700 1 -- 172.21.15.44:0/2797193184 learned_addr learned my addr 172.21.15.44:0/2797193184 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6b5d9b700 1 --2- 172.21.15.44:0/2797193184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e15a89510 0x555e15a89990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6b6d9d700 1 --2- 172.21.15.44:0/2797193184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e15a89ed0 0x555e15a96140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6b659c700 1 -- 172.21.15.44:0/2797193184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e15a89510 msgr2=0x555e15a89990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6b659c700 1 --2- 172.21.15.44:0/2797193184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e15a89510 0x555e15a89990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6b659c700 1 -- 172.21.15.44:0/2797193184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e15a89ed0 msgr2=0x555e15a96140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6b659c700 1 --2- 172.21.15.44:0/2797193184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e15a89ed0 0x555e15a96140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.908+0000 7ff6b659c700 1 -- 172.21.15.44:0/2797193184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e15805d80 con 0x555e15a8f3b0 2023-09-10T00:54:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.909+0000 7ff6b659c700 1 --2- 172.21.15.44:0/2797193184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a8f3b0 0x555e15a8f790 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000e4e0 tx=0x7ff6a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7e84e72695a0b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.909+0000 7ff6acff9700 1 -- 172.21.15.44:0/2797193184 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a000ee00 con 0x555e15a8f3b0 2023-09-10T00:54:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.909+0000 7ff6acff9700 1 -- 172.21.15.44:0/2797193184 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff6a0021070 con 0x555e15a8f3b0 2023-09-10T00:54:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.909+0000 7ff6acff9700 1 -- 172.21.15.44:0/2797193184 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a001d740 con 0x555e15a8f3b0 2023-09-10T00:54:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.909+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/2797193184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a8f3b0 msgr2=0x555e15a8f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.909+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/2797193184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a8f3b0 0x555e15a8f790 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000e4e0 tx=0x7ff6a0014000 comp rx=0 tx=0).stop 2023-09-10T00:54:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/2797193184 shutdown_connections 2023-09-10T00:54:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/2797193184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e15a89ed0 0x555e15a96140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/2797193184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a8f3b0 0x555e15a8f790 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/2797193184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e15a89510 0x555e15a89990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/2797193184 >> 172.21.15.44:0/2797193184 conn(0x555e159fc030 msgr2=0x555e15a82560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/2797193184 shutdown_connections 2023-09-10T00:54:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/2797193184 wait complete. 2023-09-10T00:54:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6c8c9ab80 1 -- start start 2023-09-10T00:54:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6c8c9ab80 1 Processor -- start 2023-09-10T00:54:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6c8c9ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e15a89510 0x555e15a95f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6c8c9ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89ed0 0x555e15a93290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6c8c9ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e15a8f3b0 0x555e15a93810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6c8c9ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e15977b10 con 0x555e15a89ed0 2023-09-10T00:54:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6c8c9ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e15977810 con 0x555e15a89510 2023-09-10T00:54:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6c8c9ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e1585b8c0 con 0x555e15a8f3b0 2023-09-10T00:54:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6c8c9ab80 10 client.0 _ll_register_callbacks cb 0x555e158c1a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:54:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6b5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89ed0 0x555e15a93290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6b5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89ed0 0x555e15a93290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53354/0 (socket says 172.21.15.44:53354) 2023-09-10T00:54:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.910+0000 7ff6b659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e15a89510 0x555e15a95f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.911+0000 7ff6b5d9b700 1 -- 172.21.15.44:0/1559958582 learned_addr learned my addr 172.21.15.44:0/1559958582 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.911+0000 7ff6b6d9d700 1 --2- 172.21.15.44:0/1559958582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e15a8f3b0 0x555e15a93810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.911+0000 7ff6b5d9b700 1 -- 172.21.15.44:0/1559958582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e15a8f3b0 msgr2=0x555e15a93810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.911+0000 7ff6b5d9b700 1 --2- 172.21.15.44:0/1559958582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e15a8f3b0 0x555e15a93810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.911+0000 7ff6b5d9b700 1 -- 172.21.15.44:0/1559958582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e15a89510 msgr2=0x555e15a95f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.911+0000 7ff6b5d9b700 1 --2- 172.21.15.44:0/1559958582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e15a89510 0x555e15a95f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.911+0000 7ff6b5d9b700 1 -- 172.21.15.44:0/1559958582 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a0017070 con 0x555e15a89ed0 2023-09-10T00:54:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.911+0000 7ff6b5d9b700 1 --2- 172.21.15.44:0/1559958582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89ed0 0x555e15a93290 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8012840 tx=0x7ff6a8012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c954a58b03315a2f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.911+0000 7ff69dffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:54:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.911+0000 7ff69dffb700 1 -- 172.21.15.44:0/1559958582 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8017070 con 0x555e15a89ed0 2023-09-10T00:54:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.911+0000 7ff69dffb700 1 -- 172.21.15.44:0/1559958582 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff6a8004830 con 0x555e15a89ed0 2023-09-10T00:54:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.911+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/1559958582 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e15931490 con 0x555e15a89ed0 2023-09-10T00:54:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.911+0000 7ff69dffb700 1 -- 172.21.15.44:0/1559958582 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6a8004a30 con 0x555e15a89ed0 2023-09-10T00:54:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.911+0000 7ff6c8c9ab80 10 client.17730 fetch_fsmap learned FSMap version 68 2023-09-10T00:54:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.911+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/1559958582 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x555e157eec00 con 0x555e15a89ed0 2023-09-10T00:54:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.912+0000 7ff69dffb700 1 -- 172.21.15.44:0/1559958582 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555e15931490 con 0x555e15a89ed0 2023-09-10T00:54:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.912+0000 7ff69dffb700 1 -- 172.21.15.44:0/1559958582 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 68) v1 ==== 7922+0+0 (secure 0 0 0) 0x7ff6a80104c0 con 0x555e15a89ed0 2023-09-10T00:54:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.912+0000 7ff6c8c9ab80 10 client.17730 fetch_fsmap finished waiting for FSMap version 68 2023-09-10T00:54:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.912+0000 7ff6c8c9ab80 10 client.17730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:54:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.912+0000 7ff6c8c9ab80 20 client.17730 populate_metadata read hostname 'smithi044' 2023-09-10T00:54:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.912+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/1559958582 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555e15aa36b0 0x555e15aa5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.912+0000 7ff6c8c9ab80 4 client.17730 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:54:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.912+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/1559958582 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e159ee840 con 0x555e15aa36b0 2023-09-10T00:54:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.912+0000 7ff6b659c700 1 --2- 172.21.15.44:0/1559958582 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555e15aa36b0 0x555e15aa5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.912+0000 7ff69dffb700 10 client.17730 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.912+0000 7ff6b659c700 1 --2- 172.21.15.44:0/1559958582 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555e15aa36b0 0x555e15aa5ad0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e437635e51594c server_cookie=e415af4f86fe1793 in_seq=0 out_seq=0 2023-09-10T00:54:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.913+0000 7ff69dffb700 1 -- 172.21.15.44:0/1559958582 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555e159ee840 con 0x555e15aa36b0 2023-09-10T00:54:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.913+0000 7ff69dffb700 10 client.17730 handle_command_reply: tid=0 2023-09-10T00:54:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.913+0000 7ff6c8c9ab80 1 client.17730 shutdown 2023-09-10T00:54:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.913+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/1559958582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89ed0 msgr2=0x555e15a93290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.913+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/1559958582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89ed0 0x555e15a93290 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8012840 tx=0x7ff6a8012b50 comp rx=0 tx=0).stop 2023-09-10T00:54:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.913+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/1559958582 shutdown_connections 2023-09-10T00:54:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.913+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/1559958582 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555e15aa36b0 0x555e15aa5ad0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.913+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/1559958582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e15a8f3b0 0x555e15a93810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.913+0000 7ff69dffb700 0 client.17730 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.913+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/1559958582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e15a89510 0x555e15a95f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.913+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/1559958582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89ed0 0x555e15a93290 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.913+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/1559958582 >> 172.21.15.44:0/1559958582 conn(0x555e159fc030 msgr2=0x555e15a817a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.913+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/1559958582 shutdown_connections 2023-09-10T00:54:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.914+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/1559958582 wait complete. 2023-09-10T00:54:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.914+0000 7ff6c8c9ab80 20 client.17730 trim_cache size 0 max 16384 2023-09-10T00:54:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6c8c9ab80 1 Processor -- start 2023-09-10T00:54:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6c8c9ab80 1 -- start start 2023-09-10T00:54:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6c8c9ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e159fc030 0x555e15a92a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6c8c9ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89510 0x555e15a88a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6c8c9ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e15a89ed0 0x555e15a88f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6c8c9ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e15a7c8e0 con 0x555e15a89510 2023-09-10T00:54:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6c8c9ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e15a8a790 con 0x555e159fc030 2023-09-10T00:54:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6c8c9ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e15a948a0 con 0x555e15a89ed0 2023-09-10T00:54:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6b5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89510 0x555e15a88a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6b5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89510 0x555e15a88a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53360/0 (socket says 172.21.15.44:53360) 2023-09-10T00:54:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6b5d9b700 1 -- 172.21.15.44:0/349114843 learned_addr learned my addr 172.21.15.44:0/349114843 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6b6d9d700 1 --2- 172.21.15.44:0/349114843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e15a89ed0 0x555e15a88f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6b659c700 1 --2- 172.21.15.44:0/349114843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e159fc030 0x555e15a92a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6b5d9b700 1 -- 172.21.15.44:0/349114843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e15a89ed0 msgr2=0x555e15a88f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6b5d9b700 1 --2- 172.21.15.44:0/349114843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e15a89ed0 0x555e15a88f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6b5d9b700 1 -- 172.21.15.44:0/349114843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e159fc030 msgr2=0x555e15a92a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6b5d9b700 1 --2- 172.21.15.44:0/349114843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e159fc030 0x555e15a92a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6b5d9b700 1 -- 172.21.15.44:0/349114843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e1582c1f0 con 0x555e15a89510 2023-09-10T00:54:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.929+0000 7ff6b5d9b700 1 --2- 172.21.15.44:0/349114843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89510 0x555e15a88a10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff6a80108f0 tx=0x7ff6a80056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7ab97452609911 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.930+0000 7ff6ad7fa700 1 -- 172.21.15.44:0/349114843 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8017070 con 0x555e15a89510 2023-09-10T00:54:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.930+0000 7ff6ad7fa700 1 -- 172.21.15.44:0/349114843 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff6a8018040 con 0x555e15a89510 2023-09-10T00:54:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.930+0000 7ff6ad7fa700 1 -- 172.21.15.44:0/349114843 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a801cd00 con 0x555e15a89510 2023-09-10T00:54:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.930+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/349114843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89510 msgr2=0x555e15a88a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.930+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/349114843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89510 0x555e15a88a10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff6a80108f0 tx=0x7ff6a80056c0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.930+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/349114843 shutdown_connections 2023-09-10T00:54:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.930+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/349114843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e15a89ed0 0x555e15a88f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.930+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/349114843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e159fc030 0x555e15a92a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.930+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/349114843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89510 0x555e15a88a10 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.930+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/349114843 >> 172.21.15.44:0/349114843 conn(0x555e15a93970 msgr2=0x555e15a862f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.930+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/349114843 shutdown_connections 2023-09-10T00:54:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.930+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/349114843 wait complete. 2023-09-10T00:54:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6c8c9ab80 1 -- start start 2023-09-10T00:54:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6c8c9ab80 1 Processor -- start 2023-09-10T00:54:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6c8c9ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e159fc030 0x555e15a96900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6c8c9ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e15a89510 0x555e15a96e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6c8c9ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89ed0 0x555e15af88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6c8c9ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e15af8df0 con 0x555e15a89ed0 2023-09-10T00:54:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6c8c9ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e15af8f30 con 0x555e15a89510 2023-09-10T00:54:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6c8c9ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e15af9070 con 0x555e159fc030 2023-09-10T00:54:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6c8c9ab80 10 client.0 _ll_register_callbacks cb 0x7ff6a8009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:54:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6b6d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89ed0 0x555e15af88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6b659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e159fc030 0x555e15a96900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6b6d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89ed0 0x555e15af88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53370/0 (socket says 172.21.15.44:53370) 2023-09-10T00:54:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6b6d9d700 1 -- 172.21.15.44:0/3972004946 learned_addr learned my addr 172.21.15.44:0/3972004946 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6b5d9b700 1 --2- 172.21.15.44:0/3972004946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e15a89510 0x555e15a96e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6b6d9d700 1 -- 172.21.15.44:0/3972004946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e159fc030 msgr2=0x555e15a96900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6b6d9d700 1 --2- 172.21.15.44:0/3972004946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e159fc030 0x555e15a96900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6b6d9d700 1 -- 172.21.15.44:0/3972004946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e15a89510 msgr2=0x555e15a96e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.931+0000 7ff6b6d9d700 1 --2- 172.21.15.44:0/3972004946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e15a89510 0x555e15a96e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff6b6d9d700 1 -- 172.21.15.44:0/3972004946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a8021040 con 0x555e15a89ed0 2023-09-10T00:54:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff6b659c700 1 --2- 172.21.15.44:0/3972004946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e159fc030 0x555e15a96900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff6b6d9d700 1 --2- 172.21.15.44:0/3972004946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89ed0 0x555e15af88b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff6b009a4f0 tx=0x7ff6b009a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911670bf6c30f639 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff69dffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:54:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff69dffb700 1 -- 172.21.15.44:0/3972004946 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0003cb0 con 0x555e15a89ed0 2023-09-10T00:54:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff69dffb700 1 -- 172.21.15.44:0/3972004946 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff6b009ac60 con 0x555e15a89ed0 2023-09-10T00:54:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/3972004946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6a0021070 con 0x555e15a89ed0 2023-09-10T00:54:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff69dffb700 1 -- 172.21.15.44:0/3972004946 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6b009e150 con 0x555e15a89ed0 2023-09-10T00:54:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff6c8c9ab80 10 client.17736 fetch_fsmap learned FSMap version 68 2023-09-10T00:54:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/3972004946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x555e15804250 con 0x555e15a89ed0 2023-09-10T00:54:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff69dffb700 1 -- 172.21.15.44:0/3972004946 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff6a0021070 con 0x555e15a89ed0 2023-09-10T00:54:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff69dffb700 1 -- 172.21.15.44:0/3972004946 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 68) v1 ==== 7922+0+0 (secure 0 0 0) 0x7ff6b0096c70 con 0x555e15a89ed0 2023-09-10T00:54:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff6c8c9ab80 10 client.17736 fetch_fsmap finished waiting for FSMap version 68 2023-09-10T00:54:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff6c8c9ab80 10 client.17736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:54:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff6c8c9ab80 20 client.17736 populate_metadata read hostname 'smithi044' 2023-09-10T00:54:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/3972004946 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555e15a8f3b0 0x555e15afc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff6c8c9ab80 4 client.17736 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:54:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.932+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/3972004946 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555e159d28c0 con 0x555e15a8f3b0 2023-09-10T00:54:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.933+0000 7ff6b659c700 1 --2- 172.21.15.44:0/3972004946 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555e15a8f3b0 0x555e15afc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.933+0000 7ff69dffb700 10 client.17736 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.933+0000 7ff6b659c700 1 --2- 172.21.15.44:0/3972004946 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555e15a8f3b0 0x555e15afc580 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=796bd90a493ee154 server_cookie=8a494989f9d33948 in_seq=0 out_seq=0 2023-09-10T00:54:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.954+0000 7ff69dffb700 1 -- 172.21.15.44:0/3972004946 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555e159d28c0 con 0x555e15a8f3b0 2023-09-10T00:54:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.954+0000 7ff69dffb700 10 client.17736 handle_command_reply: tid=0 2023-09-10T00:54:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.954+0000 7ff6c8c9ab80 1 client.17736 shutdown 2023-09-10T00:54:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.954+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/3972004946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89ed0 msgr2=0x555e15af88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.954+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/3972004946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89ed0 0x555e15af88b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff6b009a4f0 tx=0x7ff6b009a330 comp rx=0 tx=0).stop 2023-09-10T00:54:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.954+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/3972004946 shutdown_connections 2023-09-10T00:54:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.954+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/3972004946 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555e15a8f3b0 0x555e15afc580 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.954+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/3972004946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e15a89ed0 0x555e15af88b0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.954+0000 7ff69dffb700 0 client.17736 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.954+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/3972004946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e159fc030 0x555e15a96900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.954+0000 7ff6c8c9ab80 1 --2- 172.21.15.44:0/3972004946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e15a89510 0x555e15a96e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.954+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/3972004946 >> 172.21.15.44:0/3972004946 conn(0x555e15a93970 msgr2=0x555e15a952b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.954+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/3972004946 shutdown_connections 2023-09-10T00:54:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.954+0000 7ff6c8c9ab80 1 -- 172.21.15.44:0/3972004946 wait complete. 2023-09-10T00:54:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.955+0000 7ff6c8c9ab80 20 client.17736 trim_cache size 0 max 16384 2023-09-10T00:54:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:54:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:54:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:54:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:54:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.955+0000 7ff6bcad4700 1 -- 172.21.15.44:0/698456750 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff6a40a07b0 msgr2=0x7ff6a40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.955+0000 7ff6bcad4700 1 --2- 172.21.15.44:0/698456750 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff6a40a07b0 0x7ff6a40a2c70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009c60 tx=0x7ff6a0009260 comp rx=0 tx=0).stop 2023-09-10T00:54:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.955+0000 7ff6bcad4700 1 -- 172.21.15.44:0/698456750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b81055b0 msgr2=0x7ff6b8195380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.955+0000 7ff6bcad4700 1 --2- 172.21.15.44:0/698456750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b81055b0 0x7ff6b8195380 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0008e50 tx=0x7ff6b0008e80 comp rx=0 tx=0).stop 2023-09-10T00:54:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.955+0000 7ff6bcad4700 1 -- 172.21.15.44:0/698456750 shutdown_connections 2023-09-10T00:54:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.955+0000 7ff6bcad4700 1 --2- 172.21.15.44:0/698456750 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff6a40a07b0 0x7ff6a40a2c70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.955+0000 7ff6bcad4700 1 --2- 172.21.15.44:0/698456750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b81055b0 0x7ff6b8195380 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.955+0000 7ff6bcad4700 1 --2- 172.21.15.44:0/698456750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6b80ff760 0x7ff6b819b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.955+0000 7ff6bcad4700 1 --2- 172.21.15.44:0/698456750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6b8100120 0x7ff6b819ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.955+0000 7ff6bcad4700 1 -- 172.21.15.44:0/698456750 >> 172.21.15.44:0/698456750 conn(0x7ff6b8074f30 msgr2=0x7ff6b80fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.955+0000 7ff6bcad4700 1 -- 172.21.15.44:0/698456750 shutdown_connections 2023-09-10T00:54:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:19.955+0000 7ff6bcad4700 1 -- 172.21.15.44:0/698456750 wait complete. 2023-09-10T00:54:21.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:20 smithi131 ceph-mon[103713]: pgmap v429: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 3.0 MiB/s wr, 396 op/s 2023-09-10T00:54:21.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:20 smithi044 ceph-mon[98676]: pgmap v429: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 3.0 MiB/s wr, 396 op/s 2023-09-10T00:54:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:20 smithi099 ceph-mon[102692]: pgmap v429: 129 pgs: 129 active+clean; 9.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 3.0 MiB/s wr, 396 op/s 2023-09-10T00:54:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:22 smithi131 ceph-mon[103713]: pgmap v430: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 2.8 MiB/s wr, 375 op/s 2023-09-10T00:54:23.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:22 smithi044 ceph-mon[98676]: pgmap v430: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 2.8 MiB/s wr, 375 op/s 2023-09-10T00:54:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:22 smithi099 ceph-mon[102692]: pgmap v430: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 2.8 MiB/s wr, 375 op/s 2023-09-10T00:54:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:24 smithi131 ceph-mon[103713]: pgmap v431: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 2.8 MiB/s wr, 375 op/s 2023-09-10T00:54:25.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:24 smithi044 ceph-mon[98676]: pgmap v431: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 2.8 MiB/s wr, 375 op/s 2023-09-10T00:54:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:24 smithi099 ceph-mon[102692]: pgmap v431: 129 pgs: 129 active+clean; 9.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 2.8 MiB/s wr, 375 op/s 2023-09-10T00:54:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:26 smithi131 ceph-mon[103713]: pgmap v432: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.9 MiB/s wr, 389 op/s 2023-09-10T00:54:27.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:26 smithi044 ceph-mon[98676]: pgmap v432: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.9 MiB/s wr, 389 op/s 2023-09-10T00:54:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:26 smithi099 ceph-mon[102692]: pgmap v432: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.9 MiB/s wr, 389 op/s 2023-09-10T00:54:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:28 smithi131 ceph-mon[103713]: pgmap v433: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.5 MiB/s wr, 337 op/s 2023-09-10T00:54:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:28 smithi044 ceph-mon[98676]: pgmap v433: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.5 MiB/s wr, 337 op/s 2023-09-10T00:54:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:28 smithi099 ceph-mon[102692]: pgmap v433: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.5 MiB/s wr, 337 op/s 2023-09-10T00:54:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:30 smithi131 ceph-mon[103713]: pgmap v434: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.4 MiB/s wr, 324 op/s 2023-09-10T00:54:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:30 smithi044 ceph-mon[98676]: pgmap v434: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.4 MiB/s wr, 324 op/s 2023-09-10T00:54:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:30 smithi099 ceph-mon[102692]: pgmap v434: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.4 MiB/s wr, 324 op/s 2023-09-10T00:54:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:32 smithi131 ceph-mon[103713]: pgmap v435: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.8 MiB/s wr, 503 op/s 2023-09-10T00:54:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:32 smithi044 ceph-mon[98676]: pgmap v435: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.8 MiB/s wr, 503 op/s 2023-09-10T00:54:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:32 smithi099 ceph-mon[102692]: pgmap v435: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.8 MiB/s wr, 503 op/s 2023-09-10T00:54:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.161+0000 7ff2ea9b8700 1 -- 172.21.15.44:0/1707132943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2e40ff760 msgr2=0x7ff2e40ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.161+0000 7ff2ea9b8700 1 --2- 172.21.15.44:0/1707132943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2e40ff760 0x7ff2e40ffbe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009990 tx=0x7ff2d8009ca0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.161+0000 7ff2ea9b8700 1 -- 172.21.15.44:0/1707132943 shutdown_connections 2023-09-10T00:54:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.161+0000 7ff2ea9b8700 1 --2- 172.21.15.44:0/1707132943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2e4100120 0x7ff2e410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.161+0000 7ff2ea9b8700 1 --2- 172.21.15.44:0/1707132943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2e41055b0 0x7ff2e4105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.161+0000 7ff2ea9b8700 1 --2- 172.21.15.44:0/1707132943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2e40ff760 0x7ff2e40ffbe0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.161+0000 7ff2ea9b8700 1 -- 172.21.15.44:0/1707132943 >> 172.21.15.44:0/1707132943 conn(0x7ff2e4074f30 msgr2=0x7ff2e4075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.161+0000 7ff2ea9b8700 1 -- 172.21.15.44:0/1707132943 shutdown_connections 2023-09-10T00:54:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.161+0000 7ff2ea9b8700 1 -- 172.21.15.44:0/1707132943 wait complete. 2023-09-10T00:54:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.162+0000 7ff2ea9b8700 1 Processor -- start 2023-09-10T00:54:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.162+0000 7ff2ea9b8700 1 -- start start 2023-09-10T00:54:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2ea9b8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2e40ff760 0x7ff2e41975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2ea9b8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2e4100120 0x7ff2e4197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2ea9b8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2e41055b0 0x7ff2e41a2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2ea9b8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e41a2a50 con 0x7ff2e40ff760 2023-09-10T00:54:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2ea9b8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e41a2bc0 con 0x7ff2e4100120 2023-09-10T00:54:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2ea9b8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e41a2d80 con 0x7ff2e41055b0 2023-09-10T00:54:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2e40ff760 0x7ff2e41975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2e37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2e4100120 0x7ff2e4197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2e8f55700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2e41055b0 0x7ff2e41a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2e40ff760 0x7ff2e41975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53380/0 (socket says 172.21.15.44:53380) 2023-09-10T00:54:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2e37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2e4100120 0x7ff2e4197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55122/0 (socket says 172.21.15.44:55122) 2023-09-10T00:54:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2e3fff700 1 -- 172.21.15.44:0/2343313679 learned_addr learned my addr 172.21.15.44:0/2343313679 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2e37fe700 1 -- 172.21.15.44:0/2343313679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2e41055b0 msgr2=0x7ff2e41a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2e37fe700 1 --2- 172.21.15.44:0/2343313679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2e41055b0 0x7ff2e41a2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2e37fe700 1 -- 172.21.15.44:0/2343313679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2e40ff760 msgr2=0x7ff2e41975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2e37fe700 1 --2- 172.21.15.44:0/2343313679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2e40ff760 0x7ff2e41975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2e37fe700 1 -- 172.21.15.44:0/2343313679 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d80095f0 con 0x7ff2e4100120 2023-09-10T00:54:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2e3fff700 1 --2- 172.21.15.44:0/2343313679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2e40ff760 0x7ff2e41975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.163+0000 7ff2e37fe700 1 --2- 172.21.15.44:0/2343313679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2e4100120 0x7ff2e4197b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8000c00 tx=0x7ff2d8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.164+0000 7ff2e17fa700 1 -- 172.21.15.44:0/2343313679 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d801d070 con 0x7ff2e4100120 2023-09-10T00:54:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.164+0000 7ff2e17fa700 1 -- 172.21.15.44:0/2343313679 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2d8005030 con 0x7ff2e4100120 2023-09-10T00:54:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.164+0000 7ff2e17fa700 1 -- 172.21.15.44:0/2343313679 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d80049e0 con 0x7ff2e4100120 2023-09-10T00:54:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.164+0000 7ff2ea9b8700 1 -- 172.21.15.44:0/2343313679 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e41a3000 con 0x7ff2e4100120 2023-09-10T00:54:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.164+0000 7ff2ea9b8700 1 -- 172.21.15.44:0/2343313679 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e41a3440 con 0x7ff2e4100120 2023-09-10T00:54:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.166+0000 7ff2e17fa700 1 -- 172.21.15.44:0/2343313679 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ff2d8004bc0 con 0x7ff2e4100120 2023-09-10T00:54:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.166+0000 7ff2e17fa700 1 --2- 172.21.15.44:0/2343313679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff2cc0a0830 0x7ff2cc0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.167+0000 7ff2e17fa700 1 -- 172.21.15.44:0/2343313679 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff2d80cd430 con 0x7ff2e4100120 2023-09-10T00:54:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.167+0000 7ff2e3fff700 1 --2- 172.21.15.44:0/2343313679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff2cc0a0830 0x7ff2cc0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.167+0000 7ff2e3fff700 1 --2- 172.21.15.44:0/2343313679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff2cc0a0830 0x7ff2cc0a2cf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4011550 tx=0x7ff2d4009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.167+0000 7ff2f6b7eb80 1 Processor -- start 2023-09-10T00:54:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.167+0000 7ff2f6b7eb80 1 -- start start 2023-09-10T00:54:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2f6b7eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b44106a1b0 0x55b44106a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2f6b7eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064310 0x55b441064790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2f6b7eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b441064cd0 0x55b441070f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2f6b7eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b440fac6d0 con 0x55b441064310 2023-09-10T00:54:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2f6b7eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b440fa9ff0 con 0x55b44106a1b0 2023-09-10T00:54:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2f6b7eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b440fa58e0 con 0x55b441064cd0 2023-09-10T00:54:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2e37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064310 0x55b441064790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2e3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b44106a1b0 0x55b44106a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2e37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064310 0x55b441064790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53390/0 (socket says 172.21.15.44:53390) 2023-09-10T00:54:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2e37fe700 1 -- 172.21.15.44:0/2077540660 learned_addr learned my addr 172.21.15.44:0/2077540660 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2e8f55700 1 --2- 172.21.15.44:0/2077540660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b441064cd0 0x55b441070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2e3fff700 1 -- 172.21.15.44:0/2077540660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b441064cd0 msgr2=0x55b441070f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2e3fff700 1 --2- 172.21.15.44:0/2077540660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b441064cd0 0x55b441070f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2e3fff700 1 -- 172.21.15.44:0/2077540660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064310 msgr2=0x55b441064790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2e3fff700 1 --2- 172.21.15.44:0/2077540660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064310 0x55b441064790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2e3fff700 1 -- 172.21.15.44:0/2077540660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b440dd1d80 con 0x55b44106a1b0 2023-09-10T00:54:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2e37fe700 1 --2- 172.21.15.44:0/2077540660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064310 0x55b441064790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2e3fff700 1 --2- 172.21.15.44:0/2077540660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b44106a1b0 0x55b44106a590 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4011ca0 tx=0x7ff2d4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a446ea6bc6777b45 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.168+0000 7ff2ca7fc700 1 -- 172.21.15.44:0/2077540660 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4004830 con 0x55b44106a1b0 2023-09-10T00:54:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.169+0000 7ff2ca7fc700 1 -- 172.21.15.44:0/2077540660 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2d4021070 con 0x55b44106a1b0 2023-09-10T00:54:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.169+0000 7ff2ca7fc700 1 -- 172.21.15.44:0/2077540660 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d401c610 con 0x55b44106a1b0 2023-09-10T00:54:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.169+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2077540660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b44106a1b0 msgr2=0x55b44106a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.169+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/2077540660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b44106a1b0 0x55b44106a590 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4011ca0 tx=0x7ff2d4014000 comp rx=0 tx=0).stop 2023-09-10T00:54:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.169+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2077540660 shutdown_connections 2023-09-10T00:54:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.169+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/2077540660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b441064cd0 0x55b441070f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.169+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/2077540660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b44106a1b0 0x55b44106a590 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.169+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/2077540660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064310 0x55b441064790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.169+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2077540660 >> 172.21.15.44:0/2077540660 conn(0x55b440fc8030 msgr2=0x55b44104e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.170+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2077540660 shutdown_connections 2023-09-10T00:54:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.170+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2077540660 wait complete. 2023-09-10T00:54:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.170+0000 7ff2f6b7eb80 1 -- start start 2023-09-10T00:54:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.170+0000 7ff2f6b7eb80 1 Processor -- start 2023-09-10T00:54:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.170+0000 7ff2f6b7eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064310 0x55b44105b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.170+0000 7ff2f6b7eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b441064cd0 0x55b44105b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.170+0000 7ff2f6b7eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b44106a1b0 0x55b441056620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.170+0000 7ff2f6b7eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b440f43b10 con 0x55b441064310 2023-09-10T00:54:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.170+0000 7ff2f6b7eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b440f43810 con 0x55b441064cd0 2023-09-10T00:54:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.170+0000 7ff2f6b7eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b440e278c0 con 0x55b44106a1b0 2023-09-10T00:54:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2f6b7eb80 10 client.0 _ll_register_callbacks cb 0x55b440e8da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:54:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064310 0x55b44105b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2e37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b441064cd0 0x55b44105b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2e8f55700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b44106a1b0 0x55b441056620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064310 0x55b44105b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53394/0 (socket says 172.21.15.44:53394) 2023-09-10T00:54:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2e3fff700 1 -- 172.21.15.44:0/941676191 learned_addr learned my addr 172.21.15.44:0/941676191 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2e37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b441064cd0 0x55b44105b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55136/0 (socket says 172.21.15.44:55136) 2023-09-10T00:54:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2e37fe700 1 -- 172.21.15.44:0/941676191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b44106a1b0 msgr2=0x55b441056620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2e37fe700 1 --2- 172.21.15.44:0/941676191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b44106a1b0 0x55b441056620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2e37fe700 1 -- 172.21.15.44:0/941676191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064310 msgr2=0x55b44105b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2e37fe700 1 --2- 172.21.15.44:0/941676191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064310 0x55b44105b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2e37fe700 1 -- 172.21.15.44:0/941676191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e41a3440 con 0x55b441064cd0 2023-09-10T00:54:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2e3fff700 1 --2- 172.21.15.44:0/941676191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064310 0x55b44105b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2e37fe700 1 --2- 172.21.15.44:0/941676191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b441064cd0 0x55b44105b7f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80a2020 tx=0x7ff2d8099f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b832c09f376338a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:54:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2c37fe700 1 -- 172.21.15.44:0/941676191 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8050070 con 0x55b441064cd0 2023-09-10T00:54:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/941676191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b440efd490 con 0x55b441064cd0 2023-09-10T00:54:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.171+0000 7ff2c37fe700 1 -- 172.21.15.44:0/941676191 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff2d809ac70 con 0x55b441064cd0 2023-09-10T00:54:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.172+0000 7ff2c37fe700 1 -- 172.21.15.44:0/941676191 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2d809a570 con 0x55b441064cd0 2023-09-10T00:54:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.172+0000 7ff2e8f55700 1 --2- 172.21.15.44:0/941676191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b44106a1b0 0x55b441056620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:54:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.172+0000 7ff2f6b7eb80 10 client.27413 fetch_fsmap learned FSMap version 68 2023-09-10T00:54:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.172+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/941676191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7ff2d4017070 con 0x55b441064cd0 2023-09-10T00:54:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.172+0000 7ff2c37fe700 1 -- 172.21.15.44:0/941676191 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b440efd490 con 0x55b441064cd0 2023-09-10T00:54:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.172+0000 7ff2c37fe700 1 -- 172.21.15.44:0/941676191 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 68) v1 ==== 7922+0+0 (secure 0 0 0) 0x7ff2d809a7d0 con 0x55b441064cd0 2023-09-10T00:54:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.172+0000 7ff2f6b7eb80 10 client.27413 fetch_fsmap finished waiting for FSMap version 68 2023-09-10T00:54:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.172+0000 7ff2f6b7eb80 10 client.27413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:54:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.172+0000 7ff2f6b7eb80 20 client.27413 populate_metadata read hostname 'smithi044' 2023-09-10T00:54:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.172+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/941676191 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b44106d160 0x55b44106f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.172+0000 7ff2f6b7eb80 4 client.27413 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:54:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.172+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/941676191 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b440fba840 con 0x55b44106d160 2023-09-10T00:54:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.172+0000 7ff2e3fff700 1 --2- 172.21.15.44:0/941676191 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b44106d160 0x55b44106f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.173+0000 7ff2c37fe700 10 client.27413 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.173+0000 7ff2e3fff700 1 --2- 172.21.15.44:0/941676191 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b44106d160 0x55b44106f550 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6dd9f508499aba8 server_cookie=c22e519a5528bc2f in_seq=0 out_seq=0 2023-09-10T00:54:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.173+0000 7ff2c37fe700 1 -- 172.21.15.44:0/941676191 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b440fba840 con 0x55b44106d160 2023-09-10T00:54:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.173+0000 7ff2c37fe700 10 client.27413 handle_command_reply: tid=0 2023-09-10T00:54:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.173+0000 7ff2f6b7eb80 1 client.27413 shutdown 2023-09-10T00:54:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.174+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/941676191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b441064cd0 msgr2=0x55b44105b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.174+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/941676191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b441064cd0 0x55b44105b7f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80a2020 tx=0x7ff2d8099f50 comp rx=0 tx=0).stop 2023-09-10T00:54:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.174+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/941676191 shutdown_connections 2023-09-10T00:54:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.174+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/941676191 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b44106d160 0x55b44106f550 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.174+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/941676191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b44106a1b0 0x55b441056620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.174+0000 7ff2c37fe700 0 client.27413 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.174+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/941676191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064310 0x55b44105b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.174+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/941676191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b441064cd0 0x55b44105b7f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.174+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/941676191 >> 172.21.15.44:0/941676191 conn(0x55b440fc8030 msgr2=0x55b44104d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.174+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/941676191 shutdown_connections 2023-09-10T00:54:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.174+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/941676191 wait complete. 2023-09-10T00:54:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.174+0000 7ff2f6b7eb80 20 client.27413 trim_cache size 0 max 16384 2023-09-10T00:54:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.190+0000 7ff2f6b7eb80 1 Processor -- start 2023-09-10T00:54:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.190+0000 7ff2f6b7eb80 1 -- start start 2023-09-10T00:54:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2f6b7eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b441064310 0x55b4410565d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2f6b7eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064cd0 0x55b44104df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2f6b7eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b44106a1b0 0x55b44104e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2f6b7eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4410488d0 con 0x55b441064cd0 2023-09-10T00:54:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2f6b7eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b44104fd90 con 0x55b441064310 2023-09-10T00:54:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2f6b7eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b441065590 con 0x55b44106a1b0 2023-09-10T00:54:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2e37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064cd0 0x55b44104df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2e3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b441064310 0x55b4410565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2e3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b441064310 0x55b4410565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55142/0 (socket says 172.21.15.44:55142) 2023-09-10T00:54:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2e37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064cd0 0x55b44104df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53404/0 (socket says 172.21.15.44:53404) 2023-09-10T00:54:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2e3fff700 1 -- 172.21.15.44:0/2561381066 learned_addr learned my addr 172.21.15.44:0/2561381066 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2e8f55700 1 --2- 172.21.15.44:0/2561381066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b44106a1b0 0x55b44104e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2e3fff700 1 -- 172.21.15.44:0/2561381066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b44106a1b0 msgr2=0x55b44104e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2e3fff700 1 --2- 172.21.15.44:0/2561381066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b44106a1b0 0x55b44104e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2e3fff700 1 -- 172.21.15.44:0/2561381066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064cd0 msgr2=0x55b44104df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2e3fff700 1 --2- 172.21.15.44:0/2561381066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064cd0 0x55b44104df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2e3fff700 1 -- 172.21.15.44:0/2561381066 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b440dbac00 con 0x55b441064310 2023-09-10T00:54:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.191+0000 7ff2e3fff700 1 --2- 172.21.15.44:0/2561381066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b441064310 0x55b4410565d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400e5f0 tx=0x7ff2d4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbe04644d2cd10f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.192+0000 7ff2caffd700 1 -- 172.21.15.44:0/2561381066 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d402c040 con 0x55b441064310 2023-09-10T00:54:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.192+0000 7ff2caffd700 1 -- 172.21.15.44:0/2561381066 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2d40315b0 con 0x55b441064310 2023-09-10T00:54:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.192+0000 7ff2caffd700 1 -- 172.21.15.44:0/2561381066 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4026970 con 0x55b441064310 2023-09-10T00:54:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.192+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2561381066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b441064310 msgr2=0x55b4410565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.192+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/2561381066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b441064310 0x55b4410565d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400e5f0 tx=0x7ff2d4017000 comp rx=0 tx=0).stop 2023-09-10T00:54:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.192+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2561381066 shutdown_connections 2023-09-10T00:54:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.192+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/2561381066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b44106a1b0 0x55b44104e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.192+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/2561381066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b441064310 0x55b4410565d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.192+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/2561381066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064cd0 0x55b44104df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.192+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2561381066 >> 172.21.15.44:0/2561381066 conn(0x55b440fc8030 msgr2=0x55b441055ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.192+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2561381066 shutdown_connections 2023-09-10T00:54:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.192+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2561381066 wait complete. 2023-09-10T00:54:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.193+0000 7ff2f6b7eb80 1 -- start start 2023-09-10T00:54:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.193+0000 7ff2f6b7eb80 1 Processor -- start 2023-09-10T00:54:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.193+0000 7ff2f6b7eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b441064310 0x55b44105dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.193+0000 7ff2f6b7eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064cd0 0x55b44105e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.193+0000 7ff2f6b7eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b44106a1b0 0x55b4410b9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.193+0000 7ff2f6b7eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4410ba3a0 con 0x55b441064cd0 2023-09-10T00:54:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.193+0000 7ff2f6b7eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4410ba4e0 con 0x55b44106a1b0 2023-09-10T00:54:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.193+0000 7ff2f6b7eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4410ba6a0 con 0x55b441064310 2023-09-10T00:54:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.193+0000 7ff2f6b7eb80 10 client.0 _ll_register_callbacks cb 0x7ff2d809a0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:54:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.193+0000 7ff2e37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064cd0 0x55b44105e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.193+0000 7ff2e8f55700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b44106a1b0 0x55b4410b9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.193+0000 7ff2e3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b441064310 0x55b44105dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.194+0000 7ff2e37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064cd0 0x55b44105e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53410/0 (socket says 172.21.15.44:53410) 2023-09-10T00:54:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.194+0000 7ff2e8f55700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b44106a1b0 0x55b4410b9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55152/0 (socket says 172.21.15.44:55152) 2023-09-10T00:54:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.194+0000 7ff2e3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b441064310 0x55b44105dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60290/0 (socket says 172.21.15.44:60290) 2023-09-10T00:54:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.194+0000 7ff2e37fe700 1 -- 172.21.15.44:0/2422312217 learned_addr learned my addr 172.21.15.44:0/2422312217 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.194+0000 7ff2e3fff700 1 -- 172.21.15.44:0/2422312217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b44106a1b0 msgr2=0x55b4410b9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.194+0000 7ff2e3fff700 1 --2- 172.21.15.44:0/2422312217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b44106a1b0 0x55b4410b9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.194+0000 7ff2e3fff700 1 -- 172.21.15.44:0/2422312217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064cd0 msgr2=0x55b44105e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.194+0000 7ff2e3fff700 1 --2- 172.21.15.44:0/2422312217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064cd0 0x55b44105e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.194+0000 7ff2e3fff700 1 -- 172.21.15.44:0/2422312217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b440dd1d80 con 0x55b441064310 2023-09-10T00:54:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.194+0000 7ff2e37fe700 1 --2- 172.21.15.44:0/2422312217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064cd0 0x55b44105e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.194+0000 7ff2e3fff700 1 --2- 172.21.15.44:0/2422312217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b441064310 0x55b44105dff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4013000 tx=0x7ff2d40053b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30f2c383e039c5aa server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.194+0000 7ff2c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:54:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.194+0000 7ff2c37fe700 1 -- 172.21.15.44:0/2422312217 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d402c040 con 0x55b441064310 2023-09-10T00:54:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.194+0000 7ff2c37fe700 1 -- 172.21.15.44:0/2422312217 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff2d4031b00 con 0x55b441064310 2023-09-10T00:54:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.194+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2422312217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b440f9e8c0 con 0x55b441064310 2023-09-10T00:54:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.195+0000 7ff2c37fe700 1 -- 172.21.15.44:0/2422312217 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2d4031d20 con 0x55b441064310 2023-09-10T00:54:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.195+0000 7ff2f6b7eb80 10 client.27352 fetch_fsmap learned FSMap version 68 2023-09-10T00:54:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.195+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2422312217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55b440df81f0 con 0x55b441064310 2023-09-10T00:54:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.195+0000 7ff2c37fe700 1 -- 172.21.15.44:0/2422312217 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b440f9e8c0 con 0x55b441064310 2023-09-10T00:54:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.195+0000 7ff2c37fe700 1 -- 172.21.15.44:0/2422312217 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 68) v1 ==== 7922+0+0 (secure 0 0 0) 0x7ff2d403a460 con 0x55b441064310 2023-09-10T00:54:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.195+0000 7ff2f6b7eb80 10 client.27352 fetch_fsmap finished waiting for FSMap version 68 2023-09-10T00:54:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.195+0000 7ff2f6b7eb80 10 client.27352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:54:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.195+0000 7ff2f6b7eb80 20 client.27352 populate_metadata read hostname 'smithi044' 2023-09-10T00:54:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.195+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/2422312217 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b44106d160 0x55b4410c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.195+0000 7ff2f6b7eb80 4 client.27352 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:54:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.195+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2422312217 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b440f5f050 con 0x55b44106d160 2023-09-10T00:54:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.195+0000 7ff2e37fe700 1 --2- 172.21.15.44:0/2422312217 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b44106d160 0x55b4410c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.196+0000 7ff2c37fe700 10 client.27352 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.196+0000 7ff2e37fe700 1 --2- 172.21.15.44:0/2422312217 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b44106d160 0x55b4410c3d60 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5100b228455f92e5 server_cookie=437c656d29dd6454 in_seq=0 out_seq=0 2023-09-10T00:54:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.210+0000 7ff2c37fe700 1 -- 172.21.15.44:0/2422312217 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b440f5f050 con 0x55b44106d160 2023-09-10T00:54:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.210+0000 7ff2c37fe700 10 client.27352 handle_command_reply: tid=0 2023-09-10T00:54:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.210+0000 7ff2f6b7eb80 1 client.27352 shutdown 2023-09-10T00:54:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.211+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2422312217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b441064310 msgr2=0x55b44105dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.211+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/2422312217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b441064310 0x55b44105dff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4013000 tx=0x7ff2d40053b0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.211+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2422312217 shutdown_connections 2023-09-10T00:54:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.211+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/2422312217 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b44106d160 0x55b4410c3d60 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.211+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/2422312217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b44106a1b0 0x55b4410b9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.211+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/2422312217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b441064310 0x55b44105dff0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.211+0000 7ff2f6b7eb80 1 --2- 172.21.15.44:0/2422312217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b441064cd0 0x55b44105e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.211+0000 7ff2c37fe700 0 client.27352 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.211+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2422312217 >> 172.21.15.44:0/2422312217 conn(0x55b440fc8030 msgr2=0x55b441054dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.211+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2422312217 shutdown_connections 2023-09-10T00:54:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.211+0000 7ff2f6b7eb80 1 -- 172.21.15.44:0/2422312217 wait complete. 2023-09-10T00:54:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.211+0000 7ff2f6b7eb80 20 client.27352 trim_cache size 0 max 16384 2023-09-10T00:54:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:54:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:54:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:54:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:54:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.211+0000 7ff2ea9b8700 1 -- 172.21.15.44:0/2343313679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff2cc0a0830 msgr2=0x7ff2cc0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.211+0000 7ff2ea9b8700 1 --2- 172.21.15.44:0/2343313679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff2cc0a0830 0x7ff2cc0a2cf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4011550 tx=0x7ff2d4009260 comp rx=0 tx=0).stop 2023-09-10T00:54:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.211+0000 7ff2ea9b8700 1 -- 172.21.15.44:0/2343313679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2e4100120 msgr2=0x7ff2e4197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.211+0000 7ff2ea9b8700 1 --2- 172.21.15.44:0/2343313679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2e4100120 0x7ff2e4197b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8000c00 tx=0x7ff2d8004730 comp rx=0 tx=0).stop 2023-09-10T00:54:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.212+0000 7ff2ea9b8700 1 -- 172.21.15.44:0/2343313679 shutdown_connections 2023-09-10T00:54:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.212+0000 7ff2ea9b8700 1 --2- 172.21.15.44:0/2343313679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff2cc0a0830 0x7ff2cc0a2cf0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.212+0000 7ff2ea9b8700 1 --2- 172.21.15.44:0/2343313679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2e41055b0 0x7ff2e41a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.212+0000 7ff2ea9b8700 1 --2- 172.21.15.44:0/2343313679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2e40ff760 0x7ff2e41975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.212+0000 7ff2ea9b8700 1 --2- 172.21.15.44:0/2343313679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2e4100120 0x7ff2e4197b20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.212+0000 7ff2ea9b8700 1 -- 172.21.15.44:0/2343313679 >> 172.21.15.44:0/2343313679 conn(0x7ff2e4074f30 msgr2=0x7ff2e40fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.212+0000 7ff2ea9b8700 1 -- 172.21.15.44:0/2343313679 shutdown_connections 2023-09-10T00:54:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:33.212+0000 7ff2ea9b8700 1 -- 172.21.15.44:0/2343313679 wait complete. 2023-09-10T00:54:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:34 smithi131 ceph-mon[103713]: pgmap v436: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.7 MiB/s wr, 356 op/s 2023-09-10T00:54:35.065 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:34 smithi044 ceph-mon[98676]: pgmap v436: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.7 MiB/s wr, 356 op/s 2023-09-10T00:54:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:34 smithi099 ceph-mon[102692]: pgmap v436: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.7 MiB/s wr, 356 op/s 2023-09-10T00:54:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:36 smithi131 ceph-mon[103713]: pgmap v437: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.1 MiB/s wr, 537 op/s 2023-09-10T00:54:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:36 smithi044 ceph-mon[98676]: pgmap v437: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.1 MiB/s wr, 537 op/s 2023-09-10T00:54:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:36 smithi099 ceph-mon[102692]: pgmap v437: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.1 MiB/s wr, 537 op/s 2023-09-10T00:54:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:38 smithi131 ceph-mon[103713]: pgmap v438: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.7 MiB/s wr, 358 op/s 2023-09-10T00:54:39.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:38 smithi044 ceph-mon[98676]: pgmap v438: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.7 MiB/s wr, 358 op/s 2023-09-10T00:54:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:38 smithi099 ceph-mon[102692]: pgmap v438: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.7 MiB/s wr, 358 op/s 2023-09-10T00:54:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:40 smithi131 ceph-mon[103713]: pgmap v439: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.7 MiB/s wr, 358 op/s 2023-09-10T00:54:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:40 smithi044 ceph-mon[98676]: pgmap v439: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.7 MiB/s wr, 358 op/s 2023-09-10T00:54:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:40 smithi099 ceph-mon[102692]: pgmap v439: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.7 MiB/s wr, 358 op/s 2023-09-10T00:54:41.911 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:54:42.278 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.278+0000 7f5d5c18f700 1 -- 172.21.15.44:0/945892270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d540fea00 msgr2=0x7f5d540fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:42.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.278+0000 7f5d5c18f700 1 --2- 172.21.15.44:0/945892270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d540fea00 0x7f5d540fee80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5d500099c0 tx=0x7f5d50009cd0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.278+0000 7f5d5c18f700 1 -- 172.21.15.44:0/945892270 shutdown_connections 2023-09-10T00:54:42.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.278+0000 7f5d5c18f700 1 --2- 172.21.15.44:0/945892270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d540ff3c0 0x7f5d5410d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.278+0000 7f5d5c18f700 1 --2- 172.21.15.44:0/945892270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d54105570 0x7f5d54105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.278+0000 7f5d5c18f700 1 --2- 172.21.15.44:0/945892270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d540fea00 0x7f5d540fee80 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.278+0000 7f5d5c18f700 1 -- 172.21.15.44:0/945892270 >> 172.21.15.44:0/945892270 conn(0x7f5d54076a70 msgr2=0x7f5d540fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:42.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.278+0000 7f5d5c18f700 1 -- 172.21.15.44:0/945892270 shutdown_connections 2023-09-10T00:54:42.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.278+0000 7f5d5c18f700 1 -- 172.21.15.44:0/945892270 wait complete. 2023-09-10T00:54:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.279+0000 7f5d5c18f700 1 Processor -- start 2023-09-10T00:54:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.279+0000 7f5d5c18f700 1 -- start start 2023-09-10T00:54:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.279+0000 7f5d5c18f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d540fea00 0x7f5d541965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.279+0000 7f5d5c18f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d540ff3c0 0x7f5d54196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.279+0000 7f5d5c18f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d54105570 0x7f5d5419af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.279+0000 7f5d5c18f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5419b590 con 0x7f5d540ff3c0 2023-09-10T00:54:42.283 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.279+0000 7f5d5c18f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5419b700 con 0x7f5d54105570 2023-09-10T00:54:42.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.279+0000 7f5d5c18f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5419b8c0 con 0x7f5d540fea00 2023-09-10T00:54:42.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.279+0000 7f5d5972a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d540ff3c0 0x7f5d54196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.279+0000 7f5d59f2b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d540fea00 0x7f5d541965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.279+0000 7f5d5a72c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d54105570 0x7f5d5419af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.279+0000 7f5d5972a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d540ff3c0 0x7f5d54196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53424/0 (socket says 172.21.15.44:53424) 2023-09-10T00:54:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.279+0000 7f5d5972a700 1 -- 172.21.15.44:0/1362993842 learned_addr learned my addr 172.21.15.44:0/1362993842 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.280+0000 7f5d5a72c700 1 -- 172.21.15.44:0/1362993842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d540fea00 msgr2=0x7f5d541965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.280+0000 7f5d5a72c700 1 --2- 172.21.15.44:0/1362993842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d540fea00 0x7f5d541965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.280+0000 7f5d5a72c700 1 -- 172.21.15.44:0/1362993842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d540ff3c0 msgr2=0x7f5d54196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.280+0000 7f5d5a72c700 1 --2- 172.21.15.44:0/1362993842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d540ff3c0 0x7f5d54196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.280+0000 7f5d5a72c700 1 -- 172.21.15.44:0/1362993842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c009c40 con 0x7f5d54105570 2023-09-10T00:54:42.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.280+0000 7f5d59f2b700 1 --2- 172.21.15.44:0/1362993842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d540fea00 0x7f5d541965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:42.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.280+0000 7f5d5972a700 1 --2- 172.21.15.44:0/1362993842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d540ff3c0 0x7f5d54196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:42.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.280+0000 7f5d5a72c700 1 --2- 172.21.15.44:0/1362993842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d54105570 0x7f5d5419af50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00efb0 tx=0x7f5d4c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:42.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.280+0000 7f5d46ffd700 1 -- 172.21.15.44:0/1362993842 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c003a00 con 0x7f5d54105570 2023-09-10T00:54:42.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.280+0000 7f5d46ffd700 1 -- 172.21.15.44:0/1362993842 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d4c00f040 con 0x7f5d54105570 2023-09-10T00:54:42.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.280+0000 7f5d5c18f700 1 -- 172.21.15.44:0/1362993842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d48009520 con 0x7f5d54105570 2023-09-10T00:54:42.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.280+0000 7f5d46ffd700 1 -- 172.21.15.44:0/1362993842 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c0044d0 con 0x7f5d54105570 2023-09-10T00:54:42.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.280+0000 7f5d5c18f700 1 -- 172.21.15.44:0/1362993842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d500095f0 con 0x7f5d54105570 2023-09-10T00:54:42.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.283+0000 7f5d46ffd700 1 -- 172.21.15.44:0/1362993842 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f5d4c00cbc0 con 0x7f5d54105570 2023-09-10T00:54:42.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.283+0000 7f5d46ffd700 1 --2- 172.21.15.44:0/1362993842 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5d400a07b0 0x7f5d400a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.284+0000 7f5d46ffd700 1 -- 172.21.15.44:0/1362993842 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f5d4c0cbfe0 con 0x7f5d54105570 2023-09-10T00:54:42.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.284+0000 7f5d59f2b700 1 --2- 172.21.15.44:0/1362993842 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5d400a07b0 0x7f5d400a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.284+0000 7f5d59f2b700 1 --2- 172.21.15.44:0/1362993842 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5d400a07b0 0x7f5d400a2c70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009fd0 tx=0x7f5d48009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:42.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.302+0000 7f5d68355b80 1 Processor -- start 2023-09-10T00:54:42.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.302+0000 7f5d68355b80 1 -- start start 2023-09-10T00:54:42.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.302+0000 7f5d68355b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b80a710 0x55691b80aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.302+0000 7f5d68355b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80b030 0x55691b814900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.302+0000 7f5d68355b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b814e40 0x55691b817230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.302+0000 7f5d68355b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691b763730 con 0x55691b80b030 2023-09-10T00:54:42.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.302+0000 7f5d68355b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691b761050 con 0x55691b80a710 2023-09-10T00:54:42.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.302+0000 7f5d68355b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691b75c940 con 0x55691b814e40 2023-09-10T00:54:42.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.302+0000 7f5d5972a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80b030 0x55691b814900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.302+0000 7f5d59f2b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b80a710 0x55691b80aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.302+0000 7f5d5972a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80b030 0x55691b814900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53432/0 (socket says 172.21.15.44:53432) 2023-09-10T00:54:42.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.302+0000 7f5d5972a700 1 -- 172.21.15.44:0/98603922 learned_addr learned my addr 172.21.15.44:0/98603922 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:42.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.302+0000 7f5d5a72c700 1 --2- 172.21.15.44:0/98603922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b814e40 0x55691b817230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.303+0000 7f5d59f2b700 1 -- 172.21.15.44:0/98603922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b814e40 msgr2=0x55691b817230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:42.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.303+0000 7f5d59f2b700 1 --2- 172.21.15.44:0/98603922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b814e40 0x55691b817230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.303+0000 7f5d59f2b700 1 -- 172.21.15.44:0/98603922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80b030 msgr2=0x55691b814900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:42.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.303+0000 7f5d59f2b700 1 --2- 172.21.15.44:0/98603922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80b030 0x55691b814900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.303+0000 7f5d59f2b700 1 -- 172.21.15.44:0/98603922 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55691b588de0 con 0x55691b80a710 2023-09-10T00:54:42.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.303+0000 7f5d5972a700 1 --2- 172.21.15.44:0/98603922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80b030 0x55691b814900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:42.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.303+0000 7f5d5a72c700 1 --2- 172.21.15.44:0/98603922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b814e40 0x55691b817230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:42.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.303+0000 7f5d59f2b700 1 --2- 172.21.15.44:0/98603922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b80a710 0x55691b80aaf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5d48011d50 tx=0x7f5d48014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b98a09f9450ac36 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:42.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.303+0000 7f5d3ffff700 1 -- 172.21.15.44:0/98603922 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4801b600 con 0x55691b80a710 2023-09-10T00:54:42.308 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.303+0000 7f5d3ffff700 1 -- 172.21.15.44:0/98603922 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d4800ee40 con 0x55691b80a710 2023-09-10T00:54:42.308 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.303+0000 7f5d3ffff700 1 -- 172.21.15.44:0/98603922 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d480269a0 con 0x55691b80a710 2023-09-10T00:54:42.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.304+0000 7f5d68355b80 1 -- 172.21.15.44:0/98603922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b80a710 msgr2=0x55691b80aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:42.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.304+0000 7f5d68355b80 1 --2- 172.21.15.44:0/98603922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b80a710 0x55691b80aaf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5d48011d50 tx=0x7f5d48014000 comp rx=0 tx=0).stop 2023-09-10T00:54:42.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.304+0000 7f5d68355b80 1 -- 172.21.15.44:0/98603922 shutdown_connections 2023-09-10T00:54:42.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.304+0000 7f5d68355b80 1 --2- 172.21.15.44:0/98603922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b814e40 0x55691b817230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.304+0000 7f5d68355b80 1 --2- 172.21.15.44:0/98603922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b80a710 0x55691b80aaf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.312 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.304+0000 7f5d68355b80 1 --2- 172.21.15.44:0/98603922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80b030 0x55691b814900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.304+0000 7f5d68355b80 1 -- 172.21.15.44:0/98603922 >> 172.21.15.44:0/98603922 conn(0x55691b77f090 msgr2=0x55691b805bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:42.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.304+0000 7f5d68355b80 1 -- 172.21.15.44:0/98603922 shutdown_connections 2023-09-10T00:54:42.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.304+0000 7f5d68355b80 1 -- 172.21.15.44:0/98603922 wait complete. 2023-09-10T00:54:42.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.305+0000 7f5d68355b80 1 -- start start 2023-09-10T00:54:42.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.305+0000 7f5d68355b80 1 Processor -- start 2023-09-10T00:54:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.305+0000 7f5d68355b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b80a710 0x55691b814810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.305+0000 7f5d68355b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b80b030 0x55691b811a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.305+0000 7f5d68355b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b814e40 0x55691b812030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.305+0000 7f5d68355b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691b6fab70 con 0x55691b814e40 2023-09-10T00:54:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.305+0000 7f5d68355b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691b6fa870 con 0x55691b80a710 2023-09-10T00:54:42.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.305+0000 7f5d68355b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691b5de920 con 0x55691b80b030 2023-09-10T00:54:42.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.305+0000 7f5d68355b80 10 client.0 _ll_register_callbacks cb 0x55691b644a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:54:42.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.306+0000 7f5d5972a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b80b030 0x55691b811a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.306+0000 7f5d5a72c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b814e40 0x55691b812030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.306+0000 7f5d5972a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b80b030 0x55691b811a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60320/0 (socket says 172.21.15.44:60320) 2023-09-10T00:54:42.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.306+0000 7f5d5972a700 1 -- 172.21.15.44:0/970782925 learned_addr learned my addr 172.21.15.44:0/970782925 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:42.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.306+0000 7f5d5a72c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b814e40 0x55691b812030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53440/0 (socket says 172.21.15.44:53440) 2023-09-10T00:54:42.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.306+0000 7f5d59f2b700 1 --2- 172.21.15.44:0/970782925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b80a710 0x55691b814810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.306+0000 7f5d5972a700 1 -- 172.21.15.44:0/970782925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b80a710 msgr2=0x55691b814810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:42.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.306+0000 7f5d5972a700 1 --2- 172.21.15.44:0/970782925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b80a710 0x55691b814810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.306+0000 7f5d5972a700 1 -- 172.21.15.44:0/970782925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b814e40 msgr2=0x55691b812030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:42.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.306+0000 7f5d5972a700 1 --2- 172.21.15.44:0/970782925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b814e40 0x55691b812030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.306+0000 7f5d5972a700 1 -- 172.21.15.44:0/970782925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5410dd00 con 0x55691b80b030 2023-09-10T00:54:42.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.306+0000 7f5d5972a700 1 --2- 172.21.15.44:0/970782925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b80b030 0x55691b811a20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5d50005270 tx=0x7f5d500052a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c96bd3c9e68ef57d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:42.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.306+0000 7f5d3d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:54:42.317 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.306+0000 7f5d3d7fa700 1 -- 172.21.15.44:0/970782925 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50003960 con 0x55691b80b030 2023-09-10T00:54:42.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.306+0000 7f5d3d7fa700 1 -- 172.21.15.44:0/970782925 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f5d50003ac0 con 0x55691b80b030 2023-09-10T00:54:42.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.306+0000 7f5d68355b80 1 -- 172.21.15.44:0/970782925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55691b6b44f0 con 0x55691b80b030 2023-09-10T00:54:42.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.307+0000 7f5d3d7fa700 1 -- 172.21.15.44:0/970782925 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d50005710 con 0x55691b80b030 2023-09-10T00:54:42.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.307+0000 7f5d68355b80 10 client.27364 fetch_fsmap learned FSMap version 68 2023-09-10T00:54:42.318 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.307+0000 7f5d68355b80 1 -- 172.21.15.44:0/970782925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7f5d500095f0 con 0x55691b80b030 2023-09-10T00:54:42.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.307+0000 7f5d3d7fa700 1 -- 172.21.15.44:0/970782925 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55691b6b44f0 con 0x55691b80b030 2023-09-10T00:54:42.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.307+0000 7f5d3d7fa700 1 -- 172.21.15.44:0/970782925 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 68) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f5d500173f0 con 0x55691b80b030 2023-09-10T00:54:42.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.307+0000 7f5d68355b80 10 client.27364 fetch_fsmap finished waiting for FSMap version 68 2023-09-10T00:54:42.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.307+0000 7f5d68355b80 10 client.27364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:54:42.319 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.307+0000 7f5d68355b80 20 client.27364 populate_metadata read hostname 'smithi044' 2023-09-10T00:54:42.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.307+0000 7f5d68355b80 1 --2- 172.21.15.44:0/970782925 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55691b8249c0 0x55691b826de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.307+0000 7f5d68355b80 4 client.27364 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:54:42.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.307+0000 7f5d68355b80 1 -- 172.21.15.44:0/970782925 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55691b7718a0 con 0x55691b8249c0 2023-09-10T00:54:42.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.307+0000 7f5d59f2b700 1 --2- 172.21.15.44:0/970782925 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55691b8249c0 0x55691b826de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.320 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.308+0000 7f5d3d7fa700 10 client.27364 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:42.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.308+0000 7f5d59f2b700 1 --2- 172.21.15.44:0/970782925 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55691b8249c0 0x55691b826de0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fd407d6b34eb20b server_cookie=bbe1279b065c91b6 in_seq=0 out_seq=0 2023-09-10T00:54:42.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.308+0000 7f5d3d7fa700 1 -- 172.21.15.44:0/970782925 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55691b7718a0 con 0x55691b8249c0 2023-09-10T00:54:42.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.308+0000 7f5d3d7fa700 10 client.27364 handle_command_reply: tid=0 2023-09-10T00:54:42.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.309+0000 7f5d68355b80 1 client.27364 shutdown 2023-09-10T00:54:42.321 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.309+0000 7f5d68355b80 1 -- 172.21.15.44:0/970782925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b80b030 msgr2=0x55691b811a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:42.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.309+0000 7f5d68355b80 1 --2- 172.21.15.44:0/970782925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b80b030 0x55691b811a20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5d50005270 tx=0x7f5d500052a0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.309+0000 7f5d68355b80 1 -- 172.21.15.44:0/970782925 shutdown_connections 2023-09-10T00:54:42.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.309+0000 7f5d68355b80 1 --2- 172.21.15.44:0/970782925 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55691b8249c0 0x55691b826de0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.309+0000 7f5d68355b80 1 --2- 172.21.15.44:0/970782925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b814e40 0x55691b812030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.322 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.309+0000 7f5d3d7fa700 0 client.27364 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:42.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.309+0000 7f5d68355b80 1 --2- 172.21.15.44:0/970782925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b80a710 0x55691b814810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.309+0000 7f5d68355b80 1 --2- 172.21.15.44:0/970782925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b80b030 0x55691b811a20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.309+0000 7f5d68355b80 1 -- 172.21.15.44:0/970782925 >> 172.21.15.44:0/970782925 conn(0x55691b77f090 msgr2=0x55691b805780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:42.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.309+0000 7f5d68355b80 1 -- 172.21.15.44:0/970782925 shutdown_connections 2023-09-10T00:54:42.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.309+0000 7f5d68355b80 1 -- 172.21.15.44:0/970782925 wait complete. 2023-09-10T00:54:42.323 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.309+0000 7f5d68355b80 20 client.27364 trim_cache size 0 max 16384 2023-09-10T00:54:42.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.334+0000 7f5d68355b80 1 Processor -- start 2023-09-10T00:54:42.335 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.334+0000 7f5d68355b80 1 -- start start 2023-09-10T00:54:42.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.334+0000 7f5d68355b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80a710 0x55691b8111e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.334+0000 7f5d68355b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b80b030 0x55691b814340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.334+0000 7f5d68355b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b814e40 0x55691b80ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.334+0000 7f5d68355b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691b817750 con 0x55691b80a710 2023-09-10T00:54:42.336 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.334+0000 7f5d68355b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691b807240 con 0x55691b814e40 2023-09-10T00:54:42.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.334+0000 7f5d68355b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691b80b6d0 con 0x55691b80b030 2023-09-10T00:54:42.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.334+0000 7f5d59f2b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80a710 0x55691b8111e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.334+0000 7f5d5972a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b80b030 0x55691b814340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.334+0000 7f5d59f2b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80a710 0x55691b8111e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53444/0 (socket says 172.21.15.44:53444) 2023-09-10T00:54:42.337 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.334+0000 7f5d59f2b700 1 -- 172.21.15.44:0/3348454887 learned_addr learned my addr 172.21.15.44:0/3348454887 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:42.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.334+0000 7f5d5a72c700 1 --2- 172.21.15.44:0/3348454887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b814e40 0x55691b80ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.335+0000 7f5d5972a700 1 -- 172.21.15.44:0/3348454887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b814e40 msgr2=0x55691b80ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:42.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.335+0000 7f5d5972a700 1 --2- 172.21.15.44:0/3348454887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b814e40 0x55691b80ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.335+0000 7f5d5972a700 1 -- 172.21.15.44:0/3348454887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80a710 msgr2=0x55691b8111e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:42.338 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.335+0000 7f5d5972a700 1 --2- 172.21.15.44:0/3348454887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80a710 0x55691b8111e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.335+0000 7f5d5972a700 1 -- 172.21.15.44:0/3348454887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d48017050 con 0x55691b80b030 2023-09-10T00:54:42.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.335+0000 7f5d5972a700 1 --2- 172.21.15.44:0/3348454887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b80b030 0x55691b814340 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009990 tx=0x7f5d5000b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75901f5e3bb82da7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:42.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.335+0000 7f5d44ff9700 1 -- 172.21.15.44:0/3348454887 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50005710 con 0x55691b80b030 2023-09-10T00:54:42.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.335+0000 7f5d44ff9700 1 -- 172.21.15.44:0/3348454887 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d5001e040 con 0x55691b80b030 2023-09-10T00:54:42.339 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.335+0000 7f5d44ff9700 1 -- 172.21.15.44:0/3348454887 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d500246e0 con 0x55691b80b030 2023-09-10T00:54:42.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.336+0000 7f5d68355b80 1 -- 172.21.15.44:0/3348454887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b80b030 msgr2=0x55691b814340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:42.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.336+0000 7f5d68355b80 1 --2- 172.21.15.44:0/3348454887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b80b030 0x55691b814340 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009990 tx=0x7f5d5000b540 comp rx=0 tx=0).stop 2023-09-10T00:54:42.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.336+0000 7f5d68355b80 1 -- 172.21.15.44:0/3348454887 shutdown_connections 2023-09-10T00:54:42.343 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.336+0000 7f5d68355b80 1 --2- 172.21.15.44:0/3348454887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b814e40 0x55691b80ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.336+0000 7f5d68355b80 1 --2- 172.21.15.44:0/3348454887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80a710 0x55691b8111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.336+0000 7f5d68355b80 1 --2- 172.21.15.44:0/3348454887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b80b030 0x55691b814340 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.336+0000 7f5d68355b80 1 -- 172.21.15.44:0/3348454887 >> 172.21.15.44:0/3348454887 conn(0x55691b77f090 msgr2=0x55691b80cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.336+0000 7f5d68355b80 1 -- 172.21.15.44:0/3348454887 shutdown_connections 2023-09-10T00:54:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.336+0000 7f5d68355b80 1 -- 172.21.15.44:0/3348454887 wait complete. 2023-09-10T00:54:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.336+0000 7f5d68355b80 1 -- start start 2023-09-10T00:54:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.336+0000 7f5d68355b80 1 Processor -- start 2023-09-10T00:54:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.337+0000 7f5d68355b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80a710 0x55691b81b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.337+0000 7f5d68355b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b80b030 0x55691b81bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.337+0000 7f5d68355b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b814e40 0x55691b819b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.337+0000 7f5d68355b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691b81a220 con 0x55691b80a710 2023-09-10T00:54:42.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.337+0000 7f5d68355b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691b81a390 con 0x55691b80b030 2023-09-10T00:54:42.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.337+0000 7f5d68355b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691b81a550 con 0x55691b814e40 2023-09-10T00:54:42.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.337+0000 7f5d68355b80 10 client.0 _ll_register_callbacks cb 0x7f5d5000be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:54:42.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.337+0000 7f5d59f2b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80a710 0x55691b81b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.337+0000 7f5d59f2b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80a710 0x55691b81b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53450/0 (socket says 172.21.15.44:53450) 2023-09-10T00:54:42.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.337+0000 7f5d59f2b700 1 -- 172.21.15.44:0/1508233604 learned_addr learned my addr 172.21.15.44:0/1508233604 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:42.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.337+0000 7f5d5972a700 1 --2- 172.21.15.44:0/1508233604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b80b030 0x55691b81bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.338+0000 7f5d5a72c700 1 --2- 172.21.15.44:0/1508233604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b814e40 0x55691b819b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.347 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.338+0000 7f5d59f2b700 1 -- 172.21.15.44:0/1508233604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b814e40 msgr2=0x55691b819b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:42.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.338+0000 7f5d59f2b700 1 --2- 172.21.15.44:0/1508233604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b814e40 0x55691b819b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.338+0000 7f5d59f2b700 1 -- 172.21.15.44:0/1508233604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b80b030 msgr2=0x55691b81bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:42.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.338+0000 7f5d59f2b700 1 --2- 172.21.15.44:0/1508233604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b80b030 0x55691b81bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.338+0000 7f5d59f2b700 1 -- 172.21.15.44:0/1508233604 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d500095f0 con 0x55691b80a710 2023-09-10T00:54:42.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.338+0000 7f5d59f2b700 1 --2- 172.21.15.44:0/1508233604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80a710 0x55691b81b680 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800e5e0 tx=0x7f5d48011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f3968354714141 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:42.348 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.338+0000 7f5d3d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:54:42.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.338+0000 7f5d3d7fa700 1 -- 172.21.15.44:0/1508233604 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d480055b0 con 0x55691b80a710 2023-09-10T00:54:42.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.338+0000 7f5d3d7fa700 1 -- 172.21.15.44:0/1508233604 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f5d48038070 con 0x55691b80a710 2023-09-10T00:54:42.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.338+0000 7f5d68355b80 1 -- 172.21.15.44:0/1508233604 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d4800ee40 con 0x55691b80a710 2023-09-10T00:54:42.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.338+0000 7f5d3d7fa700 1 -- 172.21.15.44:0/1508233604 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d480046d0 con 0x55691b80a710 2023-09-10T00:54:42.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.338+0000 7f5d68355b80 10 client.17772 fetch_fsmap learned FSMap version 68 2023-09-10T00:54:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.338+0000 7f5d68355b80 1 -- 172.21.15.44:0/1508233604 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55691b571c60 con 0x55691b80a710 2023-09-10T00:54:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.339+0000 7f5d3d7fa700 1 -- 172.21.15.44:0/1508233604 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d4800ee40 con 0x55691b80a710 2023-09-10T00:54:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.339+0000 7f5d3d7fa700 1 -- 172.21.15.44:0/1508233604 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 68) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f5d4803a070 con 0x55691b80a710 2023-09-10T00:54:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.339+0000 7f5d68355b80 10 client.17772 fetch_fsmap finished waiting for FSMap version 68 2023-09-10T00:54:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.339+0000 7f5d68355b80 10 client.17772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:54:42.351 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.339+0000 7f5d68355b80 20 client.17772 populate_metadata read hostname 'smithi044' 2023-09-10T00:54:42.351 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.339+0000 7f5d68355b80 1 --2- 172.21.15.44:0/1508233604 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55691b8249c0 0x55691b87aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:42.351 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.339+0000 7f5d68355b80 4 client.17772 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:54:42.351 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.339+0000 7f5d68355b80 1 -- 172.21.15.44:0/1508233604 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55691b755920 con 0x55691b8249c0 2023-09-10T00:54:42.351 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.339+0000 7f5d5972a700 1 --2- 172.21.15.44:0/1508233604 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55691b8249c0 0x55691b87aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:42.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.340+0000 7f5d5972a700 1 --2- 172.21.15.44:0/1508233604 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55691b8249c0 0x55691b87aec0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dadbd581c1d7597 server_cookie=918e11a5c61ec25 in_seq=0 out_seq=0 2023-09-10T00:54:42.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.340+0000 7f5d3d7fa700 10 client.17772 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:42.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.340+0000 7f5d3d7fa700 1 -- 172.21.15.44:0/1508233604 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55691b755920 con 0x55691b8249c0 2023-09-10T00:54:42.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.340+0000 7f5d3d7fa700 10 client.17772 handle_command_reply: tid=0 2023-09-10T00:54:42.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.340+0000 7f5d68355b80 1 client.17772 shutdown 2023-09-10T00:54:42.352 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.340+0000 7f5d68355b80 1 -- 172.21.15.44:0/1508233604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80a710 msgr2=0x55691b81b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:42.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.340+0000 7f5d68355b80 1 --2- 172.21.15.44:0/1508233604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80a710 0x55691b81b680 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800e5e0 tx=0x7f5d48011dd0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d68355b80 1 -- 172.21.15.44:0/1508233604 shutdown_connections 2023-09-10T00:54:42.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d68355b80 1 --2- 172.21.15.44:0/1508233604 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55691b8249c0 0x55691b87aec0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d68355b80 1 --2- 172.21.15.44:0/1508233604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55691b814e40 0x55691b819b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.353 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d68355b80 1 --2- 172.21.15.44:0/1508233604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55691b80a710 0x55691b81b680 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d3d7fa700 0 client.17772 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:42.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d68355b80 1 --2- 172.21.15.44:0/1508233604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55691b80b030 0x55691b81bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d68355b80 1 -- 172.21.15.44:0/1508233604 >> 172.21.15.44:0/1508233604 conn(0x55691b77f090 msgr2=0x55691b809700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:42.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d68355b80 1 -- 172.21.15.44:0/1508233604 shutdown_connections 2023-09-10T00:54:42.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d68355b80 1 -- 172.21.15.44:0/1508233604 wait complete. 2023-09-10T00:54:42.354 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d68355b80 20 client.17772 trim_cache size 0 max 16384 2023-09-10T00:54:42.355 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:54:42.355 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:54:42.355 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:54:42.355 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:54:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:54:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:54:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:54:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:54:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:54:42.357 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:54:42.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d5c18f700 1 -- 172.21.15.44:0/1362993842 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5d400a07b0 msgr2=0x7f5d400a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:42.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d5c18f700 1 --2- 172.21.15.44:0/1362993842 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5d400a07b0 0x7f5d400a2c70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009fd0 tx=0x7f5d48009260 comp rx=0 tx=0).stop 2023-09-10T00:54:42.358 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d5c18f700 1 -- 172.21.15.44:0/1362993842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d54105570 msgr2=0x7f5d5419af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:42.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d5c18f700 1 --2- 172.21.15.44:0/1362993842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d54105570 0x7f5d5419af50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00efb0 tx=0x7f5d4c00c3a0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d5c18f700 1 -- 172.21.15.44:0/1362993842 shutdown_connections 2023-09-10T00:54:42.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d5c18f700 1 --2- 172.21.15.44:0/1362993842 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5d400a07b0 0x7f5d400a2c70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.341+0000 7f5d5c18f700 1 --2- 172.21.15.44:0/1362993842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d54105570 0x7f5d5419af50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.359 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.342+0000 7f5d5c18f700 1 --2- 172.21.15.44:0/1362993842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d540fea00 0x7f5d541965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.342+0000 7f5d5c18f700 1 --2- 172.21.15.44:0/1362993842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d540ff3c0 0x7f5d54196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:42.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.342+0000 7f5d5c18f700 1 -- 172.21.15.44:0/1362993842 >> 172.21.15.44:0/1362993842 conn(0x7f5d54076a70 msgr2=0x7f5d54104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:42.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.342+0000 7f5d5c18f700 1 -- 172.21.15.44:0/1362993842 shutdown_connections 2023-09-10T00:54:42.360 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:54:42.342+0000 7f5d5c18f700 1 -- 172.21.15.44:0/1362993842 wait complete. 2023-09-10T00:54:42.361 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:54:42.706 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:42 smithi044 ceph-mon[98676]: pgmap v440: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 534 op/s 2023-09-10T00:54:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:42 smithi131 ceph-mon[103713]: pgmap v440: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 534 op/s 2023-09-10T00:54:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:42 smithi099 ceph-mon[102692]: pgmap v440: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 534 op/s 2023-09-10T00:54:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:44 smithi131 ceph-mon[103713]: pgmap v441: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.7 MiB/s wr, 356 op/s 2023-09-10T00:54:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:44 smithi044 ceph-mon[98676]: pgmap v441: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.7 MiB/s wr, 356 op/s 2023-09-10T00:54:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:44 smithi099 ceph-mon[102692]: pgmap v441: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.7 MiB/s wr, 356 op/s 2023-09-10T00:54:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.429+0000 7f0842369700 1 -- 172.21.15.44:0/866775239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c1027c0 msgr2=0x7f083c102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.429+0000 7f0842369700 1 --2- 172.21.15.44:0/866775239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c1027c0 0x7f083c102c40 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f082c009990 tx=0x7f082c009ca0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.430+0000 7f0842369700 1 -- 172.21.15.44:0/866775239 shutdown_connections 2023-09-10T00:54:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.430+0000 7f0842369700 1 --2- 172.21.15.44:0/866775239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f083c103180 0x7f083c10f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.430+0000 7f0842369700 1 --2- 172.21.15.44:0/866775239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f083c1087d0 0x7f083c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.430+0000 7f0842369700 1 --2- 172.21.15.44:0/866775239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c1027c0 0x7f083c102c40 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.430+0000 7f0842369700 1 -- 172.21.15.44:0/866775239 >> 172.21.15.44:0/866775239 conn(0x7f083c0fe290 msgr2=0x7f083c1006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.430+0000 7f0842369700 1 -- 172.21.15.44:0/866775239 shutdown_connections 2023-09-10T00:54:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.430+0000 7f0842369700 1 -- 172.21.15.44:0/866775239 wait complete. 2023-09-10T00:54:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.430+0000 7f0842369700 1 Processor -- start 2023-09-10T00:54:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.430+0000 7f0842369700 1 -- start start 2023-09-10T00:54:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f0842369700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f083c1027c0 0x7f083c10ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f0842369700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f083c103180 0x7f083c10b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f0842369700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c1087d0 0x7f083c1bdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f0842369700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083c1be450 con 0x7f083c1087d0 2023-09-10T00:54:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f0842369700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083c1be5c0 con 0x7f083c103180 2023-09-10T00:54:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f0842369700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083c1be780 con 0x7f083c1027c0 2023-09-10T00:54:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f083bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f083c1027c0 0x7f083c10ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f083b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f083c103180 0x7f083c10b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f083bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f083c1027c0 0x7f083c10ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60346/0 (socket says 172.21.15.44:60346) 2023-09-10T00:54:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f083bfff700 1 -- 172.21.15.44:0/433018609 learned_addr learned my addr 172.21.15.44:0/433018609 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f0840906700 1 --2- 172.21.15.44:0/433018609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c1087d0 0x7f083c1bdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f083bfff700 1 -- 172.21.15.44:0/433018609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f083c103180 msgr2=0x7f083c10b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f083bfff700 1 --2- 172.21.15.44:0/433018609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f083c103180 0x7f083c10b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f083bfff700 1 -- 172.21.15.44:0/433018609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c1087d0 msgr2=0x7f083c1bdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f083bfff700 1 --2- 172.21.15.44:0/433018609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c1087d0 0x7f083c1bdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f083bfff700 1 -- 172.21.15.44:0/433018609 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c0095f0 con 0x7f083c1027c0 2023-09-10T00:54:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f0840906700 1 --2- 172.21.15.44:0/433018609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c1087d0 0x7f083c1bdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:54:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.431+0000 7f083b7fe700 1 --2- 172.21.15.44:0/433018609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f083c103180 0x7f083c10b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.432+0000 7f083bfff700 1 --2- 172.21.15.44:0/433018609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f083c1027c0 0x7f083c10ace0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f083000e9a0 tx=0x7f083000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.432+0000 7f08397fa700 1 -- 172.21.15.44:0/433018609 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830009790 con 0x7f083c1027c0 2023-09-10T00:54:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.432+0000 7f0842369700 1 -- 172.21.15.44:0/433018609 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f083c1bea60 con 0x7f083c1027c0 2023-09-10T00:54:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.432+0000 7f0842369700 1 -- 172.21.15.44:0/433018609 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083c1bef90 con 0x7f083c1027c0 2023-09-10T00:54:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.432+0000 7f08397fa700 1 -- 172.21.15.44:0/433018609 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f083000cbd0 con 0x7f083c1027c0 2023-09-10T00:54:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.432+0000 7f08397fa700 1 -- 172.21.15.44:0/433018609 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830018680 con 0x7f083c1027c0 2023-09-10T00:54:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.435+0000 7f08397fa700 1 -- 172.21.15.44:0/433018609 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f08300187e0 con 0x7f083c1027c0 2023-09-10T00:54:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.436+0000 7f08397fa700 1 --2- 172.21.15.44:0/433018609 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f08240a0820 0x7f08240a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.436+0000 7f08397fa700 1 -- 172.21.15.44:0/433018609 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f0830014070 con 0x7f083c1027c0 2023-09-10T00:54:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.436+0000 7f083b7fe700 1 --2- 172.21.15.44:0/433018609 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f08240a0820 0x7f08240a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.436+0000 7f084e52fb80 1 Processor -- start 2023-09-10T00:54:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.436+0000 7f083b7fe700 1 --2- 172.21.15.44:0/433018609 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f08240a0820 0x7f08240a2ce0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f083c107ea0 tx=0x7f082c004300 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.436+0000 7f084e52fb80 1 -- start start 2023-09-10T00:54:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.437+0000 7f084e52fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d5b9b0 0x55be70d5bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.437+0000 7f084e52fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d55b10 0x55be70d55f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.437+0000 7f084e52fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d564d0 0x55be70d62740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.437+0000 7f084e52fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be70ca66d0 con 0x55be70d564d0 2023-09-10T00:54:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.437+0000 7f084e52fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be70ca3ff0 con 0x55be70d5b9b0 2023-09-10T00:54:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.437+0000 7f084e52fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be70c9f8e0 con 0x55be70d55b10 2023-09-10T00:54:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.437+0000 7f0840906700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d564d0 0x55be70d62740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.437+0000 7f0840906700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d564d0 0x55be70d62740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53472/0 (socket says 172.21.15.44:53472) 2023-09-10T00:54:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.437+0000 7f0840906700 1 -- 172.21.15.44:0/930361896 learned_addr learned my addr 172.21.15.44:0/930361896 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.437+0000 7f083bfff700 1 --2- 172.21.15.44:0/930361896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d5b9b0 0x55be70d5bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.437+0000 7f083b7fe700 1 --2- 172.21.15.44:0/930361896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d55b10 0x55be70d55f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.438+0000 7f083bfff700 1 -- 172.21.15.44:0/930361896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d55b10 msgr2=0x55be70d55f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.438+0000 7f083bfff700 1 --2- 172.21.15.44:0/930361896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d55b10 0x55be70d55f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.438+0000 7f083bfff700 1 -- 172.21.15.44:0/930361896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d564d0 msgr2=0x55be70d62740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.438+0000 7f083bfff700 1 --2- 172.21.15.44:0/930361896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d564d0 0x55be70d62740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.438+0000 7f083bfff700 1 -- 172.21.15.44:0/930361896 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be70acbd80 con 0x55be70d5b9b0 2023-09-10T00:54:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.438+0000 7f083bfff700 1 --2- 172.21.15.44:0/930361896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d5b9b0 0x55be70d5bd90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f08300c97d0 tx=0x7f0830099df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba8634d697d817ab server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.438+0000 7f08227fc700 1 -- 172.21.15.44:0/930361896 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083009a490 con 0x55be70d5b9b0 2023-09-10T00:54:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.438+0000 7f08227fc700 1 -- 172.21.15.44:0/930361896 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f083009e2c0 con 0x55be70d5b9b0 2023-09-10T00:54:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.438+0000 7f08227fc700 1 -- 172.21.15.44:0/930361896 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830098930 con 0x55be70d5b9b0 2023-09-10T00:54:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.439+0000 7f084e52fb80 1 -- 172.21.15.44:0/930361896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d5b9b0 msgr2=0x55be70d5bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.439+0000 7f084e52fb80 1 --2- 172.21.15.44:0/930361896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d5b9b0 0x55be70d5bd90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f08300c97d0 tx=0x7f0830099df0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.439+0000 7f084e52fb80 1 -- 172.21.15.44:0/930361896 shutdown_connections 2023-09-10T00:54:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.439+0000 7f084e52fb80 1 --2- 172.21.15.44:0/930361896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d564d0 0x55be70d62740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.439+0000 7f084e52fb80 1 --2- 172.21.15.44:0/930361896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d5b9b0 0x55be70d5bd90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.439+0000 7f084e52fb80 1 --2- 172.21.15.44:0/930361896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d55b10 0x55be70d55f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.439+0000 7f084e52fb80 1 -- 172.21.15.44:0/930361896 >> 172.21.15.44:0/930361896 conn(0x55be70cc2030 msgr2=0x55be70d48560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.439+0000 7f084e52fb80 1 -- 172.21.15.44:0/930361896 shutdown_connections 2023-09-10T00:54:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.439+0000 7f084e52fb80 1 -- 172.21.15.44:0/930361896 wait complete. 2023-09-10T00:54:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.439+0000 7f084e52fb80 1 -- start start 2023-09-10T00:54:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.440+0000 7f084e52fb80 1 Processor -- start 2023-09-10T00:54:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.440+0000 7f084e52fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d62600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.440+0000 7f084e52fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d564d0 0x55be70d5f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.440+0000 7f084e52fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d5b9b0 0x55be70d5ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.440+0000 7f084e52fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be70c3db10 con 0x55be70d564d0 2023-09-10T00:54:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.440+0000 7f084e52fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be70c3d810 con 0x55be70d55b10 2023-09-10T00:54:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.440+0000 7f084e52fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be70b218c0 con 0x55be70d5b9b0 2023-09-10T00:54:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.440+0000 7f084e52fb80 10 client.0 _ll_register_callbacks cb 0x55be70b87a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:54:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.440+0000 7f083bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d62600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.440+0000 7f083bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d62600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55218/0 (socket says 172.21.15.44:55218) 2023-09-10T00:54:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.440+0000 7f083bfff700 1 -- 172.21.15.44:0/2276189681 learned_addr learned my addr 172.21.15.44:0/2276189681 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.440+0000 7f0840906700 1 --2- 172.21.15.44:0/2276189681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d5b9b0 0x55be70d5ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.440+0000 7f083b7fe700 1 --2- 172.21.15.44:0/2276189681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d564d0 0x55be70d5f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.441+0000 7f083bfff700 1 -- 172.21.15.44:0/2276189681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d5b9b0 msgr2=0x55be70d5ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.441+0000 7f083bfff700 1 --2- 172.21.15.44:0/2276189681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d5b9b0 0x55be70d5ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.441+0000 7f083bfff700 1 -- 172.21.15.44:0/2276189681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d564d0 msgr2=0x55be70d5f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.441+0000 7f083bfff700 1 --2- 172.21.15.44:0/2276189681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d564d0 0x55be70d5f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.441+0000 7f083bfff700 1 -- 172.21.15.44:0/2276189681 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083c1bef90 con 0x55be70d55b10 2023-09-10T00:54:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.441+0000 7f0840906700 1 --2- 172.21.15.44:0/2276189681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d5b9b0 0x55be70d5ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.441+0000 7f083b7fe700 1 --2- 172.21.15.44:0/2276189681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d564d0 0x55be70d5f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:54:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.441+0000 7f083bfff700 1 --2- 172.21.15.44:0/2276189681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d62600 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f08300a1fd0 tx=0x7f083009ad00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cad51935ad830a82 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.441+0000 7f081b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:54:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.441+0000 7f081b7fe700 1 -- 172.21.15.44:0/2276189681 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083009a8f0 con 0x55be70d55b10 2023-09-10T00:54:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.441+0000 7f084e52fb80 1 -- 172.21.15.44:0/2276189681 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be70bf7490 con 0x55be70d55b10 2023-09-10T00:54:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.441+0000 7f081b7fe700 1 -- 172.21.15.44:0/2276189681 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f083009a490 con 0x55be70d55b10 2023-09-10T00:54:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.441+0000 7f081b7fe700 1 -- 172.21.15.44:0/2276189681 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08300a2500 con 0x55be70d55b10 2023-09-10T00:54:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.441+0000 7f084e52fb80 10 client.27449 fetch_fsmap learned FSMap version 68 2023-09-10T00:54:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.441+0000 7f084e52fb80 1 -- 172.21.15.44:0/2276189681 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55be70ab4c00 con 0x55be70d55b10 2023-09-10T00:54:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.442+0000 7f081b7fe700 1 -- 172.21.15.44:0/2276189681 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55be70bf7490 con 0x55be70d55b10 2023-09-10T00:54:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.442+0000 7f081b7fe700 1 -- 172.21.15.44:0/2276189681 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 68) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f0830096280 con 0x55be70d55b10 2023-09-10T00:54:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.442+0000 7f084e52fb80 10 client.27449 fetch_fsmap finished waiting for FSMap version 68 2023-09-10T00:54:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.442+0000 7f084e52fb80 10 client.27449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:54:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.442+0000 7f084e52fb80 20 client.27449 populate_metadata read hostname 'smithi044' 2023-09-10T00:54:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.442+0000 7f084e52fb80 1 --2- 172.21.15.44:0/2276189681 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55be70d50ef0 0x55be70d53310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.442+0000 7f084e52fb80 4 client.27449 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:54:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.442+0000 7f084e52fb80 1 -- 172.21.15.44:0/2276189681 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be70cb4840 con 0x55be70d50ef0 2023-09-10T00:54:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.442+0000 7f083b7fe700 1 --2- 172.21.15.44:0/2276189681 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55be70d50ef0 0x55be70d53310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.444+0000 7f081b7fe700 10 client.27449 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.444+0000 7f083b7fe700 1 --2- 172.21.15.44:0/2276189681 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55be70d50ef0 0x55be70d53310 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d42bef595031667f server_cookie=7e81aa8e6bc402b3 in_seq=0 out_seq=0 2023-09-10T00:54:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.444+0000 7f081b7fe700 1 -- 172.21.15.44:0/2276189681 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55be70cb4840 con 0x55be70d50ef0 2023-09-10T00:54:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.444+0000 7f081b7fe700 10 client.27449 handle_command_reply: tid=0 2023-09-10T00:54:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.444+0000 7f084e52fb80 1 client.27449 shutdown 2023-09-10T00:54:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.445+0000 7f084e52fb80 1 -- 172.21.15.44:0/2276189681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 msgr2=0x55be70d62600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.445+0000 7f084e52fb80 1 --2- 172.21.15.44:0/2276189681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d62600 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f08300a1fd0 tx=0x7f083009ad00 comp rx=0 tx=0).stop 2023-09-10T00:54:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.445+0000 7f084e52fb80 1 -- 172.21.15.44:0/2276189681 shutdown_connections 2023-09-10T00:54:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.445+0000 7f084e52fb80 1 --2- 172.21.15.44:0/2276189681 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55be70d50ef0 0x55be70d53310 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.445+0000 7f084e52fb80 1 --2- 172.21.15.44:0/2276189681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d5b9b0 0x55be70d5ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.445+0000 7f084e52fb80 1 --2- 172.21.15.44:0/2276189681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d62600 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.445+0000 7f081b7fe700 0 client.27449 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.445+0000 7f084e52fb80 1 --2- 172.21.15.44:0/2276189681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d564d0 0x55be70d5f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.445+0000 7f084e52fb80 1 -- 172.21.15.44:0/2276189681 >> 172.21.15.44:0/2276189681 conn(0x55be70cc2030 msgr2=0x55be70d477a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.445+0000 7f084e52fb80 1 -- 172.21.15.44:0/2276189681 shutdown_connections 2023-09-10T00:54:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.445+0000 7f084e52fb80 1 -- 172.21.15.44:0/2276189681 wait complete. 2023-09-10T00:54:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.445+0000 7f084e52fb80 20 client.27449 trim_cache size 0 max 16384 2023-09-10T00:54:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.468+0000 7f084e52fb80 1 Processor -- start 2023-09-10T00:54:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f084e52fb80 1 -- start start 2023-09-10T00:54:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f084e52fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d50ef0 0x55be70d4ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f084e52fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d47f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f084e52fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d564d0 0x55be70d48460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f084e52fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be70bf7490 con 0x55be70d50ef0 2023-09-10T00:54:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f084e52fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be70d49140 con 0x55be70d55b10 2023-09-10T00:54:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f084e52fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be70d60520 con 0x55be70d564d0 2023-09-10T00:54:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f083b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d47f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f083b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d47f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55226/0 (socket says 172.21.15.44:55226) 2023-09-10T00:54:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f083b7fe700 1 -- 172.21.15.44:0/4243112653 learned_addr learned my addr 172.21.15.44:0/4243112653 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f0840906700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d564d0 0x55be70d48460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f083bfff700 1 --2- 172.21.15.44:0/4243112653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d50ef0 0x55be70d4ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f0840906700 1 -- 172.21.15.44:0/4243112653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 msgr2=0x55be70d47f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f0840906700 1 --2- 172.21.15.44:0/4243112653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d47f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f0840906700 1 -- 172.21.15.44:0/4243112653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d50ef0 msgr2=0x55be70d4ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f0840906700 1 --2- 172.21.15.44:0/4243112653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d50ef0 0x55be70d4ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f0840906700 1 -- 172.21.15.44:0/4243112653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be70af21f0 con 0x55be70d564d0 2023-09-10T00:54:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f083b7fe700 1 --2- 172.21.15.44:0/4243112653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d47f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.469+0000 7f083bfff700 1 --2- 172.21.15.44:0/4243112653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d50ef0 0x55be70d4ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:54:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.470+0000 7f0840906700 1 --2- 172.21.15.44:0/4243112653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d564d0 0x55be70d48460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0834007800 tx=0x7f083400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cb2f3733e4f8d83 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.470+0000 7f0822ffd700 1 -- 172.21.15.44:0/4243112653 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083400e040 con 0x55be70d564d0 2023-09-10T00:54:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.470+0000 7f0822ffd700 1 -- 172.21.15.44:0/4243112653 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f083400f040 con 0x55be70d564d0 2023-09-10T00:54:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.470+0000 7f0822ffd700 1 -- 172.21.15.44:0/4243112653 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0834004280 con 0x55be70d564d0 2023-09-10T00:54:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.471+0000 7f084e52fb80 1 -- 172.21.15.44:0/4243112653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d564d0 msgr2=0x55be70d48460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.471+0000 7f084e52fb80 1 --2- 172.21.15.44:0/4243112653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d564d0 0x55be70d48460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0834007800 tx=0x7f083400b040 comp rx=0 tx=0).stop 2023-09-10T00:54:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.471+0000 7f084e52fb80 1 -- 172.21.15.44:0/4243112653 shutdown_connections 2023-09-10T00:54:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.471+0000 7f084e52fb80 1 --2- 172.21.15.44:0/4243112653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d564d0 0x55be70d48460 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.471+0000 7f084e52fb80 1 --2- 172.21.15.44:0/4243112653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d50ef0 0x55be70d4ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.471+0000 7f084e52fb80 1 --2- 172.21.15.44:0/4243112653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d47f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.471+0000 7f084e52fb80 1 -- 172.21.15.44:0/4243112653 >> 172.21.15.44:0/4243112653 conn(0x55be70cc2030 msgr2=0x55be70d4c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.471+0000 7f084e52fb80 1 -- 172.21.15.44:0/4243112653 shutdown_connections 2023-09-10T00:54:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.471+0000 7f084e52fb80 1 -- 172.21.15.44:0/4243112653 wait complete. 2023-09-10T00:54:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.471+0000 7f084e52fb80 1 -- start start 2023-09-10T00:54:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.472+0000 7f084e52fb80 1 Processor -- start 2023-09-10T00:54:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.472+0000 7f084e52fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d50ef0 0x55be70d55340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.472+0000 7f084e52fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d53b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.472+0000 7f084e52fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d564d0 0x55be70d54080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.472+0000 7f084e52fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be70d545c0 con 0x55be70d50ef0 2023-09-10T00:54:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.472+0000 7f084e52fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be70d54730 con 0x55be70d55b10 2023-09-10T00:54:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.472+0000 7f084e52fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be70d548f0 con 0x55be70d564d0 2023-09-10T00:54:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.472+0000 7f084e52fb80 10 client.0 _ll_register_callbacks cb 0x7f0834009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:54:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.472+0000 7f083bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d50ef0 0x55be70d55340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.472+0000 7f083bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d50ef0 0x55be70d55340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53492/0 (socket says 172.21.15.44:53492) 2023-09-10T00:54:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.472+0000 7f083bfff700 1 -- 172.21.15.44:0/2700387527 learned_addr learned my addr 172.21.15.44:0/2700387527 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:54:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.472+0000 7f083b7fe700 1 --2- 172.21.15.44:0/2700387527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d53b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.473+0000 7f0840906700 1 --2- 172.21.15.44:0/2700387527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d564d0 0x55be70d54080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.473+0000 7f083bfff700 1 -- 172.21.15.44:0/2700387527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d564d0 msgr2=0x55be70d54080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.473+0000 7f083bfff700 1 --2- 172.21.15.44:0/2700387527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d564d0 0x55be70d54080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.473+0000 7f083bfff700 1 -- 172.21.15.44:0/2700387527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 msgr2=0x55be70d53b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:54:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.473+0000 7f083bfff700 1 --2- 172.21.15.44:0/2700387527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d53b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.473+0000 7f083bfff700 1 -- 172.21.15.44:0/2700387527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083c10ff90 con 0x55be70d50ef0 2023-09-10T00:54:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.473+0000 7f083b7fe700 1 --2- 172.21.15.44:0/2700387527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d53b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.473+0000 7f083bfff700 1 --2- 172.21.15.44:0/2700387527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d50ef0 0x55be70d55340 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0830096770 tx=0x7f0830099df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c000e19df60ce2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.473+0000 7f081b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.473+0000 7f081b7fe700 1 -- 172.21.15.44:0/2700387527 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08300b84e0 con 0x55be70d50ef0 2023-09-10T00:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.473+0000 7f081b7fe700 1 -- 172.21.15.44:0/2700387527 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f083009a490 con 0x55be70d50ef0 2023-09-10T00:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.473+0000 7f084e52fb80 1 -- 172.21.15.44:0/2700387527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f083009e2c0 con 0x55be70d50ef0 2023-09-10T00:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.473+0000 7f081b7fe700 1 -- 172.21.15.44:0/2700387527 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08300985d0 con 0x55be70d50ef0 2023-09-10T00:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.473+0000 7f084e52fb80 10 client.17790 fetch_fsmap learned FSMap version 68 2023-09-10T00:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.473+0000 7f084e52fb80 1 -- 172.21.15.44:0/2700387527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7f0834007500 con 0x55be70d50ef0 2023-09-10T00:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.474+0000 7f081b7fe700 1 -- 172.21.15.44:0/2700387527 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f083009e2c0 con 0x55be70d50ef0 2023-09-10T00:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.474+0000 7f081b7fe700 1 -- 172.21.15.44:0/2700387527 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 68) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f083009a6f0 con 0x55be70d50ef0 2023-09-10T00:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.474+0000 7f084e52fb80 10 client.17790 fetch_fsmap finished waiting for FSMap version 68 2023-09-10T00:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.474+0000 7f084e52fb80 10 client.17790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.474+0000 7f084e52fb80 20 client.17790 populate_metadata read hostname 'smithi044' 2023-09-10T00:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.474+0000 7f084e52fb80 1 --2- 172.21.15.44:0/2700387527 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55be70d5b9b0 0x55be70dcee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.474+0000 7f084e52fb80 4 client.17790 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.474+0000 7f084e52fb80 1 -- 172.21.15.44:0/2700387527 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55be70c988c0 con 0x55be70d5b9b0 2023-09-10T00:54:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.475+0000 7f083b7fe700 1 --2- 172.21.15.44:0/2700387527 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55be70d5b9b0 0x55be70dcee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:54:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.475+0000 7f081b7fe700 10 client.17790 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.475+0000 7f083b7fe700 1 --2- 172.21.15.44:0/2700387527 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55be70d5b9b0 0x55be70dcee60 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a338239ce8863d4 server_cookie=c7c0c906649bccc4 in_seq=0 out_seq=0 2023-09-10T00:54:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.486+0000 7f081b7fe700 1 -- 172.21.15.44:0/2700387527 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55be70c988c0 con 0x55be70d5b9b0 2023-09-10T00:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.486+0000 7f081b7fe700 10 client.17790 handle_command_reply: tid=0 2023-09-10T00:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.486+0000 7f084e52fb80 1 client.17790 shutdown 2023-09-10T00:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.486+0000 7f084e52fb80 1 -- 172.21.15.44:0/2700387527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d50ef0 msgr2=0x55be70d55340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.486+0000 7f084e52fb80 1 --2- 172.21.15.44:0/2700387527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d50ef0 0x55be70d55340 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0830096770 tx=0x7f0830099df0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.486+0000 7f084e52fb80 1 -- 172.21.15.44:0/2700387527 shutdown_connections 2023-09-10T00:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.486+0000 7f084e52fb80 1 --2- 172.21.15.44:0/2700387527 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55be70d5b9b0 0x55be70dcee60 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.486+0000 7f084e52fb80 1 --2- 172.21.15.44:0/2700387527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55be70d564d0 0x55be70d54080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.486+0000 7f084e52fb80 1 --2- 172.21.15.44:0/2700387527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be70d50ef0 0x55be70d55340 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.486+0000 7f084e52fb80 1 --2- 172.21.15.44:0/2700387527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be70d55b10 0x55be70d53b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.486+0000 7f084e52fb80 1 -- 172.21.15.44:0/2700387527 >> 172.21.15.44:0/2700387527 conn(0x55be70cc2030 msgr2=0x55be70d4cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.486+0000 7f081b7fe700 0 client.17790 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.487+0000 7f084e52fb80 1 -- 172.21.15.44:0/2700387527 shutdown_connections 2023-09-10T00:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.487+0000 7f084e52fb80 1 -- 172.21.15.44:0/2700387527 wait complete. 2023-09-10T00:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.487+0000 7f084e52fb80 20 client.17790 trim_cache size 0 max 16384 2023-09-10T00:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:54:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:54:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:54:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:54:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.487+0000 7f0842369700 1 -- 172.21.15.44:0/433018609 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f08240a0820 msgr2=0x7f08240a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.487+0000 7f0842369700 1 --2- 172.21.15.44:0/433018609 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f08240a0820 0x7f08240a2ce0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f083c107ea0 tx=0x7f082c004300 comp rx=0 tx=0).stop 2023-09-10T00:54:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.487+0000 7f0842369700 1 -- 172.21.15.44:0/433018609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f083c1027c0 msgr2=0x7f083c10ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:54:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.487+0000 7f0842369700 1 --2- 172.21.15.44:0/433018609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f083c1027c0 0x7f083c10ace0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f083000e9a0 tx=0x7f083000ecb0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.489+0000 7f0842369700 1 -- 172.21.15.44:0/433018609 shutdown_connections 2023-09-10T00:54:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.489+0000 7f0842369700 1 --2- 172.21.15.44:0/433018609 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f08240a0820 0x7f08240a2ce0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.489+0000 7f0842369700 1 --2- 172.21.15.44:0/433018609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c1087d0 0x7f083c1bdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.489+0000 7f0842369700 1 --2- 172.21.15.44:0/433018609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f083c1027c0 0x7f083c10ace0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.489+0000 7f0842369700 1 --2- 172.21.15.44:0/433018609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f083c103180 0x7f083c10b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:54:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.489+0000 7f0842369700 1 -- 172.21.15.44:0/433018609 >> 172.21.15.44:0/433018609 conn(0x7f083c0fe290 msgr2=0x7f083c1062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:54:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.489+0000 7f0842369700 1 -- 172.21.15.44:0/433018609 shutdown_connections 2023-09-10T00:54:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:54:46.489+0000 7f0842369700 1 -- 172.21.15.44:0/433018609 wait complete. 2023-09-10T00:54:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:46 smithi131 ceph-mon[103713]: pgmap v442: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.8 MiB/s wr, 504 op/s 2023-09-10T00:54:47.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:46 smithi044 ceph-mon[98676]: pgmap v442: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.8 MiB/s wr, 504 op/s 2023-09-10T00:54:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:46 smithi099 ceph-mon[102692]: pgmap v442: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.8 MiB/s wr, 504 op/s 2023-09-10T00:54:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:48 smithi131 ceph-mon[103713]: pgmap v443: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.5 MiB/s wr, 324 op/s 2023-09-10T00:54:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:48 smithi044 ceph-mon[98676]: pgmap v443: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.5 MiB/s wr, 324 op/s 2023-09-10T00:54:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:48 smithi099 ceph-mon[102692]: pgmap v443: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.5 MiB/s wr, 324 op/s 2023-09-10T00:54:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:50 smithi131 ceph-mon[103713]: pgmap v444: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.5 MiB/s wr, 324 op/s 2023-09-10T00:54:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:50 smithi044 ceph-mon[98676]: pgmap v444: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.5 MiB/s wr, 324 op/s 2023-09-10T00:54:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:50 smithi099 ceph-mon[102692]: pgmap v444: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.5 MiB/s wr, 324 op/s 2023-09-10T00:54:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:52 smithi131 ceph-mon[103713]: pgmap v445: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.8 MiB/s wr, 495 op/s 2023-09-10T00:54:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:52 smithi044 ceph-mon[98676]: pgmap v445: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.8 MiB/s wr, 495 op/s 2023-09-10T00:54:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:52 smithi099 ceph-mon[102692]: pgmap v445: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.8 MiB/s wr, 495 op/s 2023-09-10T00:54:55.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:54 smithi131 ceph-mon[103713]: pgmap v446: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.4 MiB/s wr, 319 op/s 2023-09-10T00:54:55.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:54 smithi044 ceph-mon[98676]: pgmap v446: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.4 MiB/s wr, 319 op/s 2023-09-10T00:54:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:54 smithi099 ceph-mon[102692]: pgmap v446: 129 pgs: 129 active+clean; 9.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.4 MiB/s wr, 319 op/s 2023-09-10T00:54:57.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:56 smithi131 ceph-mon[103713]: pgmap v447: 129 pgs: 129 active+clean; 9.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.5 MiB/s wr, 463 op/s 2023-09-10T00:54:57.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:56 smithi044 ceph-mon[98676]: pgmap v447: 129 pgs: 129 active+clean; 9.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.5 MiB/s wr, 463 op/s 2023-09-10T00:54:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:56 smithi099 ceph-mon[102692]: pgmap v447: 129 pgs: 129 active+clean; 9.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.5 MiB/s wr, 463 op/s 2023-09-10T00:54:59.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:54:58 smithi131 ceph-mon[103713]: pgmap v448: 129 pgs: 129 active+clean; 9.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.4 MiB/s wr, 314 op/s 2023-09-10T00:54:59.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:54:58 smithi044 ceph-mon[98676]: pgmap v448: 129 pgs: 129 active+clean; 9.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.4 MiB/s wr, 314 op/s 2023-09-10T00:54:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:54:58 smithi099 ceph-mon[102692]: pgmap v448: 129 pgs: 129 active+clean; 9.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.4 MiB/s wr, 314 op/s 2023-09-10T00:55:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.087+0000 7ff24c7a2700 1 -- 172.21.15.44:0/3668850461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2440ff760 msgr2=0x7ff2440ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.087+0000 7ff24c7a2700 1 --2- 172.21.15.44:0/3668850461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2440ff760 0x7ff2440ffbe0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff2340099c0 tx=0x7ff234009cd0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.087+0000 7ff24c7a2700 1 -- 172.21.15.44:0/3668850461 shutdown_connections 2023-09-10T00:55:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.087+0000 7ff24c7a2700 1 --2- 172.21.15.44:0/3668850461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff244100120 0x7ff24410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.087+0000 7ff24c7a2700 1 --2- 172.21.15.44:0/3668850461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2441055b0 0x7ff244105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.087+0000 7ff24c7a2700 1 --2- 172.21.15.44:0/3668850461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2440ff760 0x7ff2440ffbe0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.087+0000 7ff24c7a2700 1 -- 172.21.15.44:0/3668850461 >> 172.21.15.44:0/3668850461 conn(0x7ff244074f30 msgr2=0x7ff244075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.087+0000 7ff24c7a2700 1 -- 172.21.15.44:0/3668850461 shutdown_connections 2023-09-10T00:55:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.087+0000 7ff24c7a2700 1 -- 172.21.15.44:0/3668850461 wait complete. 2023-09-10T00:55:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.088+0000 7ff24c7a2700 1 Processor -- start 2023-09-10T00:55:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.088+0000 7ff24c7a2700 1 -- start start 2023-09-10T00:55:00.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.088+0000 7ff24c7a2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2440ff760 0x7ff24410ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.088+0000 7ff24c7a2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff244100120 0x7ff24410d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.088+0000 7ff24c7a2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2441055b0 0x7ff2441a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.088+0000 7ff24c7a2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2441a4d40 con 0x7ff244100120 2023-09-10T00:55:00.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.088+0000 7ff24c7a2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2441a4e80 con 0x7ff2441055b0 2023-09-10T00:55:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.088+0000 7ff24c7a2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2441a4ff0 con 0x7ff2440ff760 2023-09-10T00:55:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.088+0000 7ff249d3d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff244100120 0x7ff24410d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.088+0000 7ff24a53e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2440ff760 0x7ff24410ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.088+0000 7ff249d3d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff244100120 0x7ff24410d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53508/0 (socket says 172.21.15.44:53508) 2023-09-10T00:55:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.088+0000 7ff249d3d700 1 -- 172.21.15.44:0/131120400 learned_addr learned my addr 172.21.15.44:0/131120400 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.088+0000 7ff24ad3f700 1 --2- 172.21.15.44:0/131120400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2441055b0 0x7ff2441a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.089+0000 7ff24ad3f700 1 -- 172.21.15.44:0/131120400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2440ff760 msgr2=0x7ff24410ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.089+0000 7ff24ad3f700 1 --2- 172.21.15.44:0/131120400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2440ff760 0x7ff24410ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.089+0000 7ff24ad3f700 1 -- 172.21.15.44:0/131120400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff244100120 msgr2=0x7ff24410d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.089+0000 7ff24ad3f700 1 --2- 172.21.15.44:0/131120400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff244100120 0x7ff24410d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.089+0000 7ff24ad3f700 1 -- 172.21.15.44:0/131120400 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2340095f0 con 0x7ff2441055b0 2023-09-10T00:55:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.089+0000 7ff249d3d700 1 --2- 172.21.15.44:0/131120400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff244100120 0x7ff24410d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:55:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.089+0000 7ff24ad3f700 1 --2- 172.21.15.44:0/131120400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2441055b0 0x7ff2441a4800 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00efc0 tx=0x7ff23c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.089+0000 7ff23b7fe700 1 -- 172.21.15.44:0/131120400 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c018070 con 0x7ff2441055b0 2023-09-10T00:55:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.089+0000 7ff23b7fe700 1 -- 172.21.15.44:0/131120400 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff23c00f040 con 0x7ff2441055b0 2023-09-10T00:55:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.089+0000 7ff23b7fe700 1 -- 172.21.15.44:0/131120400 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c00cc70 con 0x7ff2441055b0 2023-09-10T00:55:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.089+0000 7ff24c7a2700 1 -- 172.21.15.44:0/131120400 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2441a5270 con 0x7ff2441055b0 2023-09-10T00:55:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.089+0000 7ff24c7a2700 1 -- 172.21.15.44:0/131120400 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2441a5730 con 0x7ff2441055b0 2023-09-10T00:55:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.092+0000 7ff23b7fe700 1 -- 172.21.15.44:0/131120400 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ff23c003a00 con 0x7ff2441055b0 2023-09-10T00:55:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.093+0000 7ff23b7fe700 1 --2- 172.21.15.44:0/131120400 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff2300a07b0 0x7ff2300a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.093+0000 7ff23b7fe700 1 -- 172.21.15.44:0/131120400 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff23c0cbf20 con 0x7ff2441055b0 2023-09-10T00:55:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.093+0000 7ff24a53e700 1 --2- 172.21.15.44:0/131120400 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff2300a07b0 0x7ff2300a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.094+0000 7ff258968b80 1 Processor -- start 2023-09-10T00:55:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.094+0000 7ff258968b80 1 -- start start 2023-09-10T00:55:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.094+0000 7ff24a53e700 1 --2- 172.21.15.44:0/131120400 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff2300a07b0 0x7ff2300a2c70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff240009bd0 tx=0x7ff240009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.094+0000 7ff258968b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce61a0 0x556377ce6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.094+0000 7ff258968b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0300 0x556377ce0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.094+0000 7ff258968b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce0cc0 0x556377cecf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.094+0000 7ff258968b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556377c396d0 con 0x556377ce61a0 2023-09-10T00:55:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.094+0000 7ff258968b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556377c36ff0 con 0x556377ce0cc0 2023-09-10T00:55:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.094+0000 7ff258968b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556377c328e0 con 0x556377ce0300 2023-09-10T00:55:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.094+0000 7ff24a53e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce61a0 0x556377ce6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.094+0000 7ff24a53e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce61a0 0x556377ce6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53514/0 (socket says 172.21.15.44:53514) 2023-09-10T00:55:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.094+0000 7ff24a53e700 1 -- 172.21.15.44:0/4037787263 learned_addr learned my addr 172.21.15.44:0/4037787263 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.094+0000 7ff249d3d700 1 --2- 172.21.15.44:0/4037787263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0300 0x556377ce0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.095+0000 7ff24a53e700 1 -- 172.21.15.44:0/4037787263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0300 msgr2=0x556377ce0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.095+0000 7ff24a53e700 1 --2- 172.21.15.44:0/4037787263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0300 0x556377ce0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.095+0000 7ff24a53e700 1 -- 172.21.15.44:0/4037787263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce0cc0 msgr2=0x556377cecf30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T00:55:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.095+0000 7ff24a53e700 1 --2- 172.21.15.44:0/4037787263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce0cc0 0x556377cecf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.095+0000 7ff24a53e700 1 -- 172.21.15.44:0/4037787263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556377a5ed80 con 0x556377ce61a0 2023-09-10T00:55:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.095+0000 7ff249d3d700 1 --2- 172.21.15.44:0/4037787263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0300 0x556377ce0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:55:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.095+0000 7ff24a53e700 1 --2- 172.21.15.44:0/4037787263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce61a0 0x556377ce6580 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff240005910 tx=0x7ff240013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a789805fcb37e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.095+0000 7ff238ff9700 1 -- 172.21.15.44:0/4037787263 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff240011db0 con 0x556377ce61a0 2023-09-10T00:55:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.095+0000 7ff238ff9700 1 -- 172.21.15.44:0/4037787263 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff240015070 con 0x556377ce61a0 2023-09-10T00:55:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.095+0000 7ff238ff9700 1 -- 172.21.15.44:0/4037787263 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24002e620 con 0x556377ce61a0 2023-09-10T00:55:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.096+0000 7ff258968b80 1 -- 172.21.15.44:0/4037787263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce61a0 msgr2=0x556377ce6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.096+0000 7ff258968b80 1 --2- 172.21.15.44:0/4037787263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce61a0 0x556377ce6580 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff240005910 tx=0x7ff240013000 comp rx=0 tx=0).stop 2023-09-10T00:55:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.096+0000 7ff258968b80 1 -- 172.21.15.44:0/4037787263 shutdown_connections 2023-09-10T00:55:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.096+0000 7ff258968b80 1 --2- 172.21.15.44:0/4037787263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce0cc0 0x556377cecf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.096+0000 7ff258968b80 1 --2- 172.21.15.44:0/4037787263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce61a0 0x556377ce6580 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.096+0000 7ff258968b80 1 --2- 172.21.15.44:0/4037787263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0300 0x556377ce0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.096+0000 7ff258968b80 1 -- 172.21.15.44:0/4037787263 >> 172.21.15.44:0/4037787263 conn(0x556377c55030 msgr2=0x556377cdb560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.096+0000 7ff258968b80 1 -- 172.21.15.44:0/4037787263 shutdown_connections 2023-09-10T00:55:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.096+0000 7ff258968b80 1 -- 172.21.15.44:0/4037787263 wait complete. 2023-09-10T00:55:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.097+0000 7ff258968b80 1 -- start start 2023-09-10T00:55:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.097+0000 7ff258968b80 1 Processor -- start 2023-09-10T00:55:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.097+0000 7ff258968b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce0300 0x556377cece30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.097+0000 7ff258968b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce0cc0 0x556377cea150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.097+0000 7ff258968b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce61a0 0x556377cea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.097+0000 7ff258968b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556377bd0b10 con 0x556377ce0cc0 2023-09-10T00:55:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.097+0000 7ff258968b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556377bd0810 con 0x556377ce0300 2023-09-10T00:55:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.097+0000 7ff258968b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556377ab48c0 con 0x556377ce61a0 2023-09-10T00:55:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.097+0000 7ff258968b80 10 client.0 _ll_register_callbacks cb 0x556377b1aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:55:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.098+0000 7ff249d3d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce0cc0 0x556377cea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.098+0000 7ff249d3d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce0cc0 0x556377cea150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53520/0 (socket says 172.21.15.44:53520) 2023-09-10T00:55:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.098+0000 7ff249d3d700 1 -- 172.21.15.44:0/3799486545 learned_addr learned my addr 172.21.15.44:0/3799486545 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.098+0000 7ff24ad3f700 1 --2- 172.21.15.44:0/3799486545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce61a0 0x556377cea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.098+0000 7ff24a53e700 1 --2- 172.21.15.44:0/3799486545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce0300 0x556377cece30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.098+0000 7ff249d3d700 1 -- 172.21.15.44:0/3799486545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce61a0 msgr2=0x556377cea7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.098+0000 7ff249d3d700 1 --2- 172.21.15.44:0/3799486545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce61a0 0x556377cea7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.098+0000 7ff249d3d700 1 -- 172.21.15.44:0/3799486545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce0300 msgr2=0x556377cece30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.098+0000 7ff249d3d700 1 --2- 172.21.15.44:0/3799486545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce0300 0x556377cece30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.098+0000 7ff249d3d700 1 -- 172.21.15.44:0/3799486545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2440679f0 con 0x556377ce0cc0 2023-09-10T00:55:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.098+0000 7ff24a53e700 1 --2- 172.21.15.44:0/3799486545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce0300 0x556377cece30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:55:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.098+0000 7ff249d3d700 1 --2- 172.21.15.44:0/3799486545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce0cc0 0x556377cea150 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff234009990 tx=0x7ff2340052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a3f9bd6c349ba6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.098+0000 7ff22dffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:55:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.099+0000 7ff22dffb700 1 -- 172.21.15.44:0/3799486545 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234003960 con 0x556377ce0cc0 2023-09-10T00:55:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.099+0000 7ff22dffb700 1 -- 172.21.15.44:0/3799486545 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff234003ac0 con 0x556377ce0cc0 2023-09-10T00:55:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.099+0000 7ff258968b80 1 -- 172.21.15.44:0/3799486545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556377b8a490 con 0x556377ce0cc0 2023-09-10T00:55:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.099+0000 7ff22dffb700 1 -- 172.21.15.44:0/3799486545 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff234003c80 con 0x556377ce0cc0 2023-09-10T00:55:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.099+0000 7ff258968b80 10 client.17802 fetch_fsmap learned FSMap version 68 2023-09-10T00:55:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.099+0000 7ff258968b80 1 -- 172.21.15.44:0/3799486545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7ff240010b40 con 0x556377ce0cc0 2023-09-10T00:55:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.100+0000 7ff22dffb700 1 -- 172.21.15.44:0/3799486545 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556377b8a490 con 0x556377ce0cc0 2023-09-10T00:55:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.100+0000 7ff22dffb700 1 -- 172.21.15.44:0/3799486545 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 68) v1 ==== 7922+0+0 (secure 0 0 0) 0x7ff23400f460 con 0x556377ce0cc0 2023-09-10T00:55:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.100+0000 7ff258968b80 10 client.17802 fetch_fsmap finished waiting for FSMap version 68 2023-09-10T00:55:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.100+0000 7ff258968b80 10 client.17802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:55:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.100+0000 7ff258968b80 20 client.17802 populate_metadata read hostname 'smithi044' 2023-09-10T00:55:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.100+0000 7ff258968b80 1 --2- 172.21.15.44:0/3799486545 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556377cf8f40 0x556377cfb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.100+0000 7ff258968b80 4 client.17802 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:55:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.100+0000 7ff258968b80 1 -- 172.21.15.44:0/3799486545 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556377c47840 con 0x556377cf8f40 2023-09-10T00:55:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.100+0000 7ff24a53e700 1 --2- 172.21.15.44:0/3799486545 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556377cf8f40 0x556377cfb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.101+0000 7ff24a53e700 1 --2- 172.21.15.44:0/3799486545 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556377cf8f40 0x556377cfb360 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f25940f1ba1d11e server_cookie=b7a5dee0f702599b in_seq=0 out_seq=0 2023-09-10T00:55:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.101+0000 7ff22dffb700 10 client.17802 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.101+0000 7ff22dffb700 1 -- 172.21.15.44:0/3799486545 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556377c47840 con 0x556377cf8f40 2023-09-10T00:55:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.101+0000 7ff22dffb700 10 client.17802 handle_command_reply: tid=0 2023-09-10T00:55:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.101+0000 7ff258968b80 1 client.17802 shutdown 2023-09-10T00:55:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.101+0000 7ff258968b80 1 -- 172.21.15.44:0/3799486545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce0cc0 msgr2=0x556377cea150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.102+0000 7ff258968b80 1 --2- 172.21.15.44:0/3799486545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce0cc0 0x556377cea150 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff234009990 tx=0x7ff2340052e0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.102+0000 7ff258968b80 1 -- 172.21.15.44:0/3799486545 shutdown_connections 2023-09-10T00:55:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.102+0000 7ff258968b80 1 --2- 172.21.15.44:0/3799486545 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556377cf8f40 0x556377cfb360 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.102+0000 7ff258968b80 1 --2- 172.21.15.44:0/3799486545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce61a0 0x556377cea7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.102+0000 7ff258968b80 1 --2- 172.21.15.44:0/3799486545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce0300 0x556377cece30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.102+0000 7ff258968b80 1 --2- 172.21.15.44:0/3799486545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce0cc0 0x556377cea150 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.102+0000 7ff258968b80 1 -- 172.21.15.44:0/3799486545 >> 172.21.15.44:0/3799486545 conn(0x556377c55030 msgr2=0x556377cda7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.102+0000 7ff22dffb700 0 client.17802 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.102+0000 7ff258968b80 1 -- 172.21.15.44:0/3799486545 shutdown_connections 2023-09-10T00:55:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.102+0000 7ff258968b80 1 -- 172.21.15.44:0/3799486545 wait complete. 2023-09-10T00:55:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.102+0000 7ff258968b80 20 client.17802 trim_cache size 0 max 16384 2023-09-10T00:55:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.121+0000 7ff258968b80 1 Processor -- start 2023-09-10T00:55:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.121+0000 7ff258968b80 1 -- start start 2023-09-10T00:55:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.121+0000 7ff258968b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0300 0x556377d042f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.121+0000 7ff258968b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce0cc0 0x556377cdaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.121+0000 7ff258968b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce61a0 0x556377cdb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.121+0000 7ff258968b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556377cfec80 con 0x556377ce61a0 2023-09-10T00:55:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.121+0000 7ff258968b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556377cd58a0 con 0x556377ce0cc0 2023-09-10T00:55:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.121+0000 7ff258968b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556377ce5e00 con 0x556377ce0300 2023-09-10T00:55:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.122+0000 7ff24ad3f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce61a0 0x556377cdb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.122+0000 7ff24a53e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0300 0x556377d042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.122+0000 7ff249d3d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce0cc0 0x556377cdaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.122+0000 7ff24ad3f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce61a0 0x556377cdb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53530/0 (socket says 172.21.15.44:53530) 2023-09-10T00:55:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.122+0000 7ff24a53e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0300 0x556377d042f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60408/0 (socket says 172.21.15.44:60408) 2023-09-10T00:55:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.122+0000 7ff24ad3f700 1 -- 172.21.15.44:0/1977260057 learned_addr learned my addr 172.21.15.44:0/1977260057 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.122+0000 7ff249d3d700 1 -- 172.21.15.44:0/1977260057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0300 msgr2=0x556377d042f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.122+0000 7ff249d3d700 1 --2- 172.21.15.44:0/1977260057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0300 0x556377d042f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.122+0000 7ff249d3d700 1 -- 172.21.15.44:0/1977260057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce61a0 msgr2=0x556377cdb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.122+0000 7ff249d3d700 1 --2- 172.21.15.44:0/1977260057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce61a0 0x556377cdb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.122+0000 7ff249d3d700 1 -- 172.21.15.44:0/1977260057 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556377a47c00 con 0x556377ce0cc0 2023-09-10T00:55:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.122+0000 7ff249d3d700 1 --2- 172.21.15.44:0/1977260057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce0cc0 0x556377cdaf20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff23400bf80 tx=0x7ff23400b5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9087c856c2c6cdf2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.122+0000 7ff2397fa700 1 -- 172.21.15.44:0/1977260057 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2340058e0 con 0x556377ce0cc0 2023-09-10T00:55:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.123+0000 7ff2397fa700 1 -- 172.21.15.44:0/1977260057 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff23401e040 con 0x556377ce0cc0 2023-09-10T00:55:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.123+0000 7ff2397fa700 1 -- 172.21.15.44:0/1977260057 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2340246e0 con 0x556377ce0cc0 2023-09-10T00:55:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.123+0000 7ff258968b80 1 -- 172.21.15.44:0/1977260057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce0cc0 msgr2=0x556377cdaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.123+0000 7ff258968b80 1 --2- 172.21.15.44:0/1977260057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce0cc0 0x556377cdaf20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff23400bf80 tx=0x7ff23400b5c0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.123+0000 7ff258968b80 1 -- 172.21.15.44:0/1977260057 shutdown_connections 2023-09-10T00:55:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.123+0000 7ff258968b80 1 --2- 172.21.15.44:0/1977260057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce61a0 0x556377cdb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.123+0000 7ff258968b80 1 --2- 172.21.15.44:0/1977260057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0300 0x556377d042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.123+0000 7ff258968b80 1 --2- 172.21.15.44:0/1977260057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce0cc0 0x556377cdaf20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.123+0000 7ff258968b80 1 -- 172.21.15.44:0/1977260057 >> 172.21.15.44:0/1977260057 conn(0x556377c55030 msgr2=0x556377ce41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.123+0000 7ff258968b80 1 -- 172.21.15.44:0/1977260057 shutdown_connections 2023-09-10T00:55:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.123+0000 7ff258968b80 1 -- 172.21.15.44:0/1977260057 wait complete. 2023-09-10T00:55:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.124+0000 7ff258968b80 1 -- start start 2023-09-10T00:55:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.124+0000 7ff258968b80 1 Processor -- start 2023-09-10T00:55:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.124+0000 7ff258968b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce0300 0x556377d042d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.124+0000 7ff258968b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0cc0 0x556377d02730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.124+0000 7ff258968b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce61a0 0x556377d02c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.124+0000 7ff258968b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556377d03280 con 0x556377ce0300 2023-09-10T00:55:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.124+0000 7ff258968b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556377d033f0 con 0x556377ce61a0 2023-09-10T00:55:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.124+0000 7ff258968b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556377d035b0 con 0x556377ce0cc0 2023-09-10T00:55:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.125+0000 7ff258968b80 10 client.0 _ll_register_callbacks cb 0x7ff23400be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:55:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.125+0000 7ff24a53e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce0300 0x556377d042d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.125+0000 7ff249d3d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0cc0 0x556377d02730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.125+0000 7ff24a53e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce0300 0x556377d042d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53532/0 (socket says 172.21.15.44:53532) 2023-09-10T00:55:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.125+0000 7ff24a53e700 1 -- 172.21.15.44:0/1834767628 learned_addr learned my addr 172.21.15.44:0/1834767628 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.125+0000 7ff24ad3f700 1 --2- 172.21.15.44:0/1834767628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce61a0 0x556377d02c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.125+0000 7ff24a53e700 1 -- 172.21.15.44:0/1834767628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0cc0 msgr2=0x556377d02730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.125+0000 7ff24a53e700 1 --2- 172.21.15.44:0/1834767628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0cc0 0x556377d02730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.125+0000 7ff24a53e700 1 -- 172.21.15.44:0/1834767628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce61a0 msgr2=0x556377d02c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.125+0000 7ff24a53e700 1 --2- 172.21.15.44:0/1834767628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce61a0 0x556377d02c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.125+0000 7ff24a53e700 1 -- 172.21.15.44:0/1834767628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff240010b40 con 0x556377ce0300 2023-09-10T00:55:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.125+0000 7ff249d3d700 1 --2- 172.21.15.44:0/1834767628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0cc0 0x556377d02730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:55:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.125+0000 7ff24a53e700 1 --2- 172.21.15.44:0/1834767628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce0300 0x556377d042d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff240010760 tx=0x7ff2400092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70a5d8f96e3391b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.125+0000 7ff22dffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:55:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.126+0000 7ff22dffb700 1 -- 172.21.15.44:0/1834767628 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff240043b10 con 0x556377ce0300 2023-09-10T00:55:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.126+0000 7ff22dffb700 1 -- 172.21.15.44:0/1834767628 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff240017070 con 0x556377ce0300 2023-09-10T00:55:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.126+0000 7ff258968b80 1 -- 172.21.15.44:0/1834767628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff240015070 con 0x556377ce0300 2023-09-10T00:55:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.126+0000 7ff22dffb700 1 -- 172.21.15.44:0/1834767628 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2400435c0 con 0x556377ce0300 2023-09-10T00:55:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.126+0000 7ff258968b80 10 client.17808 fetch_fsmap learned FSMap version 68 2023-09-10T00:55:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.126+0000 7ff258968b80 1 -- 172.21.15.44:0/1834767628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x556377a851f0 con 0x556377ce0300 2023-09-10T00:55:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.126+0000 7ff22dffb700 1 -- 172.21.15.44:0/1834767628 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff240015070 con 0x556377ce0300 2023-09-10T00:55:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.126+0000 7ff22dffb700 1 -- 172.21.15.44:0/1834767628 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 68) v1 ==== 7922+0+0 (secure 0 0 0) 0x7ff24002f550 con 0x556377ce0300 2023-09-10T00:55:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.126+0000 7ff258968b80 10 client.17808 fetch_fsmap finished waiting for FSMap version 68 2023-09-10T00:55:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.126+0000 7ff258968b80 10 client.17808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:55:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.126+0000 7ff258968b80 20 client.17808 populate_metadata read hostname 'smithi044' 2023-09-10T00:55:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.127+0000 7ff258968b80 1 --2- 172.21.15.44:0/1834767628 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556377d4e930 0x556377d50d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.127+0000 7ff258968b80 4 client.17808 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:55:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.127+0000 7ff258968b80 1 -- 172.21.15.44:0/1834767628 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556377c2b8c0 con 0x556377d4e930 2023-09-10T00:55:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.127+0000 7ff249d3d700 1 --2- 172.21.15.44:0/1834767628 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556377d4e930 0x556377d50d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.127+0000 7ff22dffb700 10 client.17808 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:00.127+0000 7ff249d3d700 1 --2- 172.21.15.44:0/1834767628 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556377d4e930 0x556377d50d50 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=827608ea1cb0a301 server_cookie=f6b3d70fd70d29d1 in_seq=0 out_seq=0 2023-09-10T00:55:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:00 smithi131 ceph-mon[103713]: pgmap v449: 129 pgs: 129 active+clean; 9.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.4 MiB/s wr, 314 op/s 2023-09-10T00:55:01.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:00 smithi044 ceph-mon[98676]: pgmap v449: 129 pgs: 129 active+clean; 9.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.4 MiB/s wr, 314 op/s 2023-09-10T00:55:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:00 smithi099 ceph-mon[102692]: pgmap v449: 129 pgs: 129 active+clean; 9.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.4 MiB/s wr, 314 op/s 2023-09-10T00:55:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:02 smithi131 ceph-mon[103713]: pgmap v450: 129 pgs: 129 active+clean; 9.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.7 MiB/s wr, 488 op/s 2023-09-10T00:55:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:02 smithi099 ceph-mon[102692]: pgmap v450: 129 pgs: 129 active+clean; 9.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.7 MiB/s wr, 488 op/s 2023-09-10T00:55:03.342 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:02 smithi044 ceph-mon[98676]: pgmap v450: 129 pgs: 129 active+clean; 9.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.7 MiB/s wr, 488 op/s 2023-09-10T00:55:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:55:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:55:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:55:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:55:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.413+0000 7ff22dffb700 1 -- 172.21.15.44:0/1834767628 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556377c2b8c0 con 0x556377d4e930 2023-09-10T00:55:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.413+0000 7ff22dffb700 10 client.17808 handle_command_reply: tid=0 2023-09-10T00:55:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.413+0000 7ff258968b80 1 client.17808 shutdown 2023-09-10T00:55:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.413+0000 7ff258968b80 1 -- 172.21.15.44:0/1834767628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce0300 msgr2=0x556377d042d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.413+0000 7ff258968b80 1 --2- 172.21.15.44:0/1834767628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce0300 0x556377d042d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff240010760 tx=0x7ff2400092d0 comp rx=0 tx=0).stop 2023-09-10T00:55:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.414+0000 7ff258968b80 1 -- 172.21.15.44:0/1834767628 shutdown_connections 2023-09-10T00:55:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.414+0000 7ff258968b80 1 --2- 172.21.15.44:0/1834767628 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556377d4e930 0x556377d50d50 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.414+0000 7ff258968b80 1 --2- 172.21.15.44:0/1834767628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556377ce61a0 0x556377d02c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.414+0000 7ff22dffb700 0 client.17808 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.414+0000 7ff258968b80 1 --2- 172.21.15.44:0/1834767628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556377ce0300 0x556377d042d0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.414+0000 7ff258968b80 1 --2- 172.21.15.44:0/1834767628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556377ce0cc0 0x556377d02730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.414+0000 7ff258968b80 1 -- 172.21.15.44:0/1834767628 >> 172.21.15.44:0/1834767628 conn(0x556377c55030 msgr2=0x556377cf5540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.414+0000 7ff258968b80 1 -- 172.21.15.44:0/1834767628 shutdown_connections 2023-09-10T00:55:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.414+0000 7ff258968b80 1 -- 172.21.15.44:0/1834767628 wait complete. 2023-09-10T00:55:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.414+0000 7ff258968b80 20 client.17808 trim_cache size 0 max 16384 2023-09-10T00:55:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.415+0000 7ff24c7a2700 1 -- 172.21.15.44:0/131120400 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff2300a07b0 msgr2=0x7ff2300a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.415+0000 7ff24c7a2700 1 --2- 172.21.15.44:0/131120400 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff2300a07b0 0x7ff2300a2c70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff240009bd0 tx=0x7ff240009260 comp rx=0 tx=0).stop 2023-09-10T00:55:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.415+0000 7ff24c7a2700 1 -- 172.21.15.44:0/131120400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2441055b0 msgr2=0x7ff2441a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.415+0000 7ff24c7a2700 1 --2- 172.21.15.44:0/131120400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2441055b0 0x7ff2441a4800 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00efc0 tx=0x7ff23c00c3a0 comp rx=0 tx=0).stop 2023-09-10T00:55:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.415+0000 7ff24c7a2700 1 -- 172.21.15.44:0/131120400 shutdown_connections 2023-09-10T00:55:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.415+0000 7ff24c7a2700 1 --2- 172.21.15.44:0/131120400 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff2300a07b0 0x7ff2300a2c70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.415+0000 7ff24c7a2700 1 --2- 172.21.15.44:0/131120400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2441055b0 0x7ff2441a4800 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.415+0000 7ff24c7a2700 1 --2- 172.21.15.44:0/131120400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2440ff760 0x7ff24410ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.415+0000 7ff24c7a2700 1 --2- 172.21.15.44:0/131120400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff244100120 0x7ff24410d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.415+0000 7ff24c7a2700 1 -- 172.21.15.44:0/131120400 >> 172.21.15.44:0/131120400 conn(0x7ff244074f30 msgr2=0x7ff2440fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.415+0000 7ff24c7a2700 1 -- 172.21.15.44:0/131120400 shutdown_connections 2023-09-10T00:55:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:03.415+0000 7ff24c7a2700 1 -- 172.21.15.44:0/131120400 wait complete. 2023-09-10T00:55:04.296 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: *** Caught signal (Segmentation fault) ** 2023-09-10T00:55:04.296 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: in thread 7fcaf3b3c700 thread_name:md_log_replay 2023-09-10T00:55:04.296 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T00:55:04.297 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fcb046ebcf0] 2023-09-10T00:55:04.297 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x555c647d92f2] 2023-09-10T00:55:04.297 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x555c647db543] 2023-09-10T00:55:04.297 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 4: (MDLog::_replay_thread()+0x647) [0x555c64844e97] 2023-09-10T00:55:04.297 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 5: (MDLog::ReplayThread::entry()+0x11) [0x555c6484e381] 2023-09-10T00:55:04.298 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fcb046e11ca] 2023-09-10T00:55:04.298 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 7: clone() 2023-09-10T00:55:04.298 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 2023-09-10T00:55:03.977+0000 7fcaf3b3c700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T00:55:04.298 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: in thread 7fcaf3b3c700 thread_name:md_log_replay 2023-09-10T00:55:04.298 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 2023-09-10T00:55:04.299 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T00:55:04.299 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fcb046ebcf0] 2023-09-10T00:55:04.299 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x555c647d92f2] 2023-09-10T00:55:04.299 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x555c647db543] 2023-09-10T00:55:04.299 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 4: (MDLog::_replay_thread()+0x647) [0x555c64844e97] 2023-09-10T00:55:04.300 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 5: (MDLog::ReplayThread::entry()+0x11) [0x555c6484e381] 2023-09-10T00:55:04.300 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fcb046e11ca] 2023-09-10T00:55:04.300 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 7: clone() 2023-09-10T00:55:04.300 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T00:55:04.300 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 2023-09-10T00:55:04.300 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 0> 2023-09-10T00:55:03.977+0000 7fcaf3b3c700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T00:55:04.301 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: in thread 7fcaf3b3c700 thread_name:md_log_replay 2023-09-10T00:55:04.301 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 2023-09-10T00:55:04.301 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T00:55:04.301 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fcb046ebcf0] 2023-09-10T00:55:04.301 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x555c647d92f2] 2023-09-10T00:55:04.301 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x555c647db543] 2023-09-10T00:55:04.302 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 4: (MDLog::_replay_thread()+0x647) [0x555c64844e97] 2023-09-10T00:55:04.302 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 5: (MDLog::ReplayThread::entry()+0x11) [0x555c6484e381] 2023-09-10T00:55:04.302 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fcb046e11ca] 2023-09-10T00:55:04.302 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 7: clone() 2023-09-10T00:55:04.302 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T00:55:04.303 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 2023-09-10T00:55:04.303 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 0> 2023-09-10T00:55:03.977+0000 7fcaf3b3c700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T00:55:04.303 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: in thread 7fcaf3b3c700 thread_name:md_log_replay 2023-09-10T00:55:04.303 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 2023-09-10T00:55:04.303 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T00:55:04.304 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fcb046ebcf0] 2023-09-10T00:55:04.304 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x555c647d92f2] 2023-09-10T00:55:04.304 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x555c647db543] 2023-09-10T00:55:04.304 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 4: (MDLog::_replay_thread()+0x647) [0x555c64844e97] 2023-09-10T00:55:04.304 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 5: (MDLog::ReplayThread::entry()+0x11) [0x555c6484e381] 2023-09-10T00:55:04.304 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fcb046e11ca] 2023-09-10T00:55:04.305 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 7: clone() 2023-09-10T00:55:04.305 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T00:55:04.305 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[141123]: 2023-09-10T00:55:04.789 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:04 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Main process exited, code=exited, status=139/n/a 2023-09-10T00:55:04.789 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-mon[103713]: pgmap v451: 129 pgs: 129 active+clean; 9.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.4 MiB/s wr, 317 op/s 2023-09-10T00:55:04.789 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:04 smithi131 ceph-mon[103713]: osdmap e109: 12 total, 12 up, 12 in 2023-09-10T00:55:05.045 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:05 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Failed with result 'exit-code'. 2023-09-10T00:55:05.090 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:04 smithi044 ceph-mon[98676]: pgmap v451: 129 pgs: 129 active+clean; 9.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.4 MiB/s wr, 317 op/s 2023-09-10T00:55:05.091 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:04 smithi044 ceph-mon[98676]: osdmap e109: 12 total, 12 up, 12 in 2023-09-10T00:55:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:04 smithi099 ceph-mon[102692]: pgmap v451: 129 pgs: 129 active+clean; 9.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.4 MiB/s wr, 317 op/s 2023-09-10T00:55:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:04 smithi099 ceph-mon[102692]: osdmap e109: 12 total, 12 up, 12 in 2023-09-10T00:55:06.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:05 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:55:06.054 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:05 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:55:06.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:05 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:55:07.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:06 smithi131 ceph-mon[103713]: pgmap v453: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.9 MiB/s wr, 408 op/s 2023-09-10T00:55:07.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:06 smithi044 ceph-mon[98676]: pgmap v453: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.9 MiB/s wr, 408 op/s 2023-09-10T00:55:07.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:06 smithi099 ceph-mon[102692]: pgmap v453: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.9 MiB/s wr, 408 op/s 2023-09-10T00:55:09.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:08 smithi131 ceph-mon[103713]: pgmap v454: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.9 MiB/s wr, 408 op/s 2023-09-10T00:55:09.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:08 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:55:09.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:08 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:55:09.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:08 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:55:09.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:08 smithi044 ceph-mon[98676]: pgmap v454: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.9 MiB/s wr, 408 op/s 2023-09-10T00:55:09.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:08 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:55:09.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:08 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:55:09.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:08 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:55:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:08 smithi099 ceph-mon[102692]: pgmap v454: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.9 MiB/s wr, 408 op/s 2023-09-10T00:55:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:08 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:55:09.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:08 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:55:09.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:08 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:55:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:10 smithi131 ceph-mon[103713]: pgmap v455: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.9 MiB/s wr, 408 op/s 2023-09-10T00:55:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:10 smithi131 ceph-mon[103713]: osdmap e110: 12 total, 12 up, 12 in 2023-09-10T00:55:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:10 smithi044 ceph-mon[98676]: pgmap v455: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.9 MiB/s wr, 408 op/s 2023-09-10T00:55:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:10 smithi044 ceph-mon[98676]: osdmap e110: 12 total, 12 up, 12 in 2023-09-10T00:55:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:10 smithi099 ceph-mon[102692]: pgmap v455: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.9 MiB/s wr, 408 op/s 2023-09-10T00:55:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:10 smithi099 ceph-mon[102692]: osdmap e110: 12 total, 12 up, 12 in 2023-09-10T00:55:12.362 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:55:12.529 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.528+0000 7f6985baa700 1 -- 172.21.15.44:0/3435257991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69801020d0 msgr2=0x7f698010a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:12.529 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.528+0000 7f6985baa700 1 --2- 172.21.15.44:0/3435257991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69801020d0 0x7f698010a5b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6970009990 tx=0x7f6970009ca0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.529 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.528+0000 7f6985baa700 1 -- 172.21.15.44:0/3435257991 shutdown_connections 2023-09-10T00:55:12.529 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.528+0000 7f6985baa700 1 --2- 172.21.15.44:0/3435257991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f698010ab80 0x7f698010cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.528+0000 7f6985baa700 1 --2- 172.21.15.44:0/3435257991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69801017b0 0x7f6980101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.528+0000 7f6985baa700 1 --2- 172.21.15.44:0/3435257991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69801020d0 0x7f698010a5b0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.528+0000 7f6985baa700 1 -- 172.21.15.44:0/3435257991 >> 172.21.15.44:0/3435257991 conn(0x7f69800faf50 msgr2=0x7f69800fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:12.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.528+0000 7f6985baa700 1 -- 172.21.15.44:0/3435257991 shutdown_connections 2023-09-10T00:55:12.530 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.528+0000 7f6985baa700 1 -- 172.21.15.44:0/3435257991 wait complete. 2023-09-10T00:55:12.531 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f6985baa700 1 Processor -- start 2023-09-10T00:55:12.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f6985baa700 1 -- start start 2023-09-10T00:55:12.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f6985baa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69801017b0 0x7f6980196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f6985baa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69801020d0 0x7f6980196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.533 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f6985baa700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f698010ab80 0x7f698019afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f6985baa700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698019b620 con 0x7f69801017b0 2023-09-10T00:55:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f6985baa700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698019b790 con 0x7f69801020d0 2023-09-10T00:55:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f6985baa700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698019b950 con 0x7f698010ab80 2023-09-10T00:55:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f697f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69801017b0 0x7f6980196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f697f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69801017b0 0x7f6980196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53546/0 (socket says 172.21.15.44:53546) 2023-09-10T00:55:12.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f697f7fe700 1 -- 172.21.15.44:0/2244879953 learned_addr learned my addr 172.21.15.44:0/2244879953 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:12.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f697effd700 1 --2- 172.21.15.44:0/2244879953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69801020d0 0x7f6980196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f697ffff700 1 --2- 172.21.15.44:0/2244879953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f698010ab80 0x7f698019afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f697f7fe700 1 -- 172.21.15.44:0/2244879953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f698010ab80 msgr2=0x7f698019afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:12.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f697f7fe700 1 --2- 172.21.15.44:0/2244879953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f698010ab80 0x7f698019afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f697f7fe700 1 -- 172.21.15.44:0/2244879953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69801020d0 msgr2=0x7f6980196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:12.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f697f7fe700 1 --2- 172.21.15.44:0/2244879953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69801020d0 0x7f6980196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.529+0000 7f697f7fe700 1 -- 172.21.15.44:0/2244879953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69700095f0 con 0x7f69801017b0 2023-09-10T00:55:12.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.530+0000 7f697f7fe700 1 --2- 172.21.15.44:0/2244879953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69801017b0 0x7f6980196640 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f696800e9a0 tx=0x7f696800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:12.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.530+0000 7f697cff9700 1 -- 172.21.15.44:0/2244879953 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6968009790 con 0x7f69801017b0 2023-09-10T00:55:12.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.530+0000 7f697cff9700 1 -- 172.21.15.44:0/2244879953 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6968010430 con 0x7f69801017b0 2023-09-10T00:55:12.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.530+0000 7f6985baa700 1 -- 172.21.15.44:0/2244879953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698019bc30 con 0x7f69801017b0 2023-09-10T00:55:12.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.530+0000 7f697cff9700 1 -- 172.21.15.44:0/2244879953 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696800cea0 con 0x7f69801017b0 2023-09-10T00:55:12.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.530+0000 7f6985baa700 1 -- 172.21.15.44:0/2244879953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69801a3470 con 0x7f69801017b0 2023-09-10T00:55:12.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.532+0000 7f697cff9700 1 -- 172.21.15.44:0/2244879953 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f69680105a0 con 0x7f69801017b0 2023-09-10T00:55:12.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.532+0000 7f697cff9700 1 --2- 172.21.15.44:0/2244879953 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f696c0a07b0 0x7f696c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.533+0000 7f697effd700 1 --2- 172.21.15.44:0/2244879953 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f696c0a07b0 0x7f696c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.533+0000 7f697cff9700 1 -- 172.21.15.44:0/2244879953 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f6968014070 con 0x7f69801017b0 2023-09-10T00:55:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.533+0000 7f697effd700 1 --2- 172.21.15.44:0/2244879953 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f696c0a07b0 0x7f696c0a2c70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f69700059d0 tx=0x7f6970005960 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.533+0000 7f6991d70b80 1 Processor -- start 2023-09-10T00:55:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f6991d70b80 1 -- start start 2023-09-10T00:55:12.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f6991d70b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb43710 0x55b7eeb43af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f6991d70b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb44030 0x55b7eeb4d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f6991d70b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb4de40 0x55b7eeb50230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f6991d70b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7eea9c730 con 0x55b7eeb43710 2023-09-10T00:55:12.544 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f6991d70b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7eea9a050 con 0x55b7eeb4de40 2023-09-10T00:55:12.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f6991d70b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7eea95940 con 0x55b7eeb44030 2023-09-10T00:55:12.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f697f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb43710 0x55b7eeb43af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f697ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb4de40 0x55b7eeb50230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f697f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb43710 0x55b7eeb43af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53554/0 (socket says 172.21.15.44:53554) 2023-09-10T00:55:12.545 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f697ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb4de40 0x55b7eeb50230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55296/0 (socket says 172.21.15.44:55296) 2023-09-10T00:55:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f697f7fe700 1 -- 172.21.15.44:0/3310266832 learned_addr learned my addr 172.21.15.44:0/3310266832 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f697f7fe700 1 -- 172.21.15.44:0/3310266832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb44030 msgr2=0x55b7eeb4d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f697f7fe700 1 --2- 172.21.15.44:0/3310266832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb44030 0x55b7eeb4d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f697f7fe700 1 -- 172.21.15.44:0/3310266832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb4de40 msgr2=0x55b7eeb50230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f697f7fe700 1 --2- 172.21.15.44:0/3310266832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb4de40 0x55b7eeb50230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f697f7fe700 1 -- 172.21.15.44:0/3310266832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7ee8c1de0 con 0x55b7eeb43710 2023-09-10T00:55:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f697ffff700 1 --2- 172.21.15.44:0/3310266832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb4de40 0x55b7eeb50230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:55:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.534+0000 7f697f7fe700 1 --2- 172.21.15.44:0/3310266832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb43710 0x55b7eeb43af0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f69680a1130 tx=0x7f6968098f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2846a0cbc8b349ef server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.535+0000 7f6975ffb700 1 -- 172.21.15.44:0/3310266832 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69680995f0 con 0x55b7eeb43710 2023-09-10T00:55:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.535+0000 7f6975ffb700 1 -- 172.21.15.44:0/3310266832 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f696809d420 con 0x55b7eeb43710 2023-09-10T00:55:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.535+0000 7f6975ffb700 1 -- 172.21.15.44:0/3310266832 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69680840b0 con 0x55b7eeb43710 2023-09-10T00:55:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.535+0000 7f6991d70b80 1 -- 172.21.15.44:0/3310266832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb43710 msgr2=0x55b7eeb43af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.535+0000 7f6991d70b80 1 --2- 172.21.15.44:0/3310266832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb43710 0x55b7eeb43af0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f69680a1130 tx=0x7f6968098f50 comp rx=0 tx=0).stop 2023-09-10T00:55:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.535+0000 7f6991d70b80 1 -- 172.21.15.44:0/3310266832 shutdown_connections 2023-09-10T00:55:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.535+0000 7f6991d70b80 1 --2- 172.21.15.44:0/3310266832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb4de40 0x55b7eeb50230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.535+0000 7f6991d70b80 1 --2- 172.21.15.44:0/3310266832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb43710 0x55b7eeb43af0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.535+0000 7f6991d70b80 1 --2- 172.21.15.44:0/3310266832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb44030 0x55b7eeb4d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.535+0000 7f6991d70b80 1 -- 172.21.15.44:0/3310266832 >> 172.21.15.44:0/3310266832 conn(0x55b7eeab8090 msgr2=0x55b7eeb3ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:12.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.535+0000 7f6991d70b80 1 -- 172.21.15.44:0/3310266832 shutdown_connections 2023-09-10T00:55:12.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.535+0000 7f6991d70b80 1 -- 172.21.15.44:0/3310266832 wait complete. 2023-09-10T00:55:12.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.536+0000 7f6991d70b80 1 -- start start 2023-09-10T00:55:12.549 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.536+0000 7f6991d70b80 1 Processor -- start 2023-09-10T00:55:12.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.536+0000 7f6991d70b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb43710 0x55b7eeb4aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.536+0000 7f6991d70b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb44030 0x55b7eeb4b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.536+0000 7f6991d70b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb4de40 0x55b7eeb4b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.536+0000 7f6991d70b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7eea33b70 con 0x55b7eeb44030 2023-09-10T00:55:12.550 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.536+0000 7f6991d70b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7eea33870 con 0x55b7eeb4de40 2023-09-10T00:55:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.536+0000 7f6991d70b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ee917920 con 0x55b7eeb43710 2023-09-10T00:55:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.536+0000 7f6991d70b80 10 client.0 _ll_register_callbacks cb 0x55b7ee97da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:55:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.536+0000 7f697effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb44030 0x55b7eeb4b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.536+0000 7f697f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb43710 0x55b7eeb4aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.536+0000 7f697effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb44030 0x55b7eeb4b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53560/0 (socket says 172.21.15.44:53560) 2023-09-10T00:55:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.536+0000 7f697effd700 1 -- 172.21.15.44:0/1837504425 learned_addr learned my addr 172.21.15.44:0/1837504425 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.536+0000 7f697ffff700 1 --2- 172.21.15.44:0/1837504425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb4de40 0x55b7eeb4b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.537+0000 7f697effd700 1 -- 172.21.15.44:0/1837504425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb43710 msgr2=0x55b7eeb4aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.537+0000 7f697effd700 1 --2- 172.21.15.44:0/1837504425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb43710 0x55b7eeb4aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.537+0000 7f697effd700 1 -- 172.21.15.44:0/1837504425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb4de40 msgr2=0x55b7eeb4b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.537+0000 7f697effd700 1 --2- 172.21.15.44:0/1837504425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb4de40 0x55b7eeb4b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.537+0000 7f697effd700 1 -- 172.21.15.44:0/1837504425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6978011230 con 0x55b7eeb44030 2023-09-10T00:55:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.537+0000 7f697effd700 1 --2- 172.21.15.44:0/1837504425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb44030 0x55b7eeb4b0e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f697000b2e0 tx=0x7f6970009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d606b1e04af8b86c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.537+0000 7f695effd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:55:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.537+0000 7f695effd700 1 -- 172.21.15.44:0/1837504425 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6970004dc0 con 0x55b7eeb44030 2023-09-10T00:55:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.537+0000 7f695effd700 1 -- 172.21.15.44:0/1837504425 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f697000f460 con 0x55b7eeb44030 2023-09-10T00:55:12.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.537+0000 7f6991d70b80 1 -- 172.21.15.44:0/1837504425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7ee9ed4f0 con 0x55b7eeb44030 2023-09-10T00:55:12.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.537+0000 7f695effd700 1 -- 172.21.15.44:0/1837504425 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f697000f620 con 0x55b7eeb44030 2023-09-10T00:55:12.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.537+0000 7f6991d70b80 10 client.17820 fetch_fsmap learned FSMap version 68 2023-09-10T00:55:12.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.537+0000 7f6991d70b80 1 -- 172.21.15.44:0/1837504425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7f69801a3470 con 0x55b7eeb44030 2023-09-10T00:55:12.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.537+0000 7f695effd700 1 -- 172.21.15.44:0/1837504425 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b7ee9ed4f0 con 0x55b7eeb44030 2023-09-10T00:55:12.554 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.538+0000 7f695effd700 1 -- 172.21.15.44:0/1837504425 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 68) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f697002f7a0 con 0x55b7eeb44030 2023-09-10T00:55:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.538+0000 7f6991d70b80 10 client.17820 fetch_fsmap finished waiting for FSMap version 68 2023-09-10T00:55:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.538+0000 7f6991d70b80 10 client.17820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:55:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.538+0000 7f6991d70b80 20 client.17820 populate_metadata read hostname 'smithi044' 2023-09-10T00:55:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.538+0000 7f6991d70b80 1 --2- 172.21.15.44:0/1837504425 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b7eeb5daf0 0x55b7eeb5ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.538+0000 7f6991d70b80 4 client.17820 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:55:12.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.538+0000 7f6991d70b80 1 -- 172.21.15.44:0/1837504425 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7eeaaa8a0 con 0x55b7eeb5daf0 2023-09-10T00:55:12.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.538+0000 7f697f7fe700 1 --2- 172.21.15.44:0/1837504425 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b7eeb5daf0 0x55b7eeb5ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.538+0000 7f695effd700 10 client.17820 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:12.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.538+0000 7f697f7fe700 1 --2- 172.21.15.44:0/1837504425 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b7eeb5daf0 0x55b7eeb5ff10 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b860131477d540 server_cookie=3dbb26ca9cf3da4d in_seq=0 out_seq=0 2023-09-10T00:55:12.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.539+0000 7f695effd700 1 -- 172.21.15.44:0/1837504425 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b7eeaaa8a0 con 0x55b7eeb5daf0 2023-09-10T00:55:12.556 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.539+0000 7f695effd700 10 client.17820 handle_command_reply: tid=0 2023-09-10T00:55:12.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.539+0000 7f6991d70b80 1 client.17820 shutdown 2023-09-10T00:55:12.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.539+0000 7f6991d70b80 1 -- 172.21.15.44:0/1837504425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb44030 msgr2=0x55b7eeb4b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:12.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.539+0000 7f6991d70b80 1 --2- 172.21.15.44:0/1837504425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb44030 0x55b7eeb4b0e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f697000b2e0 tx=0x7f6970009310 comp rx=0 tx=0).stop 2023-09-10T00:55:12.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.539+0000 7f6991d70b80 1 -- 172.21.15.44:0/1837504425 shutdown_connections 2023-09-10T00:55:12.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.539+0000 7f6991d70b80 1 --2- 172.21.15.44:0/1837504425 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b7eeb5daf0 0x55b7eeb5ff10 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.539+0000 7f6991d70b80 1 --2- 172.21.15.44:0/1837504425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb4de40 0x55b7eeb4b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.539+0000 7f6991d70b80 1 --2- 172.21.15.44:0/1837504425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb43710 0x55b7eeb4aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.539+0000 7f6991d70b80 1 --2- 172.21.15.44:0/1837504425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb44030 0x55b7eeb4b0e0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.539+0000 7f6991d70b80 1 -- 172.21.15.44:0/1837504425 >> 172.21.15.44:0/1837504425 conn(0x55b7eeab8090 msgr2=0x55b7eeb3e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:12.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.539+0000 7f695effd700 0 client.17820 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:12.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.539+0000 7f6991d70b80 1 -- 172.21.15.44:0/1837504425 shutdown_connections 2023-09-10T00:55:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.539+0000 7f6991d70b80 1 -- 172.21.15.44:0/1837504425 wait complete. 2023-09-10T00:55:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.539+0000 7f6991d70b80 20 client.17820 trim_cache size 0 max 16384 2023-09-10T00:55:12.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.556+0000 7f6991d70b80 1 Processor -- start 2023-09-10T00:55:12.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.556+0000 7f6991d70b80 1 -- start start 2023-09-10T00:55:12.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f6991d70b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb43710 0x55b7eeb49e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f6991d70b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb44030 0x55b7eeb54980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f6991d70b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb4de40 0x55b7eeb45ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f6991d70b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7eeb40280 con 0x55b7eeb4de40 2023-09-10T00:55:12.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f6991d70b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7eeb38da0 con 0x55b7eeb44030 2023-09-10T00:55:12.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f6991d70b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7eeb5a3d0 con 0x55b7eeb43710 2023-09-10T00:55:12.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f697ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb4de40 0x55b7eeb45ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f697ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb4de40 0x55b7eeb45ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53570/0 (socket says 172.21.15.44:53570) 2023-09-10T00:55:12.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f697ffff700 1 -- 172.21.15.44:0/1140036900 learned_addr learned my addr 172.21.15.44:0/1140036900 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:12.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f697effd700 1 --2- 172.21.15.44:0/1140036900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb44030 0x55b7eeb54980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f697f7fe700 1 --2- 172.21.15.44:0/1140036900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb43710 0x55b7eeb49e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f697ffff700 1 -- 172.21.15.44:0/1140036900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb43710 msgr2=0x55b7eeb49e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:12.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f697ffff700 1 --2- 172.21.15.44:0/1140036900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb43710 0x55b7eeb49e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.567 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f697ffff700 1 -- 172.21.15.44:0/1140036900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb44030 msgr2=0x55b7eeb54980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:12.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f697ffff700 1 --2- 172.21.15.44:0/1140036900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb44030 0x55b7eeb54980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f697ffff700 1 -- 172.21.15.44:0/1140036900 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7ee8aac60 con 0x55b7eeb4de40 2023-09-10T00:55:12.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f697effd700 1 --2- 172.21.15.44:0/1140036900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb44030 0x55b7eeb54980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:55:12.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f697ffff700 1 --2- 172.21.15.44:0/1140036900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb4de40 0x55b7eeb45ee0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6978004070 tx=0x7f6978009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be021060fafd2f21 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:12.568 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f69767fc700 1 -- 172.21.15.44:0/1140036900 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697800e040 con 0x55b7eeb4de40 2023-09-10T00:55:12.569 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f69767fc700 1 -- 172.21.15.44:0/1140036900 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f697800f040 con 0x55b7eeb4de40 2023-09-10T00:55:12.569 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.557+0000 7f69767fc700 1 -- 172.21.15.44:0/1140036900 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978005240 con 0x55b7eeb4de40 2023-09-10T00:55:12.569 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.558+0000 7f6991d70b80 1 -- 172.21.15.44:0/1140036900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb4de40 msgr2=0x55b7eeb45ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:12.569 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.558+0000 7f6991d70b80 1 --2- 172.21.15.44:0/1140036900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb4de40 0x55b7eeb45ee0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6978004070 tx=0x7f6978009920 comp rx=0 tx=0).stop 2023-09-10T00:55:12.569 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.558+0000 7f6991d70b80 1 -- 172.21.15.44:0/1140036900 shutdown_connections 2023-09-10T00:55:12.569 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.558+0000 7f6991d70b80 1 --2- 172.21.15.44:0/1140036900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb4de40 0x55b7eeb45ee0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.570 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.558+0000 7f6991d70b80 1 --2- 172.21.15.44:0/1140036900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb43710 0x55b7eeb49e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.570 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.558+0000 7f6991d70b80 1 --2- 172.21.15.44:0/1140036900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb44030 0x55b7eeb54980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.570 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.558+0000 7f6991d70b80 1 -- 172.21.15.44:0/1140036900 >> 172.21.15.44:0/1140036900 conn(0x55b7eeab8090 msgr2=0x55b7eeb590e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:12.570 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.558+0000 7f6991d70b80 1 -- 172.21.15.44:0/1140036900 shutdown_connections 2023-09-10T00:55:12.570 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.558+0000 7f6991d70b80 1 -- 172.21.15.44:0/1140036900 wait complete. 2023-09-10T00:55:12.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.558+0000 7f6991d70b80 1 -- start start 2023-09-10T00:55:12.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.559+0000 7f6991d70b80 1 Processor -- start 2023-09-10T00:55:12.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.559+0000 7f6991d70b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb43710 0x55b7eeb54520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.559+0000 7f6991d70b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb44030 0x55b7eeb52970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.559+0000 7f6991d70b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb4de40 0x55b7eeb52eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.559+0000 7f6991d70b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7eeb533f0 con 0x55b7eeb44030 2023-09-10T00:55:12.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.559+0000 7f6991d70b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7eeb53560 con 0x55b7eeb43710 2023-09-10T00:55:12.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.559+0000 7f6991d70b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7eeb53720 con 0x55b7eeb4de40 2023-09-10T00:55:12.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.559+0000 7f6991d70b80 10 client.0 _ll_register_callbacks cb 0x7f697000ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:55:12.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.559+0000 7f697effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb44030 0x55b7eeb52970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.559+0000 7f697f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb43710 0x55b7eeb54520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.559+0000 7f697f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb43710 0x55b7eeb54520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55312/0 (socket says 172.21.15.44:55312) 2023-09-10T00:55:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.559+0000 7f697effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb44030 0x55b7eeb52970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53574/0 (socket says 172.21.15.44:53574) 2023-09-10T00:55:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.559+0000 7f697f7fe700 1 -- 172.21.15.44:0/3077904674 learned_addr learned my addr 172.21.15.44:0/3077904674 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.559+0000 7f697ffff700 1 --2- 172.21.15.44:0/3077904674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb4de40 0x55b7eeb52eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.560+0000 7f697f7fe700 1 -- 172.21.15.44:0/3077904674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb4de40 msgr2=0x55b7eeb52eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.560+0000 7f697f7fe700 1 --2- 172.21.15.44:0/3077904674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb4de40 0x55b7eeb52eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.560+0000 7f697f7fe700 1 -- 172.21.15.44:0/3077904674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb44030 msgr2=0x55b7eeb52970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.560+0000 7f697f7fe700 1 --2- 172.21.15.44:0/3077904674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb44030 0x55b7eeb52970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.560+0000 7f697f7fe700 1 -- 172.21.15.44:0/3077904674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69801a3470 con 0x55b7eeb43710 2023-09-10T00:55:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.560+0000 7f697effd700 1 --2- 172.21.15.44:0/3077904674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb44030 0x55b7eeb52970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:55:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.560+0000 7f697f7fe700 1 --2- 172.21.15.44:0/3077904674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb43710 0x55b7eeb54520 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f69680c9940 tx=0x7f6968010b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f728214efab1d73 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.560+0000 7f695effd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:55:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.560+0000 7f695effd700 1 -- 172.21.15.44:0/3077904674 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69680995f0 con 0x55b7eeb43710 2023-09-10T00:55:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.560+0000 7f6991d70b80 1 -- 172.21.15.44:0/3077904674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f696809d420 con 0x55b7eeb43710 2023-09-10T00:55:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.560+0000 7f695effd700 1 -- 172.21.15.44:0/3077904674 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f69680a1660 con 0x55b7eeb43710 2023-09-10T00:55:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.560+0000 7f695effd700 1 -- 172.21.15.44:0/3077904674 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6968097990 con 0x55b7eeb43710 2023-09-10T00:55:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.560+0000 7f6991d70b80 10 client.27488 fetch_fsmap learned FSMap version 68 2023-09-10T00:55:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.560+0000 7f6991d70b80 1 -- 172.21.15.44:0/3077904674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7f6978003cd0 con 0x55b7eeb43710 2023-09-10T00:55:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.560+0000 7f695effd700 1 -- 172.21.15.44:0/3077904674 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f696809d420 con 0x55b7eeb43710 2023-09-10T00:55:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.561+0000 7f695effd700 1 -- 172.21.15.44:0/3077904674 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 68) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f69680cc1b0 con 0x55b7eeb43710 2023-09-10T00:55:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.561+0000 7f6991d70b80 10 client.27488 fetch_fsmap finished waiting for FSMap version 68 2023-09-10T00:55:12.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.561+0000 7f6991d70b80 10 client.27488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:55:12.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.561+0000 7f6991d70b80 20 client.27488 populate_metadata read hostname 'smithi044' 2023-09-10T00:55:12.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.561+0000 7f6991d70b80 1 --2- 172.21.15.44:0/3077904674 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b7eebb2a30 0x55b7eebb4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:12.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.561+0000 7f6991d70b80 4 client.27488 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:55:12.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.561+0000 7f6991d70b80 1 -- 172.21.15.44:0/3077904674 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b7eea8e920 con 0x55b7eebb2a30 2023-09-10T00:55:12.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.561+0000 7f697effd700 1 --2- 172.21.15.44:0/3077904674 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b7eebb2a30 0x55b7eebb4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:12.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.561+0000 7f697effd700 1 --2- 172.21.15.44:0/3077904674 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b7eebb2a30 0x55b7eebb4e50 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85a550cfba25df1c server_cookie=dfb4ee52c044d2f3 in_seq=0 out_seq=0 2023-09-10T00:55:12.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.561+0000 7f695effd700 10 client.27488 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:12.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.561+0000 7f695effd700 1 -- 172.21.15.44:0/3077904674 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b7eea8e920 con 0x55b7eebb2a30 2023-09-10T00:55:12.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.561+0000 7f695effd700 10 client.27488 handle_command_reply: tid=0 2023-09-10T00:55:12.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.561+0000 7f6991d70b80 1 client.27488 shutdown 2023-09-10T00:55:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6991d70b80 1 -- 172.21.15.44:0/3077904674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb43710 msgr2=0x55b7eeb54520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6991d70b80 1 --2- 172.21.15.44:0/3077904674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb43710 0x55b7eeb54520 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f69680c9940 tx=0x7f6968010b80 comp rx=0 tx=0).stop 2023-09-10T00:55:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6991d70b80 1 -- 172.21.15.44:0/3077904674 shutdown_connections 2023-09-10T00:55:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6991d70b80 1 --2- 172.21.15.44:0/3077904674 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b7eebb2a30 0x55b7eebb4e50 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6991d70b80 1 --2- 172.21.15.44:0/3077904674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7eeb4de40 0x55b7eeb52eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6991d70b80 1 --2- 172.21.15.44:0/3077904674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7eeb43710 0x55b7eeb54520 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f695effd700 0 client.27488 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:12.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6991d70b80 1 --2- 172.21.15.44:0/3077904674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b7eeb44030 0x55b7eeb52970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6991d70b80 1 -- 172.21.15.44:0/3077904674 >> 172.21.15.44:0/3077904674 conn(0x55b7eeab8090 msgr2=0x55b7eeb45ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:12.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6991d70b80 1 -- 172.21.15.44:0/3077904674 shutdown_connections 2023-09-10T00:55:12.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6991d70b80 1 -- 172.21.15.44:0/3077904674 wait complete. 2023-09-10T00:55:12.581 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6991d70b80 20 client.27488 trim_cache size 0 max 16384 2023-09-10T00:55:12.581 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:55:12.581 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:55:12.581 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:55:12.582 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:55:12.582 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:55:12.582 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:55:12.582 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:55:12.582 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:55:12.583 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:55:12.583 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:55:12.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6985baa700 1 -- 172.21.15.44:0/2244879953 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f696c0a07b0 msgr2=0x7f696c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:12.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6985baa700 1 --2- 172.21.15.44:0/2244879953 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f696c0a07b0 0x7f696c0a2c70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f69700059d0 tx=0x7f6970005960 comp rx=0 tx=0).stop 2023-09-10T00:55:12.584 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6985baa700 1 -- 172.21.15.44:0/2244879953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69801017b0 msgr2=0x7f6980196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:12.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6985baa700 1 --2- 172.21.15.44:0/2244879953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69801017b0 0x7f6980196640 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f696800e9a0 tx=0x7f696800ecb0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6985baa700 1 -- 172.21.15.44:0/2244879953 shutdown_connections 2023-09-10T00:55:12.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6985baa700 1 --2- 172.21.15.44:0/2244879953 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f696c0a07b0 0x7f696c0a2c70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6985baa700 1 --2- 172.21.15.44:0/2244879953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f698010ab80 0x7f698019afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.585 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6985baa700 1 --2- 172.21.15.44:0/2244879953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69801017b0 0x7f6980196640 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.586 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6985baa700 1 --2- 172.21.15.44:0/2244879953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69801020d0 0x7f6980196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:12.586 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6985baa700 1 -- 172.21.15.44:0/2244879953 >> 172.21.15.44:0/2244879953 conn(0x7f69800faf50 msgr2=0x7f69800fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:12.586 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.562+0000 7f6985baa700 1 -- 172.21.15.44:0/2244879953 shutdown_connections 2023-09-10T00:55:12.586 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:12.563+0000 7f6985baa700 1 -- 172.21.15.44:0/2244879953 wait complete. 2023-09-10T00:55:12.589 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:55:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:12 smithi131 ceph-mon[103713]: pgmap v457: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 468 op/s 2023-09-10T00:55:13.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:12 smithi044 ceph-mon[98676]: pgmap v457: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 468 op/s 2023-09-10T00:55:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:12 smithi099 ceph-mon[102692]: pgmap v457: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 468 op/s 2023-09-10T00:55:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:14 smithi131 ceph-mon[103713]: pgmap v458: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 394 op/s 2023-09-10T00:55:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:14 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:14 smithi044 ceph-mon[98676]: pgmap v458: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 394 op/s 2023-09-10T00:55:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:14 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:14 smithi099 ceph-mon[102692]: pgmap v458: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 394 op/s 2023-09-10T00:55:15.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:14 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:15.338 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:15 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Service RestartSec=10s expired, scheduling restart. 2023-09-10T00:55:15.338 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:15 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Scheduled restart job, restart counter is at 1. 2023-09-10T00:55:15.339 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:15 smithi131 systemd[1]: Stopped Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:55:15.339 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:15 smithi131 systemd[1]: Starting Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:55:15.597 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:15 smithi131 podman[147759]: 2023-09-10T00:55:15.598 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:15 smithi131 ceph-mds[147799]: -- 172.21.15.131:0/3032596675 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b54666d1e0 con 0x55b546642800 2023-09-10T00:55:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:15 smithi131 ceph-mon[103713]: daemon mds.c restarted 2023-09-10T00:55:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:15 smithi131 ceph-mon[103713]: osdmap e111: 12 total, 12 up, 12 in 2023-09-10T00:55:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:15 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.131:6836/2983505322,v1:172.21.15.131:6837/2983505322] up:boot 2023-09-10T00:55:16.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:15 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:16.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:15 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T00:55:16.046 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:15 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: starting mds.c at 2023-09-10T00:55:16.046 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:15 smithi131 bash[147759]: 75ff68de96dae02f776af52656d80793cb4d427979b1bc4692fc85e143f6c193 2023-09-10T00:55:16.047 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:55:15 smithi131 systemd[1]: Started Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:55:16.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:15 smithi044 ceph-mon[98676]: daemon mds.c restarted 2023-09-10T00:55:16.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:15 smithi044 ceph-mon[98676]: osdmap e111: 12 total, 12 up, 12 in 2023-09-10T00:55:16.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:15 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.131:6836/2983505322,v1:172.21.15.131:6837/2983505322] up:boot 2023-09-10T00:55:16.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:15 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:16.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:15 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T00:55:16.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:15 smithi099 ceph-mon[102692]: daemon mds.c restarted 2023-09-10T00:55:16.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:15 smithi099 ceph-mon[102692]: osdmap e111: 12 total, 12 up, 12 in 2023-09-10T00:55:16.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:15 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.131:6836/2983505322,v1:172.21.15.131:6837/2983505322] up:boot 2023-09-10T00:55:16.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:15 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:16.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:15 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T00:55:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.808+0000 7f40e2f0b700 1 -- 172.21.15.44:0/3334228980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40dc105560 msgr2=0x7f40dc105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.808+0000 7f40e2f0b700 1 --2- 172.21.15.44:0/3334228980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40dc105560 0x7f40dc105940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009920 tx=0x7f40d0009c30 comp rx=0 tx=0).stop 2023-09-10T00:55:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.808+0000 7f40e2f0b700 1 -- 172.21.15.44:0/3334228980 shutdown_connections 2023-09-10T00:55:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.808+0000 7f40e2f0b700 1 --2- 172.21.15.44:0/3334228980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40dc100050 0x7f40dc10c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.808+0000 7f40e2f0b700 1 --2- 172.21.15.44:0/3334228980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40dc105560 0x7f40dc105940 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.808+0000 7f40e2f0b700 1 --2- 172.21.15.44:0/3334228980 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40dc0ff690 0x7f40dc0ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.808+0000 7f40e2f0b700 1 -- 172.21.15.44:0/3334228980 >> 172.21.15.44:0/3334228980 conn(0x7f40dc0773d0 msgr2=0x7f40dc0777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.808+0000 7f40e2f0b700 1 -- 172.21.15.44:0/3334228980 shutdown_connections 2023-09-10T00:55:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.808+0000 7f40e2f0b700 1 -- 172.21.15.44:0/3334228980 wait complete. 2023-09-10T00:55:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.809+0000 7f40e2f0b700 1 Processor -- start 2023-09-10T00:55:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.809+0000 7f40e2f0b700 1 -- start start 2023-09-10T00:55:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.809+0000 7f40e2f0b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40dc0ff690 0x7f40dc19c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.809+0000 7f40e2f0b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40dc100050 0x7f40dc19cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.809+0000 7f40e2f0b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40dc105560 0x7f40dc196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.809+0000 7f40e2f0b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc196ad0 con 0x7f40dc105560 2023-09-10T00:55:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.809+0000 7f40e2f0b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc196c40 con 0x7f40dc100050 2023-09-10T00:55:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.809+0000 7f40e2f0b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc196e00 con 0x7f40dc0ff690 2023-09-10T00:55:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.809+0000 7f40e0ca7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40dc0ff690 0x7f40dc19c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.809+0000 7f40e0ca7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40dc0ff690 0x7f40dc19c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60468/0 (socket says 172.21.15.44:60468) 2023-09-10T00:55:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.809+0000 7f40e0ca7700 1 -- 172.21.15.44:0/1369349679 learned_addr learned my addr 172.21.15.44:0/1369349679 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.809+0000 7f40dbfff700 1 --2- 172.21.15.44:0/1369349679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40dc100050 0x7f40dc19cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.809+0000 7f40e14a8700 1 --2- 172.21.15.44:0/1369349679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40dc105560 0x7f40dc196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.810+0000 7f40e0ca7700 1 -- 172.21.15.44:0/1369349679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40dc100050 msgr2=0x7f40dc19cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.810+0000 7f40e0ca7700 1 --2- 172.21.15.44:0/1369349679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40dc100050 0x7f40dc19cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.810+0000 7f40e0ca7700 1 -- 172.21.15.44:0/1369349679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40dc105560 msgr2=0x7f40dc196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.810+0000 7f40e0ca7700 1 --2- 172.21.15.44:0/1369349679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40dc105560 0x7f40dc196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.810+0000 7f40e0ca7700 1 -- 172.21.15.44:0/1369349679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d4009d00 con 0x7f40dc0ff690 2023-09-10T00:55:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.810+0000 7f40e0ca7700 1 --2- 172.21.15.44:0/1369349679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40dc0ff690 0x7f40dc19c600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009230 tx=0x7f40d000f9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.810+0000 7f40d9ffb700 1 -- 172.21.15.44:0/1369349679 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d001c070 con 0x7f40dc0ff690 2023-09-10T00:55:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.810+0000 7f40e2f0b700 1 -- 172.21.15.44:0/1369349679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d0009520 con 0x7f40dc0ff690 2023-09-10T00:55:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.810+0000 7f40d9ffb700 1 -- 172.21.15.44:0/1369349679 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40d000ba70 con 0x7f40dc0ff690 2023-09-10T00:55:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.810+0000 7f40d9ffb700 1 -- 172.21.15.44:0/1369349679 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d00038e0 con 0x7f40dc0ff690 2023-09-10T00:55:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.810+0000 7f40e2f0b700 1 -- 172.21.15.44:0/1369349679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40dc101040 con 0x7f40dc0ff690 2023-09-10T00:55:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.813+0000 7f40d9ffb700 1 -- 172.21.15.44:0/1369349679 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f40d0020030 con 0x7f40dc0ff690 2023-09-10T00:55:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.814+0000 7f40d9ffb700 1 --2- 172.21.15.44:0/1369349679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f40cc0a07b0 0x7f40cc0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.814+0000 7f40dbfff700 1 --2- 172.21.15.44:0/1369349679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f40cc0a07b0 0x7f40cc0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.814+0000 7f40d9ffb700 1 -- 172.21.15.44:0/1369349679 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f40d00cd1f0 con 0x7f40dc0ff690 2023-09-10T00:55:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.814+0000 7f40dbfff700 1 --2- 172.21.15.44:0/1369349679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f40cc0a07b0 0x7f40cc0a2c70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f40dc104fa0 tx=0x7f40c8009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.814+0000 7f40ef0d1b80 1 Processor -- start 2023-09-10T00:55:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.814+0000 7f40ef0d1b80 1 -- start start 2023-09-10T00:55:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40ef0d1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd01021a0 0x561bd0102580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40ef0d1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fc300 0x561bd00fc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40ef0d1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd00fccc0 0x561bd0108f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40ef0d1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd00556d0 con 0x561bd01021a0 2023-09-10T00:55:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40ef0d1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd0052ff0 con 0x561bd00fccc0 2023-09-10T00:55:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40ef0d1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd004e8e0 con 0x561bd00fc300 2023-09-10T00:55:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40e0ca7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd01021a0 0x561bd0102580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40e14a8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd00fccc0 0x561bd0108f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40e0ca7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd01021a0 0x561bd0102580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53594/0 (socket says 172.21.15.44:53594) 2023-09-10T00:55:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40e0ca7700 1 -- 172.21.15.44:0/3037354174 learned_addr learned my addr 172.21.15.44:0/3037354174 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40dbfff700 1 --2- 172.21.15.44:0/3037354174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fc300 0x561bd00fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40e14a8700 1 -- 172.21.15.44:0/3037354174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fc300 msgr2=0x561bd00fc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40e14a8700 1 --2- 172.21.15.44:0/3037354174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fc300 0x561bd00fc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40e14a8700 1 -- 172.21.15.44:0/3037354174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd01021a0 msgr2=0x561bd0102580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40e14a8700 1 --2- 172.21.15.44:0/3037354174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd01021a0 0x561bd0102580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40e14a8700 1 -- 172.21.15.44:0/3037354174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bcfe7ad80 con 0x561bd00fccc0 2023-09-10T00:55:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.815+0000 7f40e14a8700 1 --2- 172.21.15.44:0/3037354174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd00fccc0 0x561bd0108f30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009fd0 tx=0x7f40d40099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b44c3a231fd2282 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.816+0000 7f40c6ffd700 1 -- 172.21.15.44:0/3037354174 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4004bb0 con 0x561bd00fccc0 2023-09-10T00:55:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.816+0000 7f40c6ffd700 1 -- 172.21.15.44:0/3037354174 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40d400f040 con 0x561bd00fccc0 2023-09-10T00:55:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.816+0000 7f40c6ffd700 1 -- 172.21.15.44:0/3037354174 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4005020 con 0x561bd00fccc0 2023-09-10T00:55:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.817+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/3037354174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd00fccc0 msgr2=0x561bd0108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.817+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/3037354174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd00fccc0 0x561bd0108f30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009fd0 tx=0x7f40d40099e0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.817+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/3037354174 shutdown_connections 2023-09-10T00:55:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.817+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/3037354174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd00fccc0 0x561bd0108f30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.817+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/3037354174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd01021a0 0x561bd0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.817+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/3037354174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fc300 0x561bd00fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.817+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/3037354174 >> 172.21.15.44:0/3037354174 conn(0x561bd0071030 msgr2=0x561bd00f7560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.817+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/3037354174 shutdown_connections 2023-09-10T00:55:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.817+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/3037354174 wait complete. 2023-09-10T00:55:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.817+0000 7f40ef0d1b80 1 -- start start 2023-09-10T00:55:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.817+0000 7f40ef0d1b80 1 Processor -- start 2023-09-10T00:55:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40ef0d1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fc300 0x561bd0108d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40ef0d1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd00fccc0 0x561bd01060f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40ef0d1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd01021a0 0x561bd0106670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40ef0d1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcffecb10 con 0x561bd01021a0 2023-09-10T00:55:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40ef0d1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcffec810 con 0x561bd00fccc0 2023-09-10T00:55:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40ef0d1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcfed08c0 con 0x561bd00fc300 2023-09-10T00:55:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40ef0d1b80 10 client.0 _ll_register_callbacks cb 0x561bcff36a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:55:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40e0ca7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fc300 0x561bd0108d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40dbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd00fccc0 0x561bd01060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40e0ca7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fc300 0x561bd0108d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60482/0 (socket says 172.21.15.44:60482) 2023-09-10T00:55:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40e0ca7700 1 -- 172.21.15.44:0/2145085022 learned_addr learned my addr 172.21.15.44:0/2145085022 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40e14a8700 1 --2- 172.21.15.44:0/2145085022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd01021a0 0x561bd0106670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40e0ca7700 1 -- 172.21.15.44:0/2145085022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd00fccc0 msgr2=0x561bd01060f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40e0ca7700 1 --2- 172.21.15.44:0/2145085022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd00fccc0 0x561bd01060f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40e0ca7700 1 -- 172.21.15.44:0/2145085022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd01021a0 msgr2=0x561bd0106670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40e0ca7700 1 --2- 172.21.15.44:0/2145085022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd01021a0 0x561bd0106670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.818+0000 7f40e0ca7700 1 -- 172.21.15.44:0/2145085022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d4009140 con 0x561bd00fc300 2023-09-10T00:55:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.819+0000 7f40e0ca7700 1 --2- 172.21.15.44:0/2145085022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fc300 0x561bd0108d60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f40d0017040 tx=0x7f40d009a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3a86135d86e2535 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.819+0000 7f40bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:55:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.819+0000 7f40bbfff700 1 -- 172.21.15.44:0/2145085022 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d0085070 con 0x561bd00fc300 2023-09-10T00:55:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.819+0000 7f40bbfff700 1 -- 172.21.15.44:0/2145085022 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f40d009aa90 con 0x561bd00fc300 2023-09-10T00:55:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.819+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/2145085022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bcffa6490 con 0x561bd00fc300 2023-09-10T00:55:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.819+0000 7f40bbfff700 1 -- 172.21.15.44:0/2145085022 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40d00cd070 con 0x561bd00fc300 2023-09-10T00:55:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.819+0000 7f40ef0d1b80 10 client.27439 fetch_fsmap learned FSMap version 70 2023-09-10T00:55:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.819+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/2145085022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x561bcfe63c00 con 0x561bd00fc300 2023-09-10T00:55:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.819+0000 7f40bbfff700 1 -- 172.21.15.44:0/2145085022 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40d00b6080 con 0x561bd00fc300 2023-09-10T00:55:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.819+0000 7f40bbfff700 1 -- 172.21.15.44:0/2145085022 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 70) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f40d0096aa0 con 0x561bd00fc300 2023-09-10T00:55:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.819+0000 7f40ef0d1b80 10 client.27439 fetch_fsmap finished waiting for FSMap version 70 2023-09-10T00:55:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.819+0000 7f40ef0d1b80 10 client.27439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:55:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.819+0000 7f40ef0d1b80 20 client.27439 populate_metadata read hostname 'smithi044' 2023-09-10T00:55:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.819+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/2145085022 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561bd01160d0 0x561bd01184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.819+0000 7f40ef0d1b80 4 client.27439 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:55:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.819+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/2145085022 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bd0063840 con 0x561bd01160d0 2023-09-10T00:55:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.820+0000 7f40dbfff700 1 --2- 172.21.15.44:0/2145085022 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561bd01160d0 0x561bd01184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.820+0000 7f40bbfff700 10 client.27439 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.820+0000 7f40dbfff700 1 --2- 172.21.15.44:0/2145085022 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561bd01160d0 0x561bd01184f0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b257e1ec8621886 server_cookie=fa71de16a993caae in_seq=0 out_seq=0 2023-09-10T00:55:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.821+0000 7f40bbfff700 1 -- 172.21.15.44:0/2145085022 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561bd0063840 con 0x561bd01160d0 2023-09-10T00:55:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.821+0000 7f40bbfff700 10 client.27439 handle_command_reply: tid=0 2023-09-10T00:55:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.821+0000 7f40ef0d1b80 1 client.27439 shutdown 2023-09-10T00:55:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.821+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/2145085022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fc300 msgr2=0x561bd0108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.821+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/2145085022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fc300 0x561bd0108d60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f40d0017040 tx=0x7f40d009a1e0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.821+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/2145085022 shutdown_connections 2023-09-10T00:55:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.821+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/2145085022 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561bd01160d0 0x561bd01184f0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.821+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/2145085022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd01021a0 0x561bd0106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.821+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/2145085022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fc300 0x561bd0108d60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.821+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/2145085022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd00fccc0 0x561bd01060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.821+0000 7f40bbfff700 0 client.27439 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.821+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/2145085022 >> 172.21.15.44:0/2145085022 conn(0x561bd0071030 msgr2=0x561bd00f67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.821+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/2145085022 shutdown_connections 2023-09-10T00:55:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.821+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/2145085022 wait complete. 2023-09-10T00:55:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.822+0000 7f40ef0d1b80 20 client.27439 trim_cache size 0 max 16384 2023-09-10T00:55:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.842+0000 7f40ef0d1b80 1 Processor -- start 2023-09-10T00:55:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.842+0000 7f40ef0d1b80 1 -- start start 2023-09-10T00:55:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.842+0000 7f40ef0d1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd00fc300 0x561bd0112a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.842+0000 7f40ef0d1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd00fccc0 0x561bd010d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.842+0000 7f40ef0d1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd01021a0 0x561bd0108890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.842+0000 7f40ef0d1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd00ffdb0 con 0x561bd00fc300 2023-09-10T00:55:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.842+0000 7f40ef0d1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd00fd580 con 0x561bd00fccc0 2023-09-10T00:55:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.842+0000 7f40ef0d1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd0107660 con 0x561bd01021a0 2023-09-10T00:55:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.842+0000 7f40e0ca7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd00fc300 0x561bd0112a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.842+0000 7f40e14a8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd01021a0 0x561bd0108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.842+0000 7f40e0ca7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd00fc300 0x561bd0112a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53608/0 (socket says 172.21.15.44:53608) 2023-09-10T00:55:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.842+0000 7f40e0ca7700 1 -- 172.21.15.44:0/1873446264 learned_addr learned my addr 172.21.15.44:0/1873446264 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.843+0000 7f40e14a8700 1 -- 172.21.15.44:0/1873446264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd00fccc0 msgr2=0x561bd010d530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T00:55:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.843+0000 7f40e14a8700 1 --2- 172.21.15.44:0/1873446264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd00fccc0 0x561bd010d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.843+0000 7f40e14a8700 1 -- 172.21.15.44:0/1873446264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd00fc300 msgr2=0x561bd0112a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.843+0000 7f40e14a8700 1 --2- 172.21.15.44:0/1873446264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd00fc300 0x561bd0112a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.843+0000 7f40e14a8700 1 -- 172.21.15.44:0/1873446264 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bcfea11f0 con 0x561bd01021a0 2023-09-10T00:55:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.843+0000 7f40e14a8700 1 --2- 172.21.15.44:0/1873446264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd01021a0 0x561bd0108890 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009ca0 tx=0x7f40d4006040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5896e71053dd9dbd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.843+0000 7f40c77fe700 1 -- 172.21.15.44:0/1873446264 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4004b10 con 0x561bd01021a0 2023-09-10T00:55:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.843+0000 7f40c77fe700 1 -- 172.21.15.44:0/1873446264 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40d4004c70 con 0x561bd01021a0 2023-09-10T00:55:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.843+0000 7f40c77fe700 1 -- 172.21.15.44:0/1873446264 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4017460 con 0x561bd01021a0 2023-09-10T00:55:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.844+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/1873446264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd01021a0 msgr2=0x561bd0108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.844+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/1873446264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd01021a0 0x561bd0108890 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009ca0 tx=0x7f40d4006040 comp rx=0 tx=0).stop 2023-09-10T00:55:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.844+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/1873446264 shutdown_connections 2023-09-10T00:55:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.844+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/1873446264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd01021a0 0x561bd0108890 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.844+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/1873446264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd00fc300 0x561bd0112a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.844+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/1873446264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd00fccc0 0x561bd010d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.844+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/1873446264 >> 172.21.15.44:0/1873446264 conn(0x561bd0071030 msgr2=0x561bd00f7090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.844+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/1873446264 shutdown_connections 2023-09-10T00:55:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.844+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/1873446264 wait complete. 2023-09-10T00:55:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.844+0000 7f40ef0d1b80 1 -- start start 2023-09-10T00:55:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.845+0000 7f40ef0d1b80 1 Processor -- start 2023-09-10T00:55:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.845+0000 7f40ef0d1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd00fc300 0x561bd0163440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.845+0000 7f40ef0d1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fccc0 0x561bd0163980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.845+0000 7f40ef0d1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd01021a0 0x561bd015de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.845+0000 7f40ef0d1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd015e5a0 con 0x561bd00fc300 2023-09-10T00:55:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.845+0000 7f40ef0d1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd015e710 con 0x561bd01021a0 2023-09-10T00:55:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.845+0000 7f40ef0d1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd015e8d0 con 0x561bd00fccc0 2023-09-10T00:55:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.845+0000 7f40ef0d1b80 10 client.0 _ll_register_callbacks cb 0x7f40d00966d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:55:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.845+0000 7f40e0ca7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd00fc300 0x561bd0163440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.845+0000 7f40dbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fccc0 0x561bd0163980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.845+0000 7f40e0ca7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd00fc300 0x561bd0163440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53614/0 (socket says 172.21.15.44:53614) 2023-09-10T00:55:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.845+0000 7f40e0ca7700 1 -- 172.21.15.44:0/1316714006 learned_addr learned my addr 172.21.15.44:0/1316714006 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.845+0000 7f40e14a8700 1 --2- 172.21.15.44:0/1316714006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd01021a0 0x561bd015de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.846+0000 7f40e0ca7700 1 -- 172.21.15.44:0/1316714006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fccc0 msgr2=0x561bd0163980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.846+0000 7f40e0ca7700 1 --2- 172.21.15.44:0/1316714006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fccc0 0x561bd0163980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.846+0000 7f40e0ca7700 1 -- 172.21.15.44:0/1316714006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd01021a0 msgr2=0x561bd015de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.846+0000 7f40e0ca7700 1 --2- 172.21.15.44:0/1316714006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd01021a0 0x561bd015de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.846+0000 7f40e0ca7700 1 -- 172.21.15.44:0/1316714006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bcfe7ad80 con 0x561bd00fc300 2023-09-10T00:55:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.846+0000 7f40dbfff700 1 --2- 172.21.15.44:0/1316714006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fccc0 0x561bd0163980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:55:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.846+0000 7f40e0ca7700 1 --2- 172.21.15.44:0/1316714006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd00fc300 0x561bd0163440 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f40d00caba0 tx=0x7f40d0005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf2958480a26525 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.846+0000 7f40bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:55:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.846+0000 7f40bbfff700 1 -- 172.21.15.44:0/1316714006 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d000bbe0 con 0x561bd00fc300 2023-09-10T00:55:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.846+0000 7f40bbfff700 1 -- 172.21.15.44:0/1316714006 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f40d009aa90 con 0x561bd00fc300 2023-09-10T00:55:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.846+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/1316714006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40d400f040 con 0x561bd00fc300 2023-09-10T00:55:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.846+0000 7f40bbfff700 1 -- 172.21.15.44:0/1316714006 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40d0099ed0 con 0x561bd00fc300 2023-09-10T00:55:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.846+0000 7f40ef0d1b80 10 client.17850 fetch_fsmap learned FSMap version 70 2023-09-10T00:55:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.846+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/1316714006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x561bcfe79250 con 0x561bd00fc300 2023-09-10T00:55:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.847+0000 7f40bbfff700 1 -- 172.21.15.44:0/1316714006 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40d400f040 con 0x561bd00fc300 2023-09-10T00:55:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.847+0000 7f40bbfff700 1 -- 172.21.15.44:0/1316714006 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 70) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f40d00a00b0 con 0x561bd00fc300 2023-09-10T00:55:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.847+0000 7f40ef0d1b80 10 client.17850 fetch_fsmap finished waiting for FSMap version 70 2023-09-10T00:55:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.847+0000 7f40ef0d1b80 10 client.17850 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:55:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.847+0000 7f40ef0d1b80 20 client.17850 populate_metadata read hostname 'smithi044' 2023-09-10T00:55:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.847+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/1316714006 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561bd016acd0 0x561bd016d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.847+0000 7f40ef0d1b80 4 client.17850 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:55:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.847+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/1316714006 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561bd00478c0 con 0x561bd016acd0 2023-09-10T00:55:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.847+0000 7f40dbfff700 1 --2- 172.21.15.44:0/1316714006 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561bd016acd0 0x561bd016d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.847+0000 7f40dbfff700 1 --2- 172.21.15.44:0/1316714006 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561bd016acd0 0x561bd016d0c0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a85c0e3bc5edb2d server_cookie=bf5cb3b6c0c56961 in_seq=0 out_seq=0 2023-09-10T00:55:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.847+0000 7f40bbfff700 10 client.17850 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.860+0000 7f40bbfff700 1 -- 172.21.15.44:0/1316714006 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561bd00478c0 con 0x561bd016acd0 2023-09-10T00:55:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.860+0000 7f40bbfff700 10 client.17850 handle_command_reply: tid=0 2023-09-10T00:55:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.860+0000 7f40ef0d1b80 1 client.17850 shutdown 2023-09-10T00:55:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.860+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/1316714006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd00fc300 msgr2=0x561bd0163440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.860+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/1316714006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd00fc300 0x561bd0163440 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f40d00caba0 tx=0x7f40d0005b40 comp rx=0 tx=0).stop 2023-09-10T00:55:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.860+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/1316714006 shutdown_connections 2023-09-10T00:55:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.860+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/1316714006 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561bd016acd0 0x561bd016d0c0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.860+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/1316714006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd01021a0 0x561bd015de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.860+0000 7f40bbfff700 0 client.17850 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.860+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/1316714006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561bd00fc300 0x561bd0163440 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.860+0000 7f40ef0d1b80 1 --2- 172.21.15.44:0/1316714006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bd00fccc0 0x561bd0163980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.860+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/1316714006 >> 172.21.15.44:0/1316714006 conn(0x561bd0071030 msgr2=0x561bd00f5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.860+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/1316714006 shutdown_connections 2023-09-10T00:55:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.860+0000 7f40ef0d1b80 1 -- 172.21.15.44:0/1316714006 wait complete. 2023-09-10T00:55:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.860+0000 7f40ef0d1b80 20 client.17850 trim_cache size 0 max 16384 2023-09-10T00:55:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:55:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:55:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:55:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:55:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.861+0000 7f40e2f0b700 1 -- 172.21.15.44:0/1369349679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f40cc0a07b0 msgr2=0x7f40cc0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.861+0000 7f40e2f0b700 1 --2- 172.21.15.44:0/1369349679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f40cc0a07b0 0x7f40cc0a2c70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f40dc104fa0 tx=0x7f40c8009260 comp rx=0 tx=0).stop 2023-09-10T00:55:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.861+0000 7f40e2f0b700 1 -- 172.21.15.44:0/1369349679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40dc0ff690 msgr2=0x7f40dc19c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.861+0000 7f40e2f0b700 1 --2- 172.21.15.44:0/1369349679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40dc0ff690 0x7f40dc19c600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009230 tx=0x7f40d000f9e0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.861+0000 7f40e2f0b700 1 -- 172.21.15.44:0/1369349679 shutdown_connections 2023-09-10T00:55:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.861+0000 7f40e2f0b700 1 --2- 172.21.15.44:0/1369349679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f40cc0a07b0 0x7f40cc0a2c70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.861+0000 7f40e2f0b700 1 --2- 172.21.15.44:0/1369349679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40dc105560 0x7f40dc196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.861+0000 7f40e2f0b700 1 --2- 172.21.15.44:0/1369349679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40dc0ff690 0x7f40dc19c600 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.861+0000 7f40e2f0b700 1 --2- 172.21.15.44:0/1369349679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40dc100050 0x7f40dc19cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.861+0000 7f40e2f0b700 1 -- 172.21.15.44:0/1369349679 >> 172.21.15.44:0/1369349679 conn(0x7f40dc0773d0 msgr2=0x7f40dc0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.861+0000 7f40e2f0b700 1 -- 172.21.15.44:0/1369349679 shutdown_connections 2023-09-10T00:55:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:16.861+0000 7f40e2f0b700 1 -- 172.21.15.44:0/1369349679 wait complete. 2023-09-10T00:55:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:16 smithi131 ceph-mon[103713]: pgmap v459: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 360 op/s 2023-09-10T00:55:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:16 smithi044 ceph-mon[98676]: pgmap v459: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 360 op/s 2023-09-10T00:55:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:16 smithi099 ceph-mon[102692]: pgmap v459: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 360 op/s 2023-09-10T00:55:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:18 smithi131 ceph-mon[103713]: pgmap v461: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 450 op/s 2023-09-10T00:55:19.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:18 smithi044 ceph-mon[98676]: pgmap v461: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 450 op/s 2023-09-10T00:55:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:18 smithi099 ceph-mon[102692]: pgmap v461: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 450 op/s 2023-09-10T00:55:21.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:20 smithi131 ceph-mon[103713]: pgmap v462: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 216 op/s 2023-09-10T00:55:21.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:20 smithi044 ceph-mon[98676]: pgmap v462: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 216 op/s 2023-09-10T00:55:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:20 smithi099 ceph-mon[102692]: pgmap v462: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 216 op/s 2023-09-10T00:55:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:22 smithi131 ceph-mon[103713]: pgmap v463: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2023-09-10T00:55:23.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:22 smithi044 ceph-mon[98676]: pgmap v463: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2023-09-10T00:55:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:22 smithi099 ceph-mon[102692]: pgmap v463: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2023-09-10T00:55:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:24 smithi131 ceph-mon[103713]: pgmap v464: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2023-09-10T00:55:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:24 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:25.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:24 smithi044 ceph-mon[98676]: pgmap v464: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2023-09-10T00:55:25.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:24 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:24 smithi099 ceph-mon[102692]: pgmap v464: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2023-09-10T00:55:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:24 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:26 smithi131 ceph-mon[103713]: pgmap v465: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 355 op/s 2023-09-10T00:55:27.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:26 smithi044 ceph-mon[98676]: pgmap v465: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 355 op/s 2023-09-10T00:55:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:26 smithi099 ceph-mon[102692]: pgmap v465: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 355 op/s 2023-09-10T00:55:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:28 smithi131 ceph-mon[103713]: pgmap v466: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2023-09-10T00:55:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:28 smithi044 ceph-mon[98676]: pgmap v466: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2023-09-10T00:55:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:28 smithi099 ceph-mon[102692]: pgmap v466: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2023-09-10T00:55:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:29 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:30.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:29 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:30.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.075+0000 7fcc0fb9a700 1 -- 172.21.15.44:0/3590382955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc081055b0 msgr2=0x7fcc08105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:30.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.075+0000 7fcc0fb9a700 1 --2- 172.21.15.44:0/3590382955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc081055b0 0x7fcc08105990 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fcc040099c0 tx=0x7fcc04009cd0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.076+0000 7fcc0fb9a700 1 -- 172.21.15.44:0/3590382955 shutdown_connections 2023-09-10T00:55:30.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.076+0000 7fcc0fb9a700 1 --2- 172.21.15.44:0/3590382955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc08100120 0x7fcc0810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.076+0000 7fcc0fb9a700 1 --2- 172.21.15.44:0/3590382955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc081055b0 0x7fcc08105990 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.076+0000 7fcc0fb9a700 1 --2- 172.21.15.44:0/3590382955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc080ff760 0x7fcc080ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.076+0000 7fcc0fb9a700 1 -- 172.21.15.44:0/3590382955 >> 172.21.15.44:0/3590382955 conn(0x7fcc08074f30 msgr2=0x7fcc08075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:30.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.076+0000 7fcc0fb9a700 1 -- 172.21.15.44:0/3590382955 shutdown_connections 2023-09-10T00:55:30.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.076+0000 7fcc0fb9a700 1 -- 172.21.15.44:0/3590382955 wait complete. 2023-09-10T00:55:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.077+0000 7fcc0fb9a700 1 Processor -- start 2023-09-10T00:55:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.077+0000 7fcc0fb9a700 1 -- start start 2023-09-10T00:55:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.077+0000 7fcc0fb9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc080ff760 0x7fcc0819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.077+0000 7fcc0fb9a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc08100120 0x7fcc0819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.077+0000 7fcc0fb9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc081055b0 0x7fcc081963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.077+0000 7fcc0fb9a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc08196a10 con 0x7fcc08100120 2023-09-10T00:55:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.077+0000 7fcc0fb9a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc08196b80 con 0x7fcc081055b0 2023-09-10T00:55:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.077+0000 7fcc0fb9a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc08196d40 con 0x7fcc080ff760 2023-09-10T00:55:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.077+0000 7fcc0d135700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc08100120 0x7fcc0819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.078+0000 7fcc0d936700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc080ff760 0x7fcc0819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.078+0000 7fcc0d135700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc08100120 0x7fcc0819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53630/0 (socket says 172.21.15.44:53630) 2023-09-10T00:55:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.078+0000 7fcc0d135700 1 -- 172.21.15.44:0/1704852558 learned_addr learned my addr 172.21.15.44:0/1704852558 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.078+0000 7fcc0d936700 1 --2- 172.21.15.44:0/1704852558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc080ff760 0x7fcc0819c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60510/0 (socket says 172.21.15.44:60510) 2023-09-10T00:55:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.078+0000 7fcc0e137700 1 --2- 172.21.15.44:0/1704852558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc081055b0 0x7fcc081963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.078+0000 7fcc0d135700 1 -- 172.21.15.44:0/1704852558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc080ff760 msgr2=0x7fcc0819c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.078+0000 7fcc0d135700 1 --2- 172.21.15.44:0/1704852558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc080ff760 0x7fcc0819c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.078+0000 7fcc0d135700 1 -- 172.21.15.44:0/1704852558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc081055b0 msgr2=0x7fcc081963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.078+0000 7fcc0d135700 1 --2- 172.21.15.44:0/1704852558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc081055b0 0x7fcc081963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.078+0000 7fcc0d135700 1 -- 172.21.15.44:0/1704852558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040095f0 con 0x7fcc08100120 2023-09-10T00:55:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.078+0000 7fcc0d135700 1 --2- 172.21.15.44:0/1704852558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc08100120 0x7fcc0819ca80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800e9a0 tx=0x7fcbf800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.078+0000 7fcbfeffd700 1 -- 172.21.15.44:0/1704852558 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8009790 con 0x7fcc08100120 2023-09-10T00:55:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.078+0000 7fcbfeffd700 1 -- 172.21.15.44:0/1704852558 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbf8010430 con 0x7fcc08100120 2023-09-10T00:55:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.078+0000 7fcc0fb9a700 1 -- 172.21.15.44:0/1704852558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc08197020 con 0x7fcc08100120 2023-09-10T00:55:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.078+0000 7fcbfeffd700 1 -- 172.21.15.44:0/1704852558 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf800cea0 con 0x7fcc08100120 2023-09-10T00:55:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.078+0000 7fcc0fb9a700 1 -- 172.21.15.44:0/1704852558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc081a3530 con 0x7fcc08100120 2023-09-10T00:55:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.080+0000 7fcbfeffd700 1 -- 172.21.15.44:0/1704852558 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fcbf80105a0 con 0x7fcc08100120 2023-09-10T00:55:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.081+0000 7fcbfeffd700 1 --2- 172.21.15.44:0/1704852558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcbf40a07b0 0x7fcbf40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.081+0000 7fcbfeffd700 1 -- 172.21.15.44:0/1704852558 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fcbf8014070 con 0x7fcc08100120 2023-09-10T00:55:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.081+0000 7fcc0d936700 1 --2- 172.21.15.44:0/1704852558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcbf40a07b0 0x7fcbf40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.082+0000 7fcc0d936700 1 --2- 172.21.15.44:0/1704852558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcbf40a07b0 0x7fcbf40a2c70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400b580 tx=0x7fcc04005c00 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.082+0000 7fcc1bd60b80 1 Processor -- start 2023-09-10T00:55:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.082+0000 7fcc1bd60b80 1 -- start start 2023-09-10T00:55:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.082+0000 7fcc1bd60b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f9465b0 0x55e07f946990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.082+0000 7fcc1bd60b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f940710 0x55e07f940b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.082+0000 7fcc1bd60b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9410d0 0x55e07f94d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.082+0000 7fcc1bd60b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f8956d0 con 0x55e07f9410d0 2023-09-10T00:55:30.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.083+0000 7fcc1bd60b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f892ff0 con 0x55e07f9465b0 2023-09-10T00:55:30.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.083+0000 7fcc1bd60b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f88e8e0 con 0x55e07f940710 2023-09-10T00:55:30.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.083+0000 7fcc0e137700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9410d0 0x55e07f94d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.083+0000 7fcc0e137700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9410d0 0x55e07f94d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53638/0 (socket says 172.21.15.44:53638) 2023-09-10T00:55:30.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.083+0000 7fcc0e137700 1 -- 172.21.15.44:0/706275238 learned_addr learned my addr 172.21.15.44:0/706275238 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:30.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.083+0000 7fcc0d135700 1 --2- 172.21.15.44:0/706275238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f940710 0x55e07f940b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.083+0000 7fcc0d936700 1 --2- 172.21.15.44:0/706275238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f9465b0 0x55e07f946990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.083+0000 7fcc0e137700 1 -- 172.21.15.44:0/706275238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f940710 msgr2=0x55e07f940b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:30.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.083+0000 7fcc0e137700 1 --2- 172.21.15.44:0/706275238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f940710 0x55e07f940b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.083+0000 7fcc0e137700 1 -- 172.21.15.44:0/706275238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f9465b0 msgr2=0x55e07f946990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:30.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.083+0000 7fcc0e137700 1 --2- 172.21.15.44:0/706275238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f9465b0 0x55e07f946990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.083+0000 7fcc0e137700 1 -- 172.21.15.44:0/706275238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e07f6bad80 con 0x55e07f9410d0 2023-09-10T00:55:30.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.083+0000 7fcc0e137700 1 --2- 172.21.15.44:0/706275238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9410d0 0x55e07f94d340 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fcc00011600 tx=0x7fcc00009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46c69840158c899 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:30.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.083+0000 7fcbf3fff700 1 -- 172.21.15.44:0/706275238 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000e040 con 0x55e07f9410d0 2023-09-10T00:55:30.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.083+0000 7fcbf3fff700 1 -- 172.21.15.44:0/706275238 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc0000f040 con 0x55e07f9410d0 2023-09-10T00:55:30.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.083+0000 7fcbf3fff700 1 -- 172.21.15.44:0/706275238 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00004630 con 0x55e07f9410d0 2023-09-10T00:55:30.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.084+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/706275238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9410d0 msgr2=0x55e07f94d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:30.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.084+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/706275238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9410d0 0x55e07f94d340 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fcc00011600 tx=0x7fcc00009920 comp rx=0 tx=0).stop 2023-09-10T00:55:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.084+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/706275238 shutdown_connections 2023-09-10T00:55:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.084+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/706275238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9410d0 0x55e07f94d340 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.084+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/706275238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f9465b0 0x55e07f946990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.084+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/706275238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f940710 0x55e07f940b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.084+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/706275238 >> 172.21.15.44:0/706275238 conn(0x55e07f8b1030 msgr2=0x55e07f937560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.084+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/706275238 shutdown_connections 2023-09-10T00:55:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.084+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/706275238 wait complete. 2023-09-10T00:55:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.085+0000 7fcc1bd60b80 1 -- start start 2023-09-10T00:55:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.085+0000 7fcc1bd60b80 1 Processor -- start 2023-09-10T00:55:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.085+0000 7fcc1bd60b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f940710 0x55e07f94d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.085+0000 7fcc1bd60b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f9410d0 0x55e07f94a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.085+0000 7fcc1bd60b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9465b0 0x55e07f94aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.085+0000 7fcc1bd60b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f82cb10 con 0x55e07f9465b0 2023-09-10T00:55:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.085+0000 7fcc1bd60b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f82c810 con 0x55e07f940710 2023-09-10T00:55:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.085+0000 7fcc1bd60b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f7108c0 con 0x55e07f9410d0 2023-09-10T00:55:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.085+0000 7fcc1bd60b80 10 client.0 _ll_register_callbacks cb 0x55e07f776a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:55:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.085+0000 7fcc0e137700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9465b0 0x55e07f94aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.085+0000 7fcc0d936700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f940710 0x55e07f94d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.085+0000 7fcc0e137700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9465b0 0x55e07f94aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53646/0 (socket says 172.21.15.44:53646) 2023-09-10T00:55:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.085+0000 7fcc0e137700 1 -- 172.21.15.44:0/2172734735 learned_addr learned my addr 172.21.15.44:0/2172734735 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.085+0000 7fcc0d936700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f940710 0x55e07f94d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55382/0 (socket says 172.21.15.44:55382) 2023-09-10T00:55:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.085+0000 7fcc0d135700 1 --2- 172.21.15.44:0/2172734735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f9410d0 0x55e07f94a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.086+0000 7fcc0e137700 1 -- 172.21.15.44:0/2172734735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f9410d0 msgr2=0x55e07f94a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.086+0000 7fcc0e137700 1 --2- 172.21.15.44:0/2172734735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f9410d0 0x55e07f94a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.086+0000 7fcc0e137700 1 -- 172.21.15.44:0/2172734735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f940710 msgr2=0x55e07f94d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.086+0000 7fcc0e137700 1 --2- 172.21.15.44:0/2172734735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f940710 0x55e07f94d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.086+0000 7fcc0e137700 1 -- 172.21.15.44:0/2172734735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc000112b0 con 0x55e07f9465b0 2023-09-10T00:55:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.086+0000 7fcc0e137700 1 --2- 172.21.15.44:0/2172734735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9465b0 0x55e07f94aa60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fcc00004f80 tx=0x7fcc00009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbce7cfd73a9ae34 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.086+0000 7fcbf17fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:55:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.086+0000 7fcbf17fa700 1 -- 172.21.15.44:0/2172734735 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000e030 con 0x55e07f9465b0 2023-09-10T00:55:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.086+0000 7fcbf17fa700 1 -- 172.21.15.44:0/2172734735 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fcc00003c10 con 0x55e07f9465b0 2023-09-10T00:55:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.086+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/2172734735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e07f7e6490 con 0x55e07f9465b0 2023-09-10T00:55:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.086+0000 7fcbf17fa700 1 -- 172.21.15.44:0/2172734735 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc00003dd0 con 0x55e07f9465b0 2023-09-10T00:55:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.086+0000 7fcc1bd60b80 10 client.17862 fetch_fsmap learned FSMap version 72 2023-09-10T00:55:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.086+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/2172734735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55e07f6a3c00 con 0x55e07f9465b0 2023-09-10T00:55:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.086+0000 7fcbf17fa700 1 -- 172.21.15.44:0/2172734735 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e07f7e6490 con 0x55e07f9465b0 2023-09-10T00:55:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.087+0000 7fcbf17fa700 1 -- 172.21.15.44:0/2172734735 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 72) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fcc00024020 con 0x55e07f9465b0 2023-09-10T00:55:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.087+0000 7fcc1bd60b80 10 client.17862 fetch_fsmap finished waiting for FSMap version 72 2023-09-10T00:55:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.087+0000 7fcc1bd60b80 10 client.17862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:55:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.087+0000 7fcc1bd60b80 20 client.17862 populate_metadata read hostname 'smithi044' 2023-09-10T00:55:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.087+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/2172734735 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e07f95a810 0x55e07f95cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.087+0000 7fcc1bd60b80 4 client.17862 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:55:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.087+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/2172734735 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e07f8a3840 con 0x55e07f95a810 2023-09-10T00:55:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.087+0000 7fcc0d936700 1 --2- 172.21.15.44:0/2172734735 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e07f95a810 0x55e07f95cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.087+0000 7fcbf17fa700 10 client.17862 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.087+0000 7fcc0d936700 1 --2- 172.21.15.44:0/2172734735 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e07f95a810 0x55e07f95cc30 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78ee4c1b65d5d6bd server_cookie=e99b1d3ae6cf65d0 in_seq=0 out_seq=0 2023-09-10T00:55:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.088+0000 7fcbf17fa700 1 -- 172.21.15.44:0/2172734735 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e07f8a3840 con 0x55e07f95a810 2023-09-10T00:55:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.088+0000 7fcbf17fa700 10 client.17862 handle_command_reply: tid=0 2023-09-10T00:55:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.088+0000 7fcc1bd60b80 1 client.17862 shutdown 2023-09-10T00:55:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.088+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/2172734735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9465b0 msgr2=0x55e07f94aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.088+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/2172734735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9465b0 0x55e07f94aa60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fcc00004f80 tx=0x7fcc00009dd0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.088+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/2172734735 shutdown_connections 2023-09-10T00:55:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.088+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/2172734735 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e07f95a810 0x55e07f95cc30 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.088+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/2172734735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9465b0 0x55e07f94aa60 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.088+0000 7fcbf17fa700 0 client.17862 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.088+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/2172734735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f940710 0x55e07f94d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.088+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/2172734735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f9410d0 0x55e07f94a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.088+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/2172734735 >> 172.21.15.44:0/2172734735 conn(0x55e07f8b1030 msgr2=0x55e07f9367a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.088+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/2172734735 shutdown_connections 2023-09-10T00:55:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.089+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/2172734735 wait complete. 2023-09-10T00:55:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.089+0000 7fcc1bd60b80 20 client.17862 trim_cache size 0 max 16384 2023-09-10T00:55:30.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:29 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.104+0000 7fcc1bd60b80 1 Processor -- start 2023-09-10T00:55:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.104+0000 7fcc1bd60b80 1 -- start start 2023-09-10T00:55:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.104+0000 7fcc1bd60b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f940710 0x55e07f94cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.104+0000 7fcc1bd60b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f9410d0 0x55e07f945560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.104+0000 7fcc1bd60b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9465b0 0x55e07f945aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.104+0000 7fcc1bd60b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f944f30 con 0x55e07f9465b0 2023-09-10T00:55:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.104+0000 7fcc1bd60b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f945fe0 con 0x55e07f9410d0 2023-09-10T00:55:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.104+0000 7fcc1bd60b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f945150 con 0x55e07f940710 2023-09-10T00:55:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.104+0000 7fcc0e137700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9465b0 0x55e07f945aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.104+0000 7fcc0e137700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9465b0 0x55e07f945aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53654/0 (socket says 172.21.15.44:53654) 2023-09-10T00:55:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.104+0000 7fcc0d936700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f940710 0x55e07f94cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.104+0000 7fcc0d135700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f9410d0 0x55e07f945560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.104+0000 7fcc0e137700 1 -- 172.21.15.44:0/466431668 learned_addr learned my addr 172.21.15.44:0/466431668 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.105+0000 7fcc0e137700 1 -- 172.21.15.44:0/466431668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f940710 msgr2=0x55e07f94cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.105+0000 7fcc0e137700 1 --2- 172.21.15.44:0/466431668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f940710 0x55e07f94cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.105+0000 7fcc0e137700 1 -- 172.21.15.44:0/466431668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f9410d0 msgr2=0x55e07f945560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.105+0000 7fcc0e137700 1 --2- 172.21.15.44:0/466431668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f9410d0 0x55e07f945560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.105+0000 7fcc0e137700 1 -- 172.21.15.44:0/466431668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e07f6e11f0 con 0x55e07f9465b0 2023-09-10T00:55:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.105+0000 7fcc0e137700 1 --2- 172.21.15.44:0/466431668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9465b0 0x55e07f945aa0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fcc00011db0 tx=0x7fcc000021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e5730a66330358 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.105+0000 7fcbfcd89700 1 -- 172.21.15.44:0/466431668 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000e030 con 0x55e07f9465b0 2023-09-10T00:55:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.105+0000 7fcbfcd89700 1 -- 172.21.15.44:0/466431668 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc00008ed0 con 0x55e07f9465b0 2023-09-10T00:55:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.105+0000 7fcbfcd89700 1 -- 172.21.15.44:0/466431668 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc000086c0 con 0x55e07f9465b0 2023-09-10T00:55:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.105+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/466431668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9465b0 msgr2=0x55e07f945aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.105+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/466431668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9465b0 0x55e07f945aa0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fcc00011db0 tx=0x7fcc000021d0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.106+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/466431668 shutdown_connections 2023-09-10T00:55:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.106+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/466431668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f9465b0 0x55e07f945aa0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.106+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/466431668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f940710 0x55e07f94cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.106+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/466431668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f9410d0 0x55e07f945560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.106+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/466431668 >> 172.21.15.44:0/466431668 conn(0x55e07f8b1030 msgr2=0x55e07f94a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.106+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/466431668 shutdown_connections 2023-09-10T00:55:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.106+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/466431668 wait complete. 2023-09-10T00:55:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.106+0000 7fcc1bd60b80 1 -- start start 2023-09-10T00:55:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.106+0000 7fcc1bd60b80 1 Processor -- start 2023-09-10T00:55:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.106+0000 7fcc1bd60b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f940710 0x55e07f94f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.106+0000 7fcc1bd60b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f9410d0 0x55e07f94fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.106+0000 7fcc1bd60b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f9465b0 0x55e07f9ad640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.106+0000 7fcc1bd60b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f950770 con 0x55e07f940710 2023-09-10T00:55:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.106+0000 7fcc1bd60b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f9adb80 con 0x55e07f9465b0 2023-09-10T00:55:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.106+0000 7fcc1bd60b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f9adcc0 con 0x55e07f9410d0 2023-09-10T00:55:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.106+0000 7fcc1bd60b80 10 client.0 _ll_register_callbacks cb 0x7fcc0400b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:55:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcc0d936700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f940710 0x55e07f94f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcc0d936700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f940710 0x55e07f94f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53656/0 (socket says 172.21.15.44:53656) 2023-09-10T00:55:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcc0d936700 1 -- 172.21.15.44:0/54763946 learned_addr learned my addr 172.21.15.44:0/54763946 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcc0d135700 1 --2- 172.21.15.44:0/54763946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f9410d0 0x55e07f94fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcc0e137700 1 --2- 172.21.15.44:0/54763946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f9465b0 0x55e07f9ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcc0d936700 1 -- 172.21.15.44:0/54763946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f9410d0 msgr2=0x55e07f94fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcc0d936700 1 --2- 172.21.15.44:0/54763946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f9410d0 0x55e07f94fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcc0d936700 1 -- 172.21.15.44:0/54763946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f9465b0 msgr2=0x55e07f9ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcc0d936700 1 --2- 172.21.15.44:0/54763946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f9465b0 0x55e07f9ad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcc0d936700 1 -- 172.21.15.44:0/54763946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e07f6a3c00 con 0x55e07f940710 2023-09-10T00:55:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcc0d135700 1 --2- 172.21.15.44:0/54763946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f9410d0 0x55e07f94fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:55:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcc0d936700 1 --2- 172.21.15.44:0/54763946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f940710 0x55e07f94f920 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fcc040099c0 tx=0x7fcc04009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf74a05ca5e923a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcbf17fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:55:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcbf17fa700 1 -- 172.21.15.44:0/54763946 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04028030 con 0x55e07f940710 2023-09-10T00:55:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcbf17fa700 1 -- 172.21.15.44:0/54763946 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fcc040320e0 con 0x55e07f940710 2023-09-10T00:55:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/54763946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc0000f040 con 0x55e07f940710 2023-09-10T00:55:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcbf17fa700 1 -- 172.21.15.44:0/54763946 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc040322a0 con 0x55e07f940710 2023-09-10T00:55:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcc1bd60b80 10 client.17868 fetch_fsmap learned FSMap version 72 2023-09-10T00:55:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.107+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/54763946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55e07f6b9250 con 0x55e07f940710 2023-09-10T00:55:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.108+0000 7fcbf17fa700 1 -- 172.21.15.44:0/54763946 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc0000f040 con 0x55e07f940710 2023-09-10T00:55:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.108+0000 7fcbf17fa700 1 -- 172.21.15.44:0/54763946 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 72) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fcc0400f4f0 con 0x55e07f940710 2023-09-10T00:55:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.108+0000 7fcc1bd60b80 10 client.17868 fetch_fsmap finished waiting for FSMap version 72 2023-09-10T00:55:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.108+0000 7fcc1bd60b80 10 client.17868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:55:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.108+0000 7fcc1bd60b80 20 client.17868 populate_metadata read hostname 'smithi044' 2023-09-10T00:55:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.108+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/54763946 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e07f9af400 0x55e07f9b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.108+0000 7fcc1bd60b80 4 client.17868 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:55:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.108+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/54763946 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e07f8878c0 con 0x55e07f9af400 2023-09-10T00:55:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.108+0000 7fcc0d135700 1 --2- 172.21.15.44:0/54763946 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e07f9af400 0x55e07f9b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.108+0000 7fcbf17fa700 10 client.17868 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.108+0000 7fcc0d135700 1 --2- 172.21.15.44:0/54763946 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e07f9af400 0x55e07f9b1820 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=141369b0bcb27669 server_cookie=c5c22ed008195377 in_seq=0 out_seq=0 2023-09-10T00:55:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.120+0000 7fcbf17fa700 1 -- 172.21.15.44:0/54763946 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e07f8878c0 con 0x55e07f9af400 2023-09-10T00:55:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.120+0000 7fcbf17fa700 10 client.17868 handle_command_reply: tid=0 2023-09-10T00:55:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.120+0000 7fcc1bd60b80 1 client.17868 shutdown 2023-09-10T00:55:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.120+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/54763946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f940710 msgr2=0x55e07f94f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.120+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/54763946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f940710 0x55e07f94f920 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fcc040099c0 tx=0x7fcc04009cd0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.120+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/54763946 shutdown_connections 2023-09-10T00:55:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.120+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/54763946 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e07f9af400 0x55e07f9b1820 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.120+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/54763946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f9465b0 0x55e07f9ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.120+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/54763946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e07f940710 0x55e07f94f920 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.120+0000 7fcbf17fa700 0 client.17868 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.120+0000 7fcc1bd60b80 1 --2- 172.21.15.44:0/54763946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e07f9410d0 0x55e07f94fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.120+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/54763946 >> 172.21.15.44:0/54763946 conn(0x55e07f8b1030 msgr2=0x55e07f945560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.120+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/54763946 shutdown_connections 2023-09-10T00:55:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.120+0000 7fcc1bd60b80 1 -- 172.21.15.44:0/54763946 wait complete. 2023-09-10T00:55:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.120+0000 7fcc1bd60b80 20 client.17868 trim_cache size 0 max 16384 2023-09-10T00:55:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:55:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:55:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:55:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:55:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.121+0000 7fcc0fb9a700 1 -- 172.21.15.44:0/1704852558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcbf40a07b0 msgr2=0x7fcbf40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.121+0000 7fcc0fb9a700 1 --2- 172.21.15.44:0/1704852558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcbf40a07b0 0x7fcbf40a2c70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400b580 tx=0x7fcc04005c00 comp rx=0 tx=0).stop 2023-09-10T00:55:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.121+0000 7fcc0fb9a700 1 -- 172.21.15.44:0/1704852558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc08100120 msgr2=0x7fcc0819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.121+0000 7fcc0fb9a700 1 --2- 172.21.15.44:0/1704852558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc08100120 0x7fcc0819ca80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800e9a0 tx=0x7fcbf800ecb0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.121+0000 7fcc0fb9a700 1 -- 172.21.15.44:0/1704852558 shutdown_connections 2023-09-10T00:55:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.121+0000 7fcc0fb9a700 1 --2- 172.21.15.44:0/1704852558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcbf40a07b0 0x7fcbf40a2c70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.121+0000 7fcc0fb9a700 1 --2- 172.21.15.44:0/1704852558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc081055b0 0x7fcc081963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.121+0000 7fcc0fb9a700 1 --2- 172.21.15.44:0/1704852558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc080ff760 0x7fcc0819c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.121+0000 7fcc0fb9a700 1 --2- 172.21.15.44:0/1704852558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc08100120 0x7fcc0819ca80 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.121+0000 7fcc0fb9a700 1 -- 172.21.15.44:0/1704852558 >> 172.21.15.44:0/1704852558 conn(0x7fcc08074f30 msgr2=0x7fcc08100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.121+0000 7fcc0fb9a700 1 -- 172.21.15.44:0/1704852558 shutdown_connections 2023-09-10T00:55:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:30.121+0000 7fcc0fb9a700 1 -- 172.21.15.44:0/1704852558 wait complete. 2023-09-10T00:55:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:30 smithi131 ceph-mon[103713]: pgmap v467: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 296 op/s 2023-09-10T00:55:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:30 smithi044 ceph-mon[98676]: pgmap v467: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 296 op/s 2023-09-10T00:55:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:30 smithi099 ceph-mon[102692]: pgmap v467: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 296 op/s 2023-09-10T00:55:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:32 smithi131 ceph-mon[103713]: pgmap v468: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 426 op/s 2023-09-10T00:55:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:32 smithi044 ceph-mon[98676]: pgmap v468: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 426 op/s 2023-09-10T00:55:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:32 smithi099 ceph-mon[102692]: pgmap v468: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 426 op/s 2023-09-10T00:55:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:34 smithi131 ceph-mon[103713]: pgmap v469: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 281 op/s 2023-09-10T00:55:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:34 smithi044 ceph-mon[98676]: pgmap v469: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 281 op/s 2023-09-10T00:55:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:34 smithi099 ceph-mon[102692]: pgmap v469: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 281 op/s 2023-09-10T00:55:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:36 smithi131 ceph-mon[103713]: pgmap v470: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 406 op/s 2023-09-10T00:55:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:36 smithi044 ceph-mon[98676]: pgmap v470: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 406 op/s 2023-09-10T00:55:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:36 smithi099 ceph-mon[102692]: pgmap v470: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 406 op/s 2023-09-10T00:55:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:38 smithi044 ceph-mon[98676]: pgmap v471: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2023-09-10T00:55:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:38 smithi099 ceph-mon[102692]: pgmap v471: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2023-09-10T00:55:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:38 smithi131 ceph-mon[103713]: pgmap v471: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2023-09-10T00:55:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:40 smithi044 ceph-mon[98676]: pgmap v472: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2023-09-10T00:55:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:40 smithi099 ceph-mon[102692]: pgmap v472: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2023-09-10T00:55:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:40 smithi131 ceph-mon[103713]: pgmap v472: 129 pgs: 129 active+clean; 9.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2023-09-10T00:55:42.591 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:55:42.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.759+0000 7fc97c486700 1 -- 172.21.15.44:0/3783458692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9740ff450 msgr2=0x7fc9740ff8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:42.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.759+0000 7fc97c486700 1 --2- 172.21.15.44:0/3783458692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9740ff450 0x7fc9740ff8d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fc964009990 tx=0x7fc964009ca0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.759+0000 7fc97c486700 1 -- 172.21.15.44:0/3783458692 shutdown_connections 2023-09-10T00:55:42.760 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.759+0000 7fc97c486700 1 --2- 172.21.15.44:0/3783458692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9740ffe10 0x7fc97410c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.759+0000 7fc97c486700 1 --2- 172.21.15.44:0/3783458692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9741054f0 0x7fc9741058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.759+0000 7fc97c486700 1 --2- 172.21.15.44:0/3783458692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9740ff450 0x7fc9740ff8d0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.759+0000 7fc97c486700 1 -- 172.21.15.44:0/3783458692 >> 172.21.15.44:0/3783458692 conn(0x7fc9740faf40 msgr2=0x7fc9740fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:42.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.759+0000 7fc97c486700 1 -- 172.21.15.44:0/3783458692 shutdown_connections 2023-09-10T00:55:42.761 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.759+0000 7fc97c486700 1 -- 172.21.15.44:0/3783458692 wait complete. 2023-09-10T00:55:42.763 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97c486700 1 Processor -- start 2023-09-10T00:55:42.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97c486700 1 -- start start 2023-09-10T00:55:42.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97c486700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9740ff450 0x7fc9741954b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97c486700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9740ffe10 0x7fc9741959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97c486700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9741054f0 0x7fc974199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.764 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97c486700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97419a490 con 0x7fc9741054f0 2023-09-10T00:55:42.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97c486700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97419a600 con 0x7fc9740ff450 2023-09-10T00:55:42.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97c486700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97419a7c0 con 0x7fc9740ffe10 2023-09-10T00:55:42.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97aa23700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9741054f0 0x7fc974199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97aa23700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9741054f0 0x7fc974199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53694/0 (socket says 172.21.15.44:53694) 2023-09-10T00:55:42.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97aa23700 1 -- 172.21.15.44:0/4204424434 learned_addr learned my addr 172.21.15.44:0/4204424434 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:42.765 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97a222700 1 --2- 172.21.15.44:0/4204424434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9740ff450 0x7fc9741954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc979a21700 1 --2- 172.21.15.44:0/4204424434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9740ffe10 0x7fc9741959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97aa23700 1 -- 172.21.15.44:0/4204424434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9740ffe10 msgr2=0x7fc9741959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:42.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97aa23700 1 --2- 172.21.15.44:0/4204424434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9740ffe10 0x7fc9741959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97aa23700 1 -- 172.21.15.44:0/4204424434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9740ff450 msgr2=0x7fc9741954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:42.766 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97aa23700 1 --2- 172.21.15.44:0/4204424434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9740ff450 0x7fc9741954b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.760+0000 7fc97aa23700 1 -- 172.21.15.44:0/4204424434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9640095f0 con 0x7fc9741054f0 2023-09-10T00:55:42.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.761+0000 7fc97aa23700 1 --2- 172.21.15.44:0/4204424434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9741054f0 0x7fc974199e50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00ef30 tx=0x7fc96c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:42.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.761+0000 7fc96b7fe700 1 -- 172.21.15.44:0/4204424434 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c018070 con 0x7fc9741054f0 2023-09-10T00:55:42.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.761+0000 7fc96b7fe700 1 -- 172.21.15.44:0/4204424434 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc96c00f040 con 0x7fc9741054f0 2023-09-10T00:55:42.767 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.761+0000 7fc97c486700 1 -- 172.21.15.44:0/4204424434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97419aa40 con 0x7fc9741054f0 2023-09-10T00:55:42.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.761+0000 7fc96b7fe700 1 -- 172.21.15.44:0/4204424434 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c00cbc0 con 0x7fc9741054f0 2023-09-10T00:55:42.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.761+0000 7fc97c486700 1 -- 172.21.15.44:0/4204424434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9741a22e0 con 0x7fc9741054f0 2023-09-10T00:55:42.768 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.763+0000 7fc96b7fe700 1 -- 172.21.15.44:0/4204424434 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc96c00cd80 con 0x7fc9741054f0 2023-09-10T00:55:42.773 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.764+0000 7fc96b7fe700 1 --2- 172.21.15.44:0/4204424434 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc9600a0900 0x7fc9600a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.764+0000 7fc96b7fe700 1 -- 172.21.15.44:0/4204424434 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc96c0cbd60 con 0x7fc9741054f0 2023-09-10T00:55:42.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.764+0000 7fc97a222700 1 --2- 172.21.15.44:0/4204424434 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc9600a0900 0x7fc9600a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.764+0000 7fc97a222700 1 --2- 172.21.15.44:0/4204424434 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc9600a0900 0x7fc9600a2dc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc970009be0 tx=0x7fc970009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:42.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.764+0000 7fc98864cb80 1 Processor -- start 2023-09-10T00:55:42.774 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc98864cb80 1 -- start start 2023-09-10T00:55:42.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc98864cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 0x55ae8d4b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc98864cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b7030 0x55ae8d4c0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc98864cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4c3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc98864cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8d40f730 con 0x55ae8d4b7030 2023-09-10T00:55:42.775 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc98864cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8d40d050 con 0x55ae8d4b6710 2023-09-10T00:55:42.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc98864cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8d408940 con 0x55ae8d4c0e40 2023-09-10T00:55:42.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc979a21700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b7030 0x55ae8d4c0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc979a21700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b7030 0x55ae8d4c0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53700/0 (socket says 172.21.15.44:53700) 2023-09-10T00:55:42.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc979a21700 1 -- 172.21.15.44:0/2664054286 learned_addr learned my addr 172.21.15.44:0/2664054286 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:42.776 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc97aa23700 1 --2- 172.21.15.44:0/2664054286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc97a222700 1 --2- 172.21.15.44:0/2664054286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 0x55ae8d4b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc979a21700 1 -- 172.21.15.44:0/2664054286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 msgr2=0x55ae8d4c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:42.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc979a21700 1 --2- 172.21.15.44:0/2664054286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4c3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc979a21700 1 -- 172.21.15.44:0/2664054286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 msgr2=0x55ae8d4b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:42.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc979a21700 1 --2- 172.21.15.44:0/2664054286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 0x55ae8d4b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.777 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.765+0000 7fc979a21700 1 -- 172.21.15.44:0/2664054286 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae8d234de0 con 0x55ae8d4b7030 2023-09-10T00:55:42.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.766+0000 7fc979a21700 1 --2- 172.21.15.44:0/2664054286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b7030 0x55ae8d4c0900 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc964009990 tx=0x7fc964005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930fe2b4239c066c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:42.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.766+0000 7fc968d89700 1 -- 172.21.15.44:0/2664054286 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96401d070 con 0x55ae8d4b7030 2023-09-10T00:55:42.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.766+0000 7fc968d89700 1 -- 172.21.15.44:0/2664054286 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc96401e040 con 0x55ae8d4b7030 2023-09-10T00:55:42.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.766+0000 7fc968d89700 1 -- 172.21.15.44:0/2664054286 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964004760 con 0x55ae8d4b7030 2023-09-10T00:55:42.778 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.766+0000 7fc98864cb80 1 -- 172.21.15.44:0/2664054286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b7030 msgr2=0x55ae8d4c0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:42.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.766+0000 7fc98864cb80 1 --2- 172.21.15.44:0/2664054286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b7030 0x55ae8d4c0900 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc964009990 tx=0x7fc964005f00 comp rx=0 tx=0).stop 2023-09-10T00:55:42.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.767+0000 7fc98864cb80 1 -- 172.21.15.44:0/2664054286 shutdown_connections 2023-09-10T00:55:42.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.767+0000 7fc98864cb80 1 --2- 172.21.15.44:0/2664054286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4c3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.767+0000 7fc98864cb80 1 --2- 172.21.15.44:0/2664054286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 0x55ae8d4b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.767+0000 7fc98864cb80 1 --2- 172.21.15.44:0/2664054286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b7030 0x55ae8d4c0900 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.779 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.767+0000 7fc98864cb80 1 -- 172.21.15.44:0/2664054286 >> 172.21.15.44:0/2664054286 conn(0x55ae8d42b090 msgr2=0x55ae8d4b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:42.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.767+0000 7fc98864cb80 1 -- 172.21.15.44:0/2664054286 shutdown_connections 2023-09-10T00:55:42.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.767+0000 7fc98864cb80 1 -- 172.21.15.44:0/2664054286 wait complete. 2023-09-10T00:55:42.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.767+0000 7fc98864cb80 1 -- start start 2023-09-10T00:55:42.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.767+0000 7fc98864cb80 1 Processor -- start 2023-09-10T00:55:42.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.767+0000 7fc98864cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 0x55ae8d4c07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.767+0000 7fc98864cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b7030 0x55ae8d4bd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.767+0000 7fc98864cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4bdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.767+0000 7fc98864cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8d3a6b70 con 0x55ae8d4b7030 2023-09-10T00:55:42.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.767+0000 7fc98864cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8d3a6870 con 0x55ae8d4b6710 2023-09-10T00:55:42.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.767+0000 7fc98864cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8d28a920 con 0x55ae8d4c0e40 2023-09-10T00:55:42.781 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.767+0000 7fc98864cb80 10 client.0 _ll_register_callbacks cb 0x55ae8d2f0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:55:42.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc979a21700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b7030 0x55ae8d4bd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc97aa23700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4bdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc97a222700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 0x55ae8d4c07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc979a21700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b7030 0x55ae8d4bd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53706/0 (socket says 172.21.15.44:53706) 2023-09-10T00:55:42.782 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc979a21700 1 -- 172.21.15.44:0/1303261769 learned_addr learned my addr 172.21.15.44:0/1303261769 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:42.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc979a21700 1 -- 172.21.15.44:0/1303261769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 msgr2=0x55ae8d4bdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:42.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc979a21700 1 --2- 172.21.15.44:0/1303261769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4bdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc979a21700 1 -- 172.21.15.44:0/1303261769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 msgr2=0x55ae8d4c07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:42.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc979a21700 1 --2- 172.21.15.44:0/1303261769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 0x55ae8d4c07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc979a21700 1 -- 172.21.15.44:0/1303261769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97410cc20 con 0x55ae8d4b7030 2023-09-10T00:55:42.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc97aa23700 1 --2- 172.21.15.44:0/1303261769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4bdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:55:42.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc979a21700 1 --2- 172.21.15.44:0/1303261769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b7030 0x55ae8d4bd9f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc964000c00 tx=0x7fc96400b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb6b7c50ade537f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:42.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc95dffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:55:42.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc95dffb700 1 -- 172.21.15.44:0/1303261769 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96401d030 con 0x55ae8d4b7030 2023-09-10T00:55:42.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc95dffb700 1 -- 172.21.15.44:0/1303261769 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc96400be60 con 0x55ae8d4b7030 2023-09-10T00:55:42.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc98864cb80 1 -- 172.21.15.44:0/1303261769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae8d3604f0 con 0x55ae8d4b7030 2023-09-10T00:55:42.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc95dffb700 1 -- 172.21.15.44:0/1303261769 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc964004710 con 0x55ae8d4b7030 2023-09-10T00:55:42.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc98864cb80 10 client.17880 fetch_fsmap learned FSMap version 72 2023-09-10T00:55:42.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.768+0000 7fc98864cb80 1 -- 172.21.15.44:0/1303261769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55ae8d21dc60 con 0x55ae8d4b7030 2023-09-10T00:55:42.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.769+0000 7fc95dffb700 1 -- 172.21.15.44:0/1303261769 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ae8d3604f0 con 0x55ae8d4b7030 2023-09-10T00:55:42.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.769+0000 7fc95dffb700 1 -- 172.21.15.44:0/1303261769 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 72) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fc96400f8a0 con 0x55ae8d4b7030 2023-09-10T00:55:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.769+0000 7fc98864cb80 10 client.17880 fetch_fsmap finished waiting for FSMap version 72 2023-09-10T00:55:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.769+0000 7fc98864cb80 10 client.17880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:55:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.769+0000 7fc98864cb80 20 client.17880 populate_metadata read hostname 'smithi044' 2023-09-10T00:55:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.769+0000 7fc98864cb80 1 --2- 172.21.15.44:0/1303261769 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ae8d4d0470 0x55ae8d4d2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.769+0000 7fc98864cb80 4 client.17880 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:55:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.769+0000 7fc98864cb80 1 -- 172.21.15.44:0/1303261769 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae8d41d8a0 con 0x55ae8d4d0470 2023-09-10T00:55:42.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.769+0000 7fc97a222700 1 --2- 172.21.15.44:0/1303261769 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ae8d4d0470 0x55ae8d4d2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.769+0000 7fc95dffb700 10 client.17880 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:42.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.769+0000 7fc97a222700 1 --2- 172.21.15.44:0/1303261769 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ae8d4d0470 0x55ae8d4d2890 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc77021960a3b2a server_cookie=303703e516e603f3 in_seq=0 out_seq=0 2023-09-10T00:55:42.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.770+0000 7fc95dffb700 1 -- 172.21.15.44:0/1303261769 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ae8d41d8a0 con 0x55ae8d4d0470 2023-09-10T00:55:42.787 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.770+0000 7fc95dffb700 10 client.17880 handle_command_reply: tid=0 2023-09-10T00:55:42.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.770+0000 7fc98864cb80 1 client.17880 shutdown 2023-09-10T00:55:42.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.770+0000 7fc98864cb80 1 -- 172.21.15.44:0/1303261769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b7030 msgr2=0x55ae8d4bd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:42.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.770+0000 7fc98864cb80 1 --2- 172.21.15.44:0/1303261769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b7030 0x55ae8d4bd9f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc964000c00 tx=0x7fc96400b940 comp rx=0 tx=0).stop 2023-09-10T00:55:42.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.770+0000 7fc98864cb80 1 -- 172.21.15.44:0/1303261769 shutdown_connections 2023-09-10T00:55:42.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.770+0000 7fc98864cb80 1 --2- 172.21.15.44:0/1303261769 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ae8d4d0470 0x55ae8d4d2890 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.770+0000 7fc98864cb80 1 --2- 172.21.15.44:0/1303261769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4bdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.770+0000 7fc95dffb700 0 client.17880 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:42.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.770+0000 7fc98864cb80 1 --2- 172.21.15.44:0/1303261769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 0x55ae8d4c07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.770+0000 7fc98864cb80 1 --2- 172.21.15.44:0/1303261769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b7030 0x55ae8d4bd9f0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.770+0000 7fc98864cb80 1 -- 172.21.15.44:0/1303261769 >> 172.21.15.44:0/1303261769 conn(0x55ae8d42b090 msgr2=0x55ae8d4b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:42.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.771+0000 7fc98864cb80 1 -- 172.21.15.44:0/1303261769 shutdown_connections 2023-09-10T00:55:42.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.771+0000 7fc98864cb80 1 -- 172.21.15.44:0/1303261769 wait complete. 2023-09-10T00:55:42.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.771+0000 7fc98864cb80 20 client.17880 trim_cache size 0 max 16384 2023-09-10T00:55:42.795 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.789+0000 7fc98864cb80 1 Processor -- start 2023-09-10T00:55:42.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.789+0000 7fc98864cb80 1 -- start start 2023-09-10T00:55:42.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.789+0000 7fc98864cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 0x55ae8d4c0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.789+0000 7fc98864cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4bac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.789+0000 7fc98864cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b9430 0x55ae8d4bb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.796 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.789+0000 7fc98864cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8d4b78c0 con 0x55ae8d4b9430 2023-09-10T00:55:42.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.789+0000 7fc98864cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8d4ae3d0 con 0x55ae8d4b6710 2023-09-10T00:55:42.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.789+0000 7fc98864cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8d4be5b0 con 0x55ae8d4c0e40 2023-09-10T00:55:42.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.789+0000 7fc97aa23700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b9430 0x55ae8d4bb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.790+0000 7fc979a21700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4bac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.790+0000 7fc97a222700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 0x55ae8d4c0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.797 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.790+0000 7fc97aa23700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b9430 0x55ae8d4bb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53716/0 (socket says 172.21.15.44:53716) 2023-09-10T00:55:42.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.790+0000 7fc979a21700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4bac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60596/0 (socket says 172.21.15.44:60596) 2023-09-10T00:55:42.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.790+0000 7fc97aa23700 1 -- 172.21.15.44:0/2917363676 learned_addr learned my addr 172.21.15.44:0/2917363676 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:42.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.790+0000 7fc97aa23700 1 -- 172.21.15.44:0/2917363676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 msgr2=0x55ae8d4bac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:42.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.790+0000 7fc97aa23700 1 --2- 172.21.15.44:0/2917363676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4bac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.798 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.790+0000 7fc97aa23700 1 -- 172.21.15.44:0/2917363676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 msgr2=0x55ae8d4c0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:42.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.790+0000 7fc97aa23700 1 --2- 172.21.15.44:0/2917363676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 0x55ae8d4c0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.790+0000 7fc97aa23700 1 -- 172.21.15.44:0/2917363676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae8d25b250 con 0x55ae8d4b9430 2023-09-10T00:55:42.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.790+0000 7fc97aa23700 1 --2- 172.21.15.44:0/2917363676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b9430 0x55ae8d4bb1c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc96c004b80 tx=0x7fc96c0cb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61fd4b4ef0fe9eb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:42.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.790+0000 7fc96958a700 1 -- 172.21.15.44:0/2917363676 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c012340 con 0x55ae8d4b9430 2023-09-10T00:55:42.799 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.790+0000 7fc96958a700 1 -- 172.21.15.44:0/2917363676 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc96c0995a0 con 0x55ae8d4b9430 2023-09-10T00:55:42.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.790+0000 7fc96958a700 1 -- 172.21.15.44:0/2917363676 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c098390 con 0x55ae8d4b9430 2023-09-10T00:55:42.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.791+0000 7fc98864cb80 1 -- 172.21.15.44:0/2917363676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b9430 msgr2=0x55ae8d4bb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:42.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.791+0000 7fc98864cb80 1 --2- 172.21.15.44:0/2917363676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b9430 0x55ae8d4bb1c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc96c004b80 tx=0x7fc96c0cb040 comp rx=0 tx=0).stop 2023-09-10T00:55:42.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.791+0000 7fc98864cb80 1 -- 172.21.15.44:0/2917363676 shutdown_connections 2023-09-10T00:55:42.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.791+0000 7fc98864cb80 1 --2- 172.21.15.44:0/2917363676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4b9430 0x55ae8d4bb1c0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.800 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.791+0000 7fc98864cb80 1 --2- 172.21.15.44:0/2917363676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 0x55ae8d4c0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.791+0000 7fc98864cb80 1 --2- 172.21.15.44:0/2917363676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4bac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.791+0000 7fc98864cb80 1 -- 172.21.15.44:0/2917363676 >> 172.21.15.44:0/2917363676 conn(0x55ae8d42b090 msgr2=0x55ae8d4b8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:42.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.791+0000 7fc98864cb80 1 -- 172.21.15.44:0/2917363676 shutdown_connections 2023-09-10T00:55:42.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.791+0000 7fc98864cb80 1 -- 172.21.15.44:0/2917363676 wait complete. 2023-09-10T00:55:42.801 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.791+0000 7fc98864cb80 1 -- start start 2023-09-10T00:55:42.802 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.791+0000 7fc98864cb80 1 Processor -- start 2023-09-10T00:55:42.802 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc98864cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 0x55ae8d4c6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.802 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc98864cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.802 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc98864cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c6b00 0x55ae8d528680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.802 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc98864cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8d4c7f40 con 0x55ae8d4c0e40 2023-09-10T00:55:42.803 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc98864cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8d528bc0 con 0x55ae8d4b6710 2023-09-10T00:55:42.803 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc98864cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8d528d50 con 0x55ae8d4c6b00 2023-09-10T00:55:42.803 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc98864cb80 10 client.0 _ll_register_callbacks cb 0x7fc96c094ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:55:42.803 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc979a21700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.803 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc979a21700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4c65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53720/0 (socket says 172.21.15.44:53720) 2023-09-10T00:55:42.803 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc97aa23700 1 --2- 172.21.15.44:0/160694633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c6b00 0x55ae8d528680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc979a21700 1 -- 172.21.15.44:0/160694633 learned_addr learned my addr 172.21.15.44:0/160694633 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:42.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc97a222700 1 --2- 172.21.15.44:0/160694633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 0x55ae8d4c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc979a21700 1 -- 172.21.15.44:0/160694633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c6b00 msgr2=0x55ae8d528680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:42.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc979a21700 1 --2- 172.21.15.44:0/160694633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c6b00 0x55ae8d528680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.804 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc979a21700 1 -- 172.21.15.44:0/160694633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 msgr2=0x55ae8d4c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:42.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc979a21700 1 --2- 172.21.15.44:0/160694633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 0x55ae8d4c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc979a21700 1 -- 172.21.15.44:0/160694633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9741a22e0 con 0x55ae8d4c0e40 2023-09-10T00:55:42.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc979a21700 1 --2- 172.21.15.44:0/160694633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4c65c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc96400b2e0 tx=0x7fc96400fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466794eb5b269bbe server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:42.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc95dffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:55:42.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc95dffb700 1 -- 172.21.15.44:0/160694633 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96400b390 con 0x55ae8d4c0e40 2023-09-10T00:55:42.805 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.792+0000 7fc95dffb700 1 -- 172.21.15.44:0/160694633 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc964000e40 con 0x55ae8d4c0e40 2023-09-10T00:55:42.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.793+0000 7fc98864cb80 1 -- 172.21.15.44:0/160694633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc96401e040 con 0x55ae8d4c0e40 2023-09-10T00:55:42.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.793+0000 7fc95dffb700 1 -- 172.21.15.44:0/160694633 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc964005bc0 con 0x55ae8d4c0e40 2023-09-10T00:55:42.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.793+0000 7fc98864cb80 10 client.17886 fetch_fsmap learned FSMap version 72 2023-09-10T00:55:42.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.793+0000 7fc98864cb80 1 -- 172.21.15.44:0/160694633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55ae8d2332b0 con 0x55ae8d4c0e40 2023-09-10T00:55:42.806 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.793+0000 7fc95dffb700 1 -- 172.21.15.44:0/160694633 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc96404b080 con 0x55ae8d4c0e40 2023-09-10T00:55:42.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.793+0000 7fc95dffb700 1 -- 172.21.15.44:0/160694633 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 72) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fc96402c840 con 0x55ae8d4c0e40 2023-09-10T00:55:42.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.793+0000 7fc98864cb80 10 client.17886 fetch_fsmap finished waiting for FSMap version 72 2023-09-10T00:55:42.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.793+0000 7fc98864cb80 10 client.17886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:55:42.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.793+0000 7fc98864cb80 20 client.17886 populate_metadata read hostname 'smithi044' 2023-09-10T00:55:42.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.793+0000 7fc98864cb80 1 --2- 172.21.15.44:0/160694633 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ae8d52a200 0x55ae8d52c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:42.807 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.793+0000 7fc98864cb80 4 client.17886 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:55:42.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.793+0000 7fc98864cb80 1 -- 172.21.15.44:0/160694633 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ae8d401920 con 0x55ae8d52a200 2023-09-10T00:55:42.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.793+0000 7fc97a222700 1 --2- 172.21.15.44:0/160694633 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ae8d52a200 0x55ae8d52c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:42.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc95dffb700 10 client.17886 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:42.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc97a222700 1 --2- 172.21.15.44:0/160694633 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ae8d52a200 0x55ae8d52c620 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d933b1c73cede974 server_cookie=31948a5d57bf52b in_seq=0 out_seq=0 2023-09-10T00:55:42.808 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc95dffb700 1 -- 172.21.15.44:0/160694633 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ae8d401920 con 0x55ae8d52a200 2023-09-10T00:55:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc95dffb700 10 client.17886 handle_command_reply: tid=0 2023-09-10T00:55:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc98864cb80 1 client.17886 shutdown 2023-09-10T00:55:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc98864cb80 1 -- 172.21.15.44:0/160694633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4c0e40 msgr2=0x55ae8d4c65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc98864cb80 1 --2- 172.21.15.44:0/160694633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4c65c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc96400b2e0 tx=0x7fc96400fa10 comp rx=0 tx=0).stop 2023-09-10T00:55:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc98864cb80 1 -- 172.21.15.44:0/160694633 shutdown_connections 2023-09-10T00:55:42.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc98864cb80 1 --2- 172.21.15.44:0/160694633 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ae8d52a200 0x55ae8d52c620 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc98864cb80 1 --2- 172.21.15.44:0/160694633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8d4c6b00 0x55ae8d528680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc98864cb80 1 --2- 172.21.15.44:0/160694633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae8d4b6710 0x55ae8d4c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc98864cb80 1 --2- 172.21.15.44:0/160694633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae8d4c0e40 0x55ae8d4c65c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.810 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc95dffb700 0 client.17886 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:42.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc98864cb80 1 -- 172.21.15.44:0/160694633 >> 172.21.15.44:0/160694633 conn(0x55ae8d42b090 msgr2=0x55ae8d4b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:42.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc98864cb80 1 -- 172.21.15.44:0/160694633 shutdown_connections 2023-09-10T00:55:42.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc98864cb80 1 -- 172.21.15.44:0/160694633 wait complete. 2023-09-10T00:55:42.811 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.794+0000 7fc98864cb80 20 client.17886 trim_cache size 0 max 16384 2023-09-10T00:55:42.812 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:55:42.812 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:55:42.812 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:55:42.812 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:55:42.812 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:55:42.813 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:55:42.813 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:55:42.813 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:55:42.813 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:55:42.813 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:55:42.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.795+0000 7fc97c486700 1 -- 172.21.15.44:0/4204424434 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc9600a0900 msgr2=0x7fc9600a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:42.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.795+0000 7fc97c486700 1 --2- 172.21.15.44:0/4204424434 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc9600a0900 0x7fc9600a2dc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc970009be0 tx=0x7fc970009260 comp rx=0 tx=0).stop 2023-09-10T00:55:42.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.795+0000 7fc97c486700 1 -- 172.21.15.44:0/4204424434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9741054f0 msgr2=0x7fc974199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:42.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.795+0000 7fc97c486700 1 --2- 172.21.15.44:0/4204424434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9741054f0 0x7fc974199e50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00ef30 tx=0x7fc96c00c3a0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.795+0000 7fc97c486700 1 -- 172.21.15.44:0/4204424434 shutdown_connections 2023-09-10T00:55:42.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.795+0000 7fc97c486700 1 --2- 172.21.15.44:0/4204424434 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc9600a0900 0x7fc9600a2dc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.795+0000 7fc97c486700 1 --2- 172.21.15.44:0/4204424434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9741054f0 0x7fc974199e50 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.795+0000 7fc97c486700 1 --2- 172.21.15.44:0/4204424434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9740ff450 0x7fc9741954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.795+0000 7fc97c486700 1 --2- 172.21.15.44:0/4204424434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9740ffe10 0x7fc9741959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:42.816 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.795+0000 7fc97c486700 1 -- 172.21.15.44:0/4204424434 >> 172.21.15.44:0/4204424434 conn(0x7fc9740faf40 msgr2=0x7fc9740fb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:42.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.795+0000 7fc97c486700 1 -- 172.21.15.44:0/4204424434 shutdown_connections 2023-09-10T00:55:42.817 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:55:42.795+0000 7fc97c486700 1 -- 172.21.15.44:0/4204424434 wait complete. 2023-09-10T00:55:42.817 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:55:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:42 smithi044 ceph-mon[98676]: pgmap v473: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 395 op/s 2023-09-10T00:55:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:42 smithi099 ceph-mon[102692]: pgmap v473: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 395 op/s 2023-09-10T00:55:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.233+0000 7f1da0399700 1 -- 172.21.15.44:0/3396045382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d981055b0 msgr2=0x7f1d98105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.233+0000 7f1da0399700 1 --2- 172.21.15.44:0/3396045382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d981055b0 0x7f1d98105990 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1d880099c0 tx=0x7f1d88009cd0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.233+0000 7f1da0399700 1 -- 172.21.15.44:0/3396045382 shutdown_connections 2023-09-10T00:55:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.233+0000 7f1da0399700 1 --2- 172.21.15.44:0/3396045382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d98100120 0x7f1d9810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.233+0000 7f1da0399700 1 --2- 172.21.15.44:0/3396045382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d981055b0 0x7f1d98105990 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.233+0000 7f1da0399700 1 --2- 172.21.15.44:0/3396045382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d980ff760 0x7f1d980ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.233+0000 7f1da0399700 1 -- 172.21.15.44:0/3396045382 >> 172.21.15.44:0/3396045382 conn(0x7f1d98074f30 msgr2=0x7f1d98075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.233+0000 7f1da0399700 1 -- 172.21.15.44:0/3396045382 shutdown_connections 2023-09-10T00:55:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.233+0000 7f1da0399700 1 -- 172.21.15.44:0/3396045382 wait complete. 2023-09-10T00:55:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.234+0000 7f1da0399700 1 Processor -- start 2023-09-10T00:55:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.234+0000 7f1da0399700 1 -- start start 2023-09-10T00:55:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.234+0000 7f1da0399700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d980ff760 0x7f1d981012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.234+0000 7f1da0399700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d98100120 0x7f1d981017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.234+0000 7f1da0399700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d981055b0 0x7f1d981a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.234+0000 7f1da0399700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d981a4c80 con 0x7f1d98100120 2023-09-10T00:55:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.234+0000 7f1da0399700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d981a4dc0 con 0x7f1d981055b0 2023-09-10T00:55:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.234+0000 7f1da0399700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d981a4f30 con 0x7f1d980ff760 2023-09-10T00:55:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.234+0000 7f1d9d934700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d98100120 0x7f1d981017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.234+0000 7f1d9e135700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d980ff760 0x7f1d981012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.234+0000 7f1d9d934700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d98100120 0x7f1d981017e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53734/0 (socket says 172.21.15.44:53734) 2023-09-10T00:55:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.234+0000 7f1d9e135700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d980ff760 0x7f1d981012a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60614/0 (socket says 172.21.15.44:60614) 2023-09-10T00:55:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.234+0000 7f1d9d934700 1 -- 172.21.15.44:0/3743936214 learned_addr learned my addr 172.21.15.44:0/3743936214 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.234+0000 7f1d9e936700 1 --2- 172.21.15.44:0/3743936214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d981055b0 0x7f1d981a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.235+0000 7f1d9d934700 1 -- 172.21.15.44:0/3743936214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d980ff760 msgr2=0x7f1d981012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.235+0000 7f1d9d934700 1 --2- 172.21.15.44:0/3743936214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d980ff760 0x7f1d981012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.235+0000 7f1d9d934700 1 -- 172.21.15.44:0/3743936214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d981055b0 msgr2=0x7f1d981a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.235+0000 7f1d9d934700 1 --2- 172.21.15.44:0/3743936214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d981055b0 0x7f1d981a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.235+0000 7f1d9d934700 1 -- 172.21.15.44:0/3743936214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d880095f0 con 0x7f1d98100120 2023-09-10T00:55:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.235+0000 7f1d9e135700 1 --2- 172.21.15.44:0/3743936214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d980ff760 0x7f1d981012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:55:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.235+0000 7f1d9d934700 1 --2- 172.21.15.44:0/3743936214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d98100120 0x7f1d981017e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400e9a0 tx=0x7f1d9400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.235+0000 7f1d8f7fe700 1 -- 172.21.15.44:0/3743936214 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94009790 con 0x7f1d98100120 2023-09-10T00:55:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.235+0000 7f1d8f7fe700 1 -- 172.21.15.44:0/3743936214 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d94010430 con 0x7f1d98100120 2023-09-10T00:55:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.235+0000 7f1da0399700 1 -- 172.21.15.44:0/3743936214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d981a5210 con 0x7f1d98100120 2023-09-10T00:55:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.235+0000 7f1d8f7fe700 1 -- 172.21.15.44:0/3743936214 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9400cea0 con 0x7f1d98100120 2023-09-10T00:55:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.235+0000 7f1da0399700 1 -- 172.21.15.44:0/3743936214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d981a5720 con 0x7f1d98100120 2023-09-10T00:55:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.238+0000 7f1d8f7fe700 1 -- 172.21.15.44:0/3743936214 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1d940105a0 con 0x7f1d98100120 2023-09-10T00:55:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.238+0000 7f1d8f7fe700 1 --2- 172.21.15.44:0/3743936214 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1d840a07b0 0x7f1d840a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.239+0000 7f1d9e135700 1 --2- 172.21.15.44:0/3743936214 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1d840a07b0 0x7f1d840a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.238+0000 7f1d8f7fe700 1 -- 172.21.15.44:0/3743936214 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f1d94014070 con 0x7f1d98100120 2023-09-10T00:55:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.239+0000 7f1dac55fb80 1 Processor -- start 2023-09-10T00:55:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.239+0000 7f1dac55fb80 1 -- start start 2023-09-10T00:55:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.239+0000 7f1dac55fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af9126e1a0 0x55af9126e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.239+0000 7f1dac55fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af91268300 0x55af91268780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.239+0000 7f1dac55fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af91268cc0 0x55af91274f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.239+0000 7f1dac55fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af911c16d0 con 0x55af9126e1a0 2023-09-10T00:55:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.239+0000 7f1dac55fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af911beff0 con 0x55af91268cc0 2023-09-10T00:55:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.239+0000 7f1dac55fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af911ba8e0 con 0x55af91268300 2023-09-10T00:55:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.240+0000 7f1d9d934700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af91268300 0x55af91268780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.240+0000 7f1d9d934700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af91268300 0x55af91268780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60624/0 (socket says 172.21.15.44:60624) 2023-09-10T00:55:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.240+0000 7f1d9d934700 1 -- 172.21.15.44:0/3804028697 learned_addr learned my addr 172.21.15.44:0/3804028697 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.240+0000 7f1d9e135700 1 --2- 172.21.15.44:0/3804028697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af9126e1a0 0x55af9126e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.240+0000 7f1d9e936700 1 --2- 172.21.15.44:0/3804028697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af91268cc0 0x55af91274f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.240+0000 7f1d9e135700 1 --2- 172.21.15.44:0/3743936214 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1d840a07b0 0x7f1d840a2c70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800b580 tx=0x7f1d8801a040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.240+0000 7f1d9d934700 1 -- 172.21.15.44:0/3804028697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af91268cc0 msgr2=0x55af91274f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.240+0000 7f1d9d934700 1 --2- 172.21.15.44:0/3804028697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af91268cc0 0x55af91274f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.240+0000 7f1d9d934700 1 -- 172.21.15.44:0/3804028697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af9126e1a0 msgr2=0x55af9126e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.240+0000 7f1d9d934700 1 --2- 172.21.15.44:0/3804028697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af9126e1a0 0x55af9126e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.240+0000 7f1d9d934700 1 -- 172.21.15.44:0/3804028697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af90fe6d80 con 0x55af91268300 2023-09-10T00:55:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.240+0000 7f1d9d934700 1 --2- 172.21.15.44:0/3804028697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af91268300 0x55af91268780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1d940a1130 tx=0x7f1d94098f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d39395e0ae5e9de8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.240+0000 7f1d8cd89700 1 -- 172.21.15.44:0/3804028697 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d940995f0 con 0x55af91268300 2023-09-10T00:55:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.240+0000 7f1d8cd89700 1 -- 172.21.15.44:0/3804028697 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d9409d420 con 0x55af91268300 2023-09-10T00:55:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.240+0000 7f1d8cd89700 1 -- 172.21.15.44:0/3804028697 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d940840b0 con 0x55af91268300 2023-09-10T00:55:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.241+0000 7f1dac55fb80 1 -- 172.21.15.44:0/3804028697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af91268300 msgr2=0x55af91268780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.241+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/3804028697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af91268300 0x55af91268780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1d940a1130 tx=0x7f1d94098f50 comp rx=0 tx=0).stop 2023-09-10T00:55:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.241+0000 7f1dac55fb80 1 -- 172.21.15.44:0/3804028697 shutdown_connections 2023-09-10T00:55:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.241+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/3804028697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af91268cc0 0x55af91274f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.241+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/3804028697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af9126e1a0 0x55af9126e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.241+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/3804028697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af91268300 0x55af91268780 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.241+0000 7f1dac55fb80 1 -- 172.21.15.44:0/3804028697 >> 172.21.15.44:0/3804028697 conn(0x55af911dd030 msgr2=0x55af91263560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.241+0000 7f1dac55fb80 1 -- 172.21.15.44:0/3804028697 shutdown_connections 2023-09-10T00:55:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.241+0000 7f1dac55fb80 1 -- 172.21.15.44:0/3804028697 wait complete. 2023-09-10T00:55:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1dac55fb80 1 -- start start 2023-09-10T00:55:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1dac55fb80 1 Processor -- start 2023-09-10T00:55:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1dac55fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268300 0x55af91272230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1dac55fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af91268cc0 0x55af91272770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1dac55fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af9126e1a0 0x55af91272cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1dac55fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af91158b10 con 0x55af91268300 2023-09-10T00:55:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1dac55fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af91158810 con 0x55af91268cc0 2023-09-10T00:55:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1dac55fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9103c8c0 con 0x55af9126e1a0 2023-09-10T00:55:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1dac55fb80 10 client.0 _ll_register_callbacks cb 0x55af910a2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:55:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1d9e135700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268300 0x55af91272230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1d9e135700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268300 0x55af91272230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53746/0 (socket says 172.21.15.44:53746) 2023-09-10T00:55:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1d9e135700 1 -- 172.21.15.44:0/410158317 learned_addr learned my addr 172.21.15.44:0/410158317 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1d9e936700 1 --2- 172.21.15.44:0/410158317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af9126e1a0 0x55af91272cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1d9d934700 1 --2- 172.21.15.44:0/410158317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af91268cc0 0x55af91272770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1d9e135700 1 -- 172.21.15.44:0/410158317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af9126e1a0 msgr2=0x55af91272cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1d9e135700 1 --2- 172.21.15.44:0/410158317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af9126e1a0 0x55af91272cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1d9e135700 1 -- 172.21.15.44:0/410158317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af91268cc0 msgr2=0x55af91272770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1d9e135700 1 --2- 172.21.15.44:0/410158317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af91268cc0 0x55af91272770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.242+0000 7f1d9e135700 1 -- 172.21.15.44:0/410158317 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d981a5720 con 0x55af91268300 2023-09-10T00:55:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.243+0000 7f1d9e135700 1 --2- 172.21.15.44:0/410158317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268300 0x55af91272230 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800bf80 tx=0x7f1d88009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c55b32e2b780caa server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.243+0000 7f1d81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:55:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.243+0000 7f1d81ffb700 1 -- 172.21.15.44:0/410158317 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8802b040 con 0x55af91268300 2023-09-10T00:55:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.243+0000 7f1d81ffb700 1 -- 172.21.15.44:0/410158317 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f1d88003760 con 0x55af91268300 2023-09-10T00:55:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.243+0000 7f1dac55fb80 1 -- 172.21.15.44:0/410158317 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af91112490 con 0x55af91268300 2023-09-10T00:55:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.243+0000 7f1d81ffb700 1 -- 172.21.15.44:0/410158317 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d880055e0 con 0x55af91268300 2023-09-10T00:55:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.243+0000 7f1dac55fb80 10 client.17898 fetch_fsmap learned FSMap version 72 2023-09-10T00:55:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.243+0000 7f1dac55fb80 1 -- 172.21.15.44:0/410158317 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55af90fcfc00 con 0x55af91268300 2023-09-10T00:55:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.243+0000 7f1d81ffb700 1 -- 172.21.15.44:0/410158317 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55af91112490 con 0x55af91268300 2023-09-10T00:55:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.243+0000 7f1d81ffb700 1 -- 172.21.15.44:0/410158317 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 72) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f1d88030870 con 0x55af91268300 2023-09-10T00:55:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.243+0000 7f1dac55fb80 10 client.17898 fetch_fsmap finished waiting for FSMap version 72 2023-09-10T00:55:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.243+0000 7f1dac55fb80 10 client.17898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:55:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.243+0000 7f1dac55fb80 20 client.17898 populate_metadata read hostname 'smithi044' 2023-09-10T00:55:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.243+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/410158317 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af912824c0 0x55af912848e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.243+0000 7f1dac55fb80 4 client.17898 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:55:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.243+0000 7f1dac55fb80 1 -- 172.21.15.44:0/410158317 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af911cf840 con 0x55af912824c0 2023-09-10T00:55:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.244+0000 7f1d9d934700 1 --2- 172.21.15.44:0/410158317 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af912824c0 0x55af912848e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.244+0000 7f1d9d934700 1 --2- 172.21.15.44:0/410158317 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af912824c0 0x55af912848e0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a727a3619a248a server_cookie=e3f85871fb0fe4a in_seq=0 out_seq=0 2023-09-10T00:55:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.244+0000 7f1d81ffb700 10 client.17898 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.245+0000 7f1d81ffb700 1 -- 172.21.15.44:0/410158317 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55af911cf840 con 0x55af912824c0 2023-09-10T00:55:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.245+0000 7f1d81ffb700 10 client.17898 handle_command_reply: tid=0 2023-09-10T00:55:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.245+0000 7f1dac55fb80 1 client.17898 shutdown 2023-09-10T00:55:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.245+0000 7f1dac55fb80 1 -- 172.21.15.44:0/410158317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268300 msgr2=0x55af91272230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.245+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/410158317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268300 0x55af91272230 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800bf80 tx=0x7f1d88009310 comp rx=0 tx=0).stop 2023-09-10T00:55:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.245+0000 7f1dac55fb80 1 -- 172.21.15.44:0/410158317 shutdown_connections 2023-09-10T00:55:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.245+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/410158317 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af912824c0 0x55af912848e0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.245+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/410158317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af9126e1a0 0x55af91272cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.245+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/410158317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268300 0x55af91272230 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.245+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/410158317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af91268cc0 0x55af91272770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.245+0000 7f1d81ffb700 0 client.17898 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.245+0000 7f1dac55fb80 1 -- 172.21.15.44:0/410158317 >> 172.21.15.44:0/410158317 conn(0x55af911dd030 msgr2=0x55af912627a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.245+0000 7f1dac55fb80 1 -- 172.21.15.44:0/410158317 shutdown_connections 2023-09-10T00:55:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.245+0000 7f1dac55fb80 1 -- 172.21.15.44:0/410158317 wait complete. 2023-09-10T00:55:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.245+0000 7f1dac55fb80 20 client.17898 trim_cache size 0 max 16384 2023-09-10T00:55:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.260+0000 7f1dac55fb80 1 Processor -- start 2023-09-10T00:55:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.260+0000 7f1dac55fb80 1 -- start start 2023-09-10T00:55:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.260+0000 7f1dac55fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af91268300 0x55af9126fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.260+0000 7f1dac55fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268cc0 0x55af91279420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.260+0000 7f1dac55fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af9126e1a0 0x55af91263090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.260+0000 7f1dac55fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af91275230 con 0x55af91268cc0 2023-09-10T00:55:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.260+0000 7f1dac55fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af91264930 con 0x55af91268300 2023-09-10T00:55:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.260+0000 7f1dac55fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af91264d60 con 0x55af9126e1a0 2023-09-10T00:55:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.260+0000 7f1d9d934700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268cc0 0x55af91279420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.260+0000 7f1d9e936700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af9126e1a0 0x55af91263090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.260+0000 7f1d9e135700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af91268300 0x55af9126fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.260+0000 7f1d9e936700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af9126e1a0 0x55af91263090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60640/0 (socket says 172.21.15.44:60640) 2023-09-10T00:55:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.260+0000 7f1d9e936700 1 -- 172.21.15.44:0/3247322248 learned_addr learned my addr 172.21.15.44:0/3247322248 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.260+0000 7f1d9d934700 1 --2- 172.21.15.44:0/3247322248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268cc0 0x55af91279420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53756/0 (socket says 172.21.15.44:53756) 2023-09-10T00:55:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.261+0000 7f1d9d934700 1 -- 172.21.15.44:0/3247322248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af9126e1a0 msgr2=0x55af91263090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.261+0000 7f1d9d934700 1 --2- 172.21.15.44:0/3247322248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af9126e1a0 0x55af91263090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.261+0000 7f1d9d934700 1 -- 172.21.15.44:0/3247322248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af91268300 msgr2=0x55af9126fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.261+0000 7f1d9d934700 1 --2- 172.21.15.44:0/3247322248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af91268300 0x55af9126fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.261+0000 7f1d9d934700 1 -- 172.21.15.44:0/3247322248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af9100d1f0 con 0x55af91268cc0 2023-09-10T00:55:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.261+0000 7f1d9e936700 1 --2- 172.21.15.44:0/3247322248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af9126e1a0 0x55af91263090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:55:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.261+0000 7f1d9d934700 1 --2- 172.21.15.44:0/3247322248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268cc0 0x55af91279420 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x55af9126eac0 tx=0x7f1d94095a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b32032cd4174c55 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.261+0000 7f1d8d58a700 1 -- 172.21.15.44:0/3247322248 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94099670 con 0x55af91268cc0 2023-09-10T00:55:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.261+0000 7f1d8d58a700 1 -- 172.21.15.44:0/3247322248 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d940a1660 con 0x55af91268cc0 2023-09-10T00:55:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.261+0000 7f1d8d58a700 1 -- 172.21.15.44:0/3247322248 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d940cc1b0 con 0x55af91268cc0 2023-09-10T00:55:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.262+0000 7f1dac55fb80 1 -- 172.21.15.44:0/3247322248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268cc0 msgr2=0x55af91279420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.262+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/3247322248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268cc0 0x55af91279420 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x55af9126eac0 tx=0x7f1d94095a60 comp rx=0 tx=0).stop 2023-09-10T00:55:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.262+0000 7f1dac55fb80 1 -- 172.21.15.44:0/3247322248 shutdown_connections 2023-09-10T00:55:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.262+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/3247322248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af9126e1a0 0x55af91263090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.262+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/3247322248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af91268300 0x55af9126fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.262+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/3247322248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268cc0 0x55af91279420 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.262+0000 7f1dac55fb80 1 -- 172.21.15.44:0/3247322248 >> 172.21.15.44:0/3247322248 conn(0x55af911dd030 msgr2=0x55af9126c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.262+0000 7f1dac55fb80 1 -- 172.21.15.44:0/3247322248 shutdown_connections 2023-09-10T00:55:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.262+0000 7f1dac55fb80 1 -- 172.21.15.44:0/3247322248 wait complete. 2023-09-10T00:55:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.262+0000 7f1dac55fb80 1 -- start start 2023-09-10T00:55:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.262+0000 7f1dac55fb80 1 Processor -- start 2023-09-10T00:55:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.262+0000 7f1dac55fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af91268300 0x55af912cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.262+0000 7f1dac55fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268cc0 0x55af912cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.262+0000 7f1dac55fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af9126e1a0 0x55af912ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.262+0000 7f1dac55fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af912ce890 con 0x55af91268cc0 2023-09-10T00:55:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.262+0000 7f1dac55fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af912cea00 con 0x55af9126e1a0 2023-09-10T00:55:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1dac55fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af912cebc0 con 0x55af91268300 2023-09-10T00:55:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1dac55fb80 10 client.0 _ll_register_callbacks cb 0x7f1d8800f7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:55:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1d9d934700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268cc0 0x55af912cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1d9e135700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af91268300 0x55af912cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1d9d934700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268cc0 0x55af912cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53762/0 (socket says 172.21.15.44:53762) 2023-09-10T00:55:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1d9d934700 1 -- 172.21.15.44:0/294333609 learned_addr learned my addr 172.21.15.44:0/294333609 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1d9e936700 1 --2- 172.21.15.44:0/294333609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af9126e1a0 0x55af912ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1d9d934700 1 -- 172.21.15.44:0/294333609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af91268300 msgr2=0x55af912cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1d9d934700 1 --2- 172.21.15.44:0/294333609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af91268300 0x55af912cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1d9d934700 1 -- 172.21.15.44:0/294333609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af9126e1a0 msgr2=0x55af912ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1d9d934700 1 --2- 172.21.15.44:0/294333609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af9126e1a0 0x55af912ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1d9d934700 1 -- 172.21.15.44:0/294333609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d900125e0 con 0x55af91268cc0 2023-09-10T00:55:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1d9d934700 1 --2- 172.21.15.44:0/294333609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268cc0 0x55af912cfb00 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1d94099580 tx=0x7f1d94099670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19d0b1494146afc server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1d81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:55:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1d81ffb700 1 -- 172.21.15.44:0/294333609 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94098f50 con 0x55af91268cc0 2023-09-10T00:55:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1dac55fb80 1 -- 172.21.15.44:0/294333609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d9409d420 con 0x55af91268cc0 2023-09-10T00:55:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1d81ffb700 1 -- 172.21.15.44:0/294333609 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f1d94097990 con 0x55af91268cc0 2023-09-10T00:55:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1d81ffb700 1 -- 172.21.15.44:0/294333609 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d940840b0 con 0x55af91268cc0 2023-09-10T00:55:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1dac55fb80 10 client.17904 fetch_fsmap learned FSMap version 72 2023-09-10T00:55:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.263+0000 7f1dac55fb80 1 -- 172.21.15.44:0/294333609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55af90fe6d80 con 0x55af91268cc0 2023-09-10T00:55:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.264+0000 7f1d81ffb700 1 -- 172.21.15.44:0/294333609 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d9409d420 con 0x55af91268cc0 2023-09-10T00:55:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.264+0000 7f1d81ffb700 1 -- 172.21.15.44:0/294333609 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 72) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f1d94098ae0 con 0x55af91268cc0 2023-09-10T00:55:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.264+0000 7f1dac55fb80 10 client.17904 fetch_fsmap finished waiting for FSMap version 72 2023-09-10T00:55:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.264+0000 7f1dac55fb80 10 client.17904 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:55:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.264+0000 7f1dac55fb80 20 client.17904 populate_metadata read hostname 'smithi044' 2023-09-10T00:55:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.264+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/294333609 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af912d7230 0x55af912d9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.264+0000 7f1dac55fb80 4 client.17904 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:55:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.264+0000 7f1dac55fb80 1 -- 172.21.15.44:0/294333609 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55af911b38c0 con 0x55af912d7230 2023-09-10T00:55:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.264+0000 7f1d9e135700 1 --2- 172.21.15.44:0/294333609 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af912d7230 0x55af912d9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.264+0000 7f1d9e135700 1 --2- 172.21.15.44:0/294333609 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af912d7230 0x55af912d9650 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81f4b444c8fd7a25 server_cookie=d02fd009254a23a3 in_seq=0 out_seq=0 2023-09-10T00:55:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.264+0000 7f1d81ffb700 10 client.17904 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.277+0000 7f1d81ffb700 1 -- 172.21.15.44:0/294333609 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55af911b38c0 con 0x55af912d7230 2023-09-10T00:55:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.277+0000 7f1d81ffb700 10 client.17904 handle_command_reply: tid=0 2023-09-10T00:55:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.277+0000 7f1dac55fb80 1 client.17904 shutdown 2023-09-10T00:55:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.277+0000 7f1dac55fb80 1 -- 172.21.15.44:0/294333609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268cc0 msgr2=0x55af912cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.277+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/294333609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268cc0 0x55af912cfb00 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1d94099580 tx=0x7f1d94099670 comp rx=0 tx=0).stop 2023-09-10T00:55:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.277+0000 7f1dac55fb80 1 -- 172.21.15.44:0/294333609 shutdown_connections 2023-09-10T00:55:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.277+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/294333609 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af912d7230 0x55af912d9650 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.277+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/294333609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af9126e1a0 0x55af912ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.277+0000 7f1d81ffb700 0 client.17904 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.277+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/294333609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af91268300 0x55af912cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.277+0000 7f1dac55fb80 1 --2- 172.21.15.44:0/294333609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af91268cc0 0x55af912cfb00 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.277+0000 7f1dac55fb80 1 -- 172.21.15.44:0/294333609 >> 172.21.15.44:0/294333609 conn(0x55af911dd030 msgr2=0x55af91262fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.278+0000 7f1dac55fb80 1 -- 172.21.15.44:0/294333609 shutdown_connections 2023-09-10T00:55:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.278+0000 7f1dac55fb80 1 -- 172.21.15.44:0/294333609 wait complete. 2023-09-10T00:55:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.278+0000 7f1dac55fb80 20 client.17904 trim_cache size 0 max 16384 2023-09-10T00:55:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:55:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:55:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:55:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:55:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.278+0000 7f1da0399700 1 -- 172.21.15.44:0/3743936214 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1d840a07b0 msgr2=0x7f1d840a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.278+0000 7f1da0399700 1 --2- 172.21.15.44:0/3743936214 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1d840a07b0 0x7f1d840a2c70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800b580 tx=0x7f1d8801a040 comp rx=0 tx=0).stop 2023-09-10T00:55:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.278+0000 7f1da0399700 1 -- 172.21.15.44:0/3743936214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d98100120 msgr2=0x7f1d981017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.278+0000 7f1da0399700 1 --2- 172.21.15.44:0/3743936214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d98100120 0x7f1d981017e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400e9a0 tx=0x7f1d9400ecb0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.278+0000 7f1da0399700 1 -- 172.21.15.44:0/3743936214 shutdown_connections 2023-09-10T00:55:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.278+0000 7f1da0399700 1 --2- 172.21.15.44:0/3743936214 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1d840a07b0 0x7f1d840a2c70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.278+0000 7f1da0399700 1 --2- 172.21.15.44:0/3743936214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d981055b0 0x7f1d981a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.278+0000 7f1da0399700 1 --2- 172.21.15.44:0/3743936214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d980ff760 0x7f1d981012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.278+0000 7f1da0399700 1 --2- 172.21.15.44:0/3743936214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d98100120 0x7f1d981017e0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.278+0000 7f1da0399700 1 -- 172.21.15.44:0/3743936214 >> 172.21.15.44:0/3743936214 conn(0x7f1d98074f30 msgr2=0x7f1d98067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.278+0000 7f1da0399700 1 -- 172.21.15.44:0/3743936214 shutdown_connections 2023-09-10T00:55:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:43.278+0000 7f1da0399700 1 -- 172.21.15.44:0/3743936214 wait complete. 2023-09-10T00:55:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:42 smithi131 ceph-mon[103713]: pgmap v473: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 395 op/s 2023-09-10T00:55:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:44 smithi044 ceph-mon[98676]: pgmap v474: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2023-09-10T00:55:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:44 smithi099 ceph-mon[102692]: pgmap v474: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2023-09-10T00:55:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:44 smithi131 ceph-mon[103713]: pgmap v474: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2023-09-10T00:55:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:46 smithi099 ceph-mon[102692]: pgmap v475: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 411 op/s 2023-09-10T00:55:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:46 smithi131 ceph-mon[103713]: pgmap v475: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 411 op/s 2023-09-10T00:55:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:46 smithi044 ceph-mon[98676]: pgmap v475: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 411 op/s 2023-09-10T00:55:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:48 smithi099 ceph-mon[102692]: pgmap v476: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2023-09-10T00:55:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:48 smithi131 ceph-mon[103713]: pgmap v476: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2023-09-10T00:55:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:48 smithi044 ceph-mon[98676]: pgmap v476: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2023-09-10T00:55:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:50 smithi099 ceph-mon[102692]: pgmap v477: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2023-09-10T00:55:51.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:50 smithi131 ceph-mon[103713]: pgmap v477: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2023-09-10T00:55:51.399 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:50 smithi044 ceph-mon[98676]: pgmap v477: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2023-09-10T00:55:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:52 smithi099 ceph-mon[102692]: pgmap v478: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 430 op/s 2023-09-10T00:55:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:52 smithi131 ceph-mon[103713]: pgmap v478: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 430 op/s 2023-09-10T00:55:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:52 smithi044 ceph-mon[98676]: pgmap v478: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 430 op/s 2023-09-10T00:55:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:54 smithi099 ceph-mon[102692]: pgmap v479: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 291 op/s 2023-09-10T00:55:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:54 smithi131 ceph-mon[103713]: pgmap v479: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 291 op/s 2023-09-10T00:55:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:54 smithi044 ceph-mon[98676]: pgmap v479: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 291 op/s 2023-09-10T00:55:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.414+0000 7f5fb9b77700 1 -- 172.21.15.44:0/866882625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb41055b0 msgr2=0x7f5fb4105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.414+0000 7f5fb9b77700 1 --2- 172.21.15.44:0/866882625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb41055b0 0x7f5fb4105990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400b9a0 tx=0x7f5fa4009260 comp rx=0 tx=0).stop 2023-09-10T00:55:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.414+0000 7f5fb9b77700 1 -- 172.21.15.44:0/866882625 shutdown_connections 2023-09-10T00:55:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.414+0000 7f5fb9b77700 1 --2- 172.21.15.44:0/866882625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fb4100050 0x7f5fb410c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.414+0000 7f5fb9b77700 1 --2- 172.21.15.44:0/866882625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb41055b0 0x7f5fb4105990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.414+0000 7f5fb9b77700 1 --2- 172.21.15.44:0/866882625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb40ff690 0x7f5fb40ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.414+0000 7f5fb9b77700 1 -- 172.21.15.44:0/866882625 >> 172.21.15.44:0/866882625 conn(0x7f5fb4077430 msgr2=0x7f5fb4077840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.414+0000 7f5fb9b77700 1 -- 172.21.15.44:0/866882625 shutdown_connections 2023-09-10T00:55:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.414+0000 7f5fb9b77700 1 -- 172.21.15.44:0/866882625 wait complete. 2023-09-10T00:55:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.415+0000 7f5fb9b77700 1 Processor -- start 2023-09-10T00:55:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.415+0000 7f5fb9b77700 1 -- start start 2023-09-10T00:55:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.415+0000 7f5fb9b77700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb40ff690 0x7f5fb4194310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.415+0000 7f5fb9b77700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fb4100050 0x7f5fb4194850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.415+0000 7f5fb9b77700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb41055b0 0x7f5fb4198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.415+0000 7f5fb9b77700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb4199500 con 0x7f5fb41055b0 2023-09-10T00:55:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.415+0000 7f5fb9b77700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb4199670 con 0x7f5fb40ff690 2023-09-10T00:55:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.415+0000 7f5fb9b77700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb4199830 con 0x7f5fb4100050 2023-09-10T00:55:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.416+0000 7f5fb2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fb4100050 0x7f5fb4194850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.416+0000 7f5fb37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb40ff690 0x7f5fb4194310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.416+0000 7f5fb3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb41055b0 0x7f5fb4198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.416+0000 7f5fb2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fb4100050 0x7f5fb4194850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60658/0 (socket says 172.21.15.44:60658) 2023-09-10T00:55:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.416+0000 7f5fb2ffd700 1 -- 172.21.15.44:0/143492677 learned_addr learned my addr 172.21.15.44:0/143492677 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.416+0000 7f5fb37fe700 1 -- 172.21.15.44:0/143492677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fb4100050 msgr2=0x7f5fb4194850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.416+0000 7f5fb37fe700 1 --2- 172.21.15.44:0/143492677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fb4100050 0x7f5fb4194850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.416+0000 7f5fb37fe700 1 -- 172.21.15.44:0/143492677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb41055b0 msgr2=0x7f5fb4198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.416+0000 7f5fb37fe700 1 --2- 172.21.15.44:0/143492677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb41055b0 0x7f5fb4198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.416+0000 7f5fb37fe700 1 -- 172.21.15.44:0/143492677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa400b5a0 con 0x7f5fb40ff690 2023-09-10T00:55:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.416+0000 7f5fb3fff700 1 --2- 172.21.15.44:0/143492677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb41055b0 0x7f5fb4198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:55:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.416+0000 7f5fb37fe700 1 --2- 172.21.15.44:0/143492677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb40ff690 0x7f5fb4194310 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4000c00 tx=0x7f5fa4005980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.417+0000 7f5fb0ff9700 1 -- 172.21.15.44:0/143492677 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4009b70 con 0x7f5fb40ff690 2023-09-10T00:55:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.417+0000 7f5fb0ff9700 1 -- 172.21.15.44:0/143492677 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5fa4005550 con 0x7f5fb40ff690 2023-09-10T00:55:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.417+0000 7f5fb9b77700 1 -- 172.21.15.44:0/143492677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb4199ab0 con 0x7f5fb40ff690 2023-09-10T00:55:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.417+0000 7f5fb0ff9700 1 -- 172.21.15.44:0/143492677 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4003e10 con 0x7f5fb40ff690 2023-09-10T00:55:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.417+0000 7f5fb9b77700 1 -- 172.21.15.44:0/143492677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb41a1350 con 0x7f5fb40ff690 2023-09-10T00:55:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.419+0000 7f5fb0ff9700 1 -- 172.21.15.44:0/143492677 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f5fa401e420 con 0x7f5fb40ff690 2023-09-10T00:55:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.420+0000 7f5fb0ff9700 1 --2- 172.21.15.44:0/143492677 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5f9c0a0900 0x7f5f9c0a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.420+0000 7f5fb2ffd700 1 --2- 172.21.15.44:0/143492677 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5f9c0a0900 0x7f5f9c0a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.420+0000 7f5fb0ff9700 1 -- 172.21.15.44:0/143492677 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f5fa40cdc40 con 0x7f5fb40ff690 2023-09-10T00:55:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fb2ffd700 1 --2- 172.21.15.44:0/143492677 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5f9c0a0900 0x7f5f9c0a2dc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4195b30 tx=0x7f5fa800f040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fc5d3db80 1 Processor -- start 2023-09-10T00:55:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fc5d3db80 1 -- start start 2023-09-10T00:55:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fc5d3db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd136dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fc5d3db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 0x55afd1367f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fc5d3db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd13684d0 0x55afd1374740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fc5d3db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd12b86d0 con 0x55afd136d9b0 2023-09-10T00:55:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fc5d3db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd12b5ff0 con 0x55afd1367b10 2023-09-10T00:55:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fc5d3db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd12b18e0 con 0x55afd13684d0 2023-09-10T00:55:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fb37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd136dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fb37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd136dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53782/0 (socket says 172.21.15.44:53782) 2023-09-10T00:55:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fb37fe700 1 -- 172.21.15.44:0/142005210 learned_addr learned my addr 172.21.15.44:0/142005210 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fb2ffd700 1 --2- 172.21.15.44:0/142005210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 0x55afd1367f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fb3fff700 1 --2- 172.21.15.44:0/142005210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd13684d0 0x55afd1374740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fb37fe700 1 -- 172.21.15.44:0/142005210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd13684d0 msgr2=0x55afd1374740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fb37fe700 1 --2- 172.21.15.44:0/142005210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd13684d0 0x55afd1374740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fb37fe700 1 -- 172.21.15.44:0/142005210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 msgr2=0x55afd1367f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.421+0000 7f5fb37fe700 1 --2- 172.21.15.44:0/142005210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 0x55afd1367f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.422+0000 7f5fb37fe700 1 -- 172.21.15.44:0/142005210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afd10ddd80 con 0x55afd136d9b0 2023-09-10T00:55:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.422+0000 7f5fb2ffd700 1 --2- 172.21.15.44:0/142005210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 0x55afd1367f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:55:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.422+0000 7f5fb37fe700 1 --2- 172.21.15.44:0/142005210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd136dd90 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4097850 tx=0x7f5fa409a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c744d611d3743272 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.422+0000 7f5f99ffb700 1 -- 172.21.15.44:0/142005210 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa409ade0 con 0x55afd136d9b0 2023-09-10T00:55:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.422+0000 7f5f99ffb700 1 -- 172.21.15.44:0/142005210 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5fa409ec10 con 0x55afd136d9b0 2023-09-10T00:55:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.422+0000 7f5f99ffb700 1 -- 172.21.15.44:0/142005210 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4003680 con 0x55afd136d9b0 2023-09-10T00:55:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.423+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/142005210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 msgr2=0x55afd136dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.423+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/142005210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd136dd90 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4097850 tx=0x7f5fa409a920 comp rx=0 tx=0).stop 2023-09-10T00:55:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.423+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/142005210 shutdown_connections 2023-09-10T00:55:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.423+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/142005210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd13684d0 0x55afd1374740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.423+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/142005210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd136dd90 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.423+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/142005210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 0x55afd1367f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.423+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/142005210 >> 172.21.15.44:0/142005210 conn(0x55afd12d4030 msgr2=0x55afd135a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.423+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/142005210 shutdown_connections 2023-09-10T00:55:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.423+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/142005210 wait complete. 2023-09-10T00:55:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.423+0000 7f5fc5d3db80 1 -- start start 2023-09-10T00:55:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fc5d3db80 1 Processor -- start 2023-09-10T00:55:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fc5d3db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 0x55afd1361a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fc5d3db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd13684d0 0x55afd1361f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fc5d3db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd135c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fc5d3db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd124fb10 con 0x55afd136d9b0 2023-09-10T00:55:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fc5d3db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd124f810 con 0x55afd1367b10 2023-09-10T00:55:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fc5d3db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd11338c0 con 0x55afd13684d0 2023-09-10T00:55:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fc5d3db80 10 client.0 _ll_register_callbacks cb 0x55afd1199a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:55:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fb3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd135c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fb3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd135c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53790/0 (socket says 172.21.15.44:53790) 2023-09-10T00:55:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fb3fff700 1 -- 172.21.15.44:0/4232891330 learned_addr learned my addr 172.21.15.44:0/4232891330 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fb37fe700 1 --2- 172.21.15.44:0/4232891330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 0x55afd1361a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fb37fe700 1 -- 172.21.15.44:0/4232891330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd13684d0 msgr2=0x55afd1361f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T00:55:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fb37fe700 1 --2- 172.21.15.44:0/4232891330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd13684d0 0x55afd1361f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fb37fe700 1 -- 172.21.15.44:0/4232891330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 msgr2=0x55afd135c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fb37fe700 1 --2- 172.21.15.44:0/4232891330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd135c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.424+0000 7f5fb37fe700 1 -- 172.21.15.44:0/4232891330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa800b3a0 con 0x55afd1367b10 2023-09-10T00:55:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5fb3fff700 1 --2- 172.21.15.44:0/4232891330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd135c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:55:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5fb37fe700 1 --2- 172.21.15.44:0/4232891330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 0x55afd1361a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55afd1368a20 tx=0x7f5fa400b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=323f253a564e6940 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:56.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5f92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:55:56.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5f92ffd700 1 -- 172.21.15.44:0/4232891330 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40b8070 con 0x55afd1367b10 2023-09-10T00:55:56.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5f92ffd700 1 -- 172.21.15.44:0/4232891330 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f5fa409ade0 con 0x55afd1367b10 2023-09-10T00:55:56.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/4232891330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afd1209490 con 0x55afd1367b10 2023-09-10T00:55:56.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5f92ffd700 1 -- 172.21.15.44:0/4232891330 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fa40a2dc0 con 0x55afd1367b10 2023-09-10T00:55:56.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5fc5d3db80 10 client.27575 fetch_fsmap learned FSMap version 72 2023-09-10T00:55:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/4232891330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7f5fb41a1350 con 0x55afd1367b10 2023-09-10T00:55:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5f92ffd700 1 -- 172.21.15.44:0/4232891330 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5fa40ab080 con 0x55afd1367b10 2023-09-10T00:55:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5f92ffd700 1 -- 172.21.15.44:0/4232891330 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 72) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f5fa40974f0 con 0x55afd1367b10 2023-09-10T00:55:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5fc5d3db80 10 client.27575 fetch_fsmap finished waiting for FSMap version 72 2023-09-10T00:55:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5fc5d3db80 10 client.27575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:55:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5fc5d3db80 20 client.27575 populate_metadata read hostname 'smithi044' 2023-09-10T00:55:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/4232891330 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55afd1389fa0 0x55afd138c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5fc5d3db80 4 client.27575 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:55:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.425+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/4232891330 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afd12c6840 con 0x55afd1389fa0 2023-09-10T00:55:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.426+0000 7f5fb2ffd700 1 --2- 172.21.15.44:0/4232891330 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55afd1389fa0 0x55afd138c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.426+0000 7f5f92ffd700 10 client.27575 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.426+0000 7f5fb2ffd700 1 --2- 172.21.15.44:0/4232891330 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55afd1389fa0 0x55afd138c390 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb57a49a80937184 server_cookie=1f99ec68e678736b in_seq=0 out_seq=0 2023-09-10T00:55:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.427+0000 7f5f92ffd700 1 -- 172.21.15.44:0/4232891330 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55afd12c6840 con 0x55afd1389fa0 2023-09-10T00:55:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.427+0000 7f5f92ffd700 10 client.27575 handle_command_reply: tid=0 2023-09-10T00:55:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.427+0000 7f5fc5d3db80 1 client.27575 shutdown 2023-09-10T00:55:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.427+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/4232891330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 msgr2=0x55afd1361a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.427+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/4232891330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 0x55afd1361a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55afd1368a20 tx=0x7f5fa400b4f0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.427+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/4232891330 shutdown_connections 2023-09-10T00:55:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.427+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/4232891330 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55afd1389fa0 0x55afd138c390 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.427+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/4232891330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd135c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.427+0000 7f5f92ffd700 0 client.27575 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.427+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/4232891330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 0x55afd1361a10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.427+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/4232891330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd13684d0 0x55afd1361f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.427+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/4232891330 >> 172.21.15.44:0/4232891330 conn(0x55afd12d4030 msgr2=0x55afd13597a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.427+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/4232891330 shutdown_connections 2023-09-10T00:55:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.427+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/4232891330 wait complete. 2023-09-10T00:55:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.427+0000 7f5fc5d3db80 20 client.27575 trim_cache size 0 max 16384 2023-09-10T00:55:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.446+0000 7f5fc5d3db80 1 Processor -- start 2023-09-10T00:55:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.446+0000 7f5fc5d3db80 1 -- start start 2023-09-10T00:55:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.446+0000 7f5fc5d3db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd1367b10 0x55afd1361580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.446+0000 7f5fc5d3db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd13684d0 0x55afd1361ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.446+0000 7f5fc5d3db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd1371830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.446+0000 7f5fc5d3db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd1356fc0 con 0x55afd136d9b0 2023-09-10T00:55:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.446+0000 7f5fc5d3db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd1368a50 con 0x55afd13684d0 2023-09-10T00:55:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.446+0000 7f5fc5d3db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd136e120 con 0x55afd1367b10 2023-09-10T00:55:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.446+0000 7f5fb3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd1371830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.446+0000 7f5fb2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd13684d0 0x55afd1361ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.446+0000 7f5fb37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd1367b10 0x55afd1361580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.446+0000 7f5fb2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd13684d0 0x55afd1361ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55538/0 (socket says 172.21.15.44:55538) 2023-09-10T00:55:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.446+0000 7f5fb2ffd700 1 -- 172.21.15.44:0/2526235638 learned_addr learned my addr 172.21.15.44:0/2526235638 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.446+0000 7f5fb2ffd700 1 -- 172.21.15.44:0/2526235638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd1367b10 msgr2=0x55afd1361580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.447+0000 7f5fb2ffd700 1 --2- 172.21.15.44:0/2526235638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd1367b10 0x55afd1361580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.447+0000 7f5fb2ffd700 1 -- 172.21.15.44:0/2526235638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 msgr2=0x55afd1371830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.447+0000 7f5fb2ffd700 1 --2- 172.21.15.44:0/2526235638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd1371830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.447+0000 7f5fb2ffd700 1 -- 172.21.15.44:0/2526235638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afd10c6c00 con 0x55afd13684d0 2023-09-10T00:55:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.447+0000 7f5fb2ffd700 1 --2- 172.21.15.44:0/2526235638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd13684d0 0x55afd1361ac0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800dfd0 tx=0x7f5fa8018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1963c927e6c7f942 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.447+0000 7f5f9a7fc700 1 -- 172.21.15.44:0/2526235638 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8012040 con 0x55afd13684d0 2023-09-10T00:55:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.447+0000 7f5f9a7fc700 1 -- 172.21.15.44:0/2526235638 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5fa8013040 con 0x55afd13684d0 2023-09-10T00:55:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.447+0000 7f5f9a7fc700 1 -- 172.21.15.44:0/2526235638 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8026440 con 0x55afd13684d0 2023-09-10T00:55:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.448+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/2526235638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd13684d0 msgr2=0x55afd1361ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.448+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/2526235638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd13684d0 0x55afd1361ac0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800dfd0 tx=0x7f5fa8018000 comp rx=0 tx=0).stop 2023-09-10T00:55:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.448+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/2526235638 shutdown_connections 2023-09-10T00:55:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.448+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/2526235638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd1371830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.448+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/2526235638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd1367b10 0x55afd1361580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.448+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/2526235638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd13684d0 0x55afd1361ac0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.448+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/2526235638 >> 172.21.15.44:0/2526235638 conn(0x55afd12d4030 msgr2=0x55afd135a090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.448+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/2526235638 shutdown_connections 2023-09-10T00:55:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.448+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/2526235638 wait complete. 2023-09-10T00:55:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.448+0000 7f5fc5d3db80 1 -- start start 2023-09-10T00:55:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.448+0000 7f5fc5d3db80 1 Processor -- start 2023-09-10T00:55:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fc5d3db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 0x55afd13d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fc5d3db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd13684d0 0x55afd13d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fc5d3db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd13dcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fc5d3db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd13d29d0 con 0x55afd136d9b0 2023-09-10T00:55:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fc5d3db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd13dd140 con 0x55afd1367b10 2023-09-10T00:55:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fc5d3db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd13dd2b0 con 0x55afd13684d0 2023-09-10T00:55:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fc5d3db80 10 client.0 _ll_register_callbacks cb 0x7f5fa800dee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:55:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fb37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 0x55afd13d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fb3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd13dcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fb37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 0x55afd13d1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55540/0 (socket says 172.21.15.44:55540) 2023-09-10T00:55:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fb37fe700 1 -- 172.21.15.44:0/3959732882 learned_addr learned my addr 172.21.15.44:0/3959732882 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:55:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fb2ffd700 1 --2- 172.21.15.44:0/3959732882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd13684d0 0x55afd13d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fb37fe700 1 -- 172.21.15.44:0/3959732882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd13684d0 msgr2=0x55afd13d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fb37fe700 1 --2- 172.21.15.44:0/3959732882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd13684d0 0x55afd13d20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fb37fe700 1 -- 172.21.15.44:0/3959732882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 msgr2=0x55afd13dcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:55:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fb37fe700 1 --2- 172.21.15.44:0/3959732882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd13dcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fb37fe700 1 -- 172.21.15.44:0/3959732882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa8029b20 con 0x55afd1367b10 2023-09-10T00:55:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5fb37fe700 1 --2- 172.21.15.44:0/3959732882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 0x55afd13d1b80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4097850 tx=0x7f5fa40b3260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d7b7a54754b5c4a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:55:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.449+0000 7f5f92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:55:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.450+0000 7f5f92ffd700 1 -- 172.21.15.44:0/3959732882 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40b8070 con 0x55afd1367b10 2023-09-10T00:55:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.450+0000 7f5f92ffd700 1 -- 172.21.15.44:0/3959732882 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f5fa409ade0 con 0x55afd1367b10 2023-09-10T00:55:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.450+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/3959732882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fa409ec10 con 0x55afd1367b10 2023-09-10T00:55:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.450+0000 7f5f92ffd700 1 -- 172.21.15.44:0/3959732882 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fa409a920 con 0x55afd1367b10 2023-09-10T00:55:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.450+0000 7f5fc5d3db80 10 client.27581 fetch_fsmap learned FSMap version 72 2023-09-10T00:55:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.450+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/3959732882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55afd11041f0 con 0x55afd1367b10 2023-09-10T00:55:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.450+0000 7f5f92ffd700 1 -- 172.21.15.44:0/3959732882 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5fa409ec10 con 0x55afd1367b10 2023-09-10T00:55:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.450+0000 7f5f92ffd700 1 -- 172.21.15.44:0/3959732882 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 72) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f5fa40a2930 con 0x55afd1367b10 2023-09-10T00:55:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.450+0000 7f5fc5d3db80 10 client.27581 fetch_fsmap finished waiting for FSMap version 72 2023-09-10T00:55:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.450+0000 7f5fc5d3db80 10 client.27581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:55:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.450+0000 7f5fc5d3db80 20 client.27581 populate_metadata read hostname 'smithi044' 2023-09-10T00:55:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.450+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/3959732882 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55afd13de4f0 0x55afd13e08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:55:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.450+0000 7f5fc5d3db80 4 client.27581 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:55:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.450+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/3959732882 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55afd12aa8c0 con 0x55afd13de4f0 2023-09-10T00:55:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.450+0000 7f5fb2ffd700 1 --2- 172.21.15.44:0/3959732882 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55afd13de4f0 0x55afd13e08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:55:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.451+0000 7f5f92ffd700 10 client.27581 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.451+0000 7f5fb2ffd700 1 --2- 172.21.15.44:0/3959732882 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55afd13de4f0 0x55afd13e08e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d08afc50e74dc8d1 server_cookie=9b9b917416691456 in_seq=0 out_seq=0 2023-09-10T00:55:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.464+0000 7f5f92ffd700 1 -- 172.21.15.44:0/3959732882 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55afd12aa8c0 con 0x55afd13de4f0 2023-09-10T00:55:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.464+0000 7f5f92ffd700 10 client.27581 handle_command_reply: tid=0 2023-09-10T00:55:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.464+0000 7f5fc5d3db80 1 client.27581 shutdown 2023-09-10T00:55:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.464+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/3959732882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 msgr2=0x55afd13d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.464+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/3959732882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 0x55afd13d1b80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4097850 tx=0x7f5fa40b3260 comp rx=0 tx=0).stop 2023-09-10T00:55:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.464+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/3959732882 shutdown_connections 2023-09-10T00:55:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.464+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/3959732882 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55afd13de4f0 0x55afd13e08e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.464+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/3959732882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afd136d9b0 0x55afd13dcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.464+0000 7f5f92ffd700 0 client.27581 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:55:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.464+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/3959732882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55afd1367b10 0x55afd13d1b80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.464+0000 7f5fc5d3db80 1 --2- 172.21.15.44:0/3959732882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afd13684d0 0x55afd13d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.464+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/3959732882 >> 172.21.15.44:0/3959732882 conn(0x55afd12d4030 msgr2=0x55afd1358f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.464+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/3959732882 shutdown_connections 2023-09-10T00:55:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.464+0000 7f5fc5d3db80 1 -- 172.21.15.44:0/3959732882 wait complete. 2023-09-10T00:55:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.464+0000 7f5fc5d3db80 20 client.27581 trim_cache size 0 max 16384 2023-09-10T00:55:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:55:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:55:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:55:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:55:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.465+0000 7f5fb9b77700 1 -- 172.21.15.44:0/143492677 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5f9c0a0900 msgr2=0x7f5f9c0a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.465+0000 7f5fb9b77700 1 --2- 172.21.15.44:0/143492677 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5f9c0a0900 0x7f5f9c0a2dc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4195b30 tx=0x7f5fa800f040 comp rx=0 tx=0).stop 2023-09-10T00:55:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.465+0000 7f5fb9b77700 1 -- 172.21.15.44:0/143492677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb40ff690 msgr2=0x7f5fb4194310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:55:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.465+0000 7f5fb9b77700 1 --2- 172.21.15.44:0/143492677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb40ff690 0x7f5fb4194310 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4000c00 tx=0x7f5fa4005980 comp rx=0 tx=0).stop 2023-09-10T00:55:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.465+0000 7f5fb9b77700 1 -- 172.21.15.44:0/143492677 shutdown_connections 2023-09-10T00:55:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.465+0000 7f5fb9b77700 1 --2- 172.21.15.44:0/143492677 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5f9c0a0900 0x7f5f9c0a2dc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.465+0000 7f5fb9b77700 1 --2- 172.21.15.44:0/143492677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb41055b0 0x7f5fb4198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.465+0000 7f5fb9b77700 1 --2- 172.21.15.44:0/143492677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb40ff690 0x7f5fb4194310 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.465+0000 7f5fb9b77700 1 --2- 172.21.15.44:0/143492677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fb4100050 0x7f5fb4194850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:55:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.465+0000 7f5fb9b77700 1 -- 172.21.15.44:0/143492677 >> 172.21.15.44:0/143492677 conn(0x7f5fb4077430 msgr2=0x7f5fb40fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:55:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.465+0000 7f5fb9b77700 1 -- 172.21.15.44:0/143492677 shutdown_connections 2023-09-10T00:55:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:55:56.465+0000 7f5fb9b77700 1 -- 172.21.15.44:0/143492677 wait complete. 2023-09-10T00:55:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:56 smithi099 ceph-mon[102692]: pgmap v480: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 417 op/s 2023-09-10T00:55:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:56 smithi131 ceph-mon[103713]: pgmap v480: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 417 op/s 2023-09-10T00:55:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:56 smithi044 ceph-mon[98676]: pgmap v480: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 417 op/s 2023-09-10T00:55:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:55:58 smithi099 ceph-mon[102692]: pgmap v481: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2023-09-10T00:55:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:55:58 smithi131 ceph-mon[103713]: pgmap v481: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2023-09-10T00:55:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:55:58 smithi044 ceph-mon[98676]: pgmap v481: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2023-09-10T00:56:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:00 smithi099 ceph-mon[102692]: pgmap v482: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2023-09-10T00:56:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:00 smithi131 ceph-mon[103713]: pgmap v482: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2023-09-10T00:56:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:00 smithi044 ceph-mon[98676]: pgmap v482: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2023-09-10T00:56:03.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:02 smithi099 ceph-mon[102692]: pgmap v483: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 392 op/s 2023-09-10T00:56:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:02 smithi131 ceph-mon[103713]: pgmap v483: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 392 op/s 2023-09-10T00:56:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:02 smithi044 ceph-mon[98676]: pgmap v483: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 392 op/s 2023-09-10T00:56:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:04 smithi099 ceph-mon[102692]: pgmap v484: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 247 op/s 2023-09-10T00:56:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:04 smithi099 ceph-mon[102692]: osdmap e112: 12 total, 12 up, 12 in 2023-09-10T00:56:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:04 smithi131 ceph-mon[103713]: pgmap v484: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 247 op/s 2023-09-10T00:56:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:04 smithi131 ceph-mon[103713]: osdmap e112: 12 total, 12 up, 12 in 2023-09-10T00:56:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:04 smithi044 ceph-mon[98676]: pgmap v484: 129 pgs: 129 active+clean; 9.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 247 op/s 2023-09-10T00:56:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:04 smithi044 ceph-mon[98676]: osdmap e112: 12 total, 12 up, 12 in 2023-09-10T00:56:07.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:06 smithi099 ceph-mon[102692]: pgmap v486: 129 pgs: 129 active+clean; 8.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2023-09-10T00:56:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:06 smithi131 ceph-mon[103713]: pgmap v486: 129 pgs: 129 active+clean; 8.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2023-09-10T00:56:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:06 smithi044 ceph-mon[98676]: pgmap v486: 129 pgs: 129 active+clean; 8.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2023-09-10T00:56:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:08 smithi099 ceph-mon[102692]: pgmap v487: 129 pgs: 129 active+clean; 8.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2023-09-10T00:56:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:08 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:56:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:08 smithi131 ceph-mon[103713]: pgmap v487: 129 pgs: 129 active+clean; 8.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2023-09-10T00:56:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:08 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:56:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:08 smithi044 ceph-mon[98676]: pgmap v487: 129 pgs: 129 active+clean; 8.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2023-09-10T00:56:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:08 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:56:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.617+0000 7ff747974700 1 -- 172.21.15.44:0/2581661829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff740106650 msgr2=0x7ff740106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.617+0000 7ff747974700 1 --2- 172.21.15.44:0/2581661829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff740106650 0x7ff740106a30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009990 tx=0x7ff73c009ca0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.618+0000 7ff747974700 1 -- 172.21.15.44:0/2581661829 shutdown_connections 2023-09-10T00:56:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.618+0000 7ff747974700 1 --2- 172.21.15.44:0/2581661829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff740100f50 0x7ff74010d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.618+0000 7ff747974700 1 --2- 172.21.15.44:0/2581661829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff740106650 0x7ff740106a30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.618+0000 7ff747974700 1 --2- 172.21.15.44:0/2581661829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff740100590 0x7ff740100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.618+0000 7ff747974700 1 -- 172.21.15.44:0/2581661829 >> 172.21.15.44:0/2581661829 conn(0x7ff7400fc060 msgr2=0x7ff7400fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.618+0000 7ff747974700 1 -- 172.21.15.44:0/2581661829 shutdown_connections 2023-09-10T00:56:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.618+0000 7ff747974700 1 -- 172.21.15.44:0/2581661829 wait complete. 2023-09-10T00:56:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff747974700 1 Processor -- start 2023-09-10T00:56:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff747974700 1 -- start start 2023-09-10T00:56:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff747974700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff740100590 0x7ff7401a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff747974700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff740100f50 0x7ff7401a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff747974700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff740106650 0x7ff74019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff747974700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74019cfc0 con 0x7ff740106650 2023-09-10T00:56:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff747974700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74019d130 con 0x7ff740100590 2023-09-10T00:56:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff747974700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74019d2f0 con 0x7ff740100f50 2023-09-10T00:56:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff745710700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff740100590 0x7ff7401a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff744f0f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff740100f50 0x7ff7401a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff744f0f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff740100f50 0x7ff7401a3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60698/0 (socket says 172.21.15.44:60698) 2023-09-10T00:56:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff745710700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff740100590 0x7ff7401a2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55554/0 (socket says 172.21.15.44:55554) 2023-09-10T00:56:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff744f0f700 1 -- 172.21.15.44:0/3937084447 learned_addr learned my addr 172.21.15.44:0/3937084447 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff745f11700 1 --2- 172.21.15.44:0/3937084447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff740106650 0x7ff74019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff745710700 1 -- 172.21.15.44:0/3937084447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff740100f50 msgr2=0x7ff7401a3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff745710700 1 --2- 172.21.15.44:0/3937084447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff740100f50 0x7ff7401a3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff745710700 1 -- 172.21.15.44:0/3937084447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff740106650 msgr2=0x7ff74019c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.619+0000 7ff745710700 1 --2- 172.21.15.44:0/3937084447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff740106650 0x7ff74019c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.620+0000 7ff745710700 1 -- 172.21.15.44:0/3937084447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c0095f0 con 0x7ff740100590 2023-09-10T00:56:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.620+0000 7ff744f0f700 1 --2- 172.21.15.44:0/3937084447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff740100f50 0x7ff7401a3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:56:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.620+0000 7ff745710700 1 --2- 172.21.15.44:0/3937084447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff740100590 0x7ff7401a2af0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff73c000c00 tx=0x7ff73c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.620+0000 7ff7367fc700 1 -- 172.21.15.44:0/3937084447 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c01d070 con 0x7ff740100590 2023-09-10T00:56:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.620+0000 7ff7367fc700 1 -- 172.21.15.44:0/3937084447 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff73c021410 con 0x7ff740100590 2023-09-10T00:56:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.620+0000 7ff747974700 1 -- 172.21.15.44:0/3937084447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74019d570 con 0x7ff740100590 2023-09-10T00:56:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.620+0000 7ff7367fc700 1 -- 172.21.15.44:0/3937084447 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c021ea0 con 0x7ff740100590 2023-09-10T00:56:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.620+0000 7ff747974700 1 -- 172.21.15.44:0/3937084447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7401a9ae0 con 0x7ff740100590 2023-09-10T00:56:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.623+0000 7ff7367fc700 1 -- 172.21.15.44:0/3937084447 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ff73c021580 con 0x7ff740100590 2023-09-10T00:56:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.624+0000 7ff7367fc700 1 --2- 172.21.15.44:0/3937084447 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff72c0a0830 0x7ff72c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.624+0000 7ff7367fc700 1 -- 172.21.15.44:0/3937084447 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9628+0+0 (secure 0 0 0) 0x7ff73c0cd480 con 0x7ff740100590 2023-09-10T00:56:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.624+0000 7ff744f0f700 1 --2- 172.21.15.44:0/3937084447 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff72c0a0830 0x7ff72c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.624+0000 7ff744f0f700 1 --2- 172.21.15.44:0/3937084447 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff72c0a0830 0x7ff72c0a2cf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff7400ff920 tx=0x7ff730009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.625+0000 7ff753b3ab80 1 Processor -- start 2023-09-10T00:56:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.625+0000 7ff753b3ab80 1 -- start start 2023-09-10T00:56:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.625+0000 7ff753b3ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad7e1a0 0x55befad7e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.625+0000 7ff753b3ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad78300 0x55befad78780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.625+0000 7ff753b3ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 0x55befad84f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.625+0000 7ff753b3ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befacd16d0 con 0x55befad78cc0 2023-09-10T00:56:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.625+0000 7ff753b3ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befacceff0 con 0x55befad78300 2023-09-10T00:56:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.626+0000 7ff753b3ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befacca8e0 con 0x55befad7e1a0 2023-09-10T00:56:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.626+0000 7ff744f0f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad78300 0x55befad78780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.626+0000 7ff745f11700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 0x55befad84f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.626+0000 7ff745710700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad7e1a0 0x55befad7e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.626+0000 7ff744f0f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad78300 0x55befad78780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55562/0 (socket says 172.21.15.44:55562) 2023-09-10T00:56:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.626+0000 7ff744f0f700 1 -- 172.21.15.44:0/2394905521 learned_addr learned my addr 172.21.15.44:0/2394905521 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.626+0000 7ff744f0f700 1 -- 172.21.15.44:0/2394905521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad7e1a0 msgr2=0x55befad7e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.626+0000 7ff744f0f700 1 --2- 172.21.15.44:0/2394905521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad7e1a0 0x55befad7e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.626+0000 7ff744f0f700 1 -- 172.21.15.44:0/2394905521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 msgr2=0x55befad84f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.626+0000 7ff744f0f700 1 --2- 172.21.15.44:0/2394905521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 0x55befad84f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.626+0000 7ff744f0f700 1 -- 172.21.15.44:0/2394905521 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55befaaf6d80 con 0x55befad78300 2023-09-10T00:56:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.626+0000 7ff744f0f700 1 --2- 172.21.15.44:0/2394905521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad78300 0x55befad78780 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff730009880 tx=0x7ff730014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e035e0ca61a99ab server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.626+0000 7ff72b7fe700 1 -- 172.21.15.44:0/2394905521 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730026630 con 0x55befad78300 2023-09-10T00:56:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.626+0000 7ff72b7fe700 1 -- 172.21.15.44:0/2394905521 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff730021070 con 0x55befad78300 2023-09-10T00:56:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.626+0000 7ff72b7fe700 1 -- 172.21.15.44:0/2394905521 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73001d670 con 0x55befad78300 2023-09-10T00:56:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.627+0000 7ff753b3ab80 1 -- 172.21.15.44:0/2394905521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad78300 msgr2=0x55befad78780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.627+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/2394905521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad78300 0x55befad78780 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff730009880 tx=0x7ff730014000 comp rx=0 tx=0).stop 2023-09-10T00:56:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.627+0000 7ff753b3ab80 1 -- 172.21.15.44:0/2394905521 shutdown_connections 2023-09-10T00:56:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.627+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/2394905521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 0x55befad84f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.627+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/2394905521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad7e1a0 0x55befad7e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.627+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/2394905521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad78300 0x55befad78780 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.627+0000 7ff753b3ab80 1 -- 172.21.15.44:0/2394905521 >> 172.21.15.44:0/2394905521 conn(0x55befaced030 msgr2=0x55befad73560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.627+0000 7ff753b3ab80 1 -- 172.21.15.44:0/2394905521 shutdown_connections 2023-09-10T00:56:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.627+0000 7ff753b3ab80 1 -- 172.21.15.44:0/2394905521 wait complete. 2023-09-10T00:56:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.628+0000 7ff753b3ab80 1 -- start start 2023-09-10T00:56:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.628+0000 7ff753b3ab80 1 Processor -- start 2023-09-10T00:56:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.628+0000 7ff753b3ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 0x55befad84e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.628+0000 7ff753b3ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad78cc0 0x55befad82080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.628+0000 7ff753b3ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad7e1a0 0x55befad82690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.628+0000 7ff753b3ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befac68b10 con 0x55befad7e1a0 2023-09-10T00:56:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.628+0000 7ff753b3ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befac68810 con 0x55befad78cc0 2023-09-10T00:56:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.628+0000 7ff753b3ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befab4c8c0 con 0x55befad78300 2023-09-10T00:56:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff753b3ab80 10 client.0 _ll_register_callbacks cb 0x55befabb2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:56:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff744f0f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad78cc0 0x55befad82080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff745710700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 0x55befad84e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff744f0f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad78cc0 0x55befad82080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55570/0 (socket says 172.21.15.44:55570) 2023-09-10T00:56:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff744f0f700 1 -- 172.21.15.44:0/1143817784 learned_addr learned my addr 172.21.15.44:0/1143817784 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff745710700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 0x55befad84e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60710/0 (socket says 172.21.15.44:60710) 2023-09-10T00:56:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff745f11700 1 --2- 172.21.15.44:0/1143817784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad7e1a0 0x55befad82690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff744f0f700 1 -- 172.21.15.44:0/1143817784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 msgr2=0x55befad84e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff744f0f700 1 --2- 172.21.15.44:0/1143817784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 0x55befad84e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff744f0f700 1 -- 172.21.15.44:0/1143817784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad7e1a0 msgr2=0x55befad82690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff744f0f700 1 --2- 172.21.15.44:0/1143817784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad7e1a0 0x55befad82690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff744f0f700 1 -- 172.21.15.44:0/1143817784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff730017070 con 0x55befad78cc0 2023-09-10T00:56:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff744f0f700 1 --2- 172.21.15.44:0/1143817784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad78cc0 0x55befad82080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff730011c70 tx=0x7ff73000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a64509ee640c6089 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff728ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:56:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff728ff9700 1 -- 172.21.15.44:0/1143817784 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73001c410 con 0x55befad78cc0 2023-09-10T00:56:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff728ff9700 1 -- 172.21.15.44:0/1143817784 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9628+0+0 (secure 0 0 0) 0x7ff73001c570 con 0x55befad78cc0 2023-09-10T00:56:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.629+0000 7ff753b3ab80 1 -- 172.21.15.44:0/1143817784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55befac22490 con 0x55befad78cc0 2023-09-10T00:56:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.630+0000 7ff728ff9700 1 -- 172.21.15.44:0/1143817784 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff73001c750 con 0x55befad78cc0 2023-09-10T00:56:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.630+0000 7ff753b3ab80 10 client.27593 fetch_fsmap learned FSMap version 72 2023-09-10T00:56:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.630+0000 7ff753b3ab80 1 -- 172.21.15.44:0/1143817784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55befaadfc00 con 0x55befad78cc0 2023-09-10T00:56:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.630+0000 7ff728ff9700 1 -- 172.21.15.44:0/1143817784 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55befac22490 con 0x55befad78cc0 2023-09-10T00:56:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.630+0000 7ff728ff9700 1 -- 172.21.15.44:0/1143817784 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 72) v1 ==== 7922+0+0 (secure 0 0 0) 0x7ff73002d020 con 0x55befad78cc0 2023-09-10T00:56:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.630+0000 7ff753b3ab80 10 client.27593 fetch_fsmap finished waiting for FSMap version 72 2023-09-10T00:56:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.630+0000 7ff753b3ab80 10 client.27593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:56:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.630+0000 7ff753b3ab80 20 client.27593 populate_metadata read hostname 'smithi044' 2023-09-10T00:56:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.630+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/1143817784 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55befada33a0 0x55befada57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.630+0000 7ff753b3ab80 4 client.27593 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:56:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.630+0000 7ff753b3ab80 1 -- 172.21.15.44:0/1143817784 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55befacdf840 con 0x55befada33a0 2023-09-10T00:56:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.630+0000 7ff745710700 1 --2- 172.21.15.44:0/1143817784 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55befada33a0 0x55befada57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.631+0000 7ff728ff9700 10 client.27593 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.631+0000 7ff745710700 1 --2- 172.21.15.44:0/1143817784 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55befada33a0 0x55befada57c0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a97af375e8227d25 server_cookie=def6e656cb408f58 in_seq=0 out_seq=0 2023-09-10T00:56:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.631+0000 7ff728ff9700 1 -- 172.21.15.44:0/1143817784 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55befacdf840 con 0x55befada33a0 2023-09-10T00:56:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.631+0000 7ff728ff9700 10 client.27593 handle_command_reply: tid=0 2023-09-10T00:56:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.631+0000 7ff753b3ab80 1 client.27593 shutdown 2023-09-10T00:56:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.632+0000 7ff753b3ab80 1 -- 172.21.15.44:0/1143817784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad78cc0 msgr2=0x55befad82080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.632+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/1143817784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad78cc0 0x55befad82080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff730011c70 tx=0x7ff73000ee40 comp rx=0 tx=0).stop 2023-09-10T00:56:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.632+0000 7ff753b3ab80 1 -- 172.21.15.44:0/1143817784 shutdown_connections 2023-09-10T00:56:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.632+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/1143817784 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55befada33a0 0x55befada57c0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.632+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/1143817784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad7e1a0 0x55befad82690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.632+0000 7ff728ff9700 0 client.27593 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.632+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/1143817784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 0x55befad84e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.632+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/1143817784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad78cc0 0x55befad82080 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.632+0000 7ff753b3ab80 1 -- 172.21.15.44:0/1143817784 >> 172.21.15.44:0/1143817784 conn(0x55befaced030 msgr2=0x55befad727a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.632+0000 7ff753b3ab80 1 -- 172.21.15.44:0/1143817784 shutdown_connections 2023-09-10T00:56:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.632+0000 7ff753b3ab80 1 -- 172.21.15.44:0/1143817784 wait complete. 2023-09-10T00:56:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.632+0000 7ff753b3ab80 20 client.27593 trim_cache size 0 max 16384 2023-09-10T00:56:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.656+0000 7ff753b3ab80 1 Processor -- start 2023-09-10T00:56:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.656+0000 7ff753b3ab80 1 -- start start 2023-09-10T00:56:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.656+0000 7ff753b3ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 0x55befad7d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.656+0000 7ff753b3ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 0x55befad7d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.656+0000 7ff753b3ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad7e1a0 0x55befad7c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.656+0000 7ff753b3ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befad82c70 con 0x55befad78cc0 2023-09-10T00:56:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.656+0000 7ff753b3ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befad83660 con 0x55befad7e1a0 2023-09-10T00:56:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.656+0000 7ff753b3ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befad7bdb0 con 0x55befad78300 2023-09-10T00:56:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.656+0000 7ff744f0f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 0x55befad7d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.656+0000 7ff745710700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 0x55befad7d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.656+0000 7ff744f0f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 0x55befad7d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53836/0 (socket says 172.21.15.44:53836) 2023-09-10T00:56:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.656+0000 7ff745f11700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad7e1a0 0x55befad7c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.656+0000 7ff744f0f700 1 -- 172.21.15.44:0/1456202209 learned_addr learned my addr 172.21.15.44:0/1456202209 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.656+0000 7ff745710700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 0x55befad7d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60718/0 (socket says 172.21.15.44:60718) 2023-09-10T00:56:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.657+0000 7ff745f11700 1 -- 172.21.15.44:0/1456202209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 msgr2=0x55befad7d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.657+0000 7ff745f11700 1 --2- 172.21.15.44:0/1456202209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 0x55befad7d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.657+0000 7ff745f11700 1 -- 172.21.15.44:0/1456202209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 msgr2=0x55befad7d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.657+0000 7ff745f11700 1 --2- 172.21.15.44:0/1456202209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 0x55befad7d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.657+0000 7ff745f11700 1 -- 172.21.15.44:0/1456202209 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55befab1d1f0 con 0x55befad7e1a0 2023-09-10T00:56:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.657+0000 7ff744f0f700 1 --2- 172.21.15.44:0/1456202209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 0x55befad7d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:56:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.657+0000 7ff745f11700 1 --2- 172.21.15.44:0/1456202209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad7e1a0 0x55befad7c290 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff73800dfd0 tx=0x7ff73800f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70dcbca9945b0182 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.657+0000 7ff72bfff700 1 -- 172.21.15.44:0/1456202209 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738012040 con 0x55befad7e1a0 2023-09-10T00:56:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.657+0000 7ff72bfff700 1 -- 172.21.15.44:0/1456202209 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff738018ad0 con 0x55befad7e1a0 2023-09-10T00:56:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.657+0000 7ff72bfff700 1 -- 172.21.15.44:0/1456202209 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738008b00 con 0x55befad7e1a0 2023-09-10T00:56:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.658+0000 7ff753b3ab80 1 -- 172.21.15.44:0/1456202209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad7e1a0 msgr2=0x55befad7c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.658+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/1456202209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad7e1a0 0x55befad7c290 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff73800dfd0 tx=0x7ff73800f040 comp rx=0 tx=0).stop 2023-09-10T00:56:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.658+0000 7ff753b3ab80 1 -- 172.21.15.44:0/1456202209 shutdown_connections 2023-09-10T00:56:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.658+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/1456202209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad7e1a0 0x55befad7c290 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.658+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/1456202209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 0x55befad7d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.658+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/1456202209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 0x55befad7d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.658+0000 7ff753b3ab80 1 -- 172.21.15.44:0/1456202209 >> 172.21.15.44:0/1456202209 conn(0x55befaced030 msgr2=0x55befad9f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.658+0000 7ff753b3ab80 1 -- 172.21.15.44:0/1456202209 shutdown_connections 2023-09-10T00:56:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.658+0000 7ff753b3ab80 1 -- 172.21.15.44:0/1456202209 wait complete. 2023-09-10T00:56:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.658+0000 7ff753b3ab80 1 -- start start 2023-09-10T00:56:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.658+0000 7ff753b3ab80 1 Processor -- start 2023-09-10T00:56:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff753b3ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 0x55befad99ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff753b3ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 0x55befad9a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff753b3ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad7e1a0 0x55befad982e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff753b3ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befad989b0 con 0x55befad78cc0 2023-09-10T00:56:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff753b3ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befad98b20 con 0x55befad7e1a0 2023-09-10T00:56:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff753b3ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befad98ce0 con 0x55befad78300 2023-09-10T00:56:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff753b3ab80 10 client.0 _ll_register_callbacks cb 0x7ff73800ba10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:56:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff745710700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 0x55befad99ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff745710700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 0x55befad99ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60722/0 (socket says 172.21.15.44:60722) 2023-09-10T00:56:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff745710700 1 -- 172.21.15.44:0/4248762652 learned_addr learned my addr 172.21.15.44:0/4248762652 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff744f0f700 1 --2- 172.21.15.44:0/4248762652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 0x55befad9a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff745f11700 1 --2- 172.21.15.44:0/4248762652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad7e1a0 0x55befad982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff745710700 1 -- 172.21.15.44:0/4248762652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad7e1a0 msgr2=0x55befad982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff745710700 1 --2- 172.21.15.44:0/4248762652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad7e1a0 0x55befad982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff745710700 1 -- 172.21.15.44:0/4248762652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 msgr2=0x55befad9a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff745710700 1 --2- 172.21.15.44:0/4248762652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 0x55befad9a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.659+0000 7ff745710700 1 -- 172.21.15.44:0/4248762652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55befaadfc00 con 0x55befad78300 2023-09-10T00:56:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.660+0000 7ff745710700 1 --2- 172.21.15.44:0/4248762652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 0x55befad99ed0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00c010 tx=0x7ff73c0cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5882c83b02457371 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.660+0000 7ff728ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:56:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.660+0000 7ff728ff9700 1 -- 172.21.15.44:0/4248762652 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0cd120 con 0x55befad78300 2023-09-10T00:56:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.660+0000 7ff753b3ab80 1 -- 172.21.15.44:0/4248762652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff730021070 con 0x55befad78300 2023-09-10T00:56:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.660+0000 7ff728ff9700 1 -- 172.21.15.44:0/4248762652 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9628+0+0 (secure 0 0 0) 0x7ff73c0cd280 con 0x55befad78300 2023-09-10T00:56:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.660+0000 7ff728ff9700 1 -- 172.21.15.44:0/4248762652 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff73c021860 con 0x55befad78300 2023-09-10T00:56:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.660+0000 7ff753b3ab80 10 client.27532 fetch_fsmap learned FSMap version 72 2023-09-10T00:56:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.660+0000 7ff753b3ab80 1 -- 172.21.15.44:0/4248762652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7ff73800b1c0 con 0x55befad78300 2023-09-10T00:56:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.661+0000 7ff728ff9700 1 -- 172.21.15.44:0/4248762652 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff730021070 con 0x55befad78300 2023-09-10T00:56:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.661+0000 7ff728ff9700 1 -- 172.21.15.44:0/4248762652 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 72) v1 ==== 7922+0+0 (secure 0 0 0) 0x7ff73c099b80 con 0x55befad78300 2023-09-10T00:56:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.661+0000 7ff753b3ab80 10 client.27532 fetch_fsmap finished waiting for FSMap version 72 2023-09-10T00:56:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.661+0000 7ff753b3ab80 10 client.27532 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:56:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.661+0000 7ff753b3ab80 20 client.27532 populate_metadata read hostname 'smithi044' 2023-09-10T00:56:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.661+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/4248762652 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55befada33a0 0x55befadfa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.661+0000 7ff753b3ab80 4 client.27532 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:56:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.661+0000 7ff753b3ab80 1 -- 172.21.15.44:0/4248762652 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55befacc38c0 con 0x55befada33a0 2023-09-10T00:56:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.661+0000 7ff744f0f700 1 --2- 172.21.15.44:0/4248762652 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55befada33a0 0x55befadfa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.661+0000 7ff728ff9700 10 client.27532 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.661+0000 7ff744f0f700 1 --2- 172.21.15.44:0/4248762652 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55befada33a0 0x55befadfa060 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=487bdaa8d4b2bf27 server_cookie=364c32d3b009e847 in_seq=0 out_seq=0 2023-09-10T00:56:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.671+0000 7ff728ff9700 1 -- 172.21.15.44:0/4248762652 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55befacc38c0 con 0x55befada33a0 2023-09-10T00:56:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.671+0000 7ff728ff9700 10 client.27532 handle_command_reply: tid=0 2023-09-10T00:56:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.671+0000 7ff753b3ab80 1 client.27532 shutdown 2023-09-10T00:56:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.671+0000 7ff753b3ab80 1 -- 172.21.15.44:0/4248762652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 msgr2=0x55befad99ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.671+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/4248762652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 0x55befad99ed0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00c010 tx=0x7ff73c0cc040 comp rx=0 tx=0).stop 2023-09-10T00:56:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.671+0000 7ff753b3ab80 1 -- 172.21.15.44:0/4248762652 shutdown_connections 2023-09-10T00:56:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.671+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/4248762652 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55befada33a0 0x55befadfa060 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.671+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/4248762652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55befad7e1a0 0x55befad982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.671+0000 7ff728ff9700 0 client.27532 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.671+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/4248762652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55befad78300 0x55befad99ed0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.671+0000 7ff753b3ab80 1 --2- 172.21.15.44:0/4248762652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55befad78cc0 0x55befad9a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.671+0000 7ff753b3ab80 1 -- 172.21.15.44:0/4248762652 >> 172.21.15.44:0/4248762652 conn(0x55befaced030 msgr2=0x55befad7c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.672+0000 7ff753b3ab80 1 -- 172.21.15.44:0/4248762652 shutdown_connections 2023-09-10T00:56:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.672+0000 7ff753b3ab80 1 -- 172.21.15.44:0/4248762652 wait complete. 2023-09-10T00:56:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.672+0000 7ff753b3ab80 20 client.27532 trim_cache size 0 max 16384 2023-09-10T00:56:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:56:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:56:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:56:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:56:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.673+0000 7ff747974700 1 -- 172.21.15.44:0/3937084447 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff72c0a0830 msgr2=0x7ff72c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.673+0000 7ff747974700 1 --2- 172.21.15.44:0/3937084447 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff72c0a0830 0x7ff72c0a2cf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff7400ff920 tx=0x7ff730009260 comp rx=0 tx=0).stop 2023-09-10T00:56:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.673+0000 7ff747974700 1 -- 172.21.15.44:0/3937084447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff740100590 msgr2=0x7ff7401a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.673+0000 7ff747974700 1 --2- 172.21.15.44:0/3937084447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff740100590 0x7ff7401a2af0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff73c000c00 tx=0x7ff73c005740 comp rx=0 tx=0).stop 2023-09-10T00:56:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.673+0000 7ff747974700 1 -- 172.21.15.44:0/3937084447 shutdown_connections 2023-09-10T00:56:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.673+0000 7ff747974700 1 --2- 172.21.15.44:0/3937084447 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff72c0a0830 0x7ff72c0a2cf0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.673+0000 7ff747974700 1 --2- 172.21.15.44:0/3937084447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff740106650 0x7ff74019c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.673+0000 7ff747974700 1 --2- 172.21.15.44:0/3937084447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff740100590 0x7ff7401a2af0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.673+0000 7ff747974700 1 --2- 172.21.15.44:0/3937084447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff740100f50 0x7ff7401a3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.673+0000 7ff747974700 1 -- 172.21.15.44:0/3937084447 >> 172.21.15.44:0/3937084447 conn(0x7ff7400fc060 msgr2=0x7ff7400fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.673+0000 7ff747974700 1 -- 172.21.15.44:0/3937084447 shutdown_connections 2023-09-10T00:56:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:09.673+0000 7ff747974700 1 -- 172.21.15.44:0/3937084447 wait complete. 2023-09-10T00:56:10.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:09 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:56:10.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:09 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:56:10.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:09 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:56:10.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:09 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:56:10.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:09 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:56:10.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:09 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:56:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:09 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:56:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:09 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:56:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:09 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:56:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:10 smithi131 ceph-mon[103713]: pgmap v488: 129 pgs: 129 active+clean; 8.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2023-09-10T00:56:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:10 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:10 smithi131 ceph-mon[103713]: osdmap e113: 12 total, 12 up, 12 in 2023-09-10T00:56:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:10 smithi044 ceph-mon[98676]: pgmap v488: 129 pgs: 129 active+clean; 8.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2023-09-10T00:56:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:10 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:10 smithi044 ceph-mon[98676]: osdmap e113: 12 total, 12 up, 12 in 2023-09-10T00:56:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:10 smithi099 ceph-mon[102692]: pgmap v488: 129 pgs: 129 active+clean; 8.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2023-09-10T00:56:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:10 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:10 smithi099 ceph-mon[102692]: osdmap e113: 12 total, 12 up, 12 in 2023-09-10T00:56:12.818 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-10T00:56:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:12 smithi131 ceph-mon[103713]: pgmap v490: 129 pgs: 129 active+clean; 8.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 435 op/s 2023-09-10T00:56:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:12 smithi044 ceph-mon[98676]: pgmap v490: 129 pgs: 129 active+clean; 8.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 435 op/s 2023-09-10T00:56:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:12 smithi099 ceph-mon[102692]: pgmap v490: 129 pgs: 129 active+clean; 8.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 435 op/s 2023-09-10T00:56:13.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.389+0000 7f7d055da700 1 -- 172.21.15.44:0/27851071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d000ffe00 msgr2=0x7f7d0010c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:13.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.389+0000 7f7d055da700 1 --2- 172.21.15.44:0/27851071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d000ffe00 0x7f7d0010c310 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800b0a0 tx=0x7f7cf800b3b0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.389+0000 7f7d055da700 1 -- 172.21.15.44:0/27851071 shutdown_connections 2023-09-10T00:56:13.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.389+0000 7f7d055da700 1 --2- 172.21.15.44:0/27851071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d000ffe00 0x7f7d0010c310 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.389+0000 7f7d055da700 1 --2- 172.21.15.44:0/27851071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d001054e0 0x7f7d001058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.389+0000 7f7d055da700 1 --2- 172.21.15.44:0/27851071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d000ff440 0x7f7d000ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.389+0000 7f7d055da700 1 -- 172.21.15.44:0/27851071 >> 172.21.15.44:0/27851071 conn(0x7f7d000faf50 msgr2=0x7f7d000fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:13.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.389+0000 7f7d055da700 1 -- 172.21.15.44:0/27851071 shutdown_connections 2023-09-10T00:56:13.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.389+0000 7f7d055da700 1 -- 172.21.15.44:0/27851071 wait complete. 2023-09-10T00:56:13.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.390+0000 7f7d055da700 1 Processor -- start 2023-09-10T00:56:13.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.390+0000 7f7d055da700 1 -- start start 2023-09-10T00:56:13.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.390+0000 7f7d055da700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d000ff440 0x7f7d001966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.390+0000 7f7d055da700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d000ffe00 0x7f7d00196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.390+0000 7f7d055da700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d001054e0 0x7f7d00199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.390+0000 7f7d055da700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0019a5e0 con 0x7f7d000ffe00 2023-09-10T00:56:13.395 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.390+0000 7f7d055da700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0019a750 con 0x7f7d001054e0 2023-09-10T00:56:13.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.390+0000 7f7d055da700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0019a910 con 0x7f7d000ff440 2023-09-10T00:56:13.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.390+0000 7f7cfe7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d000ffe00 0x7f7d00196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.390+0000 7f7cfeffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d000ff440 0x7f7d001966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.390+0000 7f7cfe7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d000ffe00 0x7f7d00196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53856/0 (socket says 172.21.15.44:53856) 2023-09-10T00:56:13.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.390+0000 7f7cfeffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d000ff440 0x7f7d001966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60736/0 (socket says 172.21.15.44:60736) 2023-09-10T00:56:13.396 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.390+0000 7f7cfe7fc700 1 -- 172.21.15.44:0/4172130832 learned_addr learned my addr 172.21.15.44:0/4172130832 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:13.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.390+0000 7f7cff7fe700 1 --2- 172.21.15.44:0/4172130832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d001054e0 0x7f7d00199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.391+0000 7f7cfe7fc700 1 -- 172.21.15.44:0/4172130832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d000ff440 msgr2=0x7f7d001966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:13.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.391+0000 7f7cfe7fc700 1 --2- 172.21.15.44:0/4172130832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d000ff440 0x7f7d001966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.391+0000 7f7cfe7fc700 1 -- 172.21.15.44:0/4172130832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d001054e0 msgr2=0x7f7d00199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:13.397 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.391+0000 7f7cfe7fc700 1 --2- 172.21.15.44:0/4172130832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d001054e0 0x7f7d00199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.391+0000 7f7cfe7fc700 1 -- 172.21.15.44:0/4172130832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf0009520 con 0x7f7d000ffe00 2023-09-10T00:56:13.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.391+0000 7f7cff7fe700 1 --2- 172.21.15.44:0/4172130832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d001054e0 0x7f7d00199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:56:13.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.391+0000 7f7cfe7fc700 1 --2- 172.21.15.44:0/4172130832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d000ffe00 0x7f7d00196c30 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400e9a0 tx=0x7f7cf400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:13.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.391+0000 7f7ce7fff700 1 -- 172.21.15.44:0/4172130832 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4009790 con 0x7f7d000ffe00 2023-09-10T00:56:13.398 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.391+0000 7f7ce7fff700 1 -- 172.21.15.44:0/4172130832 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7cf4010430 con 0x7f7d000ffe00 2023-09-10T00:56:13.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.391+0000 7f7d055da700 1 -- 172.21.15.44:0/4172130832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf8009d00 con 0x7f7d000ffe00 2023-09-10T00:56:13.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.391+0000 7f7ce7fff700 1 -- 172.21.15.44:0/4172130832 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf400cea0 con 0x7f7d000ffe00 2023-09-10T00:56:13.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.391+0000 7f7d055da700 1 -- 172.21.15.44:0/4172130832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d001a3370 con 0x7f7d000ffe00 2023-09-10T00:56:13.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.394+0000 7f7ce7fff700 1 -- 172.21.15.44:0/4172130832 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f7cf40105a0 con 0x7f7d000ffe00 2023-09-10T00:56:13.404 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.394+0000 7f7ce7fff700 1 --2- 172.21.15.44:0/4172130832 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7ce80a0830 0x7f7ce80a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.394+0000 7f7ce7fff700 1 -- 172.21.15.44:0/4172130832 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f7cf4014070 con 0x7f7d000ffe00 2023-09-10T00:56:13.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.395+0000 7f7cfeffd700 1 --2- 172.21.15.44:0/4172130832 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7ce80a0830 0x7f7ce80a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.395+0000 7f7cfeffd700 1 --2- 172.21.15.44:0/4172130832 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7ce80a0830 0x7f7ce80a2cf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009d70 tx=0x7f7cf0009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:13.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.395+0000 7f7d117a0b80 1 Processor -- start 2023-09-10T00:56:13.405 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.395+0000 7f7d117a0b80 1 -- start start 2023-09-10T00:56:13.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.395+0000 7f7d117a0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f2710 0x55c54a2f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.395+0000 7f7d117a0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f3030 0x55c54a2fc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.395+0000 7f7d117a0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 0x55c54a2ff230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.395+0000 7f7d117a0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54a24b730 con 0x55c54a2f2710 2023-09-10T00:56:13.406 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.395+0000 7f7d117a0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54a249050 con 0x55c54a2fce40 2023-09-10T00:56:13.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.395+0000 7f7d117a0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54a244940 con 0x55c54a2f3030 2023-09-10T00:56:13.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.396+0000 7f7cfeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f2710 0x55c54a2f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.396+0000 7f7cfeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f2710 0x55c54a2f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53862/0 (socket says 172.21.15.44:53862) 2023-09-10T00:56:13.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.396+0000 7f7cfe7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f3030 0x55c54a2fc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.396+0000 7f7cfeffd700 1 -- 172.21.15.44:0/2609230975 learned_addr learned my addr 172.21.15.44:0/2609230975 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:13.407 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.396+0000 7f7cff7fe700 1 --2- 172.21.15.44:0/2609230975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 0x55c54a2ff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.408 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.396+0000 7f7cfe7fc700 1 -- 172.21.15.44:0/2609230975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 msgr2=0x55c54a2ff230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:13.408 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.396+0000 7f7cfe7fc700 1 --2- 172.21.15.44:0/2609230975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 0x55c54a2ff230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.408 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.396+0000 7f7cfe7fc700 1 -- 172.21.15.44:0/2609230975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f2710 msgr2=0x55c54a2f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:13.408 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.396+0000 7f7cfe7fc700 1 --2- 172.21.15.44:0/2609230975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f2710 0x55c54a2f2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.408 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.396+0000 7f7cfe7fc700 1 -- 172.21.15.44:0/2609230975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c54a070de0 con 0x55c54a2f3030 2023-09-10T00:56:13.409 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.396+0000 7f7cfeffd700 1 --2- 172.21.15.44:0/2609230975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f2710 0x55c54a2f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:56:13.409 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.396+0000 7f7cfe7fc700 1 --2- 172.21.15.44:0/2609230975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f3030 0x55c54a2fc900 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40c9810 tx=0x7f7cf4098ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9e57302e63137dd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:13.409 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.396+0000 7f7ce57fa700 1 -- 172.21.15.44:0/2609230975 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf409d170 con 0x55c54a2f3030 2023-09-10T00:56:13.409 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.396+0000 7f7ce57fa700 1 -- 172.21.15.44:0/2609230975 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7cf4099570 con 0x55c54a2f3030 2023-09-10T00:56:13.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.396+0000 7f7ce57fa700 1 -- 172.21.15.44:0/2609230975 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40840b0 con 0x55c54a2f3030 2023-09-10T00:56:13.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.397+0000 7f7d117a0b80 1 -- 172.21.15.44:0/2609230975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f3030 msgr2=0x55c54a2fc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:13.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.397+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/2609230975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f3030 0x55c54a2fc900 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40c9810 tx=0x7f7cf4098ed0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.397+0000 7f7d117a0b80 1 -- 172.21.15.44:0/2609230975 shutdown_connections 2023-09-10T00:56:13.410 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.397+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/2609230975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 0x55c54a2ff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.397+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/2609230975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f2710 0x55c54a2f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.397+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/2609230975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f3030 0x55c54a2fc900 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.397+0000 7f7d117a0b80 1 -- 172.21.15.44:0/2609230975 >> 172.21.15.44:0/2609230975 conn(0x55c54a267090 msgr2=0x55c54a2edbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:13.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.397+0000 7f7d117a0b80 1 -- 172.21.15.44:0/2609230975 shutdown_connections 2023-09-10T00:56:13.411 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.397+0000 7f7d117a0b80 1 -- 172.21.15.44:0/2609230975 wait complete. 2023-09-10T00:56:13.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.398+0000 7f7d117a0b80 1 -- start start 2023-09-10T00:56:13.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.398+0000 7f7d117a0b80 1 Processor -- start 2023-09-10T00:56:13.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.398+0000 7f7d117a0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f2710 0x55c54a2f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.398+0000 7f7d117a0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f3030 0x55c54a2fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.398+0000 7f7d117a0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 0x55c54a2fa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.398+0000 7f7d117a0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54a1e2b70 con 0x55c54a2f3030 2023-09-10T00:56:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.398+0000 7f7d117a0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54a1e2870 con 0x55c54a2fce40 2023-09-10T00:56:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.398+0000 7f7d117a0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54a0c6920 con 0x55c54a2f2710 2023-09-10T00:56:13.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.398+0000 7f7d117a0b80 10 client.0 _ll_register_callbacks cb 0x55c54a12ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:56:13.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.398+0000 7f7cfeffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f2710 0x55c54a2f9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.398+0000 7f7cfe7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f3030 0x55c54a2fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.398+0000 7f7cfe7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f3030 0x55c54a2fa0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53870/0 (socket says 172.21.15.44:53870) 2023-09-10T00:56:13.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.398+0000 7f7cfeffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f2710 0x55c54a2f9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60750/0 (socket says 172.21.15.44:60750) 2023-09-10T00:56:13.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.398+0000 7f7cfe7fc700 1 -- 172.21.15.44:0/2699168844 learned_addr learned my addr 172.21.15.44:0/2699168844 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:13.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.399+0000 7f7cfe7fc700 1 -- 172.21.15.44:0/2699168844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f2710 msgr2=0x55c54a2f9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:13.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.399+0000 7f7cfe7fc700 1 --2- 172.21.15.44:0/2699168844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f2710 0x55c54a2f9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.399+0000 7f7cfe7fc700 1 -- 172.21.15.44:0/2699168844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 msgr2=0x55c54a2fa5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T00:56:13.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.399+0000 7f7cfe7fc700 1 --2- 172.21.15.44:0/2699168844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 0x55c54a2fa5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.399+0000 7f7cfe7fc700 1 -- 172.21.15.44:0/2699168844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf0017070 con 0x55c54a2f3030 2023-09-10T00:56:13.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.399+0000 7f7cfe7fc700 1 --2- 172.21.15.44:0/2699168844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f3030 0x55c54a2fa0a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4095a70 tx=0x7f7cf40842f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211c8a99ad2d4222 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:13.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.399+0000 7f7ce2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:56:13.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.399+0000 7f7ce2ff5700 1 -- 172.21.15.44:0/2699168844 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40aeea0 con 0x55c54a2f3030 2023-09-10T00:56:13.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.399+0000 7f7ce2ff5700 1 -- 172.21.15.44:0/2699168844 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f7cf409d3a0 con 0x55c54a2f3030 2023-09-10T00:56:13.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.399+0000 7f7d117a0b80 1 -- 172.21.15.44:0/2699168844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c54a19c4f0 con 0x55c54a2f3030 2023-09-10T00:56:13.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.399+0000 7f7ce2ff5700 1 -- 172.21.15.44:0/2699168844 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cf40a15e0 con 0x55c54a2f3030 2023-09-10T00:56:13.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.399+0000 7f7d117a0b80 10 client.17952 fetch_fsmap learned FSMap version 73 2023-09-10T00:56:13.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.399+0000 7f7d117a0b80 1 -- 172.21.15.44:0/2699168844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x7f7d001a3370 con 0x55c54a2f3030 2023-09-10T00:56:13.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.399+0000 7f7ce2ff5700 1 -- 172.21.15.44:0/2699168844 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c54a19c4f0 con 0x55c54a2f3030 2023-09-10T00:56:13.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.400+0000 7f7ce2ff5700 1 -- 172.21.15.44:0/2699168844 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 73) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7cf40a1150 con 0x55c54a2f3030 2023-09-10T00:56:13.417 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.400+0000 7f7d117a0b80 10 client.17952 fetch_fsmap finished waiting for FSMap version 73 2023-09-10T00:56:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.400+0000 7f7d117a0b80 10 client.17952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:56:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.400+0000 7f7d117a0b80 20 client.17952 populate_metadata read hostname 'smithi044' 2023-09-10T00:56:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.400+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/2699168844 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c54a314f20 0x55c54a317340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.400+0000 7f7d117a0b80 4 client.17952 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:56:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.400+0000 7f7d117a0b80 1 -- 172.21.15.44:0/2699168844 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c54a2598a0 con 0x55c54a314f20 2023-09-10T00:56:13.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.400+0000 7f7cfeffd700 1 --2- 172.21.15.44:0/2699168844 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c54a314f20 0x55c54a317340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.400+0000 7f7ce2ff5700 10 client.17952 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:13.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.400+0000 7f7cfeffd700 1 --2- 172.21.15.44:0/2699168844 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c54a314f20 0x55c54a317340 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3bacab9e9bf41d9 server_cookie=36baa52d547ac317 in_seq=0 out_seq=0 2023-09-10T00:56:13.419 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.401+0000 7f7ce2ff5700 1 -- 172.21.15.44:0/2699168844 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c54a2598a0 con 0x55c54a314f20 2023-09-10T00:56:13.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.401+0000 7f7ce2ff5700 10 client.17952 handle_command_reply: tid=0 2023-09-10T00:56:13.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.401+0000 7f7d117a0b80 1 client.17952 shutdown 2023-09-10T00:56:13.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.401+0000 7f7d117a0b80 1 -- 172.21.15.44:0/2699168844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f3030 msgr2=0x55c54a2fa0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:13.420 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.401+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/2699168844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f3030 0x55c54a2fa0a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4095a70 tx=0x7f7cf40842f0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.401+0000 7f7d117a0b80 1 -- 172.21.15.44:0/2699168844 shutdown_connections 2023-09-10T00:56:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.401+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/2699168844 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c54a314f20 0x55c54a317340 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.401+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/2699168844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 0x55c54a2fa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.401+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/2699168844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f2710 0x55c54a2f9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.401+0000 7f7ce2ff5700 0 client.17952 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.401+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/2699168844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f3030 0x55c54a2fa0a0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.401+0000 7f7d117a0b80 1 -- 172.21.15.44:0/2699168844 >> 172.21.15.44:0/2699168844 conn(0x55c54a267090 msgr2=0x55c54a2ed7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:13.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.401+0000 7f7d117a0b80 1 -- 172.21.15.44:0/2699168844 shutdown_connections 2023-09-10T00:56:13.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.401+0000 7f7d117a0b80 1 -- 172.21.15.44:0/2699168844 wait complete. 2023-09-10T00:56:13.422 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.402+0000 7f7d117a0b80 20 client.17952 trim_cache size 0 max 16384 2023-09-10T00:56:13.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.422+0000 7f7d117a0b80 1 Processor -- start 2023-09-10T00:56:13.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.422+0000 7f7d117a0b80 1 -- start start 2023-09-10T00:56:13.425 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7d117a0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f2710 0x55c54a2ed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7d117a0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f3030 0x55c54a2edc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7d117a0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 0x55c54a2f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7d117a0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54a2e7d70 con 0x55c54a2f2710 2023-09-10T00:56:13.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7d117a0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54a2ff6d0 con 0x55c54a2fce40 2023-09-10T00:56:13.426 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7d117a0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54a2e5400 con 0x55c54a2f3030 2023-09-10T00:56:13.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7cfeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f2710 0x55c54a2ed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7cfe7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f3030 0x55c54a2edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7cfeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f2710 0x55c54a2ed6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53876/0 (socket says 172.21.15.44:53876) 2023-09-10T00:56:13.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7cfeffd700 1 -- 172.21.15.44:0/3640705367 learned_addr learned my addr 172.21.15.44:0/3640705367 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:13.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7cff7fe700 1 --2- 172.21.15.44:0/3640705367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 0x55c54a2f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.427 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7cfeffd700 1 -- 172.21.15.44:0/3640705367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f3030 msgr2=0x55c54a2edc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:13.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7cfeffd700 1 --2- 172.21.15.44:0/3640705367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f3030 0x55c54a2edc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7cfeffd700 1 -- 172.21.15.44:0/3640705367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 msgr2=0x55c54a2f8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:13.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7cfeffd700 1 --2- 172.21.15.44:0/3640705367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 0x55c54a2f8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7cfeffd700 1 -- 172.21.15.44:0/3640705367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c54a059c60 con 0x55c54a2f2710 2023-09-10T00:56:13.428 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7cfe7fc700 1 --2- 172.21.15.44:0/3640705367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f3030 0x55c54a2edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:56:13.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7cfeffd700 1 --2- 172.21.15.44:0/3640705367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f2710 0x55c54a2ed6f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0013010 tx=0x7f7cf0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa916ee7b9cd4191 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:13.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.423+0000 7f7ce5ffb700 1 -- 172.21.15.44:0/3640705367 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0024040 con 0x55c54a2f2710 2023-09-10T00:56:13.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.424+0000 7f7ce5ffb700 1 -- 172.21.15.44:0/3640705367 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7cf000ee80 con 0x55c54a2f2710 2023-09-10T00:56:13.429 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.424+0000 7f7ce5ffb700 1 -- 172.21.15.44:0/3640705367 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0003680 con 0x55c54a2f2710 2023-09-10T00:56:13.435 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.424+0000 7f7d117a0b80 1 -- 172.21.15.44:0/3640705367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f2710 msgr2=0x55c54a2ed6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:13.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.424+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/3640705367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f2710 0x55c54a2ed6f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0013010 tx=0x7f7cf0014000 comp rx=0 tx=0).stop 2023-09-10T00:56:13.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.424+0000 7f7d117a0b80 1 -- 172.21.15.44:0/3640705367 shutdown_connections 2023-09-10T00:56:13.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.424+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/3640705367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 0x55c54a2f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.424+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/3640705367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f2710 0x55c54a2ed6f0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.436 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.424+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/3640705367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f3030 0x55c54a2edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.424+0000 7f7d117a0b80 1 -- 172.21.15.44:0/3640705367 >> 172.21.15.44:0/3640705367 conn(0x55c54a267090 msgr2=0x55c54a2f4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.424+0000 7f7d117a0b80 1 -- 172.21.15.44:0/3640705367 shutdown_connections 2023-09-10T00:56:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.424+0000 7f7d117a0b80 1 -- 172.21.15.44:0/3640705367 wait complete. 2023-09-10T00:56:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.425+0000 7f7d117a0b80 1 -- start start 2023-09-10T00:56:13.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.425+0000 7f7d117a0b80 1 Processor -- start 2023-09-10T00:56:13.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.425+0000 7f7d117a0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f2710 0x55c54a3021f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.425+0000 7f7d117a0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f3030 0x55c54a302730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.438 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.425+0000 7f7d117a0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 0x55c54a302c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.425+0000 7f7d117a0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54a368e60 con 0x55c54a2f3030 2023-09-10T00:56:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.425+0000 7f7d117a0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54a368fd0 con 0x55c54a2fce40 2023-09-10T00:56:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.425+0000 7f7d117a0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54a369190 con 0x55c54a2f2710 2023-09-10T00:56:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.425+0000 7f7d117a0b80 10 client.0 _ll_register_callbacks cb 0x7f7cf0009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:56:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.425+0000 7f7cfe7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f3030 0x55c54a302730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.440 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.425+0000 7f7cfeffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f2710 0x55c54a3021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.440 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.425+0000 7f7cfe7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f3030 0x55c54a302730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53884/0 (socket says 172.21.15.44:53884) 2023-09-10T00:56:13.440 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.425+0000 7f7cfe7fc700 1 -- 172.21.15.44:0/4034857667 learned_addr learned my addr 172.21.15.44:0/4034857667 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:13.440 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.425+0000 7f7cff7fe700 1 --2- 172.21.15.44:0/4034857667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 0x55c54a302c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.426+0000 7f7cfeffd700 1 -- 172.21.15.44:0/4034857667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 msgr2=0x55c54a302c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.426+0000 7f7cfeffd700 1 --2- 172.21.15.44:0/4034857667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 0x55c54a302c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.426+0000 7f7cfeffd700 1 -- 172.21.15.44:0/4034857667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f3030 msgr2=0x55c54a302730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.426+0000 7f7cfeffd700 1 --2- 172.21.15.44:0/4034857667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f3030 0x55c54a302730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.426+0000 7f7cfeffd700 1 -- 172.21.15.44:0/4034857667 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d001a3370 con 0x55c54a2f2710 2023-09-10T00:56:13.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.426+0000 7f7cfe7fc700 1 --2- 172.21.15.44:0/4034857667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f3030 0x55c54a302730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:56:13.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.426+0000 7f7cff7fe700 1 --2- 172.21.15.44:0/4034857667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 0x55c54a302c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:56:13.442 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.426+0000 7f7cfeffd700 1 --2- 172.21.15.44:0/4034857667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f2710 0x55c54a3021f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009fa0 tx=0x7f7cf00090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62a6c12e59100ebb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:13.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.426+0000 7f7ce2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:56:13.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.426+0000 7f7ce2ff5700 1 -- 172.21.15.44:0/4034857667 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0024040 con 0x55c54a2f2710 2023-09-10T00:56:13.443 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.426+0000 7f7d117a0b80 1 -- 172.21.15.44:0/4034857667 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c54a23d920 con 0x55c54a2f2710 2023-09-10T00:56:13.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.426+0000 7f7ce2ff5700 1 -- 172.21.15.44:0/4034857667 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f7cf000e9f0 con 0x55c54a2f2710 2023-09-10T00:56:13.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.426+0000 7f7ce2ff5700 1 -- 172.21.15.44:0/4034857667 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cf000ebd0 con 0x55c54a2f2710 2023-09-10T00:56:13.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.426+0000 7f7d117a0b80 10 client.27550 fetch_fsmap learned FSMap version 73 2023-09-10T00:56:13.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.426+0000 7f7d117a0b80 1 -- 172.21.15.44:0/4034857667 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x7f7cf000e3b0 con 0x55c54a2f2710 2023-09-10T00:56:13.444 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.427+0000 7f7ce2ff5700 1 -- 172.21.15.44:0/4034857667 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c54a23d920 con 0x55c54a2f2710 2023-09-10T00:56:13.445 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.427+0000 7f7ce2ff5700 1 -- 172.21.15.44:0/4034857667 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 73) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7cf0003a00 con 0x55c54a2f2710 2023-09-10T00:56:13.445 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.427+0000 7f7d117a0b80 10 client.27550 fetch_fsmap finished waiting for FSMap version 73 2023-09-10T00:56:13.445 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.427+0000 7f7d117a0b80 10 client.27550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-10T00:56:13.445 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.427+0000 7f7d117a0b80 20 client.27550 populate_metadata read hostname 'smithi044' 2023-09-10T00:56:13.445 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.427+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/4034857667 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c54a36a960 0x55c54a36cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:13.446 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.427+0000 7f7d117a0b80 4 client.27550 mds_command: new command op to 24605 tid=0[{"prefix": "scrub status"}] 2023-09-10T00:56:13.446 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.427+0000 7f7d117a0b80 1 -- 172.21.15.44:0/4034857667 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c54a1fe0b0 con 0x55c54a36a960 2023-09-10T00:56:13.446 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.427+0000 7f7cfe7fc700 1 --2- 172.21.15.44:0/4034857667 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c54a36a960 0x55c54a36cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:13.446 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.427+0000 7f7cfe7fc700 1 --2- 172.21.15.44:0/4034857667 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c54a36a960 0x55c54a36cd80 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76103974ee193c2b server_cookie=78c94709e98a6b08 in_seq=0 out_seq=0 2023-09-10T00:56:13.446 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.427+0000 7f7ce2ff5700 10 client.27550 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:13.447 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.428+0000 7f7ce2ff5700 1 -- 172.21.15.44:0/4034857667 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c54a1fe0b0 con 0x55c54a36a960 2023-09-10T00:56:13.447 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.428+0000 7f7ce2ff5700 10 client.27550 handle_command_reply: tid=0 2023-09-10T00:56:13.447 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.428+0000 7f7d117a0b80 1 client.27550 shutdown 2023-09-10T00:56:13.447 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.428+0000 7f7d117a0b80 1 -- 172.21.15.44:0/4034857667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f2710 msgr2=0x55c54a3021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:13.447 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.428+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/4034857667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f2710 0x55c54a3021f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009fa0 tx=0x7f7cf00090f0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.447 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.428+0000 7f7d117a0b80 1 -- 172.21.15.44:0/4034857667 shutdown_connections 2023-09-10T00:56:13.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.428+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/4034857667 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c54a36a960 0x55c54a36cd80 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.428+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/4034857667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c54a2fce40 0x55c54a302c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.428+0000 7f7ce2ff5700 0 client.27550 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:13.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.428+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/4034857667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c54a2f2710 0x55c54a3021f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.448 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.428+0000 7f7d117a0b80 1 --2- 172.21.15.44:0/4034857667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c54a2f3030 0x55c54a302730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.449 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.428+0000 7f7d117a0b80 1 -- 172.21.15.44:0/4034857667 >> 172.21.15.44:0/4034857667 conn(0x55c54a267090 msgr2=0x55c54a2ed440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:13.449 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.428+0000 7f7d117a0b80 1 -- 172.21.15.44:0/4034857667 shutdown_connections 2023-09-10T00:56:13.449 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.428+0000 7f7d117a0b80 1 -- 172.21.15.44:0/4034857667 wait complete. 2023-09-10T00:56:13.449 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.428+0000 7f7d117a0b80 20 client.27550 trim_cache size 0 max 16384 2023-09-10T00:56:13.450 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T00:56:13.450 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-10T00:56:13.450 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2023-09-10T00:56:13.450 INFO:teuthology.orchestra.run.smithi044.stdout: "0a809115-54e8-46e3-b407-3791c0181760": { 2023-09-10T00:56:13.450 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2023-09-10T00:56:13.451 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "0a809115-54e8-46e3-b407-3791c0181760", 2023-09-10T00:56:13.451 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2023-09-10T00:56:13.451 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:56:13.451 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T00:56:13.452 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T00:56:13.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.429+0000 7f7d055da700 1 -- 172.21.15.44:0/4172130832 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7ce80a0830 msgr2=0x7f7ce80a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:13.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.429+0000 7f7d055da700 1 --2- 172.21.15.44:0/4172130832 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7ce80a0830 0x7f7ce80a2cf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009d70 tx=0x7f7cf0009260 comp rx=0 tx=0).stop 2023-09-10T00:56:13.453 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.429+0000 7f7d055da700 1 -- 172.21.15.44:0/4172130832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d000ffe00 msgr2=0x7f7d00196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:13.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.429+0000 7f7d055da700 1 --2- 172.21.15.44:0/4172130832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d000ffe00 0x7f7d00196c30 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400e9a0 tx=0x7f7cf400ecb0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.429+0000 7f7d055da700 1 -- 172.21.15.44:0/4172130832 shutdown_connections 2023-09-10T00:56:13.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.429+0000 7f7d055da700 1 --2- 172.21.15.44:0/4172130832 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7ce80a0830 0x7f7ce80a2cf0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.454 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.429+0000 7f7d055da700 1 --2- 172.21.15.44:0/4172130832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d001054e0 0x7f7d00199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.429+0000 7f7d055da700 1 --2- 172.21.15.44:0/4172130832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d000ff440 0x7f7d001966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.429+0000 7f7d055da700 1 --2- 172.21.15.44:0/4172130832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d000ffe00 0x7f7d00196c30 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:13.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.429+0000 7f7d055da700 1 -- 172.21.15.44:0/4172130832 >> 172.21.15.44:0/4172130832 conn(0x7f7d000faf50 msgr2=0x7f7d00102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:13.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.429+0000 7f7d055da700 1 -- 172.21.15.44:0/4172130832 shutdown_connections 2023-09-10T00:56:13.455 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T00:56:13.429+0000 7f7d055da700 1 -- 172.21.15.44:0/4172130832 wait complete. 2023-09-10T00:56:13.456 INFO:tasks.cephfs.filesystem:scrub status for tag:0a809115-54e8-46e3-b407-3791c0181760 - {'path': '/', 'tag': '0a809115-54e8-46e3-b407-3791c0181760', 'options': 'recursive,force'} 2023-09-10T00:56:13.456 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/cephfs/filesystem.py", line 1730, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-09-10T00:56:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:14 smithi131 ceph-mon[103713]: pgmap v491: 129 pgs: 129 active+clean; 8.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 370 op/s 2023-09-10T00:56:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:14 smithi044 ceph-mon[98676]: pgmap v491: 129 pgs: 129 active+clean; 8.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 370 op/s 2023-09-10T00:56:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:14 smithi099 ceph-mon[102692]: pgmap v491: 129 pgs: 129 active+clean; 8.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 370 op/s 2023-09-10T00:56:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:16 smithi131 ceph-mon[103713]: pgmap v492: 129 pgs: 129 active+clean; 8.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 383 op/s 2023-09-10T00:56:17.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:16 smithi044 ceph-mon[98676]: pgmap v492: 129 pgs: 129 active+clean; 8.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 383 op/s 2023-09-10T00:56:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:16 smithi099 ceph-mon[102692]: pgmap v492: 129 pgs: 129 active+clean; 8.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 383 op/s 2023-09-10T00:56:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:18 smithi131 ceph-mon[103713]: pgmap v493: 129 pgs: 129 active+clean; 8.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 383 op/s 2023-09-10T00:56:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:18 smithi044 ceph-mon[98676]: pgmap v493: 129 pgs: 129 active+clean; 8.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 383 op/s 2023-09-10T00:56:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:18 smithi099 ceph-mon[102692]: pgmap v493: 129 pgs: 129 active+clean; 8.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 383 op/s 2023-09-10T00:56:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:20 smithi131 ceph-mon[103713]: pgmap v494: 129 pgs: 129 active+clean; 8.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 383 op/s 2023-09-10T00:56:21.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:20 smithi044 ceph-mon[98676]: pgmap v494: 129 pgs: 129 active+clean; 8.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 383 op/s 2023-09-10T00:56:21.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:20 smithi099 ceph-mon[102692]: pgmap v494: 129 pgs: 129 active+clean; 8.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 383 op/s 2023-09-10T00:56:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.841+0000 7fdc60b4e700 1 -- 172.21.15.44:0/3206872664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc5c0ff760 msgr2=0x7fdc5c0ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.841+0000 7fdc60b4e700 1 --2- 172.21.15.44:0/3206872664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc5c0ff760 0x7fdc5c0ffbe0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000b9e0 tx=0x7fdc50009260 comp rx=0 tx=0).stop 2023-09-10T00:56:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.842+0000 7fdc60b4e700 1 -- 172.21.15.44:0/3206872664 shutdown_connections 2023-09-10T00:56:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.842+0000 7fdc60b4e700 1 --2- 172.21.15.44:0/3206872664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c100120 0x7fdc5c10c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.842+0000 7fdc60b4e700 1 --2- 172.21.15.44:0/3206872664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc5c1055b0 0x7fdc5c105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.842+0000 7fdc60b4e700 1 --2- 172.21.15.44:0/3206872664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc5c0ff760 0x7fdc5c0ffbe0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.842+0000 7fdc60b4e700 1 -- 172.21.15.44:0/3206872664 >> 172.21.15.44:0/3206872664 conn(0x7fdc5c074f30 msgr2=0x7fdc5c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.842+0000 7fdc60b4e700 1 -- 172.21.15.44:0/3206872664 shutdown_connections 2023-09-10T00:56:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.842+0000 7fdc60b4e700 1 -- 172.21.15.44:0/3206872664 wait complete. 2023-09-10T00:56:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.843+0000 7fdc60b4e700 1 Processor -- start 2023-09-10T00:56:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.843+0000 7fdc60b4e700 1 -- start start 2023-09-10T00:56:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.843+0000 7fdc60b4e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c0ff760 0x7fdc5c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.843+0000 7fdc60b4e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc5c100120 0x7fdc5c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.843+0000 7fdc60b4e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc5c1055b0 0x7fdc5c194390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.843+0000 7fdc60b4e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c194a00 con 0x7fdc5c100120 2023-09-10T00:56:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.843+0000 7fdc60b4e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c194b70 con 0x7fdc5c1055b0 2023-09-10T00:56:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.843+0000 7fdc60b4e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c194d30 con 0x7fdc5c0ff760 2023-09-10T00:56:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.843+0000 7fdc5a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c0ff760 0x7fdc5c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.843+0000 7fdc5a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c0ff760 0x7fdc5c19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60778/0 (socket says 172.21.15.44:60778) 2023-09-10T00:56:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.843+0000 7fdc5a59c700 1 -- 172.21.15.44:0/2643396209 learned_addr learned my addr 172.21.15.44:0/2643396209 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.843+0000 7fdc5ad9d700 1 --2- 172.21.15.44:0/2643396209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc5c1055b0 0x7fdc5c194390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.843+0000 7fdc59d9b700 1 --2- 172.21.15.44:0/2643396209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc5c100120 0x7fdc5c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.844+0000 7fdc5ad9d700 1 -- 172.21.15.44:0/2643396209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c0ff760 msgr2=0x7fdc5c19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.844+0000 7fdc5ad9d700 1 --2- 172.21.15.44:0/2643396209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c0ff760 0x7fdc5c19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.844+0000 7fdc5ad9d700 1 -- 172.21.15.44:0/2643396209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc5c100120 msgr2=0x7fdc5c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.844+0000 7fdc5ad9d700 1 --2- 172.21.15.44:0/2643396209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc5c100120 0x7fdc5c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.844+0000 7fdc5ad9d700 1 -- 172.21.15.44:0/2643396209 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc54008490 con 0x7fdc5c1055b0 2023-09-10T00:56:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.844+0000 7fdc5a59c700 1 --2- 172.21.15.44:0/2643396209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c0ff760 0x7fdc5c19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:56:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.844+0000 7fdc5ad9d700 1 --2- 172.21.15.44:0/2643396209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc5c1055b0 0x7fdc5c194390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdc540089f0 tx=0x7fdc54008d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.844+0000 7fdc4b7fe700 1 -- 172.21.15.44:0/2643396209 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc54008e30 con 0x7fdc5c1055b0 2023-09-10T00:56:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.844+0000 7fdc60b4e700 1 -- 172.21.15.44:0/2643396209 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc5000b610 con 0x7fdc5c1055b0 2023-09-10T00:56:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.844+0000 7fdc4b7fe700 1 -- 172.21.15.44:0/2643396209 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc54004bb0 con 0x7fdc5c1055b0 2023-09-10T00:56:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.844+0000 7fdc60b4e700 1 -- 172.21.15.44:0/2643396209 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5c1a1340 con 0x7fdc5c1055b0 2023-09-10T00:56:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.844+0000 7fdc4b7fe700 1 -- 172.21.15.44:0/2643396209 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc54007b30 con 0x7fdc5c1055b0 2023-09-10T00:56:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.847+0000 7fdc4b7fe700 1 -- 172.21.15.44:0/2643396209 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fdc54007c90 con 0x7fdc5c1055b0 2023-09-10T00:56:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.847+0000 7fdc4b7fe700 1 --2- 172.21.15.44:0/2643396209 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdc440a07b0 0x7fdc440a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.848+0000 7fdc4b7fe700 1 -- 172.21.15.44:0/2643396209 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fdc540d43c0 con 0x7fdc5c1055b0 2023-09-10T00:56:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.848+0000 7fdc5a59c700 1 --2- 172.21.15.44:0/2643396209 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdc440a07b0 0x7fdc440a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.848+0000 7fdc5a59c700 1 --2- 172.21.15.44:0/2643396209 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdc440a07b0 0x7fdc440a2c70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c011550 tx=0x7fdc4c009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.849+0000 7fdc6cd14b80 1 Processor -- start 2023-09-10T00:56:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.849+0000 7fdc6cd14b80 1 -- start start 2023-09-10T00:56:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.849+0000 7fdc6cd14b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1ff1a0 0x5578ec1ff580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.849+0000 7fdc6cd14b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1f9300 0x5578ec1f9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.849+0000 7fdc6cd14b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec205f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.849+0000 7fdc6cd14b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ec1526d0 con 0x5578ec1f9cc0 2023-09-10T00:56:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.849+0000 7fdc6cd14b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ec14fff0 con 0x5578ec1f9300 2023-09-10T00:56:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.849+0000 7fdc6cd14b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ec14b8e0 con 0x5578ec1ff1a0 2023-09-10T00:56:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.850+0000 7fdc5ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec205f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.850+0000 7fdc5ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec205f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53904/0 (socket says 172.21.15.44:53904) 2023-09-10T00:56:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.850+0000 7fdc5ad9d700 1 -- 172.21.15.44:0/2569008857 learned_addr learned my addr 172.21.15.44:0/2569008857 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.850+0000 7fdc59d9b700 1 --2- 172.21.15.44:0/2569008857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1f9300 0x5578ec1f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.850+0000 7fdc5a59c700 1 --2- 172.21.15.44:0/2569008857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1ff1a0 0x5578ec1ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.850+0000 7fdc59d9b700 1 -- 172.21.15.44:0/2569008857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1ff1a0 msgr2=0x5578ec1ff580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.850+0000 7fdc59d9b700 1 --2- 172.21.15.44:0/2569008857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1ff1a0 0x5578ec1ff580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.850+0000 7fdc59d9b700 1 -- 172.21.15.44:0/2569008857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 msgr2=0x5578ec205f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.850+0000 7fdc59d9b700 1 --2- 172.21.15.44:0/2569008857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec205f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.850+0000 7fdc59d9b700 1 -- 172.21.15.44:0/2569008857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578ebf77d80 con 0x5578ec1f9300 2023-09-10T00:56:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.850+0000 7fdc5ad9d700 1 --2- 172.21.15.44:0/2569008857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec205f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:56:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.850+0000 7fdc59d9b700 1 --2- 172.21.15.44:0/2569008857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1f9300 0x5578ec1f9780 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000b900 tx=0x7fdc50005c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd967fd370ed9102 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.850+0000 7fdc48ff9700 1 -- 172.21.15.44:0/2569008857 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500098e0 con 0x5578ec1f9300 2023-09-10T00:56:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.850+0000 7fdc48ff9700 1 -- 172.21.15.44:0/2569008857 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc50009a40 con 0x5578ec1f9300 2023-09-10T00:56:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.850+0000 7fdc48ff9700 1 -- 172.21.15.44:0/2569008857 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50004a70 con 0x5578ec1f9300 2023-09-10T00:56:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.851+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/2569008857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1f9300 msgr2=0x5578ec1f9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.851+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/2569008857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1f9300 0x5578ec1f9780 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000b900 tx=0x7fdc50005c60 comp rx=0 tx=0).stop 2023-09-10T00:56:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.851+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/2569008857 shutdown_connections 2023-09-10T00:56:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.851+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/2569008857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec205f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.851+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/2569008857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1ff1a0 0x5578ec1ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.851+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/2569008857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1f9300 0x5578ec1f9780 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.851+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/2569008857 >> 172.21.15.44:0/2569008857 conn(0x5578ec16e030 msgr2=0x5578ec1f4560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.851+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/2569008857 shutdown_connections 2023-09-10T00:56:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.852+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/2569008857 wait complete. 2023-09-10T00:56:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.852+0000 7fdc6cd14b80 1 -- start start 2023-09-10T00:56:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.852+0000 7fdc6cd14b80 1 Processor -- start 2023-09-10T00:56:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.852+0000 7fdc6cd14b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1f9300 0x5578ec205e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.852+0000 7fdc6cd14b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec203090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.852+0000 7fdc6cd14b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1ff1a0 0x5578ec2036c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.852+0000 7fdc6cd14b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ec0e9b10 con 0x5578ec1f9cc0 2023-09-10T00:56:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.852+0000 7fdc6cd14b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ec0e9810 con 0x5578ec1ff1a0 2023-09-10T00:56:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.852+0000 7fdc6cd14b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ebfcd8c0 con 0x5578ec1f9300 2023-09-10T00:56:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.852+0000 7fdc6cd14b80 10 client.0 _ll_register_callbacks cb 0x5578ec033a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:56:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.852+0000 7fdc5a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1f9300 0x5578ec205e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.852+0000 7fdc5a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1f9300 0x5578ec205e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60792/0 (socket says 172.21.15.44:60792) 2023-09-10T00:56:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.852+0000 7fdc5a59c700 1 -- 172.21.15.44:0/151475906 learned_addr learned my addr 172.21.15.44:0/151475906 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.852+0000 7fdc5ad9d700 1 --2- 172.21.15.44:0/151475906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1ff1a0 0x5578ec2036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.853+0000 7fdc59d9b700 1 --2- 172.21.15.44:0/151475906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec203090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.853+0000 7fdc5a59c700 1 -- 172.21.15.44:0/151475906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1ff1a0 msgr2=0x5578ec2036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.853+0000 7fdc5a59c700 1 --2- 172.21.15.44:0/151475906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1ff1a0 0x5578ec2036c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.853+0000 7fdc5a59c700 1 -- 172.21.15.44:0/151475906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 msgr2=0x5578ec203090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.853+0000 7fdc5a59c700 1 --2- 172.21.15.44:0/151475906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec203090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.853+0000 7fdc5a59c700 1 -- 172.21.15.44:0/151475906 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5c1a1340 con 0x5578ec1f9300 2023-09-10T00:56:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.853+0000 7fdc5ad9d700 1 --2- 172.21.15.44:0/151475906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1ff1a0 0x5578ec2036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:56:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.853+0000 7fdc5a59c700 1 --2- 172.21.15.44:0/151475906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1f9300 0x5578ec205e10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00edf0 tx=0x7fdc4c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=465edaaee1538f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.853+0000 7fdc41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:56:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.853+0000 7fdc41ffb700 1 -- 172.21.15.44:0/151475906 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c00ee20 con 0x5578ec1f9300 2023-09-10T00:56:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.853+0000 7fdc41ffb700 1 -- 172.21.15.44:0/151475906 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fdc4c01a5b0 con 0x5578ec1f9300 2023-09-10T00:56:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.853+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/151475906 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578ec0a3490 con 0x5578ec1f9300 2023-09-10T00:56:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.854+0000 7fdc41ffb700 1 -- 172.21.15.44:0/151475906 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc4c01a770 con 0x5578ec1f9300 2023-09-10T00:56:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.854+0000 7fdc6cd14b80 10 client.27562 fetch_fsmap learned FSMap version 73 2023-09-10T00:56:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.854+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/151475906 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x7fdc5c10cbc0 con 0x5578ec1f9300 2023-09-10T00:56:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.854+0000 7fdc41ffb700 1 -- 172.21.15.44:0/151475906 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5578ec0a3490 con 0x5578ec1f9300 2023-09-10T00:56:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.854+0000 7fdc41ffb700 1 -- 172.21.15.44:0/151475906 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 73) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fdc4c028970 con 0x5578ec1f9300 2023-09-10T00:56:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.854+0000 7fdc6cd14b80 10 client.27562 fetch_fsmap finished waiting for FSMap version 73 2023-09-10T00:56:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.854+0000 7fdc6cd14b80 10 client.27562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:56:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.854+0000 7fdc6cd14b80 20 client.27562 populate_metadata read hostname 'smithi044' 2023-09-10T00:56:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.854+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/151475906 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5578ec2134d0 0x5578ec2158f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.854+0000 7fdc6cd14b80 4 client.27562 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:56:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.854+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/151475906 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578ec160840 con 0x5578ec2134d0 2023-09-10T00:56:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.854+0000 7fdc59d9b700 1 --2- 172.21.15.44:0/151475906 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5578ec2134d0 0x5578ec2158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.855+0000 7fdc41ffb700 10 client.27562 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.855+0000 7fdc59d9b700 1 --2- 172.21.15.44:0/151475906 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5578ec2134d0 0x5578ec2158f0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9193372ce9cc5ae0 server_cookie=338d67bfc2f7f5ae in_seq=0 out_seq=0 2023-09-10T00:56:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.855+0000 7fdc41ffb700 1 -- 172.21.15.44:0/151475906 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5578ec160840 con 0x5578ec2134d0 2023-09-10T00:56:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.855+0000 7fdc41ffb700 10 client.27562 handle_command_reply: tid=0 2023-09-10T00:56:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.855+0000 7fdc6cd14b80 1 client.27562 shutdown 2023-09-10T00:56:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.855+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/151475906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1f9300 msgr2=0x5578ec205e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.855+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/151475906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1f9300 0x5578ec205e10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00edf0 tx=0x7fdc4c0047c0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.856+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/151475906 shutdown_connections 2023-09-10T00:56:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.856+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/151475906 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5578ec2134d0 0x5578ec2158f0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.856+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/151475906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1ff1a0 0x5578ec2036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.856+0000 7fdc41ffb700 0 client.27562 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.856+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/151475906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1f9300 0x5578ec205e10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.856+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/151475906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec203090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.856+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/151475906 >> 172.21.15.44:0/151475906 conn(0x5578ec16e030 msgr2=0x5578ec1f37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.856+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/151475906 shutdown_connections 2023-09-10T00:56:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.856+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/151475906 wait complete. 2023-09-10T00:56:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.856+0000 7fdc6cd14b80 20 client.27562 trim_cache size 0 max 16384 2023-09-10T00:56:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc6cd14b80 1 Processor -- start 2023-09-10T00:56:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc6cd14b80 1 -- start start 2023-09-10T00:56:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc6cd14b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1f9300 0x5578ec20a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc6cd14b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec2059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc6cd14b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1ff1a0 0x5578ec1fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc6cd14b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ec1f0fc0 con 0x5578ec1f9cc0 2023-09-10T00:56:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc6cd14b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ec1fede0 con 0x5578ec1f9300 2023-09-10T00:56:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc6cd14b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ec20a960 con 0x5578ec1ff1a0 2023-09-10T00:56:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc5a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1f9300 0x5578ec20a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc5ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1ff1a0 0x5578ec1fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc5a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1f9300 0x5578ec20a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55658/0 (socket says 172.21.15.44:55658) 2023-09-10T00:56:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc5a59c700 1 -- 172.21.15.44:0/3799805552 learned_addr learned my addr 172.21.15.44:0/3799805552 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc59d9b700 1 --2- 172.21.15.44:0/3799805552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec2059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc5a59c700 1 -- 172.21.15.44:0/3799805552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1ff1a0 msgr2=0x5578ec1fe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc5a59c700 1 --2- 172.21.15.44:0/3799805552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1ff1a0 0x5578ec1fe0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc5a59c700 1 -- 172.21.15.44:0/3799805552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 msgr2=0x5578ec2059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc5a59c700 1 --2- 172.21.15.44:0/3799805552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec2059b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.879+0000 7fdc5a59c700 1 -- 172.21.15.44:0/3799805552 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578ebf60c00 con 0x5578ec1f9300 2023-09-10T00:56:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.880+0000 7fdc5a59c700 1 --2- 172.21.15.44:0/3799805552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1f9300 0x5578ec20a420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c013010 tx=0x7fdc4c0046b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fdd3e0c0cb651d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.880+0000 7fdc497fa700 1 -- 172.21.15.44:0/3799805552 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c03a070 con 0x5578ec1f9300 2023-09-10T00:56:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.880+0000 7fdc497fa700 1 -- 172.21.15.44:0/3799805552 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc4c011e90 con 0x5578ec1f9300 2023-09-10T00:56:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.880+0000 7fdc497fa700 1 -- 172.21.15.44:0/3799805552 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c028680 con 0x5578ec1f9300 2023-09-10T00:56:22.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.881+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/3799805552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1f9300 msgr2=0x5578ec20a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:22.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.881+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/3799805552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1f9300 0x5578ec20a420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c013010 tx=0x7fdc4c0046b0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.881+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/3799805552 shutdown_connections 2023-09-10T00:56:22.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.881+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/3799805552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1ff1a0 0x5578ec1fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.881+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/3799805552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1f9300 0x5578ec20a420 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.881+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/3799805552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec2059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.881+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/3799805552 >> 172.21.15.44:0/3799805552 conn(0x5578ec16e030 msgr2=0x5578ec1fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.881+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/3799805552 shutdown_connections 2023-09-10T00:56:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.881+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/3799805552 wait complete. 2023-09-10T00:56:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.881+0000 7fdc6cd14b80 1 -- start start 2023-09-10T00:56:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc6cd14b80 1 Processor -- start 2023-09-10T00:56:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc6cd14b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1f9300 0x5578ec207b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc6cd14b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec208040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc6cd14b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1ff1a0 0x5578ec2085a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc6cd14b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ec266490 con 0x5578ec1f9cc0 2023-09-10T00:56:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc6cd14b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ec266600 con 0x5578ec1ff1a0 2023-09-10T00:56:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc6cd14b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ec2777d0 con 0x5578ec1f9300 2023-09-10T00:56:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc6cd14b80 10 client.0 _ll_register_callbacks cb 0x7fdc4c00e9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:56:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc5a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1f9300 0x5578ec207b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc5ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1ff1a0 0x5578ec2085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc59d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec208040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc5a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1f9300 0x5578ec207b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60806/0 (socket says 172.21.15.44:60806) 2023-09-10T00:56:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc5a59c700 1 -- 172.21.15.44:0/3245693401 learned_addr learned my addr 172.21.15.44:0/3245693401 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc5a59c700 1 -- 172.21.15.44:0/3245693401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1ff1a0 msgr2=0x5578ec2085a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc5a59c700 1 --2- 172.21.15.44:0/3245693401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1ff1a0 0x5578ec2085a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc5a59c700 1 -- 172.21.15.44:0/3245693401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 msgr2=0x5578ec208040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc5a59c700 1 --2- 172.21.15.44:0/3245693401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec208040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc5a59c700 1 -- 172.21.15.44:0/3245693401 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5c10cbc0 con 0x5578ec1f9300 2023-09-10T00:56:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.882+0000 7fdc5ad9d700 1 --2- 172.21.15.44:0/3245693401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1ff1a0 0x5578ec2085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:56:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.883+0000 7fdc5a59c700 1 --2- 172.21.15.44:0/3245693401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1f9300 0x5578ec207b00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c028f60 tx=0x7fdc4c004bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10a8d123b3d1f180 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.883+0000 7fdc41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:56:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.883+0000 7fdc41ffb700 1 -- 172.21.15.44:0/3245693401 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c044070 con 0x5578ec1f9300 2023-09-10T00:56:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.883+0000 7fdc41ffb700 1 -- 172.21.15.44:0/3245693401 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fdc4c028440 con 0x5578ec1f9300 2023-09-10T00:56:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.883+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/3245693401 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc50009a40 con 0x5578ec1f9300 2023-09-10T00:56:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.883+0000 7fdc41ffb700 1 -- 172.21.15.44:0/3245693401 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc4c03a040 con 0x5578ec1f9300 2023-09-10T00:56:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.883+0000 7fdc6cd14b80 10 client.27568 fetch_fsmap learned FSMap version 73 2023-09-10T00:56:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.883+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/3245693401 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x5578ebf9e1f0 con 0x5578ec1f9300 2023-09-10T00:56:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.883+0000 7fdc41ffb700 1 -- 172.21.15.44:0/3245693401 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc50009a40 con 0x5578ec1f9300 2023-09-10T00:56:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.884+0000 7fdc41ffb700 1 -- 172.21.15.44:0/3245693401 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 73) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fdc4c047030 con 0x5578ec1f9300 2023-09-10T00:56:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.884+0000 7fdc6cd14b80 10 client.27568 fetch_fsmap finished waiting for FSMap version 73 2023-09-10T00:56:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.884+0000 7fdc6cd14b80 10 client.27568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:56:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.884+0000 7fdc6cd14b80 20 client.27568 populate_metadata read hostname 'smithi044' 2023-09-10T00:56:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.884+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/3245693401 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5578ec278b20 0x5578ec27af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.884+0000 7fdc6cd14b80 4 client.27568 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:56:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.884+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/3245693401 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5578ec1448c0 con 0x5578ec278b20 2023-09-10T00:56:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.885+0000 7fdc59d9b700 1 --2- 172.21.15.44:0/3245693401 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5578ec278b20 0x5578ec27af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.885+0000 7fdc59d9b700 1 --2- 172.21.15.44:0/3245693401 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5578ec278b20 0x5578ec27af10 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1056aa93128b6be server_cookie=a67da62c6c5d8655 in_seq=0 out_seq=0 2023-09-10T00:56:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.885+0000 7fdc41ffb700 10 client.27568 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.896+0000 7fdc41ffb700 1 -- 172.21.15.44:0/3245693401 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5578ec1448c0 con 0x5578ec278b20 2023-09-10T00:56:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.896+0000 7fdc41ffb700 10 client.27568 handle_command_reply: tid=0 2023-09-10T00:56:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.896+0000 7fdc6cd14b80 1 client.27568 shutdown 2023-09-10T00:56:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.896+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/3245693401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1f9300 msgr2=0x5578ec207b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.896+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/3245693401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1f9300 0x5578ec207b00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c028f60 tx=0x7fdc4c004bb0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.897+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/3245693401 shutdown_connections 2023-09-10T00:56:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.897+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/3245693401 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5578ec278b20 0x5578ec27af10 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.897+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/3245693401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578ec1ff1a0 0x5578ec2085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.897+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/3245693401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5578ec1f9300 0x5578ec207b00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.897+0000 7fdc6cd14b80 1 --2- 172.21.15.44:0/3245693401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578ec1f9cc0 0x5578ec208040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.897+0000 7fdc41ffb700 0 client.27568 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.897+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/3245693401 >> 172.21.15.44:0/3245693401 conn(0x5578ec16e030 msgr2=0x5578ec1f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.897+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/3245693401 shutdown_connections 2023-09-10T00:56:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.897+0000 7fdc6cd14b80 1 -- 172.21.15.44:0/3245693401 wait complete. 2023-09-10T00:56:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.897+0000 7fdc6cd14b80 20 client.27568 trim_cache size 0 max 16384 2023-09-10T00:56:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:56:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:56:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:56:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:56:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.898+0000 7fdc60b4e700 1 -- 172.21.15.44:0/2643396209 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdc440a07b0 msgr2=0x7fdc440a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.898+0000 7fdc60b4e700 1 --2- 172.21.15.44:0/2643396209 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdc440a07b0 0x7fdc440a2c70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c011550 tx=0x7fdc4c009260 comp rx=0 tx=0).stop 2023-09-10T00:56:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.898+0000 7fdc60b4e700 1 -- 172.21.15.44:0/2643396209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc5c1055b0 msgr2=0x7fdc5c194390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.898+0000 7fdc60b4e700 1 --2- 172.21.15.44:0/2643396209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc5c1055b0 0x7fdc5c194390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdc540089f0 tx=0x7fdc54008d00 comp rx=0 tx=0).stop 2023-09-10T00:56:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.898+0000 7fdc60b4e700 1 -- 172.21.15.44:0/2643396209 shutdown_connections 2023-09-10T00:56:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.898+0000 7fdc60b4e700 1 --2- 172.21.15.44:0/2643396209 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdc440a07b0 0x7fdc440a2c70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.898+0000 7fdc60b4e700 1 --2- 172.21.15.44:0/2643396209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc5c1055b0 0x7fdc5c194390 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.898+0000 7fdc60b4e700 1 --2- 172.21.15.44:0/2643396209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c0ff760 0x7fdc5c19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.898+0000 7fdc60b4e700 1 --2- 172.21.15.44:0/2643396209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc5c100120 0x7fdc5c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.898+0000 7fdc60b4e700 1 -- 172.21.15.44:0/2643396209 >> 172.21.15.44:0/2643396209 conn(0x7fdc5c074f30 msgr2=0x7fdc5c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.898+0000 7fdc60b4e700 1 -- 172.21.15.44:0/2643396209 shutdown_connections 2023-09-10T00:56:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:22.898+0000 7fdc60b4e700 1 -- 172.21.15.44:0/2643396209 wait complete. 2023-09-10T00:56:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:22 smithi131 ceph-mon[103713]: pgmap v495: 129 pgs: 129 active+clean; 8.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 331 op/s 2023-09-10T00:56:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:22 smithi044 ceph-mon[98676]: pgmap v495: 129 pgs: 129 active+clean; 8.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 331 op/s 2023-09-10T00:56:23.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:22 smithi099 ceph-mon[102692]: pgmap v495: 129 pgs: 129 active+clean; 8.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 331 op/s 2023-09-10T00:56:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:24 smithi131 ceph-mon[103713]: pgmap v496: 129 pgs: 129 active+clean; 8.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 291 op/s 2023-09-10T00:56:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:24 smithi044 ceph-mon[98676]: pgmap v496: 129 pgs: 129 active+clean; 8.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 291 op/s 2023-09-10T00:56:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:24 smithi099 ceph-mon[102692]: pgmap v496: 129 pgs: 129 active+clean; 8.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 291 op/s 2023-09-10T00:56:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:26 smithi131 ceph-mon[103713]: pgmap v497: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 433 op/s 2023-09-10T00:56:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:26 smithi044 ceph-mon[98676]: pgmap v497: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 433 op/s 2023-09-10T00:56:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:26 smithi099 ceph-mon[102692]: pgmap v497: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 433 op/s 2023-09-10T00:56:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:28 smithi131 ceph-mon[103713]: pgmap v498: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2023-09-10T00:56:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:28 smithi044 ceph-mon[98676]: pgmap v498: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2023-09-10T00:56:29.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:28 smithi099 ceph-mon[102692]: pgmap v498: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2023-09-10T00:56:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:30 smithi131 ceph-mon[103713]: pgmap v499: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2023-09-10T00:56:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:30 smithi044 ceph-mon[98676]: pgmap v499: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2023-09-10T00:56:31.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:30 smithi099 ceph-mon[102692]: pgmap v499: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2023-09-10T00:56:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:32 smithi131 ceph-mon[103713]: pgmap v500: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 427 op/s 2023-09-10T00:56:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:32 smithi044 ceph-mon[98676]: pgmap v500: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 427 op/s 2023-09-10T00:56:33.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:32 smithi099 ceph-mon[102692]: pgmap v500: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 427 op/s 2023-09-10T00:56:35.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:34 smithi131 ceph-mon[103713]: pgmap v501: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 289 op/s 2023-09-10T00:56:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:34 smithi044 ceph-mon[98676]: pgmap v501: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 289 op/s 2023-09-10T00:56:35.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:34 smithi099 ceph-mon[102692]: pgmap v501: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 289 op/s 2023-09-10T00:56:36.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:35 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:36.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:35 smithi131 ceph-mon[103713]: pgmap v502: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 440 op/s 2023-09-10T00:56:36.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:35 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:36.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:35 smithi044 ceph-mon[98676]: pgmap v502: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 440 op/s 2023-09-10T00:56:36.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:35 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:36.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:35 smithi099 ceph-mon[102692]: pgmap v502: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 440 op/s 2023-09-10T00:56:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.409+0000 7f8ff9074700 1 -- 172.21.15.44:0/824711462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff40fee80 msgr2=0x7f8ff40ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.409+0000 7f8ff9074700 1 --2- 172.21.15.44:0/824711462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff40fee80 0x7f8ff40ff260 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009990 tx=0x7f8fdc009ca0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.409+0000 7f8ff9074700 1 -- 172.21.15.44:0/824711462 shutdown_connections 2023-09-10T00:56:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.409+0000 7f8ff9074700 1 --2- 172.21.15.44:0/824711462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff4103fa0 0x7f8ff410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.409+0000 7f8ff9074700 1 --2- 172.21.15.44:0/824711462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff40fee80 0x7f8ff40ff260 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.409+0000 7f8ff9074700 1 --2- 172.21.15.44:0/824711462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff40ff830 0x7f8ff4103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.409+0000 7f8ff9074700 1 -- 172.21.15.44:0/824711462 >> 172.21.15.44:0/824711462 conn(0x7f8ff40769d0 msgr2=0x7f8ff40fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.409+0000 7f8ff9074700 1 -- 172.21.15.44:0/824711462 shutdown_connections 2023-09-10T00:56:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.409+0000 7f8ff9074700 1 -- 172.21.15.44:0/824711462 wait complete. 2023-09-10T00:56:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.410+0000 7f8ff9074700 1 Processor -- start 2023-09-10T00:56:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff9074700 1 -- start start 2023-09-10T00:56:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff9074700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff40fee80 0x7f8ff419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff9074700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff4103fa0 0x7f8ff419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff9074700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff41963f0 0x7f8ff41968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff9074700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4196e70 con 0x7f8ff40fee80 2023-09-10T00:56:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff9074700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4196fb0 con 0x7f8ff41963f0 2023-09-10T00:56:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff9074700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4197170 con 0x7f8ff4103fa0 2023-09-10T00:56:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff40fee80 0x7f8ff419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff359e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff41963f0 0x7f8ff41968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff359e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff41963f0 0x7f8ff41968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55680/0 (socket says 172.21.15.44:55680) 2023-09-10T00:56:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff40fee80 0x7f8ff419c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53938/0 (socket says 172.21.15.44:53938) 2023-09-10T00:56:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff359e700 1 -- 172.21.15.44:0/444188915 learned_addr learned my addr 172.21.15.44:0/444188915 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff259c700 1 --2- 172.21.15.44:0/444188915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff4103fa0 0x7f8ff419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff359e700 1 -- 172.21.15.44:0/444188915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff4103fa0 msgr2=0x7f8ff419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff359e700 1 --2- 172.21.15.44:0/444188915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff4103fa0 0x7f8ff419cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff359e700 1 -- 172.21.15.44:0/444188915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff40fee80 msgr2=0x7f8ff419c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff359e700 1 --2- 172.21.15.44:0/444188915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff40fee80 0x7f8ff419c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.411+0000 7f8ff359e700 1 -- 172.21.15.44:0/444188915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe4009520 con 0x7f8ff41963f0 2023-09-10T00:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.412+0000 7f8ff359e700 1 --2- 172.21.15.44:0/444188915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff41963f0 0x7f8ff41968a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00c370 tx=0x7f8fec00c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.412+0000 7f8febfff700 1 -- 172.21.15.44:0/444188915 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec017070 con 0x7f8ff41963f0 2023-09-10T00:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.412+0000 7f8ff9074700 1 -- 172.21.15.44:0/444188915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec009c40 con 0x7f8ff41963f0 2023-09-10T00:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.412+0000 7f8febfff700 1 -- 172.21.15.44:0/444188915 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fec00f040 con 0x7f8ff41963f0 2023-09-10T00:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.412+0000 7f8febfff700 1 -- 172.21.15.44:0/444188915 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00e050 con 0x7f8ff41963f0 2023-09-10T00:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.412+0000 7f8ff9074700 1 -- 172.21.15.44:0/444188915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fdc0095f0 con 0x7f8ff41963f0 2023-09-10T00:56:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.414+0000 7f8febfff700 1 -- 172.21.15.44:0/444188915 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f8fec00cb60 con 0x7f8ff41963f0 2023-09-10T00:56:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.415+0000 7f8febfff700 1 --2- 172.21.15.44:0/444188915 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fe00a07b0 0x7f8fe00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.415+0000 7f8febfff700 1 -- 172.21.15.44:0/444188915 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8fec0cbcb0 con 0x7f8ff41963f0 2023-09-10T00:56:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.415+0000 7f8ff2d9d700 1 --2- 172.21.15.44:0/444188915 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fe00a07b0 0x7f8fe00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.416+0000 7f8ff2d9d700 1 --2- 172.21.15.44:0/444188915 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fe00a07b0 0x7f8fe00a2c70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc006010 tx=0x7f8fdc005960 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.416+0000 7f900523ab80 1 Processor -- start 2023-09-10T00:56:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.416+0000 7f900523ab80 1 -- start start 2023-09-10T00:56:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f900523ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d911a0 0x5640f7d91580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f900523ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d8b300 0x5640f7d8b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f900523ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d97f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f900523ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f7ce46d0 con 0x5640f7d911a0 2023-09-10T00:56:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f900523ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f7ce1ff0 con 0x5640f7d8b300 2023-09-10T00:56:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f900523ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f7cdd8e0 con 0x5640f7d8bcc0 2023-09-10T00:56:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f8ff2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d911a0 0x5640f7d91580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f8ff259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d8b300 0x5640f7d8b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f8ff359e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d97f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f8ff259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d8b300 0x5640f7d8b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55686/0 (socket says 172.21.15.44:55686) 2023-09-10T00:56:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f8ff359e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d97f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60832/0 (socket says 172.21.15.44:60832) 2023-09-10T00:56:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f8ff259c700 1 -- 172.21.15.44:0/84358825 learned_addr learned my addr 172.21.15.44:0/84358825 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f8ff2d9d700 1 --2- 172.21.15.44:0/84358825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d911a0 0x5640f7d91580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53946/0 (socket says 172.21.15.44:53946) 2023-09-10T00:56:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f8ff259c700 1 -- 172.21.15.44:0/84358825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 msgr2=0x5640f7d97f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f8ff259c700 1 --2- 172.21.15.44:0/84358825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d97f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f8ff259c700 1 -- 172.21.15.44:0/84358825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d911a0 msgr2=0x5640f7d91580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f8ff259c700 1 --2- 172.21.15.44:0/84358825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d911a0 0x5640f7d91580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f8ff259c700 1 -- 172.21.15.44:0/84358825 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640f7b09d80 con 0x5640f7d8b300 2023-09-10T00:56:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.417+0000 7f8ff359e700 1 --2- 172.21.15.44:0/84358825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d97f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:56:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.418+0000 7f8ff259c700 1 --2- 172.21.15.44:0/84358825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d8b300 0x5640f7d8b780 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009e80 tx=0x7f8fe4011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56bbae079c4f843b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.418+0000 7f8fe97fa700 1 -- 172.21.15.44:0/84358825 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4004250 con 0x5640f7d8b300 2023-09-10T00:56:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.418+0000 7f8fe97fa700 1 -- 172.21.15.44:0/84358825 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fe40043b0 con 0x5640f7d8b300 2023-09-10T00:56:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.418+0000 7f8fe97fa700 1 -- 172.21.15.44:0/84358825 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4019800 con 0x5640f7d8b300 2023-09-10T00:56:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.419+0000 7f900523ab80 1 -- 172.21.15.44:0/84358825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d8b300 msgr2=0x5640f7d8b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.419+0000 7f900523ab80 1 --2- 172.21.15.44:0/84358825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d8b300 0x5640f7d8b780 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009e80 tx=0x7f8fe4011720 comp rx=0 tx=0).stop 2023-09-10T00:56:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.419+0000 7f900523ab80 1 -- 172.21.15.44:0/84358825 shutdown_connections 2023-09-10T00:56:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.419+0000 7f900523ab80 1 --2- 172.21.15.44:0/84358825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d97f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.419+0000 7f900523ab80 1 --2- 172.21.15.44:0/84358825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d911a0 0x5640f7d91580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.419+0000 7f900523ab80 1 --2- 172.21.15.44:0/84358825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d8b300 0x5640f7d8b780 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.419+0000 7f900523ab80 1 -- 172.21.15.44:0/84358825 >> 172.21.15.44:0/84358825 conn(0x5640f7d00030 msgr2=0x5640f7d86560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.419+0000 7f900523ab80 1 -- 172.21.15.44:0/84358825 shutdown_connections 2023-09-10T00:56:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.419+0000 7f900523ab80 1 -- 172.21.15.44:0/84358825 wait complete. 2023-09-10T00:56:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.420+0000 7f900523ab80 1 -- start start 2023-09-10T00:56:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.420+0000 7f900523ab80 1 Processor -- start 2023-09-10T00:56:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.420+0000 7f900523ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d8b300 0x5640f7d97db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.420+0000 7f900523ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d950d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.420+0000 7f900523ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d911a0 0x5640f7d95690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.420+0000 7f900523ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f7c7bb10 con 0x5640f7d8b300 2023-09-10T00:56:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.420+0000 7f900523ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f7c7b810 con 0x5640f7d911a0 2023-09-10T00:56:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.420+0000 7f900523ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f7b5f8c0 con 0x5640f7d8bcc0 2023-09-10T00:56:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.420+0000 7f900523ab80 10 client.0 _ll_register_callbacks cb 0x5640f7bc5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:56:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.420+0000 7f8ff259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.420+0000 7f8ff259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d950d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60836/0 (socket says 172.21.15.44:60836) 2023-09-10T00:56:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.420+0000 7f8ff259c700 1 -- 172.21.15.44:0/2981811334 learned_addr learned my addr 172.21.15.44:0/2981811334 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.420+0000 7f8ff2d9d700 1 --2- 172.21.15.44:0/2981811334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d8b300 0x5640f7d97db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.420+0000 7f8ff359e700 1 --2- 172.21.15.44:0/2981811334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d911a0 0x5640f7d95690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.421+0000 7f8ff259c700 1 -- 172.21.15.44:0/2981811334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d911a0 msgr2=0x5640f7d95690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.421+0000 7f8ff259c700 1 --2- 172.21.15.44:0/2981811334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d911a0 0x5640f7d95690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.421+0000 7f8ff259c700 1 -- 172.21.15.44:0/2981811334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d8b300 msgr2=0x5640f7d97db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.421+0000 7f8ff259c700 1 --2- 172.21.15.44:0/2981811334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d8b300 0x5640f7d97db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.421+0000 7f8ff259c700 1 -- 172.21.15.44:0/2981811334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fdc0095f0 con 0x5640f7d8bcc0 2023-09-10T00:56:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.421+0000 7f8ff359e700 1 --2- 172.21.15.44:0/2981811334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d911a0 0x5640f7d95690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:56:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.421+0000 7f8ff259c700 1 --2- 172.21.15.44:0/2981811334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d950d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4003fd0 tx=0x7f8fe4003ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1b5ce60072c1ef0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.421+0000 7f8fda7fc700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:56:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.421+0000 7f8fda7fc700 1 -- 172.21.15.44:0/2981811334 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4003910 con 0x5640f7d8bcc0 2023-09-10T00:56:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.421+0000 7f900523ab80 1 -- 172.21.15.44:0/2981811334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640f7c35490 con 0x5640f7d8bcc0 2023-09-10T00:56:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.421+0000 7f8fda7fc700 1 -- 172.21.15.44:0/2981811334 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8fe4003a70 con 0x5640f7d8bcc0 2023-09-10T00:56:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.421+0000 7f8fda7fc700 1 -- 172.21.15.44:0/2981811334 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fe401a440 con 0x5640f7d8bcc0 2023-09-10T00:56:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.421+0000 7f900523ab80 10 client.27580 fetch_fsmap learned FSMap version 74 2023-09-10T00:56:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.421+0000 7f900523ab80 1 -- 172.21.15.44:0/2981811334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x7f8fe4009ab0 con 0x5640f7d8bcc0 2023-09-10T00:56:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.422+0000 7f8fda7fc700 1 -- 172.21.15.44:0/2981811334 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5640f7c35490 con 0x5640f7d8bcc0 2023-09-10T00:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.422+0000 7f8fda7fc700 1 -- 172.21.15.44:0/2981811334 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 74) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f8fe4023520 con 0x5640f7d8bcc0 2023-09-10T00:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.422+0000 7f900523ab80 10 client.27580 fetch_fsmap finished waiting for FSMap version 74 2023-09-10T00:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.422+0000 7f900523ab80 10 client.27580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.422+0000 7f900523ab80 20 client.27580 populate_metadata read hostname 'smithi044' 2023-09-10T00:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.422+0000 7f900523ab80 1 --2- 172.21.15.44:0/2981811334 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5640f7d9c370 0x5640f7db0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.422+0000 7f900523ab80 4 client.27580 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.422+0000 7f900523ab80 1 -- 172.21.15.44:0/2981811334 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640f7cf2840 con 0x5640f7d9c370 2023-09-10T00:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.422+0000 7f8ff2d9d700 1 --2- 172.21.15.44:0/2981811334 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5640f7d9c370 0x5640f7db0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.423+0000 7f8ff2d9d700 1 --2- 172.21.15.44:0/2981811334 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5640f7d9c370 0x5640f7db0210 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6888baca2f46dc9 server_cookie=d5fabe6429ebe288 in_seq=0 out_seq=0 2023-09-10T00:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.423+0000 7f8fda7fc700 10 client.27580 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.423+0000 7f8fda7fc700 1 -- 172.21.15.44:0/2981811334 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5640f7cf2840 con 0x5640f7d9c370 2023-09-10T00:56:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.423+0000 7f8fda7fc700 10 client.27580 handle_command_reply: tid=0 2023-09-10T00:56:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.423+0000 7f900523ab80 1 client.27580 shutdown 2023-09-10T00:56:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.423+0000 7f900523ab80 1 -- 172.21.15.44:0/2981811334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 msgr2=0x5640f7d950d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.423+0000 7f900523ab80 1 --2- 172.21.15.44:0/2981811334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d950d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4003fd0 tx=0x7f8fe4003ec0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.423+0000 7f900523ab80 1 -- 172.21.15.44:0/2981811334 shutdown_connections 2023-09-10T00:56:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.423+0000 7f900523ab80 1 --2- 172.21.15.44:0/2981811334 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5640f7d9c370 0x5640f7db0210 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.423+0000 7f900523ab80 1 --2- 172.21.15.44:0/2981811334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d911a0 0x5640f7d95690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.424+0000 7f900523ab80 1 --2- 172.21.15.44:0/2981811334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d8b300 0x5640f7d97db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.424+0000 7f900523ab80 1 --2- 172.21.15.44:0/2981811334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d950d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.424+0000 7f900523ab80 1 -- 172.21.15.44:0/2981811334 >> 172.21.15.44:0/2981811334 conn(0x5640f7d00030 msgr2=0x5640f7d857a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.424+0000 7f8fda7fc700 0 client.27580 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.424+0000 7f900523ab80 1 -- 172.21.15.44:0/2981811334 shutdown_connections 2023-09-10T00:56:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.424+0000 7f900523ab80 1 -- 172.21.15.44:0/2981811334 wait complete. 2023-09-10T00:56:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.424+0000 7f900523ab80 20 client.27580 trim_cache size 0 max 16384 2023-09-10T00:56:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.446+0000 7f900523ab80 1 Processor -- start 2023-09-10T00:56:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.447+0000 7f900523ab80 1 -- start start 2023-09-10T00:56:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.447+0000 7f900523ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d8b300 0x5640f7da0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.447+0000 7f900523ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d8db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.447+0000 7f900523ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d911a0 0x5640f7d8e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.447+0000 7f900523ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f7cf2840 con 0x5640f7d8b300 2023-09-10T00:56:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.447+0000 7f900523ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f7c35490 con 0x5640f7d911a0 2023-09-10T00:56:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.447+0000 7f900523ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f7d8c140 con 0x5640f7d8bcc0 2023-09-10T00:56:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.447+0000 7f8ff2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d8b300 0x5640f7da0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.447+0000 7f8ff2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d8b300 0x5640f7da0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53960/0 (socket says 172.21.15.44:53960) 2023-09-10T00:56:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.447+0000 7f8ff259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d8db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.447+0000 7f8ff359e700 1 --2- 172.21.15.44:0/3140983848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d911a0 0x5640f7d8e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.447+0000 7f8ff2d9d700 1 -- 172.21.15.44:0/3140983848 learned_addr learned my addr 172.21.15.44:0/3140983848 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.448+0000 7f8ff259c700 1 -- 172.21.15.44:0/3140983848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d911a0 msgr2=0x5640f7d8e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.448+0000 7f8ff259c700 1 --2- 172.21.15.44:0/3140983848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d911a0 0x5640f7d8e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.448+0000 7f8ff259c700 1 -- 172.21.15.44:0/3140983848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d8b300 msgr2=0x5640f7da0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.448+0000 7f8ff259c700 1 --2- 172.21.15.44:0/3140983848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d8b300 0x5640f7da0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.448+0000 7f8ff259c700 1 -- 172.21.15.44:0/3140983848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640f7af2c00 con 0x5640f7d8bcc0 2023-09-10T00:56:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.448+0000 7f8ff2d9d700 1 --2- 172.21.15.44:0/3140983848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d8b300 0x5640f7da0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:56:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.448+0000 7f8ff359e700 1 --2- 172.21.15.44:0/3140983848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d911a0 0x5640f7d8e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:56:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.448+0000 7f8ff259c700 1 --2- 172.21.15.44:0/3140983848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d8db60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009da0 tx=0x7f8fe4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5546f560f614ef1c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.448+0000 7f8fe9ffb700 1 -- 172.21.15.44:0/3140983848 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4023c40 con 0x5640f7d8bcc0 2023-09-10T00:56:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.448+0000 7f8fe9ffb700 1 -- 172.21.15.44:0/3140983848 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fe4003910 con 0x5640f7d8bcc0 2023-09-10T00:56:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.448+0000 7f8fe9ffb700 1 -- 172.21.15.44:0/3140983848 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40045d0 con 0x5640f7d8bcc0 2023-09-10T00:56:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.449+0000 7f900523ab80 1 -- 172.21.15.44:0/3140983848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 msgr2=0x5640f7d8db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.449+0000 7f900523ab80 1 --2- 172.21.15.44:0/3140983848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d8db60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009da0 tx=0x7f8fe4011720 comp rx=0 tx=0).stop 2023-09-10T00:56:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.449+0000 7f900523ab80 1 -- 172.21.15.44:0/3140983848 shutdown_connections 2023-09-10T00:56:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.449+0000 7f900523ab80 1 --2- 172.21.15.44:0/3140983848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d911a0 0x5640f7d8e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.449+0000 7f900523ab80 1 --2- 172.21.15.44:0/3140983848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d8b300 0x5640f7da0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.449+0000 7f900523ab80 1 --2- 172.21.15.44:0/3140983848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d8db60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.449+0000 7f900523ab80 1 -- 172.21.15.44:0/3140983848 >> 172.21.15.44:0/3140983848 conn(0x5640f7d00030 msgr2=0x5640f7d8f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.449+0000 7f900523ab80 1 -- 172.21.15.44:0/3140983848 shutdown_connections 2023-09-10T00:56:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.449+0000 7f900523ab80 1 -- 172.21.15.44:0/3140983848 wait complete. 2023-09-10T00:56:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.449+0000 7f900523ab80 1 -- start start 2023-09-10T00:56:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.450+0000 7f900523ab80 1 Processor -- start 2023-09-10T00:56:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.450+0000 7f900523ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d8b300 0x5640f7d94920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.450+0000 7f900523ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d94e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.450+0000 7f900523ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d911a0 0x5640f7d938a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.450+0000 7f900523ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f7da0de0 con 0x5640f7d911a0 2023-09-10T00:56:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.450+0000 7f900523ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f7d93de0 con 0x5640f7d8b300 2023-09-10T00:56:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.450+0000 7f900523ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640f7d93f70 con 0x5640f7d8bcc0 2023-09-10T00:56:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.450+0000 7f900523ab80 10 client.0 _ll_register_callbacks cb 0x7f8fdc005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:56:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.450+0000 7f8ff2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d8b300 0x5640f7d94920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.450+0000 7f8ff2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d8b300 0x5640f7d94920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55706/0 (socket says 172.21.15.44:55706) 2023-09-10T00:56:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.450+0000 7f8ff2d9d700 1 -- 172.21.15.44:0/2026802244 learned_addr learned my addr 172.21.15.44:0/2026802244 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.450+0000 7f8ff259c700 1 --2- 172.21.15.44:0/2026802244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d94e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.450+0000 7f8ff359e700 1 --2- 172.21.15.44:0/2026802244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d911a0 0x5640f7d938a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.451+0000 7f8ff2d9d700 1 -- 172.21.15.44:0/2026802244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 msgr2=0x5640f7d94e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.451+0000 7f8ff2d9d700 1 --2- 172.21.15.44:0/2026802244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d94e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.451+0000 7f8ff2d9d700 1 -- 172.21.15.44:0/2026802244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d911a0 msgr2=0x5640f7d938a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.451+0000 7f8ff2d9d700 1 --2- 172.21.15.44:0/2026802244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d911a0 0x5640f7d938a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.451+0000 7f8ff2d9d700 1 -- 172.21.15.44:0/2026802244 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff40674e0 con 0x5640f7d8b300 2023-09-10T00:56:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.451+0000 7f8ff2d9d700 1 --2- 172.21.15.44:0/2026802244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d8b300 0x5640f7d94920 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc005910 tx=0x7f8fdc009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6e9ff269d373d71 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.451+0000 7f8fda7fc700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:56:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.451+0000 7f8fda7fc700 1 -- 172.21.15.44:0/2026802244 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc036360 con 0x5640f7d8b300 2023-09-10T00:56:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.451+0000 7f8fda7fc700 1 -- 172.21.15.44:0/2026802244 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8fdc0364c0 con 0x5640f7d8b300 2023-09-10T00:56:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.451+0000 7f900523ab80 1 -- 172.21.15.44:0/2026802244 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fe40043b0 con 0x5640f7d8b300 2023-09-10T00:56:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.451+0000 7f8fda7fc700 1 -- 172.21.15.44:0/2026802244 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fdc004500 con 0x5640f7d8b300 2023-09-10T00:56:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.451+0000 7f900523ab80 10 client.27650 fetch_fsmap learned FSMap version 74 2023-09-10T00:56:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.451+0000 7f900523ab80 1 -- 172.21.15.44:0/2026802244 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x7f8fec021070 con 0x5640f7d8b300 2023-09-10T00:56:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.452+0000 7f8fda7fc700 1 -- 172.21.15.44:0/2026802244 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fe40043b0 con 0x5640f7d8b300 2023-09-10T00:56:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.452+0000 7f8fda7fc700 1 -- 172.21.15.44:0/2026802244 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 74) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f8fdc02f050 con 0x5640f7d8b300 2023-09-10T00:56:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.452+0000 7f900523ab80 10 client.27650 fetch_fsmap finished waiting for FSMap version 74 2023-09-10T00:56:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.452+0000 7f900523ab80 10 client.27650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:56:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.452+0000 7f900523ab80 20 client.27650 populate_metadata read hostname 'smithi044' 2023-09-10T00:56:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.452+0000 7f900523ab80 1 --2- 172.21.15.44:0/2026802244 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5640f7e02840 0x5640f7e04c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.452+0000 7f900523ab80 4 client.27650 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:56:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.452+0000 7f900523ab80 1 -- 172.21.15.44:0/2026802244 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5640f7cd68c0 con 0x5640f7e02840 2023-09-10T00:56:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.452+0000 7f8ff259c700 1 --2- 172.21.15.44:0/2026802244 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5640f7e02840 0x5640f7e04c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.452+0000 7f8fda7fc700 10 client.27650 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.452+0000 7f8ff259c700 1 --2- 172.21.15.44:0/2026802244 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5640f7e02840 0x5640f7e04c60 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d87b37ad211f4f33 server_cookie=a893e5a869485651 in_seq=0 out_seq=0 2023-09-10T00:56:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.463+0000 7f8fda7fc700 1 -- 172.21.15.44:0/2026802244 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5640f7cd68c0 con 0x5640f7e02840 2023-09-10T00:56:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.463+0000 7f8fda7fc700 10 client.27650 handle_command_reply: tid=0 2023-09-10T00:56:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.463+0000 7f900523ab80 1 client.27650 shutdown 2023-09-10T00:56:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.463+0000 7f900523ab80 1 -- 172.21.15.44:0/2026802244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d8b300 msgr2=0x5640f7d94920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.463+0000 7f900523ab80 1 --2- 172.21.15.44:0/2026802244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d8b300 0x5640f7d94920 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc005910 tx=0x7f8fdc009ca0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.463+0000 7f900523ab80 1 -- 172.21.15.44:0/2026802244 shutdown_connections 2023-09-10T00:56:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.463+0000 7f900523ab80 1 --2- 172.21.15.44:0/2026802244 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5640f7e02840 0x5640f7e04c60 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.463+0000 7f900523ab80 1 --2- 172.21.15.44:0/2026802244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640f7d911a0 0x5640f7d938a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.463+0000 7f900523ab80 1 --2- 172.21.15.44:0/2026802244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640f7d8b300 0x5640f7d94920 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.463+0000 7f900523ab80 1 --2- 172.21.15.44:0/2026802244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640f7d8bcc0 0x5640f7d94e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.463+0000 7f900523ab80 1 -- 172.21.15.44:0/2026802244 >> 172.21.15.44:0/2026802244 conn(0x5640f7d00030 msgr2=0x5640f7d901d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.463+0000 7f8fda7fc700 0 client.27650 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.463+0000 7f900523ab80 1 -- 172.21.15.44:0/2026802244 shutdown_connections 2023-09-10T00:56:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.463+0000 7f900523ab80 1 -- 172.21.15.44:0/2026802244 wait complete. 2023-09-10T00:56:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.463+0000 7f900523ab80 20 client.27650 trim_cache size 0 max 16384 2023-09-10T00:56:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:56:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:56:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:56:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:56:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.464+0000 7f8ff9074700 1 -- 172.21.15.44:0/444188915 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fe00a07b0 msgr2=0x7f8fe00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.464+0000 7f8ff9074700 1 --2- 172.21.15.44:0/444188915 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fe00a07b0 0x7f8fe00a2c70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc006010 tx=0x7f8fdc005960 comp rx=0 tx=0).stop 2023-09-10T00:56:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.464+0000 7f8ff9074700 1 -- 172.21.15.44:0/444188915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff41963f0 msgr2=0x7f8ff41968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.464+0000 7f8ff9074700 1 --2- 172.21.15.44:0/444188915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff41963f0 0x7f8ff41968a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00c370 tx=0x7f8fec00c680 comp rx=0 tx=0).stop 2023-09-10T00:56:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.464+0000 7f8ff9074700 1 -- 172.21.15.44:0/444188915 shutdown_connections 2023-09-10T00:56:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.464+0000 7f8ff9074700 1 --2- 172.21.15.44:0/444188915 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8fe00a07b0 0x7f8fe00a2c70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.464+0000 7f8ff9074700 1 --2- 172.21.15.44:0/444188915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff41963f0 0x7f8ff41968a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.464+0000 7f8ff9074700 1 --2- 172.21.15.44:0/444188915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff40fee80 0x7f8ff419c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.464+0000 7f8ff9074700 1 --2- 172.21.15.44:0/444188915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff4103fa0 0x7f8ff419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.464+0000 7f8ff9074700 1 -- 172.21.15.44:0/444188915 >> 172.21.15.44:0/444188915 conn(0x7f8ff40769d0 msgr2=0x7f8ff4103660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.464+0000 7f8ff9074700 1 -- 172.21.15.44:0/444188915 shutdown_connections 2023-09-10T00:56:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:36.464+0000 7f8ff9074700 1 -- 172.21.15.44:0/444188915 wait complete. 2023-09-10T00:56:38.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:38 smithi099 ceph-mon[102692]: pgmap v503: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 297 op/s 2023-09-10T00:56:38.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:38 smithi131 ceph-mon[103713]: pgmap v503: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 297 op/s 2023-09-10T00:56:38.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:38 smithi044 ceph-mon[98676]: pgmap v503: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 297 op/s 2023-09-10T00:56:40.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:40 smithi099 ceph-mon[102692]: pgmap v504: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 297 op/s 2023-09-10T00:56:40.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:40 smithi131 ceph-mon[103713]: pgmap v504: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 297 op/s 2023-09-10T00:56:40.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:40 smithi044 ceph-mon[98676]: pgmap v504: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 297 op/s 2023-09-10T00:56:41.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:41 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:41.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:41 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:41.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:41 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:42 smithi131 ceph-mon[103713]: pgmap v505: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 443 op/s 2023-09-10T00:56:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:42 smithi044 ceph-mon[98676]: pgmap v505: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 443 op/s 2023-09-10T00:56:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:42 smithi099 ceph-mon[102692]: pgmap v505: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 443 op/s 2023-09-10T00:56:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:44 smithi131 ceph-mon[103713]: pgmap v506: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 295 op/s 2023-09-10T00:56:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:44 smithi044 ceph-mon[98676]: pgmap v506: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 295 op/s 2023-09-10T00:56:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:44 smithi099 ceph-mon[102692]: pgmap v506: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 295 op/s 2023-09-10T00:56:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:46 smithi131 ceph-mon[103713]: pgmap v507: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 410 op/s 2023-09-10T00:56:47.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:46 smithi044 ceph-mon[98676]: pgmap v507: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 410 op/s 2023-09-10T00:56:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:46 smithi099 ceph-mon[102692]: pgmap v507: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 410 op/s 2023-09-10T00:56:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:48 smithi131 ceph-mon[103713]: pgmap v508: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2023-09-10T00:56:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:48 smithi044 ceph-mon[98676]: pgmap v508: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2023-09-10T00:56:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:48 smithi099 ceph-mon[102692]: pgmap v508: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2023-09-10T00:56:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.609+0000 7f602dc60700 1 -- 172.21.15.44:0/2838324832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6028106680 msgr2=0x7f6028106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.609+0000 7f602dc60700 1 --2- 172.21.15.44:0/2838324832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6028106680 0x7f6028106a60 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f60180099c0 tx=0x7f6018009cd0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.609+0000 7f602dc60700 1 -- 172.21.15.44:0/2838324832 shutdown_connections 2023-09-10T00:56:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.609+0000 7f602dc60700 1 --2- 172.21.15.44:0/2838324832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6028100f80 0x7f602810d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.609+0000 7f602dc60700 1 --2- 172.21.15.44:0/2838324832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6028106680 0x7f6028106a60 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.609+0000 7f602dc60700 1 --2- 172.21.15.44:0/2838324832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60281005c0 0x7f6028100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.609+0000 7f602dc60700 1 -- 172.21.15.44:0/2838324832 >> 172.21.15.44:0/2838324832 conn(0x7f60280fc050 msgr2=0x7f60280fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.609+0000 7f602dc60700 1 -- 172.21.15.44:0/2838324832 shutdown_connections 2023-09-10T00:56:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.609+0000 7f602dc60700 1 -- 172.21.15.44:0/2838324832 wait complete. 2023-09-10T00:56:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.610+0000 7f602dc60700 1 Processor -- start 2023-09-10T00:56:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.610+0000 7f602dc60700 1 -- start start 2023-09-10T00:56:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f602dc60700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60281005c0 0x7f602819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f602dc60700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6028100f80 0x7f602819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f602dc60700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6028106680 0x7f6028196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f602dc60700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6028196be0 con 0x7f60281005c0 2023-09-10T00:56:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f602dc60700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6028196d50 con 0x7f6028100f80 2023-09-10T00:56:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f602dc60700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6028196f10 con 0x7f6028106680 2023-09-10T00:56:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f60277fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60281005c0 0x7f602819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f60277fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60281005c0 0x7f602819c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53980/0 (socket says 172.21.15.44:53980) 2023-09-10T00:56:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f60277fe700 1 -- 172.21.15.44:0/1334597393 learned_addr learned my addr 172.21.15.44:0/1334597393 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f6027fff700 1 --2- 172.21.15.44:0/1334597393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6028106680 0x7f6028196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f6026ffd700 1 --2- 172.21.15.44:0/1334597393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6028100f80 0x7f602819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f60277fe700 1 -- 172.21.15.44:0/1334597393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6028106680 msgr2=0x7f6028196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f60277fe700 1 --2- 172.21.15.44:0/1334597393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6028106680 0x7f6028196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f60277fe700 1 -- 172.21.15.44:0/1334597393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6028100f80 msgr2=0x7f602819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f60277fe700 1 --2- 172.21.15.44:0/1334597393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6028100f80 0x7f602819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f60277fe700 1 -- 172.21.15.44:0/1334597393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60180095f0 con 0x7f60281005c0 2023-09-10T00:56:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.611+0000 7f60277fe700 1 --2- 172.21.15.44:0/1334597393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60281005c0 0x7f602819c710 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6018005910 tx=0x7f601800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.612+0000 7f6024ff9700 1 -- 172.21.15.44:0/1334597393 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601801d070 con 0x7f60281005c0 2023-09-10T00:56:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.612+0000 7f6024ff9700 1 -- 172.21.15.44:0/1334597393 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6018022470 con 0x7f60281005c0 2023-09-10T00:56:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.612+0000 7f6024ff9700 1 -- 172.21.15.44:0/1334597393 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6018017650 con 0x7f60281005c0 2023-09-10T00:56:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.612+0000 7f602dc60700 1 -- 172.21.15.44:0/1334597393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6028197190 con 0x7f60281005c0 2023-09-10T00:56:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.612+0000 7f602dc60700 1 -- 172.21.15.44:0/1334597393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6028072a10 con 0x7f60281005c0 2023-09-10T00:56:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.614+0000 7f6024ff9700 1 -- 172.21.15.44:0/1334597393 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f6018022850 con 0x7f60281005c0 2023-09-10T00:56:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.615+0000 7f6024ff9700 1 --2- 172.21.15.44:0/1334597393 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f60100a07b0 0x7f60100a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.615+0000 7f6024ff9700 1 -- 172.21.15.44:0/1334597393 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f60180cd510 con 0x7f60281005c0 2023-09-10T00:56:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.615+0000 7f6026ffd700 1 --2- 172.21.15.44:0/1334597393 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f60100a07b0 0x7f60100a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.615+0000 7f6026ffd700 1 --2- 172.21.15.44:0/1334597393 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f60100a07b0 0x7f60100a2c70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f60281979e0 tx=0x7f601c009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f6039e26b80 1 Processor -- start 2023-09-10T00:56:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f6039e26b80 1 -- start start 2023-09-10T00:56:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f6039e26b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619337251a0 0x561933725580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f6039e26b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371f300 0x56193371f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f6039e26b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371fcc0 0x56193372bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f6039e26b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619336786d0 con 0x56193371fcc0 2023-09-10T00:56:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f6039e26b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561933675ff0 con 0x5619337251a0 2023-09-10T00:56:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f6039e26b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619336718e0 con 0x56193371f300 2023-09-10T00:56:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f6026ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371f300 0x56193371f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f6027fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371fcc0 0x56193372bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f6026ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371f300 0x56193371f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60872/0 (socket says 172.21.15.44:60872) 2023-09-10T00:56:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f6026ffd700 1 -- 172.21.15.44:0/4050277630 learned_addr learned my addr 172.21.15.44:0/4050277630 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f60277fe700 1 --2- 172.21.15.44:0/4050277630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619337251a0 0x561933725580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f6027fff700 1 -- 172.21.15.44:0/4050277630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371f300 msgr2=0x56193371f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f6027fff700 1 --2- 172.21.15.44:0/4050277630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371f300 0x56193371f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f6027fff700 1 -- 172.21.15.44:0/4050277630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619337251a0 msgr2=0x561933725580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.616+0000 7f6027fff700 1 --2- 172.21.15.44:0/4050277630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619337251a0 0x561933725580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.617+0000 7f6027fff700 1 -- 172.21.15.44:0/4050277630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56193349dd80 con 0x56193371fcc0 2023-09-10T00:56:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.617+0000 7f6027fff700 1 --2- 172.21.15.44:0/4050277630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371fcc0 0x56193372bf30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f6020011860 tx=0x7f602000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a73e773553ec1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.617+0000 7f600dffb700 1 -- 172.21.15.44:0/4050277630 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020017040 con 0x56193371fcc0 2023-09-10T00:56:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.617+0000 7f600dffb700 1 -- 172.21.15.44:0/4050277630 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f60200093a0 con 0x56193371fcc0 2023-09-10T00:56:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.617+0000 7f600dffb700 1 -- 172.21.15.44:0/4050277630 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020007760 con 0x56193371fcc0 2023-09-10T00:56:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.618+0000 7f6039e26b80 1 -- 172.21.15.44:0/4050277630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371fcc0 msgr2=0x56193372bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.618+0000 7f6039e26b80 1 --2- 172.21.15.44:0/4050277630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371fcc0 0x56193372bf30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f6020011860 tx=0x7f602000b040 comp rx=0 tx=0).stop 2023-09-10T00:56:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.618+0000 7f6039e26b80 1 -- 172.21.15.44:0/4050277630 shutdown_connections 2023-09-10T00:56:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.618+0000 7f6039e26b80 1 --2- 172.21.15.44:0/4050277630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371fcc0 0x56193372bf30 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.618+0000 7f6039e26b80 1 --2- 172.21.15.44:0/4050277630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619337251a0 0x561933725580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.618+0000 7f6039e26b80 1 --2- 172.21.15.44:0/4050277630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371f300 0x56193371f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.618+0000 7f6039e26b80 1 -- 172.21.15.44:0/4050277630 >> 172.21.15.44:0/4050277630 conn(0x561933694030 msgr2=0x56193371a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.618+0000 7f6039e26b80 1 -- 172.21.15.44:0/4050277630 shutdown_connections 2023-09-10T00:56:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.618+0000 7f6039e26b80 1 -- 172.21.15.44:0/4050277630 wait complete. 2023-09-10T00:56:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.618+0000 7f6039e26b80 1 -- start start 2023-09-10T00:56:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f6039e26b80 1 Processor -- start 2023-09-10T00:56:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f6039e26b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371f300 0x56193372bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f6039e26b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371fcc0 0x561933729090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f6039e26b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619337251a0 0x561933729610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f6039e26b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193360fb10 con 0x56193371f300 2023-09-10T00:56:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f6039e26b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193360f810 con 0x5619337251a0 2023-09-10T00:56:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f6039e26b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619334f38c0 con 0x56193371fcc0 2023-09-10T00:56:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f6039e26b80 10 client.0 _ll_register_callbacks cb 0x561933559a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:56:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f60277fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371f300 0x56193372bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f6026ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371fcc0 0x561933729090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f60277fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371f300 0x56193372bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53994/0 (socket says 172.21.15.44:53994) 2023-09-10T00:56:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f60277fe700 1 -- 172.21.15.44:0/2756997570 learned_addr learned my addr 172.21.15.44:0/2756997570 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f6027fff700 1 --2- 172.21.15.44:0/2756997570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619337251a0 0x561933729610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f60277fe700 1 -- 172.21.15.44:0/2756997570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371fcc0 msgr2=0x561933729090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f60277fe700 1 --2- 172.21.15.44:0/2756997570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371fcc0 0x561933729090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f60277fe700 1 -- 172.21.15.44:0/2756997570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619337251a0 msgr2=0x561933729610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f60277fe700 1 --2- 172.21.15.44:0/2756997570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619337251a0 0x561933729610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f60277fe700 1 -- 172.21.15.44:0/2756997570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6020011540 con 0x56193371f300 2023-09-10T00:56:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f6026ffd700 1 --2- 172.21.15.44:0/2756997570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371fcc0 0x561933729090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:56:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f60277fe700 1 --2- 172.21.15.44:0/2756997570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371f300 0x56193372bde0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f601800b580 tx=0x7f601809ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5f13aff0911b61 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.619+0000 7f6006ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:56:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.620+0000 7f6006ffd700 1 -- 172.21.15.44:0/2756997570 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601809e350 con 0x56193371f300 2023-09-10T00:56:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.620+0000 7f6006ffd700 1 -- 172.21.15.44:0/2756997570 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f601809e520 con 0x56193371f300 2023-09-10T00:56:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.620+0000 7f6039e26b80 1 -- 172.21.15.44:0/2756997570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619335c9490 con 0x56193371f300 2023-09-10T00:56:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.620+0000 7f6006ffd700 1 -- 172.21.15.44:0/2756997570 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60180a26d0 con 0x56193371f300 2023-09-10T00:56:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.620+0000 7f6039e26b80 10 client.18006 fetch_fsmap learned FSMap version 75 2023-09-10T00:56:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.620+0000 7f6039e26b80 1 -- 172.21.15.44:0/2756997570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) v3 -- 0x561933486c00 con 0x56193371f300 2023-09-10T00:56:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.620+0000 7f6006ffd700 1 -- 172.21.15.44:0/2756997570 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5619335c9490 con 0x56193371f300 2023-09-10T00:56:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.620+0000 7f6006ffd700 1 -- 172.21.15.44:0/2756997570 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 75) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f6018099be0 con 0x56193371f300 2023-09-10T00:56:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.620+0000 7f6039e26b80 10 client.18006 fetch_fsmap finished waiting for FSMap version 75 2023-09-10T00:56:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.620+0000 7f6039e26b80 10 client.18006 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:56:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.620+0000 7f6039e26b80 20 client.18006 populate_metadata read hostname 'smithi044' 2023-09-10T00:56:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.620+0000 7f6039e26b80 1 --2- 172.21.15.44:0/2756997570 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561933739080 0x56193373b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.620+0000 7f6039e26b80 4 client.18006 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:56:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.620+0000 7f6039e26b80 1 -- 172.21.15.44:0/2756997570 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561933686840 con 0x561933739080 2023-09-10T00:56:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.621+0000 7f6026ffd700 1 --2- 172.21.15.44:0/2756997570 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561933739080 0x56193373b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.621+0000 7f6026ffd700 1 --2- 172.21.15.44:0/2756997570 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561933739080 0x56193373b4a0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b19c2ba44187955 server_cookie=615de88eb327d0cf in_seq=0 out_seq=0 2023-09-10T00:56:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.621+0000 7f6006ffd700 10 client.18006 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.622+0000 7f6006ffd700 1 -- 172.21.15.44:0/2756997570 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561933686840 con 0x561933739080 2023-09-10T00:56:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.622+0000 7f6006ffd700 10 client.18006 handle_command_reply: tid=0 2023-09-10T00:56:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.622+0000 7f6039e26b80 1 client.18006 shutdown 2023-09-10T00:56:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.622+0000 7f6039e26b80 1 -- 172.21.15.44:0/2756997570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371f300 msgr2=0x56193372bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.622+0000 7f6039e26b80 1 --2- 172.21.15.44:0/2756997570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371f300 0x56193372bde0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f601800b580 tx=0x7f601809ae60 comp rx=0 tx=0).stop 2023-09-10T00:56:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.622+0000 7f6039e26b80 1 -- 172.21.15.44:0/2756997570 shutdown_connections 2023-09-10T00:56:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.622+0000 7f6039e26b80 1 --2- 172.21.15.44:0/2756997570 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561933739080 0x56193373b4a0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.622+0000 7f6039e26b80 1 --2- 172.21.15.44:0/2756997570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619337251a0 0x561933729610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.622+0000 7f6039e26b80 1 --2- 172.21.15.44:0/2756997570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371f300 0x56193372bde0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.622+0000 7f6039e26b80 1 --2- 172.21.15.44:0/2756997570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371fcc0 0x561933729090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.622+0000 7f6039e26b80 1 -- 172.21.15.44:0/2756997570 >> 172.21.15.44:0/2756997570 conn(0x561933694030 msgr2=0x5619337197a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.622+0000 7f6006ffd700 0 client.18006 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.622+0000 7f6039e26b80 1 -- 172.21.15.44:0/2756997570 shutdown_connections 2023-09-10T00:56:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.622+0000 7f6039e26b80 1 -- 172.21.15.44:0/2756997570 wait complete. 2023-09-10T00:56:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.622+0000 7f6039e26b80 20 client.18006 trim_cache size 0 max 16384 2023-09-10T00:56:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f6039e26b80 1 Processor -- start 2023-09-10T00:56:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f6039e26b80 1 -- start start 2023-09-10T00:56:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f6039e26b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371f300 0x561933724740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f6039e26b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371fcc0 0x56193372b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f6039e26b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619337251a0 0x56193372bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f6039e26b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193371bb40 con 0x56193371f300 2023-09-10T00:56:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f6039e26b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193372a200 con 0x5619337251a0 2023-09-10T00:56:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f6039e26b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619335c9490 con 0x56193371fcc0 2023-09-10T00:56:49.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f60277fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371f300 0x561933724740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f6026ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371fcc0 0x56193372b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f60277fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371f300 0x561933724740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54002/0 (socket says 172.21.15.44:54002) 2023-09-10T00:56:49.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f60277fe700 1 -- 172.21.15.44:0/3992419350 learned_addr learned my addr 172.21.15.44:0/3992419350 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:49.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f6027fff700 1 --2- 172.21.15.44:0/3992419350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619337251a0 0x56193372bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f6026ffd700 1 -- 172.21.15.44:0/3992419350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619337251a0 msgr2=0x56193372bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f6026ffd700 1 --2- 172.21.15.44:0/3992419350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619337251a0 0x56193372bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f6026ffd700 1 -- 172.21.15.44:0/3992419350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371f300 msgr2=0x561933724740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f6026ffd700 1 --2- 172.21.15.44:0/3992419350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371f300 0x561933724740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.640+0000 7f6026ffd700 1 -- 172.21.15.44:0/3992419350 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619334c41f0 con 0x56193371fcc0 2023-09-10T00:56:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.641+0000 7f6026ffd700 1 --2- 172.21.15.44:0/3992419350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371fcc0 0x56193372b8a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x56193373bd70 tx=0x7f601c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb499a48cd3cc167 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.641+0000 7f600e7fc700 1 -- 172.21.15.44:0/3992419350 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c022030 con 0x56193371fcc0 2023-09-10T00:56:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.641+0000 7f600e7fc700 1 -- 172.21.15.44:0/3992419350 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f601c023040 con 0x56193371fcc0 2023-09-10T00:56:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.641+0000 7f600e7fc700 1 -- 172.21.15.44:0/3992419350 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c003d00 con 0x56193371fcc0 2023-09-10T00:56:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.642+0000 7f6039e26b80 1 -- 172.21.15.44:0/3992419350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371fcc0 msgr2=0x56193372b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.642+0000 7f6039e26b80 1 --2- 172.21.15.44:0/3992419350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371fcc0 0x56193372b8a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x56193373bd70 tx=0x7f601c014000 comp rx=0 tx=0).stop 2023-09-10T00:56:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.642+0000 7f6039e26b80 1 -- 172.21.15.44:0/3992419350 shutdown_connections 2023-09-10T00:56:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.642+0000 7f6039e26b80 1 --2- 172.21.15.44:0/3992419350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619337251a0 0x56193372bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.642+0000 7f6039e26b80 1 --2- 172.21.15.44:0/3992419350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56193371f300 0x561933724740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.642+0000 7f6039e26b80 1 --2- 172.21.15.44:0/3992419350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371fcc0 0x56193372b8a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.642+0000 7f6039e26b80 1 -- 172.21.15.44:0/3992419350 >> 172.21.15.44:0/3992419350 conn(0x561933694030 msgr2=0x56193371eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.642+0000 7f6039e26b80 1 -- 172.21.15.44:0/3992419350 shutdown_connections 2023-09-10T00:56:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.642+0000 7f6039e26b80 1 -- 172.21.15.44:0/3992419350 wait complete. 2023-09-10T00:56:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.642+0000 7f6039e26b80 1 -- start start 2023-09-10T00:56:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.642+0000 7f6039e26b80 1 Processor -- start 2023-09-10T00:56:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6039e26b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371f300 0x56193372ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6039e26b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56193371fcc0 0x56193372f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6039e26b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619337251a0 0x56193372f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6039e26b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561933782e10 con 0x5619337251a0 2023-09-10T00:56:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6039e26b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561933782f80 con 0x56193371fcc0 2023-09-10T00:56:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6039e26b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561933783140 con 0x56193371f300 2023-09-10T00:56:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6039e26b80 10 client.0 _ll_register_callbacks cb 0x7f601809a300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:56:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6027fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619337251a0 0x56193372f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6027fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619337251a0 0x56193372f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54012/0 (socket says 172.21.15.44:54012) 2023-09-10T00:56:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6027fff700 1 -- 172.21.15.44:0/2785676002 learned_addr learned my addr 172.21.15.44:0/2785676002 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:56:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6026ffd700 1 --2- 172.21.15.44:0/2785676002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56193371fcc0 0x56193372f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f60277fe700 1 --2- 172.21.15.44:0/2785676002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371f300 0x56193372ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6027fff700 1 -- 172.21.15.44:0/2785676002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371f300 msgr2=0x56193372ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6027fff700 1 --2- 172.21.15.44:0/2785676002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371f300 0x56193372ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6027fff700 1 -- 172.21.15.44:0/2785676002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56193371fcc0 msgr2=0x56193372f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:56:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6027fff700 1 --2- 172.21.15.44:0/2785676002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56193371fcc0 0x56193372f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6027fff700 1 -- 172.21.15.44:0/2785676002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601c017070 con 0x5619337251a0 2023-09-10T00:56:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f60277fe700 1 --2- 172.21.15.44:0/2785676002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371f300 0x56193372ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:56:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6027fff700 1 --2- 172.21.15.44:0/2785676002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619337251a0 0x56193372f630 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6020011e50 tx=0x7f6020003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54045823025cbfe5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:56:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6006ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:56:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6006ffd700 1 -- 172.21.15.44:0/2785676002 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602000e070 con 0x5619337251a0 2023-09-10T00:56:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6006ffd700 1 -- 172.21.15.44:0/2785676002 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f6020011360 con 0x5619337251a0 2023-09-10T00:56:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.643+0000 7f6039e26b80 1 -- 172.21.15.44:0/2785676002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60200093a0 con 0x5619337251a0 2023-09-10T00:56:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.644+0000 7f6006ffd700 1 -- 172.21.15.44:0/2785676002 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6020017040 con 0x5619337251a0 2023-09-10T00:56:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.644+0000 7f6039e26b80 10 client.18012 fetch_fsmap learned FSMap version 75 2023-09-10T00:56:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.644+0000 7f6039e26b80 1 -- 172.21.15.44:0/2785676002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) v3 -- 0x56193349c250 con 0x5619337251a0 2023-09-10T00:56:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.644+0000 7f6006ffd700 1 -- 172.21.15.44:0/2785676002 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f60200093a0 con 0x5619337251a0 2023-09-10T00:56:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.644+0000 7f6006ffd700 1 -- 172.21.15.44:0/2785676002 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 75) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f60200054f0 con 0x5619337251a0 2023-09-10T00:56:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.644+0000 7f6039e26b80 10 client.18012 fetch_fsmap finished waiting for FSMap version 75 2023-09-10T00:56:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.644+0000 7f6039e26b80 10 client.18012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:56:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.644+0000 7f6039e26b80 20 client.18012 populate_metadata read hostname 'smithi044' 2023-09-10T00:56:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.644+0000 7f6039e26b80 1 --2- 172.21.15.44:0/2785676002 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5619337847b0 0x561933786bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:56:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.644+0000 7f6039e26b80 4 client.18012 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:56:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.644+0000 7f6039e26b80 1 -- 172.21.15.44:0/2785676002 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56193366a8c0 con 0x5619337847b0 2023-09-10T00:56:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.644+0000 7f60277fe700 1 --2- 172.21.15.44:0/2785676002 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5619337847b0 0x561933786bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:56:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.645+0000 7f6006ffd700 10 client.18012 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.645+0000 7f60277fe700 1 --2- 172.21.15.44:0/2785676002 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5619337847b0 0x561933786bd0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79aab5cc0661b603 server_cookie=4e9075b722000315 in_seq=0 out_seq=0 2023-09-10T00:56:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.660+0000 7f6006ffd700 1 -- 172.21.15.44:0/2785676002 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56193366a8c0 con 0x5619337847b0 2023-09-10T00:56:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.660+0000 7f6006ffd700 10 client.18012 handle_command_reply: tid=0 2023-09-10T00:56:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.660+0000 7f6039e26b80 1 client.18012 shutdown 2023-09-10T00:56:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.660+0000 7f6039e26b80 1 -- 172.21.15.44:0/2785676002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619337251a0 msgr2=0x56193372f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.660+0000 7f6039e26b80 1 --2- 172.21.15.44:0/2785676002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619337251a0 0x56193372f630 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6020011e50 tx=0x7f6020003c10 comp rx=0 tx=0).stop 2023-09-10T00:56:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.660+0000 7f6039e26b80 1 -- 172.21.15.44:0/2785676002 shutdown_connections 2023-09-10T00:56:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.660+0000 7f6039e26b80 1 --2- 172.21.15.44:0/2785676002 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5619337847b0 0x561933786bd0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.660+0000 7f6039e26b80 1 --2- 172.21.15.44:0/2785676002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619337251a0 0x56193372f630 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.660+0000 7f6039e26b80 1 --2- 172.21.15.44:0/2785676002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56193371f300 0x56193372ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.660+0000 7f6039e26b80 1 --2- 172.21.15.44:0/2785676002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56193371fcc0 0x56193372f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.660+0000 7f6006ffd700 0 client.18012 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:56:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.660+0000 7f6039e26b80 1 -- 172.21.15.44:0/2785676002 >> 172.21.15.44:0/2785676002 conn(0x561933694030 msgr2=0x56193371eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.660+0000 7f6039e26b80 1 -- 172.21.15.44:0/2785676002 shutdown_connections 2023-09-10T00:56:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.660+0000 7f6039e26b80 1 -- 172.21.15.44:0/2785676002 wait complete. 2023-09-10T00:56:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.660+0000 7f6039e26b80 20 client.18012 trim_cache size 0 max 16384 2023-09-10T00:56:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:56:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:56:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:56:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:56:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.661+0000 7f602dc60700 1 -- 172.21.15.44:0/1334597393 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f60100a07b0 msgr2=0x7f60100a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.661+0000 7f602dc60700 1 --2- 172.21.15.44:0/1334597393 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f60100a07b0 0x7f60100a2c70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f60281979e0 tx=0x7f601c009260 comp rx=0 tx=0).stop 2023-09-10T00:56:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.661+0000 7f602dc60700 1 -- 172.21.15.44:0/1334597393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60281005c0 msgr2=0x7f602819c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:56:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.661+0000 7f602dc60700 1 --2- 172.21.15.44:0/1334597393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60281005c0 0x7f602819c710 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6018005910 tx=0x7f601800f690 comp rx=0 tx=0).stop 2023-09-10T00:56:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.661+0000 7f602dc60700 1 -- 172.21.15.44:0/1334597393 shutdown_connections 2023-09-10T00:56:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.661+0000 7f602dc60700 1 --2- 172.21.15.44:0/1334597393 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f60100a07b0 0x7f60100a2c70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.661+0000 7f602dc60700 1 --2- 172.21.15.44:0/1334597393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6028106680 0x7f6028196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.661+0000 7f602dc60700 1 --2- 172.21.15.44:0/1334597393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60281005c0 0x7f602819c710 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.661+0000 7f602dc60700 1 --2- 172.21.15.44:0/1334597393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6028100f80 0x7f602819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:56:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.661+0000 7f602dc60700 1 -- 172.21.15.44:0/1334597393 >> 172.21.15.44:0/1334597393 conn(0x7f60280fc050 msgr2=0x7f60280fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:56:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.661+0000 7f602dc60700 1 -- 172.21.15.44:0/1334597393 shutdown_connections 2023-09-10T00:56:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:56:49.661+0000 7f602dc60700 1 -- 172.21.15.44:0/1334597393 wait complete. 2023-09-10T00:56:51.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:50 smithi131 ceph-mon[103713]: pgmap v509: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2023-09-10T00:56:51.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:50 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:50 smithi044 ceph-mon[98676]: pgmap v509: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2023-09-10T00:56:51.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:50 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:51.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:50 smithi099 ceph-mon[102692]: pgmap v509: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2023-09-10T00:56:51.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:50 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:52 smithi131 ceph-mon[103713]: pgmap v510: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 398 op/s 2023-09-10T00:56:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:52 smithi044 ceph-mon[98676]: pgmap v510: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 398 op/s 2023-09-10T00:56:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:52 smithi099 ceph-mon[102692]: pgmap v510: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 398 op/s 2023-09-10T00:56:55.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:54 smithi131 ceph-mon[103713]: pgmap v511: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 253 op/s 2023-09-10T00:56:55.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:54 smithi044 ceph-mon[98676]: pgmap v511: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 253 op/s 2023-09-10T00:56:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:54 smithi099 ceph-mon[102692]: pgmap v511: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 253 op/s 2023-09-10T00:56:57.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:56 smithi131 ceph-mon[103713]: pgmap v512: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 370 op/s 2023-09-10T00:56:57.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:56 smithi044 ceph-mon[98676]: pgmap v512: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 370 op/s 2023-09-10T00:56:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:56 smithi099 ceph-mon[102692]: pgmap v512: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 370 op/s 2023-09-10T00:56:59.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:56:58 smithi131 ceph-mon[103713]: pgmap v513: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 255 op/s 2023-09-10T00:56:59.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:56:58 smithi044 ceph-mon[98676]: pgmap v513: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 255 op/s 2023-09-10T00:56:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:56:58 smithi099 ceph-mon[102692]: pgmap v513: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 255 op/s 2023-09-10T00:57:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:00 smithi131 ceph-mon[103713]: pgmap v514: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 255 op/s 2023-09-10T00:57:01.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:00 smithi044 ceph-mon[98676]: pgmap v514: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 255 op/s 2023-09-10T00:57:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:00 smithi099 ceph-mon[102692]: pgmap v514: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 255 op/s 2023-09-10T00:57:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.877+0000 7f01decc9700 1 -- 172.21.15.44:0/2348617969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01d80683b0 msgr2=0x7f01d810d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.877+0000 7f01decc9700 1 --2- 172.21.15.44:0/2348617969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01d80683b0 0x7f01d810d3d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f01d000b0d0 tx=0x7f01d000b3e0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.877+0000 7f01decc9700 1 -- 172.21.15.44:0/2348617969 shutdown_connections 2023-09-10T00:57:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.877+0000 7f01decc9700 1 --2- 172.21.15.44:0/2348617969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01d80683b0 0x7f01d810d3d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.877+0000 7f01decc9700 1 --2- 172.21.15.44:0/2348617969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d8106650 0x7f01d8106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.877+0000 7f01decc9700 1 --2- 172.21.15.44:0/2348617969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01d80679f0 0x7f01d8067e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.877+0000 7f01decc9700 1 -- 172.21.15.44:0/2348617969 >> 172.21.15.44:0/2348617969 conn(0x7f01d8074f30 msgr2=0x7f01d8075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.878+0000 7f01decc9700 1 -- 172.21.15.44:0/2348617969 shutdown_connections 2023-09-10T00:57:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.878+0000 7f01decc9700 1 -- 172.21.15.44:0/2348617969 wait complete. 2023-09-10T00:57:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.878+0000 7f01decc9700 1 Processor -- start 2023-09-10T00:57:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.878+0000 7f01decc9700 1 -- start start 2023-09-10T00:57:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01decc9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d80679f0 0x7f01d819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01decc9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01d80683b0 0x7f01d819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01decc9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01d8106650 0x7f01d8196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01decc9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8196b90 con 0x7f01d80679f0 2023-09-10T00:57:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01decc9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8196d00 con 0x7f01d8106650 2023-09-10T00:57:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01decc9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8196ec0 con 0x7f01d80683b0 2023-09-10T00:57:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01dca65700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d80679f0 0x7f01d819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01dca65700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d80679f0 0x7f01d819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54022/0 (socket says 172.21.15.44:54022) 2023-09-10T00:57:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01dca65700 1 -- 172.21.15.44:0/3534022845 learned_addr learned my addr 172.21.15.44:0/3534022845 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01dd266700 1 --2- 172.21.15.44:0/3534022845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01d8106650 0x7f01d8196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01d7fff700 1 --2- 172.21.15.44:0/3534022845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01d80683b0 0x7f01d819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01dd266700 1 -- 172.21.15.44:0/3534022845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01d80683b0 msgr2=0x7f01d819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01dd266700 1 --2- 172.21.15.44:0/3534022845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01d80683b0 0x7f01d819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01dd266700 1 -- 172.21.15.44:0/3534022845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d80679f0 msgr2=0x7f01d819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01dd266700 1 --2- 172.21.15.44:0/3534022845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d80679f0 0x7f01d819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01dd266700 1 -- 172.21.15.44:0/3534022845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d0009d00 con 0x7f01d8106650 2023-09-10T00:57:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.879+0000 7f01dd266700 1 --2- 172.21.15.44:0/3534022845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01d8106650 0x7f01d8196520 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f01d000b770 tx=0x7f01d000cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.880+0000 7f01d5ffb700 1 -- 172.21.15.44:0/3534022845 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d000e040 con 0x7f01d8106650 2023-09-10T00:57:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.880+0000 7f01d5ffb700 1 -- 172.21.15.44:0/3534022845 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01d0004c50 con 0x7f01d8106650 2023-09-10T00:57:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.880+0000 7f01decc9700 1 -- 172.21.15.44:0/3534022845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d8197140 con 0x7f01d8106650 2023-09-10T00:57:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.880+0000 7f01d5ffb700 1 -- 172.21.15.44:0/3534022845 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d001c400 con 0x7f01d8106650 2023-09-10T00:57:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.880+0000 7f01decc9700 1 -- 172.21.15.44:0/3534022845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d80ff9c0 con 0x7f01d8106650 2023-09-10T00:57:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.882+0000 7f01d5ffb700 1 -- 172.21.15.44:0/3534022845 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f01d0005040 con 0x7f01d8106650 2023-09-10T00:57:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.883+0000 7f01d5ffb700 1 --2- 172.21.15.44:0/3534022845 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f01c80a0870 0x7f01c80a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.883+0000 7f01d5ffb700 1 -- 172.21.15.44:0/3534022845 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f01d00cd040 con 0x7f01d8106650 2023-09-10T00:57:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.883+0000 7f01dca65700 1 --2- 172.21.15.44:0/3534022845 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f01c80a0870 0x7f01c80a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.884+0000 7f01dca65700 1 --2- 172.21.15.44:0/3534022845 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f01c80a0870 0x7f01c80a2d30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009ad0 tx=0x7f01cc009290 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.884+0000 7f01eae8fb80 1 Processor -- start 2023-09-10T00:57:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.884+0000 7f01eae8fb80 1 -- start start 2023-09-10T00:57:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.884+0000 7f01eae8fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff9031a0 0x55d3ff903580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.884+0000 7f01eae8fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff8fd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.884+0000 7f01eae8fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff909f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.884+0000 7f01eae8fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ff8566d0 con 0x55d3ff9031a0 2023-09-10T00:57:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.884+0000 7f01eae8fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ff853ff0 con 0x55d3ff8fdcc0 2023-09-10T00:57:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.884+0000 7f01eae8fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ff84f8e0 con 0x55d3ff8fd300 2023-09-10T00:57:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.884+0000 7f01dca65700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff9031a0 0x55d3ff903580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.884+0000 7f01d7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff8fd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.884+0000 7f01d7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff8fd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60914/0 (socket says 172.21.15.44:60914) 2023-09-10T00:57:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.884+0000 7f01d7fff700 1 -- 172.21.15.44:0/67602835 learned_addr learned my addr 172.21.15.44:0/67602835 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.884+0000 7f01dd266700 1 --2- 172.21.15.44:0/67602835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff909f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.885+0000 7f01d7fff700 1 -- 172.21.15.44:0/67602835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fdcc0 msgr2=0x55d3ff909f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.885+0000 7f01d7fff700 1 --2- 172.21.15.44:0/67602835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff909f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.885+0000 7f01d7fff700 1 -- 172.21.15.44:0/67602835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff9031a0 msgr2=0x55d3ff903580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.885+0000 7f01d7fff700 1 --2- 172.21.15.44:0/67602835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff9031a0 0x55d3ff903580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.885+0000 7f01d7fff700 1 -- 172.21.15.44:0/67602835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3ff67bd80 con 0x55d3ff8fd300 2023-09-10T00:57:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.885+0000 7f01dca65700 1 --2- 172.21.15.44:0/67602835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff9031a0 0x55d3ff903580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:57:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.885+0000 7f01d7fff700 1 --2- 172.21.15.44:0/67602835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff8fd780 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009d20 tx=0x7f01c4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48c2ba38ad98e14 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.885+0000 7f01c2ffd700 1 -- 172.21.15.44:0/67602835 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4019630 con 0x55d3ff8fd300 2023-09-10T00:57:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.885+0000 7f01c2ffd700 1 -- 172.21.15.44:0/67602835 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01c4019c20 con 0x55d3ff8fd300 2023-09-10T00:57:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.885+0000 7f01c2ffd700 1 -- 172.21.15.44:0/67602835 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c400eb00 con 0x55d3ff8fd300 2023-09-10T00:57:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.886+0000 7f01eae8fb80 1 -- 172.21.15.44:0/67602835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 msgr2=0x55d3ff8fd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.886+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/67602835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff8fd780 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009d20 tx=0x7f01c4011720 comp rx=0 tx=0).stop 2023-09-10T00:57:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.886+0000 7f01eae8fb80 1 -- 172.21.15.44:0/67602835 shutdown_connections 2023-09-10T00:57:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.886+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/67602835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff909f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.886+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/67602835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff9031a0 0x55d3ff903580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.886+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/67602835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff8fd780 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.886+0000 7f01eae8fb80 1 -- 172.21.15.44:0/67602835 >> 172.21.15.44:0/67602835 conn(0x55d3ff872030 msgr2=0x55d3ff8f8560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.887+0000 7f01eae8fb80 1 -- 172.21.15.44:0/67602835 shutdown_connections 2023-09-10T00:57:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.887+0000 7f01eae8fb80 1 -- 172.21.15.44:0/67602835 wait complete. 2023-09-10T00:57:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.887+0000 7f01eae8fb80 1 -- start start 2023-09-10T00:57:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.887+0000 7f01eae8fb80 1 Processor -- start 2023-09-10T00:57:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01eae8fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff909e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01eae8fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff907140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01eae8fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff9031a0 0x55d3ff907770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01eae8fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ff7edb10 con 0x55d3ff9031a0 2023-09-10T00:57:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01eae8fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ff7ed810 con 0x55d3ff8fdcc0 2023-09-10T00:57:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01eae8fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ff6d18c0 con 0x55d3ff8fd300 2023-09-10T00:57:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01eae8fb80 10 client.0 _ll_register_callbacks cb 0x55d3ff737a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:57:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01d7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff907140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01dca65700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff909e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01dd266700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff9031a0 0x55d3ff907770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01d7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff907140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55778/0 (socket says 172.21.15.44:55778) 2023-09-10T00:57:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01dca65700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff909e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60918/0 (socket says 172.21.15.44:60918) 2023-09-10T00:57:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01d7fff700 1 -- 172.21.15.44:0/736582941 learned_addr learned my addr 172.21.15.44:0/736582941 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01d7fff700 1 -- 172.21.15.44:0/736582941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 msgr2=0x55d3ff909e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01d7fff700 1 --2- 172.21.15.44:0/736582941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff909e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01d7fff700 1 -- 172.21.15.44:0/736582941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff9031a0 msgr2=0x55d3ff907770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01d7fff700 1 --2- 172.21.15.44:0/736582941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff9031a0 0x55d3ff907770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01d7fff700 1 -- 172.21.15.44:0/736582941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c4009980 con 0x55d3ff8fdcc0 2023-09-10T00:57:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.888+0000 7f01dca65700 1 --2- 172.21.15.44:0/736582941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff909e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:57:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.889+0000 7f01d7fff700 1 --2- 172.21.15.44:0/736582941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff907140 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f01c401c040 tx=0x7f01c4019a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e182a7fc8bbe7df1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.889+0000 7f01b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:57:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.889+0000 7f01b7fff700 1 -- 172.21.15.44:0/736582941 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4005450 con 0x55d3ff8fdcc0 2023-09-10T00:57:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.889+0000 7f01b7fff700 1 -- 172.21.15.44:0/736582941 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f01c4005ce0 con 0x55d3ff8fdcc0 2023-09-10T00:57:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.889+0000 7f01eae8fb80 1 -- 172.21.15.44:0/736582941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3ff7a7490 con 0x55d3ff8fdcc0 2023-09-10T00:57:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.889+0000 7f01b7fff700 1 -- 172.21.15.44:0/736582941 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01c40055b0 con 0x55d3ff8fdcc0 2023-09-10T00:57:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.889+0000 7f01eae8fb80 10 client.27680 fetch_fsmap learned FSMap version 76 2023-09-10T00:57:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.889+0000 7f01eae8fb80 1 -- 172.21.15.44:0/736582941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55d3ff664c00 con 0x55d3ff8fdcc0 2023-09-10T00:57:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.889+0000 7f01b7fff700 1 -- 172.21.15.44:0/736582941 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d3ff7a7490 con 0x55d3ff8fdcc0 2023-09-10T00:57:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.889+0000 7f01b7fff700 1 -- 172.21.15.44:0/736582941 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 76) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f01c4004bd0 con 0x55d3ff8fdcc0 2023-09-10T00:57:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.889+0000 7f01eae8fb80 10 client.27680 fetch_fsmap finished waiting for FSMap version 76 2023-09-10T00:57:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.889+0000 7f01eae8fb80 10 client.27680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:57:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.889+0000 7f01eae8fb80 20 client.27680 populate_metadata read hostname 'smithi044' 2023-09-10T00:57:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.889+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/736582941 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d3ff91fcb0 0x55d3ff9220d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.889+0000 7f01eae8fb80 4 client.27680 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:57:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.889+0000 7f01eae8fb80 1 -- 172.21.15.44:0/736582941 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3ff864840 con 0x55d3ff91fcb0 2023-09-10T00:57:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.890+0000 7f01dca65700 1 --2- 172.21.15.44:0/736582941 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d3ff91fcb0 0x55d3ff9220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.890+0000 7f01dca65700 1 --2- 172.21.15.44:0/736582941 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d3ff91fcb0 0x55d3ff9220d0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=607decb0d48667bd server_cookie=ffc749646a2d7d0d in_seq=0 out_seq=0 2023-09-10T00:57:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.890+0000 7f01b7fff700 10 client.27680 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.891+0000 7f01b7fff700 1 -- 172.21.15.44:0/736582941 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d3ff864840 con 0x55d3ff91fcb0 2023-09-10T00:57:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.891+0000 7f01b7fff700 10 client.27680 handle_command_reply: tid=0 2023-09-10T00:57:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.891+0000 7f01eae8fb80 1 client.27680 shutdown 2023-09-10T00:57:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.891+0000 7f01eae8fb80 1 -- 172.21.15.44:0/736582941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fdcc0 msgr2=0x55d3ff907140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.891+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/736582941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff907140 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f01c401c040 tx=0x7f01c4019a20 comp rx=0 tx=0).stop 2023-09-10T00:57:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.891+0000 7f01eae8fb80 1 -- 172.21.15.44:0/736582941 shutdown_connections 2023-09-10T00:57:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.891+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/736582941 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d3ff91fcb0 0x55d3ff9220d0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.891+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/736582941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff9031a0 0x55d3ff907770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.891+0000 7f01b7fff700 0 client.27680 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.891+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/736582941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff909e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.891+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/736582941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff907140 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.891+0000 7f01eae8fb80 1 -- 172.21.15.44:0/736582941 >> 172.21.15.44:0/736582941 conn(0x55d3ff872030 msgr2=0x55d3ff8f77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.891+0000 7f01eae8fb80 1 -- 172.21.15.44:0/736582941 shutdown_connections 2023-09-10T00:57:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.891+0000 7f01eae8fb80 1 -- 172.21.15.44:0/736582941 wait complete. 2023-09-10T00:57:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.891+0000 7f01eae8fb80 20 client.27680 trim_cache size 0 max 16384 2023-09-10T00:57:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.913+0000 7f01eae8fb80 1 Processor -- start 2023-09-10T00:57:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.913+0000 7f01eae8fb80 1 -- start start 2023-09-10T00:57:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.913+0000 7f01eae8fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff904e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.913+0000 7f01eae8fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff90e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.913+0000 7f01eae8fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff9031a0 0x55d3ff8ffb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.913+0000 7f01eae8fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ff8f5010 con 0x55d3ff8fdcc0 2023-09-10T00:57:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.913+0000 7f01eae8fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ff902e00 con 0x55d3ff9031a0 2023-09-10T00:57:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.913+0000 7f01eae8fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ff908750 con 0x55d3ff8fd300 2023-09-10T00:57:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.914+0000 7f01dca65700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff904e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.914+0000 7f01dd266700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff9031a0 0x55d3ff8ffb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.914+0000 7f01dca65700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff904e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:60926/0 (socket says 172.21.15.44:60926) 2023-09-10T00:57:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.914+0000 7f01dca65700 1 -- 172.21.15.44:0/3607007041 learned_addr learned my addr 172.21.15.44:0/3607007041 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.914+0000 7f01dd266700 1 -- 172.21.15.44:0/3607007041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 msgr2=0x55d3ff904e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.914+0000 7f01dd266700 1 --2- 172.21.15.44:0/3607007041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff904e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.914+0000 7f01dd266700 1 -- 172.21.15.44:0/3607007041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff8fdcc0 msgr2=0x55d3ff90e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.914+0000 7f01dd266700 1 --2- 172.21.15.44:0/3607007041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff90e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.914+0000 7f01dd266700 1 -- 172.21.15.44:0/3607007041 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3ff6a21f0 con 0x55d3ff9031a0 2023-09-10T00:57:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.914+0000 7f01dd266700 1 --2- 172.21.15.44:0/3607007041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff9031a0 0x55d3ff8ffb60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f01d00ca360 tx=0x7f01d0009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92239d02f7a64d50 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.914+0000 7f01c37fe700 1 -- 172.21.15.44:0/3607007041 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00a00b0 con 0x55d3ff9031a0 2023-09-10T00:57:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.914+0000 7f01c37fe700 1 -- 172.21.15.44:0/3607007041 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01d009a880 con 0x55d3ff9031a0 2023-09-10T00:57:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.914+0000 7f01c37fe700 1 -- 172.21.15.44:0/3607007041 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0023310 con 0x55d3ff9031a0 2023-09-10T00:57:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.915+0000 7f01eae8fb80 1 -- 172.21.15.44:0/3607007041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff9031a0 msgr2=0x55d3ff8ffb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.915+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/3607007041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff9031a0 0x55d3ff8ffb60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f01d00ca360 tx=0x7f01d0009040 comp rx=0 tx=0).stop 2023-09-10T00:57:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.915+0000 7f01eae8fb80 1 -- 172.21.15.44:0/3607007041 shutdown_connections 2023-09-10T00:57:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.915+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/3607007041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff9031a0 0x55d3ff8ffb60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.915+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/3607007041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff8fd300 0x55d3ff904e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.915+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/3607007041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff90e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.915+0000 7f01eae8fb80 1 -- 172.21.15.44:0/3607007041 >> 172.21.15.44:0/3607007041 conn(0x55d3ff872030 msgr2=0x55d3ff9021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.915+0000 7f01eae8fb80 1 -- 172.21.15.44:0/3607007041 shutdown_connections 2023-09-10T00:57:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.915+0000 7f01eae8fb80 1 -- 172.21.15.44:0/3607007041 wait complete. 2023-09-10T00:57:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.916+0000 7f01eae8fb80 1 -- start start 2023-09-10T00:57:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.916+0000 7f01eae8fb80 1 Processor -- start 2023-09-10T00:57:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01eae8fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fd300 0x55d3ff905970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01eae8fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff905eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01eae8fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff9031a0 0x55d3ff9063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01eae8fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ff906ad0 con 0x55d3ff8fdcc0 2023-09-10T00:57:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01eae8fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ff906c40 con 0x55d3ff8fd300 2023-09-10T00:57:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01eae8fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ff906e00 con 0x55d3ff9031a0 2023-09-10T00:57:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01eae8fb80 10 client.0 _ll_register_callbacks cb 0x7f01d0007930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:57:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01dca65700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fd300 0x55d3ff905970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01dca65700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fd300 0x55d3ff905970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55790/0 (socket says 172.21.15.44:55790) 2023-09-10T00:57:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01dd266700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff9031a0 0x55d3ff9063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01dca65700 1 -- 172.21.15.44:0/639252409 learned_addr learned my addr 172.21.15.44:0/639252409 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01d7fff700 1 --2- 172.21.15.44:0/639252409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff905eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01dd266700 1 -- 172.21.15.44:0/639252409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fd300 msgr2=0x55d3ff905970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01dd266700 1 --2- 172.21.15.44:0/639252409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fd300 0x55d3ff905970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01dd266700 1 -- 172.21.15.44:0/639252409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff8fdcc0 msgr2=0x55d3ff905eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01dd266700 1 --2- 172.21.15.44:0/639252409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff905eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01dd266700 1 -- 172.21.15.44:0/639252409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d80ff9c0 con 0x55d3ff9031a0 2023-09-10T00:57:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01dd266700 1 --2- 172.21.15.44:0/639252409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff9031a0 0x55d3ff9063f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f01d000b0d0 tx=0x7f01d000b030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b0d3e18d89e91f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.917+0000 7f01b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:57:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.918+0000 7f01b7fff700 1 -- 172.21.15.44:0/639252409 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0023520 con 0x55d3ff9031a0 2023-09-10T00:57:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.918+0000 7f01b7fff700 1 -- 172.21.15.44:0/639252409 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f01d009c070 con 0x55d3ff9031a0 2023-09-10T00:57:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.918+0000 7f01eae8fb80 1 -- 172.21.15.44:0/639252409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01c4019c20 con 0x55d3ff9031a0 2023-09-10T00:57:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.918+0000 7f01b7fff700 1 -- 172.21.15.44:0/639252409 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01d009dfb0 con 0x55d3ff9031a0 2023-09-10T00:57:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.918+0000 7f01eae8fb80 10 client.27622 fetch_fsmap learned FSMap version 76 2023-09-10T00:57:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.918+0000 7f01eae8fb80 1 -- 172.21.15.44:0/639252409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55d3ff67a250 con 0x55d3ff9031a0 2023-09-10T00:57:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.918+0000 7f01b7fff700 1 -- 172.21.15.44:0/639252409 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01c4019c20 con 0x55d3ff9031a0 2023-09-10T00:57:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.918+0000 7f01b7fff700 1 -- 172.21.15.44:0/639252409 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 76) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f01d0099670 con 0x55d3ff9031a0 2023-09-10T00:57:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.918+0000 7f01eae8fb80 10 client.27622 fetch_fsmap finished waiting for FSMap version 76 2023-09-10T00:57:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.918+0000 7f01eae8fb80 10 client.27622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:57:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.918+0000 7f01eae8fb80 20 client.27622 populate_metadata read hostname 'smithi044' 2023-09-10T00:57:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.918+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/639252409 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d3ff96bd10 0x55d3ff96e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.918+0000 7f01eae8fb80 4 client.27622 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:57:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.918+0000 7f01eae8fb80 1 -- 172.21.15.44:0/639252409 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d3ff8488c0 con 0x55d3ff96bd10 2023-09-10T00:57:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.919+0000 7f01dca65700 1 --2- 172.21.15.44:0/639252409 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d3ff96bd10 0x55d3ff96e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.919+0000 7f01b7fff700 10 client.27622 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:02.919+0000 7f01dca65700 1 --2- 172.21.15.44:0/639252409 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d3ff96bd10 0x55d3ff96e130 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceb20f8add7003b0 server_cookie=714af238199e68a4 in_seq=0 out_seq=0 2023-09-10T00:57:03.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:02 smithi131 ceph-mon[103713]: pgmap v515: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 382 op/s 2023-09-10T00:57:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:02 smithi044 ceph-mon[98676]: pgmap v515: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 382 op/s 2023-09-10T00:57:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:02 smithi099 ceph-mon[102692]: pgmap v515: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 382 op/s 2023-09-10T00:57:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.414+0000 7f01b7fff700 1 -- 172.21.15.44:0/639252409 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d3ff8488c0 con 0x55d3ff96bd10 2023-09-10T00:57:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.414+0000 7f01b7fff700 10 client.27622 handle_command_reply: tid=0 2023-09-10T00:57:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.414+0000 7f01eae8fb80 1 client.27622 shutdown 2023-09-10T00:57:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.414+0000 7f01eae8fb80 1 -- 172.21.15.44:0/639252409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff9031a0 msgr2=0x55d3ff9063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.414+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/639252409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff9031a0 0x55d3ff9063f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f01d000b0d0 tx=0x7f01d000b030 comp rx=0 tx=0).stop 2023-09-10T00:57:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.414+0000 7f01eae8fb80 1 -- 172.21.15.44:0/639252409 shutdown_connections 2023-09-10T00:57:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.414+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/639252409 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d3ff96bd10 0x55d3ff96e130 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.414+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/639252409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3ff9031a0 0x55d3ff9063f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.414+0000 7f01b7fff700 0 client.27622 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.414+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/639252409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3ff8fd300 0x55d3ff905970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.414+0000 7f01eae8fb80 1 --2- 172.21.15.44:0/639252409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d3ff8fdcc0 0x55d3ff905eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.414+0000 7f01eae8fb80 1 -- 172.21.15.44:0/639252409 >> 172.21.15.44:0/639252409 conn(0x55d3ff872030 msgr2=0x55d3ff9021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.415+0000 7f01eae8fb80 1 -- 172.21.15.44:0/639252409 shutdown_connections 2023-09-10T00:57:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.415+0000 7f01eae8fb80 1 -- 172.21.15.44:0/639252409 wait complete. 2023-09-10T00:57:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.415+0000 7f01eae8fb80 20 client.27622 trim_cache size 0 max 16384 2023-09-10T00:57:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:57:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:57:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:57:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:57:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.415+0000 7f01decc9700 1 -- 172.21.15.44:0/3534022845 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f01c80a0870 msgr2=0x7f01c80a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.415+0000 7f01decc9700 1 --2- 172.21.15.44:0/3534022845 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f01c80a0870 0x7f01c80a2d30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009ad0 tx=0x7f01cc009290 comp rx=0 tx=0).stop 2023-09-10T00:57:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.415+0000 7f01decc9700 1 -- 172.21.15.44:0/3534022845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01d8106650 msgr2=0x7f01d8196520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.415+0000 7f01decc9700 1 --2- 172.21.15.44:0/3534022845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01d8106650 0x7f01d8196520 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f01d000b770 tx=0x7f01d000cce0 comp rx=0 tx=0).stop 2023-09-10T00:57:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.415+0000 7f01decc9700 1 -- 172.21.15.44:0/3534022845 shutdown_connections 2023-09-10T00:57:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.415+0000 7f01decc9700 1 --2- 172.21.15.44:0/3534022845 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f01c80a0870 0x7f01c80a2d30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.415+0000 7f01decc9700 1 --2- 172.21.15.44:0/3534022845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01d8106650 0x7f01d8196520 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.415+0000 7f01decc9700 1 --2- 172.21.15.44:0/3534022845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d80679f0 0x7f01d819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.415+0000 7f01decc9700 1 --2- 172.21.15.44:0/3534022845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01d80683b0 0x7f01d819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.416+0000 7f01decc9700 1 -- 172.21.15.44:0/3534022845 >> 172.21.15.44:0/3534022845 conn(0x7f01d8074f30 msgr2=0x7f01d80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.416+0000 7f01decc9700 1 -- 172.21.15.44:0/3534022845 shutdown_connections 2023-09-10T00:57:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:03.416+0000 7f01decc9700 1 -- 172.21.15.44:0/3534022845 wait complete. 2023-09-10T00:57:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:04 smithi131 ceph-mon[103713]: pgmap v516: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 243 op/s 2023-09-10T00:57:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:04 smithi131 ceph-mon[103713]: osdmap e114: 12 total, 12 up, 12 in 2023-09-10T00:57:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:04 smithi044 ceph-mon[98676]: pgmap v516: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 243 op/s 2023-09-10T00:57:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:04 smithi044 ceph-mon[98676]: osdmap e114: 12 total, 12 up, 12 in 2023-09-10T00:57:05.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:04 smithi099 ceph-mon[102692]: pgmap v516: 129 pgs: 129 active+clean; 9.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 243 op/s 2023-09-10T00:57:05.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:04 smithi099 ceph-mon[102692]: osdmap e114: 12 total, 12 up, 12 in 2023-09-10T00:57:06.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:05 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:06.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:05 smithi131 ceph-mon[103713]: scrub summary: active paths [/] 2023-09-10T00:57:06.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:05 smithi131 ceph-mon[103713]: scrub summary: idle+waiting paths [/] 2023-09-10T00:57:06.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:05 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:06.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:05 smithi044 ceph-mon[98676]: scrub summary: active paths [/] 2023-09-10T00:57:06.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:05 smithi044 ceph-mon[98676]: scrub summary: idle+waiting paths [/] 2023-09-10T00:57:06.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:05 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:06.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:05 smithi099 ceph-mon[102692]: scrub summary: active paths [/] 2023-09-10T00:57:06.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:05 smithi099 ceph-mon[102692]: scrub summary: idle+waiting paths [/] 2023-09-10T00:57:07.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:06 smithi131 ceph-mon[103713]: pgmap v518: 129 pgs: 129 active+clean; 8.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 341 op/s 2023-09-10T00:57:07.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:06 smithi044 ceph-mon[98676]: pgmap v518: 129 pgs: 129 active+clean; 8.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 341 op/s 2023-09-10T00:57:07.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:06 smithi099 ceph-mon[102692]: pgmap v518: 129 pgs: 129 active+clean; 8.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 341 op/s 2023-09-10T00:57:09.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:08 smithi131 ceph-mon[103713]: pgmap v519: 129 pgs: 129 active+clean; 8.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 341 op/s 2023-09-10T00:57:09.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:08 smithi044 ceph-mon[98676]: pgmap v519: 129 pgs: 129 active+clean; 8.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 341 op/s 2023-09-10T00:57:09.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:08 smithi099 ceph-mon[102692]: pgmap v519: 129 pgs: 129 active+clean; 8.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 341 op/s 2023-09-10T00:57:10.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:09 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:57:10.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:09 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:57:10.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:09 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:57:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:10 smithi131 ceph-mon[103713]: pgmap v520: 129 pgs: 129 active+clean; 8.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 341 op/s 2023-09-10T00:57:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:10 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:10 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:57:11.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:10 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:57:11.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:10 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:57:11.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:10 smithi131 ceph-mon[103713]: osdmap e115: 12 total, 12 up, 12 in 2023-09-10T00:57:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:10 smithi044 ceph-mon[98676]: pgmap v520: 129 pgs: 129 active+clean; 8.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 341 op/s 2023-09-10T00:57:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:10 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:10 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:57:11.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:10 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:57:11.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:10 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:57:11.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:10 smithi044 ceph-mon[98676]: osdmap e115: 12 total, 12 up, 12 in 2023-09-10T00:57:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:10 smithi099 ceph-mon[102692]: pgmap v520: 129 pgs: 129 active+clean; 8.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 341 op/s 2023-09-10T00:57:11.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:10 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:11.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:10 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:57:11.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:10 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:57:11.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:10 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:57:11.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:10 smithi099 ceph-mon[102692]: osdmap e115: 12 total, 12 up, 12 in 2023-09-10T00:57:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:12 smithi131 ceph-mon[103713]: pgmap v522: 129 pgs: 129 active+clean; 8.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 440 op/s 2023-09-10T00:57:13.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:12 smithi044 ceph-mon[98676]: pgmap v522: 129 pgs: 129 active+clean; 8.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 440 op/s 2023-09-10T00:57:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:12 smithi099 ceph-mon[102692]: pgmap v522: 129 pgs: 129 active+clean; 8.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 440 op/s 2023-09-10T00:57:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:14 smithi131 ceph-mon[103713]: pgmap v523: 129 pgs: 129 active+clean; 8.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 369 op/s 2023-09-10T00:57:15.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:14 smithi044 ceph-mon[98676]: pgmap v523: 129 pgs: 129 active+clean; 8.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 369 op/s 2023-09-10T00:57:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:14 smithi099 ceph-mon[102692]: pgmap v523: 129 pgs: 129 active+clean; 8.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 369 op/s 2023-09-10T00:57:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.662+0000 7fabba4e4700 1 -- 172.21.15.44:0/4172138216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb41055b0 msgr2=0x7fabb4105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.662+0000 7fabba4e4700 1 --2- 172.21.15.44:0/4172138216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb41055b0 0x7fabb4105990 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7faba4009920 tx=0x7faba4009c30 comp rx=0 tx=0).stop 2023-09-10T00:57:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.662+0000 7fabba4e4700 1 -- 172.21.15.44:0/4172138216 shutdown_connections 2023-09-10T00:57:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.662+0000 7fabba4e4700 1 --2- 172.21.15.44:0/4172138216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabb4100120 0x7fabb410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.662+0000 7fabba4e4700 1 --2- 172.21.15.44:0/4172138216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb41055b0 0x7fabb4105990 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.662+0000 7fabba4e4700 1 --2- 172.21.15.44:0/4172138216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabb40ff760 0x7fabb40ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.662+0000 7fabba4e4700 1 -- 172.21.15.44:0/4172138216 >> 172.21.15.44:0/4172138216 conn(0x7fabb4074f30 msgr2=0x7fabb4075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.662+0000 7fabba4e4700 1 -- 172.21.15.44:0/4172138216 shutdown_connections 2023-09-10T00:57:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.662+0000 7fabba4e4700 1 -- 172.21.15.44:0/4172138216 wait complete. 2023-09-10T00:57:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.663+0000 7fabba4e4700 1 Processor -- start 2023-09-10T00:57:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.663+0000 7fabba4e4700 1 -- start start 2023-09-10T00:57:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.663+0000 7fabba4e4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabb40ff760 0x7fabb4197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.663+0000 7fabba4e4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb4100120 0x7fabb4197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.663+0000 7fabba4e4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabb41055b0 0x7fabb41a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.663+0000 7fabba4e4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb41a2af0 con 0x7fabb4100120 2023-09-10T00:57:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.663+0000 7fabba4e4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb41a2c60 con 0x7fabb40ff760 2023-09-10T00:57:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.663+0000 7fabba4e4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb41a2e20 con 0x7fabb41055b0 2023-09-10T00:57:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabb37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb4100120 0x7fabb4197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabb3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabb40ff760 0x7fabb4197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabb37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb4100120 0x7fabb4197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54066/0 (socket says 172.21.15.44:54066) 2023-09-10T00:57:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabb3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabb40ff760 0x7fabb4197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55804/0 (socket says 172.21.15.44:55804) 2023-09-10T00:57:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabb37fe700 1 -- 172.21.15.44:0/3392200697 learned_addr learned my addr 172.21.15.44:0/3392200697 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabb8a81700 1 --2- 172.21.15.44:0/3392200697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabb41055b0 0x7fabb41a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabb37fe700 1 -- 172.21.15.44:0/3392200697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabb41055b0 msgr2=0x7fabb41a2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabb37fe700 1 --2- 172.21.15.44:0/3392200697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabb41055b0 0x7fabb41a2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabb37fe700 1 -- 172.21.15.44:0/3392200697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabb40ff760 msgr2=0x7fabb4197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabb37fe700 1 --2- 172.21.15.44:0/3392200697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabb40ff760 0x7fabb4197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabb37fe700 1 -- 172.21.15.44:0/3392200697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba80095f0 con 0x7fabb4100120 2023-09-10T00:57:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabb37fe700 1 --2- 172.21.15.44:0/3392200697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb4100120 0x7fabb4197bc0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7faba800ec10 tx=0x7faba800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabb17fa700 1 -- 172.21.15.44:0/3392200697 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba800cc90 con 0x7fabb4100120 2023-09-10T00:57:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabb17fa700 1 -- 172.21.15.44:0/3392200697 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faba800cdf0 con 0x7fabb4100120 2023-09-10T00:57:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabba4e4700 1 -- 172.21.15.44:0/3392200697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba4009520 con 0x7fabb4100120 2023-09-10T00:57:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabb17fa700 1 -- 172.21.15.44:0/3392200697 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8003e80 con 0x7fabb4100120 2023-09-10T00:57:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.664+0000 7fabba4e4700 1 -- 172.21.15.44:0/3392200697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb41a3480 con 0x7fabb4100120 2023-09-10T00:57:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.666+0000 7fabb17fa700 1 -- 172.21.15.44:0/3392200697 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7faba8004000 con 0x7fabb4100120 2023-09-10T00:57:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.667+0000 7fabb17fa700 1 --2- 172.21.15.44:0/3392200697 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fab9c0a07b0 0x7fab9c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.667+0000 7fabb17fa700 1 -- 172.21.15.44:0/3392200697 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9540+0+0 (secure 0 0 0) 0x7faba8014070 con 0x7fabb4100120 2023-09-10T00:57:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.667+0000 7fabb3fff700 1 --2- 172.21.15.44:0/3392200697 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fab9c0a07b0 0x7fab9c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.667+0000 7fabb3fff700 1 --2- 172.21.15.44:0/3392200697 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fab9c0a07b0 0x7fab9c0a2c70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faba4009810 tx=0x7faba4019040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.668+0000 7fabc66aab80 1 Processor -- start 2023-09-10T00:57:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.668+0000 7fabc66aab80 1 -- start start 2023-09-10T00:57:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.668+0000 7fabc66aab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630019991a0 0x563001999580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.668+0000 7fabc66aab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993300 0x563001993780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.668+0000 7fabc66aab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563001993cc0 0x56300199ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.668+0000 7fabc66aab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630018ec6d0 con 0x563001993cc0 2023-09-10T00:57:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.668+0000 7fabc66aab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630018e9ff0 con 0x5630019991a0 2023-09-10T00:57:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.668+0000 7fabc66aab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630018e58e0 con 0x563001993300 2023-09-10T00:57:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.668+0000 7fabb3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630019991a0 0x563001999580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.668+0000 7fabb8a81700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563001993cc0 0x56300199ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.668+0000 7fabb37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993300 0x563001993780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.668+0000 7fabb3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630019991a0 0x563001999580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55812/0 (socket says 172.21.15.44:55812) 2023-09-10T00:57:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.668+0000 7fabb3fff700 1 -- 172.21.15.44:0/2430706057 learned_addr learned my addr 172.21.15.44:0/2430706057 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.668+0000 7fabb8a81700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563001993cc0 0x56300199ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54076/0 (socket says 172.21.15.44:54076) 2023-09-10T00:57:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.669+0000 7fabb8a81700 1 -- 172.21.15.44:0/2430706057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993300 msgr2=0x563001993780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.669+0000 7fabb8a81700 1 --2- 172.21.15.44:0/2430706057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993300 0x563001993780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.669+0000 7fabb8a81700 1 -- 172.21.15.44:0/2430706057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630019991a0 msgr2=0x563001999580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.669+0000 7fabb8a81700 1 --2- 172.21.15.44:0/2430706057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630019991a0 0x563001999580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.669+0000 7fabb8a81700 1 -- 172.21.15.44:0/2430706057 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563001711d80 con 0x563001993cc0 2023-09-10T00:57:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.669+0000 7fabb3fff700 1 --2- 172.21.15.44:0/2430706057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630019991a0 0x563001999580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:57:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.669+0000 7fabb8a81700 1 --2- 172.21.15.44:0/2430706057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563001993cc0 0x56300199ff30 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fabac011580 tx=0x7fabac009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d42d6bf094beba server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.669+0000 7fab9a7fc700 1 -- 172.21.15.44:0/2430706057 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac00e040 con 0x563001993cc0 2023-09-10T00:57:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.669+0000 7fab9a7fc700 1 -- 172.21.15.44:0/2430706057 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fabac00f040 con 0x563001993cc0 2023-09-10T00:57:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.669+0000 7fab9a7fc700 1 -- 172.21.15.44:0/2430706057 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac0042c0 con 0x563001993cc0 2023-09-10T00:57:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.670+0000 7fabc66aab80 1 -- 172.21.15.44:0/2430706057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563001993cc0 msgr2=0x56300199ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.670+0000 7fabc66aab80 1 --2- 172.21.15.44:0/2430706057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563001993cc0 0x56300199ff30 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fabac011580 tx=0x7fabac009920 comp rx=0 tx=0).stop 2023-09-10T00:57:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.670+0000 7fabc66aab80 1 -- 172.21.15.44:0/2430706057 shutdown_connections 2023-09-10T00:57:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.670+0000 7fabc66aab80 1 --2- 172.21.15.44:0/2430706057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563001993cc0 0x56300199ff30 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.670+0000 7fabc66aab80 1 --2- 172.21.15.44:0/2430706057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630019991a0 0x563001999580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.670+0000 7fabc66aab80 1 --2- 172.21.15.44:0/2430706057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993300 0x563001993780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.670+0000 7fabc66aab80 1 -- 172.21.15.44:0/2430706057 >> 172.21.15.44:0/2430706057 conn(0x563001908030 msgr2=0x56300198e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.670+0000 7fabc66aab80 1 -- 172.21.15.44:0/2430706057 shutdown_connections 2023-09-10T00:57:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.670+0000 7fabc66aab80 1 -- 172.21.15.44:0/2430706057 wait complete. 2023-09-10T00:57:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.670+0000 7fabc66aab80 1 -- start start 2023-09-10T00:57:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.670+0000 7fabc66aab80 1 Processor -- start 2023-09-10T00:57:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabc66aab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563001993300 0x56300199fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabc66aab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993cc0 0x56300199cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabc66aab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5630019991a0 0x56300199d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabc66aab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563001883b10 con 0x5630019991a0 2023-09-10T00:57:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabc66aab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563001883810 con 0x563001993300 2023-09-10T00:57:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabc66aab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630017678c0 con 0x563001993cc0 2023-09-10T00:57:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabc66aab80 10 client.0 _ll_register_callbacks cb 0x5630017cda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:57:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabb8a81700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5630019991a0 0x56300199d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabb3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563001993300 0x56300199fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabb8a81700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5630019991a0 0x56300199d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54082/0 (socket says 172.21.15.44:54082) 2023-09-10T00:57:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabb8a81700 1 -- 172.21.15.44:0/1728403458 learned_addr learned my addr 172.21.15.44:0/1728403458 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabb37fe700 1 --2- 172.21.15.44:0/1728403458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993cc0 0x56300199cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabb3fff700 1 -- 172.21.15.44:0/1728403458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993cc0 msgr2=0x56300199cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabb3fff700 1 --2- 172.21.15.44:0/1728403458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993cc0 0x56300199cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabb3fff700 1 -- 172.21.15.44:0/1728403458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5630019991a0 msgr2=0x56300199d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabb3fff700 1 --2- 172.21.15.44:0/1728403458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5630019991a0 0x56300199d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabb3fff700 1 -- 172.21.15.44:0/1728403458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb410cbc0 con 0x563001993300 2023-09-10T00:57:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabb8a81700 1 --2- 172.21.15.44:0/1728403458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5630019991a0 0x56300199d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:57:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fabb3fff700 1 --2- 172.21.15.44:0/1728403458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563001993300 0x56300199fd10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faba4000c00 tx=0x7faba4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2043d9f98c285304 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.671+0000 7fab937fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:57:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.672+0000 7fab937fe700 1 -- 172.21.15.44:0/1728403458 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba402a050 con 0x563001993300 2023-09-10T00:57:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.672+0000 7fab937fe700 1 -- 172.21.15.44:0/1728403458 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9540+0+0 (secure 0 0 0) 0x7faba4005040 con 0x563001993300 2023-09-10T00:57:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.672+0000 7fabc66aab80 1 -- 172.21.15.44:0/1728403458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56300183d490 con 0x563001993300 2023-09-10T00:57:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.672+0000 7fab937fe700 1 -- 172.21.15.44:0/1728403458 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7faba4005940 con 0x563001993300 2023-09-10T00:57:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.672+0000 7fabc66aab80 10 client.27698 fetch_fsmap learned FSMap version 78 2023-09-10T00:57:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.672+0000 7fabc66aab80 1 -- 172.21.15.44:0/1728403458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x7fabac011230 con 0x563001993300 2023-09-10T00:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.672+0000 7fab937fe700 1 -- 172.21.15.44:0/1728403458 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56300183d490 con 0x563001993300 2023-09-10T00:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.672+0000 7fab937fe700 1 -- 172.21.15.44:0/1728403458 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 78) v1 ==== 7922+0+0 (secure 0 0 0) 0x7faba4003c90 con 0x563001993300 2023-09-10T00:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.672+0000 7fabc66aab80 10 client.27698 fetch_fsmap finished waiting for FSMap version 78 2023-09-10T00:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.672+0000 7fabc66aab80 10 client.27698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.672+0000 7fabc66aab80 20 client.27698 populate_metadata read hostname 'smithi044' 2023-09-10T00:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.672+0000 7fabc66aab80 1 --2- 172.21.15.44:0/1728403458 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5630019ad380 0x5630019af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.672+0000 7fabc66aab80 4 client.27698 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:57:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.672+0000 7fabc66aab80 1 -- 172.21.15.44:0/1728403458 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630018fa840 con 0x5630019ad380 2023-09-10T00:57:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.672+0000 7fabb37fe700 1 --2- 172.21.15.44:0/1728403458 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5630019ad380 0x5630019af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.673+0000 7fab937fe700 10 client.27698 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.673+0000 7fabb37fe700 1 --2- 172.21.15.44:0/1728403458 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5630019ad380 0x5630019af7a0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b31d880fe689ed server_cookie=50b2e9636126baa3 in_seq=0 out_seq=0 2023-09-10T00:57:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.673+0000 7fab937fe700 1 -- 172.21.15.44:0/1728403458 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5630018fa840 con 0x5630019ad380 2023-09-10T00:57:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.673+0000 7fab937fe700 10 client.27698 handle_command_reply: tid=0 2023-09-10T00:57:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.673+0000 7fabc66aab80 1 client.27698 shutdown 2023-09-10T00:57:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.674+0000 7fabc66aab80 1 -- 172.21.15.44:0/1728403458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563001993300 msgr2=0x56300199fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.674+0000 7fabc66aab80 1 --2- 172.21.15.44:0/1728403458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563001993300 0x56300199fd10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faba4000c00 tx=0x7faba4009c30 comp rx=0 tx=0).stop 2023-09-10T00:57:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.674+0000 7fabc66aab80 1 -- 172.21.15.44:0/1728403458 shutdown_connections 2023-09-10T00:57:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.674+0000 7fabc66aab80 1 --2- 172.21.15.44:0/1728403458 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5630019ad380 0x5630019af7a0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.674+0000 7fabc66aab80 1 --2- 172.21.15.44:0/1728403458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5630019991a0 0x56300199d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.674+0000 7fab937fe700 0 client.27698 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.674+0000 7fabc66aab80 1 --2- 172.21.15.44:0/1728403458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563001993300 0x56300199fd10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.674+0000 7fabc66aab80 1 --2- 172.21.15.44:0/1728403458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993cc0 0x56300199cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.674+0000 7fabc66aab80 1 -- 172.21.15.44:0/1728403458 >> 172.21.15.44:0/1728403458 conn(0x563001908030 msgr2=0x56300198d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.674+0000 7fabc66aab80 1 -- 172.21.15.44:0/1728403458 shutdown_connections 2023-09-10T00:57:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.674+0000 7fabc66aab80 1 -- 172.21.15.44:0/1728403458 wait complete. 2023-09-10T00:57:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.674+0000 7fabc66aab80 20 client.27698 trim_cache size 0 max 16384 2023-09-10T00:57:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.689+0000 7fabc66aab80 1 Processor -- start 2023-09-10T00:57:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.689+0000 7fabc66aab80 1 -- start start 2023-09-10T00:57:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.689+0000 7fabc66aab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993300 0x5630019a4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.689+0000 7fabc66aab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563001993cc0 0x56300199f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.689+0000 7fabc66aab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5630019991a0 0x56300199fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.689+0000 7fabc66aab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563001994580 con 0x5630019991a0 2023-09-10T00:57:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.689+0000 7fabc66aab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300199d870 con 0x563001993cc0 2023-09-10T00:57:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.689+0000 7fabc66aab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563001997a40 con 0x563001993300 2023-09-10T00:57:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.689+0000 7fabb8a81700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5630019991a0 0x56300199fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.689+0000 7fabb8a81700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5630019991a0 0x56300199fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54090/0 (socket says 172.21.15.44:54090) 2023-09-10T00:57:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.689+0000 7fabb8a81700 1 -- 172.21.15.44:0/2692706001 learned_addr learned my addr 172.21.15.44:0/2692706001 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.689+0000 7fabb3fff700 1 --2- 172.21.15.44:0/2692706001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993300 0x5630019a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.689+0000 7fabb37fe700 1 --2- 172.21.15.44:0/2692706001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563001993cc0 0x56300199f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.690+0000 7fabb8a81700 1 -- 172.21.15.44:0/2692706001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993300 msgr2=0x5630019a4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.690+0000 7fabb8a81700 1 --2- 172.21.15.44:0/2692706001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993300 0x5630019a4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.690+0000 7fabb8a81700 1 -- 172.21.15.44:0/2692706001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563001993cc0 msgr2=0x56300199f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.690+0000 7fabb8a81700 1 --2- 172.21.15.44:0/2692706001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563001993cc0 0x56300199f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.690+0000 7fabb8a81700 1 -- 172.21.15.44:0/2692706001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630016fac00 con 0x5630019991a0 2023-09-10T00:57:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.690+0000 7fabb8a81700 1 --2- 172.21.15.44:0/2692706001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5630019991a0 0x56300199fd10 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fabac009e60 tx=0x7fabac011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd67c7817b1a531e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.690+0000 7fab9affd700 1 -- 172.21.15.44:0/2692706001 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac00e040 con 0x5630019991a0 2023-09-10T00:57:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.690+0000 7fab9affd700 1 -- 172.21.15.44:0/2692706001 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fabac003d10 con 0x5630019991a0 2023-09-10T00:57:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.690+0000 7fab9affd700 1 -- 172.21.15.44:0/2692706001 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac005340 con 0x5630019991a0 2023-09-10T00:57:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.691+0000 7fabc66aab80 1 -- 172.21.15.44:0/2692706001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5630019991a0 msgr2=0x56300199fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.691+0000 7fabc66aab80 1 --2- 172.21.15.44:0/2692706001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5630019991a0 0x56300199fd10 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fabac009e60 tx=0x7fabac011e80 comp rx=0 tx=0).stop 2023-09-10T00:57:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.691+0000 7fabc66aab80 1 -- 172.21.15.44:0/2692706001 shutdown_connections 2023-09-10T00:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.691+0000 7fabc66aab80 1 --2- 172.21.15.44:0/2692706001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5630019991a0 0x56300199fd10 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.691+0000 7fabc66aab80 1 --2- 172.21.15.44:0/2692706001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993300 0x5630019a4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.691+0000 7fabc66aab80 1 --2- 172.21.15.44:0/2692706001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563001993cc0 0x56300199f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.691+0000 7fabc66aab80 1 -- 172.21.15.44:0/2692706001 >> 172.21.15.44:0/2692706001 conn(0x563001908030 msgr2=0x563001992b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.691+0000 7fabc66aab80 1 -- 172.21.15.44:0/2692706001 shutdown_connections 2023-09-10T00:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.691+0000 7fabc66aab80 1 -- 172.21.15.44:0/2692706001 wait complete. 2023-09-10T00:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.691+0000 7fabc66aab80 1 -- start start 2023-09-10T00:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.691+0000 7fabc66aab80 1 Processor -- start 2023-09-10T00:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabc66aab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993300 0x5630019a4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabc66aab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563001993cc0 0x5630019a2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabc66aab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630019991a0 0x5630019a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabc66aab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630019a4910 con 0x563001993cc0 2023-09-10T00:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabc66aab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630019a2db0 con 0x5630019991a0 2023-09-10T00:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabc66aab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630019a2f20 con 0x563001993300 2023-09-10T00:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabc66aab80 10 client.0 _ll_register_callbacks cb 0x5630017cda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabb37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563001993cc0 0x5630019a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabb8a81700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630019991a0 0x5630019a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabb37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563001993cc0 0x5630019a2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54094/0 (socket says 172.21.15.44:54094) 2023-09-10T00:57:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabb3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993300 0x5630019a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabb37fe700 1 -- 172.21.15.44:0/2947248032 learned_addr learned my addr 172.21.15.44:0/2947248032 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabb37fe700 1 -- 172.21.15.44:0/2947248032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993300 msgr2=0x5630019a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabb37fe700 1 --2- 172.21.15.44:0/2947248032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993300 0x5630019a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabb37fe700 1 -- 172.21.15.44:0/2947248032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630019991a0 msgr2=0x5630019a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabb37fe700 1 --2- 172.21.15.44:0/2947248032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630019991a0 0x5630019a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabb37fe700 1 -- 172.21.15.44:0/2947248032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563001711d80 con 0x563001993cc0 2023-09-10T00:57:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabb37fe700 1 --2- 172.21.15.44:0/2947248032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563001993cc0 0x5630019a2330 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7faba800c5b0 tx=0x7faba800c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd106f9e10153d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fab937fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:57:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fab937fe700 1 -- 172.21.15.44:0/2947248032 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80b1ea0 con 0x563001993cc0 2023-09-10T00:57:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fab937fe700 1 -- 172.21.15.44:0/2947248032 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9540+0+0 (secure 0 0 0) 0x7faba8063030 con 0x563001993cc0 2023-09-10T00:57:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.692+0000 7fabc66aab80 1 -- 172.21.15.44:0/2947248032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fabac00f040 con 0x563001993cc0 2023-09-10T00:57:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.693+0000 7fab937fe700 1 -- 172.21.15.44:0/2947248032 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7faba80998a0 con 0x563001993cc0 2023-09-10T00:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.693+0000 7fabc66aab80 10 client.18042 fetch_fsmap learned FSMap version 78 2023-09-10T00:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.693+0000 7fabc66aab80 1 -- 172.21.15.44:0/2947248032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x5630017381f0 con 0x563001993cc0 2023-09-10T00:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.693+0000 7fab937fe700 1 -- 172.21.15.44:0/2947248032 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fabac00f040 con 0x563001993cc0 2023-09-10T00:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.693+0000 7fab937fe700 1 -- 172.21.15.44:0/2947248032 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 78) v1 ==== 7922+0+0 (secure 0 0 0) 0x7faba807f020 con 0x563001993cc0 2023-09-10T00:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.693+0000 7fabc66aab80 10 client.18042 fetch_fsmap finished waiting for FSMap version 78 2023-09-10T00:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.693+0000 7fabc66aab80 10 client.18042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.693+0000 7fabc66aab80 20 client.18042 populate_metadata read hostname 'smithi044' 2023-09-10T00:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.693+0000 7fabc66aab80 1 --2- 172.21.15.44:0/2947248032 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563001a02670 0x563001a04a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.693+0000 7fabc66aab80 4 client.18042 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.693+0000 7fabc66aab80 1 -- 172.21.15.44:0/2947248032 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5630018de8c0 con 0x563001a02670 2023-09-10T00:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.693+0000 7fabb3fff700 1 --2- 172.21.15.44:0/2947248032 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563001a02670 0x563001a04a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.693+0000 7fab937fe700 10 client.18042 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.693+0000 7fabb3fff700 1 --2- 172.21.15.44:0/2947248032 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563001a02670 0x563001a04a90 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e3ef877975ab25f server_cookie=2d1df836e0ae967a in_seq=0 out_seq=0 2023-09-10T00:57:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.708+0000 7fab937fe700 1 -- 172.21.15.44:0/2947248032 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5630018de8c0 con 0x563001a02670 2023-09-10T00:57:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.708+0000 7fab937fe700 10 client.18042 handle_command_reply: tid=0 2023-09-10T00:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.708+0000 7fabc66aab80 1 client.18042 shutdown 2023-09-10T00:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.708+0000 7fabc66aab80 1 -- 172.21.15.44:0/2947248032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563001993cc0 msgr2=0x5630019a2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.708+0000 7fabc66aab80 1 --2- 172.21.15.44:0/2947248032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563001993cc0 0x5630019a2330 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7faba800c5b0 tx=0x7faba800c6e0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.708+0000 7fabc66aab80 1 -- 172.21.15.44:0/2947248032 shutdown_connections 2023-09-10T00:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.708+0000 7fabc66aab80 1 --2- 172.21.15.44:0/2947248032 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563001a02670 0x563001a04a90 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.708+0000 7fabc66aab80 1 --2- 172.21.15.44:0/2947248032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630019991a0 0x5630019a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.708+0000 7fab937fe700 0 client.18042 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.708+0000 7fabc66aab80 1 --2- 172.21.15.44:0/2947248032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563001993300 0x5630019a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.708+0000 7fabc66aab80 1 --2- 172.21.15.44:0/2947248032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563001993cc0 0x5630019a2330 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.708+0000 7fabc66aab80 1 -- 172.21.15.44:0/2947248032 >> 172.21.15.44:0/2947248032 conn(0x563001908030 msgr2=0x5630019a8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.708+0000 7fabc66aab80 1 -- 172.21.15.44:0/2947248032 shutdown_connections 2023-09-10T00:57:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.708+0000 7fabc66aab80 1 -- 172.21.15.44:0/2947248032 wait complete. 2023-09-10T00:57:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.708+0000 7fabc66aab80 20 client.18042 trim_cache size 0 max 16384 2023-09-10T00:57:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:57:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.709+0000 7fabba4e4700 1 -- 172.21.15.44:0/3392200697 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fab9c0a07b0 msgr2=0x7fab9c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.709+0000 7fabba4e4700 1 --2- 172.21.15.44:0/3392200697 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fab9c0a07b0 0x7fab9c0a2c70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faba4009810 tx=0x7faba4019040 comp rx=0 tx=0).stop 2023-09-10T00:57:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.709+0000 7fabba4e4700 1 -- 172.21.15.44:0/3392200697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb4100120 msgr2=0x7fabb4197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.709+0000 7fabba4e4700 1 --2- 172.21.15.44:0/3392200697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb4100120 0x7fabb4197bc0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7faba800ec10 tx=0x7faba800ef20 comp rx=0 tx=0).stop 2023-09-10T00:57:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.709+0000 7fabba4e4700 1 -- 172.21.15.44:0/3392200697 shutdown_connections 2023-09-10T00:57:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.709+0000 7fabba4e4700 1 --2- 172.21.15.44:0/3392200697 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fab9c0a07b0 0x7fab9c0a2c70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.709+0000 7fabba4e4700 1 --2- 172.21.15.44:0/3392200697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabb41055b0 0x7fabb41a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.709+0000 7fabba4e4700 1 --2- 172.21.15.44:0/3392200697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabb40ff760 0x7fabb4197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.709+0000 7fabba4e4700 1 --2- 172.21.15.44:0/3392200697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb4100120 0x7fabb4197bc0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.709+0000 7fabba4e4700 1 -- 172.21.15.44:0/3392200697 >> 172.21.15.44:0/3392200697 conn(0x7fabb4074f30 msgr2=0x7fabb40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.709+0000 7fabba4e4700 1 -- 172.21.15.44:0/3392200697 shutdown_connections 2023-09-10T00:57:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:16.709+0000 7fabba4e4700 1 -- 172.21.15.44:0/3392200697 wait complete. 2023-09-10T00:57:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:16 smithi131 ceph-mon[103713]: pgmap v524: 129 pgs: 129 active+clean; 8.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2023-09-10T00:57:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:16 smithi044 ceph-mon[98676]: pgmap v524: 129 pgs: 129 active+clean; 8.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2023-09-10T00:57:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:16 smithi099 ceph-mon[102692]: pgmap v524: 129 pgs: 129 active+clean; 8.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2023-09-10T00:57:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:18 smithi131 ceph-mon[103713]: pgmap v525: 129 pgs: 129 active+clean; 8.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2023-09-10T00:57:19.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:18 smithi044 ceph-mon[98676]: pgmap v525: 129 pgs: 129 active+clean; 8.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2023-09-10T00:57:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:18 smithi099 ceph-mon[102692]: pgmap v525: 129 pgs: 129 active+clean; 8.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2023-09-10T00:57:20.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:19 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:20.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:19 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:20.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:19 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:21.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:20 smithi131 ceph-mon[103713]: pgmap v526: 129 pgs: 129 active+clean; 8.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2023-09-10T00:57:21.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:20 smithi044 ceph-mon[98676]: pgmap v526: 129 pgs: 129 active+clean; 8.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2023-09-10T00:57:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:20 smithi099 ceph-mon[102692]: pgmap v526: 129 pgs: 129 active+clean; 8.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2023-09-10T00:57:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:22 smithi131 ceph-mon[103713]: pgmap v527: 129 pgs: 129 active+clean; 8.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 291 op/s 2023-09-10T00:57:23.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:22 smithi044 ceph-mon[98676]: pgmap v527: 129 pgs: 129 active+clean; 8.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 291 op/s 2023-09-10T00:57:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:22 smithi099 ceph-mon[102692]: pgmap v527: 129 pgs: 129 active+clean; 8.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 291 op/s 2023-09-10T00:57:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:24 smithi131 ceph-mon[103713]: pgmap v528: 129 pgs: 129 active+clean; 8.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2023-09-10T00:57:25.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:24 smithi044 ceph-mon[98676]: pgmap v528: 129 pgs: 129 active+clean; 8.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2023-09-10T00:57:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:24 smithi099 ceph-mon[102692]: pgmap v528: 129 pgs: 129 active+clean; 8.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2023-09-10T00:57:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:26 smithi131 ceph-mon[103713]: pgmap v529: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 377 op/s 2023-09-10T00:57:27.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:26 smithi044 ceph-mon[98676]: pgmap v529: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 377 op/s 2023-09-10T00:57:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:26 smithi099 ceph-mon[102692]: pgmap v529: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 377 op/s 2023-09-10T00:57:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:28 smithi044 ceph-mon[98676]: pgmap v530: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2023-09-10T00:57:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:28 smithi099 ceph-mon[102692]: pgmap v530: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2023-09-10T00:57:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:28 smithi131 ceph-mon[103713]: pgmap v530: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2023-09-10T00:57:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.824+0000 7f583a014700 1 -- 172.21.15.44:0/1694588119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5834100120 msgr2=0x7f583410c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.824+0000 7f583a014700 1 --2- 172.21.15.44:0/1694588119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5834100120 0x7f583410c330 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f582c009f90 tx=0x7f582c00b330 comp rx=0 tx=0).stop 2023-09-10T00:57:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.824+0000 7f583a014700 1 -- 172.21.15.44:0/1694588119 shutdown_connections 2023-09-10T00:57:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.824+0000 7f583a014700 1 --2- 172.21.15.44:0/1694588119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5834100120 0x7f583410c330 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.824+0000 7f583a014700 1 --2- 172.21.15.44:0/1694588119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58341055b0 0x7f5834105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.824+0000 7f583a014700 1 --2- 172.21.15.44:0/1694588119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58340ff760 0x7f58340ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.824+0000 7f583a014700 1 -- 172.21.15.44:0/1694588119 >> 172.21.15.44:0/1694588119 conn(0x7f5834074f30 msgr2=0x7f5834075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.824+0000 7f583a014700 1 -- 172.21.15.44:0/1694588119 shutdown_connections 2023-09-10T00:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.825+0000 7f583a014700 1 -- 172.21.15.44:0/1694588119 wait complete. 2023-09-10T00:57:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.825+0000 7f583a014700 1 Processor -- start 2023-09-10T00:57:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f583a014700 1 -- start start 2023-09-10T00:57:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f583a014700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58340ff760 0x7f58341975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f583a014700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5834100120 0x7f5834197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f583a014700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58341055b0 0x7f58341a2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f583a014700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58341a2a80 con 0x7f58340ff760 2023-09-10T00:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f583a014700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58341a2bf0 con 0x7f5834100120 2023-09-10T00:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f583a014700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58341a2db0 con 0x7f58341055b0 2023-09-10T00:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f58337fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58340ff760 0x7f58341975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f58337fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58340ff760 0x7f58341975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54106/0 (socket says 172.21.15.44:54106) 2023-09-10T00:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f58337fe700 1 -- 172.21.15.44:0/2959349173 learned_addr learned my addr 172.21.15.44:0/2959349173 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f5832ffd700 1 --2- 172.21.15.44:0/2959349173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5834100120 0x7f5834197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f5833fff700 1 --2- 172.21.15.44:0/2959349173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58341055b0 0x7f58341a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f58337fe700 1 -- 172.21.15.44:0/2959349173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58341055b0 msgr2=0x7f58341a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f58337fe700 1 --2- 172.21.15.44:0/2959349173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58341055b0 0x7f58341a2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f58337fe700 1 -- 172.21.15.44:0/2959349173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5834100120 msgr2=0x7f5834197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f58337fe700 1 --2- 172.21.15.44:0/2959349173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5834100120 0x7f5834197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f58337fe700 1 -- 172.21.15.44:0/2959349173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58240095f0 con 0x7f58340ff760 2023-09-10T00:57:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.826+0000 7f58337fe700 1 --2- 172.21.15.44:0/2959349173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58340ff760 0x7f58341975e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f582400ec10 tx=0x7f582400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.827+0000 7f5830ff9700 1 -- 172.21.15.44:0/2959349173 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582400cc90 con 0x7f58340ff760 2023-09-10T00:57:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.827+0000 7f583a014700 1 -- 172.21.15.44:0/2959349173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c009c40 con 0x7f58340ff760 2023-09-10T00:57:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.827+0000 7f5830ff9700 1 -- 172.21.15.44:0/2959349173 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f582400cdf0 con 0x7f58340ff760 2023-09-10T00:57:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.827+0000 7f5830ff9700 1 -- 172.21.15.44:0/2959349173 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824003e80 con 0x7f58340ff760 2023-09-10T00:57:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.827+0000 7f583a014700 1 -- 172.21.15.44:0/2959349173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58341a3410 con 0x7f58340ff760 2023-09-10T00:57:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.829+0000 7f5830ff9700 1 -- 172.21.15.44:0/2959349173 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f5824004000 con 0x7f58340ff760 2023-09-10T00:57:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.830+0000 7f5830ff9700 1 --2- 172.21.15.44:0/2959349173 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f581c0a07b0 0x7f581c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.830+0000 7f5830ff9700 1 -- 172.21.15.44:0/2959349173 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f5824014070 con 0x7f58340ff760 2023-09-10T00:57:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.830+0000 7f5832ffd700 1 --2- 172.21.15.44:0/2959349173 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f581c0a07b0 0x7f581c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.831+0000 7f5832ffd700 1 --2- 172.21.15.44:0/2959349173 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f581c0a07b0 0x7f581c0a2c70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5834068510 tx=0x7f5828009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.831+0000 7f58461dab80 1 Processor -- start 2023-09-10T00:57:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.831+0000 7f58461dab80 1 -- start start 2023-09-10T00:57:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.831+0000 7f58461dab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef151a0 0x55739ef15580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.831+0000 7f58461dab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0f300 0x55739ef0f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.831+0000 7f58461dab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef0fcc0 0x55739ef1bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.831+0000 7f58461dab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ee686d0 con 0x55739ef0f300 2023-09-10T00:57:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.831+0000 7f58461dab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ee65ff0 con 0x55739ef151a0 2023-09-10T00:57:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.831+0000 7f58461dab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ee618e0 con 0x55739ef0fcc0 2023-09-10T00:57:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.831+0000 7f5832ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0f300 0x55739ef0f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.831+0000 7f5833fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef0fcc0 0x55739ef1bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.831+0000 7f5832ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0f300 0x55739ef0f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54116/0 (socket says 172.21.15.44:54116) 2023-09-10T00:57:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.831+0000 7f5833fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef0fcc0 0x55739ef1bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:32768/0 (socket says 172.21.15.44:32768) 2023-09-10T00:57:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.831+0000 7f5832ffd700 1 -- 172.21.15.44:0/2557613681 learned_addr learned my addr 172.21.15.44:0/2557613681 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.831+0000 7f58337fe700 1 --2- 172.21.15.44:0/2557613681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef151a0 0x55739ef15580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.832+0000 7f5832ffd700 1 -- 172.21.15.44:0/2557613681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef0fcc0 msgr2=0x55739ef1bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.832+0000 7f5832ffd700 1 --2- 172.21.15.44:0/2557613681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef0fcc0 0x55739ef1bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.832+0000 7f5832ffd700 1 -- 172.21.15.44:0/2557613681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef151a0 msgr2=0x55739ef15580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.832+0000 7f5832ffd700 1 --2- 172.21.15.44:0/2557613681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef151a0 0x55739ef15580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.832+0000 7f5832ffd700 1 -- 172.21.15.44:0/2557613681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55739ec8dd80 con 0x55739ef0f300 2023-09-10T00:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.832+0000 7f5832ffd700 1 --2- 172.21.15.44:0/2557613681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0f300 0x55739ef0f780 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5828011d20 tx=0x7f5828014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05340f37250d77a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.832+0000 7f5819ffb700 1 -- 172.21.15.44:0/2557613681 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582800eb20 con 0x55739ef0f300 2023-09-10T00:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.832+0000 7f5819ffb700 1 -- 172.21.15.44:0/2557613681 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5828021070 con 0x55739ef0f300 2023-09-10T00:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.832+0000 7f5819ffb700 1 -- 172.21.15.44:0/2557613681 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582801d410 con 0x55739ef0f300 2023-09-10T00:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.833+0000 7f58461dab80 1 -- 172.21.15.44:0/2557613681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0f300 msgr2=0x55739ef0f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.833+0000 7f58461dab80 1 --2- 172.21.15.44:0/2557613681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0f300 0x55739ef0f780 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5828011d20 tx=0x7f5828014000 comp rx=0 tx=0).stop 2023-09-10T00:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.833+0000 7f58461dab80 1 -- 172.21.15.44:0/2557613681 shutdown_connections 2023-09-10T00:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.833+0000 7f58461dab80 1 --2- 172.21.15.44:0/2557613681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef0fcc0 0x55739ef1bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.833+0000 7f58461dab80 1 --2- 172.21.15.44:0/2557613681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef151a0 0x55739ef15580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.833+0000 7f58461dab80 1 --2- 172.21.15.44:0/2557613681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0f300 0x55739ef0f780 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.833+0000 7f58461dab80 1 -- 172.21.15.44:0/2557613681 >> 172.21.15.44:0/2557613681 conn(0x55739ee84030 msgr2=0x55739ef0a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.833+0000 7f58461dab80 1 -- 172.21.15.44:0/2557613681 shutdown_connections 2023-09-10T00:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.833+0000 7f58461dab80 1 -- 172.21.15.44:0/2557613681 wait complete. 2023-09-10T00:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.833+0000 7f58461dab80 1 -- start start 2023-09-10T00:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.833+0000 7f58461dab80 1 Processor -- start 2023-09-10T00:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f58461dab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef0f300 0x55739ef1bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f58461dab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef0fcc0 0x55739ef19070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f58461dab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef151a0 0x55739ef195d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f58461dab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739edffb10 con 0x55739ef151a0 2023-09-10T00:57:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f58461dab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739edff810 con 0x55739ef0f300 2023-09-10T00:57:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f58461dab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ece38c0 con 0x55739ef0fcc0 2023-09-10T00:57:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f58461dab80 10 client.0 _ll_register_callbacks cb 0x55739ed49a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:57:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f5833fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef151a0 0x55739ef195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f58337fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef0f300 0x55739ef1bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f58337fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef0f300 0x55739ef1bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55860/0 (socket says 172.21.15.44:55860) 2023-09-10T00:57:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f5833fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef151a0 0x55739ef195d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54124/0 (socket says 172.21.15.44:54124) 2023-09-10T00:57:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f58337fe700 1 -- 172.21.15.44:0/366728764 learned_addr learned my addr 172.21.15.44:0/366728764 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f5832ffd700 1 --2- 172.21.15.44:0/366728764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef0fcc0 0x55739ef19070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f58337fe700 1 -- 172.21.15.44:0/366728764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef0fcc0 msgr2=0x55739ef19070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f58337fe700 1 --2- 172.21.15.44:0/366728764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef0fcc0 0x55739ef19070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f58337fe700 1 -- 172.21.15.44:0/366728764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef151a0 msgr2=0x55739ef195d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f58337fe700 1 --2- 172.21.15.44:0/366728764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef151a0 0x55739ef195d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f58337fe700 1 -- 172.21.15.44:0/366728764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5828017070 con 0x55739ef0f300 2023-09-10T00:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f58337fe700 1 --2- 172.21.15.44:0/366728764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef0f300 0x55739ef1bd50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f58240ca140 tx=0x7f58240992f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4d2b483374a913f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.834+0000 7f5812ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:57:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.835+0000 7f5812ffd700 1 -- 172.21.15.44:0/366728764 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58240b1ea0 con 0x55739ef0f300 2023-09-10T00:57:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.835+0000 7f5812ffd700 1 -- 172.21.15.44:0/366728764 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f58240998a0 con 0x55739ef0f300 2023-09-10T00:57:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.835+0000 7f58461dab80 1 -- 172.21.15.44:0/366728764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55739edb9490 con 0x55739ef0f300 2023-09-10T00:57:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.835+0000 7f5812ffd700 1 -- 172.21.15.44:0/366728764 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58240044b0 con 0x55739ef0f300 2023-09-10T00:57:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.835+0000 7f58461dab80 10 client.27716 fetch_fsmap learned FSMap version 80 2023-09-10T00:57:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.835+0000 7f58461dab80 1 -- 172.21.15.44:0/366728764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55739ec76c00 con 0x55739ef0f300 2023-09-10T00:57:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.835+0000 7f5812ffd700 1 -- 172.21.15.44:0/366728764 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55739edb9490 con 0x55739ef0f300 2023-09-10T00:57:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.835+0000 7f5812ffd700 1 -- 172.21.15.44:0/366728764 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 80) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5824098d90 con 0x55739ef0f300 2023-09-10T00:57:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.835+0000 7f58461dab80 10 client.27716 fetch_fsmap finished waiting for FSMap version 80 2023-09-10T00:57:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.835+0000 7f58461dab80 10 client.27716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:57:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.835+0000 7f58461dab80 20 client.27716 populate_metadata read hostname 'smithi044' 2023-09-10T00:57:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.835+0000 7f58461dab80 1 --2- 172.21.15.44:0/366728764 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55739ef293b0 0x55739ef2b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.835+0000 7f58461dab80 4 client.27716 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:57:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.835+0000 7f58461dab80 1 -- 172.21.15.44:0/366728764 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55739ee76840 con 0x55739ef293b0 2023-09-10T00:57:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.835+0000 7f5832ffd700 1 --2- 172.21.15.44:0/366728764 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55739ef293b0 0x55739ef2b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.836+0000 7f5832ffd700 1 --2- 172.21.15.44:0/366728764 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55739ef293b0 0x55739ef2b7d0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b4f98ab802337ec server_cookie=334c3f66587c7f0c in_seq=0 out_seq=0 2023-09-10T00:57:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.836+0000 7f5812ffd700 10 client.27716 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.836+0000 7f5812ffd700 1 -- 172.21.15.44:0/366728764 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55739ee76840 con 0x55739ef293b0 2023-09-10T00:57:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.836+0000 7f5812ffd700 10 client.27716 handle_command_reply: tid=0 2023-09-10T00:57:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.836+0000 7f58461dab80 1 client.27716 shutdown 2023-09-10T00:57:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.836+0000 7f58461dab80 1 -- 172.21.15.44:0/366728764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef0f300 msgr2=0x55739ef1bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.836+0000 7f58461dab80 1 --2- 172.21.15.44:0/366728764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef0f300 0x55739ef1bd50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f58240ca140 tx=0x7f58240992f0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.837+0000 7f58461dab80 1 -- 172.21.15.44:0/366728764 shutdown_connections 2023-09-10T00:57:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.837+0000 7f58461dab80 1 --2- 172.21.15.44:0/366728764 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55739ef293b0 0x55739ef2b7d0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.837+0000 7f58461dab80 1 --2- 172.21.15.44:0/366728764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef151a0 0x55739ef195d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.837+0000 7f5812ffd700 0 client.27716 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.837+0000 7f58461dab80 1 --2- 172.21.15.44:0/366728764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef0f300 0x55739ef1bd50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.837+0000 7f58461dab80 1 --2- 172.21.15.44:0/366728764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef0fcc0 0x55739ef19070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.837+0000 7f58461dab80 1 -- 172.21.15.44:0/366728764 >> 172.21.15.44:0/366728764 conn(0x55739ee84030 msgr2=0x55739ef097a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.837+0000 7f58461dab80 1 -- 172.21.15.44:0/366728764 shutdown_connections 2023-09-10T00:57:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.837+0000 7f58461dab80 1 -- 172.21.15.44:0/366728764 wait complete. 2023-09-10T00:57:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.837+0000 7f58461dab80 20 client.27716 trim_cache size 0 max 16384 2023-09-10T00:57:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.852+0000 7f58461dab80 1 Processor -- start 2023-09-10T00:57:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.852+0000 7f58461dab80 1 -- start start 2023-09-10T00:57:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.852+0000 7f58461dab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0f300 0x55739ef20530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.852+0000 7f58461dab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef0fcc0 0x55739ef0ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.852+0000 7f58461dab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef151a0 0x55739ef1b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.852+0000 7f58461dab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ef26aa0 con 0x55739ef0f300 2023-09-10T00:57:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.852+0000 7f58461dab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ef0ab00 con 0x55739ef151a0 2023-09-10T00:57:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.852+0000 7f58461dab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ef105d0 con 0x55739ef0fcc0 2023-09-10T00:57:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.852+0000 7f58337fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0f300 0x55739ef20530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.852+0000 7f58337fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0f300 0x55739ef20530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54128/0 (socket says 172.21.15.44:54128) 2023-09-10T00:57:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.852+0000 7f58337fe700 1 -- 172.21.15.44:0/4172758610 learned_addr learned my addr 172.21.15.44:0/4172758610 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.852+0000 7f5833fff700 1 --2- 172.21.15.44:0/4172758610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef151a0 0x55739ef1b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.852+0000 7f5832ffd700 1 --2- 172.21.15.44:0/4172758610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef0fcc0 0x55739ef0ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.852+0000 7f58337fe700 1 -- 172.21.15.44:0/4172758610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef0fcc0 msgr2=0x55739ef0ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.852+0000 7f58337fe700 1 --2- 172.21.15.44:0/4172758610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef0fcc0 0x55739ef0ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.852+0000 7f58337fe700 1 -- 172.21.15.44:0/4172758610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef151a0 msgr2=0x55739ef1b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.853+0000 7f58337fe700 1 --2- 172.21.15.44:0/4172758610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef151a0 0x55739ef1b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.853+0000 7f58337fe700 1 -- 172.21.15.44:0/4172758610 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55739ecb41f0 con 0x55739ef0f300 2023-09-10T00:57:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.853+0000 7f58337fe700 1 --2- 172.21.15.44:0/4172758610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0f300 0x55739ef20530 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f58240b9120 tx=0x7f5824095d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a17c0181266e1a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.853+0000 7f581a7fc700 1 -- 172.21.15.44:0/4172758610 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58240b1ea0 con 0x55739ef0f300 2023-09-10T00:57:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.853+0000 7f581a7fc700 1 -- 172.21.15.44:0/4172758610 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58240998a0 con 0x55739ef0f300 2023-09-10T00:57:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.853+0000 7f581a7fc700 1 -- 172.21.15.44:0/4172758610 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582409d6d0 con 0x55739ef0f300 2023-09-10T00:57:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.853+0000 7f58461dab80 1 -- 172.21.15.44:0/4172758610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0f300 msgr2=0x55739ef20530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 1 --2- 172.21.15.44:0/4172758610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0f300 0x55739ef20530 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f58240b9120 tx=0x7f5824095d10 comp rx=0 tx=0).stop 2023-09-10T00:57:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 1 -- 172.21.15.44:0/4172758610 shutdown_connections 2023-09-10T00:57:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 1 --2- 172.21.15.44:0/4172758610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef151a0 0x55739ef1b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 1 --2- 172.21.15.44:0/4172758610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0f300 0x55739ef20530 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 1 --2- 172.21.15.44:0/4172758610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef0fcc0 0x55739ef0ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 1 -- 172.21.15.44:0/4172758610 >> 172.21.15.44:0/4172758610 conn(0x55739ee84030 msgr2=0x55739ef0e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 1 -- 172.21.15.44:0/4172758610 shutdown_connections 2023-09-10T00:57:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 1 -- 172.21.15.44:0/4172758610 wait complete. 2023-09-10T00:57:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 1 -- start start 2023-09-10T00:57:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 1 Processor -- start 2023-09-10T00:57:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef0f300 0x55739ef7ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0fcc0 0x55739ef7b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef151a0 0x55739ef75540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ef75b20 con 0x55739ef0fcc0 2023-09-10T00:57:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ef75c90 con 0x55739ef0f300 2023-09-10T00:57:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ef75e50 con 0x55739ef151a0 2023-09-10T00:57:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.854+0000 7f58461dab80 10 client.0 _ll_register_callbacks cb 0x7f5824099740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:57:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f5832ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0fcc0 0x55739ef7b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f5832ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0fcc0 0x55739ef7b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54136/0 (socket says 172.21.15.44:54136) 2023-09-10T00:57:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f58337fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef0f300 0x55739ef7ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f5832ffd700 1 -- 172.21.15.44:0/1849979521 learned_addr learned my addr 172.21.15.44:0/1849979521 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f5833fff700 1 --2- 172.21.15.44:0/1849979521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef151a0 0x55739ef75540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f5832ffd700 1 -- 172.21.15.44:0/1849979521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef151a0 msgr2=0x55739ef75540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f5832ffd700 1 --2- 172.21.15.44:0/1849979521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef151a0 0x55739ef75540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f5832ffd700 1 -- 172.21.15.44:0/1849979521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef0f300 msgr2=0x55739ef7ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f5832ffd700 1 --2- 172.21.15.44:0/1849979521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef0f300 0x55739ef7ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f5832ffd700 1 -- 172.21.15.44:0/1849979521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55739ec76c00 con 0x55739ef0fcc0 2023-09-10T00:57:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f5833fff700 1 --2- 172.21.15.44:0/1849979521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef151a0 0x55739ef75540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:57:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f5832ffd700 1 --2- 172.21.15.44:0/1849979521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0fcc0 0x55739ef7b370 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f5828011b30 tx=0x7f582800ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb5df6453e4c7b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f5812ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:57:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f5812ffd700 1 -- 172.21.15.44:0/1849979521 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828032a20 con 0x55739ef0fcc0 2023-09-10T00:57:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f5812ffd700 1 -- 172.21.15.44:0/1849979521 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f5828032b80 con 0x55739ef0fcc0 2023-09-10T00:57:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f58461dab80 1 -- 172.21.15.44:0/1849979521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5828021070 con 0x55739ef0fcc0 2023-09-10T00:57:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f5812ffd700 1 -- 172.21.15.44:0/1849979521 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5828032d40 con 0x55739ef0fcc0 2023-09-10T00:57:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f58461dab80 10 client.18060 fetch_fsmap learned FSMap version 80 2023-09-10T00:57:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.855+0000 7f58461dab80 1 -- 172.21.15.44:0/1849979521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55739ec8c250 con 0x55739ef0fcc0 2023-09-10T00:57:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.856+0000 7f5812ffd700 1 -- 172.21.15.44:0/1849979521 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5828021070 con 0x55739ef0fcc0 2023-09-10T00:57:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.856+0000 7f5812ffd700 1 -- 172.21.15.44:0/1849979521 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 80) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5828039060 con 0x55739ef0fcc0 2023-09-10T00:57:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.856+0000 7f58461dab80 10 client.18060 fetch_fsmap finished waiting for FSMap version 80 2023-09-10T00:57:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.856+0000 7f58461dab80 10 client.18060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:57:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.856+0000 7f58461dab80 20 client.18060 populate_metadata read hostname 'smithi044' 2023-09-10T00:57:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.856+0000 7f58461dab80 1 --2- 172.21.15.44:0/1849979521 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55739ef82d70 0x55739ef85190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.856+0000 7f58461dab80 4 client.18060 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:57:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.856+0000 7f58461dab80 1 -- 172.21.15.44:0/1849979521 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55739ee5a8c0 con 0x55739ef82d70 2023-09-10T00:57:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.856+0000 7f58337fe700 1 --2- 172.21.15.44:0/1849979521 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55739ef82d70 0x55739ef85190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.856+0000 7f5812ffd700 10 client.18060 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.856+0000 7f58337fe700 1 --2- 172.21.15.44:0/1849979521 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55739ef82d70 0x55739ef85190 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64617f20683dce75 server_cookie=7ffa948db8756691 in_seq=0 out_seq=0 2023-09-10T00:57:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.872+0000 7f5812ffd700 1 -- 172.21.15.44:0/1849979521 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55739ee5a8c0 con 0x55739ef82d70 2023-09-10T00:57:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.872+0000 7f5812ffd700 10 client.18060 handle_command_reply: tid=0 2023-09-10T00:57:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.872+0000 7f58461dab80 1 client.18060 shutdown 2023-09-10T00:57:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.872+0000 7f58461dab80 1 -- 172.21.15.44:0/1849979521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0fcc0 msgr2=0x55739ef7b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.872+0000 7f58461dab80 1 --2- 172.21.15.44:0/1849979521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0fcc0 0x55739ef7b370 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f5828011b30 tx=0x7f582800ea20 comp rx=0 tx=0).stop 2023-09-10T00:57:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.872+0000 7f58461dab80 1 -- 172.21.15.44:0/1849979521 shutdown_connections 2023-09-10T00:57:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f58461dab80 1 --2- 172.21.15.44:0/1849979521 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55739ef82d70 0x55739ef85190 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f58461dab80 1 --2- 172.21.15.44:0/1849979521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55739ef151a0 0x55739ef75540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f58461dab80 1 --2- 172.21.15.44:0/1849979521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55739ef0f300 0x55739ef7ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f58461dab80 1 --2- 172.21.15.44:0/1849979521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55739ef0fcc0 0x55739ef7b370 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f58461dab80 1 -- 172.21.15.44:0/1849979521 >> 172.21.15.44:0/1849979521 conn(0x55739ee84030 msgr2=0x55739ef14870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f5812ffd700 0 client.18060 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f58461dab80 1 -- 172.21.15.44:0/1849979521 shutdown_connections 2023-09-10T00:57:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f58461dab80 1 -- 172.21.15.44:0/1849979521 wait complete. 2023-09-10T00:57:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f58461dab80 20 client.18060 trim_cache size 0 max 16384 2023-09-10T00:57:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:57:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:57:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:57:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:57:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f583a014700 1 -- 172.21.15.44:0/2959349173 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f581c0a07b0 msgr2=0x7f581c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f583a014700 1 --2- 172.21.15.44:0/2959349173 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f581c0a07b0 0x7f581c0a2c70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5834068510 tx=0x7f5828009260 comp rx=0 tx=0).stop 2023-09-10T00:57:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f583a014700 1 -- 172.21.15.44:0/2959349173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58340ff760 msgr2=0x7f58341975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f583a014700 1 --2- 172.21.15.44:0/2959349173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58340ff760 0x7f58341975e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f582400ec10 tx=0x7f582400ef20 comp rx=0 tx=0).stop 2023-09-10T00:57:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f583a014700 1 -- 172.21.15.44:0/2959349173 shutdown_connections 2023-09-10T00:57:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f583a014700 1 --2- 172.21.15.44:0/2959349173 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f581c0a07b0 0x7f581c0a2c70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f583a014700 1 --2- 172.21.15.44:0/2959349173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58341055b0 0x7f58341a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f583a014700 1 --2- 172.21.15.44:0/2959349173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58340ff760 0x7f58341975e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f583a014700 1 --2- 172.21.15.44:0/2959349173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5834100120 0x7f5834197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f583a014700 1 -- 172.21.15.44:0/2959349173 >> 172.21.15.44:0/2959349173 conn(0x7f5834074f30 msgr2=0x7f58340fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f583a014700 1 -- 172.21.15.44:0/2959349173 shutdown_connections 2023-09-10T00:57:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:29.873+0000 7f583a014700 1 -- 172.21.15.44:0/2959349173 wait complete. 2023-09-10T00:57:30.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:29 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:30.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:29 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:30.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:29 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:30 smithi099 ceph-mon[102692]: pgmap v531: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2023-09-10T00:57:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:30 smithi131 ceph-mon[103713]: pgmap v531: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2023-09-10T00:57:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:30 smithi044 ceph-mon[98676]: pgmap v531: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2023-09-10T00:57:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:32 smithi099 ceph-mon[102692]: pgmap v532: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 382 op/s 2023-09-10T00:57:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:32 smithi131 ceph-mon[103713]: pgmap v532: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 382 op/s 2023-09-10T00:57:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:32 smithi044 ceph-mon[98676]: pgmap v532: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 382 op/s 2023-09-10T00:57:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:34 smithi099 ceph-mon[102692]: pgmap v533: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 246 op/s 2023-09-10T00:57:35.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:34 smithi131 ceph-mon[103713]: pgmap v533: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 246 op/s 2023-09-10T00:57:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:34 smithi044 ceph-mon[98676]: pgmap v533: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 246 op/s 2023-09-10T00:57:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:36 smithi099 ceph-mon[102692]: pgmap v534: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 352 op/s 2023-09-10T00:57:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:36 smithi131 ceph-mon[103713]: pgmap v534: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 352 op/s 2023-09-10T00:57:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:36 smithi044 ceph-mon[98676]: pgmap v534: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 352 op/s 2023-09-10T00:57:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:38 smithi099 ceph-mon[102692]: pgmap v535: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T00:57:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:38 smithi131 ceph-mon[103713]: pgmap v535: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T00:57:39.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:38 smithi044 ceph-mon[98676]: pgmap v535: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T00:57:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:40 smithi099 ceph-mon[102692]: pgmap v536: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T00:57:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:40 smithi131 ceph-mon[103713]: pgmap v536: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T00:57:41.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:40 smithi044 ceph-mon[98676]: pgmap v536: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T00:57:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.985+0000 7f9fa3f8c700 1 -- 172.21.15.44:0/1543726389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f9c100f80 msgr2=0x7f9f9c10d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.985+0000 7f9fa3f8c700 1 --2- 172.21.15.44:0/1543726389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f9c100f80 0x7f9f9c10d440 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400b0d0 tx=0x7f9f9400b3e0 comp rx=0 tx=0).stop 2023-09-10T00:57:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.986+0000 7f9fa3f8c700 1 -- 172.21.15.44:0/1543726389 shutdown_connections 2023-09-10T00:57:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.986+0000 7f9fa3f8c700 1 --2- 172.21.15.44:0/1543726389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f9c100f80 0x7f9f9c10d440 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.986+0000 7f9fa3f8c700 1 --2- 172.21.15.44:0/1543726389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f9c106680 0x7f9f9c106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.986+0000 7f9fa3f8c700 1 --2- 172.21.15.44:0/1543726389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f9c1005c0 0x7f9f9c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.986+0000 7f9fa3f8c700 1 -- 172.21.15.44:0/1543726389 >> 172.21.15.44:0/1543726389 conn(0x7f9f9c0fc0b0 msgr2=0x7f9f9c0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.986+0000 7f9fa3f8c700 1 -- 172.21.15.44:0/1543726389 shutdown_connections 2023-09-10T00:57:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.986+0000 7f9fa3f8c700 1 -- 172.21.15.44:0/1543726389 wait complete. 2023-09-10T00:57:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.986+0000 7f9fa3f8c700 1 Processor -- start 2023-09-10T00:57:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.986+0000 7f9fa3f8c700 1 -- start start 2023-09-10T00:57:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.986+0000 7f9fa3f8c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f9c1005c0 0x7f9f9c072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.986+0000 7f9fa3f8c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f9c100f80 0x7f9f9c073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.986+0000 7f9fa3f8c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f9c106680 0x7f9f9c1a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.986+0000 7f9fa3f8c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c1a4d70 con 0x7f9f9c106680 2023-09-10T00:57:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.986+0000 7f9fa3f8c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c1a4eb0 con 0x7f9f9c100f80 2023-09-10T00:57:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.986+0000 7f9fa3f8c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c1a5020 con 0x7f9f9c1005c0 2023-09-10T00:57:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9fa2529700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f9c106680 0x7f9f9c1a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9fa1d28700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f9c1005c0 0x7f9f9c072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9fa1d28700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f9c1005c0 0x7f9f9c072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:32798/0 (socket says 172.21.15.44:32798) 2023-09-10T00:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9fa2529700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f9c106680 0x7f9f9c1a4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54152/0 (socket says 172.21.15.44:54152) 2023-09-10T00:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9fa1d28700 1 -- 172.21.15.44:0/2592528952 learned_addr learned my addr 172.21.15.44:0/2592528952 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9fa1527700 1 --2- 172.21.15.44:0/2592528952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f9c100f80 0x7f9f9c073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9fa2529700 1 -- 172.21.15.44:0/2592528952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f9c1005c0 msgr2=0x7f9f9c072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9fa2529700 1 --2- 172.21.15.44:0/2592528952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f9c1005c0 0x7f9f9c072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9fa2529700 1 -- 172.21.15.44:0/2592528952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f9c100f80 msgr2=0x7f9f9c073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9fa2529700 1 --2- 172.21.15.44:0/2592528952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f9c100f80 0x7f9f9c073020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9fa2529700 1 -- 172.21.15.44:0/2592528952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f94009d00 con 0x7f9f9c106680 2023-09-10T00:57:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9fa2529700 1 --2- 172.21.15.44:0/2592528952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f9c106680 0x7f9f9c1a4830 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400b770 tx=0x7f9f94008e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9f92ffd700 1 -- 172.21.15.44:0/2592528952 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400e040 con 0x7f9f9c106680 2023-09-10T00:57:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9fa3f8c700 1 -- 172.21.15.44:0/2592528952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9c1a52a0 con 0x7f9f9c106680 2023-09-10T00:57:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9f92ffd700 1 -- 172.21.15.44:0/2592528952 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f9401d900 con 0x7f9f9c106680 2023-09-10T00:57:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9f92ffd700 1 -- 172.21.15.44:0/2592528952 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9401b7e0 con 0x7f9f9c106680 2023-09-10T00:57:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.987+0000 7f9fa3f8c700 1 -- 172.21.15.44:0/2592528952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9c1a5770 con 0x7f9f9c106680 2023-09-10T00:57:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.990+0000 7f9f92ffd700 1 -- 172.21.15.44:0/2592528952 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f9f9401da70 con 0x7f9f9c106680 2023-09-10T00:57:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.990+0000 7f9f92ffd700 1 --2- 172.21.15.44:0/2592528952 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9f880a07b0 0x7f9f880a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.990+0000 7f9f92ffd700 1 -- 172.21.15.44:0/2592528952 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f9f940cd610 con 0x7f9f9c106680 2023-09-10T00:57:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.990+0000 7f9fa1d28700 1 --2- 172.21.15.44:0/2592528952 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9f880a07b0 0x7f9f880a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.991+0000 7f9fa1d28700 1 --2- 172.21.15.44:0/2592528952 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9f880a07b0 0x7f9f880a2c70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009d20 tx=0x7f9f8c009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.991+0000 7f9fb0152b80 1 Processor -- start 2023-09-10T00:57:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.991+0000 7f9fb0152b80 1 -- start start 2023-09-10T00:57:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.991+0000 7f9fb0152b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbe5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.991+0000 7f9fb0152b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdf300 0x55bc9bbdf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9fb0152b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbdfcc0 0x55bc9bbebf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9fb0152b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bb386d0 con 0x55bc9bbe51a0 2023-09-10T00:57:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9fb0152b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bb35ff0 con 0x55bc9bbdfcc0 2023-09-10T00:57:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9fb0152b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bb318e0 con 0x55bc9bbdf300 2023-09-10T00:57:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9fa1d28700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbe5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9fa1d28700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbe5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54156/0 (socket says 172.21.15.44:54156) 2023-09-10T00:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9fa1d28700 1 -- 172.21.15.44:0/4224573246 learned_addr learned my addr 172.21.15.44:0/4224573246 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9fa1527700 1 --2- 172.21.15.44:0/4224573246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdf300 0x55bc9bbdf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9fa2529700 1 --2- 172.21.15.44:0/4224573246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbdfcc0 0x55bc9bbebf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9fa1527700 1 -- 172.21.15.44:0/4224573246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbdfcc0 msgr2=0x55bc9bbebf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9fa1527700 1 --2- 172.21.15.44:0/4224573246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbdfcc0 0x55bc9bbebf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9fa1527700 1 -- 172.21.15.44:0/4224573246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 msgr2=0x55bc9bbe5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9fa1527700 1 --2- 172.21.15.44:0/4224573246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbe5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9fa1527700 1 -- 172.21.15.44:0/4224573246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc9b95dd80 con 0x55bc9bbdf300 2023-09-10T00:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9fa1d28700 1 --2- 172.21.15.44:0/4224573246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbe5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9fa1527700 1 --2- 172.21.15.44:0/4224573246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdf300 0x55bc9bbdf780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9f98009c80 tx=0x7f9f98009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7872dd4e9874e7fc server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9f87fff700 1 -- 172.21.15.44:0/4224573246 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f98011cb0 con 0x55bc9bbdf300 2023-09-10T00:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9f87fff700 1 -- 172.21.15.44:0/4224573246 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f98011e10 con 0x55bc9bbdf300 2023-09-10T00:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.992+0000 7f9f87fff700 1 -- 172.21.15.44:0/4224573246 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f98005280 con 0x55bc9bbdf300 2023-09-10T00:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.993+0000 7f9fb0152b80 1 -- 172.21.15.44:0/4224573246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdf300 msgr2=0x55bc9bbdf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.993+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/4224573246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdf300 0x55bc9bbdf780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9f98009c80 tx=0x7f9f98009f90 comp rx=0 tx=0).stop 2023-09-10T00:57:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.993+0000 7f9fb0152b80 1 -- 172.21.15.44:0/4224573246 shutdown_connections 2023-09-10T00:57:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.993+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/4224573246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbdfcc0 0x55bc9bbebf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.993+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/4224573246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbe5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.993+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/4224573246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdf300 0x55bc9bbdf780 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.993+0000 7f9fb0152b80 1 -- 172.21.15.44:0/4224573246 >> 172.21.15.44:0/4224573246 conn(0x55bc9bb54030 msgr2=0x55bc9bbda560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.993+0000 7f9fb0152b80 1 -- 172.21.15.44:0/4224573246 shutdown_connections 2023-09-10T00:57:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.993+0000 7f9fb0152b80 1 -- 172.21.15.44:0/4224573246 wait complete. 2023-09-10T00:57:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.994+0000 7f9fb0152b80 1 -- start start 2023-09-10T00:57:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.994+0000 7f9fb0152b80 1 Processor -- start 2023-09-10T00:57:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.994+0000 7f9fb0152b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdf300 0x55bc9bbebcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.994+0000 7f9fb0152b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbdfcc0 0x55bc9bbe9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.994+0000 7f9fb0152b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbe9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.994+0000 7f9fb0152b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bacfb10 con 0x55bc9bbe51a0 2023-09-10T00:57:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.994+0000 7f9fb0152b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bacf810 con 0x55bc9bbdfcc0 2023-09-10T00:57:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.994+0000 7f9fb0152b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9b9b38c0 con 0x55bc9bbdf300 2023-09-10T00:57:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.994+0000 7f9fb0152b80 10 client.0 _ll_register_callbacks cb 0x55bc9ba19a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:57:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.994+0000 7f9fa1d28700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdf300 0x55bc9bbebcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.994+0000 7f9fa2529700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbe9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.994+0000 7f9fa1527700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbdfcc0 0x55bc9bbe9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.994+0000 7f9fa1d28700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdf300 0x55bc9bbebcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:32812/0 (socket says 172.21.15.44:32812) 2023-09-10T00:57:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.994+0000 7f9fa1d28700 1 -- 172.21.15.44:0/2939236877 learned_addr learned my addr 172.21.15.44:0/2939236877 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.995+0000 7f9fa1d28700 1 -- 172.21.15.44:0/2939236877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbdfcc0 msgr2=0x55bc9bbe9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.995+0000 7f9fa1d28700 1 --2- 172.21.15.44:0/2939236877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbdfcc0 0x55bc9bbe9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.995+0000 7f9fa1d28700 1 -- 172.21.15.44:0/2939236877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 msgr2=0x55bc9bbe9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.995+0000 7f9fa1d28700 1 --2- 172.21.15.44:0/2939236877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbe9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.995+0000 7f9fa1d28700 1 -- 172.21.15.44:0/2939236877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8c017070 con 0x55bc9bbdf300 2023-09-10T00:57:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.995+0000 7f9fa1d28700 1 --2- 172.21.15.44:0/2939236877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdf300 0x55bc9bbebcf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009820 tx=0x7f9f8c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17210e7019f1d5c0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.995+0000 7f9f857fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:57:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.995+0000 7f9f857fa700 1 -- 172.21.15.44:0/2939236877 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c022070 con 0x55bc9bbdf300 2023-09-10T00:57:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.995+0000 7f9fb0152b80 1 -- 172.21.15.44:0/2939236877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc9ba89490 con 0x55bc9bbdf300 2023-09-10T00:57:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.995+0000 7f9f857fa700 1 -- 172.21.15.44:0/2939236877 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f9f8c01db00 con 0x55bc9bbdf300 2023-09-10T00:57:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.995+0000 7f9f857fa700 1 -- 172.21.15.44:0/2939236877 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f8c01dcc0 con 0x55bc9bbdf300 2023-09-10T00:57:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.995+0000 7f9fb0152b80 10 client.27670 fetch_fsmap learned FSMap version 80 2023-09-10T00:57:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.995+0000 7f9fb0152b80 1 -- 172.21.15.44:0/2939236877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x7f9f980098e0 con 0x55bc9bbdf300 2023-09-10T00:57:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.995+0000 7f9f857fa700 1 -- 172.21.15.44:0/2939236877 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bc9ba89490 con 0x55bc9bbdf300 2023-09-10T00:57:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.996+0000 7f9f857fa700 1 -- 172.21.15.44:0/2939236877 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 80) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9f8c01c400 con 0x55bc9bbdf300 2023-09-10T00:57:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.996+0000 7f9fb0152b80 10 client.27670 fetch_fsmap finished waiting for FSMap version 80 2023-09-10T00:57:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.996+0000 7f9fb0152b80 10 client.27670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:57:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.996+0000 7f9fb0152b80 20 client.27670 populate_metadata read hostname 'smithi044' 2023-09-10T00:57:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.996+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/2939236877 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc9bbf9340 0x55bc9bbfb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.996+0000 7f9fb0152b80 4 client.27670 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:57:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.996+0000 7f9fb0152b80 1 -- 172.21.15.44:0/2939236877 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc9bb46840 con 0x55bc9bbf9340 2023-09-10T00:57:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.996+0000 7f9fa1527700 1 --2- 172.21.15.44:0/2939236877 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc9bbf9340 0x55bc9bbfb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.996+0000 7f9f857fa700 10 client.27670 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.996+0000 7f9fa1527700 1 --2- 172.21.15.44:0/2939236877 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc9bbf9340 0x55bc9bbfb760 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d07974ed69f4bb7c server_cookie=1b3818fb13000720 in_seq=0 out_seq=0 2023-09-10T00:57:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.997+0000 7f9f857fa700 1 -- 172.21.15.44:0/2939236877 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bc9bb46840 con 0x55bc9bbf9340 2023-09-10T00:57:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.997+0000 7f9f857fa700 10 client.27670 handle_command_reply: tid=0 2023-09-10T00:57:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.997+0000 7f9fb0152b80 1 client.27670 shutdown 2023-09-10T00:57:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.998+0000 7f9fb0152b80 1 -- 172.21.15.44:0/2939236877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdf300 msgr2=0x55bc9bbebcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.998+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/2939236877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdf300 0x55bc9bbebcf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009820 tx=0x7f9f8c0047c0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.998+0000 7f9fb0152b80 1 -- 172.21.15.44:0/2939236877 shutdown_connections 2023-09-10T00:57:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.998+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/2939236877 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc9bbf9340 0x55bc9bbfb760 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.998+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/2939236877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbe9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.998+0000 7f9f857fa700 0 client.27670 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.998+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/2939236877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdf300 0x55bc9bbebcf0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.998+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/2939236877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbdfcc0 0x55bc9bbe9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.998+0000 7f9fb0152b80 1 -- 172.21.15.44:0/2939236877 >> 172.21.15.44:0/2939236877 conn(0x55bc9bb54030 msgr2=0x55bc9bbd97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.998+0000 7f9fb0152b80 1 -- 172.21.15.44:0/2939236877 shutdown_connections 2023-09-10T00:57:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.998+0000 7f9fb0152b80 1 -- 172.21.15.44:0/2939236877 wait complete. 2023-09-10T00:57:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:42.998+0000 7f9fb0152b80 20 client.27670 trim_cache size 0 max 16384 2023-09-10T00:57:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.012+0000 7f9fb0152b80 1 Processor -- start 2023-09-10T00:57:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fb0152b80 1 -- start start 2023-09-10T00:57:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fb0152b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbdf300 0x55bc9bbf0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fb0152b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdfcc0 0x55bc9bbeb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fb0152b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbe1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fb0152b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bbe0580 con 0x55bc9bbdf300 2023-09-10T00:57:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fb0152b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bbea5f0 con 0x55bc9bbe51a0 2023-09-10T00:57:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fb0152b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bbf5c90 con 0x55bc9bbdfcc0 2023-09-10T00:57:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fa1d28700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbdf300 0x55bc9bbf0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fa1d28700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbdf300 0x55bc9bbf0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54170/0 (socket says 172.21.15.44:54170) 2023-09-10T00:57:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fa1d28700 1 -- 172.21.15.44:0/255611584 learned_addr learned my addr 172.21.15.44:0/255611584 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fa1527700 1 --2- 172.21.15.44:0/255611584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdfcc0 0x55bc9bbeb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fa2529700 1 --2- 172.21.15.44:0/255611584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbe1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fa1d28700 1 -- 172.21.15.44:0/255611584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdfcc0 msgr2=0x55bc9bbeb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fa1d28700 1 --2- 172.21.15.44:0/255611584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdfcc0 0x55bc9bbeb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fa1d28700 1 -- 172.21.15.44:0/255611584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbe51a0 msgr2=0x55bc9bbe1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fa1d28700 1 --2- 172.21.15.44:0/255611584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbe1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fa1d28700 1 -- 172.21.15.44:0/255611584 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc9b946c00 con 0x55bc9bbdf300 2023-09-10T00:57:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.013+0000 7f9fa1d28700 1 --2- 172.21.15.44:0/255611584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbdf300 0x55bc9bbf0520 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c000c00 tx=0x7f9f8c00e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d28d1e851512e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.014+0000 7f9f90ff9700 1 -- 172.21.15.44:0/255611584 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c022070 con 0x55bc9bbdf300 2023-09-10T00:57:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.014+0000 7f9f90ff9700 1 -- 172.21.15.44:0/255611584 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f8c01de60 con 0x55bc9bbdf300 2023-09-10T00:57:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.014+0000 7f9f90ff9700 1 -- 172.21.15.44:0/255611584 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c01cbb0 con 0x55bc9bbdf300 2023-09-10T00:57:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.014+0000 7f9fb0152b80 1 -- 172.21.15.44:0/255611584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbdf300 msgr2=0x55bc9bbf0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.014+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/255611584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbdf300 0x55bc9bbf0520 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c000c00 tx=0x7f9f8c00e430 comp rx=0 tx=0).stop 2023-09-10T00:57:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.014+0000 7f9fb0152b80 1 -- 172.21.15.44:0/255611584 shutdown_connections 2023-09-10T00:57:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.014+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/255611584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbe1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.014+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/255611584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbdf300 0x55bc9bbf0520 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.014+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/255611584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdfcc0 0x55bc9bbeb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.014+0000 7f9fb0152b80 1 -- 172.21.15.44:0/255611584 >> 172.21.15.44:0/255611584 conn(0x55bc9bb54030 msgr2=0x55bc9bbe4110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.014+0000 7f9fb0152b80 1 -- 172.21.15.44:0/255611584 shutdown_connections 2023-09-10T00:57:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.014+0000 7f9fb0152b80 1 -- 172.21.15.44:0/255611584 wait complete. 2023-09-10T00:57:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.015+0000 7f9fb0152b80 1 -- start start 2023-09-10T00:57:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.015+0000 7f9fb0152b80 1 Processor -- start 2023-09-10T00:57:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.015+0000 7f9fb0152b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbdf300 0x55bc9bbf0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.015+0000 7f9fb0152b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdfcc0 0x55bc9bbee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.015+0000 7f9fb0152b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbeea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.015+0000 7f9fb0152b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bbef030 con 0x55bc9bbe51a0 2023-09-10T00:57:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.015+0000 7f9fb0152b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bbef1a0 con 0x55bc9bbdf300 2023-09-10T00:57:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.015+0000 7f9fb0152b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bbef360 con 0x55bc9bbdfcc0 2023-09-10T00:57:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.015+0000 7f9fb0152b80 10 client.0 _ll_register_callbacks cb 0x7f9f8c00e960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:57:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.015+0000 7f9fa2529700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbeea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.015+0000 7f9fa2529700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbeea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54180/0 (socket says 172.21.15.44:54180) 2023-09-10T00:57:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.015+0000 7f9fa2529700 1 -- 172.21.15.44:0/3713105704 learned_addr learned my addr 172.21.15.44:0/3713105704 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.015+0000 7f9fa1527700 1 --2- 172.21.15.44:0/3713105704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdfcc0 0x55bc9bbee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.015+0000 7f9fa1d28700 1 --2- 172.21.15.44:0/3713105704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbdf300 0x55bc9bbf0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9fa2529700 1 -- 172.21.15.44:0/3713105704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdfcc0 msgr2=0x55bc9bbee510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9fa2529700 1 --2- 172.21.15.44:0/3713105704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdfcc0 0x55bc9bbee510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9fa2529700 1 -- 172.21.15.44:0/3713105704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbdf300 msgr2=0x55bc9bbf0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9fa2529700 1 --2- 172.21.15.44:0/3713105704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbdf300 0x55bc9bbf0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9fa2529700 1 -- 172.21.15.44:0/3713105704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f980098e0 con 0x55bc9bbe51a0 2023-09-10T00:57:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9fa2529700 1 --2- 172.21.15.44:0/3713105704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbeea50 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9f940971c0 tx=0x7f9f9409a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931e8604a5696d40 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9f857fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:57:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9f857fa700 1 -- 172.21.15.44:0/3713105704 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9409abb0 con 0x55bc9bbe51a0 2023-09-10T00:57:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9fb0152b80 1 -- 172.21.15.44:0/3713105704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f98011e10 con 0x55bc9bbe51a0 2023-09-10T00:57:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9f857fa700 1 -- 172.21.15.44:0/3713105704 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f9f940ac070 con 0x55bc9bbe51a0 2023-09-10T00:57:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9f857fa700 1 -- 172.21.15.44:0/3713105704 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f9409e580 con 0x55bc9bbe51a0 2023-09-10T00:57:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9fb0152b80 10 client.18078 fetch_fsmap learned FSMap version 80 2023-09-10T00:57:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9fb0152b80 1 -- 172.21.15.44:0/3713105704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55bc9b9841f0 con 0x55bc9bbe51a0 2023-09-10T00:57:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9f857fa700 1 -- 172.21.15.44:0/3713105704 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f940b6020 con 0x55bc9bbe51a0 2023-09-10T00:57:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9f857fa700 1 -- 172.21.15.44:0/3713105704 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 80) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9f940a22a0 con 0x55bc9bbe51a0 2023-09-10T00:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9fb0152b80 10 client.18078 fetch_fsmap finished waiting for FSMap version 80 2023-09-10T00:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9fb0152b80 10 client.18078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9fb0152b80 20 client.18078 populate_metadata read hostname 'smithi044' 2023-09-10T00:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/3713105704 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc9bc4e570 0x55bc9bc50960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9fb0152b80 4 client.18078 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.016+0000 7f9fb0152b80 1 -- 172.21.15.44:0/3713105704 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bc9bb2a8c0 con 0x55bc9bc4e570 2023-09-10T00:57:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.017+0000 7f9fa1d28700 1 --2- 172.21.15.44:0/3713105704 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc9bc4e570 0x55bc9bc50960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.017+0000 7f9f857fa700 10 client.18078 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.017+0000 7f9fa1d28700 1 --2- 172.21.15.44:0/3713105704 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc9bc4e570 0x55bc9bc50960 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8dda7b1dce660dd server_cookie=10fc57fb89f77446 in_seq=0 out_seq=0 2023-09-10T00:57:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.030+0000 7f9f857fa700 1 -- 172.21.15.44:0/3713105704 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bc9bb2a8c0 con 0x55bc9bc4e570 2023-09-10T00:57:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.030+0000 7f9f857fa700 10 client.18078 handle_command_reply: tid=0 2023-09-10T00:57:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.030+0000 7f9fb0152b80 1 client.18078 shutdown 2023-09-10T00:57:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.030+0000 7f9fb0152b80 1 -- 172.21.15.44:0/3713105704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 msgr2=0x55bc9bbeea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.030+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/3713105704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbeea50 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9f940971c0 tx=0x7f9f9409a620 comp rx=0 tx=0).stop 2023-09-10T00:57:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.030+0000 7f9fb0152b80 1 -- 172.21.15.44:0/3713105704 shutdown_connections 2023-09-10T00:57:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.030+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/3713105704 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bc9bc4e570 0x55bc9bc50960 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.030+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/3713105704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc9bbe51a0 0x55bc9bbeea50 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.030+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/3713105704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc9bbdf300 0x55bc9bbf0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.030+0000 7f9f857fa700 0 client.18078 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.030+0000 7f9fb0152b80 1 --2- 172.21.15.44:0/3713105704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc9bbdfcc0 0x55bc9bbee510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.030+0000 7f9fb0152b80 1 -- 172.21.15.44:0/3713105704 >> 172.21.15.44:0/3713105704 conn(0x55bc9bb54030 msgr2=0x55bc9bbd8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.030+0000 7f9fb0152b80 1 -- 172.21.15.44:0/3713105704 shutdown_connections 2023-09-10T00:57:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.030+0000 7f9fb0152b80 1 -- 172.21.15.44:0/3713105704 wait complete. 2023-09-10T00:57:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.030+0000 7f9fb0152b80 20 client.18078 trim_cache size 0 max 16384 2023-09-10T00:57:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:57:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:57:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:57:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:57:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.031+0000 7f9fa3f8c700 1 -- 172.21.15.44:0/2592528952 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9f880a07b0 msgr2=0x7f9f880a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.031+0000 7f9fa3f8c700 1 --2- 172.21.15.44:0/2592528952 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9f880a07b0 0x7f9f880a2c70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009d20 tx=0x7f9f8c009260 comp rx=0 tx=0).stop 2023-09-10T00:57:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.031+0000 7f9fa3f8c700 1 -- 172.21.15.44:0/2592528952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f9c106680 msgr2=0x7f9f9c1a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.031+0000 7f9fa3f8c700 1 --2- 172.21.15.44:0/2592528952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f9c106680 0x7f9f9c1a4830 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400b770 tx=0x7f9f94008e50 comp rx=0 tx=0).stop 2023-09-10T00:57:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.031+0000 7f9fa3f8c700 1 -- 172.21.15.44:0/2592528952 shutdown_connections 2023-09-10T00:57:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.031+0000 7f9fa3f8c700 1 --2- 172.21.15.44:0/2592528952 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9f880a07b0 0x7f9f880a2c70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.031+0000 7f9fa3f8c700 1 --2- 172.21.15.44:0/2592528952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f9c106680 0x7f9f9c1a4830 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.031+0000 7f9fa3f8c700 1 --2- 172.21.15.44:0/2592528952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f9c1005c0 0x7f9f9c072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.031+0000 7f9fa3f8c700 1 --2- 172.21.15.44:0/2592528952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f9c100f80 0x7f9f9c073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.031+0000 7f9fa3f8c700 1 -- 172.21.15.44:0/2592528952 >> 172.21.15.44:0/2592528952 conn(0x7f9f9c0fc0b0 msgr2=0x7f9f9c0fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.031+0000 7f9fa3f8c700 1 -- 172.21.15.44:0/2592528952 shutdown_connections 2023-09-10T00:57:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:43.031+0000 7f9fa3f8c700 1 -- 172.21.15.44:0/2592528952 wait complete. 2023-09-10T00:57:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:42 smithi099 ceph-mon[102692]: pgmap v537: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 343 op/s 2023-09-10T00:57:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:42 smithi131 ceph-mon[103713]: pgmap v537: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 343 op/s 2023-09-10T00:57:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:42 smithi044 ceph-mon[98676]: pgmap v537: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 343 op/s 2023-09-10T00:57:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:44 smithi099 ceph-mon[102692]: pgmap v538: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T00:57:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:44 smithi131 ceph-mon[103713]: pgmap v538: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T00:57:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:44 smithi044 ceph-mon[98676]: pgmap v538: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T00:57:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:46 smithi099 ceph-mon[102692]: pgmap v539: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 316 op/s 2023-09-10T00:57:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:46 smithi131 ceph-mon[103713]: pgmap v539: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 316 op/s 2023-09-10T00:57:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:46 smithi044 ceph-mon[98676]: pgmap v539: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 316 op/s 2023-09-10T00:57:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:48 smithi099 ceph-mon[102692]: pgmap v540: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 210 op/s 2023-09-10T00:57:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:48 smithi131 ceph-mon[103713]: pgmap v540: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 210 op/s 2023-09-10T00:57:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:48 smithi044 ceph-mon[98676]: pgmap v540: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 210 op/s 2023-09-10T00:57:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:50 smithi099 ceph-mon[102692]: pgmap v541: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 210 op/s 2023-09-10T00:57:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:50 smithi131 ceph-mon[103713]: pgmap v541: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 210 op/s 2023-09-10T00:57:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:50 smithi044 ceph-mon[98676]: pgmap v541: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 210 op/s 2023-09-10T00:57:53.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:52 smithi099 ceph-mon[102692]: pgmap v542: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 304 op/s 2023-09-10T00:57:54.675 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:52 smithi044 ceph-mon[98676]: pgmap v542: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 304 op/s 2023-09-10T00:57:54.675 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:52 smithi131 ceph-mon[103713]: pgmap v542: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 304 op/s 2023-09-10T00:57:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:54 smithi099 ceph-mon[102692]: pgmap v543: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2023-09-10T00:57:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:54 smithi131 ceph-mon[103713]: pgmap v543: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2023-09-10T00:57:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:54 smithi044 ceph-mon[98676]: pgmap v543: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2023-09-10T00:57:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.174+0000 7faffabef700 1 -- 172.21.15.44:0/1549586080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faff4100120 msgr2=0x7faff410c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.174+0000 7faffabef700 1 --2- 172.21.15.44:0/1549586080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faff4100120 0x7faff410c330 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fafec009fc0 tx=0x7fafec00b330 comp rx=0 tx=0).stop 2023-09-10T00:57:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.175+0000 7faffabef700 1 -- 172.21.15.44:0/1549586080 shutdown_connections 2023-09-10T00:57:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.175+0000 7faffabef700 1 --2- 172.21.15.44:0/1549586080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faff4100120 0x7faff410c330 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.175+0000 7faffabef700 1 --2- 172.21.15.44:0/1549586080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff41055b0 0x7faff4105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.175+0000 7faffabef700 1 --2- 172.21.15.44:0/1549586080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff40ff760 0x7faff40ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.175+0000 7faffabef700 1 -- 172.21.15.44:0/1549586080 >> 172.21.15.44:0/1549586080 conn(0x7faff4074f30 msgr2=0x7faff4075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.175+0000 7faffabef700 1 -- 172.21.15.44:0/1549586080 shutdown_connections 2023-09-10T00:57:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.175+0000 7faffabef700 1 -- 172.21.15.44:0/1549586080 wait complete. 2023-09-10T00:57:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.175+0000 7faffabef700 1 Processor -- start 2023-09-10T00:57:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.175+0000 7faffabef700 1 -- start start 2023-09-10T00:57:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faffabef700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faff40ff760 0x7faff419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faffabef700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff4100120 0x7faff419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faffabef700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff41055b0 0x7faff41964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faffabef700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff4196b40 con 0x7faff4100120 2023-09-10T00:57:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faffabef700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff4196cb0 con 0x7faff41055b0 2023-09-10T00:57:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faffabef700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff4196e70 con 0x7faff40ff760 2023-09-10T00:57:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faff3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff4100120 0x7faff419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faff898b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faff40ff760 0x7faff419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faff3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff4100120 0x7faff419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54192/0 (socket says 172.21.15.44:54192) 2023-09-10T00:57:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faff3fff700 1 -- 172.21.15.44:0/3680156908 learned_addr learned my addr 172.21.15.44:0/3680156908 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faff918c700 1 --2- 172.21.15.44:0/3680156908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff41055b0 0x7faff41964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faff898b700 1 -- 172.21.15.44:0/3680156908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff41055b0 msgr2=0x7faff41964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faff898b700 1 --2- 172.21.15.44:0/3680156908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff41055b0 0x7faff41964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faff898b700 1 -- 172.21.15.44:0/3680156908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff4100120 msgr2=0x7faff419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faff898b700 1 --2- 172.21.15.44:0/3680156908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff4100120 0x7faff419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faff898b700 1 -- 172.21.15.44:0/3680156908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec009c40 con 0x7faff40ff760 2023-09-10T00:57:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faff3fff700 1 --2- 172.21.15.44:0/3680156908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff4100120 0x7faff419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:57:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.176+0000 7faff898b700 1 --2- 172.21.15.44:0/3680156908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faff40ff760 0x7faff419c670 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fafe800e9c0 tx=0x7fafe800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.177+0000 7faff1ffb700 1 -- 172.21.15.44:0/3680156908 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe800c9c0 con 0x7faff40ff760 2023-09-10T00:57:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.177+0000 7faff1ffb700 1 -- 172.21.15.44:0/3680156908 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafe8010430 con 0x7faff40ff760 2023-09-10T00:57:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.177+0000 7faffabef700 1 -- 172.21.15.44:0/3680156908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff4197150 con 0x7faff40ff760 2023-09-10T00:57:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.177+0000 7faff1ffb700 1 -- 172.21.15.44:0/3680156908 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8018680 con 0x7faff40ff760 2023-09-10T00:57:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.177+0000 7faffabef700 1 -- 172.21.15.44:0/3680156908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff41a3660 con 0x7faff40ff760 2023-09-10T00:57:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.179+0000 7faff1ffb700 1 -- 172.21.15.44:0/3680156908 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fafe80105a0 con 0x7faff40ff760 2023-09-10T00:57:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.180+0000 7faff1ffb700 1 --2- 172.21.15.44:0/3680156908 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fafe40a0830 0x7fafe40a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.180+0000 7faff1ffb700 1 -- 172.21.15.44:0/3680156908 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fafe8014070 con 0x7faff40ff760 2023-09-10T00:57:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.180+0000 7faff3fff700 1 --2- 172.21.15.44:0/3680156908 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fafe40a0830 0x7fafe40a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.181+0000 7faff3fff700 1 --2- 172.21.15.44:0/3680156908 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fafe40a0830 0x7fafe40a2cf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faff4197940 tx=0x7fafe0009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.182+0000 7fb006db5b80 1 Processor -- start 2023-09-10T00:57:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.182+0000 7fb006db5b80 1 -- start start 2023-09-10T00:57:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.182+0000 7fb006db5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd31270 0x55f63dd2d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.182+0000 7fb006db5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd2d840 0x55f63dd2dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.182+0000 7fb006db5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd3ff60 0x55f63dd42350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.182+0000 7fb006db5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63dc866d0 con 0x55f63dd31270 2023-09-10T00:57:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.182+0000 7fb006db5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63dc83ff0 con 0x55f63dd3ff60 2023-09-10T00:57:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.182+0000 7fb006db5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63dc7f8e0 con 0x55f63dd2d840 2023-09-10T00:57:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.182+0000 7faff898b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd31270 0x55f63dd2d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.182+0000 7faff898b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd31270 0x55f63dd2d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54198/0 (socket says 172.21.15.44:54198) 2023-09-10T00:57:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.182+0000 7faff898b700 1 -- 172.21.15.44:0/3122923010 learned_addr learned my addr 172.21.15.44:0/3122923010 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.182+0000 7faff898b700 1 -- 172.21.15.44:0/3122923010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd2d840 msgr2=0x55f63dd2dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.182+0000 7faff898b700 1 --2- 172.21.15.44:0/3122923010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd2d840 0x55f63dd2dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.182+0000 7faff898b700 1 -- 172.21.15.44:0/3122923010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd3ff60 msgr2=0x55f63dd42350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.182+0000 7faff898b700 1 --2- 172.21.15.44:0/3122923010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd3ff60 0x55f63dd42350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.182+0000 7faff898b700 1 -- 172.21.15.44:0/3122923010 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f63daabd80 con 0x55f63dd31270 2023-09-10T00:57:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.183+0000 7faff898b700 1 --2- 172.21.15.44:0/3122923010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd31270 0x55f63dd2d300 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fafe8099460 tx=0x7fafe8099010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a807f896a6cb18 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.183+0000 7fafdeffd700 1 -- 172.21.15.44:0/3122923010 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe809d120 con 0x55f63dd31270 2023-09-10T00:57:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.183+0000 7fafdeffd700 1 -- 172.21.15.44:0/3122923010 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafe8099bd0 con 0x55f63dd31270 2023-09-10T00:57:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.183+0000 7fafdeffd700 1 -- 172.21.15.44:0/3122923010 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe80cc170 con 0x55f63dd31270 2023-09-10T00:57:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.184+0000 7fb006db5b80 1 -- 172.21.15.44:0/3122923010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd31270 msgr2=0x55f63dd2d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.184+0000 7fb006db5b80 1 --2- 172.21.15.44:0/3122923010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd31270 0x55f63dd2d300 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fafe8099460 tx=0x7fafe8099010 comp rx=0 tx=0).stop 2023-09-10T00:57:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.184+0000 7fb006db5b80 1 -- 172.21.15.44:0/3122923010 shutdown_connections 2023-09-10T00:57:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.184+0000 7fb006db5b80 1 --2- 172.21.15.44:0/3122923010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd3ff60 0x55f63dd42350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.184+0000 7fb006db5b80 1 --2- 172.21.15.44:0/3122923010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd31270 0x55f63dd2d300 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.184+0000 7fb006db5b80 1 --2- 172.21.15.44:0/3122923010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd2d840 0x55f63dd2dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.184+0000 7fb006db5b80 1 -- 172.21.15.44:0/3122923010 >> 172.21.15.44:0/3122923010 conn(0x55f63dca2030 msgr2=0x55f63dd28560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.184+0000 7fb006db5b80 1 -- 172.21.15.44:0/3122923010 shutdown_connections 2023-09-10T00:57:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.184+0000 7fb006db5b80 1 -- 172.21.15.44:0/3122923010 wait complete. 2023-09-10T00:57:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.185+0000 7fb006db5b80 1 -- start start 2023-09-10T00:57:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.185+0000 7fb006db5b80 1 Processor -- start 2023-09-10T00:57:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.185+0000 7fb006db5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd2d840 0x55f63dd39420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.185+0000 7fb006db5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd31270 0x55f63dd39960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.185+0000 7fb006db5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 0x55f63dd3f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.185+0000 7fb006db5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63dc1db10 con 0x55f63dd31270 2023-09-10T00:57:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.185+0000 7fb006db5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63dc1d810 con 0x55f63dd2d840 2023-09-10T00:57:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.185+0000 7fb006db5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63db018c0 con 0x55f63dd3ff60 2023-09-10T00:57:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.185+0000 7fb006db5b80 10 client.0 _ll_register_callbacks cb 0x55f63db67a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:57:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7faff3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd31270 0x55f63dd39960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7faff918c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 0x55f63dd3f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7faff898b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd2d840 0x55f63dd39420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7faff918c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 0x55f63dd3f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:32858/0 (socket says 172.21.15.44:32858) 2023-09-10T00:57:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7faff898b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd2d840 0x55f63dd39420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55944/0 (socket says 172.21.15.44:55944) 2023-09-10T00:57:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7faff918c700 1 -- 172.21.15.44:0/4020294942 learned_addr learned my addr 172.21.15.44:0/4020294942 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7faff918c700 1 -- 172.21.15.44:0/4020294942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd2d840 msgr2=0x55f63dd39420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7faff918c700 1 --2- 172.21.15.44:0/4020294942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd2d840 0x55f63dd39420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7faff918c700 1 -- 172.21.15.44:0/4020294942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd31270 msgr2=0x55f63dd39960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7faff918c700 1 --2- 172.21.15.44:0/4020294942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd31270 0x55f63dd39960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7faff918c700 1 -- 172.21.15.44:0/4020294942 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff41a3660 con 0x55f63dd3ff60 2023-09-10T00:57:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7faff3fff700 1 --2- 172.21.15.44:0/4020294942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd31270 0x55f63dd39960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:57:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7faff898b700 1 --2- 172.21.15.44:0/4020294942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd2d840 0x55f63dd39420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:57:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7faff918c700 1 --2- 172.21.15.44:0/4020294942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 0x55f63dd3f190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fafec007900 tx=0x7fafec009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e31753ba94a0861 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7fafd3fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:57:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7fafd3fff700 1 -- 172.21.15.44:0/4020294942 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec00e040 con 0x55f63dd3ff60 2023-09-10T00:57:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7fb006db5b80 1 -- 172.21.15.44:0/4020294942 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f63dbd7490 con 0x55f63dd3ff60 2023-09-10T00:57:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.186+0000 7fafd3fff700 1 -- 172.21.15.44:0/4020294942 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fafec003a00 con 0x55f63dd3ff60 2023-09-10T00:57:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.187+0000 7fafd3fff700 1 -- 172.21.15.44:0/4020294942 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafec003bc0 con 0x55f63dd3ff60 2023-09-10T00:57:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.187+0000 7fb006db5b80 10 client.27685 fetch_fsmap learned FSMap version 80 2023-09-10T00:57:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.187+0000 7fb006db5b80 1 -- 172.21.15.44:0/4020294942 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55f63da94c00 con 0x55f63dd3ff60 2023-09-10T00:57:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.187+0000 7fafd3fff700 1 -- 172.21.15.44:0/4020294942 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f63dbd7490 con 0x55f63dd3ff60 2023-09-10T00:57:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.187+0000 7fafd3fff700 1 -- 172.21.15.44:0/4020294942 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 80) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fafec00b050 con 0x55f63dd3ff60 2023-09-10T00:57:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.187+0000 7fb006db5b80 10 client.27685 fetch_fsmap finished waiting for FSMap version 80 2023-09-10T00:57:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.187+0000 7fb006db5b80 10 client.27685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:57:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.187+0000 7fb006db5b80 20 client.27685 populate_metadata read hostname 'smithi044' 2023-09-10T00:57:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.187+0000 7fb006db5b80 1 --2- 172.21.15.44:0/4020294942 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f63dd58360 0x55f63dd5a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.187+0000 7fb006db5b80 4 client.27685 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:57:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.187+0000 7fb006db5b80 1 -- 172.21.15.44:0/4020294942 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f63dc94840 con 0x55f63dd58360 2023-09-10T00:57:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.187+0000 7faff898b700 1 --2- 172.21.15.44:0/4020294942 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f63dd58360 0x55f63dd5a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.188+0000 7fafd3fff700 10 client.27685 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.188+0000 7faff898b700 1 --2- 172.21.15.44:0/4020294942 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f63dd58360 0x55f63dd5a780 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11c3f853eab91828 server_cookie=d35279a2ca18fb5f in_seq=0 out_seq=0 2023-09-10T00:57:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.189+0000 7fafd3fff700 1 -- 172.21.15.44:0/4020294942 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f63dc94840 con 0x55f63dd58360 2023-09-10T00:57:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.189+0000 7fafd3fff700 10 client.27685 handle_command_reply: tid=0 2023-09-10T00:57:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.189+0000 7fb006db5b80 1 client.27685 shutdown 2023-09-10T00:57:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.189+0000 7fb006db5b80 1 -- 172.21.15.44:0/4020294942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 msgr2=0x55f63dd3f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.189+0000 7fb006db5b80 1 --2- 172.21.15.44:0/4020294942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 0x55f63dd3f190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fafec007900 tx=0x7fafec009510 comp rx=0 tx=0).stop 2023-09-10T00:57:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.189+0000 7fb006db5b80 1 -- 172.21.15.44:0/4020294942 shutdown_connections 2023-09-10T00:57:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.189+0000 7fb006db5b80 1 --2- 172.21.15.44:0/4020294942 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f63dd58360 0x55f63dd5a780 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.189+0000 7fb006db5b80 1 --2- 172.21.15.44:0/4020294942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 0x55f63dd3f190 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.189+0000 7fb006db5b80 1 --2- 172.21.15.44:0/4020294942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd2d840 0x55f63dd39420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.189+0000 7fb006db5b80 1 --2- 172.21.15.44:0/4020294942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd31270 0x55f63dd39960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.189+0000 7fb006db5b80 1 -- 172.21.15.44:0/4020294942 >> 172.21.15.44:0/4020294942 conn(0x55f63dca2030 msgr2=0x55f63dd277a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.189+0000 7fafd3fff700 0 client.27685 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.189+0000 7fb006db5b80 1 -- 172.21.15.44:0/4020294942 shutdown_connections 2023-09-10T00:57:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.189+0000 7fb006db5b80 1 -- 172.21.15.44:0/4020294942 wait complete. 2023-09-10T00:57:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.189+0000 7fb006db5b80 20 client.27685 trim_cache size 0 max 16384 2023-09-10T00:57:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.209+0000 7fb006db5b80 1 Processor -- start 2023-09-10T00:57:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.209+0000 7fb006db5b80 1 -- start start 2023-09-10T00:57:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.209+0000 7fb006db5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd2d840 0x55f63dd39890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.209+0000 7fb006db5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd31270 0x55f63dd3f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.209+0000 7fb006db5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 0x55f63dd320c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7fb006db5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63dd3fde0 con 0x55f63dd2d840 2023-09-10T00:57:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7fb006db5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63dd2e0e0 con 0x55f63dd31270 2023-09-10T00:57:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7fb006db5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63dd24fc0 con 0x55f63dd3ff60 2023-09-10T00:57:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7faff898b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd2d840 0x55f63dd39890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7faff898b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd2d840 0x55f63dd39890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54212/0 (socket says 172.21.15.44:54212) 2023-09-10T00:57:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7faff898b700 1 -- 172.21.15.44:0/3655353160 learned_addr learned my addr 172.21.15.44:0/3655353160 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7faff3fff700 1 --2- 172.21.15.44:0/3655353160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd31270 0x55f63dd3f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7faff918c700 1 --2- 172.21.15.44:0/3655353160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 0x55f63dd320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7faff898b700 1 -- 172.21.15.44:0/3655353160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 msgr2=0x55f63dd320c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7faff898b700 1 --2- 172.21.15.44:0/3655353160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 0x55f63dd320c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7faff898b700 1 -- 172.21.15.44:0/3655353160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd31270 msgr2=0x55f63dd3f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7faff898b700 1 --2- 172.21.15.44:0/3655353160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd31270 0x55f63dd3f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7faff898b700 1 -- 172.21.15.44:0/3655353160 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f63dad21f0 con 0x55f63dd2d840 2023-09-10T00:57:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7faff918c700 1 --2- 172.21.15.44:0/3655353160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 0x55f63dd320c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:57:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7faff3fff700 1 --2- 172.21.15.44:0/3655353160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd31270 0x55f63dd3f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:57:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7faff898b700 1 --2- 172.21.15.44:0/3655353160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd2d840 0x55f63dd39890 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fafe8099460 tx=0x7fafe800efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9670c8deb3669352 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.210+0000 7fafdf7fe700 1 -- 172.21.15.44:0/3655353160 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe809d120 con 0x55f63dd2d840 2023-09-10T00:57:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.211+0000 7fafdf7fe700 1 -- 172.21.15.44:0/3655353160 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafe80994d0 con 0x55f63dd2d840 2023-09-10T00:57:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.211+0000 7fafdf7fe700 1 -- 172.21.15.44:0/3655353160 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8020ec0 con 0x55f63dd2d840 2023-09-10T00:57:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.211+0000 7fb006db5b80 1 -- 172.21.15.44:0/3655353160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd2d840 msgr2=0x55f63dd39890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.211+0000 7fb006db5b80 1 --2- 172.21.15.44:0/3655353160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd2d840 0x55f63dd39890 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fafe8099460 tx=0x7fafe800efd0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.211+0000 7fb006db5b80 1 -- 172.21.15.44:0/3655353160 shutdown_connections 2023-09-10T00:57:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.211+0000 7fb006db5b80 1 --2- 172.21.15.44:0/3655353160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 0x55f63dd320c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.211+0000 7fb006db5b80 1 --2- 172.21.15.44:0/3655353160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd2d840 0x55f63dd39890 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.211+0000 7fb006db5b80 1 --2- 172.21.15.44:0/3655353160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd31270 0x55f63dd3f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.211+0000 7fb006db5b80 1 -- 172.21.15.44:0/3655353160 >> 172.21.15.44:0/3655353160 conn(0x55f63dca2030 msgr2=0x55f63dd319b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.211+0000 7fb006db5b80 1 -- 172.21.15.44:0/3655353160 shutdown_connections 2023-09-10T00:57:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.211+0000 7fb006db5b80 1 -- 172.21.15.44:0/3655353160 wait complete. 2023-09-10T00:57:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.212+0000 7fb006db5b80 1 -- start start 2023-09-10T00:57:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.212+0000 7fb006db5b80 1 Processor -- start 2023-09-10T00:57:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.212+0000 7fb006db5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd2d840 0x55f63dd3c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.212+0000 7fb006db5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd31270 0x55f63dd3c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.212+0000 7fb006db5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 0x55f63dda41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.212+0000 7fb006db5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63dda4700 con 0x55f63dd2d840 2023-09-10T00:57:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.212+0000 7fb006db5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63dda4870 con 0x55f63dd31270 2023-09-10T00:57:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.212+0000 7fb006db5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63dda4a30 con 0x55f63dd3ff60 2023-09-10T00:57:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.212+0000 7fb006db5b80 10 client.0 _ll_register_callbacks cb 0x7fafec009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:57:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.213+0000 7faff898b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd2d840 0x55f63dd3c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.213+0000 7faff898b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd2d840 0x55f63dd3c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54218/0 (socket says 172.21.15.44:54218) 2023-09-10T00:57:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.213+0000 7faff898b700 1 -- 172.21.15.44:0/1055210362 learned_addr learned my addr 172.21.15.44:0/1055210362 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:57:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.213+0000 7faff3fff700 1 --2- 172.21.15.44:0/1055210362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd31270 0x55f63dd3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.213+0000 7faff918c700 1 --2- 172.21.15.44:0/1055210362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 0x55f63dda41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.213+0000 7faff3fff700 1 -- 172.21.15.44:0/1055210362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 msgr2=0x55f63dda41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.213+0000 7faff3fff700 1 --2- 172.21.15.44:0/1055210362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 0x55f63dda41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.213+0000 7faff3fff700 1 -- 172.21.15.44:0/1055210362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd2d840 msgr2=0x55f63dd3c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:57:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.213+0000 7faff3fff700 1 --2- 172.21.15.44:0/1055210362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd2d840 0x55f63dd3c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.213+0000 7faff3fff700 1 -- 172.21.15.44:0/1055210362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe0017070 con 0x55f63dd31270 2023-09-10T00:57:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.213+0000 7faff898b700 1 --2- 172.21.15.44:0/1055210362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd2d840 0x55f63dd3c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:57:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.213+0000 7faff3fff700 1 --2- 172.21.15.44:0/1055210362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd31270 0x55f63dd3c680 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fafe0005bf0 tx=0x7fafe0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d4660f3d9fb9c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:57:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.213+0000 7fafd3fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:57:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.214+0000 7fafd3fff700 1 -- 172.21.15.44:0/1055210362 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe00273f0 con 0x55f63dd31270 2023-09-10T00:57:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.214+0000 7fb006db5b80 1 -- 172.21.15.44:0/1055210362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafe8099bd0 con 0x55f63dd31270 2023-09-10T00:57:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.214+0000 7fafd3fff700 1 -- 172.21.15.44:0/1055210362 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fafe0027550 con 0x55f63dd31270 2023-09-10T00:57:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.214+0000 7fafd3fff700 1 -- 172.21.15.44:0/1055210362 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafe0027710 con 0x55f63dd31270 2023-09-10T00:57:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.214+0000 7fb006db5b80 10 client.27755 fetch_fsmap learned FSMap version 80 2023-09-10T00:57:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.214+0000 7fb006db5b80 1 -- 172.21.15.44:0/1055210362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55f63daabd80 con 0x55f63dd31270 2023-09-10T00:57:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.214+0000 7fafd3fff700 1 -- 172.21.15.44:0/1055210362 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafe8099bd0 con 0x55f63dd31270 2023-09-10T00:57:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.214+0000 7fafd3fff700 1 -- 172.21.15.44:0/1055210362 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 80) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fafe001c6e0 con 0x55f63dd31270 2023-09-10T00:57:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.214+0000 7fb006db5b80 10 client.27755 fetch_fsmap finished waiting for FSMap version 80 2023-09-10T00:57:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.214+0000 7fb006db5b80 10 client.27755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:57:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.214+0000 7fb006db5b80 20 client.27755 populate_metadata read hostname 'smithi044' 2023-09-10T00:57:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.214+0000 7fb006db5b80 1 --2- 172.21.15.44:0/1055210362 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f63ddac260 0x55f63ddae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:57:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.214+0000 7fb006db5b80 4 client.27755 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:57:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.214+0000 7fb006db5b80 1 -- 172.21.15.44:0/1055210362 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f63dc788c0 con 0x55f63ddac260 2023-09-10T00:57:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.215+0000 7faff898b700 1 --2- 172.21.15.44:0/1055210362 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f63ddac260 0x55f63ddae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:57:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.215+0000 7fafd3fff700 10 client.27755 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.215+0000 7faff898b700 1 --2- 172.21.15.44:0/1055210362 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f63ddac260 0x55f63ddae680 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef6d8c4ac478b669 server_cookie=e16b17e77633d721 in_seq=0 out_seq=0 2023-09-10T00:57:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.225+0000 7fafd3fff700 1 -- 172.21.15.44:0/1055210362 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f63dc788c0 con 0x55f63ddac260 2023-09-10T00:57:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.225+0000 7fafd3fff700 10 client.27755 handle_command_reply: tid=0 2023-09-10T00:57:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.225+0000 7fb006db5b80 1 client.27755 shutdown 2023-09-10T00:57:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.225+0000 7fb006db5b80 1 -- 172.21.15.44:0/1055210362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd31270 msgr2=0x55f63dd3c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.225+0000 7fb006db5b80 1 --2- 172.21.15.44:0/1055210362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd31270 0x55f63dd3c680 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fafe0005bf0 tx=0x7fafe0004750 comp rx=0 tx=0).stop 2023-09-10T00:57:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.225+0000 7fb006db5b80 1 -- 172.21.15.44:0/1055210362 shutdown_connections 2023-09-10T00:57:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.225+0000 7fb006db5b80 1 --2- 172.21.15.44:0/1055210362 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f63ddac260 0x55f63ddae680 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.225+0000 7fb006db5b80 1 --2- 172.21.15.44:0/1055210362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f63dd3ff60 0x55f63dda41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.225+0000 7fafd3fff700 0 client.27755 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:57:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.225+0000 7fb006db5b80 1 --2- 172.21.15.44:0/1055210362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f63dd2d840 0x55f63dd3c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.225+0000 7fb006db5b80 1 --2- 172.21.15.44:0/1055210362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f63dd31270 0x55f63dd3c680 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.225+0000 7fb006db5b80 1 -- 172.21.15.44:0/1055210362 >> 172.21.15.44:0/1055210362 conn(0x55f63dca2030 msgr2=0x55f63dd32240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.225+0000 7fb006db5b80 1 -- 172.21.15.44:0/1055210362 shutdown_connections 2023-09-10T00:57:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.225+0000 7fb006db5b80 1 -- 172.21.15.44:0/1055210362 wait complete. 2023-09-10T00:57:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.225+0000 7fb006db5b80 20 client.27755 trim_cache size 0 max 16384 2023-09-10T00:57:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:57:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:57:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:57:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:57:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.226+0000 7faffabef700 1 -- 172.21.15.44:0/3680156908 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fafe40a0830 msgr2=0x7fafe40a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.226+0000 7faffabef700 1 --2- 172.21.15.44:0/3680156908 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fafe40a0830 0x7fafe40a2cf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faff4197940 tx=0x7fafe0009260 comp rx=0 tx=0).stop 2023-09-10T00:57:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.226+0000 7faffabef700 1 -- 172.21.15.44:0/3680156908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faff40ff760 msgr2=0x7faff419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:57:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.226+0000 7faffabef700 1 --2- 172.21.15.44:0/3680156908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faff40ff760 0x7faff419c670 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fafe800e9c0 tx=0x7fafe800ecd0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.226+0000 7faffabef700 1 -- 172.21.15.44:0/3680156908 shutdown_connections 2023-09-10T00:57:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.226+0000 7faffabef700 1 --2- 172.21.15.44:0/3680156908 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fafe40a0830 0x7fafe40a2cf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.226+0000 7faffabef700 1 --2- 172.21.15.44:0/3680156908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff41055b0 0x7faff41964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.226+0000 7faffabef700 1 --2- 172.21.15.44:0/3680156908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faff40ff760 0x7faff419c670 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.226+0000 7faffabef700 1 --2- 172.21.15.44:0/3680156908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff4100120 0x7faff419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:57:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.226+0000 7faffabef700 1 -- 172.21.15.44:0/3680156908 >> 172.21.15.44:0/3680156908 conn(0x7faff4074f30 msgr2=0x7faff40fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:57:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.226+0000 7faffabef700 1 -- 172.21.15.44:0/3680156908 shutdown_connections 2023-09-10T00:57:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:57:56.226+0000 7faffabef700 1 -- 172.21.15.44:0/3680156908 wait complete. 2023-09-10T00:57:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:56 smithi099 ceph-mon[102692]: pgmap v544: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 284 op/s 2023-09-10T00:57:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:56 smithi131 ceph-mon[103713]: pgmap v544: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 284 op/s 2023-09-10T00:57:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:56 smithi044 ceph-mon[98676]: pgmap v544: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 284 op/s 2023-09-10T00:57:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:57:58 smithi099 ceph-mon[102692]: pgmap v545: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2023-09-10T00:57:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:57:58 smithi131 ceph-mon[103713]: pgmap v545: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2023-09-10T00:57:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:57:58 smithi044 ceph-mon[98676]: pgmap v545: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2023-09-10T00:58:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:00 smithi099 ceph-mon[102692]: pgmap v546: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2023-09-10T00:58:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:00 smithi131 ceph-mon[103713]: pgmap v546: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2023-09-10T00:58:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:00 smithi044 ceph-mon[98676]: pgmap v546: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2023-09-10T00:58:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:02 smithi099 ceph-mon[102692]: pgmap v547: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 283 op/s 2023-09-10T00:58:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:02 smithi131 ceph-mon[103713]: pgmap v547: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 283 op/s 2023-09-10T00:58:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:02 smithi044 ceph-mon[98676]: pgmap v547: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 283 op/s 2023-09-10T00:58:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:04 smithi099 ceph-mon[102692]: pgmap v548: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2023-09-10T00:58:05.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:04 smithi099 ceph-mon[102692]: osdmap e116: 12 total, 12 up, 12 in 2023-09-10T00:58:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:04 smithi131 ceph-mon[103713]: pgmap v548: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2023-09-10T00:58:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:04 smithi131 ceph-mon[103713]: osdmap e116: 12 total, 12 up, 12 in 2023-09-10T00:58:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:04 smithi044 ceph-mon[98676]: pgmap v548: 129 pgs: 129 active+clean; 8.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2023-09-10T00:58:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:04 smithi044 ceph-mon[98676]: osdmap e116: 12 total, 12 up, 12 in 2023-09-10T00:58:07.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:06 smithi099 ceph-mon[102692]: pgmap v550: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2023-09-10T00:58:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:06 smithi131 ceph-mon[103713]: pgmap v550: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2023-09-10T00:58:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:06 smithi044 ceph-mon[98676]: pgmap v550: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2023-09-10T00:58:09.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:08 smithi131 ceph-mon[103713]: pgmap v551: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2023-09-10T00:58:09.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:08 smithi044 ceph-mon[98676]: pgmap v551: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2023-09-10T00:58:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.341+0000 7f7708662700 1 -- 172.21.15.44:0/405440953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7700106770 msgr2=0x7f7700106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.341+0000 7f7708662700 1 --2- 172.21.15.44:0/405440953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7700106770 0x7f7700106b50 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009990 tx=0x7f76fc009ca0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.341+0000 7f7708662700 1 -- 172.21.15.44:0/405440953 shutdown_connections 2023-09-10T00:58:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.341+0000 7f7708662700 1 --2- 172.21.15.44:0/405440953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7700069350 0x7f770010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.341+0000 7f7708662700 1 --2- 172.21.15.44:0/405440953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7700106770 0x7f7700106b50 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.341+0000 7f7708662700 1 --2- 172.21.15.44:0/405440953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7700068990 0x7f7700068e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.341+0000 7f7708662700 1 -- 172.21.15.44:0/405440953 >> 172.21.15.44:0/405440953 conn(0x7f7700076630 msgr2=0x7f7700076a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.341+0000 7f7708662700 1 -- 172.21.15.44:0/405440953 shutdown_connections 2023-09-10T00:58:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.341+0000 7f7708662700 1 -- 172.21.15.44:0/405440953 wait complete. 2023-09-10T00:58:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7708662700 1 Processor -- start 2023-09-10T00:58:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7708662700 1 -- start start 2023-09-10T00:58:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7708662700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7700068990 0x7f77000ff720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7708662700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7700069350 0x7f77000ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7708662700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7700106770 0x7f77001a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7708662700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77001a4e30 con 0x7f7700106770 2023-09-10T00:58:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7708662700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77001a4f70 con 0x7f7700069350 2023-09-10T00:58:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7708662700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77001a50e0 con 0x7f7700068990 2023-09-10T00:58:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7706bff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7700106770 0x7f77001a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7706bff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7700106770 0x7f77001a48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54234/0 (socket says 172.21.15.44:54234) 2023-09-10T00:58:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7706bff700 1 -- 172.21.15.44:0/2409859825 learned_addr learned my addr 172.21.15.44:0/2409859825 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f77063fe700 1 --2- 172.21.15.44:0/2409859825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7700068990 0x7f77000ff720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7705bfd700 1 --2- 172.21.15.44:0/2409859825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7700069350 0x7f77000ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7706bff700 1 -- 172.21.15.44:0/2409859825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7700068990 msgr2=0x7f77000ff720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7706bff700 1 --2- 172.21.15.44:0/2409859825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7700068990 0x7f77000ff720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7706bff700 1 -- 172.21.15.44:0/2409859825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7700069350 msgr2=0x7f77000ffc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7706bff700 1 --2- 172.21.15.44:0/2409859825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7700069350 0x7f77000ffc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7706bff700 1 -- 172.21.15.44:0/2409859825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76fc0095f0 con 0x7f7700106770 2023-09-10T00:58:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.342+0000 7f7706bff700 1 --2- 172.21.15.44:0/2409859825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7700106770 0x7f77001a48f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f76f800ef30 tx=0x7f76f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.343+0000 7f76f77fe700 1 -- 172.21.15.44:0/2409859825 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8018070 con 0x7f7700106770 2023-09-10T00:58:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.343+0000 7f76f77fe700 1 -- 172.21.15.44:0/2409859825 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76f800f040 con 0x7f7700106770 2023-09-10T00:58:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.343+0000 7f76f77fe700 1 -- 172.21.15.44:0/2409859825 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f800cbc0 con 0x7f7700106770 2023-09-10T00:58:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.343+0000 7f7708662700 1 -- 172.21.15.44:0/2409859825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77001a5360 con 0x7f7700106770 2023-09-10T00:58:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.343+0000 7f7708662700 1 -- 172.21.15.44:0/2409859825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77001a5820 con 0x7f7700106770 2023-09-10T00:58:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.345+0000 7f76f77fe700 1 -- 172.21.15.44:0/2409859825 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f76f8007500 con 0x7f7700106770 2023-09-10T00:58:09.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:08 smithi099 ceph-mon[102692]: pgmap v551: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2023-09-10T00:58:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.345+0000 7f76f77fe700 1 --2- 172.21.15.44:0/2409859825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76ec0a0830 0x7f76ec0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.345+0000 7f76f77fe700 1 -- 172.21.15.44:0/2409859825 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f76f80cbde0 con 0x7f7700106770 2023-09-10T00:58:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.345+0000 7f77063fe700 1 --2- 172.21.15.44:0/2409859825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76ec0a0830 0x7f76ec0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.346+0000 7f77063fe700 1 --2- 172.21.15.44:0/2409859825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76ec0a0830 0x7f76ec0a2cf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f76fc006010 tx=0x7f76fc005960 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.346+0000 7f7714828b80 1 Processor -- start 2023-09-10T00:58:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7714828b80 1 -- start start 2023-09-10T00:58:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7714828b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a916e1a0 0x5555a916e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7714828b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168300 0x5555a9168780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7714828b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 0x5555a9174f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7714828b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a90c16d0 con 0x5555a9168300 2023-09-10T00:58:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7714828b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a90beff0 con 0x5555a9168cc0 2023-09-10T00:58:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7714828b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a90ba8e0 con 0x5555a916e1a0 2023-09-10T00:58:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7705bfd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168300 0x5555a9168780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f77063fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a916e1a0 0x5555a916e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7706bff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 0x5555a9174f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f77063fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a916e1a0 0x5555a916e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:32888/0 (socket says 172.21.15.44:32888) 2023-09-10T00:58:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f77063fe700 1 -- 172.21.15.44:0/1840832531 learned_addr learned my addr 172.21.15.44:0/1840832531 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7705bfd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168300 0x5555a9168780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54240/0 (socket says 172.21.15.44:54240) 2023-09-10T00:58:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7705bfd700 1 -- 172.21.15.44:0/1840832531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a916e1a0 msgr2=0x5555a916e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7705bfd700 1 --2- 172.21.15.44:0/1840832531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a916e1a0 0x5555a916e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7705bfd700 1 -- 172.21.15.44:0/1840832531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 msgr2=0x5555a9174f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7705bfd700 1 --2- 172.21.15.44:0/1840832531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 0x5555a9174f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7705bfd700 1 -- 172.21.15.44:0/1840832531 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555a8ee6d80 con 0x5555a9168300 2023-09-10T00:58:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7706bff700 1 --2- 172.21.15.44:0/1840832531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 0x5555a9174f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:58:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.347+0000 7f7705bfd700 1 --2- 172.21.15.44:0/1840832531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168300 0x5555a9168780 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009c80 tx=0x7f76f0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f0cb299d945d4e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.348+0000 7f76f4ff9700 1 -- 172.21.15.44:0/1840832531 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0011cb0 con 0x5555a9168300 2023-09-10T00:58:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.348+0000 7f76f4ff9700 1 -- 172.21.15.44:0/1840832531 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76f0011e10 con 0x5555a9168300 2023-09-10T00:58:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.348+0000 7f76f4ff9700 1 -- 172.21.15.44:0/1840832531 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0005280 con 0x5555a9168300 2023-09-10T00:58:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.348+0000 7f7714828b80 1 -- 172.21.15.44:0/1840832531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168300 msgr2=0x5555a9168780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.348+0000 7f7714828b80 1 --2- 172.21.15.44:0/1840832531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168300 0x5555a9168780 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009c80 tx=0x7f76f0009f90 comp rx=0 tx=0).stop 2023-09-10T00:58:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.348+0000 7f7714828b80 1 -- 172.21.15.44:0/1840832531 shutdown_connections 2023-09-10T00:58:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.348+0000 7f7714828b80 1 --2- 172.21.15.44:0/1840832531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 0x5555a9174f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.348+0000 7f7714828b80 1 --2- 172.21.15.44:0/1840832531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a916e1a0 0x5555a916e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.348+0000 7f7714828b80 1 --2- 172.21.15.44:0/1840832531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168300 0x5555a9168780 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.348+0000 7f7714828b80 1 -- 172.21.15.44:0/1840832531 >> 172.21.15.44:0/1840832531 conn(0x5555a90dd030 msgr2=0x5555a9163560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.348+0000 7f7714828b80 1 -- 172.21.15.44:0/1840832531 shutdown_connections 2023-09-10T00:58:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.348+0000 7f7714828b80 1 -- 172.21.15.44:0/1840832531 wait complete. 2023-09-10T00:58:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.349+0000 7f7714828b80 1 -- start start 2023-09-10T00:58:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.349+0000 7f7714828b80 1 Processor -- start 2023-09-10T00:58:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.349+0000 7f7714828b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a9168300 0x5555a9174d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.349+0000 7f7714828b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 0x5555a9171ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.349+0000 7f7714828b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a916e1a0 0x5555a9172570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.349+0000 7f7714828b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a9058b10 con 0x5555a916e1a0 2023-09-10T00:58:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.349+0000 7f7714828b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a9058810 con 0x5555a9168cc0 2023-09-10T00:58:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.349+0000 7f7714828b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a8f3c8c0 con 0x5555a9168300 2023-09-10T00:58:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.349+0000 7f7714828b80 10 client.0 _ll_register_callbacks cb 0x5555a8fa2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:58:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.349+0000 7f7706bff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a916e1a0 0x5555a9172570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.349+0000 7f7706bff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a916e1a0 0x5555a9172570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54248/0 (socket says 172.21.15.44:54248) 2023-09-10T00:58:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.349+0000 7f7706bff700 1 -- 172.21.15.44:0/2442333737 learned_addr learned my addr 172.21.15.44:0/2442333737 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.349+0000 7f77063fe700 1 --2- 172.21.15.44:0/2442333737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a9168300 0x5555a9174d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.349+0000 7f7705bfd700 1 --2- 172.21.15.44:0/2442333737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 0x5555a9171ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f7706bff700 1 -- 172.21.15.44:0/2442333737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a9168300 msgr2=0x5555a9174d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f7706bff700 1 --2- 172.21.15.44:0/2442333737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a9168300 0x5555a9174d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f7706bff700 1 -- 172.21.15.44:0/2442333737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 msgr2=0x5555a9171ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f7706bff700 1 --2- 172.21.15.44:0/2442333737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 0x5555a9171ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f7706bff700 1 -- 172.21.15.44:0/2442333737 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77001a5820 con 0x5555a916e1a0 2023-09-10T00:58:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f7705bfd700 1 --2- 172.21.15.44:0/2442333737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 0x5555a9171ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:58:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f7706bff700 1 --2- 172.21.15.44:0/2442333737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a916e1a0 0x5555a9172570 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f76f8095940 tx=0x7f76f80123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2715b591e830dd98 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f76e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:58:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f76e9ffb700 1 -- 172.21.15.44:0/2442333737 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f80b1250 con 0x5555a916e1a0 2023-09-10T00:58:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f76e9ffb700 1 -- 172.21.15.44:0/2442333737 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f76f80995d0 con 0x5555a916e1a0 2023-09-10T00:58:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f7714828b80 1 -- 172.21.15.44:0/2442333737 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555a9012490 con 0x5555a916e1a0 2023-09-10T00:58:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f76e9ffb700 1 -- 172.21.15.44:0/2442333737 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76f8098ed0 con 0x5555a916e1a0 2023-09-10T00:58:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f7714828b80 10 client.18108 fetch_fsmap learned FSMap version 80 2023-09-10T00:58:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f7714828b80 1 -- 172.21.15.44:0/2442333737 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x7f76f00098e0 con 0x5555a916e1a0 2023-09-10T00:58:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f76e9ffb700 1 -- 172.21.15.44:0/2442333737 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5555a9012490 con 0x5555a916e1a0 2023-09-10T00:58:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f76e9ffb700 1 -- 172.21.15.44:0/2442333737 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 80) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f76f80a0a20 con 0x5555a916e1a0 2023-09-10T00:58:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f7714828b80 10 client.18108 fetch_fsmap finished waiting for FSMap version 80 2023-09-10T00:58:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f7714828b80 10 client.18108 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:58:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f7714828b80 20 client.18108 populate_metadata read hostname 'smithi044' 2023-09-10T00:58:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f7714828b80 1 --2- 172.21.15.44:0/2442333737 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5555a91823e0 0x5555a9184800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f7714828b80 4 client.18108 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:58:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.350+0000 7f7714828b80 1 -- 172.21.15.44:0/2442333737 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555a90cf840 con 0x5555a91823e0 2023-09-10T00:58:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.351+0000 7f77063fe700 1 --2- 172.21.15.44:0/2442333737 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5555a91823e0 0x5555a9184800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.351+0000 7f76e9ffb700 10 client.18108 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:58:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.351+0000 7f77063fe700 1 --2- 172.21.15.44:0/2442333737 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5555a91823e0 0x5555a9184800 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c80399af28b2573d server_cookie=bc3d76e7dd9fe557 in_seq=0 out_seq=0 2023-09-10T00:58:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.351+0000 7f76e9ffb700 1 -- 172.21.15.44:0/2442333737 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5555a90cf840 con 0x5555a91823e0 2023-09-10T00:58:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.351+0000 7f76e9ffb700 10 client.18108 handle_command_reply: tid=0 2023-09-10T00:58:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.352+0000 7f7714828b80 1 client.18108 shutdown 2023-09-10T00:58:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.352+0000 7f7714828b80 1 -- 172.21.15.44:0/2442333737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a916e1a0 msgr2=0x5555a9172570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.352+0000 7f7714828b80 1 --2- 172.21.15.44:0/2442333737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a916e1a0 0x5555a9172570 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f76f8095940 tx=0x7f76f80123f0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.352+0000 7f7714828b80 1 -- 172.21.15.44:0/2442333737 shutdown_connections 2023-09-10T00:58:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.352+0000 7f7714828b80 1 --2- 172.21.15.44:0/2442333737 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5555a91823e0 0x5555a9184800 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.352+0000 7f7714828b80 1 --2- 172.21.15.44:0/2442333737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a916e1a0 0x5555a9172570 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.352+0000 7f7714828b80 1 --2- 172.21.15.44:0/2442333737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a9168300 0x5555a9174d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.352+0000 7f76e9ffb700 0 client.18108 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:58:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.352+0000 7f7714828b80 1 --2- 172.21.15.44:0/2442333737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 0x5555a9171ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.352+0000 7f7714828b80 1 -- 172.21.15.44:0/2442333737 >> 172.21.15.44:0/2442333737 conn(0x5555a90dd030 msgr2=0x5555a91627a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.352+0000 7f7714828b80 1 -- 172.21.15.44:0/2442333737 shutdown_connections 2023-09-10T00:58:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.352+0000 7f7714828b80 1 -- 172.21.15.44:0/2442333737 wait complete. 2023-09-10T00:58:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.352+0000 7f7714828b80 20 client.18108 trim_cache size 0 max 16384 2023-09-10T00:58:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7714828b80 1 Processor -- start 2023-09-10T00:58:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7714828b80 1 -- start start 2023-09-10T00:58:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7714828b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a9168300 0x5555a916d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7714828b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168cc0 0x5555a9179520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7714828b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a916e1a0 0x5555a916ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7714828b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a9164e60 con 0x5555a9168cc0 2023-09-10T00:58:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7714828b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a90cf840 con 0x5555a916e1a0 2023-09-10T00:58:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7714828b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a9012490 con 0x5555a9168300 2023-09-10T00:58:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7705bfd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168cc0 0x5555a9179520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7705bfd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168cc0 0x5555a9179520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54254/0 (socket says 172.21.15.44:54254) 2023-09-10T00:58:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7705bfd700 1 -- 172.21.15.44:0/4218618436 learned_addr learned my addr 172.21.15.44:0/4218618436 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f77063fe700 1 --2- 172.21.15.44:0/4218618436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a9168300 0x5555a916d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7706bff700 1 --2- 172.21.15.44:0/4218618436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a916e1a0 0x5555a916ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7705bfd700 1 -- 172.21.15.44:0/4218618436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a9168300 msgr2=0x5555a916d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7705bfd700 1 --2- 172.21.15.44:0/4218618436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a9168300 0x5555a916d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7705bfd700 1 -- 172.21.15.44:0/4218618436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a916e1a0 msgr2=0x5555a916ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7705bfd700 1 --2- 172.21.15.44:0/4218618436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a916e1a0 0x5555a916ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.367+0000 7f7705bfd700 1 -- 172.21.15.44:0/4218618436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555a8ecfc00 con 0x5555a9168cc0 2023-09-10T00:58:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.368+0000 7f7705bfd700 1 --2- 172.21.15.44:0/4218618436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168cc0 0x5555a9179520 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f76f0019000 tx=0x7f76f0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffb322a294b63f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.368+0000 7f76f57fa700 1 -- 172.21.15.44:0/4218618436 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0011bb0 con 0x5555a9168cc0 2023-09-10T00:58:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.368+0000 7f76f57fa700 1 -- 172.21.15.44:0/4218618436 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76f00054e0 con 0x5555a9168cc0 2023-09-10T00:58:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.368+0000 7f76f57fa700 1 -- 172.21.15.44:0/4218618436 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0003e60 con 0x5555a9168cc0 2023-09-10T00:58:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.368+0000 7f7714828b80 1 -- 172.21.15.44:0/4218618436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168cc0 msgr2=0x5555a9179520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.368+0000 7f7714828b80 1 --2- 172.21.15.44:0/4218618436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168cc0 0x5555a9179520 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f76f0019000 tx=0x7f76f0005dc0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.368+0000 7f7714828b80 1 -- 172.21.15.44:0/4218618436 shutdown_connections 2023-09-10T00:58:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.368+0000 7f7714828b80 1 --2- 172.21.15.44:0/4218618436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a916e1a0 0x5555a916ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.368+0000 7f7714828b80 1 --2- 172.21.15.44:0/4218618436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a9168300 0x5555a916d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.368+0000 7f7714828b80 1 --2- 172.21.15.44:0/4218618436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168cc0 0x5555a9179520 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.368+0000 7f7714828b80 1 -- 172.21.15.44:0/4218618436 >> 172.21.15.44:0/4218618436 conn(0x5555a90dd030 msgr2=0x5555a916d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.368+0000 7f7714828b80 1 -- 172.21.15.44:0/4218618436 shutdown_connections 2023-09-10T00:58:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.368+0000 7f7714828b80 1 -- 172.21.15.44:0/4218618436 wait complete. 2023-09-10T00:58:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.369+0000 7f7714828b80 1 -- start start 2023-09-10T00:58:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.369+0000 7f7714828b80 1 Processor -- start 2023-09-10T00:58:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.369+0000 7f7714828b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168300 0x5555a9179380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.369+0000 7f7714828b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 0x5555a91776b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.369+0000 7f7714828b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a916e1a0 0x5555a9177bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.369+0000 7f7714828b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a9178130 con 0x5555a9168300 2023-09-10T00:58:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.369+0000 7f7714828b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a91782a0 con 0x5555a9168cc0 2023-09-10T00:58:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.369+0000 7f7714828b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a9178460 con 0x5555a916e1a0 2023-09-10T00:58:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.369+0000 7f77063fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168300 0x5555a9179380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.369+0000 7f77063fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168300 0x5555a9179380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54258/0 (socket says 172.21.15.44:54258) 2023-09-10T00:58:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.369+0000 7f77063fe700 1 -- 172.21.15.44:0/2393145782 learned_addr learned my addr 172.21.15.44:0/2393145782 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.369+0000 7f7714828b80 10 client.0 _ll_register_callbacks cb 0x7f76f8098a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:58:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.369+0000 7f7706bff700 1 --2- 172.21.15.44:0/2393145782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a916e1a0 0x5555a9177bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.369+0000 7f7705bfd700 1 --2- 172.21.15.44:0/2393145782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 0x5555a91776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f77063fe700 1 -- 172.21.15.44:0/2393145782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a916e1a0 msgr2=0x5555a9177bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f77063fe700 1 --2- 172.21.15.44:0/2393145782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a916e1a0 0x5555a9177bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f77063fe700 1 -- 172.21.15.44:0/2393145782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 msgr2=0x5555a91776b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f77063fe700 1 --2- 172.21.15.44:0/2393145782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 0x5555a91776b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f77063fe700 1 -- 172.21.15.44:0/2393145782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f001d040 con 0x5555a9168300 2023-09-10T00:58:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f77063fe700 1 --2- 172.21.15.44:0/2393145782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168300 0x5555a9179380 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0055b0 tx=0x7f76fc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e3b20e8bf09680 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f76e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:58:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f76e9ffb700 1 -- 172.21.15.44:0/2393145782 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc036d10 con 0x5555a9168300 2023-09-10T00:58:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f76e9ffb700 1 -- 172.21.15.44:0/2393145782 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f76fc0360c0 con 0x5555a9168300 2023-09-10T00:58:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f7714828b80 1 -- 172.21.15.44:0/2393145782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76f0011e10 con 0x5555a9168300 2023-09-10T00:58:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f76e9ffb700 1 -- 172.21.15.44:0/2393145782 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76fc036e70 con 0x5555a9168300 2023-09-10T00:58:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f7714828b80 10 client.18114 fetch_fsmap learned FSMap version 80 2023-09-10T00:58:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f7714828b80 1 -- 172.21.15.44:0/2393145782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x5555a8f0d1f0 con 0x5555a9168300 2023-09-10T00:58:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f76e9ffb700 1 -- 172.21.15.44:0/2393145782 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76f0011e10 con 0x5555a9168300 2023-09-10T00:58:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f76e9ffb700 1 -- 172.21.15.44:0/2393145782 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 80) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f76fc004b80 con 0x5555a9168300 2023-09-10T00:58:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f7714828b80 10 client.18114 fetch_fsmap finished waiting for FSMap version 80 2023-09-10T00:58:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.370+0000 7f7714828b80 10 client.18114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:58:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.371+0000 7f7714828b80 20 client.18114 populate_metadata read hostname 'smithi044' 2023-09-10T00:58:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.371+0000 7f7714828b80 1 --2- 172.21.15.44:0/2393145782 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5555a91d77a0 0x5555a91d9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.371+0000 7f7714828b80 4 client.18114 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:58:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.371+0000 7f7714828b80 1 -- 172.21.15.44:0/2393145782 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5555a90b38c0 con 0x5555a91d77a0 2023-09-10T00:58:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.371+0000 7f7705bfd700 1 --2- 172.21.15.44:0/2393145782 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5555a91d77a0 0x5555a91d9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.371+0000 7f7705bfd700 1 --2- 172.21.15.44:0/2393145782 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5555a91d77a0 0x5555a91d9bc0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb3b5ef760c5dd78 server_cookie=acf0954a99f3059e in_seq=0 out_seq=0 2023-09-10T00:58:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.371+0000 7f76e9ffb700 10 client.18114 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:58:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.391+0000 7f76e9ffb700 1 -- 172.21.15.44:0/2393145782 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5555a90b38c0 con 0x5555a91d77a0 2023-09-10T00:58:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.391+0000 7f76e9ffb700 10 client.18114 handle_command_reply: tid=0 2023-09-10T00:58:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.391+0000 7f7714828b80 1 client.18114 shutdown 2023-09-10T00:58:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.391+0000 7f7714828b80 1 -- 172.21.15.44:0/2393145782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168300 msgr2=0x5555a9179380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.391+0000 7f7714828b80 1 --2- 172.21.15.44:0/2393145782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168300 0x5555a9179380 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0055b0 tx=0x7f76fc009ca0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.391+0000 7f7714828b80 1 -- 172.21.15.44:0/2393145782 shutdown_connections 2023-09-10T00:58:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.391+0000 7f7714828b80 1 --2- 172.21.15.44:0/2393145782 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5555a91d77a0 0x5555a91d9bc0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.391+0000 7f7714828b80 1 --2- 172.21.15.44:0/2393145782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555a916e1a0 0x5555a9177bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.391+0000 7f7714828b80 1 --2- 172.21.15.44:0/2393145782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555a9168300 0x5555a9179380 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.391+0000 7f76e9ffb700 0 client.18114 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:58:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.391+0000 7f7714828b80 1 --2- 172.21.15.44:0/2393145782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555a9168cc0 0x5555a91776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.391+0000 7f7714828b80 1 -- 172.21.15.44:0/2393145782 >> 172.21.15.44:0/2393145782 conn(0x5555a90dd030 msgr2=0x5555a916d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.391+0000 7f7714828b80 1 -- 172.21.15.44:0/2393145782 shutdown_connections 2023-09-10T00:58:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.391+0000 7f7714828b80 1 -- 172.21.15.44:0/2393145782 wait complete. 2023-09-10T00:58:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.392+0000 7f7714828b80 20 client.18114 trim_cache size 0 max 16384 2023-09-10T00:58:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:58:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:58:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:58:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:58:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.392+0000 7f7708662700 1 -- 172.21.15.44:0/2409859825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76ec0a0830 msgr2=0x7f76ec0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.392+0000 7f7708662700 1 --2- 172.21.15.44:0/2409859825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76ec0a0830 0x7f76ec0a2cf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f76fc006010 tx=0x7f76fc005960 comp rx=0 tx=0).stop 2023-09-10T00:58:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.392+0000 7f7708662700 1 -- 172.21.15.44:0/2409859825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7700106770 msgr2=0x7f77001a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.392+0000 7f7708662700 1 --2- 172.21.15.44:0/2409859825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7700106770 0x7f77001a48f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f76f800ef30 tx=0x7f76f800c3a0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.392+0000 7f7708662700 1 -- 172.21.15.44:0/2409859825 shutdown_connections 2023-09-10T00:58:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.392+0000 7f7708662700 1 --2- 172.21.15.44:0/2409859825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76ec0a0830 0x7f76ec0a2cf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.392+0000 7f7708662700 1 --2- 172.21.15.44:0/2409859825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7700106770 0x7f77001a48f0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.392+0000 7f7708662700 1 --2- 172.21.15.44:0/2409859825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7700068990 0x7f77000ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.392+0000 7f7708662700 1 --2- 172.21.15.44:0/2409859825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7700069350 0x7f77000ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.392+0000 7f7708662700 1 -- 172.21.15.44:0/2409859825 >> 172.21.15.44:0/2409859825 conn(0x7f7700076630 msgr2=0x7f7700104400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.392+0000 7f7708662700 1 -- 172.21.15.44:0/2409859825 shutdown_connections 2023-09-10T00:58:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:09.392+0000 7f7708662700 1 -- 172.21.15.44:0/2409859825 wait complete. 2023-09-10T00:58:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:10 smithi131 ceph-mon[103713]: pgmap v552: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2023-09-10T00:58:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:10 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:58:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:10 smithi131 ceph-mon[103713]: osdmap e117: 12 total, 12 up, 12 in 2023-09-10T00:58:11.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:10 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:58:11.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:10 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:58:11.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:10 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:58:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:10 smithi044 ceph-mon[98676]: pgmap v552: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2023-09-10T00:58:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:10 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:58:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:10 smithi044 ceph-mon[98676]: osdmap e117: 12 total, 12 up, 12 in 2023-09-10T00:58:11.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:10 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:58:11.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:10 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:58:11.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:10 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:58:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:10 smithi099 ceph-mon[102692]: pgmap v552: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2023-09-10T00:58:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:10 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:58:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:10 smithi099 ceph-mon[102692]: osdmap e117: 12 total, 12 up, 12 in 2023-09-10T00:58:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:10 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:58:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:10 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:58:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:10 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:58:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:12 smithi131 ceph-mon[103713]: pgmap v554: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 324 op/s 2023-09-10T00:58:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:12 smithi044 ceph-mon[98676]: pgmap v554: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 324 op/s 2023-09-10T00:58:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:12 smithi099 ceph-mon[102692]: pgmap v554: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 324 op/s 2023-09-10T00:58:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:14 smithi131 ceph-mon[103713]: pgmap v555: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2023-09-10T00:58:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:14 smithi044 ceph-mon[98676]: pgmap v555: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2023-09-10T00:58:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:14 smithi099 ceph-mon[102692]: pgmap v555: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2023-09-10T00:58:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:16 smithi131 ceph-mon[103713]: pgmap v556: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2023-09-10T00:58:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:16 smithi044 ceph-mon[98676]: pgmap v556: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2023-09-10T00:58:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:16 smithi099 ceph-mon[102692]: pgmap v556: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2023-09-10T00:58:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:18 smithi131 ceph-mon[103713]: pgmap v557: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2023-09-10T00:58:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:18 smithi044 ceph-mon[98676]: pgmap v557: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2023-09-10T00:58:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:18 smithi099 ceph-mon[102692]: pgmap v557: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2023-09-10T00:58:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:20 smithi131 ceph-mon[103713]: pgmap v558: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2023-09-10T00:58:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:20 smithi044 ceph-mon[98676]: pgmap v558: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2023-09-10T00:58:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:20 smithi099 ceph-mon[102692]: pgmap v558: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2023-09-10T00:58:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.535+0000 7f611dffe700 1 -- 172.21.15.44:0/3082765327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61181055b0 msgr2=0x7f6118105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.535+0000 7f611dffe700 1 --2- 172.21.15.44:0/3082765327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61181055b0 0x7f6118105990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f610800d960 tx=0x7f610800dc70 comp rx=0 tx=0).stop 2023-09-10T00:58:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.535+0000 7f611dffe700 1 -- 172.21.15.44:0/3082765327 shutdown_connections 2023-09-10T00:58:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.535+0000 7f611dffe700 1 --2- 172.21.15.44:0/3082765327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6118100120 0x7f611810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.535+0000 7f611dffe700 1 --2- 172.21.15.44:0/3082765327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61181055b0 0x7f6118105990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.535+0000 7f611dffe700 1 --2- 172.21.15.44:0/3082765327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61180ff760 0x7f61180ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.535+0000 7f611dffe700 1 -- 172.21.15.44:0/3082765327 >> 172.21.15.44:0/3082765327 conn(0x7f6118074f30 msgr2=0x7f6118075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.535+0000 7f611dffe700 1 -- 172.21.15.44:0/3082765327 shutdown_connections 2023-09-10T00:58:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.535+0000 7f611dffe700 1 -- 172.21.15.44:0/3082765327 wait complete. 2023-09-10T00:58:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.536+0000 7f611dffe700 1 Processor -- start 2023-09-10T00:58:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.537+0000 7f611dffe700 1 -- start start 2023-09-10T00:58:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.537+0000 7f611dffe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61180ff760 0x7f6118194400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.537+0000 7f611dffe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6118100120 0x7f6118194940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.537+0000 7f611dffe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61181055b0 0x7f6118198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.537+0000 7f611dffe700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61181995f0 con 0x7f61180ff760 2023-09-10T00:58:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.537+0000 7f611dffe700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6118199760 con 0x7f6118100120 2023-09-10T00:58:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.537+0000 7f611dffe700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6118199920 con 0x7f61181055b0 2023-09-10T00:58:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.537+0000 7f6117fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61181055b0 0x7f6118198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.537+0000 7f6117fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61181055b0 0x7f6118198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:32924/0 (socket says 172.21.15.44:32924) 2023-09-10T00:58:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.537+0000 7f6117fff700 1 -- 172.21.15.44:0/2419015400 learned_addr learned my addr 172.21.15.44:0/2419015400 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.537+0000 7f6117fff700 1 -- 172.21.15.44:0/2419015400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6118100120 msgr2=0x7f6118194940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T00:58:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.537+0000 7f6117fff700 1 --2- 172.21.15.44:0/2419015400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6118100120 0x7f6118194940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.537+0000 7f6117fff700 1 -- 172.21.15.44:0/2419015400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61180ff760 msgr2=0x7f6118194400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.537+0000 7f6117fff700 1 --2- 172.21.15.44:0/2419015400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61180ff760 0x7f6118194400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.537+0000 7f6117fff700 1 -- 172.21.15.44:0/2419015400 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f610c0095f0 con 0x7f61181055b0 2023-09-10T00:58:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.538+0000 7f6117fff700 1 --2- 172.21.15.44:0/2419015400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61181055b0 0x7f6118198fb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f611000c370 tx=0x7f611000c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.538+0000 7f6114ff9700 1 -- 172.21.15.44:0/2419015400 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110017070 con 0x7f61181055b0 2023-09-10T00:58:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.538+0000 7f611dffe700 1 -- 172.21.15.44:0/2419015400 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6110009c40 con 0x7f61181055b0 2023-09-10T00:58:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.538+0000 7f6114ff9700 1 -- 172.21.15.44:0/2419015400 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f611000f040 con 0x7f61181055b0 2023-09-10T00:58:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.538+0000 7f6114ff9700 1 -- 172.21.15.44:0/2419015400 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611000e050 con 0x7f61181055b0 2023-09-10T00:58:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.538+0000 7f611dffe700 1 -- 172.21.15.44:0/2419015400 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f610800d590 con 0x7f61181055b0 2023-09-10T00:58:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.541+0000 7f6114ff9700 1 -- 172.21.15.44:0/2419015400 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f6110003a00 con 0x7f61181055b0 2023-09-10T00:58:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.542+0000 7f6114ff9700 1 --2- 172.21.15.44:0/2419015400 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f61000a0830 0x7f61000a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.542+0000 7f61177fe700 1 --2- 172.21.15.44:0/2419015400 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f61000a0830 0x7f61000a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.542+0000 7f6114ff9700 1 -- 172.21.15.44:0/2419015400 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f61100cbd10 con 0x7f61181055b0 2023-09-10T00:58:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.543+0000 7f612a1c4b80 1 Processor -- start 2023-09-10T00:58:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.543+0000 7f612a1c4b80 1 -- start start 2023-09-10T00:58:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.543+0000 7f61177fe700 1 --2- 172.21.15.44:0/2419015400 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f61000a0830 0x7f61000a2cf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f610800cfd0 tx=0x7f610800d520 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.543+0000 7f612a1c4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd5081a0 0x557fcd508580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.543+0000 7f612a1c4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 0x557fcd502780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.543+0000 7f612a1c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502cc0 0x557fcd50ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.543+0000 7f612a1c4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd45b6d0 con 0x557fcd5081a0 2023-09-10T00:58:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.543+0000 7f612a1c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd458ff0 con 0x557fcd502cc0 2023-09-10T00:58:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.543+0000 7f612a1c4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd4548e0 con 0x557fcd502300 2023-09-10T00:58:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.543+0000 7f61177fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd5081a0 0x557fcd508580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.543+0000 7f61177fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd5081a0 0x557fcd508580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54278/0 (socket says 172.21.15.44:54278) 2023-09-10T00:58:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.543+0000 7f61177fe700 1 -- 172.21.15.44:0/2988182406 learned_addr learned my addr 172.21.15.44:0/2988182406 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.543+0000 7f6116ffd700 1 --2- 172.21.15.44:0/2988182406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 0x557fcd502780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.543+0000 7f6117fff700 1 --2- 172.21.15.44:0/2988182406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502cc0 0x557fcd50ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.544+0000 7f61177fe700 1 -- 172.21.15.44:0/2988182406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 msgr2=0x557fcd502780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.544+0000 7f61177fe700 1 --2- 172.21.15.44:0/2988182406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 0x557fcd502780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.544+0000 7f61177fe700 1 -- 172.21.15.44:0/2988182406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502cc0 msgr2=0x557fcd50ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.544+0000 7f61177fe700 1 --2- 172.21.15.44:0/2988182406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502cc0 0x557fcd50ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.544+0000 7f61177fe700 1 -- 172.21.15.44:0/2988182406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fcd280d80 con 0x557fcd5081a0 2023-09-10T00:58:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.544+0000 7f6117fff700 1 --2- 172.21.15.44:0/2988182406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502cc0 0x557fcd50ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:58:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.544+0000 7f61177fe700 1 --2- 172.21.15.44:0/2988182406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd5081a0 0x557fcd508580 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6108017000 tx=0x7f610801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a8c050c935183 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.544+0000 7f60fdffb700 1 -- 172.21.15.44:0/2988182406 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610800cc30 con 0x557fcd5081a0 2023-09-10T00:58:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.544+0000 7f60fdffb700 1 -- 172.21.15.44:0/2988182406 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61080054e0 con 0x557fcd5081a0 2023-09-10T00:58:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.544+0000 7f60fdffb700 1 -- 172.21.15.44:0/2988182406 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6108009e50 con 0x557fcd5081a0 2023-09-10T00:58:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.545+0000 7f612a1c4b80 1 -- 172.21.15.44:0/2988182406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd5081a0 msgr2=0x557fcd508580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.545+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/2988182406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd5081a0 0x557fcd508580 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6108017000 tx=0x7f610801d000 comp rx=0 tx=0).stop 2023-09-10T00:58:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.545+0000 7f612a1c4b80 1 -- 172.21.15.44:0/2988182406 shutdown_connections 2023-09-10T00:58:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.545+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/2988182406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502cc0 0x557fcd50ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.545+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/2988182406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd5081a0 0x557fcd508580 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.545+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/2988182406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 0x557fcd502780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.545+0000 7f612a1c4b80 1 -- 172.21.15.44:0/2988182406 >> 172.21.15.44:0/2988182406 conn(0x557fcd477030 msgr2=0x557fcd4fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.545+0000 7f612a1c4b80 1 -- 172.21.15.44:0/2988182406 shutdown_connections 2023-09-10T00:58:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.545+0000 7f612a1c4b80 1 -- 172.21.15.44:0/2988182406 wait complete. 2023-09-10T00:58:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.546+0000 7f612a1c4b80 1 -- start start 2023-09-10T00:58:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.546+0000 7f612a1c4b80 1 Processor -- start 2023-09-10T00:58:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.546+0000 7f612a1c4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 0x557fcd50edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.546+0000 7f612a1c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502cc0 0x557fcd50c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.546+0000 7f612a1c4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd5081a0 0x557fcd50c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.546+0000 7f612a1c4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd3f2b10 con 0x557fcd5081a0 2023-09-10T00:58:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.546+0000 7f612a1c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd3f2810 con 0x557fcd502cc0 2023-09-10T00:58:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.546+0000 7f612a1c4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd2d68c0 con 0x557fcd502300 2023-09-10T00:58:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.546+0000 7f612a1c4b80 10 client.0 _ll_register_callbacks cb 0x557fcd33ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:58:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f61177fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 0x557fcd50edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f6117fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd5081a0 0x557fcd50c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f61177fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 0x557fcd50edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:32934/0 (socket says 172.21.15.44:32934) 2023-09-10T00:58:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f61177fe700 1 -- 172.21.15.44:0/3427159973 learned_addr learned my addr 172.21.15.44:0/3427159973 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f6116ffd700 1 --2- 172.21.15.44:0/3427159973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502cc0 0x557fcd50c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f61177fe700 1 -- 172.21.15.44:0/3427159973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502cc0 msgr2=0x557fcd50c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f61177fe700 1 --2- 172.21.15.44:0/3427159973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502cc0 0x557fcd50c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f61177fe700 1 -- 172.21.15.44:0/3427159973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd5081a0 msgr2=0x557fcd50c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f61177fe700 1 --2- 172.21.15.44:0/3427159973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd5081a0 0x557fcd50c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f61177fe700 1 -- 172.21.15.44:0/3427159973 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f610800d590 con 0x557fcd502300 2023-09-10T00:58:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f6116ffd700 1 --2- 172.21.15.44:0/3427159973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502cc0 0x557fcd50c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:58:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f61177fe700 1 --2- 172.21.15.44:0/3427159973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 0x557fcd50edd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f610800d0a0 tx=0x7f610800cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9d6e450eef6e25a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f60f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:58:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f60f6ffd700 1 -- 172.21.15.44:0/3427159973 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6108009880 con 0x557fcd502300 2023-09-10T00:58:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f60f6ffd700 1 -- 172.21.15.44:0/3427159973 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f6108025040 con 0x557fcd502300 2023-09-10T00:58:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f612a1c4b80 1 -- 172.21.15.44:0/3427159973 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fcd3ac490 con 0x557fcd502300 2023-09-10T00:58:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.547+0000 7f60f6ffd700 1 -- 172.21.15.44:0/3427159973 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6108009c60 con 0x557fcd502300 2023-09-10T00:58:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.548+0000 7f612a1c4b80 10 client.27721 fetch_fsmap learned FSMap version 80 2023-09-10T00:58:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.548+0000 7f612a1c4b80 1 -- 172.21.15.44:0/3427159973 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x7f611810cbc0 con 0x557fcd502300 2023-09-10T00:58:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.548+0000 7f60f6ffd700 1 -- 172.21.15.44:0/3427159973 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557fcd3ac490 con 0x557fcd502300 2023-09-10T00:58:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.548+0000 7f60f6ffd700 1 -- 172.21.15.44:0/3427159973 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 80) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f61080154c0 con 0x557fcd502300 2023-09-10T00:58:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.548+0000 7f612a1c4b80 10 client.27721 fetch_fsmap finished waiting for FSMap version 80 2023-09-10T00:58:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.548+0000 7f612a1c4b80 10 client.27721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:58:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.548+0000 7f612a1c4b80 20 client.27721 populate_metadata read hostname 'smithi044' 2023-09-10T00:58:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.548+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/3427159973 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557fcd512e40 0x557fcd515260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.548+0000 7f612a1c4b80 4 client.27721 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:58:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.548+0000 7f612a1c4b80 1 -- 172.21.15.44:0/3427159973 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fcd469840 con 0x557fcd512e40 2023-09-10T00:58:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.548+0000 7f6116ffd700 1 --2- 172.21.15.44:0/3427159973 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557fcd512e40 0x557fcd515260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.549+0000 7f60f6ffd700 10 client.27721 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:58:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.549+0000 7f6116ffd700 1 --2- 172.21.15.44:0/3427159973 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557fcd512e40 0x557fcd515260 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb41aedebdb671e server_cookie=aa0966fef66efe51 in_seq=0 out_seq=0 2023-09-10T00:58:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.549+0000 7f60f6ffd700 1 -- 172.21.15.44:0/3427159973 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557fcd469840 con 0x557fcd512e40 2023-09-10T00:58:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.549+0000 7f60f6ffd700 10 client.27721 handle_command_reply: tid=0 2023-09-10T00:58:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.549+0000 7f612a1c4b80 1 client.27721 shutdown 2023-09-10T00:58:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.549+0000 7f612a1c4b80 1 -- 172.21.15.44:0/3427159973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 msgr2=0x557fcd50edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.549+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/3427159973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 0x557fcd50edd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f610800d0a0 tx=0x7f610800cc30 comp rx=0 tx=0).stop 2023-09-10T00:58:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.549+0000 7f612a1c4b80 1 -- 172.21.15.44:0/3427159973 shutdown_connections 2023-09-10T00:58:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.549+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/3427159973 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557fcd512e40 0x557fcd515260 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.549+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/3427159973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd5081a0 0x557fcd50c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.549+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/3427159973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 0x557fcd50edd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.549+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/3427159973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502cc0 0x557fcd50c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.549+0000 7f60f6ffd700 0 client.27721 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:58:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.549+0000 7f612a1c4b80 1 -- 172.21.15.44:0/3427159973 >> 172.21.15.44:0/3427159973 conn(0x557fcd477030 msgr2=0x557fcd4fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.550+0000 7f612a1c4b80 1 -- 172.21.15.44:0/3427159973 shutdown_connections 2023-09-10T00:58:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.550+0000 7f612a1c4b80 1 -- 172.21.15.44:0/3427159973 wait complete. 2023-09-10T00:58:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.550+0000 7f612a1c4b80 20 client.27721 trim_cache size 0 max 16384 2023-09-10T00:58:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.568+0000 7f612a1c4b80 1 Processor -- start 2023-09-10T00:58:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.568+0000 7f612a1c4b80 1 -- start start 2023-09-10T00:58:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f612a1c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502300 0x557fcd519b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f612a1c4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd502cc0 0x557fcd50e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f612a1c4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd5081a0 0x557fcd507140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f612a1c4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd4f78a0 con 0x557fcd502cc0 2023-09-10T00:58:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f612a1c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd520ba0 con 0x557fcd502300 2023-09-10T00:58:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f612a1c4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd50d630 con 0x557fcd5081a0 2023-09-10T00:58:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f61177fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502300 0x557fcd519b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f61177fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502300 0x557fcd519b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56032/0 (socket says 172.21.15.44:56032) 2023-09-10T00:58:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f6117fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd5081a0 0x557fcd507140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f61177fe700 1 -- 172.21.15.44:0/1267369884 learned_addr learned my addr 172.21.15.44:0/1267369884 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f6116ffd700 1 --2- 172.21.15.44:0/1267369884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd502cc0 0x557fcd50e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f61177fe700 1 -- 172.21.15.44:0/1267369884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd5081a0 msgr2=0x557fcd507140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f61177fe700 1 --2- 172.21.15.44:0/1267369884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd5081a0 0x557fcd507140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f61177fe700 1 -- 172.21.15.44:0/1267369884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd502cc0 msgr2=0x557fcd50e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f61177fe700 1 --2- 172.21.15.44:0/1267369884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd502cc0 0x557fcd50e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f61177fe700 1 -- 172.21.15.44:0/1267369884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fcd269c00 con 0x557fcd502300 2023-09-10T00:58:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.569+0000 7f61177fe700 1 --2- 172.21.15.44:0/1267369884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502300 0x557fcd519b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6108015a70 tx=0x7f6108015aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a0dc033b15497d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.570+0000 7f60fe7fc700 1 -- 172.21.15.44:0/1267369884 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6108005330 con 0x557fcd502300 2023-09-10T00:58:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.570+0000 7f60fe7fc700 1 -- 172.21.15.44:0/1267369884 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6108009780 con 0x557fcd502300 2023-09-10T00:58:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.570+0000 7f60fe7fc700 1 -- 172.21.15.44:0/1267369884 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6108003e00 con 0x557fcd502300 2023-09-10T00:58:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.571+0000 7f612a1c4b80 1 -- 172.21.15.44:0/1267369884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502300 msgr2=0x557fcd519b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.571+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/1267369884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502300 0x557fcd519b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6108015a70 tx=0x7f6108015aa0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.571+0000 7f612a1c4b80 1 -- 172.21.15.44:0/1267369884 shutdown_connections 2023-09-10T00:58:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.571+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/1267369884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd5081a0 0x557fcd507140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.571+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/1267369884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd502300 0x557fcd519b90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.571+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/1267369884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd502cc0 0x557fcd50e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.571+0000 7f612a1c4b80 1 -- 172.21.15.44:0/1267369884 >> 172.21.15.44:0/1267369884 conn(0x557fcd477030 msgr2=0x557fcd5061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.571+0000 7f612a1c4b80 1 -- 172.21.15.44:0/1267369884 shutdown_connections 2023-09-10T00:58:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.571+0000 7f612a1c4b80 1 -- 172.21.15.44:0/1267369884 wait complete. 2023-09-10T00:58:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.571+0000 7f612a1c4b80 1 -- start start 2023-09-10T00:58:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.572+0000 7f612a1c4b80 1 Processor -- start 2023-09-10T00:58:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.572+0000 7f612a1c4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 0x557fcd5796a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.572+0000 7f612a1c4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd502cc0 0x557fcd579be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.572+0000 7f612a1c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd5081a0 0x557fcd518f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.572+0000 7f612a1c4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd519520 con 0x557fcd502cc0 2023-09-10T00:58:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.572+0000 7f612a1c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd519690 con 0x557fcd5081a0 2023-09-10T00:58:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.572+0000 7f612a1c4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd519850 con 0x557fcd502300 2023-09-10T00:58:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.572+0000 7f612a1c4b80 10 client.0 _ll_register_callbacks cb 0x7f6110004ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:58:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.572+0000 7f6116ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd502cc0 0x557fcd579be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.572+0000 7f6116ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd502cc0 0x557fcd579be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54298/0 (socket says 172.21.15.44:54298) 2023-09-10T00:58:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.572+0000 7f6117fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd5081a0 0x557fcd518f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.572+0000 7f61177fe700 1 --2- 172.21.15.44:0/2185326078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 0x557fcd5796a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.572+0000 7f6116ffd700 1 -- 172.21.15.44:0/2185326078 learned_addr learned my addr 172.21.15.44:0/2185326078 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.573+0000 7f6117fff700 1 -- 172.21.15.44:0/2185326078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 msgr2=0x557fcd5796a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.573+0000 7f6117fff700 1 --2- 172.21.15.44:0/2185326078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 0x557fcd5796a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.573+0000 7f6117fff700 1 -- 172.21.15.44:0/2185326078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd502cc0 msgr2=0x557fcd579be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.573+0000 7f6117fff700 1 --2- 172.21.15.44:0/2185326078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd502cc0 0x557fcd579be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.573+0000 7f6117fff700 1 -- 172.21.15.44:0/2185326078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611810cbc0 con 0x557fcd5081a0 2023-09-10T00:58:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.573+0000 7f61177fe700 1 --2- 172.21.15.44:0/2185326078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 0x557fcd5796a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:58:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.573+0000 7f6116ffd700 1 --2- 172.21.15.44:0/2185326078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd502cc0 0x557fcd579be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:58:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.573+0000 7f6117fff700 1 --2- 172.21.15.44:0/2185326078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd5081a0 0x557fcd518f40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f61100c9060 tx=0x7f6110099510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56d9205e36551680 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.573+0000 7f60f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:58:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.573+0000 7f60f6ffd700 1 -- 172.21.15.44:0/2185326078 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110099220 con 0x557fcd5081a0 2023-09-10T00:58:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.573+0000 7f60f6ffd700 1 -- 172.21.15.44:0/2185326078 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f61100a0e30 con 0x557fcd5081a0 2023-09-10T00:58:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.573+0000 7f612a1c4b80 1 -- 172.21.15.44:0/2185326078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61080054e0 con 0x557fcd5081a0 2023-09-10T00:58:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.573+0000 7f60f6ffd700 1 -- 172.21.15.44:0/2185326078 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61100b1200 con 0x557fcd5081a0 2023-09-10T00:58:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.573+0000 7f612a1c4b80 10 client.27788 fetch_fsmap learned FSMap version 80 2023-09-10T00:58:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.573+0000 7f612a1c4b80 1 -- 172.21.15.44:0/2185326078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x557fcd2a71f0 con 0x557fcd5081a0 2023-09-10T00:58:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.574+0000 7f60f6ffd700 1 -- 172.21.15.44:0/2185326078 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61080054e0 con 0x557fcd5081a0 2023-09-10T00:58:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.574+0000 7f60f6ffd700 1 -- 172.21.15.44:0/2185326078 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 80) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f61100a09a0 con 0x557fcd5081a0 2023-09-10T00:58:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.574+0000 7f612a1c4b80 10 client.27788 fetch_fsmap finished waiting for FSMap version 80 2023-09-10T00:58:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.574+0000 7f612a1c4b80 10 client.27788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:58:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.574+0000 7f612a1c4b80 20 client.27788 populate_metadata read hostname 'smithi044' 2023-09-10T00:58:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.574+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/2185326078 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557fcd5815a0 0x557fcd5839c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.574+0000 7f612a1c4b80 4 client.27788 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:58:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.574+0000 7f612a1c4b80 1 -- 172.21.15.44:0/2185326078 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557fcd44d8c0 con 0x557fcd5815a0 2023-09-10T00:58:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.574+0000 7f61177fe700 1 --2- 172.21.15.44:0/2185326078 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557fcd5815a0 0x557fcd5839c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.574+0000 7f60f6ffd700 10 client.27788 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:58:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.575+0000 7f61177fe700 1 --2- 172.21.15.44:0/2185326078 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557fcd5815a0 0x557fcd5839c0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8195b13f0f2fdef server_cookie=76ce61d8f3bc15d6 in_seq=0 out_seq=0 2023-09-10T00:58:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.590+0000 7f60f6ffd700 1 -- 172.21.15.44:0/2185326078 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557fcd44d8c0 con 0x557fcd5815a0 2023-09-10T00:58:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.590+0000 7f60f6ffd700 10 client.27788 handle_command_reply: tid=0 2023-09-10T00:58:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.590+0000 7f612a1c4b80 1 client.27788 shutdown 2023-09-10T00:58:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.590+0000 7f612a1c4b80 1 -- 172.21.15.44:0/2185326078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd5081a0 msgr2=0x557fcd518f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.590+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/2185326078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd5081a0 0x557fcd518f40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f61100c9060 tx=0x7f6110099510 comp rx=0 tx=0).stop 2023-09-10T00:58:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.591+0000 7f612a1c4b80 1 -- 172.21.15.44:0/2185326078 shutdown_connections 2023-09-10T00:58:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.591+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/2185326078 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557fcd5815a0 0x557fcd5839c0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.591+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/2185326078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557fcd5081a0 0x557fcd518f40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.591+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/2185326078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557fcd502300 0x557fcd5796a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.591+0000 7f612a1c4b80 1 --2- 172.21.15.44:0/2185326078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fcd502cc0 0x557fcd579be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.591+0000 7f60f6ffd700 0 client.27788 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:58:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.591+0000 7f612a1c4b80 1 -- 172.21.15.44:0/2185326078 >> 172.21.15.44:0/2185326078 conn(0x557fcd477030 msgr2=0x557fcd4fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.591+0000 7f612a1c4b80 1 -- 172.21.15.44:0/2185326078 shutdown_connections 2023-09-10T00:58:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.591+0000 7f612a1c4b80 1 -- 172.21.15.44:0/2185326078 wait complete. 2023-09-10T00:58:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.591+0000 7f612a1c4b80 20 client.27788 trim_cache size 0 max 16384 2023-09-10T00:58:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:58:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:58:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:58:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:58:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.594+0000 7f611dffe700 1 -- 172.21.15.44:0/2419015400 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f61000a0830 msgr2=0x7f61000a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.594+0000 7f611dffe700 1 --2- 172.21.15.44:0/2419015400 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f61000a0830 0x7f61000a2cf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f610800cfd0 tx=0x7f610800d520 comp rx=0 tx=0).stop 2023-09-10T00:58:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.594+0000 7f611dffe700 1 -- 172.21.15.44:0/2419015400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61181055b0 msgr2=0x7f6118198fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.594+0000 7f611dffe700 1 --2- 172.21.15.44:0/2419015400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61181055b0 0x7f6118198fb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f611000c370 tx=0x7f611000c730 comp rx=0 tx=0).stop 2023-09-10T00:58:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.594+0000 7f611dffe700 1 -- 172.21.15.44:0/2419015400 shutdown_connections 2023-09-10T00:58:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.594+0000 7f611dffe700 1 --2- 172.21.15.44:0/2419015400 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f61000a0830 0x7f61000a2cf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.594+0000 7f611dffe700 1 --2- 172.21.15.44:0/2419015400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61181055b0 0x7f6118198fb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.594+0000 7f611dffe700 1 --2- 172.21.15.44:0/2419015400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61180ff760 0x7f6118194400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.594+0000 7f611dffe700 1 --2- 172.21.15.44:0/2419015400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6118100120 0x7f6118194940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.594+0000 7f611dffe700 1 -- 172.21.15.44:0/2419015400 >> 172.21.15.44:0/2419015400 conn(0x7f6118074f30 msgr2=0x7f61180fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.594+0000 7f611dffe700 1 -- 172.21.15.44:0/2419015400 shutdown_connections 2023-09-10T00:58:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:22.594+0000 7f611dffe700 1 -- 172.21.15.44:0/2419015400 wait complete. 2023-09-10T00:58:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:22 smithi131 ceph-mon[103713]: pgmap v559: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 259 op/s 2023-09-10T00:58:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:22 smithi044 ceph-mon[98676]: pgmap v559: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 259 op/s 2023-09-10T00:58:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:22 smithi099 ceph-mon[102692]: pgmap v559: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 259 op/s 2023-09-10T00:58:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:24 smithi131 ceph-mon[103713]: pgmap v560: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 227 op/s 2023-09-10T00:58:25.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:24 smithi044 ceph-mon[98676]: pgmap v560: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 227 op/s 2023-09-10T00:58:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:24 smithi099 ceph-mon[102692]: pgmap v560: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 227 op/s 2023-09-10T00:58:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:26 smithi131 ceph-mon[103713]: pgmap v561: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 325 op/s 2023-09-10T00:58:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:26 smithi044 ceph-mon[98676]: pgmap v561: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 325 op/s 2023-09-10T00:58:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:26 smithi099 ceph-mon[102692]: pgmap v561: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 325 op/s 2023-09-10T00:58:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:28 smithi131 ceph-mon[103713]: pgmap v562: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 209 op/s 2023-09-10T00:58:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:28 smithi044 ceph-mon[98676]: pgmap v562: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 209 op/s 2023-09-10T00:58:29.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:28 smithi099 ceph-mon[102692]: pgmap v562: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 209 op/s 2023-09-10T00:58:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:30 smithi131 ceph-mon[103713]: pgmap v563: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 209 op/s 2023-09-10T00:58:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:30 smithi044 ceph-mon[98676]: pgmap v563: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 209 op/s 2023-09-10T00:58:31.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:30 smithi099 ceph-mon[102692]: pgmap v563: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 209 op/s 2023-09-10T00:58:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:32 smithi131 ceph-mon[103713]: pgmap v564: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 298 op/s 2023-09-10T00:58:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:32 smithi044 ceph-mon[98676]: pgmap v564: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 298 op/s 2023-09-10T00:58:33.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:32 smithi099 ceph-mon[102692]: pgmap v564: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 298 op/s 2023-09-10T00:58:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:34 smithi131 ceph-mon[103713]: pgmap v565: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2023-09-10T00:58:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:34 smithi044 ceph-mon[98676]: pgmap v565: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2023-09-10T00:58:35.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:34 smithi099 ceph-mon[102692]: pgmap v565: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2023-09-10T00:58:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.736+0000 7f3fe562d700 1 -- 172.21.15.44:0/1990101251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe01055b0 msgr2=0x7f3fe0105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.736+0000 7f3fe562d700 1 --2- 172.21.15.44:0/1990101251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe01055b0 0x7f3fe0105990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009920 tx=0x7f3fd0009c30 comp rx=0 tx=0).stop 2023-09-10T00:58:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.736+0000 7f3fe562d700 1 -- 172.21.15.44:0/1990101251 shutdown_connections 2023-09-10T00:58:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.736+0000 7f3fe562d700 1 --2- 172.21.15.44:0/1990101251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fe0100120 0x7f3fe010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.736+0000 7f3fe562d700 1 --2- 172.21.15.44:0/1990101251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe01055b0 0x7f3fe0105990 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.736+0000 7f3fe562d700 1 --2- 172.21.15.44:0/1990101251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fe00ff760 0x7f3fe00ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.736+0000 7f3fe562d700 1 -- 172.21.15.44:0/1990101251 >> 172.21.15.44:0/1990101251 conn(0x7f3fe0074f30 msgr2=0x7f3fe0075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.736+0000 7f3fe562d700 1 -- 172.21.15.44:0/1990101251 shutdown_connections 2023-09-10T00:58:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.736+0000 7f3fe562d700 1 -- 172.21.15.44:0/1990101251 wait complete. 2023-09-10T00:58:35.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.737+0000 7f3fe562d700 1 Processor -- start 2023-09-10T00:58:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.738+0000 7f3fe562d700 1 -- start start 2023-09-10T00:58:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.738+0000 7f3fe562d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fe00ff760 0x7f3fe0194400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.739+0000 7f3fe562d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fe0100120 0x7f3fe0194940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.739+0000 7f3fe562d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe01055b0 0x7f3fe0198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.739+0000 7f3fe562d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe01995f0 con 0x7f3fe0100120 2023-09-10T00:58:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.739+0000 7f3fe562d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe0199760 con 0x7f3fe00ff760 2023-09-10T00:58:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.739+0000 7f3fe562d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe0199920 con 0x7f3fe01055b0 2023-09-10T00:58:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.739+0000 7f3fdeffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fe00ff760 0x7f3fe0194400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.739+0000 7f3fdeffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fe00ff760 0x7f3fe0194400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56052/0 (socket says 172.21.15.44:56052) 2023-09-10T00:58:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.739+0000 7f3fdf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe01055b0 0x7f3fe0198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.739+0000 7f3fdeffd700 1 -- 172.21.15.44:0/1446021948 learned_addr learned my addr 172.21.15.44:0/1446021948 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.739+0000 7f3fdeffd700 1 -- 172.21.15.44:0/1446021948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe01055b0 msgr2=0x7f3fe0198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.739+0000 7f3fdeffd700 1 --2- 172.21.15.44:0/1446021948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe01055b0 0x7f3fe0198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.739+0000 7f3fdeffd700 1 -- 172.21.15.44:0/1446021948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fe0100120 msgr2=0x7f3fe0194940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T00:58:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.739+0000 7f3fdeffd700 1 --2- 172.21.15.44:0/1446021948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fe0100120 0x7f3fe0194940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.739+0000 7f3fdeffd700 1 -- 172.21.15.44:0/1446021948 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd0009520 con 0x7f3fe00ff760 2023-09-10T00:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.739+0000 7f3fdf7fe700 1 --2- 172.21.15.44:0/1446021948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe01055b0 0x7f3fe0198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.739+0000 7f3fdeffd700 1 --2- 172.21.15.44:0/1446021948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fe00ff760 0x7f3fe0194400 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009840 tx=0x7f3fd000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.740+0000 7f3fc7fff700 1 -- 172.21.15.44:0/1446021948 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd001c070 con 0x7f3fe00ff760 2023-09-10T00:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.740+0000 7f3fc7fff700 1 -- 172.21.15.44:0/1446021948 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fd0005040 con 0x7f3fe00ff760 2023-09-10T00:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.740+0000 7f3fe562d700 1 -- 172.21.15.44:0/1446021948 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe0199ba0 con 0x7f3fe00ff760 2023-09-10T00:58:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.740+0000 7f3fc7fff700 1 -- 172.21.15.44:0/1446021948 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd00038c0 con 0x7f3fe00ff760 2023-09-10T00:58:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.740+0000 7f3fe562d700 1 -- 172.21.15.44:0/1446021948 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe01a1440 con 0x7f3fe00ff760 2023-09-10T00:58:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.742+0000 7f3fc7fff700 1 -- 172.21.15.44:0/1446021948 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f3fd0020030 con 0x7f3fe00ff760 2023-09-10T00:58:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.743+0000 7f3fc7fff700 1 --2- 172.21.15.44:0/1446021948 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3fc80a0830 0x7f3fc80a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.743+0000 7f3fde7fc700 1 --2- 172.21.15.44:0/1446021948 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3fc80a0830 0x7f3fc80a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.743+0000 7f3fc7fff700 1 -- 172.21.15.44:0/1446021948 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f3fd00cd4d0 con 0x7f3fe00ff760 2023-09-10T00:58:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.743+0000 7f3fde7fc700 1 --2- 172.21.15.44:0/1446021948 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3fc80a0830 0x7f3fc80a2cf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0195c20 tx=0x7f3fd400f040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.744+0000 7f3ff17f3b80 1 Processor -- start 2023-09-10T00:58:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.744+0000 7f3ff17f3b80 1 -- start start 2023-09-10T00:58:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.744+0000 7f3ff17f3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 0x559d3c712580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3ff17f3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c70c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3ff17f3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 0x559d3c718f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3ff17f3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3c6656d0 con 0x559d3c70ccc0 2023-09-10T00:58:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3ff17f3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3c662ff0 con 0x559d3c7121a0 2023-09-10T00:58:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3ff17f3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3c65e8e0 con 0x559d3c70c300 2023-09-10T00:58:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3fdeffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 0x559d3c712580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3fdeffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 0x559d3c712580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56058/0 (socket says 172.21.15.44:56058) 2023-09-10T00:58:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3fdeffd700 1 -- 172.21.15.44:0/3709985859 learned_addr learned my addr 172.21.15.44:0/3709985859 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3fdf7fe700 1 --2- 172.21.15.44:0/3709985859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 0x559d3c718f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3fde7fc700 1 --2- 172.21.15.44:0/3709985859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c70c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3fdeffd700 1 -- 172.21.15.44:0/3709985859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 msgr2=0x559d3c70c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3fdeffd700 1 --2- 172.21.15.44:0/3709985859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c70c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3fdeffd700 1 -- 172.21.15.44:0/3709985859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 msgr2=0x559d3c718f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3fdeffd700 1 --2- 172.21.15.44:0/3709985859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 0x559d3c718f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3fdeffd700 1 -- 172.21.15.44:0/3709985859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d3c48ad80 con 0x559d3c7121a0 2023-09-10T00:58:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3fdeffd700 1 --2- 172.21.15.44:0/3709985859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 0x559d3c712580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0005e90 tx=0x7f3fd0099f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d024fec8191563ce server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3fc57fa700 1 -- 172.21.15.44:0/3709985859 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd009e440 con 0x559d3c7121a0 2023-09-10T00:58:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.745+0000 7f3fc57fa700 1 -- 172.21.15.44:0/3709985859 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fd00a25f0 con 0x559d3c7121a0 2023-09-10T00:58:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.746+0000 7f3fc57fa700 1 -- 172.21.15.44:0/3709985859 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd009e200 con 0x559d3c7121a0 2023-09-10T00:58:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.746+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/3709985859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 msgr2=0x559d3c712580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.746+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/3709985859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 0x559d3c712580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0005e90 tx=0x7f3fd0099f70 comp rx=0 tx=0).stop 2023-09-10T00:58:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.746+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/3709985859 shutdown_connections 2023-09-10T00:58:35.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.746+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/3709985859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 0x559d3c718f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.746+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/3709985859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 0x559d3c712580 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.746+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/3709985859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c70c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.746+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/3709985859 >> 172.21.15.44:0/3709985859 conn(0x559d3c681030 msgr2=0x559d3c707560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:35.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.746+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/3709985859 shutdown_connections 2023-09-10T00:58:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.746+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/3709985859 wait complete. 2023-09-10T00:58:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.747+0000 7f3ff17f3b80 1 -- start start 2023-09-10T00:58:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.747+0000 7f3ff17f3b80 1 Processor -- start 2023-09-10T00:58:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.747+0000 7f3ff17f3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c71bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.747+0000 7f3ff17f3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 0x559d3c71c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.747+0000 7f3ff17f3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 0x559d3c715e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.747+0000 7f3ff17f3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3c5fcb10 con 0x559d3c70ccc0 2023-09-10T00:58:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.747+0000 7f3ff17f3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3c5fc810 con 0x559d3c7121a0 2023-09-10T00:58:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.747+0000 7f3ff17f3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3c4e08c0 con 0x559d3c70c300 2023-09-10T00:58:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.747+0000 7f3ff17f3b80 10 client.0 _ll_register_callbacks cb 0x559d3c546a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:58:35.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.747+0000 7f3fdeffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c71bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.747+0000 7f3fde7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 0x559d3c71c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.747+0000 7f3fdeffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c71bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:32974/0 (socket says 172.21.15.44:32974) 2023-09-10T00:58:35.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.747+0000 7f3fdeffd700 1 -- 172.21.15.44:0/2280404619 learned_addr learned my addr 172.21.15.44:0/2280404619 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:35.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.748+0000 7f3fdf7fe700 1 --2- 172.21.15.44:0/2280404619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 0x559d3c715e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.748+0000 7f3fdeffd700 1 -- 172.21.15.44:0/2280404619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 msgr2=0x559d3c715e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.748+0000 7f3fdeffd700 1 --2- 172.21.15.44:0/2280404619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 0x559d3c715e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.748+0000 7f3fdeffd700 1 -- 172.21.15.44:0/2280404619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 msgr2=0x559d3c71c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.748+0000 7f3fdeffd700 1 --2- 172.21.15.44:0/2280404619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 0x559d3c71c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.748+0000 7f3fdeffd700 1 -- 172.21.15.44:0/2280404619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe01a1440 con 0x559d3c70c300 2023-09-10T00:58:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.748+0000 7f3fdf7fe700 1 --2- 172.21.15.44:0/2280404619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 0x559d3c715e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:58:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.748+0000 7f3fdeffd700 1 --2- 172.21.15.44:0/2280404619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c71bb60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0097080 tx=0x7f3fd0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37fcf2f09f7007bf server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.748+0000 7f3fc2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:58:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.748+0000 7f3fc2ff5700 1 -- 172.21.15.44:0/2280404619 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd009e120 con 0x559d3c70c300 2023-09-10T00:58:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.748+0000 7f3fc2ff5700 1 -- 172.21.15.44:0/2280404619 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f3fd00caaa0 con 0x559d3c70c300 2023-09-10T00:58:35.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.748+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/2280404619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d3c5b6490 con 0x559d3c70c300 2023-09-10T00:58:35.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.748+0000 7f3fc2ff5700 1 -- 172.21.15.44:0/2280404619 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fd009e280 con 0x559d3c70c300 2023-09-10T00:58:35.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.748+0000 7f3ff17f3b80 10 client.27739 fetch_fsmap learned FSMap version 80 2023-09-10T00:58:35.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.748+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/2280404619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x559d3c473c00 con 0x559d3c70c300 2023-09-10T00:58:35.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.749+0000 7f3fc2ff5700 1 -- 172.21.15.44:0/2280404619 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559d3c5b6490 con 0x559d3c70c300 2023-09-10T00:58:35.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.749+0000 7f3fc2ff5700 1 -- 172.21.15.44:0/2280404619 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 80) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3fd009a870 con 0x559d3c70c300 2023-09-10T00:58:35.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.749+0000 7f3ff17f3b80 10 client.27739 fetch_fsmap finished waiting for FSMap version 80 2023-09-10T00:58:35.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.749+0000 7f3ff17f3b80 10 client.27739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:58:35.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.749+0000 7f3ff17f3b80 20 client.27739 populate_metadata read hostname 'smithi044' 2023-09-10T00:58:35.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.749+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/2280404619 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559d3c725c00 0x559d3c728020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.749+0000 7f3ff17f3b80 4 client.27739 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:58:35.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.749+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/2280404619 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d3c673840 con 0x559d3c725c00 2023-09-10T00:58:35.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.749+0000 7f3fde7fc700 1 --2- 172.21.15.44:0/2280404619 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559d3c725c00 0x559d3c728020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.749+0000 7f3fc2ff5700 10 client.27739 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:58:35.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.749+0000 7f3fde7fc700 1 --2- 172.21.15.44:0/2280404619 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559d3c725c00 0x559d3c728020 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a92af91fe3f3595 server_cookie=fc5e31eba6ca0739 in_seq=0 out_seq=0 2023-09-10T00:58:35.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.750+0000 7f3fc2ff5700 1 -- 172.21.15.44:0/2280404619 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559d3c673840 con 0x559d3c725c00 2023-09-10T00:58:35.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.750+0000 7f3fc2ff5700 10 client.27739 handle_command_reply: tid=0 2023-09-10T00:58:35.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.750+0000 7f3ff17f3b80 1 client.27739 shutdown 2023-09-10T00:58:35.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.750+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/2280404619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 msgr2=0x559d3c71bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:35.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.750+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/2280404619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c71bb60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0097080 tx=0x7f3fd0009260 comp rx=0 tx=0).stop 2023-09-10T00:58:35.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.750+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/2280404619 shutdown_connections 2023-09-10T00:58:35.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.750+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/2280404619 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559d3c725c00 0x559d3c728020 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.750+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/2280404619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 0x559d3c715e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.750+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/2280404619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c71bb60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.750+0000 7f3fc2ff5700 0 client.27739 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:58:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.750+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/2280404619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 0x559d3c71c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.750+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/2280404619 >> 172.21.15.44:0/2280404619 conn(0x559d3c681030 msgr2=0x559d3c7067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.750+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/2280404619 shutdown_connections 2023-09-10T00:58:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.750+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/2280404619 wait complete. 2023-09-10T00:58:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.751+0000 7f3ff17f3b80 20 client.27739 trim_cache size 0 max 16384 2023-09-10T00:58:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.773+0000 7f3ff17f3b80 1 Processor -- start 2023-09-10T00:58:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.773+0000 7f3ff17f3b80 1 -- start start 2023-09-10T00:58:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.773+0000 7f3ff17f3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c7156c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.773+0000 7f3ff17f3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c70ccc0 0x559d3c715c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.773+0000 7f3ff17f3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c7121a0 0x559d3c710f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.773+0000 7f3ff17f3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3c7143a0 con 0x559d3c7121a0 2023-09-10T00:58:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.773+0000 7f3ff17f3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3c710b40 con 0x559d3c70ccc0 2023-09-10T00:58:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.773+0000 7f3ff17f3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3c708d10 con 0x559d3c70c300 2023-09-10T00:58:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.773+0000 7f3fdf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c7121a0 0x559d3c710f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.773+0000 7f3fdeffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c7156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.773+0000 7f3fdf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c7121a0 0x559d3c710f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54336/0 (socket says 172.21.15.44:54336) 2023-09-10T00:58:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.773+0000 7f3fde7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c70ccc0 0x559d3c715c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.773+0000 7f3fdf7fe700 1 -- 172.21.15.44:0/1543843020 learned_addr learned my addr 172.21.15.44:0/1543843020 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.773+0000 7f3fdeffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c7156c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:32982/0 (socket says 172.21.15.44:32982) 2023-09-10T00:58:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.774+0000 7f3fdeffd700 1 -- 172.21.15.44:0/1543843020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c70ccc0 msgr2=0x559d3c715c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.774+0000 7f3fdeffd700 1 --2- 172.21.15.44:0/1543843020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c70ccc0 0x559d3c715c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.774+0000 7f3fdeffd700 1 -- 172.21.15.44:0/1543843020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c7121a0 msgr2=0x559d3c710f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.774+0000 7f3fdeffd700 1 --2- 172.21.15.44:0/1543843020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c7121a0 0x559d3c710f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.774+0000 7f3fdeffd700 1 -- 172.21.15.44:0/1543843020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d3c4b11f0 con 0x559d3c70c300 2023-09-10T00:58:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.774+0000 7f3fdf7fe700 1 --2- 172.21.15.44:0/1543843020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c7121a0 0x559d3c710f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:58:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.774+0000 7f3fdeffd700 1 --2- 172.21.15.44:0/1543843020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c7156c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0011040 tx=0x7f3fd00ca840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acfc08ddf966e8cd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.774+0000 7f3fc5ffb700 1 -- 172.21.15.44:0/1543843020 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0099c20 con 0x559d3c70c300 2023-09-10T00:58:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.774+0000 7f3fc5ffb700 1 -- 172.21.15.44:0/1543843020 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fd009e120 con 0x559d3c70c300 2023-09-10T00:58:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.774+0000 7f3fc5ffb700 1 -- 172.21.15.44:0/1543843020 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd009aa00 con 0x559d3c70c300 2023-09-10T00:58:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.775+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/1543843020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 msgr2=0x559d3c7156c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.775+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/1543843020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c7156c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0011040 tx=0x7f3fd00ca840 comp rx=0 tx=0).stop 2023-09-10T00:58:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.775+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/1543843020 shutdown_connections 2023-09-10T00:58:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.775+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/1543843020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c7121a0 0x559d3c710f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.775+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/1543843020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c7156c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.775+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/1543843020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c70ccc0 0x559d3c715c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.775+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/1543843020 >> 172.21.15.44:0/1543843020 conn(0x559d3c681030 msgr2=0x559d3c707090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.775+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/1543843020 shutdown_connections 2023-09-10T00:58:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.775+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/1543843020 wait complete. 2023-09-10T00:58:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.776+0000 7f3ff17f3b80 1 -- start start 2023-09-10T00:58:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.776+0000 7f3ff17f3b80 1 Processor -- start 2023-09-10T00:58:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.776+0000 7f3ff17f3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c776190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.776+0000 7f3ff17f3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 0x559d3c7766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.776+0000 7f3ff17f3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 0x559d3c776c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.776+0000 7f3ff17f3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3c7812b0 con 0x559d3c70ccc0 2023-09-10T00:58:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.776+0000 7f3ff17f3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3c7813f0 con 0x559d3c7121a0 2023-09-10T00:58:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.776+0000 7f3ff17f3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3c7815b0 con 0x559d3c70c300 2023-09-10T00:58:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.776+0000 7f3ff17f3b80 10 client.0 _ll_register_callbacks cb 0x7f3fd8009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:58:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.776+0000 7f3fde7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 0x559d3c7766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.776+0000 7f3fde7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 0x559d3c7766d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54340/0 (socket says 172.21.15.44:54340) 2023-09-10T00:58:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.776+0000 7f3fde7fc700 1 -- 172.21.15.44:0/502499549 learned_addr learned my addr 172.21.15.44:0/502499549 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.776+0000 7f3fdeffd700 1 --2- 172.21.15.44:0/502499549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c776190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.776+0000 7f3fdf7fe700 1 --2- 172.21.15.44:0/502499549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 0x559d3c776c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.777+0000 7f3fde7fc700 1 -- 172.21.15.44:0/502499549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 msgr2=0x559d3c776190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.777+0000 7f3fde7fc700 1 --2- 172.21.15.44:0/502499549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c776190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.777+0000 7f3fde7fc700 1 -- 172.21.15.44:0/502499549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 msgr2=0x559d3c776c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.777+0000 7f3fde7fc700 1 --2- 172.21.15.44:0/502499549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 0x559d3c776c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.777+0000 7f3fde7fc700 1 -- 172.21.15.44:0/502499549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd8003a00 con 0x559d3c70ccc0 2023-09-10T00:58:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.777+0000 7f3fdeffd700 1 --2- 172.21.15.44:0/502499549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c776190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:58:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.777+0000 7f3fde7fc700 1 --2- 172.21.15.44:0/502499549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 0x559d3c7766d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400c990 tx=0x7f3fd40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f2c68489c5f699 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.777+0000 7f3fc2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:58:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.777+0000 7f3fc2ff5700 1 -- 172.21.15.44:0/502499549 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4012040 con 0x559d3c70ccc0 2023-09-10T00:58:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.777+0000 7f3fc2ff5700 1 -- 172.21.15.44:0/502499549 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f3fd402a2b0 con 0x559d3c70ccc0 2023-09-10T00:58:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.777+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/502499549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fd00a25f0 con 0x559d3c70ccc0 2023-09-10T00:58:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.777+0000 7f3fc2ff5700 1 -- 172.21.15.44:0/502499549 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fd402a470 con 0x559d3c70ccc0 2023-09-10T00:58:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.777+0000 7f3ff17f3b80 10 client.18144 fetch_fsmap learned FSMap version 80 2023-09-10T00:58:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.777+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/502499549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x559d3c473c00 con 0x559d3c70ccc0 2023-09-10T00:58:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.778+0000 7f3fc2ff5700 1 -- 172.21.15.44:0/502499549 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fd00a25f0 con 0x559d3c70ccc0 2023-09-10T00:58:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.778+0000 7f3fc2ff5700 1 -- 172.21.15.44:0/502499549 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 80) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3fd4028b30 con 0x559d3c70ccc0 2023-09-10T00:58:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.778+0000 7f3ff17f3b80 10 client.18144 fetch_fsmap finished waiting for FSMap version 80 2023-09-10T00:58:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.778+0000 7f3ff17f3b80 10 client.18144 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:58:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.778+0000 7f3ff17f3b80 20 client.18144 populate_metadata read hostname 'smithi044' 2023-09-10T00:58:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.778+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/502499549 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559d3c782dd0 0x559d3c7851f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.778+0000 7f3ff17f3b80 4 client.18144 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:58:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.778+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/502499549 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559d3c6578c0 con 0x559d3c782dd0 2023-09-10T00:58:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.778+0000 7f3fdeffd700 1 --2- 172.21.15.44:0/502499549 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559d3c782dd0 0x559d3c7851f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.779+0000 7f3fdeffd700 1 --2- 172.21.15.44:0/502499549 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559d3c782dd0 0x559d3c7851f0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a3c9f8a5b53411 server_cookie=b7b9e09c39adcc63 in_seq=0 out_seq=0 2023-09-10T00:58:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.779+0000 7f3fc2ff5700 10 client.18144 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:58:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.792+0000 7f3fc2ff5700 1 -- 172.21.15.44:0/502499549 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559d3c6578c0 con 0x559d3c782dd0 2023-09-10T00:58:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.792+0000 7f3fc2ff5700 10 client.18144 handle_command_reply: tid=0 2023-09-10T00:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.792+0000 7f3ff17f3b80 1 client.18144 shutdown 2023-09-10T00:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.792+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/502499549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 msgr2=0x559d3c7766d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.792+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/502499549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 0x559d3c7766d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400c990 tx=0x7f3fd40047c0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.792+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/502499549 shutdown_connections 2023-09-10T00:58:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.792+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/502499549 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559d3c782dd0 0x559d3c7851f0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/502499549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3c7121a0 0x559d3c776c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3fc2ff5700 0 client.18144 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/502499549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d3c70c300 0x559d3c776190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3ff17f3b80 1 --2- 172.21.15.44:0/502499549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d3c70ccc0 0x559d3c7766d0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/502499549 >> 172.21.15.44:0/502499549 conn(0x559d3c681030 msgr2=0x559d3c7156c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/502499549 shutdown_connections 2023-09-10T00:58:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3ff17f3b80 1 -- 172.21.15.44:0/502499549 wait complete. 2023-09-10T00:58:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3ff17f3b80 20 client.18144 trim_cache size 0 max 16384 2023-09-10T00:58:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:58:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:58:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:58:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:58:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3fe562d700 1 -- 172.21.15.44:0/1446021948 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3fc80a0830 msgr2=0x7f3fc80a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3fe562d700 1 --2- 172.21.15.44:0/1446021948 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3fc80a0830 0x7f3fc80a2cf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0195c20 tx=0x7f3fd400f040 comp rx=0 tx=0).stop 2023-09-10T00:58:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3fe562d700 1 -- 172.21.15.44:0/1446021948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fe00ff760 msgr2=0x7f3fe0194400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3fe562d700 1 --2- 172.21.15.44:0/1446021948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fe00ff760 0x7f3fe0194400 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009840 tx=0x7f3fd000fa00 comp rx=0 tx=0).stop 2023-09-10T00:58:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3fe562d700 1 -- 172.21.15.44:0/1446021948 shutdown_connections 2023-09-10T00:58:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3fe562d700 1 --2- 172.21.15.44:0/1446021948 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3fc80a0830 0x7f3fc80a2cf0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3fe562d700 1 --2- 172.21.15.44:0/1446021948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe01055b0 0x7f3fe0198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3fe562d700 1 --2- 172.21.15.44:0/1446021948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fe00ff760 0x7f3fe0194400 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3fe562d700 1 --2- 172.21.15.44:0/1446021948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fe0100120 0x7f3fe0194940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.793+0000 7f3fe562d700 1 -- 172.21.15.44:0/1446021948 >> 172.21.15.44:0/1446021948 conn(0x7f3fe0074f30 msgr2=0x7f3fe00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.794+0000 7f3fe562d700 1 -- 172.21.15.44:0/1446021948 shutdown_connections 2023-09-10T00:58:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:35.794+0000 7f3fe562d700 1 -- 172.21.15.44:0/1446021948 wait complete. 2023-09-10T00:58:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:36 smithi131 ceph-mon[103713]: pgmap v566: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 296 op/s 2023-09-10T00:58:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:36 smithi044 ceph-mon[98676]: pgmap v566: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 296 op/s 2023-09-10T00:58:37.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:36 smithi099 ceph-mon[102692]: pgmap v566: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 296 op/s 2023-09-10T00:58:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:38 smithi131 ceph-mon[103713]: pgmap v567: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2023-09-10T00:58:39.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:38 smithi044 ceph-mon[98676]: pgmap v567: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2023-09-10T00:58:39.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:38 smithi099 ceph-mon[102692]: pgmap v567: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2023-09-10T00:58:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:40 smithi131 ceph-mon[103713]: pgmap v568: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2023-09-10T00:58:41.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:40 smithi044 ceph-mon[98676]: pgmap v568: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2023-09-10T00:58:41.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:40 smithi099 ceph-mon[102692]: pgmap v568: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2023-09-10T00:58:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:42 smithi131 ceph-mon[103713]: pgmap v569: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 318 op/s 2023-09-10T00:58:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:42 smithi044 ceph-mon[98676]: pgmap v569: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 318 op/s 2023-09-10T00:58:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:42 smithi099 ceph-mon[102692]: pgmap v569: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 318 op/s 2023-09-10T00:58:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:44 smithi131 ceph-mon[103713]: pgmap v570: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 229 op/s 2023-09-10T00:58:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:44 smithi044 ceph-mon[98676]: pgmap v570: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 229 op/s 2023-09-10T00:58:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:44 smithi099 ceph-mon[102692]: pgmap v570: 129 pgs: 129 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 229 op/s 2023-09-10T00:58:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:46 smithi131 ceph-mon[103713]: pgmap v571: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 355 op/s 2023-09-10T00:58:47.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:46 smithi044 ceph-mon[98676]: pgmap v571: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 355 op/s 2023-09-10T00:58:47.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:46 smithi099 ceph-mon[102692]: pgmap v571: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 355 op/s 2023-09-10T00:58:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.925+0000 7f297d760700 1 -- 172.21.15.44:0/2914211014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2978105590 msgr2=0x7f2978105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.926+0000 7f297d760700 1 --2- 172.21.15.44:0/2914211014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2978105590 0x7f2978105970 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f29680099c0 tx=0x7f2968009cd0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.926+0000 7f297d760700 1 -- 172.21.15.44:0/2914211014 shutdown_connections 2023-09-10T00:58:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.926+0000 7f297d760700 1 --2- 172.21.15.44:0/2914211014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29780ffe90 0x7f297810c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.926+0000 7f297d760700 1 --2- 172.21.15.44:0/2914211014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2978105590 0x7f2978105970 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.926+0000 7f297d760700 1 --2- 172.21.15.44:0/2914211014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29780ff4d0 0x7f29780ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.926+0000 7f297d760700 1 -- 172.21.15.44:0/2914211014 >> 172.21.15.44:0/2914211014 conn(0x7f29780faf60 msgr2=0x7f29780fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.926+0000 7f297d760700 1 -- 172.21.15.44:0/2914211014 shutdown_connections 2023-09-10T00:58:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.926+0000 7f297d760700 1 -- 172.21.15.44:0/2914211014 wait complete. 2023-09-10T00:58:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.926+0000 7f297d760700 1 Processor -- start 2023-09-10T00:58:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f297d760700 1 -- start start 2023-09-10T00:58:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f297d760700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29780ff4d0 0x7f2978195350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f297d760700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29780ffe90 0x7f2978195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f297d760700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2978105590 0x7f29781a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f297d760700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2978195fe0 con 0x7f29780ff4d0 2023-09-10T00:58:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f297d760700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29781a4d30 con 0x7f29780ffe90 2023-09-10T00:58:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f297d760700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29781a4ec0 con 0x7f2978105590 2023-09-10T00:58:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f2976ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29780ff4d0 0x7f2978195350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f2976ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29780ff4d0 0x7f2978195350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54350/0 (socket says 172.21.15.44:54350) 2023-09-10T00:58:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f2976ffd700 1 -- 172.21.15.44:0/2519831206 learned_addr learned my addr 172.21.15.44:0/2519831206 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f29767fc700 1 --2- 172.21.15.44:0/2519831206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29780ffe90 0x7f2978195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f29777fe700 1 --2- 172.21.15.44:0/2519831206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2978105590 0x7f29781a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f2976ffd700 1 -- 172.21.15.44:0/2519831206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2978105590 msgr2=0x7f29781a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f2976ffd700 1 --2- 172.21.15.44:0/2519831206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2978105590 0x7f29781a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f2976ffd700 1 -- 172.21.15.44:0/2519831206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29780ffe90 msgr2=0x7f2978195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f2976ffd700 1 --2- 172.21.15.44:0/2519831206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29780ffe90 0x7f2978195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f2976ffd700 1 -- 172.21.15.44:0/2519831206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29680095f0 con 0x7f29780ff4d0 2023-09-10T00:58:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.927+0000 7f2976ffd700 1 --2- 172.21.15.44:0/2519831206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29780ff4d0 0x7f2978195350 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2968005910 tx=0x7f2968009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.928+0000 7f295ffff700 1 -- 172.21.15.44:0/2519831206 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296801d070 con 0x7f29780ff4d0 2023-09-10T00:58:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.928+0000 7f297d760700 1 -- 172.21.15.44:0/2519831206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29781a5140 con 0x7f29780ff4d0 2023-09-10T00:58:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.928+0000 7f295ffff700 1 -- 172.21.15.44:0/2519831206 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2968022470 con 0x7f29780ff4d0 2023-09-10T00:58:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.928+0000 7f295ffff700 1 -- 172.21.15.44:0/2519831206 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968017650 con 0x7f29780ff4d0 2023-09-10T00:58:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.928+0000 7f297d760700 1 -- 172.21.15.44:0/2519831206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29781a5570 con 0x7f29780ff4d0 2023-09-10T00:58:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.930+0000 7f295ffff700 1 -- 172.21.15.44:0/2519831206 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f29680225e0 con 0x7f29780ff4d0 2023-09-10T00:58:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.930+0000 7f295ffff700 1 --2- 172.21.15.44:0/2519831206 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29600a07b0 0x7f29600a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.930+0000 7f295ffff700 1 -- 172.21.15.44:0/2519831206 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f29680cd3b0 con 0x7f29780ff4d0 2023-09-10T00:58:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.930+0000 7f29767fc700 1 --2- 172.21.15.44:0/2519831206 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29600a07b0 0x7f29600a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.931+0000 7f29767fc700 1 --2- 172.21.15.44:0/2519831206 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29600a07b0 0x7f29600a2c70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f29780694d0 tx=0x7f296c009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.931+0000 7f2989926b80 1 Processor -- start 2023-09-10T00:58:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.931+0000 7f2989926b80 1 -- start start 2023-09-10T00:58:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f2989926b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132d11a0 0x5616132d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f2989926b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132cb300 0x5616132cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f2989926b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132cbcc0 0x5616132d7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f2989926b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616132246d0 con 0x5616132cbcc0 2023-09-10T00:58:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f2989926b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561613221ff0 con 0x5616132cb300 2023-09-10T00:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f2989926b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161321d8e0 con 0x5616132d11a0 2023-09-10T00:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f29777fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132cbcc0 0x5616132d7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f29767fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132cb300 0x5616132cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f2976ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132d11a0 0x5616132d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f29777fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132cbcc0 0x5616132d7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54362/0 (socket says 172.21.15.44:54362) 2023-09-10T00:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f29767fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132cb300 0x5616132cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56100/0 (socket says 172.21.15.44:56100) 2023-09-10T00:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f29777fe700 1 -- 172.21.15.44:0/1620636554 learned_addr learned my addr 172.21.15.44:0/1620636554 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f29777fe700 1 -- 172.21.15.44:0/1620636554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132d11a0 msgr2=0x5616132d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f29777fe700 1 --2- 172.21.15.44:0/1620636554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132d11a0 0x5616132d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f29777fe700 1 -- 172.21.15.44:0/1620636554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132cb300 msgr2=0x5616132cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f29777fe700 1 --2- 172.21.15.44:0/1620636554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132cb300 0x5616132cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f29777fe700 1 -- 172.21.15.44:0/1620636554 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561613049d80 con 0x5616132cbcc0 2023-09-10T00:58:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f29777fe700 1 --2- 172.21.15.44:0/1620636554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132cbcc0 0x5616132d7f30 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2970011600 tx=0x7f2970009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9663eb5a1a537bd2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f295d7fa700 1 -- 172.21.15.44:0/1620636554 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297000e040 con 0x5616132cbcc0 2023-09-10T00:58:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f295d7fa700 1 -- 172.21.15.44:0/1620636554 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f297000f040 con 0x5616132cbcc0 2023-09-10T00:58:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.932+0000 7f295d7fa700 1 -- 172.21.15.44:0/1620636554 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29700045e0 con 0x5616132cbcc0 2023-09-10T00:58:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.933+0000 7f2989926b80 1 -- 172.21.15.44:0/1620636554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132cbcc0 msgr2=0x5616132d7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.933+0000 7f2989926b80 1 --2- 172.21.15.44:0/1620636554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132cbcc0 0x5616132d7f30 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2970011600 tx=0x7f2970009920 comp rx=0 tx=0).stop 2023-09-10T00:58:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.933+0000 7f2989926b80 1 -- 172.21.15.44:0/1620636554 shutdown_connections 2023-09-10T00:58:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.933+0000 7f2989926b80 1 --2- 172.21.15.44:0/1620636554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132cbcc0 0x5616132d7f30 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.933+0000 7f2989926b80 1 --2- 172.21.15.44:0/1620636554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132d11a0 0x5616132d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.933+0000 7f2989926b80 1 --2- 172.21.15.44:0/1620636554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132cb300 0x5616132cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.933+0000 7f2989926b80 1 -- 172.21.15.44:0/1620636554 >> 172.21.15.44:0/1620636554 conn(0x561613240030 msgr2=0x5616132c6560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.933+0000 7f2989926b80 1 -- 172.21.15.44:0/1620636554 shutdown_connections 2023-09-10T00:58:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.933+0000 7f2989926b80 1 -- 172.21.15.44:0/1620636554 wait complete. 2023-09-10T00:58:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.933+0000 7f2989926b80 1 -- start start 2023-09-10T00:58:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f2989926b80 1 Processor -- start 2023-09-10T00:58:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f2989926b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132cb300 0x5616132d7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f2989926b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132cbcc0 0x5616132d50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f2989926b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132d11a0 0x5616132d5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f2989926b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616131bbb10 con 0x5616132d11a0 2023-09-10T00:58:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f2989926b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616131bb810 con 0x5616132cbcc0 2023-09-10T00:58:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f2989926b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161309f8c0 con 0x5616132cb300 2023-09-10T00:58:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f2989926b80 10 client.0 _ll_register_callbacks cb 0x561613105a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:58:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f29777fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132d11a0 0x5616132d5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f29777fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132d11a0 0x5616132d5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54368/0 (socket says 172.21.15.44:54368) 2023-09-10T00:58:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f29777fe700 1 -- 172.21.15.44:0/503713387 learned_addr learned my addr 172.21.15.44:0/503713387 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f2976ffd700 1 --2- 172.21.15.44:0/503713387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132cb300 0x5616132d7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f29767fc700 1 --2- 172.21.15.44:0/503713387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132cbcc0 0x5616132d50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f29777fe700 1 -- 172.21.15.44:0/503713387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132cb300 msgr2=0x5616132d7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f29777fe700 1 --2- 172.21.15.44:0/503713387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132cb300 0x5616132d7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f29777fe700 1 -- 172.21.15.44:0/503713387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132cbcc0 msgr2=0x5616132d50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f29777fe700 1 --2- 172.21.15.44:0/503713387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132cbcc0 0x5616132d50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f29777fe700 1 -- 172.21.15.44:0/503713387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29700112b0 con 0x5616132d11a0 2023-09-10T00:58:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f29777fe700 1 --2- 172.21.15.44:0/503713387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132d11a0 0x5616132d5690 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2970006080 tx=0x7f29700087c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94258ce489f60b0e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.934+0000 7f295aff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:58:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.935+0000 7f295aff5700 1 -- 172.21.15.44:0/503713387 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297000e070 con 0x5616132d11a0 2023-09-10T00:58:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.935+0000 7f295aff5700 1 -- 172.21.15.44:0/503713387 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f29700092e0 con 0x5616132d11a0 2023-09-10T00:58:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.935+0000 7f2989926b80 1 -- 172.21.15.44:0/503713387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561613175490 con 0x5616132d11a0 2023-09-10T00:58:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.935+0000 7f295aff5700 1 -- 172.21.15.44:0/503713387 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2970008c70 con 0x5616132d11a0 2023-09-10T00:58:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.935+0000 7f2989926b80 10 client.18156 fetch_fsmap learned FSMap version 80 2023-09-10T00:58:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.935+0000 7f2989926b80 1 -- 172.21.15.44:0/503713387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x561613032c00 con 0x5616132d11a0 2023-09-10T00:58:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.935+0000 7f295aff5700 1 -- 172.21.15.44:0/503713387 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561613175490 con 0x5616132d11a0 2023-09-10T00:58:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.935+0000 7f295aff5700 1 -- 172.21.15.44:0/503713387 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 80) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f2970015450 con 0x5616132d11a0 2023-09-10T00:58:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.935+0000 7f2989926b80 10 client.18156 fetch_fsmap finished waiting for FSMap version 80 2023-09-10T00:58:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.935+0000 7f2989926b80 10 client.18156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:58:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.935+0000 7f2989926b80 20 client.18156 populate_metadata read hostname 'smithi044' 2023-09-10T00:58:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.935+0000 7f2989926b80 1 --2- 172.21.15.44:0/503713387 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5616132e5370 0x5616132e7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.935+0000 7f2989926b80 4 client.18156 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:58:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.935+0000 7f2989926b80 1 -- 172.21.15.44:0/503713387 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561613232840 con 0x5616132e5370 2023-09-10T00:58:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.935+0000 7f2976ffd700 1 --2- 172.21.15.44:0/503713387 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5616132e5370 0x5616132e7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.936+0000 7f295aff5700 10 client.18156 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:58:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.936+0000 7f2976ffd700 1 --2- 172.21.15.44:0/503713387 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5616132e5370 0x5616132e7790 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=718b0df47a57e2e3 server_cookie=ae9168dfdeb5b101 in_seq=0 out_seq=0 2023-09-10T00:58:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.936+0000 7f295aff5700 1 -- 172.21.15.44:0/503713387 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561613232840 con 0x5616132e5370 2023-09-10T00:58:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.936+0000 7f295aff5700 10 client.18156 handle_command_reply: tid=0 2023-09-10T00:58:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.936+0000 7f2989926b80 1 client.18156 shutdown 2023-09-10T00:58:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.936+0000 7f2989926b80 1 -- 172.21.15.44:0/503713387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132d11a0 msgr2=0x5616132d5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.936+0000 7f2989926b80 1 --2- 172.21.15.44:0/503713387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132d11a0 0x5616132d5690 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2970006080 tx=0x7f29700087c0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.936+0000 7f2989926b80 1 -- 172.21.15.44:0/503713387 shutdown_connections 2023-09-10T00:58:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.936+0000 7f2989926b80 1 --2- 172.21.15.44:0/503713387 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5616132e5370 0x5616132e7790 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.936+0000 7f2989926b80 1 --2- 172.21.15.44:0/503713387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132d11a0 0x5616132d5690 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.936+0000 7f2989926b80 1 --2- 172.21.15.44:0/503713387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132cb300 0x5616132d7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.936+0000 7f2989926b80 1 --2- 172.21.15.44:0/503713387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132cbcc0 0x5616132d50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.936+0000 7f295aff5700 0 client.18156 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:58:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.936+0000 7f2989926b80 1 -- 172.21.15.44:0/503713387 >> 172.21.15.44:0/503713387 conn(0x561613240030 msgr2=0x5616132c57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.937+0000 7f2989926b80 1 -- 172.21.15.44:0/503713387 shutdown_connections 2023-09-10T00:58:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.937+0000 7f2989926b80 1 -- 172.21.15.44:0/503713387 wait complete. 2023-09-10T00:58:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.937+0000 7f2989926b80 20 client.18156 trim_cache size 0 max 16384 2023-09-10T00:58:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.951+0000 7f2989926b80 1 Processor -- start 2023-09-10T00:58:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.951+0000 7f2989926b80 1 -- start start 2023-09-10T00:58:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f2989926b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132cb300 0x5616132dc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f2989926b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132cbcc0 0x5616132cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f2989926b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132d11a0 0x5616132cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f2989926b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616132c7c30 con 0x5616132cb300 2023-09-10T00:58:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f2989926b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616132c7340 con 0x5616132d11a0 2023-09-10T00:58:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f2989926b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616132cedb0 con 0x5616132cbcc0 2023-09-10T00:58:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f2976ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132cb300 0x5616132dc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f2976ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132cb300 0x5616132dc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54372/0 (socket says 172.21.15.44:54372) 2023-09-10T00:58:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f2976ffd700 1 -- 172.21.15.44:0/1694040776 learned_addr learned my addr 172.21.15.44:0/1694040776 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f29777fe700 1 --2- 172.21.15.44:0/1694040776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132d11a0 0x5616132cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f29767fc700 1 --2- 172.21.15.44:0/1694040776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132cbcc0 0x5616132cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f2976ffd700 1 -- 172.21.15.44:0/1694040776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132cbcc0 msgr2=0x5616132cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f2976ffd700 1 --2- 172.21.15.44:0/1694040776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132cbcc0 0x5616132cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f2976ffd700 1 -- 172.21.15.44:0/1694040776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132d11a0 msgr2=0x5616132cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f2976ffd700 1 --2- 172.21.15.44:0/1694040776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132d11a0 0x5616132cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f2976ffd700 1 -- 172.21.15.44:0/1694040776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616130701f0 con 0x5616132cb300 2023-09-10T00:58:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f29777fe700 1 --2- 172.21.15.44:0/1694040776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132d11a0 0x5616132cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:58:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.952+0000 7f2976ffd700 1 --2- 172.21.15.44:0/1694040776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132cb300 0x5616132dc530 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f29680969f0 tx=0x7f2968096230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2dbe9a4bc37ffa server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.953+0000 7f295dffb700 1 -- 172.21.15.44:0/1694040776 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29680b1070 con 0x5616132cb300 2023-09-10T00:58:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.953+0000 7f295dffb700 1 -- 172.21.15.44:0/1694040776 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f296809abf0 con 0x5616132cb300 2023-09-10T00:58:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.953+0000 7f295dffb700 1 -- 172.21.15.44:0/1694040776 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968098890 con 0x5616132cb300 2023-09-10T00:58:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.953+0000 7f2989926b80 1 -- 172.21.15.44:0/1694040776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132cb300 msgr2=0x5616132dc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.953+0000 7f2989926b80 1 --2- 172.21.15.44:0/1694040776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132cb300 0x5616132dc530 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f29680969f0 tx=0x7f2968096230 comp rx=0 tx=0).stop 2023-09-10T00:58:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.953+0000 7f2989926b80 1 -- 172.21.15.44:0/1694040776 shutdown_connections 2023-09-10T00:58:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.953+0000 7f2989926b80 1 --2- 172.21.15.44:0/1694040776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132d11a0 0x5616132cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.953+0000 7f2989926b80 1 --2- 172.21.15.44:0/1694040776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616132cb300 0x5616132dc530 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.953+0000 7f2989926b80 1 --2- 172.21.15.44:0/1694040776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132cbcc0 0x5616132cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.953+0000 7f2989926b80 1 -- 172.21.15.44:0/1694040776 >> 172.21.15.44:0/1694040776 conn(0x561613240030 msgr2=0x5616132e0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.953+0000 7f2989926b80 1 -- 172.21.15.44:0/1694040776 shutdown_connections 2023-09-10T00:58:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.953+0000 7f2989926b80 1 -- 172.21.15.44:0/1694040776 wait complete. 2023-09-10T00:58:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f2989926b80 1 -- start start 2023-09-10T00:58:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f2989926b80 1 Processor -- start 2023-09-10T00:58:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f2989926b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132cbcc0 0x561613335940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f2989926b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132d11a0 0x561613335e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f2989926b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616133363c0 0x5616133409c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f2989926b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561613340f50 con 0x5616133363c0 2023-09-10T00:58:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f2989926b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616133410c0 con 0x5616132d11a0 2023-09-10T00:58:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f2989926b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561613341280 con 0x5616132cbcc0 2023-09-10T00:58:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f2989926b80 10 client.0 _ll_register_callbacks cb 0x7f296c0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:58:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f29777fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616133363c0 0x5616133409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f29767fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132d11a0 0x561613335e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f29777fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616133363c0 0x5616133409c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54382/0 (socket says 172.21.15.44:54382) 2023-09-10T00:58:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f29777fe700 1 -- 172.21.15.44:0/3924251885 learned_addr learned my addr 172.21.15.44:0/3924251885 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f2976ffd700 1 --2- 172.21.15.44:0/3924251885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132cbcc0 0x561613335940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f29777fe700 1 -- 172.21.15.44:0/3924251885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132cbcc0 msgr2=0x561613335940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f29777fe700 1 --2- 172.21.15.44:0/3924251885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132cbcc0 0x561613335940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f29777fe700 1 -- 172.21.15.44:0/3924251885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132d11a0 msgr2=0x561613335e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f29777fe700 1 --2- 172.21.15.44:0/3924251885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132d11a0 0x561613335e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.954+0000 7f29777fe700 1 -- 172.21.15.44:0/3924251885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561613032c00 con 0x5616133363c0 2023-09-10T00:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f29777fe700 1 --2- 172.21.15.44:0/3924251885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616133363c0 0x5616133409c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2970004e40 tx=0x7f29700060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb5f0c03acadcc5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f295aff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f295aff5700 1 -- 172.21.15.44:0/3924251885 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297000e070 con 0x5616133363c0 2023-09-10T00:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f2989926b80 1 -- 172.21.15.44:0/3924251885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f297000f040 con 0x5616133363c0 2023-09-10T00:58:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f295aff5700 1 -- 172.21.15.44:0/3924251885 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2970008c70 con 0x5616133363c0 2023-09-10T00:58:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f295aff5700 1 -- 172.21.15.44:0/3924251885 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29700098a0 con 0x5616133363c0 2023-09-10T00:58:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f2989926b80 10 client.18162 fetch_fsmap learned FSMap version 80 2023-09-10T00:58:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f2989926b80 1 -- 172.21.15.44:0/3924251885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x7f29781a5570 con 0x5616133363c0 2023-09-10T00:58:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f295aff5700 1 -- 172.21.15.44:0/3924251885 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f297000f040 con 0x5616133363c0 2023-09-10T00:58:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f295aff5700 1 -- 172.21.15.44:0/3924251885 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 80) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f2970027d10 con 0x5616133363c0 2023-09-10T00:58:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f2989926b80 10 client.18162 fetch_fsmap finished waiting for FSMap version 80 2023-09-10T00:58:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f2989926b80 10 client.18162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:58:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f2989926b80 20 client.18162 populate_metadata read hostname 'smithi044' 2023-09-10T00:58:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f2989926b80 1 --2- 172.21.15.44:0/3924251885 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561613342680 0x561613344aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:58:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f2989926b80 4 client.18162 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:58:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f2989926b80 1 -- 172.21.15.44:0/3924251885 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5616132168c0 con 0x561613342680 2023-09-10T00:58:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.955+0000 7f2976ffd700 1 --2- 172.21.15.44:0/3924251885 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561613342680 0x561613344aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:58:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.956+0000 7f2976ffd700 1 --2- 172.21.15.44:0/3924251885 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561613342680 0x561613344aa0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b70f6e6b4ff9b68a server_cookie=6a9ea78c019ac814 in_seq=0 out_seq=0 2023-09-10T00:58:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.956+0000 7f295aff5700 10 client.18162 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:58:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f295aff5700 1 -- 172.21.15.44:0/3924251885 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5616132168c0 con 0x561613342680 2023-09-10T00:58:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f295aff5700 10 client.18162 handle_command_reply: tid=0 2023-09-10T00:58:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f2989926b80 1 client.18162 shutdown 2023-09-10T00:58:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f2989926b80 1 -- 172.21.15.44:0/3924251885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616133363c0 msgr2=0x5616133409c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f2989926b80 1 --2- 172.21.15.44:0/3924251885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616133363c0 0x5616133409c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2970004e40 tx=0x7f29700060e0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f2989926b80 1 -- 172.21.15.44:0/3924251885 shutdown_connections 2023-09-10T00:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f2989926b80 1 --2- 172.21.15.44:0/3924251885 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561613342680 0x561613344aa0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f2989926b80 1 --2- 172.21.15.44:0/3924251885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616133363c0 0x5616133409c0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f2989926b80 1 --2- 172.21.15.44:0/3924251885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5616132cbcc0 0x561613335940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f295aff5700 0 client.18162 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f2989926b80 1 --2- 172.21.15.44:0/3924251885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616132d11a0 0x561613335e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f2989926b80 1 -- 172.21.15.44:0/3924251885 >> 172.21.15.44:0/3924251885 conn(0x561613240030 msgr2=0x5616132d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f2989926b80 1 -- 172.21.15.44:0/3924251885 shutdown_connections 2023-09-10T00:58:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f2989926b80 1 -- 172.21.15.44:0/3924251885 wait complete. 2023-09-10T00:58:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f2989926b80 20 client.18162 trim_cache size 0 max 16384 2023-09-10T00:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f297d760700 1 -- 172.21.15.44:0/2519831206 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29600a07b0 msgr2=0x7f29600a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f297d760700 1 --2- 172.21.15.44:0/2519831206 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29600a07b0 0x7f29600a2c70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f29780694d0 tx=0x7f296c009260 comp rx=0 tx=0).stop 2023-09-10T00:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.968+0000 7f297d760700 1 -- 172.21.15.44:0/2519831206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29780ff4d0 msgr2=0x7f2978195350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.969+0000 7f297d760700 1 --2- 172.21.15.44:0/2519831206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29780ff4d0 0x7f2978195350 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2968005910 tx=0x7f2968009580 comp rx=0 tx=0).stop 2023-09-10T00:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.969+0000 7f297d760700 1 -- 172.21.15.44:0/2519831206 shutdown_connections 2023-09-10T00:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.969+0000 7f297d760700 1 --2- 172.21.15.44:0/2519831206 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29600a07b0 0x7f29600a2c70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.969+0000 7f297d760700 1 --2- 172.21.15.44:0/2519831206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2978105590 0x7f29781a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.969+0000 7f297d760700 1 --2- 172.21.15.44:0/2519831206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29780ff4d0 0x7f2978195350 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.969+0000 7f297d760700 1 --2- 172.21.15.44:0/2519831206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29780ffe90 0x7f2978195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:58:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.969+0000 7f297d760700 1 -- 172.21.15.44:0/2519831206 >> 172.21.15.44:0/2519831206 conn(0x7f29780faf60 msgr2=0x7f29781031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:58:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.969+0000 7f297d760700 1 -- 172.21.15.44:0/2519831206 shutdown_connections 2023-09-10T00:58:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:58:48.969+0000 7f297d760700 1 -- 172.21.15.44:0/2519831206 wait complete. 2023-09-10T00:58:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:48 smithi131 ceph-mon[103713]: pgmap v572: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 245 op/s 2023-09-10T00:58:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:48 smithi044 ceph-mon[98676]: pgmap v572: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 245 op/s 2023-09-10T00:58:49.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:48 smithi099 ceph-mon[102692]: pgmap v572: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 245 op/s 2023-09-10T00:58:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:50 smithi131 ceph-mon[103713]: pgmap v573: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 245 op/s 2023-09-10T00:58:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:50 smithi044 ceph-mon[98676]: pgmap v573: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 245 op/s 2023-09-10T00:58:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:50 smithi099 ceph-mon[102692]: pgmap v573: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 245 op/s 2023-09-10T00:58:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:52 smithi131 ceph-mon[103713]: pgmap v574: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 347 op/s 2023-09-10T00:58:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:52 smithi044 ceph-mon[98676]: pgmap v574: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 347 op/s 2023-09-10T00:58:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:52 smithi099 ceph-mon[102692]: pgmap v574: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 347 op/s 2023-09-10T00:58:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:54 smithi131 ceph-mon[103713]: pgmap v575: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 227 op/s 2023-09-10T00:58:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:54 smithi044 ceph-mon[98676]: pgmap v575: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 227 op/s 2023-09-10T00:58:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:54 smithi099 ceph-mon[102692]: pgmap v575: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 227 op/s 2023-09-10T00:58:56.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:55 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:56.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:55 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:56.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:55 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:56 smithi131 ceph-mon[103713]: pgmap v576: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 356 op/s 2023-09-10T00:58:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:56 smithi044 ceph-mon[98676]: pgmap v576: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 356 op/s 2023-09-10T00:58:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:56 smithi099 ceph-mon[102692]: pgmap v576: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 356 op/s 2023-09-10T00:58:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:58:58 smithi131 ceph-mon[103713]: pgmap v577: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 230 op/s 2023-09-10T00:58:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:58:58 smithi044 ceph-mon[98676]: pgmap v577: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 230 op/s 2023-09-10T00:58:59.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:58:58 smithi099 ceph-mon[102692]: pgmap v577: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 230 op/s 2023-09-10T00:59:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:00 smithi131 ceph-mon[103713]: pgmap v578: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 230 op/s 2023-09-10T00:59:01.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:00 smithi044 ceph-mon[98676]: pgmap v578: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 230 op/s 2023-09-10T00:59:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:00 smithi099 ceph-mon[102692]: pgmap v578: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 230 op/s 2023-09-10T00:59:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.377+0000 7fdadabf0700 1 -- 172.21.15.44:0/825425680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdad410d4a0 msgr2=0x7fdad410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.377+0000 7fdadabf0700 1 --2- 172.21.15.44:0/825425680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdad410d4a0 0x7fdad410d880 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009990 tx=0x7fdac8009ca0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.379+0000 7fdadabf0700 1 -- 172.21.15.44:0/825425680 shutdown_connections 2023-09-10T00:59:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.379+0000 7fdadabf0700 1 --2- 172.21.15.44:0/825425680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdad4108850 0x7fdad41142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.379+0000 7fdadabf0700 1 --2- 172.21.15.44:0/825425680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdad410d4a0 0x7fdad410d880 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.379+0000 7fdadabf0700 1 --2- 172.21.15.44:0/825425680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad4107e90 0x7fdad4108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.379+0000 7fdadabf0700 1 -- 172.21.15.44:0/825425680 >> 172.21.15.44:0/825425680 conn(0x7fdad406c6c0 msgr2=0x7fdad406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.379+0000 7fdadabf0700 1 -- 172.21.15.44:0/825425680 shutdown_connections 2023-09-10T00:59:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.379+0000 7fdadabf0700 1 -- 172.21.15.44:0/825425680 wait complete. 2023-09-10T00:59:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.379+0000 7fdadabf0700 1 Processor -- start 2023-09-10T00:59:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdadabf0700 1 -- start start 2023-09-10T00:59:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdadabf0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdad4107e90 0x7fdad419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdadabf0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdad4108850 0x7fdad419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdadabf0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad410d4a0 0x7fdad41a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdadabf0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad41a4030 con 0x7fdad4108850 2023-09-10T00:59:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdadabf0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad41a41a0 con 0x7fdad410d4a0 2023-09-10T00:59:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdadabf0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad41a4360 con 0x7fdad4107e90 2023-09-10T00:59:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdad3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdad4108850 0x7fdad419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdad918d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad410d4a0 0x7fdad41a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdad898c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdad4107e90 0x7fdad419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdad918d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad410d4a0 0x7fdad41a39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56134/0 (socket says 172.21.15.44:56134) 2023-09-10T00:59:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdad3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdad4108850 0x7fdad419f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54392/0 (socket says 172.21.15.44:54392) 2023-09-10T00:59:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdad898c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdad4107e90 0x7fdad419ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33040/0 (socket says 172.21.15.44:33040) 2023-09-10T00:59:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdad918d700 1 -- 172.21.15.44:0/2900869148 learned_addr learned my addr 172.21.15.44:0/2900869148 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdad918d700 1 -- 172.21.15.44:0/2900869148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdad4107e90 msgr2=0x7fdad419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdad918d700 1 --2- 172.21.15.44:0/2900869148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdad4107e90 0x7fdad419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdad918d700 1 -- 172.21.15.44:0/2900869148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdad4108850 msgr2=0x7fdad419f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdad918d700 1 --2- 172.21.15.44:0/2900869148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdad4108850 0x7fdad419f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdad918d700 1 -- 172.21.15.44:0/2900869148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac80095f0 con 0x7fdad410d4a0 2023-09-10T00:59:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.380+0000 7fdad3fff700 1 --2- 172.21.15.44:0/2900869148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdad4108850 0x7fdad419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:59:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.381+0000 7fdad918d700 1 --2- 172.21.15.44:0/2900869148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad410d4a0 0x7fdad41a39f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdacc007ee0 tx=0x7fdacc00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.381+0000 7fdad1ffb700 1 -- 172.21.15.44:0/2900869148 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc00dd20 con 0x7fdad410d4a0 2023-09-10T00:59:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.381+0000 7fdad1ffb700 1 -- 172.21.15.44:0/2900869148 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdacc00f040 con 0x7fdad410d4a0 2023-09-10T00:59:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.381+0000 7fdadabf0700 1 -- 172.21.15.44:0/2900869148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad41a4640 con 0x7fdad410d4a0 2023-09-10T00:59:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.381+0000 7fdad1ffb700 1 -- 172.21.15.44:0/2900869148 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc004750 con 0x7fdad410d4a0 2023-09-10T00:59:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.381+0000 7fdadabf0700 1 -- 172.21.15.44:0/2900869148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad4078a40 con 0x7fdad410d4a0 2023-09-10T00:59:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.383+0000 7fdad1ffb700 1 -- 172.21.15.44:0/2900869148 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fdacc003a00 con 0x7fdad410d4a0 2023-09-10T00:59:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.384+0000 7fdad1ffb700 1 --2- 172.21.15.44:0/2900869148 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdac40a0900 0x7fdac40a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.384+0000 7fdad1ffb700 1 -- 172.21.15.44:0/2900869148 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fdacc0cbaf0 con 0x7fdad410d4a0 2023-09-10T00:59:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.384+0000 7fdad898c700 1 --2- 172.21.15.44:0/2900869148 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdac40a0900 0x7fdac40a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.385+0000 7fdad898c700 1 --2- 172.21.15.44:0/2900869148 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdac40a0900 0x7fdac40a2dc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdac8006010 tx=0x7fdac800b260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.385+0000 7fdae6db6b80 1 Processor -- start 2023-09-10T00:59:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.385+0000 7fdae6db6b80 1 -- start start 2023-09-10T00:59:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdae6db6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f43580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdae6db6b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f3d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdae6db6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9f49f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdae6db6b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb9e966d0 con 0x564cb9f3d300 2023-09-10T00:59:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdae6db6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb9e93ff0 con 0x564cb9f3dcc0 2023-09-10T00:59:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdae6db6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb9e8f8e0 con 0x564cb9f431a0 2023-09-10T00:59:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdad3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f3d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdad898c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f43580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdad898c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f43580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33048/0 (socket says 172.21.15.44:33048) 2023-09-10T00:59:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdad3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f3d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54400/0 (socket says 172.21.15.44:54400) 2023-09-10T00:59:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdad898c700 1 -- 172.21.15.44:0/2708623567 learned_addr learned my addr 172.21.15.44:0/2708623567 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdad3fff700 1 -- 172.21.15.44:0/2708623567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 msgr2=0x564cb9f43580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdad3fff700 1 --2- 172.21.15.44:0/2708623567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f43580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdad3fff700 1 -- 172.21.15.44:0/2708623567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 msgr2=0x564cb9f49f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T00:59:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdad3fff700 1 --2- 172.21.15.44:0/2708623567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9f49f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdad3fff700 1 -- 172.21.15.44:0/2708623567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cb9cbbd80 con 0x564cb9f3d300 2023-09-10T00:59:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.386+0000 7fdad898c700 1 --2- 172.21.15.44:0/2708623567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f43580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:59:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.387+0000 7fdad3fff700 1 --2- 172.21.15.44:0/2708623567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f3d780 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdac00114c0 tx=0x7fdac00117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db272a2163e4c378 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.387+0000 7fdabeffd700 1 -- 172.21.15.44:0/2708623567 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0004230 con 0x564cb9f3d300 2023-09-10T00:59:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.387+0000 7fdabeffd700 1 -- 172.21.15.44:0/2708623567 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdac0004cc0 con 0x564cb9f3d300 2023-09-10T00:59:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.387+0000 7fdabeffd700 1 -- 172.21.15.44:0/2708623567 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0005520 con 0x564cb9f3d300 2023-09-10T00:59:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.388+0000 7fdae6db6b80 1 -- 172.21.15.44:0/2708623567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 msgr2=0x564cb9f3d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.388+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/2708623567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f3d780 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdac00114c0 tx=0x7fdac00117d0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.388+0000 7fdae6db6b80 1 -- 172.21.15.44:0/2708623567 shutdown_connections 2023-09-10T00:59:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.388+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/2708623567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9f49f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.388+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/2708623567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f43580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.388+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/2708623567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f3d780 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.388+0000 7fdae6db6b80 1 -- 172.21.15.44:0/2708623567 >> 172.21.15.44:0/2708623567 conn(0x564cb9eb2030 msgr2=0x564cb9f38560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.388+0000 7fdae6db6b80 1 -- 172.21.15.44:0/2708623567 shutdown_connections 2023-09-10T00:59:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.388+0000 7fdae6db6b80 1 -- 172.21.15.44:0/2708623567 wait complete. 2023-09-10T00:59:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.388+0000 7fdae6db6b80 1 -- start start 2023-09-10T00:59:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.388+0000 7fdae6db6b80 1 Processor -- start 2023-09-10T00:59:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.389+0000 7fdae6db6b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f51320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.389+0000 7fdae6db6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9f51860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.389+0000 7fdae6db6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f4b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.389+0000 7fdae6db6b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb9e2db10 con 0x564cb9f3d300 2023-09-10T00:59:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.389+0000 7fdae6db6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb9e2d810 con 0x564cb9f3dcc0 2023-09-10T00:59:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.389+0000 7fdae6db6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb9d118c0 con 0x564cb9f431a0 2023-09-10T00:59:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.389+0000 7fdae6db6b80 10 client.0 _ll_register_callbacks cb 0x564cb9d77a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:59:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.389+0000 7fdad898c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f51320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.389+0000 7fdad898c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f51320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54402/0 (socket says 172.21.15.44:54402) 2023-09-10T00:59:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.389+0000 7fdad898c700 1 -- 172.21.15.44:0/1152950744 learned_addr learned my addr 172.21.15.44:0/1152950744 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.389+0000 7fdad3fff700 1 --2- 172.21.15.44:0/1152950744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9f51860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.390+0000 7fdad898c700 1 -- 172.21.15.44:0/1152950744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 msgr2=0x564cb9f4b610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T00:59:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.390+0000 7fdad898c700 1 --2- 172.21.15.44:0/1152950744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f4b610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.390+0000 7fdad898c700 1 -- 172.21.15.44:0/1152950744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 msgr2=0x564cb9f51860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.390+0000 7fdad898c700 1 --2- 172.21.15.44:0/1152950744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9f51860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.390+0000 7fdad898c700 1 -- 172.21.15.44:0/1152950744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad4107af0 con 0x564cb9f3d300 2023-09-10T00:59:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.390+0000 7fdad3fff700 1 --2- 172.21.15.44:0/1152950744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9f51860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:59:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.390+0000 7fdad898c700 1 --2- 172.21.15.44:0/1152950744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f51320 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009990 tx=0x7fdac8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b71847805bf3f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.390+0000 7fdab3fff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:59:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.390+0000 7fdab3fff700 1 -- 172.21.15.44:0/1152950744 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8024410 con 0x564cb9f3d300 2023-09-10T00:59:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.390+0000 7fdae6db6b80 1 -- 172.21.15.44:0/1152950744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564cb9de7490 con 0x564cb9f3d300 2023-09-10T00:59:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.390+0000 7fdab3fff700 1 -- 172.21.15.44:0/1152950744 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fdac8024570 con 0x564cb9f3d300 2023-09-10T00:59:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.390+0000 7fdab3fff700 1 -- 172.21.15.44:0/1152950744 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdac8024730 con 0x564cb9f3d300 2023-09-10T00:59:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.390+0000 7fdae6db6b80 10 client.18174 fetch_fsmap learned FSMap version 81 2023-09-10T00:59:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.390+0000 7fdae6db6b80 1 -- 172.21.15.44:0/1152950744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x7fdac0009c90 con 0x564cb9f3d300 2023-09-10T00:59:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.391+0000 7fdab3fff700 1 -- 172.21.15.44:0/1152950744 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564cb9de7490 con 0x564cb9f3d300 2023-09-10T00:59:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.391+0000 7fdab3fff700 1 -- 172.21.15.44:0/1152950744 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 81) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fdac802f070 con 0x564cb9f3d300 2023-09-10T00:59:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.391+0000 7fdae6db6b80 10 client.18174 fetch_fsmap finished waiting for FSMap version 81 2023-09-10T00:59:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.391+0000 7fdae6db6b80 10 client.18174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:59:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.391+0000 7fdae6db6b80 20 client.18174 populate_metadata read hostname 'smithi044' 2023-09-10T00:59:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.391+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/1152950744 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564cb9f5b830 0x564cb9f5dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.391+0000 7fdae6db6b80 4 client.18174 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:59:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.391+0000 7fdae6db6b80 1 -- 172.21.15.44:0/1152950744 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564cb9ea4840 con 0x564cb9f5b830 2023-09-10T00:59:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.392+0000 7fdad3fff700 1 --2- 172.21.15.44:0/1152950744 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564cb9f5b830 0x564cb9f5dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.392+0000 7fdab3fff700 10 client.18174 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.392+0000 7fdad3fff700 1 --2- 172.21.15.44:0/1152950744 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564cb9f5b830 0x564cb9f5dc50 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=336104b67f757841 server_cookie=407d42413d1db289 in_seq=0 out_seq=0 2023-09-10T00:59:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.393+0000 7fdab3fff700 1 -- 172.21.15.44:0/1152950744 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564cb9ea4840 con 0x564cb9f5b830 2023-09-10T00:59:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.393+0000 7fdab3fff700 10 client.18174 handle_command_reply: tid=0 2023-09-10T00:59:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.393+0000 7fdae6db6b80 1 client.18174 shutdown 2023-09-10T00:59:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.393+0000 7fdae6db6b80 1 -- 172.21.15.44:0/1152950744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 msgr2=0x564cb9f51320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.393+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/1152950744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f51320 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009990 tx=0x7fdac8009ca0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.393+0000 7fdae6db6b80 1 -- 172.21.15.44:0/1152950744 shutdown_connections 2023-09-10T00:59:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.393+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/1152950744 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564cb9f5b830 0x564cb9f5dc50 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.393+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/1152950744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f4b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.393+0000 7fdab3fff700 0 client.18174 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.393+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/1152950744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f51320 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.393+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/1152950744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9f51860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.393+0000 7fdae6db6b80 1 -- 172.21.15.44:0/1152950744 >> 172.21.15.44:0/1152950744 conn(0x564cb9eb2030 msgr2=0x564cb9f377a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.393+0000 7fdae6db6b80 1 -- 172.21.15.44:0/1152950744 shutdown_connections 2023-09-10T00:59:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.393+0000 7fdae6db6b80 1 -- 172.21.15.44:0/1152950744 wait complete. 2023-09-10T00:59:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.393+0000 7fdae6db6b80 20 client.18174 trim_cache size 0 max 16384 2023-09-10T00:59:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.415+0000 7fdae6db6b80 1 Processor -- start 2023-09-10T00:59:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.415+0000 7fdae6db6b80 1 -- start start 2023-09-10T00:59:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.415+0000 7fdae6db6b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f3f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.415+0000 7fdae6db6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9f3f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.415+0000 7fdae6db6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f38090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.415+0000 7fdae6db6b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb9de7490 con 0x564cb9f3d300 2023-09-10T00:59:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.415+0000 7fdae6db6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb9f3e140 con 0x564cb9f3dcc0 2023-09-10T00:59:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.415+0000 7fdae6db6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb9f39e10 con 0x564cb9f431a0 2023-09-10T00:59:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.415+0000 7fdad898c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f3f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.415+0000 7fdad898c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f3f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54408/0 (socket says 172.21.15.44:54408) 2023-09-10T00:59:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.415+0000 7fdad898c700 1 -- 172.21.15.44:0/1850038565 learned_addr learned my addr 172.21.15.44:0/1850038565 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.415+0000 7fdad918d700 1 --2- 172.21.15.44:0/1850038565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f38090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.416+0000 7fdad898c700 1 -- 172.21.15.44:0/1850038565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 msgr2=0x564cb9f38090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.416+0000 7fdad898c700 1 --2- 172.21.15.44:0/1850038565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f38090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.416+0000 7fdad898c700 1 -- 172.21.15.44:0/1850038565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 msgr2=0x564cb9f3f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.416+0000 7fdad3fff700 1 --2- 172.21.15.44:0/1850038565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9f3f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.416+0000 7fdad898c700 1 --2- 172.21.15.44:0/1850038565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9f3f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.416+0000 7fdad898c700 1 -- 172.21.15.44:0/1850038565 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cb9ca4c00 con 0x564cb9f3d300 2023-09-10T00:59:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.416+0000 7fdad3fff700 1 --2- 172.21.15.44:0/1850038565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9f3f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:59:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.416+0000 7fdad898c700 1 --2- 172.21.15.44:0/1850038565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f3f050 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fdac80098e0 tx=0x7fdac8005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a445d9713e3b10 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.416+0000 7fdabf7fe700 1 -- 172.21.15.44:0/1850038565 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8018400 con 0x564cb9f3d300 2023-09-10T00:59:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.416+0000 7fdabf7fe700 1 -- 172.21.15.44:0/1850038565 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdac8028040 con 0x564cb9f3d300 2023-09-10T00:59:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.416+0000 7fdabf7fe700 1 -- 172.21.15.44:0/1850038565 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac803acc0 con 0x564cb9f3d300 2023-09-10T00:59:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.417+0000 7fdae6db6b80 1 -- 172.21.15.44:0/1850038565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 msgr2=0x564cb9f3f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.417+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/1850038565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f3f050 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fdac80098e0 tx=0x7fdac8005a00 comp rx=0 tx=0).stop 2023-09-10T00:59:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.417+0000 7fdae6db6b80 1 -- 172.21.15.44:0/1850038565 shutdown_connections 2023-09-10T00:59:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.417+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/1850038565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f38090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.417+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/1850038565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9f3f050 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.417+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/1850038565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9f3f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.417+0000 7fdae6db6b80 1 -- 172.21.15.44:0/1850038565 >> 172.21.15.44:0/1850038565 conn(0x564cb9eb2030 msgr2=0x564cb9f4bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.417+0000 7fdae6db6b80 1 -- 172.21.15.44:0/1850038565 shutdown_connections 2023-09-10T00:59:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.417+0000 7fdae6db6b80 1 -- 172.21.15.44:0/1850038565 wait complete. 2023-09-10T00:59:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.418+0000 7fdae6db6b80 1 -- start start 2023-09-10T00:59:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.418+0000 7fdae6db6b80 1 Processor -- start 2023-09-10T00:59:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.418+0000 7fdae6db6b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9fa8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.418+0000 7fdae6db6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9fa8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.418+0000 7fdae6db6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f50b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.418+0000 7fdae6db6b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb9f51120 con 0x564cb9f3d300 2023-09-10T00:59:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.418+0000 7fdae6db6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb9f51290 con 0x564cb9f3dcc0 2023-09-10T00:59:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.418+0000 7fdae6db6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cb9f51450 con 0x564cb9f431a0 2023-09-10T00:59:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.418+0000 7fdae6db6b80 10 client.0 _ll_register_callbacks cb 0x7fdac800bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:59:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.418+0000 7fdad898c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9fa8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.418+0000 7fdad918d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f50b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.418+0000 7fdad3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9fa8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.418+0000 7fdad918d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f50b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33068/0 (socket says 172.21.15.44:33068) 2023-09-10T00:59:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.418+0000 7fdad898c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9fa8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54414/0 (socket says 172.21.15.44:54414) 2023-09-10T00:59:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.418+0000 7fdad3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9fa8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56154/0 (socket says 172.21.15.44:56154) 2023-09-10T00:59:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.418+0000 7fdad918d700 1 -- 172.21.15.44:0/2530453896 learned_addr learned my addr 172.21.15.44:0/2530453896 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.419+0000 7fdad918d700 1 -- 172.21.15.44:0/2530453896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 msgr2=0x564cb9fa8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.419+0000 7fdad918d700 1 --2- 172.21.15.44:0/2530453896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9fa8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.419+0000 7fdad918d700 1 -- 172.21.15.44:0/2530453896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 msgr2=0x564cb9fa8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.419+0000 7fdad918d700 1 --2- 172.21.15.44:0/2530453896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9fa8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.419+0000 7fdad918d700 1 -- 172.21.15.44:0/2530453896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac0009c90 con 0x564cb9f431a0 2023-09-10T00:59:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.419+0000 7fdad3fff700 1 --2- 172.21.15.44:0/2530453896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9fa8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:59:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.419+0000 7fdad918d700 1 --2- 172.21.15.44:0/2530453896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f50b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0988a0 tx=0x7fdacc098790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65f0c0adaa322b76 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.419+0000 7fdab3fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T00:59:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.419+0000 7fdab3fff700 1 -- 172.21.15.44:0/2530453896 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0b0070 con 0x564cb9f431a0 2023-09-10T00:59:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.419+0000 7fdae6db6b80 1 -- 172.21.15.44:0/2530453896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdac0004cc0 con 0x564cb9f431a0 2023-09-10T00:59:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.419+0000 7fdab3fff700 1 -- 172.21.15.44:0/2530453896 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fdacc00de80 con 0x564cb9f431a0 2023-09-10T00:59:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.419+0000 7fdab3fff700 1 -- 172.21.15.44:0/2530453896 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdacc0048b0 con 0x564cb9f431a0 2023-09-10T00:59:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.419+0000 7fdae6db6b80 10 client.27778 fetch_fsmap learned FSMap version 81 2023-09-10T00:59:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.419+0000 7fdae6db6b80 1 -- 172.21.15.44:0/2530453896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x564cb9ce21f0 con 0x564cb9f431a0 2023-09-10T00:59:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.420+0000 7fdab3fff700 1 -- 172.21.15.44:0/2530453896 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdacc0a4070 con 0x564cb9f431a0 2023-09-10T00:59:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.420+0000 7fdab3fff700 1 -- 172.21.15.44:0/2530453896 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 81) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fdacc098e00 con 0x564cb9f431a0 2023-09-10T00:59:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.420+0000 7fdae6db6b80 10 client.27778 fetch_fsmap finished waiting for FSMap version 81 2023-09-10T00:59:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.420+0000 7fdae6db6b80 10 client.27778 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:59:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.420+0000 7fdae6db6b80 20 client.27778 populate_metadata read hostname 'smithi044' 2023-09-10T00:59:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.420+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/2530453896 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564cb9fafed0 0x564cb9fb22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.420+0000 7fdae6db6b80 4 client.27778 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:59:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.420+0000 7fdae6db6b80 1 -- 172.21.15.44:0/2530453896 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564cb9e888c0 con 0x564cb9fafed0 2023-09-10T00:59:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.420+0000 7fdad898c700 1 --2- 172.21.15.44:0/2530453896 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564cb9fafed0 0x564cb9fb22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.420+0000 7fdab3fff700 10 client.27778 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:02.420+0000 7fdad898c700 1 --2- 172.21.15.44:0/2530453896 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564cb9fafed0 0x564cb9fb22f0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8558b7b271a7c56c server_cookie=6728229edb52cf5f in_seq=0 out_seq=0 2023-09-10T00:59:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:02 smithi131 ceph-mon[103713]: pgmap v579: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 327 op/s 2023-09-10T00:59:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:02 smithi044 ceph-mon[98676]: pgmap v579: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 327 op/s 2023-09-10T00:59:03.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:02 smithi099 ceph-mon[102692]: pgmap v579: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 327 op/s 2023-09-10T00:59:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.421+0000 7fdab3fff700 1 -- 172.21.15.44:0/2530453896 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564cb9e888c0 con 0x564cb9fafed0 2023-09-10T00:59:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.421+0000 7fdab3fff700 10 client.27778 handle_command_reply: tid=0 2023-09-10T00:59:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.421+0000 7fdae6db6b80 1 client.27778 shutdown 2023-09-10T00:59:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.421+0000 7fdae6db6b80 1 -- 172.21.15.44:0/2530453896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 msgr2=0x564cb9f50b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.421+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/2530453896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f50b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0988a0 tx=0x7fdacc098790 comp rx=0 tx=0).stop 2023-09-10T00:59:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.421+0000 7fdae6db6b80 1 -- 172.21.15.44:0/2530453896 shutdown_connections 2023-09-10T00:59:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.421+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/2530453896 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564cb9fafed0 0x564cb9fb22f0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.421+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/2530453896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564cb9f431a0 0x564cb9f50b50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.421+0000 7fdab3fff700 0 client.27778 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.421+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/2530453896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564cb9f3d300 0x564cb9fa8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.421+0000 7fdae6db6b80 1 --2- 172.21.15.44:0/2530453896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564cb9f3dcc0 0x564cb9fa8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.421+0000 7fdae6db6b80 1 -- 172.21.15.44:0/2530453896 >> 172.21.15.44:0/2530453896 conn(0x564cb9eb2030 msgr2=0x564cb9f3f050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.421+0000 7fdae6db6b80 1 -- 172.21.15.44:0/2530453896 shutdown_connections 2023-09-10T00:59:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.421+0000 7fdae6db6b80 1 -- 172.21.15.44:0/2530453896 wait complete. 2023-09-10T00:59:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.421+0000 7fdae6db6b80 20 client.27778 trim_cache size 0 max 16384 2023-09-10T00:59:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:59:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:59:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:59:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:59:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.422+0000 7fdadabf0700 1 -- 172.21.15.44:0/2900869148 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdac40a0900 msgr2=0x7fdac40a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.422+0000 7fdadabf0700 1 --2- 172.21.15.44:0/2900869148 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdac40a0900 0x7fdac40a2dc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdac8006010 tx=0x7fdac800b260 comp rx=0 tx=0).stop 2023-09-10T00:59:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.422+0000 7fdadabf0700 1 -- 172.21.15.44:0/2900869148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad410d4a0 msgr2=0x7fdad41a39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.422+0000 7fdadabf0700 1 --2- 172.21.15.44:0/2900869148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad410d4a0 0x7fdad41a39f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdacc007ee0 tx=0x7fdacc00d3b0 comp rx=0 tx=0).stop 2023-09-10T00:59:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.422+0000 7fdadabf0700 1 -- 172.21.15.44:0/2900869148 shutdown_connections 2023-09-10T00:59:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.422+0000 7fdadabf0700 1 --2- 172.21.15.44:0/2900869148 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdac40a0900 0x7fdac40a2dc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.422+0000 7fdadabf0700 1 --2- 172.21.15.44:0/2900869148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad410d4a0 0x7fdad41a39f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.422+0000 7fdadabf0700 1 --2- 172.21.15.44:0/2900869148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdad4107e90 0x7fdad419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.422+0000 7fdadabf0700 1 --2- 172.21.15.44:0/2900869148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdad4108850 0x7fdad419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.422+0000 7fdadabf0700 1 -- 172.21.15.44:0/2900869148 >> 172.21.15.44:0/2900869148 conn(0x7fdad406c6c0 msgr2=0x7fdad4070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.422+0000 7fdadabf0700 1 -- 172.21.15.44:0/2900869148 shutdown_connections 2023-09-10T00:59:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:03.422+0000 7fdadabf0700 1 -- 172.21.15.44:0/2900869148 wait complete. 2023-09-10T00:59:04.296 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: *** Caught signal (Segmentation fault) ** 2023-09-10T00:59:04.297 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: in thread 7f88bfc99700 thread_name:md_log_replay 2023-09-10T00:59:04.297 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T00:59:04.297 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f88d0848cf0] 2023-09-10T00:59:04.297 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x55b54413d30f] 2023-09-10T00:59:04.297 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55b54413f543] 2023-09-10T00:59:04.298 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 4: (MDLog::_replay_thread()+0x647) [0x55b5441a8e97] 2023-09-10T00:59:04.298 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55b5441b2381] 2023-09-10T00:59:04.298 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f88d083e1ca] 2023-09-10T00:59:04.298 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 7: clone() 2023-09-10T00:59:04.298 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 2023-09-10T00:59:04.026+0000 7f88bfc99700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T00:59:04.299 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: in thread 7f88bfc99700 thread_name:md_log_replay 2023-09-10T00:59:04.299 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 2023-09-10T00:59:04.299 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T00:59:04.299 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f88d0848cf0] 2023-09-10T00:59:04.299 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x55b54413d30f] 2023-09-10T00:59:04.299 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55b54413f543] 2023-09-10T00:59:04.300 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 4: (MDLog::_replay_thread()+0x647) [0x55b5441a8e97] 2023-09-10T00:59:04.300 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55b5441b2381] 2023-09-10T00:59:04.300 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f88d083e1ca] 2023-09-10T00:59:04.300 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 7: clone() 2023-09-10T00:59:04.300 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T00:59:04.301 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 2023-09-10T00:59:04.301 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 0> 2023-09-10T00:59:04.026+0000 7f88bfc99700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T00:59:04.301 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: in thread 7f88bfc99700 thread_name:md_log_replay 2023-09-10T00:59:04.301 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 2023-09-10T00:59:04.301 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T00:59:04.301 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f88d0848cf0] 2023-09-10T00:59:04.302 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x55b54413d30f] 2023-09-10T00:59:04.302 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55b54413f543] 2023-09-10T00:59:04.302 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 4: (MDLog::_replay_thread()+0x647) [0x55b5441a8e97] 2023-09-10T00:59:04.302 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55b5441b2381] 2023-09-10T00:59:04.302 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f88d083e1ca] 2023-09-10T00:59:04.303 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 7: clone() 2023-09-10T00:59:04.303 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T00:59:04.303 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 2023-09-10T00:59:04.303 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 0> 2023-09-10T00:59:04.026+0000 7f88bfc99700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T00:59:04.303 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: in thread 7f88bfc99700 thread_name:md_log_replay 2023-09-10T00:59:04.304 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 2023-09-10T00:59:04.304 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T00:59:04.304 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f88d0848cf0] 2023-09-10T00:59:04.304 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x55b54413d30f] 2023-09-10T00:59:04.304 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55b54413f543] 2023-09-10T00:59:04.304 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 4: (MDLog::_replay_thread()+0x647) [0x55b5441a8e97] 2023-09-10T00:59:04.305 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55b5441b2381] 2023-09-10T00:59:04.305 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f88d083e1ca] 2023-09-10T00:59:04.305 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 7: clone() 2023-09-10T00:59:04.305 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T00:59:04.305 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[147775]: 2023-09-10T00:59:04.670 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Main process exited, code=exited, status=139/n/a 2023-09-10T00:59:04.932 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-mon[103713]: pgmap v580: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2023-09-10T00:59:04.933 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:04 smithi131 ceph-mon[103713]: osdmap e118: 12 total, 12 up, 12 in 2023-09-10T00:59:05.295 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:04 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Failed with result 'exit-code'. 2023-09-10T00:59:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:04 smithi044 ceph-mon[98676]: pgmap v580: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2023-09-10T00:59:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:04 smithi044 ceph-mon[98676]: osdmap e118: 12 total, 12 up, 12 in 2023-09-10T00:59:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:04 smithi099 ceph-mon[102692]: pgmap v580: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2023-09-10T00:59:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:04 smithi099 ceph-mon[102692]: osdmap e118: 12 total, 12 up, 12 in 2023-09-10T00:59:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:06 smithi131 ceph-mon[103713]: pgmap v582: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 258 op/s 2023-09-10T00:59:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:06 smithi044 ceph-mon[98676]: pgmap v582: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 258 op/s 2023-09-10T00:59:07.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:06 smithi099 ceph-mon[102692]: pgmap v582: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 258 op/s 2023-09-10T00:59:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:08 smithi131 ceph-mon[103713]: pgmap v583: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 258 op/s 2023-09-10T00:59:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:08 smithi044 ceph-mon[98676]: pgmap v583: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 258 op/s 2023-09-10T00:59:09.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:08 smithi099 ceph-mon[102692]: pgmap v583: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 258 op/s 2023-09-10T00:59:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:10 smithi131 ceph-mon[103713]: pgmap v584: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 258 op/s 2023-09-10T00:59:11.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:10 smithi131 ceph-mon[103713]: osdmap e119: 12 total, 12 up, 12 in 2023-09-10T00:59:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:10 smithi044 ceph-mon[98676]: pgmap v584: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 258 op/s 2023-09-10T00:59:11.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:10 smithi044 ceph-mon[98676]: osdmap e119: 12 total, 12 up, 12 in 2023-09-10T00:59:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:10 smithi099 ceph-mon[102692]: pgmap v584: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 258 op/s 2023-09-10T00:59:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:10 smithi099 ceph-mon[102692]: osdmap e119: 12 total, 12 up, 12 in 2023-09-10T00:59:12.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:11 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:59:12.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:11 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:59:12.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:11 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:59:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:12 smithi131 ceph-mon[103713]: pgmap v586: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 295 op/s 2023-09-10T00:59:13.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:12 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:59:13.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:12 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:59:13.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:12 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:59:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:12 smithi044 ceph-mon[98676]: pgmap v586: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 295 op/s 2023-09-10T00:59:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:12 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:59:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:12 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:59:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:12 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:59:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:12 smithi099 ceph-mon[102692]: pgmap v586: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 295 op/s 2023-09-10T00:59:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:12 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:59:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:12 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:59:13.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:12 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T00:59:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:14 smithi131 ceph-mon[103713]: pgmap v587: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 252 op/s 2023-09-10T00:59:15.295 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:15 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Service RestartSec=10s expired, scheduling restart. 2023-09-10T00:59:15.296 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:15 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Scheduled restart job, restart counter is at 2. 2023-09-10T00:59:15.296 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:15 smithi131 systemd[1]: Stopped Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:59:15.296 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:15 smithi131 systemd[1]: Starting Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T00:59:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:14 smithi044 ceph-mon[98676]: pgmap v587: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 252 op/s 2023-09-10T00:59:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:14 smithi099 ceph-mon[102692]: pgmap v587: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 252 op/s 2023-09-10T00:59:16.367 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:16 smithi131 podman[148576]: 2023-09-10T00:59:16.367 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:16 smithi131 ceph-mds[148616]: -- 172.21.15.131:0/3308969345 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f1f86891e0 con 0x55f1f865e800 2023-09-10T00:59:16.367 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:16 smithi131 bash[148576]: b4333d0328bf8a90eb57ca6b56d142d7ad9fc28ea2bbceda4300ef1f68cb6e38 2023-09-10T00:59:16.367 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:16 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: starting mds.c at 2023-09-10T00:59:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.535+0000 7f4b049a4700 1 -- 172.21.15.44:0/4215643635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b00106620 msgr2=0x7f4b00106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.535+0000 7f4b049a4700 1 --2- 172.21.15.44:0/4215643635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b00106620 0x7f4b00106a00 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009990 tx=0x7f4af4009ca0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.535+0000 7f4b049a4700 1 -- 172.21.15.44:0/4215643635 shutdown_connections 2023-09-10T00:59:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.535+0000 7f4b049a4700 1 --2- 172.21.15.44:0/4215643635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b000ff460 0x7f4b0010d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.535+0000 7f4b049a4700 1 --2- 172.21.15.44:0/4215643635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b00106620 0x7f4b00106a00 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.535+0000 7f4b049a4700 1 --2- 172.21.15.44:0/4215643635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b000feaa0 0x7f4b000fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.535+0000 7f4b049a4700 1 -- 172.21.15.44:0/4215643635 >> 172.21.15.44:0/4215643635 conn(0x7f4b000769d0 msgr2=0x7f4b000fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.535+0000 7f4b049a4700 1 -- 172.21.15.44:0/4215643635 shutdown_connections 2023-09-10T00:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.535+0000 7f4b049a4700 1 -- 172.21.15.44:0/4215643635 wait complete. 2023-09-10T00:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.536+0000 7f4b049a4700 1 Processor -- start 2023-09-10T00:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.536+0000 7f4b049a4700 1 -- start start 2023-09-10T00:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.536+0000 7f4b049a4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b000feaa0 0x7f4b0019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.536+0000 7f4b049a4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b000ff460 0x7f4b0019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.536+0000 7f4b049a4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b00106620 0x7f4b00196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.536+0000 7f4b049a4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b001969f0 con 0x7f4b000ff460 2023-09-10T00:59:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.536+0000 7f4b049a4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b00196b60 con 0x7f4b000feaa0 2023-09-10T00:59:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.536+0000 7f4b049a4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b00196d20 con 0x7f4b00106620 2023-09-10T00:59:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.536+0000 7f4afdd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b000ff460 0x7f4b0019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.536+0000 7f4afdd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b000ff460 0x7f4b0019ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54430/0 (socket says 172.21.15.44:54430) 2023-09-10T00:59:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.536+0000 7f4afed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b00106620 0x7f4b00196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.536+0000 7f4afe59c700 1 --2- 172.21.15.44:0/83265411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b000feaa0 0x7f4b0019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.536+0000 7f4afdd9b700 1 -- 172.21.15.44:0/83265411 learned_addr learned my addr 172.21.15.44:0/83265411 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.537+0000 7f4afdd9b700 1 -- 172.21.15.44:0/83265411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b00106620 msgr2=0x7f4b00196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.537+0000 7f4afdd9b700 1 --2- 172.21.15.44:0/83265411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b00106620 0x7f4b00196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.537+0000 7f4afdd9b700 1 -- 172.21.15.44:0/83265411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b000feaa0 msgr2=0x7f4b0019c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.537+0000 7f4afdd9b700 1 --2- 172.21.15.44:0/83265411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b000feaa0 0x7f4b0019c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.537+0000 7f4afdd9b700 1 -- 172.21.15.44:0/83265411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af40095f0 con 0x7f4b000ff460 2023-09-10T00:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.537+0000 7f4afdd9b700 1 --2- 172.21.15.44:0/83265411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b000ff460 0x7f4b0019ca60 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f4af000e9a0 tx=0x7f4af000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.537+0000 7f4aef7fe700 1 -- 172.21.15.44:0/83265411 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0009790 con 0x7f4b000ff460 2023-09-10T00:59:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.537+0000 7f4b049a4700 1 -- 172.21.15.44:0/83265411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b00197000 con 0x7f4b000ff460 2023-09-10T00:59:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.537+0000 7f4aef7fe700 1 -- 172.21.15.44:0/83265411 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4af0010430 con 0x7f4b000ff460 2023-09-10T00:59:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.537+0000 7f4aef7fe700 1 -- 172.21.15.44:0/83265411 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af000cea0 con 0x7f4b000ff460 2023-09-10T00:59:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.537+0000 7f4b049a4700 1 -- 172.21.15.44:0/83265411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b001a3510 con 0x7f4b000ff460 2023-09-10T00:59:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.539+0000 7f4aef7fe700 1 -- 172.21.15.44:0/83265411 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f4af00105a0 con 0x7f4b000ff460 2023-09-10T00:59:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.540+0000 7f4aef7fe700 1 --2- 172.21.15.44:0/83265411 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4ae80a07b0 0x7f4ae80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.540+0000 7f4afe59c700 1 --2- 172.21.15.44:0/83265411 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4ae80a07b0 0x7f4ae80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.540+0000 7f4aef7fe700 1 -- 172.21.15.44:0/83265411 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f4af0014070 con 0x7f4b000ff460 2023-09-10T00:59:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.540+0000 7f4afe59c700 1 --2- 172.21.15.44:0/83265411 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4ae80a07b0 0x7f4ae80a2c70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4af40059d0 tx=0x7f4af4005960 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.541+0000 7f4b10b6ab80 1 Processor -- start 2023-09-10T00:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.541+0000 7f4b10b6ab80 1 -- start start 2023-09-10T00:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.541+0000 7f4b10b6ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd5e1a0 0x55a89fd5e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.541+0000 7f4b10b6ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd58300 0x55a89fd58780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.541+0000 7f4b10b6ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58cc0 0x55a89fd64f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.541+0000 7f4b10b6ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89fcb16d0 con 0x55a89fd5e1a0 2023-09-10T00:59:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.541+0000 7f4b10b6ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89fcaeff0 con 0x55a89fd58300 2023-09-10T00:59:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.541+0000 7f4b10b6ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89fcaa8e0 con 0x55a89fd58cc0 2023-09-10T00:59:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.541+0000 7f4afe59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd5e1a0 0x55a89fd5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.541+0000 7f4afe59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd5e1a0 0x55a89fd5e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54436/0 (socket says 172.21.15.44:54436) 2023-09-10T00:59:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.541+0000 7f4afe59c700 1 -- 172.21.15.44:0/3501230687 learned_addr learned my addr 172.21.15.44:0/3501230687 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.541+0000 7f4afed9d700 1 --2- 172.21.15.44:0/3501230687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58cc0 0x55a89fd64f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.541+0000 7f4afdd9b700 1 --2- 172.21.15.44:0/3501230687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd58300 0x55a89fd58780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.542+0000 7f4afe59c700 1 -- 172.21.15.44:0/3501230687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58cc0 msgr2=0x55a89fd64f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.542+0000 7f4afe59c700 1 --2- 172.21.15.44:0/3501230687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58cc0 0x55a89fd64f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.542+0000 7f4afe59c700 1 -- 172.21.15.44:0/3501230687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd58300 msgr2=0x55a89fd58780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.542+0000 7f4afe59c700 1 --2- 172.21.15.44:0/3501230687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd58300 0x55a89fd58780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.542+0000 7f4afe59c700 1 -- 172.21.15.44:0/3501230687 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a89fad6d80 con 0x55a89fd5e1a0 2023-09-10T00:59:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.542+0000 7f4afe59c700 1 --2- 172.21.15.44:0/3501230687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd5e1a0 0x55a89fd5e580 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f4af400b2e0 tx=0x7f4af4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3c9fc7af4cfa36 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.542+0000 7f4aecff9700 1 -- 172.21.15.44:0/3501230687 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4004700 con 0x55a89fd5e1a0 2023-09-10T00:59:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.542+0000 7f4aecff9700 1 -- 172.21.15.44:0/3501230687 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4af401e040 con 0x55a89fd5e1a0 2023-09-10T00:59:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.542+0000 7f4aecff9700 1 -- 172.21.15.44:0/3501230687 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af400f930 con 0x55a89fd5e1a0 2023-09-10T00:59:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/3501230687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd5e1a0 msgr2=0x55a89fd5e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/3501230687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd5e1a0 0x55a89fd5e580 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f4af400b2e0 tx=0x7f4af4020000 comp rx=0 tx=0).stop 2023-09-10T00:59:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/3501230687 shutdown_connections 2023-09-10T00:59:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/3501230687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58cc0 0x55a89fd64f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/3501230687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd5e1a0 0x55a89fd5e580 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/3501230687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd58300 0x55a89fd58780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/3501230687 >> 172.21.15.44:0/3501230687 conn(0x55a89fccd030 msgr2=0x55a89fd53560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/3501230687 shutdown_connections 2023-09-10T00:59:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/3501230687 wait complete. 2023-09-10T00:59:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 -- start start 2023-09-10T00:59:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 Processor -- start 2023-09-10T00:59:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd58300 0x55a89fd64e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58cc0 0x55a89fd62060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd5e1a0 0x55a89fd62650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89fc48b10 con 0x55a89fd58300 2023-09-10T00:59:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89fc48810 con 0x55a89fd5e1a0 2023-09-10T00:59:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.543+0000 7f4b10b6ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89fb2c8c0 con 0x55a89fd58cc0 2023-09-10T00:59:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4b10b6ab80 10 client.0 _ll_register_callbacks cb 0x55a89fb92a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:59:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4afe59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd58300 0x55a89fd64e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4afe59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd58300 0x55a89fd64e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54442/0 (socket says 172.21.15.44:54442) 2023-09-10T00:59:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4afe59c700 1 -- 172.21.15.44:0/3274123145 learned_addr learned my addr 172.21.15.44:0/3274123145 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4afdd9b700 1 --2- 172.21.15.44:0/3274123145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58cc0 0x55a89fd62060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4afed9d700 1 --2- 172.21.15.44:0/3274123145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd5e1a0 0x55a89fd62650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4afe59c700 1 -- 172.21.15.44:0/3274123145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58cc0 msgr2=0x55a89fd62060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4afe59c700 1 --2- 172.21.15.44:0/3274123145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58cc0 0x55a89fd62060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4afe59c700 1 -- 172.21.15.44:0/3274123145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd5e1a0 msgr2=0x55a89fd62650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4afe59c700 1 --2- 172.21.15.44:0/3274123145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd5e1a0 0x55a89fd62650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4afe59c700 1 -- 172.21.15.44:0/3274123145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0010dd30 con 0x55a89fd58300 2023-09-10T00:59:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4afe59c700 1 --2- 172.21.15.44:0/3274123145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd58300 0x55a89fd64e50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f4af4005820 tx=0x7f4af400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca78785dac6e03b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4ae5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:59:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4ae5ffb700 1 -- 172.21.15.44:0/3274123145 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4004700 con 0x55a89fd58300 2023-09-10T00:59:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4ae5ffb700 1 -- 172.21.15.44:0/3274123145 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f4af400be50 con 0x55a89fd58300 2023-09-10T00:59:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/3274123145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a89fc02490 con 0x55a89fd58300 2023-09-10T00:59:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.544+0000 7f4ae5ffb700 1 -- 172.21.15.44:0/3274123145 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4af4004860 con 0x55a89fd58300 2023-09-10T00:59:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.545+0000 7f4b10b6ab80 10 client.18198 fetch_fsmap learned FSMap version 81 2023-09-10T00:59:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.545+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/3274123145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x55a89fabfc00 con 0x55a89fd58300 2023-09-10T00:59:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.545+0000 7f4ae5ffb700 1 -- 172.21.15.44:0/3274123145 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a89fc02490 con 0x55a89fd58300 2023-09-10T00:59:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.545+0000 7f4ae5ffb700 1 -- 172.21.15.44:0/3274123145 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 81) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f4af40249f0 con 0x55a89fd58300 2023-09-10T00:59:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.545+0000 7f4b10b6ab80 10 client.18198 fetch_fsmap finished waiting for FSMap version 81 2023-09-10T00:59:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.545+0000 7f4b10b6ab80 10 client.18198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:59:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.545+0000 7f4b10b6ab80 20 client.18198 populate_metadata read hostname 'smithi044' 2023-09-10T00:59:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.545+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/3274123145 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a89fd71e20 0x55a89fd74210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.545+0000 7f4b10b6ab80 4 client.18198 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:59:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.545+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/3274123145 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a89fcbf840 con 0x55a89fd71e20 2023-09-10T00:59:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.545+0000 7f4afdd9b700 1 --2- 172.21.15.44:0/3274123145 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a89fd71e20 0x55a89fd74210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.546+0000 7f4afdd9b700 1 --2- 172.21.15.44:0/3274123145 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a89fd71e20 0x55a89fd74210 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39f9891cf34c3826 server_cookie=1752a720096a08d3 in_seq=0 out_seq=0 2023-09-10T00:59:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.546+0000 7f4ae5ffb700 10 client.18198 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.546+0000 7f4ae5ffb700 1 -- 172.21.15.44:0/3274123145 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a89fcbf840 con 0x55a89fd71e20 2023-09-10T00:59:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.546+0000 7f4ae5ffb700 10 client.18198 handle_command_reply: tid=0 2023-09-10T00:59:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.546+0000 7f4b10b6ab80 1 client.18198 shutdown 2023-09-10T00:59:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.546+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/3274123145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd58300 msgr2=0x55a89fd64e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.546+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/3274123145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd58300 0x55a89fd64e50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f4af4005820 tx=0x7f4af400f670 comp rx=0 tx=0).stop 2023-09-10T00:59:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.546+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/3274123145 shutdown_connections 2023-09-10T00:59:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.546+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/3274123145 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a89fd71e20 0x55a89fd74210 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.546+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/3274123145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd5e1a0 0x55a89fd62650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.546+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/3274123145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd58300 0x55a89fd64e50 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.546+0000 7f4ae5ffb700 0 client.18198 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.546+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/3274123145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58cc0 0x55a89fd62060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.546+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/3274123145 >> 172.21.15.44:0/3274123145 conn(0x55a89fccd030 msgr2=0x55a89fd527a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.547+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/3274123145 shutdown_connections 2023-09-10T00:59:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.547+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/3274123145 wait complete. 2023-09-10T00:59:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.547+0000 7f4b10b6ab80 20 client.18198 trim_cache size 0 max 16384 2023-09-10T00:59:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.562+0000 7f4b10b6ab80 1 Processor -- start 2023-09-10T00:59:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.562+0000 7f4b10b6ab80 1 -- start start 2023-09-10T00:59:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.562+0000 7f4b10b6ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58300 0x55a89fd64980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.562+0000 7f4b10b6ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd58cc0 0x55a89fd5d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.562+0000 7f4b10b6ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd5e1a0 0x55a89fd5d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.562+0000 7f4b10b6ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89fd4ffc0 con 0x55a89fd58cc0 2023-09-10T00:59:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.562+0000 7f4b10b6ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89fd5e580 con 0x55a89fd5e1a0 2023-09-10T00:59:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.562+0000 7f4b10b6ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89fd605e0 con 0x55a89fd58300 2023-09-10T00:59:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.562+0000 7f4afdd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd58cc0 0x55a89fd5d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.563+0000 7f4afdd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd58cc0 0x55a89fd5d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54452/0 (socket says 172.21.15.44:54452) 2023-09-10T00:59:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.563+0000 7f4afdd9b700 1 -- 172.21.15.44:0/2803466851 learned_addr learned my addr 172.21.15.44:0/2803466851 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.563+0000 7f4afe59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58300 0x55a89fd64980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.563+0000 7f4afed9d700 1 --2- 172.21.15.44:0/2803466851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd5e1a0 0x55a89fd5d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.563+0000 7f4afdd9b700 1 -- 172.21.15.44:0/2803466851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58300 msgr2=0x55a89fd64980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.563+0000 7f4afdd9b700 1 --2- 172.21.15.44:0/2803466851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58300 0x55a89fd64980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.563+0000 7f4afdd9b700 1 -- 172.21.15.44:0/2803466851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd5e1a0 msgr2=0x55a89fd5d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.563+0000 7f4afdd9b700 1 --2- 172.21.15.44:0/2803466851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd5e1a0 0x55a89fd5d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.563+0000 7f4afdd9b700 1 -- 172.21.15.44:0/2803466851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a89fafd1f0 con 0x55a89fd58cc0 2023-09-10T00:59:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.563+0000 7f4afdd9b700 1 --2- 172.21.15.44:0/2803466851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd58cc0 0x55a89fd5d050 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f4af0099580 tx=0x7f4af000c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51c5c321d63a78a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.563+0000 7f4aed7fa700 1 -- 172.21.15.44:0/2803466851 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0099cf0 con 0x55a89fd58cc0 2023-09-10T00:59:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.563+0000 7f4aed7fa700 1 -- 172.21.15.44:0/2803466851 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4af00995f0 con 0x55a89fd58cc0 2023-09-10T00:59:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.563+0000 7f4aed7fa700 1 -- 172.21.15.44:0/2803466851 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af00840b0 con 0x55a89fd58cc0 2023-09-10T00:59:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/2803466851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd58cc0 msgr2=0x55a89fd5d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/2803466851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd58cc0 0x55a89fd5d050 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f4af0099580 tx=0x7f4af000c480 comp rx=0 tx=0).stop 2023-09-10T00:59:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/2803466851 shutdown_connections 2023-09-10T00:59:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/2803466851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd5e1a0 0x55a89fd5d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/2803466851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58300 0x55a89fd64980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/2803466851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd58cc0 0x55a89fd5d050 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/2803466851 >> 172.21.15.44:0/2803466851 conn(0x55a89fccd030 msgr2=0x55a89fd5c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/2803466851 shutdown_connections 2023-09-10T00:59:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/2803466851 wait complete. 2023-09-10T00:59:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 -- start start 2023-09-10T00:59:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 Processor -- start 2023-09-10T00:59:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd58300 0x55a89fd68d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58cc0 0x55a89fd69320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd5e1a0 0x55a89fd67440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89fd67a90 con 0x55a89fd5e1a0 2023-09-10T00:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89fd67c00 con 0x55a89fd58300 2023-09-10T00:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.564+0000 7f4b10b6ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89fd67dc0 con 0x55a89fd58cc0 2023-09-10T00:59:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4b10b6ab80 10 client.0 _ll_register_callbacks cb 0x7f4af0095270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4afed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd5e1a0 0x55a89fd67440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4afe59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd58300 0x55a89fd68d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4afed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd5e1a0 0x55a89fd67440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54460/0 (socket says 172.21.15.44:54460) 2023-09-10T00:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4afed9d700 1 -- 172.21.15.44:0/1506974243 learned_addr learned my addr 172.21.15.44:0/1506974243 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4afdd9b700 1 --2- 172.21.15.44:0/1506974243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58cc0 0x55a89fd69320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4afed9d700 1 -- 172.21.15.44:0/1506974243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58cc0 msgr2=0x55a89fd69320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4afed9d700 1 --2- 172.21.15.44:0/1506974243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58cc0 0x55a89fd69320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4afed9d700 1 -- 172.21.15.44:0/1506974243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd58300 msgr2=0x55a89fd68d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4afed9d700 1 --2- 172.21.15.44:0/1506974243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd58300 0x55a89fd68d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4afed9d700 1 -- 172.21.15.44:0/1506974243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b001a3510 con 0x55a89fd5e1a0 2023-09-10T00:59:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4afe59c700 1 --2- 172.21.15.44:0/1506974243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd58300 0x55a89fd68d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:59:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4afed9d700 1 --2- 172.21.15.44:0/1506974243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd5e1a0 0x55a89fd67440 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f4af80144e0 tx=0x7f4af80147f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d6411f2b42fa17 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4ae5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:59:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4ae5ffb700 1 -- 172.21.15.44:0/1506974243 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af801d070 con 0x55a89fd5e1a0 2023-09-10T00:59:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4ae5ffb700 1 -- 172.21.15.44:0/1506974243 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f4af8018110 con 0x55a89fd5e1a0 2023-09-10T00:59:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/1506974243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4af401e040 con 0x55a89fd5e1a0 2023-09-10T00:59:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4ae5ffb700 1 -- 172.21.15.44:0/1506974243 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4af800e040 con 0x55a89fd5e1a0 2023-09-10T00:59:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4b10b6ab80 10 client.18204 fetch_fsmap learned FSMap version 81 2023-09-10T00:59:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.565+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/1506974243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x55a89fad5250 con 0x55a89fd5e1a0 2023-09-10T00:59:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.566+0000 7f4ae5ffb700 1 -- 172.21.15.44:0/1506974243 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4af401e040 con 0x55a89fd5e1a0 2023-09-10T00:59:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.566+0000 7f4ae5ffb700 1 -- 172.21.15.44:0/1506974243 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 81) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f4af8004630 con 0x55a89fd5e1a0 2023-09-10T00:59:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.566+0000 7f4b10b6ab80 10 client.18204 fetch_fsmap finished waiting for FSMap version 81 2023-09-10T00:59:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.566+0000 7f4b10b6ab80 10 client.18204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:59:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.566+0000 7f4b10b6ab80 20 client.18204 populate_metadata read hostname 'smithi044' 2023-09-10T00:59:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.566+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/1506974243 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a89fdc6f90 0x55a89fdc93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.566+0000 7f4b10b6ab80 4 client.18204 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:59:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.566+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/1506974243 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a89fca38c0 con 0x55a89fdc6f90 2023-09-10T00:59:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.566+0000 7f4afe59c700 1 --2- 172.21.15.44:0/1506974243 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a89fdc6f90 0x55a89fdc93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.566+0000 7f4ae5ffb700 10 client.18204 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.566+0000 7f4afe59c700 1 --2- 172.21.15.44:0/1506974243 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a89fdc6f90 0x55a89fdc93b0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1aab39b554392f2 server_cookie=c7347587ed51ee23 in_seq=0 out_seq=0 2023-09-10T00:59:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.581+0000 7f4ae5ffb700 1 -- 172.21.15.44:0/1506974243 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a89fca38c0 con 0x55a89fdc6f90 2023-09-10T00:59:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.581+0000 7f4ae5ffb700 10 client.18204 handle_command_reply: tid=0 2023-09-10T00:59:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.581+0000 7f4b10b6ab80 1 client.18204 shutdown 2023-09-10T00:59:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.581+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/1506974243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd5e1a0 msgr2=0x55a89fd67440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.581+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/1506974243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd5e1a0 0x55a89fd67440 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f4af80144e0 tx=0x7f4af80147f0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.581+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/1506974243 shutdown_connections 2023-09-10T00:59:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.581+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/1506974243 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a89fdc6f90 0x55a89fdc93b0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.581+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/1506974243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a89fd5e1a0 0x55a89fd67440 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.581+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/1506974243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a89fd58300 0x55a89fd68d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.581+0000 7f4ae5ffb700 0 client.18204 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.581+0000 7f4b10b6ab80 1 --2- 172.21.15.44:0/1506974243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a89fd58cc0 0x55a89fd69320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.581+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/1506974243 >> 172.21.15.44:0/1506974243 conn(0x55a89fccd030 msgr2=0x55a89fd5d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.581+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/1506974243 shutdown_connections 2023-09-10T00:59:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.581+0000 7f4b10b6ab80 1 -- 172.21.15.44:0/1506974243 wait complete. 2023-09-10T00:59:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.581+0000 7f4b10b6ab80 20 client.18204 trim_cache size 0 max 16384 2023-09-10T00:59:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:59:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:59:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:59:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:59:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.582+0000 7f4b049a4700 1 -- 172.21.15.44:0/83265411 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4ae80a07b0 msgr2=0x7f4ae80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.582+0000 7f4b049a4700 1 --2- 172.21.15.44:0/83265411 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4ae80a07b0 0x7f4ae80a2c70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4af40059d0 tx=0x7f4af4005960 comp rx=0 tx=0).stop 2023-09-10T00:59:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.582+0000 7f4b049a4700 1 -- 172.21.15.44:0/83265411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b000ff460 msgr2=0x7f4b0019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.582+0000 7f4b049a4700 1 --2- 172.21.15.44:0/83265411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b000ff460 0x7f4b0019ca60 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f4af000e9a0 tx=0x7f4af000ecb0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.582+0000 7f4b049a4700 1 -- 172.21.15.44:0/83265411 shutdown_connections 2023-09-10T00:59:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.582+0000 7f4b049a4700 1 --2- 172.21.15.44:0/83265411 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4ae80a07b0 0x7f4ae80a2c70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.582+0000 7f4b049a4700 1 --2- 172.21.15.44:0/83265411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b00106620 0x7f4b00196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.582+0000 7f4b049a4700 1 --2- 172.21.15.44:0/83265411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b000feaa0 0x7f4b0019c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.582+0000 7f4b049a4700 1 --2- 172.21.15.44:0/83265411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b000ff460 0x7f4b0019ca60 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.582+0000 7f4b049a4700 1 -- 172.21.15.44:0/83265411 >> 172.21.15.44:0/83265411 conn(0x7f4b000769d0 msgr2=0x7f4b00104160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.582+0000 7f4b049a4700 1 -- 172.21.15.44:0/83265411 shutdown_connections 2023-09-10T00:59:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:16.582+0000 7f4b049a4700 1 -- 172.21.15.44:0/83265411 wait complete. 2023-09-10T00:59:16.795 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 00:59:16 smithi131 systemd[1]: Started Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T00:59:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:16 smithi131 ceph-mon[103713]: pgmap v588: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 247 op/s 2023-09-10T00:59:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:16 smithi131 ceph-mon[103713]: daemon mds.c restarted 2023-09-10T00:59:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:16 smithi044 ceph-mon[98676]: pgmap v588: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 247 op/s 2023-09-10T00:59:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:16 smithi044 ceph-mon[98676]: daemon mds.c restarted 2023-09-10T00:59:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:16 smithi099 ceph-mon[102692]: pgmap v588: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 247 op/s 2023-09-10T00:59:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:16 smithi099 ceph-mon[102692]: daemon mds.c restarted 2023-09-10T00:59:18.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:17 smithi131 ceph-mon[103713]: osdmap e120: 12 total, 12 up, 12 in 2023-09-10T00:59:18.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:17 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.131:6836/1086969990,v1:172.21.15.131:6837/1086969990] up:boot 2023-09-10T00:59:18.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:17 smithi131 ceph-mon[103713]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-09-10T00:59:18.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:17 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T00:59:18.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:17 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:59:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:17 smithi044 ceph-mon[98676]: osdmap e120: 12 total, 12 up, 12 in 2023-09-10T00:59:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:17 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.131:6836/1086969990,v1:172.21.15.131:6837/1086969990] up:boot 2023-09-10T00:59:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:17 smithi044 ceph-mon[98676]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-09-10T00:59:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:17 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T00:59:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:17 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:59:18.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:17 smithi099 ceph-mon[102692]: osdmap e120: 12 total, 12 up, 12 in 2023-09-10T00:59:18.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:17 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.131:6836/1086969990,v1:172.21.15.131:6837/1086969990] up:boot 2023-09-10T00:59:18.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:17 smithi099 ceph-mon[102692]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-09-10T00:59:18.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:17 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T00:59:18.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:17 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:59:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:18 smithi131 ceph-mon[103713]: pgmap v590: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 309 op/s 2023-09-10T00:59:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:18 smithi044 ceph-mon[98676]: pgmap v590: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 309 op/s 2023-09-10T00:59:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:18 smithi099 ceph-mon[102692]: pgmap v590: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 309 op/s 2023-09-10T00:59:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:20 smithi131 ceph-mon[103713]: pgmap v591: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 178 op/s 2023-09-10T00:59:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:20 smithi044 ceph-mon[98676]: pgmap v591: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 178 op/s 2023-09-10T00:59:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:20 smithi099 ceph-mon[102692]: pgmap v591: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 178 op/s 2023-09-10T00:59:22.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:22 smithi131 ceph-mon[103713]: pgmap v592: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2023-09-10T00:59:22.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:22 smithi044 ceph-mon[98676]: pgmap v592: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2023-09-10T00:59:22.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:22 smithi099 ceph-mon[102692]: pgmap v592: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2023-09-10T00:59:24.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:24 smithi099 ceph-mon[102692]: pgmap v593: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2023-09-10T00:59:24.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:24 smithi131 ceph-mon[103713]: pgmap v593: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2023-09-10T00:59:24.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:24 smithi044 ceph-mon[98676]: pgmap v593: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2023-09-10T00:59:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:26 smithi131 ceph-mon[103713]: pgmap v594: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 246 op/s 2023-09-10T00:59:27.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:26 smithi044 ceph-mon[98676]: pgmap v594: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 246 op/s 2023-09-10T00:59:27.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:26 smithi099 ceph-mon[102692]: pgmap v594: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 246 op/s 2023-09-10T00:59:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:28 smithi131 ceph-mon[103713]: pgmap v595: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 237 op/s 2023-09-10T00:59:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:28 smithi044 ceph-mon[98676]: pgmap v595: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 237 op/s 2023-09-10T00:59:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:28 smithi099 ceph-mon[102692]: pgmap v595: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 237 op/s 2023-09-10T00:59:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.736+0000 7f3bec6cc700 1 -- 172.21.15.44:0/1615043065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be4108fa0 msgr2=0x7f3be4109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.736+0000 7f3bec6cc700 1 --2- 172.21.15.44:0/1615043065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be4108fa0 0x7f3be4109380 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009990 tx=0x7f3be0009ca0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.736+0000 7f3bec6cc700 1 -- 172.21.15.44:0/1615043065 shutdown_connections 2023-09-10T00:59:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.736+0000 7f3bec6cc700 1 --2- 172.21.15.44:0/1615043065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4112550 0x7f3be4114940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.736+0000 7f3bec6cc700 1 --2- 172.21.15.44:0/1615043065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be4108fa0 0x7f3be4109380 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.736+0000 7f3bec6cc700 1 --2- 172.21.15.44:0/1615043065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be41098c0 0x7f3be4112010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.736+0000 7f3bec6cc700 1 -- 172.21.15.44:0/1615043065 >> 172.21.15.44:0/1615043065 conn(0x7f3be406c6c0 msgr2=0x7f3be406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.736+0000 7f3bec6cc700 1 -- 172.21.15.44:0/1615043065 shutdown_connections 2023-09-10T00:59:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.736+0000 7f3bec6cc700 1 -- 172.21.15.44:0/1615043065 wait complete. 2023-09-10T00:59:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3bec6cc700 1 Processor -- start 2023-09-10T00:59:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3bec6cc700 1 -- start start 2023-09-10T00:59:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3bec6cc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be4108fa0 0x7f3be419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3bec6cc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be41098c0 0x7f3be419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3bec6cc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4112550 0x7f3be41ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3bec6cc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be419e8a0 con 0x7f3be4108fa0 2023-09-10T00:59:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3bec6cc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be41ad5d0 con 0x7f3be4112550 2023-09-10T00:59:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3bec6cc700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be41ad790 con 0x7f3be41098c0 2023-09-10T00:59:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3be9c67700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be41098c0 0x7f3be419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3be9c67700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be41098c0 0x7f3be419e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33122/0 (socket says 172.21.15.44:33122) 2023-09-10T00:59:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3be9c67700 1 -- 172.21.15.44:0/379074787 learned_addr learned my addr 172.21.15.44:0/379074787 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3bea468700 1 --2- 172.21.15.44:0/379074787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be4108fa0 0x7f3be419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3beac69700 1 --2- 172.21.15.44:0/379074787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4112550 0x7f3be41ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3be9c67700 1 -- 172.21.15.44:0/379074787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4112550 msgr2=0x7f3be41ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3be9c67700 1 --2- 172.21.15.44:0/379074787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4112550 0x7f3be41ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3be9c67700 1 -- 172.21.15.44:0/379074787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be4108fa0 msgr2=0x7f3be419dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3be9c67700 1 --2- 172.21.15.44:0/379074787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be4108fa0 0x7f3be419dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3be9c67700 1 -- 172.21.15.44:0/379074787 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be00095f0 con 0x7f3be41098c0 2023-09-10T00:59:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3bea468700 1 --2- 172.21.15.44:0/379074787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be4108fa0 0x7f3be419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:59:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.737+0000 7f3beac69700 1 --2- 172.21.15.44:0/379074787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4112550 0x7f3be41ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:59:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.738+0000 7f3be9c67700 1 --2- 172.21.15.44:0/379074787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be41098c0 0x7f3be419e130 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400e9d0 tx=0x7f3bd400ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.738+0000 7f3bdb7fe700 1 -- 172.21.15.44:0/379074787 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4009790 con 0x7f3be41098c0 2023-09-10T00:59:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.738+0000 7f3bdb7fe700 1 -- 172.21.15.44:0/379074787 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3bd400cbd0 con 0x7f3be41098c0 2023-09-10T00:59:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.738+0000 7f3bdb7fe700 1 -- 172.21.15.44:0/379074787 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4018680 con 0x7f3be41098c0 2023-09-10T00:59:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.738+0000 7f3bec6cc700 1 -- 172.21.15.44:0/379074787 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be41ada70 con 0x7f3be41098c0 2023-09-10T00:59:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.738+0000 7f3bec6cc700 1 -- 172.21.15.44:0/379074787 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be41adf30 con 0x7f3be41098c0 2023-09-10T00:59:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.741+0000 7f3bdb7fe700 1 -- 172.21.15.44:0/379074787 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f3bd400cd40 con 0x7f3be41098c0 2023-09-10T00:59:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.741+0000 7f3bdb7fe700 1 --2- 172.21.15.44:0/379074787 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3bd00a07b0 0x7f3bd00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.741+0000 7f3bea468700 1 --2- 172.21.15.44:0/379074787 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3bd00a07b0 0x7f3bd00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.741+0000 7f3bdb7fe700 1 -- 172.21.15.44:0/379074787 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f3bd4014070 con 0x7f3be41098c0 2023-09-10T00:59:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.742+0000 7f3bea468700 1 --2- 172.21.15.44:0/379074787 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3bd00a07b0 0x7f3bd00a2c70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3be0006010 tx=0x7f3be000b260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.742+0000 7f3bf8892b80 1 Processor -- start 2023-09-10T00:59:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.743+0000 7f3bf8892b80 1 -- start start 2023-09-10T00:59:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.743+0000 7f3bf8892b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb810a1a0 0x560fb810a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.743+0000 7f3bf8892b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 0x560fb8104780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.743+0000 7f3bf8892b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb8104cc0 0x560fb8110f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.743+0000 7f3bf8892b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fb805d6d0 con 0x560fb810a1a0 2023-09-10T00:59:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.743+0000 7f3bf8892b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fb805aff0 con 0x560fb8104300 2023-09-10T00:59:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.743+0000 7f3bf8892b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fb80568e0 con 0x560fb8104cc0 2023-09-10T00:59:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.743+0000 7f3bea468700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb810a1a0 0x560fb810a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.743+0000 7f3bea468700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb810a1a0 0x560fb810a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54478/0 (socket says 172.21.15.44:54478) 2023-09-10T00:59:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.743+0000 7f3bea468700 1 -- 172.21.15.44:0/1979969841 learned_addr learned my addr 172.21.15.44:0/1979969841 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.743+0000 7f3beac69700 1 --2- 172.21.15.44:0/1979969841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb8104cc0 0x560fb8110f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.743+0000 7f3be9c67700 1 --2- 172.21.15.44:0/1979969841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 0x560fb8104780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.744+0000 7f3beac69700 1 -- 172.21.15.44:0/1979969841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 msgr2=0x560fb8104780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.744+0000 7f3beac69700 1 --2- 172.21.15.44:0/1979969841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 0x560fb8104780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.744+0000 7f3beac69700 1 -- 172.21.15.44:0/1979969841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb810a1a0 msgr2=0x560fb810a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.744+0000 7f3beac69700 1 --2- 172.21.15.44:0/1979969841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb810a1a0 0x560fb810a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.744+0000 7f3beac69700 1 -- 172.21.15.44:0/1979969841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fb7e82d80 con 0x560fb8104cc0 2023-09-10T00:59:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.744+0000 7f3bea468700 1 --2- 172.21.15.44:0/1979969841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb810a1a0 0x560fb810a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:59:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.744+0000 7f3beac69700 1 --2- 172.21.15.44:0/1979969841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb8104cc0 0x560fb8110f30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009eb0 tx=0x7f3bdc009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae7a2b2f736c8a64 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.744+0000 7f3bd8ff9700 1 -- 172.21.15.44:0/1979969841 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc011070 con 0x560fb8104cc0 2023-09-10T00:59:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.744+0000 7f3bd8ff9700 1 -- 172.21.15.44:0/1979969841 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3bdc00ce60 con 0x560fb8104cc0 2023-09-10T00:59:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.744+0000 7f3bd8ff9700 1 -- 172.21.15.44:0/1979969841 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc015e00 con 0x560fb8104cc0 2023-09-10T00:59:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.745+0000 7f3bf8892b80 1 -- 172.21.15.44:0/1979969841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb8104cc0 msgr2=0x560fb8110f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.745+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/1979969841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb8104cc0 0x560fb8110f30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009eb0 tx=0x7f3bdc009ee0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.745+0000 7f3bf8892b80 1 -- 172.21.15.44:0/1979969841 shutdown_connections 2023-09-10T00:59:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.745+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/1979969841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb8104cc0 0x560fb8110f30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.745+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/1979969841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb810a1a0 0x560fb810a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.745+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/1979969841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 0x560fb8104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.745+0000 7f3bf8892b80 1 -- 172.21.15.44:0/1979969841 >> 172.21.15.44:0/1979969841 conn(0x560fb8079030 msgr2=0x560fb80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.745+0000 7f3bf8892b80 1 -- 172.21.15.44:0/1979969841 shutdown_connections 2023-09-10T00:59:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.745+0000 7f3bf8892b80 1 -- 172.21.15.44:0/1979969841 wait complete. 2023-09-10T00:59:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.745+0000 7f3bf8892b80 1 -- start start 2023-09-10T00:59:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3bf8892b80 1 Processor -- start 2023-09-10T00:59:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3bf8892b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104300 0x560fb8110d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3bf8892b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104cc0 0x560fb810dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3bf8892b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb810a1a0 0x560fb810e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3bf8892b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fb7ff4b10 con 0x560fb8104300 2023-09-10T00:59:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3bf8892b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fb7ff4810 con 0x560fb8104cc0 2023-09-10T00:59:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3bf8892b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fb7ed88c0 con 0x560fb810a1a0 2023-09-10T00:59:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3bf8892b80 10 client.0 _ll_register_callbacks cb 0x560fb7f3ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:59:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3bea468700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104300 0x560fb8110d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3bea468700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104300 0x560fb8110d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54484/0 (socket says 172.21.15.44:54484) 2023-09-10T00:59:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3bea468700 1 -- 172.21.15.44:0/2960172519 learned_addr learned my addr 172.21.15.44:0/2960172519 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3be9c67700 1 --2- 172.21.15.44:0/2960172519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104cc0 0x560fb810dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3beac69700 1 --2- 172.21.15.44:0/2960172519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb810a1a0 0x560fb810e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3bea468700 1 -- 172.21.15.44:0/2960172519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb810a1a0 msgr2=0x560fb810e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3bea468700 1 --2- 172.21.15.44:0/2960172519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb810a1a0 0x560fb810e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3bea468700 1 -- 172.21.15.44:0/2960172519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104cc0 msgr2=0x560fb810dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3bea468700 1 --2- 172.21.15.44:0/2960172519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104cc0 0x560fb810dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3bea468700 1 -- 172.21.15.44:0/2960172519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be41153d0 con 0x560fb8104300 2023-09-10T00:59:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3be9c67700 1 --2- 172.21.15.44:0/2960172519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104cc0 0x560fb810dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:59:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.746+0000 7f3beac69700 1 --2- 172.21.15.44:0/2960172519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb810a1a0 0x560fb810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T00:59:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.747+0000 7f3bea468700 1 --2- 172.21.15.44:0/2960172519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104300 0x560fb8110d40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3be00098e0 tx=0x7f3be0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a954ee8b6afa6c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.747+0000 7f3bcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T00:59:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.748+0000 7f3bcdffb700 1 -- 172.21.15.44:0/2960172519 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be000f7f0 con 0x560fb8104300 2023-09-10T00:59:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.748+0000 7f3bcdffb700 1 -- 172.21.15.44:0/2960172519 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f3be000f950 con 0x560fb8104300 2023-09-10T00:59:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.748+0000 7f3bf8892b80 1 -- 172.21.15.44:0/2960172519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fb7fae490 con 0x560fb8104300 2023-09-10T00:59:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.748+0000 7f3bcdffb700 1 -- 172.21.15.44:0/2960172519 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3be000fb10 con 0x560fb8104300 2023-09-10T00:59:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.748+0000 7f3bf8892b80 10 client.18216 fetch_fsmap learned FSMap version 83 2023-09-10T00:59:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.748+0000 7f3bf8892b80 1 -- 172.21.15.44:0/2960172519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x7f3bdc009b90 con 0x560fb8104300 2023-09-10T00:59:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.748+0000 7f3bcdffb700 1 -- 172.21.15.44:0/2960172519 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560fb7fae490 con 0x560fb8104300 2023-09-10T00:59:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.748+0000 7f3bcdffb700 1 -- 172.21.15.44:0/2960172519 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 83) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3be002f070 con 0x560fb8104300 2023-09-10T00:59:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.748+0000 7f3bf8892b80 10 client.18216 fetch_fsmap finished waiting for FSMap version 83 2023-09-10T00:59:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.748+0000 7f3bf8892b80 10 client.18216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:59:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.748+0000 7f3bf8892b80 20 client.18216 populate_metadata read hostname 'smithi044' 2023-09-10T00:59:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.748+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/2960172519 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560fb811e360 0x560fb8120780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.748+0000 7f3bf8892b80 4 client.18216 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:59:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.748+0000 7f3bf8892b80 1 -- 172.21.15.44:0/2960172519 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fb806b840 con 0x560fb811e360 2023-09-10T00:59:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.748+0000 7f3be9c67700 1 --2- 172.21.15.44:0/2960172519 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560fb811e360 0x560fb8120780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.749+0000 7f3bcdffb700 10 client.18216 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.749+0000 7f3be9c67700 1 --2- 172.21.15.44:0/2960172519 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560fb811e360 0x560fb8120780 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7eceeec88c3736d server_cookie=58c292ad394e904c in_seq=0 out_seq=0 2023-09-10T00:59:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.749+0000 7f3bcdffb700 1 -- 172.21.15.44:0/2960172519 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560fb806b840 con 0x560fb811e360 2023-09-10T00:59:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.749+0000 7f3bcdffb700 10 client.18216 handle_command_reply: tid=0 2023-09-10T00:59:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.749+0000 7f3bf8892b80 1 client.18216 shutdown 2023-09-10T00:59:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.750+0000 7f3bf8892b80 1 -- 172.21.15.44:0/2960172519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104300 msgr2=0x560fb8110d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.750+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/2960172519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104300 0x560fb8110d40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3be00098e0 tx=0x7f3be0009ca0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.750+0000 7f3bf8892b80 1 -- 172.21.15.44:0/2960172519 shutdown_connections 2023-09-10T00:59:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.750+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/2960172519 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560fb811e360 0x560fb8120780 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.750+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/2960172519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb810a1a0 0x560fb810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.750+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/2960172519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104300 0x560fb8110d40 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.750+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/2960172519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104cc0 0x560fb810dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.750+0000 7f3bf8892b80 1 -- 172.21.15.44:0/2960172519 >> 172.21.15.44:0/2960172519 conn(0x560fb8079030 msgr2=0x560fb80fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.750+0000 7f3bcdffb700 0 client.18216 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.750+0000 7f3bf8892b80 1 -- 172.21.15.44:0/2960172519 shutdown_connections 2023-09-10T00:59:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.750+0000 7f3bf8892b80 1 -- 172.21.15.44:0/2960172519 wait complete. 2023-09-10T00:59:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.750+0000 7f3bf8892b80 20 client.18216 trim_cache size 0 max 16384 2023-09-10T00:59:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.776+0000 7f3bf8892b80 1 Processor -- start 2023-09-10T00:59:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.776+0000 7f3bf8892b80 1 -- start start 2023-09-10T00:59:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.777+0000 7f3bf8892b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 0x560fb8109790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.777+0000 7f3bf8892b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104cc0 0x560fb8115530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.777+0000 7f3bf8892b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb810a1a0 0x560fb81108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.777+0000 7f3bf8892b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fb80f6f80 con 0x560fb8104cc0 2023-09-10T00:59:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.777+0000 7f3bf8892b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fb8100d90 con 0x560fb8104300 2023-09-10T00:59:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.777+0000 7f3bf8892b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fb810f5a0 con 0x560fb810a1a0 2023-09-10T00:59:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.777+0000 7f3be9c67700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104cc0 0x560fb8115530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.777+0000 7f3bea468700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 0x560fb8109790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.777+0000 7f3be9c67700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104cc0 0x560fb8115530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54494/0 (socket says 172.21.15.44:54494) 2023-09-10T00:59:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.777+0000 7f3be9c67700 1 -- 172.21.15.44:0/1465960089 learned_addr learned my addr 172.21.15.44:0/1465960089 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.777+0000 7f3beac69700 1 --2- 172.21.15.44:0/1465960089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb810a1a0 0x560fb81108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.777+0000 7f3bea468700 1 -- 172.21.15.44:0/1465960089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb810a1a0 msgr2=0x560fb81108e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.777+0000 7f3bea468700 1 --2- 172.21.15.44:0/1465960089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb810a1a0 0x560fb81108e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.777+0000 7f3bea468700 1 -- 172.21.15.44:0/1465960089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104cc0 msgr2=0x560fb8115530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.777+0000 7f3bea468700 1 --2- 172.21.15.44:0/1465960089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104cc0 0x560fb8115530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.777+0000 7f3bea468700 1 -- 172.21.15.44:0/1465960089 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fb7e6bc00 con 0x560fb8104300 2023-09-10T00:59:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.778+0000 7f3bea468700 1 --2- 172.21.15.44:0/1465960089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 0x560fb8109790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009990 tx=0x7f3be0005810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c69563d9d9713e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.778+0000 7f3bd97fa700 1 -- 172.21.15.44:0/1465960089 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be000be10 con 0x560fb8104300 2023-09-10T00:59:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.778+0000 7f3bd97fa700 1 -- 172.21.15.44:0/1465960089 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3be0028040 con 0x560fb8104300 2023-09-10T00:59:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.778+0000 7f3bd97fa700 1 -- 172.21.15.44:0/1465960089 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0024920 con 0x560fb8104300 2023-09-10T00:59:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.779+0000 7f3bf8892b80 1 -- 172.21.15.44:0/1465960089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 msgr2=0x560fb8109790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.779+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/1465960089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 0x560fb8109790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009990 tx=0x7f3be0005810 comp rx=0 tx=0).stop 2023-09-10T00:59:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.779+0000 7f3bf8892b80 1 -- 172.21.15.44:0/1465960089 shutdown_connections 2023-09-10T00:59:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.779+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/1465960089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb810a1a0 0x560fb81108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.779+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/1465960089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 0x560fb8109790 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.779+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/1465960089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104cc0 0x560fb8115530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.779+0000 7f3bf8892b80 1 -- 172.21.15.44:0/1465960089 >> 172.21.15.44:0/1465960089 conn(0x560fb8079030 msgr2=0x560fb8109080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.779+0000 7f3bf8892b80 1 -- 172.21.15.44:0/1465960089 shutdown_connections 2023-09-10T00:59:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.779+0000 7f3bf8892b80 1 -- 172.21.15.44:0/1465960089 wait complete. 2023-09-10T00:59:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.779+0000 7f3bf8892b80 1 -- start start 2023-09-10T00:59:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.779+0000 7f3bf8892b80 1 Processor -- start 2023-09-10T00:59:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bf8892b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 0x560fb816aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bf8892b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104cc0 0x560fb816afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bf8892b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb810a1a0 0x560fb81148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bf8892b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fb8114ec0 con 0x560fb8104cc0 2023-09-10T00:59:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bf8892b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fb8115030 con 0x560fb8104300 2023-09-10T00:59:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bf8892b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fb81151f0 con 0x560fb810a1a0 2023-09-10T00:59:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bf8892b80 10 client.0 _ll_register_callbacks cb 0x7f3be0004500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:59:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bea468700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 0x560fb816aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3be9c67700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104cc0 0x560fb816afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bea468700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 0x560fb816aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56238/0 (socket says 172.21.15.44:56238) 2023-09-10T00:59:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bea468700 1 -- 172.21.15.44:0/3475271219 learned_addr learned my addr 172.21.15.44:0/3475271219 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3beac69700 1 --2- 172.21.15.44:0/3475271219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb810a1a0 0x560fb81148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bea468700 1 -- 172.21.15.44:0/3475271219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb810a1a0 msgr2=0x560fb81148e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bea468700 1 --2- 172.21.15.44:0/3475271219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb810a1a0 0x560fb81148e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bea468700 1 -- 172.21.15.44:0/3475271219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104cc0 msgr2=0x560fb816afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bea468700 1 --2- 172.21.15.44:0/3475271219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104cc0 0x560fb816afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bea468700 1 -- 172.21.15.44:0/3475271219 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bdc009b90 con 0x560fb8104300 2023-09-10T00:59:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bea468700 1 --2- 172.21.15.44:0/3475271219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 0x560fb816aa80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009350 tx=0x7f3be0020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8520916425fee43 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:59:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.780+0000 7f3bcdffb700 1 -- 172.21.15.44:0/3475271219 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0024a30 con 0x560fb8104300 2023-09-10T00:59:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.781+0000 7f3bcdffb700 1 -- 172.21.15.44:0/3475271219 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f3be000f7f0 con 0x560fb8104300 2023-09-10T00:59:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.781+0000 7f3bf8892b80 1 -- 172.21.15.44:0/3475271219 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3bdc00ce60 con 0x560fb8104300 2023-09-10T00:59:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.781+0000 7f3bcdffb700 1 -- 172.21.15.44:0/3475271219 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3be000f9b0 con 0x560fb8104300 2023-09-10T00:59:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.781+0000 7f3bf8892b80 10 client.27875 fetch_fsmap learned FSMap version 83 2023-09-10T00:59:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.781+0000 7f3bf8892b80 1 -- 172.21.15.44:0/3475271219 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x560fb7ea91f0 con 0x560fb8104300 2023-09-10T00:59:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.781+0000 7f3bcdffb700 1 -- 172.21.15.44:0/3475271219 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3bdc00ce60 con 0x560fb8104300 2023-09-10T00:59:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.781+0000 7f3bcdffb700 1 -- 172.21.15.44:0/3475271219 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 83) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3be0052080 con 0x560fb8104300 2023-09-10T00:59:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.781+0000 7f3bf8892b80 10 client.27875 fetch_fsmap finished waiting for FSMap version 83 2023-09-10T00:59:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.781+0000 7f3bf8892b80 10 client.27875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:59:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.781+0000 7f3bf8892b80 20 client.27875 populate_metadata read hostname 'smithi044' 2023-09-10T00:59:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.781+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/3475271219 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560fb81723d0 0x560fb81747c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.781+0000 7f3bf8892b80 4 client.27875 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:59:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.781+0000 7f3bf8892b80 1 -- 172.21.15.44:0/3475271219 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560fb804f8c0 con 0x560fb81723d0 2023-09-10T00:59:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.781+0000 7f3be9c67700 1 --2- 172.21.15.44:0/3475271219 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560fb81723d0 0x560fb81747c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.782+0000 7f3bcdffb700 10 client.27875 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.782+0000 7f3be9c67700 1 --2- 172.21.15.44:0/3475271219 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560fb81723d0 0x560fb81747c0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aec65dca0769ab0 server_cookie=26020cd00926ce69 in_seq=0 out_seq=0 2023-09-10T00:59:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.794+0000 7f3bcdffb700 1 -- 172.21.15.44:0/3475271219 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560fb804f8c0 con 0x560fb81723d0 2023-09-10T00:59:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.794+0000 7f3bcdffb700 10 client.27875 handle_command_reply: tid=0 2023-09-10T00:59:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.794+0000 7f3bf8892b80 1 client.27875 shutdown 2023-09-10T00:59:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.794+0000 7f3bf8892b80 1 -- 172.21.15.44:0/3475271219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 msgr2=0x560fb816aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/3475271219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 0x560fb816aa80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009350 tx=0x7f3be0020000 comp rx=0 tx=0).stop 2023-09-10T00:59:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bf8892b80 1 -- 172.21.15.44:0/3475271219 shutdown_connections 2023-09-10T00:59:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/3475271219 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560fb81723d0 0x560fb81747c0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/3475271219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb810a1a0 0x560fb81148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/3475271219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fb8104300 0x560fb816aa80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bf8892b80 1 --2- 172.21.15.44:0/3475271219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560fb8104cc0 0x560fb816afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bcdffb700 0 client.27875 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bf8892b80 1 -- 172.21.15.44:0/3475271219 >> 172.21.15.44:0/3475271219 conn(0x560fb8079030 msgr2=0x560fb81032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bf8892b80 1 -- 172.21.15.44:0/3475271219 shutdown_connections 2023-09-10T00:59:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bf8892b80 1 -- 172.21.15.44:0/3475271219 wait complete. 2023-09-10T00:59:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bf8892b80 20 client.27875 trim_cache size 0 max 16384 2023-09-10T00:59:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:59:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:59:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:59:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:59:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bec6cc700 1 -- 172.21.15.44:0/379074787 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3bd00a07b0 msgr2=0x7f3bd00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bec6cc700 1 --2- 172.21.15.44:0/379074787 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3bd00a07b0 0x7f3bd00a2c70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3be0006010 tx=0x7f3be000b260 comp rx=0 tx=0).stop 2023-09-10T00:59:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bec6cc700 1 -- 172.21.15.44:0/379074787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be41098c0 msgr2=0x7f3be419e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bec6cc700 1 --2- 172.21.15.44:0/379074787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be41098c0 0x7f3be419e130 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400e9d0 tx=0x7f3bd400ece0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bec6cc700 1 -- 172.21.15.44:0/379074787 shutdown_connections 2023-09-10T00:59:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bec6cc700 1 --2- 172.21.15.44:0/379074787 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3bd00a07b0 0x7f3bd00a2c70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bec6cc700 1 --2- 172.21.15.44:0/379074787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4112550 0x7f3be41ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bec6cc700 1 --2- 172.21.15.44:0/379074787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be4108fa0 0x7f3be419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bec6cc700 1 --2- 172.21.15.44:0/379074787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be41098c0 0x7f3be419e130 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.795+0000 7f3bec6cc700 1 -- 172.21.15.44:0/379074787 >> 172.21.15.44:0/379074787 conn(0x7f3be406c6c0 msgr2=0x7f3be406efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.796+0000 7f3bec6cc700 1 -- 172.21.15.44:0/379074787 shutdown_connections 2023-09-10T00:59:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:29.796+0000 7f3bec6cc700 1 -- 172.21.15.44:0/379074787 wait complete. 2023-09-10T00:59:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:30 smithi131 ceph-mon[103713]: pgmap v596: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2023-09-10T00:59:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:30 smithi044 ceph-mon[98676]: pgmap v596: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2023-09-10T00:59:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:30 smithi099 ceph-mon[102692]: pgmap v596: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2023-09-10T00:59:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:32 smithi131 ceph-mon[103713]: pgmap v597: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 300 op/s 2023-09-10T00:59:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:32 smithi044 ceph-mon[98676]: pgmap v597: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 300 op/s 2023-09-10T00:59:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:32 smithi099 ceph-mon[102692]: pgmap v597: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 300 op/s 2023-09-10T00:59:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:34 smithi131 ceph-mon[103713]: pgmap v598: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 193 op/s 2023-09-10T00:59:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:34 smithi044 ceph-mon[98676]: pgmap v598: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 193 op/s 2023-09-10T00:59:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:34 smithi099 ceph-mon[102692]: pgmap v598: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 193 op/s 2023-09-10T00:59:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:36 smithi131 ceph-mon[103713]: pgmap v599: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 304 op/s 2023-09-10T00:59:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:36 smithi044 ceph-mon[98676]: pgmap v599: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 304 op/s 2023-09-10T00:59:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:36 smithi099 ceph-mon[102692]: pgmap v599: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 304 op/s 2023-09-10T00:59:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:38 smithi131 ceph-mon[103713]: pgmap v600: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-09-10T00:59:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:38 smithi044 ceph-mon[98676]: pgmap v600: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-09-10T00:59:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:38 smithi099 ceph-mon[102692]: pgmap v600: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-09-10T00:59:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:40 smithi131 ceph-mon[103713]: pgmap v601: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-09-10T00:59:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:40 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:59:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:40 smithi044 ceph-mon[98676]: pgmap v601: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-09-10T00:59:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:40 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:59:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:40 smithi099 ceph-mon[102692]: pgmap v601: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-09-10T00:59:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:40 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:59:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.952+0000 7f4aa34d8700 1 -- 172.21.15.44:0/2518453666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9c1055b0 msgr2=0x7f4a9c105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.952+0000 7f4aa34d8700 1 --2- 172.21.15.44:0/2518453666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9c1055b0 0x7f4a9c105990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009990 tx=0x7f4a98009ca0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.952+0000 7f4aa34d8700 1 -- 172.21.15.44:0/2518453666 shutdown_connections 2023-09-10T00:59:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.952+0000 7f4aa34d8700 1 --2- 172.21.15.44:0/2518453666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9c100120 0x7f4a9c10c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.952+0000 7f4aa34d8700 1 --2- 172.21.15.44:0/2518453666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9c1055b0 0x7f4a9c105990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.952+0000 7f4aa34d8700 1 --2- 172.21.15.44:0/2518453666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c0ff760 0x7f4a9c0ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.952+0000 7f4aa34d8700 1 -- 172.21.15.44:0/2518453666 >> 172.21.15.44:0/2518453666 conn(0x7f4a9c074f30 msgr2=0x7f4a9c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.953+0000 7f4aa34d8700 1 -- 172.21.15.44:0/2518453666 shutdown_connections 2023-09-10T00:59:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.953+0000 7f4aa34d8700 1 -- 172.21.15.44:0/2518453666 wait complete. 2023-09-10T00:59:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.953+0000 7f4aa34d8700 1 Processor -- start 2023-09-10T00:59:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.953+0000 7f4aa34d8700 1 -- start start 2023-09-10T00:59:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa34d8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c0ff760 0x7f4a9c197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa34d8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9c100120 0x7f4a9c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa34d8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9c1055b0 0x7f4a9c1a24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa34d8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c1a2a30 con 0x7f4a9c0ff760 2023-09-10T00:59:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa34d8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c1a2ba0 con 0x7f4a9c1055b0 2023-09-10T00:59:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa34d8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c1a2d60 con 0x7f4a9c100120 2023-09-10T00:59:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa1274700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c0ff760 0x7f4a9c197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa1a75700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9c1055b0 0x7f4a9c1a24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa1a75700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9c1055b0 0x7f4a9c1a24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56256/0 (socket says 172.21.15.44:56256) 2023-09-10T00:59:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa0a73700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9c100120 0x7f4a9c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa1a75700 1 -- 172.21.15.44:0/472407653 learned_addr learned my addr 172.21.15.44:0/472407653 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa1274700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c0ff760 0x7f4a9c197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54512/0 (socket says 172.21.15.44:54512) 2023-09-10T00:59:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa1a75700 1 -- 172.21.15.44:0/472407653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9c100120 msgr2=0x7f4a9c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa1a75700 1 --2- 172.21.15.44:0/472407653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9c100120 0x7f4a9c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa1a75700 1 -- 172.21.15.44:0/472407653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c0ff760 msgr2=0x7f4a9c197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa1a75700 1 --2- 172.21.15.44:0/472407653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c0ff760 0x7f4a9c197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa1a75700 1 -- 172.21.15.44:0/472407653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a980095f0 con 0x7f4a9c1055b0 2023-09-10T00:59:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.954+0000 7f4aa1a75700 1 --2- 172.21.15.44:0/472407653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9c1055b0 0x7f4a9c1a24c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4a94007ee0 tx=0x7f4a9400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.955+0000 7f4a927fc700 1 -- 172.21.15.44:0/472407653 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9400dd20 con 0x7f4a9c1055b0 2023-09-10T00:59:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.955+0000 7f4a927fc700 1 -- 172.21.15.44:0/472407653 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a9400f040 con 0x7f4a9c1055b0 2023-09-10T00:59:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.955+0000 7f4aa34d8700 1 -- 172.21.15.44:0/472407653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9c1a3040 con 0x7f4a9c1055b0 2023-09-10T00:59:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.955+0000 7f4a927fc700 1 -- 172.21.15.44:0/472407653 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94004750 con 0x7f4a9c1055b0 2023-09-10T00:59:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.955+0000 7f4aa34d8700 1 -- 172.21.15.44:0/472407653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9c1a3520 con 0x7f4a9c1055b0 2023-09-10T00:59:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.957+0000 7f4a927fc700 1 -- 172.21.15.44:0/472407653 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f4a94003a00 con 0x7f4a9c1055b0 2023-09-10T00:59:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.958+0000 7f4a927fc700 1 --2- 172.21.15.44:0/472407653 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4a880a0830 0x7f4a880a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.958+0000 7f4a927fc700 1 -- 172.21.15.44:0/472407653 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f4a940cbbb0 con 0x7f4a9c1055b0 2023-09-10T00:59:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.958+0000 7f4aa1274700 1 --2- 172.21.15.44:0/472407653 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4a880a0830 0x7f4a880a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.958+0000 7f4aa1274700 1 --2- 172.21.15.44:0/472407653 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4a880a0830 0x7f4a880a2cf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800b2e0 tx=0x7f4a98005fb0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.959+0000 7f4aaf69eb80 1 Processor -- start 2023-09-10T00:59:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.959+0000 7f4aaf69eb80 1 -- start start 2023-09-10T00:59:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.959+0000 7f4aaf69eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fdc1a0 0x558902fdc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.959+0000 7f4aaf69eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 0x558902fd6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.959+0000 7f4aaf69eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6cc0 0x558902fe2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.959+0000 7f4aaf69eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558902f2f6d0 con 0x558902fdc1a0 2023-09-10T00:59:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.959+0000 7f4aaf69eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558902f2cff0 con 0x558902fd6cc0 2023-09-10T00:59:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4aaf69eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558902f288e0 con 0x558902fd6300 2023-09-10T00:59:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4aa1274700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fdc1a0 0x558902fdc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4aa0a73700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 0x558902fd6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4aa1274700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fdc1a0 0x558902fdc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54520/0 (socket says 172.21.15.44:54520) 2023-09-10T00:59:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4aa1274700 1 -- 172.21.15.44:0/3870350739 learned_addr learned my addr 172.21.15.44:0/3870350739 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4aa0a73700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 0x558902fd6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33172/0 (socket says 172.21.15.44:33172) 2023-09-10T00:59:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4aa1a75700 1 --2- 172.21.15.44:0/3870350739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6cc0 0x558902fe2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4aa1274700 1 -- 172.21.15.44:0/3870350739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 msgr2=0x558902fd6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4aa1274700 1 --2- 172.21.15.44:0/3870350739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 0x558902fd6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4aa1274700 1 -- 172.21.15.44:0/3870350739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6cc0 msgr2=0x558902fe2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4aa1274700 1 --2- 172.21.15.44:0/3870350739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6cc0 0x558902fe2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4aa1274700 1 -- 172.21.15.44:0/3870350739 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558902d54d80 con 0x558902fdc1a0 2023-09-10T00:59:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4aa0a73700 1 --2- 172.21.15.44:0/3870350739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 0x558902fd6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:59:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4aa1274700 1 --2- 172.21.15.44:0/3870350739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fdc1a0 0x558902fdc580 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800b7b0 tx=0x7f4a98020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c7247e273f5d3d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4a877fe700 1 -- 172.21.15.44:0/3870350739 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9802c070 con 0x558902fdc1a0 2023-09-10T00:59:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4a877fe700 1 -- 172.21.15.44:0/3870350739 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a9801e040 con 0x558902fdc1a0 2023-09-10T00:59:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.960+0000 7f4a877fe700 1 -- 172.21.15.44:0/3870350739 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9800f660 con 0x558902fdc1a0 2023-09-10T00:59:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.961+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/3870350739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fdc1a0 msgr2=0x558902fdc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.961+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/3870350739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fdc1a0 0x558902fdc580 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800b7b0 tx=0x7f4a98020000 comp rx=0 tx=0).stop 2023-09-10T00:59:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.961+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/3870350739 shutdown_connections 2023-09-10T00:59:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.961+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/3870350739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6cc0 0x558902fe2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.961+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/3870350739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fdc1a0 0x558902fdc580 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.961+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/3870350739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 0x558902fd6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.961+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/3870350739 >> 172.21.15.44:0/3870350739 conn(0x558902f4b030 msgr2=0x558902fd1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.961+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/3870350739 shutdown_connections 2023-09-10T00:59:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.961+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/3870350739 wait complete. 2023-09-10T00:59:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.962+0000 7f4aaf69eb80 1 -- start start 2023-09-10T00:59:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.962+0000 7f4aaf69eb80 1 Processor -- start 2023-09-10T00:59:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.962+0000 7f4aaf69eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6300 0x558902fe2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.962+0000 7f4aaf69eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fd6cc0 0x558902fe0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.962+0000 7f4aaf69eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fdc1a0 0x558902fe0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.962+0000 7f4aaf69eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558902ec6b10 con 0x558902fd6cc0 2023-09-10T00:59:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.962+0000 7f4aaf69eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558902ec6810 con 0x558902fd6300 2023-09-10T00:59:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.962+0000 7f4aaf69eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558902daa8c0 con 0x558902fdc1a0 2023-09-10T00:59:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.962+0000 7f4aaf69eb80 10 client.0 _ll_register_callbacks cb 0x558902e10a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:59:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.963+0000 7f4aa1274700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6300 0x558902fe2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.963+0000 7f4aa1274700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6300 0x558902fe2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56266/0 (socket says 172.21.15.44:56266) 2023-09-10T00:59:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.963+0000 7f4aa1274700 1 -- 172.21.15.44:0/1655795138 learned_addr learned my addr 172.21.15.44:0/1655795138 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.963+0000 7f4aa0a73700 1 --2- 172.21.15.44:0/1655795138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fd6cc0 0x558902fe0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.963+0000 7f4aa1a75700 1 --2- 172.21.15.44:0/1655795138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fdc1a0 0x558902fe0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.963+0000 7f4aa1274700 1 -- 172.21.15.44:0/1655795138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fdc1a0 msgr2=0x558902fe0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.963+0000 7f4aa1274700 1 --2- 172.21.15.44:0/1655795138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fdc1a0 0x558902fe0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.963+0000 7f4aa1274700 1 -- 172.21.15.44:0/1655795138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fd6cc0 msgr2=0x558902fe0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.963+0000 7f4aa1274700 1 --2- 172.21.15.44:0/1655795138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fd6cc0 0x558902fe0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.963+0000 7f4aa1274700 1 -- 172.21.15.44:0/1655795138 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8c0098e0 con 0x558902fd6300 2023-09-10T00:59:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.963+0000 7f4aa1274700 1 --2- 172.21.15.44:0/1655795138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6300 0x558902fe2eb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009ce0 tx=0x7f4a98031690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ce30763c66910a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.963+0000 7f4a84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:59:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.963+0000 7f4a84ff9700 1 -- 172.21.15.44:0/1655795138 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9802c050 con 0x558902fd6300 2023-09-10T00:59:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.963+0000 7f4a84ff9700 1 -- 172.21.15.44:0/1655795138 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f4a98031930 con 0x558902fd6300 2023-09-10T00:59:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.963+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/1655795138 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558902e80490 con 0x558902fd6300 2023-09-10T00:59:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.964+0000 7f4a84ff9700 1 -- 172.21.15.44:0/1655795138 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a9800bed0 con 0x558902fd6300 2023-09-10T00:59:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.964+0000 7f4aaf69eb80 10 client.27887 fetch_fsmap learned FSMap version 84 2023-09-10T00:59:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.964+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/1655795138 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x7f4a9c10cbc0 con 0x558902fd6300 2023-09-10T00:59:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.964+0000 7f4a84ff9700 1 -- 172.21.15.44:0/1655795138 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558902e80490 con 0x558902fd6300 2023-09-10T00:59:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.964+0000 7f4a84ff9700 1 -- 172.21.15.44:0/1655795138 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 84) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4a9803abd0 con 0x558902fd6300 2023-09-10T00:59:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.964+0000 7f4aaf69eb80 10 client.27887 fetch_fsmap finished waiting for FSMap version 84 2023-09-10T00:59:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.964+0000 7f4aaf69eb80 10 client.27887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:59:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.964+0000 7f4aaf69eb80 20 client.27887 populate_metadata read hostname 'smithi044' 2023-09-10T00:59:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.964+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/1655795138 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558902ff0020 0x558902ff2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.964+0000 7f4aaf69eb80 4 client.27887 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:59:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.964+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/1655795138 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558902f3d840 con 0x558902ff0020 2023-09-10T00:59:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.964+0000 7f4aa0a73700 1 --2- 172.21.15.44:0/1655795138 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558902ff0020 0x558902ff2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.965+0000 7f4a84ff9700 10 client.27887 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.965+0000 7f4aa0a73700 1 --2- 172.21.15.44:0/1655795138 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558902ff0020 0x558902ff2440 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef84f416c55a1b79 server_cookie=3e785d05a665b91d in_seq=0 out_seq=0 2023-09-10T00:59:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.965+0000 7f4a84ff9700 1 -- 172.21.15.44:0/1655795138 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558902f3d840 con 0x558902ff0020 2023-09-10T00:59:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.965+0000 7f4a84ff9700 10 client.27887 handle_command_reply: tid=0 2023-09-10T00:59:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.965+0000 7f4aaf69eb80 1 client.27887 shutdown 2023-09-10T00:59:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.966+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/1655795138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6300 msgr2=0x558902fe2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.966+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/1655795138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6300 0x558902fe2eb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009ce0 tx=0x7f4a98031690 comp rx=0 tx=0).stop 2023-09-10T00:59:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.966+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/1655795138 shutdown_connections 2023-09-10T00:59:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.966+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/1655795138 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558902ff0020 0x558902ff2440 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.966+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/1655795138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fdc1a0 0x558902fe0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.966+0000 7f4a84ff9700 0 client.27887 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.966+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/1655795138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6300 0x558902fe2eb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.966+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/1655795138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fd6cc0 0x558902fe0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.966+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/1655795138 >> 172.21.15.44:0/1655795138 conn(0x558902f4b030 msgr2=0x558902fd07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.966+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/1655795138 shutdown_connections 2023-09-10T00:59:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.966+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/1655795138 wait complete. 2023-09-10T00:59:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.966+0000 7f4aaf69eb80 20 client.27887 trim_cache size 0 max 16384 2023-09-10T00:59:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aaf69eb80 1 Processor -- start 2023-09-10T00:59:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aaf69eb80 1 -- start start 2023-09-10T00:59:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aaf69eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 0x558902fe2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aaf69eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fd6cc0 0x558902fd1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aaf69eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fdc1a0 0x558902fd15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aaf69eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558902f3d840 con 0x558902fd6cc0 2023-09-10T00:59:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aaf69eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558902e80490 con 0x558902fdc1a0 2023-09-10T00:59:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aaf69eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558902fcb8a0 con 0x558902fd6300 2023-09-10T00:59:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aa0a73700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fd6cc0 0x558902fd1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aa0a73700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fd6cc0 0x558902fd1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54536/0 (socket says 172.21.15.44:54536) 2023-09-10T00:59:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aa0a73700 1 -- 172.21.15.44:0/389567032 learned_addr learned my addr 172.21.15.44:0/389567032 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aa1a75700 1 --2- 172.21.15.44:0/389567032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fdc1a0 0x558902fd15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aa1274700 1 --2- 172.21.15.44:0/389567032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 0x558902fe2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aa0a73700 1 -- 172.21.15.44:0/389567032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 msgr2=0x558902fe2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aa0a73700 1 --2- 172.21.15.44:0/389567032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 0x558902fe2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aa0a73700 1 -- 172.21.15.44:0/389567032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fdc1a0 msgr2=0x558902fd15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aa0a73700 1 --2- 172.21.15.44:0/389567032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fdc1a0 0x558902fd15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aa0a73700 1 -- 172.21.15.44:0/389567032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558902d3dc00 con 0x558902fd6cc0 2023-09-10T00:59:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.985+0000 7f4aa1a75700 1 --2- 172.21.15.44:0/389567032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fdc1a0 0x558902fd15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:59:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.986+0000 7f4aa0a73700 1 --2- 172.21.15.44:0/389567032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fd6cc0 0x558902fd1090 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x558902fd1060 tx=0x7f4a8c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65979f57745f863f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.986+0000 7f4a87fff700 1 -- 172.21.15.44:0/389567032 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c017040 con 0x558902fd6cc0 2023-09-10T00:59:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.986+0000 7f4a87fff700 1 -- 172.21.15.44:0/389567032 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a8c025070 con 0x558902fd6cc0 2023-09-10T00:59:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.986+0000 7f4a87fff700 1 -- 172.21.15.44:0/389567032 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0128f0 con 0x558902fd6cc0 2023-09-10T00:59:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.987+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/389567032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fd6cc0 msgr2=0x558902fd1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.987+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/389567032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fd6cc0 0x558902fd1090 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x558902fd1060 tx=0x7f4a8c009260 comp rx=0 tx=0).stop 2023-09-10T00:59:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.987+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/389567032 shutdown_connections 2023-09-10T00:59:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.987+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/389567032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fdc1a0 0x558902fd15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.987+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/389567032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 0x558902fe2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.987+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/389567032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fd6cc0 0x558902fd1090 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.987+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/389567032 >> 172.21.15.44:0/389567032 conn(0x558902f4b030 msgr2=0x558902fda1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.987+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/389567032 shutdown_connections 2023-09-10T00:59:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.987+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/389567032 wait complete. 2023-09-10T00:59:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.987+0000 7f4aaf69eb80 1 -- start start 2023-09-10T00:59:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.987+0000 7f4aaf69eb80 1 Processor -- start 2023-09-10T00:59:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aaf69eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 0x558902fe52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aaf69eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6cc0 0x558902fe5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aaf69eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fdc1a0 0x558903042e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aaf69eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558903047790 con 0x558902fdc1a0 2023-09-10T00:59:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aaf69eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558903047900 con 0x558902fd6cc0 2023-09-10T00:59:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aaf69eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558903047ac0 con 0x558902fd6300 2023-09-10T00:59:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aaf69eb80 10 client.0 _ll_register_callbacks cb 0x7f4a94007f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:59:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aa1a75700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fdc1a0 0x558903042e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aa1274700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 0x558902fe52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aa0a73700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6cc0 0x558902fe5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aa1a75700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fdc1a0 0x558903042e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54544/0 (socket says 172.21.15.44:54544) 2023-09-10T00:59:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aa1274700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 0x558902fe52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33190/0 (socket says 172.21.15.44:33190) 2023-09-10T00:59:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aa1a75700 1 -- 172.21.15.44:0/4193812680 learned_addr learned my addr 172.21.15.44:0/4193812680 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aa0a73700 1 -- 172.21.15.44:0/4193812680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 msgr2=0x558902fe52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aa0a73700 1 --2- 172.21.15.44:0/4193812680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 0x558902fe52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aa0a73700 1 -- 172.21.15.44:0/4193812680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fdc1a0 msgr2=0x558903042e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aa0a73700 1 --2- 172.21.15.44:0/4193812680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fdc1a0 0x558903042e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.988+0000 7f4aa0a73700 1 -- 172.21.15.44:0/4193812680 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9c1a3520 con 0x558902fd6cc0 2023-09-10T00:59:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.989+0000 7f4aa0a73700 1 --2- 172.21.15.44:0/4193812680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6cc0 0x558902fe5820 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x558902fefef0 tx=0x7f4a8c012150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6681bde905b0d1be server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.989+0000 7f4a84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:59:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.989+0000 7f4a84ff9700 1 -- 172.21.15.44:0/4193812680 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c004380 con 0x558902fd6cc0 2023-09-10T00:59:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.989+0000 7f4a84ff9700 1 -- 172.21.15.44:0/4193812680 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f4a8c0044e0 con 0x558902fd6cc0 2023-09-10T00:59:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.989+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/4193812680 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a9801e040 con 0x558902fd6cc0 2023-09-10T00:59:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.989+0000 7f4a84ff9700 1 -- 172.21.15.44:0/4193812680 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a8c0210f0 con 0x558902fd6cc0 2023-09-10T00:59:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.989+0000 7f4aaf69eb80 10 client.27893 fetch_fsmap learned FSMap version 84 2023-09-10T00:59:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.989+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/4193812680 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x7f4a8c021a00 con 0x558902fd6cc0 2023-09-10T00:59:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.989+0000 7f4a84ff9700 1 -- 172.21.15.44:0/4193812680 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a9801e040 con 0x558902fd6cc0 2023-09-10T00:59:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.989+0000 7f4a84ff9700 1 -- 172.21.15.44:0/4193812680 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 84) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4a8c02b5f0 con 0x558902fd6cc0 2023-09-10T00:59:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.989+0000 7f4aaf69eb80 10 client.27893 fetch_fsmap finished waiting for FSMap version 84 2023-09-10T00:59:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.989+0000 7f4aaf69eb80 10 client.27893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:59:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.989+0000 7f4aaf69eb80 20 client.27893 populate_metadata read hostname 'smithi044' 2023-09-10T00:59:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.989+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/4193812680 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558903043ce0 0x558903046100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.989+0000 7f4aaf69eb80 4 client.27893 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:59:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.989+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/4193812680 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558902f218c0 con 0x558903043ce0 2023-09-10T00:59:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.990+0000 7f4aa1274700 1 --2- 172.21.15.44:0/4193812680 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558903043ce0 0x558903046100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.990+0000 7f4a84ff9700 10 client.27893 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.990+0000 7f4aa1274700 1 --2- 172.21.15.44:0/4193812680 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558903043ce0 0x558903046100 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a01b0dea20102db server_cookie=61bc792232791653 in_seq=0 out_seq=0 2023-09-10T00:59:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.999+0000 7f4a84ff9700 1 -- 172.21.15.44:0/4193812680 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558902f218c0 con 0x558903043ce0 2023-09-10T00:59:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:42.999+0000 7f4a84ff9700 10 client.27893 handle_command_reply: tid=0 2023-09-10T00:59:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.000+0000 7f4aaf69eb80 1 client.27893 shutdown 2023-09-10T00:59:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.000+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/4193812680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6cc0 msgr2=0x558902fe5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.000+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/4193812680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6cc0 0x558902fe5820 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x558902fefef0 tx=0x7f4a8c012150 comp rx=0 tx=0).stop 2023-09-10T00:59:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.000+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/4193812680 shutdown_connections 2023-09-10T00:59:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.000+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/4193812680 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558903043ce0 0x558903046100 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.000+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/4193812680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558902fdc1a0 0x558903042e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.000+0000 7f4a84ff9700 0 client.27893 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.000+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/4193812680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558902fd6300 0x558902fe52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.000+0000 7f4aaf69eb80 1 --2- 172.21.15.44:0/4193812680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558902fd6cc0 0x558902fe5820 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.000+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/4193812680 >> 172.21.15.44:0/4193812680 conn(0x558902f4b030 msgr2=0x558902fdb170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.000+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/4193812680 shutdown_connections 2023-09-10T00:59:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.000+0000 7f4aaf69eb80 1 -- 172.21.15.44:0/4193812680 wait complete. 2023-09-10T00:59:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.000+0000 7f4aaf69eb80 20 client.27893 trim_cache size 0 max 16384 2023-09-10T00:59:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:59:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:59:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:59:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:59:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.001+0000 7f4aa34d8700 1 -- 172.21.15.44:0/472407653 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4a880a0830 msgr2=0x7f4a880a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.001+0000 7f4aa34d8700 1 --2- 172.21.15.44:0/472407653 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4a880a0830 0x7f4a880a2cf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800b2e0 tx=0x7f4a98005fb0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.001+0000 7f4aa34d8700 1 -- 172.21.15.44:0/472407653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9c1055b0 msgr2=0x7f4a9c1a24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.001+0000 7f4aa34d8700 1 --2- 172.21.15.44:0/472407653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9c1055b0 0x7f4a9c1a24c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4a94007ee0 tx=0x7f4a9400d3b0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.001+0000 7f4aa34d8700 1 -- 172.21.15.44:0/472407653 shutdown_connections 2023-09-10T00:59:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.001+0000 7f4aa34d8700 1 --2- 172.21.15.44:0/472407653 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4a880a0830 0x7f4a880a2cf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.001+0000 7f4aa34d8700 1 --2- 172.21.15.44:0/472407653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9c1055b0 0x7f4a9c1a24c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.001+0000 7f4aa34d8700 1 --2- 172.21.15.44:0/472407653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a9c0ff760 0x7f4a9c197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.001+0000 7f4aa34d8700 1 --2- 172.21.15.44:0/472407653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9c100120 0x7f4a9c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.001+0000 7f4aa34d8700 1 -- 172.21.15.44:0/472407653 >> 172.21.15.44:0/472407653 conn(0x7f4a9c074f30 msgr2=0x7f4a9c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.001+0000 7f4aa34d8700 1 -- 172.21.15.44:0/472407653 shutdown_connections 2023-09-10T00:59:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:43.001+0000 7f4aa34d8700 1 -- 172.21.15.44:0/472407653 wait complete. 2023-09-10T00:59:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:42 smithi131 ceph-mon[103713]: pgmap v602: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 305 op/s 2023-09-10T00:59:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:42 smithi044 ceph-mon[98676]: pgmap v602: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 305 op/s 2023-09-10T00:59:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:42 smithi099 ceph-mon[102692]: pgmap v602: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 305 op/s 2023-09-10T00:59:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:44 smithi131 ceph-mon[103713]: pgmap v603: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 210 op/s 2023-09-10T00:59:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:44 smithi044 ceph-mon[98676]: pgmap v603: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 210 op/s 2023-09-10T00:59:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:44 smithi099 ceph-mon[102692]: pgmap v603: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 210 op/s 2023-09-10T00:59:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:46 smithi131 ceph-mon[103713]: pgmap v604: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 316 op/s 2023-09-10T00:59:47.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:46 smithi044 ceph-mon[98676]: pgmap v604: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 316 op/s 2023-09-10T00:59:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:46 smithi099 ceph-mon[102692]: pgmap v604: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 316 op/s 2023-09-10T00:59:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:48 smithi131 ceph-mon[103713]: pgmap v605: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-09-10T00:59:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:48 smithi044 ceph-mon[98676]: pgmap v605: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-09-10T00:59:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:48 smithi099 ceph-mon[102692]: pgmap v605: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-09-10T00:59:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:50 smithi131 ceph-mon[103713]: pgmap v606: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-09-10T00:59:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:50 smithi044 ceph-mon[98676]: pgmap v606: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-09-10T00:59:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:50 smithi099 ceph-mon[102692]: pgmap v606: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-09-10T00:59:53.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:52 smithi131 ceph-mon[103713]: pgmap v607: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 299 op/s 2023-09-10T00:59:53.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:52 smithi044 ceph-mon[98676]: pgmap v607: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 299 op/s 2023-09-10T00:59:53.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:52 smithi099 ceph-mon[102692]: pgmap v607: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 299 op/s 2023-09-10T00:59:55.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:54 smithi131 ceph-mon[103713]: pgmap v608: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 200 op/s 2023-09-10T00:59:55.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:54 smithi044 ceph-mon[98676]: pgmap v608: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 200 op/s 2023-09-10T00:59:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:54 smithi099 ceph-mon[102692]: pgmap v608: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 200 op/s 2023-09-10T00:59:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.132+0000 7fc9c702e700 1 -- 172.21.15.44:0/2249735022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9c0100120 msgr2=0x7fc9c010c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.132+0000 7fc9c702e700 1 --2- 172.21.15.44:0/2249735022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9c0100120 0x7fc9c010c330 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fc9b800b0d0 tx=0x7fc9b800b3e0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.132+0000 7fc9c702e700 1 -- 172.21.15.44:0/2249735022 shutdown_connections 2023-09-10T00:59:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.132+0000 7fc9c702e700 1 --2- 172.21.15.44:0/2249735022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9c0100120 0x7fc9c010c330 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.132+0000 7fc9c702e700 1 --2- 172.21.15.44:0/2249735022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9c01055b0 0x7fc9c0105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.132+0000 7fc9c702e700 1 --2- 172.21.15.44:0/2249735022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9c00ff760 0x7fc9c00ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.132+0000 7fc9c702e700 1 -- 172.21.15.44:0/2249735022 >> 172.21.15.44:0/2249735022 conn(0x7fc9c0074f30 msgr2=0x7fc9c0075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.132+0000 7fc9c702e700 1 -- 172.21.15.44:0/2249735022 shutdown_connections 2023-09-10T00:59:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.132+0000 7fc9c702e700 1 -- 172.21.15.44:0/2249735022 wait complete. 2023-09-10T00:59:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c702e700 1 Processor -- start 2023-09-10T00:59:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c702e700 1 -- start start 2023-09-10T00:59:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c702e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9c00ff760 0x7fc9c01012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c702e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9c0100120 0x7fc9c01017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c702e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9c01055b0 0x7fc9c01a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c702e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c01a4d90 con 0x7fc9c00ff760 2023-09-10T00:59:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c702e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c01a4ed0 con 0x7fc9c01055b0 2023-09-10T00:59:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c702e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c01a5040 con 0x7fc9c0100120 2023-09-10T00:59:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c4dca700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9c00ff760 0x7fc9c01012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c4dca700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9c00ff760 0x7fc9c01012a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54554/0 (socket says 172.21.15.44:54554) 2023-09-10T00:59:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c4dca700 1 -- 172.21.15.44:0/1231121583 learned_addr learned my addr 172.21.15.44:0/1231121583 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c55cb700 1 --2- 172.21.15.44:0/1231121583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9c01055b0 0x7fc9c01a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9bffff700 1 --2- 172.21.15.44:0/1231121583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9c0100120 0x7fc9c01017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c4dca700 1 -- 172.21.15.44:0/1231121583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9c0100120 msgr2=0x7fc9c01017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c4dca700 1 --2- 172.21.15.44:0/1231121583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9c0100120 0x7fc9c01017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c4dca700 1 -- 172.21.15.44:0/1231121583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9c01055b0 msgr2=0x7fc9c01a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c4dca700 1 --2- 172.21.15.44:0/1231121583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9c01055b0 0x7fc9c01a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c4dca700 1 -- 172.21.15.44:0/1231121583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b8009d00 con 0x7fc9c00ff760 2023-09-10T00:59:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.133+0000 7fc9c55cb700 1 --2- 172.21.15.44:0/1231121583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9c01055b0 0x7fc9c01a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:59:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.134+0000 7fc9c4dca700 1 --2- 172.21.15.44:0/1231121583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9c00ff760 0x7fc9c01012a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400e9a0 tx=0x7fc9b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.134+0000 7fc9bdffb700 1 -- 172.21.15.44:0/1231121583 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4009790 con 0x7fc9c00ff760 2023-09-10T00:59:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.134+0000 7fc9bdffb700 1 -- 172.21.15.44:0/1231121583 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9b4010430 con 0x7fc9c00ff760 2023-09-10T00:59:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.134+0000 7fc9c702e700 1 -- 172.21.15.44:0/1231121583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c01a5320 con 0x7fc9c00ff760 2023-09-10T00:59:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.134+0000 7fc9bdffb700 1 -- 172.21.15.44:0/1231121583 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b400cea0 con 0x7fc9c00ff760 2023-09-10T00:59:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.134+0000 7fc9c702e700 1 -- 172.21.15.44:0/1231121583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c01a5830 con 0x7fc9c00ff760 2023-09-10T00:59:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.136+0000 7fc9bdffb700 1 -- 172.21.15.44:0/1231121583 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc9b40105a0 con 0x7fc9c00ff760 2023-09-10T00:59:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.137+0000 7fc9bdffb700 1 --2- 172.21.15.44:0/1231121583 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc9b00a07b0 0x7fc9b00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.137+0000 7fc9bffff700 1 --2- 172.21.15.44:0/1231121583 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc9b00a07b0 0x7fc9b00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.137+0000 7fc9bdffb700 1 -- 172.21.15.44:0/1231121583 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc9b4014070 con 0x7fc9c00ff760 2023-09-10T00:59:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.137+0000 7fc9bffff700 1 --2- 172.21.15.44:0/1231121583 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc9b00a07b0 0x7fc9b00a2c70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0102ac0 tx=0x7fc9ac009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.138+0000 7fc9d31f4b80 1 Processor -- start 2023-09-10T00:59:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.138+0000 7fc9d31f4b80 1 -- start start 2023-09-10T00:59:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.138+0000 7fc9d31f4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb61a0 0x5608dddb6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.138+0000 7fc9d31f4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb0300 0x5608dddb0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.138+0000 7fc9d31f4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0cc0 0x5608dddbcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.138+0000 7fc9d31f4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ddd096d0 con 0x5608dddb0300 2023-09-10T00:59:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.138+0000 7fc9d31f4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ddd06ff0 con 0x5608dddb0cc0 2023-09-10T00:59:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.138+0000 7fc9d31f4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ddd028e0 con 0x5608dddb61a0 2023-09-10T00:59:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.138+0000 7fc9bffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb0300 0x5608dddb0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.138+0000 7fc9bffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb0300 0x5608dddb0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54564/0 (socket says 172.21.15.44:54564) 2023-09-10T00:59:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.138+0000 7fc9bffff700 1 -- 172.21.15.44:0/3073827104 learned_addr learned my addr 172.21.15.44:0/3073827104 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.138+0000 7fc9c4dca700 1 --2- 172.21.15.44:0/3073827104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb61a0 0x5608dddb6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.138+0000 7fc9c55cb700 1 --2- 172.21.15.44:0/3073827104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0cc0 0x5608dddbcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.139+0000 7fc9bffff700 1 -- 172.21.15.44:0/3073827104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb61a0 msgr2=0x5608dddb6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.139+0000 7fc9bffff700 1 --2- 172.21.15.44:0/3073827104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb61a0 0x5608dddb6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.139+0000 7fc9bffff700 1 -- 172.21.15.44:0/3073827104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0cc0 msgr2=0x5608dddbcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.139+0000 7fc9bffff700 1 --2- 172.21.15.44:0/3073827104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0cc0 0x5608dddbcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.139+0000 7fc9bffff700 1 -- 172.21.15.44:0/3073827104 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608ddb2ed80 con 0x5608dddb0300 2023-09-10T00:59:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.139+0000 7fc9bffff700 1 --2- 172.21.15.44:0/3073827104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb0300 0x5608dddb0780 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac011da0 tx=0x7fc9ac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9140ba921c3104bd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.139+0000 7fc9aaffd700 1 -- 172.21.15.44:0/3073827104 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac004a30 con 0x5608dddb0300 2023-09-10T00:59:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.139+0000 7fc9aaffd700 1 -- 172.21.15.44:0/3073827104 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9ac0054b0 con 0x5608dddb0300 2023-09-10T00:59:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.139+0000 7fc9aaffd700 1 -- 172.21.15.44:0/3073827104 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac003a60 con 0x5608dddb0300 2023-09-10T00:59:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.140+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/3073827104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb0300 msgr2=0x5608dddb0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.140+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/3073827104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb0300 0x5608dddb0780 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac011da0 tx=0x7fc9ac014000 comp rx=0 tx=0).stop 2023-09-10T00:59:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.140+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/3073827104 shutdown_connections 2023-09-10T00:59:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.140+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/3073827104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0cc0 0x5608dddbcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.140+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/3073827104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb61a0 0x5608dddb6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.140+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/3073827104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb0300 0x5608dddb0780 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.140+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/3073827104 >> 172.21.15.44:0/3073827104 conn(0x5608ddd25030 msgr2=0x5608dddab560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.140+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/3073827104 shutdown_connections 2023-09-10T00:59:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.140+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/3073827104 wait complete. 2023-09-10T00:59:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.140+0000 7fc9d31f4b80 1 -- start start 2023-09-10T00:59:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.140+0000 7fc9d31f4b80 1 Processor -- start 2023-09-10T00:59:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9d31f4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb0300 0x5608dddbcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9d31f4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0cc0 0x5608dddb9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9d31f4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb61a0 0x5608dddba560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9d31f4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ddca0b10 con 0x5608dddb0300 2023-09-10T00:59:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9d31f4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ddca0810 con 0x5608dddb0cc0 2023-09-10T00:59:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9d31f4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ddb848c0 con 0x5608dddb61a0 2023-09-10T00:59:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9d31f4b80 10 client.0 _ll_register_callbacks cb 0x5608ddbeaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:59:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9c4dca700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb0300 0x5608dddbcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9c55cb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb61a0 0x5608dddba560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9c4dca700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb0300 0x5608dddbcd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54568/0 (socket says 172.21.15.44:54568) 2023-09-10T00:59:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9bffff700 1 --2- 172.21.15.44:0/4199760931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0cc0 0x5608dddb9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9c4dca700 1 -- 172.21.15.44:0/4199760931 learned_addr learned my addr 172.21.15.44:0/4199760931 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9bffff700 1 -- 172.21.15.44:0/4199760931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb61a0 msgr2=0x5608dddba560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9bffff700 1 --2- 172.21.15.44:0/4199760931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb61a0 0x5608dddba560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9bffff700 1 -- 172.21.15.44:0/4199760931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb0300 msgr2=0x5608dddbcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9bffff700 1 --2- 172.21.15.44:0/4199760931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb0300 0x5608dddbcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9bffff700 1 -- 172.21.15.44:0/4199760931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ac017070 con 0x5608dddb0cc0 2023-09-10T00:59:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9c4dca700 1 --2- 172.21.15.44:0/4199760931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb0300 0x5608dddbcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T00:59:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc9bffff700 1 --2- 172.21.15.44:0/4199760931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0cc0 0x5608dddb9fe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009fd0 tx=0x7fc9ac011ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32c403e97aaa555e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc99ffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:59:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.141+0000 7fc99ffff700 1 -- 172.21.15.44:0/4199760931 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac02c070 con 0x5608dddb0cc0 2023-09-10T00:59:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.142+0000 7fc99ffff700 1 -- 172.21.15.44:0/4199760931 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc9ac004940 con 0x5608dddb0cc0 2023-09-10T00:59:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.142+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/4199760931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608ddc5a490 con 0x5608dddb0cc0 2023-09-10T00:59:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.142+0000 7fc99ffff700 1 -- 172.21.15.44:0/4199760931 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9ac004b00 con 0x5608dddb0cc0 2023-09-10T00:59:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.142+0000 7fc9d31f4b80 10 client.27905 fetch_fsmap learned FSMap version 84 2023-09-10T00:59:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.142+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/4199760931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x5608ddb17c00 con 0x5608dddb0cc0 2023-09-10T00:59:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.142+0000 7fc99ffff700 1 -- 172.21.15.44:0/4199760931 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5608ddc5a490 con 0x5608dddb0cc0 2023-09-10T00:59:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.142+0000 7fc99ffff700 1 -- 172.21.15.44:0/4199760931 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 84) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc9ac024030 con 0x5608dddb0cc0 2023-09-10T00:59:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.142+0000 7fc9d31f4b80 10 client.27905 fetch_fsmap finished waiting for FSMap version 84 2023-09-10T00:59:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.142+0000 7fc9d31f4b80 10 client.27905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:59:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.142+0000 7fc9d31f4b80 20 client.27905 populate_metadata read hostname 'smithi044' 2023-09-10T00:59:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.142+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/4199760931 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5608dddca340 0x5608dddcc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.142+0000 7fc9d31f4b80 4 client.27905 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T00:59:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.142+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/4199760931 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608ddd17840 con 0x5608dddca340 2023-09-10T00:59:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.142+0000 7fc9c4dca700 1 --2- 172.21.15.44:0/4199760931 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5608dddca340 0x5608dddcc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.143+0000 7fc9c4dca700 1 --2- 172.21.15.44:0/4199760931 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5608dddca340 0x5608dddcc760 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a2ec14c9e703e2 server_cookie=4d18503a3a163471 in_seq=0 out_seq=0 2023-09-10T00:59:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.143+0000 7fc99ffff700 10 client.27905 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.143+0000 7fc99ffff700 1 -- 172.21.15.44:0/4199760931 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5608ddd17840 con 0x5608dddca340 2023-09-10T00:59:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.143+0000 7fc99ffff700 10 client.27905 handle_command_reply: tid=0 2023-09-10T00:59:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.143+0000 7fc9d31f4b80 1 client.27905 shutdown 2023-09-10T00:59:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.143+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/4199760931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0cc0 msgr2=0x5608dddb9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.143+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/4199760931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0cc0 0x5608dddb9fe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009fd0 tx=0x7fc9ac011ac0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.143+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/4199760931 shutdown_connections 2023-09-10T00:59:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.143+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/4199760931 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5608dddca340 0x5608dddcc760 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.143+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/4199760931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb61a0 0x5608dddba560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.143+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/4199760931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb0300 0x5608dddbcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.143+0000 7fc99ffff700 0 client.27905 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.143+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/4199760931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0cc0 0x5608dddb9fe0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.143+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/4199760931 >> 172.21.15.44:0/4199760931 conn(0x5608ddd25030 msgr2=0x5608dddaa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.144+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/4199760931 shutdown_connections 2023-09-10T00:59:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.144+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/4199760931 wait complete. 2023-09-10T00:59:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.144+0000 7fc9d31f4b80 20 client.27905 trim_cache size 0 max 16384 2023-09-10T00:59:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.159+0000 7fc9d31f4b80 1 Processor -- start 2023-09-10T00:59:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.159+0000 7fc9d31f4b80 1 -- start start 2023-09-10T00:59:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.159+0000 7fc9d31f4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0300 0x5608dddc1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.159+0000 7fc9d31f4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb0cc0 0x5608dddafa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.159+0000 7fc9d31f4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb61a0 0x5608dddaaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.159+0000 7fc9d31f4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ddda58a0 con 0x5608dddb61a0 2023-09-10T00:59:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.159+0000 7fc9d31f4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608dddc1a70 con 0x5608dddb0300 2023-09-10T00:59:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.159+0000 7fc9d31f4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608dddb6940 con 0x5608dddb0cc0 2023-09-10T00:59:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.159+0000 7fc9c4dca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0300 0x5608dddc1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.159+0000 7fc9c55cb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb61a0 0x5608dddaaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.159+0000 7fc9c4dca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0300 0x5608dddc1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56316/0 (socket says 172.21.15.44:56316) 2023-09-10T00:59:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.159+0000 7fc9c4dca700 1 -- 172.21.15.44:0/2836144200 learned_addr learned my addr 172.21.15.44:0/2836144200 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.159+0000 7fc9bffff700 1 --2- 172.21.15.44:0/2836144200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb0cc0 0x5608dddafa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.160+0000 7fc9c4dca700 1 -- 172.21.15.44:0/2836144200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb0cc0 msgr2=0x5608dddafa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.160+0000 7fc9c4dca700 1 --2- 172.21.15.44:0/2836144200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb0cc0 0x5608dddafa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.160+0000 7fc9c4dca700 1 -- 172.21.15.44:0/2836144200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb61a0 msgr2=0x5608dddaaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.160+0000 7fc9c4dca700 1 --2- 172.21.15.44:0/2836144200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb61a0 0x5608dddaaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.160+0000 7fc9c4dca700 1 -- 172.21.15.44:0/2836144200 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608ddb551f0 con 0x5608dddb0300 2023-09-10T00:59:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.160+0000 7fc9bffff700 1 --2- 172.21.15.44:0/2836144200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb0cc0 0x5608dddafa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T00:59:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.160+0000 7fc9c4dca700 1 --2- 172.21.15.44:0/2836144200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0300 0x5608dddc1530 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc9b40992f0 tx=0x7fc9b400c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=824451031d364508 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.160+0000 7fc9ab7fe700 1 -- 172.21.15.44:0/2836144200 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4099cf0 con 0x5608dddb0300 2023-09-10T00:59:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.160+0000 7fc9ab7fe700 1 -- 172.21.15.44:0/2836144200 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9b40995f0 con 0x5608dddb0300 2023-09-10T00:59:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.160+0000 7fc9ab7fe700 1 -- 172.21.15.44:0/2836144200 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4099a90 con 0x5608dddb0300 2023-09-10T00:59:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.161+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/2836144200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0300 msgr2=0x5608dddc1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.161+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/2836144200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0300 0x5608dddc1530 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc9b40992f0 tx=0x7fc9b400c480 comp rx=0 tx=0).stop 2023-09-10T00:59:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.161+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/2836144200 shutdown_connections 2023-09-10T00:59:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.161+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/2836144200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb61a0 0x5608dddaaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.161+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/2836144200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0300 0x5608dddc1530 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.161+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/2836144200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb0cc0 0x5608dddafa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.161+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/2836144200 >> 172.21.15.44:0/2836144200 conn(0x5608ddd25030 msgr2=0x5608dddaf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.161+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/2836144200 shutdown_connections 2023-09-10T00:59:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.161+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/2836144200 wait complete. 2023-09-10T00:59:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.161+0000 7fc9d31f4b80 1 -- start start 2023-09-10T00:59:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.161+0000 7fc9d31f4b80 1 Processor -- start 2023-09-10T00:59:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.161+0000 7fc9d31f4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0300 0x5608dde17950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.161+0000 7fc9d31f4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb0cc0 0x5608dde17e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9d31f4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb61a0 0x5608dde12100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9d31f4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608dde126e0 con 0x5608dddb61a0 2023-09-10T00:59:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9d31f4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608dde12850 con 0x5608dddb0300 2023-09-10T00:59:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9d31f4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608dde12a10 con 0x5608dddb0cc0 2023-09-10T00:59:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9d31f4b80 10 client.0 _ll_register_callbacks cb 0x7fc9b4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T00:59:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9c4dca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0300 0x5608dde17950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9c55cb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb61a0 0x5608dde12100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9c4dca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0300 0x5608dde17950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56322/0 (socket says 172.21.15.44:56322) 2023-09-10T00:59:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9c4dca700 1 -- 172.21.15.44:0/3695639431 learned_addr learned my addr 172.21.15.44:0/3695639431 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T00:59:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9bffff700 1 --2- 172.21.15.44:0/3695639431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb0cc0 0x5608dde17e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9c4dca700 1 -- 172.21.15.44:0/3695639431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb0cc0 msgr2=0x5608dde17e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9c4dca700 1 --2- 172.21.15.44:0/3695639431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb0cc0 0x5608dde17e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9c4dca700 1 -- 172.21.15.44:0/3695639431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb61a0 msgr2=0x5608dde12100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T00:59:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9c4dca700 1 --2- 172.21.15.44:0/3695639431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb61a0 0x5608dde12100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9c4dca700 1 -- 172.21.15.44:0/3695639431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c01a5830 con 0x5608dddb0300 2023-09-10T00:59:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9c4dca700 1 --2- 172.21.15.44:0/3695639431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0300 0x5608dde17950 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc9b40c9830 tx=0x7fc9b4005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68e11dc503501dc5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T00:59:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc99ffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T00:59:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc99ffff700 1 -- 172.21.15.44:0/3695639431 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4099cf0 con 0x5608dddb0300 2023-09-10T00:59:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc99ffff700 1 -- 172.21.15.44:0/3695639431 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc9b4097990 con 0x5608dddb0300 2023-09-10T00:59:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/3695639431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9ac0054b0 con 0x5608dddb0300 2023-09-10T00:59:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc99ffff700 1 -- 172.21.15.44:0/3695639431 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9b40cc130 con 0x5608dddb0300 2023-09-10T00:59:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9d31f4b80 10 client.27911 fetch_fsmap learned FSMap version 84 2023-09-10T00:59:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.162+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/3695639431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x5608ddb2d250 con 0x5608dddb0300 2023-09-10T00:59:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.163+0000 7fc99ffff700 1 -- 172.21.15.44:0/3695639431 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9ac0054b0 con 0x5608dddb0300 2023-09-10T00:59:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.163+0000 7fc99ffff700 1 -- 172.21.15.44:0/3695639431 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 84) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc9b4065110 con 0x5608dddb0300 2023-09-10T00:59:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.163+0000 7fc9d31f4b80 10 client.27911 fetch_fsmap finished waiting for FSMap version 84 2023-09-10T00:59:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.163+0000 7fc9d31f4b80 10 client.27911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T00:59:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.163+0000 7fc9d31f4b80 20 client.27911 populate_metadata read hostname 'smithi044' 2023-09-10T00:59:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.163+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/3695639431 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5608dde1f4f0 0x5608dde21910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T00:59:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.163+0000 7fc9d31f4b80 4 client.27911 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T00:59:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.163+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/3695639431 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5608ddcfb8c0 con 0x5608dde1f4f0 2023-09-10T00:59:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.163+0000 7fc9bffff700 1 --2- 172.21.15.44:0/3695639431 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5608dde1f4f0 0x5608dde21910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T00:59:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.163+0000 7fc99ffff700 10 client.27911 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.163+0000 7fc9bffff700 1 --2- 172.21.15.44:0/3695639431 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5608dde1f4f0 0x5608dde21910 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d155903d4b926a4 server_cookie=c4a92fe16c6b5440 in_seq=0 out_seq=0 2023-09-10T00:59:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.180+0000 7fc99ffff700 1 -- 172.21.15.44:0/3695639431 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5608ddcfb8c0 con 0x5608dde1f4f0 2023-09-10T00:59:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.180+0000 7fc99ffff700 10 client.27911 handle_command_reply: tid=0 2023-09-10T00:59:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.180+0000 7fc9d31f4b80 1 client.27911 shutdown 2023-09-10T00:59:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.180+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/3695639431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0300 msgr2=0x5608dde17950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.180+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/3695639431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0300 0x5608dde17950 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc9b40c9830 tx=0x7fc9b4005d30 comp rx=0 tx=0).stop 2023-09-10T00:59:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.180+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/3695639431 shutdown_connections 2023-09-10T00:59:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.180+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/3695639431 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5608dde1f4f0 0x5608dde21910 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.180+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/3695639431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608dddb61a0 0x5608dde12100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.180+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/3695639431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608dddb0300 0x5608dde17950 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.180+0000 7fc99ffff700 0 client.27911 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T00:59:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.180+0000 7fc9d31f4b80 1 --2- 172.21.15.44:0/3695639431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5608dddb0cc0 0x5608dde17e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.180+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/3695639431 >> 172.21.15.44:0/3695639431 conn(0x5608ddd25030 msgr2=0x5608dddafb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.180+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/3695639431 shutdown_connections 2023-09-10T00:59:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.180+0000 7fc9d31f4b80 1 -- 172.21.15.44:0/3695639431 wait complete. 2023-09-10T00:59:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.180+0000 7fc9d31f4b80 20 client.27911 trim_cache size 0 max 16384 2023-09-10T00:59:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T00:59:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T00:59:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T00:59:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T00:59:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.181+0000 7fc9c702e700 1 -- 172.21.15.44:0/1231121583 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc9b00a07b0 msgr2=0x7fc9b00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.181+0000 7fc9c702e700 1 --2- 172.21.15.44:0/1231121583 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc9b00a07b0 0x7fc9b00a2c70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0102ac0 tx=0x7fc9ac009260 comp rx=0 tx=0).stop 2023-09-10T00:59:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.181+0000 7fc9c702e700 1 -- 172.21.15.44:0/1231121583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9c00ff760 msgr2=0x7fc9c01012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T00:59:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.181+0000 7fc9c702e700 1 --2- 172.21.15.44:0/1231121583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9c00ff760 0x7fc9c01012a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400e9a0 tx=0x7fc9b400ecb0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.181+0000 7fc9c702e700 1 -- 172.21.15.44:0/1231121583 shutdown_connections 2023-09-10T00:59:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.181+0000 7fc9c702e700 1 --2- 172.21.15.44:0/1231121583 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc9b00a07b0 0x7fc9b00a2c70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.181+0000 7fc9c702e700 1 --2- 172.21.15.44:0/1231121583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9c01055b0 0x7fc9c01a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.181+0000 7fc9c702e700 1 --2- 172.21.15.44:0/1231121583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9c00ff760 0x7fc9c01012a0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.181+0000 7fc9c702e700 1 --2- 172.21.15.44:0/1231121583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9c0100120 0x7fc9c01017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T00:59:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.181+0000 7fc9c702e700 1 -- 172.21.15.44:0/1231121583 >> 172.21.15.44:0/1231121583 conn(0x7fc9c0074f30 msgr2=0x7fc9c0067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T00:59:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.181+0000 7fc9c702e700 1 -- 172.21.15.44:0/1231121583 shutdown_connections 2023-09-10T00:59:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T00:59:56.181+0000 7fc9c702e700 1 -- 172.21.15.44:0/1231121583 wait complete. 2023-09-10T00:59:57.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:56 smithi044 ceph-mon[98676]: pgmap v609: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 274 op/s 2023-09-10T00:59:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:56 smithi099 ceph-mon[102692]: pgmap v609: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 274 op/s 2023-09-10T00:59:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:56 smithi131 ceph-mon[103713]: pgmap v609: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 274 op/s 2023-09-10T00:59:59.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 00:59:58 smithi044 ceph-mon[98676]: pgmap v610: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2023-09-10T00:59:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 00:59:58 smithi099 ceph-mon[102692]: pgmap v610: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2023-09-10T00:59:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 00:59:58 smithi131 ceph-mon[103713]: pgmap v610: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2023-09-10T01:00:01.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:00 smithi044 ceph-mon[98676]: pgmap v611: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2023-09-10T01:00:01.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:00 smithi044 ceph-mon[98676]: overall HEALTH_OK 2023-09-10T01:00:01.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:00 smithi099 ceph-mon[102692]: pgmap v611: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2023-09-10T01:00:01.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:00 smithi099 ceph-mon[102692]: overall HEALTH_OK 2023-09-10T01:00:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:00 smithi131 ceph-mon[103713]: pgmap v611: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2023-09-10T01:00:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:00 smithi131 ceph-mon[103713]: overall HEALTH_OK 2023-09-10T01:00:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:02 smithi044 ceph-mon[98676]: pgmap v612: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2023-09-10T01:00:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:02 smithi099 ceph-mon[102692]: pgmap v612: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2023-09-10T01:00:03.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:02 smithi131 ceph-mon[103713]: pgmap v612: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2023-09-10T01:00:05.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:04 smithi044 ceph-mon[98676]: pgmap v613: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 163 op/s 2023-09-10T01:00:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:04 smithi099 ceph-mon[102692]: pgmap v613: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 163 op/s 2023-09-10T01:00:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:04 smithi131 ceph-mon[103713]: pgmap v613: 129 pgs: 129 active+clean; 8.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 163 op/s 2023-09-10T01:00:07.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:06 smithi099 ceph-mon[102692]: pgmap v614: 129 pgs: 129 active+clean; 8.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 283 op/s 2023-09-10T01:00:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:06 smithi131 ceph-mon[103713]: pgmap v614: 129 pgs: 129 active+clean; 8.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 283 op/s 2023-09-10T01:00:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:06 smithi044 ceph-mon[98676]: pgmap v614: 129 pgs: 129 active+clean; 8.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 283 op/s 2023-09-10T01:00:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:08 smithi099 ceph-mon[102692]: pgmap v615: 129 pgs: 129 active+clean; 8.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 209 op/s 2023-09-10T01:00:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:08 smithi131 ceph-mon[103713]: pgmap v615: 129 pgs: 129 active+clean; 8.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 209 op/s 2023-09-10T01:00:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:08 smithi044 ceph-mon[98676]: pgmap v615: 129 pgs: 129 active+clean; 8.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 209 op/s 2023-09-10T01:00:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.305+0000 7fe05b0ad700 1 -- 172.21.15.44:0/4142130796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0541055b0 msgr2=0x7fe054105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.305+0000 7fe05b0ad700 1 --2- 172.21.15.44:0/4142130796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0541055b0 0x7fe054105990 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe0480099c0 tx=0x7fe048009cd0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.305+0000 7fe05b0ad700 1 -- 172.21.15.44:0/4142130796 shutdown_connections 2023-09-10T01:00:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.305+0000 7fe05b0ad700 1 --2- 172.21.15.44:0/4142130796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe054100120 0x7fe05410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.305+0000 7fe05b0ad700 1 --2- 172.21.15.44:0/4142130796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0541055b0 0x7fe054105990 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.305+0000 7fe05b0ad700 1 --2- 172.21.15.44:0/4142130796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0540ff760 0x7fe0540ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.305+0000 7fe05b0ad700 1 -- 172.21.15.44:0/4142130796 >> 172.21.15.44:0/4142130796 conn(0x7fe054074f30 msgr2=0x7fe054075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.305+0000 7fe05b0ad700 1 -- 172.21.15.44:0/4142130796 shutdown_connections 2023-09-10T01:00:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.305+0000 7fe05b0ad700 1 -- 172.21.15.44:0/4142130796 wait complete. 2023-09-10T01:00:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.307+0000 7fe05b0ad700 1 Processor -- start 2023-09-10T01:00:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.307+0000 7fe05b0ad700 1 -- start start 2023-09-10T01:00:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.307+0000 7fe05b0ad700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0540ff760 0x7fe05419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.307+0000 7fe05b0ad700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe054100120 0x7fe05419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.307+0000 7fe05b0ad700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0541055b0 0x7fe054196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.307+0000 7fe05b0ad700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe054196bb0 con 0x7fe054100120 2023-09-10T01:00:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.307+0000 7fe05b0ad700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe054196d20 con 0x7fe0540ff760 2023-09-10T01:00:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.307+0000 7fe05b0ad700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe054196ee0 con 0x7fe0541055b0 2023-09-10T01:00:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.307+0000 7fe053fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe054100120 0x7fe05419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.307+0000 7fe053fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe054100120 0x7fe05419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54596/0 (socket says 172.21.15.44:54596) 2023-09-10T01:00:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.307+0000 7fe053fff700 1 -- 172.21.15.44:0/3046706968 learned_addr learned my addr 172.21.15.44:0/3046706968 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.307+0000 7fe058e49700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0540ff760 0x7fe05419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.307+0000 7fe05964a700 1 --2- 172.21.15.44:0/3046706968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0541055b0 0x7fe054196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.308+0000 7fe058e49700 1 -- 172.21.15.44:0/3046706968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0541055b0 msgr2=0x7fe054196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.308+0000 7fe058e49700 1 --2- 172.21.15.44:0/3046706968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0541055b0 0x7fe054196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.308+0000 7fe058e49700 1 -- 172.21.15.44:0/3046706968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe054100120 msgr2=0x7fe05419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.308+0000 7fe058e49700 1 --2- 172.21.15.44:0/3046706968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe054100120 0x7fe05419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.308+0000 7fe058e49700 1 -- 172.21.15.44:0/3046706968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0480095f0 con 0x7fe0540ff760 2023-09-10T01:00:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.308+0000 7fe05964a700 1 --2- 172.21.15.44:0/3046706968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0541055b0 0x7fe054196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:00:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.308+0000 7fe053fff700 1 --2- 172.21.15.44:0/3046706968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe054100120 0x7fe05419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:00:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.308+0000 7fe058e49700 1 --2- 172.21.15.44:0/3046706968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0540ff760 0x7fe05419c6e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe048009990 tx=0x7fe04800f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.308+0000 7fe051ffb700 1 -- 172.21.15.44:0/3046706968 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04801d070 con 0x7fe0540ff760 2023-09-10T01:00:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.308+0000 7fe05b0ad700 1 -- 172.21.15.44:0/3046706968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe054197160 con 0x7fe0540ff760 2023-09-10T01:00:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.308+0000 7fe051ffb700 1 -- 172.21.15.44:0/3046706968 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe048005030 con 0x7fe0540ff760 2023-09-10T01:00:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.308+0000 7fe051ffb700 1 -- 172.21.15.44:0/3046706968 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048004950 con 0x7fe0540ff760 2023-09-10T01:00:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.308+0000 7fe05b0ad700 1 -- 172.21.15.44:0/3046706968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05410e950 con 0x7fe0540ff760 2023-09-10T01:00:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.311+0000 7fe051ffb700 1 -- 172.21.15.44:0/3046706968 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fe048005410 con 0x7fe0540ff760 2023-09-10T01:00:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.311+0000 7fe051ffb700 1 --2- 172.21.15.44:0/3046706968 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe0440a4bc0 0x7fe0440a7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.312+0000 7fe051ffb700 1 -- 172.21.15.44:0/3046706968 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fe0480ccf60 con 0x7fe0540ff760 2023-09-10T01:00:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.312+0000 7fe053fff700 1 --2- 172.21.15.44:0/3046706968 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe0440a4bc0 0x7fe0440a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.312+0000 7fe067273b80 1 Processor -- start 2023-09-10T01:00:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.312+0000 7fe067273b80 1 -- start start 2023-09-10T01:00:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.312+0000 7fe053fff700 1 --2- 172.21.15.44:0/3046706968 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe0440a4bc0 0x7fe0440a7080 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe0541979b0 tx=0x7fe040009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.312+0000 7fe067273b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecec31a0 0x55e5ecec3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.312+0000 7fe067273b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecebd300 0x55e5ecebd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.312+0000 7fe067273b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebdcc0 0x55e5ecec9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.312+0000 7fe067273b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ece166d0 con 0x55e5ecec31a0 2023-09-10T01:00:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.312+0000 7fe067273b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ece13ff0 con 0x55e5ecebdcc0 2023-09-10T01:00:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.312+0000 7fe067273b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ece0f8e0 con 0x55e5ecebd300 2023-09-10T01:00:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.312+0000 7fe058e49700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecec31a0 0x55e5ecec3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.312+0000 7fe058e49700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecec31a0 0x55e5ecec3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54602/0 (socket says 172.21.15.44:54602) 2023-09-10T01:00:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.312+0000 7fe058e49700 1 -- 172.21.15.44:0/4174930762 learned_addr learned my addr 172.21.15.44:0/4174930762 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.312+0000 7fe053fff700 1 --2- 172.21.15.44:0/4174930762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecebd300 0x55e5ecebd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.313+0000 7fe05964a700 1 --2- 172.21.15.44:0/4174930762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebdcc0 0x55e5ecec9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.313+0000 7fe058e49700 1 -- 172.21.15.44:0/4174930762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecebd300 msgr2=0x55e5ecebd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.313+0000 7fe058e49700 1 --2- 172.21.15.44:0/4174930762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecebd300 0x55e5ecebd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.313+0000 7fe058e49700 1 -- 172.21.15.44:0/4174930762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebdcc0 msgr2=0x55e5ecec9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.313+0000 7fe058e49700 1 --2- 172.21.15.44:0/4174930762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebdcc0 0x55e5ecec9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.313+0000 7fe058e49700 1 -- 172.21.15.44:0/4174930762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5ecc3bd80 con 0x55e5ecec31a0 2023-09-10T01:00:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.313+0000 7fe058e49700 1 --2- 172.21.15.44:0/4174930762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecec31a0 0x55e5ecec3580 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe048096550 tx=0x7fe048099a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e009d63fcb911ed server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.313+0000 7fe03effd700 1 -- 172.21.15.44:0/4174930762 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04809de80 con 0x55e5ecec31a0 2023-09-10T01:00:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.313+0000 7fe03effd700 1 -- 172.21.15.44:0/4174930762 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe0480a2030 con 0x55e5ecec31a0 2023-09-10T01:00:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.313+0000 7fe03effd700 1 -- 172.21.15.44:0/4174930762 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0480997a0 con 0x55e5ecec31a0 2023-09-10T01:00:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.314+0000 7fe067273b80 1 -- 172.21.15.44:0/4174930762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecec31a0 msgr2=0x55e5ecec3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.314+0000 7fe067273b80 1 --2- 172.21.15.44:0/4174930762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecec31a0 0x55e5ecec3580 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe048096550 tx=0x7fe048099a30 comp rx=0 tx=0).stop 2023-09-10T01:00:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.314+0000 7fe067273b80 1 -- 172.21.15.44:0/4174930762 shutdown_connections 2023-09-10T01:00:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.314+0000 7fe067273b80 1 --2- 172.21.15.44:0/4174930762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebdcc0 0x55e5ecec9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.314+0000 7fe067273b80 1 --2- 172.21.15.44:0/4174930762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecec31a0 0x55e5ecec3580 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.314+0000 7fe067273b80 1 --2- 172.21.15.44:0/4174930762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecebd300 0x55e5ecebd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.314+0000 7fe067273b80 1 -- 172.21.15.44:0/4174930762 >> 172.21.15.44:0/4174930762 conn(0x55e5ece32030 msgr2=0x55e5eceb8560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.314+0000 7fe067273b80 1 -- 172.21.15.44:0/4174930762 shutdown_connections 2023-09-10T01:00:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.314+0000 7fe067273b80 1 -- 172.21.15.44:0/4174930762 wait complete. 2023-09-10T01:00:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.314+0000 7fe067273b80 1 -- start start 2023-09-10T01:00:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.314+0000 7fe067273b80 1 Processor -- start 2023-09-10T01:00:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.315+0000 7fe067273b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebd300 0x55e5ececccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.315+0000 7fe067273b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebdcc0 0x55e5ececd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.315+0000 7fe067273b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecec6f50 0x55e5ecec73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.315+0000 7fe067273b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ecdadb10 con 0x55e5ecebdcc0 2023-09-10T01:00:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.315+0000 7fe067273b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ecdad810 con 0x55e5ecebd300 2023-09-10T01:00:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.315+0000 7fe067273b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ecc918c0 con 0x55e5ecec6f50 2023-09-10T01:00:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.315+0000 7fe067273b80 10 client.0 _ll_register_callbacks cb 0x55e5eccf7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:00:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.315+0000 7fe058e49700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebd300 0x55e5ececccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.315+0000 7fe05964a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecec6f50 0x55e5ecec73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.315+0000 7fe058e49700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebd300 0x55e5ececccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56348/0 (socket says 172.21.15.44:56348) 2023-09-10T01:00:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.315+0000 7fe058e49700 1 -- 172.21.15.44:0/1160810044 learned_addr learned my addr 172.21.15.44:0/1160810044 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.315+0000 7fe053fff700 1 --2- 172.21.15.44:0/1160810044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebdcc0 0x55e5ececd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.315+0000 7fe058e49700 1 -- 172.21.15.44:0/1160810044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecec6f50 msgr2=0x55e5ecec73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.315+0000 7fe058e49700 1 --2- 172.21.15.44:0/1160810044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecec6f50 0x55e5ecec73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.315+0000 7fe058e49700 1 -- 172.21.15.44:0/1160810044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebdcc0 msgr2=0x55e5ececd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.315+0000 7fe058e49700 1 --2- 172.21.15.44:0/1160810044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebdcc0 0x55e5ececd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.316+0000 7fe058e49700 1 -- 172.21.15.44:0/1160810044 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05410e950 con 0x55e5ecebd300 2023-09-10T01:00:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.316+0000 7fe05964a700 1 --2- 172.21.15.44:0/1160810044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecec6f50 0x55e5ecec73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:00:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.316+0000 7fe058e49700 1 --2- 172.21.15.44:0/1160810044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebd300 0x55e5ececccc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe048099d50 tx=0x7fe0480ca220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=637cfeceb887017c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.316+0000 7fe033fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:00:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.316+0000 7fe033fff700 1 -- 172.21.15.44:0/1160810044 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04809de80 con 0x55e5ecebd300 2023-09-10T01:00:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.316+0000 7fe067273b80 1 -- 172.21.15.44:0/1160810044 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5ecd67490 con 0x55e5ecebd300 2023-09-10T01:00:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.316+0000 7fe033fff700 1 -- 172.21.15.44:0/1160810044 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fe04809a2b0 con 0x55e5ecebd300 2023-09-10T01:00:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.316+0000 7fe033fff700 1 -- 172.21.15.44:0/1160810044 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe04809a4b0 con 0x55e5ecebd300 2023-09-10T01:00:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.316+0000 7fe067273b80 10 client.27923 fetch_fsmap learned FSMap version 84 2023-09-10T01:00:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.316+0000 7fe067273b80 1 -- 172.21.15.44:0/1160810044 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x55e5ecc24c00 con 0x55e5ecebd300 2023-09-10T01:00:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.316+0000 7fe033fff700 1 -- 172.21.15.44:0/1160810044 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e5ecd67490 con 0x55e5ecebd300 2023-09-10T01:00:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.317+0000 7fe033fff700 1 -- 172.21.15.44:0/1160810044 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 84) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe04809ebc0 con 0x55e5ecebd300 2023-09-10T01:00:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.317+0000 7fe067273b80 10 client.27923 fetch_fsmap finished waiting for FSMap version 84 2023-09-10T01:00:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.317+0000 7fe067273b80 10 client.27923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:00:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.317+0000 7fe067273b80 20 client.27923 populate_metadata read hostname 'smithi044' 2023-09-10T01:00:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.317+0000 7fe067273b80 1 --2- 172.21.15.44:0/1160810044 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e5eced70c0 0x55e5eced94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.317+0000 7fe067273b80 4 client.27923 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:00:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.317+0000 7fe067273b80 1 -- 172.21.15.44:0/1160810044 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5ece24840 con 0x55e5eced70c0 2023-09-10T01:00:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.317+0000 7fe053fff700 1 --2- 172.21.15.44:0/1160810044 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e5eced70c0 0x55e5eced94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.317+0000 7fe033fff700 10 client.27923 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:00:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.317+0000 7fe053fff700 1 --2- 172.21.15.44:0/1160810044 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e5eced70c0 0x55e5eced94b0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=578a60d09d8c0e29 server_cookie=3af400f36705a0a8 in_seq=0 out_seq=0 2023-09-10T01:00:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.318+0000 7fe033fff700 1 -- 172.21.15.44:0/1160810044 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e5ece24840 con 0x55e5eced70c0 2023-09-10T01:00:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.318+0000 7fe033fff700 10 client.27923 handle_command_reply: tid=0 2023-09-10T01:00:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.318+0000 7fe067273b80 1 client.27923 shutdown 2023-09-10T01:00:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.318+0000 7fe067273b80 1 -- 172.21.15.44:0/1160810044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebd300 msgr2=0x55e5ececccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.318+0000 7fe067273b80 1 --2- 172.21.15.44:0/1160810044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebd300 0x55e5ececccc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe048099d50 tx=0x7fe0480ca220 comp rx=0 tx=0).stop 2023-09-10T01:00:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.318+0000 7fe067273b80 1 -- 172.21.15.44:0/1160810044 shutdown_connections 2023-09-10T01:00:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.318+0000 7fe067273b80 1 --2- 172.21.15.44:0/1160810044 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e5eced70c0 0x55e5eced94b0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.318+0000 7fe067273b80 1 --2- 172.21.15.44:0/1160810044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecec6f50 0x55e5ecec73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.318+0000 7fe067273b80 1 --2- 172.21.15.44:0/1160810044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebd300 0x55e5ececccc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.318+0000 7fe067273b80 1 --2- 172.21.15.44:0/1160810044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebdcc0 0x55e5ececd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.318+0000 7fe033fff700 0 client.27923 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:00:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.318+0000 7fe067273b80 1 -- 172.21.15.44:0/1160810044 >> 172.21.15.44:0/1160810044 conn(0x55e5ece32030 msgr2=0x55e5eceb77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.318+0000 7fe067273b80 1 -- 172.21.15.44:0/1160810044 shutdown_connections 2023-09-10T01:00:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.318+0000 7fe067273b80 1 -- 172.21.15.44:0/1160810044 wait complete. 2023-09-10T01:00:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.318+0000 7fe067273b80 20 client.27923 trim_cache size 0 max 16384 2023-09-10T01:00:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.334+0000 7fe067273b80 1 Processor -- start 2023-09-10T01:00:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.334+0000 7fe067273b80 1 -- start start 2023-09-10T01:00:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe067273b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecebd300 0x55e5ecec67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe067273b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebdcc0 0x55e5eced3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe067273b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecec6f50 0x55e5ecec21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe067273b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5eceaff30 con 0x55e5ecebdcc0 2023-09-10T01:00:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe067273b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ecec7eb0 con 0x55e5ecec6f50 2023-09-10T01:00:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe067273b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ecebe550 con 0x55e5ecebd300 2023-09-10T01:00:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe053fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebdcc0 0x55e5eced3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe053fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebdcc0 0x55e5eced3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54618/0 (socket says 172.21.15.44:54618) 2023-09-10T01:00:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe053fff700 1 -- 172.21.15.44:0/1605882851 learned_addr learned my addr 172.21.15.44:0/1605882851 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe058e49700 1 --2- 172.21.15.44:0/1605882851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecebd300 0x55e5ecec67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe05964a700 1 --2- 172.21.15.44:0/1605882851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecec6f50 0x55e5ecec21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe058e49700 1 -- 172.21.15.44:0/1605882851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecec6f50 msgr2=0x55e5ecec21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe058e49700 1 --2- 172.21.15.44:0/1605882851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecec6f50 0x55e5ecec21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe058e49700 1 -- 172.21.15.44:0/1605882851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebdcc0 msgr2=0x55e5eced3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe058e49700 1 --2- 172.21.15.44:0/1605882851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebdcc0 0x55e5eced3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe058e49700 1 -- 172.21.15.44:0/1605882851 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5ecc621f0 con 0x55e5ecebd300 2023-09-10T01:00:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe053fff700 1 --2- 172.21.15.44:0/1605882851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebdcc0 0x55e5eced3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:00:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.335+0000 7fe058e49700 1 --2- 172.21.15.44:0/1605882851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecebd300 0x55e5ecec67f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe0480098e0 tx=0x7fe04800b580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9553b8e981d554b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.336+0000 7fe03f7fe700 1 -- 172.21.15.44:0/1605882851 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048095f60 con 0x55e5ecebd300 2023-09-10T01:00:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.336+0000 7fe03f7fe700 1 -- 172.21.15.44:0/1605882851 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe048096390 con 0x55e5ecebd300 2023-09-10T01:00:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.336+0000 7fe03f7fe700 1 -- 172.21.15.44:0/1605882851 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04809ebc0 con 0x55e5ecebd300 2023-09-10T01:00:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.336+0000 7fe067273b80 1 -- 172.21.15.44:0/1605882851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecebd300 msgr2=0x55e5ecec67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.336+0000 7fe067273b80 1 --2- 172.21.15.44:0/1605882851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecebd300 0x55e5ecec67f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe0480098e0 tx=0x7fe04800b580 comp rx=0 tx=0).stop 2023-09-10T01:00:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.336+0000 7fe067273b80 1 -- 172.21.15.44:0/1605882851 shutdown_connections 2023-09-10T01:00:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.336+0000 7fe067273b80 1 --2- 172.21.15.44:0/1605882851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecec6f50 0x55e5ecec21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.336+0000 7fe067273b80 1 --2- 172.21.15.44:0/1605882851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecebd300 0x55e5ecec67f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.336+0000 7fe067273b80 1 --2- 172.21.15.44:0/1605882851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebdcc0 0x55e5eced3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.336+0000 7fe067273b80 1 -- 172.21.15.44:0/1605882851 >> 172.21.15.44:0/1605882851 conn(0x55e5ece32030 msgr2=0x55e5ecedde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.336+0000 7fe067273b80 1 -- 172.21.15.44:0/1605882851 shutdown_connections 2023-09-10T01:00:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.336+0000 7fe067273b80 1 -- 172.21.15.44:0/1605882851 wait complete. 2023-09-10T01:00:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.337+0000 7fe067273b80 1 -- start start 2023-09-10T01:00:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.337+0000 7fe067273b80 1 Processor -- start 2023-09-10T01:00:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.337+0000 7fe067273b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebd300 0x55e5ecf23950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.337+0000 7fe067273b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebdcc0 0x55e5ecf23e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.337+0000 7fe067273b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecec6f50 0x55e5ececc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.337+0000 7fe067273b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ececcaf0 con 0x55e5ecebd300 2023-09-10T01:00:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.337+0000 7fe067273b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ececcc60 con 0x55e5ecebdcc0 2023-09-10T01:00:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.337+0000 7fe067273b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ececce20 con 0x55e5ecec6f50 2023-09-10T01:00:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.337+0000 7fe067273b80 10 client.0 _ll_register_callbacks cb 0x7fe04c009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:00:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe05964a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecec6f50 0x55e5ececc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe058e49700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebd300 0x55e5ecf23950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe05964a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecec6f50 0x55e5ececc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33274/0 (socket says 172.21.15.44:33274) 2023-09-10T01:00:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe05964a700 1 -- 172.21.15.44:0/1129840309 learned_addr learned my addr 172.21.15.44:0/1129840309 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe053fff700 1 --2- 172.21.15.44:0/1129840309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebdcc0 0x55e5ecf23e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe05964a700 1 -- 172.21.15.44:0/1129840309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebdcc0 msgr2=0x55e5ecf23e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe05964a700 1 --2- 172.21.15.44:0/1129840309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebdcc0 0x55e5ecf23e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe05964a700 1 -- 172.21.15.44:0/1129840309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebd300 msgr2=0x55e5ecf23950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe05964a700 1 --2- 172.21.15.44:0/1129840309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebd300 0x55e5ecf23950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe05964a700 1 -- 172.21.15.44:0/1129840309 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe040017070 con 0x55e5ecec6f50 2023-09-10T01:00:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe058e49700 1 --2- 172.21.15.44:0/1129840309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebd300 0x55e5ecf23950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:00:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe05964a700 1 --2- 172.21.15.44:0/1129840309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecec6f50 0x55e5ececc520 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00ed50 tx=0x7fe04c0077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6914974be962190 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe033fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:00:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe033fff700 1 -- 172.21.15.44:0/1129840309 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c01c070 con 0x55e5ecec6f50 2023-09-10T01:00:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe067273b80 1 -- 172.21.15.44:0/1129840309 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0480a2030 con 0x55e5ecec6f50 2023-09-10T01:00:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe033fff700 1 -- 172.21.15.44:0/1129840309 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fe04c004bb0 con 0x55e5ecec6f50 2023-09-10T01:00:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe033fff700 1 -- 172.21.15.44:0/1129840309 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe04c007e50 con 0x55e5ecec6f50 2023-09-10T01:00:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.338+0000 7fe067273b80 10 client.27871 fetch_fsmap learned FSMap version 84 2023-09-10T01:00:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.339+0000 7fe067273b80 1 -- 172.21.15.44:0/1129840309 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x55e5ecc24c00 con 0x55e5ecec6f50 2023-09-10T01:00:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.339+0000 7fe033fff700 1 -- 172.21.15.44:0/1129840309 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe0480a2030 con 0x55e5ecec6f50 2023-09-10T01:00:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.339+0000 7fe033fff700 1 -- 172.21.15.44:0/1129840309 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 84) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe04c020410 con 0x55e5ecec6f50 2023-09-10T01:00:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.339+0000 7fe067273b80 10 client.27871 fetch_fsmap finished waiting for FSMap version 84 2023-09-10T01:00:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.339+0000 7fe067273b80 10 client.27871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:00:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.339+0000 7fe067273b80 20 client.27871 populate_metadata read hostname 'smithi044' 2023-09-10T01:00:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.339+0000 7fe067273b80 1 --2- 172.21.15.44:0/1129840309 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e5ecf2b860 0x55e5ecf2dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.339+0000 7fe067273b80 4 client.27871 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:00:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.339+0000 7fe067273b80 1 -- 172.21.15.44:0/1129840309 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e5ece088c0 con 0x55e5ecf2b860 2023-09-10T01:00:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.339+0000 7fe058e49700 1 --2- 172.21.15.44:0/1129840309 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e5ecf2b860 0x55e5ecf2dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.340+0000 7fe033fff700 10 client.27871 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:00:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.340+0000 7fe058e49700 1 --2- 172.21.15.44:0/1129840309 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e5ecf2b860 0x55e5ecf2dc80 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57229ac1e533620e server_cookie=b4cc2e67bec768b5 in_seq=0 out_seq=0 2023-09-10T01:00:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.353+0000 7fe033fff700 1 -- 172.21.15.44:0/1129840309 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e5ece088c0 con 0x55e5ecf2b860 2023-09-10T01:00:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.353+0000 7fe033fff700 10 client.27871 handle_command_reply: tid=0 2023-09-10T01:00:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.353+0000 7fe067273b80 1 client.27871 shutdown 2023-09-10T01:00:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.353+0000 7fe067273b80 1 -- 172.21.15.44:0/1129840309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecec6f50 msgr2=0x55e5ececc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.353+0000 7fe067273b80 1 --2- 172.21.15.44:0/1129840309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecec6f50 0x55e5ececc520 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00ed50 tx=0x7fe04c0077e0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.353+0000 7fe067273b80 1 -- 172.21.15.44:0/1129840309 shutdown_connections 2023-09-10T01:00:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.353+0000 7fe067273b80 1 --2- 172.21.15.44:0/1129840309 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e5ecf2b860 0x55e5ecf2dc80 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.353+0000 7fe067273b80 1 --2- 172.21.15.44:0/1129840309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5ecec6f50 0x55e5ececc520 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.353+0000 7fe067273b80 1 --2- 172.21.15.44:0/1129840309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5ecebd300 0x55e5ecf23950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.353+0000 7fe067273b80 1 --2- 172.21.15.44:0/1129840309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5ecebdcc0 0x55e5ecf23e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.353+0000 7fe033fff700 0 client.27871 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:00:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.353+0000 7fe067273b80 1 -- 172.21.15.44:0/1129840309 >> 172.21.15.44:0/1129840309 conn(0x55e5ece32030 msgr2=0x55e5ecec67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.353+0000 7fe067273b80 1 -- 172.21.15.44:0/1129840309 shutdown_connections 2023-09-10T01:00:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.353+0000 7fe067273b80 1 -- 172.21.15.44:0/1129840309 wait complete. 2023-09-10T01:00:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.353+0000 7fe067273b80 20 client.27871 trim_cache size 0 max 16384 2023-09-10T01:00:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:00:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:00:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:00:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:00:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.354+0000 7fe05b0ad700 1 -- 172.21.15.44:0/3046706968 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe0440a4bc0 msgr2=0x7fe0440a7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.354+0000 7fe05b0ad700 1 --2- 172.21.15.44:0/3046706968 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe0440a4bc0 0x7fe0440a7080 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe0541979b0 tx=0x7fe040009260 comp rx=0 tx=0).stop 2023-09-10T01:00:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.354+0000 7fe05b0ad700 1 -- 172.21.15.44:0/3046706968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0540ff760 msgr2=0x7fe05419c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.354+0000 7fe05b0ad700 1 --2- 172.21.15.44:0/3046706968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0540ff760 0x7fe05419c6e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe048009990 tx=0x7fe04800f670 comp rx=0 tx=0).stop 2023-09-10T01:00:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.354+0000 7fe05b0ad700 1 -- 172.21.15.44:0/3046706968 shutdown_connections 2023-09-10T01:00:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.354+0000 7fe05b0ad700 1 --2- 172.21.15.44:0/3046706968 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe0440a4bc0 0x7fe0440a7080 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.354+0000 7fe05b0ad700 1 --2- 172.21.15.44:0/3046706968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0541055b0 0x7fe054196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.354+0000 7fe05b0ad700 1 --2- 172.21.15.44:0/3046706968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0540ff760 0x7fe05419c6e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.354+0000 7fe05b0ad700 1 --2- 172.21.15.44:0/3046706968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe054100120 0x7fe05419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.354+0000 7fe05b0ad700 1 -- 172.21.15.44:0/3046706968 >> 172.21.15.44:0/3046706968 conn(0x7fe054074f30 msgr2=0x7fe0540fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.354+0000 7fe05b0ad700 1 -- 172.21.15.44:0/3046706968 shutdown_connections 2023-09-10T01:00:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:09.354+0000 7fe05b0ad700 1 -- 172.21.15.44:0/3046706968 wait complete. 2023-09-10T01:00:10.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:09 smithi099 ceph-mon[102692]: osdmap e121: 12 total, 12 up, 12 in 2023-09-10T01:00:10.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:09 smithi131 ceph-mon[103713]: osdmap e121: 12 total, 12 up, 12 in 2023-09-10T01:00:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:09 smithi044 ceph-mon[98676]: osdmap e121: 12 total, 12 up, 12 in 2023-09-10T01:00:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:10 smithi099 ceph-mon[102692]: pgmap v617: 129 pgs: 129 active+clean; 8.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 251 op/s 2023-09-10T01:00:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:10 smithi131 ceph-mon[103713]: pgmap v617: 129 pgs: 129 active+clean; 8.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 251 op/s 2023-09-10T01:00:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:10 smithi044 ceph-mon[98676]: pgmap v617: 129 pgs: 129 active+clean; 8.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 251 op/s 2023-09-10T01:00:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:12 smithi099 ceph-mon[102692]: pgmap v618: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 309 op/s 2023-09-10T01:00:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:12 smithi131 ceph-mon[103713]: pgmap v618: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 309 op/s 2023-09-10T01:00:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:12 smithi044 ceph-mon[98676]: pgmap v618: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 309 op/s 2023-09-10T01:00:14.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:13 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:00:14.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:13 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:00:14.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:13 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:00:14.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:13 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:00:14.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:13 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:00:14.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:13 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:00:14.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:13 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:00:14.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:13 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:00:14.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:13 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:00:14.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:13 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:00:14.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:13 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:00:14.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:13 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:00:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:14 smithi099 ceph-mon[102692]: pgmap v619: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 309 op/s 2023-09-10T01:00:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:14 smithi131 ceph-mon[103713]: pgmap v619: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 309 op/s 2023-09-10T01:00:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:14 smithi044 ceph-mon[98676]: pgmap v619: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 309 op/s 2023-09-10T01:00:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:16 smithi131 ceph-mon[103713]: pgmap v620: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 312 op/s 2023-09-10T01:00:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:16 smithi131 ceph-mon[103713]: osdmap e122: 12 total, 12 up, 12 in 2023-09-10T01:00:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:16 smithi044 ceph-mon[98676]: pgmap v620: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 312 op/s 2023-09-10T01:00:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:16 smithi044 ceph-mon[98676]: osdmap e122: 12 total, 12 up, 12 in 2023-09-10T01:00:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:16 smithi099 ceph-mon[102692]: pgmap v620: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 312 op/s 2023-09-10T01:00:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:16 smithi099 ceph-mon[102692]: osdmap e122: 12 total, 12 up, 12 in 2023-09-10T01:00:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:18 smithi131 ceph-mon[103713]: pgmap v622: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 367 op/s 2023-09-10T01:00:19.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:18 smithi044 ceph-mon[98676]: pgmap v622: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 367 op/s 2023-09-10T01:00:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:18 smithi099 ceph-mon[102692]: pgmap v622: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 367 op/s 2023-09-10T01:00:21.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:20 smithi131 ceph-mon[103713]: pgmap v623: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 312 op/s 2023-09-10T01:00:21.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:20 smithi044 ceph-mon[98676]: pgmap v623: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 312 op/s 2023-09-10T01:00:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:20 smithi099 ceph-mon[102692]: pgmap v623: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 312 op/s 2023-09-10T01:00:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.500+0000 7f460e3e1700 1 -- 172.21.15.44:0/1826566789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4608105560 msgr2=0x7f4608105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.500+0000 7f460e3e1700 1 --2- 172.21.15.44:0/1826566789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4608105560 0x7f4608105940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f45f80098f0 tx=0x7f45f8009c00 comp rx=0 tx=0).stop 2023-09-10T01:00:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.500+0000 7f460e3e1700 1 -- 172.21.15.44:0/1826566789 shutdown_connections 2023-09-10T01:00:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.500+0000 7f460e3e1700 1 --2- 172.21.15.44:0/1826566789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4608100050 0x7f460810c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.500+0000 7f460e3e1700 1 --2- 172.21.15.44:0/1826566789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4608105560 0x7f4608105940 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.500+0000 7f460e3e1700 1 --2- 172.21.15.44:0/1826566789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46080ff690 0x7f46080ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.500+0000 7f460e3e1700 1 -- 172.21.15.44:0/1826566789 >> 172.21.15.44:0/1826566789 conn(0x7f46080773d0 msgr2=0x7f46080777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.500+0000 7f460e3e1700 1 -- 172.21.15.44:0/1826566789 shutdown_connections 2023-09-10T01:00:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.500+0000 7f460e3e1700 1 -- 172.21.15.44:0/1826566789 wait complete. 2023-09-10T01:00:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.501+0000 7f460e3e1700 1 Processor -- start 2023-09-10T01:00:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.501+0000 7f460e3e1700 1 -- start start 2023-09-10T01:00:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.501+0000 7f460e3e1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46080ff690 0x7f4608072240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.501+0000 7f460e3e1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4608100050 0x7f460806e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.501+0000 7f460e3e1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4608105560 0x7f460806e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.501+0000 7f460e3e1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460806ee60 con 0x7f4608105560 2023-09-10T01:00:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.501+0000 7f460e3e1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4608117980 con 0x7f46080ff690 2023-09-10T01:00:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.501+0000 7f460e3e1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4608117b40 con 0x7f4608100050 2023-09-10T01:00:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.501+0000 7f4607fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46080ff690 0x7f4608072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.501+0000 7f46077fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4608100050 0x7f460806e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.501+0000 7f460c97e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4608105560 0x7f460806e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.501+0000 7f4607fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46080ff690 0x7f4608072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56376/0 (socket says 172.21.15.44:56376) 2023-09-10T01:00:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.501+0000 7f46077fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4608100050 0x7f460806e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33288/0 (socket says 172.21.15.44:33288) 2023-09-10T01:00:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.501+0000 7f4607fff700 1 -- 172.21.15.44:0/3820400140 learned_addr learned my addr 172.21.15.44:0/3820400140 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.502+0000 7f4607fff700 1 -- 172.21.15.44:0/3820400140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4608100050 msgr2=0x7f460806e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.502+0000 7f4607fff700 1 --2- 172.21.15.44:0/3820400140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4608100050 0x7f460806e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.502+0000 7f4607fff700 1 -- 172.21.15.44:0/3820400140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4608105560 msgr2=0x7f460806e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.502+0000 7f4607fff700 1 --2- 172.21.15.44:0/3820400140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4608105560 0x7f460806e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.502+0000 7f4607fff700 1 -- 172.21.15.44:0/3820400140 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc0095f0 con 0x7f46080ff690 2023-09-10T01:00:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.502+0000 7f460c97e700 1 --2- 172.21.15.44:0/3820400140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4608105560 0x7f460806e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:00:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.502+0000 7f4607fff700 1 --2- 172.21.15.44:0/3820400140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46080ff690 0x7f4608072240 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009810 tx=0x7f45f800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.502+0000 7f46057fa700 1 -- 172.21.15.44:0/3820400140 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f801c070 con 0x7f46080ff690 2023-09-10T01:00:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.502+0000 7f46057fa700 1 -- 172.21.15.44:0/3820400140 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45f800baf0 con 0x7f46080ff690 2023-09-10T01:00:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.502+0000 7f46057fa700 1 -- 172.21.15.44:0/3820400140 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8004930 con 0x7f46080ff690 2023-09-10T01:00:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.502+0000 7f460e3e1700 1 -- 172.21.15.44:0/3820400140 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f8009520 con 0x7f46080ff690 2023-09-10T01:00:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.502+0000 7f460e3e1700 1 -- 172.21.15.44:0/3820400140 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4608118000 con 0x7f46080ff690 2023-09-10T01:00:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.505+0000 7f46057fa700 1 -- 172.21.15.44:0/3820400140 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f45f8020070 con 0x7f46080ff690 2023-09-10T01:00:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.505+0000 7f46057fa700 1 --2- 172.21.15.44:0/3820400140 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45f00a0830 0x7f45f00a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.506+0000 7f46057fa700 1 -- 172.21.15.44:0/3820400140 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f45f80cd030 con 0x7f46080ff690 2023-09-10T01:00:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.506+0000 7f46077fe700 1 --2- 172.21.15.44:0/3820400140 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45f00a0830 0x7f45f00a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.506+0000 7f461a5a7b80 1 Processor -- start 2023-09-10T01:00:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.506+0000 7f461a5a7b80 1 -- start start 2023-09-10T01:00:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.506+0000 7f46077fe700 1 --2- 172.21.15.44:0/3820400140 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45f00a0830 0x7f45f00a2cf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f460806fa30 tx=0x7f45fc009310 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.506+0000 7f461a5a7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 0x5562c729c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.506+0000 7f461a5a7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 0x5562c7296780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.506+0000 7f461a5a7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296cc0 0x5562c72a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.506+0000 7f461a5a7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c71ef6d0 con 0x5562c7296cc0 2023-09-10T01:00:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.506+0000 7f461a5a7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c71ecff0 con 0x5562c7296300 2023-09-10T01:00:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.506+0000 7f461a5a7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c71e88e0 con 0x5562c729c1a0 2023-09-10T01:00:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.507+0000 7f4607fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 0x5562c729c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.507+0000 7f460c97e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296cc0 0x5562c72a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.507+0000 7f4607fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 0x5562c729c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33294/0 (socket says 172.21.15.44:33294) 2023-09-10T01:00:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.507+0000 7f46077fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 0x5562c7296780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.507+0000 7f4607fff700 1 -- 172.21.15.44:0/892374699 learned_addr learned my addr 172.21.15.44:0/892374699 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.507+0000 7f4607fff700 1 -- 172.21.15.44:0/892374699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 msgr2=0x5562c7296780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.507+0000 7f4607fff700 1 --2- 172.21.15.44:0/892374699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 0x5562c7296780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.507+0000 7f4607fff700 1 -- 172.21.15.44:0/892374699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296cc0 msgr2=0x5562c72a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.507+0000 7f4607fff700 1 --2- 172.21.15.44:0/892374699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296cc0 0x5562c72a2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.507+0000 7f4607fff700 1 -- 172.21.15.44:0/892374699 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562c7014d80 con 0x5562c729c1a0 2023-09-10T01:00:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.507+0000 7f46077fe700 1 --2- 172.21.15.44:0/892374699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 0x5562c7296780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:00:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.507+0000 7f460c97e700 1 --2- 172.21.15.44:0/892374699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296cc0 0x5562c72a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:00:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.507+0000 7f4607fff700 1 --2- 172.21.15.44:0/892374699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 0x5562c729c580 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f45f8096be0 tx=0x7f45f800bc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0980469b636fb82 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.507+0000 7f45ee7fc700 1 -- 172.21.15.44:0/892374699 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f809a870 con 0x5562c729c1a0 2023-09-10T01:00:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.507+0000 7f45ee7fc700 1 -- 172.21.15.44:0/892374699 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45f809a170 con 0x5562c729c1a0 2023-09-10T01:00:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.507+0000 7f45ee7fc700 1 -- 172.21.15.44:0/892374699 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80ca790 con 0x5562c729c1a0 2023-09-10T01:00:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.508+0000 7f461a5a7b80 1 -- 172.21.15.44:0/892374699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 msgr2=0x5562c729c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.508+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/892374699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 0x5562c729c580 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f45f8096be0 tx=0x7f45f800bc60 comp rx=0 tx=0).stop 2023-09-10T01:00:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.508+0000 7f461a5a7b80 1 -- 172.21.15.44:0/892374699 shutdown_connections 2023-09-10T01:00:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.508+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/892374699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296cc0 0x5562c72a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.508+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/892374699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 0x5562c729c580 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.508+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/892374699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 0x5562c7296780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.508+0000 7f461a5a7b80 1 -- 172.21.15.44:0/892374699 >> 172.21.15.44:0/892374699 conn(0x5562c720b030 msgr2=0x5562c7291560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.508+0000 7f461a5a7b80 1 -- 172.21.15.44:0/892374699 shutdown_connections 2023-09-10T01:00:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.508+0000 7f461a5a7b80 1 -- 172.21.15.44:0/892374699 wait complete. 2023-09-10T01:00:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.509+0000 7f461a5a7b80 1 -- start start 2023-09-10T01:00:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.509+0000 7f461a5a7b80 1 Processor -- start 2023-09-10T01:00:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.509+0000 7f461a5a7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 0x5562c72a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.509+0000 7f461a5a7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c7296cc0 0x5562c72a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.509+0000 7f461a5a7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c729c1a0 0x5562c729fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.509+0000 7f461a5a7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c7186b10 con 0x5562c729c1a0 2023-09-10T01:00:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.509+0000 7f461a5a7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c7186810 con 0x5562c7296300 2023-09-10T01:00:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.509+0000 7f461a5a7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c706a8c0 con 0x5562c7296cc0 2023-09-10T01:00:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.509+0000 7f461a5a7b80 10 client.0 _ll_register_callbacks cb 0x5562c70d0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:00:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.509+0000 7f46077fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c7296cc0 0x5562c72a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.509+0000 7f46077fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c7296cc0 0x5562c72a60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33302/0 (socket says 172.21.15.44:33302) 2023-09-10T01:00:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.509+0000 7f46077fe700 1 -- 172.21.15.44:0/3628763824 learned_addr learned my addr 172.21.15.44:0/3628763824 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.509+0000 7f4607fff700 1 --2- 172.21.15.44:0/3628763824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 0x5562c72a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.510+0000 7f46077fe700 1 -- 172.21.15.44:0/3628763824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 msgr2=0x5562c72a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.510+0000 7f460c97e700 1 --2- 172.21.15.44:0/3628763824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c729c1a0 0x5562c729fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.510+0000 7f46077fe700 1 --2- 172.21.15.44:0/3628763824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 0x5562c72a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.510+0000 7f46077fe700 1 -- 172.21.15.44:0/3628763824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c729c1a0 msgr2=0x5562c729fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.510+0000 7f46077fe700 1 --2- 172.21.15.44:0/3628763824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c729c1a0 0x5562c729fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.510+0000 7f46077fe700 1 -- 172.21.15.44:0/3628763824 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45fc017070 con 0x5562c7296cc0 2023-09-10T01:00:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.510+0000 7f460c97e700 1 --2- 172.21.15.44:0/3628763824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c729c1a0 0x5562c729fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:00:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.510+0000 7f46077fe700 1 --2- 172.21.15.44:0/3628763824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c7296cc0 0x5562c72a60b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f45fc005fd0 tx=0x7f45fc0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae915388d3c400db server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.512+0000 7f45e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:00:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.512+0000 7f45e77fe700 1 -- 172.21.15.44:0/3628763824 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc022070 con 0x5562c7296cc0 2023-09-10T01:00:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.512+0000 7f45e77fe700 1 -- 172.21.15.44:0/3628763824 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f45fc00ee00 con 0x5562c7296cc0 2023-09-10T01:00:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.512+0000 7f461a5a7b80 1 -- 172.21.15.44:0/3628763824 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562c7140490 con 0x5562c7296cc0 2023-09-10T01:00:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.512+0000 7f45e77fe700 1 -- 172.21.15.44:0/3628763824 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45fc01ddc0 con 0x5562c7296cc0 2023-09-10T01:00:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.512+0000 7f461a5a7b80 10 client.27883 fetch_fsmap learned FSMap version 84 2023-09-10T01:00:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.512+0000 7f461a5a7b80 1 -- 172.21.15.44:0/3628763824 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x7f4600003a00 con 0x5562c7296cc0 2023-09-10T01:00:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.512+0000 7f45e77fe700 1 -- 172.21.15.44:0/3628763824 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5562c7140490 con 0x5562c7296cc0 2023-09-10T01:00:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.512+0000 7f45e77fe700 1 -- 172.21.15.44:0/3628763824 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 84) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f45fc003b00 con 0x5562c7296cc0 2023-09-10T01:00:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.513+0000 7f461a5a7b80 10 client.27883 fetch_fsmap finished waiting for FSMap version 84 2023-09-10T01:00:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.513+0000 7f461a5a7b80 10 client.27883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:00:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.513+0000 7f461a5a7b80 20 client.27883 populate_metadata read hostname 'smithi044' 2023-09-10T01:00:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.513+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/3628763824 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5562c72b00f0 0x5562c72b2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.513+0000 7f461a5a7b80 4 client.27883 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:00:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.513+0000 7f461a5a7b80 1 -- 172.21.15.44:0/3628763824 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562c71fd840 con 0x5562c72b00f0 2023-09-10T01:00:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.513+0000 7f4607fff700 1 --2- 172.21.15.44:0/3628763824 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5562c72b00f0 0x5562c72b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.513+0000 7f45e77fe700 10 client.27883 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:00:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.513+0000 7f4607fff700 1 --2- 172.21.15.44:0/3628763824 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5562c72b00f0 0x5562c72b2510 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a5144b1b1a54e1a server_cookie=a8147c681197e7a9 in_seq=0 out_seq=0 2023-09-10T01:00:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.514+0000 7f45e77fe700 1 -- 172.21.15.44:0/3628763824 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5562c71fd840 con 0x5562c72b00f0 2023-09-10T01:00:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.514+0000 7f45e77fe700 10 client.27883 handle_command_reply: tid=0 2023-09-10T01:00:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.514+0000 7f461a5a7b80 1 client.27883 shutdown 2023-09-10T01:00:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.514+0000 7f461a5a7b80 1 -- 172.21.15.44:0/3628763824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c7296cc0 msgr2=0x5562c72a60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.514+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/3628763824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c7296cc0 0x5562c72a60b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f45fc005fd0 tx=0x7f45fc0047c0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.514+0000 7f461a5a7b80 1 -- 172.21.15.44:0/3628763824 shutdown_connections 2023-09-10T01:00:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.514+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/3628763824 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5562c72b00f0 0x5562c72b2510 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.514+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/3628763824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c729c1a0 0x5562c729fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.514+0000 7f45e77fe700 0 client.27883 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:00:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.514+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/3628763824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 0x5562c72a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.514+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/3628763824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c7296cc0 0x5562c72a60b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.514+0000 7f461a5a7b80 1 -- 172.21.15.44:0/3628763824 >> 172.21.15.44:0/3628763824 conn(0x5562c720b030 msgr2=0x5562c72907a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.514+0000 7f461a5a7b80 1 -- 172.21.15.44:0/3628763824 shutdown_connections 2023-09-10T01:00:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.514+0000 7f461a5a7b80 1 -- 172.21.15.44:0/3628763824 wait complete. 2023-09-10T01:00:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.515+0000 7f461a5a7b80 20 client.27883 trim_cache size 0 max 16384 2023-09-10T01:00:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.535+0000 7f461a5a7b80 1 Processor -- start 2023-09-10T01:00:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.535+0000 7f461a5a7b80 1 -- start start 2023-09-10T01:00:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.535+0000 7f461a5a7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296300 0x5562c729b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.535+0000 7f461a5a7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296cc0 0x5562c7295a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.535+0000 7f461a5a7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 0x5562c7290f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.535+0000 7f461a5a7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c71fd840 con 0x5562c7296300 2023-09-10T01:00:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.535+0000 7f461a5a7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c7140490 con 0x5562c7296cc0 2023-09-10T01:00:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.535+0000 7f461a5a7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c72ad730 con 0x5562c729c1a0 2023-09-10T01:00:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.536+0000 7f46077fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296cc0 0x5562c7295a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.536+0000 7f4607fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296300 0x5562c729b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.536+0000 7f46077fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296cc0 0x5562c7295a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56400/0 (socket says 172.21.15.44:56400) 2023-09-10T01:00:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.536+0000 7f46077fe700 1 -- 172.21.15.44:0/2868902826 learned_addr learned my addr 172.21.15.44:0/2868902826 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.536+0000 7f460c97e700 1 --2- 172.21.15.44:0/2868902826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 0x5562c7290f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.536+0000 7f46077fe700 1 -- 172.21.15.44:0/2868902826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 msgr2=0x5562c7290f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.536+0000 7f46077fe700 1 --2- 172.21.15.44:0/2868902826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 0x5562c7290f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.536+0000 7f46077fe700 1 -- 172.21.15.44:0/2868902826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296300 msgr2=0x5562c729b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.536+0000 7f46077fe700 1 --2- 172.21.15.44:0/2868902826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296300 0x5562c729b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.536+0000 7f46077fe700 1 -- 172.21.15.44:0/2868902826 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4608118000 con 0x5562c7296cc0 2023-09-10T01:00:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.536+0000 7f46077fe700 1 --2- 172.21.15.44:0/2868902826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296cc0 0x5562c7295a00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f45fc013010 tx=0x7f45fc00e460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a367b9c15024449 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.536+0000 7f45eeffd700 1 -- 172.21.15.44:0/2868902826 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc022030 con 0x5562c7296cc0 2023-09-10T01:00:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.536+0000 7f45eeffd700 1 -- 172.21.15.44:0/2868902826 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45fc00ee00 con 0x5562c7296cc0 2023-09-10T01:00:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.536+0000 7f45eeffd700 1 -- 172.21.15.44:0/2868902826 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc003a70 con 0x5562c7296cc0 2023-09-10T01:00:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.537+0000 7f461a5a7b80 1 -- 172.21.15.44:0/2868902826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296cc0 msgr2=0x5562c7295a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.537+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/2868902826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296cc0 0x5562c7295a00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f45fc013010 tx=0x7f45fc00e460 comp rx=0 tx=0).stop 2023-09-10T01:00:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.537+0000 7f461a5a7b80 1 -- 172.21.15.44:0/2868902826 shutdown_connections 2023-09-10T01:00:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.537+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/2868902826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 0x5562c7290f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.537+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/2868902826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296300 0x5562c729b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.537+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/2868902826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296cc0 0x5562c7295a00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.537+0000 7f461a5a7b80 1 -- 172.21.15.44:0/2868902826 >> 172.21.15.44:0/2868902826 conn(0x5562c720b030 msgr2=0x5562c72952f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.537+0000 7f461a5a7b80 1 -- 172.21.15.44:0/2868902826 shutdown_connections 2023-09-10T01:00:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.537+0000 7f461a5a7b80 1 -- 172.21.15.44:0/2868902826 wait complete. 2023-09-10T01:00:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.538+0000 7f461a5a7b80 1 -- start start 2023-09-10T01:00:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.538+0000 7f461a5a7b80 1 Processor -- start 2023-09-10T01:00:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.538+0000 7f461a5a7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 0x5562c72f8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.538+0000 7f461a5a7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296cc0 0x5562c72f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.538+0000 7f461a5a7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 0x5562c7302ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.538+0000 7f461a5a7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c7303430 con 0x5562c7296cc0 2023-09-10T01:00:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.538+0000 7f461a5a7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c7303570 con 0x5562c7296300 2023-09-10T01:00:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.538+0000 7f461a5a7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c73036b0 con 0x5562c729c1a0 2023-09-10T01:00:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.538+0000 7f461a5a7b80 10 client.0 _ll_register_callbacks cb 0x7f45fc00e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:00:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.538+0000 7f460c97e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 0x5562c7302ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.538+0000 7f46077fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296cc0 0x5562c72f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.538+0000 7f460c97e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 0x5562c7302ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33318/0 (socket says 172.21.15.44:33318) 2023-09-10T01:00:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.538+0000 7f460c97e700 1 -- 172.21.15.44:0/1400888230 learned_addr learned my addr 172.21.15.44:0/1400888230 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.538+0000 7f46077fe700 1 --2- 172.21.15.44:0/1400888230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296cc0 0x5562c72f85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54666/0 (socket says 172.21.15.44:54666) 2023-09-10T01:00:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.539+0000 7f4607fff700 1 --2- 172.21.15.44:0/1400888230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 0x5562c72f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.539+0000 7f46077fe700 1 -- 172.21.15.44:0/1400888230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 msgr2=0x5562c7302ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.539+0000 7f46077fe700 1 --2- 172.21.15.44:0/1400888230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 0x5562c7302ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.539+0000 7f46077fe700 1 -- 172.21.15.44:0/1400888230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 msgr2=0x5562c72f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.539+0000 7f46077fe700 1 --2- 172.21.15.44:0/1400888230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 0x5562c72f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.539+0000 7f46077fe700 1 -- 172.21.15.44:0/1400888230 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4600003a00 con 0x5562c7296cc0 2023-09-10T01:00:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.539+0000 7f4607fff700 1 --2- 172.21.15.44:0/1400888230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 0x5562c72f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:00:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.539+0000 7f46077fe700 1 --2- 172.21.15.44:0/1400888230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296cc0 0x5562c72f85a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x5562c72a1270 tx=0x7f45fc004330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ce27f22e0b268d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.539+0000 7f45e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:00:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.539+0000 7f45e77fe700 1 -- 172.21.15.44:0/1400888230 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc022040 con 0x5562c7296cc0 2023-09-10T01:00:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.539+0000 7f45e77fe700 1 -- 172.21.15.44:0/1400888230 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f45fc023070 con 0x5562c7296cc0 2023-09-10T01:00:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.539+0000 7f461a5a7b80 1 -- 172.21.15.44:0/1400888230 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45f809a170 con 0x5562c7296cc0 2023-09-10T01:00:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.539+0000 7f45e77fe700 1 -- 172.21.15.44:0/1400888230 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45fc004520 con 0x5562c7296cc0 2023-09-10T01:00:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.539+0000 7f461a5a7b80 10 client.18291 fetch_fsmap learned FSMap version 84 2023-09-10T01:00:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.539+0000 7f461a5a7b80 1 -- 172.21.15.44:0/1400888230 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x5562c6ffdc00 con 0x5562c7296cc0 2023-09-10T01:00:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.540+0000 7f45e77fe700 1 -- 172.21.15.44:0/1400888230 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45f809a170 con 0x5562c7296cc0 2023-09-10T01:00:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.540+0000 7f45e77fe700 1 -- 172.21.15.44:0/1400888230 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 84) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f45fc004710 con 0x5562c7296cc0 2023-09-10T01:00:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.540+0000 7f461a5a7b80 10 client.18291 fetch_fsmap finished waiting for FSMap version 84 2023-09-10T01:00:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.540+0000 7f461a5a7b80 10 client.18291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:00:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.540+0000 7f461a5a7b80 20 client.18291 populate_metadata read hostname 'smithi044' 2023-09-10T01:00:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.540+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/1400888230 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5562c7304890 0x5562c7306c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.540+0000 7f461a5a7b80 4 client.18291 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:00:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.540+0000 7f461a5a7b80 1 -- 172.21.15.44:0/1400888230 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5562c71e18c0 con 0x5562c7304890 2023-09-10T01:00:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.540+0000 7f4607fff700 1 --2- 172.21.15.44:0/1400888230 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5562c7304890 0x5562c7306c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.541+0000 7f45e77fe700 10 client.18291 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:00:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.541+0000 7f4607fff700 1 --2- 172.21.15.44:0/1400888230 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5562c7304890 0x5562c7306c80 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cf9879e50881129 server_cookie=19908dc0dc5318aa in_seq=0 out_seq=0 2023-09-10T01:00:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.551+0000 7f45e77fe700 1 -- 172.21.15.44:0/1400888230 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5562c71e18c0 con 0x5562c7304890 2023-09-10T01:00:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.551+0000 7f45e77fe700 10 client.18291 handle_command_reply: tid=0 2023-09-10T01:00:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.551+0000 7f461a5a7b80 1 client.18291 shutdown 2023-09-10T01:00:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.551+0000 7f461a5a7b80 1 -- 172.21.15.44:0/1400888230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296cc0 msgr2=0x5562c72f85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.551+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/1400888230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296cc0 0x5562c72f85a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x5562c72a1270 tx=0x7f45fc004330 comp rx=0 tx=0).stop 2023-09-10T01:00:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.551+0000 7f461a5a7b80 1 -- 172.21.15.44:0/1400888230 shutdown_connections 2023-09-10T01:00:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.551+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/1400888230 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5562c7304890 0x5562c7306c80 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.551+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/1400888230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562c729c1a0 0x5562c7302ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.551+0000 7f45e77fe700 0 client.18291 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:00:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.551+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/1400888230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562c7296300 0x5562c72f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.551+0000 7f461a5a7b80 1 --2- 172.21.15.44:0/1400888230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562c7296cc0 0x5562c72f85a0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.551+0000 7f461a5a7b80 1 -- 172.21.15.44:0/1400888230 >> 172.21.15.44:0/1400888230 conn(0x5562c720b030 msgr2=0x5562c7295b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.552+0000 7f461a5a7b80 1 -- 172.21.15.44:0/1400888230 shutdown_connections 2023-09-10T01:00:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.552+0000 7f461a5a7b80 1 -- 172.21.15.44:0/1400888230 wait complete. 2023-09-10T01:00:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.552+0000 7f461a5a7b80 20 client.18291 trim_cache size 0 max 16384 2023-09-10T01:00:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:00:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:00:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:00:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:00:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.552+0000 7f460e3e1700 1 -- 172.21.15.44:0/3820400140 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45f00a0830 msgr2=0x7f45f00a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.552+0000 7f460e3e1700 1 --2- 172.21.15.44:0/3820400140 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45f00a0830 0x7f45f00a2cf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f460806fa30 tx=0x7f45fc009310 comp rx=0 tx=0).stop 2023-09-10T01:00:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.552+0000 7f460e3e1700 1 -- 172.21.15.44:0/3820400140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46080ff690 msgr2=0x7f4608072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.552+0000 7f460e3e1700 1 --2- 172.21.15.44:0/3820400140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46080ff690 0x7f4608072240 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009810 tx=0x7f45f800fa00 comp rx=0 tx=0).stop 2023-09-10T01:00:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.552+0000 7f460e3e1700 1 -- 172.21.15.44:0/3820400140 shutdown_connections 2023-09-10T01:00:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.552+0000 7f460e3e1700 1 --2- 172.21.15.44:0/3820400140 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45f00a0830 0x7f45f00a2cf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.552+0000 7f460e3e1700 1 --2- 172.21.15.44:0/3820400140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4608105560 0x7f460806e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.552+0000 7f460e3e1700 1 --2- 172.21.15.44:0/3820400140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46080ff690 0x7f4608072240 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.552+0000 7f460e3e1700 1 --2- 172.21.15.44:0/3820400140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4608100050 0x7f460806e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.552+0000 7f460e3e1700 1 -- 172.21.15.44:0/3820400140 >> 172.21.15.44:0/3820400140 conn(0x7f46080773d0 msgr2=0x7f46080fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.552+0000 7f460e3e1700 1 -- 172.21.15.44:0/3820400140 shutdown_connections 2023-09-10T01:00:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:22.552+0000 7f460e3e1700 1 -- 172.21.15.44:0/3820400140 wait complete. 2023-09-10T01:00:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:22 smithi131 ceph-mon[103713]: pgmap v624: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 291 op/s 2023-09-10T01:00:23.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:22 smithi044 ceph-mon[98676]: pgmap v624: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 291 op/s 2023-09-10T01:00:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:22 smithi099 ceph-mon[102692]: pgmap v624: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 291 op/s 2023-09-10T01:00:25.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:24 smithi131 ceph-mon[103713]: pgmap v625: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 291 op/s 2023-09-10T01:00:25.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:24 smithi044 ceph-mon[98676]: pgmap v625: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 291 op/s 2023-09-10T01:00:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:24 smithi099 ceph-mon[102692]: pgmap v625: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 291 op/s 2023-09-10T01:00:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:26 smithi131 ceph-mon[103713]: pgmap v626: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 288 op/s 2023-09-10T01:00:27.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:26 smithi044 ceph-mon[98676]: pgmap v626: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 288 op/s 2023-09-10T01:00:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:26 smithi099 ceph-mon[102692]: pgmap v626: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 288 op/s 2023-09-10T01:00:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:28 smithi131 ceph-mon[103713]: pgmap v627: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 249 op/s 2023-09-10T01:00:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:28 smithi044 ceph-mon[98676]: pgmap v627: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 249 op/s 2023-09-10T01:00:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:28 smithi099 ceph-mon[102692]: pgmap v627: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 249 op/s 2023-09-10T01:00:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:30 smithi131 ceph-mon[103713]: pgmap v628: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2023-09-10T01:00:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:30 smithi044 ceph-mon[98676]: pgmap v628: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2023-09-10T01:00:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:30 smithi099 ceph-mon[102692]: pgmap v628: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2023-09-10T01:00:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:32 smithi131 ceph-mon[103713]: pgmap v629: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 310 op/s 2023-09-10T01:00:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:32 smithi044 ceph-mon[98676]: pgmap v629: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 310 op/s 2023-09-10T01:00:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:32 smithi099 ceph-mon[102692]: pgmap v629: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 310 op/s 2023-09-10T01:00:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:34 smithi131 ceph-mon[103713]: pgmap v630: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 190 op/s 2023-09-10T01:00:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:34 smithi044 ceph-mon[98676]: pgmap v630: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 190 op/s 2023-09-10T01:00:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:34 smithi099 ceph-mon[102692]: pgmap v630: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 190 op/s 2023-09-10T01:00:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.696+0000 7fb65b534700 1 -- 172.21.15.44:0/1591041521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6540ff830 msgr2=0x7fb654103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.696+0000 7fb65b534700 1 --2- 172.21.15.44:0/1591041521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6540ff830 0x7fb654103870 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb6440098f0 tx=0x7fb644009c00 comp rx=0 tx=0).stop 2023-09-10T01:00:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.698+0000 7fb65b534700 1 -- 172.21.15.44:0/1591041521 shutdown_connections 2023-09-10T01:00:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.698+0000 7fb65b534700 1 --2- 172.21.15.44:0/1591041521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb654103fa0 0x7fb65410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.698+0000 7fb65b534700 1 --2- 172.21.15.44:0/1591041521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6540fee80 0x7fb6540ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.698+0000 7fb65b534700 1 --2- 172.21.15.44:0/1591041521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6540ff830 0x7fb654103870 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.698+0000 7fb65b534700 1 -- 172.21.15.44:0/1591041521 >> 172.21.15.44:0/1591041521 conn(0x7fb6540769d0 msgr2=0x7fb6540fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.698+0000 7fb65b534700 1 -- 172.21.15.44:0/1591041521 shutdown_connections 2023-09-10T01:00:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.698+0000 7fb65b534700 1 -- 172.21.15.44:0/1591041521 wait complete. 2023-09-10T01:00:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.698+0000 7fb65b534700 1 Processor -- start 2023-09-10T01:00:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.698+0000 7fb65b534700 1 -- start start 2023-09-10T01:00:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb65b534700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6540fee80 0x7fb65410d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb65b534700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb654103fa0 0x7fb65410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb65b534700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6541a2880 0x7fb6541a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb65b534700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6541a51b0 con 0x7fb6540fee80 2023-09-10T01:00:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb65b534700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6541a52f0 con 0x7fb6541a2880 2023-09-10T01:00:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb65b534700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6541a5460 con 0x7fb654103fa0 2023-09-10T01:00:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb6592d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6540fee80 0x7fb65410d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb659ad1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6541a2880 0x7fb6541a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb6592d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6540fee80 0x7fb65410d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54678/0 (socket says 172.21.15.44:54678) 2023-09-10T01:00:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb6592d0700 1 -- 172.21.15.44:0/3317491814 learned_addr learned my addr 172.21.15.44:0/3317491814 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb658acf700 1 --2- 172.21.15.44:0/3317491814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb654103fa0 0x7fb65410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb659ad1700 1 -- 172.21.15.44:0/3317491814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb654103fa0 msgr2=0x7fb65410d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb659ad1700 1 --2- 172.21.15.44:0/3317491814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb654103fa0 0x7fb65410d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb659ad1700 1 -- 172.21.15.44:0/3317491814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6540fee80 msgr2=0x7fb65410d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb659ad1700 1 --2- 172.21.15.44:0/3317491814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6540fee80 0x7fb65410d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb659ad1700 1 -- 172.21.15.44:0/3317491814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6500095f0 con 0x7fb6541a2880 2023-09-10T01:00:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.699+0000 7fb659ad1700 1 --2- 172.21.15.44:0/3317491814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6541a2880 0x7fb6541a4c70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00efc0 tx=0x7fb64c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.700+0000 7fb64a7fc700 1 -- 172.21.15.44:0/3317491814 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c018070 con 0x7fb6541a2880 2023-09-10T01:00:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.700+0000 7fb65b534700 1 -- 172.21.15.44:0/3317491814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb644009520 con 0x7fb6541a2880 2023-09-10T01:00:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.700+0000 7fb64a7fc700 1 -- 172.21.15.44:0/3317491814 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb64c00f040 con 0x7fb6541a2880 2023-09-10T01:00:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.700+0000 7fb64a7fc700 1 -- 172.21.15.44:0/3317491814 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c00cba0 con 0x7fb6541a2880 2023-09-10T01:00:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.701+0000 7fb65b534700 1 -- 172.21.15.44:0/3317491814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6541a5a80 con 0x7fb6541a2880 2023-09-10T01:00:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.703+0000 7fb64a7fc700 1 -- 172.21.15.44:0/3317491814 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fb64c003a00 con 0x7fb6541a2880 2023-09-10T01:00:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.704+0000 7fb64a7fc700 1 --2- 172.21.15.44:0/3317491814 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb6400a07b0 0x7fb6400a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.704+0000 7fb64a7fc700 1 -- 172.21.15.44:0/3317491814 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb64c0cba40 con 0x7fb6541a2880 2023-09-10T01:00:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.704+0000 7fb6592d0700 1 --2- 172.21.15.44:0/3317491814 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb6400a07b0 0x7fb6400a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.705+0000 7fb6592d0700 1 --2- 172.21.15.44:0/3317491814 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb6400a07b0 0x7fb6400a2c70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb650005d10 tx=0x7fb650009310 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.705+0000 7fb6676fab80 1 Processor -- start 2023-09-10T01:00:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.705+0000 7fb6676fab80 1 -- start start 2023-09-10T01:00:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.705+0000 7fb6676fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2689291a0 0x55d268929580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.705+0000 7fb6676fab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d268923300 0x55d268923780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.705+0000 7fb6676fab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923cc0 0x55d26892ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.705+0000 7fb6676fab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26887c6d0 con 0x55d268923300 2023-09-10T01:00:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.705+0000 7fb6676fab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d268879ff0 con 0x55d268923cc0 2023-09-10T01:00:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.705+0000 7fb6676fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2688758e0 con 0x55d2689291a0 2023-09-10T01:00:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.706+0000 7fb6592d0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2689291a0 0x55d268929580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.706+0000 7fb6592d0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2689291a0 0x55d268929580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33336/0 (socket says 172.21.15.44:33336) 2023-09-10T01:00:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.706+0000 7fb6592d0700 1 -- 172.21.15.44:0/1882697486 learned_addr learned my addr 172.21.15.44:0/1882697486 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.706+0000 7fb659ad1700 1 --2- 172.21.15.44:0/1882697486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923cc0 0x55d26892ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.706+0000 7fb658acf700 1 --2- 172.21.15.44:0/1882697486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d268923300 0x55d268923780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.706+0000 7fb6592d0700 1 -- 172.21.15.44:0/1882697486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923cc0 msgr2=0x55d26892ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.706+0000 7fb6592d0700 1 --2- 172.21.15.44:0/1882697486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923cc0 0x55d26892ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.706+0000 7fb6592d0700 1 -- 172.21.15.44:0/1882697486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d268923300 msgr2=0x55d268923780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.706+0000 7fb6592d0700 1 --2- 172.21.15.44:0/1882697486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d268923300 0x55d268923780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.706+0000 7fb6592d0700 1 -- 172.21.15.44:0/1882697486 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2686a1d80 con 0x55d2689291a0 2023-09-10T01:00:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.706+0000 7fb6592d0700 1 --2- 172.21.15.44:0/1882697486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2689291a0 0x55d268929580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb65000e6a0 tx=0x7fb650014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68c3180d1db54e76 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.706+0000 7fb63f7fe700 1 -- 172.21.15.44:0/1882697486 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65000e3b0 con 0x55d2689291a0 2023-09-10T01:00:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.706+0000 7fb63f7fe700 1 -- 172.21.15.44:0/1882697486 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb650021070 con 0x55d2689291a0 2023-09-10T01:00:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.706+0000 7fb63f7fe700 1 -- 172.21.15.44:0/1882697486 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6500264c0 con 0x55d2689291a0 2023-09-10T01:00:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.707+0000 7fb6676fab80 1 -- 172.21.15.44:0/1882697486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2689291a0 msgr2=0x55d268929580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.707+0000 7fb6676fab80 1 --2- 172.21.15.44:0/1882697486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2689291a0 0x55d268929580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb65000e6a0 tx=0x7fb650014000 comp rx=0 tx=0).stop 2023-09-10T01:00:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.707+0000 7fb6676fab80 1 -- 172.21.15.44:0/1882697486 shutdown_connections 2023-09-10T01:00:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.707+0000 7fb6676fab80 1 --2- 172.21.15.44:0/1882697486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923cc0 0x55d26892ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.707+0000 7fb6676fab80 1 --2- 172.21.15.44:0/1882697486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2689291a0 0x55d268929580 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.707+0000 7fb6676fab80 1 --2- 172.21.15.44:0/1882697486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d268923300 0x55d268923780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.707+0000 7fb6676fab80 1 -- 172.21.15.44:0/1882697486 >> 172.21.15.44:0/1882697486 conn(0x55d268898030 msgr2=0x55d26891e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.707+0000 7fb6676fab80 1 -- 172.21.15.44:0/1882697486 shutdown_connections 2023-09-10T01:00:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.707+0000 7fb6676fab80 1 -- 172.21.15.44:0/1882697486 wait complete. 2023-09-10T01:00:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.707+0000 7fb6676fab80 1 -- start start 2023-09-10T01:00:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.707+0000 7fb6676fab80 1 Processor -- start 2023-09-10T01:00:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb6676fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d268923300 0x55d26892fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb6676fab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923cc0 0x55d26892d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb6676fab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2689291a0 0x55d26892d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb6676fab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d268813b10 con 0x55d2689291a0 2023-09-10T01:00:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb6676fab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d268813810 con 0x55d268923cc0 2023-09-10T01:00:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb6676fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2686f78c0 con 0x55d268923300 2023-09-10T01:00:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb6676fab80 10 client.0 _ll_register_callbacks cb 0x55d26875da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:00:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb6592d0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d268923300 0x55d26892fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb658acf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923cc0 0x55d26892d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb6592d0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d268923300 0x55d26892fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33342/0 (socket says 172.21.15.44:33342) 2023-09-10T01:00:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb659ad1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2689291a0 0x55d26892d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb6592d0700 1 -- 172.21.15.44:0/1089950123 learned_addr learned my addr 172.21.15.44:0/1089950123 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb659ad1700 1 -- 172.21.15.44:0/1089950123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d268923300 msgr2=0x55d26892fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb659ad1700 1 --2- 172.21.15.44:0/1089950123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d268923300 0x55d26892fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb659ad1700 1 -- 172.21.15.44:0/1089950123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923cc0 msgr2=0x55d26892d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb659ad1700 1 --2- 172.21.15.44:0/1089950123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923cc0 0x55d26892d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.708+0000 7fb659ad1700 1 -- 172.21.15.44:0/1089950123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb650017070 con 0x55d2689291a0 2023-09-10T01:00:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.709+0000 7fb659ad1700 1 --2- 172.21.15.44:0/1089950123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2689291a0 0x55d26892d660 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0c8d90 tx=0x7fb64c0986c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994b7c8b708fcb9c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.709+0000 7fb63cff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:00:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.709+0000 7fb63cff9700 1 -- 172.21.15.44:0/1089950123 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c098fe0 con 0x55d2689291a0 2023-09-10T01:00:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.709+0000 7fb6676fab80 1 -- 172.21.15.44:0/1089950123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2687cd490 con 0x55d2689291a0 2023-09-10T01:00:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.709+0000 7fb63cff9700 1 -- 172.21.15.44:0/1089950123 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb64c099280 con 0x55d2689291a0 2023-09-10T01:00:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.709+0000 7fb63cff9700 1 -- 172.21.15.44:0/1089950123 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb64c098b80 con 0x55d2689291a0 2023-09-10T01:00:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.709+0000 7fb6676fab80 10 client.18303 fetch_fsmap learned FSMap version 84 2023-09-10T01:00:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.709+0000 7fb6676fab80 1 -- 172.21.15.44:0/1089950123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x55d26868ac00 con 0x55d2689291a0 2023-09-10T01:00:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.709+0000 7fb63cff9700 1 -- 172.21.15.44:0/1089950123 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb64c0b3030 con 0x55d2689291a0 2023-09-10T01:00:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.709+0000 7fb63cff9700 1 -- 172.21.15.44:0/1089950123 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 84) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb64c0a06d0 con 0x55d2689291a0 2023-09-10T01:00:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.709+0000 7fb6676fab80 10 client.18303 fetch_fsmap finished waiting for FSMap version 84 2023-09-10T01:00:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.709+0000 7fb6676fab80 10 client.18303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:00:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.709+0000 7fb6676fab80 20 client.18303 populate_metadata read hostname 'smithi044' 2023-09-10T01:00:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.709+0000 7fb6676fab80 1 --2- 172.21.15.44:0/1089950123 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d26893d3c0 0x55d26893f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.710+0000 7fb6676fab80 4 client.18303 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:00:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.710+0000 7fb6676fab80 1 -- 172.21.15.44:0/1089950123 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d26888a840 con 0x55d26893d3c0 2023-09-10T01:00:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.710+0000 7fb6592d0700 1 --2- 172.21.15.44:0/1089950123 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d26893d3c0 0x55d26893f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.710+0000 7fb63cff9700 10 client.18303 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:00:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.710+0000 7fb6592d0700 1 --2- 172.21.15.44:0/1089950123 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d26893d3c0 0x55d26893f880 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f68188238a2f3a0 server_cookie=c369c4809c23e794 in_seq=0 out_seq=0 2023-09-10T01:00:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.711+0000 7fb63cff9700 1 -- 172.21.15.44:0/1089950123 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d26888a840 con 0x55d26893d3c0 2023-09-10T01:00:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.711+0000 7fb63cff9700 10 client.18303 handle_command_reply: tid=0 2023-09-10T01:00:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.711+0000 7fb6676fab80 1 client.18303 shutdown 2023-09-10T01:00:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.711+0000 7fb6676fab80 1 -- 172.21.15.44:0/1089950123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2689291a0 msgr2=0x55d26892d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:35.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.711+0000 7fb6676fab80 1 --2- 172.21.15.44:0/1089950123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2689291a0 0x55d26892d660 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0c8d90 tx=0x7fb64c0986c0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.711+0000 7fb6676fab80 1 -- 172.21.15.44:0/1089950123 shutdown_connections 2023-09-10T01:00:35.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.711+0000 7fb6676fab80 1 --2- 172.21.15.44:0/1089950123 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d26893d3c0 0x55d26893f880 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.711+0000 7fb6676fab80 1 --2- 172.21.15.44:0/1089950123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2689291a0 0x55d26892d660 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.711+0000 7fb6676fab80 1 --2- 172.21.15.44:0/1089950123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d268923300 0x55d26892fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.711+0000 7fb63cff9700 0 client.18303 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:00:35.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.711+0000 7fb6676fab80 1 --2- 172.21.15.44:0/1089950123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923cc0 0x55d26892d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.711+0000 7fb6676fab80 1 -- 172.21.15.44:0/1089950123 >> 172.21.15.44:0/1089950123 conn(0x55d268898030 msgr2=0x55d26891d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:35.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.711+0000 7fb6676fab80 1 -- 172.21.15.44:0/1089950123 shutdown_connections 2023-09-10T01:00:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.711+0000 7fb6676fab80 1 -- 172.21.15.44:0/1089950123 wait complete. 2023-09-10T01:00:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.711+0000 7fb6676fab80 20 client.18303 trim_cache size 0 max 16384 2023-09-10T01:00:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.731+0000 7fb6676fab80 1 Processor -- start 2023-09-10T01:00:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.731+0000 7fb6676fab80 1 -- start start 2023-09-10T01:00:35.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb6676fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d268923300 0x55d268922a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb6676fab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d268923cc0 0x55d26892f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb6676fab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2689291a0 0x55d26891df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb6676fab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2689245b0 con 0x55d268923cc0 2023-09-10T01:00:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb6676fab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d268924140 con 0x55d2689291a0 2023-09-10T01:00:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb6676fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26892d900 con 0x55d268923300 2023-09-10T01:00:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb659ad1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2689291a0 0x55d26891df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb6592d0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d268923300 0x55d268922a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb659ad1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2689291a0 0x55d26891df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56444/0 (socket says 172.21.15.44:56444) 2023-09-10T01:00:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb659ad1700 1 -- 172.21.15.44:0/3434073605 learned_addr learned my addr 172.21.15.44:0/3434073605 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb658acf700 1 --2- 172.21.15.44:0/3434073605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d268923cc0 0x55d26892f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb659ad1700 1 -- 172.21.15.44:0/3434073605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d268923300 msgr2=0x55d268922a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb659ad1700 1 --2- 172.21.15.44:0/3434073605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d268923300 0x55d268922a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb659ad1700 1 -- 172.21.15.44:0/3434073605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d268923cc0 msgr2=0x55d26892f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb659ad1700 1 --2- 172.21.15.44:0/3434073605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d268923cc0 0x55d26892f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb659ad1700 1 -- 172.21.15.44:0/3434073605 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2686c81f0 con 0x55d2689291a0 2023-09-10T01:00:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb658acf700 1 --2- 172.21.15.44:0/3434073605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d268923cc0 0x55d26892f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:00:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.732+0000 7fb6592d0700 1 --2- 172.21.15.44:0/3434073605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d268923300 0x55d268922a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:00:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.733+0000 7fb659ad1700 1 --2- 172.21.15.44:0/3434073605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2689291a0 0x55d26891df20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0c9390 tx=0x7fb64c0094d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e42a19657f319b2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.733+0000 7fb63ffff700 1 -- 172.21.15.44:0/3434073605 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c098070 con 0x55d2689291a0 2023-09-10T01:00:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.733+0000 7fb63ffff700 1 -- 172.21.15.44:0/3434073605 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb64c0b3040 con 0x55d2689291a0 2023-09-10T01:00:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.733+0000 7fb63ffff700 1 -- 172.21.15.44:0/3434073605 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c099e40 con 0x55d2689291a0 2023-09-10T01:00:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.733+0000 7fb6676fab80 1 -- 172.21.15.44:0/3434073605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2689291a0 msgr2=0x55d26891df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.733+0000 7fb6676fab80 1 --2- 172.21.15.44:0/3434073605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2689291a0 0x55d26891df20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0c9390 tx=0x7fb64c0094d0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.734+0000 7fb6676fab80 1 -- 172.21.15.44:0/3434073605 shutdown_connections 2023-09-10T01:00:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.734+0000 7fb6676fab80 1 --2- 172.21.15.44:0/3434073605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2689291a0 0x55d26891df20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.734+0000 7fb6676fab80 1 --2- 172.21.15.44:0/3434073605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d268923300 0x55d268922a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.734+0000 7fb6676fab80 1 --2- 172.21.15.44:0/3434073605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d268923cc0 0x55d26892f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.734+0000 7fb6676fab80 1 -- 172.21.15.44:0/3434073605 >> 172.21.15.44:0/3434073605 conn(0x55d268898030 msgr2=0x55d2689222f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.734+0000 7fb6676fab80 1 -- 172.21.15.44:0/3434073605 shutdown_connections 2023-09-10T01:00:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.734+0000 7fb6676fab80 1 -- 172.21.15.44:0/3434073605 wait complete. 2023-09-10T01:00:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.734+0000 7fb6676fab80 1 -- start start 2023-09-10T01:00:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.734+0000 7fb6676fab80 1 Processor -- start 2023-09-10T01:00:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb6676fab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923300 0x55d26898c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb6676fab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d268923cc0 0x55d26898cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb6676fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2689291a0 0x55d268933c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb6676fab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d268934330 con 0x55d268923cc0 2023-09-10T01:00:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb6676fab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2689344a0 con 0x55d268923300 2023-09-10T01:00:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb6676fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d268934660 con 0x55d2689291a0 2023-09-10T01:00:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb6676fab80 10 client.0 _ll_register_callbacks cb 0x55d26875da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:00:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb6592d0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923300 0x55d26898c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb6592d0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923300 0x55d26898c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56446/0 (socket says 172.21.15.44:56446) 2023-09-10T01:00:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb658acf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d268923cc0 0x55d26898cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb6592d0700 1 -- 172.21.15.44:0/3035262921 learned_addr learned my addr 172.21.15.44:0/3035262921 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb659ad1700 1 --2- 172.21.15.44:0/3035262921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2689291a0 0x55d268933c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb6592d0700 1 -- 172.21.15.44:0/3035262921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2689291a0 msgr2=0x55d268933c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:35.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb6592d0700 1 --2- 172.21.15.44:0/3035262921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2689291a0 0x55d268933c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb6592d0700 1 -- 172.21.15.44:0/3035262921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d268923cc0 msgr2=0x55d26898cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:35.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb6592d0700 1 --2- 172.21.15.44:0/3035262921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d268923cc0 0x55d26898cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.735+0000 7fb6592d0700 1 -- 172.21.15.44:0/3035262921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d26868ac00 con 0x55d268923300 2023-09-10T01:00:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.736+0000 7fb6592d0700 1 --2- 172.21.15.44:0/3035262921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923300 0x55d26898c6b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb650009ac0 tx=0x7fb650011c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2ae95111cc22219 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.736+0000 7fb63cff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:00:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.736+0000 7fb63cff9700 1 -- 172.21.15.44:0/3035262921 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65002c040 con 0x55d268923300 2023-09-10T01:00:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.736+0000 7fb63cff9700 1 -- 172.21.15.44:0/3035262921 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb6500313a0 con 0x55d268923300 2023-09-10T01:00:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.736+0000 7fb6676fab80 1 -- 172.21.15.44:0/3035262921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb650021070 con 0x55d268923300 2023-09-10T01:00:35.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.736+0000 7fb63cff9700 1 -- 172.21.15.44:0/3035262921 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb650031560 con 0x55d268923300 2023-09-10T01:00:35.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.736+0000 7fb6676fab80 10 client.27965 fetch_fsmap learned FSMap version 84 2023-09-10T01:00:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.736+0000 7fb6676fab80 1 -- 172.21.15.44:0/3035262921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x55d2686a0250 con 0x55d268923300 2023-09-10T01:00:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.736+0000 7fb63cff9700 1 -- 172.21.15.44:0/3035262921 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb650021070 con 0x55d268923300 2023-09-10T01:00:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.736+0000 7fb63cff9700 1 -- 172.21.15.44:0/3035262921 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 84) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb6500053c0 con 0x55d268923300 2023-09-10T01:00:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.737+0000 7fb6676fab80 10 client.27965 fetch_fsmap finished waiting for FSMap version 84 2023-09-10T01:00:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.737+0000 7fb6676fab80 10 client.27965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:00:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.737+0000 7fb6676fab80 20 client.27965 populate_metadata read hostname 'smithi044' 2023-09-10T01:00:35.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.737+0000 7fb6676fab80 1 --2- 172.21.15.44:0/3035262921 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d26898a3e0 0x55d26898a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:35.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.737+0000 7fb6676fab80 4 client.27965 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:00:35.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.737+0000 7fb6676fab80 1 -- 172.21.15.44:0/3035262921 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d26886e8c0 con 0x55d26898a3e0 2023-09-10T01:00:35.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.737+0000 7fb658acf700 1 --2- 172.21.15.44:0/3035262921 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d26898a3e0 0x55d26898a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:35.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.737+0000 7fb658acf700 1 --2- 172.21.15.44:0/3035262921 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d26898a3e0 0x55d26898a7f0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eac9ebfd3266dd server_cookie=93c783a2500a979a in_seq=0 out_seq=0 2023-09-10T01:00:35.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.737+0000 7fb63cff9700 10 client.27965 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:00:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.751+0000 7fb63cff9700 1 -- 172.21.15.44:0/3035262921 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d26886e8c0 con 0x55d26898a3e0 2023-09-10T01:00:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.751+0000 7fb63cff9700 10 client.27965 handle_command_reply: tid=0 2023-09-10T01:00:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.751+0000 7fb6676fab80 1 client.27965 shutdown 2023-09-10T01:00:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.751+0000 7fb6676fab80 1 -- 172.21.15.44:0/3035262921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923300 msgr2=0x55d26898c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.751+0000 7fb6676fab80 1 --2- 172.21.15.44:0/3035262921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923300 0x55d26898c6b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb650009ac0 tx=0x7fb650011c10 comp rx=0 tx=0).stop 2023-09-10T01:00:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.751+0000 7fb6676fab80 1 -- 172.21.15.44:0/3035262921 shutdown_connections 2023-09-10T01:00:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.751+0000 7fb6676fab80 1 --2- 172.21.15.44:0/3035262921 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d26898a3e0 0x55d26898a7f0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.751+0000 7fb6676fab80 1 --2- 172.21.15.44:0/3035262921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2689291a0 0x55d268933c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.751+0000 7fb63cff9700 0 client.27965 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:00:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.751+0000 7fb6676fab80 1 --2- 172.21.15.44:0/3035262921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d268923300 0x55d26898c6b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.751+0000 7fb6676fab80 1 --2- 172.21.15.44:0/3035262921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d268923cc0 0x55d26898cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.751+0000 7fb6676fab80 1 -- 172.21.15.44:0/3035262921 >> 172.21.15.44:0/3035262921 conn(0x55d268898030 msgr2=0x55d268922b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.751+0000 7fb6676fab80 1 -- 172.21.15.44:0/3035262921 shutdown_connections 2023-09-10T01:00:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.751+0000 7fb6676fab80 1 -- 172.21.15.44:0/3035262921 wait complete. 2023-09-10T01:00:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.751+0000 7fb6676fab80 20 client.27965 trim_cache size 0 max 16384 2023-09-10T01:00:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:00:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:00:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:00:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:00:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.752+0000 7fb65b534700 1 -- 172.21.15.44:0/3317491814 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb6400a07b0 msgr2=0x7fb6400a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.752+0000 7fb65b534700 1 --2- 172.21.15.44:0/3317491814 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb6400a07b0 0x7fb6400a2c70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb650005d10 tx=0x7fb650009310 comp rx=0 tx=0).stop 2023-09-10T01:00:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.752+0000 7fb65b534700 1 -- 172.21.15.44:0/3317491814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6541a2880 msgr2=0x7fb6541a4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.752+0000 7fb65b534700 1 --2- 172.21.15.44:0/3317491814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6541a2880 0x7fb6541a4c70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00efc0 tx=0x7fb64c00c3a0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.752+0000 7fb65b534700 1 -- 172.21.15.44:0/3317491814 shutdown_connections 2023-09-10T01:00:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.752+0000 7fb65b534700 1 --2- 172.21.15.44:0/3317491814 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb6400a07b0 0x7fb6400a2c70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.752+0000 7fb65b534700 1 --2- 172.21.15.44:0/3317491814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6541a2880 0x7fb6541a4c70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.752+0000 7fb65b534700 1 --2- 172.21.15.44:0/3317491814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6540fee80 0x7fb65410d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.752+0000 7fb65b534700 1 --2- 172.21.15.44:0/3317491814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb654103fa0 0x7fb65410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.752+0000 7fb65b534700 1 -- 172.21.15.44:0/3317491814 >> 172.21.15.44:0/3317491814 conn(0x7fb6540769d0 msgr2=0x7fb654103620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.752+0000 7fb65b534700 1 -- 172.21.15.44:0/3317491814 shutdown_connections 2023-09-10T01:00:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:35.752+0000 7fb65b534700 1 -- 172.21.15.44:0/3317491814 wait complete. 2023-09-10T01:00:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:36 smithi131 ceph-mon[103713]: pgmap v631: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 282 op/s 2023-09-10T01:00:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:36 smithi044 ceph-mon[98676]: pgmap v631: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 282 op/s 2023-09-10T01:00:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:36 smithi099 ceph-mon[102692]: pgmap v631: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 282 op/s 2023-09-10T01:00:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:38 smithi131 ceph-mon[103713]: pgmap v632: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 161 op/s 2023-09-10T01:00:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:38 smithi044 ceph-mon[98676]: pgmap v632: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 161 op/s 2023-09-10T01:00:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:38 smithi099 ceph-mon[102692]: pgmap v632: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 161 op/s 2023-09-10T01:00:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:40 smithi131 ceph-mon[103713]: pgmap v633: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 161 op/s 2023-09-10T01:00:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:40 smithi044 ceph-mon[98676]: pgmap v633: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 161 op/s 2023-09-10T01:00:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:40 smithi099 ceph-mon[102692]: pgmap v633: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 161 op/s 2023-09-10T01:00:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:42 smithi131 ceph-mon[103713]: pgmap v634: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 264 op/s 2023-09-10T01:00:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:42 smithi044 ceph-mon[98676]: pgmap v634: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 264 op/s 2023-09-10T01:00:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:42 smithi099 ceph-mon[102692]: pgmap v634: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 264 op/s 2023-09-10T01:00:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:44 smithi044 ceph-mon[98676]: pgmap v635: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2023-09-10T01:00:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:44 smithi099 ceph-mon[102692]: pgmap v635: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2023-09-10T01:00:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:44 smithi131 ceph-mon[103713]: pgmap v635: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2023-09-10T01:00:47.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:46 smithi044 ceph-mon[98676]: pgmap v636: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 319 op/s 2023-09-10T01:00:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:46 smithi099 ceph-mon[102692]: pgmap v636: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 319 op/s 2023-09-10T01:00:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:46 smithi131 ceph-mon[103713]: pgmap v636: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 319 op/s 2023-09-10T01:00:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.870+0000 7ff172fb0700 1 -- 172.21.15.44:0/428916666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16c0ff690 msgr2=0x7ff16c0ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.870+0000 7ff172fb0700 1 --2- 172.21.15.44:0/428916666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16c0ff690 0x7ff16c0ffb10 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff1580099c0 tx=0x7ff158009cd0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.870+0000 7ff172fb0700 1 -- 172.21.15.44:0/428916666 shutdown_connections 2023-09-10T01:00:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.870+0000 7ff172fb0700 1 --2- 172.21.15.44:0/428916666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff16c100050 0x7ff16c10c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.870+0000 7ff172fb0700 1 --2- 172.21.15.44:0/428916666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff16c105560 0x7ff16c105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.870+0000 7ff172fb0700 1 --2- 172.21.15.44:0/428916666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16c0ff690 0x7ff16c0ffb10 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.870+0000 7ff172fb0700 1 -- 172.21.15.44:0/428916666 >> 172.21.15.44:0/428916666 conn(0x7ff16c0773d0 msgr2=0x7ff16c0777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.871+0000 7ff172fb0700 1 -- 172.21.15.44:0/428916666 shutdown_connections 2023-09-10T01:00:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.871+0000 7ff172fb0700 1 -- 172.21.15.44:0/428916666 wait complete. 2023-09-10T01:00:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.871+0000 7ff172fb0700 1 Processor -- start 2023-09-10T01:00:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.871+0000 7ff172fb0700 1 -- start start 2023-09-10T01:00:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.871+0000 7ff172fb0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff16c0ff690 0x7ff16c1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.871+0000 7ff172fb0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff16c100050 0x7ff16c196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.871+0000 7ff172fb0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16c105560 0x7ff16c199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.871+0000 7ff172fb0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c19a5e0 con 0x7ff16c105560 2023-09-10T01:00:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.871+0000 7ff172fb0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c19a750 con 0x7ff16c100050 2023-09-10T01:00:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.871+0000 7ff172fb0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c19a910 con 0x7ff16c0ff690 2023-09-10T01:00:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.872+0000 7ff17154d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16c105560 0x7ff16c199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.872+0000 7ff170d4c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff16c0ff690 0x7ff16c1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.872+0000 7ff17154d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16c105560 0x7ff16c199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54724/0 (socket says 172.21.15.44:54724) 2023-09-10T01:00:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.872+0000 7ff17154d700 1 -- 172.21.15.44:0/111016818 learned_addr learned my addr 172.21.15.44:0/111016818 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.872+0000 7ff16bfff700 1 --2- 172.21.15.44:0/111016818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff16c100050 0x7ff16c196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.872+0000 7ff17154d700 1 -- 172.21.15.44:0/111016818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff16c0ff690 msgr2=0x7ff16c1964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.872+0000 7ff17154d700 1 --2- 172.21.15.44:0/111016818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff16c0ff690 0x7ff16c1964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.872+0000 7ff17154d700 1 -- 172.21.15.44:0/111016818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff16c100050 msgr2=0x7ff16c196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.872+0000 7ff17154d700 1 --2- 172.21.15.44:0/111016818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff16c100050 0x7ff16c196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.872+0000 7ff17154d700 1 -- 172.21.15.44:0/111016818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1580095f0 con 0x7ff16c105560 2023-09-10T01:00:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.872+0000 7ff17154d700 1 --2- 172.21.15.44:0/111016818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16c105560 0x7ff16c199fe0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff16400efc0 tx=0x7ff16400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.872+0000 7ff169ffb700 1 -- 172.21.15.44:0/111016818 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff164018070 con 0x7ff16c105560 2023-09-10T01:00:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.872+0000 7ff169ffb700 1 -- 172.21.15.44:0/111016818 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff16400f040 con 0x7ff16c105560 2023-09-10T01:00:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.872+0000 7ff169ffb700 1 -- 172.21.15.44:0/111016818 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16400cb00 con 0x7ff16c105560 2023-09-10T01:00:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.872+0000 7ff172fb0700 1 -- 172.21.15.44:0/111016818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16c19ab90 con 0x7ff16c105560 2023-09-10T01:00:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.872+0000 7ff172fb0700 1 -- 172.21.15.44:0/111016818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16c19ae40 con 0x7ff16c105560 2023-09-10T01:00:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.875+0000 7ff169ffb700 1 -- 172.21.15.44:0/111016818 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ff16400cce0 con 0x7ff16c105560 2023-09-10T01:00:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.875+0000 7ff169ffb700 1 --2- 172.21.15.44:0/111016818 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff15c0a0830 0x7ff15c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.875+0000 7ff169ffb700 1 -- 172.21.15.44:0/111016818 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff1640cbdf0 con 0x7ff16c105560 2023-09-10T01:00:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.875+0000 7ff170d4c700 1 --2- 172.21.15.44:0/111016818 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff15c0a0830 0x7ff15c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.876+0000 7ff170d4c700 1 --2- 172.21.15.44:0/111016818 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff15c0a0830 0x7ff15c0a2cf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff160009c60 tx=0x7ff160009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.876+0000 7ff17f176b80 1 Processor -- start 2023-09-10T01:00:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.876+0000 7ff17f176b80 1 -- start start 2023-09-10T01:00:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.876+0000 7ff17f176b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc6805b0 0x55cebc680990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.876+0000 7ff17f176b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67a710 0x55cebc67ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.876+0000 7ff17f176b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc67b0d0 0x55cebc687340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.876+0000 7ff17f176b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebc5cf6d0 con 0x55cebc67b0d0 2023-09-10T01:00:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.876+0000 7ff17f176b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebc5ccff0 con 0x55cebc67a710 2023-09-10T01:00:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.876+0000 7ff17f176b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebc5c88e0 con 0x55cebc6805b0 2023-09-10T01:00:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.876+0000 7ff17154d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc67b0d0 0x55cebc687340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.876+0000 7ff17154d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc67b0d0 0x55cebc687340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54732/0 (socket says 172.21.15.44:54732) 2023-09-10T01:00:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.876+0000 7ff170d4c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc6805b0 0x55cebc680990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.876+0000 7ff17154d700 1 -- 172.21.15.44:0/804685692 learned_addr learned my addr 172.21.15.44:0/804685692 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.876+0000 7ff16bfff700 1 --2- 172.21.15.44:0/804685692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67a710 0x55cebc67ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.877+0000 7ff17154d700 1 -- 172.21.15.44:0/804685692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc6805b0 msgr2=0x55cebc680990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.877+0000 7ff17154d700 1 --2- 172.21.15.44:0/804685692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc6805b0 0x55cebc680990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.877+0000 7ff17154d700 1 -- 172.21.15.44:0/804685692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67a710 msgr2=0x55cebc67ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.877+0000 7ff17154d700 1 --2- 172.21.15.44:0/804685692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67a710 0x55cebc67ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.877+0000 7ff17154d700 1 -- 172.21.15.44:0/804685692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cebc3f4d80 con 0x55cebc67b0d0 2023-09-10T01:00:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.877+0000 7ff17154d700 1 --2- 172.21.15.44:0/804685692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc67b0d0 0x55cebc687340 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff1640090a0 tx=0x7ff164095300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290d6d333cc35726 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.877+0000 7ff156ffd700 1 -- 172.21.15.44:0/804685692 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff164099ea0 con 0x55cebc67b0d0 2023-09-10T01:00:48.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.877+0000 7ff156ffd700 1 -- 172.21.15.44:0/804685692 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff164099590 con 0x55cebc67b0d0 2023-09-10T01:00:48.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.877+0000 7ff156ffd700 1 -- 172.21.15.44:0/804685692 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff164099350 con 0x55cebc67b0d0 2023-09-10T01:00:48.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 -- 172.21.15.44:0/804685692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc67b0d0 msgr2=0x55cebc687340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:48.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 --2- 172.21.15.44:0/804685692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc67b0d0 0x55cebc687340 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff1640090a0 tx=0x7ff164095300 comp rx=0 tx=0).stop 2023-09-10T01:00:48.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 -- 172.21.15.44:0/804685692 shutdown_connections 2023-09-10T01:00:48.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 --2- 172.21.15.44:0/804685692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc67b0d0 0x55cebc687340 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 --2- 172.21.15.44:0/804685692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc6805b0 0x55cebc680990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 --2- 172.21.15.44:0/804685692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67a710 0x55cebc67ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 -- 172.21.15.44:0/804685692 >> 172.21.15.44:0/804685692 conn(0x55cebc5eb030 msgr2=0x55cebc671560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:48.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 -- 172.21.15.44:0/804685692 shutdown_connections 2023-09-10T01:00:48.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 -- 172.21.15.44:0/804685692 wait complete. 2023-09-10T01:00:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 -- start start 2023-09-10T01:00:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 Processor -- start 2023-09-10T01:00:48.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc67a710 0x55cebc687220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67b0d0 0x55cebc6844a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc684ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebc566b10 con 0x55cebc6805b0 2023-09-10T01:00:48.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebc566810 con 0x55cebc67b0d0 2023-09-10T01:00:48.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebc44a8c0 con 0x55cebc67a710 2023-09-10T01:00:48.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17f176b80 10 client.0 _ll_register_callbacks cb 0x55cebc4b0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:00:48.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.878+0000 7ff17154d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc684ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff170d4c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc67a710 0x55cebc687220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff17154d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc684ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54738/0 (socket says 172.21.15.44:54738) 2023-09-10T01:00:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff16bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67b0d0 0x55cebc6844a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff17154d700 1 -- 172.21.15.44:0/2272429893 learned_addr learned my addr 172.21.15.44:0/2272429893 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff17154d700 1 -- 172.21.15.44:0/2272429893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc67a710 msgr2=0x55cebc687220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff17154d700 1 --2- 172.21.15.44:0/2272429893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc67a710 0x55cebc687220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff17154d700 1 -- 172.21.15.44:0/2272429893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67b0d0 msgr2=0x55cebc6844a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff17154d700 1 --2- 172.21.15.44:0/2272429893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67b0d0 0x55cebc6844a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff17154d700 1 -- 172.21.15.44:0/2272429893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16c19ae40 con 0x55cebc6805b0 2023-09-10T01:00:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff16bfff700 1 --2- 172.21.15.44:0/2272429893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67b0d0 0x55cebc6844a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:00:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff17154d700 1 --2- 172.21.15.44:0/2272429893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc684ad0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff1640c9740 tx=0x7ff164099320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78ba829856fa88d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff14bfff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:00:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff14bfff700 1 -- 172.21.15.44:0/2272429893 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff164099ea0 con 0x55cebc6805b0 2023-09-10T01:00:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff14bfff700 1 -- 172.21.15.44:0/2272429893 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff164098e90 con 0x55cebc6805b0 2023-09-10T01:00:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff17f176b80 1 -- 172.21.15.44:0/2272429893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cebc520490 con 0x55cebc6805b0 2023-09-10T01:00:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff14bfff700 1 -- 172.21.15.44:0/2272429893 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1640b1230 con 0x55cebc6805b0 2023-09-10T01:00:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff17f176b80 10 client.18321 fetch_fsmap learned FSMap version 84 2023-09-10T01:00:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.879+0000 7ff17f176b80 1 -- 172.21.15.44:0/2272429893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x55cebc3ddc00 con 0x55cebc6805b0 2023-09-10T01:00:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.880+0000 7ff14bfff700 1 -- 172.21.15.44:0/2272429893 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cebc520490 con 0x55cebc6805b0 2023-09-10T01:00:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.880+0000 7ff14bfff700 1 -- 172.21.15.44:0/2272429893 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 84) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff164097df0 con 0x55cebc6805b0 2023-09-10T01:00:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.880+0000 7ff17f176b80 10 client.18321 fetch_fsmap finished waiting for FSMap version 84 2023-09-10T01:00:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.880+0000 7ff17f176b80 10 client.18321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:00:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.880+0000 7ff17f176b80 20 client.18321 populate_metadata read hostname 'smithi044' 2023-09-10T01:00:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.880+0000 7ff17f176b80 1 --2- 172.21.15.44:0/2272429893 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cebc694380 0x55cebc696770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.880+0000 7ff17f176b80 4 client.18321 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:00:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.880+0000 7ff17f176b80 1 -- 172.21.15.44:0/2272429893 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cebc5dd840 con 0x55cebc694380 2023-09-10T01:00:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.880+0000 7ff170d4c700 1 --2- 172.21.15.44:0/2272429893 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cebc694380 0x55cebc696770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.880+0000 7ff14bfff700 10 client.18321 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:00:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.880+0000 7ff170d4c700 1 --2- 172.21.15.44:0/2272429893 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cebc694380 0x55cebc696770 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=113fd21124f1bf42 server_cookie=5be5d13d2a825924 in_seq=0 out_seq=0 2023-09-10T01:00:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.881+0000 7ff14bfff700 1 -- 172.21.15.44:0/2272429893 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cebc5dd840 con 0x55cebc694380 2023-09-10T01:00:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.881+0000 7ff14bfff700 10 client.18321 handle_command_reply: tid=0 2023-09-10T01:00:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.881+0000 7ff17f176b80 1 client.18321 shutdown 2023-09-10T01:00:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.881+0000 7ff17f176b80 1 -- 172.21.15.44:0/2272429893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 msgr2=0x55cebc684ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.881+0000 7ff17f176b80 1 --2- 172.21.15.44:0/2272429893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc684ad0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff1640c9740 tx=0x7ff164099320 comp rx=0 tx=0).stop 2023-09-10T01:00:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.881+0000 7ff17f176b80 1 -- 172.21.15.44:0/2272429893 shutdown_connections 2023-09-10T01:00:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.881+0000 7ff17f176b80 1 --2- 172.21.15.44:0/2272429893 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cebc694380 0x55cebc696770 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.881+0000 7ff17f176b80 1 --2- 172.21.15.44:0/2272429893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc684ad0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.881+0000 7ff14bfff700 0 client.18321 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:00:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.881+0000 7ff17f176b80 1 --2- 172.21.15.44:0/2272429893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc67a710 0x55cebc687220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.881+0000 7ff17f176b80 1 --2- 172.21.15.44:0/2272429893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67b0d0 0x55cebc6844a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.881+0000 7ff17f176b80 1 -- 172.21.15.44:0/2272429893 >> 172.21.15.44:0/2272429893 conn(0x55cebc5eb030 msgr2=0x55cebc6707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.881+0000 7ff17f176b80 1 -- 172.21.15.44:0/2272429893 shutdown_connections 2023-09-10T01:00:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.881+0000 7ff17f176b80 1 -- 172.21.15.44:0/2272429893 wait complete. 2023-09-10T01:00:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.881+0000 7ff17f176b80 20 client.18321 trim_cache size 0 max 16384 2023-09-10T01:00:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17f176b80 1 Processor -- start 2023-09-10T01:00:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17f176b80 1 -- start start 2023-09-10T01:00:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17f176b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67a710 0x55cebc68b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17f176b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc67b0d0 0x55cebc67f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17f176b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc67fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17f176b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebc668e00 con 0x55cebc6805b0 2023-09-10T01:00:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17f176b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebc685a80 con 0x55cebc67a710 2023-09-10T01:00:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17f176b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebc672c30 con 0x55cebc67b0d0 2023-09-10T01:00:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17154d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc67fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17154d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc67fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54746/0 (socket says 172.21.15.44:54746) 2023-09-10T01:00:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17154d700 1 -- 172.21.15.44:0/4149492361 learned_addr learned my addr 172.21.15.44:0/4149492361 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff170d4c700 1 --2- 172.21.15.44:0/4149492361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67a710 0x55cebc68b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff16bfff700 1 --2- 172.21.15.44:0/4149492361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc67b0d0 0x55cebc67f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17154d700 1 -- 172.21.15.44:0/4149492361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc67b0d0 msgr2=0x55cebc67f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17154d700 1 --2- 172.21.15.44:0/4149492361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc67b0d0 0x55cebc67f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17154d700 1 -- 172.21.15.44:0/4149492361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67a710 msgr2=0x55cebc68b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17154d700 1 --2- 172.21.15.44:0/4149492361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67a710 0x55cebc68b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17154d700 1 -- 172.21.15.44:0/4149492361 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cebc41b1f0 con 0x55cebc6805b0 2023-09-10T01:00:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.897+0000 7ff17154d700 1 --2- 172.21.15.44:0/4149492361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc67fa10 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff1640a1030 tx=0x7ff164004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29924dcf621c5a6c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.898+0000 7ff1577fe700 1 -- 172.21.15.44:0/4149492361 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff164099ea0 con 0x55cebc6805b0 2023-09-10T01:00:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.898+0000 7ff1577fe700 1 -- 172.21.15.44:0/4149492361 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1640b1230 con 0x55cebc6805b0 2023-09-10T01:00:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.898+0000 7ff1577fe700 1 -- 172.21.15.44:0/4149492361 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1640972a0 con 0x55cebc6805b0 2023-09-10T01:00:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.898+0000 7ff17f176b80 1 -- 172.21.15.44:0/4149492361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 msgr2=0x55cebc67fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.898+0000 7ff17f176b80 1 --2- 172.21.15.44:0/4149492361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc67fa10 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff1640a1030 tx=0x7ff164004cc0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.898+0000 7ff17f176b80 1 -- 172.21.15.44:0/4149492361 shutdown_connections 2023-09-10T01:00:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.898+0000 7ff17f176b80 1 --2- 172.21.15.44:0/4149492361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc67fa10 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.898+0000 7ff17f176b80 1 --2- 172.21.15.44:0/4149492361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67a710 0x55cebc68b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.898+0000 7ff17f176b80 1 --2- 172.21.15.44:0/4149492361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc67b0d0 0x55cebc67f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.898+0000 7ff17f176b80 1 -- 172.21.15.44:0/4149492361 >> 172.21.15.44:0/4149492361 conn(0x55cebc5eb030 msgr2=0x55cebc68ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.898+0000 7ff17f176b80 1 -- 172.21.15.44:0/4149492361 shutdown_connections 2023-09-10T01:00:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.898+0000 7ff17f176b80 1 -- 172.21.15.44:0/4149492361 wait complete. 2023-09-10T01:00:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff17f176b80 1 -- start start 2023-09-10T01:00:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff17f176b80 1 Processor -- start 2023-09-10T01:00:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff17f176b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc67a710 0x55cebc6e0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff17f176b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67b0d0 0x55cebc6e1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff17f176b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc68ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff17f176b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebc68b220 con 0x55cebc6805b0 2023-09-10T01:00:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff17f176b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebc68b390 con 0x55cebc67b0d0 2023-09-10T01:00:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff17f176b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebc68b550 con 0x55cebc67a710 2023-09-10T01:00:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff17f176b80 10 client.0 _ll_register_callbacks cb 0x7ff160005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:00:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff17154d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc68ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff16bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67b0d0 0x55cebc6e1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff16bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67b0d0 0x55cebc6e1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56490/0 (socket says 172.21.15.44:56490) 2023-09-10T01:00:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff17154d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc68ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54752/0 (socket says 172.21.15.44:54752) 2023-09-10T01:00:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff170d4c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc67a710 0x55cebc6e0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff16bfff700 1 -- 172.21.15.44:0/96968112 learned_addr learned my addr 172.21.15.44:0/96968112 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:00:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff16bfff700 1 -- 172.21.15.44:0/96968112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc67a710 msgr2=0x55cebc6e0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff16bfff700 1 --2- 172.21.15.44:0/96968112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc67a710 0x55cebc6e0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff16bfff700 1 -- 172.21.15.44:0/96968112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 msgr2=0x55cebc68ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:00:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff16bfff700 1 --2- 172.21.15.44:0/96968112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc68ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff16bfff700 1 -- 172.21.15.44:0/96968112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cebc3ddc00 con 0x55cebc67b0d0 2023-09-10T01:00:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.899+0000 7ff17154d700 1 --2- 172.21.15.44:0/96968112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc68ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:00:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.900+0000 7ff16bfff700 1 --2- 172.21.15.44:0/96968112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67b0d0 0x55cebc6e1180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff1580098e0 tx=0x7ff158005790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ff38fc8beff544 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:00:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.900+0000 7ff14bfff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:00:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.900+0000 7ff14bfff700 1 -- 172.21.15.44:0/96968112 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158003960 con 0x55cebc67b0d0 2023-09-10T01:00:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.900+0000 7ff17f176b80 1 -- 172.21.15.44:0/96968112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff164099590 con 0x55cebc67b0d0 2023-09-10T01:00:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.900+0000 7ff14bfff700 1 -- 172.21.15.44:0/96968112 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff158003ac0 con 0x55cebc67b0d0 2023-09-10T01:00:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.900+0000 7ff14bfff700 1 -- 172.21.15.44:0/96968112 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff158003c80 con 0x55cebc67b0d0 2023-09-10T01:00:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.900+0000 7ff17f176b80 10 client.27983 fetch_fsmap learned FSMap version 84 2023-09-10T01:00:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.900+0000 7ff17f176b80 1 -- 172.21.15.44:0/96968112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x55cebc3f3250 con 0x55cebc67b0d0 2023-09-10T01:00:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.900+0000 7ff14bfff700 1 -- 172.21.15.44:0/96968112 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff164099590 con 0x55cebc67b0d0 2023-09-10T01:00:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.900+0000 7ff14bfff700 1 -- 172.21.15.44:0/96968112 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 84) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff1580099f0 con 0x55cebc67b0d0 2023-09-10T01:00:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.900+0000 7ff17f176b80 10 client.27983 fetch_fsmap finished waiting for FSMap version 84 2023-09-10T01:00:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.900+0000 7ff17f176b80 10 client.27983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:00:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.900+0000 7ff17f176b80 20 client.27983 populate_metadata read hostname 'smithi044' 2023-09-10T01:00:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.900+0000 7ff17f176b80 1 --2- 172.21.15.44:0/96968112 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cebc6e89e0 0x55cebc6eae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:00:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.900+0000 7ff17f176b80 4 client.27983 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:00:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.900+0000 7ff17f176b80 1 -- 172.21.15.44:0/96968112 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cebc5c18c0 con 0x55cebc6e89e0 2023-09-10T01:00:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.901+0000 7ff170d4c700 1 --2- 172.21.15.44:0/96968112 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cebc6e89e0 0x55cebc6eae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:00:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.901+0000 7ff14bfff700 10 client.27983 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:00:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.901+0000 7ff170d4c700 1 --2- 172.21.15.44:0/96968112 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cebc6e89e0 0x55cebc6eae00 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce9957b9303a6ae2 server_cookie=3839c46bd036c831 in_seq=0 out_seq=0 2023-09-10T01:00:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.930+0000 7ff14bfff700 1 -- 172.21.15.44:0/96968112 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cebc5c18c0 con 0x55cebc6e89e0 2023-09-10T01:00:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.930+0000 7ff14bfff700 10 client.27983 handle_command_reply: tid=0 2023-09-10T01:00:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.930+0000 7ff17f176b80 1 client.27983 shutdown 2023-09-10T01:00:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.930+0000 7ff17f176b80 1 -- 172.21.15.44:0/96968112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67b0d0 msgr2=0x55cebc6e1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.930+0000 7ff17f176b80 1 --2- 172.21.15.44:0/96968112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67b0d0 0x55cebc6e1180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff1580098e0 tx=0x7ff158005790 comp rx=0 tx=0).stop 2023-09-10T01:00:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.930+0000 7ff17f176b80 1 -- 172.21.15.44:0/96968112 shutdown_connections 2023-09-10T01:00:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.930+0000 7ff17f176b80 1 --2- 172.21.15.44:0/96968112 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cebc6e89e0 0x55cebc6eae00 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff17f176b80 1 --2- 172.21.15.44:0/96968112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cebc6805b0 0x55cebc68ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff17f176b80 1 --2- 172.21.15.44:0/96968112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cebc67a710 0x55cebc6e0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff17f176b80 1 --2- 172.21.15.44:0/96968112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cebc67b0d0 0x55cebc6e1180 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff14bfff700 0 client.27983 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:00:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff17f176b80 1 -- 172.21.15.44:0/96968112 >> 172.21.15.44:0/96968112 conn(0x55cebc5eb030 msgr2=0x55cebc67f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff17f176b80 1 -- 172.21.15.44:0/96968112 shutdown_connections 2023-09-10T01:00:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff17f176b80 1 -- 172.21.15.44:0/96968112 wait complete. 2023-09-10T01:00:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff17f176b80 20 client.27983 trim_cache size 0 max 16384 2023-09-10T01:00:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:00:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:00:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:00:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:00:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff172fb0700 1 -- 172.21.15.44:0/111016818 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff15c0a0830 msgr2=0x7ff15c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff172fb0700 1 --2- 172.21.15.44:0/111016818 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff15c0a0830 0x7ff15c0a2cf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff160009c60 tx=0x7ff160009260 comp rx=0 tx=0).stop 2023-09-10T01:00:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff172fb0700 1 -- 172.21.15.44:0/111016818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16c105560 msgr2=0x7ff16c199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:00:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff172fb0700 1 --2- 172.21.15.44:0/111016818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16c105560 0x7ff16c199fe0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff16400efc0 tx=0x7ff16400c3a0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff172fb0700 1 -- 172.21.15.44:0/111016818 shutdown_connections 2023-09-10T01:00:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff172fb0700 1 --2- 172.21.15.44:0/111016818 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff15c0a0830 0x7ff15c0a2cf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff172fb0700 1 --2- 172.21.15.44:0/111016818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16c105560 0x7ff16c199fe0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff172fb0700 1 --2- 172.21.15.44:0/111016818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff16c0ff690 0x7ff16c1964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff172fb0700 1 --2- 172.21.15.44:0/111016818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff16c100050 0x7ff16c196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:00:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff172fb0700 1 -- 172.21.15.44:0/111016818 >> 172.21.15.44:0/111016818 conn(0x7ff16c0773d0 msgr2=0x7ff16c103490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:00:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff172fb0700 1 -- 172.21.15.44:0/111016818 shutdown_connections 2023-09-10T01:00:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:00:48.931+0000 7ff172fb0700 1 -- 172.21.15.44:0/111016818 wait complete. 2023-09-10T01:00:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:48 smithi044 ceph-mon[98676]: pgmap v637: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-09-10T01:00:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:48 smithi099 ceph-mon[102692]: pgmap v637: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-09-10T01:00:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:48 smithi131 ceph-mon[103713]: pgmap v637: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-09-10T01:00:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:50 smithi099 ceph-mon[102692]: pgmap v638: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-09-10T01:00:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:50 smithi131 ceph-mon[103713]: pgmap v638: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-09-10T01:00:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:50 smithi044 ceph-mon[98676]: pgmap v638: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-09-10T01:00:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:52 smithi099 ceph-mon[102692]: pgmap v639: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 319 op/s 2023-09-10T01:00:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:52 smithi131 ceph-mon[103713]: pgmap v639: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 319 op/s 2023-09-10T01:00:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:52 smithi044 ceph-mon[98676]: pgmap v639: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 319 op/s 2023-09-10T01:00:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:54 smithi099 ceph-mon[102692]: pgmap v640: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 217 op/s 2023-09-10T01:00:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:54 smithi131 ceph-mon[103713]: pgmap v640: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 217 op/s 2023-09-10T01:00:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:54 smithi044 ceph-mon[98676]: pgmap v640: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 217 op/s 2023-09-10T01:00:56.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:55 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:00:56.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:55 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:00:56.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:55 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:00:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:56 smithi099 ceph-mon[102692]: pgmap v641: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 312 op/s 2023-09-10T01:00:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:56 smithi131 ceph-mon[103713]: pgmap v641: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 312 op/s 2023-09-10T01:00:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:56 smithi044 ceph-mon[98676]: pgmap v641: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 312 op/s 2023-09-10T01:00:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:00:58 smithi099 ceph-mon[102692]: pgmap v642: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2023-09-10T01:00:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:00:58 smithi131 ceph-mon[103713]: pgmap v642: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2023-09-10T01:00:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:00:58 smithi044 ceph-mon[98676]: pgmap v642: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2023-09-10T01:01:01.085 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:00 smithi099 ceph-mon[102692]: pgmap v643: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2023-09-10T01:01:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:00 smithi131 ceph-mon[103713]: pgmap v643: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2023-09-10T01:01:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:00 smithi044 ceph-mon[98676]: pgmap v643: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2023-09-10T01:01:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.043+0000 7f1f86c42700 1 -- 172.21.15.44:0/3180641491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f800ff760 msgr2=0x7f1f800ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.043+0000 7f1f86c42700 1 --2- 172.21.15.44:0/3180641491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f800ff760 0x7f1f800ffbe0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f1f680099c0 tx=0x7f1f68009cd0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.044+0000 7f1f86c42700 1 -- 172.21.15.44:0/3180641491 shutdown_connections 2023-09-10T01:01:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.044+0000 7f1f86c42700 1 --2- 172.21.15.44:0/3180641491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f80100120 0x7f1f8010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.044+0000 7f1f86c42700 1 --2- 172.21.15.44:0/3180641491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f801055b0 0x7f1f80105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.044+0000 7f1f86c42700 1 --2- 172.21.15.44:0/3180641491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f800ff760 0x7f1f800ffbe0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.044+0000 7f1f86c42700 1 -- 172.21.15.44:0/3180641491 >> 172.21.15.44:0/3180641491 conn(0x7f1f80074f30 msgr2=0x7f1f80075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.044+0000 7f1f86c42700 1 -- 172.21.15.44:0/3180641491 shutdown_connections 2023-09-10T01:01:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.044+0000 7f1f86c42700 1 -- 172.21.15.44:0/3180641491 wait complete. 2023-09-10T01:01:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.044+0000 7f1f86c42700 1 Processor -- start 2023-09-10T01:01:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.044+0000 7f1f86c42700 1 -- start start 2023-09-10T01:01:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.044+0000 7f1f86c42700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f800ff760 0x7f1f801964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.044+0000 7f1f86c42700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f80100120 0x7f1f80196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.044+0000 7f1f86c42700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f801055b0 0x7f1f80199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.044+0000 7f1f86c42700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8019a5e0 con 0x7f1f800ff760 2023-09-10T01:01:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.044+0000 7f1f86c42700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8019a750 con 0x7f1f80100120 2023-09-10T01:01:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.044+0000 7f1f86c42700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8019a910 con 0x7f1f801055b0 2023-09-10T01:01:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f849de700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f800ff760 0x7f1f801964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f849de700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f800ff760 0x7f1f801964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54762/0 (socket says 172.21.15.44:54762) 2023-09-10T01:01:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f7ffff700 1 --2- 172.21.15.44:0/2277006868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f80100120 0x7f1f80196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f849de700 1 -- 172.21.15.44:0/2277006868 learned_addr learned my addr 172.21.15.44:0/2277006868 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f851df700 1 --2- 172.21.15.44:0/2277006868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f801055b0 0x7f1f80199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f849de700 1 -- 172.21.15.44:0/2277006868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f801055b0 msgr2=0x7f1f80199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f849de700 1 --2- 172.21.15.44:0/2277006868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f801055b0 0x7f1f80199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f849de700 1 -- 172.21.15.44:0/2277006868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f80100120 msgr2=0x7f1f80196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f849de700 1 --2- 172.21.15.44:0/2277006868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f80100120 0x7f1f80196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f849de700 1 -- 172.21.15.44:0/2277006868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f78009c40 con 0x7f1f800ff760 2023-09-10T01:01:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f7ffff700 1 --2- 172.21.15.44:0/2277006868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f80100120 0x7f1f80196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f849de700 1 --2- 172.21.15.44:0/2277006868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f800ff760 0x7f1f801964e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000e9a0 tx=0x7f1f7000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f7dffb700 1 -- 172.21.15.44:0/2277006868 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70009790 con 0x7f1f800ff760 2023-09-10T01:01:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f7dffb700 1 -- 172.21.15.44:0/2277006868 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f7000ce50 con 0x7f1f800ff760 2023-09-10T01:01:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f7dffb700 1 -- 172.21.15.44:0/2277006868 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70018420 con 0x7f1f800ff760 2023-09-10T01:01:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f86c42700 1 -- 172.21.15.44:0/2277006868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f680095f0 con 0x7f1f800ff760 2023-09-10T01:01:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.045+0000 7f1f86c42700 1 -- 172.21.15.44:0/2277006868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f801a3370 con 0x7f1f800ff760 2023-09-10T01:01:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.047+0000 7f1f7dffb700 1 -- 172.21.15.44:0/2277006868 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1f70010430 con 0x7f1f800ff760 2023-09-10T01:01:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.048+0000 7f1f7dffb700 1 --2- 172.21.15.44:0/2277006868 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1f6c0a07b0 0x7f1f6c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.048+0000 7f1f7ffff700 1 --2- 172.21.15.44:0/2277006868 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1f6c0a07b0 0x7f1f6c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.049+0000 7f1f7dffb700 1 -- 172.21.15.44:0/2277006868 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1f70014070 con 0x7f1f800ff760 2023-09-10T01:01:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.049+0000 7f1f7ffff700 1 --2- 172.21.15.44:0/2277006868 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1f6c0a07b0 0x7f1f6c0a2c70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800b580 tx=0x7f1f6801a040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.049+0000 7f1f92e08b80 1 Processor -- start 2023-09-10T01:01:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.049+0000 7f1f92e08b80 1 -- start start 2023-09-10T01:01:02.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.049+0000 7f1f92e08b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 0x55c3c34c1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.049+0000 7f1f92e08b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34bb300 0x55c3c34bb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.049+0000 7f1f92e08b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.049+0000 7f1f92e08b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c34146d0 con 0x55c3c34bbcc0 2023-09-10T01:01:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.049+0000 7f1f92e08b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c3411ff0 con 0x55c3c34c11a0 2023-09-10T01:01:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.049+0000 7f1f92e08b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c340d8e0 con 0x55c3c34bb300 2023-09-10T01:01:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.049+0000 7f1f849de700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 0x55c3c34c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.050+0000 7f1f851df700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.050+0000 7f1f849de700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 0x55c3c34c1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56510/0 (socket says 172.21.15.44:56510) 2023-09-10T01:01:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.050+0000 7f1f849de700 1 -- 172.21.15.44:0/3646867696 learned_addr learned my addr 172.21.15.44:0/3646867696 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.050+0000 7f1f7ffff700 1 --2- 172.21.15.44:0/3646867696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34bb300 0x55c3c34bb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.050+0000 7f1f851df700 1 -- 172.21.15.44:0/3646867696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34bb300 msgr2=0x55c3c34bb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.050+0000 7f1f851df700 1 --2- 172.21.15.44:0/3646867696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34bb300 0x55c3c34bb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.050+0000 7f1f851df700 1 -- 172.21.15.44:0/3646867696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 msgr2=0x55c3c34c1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.050+0000 7f1f851df700 1 --2- 172.21.15.44:0/3646867696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 0x55c3c34c1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.050+0000 7f1f851df700 1 -- 172.21.15.44:0/3646867696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3c3239d80 con 0x55c3c34bbcc0 2023-09-10T01:01:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.050+0000 7f1f849de700 1 --2- 172.21.15.44:0/3646867696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 0x55c3c34c1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.050+0000 7f1f851df700 1 --2- 172.21.15.44:0/3646867696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34c7f30 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800e650 tx=0x7f1f78009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb5a7b1ab550d2e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.050+0000 7f1f76ffd700 1 -- 172.21.15.44:0/3646867696 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7800e040 con 0x55c3c34bbcc0 2023-09-10T01:01:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.050+0000 7f1f76ffd700 1 -- 172.21.15.44:0/3646867696 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f7800f040 con 0x55c3c34bbcc0 2023-09-10T01:01:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.050+0000 7f1f76ffd700 1 -- 172.21.15.44:0/3646867696 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f780048c0 con 0x55c3c34bbcc0 2023-09-10T01:01:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.051+0000 7f1f92e08b80 1 -- 172.21.15.44:0/3646867696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 msgr2=0x55c3c34c7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.051+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/3646867696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34c7f30 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800e650 tx=0x7f1f78009920 comp rx=0 tx=0).stop 2023-09-10T01:01:02.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.051+0000 7f1f92e08b80 1 -- 172.21.15.44:0/3646867696 shutdown_connections 2023-09-10T01:01:02.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.051+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/3646867696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34c7f30 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.051+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/3646867696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 0x55c3c34c1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.051+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/3646867696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34bb300 0x55c3c34bb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.051+0000 7f1f92e08b80 1 -- 172.21.15.44:0/3646867696 >> 172.21.15.44:0/3646867696 conn(0x55c3c3430030 msgr2=0x55c3c34b6560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.051+0000 7f1f92e08b80 1 -- 172.21.15.44:0/3646867696 shutdown_connections 2023-09-10T01:01:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.051+0000 7f1f92e08b80 1 -- 172.21.15.44:0/3646867696 wait complete. 2023-09-10T01:01:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.051+0000 7f1f92e08b80 1 -- start start 2023-09-10T01:01:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.051+0000 7f1f92e08b80 1 Processor -- start 2023-09-10T01:01:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.051+0000 7f1f92e08b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34bb300 0x55c3c34c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.051+0000 7f1f92e08b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f92e08b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 0x55c3c34c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f92e08b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c33abb10 con 0x55c3c34bbcc0 2023-09-10T01:01:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f92e08b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c33ab810 con 0x55c3c34c11a0 2023-09-10T01:01:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f92e08b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c328f8c0 con 0x55c3c34bb300 2023-09-10T01:01:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f92e08b80 10 client.0 _ll_register_callbacks cb 0x55c3c32f5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:01:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f7ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f7ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54778/0 (socket says 172.21.15.44:54778) 2023-09-10T01:01:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f7ffff700 1 -- 172.21.15.44:0/3087347083 learned_addr learned my addr 172.21.15.44:0/3087347083 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f849de700 1 --2- 172.21.15.44:0/3087347083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34bb300 0x55c3c34c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f851df700 1 --2- 172.21.15.44:0/3087347083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 0x55c3c34c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f7ffff700 1 -- 172.21.15.44:0/3087347083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34bb300 msgr2=0x55c3c34c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f7ffff700 1 --2- 172.21.15.44:0/3087347083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34bb300 0x55c3c34c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f7ffff700 1 -- 172.21.15.44:0/3087347083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 msgr2=0x55c3c34c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f7ffff700 1 --2- 172.21.15.44:0/3087347083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 0x55c3c34c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f7ffff700 1 -- 172.21.15.44:0/3087347083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f801a3370 con 0x55c3c34bbcc0 2023-09-10T01:01:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f7ffff700 1 --2- 172.21.15.44:0/3087347083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34c50a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1f68005cd0 tx=0x7f1f68009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca82d1817b3614a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f5ffff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:01:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f5ffff700 1 -- 172.21.15.44:0/3087347083 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6802c070 con 0x55c3c34bbcc0 2023-09-10T01:01:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f5ffff700 1 -- 172.21.15.44:0/3087347083 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1f68004d60 con 0x55c3c34bbcc0 2023-09-10T01:01:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f92e08b80 1 -- 172.21.15.44:0/3087347083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3c3365490 con 0x55c3c34bbcc0 2023-09-10T01:01:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f5ffff700 1 -- 172.21.15.44:0/3087347083 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f68026430 con 0x55c3c34bbcc0 2023-09-10T01:01:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f92e08b80 10 client.18339 fetch_fsmap learned FSMap version 85 2023-09-10T01:01:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.052+0000 7f1f92e08b80 1 -- 172.21.15.44:0/3087347083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=4+}) v3 -- 0x7f1f7800e330 con 0x55c3c34bbcc0 2023-09-10T01:01:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.053+0000 7f1f5ffff700 1 -- 172.21.15.44:0/3087347083 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f6803a080 con 0x55c3c34bbcc0 2023-09-10T01:01:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.053+0000 7f1f5ffff700 1 -- 172.21.15.44:0/3087347083 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 85) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1f68031420 con 0x55c3c34bbcc0 2023-09-10T01:01:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.053+0000 7f1f92e08b80 10 client.18339 fetch_fsmap finished waiting for FSMap version 85 2023-09-10T01:01:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.053+0000 7f1f92e08b80 10 client.18339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:01:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.053+0000 7f1f92e08b80 20 client.18339 populate_metadata read hostname 'smithi044' 2023-09-10T01:01:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.053+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/3087347083 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c3c34d5400 0x55c3c34d7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.053+0000 7f1f92e08b80 4 client.18339 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:01:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.053+0000 7f1f92e08b80 1 -- 172.21.15.44:0/3087347083 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3c3422840 con 0x55c3c34d5400 2023-09-10T01:01:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.053+0000 7f1f849de700 1 --2- 172.21.15.44:0/3087347083 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c3c34d5400 0x55c3c34d7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.053+0000 7f1f5ffff700 10 client.18339 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.053+0000 7f1f849de700 1 --2- 172.21.15.44:0/3087347083 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c3c34d5400 0x55c3c34d7820 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60ff2e2f9e6753f8 server_cookie=6d57be3781b51af2 in_seq=0 out_seq=0 2023-09-10T01:01:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.054+0000 7f1f5ffff700 1 -- 172.21.15.44:0/3087347083 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c3c3422840 con 0x55c3c34d5400 2023-09-10T01:01:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.054+0000 7f1f5ffff700 10 client.18339 handle_command_reply: tid=0 2023-09-10T01:01:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.054+0000 7f1f92e08b80 1 client.18339 shutdown 2023-09-10T01:01:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.054+0000 7f1f92e08b80 1 -- 172.21.15.44:0/3087347083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 msgr2=0x55c3c34c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.054+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/3087347083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34c50a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1f68005cd0 tx=0x7f1f68009310 comp rx=0 tx=0).stop 2023-09-10T01:01:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.054+0000 7f1f92e08b80 1 -- 172.21.15.44:0/3087347083 shutdown_connections 2023-09-10T01:01:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.054+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/3087347083 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c3c34d5400 0x55c3c34d7820 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.054+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/3087347083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 0x55c3c34c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.054+0000 7f1f5ffff700 0 client.18339 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.054+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/3087347083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34bb300 0x55c3c34c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.054+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/3087347083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34c50a0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.054+0000 7f1f92e08b80 1 -- 172.21.15.44:0/3087347083 >> 172.21.15.44:0/3087347083 conn(0x55c3c3430030 msgr2=0x55c3c34b57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.054+0000 7f1f92e08b80 1 -- 172.21.15.44:0/3087347083 shutdown_connections 2023-09-10T01:01:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.054+0000 7f1f92e08b80 1 -- 172.21.15.44:0/3087347083 wait complete. 2023-09-10T01:01:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.054+0000 7f1f92e08b80 20 client.18339 trim_cache size 0 max 16384 2023-09-10T01:01:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f92e08b80 1 Processor -- start 2023-09-10T01:01:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f92e08b80 1 -- start start 2023-09-10T01:01:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f92e08b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34bb300 0x55c3c34b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f92e08b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34b6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f92e08b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34c11a0 0x55c3c34c4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f92e08b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c34b2fc0 con 0x55c3c34bbcc0 2023-09-10T01:01:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f92e08b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c34bedb0 con 0x55c3c34bb300 2023-09-10T01:01:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f92e08b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c34b7320 con 0x55c3c34c11a0 2023-09-10T01:01:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f7ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f7ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34b6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54786/0 (socket says 172.21.15.44:54786) 2023-09-10T01:01:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f7ffff700 1 -- 172.21.15.44:0/4009137981 learned_addr learned my addr 172.21.15.44:0/4009137981 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f849de700 1 --2- 172.21.15.44:0/4009137981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34bb300 0x55c3c34b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f851df700 1 --2- 172.21.15.44:0/4009137981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34c11a0 0x55c3c34c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f7ffff700 1 -- 172.21.15.44:0/4009137981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34c11a0 msgr2=0x55c3c34c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f7ffff700 1 --2- 172.21.15.44:0/4009137981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34c11a0 0x55c3c34c4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f7ffff700 1 -- 172.21.15.44:0/4009137981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34bb300 msgr2=0x55c3c34b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f7ffff700 1 --2- 172.21.15.44:0/4009137981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34bb300 0x55c3c34b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f7ffff700 1 -- 172.21.15.44:0/4009137981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3c3222c00 con 0x55c3c34bbcc0 2023-09-10T01:01:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f849de700 1 --2- 172.21.15.44:0/4009137981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34bb300 0x55c3c34b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:01:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f7ffff700 1 --2- 172.21.15.44:0/4009137981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34b6460 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f1f680318d0 tx=0x7f1f68004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc76774599746ac server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.070+0000 7f1f777fe700 1 -- 172.21.15.44:0/4009137981 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6802c070 con 0x55c3c34bbcc0 2023-09-10T01:01:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.071+0000 7f1f777fe700 1 -- 172.21.15.44:0/4009137981 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f6802d040 con 0x55c3c34bbcc0 2023-09-10T01:01:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.071+0000 7f1f777fe700 1 -- 172.21.15.44:0/4009137981 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f680268c0 con 0x55c3c34bbcc0 2023-09-10T01:01:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.071+0000 7f1f92e08b80 1 -- 172.21.15.44:0/4009137981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 msgr2=0x55c3c34b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.071+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/4009137981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34b6460 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f1f680318d0 tx=0x7f1f68004a50 comp rx=0 tx=0).stop 2023-09-10T01:01:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.071+0000 7f1f92e08b80 1 -- 172.21.15.44:0/4009137981 shutdown_connections 2023-09-10T01:01:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.071+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/4009137981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34c11a0 0x55c3c34c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.071+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/4009137981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34bb300 0x55c3c34b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.071+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/4009137981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34b6460 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.071+0000 7f1f92e08b80 1 -- 172.21.15.44:0/4009137981 >> 172.21.15.44:0/4009137981 conn(0x55c3c3430030 msgr2=0x55c3c34c0180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.071+0000 7f1f92e08b80 1 -- 172.21.15.44:0/4009137981 shutdown_connections 2023-09-10T01:01:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.071+0000 7f1f92e08b80 1 -- 172.21.15.44:0/4009137981 wait complete. 2023-09-10T01:01:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f92e08b80 1 -- start start 2023-09-10T01:01:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f92e08b80 1 Processor -- start 2023-09-10T01:01:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f92e08b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34bb300 0x55c3c34ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f92e08b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f92e08b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 0x55c3c35222e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f92e08b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c3522820 con 0x55c3c34bbcc0 2023-09-10T01:01:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f92e08b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c3522990 con 0x55c3c34c11a0 2023-09-10T01:01:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f92e08b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c3522b50 con 0x55c3c34bb300 2023-09-10T01:01:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f92e08b80 10 client.0 _ll_register_callbacks cb 0x55c3c32f5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:01:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f7ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f7ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54792/0 (socket says 172.21.15.44:54792) 2023-09-10T01:01:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f849de700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34bb300 0x55c3c34ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f7ffff700 1 -- 172.21.15.44:0/1649621451 learned_addr learned my addr 172.21.15.44:0/1649621451 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f851df700 1 --2- 172.21.15.44:0/1649621451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 0x55c3c35222e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f7ffff700 1 -- 172.21.15.44:0/1649621451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34bb300 msgr2=0x55c3c34ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f7ffff700 1 --2- 172.21.15.44:0/1649621451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34bb300 0x55c3c34ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f7ffff700 1 -- 172.21.15.44:0/1649621451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 msgr2=0x55c3c35222e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f7ffff700 1 --2- 172.21.15.44:0/1649621451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 0x55c3c35222e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.072+0000 7f1f7ffff700 1 -- 172.21.15.44:0/1649621451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7800e330 con 0x55c3c34bbcc0 2023-09-10T01:01:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.073+0000 7f1f7ffff700 1 --2- 172.21.15.44:0/1649621451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34cab40 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009d10 tx=0x7f1f6801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9ae80b5f85a5da server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.073+0000 7f1f5ffff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:01:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.073+0000 7f1f5ffff700 1 -- 172.21.15.44:0/1649621451 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6802c070 con 0x55c3c34bbcc0 2023-09-10T01:01:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.073+0000 7f1f5ffff700 1 -- 172.21.15.44:0/1649621451 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1f68042670 con 0x55c3c34bbcc0 2023-09-10T01:01:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.073+0000 7f1f92e08b80 1 -- 172.21.15.44:0/1649621451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f7800f040 con 0x55c3c34bbcc0 2023-09-10T01:01:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.073+0000 7f1f5ffff700 1 -- 172.21.15.44:0/1649621451 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f68042830 con 0x55c3c34bbcc0 2023-09-10T01:01:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.073+0000 7f1f92e08b80 10 client.18345 fetch_fsmap learned FSMap version 85 2023-09-10T01:01:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.073+0000 7f1f92e08b80 1 -- 172.21.15.44:0/1649621451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=4+}) v3 -- 0x55c3c32601f0 con 0x55c3c34bbcc0 2023-09-10T01:01:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.073+0000 7f1f5ffff700 1 -- 172.21.15.44:0/1649621451 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f7800f040 con 0x55c3c34bbcc0 2023-09-10T01:01:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.073+0000 7f1f5ffff700 1 -- 172.21.15.44:0/1649621451 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 85) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1f68026990 con 0x55c3c34bbcc0 2023-09-10T01:01:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.073+0000 7f1f92e08b80 10 client.18345 fetch_fsmap finished waiting for FSMap version 85 2023-09-10T01:01:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.073+0000 7f1f92e08b80 10 client.18345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:01:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.073+0000 7f1f92e08b80 20 client.18345 populate_metadata read hostname 'smithi044' 2023-09-10T01:01:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.073+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/1649621451 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c3c3529de0 0x55c3c352c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.073+0000 7f1f92e08b80 4 client.18345 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:01:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.073+0000 7f1f92e08b80 1 -- 172.21.15.44:0/1649621451 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c3c34068c0 con 0x55c3c3529de0 2023-09-10T01:01:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.074+0000 7f1f849de700 1 --2- 172.21.15.44:0/1649621451 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c3c3529de0 0x55c3c352c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.074+0000 7f1f5ffff700 10 client.18345 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:02.074+0000 7f1f849de700 1 --2- 172.21.15.44:0/1649621451 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c3c3529de0 0x55c3c352c1d0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a2b342dac3308fb server_cookie=3143ecedffe0fca1 in_seq=0 out_seq=0 2023-09-10T01:01:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:02 smithi099 ceph-mon[102692]: pgmap v644: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 273 op/s 2023-09-10T01:01:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:02 smithi131 ceph-mon[103713]: pgmap v644: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 273 op/s 2023-09-10T01:01:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:02 smithi044 ceph-mon[98676]: pgmap v644: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 273 op/s 2023-09-10T01:01:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.426+0000 7f1f5ffff700 1 -- 172.21.15.44:0/1649621451 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c3c34068c0 con 0x55c3c3529de0 2023-09-10T01:01:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.427+0000 7f1f5ffff700 10 client.18345 handle_command_reply: tid=0 2023-09-10T01:01:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.427+0000 7f1f92e08b80 1 client.18345 shutdown 2023-09-10T01:01:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.427+0000 7f1f92e08b80 1 -- 172.21.15.44:0/1649621451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 msgr2=0x55c3c34cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.427+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/1649621451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34cab40 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009d10 tx=0x7f1f6801f000 comp rx=0 tx=0).stop 2023-09-10T01:01:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.427+0000 7f1f92e08b80 1 -- 172.21.15.44:0/1649621451 shutdown_connections 2023-09-10T01:01:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.427+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/1649621451 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c3c3529de0 0x55c3c352c1d0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.427+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/1649621451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3c34c11a0 0x55c3c35222e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.427+0000 7f1f5ffff700 0 client.18345 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.427+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/1649621451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c34bb300 0x55c3c34ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.427+0000 7f1f92e08b80 1 --2- 172.21.15.44:0/1649621451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3c34bbcc0 0x55c3c34cab40 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.427+0000 7f1f92e08b80 1 -- 172.21.15.44:0/1649621451 >> 172.21.15.44:0/1649621451 conn(0x55c3c3430030 msgr2=0x55c3c34ba2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.427+0000 7f1f92e08b80 1 -- 172.21.15.44:0/1649621451 shutdown_connections 2023-09-10T01:01:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.427+0000 7f1f92e08b80 1 -- 172.21.15.44:0/1649621451 wait complete. 2023-09-10T01:01:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.427+0000 7f1f92e08b80 20 client.18345 trim_cache size 0 max 16384 2023-09-10T01:01:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:01:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:01:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:01:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:01:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.428+0000 7f1f86c42700 1 -- 172.21.15.44:0/2277006868 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1f6c0a07b0 msgr2=0x7f1f6c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.428+0000 7f1f86c42700 1 --2- 172.21.15.44:0/2277006868 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1f6c0a07b0 0x7f1f6c0a2c70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800b580 tx=0x7f1f6801a040 comp rx=0 tx=0).stop 2023-09-10T01:01:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.428+0000 7f1f86c42700 1 -- 172.21.15.44:0/2277006868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f800ff760 msgr2=0x7f1f801964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.428+0000 7f1f86c42700 1 --2- 172.21.15.44:0/2277006868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f800ff760 0x7f1f801964e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000e9a0 tx=0x7f1f7000ecb0 comp rx=0 tx=0).stop 2023-09-10T01:01:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.428+0000 7f1f86c42700 1 -- 172.21.15.44:0/2277006868 shutdown_connections 2023-09-10T01:01:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.428+0000 7f1f86c42700 1 --2- 172.21.15.44:0/2277006868 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1f6c0a07b0 0x7f1f6c0a2c70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.428+0000 7f1f86c42700 1 --2- 172.21.15.44:0/2277006868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f801055b0 0x7f1f80199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.428+0000 7f1f86c42700 1 --2- 172.21.15.44:0/2277006868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f800ff760 0x7f1f801964e0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.428+0000 7f1f86c42700 1 --2- 172.21.15.44:0/2277006868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f80100120 0x7f1f80196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.428+0000 7f1f86c42700 1 -- 172.21.15.44:0/2277006868 >> 172.21.15.44:0/2277006868 conn(0x7f1f80074f30 msgr2=0x7f1f80067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.428+0000 7f1f86c42700 1 -- 172.21.15.44:0/2277006868 shutdown_connections 2023-09-10T01:01:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:03.428+0000 7f1f86c42700 1 -- 172.21.15.44:0/2277006868 wait complete. 2023-09-10T01:01:04.546 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: *** Caught signal (Segmentation fault) ** 2023-09-10T01:01:04.546 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: in thread 7f6a1990c700 thread_name:md_log_replay 2023-09-10T01:01:04.547 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:01:04.547 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f6a2a4bbcf0] 2023-09-10T01:01:04.547 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x55f1f77442f2] 2023-09-10T01:01:04.547 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55f1f7746543] 2023-09-10T01:01:04.547 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 4: (MDLog::_replay_thread()+0x647) [0x55f1f77afe97] 2023-09-10T01:01:04.548 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55f1f77b9381] 2023-09-10T01:01:04.548 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f6a2a4b11ca] 2023-09-10T01:01:04.548 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 7: clone() 2023-09-10T01:01:04.548 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 2023-09-10T01:01:04.137+0000 7f6a1990c700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:01:04.548 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: in thread 7f6a1990c700 thread_name:md_log_replay 2023-09-10T01:01:04.548 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 2023-09-10T01:01:04.549 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:01:04.549 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f6a2a4bbcf0] 2023-09-10T01:01:04.549 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x55f1f77442f2] 2023-09-10T01:01:04.549 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55f1f7746543] 2023-09-10T01:01:04.549 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 4: (MDLog::_replay_thread()+0x647) [0x55f1f77afe97] 2023-09-10T01:01:04.550 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55f1f77b9381] 2023-09-10T01:01:04.550 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f6a2a4b11ca] 2023-09-10T01:01:04.550 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 7: clone() 2023-09-10T01:01:04.550 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:01:04.550 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 2023-09-10T01:01:04.551 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 0> 2023-09-10T01:01:04.137+0000 7f6a1990c700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:01:04.551 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: in thread 7f6a1990c700 thread_name:md_log_replay 2023-09-10T01:01:04.551 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 2023-09-10T01:01:04.551 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:01:04.551 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f6a2a4bbcf0] 2023-09-10T01:01:04.552 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x55f1f77442f2] 2023-09-10T01:01:04.552 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55f1f7746543] 2023-09-10T01:01:04.552 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 4: (MDLog::_replay_thread()+0x647) [0x55f1f77afe97] 2023-09-10T01:01:04.552 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55f1f77b9381] 2023-09-10T01:01:04.552 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f6a2a4b11ca] 2023-09-10T01:01:04.552 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 7: clone() 2023-09-10T01:01:04.553 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:01:04.553 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 2023-09-10T01:01:04.553 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 0> 2023-09-10T01:01:04.137+0000 7f6a1990c700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:01:04.553 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: in thread 7f6a1990c700 thread_name:md_log_replay 2023-09-10T01:01:04.553 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 2023-09-10T01:01:04.554 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:01:04.554 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f6a2a4bbcf0] 2023-09-10T01:01:04.554 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x55f1f77442f2] 2023-09-10T01:01:04.555 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55f1f7746543] 2023-09-10T01:01:04.555 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 4: (MDLog::_replay_thread()+0x647) [0x55f1f77afe97] 2023-09-10T01:01:04.555 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55f1f77b9381] 2023-09-10T01:01:04.555 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f6a2a4b11ca] 2023-09-10T01:01:04.555 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 7: clone() 2023-09-10T01:01:04.555 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:01:04.556 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[148592]: 2023-09-10T01:01:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:04 smithi099 ceph-mon[102692]: pgmap v645: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 181 op/s 2023-09-10T01:01:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:04 smithi099 ceph-mon[102692]: osdmap e123: 12 total, 12 up, 12 in 2023-09-10T01:01:05.207 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:05 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Main process exited, code=exited, status=139/n/a 2023-09-10T01:01:05.207 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-mon[103713]: pgmap v645: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 181 op/s 2023-09-10T01:01:05.208 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:04 smithi131 ceph-mon[103713]: osdmap e123: 12 total, 12 up, 12 in 2023-09-10T01:01:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:04 smithi044 ceph-mon[98676]: pgmap v645: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 181 op/s 2023-09-10T01:01:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:04 smithi044 ceph-mon[98676]: osdmap e123: 12 total, 12 up, 12 in 2023-09-10T01:01:05.795 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:05 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Failed with result 'exit-code'. 2023-09-10T01:01:07.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:06 smithi099 ceph-mon[102692]: pgmap v647: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 217 op/s 2023-09-10T01:01:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:06 smithi131 ceph-mon[103713]: pgmap v647: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 217 op/s 2023-09-10T01:01:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:06 smithi044 ceph-mon[98676]: pgmap v647: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 217 op/s 2023-09-10T01:01:09.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:08 smithi099 ceph-mon[102692]: pgmap v648: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 217 op/s 2023-09-10T01:01:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:08 smithi131 ceph-mon[103713]: pgmap v648: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 217 op/s 2023-09-10T01:01:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:08 smithi044 ceph-mon[98676]: pgmap v648: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 217 op/s 2023-09-10T01:01:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:10 smithi099 ceph-mon[102692]: pgmap v649: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 217 op/s 2023-09-10T01:01:11.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:10 smithi099 ceph-mon[102692]: osdmap e124: 12 total, 12 up, 12 in 2023-09-10T01:01:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:10 smithi131 ceph-mon[103713]: pgmap v649: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 217 op/s 2023-09-10T01:01:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:10 smithi131 ceph-mon[103713]: osdmap e124: 12 total, 12 up, 12 in 2023-09-10T01:01:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:10 smithi044 ceph-mon[98676]: pgmap v649: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 217 op/s 2023-09-10T01:01:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:10 smithi044 ceph-mon[98676]: osdmap e124: 12 total, 12 up, 12 in 2023-09-10T01:01:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:12 smithi099 ceph-mon[102692]: pgmap v651: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 278 op/s 2023-09-10T01:01:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:12 smithi131 ceph-mon[103713]: pgmap v651: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 278 op/s 2023-09-10T01:01:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:12 smithi044 ceph-mon[98676]: pgmap v651: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 278 op/s 2023-09-10T01:01:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:14 smithi099 ceph-mon[102692]: pgmap v652: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 234 op/s 2023-09-10T01:01:15.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:14 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:01:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:14 smithi131 ceph-mon[103713]: pgmap v652: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 234 op/s 2023-09-10T01:01:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:14 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:01:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:14 smithi044 ceph-mon[98676]: pgmap v652: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 234 op/s 2023-09-10T01:01:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:14 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:01:15.809 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:15 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Service RestartSec=10s expired, scheduling restart. 2023-09-10T01:01:15.809 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:15 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Scheduled restart job, restart counter is at 3. 2023-09-10T01:01:15.809 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:15 smithi131 systemd[1]: Stopped Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:01:15.809 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:15 smithi131 systemd[1]: Starting Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:01:16.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:15 smithi099 ceph-mon[102692]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2023-09-10T01:01:16.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:15 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:01:16.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:15 smithi131 ceph-mon[103713]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2023-09-10T01:01:16.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:15 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:01:16.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:15 smithi044 ceph-mon[98676]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2023-09-10T01:01:16.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:15 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:01:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.640+0000 7f76baeb6700 1 -- 172.21.15.44:0/2387345621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b41055b0 msgr2=0x7f76b4105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.640+0000 7f76baeb6700 1 --2- 172.21.15.44:0/2387345621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b41055b0 0x7f76b4105990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009920 tx=0x7f76a8009c30 comp rx=0 tx=0).stop 2023-09-10T01:01:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.640+0000 7f76baeb6700 1 -- 172.21.15.44:0/2387345621 shutdown_connections 2023-09-10T01:01:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.640+0000 7f76baeb6700 1 --2- 172.21.15.44:0/2387345621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76b4100050 0x7f76b410c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.640+0000 7f76baeb6700 1 --2- 172.21.15.44:0/2387345621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b41055b0 0x7f76b4105990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.640+0000 7f76baeb6700 1 --2- 172.21.15.44:0/2387345621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b40ff690 0x7f76b40ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.640+0000 7f76baeb6700 1 -- 172.21.15.44:0/2387345621 >> 172.21.15.44:0/2387345621 conn(0x7f76b4077430 msgr2=0x7f76b4077840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.640+0000 7f76baeb6700 1 -- 172.21.15.44:0/2387345621 shutdown_connections 2023-09-10T01:01:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.640+0000 7f76baeb6700 1 -- 172.21.15.44:0/2387345621 wait complete. 2023-09-10T01:01:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.641+0000 7f76baeb6700 1 Processor -- start 2023-09-10T01:01:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.641+0000 7f76baeb6700 1 -- start start 2023-09-10T01:01:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76baeb6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b40ff690 0x7f76b4198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76baeb6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76b4100050 0x7f76b4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76baeb6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b41055b0 0x7f76b419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76baeb6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b419da60 con 0x7f76b40ff690 2023-09-10T01:01:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76baeb6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b419dbd0 con 0x7f76b41055b0 2023-09-10T01:01:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76baeb6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b419dd90 con 0x7f76b4100050 2023-09-10T01:01:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76b3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76b4100050 0x7f76b4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76b3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76b4100050 0x7f76b4199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33468/0 (socket says 172.21.15.44:33468) 2023-09-10T01:01:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76b9453700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b41055b0 0x7f76b419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76b3fff700 1 -- 172.21.15.44:0/1220054559 learned_addr learned my addr 172.21.15.44:0/1220054559 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76b8c52700 1 --2- 172.21.15.44:0/1220054559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b40ff690 0x7f76b4198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76b9453700 1 -- 172.21.15.44:0/1220054559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76b4100050 msgr2=0x7f76b4199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76b9453700 1 --2- 172.21.15.44:0/1220054559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76b4100050 0x7f76b4199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76b9453700 1 -- 172.21.15.44:0/1220054559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b40ff690 msgr2=0x7f76b4198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76b9453700 1 --2- 172.21.15.44:0/1220054559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b40ff690 0x7f76b4198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76b9453700 1 -- 172.21.15.44:0/1220054559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac009c40 con 0x7f76b41055b0 2023-09-10T01:01:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76b3fff700 1 --2- 172.21.15.44:0/1220054559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76b4100050 0x7f76b4199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76b8c52700 1 --2- 172.21.15.44:0/1220054559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b40ff690 0x7f76b4198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.642+0000 7f76b9453700 1 --2- 172.21.15.44:0/1220054559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b41055b0 0x7f76b419d440 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f76a400ea80 tx=0x7f76a400ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.643+0000 7f76b1ffb700 1 -- 172.21.15.44:0/1220054559 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a40098a0 con 0x7f76b41055b0 2023-09-10T01:01:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.643+0000 7f76b1ffb700 1 -- 172.21.15.44:0/1220054559 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76a4004030 con 0x7f76b41055b0 2023-09-10T01:01:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.643+0000 7f76b1ffb700 1 -- 172.21.15.44:0/1220054559 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a4003d60 con 0x7f76b41055b0 2023-09-10T01:01:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.643+0000 7f76baeb6700 1 -- 172.21.15.44:0/1220054559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a40095f0 con 0x7f76b41055b0 2023-09-10T01:01:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.643+0000 7f76baeb6700 1 -- 172.21.15.44:0/1220054559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a8009520 con 0x7f76b41055b0 2023-09-10T01:01:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.646+0000 7f76b1ffb700 1 -- 172.21.15.44:0/1220054559 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f76a40041a0 con 0x7f76b41055b0 2023-09-10T01:01:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.647+0000 7f76b1ffb700 1 --2- 172.21.15.44:0/1220054559 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f769c0a0830 0x7f769c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.647+0000 7f76b1ffb700 1 -- 172.21.15.44:0/1220054559 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f76a4014070 con 0x7f76b41055b0 2023-09-10T01:01:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.647+0000 7f76b8c52700 1 --2- 172.21.15.44:0/1220054559 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f769c0a0830 0x7f769c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.648+0000 7f76b8c52700 1 --2- 172.21.15.44:0/1220054559 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f769c0a0830 0x7f769c0a2cf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f76a8011040 tx=0x7f76a8019040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.648+0000 7f76c707cb80 1 Processor -- start 2023-09-10T01:01:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.648+0000 7f76c707cb80 1 -- start start 2023-09-10T01:01:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.648+0000 7f76c707cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac481a0 0x55bebac48580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.648+0000 7f76c707cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42300 0x55bebac42780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.648+0000 7f76c707cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42cc0 0x55bebac4ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.648+0000 7f76c707cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebab9b6d0 con 0x55bebac481a0 2023-09-10T01:01:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.648+0000 7f76c707cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebab98ff0 con 0x55bebac42cc0 2023-09-10T01:01:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.648+0000 7f76c707cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebab948e0 con 0x55bebac42300 2023-09-10T01:01:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.649+0000 7f76b8c52700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac481a0 0x55bebac48580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.649+0000 7f76b8c52700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac481a0 0x55bebac48580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54824/0 (socket says 172.21.15.44:54824) 2023-09-10T01:01:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.649+0000 7f76b8c52700 1 -- 172.21.15.44:0/1316480767 learned_addr learned my addr 172.21.15.44:0/1316480767 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.649+0000 7f76b3fff700 1 --2- 172.21.15.44:0/1316480767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42300 0x55bebac42780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.649+0000 7f76b9453700 1 --2- 172.21.15.44:0/1316480767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42cc0 0x55bebac4ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.649+0000 7f76b8c52700 1 -- 172.21.15.44:0/1316480767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42300 msgr2=0x55bebac42780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.649+0000 7f76b8c52700 1 --2- 172.21.15.44:0/1316480767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42300 0x55bebac42780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.649+0000 7f76b8c52700 1 -- 172.21.15.44:0/1316480767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42cc0 msgr2=0x55bebac4ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.649+0000 7f76b8c52700 1 --2- 172.21.15.44:0/1316480767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42cc0 0x55bebac4ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.649+0000 7f76b8c52700 1 -- 172.21.15.44:0/1316480767 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55beba9c0d80 con 0x55bebac481a0 2023-09-10T01:01:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.649+0000 7f76b9453700 1 --2- 172.21.15.44:0/1316480767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42cc0 0x55bebac4ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.649+0000 7f76b8c52700 1 --2- 172.21.15.44:0/1316480767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac481a0 0x55bebac48580 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f76a8016000 tx=0x7f76a801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c93929e66d6640 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.649+0000 7f769affd700 1 -- 172.21.15.44:0/1316480767 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a800b7e0 con 0x55bebac481a0 2023-09-10T01:01:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.650+0000 7f769affd700 1 -- 172.21.15.44:0/1316480767 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76a801d040 con 0x55bebac481a0 2023-09-10T01:01:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.650+0000 7f769affd700 1 -- 172.21.15.44:0/1316480767 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a80038a0 con 0x55bebac481a0 2023-09-10T01:01:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.650+0000 7f76c707cb80 1 -- 172.21.15.44:0/1316480767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac481a0 msgr2=0x55bebac48580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.650+0000 7f76c707cb80 1 --2- 172.21.15.44:0/1316480767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac481a0 0x55bebac48580 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f76a8016000 tx=0x7f76a801f000 comp rx=0 tx=0).stop 2023-09-10T01:01:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.651+0000 7f76c707cb80 1 -- 172.21.15.44:0/1316480767 shutdown_connections 2023-09-10T01:01:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.651+0000 7f76c707cb80 1 --2- 172.21.15.44:0/1316480767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42cc0 0x55bebac4ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.651+0000 7f76c707cb80 1 --2- 172.21.15.44:0/1316480767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac481a0 0x55bebac48580 secure :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f76a8016000 tx=0x7f76a801f000 comp rx=0 tx=0).stop 2023-09-10T01:01:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.651+0000 7f76c707cb80 1 --2- 172.21.15.44:0/1316480767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42300 0x55bebac42780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.651+0000 7f76c707cb80 1 -- 172.21.15.44:0/1316480767 >> 172.21.15.44:0/1316480767 conn(0x55bebabb7030 msgr2=0x55bebac3d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.653+0000 7f76c707cb80 1 -- 172.21.15.44:0/1316480767 shutdown_connections 2023-09-10T01:01:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.653+0000 7f76c707cb80 1 -- 172.21.15.44:0/1316480767 wait complete. 2023-09-10T01:01:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.653+0000 7f76c707cb80 1 -- start start 2023-09-10T01:01:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.653+0000 7f76c707cb80 1 Processor -- start 2023-09-10T01:01:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76c707cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac42300 0x55bebac4c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76c707cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42cc0 0x55bebac4c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76c707cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac4cda0 0x55bebac60d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76c707cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebab32b10 con 0x55bebac42300 2023-09-10T01:01:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76c707cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebab32810 con 0x55bebac4cda0 2023-09-10T01:01:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76c707cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebaa168c0 con 0x55bebac42cc0 2023-09-10T01:01:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76c707cb80 10 client.0 _ll_register_callbacks cb 0x55bebaa7ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:01:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76b8c52700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac42300 0x55bebac4c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76b8c52700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac42300 0x55bebac4c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54830/0 (socket says 172.21.15.44:54830) 2023-09-10T01:01:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76b8c52700 1 -- 172.21.15.44:0/1291472675 learned_addr learned my addr 172.21.15.44:0/1291472675 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76b3fff700 1 --2- 172.21.15.44:0/1291472675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42cc0 0x55bebac4c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76b9453700 1 --2- 172.21.15.44:0/1291472675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac4cda0 0x55bebac60d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76b8c52700 1 -- 172.21.15.44:0/1291472675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42cc0 msgr2=0x55bebac4c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76b8c52700 1 --2- 172.21.15.44:0/1291472675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42cc0 0x55bebac4c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76b8c52700 1 -- 172.21.15.44:0/1291472675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac4cda0 msgr2=0x55bebac60d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76b8c52700 1 --2- 172.21.15.44:0/1291472675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac4cda0 0x55bebac60d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76b8c52700 1 -- 172.21.15.44:0/1291472675 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a8009520 con 0x55bebac42300 2023-09-10T01:01:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76b3fff700 1 --2- 172.21.15.44:0/1291472675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42cc0 0x55bebac4c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.654+0000 7f76b9453700 1 --2- 172.21.15.44:0/1291472675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac4cda0 0x55bebac60d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:01:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.655+0000 7f76b8c52700 1 --2- 172.21.15.44:0/1291472675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac42300 0x55bebac4c320 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f76a8005a80 tx=0x7f76a80098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c902ae7f7a150e08 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.656+0000 7f7693fff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:01:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.656+0000 7f7693fff700 1 -- 172.21.15.44:0/1291472675 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8003880 con 0x55bebac42300 2023-09-10T01:01:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.656+0000 7f7693fff700 1 -- 172.21.15.44:0/1291472675 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f76a800bb60 con 0x55bebac42300 2023-09-10T01:01:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.656+0000 7f76c707cb80 1 -- 172.21.15.44:0/1291472675 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bebaaec490 con 0x55bebac42300 2023-09-10T01:01:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.656+0000 7f7693fff700 1 -- 172.21.15.44:0/1291472675 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76a80039e0 con 0x55bebac42300 2023-09-10T01:01:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.656+0000 7f76c707cb80 10 client.18357 fetch_fsmap learned FSMap version 86 2023-09-10T01:01:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.656+0000 7f76c707cb80 1 -- 172.21.15.44:0/1291472675 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7f76b40684b0 con 0x55bebac42300 2023-09-10T01:01:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.656+0000 7f7693fff700 1 -- 172.21.15.44:0/1291472675 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bebaaec490 con 0x55bebac42300 2023-09-10T01:01:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.656+0000 7f7693fff700 1 -- 172.21.15.44:0/1291472675 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 86) v1 ==== 7319+0+0 (secure 0 0 0) 0x7f76a8038a10 con 0x55bebac42300 2023-09-10T01:01:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.656+0000 7f76c707cb80 10 client.18357 fetch_fsmap finished waiting for FSMap version 86 2023-09-10T01:01:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.656+0000 7f76c707cb80 10 client.18357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:01:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.656+0000 7f76c707cb80 20 client.18357 populate_metadata read hostname 'smithi044' 2023-09-10T01:01:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.656+0000 7f76c707cb80 1 --2- 172.21.15.44:0/1291472675 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bebac52ed0 0x55bebac552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.656+0000 7f76c707cb80 4 client.18357 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:01:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.656+0000 7f76c707cb80 1 -- 172.21.15.44:0/1291472675 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bebaba9840 con 0x55bebac52ed0 2023-09-10T01:01:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.657+0000 7f76b3fff700 1 --2- 172.21.15.44:0/1291472675 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bebac52ed0 0x55bebac552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.657+0000 7f7693fff700 10 client.18357 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.657+0000 7f76b3fff700 1 --2- 172.21.15.44:0/1291472675 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bebac52ed0 0x55bebac552c0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f088cc90883cdc22 server_cookie=9d70ded644adae6 in_seq=0 out_seq=0 2023-09-10T01:01:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.658+0000 7f7693fff700 1 -- 172.21.15.44:0/1291472675 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bebaba9840 con 0x55bebac52ed0 2023-09-10T01:01:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.658+0000 7f7693fff700 10 client.18357 handle_command_reply: tid=0 2023-09-10T01:01:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.658+0000 7f76c707cb80 1 client.18357 shutdown 2023-09-10T01:01:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.658+0000 7f76c707cb80 1 -- 172.21.15.44:0/1291472675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac42300 msgr2=0x55bebac4c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.658+0000 7f76c707cb80 1 --2- 172.21.15.44:0/1291472675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac42300 0x55bebac4c320 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f76a8005a80 tx=0x7f76a80098e0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.658+0000 7f76c707cb80 1 -- 172.21.15.44:0/1291472675 shutdown_connections 2023-09-10T01:01:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.658+0000 7f76c707cb80 1 --2- 172.21.15.44:0/1291472675 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bebac52ed0 0x55bebac552c0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.658+0000 7f76c707cb80 1 --2- 172.21.15.44:0/1291472675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac4cda0 0x55bebac60d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.658+0000 7f76c707cb80 1 --2- 172.21.15.44:0/1291472675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac42300 0x55bebac4c320 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.658+0000 7f76c707cb80 1 --2- 172.21.15.44:0/1291472675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42cc0 0x55bebac4c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.658+0000 7f76c707cb80 1 -- 172.21.15.44:0/1291472675 >> 172.21.15.44:0/1291472675 conn(0x55bebabb7030 msgr2=0x55bebac3c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.658+0000 7f7693fff700 0 client.18357 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.658+0000 7f76c707cb80 1 -- 172.21.15.44:0/1291472675 shutdown_connections 2023-09-10T01:01:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.658+0000 7f76c707cb80 1 -- 172.21.15.44:0/1291472675 wait complete. 2023-09-10T01:01:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.658+0000 7f76c707cb80 20 client.18357 trim_cache size 0 max 16384 2023-09-10T01:01:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.678+0000 7f76c707cb80 1 Processor -- start 2023-09-10T01:01:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.678+0000 7f76c707cb80 1 -- start start 2023-09-10T01:01:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.678+0000 7f76c707cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42300 0x55bebac49df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.678+0000 7f76c707cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42cc0 0x55bebac59c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.678+0000 7f76c707cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac4cda0 0x55bebac603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.678+0000 7f76c707cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebac46c60 con 0x55bebac4cda0 2023-09-10T01:01:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.678+0000 7f76c707cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebac4cbd0 con 0x55bebac42cc0 2023-09-10T01:01:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.678+0000 7f76c707cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebac47db0 con 0x55bebac42300 2023-09-10T01:01:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.679+0000 7f76b9453700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac4cda0 0x55bebac603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.679+0000 7f76b9453700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac4cda0 0x55bebac603b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54842/0 (socket says 172.21.15.44:54842) 2023-09-10T01:01:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.679+0000 7f76b8c52700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42300 0x55bebac49df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.679+0000 7f76b9453700 1 -- 172.21.15.44:0/3625480993 learned_addr learned my addr 172.21.15.44:0/3625480993 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.679+0000 7f76b3fff700 1 --2- 172.21.15.44:0/3625480993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42cc0 0x55bebac59c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.679+0000 7f76b8c52700 1 -- 172.21.15.44:0/3625480993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42cc0 msgr2=0x55bebac59c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.679+0000 7f76b8c52700 1 --2- 172.21.15.44:0/3625480993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42cc0 0x55bebac59c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.679+0000 7f76b8c52700 1 -- 172.21.15.44:0/3625480993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac4cda0 msgr2=0x55bebac603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.679+0000 7f76b8c52700 1 --2- 172.21.15.44:0/3625480993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac4cda0 0x55bebac603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.679+0000 7f76b8c52700 1 -- 172.21.15.44:0/3625480993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55beba9a9c00 con 0x55bebac42300 2023-09-10T01:01:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.679+0000 7f76b3fff700 1 --2- 172.21.15.44:0/3625480993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42cc0 0x55bebac59c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.679+0000 7f76b9453700 1 --2- 172.21.15.44:0/3625480993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac4cda0 0x55bebac603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.679+0000 7f76b8c52700 1 --2- 172.21.15.44:0/3625480993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42300 0x55bebac49df0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f76a8038dd0 tx=0x7f76a8005450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13d3866378ea1940 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.680+0000 7f769b7fe700 1 -- 172.21.15.44:0/3625480993 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a800b2e0 con 0x55bebac42300 2023-09-10T01:01:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.680+0000 7f769b7fe700 1 -- 172.21.15.44:0/3625480993 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76a802e440 con 0x55bebac42300 2023-09-10T01:01:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.680+0000 7f769b7fe700 1 -- 172.21.15.44:0/3625480993 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8038c20 con 0x55bebac42300 2023-09-10T01:01:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.680+0000 7f76c707cb80 1 -- 172.21.15.44:0/3625480993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42300 msgr2=0x55bebac49df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.680+0000 7f76c707cb80 1 --2- 172.21.15.44:0/3625480993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42300 0x55bebac49df0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f76a8038dd0 tx=0x7f76a8005450 comp rx=0 tx=0).stop 2023-09-10T01:01:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.680+0000 7f76c707cb80 1 -- 172.21.15.44:0/3625480993 shutdown_connections 2023-09-10T01:01:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.680+0000 7f76c707cb80 1 --2- 172.21.15.44:0/3625480993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac4cda0 0x55bebac603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.680+0000 7f76c707cb80 1 --2- 172.21.15.44:0/3625480993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac42300 0x55bebac49df0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.680+0000 7f76c707cb80 1 --2- 172.21.15.44:0/3625480993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42cc0 0x55bebac59c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.680+0000 7f76c707cb80 1 -- 172.21.15.44:0/3625480993 >> 172.21.15.44:0/3625480993 conn(0x55bebabb7030 msgr2=0x55bebac3d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.680+0000 7f76c707cb80 1 -- 172.21.15.44:0/3625480993 shutdown_connections 2023-09-10T01:01:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.680+0000 7f76c707cb80 1 -- 172.21.15.44:0/3625480993 wait complete. 2023-09-10T01:01:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.681+0000 7f76c707cb80 1 -- start start 2023-09-10T01:01:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.681+0000 7f76c707cb80 1 Processor -- start 2023-09-10T01:01:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.681+0000 7f76c707cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42300 0x55bebac49e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.681+0000 7f76c707cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac42cc0 0x55bebac57c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.681+0000 7f76c707cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac4cda0 0x55bebac58370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.681+0000 7f76c707cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebac58a40 con 0x55bebac42cc0 2023-09-10T01:01:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.681+0000 7f76c707cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebacc0330 con 0x55bebac42300 2023-09-10T01:01:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.681+0000 7f76c707cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebacc0470 con 0x55bebac4cda0 2023-09-10T01:01:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.681+0000 7f76c707cb80 10 client.0 _ll_register_callbacks cb 0x7f76a4003f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:01:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.682+0000 7f76b3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac42cc0 0x55bebac57c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.682+0000 7f76b3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac42cc0 0x55bebac57c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54846/0 (socket says 172.21.15.44:54846) 2023-09-10T01:01:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.682+0000 7f76b3fff700 1 -- 172.21.15.44:0/3303896624 learned_addr learned my addr 172.21.15.44:0/3303896624 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.682+0000 7f76b8c52700 1 --2- 172.21.15.44:0/3303896624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42300 0x55bebac49e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.682+0000 7f76b3fff700 1 -- 172.21.15.44:0/3303896624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac4cda0 msgr2=0x55bebac58370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:01:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.682+0000 7f76b3fff700 1 --2- 172.21.15.44:0/3303896624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac4cda0 0x55bebac58370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.682+0000 7f76b3fff700 1 -- 172.21.15.44:0/3303896624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42300 msgr2=0x55bebac49e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.682+0000 7f76b3fff700 1 --2- 172.21.15.44:0/3303896624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42300 0x55bebac49e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.682+0000 7f76b3fff700 1 -- 172.21.15.44:0/3303896624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ac00ecc0 con 0x55bebac42cc0 2023-09-10T01:01:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.682+0000 7f76b8c52700 1 --2- 172.21.15.44:0/3303896624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42300 0x55bebac49e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.682+0000 7f76b3fff700 1 --2- 172.21.15.44:0/3303896624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac42cc0 0x55bebac57c30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x55bebac4d9d0 tx=0x7f76ac009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d55d9e1698b3de server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.682+0000 7f7693fff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:01:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.682+0000 7f7693fff700 1 -- 172.21.15.44:0/3303896624 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac004db0 con 0x55bebac42cc0 2023-09-10T01:01:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.682+0000 7f7693fff700 1 -- 172.21.15.44:0/3303896624 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f76ac01c040 con 0x55bebac42cc0 2023-09-10T01:01:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.682+0000 7f76c707cb80 1 -- 172.21.15.44:0/3303896624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76a801d040 con 0x55bebac42cc0 2023-09-10T01:01:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.683+0000 7f7693fff700 1 -- 172.21.15.44:0/3303896624 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76ac003a00 con 0x55bebac42cc0 2023-09-10T01:01:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.683+0000 7f76c707cb80 10 client.18363 fetch_fsmap learned FSMap version 86 2023-09-10T01:01:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.683+0000 7f76c707cb80 1 -- 172.21.15.44:0/3303896624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7f76a40095f0 con 0x55bebac42cc0 2023-09-10T01:01:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.683+0000 7f7693fff700 1 -- 172.21.15.44:0/3303896624 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76a801d040 con 0x55bebac42cc0 2023-09-10T01:01:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.683+0000 7f7693fff700 1 -- 172.21.15.44:0/3303896624 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 86) v1 ==== 7319+0+0 (secure 0 0 0) 0x7f76ac017740 con 0x55bebac42cc0 2023-09-10T01:01:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.683+0000 7f76c707cb80 10 client.18363 fetch_fsmap finished waiting for FSMap version 86 2023-09-10T01:01:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.683+0000 7f76c707cb80 10 client.18363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:01:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.683+0000 7f76c707cb80 20 client.18363 populate_metadata read hostname 'smithi044' 2023-09-10T01:01:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.683+0000 7f76c707cb80 1 --2- 172.21.15.44:0/3303896624 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bebacc1bd0 0x55bebacc3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.683+0000 7f76c707cb80 4 client.18363 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:01:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.683+0000 7f76c707cb80 1 -- 172.21.15.44:0/3303896624 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bebab8d8c0 con 0x55bebacc1bd0 2023-09-10T01:01:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.683+0000 7f76b8c52700 1 --2- 172.21.15.44:0/3303896624 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bebacc1bd0 0x55bebacc3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.684+0000 7f7693fff700 10 client.18363 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.684+0000 7f76b8c52700 1 --2- 172.21.15.44:0/3303896624 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bebacc1bd0 0x55bebacc3ff0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8617b4dae4958de server_cookie=11381fefb2911d0b in_seq=0 out_seq=0 2023-09-10T01:01:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.696+0000 7f7693fff700 1 -- 172.21.15.44:0/3303896624 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bebab8d8c0 con 0x55bebacc1bd0 2023-09-10T01:01:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.696+0000 7f7693fff700 10 client.18363 handle_command_reply: tid=0 2023-09-10T01:01:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.696+0000 7f76c707cb80 1 client.18363 shutdown 2023-09-10T01:01:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76c707cb80 1 -- 172.21.15.44:0/3303896624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac42cc0 msgr2=0x55bebac57c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76c707cb80 1 --2- 172.21.15.44:0/3303896624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac42cc0 0x55bebac57c30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x55bebac4d9d0 tx=0x7f76ac009920 comp rx=0 tx=0).stop 2023-09-10T01:01:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76c707cb80 1 -- 172.21.15.44:0/3303896624 shutdown_connections 2023-09-10T01:01:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76c707cb80 1 --2- 172.21.15.44:0/3303896624 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bebacc1bd0 0x55bebacc3ff0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76c707cb80 1 --2- 172.21.15.44:0/3303896624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bebac4cda0 0x55bebac58370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76c707cb80 1 --2- 172.21.15.44:0/3303896624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bebac42300 0x55bebac49e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76c707cb80 1 --2- 172.21.15.44:0/3303896624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bebac42cc0 0x55bebac57c30 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76c707cb80 1 -- 172.21.15.44:0/3303896624 >> 172.21.15.44:0/3303896624 conn(0x55bebabb7030 msgr2=0x55bebac603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f7693fff700 0 client.18363 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76c707cb80 1 -- 172.21.15.44:0/3303896624 shutdown_connections 2023-09-10T01:01:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76c707cb80 1 -- 172.21.15.44:0/3303896624 wait complete. 2023-09-10T01:01:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76c707cb80 20 client.18363 trim_cache size 0 max 16384 2023-09-10T01:01:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:01:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:01:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:01:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:01:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76baeb6700 1 -- 172.21.15.44:0/1220054559 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f769c0a0830 msgr2=0x7f769c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76baeb6700 1 --2- 172.21.15.44:0/1220054559 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f769c0a0830 0x7f769c0a2cf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f76a8011040 tx=0x7f76a8019040 comp rx=0 tx=0).stop 2023-09-10T01:01:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76baeb6700 1 -- 172.21.15.44:0/1220054559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b41055b0 msgr2=0x7f76b419d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76baeb6700 1 --2- 172.21.15.44:0/1220054559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b41055b0 0x7f76b419d440 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f76a400ea80 tx=0x7f76a400ed90 comp rx=0 tx=0).stop 2023-09-10T01:01:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76baeb6700 1 -- 172.21.15.44:0/1220054559 shutdown_connections 2023-09-10T01:01:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76baeb6700 1 --2- 172.21.15.44:0/1220054559 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f769c0a0830 0x7f769c0a2cf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76baeb6700 1 --2- 172.21.15.44:0/1220054559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b41055b0 0x7f76b419d440 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76baeb6700 1 --2- 172.21.15.44:0/1220054559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b40ff690 0x7f76b4198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76baeb6700 1 --2- 172.21.15.44:0/1220054559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76b4100050 0x7f76b4199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.697+0000 7f76baeb6700 1 -- 172.21.15.44:0/1220054559 >> 172.21.15.44:0/1220054559 conn(0x7f76b4077430 msgr2=0x7f76b40fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.698+0000 7f76baeb6700 1 -- 172.21.15.44:0/1220054559 shutdown_connections 2023-09-10T01:01:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:16.698+0000 7f76baeb6700 1 -- 172.21.15.44:0/1220054559 wait complete. 2023-09-10T01:01:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:16 smithi099 ceph-mon[102692]: pgmap v653: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 235 op/s 2023-09-10T01:01:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:16 smithi131 ceph-mon[103713]: pgmap v653: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 235 op/s 2023-09-10T01:01:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:16 smithi044 ceph-mon[98676]: pgmap v653: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 235 op/s 2023-09-10T01:01:18.045 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:17 smithi131 podman[149174]: 2023-09-10T01:01:18.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:18.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:18.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-mon[103713]: pgmap v654: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 235 op/s 2023-09-10T01:01:18.331 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.331 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.331 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.331 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.331 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.332 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.332 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.332 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.332 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T01:01:18.332 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:01:18.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:18.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:18.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:18 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:18.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:18 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:18.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:18 smithi099 ceph-mon[102692]: pgmap v654: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 235 op/s 2023-09-10T01:01:18.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:18 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:18 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:18 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:18 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:18 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:18 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:18 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:18 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:18 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T01:01:18.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:18 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:01:18.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:18 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:18.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:18 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:18.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:18 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:18.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:18 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:18.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:18 smithi044 ceph-mon[98676]: pgmap v654: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 235 op/s 2023-09-10T01:01:18.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:18 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:18 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:18 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:18 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:18 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:18 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:18 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:18 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:18.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:18 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T01:01:18.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:18 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:01:18.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:18 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:18.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:18 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:18.795 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-mds[149299]: -- 172.21.15.131:0/2429601430 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5592390531e0 con 0x559239028c00 2023-09-10T01:01:18.796 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:18 smithi131 bash[149174]: ff5fe96739f288f3450d3b1112201e2b7d65cc052a01f7f7452bbd8240ed5385 2023-09-10T01:01:18.796 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:18 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: starting mds.c at 2023-09-10T01:01:18.796 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:01:18 smithi131 systemd[1]: Started Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:01:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:19 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:19 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:19 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:19 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:19 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:19 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:19 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:19 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:19 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T01:01:19.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:19 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T01:01:19.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:19 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:19 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:19 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:19 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:19 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:19 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:19 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:19 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:19 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T01:01:19.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:19 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T01:01:19.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:19 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:19 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:19 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:19 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:19 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:19 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:19 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:19 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:19.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:19 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T01:01:19.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:19 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T01:01:20.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.131:6836/1641988010,v1:172.21.15.131:6837/1641988010] up:boot 2023-09-10T01:01:20.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:01:20.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T01:01:20.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:20.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:20.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: pgmap v655: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 235 op/s 2023-09-10T01:01:20.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T01:01:20.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T01:01:20.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:01:20.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:01:20.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:20 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:20.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.131:6836/1641988010,v1:172.21.15.131:6837/1641988010] up:boot 2023-09-10T01:01:20.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:01:20.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T01:01:20.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:20.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:20.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: pgmap v655: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 235 op/s 2023-09-10T01:01:20.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T01:01:20.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T01:01:20.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:01:20.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:01:20.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:20 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:20.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.131:6836/1641988010,v1:172.21.15.131:6837/1641988010] up:boot 2023-09-10T01:01:20.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:01:20.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T01:01:20.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:20.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:20.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: pgmap v655: 129 pgs: 129 active+clean; 8.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 235 op/s 2023-09-10T01:01:20.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:01:20.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T01:01:20.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T01:01:20.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:01:20.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:01:20.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:20 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:01:23.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:22 smithi131 ceph-mon[103713]: pgmap v656: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2023-09-10T01:01:23.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:22 smithi044 ceph-mon[98676]: pgmap v656: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2023-09-10T01:01:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:22 smithi099 ceph-mon[102692]: pgmap v656: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2023-09-10T01:01:25.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:24 smithi131 ceph-mon[103713]: pgmap v657: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2023-09-10T01:01:25.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:24 smithi044 ceph-mon[98676]: pgmap v657: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2023-09-10T01:01:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:24 smithi099 ceph-mon[102692]: pgmap v657: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2023-09-10T01:01:27.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:26 smithi131 ceph-mon[103713]: pgmap v658: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 286 op/s 2023-09-10T01:01:27.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:26 smithi044 ceph-mon[98676]: pgmap v658: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 286 op/s 2023-09-10T01:01:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:26 smithi099 ceph-mon[102692]: pgmap v658: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 286 op/s 2023-09-10T01:01:29.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:28 smithi131 ceph-mon[103713]: pgmap v659: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2023-09-10T01:01:29.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:28 smithi044 ceph-mon[98676]: pgmap v659: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2023-09-10T01:01:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:28 smithi099 ceph-mon[102692]: pgmap v659: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2023-09-10T01:01:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.843+0000 7f2df4e89700 1 -- 172.21.15.44:0/3679254233 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df00ffe70 msgr2=0x7f2df010c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.843+0000 7f2df4e89700 1 --2- 172.21.15.44:0/3679254233 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df00ffe70 0x7f2df010c310 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009f90 tx=0x7f2de800b330 comp rx=0 tx=0).stop 2023-09-10T01:01:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.843+0000 7f2df4e89700 1 -- 172.21.15.44:0/3679254233 shutdown_connections 2023-09-10T01:01:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.843+0000 7f2df4e89700 1 --2- 172.21.15.44:0/3679254233 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df00ffe70 0x7f2df010c310 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.843+0000 7f2df4e89700 1 --2- 172.21.15.44:0/3679254233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df0104420 0x7f2df0104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.843+0000 7f2df4e89700 1 --2- 172.21.15.44:0/3679254233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2df00ff4b0 0x7f2df00ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.843+0000 7f2df4e89700 1 -- 172.21.15.44:0/3679254233 >> 172.21.15.44:0/3679254233 conn(0x7f2df0076600 msgr2=0x7f2df0076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.843+0000 7f2df4e89700 1 -- 172.21.15.44:0/3679254233 shutdown_connections 2023-09-10T01:01:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.843+0000 7f2df4e89700 1 -- 172.21.15.44:0/3679254233 wait complete. 2023-09-10T01:01:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.844+0000 7f2df4e89700 1 Processor -- start 2023-09-10T01:01:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.844+0000 7f2df4e89700 1 -- start start 2023-09-10T01:01:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2df4e89700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2df00ff4b0 0x7f2df0071d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2df4e89700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df00ffe70 0x7f2df0072290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2df4e89700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df0104420 0x7f2df006ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2df4e89700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df006b510 con 0x7f2df00ffe70 2023-09-10T01:01:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2df4e89700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df006b680 con 0x7f2df00ff4b0 2023-09-10T01:01:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2df4e89700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df006b840 con 0x7f2df0104420 2023-09-10T01:01:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2deed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df0104420 0x7f2df006ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2dedd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df00ffe70 0x7f2df0072290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2deed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df0104420 0x7f2df006ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33510/0 (socket says 172.21.15.44:33510) 2023-09-10T01:01:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2deed9d700 1 -- 172.21.15.44:0/1877260634 learned_addr learned my addr 172.21.15.44:0/1877260634 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2dee59c700 1 --2- 172.21.15.44:0/1877260634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2df00ff4b0 0x7f2df0071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2deed9d700 1 -- 172.21.15.44:0/1877260634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2df00ff4b0 msgr2=0x7f2df0071d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2deed9d700 1 --2- 172.21.15.44:0/1877260634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2df00ff4b0 0x7f2df0071d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2deed9d700 1 -- 172.21.15.44:0/1877260634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df00ffe70 msgr2=0x7f2df0072290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2deed9d700 1 --2- 172.21.15.44:0/1877260634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df00ffe70 0x7f2df0072290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2deed9d700 1 -- 172.21.15.44:0/1877260634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de8009c40 con 0x7f2df0104420 2023-09-10T01:01:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2dedd9b700 1 --2- 172.21.15.44:0/1877260634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df00ffe70 0x7f2df0072290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.845+0000 7f2deed9d700 1 --2- 172.21.15.44:0/1877260634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df0104420 0x7f2df006ad40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009f60 tx=0x7f2de8008d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.846+0000 7f2ddf7fe700 1 -- 172.21.15.44:0/1877260634 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de800e040 con 0x7f2df0104420 2023-09-10T01:01:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.846+0000 7f2ddf7fe700 1 -- 172.21.15.44:0/1877260634 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2de800be00 con 0x7f2df0104420 2023-09-10T01:01:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.846+0000 7f2df4e89700 1 -- 172.21.15.44:0/1877260634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df006f8a0 con 0x7f2df0104420 2023-09-10T01:01:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.846+0000 7f2ddf7fe700 1 -- 172.21.15.44:0/1877260634 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de80052a0 con 0x7f2df0104420 2023-09-10T01:01:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.846+0000 7f2df4e89700 1 -- 172.21.15.44:0/1877260634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df006fd80 con 0x7f2df0104420 2023-09-10T01:01:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.848+0000 7f2ddf7fe700 1 -- 172.21.15.44:0/1877260634 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f2de801d470 con 0x7f2df0104420 2023-09-10T01:01:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.849+0000 7f2ddf7fe700 1 --2- 172.21.15.44:0/1877260634 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2dd80a0830 0x7f2dd80a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.849+0000 7f2dee59c700 1 --2- 172.21.15.44:0/1877260634 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2dd80a0830 0x7f2dd80a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.849+0000 7f2ddf7fe700 1 -- 172.21.15.44:0/1877260634 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2de80cd7c0 con 0x7f2df0104420 2023-09-10T01:01:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.850+0000 7f2dee59c700 1 --2- 172.21.15.44:0/1877260634 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2dd80a0830 0x7f2dd80a2cf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2de0005fd0 tx=0x7f2de0009310 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.850+0000 7f2e0104fb80 1 Processor -- start 2023-09-10T01:01:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.851+0000 7f2e0104fb80 1 -- start start 2023-09-10T01:01:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.851+0000 7f2e0104fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f91a0 0x55cb9c4f9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.851+0000 7f2e0104fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f3300 0x55cb9c4f3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.851+0000 7f2e0104fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c4fff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.851+0000 7f2e0104fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9c44c6d0 con 0x55cb9c4f91a0 2023-09-10T01:01:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.851+0000 7f2e0104fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9c449ff0 con 0x55cb9c4f3300 2023-09-10T01:01:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.851+0000 7f2e0104fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9c4458e0 con 0x55cb9c4f3cc0 2023-09-10T01:01:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.851+0000 7f2dee59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f91a0 0x55cb9c4f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.851+0000 7f2dee59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f91a0 0x55cb9c4f9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54864/0 (socket says 172.21.15.44:54864) 2023-09-10T01:01:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.851+0000 7f2dee59c700 1 -- 172.21.15.44:0/386100018 learned_addr learned my addr 172.21.15.44:0/386100018 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.851+0000 7f2deed9d700 1 --2- 172.21.15.44:0/386100018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c4fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.851+0000 7f2dee59c700 1 -- 172.21.15.44:0/386100018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f3cc0 msgr2=0x55cb9c4fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.851+0000 7f2dee59c700 1 --2- 172.21.15.44:0/386100018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c4fff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.851+0000 7f2dee59c700 1 -- 172.21.15.44:0/386100018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f3300 msgr2=0x55cb9c4f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.851+0000 7f2dee59c700 1 --2- 172.21.15.44:0/386100018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f3300 0x55cb9c4f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.851+0000 7f2dee59c700 1 -- 172.21.15.44:0/386100018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb9c271d80 con 0x55cb9c4f91a0 2023-09-10T01:01:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.852+0000 7f2dee59c700 1 --2- 172.21.15.44:0/386100018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f91a0 0x55cb9c4f9580 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f2de0011d80 tx=0x7f2de0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff01bfab9bccc9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.852+0000 7f2ddcff9700 1 -- 172.21.15.44:0/386100018 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0004a30 con 0x55cb9c4f91a0 2023-09-10T01:01:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.852+0000 7f2ddcff9700 1 -- 172.21.15.44:0/386100018 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2de00054b0 con 0x55cb9c4f91a0 2023-09-10T01:01:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.852+0000 7f2ddcff9700 1 -- 172.21.15.44:0/386100018 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0003a00 con 0x55cb9c4f91a0 2023-09-10T01:01:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.853+0000 7f2e0104fb80 1 -- 172.21.15.44:0/386100018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f91a0 msgr2=0x55cb9c4f9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.853+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/386100018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f91a0 0x55cb9c4f9580 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f2de0011d80 tx=0x7f2de0014000 comp rx=0 tx=0).stop 2023-09-10T01:01:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.853+0000 7f2e0104fb80 1 -- 172.21.15.44:0/386100018 shutdown_connections 2023-09-10T01:01:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.853+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/386100018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c4fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.853+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/386100018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f91a0 0x55cb9c4f9580 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.853+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/386100018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f3300 0x55cb9c4f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.853+0000 7f2e0104fb80 1 -- 172.21.15.44:0/386100018 >> 172.21.15.44:0/386100018 conn(0x55cb9c468030 msgr2=0x55cb9c4ee560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.853+0000 7f2e0104fb80 1 -- 172.21.15.44:0/386100018 shutdown_connections 2023-09-10T01:01:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.853+0000 7f2e0104fb80 1 -- 172.21.15.44:0/386100018 wait complete. 2023-09-10T01:01:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.853+0000 7f2e0104fb80 1 -- start start 2023-09-10T01:01:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.853+0000 7f2e0104fb80 1 Processor -- start 2023-09-10T01:01:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.854+0000 7f2e0104fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f3300 0x55cb9c5064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.854+0000 7f2e0104fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c503700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.854+0000 7f2e0104fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f91a0 0x55cb9c503e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.854+0000 7f2e0104fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9c3e3b10 con 0x55cb9c4f3cc0 2023-09-10T01:01:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.854+0000 7f2e0104fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9c3e3810 con 0x55cb9c4f3300 2023-09-10T01:01:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.854+0000 7f2e0104fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9c2c78c0 con 0x55cb9c4f91a0 2023-09-10T01:01:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.854+0000 7f2e0104fb80 10 client.0 _ll_register_callbacks cb 0x55cb9c32da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:01:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.854+0000 7f2dee59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f3300 0x55cb9c5064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.854+0000 7f2dee59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f3300 0x55cb9c5064f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56610/0 (socket says 172.21.15.44:56610) 2023-09-10T01:01:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.854+0000 7f2dee59c700 1 -- 172.21.15.44:0/1346421648 learned_addr learned my addr 172.21.15.44:0/1346421648 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.854+0000 7f2deed9d700 1 --2- 172.21.15.44:0/1346421648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f91a0 0x55cb9c503e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.854+0000 7f2dee59c700 1 -- 172.21.15.44:0/1346421648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f91a0 msgr2=0x55cb9c503e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.854+0000 7f2dee59c700 1 --2- 172.21.15.44:0/1346421648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f91a0 0x55cb9c503e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.854+0000 7f2dee59c700 1 -- 172.21.15.44:0/1346421648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f3cc0 msgr2=0x55cb9c503700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:01:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.854+0000 7f2dee59c700 1 --2- 172.21.15.44:0/1346421648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c503700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.854+0000 7f2dee59c700 1 -- 172.21.15.44:0/1346421648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de0017070 con 0x55cb9c4f3300 2023-09-10T01:01:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.855+0000 7f2dee59c700 1 --2- 172.21.15.44:0/1346421648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f3300 0x55cb9c5064f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2de00092e0 tx=0x7f2de0009a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=384a7ec06465e22f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.855+0000 7f2dd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:01:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.855+0000 7f2dd5ffb700 1 -- 172.21.15.44:0/1346421648 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de002c070 con 0x55cb9c4f3300 2023-09-10T01:01:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.855+0000 7f2dd5ffb700 1 -- 172.21.15.44:0/1346421648 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2de0004b20 con 0x55cb9c4f3300 2023-09-10T01:01:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.855+0000 7f2e0104fb80 1 -- 172.21.15.44:0/1346421648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb9c39d490 con 0x55cb9c4f3300 2023-09-10T01:01:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.855+0000 7f2dd5ffb700 1 -- 172.21.15.44:0/1346421648 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2de0003680 con 0x55cb9c4f3300 2023-09-10T01:01:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.855+0000 7f2e0104fb80 10 client.28034 fetch_fsmap learned FSMap version 87 2023-09-10T01:01:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.855+0000 7f2e0104fb80 1 -- 172.21.15.44:0/1346421648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x55cb9c25ac00 con 0x55cb9c4f3300 2023-09-10T01:01:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.855+0000 7f2dd5ffb700 1 -- 172.21.15.44:0/1346421648 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cb9c39d490 con 0x55cb9c4f3300 2023-09-10T01:01:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.855+0000 7f2dd5ffb700 1 -- 172.21.15.44:0/1346421648 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 87) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f2de0024030 con 0x55cb9c4f3300 2023-09-10T01:01:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.855+0000 7f2e0104fb80 10 client.28034 fetch_fsmap finished waiting for FSMap version 87 2023-09-10T01:01:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.855+0000 7f2e0104fb80 10 client.28034 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:01:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.855+0000 7f2e0104fb80 20 client.28034 populate_metadata read hostname 'smithi044' 2023-09-10T01:01:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.855+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/1346421648 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb9c515b00 0x55cb9c517ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.855+0000 7f2e0104fb80 4 client.28034 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:01:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.855+0000 7f2e0104fb80 1 -- 172.21.15.44:0/1346421648 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb9c45a840 con 0x55cb9c515b00 2023-09-10T01:01:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.856+0000 7f2dedd9b700 1 --2- 172.21.15.44:0/1346421648 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb9c515b00 0x55cb9c517ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.856+0000 7f2dd5ffb700 10 client.28034 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.856+0000 7f2dedd9b700 1 --2- 172.21.15.44:0/1346421648 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb9c515b00 0x55cb9c517ef0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb0d28c9addd48d server_cookie=af9b1fdd24280e33 in_seq=0 out_seq=0 2023-09-10T01:01:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.857+0000 7f2dd5ffb700 1 -- 172.21.15.44:0/1346421648 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cb9c45a840 con 0x55cb9c515b00 2023-09-10T01:01:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.857+0000 7f2dd5ffb700 10 client.28034 handle_command_reply: tid=0 2023-09-10T01:01:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.857+0000 7f2e0104fb80 1 client.28034 shutdown 2023-09-10T01:01:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.857+0000 7f2e0104fb80 1 -- 172.21.15.44:0/1346421648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f3300 msgr2=0x55cb9c5064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.857+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/1346421648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f3300 0x55cb9c5064f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2de00092e0 tx=0x7f2de0009a50 comp rx=0 tx=0).stop 2023-09-10T01:01:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.857+0000 7f2e0104fb80 1 -- 172.21.15.44:0/1346421648 shutdown_connections 2023-09-10T01:01:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.857+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/1346421648 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb9c515b00 0x55cb9c517ef0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.857+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/1346421648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f91a0 0x55cb9c503e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.857+0000 7f2dd5ffb700 0 client.28034 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.857+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/1346421648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f3300 0x55cb9c5064f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.857+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/1346421648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c503700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.857+0000 7f2e0104fb80 1 -- 172.21.15.44:0/1346421648 >> 172.21.15.44:0/1346421648 conn(0x55cb9c468030 msgr2=0x55cb9c4ed7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.857+0000 7f2e0104fb80 1 -- 172.21.15.44:0/1346421648 shutdown_connections 2023-09-10T01:01:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.857+0000 7f2e0104fb80 1 -- 172.21.15.44:0/1346421648 wait complete. 2023-09-10T01:01:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.857+0000 7f2e0104fb80 20 client.28034 trim_cache size 0 max 16384 2023-09-10T01:01:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.875+0000 7f2e0104fb80 1 Processor -- start 2023-09-10T01:01:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.875+0000 7f2e0104fb80 1 -- start start 2023-09-10T01:01:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.875+0000 7f2e0104fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f3300 0x55cb9c506020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.875+0000 7f2e0104fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c4ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.875+0000 7f2e0104fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f91a0 0x55cb9c4ee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.875+0000 7f2e0104fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9c4ef380 con 0x55cb9c4f3300 2023-09-10T01:01:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.875+0000 7f2e0104fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9c501c60 con 0x55cb9c4f91a0 2023-09-10T01:01:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.875+0000 7f2e0104fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9c4f7bc0 con 0x55cb9c4f3cc0 2023-09-10T01:01:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.875+0000 7f2deed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f91a0 0x55cb9c4ee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.875+0000 7f2dee59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f3300 0x55cb9c506020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.875+0000 7f2dedd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c4ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.875+0000 7f2deed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f91a0 0x55cb9c4ee5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56618/0 (socket says 172.21.15.44:56618) 2023-09-10T01:01:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.875+0000 7f2deed9d700 1 -- 172.21.15.44:0/3465556894 learned_addr learned my addr 172.21.15.44:0/3465556894 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.875+0000 7f2deed9d700 1 -- 172.21.15.44:0/3465556894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f3cc0 msgr2=0x55cb9c4ee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.876+0000 7f2deed9d700 1 --2- 172.21.15.44:0/3465556894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c4ee090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.876+0000 7f2deed9d700 1 -- 172.21.15.44:0/3465556894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f3300 msgr2=0x55cb9c506020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.876+0000 7f2deed9d700 1 --2- 172.21.15.44:0/3465556894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f3300 0x55cb9c506020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.876+0000 7f2deed9d700 1 -- 172.21.15.44:0/3465556894 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb9c2981f0 con 0x55cb9c4f91a0 2023-09-10T01:01:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.876+0000 7f2dedd9b700 1 --2- 172.21.15.44:0/3465556894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c4ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.876+0000 7f2deed9d700 1 --2- 172.21.15.44:0/3465556894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f91a0 0x55cb9c4ee5d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2de8096da0 tx=0x7f2de800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8251c37082a7613d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.876+0000 7f2ddd7fa700 1 -- 172.21.15.44:0/3465556894 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de80af020 con 0x55cb9c4f91a0 2023-09-10T01:01:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.876+0000 7f2ddd7fa700 1 -- 172.21.15.44:0/3465556894 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2de809a200 con 0x55cb9c4f91a0 2023-09-10T01:01:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.876+0000 7f2ddd7fa700 1 -- 172.21.15.44:0/3465556894 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de809e6d0 con 0x55cb9c4f91a0 2023-09-10T01:01:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.877+0000 7f2e0104fb80 1 -- 172.21.15.44:0/3465556894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f91a0 msgr2=0x55cb9c4ee5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.877+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/3465556894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f91a0 0x55cb9c4ee5d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2de8096da0 tx=0x7f2de800b330 comp rx=0 tx=0).stop 2023-09-10T01:01:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.877+0000 7f2e0104fb80 1 -- 172.21.15.44:0/3465556894 shutdown_connections 2023-09-10T01:01:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.877+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/3465556894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f91a0 0x55cb9c4ee5d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.877+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/3465556894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f3300 0x55cb9c506020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.877+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/3465556894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c4ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.877+0000 7f2e0104fb80 1 -- 172.21.15.44:0/3465556894 >> 172.21.15.44:0/3465556894 conn(0x55cb9c468030 msgr2=0x55cb9c4f7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.877+0000 7f2e0104fb80 1 -- 172.21.15.44:0/3465556894 shutdown_connections 2023-09-10T01:01:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.877+0000 7f2e0104fb80 1 -- 172.21.15.44:0/3465556894 wait complete. 2023-09-10T01:01:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.877+0000 7f2e0104fb80 1 -- start start 2023-09-10T01:01:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.878+0000 7f2e0104fb80 1 Processor -- start 2023-09-10T01:01:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.878+0000 7f2e0104fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f3300 0x55cb9c4fe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.878+0000 7f2e0104fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c4fc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.878+0000 7f2e0104fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f91a0 0x55cb9c4fcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.878+0000 7f2e0104fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9c4fd290 con 0x55cb9c4f3cc0 2023-09-10T01:01:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.878+0000 7f2e0104fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9c4fd400 con 0x55cb9c4f91a0 2023-09-10T01:01:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.878+0000 7f2e0104fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9c4fd5c0 con 0x55cb9c4f3300 2023-09-10T01:01:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.878+0000 7f2e0104fb80 10 client.0 _ll_register_callbacks cb 0x7f2de801d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:01:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.878+0000 7f2dedd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c4fc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.878+0000 7f2dedd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c4fc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54882/0 (socket says 172.21.15.44:54882) 2023-09-10T01:01:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.878+0000 7f2dedd9b700 1 -- 172.21.15.44:0/401260250 learned_addr learned my addr 172.21.15.44:0/401260250 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.878+0000 7f2deed9d700 1 --2- 172.21.15.44:0/401260250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f91a0 0x55cb9c4fcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.878+0000 7f2dee59c700 1 --2- 172.21.15.44:0/401260250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f3300 0x55cb9c4fe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.879+0000 7f2dedd9b700 1 -- 172.21.15.44:0/401260250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f3300 msgr2=0x55cb9c4fe4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.879+0000 7f2dedd9b700 1 --2- 172.21.15.44:0/401260250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f3300 0x55cb9c4fe4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.879+0000 7f2dedd9b700 1 -- 172.21.15.44:0/401260250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f91a0 msgr2=0x55cb9c4fcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.879+0000 7f2dedd9b700 1 --2- 172.21.15.44:0/401260250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f91a0 0x55cb9c4fcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.879+0000 7f2dedd9b700 1 -- 172.21.15.44:0/401260250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de4007be0 con 0x55cb9c4f3cc0 2023-09-10T01:01:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.879+0000 7f2dedd9b700 1 --2- 172.21.15.44:0/401260250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c4fc810 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x55cb9c518820 tx=0x7f2de4019b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989a7867a8466e24 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.879+0000 7f2dd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:01:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.879+0000 7f2dd5ffb700 1 -- 172.21.15.44:0/401260250 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4004440 con 0x55cb9c4f3cc0 2023-09-10T01:01:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.879+0000 7f2dd5ffb700 1 -- 172.21.15.44:0/401260250 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2de4016050 con 0x55cb9c4f3cc0 2023-09-10T01:01:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.879+0000 7f2e0104fb80 1 -- 172.21.15.44:0/401260250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2de00054b0 con 0x55cb9c4f3cc0 2023-09-10T01:01:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.879+0000 7f2dd5ffb700 1 -- 172.21.15.44:0/401260250 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2de40045a0 con 0x55cb9c4f3cc0 2023-09-10T01:01:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.879+0000 7f2e0104fb80 10 client.18384 fetch_fsmap learned FSMap version 87 2023-09-10T01:01:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.879+0000 7f2e0104fb80 1 -- 172.21.15.44:0/401260250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x7f2df006fd80 con 0x55cb9c4f3cc0 2023-09-10T01:01:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.880+0000 7f2dd5ffb700 1 -- 172.21.15.44:0/401260250 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2de00054b0 con 0x55cb9c4f3cc0 2023-09-10T01:01:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.880+0000 7f2dd5ffb700 1 -- 172.21.15.44:0/401260250 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 87) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f2de40117b0 con 0x55cb9c4f3cc0 2023-09-10T01:01:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.880+0000 7f2e0104fb80 10 client.18384 fetch_fsmap finished waiting for FSMap version 87 2023-09-10T01:01:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.880+0000 7f2e0104fb80 10 client.18384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:01:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.880+0000 7f2e0104fb80 20 client.18384 populate_metadata read hostname 'smithi044' 2023-09-10T01:01:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.880+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/401260250 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb9c56af10 0x55cb9c56d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.880+0000 7f2e0104fb80 4 client.18384 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:01:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.880+0000 7f2e0104fb80 1 -- 172.21.15.44:0/401260250 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cb9c43e8c0 con 0x55cb9c56af10 2023-09-10T01:01:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.880+0000 7f2dee59c700 1 --2- 172.21.15.44:0/401260250 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb9c56af10 0x55cb9c56d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.880+0000 7f2dd5ffb700 10 client.18384 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.880+0000 7f2dee59c700 1 --2- 172.21.15.44:0/401260250 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb9c56af10 0x55cb9c56d330 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42acd850b8a903c2 server_cookie=270868c4960cc9f in_seq=0 out_seq=0 2023-09-10T01:01:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.891+0000 7f2dd5ffb700 1 -- 172.21.15.44:0/401260250 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cb9c43e8c0 con 0x55cb9c56af10 2023-09-10T01:01:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.891+0000 7f2dd5ffb700 10 client.18384 handle_command_reply: tid=0 2023-09-10T01:01:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.891+0000 7f2e0104fb80 1 client.18384 shutdown 2023-09-10T01:01:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.891+0000 7f2e0104fb80 1 -- 172.21.15.44:0/401260250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f3cc0 msgr2=0x55cb9c4fc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.891+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/401260250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c4fc810 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x55cb9c518820 tx=0x7f2de4019b60 comp rx=0 tx=0).stop 2023-09-10T01:01:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.891+0000 7f2e0104fb80 1 -- 172.21.15.44:0/401260250 shutdown_connections 2023-09-10T01:01:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.891+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/401260250 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb9c56af10 0x55cb9c56d330 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.891+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/401260250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb9c4f91a0 0x55cb9c4fcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.891+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/401260250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb9c4f3300 0x55cb9c4fe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.891+0000 7f2dd5ffb700 0 client.18384 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.891+0000 7f2e0104fb80 1 --2- 172.21.15.44:0/401260250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb9c4f3cc0 0x55cb9c4fc810 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.892+0000 7f2e0104fb80 1 -- 172.21.15.44:0/401260250 >> 172.21.15.44:0/401260250 conn(0x55cb9c468030 msgr2=0x55cb9c4f7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.892+0000 7f2e0104fb80 1 -- 172.21.15.44:0/401260250 shutdown_connections 2023-09-10T01:01:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.892+0000 7f2e0104fb80 1 -- 172.21.15.44:0/401260250 wait complete. 2023-09-10T01:01:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.892+0000 7f2e0104fb80 20 client.18384 trim_cache size 0 max 16384 2023-09-10T01:01:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:01:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:01:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:01:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:01:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.892+0000 7f2df4e89700 1 -- 172.21.15.44:0/1877260634 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2dd80a0830 msgr2=0x7f2dd80a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.892+0000 7f2df4e89700 1 --2- 172.21.15.44:0/1877260634 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2dd80a0830 0x7f2dd80a2cf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2de0005fd0 tx=0x7f2de0009310 comp rx=0 tx=0).stop 2023-09-10T01:01:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.892+0000 7f2df4e89700 1 -- 172.21.15.44:0/1877260634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df0104420 msgr2=0x7f2df006ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.892+0000 7f2df4e89700 1 --2- 172.21.15.44:0/1877260634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df0104420 0x7f2df006ad40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009f60 tx=0x7f2de8008d20 comp rx=0 tx=0).stop 2023-09-10T01:01:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.892+0000 7f2df4e89700 1 -- 172.21.15.44:0/1877260634 shutdown_connections 2023-09-10T01:01:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.892+0000 7f2df4e89700 1 --2- 172.21.15.44:0/1877260634 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2dd80a0830 0x7f2dd80a2cf0 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2de0005fd0 tx=0x7f2de0009310 comp rx=0 tx=0).stop 2023-09-10T01:01:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.892+0000 7f2df4e89700 1 --2- 172.21.15.44:0/1877260634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df0104420 0x7f2df006ad40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.893+0000 7f2df4e89700 1 --2- 172.21.15.44:0/1877260634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2df00ff4b0 0x7f2df0071d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.893+0000 7f2df4e89700 1 --2- 172.21.15.44:0/1877260634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df00ffe70 0x7f2df0072290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.893+0000 7f2df4e89700 1 -- 172.21.15.44:0/1877260634 >> 172.21.15.44:0/1877260634 conn(0x7f2df0076600 msgr2=0x7f2df00fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.896+0000 7f2df4e89700 1 -- 172.21.15.44:0/1877260634 shutdown_connections 2023-09-10T01:01:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:29.896+0000 7f2df4e89700 1 -- 172.21.15.44:0/1877260634 wait complete. 2023-09-10T01:01:31.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:30 smithi131 ceph-mon[103713]: pgmap v660: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2023-09-10T01:01:31.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:30 smithi044 ceph-mon[98676]: pgmap v660: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2023-09-10T01:01:31.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:30 smithi099 ceph-mon[102692]: pgmap v660: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2023-09-10T01:01:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:32 smithi131 ceph-mon[103713]: pgmap v661: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 277 op/s 2023-09-10T01:01:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:32 smithi044 ceph-mon[98676]: pgmap v661: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 277 op/s 2023-09-10T01:01:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:32 smithi099 ceph-mon[102692]: pgmap v661: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 277 op/s 2023-09-10T01:01:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:34 smithi131 ceph-mon[103713]: pgmap v662: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2023-09-10T01:01:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:34 smithi044 ceph-mon[98676]: pgmap v662: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2023-09-10T01:01:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:34 smithi099 ceph-mon[102692]: pgmap v662: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2023-09-10T01:01:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:36 smithi044 ceph-mon[98676]: pgmap v663: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2023-09-10T01:01:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:36 smithi099 ceph-mon[102692]: pgmap v663: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2023-09-10T01:01:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:36 smithi131 ceph-mon[103713]: pgmap v663: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2023-09-10T01:01:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:38 smithi044 ceph-mon[98676]: pgmap v664: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:01:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:38 smithi099 ceph-mon[102692]: pgmap v664: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:01:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:38 smithi131 ceph-mon[103713]: pgmap v664: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:01:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:40 smithi044 ceph-mon[98676]: pgmap v665: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:01:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:40 smithi099 ceph-mon[102692]: pgmap v665: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:01:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:40 smithi131 ceph-mon[103713]: pgmap v665: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:01:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.028+0000 7fa68a203700 1 -- 172.21.15.44:0/3303229836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa684103180 msgr2=0x7fa68410f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.028+0000 7fa68a203700 1 --2- 172.21.15.44:0/3303229836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa684103180 0x7fa68410f690 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00b0a0 tx=0x7fa67c00b3b0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.029+0000 7fa68a203700 1 -- 172.21.15.44:0/3303229836 shutdown_connections 2023-09-10T01:01:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.029+0000 7fa68a203700 1 --2- 172.21.15.44:0/3303229836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa684103180 0x7fa68410f690 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.029+0000 7fa68a203700 1 --2- 172.21.15.44:0/3303229836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6841087d0 0x7fa684108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.029+0000 7fa68a203700 1 --2- 172.21.15.44:0/3303229836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6841027c0 0x7fa684102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.029+0000 7fa68a203700 1 -- 172.21.15.44:0/3303229836 >> 172.21.15.44:0/3303229836 conn(0x7fa6840fe290 msgr2=0x7fa6841006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.029+0000 7fa68a203700 1 -- 172.21.15.44:0/3303229836 shutdown_connections 2023-09-10T01:01:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.029+0000 7fa68a203700 1 -- 172.21.15.44:0/3303229836 wait complete. 2023-09-10T01:01:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.029+0000 7fa68a203700 1 Processor -- start 2023-09-10T01:01:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa68a203700 1 -- start start 2023-09-10T01:01:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa68a203700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6841027c0 0x7fa684074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa68a203700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa684103180 0x7fa684075050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa68a203700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6841087d0 0x7fa6841a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa68a203700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6840758b0 con 0x7fa684103180 2023-09-10T01:01:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa68a203700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa684198440 con 0x7fa6841087d0 2023-09-10T01:01:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa68a203700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa684198600 con 0x7fa6841027c0 2023-09-10T01:01:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa6837fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6841027c0 0x7fa684074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa682ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa684103180 0x7fa684075050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa683fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6841087d0 0x7fa6841a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa682ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa684103180 0x7fa684075050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54898/0 (socket says 172.21.15.44:54898) 2023-09-10T01:01:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa6837fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6841027c0 0x7fa684074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33546/0 (socket says 172.21.15.44:33546) 2023-09-10T01:01:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa682ffd700 1 -- 172.21.15.44:0/864862945 learned_addr learned my addr 172.21.15.44:0/864862945 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa682ffd700 1 -- 172.21.15.44:0/864862945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6841027c0 msgr2=0x7fa684074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa682ffd700 1 --2- 172.21.15.44:0/864862945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6841027c0 0x7fa684074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa682ffd700 1 -- 172.21.15.44:0/864862945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6841087d0 msgr2=0x7fa6841a8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa682ffd700 1 --2- 172.21.15.44:0/864862945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6841087d0 0x7fa6841a8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa682ffd700 1 -- 172.21.15.44:0/864862945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c009d00 con 0x7fa684103180 2023-09-10T01:01:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa683fff700 1 --2- 172.21.15.44:0/864862945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6841087d0 0x7fa6841a8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.030+0000 7fa6837fe700 1 --2- 172.21.15.44:0/864862945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6841027c0 0x7fa684074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.031+0000 7fa682ffd700 1 --2- 172.21.15.44:0/864862945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa684103180 0x7fa684075050 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa67800e9a0 tx=0x7fa67800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.031+0000 7fa680ff9700 1 -- 172.21.15.44:0/864862945 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678009790 con 0x7fa684103180 2023-09-10T01:01:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.031+0000 7fa68a203700 1 -- 172.21.15.44:0/864862945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6841988e0 con 0x7fa684103180 2023-09-10T01:01:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.031+0000 7fa680ff9700 1 -- 172.21.15.44:0/864862945 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa678010430 con 0x7fa684103180 2023-09-10T01:01:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.031+0000 7fa680ff9700 1 -- 172.21.15.44:0/864862945 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67800cea0 con 0x7fa684103180 2023-09-10T01:01:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.031+0000 7fa68a203700 1 -- 172.21.15.44:0/864862945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa684198df0 con 0x7fa684103180 2023-09-10T01:01:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.034+0000 7fa680ff9700 1 -- 172.21.15.44:0/864862945 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa6780105a0 con 0x7fa684103180 2023-09-10T01:01:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.035+0000 7fa680ff9700 1 --2- 172.21.15.44:0/864862945 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa66c0a07b0 0x7fa66c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.035+0000 7fa6963c9b80 1 Processor -- start 2023-09-10T01:01:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.035+0000 7fa680ff9700 1 -- 172.21.15.44:0/864862945 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa678014070 con 0x7fa684103180 2023-09-10T01:01:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.035+0000 7fa6963c9b80 1 -- start start 2023-09-10T01:01:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.035+0000 7fa6963c9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a8e1a0 0x5560d0a8e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.035+0000 7fa6963c9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88300 0x5560d0a88780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.035+0000 7fa6963c9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a88cc0 0x5560d0a94f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.035+0000 7fa6963c9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d09e16d0 con 0x5560d0a88300 2023-09-10T01:01:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.035+0000 7fa6963c9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d09deff0 con 0x5560d0a8e1a0 2023-09-10T01:01:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.035+0000 7fa6963c9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d09da8e0 con 0x5560d0a88cc0 2023-09-10T01:01:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.036+0000 7fa682ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88300 0x5560d0a88780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.036+0000 7fa683fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a88cc0 0x5560d0a94f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.036+0000 7fa683fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a88cc0 0x5560d0a94f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33556/0 (socket says 172.21.15.44:33556) 2023-09-10T01:01:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.036+0000 7fa682ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88300 0x5560d0a88780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54908/0 (socket says 172.21.15.44:54908) 2023-09-10T01:01:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.036+0000 7fa683fff700 1 -- 172.21.15.44:0/2121400058 learned_addr learned my addr 172.21.15.44:0/2121400058 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.036+0000 7fa6837fe700 1 --2- 172.21.15.44:0/2121400058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a8e1a0 0x5560d0a8e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.036+0000 7fa6837fe700 1 --2- 172.21.15.44:0/864862945 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa66c0a07b0 0x7fa66c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.036+0000 7fa6837fe700 1 -- 172.21.15.44:0/2121400058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a88cc0 msgr2=0x5560d0a94f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.036+0000 7fa6837fe700 1 --2- 172.21.15.44:0/2121400058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a88cc0 0x5560d0a94f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.036+0000 7fa6837fe700 1 -- 172.21.15.44:0/2121400058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88300 msgr2=0x5560d0a88780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.036+0000 7fa6837fe700 1 --2- 172.21.15.44:0/2121400058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88300 0x5560d0a88780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.036+0000 7fa6837fe700 1 -- 172.21.15.44:0/2121400058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560d0806d80 con 0x5560d0a8e1a0 2023-09-10T01:01:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.036+0000 7fa683fff700 1 --2- 172.21.15.44:0/2121400058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a88cc0 0x5560d0a94f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.036+0000 7fa6837fe700 1 --2- 172.21.15.44:0/2121400058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a8e1a0 0x5560d0a8e580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa674005950 tx=0x7fa674013e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=293396794eea1789 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.036+0000 7fa6837fe700 1 --2- 172.21.15.44:0/864862945 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa66c0a07b0 0x7fa66c0a2c70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa67401c000 tx=0x7fa67401e000 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.036+0000 7fa669ffb700 1 -- 172.21.15.44:0/2121400058 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa674004050 con 0x5560d0a8e1a0 2023-09-10T01:01:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.037+0000 7fa669ffb700 1 -- 172.21.15.44:0/2121400058 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa674003680 con 0x5560d0a8e1a0 2023-09-10T01:01:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.037+0000 7fa669ffb700 1 -- 172.21.15.44:0/2121400058 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67400f8a0 con 0x5560d0a8e1a0 2023-09-10T01:01:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.037+0000 7fa6963c9b80 1 -- 172.21.15.44:0/2121400058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a8e1a0 msgr2=0x5560d0a8e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.037+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/2121400058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a8e1a0 0x5560d0a8e580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa674005950 tx=0x7fa674013e60 comp rx=0 tx=0).stop 2023-09-10T01:01:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.037+0000 7fa6963c9b80 1 -- 172.21.15.44:0/2121400058 shutdown_connections 2023-09-10T01:01:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.038+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/2121400058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a88cc0 0x5560d0a94f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.038+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/2121400058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a8e1a0 0x5560d0a8e580 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.038+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/2121400058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88300 0x5560d0a88780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.038+0000 7fa6963c9b80 1 -- 172.21.15.44:0/2121400058 >> 172.21.15.44:0/2121400058 conn(0x5560d09fd030 msgr2=0x5560d0a83560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.038+0000 7fa6963c9b80 1 -- 172.21.15.44:0/2121400058 shutdown_connections 2023-09-10T01:01:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.038+0000 7fa6963c9b80 1 -- 172.21.15.44:0/2121400058 wait complete. 2023-09-10T01:01:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.038+0000 7fa6963c9b80 1 -- start start 2023-09-10T01:01:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.038+0000 7fa6963c9b80 1 Processor -- start 2023-09-10T01:01:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.038+0000 7fa6963c9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a94e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa6963c9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88cc0 0x5560d0a92080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa6963c9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a8e1a0 0x5560d0a92690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa6963c9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d0978b10 con 0x5560d0a88cc0 2023-09-10T01:01:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa6963c9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d0978810 con 0x5560d0a88300 2023-09-10T01:01:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa6963c9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d085c8c0 con 0x5560d0a8e1a0 2023-09-10T01:01:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa6963c9b80 10 client.0 _ll_register_callbacks cb 0x5560d08c2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:01:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa682ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88cc0 0x5560d0a92080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa6837fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a94e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa682ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88cc0 0x5560d0a92080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54912/0 (socket says 172.21.15.44:54912) 2023-09-10T01:01:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa6837fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a94e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56650/0 (socket says 172.21.15.44:56650) 2023-09-10T01:01:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa682ffd700 1 -- 172.21.15.44:0/3905073467 learned_addr learned my addr 172.21.15.44:0/3905073467 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa683fff700 1 --2- 172.21.15.44:0/3905073467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a8e1a0 0x5560d0a92690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa6837fe700 1 -- 172.21.15.44:0/3905073467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a8e1a0 msgr2=0x5560d0a92690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa6837fe700 1 --2- 172.21.15.44:0/3905073467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a8e1a0 0x5560d0a92690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa6837fe700 1 -- 172.21.15.44:0/3905073467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88cc0 msgr2=0x5560d0a92080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa6837fe700 1 --2- 172.21.15.44:0/3905073467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88cc0 0x5560d0a92080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa6837fe700 1 -- 172.21.15.44:0/3905073467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68410ff90 con 0x5560d0a88300 2023-09-10T01:01:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa683fff700 1 --2- 172.21.15.44:0/3905073467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a8e1a0 0x5560d0a92690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa6837fe700 1 --2- 172.21.15.44:0/3905073467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a94e00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa674005950 tx=0x7fa6740052e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4652f10fc555ebb6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa662ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:01:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa662ffd700 1 -- 172.21.15.44:0/3905073467 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa674005ce0 con 0x5560d0a88300 2023-09-10T01:01:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa6963c9b80 1 -- 172.21.15.44:0/3905073467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560d0932490 con 0x5560d0a88300 2023-09-10T01:01:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.039+0000 7fa662ffd700 1 -- 172.21.15.44:0/3905073467 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa67400f8a0 con 0x5560d0a88300 2023-09-10T01:01:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.040+0000 7fa662ffd700 1 -- 172.21.15.44:0/3905073467 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa674003de0 con 0x5560d0a88300 2023-09-10T01:01:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.040+0000 7fa6963c9b80 10 client.28052 fetch_fsmap learned FSMap version 87 2023-09-10T01:01:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.040+0000 7fa6963c9b80 1 -- 172.21.15.44:0/3905073467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x7fa674013760 con 0x5560d0a88300 2023-09-10T01:01:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.040+0000 7fa662ffd700 1 -- 172.21.15.44:0/3905073467 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5560d0932490 con 0x5560d0a88300 2023-09-10T01:01:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.040+0000 7fa662ffd700 1 -- 172.21.15.44:0/3905073467 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 87) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa674025070 con 0x5560d0a88300 2023-09-10T01:01:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.040+0000 7fa6963c9b80 10 client.28052 fetch_fsmap finished waiting for FSMap version 87 2023-09-10T01:01:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.040+0000 7fa6963c9b80 10 client.28052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:01:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.040+0000 7fa6963c9b80 20 client.28052 populate_metadata read hostname 'smithi044' 2023-09-10T01:01:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.040+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/3905073467 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5560d0aa2320 0x5560d0aa4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.040+0000 7fa6963c9b80 4 client.28052 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:01:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.040+0000 7fa6963c9b80 1 -- 172.21.15.44:0/3905073467 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560d09ef840 con 0x5560d0aa2320 2023-09-10T01:01:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.040+0000 7fa682ffd700 1 --2- 172.21.15.44:0/3905073467 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5560d0aa2320 0x5560d0aa4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.041+0000 7fa662ffd700 10 client.28052 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.041+0000 7fa682ffd700 1 --2- 172.21.15.44:0/3905073467 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5560d0aa2320 0x5560d0aa4740 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7691b61cd6513d40 server_cookie=c0fd82d1833d208 in_seq=0 out_seq=0 2023-09-10T01:01:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.041+0000 7fa662ffd700 1 -- 172.21.15.44:0/3905073467 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5560d09ef840 con 0x5560d0aa2320 2023-09-10T01:01:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.041+0000 7fa662ffd700 10 client.28052 handle_command_reply: tid=0 2023-09-10T01:01:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.041+0000 7fa6963c9b80 1 client.28052 shutdown 2023-09-10T01:01:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.041+0000 7fa6963c9b80 1 -- 172.21.15.44:0/3905073467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 msgr2=0x5560d0a94e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.041+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/3905073467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a94e00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa674005950 tx=0x7fa6740052e0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.041+0000 7fa6963c9b80 1 -- 172.21.15.44:0/3905073467 shutdown_connections 2023-09-10T01:01:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.041+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/3905073467 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5560d0aa2320 0x5560d0aa4740 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.042+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/3905073467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a8e1a0 0x5560d0a92690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.042+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/3905073467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a94e00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.042+0000 7fa662ffd700 0 client.28052 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:43.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.042+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/3905073467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88cc0 0x5560d0a92080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.042+0000 7fa6963c9b80 1 -- 172.21.15.44:0/3905073467 >> 172.21.15.44:0/3905073467 conn(0x5560d09fd030 msgr2=0x5560d0a827a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.042+0000 7fa6963c9b80 1 -- 172.21.15.44:0/3905073467 shutdown_connections 2023-09-10T01:01:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.042+0000 7fa6963c9b80 1 -- 172.21.15.44:0/3905073467 wait complete. 2023-09-10T01:01:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.042+0000 7fa6963c9b80 20 client.28052 trim_cache size 0 max 16384 2023-09-10T01:01:43.078 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:42 smithi044 ceph-mon[98676]: pgmap v666: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 273 op/s 2023-09-10T01:01:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.058+0000 7fa6963c9b80 1 Processor -- start 2023-09-10T01:01:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.058+0000 7fa6963c9b80 1 -- start start 2023-09-10T01:01:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.058+0000 7fa6963c9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a994a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.058+0000 7fa6963c9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a88cc0 0x5560d0a87a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.058+0000 7fa6963c9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a8e1a0 0x5560d0a82f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.058+0000 7fa6963c9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d0a7ffc0 con 0x5560d0a8e1a0 2023-09-10T01:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.058+0000 7fa6963c9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d0a890a0 con 0x5560d0a88300 2023-09-10T01:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.058+0000 7fa6963c9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d0a93660 con 0x5560d0a88cc0 2023-09-10T01:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa682ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a88cc0 0x5560d0a87a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa6837fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa683fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a8e1a0 0x5560d0a82f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa682ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a88cc0 0x5560d0a87a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33570/0 (socket says 172.21.15.44:33570) 2023-09-10T01:01:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa682ffd700 1 -- 172.21.15.44:0/296621823 learned_addr learned my addr 172.21.15.44:0/296621823 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa6837fe700 1 --2- 172.21.15.44:0/296621823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56658/0 (socket says 172.21.15.44:56658) 2023-09-10T01:01:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa682ffd700 1 -- 172.21.15.44:0/296621823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 msgr2=0x5560d0a994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa682ffd700 1 --2- 172.21.15.44:0/296621823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa682ffd700 1 -- 172.21.15.44:0/296621823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a8e1a0 msgr2=0x5560d0a82f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa682ffd700 1 --2- 172.21.15.44:0/296621823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a8e1a0 0x5560d0a82f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa682ffd700 1 -- 172.21.15.44:0/296621823 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560d07efc00 con 0x5560d0a88cc0 2023-09-10T01:01:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa683fff700 1 --2- 172.21.15.44:0/296621823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a8e1a0 0x5560d0a82f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa6837fe700 1 --2- 172.21.15.44:0/296621823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa682ffd700 1 --2- 172.21.15.44:0/296621823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a88cc0 0x5560d0a87a00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa678095af0 tx=0x7fa6780cb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47a054eebfcfe882 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa66a7fc700 1 -- 172.21.15.44:0/296621823 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678099cf0 con 0x5560d0a88cc0 2023-09-10T01:01:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa66a7fc700 1 -- 172.21.15.44:0/296621823 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa6780995f0 con 0x5560d0a88cc0 2023-09-10T01:01:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.059+0000 7fa66a7fc700 1 -- 172.21.15.44:0/296621823 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6780840b0 con 0x5560d0a88cc0 2023-09-10T01:01:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.060+0000 7fa6963c9b80 1 -- 172.21.15.44:0/296621823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a88cc0 msgr2=0x5560d0a87a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.060+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/296621823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a88cc0 0x5560d0a87a00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa678095af0 tx=0x7fa6780cb040 comp rx=0 tx=0).stop 2023-09-10T01:01:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.060+0000 7fa6963c9b80 1 -- 172.21.15.44:0/296621823 shutdown_connections 2023-09-10T01:01:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.060+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/296621823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a8e1a0 0x5560d0a82f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.060+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/296621823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.060+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/296621823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a88cc0 0x5560d0a87a00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.060+0000 7fa6963c9b80 1 -- 172.21.15.44:0/296621823 >> 172.21.15.44:0/296621823 conn(0x5560d09fd030 msgr2=0x5560d0a872f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.060+0000 7fa6963c9b80 1 -- 172.21.15.44:0/296621823 shutdown_connections 2023-09-10T01:01:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.060+0000 7fa6963c9b80 1 -- 172.21.15.44:0/296621823 wait complete. 2023-09-10T01:01:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.061+0000 7fa6963c9b80 1 -- start start 2023-09-10T01:01:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.061+0000 7fa6963c9b80 1 Processor -- start 2023-09-10T01:01:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.061+0000 7fa6963c9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a990e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.061+0000 7fa6963c9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88cc0 0x5560d0a99620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.061+0000 7fa6963c9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a8e1a0 0x5560d0a97660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.061+0000 7fa6963c9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d0a97d70 con 0x5560d0a88cc0 2023-09-10T01:01:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.061+0000 7fa6963c9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d0a97ee0 con 0x5560d0a88300 2023-09-10T01:01:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.061+0000 7fa6963c9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d0a980a0 con 0x5560d0a8e1a0 2023-09-10T01:01:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.061+0000 7fa6963c9b80 10 client.0 _ll_register_callbacks cb 0x7fa678095270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:01:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.062+0000 7fa683fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a8e1a0 0x5560d0a97660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.062+0000 7fa683fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a8e1a0 0x5560d0a97660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33576/0 (socket says 172.21.15.44:33576) 2023-09-10T01:01:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.062+0000 7fa6837fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.062+0000 7fa683fff700 1 -- 172.21.15.44:0/821171542 learned_addr learned my addr 172.21.15.44:0/821171542 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.062+0000 7fa682ffd700 1 --2- 172.21.15.44:0/821171542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88cc0 0x5560d0a99620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.062+0000 7fa683fff700 1 -- 172.21.15.44:0/821171542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 msgr2=0x5560d0a990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.062+0000 7fa683fff700 1 --2- 172.21.15.44:0/821171542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.062+0000 7fa683fff700 1 -- 172.21.15.44:0/821171542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88cc0 msgr2=0x5560d0a99620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.062+0000 7fa683fff700 1 --2- 172.21.15.44:0/821171542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88cc0 0x5560d0a99620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.062+0000 7fa683fff700 1 -- 172.21.15.44:0/821171542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa674013760 con 0x5560d0a8e1a0 2023-09-10T01:01:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.062+0000 7fa6837fe700 1 --2- 172.21.15.44:0/821171542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.062+0000 7fa683fff700 1 --2- 172.21.15.44:0/821171542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a8e1a0 0x5560d0a97660 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa67c003c60 tx=0x7fa67c003c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2f066251dc6309d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.062+0000 7fa662ffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:01:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.062+0000 7fa662ffd700 1 -- 172.21.15.44:0/821171542 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c023070 con 0x5560d0a8e1a0 2023-09-10T01:01:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.062+0000 7fa6963c9b80 1 -- 172.21.15.44:0/821171542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa674003680 con 0x5560d0a8e1a0 2023-09-10T01:01:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.062+0000 7fa662ffd700 1 -- 172.21.15.44:0/821171542 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa67c005250 con 0x5560d0a8e1a0 2023-09-10T01:01:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.063+0000 7fa662ffd700 1 -- 172.21.15.44:0/821171542 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa67c00e040 con 0x5560d0a8e1a0 2023-09-10T01:01:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.063+0000 7fa6963c9b80 10 client.27997 fetch_fsmap learned FSMap version 87 2023-09-10T01:01:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.063+0000 7fa6963c9b80 1 -- 172.21.15.44:0/821171542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x7fa67c00bdc0 con 0x5560d0a8e1a0 2023-09-10T01:01:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.063+0000 7fa662ffd700 1 -- 172.21.15.44:0/821171542 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa674003680 con 0x5560d0a8e1a0 2023-09-10T01:01:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.063+0000 7fa662ffd700 1 -- 172.21.15.44:0/821171542 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 87) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa67c00b0d0 con 0x5560d0a8e1a0 2023-09-10T01:01:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.063+0000 7fa6963c9b80 10 client.27997 fetch_fsmap finished waiting for FSMap version 87 2023-09-10T01:01:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.063+0000 7fa6963c9b80 10 client.27997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:01:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.063+0000 7fa6963c9b80 20 client.27997 populate_metadata read hostname 'smithi044' 2023-09-10T01:01:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.063+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/821171542 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5560d0af74a0 0x5560d0af9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.063+0000 7fa6963c9b80 4 client.27997 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:01:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.063+0000 7fa6963c9b80 1 -- 172.21.15.44:0/821171542 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5560d09d38c0 con 0x5560d0af74a0 2023-09-10T01:01:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.063+0000 7fa6837fe700 1 --2- 172.21.15.44:0/821171542 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5560d0af74a0 0x5560d0af9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.064+0000 7fa662ffd700 10 client.27997 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.064+0000 7fa6837fe700 1 --2- 172.21.15.44:0/821171542 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5560d0af74a0 0x5560d0af9890 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86592374f2eb742c server_cookie=5081a4499ca761 in_seq=0 out_seq=0 2023-09-10T01:01:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.073+0000 7fa662ffd700 1 -- 172.21.15.44:0/821171542 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5560d09d38c0 con 0x5560d0af74a0 2023-09-10T01:01:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.073+0000 7fa662ffd700 10 client.27997 handle_command_reply: tid=0 2023-09-10T01:01:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.073+0000 7fa6963c9b80 1 client.27997 shutdown 2023-09-10T01:01:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.073+0000 7fa6963c9b80 1 -- 172.21.15.44:0/821171542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a8e1a0 msgr2=0x5560d0a97660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.073+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/821171542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a8e1a0 0x5560d0a97660 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa67c003c60 tx=0x7fa67c003c90 comp rx=0 tx=0).stop 2023-09-10T01:01:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.073+0000 7fa6963c9b80 1 -- 172.21.15.44:0/821171542 shutdown_connections 2023-09-10T01:01:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.073+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/821171542 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5560d0af74a0 0x5560d0af9890 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.073+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/821171542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560d0a8e1a0 0x5560d0a97660 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.073+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/821171542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560d0a88300 0x5560d0a990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.073+0000 7fa662ffd700 0 client.27997 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.073+0000 7fa6963c9b80 1 --2- 172.21.15.44:0/821171542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5560d0a88cc0 0x5560d0a99620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.073+0000 7fa6963c9b80 1 -- 172.21.15.44:0/821171542 >> 172.21.15.44:0/821171542 conn(0x5560d09fd030 msgr2=0x5560d0a87b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.073+0000 7fa6963c9b80 1 -- 172.21.15.44:0/821171542 shutdown_connections 2023-09-10T01:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.073+0000 7fa6963c9b80 1 -- 172.21.15.44:0/821171542 wait complete. 2023-09-10T01:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.073+0000 7fa6963c9b80 20 client.27997 trim_cache size 0 max 16384 2023-09-10T01:01:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:01:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.074+0000 7fa68a203700 1 -- 172.21.15.44:0/864862945 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa66c0a07b0 msgr2=0x7fa66c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.074+0000 7fa68a203700 1 --2- 172.21.15.44:0/864862945 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa66c0a07b0 0x7fa66c0a2c70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa67401c000 tx=0x7fa67401e000 comp rx=0 tx=0).stop 2023-09-10T01:01:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.074+0000 7fa68a203700 1 -- 172.21.15.44:0/864862945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa684103180 msgr2=0x7fa684075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.074+0000 7fa68a203700 1 --2- 172.21.15.44:0/864862945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa684103180 0x7fa684075050 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa67800e9a0 tx=0x7fa67800ecb0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.074+0000 7fa68a203700 1 -- 172.21.15.44:0/864862945 shutdown_connections 2023-09-10T01:01:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.074+0000 7fa68a203700 1 --2- 172.21.15.44:0/864862945 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa66c0a07b0 0x7fa66c0a2c70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.074+0000 7fa68a203700 1 --2- 172.21.15.44:0/864862945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6841087d0 0x7fa6841a8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.074+0000 7fa68a203700 1 --2- 172.21.15.44:0/864862945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6841027c0 0x7fa684074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.074+0000 7fa68a203700 1 --2- 172.21.15.44:0/864862945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa684103180 0x7fa684075050 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.074+0000 7fa68a203700 1 -- 172.21.15.44:0/864862945 >> 172.21.15.44:0/864862945 conn(0x7fa6840fe290 msgr2=0x7fa6840fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.074+0000 7fa68a203700 1 -- 172.21.15.44:0/864862945 shutdown_connections 2023-09-10T01:01:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:43.074+0000 7fa68a203700 1 -- 172.21.15.44:0/864862945 wait complete. 2023-09-10T01:01:43.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:42 smithi099 ceph-mon[102692]: pgmap v666: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 273 op/s 2023-09-10T01:01:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:42 smithi131 ceph-mon[103713]: pgmap v666: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 273 op/s 2023-09-10T01:01:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:44 smithi099 ceph-mon[102692]: pgmap v667: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:01:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:44 smithi131 ceph-mon[103713]: pgmap v667: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:01:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:44 smithi044 ceph-mon[98676]: pgmap v667: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:01:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:46 smithi099 ceph-mon[102692]: pgmap v668: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 277 op/s 2023-09-10T01:01:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:46 smithi131 ceph-mon[103713]: pgmap v668: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 277 op/s 2023-09-10T01:01:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:46 smithi044 ceph-mon[98676]: pgmap v668: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 277 op/s 2023-09-10T01:01:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:48 smithi099 ceph-mon[102692]: pgmap v669: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2023-09-10T01:01:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:48 smithi131 ceph-mon[103713]: pgmap v669: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2023-09-10T01:01:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:48 smithi044 ceph-mon[98676]: pgmap v669: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2023-09-10T01:01:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:50 smithi099 ceph-mon[102692]: pgmap v670: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2023-09-10T01:01:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:50 smithi131 ceph-mon[103713]: pgmap v670: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2023-09-10T01:01:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:50 smithi044 ceph-mon[98676]: pgmap v670: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2023-09-10T01:01:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:52 smithi099 ceph-mon[102692]: pgmap v671: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 276 op/s 2023-09-10T01:01:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:52 smithi131 ceph-mon[103713]: pgmap v671: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 276 op/s 2023-09-10T01:01:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:52 smithi044 ceph-mon[98676]: pgmap v671: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 276 op/s 2023-09-10T01:01:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:54 smithi099 ceph-mon[102692]: pgmap v672: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 179 op/s 2023-09-10T01:01:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:54 smithi131 ceph-mon[103713]: pgmap v672: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 179 op/s 2023-09-10T01:01:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:54 smithi044 ceph-mon[98676]: pgmap v672: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 179 op/s 2023-09-10T01:01:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.210+0000 7f557298e700 1 -- 172.21.15.44:0/1724296503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f556c0679f0 msgr2=0x7f556c067e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.210+0000 7f557298e700 1 --2- 172.21.15.44:0/1724296503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f556c0679f0 0x7f556c067e70 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f55600099c0 tx=0x7f5560009cd0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.210+0000 7f557298e700 1 -- 172.21.15.44:0/1724296503 shutdown_connections 2023-09-10T01:01:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.210+0000 7f557298e700 1 --2- 172.21.15.44:0/1724296503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f556c0683b0 0x7f556c10d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.210+0000 7f557298e700 1 --2- 172.21.15.44:0/1724296503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f556c106650 0x7f556c106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.210+0000 7f557298e700 1 --2- 172.21.15.44:0/1724296503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f556c0679f0 0x7f556c067e70 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.210+0000 7f557298e700 1 -- 172.21.15.44:0/1724296503 >> 172.21.15.44:0/1724296503 conn(0x7f556c074f30 msgr2=0x7f556c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.210+0000 7f557298e700 1 -- 172.21.15.44:0/1724296503 shutdown_connections 2023-09-10T01:01:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.210+0000 7f557298e700 1 -- 172.21.15.44:0/1724296503 wait complete. 2023-09-10T01:01:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.210+0000 7f557298e700 1 Processor -- start 2023-09-10T01:01:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.210+0000 7f557298e700 1 -- start start 2023-09-10T01:01:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f557298e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f556c0679f0 0x7f556c196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f557298e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f556c0683b0 0x7f556c196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f557298e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f556c106650 0x7f556c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f557298e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c19a600 con 0x7f556c0683b0 2023-09-10T01:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f557298e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c19a770 con 0x7f556c0679f0 2023-09-10T01:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f557298e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c19a930 con 0x7f556c106650 2023-09-10T01:01:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f556b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f556c0683b0 0x7f556c196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f556b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f556c0683b0 0x7f556c196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54940/0 (socket says 172.21.15.44:54940) 2023-09-10T01:01:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f556bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f556c0679f0 0x7f556c196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f556b7fe700 1 -- 172.21.15.44:0/949763549 learned_addr learned my addr 172.21.15.44:0/949763549 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f5570f2b700 1 --2- 172.21.15.44:0/949763549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f556c106650 0x7f556c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f556b7fe700 1 -- 172.21.15.44:0/949763549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f556c106650 msgr2=0x7f556c19a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f556b7fe700 1 --2- 172.21.15.44:0/949763549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f556c106650 0x7f556c19a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f556b7fe700 1 -- 172.21.15.44:0/949763549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f556c0679f0 msgr2=0x7f556c196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f556b7fe700 1 --2- 172.21.15.44:0/949763549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f556c0679f0 0x7f556c196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f556b7fe700 1 -- 172.21.15.44:0/949763549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55600095f0 con 0x7f556c0683b0 2023-09-10T01:01:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f556bfff700 1 --2- 172.21.15.44:0/949763549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f556c0679f0 0x7f556c196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f556b7fe700 1 --2- 172.21.15.44:0/949763549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f556c0683b0 0x7f556c196a40 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f5560005910 tx=0x7f5560009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f55697fa700 1 -- 172.21.15.44:0/949763549 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556001d070 con 0x7f556c0683b0 2023-09-10T01:01:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f55697fa700 1 -- 172.21.15.44:0/949763549 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5560022470 con 0x7f556c0683b0 2023-09-10T01:01:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f557298e700 1 -- 172.21.15.44:0/949763549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556c19abb0 con 0x7f556c0683b0 2023-09-10T01:01:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f55697fa700 1 -- 172.21.15.44:0/949763549 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5560017650 con 0x7f556c0683b0 2023-09-10T01:01:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.211+0000 7f557298e700 1 -- 172.21.15.44:0/949763549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556c1a3570 con 0x7f556c0683b0 2023-09-10T01:01:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.214+0000 7f55697fa700 1 -- 172.21.15.44:0/949763549 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f55600225e0 con 0x7f556c0683b0 2023-09-10T01:01:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.214+0000 7f55697fa700 1 --2- 172.21.15.44:0/949763549 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55540a0830 0x7f55540a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.214+0000 7f55697fa700 1 -- 172.21.15.44:0/949763549 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f55600cd200 con 0x7f556c0683b0 2023-09-10T01:01:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.214+0000 7f556bfff700 1 --2- 172.21.15.44:0/949763549 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55540a0830 0x7f55540a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.215+0000 7f556bfff700 1 --2- 172.21.15.44:0/949763549 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55540a0830 0x7f55540a2cf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f555c011490 tx=0x7f555c009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.215+0000 7f557eb54b80 1 Processor -- start 2023-09-10T01:01:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.215+0000 7f557eb54b80 1 -- start start 2023-09-10T01:01:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.215+0000 7f557eb54b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e91a0 0x55a9eb7e9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.215+0000 7f557eb54b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e3300 0x55a9eb7e3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.215+0000 7f557eb54b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7eff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.215+0000 7f557eb54b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9eb73c6d0 con 0x55a9eb7e91a0 2023-09-10T01:01:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.215+0000 7f557eb54b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9eb739ff0 con 0x55a9eb7e3cc0 2023-09-10T01:01:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.215+0000 7f557eb54b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9eb7358e0 con 0x55a9eb7e3300 2023-09-10T01:01:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.215+0000 7f556bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e91a0 0x55a9eb7e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.216+0000 7f556bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e91a0 0x55a9eb7e9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54946/0 (socket says 172.21.15.44:54946) 2023-09-10T01:01:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.216+0000 7f556bfff700 1 -- 172.21.15.44:0/832822237 learned_addr learned my addr 172.21.15.44:0/832822237 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.216+0000 7f5570f2b700 1 --2- 172.21.15.44:0/832822237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7eff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.216+0000 7f556b7fe700 1 --2- 172.21.15.44:0/832822237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e3300 0x55a9eb7e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.216+0000 7f556bfff700 1 -- 172.21.15.44:0/832822237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e3300 msgr2=0x55a9eb7e3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.216+0000 7f556bfff700 1 --2- 172.21.15.44:0/832822237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e3300 0x55a9eb7e3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.216+0000 7f556bfff700 1 -- 172.21.15.44:0/832822237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e3cc0 msgr2=0x55a9eb7eff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.216+0000 7f556bfff700 1 --2- 172.21.15.44:0/832822237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7eff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.216+0000 7f556bfff700 1 -- 172.21.15.44:0/832822237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9eb561d80 con 0x55a9eb7e91a0 2023-09-10T01:01:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.216+0000 7f5570f2b700 1 --2- 172.21.15.44:0/832822237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7eff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:01:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.216+0000 7f556bfff700 1 --2- 172.21.15.44:0/832822237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e91a0 0x55a9eb7e9580 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f555c0043c0 tx=0x7f555c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869213f92674868e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.216+0000 7f55527fc700 1 -- 172.21.15.44:0/832822237 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c003d10 con 0x55a9eb7e91a0 2023-09-10T01:01:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.216+0000 7f55527fc700 1 -- 172.21.15.44:0/832822237 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f555c003e70 con 0x55a9eb7e91a0 2023-09-10T01:01:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.216+0000 7f55527fc700 1 -- 172.21.15.44:0/832822237 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c01e750 con 0x55a9eb7e91a0 2023-09-10T01:01:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.217+0000 7f557eb54b80 1 -- 172.21.15.44:0/832822237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e91a0 msgr2=0x55a9eb7e9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.217+0000 7f557eb54b80 1 --2- 172.21.15.44:0/832822237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e91a0 0x55a9eb7e9580 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f555c0043c0 tx=0x7f555c014000 comp rx=0 tx=0).stop 2023-09-10T01:01:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.217+0000 7f557eb54b80 1 -- 172.21.15.44:0/832822237 shutdown_connections 2023-09-10T01:01:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.217+0000 7f557eb54b80 1 --2- 172.21.15.44:0/832822237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7eff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.217+0000 7f557eb54b80 1 --2- 172.21.15.44:0/832822237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e91a0 0x55a9eb7e9580 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.217+0000 7f557eb54b80 1 --2- 172.21.15.44:0/832822237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e3300 0x55a9eb7e3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.217+0000 7f557eb54b80 1 -- 172.21.15.44:0/832822237 >> 172.21.15.44:0/832822237 conn(0x55a9eb758030 msgr2=0x55a9eb7de560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.217+0000 7f557eb54b80 1 -- 172.21.15.44:0/832822237 shutdown_connections 2023-09-10T01:01:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.217+0000 7f557eb54b80 1 -- 172.21.15.44:0/832822237 wait complete. 2023-09-10T01:01:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.217+0000 7f557eb54b80 1 -- start start 2023-09-10T01:01:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.217+0000 7f557eb54b80 1 Processor -- start 2023-09-10T01:01:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f557eb54b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e3300 0x55a9eb7efe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f557eb54b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7ed130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f557eb54b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e91a0 0x55a9eb7ed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f557eb54b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9eb6d3b10 con 0x55a9eb7e3cc0 2023-09-10T01:01:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f557eb54b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9eb6d3810 con 0x55a9eb7e91a0 2023-09-10T01:01:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f557eb54b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9eb5b78c0 con 0x55a9eb7e3300 2023-09-10T01:01:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f557eb54b80 10 client.0 _ll_register_callbacks cb 0x55a9eb61da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:01:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f556b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7ed130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f556b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7ed130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54954/0 (socket says 172.21.15.44:54954) 2023-09-10T01:01:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f556b7fe700 1 -- 172.21.15.44:0/2450491053 learned_addr learned my addr 172.21.15.44:0/2450491053 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f556bfff700 1 --2- 172.21.15.44:0/2450491053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e3300 0x55a9eb7efe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f5570f2b700 1 --2- 172.21.15.44:0/2450491053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e91a0 0x55a9eb7ed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f556b7fe700 1 -- 172.21.15.44:0/2450491053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e3300 msgr2=0x55a9eb7efe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f556b7fe700 1 --2- 172.21.15.44:0/2450491053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e3300 0x55a9eb7efe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f556b7fe700 1 -- 172.21.15.44:0/2450491053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e91a0 msgr2=0x55a9eb7ed760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f556b7fe700 1 --2- 172.21.15.44:0/2450491053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e91a0 0x55a9eb7ed760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f556b7fe700 1 -- 172.21.15.44:0/2450491053 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5564011230 con 0x55a9eb7e3cc0 2023-09-10T01:01:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f556b7fe700 1 --2- 172.21.15.44:0/2450491053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7ed130 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f556000b580 tx=0x7f5560099f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7334ca81511dfda server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f554b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:01:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f554b7fe700 1 -- 172.21.15.44:0/2450491053 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556009a7a0 con 0x55a9eb7e3cc0 2023-09-10T01:01:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f554b7fe700 1 -- 172.21.15.44:0/2450491053 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f556009aa40 con 0x55a9eb7e3cc0 2023-09-10T01:01:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f557eb54b80 1 -- 172.21.15.44:0/2450491053 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9eb68d490 con 0x55a9eb7e3cc0 2023-09-10T01:01:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f554b7fe700 1 -- 172.21.15.44:0/2450491053 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55600b1070 con 0x55a9eb7e3cc0 2023-09-10T01:01:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f557eb54b80 10 client.18414 fetch_fsmap learned FSMap version 87 2023-09-10T01:01:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.218+0000 7f557eb54b80 1 -- 172.21.15.44:0/2450491053 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x7f555c017070 con 0x55a9eb7e3cc0 2023-09-10T01:01:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.219+0000 7f554b7fe700 1 -- 172.21.15.44:0/2450491053 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a9eb68d490 con 0x55a9eb7e3cc0 2023-09-10T01:01:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.219+0000 7f554b7fe700 1 -- 172.21.15.44:0/2450491053 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 87) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f55600998c0 con 0x55a9eb7e3cc0 2023-09-10T01:01:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.219+0000 7f557eb54b80 10 client.18414 fetch_fsmap finished waiting for FSMap version 87 2023-09-10T01:01:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.219+0000 7f557eb54b80 10 client.18414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:01:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.219+0000 7f557eb54b80 20 client.18414 populate_metadata read hostname 'smithi044' 2023-09-10T01:01:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.219+0000 7f557eb54b80 1 --2- 172.21.15.44:0/2450491053 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a9eb7fcfd0 0x55a9eb7ff3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.219+0000 7f557eb54b80 4 client.18414 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:01:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.219+0000 7f557eb54b80 1 -- 172.21.15.44:0/2450491053 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9eb74a840 con 0x55a9eb7fcfd0 2023-09-10T01:01:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.219+0000 7f556bfff700 1 --2- 172.21.15.44:0/2450491053 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a9eb7fcfd0 0x55a9eb7ff3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.219+0000 7f554b7fe700 10 client.18414 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.219+0000 7f556bfff700 1 --2- 172.21.15.44:0/2450491053 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a9eb7fcfd0 0x55a9eb7ff3f0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa86cb77e5190bcd server_cookie=f82cbf2a22c5d27d in_seq=0 out_seq=0 2023-09-10T01:01:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.220+0000 7f554b7fe700 1 -- 172.21.15.44:0/2450491053 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a9eb74a840 con 0x55a9eb7fcfd0 2023-09-10T01:01:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.220+0000 7f554b7fe700 10 client.18414 handle_command_reply: tid=0 2023-09-10T01:01:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.220+0000 7f557eb54b80 1 client.18414 shutdown 2023-09-10T01:01:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.220+0000 7f557eb54b80 1 -- 172.21.15.44:0/2450491053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3cc0 msgr2=0x55a9eb7ed130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.220+0000 7f557eb54b80 1 --2- 172.21.15.44:0/2450491053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7ed130 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f556000b580 tx=0x7f5560099f00 comp rx=0 tx=0).stop 2023-09-10T01:01:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.220+0000 7f557eb54b80 1 -- 172.21.15.44:0/2450491053 shutdown_connections 2023-09-10T01:01:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.220+0000 7f557eb54b80 1 --2- 172.21.15.44:0/2450491053 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a9eb7fcfd0 0x55a9eb7ff3f0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.220+0000 7f557eb54b80 1 --2- 172.21.15.44:0/2450491053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e91a0 0x55a9eb7ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.220+0000 7f554b7fe700 0 client.18414 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.220+0000 7f557eb54b80 1 --2- 172.21.15.44:0/2450491053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e3300 0x55a9eb7efe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.220+0000 7f557eb54b80 1 --2- 172.21.15.44:0/2450491053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7ed130 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.220+0000 7f557eb54b80 1 -- 172.21.15.44:0/2450491053 >> 172.21.15.44:0/2450491053 conn(0x55a9eb758030 msgr2=0x55a9eb7dd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.220+0000 7f557eb54b80 1 -- 172.21.15.44:0/2450491053 shutdown_connections 2023-09-10T01:01:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.220+0000 7f557eb54b80 1 -- 172.21.15.44:0/2450491053 wait complete. 2023-09-10T01:01:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.220+0000 7f557eb54b80 20 client.18414 trim_cache size 0 max 16384 2023-09-10T01:01:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.235+0000 7f557eb54b80 1 Processor -- start 2023-09-10T01:01:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.235+0000 7f557eb54b80 1 -- start start 2023-09-10T01:01:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f557eb54b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3300 0x55a9eb7ddf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f557eb54b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7de460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f557eb54b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e91a0 0x55a9eb7ec6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f557eb54b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9eb74a840 con 0x55a9eb7e3300 2023-09-10T01:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f557eb54b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9eb68d490 con 0x55a9eb7e91a0 2023-09-10T01:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f557eb54b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9eb7e8c20 con 0x55a9eb7e3cc0 2023-09-10T01:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f556bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3300 0x55a9eb7ddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f556bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3300 0x55a9eb7ddf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54960/0 (socket says 172.21.15.44:54960) 2023-09-10T01:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f556bfff700 1 -- 172.21.15.44:0/2800386145 learned_addr learned my addr 172.21.15.44:0/2800386145 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f5570f2b700 1 --2- 172.21.15.44:0/2800386145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e91a0 0x55a9eb7ec6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f556b7fe700 1 --2- 172.21.15.44:0/2800386145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7de460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f556bfff700 1 -- 172.21.15.44:0/2800386145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e3cc0 msgr2=0x55a9eb7de460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f556bfff700 1 --2- 172.21.15.44:0/2800386145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7de460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f556bfff700 1 -- 172.21.15.44:0/2800386145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e91a0 msgr2=0x55a9eb7ec6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f556bfff700 1 --2- 172.21.15.44:0/2800386145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e91a0 0x55a9eb7ec6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f556bfff700 1 -- 172.21.15.44:0/2800386145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9eb54ac00 con 0x55a9eb7e3300 2023-09-10T01:01:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f556bfff700 1 --2- 172.21.15.44:0/2800386145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3300 0x55a9eb7ddf20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f555c000c00 tx=0x7f555c026f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93ba96ff2f60528 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f5552ffd700 1 -- 172.21.15.44:0/2800386145 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c02b070 con 0x55a9eb7e3300 2023-09-10T01:01:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f5552ffd700 1 -- 172.21.15.44:0/2800386145 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f555c02c040 con 0x55a9eb7e3300 2023-09-10T01:01:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.236+0000 7f5552ffd700 1 -- 172.21.15.44:0/2800386145 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c0055c0 con 0x55a9eb7e3300 2023-09-10T01:01:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.237+0000 7f557eb54b80 1 -- 172.21.15.44:0/2800386145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3300 msgr2=0x55a9eb7ddf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.237+0000 7f557eb54b80 1 --2- 172.21.15.44:0/2800386145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3300 0x55a9eb7ddf20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f555c000c00 tx=0x7f555c026f10 comp rx=0 tx=0).stop 2023-09-10T01:01:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.237+0000 7f557eb54b80 1 -- 172.21.15.44:0/2800386145 shutdown_connections 2023-09-10T01:01:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.237+0000 7f557eb54b80 1 --2- 172.21.15.44:0/2800386145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e91a0 0x55a9eb7ec6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.237+0000 7f557eb54b80 1 --2- 172.21.15.44:0/2800386145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3300 0x55a9eb7ddf20 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.237+0000 7f557eb54b80 1 --2- 172.21.15.44:0/2800386145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7de460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.237+0000 7f557eb54b80 1 -- 172.21.15.44:0/2800386145 >> 172.21.15.44:0/2800386145 conn(0x55a9eb758030 msgr2=0x55a9eb7e71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.237+0000 7f557eb54b80 1 -- 172.21.15.44:0/2800386145 shutdown_connections 2023-09-10T01:01:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.237+0000 7f557eb54b80 1 -- 172.21.15.44:0/2800386145 wait complete. 2023-09-10T01:01:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.237+0000 7f557eb54b80 1 -- start start 2023-09-10T01:01:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.237+0000 7f557eb54b80 1 Processor -- start 2023-09-10T01:01:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f557eb54b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e3300 0x55a9eb7f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f557eb54b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7f2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f557eb54b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e91a0 0x55a9eb849e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f557eb54b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9eb84a340 con 0x55a9eb7e3cc0 2023-09-10T01:01:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f557eb54b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9eb84a4b0 con 0x55a9eb7e3300 2023-09-10T01:01:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f557eb54b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9eb84a670 con 0x55a9eb7e91a0 2023-09-10T01:01:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f557eb54b80 10 client.0 _ll_register_callbacks cb 0x7f5560085050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:01:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f556b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f556bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e3300 0x55a9eb7f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f556bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e3300 0x55a9eb7f23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56706/0 (socket says 172.21.15.44:56706) 2023-09-10T01:01:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f556b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7f2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54968/0 (socket says 172.21.15.44:54968) 2023-09-10T01:01:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f556bfff700 1 -- 172.21.15.44:0/3832737613 learned_addr learned my addr 172.21.15.44:0/3832737613 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:01:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f5570f2b700 1 --2- 172.21.15.44:0/3832737613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e91a0 0x55a9eb849e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f556bfff700 1 -- 172.21.15.44:0/3832737613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e91a0 msgr2=0x55a9eb849e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f556bfff700 1 --2- 172.21.15.44:0/3832737613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e91a0 0x55a9eb849e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f556bfff700 1 -- 172.21.15.44:0/3832737613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3cc0 msgr2=0x55a9eb7f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:01:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f556bfff700 1 --2- 172.21.15.44:0/3832737613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7f2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f556bfff700 1 -- 172.21.15.44:0/3832737613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9eb561d80 con 0x55a9eb7e3300 2023-09-10T01:01:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f556bfff700 1 --2- 172.21.15.44:0/3832737613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e3300 0x55a9eb7f23f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f555c028010 tx=0x7f555c003d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2af47b91861cf13 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:01:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f554b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:01:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.238+0000 7f554b7fe700 1 -- 172.21.15.44:0/3832737613 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c02b040 con 0x55a9eb7e3300 2023-09-10T01:01:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.239+0000 7f557eb54b80 1 -- 172.21.15.44:0/3832737613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f555c003e70 con 0x55a9eb7e3300 2023-09-10T01:01:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.239+0000 7f554b7fe700 1 -- 172.21.15.44:0/3832737613 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f555c032050 con 0x55a9eb7e3300 2023-09-10T01:01:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.239+0000 7f554b7fe700 1 -- 172.21.15.44:0/3832737613 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f555c032e50 con 0x55a9eb7e3300 2023-09-10T01:01:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.239+0000 7f557eb54b80 10 client.28076 fetch_fsmap learned FSMap version 87 2023-09-10T01:01:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.239+0000 7f557eb54b80 1 -- 172.21.15.44:0/3832737613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x55a9eb5881f0 con 0x55a9eb7e3300 2023-09-10T01:01:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.239+0000 7f554b7fe700 1 -- 172.21.15.44:0/3832737613 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f555c003e70 con 0x55a9eb7e3300 2023-09-10T01:01:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.239+0000 7f554b7fe700 1 -- 172.21.15.44:0/3832737613 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 87) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f555c024020 con 0x55a9eb7e3300 2023-09-10T01:01:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.239+0000 7f557eb54b80 10 client.28076 fetch_fsmap finished waiting for FSMap version 87 2023-09-10T01:01:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.239+0000 7f557eb54b80 10 client.28076 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:01:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.239+0000 7f557eb54b80 20 client.28076 populate_metadata read hostname 'smithi044' 2023-09-10T01:01:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.239+0000 7f557eb54b80 1 --2- 172.21.15.44:0/3832737613 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a9eb8518e0 0x55a9eb853cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:01:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.239+0000 7f557eb54b80 4 client.28076 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:01:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.239+0000 7f557eb54b80 1 -- 172.21.15.44:0/3832737613 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a9eb72e8c0 con 0x55a9eb8518e0 2023-09-10T01:01:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.239+0000 7f556b7fe700 1 --2- 172.21.15.44:0/3832737613 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a9eb8518e0 0x55a9eb853cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:01:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.240+0000 7f554b7fe700 10 client.28076 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.240+0000 7f556b7fe700 1 --2- 172.21.15.44:0/3832737613 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a9eb8518e0 0x55a9eb853cd0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc3f2428f9c4e142 server_cookie=5fcc90dcdf445fbf in_seq=0 out_seq=0 2023-09-10T01:01:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.261+0000 7f554b7fe700 1 -- 172.21.15.44:0/3832737613 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a9eb72e8c0 con 0x55a9eb8518e0 2023-09-10T01:01:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.261+0000 7f554b7fe700 10 client.28076 handle_command_reply: tid=0 2023-09-10T01:01:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.261+0000 7f557eb54b80 1 client.28076 shutdown 2023-09-10T01:01:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.261+0000 7f557eb54b80 1 -- 172.21.15.44:0/3832737613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e3300 msgr2=0x55a9eb7f23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.261+0000 7f557eb54b80 1 --2- 172.21.15.44:0/3832737613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e3300 0x55a9eb7f23f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f555c028010 tx=0x7f555c003d50 comp rx=0 tx=0).stop 2023-09-10T01:01:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557eb54b80 1 -- 172.21.15.44:0/3832737613 shutdown_connections 2023-09-10T01:01:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557eb54b80 1 --2- 172.21.15.44:0/3832737613 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a9eb8518e0 0x55a9eb853cd0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557eb54b80 1 --2- 172.21.15.44:0/3832737613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9eb7e91a0 0x55a9eb849e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f554b7fe700 0 client.28076 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:01:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557eb54b80 1 --2- 172.21.15.44:0/3832737613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9eb7e3300 0x55a9eb7f23f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557eb54b80 1 --2- 172.21.15.44:0/3832737613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a9eb7e3cc0 0x55a9eb7f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557eb54b80 1 -- 172.21.15.44:0/3832737613 >> 172.21.15.44:0/3832737613 conn(0x55a9eb758030 msgr2=0x55a9eb7e22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557eb54b80 1 -- 172.21.15.44:0/3832737613 shutdown_connections 2023-09-10T01:01:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557eb54b80 1 -- 172.21.15.44:0/3832737613 wait complete. 2023-09-10T01:01:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557eb54b80 20 client.28076 trim_cache size 0 max 16384 2023-09-10T01:01:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:01:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:01:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:01:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:01:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557298e700 1 -- 172.21.15.44:0/949763549 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55540a0830 msgr2=0x7f55540a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557298e700 1 --2- 172.21.15.44:0/949763549 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55540a0830 0x7f55540a2cf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f555c011490 tx=0x7f555c009260 comp rx=0 tx=0).stop 2023-09-10T01:01:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557298e700 1 -- 172.21.15.44:0/949763549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f556c0683b0 msgr2=0x7f556c196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:01:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557298e700 1 --2- 172.21.15.44:0/949763549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f556c0683b0 0x7f556c196a40 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f5560005910 tx=0x7f5560009580 comp rx=0 tx=0).stop 2023-09-10T01:01:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557298e700 1 -- 172.21.15.44:0/949763549 shutdown_connections 2023-09-10T01:01:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557298e700 1 --2- 172.21.15.44:0/949763549 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55540a0830 0x7f55540a2cf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557298e700 1 --2- 172.21.15.44:0/949763549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f556c106650 0x7f556c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557298e700 1 --2- 172.21.15.44:0/949763549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f556c0679f0 0x7f556c196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557298e700 1 --2- 172.21.15.44:0/949763549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f556c0683b0 0x7f556c196a40 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:01:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557298e700 1 -- 172.21.15.44:0/949763549 >> 172.21.15.44:0/949763549 conn(0x7f556c074f30 msgr2=0x7f556c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:01:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557298e700 1 -- 172.21.15.44:0/949763549 shutdown_connections 2023-09-10T01:01:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:01:56.262+0000 7f557298e700 1 -- 172.21.15.44:0/949763549 wait complete. 2023-09-10T01:01:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:56 smithi099 ceph-mon[102692]: pgmap v673: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 261 op/s 2023-09-10T01:01:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:56 smithi131 ceph-mon[103713]: pgmap v673: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 261 op/s 2023-09-10T01:01:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:56 smithi044 ceph-mon[98676]: pgmap v673: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 261 op/s 2023-09-10T01:01:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:01:58 smithi099 ceph-mon[102692]: pgmap v674: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2023-09-10T01:01:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:01:58 smithi131 ceph-mon[103713]: pgmap v674: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2023-09-10T01:01:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:01:58 smithi044 ceph-mon[98676]: pgmap v674: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2023-09-10T01:02:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:00 smithi099 ceph-mon[102692]: pgmap v675: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2023-09-10T01:02:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:00 smithi131 ceph-mon[103713]: pgmap v675: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2023-09-10T01:02:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:00 smithi044 ceph-mon[98676]: pgmap v675: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2023-09-10T01:02:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:02 smithi099 ceph-mon[102692]: pgmap v676: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 235 op/s 2023-09-10T01:02:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:02 smithi131 ceph-mon[103713]: pgmap v676: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 235 op/s 2023-09-10T01:02:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:02 smithi044 ceph-mon[98676]: pgmap v676: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 235 op/s 2023-09-10T01:02:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:04 smithi099 ceph-mon[102692]: pgmap v677: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 157 op/s 2023-09-10T01:02:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:04 smithi131 ceph-mon[103713]: pgmap v677: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 157 op/s 2023-09-10T01:02:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:04 smithi044 ceph-mon[98676]: pgmap v677: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 157 op/s 2023-09-10T01:02:07.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:06 smithi099 ceph-mon[102692]: pgmap v678: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 286 op/s 2023-09-10T01:02:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:06 smithi131 ceph-mon[103713]: pgmap v678: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 286 op/s 2023-09-10T01:02:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:06 smithi044 ceph-mon[98676]: pgmap v678: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 286 op/s 2023-09-10T01:02:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:08 smithi131 ceph-mon[103713]: pgmap v679: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:02:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:08 smithi044 ceph-mon[98676]: pgmap v679: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:02:09.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:08 smithi099 ceph-mon[102692]: pgmap v679: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:02:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.405+0000 7fe916115700 1 -- 172.21.15.44:0/1086596355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe910106650 msgr2=0x7fe910106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.405+0000 7fe916115700 1 --2- 172.21.15.44:0/1086596355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe910106650 0x7fe910106a30 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fe9000099c0 tx=0x7fe900009cd0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.405+0000 7fe916115700 1 -- 172.21.15.44:0/1086596355 shutdown_connections 2023-09-10T01:02:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.405+0000 7fe916115700 1 --2- 172.21.15.44:0/1086596355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe910100f50 0x7fe91010d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.405+0000 7fe916115700 1 --2- 172.21.15.44:0/1086596355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe910106650 0x7fe910106a30 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.405+0000 7fe916115700 1 --2- 172.21.15.44:0/1086596355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe910100590 0x7fe910100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.405+0000 7fe916115700 1 -- 172.21.15.44:0/1086596355 >> 172.21.15.44:0/1086596355 conn(0x7fe9100fc060 msgr2=0x7fe9100fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.405+0000 7fe916115700 1 -- 172.21.15.44:0/1086596355 shutdown_connections 2023-09-10T01:02:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.405+0000 7fe916115700 1 -- 172.21.15.44:0/1086596355 wait complete. 2023-09-10T01:02:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.406+0000 7fe916115700 1 Processor -- start 2023-09-10T01:02:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.406+0000 7fe916115700 1 -- start start 2023-09-10T01:02:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.406+0000 7fe916115700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe910100590 0x7fe910198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.406+0000 7fe916115700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe910100f50 0x7fe910198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.406+0000 7fe916115700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe910106650 0x7fe91019d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.406+0000 7fe916115700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91019d930 con 0x7fe910106650 2023-09-10T01:02:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.406+0000 7fe916115700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91019daa0 con 0x7fe910100590 2023-09-10T01:02:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.406+0000 7fe916115700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91019dc60 con 0x7fe910100f50 2023-09-10T01:02:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe90f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe910100590 0x7fe910198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe90f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe910100590 0x7fe910198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56720/0 (socket says 172.21.15.44:56720) 2023-09-10T01:02:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe90f7fe700 1 -- 172.21.15.44:0/199403563 learned_addr learned my addr 172.21.15.44:0/199403563 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe906ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe910100f50 0x7fe910198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe90ffff700 1 --2- 172.21.15.44:0/199403563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe910106650 0x7fe91019d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe90f7fe700 1 -- 172.21.15.44:0/199403563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe910100f50 msgr2=0x7fe910198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe90f7fe700 1 --2- 172.21.15.44:0/199403563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe910100f50 0x7fe910198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe90f7fe700 1 -- 172.21.15.44:0/199403563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe910106650 msgr2=0x7fe91019d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe90f7fe700 1 --2- 172.21.15.44:0/199403563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe910106650 0x7fe91019d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe90f7fe700 1 -- 172.21.15.44:0/199403563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9000095f0 con 0x7fe910100590 2023-09-10T01:02:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe906ffd700 1 --2- 172.21.15.44:0/199403563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe910100f50 0x7fe910198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:02:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe90f7fe700 1 --2- 172.21.15.44:0/199403563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe910100590 0x7fe910198780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe900005910 tx=0x7fe90000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe90d7fa700 1 -- 172.21.15.44:0/199403563 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90001d070 con 0x7fe910100590 2023-09-10T01:02:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe916115700 1 -- 172.21.15.44:0/199403563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91019dee0 con 0x7fe910100590 2023-09-10T01:02:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe90d7fa700 1 -- 172.21.15.44:0/199403563 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe900005050 con 0x7fe910100590 2023-09-10T01:02:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe90d7fa700 1 -- 172.21.15.44:0/199403563 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe900004950 con 0x7fe910100590 2023-09-10T01:02:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.407+0000 7fe916115700 1 -- 172.21.15.44:0/199403563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91019e1b0 con 0x7fe910100590 2023-09-10T01:02:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.410+0000 7fe90d7fa700 1 -- 172.21.15.44:0/199403563 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fe90000fad0 con 0x7fe910100590 2023-09-10T01:02:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.411+0000 7fe90d7fa700 1 --2- 172.21.15.44:0/199403563 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe8fc0a07b0 0x7fe8fc0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.411+0000 7fe90d7fa700 1 -- 172.21.15.44:0/199403563 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fe9000cd0e0 con 0x7fe910100590 2023-09-10T01:02:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.411+0000 7fe906ffd700 1 --2- 172.21.15.44:0/199403563 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe8fc0a07b0 0x7fe8fc0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.412+0000 7fe906ffd700 1 --2- 172.21.15.44:0/199403563 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe8fc0a07b0 0x7fe8fc0a2c70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe910199fa0 tx=0x7fe8f8009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.412+0000 7fe9222dbb80 1 Processor -- start 2023-09-10T01:02:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.412+0000 7fe9222dbb80 1 -- start start 2023-09-10T01:02:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.412+0000 7fe9222dbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b561a021a0 0x55b561a02580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.412+0000 7fe9222dbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5619fc300 0x55b5619fc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.412+0000 7fe9222dbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fccc0 0x55b561a08f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.412+0000 7fe9222dbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5619556d0 con 0x55b561a021a0 2023-09-10T01:02:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.412+0000 7fe9222dbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b561952ff0 con 0x55b5619fc300 2023-09-10T01:02:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.412+0000 7fe9222dbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b56194e8e0 con 0x55b5619fccc0 2023-09-10T01:02:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.412+0000 7fe90f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b561a021a0 0x55b561a02580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.412+0000 7fe90f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b561a021a0 0x55b561a02580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54988/0 (socket says 172.21.15.44:54988) 2023-09-10T01:02:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.412+0000 7fe90f7fe700 1 -- 172.21.15.44:0/1967692725 learned_addr learned my addr 172.21.15.44:0/1967692725 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.412+0000 7fe906ffd700 1 --2- 172.21.15.44:0/1967692725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5619fc300 0x55b5619fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.413+0000 7fe90f7fe700 1 -- 172.21.15.44:0/1967692725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fccc0 msgr2=0x55b561a08f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:02:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.413+0000 7fe90f7fe700 1 --2- 172.21.15.44:0/1967692725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fccc0 0x55b561a08f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.413+0000 7fe90f7fe700 1 -- 172.21.15.44:0/1967692725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5619fc300 msgr2=0x55b5619fc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.413+0000 7fe90f7fe700 1 --2- 172.21.15.44:0/1967692725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5619fc300 0x55b5619fc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.413+0000 7fe90f7fe700 1 -- 172.21.15.44:0/1967692725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b56177ad80 con 0x55b561a021a0 2023-09-10T01:02:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.413+0000 7fe906ffd700 1 --2- 172.21.15.44:0/1967692725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5619fc300 0x55b5619fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:02:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.413+0000 7fe90f7fe700 1 --2- 172.21.15.44:0/1967692725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b561a021a0 0x55b561a02580 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fe9000a1c80 tx=0x7fe900099c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0a70643c8a572e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.413+0000 7fe905ffb700 1 -- 172.21.15.44:0/1967692725 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90009e000 con 0x55b561a021a0 2023-09-10T01:02:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.413+0000 7fe905ffb700 1 -- 172.21.15.44:0/1967692725 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9000a21b0 con 0x55b561a021a0 2023-09-10T01:02:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.413+0000 7fe905ffb700 1 -- 172.21.15.44:0/1967692725 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90009a430 con 0x55b561a021a0 2023-09-10T01:02:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.414+0000 7fe9222dbb80 1 -- 172.21.15.44:0/1967692725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b561a021a0 msgr2=0x55b561a02580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.414+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/1967692725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b561a021a0 0x55b561a02580 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fe9000a1c80 tx=0x7fe900099c40 comp rx=0 tx=0).stop 2023-09-10T01:02:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.414+0000 7fe9222dbb80 1 -- 172.21.15.44:0/1967692725 shutdown_connections 2023-09-10T01:02:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.414+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/1967692725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fccc0 0x55b561a08f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.414+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/1967692725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b561a021a0 0x55b561a02580 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.414+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/1967692725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5619fc300 0x55b5619fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.414+0000 7fe9222dbb80 1 -- 172.21.15.44:0/1967692725 >> 172.21.15.44:0/1967692725 conn(0x55b561971030 msgr2=0x55b5619f7560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.414+0000 7fe9222dbb80 1 -- 172.21.15.44:0/1967692725 shutdown_connections 2023-09-10T01:02:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.414+0000 7fe9222dbb80 1 -- 172.21.15.44:0/1967692725 wait complete. 2023-09-10T01:02:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.414+0000 7fe9222dbb80 1 -- start start 2023-09-10T01:02:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.415+0000 7fe9222dbb80 1 Processor -- start 2023-09-10T01:02:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.415+0000 7fe9222dbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fc300 0x55b561a0ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.415+0000 7fe9222dbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5619fccc0 0x55b561a0bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.415+0000 7fe9222dbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b561a021a0 0x55b561a05ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.415+0000 7fe9222dbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5618ecb10 con 0x55b5619fccc0 2023-09-10T01:02:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.415+0000 7fe9222dbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5618ec810 con 0x55b561a021a0 2023-09-10T01:02:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.415+0000 7fe9222dbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5617d08c0 con 0x55b5619fc300 2023-09-10T01:02:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.415+0000 7fe9222dbb80 10 client.0 _ll_register_callbacks cb 0x55b561836a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:02:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.415+0000 7fe906ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5619fccc0 0x55b561a0bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.415+0000 7fe90f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fc300 0x55b561a0ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.415+0000 7fe906ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5619fccc0 0x55b561a0bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54994/0 (socket says 172.21.15.44:54994) 2023-09-10T01:02:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.415+0000 7fe906ffd700 1 -- 172.21.15.44:0/402936550 learned_addr learned my addr 172.21.15.44:0/402936550 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.415+0000 7fe90ffff700 1 --2- 172.21.15.44:0/402936550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b561a021a0 0x55b561a05ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.415+0000 7fe90f7fe700 1 -- 172.21.15.44:0/402936550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b561a021a0 msgr2=0x55b561a05ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.415+0000 7fe90f7fe700 1 --2- 172.21.15.44:0/402936550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b561a021a0 0x55b561a05ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.415+0000 7fe90f7fe700 1 -- 172.21.15.44:0/402936550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5619fccc0 msgr2=0x55b561a0bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.416+0000 7fe90f7fe700 1 --2- 172.21.15.44:0/402936550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5619fccc0 0x55b561a0bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.416+0000 7fe90f7fe700 1 -- 172.21.15.44:0/402936550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f8017070 con 0x55b5619fc300 2023-09-10T01:02:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.416+0000 7fe906ffd700 1 --2- 172.21.15.44:0/402936550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5619fccc0 0x55b561a0bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:02:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.416+0000 7fe90f7fe700 1 --2- 172.21.15.44:0/402936550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fc300 0x55b561a0ba50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe900098520 tx=0x7fe90009e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=534e5e6e2efaf1d8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.416+0000 7fe8eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:02:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.416+0000 7fe8eeffd700 1 -- 172.21.15.44:0/402936550 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9000996c0 con 0x55b5619fc300 2023-09-10T01:02:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.416+0000 7fe9222dbb80 1 -- 172.21.15.44:0/402936550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5618a6490 con 0x55b5619fc300 2023-09-10T01:02:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.416+0000 7fe8eeffd700 1 -- 172.21.15.44:0/402936550 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fe900099820 con 0x55b5619fc300 2023-09-10T01:02:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.416+0000 7fe8eeffd700 1 -- 172.21.15.44:0/402936550 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe90009a430 con 0x55b5619fc300 2023-09-10T01:02:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.416+0000 7fe9222dbb80 10 client.28024 fetch_fsmap learned FSMap version 87 2023-09-10T01:02:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.416+0000 7fe9222dbb80 1 -- 172.21.15.44:0/402936550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x7fe91019e1b0 con 0x55b5619fc300 2023-09-10T01:02:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.416+0000 7fe8eeffd700 1 -- 172.21.15.44:0/402936550 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b5618a6490 con 0x55b5619fc300 2023-09-10T01:02:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.417+0000 7fe8eeffd700 1 -- 172.21.15.44:0/402936550 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 87) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fe900095fc0 con 0x55b5619fc300 2023-09-10T01:02:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.417+0000 7fe9222dbb80 10 client.28024 fetch_fsmap finished waiting for FSMap version 87 2023-09-10T01:02:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.417+0000 7fe9222dbb80 10 client.28024 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:02:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.417+0000 7fe9222dbb80 20 client.28024 populate_metadata read hostname 'smithi044' 2023-09-10T01:02:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.417+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/402936550 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b561a15a60 0x55b561a17e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.417+0000 7fe9222dbb80 4 client.28024 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:02:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.417+0000 7fe9222dbb80 1 -- 172.21.15.44:0/402936550 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b561963840 con 0x55b561a15a60 2023-09-10T01:02:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.417+0000 7fe906ffd700 1 --2- 172.21.15.44:0/402936550 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b561a15a60 0x55b561a17e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.417+0000 7fe8eeffd700 10 client.28024 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:02:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.417+0000 7fe906ffd700 1 --2- 172.21.15.44:0/402936550 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b561a15a60 0x55b561a17e50 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab5d3dbabf8d0460 server_cookie=9d88857e65c8bf33 in_seq=0 out_seq=0 2023-09-10T01:02:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.418+0000 7fe8eeffd700 1 -- 172.21.15.44:0/402936550 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b561963840 con 0x55b561a15a60 2023-09-10T01:02:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.418+0000 7fe8eeffd700 10 client.28024 handle_command_reply: tid=0 2023-09-10T01:02:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.418+0000 7fe9222dbb80 1 client.28024 shutdown 2023-09-10T01:02:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.418+0000 7fe9222dbb80 1 -- 172.21.15.44:0/402936550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fc300 msgr2=0x55b561a0ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.418+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/402936550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fc300 0x55b561a0ba50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe900098520 tx=0x7fe90009e000 comp rx=0 tx=0).stop 2023-09-10T01:02:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.418+0000 7fe9222dbb80 1 -- 172.21.15.44:0/402936550 shutdown_connections 2023-09-10T01:02:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.418+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/402936550 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b561a15a60 0x55b561a17e50 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.419+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/402936550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b561a021a0 0x55b561a05ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.419+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/402936550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fc300 0x55b561a0ba50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.419+0000 7fe8eeffd700 0 client.28024 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:02:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.419+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/402936550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5619fccc0 0x55b561a0bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.419+0000 7fe9222dbb80 1 -- 172.21.15.44:0/402936550 >> 172.21.15.44:0/402936550 conn(0x55b561971030 msgr2=0x55b5619f67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.419+0000 7fe9222dbb80 1 -- 172.21.15.44:0/402936550 shutdown_connections 2023-09-10T01:02:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.419+0000 7fe9222dbb80 1 -- 172.21.15.44:0/402936550 wait complete. 2023-09-10T01:02:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.419+0000 7fe9222dbb80 20 client.28024 trim_cache size 0 max 16384 2023-09-10T01:02:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.437+0000 7fe9222dbb80 1 Processor -- start 2023-09-10T01:02:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.437+0000 7fe9222dbb80 1 -- start start 2023-09-10T01:02:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe9222dbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5619fc300 0x55b561a01660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe9222dbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fccc0 0x55b5619f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe9222dbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b561a021a0 0x55b5619f7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe9222dbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5619fd240 con 0x55b5619fc300 2023-09-10T01:02:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe9222dbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b561a04240 con 0x55b561a021a0 2023-09-10T01:02:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe9222dbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b561a00b40 con 0x55b5619fccc0 2023-09-10T01:02:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe90f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5619fc300 0x55b561a01660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe906ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fccc0 0x55b5619f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe90ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b561a021a0 0x55b5619f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe90f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5619fc300 0x55b561a01660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55000/0 (socket says 172.21.15.44:55000) 2023-09-10T01:02:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe90f7fe700 1 -- 172.21.15.44:0/2905170188 learned_addr learned my addr 172.21.15.44:0/2905170188 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe90f7fe700 1 -- 172.21.15.44:0/2905170188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fccc0 msgr2=0x55b5619f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe90f7fe700 1 --2- 172.21.15.44:0/2905170188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fccc0 0x55b5619f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe90f7fe700 1 -- 172.21.15.44:0/2905170188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b561a021a0 msgr2=0x55b5619f7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe90f7fe700 1 --2- 172.21.15.44:0/2905170188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b561a021a0 0x55b5619f7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe90f7fe700 1 -- 172.21.15.44:0/2905170188 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b561763c00 con 0x55b5619fc300 2023-09-10T01:02:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe906ffd700 1 --2- 172.21.15.44:0/2905170188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fccc0 0x55b5619f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:02:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe90ffff700 1 --2- 172.21.15.44:0/2905170188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b561a021a0 0x55b5619f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:02:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.438+0000 7fe90f7fe700 1 --2- 172.21.15.44:0/2905170188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5619fc300 0x55b561a01660 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fe900009fd0 tx=0x7fe9000b8250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a82c236132f290c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.439+0000 7fe9067fc700 1 -- 172.21.15.44:0/2905170188 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90009e070 con 0x55b5619fc300 2023-09-10T01:02:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.439+0000 7fe9067fc700 1 -- 172.21.15.44:0/2905170188 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe900099fa0 con 0x55b5619fc300 2023-09-10T01:02:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.439+0000 7fe9067fc700 1 -- 172.21.15.44:0/2905170188 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9000a1f80 con 0x55b5619fc300 2023-09-10T01:02:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.439+0000 7fe9222dbb80 1 -- 172.21.15.44:0/2905170188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5619fc300 msgr2=0x55b561a01660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.439+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/2905170188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5619fc300 0x55b561a01660 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fe900009fd0 tx=0x7fe9000b8250 comp rx=0 tx=0).stop 2023-09-10T01:02:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.440+0000 7fe9222dbb80 1 -- 172.21.15.44:0/2905170188 shutdown_connections 2023-09-10T01:02:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.440+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/2905170188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b561a021a0 0x55b5619f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.440+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/2905170188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5619fc300 0x55b561a01660 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.440+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/2905170188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fccc0 0x55b5619f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.440+0000 7fe9222dbb80 1 -- 172.21.15.44:0/2905170188 >> 172.21.15.44:0/2905170188 conn(0x55b561971030 msgr2=0x55b561a00f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.440+0000 7fe9222dbb80 1 -- 172.21.15.44:0/2905170188 shutdown_connections 2023-09-10T01:02:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.440+0000 7fe9222dbb80 1 -- 172.21.15.44:0/2905170188 wait complete. 2023-09-10T01:02:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.440+0000 7fe9222dbb80 1 -- start start 2023-09-10T01:02:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.440+0000 7fe9222dbb80 1 Processor -- start 2023-09-10T01:02:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.440+0000 7fe9222dbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5619fc300 0x55b561a5d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.440+0000 7fe9222dbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fccc0 0x55b561a5df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.440+0000 7fe9222dbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b561a021a0 0x55b561a68860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.440+0000 7fe9222dbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b561a68da0 con 0x55b561a021a0 2023-09-10T01:02:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.440+0000 7fe9222dbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b561a68ee0 con 0x55b5619fc300 2023-09-10T01:02:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.440+0000 7fe9222dbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b561a69020 con 0x55b5619fccc0 2023-09-10T01:02:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe9222dbb80 10 client.0 _ll_register_callbacks cb 0x55b561836a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:02:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe906ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fccc0 0x55b561a5df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe90f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5619fc300 0x55b561a5d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe90f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5619fc300 0x55b561a5d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56744/0 (socket says 172.21.15.44:56744) 2023-09-10T01:02:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe90ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b561a021a0 0x55b561a68860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe90f7fe700 1 -- 172.21.15.44:0/3963095389 learned_addr learned my addr 172.21.15.44:0/3963095389 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe906ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fccc0 0x55b561a5df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33654/0 (socket says 172.21.15.44:33654) 2023-09-10T01:02:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe90f7fe700 1 -- 172.21.15.44:0/3963095389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fccc0 msgr2=0x55b561a5df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe90f7fe700 1 --2- 172.21.15.44:0/3963095389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fccc0 0x55b561a5df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe90f7fe700 1 -- 172.21.15.44:0/3963095389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b561a021a0 msgr2=0x55b561a68860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe90f7fe700 1 --2- 172.21.15.44:0/3963095389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b561a021a0 0x55b561a68860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe90f7fe700 1 -- 172.21.15.44:0/3963095389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f8003960 con 0x55b5619fc300 2023-09-10T01:02:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe90f7fe700 1 --2- 172.21.15.44:0/3963095389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5619fc300 0x55b561a5d9d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe9000a1c80 tx=0x7fe900009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=301a7b0cf49fed66 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe8eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:02:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe8eeffd700 1 -- 172.21.15.44:0/3963095389 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90009e000 con 0x55b5619fc300 2023-09-10T01:02:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe9222dbb80 1 -- 172.21.15.44:0/3963095389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9000a21b0 con 0x55b5619fc300 2023-09-10T01:02:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.441+0000 7fe8eeffd700 1 -- 172.21.15.44:0/3963095389 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fe9000996c0 con 0x55b5619fc300 2023-09-10T01:02:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.442+0000 7fe8eeffd700 1 -- 172.21.15.44:0/3963095389 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe900099880 con 0x55b5619fc300 2023-09-10T01:02:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.442+0000 7fe9222dbb80 10 client.28094 fetch_fsmap learned FSMap version 87 2023-09-10T01:02:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.442+0000 7fe9222dbb80 1 -- 172.21.15.44:0/3963095389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x7fe908012350 con 0x55b5619fc300 2023-09-10T01:02:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.442+0000 7fe8eeffd700 1 -- 172.21.15.44:0/3963095389 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9000a9080 con 0x55b5619fc300 2023-09-10T01:02:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.442+0000 7fe8eeffd700 1 -- 172.21.15.44:0/3963095389 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 87) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fe900099250 con 0x55b5619fc300 2023-09-10T01:02:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.442+0000 7fe9222dbb80 10 client.28094 fetch_fsmap finished waiting for FSMap version 87 2023-09-10T01:02:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.442+0000 7fe9222dbb80 10 client.28094 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:02:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.442+0000 7fe9222dbb80 20 client.28094 populate_metadata read hostname 'smithi044' 2023-09-10T01:02:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.442+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/3963095389 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b561a6a280 0x55b561a6c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.442+0000 7fe9222dbb80 4 client.28094 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:02:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.442+0000 7fe9222dbb80 1 -- 172.21.15.44:0/3963095389 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b5619478c0 con 0x55b561a6a280 2023-09-10T01:02:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.442+0000 7fe906ffd700 1 --2- 172.21.15.44:0/3963095389 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b561a6a280 0x55b561a6c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.443+0000 7fe8eeffd700 10 client.28094 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:02:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.443+0000 7fe906ffd700 1 --2- 172.21.15.44:0/3963095389 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b561a6a280 0x55b561a6c670 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2ef7adf251c55c8 server_cookie=2962f53c9a1986bd in_seq=0 out_seq=0 2023-09-10T01:02:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.452+0000 7fe8eeffd700 1 -- 172.21.15.44:0/3963095389 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b5619478c0 con 0x55b561a6a280 2023-09-10T01:02:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.452+0000 7fe8eeffd700 10 client.28094 handle_command_reply: tid=0 2023-09-10T01:02:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.453+0000 7fe9222dbb80 1 client.28094 shutdown 2023-09-10T01:02:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.453+0000 7fe9222dbb80 1 -- 172.21.15.44:0/3963095389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5619fc300 msgr2=0x55b561a5d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.453+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/3963095389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5619fc300 0x55b561a5d9d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe9000a1c80 tx=0x7fe900009cd0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.453+0000 7fe9222dbb80 1 -- 172.21.15.44:0/3963095389 shutdown_connections 2023-09-10T01:02:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.453+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/3963095389 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b561a6a280 0x55b561a6c670 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.453+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/3963095389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b561a021a0 0x55b561a68860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.453+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/3963095389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5619fc300 0x55b561a5d9d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.453+0000 7fe8eeffd700 0 client.28094 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:02:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.453+0000 7fe9222dbb80 1 --2- 172.21.15.44:0/3963095389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5619fccc0 0x55b561a5df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.453+0000 7fe9222dbb80 1 -- 172.21.15.44:0/3963095389 >> 172.21.15.44:0/3963095389 conn(0x55b561971030 msgr2=0x55b5619fbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.453+0000 7fe9222dbb80 1 -- 172.21.15.44:0/3963095389 shutdown_connections 2023-09-10T01:02:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.453+0000 7fe9222dbb80 1 -- 172.21.15.44:0/3963095389 wait complete. 2023-09-10T01:02:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.453+0000 7fe9222dbb80 20 client.28094 trim_cache size 0 max 16384 2023-09-10T01:02:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:02:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:02:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:02:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:02:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.454+0000 7fe916115700 1 -- 172.21.15.44:0/199403563 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe8fc0a07b0 msgr2=0x7fe8fc0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.454+0000 7fe916115700 1 --2- 172.21.15.44:0/199403563 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe8fc0a07b0 0x7fe8fc0a2c70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe910199fa0 tx=0x7fe8f8009260 comp rx=0 tx=0).stop 2023-09-10T01:02:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.454+0000 7fe916115700 1 -- 172.21.15.44:0/199403563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe910100590 msgr2=0x7fe910198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.454+0000 7fe916115700 1 --2- 172.21.15.44:0/199403563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe910100590 0x7fe910198780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe900005910 tx=0x7fe90000f690 comp rx=0 tx=0).stop 2023-09-10T01:02:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.454+0000 7fe916115700 1 -- 172.21.15.44:0/199403563 shutdown_connections 2023-09-10T01:02:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.454+0000 7fe916115700 1 --2- 172.21.15.44:0/199403563 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe8fc0a07b0 0x7fe8fc0a2c70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.454+0000 7fe916115700 1 --2- 172.21.15.44:0/199403563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe910106650 0x7fe91019d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.454+0000 7fe916115700 1 --2- 172.21.15.44:0/199403563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe910100590 0x7fe910198780 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.454+0000 7fe916115700 1 --2- 172.21.15.44:0/199403563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe910100f50 0x7fe910198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.454+0000 7fe916115700 1 -- 172.21.15.44:0/199403563 >> 172.21.15.44:0/199403563 conn(0x7fe9100fc060 msgr2=0x7fe9100fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.454+0000 7fe916115700 1 -- 172.21.15.44:0/199403563 shutdown_connections 2023-09-10T01:02:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:09.454+0000 7fe916115700 1 -- 172.21.15.44:0/199403563 wait complete. 2023-09-10T01:02:10.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:09 smithi099 ceph-mon[102692]: osdmap e125: 12 total, 12 up, 12 in 2023-09-10T01:02:10.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:09 smithi131 ceph-mon[103713]: osdmap e125: 12 total, 12 up, 12 in 2023-09-10T01:02:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:09 smithi044 ceph-mon[98676]: osdmap e125: 12 total, 12 up, 12 in 2023-09-10T01:02:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:10 smithi099 ceph-mon[102692]: pgmap v681: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 245 op/s 2023-09-10T01:02:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:10 smithi131 ceph-mon[103713]: pgmap v681: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 245 op/s 2023-09-10T01:02:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:10 smithi044 ceph-mon[98676]: pgmap v681: 129 pgs: 129 active+clean; 8.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 245 op/s 2023-09-10T01:02:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:12 smithi099 ceph-mon[102692]: pgmap v682: 129 pgs: 129 active+clean; 7.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 265 op/s 2023-09-10T01:02:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:12 smithi131 ceph-mon[103713]: pgmap v682: 129 pgs: 129 active+clean; 7.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 265 op/s 2023-09-10T01:02:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:12 smithi044 ceph-mon[98676]: pgmap v682: 129 pgs: 129 active+clean; 7.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 265 op/s 2023-09-10T01:02:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:14 smithi131 ceph-mon[103713]: pgmap v683: 129 pgs: 129 active+clean; 7.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 265 op/s 2023-09-10T01:02:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:14 smithi044 ceph-mon[98676]: pgmap v683: 129 pgs: 129 active+clean; 7.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 265 op/s 2023-09-10T01:02:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:14 smithi099 ceph-mon[102692]: pgmap v683: 129 pgs: 129 active+clean; 7.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 265 op/s 2023-09-10T01:02:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:16 smithi131 ceph-mon[103713]: pgmap v684: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 22 MiB/s wr, 134 op/s 2023-09-10T01:02:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:16 smithi131 ceph-mon[103713]: osdmap e126: 12 total, 12 up, 12 in 2023-09-10T01:02:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:16 smithi044 ceph-mon[98676]: pgmap v684: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 22 MiB/s wr, 134 op/s 2023-09-10T01:02:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:16 smithi044 ceph-mon[98676]: osdmap e126: 12 total, 12 up, 12 in 2023-09-10T01:02:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:16 smithi099 ceph-mon[102692]: pgmap v684: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 22 MiB/s wr, 134 op/s 2023-09-10T01:02:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:16 smithi099 ceph-mon[102692]: osdmap e126: 12 total, 12 up, 12 in 2023-09-10T01:02:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:18 smithi131 ceph-mon[103713]: pgmap v686: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 26 MiB/s wr, 158 op/s 2023-09-10T01:02:19.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:18 smithi044 ceph-mon[98676]: pgmap v686: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 26 MiB/s wr, 158 op/s 2023-09-10T01:02:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:18 smithi099 ceph-mon[102692]: pgmap v686: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 26 MiB/s wr, 158 op/s 2023-09-10T01:02:21.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:20 smithi131 ceph-mon[103713]: pgmap v687: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 22 MiB/s wr, 134 op/s 2023-09-10T01:02:21.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:20 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:02:21.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:20 smithi044 ceph-mon[98676]: pgmap v687: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 22 MiB/s wr, 134 op/s 2023-09-10T01:02:21.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:20 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:02:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:20 smithi099 ceph-mon[102692]: pgmap v687: 129 pgs: 129 active+clean; 8.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 22 MiB/s wr, 134 op/s 2023-09-10T01:02:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:20 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:02:22.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:21 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:02:22.876 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:21 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:02:22.876 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:21 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:02:22.877 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:21 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:02:22.877 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:21 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:02:22.878 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:21 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:02:22.878 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:21 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:02:22.878 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:21 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:02:22.879 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:21 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:02:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.588+0000 7ff61bae4700 1 -- 172.21.15.44:0/1214093466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6140ffe90 msgr2=0x7ff61410c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.588+0000 7ff61bae4700 1 --2- 172.21.15.44:0/1214093466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6140ffe90 0x7ff61410c350 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009f90 tx=0x7ff60c00b330 comp rx=0 tx=0).stop 2023-09-10T01:02:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.588+0000 7ff61bae4700 1 -- 172.21.15.44:0/1214093466 shutdown_connections 2023-09-10T01:02:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.588+0000 7ff61bae4700 1 --2- 172.21.15.44:0/1214093466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6140ffe90 0x7ff61410c350 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.588+0000 7ff61bae4700 1 --2- 172.21.15.44:0/1214093466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff614105590 0x7ff614105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.588+0000 7ff61bae4700 1 --2- 172.21.15.44:0/1214093466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6140ff4d0 0x7ff6140ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.588+0000 7ff61bae4700 1 -- 172.21.15.44:0/1214093466 >> 172.21.15.44:0/1214093466 conn(0x7ff6140faf80 msgr2=0x7ff6140fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.588+0000 7ff61bae4700 1 -- 172.21.15.44:0/1214093466 shutdown_connections 2023-09-10T01:02:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.588+0000 7ff61bae4700 1 -- 172.21.15.44:0/1214093466 wait complete. 2023-09-10T01:02:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.589+0000 7ff61bae4700 1 Processor -- start 2023-09-10T01:02:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.589+0000 7ff61bae4700 1 -- start start 2023-09-10T01:02:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.589+0000 7ff61bae4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6140ff4d0 0x7ff614197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.589+0000 7ff61bae4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6140ffe90 0x7ff614197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.589+0000 7ff61bae4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff614105590 0x7ff6141a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.589+0000 7ff61bae4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6141a2a80 con 0x7ff6140ffe90 2023-09-10T01:02:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.589+0000 7ff61bae4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6141a2bf0 con 0x7ff6140ff4d0 2023-09-10T01:02:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.589+0000 7ff61bae4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6141a2db0 con 0x7ff614105590 2023-09-10T01:02:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.590+0000 7ff61907f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6140ffe90 0x7ff614197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.590+0000 7ff61a081700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff614105590 0x7ff6141a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.590+0000 7ff61907f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6140ffe90 0x7ff614197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55018/0 (socket says 172.21.15.44:55018) 2023-09-10T01:02:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.590+0000 7ff61907f700 1 -- 172.21.15.44:0/1709846450 learned_addr learned my addr 172.21.15.44:0/1709846450 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.590+0000 7ff619880700 1 --2- 172.21.15.44:0/1709846450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6140ff4d0 0x7ff614197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.590+0000 7ff61a081700 1 -- 172.21.15.44:0/1709846450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6140ff4d0 msgr2=0x7ff614197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.590+0000 7ff61a081700 1 --2- 172.21.15.44:0/1709846450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6140ff4d0 0x7ff614197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.590+0000 7ff61a081700 1 -- 172.21.15.44:0/1709846450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6140ffe90 msgr2=0x7ff614197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.590+0000 7ff61a081700 1 --2- 172.21.15.44:0/1709846450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6140ffe90 0x7ff614197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.590+0000 7ff61a081700 1 -- 172.21.15.44:0/1709846450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c009c40 con 0x7ff614105590 2023-09-10T01:02:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.590+0000 7ff61a081700 1 --2- 172.21.15.44:0/1709846450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff614105590 0x7ff6141a24b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00b6c0 tx=0x7ff60c008aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.590+0000 7ff60affd700 1 -- 172.21.15.44:0/1709846450 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c00e040 con 0x7ff614105590 2023-09-10T01:02:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.590+0000 7ff60affd700 1 -- 172.21.15.44:0/1709846450 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff60c00bbf0 con 0x7ff614105590 2023-09-10T01:02:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.590+0000 7ff61bae4700 1 -- 172.21.15.44:0/1709846450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6141a3030 con 0x7ff614105590 2023-09-10T01:02:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.590+0000 7ff60affd700 1 -- 172.21.15.44:0/1709846450 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0052a0 con 0x7ff614105590 2023-09-10T01:02:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.590+0000 7ff61bae4700 1 -- 172.21.15.44:0/1709846450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6141a3460 con 0x7ff614105590 2023-09-10T01:02:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.593+0000 7ff60affd700 1 -- 172.21.15.44:0/1709846450 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ff60c01d470 con 0x7ff614105590 2023-09-10T01:02:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.594+0000 7ff60affd700 1 --2- 172.21.15.44:0/1709846450 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff6000a07b0 0x7ff6000a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.594+0000 7ff619880700 1 --2- 172.21.15.44:0/1709846450 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff6000a07b0 0x7ff6000a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.594+0000 7ff60affd700 1 -- 172.21.15.44:0/1709846450 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff60c0cd850 con 0x7ff614105590 2023-09-10T01:02:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.594+0000 7ff627caab80 1 Processor -- start 2023-09-10T01:02:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.594+0000 7ff619880700 1 --2- 172.21.15.44:0/1709846450 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff6000a07b0 0x7ff6000a2c70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff610009bf0 tx=0x7ff610009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.594+0000 7ff627caab80 1 -- start start 2023-09-10T01:02:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff627caab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae531a0 0x564c2ae53580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff627caab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4d300 0x564c2ae4d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff627caab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae4dcc0 0x564c2ae59f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff627caab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2ada66d0 con 0x564c2ae531a0 2023-09-10T01:02:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff627caab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2ada3ff0 con 0x564c2ae4dcc0 2023-09-10T01:02:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff627caab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2ad9f8e0 con 0x564c2ae4d300 2023-09-10T01:02:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff619880700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae531a0 0x564c2ae53580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff619880700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae531a0 0x564c2ae53580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55024/0 (socket says 172.21.15.44:55024) 2023-09-10T01:02:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff619880700 1 -- 172.21.15.44:0/3176227140 learned_addr learned my addr 172.21.15.44:0/3176227140 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff61907f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4d300 0x564c2ae4d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff61a081700 1 --2- 172.21.15.44:0/3176227140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae4dcc0 0x564c2ae59f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff619880700 1 -- 172.21.15.44:0/3176227140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4d300 msgr2=0x564c2ae4d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff619880700 1 --2- 172.21.15.44:0/3176227140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4d300 0x564c2ae4d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff619880700 1 -- 172.21.15.44:0/3176227140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae4dcc0 msgr2=0x564c2ae59f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff619880700 1 --2- 172.21.15.44:0/3176227140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae4dcc0 0x564c2ae59f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff619880700 1 -- 172.21.15.44:0/3176227140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2abcbd80 con 0x564c2ae531a0 2023-09-10T01:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff61a081700 1 --2- 172.21.15.44:0/3176227140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae4dcc0 0x564c2ae59f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff61907f700 1 --2- 172.21.15.44:0/3176227140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4d300 0x564c2ae4d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:02:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.595+0000 7ff619880700 1 --2- 172.21.15.44:0/3176227140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae531a0 0x564c2ae53580 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ff610011e40 tx=0x7ff610014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897c8e17fab9b382 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.596+0000 7ff5fffff700 1 -- 172.21.15.44:0/3176227140 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610004220 con 0x564c2ae531a0 2023-09-10T01:02:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.596+0000 7ff5fffff700 1 -- 172.21.15.44:0/3176227140 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff610016070 con 0x564c2ae531a0 2023-09-10T01:02:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.596+0000 7ff5fffff700 1 -- 172.21.15.44:0/3176227140 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61001f680 con 0x564c2ae531a0 2023-09-10T01:02:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.596+0000 7ff627caab80 1 -- 172.21.15.44:0/3176227140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae531a0 msgr2=0x564c2ae53580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.596+0000 7ff627caab80 1 --2- 172.21.15.44:0/3176227140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae531a0 0x564c2ae53580 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ff610011e40 tx=0x7ff610014000 comp rx=0 tx=0).stop 2023-09-10T01:02:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.596+0000 7ff627caab80 1 -- 172.21.15.44:0/3176227140 shutdown_connections 2023-09-10T01:02:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.596+0000 7ff627caab80 1 --2- 172.21.15.44:0/3176227140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae4dcc0 0x564c2ae59f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.596+0000 7ff627caab80 1 --2- 172.21.15.44:0/3176227140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae531a0 0x564c2ae53580 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.596+0000 7ff627caab80 1 --2- 172.21.15.44:0/3176227140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4d300 0x564c2ae4d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.597+0000 7ff627caab80 1 -- 172.21.15.44:0/3176227140 >> 172.21.15.44:0/3176227140 conn(0x564c2adc2030 msgr2=0x564c2ae48560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.597+0000 7ff627caab80 1 -- 172.21.15.44:0/3176227140 shutdown_connections 2023-09-10T01:02:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.597+0000 7ff627caab80 1 -- 172.21.15.44:0/3176227140 wait complete. 2023-09-10T01:02:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.597+0000 7ff627caab80 1 -- start start 2023-09-10T01:02:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.597+0000 7ff627caab80 1 Processor -- start 2023-09-10T01:02:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff627caab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4d300 0x564c2ae59e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff627caab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4dcc0 0x564c2ae57160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff627caab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae531a0 0x564c2ae577e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff627caab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2ad3db10 con 0x564c2ae4d300 2023-09-10T01:02:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff627caab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2ad3d810 con 0x564c2ae531a0 2023-09-10T01:02:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff627caab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2ac218c0 con 0x564c2ae4dcc0 2023-09-10T01:02:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff627caab80 10 client.0 _ll_register_callbacks cb 0x564c2ac87a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:02:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff61907f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4dcc0 0x564c2ae57160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff619880700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4d300 0x564c2ae59e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff61a081700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae531a0 0x564c2ae577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff61907f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4dcc0 0x564c2ae57160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33682/0 (socket says 172.21.15.44:33682) 2023-09-10T01:02:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff61907f700 1 -- 172.21.15.44:0/1956176432 learned_addr learned my addr 172.21.15.44:0/1956176432 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff61a081700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae531a0 0x564c2ae577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56774/0 (socket says 172.21.15.44:56774) 2023-09-10T01:02:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff61a081700 1 -- 172.21.15.44:0/1956176432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4dcc0 msgr2=0x564c2ae57160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff61a081700 1 --2- 172.21.15.44:0/1956176432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4dcc0 0x564c2ae57160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff61a081700 1 -- 172.21.15.44:0/1956176432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4d300 msgr2=0x564c2ae59e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff61a081700 1 --2- 172.21.15.44:0/1956176432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4d300 0x564c2ae59e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff61a081700 1 -- 172.21.15.44:0/1956176432 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6040099a0 con 0x564c2ae531a0 2023-09-10T01:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff61a081700 1 --2- 172.21.15.44:0/1956176432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae531a0 0x564c2ae577e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0caba0 tx=0x7ff60c09a7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bedb7baffbbfa1d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.598+0000 7ff5fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.599+0000 7ff5fd7fa700 1 -- 172.21.15.44:0/1956176432 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c011e70 con 0x564c2ae531a0 2023-09-10T01:02:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.599+0000 7ff5fd7fa700 1 -- 172.21.15.44:0/1956176432 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff60c085040 con 0x564c2ae531a0 2023-09-10T01:02:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.599+0000 7ff627caab80 1 -- 172.21.15.44:0/1956176432 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c2acf7490 con 0x564c2ae531a0 2023-09-10T01:02:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.599+0000 7ff5fd7fa700 1 -- 172.21.15.44:0/1956176432 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff60c00bd60 con 0x564c2ae531a0 2023-09-10T01:02:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.599+0000 7ff627caab80 10 client.28106 fetch_fsmap learned FSMap version 87 2023-09-10T01:02:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.599+0000 7ff627caab80 1 -- 172.21.15.44:0/1956176432 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x7ff610011b20 con 0x564c2ae531a0 2023-09-10T01:02:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.599+0000 7ff5fd7fa700 1 -- 172.21.15.44:0/1956176432 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff60c064020 con 0x564c2ae531a0 2023-09-10T01:02:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.599+0000 7ff5fd7fa700 1 -- 172.21.15.44:0/1956176432 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 87) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff60c060be0 con 0x564c2ae531a0 2023-09-10T01:02:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.599+0000 7ff627caab80 10 client.28106 fetch_fsmap finished waiting for FSMap version 87 2023-09-10T01:02:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.599+0000 7ff627caab80 10 client.28106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:02:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.599+0000 7ff627caab80 20 client.28106 populate_metadata read hostname 'smithi044' 2023-09-10T01:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.599+0000 7ff627caab80 1 --2- 172.21.15.44:0/1956176432 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564c2ae67330 0x564c2ae69750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.599+0000 7ff627caab80 4 client.28106 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.599+0000 7ff627caab80 1 -- 172.21.15.44:0/1956176432 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c2adb4840 con 0x564c2ae67330 2023-09-10T01:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.599+0000 7ff619880700 1 --2- 172.21.15.44:0/1956176432 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564c2ae67330 0x564c2ae69750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.600+0000 7ff619880700 1 --2- 172.21.15.44:0/1956176432 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564c2ae67330 0x564c2ae69750 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38329d5a2529d78 server_cookie=fe34e6ce8d6c475 in_seq=0 out_seq=0 2023-09-10T01:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.600+0000 7ff5fd7fa700 10 client.28106 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:02:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.600+0000 7ff5fd7fa700 1 -- 172.21.15.44:0/1956176432 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564c2adb4840 con 0x564c2ae67330 2023-09-10T01:02:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.600+0000 7ff5fd7fa700 10 client.28106 handle_command_reply: tid=0 2023-09-10T01:02:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.600+0000 7ff627caab80 1 client.28106 shutdown 2023-09-10T01:02:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.601+0000 7ff627caab80 1 -- 172.21.15.44:0/1956176432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae531a0 msgr2=0x564c2ae577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.601+0000 7ff627caab80 1 --2- 172.21.15.44:0/1956176432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae531a0 0x564c2ae577e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0caba0 tx=0x7ff60c09a7e0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.601+0000 7ff627caab80 1 -- 172.21.15.44:0/1956176432 shutdown_connections 2023-09-10T01:02:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.601+0000 7ff627caab80 1 --2- 172.21.15.44:0/1956176432 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564c2ae67330 0x564c2ae69750 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.601+0000 7ff627caab80 1 --2- 172.21.15.44:0/1956176432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae531a0 0x564c2ae577e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.601+0000 7ff627caab80 1 --2- 172.21.15.44:0/1956176432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4d300 0x564c2ae59e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.601+0000 7ff5fd7fa700 0 client.28106 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:02:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.601+0000 7ff627caab80 1 --2- 172.21.15.44:0/1956176432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4dcc0 0x564c2ae57160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.601+0000 7ff627caab80 1 -- 172.21.15.44:0/1956176432 >> 172.21.15.44:0/1956176432 conn(0x564c2adc2030 msgr2=0x564c2ae477a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.601+0000 7ff627caab80 1 -- 172.21.15.44:0/1956176432 shutdown_connections 2023-09-10T01:02:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.601+0000 7ff627caab80 1 -- 172.21.15.44:0/1956176432 wait complete. 2023-09-10T01:02:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.601+0000 7ff627caab80 20 client.28106 trim_cache size 0 max 16384 2023-09-10T01:02:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.620+0000 7ff627caab80 1 Processor -- start 2023-09-10T01:02:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.620+0000 7ff627caab80 1 -- start start 2023-09-10T01:02:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.621+0000 7ff627caab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4d300 0x564c2ae5e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.621+0000 7ff627caab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4dcc0 0x564c2ae47f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.621+0000 7ff627caab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae531a0 0x564c2ae48460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.621+0000 7ff627caab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2ae428a0 con 0x564c2ae4dcc0 2023-09-10T01:02:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.621+0000 7ff627caab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2ae50db0 con 0x564c2ae531a0 2023-09-10T01:02:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.621+0000 7ff627caab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2ae4e580 con 0x564c2ae4d300 2023-09-10T01:02:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.621+0000 7ff61907f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4dcc0 0x564c2ae47f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.621+0000 7ff61907f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4dcc0 0x564c2ae47f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55038/0 (socket says 172.21.15.44:55038) 2023-09-10T01:02:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.621+0000 7ff61907f700 1 -- 172.21.15.44:0/796352820 learned_addr learned my addr 172.21.15.44:0/796352820 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.621+0000 7ff61a081700 1 --2- 172.21.15.44:0/796352820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae531a0 0x564c2ae48460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.621+0000 7ff619880700 1 --2- 172.21.15.44:0/796352820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4d300 0x564c2ae5e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.621+0000 7ff61a081700 1 -- 172.21.15.44:0/796352820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4d300 msgr2=0x564c2ae5e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.621+0000 7ff61a081700 1 --2- 172.21.15.44:0/796352820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4d300 0x564c2ae5e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.621+0000 7ff61a081700 1 -- 172.21.15.44:0/796352820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4dcc0 msgr2=0x564c2ae47f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.621+0000 7ff61a081700 1 --2- 172.21.15.44:0/796352820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4dcc0 0x564c2ae47f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.621+0000 7ff61a081700 1 -- 172.21.15.44:0/796352820 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2abb4c00 con 0x564c2ae531a0 2023-09-10T01:02:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.622+0000 7ff61a081700 1 --2- 172.21.15.44:0/796352820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae531a0 0x564c2ae48460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff60c096e90 tx=0x7ff60c00bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3a7a1c1363d8be8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.622+0000 7ff608d89700 1 -- 172.21.15.44:0/796352820 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c09a4d0 con 0x564c2ae531a0 2023-09-10T01:02:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.622+0000 7ff608d89700 1 -- 172.21.15.44:0/796352820 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff60c064020 con 0x564c2ae531a0 2023-09-10T01:02:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.622+0000 7ff608d89700 1 -- 172.21.15.44:0/796352820 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0a2970 con 0x564c2ae531a0 2023-09-10T01:02:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.623+0000 7ff627caab80 1 -- 172.21.15.44:0/796352820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae531a0 msgr2=0x564c2ae48460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.623+0000 7ff627caab80 1 --2- 172.21.15.44:0/796352820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae531a0 0x564c2ae48460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff60c096e90 tx=0x7ff60c00bf60 comp rx=0 tx=0).stop 2023-09-10T01:02:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.623+0000 7ff627caab80 1 -- 172.21.15.44:0/796352820 shutdown_connections 2023-09-10T01:02:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.623+0000 7ff627caab80 1 --2- 172.21.15.44:0/796352820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae531a0 0x564c2ae48460 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.623+0000 7ff627caab80 1 --2- 172.21.15.44:0/796352820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae4d300 0x564c2ae5e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.623+0000 7ff627caab80 1 --2- 172.21.15.44:0/796352820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4dcc0 0x564c2ae47f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.623+0000 7ff627caab80 1 -- 172.21.15.44:0/796352820 >> 172.21.15.44:0/796352820 conn(0x564c2adc2030 msgr2=0x564c2ae51230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.623+0000 7ff627caab80 1 -- 172.21.15.44:0/796352820 shutdown_connections 2023-09-10T01:02:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.623+0000 7ff627caab80 1 -- 172.21.15.44:0/796352820 wait complete. 2023-09-10T01:02:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.623+0000 7ff627caab80 1 -- start start 2023-09-10T01:02:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.623+0000 7ff627caab80 1 Processor -- start 2023-09-10T01:02:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff627caab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4d300 0x564c2aeb4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff627caab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae4dcc0 0x564c2aeb4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff627caab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae531a0 0x564c2aeaf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff627caab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2aeaf6b0 con 0x564c2ae4d300 2023-09-10T01:02:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff627caab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2aeaf820 con 0x564c2ae4dcc0 2023-09-10T01:02:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff627caab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2aeaf9e0 con 0x564c2ae531a0 2023-09-10T01:02:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff627caab80 10 client.0 _ll_register_callbacks cb 0x564c2ac87a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:02:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff619880700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4d300 0x564c2aeb4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff61907f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae4dcc0 0x564c2aeb4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff61907f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae4dcc0 0x564c2aeb4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56784/0 (socket says 172.21.15.44:56784) 2023-09-10T01:02:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff619880700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4d300 0x564c2aeb4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55042/0 (socket says 172.21.15.44:55042) 2023-09-10T01:02:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff61a081700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae531a0 0x564c2aeaf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff61907f700 1 -- 172.21.15.44:0/1247623750 learned_addr learned my addr 172.21.15.44:0/1247623750 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff61907f700 1 -- 172.21.15.44:0/1247623750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae531a0 msgr2=0x564c2aeaf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff61907f700 1 --2- 172.21.15.44:0/1247623750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae531a0 0x564c2aeaf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff61907f700 1 -- 172.21.15.44:0/1247623750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4d300 msgr2=0x564c2aeb4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff61907f700 1 --2- 172.21.15.44:0/1247623750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4d300 0x564c2aeb4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff61907f700 1 -- 172.21.15.44:0/1247623750 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff610011b20 con 0x564c2ae4dcc0 2023-09-10T01:02:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff61907f700 1 --2- 172.21.15.44:0/1247623750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae4dcc0 0x564c2aeb4bb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff604012d20 tx=0x7ff60400f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3c19b63087d5931 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.624+0000 7ff5fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:02:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.625+0000 7ff5fd7fa700 1 -- 172.21.15.44:0/1247623750 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604017040 con 0x564c2ae4dcc0 2023-09-10T01:02:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.625+0000 7ff5fd7fa700 1 -- 172.21.15.44:0/1247623750 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff60400fd80 con 0x564c2ae4dcc0 2023-09-10T01:02:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.625+0000 7ff627caab80 1 -- 172.21.15.44:0/1247623750 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff610016070 con 0x564c2ae4dcc0 2023-09-10T01:02:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.625+0000 7ff5fd7fa700 1 -- 172.21.15.44:0/1247623750 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff604004050 con 0x564c2ae4dcc0 2023-09-10T01:02:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.625+0000 7ff627caab80 10 client.28112 fetch_fsmap learned FSMap version 87 2023-09-10T01:02:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.625+0000 7ff627caab80 1 -- 172.21.15.44:0/1247623750 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x564c2abf21f0 con 0x564c2ae4dcc0 2023-09-10T01:02:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.625+0000 7ff5fd7fa700 1 -- 172.21.15.44:0/1247623750 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff610016070 con 0x564c2ae4dcc0 2023-09-10T01:02:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.625+0000 7ff5fd7fa700 1 -- 172.21.15.44:0/1247623750 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 87) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff604010430 con 0x564c2ae4dcc0 2023-09-10T01:02:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.625+0000 7ff627caab80 10 client.28112 fetch_fsmap finished waiting for FSMap version 87 2023-09-10T01:02:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.625+0000 7ff627caab80 10 client.28112 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:02:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.625+0000 7ff627caab80 20 client.28112 populate_metadata read hostname 'smithi044' 2023-09-10T01:02:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.625+0000 7ff627caab80 1 --2- 172.21.15.44:0/1247623750 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564c2aebc6a0 0x564c2aebeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.625+0000 7ff627caab80 4 client.28112 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:02:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.625+0000 7ff627caab80 1 -- 172.21.15.44:0/1247623750 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564c2ad988c0 con 0x564c2aebc6a0 2023-09-10T01:02:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.626+0000 7ff619880700 1 --2- 172.21.15.44:0/1247623750 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564c2aebc6a0 0x564c2aebeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.626+0000 7ff5fd7fa700 10 client.28112 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:02:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.626+0000 7ff619880700 1 --2- 172.21.15.44:0/1247623750 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564c2aebc6a0 0x564c2aebeac0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc87096458da973f server_cookie=9ca6d40d26bea0a2 in_seq=0 out_seq=0 2023-09-10T01:02:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.644+0000 7ff5fd7fa700 1 -- 172.21.15.44:0/1247623750 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564c2ad988c0 con 0x564c2aebc6a0 2023-09-10T01:02:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.644+0000 7ff5fd7fa700 10 client.28112 handle_command_reply: tid=0 2023-09-10T01:02:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.644+0000 7ff627caab80 1 client.28112 shutdown 2023-09-10T01:02:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.644+0000 7ff627caab80 1 -- 172.21.15.44:0/1247623750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae4dcc0 msgr2=0x564c2aeb4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.644+0000 7ff627caab80 1 --2- 172.21.15.44:0/1247623750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae4dcc0 0x564c2aeb4bb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff604012d20 tx=0x7ff60400f6a0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff627caab80 1 -- 172.21.15.44:0/1247623750 shutdown_connections 2023-09-10T01:02:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff627caab80 1 --2- 172.21.15.44:0/1247623750 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564c2aebc6a0 0x564c2aebeac0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff5fd7fa700 0 client.28112 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:02:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff627caab80 1 --2- 172.21.15.44:0/1247623750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c2ae531a0 0x564c2aeaf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff627caab80 1 --2- 172.21.15.44:0/1247623750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c2ae4d300 0x564c2aeb4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff627caab80 1 --2- 172.21.15.44:0/1247623750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c2ae4dcc0 0x564c2aeb4bb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff627caab80 1 -- 172.21.15.44:0/1247623750 >> 172.21.15.44:0/1247623750 conn(0x564c2adc2030 msgr2=0x564c2ae4c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff627caab80 1 -- 172.21.15.44:0/1247623750 shutdown_connections 2023-09-10T01:02:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff627caab80 1 -- 172.21.15.44:0/1247623750 wait complete. 2023-09-10T01:02:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff627caab80 20 client.28112 trim_cache size 0 max 16384 2023-09-10T01:02:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:02:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:02:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:02:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:02:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff61bae4700 1 -- 172.21.15.44:0/1709846450 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff6000a07b0 msgr2=0x7ff6000a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff61bae4700 1 --2- 172.21.15.44:0/1709846450 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff6000a07b0 0x7ff6000a2c70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff610009bf0 tx=0x7ff610009260 comp rx=0 tx=0).stop 2023-09-10T01:02:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff61bae4700 1 -- 172.21.15.44:0/1709846450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff614105590 msgr2=0x7ff6141a24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff61bae4700 1 --2- 172.21.15.44:0/1709846450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff614105590 0x7ff6141a24b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00b6c0 tx=0x7ff60c008aa0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff61bae4700 1 -- 172.21.15.44:0/1709846450 shutdown_connections 2023-09-10T01:02:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff61bae4700 1 --2- 172.21.15.44:0/1709846450 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff6000a07b0 0x7ff6000a2c70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff61bae4700 1 --2- 172.21.15.44:0/1709846450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff614105590 0x7ff6141a24b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff61bae4700 1 --2- 172.21.15.44:0/1709846450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6140ff4d0 0x7ff614197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff61bae4700 1 --2- 172.21.15.44:0/1709846450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6140ffe90 0x7ff614197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.645+0000 7ff61bae4700 1 -- 172.21.15.44:0/1709846450 >> 172.21.15.44:0/1709846450 conn(0x7ff6140faf80 msgr2=0x7ff6141031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.646+0000 7ff61bae4700 1 -- 172.21.15.44:0/1709846450 shutdown_connections 2023-09-10T01:02:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:22.646+0000 7ff61bae4700 1 -- 172.21.15.44:0/1709846450 wait complete. 2023-09-10T01:02:23.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:22 smithi044 ceph-mon[98676]: pgmap v688: 129 pgs: 129 active+clean; 8.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 44 MiB/s wr, 46 op/s 2023-09-10T01:02:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:22 smithi099 ceph-mon[102692]: pgmap v688: 129 pgs: 129 active+clean; 8.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 44 MiB/s wr, 46 op/s 2023-09-10T01:02:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:22 smithi131 ceph-mon[103713]: pgmap v688: 129 pgs: 129 active+clean; 8.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 44 MiB/s wr, 46 op/s 2023-09-10T01:02:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:24 smithi099 ceph-mon[102692]: pgmap v689: 129 pgs: 129 active+clean; 8.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 44 MiB/s wr, 46 op/s 2023-09-10T01:02:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:24 smithi131 ceph-mon[103713]: pgmap v689: 129 pgs: 129 active+clean; 8.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 44 MiB/s wr, 46 op/s 2023-09-10T01:02:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:24 smithi044 ceph-mon[98676]: pgmap v689: 129 pgs: 129 active+clean; 8.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 44 MiB/s wr, 46 op/s 2023-09-10T01:02:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:26 smithi099 ceph-mon[102692]: pgmap v690: 129 pgs: 129 active+clean; 8.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 44 MiB/s wr, 44 op/s 2023-09-10T01:02:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:26 smithi131 ceph-mon[103713]: pgmap v690: 129 pgs: 129 active+clean; 8.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 44 MiB/s wr, 44 op/s 2023-09-10T01:02:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:26 smithi044 ceph-mon[98676]: pgmap v690: 129 pgs: 129 active+clean; 8.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 44 MiB/s wr, 44 op/s 2023-09-10T01:02:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:28 smithi099 ceph-mon[102692]: pgmap v691: 129 pgs: 129 active+clean; 8.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 38 MiB/s wr, 38 op/s 2023-09-10T01:02:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:28 smithi131 ceph-mon[103713]: pgmap v691: 129 pgs: 129 active+clean; 8.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 38 MiB/s wr, 38 op/s 2023-09-10T01:02:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:28 smithi044 ceph-mon[98676]: pgmap v691: 129 pgs: 129 active+clean; 8.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 38 MiB/s wr, 38 op/s 2023-09-10T01:02:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:30 smithi099 ceph-mon[102692]: pgmap v692: 129 pgs: 129 active+clean; 8.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 37 MiB/s wr, 36 op/s 2023-09-10T01:02:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:30 smithi131 ceph-mon[103713]: pgmap v692: 129 pgs: 129 active+clean; 8.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 37 MiB/s wr, 36 op/s 2023-09-10T01:02:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:30 smithi044 ceph-mon[98676]: pgmap v692: 129 pgs: 129 active+clean; 8.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 37 MiB/s wr, 36 op/s 2023-09-10T01:02:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:32 smithi099 ceph-mon[102692]: pgmap v693: 129 pgs: 129 active+clean; 8.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 55 MiB/s wr, 54 op/s 2023-09-10T01:02:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:32 smithi131 ceph-mon[103713]: pgmap v693: 129 pgs: 129 active+clean; 8.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 55 MiB/s wr, 54 op/s 2023-09-10T01:02:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:32 smithi044 ceph-mon[98676]: pgmap v693: 129 pgs: 129 active+clean; 8.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 55 MiB/s wr, 54 op/s 2023-09-10T01:02:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:34 smithi099 ceph-mon[102692]: pgmap v694: 129 pgs: 129 active+clean; 8.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 37 MiB/s wr, 37 op/s 2023-09-10T01:02:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:34 smithi131 ceph-mon[103713]: pgmap v694: 129 pgs: 129 active+clean; 8.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 37 MiB/s wr, 37 op/s 2023-09-10T01:02:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:34 smithi044 ceph-mon[98676]: pgmap v694: 129 pgs: 129 active+clean; 8.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 37 MiB/s wr, 37 op/s 2023-09-10T01:02:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.789+0000 7f044a647700 1 -- 172.21.15.44:0/4262995558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0444106650 msgr2=0x7f0444106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.789+0000 7f044a647700 1 --2- 172.21.15.44:0/4262995558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0444106650 0x7f0444106a30 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f04340099c0 tx=0x7f0434009cd0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.789+0000 7f044a647700 1 -- 172.21.15.44:0/4262995558 shutdown_connections 2023-09-10T01:02:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.789+0000 7f044a647700 1 --2- 172.21.15.44:0/4262995558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04440683b0 0x7f044410d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.789+0000 7f044a647700 1 --2- 172.21.15.44:0/4262995558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0444106650 0x7f0444106a30 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.789+0000 7f044a647700 1 --2- 172.21.15.44:0/4262995558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04440679f0 0x7f0444067e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.789+0000 7f044a647700 1 -- 172.21.15.44:0/4262995558 >> 172.21.15.44:0/4262995558 conn(0x7f0444074f30 msgr2=0x7f0444075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.789+0000 7f044a647700 1 -- 172.21.15.44:0/4262995558 shutdown_connections 2023-09-10T01:02:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.789+0000 7f044a647700 1 -- 172.21.15.44:0/4262995558 wait complete. 2023-09-10T01:02:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.790+0000 7f044a647700 1 Processor -- start 2023-09-10T01:02:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.790+0000 7f044a647700 1 -- start start 2023-09-10T01:02:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f044a647700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04440679f0 0x7f044419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f044a647700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04440683b0 0x7f044419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f044a647700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0444106650 0x7f0444196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f044a647700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04441969e0 con 0x7f0444106650 2023-09-10T01:02:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f044a647700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0444196b50 con 0x7f04440679f0 2023-09-10T01:02:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f044a647700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0444196d10 con 0x7f04440683b0 2023-09-10T01:02:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f0448be4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0444106650 0x7f0444196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f0448be4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0444106650 0x7f0444196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55058/0 (socket says 172.21.15.44:55058) 2023-09-10T01:02:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f0448be4700 1 -- 172.21.15.44:0/206510985 learned_addr learned my addr 172.21.15.44:0/206510985 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f0443fff700 1 --2- 172.21.15.44:0/206510985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04440679f0 0x7f044419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f04437fe700 1 --2- 172.21.15.44:0/206510985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04440683b0 0x7f044419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f0443fff700 1 -- 172.21.15.44:0/206510985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04440683b0 msgr2=0x7f044419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f0443fff700 1 --2- 172.21.15.44:0/206510985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04440683b0 0x7f044419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f0443fff700 1 -- 172.21.15.44:0/206510985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0444106650 msgr2=0x7f0444196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f0443fff700 1 --2- 172.21.15.44:0/206510985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0444106650 0x7f0444196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f0443fff700 1 -- 172.21.15.44:0/206510985 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04340095f0 con 0x7f04440679f0 2023-09-10T01:02:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f0448be4700 1 --2- 172.21.15.44:0/206510985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0444106650 0x7f0444196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:02:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f0443fff700 1 --2- 172.21.15.44:0/206510985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04440679f0 0x7f044419c510 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0434000c00 tx=0x7f043400b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.791+0000 7f04417fa700 1 -- 172.21.15.44:0/206510985 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0434021070 con 0x7f04440679f0 2023-09-10T01:02:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.792+0000 7f04417fa700 1 -- 172.21.15.44:0/206510985 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04340059a0 con 0x7f04440679f0 2023-09-10T01:02:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.792+0000 7f044a647700 1 -- 172.21.15.44:0/206510985 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0444196f90 con 0x7f04440679f0 2023-09-10T01:02:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.792+0000 7f04417fa700 1 -- 172.21.15.44:0/206510985 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0434004a70 con 0x7f04440679f0 2023-09-10T01:02:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.792+0000 7f044a647700 1 -- 172.21.15.44:0/206510985 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04440ff940 con 0x7f04440679f0 2023-09-10T01:02:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.794+0000 7f04417fa700 1 -- 172.21.15.44:0/206510985 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f0434025070 con 0x7f04440679f0 2023-09-10T01:02:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.794+0000 7f04417fa700 1 --2- 172.21.15.44:0/206510985 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f042c0a0830 0x7f042c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.795+0000 7f04437fe700 1 --2- 172.21.15.44:0/206510985 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f042c0a0830 0x7f042c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.795+0000 7f04417fa700 1 -- 172.21.15.44:0/206510985 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f04340d1120 con 0x7f04440679f0 2023-09-10T01:02:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.795+0000 7f04437fe700 1 --2- 172.21.15.44:0/206510985 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f042c0a0830 0x7f042c0a2cf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f04440fdee0 tx=0x7f0438009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.796+0000 7f045680db80 1 Processor -- start 2023-09-10T01:02:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.796+0000 7f045680db80 1 -- start start 2023-09-10T01:02:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.796+0000 7f045680db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe988391a0 0x55fe98839580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.796+0000 7f045680db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe98833300 0x55fe98833780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.796+0000 7f045680db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833cc0 0x55fe9883ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.796+0000 7f045680db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9878c6d0 con 0x55fe98833cc0 2023-09-10T01:02:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.796+0000 7f045680db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe98789ff0 con 0x55fe98833300 2023-09-10T01:02:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.796+0000 7f045680db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe987858e0 con 0x55fe988391a0 2023-09-10T01:02:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.796+0000 7f0448be4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833cc0 0x55fe9883ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.796+0000 7f0443fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe988391a0 0x55fe98839580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.796+0000 7f0448be4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833cc0 0x55fe9883ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55068/0 (socket says 172.21.15.44:55068) 2023-09-10T01:02:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.796+0000 7f0443fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe988391a0 0x55fe98839580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33714/0 (socket says 172.21.15.44:33714) 2023-09-10T01:02:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.796+0000 7f0448be4700 1 -- 172.21.15.44:0/2969835178 learned_addr learned my addr 172.21.15.44:0/2969835178 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.796+0000 7f04437fe700 1 --2- 172.21.15.44:0/2969835178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe98833300 0x55fe98833780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.797+0000 7f0448be4700 1 -- 172.21.15.44:0/2969835178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe988391a0 msgr2=0x55fe98839580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.797+0000 7f0448be4700 1 --2- 172.21.15.44:0/2969835178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe988391a0 0x55fe98839580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.797+0000 7f0448be4700 1 -- 172.21.15.44:0/2969835178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe98833300 msgr2=0x55fe98833780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.797+0000 7f0448be4700 1 --2- 172.21.15.44:0/2969835178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe98833300 0x55fe98833780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.797+0000 7f0448be4700 1 -- 172.21.15.44:0/2969835178 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe985b1d80 con 0x55fe98833cc0 2023-09-10T01:02:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.797+0000 7f04437fe700 1 --2- 172.21.15.44:0/2969835178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe98833300 0x55fe98833780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:02:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.797+0000 7f0448be4700 1 --2- 172.21.15.44:0/2969835178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833cc0 0x55fe9883ff30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f043c0094c0 tx=0x7f043c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb637ee0b4b2ef82 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.797+0000 7f042a7fc700 1 -- 172.21.15.44:0/2969835178 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c004db0 con 0x55fe98833cc0 2023-09-10T01:02:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.797+0000 7f042a7fc700 1 -- 172.21.15.44:0/2969835178 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f043c00f040 con 0x55fe98833cc0 2023-09-10T01:02:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.797+0000 7f042a7fc700 1 -- 172.21.15.44:0/2969835178 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c005480 con 0x55fe98833cc0 2023-09-10T01:02:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.798+0000 7f045680db80 1 -- 172.21.15.44:0/2969835178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833cc0 msgr2=0x55fe9883ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.798+0000 7f045680db80 1 --2- 172.21.15.44:0/2969835178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833cc0 0x55fe9883ff30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f043c0094c0 tx=0x7f043c009920 comp rx=0 tx=0).stop 2023-09-10T01:02:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.798+0000 7f045680db80 1 -- 172.21.15.44:0/2969835178 shutdown_connections 2023-09-10T01:02:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.798+0000 7f045680db80 1 --2- 172.21.15.44:0/2969835178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833cc0 0x55fe9883ff30 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.798+0000 7f045680db80 1 --2- 172.21.15.44:0/2969835178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe988391a0 0x55fe98839580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.798+0000 7f045680db80 1 --2- 172.21.15.44:0/2969835178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe98833300 0x55fe98833780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.798+0000 7f045680db80 1 -- 172.21.15.44:0/2969835178 >> 172.21.15.44:0/2969835178 conn(0x55fe987a8030 msgr2=0x55fe9882e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.798+0000 7f045680db80 1 -- 172.21.15.44:0/2969835178 shutdown_connections 2023-09-10T01:02:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.798+0000 7f045680db80 1 -- 172.21.15.44:0/2969835178 wait complete. 2023-09-10T01:02:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.798+0000 7f045680db80 1 -- start start 2023-09-10T01:02:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f045680db80 1 Processor -- start 2023-09-10T01:02:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f045680db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe98833300 0x55fe9883fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f045680db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe98833cc0 0x55fe9883d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f045680db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe988391a0 0x55fe9883d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f045680db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe98723b10 con 0x55fe988391a0 2023-09-10T01:02:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f045680db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe98723810 con 0x55fe98833cc0 2023-09-10T01:02:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f045680db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe986078c0 con 0x55fe98833300 2023-09-10T01:02:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f045680db80 10 client.0 _ll_register_callbacks cb 0x55fe9866da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:02:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f0443fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe98833300 0x55fe9883fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f0448be4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe988391a0 0x55fe9883d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f0443fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe98833300 0x55fe9883fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33720/0 (socket says 172.21.15.44:33720) 2023-09-10T01:02:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f0443fff700 1 -- 172.21.15.44:0/59320399 learned_addr learned my addr 172.21.15.44:0/59320399 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f0448be4700 1 --2- 172.21.15.44:0/59320399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe988391a0 0x55fe9883d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55072/0 (socket says 172.21.15.44:55072) 2023-09-10T01:02:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f04437fe700 1 --2- 172.21.15.44:0/59320399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe98833cc0 0x55fe9883d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f0443fff700 1 -- 172.21.15.44:0/59320399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe98833cc0 msgr2=0x55fe9883d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f0443fff700 1 --2- 172.21.15.44:0/59320399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe98833cc0 0x55fe9883d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f0443fff700 1 -- 172.21.15.44:0/59320399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe988391a0 msgr2=0x55fe9883d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f0443fff700 1 --2- 172.21.15.44:0/59320399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe988391a0 0x55fe9883d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.799+0000 7f0443fff700 1 -- 172.21.15.44:0/59320399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0438017070 con 0x55fe98833300 2023-09-10T01:02:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.800+0000 7f0443fff700 1 --2- 172.21.15.44:0/59320399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe98833300 0x55fe9883fe30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f04340ce520 tx=0x7f043409dca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=608f4339b8d18669 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.800+0000 7f04237fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:02:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.800+0000 7f04237fe700 1 -- 172.21.15.44:0/59320399 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04340a42b0 con 0x55fe98833300 2023-09-10T01:02:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.800+0000 7f04237fe700 1 -- 172.21.15.44:0/59320399 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f043409e340 con 0x55fe98833300 2023-09-10T01:02:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.800+0000 7f045680db80 1 -- 172.21.15.44:0/59320399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe986dd490 con 0x55fe98833300 2023-09-10T01:02:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.800+0000 7f04237fe700 1 -- 172.21.15.44:0/59320399 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04340a2170 con 0x55fe98833300 2023-09-10T01:02:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.800+0000 7f045680db80 10 client.28060 fetch_fsmap learned FSMap version 87 2023-09-10T01:02:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.800+0000 7f045680db80 1 -- 172.21.15.44:0/59320399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x7f043c009170 con 0x55fe98833300 2023-09-10T01:02:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.800+0000 7f04237fe700 1 -- 172.21.15.44:0/59320399 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04340bc030 con 0x55fe98833300 2023-09-10T01:02:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.800+0000 7f04237fe700 1 -- 172.21.15.44:0/59320399 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 87) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f043409e5a0 con 0x55fe98833300 2023-09-10T01:02:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.800+0000 7f045680db80 10 client.28060 fetch_fsmap finished waiting for FSMap version 87 2023-09-10T01:02:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.800+0000 7f045680db80 10 client.28060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:02:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.800+0000 7f045680db80 20 client.28060 populate_metadata read hostname 'smithi044' 2023-09-10T01:02:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.800+0000 7f045680db80 1 --2- 172.21.15.44:0/59320399 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe9884d0e0 0x55fe9884f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.800+0000 7f045680db80 4 client.28060 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:02:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.800+0000 7f045680db80 1 -- 172.21.15.44:0/59320399 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe9879a840 con 0x55fe9884d0e0 2023-09-10T01:02:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.801+0000 7f04437fe700 1 --2- 172.21.15.44:0/59320399 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe9884d0e0 0x55fe9884f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.801+0000 7f04237fe700 10 client.28060 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:02:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.801+0000 7f04437fe700 1 --2- 172.21.15.44:0/59320399 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe9884d0e0 0x55fe9884f4d0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4712843c05063cd8 server_cookie=ce6a177cf4b3255a in_seq=0 out_seq=0 2023-09-10T01:02:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.802+0000 7f04237fe700 1 -- 172.21.15.44:0/59320399 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fe9879a840 con 0x55fe9884d0e0 2023-09-10T01:02:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.802+0000 7f04237fe700 10 client.28060 handle_command_reply: tid=0 2023-09-10T01:02:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.802+0000 7f045680db80 1 client.28060 shutdown 2023-09-10T01:02:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.802+0000 7f045680db80 1 -- 172.21.15.44:0/59320399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe98833300 msgr2=0x55fe9883fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.802+0000 7f045680db80 1 --2- 172.21.15.44:0/59320399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe98833300 0x55fe9883fe30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f04340ce520 tx=0x7f043409dca0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.802+0000 7f045680db80 1 -- 172.21.15.44:0/59320399 shutdown_connections 2023-09-10T01:02:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.802+0000 7f045680db80 1 --2- 172.21.15.44:0/59320399 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe9884d0e0 0x55fe9884f4d0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.802+0000 7f045680db80 1 --2- 172.21.15.44:0/59320399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe988391a0 0x55fe9883d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.802+0000 7f045680db80 1 --2- 172.21.15.44:0/59320399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe98833300 0x55fe9883fe30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.802+0000 7f04237fe700 0 client.28060 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:02:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.802+0000 7f045680db80 1 --2- 172.21.15.44:0/59320399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe98833cc0 0x55fe9883d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.802+0000 7f045680db80 1 -- 172.21.15.44:0/59320399 >> 172.21.15.44:0/59320399 conn(0x55fe987a8030 msgr2=0x55fe9882d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.802+0000 7f045680db80 1 -- 172.21.15.44:0/59320399 shutdown_connections 2023-09-10T01:02:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.802+0000 7f045680db80 1 -- 172.21.15.44:0/59320399 wait complete. 2023-09-10T01:02:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.802+0000 7f045680db80 20 client.28060 trim_cache size 0 max 16384 2023-09-10T01:02:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.818+0000 7f045680db80 1 Processor -- start 2023-09-10T01:02:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.818+0000 7f045680db80 1 -- start start 2023-09-10T01:02:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.818+0000 7f045680db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833300 0x55fe98844530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.818+0000 7f045680db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe98833cc0 0x55fe9883f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.818+0000 7f045680db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe988391a0 0x55fe9882e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.818+0000 7f045680db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe98838ab0 con 0x55fe98833300 2023-09-10T01:02:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.818+0000 7f045680db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe98844a70 con 0x55fe988391a0 2023-09-10T01:02:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.818+0000 7f045680db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe988340a0 con 0x55fe98833cc0 2023-09-10T01:02:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.819+0000 7f0448be4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe988391a0 0x55fe9882e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.819+0000 7f0443fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833300 0x55fe98844530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.819+0000 7f0448be4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe988391a0 0x55fe9882e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56822/0 (socket says 172.21.15.44:56822) 2023-09-10T01:02:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.819+0000 7f0448be4700 1 -- 172.21.15.44:0/1429152516 learned_addr learned my addr 172.21.15.44:0/1429152516 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.819+0000 7f0443fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833300 0x55fe98844530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55078/0 (socket says 172.21.15.44:55078) 2023-09-10T01:02:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.819+0000 7f04437fe700 1 --2- 172.21.15.44:0/1429152516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe98833cc0 0x55fe9883f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.819+0000 7f0448be4700 1 -- 172.21.15.44:0/1429152516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe98833cc0 msgr2=0x55fe9883f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.819+0000 7f0448be4700 1 --2- 172.21.15.44:0/1429152516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe98833cc0 0x55fe9883f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.819+0000 7f0448be4700 1 -- 172.21.15.44:0/1429152516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833300 msgr2=0x55fe98844530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.819+0000 7f0448be4700 1 --2- 172.21.15.44:0/1429152516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833300 0x55fe98844530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.819+0000 7f0448be4700 1 -- 172.21.15.44:0/1429152516 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe9859ac00 con 0x55fe988391a0 2023-09-10T01:02:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.819+0000 7f0448be4700 1 --2- 172.21.15.44:0/1429152516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe988391a0 0x55fe9882e090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f043c009460 tx=0x7f043c007500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6581bc90bb34f74 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.821+0000 7f042affd700 1 -- 172.21.15.44:0/1429152516 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c022040 con 0x55fe988391a0 2023-09-10T01:02:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.821+0000 7f042affd700 1 -- 172.21.15.44:0/1429152516 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f043c003c00 con 0x55fe988391a0 2023-09-10T01:02:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.821+0000 7f042affd700 1 -- 172.21.15.44:0/1429152516 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c0052d0 con 0x55fe988391a0 2023-09-10T01:02:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.822+0000 7f045680db80 1 -- 172.21.15.44:0/1429152516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe988391a0 msgr2=0x55fe9882e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.822+0000 7f045680db80 1 --2- 172.21.15.44:0/1429152516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe988391a0 0x55fe9882e090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f043c009460 tx=0x7f043c007500 comp rx=0 tx=0).stop 2023-09-10T01:02:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.822+0000 7f045680db80 1 -- 172.21.15.44:0/1429152516 shutdown_connections 2023-09-10T01:02:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.822+0000 7f045680db80 1 --2- 172.21.15.44:0/1429152516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe988391a0 0x55fe9882e090 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.822+0000 7f045680db80 1 --2- 172.21.15.44:0/1429152516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833300 0x55fe98844530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.822+0000 7f045680db80 1 --2- 172.21.15.44:0/1429152516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe98833cc0 0x55fe9883f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.822+0000 7f045680db80 1 -- 172.21.15.44:0/1429152516 >> 172.21.15.44:0/1429152516 conn(0x55fe987a8030 msgr2=0x55fe98837330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.822+0000 7f045680db80 1 -- 172.21.15.44:0/1429152516 shutdown_connections 2023-09-10T01:02:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.822+0000 7f045680db80 1 -- 172.21.15.44:0/1429152516 wait complete. 2023-09-10T01:02:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.822+0000 7f045680db80 1 -- start start 2023-09-10T01:02:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.823+0000 7f045680db80 1 Processor -- start 2023-09-10T01:02:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.823+0000 7f045680db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe98833300 0x55fe9889a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.823+0000 7f045680db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833cc0 0x55fe9889ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.823+0000 7f045680db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe988391a0 0x55fe98894e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.823+0000 7f045680db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe98895480 con 0x55fe98833cc0 2023-09-10T01:02:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.823+0000 7f045680db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe988955f0 con 0x55fe988391a0 2023-09-10T01:02:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.823+0000 7f045680db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe988957b0 con 0x55fe98833300 2023-09-10T01:02:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.823+0000 7f045680db80 10 client.0 _ll_register_callbacks cb 0x7f043409e1a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:02:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.824+0000 7f0448be4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe988391a0 0x55fe98894e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.824+0000 7f0448be4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe988391a0 0x55fe98894e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56824/0 (socket says 172.21.15.44:56824) 2023-09-10T01:02:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.824+0000 7f0448be4700 1 -- 172.21.15.44:0/1714684974 learned_addr learned my addr 172.21.15.44:0/1714684974 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.824+0000 7f0448be4700 1 -- 172.21.15.44:0/1714684974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe98833300 msgr2=0x55fe9889a740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:02:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.824+0000 7f0448be4700 1 --2- 172.21.15.44:0/1714684974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe98833300 0x55fe9889a740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.824+0000 7f0448be4700 1 -- 172.21.15.44:0/1714684974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833cc0 msgr2=0x55fe9889ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.824+0000 7f0448be4700 1 --2- 172.21.15.44:0/1714684974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833cc0 0x55fe9889ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.824+0000 7f0448be4700 1 -- 172.21.15.44:0/1714684974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe985b1d80 con 0x55fe988391a0 2023-09-10T01:02:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.824+0000 7f0448be4700 1 --2- 172.21.15.44:0/1714684974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe988391a0 0x55fe98894e70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f043c009ed0 tx=0x7f043c003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a92e916bbbe2c64d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.824+0000 7f04237fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:02:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.824+0000 7f04237fe700 1 -- 172.21.15.44:0/1714684974 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c022040 con 0x55fe988391a0 2023-09-10T01:02:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.824+0000 7f04237fe700 1 -- 172.21.15.44:0/1714684974 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f043c005110 con 0x55fe988391a0 2023-09-10T01:02:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.824+0000 7f045680db80 1 -- 172.21.15.44:0/1714684974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f043c00f040 con 0x55fe988391a0 2023-09-10T01:02:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.831+0000 7f04237fe700 1 -- 172.21.15.44:0/1714684974 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f043c0052f0 con 0x55fe988391a0 2023-09-10T01:02:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.831+0000 7f045680db80 10 client.28130 fetch_fsmap learned FSMap version 87 2023-09-10T01:02:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.831+0000 7f045680db80 1 -- 172.21.15.44:0/1714684974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x55fe985d81f0 con 0x55fe988391a0 2023-09-10T01:02:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.831+0000 7f04237fe700 1 -- 172.21.15.44:0/1714684974 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f043c00f040 con 0x55fe988391a0 2023-09-10T01:02:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.831+0000 7f04237fe700 1 -- 172.21.15.44:0/1714684974 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 87) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f043c027b00 con 0x55fe988391a0 2023-09-10T01:02:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.831+0000 7f045680db80 10 client.28130 fetch_fsmap finished waiting for FSMap version 87 2023-09-10T01:02:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.831+0000 7f045680db80 10 client.28130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:02:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.831+0000 7f045680db80 20 client.28130 populate_metadata read hostname 'smithi044' 2023-09-10T01:02:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.831+0000 7f045680db80 1 --2- 172.21.15.44:0/1714684974 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe988a2140 0x55fe988a4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.831+0000 7f045680db80 4 client.28130 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:02:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.831+0000 7f045680db80 1 -- 172.21.15.44:0/1714684974 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fe9877e8c0 con 0x55fe988a2140 2023-09-10T01:02:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.832+0000 7f0443fff700 1 --2- 172.21.15.44:0/1714684974 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe988a2140 0x55fe988a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.833+0000 7f0443fff700 1 --2- 172.21.15.44:0/1714684974 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe988a2140 0x55fe988a4530 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5bada8585d938df server_cookie=1e88670b20290ccc in_seq=0 out_seq=0 2023-09-10T01:02:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.833+0000 7f04237fe700 10 client.28130 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:02:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.857+0000 7f04237fe700 1 -- 172.21.15.44:0/1714684974 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fe9877e8c0 con 0x55fe988a2140 2023-09-10T01:02:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.857+0000 7f04237fe700 10 client.28130 handle_command_reply: tid=0 2023-09-10T01:02:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.857+0000 7f045680db80 1 client.28130 shutdown 2023-09-10T01:02:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.857+0000 7f045680db80 1 -- 172.21.15.44:0/1714684974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe988391a0 msgr2=0x55fe98894e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.857+0000 7f045680db80 1 --2- 172.21.15.44:0/1714684974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe988391a0 0x55fe98894e70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f043c009ed0 tx=0x7f043c003b20 comp rx=0 tx=0).stop 2023-09-10T01:02:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.857+0000 7f045680db80 1 -- 172.21.15.44:0/1714684974 shutdown_connections 2023-09-10T01:02:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.857+0000 7f045680db80 1 --2- 172.21.15.44:0/1714684974 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe988a2140 0x55fe988a4530 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.857+0000 7f045680db80 1 --2- 172.21.15.44:0/1714684974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe988391a0 0x55fe98894e70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.857+0000 7f045680db80 1 --2- 172.21.15.44:0/1714684974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe98833300 0x55fe9889a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.857+0000 7f04237fe700 0 client.28130 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:02:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.857+0000 7f045680db80 1 --2- 172.21.15.44:0/1714684974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe98833cc0 0x55fe9889ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.857+0000 7f045680db80 1 -- 172.21.15.44:0/1714684974 >> 172.21.15.44:0/1714684974 conn(0x55fe987a8030 msgr2=0x55fe9882e090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.857+0000 7f045680db80 1 -- 172.21.15.44:0/1714684974 shutdown_connections 2023-09-10T01:02:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.857+0000 7f045680db80 1 -- 172.21.15.44:0/1714684974 wait complete. 2023-09-10T01:02:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.857+0000 7f045680db80 20 client.28130 trim_cache size 0 max 16384 2023-09-10T01:02:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:02:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:02:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:02:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:02:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.858+0000 7f044a647700 1 -- 172.21.15.44:0/206510985 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f042c0a0830 msgr2=0x7f042c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.858+0000 7f044a647700 1 --2- 172.21.15.44:0/206510985 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f042c0a0830 0x7f042c0a2cf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f04440fdee0 tx=0x7f0438009260 comp rx=0 tx=0).stop 2023-09-10T01:02:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.858+0000 7f044a647700 1 -- 172.21.15.44:0/206510985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04440679f0 msgr2=0x7f044419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.858+0000 7f044a647700 1 --2- 172.21.15.44:0/206510985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04440679f0 0x7f044419c510 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0434000c00 tx=0x7f043400b9c0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.858+0000 7f044a647700 1 -- 172.21.15.44:0/206510985 shutdown_connections 2023-09-10T01:02:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.858+0000 7f044a647700 1 --2- 172.21.15.44:0/206510985 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f042c0a0830 0x7f042c0a2cf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.858+0000 7f044a647700 1 --2- 172.21.15.44:0/206510985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0444106650 0x7f0444196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.858+0000 7f044a647700 1 --2- 172.21.15.44:0/206510985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04440679f0 0x7f044419c510 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.858+0000 7f044a647700 1 --2- 172.21.15.44:0/206510985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04440683b0 0x7f044419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.858+0000 7f044a647700 1 -- 172.21.15.44:0/206510985 >> 172.21.15.44:0/206510985 conn(0x7f0444074f30 msgr2=0x7f04440fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.858+0000 7f044a647700 1 -- 172.21.15.44:0/206510985 shutdown_connections 2023-09-10T01:02:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:35.858+0000 7f044a647700 1 -- 172.21.15.44:0/206510985 wait complete. 2023-09-10T01:02:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:36 smithi099 ceph-mon[102692]: pgmap v695: 129 pgs: 129 active+clean; 9.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 55 MiB/s wr, 54 op/s 2023-09-10T01:02:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:36 smithi131 ceph-mon[103713]: pgmap v695: 129 pgs: 129 active+clean; 9.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 55 MiB/s wr, 54 op/s 2023-09-10T01:02:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:36 smithi044 ceph-mon[98676]: pgmap v695: 129 pgs: 129 active+clean; 9.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 55 MiB/s wr, 54 op/s 2023-09-10T01:02:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:38 smithi099 ceph-mon[102692]: pgmap v696: 129 pgs: 129 active+clean; 9.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 38 MiB/s wr, 35 op/s 2023-09-10T01:02:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:38 smithi131 ceph-mon[103713]: pgmap v696: 129 pgs: 129 active+clean; 9.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 38 MiB/s wr, 35 op/s 2023-09-10T01:02:39.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:38 smithi044 ceph-mon[98676]: pgmap v696: 129 pgs: 129 active+clean; 9.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 38 MiB/s wr, 35 op/s 2023-09-10T01:02:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:40 smithi099 ceph-mon[102692]: pgmap v697: 129 pgs: 129 active+clean; 9.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 38 MiB/s wr, 35 op/s 2023-09-10T01:02:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:40 smithi131 ceph-mon[103713]: pgmap v697: 129 pgs: 129 active+clean; 9.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 38 MiB/s wr, 35 op/s 2023-09-10T01:02:41.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:40 smithi044 ceph-mon[98676]: pgmap v697: 129 pgs: 129 active+clean; 9.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 38 MiB/s wr, 35 op/s 2023-09-10T01:02:43.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:42 smithi099 ceph-mon[102692]: pgmap v698: 129 pgs: 129 active+clean; 9.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 56 MiB/s wr, 55 op/s 2023-09-10T01:02:43.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:42 smithi131 ceph-mon[103713]: pgmap v698: 129 pgs: 129 active+clean; 9.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 56 MiB/s wr, 55 op/s 2023-09-10T01:02:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:42 smithi044 ceph-mon[98676]: pgmap v698: 129 pgs: 129 active+clean; 9.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 56 MiB/s wr, 55 op/s 2023-09-10T01:02:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:44 smithi099 ceph-mon[102692]: pgmap v699: 129 pgs: 129 active+clean; 9.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 37 MiB/s wr, 37 op/s 2023-09-10T01:02:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:44 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:02:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:44 smithi131 ceph-mon[103713]: pgmap v699: 129 pgs: 129 active+clean; 9.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 37 MiB/s wr, 37 op/s 2023-09-10T01:02:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:44 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:02:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:44 smithi044 ceph-mon[98676]: pgmap v699: 129 pgs: 129 active+clean; 9.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 37 MiB/s wr, 37 op/s 2023-09-10T01:02:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:44 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:02:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:46 smithi099 ceph-mon[102692]: pgmap v700: 129 pgs: 129 active+clean; 9.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 56 MiB/s wr, 52 op/s 2023-09-10T01:02:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:46 smithi131 ceph-mon[103713]: pgmap v700: 129 pgs: 129 active+clean; 9.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 56 MiB/s wr, 52 op/s 2023-09-10T01:02:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:46 smithi044 ceph-mon[98676]: pgmap v700: 129 pgs: 129 active+clean; 9.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 56 MiB/s wr, 52 op/s 2023-09-10T01:02:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.995+0000 7fa7d115a700 1 -- 172.21.15.44:0/8385547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7cc100120 msgr2=0x7fa7cc10c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.995+0000 7fa7d115a700 1 --2- 172.21.15.44:0/8385547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7cc100120 0x7fa7cc10c330 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400b0d0 tx=0x7fa7c400b3e0 comp rx=0 tx=0).stop 2023-09-10T01:02:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.995+0000 7fa7d115a700 1 -- 172.21.15.44:0/8385547 shutdown_connections 2023-09-10T01:02:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.995+0000 7fa7d115a700 1 --2- 172.21.15.44:0/8385547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7cc100120 0x7fa7cc10c330 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.995+0000 7fa7d115a700 1 --2- 172.21.15.44:0/8385547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7cc1055b0 0x7fa7cc105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.995+0000 7fa7d115a700 1 --2- 172.21.15.44:0/8385547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7cc0ff760 0x7fa7cc0ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.995+0000 7fa7d115a700 1 -- 172.21.15.44:0/8385547 >> 172.21.15.44:0/8385547 conn(0x7fa7cc074f30 msgr2=0x7fa7cc075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.995+0000 7fa7d115a700 1 -- 172.21.15.44:0/8385547 shutdown_connections 2023-09-10T01:02:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.995+0000 7fa7d115a700 1 -- 172.21.15.44:0/8385547 wait complete. 2023-09-10T01:02:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.995+0000 7fa7d115a700 1 Processor -- start 2023-09-10T01:02:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7d115a700 1 -- start start 2023-09-10T01:02:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7d115a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7cc0ff760 0x7fa7cc19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7d115a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7cc100120 0x7fa7cc19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7d115a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7cc1055b0 0x7fa7cc198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7d115a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc198dc0 con 0x7fa7cc100120 2023-09-10T01:02:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7d115a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc198f30 con 0x7fa7cc1055b0 2023-09-10T01:02:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7d115a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc1990f0 con 0x7fa7cc0ff760 2023-09-10T01:02:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7ca59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7cc100120 0x7fa7cc19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7cad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7cc0ff760 0x7fa7cc19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7ca59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7cc100120 0x7fa7cc19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55098/0 (socket says 172.21.15.44:55098) 2023-09-10T01:02:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7ca59c700 1 -- 172.21.15.44:0/926651972 learned_addr learned my addr 172.21.15.44:0/926651972 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7cb59e700 1 --2- 172.21.15.44:0/926651972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7cc1055b0 0x7fa7cc198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7ca59c700 1 -- 172.21.15.44:0/926651972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7cc0ff760 msgr2=0x7fa7cc19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7ca59c700 1 --2- 172.21.15.44:0/926651972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7cc0ff760 0x7fa7cc19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7ca59c700 1 -- 172.21.15.44:0/926651972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7cc1055b0 msgr2=0x7fa7cc198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7ca59c700 1 --2- 172.21.15.44:0/926651972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7cc1055b0 0x7fa7cc198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.996+0000 7fa7ca59c700 1 -- 172.21.15.44:0/926651972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc009520 con 0x7fa7cc100120 2023-09-10T01:02:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.997+0000 7fa7ca59c700 1 --2- 172.21.15.44:0/926651972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7cc100120 0x7fa7cc19ee30 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000e9a0 tx=0x7fa7c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.997+0000 7fa7b3fff700 1 -- 172.21.15.44:0/926651972 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0009790 con 0x7fa7cc100120 2023-09-10T01:02:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.997+0000 7fa7b3fff700 1 -- 172.21.15.44:0/926651972 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa7c000cbd0 con 0x7fa7cc100120 2023-09-10T01:02:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.997+0000 7fa7d115a700 1 -- 172.21.15.44:0/926651972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c4009d00 con 0x7fa7cc100120 2023-09-10T01:02:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.997+0000 7fa7b3fff700 1 -- 172.21.15.44:0/926651972 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0018680 con 0x7fa7cc100120 2023-09-10T01:02:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.997+0000 7fa7d115a700 1 -- 172.21.15.44:0/926651972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7cc1995c0 con 0x7fa7cc100120 2023-09-10T01:02:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:48.999+0000 7fa7b3fff700 1 -- 172.21.15.44:0/926651972 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa7c000cd40 con 0x7fa7cc100120 2023-09-10T01:02:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.000+0000 7fa7b3fff700 1 --2- 172.21.15.44:0/926651972 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa7b40a0830 0x7fa7b40a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.000+0000 7fa7b3fff700 1 -- 172.21.15.44:0/926651972 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa7c0014070 con 0x7fa7cc100120 2023-09-10T01:02:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.000+0000 7fa7cad9d700 1 --2- 172.21.15.44:0/926651972 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa7b40a0830 0x7fa7b40a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.001+0000 7fa7cad9d700 1 --2- 172.21.15.44:0/926651972 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa7b40a0830 0x7fa7b40a2cf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009d70 tx=0x7fa7bc009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.001+0000 7fa7dd320b80 1 Processor -- start 2023-09-10T01:02:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.001+0000 7fa7dd320b80 1 -- start start 2023-09-10T01:02:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7dd320b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf60401a0 0x55adf6040580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7dd320b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf603a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7dd320b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf603acc0 0x55adf6046f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7dd320b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf5f936d0 con 0x55adf603a300 2023-09-10T01:02:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7dd320b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf5f90ff0 con 0x55adf603acc0 2023-09-10T01:02:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7dd320b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf5f8c8e0 con 0x55adf60401a0 2023-09-10T01:02:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7ca59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf603a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7cad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf60401a0 0x55adf6040580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7ca59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf603a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55106/0 (socket says 172.21.15.44:55106) 2023-09-10T01:02:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7ca59c700 1 -- 172.21.15.44:0/593797757 learned_addr learned my addr 172.21.15.44:0/593797757 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7cb59e700 1 --2- 172.21.15.44:0/593797757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf603acc0 0x55adf6046f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7ca59c700 1 -- 172.21.15.44:0/593797757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf60401a0 msgr2=0x55adf6040580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7ca59c700 1 --2- 172.21.15.44:0/593797757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf60401a0 0x55adf6040580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7ca59c700 1 -- 172.21.15.44:0/593797757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf603acc0 msgr2=0x55adf6046f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7ca59c700 1 --2- 172.21.15.44:0/593797757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf603acc0 0x55adf6046f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7ca59c700 1 -- 172.21.15.44:0/593797757 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adf5db8d80 con 0x55adf603a300 2023-09-10T01:02:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7cad9d700 1 --2- 172.21.15.44:0/593797757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf60401a0 0x55adf6040580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:02:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7ca59c700 1 --2- 172.21.15.44:0/593797757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf603a780 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00a1000 tx=0x7fa7c0098e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54eb7fad0e9b8c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.002+0000 7fa7b17fa700 1 -- 172.21.15.44:0/593797757 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c00994c0 con 0x55adf603a300 2023-09-10T01:02:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.003+0000 7fa7b17fa700 1 -- 172.21.15.44:0/593797757 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa7c009d2f0 con 0x55adf603a300 2023-09-10T01:02:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.003+0000 7fa7b17fa700 1 -- 172.21.15.44:0/593797757 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0099980 con 0x55adf603a300 2023-09-10T01:02:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.003+0000 7fa7dd320b80 1 -- 172.21.15.44:0/593797757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 msgr2=0x55adf603a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.003+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/593797757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf603a780 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00a1000 tx=0x7fa7c0098e20 comp rx=0 tx=0).stop 2023-09-10T01:02:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.003+0000 7fa7dd320b80 1 -- 172.21.15.44:0/593797757 shutdown_connections 2023-09-10T01:02:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.003+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/593797757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf603acc0 0x55adf6046f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.003+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/593797757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf60401a0 0x55adf6040580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.003+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/593797757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf603a780 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.003+0000 7fa7dd320b80 1 -- 172.21.15.44:0/593797757 >> 172.21.15.44:0/593797757 conn(0x55adf5faf030 msgr2=0x55adf6035560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.003+0000 7fa7dd320b80 1 -- 172.21.15.44:0/593797757 shutdown_connections 2023-09-10T01:02:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.003+0000 7fa7dd320b80 1 -- 172.21.15.44:0/593797757 wait complete. 2023-09-10T01:02:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.004+0000 7fa7dd320b80 1 -- start start 2023-09-10T01:02:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.004+0000 7fa7dd320b80 1 Processor -- start 2023-09-10T01:02:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.004+0000 7fa7dd320b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf6049e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.004+0000 7fa7dd320b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf603acc0 0x55adf604a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.004+0000 7fa7dd320b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf60401a0 0x55adf60440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.004+0000 7fa7dd320b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf5f2ab10 con 0x55adf603a300 2023-09-10T01:02:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.004+0000 7fa7dd320b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf5f2a810 con 0x55adf60401a0 2023-09-10T01:02:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.004+0000 7fa7dd320b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf5e0e8c0 con 0x55adf603acc0 2023-09-10T01:02:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.004+0000 7fa7dd320b80 10 client.0 _ll_register_callbacks cb 0x55adf5e74a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:02:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.004+0000 7fa7cad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf6049e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.004+0000 7fa7cad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf6049e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55110/0 (socket says 172.21.15.44:55110) 2023-09-10T01:02:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.004+0000 7fa7cad9d700 1 -- 172.21.15.44:0/1495132133 learned_addr learned my addr 172.21.15.44:0/1495132133 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.004+0000 7fa7ca59c700 1 --2- 172.21.15.44:0/1495132133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf603acc0 0x55adf604a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.005+0000 7fa7cad9d700 1 -- 172.21.15.44:0/1495132133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf603acc0 msgr2=0x55adf604a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.005+0000 7fa7cad9d700 1 --2- 172.21.15.44:0/1495132133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf603acc0 0x55adf604a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.005+0000 7fa7cad9d700 1 -- 172.21.15.44:0/1495132133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf60401a0 msgr2=0x55adf60440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.005+0000 7fa7cb59e700 1 --2- 172.21.15.44:0/1495132133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf60401a0 0x55adf60440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.005+0000 7fa7cad9d700 1 --2- 172.21.15.44:0/1495132133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf60401a0 0x55adf60440d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.005+0000 7fa7cad9d700 1 -- 172.21.15.44:0/1495132133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c4009d00 con 0x55adf603a300 2023-09-10T01:02:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.005+0000 7fa7cb59e700 1 --2- 172.21.15.44:0/1495132133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf60401a0 0x55adf60440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:02:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.005+0000 7fa7cad9d700 1 --2- 172.21.15.44:0/1495132133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf6049e80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc01ed50 tx=0x7fa7bc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b72fde6472b7af server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.005+0000 7fa7aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:02:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.005+0000 7fa7aeff5700 1 -- 172.21.15.44:0/1495132133 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc01ee40 con 0x55adf603a300 2023-09-10T01:02:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.005+0000 7fa7aeff5700 1 -- 172.21.15.44:0/1495132133 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa7bc00ede0 con 0x55adf603a300 2023-09-10T01:02:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.005+0000 7fa7dd320b80 1 -- 172.21.15.44:0/1495132133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55adf5ee4490 con 0x55adf603a300 2023-09-10T01:02:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.005+0000 7fa7aeff5700 1 -- 172.21.15.44:0/1495132133 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7bc02c610 con 0x55adf603a300 2023-09-10T01:02:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.005+0000 7fa7dd320b80 10 client.18483 fetch_fsmap learned FSMap version 89 2023-09-10T01:02:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.005+0000 7fa7dd320b80 1 -- 172.21.15.44:0/1495132133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x55adf5da1c00 con 0x55adf603a300 2023-09-10T01:02:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.005+0000 7fa7aeff5700 1 -- 172.21.15.44:0/1495132133 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55adf5ee4490 con 0x55adf603a300 2023-09-10T01:02:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.006+0000 7fa7aeff5700 1 -- 172.21.15.44:0/1495132133 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 89) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fa7bc01f430 con 0x55adf603a300 2023-09-10T01:02:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.006+0000 7fa7dd320b80 10 client.18483 fetch_fsmap finished waiting for FSMap version 89 2023-09-10T01:02:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.006+0000 7fa7dd320b80 10 client.18483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:02:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.006+0000 7fa7dd320b80 20 client.18483 populate_metadata read hostname 'smithi044' 2023-09-10T01:02:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.006+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/1495132133 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55adf60544a0 0x55adf60568c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.006+0000 7fa7dd320b80 4 client.18483 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:02:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.006+0000 7fa7dd320b80 1 -- 172.21.15.44:0/1495132133 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55adf5fa1840 con 0x55adf60544a0 2023-09-10T01:02:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.006+0000 7fa7ca59c700 1 --2- 172.21.15.44:0/1495132133 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55adf60544a0 0x55adf60568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.006+0000 7fa7ca59c700 1 --2- 172.21.15.44:0/1495132133 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55adf60544a0 0x55adf60568c0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b11fcdc4798ad21 server_cookie=478186b0f35ed21c in_seq=0 out_seq=0 2023-09-10T01:02:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.006+0000 7fa7aeff5700 10 client.18483 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:02:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.007+0000 7fa7aeff5700 1 -- 172.21.15.44:0/1495132133 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55adf5fa1840 con 0x55adf60544a0 2023-09-10T01:02:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.007+0000 7fa7aeff5700 10 client.18483 handle_command_reply: tid=0 2023-09-10T01:02:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.007+0000 7fa7dd320b80 1 client.18483 shutdown 2023-09-10T01:02:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.007+0000 7fa7dd320b80 1 -- 172.21.15.44:0/1495132133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 msgr2=0x55adf6049e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.007+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/1495132133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf6049e80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc01ed50 tx=0x7fa7bc0047c0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.007+0000 7fa7dd320b80 1 -- 172.21.15.44:0/1495132133 shutdown_connections 2023-09-10T01:02:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.007+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/1495132133 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55adf60544a0 0x55adf60568c0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.007+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/1495132133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf60401a0 0x55adf60440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.007+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/1495132133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf6049e80 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.007+0000 7fa7aeff5700 0 client.18483 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:02:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.007+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/1495132133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf603acc0 0x55adf604a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.007+0000 7fa7dd320b80 1 -- 172.21.15.44:0/1495132133 >> 172.21.15.44:0/1495132133 conn(0x55adf5faf030 msgr2=0x55adf60347a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.007+0000 7fa7dd320b80 1 -- 172.21.15.44:0/1495132133 shutdown_connections 2023-09-10T01:02:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.007+0000 7fa7dd320b80 1 -- 172.21.15.44:0/1495132133 wait complete. 2023-09-10T01:02:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.007+0000 7fa7dd320b80 20 client.18483 trim_cache size 0 max 16384 2023-09-10T01:02:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7dd320b80 1 Processor -- start 2023-09-10T01:02:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7dd320b80 1 -- start start 2023-09-10T01:02:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7dd320b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf6044080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7dd320b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf603acc0 0x55adf6034f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7dd320b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf60401a0 0x55adf6035460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7dd320b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf602cf30 con 0x55adf603a300 2023-09-10T01:02:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7dd320b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf60445c0 con 0x55adf60401a0 2023-09-10T01:02:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7dd320b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf603b580 con 0x55adf603acc0 2023-09-10T01:02:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7cad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf6044080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7ca59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf603acc0 0x55adf6034f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7ca59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf603acc0 0x55adf6034f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33770/0 (socket says 172.21.15.44:33770) 2023-09-10T01:02:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7cad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf6044080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55118/0 (socket says 172.21.15.44:55118) 2023-09-10T01:02:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7ca59c700 1 -- 172.21.15.44:0/3748502378 learned_addr learned my addr 172.21.15.44:0/3748502378 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7cb59e700 1 --2- 172.21.15.44:0/3748502378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf60401a0 0x55adf6035460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7ca59c700 1 -- 172.21.15.44:0/3748502378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf60401a0 msgr2=0x55adf6035460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7ca59c700 1 --2- 172.21.15.44:0/3748502378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf60401a0 0x55adf6035460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7ca59c700 1 -- 172.21.15.44:0/3748502378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 msgr2=0x55adf6044080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7ca59c700 1 --2- 172.21.15.44:0/3748502378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf6044080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7ca59c700 1 -- 172.21.15.44:0/3748502378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adf5ddf1f0 con 0x55adf603acc0 2023-09-10T01:02:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.023+0000 7fa7cad9d700 1 --2- 172.21.15.44:0/3748502378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf6044080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:02:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.024+0000 7fa7ca59c700 1 --2- 172.21.15.44:0/3748502378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf603acc0 0x55adf6034f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0099450 tx=0x7fa7c0095800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7f8eeb8902034bb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.024+0000 7fa7b1ffb700 1 -- 172.21.15.44:0/3748502378 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0099540 con 0x55adf603acc0 2023-09-10T01:02:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.024+0000 7fa7b1ffb700 1 -- 172.21.15.44:0/3748502378 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa7c00a1530 con 0x55adf603acc0 2023-09-10T01:02:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.024+0000 7fa7b1ffb700 1 -- 172.21.15.44:0/3748502378 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0099720 con 0x55adf603acc0 2023-09-10T01:02:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 -- 172.21.15.44:0/3748502378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf603acc0 msgr2=0x55adf6034f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/3748502378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf603acc0 0x55adf6034f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0099450 tx=0x7fa7c0095800 comp rx=0 tx=0).stop 2023-09-10T01:02:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 -- 172.21.15.44:0/3748502378 shutdown_connections 2023-09-10T01:02:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/3748502378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf60401a0 0x55adf6035460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/3748502378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603a300 0x55adf6044080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/3748502378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf603acc0 0x55adf6034f20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 -- 172.21.15.44:0/3748502378 >> 172.21.15.44:0/3748502378 conn(0x55adf5faf030 msgr2=0x55adf6043970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 -- 172.21.15.44:0/3748502378 shutdown_connections 2023-09-10T01:02:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 -- 172.21.15.44:0/3748502378 wait complete. 2023-09-10T01:02:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 -- start start 2023-09-10T01:02:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 Processor -- start 2023-09-10T01:02:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf603a300 0x55adf60a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603acc0 0x55adf60a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf60401a0 0x55adf60a0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf60a07f0 con 0x55adf603acc0 2023-09-10T01:02:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf60a0960 con 0x55adf603a300 2023-09-10T01:02:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.025+0000 7fa7dd320b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf60a0b20 con 0x55adf60401a0 2023-09-10T01:02:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.026+0000 7fa7dd320b80 10 client.0 _ll_register_callbacks cb 0x7fa7bc01e880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:02:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.026+0000 7fa7ca59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603acc0 0x55adf60a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.026+0000 7fa7ca59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603acc0 0x55adf60a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55126/0 (socket says 172.21.15.44:55126) 2023-09-10T01:02:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.026+0000 7fa7ca59c700 1 -- 172.21.15.44:0/2169547694 learned_addr learned my addr 172.21.15.44:0/2169547694 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:02:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.026+0000 7fa7cb59e700 1 --2- 172.21.15.44:0/2169547694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf60401a0 0x55adf60a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.026+0000 7fa7cad9d700 1 --2- 172.21.15.44:0/2169547694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf603a300 0x55adf60a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.026+0000 7fa7ca59c700 1 -- 172.21.15.44:0/2169547694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf60401a0 msgr2=0x55adf60a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.026+0000 7fa7ca59c700 1 --2- 172.21.15.44:0/2169547694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf60401a0 0x55adf60a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.026+0000 7fa7ca59c700 1 -- 172.21.15.44:0/2169547694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf603a300 msgr2=0x55adf60a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:02:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.026+0000 7fa7ca59c700 1 --2- 172.21.15.44:0/2169547694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf603a300 0x55adf60a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.026+0000 7fa7ca59c700 1 -- 172.21.15.44:0/2169547694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55adf5da1c00 con 0x55adf603acc0 2023-09-10T01:02:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.026+0000 7fa7ca59c700 1 --2- 172.21.15.44:0/2169547694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603acc0 0x55adf60a1a30 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00a1000 tx=0x7fa7c0099d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147a569b85aef21b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:02:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.026+0000 7fa7aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:02:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.026+0000 7fa7aeff5700 1 -- 172.21.15.44:0/2169547694 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0099720 con 0x55adf603acc0 2023-09-10T01:02:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.026+0000 7fa7aeff5700 1 -- 172.21.15.44:0/2169547694 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa7c0099880 con 0x55adf603acc0 2023-09-10T01:02:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.026+0000 7fa7dd320b80 1 -- 172.21.15.44:0/2169547694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7c009d2f0 con 0x55adf603acc0 2023-09-10T01:02:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.027+0000 7fa7aeff5700 1 -- 172.21.15.44:0/2169547694 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7c00989b0 con 0x55adf603acc0 2023-09-10T01:02:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.027+0000 7fa7dd320b80 10 client.18489 fetch_fsmap learned FSMap version 89 2023-09-10T01:02:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.027+0000 7fa7dd320b80 1 -- 172.21.15.44:0/2169547694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x55adf5db8d80 con 0x55adf603acc0 2023-09-10T01:02:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.027+0000 7fa7aeff5700 1 -- 172.21.15.44:0/2169547694 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa7c009d2f0 con 0x55adf603acc0 2023-09-10T01:02:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.027+0000 7fa7aeff5700 1 -- 172.21.15.44:0/2169547694 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 89) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fa7c0010430 con 0x55adf603acc0 2023-09-10T01:02:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.027+0000 7fa7dd320b80 10 client.18489 fetch_fsmap finished waiting for FSMap version 89 2023-09-10T01:02:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.027+0000 7fa7dd320b80 10 client.18489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:02:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.027+0000 7fa7dd320b80 20 client.18489 populate_metadata read hostname 'smithi044' 2023-09-10T01:02:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.027+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/2169547694 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55adf60a9070 0x55adf60ab490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:02:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.027+0000 7fa7dd320b80 4 client.18489 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:02:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.027+0000 7fa7dd320b80 1 -- 172.21.15.44:0/2169547694 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55adf5f858c0 con 0x55adf60a9070 2023-09-10T01:02:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.027+0000 7fa7cad9d700 1 --2- 172.21.15.44:0/2169547694 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55adf60a9070 0x55adf60ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:02:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.028+0000 7fa7aeff5700 10 client.18489 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:02:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.028+0000 7fa7cad9d700 1 --2- 172.21.15.44:0/2169547694 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55adf60a9070 0x55adf60ab490 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac583fa27a154a72 server_cookie=6d1cfea2e0b0bc49 in_seq=0 out_seq=0 2023-09-10T01:02:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.050+0000 7fa7aeff5700 1 -- 172.21.15.44:0/2169547694 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55adf5f858c0 con 0x55adf60a9070 2023-09-10T01:02:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.050+0000 7fa7aeff5700 10 client.18489 handle_command_reply: tid=0 2023-09-10T01:02:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.050+0000 7fa7dd320b80 1 client.18489 shutdown 2023-09-10T01:02:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.050+0000 7fa7dd320b80 1 -- 172.21.15.44:0/2169547694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603acc0 msgr2=0x55adf60a1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.050+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/2169547694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603acc0 0x55adf60a1a30 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00a1000 tx=0x7fa7c0099d00 comp rx=0 tx=0).stop 2023-09-10T01:02:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.050+0000 7fa7dd320b80 1 -- 172.21.15.44:0/2169547694 shutdown_connections 2023-09-10T01:02:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.050+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/2169547694 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55adf60a9070 0x55adf60ab490 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.050+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/2169547694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adf60401a0 0x55adf60a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.050+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/2169547694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adf603a300 0x55adf60a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.050+0000 7fa7aeff5700 0 client.18489 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:02:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.050+0000 7fa7dd320b80 1 --2- 172.21.15.44:0/2169547694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55adf603acc0 0x55adf60a1a30 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.050+0000 7fa7dd320b80 1 -- 172.21.15.44:0/2169547694 >> 172.21.15.44:0/2169547694 conn(0x55adf5faf030 msgr2=0x55adf60392f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.050+0000 7fa7dd320b80 1 -- 172.21.15.44:0/2169547694 shutdown_connections 2023-09-10T01:02:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.050+0000 7fa7dd320b80 1 -- 172.21.15.44:0/2169547694 wait complete. 2023-09-10T01:02:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.051+0000 7fa7dd320b80 20 client.18489 trim_cache size 0 max 16384 2023-09-10T01:02:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:02:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:02:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:02:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:02:49.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.051+0000 7fa7d115a700 1 -- 172.21.15.44:0/926651972 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa7b40a0830 msgr2=0x7fa7b40a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:49.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.051+0000 7fa7d115a700 1 --2- 172.21.15.44:0/926651972 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa7b40a0830 0x7fa7b40a2cf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009d70 tx=0x7fa7bc009260 comp rx=0 tx=0).stop 2023-09-10T01:02:49.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.051+0000 7fa7d115a700 1 -- 172.21.15.44:0/926651972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7cc100120 msgr2=0x7fa7cc19ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:02:49.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.051+0000 7fa7d115a700 1 --2- 172.21.15.44:0/926651972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7cc100120 0x7fa7cc19ee30 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000e9a0 tx=0x7fa7c000ecb0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.051+0000 7fa7d115a700 1 -- 172.21.15.44:0/926651972 shutdown_connections 2023-09-10T01:02:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.051+0000 7fa7d115a700 1 --2- 172.21.15.44:0/926651972 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa7b40a0830 0x7fa7b40a2cf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.051+0000 7fa7d115a700 1 --2- 172.21.15.44:0/926651972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7cc1055b0 0x7fa7cc198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.051+0000 7fa7d115a700 1 --2- 172.21.15.44:0/926651972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7cc0ff760 0x7fa7cc19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.051+0000 7fa7d115a700 1 --2- 172.21.15.44:0/926651972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7cc100120 0x7fa7cc19ee30 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:02:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.051+0000 7fa7d115a700 1 -- 172.21.15.44:0/926651972 >> 172.21.15.44:0/926651972 conn(0x7fa7cc074f30 msgr2=0x7fa7cc0679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:02:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.051+0000 7fa7d115a700 1 -- 172.21.15.44:0/926651972 shutdown_connections 2023-09-10T01:02:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:02:49.051+0000 7fa7d115a700 1 -- 172.21.15.44:0/926651972 wait complete. 2023-09-10T01:02:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:48 smithi099 ceph-mon[102692]: pgmap v701: 129 pgs: 129 active+clean; 9.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 38 MiB/s wr, 34 op/s 2023-09-10T01:02:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:48 smithi131 ceph-mon[103713]: pgmap v701: 129 pgs: 129 active+clean; 9.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 38 MiB/s wr, 34 op/s 2023-09-10T01:02:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:48 smithi044 ceph-mon[98676]: pgmap v701: 129 pgs: 129 active+clean; 9.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 38 MiB/s wr, 34 op/s 2023-09-10T01:02:50.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:49 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:02:50.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:49 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:02:50.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:49 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:02:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:50 smithi131 ceph-mon[103713]: pgmap v702: 129 pgs: 129 active+clean; 9.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 38 MiB/s wr, 34 op/s 2023-09-10T01:02:51.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:50 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:02:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:50 smithi044 ceph-mon[98676]: pgmap v702: 129 pgs: 129 active+clean; 9.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 38 MiB/s wr, 34 op/s 2023-09-10T01:02:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:50 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:02:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:50 smithi099 ceph-mon[102692]: pgmap v702: 129 pgs: 129 active+clean; 9.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 38 MiB/s wr, 34 op/s 2023-09-10T01:02:51.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:50 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:02:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:52 smithi131 ceph-mon[103713]: pgmap v703: 129 pgs: 129 active+clean; 9.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 57 MiB/s wr, 49 op/s 2023-09-10T01:02:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:52 smithi044 ceph-mon[98676]: pgmap v703: 129 pgs: 129 active+clean; 9.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 57 MiB/s wr, 49 op/s 2023-09-10T01:02:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:52 smithi099 ceph-mon[102692]: pgmap v703: 129 pgs: 129 active+clean; 9.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 57 MiB/s wr, 49 op/s 2023-09-10T01:02:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:54 smithi131 ceph-mon[103713]: pgmap v704: 129 pgs: 129 active+clean; 9.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 39 MiB/s wr, 29 op/s 2023-09-10T01:02:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:54 smithi044 ceph-mon[98676]: pgmap v704: 129 pgs: 129 active+clean; 9.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 39 MiB/s wr, 29 op/s 2023-09-10T01:02:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:54 smithi099 ceph-mon[102692]: pgmap v704: 129 pgs: 129 active+clean; 9.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 39 MiB/s wr, 29 op/s 2023-09-10T01:02:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:56 smithi131 ceph-mon[103713]: pgmap v705: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 58 MiB/s wr, 42 op/s 2023-09-10T01:02:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:56 smithi044 ceph-mon[98676]: pgmap v705: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 58 MiB/s wr, 42 op/s 2023-09-10T01:02:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:56 smithi099 ceph-mon[102692]: pgmap v705: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 58 MiB/s wr, 42 op/s 2023-09-10T01:02:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:02:58 smithi131 ceph-mon[103713]: pgmap v706: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 39 MiB/s wr, 27 op/s 2023-09-10T01:02:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:02:58 smithi044 ceph-mon[98676]: pgmap v706: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 39 MiB/s wr, 27 op/s 2023-09-10T01:02:59.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:02:58 smithi099 ceph-mon[102692]: pgmap v706: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 39 MiB/s wr, 27 op/s 2023-09-10T01:03:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:00 smithi131 ceph-mon[103713]: pgmap v707: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 39 MiB/s wr, 27 op/s 2023-09-10T01:03:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:00 smithi044 ceph-mon[98676]: pgmap v707: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 39 MiB/s wr, 27 op/s 2023-09-10T01:03:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:00 smithi099 ceph-mon[102692]: pgmap v707: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 39 MiB/s wr, 27 op/s 2023-09-10T01:03:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.196+0000 7f6b6a578700 1 -- 172.21.15.44:0/2840573400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b640ff760 msgr2=0x7f6b640ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.196+0000 7f6b6a578700 1 --2- 172.21.15.44:0/2840573400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b640ff760 0x7f6b640ffbe0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f6b580099c0 tx=0x7f6b58009cd0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.196+0000 7f6b6a578700 1 -- 172.21.15.44:0/2840573400 shutdown_connections 2023-09-10T01:03:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.196+0000 7f6b6a578700 1 --2- 172.21.15.44:0/2840573400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b64100120 0x7f6b6410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.196+0000 7f6b6a578700 1 --2- 172.21.15.44:0/2840573400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b641055b0 0x7f6b64105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.196+0000 7f6b6a578700 1 --2- 172.21.15.44:0/2840573400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b640ff760 0x7f6b640ffbe0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.196+0000 7f6b6a578700 1 -- 172.21.15.44:0/2840573400 >> 172.21.15.44:0/2840573400 conn(0x7f6b64074f30 msgr2=0x7f6b64075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.196+0000 7f6b6a578700 1 -- 172.21.15.44:0/2840573400 shutdown_connections 2023-09-10T01:03:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.196+0000 7f6b6a578700 1 -- 172.21.15.44:0/2840573400 wait complete. 2023-09-10T01:03:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.197+0000 7f6b6a578700 1 Processor -- start 2023-09-10T01:03:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.197+0000 7f6b6a578700 1 -- start start 2023-09-10T01:03:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b6a578700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b640ff760 0x7f6b64197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b6a578700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b64100120 0x7f6b64197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b6a578700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b641055b0 0x7f6b641a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b6a578700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b641a2b60 con 0x7f6b64100120 2023-09-10T01:03:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b6a578700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b641a2cd0 con 0x7f6b641055b0 2023-09-10T01:03:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b6a578700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b641a2e90 con 0x7f6b640ff760 2023-09-10T01:03:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b63fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b640ff760 0x7f6b64197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b637fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b64100120 0x7f6b64197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b63fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b640ff760 0x7f6b64197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33788/0 (socket says 172.21.15.44:33788) 2023-09-10T01:03:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b63fff700 1 -- 172.21.15.44:0/1319479138 learned_addr learned my addr 172.21.15.44:0/1319479138 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b68b15700 1 --2- 172.21.15.44:0/1319479138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b641055b0 0x7f6b641a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b63fff700 1 -- 172.21.15.44:0/1319479138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b641055b0 msgr2=0x7f6b641a24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b63fff700 1 --2- 172.21.15.44:0/1319479138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b641055b0 0x7f6b641a24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b63fff700 1 -- 172.21.15.44:0/1319479138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b64100120 msgr2=0x7f6b64197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b63fff700 1 --2- 172.21.15.44:0/1319479138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b64100120 0x7f6b64197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b63fff700 1 -- 172.21.15.44:0/1319479138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b580095f0 con 0x7f6b640ff760 2023-09-10T01:03:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.198+0000 7f6b637fe700 1 --2- 172.21.15.44:0/1319479138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b64100120 0x7f6b64197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:03:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.199+0000 7f6b63fff700 1 --2- 172.21.15.44:0/1319479138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b640ff760 0x7f6b64197630 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400e9a0 tx=0x7f6b5400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.199+0000 7f6b617fa700 1 -- 172.21.15.44:0/1319479138 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54009790 con 0x7f6b640ff760 2023-09-10T01:03:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.199+0000 7f6b6a578700 1 -- 172.21.15.44:0/1319479138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b641a3170 con 0x7f6b640ff760 2023-09-10T01:03:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.199+0000 7f6b6a578700 1 -- 172.21.15.44:0/1319479138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b641a3650 con 0x7f6b640ff760 2023-09-10T01:03:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.199+0000 7f6b617fa700 1 -- 172.21.15.44:0/1319479138 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b54010430 con 0x7f6b640ff760 2023-09-10T01:03:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.199+0000 7f6b617fa700 1 -- 172.21.15.44:0/1319479138 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5400cea0 con 0x7f6b640ff760 2023-09-10T01:03:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.202+0000 7f6b617fa700 1 -- 172.21.15.44:0/1319479138 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f6b5400ca10 con 0x7f6b640ff760 2023-09-10T01:03:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.203+0000 7f6b617fa700 1 --2- 172.21.15.44:0/1319479138 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6b4c0a07b0 0x7f6b4c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.203+0000 7f6b617fa700 1 -- 172.21.15.44:0/1319479138 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f6b54014070 con 0x7f6b640ff760 2023-09-10T01:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.203+0000 7f6b637fe700 1 --2- 172.21.15.44:0/1319479138 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6b4c0a07b0 0x7f6b4c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.203+0000 7f6b637fe700 1 --2- 172.21.15.44:0/1319479138 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6b4c0a07b0 0x7f6b4c0a2c70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6b58006010 tx=0x7f6b5801a040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.203+0000 7f6b7673eb80 1 Processor -- start 2023-09-10T01:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b7673eb80 1 -- start start 2023-09-10T01:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b7673eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624405035b0 0x562440503990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b7673eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fd710 0x5624404fdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b7673eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624404fe0d0 0x56244050a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b7673eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624404526d0 con 0x5624404fe0d0 2023-09-10T01:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b7673eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244044fff0 con 0x5624404fd710 2023-09-10T01:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b7673eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244044b8e0 con 0x5624405035b0 2023-09-10T01:03:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b63fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624405035b0 0x562440503990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b63fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624405035b0 0x562440503990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33796/0 (socket says 172.21.15.44:33796) 2023-09-10T01:03:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b63fff700 1 -- 172.21.15.44:0/4026279091 learned_addr learned my addr 172.21.15.44:0/4026279091 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b68b15700 1 --2- 172.21.15.44:0/4026279091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624404fe0d0 0x56244050a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b637fe700 1 --2- 172.21.15.44:0/4026279091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fd710 0x5624404fdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b63fff700 1 -- 172.21.15.44:0/4026279091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fd710 msgr2=0x5624404fdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b63fff700 1 --2- 172.21.15.44:0/4026279091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fd710 0x5624404fdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b63fff700 1 -- 172.21.15.44:0/4026279091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624404fe0d0 msgr2=0x56244050a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b63fff700 1 --2- 172.21.15.44:0/4026279091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624404fe0d0 0x56244050a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b63fff700 1 -- 172.21.15.44:0/4026279091 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562440277d80 con 0x5624405035b0 2023-09-10T01:03:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b68b15700 1 --2- 172.21.15.44:0/4026279091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624404fe0d0 0x56244050a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:03:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.204+0000 7f6b63fff700 1 --2- 172.21.15.44:0/4026279091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624405035b0 0x562440503990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009230 tx=0x7f6b5400cd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbe42dd7936d965c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.205+0000 7f6b4a7fc700 1 -- 172.21.15.44:0/4026279091 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5409d160 con 0x5624405035b0 2023-09-10T01:03:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.205+0000 7f6b4a7fc700 1 -- 172.21.15.44:0/4026279091 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b5409d320 con 0x5624405035b0 2023-09-10T01:03:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.205+0000 7f6b4a7fc700 1 -- 172.21.15.44:0/4026279091 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b540999b0 con 0x5624405035b0 2023-09-10T01:03:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.206+0000 7f6b7673eb80 1 -- 172.21.15.44:0/4026279091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624405035b0 msgr2=0x562440503990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.206+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/4026279091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624405035b0 0x562440503990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009230 tx=0x7f6b5400cd40 comp rx=0 tx=0).stop 2023-09-10T01:03:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.206+0000 7f6b7673eb80 1 -- 172.21.15.44:0/4026279091 shutdown_connections 2023-09-10T01:03:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.206+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/4026279091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624404fe0d0 0x56244050a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.206+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/4026279091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624405035b0 0x562440503990 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.206+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/4026279091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fd710 0x5624404fdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.206+0000 7f6b7673eb80 1 -- 172.21.15.44:0/4026279091 >> 172.21.15.44:0/4026279091 conn(0x56244046e030 msgr2=0x5624404f4560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.206+0000 7f6b7673eb80 1 -- 172.21.15.44:0/4026279091 shutdown_connections 2023-09-10T01:03:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.206+0000 7f6b7673eb80 1 -- 172.21.15.44:0/4026279091 wait complete. 2023-09-10T01:03:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.206+0000 7f6b7673eb80 1 -- start start 2023-09-10T01:03:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.206+0000 7f6b7673eb80 1 Processor -- start 2023-09-10T01:03:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b7673eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fd710 0x56244050a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b7673eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fe0d0 0x562440507560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b7673eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624405035b0 0x562440507ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b7673eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624403e9b10 con 0x5624405035b0 2023-09-10T01:03:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b7673eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624403e9810 con 0x5624404fe0d0 2023-09-10T01:03:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b7673eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624402cd8c0 con 0x5624404fd710 2023-09-10T01:03:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b7673eb80 10 client.0 _ll_register_callbacks cb 0x562440333a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:03:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b637fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fe0d0 0x562440507560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b63fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fd710 0x56244050a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b637fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fe0d0 0x562440507560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56894/0 (socket says 172.21.15.44:56894) 2023-09-10T01:03:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b637fe700 1 -- 172.21.15.44:0/1101537245 learned_addr learned my addr 172.21.15.44:0/1101537245 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b68b15700 1 --2- 172.21.15.44:0/1101537245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624405035b0 0x562440507ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b637fe700 1 -- 172.21.15.44:0/1101537245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fd710 msgr2=0x56244050a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b637fe700 1 --2- 172.21.15.44:0/1101537245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fd710 0x56244050a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b637fe700 1 -- 172.21.15.44:0/1101537245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624405035b0 msgr2=0x562440507ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b637fe700 1 --2- 172.21.15.44:0/1101537245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624405035b0 0x562440507ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b637fe700 1 -- 172.21.15.44:0/1101537245 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b5c011230 con 0x5624404fe0d0 2023-09-10T01:03:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.207+0000 7f6b63fff700 1 --2- 172.21.15.44:0/1101537245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fd710 0x56244050a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:03:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.208+0000 7f6b637fe700 1 --2- 172.21.15.44:0/1101537245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fe0d0 0x562440507560 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6b580099c0 tx=0x7f6b58009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=979500e7908b2072 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.208+0000 7f6b437fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:03:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.208+0000 7f6b437fe700 1 -- 172.21.15.44:0/1101537245 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800f460 con 0x5624404fe0d0 2023-09-10T01:03:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.208+0000 7f6b437fe700 1 -- 172.21.15.44:0/1101537245 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f6b5800f5c0 con 0x5624404fe0d0 2023-09-10T01:03:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.208+0000 7f6b7673eb80 1 -- 172.21.15.44:0/1101537245 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624403a3490 con 0x5624404fe0d0 2023-09-10T01:03:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.208+0000 7f6b437fe700 1 -- 172.21.15.44:0/1101537245 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b58049800 con 0x5624404fe0d0 2023-09-10T01:03:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.208+0000 7f6b7673eb80 10 client.28157 fetch_fsmap learned FSMap version 90 2023-09-10T01:03:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.208+0000 7f6b7673eb80 1 -- 172.21.15.44:0/1101537245 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) v3 -- 0x7f6b641a3650 con 0x5624404fe0d0 2023-09-10T01:03:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.208+0000 7f6b437fe700 1 -- 172.21.15.44:0/1101537245 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5624403a3490 con 0x5624404fe0d0 2023-09-10T01:03:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.212+0000 7f6b437fe700 1 -- 172.21.15.44:0/1101537245 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 90) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6b5800f780 con 0x5624404fe0d0 2023-09-10T01:03:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.212+0000 7f6b7673eb80 10 client.28157 fetch_fsmap finished waiting for FSMap version 90 2023-09-10T01:03:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.212+0000 7f6b7673eb80 10 client.28157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:03:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.212+0000 7f6b7673eb80 20 client.28157 populate_metadata read hostname 'smithi044' 2023-09-10T01:03:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.212+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/1101537245 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5624404fd300 0x5624405159c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.212+0000 7f6b7673eb80 4 client.28157 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:03:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.212+0000 7f6b7673eb80 1 -- 172.21.15.44:0/1101537245 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562440460840 con 0x5624404fd300 2023-09-10T01:03:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.212+0000 7f6b63fff700 1 --2- 172.21.15.44:0/1101537245 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5624404fd300 0x5624405159c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.212+0000 7f6b437fe700 10 client.28157 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.212+0000 7f6b63fff700 1 --2- 172.21.15.44:0/1101537245 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5624404fd300 0x5624405159c0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a59d25bbc9be91c4 server_cookie=636bde0f6fd18b8f in_seq=0 out_seq=0 2023-09-10T01:03:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.213+0000 7f6b437fe700 1 -- 172.21.15.44:0/1101537245 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562440460840 con 0x5624404fd300 2023-09-10T01:03:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.213+0000 7f6b437fe700 10 client.28157 handle_command_reply: tid=0 2023-09-10T01:03:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.213+0000 7f6b7673eb80 1 client.28157 shutdown 2023-09-10T01:03:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.213+0000 7f6b7673eb80 1 -- 172.21.15.44:0/1101537245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fe0d0 msgr2=0x562440507560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.213+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/1101537245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fe0d0 0x562440507560 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6b580099c0 tx=0x7f6b58009310 comp rx=0 tx=0).stop 2023-09-10T01:03:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.215+0000 7f6b7673eb80 1 -- 172.21.15.44:0/1101537245 shutdown_connections 2023-09-10T01:03:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.215+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/1101537245 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5624404fd300 0x5624405159c0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.215+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/1101537245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624405035b0 0x562440507ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.215+0000 7f6b437fe700 0 client.28157 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.215+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/1101537245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fd710 0x56244050a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.215+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/1101537245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fe0d0 0x562440507560 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.215+0000 7f6b7673eb80 1 -- 172.21.15.44:0/1101537245 >> 172.21.15.44:0/1101537245 conn(0x56244046e030 msgr2=0x5624404f37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.215+0000 7f6b7673eb80 1 -- 172.21.15.44:0/1101537245 shutdown_connections 2023-09-10T01:03:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.215+0000 7f6b7673eb80 1 -- 172.21.15.44:0/1101537245 wait complete. 2023-09-10T01:03:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.215+0000 7f6b7673eb80 20 client.28157 trim_cache size 0 max 16384 2023-09-10T01:03:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b7673eb80 1 Processor -- start 2023-09-10T01:03:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b7673eb80 1 -- start start 2023-09-10T01:03:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b7673eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fd710 0x562440505190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b7673eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fe0d0 0x5624405157b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b7673eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624405035b0 0x562440509e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b7673eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624404fe990 con 0x5624405035b0 2023-09-10T01:03:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b7673eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562440508670 con 0x5624404fd710 2023-09-10T01:03:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b7673eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562440508280 con 0x5624404fe0d0 2023-09-10T01:03:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b637fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fe0d0 0x5624405157b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b63fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fd710 0x562440505190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b68b15700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624405035b0 0x562440509e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b637fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fe0d0 0x5624405157b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33812/0 (socket says 172.21.15.44:33812) 2023-09-10T01:03:02.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b637fe700 1 -- 172.21.15.44:0/842875687 learned_addr learned my addr 172.21.15.44:0/842875687 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:02.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b68b15700 1 -- 172.21.15.44:0/842875687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fe0d0 msgr2=0x5624405157b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:02.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b68b15700 1 --2- 172.21.15.44:0/842875687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fe0d0 0x5624405157b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b68b15700 1 -- 172.21.15.44:0/842875687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fd710 msgr2=0x562440505190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b68b15700 1 --2- 172.21.15.44:0/842875687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fd710 0x562440505190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b68b15700 1 -- 172.21.15.44:0/842875687 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562440260c00 con 0x5624405035b0 2023-09-10T01:03:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.235+0000 7f6b63fff700 1 --2- 172.21.15.44:0/842875687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fd710 0x562440505190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:03:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.234+0000 7f6b637fe700 1 --2- 172.21.15.44:0/842875687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fe0d0 0x5624405157b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:03:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.235+0000 7f6b68b15700 1 --2- 172.21.15.44:0/842875687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624405035b0 0x562440509e10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c007660 tx=0x7f6b5c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b18ce4cc6fb54d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.235+0000 7f6b4affd700 1 -- 172.21.15.44:0/842875687 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00e040 con 0x5624405035b0 2023-09-10T01:03:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.235+0000 7f6b4affd700 1 -- 172.21.15.44:0/842875687 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b5c00f040 con 0x5624405035b0 2023-09-10T01:03:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.235+0000 7f6b4affd700 1 -- 172.21.15.44:0/842875687 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c016a30 con 0x5624405035b0 2023-09-10T01:03:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.236+0000 7f6b7673eb80 1 -- 172.21.15.44:0/842875687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624405035b0 msgr2=0x562440509e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.236+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/842875687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624405035b0 0x562440509e10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c007660 tx=0x7f6b5c009920 comp rx=0 tx=0).stop 2023-09-10T01:03:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.236+0000 7f6b7673eb80 1 -- 172.21.15.44:0/842875687 shutdown_connections 2023-09-10T01:03:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.236+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/842875687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624405035b0 0x562440509e10 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.236+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/842875687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fd710 0x562440505190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.236+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/842875687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fe0d0 0x5624405157b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.236+0000 7f6b7673eb80 1 -- 172.21.15.44:0/842875687 >> 172.21.15.44:0/842875687 conn(0x56244046e030 msgr2=0x5624405150a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.236+0000 7f6b7673eb80 1 -- 172.21.15.44:0/842875687 shutdown_connections 2023-09-10T01:03:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.236+0000 7f6b7673eb80 1 -- 172.21.15.44:0/842875687 wait complete. 2023-09-10T01:03:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.236+0000 7f6b7673eb80 1 -- start start 2023-09-10T01:03:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b7673eb80 1 Processor -- start 2023-09-10T01:03:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b7673eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fd710 0x56244050e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b7673eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fe0d0 0x56244050c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b7673eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624405035b0 0x56244050ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b7673eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244050d3a0 con 0x5624405035b0 2023-09-10T01:03:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b7673eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244050d510 con 0x5624404fe0d0 2023-09-10T01:03:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b7673eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244050d6d0 con 0x5624404fd710 2023-09-10T01:03:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b7673eb80 10 client.0 _ll_register_callbacks cb 0x7f6b58005370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:03:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b63fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fd710 0x56244050e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b63fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fd710 0x56244050e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33816/0 (socket says 172.21.15.44:33816) 2023-09-10T01:03:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b63fff700 1 -- 172.21.15.44:0/575457703 learned_addr learned my addr 172.21.15.44:0/575457703 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b68b15700 1 --2- 172.21.15.44:0/575457703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624405035b0 0x56244050ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b63fff700 1 -- 172.21.15.44:0/575457703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fe0d0 msgr2=0x56244050c920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:03:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b63fff700 1 --2- 172.21.15.44:0/575457703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fe0d0 0x56244050c920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b63fff700 1 -- 172.21.15.44:0/575457703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624405035b0 msgr2=0x56244050ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b63fff700 1 --2- 172.21.15.44:0/575457703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624405035b0 0x56244050ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.237+0000 7f6b63fff700 1 -- 172.21.15.44:0/575457703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562440277d80 con 0x5624404fd710 2023-09-10T01:03:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.238+0000 7f6b63fff700 1 --2- 172.21.15.44:0/575457703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fd710 0x56244050e680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400c2d0 tx=0x7f6b54099670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c55c47374543f61 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.238+0000 7f6b437fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:03:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.238+0000 7f6b437fe700 1 -- 172.21.15.44:0/575457703 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b540b7070 con 0x5624404fd710 2023-09-10T01:03:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.238+0000 7f6b7673eb80 1 -- 172.21.15.44:0/575457703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b5409d320 con 0x5624404fd710 2023-09-10T01:03:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.238+0000 7f6b437fe700 1 -- 172.21.15.44:0/575457703 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f6b540a1560 con 0x5624404fd710 2023-09-10T01:03:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.238+0000 7f6b437fe700 1 -- 172.21.15.44:0/575457703 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b5400ccc0 con 0x5624404fd710 2023-09-10T01:03:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.238+0000 7f6b7673eb80 10 client.28099 fetch_fsmap learned FSMap version 90 2023-09-10T01:03:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.238+0000 7f6b7673eb80 1 -- 172.21.15.44:0/575457703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) v3 -- 0x7f6b641a3650 con 0x5624404fd710 2023-09-10T01:03:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.238+0000 7f6b437fe700 1 -- 172.21.15.44:0/575457703 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b5407d080 con 0x5624404fd710 2023-09-10T01:03:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.238+0000 7f6b437fe700 1 -- 172.21.15.44:0/575457703 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 90) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6b54098b00 con 0x5624404fd710 2023-09-10T01:03:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.239+0000 7f6b7673eb80 10 client.28099 fetch_fsmap finished waiting for FSMap version 90 2023-09-10T01:03:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.239+0000 7f6b7673eb80 10 client.28099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:03:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.239+0000 7f6b7673eb80 20 client.28099 populate_metadata read hostname 'smithi044' 2023-09-10T01:03:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.239+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/575457703 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562440568370 0x56244056a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.239+0000 7f6b7673eb80 4 client.28099 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:03:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.239+0000 7f6b7673eb80 1 -- 172.21.15.44:0/575457703 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5624404448c0 con 0x562440568370 2023-09-10T01:03:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.240+0000 7f6b637fe700 1 --2- 172.21.15.44:0/575457703 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562440568370 0x56244056a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.240+0000 7f6b637fe700 1 --2- 172.21.15.44:0/575457703 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562440568370 0x56244056a790 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dfd852d4670e332 server_cookie=59450fd92514438a in_seq=0 out_seq=0 2023-09-10T01:03:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:02.240+0000 7f6b437fe700 10 client.28099 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:02 smithi131 ceph-mon[103713]: pgmap v708: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 42 MiB/s wr, 99 op/s 2023-09-10T01:03:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:02 smithi044 ceph-mon[98676]: pgmap v708: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 42 MiB/s wr, 99 op/s 2023-09-10T01:03:03.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:02 smithi099 ceph-mon[102692]: pgmap v708: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 42 MiB/s wr, 99 op/s 2023-09-10T01:03:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.444+0000 7f6b437fe700 1 -- 172.21.15.44:0/575457703 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5624404448c0 con 0x562440568370 2023-09-10T01:03:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.444+0000 7f6b437fe700 10 client.28099 handle_command_reply: tid=0 2023-09-10T01:03:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.444+0000 7f6b7673eb80 1 client.28099 shutdown 2023-09-10T01:03:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.444+0000 7f6b7673eb80 1 -- 172.21.15.44:0/575457703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fd710 msgr2=0x56244050e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.444+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/575457703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fd710 0x56244050e680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400c2d0 tx=0x7f6b54099670 comp rx=0 tx=0).stop 2023-09-10T01:03:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.444+0000 7f6b7673eb80 1 -- 172.21.15.44:0/575457703 shutdown_connections 2023-09-10T01:03:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.444+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/575457703 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562440568370 0x56244056a790 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.444+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/575457703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624405035b0 0x56244050ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.444+0000 7f6b437fe700 0 client.28099 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.444+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/575457703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5624404fd710 0x56244050e680 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.444+0000 7f6b7673eb80 1 --2- 172.21.15.44:0/575457703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624404fe0d0 0x56244050c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.444+0000 7f6b7673eb80 1 -- 172.21.15.44:0/575457703 >> 172.21.15.44:0/575457703 conn(0x56244046e030 msgr2=0x5624404f82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.444+0000 7f6b7673eb80 1 -- 172.21.15.44:0/575457703 shutdown_connections 2023-09-10T01:03:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.445+0000 7f6b7673eb80 1 -- 172.21.15.44:0/575457703 wait complete. 2023-09-10T01:03:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.445+0000 7f6b7673eb80 20 client.28099 trim_cache size 0 max 16384 2023-09-10T01:03:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:03:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:03:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:03:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:03:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.445+0000 7f6b6a578700 1 -- 172.21.15.44:0/1319479138 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6b4c0a07b0 msgr2=0x7f6b4c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.445+0000 7f6b6a578700 1 --2- 172.21.15.44:0/1319479138 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6b4c0a07b0 0x7f6b4c0a2c70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6b58006010 tx=0x7f6b5801a040 comp rx=0 tx=0).stop 2023-09-10T01:03:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.445+0000 7f6b6a578700 1 -- 172.21.15.44:0/1319479138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b640ff760 msgr2=0x7f6b64197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.445+0000 7f6b6a578700 1 --2- 172.21.15.44:0/1319479138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b640ff760 0x7f6b64197630 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400e9a0 tx=0x7f6b5400ecb0 comp rx=0 tx=0).stop 2023-09-10T01:03:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.445+0000 7f6b6a578700 1 -- 172.21.15.44:0/1319479138 shutdown_connections 2023-09-10T01:03:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.445+0000 7f6b6a578700 1 --2- 172.21.15.44:0/1319479138 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6b4c0a07b0 0x7f6b4c0a2c70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.445+0000 7f6b6a578700 1 --2- 172.21.15.44:0/1319479138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b641055b0 0x7f6b641a24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.445+0000 7f6b6a578700 1 --2- 172.21.15.44:0/1319479138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b640ff760 0x7f6b64197630 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.445+0000 7f6b6a578700 1 --2- 172.21.15.44:0/1319479138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b64100120 0x7f6b64197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.445+0000 7f6b6a578700 1 -- 172.21.15.44:0/1319479138 >> 172.21.15.44:0/1319479138 conn(0x7f6b64074f30 msgr2=0x7f6b640fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.445+0000 7f6b6a578700 1 -- 172.21.15.44:0/1319479138 shutdown_connections 2023-09-10T01:03:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:03.446+0000 7f6b6a578700 1 -- 172.21.15.44:0/1319479138 wait complete. 2023-09-10T01:03:04.307 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: *** Caught signal (Segmentation fault) ** 2023-09-10T01:03:04.307 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: in thread 7ff2b9f29700 thread_name:md_log_replay 2023-09-10T01:03:04.307 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:03:04.308 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7ff2caad8cf0] 2023-09-10T01:03:04.308 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x559a4b06b30f] 2023-09-10T01:03:04.308 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559a4b06d543] 2023-09-10T01:03:04.308 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 4: (MDLog::_replay_thread()+0x647) [0x559a4b0d6e97] 2023-09-10T01:03:04.309 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559a4b0e0381] 2023-09-10T01:03:04.309 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff2caace1ca] 2023-09-10T01:03:04.309 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 7: clone() 2023-09-10T01:03:04.309 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 2023-09-10T01:03:03.993+0000 7ff2b9f29700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:03:04.309 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: in thread 7ff2b9f29700 thread_name:md_log_replay 2023-09-10T01:03:04.310 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 2023-09-10T01:03:04.310 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:03:04.310 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7ff2caad8cf0] 2023-09-10T01:03:04.310 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x559a4b06b30f] 2023-09-10T01:03:04.310 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559a4b06d543] 2023-09-10T01:03:04.310 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 4: (MDLog::_replay_thread()+0x647) [0x559a4b0d6e97] 2023-09-10T01:03:04.311 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559a4b0e0381] 2023-09-10T01:03:04.311 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff2caace1ca] 2023-09-10T01:03:04.311 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 7: clone() 2023-09-10T01:03:04.311 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:03:04.311 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 2023-09-10T01:03:04.312 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 0> 2023-09-10T01:03:03.993+0000 7ff2b9f29700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:03:04.312 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: in thread 7ff2b9f29700 thread_name:md_log_replay 2023-09-10T01:03:04.312 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 2023-09-10T01:03:04.312 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:03:04.312 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7ff2caad8cf0] 2023-09-10T01:03:04.313 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x559a4b06b30f] 2023-09-10T01:03:04.313 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559a4b06d543] 2023-09-10T01:03:04.313 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 4: (MDLog::_replay_thread()+0x647) [0x559a4b0d6e97] 2023-09-10T01:03:04.313 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559a4b0e0381] 2023-09-10T01:03:04.313 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff2caace1ca] 2023-09-10T01:03:04.313 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 7: clone() 2023-09-10T01:03:04.314 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:03:04.314 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 2023-09-10T01:03:04.314 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 0> 2023-09-10T01:03:03.993+0000 7ff2b9f29700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:03:04.314 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: in thread 7ff2b9f29700 thread_name:md_log_replay 2023-09-10T01:03:04.314 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 2023-09-10T01:03:04.315 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:03:04.315 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7ff2caad8cf0] 2023-09-10T01:03:04.315 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x559a4b06b30f] 2023-09-10T01:03:04.315 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559a4b06d543] 2023-09-10T01:03:04.315 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 4: (MDLog::_replay_thread()+0x647) [0x559a4b0d6e97] 2023-09-10T01:03:04.316 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559a4b0e0381] 2023-09-10T01:03:04.316 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff2caace1ca] 2023-09-10T01:03:04.316 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 7: clone() 2023-09-10T01:03:04.316 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:03:04.316 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[154129]: 2023-09-10T01:03:04.883 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:04 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Main process exited, code=exited, status=139/n/a 2023-09-10T01:03:05.120 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-mon[98676]: pgmap v709: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 23 MiB/s wr, 85 op/s 2023-09-10T01:03:05.121 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:04 smithi044 ceph-mon[98676]: osdmap e127: 12 total, 12 up, 12 in 2023-09-10T01:03:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:04 smithi131 ceph-mon[103713]: pgmap v709: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 23 MiB/s wr, 85 op/s 2023-09-10T01:03:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:04 smithi131 ceph-mon[103713]: osdmap e127: 12 total, 12 up, 12 in 2023-09-10T01:03:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:04 smithi099 ceph-mon[102692]: pgmap v709: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 23 MiB/s wr, 85 op/s 2023-09-10T01:03:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:04 smithi099 ceph-mon[102692]: osdmap e127: 12 total, 12 up, 12 in 2023-09-10T01:03:05.555 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:05 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Failed with result 'exit-code'. 2023-09-10T01:03:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:06 smithi131 ceph-mon[103713]: pgmap v711: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 195 op/s 2023-09-10T01:03:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:06 smithi044 ceph-mon[98676]: pgmap v711: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 195 op/s 2023-09-10T01:03:07.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:06 smithi099 ceph-mon[102692]: pgmap v711: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 195 op/s 2023-09-10T01:03:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:08 smithi131 ceph-mon[103713]: pgmap v712: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 195 op/s 2023-09-10T01:03:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:08 smithi044 ceph-mon[98676]: pgmap v712: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 195 op/s 2023-09-10T01:03:09.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:08 smithi099 ceph-mon[102692]: pgmap v712: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 195 op/s 2023-09-10T01:03:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:10 smithi131 ceph-mon[103713]: pgmap v713: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 195 op/s 2023-09-10T01:03:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:10 smithi131 ceph-mon[103713]: osdmap e128: 12 total, 12 up, 12 in 2023-09-10T01:03:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:10 smithi044 ceph-mon[98676]: pgmap v713: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 195 op/s 2023-09-10T01:03:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:10 smithi044 ceph-mon[98676]: osdmap e128: 12 total, 12 up, 12 in 2023-09-10T01:03:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:10 smithi099 ceph-mon[102692]: pgmap v713: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 195 op/s 2023-09-10T01:03:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:10 smithi099 ceph-mon[102692]: osdmap e128: 12 total, 12 up, 12 in 2023-09-10T01:03:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:12 smithi131 ceph-mon[103713]: pgmap v715: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 282 op/s 2023-09-10T01:03:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:12 smithi044 ceph-mon[98676]: pgmap v715: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 282 op/s 2023-09-10T01:03:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:12 smithi099 ceph-mon[102692]: pgmap v715: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 282 op/s 2023-09-10T01:03:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:14 smithi131 ceph-mon[103713]: pgmap v716: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 238 op/s 2023-09-10T01:03:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:14 smithi044 ceph-mon[98676]: pgmap v716: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 238 op/s 2023-09-10T01:03:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:14 smithi099 ceph-mon[102692]: pgmap v716: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 238 op/s 2023-09-10T01:03:15.648 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:15 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Service RestartSec=10s expired, scheduling restart. 2023-09-10T01:03:15.648 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:15 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Scheduled restart job, restart counter is at 1. 2023-09-10T01:03:15.648 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:15 smithi044 systemd[1]: Stopped Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:03:15.648 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:15 smithi044 systemd[1]: Starting Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:03:15.972 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:15 smithi044 podman[187048]: 2023-09-10T01:03:16.305 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:15 smithi044 bash[187048]: b732f6f9e3242cee953e7a67e463b59fc3eb4ee3f83a2886421bd90844ef02f9 2023-09-10T01:03:16.305 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:16 smithi044 ceph-mds[187088]: -- 172.21.15.44:0/3225017258 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bc297911e0 con 0x55bc29767000 2023-09-10T01:03:16.305 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:16 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: starting mds.d at 2023-09-10T01:03:16.306 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:03:16 smithi044 systemd[1]: Started Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:03:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.570+0000 7f882d466700 1 -- 172.21.15.44:0/4066682668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88280fea00 msgr2=0x7f88280fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.570+0000 7f882d466700 1 --2- 172.21.15.44:0/4066682668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88280fea00 0x7f88280fee80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f881c0099c0 tx=0x7f881c009cd0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.570+0000 7f882d466700 1 -- 172.21.15.44:0/4066682668 shutdown_connections 2023-09-10T01:03:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.570+0000 7f882d466700 1 --2- 172.21.15.44:0/4066682668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88280ff3c0 0x7f882810d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.570+0000 7f882d466700 1 --2- 172.21.15.44:0/4066682668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8828105570 0x7f8828105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.570+0000 7f882d466700 1 --2- 172.21.15.44:0/4066682668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88280fea00 0x7f88280fee80 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.570+0000 7f882d466700 1 -- 172.21.15.44:0/4066682668 >> 172.21.15.44:0/4066682668 conn(0x7f88280769d0 msgr2=0x7f88280fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.570+0000 7f882d466700 1 -- 172.21.15.44:0/4066682668 shutdown_connections 2023-09-10T01:03:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.570+0000 7f882d466700 1 -- 172.21.15.44:0/4066682668 wait complete. 2023-09-10T01:03:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.571+0000 7f882d466700 1 Processor -- start 2023-09-10T01:03:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.571+0000 7f882d466700 1 -- start start 2023-09-10T01:03:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f882d466700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88280fea00 0x7f882819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f882d466700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88280ff3c0 0x7f882819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f882d466700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8828105570 0x7f88281963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f882d466700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8828196a40 con 0x7f8828105570 2023-09-10T01:03:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f882d466700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8828196bb0 con 0x7f88280fea00 2023-09-10T01:03:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f882d466700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8828196d70 con 0x7f88280ff3c0 2023-09-10T01:03:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f8826ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88280fea00 0x7f882819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f88277fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8828105570 0x7f88281963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f8826ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88280fea00 0x7f882819c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56926/0 (socket says 172.21.15.44:56926) 2023-09-10T01:03:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f8826ffd700 1 -- 172.21.15.44:0/660219802 learned_addr learned my addr 172.21.15.44:0/660219802 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f88267fc700 1 --2- 172.21.15.44:0/660219802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88280ff3c0 0x7f882819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f8826ffd700 1 -- 172.21.15.44:0/660219802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88280ff3c0 msgr2=0x7f882819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f8826ffd700 1 --2- 172.21.15.44:0/660219802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88280ff3c0 0x7f882819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f8826ffd700 1 -- 172.21.15.44:0/660219802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8828105570 msgr2=0x7f88281963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f8826ffd700 1 --2- 172.21.15.44:0/660219802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8828105570 0x7f88281963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f8826ffd700 1 -- 172.21.15.44:0/660219802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881c0095f0 con 0x7f88280fea00 2023-09-10T01:03:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f88267fc700 1 --2- 172.21.15.44:0/660219802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88280ff3c0 0x7f882819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:03:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f88277fe700 1 --2- 172.21.15.44:0/660219802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8828105570 0x7f88281963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:03:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f8826ffd700 1 --2- 172.21.15.44:0/660219802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88280fea00 0x7f882819c570 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f881800e9a0 tx=0x7f881800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.572+0000 7f880ffff700 1 -- 172.21.15.44:0/660219802 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8818009790 con 0x7f88280fea00 2023-09-10T01:03:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.573+0000 7f880ffff700 1 -- 172.21.15.44:0/660219802 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f881800cbd0 con 0x7f88280fea00 2023-09-10T01:03:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.573+0000 7f882d466700 1 -- 172.21.15.44:0/660219802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8828197050 con 0x7f88280fea00 2023-09-10T01:03:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.573+0000 7f880ffff700 1 -- 172.21.15.44:0/660219802 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8818018420 con 0x7f88280fea00 2023-09-10T01:03:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.573+0000 7f882d466700 1 -- 172.21.15.44:0/660219802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88280ffcf0 con 0x7f88280fea00 2023-09-10T01:03:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.575+0000 7f880ffff700 1 -- 172.21.15.44:0/660219802 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f881800cd40 con 0x7f88280fea00 2023-09-10T01:03:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.575+0000 7f880ffff700 1 --2- 172.21.15.44:0/660219802 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f88100a07b0 0x7f88100a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.575+0000 7f880ffff700 1 -- 172.21.15.44:0/660219802 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8818014070 con 0x7f88280fea00 2023-09-10T01:03:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.576+0000 7f88267fc700 1 --2- 172.21.15.44:0/660219802 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f88100a07b0 0x7f88100a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.576+0000 7f88267fc700 1 --2- 172.21.15.44:0/660219802 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f88100a07b0 0x7f88100a2c70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f881c006010 tx=0x7f881c00b540 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.576+0000 7f883962cb80 1 Processor -- start 2023-09-10T01:03:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.576+0000 7f883962cb80 1 -- start start 2023-09-10T01:03:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.576+0000 7f883962cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c411a0 0x559d15c41580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.576+0000 7f883962cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c3b300 0x559d15c3b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.576+0000 7f883962cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c3bcc0 0x559d15c47f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.576+0000 7f883962cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d15b946d0 con 0x559d15c411a0 2023-09-10T01:03:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.576+0000 7f883962cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d15b91ff0 con 0x559d15c3b300 2023-09-10T01:03:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.576+0000 7f883962cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d15b8d8e0 con 0x559d15c3bcc0 2023-09-10T01:03:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.577+0000 7f88267fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c3b300 0x559d15c3b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.577+0000 7f88267fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c3b300 0x559d15c3b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56936/0 (socket says 172.21.15.44:56936) 2023-09-10T01:03:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.577+0000 7f88267fc700 1 -- 172.21.15.44:0/2918794676 learned_addr learned my addr 172.21.15.44:0/2918794676 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.577+0000 7f88277fe700 1 --2- 172.21.15.44:0/2918794676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c3bcc0 0x559d15c47f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.577+0000 7f88267fc700 1 -- 172.21.15.44:0/2918794676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c3bcc0 msgr2=0x559d15c47f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.577+0000 7f88267fc700 1 --2- 172.21.15.44:0/2918794676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c3bcc0 0x559d15c47f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.577+0000 7f88267fc700 1 -- 172.21.15.44:0/2918794676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c411a0 msgr2=0x559d15c41580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.577+0000 7f88267fc700 1 --2- 172.21.15.44:0/2918794676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c411a0 0x559d15c41580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.577+0000 7f88267fc700 1 -- 172.21.15.44:0/2918794676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d159b9d80 con 0x559d15c3b300 2023-09-10T01:03:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.577+0000 7f88267fc700 1 --2- 172.21.15.44:0/2918794676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c3b300 0x559d15c3b780 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f881c0053c0 tx=0x7f881c01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=579b5cfbd37f354e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.577+0000 7f880d7fa700 1 -- 172.21.15.44:0/2918794676 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c01d070 con 0x559d15c3b300 2023-09-10T01:03:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.577+0000 7f880d7fa700 1 -- 172.21.15.44:0/2918794676 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f881c0056d0 con 0x559d15c3b300 2023-09-10T01:03:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.577+0000 7f880d7fa700 1 -- 172.21.15.44:0/2918794676 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c02f7a0 con 0x559d15c3b300 2023-09-10T01:03:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.578+0000 7f883962cb80 1 -- 172.21.15.44:0/2918794676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c3b300 msgr2=0x559d15c3b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.578+0000 7f883962cb80 1 --2- 172.21.15.44:0/2918794676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c3b300 0x559d15c3b780 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f881c0053c0 tx=0x7f881c01d000 comp rx=0 tx=0).stop 2023-09-10T01:03:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.578+0000 7f883962cb80 1 -- 172.21.15.44:0/2918794676 shutdown_connections 2023-09-10T01:03:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.578+0000 7f883962cb80 1 --2- 172.21.15.44:0/2918794676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c3bcc0 0x559d15c47f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.578+0000 7f883962cb80 1 --2- 172.21.15.44:0/2918794676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c411a0 0x559d15c41580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.578+0000 7f883962cb80 1 --2- 172.21.15.44:0/2918794676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c3b300 0x559d15c3b780 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.578+0000 7f883962cb80 1 -- 172.21.15.44:0/2918794676 >> 172.21.15.44:0/2918794676 conn(0x559d15bb0030 msgr2=0x559d15c36560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.578+0000 7f883962cb80 1 -- 172.21.15.44:0/2918794676 shutdown_connections 2023-09-10T01:03:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.578+0000 7f883962cb80 1 -- 172.21.15.44:0/2918794676 wait complete. 2023-09-10T01:03:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.579+0000 7f883962cb80 1 -- start start 2023-09-10T01:03:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.579+0000 7f883962cb80 1 Processor -- start 2023-09-10T01:03:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.579+0000 7f883962cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c3b300 0x559d15c47e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.579+0000 7f883962cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 0x559d15c45200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.579+0000 7f883962cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c411a0 0x559d15c45880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.579+0000 7f883962cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d15b2bb10 con 0x559d15c3bcc0 2023-09-10T01:03:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.579+0000 7f883962cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d15b2b810 con 0x559d15c411a0 2023-09-10T01:03:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.579+0000 7f883962cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d15a0f8c0 con 0x559d15c3b300 2023-09-10T01:03:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.579+0000 7f883962cb80 10 client.0 _ll_register_callbacks cb 0x559d15a75a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:03:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.579+0000 7f88267fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 0x559d15c45200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.579+0000 7f88267fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 0x559d15c45200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55202/0 (socket says 172.21.15.44:55202) 2023-09-10T01:03:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.579+0000 7f88267fc700 1 -- 172.21.15.44:0/672831648 learned_addr learned my addr 172.21.15.44:0/672831648 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.579+0000 7f88277fe700 1 --2- 172.21.15.44:0/672831648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c411a0 0x559d15c45880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.579+0000 7f8826ffd700 1 --2- 172.21.15.44:0/672831648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c3b300 0x559d15c47e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.580+0000 7f88277fe700 1 -- 172.21.15.44:0/672831648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c3b300 msgr2=0x559d15c47e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.580+0000 7f88277fe700 1 --2- 172.21.15.44:0/672831648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c3b300 0x559d15c47e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.580+0000 7f88277fe700 1 -- 172.21.15.44:0/672831648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 msgr2=0x559d15c45200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.580+0000 7f88277fe700 1 --2- 172.21.15.44:0/672831648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 0x559d15c45200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.580+0000 7f88277fe700 1 -- 172.21.15.44:0/672831648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882810dd00 con 0x559d15c411a0 2023-09-10T01:03:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.580+0000 7f88267fc700 1 --2- 172.21.15.44:0/672831648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 0x559d15c45200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:03:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.580+0000 7f88277fe700 1 --2- 172.21.15.44:0/672831648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c411a0 0x559d15c45880 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8820010ee0 tx=0x7f88200086d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4d2528ac50b71ca server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.580+0000 7f880aff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:03:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.580+0000 7f880aff5700 1 -- 172.21.15.44:0/672831648 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820020070 con 0x559d15c411a0 2023-09-10T01:03:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.580+0000 7f883962cb80 1 -- 172.21.15.44:0/672831648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d15ae5490 con 0x559d15c411a0 2023-09-10T01:03:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.580+0000 7f880aff5700 1 -- 172.21.15.44:0/672831648 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8820004bb0 con 0x559d15c411a0 2023-09-10T01:03:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.580+0000 7f880aff5700 1 -- 172.21.15.44:0/672831648 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8820015070 con 0x559d15c411a0 2023-09-10T01:03:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.580+0000 7f883962cb80 10 client.28178 fetch_fsmap learned FSMap version 90 2023-09-10T01:03:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.580+0000 7f883962cb80 1 -- 172.21.15.44:0/672831648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) v3 -- 0x559d159a2c00 con 0x559d15c411a0 2023-09-10T01:03:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.580+0000 7f880aff5700 1 -- 172.21.15.44:0/672831648 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559d15ae5490 con 0x559d15c411a0 2023-09-10T01:03:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.581+0000 7f880aff5700 1 -- 172.21.15.44:0/672831648 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 90) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8820023020 con 0x559d15c411a0 2023-09-10T01:03:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.581+0000 7f883962cb80 10 client.28178 fetch_fsmap finished waiting for FSMap version 90 2023-09-10T01:03:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.581+0000 7f883962cb80 10 client.28178 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:03:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.581+0000 7f883962cb80 20 client.28178 populate_metadata read hostname 'smithi044' 2023-09-10T01:03:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.581+0000 7f883962cb80 1 --2- 172.21.15.44:0/672831648 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559d15c55680 0x559d15c57aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.581+0000 7f883962cb80 4 client.28178 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:03:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.581+0000 7f883962cb80 1 -- 172.21.15.44:0/672831648 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d15ba2840 con 0x559d15c55680 2023-09-10T01:03:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.581+0000 7f8826ffd700 1 --2- 172.21.15.44:0/672831648 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559d15c55680 0x559d15c57aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.581+0000 7f880aff5700 10 client.28178 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.581+0000 7f8826ffd700 1 --2- 172.21.15.44:0/672831648 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559d15c55680 0x559d15c57aa0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f96f4cee8e8e1a6f server_cookie=198dbf436bd92fe0 in_seq=0 out_seq=0 2023-09-10T01:03:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.582+0000 7f880aff5700 1 -- 172.21.15.44:0/672831648 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559d15ba2840 con 0x559d15c55680 2023-09-10T01:03:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.582+0000 7f880aff5700 10 client.28178 handle_command_reply: tid=0 2023-09-10T01:03:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.582+0000 7f883962cb80 1 client.28178 shutdown 2023-09-10T01:03:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.582+0000 7f883962cb80 1 -- 172.21.15.44:0/672831648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c411a0 msgr2=0x559d15c45880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.582+0000 7f883962cb80 1 --2- 172.21.15.44:0/672831648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c411a0 0x559d15c45880 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8820010ee0 tx=0x7f88200086d0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.582+0000 7f883962cb80 1 -- 172.21.15.44:0/672831648 shutdown_connections 2023-09-10T01:03:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.582+0000 7f883962cb80 1 --2- 172.21.15.44:0/672831648 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559d15c55680 0x559d15c57aa0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.582+0000 7f883962cb80 1 --2- 172.21.15.44:0/672831648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c411a0 0x559d15c45880 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.582+0000 7f880aff5700 0 client.28178 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.582+0000 7f883962cb80 1 --2- 172.21.15.44:0/672831648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c3b300 0x559d15c47e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.582+0000 7f883962cb80 1 --2- 172.21.15.44:0/672831648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 0x559d15c45200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.582+0000 7f883962cb80 1 -- 172.21.15.44:0/672831648 >> 172.21.15.44:0/672831648 conn(0x559d15bb0030 msgr2=0x559d15c357a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.582+0000 7f883962cb80 1 -- 172.21.15.44:0/672831648 shutdown_connections 2023-09-10T01:03:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.582+0000 7f883962cb80 1 -- 172.21.15.44:0/672831648 wait complete. 2023-09-10T01:03:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.582+0000 7f883962cb80 20 client.28178 trim_cache size 0 max 16384 2023-09-10T01:03:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.598+0000 7f883962cb80 1 Processor -- start 2023-09-10T01:03:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.598+0000 7f883962cb80 1 -- start start 2023-09-10T01:03:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.598+0000 7f883962cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c3b300 0x559d15c42e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.598+0000 7f883962cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 0x559d15c4c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.598+0000 7f883962cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c411a0 0x559d15c36060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.598+0000 7f883962cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d15c3c580 con 0x559d15c3bcc0 2023-09-10T01:03:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.598+0000 7f883962cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d15c45c10 con 0x559d15c411a0 2023-09-10T01:03:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.598+0000 7f883962cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d15c37d60 con 0x559d15c3b300 2023-09-10T01:03:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.599+0000 7f88267fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 0x559d15c4c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.599+0000 7f88267fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 0x559d15c4c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55210/0 (socket says 172.21.15.44:55210) 2023-09-10T01:03:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.599+0000 7f88277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c411a0 0x559d15c36060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.599+0000 7f8826ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c3b300 0x559d15c42e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.599+0000 7f88267fc700 1 -- 172.21.15.44:0/3562341525 learned_addr learned my addr 172.21.15.44:0/3562341525 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.599+0000 7f88267fc700 1 -- 172.21.15.44:0/3562341525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c3b300 msgr2=0x559d15c42e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.599+0000 7f88267fc700 1 --2- 172.21.15.44:0/3562341525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c3b300 0x559d15c42e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.599+0000 7f88267fc700 1 -- 172.21.15.44:0/3562341525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c411a0 msgr2=0x559d15c36060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.599+0000 7f88267fc700 1 --2- 172.21.15.44:0/3562341525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c411a0 0x559d15c36060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.599+0000 7f88267fc700 1 -- 172.21.15.44:0/3562341525 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d159e01f0 con 0x559d15c3bcc0 2023-09-10T01:03:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.599+0000 7f88277fe700 1 --2- 172.21.15.44:0/3562341525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c411a0 0x559d15c36060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:03:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.599+0000 7f88267fc700 1 --2- 172.21.15.44:0/3562341525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 0x559d15c4c530 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f881c0041a0 tx=0x7f881c003810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da6053be6ac8d97 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.599+0000 7f880dffb700 1 -- 172.21.15.44:0/3562341525 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c034070 con 0x559d15c3bcc0 2023-09-10T01:03:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.599+0000 7f880dffb700 1 -- 172.21.15.44:0/3562341525 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f881c003d90 con 0x559d15c3bcc0 2023-09-10T01:03:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.599+0000 7f880dffb700 1 -- 172.21.15.44:0/3562341525 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c00f460 con 0x559d15c3bcc0 2023-09-10T01:03:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.600+0000 7f883962cb80 1 -- 172.21.15.44:0/3562341525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 msgr2=0x559d15c4c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.600+0000 7f883962cb80 1 --2- 172.21.15.44:0/3562341525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 0x559d15c4c530 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f881c0041a0 tx=0x7f881c003810 comp rx=0 tx=0).stop 2023-09-10T01:03:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.600+0000 7f883962cb80 1 -- 172.21.15.44:0/3562341525 shutdown_connections 2023-09-10T01:03:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.600+0000 7f883962cb80 1 --2- 172.21.15.44:0/3562341525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c411a0 0x559d15c36060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.600+0000 7f883962cb80 1 --2- 172.21.15.44:0/3562341525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c3b300 0x559d15c42e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.600+0000 7f883962cb80 1 --2- 172.21.15.44:0/3562341525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 0x559d15c4c530 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.600+0000 7f883962cb80 1 -- 172.21.15.44:0/3562341525 >> 172.21.15.44:0/3562341525 conn(0x559d15bb0030 msgr2=0x559d15c3f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.600+0000 7f883962cb80 1 -- 172.21.15.44:0/3562341525 shutdown_connections 2023-09-10T01:03:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.600+0000 7f883962cb80 1 -- 172.21.15.44:0/3562341525 wait complete. 2023-09-10T01:03:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.601+0000 7f883962cb80 1 -- start start 2023-09-10T01:03:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.601+0000 7f883962cb80 1 Processor -- start 2023-09-10T01:03:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.601+0000 7f883962cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c3b300 0x559d15c9d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.601+0000 7f883962cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 0x559d15c9d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.601+0000 7f883962cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c411a0 0x559d15c9dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.601+0000 7f883962cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d15ca8560 con 0x559d15c3bcc0 2023-09-10T01:03:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.601+0000 7f883962cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d15ca86a0 con 0x559d15c3b300 2023-09-10T01:03:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.601+0000 7f883962cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d15ca8860 con 0x559d15c411a0 2023-09-10T01:03:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.601+0000 7f883962cb80 10 client.0 _ll_register_callbacks cb 0x7f8820009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:03:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.601+0000 7f8826ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c3b300 0x559d15c9d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.601+0000 7f8826ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c3b300 0x559d15c9d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56954/0 (socket says 172.21.15.44:56954) 2023-09-10T01:03:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.601+0000 7f88277fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c411a0 0x559d15c9dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.601+0000 7f8826ffd700 1 -- 172.21.15.44:0/1597619879 learned_addr learned my addr 172.21.15.44:0/1597619879 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.602+0000 7f8826ffd700 1 -- 172.21.15.44:0/1597619879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c411a0 msgr2=0x559d15c9dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.602+0000 7f8826ffd700 1 --2- 172.21.15.44:0/1597619879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c411a0 0x559d15c9dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.602+0000 7f8826ffd700 1 -- 172.21.15.44:0/1597619879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 msgr2=0x559d15c9d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.602+0000 7f8826ffd700 1 --2- 172.21.15.44:0/1597619879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 0x559d15c9d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.602+0000 7f8826ffd700 1 -- 172.21.15.44:0/1597619879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d159a2c00 con 0x559d15c3b300 2023-09-10T01:03:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.602+0000 7f8826ffd700 1 --2- 172.21.15.44:0/1597619879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c3b300 0x559d15c9d440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8818095ba0 tx=0x7f8818084040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4830c82bd4896fb8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.602+0000 7f880aff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:03:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.602+0000 7f880aff5700 1 -- 172.21.15.44:0/1597619879 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881809d360 con 0x559d15c3b300 2023-09-10T01:03:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.602+0000 7f880aff5700 1 -- 172.21.15.44:0/1597619879 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f88180a1760 con 0x559d15c3b300 2023-09-10T01:03:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.602+0000 7f883962cb80 1 -- 172.21.15.44:0/1597619879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f881c0056d0 con 0x559d15c3b300 2023-09-10T01:03:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.602+0000 7f880aff5700 1 -- 172.21.15.44:0/1597619879 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8818061ea0 con 0x559d15c3b300 2023-09-10T01:03:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.602+0000 7f883962cb80 10 client.28184 fetch_fsmap learned FSMap version 90 2023-09-10T01:03:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.602+0000 7f883962cb80 1 -- 172.21.15.44:0/1597619879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) v3 -- 0x559d159b9d80 con 0x559d15c3b300 2023-09-10T01:03:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.602+0000 7f880aff5700 1 -- 172.21.15.44:0/1597619879 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8818065030 con 0x559d15c3b300 2023-09-10T01:03:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.603+0000 7f880aff5700 1 -- 172.21.15.44:0/1597619879 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 90) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f88180a12d0 con 0x559d15c3b300 2023-09-10T01:03:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.603+0000 7f883962cb80 10 client.28184 fetch_fsmap finished waiting for FSMap version 90 2023-09-10T01:03:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.603+0000 7f883962cb80 10 client.28184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:03:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.603+0000 7f883962cb80 20 client.28184 populate_metadata read hostname 'smithi044' 2023-09-10T01:03:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.603+0000 7f883962cb80 1 --2- 172.21.15.44:0/1597619879 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559d15caa080 0x559d15cac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.603+0000 7f883962cb80 4 client.28184 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:03:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.603+0000 7f883962cb80 1 -- 172.21.15.44:0/1597619879 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559d15b868c0 con 0x559d15caa080 2023-09-10T01:03:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.603+0000 7f88267fc700 1 --2- 172.21.15.44:0/1597619879 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559d15caa080 0x559d15cac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.603+0000 7f880aff5700 10 client.28184 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.603+0000 7f88267fc700 1 --2- 172.21.15.44:0/1597619879 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559d15caa080 0x559d15cac4a0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=173e09b276e57e97 server_cookie=7293cb3cbfbab468 in_seq=0 out_seq=0 2023-09-10T01:03:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.618+0000 7f880aff5700 1 -- 172.21.15.44:0/1597619879 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559d15b868c0 con 0x559d15caa080 2023-09-10T01:03:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.618+0000 7f880aff5700 10 client.28184 handle_command_reply: tid=0 2023-09-10T01:03:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.618+0000 7f883962cb80 1 client.28184 shutdown 2023-09-10T01:03:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.619+0000 7f883962cb80 1 -- 172.21.15.44:0/1597619879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c3b300 msgr2=0x559d15c9d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.619+0000 7f883962cb80 1 --2- 172.21.15.44:0/1597619879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c3b300 0x559d15c9d440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8818095ba0 tx=0x7f8818084040 comp rx=0 tx=0).stop 2023-09-10T01:03:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.619+0000 7f883962cb80 1 -- 172.21.15.44:0/1597619879 shutdown_connections 2023-09-10T01:03:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.619+0000 7f883962cb80 1 --2- 172.21.15.44:0/1597619879 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559d15caa080 0x559d15cac4a0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.619+0000 7f883962cb80 1 --2- 172.21.15.44:0/1597619879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d15c411a0 0x559d15c9dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.619+0000 7f883962cb80 1 --2- 172.21.15.44:0/1597619879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d15c3b300 0x559d15c9d440 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.619+0000 7f883962cb80 1 --2- 172.21.15.44:0/1597619879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d15c3bcc0 0x559d15c9d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.619+0000 7f883962cb80 1 -- 172.21.15.44:0/1597619879 >> 172.21.15.44:0/1597619879 conn(0x559d15bb0030 msgr2=0x559d15c36060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.619+0000 7f880aff5700 0 client.28184 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.619+0000 7f883962cb80 1 -- 172.21.15.44:0/1597619879 shutdown_connections 2023-09-10T01:03:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.619+0000 7f883962cb80 1 -- 172.21.15.44:0/1597619879 wait complete. 2023-09-10T01:03:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.619+0000 7f883962cb80 20 client.28184 trim_cache size 0 max 16384 2023-09-10T01:03:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:03:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:03:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:03:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:03:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.619+0000 7f882d466700 1 -- 172.21.15.44:0/660219802 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f88100a07b0 msgr2=0x7f88100a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.619+0000 7f882d466700 1 --2- 172.21.15.44:0/660219802 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f88100a07b0 0x7f88100a2c70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f881c006010 tx=0x7f881c00b540 comp rx=0 tx=0).stop 2023-09-10T01:03:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.619+0000 7f882d466700 1 -- 172.21.15.44:0/660219802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88280fea00 msgr2=0x7f882819c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.619+0000 7f882d466700 1 --2- 172.21.15.44:0/660219802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88280fea00 0x7f882819c570 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f881800e9a0 tx=0x7f881800ecb0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.620+0000 7f882d466700 1 -- 172.21.15.44:0/660219802 shutdown_connections 2023-09-10T01:03:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.620+0000 7f882d466700 1 --2- 172.21.15.44:0/660219802 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f88100a07b0 0x7f88100a2c70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.620+0000 7f882d466700 1 --2- 172.21.15.44:0/660219802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8828105570 0x7f88281963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.620+0000 7f882d466700 1 --2- 172.21.15.44:0/660219802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88280fea00 0x7f882819c570 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.620+0000 7f882d466700 1 --2- 172.21.15.44:0/660219802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88280ff3c0 0x7f882819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.620+0000 7f882d466700 1 -- 172.21.15.44:0/660219802 >> 172.21.15.44:0/660219802 conn(0x7f88280769d0 msgr2=0x7f8828103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.620+0000 7f882d466700 1 -- 172.21.15.44:0/660219802 shutdown_connections 2023-09-10T01:03:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:16.620+0000 7f882d466700 1 -- 172.21.15.44:0/660219802 wait complete. 2023-09-10T01:03:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:16 smithi131 ceph-mon[103713]: pgmap v717: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2023-09-10T01:03:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:16 smithi131 ceph-mon[103713]: daemon mds.d restarted 2023-09-10T01:03:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:16 smithi044 ceph-mon[98676]: pgmap v717: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2023-09-10T01:03:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:16 smithi044 ceph-mon[98676]: daemon mds.d restarted 2023-09-10T01:03:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:16 smithi099 ceph-mon[102692]: pgmap v717: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2023-09-10T01:03:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:16 smithi099 ceph-mon[102692]: daemon mds.d restarted 2023-09-10T01:03:18.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:17 smithi131 ceph-mon[103713]: osdmap e129: 12 total, 12 up, 12 in 2023-09-10T01:03:18.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:17 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.44:6836/2331511231,v1:172.21.15.44:6837/2331511231] up:boot 2023-09-10T01:03:18.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:17 smithi131 ceph-mon[103713]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-09-10T01:03:18.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:17 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T01:03:18.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:17 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:17 smithi044 ceph-mon[98676]: osdmap e129: 12 total, 12 up, 12 in 2023-09-10T01:03:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:17 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.44:6836/2331511231,v1:172.21.15.44:6837/2331511231] up:boot 2023-09-10T01:03:18.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:17 smithi044 ceph-mon[98676]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-09-10T01:03:18.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:17 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T01:03:18.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:17 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:18.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:17 smithi099 ceph-mon[102692]: osdmap e129: 12 total, 12 up, 12 in 2023-09-10T01:03:18.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:17 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.44:6836/2331511231,v1:172.21.15.44:6837/2331511231] up:boot 2023-09-10T01:03:18.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:17 smithi099 ceph-mon[102692]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-09-10T01:03:18.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:17 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T01:03:18.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:17 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:18 smithi131 ceph-mon[103713]: pgmap v719: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 273 op/s 2023-09-10T01:03:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:18 smithi044 ceph-mon[98676]: pgmap v719: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 273 op/s 2023-09-10T01:03:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:18 smithi099 ceph-mon[102692]: pgmap v719: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 273 op/s 2023-09-10T01:03:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:20 smithi131 ceph-mon[103713]: pgmap v720: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 917 KiB/s wr, 118 op/s 2023-09-10T01:03:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:20 smithi044 ceph-mon[98676]: pgmap v720: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 917 KiB/s wr, 118 op/s 2023-09-10T01:03:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:20 smithi099 ceph-mon[102692]: pgmap v720: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 917 KiB/s wr, 118 op/s 2023-09-10T01:03:22.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:21 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:03:22.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:21 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:03:22.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:21 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:03:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:22 smithi131 ceph-mon[103713]: pgmap v721: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 208 op/s 2023-09-10T01:03:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:22 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:22 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:22 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:22 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:22 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:22 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:22 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:03:23.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:22 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:03:23.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:22 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:22 smithi044 ceph-mon[98676]: pgmap v721: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 208 op/s 2023-09-10T01:03:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:22 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:22 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:22 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:22 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:22 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:22 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:22 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:03:23.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:22 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:03:23.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:22 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:22 smithi099 ceph-mon[102692]: pgmap v721: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 208 op/s 2023-09-10T01:03:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:22 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:22 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:22 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:22 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:22 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:22 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:23.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:22 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:03:23.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:22 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:03:23.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:22 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:03:25.299 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:24 smithi131 ceph-mon[103713]: pgmap v722: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 208 op/s 2023-09-10T01:03:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:24 smithi044 ceph-mon[98676]: pgmap v722: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 208 op/s 2023-09-10T01:03:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:24 smithi099 ceph-mon[102692]: pgmap v722: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 208 op/s 2023-09-10T01:03:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:26 smithi131 ceph-mon[103713]: pgmap v723: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 215 op/s 2023-09-10T01:03:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:26 smithi044 ceph-mon[98676]: pgmap v723: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 215 op/s 2023-09-10T01:03:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:26 smithi099 ceph-mon[102692]: pgmap v723: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 215 op/s 2023-09-10T01:03:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:28 smithi131 ceph-mon[103713]: pgmap v724: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-09-10T01:03:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:28 smithi044 ceph-mon[98676]: pgmap v724: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-09-10T01:03:29.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:28 smithi099 ceph-mon[102692]: pgmap v724: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-09-10T01:03:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.742+0000 7f858c58f700 1 -- 172.21.15.44:0/2626757740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85840ff080 msgr2=0x7f85840ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.742+0000 7f858c58f700 1 --2- 172.21.15.44:0/2626757740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85840ff080 0x7f85840ff460 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8580009990 tx=0x7f8580009ca0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.742+0000 7f858c58f700 1 -- 172.21.15.44:0/2626757740 shutdown_connections 2023-09-10T01:03:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.742+0000 7f858c58f700 1 --2- 172.21.15.44:0/2626757740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8584105950 0x7f858410d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.742+0000 7f858c58f700 1 --2- 172.21.15.44:0/2626757740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85840ff080 0x7f85840ff460 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.742+0000 7f858c58f700 1 --2- 172.21.15.44:0/2626757740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85840ffa30 0x7f8584105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.742+0000 7f858c58f700 1 -- 172.21.15.44:0/2626757740 >> 172.21.15.44:0/2626757740 conn(0x7f8584075960 msgr2=0x7f8584075d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.742+0000 7f858c58f700 1 -- 172.21.15.44:0/2626757740 shutdown_connections 2023-09-10T01:03:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.742+0000 7f858c58f700 1 -- 172.21.15.44:0/2626757740 wait complete. 2023-09-10T01:03:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.743+0000 7f858c58f700 1 Processor -- start 2023-09-10T01:03:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.743+0000 7f858c58f700 1 -- start start 2023-09-10T01:03:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.743+0000 7f858c58f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85840ff080 0x7f85841001d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.743+0000 7f858c58f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8584105950 0x7f8584100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.743+0000 7f858c58f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85841a2790 0x7f85841a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.743+0000 7f858c58f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85841a50c0 con 0x7f85841a2790 2023-09-10T01:03:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.743+0000 7f858c58f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85841a5200 con 0x7f85840ff080 2023-09-10T01:03:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.743+0000 7f858c58f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85841a5370 con 0x7f8584105950 2023-09-10T01:03:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.743+0000 7f858ab2c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85841a2790 0x7f85841a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.743+0000 7f8589b2a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8584105950 0x7f8584100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.743+0000 7f858ab2c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85841a2790 0x7f85841a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55234/0 (socket says 172.21.15.44:55234) 2023-09-10T01:03:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.743+0000 7f858a32b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85840ff080 0x7f85841001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.743+0000 7f858ab2c700 1 -- 172.21.15.44:0/2468186238 learned_addr learned my addr 172.21.15.44:0/2468186238 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.744+0000 7f858ab2c700 1 -- 172.21.15.44:0/2468186238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8584105950 msgr2=0x7f8584100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.744+0000 7f858ab2c700 1 --2- 172.21.15.44:0/2468186238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8584105950 0x7f8584100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.744+0000 7f858ab2c700 1 -- 172.21.15.44:0/2468186238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85840ff080 msgr2=0x7f85841001d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.744+0000 7f858ab2c700 1 --2- 172.21.15.44:0/2468186238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85840ff080 0x7f85841001d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.744+0000 7f858ab2c700 1 -- 172.21.15.44:0/2468186238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85800095f0 con 0x7f85841a2790 2023-09-10T01:03:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.744+0000 7f858ab2c700 1 --2- 172.21.15.44:0/2468186238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85841a2790 0x7f85841a4b80 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f857c00efc0 tx=0x7f857c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.744+0000 7f857b7fe700 1 -- 172.21.15.44:0/2468186238 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c018070 con 0x7f85841a2790 2023-09-10T01:03:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.744+0000 7f857b7fe700 1 -- 172.21.15.44:0/2468186238 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f857c00f040 con 0x7f85841a2790 2023-09-10T01:03:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.744+0000 7f858c58f700 1 -- 172.21.15.44:0/2468186238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85841a55f0 con 0x7f85841a2790 2023-09-10T01:03:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.744+0000 7f857b7fe700 1 -- 172.21.15.44:0/2468186238 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c00cb00 con 0x7f85841a2790 2023-09-10T01:03:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.744+0000 7f858c58f700 1 -- 172.21.15.44:0/2468186238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85841a5ab0 con 0x7f85841a2790 2023-09-10T01:03:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.747+0000 7f857b7fe700 1 -- 172.21.15.44:0/2468186238 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f857c00cce0 con 0x7f85841a2790 2023-09-10T01:03:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.747+0000 7f857b7fe700 1 --2- 172.21.15.44:0/2468186238 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f85700a07b0 0x7f85700a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.748+0000 7f857b7fe700 1 -- 172.21.15.44:0/2468186238 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f857c0cbde0 con 0x7f85841a2790 2023-09-10T01:03:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.748+0000 7f858a32b700 1 --2- 172.21.15.44:0/2468186238 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f85700a07b0 0x7f85700a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.748+0000 7f858a32b700 1 --2- 172.21.15.44:0/2468186238 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f85700a07b0 0x7f85700a2c70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f85800059d0 tx=0x7f8580005960 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.748+0000 7f8598755b80 1 Processor -- start 2023-09-10T01:03:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f8598755b80 1 -- start start 2023-09-10T01:03:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f8598755b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257da31a0 0x55a257da3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f8598755b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257d9d300 0x55a257d9d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f8598755b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 0x55a257da9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f8598755b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a257cf66d0 con 0x55a257d9dcc0 2023-09-10T01:03:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f8598755b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a257cf3ff0 con 0x55a257da31a0 2023-09-10T01:03:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f8598755b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a257cef8e0 con 0x55a257d9d300 2023-09-10T01:03:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f858a32b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257da31a0 0x55a257da3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f858ab2c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 0x55a257da9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f8589b2a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257d9d300 0x55a257d9d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f858a32b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257da31a0 0x55a257da3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56978/0 (socket says 172.21.15.44:56978) 2023-09-10T01:03:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f858a32b700 1 -- 172.21.15.44:0/1780591645 learned_addr learned my addr 172.21.15.44:0/1780591645 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f858ab2c700 1 -- 172.21.15.44:0/1780591645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257d9d300 msgr2=0x55a257d9d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f858ab2c700 1 --2- 172.21.15.44:0/1780591645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257d9d300 0x55a257d9d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f858ab2c700 1 -- 172.21.15.44:0/1780591645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257da31a0 msgr2=0x55a257da3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f858ab2c700 1 --2- 172.21.15.44:0/1780591645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257da31a0 0x55a257da3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.749+0000 7f858ab2c700 1 -- 172.21.15.44:0/1780591645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a257b1bd80 con 0x55a257d9dcc0 2023-09-10T01:03:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.750+0000 7f858ab2c700 1 --2- 172.21.15.44:0/1780591645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 0x55a257da9f30 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f857c0c9130 tx=0x7f857c098cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba378eba4231cd1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.750+0000 7f8578ff9700 1 -- 172.21.15.44:0/1780591645 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c099580 con 0x55a257d9dcc0 2023-09-10T01:03:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.750+0000 7f8578ff9700 1 -- 172.21.15.44:0/1780591645 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f857c099e90 con 0x55a257d9dcc0 2023-09-10T01:03:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.750+0000 7f8578ff9700 1 -- 172.21.15.44:0/1780591645 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c099340 con 0x55a257d9dcc0 2023-09-10T01:03:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.750+0000 7f8598755b80 1 -- 172.21.15.44:0/1780591645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 msgr2=0x55a257da9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.750+0000 7f8598755b80 1 --2- 172.21.15.44:0/1780591645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 0x55a257da9f30 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f857c0c9130 tx=0x7f857c098cd0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.751+0000 7f8598755b80 1 -- 172.21.15.44:0/1780591645 shutdown_connections 2023-09-10T01:03:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.751+0000 7f8598755b80 1 --2- 172.21.15.44:0/1780591645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 0x55a257da9f30 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.751+0000 7f8598755b80 1 --2- 172.21.15.44:0/1780591645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257da31a0 0x55a257da3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.751+0000 7f8598755b80 1 --2- 172.21.15.44:0/1780591645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257d9d300 0x55a257d9d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.751+0000 7f8598755b80 1 -- 172.21.15.44:0/1780591645 >> 172.21.15.44:0/1780591645 conn(0x55a257d12030 msgr2=0x55a257d98560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.751+0000 7f8598755b80 1 -- 172.21.15.44:0/1780591645 shutdown_connections 2023-09-10T01:03:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.751+0000 7f8598755b80 1 -- 172.21.15.44:0/1780591645 wait complete. 2023-09-10T01:03:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.751+0000 7f8598755b80 1 -- start start 2023-09-10T01:03:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.751+0000 7f8598755b80 1 Processor -- start 2023-09-10T01:03:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.751+0000 7f8598755b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 0x55a257da9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.751+0000 7f8598755b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 0x55a257da70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.751+0000 7f8598755b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257da31a0 0x55a257da7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.751+0000 7f8598755b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a257c8db10 con 0x55a257d9dcc0 2023-09-10T01:03:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.751+0000 7f8598755b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a257c8d810 con 0x55a257d9d300 2023-09-10T01:03:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.751+0000 7f8598755b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a257b718c0 con 0x55a257da31a0 2023-09-10T01:03:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.752+0000 7f8598755b80 10 client.0 _ll_register_callbacks cb 0x55a257bd7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:03:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.752+0000 7f858a32b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 0x55a257da9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.752+0000 7f858a32b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 0x55a257da9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:56984/0 (socket says 172.21.15.44:56984) 2023-09-10T01:03:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.752+0000 7f858ab2c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257da31a0 0x55a257da7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.752+0000 7f858a32b700 1 -- 172.21.15.44:0/754181785 learned_addr learned my addr 172.21.15.44:0/754181785 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.752+0000 7f8589b2a700 1 --2- 172.21.15.44:0/754181785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 0x55a257da70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.752+0000 7f858a32b700 1 -- 172.21.15.44:0/754181785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257da31a0 msgr2=0x55a257da7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.752+0000 7f858a32b700 1 --2- 172.21.15.44:0/754181785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257da31a0 0x55a257da7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.752+0000 7f858a32b700 1 -- 172.21.15.44:0/754181785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 msgr2=0x55a257da70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.752+0000 7f858a32b700 1 --2- 172.21.15.44:0/754181785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 0x55a257da70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.752+0000 7f858a32b700 1 -- 172.21.15.44:0/754181785 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85841a5ab0 con 0x55a257d9d300 2023-09-10T01:03:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.752+0000 7f858a32b700 1 --2- 172.21.15.44:0/754181785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 0x55a257da9e40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8580006010 tx=0x7f8580009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8edeeec760d84de server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.752+0000 7f856dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:03:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.752+0000 7f856dffb700 1 -- 172.21.15.44:0/754181785 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858000be80 con 0x55a257d9d300 2023-09-10T01:03:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.752+0000 7f8598755b80 1 -- 172.21.15.44:0/754181785 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a257c47490 con 0x55a257d9d300 2023-09-10T01:03:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.752+0000 7f856dffb700 1 -- 172.21.15.44:0/754181785 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f8580004dd0 con 0x55a257d9d300 2023-09-10T01:03:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.753+0000 7f856dffb700 1 -- 172.21.15.44:0/754181785 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f858000f460 con 0x55a257d9d300 2023-09-10T01:03:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.753+0000 7f8598755b80 10 client.28196 fetch_fsmap learned FSMap version 92 2023-09-10T01:03:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.753+0000 7f8598755b80 1 -- 172.21.15.44:0/754181785 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) v3 -- 0x55a257b04c00 con 0x55a257d9d300 2023-09-10T01:03:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.753+0000 7f856dffb700 1 -- 172.21.15.44:0/754181785 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a257c47490 con 0x55a257d9d300 2023-09-10T01:03:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.753+0000 7f856dffb700 1 -- 172.21.15.44:0/754181785 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 92) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f858002f070 con 0x55a257d9d300 2023-09-10T01:03:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.753+0000 7f8598755b80 10 client.28196 fetch_fsmap finished waiting for FSMap version 92 2023-09-10T01:03:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.753+0000 7f8598755b80 10 client.28196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:03:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.753+0000 7f8598755b80 20 client.28196 populate_metadata read hostname 'smithi044' 2023-09-10T01:03:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.753+0000 7f8598755b80 1 --2- 172.21.15.44:0/754181785 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a257db7520 0x55a257db9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.753+0000 7f8598755b80 4 client.28196 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:03:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.753+0000 7f8598755b80 1 -- 172.21.15.44:0/754181785 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a257d04840 con 0x55a257db7520 2023-09-10T01:03:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.753+0000 7f8589b2a700 1 --2- 172.21.15.44:0/754181785 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a257db7520 0x55a257db9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.754+0000 7f856dffb700 10 client.28196 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.754+0000 7f8589b2a700 1 --2- 172.21.15.44:0/754181785 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a257db7520 0x55a257db9940 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dfac23d3be26127 server_cookie=1ccdb548173adfa8 in_seq=0 out_seq=0 2023-09-10T01:03:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.754+0000 7f856dffb700 1 -- 172.21.15.44:0/754181785 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a257d04840 con 0x55a257db7520 2023-09-10T01:03:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.754+0000 7f856dffb700 10 client.28196 handle_command_reply: tid=0 2023-09-10T01:03:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.754+0000 7f8598755b80 1 client.28196 shutdown 2023-09-10T01:03:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.754+0000 7f8598755b80 1 -- 172.21.15.44:0/754181785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 msgr2=0x55a257da9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.754+0000 7f8598755b80 1 --2- 172.21.15.44:0/754181785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 0x55a257da9e40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8580006010 tx=0x7f8580009ca0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.754+0000 7f8598755b80 1 -- 172.21.15.44:0/754181785 shutdown_connections 2023-09-10T01:03:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.754+0000 7f8598755b80 1 --2- 172.21.15.44:0/754181785 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a257db7520 0x55a257db9940 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.754+0000 7f8598755b80 1 --2- 172.21.15.44:0/754181785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257da31a0 0x55a257da7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.754+0000 7f8598755b80 1 --2- 172.21.15.44:0/754181785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 0x55a257da9e40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.754+0000 7f856dffb700 0 client.28196 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.754+0000 7f8598755b80 1 --2- 172.21.15.44:0/754181785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 0x55a257da70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.754+0000 7f8598755b80 1 -- 172.21.15.44:0/754181785 >> 172.21.15.44:0/754181785 conn(0x55a257d12030 msgr2=0x55a257d977a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.755+0000 7f8598755b80 1 -- 172.21.15.44:0/754181785 shutdown_connections 2023-09-10T01:03:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.755+0000 7f8598755b80 1 -- 172.21.15.44:0/754181785 wait complete. 2023-09-10T01:03:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.755+0000 7f8598755b80 20 client.28196 trim_cache size 0 max 16384 2023-09-10T01:03:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.772+0000 7f8598755b80 1 Processor -- start 2023-09-10T01:03:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.772+0000 7f8598755b80 1 -- start start 2023-09-10T01:03:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f8598755b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 0x55a257da9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f8598755b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257d9dcc0 0x55a257da9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f8598755b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257da31a0 0x55a257da6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f8598755b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a257d99930 con 0x55a257da31a0 2023-09-10T01:03:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f8598755b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a257db4a50 con 0x55a257d9d300 2023-09-10T01:03:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f8598755b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a257d99320 con 0x55a257d9dcc0 2023-09-10T01:03:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f858ab2c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257da31a0 0x55a257da6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f8589b2a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257d9dcc0 0x55a257da9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f8589b2a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257d9dcc0 0x55a257da9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33904/0 (socket says 172.21.15.44:33904) 2023-09-10T01:03:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f858a32b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 0x55a257da9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f858ab2c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257da31a0 0x55a257da6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55256/0 (socket says 172.21.15.44:55256) 2023-09-10T01:03:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f8589b2a700 1 -- 172.21.15.44:0/246785274 learned_addr learned my addr 172.21.15.44:0/246785274 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f858ab2c700 1 -- 172.21.15.44:0/246785274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257d9dcc0 msgr2=0x55a257da9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f858ab2c700 1 --2- 172.21.15.44:0/246785274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257d9dcc0 0x55a257da9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f858ab2c700 1 -- 172.21.15.44:0/246785274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 msgr2=0x55a257da9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f858ab2c700 1 --2- 172.21.15.44:0/246785274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 0x55a257da9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f858ab2c700 1 -- 172.21.15.44:0/246785274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a257b421f0 con 0x55a257da31a0 2023-09-10T01:03:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f8589b2a700 1 --2- 172.21.15.44:0/246785274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257d9dcc0 0x55a257da9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:03:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.773+0000 7f858ab2c700 1 --2- 172.21.15.44:0/246785274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257da31a0 0x55a257da6990 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f857c0971d0 tx=0x7f857c0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2647b62fc3de3a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.774+0000 7f85797fa700 1 -- 172.21.15.44:0/246785274 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c099580 con 0x55a257da31a0 2023-09-10T01:03:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.774+0000 7f85797fa700 1 -- 172.21.15.44:0/246785274 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f857c09ccb0 con 0x55a257da31a0 2023-09-10T01:03:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.774+0000 7f85797fa700 1 -- 172.21.15.44:0/246785274 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c0990e0 con 0x55a257da31a0 2023-09-10T01:03:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.774+0000 7f8598755b80 1 -- 172.21.15.44:0/246785274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257da31a0 msgr2=0x55a257da6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.774+0000 7f8598755b80 1 --2- 172.21.15.44:0/246785274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257da31a0 0x55a257da6990 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f857c0971d0 tx=0x7f857c0092e0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.774+0000 7f8598755b80 1 -- 172.21.15.44:0/246785274 shutdown_connections 2023-09-10T01:03:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.774+0000 7f8598755b80 1 --2- 172.21.15.44:0/246785274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257da31a0 0x55a257da6990 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.774+0000 7f8598755b80 1 --2- 172.21.15.44:0/246785274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 0x55a257da9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.774+0000 7f8598755b80 1 --2- 172.21.15.44:0/246785274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257d9dcc0 0x55a257da9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.774+0000 7f8598755b80 1 -- 172.21.15.44:0/246785274 >> 172.21.15.44:0/246785274 conn(0x55a257d12030 msgr2=0x55a257d98090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.774+0000 7f8598755b80 1 -- 172.21.15.44:0/246785274 shutdown_connections 2023-09-10T01:03:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.774+0000 7f8598755b80 1 -- 172.21.15.44:0/246785274 wait complete. 2023-09-10T01:03:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.775+0000 7f8598755b80 1 -- start start 2023-09-10T01:03:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.775+0000 7f8598755b80 1 Processor -- start 2023-09-10T01:03:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.775+0000 7f8598755b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 0x55a257dac700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.775+0000 7f8598755b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 0x55a257dacc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.775+0000 7f8598755b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257da31a0 0x55a257e0a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.775+0000 7f8598755b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a257e0a850 con 0x55a257d9dcc0 2023-09-10T01:03:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.775+0000 7f8598755b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a257e0a990 con 0x55a257d9d300 2023-09-10T01:03:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.775+0000 7f8598755b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a257e0ab50 con 0x55a257da31a0 2023-09-10T01:03:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.775+0000 7f8598755b80 10 client.0 _ll_register_callbacks cb 0x55a257bd7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:03:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.775+0000 7f8589b2a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 0x55a257dacc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.775+0000 7f858a32b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 0x55a257dac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.775+0000 7f8589b2a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 0x55a257dacc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55260/0 (socket says 172.21.15.44:55260) 2023-09-10T01:03:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.775+0000 7f8589b2a700 1 -- 172.21.15.44:0/1445925773 learned_addr learned my addr 172.21.15.44:0/1445925773 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.775+0000 7f858ab2c700 1 --2- 172.21.15.44:0/1445925773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257da31a0 0x55a257e0a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.776+0000 7f858a32b700 1 -- 172.21.15.44:0/1445925773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257da31a0 msgr2=0x55a257e0a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.776+0000 7f858a32b700 1 --2- 172.21.15.44:0/1445925773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257da31a0 0x55a257e0a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.776+0000 7f858a32b700 1 -- 172.21.15.44:0/1445925773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 msgr2=0x55a257dacc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.776+0000 7f858a32b700 1 --2- 172.21.15.44:0/1445925773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 0x55a257dacc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.776+0000 7f858a32b700 1 -- 172.21.15.44:0/1445925773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8574004040 con 0x55a257d9d300 2023-09-10T01:03:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.776+0000 7f858a32b700 1 --2- 172.21.15.44:0/1445925773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 0x55a257dac700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f85800246e0 tx=0x7f8580005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38bc600acb9a21ad server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.776+0000 7f856dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:03:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.776+0000 7f856dffb700 1 -- 172.21.15.44:0/1445925773 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8580024a60 con 0x55a257d9d300 2023-09-10T01:03:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.776+0000 7f856dffb700 1 -- 172.21.15.44:0/1445925773 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f858000f460 con 0x55a257d9d300 2023-09-10T01:03:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.776+0000 7f8598755b80 1 -- 172.21.15.44:0/1445925773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f857c099e90 con 0x55a257d9d300 2023-09-10T01:03:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.776+0000 7f856dffb700 1 -- 172.21.15.44:0/1445925773 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f858000f620 con 0x55a257d9d300 2023-09-10T01:03:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.776+0000 7f8598755b80 10 client.28202 fetch_fsmap learned FSMap version 92 2023-09-10T01:03:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.776+0000 7f8598755b80 1 -- 172.21.15.44:0/1445925773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) v3 -- 0x55a257b1bd80 con 0x55a257d9d300 2023-09-10T01:03:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.776+0000 7f856dffb700 1 -- 172.21.15.44:0/1445925773 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f857c099e90 con 0x55a257d9d300 2023-09-10T01:03:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.777+0000 7f856dffb700 1 -- 172.21.15.44:0/1445925773 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 92) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8580004500 con 0x55a257d9d300 2023-09-10T01:03:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.777+0000 7f8598755b80 10 client.28202 fetch_fsmap finished waiting for FSMap version 92 2023-09-10T01:03:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.777+0000 7f8598755b80 10 client.28202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:03:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.777+0000 7f8598755b80 20 client.28202 populate_metadata read hostname 'smithi044' 2023-09-10T01:03:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.777+0000 7f8598755b80 1 --2- 172.21.15.44:0/1445925773 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a257e0bf40 0x55a257e0e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.777+0000 7f8598755b80 4 client.28202 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:03:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.777+0000 7f8598755b80 1 -- 172.21.15.44:0/1445925773 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a257ce88c0 con 0x55a257e0bf40 2023-09-10T01:03:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.777+0000 7f8589b2a700 1 --2- 172.21.15.44:0/1445925773 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a257e0bf40 0x55a257e0e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.778+0000 7f856dffb700 10 client.28202 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.778+0000 7f8589b2a700 1 --2- 172.21.15.44:0/1445925773 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a257e0bf40 0x55a257e0e360 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fdc60739b0fb047 server_cookie=6d1e53d30faaec2a in_seq=0 out_seq=0 2023-09-10T01:03:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.788+0000 7f856dffb700 1 -- 172.21.15.44:0/1445925773 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a257ce88c0 con 0x55a257e0bf40 2023-09-10T01:03:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.788+0000 7f856dffb700 10 client.28202 handle_command_reply: tid=0 2023-09-10T01:03:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.788+0000 7f8598755b80 1 client.28202 shutdown 2023-09-10T01:03:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.788+0000 7f8598755b80 1 -- 172.21.15.44:0/1445925773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 msgr2=0x55a257dac700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.788+0000 7f8598755b80 1 --2- 172.21.15.44:0/1445925773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 0x55a257dac700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f85800246e0 tx=0x7f8580005760 comp rx=0 tx=0).stop 2023-09-10T01:03:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.788+0000 7f8598755b80 1 -- 172.21.15.44:0/1445925773 shutdown_connections 2023-09-10T01:03:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.788+0000 7f8598755b80 1 --2- 172.21.15.44:0/1445925773 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a257e0bf40 0x55a257e0e360 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.788+0000 7f8598755b80 1 --2- 172.21.15.44:0/1445925773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a257da31a0 0x55a257e0a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.788+0000 7f856dffb700 0 client.28202 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.788+0000 7f8598755b80 1 --2- 172.21.15.44:0/1445925773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a257d9d300 0x55a257dac700 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.788+0000 7f8598755b80 1 --2- 172.21.15.44:0/1445925773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a257d9dcc0 0x55a257dacc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.788+0000 7f8598755b80 1 -- 172.21.15.44:0/1445925773 >> 172.21.15.44:0/1445925773 conn(0x55a257d12030 msgr2=0x55a257da6920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.788+0000 7f8598755b80 1 -- 172.21.15.44:0/1445925773 shutdown_connections 2023-09-10T01:03:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.788+0000 7f8598755b80 1 -- 172.21.15.44:0/1445925773 wait complete. 2023-09-10T01:03:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.788+0000 7f8598755b80 20 client.28202 trim_cache size 0 max 16384 2023-09-10T01:03:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:03:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:03:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:03:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:03:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.789+0000 7f858c58f700 1 -- 172.21.15.44:0/2468186238 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f85700a07b0 msgr2=0x7f85700a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.789+0000 7f858c58f700 1 --2- 172.21.15.44:0/2468186238 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f85700a07b0 0x7f85700a2c70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f85800059d0 tx=0x7f8580005960 comp rx=0 tx=0).stop 2023-09-10T01:03:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.789+0000 7f858c58f700 1 -- 172.21.15.44:0/2468186238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85841a2790 msgr2=0x7f85841a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.789+0000 7f858c58f700 1 --2- 172.21.15.44:0/2468186238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85841a2790 0x7f85841a4b80 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f857c00efc0 tx=0x7f857c00c3a0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.789+0000 7f858c58f700 1 -- 172.21.15.44:0/2468186238 shutdown_connections 2023-09-10T01:03:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.789+0000 7f858c58f700 1 --2- 172.21.15.44:0/2468186238 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f85700a07b0 0x7f85700a2c70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.789+0000 7f858c58f700 1 --2- 172.21.15.44:0/2468186238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85841a2790 0x7f85841a4b80 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.789+0000 7f858c58f700 1 --2- 172.21.15.44:0/2468186238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85840ff080 0x7f85841001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.789+0000 7f858c58f700 1 --2- 172.21.15.44:0/2468186238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8584105950 0x7f8584100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.789+0000 7f858c58f700 1 -- 172.21.15.44:0/2468186238 >> 172.21.15.44:0/2468186238 conn(0x7f8584075960 msgr2=0x7f85840fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.789+0000 7f858c58f700 1 -- 172.21.15.44:0/2468186238 shutdown_connections 2023-09-10T01:03:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:29.789+0000 7f858c58f700 1 -- 172.21.15.44:0/2468186238 wait complete. 2023-09-10T01:03:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:30 smithi131 ceph-mon[103713]: pgmap v725: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 179 op/s 2023-09-10T01:03:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:30 smithi044 ceph-mon[98676]: pgmap v725: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 179 op/s 2023-09-10T01:03:31.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:30 smithi099 ceph-mon[102692]: pgmap v725: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 179 op/s 2023-09-10T01:03:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:32 smithi131 ceph-mon[103713]: pgmap v726: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 267 op/s 2023-09-10T01:03:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:32 smithi044 ceph-mon[98676]: pgmap v726: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 267 op/s 2023-09-10T01:03:33.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:32 smithi099 ceph-mon[102692]: pgmap v726: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 267 op/s 2023-09-10T01:03:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:34 smithi131 ceph-mon[103713]: pgmap v727: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:03:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:34 smithi044 ceph-mon[98676]: pgmap v727: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:03:35.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:34 smithi099 ceph-mon[102692]: pgmap v727: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:03:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:36 smithi131 ceph-mon[103713]: pgmap v728: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 258 op/s 2023-09-10T01:03:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:36 smithi044 ceph-mon[98676]: pgmap v728: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 258 op/s 2023-09-10T01:03:37.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:36 smithi099 ceph-mon[102692]: pgmap v728: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 258 op/s 2023-09-10T01:03:38.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:37 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:38.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:37 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:38.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:37 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:38 smithi131 ceph-mon[103713]: pgmap v729: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2023-09-10T01:03:39.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:38 smithi044 ceph-mon[98676]: pgmap v729: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2023-09-10T01:03:39.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:38 smithi099 ceph-mon[102692]: pgmap v729: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2023-09-10T01:03:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:40 smithi131 ceph-mon[103713]: pgmap v730: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2023-09-10T01:03:41.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:40 smithi044 ceph-mon[98676]: pgmap v730: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2023-09-10T01:03:41.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:40 smithi099 ceph-mon[102692]: pgmap v730: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2023-09-10T01:03:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.909+0000 7f77eaebf700 1 -- 172.21.15.44:0/1298108194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77e41055b0 msgr2=0x7f77e4105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.909+0000 7f77eaebf700 1 --2- 172.21.15.44:0/1298108194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77e41055b0 0x7f77e4105990 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f77d80099c0 tx=0x7f77d8009cd0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.909+0000 7f77eaebf700 1 -- 172.21.15.44:0/1298108194 shutdown_connections 2023-09-10T01:03:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.909+0000 7f77eaebf700 1 --2- 172.21.15.44:0/1298108194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e4100120 0x7f77e410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.909+0000 7f77eaebf700 1 --2- 172.21.15.44:0/1298108194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77e41055b0 0x7f77e4105990 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.909+0000 7f77eaebf700 1 --2- 172.21.15.44:0/1298108194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77e40ff760 0x7f77e40ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.909+0000 7f77eaebf700 1 -- 172.21.15.44:0/1298108194 >> 172.21.15.44:0/1298108194 conn(0x7f77e4074f30 msgr2=0x7f77e4075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.909+0000 7f77eaebf700 1 -- 172.21.15.44:0/1298108194 shutdown_connections 2023-09-10T01:03:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.909+0000 7f77eaebf700 1 -- 172.21.15.44:0/1298108194 wait complete. 2023-09-10T01:03:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.910+0000 7f77eaebf700 1 Processor -- start 2023-09-10T01:03:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.910+0000 7f77eaebf700 1 -- start start 2023-09-10T01:03:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.910+0000 7f77eaebf700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e40ff760 0x7f77e419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.910+0000 7f77eaebf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77e4100120 0x7f77e419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.910+0000 7f77eaebf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77e41055b0 0x7f77e41964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.910+0000 7f77eaebf700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e4196b90 con 0x7f77e41055b0 2023-09-10T01:03:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.910+0000 7f77eaebf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e4196d00 con 0x7f77e4100120 2023-09-10T01:03:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.910+0000 7f77eaebf700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e4196ec0 con 0x7f77e40ff760 2023-09-10T01:03:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.910+0000 7f77e8c5b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e40ff760 0x7f77e419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.911+0000 7f77e945c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77e41055b0 0x7f77e41964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.911+0000 7f77e8c5b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e40ff760 0x7f77e419c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:33922/0 (socket says 172.21.15.44:33922) 2023-09-10T01:03:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.911+0000 7f77e8c5b700 1 -- 172.21.15.44:0/1942542274 learned_addr learned my addr 172.21.15.44:0/1942542274 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.911+0000 7f77e3fff700 1 --2- 172.21.15.44:0/1942542274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77e4100120 0x7f77e419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.911+0000 7f77e945c700 1 -- 172.21.15.44:0/1942542274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e40ff760 msgr2=0x7f77e419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.911+0000 7f77e945c700 1 --2- 172.21.15.44:0/1942542274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e40ff760 0x7f77e419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.911+0000 7f77e945c700 1 -- 172.21.15.44:0/1942542274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77e4100120 msgr2=0x7f77e419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.911+0000 7f77e945c700 1 --2- 172.21.15.44:0/1942542274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77e4100120 0x7f77e419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.911+0000 7f77e945c700 1 -- 172.21.15.44:0/1942542274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d80095f0 con 0x7f77e41055b0 2023-09-10T01:03:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.911+0000 7f77e3fff700 1 --2- 172.21.15.44:0/1942542274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77e4100120 0x7f77e419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:03:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.911+0000 7f77e945c700 1 --2- 172.21.15.44:0/1942542274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77e41055b0 0x7f77e41964e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00ef30 tx=0x7f77dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.911+0000 7f77e1ffb700 1 -- 172.21.15.44:0/1942542274 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc004bb0 con 0x7f77e41055b0 2023-09-10T01:03:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.911+0000 7f77eaebf700 1 -- 172.21.15.44:0/1942542274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e41971a0 con 0x7f77e41055b0 2023-09-10T01:03:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.911+0000 7f77e1ffb700 1 -- 172.21.15.44:0/1942542274 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77dc00f040 con 0x7f77e41055b0 2023-09-10T01:03:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.911+0000 7f77e1ffb700 1 -- 172.21.15.44:0/1942542274 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0135c0 con 0x7f77e41055b0 2023-09-10T01:03:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.912+0000 7f77eaebf700 1 -- 172.21.15.44:0/1942542274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e41011d0 con 0x7f77e41055b0 2023-09-10T01:03:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.914+0000 7f77e1ffb700 1 -- 172.21.15.44:0/1942542274 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f77dc003a00 con 0x7f77e41055b0 2023-09-10T01:03:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.915+0000 7f77e1ffb700 1 --2- 172.21.15.44:0/1942542274 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f77d40a07b0 0x7f77d40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.915+0000 7f77e1ffb700 1 -- 172.21.15.44:0/1942542274 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f77dc0cfd00 con 0x7f77e41055b0 2023-09-10T01:03:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.915+0000 7f77e8c5b700 1 --2- 172.21.15.44:0/1942542274 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f77d40a07b0 0x7f77d40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.915+0000 7f77e8c5b700 1 --2- 172.21.15.44:0/1942542274 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f77d40a07b0 0x7f77d40a2c70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f77d8005b00 tx=0x7f77d8005a90 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.916+0000 7f77f7085b80 1 Processor -- start 2023-09-10T01:03:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.916+0000 7f77f7085b80 1 -- start start 2023-09-10T01:03:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.916+0000 7f77f7085b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665cf1a0 0x55fe665cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.916+0000 7f77f7085b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9300 0x55fe665c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77f7085b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665c9cc0 0x55fe665d5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77f7085b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe665226d0 con 0x55fe665c9300 2023-09-10T01:03:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77f7085b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6651fff0 con 0x55fe665cf1a0 2023-09-10T01:03:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77f7085b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6651b8e0 con 0x55fe665c9cc0 2023-09-10T01:03:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9300 0x55fe665c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9300 0x55fe665c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55282/0 (socket says 172.21.15.44:55282) 2023-09-10T01:03:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77e3fff700 1 -- 172.21.15.44:0/3175170555 learned_addr learned my addr 172.21.15.44:0/3175170555 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77e945c700 1 --2- 172.21.15.44:0/3175170555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665c9cc0 0x55fe665d5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77e8c5b700 1 --2- 172.21.15.44:0/3175170555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665cf1a0 0x55fe665cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77e3fff700 1 -- 172.21.15.44:0/3175170555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665c9cc0 msgr2=0x55fe665d5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77e3fff700 1 --2- 172.21.15.44:0/3175170555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665c9cc0 0x55fe665d5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77e3fff700 1 -- 172.21.15.44:0/3175170555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665cf1a0 msgr2=0x55fe665cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77e3fff700 1 --2- 172.21.15.44:0/3175170555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665cf1a0 0x55fe665cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77e3fff700 1 -- 172.21.15.44:0/3175170555 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe66347d80 con 0x55fe665c9300 2023-09-10T01:03:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77e3fff700 1 --2- 172.21.15.44:0/3175170555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9300 0x55fe665c9780 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f77d0008c50 tx=0x7f77d0008f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926f6113d2e1ad88 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77ceffd700 1 -- 172.21.15.44:0/3175170555 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d000ec80 con 0x55fe665c9300 2023-09-10T01:03:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77ceffd700 1 -- 172.21.15.44:0/3175170555 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77d000ede0 con 0x55fe665c9300 2023-09-10T01:03:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.917+0000 7f77ceffd700 1 -- 172.21.15.44:0/3175170555 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0005680 con 0x55fe665c9300 2023-09-10T01:03:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.918+0000 7f77f7085b80 1 -- 172.21.15.44:0/3175170555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9300 msgr2=0x55fe665c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.918+0000 7f77f7085b80 1 --2- 172.21.15.44:0/3175170555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9300 0x55fe665c9780 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f77d0008c50 tx=0x7f77d0008f60 comp rx=0 tx=0).stop 2023-09-10T01:03:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.918+0000 7f77f7085b80 1 -- 172.21.15.44:0/3175170555 shutdown_connections 2023-09-10T01:03:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.918+0000 7f77f7085b80 1 --2- 172.21.15.44:0/3175170555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665c9cc0 0x55fe665d5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.918+0000 7f77f7085b80 1 --2- 172.21.15.44:0/3175170555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665cf1a0 0x55fe665cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.918+0000 7f77f7085b80 1 --2- 172.21.15.44:0/3175170555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9300 0x55fe665c9780 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.918+0000 7f77f7085b80 1 -- 172.21.15.44:0/3175170555 >> 172.21.15.44:0/3175170555 conn(0x55fe6653e030 msgr2=0x55fe665c4560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.918+0000 7f77f7085b80 1 -- 172.21.15.44:0/3175170555 shutdown_connections 2023-09-10T01:03:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.918+0000 7f77f7085b80 1 -- 172.21.15.44:0/3175170555 wait complete. 2023-09-10T01:03:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.919+0000 7f77f7085b80 1 -- start start 2023-09-10T01:03:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.919+0000 7f77f7085b80 1 Processor -- start 2023-09-10T01:03:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.919+0000 7f77f7085b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9300 0x55fe665d5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.919+0000 7f77f7085b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665c9cc0 0x55fe665d3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.919+0000 7f77f7085b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665cf1a0 0x55fe665d3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.919+0000 7f77f7085b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe664b9b10 con 0x55fe665c9300 2023-09-10T01:03:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.919+0000 7f77f7085b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe664b9810 con 0x55fe665c9cc0 2023-09-10T01:03:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.919+0000 7f77f7085b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6639d8c0 con 0x55fe665cf1a0 2023-09-10T01:03:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.919+0000 7f77f7085b80 10 client.0 _ll_register_callbacks cb 0x55fe66403a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:03:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.919+0000 7f77e8c5b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9300 0x55fe665d5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.919+0000 7f77e8c5b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9300 0x55fe665d5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55286/0 (socket says 172.21.15.44:55286) 2023-09-10T01:03:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.919+0000 7f77e8c5b700 1 -- 172.21.15.44:0/4178402347 learned_addr learned my addr 172.21.15.44:0/4178402347 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.919+0000 7f77e945c700 1 --2- 172.21.15.44:0/4178402347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665cf1a0 0x55fe665d3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.920+0000 7f77e3fff700 1 --2- 172.21.15.44:0/4178402347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665c9cc0 0x55fe665d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.920+0000 7f77e8c5b700 1 -- 172.21.15.44:0/4178402347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665cf1a0 msgr2=0x55fe665d3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.920+0000 7f77e8c5b700 1 --2- 172.21.15.44:0/4178402347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665cf1a0 0x55fe665d3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.920+0000 7f77e8c5b700 1 -- 172.21.15.44:0/4178402347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665c9cc0 msgr2=0x55fe665d3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.920+0000 7f77e8c5b700 1 --2- 172.21.15.44:0/4178402347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665c9cc0 0x55fe665d3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.920+0000 7f77e8c5b700 1 -- 172.21.15.44:0/4178402347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d00088b0 con 0x55fe665c9300 2023-09-10T01:03:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.920+0000 7f77e8c5b700 1 --2- 172.21.15.44:0/4178402347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9300 0x55fe665d5d60 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f77d80057d0 tx=0x7f77d8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51909d7b94651276 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.920+0000 7f77c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:03:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.920+0000 7f77c3fff700 1 -- 172.21.15.44:0/4178402347 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d8029050 con 0x55fe665c9300 2023-09-10T01:03:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.920+0000 7f77c3fff700 1 -- 172.21.15.44:0/4178402347 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f77d800f660 con 0x55fe665c9300 2023-09-10T01:03:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.920+0000 7f77f7085b80 1 -- 172.21.15.44:0/4178402347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe66473490 con 0x55fe665c9300 2023-09-10T01:03:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.920+0000 7f77c3fff700 1 -- 172.21.15.44:0/4178402347 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77d800f820 con 0x55fe665c9300 2023-09-10T01:03:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.920+0000 7f77f7085b80 10 client.18555 fetch_fsmap learned FSMap version 94 2023-09-10T01:03:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.920+0000 7f77f7085b80 1 -- 172.21.15.44:0/4178402347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=4+}) v3 -- 0x55fe66330c00 con 0x55fe665c9300 2023-09-10T01:03:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.921+0000 7f77c3fff700 1 -- 172.21.15.44:0/4178402347 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fe66473490 con 0x55fe665c9300 2023-09-10T01:03:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.921+0000 7f77c3fff700 1 -- 172.21.15.44:0/4178402347 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 94) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f77d8022890 con 0x55fe665c9300 2023-09-10T01:03:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.921+0000 7f77f7085b80 10 client.18555 fetch_fsmap finished waiting for FSMap version 94 2023-09-10T01:03:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.921+0000 7f77f7085b80 10 client.18555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:03:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.921+0000 7f77f7085b80 20 client.18555 populate_metadata read hostname 'smithi044' 2023-09-10T01:03:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.921+0000 7f77f7085b80 1 --2- 172.21.15.44:0/4178402347 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe665e3470 0x55fe665e5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.921+0000 7f77f7085b80 4 client.18555 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:03:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.921+0000 7f77f7085b80 1 -- 172.21.15.44:0/4178402347 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe66530840 con 0x55fe665e3470 2023-09-10T01:03:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.921+0000 7f77e3fff700 1 --2- 172.21.15.44:0/4178402347 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe665e3470 0x55fe665e5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.922+0000 7f77c3fff700 10 client.18555 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.922+0000 7f77e3fff700 1 --2- 172.21.15.44:0/4178402347 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe665e3470 0x55fe665e5890 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b84f1216bd1fc9e5 server_cookie=6f1a558c97655a52 in_seq=0 out_seq=0 2023-09-10T01:03:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.922+0000 7f77c3fff700 1 -- 172.21.15.44:0/4178402347 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fe66530840 con 0x55fe665e3470 2023-09-10T01:03:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.922+0000 7f77c3fff700 10 client.18555 handle_command_reply: tid=0 2023-09-10T01:03:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.922+0000 7f77f7085b80 1 client.18555 shutdown 2023-09-10T01:03:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.923+0000 7f77f7085b80 1 -- 172.21.15.44:0/4178402347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9300 msgr2=0x55fe665d5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.923+0000 7f77f7085b80 1 --2- 172.21.15.44:0/4178402347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9300 0x55fe665d5d60 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f77d80057d0 tx=0x7f77d8009cd0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.923+0000 7f77f7085b80 1 -- 172.21.15.44:0/4178402347 shutdown_connections 2023-09-10T01:03:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.923+0000 7f77f7085b80 1 --2- 172.21.15.44:0/4178402347 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe665e3470 0x55fe665e5890 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.923+0000 7f77f7085b80 1 --2- 172.21.15.44:0/4178402347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665cf1a0 0x55fe665d3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.923+0000 7f77c3fff700 0 client.18555 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.923+0000 7f77f7085b80 1 --2- 172.21.15.44:0/4178402347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9300 0x55fe665d5d60 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.923+0000 7f77f7085b80 1 --2- 172.21.15.44:0/4178402347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665c9cc0 0x55fe665d3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.923+0000 7f77f7085b80 1 -- 172.21.15.44:0/4178402347 >> 172.21.15.44:0/4178402347 conn(0x55fe6653e030 msgr2=0x55fe665c37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.923+0000 7f77f7085b80 1 -- 172.21.15.44:0/4178402347 shutdown_connections 2023-09-10T01:03:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.923+0000 7f77f7085b80 1 -- 172.21.15.44:0/4178402347 wait complete. 2023-09-10T01:03:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.923+0000 7f77f7085b80 20 client.18555 trim_cache size 0 max 16384 2023-09-10T01:03:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.939+0000 7f77f7085b80 1 Processor -- start 2023-09-10T01:03:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77f7085b80 1 -- start start 2023-09-10T01:03:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77f7085b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665c9300 0x55fe665dfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77f7085b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9cc0 0x55fe665da520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77f7085b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665cf1a0 0x55fe665d5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77f7085b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe665ca580 con 0x55fe665c9cc0 2023-09-10T01:03:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77f7085b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe665cee00 con 0x55fe665cf1a0 2023-09-10T01:03:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77f7085b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe665d4660 con 0x55fe665c9300 2023-09-10T01:03:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9cc0 0x55fe665da520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9cc0 0x55fe665da520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55296/0 (socket says 172.21.15.44:55296) 2023-09-10T01:03:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77e3fff700 1 -- 172.21.15.44:0/3528308640 learned_addr learned my addr 172.21.15.44:0/3528308640 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77e8c5b700 1 --2- 172.21.15.44:0/3528308640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665c9300 0x55fe665dfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77e945c700 1 --2- 172.21.15.44:0/3528308640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665cf1a0 0x55fe665d5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77e3fff700 1 -- 172.21.15.44:0/3528308640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665c9300 msgr2=0x55fe665dfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77e3fff700 1 --2- 172.21.15.44:0/3528308640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665c9300 0x55fe665dfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77e3fff700 1 -- 172.21.15.44:0/3528308640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665cf1a0 msgr2=0x55fe665d5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77e3fff700 1 --2- 172.21.15.44:0/3528308640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665cf1a0 0x55fe665d5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77e3fff700 1 -- 172.21.15.44:0/3528308640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe6636e1f0 con 0x55fe665c9cc0 2023-09-10T01:03:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.940+0000 7f77e945c700 1 --2- 172.21.15.44:0/3528308640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665cf1a0 0x55fe665d5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:03:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.941+0000 7f77e3fff700 1 --2- 172.21.15.44:0/3528308640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9cc0 0x55fe665da520 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x55fe665c4060 tx=0x7f77d001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2e4dc4d730b5b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.941+0000 7f77cf7fe700 1 -- 172.21.15.44:0/3528308640 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d001f030 con 0x55fe665c9cc0 2023-09-10T01:03:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.941+0000 7f77cf7fe700 1 -- 172.21.15.44:0/3528308640 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77d0020040 con 0x55fe665c9cc0 2023-09-10T01:03:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.941+0000 7f77cf7fe700 1 -- 172.21.15.44:0/3528308640 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d00272b0 con 0x55fe665c9cc0 2023-09-10T01:03:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.941+0000 7f77f7085b80 1 -- 172.21.15.44:0/3528308640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9cc0 msgr2=0x55fe665da520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.941+0000 7f77f7085b80 1 --2- 172.21.15.44:0/3528308640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9cc0 0x55fe665da520 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x55fe665c4060 tx=0x7f77d001c040 comp rx=0 tx=0).stop 2023-09-10T01:03:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.941+0000 7f77f7085b80 1 -- 172.21.15.44:0/3528308640 shutdown_connections 2023-09-10T01:03:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.941+0000 7f77f7085b80 1 --2- 172.21.15.44:0/3528308640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665cf1a0 0x55fe665d5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.941+0000 7f77f7085b80 1 --2- 172.21.15.44:0/3528308640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665c9300 0x55fe665dfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.941+0000 7f77f7085b80 1 --2- 172.21.15.44:0/3528308640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9cc0 0x55fe665da520 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.941+0000 7f77f7085b80 1 -- 172.21.15.44:0/3528308640 >> 172.21.15.44:0/3528308640 conn(0x55fe6653e030 msgr2=0x55fe665cd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.942+0000 7f77f7085b80 1 -- 172.21.15.44:0/3528308640 shutdown_connections 2023-09-10T01:03:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.942+0000 7f77f7085b80 1 -- 172.21.15.44:0/3528308640 wait complete. 2023-09-10T01:03:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.942+0000 7f77f7085b80 1 -- start start 2023-09-10T01:03:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.942+0000 7f77f7085b80 1 Processor -- start 2023-09-10T01:03:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.942+0000 7f77f7085b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665c9300 0x55fe666305a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.942+0000 7f77f7085b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9cc0 0x55fe66630ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.942+0000 7f77f7085b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665cf1a0 0x55fe665d6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.942+0000 7f77f7085b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe665d6ad0 con 0x55fe665c9cc0 2023-09-10T01:03:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.942+0000 7f77f7085b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe665d6c40 con 0x55fe665c9300 2023-09-10T01:03:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.942+0000 7f77f7085b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe665d6e00 con 0x55fe665cf1a0 2023-09-10T01:03:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.942+0000 7f77f7085b80 10 client.0 _ll_register_callbacks cb 0x7f77d8005380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:03:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.943+0000 7f77e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9cc0 0x55fe66630ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.943+0000 7f77e945c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665cf1a0 0x55fe665d6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.943+0000 7f77e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9cc0 0x55fe66630ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55300/0 (socket says 172.21.15.44:55300) 2023-09-10T01:03:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.943+0000 7f77e3fff700 1 -- 172.21.15.44:0/2522576638 learned_addr learned my addr 172.21.15.44:0/2522576638 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.943+0000 7f77e8c5b700 1 --2- 172.21.15.44:0/2522576638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665c9300 0x55fe666305a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.943+0000 7f77e945c700 1 -- 172.21.15.44:0/2522576638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665c9300 msgr2=0x55fe666305a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.943+0000 7f77e945c700 1 --2- 172.21.15.44:0/2522576638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665c9300 0x55fe666305a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.943+0000 7f77e945c700 1 -- 172.21.15.44:0/2522576638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9cc0 msgr2=0x55fe66630ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.943+0000 7f77e945c700 1 --2- 172.21.15.44:0/2522576638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9cc0 0x55fe66630ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.943+0000 7f77e945c700 1 -- 172.21.15.44:0/2522576638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe66347d80 con 0x55fe665cf1a0 2023-09-10T01:03:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.943+0000 7f77e3fff700 1 --2- 172.21.15.44:0/2522576638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9cc0 0x55fe66630ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:03:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.943+0000 7f77e945c700 1 --2- 172.21.15.44:0/2522576638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665cf1a0 0x55fe665d6500 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0cd160 tx=0x7f77dc0cd5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea4b6de8d30f812b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.943+0000 7f77c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:03:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.943+0000 7f77c3fff700 1 -- 172.21.15.44:0/2522576638 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc09c5b0 con 0x55fe665cf1a0 2023-09-10T01:03:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.943+0000 7f77c3fff700 1 -- 172.21.15.44:0/2522576638 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f77dc0b5280 con 0x55fe665cf1a0 2023-09-10T01:03:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.943+0000 7f77f7085b80 1 -- 172.21.15.44:0/2522576638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77d000ede0 con 0x55fe665cf1a0 2023-09-10T01:03:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.944+0000 7f77c3fff700 1 -- 172.21.15.44:0/2522576638 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77dc09cec0 con 0x55fe665cf1a0 2023-09-10T01:03:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.944+0000 7f77f7085b80 10 client.28156 fetch_fsmap learned FSMap version 94 2023-09-10T01:03:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.944+0000 7f77f7085b80 1 -- 172.21.15.44:0/2522576638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=4+}) v3 -- 0x55fe66346250 con 0x55fe665cf1a0 2023-09-10T01:03:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.944+0000 7f77c3fff700 1 -- 172.21.15.44:0/2522576638 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77dc07f020 con 0x55fe665cf1a0 2023-09-10T01:03:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.944+0000 7f77c3fff700 1 -- 172.21.15.44:0/2522576638 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 94) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f77dc097ad0 con 0x55fe665cf1a0 2023-09-10T01:03:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.944+0000 7f77f7085b80 10 client.28156 fetch_fsmap finished waiting for FSMap version 94 2023-09-10T01:03:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.944+0000 7f77f7085b80 10 client.28156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:03:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.944+0000 7f77f7085b80 20 client.28156 populate_metadata read hostname 'smithi044' 2023-09-10T01:03:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.944+0000 7f77f7085b80 1 --2- 172.21.15.44:0/2522576638 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe66638480 0x55fe6663a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.944+0000 7f77f7085b80 4 client.28156 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:03:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.944+0000 7f77f7085b80 1 -- 172.21.15.44:0/2522576638 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fe665148c0 con 0x55fe66638480 2023-09-10T01:03:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.945+0000 7f77e8c5b700 1 --2- 172.21.15.44:0/2522576638 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe66638480 0x55fe6663a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.945+0000 7f77e8c5b700 1 --2- 172.21.15.44:0/2522576638 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe66638480 0x55fe6663a8a0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44be2d193010b1e server_cookie=7bfac80d2ade83d2 in_seq=0 out_seq=0 2023-09-10T01:03:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.945+0000 7f77c3fff700 10 client.28156 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.959+0000 7f77c3fff700 1 -- 172.21.15.44:0/2522576638 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fe665148c0 con 0x55fe66638480 2023-09-10T01:03:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.959+0000 7f77c3fff700 10 client.28156 handle_command_reply: tid=0 2023-09-10T01:03:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.959+0000 7f77f7085b80 1 client.28156 shutdown 2023-09-10T01:03:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.959+0000 7f77f7085b80 1 -- 172.21.15.44:0/2522576638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665cf1a0 msgr2=0x55fe665d6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.959+0000 7f77f7085b80 1 --2- 172.21.15.44:0/2522576638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665cf1a0 0x55fe665d6500 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0cd160 tx=0x7f77dc0cd5b0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.959+0000 7f77f7085b80 1 -- 172.21.15.44:0/2522576638 shutdown_connections 2023-09-10T01:03:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.959+0000 7f77f7085b80 1 --2- 172.21.15.44:0/2522576638 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe66638480 0x55fe6663a8a0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.959+0000 7f77f7085b80 1 --2- 172.21.15.44:0/2522576638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe665cf1a0 0x55fe665d6500 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.959+0000 7f77f7085b80 1 --2- 172.21.15.44:0/2522576638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe665c9300 0x55fe666305a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.959+0000 7f77f7085b80 1 --2- 172.21.15.44:0/2522576638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe665c9cc0 0x55fe66630ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.959+0000 7f77c3fff700 0 client.28156 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.959+0000 7f77f7085b80 1 -- 172.21.15.44:0/2522576638 >> 172.21.15.44:0/2522576638 conn(0x55fe6653e030 msgr2=0x55fe665ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.960+0000 7f77f7085b80 1 -- 172.21.15.44:0/2522576638 shutdown_connections 2023-09-10T01:03:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.960+0000 7f77f7085b80 1 -- 172.21.15.44:0/2522576638 wait complete. 2023-09-10T01:03:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.960+0000 7f77f7085b80 20 client.28156 trim_cache size 0 max 16384 2023-09-10T01:03:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:03:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:03:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:03:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:03:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.960+0000 7f77eaebf700 1 -- 172.21.15.44:0/1942542274 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f77d40a07b0 msgr2=0x7f77d40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.960+0000 7f77eaebf700 1 --2- 172.21.15.44:0/1942542274 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f77d40a07b0 0x7f77d40a2c70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f77d8005b00 tx=0x7f77d8005a90 comp rx=0 tx=0).stop 2023-09-10T01:03:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.960+0000 7f77eaebf700 1 -- 172.21.15.44:0/1942542274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77e41055b0 msgr2=0x7f77e41964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.960+0000 7f77eaebf700 1 --2- 172.21.15.44:0/1942542274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77e41055b0 0x7f77e41964e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00ef30 tx=0x7f77dc00c3a0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.960+0000 7f77eaebf700 1 -- 172.21.15.44:0/1942542274 shutdown_connections 2023-09-10T01:03:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.960+0000 7f77eaebf700 1 --2- 172.21.15.44:0/1942542274 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f77d40a07b0 0x7f77d40a2c70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.960+0000 7f77eaebf700 1 --2- 172.21.15.44:0/1942542274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77e41055b0 0x7f77e41964e0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.960+0000 7f77eaebf700 1 --2- 172.21.15.44:0/1942542274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e40ff760 0x7f77e419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.960+0000 7f77eaebf700 1 --2- 172.21.15.44:0/1942542274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77e4100120 0x7f77e419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.960+0000 7f77eaebf700 1 -- 172.21.15.44:0/1942542274 >> 172.21.15.44:0/1942542274 conn(0x7f77e4074f30 msgr2=0x7f77e4067bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.960+0000 7f77eaebf700 1 -- 172.21.15.44:0/1942542274 shutdown_connections 2023-09-10T01:03:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:42.960+0000 7f77eaebf700 1 -- 172.21.15.44:0/1942542274 wait complete. 2023-09-10T01:03:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:42 smithi131 ceph-mon[103713]: pgmap v731: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 267 op/s 2023-09-10T01:03:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:42 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:42 smithi044 ceph-mon[98676]: pgmap v731: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 267 op/s 2023-09-10T01:03:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:42 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:42 smithi099 ceph-mon[102692]: pgmap v731: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 267 op/s 2023-09-10T01:03:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:42 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:44 smithi131 ceph-mon[103713]: pgmap v732: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 178 op/s 2023-09-10T01:03:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:44 smithi044 ceph-mon[98676]: pgmap v732: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 178 op/s 2023-09-10T01:03:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:44 smithi099 ceph-mon[102692]: pgmap v732: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 178 op/s 2023-09-10T01:03:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:46 smithi131 ceph-mon[103713]: pgmap v733: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 266 op/s 2023-09-10T01:03:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:46 smithi044 ceph-mon[98676]: pgmap v733: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 266 op/s 2023-09-10T01:03:47.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:46 smithi099 ceph-mon[102692]: pgmap v733: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 266 op/s 2023-09-10T01:03:48.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:47 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:48.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:47 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:48.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:47 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:48 smithi131 ceph-mon[103713]: pgmap v734: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2023-09-10T01:03:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:48 smithi044 ceph-mon[98676]: pgmap v734: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2023-09-10T01:03:49.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:48 smithi099 ceph-mon[102692]: pgmap v734: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2023-09-10T01:03:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:50 smithi131 ceph-mon[103713]: pgmap v735: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2023-09-10T01:03:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:50 smithi044 ceph-mon[98676]: pgmap v735: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2023-09-10T01:03:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:50 smithi099 ceph-mon[102692]: pgmap v735: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2023-09-10T01:03:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:52 smithi131 ceph-mon[103713]: pgmap v736: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 284 op/s 2023-09-10T01:03:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:52 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:52 smithi044 ceph-mon[98676]: pgmap v736: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 284 op/s 2023-09-10T01:03:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:52 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:52 smithi099 ceph-mon[102692]: pgmap v736: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 284 op/s 2023-09-10T01:03:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:52 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:55.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:54 smithi131 ceph-mon[103713]: pgmap v737: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2023-09-10T01:03:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:54 smithi044 ceph-mon[98676]: pgmap v737: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2023-09-10T01:03:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:54 smithi099 ceph-mon[102692]: pgmap v737: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2023-09-10T01:03:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.074+0000 7f33b92b2700 1 -- 172.21.15.44:0/886979794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b40ffea0 msgr2=0x7f33b410c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.074+0000 7f33b92b2700 1 --2- 172.21.15.44:0/886979794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b40ffea0 0x7f33b410c360 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00b0d0 tx=0x7f33ac00b3e0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.074+0000 7f33b92b2700 1 -- 172.21.15.44:0/886979794 shutdown_connections 2023-09-10T01:03:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.074+0000 7f33b92b2700 1 --2- 172.21.15.44:0/886979794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b40ffea0 0x7f33b410c360 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.074+0000 7f33b92b2700 1 --2- 172.21.15.44:0/886979794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33b41055a0 0x7f33b4105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.074+0000 7f33b92b2700 1 --2- 172.21.15.44:0/886979794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b40ff4e0 0x7f33b40ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.074+0000 7f33b92b2700 1 -- 172.21.15.44:0/886979794 >> 172.21.15.44:0/886979794 conn(0x7f33b4076670 msgr2=0x7f33b4076a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.074+0000 7f33b92b2700 1 -- 172.21.15.44:0/886979794 shutdown_connections 2023-09-10T01:03:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.074+0000 7f33b92b2700 1 -- 172.21.15.44:0/886979794 wait complete. 2023-09-10T01:03:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.074+0000 7f33b92b2700 1 Processor -- start 2023-09-10T01:03:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b92b2700 1 -- start start 2023-09-10T01:03:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b92b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b40ff4e0 0x7f33b4197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b92b2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33b40ffea0 0x7f33b4197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b92b2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b41055a0 0x7f33b41a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b92b2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41a2af0 con 0x7f33b40ff4e0 2023-09-10T01:03:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b92b2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41a2c60 con 0x7f33b41055a0 2023-09-10T01:03:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b92b2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41a2e20 con 0x7f33b40ffea0 2023-09-10T01:03:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b40ff4e0 0x7f33b4197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b40ff4e0 0x7f33b4197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55314/0 (socket says 172.21.15.44:55314) 2023-09-10T01:03:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b41055a0 0x7f33b41a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b2ffd700 1 -- 172.21.15.44:0/496104 learned_addr learned my addr 172.21.15.44:0/496104 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b27fc700 1 --2- 172.21.15.44:0/496104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33b40ffea0 0x7f33b4197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b37fe700 1 -- 172.21.15.44:0/496104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33b40ffea0 msgr2=0x7f33b4197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b37fe700 1 --2- 172.21.15.44:0/496104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33b40ffea0 0x7f33b4197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b37fe700 1 -- 172.21.15.44:0/496104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b40ff4e0 msgr2=0x7f33b4197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b37fe700 1 --2- 172.21.15.44:0/496104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b40ff4e0 0x7f33b4197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b37fe700 1 -- 172.21.15.44:0/496104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac009d00 con 0x7f33b41055a0 2023-09-10T01:03:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.075+0000 7f33b2ffd700 1 --2- 172.21.15.44:0/496104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b40ff4e0 0x7f33b4197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:03:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.076+0000 7f33b37fe700 1 --2- 172.21.15.44:0/496104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b41055a0 0x7f33b41a2520 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00b770 tx=0x7f33ac008e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.076+0000 7f339bfff700 1 -- 172.21.15.44:0/496104 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00e040 con 0x7f33b41055a0 2023-09-10T01:03:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.076+0000 7f33b92b2700 1 -- 172.21.15.44:0/496104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b41a30a0 con 0x7f33b41055a0 2023-09-10T01:03:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.076+0000 7f339bfff700 1 -- 172.21.15.44:0/496104 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f33ac01d900 con 0x7f33b41055a0 2023-09-10T01:03:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.076+0000 7f339bfff700 1 -- 172.21.15.44:0/496104 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac01b7e0 con 0x7f33b41055a0 2023-09-10T01:03:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.076+0000 7f33b92b2700 1 -- 172.21.15.44:0/496104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b41a3570 con 0x7f33b41055a0 2023-09-10T01:03:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.078+0000 7f339bfff700 1 -- 172.21.15.44:0/496104 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f33ac01da70 con 0x7f33b41055a0 2023-09-10T01:03:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.079+0000 7f339bfff700 1 --2- 172.21.15.44:0/496104 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f339c0a0830 0x7f339c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.079+0000 7f33b2ffd700 1 --2- 172.21.15.44:0/496104 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f339c0a0830 0x7f339c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.079+0000 7f339bfff700 1 -- 172.21.15.44:0/496104 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f33ac0cd610 con 0x7f33b41055a0 2023-09-10T01:03:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.079+0000 7f33c5478b80 1 Processor -- start 2023-09-10T01:03:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.079+0000 7f33b2ffd700 1 --2- 172.21.15.44:0/496104 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f339c0a0830 0x7f339c0a2cf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f33a4011550 tx=0x7f33a4009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.079+0000 7f33c5478b80 1 -- start start 2023-09-10T01:03:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.079+0000 7f33c5478b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e2e1a0 0x55e708e2e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.079+0000 7f33c5478b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e28300 0x55e708e28780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.079+0000 7f33c5478b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28cc0 0x55e708e34f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33c5478b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e708d816d0 con 0x55e708e2e1a0 2023-09-10T01:03:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33c5478b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e708d7eff0 con 0x55e708e28300 2023-09-10T01:03:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33c5478b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e708d7a8e0 con 0x55e708e28cc0 2023-09-10T01:03:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33b2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e2e1a0 0x55e708e2e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33b2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e2e1a0 0x55e708e2e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55322/0 (socket says 172.21.15.44:55322) 2023-09-10T01:03:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33b27fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e28300 0x55e708e28780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33b2ffd700 1 -- 172.21.15.44:0/4034722946 learned_addr learned my addr 172.21.15.44:0/4034722946 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33b37fe700 1 --2- 172.21.15.44:0/4034722946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28cc0 0x55e708e34f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33b2ffd700 1 -- 172.21.15.44:0/4034722946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28cc0 msgr2=0x55e708e34f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33b2ffd700 1 --2- 172.21.15.44:0/4034722946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28cc0 0x55e708e34f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33b2ffd700 1 -- 172.21.15.44:0/4034722946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e28300 msgr2=0x55e708e28780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33b2ffd700 1 --2- 172.21.15.44:0/4034722946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e28300 0x55e708e28780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33b2ffd700 1 -- 172.21.15.44:0/4034722946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e708ba6d80 con 0x55e708e2e1a0 2023-09-10T01:03:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33b2ffd700 1 --2- 172.21.15.44:0/4034722946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e2e1a0 0x55e708e2e580 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009880 tx=0x7f33a4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ef99d5fcfdc2b2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33997fa700 1 -- 172.21.15.44:0/4034722946 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4026630 con 0x55e708e2e1a0 2023-09-10T01:03:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33997fa700 1 -- 172.21.15.44:0/4034722946 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f33a4021070 con 0x55e708e2e1a0 2023-09-10T01:03:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.080+0000 7f33997fa700 1 -- 172.21.15.44:0/4034722946 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a401d610 con 0x55e708e2e1a0 2023-09-10T01:03:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.081+0000 7f33c5478b80 1 -- 172.21.15.44:0/4034722946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e2e1a0 msgr2=0x55e708e2e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.081+0000 7f33c5478b80 1 --2- 172.21.15.44:0/4034722946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e2e1a0 0x55e708e2e580 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009880 tx=0x7f33a4014000 comp rx=0 tx=0).stop 2023-09-10T01:03:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.081+0000 7f33c5478b80 1 -- 172.21.15.44:0/4034722946 shutdown_connections 2023-09-10T01:03:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.081+0000 7f33c5478b80 1 --2- 172.21.15.44:0/4034722946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28cc0 0x55e708e34f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.081+0000 7f33c5478b80 1 --2- 172.21.15.44:0/4034722946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e2e1a0 0x55e708e2e580 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.081+0000 7f33c5478b80 1 --2- 172.21.15.44:0/4034722946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e28300 0x55e708e28780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.081+0000 7f33c5478b80 1 -- 172.21.15.44:0/4034722946 >> 172.21.15.44:0/4034722946 conn(0x55e708d9d030 msgr2=0x55e708e23560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.081+0000 7f33c5478b80 1 -- 172.21.15.44:0/4034722946 shutdown_connections 2023-09-10T01:03:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.081+0000 7f33c5478b80 1 -- 172.21.15.44:0/4034722946 wait complete. 2023-09-10T01:03:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.081+0000 7f33c5478b80 1 -- start start 2023-09-10T01:03:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33c5478b80 1 Processor -- start 2023-09-10T01:03:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33c5478b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e28300 0x55e708e34d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33c5478b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28cc0 0x55e708e32080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33c5478b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e2e1a0 0x55e708e32600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33c5478b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e708d18b10 con 0x55e708e28300 2023-09-10T01:03:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33c5478b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e708d18810 con 0x55e708e2e1a0 2023-09-10T01:03:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33c5478b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e708bfc8c0 con 0x55e708e28cc0 2023-09-10T01:03:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33b2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e28300 0x55e708e34d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33c5478b80 10 client.0 _ll_register_callbacks cb 0x55e708c62a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:03:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33b2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e28300 0x55e708e34d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55328/0 (socket says 172.21.15.44:55328) 2023-09-10T01:03:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33b2ffd700 1 -- 172.21.15.44:0/1762400293 learned_addr learned my addr 172.21.15.44:0/1762400293 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33b27fc700 1 --2- 172.21.15.44:0/1762400293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28cc0 0x55e708e32080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33b37fe700 1 --2- 172.21.15.44:0/1762400293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e2e1a0 0x55e708e32600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33b2ffd700 1 -- 172.21.15.44:0/1762400293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28cc0 msgr2=0x55e708e32080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33b2ffd700 1 --2- 172.21.15.44:0/1762400293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28cc0 0x55e708e32080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33b2ffd700 1 -- 172.21.15.44:0/1762400293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e2e1a0 msgr2=0x55e708e32600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33b2ffd700 1 --2- 172.21.15.44:0/1762400293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e2e1a0 0x55e708e32600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.082+0000 7f33b2ffd700 1 -- 172.21.15.44:0/1762400293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a4017070 con 0x55e708e28300 2023-09-10T01:03:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.083+0000 7f33b2ffd700 1 --2- 172.21.15.44:0/1762400293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e28300 0x55e708e34d60 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f33a4011ca0 tx=0x7f33a400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5e0bda9143a1db server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.083+0000 7f3396ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:03:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.083+0000 7f3396ff5700 1 -- 172.21.15.44:0/1762400293 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a401c410 con 0x55e708e28300 2023-09-10T01:03:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.083+0000 7f33c5478b80 1 -- 172.21.15.44:0/1762400293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e708cd2490 con 0x55e708e28300 2023-09-10T01:03:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.083+0000 7f3396ff5700 1 -- 172.21.15.44:0/1762400293 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f33a401c570 con 0x55e708e28300 2023-09-10T01:03:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.083+0000 7f3396ff5700 1 -- 172.21.15.44:0/1762400293 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33a401c730 con 0x55e708e28300 2023-09-10T01:03:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.083+0000 7f33c5478b80 10 client.18573 fetch_fsmap learned FSMap version 96 2023-09-10T01:03:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.083+0000 7f33c5478b80 1 -- 172.21.15.44:0/1762400293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) v3 -- 0x55e708b8fc00 con 0x55e708e28300 2023-09-10T01:03:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.083+0000 7f3396ff5700 1 -- 172.21.15.44:0/1762400293 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e708cd2490 con 0x55e708e28300 2023-09-10T01:03:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.083+0000 7f3396ff5700 1 -- 172.21.15.44:0/1762400293 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 96) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f33a4004be0 con 0x55e708e28300 2023-09-10T01:03:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.083+0000 7f33c5478b80 10 client.18573 fetch_fsmap finished waiting for FSMap version 96 2023-09-10T01:03:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.083+0000 7f33c5478b80 10 client.18573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:03:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.083+0000 7f33c5478b80 20 client.18573 populate_metadata read hostname 'smithi044' 2023-09-10T01:03:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.083+0000 7f33c5478b80 1 --2- 172.21.15.44:0/1762400293 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e708e42390 0x55e708e447b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.083+0000 7f33c5478b80 4 client.18573 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:03:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.083+0000 7f33c5478b80 1 -- 172.21.15.44:0/1762400293 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e708d8f840 con 0x55e708e42390 2023-09-10T01:03:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.084+0000 7f33b27fc700 1 --2- 172.21.15.44:0/1762400293 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e708e42390 0x55e708e447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.084+0000 7f3396ff5700 10 client.18573 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.084+0000 7f33b27fc700 1 --2- 172.21.15.44:0/1762400293 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e708e42390 0x55e708e447b0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9853fb3981d772ea server_cookie=771c538e6cf3d51d in_seq=0 out_seq=0 2023-09-10T01:03:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.084+0000 7f3396ff5700 1 -- 172.21.15.44:0/1762400293 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e708d8f840 con 0x55e708e42390 2023-09-10T01:03:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.085+0000 7f3396ff5700 10 client.18573 handle_command_reply: tid=0 2023-09-10T01:03:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.085+0000 7f33c5478b80 1 client.18573 shutdown 2023-09-10T01:03:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.085+0000 7f33c5478b80 1 -- 172.21.15.44:0/1762400293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e28300 msgr2=0x55e708e34d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.085+0000 7f33c5478b80 1 --2- 172.21.15.44:0/1762400293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e28300 0x55e708e34d60 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f33a4011ca0 tx=0x7f33a400ee40 comp rx=0 tx=0).stop 2023-09-10T01:03:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.085+0000 7f33c5478b80 1 -- 172.21.15.44:0/1762400293 shutdown_connections 2023-09-10T01:03:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.085+0000 7f33c5478b80 1 --2- 172.21.15.44:0/1762400293 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e708e42390 0x55e708e447b0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.085+0000 7f33c5478b80 1 --2- 172.21.15.44:0/1762400293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e2e1a0 0x55e708e32600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.085+0000 7f33c5478b80 1 --2- 172.21.15.44:0/1762400293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e28300 0x55e708e34d60 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.085+0000 7f3396ff5700 0 client.18573 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.085+0000 7f33c5478b80 1 --2- 172.21.15.44:0/1762400293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28cc0 0x55e708e32080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.085+0000 7f33c5478b80 1 -- 172.21.15.44:0/1762400293 >> 172.21.15.44:0/1762400293 conn(0x55e708d9d030 msgr2=0x55e708e227a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.085+0000 7f33c5478b80 1 -- 172.21.15.44:0/1762400293 shutdown_connections 2023-09-10T01:03:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.085+0000 7f33c5478b80 1 -- 172.21.15.44:0/1762400293 wait complete. 2023-09-10T01:03:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.085+0000 7f33c5478b80 20 client.18573 trim_cache size 0 max 16384 2023-09-10T01:03:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33c5478b80 1 Processor -- start 2023-09-10T01:03:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33c5478b80 1 -- start start 2023-09-10T01:03:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33c5478b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e28300 0x55e708e34890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33c5478b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28cc0 0x55e708e2d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33c5478b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e2e1a0 0x55e708e2d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33c5478b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e708e1d8a0 con 0x55e708e28300 2023-09-10T01:03:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33c5478b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e708e2eb50 con 0x55e708e2e1a0 2023-09-10T01:03:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33c5478b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e708e33660 con 0x55e708e28cc0 2023-09-10T01:03:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33b2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e28300 0x55e708e34890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33b2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e28300 0x55e708e34890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55336/0 (socket says 172.21.15.44:55336) 2023-09-10T01:03:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33b2ffd700 1 -- 172.21.15.44:0/256418508 learned_addr learned my addr 172.21.15.44:0/256418508 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33b37fe700 1 --2- 172.21.15.44:0/256418508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e2e1a0 0x55e708e2d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33b27fc700 1 --2- 172.21.15.44:0/256418508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28cc0 0x55e708e2d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33b2ffd700 1 -- 172.21.15.44:0/256418508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28cc0 msgr2=0x55e708e2d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33b2ffd700 1 --2- 172.21.15.44:0/256418508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28cc0 0x55e708e2d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33b2ffd700 1 -- 172.21.15.44:0/256418508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e2e1a0 msgr2=0x55e708e2d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33b2ffd700 1 --2- 172.21.15.44:0/256418508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e2e1a0 0x55e708e2d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.100+0000 7f33b2ffd700 1 -- 172.21.15.44:0/256418508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e708bcd1f0 con 0x55e708e28300 2023-09-10T01:03:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.101+0000 7f33b2ffd700 1 --2- 172.21.15.44:0/256418508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e28300 0x55e708e34890 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f33a401ca60 tx=0x7f33a40093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3a47084f34b246 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.101+0000 7f3399ffb700 1 -- 172.21.15.44:0/256418508 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a401ccb0 con 0x55e708e28300 2023-09-10T01:03:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.101+0000 7f3399ffb700 1 -- 172.21.15.44:0/256418508 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f33a401c8a0 con 0x55e708e28300 2023-09-10T01:03:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.101+0000 7f3399ffb700 1 -- 172.21.15.44:0/256418508 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a40269a0 con 0x55e708e28300 2023-09-10T01:03:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.101+0000 7f33c5478b80 1 -- 172.21.15.44:0/256418508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e28300 msgr2=0x55e708e34890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.101+0000 7f33c5478b80 1 --2- 172.21.15.44:0/256418508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e28300 0x55e708e34890 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f33a401ca60 tx=0x7f33a40093c0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.101+0000 7f33c5478b80 1 -- 172.21.15.44:0/256418508 shutdown_connections 2023-09-10T01:03:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.101+0000 7f33c5478b80 1 --2- 172.21.15.44:0/256418508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e2e1a0 0x55e708e2d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.101+0000 7f33c5478b80 1 --2- 172.21.15.44:0/256418508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e28300 0x55e708e34890 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.101+0000 7f33c5478b80 1 --2- 172.21.15.44:0/256418508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28cc0 0x55e708e2d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.101+0000 7f33c5478b80 1 -- 172.21.15.44:0/256418508 >> 172.21.15.44:0/256418508 conn(0x55e708d9d030 msgr2=0x55e708e2c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.101+0000 7f33c5478b80 1 -- 172.21.15.44:0/256418508 shutdown_connections 2023-09-10T01:03:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.101+0000 7f33c5478b80 1 -- 172.21.15.44:0/256418508 wait complete. 2023-09-10T01:03:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33c5478b80 1 -- start start 2023-09-10T01:03:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33c5478b80 1 Processor -- start 2023-09-10T01:03:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33c5478b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28300 0x55e708e37510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33c5478b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e28cc0 0x55e708e37a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33c5478b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e2e1a0 0x55e708e37fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33c5478b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e708e95360 con 0x55e708e2e1a0 2023-09-10T01:03:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33c5478b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e708e954d0 con 0x55e708e28cc0 2023-09-10T01:03:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33c5478b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e708e95690 con 0x55e708e28300 2023-09-10T01:03:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33c5478b80 10 client.0 _ll_register_callbacks cb 0x7f33a8009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:03:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33b37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e2e1a0 0x55e708e37fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33b37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e2e1a0 0x55e708e37fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55346/0 (socket says 172.21.15.44:55346) 2023-09-10T01:03:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33b2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28300 0x55e708e37510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33b37fe700 1 -- 172.21.15.44:0/3957595599 learned_addr learned my addr 172.21.15.44:0/3957595599 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:03:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33b27fc700 1 --2- 172.21.15.44:0/3957595599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e28cc0 0x55e708e37a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33b37fe700 1 -- 172.21.15.44:0/3957595599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28300 msgr2=0x55e708e37510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33b37fe700 1 --2- 172.21.15.44:0/3957595599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28300 0x55e708e37510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.102+0000 7f33b37fe700 1 -- 172.21.15.44:0/3957595599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e28cc0 msgr2=0x55e708e37a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:03:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f33b37fe700 1 --2- 172.21.15.44:0/3957595599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e28cc0 0x55e708e37a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f33b37fe700 1 -- 172.21.15.44:0/3957595599 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e708b8fc00 con 0x55e708e2e1a0 2023-09-10T01:03:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f33b37fe700 1 --2- 172.21.15.44:0/3957595599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e2e1a0 0x55e708e37fb0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0ca9c0 tx=0x7f33ac09a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6f558d46f735bb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:03:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f3396ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:03:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f3396ff5700 1 -- 172.21.15.44:0/3957595599 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac09e370 con 0x55e708e2e1a0 2023-09-10T01:03:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f33c5478b80 1 -- 172.21.15.44:0/3957595599 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e708d738c0 con 0x55e708e2e1a0 2023-09-10T01:03:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f3396ff5700 1 -- 172.21.15.44:0/3957595599 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f33ac09a750 con 0x55e708e2e1a0 2023-09-10T01:03:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f3396ff5700 1 -- 172.21.15.44:0/3957595599 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33ac09e580 con 0x55e708e2e1a0 2023-09-10T01:03:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f33c5478b80 10 client.18579 fetch_fsmap learned FSMap version 96 2023-09-10T01:03:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f33c5478b80 1 -- 172.21.15.44:0/3957595599 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) v3 -- 0x55e708ba5250 con 0x55e708e2e1a0 2023-09-10T01:03:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f3396ff5700 1 -- 172.21.15.44:0/3957595599 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e708d738c0 con 0x55e708e2e1a0 2023-09-10T01:03:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f3396ff5700 1 -- 172.21.15.44:0/3957595599 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 96) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f33ac09aa20 con 0x55e708e2e1a0 2023-09-10T01:03:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f33c5478b80 10 client.18579 fetch_fsmap finished waiting for FSMap version 96 2023-09-10T01:03:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f33c5478b80 10 client.18579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:03:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f33c5478b80 20 client.18579 populate_metadata read hostname 'smithi044' 2023-09-10T01:03:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f33c5478b80 1 --2- 172.21.15.44:0/3957595599 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e708e96d00 0x55e708e99120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:03:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f33c5478b80 4 client.18579 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:03:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.103+0000 7f33c5478b80 1 -- 172.21.15.44:0/3957595599 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e708d34050 con 0x55e708e96d00 2023-09-10T01:03:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.104+0000 7f33b2ffd700 1 --2- 172.21.15.44:0/3957595599 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e708e96d00 0x55e708e99120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:03:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.104+0000 7f3396ff5700 10 client.18579 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.104+0000 7f33b2ffd700 1 --2- 172.21.15.44:0/3957595599 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e708e96d00 0x55e708e99120 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ffa162b57a4662e server_cookie=35cf6bec322ff25c in_seq=0 out_seq=0 2023-09-10T01:03:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.127+0000 7f3396ff5700 1 -- 172.21.15.44:0/3957595599 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e708d34050 con 0x55e708e96d00 2023-09-10T01:03:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.127+0000 7f3396ff5700 10 client.18579 handle_command_reply: tid=0 2023-09-10T01:03:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.127+0000 7f33c5478b80 1 client.18579 shutdown 2023-09-10T01:03:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.127+0000 7f33c5478b80 1 -- 172.21.15.44:0/3957595599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e2e1a0 msgr2=0x55e708e37fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.127+0000 7f33c5478b80 1 --2- 172.21.15.44:0/3957595599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e2e1a0 0x55e708e37fb0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0ca9c0 tx=0x7f33ac09a1a0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.127+0000 7f33c5478b80 1 -- 172.21.15.44:0/3957595599 shutdown_connections 2023-09-10T01:03:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.127+0000 7f33c5478b80 1 --2- 172.21.15.44:0/3957595599 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e708e96d00 0x55e708e99120 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.127+0000 7f33c5478b80 1 --2- 172.21.15.44:0/3957595599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e708e2e1a0 0x55e708e37fb0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.127+0000 7f33c5478b80 1 --2- 172.21.15.44:0/3957595599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e708e28300 0x55e708e37510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.127+0000 7f33c5478b80 1 --2- 172.21.15.44:0/3957595599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e708e28cc0 0x55e708e37a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.127+0000 7f3396ff5700 0 client.18579 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:03:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.127+0000 7f33c5478b80 1 -- 172.21.15.44:0/3957595599 >> 172.21.15.44:0/3957595599 conn(0x55e708d9d030 msgr2=0x55e708e2c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.127+0000 7f33c5478b80 1 -- 172.21.15.44:0/3957595599 shutdown_connections 2023-09-10T01:03:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.127+0000 7f33c5478b80 1 -- 172.21.15.44:0/3957595599 wait complete. 2023-09-10T01:03:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.128+0000 7f33c5478b80 20 client.18579 trim_cache size 0 max 16384 2023-09-10T01:03:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:03:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:03:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:03:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:03:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.128+0000 7f33b92b2700 1 -- 172.21.15.44:0/496104 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f339c0a0830 msgr2=0x7f339c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.128+0000 7f33b92b2700 1 --2- 172.21.15.44:0/496104 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f339c0a0830 0x7f339c0a2cf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f33a4011550 tx=0x7f33a4009260 comp rx=0 tx=0).stop 2023-09-10T01:03:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.128+0000 7f33b92b2700 1 -- 172.21.15.44:0/496104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b41055a0 msgr2=0x7f33b41a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:03:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.128+0000 7f33b92b2700 1 --2- 172.21.15.44:0/496104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b41055a0 0x7f33b41a2520 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00b770 tx=0x7f33ac008e50 comp rx=0 tx=0).stop 2023-09-10T01:03:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.128+0000 7f33b92b2700 1 -- 172.21.15.44:0/496104 shutdown_connections 2023-09-10T01:03:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.128+0000 7f33b92b2700 1 --2- 172.21.15.44:0/496104 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f339c0a0830 0x7f339c0a2cf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.128+0000 7f33b92b2700 1 --2- 172.21.15.44:0/496104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b41055a0 0x7f33b41a2520 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.128+0000 7f33b92b2700 1 --2- 172.21.15.44:0/496104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b40ff4e0 0x7f33b4197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.128+0000 7f33b92b2700 1 --2- 172.21.15.44:0/496104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33b40ffea0 0x7f33b4197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:03:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.128+0000 7f33b92b2700 1 -- 172.21.15.44:0/496104 >> 172.21.15.44:0/496104 conn(0x7f33b4076670 msgr2=0x7f33b40fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:03:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.128+0000 7f33b92b2700 1 -- 172.21.15.44:0/496104 shutdown_connections 2023-09-10T01:03:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:03:56.128+0000 7f33b92b2700 1 -- 172.21.15.44:0/496104 wait complete. 2023-09-10T01:03:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:56 smithi131 ceph-mon[103713]: pgmap v738: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 254 op/s 2023-09-10T01:03:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:56 smithi044 ceph-mon[98676]: pgmap v738: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 254 op/s 2023-09-10T01:03:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:56 smithi099 ceph-mon[102692]: pgmap v738: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 254 op/s 2023-09-10T01:03:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:03:58 smithi131 ceph-mon[103713]: pgmap v739: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2023-09-10T01:03:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:03:58 smithi044 ceph-mon[98676]: pgmap v739: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2023-09-10T01:03:59.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:03:58 smithi099 ceph-mon[102692]: pgmap v739: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2023-09-10T01:04:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:00 smithi131 ceph-mon[103713]: pgmap v740: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2023-09-10T01:04:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:00 smithi044 ceph-mon[98676]: pgmap v740: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2023-09-10T01:04:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:00 smithi099 ceph-mon[102692]: pgmap v740: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2023-09-10T01:04:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:02 smithi131 ceph-mon[103713]: pgmap v741: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 232 op/s 2023-09-10T01:04:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:02 smithi044 ceph-mon[98676]: pgmap v741: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 232 op/s 2023-09-10T01:04:03.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:02 smithi099 ceph-mon[102692]: pgmap v741: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 232 op/s 2023-09-10T01:04:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:04 smithi131 ceph-mon[103713]: pgmap v742: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 934 KiB/s wr, 134 op/s 2023-09-10T01:04:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:04 smithi044 ceph-mon[98676]: pgmap v742: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 934 KiB/s wr, 134 op/s 2023-09-10T01:04:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:04 smithi099 ceph-mon[102692]: pgmap v742: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 934 KiB/s wr, 134 op/s 2023-09-10T01:04:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:06 smithi131 ceph-mon[103713]: pgmap v743: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 240 op/s 2023-09-10T01:04:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:06 smithi044 ceph-mon[98676]: pgmap v743: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 240 op/s 2023-09-10T01:04:07.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:06 smithi099 ceph-mon[102692]: pgmap v743: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 240 op/s 2023-09-10T01:04:09.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:08 smithi131 ceph-mon[103713]: pgmap v744: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 171 op/s 2023-09-10T01:04:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:08 smithi044 ceph-mon[98676]: pgmap v744: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 171 op/s 2023-09-10T01:04:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.320+0000 7fa402c34700 1 -- 172.21.15.44:0/2751577366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3fc107870 msgr2=0x7fa3fc107c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.320+0000 7fa402c34700 1 --2- 172.21.15.44:0/2751577366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3fc107870 0x7fa3fc107c50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009990 tx=0x7fa3f0009ca0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.320+0000 7fa402c34700 1 -- 172.21.15.44:0/2751577366 shutdown_connections 2023-09-10T01:04:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.320+0000 7fa402c34700 1 --2- 172.21.15.44:0/2751577366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3fc10bf10 0x7fa3fc0788f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.320+0000 7fa402c34700 1 --2- 172.21.15.44:0/2751577366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3fc107870 0x7fa3fc107c50 secure :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009990 tx=0x7fa3f0009ca0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.320+0000 7fa402c34700 1 --2- 172.21.15.44:0/2751577366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3fc108220 0x7fa3fc10b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.320+0000 7fa402c34700 1 -- 172.21.15.44:0/2751577366 >> 172.21.15.44:0/2751577366 conn(0x7fa3fc06c6c0 msgr2=0x7fa3fc06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.330+0000 7fa402c34700 1 -- 172.21.15.44:0/2751577366 shutdown_connections 2023-09-10T01:04:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.330+0000 7fa402c34700 1 -- 172.21.15.44:0/2751577366 wait complete. 2023-09-10T01:04:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.331+0000 7fa402c34700 1 Processor -- start 2023-09-10T01:04:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.331+0000 7fa402c34700 1 -- start start 2023-09-10T01:04:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.331+0000 7fa402c34700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3fc10bf10 0x7fa3fc0830c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.331+0000 7fa402c34700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3fc07ce90 0x7fa3fc07d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.331+0000 7fa402c34700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3fc07d850 0x7fa3fc1c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.331+0000 7fa402c34700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc1c34e0 con 0x7fa3fc10bf10 2023-09-10T01:04:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.331+0000 7fa402c34700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc1c3620 con 0x7fa3fc07ce90 2023-09-10T01:04:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.331+0000 7fa402c34700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc1c3790 con 0x7fa3fc07d850 2023-09-10T01:04:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.332+0000 7fa3fbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3fc07ce90 0x7fa3fc07d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.332+0000 7fa3fbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3fc07ce90 0x7fa3fc07d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57094/0 (socket says 172.21.15.44:57094) 2023-09-10T01:04:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.332+0000 7fa3fbfff700 1 -- 172.21.15.44:0/1756995280 learned_addr learned my addr 172.21.15.44:0/1756995280 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.332+0000 7fa4009d0700 1 --2- 172.21.15.44:0/1756995280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3fc10bf10 0x7fa3fc0830c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.334+0000 7fa4011d1700 1 --2- 172.21.15.44:0/1756995280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3fc07d850 0x7fa3fc1c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.334+0000 7fa4011d1700 1 -- 172.21.15.44:0/1756995280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3fc07ce90 msgr2=0x7fa3fc07d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.334+0000 7fa4011d1700 1 --2- 172.21.15.44:0/1756995280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3fc07ce90 0x7fa3fc07d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.334+0000 7fa4011d1700 1 -- 172.21.15.44:0/1756995280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3fc10bf10 msgr2=0x7fa3fc0830c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.334+0000 7fa4011d1700 1 --2- 172.21.15.44:0/1756995280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3fc10bf10 0x7fa3fc0830c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.334+0000 7fa4011d1700 1 -- 172.21.15.44:0/1756995280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f00095f0 con 0x7fa3fc07d850 2023-09-10T01:04:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.334+0000 7fa4011d1700 1 --2- 172.21.15.44:0/1756995280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3fc07d850 0x7fa3fc1c2fa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00b3c0 tx=0x7fa3ec00b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.336+0000 7fa3f9ffb700 1 -- 172.21.15.44:0/1756995280 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec011620 con 0x7fa3fc07d850 2023-09-10T01:04:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.336+0000 7fa3f9ffb700 1 -- 172.21.15.44:0/1756995280 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3ec011c10 con 0x7fa3fc07d850 2023-09-10T01:04:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.336+0000 7fa3f9ffb700 1 -- 172.21.15.44:0/1756995280 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec010be0 con 0x7fa3fc07d850 2023-09-10T01:04:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.337+0000 7fa402c34700 1 -- 172.21.15.44:0/1756995280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3fc1c3a70 con 0x7fa3fc07d850 2023-09-10T01:04:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.337+0000 7fa402c34700 1 -- 172.21.15.44:0/1756995280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc1c3f00 con 0x7fa3fc07d850 2023-09-10T01:04:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.341+0000 7fa3f9ffb700 1 -- 172.21.15.44:0/1756995280 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa3ec010d40 con 0x7fa3fc07d850 2023-09-10T01:04:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.342+0000 7fa3f9ffb700 1 --2- 172.21.15.44:0/1756995280 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa3e40a0940 0x7fa3e40a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.343+0000 7fa40edfab80 1 Processor -- start 2023-09-10T01:04:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.343+0000 7fa3f9ffb700 1 -- 172.21.15.44:0/1756995280 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fa3ec0cb8f0 con 0x7fa3fc07d850 2023-09-10T01:04:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.343+0000 7fa40edfab80 1 -- start start 2023-09-10T01:04:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.343+0000 7fa40edfab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac03c1b0 0x5595ac03c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.343+0000 7fa40edfab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac036310 0x5595ac036790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.343+0000 7fa40edfab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036cd0 0x5595ac042f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.343+0000 7fa40edfab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595abf7e6d0 con 0x5595ac036cd0 2023-09-10T01:04:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.343+0000 7fa40edfab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595abf7bff0 con 0x5595ac036310 2023-09-10T01:04:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.343+0000 7fa40edfab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595abf778e0 con 0x5595ac03c1b0 2023-09-10T01:04:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.343+0000 7fa4011d1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036cd0 0x5595ac042f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.343+0000 7fa4011d1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036cd0 0x5595ac042f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55362/0 (socket says 172.21.15.44:55362) 2023-09-10T01:04:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.343+0000 7fa4011d1700 1 -- 172.21.15.44:0/4181944613 learned_addr learned my addr 172.21.15.44:0/4181944613 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.343+0000 7fa3fbfff700 1 --2- 172.21.15.44:0/4181944613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac036310 0x5595ac036790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.344+0000 7fa4011d1700 1 -- 172.21.15.44:0/4181944613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac03c1b0 msgr2=0x5595ac03c590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:04:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.344+0000 7fa4011d1700 1 --2- 172.21.15.44:0/4181944613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac03c1b0 0x5595ac03c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.344+0000 7fa4011d1700 1 -- 172.21.15.44:0/4181944613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac036310 msgr2=0x5595ac036790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.344+0000 7fa4011d1700 1 --2- 172.21.15.44:0/4181944613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac036310 0x5595ac036790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.344+0000 7fa4011d1700 1 -- 172.21.15.44:0/4181944613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595abda3d80 con 0x5595ac036cd0 2023-09-10T01:04:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.344+0000 7fa4011d1700 1 --2- 172.21.15.44:0/4181944613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036cd0 0x5595ac042f40 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec094f00 tx=0x7fa3ec094e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb623a7a4e5518d8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.349+0000 7fa4009d0700 1 --2- 172.21.15.44:0/1756995280 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa3e40a0940 0x7fa3e40a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.350+0000 7fa4009d0700 1 --2- 172.21.15.44:0/1756995280 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa3e40a0940 0x7fa3e40a2e00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0000c00 tx=0x7fa3f0005fb0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.351+0000 7fa3e2ffd700 1 -- 172.21.15.44:0/4181944613 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec099100 con 0x5595ac036cd0 2023-09-10T01:04:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.351+0000 7fa3e2ffd700 1 -- 172.21.15.44:0/4181944613 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3ec098a00 con 0x5595ac036cd0 2023-09-10T01:04:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.351+0000 7fa3e2ffd700 1 -- 172.21.15.44:0/4181944613 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec056f60 con 0x5595ac036cd0 2023-09-10T01:04:09.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:08 smithi099 ceph-mon[102692]: pgmap v744: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 171 op/s 2023-09-10T01:04:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.352+0000 7fa40edfab80 1 -- 172.21.15.44:0/4181944613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036cd0 msgr2=0x5595ac042f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.352+0000 7fa40edfab80 1 --2- 172.21.15.44:0/4181944613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036cd0 0x5595ac042f40 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec094f00 tx=0x7fa3ec094e70 comp rx=0 tx=0).stop 2023-09-10T01:04:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.352+0000 7fa40edfab80 1 -- 172.21.15.44:0/4181944613 shutdown_connections 2023-09-10T01:04:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.352+0000 7fa40edfab80 1 --2- 172.21.15.44:0/4181944613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036cd0 0x5595ac042f40 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.352+0000 7fa40edfab80 1 --2- 172.21.15.44:0/4181944613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac03c1b0 0x5595ac03c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.352+0000 7fa40edfab80 1 --2- 172.21.15.44:0/4181944613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac036310 0x5595ac036790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.352+0000 7fa40edfab80 1 -- 172.21.15.44:0/4181944613 >> 172.21.15.44:0/4181944613 conn(0x5595abf9a030 msgr2=0x5595ac020560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.354+0000 7fa40edfab80 1 -- 172.21.15.44:0/4181944613 shutdown_connections 2023-09-10T01:04:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.354+0000 7fa40edfab80 1 -- 172.21.15.44:0/4181944613 wait complete. 2023-09-10T01:04:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.354+0000 7fa40edfab80 1 -- start start 2023-09-10T01:04:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.354+0000 7fa40edfab80 1 Processor -- start 2023-09-10T01:04:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.359+0000 7fa40edfab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036310 0x5595ac02d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.359+0000 7fa40edfab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac036cd0 0x5595ac02d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.359+0000 7fa40edfab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac03c1b0 0x5595ac0285c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.359+0000 7fa40edfab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595abf15b10 con 0x5595ac036310 2023-09-10T01:04:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.359+0000 7fa40edfab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595abf15810 con 0x5595ac03c1b0 2023-09-10T01:04:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.359+0000 7fa40edfab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595abdf98c0 con 0x5595ac036cd0 2023-09-10T01:04:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.359+0000 7fa40edfab80 10 client.0 _ll_register_callbacks cb 0x5595abe5fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:04:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.359+0000 7fa3fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac036cd0 0x5595ac02d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.359+0000 7fa3fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac036cd0 0x5595ac02d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34016/0 (socket says 172.21.15.44:34016) 2023-09-10T01:04:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.359+0000 7fa3fbfff700 1 -- 172.21.15.44:0/1471514317 learned_addr learned my addr 172.21.15.44:0/1471514317 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.360+0000 7fa3fbfff700 1 -- 172.21.15.44:0/1471514317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac03c1b0 msgr2=0x5595ac0285c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:04:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.360+0000 7fa3fbfff700 1 --2- 172.21.15.44:0/1471514317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac03c1b0 0x5595ac0285c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.360+0000 7fa3fbfff700 1 -- 172.21.15.44:0/1471514317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036310 msgr2=0x5595ac02d2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:04:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.360+0000 7fa3fbfff700 1 --2- 172.21.15.44:0/1471514317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036310 0x5595ac02d2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.360+0000 7fa3fbfff700 1 -- 172.21.15.44:0/1471514317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc1c3f00 con 0x5595ac036cd0 2023-09-10T01:04:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.360+0000 7fa3fbfff700 1 --2- 172.21.15.44:0/1471514317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac036cd0 0x5595ac02d830 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4011ca0 tx=0x7fa3f400f490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcd665870bf0e109 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.360+0000 7fa3dbfff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:04:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.360+0000 7fa3dbfff700 1 -- 172.21.15.44:0/1471514317 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f400e040 con 0x5595ac036cd0 2023-09-10T01:04:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.360+0000 7fa40edfab80 1 -- 172.21.15.44:0/1471514317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595abecf490 con 0x5595ac036cd0 2023-09-10T01:04:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.360+0000 7fa3dbfff700 1 -- 172.21.15.44:0/1471514317 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fa3f400fc20 con 0x5595ac036cd0 2023-09-10T01:04:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.360+0000 7fa3dbfff700 1 -- 172.21.15.44:0/1471514317 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3f400fe20 con 0x5595ac036cd0 2023-09-10T01:04:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.362+0000 7fa40edfab80 10 client.28180 fetch_fsmap learned FSMap version 96 2023-09-10T01:04:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.362+0000 7fa40edfab80 1 -- 172.21.15.44:0/1471514317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) v3 -- 0x5595abd8cc00 con 0x5595ac036cd0 2023-09-10T01:04:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.362+0000 7fa3dbfff700 1 -- 172.21.15.44:0/1471514317 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5595abecf490 con 0x5595ac036cd0 2023-09-10T01:04:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.363+0000 7fa3dbfff700 1 -- 172.21.15.44:0/1471514317 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 96) v1 ==== 7938+0+0 (secure 0 0 0) 0x7fa3f401b020 con 0x5595ac036cd0 2023-09-10T01:04:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.363+0000 7fa40edfab80 10 client.28180 fetch_fsmap finished waiting for FSMap version 96 2023-09-10T01:04:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.363+0000 7fa40edfab80 10 client.28180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:04:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.363+0000 7fa40edfab80 20 client.28180 populate_metadata read hostname 'smithi044' 2023-09-10T01:04:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.363+0000 7fa40edfab80 1 --2- 172.21.15.44:0/1471514317 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5595ac03f570 0x5595ac041990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.363+0000 7fa40edfab80 4 client.28180 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:04:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.363+0000 7fa40edfab80 1 -- 172.21.15.44:0/1471514317 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595abf8c840 con 0x5595ac03f570 2023-09-10T01:04:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.363+0000 7fa4009d0700 1 --2- 172.21.15.44:0/1471514317 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5595ac03f570 0x5595ac041990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.368+0000 7fa4009d0700 1 --2- 172.21.15.44:0/1471514317 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5595ac03f570 0x5595ac041990 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44e8b106fddb0e4e server_cookie=349c2cfca02aefc0 in_seq=0 out_seq=0 2023-09-10T01:04:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.368+0000 7fa3dbfff700 10 client.28180 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:04:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.369+0000 7fa3dbfff700 1 -- 172.21.15.44:0/1471514317 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5595abf8c840 con 0x5595ac03f570 2023-09-10T01:04:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.369+0000 7fa3dbfff700 10 client.28180 handle_command_reply: tid=0 2023-09-10T01:04:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.370+0000 7fa40edfab80 1 client.28180 shutdown 2023-09-10T01:04:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.380+0000 7fa40edfab80 1 -- 172.21.15.44:0/1471514317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac036cd0 msgr2=0x5595ac02d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.380+0000 7fa40edfab80 1 --2- 172.21.15.44:0/1471514317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac036cd0 0x5595ac02d830 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4011ca0 tx=0x7fa3f400f490 comp rx=0 tx=0).stop 2023-09-10T01:04:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.387+0000 7fa40edfab80 1 -- 172.21.15.44:0/1471514317 shutdown_connections 2023-09-10T01:04:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.387+0000 7fa40edfab80 1 --2- 172.21.15.44:0/1471514317 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5595ac03f570 0x5595ac041990 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.387+0000 7fa40edfab80 1 --2- 172.21.15.44:0/1471514317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac03c1b0 0x5595ac0285c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.387+0000 7fa40edfab80 1 --2- 172.21.15.44:0/1471514317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036310 0x5595ac02d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.387+0000 7fa40edfab80 1 --2- 172.21.15.44:0/1471514317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac036cd0 0x5595ac02d830 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.387+0000 7fa40edfab80 1 -- 172.21.15.44:0/1471514317 >> 172.21.15.44:0/1471514317 conn(0x5595abf9a030 msgr2=0x5595ac01f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.387+0000 7fa3dbfff700 0 client.28180 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:04:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.392+0000 7fa40edfab80 1 -- 172.21.15.44:0/1471514317 shutdown_connections 2023-09-10T01:04:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.396+0000 7fa40edfab80 1 -- 172.21.15.44:0/1471514317 wait complete. 2023-09-10T01:04:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.406+0000 7fa40edfab80 20 client.28180 trim_cache size 0 max 16384 2023-09-10T01:04:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.431+0000 7fa40edfab80 1 Processor -- start 2023-09-10T01:04:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.431+0000 7fa40edfab80 1 -- start start 2023-09-10T01:04:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.432+0000 7fa40edfab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036310 0x5595ac034df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.432+0000 7fa40edfab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac036cd0 0x5595ac01ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.432+0000 7fa40edfab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac03c1b0 0x5595ac020460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.432+0000 7fa40edfab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ac037600 con 0x5595ac036310 2023-09-10T01:04:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.432+0000 7fa40edfab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ac029430 con 0x5595ac03c1b0 2023-09-10T01:04:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.432+0000 7fa40edfab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ac039dc0 con 0x5595ac036cd0 2023-09-10T01:04:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.432+0000 7fa4011d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac03c1b0 0x5595ac020460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.433+0000 7fa4009d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036310 0x5595ac034df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.433+0000 7fa4009d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036310 0x5595ac034df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55372/0 (socket says 172.21.15.44:55372) 2023-09-10T01:04:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.433+0000 7fa4009d0700 1 -- 172.21.15.44:0/1837782014 learned_addr learned my addr 172.21.15.44:0/1837782014 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.433+0000 7fa4009d0700 1 -- 172.21.15.44:0/1837782014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac036cd0 msgr2=0x5595ac01ff20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:04:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.433+0000 7fa4009d0700 1 --2- 172.21.15.44:0/1837782014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac036cd0 0x5595ac01ff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.433+0000 7fa4009d0700 1 -- 172.21.15.44:0/1837782014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac03c1b0 msgr2=0x5595ac020460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.433+0000 7fa4009d0700 1 --2- 172.21.15.44:0/1837782014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac03c1b0 0x5595ac020460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.433+0000 7fa4009d0700 1 -- 172.21.15.44:0/1837782014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595abdca1f0 con 0x5595ac036310 2023-09-10T01:04:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.433+0000 7fa4009d0700 1 --2- 172.21.15.44:0/1837782014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036310 0x5595ac034df0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000b5c0 tx=0x7fa3f0023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd89071128445fd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.437+0000 7fa3e37fe700 1 -- 172.21.15.44:0/1837782014 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f002b040 con 0x5595ac036310 2023-09-10T01:04:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.437+0000 7fa3e37fe700 1 -- 172.21.15.44:0/1837782014 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3f001e040 con 0x5595ac036310 2023-09-10T01:04:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.437+0000 7fa3e37fe700 1 -- 172.21.15.44:0/1837782014 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f000bec0 con 0x5595ac036310 2023-09-10T01:04:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.444+0000 7fa40edfab80 1 -- 172.21.15.44:0/1837782014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036310 msgr2=0x5595ac034df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.444+0000 7fa40edfab80 1 --2- 172.21.15.44:0/1837782014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036310 0x5595ac034df0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000b5c0 tx=0x7fa3f0023000 comp rx=0 tx=0).stop 2023-09-10T01:04:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.444+0000 7fa40edfab80 1 -- 172.21.15.44:0/1837782014 shutdown_connections 2023-09-10T01:04:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.444+0000 7fa40edfab80 1 --2- 172.21.15.44:0/1837782014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac03c1b0 0x5595ac020460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.444+0000 7fa40edfab80 1 --2- 172.21.15.44:0/1837782014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036310 0x5595ac034df0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.444+0000 7fa40edfab80 1 --2- 172.21.15.44:0/1837782014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac036cd0 0x5595ac01ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.444+0000 7fa40edfab80 1 -- 172.21.15.44:0/1837782014 >> 172.21.15.44:0/1837782014 conn(0x5595abf9a030 msgr2=0x5595ac0346e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.444+0000 7fa40edfab80 1 -- 172.21.15.44:0/1837782014 shutdown_connections 2023-09-10T01:04:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.445+0000 7fa40edfab80 1 -- 172.21.15.44:0/1837782014 wait complete. 2023-09-10T01:04:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.445+0000 7fa40edfab80 1 -- start start 2023-09-10T01:04:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.445+0000 7fa40edfab80 1 Processor -- start 2023-09-10T01:04:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.447+0000 7fa40edfab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036310 0x5595ac032040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.447+0000 7fa40edfab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac036cd0 0x5595ac030460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.447+0000 7fa40edfab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac03c1b0 0x5595ac030a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.447+0000 7fa40edfab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ac031140 con 0x5595ac036310 2023-09-10T01:04:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.447+0000 7fa40edfab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ac0312b0 con 0x5595ac036cd0 2023-09-10T01:04:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.447+0000 7fa40edfab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ac093390 con 0x5595ac03c1b0 2023-09-10T01:04:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.447+0000 7fa40edfab80 10 client.0 _ll_register_callbacks cb 0x7fa3f40115d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:04:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.447+0000 7fa3fbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac036cd0 0x5595ac030460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.447+0000 7fa3fbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac036cd0 0x5595ac030460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57114/0 (socket says 172.21.15.44:57114) 2023-09-10T01:04:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.447+0000 7fa3fbfff700 1 -- 172.21.15.44:0/2794942291 learned_addr learned my addr 172.21.15.44:0/2794942291 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.448+0000 7fa3fbfff700 1 -- 172.21.15.44:0/2794942291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac03c1b0 msgr2=0x5595ac030a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:04:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.448+0000 7fa3fbfff700 1 --2- 172.21.15.44:0/2794942291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac03c1b0 0x5595ac030a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.448+0000 7fa3fbfff700 1 -- 172.21.15.44:0/2794942291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036310 msgr2=0x5595ac032040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:04:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.448+0000 7fa3fbfff700 1 --2- 172.21.15.44:0/2794942291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036310 0x5595ac032040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.448+0000 7fa3fbfff700 1 -- 172.21.15.44:0/2794942291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc0791b0 con 0x5595ac036cd0 2023-09-10T01:04:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.451+0000 7fa3fbfff700 1 --2- 172.21.15.44:0/2794942291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac036cd0 0x5595ac030460 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4014040 tx=0x7fa3f4009b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a71e1ef6b8a411a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.451+0000 7fa3dbfff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:04:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.452+0000 7fa3dbfff700 1 -- 172.21.15.44:0/2794942291 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f400e040 con 0x5595ac036cd0 2023-09-10T01:04:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.452+0000 7fa3dbfff700 1 -- 172.21.15.44:0/2794942291 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fa3f400fa20 con 0x5595ac036cd0 2023-09-10T01:04:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.452+0000 7fa40edfab80 1 -- 172.21.15.44:0/2794942291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3ec098a00 con 0x5595ac036cd0 2023-09-10T01:04:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.458+0000 7fa3dbfff700 1 -- 172.21.15.44:0/2794942291 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3f400fbe0 con 0x5595ac036cd0 2023-09-10T01:04:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.459+0000 7fa40edfab80 10 client.28244 fetch_fsmap learned FSMap version 96 2023-09-10T01:04:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.459+0000 7fa40edfab80 1 -- 172.21.15.44:0/2794942291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) v3 -- 0x5595abda2250 con 0x5595ac036cd0 2023-09-10T01:04:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.462+0000 7fa3dbfff700 1 -- 172.21.15.44:0/2794942291 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3ec098a00 con 0x5595ac036cd0 2023-09-10T01:04:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.462+0000 7fa3dbfff700 1 -- 172.21.15.44:0/2794942291 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 96) v1 ==== 7938+0+0 (secure 0 0 0) 0x7fa3f400f1b0 con 0x5595ac036cd0 2023-09-10T01:04:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.462+0000 7fa40edfab80 10 client.28244 fetch_fsmap finished waiting for FSMap version 96 2023-09-10T01:04:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.462+0000 7fa40edfab80 10 client.28244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:04:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.462+0000 7fa40edfab80 20 client.28244 populate_metadata read hostname 'smithi044' 2023-09-10T01:04:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.462+0000 7fa40edfab80 1 --2- 172.21.15.44:0/2794942291 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5595ac094c10 0x5595ac097030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.462+0000 7fa40edfab80 4 client.28244 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:04:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.462+0000 7fa40edfab80 1 -- 172.21.15.44:0/2794942291 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5595abf708c0 con 0x5595ac094c10 2023-09-10T01:04:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.463+0000 7fa4009d0700 1 --2- 172.21.15.44:0/2794942291 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5595ac094c10 0x5595ac097030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.463+0000 7fa4009d0700 1 --2- 172.21.15.44:0/2794942291 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5595ac094c10 0x5595ac097030 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81c13e7ae4d70f8c server_cookie=a79394edf5e18fe1 in_seq=0 out_seq=0 2023-09-10T01:04:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.463+0000 7fa3dbfff700 10 client.28244 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:04:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.559+0000 7fa3dbfff700 1 -- 172.21.15.44:0/2794942291 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5595abf708c0 con 0x5595ac094c10 2023-09-10T01:04:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.559+0000 7fa3dbfff700 10 client.28244 handle_command_reply: tid=0 2023-09-10T01:04:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.563+0000 7fa40edfab80 1 client.28244 shutdown 2023-09-10T01:04:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.565+0000 7fa40edfab80 1 -- 172.21.15.44:0/2794942291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac036cd0 msgr2=0x5595ac030460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.565+0000 7fa40edfab80 1 --2- 172.21.15.44:0/2794942291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac036cd0 0x5595ac030460 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4014040 tx=0x7fa3f4009b70 comp rx=0 tx=0).stop 2023-09-10T01:04:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.572+0000 7fa40edfab80 1 -- 172.21.15.44:0/2794942291 shutdown_connections 2023-09-10T01:04:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.572+0000 7fa40edfab80 1 --2- 172.21.15.44:0/2794942291 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5595ac094c10 0x5595ac097030 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.572+0000 7fa40edfab80 1 --2- 172.21.15.44:0/2794942291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595ac03c1b0 0x5595ac030a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.572+0000 7fa40edfab80 1 --2- 172.21.15.44:0/2794942291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ac036310 0x5595ac032040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.572+0000 7fa40edfab80 1 --2- 172.21.15.44:0/2794942291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595ac036cd0 0x5595ac030460 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.572+0000 7fa40edfab80 1 -- 172.21.15.44:0/2794942291 >> 172.21.15.44:0/2794942291 conn(0x5595abf9a030 msgr2=0x5595ac0242f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.573+0000 7fa3dbfff700 0 client.28244 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:04:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.584+0000 7fa40edfab80 1 -- 172.21.15.44:0/2794942291 shutdown_connections 2023-09-10T01:04:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.586+0000 7fa40edfab80 1 -- 172.21.15.44:0/2794942291 wait complete. 2023-09-10T01:04:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.591+0000 7fa40edfab80 20 client.28244 trim_cache size 0 max 16384 2023-09-10T01:04:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:04:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:04:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:04:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:04:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.592+0000 7fa402c34700 1 -- 172.21.15.44:0/1756995280 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa3e40a0940 msgr2=0x7fa3e40a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.592+0000 7fa402c34700 1 --2- 172.21.15.44:0/1756995280 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa3e40a0940 0x7fa3e40a2e00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0000c00 tx=0x7fa3f0005fb0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.592+0000 7fa402c34700 1 -- 172.21.15.44:0/1756995280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3fc07d850 msgr2=0x7fa3fc1c2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.592+0000 7fa402c34700 1 --2- 172.21.15.44:0/1756995280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3fc07d850 0x7fa3fc1c2fa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00b3c0 tx=0x7fa3ec00b6d0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.592+0000 7fa402c34700 1 -- 172.21.15.44:0/1756995280 shutdown_connections 2023-09-10T01:04:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.592+0000 7fa402c34700 1 --2- 172.21.15.44:0/1756995280 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa3e40a0940 0x7fa3e40a2e00 secure :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0000c00 tx=0x7fa3f0005fb0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.592+0000 7fa402c34700 1 --2- 172.21.15.44:0/1756995280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3fc07d850 0x7fa3fc1c2fa0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.592+0000 7fa402c34700 1 --2- 172.21.15.44:0/1756995280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3fc10bf10 0x7fa3fc0830c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.592+0000 7fa402c34700 1 --2- 172.21.15.44:0/1756995280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3fc07ce90 0x7fa3fc07d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.592+0000 7fa402c34700 1 -- 172.21.15.44:0/1756995280 >> 172.21.15.44:0/1756995280 conn(0x7fa3fc06c6c0 msgr2=0x7fa3fc1099b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.602+0000 7fa402c34700 1 -- 172.21.15.44:0/1756995280 shutdown_connections 2023-09-10T01:04:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:09.602+0000 7fa402c34700 1 -- 172.21.15.44:0/1756995280 wait complete. 2023-09-10T01:04:10.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:09 smithi131 ceph-mon[103713]: osdmap e130: 12 total, 12 up, 12 in 2023-09-10T01:04:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:09 smithi044 ceph-mon[98676]: osdmap e130: 12 total, 12 up, 12 in 2023-09-10T01:04:10.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:09 smithi099 ceph-mon[102692]: osdmap e130: 12 total, 12 up, 12 in 2023-09-10T01:04:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:11 smithi131 ceph-mon[103713]: pgmap v746: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 205 op/s 2023-09-10T01:04:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:11 smithi044 ceph-mon[98676]: pgmap v746: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 205 op/s 2023-09-10T01:04:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:11 smithi099 ceph-mon[102692]: pgmap v746: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 205 op/s 2023-09-10T01:04:12.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:12 smithi131 ceph-mon[103713]: pgmap v747: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2023-09-10T01:04:12.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:12 smithi044 ceph-mon[98676]: pgmap v747: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2023-09-10T01:04:12.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:12 smithi099 ceph-mon[102692]: pgmap v747: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2023-09-10T01:04:14.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:14 smithi131 ceph-mon[103713]: pgmap v748: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2023-09-10T01:04:14.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:14 smithi044 ceph-mon[98676]: pgmap v748: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2023-09-10T01:04:14.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:14 smithi099 ceph-mon[102692]: pgmap v748: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2023-09-10T01:04:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:16 smithi099 ceph-mon[102692]: pgmap v749: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 183 op/s 2023-09-10T01:04:17.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:16 smithi099 ceph-mon[102692]: osdmap e131: 12 total, 12 up, 12 in 2023-09-10T01:04:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:16 smithi131 ceph-mon[103713]: pgmap v749: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 183 op/s 2023-09-10T01:04:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:16 smithi131 ceph-mon[103713]: osdmap e131: 12 total, 12 up, 12 in 2023-09-10T01:04:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:16 smithi044 ceph-mon[98676]: pgmap v749: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 183 op/s 2023-09-10T01:04:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:16 smithi044 ceph-mon[98676]: osdmap e131: 12 total, 12 up, 12 in 2023-09-10T01:04:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:18 smithi099 ceph-mon[102692]: pgmap v751: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T01:04:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:18 smithi131 ceph-mon[103713]: pgmap v751: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T01:04:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:18 smithi044 ceph-mon[98676]: pgmap v751: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T01:04:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:20 smithi099 ceph-mon[102692]: pgmap v752: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 183 op/s 2023-09-10T01:04:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:20 smithi131 ceph-mon[103713]: pgmap v752: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 183 op/s 2023-09-10T01:04:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:20 smithi044 ceph-mon[98676]: pgmap v752: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 183 op/s 2023-09-10T01:04:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.749+0000 7faac826e700 1 -- 172.21.15.44:0/1817122941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac01005d0 msgr2=0x7faac0100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.749+0000 7faac826e700 1 --2- 172.21.15.44:0/1817122941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac01005d0 0x7faac0100a50 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7faab00099c0 tx=0x7faab0009cd0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.749+0000 7faac826e700 1 -- 172.21.15.44:0/1817122941 shutdown_connections 2023-09-10T01:04:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.749+0000 7faac826e700 1 --2- 172.21.15.44:0/1817122941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faac0100f90 0x7faac010d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.749+0000 7faac826e700 1 --2- 172.21.15.44:0/1817122941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac0106690 0x7faac0106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.749+0000 7faac826e700 1 --2- 172.21.15.44:0/1817122941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac01005d0 0x7faac0100a50 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.749+0000 7faac826e700 1 -- 172.21.15.44:0/1817122941 >> 172.21.15.44:0/1817122941 conn(0x7faac00fc0c0 msgr2=0x7faac00fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:22.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.749+0000 7faac826e700 1 -- 172.21.15.44:0/1817122941 shutdown_connections 2023-09-10T01:04:22.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.749+0000 7faac826e700 1 -- 172.21.15.44:0/1817122941 wait complete. 2023-09-10T01:04:22.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.750+0000 7faac826e700 1 Processor -- start 2023-09-10T01:04:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.750+0000 7faac826e700 1 -- start start 2023-09-10T01:04:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.750+0000 7faac826e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faac01005d0 0x7faac019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.750+0000 7faac826e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac0100f90 0x7faac019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.750+0000 7faac826e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac0106690 0x7faac0196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.750+0000 7faac826e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac01969f0 con 0x7faac0106690 2023-09-10T01:04:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.750+0000 7faac826e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac0196b60 con 0x7faac0100f90 2023-09-10T01:04:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.750+0000 7faac826e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac0196d20 con 0x7faac01005d0 2023-09-10T01:04:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.750+0000 7faac680b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac0106690 0x7faac0196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.750+0000 7faac5809700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac0100f90 0x7faac019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.750+0000 7faac600a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faac01005d0 0x7faac019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.750+0000 7faac680b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac0106690 0x7faac0196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55388/0 (socket says 172.21.15.44:55388) 2023-09-10T01:04:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.750+0000 7faac680b700 1 -- 172.21.15.44:0/600853221 learned_addr learned my addr 172.21.15.44:0/600853221 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.751+0000 7faac5809700 1 -- 172.21.15.44:0/600853221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faac01005d0 msgr2=0x7faac019c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.751+0000 7faac5809700 1 --2- 172.21.15.44:0/600853221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faac01005d0 0x7faac019c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.751+0000 7faac5809700 1 -- 172.21.15.44:0/600853221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac0106690 msgr2=0x7faac0196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.751+0000 7faac5809700 1 --2- 172.21.15.44:0/600853221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac0106690 0x7faac0196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.751+0000 7faac5809700 1 -- 172.21.15.44:0/600853221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab00095f0 con 0x7faac0100f90 2023-09-10T01:04:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.751+0000 7faac680b700 1 --2- 172.21.15.44:0/600853221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac0106690 0x7faac0196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:04:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.751+0000 7faac5809700 1 --2- 172.21.15.44:0/600853221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac0100f90 0x7faac019ca60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faab0009fd0 tx=0x7faab0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.751+0000 7faab77fe700 1 -- 172.21.15.44:0/600853221 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab001d070 con 0x7faac0100f90 2023-09-10T01:04:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.751+0000 7faab77fe700 1 -- 172.21.15.44:0/600853221 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faab00059a0 con 0x7faac0100f90 2023-09-10T01:04:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.751+0000 7faab77fe700 1 -- 172.21.15.44:0/600853221 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab0022cb0 con 0x7faac0100f90 2023-09-10T01:04:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.751+0000 7faac826e700 1 -- 172.21.15.44:0/600853221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac0196fa0 con 0x7faac0100f90 2023-09-10T01:04:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.751+0000 7faac826e700 1 -- 172.21.15.44:0/600853221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac01a3510 con 0x7faac0100f90 2023-09-10T01:04:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.754+0000 7faab77fe700 1 -- 172.21.15.44:0/600853221 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7faab0022470 con 0x7faac0100f90 2023-09-10T01:04:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.754+0000 7faab77fe700 1 --2- 172.21.15.44:0/600853221 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faaac0a07b0 0x7faaac0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.754+0000 7faab77fe700 1 -- 172.21.15.44:0/600853221 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9540+0+0 (secure 0 0 0) 0x7faab00cd450 con 0x7faac0100f90 2023-09-10T01:04:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.754+0000 7faac600a700 1 --2- 172.21.15.44:0/600853221 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faaac0a07b0 0x7faaac0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.755+0000 7faac600a700 1 --2- 172.21.15.44:0/600853221 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faaac0a07b0 0x7faaac0a2c70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faab8009c60 tx=0x7faab8009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.756+0000 7faad4434b80 1 Processor -- start 2023-09-10T01:04:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.756+0000 7faad4434b80 1 -- start start 2023-09-10T01:04:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.756+0000 7faad4434b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b321f1a0 0x55b8b321f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.756+0000 7faad4434b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b3219300 0x55b8b3219780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.756+0000 7faad4434b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219cc0 0x55b8b3225f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.756+0000 7faad4434b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8b31726d0 con 0x55b8b3219cc0 2023-09-10T01:04:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.756+0000 7faad4434b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8b316fff0 con 0x55b8b321f1a0 2023-09-10T01:04:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.756+0000 7faad4434b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8b316b8e0 con 0x55b8b3219300 2023-09-10T01:04:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.757+0000 7faac5809700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b3219300 0x55b8b3219780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.757+0000 7faac600a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b321f1a0 0x55b8b321f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.757+0000 7faac5809700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b3219300 0x55b8b3219780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34044/0 (socket says 172.21.15.44:34044) 2023-09-10T01:04:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.757+0000 7faac5809700 1 -- 172.21.15.44:0/4090096053 learned_addr learned my addr 172.21.15.44:0/4090096053 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.757+0000 7faac680b700 1 --2- 172.21.15.44:0/4090096053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219cc0 0x55b8b3225f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.757+0000 7faac5809700 1 -- 172.21.15.44:0/4090096053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b321f1a0 msgr2=0x55b8b321f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.757+0000 7faac5809700 1 --2- 172.21.15.44:0/4090096053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b321f1a0 0x55b8b321f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.757+0000 7faac5809700 1 -- 172.21.15.44:0/4090096053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219cc0 msgr2=0x55b8b3225f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.757+0000 7faac5809700 1 --2- 172.21.15.44:0/4090096053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219cc0 0x55b8b3225f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.757+0000 7faac5809700 1 -- 172.21.15.44:0/4090096053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8b2f97d80 con 0x55b8b3219300 2023-09-10T01:04:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.757+0000 7faac5809700 1 --2- 172.21.15.44:0/4090096053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b3219300 0x55b8b3219780 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faab0096a90 tx=0x7faab009a120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7b32c60499575c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.757+0000 7faab4d89700 1 -- 172.21.15.44:0/4090096053 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab009a590 con 0x55b8b3219300 2023-09-10T01:04:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.757+0000 7faab4d89700 1 -- 172.21.15.44:0/4090096053 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faab009e3c0 con 0x55b8b3219300 2023-09-10T01:04:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.757+0000 7faab4d89700 1 -- 172.21.15.44:0/4090096053 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab009aa50 con 0x55b8b3219300 2023-09-10T01:04:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.758+0000 7faad4434b80 1 -- 172.21.15.44:0/4090096053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b3219300 msgr2=0x55b8b3219780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.758+0000 7faad4434b80 1 --2- 172.21.15.44:0/4090096053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b3219300 0x55b8b3219780 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faab0096a90 tx=0x7faab009a120 comp rx=0 tx=0).stop 2023-09-10T01:04:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.758+0000 7faad4434b80 1 -- 172.21.15.44:0/4090096053 shutdown_connections 2023-09-10T01:04:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.758+0000 7faad4434b80 1 --2- 172.21.15.44:0/4090096053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219cc0 0x55b8b3225f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.758+0000 7faad4434b80 1 --2- 172.21.15.44:0/4090096053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b321f1a0 0x55b8b321f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.758+0000 7faad4434b80 1 --2- 172.21.15.44:0/4090096053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b3219300 0x55b8b3219780 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.758+0000 7faad4434b80 1 -- 172.21.15.44:0/4090096053 >> 172.21.15.44:0/4090096053 conn(0x55b8b318e030 msgr2=0x55b8b3214560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.759+0000 7faad4434b80 1 -- 172.21.15.44:0/4090096053 shutdown_connections 2023-09-10T01:04:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.759+0000 7faad4434b80 1 -- 172.21.15.44:0/4090096053 wait complete. 2023-09-10T01:04:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.759+0000 7faad4434b80 1 -- start start 2023-09-10T01:04:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.759+0000 7faad4434b80 1 Processor -- start 2023-09-10T01:04:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.759+0000 7faad4434b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b3219300 0x55b8b3228a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.759+0000 7faad4434b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219cc0 0x55b8b3228fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.759+0000 7faad4434b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b321f1a0 0x55b8b3222ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.759+0000 7faad4434b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8b3109b10 con 0x55b8b3219cc0 2023-09-10T01:04:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.759+0000 7faad4434b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8b3109810 con 0x55b8b3219300 2023-09-10T01:04:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.759+0000 7faad4434b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8b2fed8c0 con 0x55b8b321f1a0 2023-09-10T01:04:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.760+0000 7faad4434b80 10 client.0 _ll_register_callbacks cb 0x55b8b3053a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:04:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.760+0000 7faac5809700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219cc0 0x55b8b3228fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.760+0000 7faac5809700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219cc0 0x55b8b3228fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55400/0 (socket says 172.21.15.44:55400) 2023-09-10T01:04:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.760+0000 7faac5809700 1 -- 172.21.15.44:0/3487599681 learned_addr learned my addr 172.21.15.44:0/3487599681 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.760+0000 7faac600a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b3219300 0x55b8b3228a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.760+0000 7faac680b700 1 --2- 172.21.15.44:0/3487599681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b321f1a0 0x55b8b3222ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.760+0000 7faac600a700 1 -- 172.21.15.44:0/3487599681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b321f1a0 msgr2=0x55b8b3222ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.760+0000 7faac600a700 1 --2- 172.21.15.44:0/3487599681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b321f1a0 0x55b8b3222ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.760+0000 7faac600a700 1 -- 172.21.15.44:0/3487599681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219cc0 msgr2=0x55b8b3228fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.760+0000 7faac600a700 1 --2- 172.21.15.44:0/3487599681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219cc0 0x55b8b3228fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.760+0000 7faac600a700 1 -- 172.21.15.44:0/3487599681 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac01a3510 con 0x55b8b3219300 2023-09-10T01:04:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.760+0000 7faac600a700 1 --2- 172.21.15.44:0/3487599681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b3219300 0x55b8b3228a90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faab8011ab0 tx=0x7faab80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9f575c79c56fd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.760+0000 7faaa9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:04:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.760+0000 7faaa9ffb700 1 -- 172.21.15.44:0/3487599681 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8022070 con 0x55b8b3219300 2023-09-10T01:04:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.761+0000 7faad4434b80 1 -- 172.21.15.44:0/3487599681 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8b30c3490 con 0x55b8b3219300 2023-09-10T01:04:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.761+0000 7faaa9ffb700 1 -- 172.21.15.44:0/3487599681 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 9540+0+0 (secure 0 0 0) 0x7faab801d620 con 0x55b8b3219300 2023-09-10T01:04:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.761+0000 7faaa9ffb700 1 -- 172.21.15.44:0/3487599681 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7faab801d7e0 con 0x55b8b3219300 2023-09-10T01:04:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.761+0000 7faad4434b80 10 client.28256 fetch_fsmap learned FSMap version 96 2023-09-10T01:04:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.761+0000 7faad4434b80 1 -- 172.21.15.44:0/3487599681 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) v3 -- 0x55b8b2f80c00 con 0x55b8b3219300 2023-09-10T01:04:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.761+0000 7faaa9ffb700 1 -- 172.21.15.44:0/3487599681 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b8b30c3490 con 0x55b8b3219300 2023-09-10T01:04:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.761+0000 7faaa9ffb700 1 -- 172.21.15.44:0/3487599681 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 96) v1 ==== 7938+0+0 (secure 0 0 0) 0x7faab8027a00 con 0x55b8b3219300 2023-09-10T01:04:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.761+0000 7faad4434b80 10 client.28256 fetch_fsmap finished waiting for FSMap version 96 2023-09-10T01:04:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.761+0000 7faad4434b80 10 client.28256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:04:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.761+0000 7faad4434b80 20 client.28256 populate_metadata read hostname 'smithi044' 2023-09-10T01:04:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.761+0000 7faad4434b80 1 --2- 172.21.15.44:0/3487599681 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8b3237480 0x55b8b32398a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.761+0000 7faad4434b80 4 client.28256 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:04:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.761+0000 7faad4434b80 1 -- 172.21.15.44:0/3487599681 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8b3180840 con 0x55b8b3237480 2023-09-10T01:04:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.761+0000 7faac5809700 1 --2- 172.21.15.44:0/3487599681 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8b3237480 0x55b8b32398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.762+0000 7faaa9ffb700 10 client.28256 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:04:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.762+0000 7faac5809700 1 --2- 172.21.15.44:0/3487599681 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8b3237480 0x55b8b32398a0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43e269297b43eeba server_cookie=ebc9001a832b5b27 in_seq=0 out_seq=0 2023-09-10T01:04:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.762+0000 7faaa9ffb700 1 -- 172.21.15.44:0/3487599681 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b8b3180840 con 0x55b8b3237480 2023-09-10T01:04:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.762+0000 7faaa9ffb700 10 client.28256 handle_command_reply: tid=0 2023-09-10T01:04:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.762+0000 7faad4434b80 1 client.28256 shutdown 2023-09-10T01:04:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.763+0000 7faad4434b80 1 -- 172.21.15.44:0/3487599681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b3219300 msgr2=0x55b8b3228a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.763+0000 7faad4434b80 1 --2- 172.21.15.44:0/3487599681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b3219300 0x55b8b3228a90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faab8011ab0 tx=0x7faab80047c0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.763+0000 7faad4434b80 1 -- 172.21.15.44:0/3487599681 shutdown_connections 2023-09-10T01:04:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.763+0000 7faad4434b80 1 --2- 172.21.15.44:0/3487599681 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8b3237480 0x55b8b32398a0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.763+0000 7faad4434b80 1 --2- 172.21.15.44:0/3487599681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b321f1a0 0x55b8b3222ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.763+0000 7faad4434b80 1 --2- 172.21.15.44:0/3487599681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b3219300 0x55b8b3228a90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.763+0000 7faad4434b80 1 --2- 172.21.15.44:0/3487599681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219cc0 0x55b8b3228fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.763+0000 7faaa9ffb700 0 client.28256 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:04:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.763+0000 7faad4434b80 1 -- 172.21.15.44:0/3487599681 >> 172.21.15.44:0/3487599681 conn(0x55b8b318e030 msgr2=0x55b8b32137a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.763+0000 7faad4434b80 1 -- 172.21.15.44:0/3487599681 shutdown_connections 2023-09-10T01:04:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.763+0000 7faad4434b80 1 -- 172.21.15.44:0/3487599681 wait complete. 2023-09-10T01:04:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.763+0000 7faad4434b80 20 client.28256 trim_cache size 0 max 16384 2023-09-10T01:04:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.780+0000 7faad4434b80 1 Processor -- start 2023-09-10T01:04:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.780+0000 7faad4434b80 1 -- start start 2023-09-10T01:04:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.780+0000 7faad4434b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219300 0x55b8b321e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.780+0000 7faad4434b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b3219cc0 0x55b8b3220590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.780+0000 7faad4434b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b321f1a0 0x55b8b3228620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.780+0000 7faad4434b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8b3210fc0 con 0x55b8b3219300 2023-09-10T01:04:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.780+0000 7faad4434b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8b32349e0 con 0x55b8b321f1a0 2023-09-10T01:04:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.780+0000 7faad4434b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8b321a580 con 0x55b8b3219cc0 2023-09-10T01:04:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.780+0000 7faac600a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219300 0x55b8b321e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.780+0000 7faac5809700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b3219cc0 0x55b8b3220590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.780+0000 7faac600a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219300 0x55b8b321e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55406/0 (socket says 172.21.15.44:55406) 2023-09-10T01:04:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.780+0000 7faac680b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b321f1a0 0x55b8b3228620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.780+0000 7faac600a700 1 -- 172.21.15.44:0/3256792057 learned_addr learned my addr 172.21.15.44:0/3256792057 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.780+0000 7faac5809700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b3219cc0 0x55b8b3220590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34058/0 (socket says 172.21.15.44:34058) 2023-09-10T01:04:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.781+0000 7faac680b700 1 -- 172.21.15.44:0/3256792057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b3219cc0 msgr2=0x55b8b3220590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.781+0000 7faac680b700 1 --2- 172.21.15.44:0/3256792057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b3219cc0 0x55b8b3220590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.781+0000 7faac680b700 1 -- 172.21.15.44:0/3256792057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219300 msgr2=0x55b8b321e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.781+0000 7faac680b700 1 --2- 172.21.15.44:0/3256792057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219300 0x55b8b321e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.781+0000 7faac680b700 1 -- 172.21.15.44:0/3256792057 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8b2fbe1f0 con 0x55b8b321f1a0 2023-09-10T01:04:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.781+0000 7faac680b700 1 --2- 172.21.15.44:0/3256792057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b321f1a0 0x55b8b3228620 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faac019d220 tx=0x7faabc00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d8673f71635ee74 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.781+0000 7faab558a700 1 -- 172.21.15.44:0/3256792057 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc003c00 con 0x55b8b321f1a0 2023-09-10T01:04:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.781+0000 7faab558a700 1 -- 172.21.15.44:0/3256792057 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faabc01b070 con 0x55b8b321f1a0 2023-09-10T01:04:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.781+0000 7faab558a700 1 -- 172.21.15.44:0/3256792057 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc00e810 con 0x55b8b321f1a0 2023-09-10T01:04:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.782+0000 7faad4434b80 1 -- 172.21.15.44:0/3256792057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b321f1a0 msgr2=0x55b8b3228620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.782+0000 7faad4434b80 1 --2- 172.21.15.44:0/3256792057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b321f1a0 0x55b8b3228620 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faac019d220 tx=0x7faabc00b040 comp rx=0 tx=0).stop 2023-09-10T01:04:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.782+0000 7faad4434b80 1 -- 172.21.15.44:0/3256792057 shutdown_connections 2023-09-10T01:04:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.782+0000 7faad4434b80 1 --2- 172.21.15.44:0/3256792057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b321f1a0 0x55b8b3228620 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.782+0000 7faad4434b80 1 --2- 172.21.15.44:0/3256792057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219300 0x55b8b321e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.782+0000 7faad4434b80 1 --2- 172.21.15.44:0/3256792057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b3219cc0 0x55b8b3220590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.782+0000 7faad4434b80 1 -- 172.21.15.44:0/3256792057 >> 172.21.15.44:0/3256792057 conn(0x55b8b318e030 msgr2=0x55b8b3218b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.782+0000 7faad4434b80 1 -- 172.21.15.44:0/3256792057 shutdown_connections 2023-09-10T01:04:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.782+0000 7faad4434b80 1 -- 172.21.15.44:0/3256792057 wait complete. 2023-09-10T01:04:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.783+0000 7faad4434b80 1 -- start start 2023-09-10T01:04:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.783+0000 7faad4434b80 1 Processor -- start 2023-09-10T01:04:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.783+0000 7faad4434b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219300 0x55b8b327f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.783+0000 7faad4434b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b3219cc0 0x55b8b327f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.783+0000 7faad4434b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b321f1a0 0x55b8b328a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.783+0000 7faad4434b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8b32800c0 con 0x55b8b3219300 2023-09-10T01:04:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.783+0000 7faad4434b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8b328a800 con 0x55b8b3219cc0 2023-09-10T01:04:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.783+0000 7faad4434b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8b328a9c0 con 0x55b8b321f1a0 2023-09-10T01:04:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.783+0000 7faad4434b80 10 client.0 _ll_register_callbacks cb 0x55b8b3053a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:04:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.783+0000 7faac600a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219300 0x55b8b327f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.783+0000 7faac600a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219300 0x55b8b327f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55412/0 (socket says 172.21.15.44:55412) 2023-09-10T01:04:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.783+0000 7faac600a700 1 -- 172.21.15.44:0/515506626 learned_addr learned my addr 172.21.15.44:0/515506626 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.783+0000 7faac5809700 1 --2- 172.21.15.44:0/515506626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b3219cc0 0x55b8b327f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.783+0000 7faac680b700 1 --2- 172.21.15.44:0/515506626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b321f1a0 0x55b8b328a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.783+0000 7faac600a700 1 -- 172.21.15.44:0/515506626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b321f1a0 msgr2=0x55b8b328a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.784+0000 7faac600a700 1 --2- 172.21.15.44:0/515506626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b321f1a0 0x55b8b328a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.784+0000 7faac600a700 1 -- 172.21.15.44:0/515506626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b3219cc0 msgr2=0x55b8b327f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.784+0000 7faac600a700 1 --2- 172.21.15.44:0/515506626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b3219cc0 0x55b8b327f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.784+0000 7faac600a700 1 -- 172.21.15.44:0/515506626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faabc009390 con 0x55b8b3219300 2023-09-10T01:04:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.784+0000 7faac680b700 1 --2- 172.21.15.44:0/515506626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b321f1a0 0x55b8b328a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:04:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.784+0000 7faac5809700 1 --2- 172.21.15.44:0/515506626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b3219cc0 0x55b8b327f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:04:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.784+0000 7faac600a700 1 --2- 172.21.15.44:0/515506626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219300 0x55b8b327f430 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7faab8027e80 tx=0x7faab8027a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199f648078812a94 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.784+0000 7faaa9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:04:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.784+0000 7faaa9ffb700 1 -- 172.21.15.44:0/515506626 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8022040 con 0x55b8b3219300 2023-09-10T01:04:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.784+0000 7faad4434b80 1 -- 172.21.15.44:0/515506626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faab009e3c0 con 0x55b8b3219300 2023-09-10T01:04:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.784+0000 7faaa9ffb700 1 -- 172.21.15.44:0/515506626 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 9540+0+0 (secure 0 0 0) 0x7faab801d830 con 0x55b8b3219300 2023-09-10T01:04:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.784+0000 7faaa9ffb700 1 -- 172.21.15.44:0/515506626 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7faab801d9f0 con 0x55b8b3219300 2023-09-10T01:04:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.784+0000 7faad4434b80 10 client.18606 fetch_fsmap learned FSMap version 96 2023-09-10T01:04:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.784+0000 7faad4434b80 1 -- 172.21.15.44:0/515506626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) v3 -- 0x55b8b2f96250 con 0x55b8b3219300 2023-09-10T01:04:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.784+0000 7faaa9ffb700 1 -- 172.21.15.44:0/515506626 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faab009e3c0 con 0x55b8b3219300 2023-09-10T01:04:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.785+0000 7faaa9ffb700 1 -- 172.21.15.44:0/515506626 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 96) v1 ==== 7938+0+0 (secure 0 0 0) 0x7faab8037a10 con 0x55b8b3219300 2023-09-10T01:04:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.785+0000 7faad4434b80 10 client.18606 fetch_fsmap finished waiting for FSMap version 96 2023-09-10T01:04:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.785+0000 7faad4434b80 10 client.18606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:04:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.785+0000 7faad4434b80 20 client.18606 populate_metadata read hostname 'smithi044' 2023-09-10T01:04:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.785+0000 7faad4434b80 1 --2- 172.21.15.44:0/515506626 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8b328c230 0x55b8b328e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.785+0000 7faad4434b80 4 client.18606 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:04:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.785+0000 7faad4434b80 1 -- 172.21.15.44:0/515506626 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b8b31648c0 con 0x55b8b328c230 2023-09-10T01:04:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.785+0000 7faac5809700 1 --2- 172.21.15.44:0/515506626 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8b328c230 0x55b8b328e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.785+0000 7faaa9ffb700 10 client.18606 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:04:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.785+0000 7faac5809700 1 --2- 172.21.15.44:0/515506626 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8b328c230 0x55b8b328e650 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4f81a68c53f2ac4 server_cookie=aee74d476b6aadc4 in_seq=0 out_seq=0 2023-09-10T01:04:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.799+0000 7faaa9ffb700 1 -- 172.21.15.44:0/515506626 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b8b31648c0 con 0x55b8b328c230 2023-09-10T01:04:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.799+0000 7faaa9ffb700 10 client.18606 handle_command_reply: tid=0 2023-09-10T01:04:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.799+0000 7faad4434b80 1 client.18606 shutdown 2023-09-10T01:04:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.800+0000 7faad4434b80 1 -- 172.21.15.44:0/515506626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219300 msgr2=0x55b8b327f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.800+0000 7faad4434b80 1 --2- 172.21.15.44:0/515506626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219300 0x55b8b327f430 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7faab8027e80 tx=0x7faab8027a50 comp rx=0 tx=0).stop 2023-09-10T01:04:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.800+0000 7faad4434b80 1 -- 172.21.15.44:0/515506626 shutdown_connections 2023-09-10T01:04:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.800+0000 7faad4434b80 1 --2- 172.21.15.44:0/515506626 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8b328c230 0x55b8b328e650 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.800+0000 7faad4434b80 1 --2- 172.21.15.44:0/515506626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8b321f1a0 0x55b8b328a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.800+0000 7faad4434b80 1 --2- 172.21.15.44:0/515506626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8b3219300 0x55b8b327f430 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.800+0000 7faad4434b80 1 --2- 172.21.15.44:0/515506626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8b3219cc0 0x55b8b327f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.800+0000 7faad4434b80 1 -- 172.21.15.44:0/515506626 >> 172.21.15.44:0/515506626 conn(0x55b8b318e030 msgr2=0x55b8b321f580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.800+0000 7faaa9ffb700 0 client.18606 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:04:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.800+0000 7faad4434b80 1 -- 172.21.15.44:0/515506626 shutdown_connections 2023-09-10T01:04:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.800+0000 7faad4434b80 1 -- 172.21.15.44:0/515506626 wait complete. 2023-09-10T01:04:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.800+0000 7faad4434b80 20 client.18606 trim_cache size 0 max 16384 2023-09-10T01:04:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:04:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:04:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:04:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:04:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.800+0000 7faac826e700 1 -- 172.21.15.44:0/600853221 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faaac0a07b0 msgr2=0x7faaac0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.800+0000 7faac826e700 1 --2- 172.21.15.44:0/600853221 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faaac0a07b0 0x7faaac0a2c70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faab8009c60 tx=0x7faab8009260 comp rx=0 tx=0).stop 2023-09-10T01:04:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.800+0000 7faac826e700 1 -- 172.21.15.44:0/600853221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac0100f90 msgr2=0x7faac019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.800+0000 7faac826e700 1 --2- 172.21.15.44:0/600853221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac0100f90 0x7faac019ca60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faab0009fd0 tx=0x7faab0009580 comp rx=0 tx=0).stop 2023-09-10T01:04:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.801+0000 7faac826e700 1 -- 172.21.15.44:0/600853221 shutdown_connections 2023-09-10T01:04:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.801+0000 7faac826e700 1 --2- 172.21.15.44:0/600853221 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faaac0a07b0 0x7faaac0a2c70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.801+0000 7faac826e700 1 --2- 172.21.15.44:0/600853221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac0106690 0x7faac0196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.801+0000 7faac826e700 1 --2- 172.21.15.44:0/600853221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faac01005d0 0x7faac019c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.801+0000 7faac826e700 1 --2- 172.21.15.44:0/600853221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac0100f90 0x7faac019ca60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.801+0000 7faac826e700 1 -- 172.21.15.44:0/600853221 >> 172.21.15.44:0/600853221 conn(0x7faac00fc0c0 msgr2=0x7faac00fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.801+0000 7faac826e700 1 -- 172.21.15.44:0/600853221 shutdown_connections 2023-09-10T01:04:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:22.801+0000 7faac826e700 1 -- 172.21.15.44:0/600853221 wait complete. 2023-09-10T01:04:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:22 smithi099 ceph-mon[102692]: pgmap v753: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 181 op/s 2023-09-10T01:04:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:22 smithi131 ceph-mon[103713]: pgmap v753: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 181 op/s 2023-09-10T01:04:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:22 smithi044 ceph-mon[98676]: pgmap v753: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 181 op/s 2023-09-10T01:04:24.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:23 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:04:24.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:23 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:04:24.426 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:23 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:04:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:24 smithi099 ceph-mon[102692]: pgmap v754: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 181 op/s 2023-09-10T01:04:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:24 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:04:25.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:24 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:04:25.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:24 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:04:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:24 smithi131 ceph-mon[103713]: pgmap v754: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 181 op/s 2023-09-10T01:04:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:24 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:04:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:24 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:04:25.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:24 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:04:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:24 smithi044 ceph-mon[98676]: pgmap v754: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 181 op/s 2023-09-10T01:04:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:24 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:04:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:24 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:04:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:24 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:04:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:26 smithi099 ceph-mon[102692]: pgmap v755: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2023-09-10T01:04:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:26 smithi131 ceph-mon[103713]: pgmap v755: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2023-09-10T01:04:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:26 smithi044 ceph-mon[98676]: pgmap v755: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2023-09-10T01:04:29.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:28 smithi099 ceph-mon[102692]: pgmap v756: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-09-10T01:04:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:28 smithi131 ceph-mon[103713]: pgmap v756: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-09-10T01:04:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:28 smithi044 ceph-mon[98676]: pgmap v756: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-09-10T01:04:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:30 smithi099 ceph-mon[102692]: pgmap v757: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2023-09-10T01:04:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:30 smithi131 ceph-mon[103713]: pgmap v757: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2023-09-10T01:04:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:30 smithi044 ceph-mon[98676]: pgmap v757: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2023-09-10T01:04:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:32 smithi099 ceph-mon[102692]: pgmap v758: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-09-10T01:04:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:32 smithi131 ceph-mon[103713]: pgmap v758: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-09-10T01:04:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:32 smithi044 ceph-mon[98676]: pgmap v758: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-09-10T01:04:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:34 smithi099 ceph-mon[102692]: pgmap v759: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 157 op/s 2023-09-10T01:04:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:34 smithi131 ceph-mon[103713]: pgmap v759: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 157 op/s 2023-09-10T01:04:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:34 smithi044 ceph-mon[98676]: pgmap v759: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 157 op/s 2023-09-10T01:04:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.926+0000 7f8d46421700 1 -- 172.21.15.44:0/2669333738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d401055b0 msgr2=0x7f8d40105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.926+0000 7f8d46421700 1 --2- 172.21.15.44:0/2669333738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d401055b0 0x7f8d40105990 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f8d300099c0 tx=0x7f8d30009cd0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.926+0000 7f8d46421700 1 -- 172.21.15.44:0/2669333738 shutdown_connections 2023-09-10T01:04:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.926+0000 7f8d46421700 1 --2- 172.21.15.44:0/2669333738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40100120 0x7f8d4010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.926+0000 7f8d46421700 1 --2- 172.21.15.44:0/2669333738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d401055b0 0x7f8d40105990 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.926+0000 7f8d46421700 1 --2- 172.21.15.44:0/2669333738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d400ff760 0x7f8d400ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.926+0000 7f8d46421700 1 -- 172.21.15.44:0/2669333738 >> 172.21.15.44:0/2669333738 conn(0x7f8d40074f30 msgr2=0x7f8d40075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.926+0000 7f8d46421700 1 -- 172.21.15.44:0/2669333738 shutdown_connections 2023-09-10T01:04:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.926+0000 7f8d46421700 1 -- 172.21.15.44:0/2669333738 wait complete. 2023-09-10T01:04:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.927+0000 7f8d46421700 1 Processor -- start 2023-09-10T01:04:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.927+0000 7f8d46421700 1 -- start start 2023-09-10T01:04:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.927+0000 7f8d46421700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d400ff760 0x7f8d4019b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.927+0000 7f8d46421700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d40100120 0x7f8d4019ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.927+0000 7f8d46421700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d401055b0 0x7f8d401953b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.927+0000 7f8d46421700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40195a20 con 0x7f8d400ff760 2023-09-10T01:04:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.927+0000 7f8d46421700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40195b90 con 0x7f8d401055b0 2023-09-10T01:04:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.927+0000 7f8d46421700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40195d50 con 0x7f8d40100120 2023-09-10T01:04:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.927+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d400ff760 0x7f8d4019b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.927+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d400ff760 0x7f8d4019b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55424/0 (socket says 172.21.15.44:55424) 2023-09-10T01:04:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.927+0000 7f8d3ffff700 1 -- 172.21.15.44:0/3417739965 learned_addr learned my addr 172.21.15.44:0/3417739965 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.927+0000 7f8d3f7fe700 1 --2- 172.21.15.44:0/3417739965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d40100120 0x7f8d4019ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.927+0000 7f8d449be700 1 --2- 172.21.15.44:0/3417739965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d401055b0 0x7f8d401953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.928+0000 7f8d3ffff700 1 -- 172.21.15.44:0/3417739965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d40100120 msgr2=0x7f8d4019ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.928+0000 7f8d3ffff700 1 --2- 172.21.15.44:0/3417739965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d40100120 0x7f8d4019ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.928+0000 7f8d3ffff700 1 -- 172.21.15.44:0/3417739965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d401055b0 msgr2=0x7f8d401953b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.928+0000 7f8d3ffff700 1 --2- 172.21.15.44:0/3417739965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d401055b0 0x7f8d401953b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.928+0000 7f8d3ffff700 1 -- 172.21.15.44:0/3417739965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d300095f0 con 0x7f8d400ff760 2023-09-10T01:04:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.928+0000 7f8d449be700 1 --2- 172.21.15.44:0/3417739965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d401055b0 0x7f8d401953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:04:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.928+0000 7f8d3ffff700 1 --2- 172.21.15.44:0/3417739965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d400ff760 0x7f8d4019b550 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8d30006010 tx=0x7f8d3000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.928+0000 7f8d3d7fa700 1 -- 172.21.15.44:0/3417739965 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3001d070 con 0x7f8d400ff760 2023-09-10T01:04:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.928+0000 7f8d46421700 1 -- 172.21.15.44:0/3417739965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d40195fd0 con 0x7f8d400ff760 2023-09-10T01:04:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.928+0000 7f8d3d7fa700 1 -- 172.21.15.44:0/3417739965 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d30005030 con 0x7f8d400ff760 2023-09-10T01:04:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.928+0000 7f8d3d7fa700 1 -- 172.21.15.44:0/3417739965 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d300049c0 con 0x7f8d400ff760 2023-09-10T01:04:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.928+0000 7f8d46421700 1 -- 172.21.15.44:0/3417739965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d400679f0 con 0x7f8d400ff760 2023-09-10T01:04:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.931+0000 7f8d3d7fa700 1 -- 172.21.15.44:0/3417739965 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f8d300051a0 con 0x7f8d400ff760 2023-09-10T01:04:35.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.932+0000 7f8d3d7fa700 1 --2- 172.21.15.44:0/3417739965 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8d280a07b0 0x7f8d280a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.932+0000 7f8d3d7fa700 1 -- 172.21.15.44:0/3417739965 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8d300cd040 con 0x7f8d400ff760 2023-09-10T01:04:35.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.932+0000 7f8d3f7fe700 1 --2- 172.21.15.44:0/3417739965 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8d280a07b0 0x7f8d280a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.932+0000 7f8d3f7fe700 1 --2- 172.21.15.44:0/3417739965 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8d280a07b0 0x7f8d280a2c70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8d40104ad0 tx=0x7f8d34009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:35.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.932+0000 7f8d525e7b80 1 Processor -- start 2023-09-10T01:04:35.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d525e7b80 1 -- start start 2023-09-10T01:04:35.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d525e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e9d1a0 0x5555b7e9d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d525e7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97300 0x5555b7e97780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d525e7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97cc0 0x5555b7ea3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d525e7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b7df06d0 con 0x5555b7e97cc0 2023-09-10T01:04:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d525e7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b7dedff0 con 0x5555b7e97300 2023-09-10T01:04:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d525e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b7de98e0 con 0x5555b7e9d1a0 2023-09-10T01:04:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d3f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97300 0x5555b7e97780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e9d1a0 0x5555b7e9d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d3f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97300 0x5555b7e97780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57174/0 (socket says 172.21.15.44:57174) 2023-09-10T01:04:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e9d1a0 0x5555b7e9d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34082/0 (socket says 172.21.15.44:34082) 2023-09-10T01:04:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d3f7fe700 1 -- 172.21.15.44:0/1966396928 learned_addr learned my addr 172.21.15.44:0/1966396928 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d449be700 1 --2- 172.21.15.44:0/1966396928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97cc0 0x5555b7ea3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d3ffff700 1 -- 172.21.15.44:0/1966396928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97300 msgr2=0x5555b7e97780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d3ffff700 1 --2- 172.21.15.44:0/1966396928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97300 0x5555b7e97780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d3ffff700 1 -- 172.21.15.44:0/1966396928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97cc0 msgr2=0x5555b7ea3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d3ffff700 1 --2- 172.21.15.44:0/1966396928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97cc0 0x5555b7ea3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d3ffff700 1 -- 172.21.15.44:0/1966396928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555b7c15d80 con 0x5555b7e9d1a0 2023-09-10T01:04:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d3f7fe700 1 --2- 172.21.15.44:0/1966396928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97300 0x5555b7e97780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:04:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.933+0000 7f8d449be700 1 --2- 172.21.15.44:0/1966396928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97cc0 0x5555b7ea3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:04:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.934+0000 7f8d3ffff700 1 --2- 172.21.15.44:0/1966396928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e9d1a0 0x5555b7e9d580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8d300ca390 tx=0x7f8d30099d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cbc17c6ce8a761f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.934+0000 7f8d267fc700 1 -- 172.21.15.44:0/1966396928 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d300a2280 con 0x5555b7e9d1a0 2023-09-10T01:04:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.934+0000 7f8d267fc700 1 -- 172.21.15.44:0/1966396928 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d300969b0 con 0x5555b7e9d1a0 2023-09-10T01:04:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.934+0000 7f8d267fc700 1 -- 172.21.15.44:0/1966396928 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30099790 con 0x5555b7e9d1a0 2023-09-10T01:04:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.935+0000 7f8d525e7b80 1 -- 172.21.15.44:0/1966396928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e9d1a0 msgr2=0x5555b7e9d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.935+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/1966396928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e9d1a0 0x5555b7e9d580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8d300ca390 tx=0x7f8d30099d10 comp rx=0 tx=0).stop 2023-09-10T01:04:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.935+0000 7f8d525e7b80 1 -- 172.21.15.44:0/1966396928 shutdown_connections 2023-09-10T01:04:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.935+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/1966396928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97cc0 0x5555b7ea3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.935+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/1966396928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e9d1a0 0x5555b7e9d580 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.935+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/1966396928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97300 0x5555b7e97780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.935+0000 7f8d525e7b80 1 -- 172.21.15.44:0/1966396928 >> 172.21.15.44:0/1966396928 conn(0x5555b7e0c030 msgr2=0x5555b7e92560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.935+0000 7f8d525e7b80 1 -- 172.21.15.44:0/1966396928 shutdown_connections 2023-09-10T01:04:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.935+0000 7f8d525e7b80 1 -- 172.21.15.44:0/1966396928 wait complete. 2023-09-10T01:04:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.935+0000 7f8d525e7b80 1 -- start start 2023-09-10T01:04:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d525e7b80 1 Processor -- start 2023-09-10T01:04:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d525e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e97300 0x5555b7eaf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d525e7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97cc0 0x5555b7eaf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d525e7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e9d1a0 0x5555b7ea9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d525e7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b7d87b10 con 0x5555b7e97cc0 2023-09-10T01:04:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d525e7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b7d87810 con 0x5555b7e9d1a0 2023-09-10T01:04:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d525e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b7c6b8c0 con 0x5555b7e97300 2023-09-10T01:04:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d525e7b80 10 client.0 _ll_register_callbacks cb 0x5555b7cd1a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:04:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e97300 0x5555b7eaf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d3f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97cc0 0x5555b7eaf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e97300 0x5555b7eaf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34088/0 (socket says 172.21.15.44:34088) 2023-09-10T01:04:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d3f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97cc0 0x5555b7eaf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55440/0 (socket says 172.21.15.44:55440) 2023-09-10T01:04:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d3ffff700 1 -- 172.21.15.44:0/3841226192 learned_addr learned my addr 172.21.15.44:0/3841226192 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d449be700 1 --2- 172.21.15.44:0/3841226192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e9d1a0 0x5555b7ea9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d3ffff700 1 -- 172.21.15.44:0/3841226192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e9d1a0 msgr2=0x5555b7ea9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d3ffff700 1 --2- 172.21.15.44:0/3841226192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e9d1a0 0x5555b7ea9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d3ffff700 1 -- 172.21.15.44:0/3841226192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97cc0 msgr2=0x5555b7eaf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d3ffff700 1 --2- 172.21.15.44:0/3841226192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97cc0 0x5555b7eaf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d3ffff700 1 -- 172.21.15.44:0/3841226192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d34017070 con 0x5555b7e97300 2023-09-10T01:04:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.936+0000 7f8d3f7fe700 1 --2- 172.21.15.44:0/3841226192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97cc0 0x5555b7eaf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:04:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.937+0000 7f8d3ffff700 1 --2- 172.21.15.44:0/3841226192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e97300 0x5555b7eaf400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8d300ca440 tx=0x7f8d300999c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1d6f84849335002 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.937+0000 7f8d1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:04:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.937+0000 7f8d1f7fe700 1 -- 172.21.15.44:0/3841226192 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d300a2280 con 0x5555b7e97300 2023-09-10T01:04:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.937+0000 7f8d525e7b80 1 -- 172.21.15.44:0/3841226192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555b7d41490 con 0x5555b7e97300 2023-09-10T01:04:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.937+0000 7f8d1f7fe700 1 -- 172.21.15.44:0/3841226192 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8d3009a710 con 0x5555b7e97300 2023-09-10T01:04:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.937+0000 7f8d1f7fe700 1 -- 172.21.15.44:0/3841226192 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d300b8100 con 0x5555b7e97300 2023-09-10T01:04:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.937+0000 7f8d525e7b80 10 client.28210 fetch_fsmap learned FSMap version 96 2023-09-10T01:04:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.937+0000 7f8d525e7b80 1 -- 172.21.15.44:0/3841226192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) v3 -- 0x7f8d38009050 con 0x5555b7e97300 2023-09-10T01:04:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.937+0000 7f8d1f7fe700 1 -- 172.21.15.44:0/3841226192 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d30066020 con 0x5555b7e97300 2023-09-10T01:04:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.937+0000 7f8d1f7fe700 1 -- 172.21.15.44:0/3841226192 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 96) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f8d3009c030 con 0x5555b7e97300 2023-09-10T01:04:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.938+0000 7f8d525e7b80 10 client.28210 fetch_fsmap finished waiting for FSMap version 96 2023-09-10T01:04:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.938+0000 7f8d525e7b80 10 client.28210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:04:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.938+0000 7f8d525e7b80 20 client.28210 populate_metadata read hostname 'smithi044' 2023-09-10T01:04:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.938+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/3841226192 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5555b7eb9520 0x5555b7ebb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.938+0000 7f8d525e7b80 4 client.28210 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:04:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.938+0000 7f8d525e7b80 1 -- 172.21.15.44:0/3841226192 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555b7dfe840 con 0x5555b7eb9520 2023-09-10T01:04:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.938+0000 7f8d3f7fe700 1 --2- 172.21.15.44:0/3841226192 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5555b7eb9520 0x5555b7ebb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.938+0000 7f8d1f7fe700 10 client.28210 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:04:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.938+0000 7f8d3f7fe700 1 --2- 172.21.15.44:0/3841226192 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5555b7eb9520 0x5555b7ebb940 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=914893afa28fdfcc server_cookie=a058892ea7cd249b in_seq=0 out_seq=0 2023-09-10T01:04:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.939+0000 7f8d1f7fe700 1 -- 172.21.15.44:0/3841226192 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5555b7dfe840 con 0x5555b7eb9520 2023-09-10T01:04:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.939+0000 7f8d1f7fe700 10 client.28210 handle_command_reply: tid=0 2023-09-10T01:04:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.939+0000 7f8d525e7b80 1 client.28210 shutdown 2023-09-10T01:04:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.939+0000 7f8d525e7b80 1 -- 172.21.15.44:0/3841226192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e97300 msgr2=0x5555b7eaf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.939+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/3841226192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e97300 0x5555b7eaf400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8d300ca440 tx=0x7f8d300999c0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.939+0000 7f8d525e7b80 1 -- 172.21.15.44:0/3841226192 shutdown_connections 2023-09-10T01:04:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.939+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/3841226192 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5555b7eb9520 0x5555b7ebb940 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.939+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/3841226192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e9d1a0 0x5555b7ea9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.939+0000 7f8d1f7fe700 0 client.28210 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:04:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.939+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/3841226192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e97300 0x5555b7eaf400 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.939+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/3841226192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97cc0 0x5555b7eaf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.939+0000 7f8d525e7b80 1 -- 172.21.15.44:0/3841226192 >> 172.21.15.44:0/3841226192 conn(0x5555b7e0c030 msgr2=0x5555b7e917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.939+0000 7f8d525e7b80 1 -- 172.21.15.44:0/3841226192 shutdown_connections 2023-09-10T01:04:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.939+0000 7f8d525e7b80 1 -- 172.21.15.44:0/3841226192 wait complete. 2023-09-10T01:04:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.939+0000 7f8d525e7b80 20 client.28210 trim_cache size 0 max 16384 2023-09-10T01:04:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.954+0000 7f8d525e7b80 1 Processor -- start 2023-09-10T01:04:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.954+0000 7f8d525e7b80 1 -- start start 2023-09-10T01:04:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.954+0000 7f8d525e7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97300 0x5555b7e9be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.954+0000 7f8d525e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e97cc0 0x5555b7e9c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.954+0000 7f8d525e7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e9d1a0 0x5555b7ea8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.954+0000 7f8d525e7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b7e93210 con 0x5555b7e9d1a0 2023-09-10T01:04:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.954+0000 7f8d525e7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b7ea7bb0 con 0x5555b7e97300 2023-09-10T01:04:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.954+0000 7f8d525e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b7e9ba20 con 0x5555b7e97cc0 2023-09-10T01:04:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97300 0x5555b7e9be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d449be700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e9d1a0 0x5555b7ea8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d3f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e97cc0 0x5555b7e9c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97300 0x5555b7e9be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57186/0 (socket says 172.21.15.44:57186) 2023-09-10T01:04:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d3f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e97cc0 0x5555b7e9c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34098/0 (socket says 172.21.15.44:34098) 2023-09-10T01:04:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d449be700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e9d1a0 0x5555b7ea8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55450/0 (socket says 172.21.15.44:55450) 2023-09-10T01:04:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d3ffff700 1 -- 172.21.15.44:0/350779866 learned_addr learned my addr 172.21.15.44:0/350779866 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d3ffff700 1 -- 172.21.15.44:0/350779866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e97cc0 msgr2=0x5555b7e9c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d3ffff700 1 --2- 172.21.15.44:0/350779866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e97cc0 0x5555b7e9c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d3ffff700 1 -- 172.21.15.44:0/350779866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e9d1a0 msgr2=0x5555b7ea8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d3ffff700 1 --2- 172.21.15.44:0/350779866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e9d1a0 0x5555b7ea8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d3ffff700 1 -- 172.21.15.44:0/350779866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d400679f0 con 0x5555b7e97300 2023-09-10T01:04:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d449be700 1 --2- 172.21.15.44:0/350779866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e9d1a0 0x5555b7ea8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:04:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d3ffff700 1 --2- 172.21.15.44:0/350779866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97300 0x5555b7e9be30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000b580 tx=0x7f8d30009d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f722dc7e470f27f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d26ffd700 1 -- 172.21.15.44:0/350779866 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3009a2a0 con 0x5555b7e97300 2023-09-10T01:04:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d26ffd700 1 -- 172.21.15.44:0/350779866 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d30066040 con 0x5555b7e97300 2023-09-10T01:04:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.955+0000 7f8d26ffd700 1 -- 172.21.15.44:0/350779866 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d300b8100 con 0x5555b7e97300 2023-09-10T01:04:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.956+0000 7f8d525e7b80 1 -- 172.21.15.44:0/350779866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97300 msgr2=0x5555b7e9be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.956+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/350779866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97300 0x5555b7e9be30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000b580 tx=0x7f8d30009d10 comp rx=0 tx=0).stop 2023-09-10T01:04:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.956+0000 7f8d525e7b80 1 -- 172.21.15.44:0/350779866 shutdown_connections 2023-09-10T01:04:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.956+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/350779866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e9d1a0 0x5555b7ea8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.956+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/350779866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97300 0x5555b7e9be30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.956+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/350779866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e97cc0 0x5555b7e9c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.956+0000 7f8d525e7b80 1 -- 172.21.15.44:0/350779866 >> 172.21.15.44:0/350779866 conn(0x5555b7e0c030 msgr2=0x5555b7ea9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.956+0000 7f8d525e7b80 1 -- 172.21.15.44:0/350779866 shutdown_connections 2023-09-10T01:04:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.956+0000 7f8d525e7b80 1 -- 172.21.15.44:0/350779866 wait complete. 2023-09-10T01:04:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.956+0000 7f8d525e7b80 1 -- start start 2023-09-10T01:04:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.957+0000 7f8d525e7b80 1 Processor -- start 2023-09-10T01:04:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.957+0000 7f8d525e7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97300 0x5555b7f012e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.957+0000 7f8d525e7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97cc0 0x5555b7f01820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.957+0000 7f8d525e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e9d1a0 0x5555b7f01d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.957+0000 7f8d525e7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b7f0c4e0 con 0x5555b7e97300 2023-09-10T01:04:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.957+0000 7f8d525e7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b7f0c650 con 0x5555b7e97cc0 2023-09-10T01:04:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.957+0000 7f8d525e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b7f0c810 con 0x5555b7e9d1a0 2023-09-10T01:04:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.957+0000 7f8d525e7b80 10 client.0 _ll_register_callbacks cb 0x5555b7cd1a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:04:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.957+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97300 0x5555b7f012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.957+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97300 0x5555b7f012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55452/0 (socket says 172.21.15.44:55452) 2023-09-10T01:04:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.957+0000 7f8d3ffff700 1 -- 172.21.15.44:0/2625635528 learned_addr learned my addr 172.21.15.44:0/2625635528 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.957+0000 7f8d449be700 1 --2- 172.21.15.44:0/2625635528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e9d1a0 0x5555b7f01d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.957+0000 7f8d3f7fe700 1 --2- 172.21.15.44:0/2625635528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97cc0 0x5555b7f01820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d3ffff700 1 -- 172.21.15.44:0/2625635528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e9d1a0 msgr2=0x5555b7f01d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d3ffff700 1 --2- 172.21.15.44:0/2625635528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e9d1a0 0x5555b7f01d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d3ffff700 1 -- 172.21.15.44:0/2625635528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97cc0 msgr2=0x5555b7f01820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d3ffff700 1 --2- 172.21.15.44:0/2625635528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97cc0 0x5555b7f01820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d3ffff700 1 -- 172.21.15.44:0/2625635528 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d380092c0 con 0x5555b7e97300 2023-09-10T01:04:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d449be700 1 --2- 172.21.15.44:0/2625635528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e9d1a0 0x5555b7f01d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:04:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d3f7fe700 1 --2- 172.21.15.44:0/2625635528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97cc0 0x5555b7f01820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:04:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d3ffff700 1 --2- 172.21.15.44:0/2625635528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97300 0x5555b7f012e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8d3009a230 tx=0x7f8d300a2160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c507ba4be8a3fed server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:04:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d1f7fe700 1 -- 172.21.15.44:0/2625635528 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30096b20 con 0x5555b7e97300 2023-09-10T01:04:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d1f7fe700 1 -- 172.21.15.44:0/2625635528 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8d3009a2a0 con 0x5555b7e97300 2023-09-10T01:04:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d525e7b80 1 -- 172.21.15.44:0/2625635528 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d300969b0 con 0x5555b7e97300 2023-09-10T01:04:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d1f7fe700 1 -- 172.21.15.44:0/2625635528 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d300a2280 con 0x5555b7e97300 2023-09-10T01:04:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d525e7b80 10 client.18624 fetch_fsmap learned FSMap version 96 2023-09-10T01:04:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d525e7b80 1 -- 172.21.15.44:0/2625635528 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) v3 -- 0x7f8d38009050 con 0x5555b7e97300 2023-09-10T01:04:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d1f7fe700 1 -- 172.21.15.44:0/2625635528 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d300969b0 con 0x5555b7e97300 2023-09-10T01:04:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.958+0000 7f8d1f7fe700 1 -- 172.21.15.44:0/2625635528 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 96) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f8d3009c070 con 0x5555b7e97300 2023-09-10T01:04:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.959+0000 7f8d525e7b80 10 client.18624 fetch_fsmap finished waiting for FSMap version 96 2023-09-10T01:04:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.959+0000 7f8d525e7b80 10 client.18624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:04:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.959+0000 7f8d525e7b80 20 client.18624 populate_metadata read hostname 'smithi044' 2023-09-10T01:04:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.959+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/2625635528 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5555b7f0dac0 0x5555b7f0fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.959+0000 7f8d525e7b80 4 client.18624 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:04:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.959+0000 7f8d525e7b80 1 -- 172.21.15.44:0/2625635528 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5555b7de28c0 con 0x5555b7f0dac0 2023-09-10T01:04:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.959+0000 7f8d3f7fe700 1 --2- 172.21.15.44:0/2625635528 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5555b7f0dac0 0x5555b7f0fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.959+0000 7f8d1f7fe700 10 client.18624 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:04:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.959+0000 7f8d3f7fe700 1 --2- 172.21.15.44:0/2625635528 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5555b7f0dac0 0x5555b7f0fee0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b64d809e5c8208 server_cookie=e37c37c95277b694 in_seq=0 out_seq=0 2023-09-10T01:04:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.972+0000 7f8d1f7fe700 1 -- 172.21.15.44:0/2625635528 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5555b7de28c0 con 0x5555b7f0dac0 2023-09-10T01:04:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.972+0000 7f8d1f7fe700 10 client.18624 handle_command_reply: tid=0 2023-09-10T01:04:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.972+0000 7f8d525e7b80 1 client.18624 shutdown 2023-09-10T01:04:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.972+0000 7f8d525e7b80 1 -- 172.21.15.44:0/2625635528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97300 msgr2=0x5555b7f012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.972+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/2625635528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97300 0x5555b7f012e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8d3009a230 tx=0x7f8d300a2160 comp rx=0 tx=0).stop 2023-09-10T01:04:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.972+0000 7f8d525e7b80 1 -- 172.21.15.44:0/2625635528 shutdown_connections 2023-09-10T01:04:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.972+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/2625635528 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5555b7f0dac0 0x5555b7f0fee0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.972+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/2625635528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5555b7e9d1a0 0x5555b7f01d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.972+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/2625635528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5555b7e97300 0x5555b7f012e0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.972+0000 7f8d1f7fe700 0 client.18624 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:04:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.972+0000 7f8d525e7b80 1 --2- 172.21.15.44:0/2625635528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5555b7e97cc0 0x5555b7f01820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.972+0000 7f8d525e7b80 1 -- 172.21.15.44:0/2625635528 >> 172.21.15.44:0/2625635528 conn(0x5555b7e0c030 msgr2=0x5555b7e9be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.973+0000 7f8d525e7b80 1 -- 172.21.15.44:0/2625635528 shutdown_connections 2023-09-10T01:04:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.973+0000 7f8d525e7b80 1 -- 172.21.15.44:0/2625635528 wait complete. 2023-09-10T01:04:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.973+0000 7f8d525e7b80 20 client.18624 trim_cache size 0 max 16384 2023-09-10T01:04:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:04:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:04:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:04:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:04:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.973+0000 7f8d46421700 1 -- 172.21.15.44:0/3417739965 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8d280a07b0 msgr2=0x7f8d280a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.973+0000 7f8d46421700 1 --2- 172.21.15.44:0/3417739965 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8d280a07b0 0x7f8d280a2c70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8d40104ad0 tx=0x7f8d34009260 comp rx=0 tx=0).stop 2023-09-10T01:04:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.973+0000 7f8d46421700 1 -- 172.21.15.44:0/3417739965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d400ff760 msgr2=0x7f8d4019b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.973+0000 7f8d46421700 1 --2- 172.21.15.44:0/3417739965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d400ff760 0x7f8d4019b550 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8d30006010 tx=0x7f8d3000f670 comp rx=0 tx=0).stop 2023-09-10T01:04:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.973+0000 7f8d46421700 1 -- 172.21.15.44:0/3417739965 shutdown_connections 2023-09-10T01:04:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.973+0000 7f8d46421700 1 --2- 172.21.15.44:0/3417739965 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8d280a07b0 0x7f8d280a2c70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.973+0000 7f8d46421700 1 --2- 172.21.15.44:0/3417739965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d401055b0 0x7f8d401953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.973+0000 7f8d46421700 1 --2- 172.21.15.44:0/3417739965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d400ff760 0x7f8d4019b550 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.973+0000 7f8d46421700 1 --2- 172.21.15.44:0/3417739965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d40100120 0x7f8d4019ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.973+0000 7f8d46421700 1 -- 172.21.15.44:0/3417739965 >> 172.21.15.44:0/3417739965 conn(0x7f8d40074f30 msgr2=0x7f8d400fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.973+0000 7f8d46421700 1 -- 172.21.15.44:0/3417739965 shutdown_connections 2023-09-10T01:04:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:35.973+0000 7f8d46421700 1 -- 172.21.15.44:0/3417739965 wait complete. 2023-09-10T01:04:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:36 smithi099 ceph-mon[102692]: pgmap v760: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T01:04:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:36 smithi131 ceph-mon[103713]: pgmap v760: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T01:04:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:36 smithi044 ceph-mon[98676]: pgmap v760: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T01:04:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:38 smithi099 ceph-mon[102692]: pgmap v761: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-09-10T01:04:39.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:38 smithi131 ceph-mon[103713]: pgmap v761: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-09-10T01:04:39.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:38 smithi044 ceph-mon[98676]: pgmap v761: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-09-10T01:04:40.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:39 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:04:40.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:39 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:04:40.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:39 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:04:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:40 smithi099 ceph-mon[102692]: pgmap v762: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-09-10T01:04:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:40 smithi131 ceph-mon[103713]: pgmap v762: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-09-10T01:04:41.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:40 smithi044 ceph-mon[98676]: pgmap v762: 129 pgs: 129 active+clean; 9.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-09-10T01:04:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:42 smithi131 ceph-mon[103713]: pgmap v763: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 259 op/s 2023-09-10T01:04:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:42 smithi044 ceph-mon[98676]: pgmap v763: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 259 op/s 2023-09-10T01:04:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:42 smithi099 ceph-mon[102692]: pgmap v763: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 259 op/s 2023-09-10T01:04:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:44 smithi131 ceph-mon[103713]: pgmap v764: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 177 op/s 2023-09-10T01:04:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:44 smithi044 ceph-mon[98676]: pgmap v764: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 177 op/s 2023-09-10T01:04:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:44 smithi099 ceph-mon[102692]: pgmap v764: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 177 op/s 2023-09-10T01:04:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:46 smithi131 ceph-mon[103713]: pgmap v765: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 270 op/s 2023-09-10T01:04:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:46 smithi044 ceph-mon[98676]: pgmap v765: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 270 op/s 2023-09-10T01:04:47.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:46 smithi099 ceph-mon[102692]: pgmap v765: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 270 op/s 2023-09-10T01:04:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.087+0000 7f6bd6ae9700 1 -- 172.21.15.44:0/459211151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bd01055b0 msgr2=0x7f6bd0105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.087+0000 7f6bd6ae9700 1 --2- 172.21.15.44:0/459211151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bd01055b0 0x7f6bd0105990 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40099c0 tx=0x7f6bc4009cd0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.087+0000 7f6bd6ae9700 1 -- 172.21.15.44:0/459211151 shutdown_connections 2023-09-10T01:04:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.087+0000 7f6bd6ae9700 1 --2- 172.21.15.44:0/459211151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bd0100120 0x7f6bd010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.087+0000 7f6bd6ae9700 1 --2- 172.21.15.44:0/459211151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bd01055b0 0x7f6bd0105990 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.087+0000 7f6bd6ae9700 1 --2- 172.21.15.44:0/459211151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bd00ff760 0x7f6bd00ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.087+0000 7f6bd6ae9700 1 -- 172.21.15.44:0/459211151 >> 172.21.15.44:0/459211151 conn(0x7f6bd0074f30 msgr2=0x7f6bd0075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.087+0000 7f6bd6ae9700 1 -- 172.21.15.44:0/459211151 shutdown_connections 2023-09-10T01:04:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.087+0000 7f6bd6ae9700 1 -- 172.21.15.44:0/459211151 wait complete. 2023-09-10T01:04:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.088+0000 7f6bd6ae9700 1 Processor -- start 2023-09-10T01:04:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.088+0000 7f6bd6ae9700 1 -- start start 2023-09-10T01:04:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.088+0000 7f6bd6ae9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bd00ff760 0x7f6bd0197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.088+0000 7f6bd6ae9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bd0100120 0x7f6bd0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.088+0000 7f6bd6ae9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bd01055b0 0x7f6bd01a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.088+0000 7f6bd6ae9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd01a29c0 con 0x7f6bd0100120 2023-09-10T01:04:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.088+0000 7f6bd6ae9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd01a2b30 con 0x7f6bd01055b0 2023-09-10T01:04:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.088+0000 7f6bd6ae9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd01a2cf0 con 0x7f6bd00ff760 2023-09-10T01:04:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bcffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bd0100120 0x7f6bd0197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bcffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bd0100120 0x7f6bd0197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55468/0 (socket says 172.21.15.44:55468) 2023-09-10T01:04:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bd5086700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bd01055b0 0x7f6bd01a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bd4885700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bd00ff760 0x7f6bd0197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bcffff700 1 -- 172.21.15.44:0/2567738490 learned_addr learned my addr 172.21.15.44:0/2567738490 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bcffff700 1 -- 172.21.15.44:0/2567738490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bd00ff760 msgr2=0x7f6bd0197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bcffff700 1 --2- 172.21.15.44:0/2567738490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bd00ff760 0x7f6bd0197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bcffff700 1 -- 172.21.15.44:0/2567738490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bd01055b0 msgr2=0x7f6bd01a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bcffff700 1 --2- 172.21.15.44:0/2567738490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bd01055b0 0x7f6bd01a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bcffff700 1 -- 172.21.15.44:0/2567738490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc40095f0 con 0x7f6bd0100120 2023-09-10T01:04:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bd5086700 1 --2- 172.21.15.44:0/2567738490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bd01055b0 0x7f6bd01a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:04:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bd4885700 1 --2- 172.21.15.44:0/2567738490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bd00ff760 0x7f6bd0197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:04:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bcffff700 1 --2- 172.21.15.44:0/2567738490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bd0100120 0x7f6bd0197a90 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00e9a0 tx=0x7f6bbc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bcdffb700 1 -- 172.21.15.44:0/2567738490 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc009790 con 0x7f6bd0100120 2023-09-10T01:04:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bcdffb700 1 -- 172.21.15.44:0/2567738490 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bbc010430 con 0x7f6bd0100120 2023-09-10T01:04:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bd6ae9700 1 -- 172.21.15.44:0/2567738490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd01a2fd0 con 0x7f6bd0100120 2023-09-10T01:04:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.089+0000 7f6bcdffb700 1 -- 172.21.15.44:0/2567738490 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc00cea0 con 0x7f6bd0100120 2023-09-10T01:04:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.090+0000 7f6bd6ae9700 1 -- 172.21.15.44:0/2567738490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd01a34a0 con 0x7f6bd0100120 2023-09-10T01:04:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.092+0000 7f6bcdffb700 1 -- 172.21.15.44:0/2567738490 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f6bbc0105a0 con 0x7f6bd0100120 2023-09-10T01:04:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.093+0000 7f6bcdffb700 1 --2- 172.21.15.44:0/2567738490 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6bc00a0830 0x7f6bc00a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.093+0000 7f6bd4885700 1 --2- 172.21.15.44:0/2567738490 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6bc00a0830 0x7f6bc00a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.093+0000 7f6bcdffb700 1 -- 172.21.15.44:0/2567738490 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f6bbc014070 con 0x7f6bd0100120 2023-09-10T01:04:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.093+0000 7f6bd4885700 1 --2- 172.21.15.44:0/2567738490 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6bc00a0830 0x7f6bc00a2cf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400b580 tx=0x7f6bc401a040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6be2cafb80 1 Processor -- start 2023-09-10T01:04:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6be2cafb80 1 -- start start 2023-09-10T01:04:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6be2cafb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956fd1a0 0x556e956fd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6be2cafb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956f7300 0x556e956f7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6be2cafb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e95703f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6be2cafb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e956506d0 con 0x556e956f7cc0 2023-09-10T01:04:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6be2cafb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9564dff0 con 0x556e956f7300 2023-09-10T01:04:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6be2cafb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e956498e0 con 0x556e956fd1a0 2023-09-10T01:04:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6bd5086700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e95703f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6bd4885700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956fd1a0 0x556e956fd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6bd5086700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e95703f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55478/0 (socket says 172.21.15.44:55478) 2023-09-10T01:04:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6bd5086700 1 -- 172.21.15.44:0/1068359544 learned_addr learned my addr 172.21.15.44:0/1068359544 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6bcffff700 1 --2- 172.21.15.44:0/1068359544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956f7300 0x556e956f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6bd5086700 1 -- 172.21.15.44:0/1068359544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956fd1a0 msgr2=0x556e956fd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6bd5086700 1 --2- 172.21.15.44:0/1068359544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956fd1a0 0x556e956fd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6bd5086700 1 -- 172.21.15.44:0/1068359544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956f7300 msgr2=0x556e956f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6bd5086700 1 --2- 172.21.15.44:0/1068359544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956f7300 0x556e956f7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6bd5086700 1 -- 172.21.15.44:0/1068359544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e95475d80 con 0x556e956f7cc0 2023-09-10T01:04:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.094+0000 7f6bd5086700 1 --2- 172.21.15.44:0/1068359544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e95703f30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8003eb0 tx=0x7f6bc8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7972c546d64d76f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.095+0000 7f6bbaffd700 1 -- 172.21.15.44:0/1068359544 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc80092d0 con 0x556e956f7cc0 2023-09-10T01:04:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.095+0000 7f6bbaffd700 1 -- 172.21.15.44:0/1068359544 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bc800f040 con 0x556e956f7cc0 2023-09-10T01:04:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.095+0000 7f6bbaffd700 1 -- 172.21.15.44:0/1068359544 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc800e750 con 0x556e956f7cc0 2023-09-10T01:04:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.095+0000 7f6be2cafb80 1 -- 172.21.15.44:0/1068359544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 msgr2=0x556e95703f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.095+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/1068359544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e95703f30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8003eb0 tx=0x7f6bc8009920 comp rx=0 tx=0).stop 2023-09-10T01:04:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.095+0000 7f6be2cafb80 1 -- 172.21.15.44:0/1068359544 shutdown_connections 2023-09-10T01:04:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.095+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/1068359544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e95703f30 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.095+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/1068359544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956fd1a0 0x556e956fd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.095+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/1068359544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956f7300 0x556e956f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.095+0000 7f6be2cafb80 1 -- 172.21.15.44:0/1068359544 >> 172.21.15.44:0/1068359544 conn(0x556e9566c030 msgr2=0x556e956f2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.095+0000 7f6be2cafb80 1 -- 172.21.15.44:0/1068359544 shutdown_connections 2023-09-10T01:04:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.095+0000 7f6be2cafb80 1 -- 172.21.15.44:0/1068359544 wait complete. 2023-09-10T01:04:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.096+0000 7f6be2cafb80 1 -- start start 2023-09-10T01:04:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.096+0000 7f6be2cafb80 1 Processor -- start 2023-09-10T01:04:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.096+0000 7f6be2cafb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7300 0x556e95703d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.096+0000 7f6be2cafb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956f7cc0 0x556e95701010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.096+0000 7f6be2cafb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956fd1a0 0x556e95701550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.096+0000 7f6be2cafb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e955e7b10 con 0x556e956f7300 2023-09-10T01:04:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.096+0000 7f6be2cafb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e955e7810 con 0x556e956fd1a0 2023-09-10T01:04:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.096+0000 7f6be2cafb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e954cb8c0 con 0x556e956f7cc0 2023-09-10T01:04:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.096+0000 7f6be2cafb80 10 client.0 _ll_register_callbacks cb 0x556e95531a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:04:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.096+0000 7f6bd4885700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7300 0x556e95703d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.096+0000 7f6bd4885700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7300 0x556e95703d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55480/0 (socket says 172.21.15.44:55480) 2023-09-10T01:04:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.096+0000 7f6bd4885700 1 -- 172.21.15.44:0/2475190163 learned_addr learned my addr 172.21.15.44:0/2475190163 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.097+0000 7f6bcffff700 1 --2- 172.21.15.44:0/2475190163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956f7cc0 0x556e95701010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.097+0000 7f6bd5086700 1 --2- 172.21.15.44:0/2475190163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956fd1a0 0x556e95701550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.097+0000 7f6bd4885700 1 -- 172.21.15.44:0/2475190163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956f7cc0 msgr2=0x556e95701010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.097+0000 7f6bd4885700 1 --2- 172.21.15.44:0/2475190163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956f7cc0 0x556e95701010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.097+0000 7f6bd4885700 1 -- 172.21.15.44:0/2475190163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956fd1a0 msgr2=0x556e95701550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.097+0000 7f6bd4885700 1 --2- 172.21.15.44:0/2475190163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956fd1a0 0x556e95701550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.097+0000 7f6bd4885700 1 -- 172.21.15.44:0/2475190163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc8003b90 con 0x556e956f7300 2023-09-10T01:04:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.097+0000 7f6bd5086700 1 --2- 172.21.15.44:0/2475190163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956fd1a0 0x556e95701550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:04:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.097+0000 7f6bd4885700 1 --2- 172.21.15.44:0/2475190163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7300 0x556e95703d60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40098e0 tx=0x7f6bc4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a99506104743228 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.097+0000 7f6baffff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:04:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.097+0000 7f6baffff700 1 -- 172.21.15.44:0/2475190163 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc402c040 con 0x556e956f7300 2023-09-10T01:04:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.097+0000 7f6be2cafb80 1 -- 172.21.15.44:0/2475190163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e955a1490 con 0x556e956f7300 2023-09-10T01:04:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.097+0000 7f6baffff700 1 -- 172.21.15.44:0/2475190163 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f6bc400f460 con 0x556e956f7300 2023-09-10T01:04:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.097+0000 7f6baffff700 1 -- 172.21.15.44:0/2475190163 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bc400f620 con 0x556e956f7300 2023-09-10T01:04:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.097+0000 7f6be2cafb80 10 client.18636 fetch_fsmap learned FSMap version 98 2023-09-10T01:04:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.097+0000 7f6be2cafb80 1 -- 172.21.15.44:0/2475190163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) v3 -- 0x556e9545ec00 con 0x556e956f7300 2023-09-10T01:04:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.098+0000 7f6baffff700 1 -- 172.21.15.44:0/2475190163 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556e955a1490 con 0x556e956f7300 2023-09-10T01:04:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.098+0000 7f6baffff700 1 -- 172.21.15.44:0/2475190163 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 98) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f6bc4026760 con 0x556e956f7300 2023-09-10T01:04:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.098+0000 7f6be2cafb80 10 client.18636 fetch_fsmap finished waiting for FSMap version 98 2023-09-10T01:04:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.098+0000 7f6be2cafb80 10 client.18636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:04:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.098+0000 7f6be2cafb80 20 client.18636 populate_metadata read hostname 'smithi044' 2023-09-10T01:04:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.098+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/2475190163 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556e95715780 0x556e95717ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.098+0000 7f6be2cafb80 4 client.18636 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:04:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.098+0000 7f6be2cafb80 1 -- 172.21.15.44:0/2475190163 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e9565e840 con 0x556e95715780 2023-09-10T01:04:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.098+0000 7f6bcffff700 1 --2- 172.21.15.44:0/2475190163 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556e95715780 0x556e95717ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.099+0000 7f6baffff700 10 client.18636 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:04:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.099+0000 7f6bcffff700 1 --2- 172.21.15.44:0/2475190163 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556e95715780 0x556e95717ba0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f8a28cc10d13517 server_cookie=12b1cdb2e3a86392 in_seq=0 out_seq=0 2023-09-10T01:04:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.099+0000 7f6baffff700 1 -- 172.21.15.44:0/2475190163 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556e9565e840 con 0x556e95715780 2023-09-10T01:04:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.099+0000 7f6baffff700 10 client.18636 handle_command_reply: tid=0 2023-09-10T01:04:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.099+0000 7f6be2cafb80 1 client.18636 shutdown 2023-09-10T01:04:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.099+0000 7f6be2cafb80 1 -- 172.21.15.44:0/2475190163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7300 msgr2=0x556e95703d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.099+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/2475190163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7300 0x556e95703d60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40098e0 tx=0x7f6bc4009cd0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.099+0000 7f6be2cafb80 1 -- 172.21.15.44:0/2475190163 shutdown_connections 2023-09-10T01:04:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.099+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/2475190163 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556e95715780 0x556e95717ba0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.099+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/2475190163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956fd1a0 0x556e95701550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.099+0000 7f6baffff700 0 client.18636 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:04:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.099+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/2475190163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7300 0x556e95703d60 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.099+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/2475190163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956f7cc0 0x556e95701010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.099+0000 7f6be2cafb80 1 -- 172.21.15.44:0/2475190163 >> 172.21.15.44:0/2475190163 conn(0x556e9566c030 msgr2=0x556e956f17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.100+0000 7f6be2cafb80 1 -- 172.21.15.44:0/2475190163 shutdown_connections 2023-09-10T01:04:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.100+0000 7f6be2cafb80 1 -- 172.21.15.44:0/2475190163 wait complete. 2023-09-10T01:04:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.100+0000 7f6be2cafb80 20 client.18636 trim_cache size 0 max 16384 2023-09-10T01:04:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.114+0000 7f6be2cafb80 1 Processor -- start 2023-09-10T01:04:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.114+0000 7f6be2cafb80 1 -- start start 2023-09-10T01:04:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6be2cafb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956f7300 0x556e95708520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6be2cafb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e956fc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6be2cafb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956fd1a0 0x556e95703820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6be2cafb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e956fbb80 con 0x556e956f7cc0 2023-09-10T01:04:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6be2cafb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e956f3240 con 0x556e956fd1a0 2023-09-10T01:04:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6be2cafb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e956eefc0 con 0x556e956f7300 2023-09-10T01:04:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6bcffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e956fc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6bcffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e956fc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55490/0 (socket says 172.21.15.44:55490) 2023-09-10T01:04:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6bd4885700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956f7300 0x556e95708520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6bcffff700 1 -- 172.21.15.44:0/110678283 learned_addr learned my addr 172.21.15.44:0/110678283 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6bd5086700 1 --2- 172.21.15.44:0/110678283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956fd1a0 0x556e95703820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6bcffff700 1 -- 172.21.15.44:0/110678283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956f7300 msgr2=0x556e95708520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6bcffff700 1 --2- 172.21.15.44:0/110678283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956f7300 0x556e95708520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6bcffff700 1 -- 172.21.15.44:0/110678283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956fd1a0 msgr2=0x556e95703820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6bcffff700 1 --2- 172.21.15.44:0/110678283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956fd1a0 0x556e95703820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6bcffff700 1 -- 172.21.15.44:0/110678283 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e9549c1f0 con 0x556e956f7cc0 2023-09-10T01:04:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6bcffff700 1 --2- 172.21.15.44:0/110678283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e956fc790 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0994d0 tx=0x7f6bbc00c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47697f8b20c44f07 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.115+0000 7f6bbb7fe700 1 -- 172.21.15.44:0/110678283 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc099540 con 0x556e956f7cc0 2023-09-10T01:04:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.116+0000 7f6bbb7fe700 1 -- 172.21.15.44:0/110678283 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bbc09d370 con 0x556e956f7cc0 2023-09-10T01:04:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.116+0000 7f6bbb7fe700 1 -- 172.21.15.44:0/110678283 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc0997a0 con 0x556e956f7cc0 2023-09-10T01:04:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.116+0000 7f6be2cafb80 1 -- 172.21.15.44:0/110678283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 msgr2=0x556e956fc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.116+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/110678283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e956fc790 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0994d0 tx=0x7f6bbc00c480 comp rx=0 tx=0).stop 2023-09-10T01:04:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.116+0000 7f6be2cafb80 1 -- 172.21.15.44:0/110678283 shutdown_connections 2023-09-10T01:04:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.116+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/110678283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956fd1a0 0x556e95703820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.116+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/110678283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956f7300 0x556e95708520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.116+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/110678283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e956fc790 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.116+0000 7f6be2cafb80 1 -- 172.21.15.44:0/110678283 >> 172.21.15.44:0/110678283 conn(0x556e9566c030 msgr2=0x556e956f6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.116+0000 7f6be2cafb80 1 -- 172.21.15.44:0/110678283 shutdown_connections 2023-09-10T01:04:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.116+0000 7f6be2cafb80 1 -- 172.21.15.44:0/110678283 wait complete. 2023-09-10T01:04:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.116+0000 7f6be2cafb80 1 -- start start 2023-09-10T01:04:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6be2cafb80 1 Processor -- start 2023-09-10T01:04:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6be2cafb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956f7300 0x556e95706210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6be2cafb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e95706750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6be2cafb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956fd1a0 0x556e957685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6be2cafb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e95706ea0 con 0x556e956f7cc0 2023-09-10T01:04:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6be2cafb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e95768af0 con 0x556e956f7300 2023-09-10T01:04:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6be2cafb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e95768cb0 con 0x556e956fd1a0 2023-09-10T01:04:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6be2cafb80 10 client.0 _ll_register_callbacks cb 0x7f6bc4005410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:04:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6bcffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e95706750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6bcffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e95706750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55496/0 (socket says 172.21.15.44:55496) 2023-09-10T01:04:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6bd4885700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956f7300 0x556e95706210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6bcffff700 1 -- 172.21.15.44:0/3031379478 learned_addr learned my addr 172.21.15.44:0/3031379478 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:04:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6bd5086700 1 --2- 172.21.15.44:0/3031379478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956fd1a0 0x556e957685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6bcffff700 1 -- 172.21.15.44:0/3031379478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956fd1a0 msgr2=0x556e957685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6bcffff700 1 --2- 172.21.15.44:0/3031379478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956fd1a0 0x556e957685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6bcffff700 1 -- 172.21.15.44:0/3031379478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956f7300 msgr2=0x556e95706210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:04:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6bcffff700 1 --2- 172.21.15.44:0/3031379478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956f7300 0x556e95706210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.117+0000 7f6bcffff700 1 -- 172.21.15.44:0/3031379478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd01a34a0 con 0x556e956f7cc0 2023-09-10T01:04:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.118+0000 7f6bcffff700 1 --2- 172.21.15.44:0/3031379478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e95706750 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0c9de0 tx=0x7f6bbc095880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9fa9f0941d74554 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:04:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.118+0000 7f6baffff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:04:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.118+0000 7f6baffff700 1 -- 172.21.15.44:0/3031379478 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc099540 con 0x556e956f7cc0 2023-09-10T01:04:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.118+0000 7f6baffff700 1 -- 172.21.15.44:0/3031379478 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f6bbc098a30 con 0x556e956f7cc0 2023-09-10T01:04:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.118+0000 7f6be2cafb80 1 -- 172.21.15.44:0/3031379478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6bc800f040 con 0x556e956f7cc0 2023-09-10T01:04:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.118+0000 7f6baffff700 1 -- 172.21.15.44:0/3031379478 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bbc098bf0 con 0x556e956f7cc0 2023-09-10T01:04:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.118+0000 7f6be2cafb80 10 client.18642 fetch_fsmap learned FSMap version 98 2023-09-10T01:04:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.118+0000 7f6be2cafb80 1 -- 172.21.15.44:0/3031379478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) v3 -- 0x556e95474250 con 0x556e956f7cc0 2023-09-10T01:04:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.118+0000 7f6baffff700 1 -- 172.21.15.44:0/3031379478 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bbc07c080 con 0x556e956f7cc0 2023-09-10T01:04:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.118+0000 7f6baffff700 1 -- 172.21.15.44:0/3031379478 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 98) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f6bbc0a1240 con 0x556e956f7cc0 2023-09-10T01:04:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.118+0000 7f6be2cafb80 10 client.18642 fetch_fsmap finished waiting for FSMap version 98 2023-09-10T01:04:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.118+0000 7f6be2cafb80 10 client.18642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:04:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.118+0000 7f6be2cafb80 20 client.18642 populate_metadata read hostname 'smithi044' 2023-09-10T01:04:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.118+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/3031379478 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556e9576a220 0x556e9576c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:04:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.118+0000 7f6be2cafb80 4 client.18642 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:04:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.118+0000 7f6be2cafb80 1 -- 172.21.15.44:0/3031379478 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556e956428c0 con 0x556e9576a220 2023-09-10T01:04:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.119+0000 7f6bd4885700 1 --2- 172.21.15.44:0/3031379478 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556e9576a220 0x556e9576c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:04:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.119+0000 7f6baffff700 10 client.18642 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:04:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.119+0000 7f6bd4885700 1 --2- 172.21.15.44:0/3031379478 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556e9576a220 0x556e9576c640 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e87f38484e9e4d4 server_cookie=785ca4dd3326c43c in_seq=0 out_seq=0 2023-09-10T01:04:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.133+0000 7f6baffff700 1 -- 172.21.15.44:0/3031379478 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556e956428c0 con 0x556e9576a220 2023-09-10T01:04:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.133+0000 7f6baffff700 10 client.18642 handle_command_reply: tid=0 2023-09-10T01:04:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.133+0000 7f6be2cafb80 1 client.18642 shutdown 2023-09-10T01:04:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.133+0000 7f6be2cafb80 1 -- 172.21.15.44:0/3031379478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 msgr2=0x556e95706750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.133+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/3031379478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e95706750 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0c9de0 tx=0x7f6bbc095880 comp rx=0 tx=0).stop 2023-09-10T01:04:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.133+0000 7f6be2cafb80 1 -- 172.21.15.44:0/3031379478 shutdown_connections 2023-09-10T01:04:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.133+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/3031379478 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556e9576a220 0x556e9576c640 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.133+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/3031379478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e956fd1a0 0x556e957685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.133+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/3031379478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e956f7300 0x556e95706210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.133+0000 7f6baffff700 0 client.18642 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:04:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.133+0000 7f6be2cafb80 1 --2- 172.21.15.44:0/3031379478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e956f7cc0 0x556e95706750 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.133+0000 7f6be2cafb80 1 -- 172.21.15.44:0/3031379478 >> 172.21.15.44:0/3031379478 conn(0x556e9566c030 msgr2=0x556e956f6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.133+0000 7f6be2cafb80 1 -- 172.21.15.44:0/3031379478 shutdown_connections 2023-09-10T01:04:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.133+0000 7f6be2cafb80 1 -- 172.21.15.44:0/3031379478 wait complete. 2023-09-10T01:04:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.133+0000 7f6be2cafb80 20 client.18642 trim_cache size 0 max 16384 2023-09-10T01:04:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:04:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:04:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:04:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:04:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.134+0000 7f6bd6ae9700 1 -- 172.21.15.44:0/2567738490 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6bc00a0830 msgr2=0x7f6bc00a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.134+0000 7f6bd6ae9700 1 --2- 172.21.15.44:0/2567738490 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6bc00a0830 0x7f6bc00a2cf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400b580 tx=0x7f6bc401a040 comp rx=0 tx=0).stop 2023-09-10T01:04:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.134+0000 7f6bd6ae9700 1 -- 172.21.15.44:0/2567738490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bd0100120 msgr2=0x7f6bd0197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:04:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.134+0000 7f6bd6ae9700 1 --2- 172.21.15.44:0/2567738490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bd0100120 0x7f6bd0197a90 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00e9a0 tx=0x7f6bbc00ecb0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.134+0000 7f6bd6ae9700 1 -- 172.21.15.44:0/2567738490 shutdown_connections 2023-09-10T01:04:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.134+0000 7f6bd6ae9700 1 --2- 172.21.15.44:0/2567738490 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6bc00a0830 0x7f6bc00a2cf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.134+0000 7f6bd6ae9700 1 --2- 172.21.15.44:0/2567738490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bd01055b0 0x7f6bd01a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.134+0000 7f6bd6ae9700 1 --2- 172.21.15.44:0/2567738490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bd00ff760 0x7f6bd0197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.134+0000 7f6bd6ae9700 1 --2- 172.21.15.44:0/2567738490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bd0100120 0x7f6bd0197a90 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:04:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.134+0000 7f6bd6ae9700 1 -- 172.21.15.44:0/2567738490 >> 172.21.15.44:0/2567738490 conn(0x7f6bd0074f30 msgr2=0x7f6bd00fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:04:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.134+0000 7f6bd6ae9700 1 -- 172.21.15.44:0/2567738490 shutdown_connections 2023-09-10T01:04:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:04:49.134+0000 7f6bd6ae9700 1 -- 172.21.15.44:0/2567738490 wait complete. 2023-09-10T01:04:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:48 smithi131 ceph-mon[103713]: pgmap v766: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2023-09-10T01:04:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:48 smithi044 ceph-mon[98676]: pgmap v766: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2023-09-10T01:04:49.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:48 smithi099 ceph-mon[102692]: pgmap v766: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2023-09-10T01:04:50.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:49 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:04:50.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:49 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:04:50.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:49 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:04:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:50 smithi131 ceph-mon[103713]: pgmap v767: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2023-09-10T01:04:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:50 smithi044 ceph-mon[98676]: pgmap v767: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2023-09-10T01:04:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:50 smithi099 ceph-mon[102692]: pgmap v767: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2023-09-10T01:04:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:52 smithi131 ceph-mon[103713]: pgmap v768: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 276 op/s 2023-09-10T01:04:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:52 smithi044 ceph-mon[98676]: pgmap v768: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 276 op/s 2023-09-10T01:04:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:52 smithi099 ceph-mon[102692]: pgmap v768: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 276 op/s 2023-09-10T01:04:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:54 smithi131 ceph-mon[103713]: pgmap v769: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 173 op/s 2023-09-10T01:04:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:54 smithi044 ceph-mon[98676]: pgmap v769: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 173 op/s 2023-09-10T01:04:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:54 smithi099 ceph-mon[102692]: pgmap v769: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 173 op/s 2023-09-10T01:04:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:56 smithi131 ceph-mon[103713]: pgmap v770: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 245 op/s 2023-09-10T01:04:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:56 smithi044 ceph-mon[98676]: pgmap v770: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 245 op/s 2023-09-10T01:04:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:56 smithi099 ceph-mon[102692]: pgmap v770: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 245 op/s 2023-09-10T01:04:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:58 smithi131 ceph-mon[103713]: pgmap v771: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-09-10T01:04:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:58 smithi044 ceph-mon[98676]: pgmap v771: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-09-10T01:04:59.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:58 smithi099 ceph-mon[102692]: pgmap v771: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-09-10T01:05:00.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:04:59 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:00.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:04:59 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:00.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:04:59 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:00 smithi131 ceph-mon[103713]: pgmap v772: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-09-10T01:05:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:00 smithi044 ceph-mon[98676]: pgmap v772: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-09-10T01:05:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:00 smithi099 ceph-mon[102692]: pgmap v772: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-09-10T01:05:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.274+0000 7f164ef4d700 1 -- 172.21.15.44:0/4101278296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1648112550 msgr2=0x7f1648114940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.274+0000 7f164ef4d700 1 --2- 172.21.15.44:0/4101278296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1648112550 0x7f1648114940 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f164000b0a0 tx=0x7f164000b3b0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.274+0000 7f164ef4d700 1 -- 172.21.15.44:0/4101278296 shutdown_connections 2023-09-10T01:05:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.274+0000 7f164ef4d700 1 --2- 172.21.15.44:0/4101278296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1648112550 0x7f1648114940 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.274+0000 7f164ef4d700 1 --2- 172.21.15.44:0/4101278296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1648108fa0 0x7f1648109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.274+0000 7f164ef4d700 1 --2- 172.21.15.44:0/4101278296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16481098c0 0x7f1648112010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.274+0000 7f164ef4d700 1 -- 172.21.15.44:0/4101278296 >> 172.21.15.44:0/4101278296 conn(0x7f164806c6c0 msgr2=0x7f164806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.274+0000 7f164ef4d700 1 -- 172.21.15.44:0/4101278296 shutdown_connections 2023-09-10T01:05:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.274+0000 7f164ef4d700 1 -- 172.21.15.44:0/4101278296 wait complete. 2023-09-10T01:05:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.275+0000 7f164ef4d700 1 Processor -- start 2023-09-10T01:05:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.275+0000 7f164ef4d700 1 -- start start 2023-09-10T01:05:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.275+0000 7f164ef4d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1648108fa0 0x7f16481a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.275+0000 7f164ef4d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16481098c0 0x7f16481a3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.275+0000 7f164ef4d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1648112550 0x7f164819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.275+0000 7f164ef4d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164819d100 con 0x7f16481098c0 2023-09-10T01:05:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.275+0000 7f164ef4d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164819d270 con 0x7f1648108fa0 2023-09-10T01:05:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.275+0000 7f164ef4d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164819d430 con 0x7f1648112550 2023-09-10T01:05:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.275+0000 7f164d4ea700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1648112550 0x7f164819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.275+0000 7f164cce9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1648108fa0 0x7f16481a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.275+0000 7f164cce9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1648108fa0 0x7f16481a2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57248/0 (socket says 172.21.15.44:57248) 2023-09-10T01:05:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.275+0000 7f164d4ea700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1648112550 0x7f164819ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34160/0 (socket says 172.21.15.44:34160) 2023-09-10T01:05:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.275+0000 7f164cce9700 1 -- 172.21.15.44:0/2610769492 learned_addr learned my addr 172.21.15.44:0/2610769492 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.275+0000 7f1647fff700 1 --2- 172.21.15.44:0/2610769492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16481098c0 0x7f16481a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.276+0000 7f164cce9700 1 -- 172.21.15.44:0/2610769492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1648112550 msgr2=0x7f164819ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.276+0000 7f164cce9700 1 --2- 172.21.15.44:0/2610769492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1648112550 0x7f164819ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.276+0000 7f164cce9700 1 -- 172.21.15.44:0/2610769492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16481098c0 msgr2=0x7f16481a3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.276+0000 7f164cce9700 1 --2- 172.21.15.44:0/2610769492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16481098c0 0x7f16481a3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.276+0000 7f164cce9700 1 -- 172.21.15.44:0/2610769492 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c009520 con 0x7f1648108fa0 2023-09-10T01:05:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.276+0000 7f164cce9700 1 --2- 172.21.15.44:0/2610769492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1648108fa0 0x7f16481a2c30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f163c00eb30 tx=0x7f163c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.276+0000 7f1645ffb700 1 -- 172.21.15.44:0/2610769492 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c009880 con 0x7f1648108fa0 2023-09-10T01:05:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.276+0000 7f164ef4d700 1 -- 172.21.15.44:0/2610769492 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1640009d00 con 0x7f1648108fa0 2023-09-10T01:05:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.276+0000 7f1645ffb700 1 -- 172.21.15.44:0/2610769492 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f163c0044b0 con 0x7f1648108fa0 2023-09-10T01:05:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.276+0000 7f1645ffb700 1 -- 172.21.15.44:0/2610769492 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c003ce0 con 0x7f1648108fa0 2023-09-10T01:05:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.276+0000 7f164ef4d700 1 -- 172.21.15.44:0/2610769492 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164819d900 con 0x7f1648108fa0 2023-09-10T01:05:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.279+0000 7f1645ffb700 1 -- 172.21.15.44:0/2610769492 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f163c003e40 con 0x7f1648108fa0 2023-09-10T01:05:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.280+0000 7f1645ffb700 1 --2- 172.21.15.44:0/2610769492 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f16380a0830 0x7f16380a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.280+0000 7f1645ffb700 1 -- 172.21.15.44:0/2610769492 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f163c014070 con 0x7f1648108fa0 2023-09-10T01:05:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.280+0000 7f1647fff700 1 --2- 172.21.15.44:0/2610769492 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f16380a0830 0x7f16380a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.280+0000 7f1647fff700 1 --2- 172.21.15.44:0/2610769492 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f16380a0830 0x7f16380a2cf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1648071f60 tx=0x7f163400b580 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.281+0000 7f165b113b80 1 Processor -- start 2023-09-10T01:05:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.281+0000 7f165b113b80 1 -- start start 2023-09-10T01:05:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.281+0000 7f165b113b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d71a0 0x55f9d05d7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.281+0000 7f165b113b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d1300 0x55f9d05d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.281+0000 7f165b113b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1cc0 0x55f9d05ddf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.281+0000 7f165b113b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d052a6d0 con 0x55f9d05d71a0 2023-09-10T01:05:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.281+0000 7f165b113b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d0527ff0 con 0x55f9d05d1300 2023-09-10T01:05:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.281+0000 7f165b113b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d05238e0 con 0x55f9d05d1cc0 2023-09-10T01:05:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.281+0000 7f164cce9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d71a0 0x55f9d05d7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.281+0000 7f164cce9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d71a0 0x55f9d05d7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55516/0 (socket says 172.21.15.44:55516) 2023-09-10T01:05:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.281+0000 7f164d4ea700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1cc0 0x55f9d05ddf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.281+0000 7f164cce9700 1 -- 172.21.15.44:0/3261276772 learned_addr learned my addr 172.21.15.44:0/3261276772 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.281+0000 7f1647fff700 1 --2- 172.21.15.44:0/3261276772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d1300 0x55f9d05d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.282+0000 7f164cce9700 1 -- 172.21.15.44:0/3261276772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1cc0 msgr2=0x55f9d05ddf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.282+0000 7f164cce9700 1 --2- 172.21.15.44:0/3261276772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1cc0 0x55f9d05ddf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.282+0000 7f164cce9700 1 -- 172.21.15.44:0/3261276772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d1300 msgr2=0x55f9d05d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.282+0000 7f164cce9700 1 --2- 172.21.15.44:0/3261276772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d1300 0x55f9d05d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.282+0000 7f164cce9700 1 -- 172.21.15.44:0/3261276772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9d034fd80 con 0x55f9d05d71a0 2023-09-10T01:05:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.282+0000 7f164cce9700 1 --2- 172.21.15.44:0/3261276772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d71a0 0x55f9d05d7580 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f163c00c5b0 tx=0x7f163c099100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8901256730a4a99 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.282+0000 7f1632ffd700 1 -- 172.21.15.44:0/3261276772 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c0cc070 con 0x55f9d05d71a0 2023-09-10T01:05:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.282+0000 7f1632ffd700 1 -- 172.21.15.44:0/3261276772 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f163c099da0 con 0x55f9d05d71a0 2023-09-10T01:05:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.282+0000 7f1632ffd700 1 -- 172.21.15.44:0/3261276772 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c082ea0 con 0x55f9d05d71a0 2023-09-10T01:05:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.283+0000 7f165b113b80 1 -- 172.21.15.44:0/3261276772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d71a0 msgr2=0x55f9d05d7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.283+0000 7f165b113b80 1 --2- 172.21.15.44:0/3261276772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d71a0 0x55f9d05d7580 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f163c00c5b0 tx=0x7f163c099100 comp rx=0 tx=0).stop 2023-09-10T01:05:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.283+0000 7f165b113b80 1 -- 172.21.15.44:0/3261276772 shutdown_connections 2023-09-10T01:05:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.283+0000 7f165b113b80 1 --2- 172.21.15.44:0/3261276772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1cc0 0x55f9d05ddf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.283+0000 7f165b113b80 1 --2- 172.21.15.44:0/3261276772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d71a0 0x55f9d05d7580 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.283+0000 7f165b113b80 1 --2- 172.21.15.44:0/3261276772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d1300 0x55f9d05d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.283+0000 7f165b113b80 1 -- 172.21.15.44:0/3261276772 >> 172.21.15.44:0/3261276772 conn(0x55f9d0546030 msgr2=0x55f9d05cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.283+0000 7f165b113b80 1 -- 172.21.15.44:0/3261276772 shutdown_connections 2023-09-10T01:05:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.283+0000 7f165b113b80 1 -- 172.21.15.44:0/3261276772 wait complete. 2023-09-10T01:05:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.284+0000 7f165b113b80 1 -- start start 2023-09-10T01:05:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.284+0000 7f165b113b80 1 Processor -- start 2023-09-10T01:05:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.284+0000 7f165b113b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1300 0x55f9d05dddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.284+0000 7f165b113b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d1cc0 0x55f9d05db0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f165b113b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d71a0 0x55f9d05db6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f165b113b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d04c1b10 con 0x55f9d05d71a0 2023-09-10T01:05:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f165b113b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d04c1810 con 0x55f9d05d1cc0 2023-09-10T01:05:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f165b113b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d03a58c0 con 0x55f9d05d1300 2023-09-10T01:05:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f165b113b80 10 client.0 _ll_register_callbacks cb 0x55f9d040ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:05:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f164cce9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1300 0x55f9d05dddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f1647fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d1cc0 0x55f9d05db0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f164cce9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1300 0x55f9d05dddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34172/0 (socket says 172.21.15.44:34172) 2023-09-10T01:05:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f1647fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d1cc0 0x55f9d05db0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57264/0 (socket says 172.21.15.44:57264) 2023-09-10T01:05:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f164cce9700 1 -- 172.21.15.44:0/2210609661 learned_addr learned my addr 172.21.15.44:0/2210609661 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f164d4ea700 1 --2- 172.21.15.44:0/2210609661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d71a0 0x55f9d05db6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f164cce9700 1 -- 172.21.15.44:0/2210609661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d1cc0 msgr2=0x55f9d05db0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f164cce9700 1 --2- 172.21.15.44:0/2210609661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d1cc0 0x55f9d05db0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f164cce9700 1 -- 172.21.15.44:0/2210609661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d71a0 msgr2=0x55f9d05db6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f164cce9700 1 --2- 172.21.15.44:0/2210609661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d71a0 0x55f9d05db6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f164cce9700 1 -- 172.21.15.44:0/2210609661 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1640009d00 con 0x55f9d05d1300 2023-09-10T01:05:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f164d4ea700 1 --2- 172.21.15.44:0/2210609661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d71a0 0x55f9d05db6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:05:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f164cce9700 1 --2- 172.21.15.44:0/2210609661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1300 0x55f9d05dddd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f163c0a11e0 tx=0x7f163c0c9960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3fa4f835d0eb070 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f1627fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:05:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.285+0000 7f1627fff700 1 -- 172.21.15.44:0/2210609661 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c0cc050 con 0x55f9d05d1300 2023-09-10T01:05:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.286+0000 7f1627fff700 1 -- 172.21.15.44:0/2210609661 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f163c0a1710 con 0x55f9d05d1300 2023-09-10T01:05:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.286+0000 7f165b113b80 1 -- 172.21.15.44:0/2210609661 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9d047b490 con 0x55f9d05d1300 2023-09-10T01:05:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.286+0000 7f1627fff700 1 -- 172.21.15.44:0/2210609661 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f163c082ea0 con 0x55f9d05d1300 2023-09-10T01:05:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.286+0000 7f165b113b80 10 client.28246 fetch_fsmap learned FSMap version 99 2023-09-10T01:05:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.286+0000 7f165b113b80 1 -- 172.21.15.44:0/2210609661 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=4+}) v3 -- 0x55f9d0338c00 con 0x55f9d05d1300 2023-09-10T01:05:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.286+0000 7f1627fff700 1 -- 172.21.15.44:0/2210609661 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f9d047b490 con 0x55f9d05d1300 2023-09-10T01:05:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.286+0000 7f1627fff700 1 -- 172.21.15.44:0/2210609661 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 99) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f163c099900 con 0x55f9d05d1300 2023-09-10T01:05:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.286+0000 7f165b113b80 10 client.28246 fetch_fsmap finished waiting for FSMap version 99 2023-09-10T01:05:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.286+0000 7f165b113b80 10 client.28246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:05:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.286+0000 7f165b113b80 20 client.28246 populate_metadata read hostname 'smithi044' 2023-09-10T01:05:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.286+0000 7f165b113b80 1 --2- 172.21.15.44:0/2210609661 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f9d05eaf90 0x55f9d05ed3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.286+0000 7f165b113b80 4 client.28246 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:05:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.286+0000 7f165b113b80 1 -- 172.21.15.44:0/2210609661 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9d0538840 con 0x55f9d05eaf90 2023-09-10T01:05:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.286+0000 7f1647fff700 1 --2- 172.21.15.44:0/2210609661 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f9d05eaf90 0x55f9d05ed3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.287+0000 7f1627fff700 10 client.28246 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.287+0000 7f1647fff700 1 --2- 172.21.15.44:0/2210609661 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f9d05eaf90 0x55f9d05ed3b0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0e2dc9f187bd52e server_cookie=fc60747d4f81b9ca in_seq=0 out_seq=0 2023-09-10T01:05:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.287+0000 7f1627fff700 1 -- 172.21.15.44:0/2210609661 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f9d0538840 con 0x55f9d05eaf90 2023-09-10T01:05:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.287+0000 7f1627fff700 10 client.28246 handle_command_reply: tid=0 2023-09-10T01:05:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.287+0000 7f165b113b80 1 client.28246 shutdown 2023-09-10T01:05:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.288+0000 7f165b113b80 1 -- 172.21.15.44:0/2210609661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1300 msgr2=0x55f9d05dddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.288+0000 7f165b113b80 1 --2- 172.21.15.44:0/2210609661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1300 0x55f9d05dddd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f163c0a11e0 tx=0x7f163c0c9960 comp rx=0 tx=0).stop 2023-09-10T01:05:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.288+0000 7f165b113b80 1 -- 172.21.15.44:0/2210609661 shutdown_connections 2023-09-10T01:05:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.288+0000 7f165b113b80 1 --2- 172.21.15.44:0/2210609661 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f9d05eaf90 0x55f9d05ed3b0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.288+0000 7f165b113b80 1 --2- 172.21.15.44:0/2210609661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d71a0 0x55f9d05db6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.288+0000 7f1627fff700 0 client.28246 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.288+0000 7f165b113b80 1 --2- 172.21.15.44:0/2210609661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1300 0x55f9d05dddd0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.288+0000 7f165b113b80 1 --2- 172.21.15.44:0/2210609661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d1cc0 0x55f9d05db0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.288+0000 7f165b113b80 1 -- 172.21.15.44:0/2210609661 >> 172.21.15.44:0/2210609661 conn(0x55f9d0546030 msgr2=0x55f9d05cb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.288+0000 7f165b113b80 1 -- 172.21.15.44:0/2210609661 shutdown_connections 2023-09-10T01:05:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.288+0000 7f165b113b80 1 -- 172.21.15.44:0/2210609661 wait complete. 2023-09-10T01:05:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.288+0000 7f165b113b80 20 client.28246 trim_cache size 0 max 16384 2023-09-10T01:05:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.310+0000 7f165b113b80 1 Processor -- start 2023-09-10T01:05:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.310+0000 7f165b113b80 1 -- start start 2023-09-10T01:05:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.310+0000 7f165b113b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d1300 0x55f9d05dd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.310+0000 7f165b113b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d1cc0 0x55f9d05d60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.310+0000 7f165b113b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d71a0 0x55f9d05d6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.310+0000 7f165b113b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d05c68a0 con 0x55f9d05d1cc0 2023-09-10T01:05:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.310+0000 7f165b113b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d05d7b50 con 0x55f9d05d1300 2023-09-10T01:05:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.310+0000 7f165b113b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d05d6e00 con 0x55f9d05d71a0 2023-09-10T01:05:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.311+0000 7f1647fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d1cc0 0x55f9d05d60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.311+0000 7f164d4ea700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d71a0 0x55f9d05d6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.311+0000 7f1647fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d1cc0 0x55f9d05d60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55530/0 (socket says 172.21.15.44:55530) 2023-09-10T01:05:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.311+0000 7f164d4ea700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d71a0 0x55f9d05d6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34182/0 (socket says 172.21.15.44:34182) 2023-09-10T01:05:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.311+0000 7f1647fff700 1 -- 172.21.15.44:0/1366116435 learned_addr learned my addr 172.21.15.44:0/1366116435 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.311+0000 7f1647fff700 1 -- 172.21.15.44:0/1366116435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d71a0 msgr2=0x55f9d05d6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.311+0000 7f1647fff700 1 --2- 172.21.15.44:0/1366116435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d71a0 0x55f9d05d6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.311+0000 7f1647fff700 1 -- 172.21.15.44:0/1366116435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d1300 msgr2=0x55f9d05dd900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:05:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.311+0000 7f1647fff700 1 --2- 172.21.15.44:0/1366116435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d1300 0x55f9d05dd900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.311+0000 7f1647fff700 1 -- 172.21.15.44:0/1366116435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9d03761f0 con 0x55f9d05d1cc0 2023-09-10T01:05:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.311+0000 7f1647fff700 1 --2- 172.21.15.44:0/1366116435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d1cc0 0x55f9d05d60e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f163400bb70 tx=0x7f1634018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37bfa3532e22734 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.311+0000 7f16337fe700 1 -- 172.21.15.44:0/1366116435 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1634030070 con 0x55f9d05d1cc0 2023-09-10T01:05:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.311+0000 7f16337fe700 1 -- 172.21.15.44:0/1366116435 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1634011040 con 0x55f9d05d1cc0 2023-09-10T01:05:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.311+0000 7f16337fe700 1 -- 172.21.15.44:0/1366116435 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1634004c20 con 0x55f9d05d1cc0 2023-09-10T01:05:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.312+0000 7f165b113b80 1 -- 172.21.15.44:0/1366116435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d1cc0 msgr2=0x55f9d05d60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.312+0000 7f165b113b80 1 --2- 172.21.15.44:0/1366116435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d1cc0 0x55f9d05d60e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f163400bb70 tx=0x7f1634018000 comp rx=0 tx=0).stop 2023-09-10T01:05:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.313+0000 7f165b113b80 1 -- 172.21.15.44:0/1366116435 shutdown_connections 2023-09-10T01:05:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.313+0000 7f165b113b80 1 --2- 172.21.15.44:0/1366116435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d71a0 0x55f9d05d6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.313+0000 7f165b113b80 1 --2- 172.21.15.44:0/1366116435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d1300 0x55f9d05dd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.313+0000 7f165b113b80 1 --2- 172.21.15.44:0/1366116435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d1cc0 0x55f9d05d60e0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.313+0000 7f165b113b80 1 -- 172.21.15.44:0/1366116435 >> 172.21.15.44:0/1366116435 conn(0x55f9d0546030 msgr2=0x55f9d05d5190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.313+0000 7f165b113b80 1 -- 172.21.15.44:0/1366116435 shutdown_connections 2023-09-10T01:05:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.313+0000 7f165b113b80 1 -- 172.21.15.44:0/1366116435 wait complete. 2023-09-10T01:05:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.313+0000 7f165b113b80 1 -- start start 2023-09-10T01:05:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.313+0000 7f165b113b80 1 Processor -- start 2023-09-10T01:05:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.314+0000 7f165b113b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d1300 0x55f9d05e0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.314+0000 7f165b113b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1cc0 0x55f9d05e0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.314+0000 7f165b113b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d71a0 0x55f9d06421d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.314+0000 7f165b113b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d0642710 con 0x55f9d05d1300 2023-09-10T01:05:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.314+0000 7f165b113b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d0642850 con 0x55f9d05d71a0 2023-09-10T01:05:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.314+0000 7f165b113b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d0642990 con 0x55f9d05d1cc0 2023-09-10T01:05:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.314+0000 7f165b113b80 10 client.0 _ll_register_callbacks cb 0x7f163c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:05:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.314+0000 7f1647fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1cc0 0x55f9d05e0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.314+0000 7f164cce9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d1300 0x55f9d05e0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.314+0000 7f1647fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1cc0 0x55f9d05e0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34186/0 (socket says 172.21.15.44:34186) 2023-09-10T01:05:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.314+0000 7f1647fff700 1 -- 172.21.15.44:0/2627783985 learned_addr learned my addr 172.21.15.44:0/2627783985 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.314+0000 7f164d4ea700 1 --2- 172.21.15.44:0/2627783985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d71a0 0x55f9d06421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.314+0000 7f1647fff700 1 -- 172.21.15.44:0/2627783985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d71a0 msgr2=0x55f9d06421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.314+0000 7f1647fff700 1 --2- 172.21.15.44:0/2627783985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d71a0 0x55f9d06421d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.314+0000 7f1647fff700 1 -- 172.21.15.44:0/2627783985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d1300 msgr2=0x55f9d05e0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.315+0000 7f1647fff700 1 --2- 172.21.15.44:0/2627783985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d1300 0x55f9d05e0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.315+0000 7f1647fff700 1 -- 172.21.15.44:0/2627783985 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1634009cd0 con 0x55f9d05d1cc0 2023-09-10T01:05:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.315+0000 7f1647fff700 1 --2- 172.21.15.44:0/2627783985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1cc0 0x55f9d05e0c40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55f9d05cd140 tx=0x7f1634017e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=911e682c821b42d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.315+0000 7f1627fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:05:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.315+0000 7f1627fff700 1 -- 172.21.15.44:0/2627783985 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1634030040 con 0x55f9d05d1cc0 2023-09-10T01:05:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.315+0000 7f1627fff700 1 -- 172.21.15.44:0/2627783985 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1634026850 con 0x55f9d05d1cc0 2023-09-10T01:05:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.315+0000 7f165b113b80 1 -- 172.21.15.44:0/2627783985 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f163c099da0 con 0x55f9d05d1cc0 2023-09-10T01:05:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.315+0000 7f1627fff700 1 -- 172.21.15.44:0/2627783985 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1634026a10 con 0x55f9d05d1cc0 2023-09-10T01:05:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.315+0000 7f165b113b80 10 client.28252 fetch_fsmap learned FSMap version 99 2023-09-10T01:05:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.315+0000 7f165b113b80 1 -- 172.21.15.44:0/2627783985 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=4+}) v3 -- 0x55f9d034e250 con 0x55f9d05d1cc0 2023-09-10T01:05:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.315+0000 7f1627fff700 1 -- 172.21.15.44:0/2627783985 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f163c099da0 con 0x55f9d05d1cc0 2023-09-10T01:05:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.316+0000 7f1627fff700 1 -- 172.21.15.44:0/2627783985 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 99) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f163403c070 con 0x55f9d05d1cc0 2023-09-10T01:05:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.316+0000 7f165b113b80 10 client.28252 fetch_fsmap finished waiting for FSMap version 99 2023-09-10T01:05:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.316+0000 7f165b113b80 10 client.28252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:05:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.316+0000 7f165b113b80 20 client.28252 populate_metadata read hostname 'smithi044' 2023-09-10T01:05:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.316+0000 7f165b113b80 1 --2- 172.21.15.44:0/2627783985 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f9d0644080 0x55f9d06464a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.316+0000 7f165b113b80 4 client.28252 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:05:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.316+0000 7f165b113b80 1 -- 172.21.15.44:0/2627783985 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f9d051c8c0 con 0x55f9d0644080 2023-09-10T01:05:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.316+0000 7f164cce9700 1 --2- 172.21.15.44:0/2627783985 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f9d0644080 0x55f9d06464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.316+0000 7f1627fff700 10 client.28252 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:02.316+0000 7f164cce9700 1 --2- 172.21.15.44:0/2627783985 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f9d0644080 0x55f9d06464a0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ce57b160852981f server_cookie=e462451bcdc8c8d9 in_seq=0 out_seq=0 2023-09-10T01:05:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:02 smithi131 ceph-mon[103713]: pgmap v773: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T01:05:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:02 smithi044 ceph-mon[98676]: pgmap v773: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T01:05:03.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:02 smithi099 ceph-mon[102692]: pgmap v773: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T01:05:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.431+0000 7f1627fff700 1 -- 172.21.15.44:0/2627783985 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f9d051c8c0 con 0x55f9d0644080 2023-09-10T01:05:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.431+0000 7f1627fff700 10 client.28252 handle_command_reply: tid=0 2023-09-10T01:05:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.431+0000 7f165b113b80 1 client.28252 shutdown 2023-09-10T01:05:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f165b113b80 1 -- 172.21.15.44:0/2627783985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1cc0 msgr2=0x55f9d05e0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f165b113b80 1 --2- 172.21.15.44:0/2627783985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1cc0 0x55f9d05e0c40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55f9d05cd140 tx=0x7f1634017e50 comp rx=0 tx=0).stop 2023-09-10T01:05:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f165b113b80 1 -- 172.21.15.44:0/2627783985 shutdown_connections 2023-09-10T01:05:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f165b113b80 1 --2- 172.21.15.44:0/2627783985 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f9d0644080 0x55f9d06464a0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f165b113b80 1 --2- 172.21.15.44:0/2627783985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9d05d71a0 0x55f9d06421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f165b113b80 1 --2- 172.21.15.44:0/2627783985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d05d1300 0x55f9d05e0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f165b113b80 1 --2- 172.21.15.44:0/2627783985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d05d1cc0 0x55f9d05e0c40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f1627fff700 0 client.28252 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f165b113b80 1 -- 172.21.15.44:0/2627783985 >> 172.21.15.44:0/2627783985 conn(0x55f9d0546030 msgr2=0x55f9d05d60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f165b113b80 1 -- 172.21.15.44:0/2627783985 shutdown_connections 2023-09-10T01:05:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f165b113b80 1 -- 172.21.15.44:0/2627783985 wait complete. 2023-09-10T01:05:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f165b113b80 20 client.28252 trim_cache size 0 max 16384 2023-09-10T01:05:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:05:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:05:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:05:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:05:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f164ef4d700 1 -- 172.21.15.44:0/2610769492 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f16380a0830 msgr2=0x7f16380a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f164ef4d700 1 --2- 172.21.15.44:0/2610769492 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f16380a0830 0x7f16380a2cf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1648071f60 tx=0x7f163400b580 comp rx=0 tx=0).stop 2023-09-10T01:05:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f164ef4d700 1 -- 172.21.15.44:0/2610769492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1648108fa0 msgr2=0x7f16481a2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f164ef4d700 1 --2- 172.21.15.44:0/2610769492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1648108fa0 0x7f16481a2c30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f163c00eb30 tx=0x7f163c00ee40 comp rx=0 tx=0).stop 2023-09-10T01:05:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f164ef4d700 1 -- 172.21.15.44:0/2610769492 shutdown_connections 2023-09-10T01:05:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f164ef4d700 1 --2- 172.21.15.44:0/2610769492 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f16380a0830 0x7f16380a2cf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f164ef4d700 1 --2- 172.21.15.44:0/2610769492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1648112550 0x7f164819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f164ef4d700 1 --2- 172.21.15.44:0/2610769492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1648108fa0 0x7f16481a2c30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f164ef4d700 1 --2- 172.21.15.44:0/2610769492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16481098c0 0x7f16481a3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.432+0000 7f164ef4d700 1 -- 172.21.15.44:0/2610769492 >> 172.21.15.44:0/2610769492 conn(0x7f164806c6c0 msgr2=0x7f164806f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.433+0000 7f164ef4d700 1 -- 172.21.15.44:0/2610769492 shutdown_connections 2023-09-10T01:05:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:03.433+0000 7f164ef4d700 1 -- 172.21.15.44:0/2610769492 wait complete. 2023-09-10T01:05:04.546 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: *** Caught signal (Segmentation fault) ** 2023-09-10T01:05:04.546 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: in thread 7f34d51a2700 thread_name:md_log_replay 2023-09-10T01:05:04.546 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:05:04.546 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f34e5d51cf0] 2023-09-10T01:05:04.547 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x559236f802f2] 2023-09-10T01:05:04.547 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559236f82543] 2023-09-10T01:05:04.547 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 4: (MDLog::_replay_thread()+0x647) [0x559236febe97] 2023-09-10T01:05:04.547 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559236ff5381] 2023-09-10T01:05:04.548 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f34e5d471ca] 2023-09-10T01:05:04.548 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 7: clone() 2023-09-10T01:05:04.548 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 2023-09-10T01:05:04.105+0000 7f34d51a2700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:05:04.548 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: in thread 7f34d51a2700 thread_name:md_log_replay 2023-09-10T01:05:04.548 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 2023-09-10T01:05:04.548 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:05:04.549 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f34e5d51cf0] 2023-09-10T01:05:04.549 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x559236f802f2] 2023-09-10T01:05:04.549 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559236f82543] 2023-09-10T01:05:04.549 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 4: (MDLog::_replay_thread()+0x647) [0x559236febe97] 2023-09-10T01:05:04.549 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559236ff5381] 2023-09-10T01:05:04.550 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f34e5d471ca] 2023-09-10T01:05:04.550 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 7: clone() 2023-09-10T01:05:04.550 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:05:04.550 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 2023-09-10T01:05:04.550 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 0> 2023-09-10T01:05:04.105+0000 7f34d51a2700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:05:04.551 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: in thread 7f34d51a2700 thread_name:md_log_replay 2023-09-10T01:05:04.551 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 2023-09-10T01:05:04.551 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:05:04.551 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f34e5d51cf0] 2023-09-10T01:05:04.551 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x559236f802f2] 2023-09-10T01:05:04.552 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559236f82543] 2023-09-10T01:05:04.552 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 4: (MDLog::_replay_thread()+0x647) [0x559236febe97] 2023-09-10T01:05:04.552 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559236ff5381] 2023-09-10T01:05:04.552 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f34e5d471ca] 2023-09-10T01:05:04.552 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 7: clone() 2023-09-10T01:05:04.553 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:05:04.553 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 2023-09-10T01:05:04.553 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 0> 2023-09-10T01:05:04.105+0000 7f34d51a2700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:05:04.553 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: in thread 7f34d51a2700 thread_name:md_log_replay 2023-09-10T01:05:04.553 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 2023-09-10T01:05:04.554 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:05:04.554 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f34e5d51cf0] 2023-09-10T01:05:04.554 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x559236f802f2] 2023-09-10T01:05:04.554 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559236f82543] 2023-09-10T01:05:04.554 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 4: (MDLog::_replay_thread()+0x647) [0x559236febe97] 2023-09-10T01:05:04.555 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559236ff5381] 2023-09-10T01:05:04.555 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f34e5d471ca] 2023-09-10T01:05:04.555 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 7: clone() 2023-09-10T01:05:04.555 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:05:04.555 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[149244]: 2023-09-10T01:05:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:04 smithi131 ceph-mon[103713]: pgmap v774: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 973 KiB/s wr, 138 op/s 2023-09-10T01:05:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:04 smithi044 ceph-mon[98676]: pgmap v774: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 973 KiB/s wr, 138 op/s 2023-09-10T01:05:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:04 smithi099 ceph-mon[102692]: pgmap v774: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 973 KiB/s wr, 138 op/s 2023-09-10T01:05:06.295 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:05 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Main process exited, code=exited, status=139/n/a 2023-09-10T01:05:06.296 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:06 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Failed with result 'exit-code'. 2023-09-10T01:05:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:06 smithi131 ceph-mon[103713]: pgmap v775: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 244 op/s 2023-09-10T01:05:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:06 smithi044 ceph-mon[98676]: pgmap v775: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 244 op/s 2023-09-10T01:05:07.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:06 smithi099 ceph-mon[102692]: pgmap v775: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 244 op/s 2023-09-10T01:05:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:08 smithi131 ceph-mon[103713]: pgmap v776: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 173 op/s 2023-09-10T01:05:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:08 smithi044 ceph-mon[98676]: pgmap v776: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 173 op/s 2023-09-10T01:05:09.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:08 smithi099 ceph-mon[102692]: pgmap v776: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 173 op/s 2023-09-10T01:05:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:10 smithi131 ceph-mon[103713]: pgmap v777: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 173 op/s 2023-09-10T01:05:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:10 smithi044 ceph-mon[98676]: pgmap v777: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 173 op/s 2023-09-10T01:05:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:10 smithi099 ceph-mon[102692]: pgmap v777: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 173 op/s 2023-09-10T01:05:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:12 smithi131 ceph-mon[103713]: pgmap v778: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 243 op/s 2023-09-10T01:05:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:12 smithi044 ceph-mon[98676]: pgmap v778: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 243 op/s 2023-09-10T01:05:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:12 smithi099 ceph-mon[102692]: pgmap v778: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 243 op/s 2023-09-10T01:05:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:14 smithi131 ceph-mon[103713]: pgmap v779: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 176 op/s 2023-09-10T01:05:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:14 smithi044 ceph-mon[98676]: pgmap v779: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 176 op/s 2023-09-10T01:05:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:14 smithi099 ceph-mon[102692]: pgmap v779: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 176 op/s 2023-09-10T01:05:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.567+0000 7f7711fe4700 1 -- 172.21.15.44:0/3043373467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f770c100f50 msgr2=0x7f770c10d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.567+0000 7f7711fe4700 1 --2- 172.21.15.44:0/3043373467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f770c100f50 0x7f770c10d410 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f770400b0d0 tx=0x7f770400b3e0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.567+0000 7f7711fe4700 1 -- 172.21.15.44:0/3043373467 shutdown_connections 2023-09-10T01:05:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.567+0000 7f7711fe4700 1 --2- 172.21.15.44:0/3043373467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f770c100f50 0x7f770c10d410 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.567+0000 7f7711fe4700 1 --2- 172.21.15.44:0/3043373467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f770c106650 0x7f770c106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.567+0000 7f7711fe4700 1 --2- 172.21.15.44:0/3043373467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c100590 0x7f770c100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.567+0000 7f7711fe4700 1 -- 172.21.15.44:0/3043373467 >> 172.21.15.44:0/3043373467 conn(0x7f770c0fc060 msgr2=0x7f770c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.567+0000 7f7711fe4700 1 -- 172.21.15.44:0/3043373467 shutdown_connections 2023-09-10T01:05:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.567+0000 7f7711fe4700 1 -- 172.21.15.44:0/3043373467 wait complete. 2023-09-10T01:05:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.568+0000 7f7711fe4700 1 Processor -- start 2023-09-10T01:05:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.568+0000 7f7711fe4700 1 -- start start 2023-09-10T01:05:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.568+0000 7f7711fe4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c100590 0x7f770c072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.568+0000 7f7711fe4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f770c100f50 0x7f770c073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.568+0000 7f7711fe4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f770c106650 0x7f770c1a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.568+0000 7f7711fe4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c1a4d60 con 0x7f770c100f50 2023-09-10T01:05:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.568+0000 7f7711fe4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c1a4ea0 con 0x7f770c106650 2023-09-10T01:05:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.568+0000 7f7711fe4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c1a5010 con 0x7f770c100590 2023-09-10T01:05:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.568+0000 7f7703fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f770c100f50 0x7f770c073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.568+0000 7f770b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c100590 0x7f770c072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.568+0000 7f7703fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f770c100f50 0x7f770c073020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55550/0 (socket says 172.21.15.44:55550) 2023-09-10T01:05:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.568+0000 7f7703fff700 1 -- 172.21.15.44:0/3775857135 learned_addr learned my addr 172.21.15.44:0/3775857135 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.568+0000 7f770bfff700 1 --2- 172.21.15.44:0/3775857135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f770c106650 0x7f770c1a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.569+0000 7f770bfff700 1 -- 172.21.15.44:0/3775857135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c100590 msgr2=0x7f770c072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.569+0000 7f770bfff700 1 --2- 172.21.15.44:0/3775857135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c100590 0x7f770c072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.569+0000 7f770bfff700 1 -- 172.21.15.44:0/3775857135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f770c100f50 msgr2=0x7f770c073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.569+0000 7f770bfff700 1 --2- 172.21.15.44:0/3775857135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f770c100f50 0x7f770c073020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.569+0000 7f770bfff700 1 -- 172.21.15.44:0/3775857135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76fc009520 con 0x7f770c106650 2023-09-10T01:05:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.569+0000 7f7703fff700 1 --2- 172.21.15.44:0/3775857135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f770c100f50 0x7f770c073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:05:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.569+0000 7f770bfff700 1 --2- 172.21.15.44:0/3775857135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f770c106650 0x7f770c1a4820 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f770400b770 tx=0x7f7704008e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.569+0000 7f77097fa700 1 -- 172.21.15.44:0/3775857135 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770400e040 con 0x7f770c106650 2023-09-10T01:05:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.569+0000 7f7711fe4700 1 -- 172.21.15.44:0/3775857135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7704009d00 con 0x7f770c106650 2023-09-10T01:05:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.569+0000 7f77097fa700 1 -- 172.21.15.44:0/3775857135 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f770401d900 con 0x7f770c106650 2023-09-10T01:05:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.569+0000 7f77097fa700 1 -- 172.21.15.44:0/3775857135 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770401b7e0 con 0x7f770c106650 2023-09-10T01:05:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.569+0000 7f7711fe4700 1 -- 172.21.15.44:0/3775857135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770c1a55d0 con 0x7f770c106650 2023-09-10T01:05:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.571+0000 7f77097fa700 1 -- 172.21.15.44:0/3775857135 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f770401da70 con 0x7f770c106650 2023-09-10T01:05:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.572+0000 7f77097fa700 1 --2- 172.21.15.44:0/3775857135 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76f80a07b0 0x7f76f80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.572+0000 7f770b7fe700 1 --2- 172.21.15.44:0/3775857135 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76f80a07b0 0x7f76f80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.572+0000 7f77097fa700 1 -- 172.21.15.44:0/3775857135 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f77040cd670 con 0x7f770c106650 2023-09-10T01:05:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.572+0000 7f770b7fe700 1 --2- 172.21.15.44:0/3775857135 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76f80a07b0 0x7f76f80a2c70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009df0 tx=0x7f76fc009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f771e1aab80 1 Processor -- start 2023-09-10T01:05:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f771e1aab80 1 -- start start 2023-09-10T01:05:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f771e1aab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 0x560f2d918580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f771e1aab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d912780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f771e1aab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 0x560f2d91ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f771e1aab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2d86b6d0 con 0x560f2d912300 2023-09-10T01:05:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f771e1aab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2d868ff0 con 0x560f2d912cc0 2023-09-10T01:05:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f771e1aab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2d8648e0 con 0x560f2d9181a0 2023-09-10T01:05:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f7703fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d912780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f7703fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d912780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55558/0 (socket says 172.21.15.44:55558) 2023-09-10T01:05:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f7703fff700 1 -- 172.21.15.44:0/3469964539 learned_addr learned my addr 172.21.15.44:0/3469964539 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f770b7fe700 1 --2- 172.21.15.44:0/3469964539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 0x560f2d918580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f770bfff700 1 --2- 172.21.15.44:0/3469964539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 0x560f2d91ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f7703fff700 1 -- 172.21.15.44:0/3469964539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 msgr2=0x560f2d918580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f7703fff700 1 --2- 172.21.15.44:0/3469964539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 0x560f2d918580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f7703fff700 1 -- 172.21.15.44:0/3469964539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 msgr2=0x560f2d91ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f7703fff700 1 --2- 172.21.15.44:0/3469964539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 0x560f2d91ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.573+0000 7f7703fff700 1 -- 172.21.15.44:0/3469964539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f2d690d80 con 0x560f2d912300 2023-09-10T01:05:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.574+0000 7f7703fff700 1 --2- 172.21.15.44:0/3469964539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d912780 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f76f4011680 tx=0x7f76f4011990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6dd946e3f3f790 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.574+0000 7f7701ffb700 1 -- 172.21.15.44:0/3469964539 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4004030 con 0x560f2d912300 2023-09-10T01:05:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.574+0000 7f7701ffb700 1 -- 172.21.15.44:0/3469964539 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76f4003680 con 0x560f2d912300 2023-09-10T01:05:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.574+0000 7f7701ffb700 1 -- 172.21.15.44:0/3469964539 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f40196b0 con 0x560f2d912300 2023-09-10T01:05:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.575+0000 7f771e1aab80 1 -- 172.21.15.44:0/3469964539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 msgr2=0x560f2d912780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.575+0000 7f771e1aab80 1 --2- 172.21.15.44:0/3469964539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d912780 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f76f4011680 tx=0x7f76f4011990 comp rx=0 tx=0).stop 2023-09-10T01:05:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.575+0000 7f771e1aab80 1 -- 172.21.15.44:0/3469964539 shutdown_connections 2023-09-10T01:05:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.575+0000 7f771e1aab80 1 --2- 172.21.15.44:0/3469964539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 0x560f2d91ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.575+0000 7f771e1aab80 1 --2- 172.21.15.44:0/3469964539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 0x560f2d918580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.575+0000 7f771e1aab80 1 --2- 172.21.15.44:0/3469964539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d912780 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.575+0000 7f771e1aab80 1 -- 172.21.15.44:0/3469964539 >> 172.21.15.44:0/3469964539 conn(0x560f2d887030 msgr2=0x560f2d90d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.575+0000 7f771e1aab80 1 -- 172.21.15.44:0/3469964539 shutdown_connections 2023-09-10T01:05:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.575+0000 7f771e1aab80 1 -- 172.21.15.44:0/3469964539 wait complete. 2023-09-10T01:05:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.575+0000 7f771e1aab80 1 -- start start 2023-09-10T01:05:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.575+0000 7f771e1aab80 1 Processor -- start 2023-09-10T01:05:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f771e1aab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d91edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f771e1aab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 0x560f2d91c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f771e1aab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 0x560f2d91c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f771e1aab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2d802b10 con 0x560f2d912300 2023-09-10T01:05:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f771e1aab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2d802810 con 0x560f2d912cc0 2023-09-10T01:05:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f771e1aab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2d6e68c0 con 0x560f2d9181a0 2023-09-10T01:05:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f771e1aab80 10 client.0 _ll_register_callbacks cb 0x560f2d74ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:05:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f770b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d91edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f7703fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 0x560f2d91c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f770bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 0x560f2d91c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f7703fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 0x560f2d91c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57304/0 (socket says 172.21.15.44:57304) 2023-09-10T01:05:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f770b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d91edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55562/0 (socket says 172.21.15.44:55562) 2023-09-10T01:05:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f7703fff700 1 -- 172.21.15.44:0/921637876 learned_addr learned my addr 172.21.15.44:0/921637876 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f770bfff700 1 --2- 172.21.15.44:0/921637876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 0x560f2d91c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34216/0 (socket says 172.21.15.44:34216) 2023-09-10T01:05:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f7703fff700 1 -- 172.21.15.44:0/921637876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 msgr2=0x560f2d91c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f7703fff700 1 --2- 172.21.15.44:0/921637876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 0x560f2d91c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f7703fff700 1 -- 172.21.15.44:0/921637876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 msgr2=0x560f2d91edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f7703fff700 1 --2- 172.21.15.44:0/921637876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d91edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f7703fff700 1 -- 172.21.15.44:0/921637876 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f4009e10 con 0x560f2d912cc0 2023-09-10T01:05:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f7703fff700 1 --2- 172.21.15.44:0/921637876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 0x560f2d91c070 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f76f4011650 tx=0x7f76f4004130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5b5672e3a3f1476 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f76eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:05:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f76eaffd700 1 -- 172.21.15.44:0/921637876 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4005ce0 con 0x560f2d912cc0 2023-09-10T01:05:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f771e1aab80 1 -- 172.21.15.44:0/921637876 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f2d7bc490 con 0x560f2d912cc0 2023-09-10T01:05:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.576+0000 7f76eaffd700 1 -- 172.21.15.44:0/921637876 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f76f4004530 con 0x560f2d912cc0 2023-09-10T01:05:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.577+0000 7f76eaffd700 1 -- 172.21.15.44:0/921637876 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76f4005050 con 0x560f2d912cc0 2023-09-10T01:05:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.577+0000 7f771e1aab80 10 client.28325 fetch_fsmap learned FSMap version 99 2023-09-10T01:05:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.577+0000 7f771e1aab80 1 -- 172.21.15.44:0/921637876 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=4+}) v3 -- 0x560f2d679c00 con 0x560f2d912cc0 2023-09-10T01:05:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.577+0000 7f76eaffd700 1 -- 172.21.15.44:0/921637876 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560f2d7bc490 con 0x560f2d912cc0 2023-09-10T01:05:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.577+0000 7f76eaffd700 1 -- 172.21.15.44:0/921637876 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 99) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f76f4022480 con 0x560f2d912cc0 2023-09-10T01:05:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.577+0000 7f771e1aab80 10 client.28325 fetch_fsmap finished waiting for FSMap version 99 2023-09-10T01:05:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.577+0000 7f771e1aab80 10 client.28325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:05:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.577+0000 7f771e1aab80 20 client.28325 populate_metadata read hostname 'smithi044' 2023-09-10T01:05:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.577+0000 7f771e1aab80 1 --2- 172.21.15.44:0/921637876 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560f2d92c270 0x560f2d92e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.577+0000 7f771e1aab80 4 client.28325 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:05:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.577+0000 7f771e1aab80 1 -- 172.21.15.44:0/921637876 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f2d879840 con 0x560f2d92c270 2023-09-10T01:05:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.577+0000 7f770b7fe700 1 --2- 172.21.15.44:0/921637876 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560f2d92c270 0x560f2d92e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.578+0000 7f770b7fe700 1 --2- 172.21.15.44:0/921637876 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560f2d92c270 0x560f2d92e690 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c4fe92e536cf41b server_cookie=724f9b83c7993da8 in_seq=0 out_seq=0 2023-09-10T01:05:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.578+0000 7f76eaffd700 10 client.28325 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.578+0000 7f76eaffd700 1 -- 172.21.15.44:0/921637876 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560f2d879840 con 0x560f2d92c270 2023-09-10T01:05:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.578+0000 7f76eaffd700 10 client.28325 handle_command_reply: tid=0 2023-09-10T01:05:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.578+0000 7f771e1aab80 1 client.28325 shutdown 2023-09-10T01:05:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.578+0000 7f771e1aab80 1 -- 172.21.15.44:0/921637876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 msgr2=0x560f2d91c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.578+0000 7f771e1aab80 1 --2- 172.21.15.44:0/921637876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 0x560f2d91c070 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f76f4011650 tx=0x7f76f4004130 comp rx=0 tx=0).stop 2023-09-10T01:05:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.579+0000 7f771e1aab80 1 -- 172.21.15.44:0/921637876 shutdown_connections 2023-09-10T01:05:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.579+0000 7f771e1aab80 1 --2- 172.21.15.44:0/921637876 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560f2d92c270 0x560f2d92e690 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.579+0000 7f771e1aab80 1 --2- 172.21.15.44:0/921637876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 0x560f2d91c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.579+0000 7f76eaffd700 0 client.28325 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.579+0000 7f771e1aab80 1 --2- 172.21.15.44:0/921637876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d91edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.579+0000 7f771e1aab80 1 --2- 172.21.15.44:0/921637876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 0x560f2d91c070 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.579+0000 7f771e1aab80 1 -- 172.21.15.44:0/921637876 >> 172.21.15.44:0/921637876 conn(0x560f2d887030 msgr2=0x560f2d90c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.579+0000 7f771e1aab80 1 -- 172.21.15.44:0/921637876 shutdown_connections 2023-09-10T01:05:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.579+0000 7f771e1aab80 1 -- 172.21.15.44:0/921637876 wait complete. 2023-09-10T01:05:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.579+0000 7f771e1aab80 20 client.28325 trim_cache size 0 max 16384 2023-09-10T01:05:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.594+0000 7f771e1aab80 1 Processor -- start 2023-09-10T01:05:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f771e1aab80 1 -- start start 2023-09-10T01:05:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f771e1aab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d912300 0x560f2d923520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f771e1aab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912cc0 0x560f2d91e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f771e1aab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d9181a0 0x560f2d91edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f771e1aab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2d923b20 con 0x560f2d912cc0 2023-09-10T01:05:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f771e1aab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2d916ca0 con 0x560f2d9181a0 2023-09-10T01:05:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f771e1aab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2d9126e0 con 0x560f2d912300 2023-09-10T01:05:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f7703fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912cc0 0x560f2d91e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f770b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d912300 0x560f2d923520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f770bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d9181a0 0x560f2d91edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f7703fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912cc0 0x560f2d91e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55572/0 (socket says 172.21.15.44:55572) 2023-09-10T01:05:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f770b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d912300 0x560f2d923520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34220/0 (socket says 172.21.15.44:34220) 2023-09-10T01:05:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f7703fff700 1 -- 172.21.15.44:0/3047830975 learned_addr learned my addr 172.21.15.44:0/3047830975 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f7703fff700 1 -- 172.21.15.44:0/3047830975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d912300 msgr2=0x560f2d923520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f7703fff700 1 --2- 172.21.15.44:0/3047830975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d912300 0x560f2d923520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f7703fff700 1 -- 172.21.15.44:0/3047830975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d9181a0 msgr2=0x560f2d91edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f7703fff700 1 --2- 172.21.15.44:0/3047830975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d9181a0 0x560f2d91edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f7703fff700 1 -- 172.21.15.44:0/3047830975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f2d6b71f0 con 0x560f2d912cc0 2023-09-10T01:05:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f770bfff700 1 --2- 172.21.15.44:0/3047830975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d9181a0 0x560f2d91edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:05:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.595+0000 7f7703fff700 1 --2- 172.21.15.44:0/3047830975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912cc0 0x560f2d91e880 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f76f40115a0 tx=0x7f76f4005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d58a65df8fa057 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.596+0000 7f77027fc700 1 -- 172.21.15.44:0/3047830975 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4011d90 con 0x560f2d912cc0 2023-09-10T01:05:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.596+0000 7f77027fc700 1 -- 172.21.15.44:0/3047830975 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76f40041a0 con 0x560f2d912cc0 2023-09-10T01:05:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.596+0000 7f77027fc700 1 -- 172.21.15.44:0/3047830975 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4004530 con 0x560f2d912cc0 2023-09-10T01:05:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.596+0000 7f771e1aab80 1 -- 172.21.15.44:0/3047830975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912cc0 msgr2=0x560f2d91e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.596+0000 7f771e1aab80 1 --2- 172.21.15.44:0/3047830975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912cc0 0x560f2d91e880 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f76f40115a0 tx=0x7f76f4005a40 comp rx=0 tx=0).stop 2023-09-10T01:05:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.596+0000 7f771e1aab80 1 -- 172.21.15.44:0/3047830975 shutdown_connections 2023-09-10T01:05:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.596+0000 7f771e1aab80 1 --2- 172.21.15.44:0/3047830975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d9181a0 0x560f2d91edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.596+0000 7f771e1aab80 1 --2- 172.21.15.44:0/3047830975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d912300 0x560f2d923520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.596+0000 7f771e1aab80 1 --2- 172.21.15.44:0/3047830975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912cc0 0x560f2d91e880 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.596+0000 7f771e1aab80 1 -- 172.21.15.44:0/3047830975 >> 172.21.15.44:0/3047830975 conn(0x560f2d887030 msgr2=0x560f2d916190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.597+0000 7f771e1aab80 1 -- 172.21.15.44:0/3047830975 shutdown_connections 2023-09-10T01:05:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.597+0000 7f771e1aab80 1 -- 172.21.15.44:0/3047830975 wait complete. 2023-09-10T01:05:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.597+0000 7f771e1aab80 1 -- start start 2023-09-10T01:05:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.597+0000 7f771e1aab80 1 Processor -- start 2023-09-10T01:05:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.597+0000 7f771e1aab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d923530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.597+0000 7f771e1aab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 0x560f2d90bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.597+0000 7f771e1aab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 0x560f2d90c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.597+0000 7f771e1aab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2d90c990 con 0x560f2d912300 2023-09-10T01:05:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.597+0000 7f771e1aab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2d90cb00 con 0x560f2d912cc0 2023-09-10T01:05:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.597+0000 7f771e1aab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2d90ccc0 con 0x560f2d9181a0 2023-09-10T01:05:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.597+0000 7f771e1aab80 10 client.0 _ll_register_callbacks cb 0x7f77040963d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:05:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.597+0000 7f770b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d923530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f770b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d923530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55576/0 (socket says 172.21.15.44:55576) 2023-09-10T01:05:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f7703fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 0x560f2d90bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f770b7fe700 1 -- 172.21.15.44:0/3375071219 learned_addr learned my addr 172.21.15.44:0/3375071219 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f770bfff700 1 --2- 172.21.15.44:0/3375071219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 0x560f2d90c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f770b7fe700 1 -- 172.21.15.44:0/3375071219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 msgr2=0x560f2d90c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f770b7fe700 1 --2- 172.21.15.44:0/3375071219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 0x560f2d90c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f770b7fe700 1 -- 172.21.15.44:0/3375071219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 msgr2=0x560f2d90bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f770b7fe700 1 --2- 172.21.15.44:0/3375071219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 0x560f2d90bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f770b7fe700 1 -- 172.21.15.44:0/3375071219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770c1a55d0 con 0x560f2d912300 2023-09-10T01:05:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f770bfff700 1 --2- 172.21.15.44:0/3375071219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 0x560f2d90c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:05:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f770b7fe700 1 --2- 172.21.15.44:0/3375071219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d923530 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x560f2d92ef30 tx=0x7f76fc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac8868efad4ceb5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f76eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:05:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f76eaffd700 1 -- 172.21.15.44:0/3375071219 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc022040 con 0x560f2d912300 2023-09-10T01:05:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f76eaffd700 1 -- 172.21.15.44:0/3375071219 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f76fc00e8d0 con 0x560f2d912300 2023-09-10T01:05:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f771e1aab80 1 -- 172.21.15.44:0/3375071219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f2d85d8c0 con 0x560f2d912300 2023-09-10T01:05:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f76eaffd700 1 -- 172.21.15.44:0/3375071219 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76fc00ea90 con 0x560f2d912300 2023-09-10T01:05:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f771e1aab80 10 client.18678 fetch_fsmap learned FSMap version 99 2023-09-10T01:05:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.598+0000 7f771e1aab80 1 -- 172.21.15.44:0/3375071219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=4+}) v3 -- 0x560f2d679c00 con 0x560f2d912300 2023-09-10T01:05:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.599+0000 7f76eaffd700 1 -- 172.21.15.44:0/3375071219 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560f2d85d8c0 con 0x560f2d912300 2023-09-10T01:05:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.599+0000 7f76eaffd700 1 -- 172.21.15.44:0/3375071219 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 99) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f76fc034c00 con 0x560f2d912300 2023-09-10T01:05:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.599+0000 7f771e1aab80 10 client.18678 fetch_fsmap finished waiting for FSMap version 99 2023-09-10T01:05:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.599+0000 7f771e1aab80 10 client.18678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:05:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.599+0000 7f771e1aab80 20 client.18678 populate_metadata read hostname 'smithi044' 2023-09-10T01:05:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.599+0000 7f771e1aab80 1 --2- 172.21.15.44:0/3375071219 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560f2d981560 0x560f2d983980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.599+0000 7f771e1aab80 4 client.18678 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:05:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.599+0000 7f771e1aab80 1 -- 172.21.15.44:0/3375071219 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560f2d81e050 con 0x560f2d981560 2023-09-10T01:05:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.599+0000 7f7703fff700 1 --2- 172.21.15.44:0/3375071219 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560f2d981560 0x560f2d983980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.599+0000 7f7703fff700 1 --2- 172.21.15.44:0/3375071219 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560f2d981560 0x560f2d983980 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=903f79112d4f116b server_cookie=957e6fd069c668a8 in_seq=0 out_seq=0 2023-09-10T01:05:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.599+0000 7f76eaffd700 10 client.18678 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.613+0000 7f76eaffd700 1 -- 172.21.15.44:0/3375071219 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560f2d81e050 con 0x560f2d981560 2023-09-10T01:05:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.613+0000 7f76eaffd700 10 client.18678 handle_command_reply: tid=0 2023-09-10T01:05:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.613+0000 7f771e1aab80 1 client.18678 shutdown 2023-09-10T01:05:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.613+0000 7f771e1aab80 1 -- 172.21.15.44:0/3375071219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 msgr2=0x560f2d923530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.613+0000 7f771e1aab80 1 --2- 172.21.15.44:0/3375071219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d923530 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x560f2d92ef30 tx=0x7f76fc0047c0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.613+0000 7f771e1aab80 1 -- 172.21.15.44:0/3375071219 shutdown_connections 2023-09-10T01:05:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.613+0000 7f771e1aab80 1 --2- 172.21.15.44:0/3375071219 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560f2d981560 0x560f2d983980 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.613+0000 7f771e1aab80 1 --2- 172.21.15.44:0/3375071219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f2d9181a0 0x560f2d90c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.613+0000 7f771e1aab80 1 --2- 172.21.15.44:0/3375071219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2d912300 0x560f2d923530 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.613+0000 7f76eaffd700 0 client.18678 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.613+0000 7f771e1aab80 1 --2- 172.21.15.44:0/3375071219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d912cc0 0x560f2d90bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.613+0000 7f771e1aab80 1 -- 172.21.15.44:0/3375071219 >> 172.21.15.44:0/3375071219 conn(0x560f2d887030 msgr2=0x560f2d91c010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.613+0000 7f771e1aab80 1 -- 172.21.15.44:0/3375071219 shutdown_connections 2023-09-10T01:05:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.613+0000 7f771e1aab80 1 -- 172.21.15.44:0/3375071219 wait complete. 2023-09-10T01:05:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.613+0000 7f771e1aab80 20 client.18678 trim_cache size 0 max 16384 2023-09-10T01:05:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:05:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:05:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:05:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:05:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.614+0000 7f7711fe4700 1 -- 172.21.15.44:0/3775857135 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76f80a07b0 msgr2=0x7f76f80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.614+0000 7f7711fe4700 1 --2- 172.21.15.44:0/3775857135 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76f80a07b0 0x7f76f80a2c70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009df0 tx=0x7f76fc009260 comp rx=0 tx=0).stop 2023-09-10T01:05:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.614+0000 7f7711fe4700 1 -- 172.21.15.44:0/3775857135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f770c106650 msgr2=0x7f770c1a4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.614+0000 7f7711fe4700 1 --2- 172.21.15.44:0/3775857135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f770c106650 0x7f770c1a4820 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f770400b770 tx=0x7f7704008e50 comp rx=0 tx=0).stop 2023-09-10T01:05:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.614+0000 7f7711fe4700 1 -- 172.21.15.44:0/3775857135 shutdown_connections 2023-09-10T01:05:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.614+0000 7f7711fe4700 1 --2- 172.21.15.44:0/3775857135 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76f80a07b0 0x7f76f80a2c70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.614+0000 7f7711fe4700 1 --2- 172.21.15.44:0/3775857135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f770c106650 0x7f770c1a4820 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.614+0000 7f7711fe4700 1 --2- 172.21.15.44:0/3775857135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c100590 0x7f770c072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.614+0000 7f7711fe4700 1 --2- 172.21.15.44:0/3775857135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f770c100f50 0x7f770c073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.614+0000 7f7711fe4700 1 -- 172.21.15.44:0/3775857135 >> 172.21.15.44:0/3775857135 conn(0x7f770c0fc060 msgr2=0x7f770c0fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.614+0000 7f7711fe4700 1 -- 172.21.15.44:0/3775857135 shutdown_connections 2023-09-10T01:05:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:16.614+0000 7f7711fe4700 1 -- 172.21.15.44:0/3775857135 wait complete. 2023-09-10T01:05:16.637 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:16 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Service RestartSec=10s expired, scheduling restart. 2023-09-10T01:05:16.637 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:16 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Scheduled restart job, restart counter is at 4. 2023-09-10T01:05:16.637 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:16 smithi131 systemd[1]: Stopped Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:05:16.638 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:16 smithi131 systemd[1]: Starting Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:05:16.638 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:16 smithi131 podman[150178]: 2023-09-10T01:05:17.205 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:16 smithi131 ceph-mon[103713]: pgmap v780: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 238 op/s 2023-09-10T01:05:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:16 smithi044 ceph-mon[98676]: pgmap v780: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 238 op/s 2023-09-10T01:05:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:16 smithi099 ceph-mon[102692]: pgmap v780: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 238 op/s 2023-09-10T01:05:17.915 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:17 smithi131 bash[150178]: 0a6ccee473a34e5604f03cf96ac9453ead3c7b8991f1d631b3fe6eb14921caf8 2023-09-10T01:05:17.916 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:17 smithi131 systemd[1]: Started Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:05:17.916 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:17 smithi131 ceph-mds[150218]: -- 172.21.15.131:0/347863708 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559bae4ad1e0 con 0x559bae483000 2023-09-10T01:05:17.916 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:05:17 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: starting mds.c at 2023-09-10T01:05:18.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:17 smithi131 ceph-mon[103713]: daemon mds.c restarted 2023-09-10T01:05:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:17 smithi044 ceph-mon[98676]: daemon mds.c restarted 2023-09-10T01:05:18.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:17 smithi099 ceph-mon[102692]: daemon mds.c restarted 2023-09-10T01:05:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:18 smithi131 ceph-mon[103713]: pgmap v781: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 936 KiB/s wr, 132 op/s 2023-09-10T01:05:19.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:18 smithi131 ceph-mon[103713]: osdmap e132: 12 total, 12 up, 12 in 2023-09-10T01:05:19.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:18 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.131:6836/1255900937,v1:172.21.15.131:6837/1255900937] up:boot 2023-09-10T01:05:19.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:18 smithi131 ceph-mon[103713]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-09-10T01:05:19.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:18 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T01:05:19.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:18 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:18 smithi044 ceph-mon[98676]: pgmap v781: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 936 KiB/s wr, 132 op/s 2023-09-10T01:05:19.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:18 smithi044 ceph-mon[98676]: osdmap e132: 12 total, 12 up, 12 in 2023-09-10T01:05:19.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:18 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.131:6836/1255900937,v1:172.21.15.131:6837/1255900937] up:boot 2023-09-10T01:05:19.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:18 smithi044 ceph-mon[98676]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-09-10T01:05:19.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:18 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T01:05:19.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:18 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:18 smithi099 ceph-mon[102692]: pgmap v781: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 936 KiB/s wr, 132 op/s 2023-09-10T01:05:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:18 smithi099 ceph-mon[102692]: osdmap e132: 12 total, 12 up, 12 in 2023-09-10T01:05:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:18 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.131:6836/1255900937,v1:172.21.15.131:6837/1255900937] up:boot 2023-09-10T01:05:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:18 smithi099 ceph-mon[102692]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-09-10T01:05:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:18 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T01:05:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:18 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:20.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:19 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:20.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:19 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:20.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:19 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:20 smithi131 ceph-mon[103713]: pgmap v783: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:05:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:20 smithi044 ceph-mon[98676]: pgmap v783: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:05:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:20 smithi099 ceph-mon[102692]: pgmap v783: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:05:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:22 smithi131 ceph-mon[103713]: pgmap v784: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 153 op/s 2023-09-10T01:05:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:22 smithi044 ceph-mon[98676]: pgmap v784: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 153 op/s 2023-09-10T01:05:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:22 smithi099 ceph-mon[102692]: pgmap v784: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 153 op/s 2023-09-10T01:05:25.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:24 smithi131 ceph-mon[103713]: pgmap v785: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 153 op/s 2023-09-10T01:05:25.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:24 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:05:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:24 smithi044 ceph-mon[98676]: pgmap v785: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 153 op/s 2023-09-10T01:05:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:24 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:05:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:24 smithi099 ceph-mon[102692]: pgmap v785: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 153 op/s 2023-09-10T01:05:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:24 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:05:26.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:25 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:05:26.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:25 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:05:26.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:25 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:05:26.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:25 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:05:26.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:25 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:05:26.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:25 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:05:26.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:25 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:05:26.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:25 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:05:26.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:25 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:05:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:26 smithi131 ceph-mon[103713]: pgmap v786: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 154 op/s 2023-09-10T01:05:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:26 smithi044 ceph-mon[98676]: pgmap v786: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 154 op/s 2023-09-10T01:05:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:26 smithi099 ceph-mon[102692]: pgmap v786: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 154 op/s 2023-09-10T01:05:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:28 smithi131 ceph-mon[103713]: pgmap v787: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 154 op/s 2023-09-10T01:05:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:28 smithi044 ceph-mon[98676]: pgmap v787: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 154 op/s 2023-09-10T01:05:29.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:28 smithi099 ceph-mon[102692]: pgmap v787: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 154 op/s 2023-09-10T01:05:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.759+0000 7f5005f87700 1 -- 172.21.15.44:0/720544703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50000ff760 msgr2=0x7f50000ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.759+0000 7f5005f87700 1 --2- 172.21.15.44:0/720544703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50000ff760 0x7f50000ffbe0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80099c0 tx=0x7f4fe8009cd0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.759+0000 7f5005f87700 1 -- 172.21.15.44:0/720544703 shutdown_connections 2023-09-10T01:05:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.759+0000 7f5005f87700 1 --2- 172.21.15.44:0/720544703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5000100120 0x7f500010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.759+0000 7f5005f87700 1 --2- 172.21.15.44:0/720544703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50001055b0 0x7f5000105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.759+0000 7f5005f87700 1 --2- 172.21.15.44:0/720544703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50000ff760 0x7f50000ffbe0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.759+0000 7f5005f87700 1 -- 172.21.15.44:0/720544703 >> 172.21.15.44:0/720544703 conn(0x7f5000074f30 msgr2=0x7f5000075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.759+0000 7f5005f87700 1 -- 172.21.15.44:0/720544703 shutdown_connections 2023-09-10T01:05:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.759+0000 7f5005f87700 1 -- 172.21.15.44:0/720544703 wait complete. 2023-09-10T01:05:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.760+0000 7f5005f87700 1 Processor -- start 2023-09-10T01:05:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.760+0000 7f5005f87700 1 -- start start 2023-09-10T01:05:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.760+0000 7f5005f87700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50000ff760 0x7f5000197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.760+0000 7f5005f87700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5000100120 0x7f5000197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.760+0000 7f5005f87700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50001055b0 0x7f50001a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.760+0000 7f5005f87700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50001a2b60 con 0x7f5000100120 2023-09-10T01:05:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.760+0000 7f5005f87700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50001a2cd0 con 0x7f50001055b0 2023-09-10T01:05:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.760+0000 7f5005f87700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50001a2e90 con 0x7f50000ff760 2023-09-10T01:05:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4ffeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5000100120 0x7f5000197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4fff7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50000ff760 0x7f5000197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4ffffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50001055b0 0x7f50001a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4fff7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50000ff760 0x7f5000197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34240/0 (socket says 172.21.15.44:34240) 2023-09-10T01:05:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4ffffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50001055b0 0x7f50001a24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57334/0 (socket says 172.21.15.44:57334) 2023-09-10T01:05:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4ffeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5000100120 0x7f5000197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55592/0 (socket says 172.21.15.44:55592) 2023-09-10T01:05:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4fff7fe700 1 -- 172.21.15.44:0/4198392553 learned_addr learned my addr 172.21.15.44:0/4198392553 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4ffffff700 1 -- 172.21.15.44:0/4198392553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50000ff760 msgr2=0x7f5000197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4ffffff700 1 --2- 172.21.15.44:0/4198392553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50000ff760 0x7f5000197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4ffffff700 1 -- 172.21.15.44:0/4198392553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5000100120 msgr2=0x7f5000197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4ffffff700 1 --2- 172.21.15.44:0/4198392553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5000100120 0x7f5000197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4ffffff700 1 -- 172.21.15.44:0/4198392553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe80095f0 con 0x7f50001055b0 2023-09-10T01:05:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4ffeffd700 1 --2- 172.21.15.44:0/4198392553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5000100120 0x7f5000197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:05:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4ffffff700 1 --2- 172.21.15.44:0/4198392553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50001055b0 0x7f50001a24d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800efc0 tx=0x7f4ff800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4ffcff9700 1 -- 172.21.15.44:0/4198392553 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8018070 con 0x7f50001055b0 2023-09-10T01:05:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f5005f87700 1 -- 172.21.15.44:0/4198392553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50001a3110 con 0x7f50001055b0 2023-09-10T01:05:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4ffcff9700 1 -- 172.21.15.44:0/4198392553 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ff800f040 con 0x7f50001055b0 2023-09-10T01:05:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f4ffcff9700 1 -- 172.21.15.44:0/4198392553 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff800cba0 con 0x7f50001055b0 2023-09-10T01:05:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.761+0000 7f5005f87700 1 -- 172.21.15.44:0/4198392553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50001a35f0 con 0x7f50001055b0 2023-09-10T01:05:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.764+0000 7f4ffcff9700 1 -- 172.21.15.44:0/4198392553 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f4ff8003a00 con 0x7f50001055b0 2023-09-10T01:05:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.765+0000 7f4ffcff9700 1 --2- 172.21.15.44:0/4198392553 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4fec0a07b0 0x7f4fec0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.765+0000 7f4fff7fe700 1 --2- 172.21.15.44:0/4198392553 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4fec0a07b0 0x7f4fec0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.765+0000 7f4ffcff9700 1 -- 172.21.15.44:0/4198392553 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f4ff80cbe20 con 0x7f50001055b0 2023-09-10T01:05:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.765+0000 7f4fff7fe700 1 --2- 172.21.15.44:0/4198392553 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4fec0a07b0 0x7f4fec0a2c70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0007c10 tx=0x7f4ff0007480 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.765+0000 7f501214db80 1 Processor -- start 2023-09-10T01:05:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f501214db80 1 -- start start 2023-09-10T01:05:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f501214db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c65b0 0x55f4834c6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f501214db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c0710 0x55f4834c0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f501214db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f501214db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4834156d0 con 0x55f4834c10d0 2023-09-10T01:05:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f501214db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f483412ff0 con 0x55f4834c0710 2023-09-10T01:05:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f501214db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48340e8e0 con 0x55f4834c65b0 2023-09-10T01:05:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f4ffffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f4ffeffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c0710 0x55f4834c0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f4ffeffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c0710 0x55f4834c0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57340/0 (socket says 172.21.15.44:57340) 2023-09-10T01:05:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f4ffeffd700 1 -- 172.21.15.44:0/2750433156 learned_addr learned my addr 172.21.15.44:0/2750433156 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f4fff7fe700 1 --2- 172.21.15.44:0/2750433156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c65b0 0x55f4834c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f4ffffff700 1 -- 172.21.15.44:0/2750433156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c65b0 msgr2=0x55f4834c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f4ffffff700 1 --2- 172.21.15.44:0/2750433156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c65b0 0x55f4834c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f4ffffff700 1 -- 172.21.15.44:0/2750433156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c0710 msgr2=0x55f4834c0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f4ffffff700 1 --2- 172.21.15.44:0/2750433156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c0710 0x55f4834c0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f4ffffff700 1 -- 172.21.15.44:0/2750433156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f48323ad80 con 0x55f4834c10d0 2023-09-10T01:05:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.766+0000 7f4ffffff700 1 --2- 172.21.15.44:0/2750433156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834cd340 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80c91d0 tx=0x7f4ff8004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada768f75d1dbd5c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.767+0000 7f4ff5ffb700 1 -- 172.21.15.44:0/2750433156 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80996c0 con 0x55f4834c10d0 2023-09-10T01:05:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.767+0000 7f4ff5ffb700 1 -- 172.21.15.44:0/2750433156 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ff8098fc0 con 0x55f4834c10d0 2023-09-10T01:05:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.767+0000 7f4ff5ffb700 1 -- 172.21.15.44:0/2750433156 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8099220 con 0x55f4834c10d0 2023-09-10T01:05:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.767+0000 7f501214db80 1 -- 172.21.15.44:0/2750433156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 msgr2=0x55f4834cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.767+0000 7f501214db80 1 --2- 172.21.15.44:0/2750433156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834cd340 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80c91d0 tx=0x7f4ff8004bb0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.767+0000 7f501214db80 1 -- 172.21.15.44:0/2750433156 shutdown_connections 2023-09-10T01:05:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.767+0000 7f501214db80 1 --2- 172.21.15.44:0/2750433156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834cd340 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.767+0000 7f501214db80 1 --2- 172.21.15.44:0/2750433156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c65b0 0x55f4834c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.767+0000 7f501214db80 1 --2- 172.21.15.44:0/2750433156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c0710 0x55f4834c0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.767+0000 7f501214db80 1 -- 172.21.15.44:0/2750433156 >> 172.21.15.44:0/2750433156 conn(0x55f483431030 msgr2=0x55f4834b7560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.767+0000 7f501214db80 1 -- 172.21.15.44:0/2750433156 shutdown_connections 2023-09-10T01:05:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.767+0000 7f501214db80 1 -- 172.21.15.44:0/2750433156 wait complete. 2023-09-10T01:05:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f501214db80 1 -- start start 2023-09-10T01:05:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f501214db80 1 Processor -- start 2023-09-10T01:05:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f501214db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c0710 0x55f4834cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f501214db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834ca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f501214db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c65b0 0x55f4834caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f501214db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4833acb10 con 0x55f4834c10d0 2023-09-10T01:05:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f501214db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4833ac810 con 0x55f4834c0710 2023-09-10T01:05:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f501214db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4832908c0 con 0x55f4834c65b0 2023-09-10T01:05:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f501214db80 10 client.0 _ll_register_callbacks cb 0x55f4832f6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:05:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f4ffeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f4ffeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834ca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55606/0 (socket says 172.21.15.44:55606) 2023-09-10T01:05:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f4ffeffd700 1 -- 172.21.15.44:0/2035853627 learned_addr learned my addr 172.21.15.44:0/2035853627 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f4ffffff700 1 --2- 172.21.15.44:0/2035853627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c65b0 0x55f4834caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f4fff7fe700 1 --2- 172.21.15.44:0/2035853627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c0710 0x55f4834cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f4ffeffd700 1 -- 172.21.15.44:0/2035853627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c65b0 msgr2=0x55f4834caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f4ffeffd700 1 --2- 172.21.15.44:0/2035853627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c65b0 0x55f4834caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f4ffeffd700 1 -- 172.21.15.44:0/2035853627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c0710 msgr2=0x55f4834cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f4ffeffd700 1 --2- 172.21.15.44:0/2035853627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c0710 0x55f4834cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.768+0000 7f4ffeffd700 1 -- 172.21.15.44:0/2035853627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50001a35f0 con 0x55f4834c10d0 2023-09-10T01:05:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.769+0000 7f4ffeffd700 1 --2- 172.21.15.44:0/2035853627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834ca560 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80098e0 tx=0x7f4fe8005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5943579f4440a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.769+0000 7f4fdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:05:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.769+0000 7f4fdeffd700 1 -- 172.21.15.44:0/2035853627 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8003960 con 0x55f4834c10d0 2023-09-10T01:05:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.769+0000 7f501214db80 1 -- 172.21.15.44:0/2035853627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f483366490 con 0x55f4834c10d0 2023-09-10T01:05:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.769+0000 7f4fdeffd700 1 -- 172.21.15.44:0/2035853627 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f4fe8003ac0 con 0x55f4834c10d0 2023-09-10T01:05:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.769+0000 7f4fdeffd700 1 -- 172.21.15.44:0/2035853627 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fe8003c80 con 0x55f4834c10d0 2023-09-10T01:05:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.769+0000 7f501214db80 10 client.18696 fetch_fsmap learned FSMap version 102 2023-09-10T01:05:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.769+0000 7f501214db80 1 -- 172.21.15.44:0/2035853627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) v3 -- 0x55f483223c00 con 0x55f4834c10d0 2023-09-10T01:05:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.769+0000 7f4fdeffd700 1 -- 172.21.15.44:0/2035853627 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f483366490 con 0x55f4834c10d0 2023-09-10T01:05:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.769+0000 7f4fdeffd700 1 -- 172.21.15.44:0/2035853627 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 102) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f4fe800f460 con 0x55f4834c10d0 2023-09-10T01:05:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.769+0000 7f501214db80 10 client.18696 fetch_fsmap finished waiting for FSMap version 102 2023-09-10T01:05:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.769+0000 7f501214db80 10 client.18696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:05:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.769+0000 7f501214db80 20 client.18696 populate_metadata read hostname 'smithi044' 2023-09-10T01:05:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.769+0000 7f501214db80 1 --2- 172.21.15.44:0/2035853627 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f4834da840 0x55f4834dcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.769+0000 7f501214db80 4 client.18696 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:05:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.769+0000 7f501214db80 1 -- 172.21.15.44:0/2035853627 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f483423840 con 0x55f4834da840 2023-09-10T01:05:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.770+0000 7f4fff7fe700 1 --2- 172.21.15.44:0/2035853627 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f4834da840 0x55f4834dcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.770+0000 7f4fdeffd700 10 client.18696 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.770+0000 7f4fff7fe700 1 --2- 172.21.15.44:0/2035853627 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f4834da840 0x55f4834dcc60 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51459ac31c51df54 server_cookie=ccd093306f9fc019 in_seq=0 out_seq=0 2023-09-10T01:05:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.771+0000 7f4fdeffd700 1 -- 172.21.15.44:0/2035853627 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f483423840 con 0x55f4834da840 2023-09-10T01:05:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.771+0000 7f4fdeffd700 10 client.18696 handle_command_reply: tid=0 2023-09-10T01:05:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.771+0000 7f501214db80 1 client.18696 shutdown 2023-09-10T01:05:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.771+0000 7f501214db80 1 -- 172.21.15.44:0/2035853627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 msgr2=0x55f4834ca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.771+0000 7f501214db80 1 --2- 172.21.15.44:0/2035853627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834ca560 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80098e0 tx=0x7f4fe8005420 comp rx=0 tx=0).stop 2023-09-10T01:05:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.771+0000 7f501214db80 1 -- 172.21.15.44:0/2035853627 shutdown_connections 2023-09-10T01:05:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.771+0000 7f501214db80 1 --2- 172.21.15.44:0/2035853627 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f4834da840 0x55f4834dcc60 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.771+0000 7f501214db80 1 --2- 172.21.15.44:0/2035853627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c65b0 0x55f4834caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.771+0000 7f4fdeffd700 0 client.18696 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.771+0000 7f501214db80 1 --2- 172.21.15.44:0/2035853627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c0710 0x55f4834cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.771+0000 7f501214db80 1 --2- 172.21.15.44:0/2035853627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834ca560 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.771+0000 7f501214db80 1 -- 172.21.15.44:0/2035853627 >> 172.21.15.44:0/2035853627 conn(0x55f483431030 msgr2=0x55f4834b67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.771+0000 7f501214db80 1 -- 172.21.15.44:0/2035853627 shutdown_connections 2023-09-10T01:05:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.771+0000 7f501214db80 1 -- 172.21.15.44:0/2035853627 wait complete. 2023-09-10T01:05:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.771+0000 7f501214db80 20 client.18696 trim_cache size 0 max 16384 2023-09-10T01:05:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f501214db80 1 Processor -- start 2023-09-10T01:05:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f501214db80 1 -- start start 2023-09-10T01:05:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f501214db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c0710 0x55f4834c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f501214db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834d6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f501214db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c65b0 0x55f4834ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f501214db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4834c5df0 con 0x55f4834c10d0 2023-09-10T01:05:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f501214db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4834b8320 con 0x55f4834c65b0 2023-09-10T01:05:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f501214db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4834c5150 con 0x55f4834c0710 2023-09-10T01:05:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f4ffeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834d6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f4ffeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834d6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55614/0 (socket says 172.21.15.44:55614) 2023-09-10T01:05:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f4ffeffd700 1 -- 172.21.15.44:0/2705939676 learned_addr learned my addr 172.21.15.44:0/2705939676 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f4ffffff700 1 --2- 172.21.15.44:0/2705939676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c65b0 0x55f4834ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f4fff7fe700 1 --2- 172.21.15.44:0/2705939676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c0710 0x55f4834c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f4ffeffd700 1 -- 172.21.15.44:0/2705939676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c0710 msgr2=0x55f4834c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f4ffeffd700 1 --2- 172.21.15.44:0/2705939676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c0710 0x55f4834c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f4ffeffd700 1 -- 172.21.15.44:0/2705939676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c65b0 msgr2=0x55f4834ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f4ffeffd700 1 --2- 172.21.15.44:0/2705939676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c65b0 0x55f4834ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.786+0000 7f4ffeffd700 1 -- 172.21.15.44:0/2705939676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4832611f0 con 0x55f4834c10d0 2023-09-10T01:05:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.787+0000 7f4ffeffd700 1 --2- 172.21.15.44:0/2705939676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834d6b50 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009ff0 tx=0x7f4fe8009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8663299496f357d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.787+0000 7f4ff67fc700 1 -- 172.21.15.44:0/2705939676 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe80058e0 con 0x55f4834c10d0 2023-09-10T01:05:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.787+0000 7f4ff67fc700 1 -- 172.21.15.44:0/2705939676 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4fe801e040 con 0x55f4834c10d0 2023-09-10T01:05:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.787+0000 7f4ff67fc700 1 -- 172.21.15.44:0/2705939676 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8003e30 con 0x55f4834c10d0 2023-09-10T01:05:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.787+0000 7f501214db80 1 -- 172.21.15.44:0/2705939676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 msgr2=0x55f4834d6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.787+0000 7f501214db80 1 --2- 172.21.15.44:0/2705939676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834d6b50 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009ff0 tx=0x7f4fe8009290 comp rx=0 tx=0).stop 2023-09-10T01:05:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.787+0000 7f501214db80 1 -- 172.21.15.44:0/2705939676 shutdown_connections 2023-09-10T01:05:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.787+0000 7f501214db80 1 --2- 172.21.15.44:0/2705939676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c65b0 0x55f4834ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.787+0000 7f501214db80 1 --2- 172.21.15.44:0/2705939676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c0710 0x55f4834c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.787+0000 7f501214db80 1 --2- 172.21.15.44:0/2705939676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c10d0 0x55f4834d6b50 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.787+0000 7f501214db80 1 -- 172.21.15.44:0/2705939676 >> 172.21.15.44:0/2705939676 conn(0x55f483431030 msgr2=0x55f4834e15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.787+0000 7f501214db80 1 -- 172.21.15.44:0/2705939676 shutdown_connections 2023-09-10T01:05:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.787+0000 7f501214db80 1 -- 172.21.15.44:0/2705939676 wait complete. 2023-09-10T01:05:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f501214db80 1 -- start start 2023-09-10T01:05:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f501214db80 1 Processor -- start 2023-09-10T01:05:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f501214db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c0710 0x55f4834d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f501214db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c10d0 0x55f4834d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f501214db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c65b0 0x55f4834cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f501214db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4834d0230 con 0x55f4834c0710 2023-09-10T01:05:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f501214db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4834d03a0 con 0x55f4834c65b0 2023-09-10T01:05:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f501214db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4834d0560 con 0x55f4834c10d0 2023-09-10T01:05:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f501214db80 10 client.0 _ll_register_callbacks cb 0x7f4fe800be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:05:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f4fff7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c0710 0x55f4834d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f4fff7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c0710 0x55f4834d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55618/0 (socket says 172.21.15.44:55618) 2023-09-10T01:05:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f4fff7fe700 1 -- 172.21.15.44:0/457614119 learned_addr learned my addr 172.21.15.44:0/457614119 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f4ffeffd700 1 --2- 172.21.15.44:0/457614119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c10d0 0x55f4834d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f4ffffff700 1 --2- 172.21.15.44:0/457614119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c65b0 0x55f4834cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f4fff7fe700 1 -- 172.21.15.44:0/457614119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c10d0 msgr2=0x55f4834d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f4fff7fe700 1 --2- 172.21.15.44:0/457614119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c10d0 0x55f4834d1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f4fff7fe700 1 -- 172.21.15.44:0/457614119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c65b0 msgr2=0x55f4834cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f4fff7fe700 1 --2- 172.21.15.44:0/457614119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c65b0 0x55f4834cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.788+0000 7f4fff7fe700 1 -- 172.21.15.44:0/457614119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f483223c00 con 0x55f4834c0710 2023-09-10T01:05:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.789+0000 7f4fff7fe700 1 --2- 172.21.15.44:0/457614119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c0710 0x55f4834d1580 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0005e90 tx=0x7f4ff00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad5d8aaafb39364 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.789+0000 7f4fdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:05:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.789+0000 7f4fdeffd700 1 -- 172.21.15.44:0/457614119 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0024040 con 0x55f4834c0710 2023-09-10T01:05:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.789+0000 7f4fdeffd700 1 -- 172.21.15.44:0/457614119 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f4ff0017070 con 0x55f4834c0710 2023-09-10T01:05:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.789+0000 7f501214db80 1 -- 172.21.15.44:0/457614119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ff8098fc0 con 0x55f4834c0710 2023-09-10T01:05:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.789+0000 7f4fdeffd700 1 -- 172.21.15.44:0/457614119 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ff0029e50 con 0x55f4834c0710 2023-09-10T01:05:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.789+0000 7f501214db80 10 client.18702 fetch_fsmap learned FSMap version 102 2023-09-10T01:05:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.789+0000 7f501214db80 1 -- 172.21.15.44:0/457614119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) v3 -- 0x55f483239250 con 0x55f4834c0710 2023-09-10T01:05:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.789+0000 7f4fdeffd700 1 -- 172.21.15.44:0/457614119 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ff8098fc0 con 0x55f4834c0710 2023-09-10T01:05:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.789+0000 7f4fdeffd700 1 -- 172.21.15.44:0/457614119 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 102) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f4ff0032680 con 0x55f4834c0710 2023-09-10T01:05:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.789+0000 7f501214db80 10 client.18702 fetch_fsmap finished waiting for FSMap version 102 2023-09-10T01:05:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.789+0000 7f501214db80 10 client.18702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:05:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.789+0000 7f501214db80 20 client.18702 populate_metadata read hostname 'smithi044' 2023-09-10T01:05:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.789+0000 7f501214db80 1 --2- 172.21.15.44:0/457614119 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f4834da840 0x55f4835319d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.789+0000 7f501214db80 4 client.18702 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:05:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.789+0000 7f501214db80 1 -- 172.21.15.44:0/457614119 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f4834078c0 con 0x55f4834da840 2023-09-10T01:05:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.790+0000 7f4ffeffd700 1 --2- 172.21.15.44:0/457614119 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f4834da840 0x55f4835319d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.790+0000 7f4fdeffd700 10 client.18702 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.790+0000 7f4ffeffd700 1 --2- 172.21.15.44:0/457614119 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f4834da840 0x55f4835319d0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=330cb4a7421ea695 server_cookie=65b58f85ceee7da9 in_seq=0 out_seq=0 2023-09-10T01:05:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.811+0000 7f4fdeffd700 1 -- 172.21.15.44:0/457614119 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f4834078c0 con 0x55f4834da840 2023-09-10T01:05:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.811+0000 7f4fdeffd700 10 client.18702 handle_command_reply: tid=0 2023-09-10T01:05:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.811+0000 7f501214db80 1 client.18702 shutdown 2023-09-10T01:05:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f501214db80 1 -- 172.21.15.44:0/457614119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c0710 msgr2=0x55f4834d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f501214db80 1 --2- 172.21.15.44:0/457614119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c0710 0x55f4834d1580 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0005e90 tx=0x7f4ff00047c0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f501214db80 1 -- 172.21.15.44:0/457614119 shutdown_connections 2023-09-10T01:05:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f501214db80 1 --2- 172.21.15.44:0/457614119 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f4834da840 0x55f4835319d0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f501214db80 1 --2- 172.21.15.44:0/457614119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4834c65b0 0x55f4834cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f4fdeffd700 0 client.18702 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f501214db80 1 --2- 172.21.15.44:0/457614119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4834c0710 0x55f4834d1580 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f501214db80 1 --2- 172.21.15.44:0/457614119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4834c10d0 0x55f4834d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f501214db80 1 -- 172.21.15.44:0/457614119 >> 172.21.15.44:0/457614119 conn(0x55f483431030 msgr2=0x55f4834c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f501214db80 1 -- 172.21.15.44:0/457614119 shutdown_connections 2023-09-10T01:05:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f501214db80 1 -- 172.21.15.44:0/457614119 wait complete. 2023-09-10T01:05:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f501214db80 20 client.18702 trim_cache size 0 max 16384 2023-09-10T01:05:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:05:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:05:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:05:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:05:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f5005f87700 1 -- 172.21.15.44:0/4198392553 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4fec0a07b0 msgr2=0x7f4fec0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f5005f87700 1 --2- 172.21.15.44:0/4198392553 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4fec0a07b0 0x7f4fec0a2c70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0007c10 tx=0x7f4ff0007480 comp rx=0 tx=0).stop 2023-09-10T01:05:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f5005f87700 1 -- 172.21.15.44:0/4198392553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50001055b0 msgr2=0x7f50001a24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f5005f87700 1 --2- 172.21.15.44:0/4198392553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50001055b0 0x7f50001a24d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800efc0 tx=0x7f4ff800c3a0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f5005f87700 1 -- 172.21.15.44:0/4198392553 shutdown_connections 2023-09-10T01:05:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f5005f87700 1 --2- 172.21.15.44:0/4198392553 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4fec0a07b0 0x7f4fec0a2c70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f5005f87700 1 --2- 172.21.15.44:0/4198392553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50001055b0 0x7f50001a24d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f5005f87700 1 --2- 172.21.15.44:0/4198392553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50000ff760 0x7f5000197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f5005f87700 1 --2- 172.21.15.44:0/4198392553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5000100120 0x7f5000197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.812+0000 7f5005f87700 1 -- 172.21.15.44:0/4198392553 >> 172.21.15.44:0/4198392553 conn(0x7f5000074f30 msgr2=0x7f50000fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.813+0000 7f5005f87700 1 -- 172.21.15.44:0/4198392553 shutdown_connections 2023-09-10T01:05:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:29.813+0000 7f5005f87700 1 -- 172.21.15.44:0/4198392553 wait complete. 2023-09-10T01:05:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:30 smithi131 ceph-mon[103713]: pgmap v788: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 932 KiB/s wr, 134 op/s 2023-09-10T01:05:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:30 smithi044 ceph-mon[98676]: pgmap v788: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 932 KiB/s wr, 134 op/s 2023-09-10T01:05:31.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:30 smithi099 ceph-mon[102692]: pgmap v788: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 932 KiB/s wr, 134 op/s 2023-09-10T01:05:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:32 smithi131 ceph-mon[103713]: pgmap v789: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2023-09-10T01:05:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:32 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:32 smithi044 ceph-mon[98676]: pgmap v789: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2023-09-10T01:05:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:32 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:33.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:32 smithi099 ceph-mon[102692]: pgmap v789: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2023-09-10T01:05:33.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:32 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:34 smithi131 ceph-mon[103713]: pgmap v790: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 988 KiB/s wr, 141 op/s 2023-09-10T01:05:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:34 smithi044 ceph-mon[98676]: pgmap v790: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 988 KiB/s wr, 141 op/s 2023-09-10T01:05:35.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:34 smithi099 ceph-mon[102692]: pgmap v790: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 988 KiB/s wr, 141 op/s 2023-09-10T01:05:37.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:36 smithi131 ceph-mon[103713]: pgmap v791: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2023-09-10T01:05:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:36 smithi044 ceph-mon[98676]: pgmap v791: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2023-09-10T01:05:37.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:36 smithi099 ceph-mon[102692]: pgmap v791: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2023-09-10T01:05:38.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:37 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:38.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:37 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:38.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:37 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:38 smithi131 ceph-mon[103713]: pgmap v792: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-09-10T01:05:39.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:38 smithi044 ceph-mon[98676]: pgmap v792: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-09-10T01:05:39.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:38 smithi099 ceph-mon[102692]: pgmap v792: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-09-10T01:05:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:40 smithi131 ceph-mon[103713]: pgmap v793: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-09-10T01:05:41.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:40 smithi044 ceph-mon[98676]: pgmap v793: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-09-10T01:05:41.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:40 smithi099 ceph-mon[102692]: pgmap v793: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-09-10T01:05:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.956+0000 7f13550f6700 1 -- 172.21.15.44:0/2695194175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13500feaa0 msgr2=0x7f13500fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.956+0000 7f13550f6700 1 --2- 172.21.15.44:0/2695194175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13500feaa0 0x7f13500fef20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1344009990 tx=0x7f1344009ca0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.956+0000 7f13550f6700 1 -- 172.21.15.44:0/2695194175 shutdown_connections 2023-09-10T01:05:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.956+0000 7f13550f6700 1 --2- 172.21.15.44:0/2695194175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13500ff460 0x7f135010d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.956+0000 7f13550f6700 1 --2- 172.21.15.44:0/2695194175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1350106620 0x7f1350106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.956+0000 7f13550f6700 1 --2- 172.21.15.44:0/2695194175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13500feaa0 0x7f13500fef20 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.956+0000 7f13550f6700 1 -- 172.21.15.44:0/2695194175 >> 172.21.15.44:0/2695194175 conn(0x7f13500769d0 msgr2=0x7f13500fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.956+0000 7f13550f6700 1 -- 172.21.15.44:0/2695194175 shutdown_connections 2023-09-10T01:05:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.956+0000 7f13550f6700 1 -- 172.21.15.44:0/2695194175 wait complete. 2023-09-10T01:05:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.957+0000 7f13550f6700 1 Processor -- start 2023-09-10T01:05:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.957+0000 7f13550f6700 1 -- start start 2023-09-10T01:05:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.957+0000 7f13550f6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13500feaa0 0x7f13500ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.957+0000 7f13550f6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13500ff460 0x7f13501003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.957+0000 7f13550f6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1350106620 0x7f13501a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.957+0000 7f13550f6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13501a4c90 con 0x7f1350106620 2023-09-10T01:05:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.957+0000 7f13550f6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13501a4dd0 con 0x7f13500ff460 2023-09-10T01:05:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.957+0000 7f13550f6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13501a4f40 con 0x7f13500feaa0 2023-09-10T01:05:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f134f59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1350106620 0x7f13501a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f134e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13500ff460 0x7f13501003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f134e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13500ff460 0x7f13501003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57374/0 (socket says 172.21.15.44:57374) 2023-09-10T01:05:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f134f59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1350106620 0x7f13501a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55636/0 (socket says 172.21.15.44:55636) 2023-09-10T01:05:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f134e59c700 1 -- 172.21.15.44:0/860233119 learned_addr learned my addr 172.21.15.44:0/860233119 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f134ed9d700 1 --2- 172.21.15.44:0/860233119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13500feaa0 0x7f13500ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f134e59c700 1 -- 172.21.15.44:0/860233119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13500feaa0 msgr2=0x7f13500ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f134e59c700 1 --2- 172.21.15.44:0/860233119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13500feaa0 0x7f13500ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f134e59c700 1 -- 172.21.15.44:0/860233119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1350106620 msgr2=0x7f13501a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f134e59c700 1 --2- 172.21.15.44:0/860233119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1350106620 0x7f13501a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f134e59c700 1 -- 172.21.15.44:0/860233119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13440095f0 con 0x7f13500ff460 2023-09-10T01:05:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f134ed9d700 1 --2- 172.21.15.44:0/860233119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13500feaa0 0x7f13500ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:05:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f134e59c700 1 --2- 172.21.15.44:0/860233119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13500ff460 0x7f13501003a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1344000c00 tx=0x7f1344004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f1337fff700 1 -- 172.21.15.44:0/860233119 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134401d070 con 0x7f13500ff460 2023-09-10T01:05:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f13550f6700 1 -- 172.21.15.44:0/860233119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13501a51c0 con 0x7f13500ff460 2023-09-10T01:05:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f1337fff700 1 -- 172.21.15.44:0/860233119 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1344022470 con 0x7f13500ff460 2023-09-10T01:05:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.958+0000 7f1337fff700 1 -- 172.21.15.44:0/860233119 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1344017650 con 0x7f13500ff460 2023-09-10T01:05:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.959+0000 7f13550f6700 1 -- 172.21.15.44:0/860233119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13501a5640 con 0x7f13500ff460 2023-09-10T01:05:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.961+0000 7f1337fff700 1 -- 172.21.15.44:0/860233119 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f13440225e0 con 0x7f13500ff460 2023-09-10T01:05:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.962+0000 7f1337fff700 1 --2- 172.21.15.44:0/860233119 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f13380a07b0 0x7f13380a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.962+0000 7f13612bcb80 1 Processor -- start 2023-09-10T01:05:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.962+0000 7f1337fff700 1 -- 172.21.15.44:0/860233119 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f13440cd7f0 con 0x7f13500ff460 2023-09-10T01:05:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.962+0000 7f134ed9d700 1 --2- 172.21.15.44:0/860233119 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f13380a07b0 0x7f13380a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.962+0000 7f13612bcb80 1 -- start start 2023-09-10T01:05:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f13612bcb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff28171a0 0x555ff2817580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f13612bcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff2811300 0x555ff2811780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f13612bcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811cc0 0x555ff281df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f13612bcb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff276a6d0 con 0x555ff28171a0 2023-09-10T01:05:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f13612bcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff2767ff0 con 0x555ff2811300 2023-09-10T01:05:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f13612bcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff27638e0 con 0x555ff2811cc0 2023-09-10T01:05:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f134f59e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811cc0 0x555ff281df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f134f59e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811cc0 0x555ff281df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34290/0 (socket says 172.21.15.44:34290) 2023-09-10T01:05:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f134f59e700 1 -- 172.21.15.44:0/2060491491 learned_addr learned my addr 172.21.15.44:0/2060491491 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f134ed9d700 1 --2- 172.21.15.44:0/860233119 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f13380a07b0 0x7f13380a2c70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1340011490 tx=0x7f1340009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f134ed9d700 1 --2- 172.21.15.44:0/2060491491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff28171a0 0x555ff2817580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f134e59c700 1 --2- 172.21.15.44:0/2060491491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff2811300 0x555ff2811780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f134f59e700 1 -- 172.21.15.44:0/2060491491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff2811300 msgr2=0x555ff2811780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f134f59e700 1 --2- 172.21.15.44:0/2060491491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff2811300 0x555ff2811780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f134f59e700 1 -- 172.21.15.44:0/2060491491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff28171a0 msgr2=0x555ff2817580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f134f59e700 1 --2- 172.21.15.44:0/2060491491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff28171a0 0x555ff2817580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.963+0000 7f134f59e700 1 -- 172.21.15.44:0/2060491491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ff258fd80 con 0x555ff2811cc0 2023-09-10T01:05:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.964+0000 7f134f59e700 1 --2- 172.21.15.44:0/2060491491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811cc0 0x555ff281df30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1348011680 tx=0x7f1348009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=265ac6bb89f00f57 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.964+0000 7f13357fa700 1 -- 172.21.15.44:0/2060491491 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134800e040 con 0x555ff2811cc0 2023-09-10T01:05:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.964+0000 7f13357fa700 1 -- 172.21.15.44:0/2060491491 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f134800f040 con 0x555ff2811cc0 2023-09-10T01:05:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.964+0000 7f13357fa700 1 -- 172.21.15.44:0/2060491491 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1348004730 con 0x555ff2811cc0 2023-09-10T01:05:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.965+0000 7f13612bcb80 1 -- 172.21.15.44:0/2060491491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811cc0 msgr2=0x555ff281df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.965+0000 7f13612bcb80 1 --2- 172.21.15.44:0/2060491491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811cc0 0x555ff281df30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1348011680 tx=0x7f1348009920 comp rx=0 tx=0).stop 2023-09-10T01:05:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.965+0000 7f13612bcb80 1 -- 172.21.15.44:0/2060491491 shutdown_connections 2023-09-10T01:05:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.965+0000 7f13612bcb80 1 --2- 172.21.15.44:0/2060491491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811cc0 0x555ff281df30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.965+0000 7f13612bcb80 1 --2- 172.21.15.44:0/2060491491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff28171a0 0x555ff2817580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.965+0000 7f13612bcb80 1 --2- 172.21.15.44:0/2060491491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff2811300 0x555ff2811780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.965+0000 7f13612bcb80 1 -- 172.21.15.44:0/2060491491 >> 172.21.15.44:0/2060491491 conn(0x555ff2786030 msgr2=0x555ff280c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.965+0000 7f13612bcb80 1 -- 172.21.15.44:0/2060491491 shutdown_connections 2023-09-10T01:05:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.965+0000 7f13612bcb80 1 -- 172.21.15.44:0/2060491491 wait complete. 2023-09-10T01:05:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.965+0000 7f13612bcb80 1 -- start start 2023-09-10T01:05:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.965+0000 7f13612bcb80 1 Processor -- start 2023-09-10T01:05:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f13612bcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811300 0x555ff281dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f13612bcb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 0x555ff281b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f13612bcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff28171a0 0x555ff281b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f13612bcb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff2701b10 con 0x555ff2811cc0 2023-09-10T01:05:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f13612bcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff2701810 con 0x555ff28171a0 2023-09-10T01:05:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f13612bcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff25e58c0 con 0x555ff2811300 2023-09-10T01:05:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f13612bcb80 10 client.0 _ll_register_callbacks cb 0x555ff264ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:05:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f134ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811300 0x555ff281dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f134ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811300 0x555ff281dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34296/0 (socket says 172.21.15.44:34296) 2023-09-10T01:05:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f134ed9d700 1 -- 172.21.15.44:0/4107299175 learned_addr learned my addr 172.21.15.44:0/4107299175 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f134f59e700 1 --2- 172.21.15.44:0/4107299175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff28171a0 0x555ff281b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f134e59c700 1 --2- 172.21.15.44:0/4107299175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 0x555ff281b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f134ed9d700 1 -- 172.21.15.44:0/4107299175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff28171a0 msgr2=0x555ff281b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f134ed9d700 1 --2- 172.21.15.44:0/4107299175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff28171a0 0x555ff281b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f134ed9d700 1 -- 172.21.15.44:0/4107299175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 msgr2=0x555ff281b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f134ed9d700 1 --2- 172.21.15.44:0/4107299175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 0x555ff281b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f134ed9d700 1 -- 172.21.15.44:0/4107299175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1348011330 con 0x555ff2811300 2023-09-10T01:05:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f134ed9d700 1 --2- 172.21.15.44:0/4107299175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811300 0x555ff281dd80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1340003ad0 tx=0x7f13400047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5bd3006b846f32b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.966+0000 7f1332ff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:05:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.967+0000 7f1332ff5700 1 -- 172.21.15.44:0/4107299175 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134001fa90 con 0x555ff2811300 2023-09-10T01:05:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.967+0000 7f13612bcb80 1 -- 172.21.15.44:0/4107299175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ff26bb490 con 0x555ff2811300 2023-09-10T01:05:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.967+0000 7f1332ff5700 1 -- 172.21.15.44:0/4107299175 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f134001fbf0 con 0x555ff2811300 2023-09-10T01:05:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.967+0000 7f1332ff5700 1 -- 172.21.15.44:0/4107299175 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f134001fdb0 con 0x555ff2811300 2023-09-10T01:05:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.967+0000 7f13612bcb80 10 client.28306 fetch_fsmap learned FSMap version 105 2023-09-10T01:05:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.967+0000 7f13612bcb80 1 -- 172.21.15.44:0/4107299175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x555ff2578c00 con 0x555ff2811300 2023-09-10T01:05:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.967+0000 7f1332ff5700 1 -- 172.21.15.44:0/4107299175 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555ff26bb490 con 0x555ff2811300 2023-09-10T01:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.967+0000 7f1332ff5700 1 -- 172.21.15.44:0/4107299175 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 105) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f1340027600 con 0x555ff2811300 2023-09-10T01:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.967+0000 7f13612bcb80 10 client.28306 fetch_fsmap finished waiting for FSMap version 105 2023-09-10T01:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.967+0000 7f13612bcb80 10 client.28306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.967+0000 7f13612bcb80 20 client.28306 populate_metadata read hostname 'smithi044' 2023-09-10T01:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.967+0000 7f13612bcb80 1 --2- 172.21.15.44:0/4107299175 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555ff282b420 0x555ff282d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.967+0000 7f13612bcb80 4 client.28306 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:05:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.967+0000 7f13612bcb80 1 -- 172.21.15.44:0/4107299175 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ff2778840 con 0x555ff282b420 2023-09-10T01:05:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.968+0000 7f134e59c700 1 --2- 172.21.15.44:0/4107299175 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555ff282b420 0x555ff282d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.968+0000 7f1332ff5700 10 client.28306 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.968+0000 7f134e59c700 1 --2- 172.21.15.44:0/4107299175 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555ff282b420 0x555ff282d840 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f1048d1902e1a5 server_cookie=da6828d36f037828 in_seq=0 out_seq=0 2023-09-10T01:05:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.969+0000 7f1332ff5700 1 -- 172.21.15.44:0/4107299175 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555ff2778840 con 0x555ff282b420 2023-09-10T01:05:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.969+0000 7f1332ff5700 10 client.28306 handle_command_reply: tid=0 2023-09-10T01:05:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.969+0000 7f13612bcb80 1 client.28306 shutdown 2023-09-10T01:05:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.969+0000 7f13612bcb80 1 -- 172.21.15.44:0/4107299175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811300 msgr2=0x555ff281dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.969+0000 7f13612bcb80 1 --2- 172.21.15.44:0/4107299175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811300 0x555ff281dd80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1340003ad0 tx=0x7f13400047c0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.969+0000 7f13612bcb80 1 -- 172.21.15.44:0/4107299175 shutdown_connections 2023-09-10T01:05:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.969+0000 7f13612bcb80 1 --2- 172.21.15.44:0/4107299175 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555ff282b420 0x555ff282d840 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.969+0000 7f13612bcb80 1 --2- 172.21.15.44:0/4107299175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff28171a0 0x555ff281b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.969+0000 7f13612bcb80 1 --2- 172.21.15.44:0/4107299175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811300 0x555ff281dd80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.969+0000 7f1332ff5700 0 client.28306 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.969+0000 7f13612bcb80 1 --2- 172.21.15.44:0/4107299175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 0x555ff281b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.969+0000 7f13612bcb80 1 -- 172.21.15.44:0/4107299175 >> 172.21.15.44:0/4107299175 conn(0x555ff2786030 msgr2=0x555ff280b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.969+0000 7f13612bcb80 1 -- 172.21.15.44:0/4107299175 shutdown_connections 2023-09-10T01:05:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.969+0000 7f13612bcb80 1 -- 172.21.15.44:0/4107299175 wait complete. 2023-09-10T01:05:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.969+0000 7f13612bcb80 20 client.28306 trim_cache size 0 max 16384 2023-09-10T01:05:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.987+0000 7f13612bcb80 1 Processor -- start 2023-09-10T01:05:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.987+0000 7f13612bcb80 1 -- start start 2023-09-10T01:05:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.987+0000 7f13612bcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811300 0x555ff2822520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f13612bcb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 0x555ff281d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f13612bcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff28171a0 0x555ff281dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f13612bcb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff2822a60 con 0x555ff2811cc0 2023-09-10T01:05:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f13612bcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff28195b0 con 0x555ff28171a0 2023-09-10T01:05:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f13612bcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff2827c90 con 0x555ff2811300 2023-09-10T01:05:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f134e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 0x555ff281d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f134ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811300 0x555ff2822520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f134ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811300 0x555ff2822520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34304/0 (socket says 172.21.15.44:34304) 2023-09-10T01:05:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f134e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 0x555ff281d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55656/0 (socket says 172.21.15.44:55656) 2023-09-10T01:05:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f134ed9d700 1 -- 172.21.15.44:0/2454980403 learned_addr learned my addr 172.21.15.44:0/2454980403 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f134f59e700 1 --2- 172.21.15.44:0/2454980403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff28171a0 0x555ff281dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f134ed9d700 1 -- 172.21.15.44:0/2454980403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff28171a0 msgr2=0x555ff281dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f134ed9d700 1 --2- 172.21.15.44:0/2454980403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff28171a0 0x555ff281dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f134ed9d700 1 -- 172.21.15.44:0/2454980403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 msgr2=0x555ff281d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f134ed9d700 1 --2- 172.21.15.44:0/2454980403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 0x555ff281d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f134ed9d700 1 -- 172.21.15.44:0/2454980403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ff25b61f0 con 0x555ff2811300 2023-09-10T01:05:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f134e59c700 1 --2- 172.21.15.44:0/2454980403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 0x555ff281d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:05:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f134f59e700 1 --2- 172.21.15.44:0/2454980403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff28171a0 0x555ff281dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:05:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.988+0000 7f134ed9d700 1 --2- 172.21.15.44:0/2454980403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811300 0x555ff2822520 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1340000c00 tx=0x7f1340004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=394d716cb16e4e96 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.989+0000 7f1335ffb700 1 -- 172.21.15.44:0/2454980403 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340027600 con 0x555ff2811300 2023-09-10T01:05:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.989+0000 7f1335ffb700 1 -- 172.21.15.44:0/2454980403 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1340027760 con 0x555ff2811300 2023-09-10T01:05:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.989+0000 7f1335ffb700 1 -- 172.21.15.44:0/2454980403 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13400288b0 con 0x555ff2811300 2023-09-10T01:05:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.989+0000 7f13612bcb80 1 -- 172.21.15.44:0/2454980403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811300 msgr2=0x555ff2822520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.989+0000 7f13612bcb80 1 --2- 172.21.15.44:0/2454980403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811300 0x555ff2822520 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1340000c00 tx=0x7f1340004750 comp rx=0 tx=0).stop 2023-09-10T01:05:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.989+0000 7f13612bcb80 1 -- 172.21.15.44:0/2454980403 shutdown_connections 2023-09-10T01:05:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.989+0000 7f13612bcb80 1 --2- 172.21.15.44:0/2454980403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff28171a0 0x555ff281dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.989+0000 7f13612bcb80 1 --2- 172.21.15.44:0/2454980403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff2811300 0x555ff2822520 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.989+0000 7f13612bcb80 1 --2- 172.21.15.44:0/2454980403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 0x555ff281d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.989+0000 7f13612bcb80 1 -- 172.21.15.44:0/2454980403 >> 172.21.15.44:0/2454980403 conn(0x555ff2786030 msgr2=0x555ff2810b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.989+0000 7f13612bcb80 1 -- 172.21.15.44:0/2454980403 shutdown_connections 2023-09-10T01:05:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.989+0000 7f13612bcb80 1 -- 172.21.15.44:0/2454980403 wait complete. 2023-09-10T01:05:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.990+0000 7f13612bcb80 1 -- start start 2023-09-10T01:05:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.990+0000 7f13612bcb80 1 Processor -- start 2023-09-10T01:05:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.990+0000 7f13612bcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff2811300 0x555ff2821cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.990+0000 7f13612bcb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 0x555ff28221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.990+0000 7f13612bcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff28171a0 0x555ff28200e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.990+0000 7f13612bcb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff28206b0 con 0x555ff2811cc0 2023-09-10T01:05:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.990+0000 7f13612bcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff2820820 con 0x555ff2811300 2023-09-10T01:05:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.990+0000 7f13612bcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff28209e0 con 0x555ff28171a0 2023-09-10T01:05:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.990+0000 7f13612bcb80 10 client.0 _ll_register_callbacks cb 0x7f1340003680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:05:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.990+0000 7f134e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 0x555ff28221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.990+0000 7f134ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff2811300 0x555ff2821cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.990+0000 7f134e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 0x555ff28221f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55662/0 (socket says 172.21.15.44:55662) 2023-09-10T01:05:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.990+0000 7f134e59c700 1 -- 172.21.15.44:0/2318528527 learned_addr learned my addr 172.21.15.44:0/2318528527 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.990+0000 7f134f59e700 1 --2- 172.21.15.44:0/2318528527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff28171a0 0x555ff28200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.991+0000 7f134e59c700 1 -- 172.21.15.44:0/2318528527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff28171a0 msgr2=0x555ff28200e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.991+0000 7f134e59c700 1 --2- 172.21.15.44:0/2318528527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff28171a0 0x555ff28200e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.991+0000 7f134e59c700 1 -- 172.21.15.44:0/2318528527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff2811300 msgr2=0x555ff2821cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.991+0000 7f134e59c700 1 --2- 172.21.15.44:0/2318528527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff2811300 0x555ff2821cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.991+0000 7f134e59c700 1 -- 172.21.15.44:0/2318528527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13501a5640 con 0x555ff2811cc0 2023-09-10T01:05:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.991+0000 7f134ed9d700 1 --2- 172.21.15.44:0/2318528527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff2811300 0x555ff2821cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:05:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.991+0000 7f134e59c700 1 --2- 172.21.15.44:0/2318528527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 0x555ff28221f0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f1344096fb0 tx=0x7f134400f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a5148463391e5a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.991+0000 7f1332ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:05:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.991+0000 7f1332ff5700 1 -- 172.21.15.44:0/2318528527 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134409af10 con 0x555ff2811cc0 2023-09-10T01:05:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.991+0000 7f13612bcb80 1 -- 172.21.15.44:0/2318528527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f134800f040 con 0x555ff2811cc0 2023-09-10T01:05:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.991+0000 7f1332ff5700 1 -- 172.21.15.44:0/2318528527 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f13440228c0 con 0x555ff2811cc0 2023-09-10T01:05:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.991+0000 7f1332ff5700 1 -- 172.21.15.44:0/2318528527 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f134409b1b0 con 0x555ff2811cc0 2023-09-10T01:05:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.991+0000 7f13612bcb80 10 client.18720 fetch_fsmap learned FSMap version 105 2023-09-10T01:05:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.991+0000 7f13612bcb80 1 -- 172.21.15.44:0/2318528527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x555ff258fd80 con 0x555ff2811cc0 2023-09-10T01:05:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.991+0000 7f1332ff5700 1 -- 172.21.15.44:0/2318528527 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f134800f040 con 0x555ff2811cc0 2023-09-10T01:05:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.992+0000 7f1332ff5700 1 -- 172.21.15.44:0/2318528527 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 105) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f134409a030 con 0x555ff2811cc0 2023-09-10T01:05:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.992+0000 7f13612bcb80 10 client.18720 fetch_fsmap finished waiting for FSMap version 105 2023-09-10T01:05:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.992+0000 7f13612bcb80 10 client.18720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:05:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.992+0000 7f13612bcb80 20 client.18720 populate_metadata read hostname 'smithi044' 2023-09-10T01:05:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.992+0000 7f13612bcb80 1 --2- 172.21.15.44:0/2318528527 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555ff2880070 0x555ff2882490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.992+0000 7f13612bcb80 4 client.18720 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:05:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.992+0000 7f13612bcb80 1 -- 172.21.15.44:0/2318528527 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555ff275c8c0 con 0x555ff2880070 2023-09-10T01:05:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.992+0000 7f134ed9d700 1 --2- 172.21.15.44:0/2318528527 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555ff2880070 0x555ff2882490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.992+0000 7f134ed9d700 1 --2- 172.21.15.44:0/2318528527 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555ff2880070 0x555ff2882490 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a61ffa5b625da6e5 server_cookie=25e42c22368ad1fe in_seq=0 out_seq=0 2023-09-10T01:05:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:42.992+0000 7f1332ff5700 10 client.18720 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.006+0000 7f1332ff5700 1 -- 172.21.15.44:0/2318528527 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555ff275c8c0 con 0x555ff2880070 2023-09-10T01:05:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.006+0000 7f1332ff5700 10 client.18720 handle_command_reply: tid=0 2023-09-10T01:05:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.006+0000 7f13612bcb80 1 client.18720 shutdown 2023-09-10T01:05:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.006+0000 7f13612bcb80 1 -- 172.21.15.44:0/2318528527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 msgr2=0x555ff28221f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.006+0000 7f13612bcb80 1 --2- 172.21.15.44:0/2318528527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 0x555ff28221f0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f1344096fb0 tx=0x7f134400f830 comp rx=0 tx=0).stop 2023-09-10T01:05:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.006+0000 7f13612bcb80 1 -- 172.21.15.44:0/2318528527 shutdown_connections 2023-09-10T01:05:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.006+0000 7f13612bcb80 1 --2- 172.21.15.44:0/2318528527 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555ff2880070 0x555ff2882490 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.006+0000 7f13612bcb80 1 --2- 172.21.15.44:0/2318528527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ff28171a0 0x555ff28200e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.006+0000 7f1332ff5700 0 client.18720 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.006+0000 7f13612bcb80 1 --2- 172.21.15.44:0/2318528527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ff2811300 0x555ff2821cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.006+0000 7f13612bcb80 1 --2- 172.21.15.44:0/2318528527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ff2811cc0 0x555ff28221f0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.006+0000 7f13612bcb80 1 -- 172.21.15.44:0/2318528527 >> 172.21.15.44:0/2318528527 conn(0x555ff2786030 msgr2=0x555ff2826c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.006+0000 7f13612bcb80 1 -- 172.21.15.44:0/2318528527 shutdown_connections 2023-09-10T01:05:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.006+0000 7f13612bcb80 1 -- 172.21.15.44:0/2318528527 wait complete. 2023-09-10T01:05:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.006+0000 7f13612bcb80 20 client.18720 trim_cache size 0 max 16384 2023-09-10T01:05:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:05:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:05:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:05:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:05:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.007+0000 7f13550f6700 1 -- 172.21.15.44:0/860233119 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f13380a07b0 msgr2=0x7f13380a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.007+0000 7f13550f6700 1 --2- 172.21.15.44:0/860233119 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f13380a07b0 0x7f13380a2c70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1340011490 tx=0x7f1340009260 comp rx=0 tx=0).stop 2023-09-10T01:05:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.007+0000 7f13550f6700 1 -- 172.21.15.44:0/860233119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13500ff460 msgr2=0x7f13501003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.007+0000 7f13550f6700 1 --2- 172.21.15.44:0/860233119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13500ff460 0x7f13501003a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1344000c00 tx=0x7f1344004730 comp rx=0 tx=0).stop 2023-09-10T01:05:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.007+0000 7f13550f6700 1 -- 172.21.15.44:0/860233119 shutdown_connections 2023-09-10T01:05:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.007+0000 7f13550f6700 1 --2- 172.21.15.44:0/860233119 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f13380a07b0 0x7f13380a2c70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.007+0000 7f13550f6700 1 --2- 172.21.15.44:0/860233119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1350106620 0x7f13501a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.007+0000 7f13550f6700 1 --2- 172.21.15.44:0/860233119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13500feaa0 0x7f13500ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.007+0000 7f13550f6700 1 --2- 172.21.15.44:0/860233119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13500ff460 0x7f13501003a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.007+0000 7f13550f6700 1 -- 172.21.15.44:0/860233119 >> 172.21.15.44:0/860233119 conn(0x7f13500769d0 msgr2=0x7f13500fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.007+0000 7f13550f6700 1 -- 172.21.15.44:0/860233119 shutdown_connections 2023-09-10T01:05:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:43.007+0000 7f13550f6700 1 -- 172.21.15.44:0/860233119 wait complete. 2023-09-10T01:05:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:42 smithi131 ceph-mon[103713]: pgmap v794: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T01:05:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:42 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:42 smithi044 ceph-mon[98676]: pgmap v794: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T01:05:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:42 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:42 smithi099 ceph-mon[102692]: pgmap v794: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T01:05:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:42 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:45.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:44 smithi131 ceph-mon[103713]: pgmap v795: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 152 op/s 2023-09-10T01:05:45.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:44 smithi044 ceph-mon[98676]: pgmap v795: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 152 op/s 2023-09-10T01:05:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:44 smithi099 ceph-mon[102692]: pgmap v795: 129 pgs: 129 active+clean; 9.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 152 op/s 2023-09-10T01:05:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:46 smithi131 ceph-mon[103713]: pgmap v796: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2023-09-10T01:05:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:46 smithi044 ceph-mon[98676]: pgmap v796: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2023-09-10T01:05:47.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:46 smithi099 ceph-mon[102692]: pgmap v796: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2023-09-10T01:05:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:48 smithi131 ceph-mon[103713]: pgmap v797: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 150 op/s 2023-09-10T01:05:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:48 smithi044 ceph-mon[98676]: pgmap v797: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 150 op/s 2023-09-10T01:05:49.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:48 smithi099 ceph-mon[102692]: pgmap v797: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 150 op/s 2023-09-10T01:05:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:50 smithi131 ceph-mon[103713]: pgmap v798: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 150 op/s 2023-09-10T01:05:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:50 smithi044 ceph-mon[98676]: pgmap v798: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 150 op/s 2023-09-10T01:05:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:50 smithi099 ceph-mon[102692]: pgmap v798: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 150 op/s 2023-09-10T01:05:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:52 smithi131 ceph-mon[103713]: pgmap v799: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 248 op/s 2023-09-10T01:05:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:52 smithi044 ceph-mon[98676]: pgmap v799: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 248 op/s 2023-09-10T01:05:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:52 smithi099 ceph-mon[102692]: pgmap v799: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 248 op/s 2023-09-10T01:05:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:54 smithi131 ceph-mon[103713]: pgmap v800: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 171 op/s 2023-09-10T01:05:55.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:54 smithi044 ceph-mon[98676]: pgmap v800: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 171 op/s 2023-09-10T01:05:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:54 smithi099 ceph-mon[102692]: pgmap v800: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 171 op/s 2023-09-10T01:05:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.130+0000 7f237d98b700 1 -- 172.21.15.44:0/3584694729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23780ff4b0 msgr2=0x7f23780ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.130+0000 7f237d98b700 1 --2- 172.21.15.44:0/3584694729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23780ff4b0 0x7f23780ff930 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f236c009990 tx=0x7f236c009ca0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.130+0000 7f237d98b700 1 -- 172.21.15.44:0/3584694729 shutdown_connections 2023-09-10T01:05:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.130+0000 7f237d98b700 1 --2- 172.21.15.44:0/3584694729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23780ffe70 0x7f237810c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.130+0000 7f237d98b700 1 --2- 172.21.15.44:0/3584694729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2378104410 0x7f23781047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.130+0000 7f237d98b700 1 --2- 172.21.15.44:0/3584694729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23780ff4b0 0x7f23780ff930 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.130+0000 7f237d98b700 1 -- 172.21.15.44:0/3584694729 >> 172.21.15.44:0/3584694729 conn(0x7f23780765b0 msgr2=0x7f23780769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.130+0000 7f237d98b700 1 -- 172.21.15.44:0/3584694729 shutdown_connections 2023-09-10T01:05:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.130+0000 7f237d98b700 1 -- 172.21.15.44:0/3584694729 wait complete. 2023-09-10T01:05:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.130+0000 7f237d98b700 1 Processor -- start 2023-09-10T01:05:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f237d98b700 1 -- start start 2023-09-10T01:05:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f237d98b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23780ff4b0 0x7f237819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f237d98b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23780ffe70 0x7f237819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f237d98b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2378104410 0x7f2378196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f237d98b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2378196ba0 con 0x7f23780ffe70 2023-09-10T01:05:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f237d98b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2378196d10 con 0x7f23780ff4b0 2023-09-10T01:05:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f237d98b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2378196ed0 con 0x7f2378104410 2023-09-10T01:05:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f23767fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23780ffe70 0x7f237819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f23767fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23780ffe70 0x7f237819cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55676/0 (socket says 172.21.15.44:55676) 2023-09-10T01:05:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f23767fc700 1 -- 172.21.15.44:0/736162743 learned_addr learned my addr 172.21.15.44:0/736162743 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f2376ffd700 1 --2- 172.21.15.44:0/736162743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23780ff4b0 0x7f237819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f23777fe700 1 --2- 172.21.15.44:0/736162743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2378104410 0x7f2378196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f23767fc700 1 -- 172.21.15.44:0/736162743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2378104410 msgr2=0x7f2378196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f23767fc700 1 --2- 172.21.15.44:0/736162743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2378104410 0x7f2378196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f23767fc700 1 -- 172.21.15.44:0/736162743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23780ff4b0 msgr2=0x7f237819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f23767fc700 1 --2- 172.21.15.44:0/736162743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23780ff4b0 0x7f237819c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f23767fc700 1 -- 172.21.15.44:0/736162743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c0095f0 con 0x7f23780ffe70 2023-09-10T01:05:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.131+0000 7f23767fc700 1 --2- 172.21.15.44:0/736162743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23780ffe70 0x7f237819cc10 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f236c000c00 tx=0x7f236c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.132+0000 7f237c989700 1 -- 172.21.15.44:0/736162743 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c01d070 con 0x7f23780ffe70 2023-09-10T01:05:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.132+0000 7f237d98b700 1 -- 172.21.15.44:0/736162743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2378197150 con 0x7f23780ffe70 2023-09-10T01:05:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.132+0000 7f237c989700 1 -- 172.21.15.44:0/736162743 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f236c022470 con 0x7f23780ffe70 2023-09-10T01:05:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.132+0000 7f237c989700 1 -- 172.21.15.44:0/736162743 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c017650 con 0x7f23780ffe70 2023-09-10T01:05:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.132+0000 7f237d98b700 1 -- 172.21.15.44:0/736162743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23781a36c0 con 0x7f23780ffe70 2023-09-10T01:05:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.134+0000 7f237c989700 1 -- 172.21.15.44:0/736162743 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f236c0225e0 con 0x7f23780ffe70 2023-09-10T01:05:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.135+0000 7f237c989700 1 --2- 172.21.15.44:0/736162743 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f23600a07b0 0x7f23600a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.135+0000 7f237c989700 1 -- 172.21.15.44:0/736162743 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f236c0cd930 con 0x7f23780ffe70 2023-09-10T01:05:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.135+0000 7f2376ffd700 1 --2- 172.21.15.44:0/736162743 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f23600a07b0 0x7f23600a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.136+0000 7f2376ffd700 1 --2- 172.21.15.44:0/736162743 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f23600a07b0 0x7f23600a2c70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2368009be0 tx=0x7f2368009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.136+0000 7f2389b51b80 1 Processor -- start 2023-09-10T01:05:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.136+0000 7f2389b51b80 1 -- start start 2023-09-10T01:05:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f2389b51b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e635e5b0 0x5603e635e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f2389b51b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e6358710 0x5603e6358b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f2389b51b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 0x5603e6365340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f2389b51b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603e62ad6d0 con 0x5603e635e5b0 2023-09-10T01:05:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f2389b51b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603e62aaff0 con 0x5603e6358710 2023-09-10T01:05:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f2389b51b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603e62a68e0 con 0x5603e63590d0 2023-09-10T01:05:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f2376ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e635e5b0 0x5603e635e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f23767fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e6358710 0x5603e6358b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f2376ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e635e5b0 0x5603e635e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55682/0 (socket says 172.21.15.44:55682) 2023-09-10T01:05:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f2376ffd700 1 -- 172.21.15.44:0/1928359083 learned_addr learned my addr 172.21.15.44:0/1928359083 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f23777fe700 1 --2- 172.21.15.44:0/1928359083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 0x5603e6365340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f23767fc700 1 -- 172.21.15.44:0/1928359083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 msgr2=0x5603e6365340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f23767fc700 1 --2- 172.21.15.44:0/1928359083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 0x5603e6365340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f23767fc700 1 -- 172.21.15.44:0/1928359083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e635e5b0 msgr2=0x5603e635e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f23767fc700 1 --2- 172.21.15.44:0/1928359083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e635e5b0 0x5603e635e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f23767fc700 1 -- 172.21.15.44:0/1928359083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603e60d2d80 con 0x5603e6358710 2023-09-10T01:05:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f2376ffd700 1 --2- 172.21.15.44:0/1928359083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e635e5b0 0x5603e635e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:05:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.137+0000 7f23767fc700 1 --2- 172.21.15.44:0/1928359083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e6358710 0x5603e6358b90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f236c0cace0 tx=0x7f236c096fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9146d2713c2b49df server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.138+0000 7f235dffb700 1 -- 172.21.15.44:0/1928359083 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c09a550 con 0x5603e6358710 2023-09-10T01:05:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.138+0000 7f235dffb700 1 -- 172.21.15.44:0/1928359083 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f236c0228c0 con 0x5603e6358710 2023-09-10T01:05:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.138+0000 7f235dffb700 1 -- 172.21.15.44:0/1928359083 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0a2c60 con 0x5603e6358710 2023-09-10T01:05:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.138+0000 7f2389b51b80 1 -- 172.21.15.44:0/1928359083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e6358710 msgr2=0x5603e6358b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.138+0000 7f2389b51b80 1 --2- 172.21.15.44:0/1928359083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e6358710 0x5603e6358b90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f236c0cace0 tx=0x7f236c096fb0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.138+0000 7f2389b51b80 1 -- 172.21.15.44:0/1928359083 shutdown_connections 2023-09-10T01:05:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.138+0000 7f2389b51b80 1 --2- 172.21.15.44:0/1928359083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 0x5603e6365340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.138+0000 7f2389b51b80 1 --2- 172.21.15.44:0/1928359083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e635e5b0 0x5603e635e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.138+0000 7f2389b51b80 1 --2- 172.21.15.44:0/1928359083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e6358710 0x5603e6358b90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.138+0000 7f2389b51b80 1 -- 172.21.15.44:0/1928359083 >> 172.21.15.44:0/1928359083 conn(0x5603e62c9030 msgr2=0x5603e634f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.138+0000 7f2389b51b80 1 -- 172.21.15.44:0/1928359083 shutdown_connections 2023-09-10T01:05:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.139+0000 7f2389b51b80 1 -- 172.21.15.44:0/1928359083 wait complete. 2023-09-10T01:05:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.139+0000 7f2389b51b80 1 -- start start 2023-09-10T01:05:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.139+0000 7f2389b51b80 1 Processor -- start 2023-09-10T01:05:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.139+0000 7f2389b51b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e6358710 0x5603e6367f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.139+0000 7f2389b51b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e63590d0 0x5603e6368450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.139+0000 7f2389b51b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e635e5b0 0x5603e63621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.139+0000 7f2389b51b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603e6244b10 con 0x5603e6358710 2023-09-10T01:05:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.139+0000 7f2389b51b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603e6244810 con 0x5603e63590d0 2023-09-10T01:05:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.139+0000 7f2389b51b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603e61288c0 con 0x5603e635e5b0 2023-09-10T01:05:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.139+0000 7f2389b51b80 10 client.0 _ll_register_callbacks cb 0x5603e618ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:05:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.139+0000 7f2376ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e6358710 0x5603e6367f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.139+0000 7f2376ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e6358710 0x5603e6367f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55688/0 (socket says 172.21.15.44:55688) 2023-09-10T01:05:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.139+0000 7f23767fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e63590d0 0x5603e6368450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.139+0000 7f2376ffd700 1 -- 172.21.15.44:0/3520712074 learned_addr learned my addr 172.21.15.44:0/3520712074 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.140+0000 7f23777fe700 1 --2- 172.21.15.44:0/3520712074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e635e5b0 0x5603e63621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.140+0000 7f2376ffd700 1 -- 172.21.15.44:0/3520712074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e635e5b0 msgr2=0x5603e63621d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.140+0000 7f2376ffd700 1 --2- 172.21.15.44:0/3520712074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e635e5b0 0x5603e63621d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.140+0000 7f2376ffd700 1 -- 172.21.15.44:0/3520712074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e63590d0 msgr2=0x5603e6368450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.140+0000 7f2376ffd700 1 --2- 172.21.15.44:0/3520712074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e63590d0 0x5603e6368450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.140+0000 7f2376ffd700 1 -- 172.21.15.44:0/3520712074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2368017070 con 0x5603e6358710 2023-09-10T01:05:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.140+0000 7f23767fc700 1 --2- 172.21.15.44:0/3520712074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e63590d0 0x5603e6368450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:05:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.140+0000 7f2376ffd700 1 --2- 172.21.15.44:0/3520712074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e6358710 0x5603e6367f10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f236801f900 tx=0x7f23680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c908efa9af5dfa3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.140+0000 7f2356ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:05:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.140+0000 7f2356ffd700 1 -- 172.21.15.44:0/3520712074 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236801d640 con 0x5603e6358710 2023-09-10T01:05:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.140+0000 7f2356ffd700 1 -- 172.21.15.44:0/3520712074 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f236801d7a0 con 0x5603e6358710 2023-09-10T01:05:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.140+0000 7f2389b51b80 1 -- 172.21.15.44:0/3520712074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603e61fe490 con 0x5603e6358710 2023-09-10T01:05:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.140+0000 7f2356ffd700 1 -- 172.21.15.44:0/3520712074 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f236801d9a0 con 0x5603e6358710 2023-09-10T01:05:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.140+0000 7f2389b51b80 10 client.18732 fetch_fsmap learned FSMap version 105 2023-09-10T01:05:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.140+0000 7f2389b51b80 1 -- 172.21.15.44:0/3520712074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x7f23781a36c0 con 0x5603e6358710 2023-09-10T01:05:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.141+0000 7f2356ffd700 1 -- 172.21.15.44:0/3520712074 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5603e61fe490 con 0x5603e6358710 2023-09-10T01:05:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.141+0000 7f2356ffd700 1 -- 172.21.15.44:0/3520712074 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 105) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2368028c00 con 0x5603e6358710 2023-09-10T01:05:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.141+0000 7f2389b51b80 10 client.18732 fetch_fsmap finished waiting for FSMap version 105 2023-09-10T01:05:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.141+0000 7f2389b51b80 10 client.18732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:05:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.141+0000 7f2389b51b80 20 client.18732 populate_metadata read hostname 'smithi044' 2023-09-10T01:05:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.141+0000 7f2389b51b80 1 --2- 172.21.15.44:0/3520712074 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5603e6372410 0x5603e6374830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.141+0000 7f2389b51b80 4 client.18732 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:05:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.141+0000 7f2389b51b80 1 -- 172.21.15.44:0/3520712074 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603e62bb840 con 0x5603e6372410 2023-09-10T01:05:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.141+0000 7f23767fc700 1 --2- 172.21.15.44:0/3520712074 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5603e6372410 0x5603e6374830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.142+0000 7f2356ffd700 10 client.18732 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.142+0000 7f23767fc700 1 --2- 172.21.15.44:0/3520712074 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5603e6372410 0x5603e6374830 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6919531056d4e6ac server_cookie=ac4e5ddee8b2fd65 in_seq=0 out_seq=0 2023-09-10T01:05:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.142+0000 7f2356ffd700 1 -- 172.21.15.44:0/3520712074 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5603e62bb840 con 0x5603e6372410 2023-09-10T01:05:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.142+0000 7f2356ffd700 10 client.18732 handle_command_reply: tid=0 2023-09-10T01:05:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.142+0000 7f2389b51b80 1 client.18732 shutdown 2023-09-10T01:05:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.142+0000 7f2389b51b80 1 -- 172.21.15.44:0/3520712074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e6358710 msgr2=0x5603e6367f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.142+0000 7f2389b51b80 1 --2- 172.21.15.44:0/3520712074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e6358710 0x5603e6367f10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f236801f900 tx=0x7f23680047c0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.143+0000 7f2389b51b80 1 -- 172.21.15.44:0/3520712074 shutdown_connections 2023-09-10T01:05:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.143+0000 7f2389b51b80 1 --2- 172.21.15.44:0/3520712074 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5603e6372410 0x5603e6374830 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.143+0000 7f2389b51b80 1 --2- 172.21.15.44:0/3520712074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e635e5b0 0x5603e63621d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.143+0000 7f2356ffd700 0 client.18732 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.143+0000 7f2389b51b80 1 --2- 172.21.15.44:0/3520712074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e6358710 0x5603e6367f10 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.143+0000 7f2389b51b80 1 --2- 172.21.15.44:0/3520712074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e63590d0 0x5603e6368450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.143+0000 7f2389b51b80 1 -- 172.21.15.44:0/3520712074 >> 172.21.15.44:0/3520712074 conn(0x5603e62c9030 msgr2=0x5603e634e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.143+0000 7f2389b51b80 1 -- 172.21.15.44:0/3520712074 shutdown_connections 2023-09-10T01:05:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.143+0000 7f2389b51b80 1 -- 172.21.15.44:0/3520712074 wait complete. 2023-09-10T01:05:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.143+0000 7f2389b51b80 20 client.18732 trim_cache size 0 max 16384 2023-09-10T01:05:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.158+0000 7f2389b51b80 1 Processor -- start 2023-09-10T01:05:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.158+0000 7f2389b51b80 1 -- start start 2023-09-10T01:05:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f2389b51b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e6358710 0x5603e634ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f2389b51b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 0x5603e634f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f2389b51b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e635e5b0 0x5603e635d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f2389b51b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603e634bfc0 con 0x5603e6358710 2023-09-10T01:05:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f2389b51b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603e6359990 con 0x5603e635e5b0 2023-09-10T01:05:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f2389b51b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603e6363740 con 0x5603e63590d0 2023-09-10T01:05:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f2376ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e6358710 0x5603e634ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f2376ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e6358710 0x5603e634ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55696/0 (socket says 172.21.15.44:55696) 2023-09-10T01:05:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f2376ffd700 1 -- 172.21.15.44:0/3643493426 learned_addr learned my addr 172.21.15.44:0/3643493426 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f23767fc700 1 --2- 172.21.15.44:0/3643493426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 0x5603e634f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f23777fe700 1 --2- 172.21.15.44:0/3643493426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e635e5b0 0x5603e635d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f2376ffd700 1 -- 172.21.15.44:0/3643493426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 msgr2=0x5603e634f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f2376ffd700 1 --2- 172.21.15.44:0/3643493426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 0x5603e634f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f2376ffd700 1 -- 172.21.15.44:0/3643493426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e635e5b0 msgr2=0x5603e635d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f2376ffd700 1 --2- 172.21.15.44:0/3643493426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e635e5b0 0x5603e635d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f2376ffd700 1 -- 172.21.15.44:0/3643493426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603e60bbc00 con 0x5603e6358710 2023-09-10T01:05:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.159+0000 7f2376ffd700 1 --2- 172.21.15.44:0/3643493426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e6358710 0x5603e634ef20 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f2368011850 tx=0x7f2368009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6f28bceffc3414 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.160+0000 7f235e7fc700 1 -- 172.21.15.44:0/3643493426 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236801d640 con 0x5603e6358710 2023-09-10T01:05:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.160+0000 7f235e7fc700 1 -- 172.21.15.44:0/3643493426 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f236801dc30 con 0x5603e6358710 2023-09-10T01:05:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.160+0000 7f235e7fc700 1 -- 172.21.15.44:0/3643493426 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368028c20 con 0x5603e6358710 2023-09-10T01:05:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.160+0000 7f2389b51b80 1 -- 172.21.15.44:0/3643493426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e6358710 msgr2=0x5603e634ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.160+0000 7f2389b51b80 1 --2- 172.21.15.44:0/3643493426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e6358710 0x5603e634ef20 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f2368011850 tx=0x7f2368009350 comp rx=0 tx=0).stop 2023-09-10T01:05:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.160+0000 7f2389b51b80 1 -- 172.21.15.44:0/3643493426 shutdown_connections 2023-09-10T01:05:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.161+0000 7f2389b51b80 1 --2- 172.21.15.44:0/3643493426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e635e5b0 0x5603e635d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.161+0000 7f2389b51b80 1 --2- 172.21.15.44:0/3643493426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e6358710 0x5603e634ef20 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.161+0000 7f2389b51b80 1 --2- 172.21.15.44:0/3643493426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 0x5603e634f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.161+0000 7f2389b51b80 1 -- 172.21.15.44:0/3643493426 >> 172.21.15.44:0/3643493426 conn(0x5603e62c9030 msgr2=0x5603e6362ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.161+0000 7f2389b51b80 1 -- 172.21.15.44:0/3643493426 shutdown_connections 2023-09-10T01:05:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.161+0000 7f2389b51b80 1 -- 172.21.15.44:0/3643493426 wait complete. 2023-09-10T01:05:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.161+0000 7f2389b51b80 1 -- start start 2023-09-10T01:05:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.161+0000 7f2389b51b80 1 Processor -- start 2023-09-10T01:05:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.161+0000 7f2389b51b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e6358710 0x5603e63ba1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.161+0000 7f2389b51b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 0x5603e63ba730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.161+0000 7f2389b51b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e635e5b0 0x5603e63bac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.161+0000 7f2389b51b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603e63c5350 con 0x5603e635e5b0 2023-09-10T01:05:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.161+0000 7f2389b51b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603e63c54c0 con 0x5603e6358710 2023-09-10T01:05:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.161+0000 7f2389b51b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603e63c5680 con 0x5603e63590d0 2023-09-10T01:05:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.161+0000 7f2389b51b80 10 client.0 _ll_register_callbacks cb 0x7f236801f4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:05:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f23767fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 0x5603e63ba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f23767fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 0x5603e63ba730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34354/0 (socket says 172.21.15.44:34354) 2023-09-10T01:05:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f2376ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e6358710 0x5603e63ba1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f23767fc700 1 -- 172.21.15.44:0/1576544253 learned_addr learned my addr 172.21.15.44:0/1576544253 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:05:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f23777fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e635e5b0 0x5603e63bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f23767fc700 1 -- 172.21.15.44:0/1576544253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e6358710 msgr2=0x5603e63ba1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f23767fc700 1 --2- 172.21.15.44:0/1576544253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e6358710 0x5603e63ba1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f23767fc700 1 -- 172.21.15.44:0/1576544253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e635e5b0 msgr2=0x5603e63bac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:05:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f23767fc700 1 --2- 172.21.15.44:0/1576544253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e635e5b0 0x5603e63bac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f23767fc700 1 -- 172.21.15.44:0/1576544253 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23781a36c0 con 0x5603e63590d0 2023-09-10T01:05:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f23767fc700 1 --2- 172.21.15.44:0/1576544253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 0x5603e63ba730 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f236c00f740 tx=0x7f236c00b310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fbe9f8758be2e89 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:05:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f2356ffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:05:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f2356ffd700 1 -- 172.21.15.44:0/1576544253 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c09a550 con 0x5603e63590d0 2023-09-10T01:05:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f2389b51b80 1 -- 172.21.15.44:0/1576544253 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f236c0228c0 con 0x5603e63590d0 2023-09-10T01:05:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f2356ffd700 1 -- 172.21.15.44:0/1576544253 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f236c09b2f0 con 0x5603e63590d0 2023-09-10T01:05:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f2356ffd700 1 -- 172.21.15.44:0/1576544253 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f236c09abf0 con 0x5603e63590d0 2023-09-10T01:05:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f2389b51b80 10 client.28330 fetch_fsmap learned FSMap version 105 2023-09-10T01:05:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.162+0000 7f2389b51b80 1 -- 172.21.15.44:0/1576544253 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x5603e60f91f0 con 0x5603e63590d0 2023-09-10T01:05:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.163+0000 7f2356ffd700 1 -- 172.21.15.44:0/1576544253 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f236c0228c0 con 0x5603e63590d0 2023-09-10T01:05:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.163+0000 7f2356ffd700 1 -- 172.21.15.44:0/1576544253 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 105) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f236c079be0 con 0x5603e63590d0 2023-09-10T01:05:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.163+0000 7f2389b51b80 10 client.28330 fetch_fsmap finished waiting for FSMap version 105 2023-09-10T01:05:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.163+0000 7f2389b51b80 10 client.28330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:05:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.163+0000 7f2389b51b80 20 client.28330 populate_metadata read hostname 'smithi044' 2023-09-10T01:05:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.163+0000 7f2389b51b80 1 --2- 172.21.15.44:0/1576544253 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5603e63c6910 0x5603e63c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:05:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.163+0000 7f2389b51b80 4 client.28330 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:05:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.163+0000 7f2389b51b80 1 -- 172.21.15.44:0/1576544253 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5603e629f8c0 con 0x5603e63c6910 2023-09-10T01:05:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.163+0000 7f2376ffd700 1 --2- 172.21.15.44:0/1576544253 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5603e63c6910 0x5603e63c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:05:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.164+0000 7f2356ffd700 10 client.28330 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.164+0000 7f2376ffd700 1 --2- 172.21.15.44:0/1576544253 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5603e63c6910 0x5603e63c8d00 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d58dfa5662280c0c server_cookie=6528812af7d00770 in_seq=0 out_seq=0 2023-09-10T01:05:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.174+0000 7f2356ffd700 1 -- 172.21.15.44:0/1576544253 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5603e629f8c0 con 0x5603e63c6910 2023-09-10T01:05:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.174+0000 7f2356ffd700 10 client.28330 handle_command_reply: tid=0 2023-09-10T01:05:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.174+0000 7f2389b51b80 1 client.28330 shutdown 2023-09-10T01:05:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.174+0000 7f2389b51b80 1 -- 172.21.15.44:0/1576544253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 msgr2=0x5603e63ba730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.174+0000 7f2389b51b80 1 --2- 172.21.15.44:0/1576544253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 0x5603e63ba730 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f236c00f740 tx=0x7f236c00b310 comp rx=0 tx=0).stop 2023-09-10T01:05:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.174+0000 7f2389b51b80 1 -- 172.21.15.44:0/1576544253 shutdown_connections 2023-09-10T01:05:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.174+0000 7f2389b51b80 1 --2- 172.21.15.44:0/1576544253 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5603e63c6910 0x5603e63c8d00 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.174+0000 7f2389b51b80 1 --2- 172.21.15.44:0/1576544253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603e635e5b0 0x5603e63bac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.174+0000 7f2356ffd700 0 client.28330 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:05:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.174+0000 7f2389b51b80 1 --2- 172.21.15.44:0/1576544253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603e6358710 0x5603e63ba1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.174+0000 7f2389b51b80 1 --2- 172.21.15.44:0/1576544253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603e63590d0 0x5603e63ba730 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.174+0000 7f2389b51b80 1 -- 172.21.15.44:0/1576544253 >> 172.21.15.44:0/1576544253 conn(0x5603e62c9030 msgr2=0x5603e6361a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.174+0000 7f2389b51b80 1 -- 172.21.15.44:0/1576544253 shutdown_connections 2023-09-10T01:05:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.174+0000 7f2389b51b80 1 -- 172.21.15.44:0/1576544253 wait complete. 2023-09-10T01:05:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.174+0000 7f2389b51b80 20 client.28330 trim_cache size 0 max 16384 2023-09-10T01:05:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:05:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:05:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:05:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:05:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.175+0000 7f237d98b700 1 -- 172.21.15.44:0/736162743 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f23600a07b0 msgr2=0x7f23600a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.175+0000 7f237d98b700 1 --2- 172.21.15.44:0/736162743 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f23600a07b0 0x7f23600a2c70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2368009be0 tx=0x7f2368009260 comp rx=0 tx=0).stop 2023-09-10T01:05:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.175+0000 7f237d98b700 1 -- 172.21.15.44:0/736162743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23780ffe70 msgr2=0x7f237819cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:05:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.175+0000 7f237d98b700 1 --2- 172.21.15.44:0/736162743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23780ffe70 0x7f237819cc10 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f236c000c00 tx=0x7f236c004730 comp rx=0 tx=0).stop 2023-09-10T01:05:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.175+0000 7f237d98b700 1 -- 172.21.15.44:0/736162743 shutdown_connections 2023-09-10T01:05:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.175+0000 7f237d98b700 1 --2- 172.21.15.44:0/736162743 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f23600a07b0 0x7f23600a2c70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.175+0000 7f237d98b700 1 --2- 172.21.15.44:0/736162743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2378104410 0x7f2378196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.175+0000 7f237d98b700 1 --2- 172.21.15.44:0/736162743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23780ff4b0 0x7f237819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.175+0000 7f237d98b700 1 --2- 172.21.15.44:0/736162743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23780ffe70 0x7f237819cc10 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:05:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.175+0000 7f237d98b700 1 -- 172.21.15.44:0/736162743 >> 172.21.15.44:0/736162743 conn(0x7f23780765b0 msgr2=0x7f2378068a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:05:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.175+0000 7f237d98b700 1 -- 172.21.15.44:0/736162743 shutdown_connections 2023-09-10T01:05:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:05:56.175+0000 7f237d98b700 1 -- 172.21.15.44:0/736162743 wait complete. 2023-09-10T01:05:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:56 smithi131 ceph-mon[103713]: pgmap v801: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2023-09-10T01:05:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:56 smithi044 ceph-mon[98676]: pgmap v801: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2023-09-10T01:05:57.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:56 smithi099 ceph-mon[102692]: pgmap v801: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2023-09-10T01:05:58.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:58 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:58.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:58 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:58.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:58 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:05:59 smithi131 ceph-mon[103713]: pgmap v802: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:05:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:05:59 smithi044 ceph-mon[98676]: pgmap v802: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:05:59.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:05:59 smithi099 ceph-mon[102692]: pgmap v802: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:06:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:01 smithi131 ceph-mon[103713]: pgmap v803: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:06:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:01 smithi044 ceph-mon[98676]: pgmap v803: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:06:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:01 smithi099 ceph-mon[102692]: pgmap v803: 129 pgs: 129 active+clean; 9.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:06:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:03 smithi131 ceph-mon[103713]: pgmap v804: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 269 op/s 2023-09-10T01:06:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:03 smithi044 ceph-mon[98676]: pgmap v804: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 269 op/s 2023-09-10T01:06:03.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:03 smithi099 ceph-mon[102692]: pgmap v804: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 269 op/s 2023-09-10T01:06:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:05 smithi131 ceph-mon[103713]: pgmap v805: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 171 op/s 2023-09-10T01:06:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:05 smithi044 ceph-mon[98676]: pgmap v805: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 171 op/s 2023-09-10T01:06:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:05 smithi099 ceph-mon[102692]: pgmap v805: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 171 op/s 2023-09-10T01:06:07.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:06 smithi099 ceph-mon[102692]: pgmap v806: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 270 op/s 2023-09-10T01:06:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:06 smithi131 ceph-mon[103713]: pgmap v806: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 270 op/s 2023-09-10T01:06:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:06 smithi044 ceph-mon[98676]: pgmap v806: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 270 op/s 2023-09-10T01:06:08.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:08 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:06:08.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:08 smithi131 ceph-mon[103713]: pgmap v807: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2023-09-10T01:06:08.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:08 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:06:08.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:08 smithi044 ceph-mon[98676]: pgmap v807: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2023-09-10T01:06:08.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:08 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:06:08.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:08 smithi099 ceph-mon[102692]: pgmap v807: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2023-09-10T01:06:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.377+0000 7f49c48df700 1 -- 172.21.15.44:0/377695155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c0107e90 msgr2=0x7f49c0108310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.377+0000 7f49c48df700 1 --2- 172.21.15.44:0/377695155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c0107e90 0x7f49c0108310 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009990 tx=0x7f49b0009ca0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.386+0000 7f49c48df700 1 -- 172.21.15.44:0/377695155 shutdown_connections 2023-09-10T01:06:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.386+0000 7f49c48df700 1 --2- 172.21.15.44:0/377695155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c0108850 0x7f49c01142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.386+0000 7f49c48df700 1 --2- 172.21.15.44:0/377695155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c010d4a0 0x7f49c010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.386+0000 7f49c48df700 1 --2- 172.21.15.44:0/377695155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c0107e90 0x7f49c0108310 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.386+0000 7f49c48df700 1 -- 172.21.15.44:0/377695155 >> 172.21.15.44:0/377695155 conn(0x7f49c006c6c0 msgr2=0x7f49c006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.388+0000 7f49c48df700 1 -- 172.21.15.44:0/377695155 shutdown_connections 2023-09-10T01:06:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.389+0000 7f49c48df700 1 -- 172.21.15.44:0/377695155 wait complete. 2023-09-10T01:06:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.389+0000 7f49c48df700 1 Processor -- start 2023-09-10T01:06:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.389+0000 7f49c48df700 1 -- start start 2023-09-10T01:06:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.390+0000 7f49c48df700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c0107e90 0x7f49c01ab520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.390+0000 7f49c48df700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c0108850 0x7f49c01aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.390+0000 7f49c48df700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c010d4a0 0x7f49c01a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.390+0000 7f49c48df700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c01a5a40 con 0x7f49c0108850 2023-09-10T01:06:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.390+0000 7f49c48df700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c01a5bb0 con 0x7f49c0107e90 2023-09-10T01:06:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.390+0000 7f49c48df700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c01a5d70 con 0x7f49c010d4a0 2023-09-10T01:06:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.390+0000 7f49bdd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c0108850 0x7f49c01aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.390+0000 7f49bdd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c0108850 0x7f49c01aba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55716/0 (socket says 172.21.15.44:55716) 2023-09-10T01:06:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.390+0000 7f49bdd9b700 1 -- 172.21.15.44:0/4197087621 learned_addr learned my addr 172.21.15.44:0/4197087621 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.390+0000 7f49bed9d700 1 --2- 172.21.15.44:0/4197087621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c010d4a0 0x7f49c01a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.391+0000 7f49bdd9b700 1 -- 172.21.15.44:0/4197087621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c010d4a0 msgr2=0x7f49c01a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.391+0000 7f49bdd9b700 1 --2- 172.21.15.44:0/4197087621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c010d4a0 0x7f49c01a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.391+0000 7f49bdd9b700 1 -- 172.21.15.44:0/4197087621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c0107e90 msgr2=0x7f49c01ab520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.391+0000 7f49bdd9b700 1 --2- 172.21.15.44:0/4197087621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c0107e90 0x7f49c01ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.391+0000 7f49bdd9b700 1 -- 172.21.15.44:0/4197087621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b00095f0 con 0x7f49c0108850 2023-09-10T01:06:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.391+0000 7f49bdd9b700 1 --2- 172.21.15.44:0/4197087621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c0108850 0x7f49c01aba60 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f49b00098e0 tx=0x7f49b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.391+0000 7f49af7fe700 1 -- 172.21.15.44:0/4197087621 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b001d070 con 0x7f49c0108850 2023-09-10T01:06:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.391+0000 7f49c48df700 1 -- 172.21.15.44:0/4197087621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c01a5ff0 con 0x7f49c0108850 2023-09-10T01:06:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.391+0000 7f49c48df700 1 -- 172.21.15.44:0/4197087621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c019ccb0 con 0x7f49c0108850 2023-09-10T01:06:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.391+0000 7f49af7fe700 1 -- 172.21.15.44:0/4197087621 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49b0004120 con 0x7f49c0108850 2023-09-10T01:06:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.391+0000 7f49af7fe700 1 -- 172.21.15.44:0/4197087621 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b000fdf0 con 0x7f49c0108850 2023-09-10T01:06:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.395+0000 7f49af7fe700 1 -- 172.21.15.44:0/4197087621 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f49b000f460 con 0x7f49c0108850 2023-09-10T01:06:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.395+0000 7f49af7fe700 1 --2- 172.21.15.44:0/4197087621 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f49a80a0880 0x7f49a80a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.396+0000 7f49af7fe700 1 -- 172.21.15.44:0/4197087621 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f49b00cd320 con 0x7f49c0108850 2023-09-10T01:06:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.396+0000 7f49d0aa5b80 1 Processor -- start 2023-09-10T01:06:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.396+0000 7f49d0aa5b80 1 -- start start 2023-09-10T01:06:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.397+0000 7f49be59c700 1 --2- 172.21.15.44:0/4197087621 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f49a80a0880 0x7f49a80a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.397+0000 7f49d0aa5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a7f1a0 0x55aed9a7f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.397+0000 7f49d0aa5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a79300 0x55aed9a79780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.397+0000 7f49d0aa5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a79cc0 0x55aed9a85f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.397+0000 7f49d0aa5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed99d26d0 con 0x55aed9a79cc0 2023-09-10T01:06:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.397+0000 7f49d0aa5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed99cfff0 con 0x55aed9a7f1a0 2023-09-10T01:06:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.397+0000 7f49d0aa5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed99cb8e0 con 0x55aed9a79300 2023-09-10T01:06:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.397+0000 7f49bed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a79cc0 0x55aed9a85f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.397+0000 7f49bed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a79cc0 0x55aed9a85f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55728/0 (socket says 172.21.15.44:55728) 2023-09-10T01:06:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.397+0000 7f49bed9d700 1 -- 172.21.15.44:0/3291790196 learned_addr learned my addr 172.21.15.44:0/3291790196 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.397+0000 7f49be59c700 1 --2- 172.21.15.44:0/3291790196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a7f1a0 0x55aed9a7f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.398+0000 7f49be59c700 1 --2- 172.21.15.44:0/4197087621 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f49a80a0880 0x7f49a80a2d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f49b800e4a0 tx=0x7f49b800e860 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.398+0000 7f49be59c700 1 -- 172.21.15.44:0/3291790196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a79300 msgr2=0x55aed9a79780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:06:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.398+0000 7f49be59c700 1 --2- 172.21.15.44:0/3291790196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a79300 0x55aed9a79780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.398+0000 7f49be59c700 1 -- 172.21.15.44:0/3291790196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a79cc0 msgr2=0x55aed9a85f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.398+0000 7f49be59c700 1 --2- 172.21.15.44:0/3291790196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a79cc0 0x55aed9a85f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.398+0000 7f49be59c700 1 -- 172.21.15.44:0/3291790196 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aed97f7d80 con 0x55aed9a7f1a0 2023-09-10T01:06:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.398+0000 7f49be59c700 1 --2- 172.21.15.44:0/3291790196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a7f1a0 0x55aed9a7f580 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f49b800a4c0 tx=0x7f49b8018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4101fb18ce467d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.398+0000 7f49acff9700 1 -- 172.21.15.44:0/3291790196 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8009e90 con 0x55aed9a7f1a0 2023-09-10T01:06:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.398+0000 7f49acff9700 1 -- 172.21.15.44:0/3291790196 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49b801f070 con 0x55aed9a7f1a0 2023-09-10T01:06:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.398+0000 7f49acff9700 1 -- 172.21.15.44:0/3291790196 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b800b680 con 0x55aed9a7f1a0 2023-09-10T01:06:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.399+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/3291790196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a7f1a0 msgr2=0x55aed9a7f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.399+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/3291790196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a7f1a0 0x55aed9a7f580 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f49b800a4c0 tx=0x7f49b8018000 comp rx=0 tx=0).stop 2023-09-10T01:06:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.399+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/3291790196 shutdown_connections 2023-09-10T01:06:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.399+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/3291790196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a79cc0 0x55aed9a85f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.399+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/3291790196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a7f1a0 0x55aed9a7f580 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.399+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/3291790196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a79300 0x55aed9a79780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.399+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/3291790196 >> 172.21.15.44:0/3291790196 conn(0x55aed99ee030 msgr2=0x55aed9a74560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.403+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/3291790196 shutdown_connections 2023-09-10T01:06:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.403+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/3291790196 wait complete. 2023-09-10T01:06:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.403+0000 7f49d0aa5b80 1 -- start start 2023-09-10T01:06:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.403+0000 7f49d0aa5b80 1 Processor -- start 2023-09-10T01:06:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.404+0000 7f49d0aa5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a79cc0 0x55aed9a85f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.404+0000 7f49d0aa5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a7f1a0 0x55aed9a83180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.404+0000 7f49d0aa5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a83820 0x55aed9ab15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.404+0000 7f49d0aa5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed9969b10 con 0x55aed9a79cc0 2023-09-10T01:06:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.404+0000 7f49d0aa5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed9969810 con 0x55aed9a83820 2023-09-10T01:06:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.404+0000 7f49d0aa5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed984d8c0 con 0x55aed9a7f1a0 2023-09-10T01:06:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.404+0000 7f49d0aa5b80 10 client.0 _ll_register_callbacks cb 0x55aed98b3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:06:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.404+0000 7f49be59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a79cc0 0x55aed9a85f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.404+0000 7f49be59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a79cc0 0x55aed9a85f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55730/0 (socket says 172.21.15.44:55730) 2023-09-10T01:06:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.404+0000 7f49be59c700 1 -- 172.21.15.44:0/1245041684 learned_addr learned my addr 172.21.15.44:0/1245041684 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.404+0000 7f49bed9d700 1 --2- 172.21.15.44:0/1245041684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a83820 0x55aed9ab15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.404+0000 7f49be59c700 1 -- 172.21.15.44:0/1245041684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a7f1a0 msgr2=0x55aed9a83180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.404+0000 7f49be59c700 1 --2- 172.21.15.44:0/1245041684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a7f1a0 0x55aed9a83180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.404+0000 7f49be59c700 1 -- 172.21.15.44:0/1245041684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a83820 msgr2=0x55aed9ab15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.404+0000 7f49be59c700 1 --2- 172.21.15.44:0/1245041684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a83820 0x55aed9ab15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.404+0000 7f49be59c700 1 -- 172.21.15.44:0/1245041684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b80093a0 con 0x55aed9a79cc0 2023-09-10T01:06:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.405+0000 7f49be59c700 1 --2- 172.21.15.44:0/1245041684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a79cc0 0x55aed9a85f00 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f49b801be70 tx=0x7f49b801b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7653ac4605595fda server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.409+0000 7f49a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:06:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.409+0000 7f49a5ffb700 1 -- 172.21.15.44:0/1245041684 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8024440 con 0x55aed9a79cc0 2023-09-10T01:06:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.409+0000 7f49a5ffb700 1 -- 172.21.15.44:0/1245041684 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f49b80245a0 con 0x55aed9a79cc0 2023-09-10T01:06:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.412+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/1245041684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aed9923490 con 0x55aed9a79cc0 2023-09-10T01:06:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.412+0000 7f49a5ffb700 1 -- 172.21.15.44:0/1245041684 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49b8024780 con 0x55aed9a79cc0 2023-09-10T01:06:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.414+0000 7f49d0aa5b80 10 client.18747 fetch_fsmap learned FSMap version 107 2023-09-10T01:06:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.414+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/1245041684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) v3 -- 0x55aed97e0c00 con 0x55aed9a79cc0 2023-09-10T01:06:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.415+0000 7f49a5ffb700 1 -- 172.21.15.44:0/1245041684 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aed9923490 con 0x55aed9a79cc0 2023-09-10T01:06:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.415+0000 7f49a5ffb700 1 -- 172.21.15.44:0/1245041684 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 107) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f49b800b940 con 0x55aed9a79cc0 2023-09-10T01:06:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.416+0000 7f49d0aa5b80 10 client.18747 fetch_fsmap finished waiting for FSMap version 107 2023-09-10T01:06:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.416+0000 7f49d0aa5b80 10 client.18747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:06:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.416+0000 7f49d0aa5b80 20 client.18747 populate_metadata read hostname 'smithi044' 2023-09-10T01:06:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.416+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/1245041684 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aed9ab5840 0x55aed9ab7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.416+0000 7f49d0aa5b80 4 client.18747 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:06:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.416+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/1245041684 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aed99e0840 con 0x55aed9ab5840 2023-09-10T01:06:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.422+0000 7f49bdd9b700 1 --2- 172.21.15.44:0/1245041684 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aed9ab5840 0x55aed9ab7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.439+0000 7f49a5ffb700 10 client.18747 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:06:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.439+0000 7f49bdd9b700 1 --2- 172.21.15.44:0/1245041684 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aed9ab5840 0x55aed9ab7c60 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a26e4090deeb4f12 server_cookie=2a161af57a3019ef in_seq=0 out_seq=0 2023-09-10T01:06:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.441+0000 7f49a5ffb700 1 -- 172.21.15.44:0/1245041684 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aed99e0840 con 0x55aed9ab5840 2023-09-10T01:06:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.441+0000 7f49a5ffb700 10 client.18747 handle_command_reply: tid=0 2023-09-10T01:06:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.441+0000 7f49d0aa5b80 1 client.18747 shutdown 2023-09-10T01:06:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.441+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/1245041684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a79cc0 msgr2=0x55aed9a85f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.441+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/1245041684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a79cc0 0x55aed9a85f00 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f49b801be70 tx=0x7f49b801b570 comp rx=0 tx=0).stop 2023-09-10T01:06:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.442+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/1245041684 shutdown_connections 2023-09-10T01:06:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.442+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/1245041684 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aed9ab5840 0x55aed9ab7c60 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.442+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/1245041684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a83820 0x55aed9ab15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.442+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/1245041684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a79cc0 0x55aed9a85f00 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.442+0000 7f49a5ffb700 0 client.18747 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:06:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.442+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/1245041684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a7f1a0 0x55aed9a83180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.442+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/1245041684 >> 172.21.15.44:0/1245041684 conn(0x55aed99ee030 msgr2=0x55aed9a737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.442+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/1245041684 shutdown_connections 2023-09-10T01:06:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.442+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/1245041684 wait complete. 2023-09-10T01:06:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.442+0000 7f49d0aa5b80 20 client.18747 trim_cache size 0 max 16384 2023-09-10T01:06:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.467+0000 7f49d0aa5b80 1 Processor -- start 2023-09-10T01:06:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.467+0000 7f49d0aa5b80 1 -- start start 2023-09-10T01:06:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.467+0000 7f49d0aa5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a79cc0 0x55aed9ab1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.467+0000 7f49d0aa5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a7f1a0 0x55aed9a80b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.467+0000 7f49d0aa5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a83820 0x55aed9a78a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.467+0000 7f49d0aa5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed9a7a560 con 0x55aed9a83820 2023-09-10T01:06:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.467+0000 7f49d0aa5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed9a81700 con 0x55aed9a7f1a0 2023-09-10T01:06:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.467+0000 7f49d0aa5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed9a7dd00 con 0x55aed9a79cc0 2023-09-10T01:06:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.467+0000 7f49be59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a79cc0 0x55aed9ab1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.467+0000 7f49bed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a83820 0x55aed9a78a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.468+0000 7f49bed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a83820 0x55aed9a78a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55738/0 (socket says 172.21.15.44:55738) 2023-09-10T01:06:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.468+0000 7f49bed9d700 1 -- 172.21.15.44:0/539172045 learned_addr learned my addr 172.21.15.44:0/539172045 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.468+0000 7f49bdd9b700 1 --2- 172.21.15.44:0/539172045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a7f1a0 0x55aed9a80b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.468+0000 7f49bdd9b700 1 -- 172.21.15.44:0/539172045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a79cc0 msgr2=0x55aed9ab1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.468+0000 7f49bdd9b700 1 --2- 172.21.15.44:0/539172045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a79cc0 0x55aed9ab1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.468+0000 7f49bdd9b700 1 -- 172.21.15.44:0/539172045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a83820 msgr2=0x55aed9a78a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.468+0000 7f49bdd9b700 1 --2- 172.21.15.44:0/539172045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a83820 0x55aed9a78a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.468+0000 7f49bdd9b700 1 -- 172.21.15.44:0/539172045 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aed981e1f0 con 0x55aed9a7f1a0 2023-09-10T01:06:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.468+0000 7f49bdd9b700 1 --2- 172.21.15.44:0/539172045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a7f1a0 0x55aed9a80b60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f49b00ca620 tx=0x7f49b0085040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a06168f437454c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.470+0000 7f49ad7fa700 1 -- 172.21.15.44:0/539172045 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0099f50 con 0x55aed9a7f1a0 2023-09-10T01:06:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.470+0000 7f49ad7fa700 1 -- 172.21.15.44:0/539172045 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49b0099d70 con 0x55aed9a7f1a0 2023-09-10T01:06:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.471+0000 7f49ad7fa700 1 -- 172.21.15.44:0/539172045 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00a23f0 con 0x55aed9a7f1a0 2023-09-10T01:06:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.472+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/539172045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a7f1a0 msgr2=0x55aed9a80b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.472+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/539172045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a7f1a0 0x55aed9a80b60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f49b00ca620 tx=0x7f49b0085040 comp rx=0 tx=0).stop 2023-09-10T01:06:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.475+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/539172045 shutdown_connections 2023-09-10T01:06:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.475+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/539172045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a83820 0x55aed9a78a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.475+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/539172045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a79cc0 0x55aed9ab1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.475+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/539172045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a7f1a0 0x55aed9a80b60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.475+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/539172045 >> 172.21.15.44:0/539172045 conn(0x55aed99ee030 msgr2=0x55aed9a782f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.484+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/539172045 shutdown_connections 2023-09-10T01:06:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.484+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/539172045 wait complete. 2023-09-10T01:06:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.484+0000 7f49d0aa5b80 1 -- start start 2023-09-10T01:06:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.484+0000 7f49d0aa5b80 1 Processor -- start 2023-09-10T01:06:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.485+0000 7f49d0aa5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a79cc0 0x55aed9aaa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.485+0000 7f49d0aa5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a7f1a0 0x55aed9aaaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.485+0000 7f49d0aa5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a83820 0x55aed9aaafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.485+0000 7f49d0aa5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed9b08690 con 0x55aed9a83820 2023-09-10T01:06:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.485+0000 7f49d0aa5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed9b087d0 con 0x55aed9a79cc0 2023-09-10T01:06:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.485+0000 7f49d0aa5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed9b08910 con 0x55aed9a7f1a0 2023-09-10T01:06:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.485+0000 7f49d0aa5b80 10 client.0 _ll_register_callbacks cb 0x7f49b400b220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:06:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.485+0000 7f49bed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a83820 0x55aed9aaafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.486+0000 7f49bed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a83820 0x55aed9aaafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55746/0 (socket says 172.21.15.44:55746) 2023-09-10T01:06:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.486+0000 7f49bed9d700 1 -- 172.21.15.44:0/482596992 learned_addr learned my addr 172.21.15.44:0/482596992 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.486+0000 7f49bed9d700 1 -- 172.21.15.44:0/482596992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a7f1a0 msgr2=0x55aed9aaaa60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:06:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.486+0000 7f49bed9d700 1 --2- 172.21.15.44:0/482596992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a7f1a0 0x55aed9aaaa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.486+0000 7f49bed9d700 1 -- 172.21.15.44:0/482596992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a79cc0 msgr2=0x55aed9aaa520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:06:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.486+0000 7f49bed9d700 1 --2- 172.21.15.44:0/482596992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a79cc0 0x55aed9aaa520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.486+0000 7f49bed9d700 1 -- 172.21.15.44:0/482596992 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c019ccb0 con 0x55aed9a83820 2023-09-10T01:06:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.487+0000 7f49bed9d700 1 --2- 172.21.15.44:0/482596992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a83820 0x55aed9aaafa0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f49b4015690 tx=0x7f49b4015a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8329727c1fc45c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.487+0000 7f49a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:06:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.487+0000 7f49a5ffb700 1 -- 172.21.15.44:0/482596992 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4016550 con 0x55aed9a83820 2023-09-10T01:06:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.487+0000 7f49a5ffb700 1 -- 172.21.15.44:0/482596992 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f49b40166b0 con 0x55aed9a83820 2023-09-10T01:06:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.487+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/482596992 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49b801f070 con 0x55aed9a83820 2023-09-10T01:06:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.488+0000 7f49a5ffb700 1 -- 172.21.15.44:0/482596992 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49b4016870 con 0x55aed9a83820 2023-09-10T01:06:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.488+0000 7f49d0aa5b80 10 client.18753 fetch_fsmap learned FSMap version 107 2023-09-10T01:06:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.488+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/482596992 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) v3 -- 0x55aed97f6250 con 0x55aed9a83820 2023-09-10T01:06:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.488+0000 7f49a5ffb700 1 -- 172.21.15.44:0/482596992 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49b801f070 con 0x55aed9a83820 2023-09-10T01:06:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.489+0000 7f49a5ffb700 1 -- 172.21.15.44:0/482596992 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 107) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f49b4029c10 con 0x55aed9a83820 2023-09-10T01:06:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.489+0000 7f49d0aa5b80 10 client.18753 fetch_fsmap finished waiting for FSMap version 107 2023-09-10T01:06:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.489+0000 7f49d0aa5b80 10 client.18753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:06:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.489+0000 7f49d0aa5b80 20 client.18753 populate_metadata read hostname 'smithi044' 2023-09-10T01:06:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.489+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/482596992 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aed9b0a0d0 0x55aed9b0c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.489+0000 7f49d0aa5b80 4 client.18753 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:06:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.489+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/482596992 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aed99c48c0 con 0x55aed9b0a0d0 2023-09-10T01:06:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.490+0000 7f49be59c700 1 --2- 172.21.15.44:0/482596992 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aed9b0a0d0 0x55aed9b0c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.490+0000 7f49a5ffb700 10 client.18753 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:06:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.490+0000 7f49be59c700 1 --2- 172.21.15.44:0/482596992 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aed9b0a0d0 0x55aed9b0c4f0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21903982811f0cf3 server_cookie=5cdcf1986780456f in_seq=0 out_seq=0 2023-09-10T01:06:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.610+0000 7f49a5ffb700 1 -- 172.21.15.44:0/482596992 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aed99c48c0 con 0x55aed9b0a0d0 2023-09-10T01:06:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.610+0000 7f49a5ffb700 10 client.18753 handle_command_reply: tid=0 2023-09-10T01:06:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.610+0000 7f49d0aa5b80 1 client.18753 shutdown 2023-09-10T01:06:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.614+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/482596992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a83820 msgr2=0x55aed9aaafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.614+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/482596992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a83820 0x55aed9aaafa0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f49b4015690 tx=0x7f49b4015a50 comp rx=0 tx=0).stop 2023-09-10T01:06:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.619+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/482596992 shutdown_connections 2023-09-10T01:06:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.619+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/482596992 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aed9b0a0d0 0x55aed9b0c4f0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.619+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/482596992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aed9a83820 0x55aed9aaafa0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.619+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/482596992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aed9a79cc0 0x55aed9aaa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.619+0000 7f49d0aa5b80 1 --2- 172.21.15.44:0/482596992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aed9a7f1a0 0x55aed9aaaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.619+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/482596992 >> 172.21.15.44:0/482596992 conn(0x55aed99ee030 msgr2=0x55aed9a78b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.619+0000 7f49a5ffb700 0 client.18753 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:06:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.622+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/482596992 shutdown_connections 2023-09-10T01:06:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.627+0000 7f49d0aa5b80 1 -- 172.21.15.44:0/482596992 wait complete. 2023-09-10T01:06:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:06:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:06:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:06:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:06:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.631+0000 7f49d0aa5b80 20 client.18753 trim_cache size 0 max 16384 2023-09-10T01:06:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.638+0000 7f49c48df700 1 -- 172.21.15.44:0/4197087621 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f49a80a0880 msgr2=0x7f49a80a2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.638+0000 7f49c48df700 1 --2- 172.21.15.44:0/4197087621 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f49a80a0880 0x7f49a80a2d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f49b800e4a0 tx=0x7f49b800e860 comp rx=0 tx=0).stop 2023-09-10T01:06:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.638+0000 7f49c48df700 1 -- 172.21.15.44:0/4197087621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c0108850 msgr2=0x7f49c01aba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.638+0000 7f49c48df700 1 --2- 172.21.15.44:0/4197087621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c0108850 0x7f49c01aba60 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f49b00098e0 tx=0x7f49b00057f0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.640+0000 7f49c48df700 1 -- 172.21.15.44:0/4197087621 shutdown_connections 2023-09-10T01:06:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.640+0000 7f49c48df700 1 --2- 172.21.15.44:0/4197087621 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f49a80a0880 0x7f49a80a2d40 secure :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f49b800e4a0 tx=0x7f49b800e860 comp rx=0 tx=0).stop 2023-09-10T01:06:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.640+0000 7f49c48df700 1 --2- 172.21.15.44:0/4197087621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c010d4a0 0x7f49c01a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.640+0000 7f49c48df700 1 --2- 172.21.15.44:0/4197087621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c0107e90 0x7f49c01ab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.640+0000 7f49c48df700 1 --2- 172.21.15.44:0/4197087621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c0108850 0x7f49c01aba60 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.640+0000 7f49c48df700 1 -- 172.21.15.44:0/4197087621 >> 172.21.15.44:0/4197087621 conn(0x7f49c006c6c0 msgr2=0x7f49c0070ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.647+0000 7f49c48df700 1 -- 172.21.15.44:0/4197087621 shutdown_connections 2023-09-10T01:06:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:09.650+0000 7f49c48df700 1 -- 172.21.15.44:0/4197087621 wait complete. 2023-09-10T01:06:10.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:10 smithi131 ceph-mon[103713]: osdmap e133: 12 total, 12 up, 12 in 2023-09-10T01:06:10.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:10 smithi131 ceph-mon[103713]: pgmap v809: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 230 op/s 2023-09-10T01:06:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:10 smithi044 ceph-mon[98676]: osdmap e133: 12 total, 12 up, 12 in 2023-09-10T01:06:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:10 smithi044 ceph-mon[98676]: pgmap v809: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 230 op/s 2023-09-10T01:06:10.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:10 smithi099 ceph-mon[102692]: osdmap e133: 12 total, 12 up, 12 in 2023-09-10T01:06:10.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:10 smithi099 ceph-mon[102692]: pgmap v809: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 230 op/s 2023-09-10T01:06:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:12 smithi099 ceph-mon[102692]: pgmap v810: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2023-09-10T01:06:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:12 smithi131 ceph-mon[103713]: pgmap v810: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2023-09-10T01:06:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:12 smithi044 ceph-mon[98676]: pgmap v810: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2023-09-10T01:06:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:14 smithi099 ceph-mon[102692]: pgmap v811: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2023-09-10T01:06:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:14 smithi131 ceph-mon[103713]: pgmap v811: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2023-09-10T01:06:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:14 smithi044 ceph-mon[98676]: pgmap v811: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2023-09-10T01:06:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:16 smithi131 ceph-mon[103713]: pgmap v812: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.0 MiB/s wr, 149 op/s 2023-09-10T01:06:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:16 smithi131 ceph-mon[103713]: osdmap e134: 12 total, 12 up, 12 in 2023-09-10T01:06:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:16 smithi099 ceph-mon[102692]: pgmap v812: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.0 MiB/s wr, 149 op/s 2023-09-10T01:06:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:16 smithi099 ceph-mon[102692]: osdmap e134: 12 total, 12 up, 12 in 2023-09-10T01:06:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:16 smithi044 ceph-mon[98676]: pgmap v812: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.0 MiB/s wr, 149 op/s 2023-09-10T01:06:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:16 smithi044 ceph-mon[98676]: osdmap e134: 12 total, 12 up, 12 in 2023-09-10T01:06:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:18 smithi099 ceph-mon[102692]: pgmap v814: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:06:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:18 smithi131 ceph-mon[103713]: pgmap v814: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:06:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:18 smithi044 ceph-mon[98676]: pgmap v814: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:06:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:20 smithi099 ceph-mon[102692]: pgmap v815: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.0 MiB/s wr, 149 op/s 2023-09-10T01:06:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:20 smithi131 ceph-mon[103713]: pgmap v815: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.0 MiB/s wr, 149 op/s 2023-09-10T01:06:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:20 smithi044 ceph-mon[98676]: pgmap v815: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.0 MiB/s wr, 149 op/s 2023-09-10T01:06:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.802+0000 7fcccfa28700 1 -- 172.21.15.44:0/2940015326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc8108fa0 msgr2=0x7fccc8109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.802+0000 7fcccfa28700 1 --2- 172.21.15.44:0/2940015326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc8108fa0 0x7fccc8109380 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009990 tx=0x7fccc4009ca0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.802+0000 7fcccfa28700 1 -- 172.21.15.44:0/2940015326 shutdown_connections 2023-09-10T01:06:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.802+0000 7fcccfa28700 1 --2- 172.21.15.44:0/2940015326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccc8112550 0x7fccc8114940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.802+0000 7fcccfa28700 1 --2- 172.21.15.44:0/2940015326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc8108fa0 0x7fccc8109380 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.802+0000 7fcccfa28700 1 --2- 172.21.15.44:0/2940015326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccc81098c0 0x7fccc8112010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.802+0000 7fcccfa28700 1 -- 172.21.15.44:0/2940015326 >> 172.21.15.44:0/2940015326 conn(0x7fccc806c6c0 msgr2=0x7fccc806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.803+0000 7fcccfa28700 1 -- 172.21.15.44:0/2940015326 shutdown_connections 2023-09-10T01:06:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.803+0000 7fcccfa28700 1 -- 172.21.15.44:0/2940015326 wait complete. 2023-09-10T01:06:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.804+0000 7fcccfa28700 1 Processor -- start 2023-09-10T01:06:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.804+0000 7fcccfa28700 1 -- start start 2023-09-10T01:06:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.804+0000 7fcccfa28700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc8108fa0 0x7fccc819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.804+0000 7fcccfa28700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccc81098c0 0x7fccc81a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.804+0000 7fcccfa28700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccc8112550 0x7fccc81aac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.804+0000 7fcccfa28700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc81ab260 con 0x7fccc81098c0 2023-09-10T01:06:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.804+0000 7fcccfa28700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc819da00 con 0x7fccc8108fa0 2023-09-10T01:06:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.804+0000 7fcccfa28700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc819dbc0 con 0x7fccc8112550 2023-09-10T01:06:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.804+0000 7fccccfc3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccc81098c0 0x7fccc81a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.804+0000 7fccccfc3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccc81098c0 0x7fccc81a0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55758/0 (socket says 172.21.15.44:55758) 2023-09-10T01:06:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.804+0000 7fccccfc3700 1 -- 172.21.15.44:0/2371922381 learned_addr learned my addr 172.21.15.44:0/2371922381 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.805+0000 7fccccfc3700 1 -- 172.21.15.44:0/2371922381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccc8112550 msgr2=0x7fccc81aac90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T01:06:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.805+0000 7fccccfc3700 1 --2- 172.21.15.44:0/2371922381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccc8112550 0x7fccc81aac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.805+0000 7fccccfc3700 1 -- 172.21.15.44:0/2371922381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc8108fa0 msgr2=0x7fccc819fdf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T01:06:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.805+0000 7fccccfc3700 1 --2- 172.21.15.44:0/2371922381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc8108fa0 0x7fccc819fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.805+0000 7fccccfc3700 1 -- 172.21.15.44:0/2371922381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc40095f0 con 0x7fccc81098c0 2023-09-10T01:06:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.805+0000 7fccccfc3700 1 --2- 172.21.15.44:0/2371922381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccc81098c0 0x7fccc81a0330 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fccb8007f10 tx=0x7fccb8009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.805+0000 7fccbe7fc700 1 -- 172.21.15.44:0/2371922381 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb8004220 con 0x7fccc81098c0 2023-09-10T01:06:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.805+0000 7fcccfa28700 1 -- 172.21.15.44:0/2371922381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc819dea0 con 0x7fccc81098c0 2023-09-10T01:06:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.805+0000 7fccbe7fc700 1 -- 172.21.15.44:0/2371922381 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fccb8009e60 con 0x7fccc81098c0 2023-09-10T01:06:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.805+0000 7fccbe7fc700 1 -- 172.21.15.44:0/2371922381 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb80054c0 con 0x7fccc81098c0 2023-09-10T01:06:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.805+0000 7fcccfa28700 1 -- 172.21.15.44:0/2371922381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc819e3b0 con 0x7fccc81098c0 2023-09-10T01:06:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.808+0000 7fccbe7fc700 1 -- 172.21.15.44:0/2371922381 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fccb8005620 con 0x7fccc81098c0 2023-09-10T01:06:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.809+0000 7fccbe7fc700 1 --2- 172.21.15.44:0/2371922381 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fccb40a07b0 0x7fccb40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.809+0000 7fccbe7fc700 1 -- 172.21.15.44:0/2371922381 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fccb80dc810 con 0x7fccc81098c0 2023-09-10T01:06:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.809+0000 7fcccd7c4700 1 --2- 172.21.15.44:0/2371922381 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fccb40a07b0 0x7fccb40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.810+0000 7fcccd7c4700 1 --2- 172.21.15.44:0/2371922381 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fccb40a07b0 0x7fccb40a2c70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fccc4005910 tx=0x7fccc4009510 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.812+0000 7fccdbbeeb80 1 Processor -- start 2023-09-10T01:06:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.812+0000 7fccdbbeeb80 1 -- start start 2023-09-10T01:06:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.812+0000 7fccdbbeeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720b41a0 0x5574720b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.812+0000 7fccdbbeeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720ae300 0x5574720ae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.812+0000 7fccdbbeeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720aecc0 0x5574720baf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.812+0000 7fccdbbeeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574720076d0 con 0x5574720b41a0 2023-09-10T01:06:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.812+0000 7fccdbbeeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557472004ff0 con 0x5574720aecc0 2023-09-10T01:06:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.812+0000 7fccdbbeeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574720008e0 con 0x5574720ae300 2023-09-10T01:06:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.812+0000 7fcccdfc5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720aecc0 0x5574720baf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.812+0000 7fccccfc3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720ae300 0x5574720ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.812+0000 7fcccdfc5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720aecc0 0x5574720baf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57504/0 (socket says 172.21.15.44:57504) 2023-09-10T01:06:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.812+0000 7fcccdfc5700 1 -- 172.21.15.44:0/3366027110 learned_addr learned my addr 172.21.15.44:0/3366027110 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.812+0000 7fcccd7c4700 1 --2- 172.21.15.44:0/3366027110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720b41a0 0x5574720b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.813+0000 7fcccdfc5700 1 -- 172.21.15.44:0/3366027110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720ae300 msgr2=0x5574720ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.813+0000 7fcccdfc5700 1 --2- 172.21.15.44:0/3366027110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720ae300 0x5574720ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.813+0000 7fcccdfc5700 1 -- 172.21.15.44:0/3366027110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720b41a0 msgr2=0x5574720b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.813+0000 7fcccdfc5700 1 --2- 172.21.15.44:0/3366027110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720b41a0 0x5574720b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.813+0000 7fcccdfc5700 1 -- 172.21.15.44:0/3366027110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557471e2cd80 con 0x5574720aecc0 2023-09-10T01:06:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.813+0000 7fccccfc3700 1 --2- 172.21.15.44:0/3366027110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720ae300 0x5574720ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:06:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.813+0000 7fcccd7c4700 1 --2- 172.21.15.44:0/3366027110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720b41a0 0x5574720b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:06:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.813+0000 7fcccdfc5700 1 --2- 172.21.15.44:0/3366027110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720aecc0 0x5574720baf30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fccc000e380 tx=0x7fccc000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e1d6b12024bec37 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.813+0000 7fccb37fe700 1 -- 172.21.15.44:0/3366027110 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0018070 con 0x5574720aecc0 2023-09-10T01:06:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.813+0000 7fccb37fe700 1 -- 172.21.15.44:0/3366027110 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fccc000ee20 con 0x5574720aecc0 2023-09-10T01:06:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.813+0000 7fccb37fe700 1 -- 172.21.15.44:0/3366027110 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0010040 con 0x5574720aecc0 2023-09-10T01:06:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.814+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/3366027110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720aecc0 msgr2=0x5574720baf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.814+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/3366027110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720aecc0 0x5574720baf30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fccc000e380 tx=0x7fccc000b040 comp rx=0 tx=0).stop 2023-09-10T01:06:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.814+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/3366027110 shutdown_connections 2023-09-10T01:06:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.814+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/3366027110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720aecc0 0x5574720baf30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.814+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/3366027110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720b41a0 0x5574720b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.814+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/3366027110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720ae300 0x5574720ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.814+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/3366027110 >> 172.21.15.44:0/3366027110 conn(0x557472023030 msgr2=0x5574720a9560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.814+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/3366027110 shutdown_connections 2023-09-10T01:06:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.814+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/3366027110 wait complete. 2023-09-10T01:06:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.815+0000 7fccdbbeeb80 1 -- start start 2023-09-10T01:06:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.815+0000 7fccdbbeeb80 1 Processor -- start 2023-09-10T01:06:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.815+0000 7fccdbbeeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720ae300 0x5574720badb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.815+0000 7fccdbbeeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720aecc0 0x5574720b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.815+0000 7fccdbbeeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720b41a0 0x5574720b8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.815+0000 7fccdbbeeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557471f9eb10 con 0x5574720aecc0 2023-09-10T01:06:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.815+0000 7fccdbbeeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557471f9e810 con 0x5574720b41a0 2023-09-10T01:06:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.815+0000 7fccdbbeeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557471e828c0 con 0x5574720ae300 2023-09-10T01:06:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.815+0000 7fccdbbeeb80 10 client.0 _ll_register_callbacks cb 0x557471ee8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:06:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.815+0000 7fcccd7c4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720ae300 0x5574720badb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.815+0000 7fcccd7c4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720ae300 0x5574720badb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34418/0 (socket says 172.21.15.44:34418) 2023-09-10T01:06:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.815+0000 7fcccd7c4700 1 -- 172.21.15.44:0/4133674186 learned_addr learned my addr 172.21.15.44:0/4133674186 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.815+0000 7fcccdfc5700 1 --2- 172.21.15.44:0/4133674186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720b41a0 0x5574720b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.816+0000 7fcccdfc5700 1 -- 172.21.15.44:0/4133674186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720ae300 msgr2=0x5574720badb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.816+0000 7fcccdfc5700 1 --2- 172.21.15.44:0/4133674186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720ae300 0x5574720badb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.816+0000 7fcccdfc5700 1 -- 172.21.15.44:0/4133674186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720aecc0 msgr2=0x5574720b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.816+0000 7fcccdfc5700 1 --2- 172.21.15.44:0/4133674186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720aecc0 0x5574720b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.816+0000 7fcccdfc5700 1 -- 172.21.15.44:0/4133674186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc819e3b0 con 0x5574720b41a0 2023-09-10T01:06:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.816+0000 7fcccd7c4700 1 --2- 172.21.15.44:0/4133674186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720ae300 0x5574720badb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:06:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.816+0000 7fcccdfc5700 1 --2- 172.21.15.44:0/4133674186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720b41a0 0x5574720b8660 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b010 tx=0x7fccc00092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e1dd14436b54e51 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.816+0000 7fccb0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:06:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.816+0000 7fccb0ff9700 1 -- 172.21.15.44:0/4133674186 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0022070 con 0x5574720b41a0 2023-09-10T01:06:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.816+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/4133674186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557471f58490 con 0x5574720b41a0 2023-09-10T01:06:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.816+0000 7fccb0ff9700 1 -- 172.21.15.44:0/4133674186 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fccc0004ce0 con 0x5574720b41a0 2023-09-10T01:06:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.816+0000 7fccb0ff9700 1 -- 172.21.15.44:0/4133674186 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccc0018070 con 0x5574720b41a0 2023-09-10T01:06:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.816+0000 7fccdbbeeb80 10 client.28409 fetch_fsmap learned FSMap version 107 2023-09-10T01:06:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.816+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/4133674186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) v3 -- 0x7fccc000e0c0 con 0x5574720b41a0 2023-09-10T01:06:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.816+0000 7fccb0ff9700 1 -- 172.21.15.44:0/4133674186 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557471f58490 con 0x5574720b41a0 2023-09-10T01:06:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.817+0000 7fccb0ff9700 1 -- 172.21.15.44:0/4133674186 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 107) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fccc0025020 con 0x5574720b41a0 2023-09-10T01:06:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.817+0000 7fccdbbeeb80 10 client.28409 fetch_fsmap finished waiting for FSMap version 107 2023-09-10T01:06:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.817+0000 7fccdbbeeb80 10 client.28409 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:06:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.817+0000 7fccdbbeeb80 20 client.28409 populate_metadata read hostname 'smithi044' 2023-09-10T01:06:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.817+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/4133674186 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5574720c8210 0x5574720ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.817+0000 7fccdbbeeb80 4 client.28409 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:06:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.817+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/4133674186 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557472015840 con 0x5574720c8210 2023-09-10T01:06:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.817+0000 7fcccd7c4700 1 --2- 172.21.15.44:0/4133674186 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5574720c8210 0x5574720ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.817+0000 7fccb0ff9700 10 client.28409 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:06:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.817+0000 7fcccd7c4700 1 --2- 172.21.15.44:0/4133674186 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5574720c8210 0x5574720ca600 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db3b36b3ac1f5d66 server_cookie=9539106dd5e96e68 in_seq=0 out_seq=0 2023-09-10T01:06:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.818+0000 7fccb0ff9700 1 -- 172.21.15.44:0/4133674186 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557472015840 con 0x5574720c8210 2023-09-10T01:06:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.818+0000 7fccb0ff9700 10 client.28409 handle_command_reply: tid=0 2023-09-10T01:06:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.818+0000 7fccdbbeeb80 1 client.28409 shutdown 2023-09-10T01:06:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.818+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/4133674186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720b41a0 msgr2=0x5574720b8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.818+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/4133674186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720b41a0 0x5574720b8660 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b010 tx=0x7fccc00092e0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.818+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/4133674186 shutdown_connections 2023-09-10T01:06:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.818+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/4133674186 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5574720c8210 0x5574720ca600 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.818+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/4133674186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720b41a0 0x5574720b8660 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.818+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/4133674186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720ae300 0x5574720badb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.818+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/4133674186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720aecc0 0x5574720b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.818+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/4133674186 >> 172.21.15.44:0/4133674186 conn(0x557472023030 msgr2=0x5574720a87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.818+0000 7fccb0ff9700 0 client.28409 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:06:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.818+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/4133674186 shutdown_connections 2023-09-10T01:06:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.818+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/4133674186 wait complete. 2023-09-10T01:06:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.819+0000 7fccdbbeeb80 20 client.28409 trim_cache size 0 max 16384 2023-09-10T01:06:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.837+0000 7fccdbbeeb80 1 Processor -- start 2023-09-10T01:06:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.837+0000 7fccdbbeeb80 1 -- start start 2023-09-10T01:06:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.838+0000 7fccdbbeeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720ae300 0x5574720bf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.838+0000 7fccdbbeeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720aecc0 0x5574720b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.838+0000 7fccdbbeeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720b41a0 0x5574720b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.838+0000 7fccdbbeeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557471f58490 con 0x5574720ae300 2023-09-10T01:06:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.838+0000 7fccdbbeeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574720a38a0 con 0x5574720aecc0 2023-09-10T01:06:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.838+0000 7fccdbbeeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574720aad10 con 0x5574720b41a0 2023-09-10T01:06:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.838+0000 7fcccdfc5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720b41a0 0x5574720b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.838+0000 7fcccd7c4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720ae300 0x5574720bf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.838+0000 7fcccdfc5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720b41a0 0x5574720b36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34426/0 (socket says 172.21.15.44:34426) 2023-09-10T01:06:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.838+0000 7fcccdfc5700 1 -- 172.21.15.44:0/299360123 learned_addr learned my addr 172.21.15.44:0/299360123 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.838+0000 7fccccfc3700 1 --2- 172.21.15.44:0/299360123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720aecc0 0x5574720b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.838+0000 7fcccdfc5700 1 -- 172.21.15.44:0/299360123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720aecc0 msgr2=0x5574720b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.838+0000 7fcccdfc5700 1 --2- 172.21.15.44:0/299360123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720aecc0 0x5574720b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.838+0000 7fcccdfc5700 1 -- 172.21.15.44:0/299360123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720ae300 msgr2=0x5574720bf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.838+0000 7fcccdfc5700 1 --2- 172.21.15.44:0/299360123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720ae300 0x5574720bf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.838+0000 7fcccdfc5700 1 -- 172.21.15.44:0/299360123 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557471e15c00 con 0x5574720b41a0 2023-09-10T01:06:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.839+0000 7fcccdfc5700 1 --2- 172.21.15.44:0/299360123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720b41a0 0x5574720b36a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fccc00021d0 tx=0x7fccc0009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3abb86a3701b554e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.839+0000 7fccb3fff700 1 -- 172.21.15.44:0/299360123 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0022070 con 0x5574720b41a0 2023-09-10T01:06:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.839+0000 7fccb3fff700 1 -- 172.21.15.44:0/299360123 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fccc0019040 con 0x5574720b41a0 2023-09-10T01:06:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.839+0000 7fccb3fff700 1 -- 172.21.15.44:0/299360123 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0018040 con 0x5574720b41a0 2023-09-10T01:06:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.840+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/299360123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720b41a0 msgr2=0x5574720b36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.840+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/299360123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720b41a0 0x5574720b36a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fccc00021d0 tx=0x7fccc0009ed0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.840+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/299360123 shutdown_connections 2023-09-10T01:06:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.840+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/299360123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720b41a0 0x5574720b36a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.840+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/299360123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720ae300 0x5574720bf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.840+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/299360123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720aecc0 0x5574720b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.840+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/299360123 >> 172.21.15.44:0/299360123 conn(0x557472023030 msgr2=0x5574720b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.841+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/299360123 shutdown_connections 2023-09-10T01:06:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.841+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/299360123 wait complete. 2023-09-10T01:06:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.841+0000 7fccdbbeeb80 1 -- start start 2023-09-10T01:06:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.841+0000 7fccdbbeeb80 1 Processor -- start 2023-09-10T01:06:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.841+0000 7fccdbbeeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720ae300 0x557472110190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.841+0000 7fccdbbeeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720aecc0 0x5574721106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.841+0000 7fccdbbeeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720b41a0 0x55747211b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.841+0000 7fccdbbeeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557472110dc0 con 0x5574720aecc0 2023-09-10T01:06:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.841+0000 7fccdbbeeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747211b560 con 0x5574720ae300 2023-09-10T01:06:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.841+0000 7fccdbbeeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747211b6a0 con 0x5574720b41a0 2023-09-10T01:06:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.841+0000 7fccdbbeeb80 10 client.0 _ll_register_callbacks cb 0x7fccc000e250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:06:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.842+0000 7fcccd7c4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720ae300 0x557472110190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.842+0000 7fccccfc3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720aecc0 0x5574721106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.842+0000 7fcccd7c4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720ae300 0x557472110190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57520/0 (socket says 172.21.15.44:57520) 2023-09-10T01:06:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.842+0000 7fcccd7c4700 1 -- 172.21.15.44:0/3084118221 learned_addr learned my addr 172.21.15.44:0/3084118221 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.842+0000 7fcccdfc5700 1 --2- 172.21.15.44:0/3084118221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720b41a0 0x55747211b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.842+0000 7fcccd7c4700 1 -- 172.21.15.44:0/3084118221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720b41a0 msgr2=0x55747211b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.842+0000 7fcccd7c4700 1 --2- 172.21.15.44:0/3084118221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720b41a0 0x55747211b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.842+0000 7fcccd7c4700 1 -- 172.21.15.44:0/3084118221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720aecc0 msgr2=0x5574721106d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.842+0000 7fcccd7c4700 1 --2- 172.21.15.44:0/3084118221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720aecc0 0x5574721106d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.842+0000 7fcccd7c4700 1 -- 172.21.15.44:0/3084118221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc000e0c0 con 0x5574720ae300 2023-09-10T01:06:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.842+0000 7fcccd7c4700 1 --2- 172.21.15.44:0/3084118221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720ae300 0x557472110190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5574720caee0 tx=0x7fccc4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f9c6144e8fc3816 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.842+0000 7fccb0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:06:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.842+0000 7fccb0ff9700 1 -- 172.21.15.44:0/3084118221 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc401d070 con 0x5574720ae300 2023-09-10T01:06:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.842+0000 7fccb0ff9700 1 -- 172.21.15.44:0/3084118221 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fccc4032cd0 con 0x5574720ae300 2023-09-10T01:06:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.842+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/3084118221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccc000ee20 con 0x5574720ae300 2023-09-10T01:06:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.843+0000 7fccb0ff9700 1 -- 172.21.15.44:0/3084118221 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccc4032e90 con 0x5574720ae300 2023-09-10T01:06:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.843+0000 7fccdbbeeb80 10 client.28415 fetch_fsmap learned FSMap version 107 2023-09-10T01:06:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.843+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/3084118221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) v3 -- 0x557471e531f0 con 0x5574720ae300 2023-09-10T01:06:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.843+0000 7fccb0ff9700 1 -- 172.21.15.44:0/3084118221 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fccc000ee20 con 0x5574720ae300 2023-09-10T01:06:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.843+0000 7fccb0ff9700 1 -- 172.21.15.44:0/3084118221 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 107) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fccc4004c80 con 0x5574720ae300 2023-09-10T01:06:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.843+0000 7fccdbbeeb80 10 client.28415 fetch_fsmap finished waiting for FSMap version 107 2023-09-10T01:06:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.843+0000 7fccdbbeeb80 10 client.28415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:06:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.843+0000 7fccdbbeeb80 20 client.28415 populate_metadata read hostname 'smithi044' 2023-09-10T01:06:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.843+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/3084118221 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55747211ce60 0x55747211f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.843+0000 7fccdbbeeb80 4 client.28415 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:06:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.843+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/3084118221 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557471ff98c0 con 0x55747211ce60 2023-09-10T01:06:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.843+0000 7fccccfc3700 1 --2- 172.21.15.44:0/3084118221 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55747211ce60 0x55747211f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.844+0000 7fccb0ff9700 10 client.28415 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:06:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.844+0000 7fccccfc3700 1 --2- 172.21.15.44:0/3084118221 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55747211ce60 0x55747211f280 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc3b2c5ff0d36624 server_cookie=564e9f9fae6a6361 in_seq=0 out_seq=0 2023-09-10T01:06:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.855+0000 7fccb0ff9700 1 -- 172.21.15.44:0/3084118221 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557471ff98c0 con 0x55747211ce60 2023-09-10T01:06:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.855+0000 7fccb0ff9700 10 client.28415 handle_command_reply: tid=0 2023-09-10T01:06:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.855+0000 7fccdbbeeb80 1 client.28415 shutdown 2023-09-10T01:06:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.856+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/3084118221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720ae300 msgr2=0x557472110190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.856+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/3084118221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720ae300 0x557472110190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5574720caee0 tx=0x7fccc4009310 comp rx=0 tx=0).stop 2023-09-10T01:06:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.856+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/3084118221 shutdown_connections 2023-09-10T01:06:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.856+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/3084118221 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55747211ce60 0x55747211f280 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.856+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/3084118221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574720b41a0 0x55747211b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.856+0000 7fccb0ff9700 0 client.28415 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:06:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.856+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/3084118221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574720ae300 0x557472110190 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.856+0000 7fccdbbeeb80 1 --2- 172.21.15.44:0/3084118221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574720aecc0 0x5574721106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.856+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/3084118221 >> 172.21.15.44:0/3084118221 conn(0x557472023030 msgr2=0x5574720b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.856+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/3084118221 shutdown_connections 2023-09-10T01:06:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.856+0000 7fccdbbeeb80 1 -- 172.21.15.44:0/3084118221 wait complete. 2023-09-10T01:06:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.856+0000 7fccdbbeeb80 20 client.28415 trim_cache size 0 max 16384 2023-09-10T01:06:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:06:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:06:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:06:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:06:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.857+0000 7fcccfa28700 1 -- 172.21.15.44:0/2371922381 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fccb40a07b0 msgr2=0x7fccb40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.857+0000 7fcccfa28700 1 --2- 172.21.15.44:0/2371922381 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fccb40a07b0 0x7fccb40a2c70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fccc4005910 tx=0x7fccc4009510 comp rx=0 tx=0).stop 2023-09-10T01:06:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.857+0000 7fcccfa28700 1 -- 172.21.15.44:0/2371922381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccc81098c0 msgr2=0x7fccc81a0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.857+0000 7fcccfa28700 1 --2- 172.21.15.44:0/2371922381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccc81098c0 0x7fccc81a0330 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fccb8007f10 tx=0x7fccb8009520 comp rx=0 tx=0).stop 2023-09-10T01:06:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.857+0000 7fcccfa28700 1 -- 172.21.15.44:0/2371922381 shutdown_connections 2023-09-10T01:06:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.857+0000 7fcccfa28700 1 --2- 172.21.15.44:0/2371922381 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fccb40a07b0 0x7fccb40a2c70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.857+0000 7fcccfa28700 1 --2- 172.21.15.44:0/2371922381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccc8112550 0x7fccc81aac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.857+0000 7fcccfa28700 1 --2- 172.21.15.44:0/2371922381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc8108fa0 0x7fccc819fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.857+0000 7fcccfa28700 1 --2- 172.21.15.44:0/2371922381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccc81098c0 0x7fccc81a0330 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.857+0000 7fcccfa28700 1 -- 172.21.15.44:0/2371922381 >> 172.21.15.44:0/2371922381 conn(0x7fccc806c6c0 msgr2=0x7fccc8070490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.857+0000 7fcccfa28700 1 -- 172.21.15.44:0/2371922381 shutdown_connections 2023-09-10T01:06:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:22.857+0000 7fcccfa28700 1 -- 172.21.15.44:0/2371922381 wait complete. 2023-09-10T01:06:23.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:22 smithi099 ceph-mon[102692]: pgmap v816: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2023-09-10T01:06:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:22 smithi131 ceph-mon[103713]: pgmap v816: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2023-09-10T01:06:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:22 smithi044 ceph-mon[98676]: pgmap v816: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2023-09-10T01:06:25.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:24 smithi099 ceph-mon[102692]: pgmap v817: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2023-09-10T01:06:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:24 smithi131 ceph-mon[103713]: pgmap v817: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2023-09-10T01:06:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:24 smithi044 ceph-mon[98676]: pgmap v817: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2023-09-10T01:06:26.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:25 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:06:26.543 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:25 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:06:26.544 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:25 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:06:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:26 smithi131 ceph-mon[103713]: pgmap v818: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:06:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:26 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:06:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:26 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:06:27.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:26 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:06:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:26 smithi044 ceph-mon[98676]: pgmap v818: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:06:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:26 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:06:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:26 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:06:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:26 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:06:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:26 smithi099 ceph-mon[102692]: pgmap v818: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:06:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:26 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:06:27.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:26 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:06:27.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:26 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:06:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:28 smithi131 ceph-mon[103713]: pgmap v819: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 954 KiB/s wr, 136 op/s 2023-09-10T01:06:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:28 smithi044 ceph-mon[98676]: pgmap v819: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 954 KiB/s wr, 136 op/s 2023-09-10T01:06:29.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:28 smithi099 ceph-mon[102692]: pgmap v819: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 954 KiB/s wr, 136 op/s 2023-09-10T01:06:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:30 smithi131 ceph-mon[103713]: pgmap v820: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 920 KiB/s wr, 132 op/s 2023-09-10T01:06:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:30 smithi044 ceph-mon[98676]: pgmap v820: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 920 KiB/s wr, 132 op/s 2023-09-10T01:06:31.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:30 smithi099 ceph-mon[102692]: pgmap v820: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 920 KiB/s wr, 132 op/s 2023-09-10T01:06:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:32 smithi131 ceph-mon[103713]: pgmap v821: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:06:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:32 smithi044 ceph-mon[98676]: pgmap v821: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:06:33.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:32 smithi099 ceph-mon[102692]: pgmap v821: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:06:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:34 smithi131 ceph-mon[103713]: pgmap v822: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 949 KiB/s wr, 136 op/s 2023-09-10T01:06:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:34 smithi044 ceph-mon[98676]: pgmap v822: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 949 KiB/s wr, 136 op/s 2023-09-10T01:06:35.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:34 smithi099 ceph-mon[102692]: pgmap v822: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 949 KiB/s wr, 136 op/s 2023-09-10T01:06:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.994+0000 7f80e73df700 1 -- 172.21.15.44:0/1279458313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80e01055b0 msgr2=0x7f80e0105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.994+0000 7f80e73df700 1 --2- 172.21.15.44:0/1279458313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80e01055b0 0x7f80e0105990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0098f0 tx=0x7f80dc009c00 comp rx=0 tx=0).stop 2023-09-10T01:06:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.994+0000 7f80e73df700 1 -- 172.21.15.44:0/1279458313 shutdown_connections 2023-09-10T01:06:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.994+0000 7f80e73df700 1 --2- 172.21.15.44:0/1279458313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80e0100120 0x7f80e010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.994+0000 7f80e73df700 1 --2- 172.21.15.44:0/1279458313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80e01055b0 0x7f80e0105990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.994+0000 7f80e73df700 1 --2- 172.21.15.44:0/1279458313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e00ff760 0x7f80e00ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.994+0000 7f80e73df700 1 -- 172.21.15.44:0/1279458313 >> 172.21.15.44:0/1279458313 conn(0x7f80e0074f30 msgr2=0x7f80e0075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.994+0000 7f80e73df700 1 -- 172.21.15.44:0/1279458313 shutdown_connections 2023-09-10T01:06:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.994+0000 7f80e73df700 1 -- 172.21.15.44:0/1279458313 wait complete. 2023-09-10T01:06:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e73df700 1 Processor -- start 2023-09-10T01:06:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e73df700 1 -- start start 2023-09-10T01:06:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e73df700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80e00ff760 0x7f80e01012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e73df700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e0100120 0x7f80e01017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e73df700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80e01055b0 0x7f80e01a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e73df700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e01a4cd0 con 0x7f80e0100120 2023-09-10T01:06:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e73df700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e01a4e10 con 0x7f80e01055b0 2023-09-10T01:06:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e73df700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e01a4f80 con 0x7f80e00ff760 2023-09-10T01:06:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e497a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e0100120 0x7f80e01017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e517b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80e00ff760 0x7f80e01012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e517b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80e00ff760 0x7f80e01012a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34442/0 (socket says 172.21.15.44:34442) 2023-09-10T01:06:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e517b700 1 -- 172.21.15.44:0/2782809687 learned_addr learned my addr 172.21.15.44:0/2782809687 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e497a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e0100120 0x7f80e01017e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55794/0 (socket says 172.21.15.44:55794) 2023-09-10T01:06:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e597c700 1 --2- 172.21.15.44:0/2782809687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80e01055b0 0x7f80e01a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e517b700 1 -- 172.21.15.44:0/2782809687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80e01055b0 msgr2=0x7f80e01a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e517b700 1 --2- 172.21.15.44:0/2782809687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80e01055b0 0x7f80e01a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e517b700 1 -- 172.21.15.44:0/2782809687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e0100120 msgr2=0x7f80e01017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.995+0000 7f80e517b700 1 --2- 172.21.15.44:0/2782809687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e0100120 0x7f80e01017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.996+0000 7f80e517b700 1 -- 172.21.15.44:0/2782809687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80dc009520 con 0x7f80e00ff760 2023-09-10T01:06:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.996+0000 7f80e517b700 1 --2- 172.21.15.44:0/2782809687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80e00ff760 0x7f80e01012a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009810 tx=0x7f80dc00b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.996+0000 7f80d67fc700 1 -- 172.21.15.44:0/2782809687 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc00ba10 con 0x7f80e00ff760 2023-09-10T01:06:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.996+0000 7f80e73df700 1 -- 172.21.15.44:0/2782809687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e01a5200 con 0x7f80e00ff760 2023-09-10T01:06:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.996+0000 7f80d67fc700 1 -- 172.21.15.44:0/2782809687 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80dc00bb70 con 0x7f80e00ff760 2023-09-10T01:06:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.996+0000 7f80d67fc700 1 -- 172.21.15.44:0/2782809687 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc004c90 con 0x7f80e00ff760 2023-09-10T01:06:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.996+0000 7f80e73df700 1 -- 172.21.15.44:0/2782809687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e01a5620 con 0x7f80e00ff760 2023-09-10T01:06:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.999+0000 7f80d67fc700 1 -- 172.21.15.44:0/2782809687 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f80dc005040 con 0x7f80e00ff760 2023-09-10T01:06:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:35.999+0000 7f80d67fc700 1 --2- 172.21.15.44:0/2782809687 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80cc0a07b0 0x7f80cc0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.000+0000 7f80e497a700 1 --2- 172.21.15.44:0/2782809687 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80cc0a07b0 0x7f80cc0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.000+0000 7f80d67fc700 1 -- 172.21.15.44:0/2782809687 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f80dc0cd1f0 con 0x7f80e00ff760 2023-09-10T01:06:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.000+0000 7f80e497a700 1 --2- 172.21.15.44:0/2782809687 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80cc0a07b0 0x7f80cc0a2c70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f80e0102ac0 tx=0x7f80d0009310 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.000+0000 7f80f35a5b80 1 Processor -- start 2023-09-10T01:06:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80f35a5b80 1 -- start start 2023-09-10T01:06:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80f35a5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd101a0 0x55f55bd10580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80f35a5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0a300 0x55f55bd0a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80f35a5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd0acc0 0x55f55bd16f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80f35a5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55bc636d0 con 0x55f55bd101a0 2023-09-10T01:06:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80f35a5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55bc60ff0 con 0x55f55bd0acc0 2023-09-10T01:06:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80f35a5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55bc5c8e0 con 0x55f55bd0a300 2023-09-10T01:06:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80e497a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0a300 0x55f55bd0a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80e597c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd0acc0 0x55f55bd16f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80e497a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0a300 0x55f55bd0a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34452/0 (socket says 172.21.15.44:34452) 2023-09-10T01:06:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80e497a700 1 -- 172.21.15.44:0/1018266165 learned_addr learned my addr 172.21.15.44:0/1018266165 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80e597c700 1 -- 172.21.15.44:0/1018266165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0a300 msgr2=0x55f55bd0a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80e597c700 1 --2- 172.21.15.44:0/1018266165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0a300 0x55f55bd0a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80e597c700 1 -- 172.21.15.44:0/1018266165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd101a0 msgr2=0x55f55bd10580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80e597c700 1 --2- 172.21.15.44:0/1018266165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd101a0 0x55f55bd10580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80e597c700 1 -- 172.21.15.44:0/1018266165 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f55ba88d80 con 0x55f55bd0acc0 2023-09-10T01:06:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.001+0000 7f80e497a700 1 --2- 172.21.15.44:0/1018266165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0a300 0x55f55bd0a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:06:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.002+0000 7f80e597c700 1 --2- 172.21.15.44:0/1018266165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd0acc0 0x55f55bd16f30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f80d8011580 tx=0x7f80d8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9211563d184d3ef server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.002+0000 7f80cb7fe700 1 -- 172.21.15.44:0/1018266165 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d800e040 con 0x55f55bd0acc0 2023-09-10T01:06:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.002+0000 7f80cb7fe700 1 -- 172.21.15.44:0/1018266165 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80d800f040 con 0x55f55bd0acc0 2023-09-10T01:06:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.002+0000 7f80cb7fe700 1 -- 172.21.15.44:0/1018266165 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8004320 con 0x55f55bd0acc0 2023-09-10T01:06:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 -- 172.21.15.44:0/1018266165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd0acc0 msgr2=0x55f55bd16f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/1018266165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd0acc0 0x55f55bd16f30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f80d8011580 tx=0x7f80d8009920 comp rx=0 tx=0).stop 2023-09-10T01:06:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 -- 172.21.15.44:0/1018266165 shutdown_connections 2023-09-10T01:06:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/1018266165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd0acc0 0x55f55bd16f30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/1018266165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd101a0 0x55f55bd10580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/1018266165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0a300 0x55f55bd0a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 -- 172.21.15.44:0/1018266165 >> 172.21.15.44:0/1018266165 conn(0x55f55bc7f030 msgr2=0x55f55bd05560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 -- 172.21.15.44:0/1018266165 shutdown_connections 2023-09-10T01:06:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 -- 172.21.15.44:0/1018266165 wait complete. 2023-09-10T01:06:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 -- start start 2023-09-10T01:06:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 Processor -- start 2023-09-10T01:06:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd0a300 0x55f55bd16e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0acc0 0x55f55bd14030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd101a0 0x55f55bd145f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55bbfab10 con 0x55f55bd0a300 2023-09-10T01:06:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55bbfa810 con 0x55f55bd101a0 2023-09-10T01:06:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.003+0000 7f80f35a5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55bade8c0 con 0x55f55bd0acc0 2023-09-10T01:06:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80f35a5b80 10 client.0 _ll_register_callbacks cb 0x55f55bb44a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:06:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80e517b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd0a300 0x55f55bd16e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80e517b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd0a300 0x55f55bd16e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55806/0 (socket says 172.21.15.44:55806) 2023-09-10T01:06:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80e517b700 1 -- 172.21.15.44:0/1313829646 learned_addr learned my addr 172.21.15.44:0/1313829646 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80e497a700 1 --2- 172.21.15.44:0/1313829646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0acc0 0x55f55bd14030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80e597c700 1 --2- 172.21.15.44:0/1313829646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd101a0 0x55f55bd145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80e517b700 1 -- 172.21.15.44:0/1313829646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0acc0 msgr2=0x55f55bd14030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80e517b700 1 --2- 172.21.15.44:0/1313829646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0acc0 0x55f55bd14030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80e517b700 1 -- 172.21.15.44:0/1313829646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd101a0 msgr2=0x55f55bd145f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80e517b700 1 --2- 172.21.15.44:0/1313829646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd101a0 0x55f55bd145f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80e517b700 1 -- 172.21.15.44:0/1313829646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d0017070 con 0x55f55bd0a300 2023-09-10T01:06:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80e497a700 1 --2- 172.21.15.44:0/1313829646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0acc0 0x55f55bd14030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:06:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80e517b700 1 --2- 172.21.15.44:0/1313829646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd0a300 0x55f55bd16e20 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0967d0 tx=0x7f80dc099cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5165122a4385da server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:06:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80c8ff9700 1 -- 172.21.15.44:0/1313829646 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc09a2d0 con 0x55f55bd0a300 2023-09-10T01:06:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80c8ff9700 1 -- 172.21.15.44:0/1313829646 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f80dc09ee40 con 0x55f55bd0a300 2023-09-10T01:06:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.004+0000 7f80f35a5b80 1 -- 172.21.15.44:0/1313829646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f55bbb4490 con 0x55f55bd0a300 2023-09-10T01:06:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.005+0000 7f80c8ff9700 1 -- 172.21.15.44:0/1313829646 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80dc09e100 con 0x55f55bd0a300 2023-09-10T01:06:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.005+0000 7f80f35a5b80 10 client.18777 fetch_fsmap learned FSMap version 107 2023-09-10T01:06:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.005+0000 7f80f35a5b80 1 -- 172.21.15.44:0/1313829646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) v3 -- 0x7f80d8011230 con 0x55f55bd0a300 2023-09-10T01:06:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.005+0000 7f80c8ff9700 1 -- 172.21.15.44:0/1313829646 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80dc0b1080 con 0x55f55bd0a300 2023-09-10T01:06:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.005+0000 7f80c8ff9700 1 -- 172.21.15.44:0/1313829646 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 107) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f80dc0997c0 con 0x55f55bd0a300 2023-09-10T01:06:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.005+0000 7f80f35a5b80 10 client.18777 fetch_fsmap finished waiting for FSMap version 107 2023-09-10T01:06:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.005+0000 7f80f35a5b80 10 client.18777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:06:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.005+0000 7f80f35a5b80 20 client.18777 populate_metadata read hostname 'smithi044' 2023-09-10T01:06:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.005+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/1313829646 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f55bd240d0 0x55f55bd264f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.005+0000 7f80f35a5b80 4 client.18777 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:06:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.005+0000 7f80f35a5b80 1 -- 172.21.15.44:0/1313829646 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f55bc71840 con 0x55f55bd240d0 2023-09-10T01:06:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.005+0000 7f80e497a700 1 --2- 172.21.15.44:0/1313829646 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f55bd240d0 0x55f55bd264f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.006+0000 7f80c8ff9700 10 client.18777 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:06:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.006+0000 7f80e497a700 1 --2- 172.21.15.44:0/1313829646 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f55bd240d0 0x55f55bd264f0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c476a5eb813bb1f server_cookie=d9eaf8f93b5714a4 in_seq=0 out_seq=0 2023-09-10T01:06:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.006+0000 7f80c8ff9700 1 -- 172.21.15.44:0/1313829646 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f55bc71840 con 0x55f55bd240d0 2023-09-10T01:06:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.006+0000 7f80c8ff9700 10 client.18777 handle_command_reply: tid=0 2023-09-10T01:06:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.006+0000 7f80f35a5b80 1 client.18777 shutdown 2023-09-10T01:06:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.007+0000 7f80f35a5b80 1 -- 172.21.15.44:0/1313829646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd0a300 msgr2=0x55f55bd16e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.007+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/1313829646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd0a300 0x55f55bd16e20 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0967d0 tx=0x7f80dc099cb0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.007+0000 7f80f35a5b80 1 -- 172.21.15.44:0/1313829646 shutdown_connections 2023-09-10T01:06:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.007+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/1313829646 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f55bd240d0 0x55f55bd264f0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.007+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/1313829646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd101a0 0x55f55bd145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.007+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/1313829646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd0a300 0x55f55bd16e20 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.007+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/1313829646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0acc0 0x55f55bd14030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.007+0000 7f80f35a5b80 1 -- 172.21.15.44:0/1313829646 >> 172.21.15.44:0/1313829646 conn(0x55f55bc7f030 msgr2=0x55f55bd047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.007+0000 7f80c8ff9700 0 client.18777 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:06:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.007+0000 7f80f35a5b80 1 -- 172.21.15.44:0/1313829646 shutdown_connections 2023-09-10T01:06:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.007+0000 7f80f35a5b80 1 -- 172.21.15.44:0/1313829646 wait complete. 2023-09-10T01:06:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.007+0000 7f80f35a5b80 20 client.18777 trim_cache size 0 max 16384 2023-09-10T01:06:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.025+0000 7f80f35a5b80 1 Processor -- start 2023-09-10T01:06:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.025+0000 7f80f35a5b80 1 -- start start 2023-09-10T01:06:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.025+0000 7f80f35a5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd0a300 0x55f55bd16950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.025+0000 7f80f35a5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0acc0 0x55f55bd05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.025+0000 7f80f35a5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd101a0 0x55f55bd055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.025+0000 7f80f35a5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55bd10580 con 0x55f55bd0a300 2023-09-10T01:06:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.025+0000 7f80f35a5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55bd01fc0 con 0x55f55bd101a0 2023-09-10T01:06:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.025+0000 7f80f35a5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55bd14bd0 con 0x55f55bd0acc0 2023-09-10T01:06:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.026+0000 7f80e497a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0acc0 0x55f55bd05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.026+0000 7f80e497a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0acc0 0x55f55bd05090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34466/0 (socket says 172.21.15.44:34466) 2023-09-10T01:06:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.026+0000 7f80e497a700 1 -- 172.21.15.44:0/948295525 learned_addr learned my addr 172.21.15.44:0/948295525 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.026+0000 7f80e497a700 1 -- 172.21.15.44:0/948295525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd101a0 msgr2=0x55f55bd055d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:06:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.026+0000 7f80e497a700 1 --2- 172.21.15.44:0/948295525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd101a0 0x55f55bd055d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.026+0000 7f80e497a700 1 -- 172.21.15.44:0/948295525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd0a300 msgr2=0x55f55bd16950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.026+0000 7f80e497a700 1 --2- 172.21.15.44:0/948295525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd0a300 0x55f55bd16950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.026+0000 7f80e497a700 1 -- 172.21.15.44:0/948295525 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f55ba71c00 con 0x55f55bd0acc0 2023-09-10T01:06:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.026+0000 7f80e497a700 1 --2- 172.21.15.44:0/948295525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0acc0 0x55f55bd05090 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009a50 tx=0x7f80d0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2138fbb1c3453b2f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.027+0000 7f80cbfff700 1 -- 172.21.15.44:0/948295525 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0024040 con 0x55f55bd0acc0 2023-09-10T01:06:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.027+0000 7f80cbfff700 1 -- 172.21.15.44:0/948295525 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80d0003e70 con 0x55f55bd0acc0 2023-09-10T01:06:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.027+0000 7f80cbfff700 1 -- 172.21.15.44:0/948295525 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d001ead0 con 0x55f55bd0acc0 2023-09-10T01:06:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.027+0000 7f80f35a5b80 1 -- 172.21.15.44:0/948295525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0acc0 msgr2=0x55f55bd05090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.027+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/948295525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0acc0 0x55f55bd05090 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009a50 tx=0x7f80d0014000 comp rx=0 tx=0).stop 2023-09-10T01:06:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.027+0000 7f80f35a5b80 1 -- 172.21.15.44:0/948295525 shutdown_connections 2023-09-10T01:06:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.027+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/948295525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd101a0 0x55f55bd055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.027+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/948295525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd0a300 0x55f55bd16950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.027+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/948295525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd0acc0 0x55f55bd05090 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.027+0000 7f80f35a5b80 1 -- 172.21.15.44:0/948295525 >> 172.21.15.44:0/948295525 conn(0x55f55bc7f030 msgr2=0x55f55bd0f020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.027+0000 7f80f35a5b80 1 -- 172.21.15.44:0/948295525 shutdown_connections 2023-09-10T01:06:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.027+0000 7f80f35a5b80 1 -- 172.21.15.44:0/948295525 wait complete. 2023-09-10T01:06:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.028+0000 7f80f35a5b80 1 -- start start 2023-09-10T01:06:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.028+0000 7f80f35a5b80 1 Processor -- start 2023-09-10T01:06:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.028+0000 7f80f35a5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd0a300 0x55f55bd1b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.028+0000 7f80f35a5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd0acc0 0x55f55bd1b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.028+0000 7f80f35a5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd101a0 0x55f55bd19710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.028+0000 7f80f35a5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55bd19db0 con 0x55f55bd0acc0 2023-09-10T01:06:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.028+0000 7f80f35a5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55bd19f20 con 0x55f55bd0a300 2023-09-10T01:06:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.028+0000 7f80f35a5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55bd1a0e0 con 0x55f55bd101a0 2023-09-10T01:06:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.028+0000 7f80f35a5b80 10 client.0 _ll_register_callbacks cb 0x55f55bb44a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:06:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.028+0000 7f80e517b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd0a300 0x55f55bd1b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.028+0000 7f80e517b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd0a300 0x55f55bd1b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57558/0 (socket says 172.21.15.44:57558) 2023-09-10T01:06:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.028+0000 7f80e517b700 1 -- 172.21.15.44:0/2036089090 learned_addr learned my addr 172.21.15.44:0/2036089090 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.029+0000 7f80e597c700 1 --2- 172.21.15.44:0/2036089090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd101a0 0x55f55bd19710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.029+0000 7f80e517b700 1 -- 172.21.15.44:0/2036089090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd101a0 msgr2=0x55f55bd19710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.029+0000 7f80e517b700 1 --2- 172.21.15.44:0/2036089090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd101a0 0x55f55bd19710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.029+0000 7f80e517b700 1 -- 172.21.15.44:0/2036089090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd0acc0 msgr2=0x55f55bd1b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.029+0000 7f80e517b700 1 --2- 172.21.15.44:0/2036089090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd0acc0 0x55f55bd1b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.029+0000 7f80e517b700 1 -- 172.21.15.44:0/2036089090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d00043f0 con 0x55f55bd0a300 2023-09-10T01:06:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.029+0000 7f80e517b700 1 --2- 172.21.15.44:0/2036089090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd0a300 0x55f55bd1b1b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f80dc096d40 tx=0x7f80dc00bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c6de733f515d670 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.029+0000 7f80c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:06:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.029+0000 7f80c8ff9700 1 -- 172.21.15.44:0/2036089090 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc09a9d0 con 0x55f55bd0a300 2023-09-10T01:06:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.029+0000 7f80f35a5b80 1 -- 172.21.15.44:0/2036089090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80d800f040 con 0x55f55bd0a300 2023-09-10T01:06:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.029+0000 7f80c8ff9700 1 -- 172.21.15.44:0/2036089090 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f80dc09ee40 con 0x55f55bd0a300 2023-09-10T01:06:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.029+0000 7f80c8ff9700 1 -- 172.21.15.44:0/2036089090 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80dc09a2d0 con 0x55f55bd0a300 2023-09-10T01:06:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.029+0000 7f80f35a5b80 10 client.28430 fetch_fsmap learned FSMap version 107 2023-09-10T01:06:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.029+0000 7f80f35a5b80 1 -- 172.21.15.44:0/2036089090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) v3 -- 0x55f55baaf1f0 con 0x55f55bd0a300 2023-09-10T01:06:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.030+0000 7f80c8ff9700 1 -- 172.21.15.44:0/2036089090 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80dc0b8020 con 0x55f55bd0a300 2023-09-10T01:06:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.030+0000 7f80c8ff9700 1 -- 172.21.15.44:0/2036089090 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 107) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f80dc0052f0 con 0x55f55bd0a300 2023-09-10T01:06:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.030+0000 7f80f35a5b80 10 client.28430 fetch_fsmap finished waiting for FSMap version 107 2023-09-10T01:06:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.030+0000 7f80f35a5b80 10 client.28430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:06:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.030+0000 7f80f35a5b80 20 client.28430 populate_metadata read hostname 'smithi044' 2023-09-10T01:06:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.030+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/2036089090 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f55bd78c70 0x55f55bd7b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.030+0000 7f80f35a5b80 4 client.28430 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:06:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.030+0000 7f80f35a5b80 1 -- 172.21.15.44:0/2036089090 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f55bc558c0 con 0x55f55bd78c70 2023-09-10T01:06:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.030+0000 7f80e497a700 1 --2- 172.21.15.44:0/2036089090 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f55bd78c70 0x55f55bd7b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.030+0000 7f80e497a700 1 --2- 172.21.15.44:0/2036089090 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f55bd78c70 0x55f55bd7b060 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=281d316808b92c82 server_cookie=2a0e7cae91fe0461 in_seq=0 out_seq=0 2023-09-10T01:06:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.030+0000 7f80c8ff9700 10 client.28430 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:06:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.038+0000 7f80c8ff9700 1 -- 172.21.15.44:0/2036089090 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f55bc558c0 con 0x55f55bd78c70 2023-09-10T01:06:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.038+0000 7f80c8ff9700 10 client.28430 handle_command_reply: tid=0 2023-09-10T01:06:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.038+0000 7f80f35a5b80 1 client.28430 shutdown 2023-09-10T01:06:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.039+0000 7f80f35a5b80 1 -- 172.21.15.44:0/2036089090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd0a300 msgr2=0x55f55bd1b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.039+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/2036089090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd0a300 0x55f55bd1b1b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f80dc096d40 tx=0x7f80dc00bce0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.039+0000 7f80f35a5b80 1 -- 172.21.15.44:0/2036089090 shutdown_connections 2023-09-10T01:06:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.039+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/2036089090 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f55bd78c70 0x55f55bd7b060 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.039+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/2036089090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f55bd101a0 0x55f55bd19710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.039+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/2036089090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f55bd0a300 0x55f55bd1b1b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.039+0000 7f80f35a5b80 1 --2- 172.21.15.44:0/2036089090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f55bd0acc0 0x55f55bd1b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.039+0000 7f80c8ff9700 0 client.28430 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:06:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.039+0000 7f80f35a5b80 1 -- 172.21.15.44:0/2036089090 >> 172.21.15.44:0/2036089090 conn(0x55f55bc7f030 msgr2=0x55f55bd0f020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.039+0000 7f80f35a5b80 1 -- 172.21.15.44:0/2036089090 shutdown_connections 2023-09-10T01:06:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.039+0000 7f80f35a5b80 1 -- 172.21.15.44:0/2036089090 wait complete. 2023-09-10T01:06:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.039+0000 7f80f35a5b80 20 client.28430 trim_cache size 0 max 16384 2023-09-10T01:06:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:06:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:06:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:06:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:06:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.039+0000 7f80e73df700 1 -- 172.21.15.44:0/2782809687 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80cc0a07b0 msgr2=0x7f80cc0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.039+0000 7f80e73df700 1 --2- 172.21.15.44:0/2782809687 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80cc0a07b0 0x7f80cc0a2c70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f80e0102ac0 tx=0x7f80d0009310 comp rx=0 tx=0).stop 2023-09-10T01:06:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.039+0000 7f80e73df700 1 -- 172.21.15.44:0/2782809687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80e00ff760 msgr2=0x7f80e01012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.039+0000 7f80e73df700 1 --2- 172.21.15.44:0/2782809687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80e00ff760 0x7f80e01012a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009810 tx=0x7f80dc00b560 comp rx=0 tx=0).stop 2023-09-10T01:06:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.040+0000 7f80e73df700 1 -- 172.21.15.44:0/2782809687 shutdown_connections 2023-09-10T01:06:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.040+0000 7f80e73df700 1 --2- 172.21.15.44:0/2782809687 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80cc0a07b0 0x7f80cc0a2c70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.040+0000 7f80e73df700 1 --2- 172.21.15.44:0/2782809687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80e01055b0 0x7f80e01a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.040+0000 7f80e73df700 1 --2- 172.21.15.44:0/2782809687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80e00ff760 0x7f80e01012a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.040+0000 7f80e73df700 1 --2- 172.21.15.44:0/2782809687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e0100120 0x7f80e01017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.040+0000 7f80e73df700 1 -- 172.21.15.44:0/2782809687 >> 172.21.15.44:0/2782809687 conn(0x7f80e0074f30 msgr2=0x7f80e0067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.040+0000 7f80e73df700 1 -- 172.21.15.44:0/2782809687 shutdown_connections 2023-09-10T01:06:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:36.040+0000 7f80e73df700 1 -- 172.21.15.44:0/2782809687 wait complete. 2023-09-10T01:06:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:36 smithi131 ceph-mon[103713]: pgmap v823: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 206 op/s 2023-09-10T01:06:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:36 smithi044 ceph-mon[98676]: pgmap v823: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 206 op/s 2023-09-10T01:06:37.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:36 smithi099 ceph-mon[102692]: pgmap v823: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 206 op/s 2023-09-10T01:06:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:38 smithi131 ceph-mon[103713]: pgmap v824: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 997 KiB/s wr, 142 op/s 2023-09-10T01:06:39.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:38 smithi044 ceph-mon[98676]: pgmap v824: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 997 KiB/s wr, 142 op/s 2023-09-10T01:06:39.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:38 smithi099 ceph-mon[102692]: pgmap v824: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 997 KiB/s wr, 142 op/s 2023-09-10T01:06:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:40 smithi131 ceph-mon[103713]: pgmap v825: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 997 KiB/s wr, 142 op/s 2023-09-10T01:06:41.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:40 smithi044 ceph-mon[98676]: pgmap v825: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 997 KiB/s wr, 142 op/s 2023-09-10T01:06:41.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:40 smithi099 ceph-mon[102692]: pgmap v825: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 997 KiB/s wr, 142 op/s 2023-09-10T01:06:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:42 smithi131 ceph-mon[103713]: pgmap v826: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 248 op/s 2023-09-10T01:06:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:42 smithi044 ceph-mon[98676]: pgmap v826: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 248 op/s 2023-09-10T01:06:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:42 smithi099 ceph-mon[102692]: pgmap v826: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 248 op/s 2023-09-10T01:06:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:44 smithi131 ceph-mon[103713]: pgmap v827: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:06:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:44 smithi044 ceph-mon[98676]: pgmap v827: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:06:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:44 smithi099 ceph-mon[102692]: pgmap v827: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-09-10T01:06:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:46 smithi131 ceph-mon[103713]: pgmap v828: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2023-09-10T01:06:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:46 smithi044 ceph-mon[98676]: pgmap v828: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2023-09-10T01:06:47.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:46 smithi099 ceph-mon[102692]: pgmap v828: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2023-09-10T01:06:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.162+0000 7fece1e78700 1 -- 172.21.15.44:0/4257876973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecdc100120 msgr2=0x7fecdc10c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.162+0000 7fece1e78700 1 --2- 172.21.15.44:0/4257876973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecdc100120 0x7fecdc10c330 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fecd400b0d0 tx=0x7fecd400b3e0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.162+0000 7fece1e78700 1 -- 172.21.15.44:0/4257876973 shutdown_connections 2023-09-10T01:06:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.162+0000 7fece1e78700 1 --2- 172.21.15.44:0/4257876973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecdc100120 0x7fecdc10c330 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.162+0000 7fece1e78700 1 --2- 172.21.15.44:0/4257876973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecdc1055b0 0x7fecdc105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.162+0000 7fece1e78700 1 --2- 172.21.15.44:0/4257876973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecdc0ff760 0x7fecdc0ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.162+0000 7fece1e78700 1 -- 172.21.15.44:0/4257876973 >> 172.21.15.44:0/4257876973 conn(0x7fecdc074f30 msgr2=0x7fecdc075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.162+0000 7fece1e78700 1 -- 172.21.15.44:0/4257876973 shutdown_connections 2023-09-10T01:06:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.162+0000 7fece1e78700 1 -- 172.21.15.44:0/4257876973 wait complete. 2023-09-10T01:06:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.163+0000 7fece1e78700 1 Processor -- start 2023-09-10T01:06:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.163+0000 7fece1e78700 1 -- start start 2023-09-10T01:06:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.163+0000 7fece1e78700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecdc0ff760 0x7fecdc19b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.163+0000 7fece1e78700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecdc100120 0x7fecdc19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.163+0000 7fece1e78700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecdc1055b0 0x7fecdc195450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.163+0000 7fece1e78700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc195c20 con 0x7fecdc1055b0 2023-09-10T01:06:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.163+0000 7fece1e78700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc195d90 con 0x7fecdc100120 2023-09-10T01:06:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.163+0000 7fece1e78700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc195f50 con 0x7fecdc0ff760 2023-09-10T01:06:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.163+0000 7fecdb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecdc0ff760 0x7fecdc19b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fecdaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecdc100120 0x7fecdc19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fecdb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecdc0ff760 0x7fecdc19b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34480/0 (socket says 172.21.15.44:34480) 2023-09-10T01:06:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fecdb7fe700 1 -- 172.21.15.44:0/60250894 learned_addr learned my addr 172.21.15.44:0/60250894 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fecdbfff700 1 --2- 172.21.15.44:0/60250894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecdc1055b0 0x7fecdc195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fecdbfff700 1 -- 172.21.15.44:0/60250894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecdc0ff760 msgr2=0x7fecdc19b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fecdbfff700 1 --2- 172.21.15.44:0/60250894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecdc0ff760 0x7fecdc19b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fecdbfff700 1 -- 172.21.15.44:0/60250894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecdc100120 msgr2=0x7fecdc19bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fecdbfff700 1 --2- 172.21.15.44:0/60250894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecdc100120 0x7fecdc19bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fecdbfff700 1 -- 172.21.15.44:0/60250894 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd4009d00 con 0x7fecdc1055b0 2023-09-10T01:06:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fecdaffd700 1 --2- 172.21.15.44:0/60250894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecdc100120 0x7fecdc19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:06:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fecdb7fe700 1 --2- 172.21.15.44:0/60250894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecdc0ff760 0x7fecdc19b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:06:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fecdbfff700 1 --2- 172.21.15.44:0/60250894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecdc1055b0 0x7fecdc195450 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009fd0 tx=0x7fecd4007910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fecd8ff9700 1 -- 172.21.15.44:0/60250894 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd400e040 con 0x7fecdc1055b0 2023-09-10T01:06:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fece1e78700 1 -- 172.21.15.44:0/60250894 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecdc1961d0 con 0x7fecdc1055b0 2023-09-10T01:06:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fecd8ff9700 1 -- 172.21.15.44:0/60250894 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fecd401d480 con 0x7fecdc1055b0 2023-09-10T01:06:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fecd8ff9700 1 -- 172.21.15.44:0/60250894 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd401b6b0 con 0x7fecdc1055b0 2023-09-10T01:06:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.164+0000 7fece1e78700 1 -- 172.21.15.44:0/60250894 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecdc067bd0 con 0x7fecdc1055b0 2023-09-10T01:06:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.167+0000 7fecd8ff9700 1 -- 172.21.15.44:0/60250894 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fecd401d5f0 con 0x7fecdc1055b0 2023-09-10T01:06:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.167+0000 7fecd8ff9700 1 --2- 172.21.15.44:0/60250894 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fecc40a07b0 0x7fecc40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.168+0000 7fecd8ff9700 1 -- 172.21.15.44:0/60250894 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fecd40cdb60 con 0x7fecdc1055b0 2023-09-10T01:06:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.168+0000 7fecdb7fe700 1 --2- 172.21.15.44:0/60250894 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fecc40a07b0 0x7fecc40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.168+0000 7fecdb7fe700 1 --2- 172.21.15.44:0/60250894 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fecc40a07b0 0x7fecc40a2c70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feccc00e5f0 tx=0x7feccc010040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.168+0000 7fecee03eb80 1 Processor -- start 2023-09-10T01:06:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecee03eb80 1 -- start start 2023-09-10T01:06:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecee03eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da6c5b0 0x55650da6c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecee03eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da66710 0x55650da66b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecee03eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da670d0 0x55650da73340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecee03eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650d9bb6d0 con 0x55650da6c5b0 2023-09-10T01:06:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecee03eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650d9b8ff0 con 0x55650da670d0 2023-09-10T01:06:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecee03eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650d9b48e0 con 0x55650da66710 2023-09-10T01:06:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecdb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da6c5b0 0x55650da6c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecdb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da6c5b0 0x55650da6c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55838/0 (socket says 172.21.15.44:55838) 2023-09-10T01:06:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecdb7fe700 1 -- 172.21.15.44:0/3739976882 learned_addr learned my addr 172.21.15.44:0/3739976882 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecdaffd700 1 --2- 172.21.15.44:0/3739976882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da66710 0x55650da66b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecdbfff700 1 --2- 172.21.15.44:0/3739976882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da670d0 0x55650da73340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecdb7fe700 1 -- 172.21.15.44:0/3739976882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da66710 msgr2=0x55650da66b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecdb7fe700 1 --2- 172.21.15.44:0/3739976882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da66710 0x55650da66b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecdb7fe700 1 -- 172.21.15.44:0/3739976882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da670d0 msgr2=0x55650da73340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecdb7fe700 1 --2- 172.21.15.44:0/3739976882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da670d0 0x55650da73340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecdb7fe700 1 -- 172.21.15.44:0/3739976882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55650d7e0d80 con 0x55650da6c5b0 2023-09-10T01:06:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.169+0000 7fecdb7fe700 1 --2- 172.21.15.44:0/3739976882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da6c5b0 0x55650da6c990 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7feccc01a690 tx=0x7feccc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dae9f262fec7dcb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.170+0000 7fecc1ffb700 1 -- 172.21.15.44:0/3739976882 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc009dc0 con 0x55650da6c5b0 2023-09-10T01:06:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.170+0000 7fecc1ffb700 1 -- 172.21.15.44:0/3739976882 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feccc014040 con 0x55650da6c5b0 2023-09-10T01:06:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.170+0000 7fecc1ffb700 1 -- 172.21.15.44:0/3739976882 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc005680 con 0x55650da6c5b0 2023-09-10T01:06:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.170+0000 7fecee03eb80 1 -- 172.21.15.44:0/3739976882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da6c5b0 msgr2=0x55650da6c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.170+0000 7fecee03eb80 1 --2- 172.21.15.44:0/3739976882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da6c5b0 0x55650da6c990 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7feccc01a690 tx=0x7feccc016000 comp rx=0 tx=0).stop 2023-09-10T01:06:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.170+0000 7fecee03eb80 1 -- 172.21.15.44:0/3739976882 shutdown_connections 2023-09-10T01:06:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.170+0000 7fecee03eb80 1 --2- 172.21.15.44:0/3739976882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da670d0 0x55650da73340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.170+0000 7fecee03eb80 1 --2- 172.21.15.44:0/3739976882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da6c5b0 0x55650da6c990 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.170+0000 7fecee03eb80 1 --2- 172.21.15.44:0/3739976882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da66710 0x55650da66b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.170+0000 7fecee03eb80 1 -- 172.21.15.44:0/3739976882 >> 172.21.15.44:0/3739976882 conn(0x55650d9d7030 msgr2=0x55650da5d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.170+0000 7fecee03eb80 1 -- 172.21.15.44:0/3739976882 shutdown_connections 2023-09-10T01:06:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.170+0000 7fecee03eb80 1 -- 172.21.15.44:0/3739976882 wait complete. 2023-09-10T01:06:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.171+0000 7fecee03eb80 1 -- start start 2023-09-10T01:06:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.171+0000 7fecee03eb80 1 Processor -- start 2023-09-10T01:06:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.171+0000 7fecee03eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da66710 0x55650da73180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.171+0000 7fecee03eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da670d0 0x55650da70400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.171+0000 7fecee03eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da6c5b0 0x55650da70980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.171+0000 7fecee03eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650d952b10 con 0x55650da66710 2023-09-10T01:06:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.171+0000 7fecee03eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650d952810 con 0x55650da6c5b0 2023-09-10T01:06:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.171+0000 7fecee03eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650d8368c0 con 0x55650da670d0 2023-09-10T01:06:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.171+0000 7fecee03eb80 10 client.0 _ll_register_callbacks cb 0x55650d89ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:06:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.171+0000 7fecdb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da66710 0x55650da73180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.171+0000 7fecdb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da66710 0x55650da73180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55844/0 (socket says 172.21.15.44:55844) 2023-09-10T01:06:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.171+0000 7fecdb7fe700 1 -- 172.21.15.44:0/1939530958 learned_addr learned my addr 172.21.15.44:0/1939530958 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.171+0000 7fecdaffd700 1 --2- 172.21.15.44:0/1939530958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da670d0 0x55650da70400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.171+0000 7fecdbfff700 1 --2- 172.21.15.44:0/1939530958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da6c5b0 0x55650da70980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.172+0000 7fecdb7fe700 1 -- 172.21.15.44:0/1939530958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da670d0 msgr2=0x55650da70400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.172+0000 7fecdb7fe700 1 --2- 172.21.15.44:0/1939530958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da670d0 0x55650da70400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.172+0000 7fecdb7fe700 1 -- 172.21.15.44:0/1939530958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da6c5b0 msgr2=0x55650da70980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.172+0000 7fecdb7fe700 1 --2- 172.21.15.44:0/1939530958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da6c5b0 0x55650da70980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.172+0000 7fecdb7fe700 1 -- 172.21.15.44:0/1939530958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feccc01a3f0 con 0x55650da66710 2023-09-10T01:06:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.172+0000 7fecdbfff700 1 --2- 172.21.15.44:0/1939530958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da6c5b0 0x55650da70980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:06:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.172+0000 7fecdaffd700 1 --2- 172.21.15.44:0/1939530958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da670d0 0x55650da70400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:06:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.172+0000 7fecdb7fe700 1 --2- 172.21.15.44:0/1939530958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da66710 0x55650da73180 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7feccc005500 tx=0x7feccc005530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36702fd924ea9add server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.172+0000 7fecbaffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:06:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.172+0000 7fecbaffd700 1 -- 172.21.15.44:0/1939530958 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc009dc0 con 0x55650da66710 2023-09-10T01:06:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.172+0000 7fecbaffd700 1 -- 172.21.15.44:0/1939530958 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 9540+0+0 (secure 0 0 0) 0x7feccc0046b0 con 0x55650da66710 2023-09-10T01:06:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.172+0000 7fecee03eb80 1 -- 172.21.15.44:0/1939530958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55650d90c490 con 0x55650da66710 2023-09-10T01:06:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.172+0000 7fecbaffd700 1 -- 172.21.15.44:0/1939530958 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7feccc004870 con 0x55650da66710 2023-09-10T01:06:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.172+0000 7fecee03eb80 10 client.18789 fetch_fsmap learned FSMap version 107 2023-09-10T01:06:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.172+0000 7fecee03eb80 1 -- 172.21.15.44:0/1939530958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) v3 -- 0x55650d7c9c00 con 0x55650da66710 2023-09-10T01:06:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.172+0000 7fecbaffd700 1 -- 172.21.15.44:0/1939530958 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55650d90c490 con 0x55650da66710 2023-09-10T01:06:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.173+0000 7fecbaffd700 1 -- 172.21.15.44:0/1939530958 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 107) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feccc02d970 con 0x55650da66710 2023-09-10T01:06:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.173+0000 7fecee03eb80 10 client.18789 fetch_fsmap finished waiting for FSMap version 107 2023-09-10T01:06:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.173+0000 7fecee03eb80 10 client.18789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:06:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.173+0000 7fecee03eb80 20 client.18789 populate_metadata read hostname 'smithi044' 2023-09-10T01:06:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.173+0000 7fecee03eb80 1 --2- 172.21.15.44:0/1939530958 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55650da807e0 0x55650da82c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.173+0000 7fecee03eb80 4 client.18789 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:06:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.173+0000 7fecee03eb80 1 -- 172.21.15.44:0/1939530958 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55650d9c9840 con 0x55650da807e0 2023-09-10T01:06:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.173+0000 7fecdaffd700 1 --2- 172.21.15.44:0/1939530958 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55650da807e0 0x55650da82c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.173+0000 7fecbaffd700 10 client.18789 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:06:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.173+0000 7fecdaffd700 1 --2- 172.21.15.44:0/1939530958 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55650da807e0 0x55650da82c00 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1315eca83c70f770 server_cookie=734951c4d80b3072 in_seq=0 out_seq=0 2023-09-10T01:06:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.174+0000 7fecbaffd700 1 -- 172.21.15.44:0/1939530958 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55650d9c9840 con 0x55650da807e0 2023-09-10T01:06:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.174+0000 7fecbaffd700 10 client.18789 handle_command_reply: tid=0 2023-09-10T01:06:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.174+0000 7fecee03eb80 1 client.18789 shutdown 2023-09-10T01:06:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.174+0000 7fecee03eb80 1 -- 172.21.15.44:0/1939530958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da66710 msgr2=0x55650da73180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.174+0000 7fecee03eb80 1 --2- 172.21.15.44:0/1939530958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da66710 0x55650da73180 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7feccc005500 tx=0x7feccc005530 comp rx=0 tx=0).stop 2023-09-10T01:06:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.174+0000 7fecee03eb80 1 -- 172.21.15.44:0/1939530958 shutdown_connections 2023-09-10T01:06:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.174+0000 7fecee03eb80 1 --2- 172.21.15.44:0/1939530958 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55650da807e0 0x55650da82c00 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.174+0000 7fecee03eb80 1 --2- 172.21.15.44:0/1939530958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da6c5b0 0x55650da70980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.174+0000 7fecee03eb80 1 --2- 172.21.15.44:0/1939530958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da66710 0x55650da73180 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.174+0000 7fecbaffd700 0 client.18789 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:06:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.174+0000 7fecee03eb80 1 --2- 172.21.15.44:0/1939530958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da670d0 0x55650da70400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.174+0000 7fecee03eb80 1 -- 172.21.15.44:0/1939530958 >> 172.21.15.44:0/1939530958 conn(0x55650d9d7030 msgr2=0x55650da5c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.174+0000 7fecee03eb80 1 -- 172.21.15.44:0/1939530958 shutdown_connections 2023-09-10T01:06:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.174+0000 7fecee03eb80 1 -- 172.21.15.44:0/1939530958 wait complete. 2023-09-10T01:06:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.174+0000 7fecee03eb80 20 client.18789 trim_cache size 0 max 16384 2023-09-10T01:06:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.191+0000 7fecee03eb80 1 Processor -- start 2023-09-10T01:06:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.191+0000 7fecee03eb80 1 -- start start 2023-09-10T01:06:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.191+0000 7fecee03eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da66710 0x55650da77930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.191+0000 7fecee03eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da670d0 0x55650da72d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.191+0000 7fecee03eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da6c5b0 0x55650da6fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.191+0000 7fecee03eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650da578a0 con 0x55650da6c5b0 2023-09-10T01:06:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.191+0000 7fecee03eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650da719e0 con 0x55650da670d0 2023-09-10T01:06:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.191+0000 7fecee03eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650da6c1e0 con 0x55650da66710 2023-09-10T01:06:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.192+0000 7fecdbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da6c5b0 0x55650da6fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.192+0000 7fecdb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da66710 0x55650da77930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.192+0000 7fecdaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da670d0 0x55650da72d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.192+0000 7fecdb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da66710 0x55650da77930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34502/0 (socket says 172.21.15.44:34502) 2023-09-10T01:06:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.192+0000 7fecdb7fe700 1 -- 172.21.15.44:0/199182612 learned_addr learned my addr 172.21.15.44:0/199182612 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.192+0000 7fecdbfff700 1 -- 172.21.15.44:0/199182612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da66710 msgr2=0x55650da77930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.192+0000 7fecdbfff700 1 --2- 172.21.15.44:0/199182612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da66710 0x55650da77930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.192+0000 7fecdbfff700 1 -- 172.21.15.44:0/199182612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da670d0 msgr2=0x55650da72d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.192+0000 7fecdbfff700 1 --2- 172.21.15.44:0/199182612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da670d0 0x55650da72d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.192+0000 7fecdbfff700 1 -- 172.21.15.44:0/199182612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55650d8071f0 con 0x55650da6c5b0 2023-09-10T01:06:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.192+0000 7fecdb7fe700 1 --2- 172.21.15.44:0/199182612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da66710 0x55650da77930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:06:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.192+0000 7fecdbfff700 1 --2- 172.21.15.44:0/199182612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da6c5b0 0x55650da6fbc0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fecd40caeb0 tx=0x7fecd40ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d514b0a12e87d73 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.192+0000 7fecc27fc700 1 -- 172.21.15.44:0/199182612 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd409b3a0 con 0x55650da6c5b0 2023-09-10T01:06:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.192+0000 7fecc27fc700 1 -- 172.21.15.44:0/199182612 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fecd409aca0 con 0x55650da6c5b0 2023-09-10T01:06:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.192+0000 7fecc27fc700 1 -- 172.21.15.44:0/199182612 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd40ba0b0 con 0x55650da6c5b0 2023-09-10T01:06:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.193+0000 7fecee03eb80 1 -- 172.21.15.44:0/199182612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da6c5b0 msgr2=0x55650da6fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.193+0000 7fecee03eb80 1 --2- 172.21.15.44:0/199182612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da6c5b0 0x55650da6fbc0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fecd40caeb0 tx=0x7fecd40ba040 comp rx=0 tx=0).stop 2023-09-10T01:06:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.193+0000 7fecee03eb80 1 -- 172.21.15.44:0/199182612 shutdown_connections 2023-09-10T01:06:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.193+0000 7fecee03eb80 1 --2- 172.21.15.44:0/199182612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da6c5b0 0x55650da6fbc0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.193+0000 7fecee03eb80 1 --2- 172.21.15.44:0/199182612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da66710 0x55650da77930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.193+0000 7fecee03eb80 1 --2- 172.21.15.44:0/199182612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da670d0 0x55650da72d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.193+0000 7fecee03eb80 1 -- 172.21.15.44:0/199182612 >> 172.21.15.44:0/199182612 conn(0x55650d9d7030 msgr2=0x55650da6b520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.193+0000 7fecee03eb80 1 -- 172.21.15.44:0/199182612 shutdown_connections 2023-09-10T01:06:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.193+0000 7fecee03eb80 1 -- 172.21.15.44:0/199182612 wait complete. 2023-09-10T01:06:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.194+0000 7fecee03eb80 1 -- start start 2023-09-10T01:06:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.194+0000 7fecee03eb80 1 Processor -- start 2023-09-10T01:06:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.194+0000 7fecee03eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da66710 0x55650da75920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.194+0000 7fecee03eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da670d0 0x55650da75e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.194+0000 7fecee03eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da6c5b0 0x55650dad35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.194+0000 7fecee03eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650da76750 con 0x55650da670d0 2023-09-10T01:06:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.194+0000 7fecee03eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650dad3b10 con 0x55650da66710 2023-09-10T01:06:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.194+0000 7fecee03eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650dad3c50 con 0x55650da6c5b0 2023-09-10T01:06:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.194+0000 7fecee03eb80 10 client.0 _ll_register_callbacks cb 0x7fecd4009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:06:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.194+0000 7fecdaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da670d0 0x55650da75e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.194+0000 7fecdaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da670d0 0x55650da75e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55860/0 (socket says 172.21.15.44:55860) 2023-09-10T01:06:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.194+0000 7fecdaffd700 1 -- 172.21.15.44:0/2442620459 learned_addr learned my addr 172.21.15.44:0/2442620459 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:06:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.194+0000 7fecdb7fe700 1 --2- 172.21.15.44:0/2442620459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da66710 0x55650da75920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.194+0000 7fecdbfff700 1 --2- 172.21.15.44:0/2442620459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da6c5b0 0x55650dad35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.195+0000 7fecdaffd700 1 -- 172.21.15.44:0/2442620459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da6c5b0 msgr2=0x55650dad35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.195+0000 7fecdaffd700 1 --2- 172.21.15.44:0/2442620459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da6c5b0 0x55650dad35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.195+0000 7fecdaffd700 1 -- 172.21.15.44:0/2442620459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da66710 msgr2=0x55650da75920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:06:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.195+0000 7fecdaffd700 1 --2- 172.21.15.44:0/2442620459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da66710 0x55650da75920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.195+0000 7fecdaffd700 1 -- 172.21.15.44:0/2442620459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55650d7c9c00 con 0x55650da670d0 2023-09-10T01:06:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.195+0000 7fecdaffd700 1 --2- 172.21.15.44:0/2442620459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da670d0 0x55650da75e60 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fecd0000c00 tx=0x7fecd0012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50609f73ad878aee server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:06:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.195+0000 7fecbaffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:06:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.195+0000 7fecbaffd700 1 -- 172.21.15.44:0/2442620459 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd0017040 con 0x55650da670d0 2023-09-10T01:06:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.195+0000 7fecbaffd700 1 -- 172.21.15.44:0/2442620459 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fecd00212b0 con 0x55650da670d0 2023-09-10T01:06:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.195+0000 7fecee03eb80 1 -- 172.21.15.44:0/2442620459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feccc014040 con 0x55650da670d0 2023-09-10T01:06:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.195+0000 7fecbaffd700 1 -- 172.21.15.44:0/2442620459 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecd0021470 con 0x55650da670d0 2023-09-10T01:06:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.195+0000 7fecee03eb80 10 client.18795 fetch_fsmap learned FSMap version 107 2023-09-10T01:06:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.195+0000 7fecee03eb80 1 -- 172.21.15.44:0/2442620459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) v3 -- 0x7fecdc067bd0 con 0x55650da670d0 2023-09-10T01:06:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.195+0000 7fecbaffd700 1 -- 172.21.15.44:0/2442620459 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feccc014040 con 0x55650da670d0 2023-09-10T01:06:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.196+0000 7fecbaffd700 1 -- 172.21.15.44:0/2442620459 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 107) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fecd0004c80 con 0x55650da670d0 2023-09-10T01:06:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.196+0000 7fecee03eb80 10 client.18795 fetch_fsmap finished waiting for FSMap version 107 2023-09-10T01:06:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.196+0000 7fecee03eb80 10 client.18795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:06:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.196+0000 7fecee03eb80 20 client.18795 populate_metadata read hostname 'smithi044' 2023-09-10T01:06:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.196+0000 7fecee03eb80 1 --2- 172.21.15.44:0/2442620459 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55650dad53d0 0x55650dad77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:06:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.196+0000 7fecee03eb80 4 client.18795 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:06:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.196+0000 7fecee03eb80 1 -- 172.21.15.44:0/2442620459 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55650d9ad8c0 con 0x55650dad53d0 2023-09-10T01:06:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.196+0000 7fecdb7fe700 1 --2- 172.21.15.44:0/2442620459 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55650dad53d0 0x55650dad77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:06:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.196+0000 7fecbaffd700 10 client.18795 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:06:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.196+0000 7fecdb7fe700 1 --2- 172.21.15.44:0/2442620459 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55650dad53d0 0x55650dad77f0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a50e59440dd65027 server_cookie=18d147f1c44ffd10 in_seq=0 out_seq=0 2023-09-10T01:06:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.200+0000 7fecbaffd700 1 -- 172.21.15.44:0/2442620459 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55650d9ad8c0 con 0x55650dad53d0 2023-09-10T01:06:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.200+0000 7fecbaffd700 10 client.18795 handle_command_reply: tid=0 2023-09-10T01:06:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.200+0000 7fecee03eb80 1 client.18795 shutdown 2023-09-10T01:06:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.201+0000 7fecee03eb80 1 -- 172.21.15.44:0/2442620459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da670d0 msgr2=0x55650da75e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.201+0000 7fecee03eb80 1 --2- 172.21.15.44:0/2442620459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da670d0 0x55650da75e60 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fecd0000c00 tx=0x7fecd0012f90 comp rx=0 tx=0).stop 2023-09-10T01:06:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.201+0000 7fecee03eb80 1 -- 172.21.15.44:0/2442620459 shutdown_connections 2023-09-10T01:06:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.201+0000 7fecee03eb80 1 --2- 172.21.15.44:0/2442620459 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55650dad53d0 0x55650dad77f0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.201+0000 7fecee03eb80 1 --2- 172.21.15.44:0/2442620459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55650da6c5b0 0x55650dad35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.201+0000 7fecbaffd700 0 client.18795 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:06:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.201+0000 7fecee03eb80 1 --2- 172.21.15.44:0/2442620459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650da66710 0x55650da75920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.201+0000 7fecee03eb80 1 --2- 172.21.15.44:0/2442620459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55650da670d0 0x55650da75e60 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.201+0000 7fecee03eb80 1 -- 172.21.15.44:0/2442620459 >> 172.21.15.44:0/2442620459 conn(0x55650d9d7030 msgr2=0x55650da612f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.201+0000 7fecee03eb80 1 -- 172.21.15.44:0/2442620459 shutdown_connections 2023-09-10T01:06:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.201+0000 7fecee03eb80 1 -- 172.21.15.44:0/2442620459 wait complete. 2023-09-10T01:06:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.201+0000 7fecee03eb80 20 client.18795 trim_cache size 0 max 16384 2023-09-10T01:06:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:06:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:06:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:06:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:06:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.201+0000 7fece1e78700 1 -- 172.21.15.44:0/60250894 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fecc40a07b0 msgr2=0x7fecc40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.201+0000 7fece1e78700 1 --2- 172.21.15.44:0/60250894 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fecc40a07b0 0x7fecc40a2c70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feccc00e5f0 tx=0x7feccc010040 comp rx=0 tx=0).stop 2023-09-10T01:06:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.202+0000 7fece1e78700 1 -- 172.21.15.44:0/60250894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecdc1055b0 msgr2=0x7fecdc195450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:06:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.202+0000 7fece1e78700 1 --2- 172.21.15.44:0/60250894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecdc1055b0 0x7fecdc195450 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009fd0 tx=0x7fecd4007910 comp rx=0 tx=0).stop 2023-09-10T01:06:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.202+0000 7fece1e78700 1 -- 172.21.15.44:0/60250894 shutdown_connections 2023-09-10T01:06:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.202+0000 7fece1e78700 1 --2- 172.21.15.44:0/60250894 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fecc40a07b0 0x7fecc40a2c70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.202+0000 7fece1e78700 1 --2- 172.21.15.44:0/60250894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecdc1055b0 0x7fecdc195450 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.202+0000 7fece1e78700 1 --2- 172.21.15.44:0/60250894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecdc0ff760 0x7fecdc19b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.202+0000 7fece1e78700 1 --2- 172.21.15.44:0/60250894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecdc100120 0x7fecdc19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:06:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.202+0000 7fece1e78700 1 -- 172.21.15.44:0/60250894 >> 172.21.15.44:0/60250894 conn(0x7fecdc074f30 msgr2=0x7fecdc0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:06:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.202+0000 7fece1e78700 1 -- 172.21.15.44:0/60250894 shutdown_connections 2023-09-10T01:06:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:06:49.202+0000 7fece1e78700 1 -- 172.21.15.44:0/60250894 wait complete. 2023-09-10T01:06:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:48 smithi131 ceph-mon[103713]: pgmap v829: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 179 op/s 2023-09-10T01:06:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:48 smithi044 ceph-mon[98676]: pgmap v829: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 179 op/s 2023-09-10T01:06:49.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:48 smithi099 ceph-mon[102692]: pgmap v829: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 179 op/s 2023-09-10T01:06:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:50 smithi131 ceph-mon[103713]: pgmap v830: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 179 op/s 2023-09-10T01:06:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:50 smithi044 ceph-mon[98676]: pgmap v830: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 179 op/s 2023-09-10T01:06:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:50 smithi099 ceph-mon[102692]: pgmap v830: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 179 op/s 2023-09-10T01:06:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:52 smithi131 ceph-mon[103713]: pgmap v831: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2023-09-10T01:06:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:52 smithi044 ceph-mon[98676]: pgmap v831: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2023-09-10T01:06:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:52 smithi099 ceph-mon[102692]: pgmap v831: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2023-09-10T01:06:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:54 smithi131 ceph-mon[103713]: pgmap v832: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 174 op/s 2023-09-10T01:06:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:54 smithi044 ceph-mon[98676]: pgmap v832: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 174 op/s 2023-09-10T01:06:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:54 smithi099 ceph-mon[102692]: pgmap v832: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 174 op/s 2023-09-10T01:06:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:56 smithi131 ceph-mon[103713]: pgmap v833: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 239 op/s 2023-09-10T01:06:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:56 smithi044 ceph-mon[98676]: pgmap v833: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 239 op/s 2023-09-10T01:06:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:56 smithi099 ceph-mon[102692]: pgmap v833: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 239 op/s 2023-09-10T01:06:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:06:58 smithi131 ceph-mon[103713]: pgmap v834: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2023-09-10T01:06:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:06:58 smithi044 ceph-mon[98676]: pgmap v834: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2023-09-10T01:06:59.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:06:58 smithi099 ceph-mon[102692]: pgmap v834: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2023-09-10T01:07:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:00 smithi131 ceph-mon[103713]: pgmap v835: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2023-09-10T01:07:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:00 smithi044 ceph-mon[98676]: pgmap v835: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2023-09-10T01:07:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:00 smithi099 ceph-mon[102692]: pgmap v835: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2023-09-10T01:07:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.315+0000 7f3be092a700 1 -- 172.21.15.44:0/1129808300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bdc1005d0 msgr2=0x7f3bdc100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.315+0000 7f3be092a700 1 --2- 172.21.15.44:0/1129808300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bdc1005d0 0x7f3bdc100a50 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00099c0 tx=0x7f3bd0009cd0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.315+0000 7f3be092a700 1 -- 172.21.15.44:0/1129808300 shutdown_connections 2023-09-10T01:07:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.315+0000 7f3be092a700 1 --2- 172.21.15.44:0/1129808300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bdc100f90 0x7f3bdc10d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.315+0000 7f3be092a700 1 --2- 172.21.15.44:0/1129808300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bdc106690 0x7f3bdc106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.315+0000 7f3be092a700 1 --2- 172.21.15.44:0/1129808300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bdc1005d0 0x7f3bdc100a50 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.315+0000 7f3be092a700 1 -- 172.21.15.44:0/1129808300 >> 172.21.15.44:0/1129808300 conn(0x7f3bdc0fc0c0 msgr2=0x7f3bdc0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.316+0000 7f3be092a700 1 -- 172.21.15.44:0/1129808300 shutdown_connections 2023-09-10T01:07:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.316+0000 7f3be092a700 1 -- 172.21.15.44:0/1129808300 wait complete. 2023-09-10T01:07:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.316+0000 7f3be092a700 1 Processor -- start 2023-09-10T01:07:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.316+0000 7f3be092a700 1 -- start start 2023-09-10T01:07:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.316+0000 7f3be092a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bdc1005d0 0x7f3bdc19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.316+0000 7f3be092a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bdc100f90 0x7f3bdc19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.316+0000 7f3be092a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bdc106690 0x7f3bdc1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.316+0000 7f3be092a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc196a10 con 0x7f3bdc106690 2023-09-10T01:07:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.316+0000 7f3be092a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc196b80 con 0x7f3bdc1005d0 2023-09-10T01:07:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.316+0000 7f3be092a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc196d40 con 0x7f3bdc100f90 2023-09-10T01:07:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.317+0000 7f3bdad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bdc106690 0x7f3bdc1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.317+0000 7f3bdad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bdc106690 0x7f3bdc1963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55876/0 (socket says 172.21.15.44:55876) 2023-09-10T01:07:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.317+0000 7f3bdad9d700 1 -- 172.21.15.44:0/62088298 learned_addr learned my addr 172.21.15.44:0/62088298 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.317+0000 7f3bd9d9b700 1 --2- 172.21.15.44:0/62088298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bdc100f90 0x7f3bdc19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.317+0000 7f3bda59c700 1 --2- 172.21.15.44:0/62088298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bdc1005d0 0x7f3bdc19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.317+0000 7f3bdad9d700 1 -- 172.21.15.44:0/62088298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bdc100f90 msgr2=0x7f3bdc19ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.317+0000 7f3bdad9d700 1 --2- 172.21.15.44:0/62088298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bdc100f90 0x7f3bdc19ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.317+0000 7f3bdad9d700 1 -- 172.21.15.44:0/62088298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bdc1005d0 msgr2=0x7f3bdc19c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.317+0000 7f3bdad9d700 1 --2- 172.21.15.44:0/62088298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bdc1005d0 0x7f3bdc19c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.317+0000 7f3bdad9d700 1 -- 172.21.15.44:0/62088298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc009520 con 0x7f3bdc106690 2023-09-10T01:07:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.317+0000 7f3bdad9d700 1 --2- 172.21.15.44:0/62088298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bdc106690 0x7f3bdc1963a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400ef30 tx=0x7f3bd400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.317+0000 7f3bcb7fe700 1 -- 172.21.15.44:0/62088298 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4018070 con 0x7f3bdc106690 2023-09-10T01:07:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.317+0000 7f3bcb7fe700 1 -- 172.21.15.44:0/62088298 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3bd400f040 con 0x7f3bdc106690 2023-09-10T01:07:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.317+0000 7f3be092a700 1 -- 172.21.15.44:0/62088298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd00095f0 con 0x7f3bdc106690 2023-09-10T01:07:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.317+0000 7f3bcb7fe700 1 -- 172.21.15.44:0/62088298 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd400cbc0 con 0x7f3bdc106690 2023-09-10T01:07:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.317+0000 7f3be092a700 1 -- 172.21.15.44:0/62088298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bdc1a3350 con 0x7f3bdc106690 2023-09-10T01:07:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.319+0000 7f3bcb7fe700 1 -- 172.21.15.44:0/62088298 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f3bd400cd80 con 0x7f3bdc106690 2023-09-10T01:07:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.320+0000 7f3bcb7fe700 1 --2- 172.21.15.44:0/62088298 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3bc40a07b0 0x7f3bc40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.320+0000 7f3bcb7fe700 1 -- 172.21.15.44:0/62088298 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f3bd40cbeb0 con 0x7f3bdc106690 2023-09-10T01:07:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.320+0000 7f3bda59c700 1 --2- 172.21.15.44:0/62088298 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3bc40a07b0 0x7f3bc40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.321+0000 7f3bda59c700 1 --2- 172.21.15.44:0/62088298 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3bc40a07b0 0x7f3bc40a2c70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009d70 tx=0x7f3bcc009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.321+0000 7f3becaf0b80 1 Processor -- start 2023-09-10T01:07:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.321+0000 7f3becaf0b80 1 -- start start 2023-09-10T01:07:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.321+0000 7f3becaf0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a9e1a0 0x5601f3a9e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.321+0000 7f3becaf0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98300 0x5601f3a98780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.321+0000 7f3becaf0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a98cc0 0x5601f3aa4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.321+0000 7f3becaf0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f39f16d0 con 0x5601f3a98300 2023-09-10T01:07:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.321+0000 7f3becaf0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f39eeff0 con 0x5601f3a9e1a0 2023-09-10T01:07:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.321+0000 7f3becaf0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f39ea8e0 con 0x5601f3a98cc0 2023-09-10T01:07:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.321+0000 7f3bd9d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98300 0x5601f3a98780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.321+0000 7f3bda59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a9e1a0 0x5601f3a9e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.321+0000 7f3bd9d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98300 0x5601f3a98780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55882/0 (socket says 172.21.15.44:55882) 2023-09-10T01:07:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.321+0000 7f3bda59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a9e1a0 0x5601f3a9e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57620/0 (socket says 172.21.15.44:57620) 2023-09-10T01:07:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.321+0000 7f3bd9d9b700 1 -- 172.21.15.44:0/1802347028 learned_addr learned my addr 172.21.15.44:0/1802347028 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.321+0000 7f3bdad9d700 1 --2- 172.21.15.44:0/1802347028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a98cc0 0x5601f3aa4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.322+0000 7f3bd9d9b700 1 -- 172.21.15.44:0/1802347028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a98cc0 msgr2=0x5601f3aa4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.322+0000 7f3bd9d9b700 1 --2- 172.21.15.44:0/1802347028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a98cc0 0x5601f3aa4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.322+0000 7f3bd9d9b700 1 -- 172.21.15.44:0/1802347028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a9e1a0 msgr2=0x5601f3a9e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.322+0000 7f3bd9d9b700 1 --2- 172.21.15.44:0/1802347028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a9e1a0 0x5601f3a9e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.322+0000 7f3bd9d9b700 1 -- 172.21.15.44:0/1802347028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601f3816d80 con 0x5601f3a98300 2023-09-10T01:07:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.322+0000 7f3bda59c700 1 --2- 172.21.15.44:0/1802347028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a9e1a0 0x5601f3a9e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.322+0000 7f3bd9d9b700 1 --2- 172.21.15.44:0/1802347028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98300 0x5601f3a98780 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00098e0 tx=0x7f3bd0005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ba02f0cd49a7c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.322+0000 7f3bc8ff9700 1 -- 172.21.15.44:0/1802347028 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd000ba90 con 0x5601f3a98300 2023-09-10T01:07:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.322+0000 7f3bc8ff9700 1 -- 172.21.15.44:0/1802347028 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3bd001e040 con 0x5601f3a98300 2023-09-10T01:07:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.322+0000 7f3bc8ff9700 1 -- 172.21.15.44:0/1802347028 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00039c0 con 0x5601f3a98300 2023-09-10T01:07:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 -- 172.21.15.44:0/1802347028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98300 msgr2=0x5601f3a98780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/1802347028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98300 0x5601f3a98780 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00098e0 tx=0x7f3bd0005bc0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 -- 172.21.15.44:0/1802347028 shutdown_connections 2023-09-10T01:07:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/1802347028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a98cc0 0x5601f3aa4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/1802347028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a9e1a0 0x5601f3a9e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/1802347028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98300 0x5601f3a98780 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 -- 172.21.15.44:0/1802347028 >> 172.21.15.44:0/1802347028 conn(0x5601f3a0d030 msgr2=0x5601f3a93560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 -- 172.21.15.44:0/1802347028 shutdown_connections 2023-09-10T01:07:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 -- 172.21.15.44:0/1802347028 wait complete. 2023-09-10T01:07:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 -- start start 2023-09-10T01:07:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 Processor -- start 2023-09-10T01:07:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a98300 0x5601f3aa4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98cc0 0x5601f3aa2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a9e1a0 0x5601f3aa2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f3988b10 con 0x5601f3a9e1a0 2023-09-10T01:07:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f3988810 con 0x5601f3a98cc0 2023-09-10T01:07:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.323+0000 7f3becaf0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f386c8c0 con 0x5601f3a98300 2023-09-10T01:07:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3becaf0b80 10 client.0 _ll_register_callbacks cb 0x5601f38d2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:07:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bda59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a98300 0x5601f3aa4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bd9d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98cc0 0x5601f3aa2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bda59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a98300 0x5601f3aa4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34536/0 (socket says 172.21.15.44:34536) 2023-09-10T01:07:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bda59c700 1 -- 172.21.15.44:0/3146322835 learned_addr learned my addr 172.21.15.44:0/3146322835 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bdad9d700 1 --2- 172.21.15.44:0/3146322835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a9e1a0 0x5601f3aa2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bd9d9b700 1 -- 172.21.15.44:0/3146322835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a98300 msgr2=0x5601f3aa4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bd9d9b700 1 --2- 172.21.15.44:0/3146322835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a98300 0x5601f3aa4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bd9d9b700 1 -- 172.21.15.44:0/3146322835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a9e1a0 msgr2=0x5601f3aa2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bd9d9b700 1 --2- 172.21.15.44:0/3146322835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a9e1a0 0x5601f3aa2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bd9d9b700 1 -- 172.21.15.44:0/3146322835 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc017070 con 0x5601f3a98cc0 2023-09-10T01:07:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bdad9d700 1 --2- 172.21.15.44:0/3146322835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a9e1a0 0x5601f3aa2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:07:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bda59c700 1 --2- 172.21.15.44:0/3146322835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a98300 0x5601f3aa4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bd9d9b700 1 --2- 172.21.15.44:0/3146322835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98cc0 0x5601f3aa2060 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0000c00 tx=0x7f3bd000bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b0bc80018c4270 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bc1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:07:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bc1ffb700 1 -- 172.21.15.44:0/3146322835 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0005510 con 0x5601f3a98cc0 2023-09-10T01:07:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bc1ffb700 1 -- 172.21.15.44:0/3146322835 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f3bd0005670 con 0x5601f3a98cc0 2023-09-10T01:07:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3becaf0b80 1 -- 172.21.15.44:0/3146322835 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601f3942490 con 0x5601f3a98cc0 2023-09-10T01:07:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3bc1ffb700 1 -- 172.21.15.44:0/3146322835 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bd0003760 con 0x5601f3a98cc0 2023-09-10T01:07:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3becaf0b80 10 client.28460 fetch_fsmap learned FSMap version 107 2023-09-10T01:07:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.324+0000 7f3becaf0b80 1 -- 172.21.15.44:0/3146322835 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) v3 -- 0x7f3bdc10dd50 con 0x5601f3a98cc0 2023-09-10T01:07:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.325+0000 7f3bc1ffb700 1 -- 172.21.15.44:0/3146322835 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5601f3942490 con 0x5601f3a98cc0 2023-09-10T01:07:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.325+0000 7f3bc1ffb700 1 -- 172.21.15.44:0/3146322835 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 107) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3bd000f810 con 0x5601f3a98cc0 2023-09-10T01:07:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.325+0000 7f3becaf0b80 10 client.28460 fetch_fsmap finished waiting for FSMap version 107 2023-09-10T01:07:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.325+0000 7f3becaf0b80 10 client.28460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:07:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.325+0000 7f3becaf0b80 20 client.28460 populate_metadata read hostname 'smithi044' 2023-09-10T01:07:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.325+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/3146322835 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5601f3ab1e80 0x5601f3ab42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.325+0000 7f3becaf0b80 4 client.28460 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:07:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.325+0000 7f3becaf0b80 1 -- 172.21.15.44:0/3146322835 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601f39ff840 con 0x5601f3ab1e80 2023-09-10T01:07:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.325+0000 7f3bda59c700 1 --2- 172.21.15.44:0/3146322835 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5601f3ab1e80 0x5601f3ab42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.325+0000 7f3bda59c700 1 --2- 172.21.15.44:0/3146322835 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5601f3ab1e80 0x5601f3ab42a0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db7e8470b05b8d4d server_cookie=56d6fd3a1dbb4070 in_seq=0 out_seq=0 2023-09-10T01:07:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.325+0000 7f3bc1ffb700 10 client.28460 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.326+0000 7f3bc1ffb700 1 -- 172.21.15.44:0/3146322835 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5601f39ff840 con 0x5601f3ab1e80 2023-09-10T01:07:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.326+0000 7f3bc1ffb700 10 client.28460 handle_command_reply: tid=0 2023-09-10T01:07:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.326+0000 7f3becaf0b80 1 client.28460 shutdown 2023-09-10T01:07:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.326+0000 7f3becaf0b80 1 -- 172.21.15.44:0/3146322835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98cc0 msgr2=0x5601f3aa2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.326+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/3146322835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98cc0 0x5601f3aa2060 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0000c00 tx=0x7f3bd000bcd0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.326+0000 7f3becaf0b80 1 -- 172.21.15.44:0/3146322835 shutdown_connections 2023-09-10T01:07:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.326+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/3146322835 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5601f3ab1e80 0x5601f3ab42a0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.326+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/3146322835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a9e1a0 0x5601f3aa2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.326+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/3146322835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a98300 0x5601f3aa4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.326+0000 7f3bc1ffb700 0 client.28460 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.326+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/3146322835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98cc0 0x5601f3aa2060 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.326+0000 7f3becaf0b80 1 -- 172.21.15.44:0/3146322835 >> 172.21.15.44:0/3146322835 conn(0x5601f3a0d030 msgr2=0x5601f3a927a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.326+0000 7f3becaf0b80 1 -- 172.21.15.44:0/3146322835 shutdown_connections 2023-09-10T01:07:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.326+0000 7f3becaf0b80 1 -- 172.21.15.44:0/3146322835 wait complete. 2023-09-10T01:07:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.327+0000 7f3becaf0b80 20 client.28460 trim_cache size 0 max 16384 2023-09-10T01:07:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3becaf0b80 1 Processor -- start 2023-09-10T01:07:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3becaf0b80 1 -- start start 2023-09-10T01:07:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3becaf0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98300 0x5601f3aa93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3becaf0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98cc0 0x5601f3aadb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3becaf0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a9e1a0 0x5601f3a9c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3becaf0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f3a99560 con 0x5601f3a98cc0 2023-09-10T01:07:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3becaf0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f3aa9920 con 0x5601f3a98300 2023-09-10T01:07:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3becaf0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f3a9cd90 con 0x5601f3a9e1a0 2023-09-10T01:07:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3bd9d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98cc0 0x5601f3aadb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3bd9d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98cc0 0x5601f3aadb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55896/0 (socket says 172.21.15.44:55896) 2023-09-10T01:07:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3bda59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98300 0x5601f3aa93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3bd9d9b700 1 -- 172.21.15.44:0/3108147748 learned_addr learned my addr 172.21.15.44:0/3108147748 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3bdad9d700 1 --2- 172.21.15.44:0/3108147748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a9e1a0 0x5601f3a9c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3bd9d9b700 1 -- 172.21.15.44:0/3108147748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a9e1a0 msgr2=0x5601f3a9c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3bd9d9b700 1 --2- 172.21.15.44:0/3108147748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a9e1a0 0x5601f3a9c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3bd9d9b700 1 -- 172.21.15.44:0/3108147748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98300 msgr2=0x5601f3aa93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3bd9d9b700 1 --2- 172.21.15.44:0/3108147748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98300 0x5601f3aa93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3bd9d9b700 1 -- 172.21.15.44:0/3108147748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601f37ffc00 con 0x5601f3a98cc0 2023-09-10T01:07:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.342+0000 7f3bda59c700 1 --2- 172.21.15.44:0/3108147748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98300 0x5601f3aa93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.343+0000 7f3bd9d9b700 1 --2- 172.21.15.44:0/3108147748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98cc0 0x5601f3aadb40 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0001eb0 tx=0x7f3bd0009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6a8d30bd1055bf server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.343+0000 7f3bc97fa700 1 -- 172.21.15.44:0/3108147748 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00058e0 con 0x5601f3a98cc0 2023-09-10T01:07:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.343+0000 7f3bc97fa700 1 -- 172.21.15.44:0/3108147748 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3bd000f460 con 0x5601f3a98cc0 2023-09-10T01:07:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.343+0000 7f3bc97fa700 1 -- 172.21.15.44:0/3108147748 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00039c0 con 0x5601f3a98cc0 2023-09-10T01:07:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.343+0000 7f3becaf0b80 1 -- 172.21.15.44:0/3108147748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98cc0 msgr2=0x5601f3aadb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.343+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/3108147748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98cc0 0x5601f3aadb40 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0001eb0 tx=0x7f3bd0009fd0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.343+0000 7f3becaf0b80 1 -- 172.21.15.44:0/3108147748 shutdown_connections 2023-09-10T01:07:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.343+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/3108147748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a9e1a0 0x5601f3a9c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.343+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/3108147748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98300 0x5601f3aa93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.343+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/3108147748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98cc0 0x5601f3aadb40 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.343+0000 7f3becaf0b80 1 -- 172.21.15.44:0/3108147748 >> 172.21.15.44:0/3108147748 conn(0x5601f3a0d030 msgr2=0x5601f3aae570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.343+0000 7f3becaf0b80 1 -- 172.21.15.44:0/3108147748 shutdown_connections 2023-09-10T01:07:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.343+0000 7f3becaf0b80 1 -- 172.21.15.44:0/3108147748 wait complete. 2023-09-10T01:07:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.344+0000 7f3becaf0b80 1 -- start start 2023-09-10T01:07:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.344+0000 7f3becaf0b80 1 Processor -- start 2023-09-10T01:07:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.344+0000 7f3becaf0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98300 0x5601f3aa53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.344+0000 7f3becaf0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98cc0 0x5601f3aa5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.344+0000 7f3becaf0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a9e1a0 0x5601f3aa5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.344+0000 7f3becaf0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f3aa9770 con 0x5601f3a98300 2023-09-10T01:07:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.344+0000 7f3becaf0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f3aa98e0 con 0x5601f3a98cc0 2023-09-10T01:07:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.344+0000 7f3becaf0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f3b04cf0 con 0x5601f3a9e1a0 2023-09-10T01:07:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.344+0000 7f3becaf0b80 10 client.0 _ll_register_callbacks cb 0x7f3bcc009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:07:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.344+0000 7f3bda59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98300 0x5601f3aa53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.344+0000 7f3bda59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98300 0x5601f3aa53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55900/0 (socket says 172.21.15.44:55900) 2023-09-10T01:07:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.344+0000 7f3bda59c700 1 -- 172.21.15.44:0/2692352 learned_addr learned my addr 172.21.15.44:0/2692352 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.344+0000 7f3bd9d9b700 1 --2- 172.21.15.44:0/2692352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98cc0 0x5601f3aa5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.344+0000 7f3bdad9d700 1 --2- 172.21.15.44:0/2692352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a9e1a0 0x5601f3aa5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.345+0000 7f3bd9d9b700 1 -- 172.21.15.44:0/2692352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a9e1a0 msgr2=0x5601f3aa5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.345+0000 7f3bd9d9b700 1 --2- 172.21.15.44:0/2692352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a9e1a0 0x5601f3aa5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.345+0000 7f3bd9d9b700 1 -- 172.21.15.44:0/2692352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98300 msgr2=0x5601f3aa53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.345+0000 7f3bd9d9b700 1 --2- 172.21.15.44:0/2692352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98300 0x5601f3aa53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.345+0000 7f3bd9d9b700 1 -- 172.21.15.44:0/2692352 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc009ab0 con 0x5601f3a98cc0 2023-09-10T01:07:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.345+0000 7f3bda59c700 1 --2- 172.21.15.44:0/2692352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98300 0x5601f3aa53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.345+0000 7f3bd9d9b700 1 --2- 172.21.15.44:0/2692352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98cc0 0x5601f3aa5900 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000b580 tx=0x7f3bd000f800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7056ac9ed39a970 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.345+0000 7f3bc1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:07:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.345+0000 7f3bc1ffb700 1 -- 172.21.15.44:0/2692352 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00058e0 con 0x5601f3a98cc0 2023-09-10T01:07:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.345+0000 7f3becaf0b80 1 -- 172.21.15.44:0/2692352 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3bd001e040 con 0x5601f3a98cc0 2023-09-10T01:07:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.345+0000 7f3bc1ffb700 1 -- 172.21.15.44:0/2692352 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f3bd0000e40 con 0x5601f3a98cc0 2023-09-10T01:07:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.345+0000 7f3bc1ffb700 1 -- 172.21.15.44:0/2692352 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bd000bba0 con 0x5601f3a98cc0 2023-09-10T01:07:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.345+0000 7f3becaf0b80 10 client.28466 fetch_fsmap learned FSMap version 107 2023-09-10T01:07:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.345+0000 7f3becaf0b80 1 -- 172.21.15.44:0/2692352 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) v3 -- 0x7f3bdc10dd50 con 0x5601f3a98cc0 2023-09-10T01:07:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.346+0000 7f3bc1ffb700 1 -- 172.21.15.44:0/2692352 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3bd001e040 con 0x5601f3a98cc0 2023-09-10T01:07:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.346+0000 7f3bc1ffb700 1 -- 172.21.15.44:0/2692352 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 107) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3bd002b700 con 0x5601f3a98cc0 2023-09-10T01:07:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.346+0000 7f3becaf0b80 10 client.28466 fetch_fsmap finished waiting for FSMap version 107 2023-09-10T01:07:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.346+0000 7f3becaf0b80 10 client.28466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:07:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.346+0000 7f3becaf0b80 20 client.28466 populate_metadata read hostname 'smithi044' 2023-09-10T01:07:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.346+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/2692352 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5601f3b06020 0x5601f3b08440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.346+0000 7f3becaf0b80 4 client.28466 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:07:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.346+0000 7f3becaf0b80 1 -- 172.21.15.44:0/2692352 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5601f39e38c0 con 0x5601f3b06020 2023-09-10T01:07:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.346+0000 7f3bda59c700 1 --2- 172.21.15.44:0/2692352 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5601f3b06020 0x5601f3b08440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.346+0000 7f3bc1ffb700 10 client.28466 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:02.346+0000 7f3bda59c700 1 --2- 172.21.15.44:0/2692352 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5601f3b06020 0x5601f3b08440 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e18a5915fd97bbd8 server_cookie=d5e2d264e53dec14 in_seq=0 out_seq=0 2023-09-10T01:07:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:02 smithi131 ceph-mon[103713]: pgmap v836: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 230 op/s 2023-09-10T01:07:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:02 smithi044 ceph-mon[98676]: pgmap v836: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 230 op/s 2023-09-10T01:07:03.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:02 smithi099 ceph-mon[102692]: pgmap v836: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 230 op/s 2023-09-10T01:07:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.426+0000 7f3bc1ffb700 1 -- 172.21.15.44:0/2692352 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5601f39e38c0 con 0x5601f3b06020 2023-09-10T01:07:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.426+0000 7f3bc1ffb700 10 client.28466 handle_command_reply: tid=0 2023-09-10T01:07:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.426+0000 7f3becaf0b80 1 client.28466 shutdown 2023-09-10T01:07:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.426+0000 7f3becaf0b80 1 -- 172.21.15.44:0/2692352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98cc0 msgr2=0x5601f3aa5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.426+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/2692352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98cc0 0x5601f3aa5900 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000b580 tx=0x7f3bd000f800 comp rx=0 tx=0).stop 2023-09-10T01:07:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.426+0000 7f3becaf0b80 1 -- 172.21.15.44:0/2692352 shutdown_connections 2023-09-10T01:07:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.426+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/2692352 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5601f3b06020 0x5601f3b08440 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.426+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/2692352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601f3a9e1a0 0x5601f3aa5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.426+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/2692352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601f3a98300 0x5601f3aa53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.426+0000 7f3becaf0b80 1 --2- 172.21.15.44:0/2692352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601f3a98cc0 0x5601f3aa5900 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.426+0000 7f3bc1ffb700 0 client.28466 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.426+0000 7f3becaf0b80 1 -- 172.21.15.44:0/2692352 >> 172.21.15.44:0/2692352 conn(0x5601f3a0d030 msgr2=0x5601f3a9c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.426+0000 7f3becaf0b80 1 -- 172.21.15.44:0/2692352 shutdown_connections 2023-09-10T01:07:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.426+0000 7f3becaf0b80 1 -- 172.21.15.44:0/2692352 wait complete. 2023-09-10T01:07:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.426+0000 7f3becaf0b80 20 client.28466 trim_cache size 0 max 16384 2023-09-10T01:07:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:07:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:07:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:07:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:07:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.427+0000 7f3be092a700 1 -- 172.21.15.44:0/62088298 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3bc40a07b0 msgr2=0x7f3bc40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.427+0000 7f3be092a700 1 --2- 172.21.15.44:0/62088298 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3bc40a07b0 0x7f3bc40a2c70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009d70 tx=0x7f3bcc009260 comp rx=0 tx=0).stop 2023-09-10T01:07:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.427+0000 7f3be092a700 1 -- 172.21.15.44:0/62088298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bdc106690 msgr2=0x7f3bdc1963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.427+0000 7f3be092a700 1 --2- 172.21.15.44:0/62088298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bdc106690 0x7f3bdc1963a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400ef30 tx=0x7f3bd400c3a0 comp rx=0 tx=0).stop 2023-09-10T01:07:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.427+0000 7f3be092a700 1 -- 172.21.15.44:0/62088298 shutdown_connections 2023-09-10T01:07:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.427+0000 7f3be092a700 1 --2- 172.21.15.44:0/62088298 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3bc40a07b0 0x7f3bc40a2c70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.427+0000 7f3be092a700 1 --2- 172.21.15.44:0/62088298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bdc106690 0x7f3bdc1963a0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.427+0000 7f3be092a700 1 --2- 172.21.15.44:0/62088298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bdc1005d0 0x7f3bdc19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.427+0000 7f3be092a700 1 --2- 172.21.15.44:0/62088298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bdc100f90 0x7f3bdc19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.427+0000 7f3be092a700 1 -- 172.21.15.44:0/62088298 >> 172.21.15.44:0/62088298 conn(0x7f3bdc0fc0c0 msgr2=0x7f3bdc0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.427+0000 7f3be092a700 1 -- 172.21.15.44:0/62088298 shutdown_connections 2023-09-10T01:07:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:03.427+0000 7f3be092a700 1 -- 172.21.15.44:0/62088298 wait complete. 2023-09-10T01:07:04.470 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: *** Caught signal (Segmentation fault) ** 2023-09-10T01:07:04.470 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: in thread 7fea1be8c700 thread_name:md_log_replay 2023-09-10T01:07:04.470 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:07:04.471 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fea2ca3bcf0] 2023-09-10T01:07:04.471 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x559bad5472f2] 2023-09-10T01:07:04.471 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559bad549543] 2023-09-10T01:07:04.471 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 4: (MDLog::_replay_thread()+0x647) [0x559bad5b2e97] 2023-09-10T01:07:04.472 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559bad5bc381] 2023-09-10T01:07:04.472 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fea2ca311ca] 2023-09-10T01:07:04.472 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 7: clone() 2023-09-10T01:07:04.472 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 2023-09-10T01:07:04.098+0000 7fea1be8c700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:07:04.472 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: in thread 7fea1be8c700 thread_name:md_log_replay 2023-09-10T01:07:04.472 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 2023-09-10T01:07:04.473 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:07:04.473 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fea2ca3bcf0] 2023-09-10T01:07:04.473 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x559bad5472f2] 2023-09-10T01:07:04.473 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559bad549543] 2023-09-10T01:07:04.473 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 4: (MDLog::_replay_thread()+0x647) [0x559bad5b2e97] 2023-09-10T01:07:04.474 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559bad5bc381] 2023-09-10T01:07:04.474 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fea2ca311ca] 2023-09-10T01:07:04.474 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 7: clone() 2023-09-10T01:07:04.474 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:07:04.474 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 2023-09-10T01:07:04.475 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 0> 2023-09-10T01:07:04.098+0000 7fea1be8c700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:07:04.475 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: in thread 7fea1be8c700 thread_name:md_log_replay 2023-09-10T01:07:04.475 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 2023-09-10T01:07:04.475 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:07:04.475 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fea2ca3bcf0] 2023-09-10T01:07:04.476 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x559bad5472f2] 2023-09-10T01:07:04.476 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559bad549543] 2023-09-10T01:07:04.476 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 4: (MDLog::_replay_thread()+0x647) [0x559bad5b2e97] 2023-09-10T01:07:04.476 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559bad5bc381] 2023-09-10T01:07:04.476 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fea2ca311ca] 2023-09-10T01:07:04.476 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 7: clone() 2023-09-10T01:07:04.477 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:07:04.477 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 2023-09-10T01:07:04.477 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 0> 2023-09-10T01:07:04.098+0000 7fea1be8c700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:07:04.477 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: in thread 7fea1be8c700 thread_name:md_log_replay 2023-09-10T01:07:04.477 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 2023-09-10T01:07:04.478 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:07:04.478 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fea2ca3bcf0] 2023-09-10T01:07:04.478 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x559bad5472f2] 2023-09-10T01:07:04.478 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559bad549543] 2023-09-10T01:07:04.478 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 4: (MDLog::_replay_thread()+0x647) [0x559bad5b2e97] 2023-09-10T01:07:04.479 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559bad5bc381] 2023-09-10T01:07:04.479 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fea2ca311ca] 2023-09-10T01:07:04.479 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 7: clone() 2023-09-10T01:07:04.479 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:07:04.479 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150192]: 2023-09-10T01:07:04.754 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:04 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Main process exited, code=exited, status=139/n/a 2023-09-10T01:07:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-mon[103713]: pgmap v837: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 911 KiB/s wr, 129 op/s 2023-09-10T01:07:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:04 smithi131 ceph-mon[103713]: osdmap e135: 12 total, 12 up, 12 in 2023-09-10T01:07:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:04 smithi044 ceph-mon[98676]: pgmap v837: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 911 KiB/s wr, 129 op/s 2023-09-10T01:07:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:04 smithi044 ceph-mon[98676]: osdmap e135: 12 total, 12 up, 12 in 2023-09-10T01:07:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:04 smithi099 ceph-mon[102692]: pgmap v837: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 911 KiB/s wr, 129 op/s 2023-09-10T01:07:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:04 smithi099 ceph-mon[102692]: osdmap e135: 12 total, 12 up, 12 in 2023-09-10T01:07:05.545 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:05 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Failed with result 'exit-code'. 2023-09-10T01:07:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:06 smithi131 ceph-mon[103713]: pgmap v839: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2023-09-10T01:07:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:06 smithi044 ceph-mon[98676]: pgmap v839: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2023-09-10T01:07:07.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:06 smithi099 ceph-mon[102692]: pgmap v839: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2023-09-10T01:07:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:08 smithi131 ceph-mon[103713]: pgmap v840: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2023-09-10T01:07:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:08 smithi044 ceph-mon[98676]: pgmap v840: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2023-09-10T01:07:09.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:08 smithi099 ceph-mon[102692]: pgmap v840: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2023-09-10T01:07:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:10 smithi131 ceph-mon[103713]: pgmap v841: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2023-09-10T01:07:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:10 smithi131 ceph-mon[103713]: osdmap e136: 12 total, 12 up, 12 in 2023-09-10T01:07:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:10 smithi044 ceph-mon[98676]: pgmap v841: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2023-09-10T01:07:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:10 smithi044 ceph-mon[98676]: osdmap e136: 12 total, 12 up, 12 in 2023-09-10T01:07:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:10 smithi099 ceph-mon[102692]: pgmap v841: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2023-09-10T01:07:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:10 smithi099 ceph-mon[102692]: osdmap e136: 12 total, 12 up, 12 in 2023-09-10T01:07:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:12 smithi131 ceph-mon[103713]: pgmap v843: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 284 op/s 2023-09-10T01:07:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:12 smithi044 ceph-mon[98676]: pgmap v843: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 284 op/s 2023-09-10T01:07:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:12 smithi099 ceph-mon[102692]: pgmap v843: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 284 op/s 2023-09-10T01:07:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:14 smithi131 ceph-mon[103713]: pgmap v844: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 238 op/s 2023-09-10T01:07:15.297 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:15 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Service RestartSec=10s expired, scheduling restart. 2023-09-10T01:07:15.297 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:15 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Scheduled restart job, restart counter is at 5. 2023-09-10T01:07:15.297 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:15 smithi131 systemd[1]: Stopped Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:07:15.297 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:15 smithi131 systemd[1]: Starting Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:07:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:14 smithi044 ceph-mon[98676]: pgmap v844: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 238 op/s 2023-09-10T01:07:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:14 smithi099 ceph-mon[102692]: pgmap v844: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 238 op/s 2023-09-10T01:07:15.696 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:15 smithi131 podman[150800]: 2023-09-10T01:07:16.295 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:16 smithi131 bash[150800]: e53c2d30eb3238c753289186d6898900c32cb6602e4c7d26bacaaa3e8166f3be 2023-09-10T01:07:16.295 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:16 smithi131 systemd[1]: Started Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:07:16.296 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:16 smithi131 ceph-mds[150840]: -- 172.21.15.131:0/327572326 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5603d7e771e0 con 0x5603d7e4cc00 2023-09-10T01:07:16.296 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:07:16 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: starting mds.c at 2023-09-10T01:07:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.562+0000 7efccdff7700 1 -- 172.21.15.44:0/3460153871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc80feaa0 msgr2=0x7efcc80fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.562+0000 7efccdff7700 1 --2- 172.21.15.44:0/3460153871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc80feaa0 0x7efcc80fef20 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7efcbc009990 tx=0x7efcbc009ca0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.562+0000 7efccdff7700 1 -- 172.21.15.44:0/3460153871 shutdown_connections 2023-09-10T01:07:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.562+0000 7efccdff7700 1 --2- 172.21.15.44:0/3460153871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc80ff460 0x7efcc810d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.562+0000 7efccdff7700 1 --2- 172.21.15.44:0/3460153871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcc8106620 0x7efcc8106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.562+0000 7efccdff7700 1 --2- 172.21.15.44:0/3460153871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc80feaa0 0x7efcc80fef20 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.562+0000 7efccdff7700 1 -- 172.21.15.44:0/3460153871 >> 172.21.15.44:0/3460153871 conn(0x7efcc80769d0 msgr2=0x7efcc80fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.562+0000 7efccdff7700 1 -- 172.21.15.44:0/3460153871 shutdown_connections 2023-09-10T01:07:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.562+0000 7efccdff7700 1 -- 172.21.15.44:0/3460153871 wait complete. 2023-09-10T01:07:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.562+0000 7efccdff7700 1 Processor -- start 2023-09-10T01:07:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efccdff7700 1 -- start start 2023-09-10T01:07:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efccdff7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc80feaa0 0x7efcc80ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efccdff7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc80ff460 0x7efcc81003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efccdff7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcc8106620 0x7efcc81a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efccdff7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc81a4c90 con 0x7efcc80feaa0 2023-09-10T01:07:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efccdff7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc81a4dd0 con 0x7efcc80ff460 2023-09-10T01:07:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efccdff7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc81a4f40 con 0x7efcc8106620 2023-09-10T01:07:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efcc77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc80feaa0 0x7efcc80ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efcc77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc80feaa0 0x7efcc80ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55914/0 (socket says 172.21.15.44:55914) 2023-09-10T01:07:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc80ff460 0x7efcc81003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efcc77fe700 1 -- 172.21.15.44:0/2067751720 learned_addr learned my addr 172.21.15.44:0/2067751720 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efcc7fff700 1 --2- 172.21.15.44:0/2067751720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcc8106620 0x7efcc81a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efcc77fe700 1 -- 172.21.15.44:0/2067751720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcc8106620 msgr2=0x7efcc81a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efcc77fe700 1 --2- 172.21.15.44:0/2067751720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcc8106620 0x7efcc81a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efcc77fe700 1 -- 172.21.15.44:0/2067751720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc80ff460 msgr2=0x7efcc81003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efcc77fe700 1 --2- 172.21.15.44:0/2067751720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc80ff460 0x7efcc81003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efcc77fe700 1 -- 172.21.15.44:0/2067751720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb8009520 con 0x7efcc80feaa0 2023-09-10T01:07:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.563+0000 7efcc7fff700 1 --2- 172.21.15.44:0/2067751720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcc8106620 0x7efcc81a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.564+0000 7efcc77fe700 1 --2- 172.21.15.44:0/2067751720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc80feaa0 0x7efcc80ffe60 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7efcb800eb30 tx=0x7efcb800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.564+0000 7efcc4ff9700 1 -- 172.21.15.44:0/2067751720 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8009880 con 0x7efcc80feaa0 2023-09-10T01:07:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.564+0000 7efccdff7700 1 -- 172.21.15.44:0/2067751720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcbc0095f0 con 0x7efcc80feaa0 2023-09-10T01:07:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.564+0000 7efcc4ff9700 1 -- 172.21.15.44:0/2067751720 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efcb80044b0 con 0x7efcc80feaa0 2023-09-10T01:07:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.564+0000 7efcc4ff9700 1 -- 172.21.15.44:0/2067751720 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8003ce0 con 0x7efcc80feaa0 2023-09-10T01:07:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.564+0000 7efccdff7700 1 -- 172.21.15.44:0/2067751720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc81a55a0 con 0x7efcc80feaa0 2023-09-10T01:07:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.566+0000 7efcc4ff9700 1 -- 172.21.15.44:0/2067751720 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7efcb8003e40 con 0x7efcc80feaa0 2023-09-10T01:07:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.567+0000 7efcc4ff9700 1 --2- 172.21.15.44:0/2067751720 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efcb00a07b0 0x7efcb00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.567+0000 7efcc4ff9700 1 -- 172.21.15.44:0/2067751720 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 9540+0+0 (secure 0 0 0) 0x7efcb8014070 con 0x7efcc80feaa0 2023-09-10T01:07:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.567+0000 7efcc6ffd700 1 --2- 172.21.15.44:0/2067751720 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efcb00a07b0 0x7efcb00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.568+0000 7efcc6ffd700 1 --2- 172.21.15.44:0/2067751720 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efcb00a07b0 0x7efcb00a2c70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efcc8101680 tx=0x7efcbc00b260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.568+0000 7efcda1bdb80 1 Processor -- start 2023-09-10T01:07:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.568+0000 7efcda1bdb80 1 -- start start 2023-09-10T01:07:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.568+0000 7efcda1bdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce9f1a0 0x56107ce9f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.568+0000 7efcda1bdb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99300 0x56107ce99780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.568+0000 7efcda1bdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 0x56107cea5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.568+0000 7efcda1bdb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107cdf26d0 con 0x56107ce99300 2023-09-10T01:07:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.568+0000 7efcda1bdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107cdefff0 con 0x56107ce9f1a0 2023-09-10T01:07:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.568+0000 7efcda1bdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107cdeb8e0 con 0x56107ce99cc0 2023-09-10T01:07:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.569+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99300 0x56107ce99780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.569+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99300 0x56107ce99780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55924/0 (socket says 172.21.15.44:55924) 2023-09-10T01:07:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.569+0000 7efcc6ffd700 1 -- 172.21.15.44:0/2026601516 learned_addr learned my addr 172.21.15.44:0/2026601516 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.569+0000 7efcc7fff700 1 --2- 172.21.15.44:0/2026601516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 0x56107cea5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.569+0000 7efcc77fe700 1 --2- 172.21.15.44:0/2026601516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce9f1a0 0x56107ce9f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.569+0000 7efcc6ffd700 1 -- 172.21.15.44:0/2026601516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 msgr2=0x56107cea5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.569+0000 7efcc6ffd700 1 --2- 172.21.15.44:0/2026601516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 0x56107cea5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.569+0000 7efcc6ffd700 1 -- 172.21.15.44:0/2026601516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce9f1a0 msgr2=0x56107ce9f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.569+0000 7efcc6ffd700 1 --2- 172.21.15.44:0/2026601516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce9f1a0 0x56107ce9f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.569+0000 7efcc6ffd700 1 -- 172.21.15.44:0/2026601516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56107cc17d80 con 0x56107ce99300 2023-09-10T01:07:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.569+0000 7efcc77fe700 1 --2- 172.21.15.44:0/2026601516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce9f1a0 0x56107ce9f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:07:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.569+0000 7efcc6ffd700 1 --2- 172.21.15.44:0/2026601516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99300 0x56107ce99780 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7efcbc015000 tx=0x7efcbc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1be19f628590d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.569+0000 7efcadffb700 1 -- 172.21.15.44:0/2026601516 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc00f660 con 0x56107ce99300 2023-09-10T01:07:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.569+0000 7efcadffb700 1 -- 172.21.15.44:0/2026601516 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efcbc01e040 con 0x56107ce99300 2023-09-10T01:07:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.570+0000 7efcadffb700 1 -- 172.21.15.44:0/2026601516 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc02f950 con 0x56107ce99300 2023-09-10T01:07:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.570+0000 7efcda1bdb80 1 -- 172.21.15.44:0/2026601516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99300 msgr2=0x56107ce99780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.570+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/2026601516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99300 0x56107ce99780 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7efcbc015000 tx=0x7efcbc020000 comp rx=0 tx=0).stop 2023-09-10T01:07:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.570+0000 7efcda1bdb80 1 -- 172.21.15.44:0/2026601516 shutdown_connections 2023-09-10T01:07:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.570+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/2026601516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 0x56107cea5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.570+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/2026601516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce9f1a0 0x56107ce9f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.570+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/2026601516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99300 0x56107ce99780 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.570+0000 7efcda1bdb80 1 -- 172.21.15.44:0/2026601516 >> 172.21.15.44:0/2026601516 conn(0x56107ce0e030 msgr2=0x56107ce94560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.571+0000 7efcda1bdb80 1 -- 172.21.15.44:0/2026601516 shutdown_connections 2023-09-10T01:07:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.571+0000 7efcda1bdb80 1 -- 172.21.15.44:0/2026601516 wait complete. 2023-09-10T01:07:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.571+0000 7efcda1bdb80 1 -- start start 2023-09-10T01:07:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.571+0000 7efcda1bdb80 1 Processor -- start 2023-09-10T01:07:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.571+0000 7efcda1bdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99300 0x56107cea5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.571+0000 7efcda1bdb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99cc0 0x56107cea3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.571+0000 7efcda1bdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce9f1a0 0x56107cea36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.571+0000 7efcda1bdb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107cd89b10 con 0x56107ce99cc0 2023-09-10T01:07:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.571+0000 7efcda1bdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107cd89810 con 0x56107ce9f1a0 2023-09-10T01:07:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.571+0000 7efcda1bdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107cc6d8c0 con 0x56107ce99300 2023-09-10T01:07:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.571+0000 7efcda1bdb80 10 client.0 _ll_register_callbacks cb 0x56107ccd3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:07:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.571+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99cc0 0x56107cea3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.571+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99cc0 0x56107cea3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55930/0 (socket says 172.21.15.44:55930) 2023-09-10T01:07:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.571+0000 7efcc77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99300 0x56107cea5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.571+0000 7efcc6ffd700 1 -- 172.21.15.44:0/2413863875 learned_addr learned my addr 172.21.15.44:0/2413863875 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.572+0000 7efcc7fff700 1 --2- 172.21.15.44:0/2413863875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce9f1a0 0x56107cea36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.572+0000 7efcc6ffd700 1 -- 172.21.15.44:0/2413863875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99300 msgr2=0x56107cea5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.572+0000 7efcc6ffd700 1 --2- 172.21.15.44:0/2413863875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99300 0x56107cea5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.572+0000 7efcc6ffd700 1 -- 172.21.15.44:0/2413863875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce9f1a0 msgr2=0x56107cea36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.572+0000 7efcc6ffd700 1 --2- 172.21.15.44:0/2413863875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce9f1a0 0x56107cea36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.572+0000 7efcc6ffd700 1 -- 172.21.15.44:0/2413863875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc810dd30 con 0x56107ce99cc0 2023-09-10T01:07:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.572+0000 7efcc77fe700 1 --2- 172.21.15.44:0/2413863875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99300 0x56107cea5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.572+0000 7efcc6ffd700 1 --2- 172.21.15.44:0/2413863875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99cc0 0x56107cea3070 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7efcbc000c00 tx=0x7efcbc00feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8bb0f13bee1fd6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.572+0000 7efca6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:07:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.572+0000 7efca6ffd700 1 -- 172.21.15.44:0/2413863875 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc00f870 con 0x56107ce99cc0 2023-09-10T01:07:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.572+0000 7efca6ffd700 1 -- 172.21.15.44:0/2413863875 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 9540+0+0 (secure 0 0 0) 0x7efcbc00f9d0 con 0x56107ce99cc0 2023-09-10T01:07:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.572+0000 7efcda1bdb80 1 -- 172.21.15.44:0/2413863875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56107cd43490 con 0x56107ce99cc0 2023-09-10T01:07:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.572+0000 7efca6ffd700 1 -- 172.21.15.44:0/2413863875 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcbc00bed0 con 0x56107ce99cc0 2023-09-10T01:07:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.572+0000 7efcda1bdb80 10 client.18831 fetch_fsmap learned FSMap version 107 2023-09-10T01:07:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.572+0000 7efcda1bdb80 1 -- 172.21.15.44:0/2413863875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) v3 -- 0x56107cc00c00 con 0x56107ce99cc0 2023-09-10T01:07:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.573+0000 7efca6ffd700 1 -- 172.21.15.44:0/2413863875 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56107cd43490 con 0x56107ce99cc0 2023-09-10T01:07:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.573+0000 7efca6ffd700 1 -- 172.21.15.44:0/2413863875 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 107) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efcbc042070 con 0x56107ce99cc0 2023-09-10T01:07:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.573+0000 7efcda1bdb80 10 client.18831 fetch_fsmap finished waiting for FSMap version 107 2023-09-10T01:07:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.573+0000 7efcda1bdb80 10 client.18831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:07:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.573+0000 7efcda1bdb80 20 client.18831 populate_metadata read hostname 'smithi044' 2023-09-10T01:07:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.573+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/2413863875 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56107ceb2df0 0x56107ceb51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.573+0000 7efcda1bdb80 4 client.18831 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:07:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.573+0000 7efcda1bdb80 1 -- 172.21.15.44:0/2413863875 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56107ce00840 con 0x56107ceb2df0 2023-09-10T01:07:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.573+0000 7efcc77fe700 1 --2- 172.21.15.44:0/2413863875 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56107ceb2df0 0x56107ceb51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.574+0000 7efcc77fe700 1 --2- 172.21.15.44:0/2413863875 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56107ceb2df0 0x56107ceb51e0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1546a6994dad0c59 server_cookie=be069cc15023248b in_seq=0 out_seq=0 2023-09-10T01:07:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.574+0000 7efca6ffd700 10 client.18831 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.574+0000 7efca6ffd700 1 -- 172.21.15.44:0/2413863875 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56107ce00840 con 0x56107ceb2df0 2023-09-10T01:07:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.574+0000 7efca6ffd700 10 client.18831 handle_command_reply: tid=0 2023-09-10T01:07:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.574+0000 7efcda1bdb80 1 client.18831 shutdown 2023-09-10T01:07:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.574+0000 7efcda1bdb80 1 -- 172.21.15.44:0/2413863875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99cc0 msgr2=0x56107cea3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.574+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/2413863875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99cc0 0x56107cea3070 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7efcbc000c00 tx=0x7efcbc00feb0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.575+0000 7efcda1bdb80 1 -- 172.21.15.44:0/2413863875 shutdown_connections 2023-09-10T01:07:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.575+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/2413863875 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56107ceb2df0 0x56107ceb51e0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.575+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/2413863875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce9f1a0 0x56107cea36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.575+0000 7efca6ffd700 0 client.18831 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.575+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/2413863875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99300 0x56107cea5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.575+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/2413863875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99cc0 0x56107cea3070 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.575+0000 7efcda1bdb80 1 -- 172.21.15.44:0/2413863875 >> 172.21.15.44:0/2413863875 conn(0x56107ce0e030 msgr2=0x56107ce937a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.575+0000 7efcda1bdb80 1 -- 172.21.15.44:0/2413863875 shutdown_connections 2023-09-10T01:07:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.575+0000 7efcda1bdb80 1 -- 172.21.15.44:0/2413863875 wait complete. 2023-09-10T01:07:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.575+0000 7efcda1bdb80 20 client.18831 trim_cache size 0 max 16384 2023-09-10T01:07:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.591+0000 7efcda1bdb80 1 Processor -- start 2023-09-10T01:07:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.591+0000 7efcda1bdb80 1 -- start start 2023-09-10T01:07:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcda1bdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce99300 0x56107ceaeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcda1bdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 0x56107ce9d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcda1bdb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce9f1a0 0x56107ce9d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcda1bdb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107ce8e8e0 con 0x56107ce9f1a0 2023-09-10T01:07:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcda1bdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107ce95cd0 con 0x56107ce99300 2023-09-10T01:07:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcda1bdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107cea16c0 con 0x56107ce99cc0 2023-09-10T01:07:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 0x56107ce9d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcc7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce9f1a0 0x56107ce9d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcc77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce99300 0x56107ceaeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 0x56107ce9d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34588/0 (socket says 172.21.15.44:34588) 2023-09-10T01:07:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcc77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce99300 0x56107ceaeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57676/0 (socket says 172.21.15.44:57676) 2023-09-10T01:07:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcc6ffd700 1 -- 172.21.15.44:0/389285553 learned_addr learned my addr 172.21.15.44:0/389285553 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcc7fff700 1 -- 172.21.15.44:0/389285553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 msgr2=0x56107ce9d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcc7fff700 1 --2- 172.21.15.44:0/389285553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 0x56107ce9d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcc7fff700 1 -- 172.21.15.44:0/389285553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce99300 msgr2=0x56107ceaeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcc7fff700 1 --2- 172.21.15.44:0/389285553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce99300 0x56107ceaeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcc7fff700 1 -- 172.21.15.44:0/389285553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56107cc3e1f0 con 0x56107ce9f1a0 2023-09-10T01:07:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcc6ffd700 1 --2- 172.21.15.44:0/389285553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 0x56107ce9d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcc77fe700 1 --2- 172.21.15.44:0/389285553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce99300 0x56107ceaeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.592+0000 7efcc7fff700 1 --2- 172.21.15.44:0/389285553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce9f1a0 0x56107ce9d6f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009530 tx=0x7efcc0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c45bf8a7d2ef4d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.593+0000 7efcae7fc700 1 -- 172.21.15.44:0/389285553 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc0020430 con 0x56107ce9f1a0 2023-09-10T01:07:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.593+0000 7efcae7fc700 1 -- 172.21.15.44:0/389285553 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efcc001b070 con 0x56107ce9f1a0 2023-09-10T01:07:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.593+0000 7efcae7fc700 1 -- 172.21.15.44:0/389285553 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc0007710 con 0x56107ce9f1a0 2023-09-10T01:07:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.593+0000 7efcda1bdb80 1 -- 172.21.15.44:0/389285553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce9f1a0 msgr2=0x56107ce9d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.593+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/389285553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce9f1a0 0x56107ce9d6f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009530 tx=0x7efcc0009920 comp rx=0 tx=0).stop 2023-09-10T01:07:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.594+0000 7efcda1bdb80 1 -- 172.21.15.44:0/389285553 shutdown_connections 2023-09-10T01:07:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.594+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/389285553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce9f1a0 0x56107ce9d6f0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.594+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/389285553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce99300 0x56107ceaeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.594+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/389285553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 0x56107ce9d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.594+0000 7efcda1bdb80 1 -- 172.21.15.44:0/389285553 >> 172.21.15.44:0/389285553 conn(0x56107ce0e030 msgr2=0x56107ceb9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.594+0000 7efcda1bdb80 1 -- 172.21.15.44:0/389285553 shutdown_connections 2023-09-10T01:07:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.594+0000 7efcda1bdb80 1 -- 172.21.15.44:0/389285553 wait complete. 2023-09-10T01:07:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.594+0000 7efcda1bdb80 1 -- start start 2023-09-10T01:07:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.594+0000 7efcda1bdb80 1 Processor -- start 2023-09-10T01:07:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcda1bdb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99300 0x56107cea83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcda1bdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 0x56107cea8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcda1bdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce9f1a0 0x56107cea8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcda1bdb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107cf05cf0 con 0x56107ce99300 2023-09-10T01:07:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcda1bdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107cf05e60 con 0x56107ce9f1a0 2023-09-10T01:07:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcda1bdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107cf06020 con 0x56107ce99cc0 2023-09-10T01:07:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcda1bdb80 10 client.0 _ll_register_callbacks cb 0x7efcc0009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:07:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcc77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99300 0x56107cea83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcc77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99300 0x56107cea83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55942/0 (socket says 172.21.15.44:55942) 2023-09-10T01:07:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 0x56107cea8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcc77fe700 1 -- 172.21.15.44:0/1525367295 learned_addr learned my addr 172.21.15.44:0/1525367295 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcc7fff700 1 --2- 172.21.15.44:0/1525367295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce9f1a0 0x56107cea8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcc77fe700 1 -- 172.21.15.44:0/1525367295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 msgr2=0x56107cea8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcc77fe700 1 --2- 172.21.15.44:0/1525367295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 0x56107cea8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcc77fe700 1 -- 172.21.15.44:0/1525367295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce9f1a0 msgr2=0x56107cea8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcc77fe700 1 --2- 172.21.15.44:0/1525367295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce9f1a0 0x56107cea8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcc77fe700 1 -- 172.21.15.44:0/1525367295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc81a55a0 con 0x56107ce99300 2023-09-10T01:07:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcc6ffd700 1 --2- 172.21.15.44:0/1525367295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 0x56107cea8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcc77fe700 1 --2- 172.21.15.44:0/1525367295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99300 0x56107cea83e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7efcb80c9c60 tx=0x7efcb8004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b5c43167f3dc0f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efca6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:07:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efca6ffd700 1 -- 172.21.15.44:0/1525367295 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8099910 con 0x56107ce99300 2023-09-10T01:07:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efca6ffd700 1 -- 172.21.15.44:0/1525367295 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 9540+0+0 (secure 0 0 0) 0x7efcb809d740 con 0x56107ce99300 2023-09-10T01:07:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.595+0000 7efcda1bdb80 1 -- 172.21.15.44:0/1525367295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcbc01e040 con 0x56107ce99300 2023-09-10T01:07:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.596+0000 7efca6ffd700 1 -- 172.21.15.44:0/1525367295 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcb80a1980 con 0x56107ce99300 2023-09-10T01:07:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.596+0000 7efcda1bdb80 10 client.18837 fetch_fsmap learned FSMap version 107 2023-09-10T01:07:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.596+0000 7efcda1bdb80 1 -- 172.21.15.44:0/1525367295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) v3 -- 0x56107cc00c00 con 0x56107ce99300 2023-09-10T01:07:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.596+0000 7efca6ffd700 1 -- 172.21.15.44:0/1525367295 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efcbc01e040 con 0x56107ce99300 2023-09-10T01:07:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.596+0000 7efca6ffd700 1 -- 172.21.15.44:0/1525367295 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 107) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efcb8099c70 con 0x56107ce99300 2023-09-10T01:07:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.596+0000 7efcda1bdb80 10 client.18837 fetch_fsmap finished waiting for FSMap version 107 2023-09-10T01:07:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.596+0000 7efcda1bdb80 10 client.18837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:07:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.596+0000 7efcda1bdb80 20 client.18837 populate_metadata read hostname 'smithi044' 2023-09-10T01:07:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.596+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/1525367295 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56107cf076a0 0x56107cf09ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.596+0000 7efcda1bdb80 4 client.18837 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:07:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.596+0000 7efcda1bdb80 1 -- 172.21.15.44:0/1525367295 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56107cde48c0 con 0x56107cf076a0 2023-09-10T01:07:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.596+0000 7efcc6ffd700 1 --2- 172.21.15.44:0/1525367295 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56107cf076a0 0x56107cf09ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.597+0000 7efca6ffd700 10 client.18837 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.597+0000 7efcc6ffd700 1 --2- 172.21.15.44:0/1525367295 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56107cf076a0 0x56107cf09ac0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7ec74a0ea0b8783 server_cookie=b5b3b9087da59d87 in_seq=0 out_seq=0 2023-09-10T01:07:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.610+0000 7efca6ffd700 1 -- 172.21.15.44:0/1525367295 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56107cde48c0 con 0x56107cf076a0 2023-09-10T01:07:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.610+0000 7efca6ffd700 10 client.18837 handle_command_reply: tid=0 2023-09-10T01:07:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.610+0000 7efcda1bdb80 1 client.18837 shutdown 2023-09-10T01:07:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.611+0000 7efcda1bdb80 1 -- 172.21.15.44:0/1525367295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99300 msgr2=0x56107cea83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.611+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/1525367295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99300 0x56107cea83e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7efcb80c9c60 tx=0x7efcb8004620 comp rx=0 tx=0).stop 2023-09-10T01:07:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.611+0000 7efcda1bdb80 1 -- 172.21.15.44:0/1525367295 shutdown_connections 2023-09-10T01:07:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.611+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/1525367295 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56107cf076a0 0x56107cf09ac0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.611+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/1525367295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56107ce9f1a0 0x56107cea8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.611+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/1525367295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56107ce99300 0x56107cea83e0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.611+0000 7efca6ffd700 0 client.18837 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.611+0000 7efcda1bdb80 1 --2- 172.21.15.44:0/1525367295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56107ce99cc0 0x56107cea8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.611+0000 7efcda1bdb80 1 -- 172.21.15.44:0/1525367295 >> 172.21.15.44:0/1525367295 conn(0x56107ce0e030 msgr2=0x56107ceaeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.611+0000 7efcda1bdb80 1 -- 172.21.15.44:0/1525367295 shutdown_connections 2023-09-10T01:07:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.611+0000 7efcda1bdb80 1 -- 172.21.15.44:0/1525367295 wait complete. 2023-09-10T01:07:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.611+0000 7efcda1bdb80 20 client.18837 trim_cache size 0 max 16384 2023-09-10T01:07:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:07:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:07:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:07:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:07:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.612+0000 7efccdff7700 1 -- 172.21.15.44:0/2067751720 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efcb00a07b0 msgr2=0x7efcb00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.612+0000 7efccdff7700 1 --2- 172.21.15.44:0/2067751720 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efcb00a07b0 0x7efcb00a2c70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efcc8101680 tx=0x7efcbc00b260 comp rx=0 tx=0).stop 2023-09-10T01:07:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.612+0000 7efccdff7700 1 -- 172.21.15.44:0/2067751720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc80feaa0 msgr2=0x7efcc80ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.612+0000 7efccdff7700 1 --2- 172.21.15.44:0/2067751720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc80feaa0 0x7efcc80ffe60 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7efcb800eb30 tx=0x7efcb800ee40 comp rx=0 tx=0).stop 2023-09-10T01:07:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.612+0000 7efccdff7700 1 -- 172.21.15.44:0/2067751720 shutdown_connections 2023-09-10T01:07:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.612+0000 7efccdff7700 1 --2- 172.21.15.44:0/2067751720 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efcb00a07b0 0x7efcb00a2c70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.612+0000 7efccdff7700 1 --2- 172.21.15.44:0/2067751720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcc8106620 0x7efcc81a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.612+0000 7efccdff7700 1 --2- 172.21.15.44:0/2067751720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc80feaa0 0x7efcc80ffe60 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.612+0000 7efccdff7700 1 --2- 172.21.15.44:0/2067751720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc80ff460 0x7efcc81003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.612+0000 7efccdff7700 1 -- 172.21.15.44:0/2067751720 >> 172.21.15.44:0/2067751720 conn(0x7efcc80769d0 msgr2=0x7efcc80fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.612+0000 7efccdff7700 1 -- 172.21.15.44:0/2067751720 shutdown_connections 2023-09-10T01:07:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:16.612+0000 7efccdff7700 1 -- 172.21.15.44:0/2067751720 wait complete. 2023-09-10T01:07:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:16 smithi131 ceph-mon[103713]: pgmap v845: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2023-09-10T01:07:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:16 smithi131 ceph-mon[103713]: daemon mds.c restarted 2023-09-10T01:07:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:16 smithi044 ceph-mon[98676]: pgmap v845: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2023-09-10T01:07:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:16 smithi044 ceph-mon[98676]: daemon mds.c restarted 2023-09-10T01:07:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:16 smithi099 ceph-mon[102692]: pgmap v845: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2023-09-10T01:07:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:16 smithi099 ceph-mon[102692]: daemon mds.c restarted 2023-09-10T01:07:18.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:17 smithi131 ceph-mon[103713]: osdmap e137: 12 total, 12 up, 12 in 2023-09-10T01:07:18.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:17 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.131:6836/849919326,v1:172.21.15.131:6837/849919326] up:boot 2023-09-10T01:07:18.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:17 smithi131 ceph-mon[103713]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-09-10T01:07:18.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:17 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T01:07:18.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:17 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:07:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:17 smithi044 ceph-mon[98676]: osdmap e137: 12 total, 12 up, 12 in 2023-09-10T01:07:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:17 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.131:6836/849919326,v1:172.21.15.131:6837/849919326] up:boot 2023-09-10T01:07:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:17 smithi044 ceph-mon[98676]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-09-10T01:07:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:17 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T01:07:18.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:17 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:07:18.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:17 smithi099 ceph-mon[102692]: osdmap e137: 12 total, 12 up, 12 in 2023-09-10T01:07:18.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:17 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.131:6836/849919326,v1:172.21.15.131:6837/849919326] up:boot 2023-09-10T01:07:18.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:17 smithi099 ceph-mon[102692]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-09-10T01:07:18.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:17 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-10T01:07:18.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:17 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:07:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:18 smithi131 ceph-mon[103713]: pgmap v847: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 276 op/s 2023-09-10T01:07:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:18 smithi044 ceph-mon[98676]: pgmap v847: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 276 op/s 2023-09-10T01:07:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:18 smithi099 ceph-mon[102692]: pgmap v847: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 276 op/s 2023-09-10T01:07:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:20 smithi131 ceph-mon[103713]: pgmap v848: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 855 KiB/s wr, 118 op/s 2023-09-10T01:07:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:20 smithi044 ceph-mon[98676]: pgmap v848: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 855 KiB/s wr, 118 op/s 2023-09-10T01:07:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:20 smithi099 ceph-mon[102692]: pgmap v848: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 855 KiB/s wr, 118 op/s 2023-09-10T01:07:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:22 smithi131 ceph-mon[103713]: pgmap v849: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2023-09-10T01:07:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:22 smithi044 ceph-mon[98676]: pgmap v849: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2023-09-10T01:07:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:22 smithi099 ceph-mon[102692]: pgmap v849: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2023-09-10T01:07:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:24 smithi131 ceph-mon[103713]: pgmap v850: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2023-09-10T01:07:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:24 smithi044 ceph-mon[98676]: pgmap v850: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2023-09-10T01:07:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:24 smithi099 ceph-mon[102692]: pgmap v850: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2023-09-10T01:07:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:26 smithi131 ceph-mon[103713]: pgmap v851: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2023-09-10T01:07:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:26 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:07:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:26 smithi044 ceph-mon[98676]: pgmap v851: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2023-09-10T01:07:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:26 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:07:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:26 smithi099 ceph-mon[102692]: pgmap v851: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2023-09-10T01:07:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:26 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:07:28.958 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:28 smithi044 ceph-mon[98676]: pgmap v852: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 174 op/s 2023-09-10T01:07:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:28 smithi131 ceph-mon[103713]: pgmap v852: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 174 op/s 2023-09-10T01:07:29.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:28 smithi099 ceph-mon[102692]: pgmap v852: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 174 op/s 2023-09-10T01:07:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.748+0000 7fa775c33700 1 -- 172.21.15.44:0/3479191388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7701017e0 msgr2=0x7fa770101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.748+0000 7fa775c33700 1 --2- 172.21.15.44:0/3479191388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7701017e0 0x7fa770101bc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa7600098f0 tx=0x7fa760009c00 comp rx=0 tx=0).stop 2023-09-10T01:07:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.748+0000 7fa775c33700 1 -- 172.21.15.44:0/3479191388 shutdown_connections 2023-09-10T01:07:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.748+0000 7fa775c33700 1 --2- 172.21.15.44:0/3479191388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa77010abb0 0x7fa77010d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.748+0000 7fa775c33700 1 --2- 172.21.15.44:0/3479191388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7701017e0 0x7fa770101bc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.748+0000 7fa775c33700 1 --2- 172.21.15.44:0/3479191388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa770102100 0x7fa77010a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.748+0000 7fa775c33700 1 -- 172.21.15.44:0/3479191388 >> 172.21.15.44:0/3479191388 conn(0x7fa7700faf40 msgr2=0x7fa7700fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.748+0000 7fa775c33700 1 -- 172.21.15.44:0/3479191388 shutdown_connections 2023-09-10T01:07:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.748+0000 7fa775c33700 1 -- 172.21.15.44:0/3479191388 wait complete. 2023-09-10T01:07:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.749+0000 7fa775c33700 1 Processor -- start 2023-09-10T01:07:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.749+0000 7fa775c33700 1 -- start start 2023-09-10T01:07:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa775c33700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7701017e0 0x7fa770109fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa775c33700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa770102100 0x7fa77010a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa775c33700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa77010abb0 0x7fa770104e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa775c33700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa770105610 con 0x7fa77010abb0 2023-09-10T01:07:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa775c33700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa770105780 con 0x7fa7701017e0 2023-09-10T01:07:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa775c33700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa770105940 con 0x7fa770102100 2023-09-10T01:07:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa76ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa77010abb0 0x7fa770104e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa76f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7701017e0 0x7fa770109fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa76effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa770102100 0x7fa77010a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa76f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7701017e0 0x7fa770109fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57696/0 (socket says 172.21.15.44:57696) 2023-09-10T01:07:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa76f7fe700 1 -- 172.21.15.44:0/326141875 learned_addr learned my addr 172.21.15.44:0/326141875 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa76f7fe700 1 -- 172.21.15.44:0/326141875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa770102100 msgr2=0x7fa77010a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa76f7fe700 1 --2- 172.21.15.44:0/326141875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa770102100 0x7fa77010a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa76f7fe700 1 -- 172.21.15.44:0/326141875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa77010abb0 msgr2=0x7fa770104e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa76f7fe700 1 --2- 172.21.15.44:0/326141875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa77010abb0 0x7fa770104e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa76f7fe700 1 -- 172.21.15.44:0/326141875 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa760009520 con 0x7fa7701017e0 2023-09-10T01:07:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa76ffff700 1 --2- 172.21.15.44:0/326141875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa77010abb0 0x7fa770104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.750+0000 7fa76f7fe700 1 --2- 172.21.15.44:0/326141875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7701017e0 0x7fa770109fa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa760009230 tx=0x7fa76000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.751+0000 7fa76cff9700 1 -- 172.21.15.44:0/326141875 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76001c070 con 0x7fa7701017e0 2023-09-10T01:07:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.751+0000 7fa76cff9700 1 -- 172.21.15.44:0/326141875 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa76000bb70 con 0x7fa7701017e0 2023-09-10T01:07:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.751+0000 7fa775c33700 1 -- 172.21.15.44:0/326141875 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa770105bc0 con 0x7fa7701017e0 2023-09-10T01:07:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.751+0000 7fa76cff9700 1 -- 172.21.15.44:0/326141875 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7600038e0 con 0x7fa7701017e0 2023-09-10T01:07:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.751+0000 7fa775c33700 1 -- 172.21.15.44:0/326141875 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7701abd30 con 0x7fa7701017e0 2023-09-10T01:07:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.753+0000 7fa76cff9700 1 -- 172.21.15.44:0/326141875 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa760020030 con 0x7fa7701017e0 2023-09-10T01:07:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.754+0000 7fa76cff9700 1 --2- 172.21.15.44:0/326141875 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa7580a0830 0x7fa7580a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.754+0000 7fa76cff9700 1 -- 172.21.15.44:0/326141875 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa7600cd170 con 0x7fa7701017e0 2023-09-10T01:07:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.754+0000 7fa76effd700 1 --2- 172.21.15.44:0/326141875 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa7580a0830 0x7fa7580a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.755+0000 7fa781df9b80 1 Processor -- start 2023-09-10T01:07:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.755+0000 7fa76effd700 1 --2- 172.21.15.44:0/326141875 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa7580a0830 0x7fa7580a2cf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa770106780 tx=0x7fa764009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.755+0000 7fa781df9b80 1 -- start start 2023-09-10T01:07:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.755+0000 7fa781df9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f26781a0 0x5593f2678580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.755+0000 7fa781df9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 0x5593f2672780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.755+0000 7fa781df9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672cc0 0x5593f267ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.755+0000 7fa781df9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f25cb6d0 con 0x5593f26781a0 2023-09-10T01:07:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.755+0000 7fa781df9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f25c8ff0 con 0x5593f2672cc0 2023-09-10T01:07:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.755+0000 7fa781df9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f25c48e0 con 0x5593f2672300 2023-09-10T01:07:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.755+0000 7fa76effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 0x5593f2672780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.755+0000 7fa76f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f26781a0 0x5593f2678580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.755+0000 7fa76effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 0x5593f2672780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34616/0 (socket says 172.21.15.44:34616) 2023-09-10T01:07:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.755+0000 7fa76effd700 1 -- 172.21.15.44:0/687848633 learned_addr learned my addr 172.21.15.44:0/687848633 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.755+0000 7fa76ffff700 1 --2- 172.21.15.44:0/687848633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672cc0 0x5593f267ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.756+0000 7fa76effd700 1 -- 172.21.15.44:0/687848633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672cc0 msgr2=0x5593f267ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.756+0000 7fa76effd700 1 --2- 172.21.15.44:0/687848633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672cc0 0x5593f267ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.756+0000 7fa76effd700 1 -- 172.21.15.44:0/687848633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f26781a0 msgr2=0x5593f2678580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.756+0000 7fa76effd700 1 --2- 172.21.15.44:0/687848633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f26781a0 0x5593f2678580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.756+0000 7fa76effd700 1 -- 172.21.15.44:0/687848633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593f23f0d80 con 0x5593f2672300 2023-09-10T01:07:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.756+0000 7fa76ffff700 1 --2- 172.21.15.44:0/687848633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672cc0 0x5593f267ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.756+0000 7fa76f7fe700 1 --2- 172.21.15.44:0/687848633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f26781a0 0x5593f2678580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.756+0000 7fa76effd700 1 --2- 172.21.15.44:0/687848633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 0x5593f2672780 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa764011d20 tx=0x7fa764014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42decc0256cd49c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.756+0000 7fa755ffb700 1 -- 172.21.15.44:0/687848633 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76400eb20 con 0x5593f2672300 2023-09-10T01:07:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.756+0000 7fa755ffb700 1 -- 172.21.15.44:0/687848633 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa764021070 con 0x5593f2672300 2023-09-10T01:07:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.756+0000 7fa755ffb700 1 -- 172.21.15.44:0/687848633 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76401d410 con 0x5593f2672300 2023-09-10T01:07:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.757+0000 7fa781df9b80 1 -- 172.21.15.44:0/687848633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 msgr2=0x5593f2672780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.757+0000 7fa781df9b80 1 --2- 172.21.15.44:0/687848633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 0x5593f2672780 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa764011d20 tx=0x7fa764014000 comp rx=0 tx=0).stop 2023-09-10T01:07:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.757+0000 7fa781df9b80 1 -- 172.21.15.44:0/687848633 shutdown_connections 2023-09-10T01:07:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.757+0000 7fa781df9b80 1 --2- 172.21.15.44:0/687848633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672cc0 0x5593f267ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.757+0000 7fa781df9b80 1 --2- 172.21.15.44:0/687848633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f26781a0 0x5593f2678580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.757+0000 7fa781df9b80 1 --2- 172.21.15.44:0/687848633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 0x5593f2672780 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.757+0000 7fa781df9b80 1 -- 172.21.15.44:0/687848633 >> 172.21.15.44:0/687848633 conn(0x5593f25e7030 msgr2=0x5593f266d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.757+0000 7fa781df9b80 1 -- 172.21.15.44:0/687848633 shutdown_connections 2023-09-10T01:07:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.757+0000 7fa781df9b80 1 -- 172.21.15.44:0/687848633 wait complete. 2023-09-10T01:07:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.757+0000 7fa781df9b80 1 -- start start 2023-09-10T01:07:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa781df9b80 1 Processor -- start 2023-09-10T01:07:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa781df9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 0x5593f267ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa781df9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672cc0 0x5593f267c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa781df9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f26781a0 0x5593f267c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa781df9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2562b10 con 0x5593f26781a0 2023-09-10T01:07:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa781df9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2562810 con 0x5593f2672cc0 2023-09-10T01:07:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa781df9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f24468c0 con 0x5593f2672300 2023-09-10T01:07:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa781df9b80 10 client.0 _ll_register_callbacks cb 0x5593f24aca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:07:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa76f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 0x5593f267ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa76effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672cc0 0x5593f267c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa76ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f26781a0 0x5593f267c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa76effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672cc0 0x5593f267c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57712/0 (socket says 172.21.15.44:57712) 2023-09-10T01:07:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa76f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 0x5593f267ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34620/0 (socket says 172.21.15.44:34620) 2023-09-10T01:07:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa76effd700 1 -- 172.21.15.44:0/3061766729 learned_addr learned my addr 172.21.15.44:0/3061766729 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa76effd700 1 -- 172.21.15.44:0/3061766729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 msgr2=0x5593f267ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa76effd700 1 --2- 172.21.15.44:0/3061766729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 0x5593f267ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa76effd700 1 -- 172.21.15.44:0/3061766729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f26781a0 msgr2=0x5593f267c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa76effd700 1 --2- 172.21.15.44:0/3061766729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f26781a0 0x5593f267c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.758+0000 7fa76effd700 1 -- 172.21.15.44:0/3061766729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa768003a60 con 0x5593f2672cc0 2023-09-10T01:07:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.759+0000 7fa76ffff700 1 --2- 172.21.15.44:0/3061766729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f26781a0 0x5593f267c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.759+0000 7fa76f7fe700 1 --2- 172.21.15.44:0/3061766729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 0x5593f267ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.759+0000 7fa76effd700 1 --2- 172.21.15.44:0/3061766729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672cc0 0x5593f267c030 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa76401cb60 tx=0x7fa76401cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7333e8d8dd2a1824 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.759+0000 7fa74effd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:07:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.759+0000 7fa74effd700 1 -- 172.21.15.44:0/3061766729 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76401cee0 con 0x5593f2672cc0 2023-09-10T01:07:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.759+0000 7fa74effd700 1 -- 172.21.15.44:0/3061766729 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa76400ed30 con 0x5593f2672cc0 2023-09-10T01:07:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.759+0000 7fa781df9b80 1 -- 172.21.15.44:0/3061766729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593f251c490 con 0x5593f2672cc0 2023-09-10T01:07:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.759+0000 7fa74effd700 1 -- 172.21.15.44:0/3061766729 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa76401c410 con 0x5593f2672cc0 2023-09-10T01:07:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.759+0000 7fa781df9b80 10 client.28499 fetch_fsmap learned FSMap version 109 2023-09-10T01:07:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.759+0000 7fa781df9b80 1 -- 172.21.15.44:0/3061766729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=4+}) v3 -- 0x7fa7701abd30 con 0x5593f2672cc0 2023-09-10T01:07:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.759+0000 7fa74effd700 1 -- 172.21.15.44:0/3061766729 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5593f251c490 con 0x5593f2672cc0 2023-09-10T01:07:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.760+0000 7fa74effd700 1 -- 172.21.15.44:0/3061766729 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 109) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa764003680 con 0x5593f2672cc0 2023-09-10T01:07:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.760+0000 7fa781df9b80 10 client.28499 fetch_fsmap finished waiting for FSMap version 109 2023-09-10T01:07:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.760+0000 7fa781df9b80 10 client.28499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:07:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.760+0000 7fa781df9b80 20 client.28499 populate_metadata read hostname 'smithi044' 2023-09-10T01:07:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.760+0000 7fa781df9b80 1 --2- 172.21.15.44:0/3061766729 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5593f268c220 0x5593f268e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.760+0000 7fa781df9b80 4 client.28499 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:07:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.760+0000 7fa781df9b80 1 -- 172.21.15.44:0/3061766729 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593f25d9840 con 0x5593f268c220 2023-09-10T01:07:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.760+0000 7fa76f7fe700 1 --2- 172.21.15.44:0/3061766729 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5593f268c220 0x5593f268e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.760+0000 7fa74effd700 10 client.28499 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.760+0000 7fa76f7fe700 1 --2- 172.21.15.44:0/3061766729 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5593f268c220 0x5593f268e640 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0a7033b06f1a77 server_cookie=46e748b0d90ea3d8 in_seq=0 out_seq=0 2023-09-10T01:07:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.761+0000 7fa74effd700 1 -- 172.21.15.44:0/3061766729 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5593f25d9840 con 0x5593f268c220 2023-09-10T01:07:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.761+0000 7fa74effd700 10 client.28499 handle_command_reply: tid=0 2023-09-10T01:07:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.761+0000 7fa781df9b80 1 client.28499 shutdown 2023-09-10T01:07:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.761+0000 7fa781df9b80 1 -- 172.21.15.44:0/3061766729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672cc0 msgr2=0x5593f267c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.761+0000 7fa781df9b80 1 --2- 172.21.15.44:0/3061766729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672cc0 0x5593f267c030 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa76401cb60 tx=0x7fa76401cbc0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.761+0000 7fa781df9b80 1 -- 172.21.15.44:0/3061766729 shutdown_connections 2023-09-10T01:07:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.761+0000 7fa781df9b80 1 --2- 172.21.15.44:0/3061766729 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5593f268c220 0x5593f268e640 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.761+0000 7fa781df9b80 1 --2- 172.21.15.44:0/3061766729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f26781a0 0x5593f267c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.761+0000 7fa74effd700 0 client.28499 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.761+0000 7fa781df9b80 1 --2- 172.21.15.44:0/3061766729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 0x5593f267ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.761+0000 7fa781df9b80 1 --2- 172.21.15.44:0/3061766729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672cc0 0x5593f267c030 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.761+0000 7fa781df9b80 1 -- 172.21.15.44:0/3061766729 >> 172.21.15.44:0/3061766729 conn(0x5593f25e7030 msgr2=0x5593f266c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.761+0000 7fa781df9b80 1 -- 172.21.15.44:0/3061766729 shutdown_connections 2023-09-10T01:07:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.761+0000 7fa781df9b80 1 -- 172.21.15.44:0/3061766729 wait complete. 2023-09-10T01:07:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.761+0000 7fa781df9b80 20 client.28499 trim_cache size 0 max 16384 2023-09-10T01:07:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.778+0000 7fa781df9b80 1 Processor -- start 2023-09-10T01:07:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.778+0000 7fa781df9b80 1 -- start start 2023-09-10T01:07:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.778+0000 7fa781df9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 0x5593f2671a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.778+0000 7fa781df9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f2672cc0 0x5593f267e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.778+0000 7fa781df9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f26781a0 0x5593f2674b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.778+0000 7fa781df9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f266ed10 con 0x5593f2672cc0 2023-09-10T01:07:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.778+0000 7fa781df9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2677db0 con 0x5593f26781a0 2023-09-10T01:07:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.778+0000 7fa781df9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f267a5b0 con 0x5593f2672300 2023-09-10T01:07:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.778+0000 7fa76effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f2672cc0 0x5593f267e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.778+0000 7fa76effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f2672cc0 0x5593f267e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55980/0 (socket says 172.21.15.44:55980) 2023-09-10T01:07:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.778+0000 7fa76effd700 1 -- 172.21.15.44:0/3618110773 learned_addr learned my addr 172.21.15.44:0/3618110773 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.778+0000 7fa76f7fe700 1 --2- 172.21.15.44:0/3618110773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 0x5593f2671a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.778+0000 7fa76ffff700 1 --2- 172.21.15.44:0/3618110773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f26781a0 0x5593f2674b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.779+0000 7fa76effd700 1 -- 172.21.15.44:0/3618110773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 msgr2=0x5593f2671a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.779+0000 7fa76effd700 1 --2- 172.21.15.44:0/3618110773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 0x5593f2671a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.779+0000 7fa76effd700 1 -- 172.21.15.44:0/3618110773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f26781a0 msgr2=0x5593f2674b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.779+0000 7fa76effd700 1 --2- 172.21.15.44:0/3618110773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f26781a0 0x5593f2674b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.779+0000 7fa76effd700 1 -- 172.21.15.44:0/3618110773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa764017070 con 0x5593f2672cc0 2023-09-10T01:07:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.779+0000 7fa76ffff700 1 --2- 172.21.15.44:0/3618110773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f26781a0 0x5593f2674b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:07:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.779+0000 7fa76effd700 1 --2- 172.21.15.44:0/3618110773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f2672cc0 0x5593f267e840 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa764016040 tx=0x7fa76400e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170c58e2fb0e33e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.779+0000 7fa7567fc700 1 -- 172.21.15.44:0/3618110773 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76401ccb0 con 0x5593f2672cc0 2023-09-10T01:07:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.779+0000 7fa7567fc700 1 -- 172.21.15.44:0/3618110773 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa76401c8a0 con 0x5593f2672cc0 2023-09-10T01:07:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.779+0000 7fa7567fc700 1 -- 172.21.15.44:0/3618110773 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764005610 con 0x5593f2672cc0 2023-09-10T01:07:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.780+0000 7fa781df9b80 1 -- 172.21.15.44:0/3618110773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f2672cc0 msgr2=0x5593f267e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.780+0000 7fa781df9b80 1 --2- 172.21.15.44:0/3618110773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f2672cc0 0x5593f267e840 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa764016040 tx=0x7fa76400e970 comp rx=0 tx=0).stop 2023-09-10T01:07:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.781+0000 7fa781df9b80 1 -- 172.21.15.44:0/3618110773 shutdown_connections 2023-09-10T01:07:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.781+0000 7fa781df9b80 1 --2- 172.21.15.44:0/3618110773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f26781a0 0x5593f2674b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.781+0000 7fa781df9b80 1 --2- 172.21.15.44:0/3618110773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f2672300 0x5593f2671a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.781+0000 7fa781df9b80 1 --2- 172.21.15.44:0/3618110773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f2672cc0 0x5593f267e840 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.781+0000 7fa781df9b80 1 -- 172.21.15.44:0/3618110773 >> 172.21.15.44:0/3618110773 conn(0x5593f25e7030 msgr2=0x5593f26712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.781+0000 7fa781df9b80 1 -- 172.21.15.44:0/3618110773 shutdown_connections 2023-09-10T01:07:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.781+0000 7fa781df9b80 1 -- 172.21.15.44:0/3618110773 wait complete. 2023-09-10T01:07:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.782+0000 7fa781df9b80 1 -- start start 2023-09-10T01:07:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.782+0000 7fa781df9b80 1 Processor -- start 2023-09-10T01:07:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.782+0000 7fa781df9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672300 0x5593f2683030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.782+0000 7fa781df9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f2672cc0 0x5593f2683570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.782+0000 7fa781df9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f26781a0 0x5593f2681510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.782+0000 7fa781df9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2681bf0 con 0x5593f2672cc0 2023-09-10T01:07:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.782+0000 7fa781df9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2681d60 con 0x5593f2672300 2023-09-10T01:07:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.782+0000 7fa781df9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2681f20 con 0x5593f26781a0 2023-09-10T01:07:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.782+0000 7fa781df9b80 10 client.0 _ll_register_callbacks cb 0x7fa760096650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:07:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.782+0000 7fa76effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f2672cc0 0x5593f2683570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.782+0000 7fa76f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672300 0x5593f2683030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.782+0000 7fa76effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f2672cc0 0x5593f2683570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55986/0 (socket says 172.21.15.44:55986) 2023-09-10T01:07:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.782+0000 7fa76effd700 1 -- 172.21.15.44:0/173406035 learned_addr learned my addr 172.21.15.44:0/173406035 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.782+0000 7fa76ffff700 1 --2- 172.21.15.44:0/173406035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f26781a0 0x5593f2681510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.783+0000 7fa76f7fe700 1 -- 172.21.15.44:0/173406035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f26781a0 msgr2=0x5593f2681510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.783+0000 7fa76f7fe700 1 --2- 172.21.15.44:0/173406035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f26781a0 0x5593f2681510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.783+0000 7fa76f7fe700 1 -- 172.21.15.44:0/173406035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f2672cc0 msgr2=0x5593f2683570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.783+0000 7fa76f7fe700 1 --2- 172.21.15.44:0/173406035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f2672cc0 0x5593f2683570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.783+0000 7fa76f7fe700 1 -- 172.21.15.44:0/173406035 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7701abd30 con 0x5593f2672300 2023-09-10T01:07:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.783+0000 7fa76f7fe700 1 --2- 172.21.15.44:0/173406035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672300 0x5593f2683030 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa7600ca520 tx=0x7fa760009470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dde880569ee51243 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.783+0000 7fa74effd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:07:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.783+0000 7fa74effd700 1 -- 172.21.15.44:0/173406035 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76009e140 con 0x5593f2672300 2023-09-10T01:07:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.783+0000 7fa74effd700 1 -- 172.21.15.44:0/173406035 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa7600a22f0 con 0x5593f2672300 2023-09-10T01:07:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.783+0000 7fa781df9b80 1 -- 172.21.15.44:0/173406035 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593f25bd8c0 con 0x5593f2672300 2023-09-10T01:07:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.783+0000 7fa74effd700 1 -- 172.21.15.44:0/173406035 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7600b81b0 con 0x5593f2672300 2023-09-10T01:07:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.783+0000 7fa781df9b80 10 client.28505 fetch_fsmap learned FSMap version 109 2023-09-10T01:07:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.783+0000 7fa781df9b80 1 -- 172.21.15.44:0/173406035 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=4+}) v3 -- 0x5593f23d9c00 con 0x5593f2672300 2023-09-10T01:07:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.783+0000 7fa74effd700 1 -- 172.21.15.44:0/173406035 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5593f25bd8c0 con 0x5593f2672300 2023-09-10T01:07:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.784+0000 7fa74effd700 1 -- 172.21.15.44:0/173406035 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 109) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa7600a00c0 con 0x5593f2672300 2023-09-10T01:07:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.784+0000 7fa781df9b80 10 client.28505 fetch_fsmap finished waiting for FSMap version 109 2023-09-10T01:07:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.784+0000 7fa781df9b80 10 client.28505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:07:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.784+0000 7fa781df9b80 20 client.28505 populate_metadata read hostname 'smithi044' 2023-09-10T01:07:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.784+0000 7fa781df9b80 1 --2- 172.21.15.44:0/173406035 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5593f26e0f60 0x5593f26e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.784+0000 7fa781df9b80 4 client.28505 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:07:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.784+0000 7fa781df9b80 1 -- 172.21.15.44:0/173406035 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5593f257e050 con 0x5593f26e0f60 2023-09-10T01:07:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.784+0000 7fa76effd700 1 --2- 172.21.15.44:0/173406035 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5593f26e0f60 0x5593f26e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.784+0000 7fa76effd700 1 --2- 172.21.15.44:0/173406035 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5593f26e0f60 0x5593f26e3380 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b81c26b5a01a0b server_cookie=9c114238f1e2bb1b in_seq=0 out_seq=0 2023-09-10T01:07:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.784+0000 7fa74effd700 10 client.28505 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.797+0000 7fa74effd700 1 -- 172.21.15.44:0/173406035 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5593f257e050 con 0x5593f26e0f60 2023-09-10T01:07:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.797+0000 7fa74effd700 10 client.28505 handle_command_reply: tid=0 2023-09-10T01:07:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.797+0000 7fa781df9b80 1 client.28505 shutdown 2023-09-10T01:07:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.798+0000 7fa781df9b80 1 -- 172.21.15.44:0/173406035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672300 msgr2=0x5593f2683030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.798+0000 7fa781df9b80 1 --2- 172.21.15.44:0/173406035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672300 0x5593f2683030 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa7600ca520 tx=0x7fa760009470 comp rx=0 tx=0).stop 2023-09-10T01:07:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.798+0000 7fa781df9b80 1 -- 172.21.15.44:0/173406035 shutdown_connections 2023-09-10T01:07:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.798+0000 7fa781df9b80 1 --2- 172.21.15.44:0/173406035 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5593f26e0f60 0x5593f26e3380 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.798+0000 7fa781df9b80 1 --2- 172.21.15.44:0/173406035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593f26781a0 0x5593f2681510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.798+0000 7fa781df9b80 1 --2- 172.21.15.44:0/173406035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593f2672300 0x5593f2683030 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.798+0000 7fa781df9b80 1 --2- 172.21.15.44:0/173406035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593f2672cc0 0x5593f2683570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.798+0000 7fa781df9b80 1 -- 172.21.15.44:0/173406035 >> 172.21.15.44:0/173406035 conn(0x5593f25e7030 msgr2=0x5593f2671b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.798+0000 7fa74effd700 0 client.28505 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.798+0000 7fa781df9b80 1 -- 172.21.15.44:0/173406035 shutdown_connections 2023-09-10T01:07:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.798+0000 7fa781df9b80 1 -- 172.21.15.44:0/173406035 wait complete. 2023-09-10T01:07:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.798+0000 7fa781df9b80 20 client.28505 trim_cache size 0 max 16384 2023-09-10T01:07:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:07:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:07:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:07:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:07:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.799+0000 7fa775c33700 1 -- 172.21.15.44:0/326141875 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa7580a0830 msgr2=0x7fa7580a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.799+0000 7fa775c33700 1 --2- 172.21.15.44:0/326141875 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa7580a0830 0x7fa7580a2cf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa770106780 tx=0x7fa764009260 comp rx=0 tx=0).stop 2023-09-10T01:07:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.799+0000 7fa775c33700 1 -- 172.21.15.44:0/326141875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7701017e0 msgr2=0x7fa770109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.799+0000 7fa775c33700 1 --2- 172.21.15.44:0/326141875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7701017e0 0x7fa770109fa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa760009230 tx=0x7fa76000fa00 comp rx=0 tx=0).stop 2023-09-10T01:07:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.799+0000 7fa775c33700 1 -- 172.21.15.44:0/326141875 shutdown_connections 2023-09-10T01:07:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.799+0000 7fa775c33700 1 --2- 172.21.15.44:0/326141875 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa7580a0830 0x7fa7580a2cf0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.799+0000 7fa775c33700 1 --2- 172.21.15.44:0/326141875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa77010abb0 0x7fa770104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.799+0000 7fa775c33700 1 --2- 172.21.15.44:0/326141875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7701017e0 0x7fa770109fa0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.799+0000 7fa775c33700 1 --2- 172.21.15.44:0/326141875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa770102100 0x7fa77010a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.799+0000 7fa775c33700 1 -- 172.21.15.44:0/326141875 >> 172.21.15.44:0/326141875 conn(0x7fa7700faf40 msgr2=0x7fa7700fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.799+0000 7fa775c33700 1 -- 172.21.15.44:0/326141875 shutdown_connections 2023-09-10T01:07:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:29.799+0000 7fa775c33700 1 -- 172.21.15.44:0/326141875 wait complete. 2023-09-10T01:07:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:30 smithi131 ceph-mon[103713]: pgmap v853: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2023-09-10T01:07:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:30 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:07:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:30 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:07:31.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:30 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:07:31.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:30 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:07:31.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:30 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:07:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:30 smithi044 ceph-mon[98676]: pgmap v853: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2023-09-10T01:07:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:30 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:07:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:30 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:07:31.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:30 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:07:31.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:30 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:07:31.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:30 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:07:31.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:30 smithi099 ceph-mon[102692]: pgmap v853: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2023-09-10T01:07:31.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:30 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:07:31.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:30 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:07:31.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:30 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:07:31.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:30 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:07:31.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:30 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:07:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:32 smithi131 ceph-mon[103713]: pgmap v854: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 242 op/s 2023-09-10T01:07:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:32 smithi044 ceph-mon[98676]: pgmap v854: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 242 op/s 2023-09-10T01:07:33.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:32 smithi099 ceph-mon[102692]: pgmap v854: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 242 op/s 2023-09-10T01:07:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:34 smithi131 ceph-mon[103713]: pgmap v855: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2023-09-10T01:07:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:34 smithi044 ceph-mon[98676]: pgmap v855: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2023-09-10T01:07:35.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:34 smithi099 ceph-mon[102692]: pgmap v855: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2023-09-10T01:07:37.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:36 smithi131 ceph-mon[103713]: pgmap v856: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 234 op/s 2023-09-10T01:07:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:36 smithi044 ceph-mon[98676]: pgmap v856: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 234 op/s 2023-09-10T01:07:37.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:36 smithi099 ceph-mon[102692]: pgmap v856: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 234 op/s 2023-09-10T01:07:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:38 smithi131 ceph-mon[103713]: pgmap v857: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:07:39.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:38 smithi044 ceph-mon[98676]: pgmap v857: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:07:39.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:38 smithi099 ceph-mon[102692]: pgmap v857: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:07:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:40 smithi131 ceph-mon[103713]: pgmap v858: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:07:41.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:40 smithi044 ceph-mon[98676]: pgmap v858: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:07:41.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:40 smithi099 ceph-mon[102692]: pgmap v858: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:07:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.917+0000 7f7733634700 1 -- 172.21.15.44:0/2568398384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c103fe0 msgr2=0x7f772c10bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.917+0000 7f7733634700 1 --2- 172.21.15.44:0/2568398384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c103fe0 0x7f772c10bb60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f772400c180 tx=0x7f772400c490 comp rx=0 tx=0).stop 2023-09-10T01:07:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.917+0000 7f7733634700 1 -- 172.21.15.44:0/2568398384 shutdown_connections 2023-09-10T01:07:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.917+0000 7f7733634700 1 --2- 172.21.15.44:0/2568398384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c103fe0 0x7f772c10bb60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.917+0000 7f7733634700 1 --2- 172.21.15.44:0/2568398384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f772c0ff080 0x7f772c0ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.917+0000 7f7733634700 1 --2- 172.21.15.44:0/2568398384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f772c0ffa30 0x7f772c1038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.917+0000 7f7733634700 1 -- 172.21.15.44:0/2568398384 >> 172.21.15.44:0/2568398384 conn(0x7f772c075920 msgr2=0x7f772c075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.917+0000 7f7733634700 1 -- 172.21.15.44:0/2568398384 shutdown_connections 2023-09-10T01:07:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.917+0000 7f7733634700 1 -- 172.21.15.44:0/2568398384 wait complete. 2023-09-10T01:07:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.918+0000 7f7733634700 1 Processor -- start 2023-09-10T01:07:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.918+0000 7f7733634700 1 -- start start 2023-09-10T01:07:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.918+0000 7f7733634700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f772c0ff080 0x7f772c19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.918+0000 7f7733634700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c103fe0 0x7f772c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.918+0000 7f7733634700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f772c196520 0x7f772c1969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.918+0000 7f7733634700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c196fb0 con 0x7f772c196520 2023-09-10T01:07:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.918+0000 7f7733634700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c197120 con 0x7f772c103fe0 2023-09-10T01:07:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.918+0000 7f7733634700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c1972e0 con 0x7f772c0ff080 2023-09-10T01:07:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.918+0000 7f7731bd1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f772c196520 0x7f772c1969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.918+0000 7f77313d0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f772c0ff080 0x7f772c19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.918+0000 7f7730bcf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c103fe0 0x7f772c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.918+0000 7f77313d0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f772c0ff080 0x7f772c19c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34648/0 (socket says 172.21.15.44:34648) 2023-09-10T01:07:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.918+0000 7f77313d0700 1 -- 172.21.15.44:0/2482199318 learned_addr learned my addr 172.21.15.44:0/2482199318 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.918+0000 7f7731bd1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f772c196520 0x7f772c1969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56002/0 (socket says 172.21.15.44:56002) 2023-09-10T01:07:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.919+0000 7f77313d0700 1 -- 172.21.15.44:0/2482199318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c103fe0 msgr2=0x7f772c19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.919+0000 7f77313d0700 1 --2- 172.21.15.44:0/2482199318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c103fe0 0x7f772c19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.919+0000 7f77313d0700 1 -- 172.21.15.44:0/2482199318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f772c196520 msgr2=0x7f772c1969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.919+0000 7f77313d0700 1 --2- 172.21.15.44:0/2482199318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f772c196520 0x7f772c1969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.919+0000 7f77313d0700 1 -- 172.21.15.44:0/2482199318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77280095f0 con 0x7f772c0ff080 2023-09-10T01:07:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.919+0000 7f7731bd1700 1 --2- 172.21.15.44:0/2482199318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f772c196520 0x7f772c1969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.919+0000 7f77313d0700 1 --2- 172.21.15.44:0/2482199318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f772c0ff080 0x7f772c19c6a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f772800ec10 tx=0x7f772800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.919+0000 7f77227fc700 1 -- 172.21.15.44:0/2482199318 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772800cc90 con 0x7f772c0ff080 2023-09-10T01:07:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.919+0000 7f77227fc700 1 -- 172.21.15.44:0/2482199318 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f772800cdf0 con 0x7f772c0ff080 2023-09-10T01:07:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.919+0000 7f7733634700 1 -- 172.21.15.44:0/2482199318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772400ad50 con 0x7f772c0ff080 2023-09-10T01:07:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.919+0000 7f77227fc700 1 -- 172.21.15.44:0/2482199318 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728003e80 con 0x7f772c0ff080 2023-09-10T01:07:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.919+0000 7f7733634700 1 -- 172.21.15.44:0/2482199318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772c1a38d0 con 0x7f772c0ff080 2023-09-10T01:07:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.922+0000 7f77227fc700 1 -- 172.21.15.44:0/2482199318 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f7728004000 con 0x7f772c0ff080 2023-09-10T01:07:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.923+0000 7f77227fc700 1 --2- 172.21.15.44:0/2482199318 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f77180a07b0 0x7f77180a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.923+0000 7f7730bcf700 1 --2- 172.21.15.44:0/2482199318 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f77180a07b0 0x7f77180a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.923+0000 7f77227fc700 1 -- 172.21.15.44:0/2482199318 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7728014070 con 0x7f772c0ff080 2023-09-10T01:07:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.923+0000 7f773f7fab80 1 Processor -- start 2023-09-10T01:07:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.923+0000 7f773f7fab80 1 -- start start 2023-09-10T01:07:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.923+0000 7f7730bcf700 1 --2- 172.21.15.44:0/2482199318 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f77180a07b0 0x7f77180a2c70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f772c197f20 tx=0x7f771c009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f773f7fab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eb21a0 0x55a3c8eb2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f773f7fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eac300 0x55a3c8eac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f773f7fab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eaccc0 0x55a3c8eb8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f773f7fab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c8e056d0 con 0x55a3c8eb21a0 2023-09-10T01:07:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f773f7fab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c8e02ff0 con 0x55a3c8eaccc0 2023-09-10T01:07:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f773f7fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c8dfe8e0 con 0x55a3c8eac300 2023-09-10T01:07:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f77313d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eb21a0 0x55a3c8eb2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f77313d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eb21a0 0x55a3c8eb2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56006/0 (socket says 172.21.15.44:56006) 2023-09-10T01:07:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f77313d0700 1 -- 172.21.15.44:0/2322385612 learned_addr learned my addr 172.21.15.44:0/2322385612 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f7730bcf700 1 --2- 172.21.15.44:0/2322385612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eac300 0x55a3c8eac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f7731bd1700 1 --2- 172.21.15.44:0/2322385612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eaccc0 0x55a3c8eb8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f77313d0700 1 -- 172.21.15.44:0/2322385612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eac300 msgr2=0x55a3c8eac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f77313d0700 1 --2- 172.21.15.44:0/2322385612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eac300 0x55a3c8eac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f77313d0700 1 -- 172.21.15.44:0/2322385612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eaccc0 msgr2=0x55a3c8eb8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f77313d0700 1 --2- 172.21.15.44:0/2322385612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eaccc0 0x55a3c8eb8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f77313d0700 1 -- 172.21.15.44:0/2322385612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3c8c2ad80 con 0x55a3c8eb21a0 2023-09-10T01:07:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f7731bd1700 1 --2- 172.21.15.44:0/2322385612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eaccc0 0x55a3c8eb8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:07:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f77313d0700 1 --2- 172.21.15.44:0/2322385612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eb21a0 0x55a3c8eb2580 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f77280a13e0 tx=0x7f77280997a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c10f3aa099834a5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f77177fe700 1 -- 172.21.15.44:0/2322385612 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77280b1ea0 con 0x55a3c8eb21a0 2023-09-10T01:07:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f77177fe700 1 -- 172.21.15.44:0/2322385612 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7728099fa0 con 0x55a3c8eb21a0 2023-09-10T01:07:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.924+0000 7f77177fe700 1 -- 172.21.15.44:0/2322385612 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728097d10 con 0x55a3c8eb21a0 2023-09-10T01:07:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.925+0000 7f773f7fab80 1 -- 172.21.15.44:0/2322385612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eb21a0 msgr2=0x55a3c8eb2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.925+0000 7f773f7fab80 1 --2- 172.21.15.44:0/2322385612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eb21a0 0x55a3c8eb2580 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f77280a13e0 tx=0x7f77280997a0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.925+0000 7f773f7fab80 1 -- 172.21.15.44:0/2322385612 shutdown_connections 2023-09-10T01:07:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.925+0000 7f773f7fab80 1 --2- 172.21.15.44:0/2322385612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eaccc0 0x55a3c8eb8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.925+0000 7f773f7fab80 1 --2- 172.21.15.44:0/2322385612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eb21a0 0x55a3c8eb2580 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.925+0000 7f773f7fab80 1 --2- 172.21.15.44:0/2322385612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eac300 0x55a3c8eac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.925+0000 7f773f7fab80 1 -- 172.21.15.44:0/2322385612 >> 172.21.15.44:0/2322385612 conn(0x55a3c8e21030 msgr2=0x55a3c8ea7560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.925+0000 7f773f7fab80 1 -- 172.21.15.44:0/2322385612 shutdown_connections 2023-09-10T01:07:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.925+0000 7f773f7fab80 1 -- 172.21.15.44:0/2322385612 wait complete. 2023-09-10T01:07:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.925+0000 7f773f7fab80 1 -- start start 2023-09-10T01:07:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.926+0000 7f773f7fab80 1 Processor -- start 2023-09-10T01:07:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.926+0000 7f773f7fab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 0x55a3c8eb8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.926+0000 7f773f7fab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8eb60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.926+0000 7f773f7fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 0x55a3c8eb66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.926+0000 7f773f7fab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c8d9cb10 con 0x55a3c8eaccc0 2023-09-10T01:07:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.926+0000 7f773f7fab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c8d9c810 con 0x55a3c8eac300 2023-09-10T01:07:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.926+0000 7f773f7fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c8c808c0 con 0x55a3c8eb21a0 2023-09-10T01:07:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.926+0000 7f773f7fab80 10 client.0 _ll_register_callbacks cb 0x55a3c8ce6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:07:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.926+0000 7f77313d0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 0x55a3c8eb8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.926+0000 7f7730bcf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8eb60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.926+0000 7f77313d0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 0x55a3c8eb8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57752/0 (socket says 172.21.15.44:57752) 2023-09-10T01:07:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.926+0000 7f7731bd1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 0x55a3c8eb66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.926+0000 7f77313d0700 1 -- 172.21.15.44:0/2277011713 learned_addr learned my addr 172.21.15.44:0/2277011713 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.927+0000 7f7730bcf700 1 -- 172.21.15.44:0/2277011713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 msgr2=0x55a3c8eb66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.927+0000 7f7730bcf700 1 --2- 172.21.15.44:0/2277011713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 0x55a3c8eb66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.927+0000 7f7730bcf700 1 -- 172.21.15.44:0/2277011713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 msgr2=0x55a3c8eb8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.927+0000 7f7730bcf700 1 --2- 172.21.15.44:0/2277011713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 0x55a3c8eb8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.927+0000 7f7730bcf700 1 -- 172.21.15.44:0/2277011713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772c1a38d0 con 0x55a3c8eaccc0 2023-09-10T01:07:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.927+0000 7f7731bd1700 1 --2- 172.21.15.44:0/2277011713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 0x55a3c8eb66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.927+0000 7f77313d0700 1 --2- 172.21.15.44:0/2277011713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 0x55a3c8eb8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.927+0000 7f7730bcf700 1 --2- 172.21.15.44:0/2277011713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8eb60f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x55a3c8ea7060 tx=0x7f771c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a3808ca00cc0aa server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.927+0000 7f7714ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:07:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.927+0000 7f7714ff9700 1 -- 172.21.15.44:0/2277011713 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c01d640 con 0x55a3c8eaccc0 2023-09-10T01:07:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.927+0000 7f7714ff9700 1 -- 172.21.15.44:0/2277011713 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f771c01d7a0 con 0x55a3c8eaccc0 2023-09-10T01:07:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.927+0000 7f773f7fab80 1 -- 172.21.15.44:0/2277011713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3c8d56490 con 0x55a3c8eaccc0 2023-09-10T01:07:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.927+0000 7f7714ff9700 1 -- 172.21.15.44:0/2277011713 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f771c01d960 con 0x55a3c8eaccc0 2023-09-10T01:07:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.927+0000 7f773f7fab80 10 client.18867 fetch_fsmap learned FSMap version 109 2023-09-10T01:07:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.927+0000 7f773f7fab80 1 -- 172.21.15.44:0/2277011713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=4+}) v3 -- 0x55a3c8c13c00 con 0x55a3c8eaccc0 2023-09-10T01:07:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.927+0000 7f7714ff9700 1 -- 172.21.15.44:0/2277011713 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f771c031080 con 0x55a3c8eaccc0 2023-09-10T01:07:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.928+0000 7f7714ff9700 1 -- 172.21.15.44:0/2277011713 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 109) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f771c01aba0 con 0x55a3c8eaccc0 2023-09-10T01:07:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.928+0000 7f773f7fab80 10 client.18867 fetch_fsmap finished waiting for FSMap version 109 2023-09-10T01:07:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.928+0000 7f773f7fab80 10 client.18867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:07:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.928+0000 7f773f7fab80 20 client.18867 populate_metadata read hostname 'smithi044' 2023-09-10T01:07:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.928+0000 7f773f7fab80 1 --2- 172.21.15.44:0/2277011713 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a3c8ec6430 0x55a3c8ec8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.928+0000 7f773f7fab80 4 client.18867 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:07:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.928+0000 7f773f7fab80 1 -- 172.21.15.44:0/2277011713 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3c8e13840 con 0x55a3c8ec6430 2023-09-10T01:07:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.928+0000 7f77313d0700 1 --2- 172.21.15.44:0/2277011713 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a3c8ec6430 0x55a3c8ec8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.928+0000 7f7714ff9700 10 client.18867 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.928+0000 7f77313d0700 1 --2- 172.21.15.44:0/2277011713 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a3c8ec6430 0x55a3c8ec8850 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bf49de9a1857681 server_cookie=624e30606d5cc8d4 in_seq=0 out_seq=0 2023-09-10T01:07:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.929+0000 7f7714ff9700 1 -- 172.21.15.44:0/2277011713 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a3c8e13840 con 0x55a3c8ec6430 2023-09-10T01:07:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.929+0000 7f7714ff9700 10 client.18867 handle_command_reply: tid=0 2023-09-10T01:07:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.929+0000 7f773f7fab80 1 client.18867 shutdown 2023-09-10T01:07:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.929+0000 7f773f7fab80 1 -- 172.21.15.44:0/2277011713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 msgr2=0x55a3c8eb60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.929+0000 7f773f7fab80 1 --2- 172.21.15.44:0/2277011713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8eb60f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x55a3c8ea7060 tx=0x7f771c0047c0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.929+0000 7f773f7fab80 1 -- 172.21.15.44:0/2277011713 shutdown_connections 2023-09-10T01:07:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.929+0000 7f773f7fab80 1 --2- 172.21.15.44:0/2277011713 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a3c8ec6430 0x55a3c8ec8850 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.929+0000 7f773f7fab80 1 --2- 172.21.15.44:0/2277011713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 0x55a3c8eb66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.929+0000 7f773f7fab80 1 --2- 172.21.15.44:0/2277011713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 0x55a3c8eb8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.929+0000 7f773f7fab80 1 --2- 172.21.15.44:0/2277011713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8eb60f0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.929+0000 7f7714ff9700 0 client.18867 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.929+0000 7f773f7fab80 1 -- 172.21.15.44:0/2277011713 >> 172.21.15.44:0/2277011713 conn(0x55a3c8e21030 msgr2=0x55a3c8ea67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.929+0000 7f773f7fab80 1 -- 172.21.15.44:0/2277011713 shutdown_connections 2023-09-10T01:07:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.929+0000 7f773f7fab80 1 -- 172.21.15.44:0/2277011713 wait complete. 2023-09-10T01:07:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.929+0000 7f773f7fab80 20 client.18867 trim_cache size 0 max 16384 2023-09-10T01:07:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.944+0000 7f773f7fab80 1 Processor -- start 2023-09-10T01:07:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.944+0000 7f773f7fab80 1 -- start start 2023-09-10T01:07:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.944+0000 7f773f7fab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 0x55a3c8eb8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.944+0000 7f773f7fab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8eb58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.944+0000 7f773f7fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 0x55a3c8eb5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.944+0000 7f773f7fab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c8ea3fc0 con 0x55a3c8eaccc0 2023-09-10T01:07:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.944+0000 7f773f7fab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c8ead580 con 0x55a3c8eac300 2023-09-10T01:07:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.944+0000 7f773f7fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c8eb1e30 con 0x55a3c8eb21a0 2023-09-10T01:07:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.945+0000 7f7730bcf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8eb58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.945+0000 7f7730bcf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8eb58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56022/0 (socket says 172.21.15.44:56022) 2023-09-10T01:07:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.945+0000 7f7730bcf700 1 -- 172.21.15.44:0/1619186087 learned_addr learned my addr 172.21.15.44:0/1619186087 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.945+0000 7f77313d0700 1 --2- 172.21.15.44:0/1619186087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 0x55a3c8eb8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.945+0000 7f7731bd1700 1 --2- 172.21.15.44:0/1619186087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 0x55a3c8eb5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.945+0000 7f7730bcf700 1 -- 172.21.15.44:0/1619186087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 msgr2=0x55a3c8eb5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.945+0000 7f7730bcf700 1 --2- 172.21.15.44:0/1619186087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 0x55a3c8eb5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.945+0000 7f7730bcf700 1 -- 172.21.15.44:0/1619186087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 msgr2=0x55a3c8eb8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.945+0000 7f7730bcf700 1 --2- 172.21.15.44:0/1619186087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 0x55a3c8eb8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.945+0000 7f7730bcf700 1 -- 172.21.15.44:0/1619186087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3c8c511f0 con 0x55a3c8eaccc0 2023-09-10T01:07:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.945+0000 7f7731bd1700 1 --2- 172.21.15.44:0/1619186087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 0x55a3c8eb5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:07:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.945+0000 7f7730bcf700 1 --2- 172.21.15.44:0/1619186087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8eb58d0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f771c013010 tx=0x7f771c009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714186a244d7cee4 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.945+0000 7f7717fff700 1 -- 172.21.15.44:0/1619186087 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c01fea0 con 0x55a3c8eaccc0 2023-09-10T01:07:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.945+0000 7f7717fff700 1 -- 172.21.15.44:0/1619186087 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a3c8d56490 con 0x55a3c8eaccc0 2023-09-10T01:07:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.945+0000 7f7717fff700 1 -- 172.21.15.44:0/1619186087 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c01da40 con 0x55a3c8eaccc0 2023-09-10T01:07:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.946+0000 7f773f7fab80 1 -- 172.21.15.44:0/1619186087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 msgr2=0x55a3c8eb58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.946+0000 7f773f7fab80 1 --2- 172.21.15.44:0/1619186087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8eb58d0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f771c013010 tx=0x7f771c009370 comp rx=0 tx=0).stop 2023-09-10T01:07:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.946+0000 7f773f7fab80 1 -- 172.21.15.44:0/1619186087 shutdown_connections 2023-09-10T01:07:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.946+0000 7f773f7fab80 1 --2- 172.21.15.44:0/1619186087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 0x55a3c8eb5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.946+0000 7f773f7fab80 1 --2- 172.21.15.44:0/1619186087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 0x55a3c8eb8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.946+0000 7f773f7fab80 1 --2- 172.21.15.44:0/1619186087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8eb58d0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.946+0000 7f773f7fab80 1 -- 172.21.15.44:0/1619186087 >> 172.21.15.44:0/1619186087 conn(0x55a3c8e21030 msgr2=0x55a3c8ea7090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.946+0000 7f773f7fab80 1 -- 172.21.15.44:0/1619186087 shutdown_connections 2023-09-10T01:07:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.946+0000 7f773f7fab80 1 -- 172.21.15.44:0/1619186087 wait complete. 2023-09-10T01:07:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.946+0000 7f773f7fab80 1 -- start start 2023-09-10T01:07:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.946+0000 7f773f7fab80 1 Processor -- start 2023-09-10T01:07:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f773f7fab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 0x55a3c8ebd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f773f7fab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8ebd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f773f7fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 0x55a3c8ebb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f773f7fab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c8ebbde0 con 0x55a3c8eaccc0 2023-09-10T01:07:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f773f7fab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c8ebbf50 con 0x55a3c8eac300 2023-09-10T01:07:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f773f7fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c8ebc110 con 0x55a3c8eb21a0 2023-09-10T01:07:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f773f7fab80 10 client.0 _ll_register_callbacks cb 0x7f771c01f4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:07:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f7730bcf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8ebd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f77313d0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 0x55a3c8ebd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f7730bcf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8ebd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56028/0 (socket says 172.21.15.44:56028) 2023-09-10T01:07:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f7731bd1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 0x55a3c8ebb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f7730bcf700 1 -- 172.21.15.44:0/870368424 learned_addr learned my addr 172.21.15.44:0/870368424 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f7730bcf700 1 -- 172.21.15.44:0/870368424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 msgr2=0x55a3c8ebb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f7730bcf700 1 --2- 172.21.15.44:0/870368424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 0x55a3c8ebb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f7730bcf700 1 -- 172.21.15.44:0/870368424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 msgr2=0x55a3c8ebd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f7730bcf700 1 --2- 172.21.15.44:0/870368424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 0x55a3c8ebd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f7730bcf700 1 -- 172.21.15.44:0/870368424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3c8c13c00 con 0x55a3c8eaccc0 2023-09-10T01:07:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f7730bcf700 1 --2- 172.21.15.44:0/870368424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8ebd6b0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f771c016040 tx=0x7f771c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48418fc86fbd8f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.947+0000 7f7714ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:07:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.948+0000 7f7714ff9700 1 -- 172.21.15.44:0/870368424 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c01fea0 con 0x55a3c8eaccc0 2023-09-10T01:07:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.948+0000 7f7714ff9700 1 -- 172.21.15.44:0/870368424 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f771c004bd0 con 0x55a3c8eaccc0 2023-09-10T01:07:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.948+0000 7f773f7fab80 1 -- 172.21.15.44:0/870368424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7728099fa0 con 0x55a3c8eaccc0 2023-09-10T01:07:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.948+0000 7f7714ff9700 1 -- 172.21.15.44:0/870368424 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f771c0090c0 con 0x55a3c8eaccc0 2023-09-10T01:07:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.948+0000 7f773f7fab80 10 client.18873 fetch_fsmap learned FSMap version 109 2023-09-10T01:07:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.948+0000 7f773f7fab80 1 -- 172.21.15.44:0/870368424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=4+}) v3 -- 0x55a3c8c29250 con 0x55a3c8eaccc0 2023-09-10T01:07:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.948+0000 7f7714ff9700 1 -- 172.21.15.44:0/870368424 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7728099fa0 con 0x55a3c8eaccc0 2023-09-10T01:07:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.948+0000 7f7714ff9700 1 -- 172.21.15.44:0/870368424 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 109) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f771c048070 con 0x55a3c8eaccc0 2023-09-10T01:07:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.948+0000 7f773f7fab80 10 client.18873 fetch_fsmap finished waiting for FSMap version 109 2023-09-10T01:07:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.948+0000 7f773f7fab80 10 client.18873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:07:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.948+0000 7f773f7fab80 20 client.18873 populate_metadata read hostname 'smithi044' 2023-09-10T01:07:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.948+0000 7f773f7fab80 1 --2- 172.21.15.44:0/870368424 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a3c8f1b170 0x55a3c8f1d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.948+0000 7f773f7fab80 4 client.18873 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:07:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.948+0000 7f773f7fab80 1 -- 172.21.15.44:0/870368424 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a3c8df78c0 con 0x55a3c8f1b170 2023-09-10T01:07:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.948+0000 7f77313d0700 1 --2- 172.21.15.44:0/870368424 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a3c8f1b170 0x55a3c8f1d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.949+0000 7f7714ff9700 10 client.18873 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.949+0000 7f77313d0700 1 --2- 172.21.15.44:0/870368424 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a3c8f1b170 0x55a3c8f1d590 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126b9ac2634f28d server_cookie=ed7a664ac7bd76c3 in_seq=0 out_seq=0 2023-09-10T01:07:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.962+0000 7f7714ff9700 1 -- 172.21.15.44:0/870368424 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a3c8df78c0 con 0x55a3c8f1b170 2023-09-10T01:07:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.962+0000 7f7714ff9700 10 client.18873 handle_command_reply: tid=0 2023-09-10T01:07:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.962+0000 7f773f7fab80 1 client.18873 shutdown 2023-09-10T01:07:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.962+0000 7f773f7fab80 1 -- 172.21.15.44:0/870368424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 msgr2=0x55a3c8ebd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.962+0000 7f773f7fab80 1 --2- 172.21.15.44:0/870368424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8ebd6b0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f771c016040 tx=0x7f771c0056a0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.962+0000 7f773f7fab80 1 -- 172.21.15.44:0/870368424 shutdown_connections 2023-09-10T01:07:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.962+0000 7f773f7fab80 1 --2- 172.21.15.44:0/870368424 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a3c8f1b170 0x55a3c8f1d590 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.962+0000 7f773f7fab80 1 --2- 172.21.15.44:0/870368424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3c8eb21a0 0x55a3c8ebb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.962+0000 7f7714ff9700 0 client.18873 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.962+0000 7f773f7fab80 1 --2- 172.21.15.44:0/870368424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3c8eac300 0x55a3c8ebd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.962+0000 7f773f7fab80 1 --2- 172.21.15.44:0/870368424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a3c8eaccc0 0x55a3c8ebd6b0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.962+0000 7f773f7fab80 1 -- 172.21.15.44:0/870368424 >> 172.21.15.44:0/870368424 conn(0x55a3c8e21030 msgr2=0x55a3c8eabb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.962+0000 7f773f7fab80 1 -- 172.21.15.44:0/870368424 shutdown_connections 2023-09-10T01:07:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.962+0000 7f773f7fab80 1 -- 172.21.15.44:0/870368424 wait complete. 2023-09-10T01:07:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.962+0000 7f773f7fab80 20 client.18873 trim_cache size 0 max 16384 2023-09-10T01:07:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:07:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:07:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:07:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:07:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.963+0000 7f7733634700 1 -- 172.21.15.44:0/2482199318 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f77180a07b0 msgr2=0x7f77180a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.963+0000 7f7733634700 1 --2- 172.21.15.44:0/2482199318 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f77180a07b0 0x7f77180a2c70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f772c197f20 tx=0x7f771c009260 comp rx=0 tx=0).stop 2023-09-10T01:07:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.963+0000 7f7733634700 1 -- 172.21.15.44:0/2482199318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f772c0ff080 msgr2=0x7f772c19c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.963+0000 7f7733634700 1 --2- 172.21.15.44:0/2482199318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f772c0ff080 0x7f772c19c6a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f772800ec10 tx=0x7f772800ef20 comp rx=0 tx=0).stop 2023-09-10T01:07:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.963+0000 7f7733634700 1 -- 172.21.15.44:0/2482199318 shutdown_connections 2023-09-10T01:07:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.963+0000 7f7733634700 1 --2- 172.21.15.44:0/2482199318 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f77180a07b0 0x7f77180a2c70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.963+0000 7f7733634700 1 --2- 172.21.15.44:0/2482199318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f772c196520 0x7f772c1969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.963+0000 7f7733634700 1 --2- 172.21.15.44:0/2482199318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f772c0ff080 0x7f772c19c6a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.963+0000 7f7733634700 1 --2- 172.21.15.44:0/2482199318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c103fe0 0x7f772c19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.963+0000 7f7733634700 1 -- 172.21.15.44:0/2482199318 >> 172.21.15.44:0/2482199318 conn(0x7f772c075920 msgr2=0x7f772c102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.963+0000 7f7733634700 1 -- 172.21.15.44:0/2482199318 shutdown_connections 2023-09-10T01:07:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:42.963+0000 7f7733634700 1 -- 172.21.15.44:0/2482199318 wait complete. 2023-09-10T01:07:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:42 smithi131 ceph-mon[103713]: pgmap v859: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 238 op/s 2023-09-10T01:07:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:42 smithi044 ceph-mon[98676]: pgmap v859: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 238 op/s 2023-09-10T01:07:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:42 smithi099 ceph-mon[102692]: pgmap v859: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 238 op/s 2023-09-10T01:07:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:44 smithi131 ceph-mon[103713]: pgmap v860: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 148 op/s 2023-09-10T01:07:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:44 smithi044 ceph-mon[98676]: pgmap v860: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 148 op/s 2023-09-10T01:07:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:44 smithi099 ceph-mon[102692]: pgmap v860: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 148 op/s 2023-09-10T01:07:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:46 smithi131 ceph-mon[103713]: pgmap v861: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 220 op/s 2023-09-10T01:07:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:46 smithi044 ceph-mon[98676]: pgmap v861: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 220 op/s 2023-09-10T01:07:47.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:46 smithi099 ceph-mon[102692]: pgmap v861: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 220 op/s 2023-09-10T01:07:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:48 smithi131 ceph-mon[103713]: pgmap v862: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2023-09-10T01:07:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:48 smithi044 ceph-mon[98676]: pgmap v862: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2023-09-10T01:07:49.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:48 smithi099 ceph-mon[102692]: pgmap v862: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2023-09-10T01:07:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:50 smithi131 ceph-mon[103713]: pgmap v863: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2023-09-10T01:07:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:50 smithi044 ceph-mon[98676]: pgmap v863: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2023-09-10T01:07:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:50 smithi099 ceph-mon[102692]: pgmap v863: 129 pgs: 129 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2023-09-10T01:07:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:52 smithi131 ceph-mon[103713]: pgmap v864: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 239 op/s 2023-09-10T01:07:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:52 smithi044 ceph-mon[98676]: pgmap v864: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 239 op/s 2023-09-10T01:07:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:52 smithi099 ceph-mon[102692]: pgmap v864: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 239 op/s 2023-09-10T01:07:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:54 smithi131 ceph-mon[103713]: pgmap v865: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:07:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:54 smithi044 ceph-mon[98676]: pgmap v865: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:07:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:54 smithi099 ceph-mon[102692]: pgmap v865: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:07:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.102+0000 7f443ef24700 1 -- 172.21.15.44:0/1691375399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4438106680 msgr2=0x7f4438106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.102+0000 7f443ef24700 1 --2- 172.21.15.44:0/1691375399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4438106680 0x7f4438106a60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f442c0098f0 tx=0x7f442c009c00 comp rx=0 tx=0).stop 2023-09-10T01:07:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.102+0000 7f443ef24700 1 -- 172.21.15.44:0/1691375399 shutdown_connections 2023-09-10T01:07:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.102+0000 7f443ef24700 1 --2- 172.21.15.44:0/1691375399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4438068e70 0x7f443810d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.102+0000 7f443ef24700 1 --2- 172.21.15.44:0/1691375399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4438106680 0x7f4438106a60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.102+0000 7f443ef24700 1 --2- 172.21.15.44:0/1691375399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44380684b0 0x7f4438068930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.102+0000 7f443ef24700 1 -- 172.21.15.44:0/1691375399 >> 172.21.15.44:0/1691375399 conn(0x7f4438077430 msgr2=0x7f4438077840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.102+0000 7f443ef24700 1 -- 172.21.15.44:0/1691375399 shutdown_connections 2023-09-10T01:07:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.102+0000 7f443ef24700 1 -- 172.21.15.44:0/1691375399 wait complete. 2023-09-10T01:07:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.103+0000 7f443ef24700 1 Processor -- start 2023-09-10T01:07:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.103+0000 7f443ef24700 1 -- start start 2023-09-10T01:07:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.103+0000 7f443ef24700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44380684b0 0x7f443819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.103+0000 7f443ef24700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4438068e70 0x7f443819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.103+0000 7f443ef24700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4438106680 0x7f4438196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.103+0000 7f443ef24700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4438196af0 con 0x7f4438068e70 2023-09-10T01:07:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.103+0000 7f443ef24700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4438196c60 con 0x7f4438106680 2023-09-10T01:07:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.103+0000 7f443ef24700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4438196e20 con 0x7f44380684b0 2023-09-10T01:07:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.104+0000 7f443ccc0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44380684b0 0x7f443819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.104+0000 7f443ccc0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44380684b0 0x7f443819c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34690/0 (socket says 172.21.15.44:34690) 2023-09-10T01:07:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.104+0000 7f443ccc0700 1 -- 172.21.15.44:0/3871778247 learned_addr learned my addr 172.21.15.44:0/3871778247 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.104+0000 7f4437fff700 1 --2- 172.21.15.44:0/3871778247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4438068e70 0x7f443819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.104+0000 7f443d4c1700 1 --2- 172.21.15.44:0/3871778247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4438106680 0x7f4438196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.104+0000 7f443ccc0700 1 -- 172.21.15.44:0/3871778247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4438106680 msgr2=0x7f4438196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.104+0000 7f443ccc0700 1 --2- 172.21.15.44:0/3871778247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4438106680 0x7f4438196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.104+0000 7f443ccc0700 1 -- 172.21.15.44:0/3871778247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4438068e70 msgr2=0x7f443819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.104+0000 7f443ccc0700 1 --2- 172.21.15.44:0/3871778247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4438068e70 0x7f443819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.104+0000 7f443ccc0700 1 -- 172.21.15.44:0/3871778247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4430009d00 con 0x7f44380684b0 2023-09-10T01:07:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.104+0000 7f443ccc0700 1 --2- 172.21.15.44:0/3871778247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44380684b0 0x7f443819c620 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f442c009810 tx=0x7f442c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.104+0000 7f4435ffb700 1 -- 172.21.15.44:0/3871778247 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c01c070 con 0x7f44380684b0 2023-09-10T01:07:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.105+0000 7f4435ffb700 1 -- 172.21.15.44:0/3871778247 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f442c00baf0 con 0x7f44380684b0 2023-09-10T01:07:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.105+0000 7f4435ffb700 1 -- 172.21.15.44:0/3871778247 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c004930 con 0x7f44380684b0 2023-09-10T01:07:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.105+0000 7f443ef24700 1 -- 172.21.15.44:0/3871778247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442c009520 con 0x7f44380684b0 2023-09-10T01:07:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.105+0000 7f443ef24700 1 -- 172.21.15.44:0/3871778247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44381972e0 con 0x7f44380684b0 2023-09-10T01:07:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.108+0000 7f4435ffb700 1 -- 172.21.15.44:0/3871778247 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f442c020070 con 0x7f44380684b0 2023-09-10T01:07:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.109+0000 7f4435ffb700 1 --2- 172.21.15.44:0/3871778247 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44280a0830 0x7f44280a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.109+0000 7f4435ffb700 1 -- 172.21.15.44:0/3871778247 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f442c0ccfd0 con 0x7f44380684b0 2023-09-10T01:07:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.109+0000 7f4437fff700 1 --2- 172.21.15.44:0/3871778247 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44280a0830 0x7f44280a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.110+0000 7f444b0eab80 1 Processor -- start 2023-09-10T01:07:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.114+0000 7f4437fff700 1 --2- 172.21.15.44:0/3871778247 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44280a0830 0x7f44280a2cf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f44380779f0 tx=0x7f4424009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.115+0000 7f444b0eab80 1 -- start start 2023-09-10T01:07:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.115+0000 7f444b0eab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c51387990 0x560c51387d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.115+0000 7f444b0eab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c5137a300 0x560c5137a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.115+0000 7f444b0eab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c5137acc0 0x560c5138f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.115+0000 7f444b0eab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c512d36d0 con 0x560c5137a300 2023-09-10T01:07:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.115+0000 7f444b0eab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c512d0ff0 con 0x560c5137acc0 2023-09-10T01:07:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.115+0000 7f444b0eab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c512cc8e0 con 0x560c51387990 2023-09-10T01:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.115+0000 7f4437fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c5137a300 0x560c5137a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.115+0000 7f443ccc0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c51387990 0x560c51387d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.115+0000 7f443ccc0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c51387990 0x560c51387d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34698/0 (socket says 172.21.15.44:34698) 2023-09-10T01:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.115+0000 7f443ccc0700 1 -- 172.21.15.44:0/2241159302 learned_addr learned my addr 172.21.15.44:0/2241159302 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.115+0000 7f4437fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c5137a300 0x560c5137a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56050/0 (socket says 172.21.15.44:56050) 2023-09-10T01:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.116+0000 7f443ccc0700 1 -- 172.21.15.44:0/2241159302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c5137acc0 msgr2=0x560c5138f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.116+0000 7f443ccc0700 1 --2- 172.21.15.44:0/2241159302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c5137acc0 0x560c5138f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.116+0000 7f443ccc0700 1 -- 172.21.15.44:0/2241159302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c5137a300 msgr2=0x560c5137a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.116+0000 7f443ccc0700 1 --2- 172.21.15.44:0/2241159302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c5137a300 0x560c5137a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.116+0000 7f443ccc0700 1 -- 172.21.15.44:0/2241159302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c510f8d80 con 0x560c51387990 2023-09-10T01:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.116+0000 7f4437fff700 1 --2- 172.21.15.44:0/2241159302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c5137a300 0x560c5137a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.116+0000 7f443ccc0700 1 --2- 172.21.15.44:0/2241159302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c51387990 0x560c51387d70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f442c096be0 tx=0x7f442c00bc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a32f8e5301965192 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.116+0000 7f4422ffd700 1 -- 172.21.15.44:0/2241159302 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c0a00b0 con 0x560c51387990 2023-09-10T01:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.116+0000 7f4422ffd700 1 -- 172.21.15.44:0/2241159302 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f442c09a870 con 0x560c51387990 2023-09-10T01:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.116+0000 7f4422ffd700 1 -- 172.21.15.44:0/2241159302 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c0ca760 con 0x560c51387990 2023-09-10T01:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.117+0000 7f444b0eab80 1 -- 172.21.15.44:0/2241159302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c51387990 msgr2=0x560c51387d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.117+0000 7f444b0eab80 1 --2- 172.21.15.44:0/2241159302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c51387990 0x560c51387d70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f442c096be0 tx=0x7f442c00bc60 comp rx=0 tx=0).stop 2023-09-10T01:07:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.117+0000 7f444b0eab80 1 -- 172.21.15.44:0/2241159302 shutdown_connections 2023-09-10T01:07:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.117+0000 7f444b0eab80 1 --2- 172.21.15.44:0/2241159302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c5137acc0 0x560c5138f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.117+0000 7f444b0eab80 1 --2- 172.21.15.44:0/2241159302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c51387990 0x560c51387d70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.117+0000 7f444b0eab80 1 --2- 172.21.15.44:0/2241159302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c5137a300 0x560c5137a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.117+0000 7f444b0eab80 1 -- 172.21.15.44:0/2241159302 >> 172.21.15.44:0/2241159302 conn(0x560c512ef030 msgr2=0x560c51375240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.117+0000 7f444b0eab80 1 -- 172.21.15.44:0/2241159302 shutdown_connections 2023-09-10T01:07:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.117+0000 7f444b0eab80 1 -- 172.21.15.44:0/2241159302 wait complete. 2023-09-10T01:07:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.118+0000 7f444b0eab80 1 -- start start 2023-09-10T01:07:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.118+0000 7f444b0eab80 1 Processor -- start 2023-09-10T01:07:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.118+0000 7f444b0eab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137a300 0x560c5138f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.118+0000 7f444b0eab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c5137acc0 0x560c5138c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.118+0000 7f444b0eab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c51387990 0x560c5138cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.118+0000 7f444b0eab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5126ab10 con 0x560c51387990 2023-09-10T01:07:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.118+0000 7f444b0eab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5126a810 con 0x560c5137acc0 2023-09-10T01:07:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.118+0000 7f444b0eab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5114e8c0 con 0x560c5137a300 2023-09-10T01:07:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.118+0000 7f444b0eab80 10 client.0 _ll_register_callbacks cb 0x560c511b4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:07:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.118+0000 7f443ccc0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137a300 0x560c5138f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.118+0000 7f443ccc0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137a300 0x560c5138f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34704/0 (socket says 172.21.15.44:34704) 2023-09-10T01:07:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.119+0000 7f443ccc0700 1 -- 172.21.15.44:0/2456777018 learned_addr learned my addr 172.21.15.44:0/2456777018 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.119+0000 7f4437fff700 1 --2- 172.21.15.44:0/2456777018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c5137acc0 0x560c5138c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.119+0000 7f443ccc0700 1 -- 172.21.15.44:0/2456777018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c5137acc0 msgr2=0x560c5138c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.119+0000 7f443ccc0700 1 --2- 172.21.15.44:0/2456777018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c5137acc0 0x560c5138c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.119+0000 7f443ccc0700 1 -- 172.21.15.44:0/2456777018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c51387990 msgr2=0x560c5138cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.119+0000 7f443ccc0700 1 --2- 172.21.15.44:0/2456777018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c51387990 0x560c5138cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.119+0000 7f443ccc0700 1 -- 172.21.15.44:0/2456777018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4424017070 con 0x560c5137a300 2023-09-10T01:07:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.119+0000 7f4437fff700 1 --2- 172.21.15.44:0/2456777018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c5137acc0 0x560c5138c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:07:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.119+0000 7f443ccc0700 1 --2- 172.21.15.44:0/2456777018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137a300 0x560c5138f470 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f442c096670 tx=0x7f442c0ca360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a226e3e6c5f9f17 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.119+0000 7f4417fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:07:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.119+0000 7f4417fff700 1 -- 172.21.15.44:0/2456777018 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c0053c0 con 0x560c5137a300 2023-09-10T01:07:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.119+0000 7f444b0eab80 1 -- 172.21.15.44:0/2456777018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c51224490 con 0x560c5137a300 2023-09-10T01:07:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.119+0000 7f4417fff700 1 -- 172.21.15.44:0/2456777018 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f442c09dfa0 con 0x560c5137a300 2023-09-10T01:07:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.119+0000 7f4417fff700 1 -- 172.21.15.44:0/2456777018 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f442c09a3d0 con 0x560c5137a300 2023-09-10T01:07:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.119+0000 7f444b0eab80 10 client.28477 fetch_fsmap learned FSMap version 109 2023-09-10T01:07:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.119+0000 7f444b0eab80 1 -- 172.21.15.44:0/2456777018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=4+}) v3 -- 0x7f44381972e0 con 0x560c5137a300 2023-09-10T01:07:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.120+0000 7f4417fff700 1 -- 172.21.15.44:0/2456777018 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560c51224490 con 0x560c5137a300 2023-09-10T01:07:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.120+0000 7f4417fff700 1 -- 172.21.15.44:0/2456777018 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 109) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f442c0a1cc0 con 0x560c5137a300 2023-09-10T01:07:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.120+0000 7f444b0eab80 10 client.28477 fetch_fsmap finished waiting for FSMap version 109 2023-09-10T01:07:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.120+0000 7f444b0eab80 10 client.28477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:07:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.120+0000 7f444b0eab80 20 client.28477 populate_metadata read hostname 'smithi044' 2023-09-10T01:07:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.120+0000 7f444b0eab80 1 --2- 172.21.15.44:0/2456777018 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c513a4f30 0x560c513a7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.120+0000 7f444b0eab80 4 client.28477 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:07:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.120+0000 7f444b0eab80 1 -- 172.21.15.44:0/2456777018 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c512e1840 con 0x560c513a4f30 2023-09-10T01:07:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.120+0000 7f4437fff700 1 --2- 172.21.15.44:0/2456777018 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c513a4f30 0x560c513a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.121+0000 7f4417fff700 10 client.28477 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.121+0000 7f4437fff700 1 --2- 172.21.15.44:0/2456777018 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c513a4f30 0x560c513a7320 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4757ecf21daba3bb server_cookie=e01f0c33ac3b996a in_seq=0 out_seq=0 2023-09-10T01:07:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.121+0000 7f4417fff700 1 -- 172.21.15.44:0/2456777018 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560c512e1840 con 0x560c513a4f30 2023-09-10T01:07:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.121+0000 7f4417fff700 10 client.28477 handle_command_reply: tid=0 2023-09-10T01:07:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.121+0000 7f444b0eab80 1 client.28477 shutdown 2023-09-10T01:07:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.122+0000 7f444b0eab80 1 -- 172.21.15.44:0/2456777018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137a300 msgr2=0x560c5138f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.122+0000 7f444b0eab80 1 --2- 172.21.15.44:0/2456777018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137a300 0x560c5138f470 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f442c096670 tx=0x7f442c0ca360 comp rx=0 tx=0).stop 2023-09-10T01:07:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.122+0000 7f444b0eab80 1 -- 172.21.15.44:0/2456777018 shutdown_connections 2023-09-10T01:07:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.122+0000 7f444b0eab80 1 --2- 172.21.15.44:0/2456777018 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c513a4f30 0x560c513a7320 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.122+0000 7f444b0eab80 1 --2- 172.21.15.44:0/2456777018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c51387990 0x560c5138cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.122+0000 7f4417fff700 0 client.28477 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.122+0000 7f444b0eab80 1 --2- 172.21.15.44:0/2456777018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137a300 0x560c5138f470 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.122+0000 7f444b0eab80 1 --2- 172.21.15.44:0/2456777018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c5137acc0 0x560c5138c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.122+0000 7f444b0eab80 1 -- 172.21.15.44:0/2456777018 >> 172.21.15.44:0/2456777018 conn(0x560c512ef030 msgr2=0x560c513747a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.122+0000 7f444b0eab80 1 -- 172.21.15.44:0/2456777018 shutdown_connections 2023-09-10T01:07:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.122+0000 7f444b0eab80 1 -- 172.21.15.44:0/2456777018 wait complete. 2023-09-10T01:07:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.122+0000 7f444b0eab80 20 client.28477 trim_cache size 0 max 16384 2023-09-10T01:07:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.143+0000 7f444b0eab80 1 Processor -- start 2023-09-10T01:07:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.143+0000 7f444b0eab80 1 -- start start 2023-09-10T01:07:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f444b0eab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c5137a300 0x560c51379d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f444b0eab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137acc0 0x560c5138efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f444b0eab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c51387990 0x560c51374f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f444b0eab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5138d270 con 0x560c51387990 2023-09-10T01:07:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f444b0eab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5138db80 con 0x560c5137a300 2023-09-10T01:07:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f444b0eab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c51383e20 con 0x560c5137acc0 2023-09-10T01:07:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f443d4c1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c51387990 0x560c51374f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f4437fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137acc0 0x560c5138efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f4437fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137acc0 0x560c5138efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34714/0 (socket says 172.21.15.44:34714) 2023-09-10T01:07:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f443d4c1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c51387990 0x560c51374f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56066/0 (socket says 172.21.15.44:56066) 2023-09-10T01:07:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f4437fff700 1 -- 172.21.15.44:0/1961002855 learned_addr learned my addr 172.21.15.44:0/1961002855 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f443ccc0700 1 --2- 172.21.15.44:0/1961002855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c5137a300 0x560c51379d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f4437fff700 1 -- 172.21.15.44:0/1961002855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c5137a300 msgr2=0x560c51379d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f4437fff700 1 --2- 172.21.15.44:0/1961002855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c5137a300 0x560c51379d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f4437fff700 1 -- 172.21.15.44:0/1961002855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c51387990 msgr2=0x560c51374f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f4437fff700 1 --2- 172.21.15.44:0/1961002855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c51387990 0x560c51374f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f4437fff700 1 -- 172.21.15.44:0/1961002855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c513866a0 con 0x560c5137acc0 2023-09-10T01:07:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.144+0000 7f443ccc0700 1 --2- 172.21.15.44:0/1961002855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c5137a300 0x560c51379d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.145+0000 7f4437fff700 1 --2- 172.21.15.44:0/1961002855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137acc0 0x560c5138efa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f44240097c0 tx=0x7f4424014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9cff480803d2a05 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.145+0000 7f44237fe700 1 -- 172.21.15.44:0/1961002855 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424003a60 con 0x560c5137acc0 2023-09-10T01:07:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.145+0000 7f44237fe700 1 -- 172.21.15.44:0/1961002855 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4424023040 con 0x560c5137acc0 2023-09-10T01:07:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.145+0000 7f44237fe700 1 -- 172.21.15.44:0/1961002855 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442401d410 con 0x560c5137acc0 2023-09-10T01:07:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.146+0000 7f444b0eab80 1 -- 172.21.15.44:0/1961002855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137acc0 msgr2=0x560c5138efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.146+0000 7f444b0eab80 1 --2- 172.21.15.44:0/1961002855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137acc0 0x560c5138efa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f44240097c0 tx=0x7f4424014000 comp rx=0 tx=0).stop 2023-09-10T01:07:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.146+0000 7f444b0eab80 1 -- 172.21.15.44:0/1961002855 shutdown_connections 2023-09-10T01:07:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.146+0000 7f444b0eab80 1 --2- 172.21.15.44:0/1961002855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c51387990 0x560c51374f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.146+0000 7f444b0eab80 1 --2- 172.21.15.44:0/1961002855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c5137a300 0x560c51379d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.146+0000 7f444b0eab80 1 --2- 172.21.15.44:0/1961002855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137acc0 0x560c5138efa0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.146+0000 7f444b0eab80 1 -- 172.21.15.44:0/1961002855 >> 172.21.15.44:0/1961002855 conn(0x560c512ef030 msgr2=0x560c5138b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.146+0000 7f444b0eab80 1 -- 172.21.15.44:0/1961002855 shutdown_connections 2023-09-10T01:07:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.146+0000 7f444b0eab80 1 -- 172.21.15.44:0/1961002855 wait complete. 2023-09-10T01:07:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.146+0000 7f444b0eab80 1 -- start start 2023-09-10T01:07:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.146+0000 7f444b0eab80 1 Processor -- start 2023-09-10T01:07:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f444b0eab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137a300 0x560c5137ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f444b0eab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c5137acc0 0x560c5137d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f444b0eab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c51387990 0x560c5137d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f444b0eab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5137f810 con 0x560c5137acc0 2023-09-10T01:07:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f444b0eab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5137dce0 con 0x560c51387990 2023-09-10T01:07:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f444b0eab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5137dea0 con 0x560c5137a300 2023-09-10T01:07:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f444b0eab80 10 client.0 _ll_register_callbacks cb 0x7f44240096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:07:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f443ccc0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137a300 0x560c5137ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f443ccc0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137a300 0x560c5137ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34718/0 (socket says 172.21.15.44:34718) 2023-09-10T01:07:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f443ccc0700 1 -- 172.21.15.44:0/1216430059 learned_addr learned my addr 172.21.15.44:0/1216430059 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:07:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f443d4c1700 1 --2- 172.21.15.44:0/1216430059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c51387990 0x560c5137d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f4437fff700 1 --2- 172.21.15.44:0/1216430059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c5137acc0 0x560c5137d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f443ccc0700 1 -- 172.21.15.44:0/1216430059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c51387990 msgr2=0x560c5137d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f443ccc0700 1 --2- 172.21.15.44:0/1216430059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c51387990 0x560c5137d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f443ccc0700 1 -- 172.21.15.44:0/1216430059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c5137acc0 msgr2=0x560c5137d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:07:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f443ccc0700 1 --2- 172.21.15.44:0/1216430059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c5137acc0 0x560c5137d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f443ccc0700 1 -- 172.21.15.44:0/1216430059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44381972e0 con 0x560c5137a300 2023-09-10T01:07:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f4437fff700 1 --2- 172.21.15.44:0/1216430059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c5137acc0 0x560c5137d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.147+0000 7f443d4c1700 1 --2- 172.21.15.44:0/1216430059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c51387990 0x560c5137d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:07:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.148+0000 7f443ccc0700 1 --2- 172.21.15.44:0/1216430059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137a300 0x560c5137ef40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f442c096be0 tx=0x7f442c099890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9eaf6863a158666 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:07:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.148+0000 7f4417fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:07:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.148+0000 7f4417fff700 1 -- 172.21.15.44:0/1216430059 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c0053c0 con 0x560c5137a300 2023-09-10T01:07:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.148+0000 7f4417fff700 1 -- 172.21.15.44:0/1216430059 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f442c09dfa0 con 0x560c5137a300 2023-09-10T01:07:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.148+0000 7f444b0eab80 1 -- 172.21.15.44:0/1216430059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f442c09a870 con 0x560c5137a300 2023-09-10T01:07:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.148+0000 7f4417fff700 1 -- 172.21.15.44:0/1216430059 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f442c0b2510 con 0x560c5137a300 2023-09-10T01:07:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.148+0000 7f444b0eab80 10 client.28483 fetch_fsmap learned FSMap version 109 2023-09-10T01:07:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.148+0000 7f444b0eab80 1 -- 172.21.15.44:0/1216430059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=4+}) v3 -- 0x7f4424004040 con 0x560c5137a300 2023-09-10T01:07:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.148+0000 7f4417fff700 1 -- 172.21.15.44:0/1216430059 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f442c0aa080 con 0x560c5137a300 2023-09-10T01:07:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.148+0000 7f4417fff700 1 -- 172.21.15.44:0/1216430059 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 109) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f442c095f60 con 0x560c5137a300 2023-09-10T01:07:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.148+0000 7f444b0eab80 10 client.28483 fetch_fsmap finished waiting for FSMap version 109 2023-09-10T01:07:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.148+0000 7f444b0eab80 10 client.28483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:07:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.148+0000 7f444b0eab80 20 client.28483 populate_metadata read hostname 'smithi044' 2023-09-10T01:07:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.148+0000 7f444b0eab80 1 --2- 172.21.15.44:0/1216430059 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c5138fe20 0x560c51392240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:07:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.148+0000 7f444b0eab80 4 client.28483 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:07:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.148+0000 7f444b0eab80 1 -- 172.21.15.44:0/1216430059 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c512c58c0 con 0x560c5138fe20 2023-09-10T01:07:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.149+0000 7f4437fff700 1 --2- 172.21.15.44:0/1216430059 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c5138fe20 0x560c51392240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:07:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.149+0000 7f4417fff700 10 client.28483 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.149+0000 7f4437fff700 1 --2- 172.21.15.44:0/1216430059 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c5138fe20 0x560c51392240 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cbb578c6bf4f5b5 server_cookie=dc9f8c7f6b074e26 in_seq=0 out_seq=0 2023-09-10T01:07:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.158+0000 7f4417fff700 1 -- 172.21.15.44:0/1216430059 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c512c58c0 con 0x560c5138fe20 2023-09-10T01:07:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.158+0000 7f4417fff700 10 client.28483 handle_command_reply: tid=0 2023-09-10T01:07:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.158+0000 7f444b0eab80 1 client.28483 shutdown 2023-09-10T01:07:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.158+0000 7f444b0eab80 1 -- 172.21.15.44:0/1216430059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137a300 msgr2=0x560c5137ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.158+0000 7f444b0eab80 1 --2- 172.21.15.44:0/1216430059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137a300 0x560c5137ef40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f442c096be0 tx=0x7f442c099890 comp rx=0 tx=0).stop 2023-09-10T01:07:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.158+0000 7f444b0eab80 1 -- 172.21.15.44:0/1216430059 shutdown_connections 2023-09-10T01:07:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.158+0000 7f444b0eab80 1 --2- 172.21.15.44:0/1216430059 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c5138fe20 0x560c51392240 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.158+0000 7f444b0eab80 1 --2- 172.21.15.44:0/1216430059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c51387990 0x560c5137d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.158+0000 7f444b0eab80 1 --2- 172.21.15.44:0/1216430059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5137a300 0x560c5137ef40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.158+0000 7f4417fff700 0 client.28483 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:07:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.158+0000 7f444b0eab80 1 --2- 172.21.15.44:0/1216430059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c5137acc0 0x560c5137d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.158+0000 7f444b0eab80 1 -- 172.21.15.44:0/1216430059 >> 172.21.15.44:0/1216430059 conn(0x560c512ef030 msgr2=0x560c5138bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.158+0000 7f444b0eab80 1 -- 172.21.15.44:0/1216430059 shutdown_connections 2023-09-10T01:07:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.158+0000 7f444b0eab80 1 -- 172.21.15.44:0/1216430059 wait complete. 2023-09-10T01:07:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.158+0000 7f444b0eab80 20 client.28483 trim_cache size 0 max 16384 2023-09-10T01:07:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:07:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:07:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:07:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:07:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.159+0000 7f443ef24700 1 -- 172.21.15.44:0/3871778247 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44280a0830 msgr2=0x7f44280a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.159+0000 7f443ef24700 1 --2- 172.21.15.44:0/3871778247 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44280a0830 0x7f44280a2cf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f44380779f0 tx=0x7f4424009260 comp rx=0 tx=0).stop 2023-09-10T01:07:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.159+0000 7f443ef24700 1 -- 172.21.15.44:0/3871778247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44380684b0 msgr2=0x7f443819c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:07:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.159+0000 7f443ef24700 1 --2- 172.21.15.44:0/3871778247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44380684b0 0x7f443819c620 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f442c009810 tx=0x7f442c00fa00 comp rx=0 tx=0).stop 2023-09-10T01:07:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.159+0000 7f443ef24700 1 -- 172.21.15.44:0/3871778247 shutdown_connections 2023-09-10T01:07:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.159+0000 7f443ef24700 1 --2- 172.21.15.44:0/3871778247 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44280a0830 0x7f44280a2cf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.159+0000 7f443ef24700 1 --2- 172.21.15.44:0/3871778247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4438106680 0x7f4438196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.159+0000 7f443ef24700 1 --2- 172.21.15.44:0/3871778247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44380684b0 0x7f443819c620 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.159+0000 7f443ef24700 1 --2- 172.21.15.44:0/3871778247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4438068e70 0x7f443819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:07:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.159+0000 7f443ef24700 1 -- 172.21.15.44:0/3871778247 >> 172.21.15.44:0/3871778247 conn(0x7f4438077430 msgr2=0x7f44380fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:07:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.159+0000 7f443ef24700 1 -- 172.21.15.44:0/3871778247 shutdown_connections 2023-09-10T01:07:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:07:56.159+0000 7f443ef24700 1 -- 172.21.15.44:0/3871778247 wait complete. 2023-09-10T01:07:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:56 smithi131 ceph-mon[103713]: pgmap v866: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 236 op/s 2023-09-10T01:07:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:56 smithi044 ceph-mon[98676]: pgmap v866: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 236 op/s 2023-09-10T01:07:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:56 smithi099 ceph-mon[102692]: pgmap v866: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 236 op/s 2023-09-10T01:07:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:07:58 smithi131 ceph-mon[103713]: pgmap v867: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:07:59.504 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:07:58 smithi099 ceph-mon[102692]: pgmap v867: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:07:59.504 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:07:58 smithi044 ceph-mon[98676]: pgmap v867: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:08:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:00 smithi131 ceph-mon[103713]: pgmap v868: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:08:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:00 smithi044 ceph-mon[98676]: pgmap v868: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:08:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:00 smithi099 ceph-mon[102692]: pgmap v868: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:08:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:02 smithi131 ceph-mon[103713]: pgmap v869: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 230 op/s 2023-09-10T01:08:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:02 smithi044 ceph-mon[98676]: pgmap v869: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 230 op/s 2023-09-10T01:08:03.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:02 smithi099 ceph-mon[102692]: pgmap v869: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 230 op/s 2023-09-10T01:08:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:04 smithi131 ceph-mon[103713]: pgmap v870: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1007 KiB/s wr, 143 op/s 2023-09-10T01:08:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:04 smithi044 ceph-mon[98676]: pgmap v870: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1007 KiB/s wr, 143 op/s 2023-09-10T01:08:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:04 smithi099 ceph-mon[102692]: pgmap v870: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1007 KiB/s wr, 143 op/s 2023-09-10T01:08:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:07 smithi131 ceph-mon[103713]: pgmap v871: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 243 op/s 2023-09-10T01:08:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:07 smithi044 ceph-mon[98676]: pgmap v871: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 243 op/s 2023-09-10T01:08:07.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:07 smithi099 ceph-mon[102692]: pgmap v871: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 243 op/s 2023-09-10T01:08:09.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:09 smithi131 ceph-mon[103713]: pgmap v872: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-09-10T01:08:09.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:09 smithi044 ceph-mon[98676]: pgmap v872: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-09-10T01:08:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.354+0000 7fbc0ae47700 1 -- 172.21.15.44:0/886634025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc0410bf60 msgr2=0x7fbc04113ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.354+0000 7fbc0ae47700 1 --2- 172.21.15.44:0/886634025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc0410bf60 0x7fbc04113ae0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009f90 tx=0x7fbbfc00b330 comp rx=0 tx=0).stop 2023-09-10T01:08:09.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:09 smithi099 ceph-mon[102692]: pgmap v872: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-09-10T01:08:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.357+0000 7fbc0ae47700 1 -- 172.21.15.44:0/886634025 shutdown_connections 2023-09-10T01:08:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.357+0000 7fbc0ae47700 1 --2- 172.21.15.44:0/886634025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc0410bf60 0x7fbc04113ae0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.357+0000 7fbc0ae47700 1 --2- 172.21.15.44:0/886634025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc04106f90 0x7fbc04107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.357+0000 7fbc0ae47700 1 --2- 172.21.15.44:0/886634025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc04107940 0x7fbc0410b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.357+0000 7fbc0ae47700 1 -- 172.21.15.44:0/886634025 >> 172.21.15.44:0/886634025 conn(0x7fbc040764a0 msgr2=0x7fbc04105420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.360+0000 7fbc0ae47700 1 -- 172.21.15.44:0/886634025 shutdown_connections 2023-09-10T01:08:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.360+0000 7fbc0ae47700 1 -- 172.21.15.44:0/886634025 wait complete. 2023-09-10T01:08:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.361+0000 7fbc0ae47700 1 Processor -- start 2023-09-10T01:08:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.361+0000 7fbc0ae47700 1 -- start start 2023-09-10T01:08:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.362+0000 7fbc0ae47700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc04106f90 0x7fbc0406c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.362+0000 7fbc0ae47700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc0410bf60 0x7fbc0406ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.362+0000 7fbc0ae47700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041af420 0x7fbc041b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.362+0000 7fbc0ae47700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc041b1d80 con 0x7fbc04106f90 2023-09-10T01:08:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.362+0000 7fbc0ae47700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc04078e80 con 0x7fbc0410bf60 2023-09-10T01:08:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.362+0000 7fbc0ae47700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc04079040 con 0x7fbc041af420 2023-09-10T01:08:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.362+0000 7fbc03fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc0410bf60 0x7fbc0406ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.362+0000 7fbc093e4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041af420 0x7fbc041b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.362+0000 7fbc03fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc0410bf60 0x7fbc0406ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57822/0 (socket says 172.21.15.44:57822) 2023-09-10T01:08:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.362+0000 7fbc093e4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041af420 0x7fbc041b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34734/0 (socket says 172.21.15.44:34734) 2023-09-10T01:08:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.362+0000 7fbc03fff700 1 -- 172.21.15.44:0/3426465237 learned_addr learned my addr 172.21.15.44:0/3426465237 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.363+0000 7fbc03fff700 1 -- 172.21.15.44:0/3426465237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041af420 msgr2=0x7fbc041b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.363+0000 7fbc03fff700 1 --2- 172.21.15.44:0/3426465237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041af420 0x7fbc041b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.363+0000 7fbc03fff700 1 -- 172.21.15.44:0/3426465237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc04106f90 msgr2=0x7fbc0406c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.363+0000 7fbc03fff700 1 --2- 172.21.15.44:0/3426465237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc04106f90 0x7fbc0406c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.363+0000 7fbc03fff700 1 -- 172.21.15.44:0/3426465237 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc009c40 con 0x7fbc0410bf60 2023-09-10T01:08:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.363+0000 7fbc03fff700 1 --2- 172.21.15.44:0/3426465237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc0410bf60 0x7fbc0406ceb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800d980 tx=0x7fbbf800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.371+0000 7fbc01ffb700 1 -- 172.21.15.44:0/3426465237 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8009b40 con 0x7fbc0410bf60 2023-09-10T01:08:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.371+0000 7fbc01ffb700 1 -- 172.21.15.44:0/3426465237 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbbf800de60 con 0x7fbc0410bf60 2023-09-10T01:08:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.372+0000 7fbc01ffb700 1 -- 172.21.15.44:0/3426465237 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf800f620 con 0x7fbc0410bf60 2023-09-10T01:08:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.372+0000 7fbc0ae47700 1 -- 172.21.15.44:0/3426465237 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc04079320 con 0x7fbc0410bf60 2023-09-10T01:08:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.372+0000 7fbc0ae47700 1 -- 172.21.15.44:0/3426465237 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc040797e0 con 0x7fbc0410bf60 2023-09-10T01:08:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.378+0000 7fbc1700db80 1 Processor -- start 2023-09-10T01:08:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.379+0000 7fbc01ffb700 1 -- 172.21.15.44:0/3426465237 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fbbf8010460 con 0x7fbc0410bf60 2023-09-10T01:08:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.380+0000 7fbc01ffb700 1 --2- 172.21.15.44:0/3426465237 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbbf40a08b0 0x7fbbf40a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.380+0000 7fbc01ffb700 1 -- 172.21.15.44:0/3426465237 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fbbf80cbcf0 con 0x7fbc0410bf60 2023-09-10T01:08:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.380+0000 7fbc1700db80 1 -- start start 2023-09-10T01:08:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.381+0000 7fbc1700db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc241a0 0x55f72bc24580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.381+0000 7fbc1700db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc1e300 0x55f72bc1e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.381+0000 7fbc1700db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc1ecc0 0x55f72bc2af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.381+0000 7fbc1700db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72bb776d0 con 0x55f72bc1e300 2023-09-10T01:08:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.381+0000 7fbc1700db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72bb74ff0 con 0x55f72bc241a0 2023-09-10T01:08:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.381+0000 7fbc1700db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72bb708e0 con 0x55f72bc1ecc0 2023-09-10T01:08:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.381+0000 7fbc093e4700 1 --2- 172.21.15.44:0/3426465237 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbbf40a08b0 0x7fbbf40a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.381+0000 7fbc093e4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc1ecc0 0x55f72bc2af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.381+0000 7fbc093e4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc1ecc0 0x55f72bc2af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34738/0 (socket says 172.21.15.44:34738) 2023-09-10T01:08:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.381+0000 7fbc093e4700 1 -- 172.21.15.44:0/269626693 learned_addr learned my addr 172.21.15.44:0/269626693 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.382+0000 7fbc08be3700 1 --2- 172.21.15.44:0/269626693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc241a0 0x55f72bc24580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.382+0000 7fbc093e4700 1 -- 172.21.15.44:0/269626693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc241a0 msgr2=0x55f72bc24580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.382+0000 7fbc093e4700 1 --2- 172.21.15.44:0/269626693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc241a0 0x55f72bc24580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.382+0000 7fbc093e4700 1 -- 172.21.15.44:0/269626693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc1e300 msgr2=0x55f72bc1e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.382+0000 7fbc093e4700 1 --2- 172.21.15.44:0/269626693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc1e300 0x55f72bc1e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.382+0000 7fbc093e4700 1 -- 172.21.15.44:0/269626693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f72b99cd80 con 0x55f72bc1ecc0 2023-09-10T01:08:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.382+0000 7fbc093e4700 1 --2- 172.21.15.44:0/269626693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc1ecc0 0x55f72bc2af30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009640 tx=0x7fbbfc022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=737617304e024433 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.382+0000 7fbc093e4700 1 --2- 172.21.15.44:0/3426465237 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbbf40a08b0 0x7fbbf40a2d70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc007fd0 tx=0x7fbbfc00bcd0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.384+0000 7fbbeeffd700 1 -- 172.21.15.44:0/269626693 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc0073e0 con 0x55f72bc1ecc0 2023-09-10T01:08:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.384+0000 7fbbeeffd700 1 -- 172.21.15.44:0/269626693 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbbfc0050b0 con 0x55f72bc1ecc0 2023-09-10T01:08:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.384+0000 7fbbeeffd700 1 -- 172.21.15.44:0/269626693 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc01e6c0 con 0x55f72bc1ecc0 2023-09-10T01:08:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.388+0000 7fbc1700db80 1 -- 172.21.15.44:0/269626693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc1ecc0 msgr2=0x55f72bc2af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.388+0000 7fbc1700db80 1 --2- 172.21.15.44:0/269626693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc1ecc0 0x55f72bc2af30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009640 tx=0x7fbbfc022000 comp rx=0 tx=0).stop 2023-09-10T01:08:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.389+0000 7fbc1700db80 1 -- 172.21.15.44:0/269626693 shutdown_connections 2023-09-10T01:08:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.389+0000 7fbc1700db80 1 --2- 172.21.15.44:0/269626693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc1ecc0 0x55f72bc2af30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.389+0000 7fbc1700db80 1 --2- 172.21.15.44:0/269626693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc241a0 0x55f72bc24580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.389+0000 7fbc1700db80 1 --2- 172.21.15.44:0/269626693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc1e300 0x55f72bc1e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.389+0000 7fbc1700db80 1 -- 172.21.15.44:0/269626693 >> 172.21.15.44:0/269626693 conn(0x55f72bb93030 msgr2=0x55f72bc19560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.394+0000 7fbc1700db80 1 -- 172.21.15.44:0/269626693 shutdown_connections 2023-09-10T01:08:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.401+0000 7fbc1700db80 1 -- 172.21.15.44:0/269626693 wait complete. 2023-09-10T01:08:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.401+0000 7fbc1700db80 1 -- start start 2023-09-10T01:08:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.401+0000 7fbc1700db80 1 Processor -- start 2023-09-10T01:08:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.402+0000 7fbc1700db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc1e300 0x55f72bc28230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.402+0000 7fbc1700db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc1ecc0 0x55f72bc28770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.402+0000 7fbc1700db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc241a0 0x55f72bc28cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.402+0000 7fbc1700db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72bb0eb10 con 0x55f72bc1e300 2023-09-10T01:08:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.402+0000 7fbc1700db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72bb0e810 con 0x55f72bc1ecc0 2023-09-10T01:08:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.402+0000 7fbc1700db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72b9f28c0 con 0x55f72bc241a0 2023-09-10T01:08:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.402+0000 7fbc1700db80 10 client.0 _ll_register_callbacks cb 0x55f72ba58a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:08:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.406+0000 7fbc093e4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc241a0 0x55f72bc28cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.407+0000 7fbc093e4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc241a0 0x55f72bc28cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34744/0 (socket says 172.21.15.44:34744) 2023-09-10T01:08:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.407+0000 7fbc093e4700 1 -- 172.21.15.44:0/1557861239 learned_addr learned my addr 172.21.15.44:0/1557861239 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.407+0000 7fbc03fff700 1 --2- 172.21.15.44:0/1557861239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc1ecc0 0x55f72bc28770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.413+0000 7fbc093e4700 1 -- 172.21.15.44:0/1557861239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc1ecc0 msgr2=0x55f72bc28770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.413+0000 7fbc093e4700 1 --2- 172.21.15.44:0/1557861239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc1ecc0 0x55f72bc28770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.413+0000 7fbc093e4700 1 -- 172.21.15.44:0/1557861239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc1e300 msgr2=0x55f72bc28230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.413+0000 7fbc093e4700 1 --2- 172.21.15.44:0/1557861239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc1e300 0x55f72bc28230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.413+0000 7fbc093e4700 1 -- 172.21.15.44:0/1557861239 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc04114350 con 0x55f72bc241a0 2023-09-10T01:08:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.426+0000 7fbc093e4700 1 --2- 172.21.15.44:0/1557861239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc241a0 0x55f72bc28cb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc00b8e0 tx=0x7fbbfc006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c7d91cc3de88bf6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.426+0000 7fbbe3fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:08:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.427+0000 7fbbe3fff700 1 -- 172.21.15.44:0/1557861239 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc0075e0 con 0x55f72bc241a0 2023-09-10T01:08:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.427+0000 7fbbe3fff700 1 -- 172.21.15.44:0/1557861239 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fbbfc026070 con 0x55f72bc241a0 2023-09-10T01:08:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.427+0000 7fbc1700db80 1 -- 172.21.15.44:0/1557861239 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f72bac8490 con 0x55f72bc241a0 2023-09-10T01:08:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.427+0000 7fbbe3fff700 1 -- 172.21.15.44:0/1557861239 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbfc007740 con 0x55f72bc241a0 2023-09-10T01:08:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.427+0000 7fbc1700db80 10 client.28492 fetch_fsmap learned FSMap version 109 2023-09-10T01:08:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.427+0000 7fbc1700db80 1 -- 172.21.15.44:0/1557861239 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=4+}) v3 -- 0x55f72b985c00 con 0x55f72bc241a0 2023-09-10T01:08:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.427+0000 7fbbe3fff700 1 -- 172.21.15.44:0/1557861239 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f72bac8490 con 0x55f72bc241a0 2023-09-10T01:08:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.428+0000 7fbbe3fff700 1 -- 172.21.15.44:0/1557861239 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 109) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbbfc020450 con 0x55f72bc241a0 2023-09-10T01:08:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.428+0000 7fbc1700db80 10 client.28492 fetch_fsmap finished waiting for FSMap version 109 2023-09-10T01:08:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.428+0000 7fbc1700db80 10 client.28492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:08:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.428+0000 7fbc1700db80 20 client.28492 populate_metadata read hostname 'smithi044' 2023-09-10T01:08:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.428+0000 7fbc1700db80 1 --2- 172.21.15.44:0/1557861239 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f72bc5a090 0x55f72bc5c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.428+0000 7fbc1700db80 4 client.28492 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:08:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.428+0000 7fbc1700db80 1 -- 172.21.15.44:0/1557861239 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f72bb85840 con 0x55f72bc5a090 2023-09-10T01:08:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.428+0000 7fbc08be3700 1 --2- 172.21.15.44:0/1557861239 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f72bc5a090 0x55f72bc5c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.429+0000 7fbbe3fff700 10 client.28492 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:08:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.429+0000 7fbc08be3700 1 --2- 172.21.15.44:0/1557861239 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f72bc5a090 0x55f72bc5c480 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6249bb15a40dc1c7 server_cookie=e35f06c959f48c76 in_seq=0 out_seq=0 2023-09-10T01:08:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.439+0000 7fbbe3fff700 1 -- 172.21.15.44:0/1557861239 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f72bb85840 con 0x55f72bc5a090 2023-09-10T01:08:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.439+0000 7fbbe3fff700 10 client.28492 handle_command_reply: tid=0 2023-09-10T01:08:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.439+0000 7fbc1700db80 1 client.28492 shutdown 2023-09-10T01:08:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.444+0000 7fbc1700db80 1 -- 172.21.15.44:0/1557861239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc241a0 msgr2=0x55f72bc28cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.444+0000 7fbc1700db80 1 --2- 172.21.15.44:0/1557861239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc241a0 0x55f72bc28cb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc00b8e0 tx=0x7fbbfc006000 comp rx=0 tx=0).stop 2023-09-10T01:08:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.446+0000 7fbc1700db80 1 -- 172.21.15.44:0/1557861239 shutdown_connections 2023-09-10T01:08:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.446+0000 7fbc1700db80 1 --2- 172.21.15.44:0/1557861239 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f72bc5a090 0x55f72bc5c480 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.446+0000 7fbc1700db80 1 --2- 172.21.15.44:0/1557861239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc241a0 0x55f72bc28cb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.446+0000 7fbc1700db80 1 --2- 172.21.15.44:0/1557861239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc1e300 0x55f72bc28230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.446+0000 7fbc1700db80 1 --2- 172.21.15.44:0/1557861239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc1ecc0 0x55f72bc28770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.446+0000 7fbc1700db80 1 -- 172.21.15.44:0/1557861239 >> 172.21.15.44:0/1557861239 conn(0x55f72bb93030 msgr2=0x55f72bc187a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.447+0000 7fbbe3fff700 0 client.28492 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:08:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.447+0000 7fbc1700db80 1 -- 172.21.15.44:0/1557861239 shutdown_connections 2023-09-10T01:08:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.450+0000 7fbc1700db80 1 -- 172.21.15.44:0/1557861239 wait complete. 2023-09-10T01:08:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.460+0000 7fbc1700db80 20 client.28492 trim_cache size 0 max 16384 2023-09-10T01:08:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.492+0000 7fbc1700db80 1 Processor -- start 2023-09-10T01:08:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.498+0000 7fbc1700db80 1 -- start start 2023-09-10T01:08:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.499+0000 7fbc1700db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc1e300 0x55f72bc25e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.499+0000 7fbc1700db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc1ecc0 0x55f72bc2f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.499+0000 7fbc1700db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc241a0 0x55f72bc223d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.499+0000 7fbc1700db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72bc2f860 con 0x55f72bc1ecc0 2023-09-10T01:08:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.499+0000 7fbc1700db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72bc26790 con 0x55f72bc1e300 2023-09-10T01:08:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.499+0000 7fbc1700db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72bc22ee0 con 0x55f72bc241a0 2023-09-10T01:08:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.505+0000 7fbc093e4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc241a0 0x55f72bc223d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.505+0000 7fbc093e4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc241a0 0x55f72bc223d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34752/0 (socket says 172.21.15.44:34752) 2023-09-10T01:08:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.505+0000 7fbc093e4700 1 -- 172.21.15.44:0/1876246679 learned_addr learned my addr 172.21.15.44:0/1876246679 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.505+0000 7fbc08be3700 1 --2- 172.21.15.44:0/1876246679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc1e300 0x55f72bc25e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.505+0000 7fbc03fff700 1 --2- 172.21.15.44:0/1876246679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc1ecc0 0x55f72bc2f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.505+0000 7fbc03fff700 1 -- 172.21.15.44:0/1876246679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc241a0 msgr2=0x55f72bc223d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.505+0000 7fbc03fff700 1 --2- 172.21.15.44:0/1876246679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc241a0 0x55f72bc223d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.505+0000 7fbc03fff700 1 -- 172.21.15.44:0/1876246679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc1e300 msgr2=0x55f72bc25e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.505+0000 7fbc03fff700 1 --2- 172.21.15.44:0/1876246679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc1e300 0x55f72bc25e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.505+0000 7fbc03fff700 1 -- 172.21.15.44:0/1876246679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f72b9c31f0 con 0x55f72bc1ecc0 2023-09-10T01:08:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.506+0000 7fbc03fff700 1 --2- 172.21.15.44:0/1876246679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc1ecc0 0x55f72bc2f320 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80c8f90 tx=0x7fbbf8098960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca618ccd6b433bfc server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.506+0000 7fbbef7fe700 1 -- 172.21.15.44:0/1876246679 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80993f0 con 0x55f72bc1ecc0 2023-09-10T01:08:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.506+0000 7fbbef7fe700 1 -- 172.21.15.44:0/1876246679 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbbf8098cf0 con 0x55f72bc1ecc0 2023-09-10T01:08:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.506+0000 7fbbef7fe700 1 -- 172.21.15.44:0/1876246679 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8098f50 con 0x55f72bc1ecc0 2023-09-10T01:08:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.507+0000 7fbc1700db80 1 -- 172.21.15.44:0/1876246679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc1ecc0 msgr2=0x55f72bc2f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.507+0000 7fbc1700db80 1 --2- 172.21.15.44:0/1876246679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc1ecc0 0x55f72bc2f320 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80c8f90 tx=0x7fbbf8098960 comp rx=0 tx=0).stop 2023-09-10T01:08:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.509+0000 7fbc1700db80 1 -- 172.21.15.44:0/1876246679 shutdown_connections 2023-09-10T01:08:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.509+0000 7fbc1700db80 1 --2- 172.21.15.44:0/1876246679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc241a0 0x55f72bc223d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.509+0000 7fbc1700db80 1 --2- 172.21.15.44:0/1876246679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc1e300 0x55f72bc25e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.509+0000 7fbc1700db80 1 --2- 172.21.15.44:0/1876246679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc1ecc0 0x55f72bc2f320 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.509+0000 7fbc1700db80 1 -- 172.21.15.44:0/1876246679 >> 172.21.15.44:0/1876246679 conn(0x55f72bb93030 msgr2=0x55f72bc33a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.517+0000 7fbc1700db80 1 -- 172.21.15.44:0/1876246679 shutdown_connections 2023-09-10T01:08:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.523+0000 7fbc1700db80 1 -- 172.21.15.44:0/1876246679 wait complete. 2023-09-10T01:08:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.523+0000 7fbc1700db80 1 -- start start 2023-09-10T01:08:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.523+0000 7fbc1700db80 1 Processor -- start 2023-09-10T01:08:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.526+0000 7fbc1700db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc1e300 0x55f72bca61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.526+0000 7fbc1700db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc1ecc0 0x55f72bca6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.526+0000 7fbc1700db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc241a0 0x55f72bca1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.526+0000 7fbc1700db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72bca24c0 con 0x55f72bc241a0 2023-09-10T01:08:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.526+0000 7fbc1700db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72bca2630 con 0x55f72bc1e300 2023-09-10T01:08:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.526+0000 7fbc1700db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72bca27f0 con 0x55f72bc1ecc0 2023-09-10T01:08:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.527+0000 7fbc1700db80 10 client.0 _ll_register_callbacks cb 0x7fbbfc007d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:08:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.530+0000 7fbc03fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc1ecc0 0x55f72bca6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.530+0000 7fbc08be3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc1e300 0x55f72bca61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.530+0000 7fbc03fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc1ecc0 0x55f72bca6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34762/0 (socket says 172.21.15.44:34762) 2023-09-10T01:08:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.530+0000 7fbc03fff700 1 -- 172.21.15.44:0/2392122476 learned_addr learned my addr 172.21.15.44:0/2392122476 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.530+0000 7fbc08be3700 1 -- 172.21.15.44:0/2392122476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc1ecc0 msgr2=0x55f72bca6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.530+0000 7fbc08be3700 1 --2- 172.21.15.44:0/2392122476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc1ecc0 0x55f72bca6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.530+0000 7fbc08be3700 1 -- 172.21.15.44:0/2392122476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc241a0 msgr2=0x55f72bca1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.530+0000 7fbc08be3700 1 --2- 172.21.15.44:0/2392122476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc241a0 0x55f72bca1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.530+0000 7fbc08be3700 1 -- 172.21.15.44:0/2392122476 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc040797e0 con 0x55f72bc1e300 2023-09-10T01:08:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.531+0000 7fbc08be3700 1 --2- 172.21.15.44:0/2392122476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc1e300 0x55f72bca61c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000a920 tx=0x7fbbf0016420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b01ff1ff421ceaa server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.531+0000 7fbbe3fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:08:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.531+0000 7fbbe3fff700 1 -- 172.21.15.44:0/2392122476 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0016540 con 0x55f72bc1e300 2023-09-10T01:08:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.531+0000 7fbbe3fff700 1 -- 172.21.15.44:0/2392122476 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fbbf001f480 con 0x55f72bc1e300 2023-09-10T01:08:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.531+0000 7fbc1700db80 1 -- 172.21.15.44:0/2392122476 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbfc0050b0 con 0x55f72bc1e300 2023-09-10T01:08:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.531+0000 7fbbe3fff700 1 -- 172.21.15.44:0/2392122476 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbf001f640 con 0x55f72bc1e300 2023-09-10T01:08:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.531+0000 7fbc1700db80 10 client.28556 fetch_fsmap learned FSMap version 109 2023-09-10T01:08:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.531+0000 7fbc1700db80 1 -- 172.21.15.44:0/2392122476 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=4+}) v3 -- 0x55f72b99b250 con 0x55f72bc1e300 2023-09-10T01:08:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.532+0000 7fbbe3fff700 1 -- 172.21.15.44:0/2392122476 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbbfc0050b0 con 0x55f72bc1e300 2023-09-10T01:08:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.532+0000 7fbbe3fff700 1 -- 172.21.15.44:0/2392122476 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 109) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbbf000f3e0 con 0x55f72bc1e300 2023-09-10T01:08:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.532+0000 7fbc1700db80 10 client.28556 fetch_fsmap finished waiting for FSMap version 109 2023-09-10T01:08:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.532+0000 7fbc1700db80 10 client.28556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:08:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.532+0000 7fbc1700db80 20 client.28556 populate_metadata read hostname 'smithi044' 2023-09-10T01:08:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.532+0000 7fbc1700db80 1 --2- 172.21.15.44:0/2392122476 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f72bcaf070 0x55f72bcb1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.532+0000 7fbc1700db80 4 client.28556 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:08:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.532+0000 7fbc1700db80 1 -- 172.21.15.44:0/2392122476 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f72bb698c0 con 0x55f72bcaf070 2023-09-10T01:08:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.536+0000 7fbc03fff700 1 --2- 172.21.15.44:0/2392122476 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f72bcaf070 0x55f72bcb1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.538+0000 7fbbe3fff700 10 client.28556 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:08:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.539+0000 7fbc03fff700 1 --2- 172.21.15.44:0/2392122476 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f72bcaf070 0x55f72bcb1490 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d6072bdf89916c4 server_cookie=d8a8d0ad039b5b9 in_seq=0 out_seq=0 2023-09-10T01:08:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.581+0000 7fbbe3fff700 1 -- 172.21.15.44:0/2392122476 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f72bb698c0 con 0x55f72bcaf070 2023-09-10T01:08:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.581+0000 7fbbe3fff700 10 client.28556 handle_command_reply: tid=0 2023-09-10T01:08:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.583+0000 7fbc1700db80 1 client.28556 shutdown 2023-09-10T01:08:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.588+0000 7fbc1700db80 1 -- 172.21.15.44:0/2392122476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc1e300 msgr2=0x55f72bca61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.588+0000 7fbc1700db80 1 --2- 172.21.15.44:0/2392122476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc1e300 0x55f72bca61c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000a920 tx=0x7fbbf0016420 comp rx=0 tx=0).stop 2023-09-10T01:08:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.594+0000 7fbc1700db80 1 -- 172.21.15.44:0/2392122476 shutdown_connections 2023-09-10T01:08:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.594+0000 7fbc1700db80 1 --2- 172.21.15.44:0/2392122476 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f72bcaf070 0x55f72bcb1490 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.594+0000 7fbc1700db80 1 --2- 172.21.15.44:0/2392122476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72bc241a0 0x55f72bca1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.594+0000 7fbc1700db80 1 --2- 172.21.15.44:0/2392122476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72bc1e300 0x55f72bca61c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.594+0000 7fbc1700db80 1 --2- 172.21.15.44:0/2392122476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f72bc1ecc0 0x55f72bca6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.594+0000 7fbc1700db80 1 -- 172.21.15.44:0/2392122476 >> 172.21.15.44:0/2392122476 conn(0x55f72bb93030 msgr2=0x55f72bc223d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.594+0000 7fbbe3fff700 0 client.28556 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:08:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.595+0000 7fbc1700db80 1 -- 172.21.15.44:0/2392122476 shutdown_connections 2023-09-10T01:08:09.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.600+0000 7fbc1700db80 1 -- 172.21.15.44:0/2392122476 wait complete. 2023-09-10T01:08:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.613+0000 7fbc1700db80 20 client.28556 trim_cache size 0 max 16384 2023-09-10T01:08:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:08:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:08:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:08:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:08:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.620+0000 7fbc0ae47700 1 -- 172.21.15.44:0/3426465237 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbbf40a08b0 msgr2=0x7fbbf40a2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.620+0000 7fbc0ae47700 1 --2- 172.21.15.44:0/3426465237 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbbf40a08b0 0x7fbbf40a2d70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc007fd0 tx=0x7fbbfc00bcd0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.620+0000 7fbc0ae47700 1 -- 172.21.15.44:0/3426465237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc0410bf60 msgr2=0x7fbc0406ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.620+0000 7fbc0ae47700 1 --2- 172.21.15.44:0/3426465237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc0410bf60 0x7fbc0406ceb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800d980 tx=0x7fbbf800dd40 comp rx=0 tx=0).stop 2023-09-10T01:08:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.626+0000 7fbc0ae47700 1 -- 172.21.15.44:0/3426465237 shutdown_connections 2023-09-10T01:08:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.626+0000 7fbc0ae47700 1 --2- 172.21.15.44:0/3426465237 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbbf40a08b0 0x7fbbf40a2d70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.626+0000 7fbc0ae47700 1 --2- 172.21.15.44:0/3426465237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041af420 0x7fbc041b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.626+0000 7fbc0ae47700 1 --2- 172.21.15.44:0/3426465237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc04106f90 0x7fbc0406c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.626+0000 7fbc0ae47700 1 --2- 172.21.15.44:0/3426465237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc0410bf60 0x7fbc0406ceb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.626+0000 7fbc0ae47700 1 -- 172.21.15.44:0/3426465237 >> 172.21.15.44:0/3426465237 conn(0x7fbc040764a0 msgr2=0x7fbc04076880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.632+0000 7fbc0ae47700 1 -- 172.21.15.44:0/3426465237 shutdown_connections 2023-09-10T01:08:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:09.636+0000 7fbc0ae47700 1 -- 172.21.15.44:0/3426465237 wait complete. 2023-09-10T01:08:10.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:10 smithi131 ceph-mon[103713]: osdmap e138: 12 total, 12 up, 12 in 2023-09-10T01:08:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:10 smithi044 ceph-mon[98676]: osdmap e138: 12 total, 12 up, 12 in 2023-09-10T01:08:10.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:10 smithi099 ceph-mon[102692]: osdmap e138: 12 total, 12 up, 12 in 2023-09-10T01:08:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:11 smithi131 ceph-mon[103713]: pgmap v874: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 199 op/s 2023-09-10T01:08:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:11 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:08:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:11 smithi044 ceph-mon[98676]: pgmap v874: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 199 op/s 2023-09-10T01:08:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:11 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:08:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:11 smithi099 ceph-mon[102692]: pgmap v874: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 199 op/s 2023-09-10T01:08:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:11 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:08:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:13 smithi131 ceph-mon[103713]: pgmap v875: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 199 op/s 2023-09-10T01:08:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:13 smithi044 ceph-mon[98676]: pgmap v875: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 199 op/s 2023-09-10T01:08:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:13 smithi099 ceph-mon[102692]: pgmap v875: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 199 op/s 2023-09-10T01:08:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:15 smithi131 ceph-mon[103713]: pgmap v876: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 199 op/s 2023-09-10T01:08:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:15 smithi044 ceph-mon[98676]: pgmap v876: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 199 op/s 2023-09-10T01:08:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:15 smithi099 ceph-mon[102692]: pgmap v876: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 199 op/s 2023-09-10T01:08:16.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:16 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:08:16.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:16 smithi044 ceph-mon[98676]: pgmap v877: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:08:16.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:16 smithi044 ceph-mon[98676]: osdmap e139: 12 total, 12 up, 12 in 2023-09-10T01:08:16.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:16 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:08:16.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:16 smithi099 ceph-mon[102692]: pgmap v877: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:08:16.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:16 smithi099 ceph-mon[102692]: osdmap e139: 12 total, 12 up, 12 in 2023-09-10T01:08:16.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:16 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:08:16.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:16 smithi131 ceph-mon[103713]: pgmap v877: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:08:16.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:16 smithi131 ceph-mon[103713]: osdmap e139: 12 total, 12 up, 12 in 2023-09-10T01:08:18.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:18 smithi131 ceph-mon[103713]: pgmap v879: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2023-09-10T01:08:18.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:18 smithi044 ceph-mon[98676]: pgmap v879: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2023-09-10T01:08:18.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:18 smithi099 ceph-mon[102692]: pgmap v879: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2023-09-10T01:08:20.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:20 smithi131 ceph-mon[103713]: pgmap v880: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:08:20.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:20 smithi044 ceph-mon[98676]: pgmap v880: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:08:20.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:20 smithi099 ceph-mon[102692]: pgmap v880: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:08:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.781+0000 7f1dcc30b700 1 -- 172.21.15.44:0/2056459788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4106660 msgr2=0x7f1dc4106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.781+0000 7f1dcc30b700 1 --2- 172.21.15.44:0/2056459788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4106660 0x7f1dc4106a40 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00099c0 tx=0x7f1dc0009cd0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.781+0000 7f1dcc30b700 1 -- 172.21.15.44:0/2056459788 shutdown_connections 2023-09-10T01:08:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.781+0000 7f1dcc30b700 1 --2- 172.21.15.44:0/2056459788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc4100f60 0x7f1dc410d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.781+0000 7f1dcc30b700 1 --2- 172.21.15.44:0/2056459788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4106660 0x7f1dc4106a40 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.781+0000 7f1dcc30b700 1 --2- 172.21.15.44:0/2056459788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dc41005a0 0x7f1dc4100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.781+0000 7f1dcc30b700 1 -- 172.21.15.44:0/2056459788 >> 172.21.15.44:0/2056459788 conn(0x7f1dc40fc0b0 msgr2=0x7f1dc40fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.781+0000 7f1dcc30b700 1 -- 172.21.15.44:0/2056459788 shutdown_connections 2023-09-10T01:08:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.781+0000 7f1dcc30b700 1 -- 172.21.15.44:0/2056459788 wait complete. 2023-09-10T01:08:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.782+0000 7f1dcc30b700 1 Processor -- start 2023-09-10T01:08:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.782+0000 7f1dcc30b700 1 -- start start 2023-09-10T01:08:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.782+0000 7f1dcc30b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc41005a0 0x7f1dc419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.782+0000 7f1dcc30b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4100f60 0x7f1dc419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.782+0000 7f1dcc30b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dc4106660 0x7f1dc4196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dcc30b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc4196ad0 con 0x7f1dc4100f60 2023-09-10T01:08:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dcc30b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc4196c40 con 0x7f1dc4106660 2023-09-10T01:08:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dcc30b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc4196e00 con 0x7f1dc41005a0 2023-09-10T01:08:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dc98a6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4100f60 0x7f1dc419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dc98a6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4100f60 0x7f1dc419cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56124/0 (socket says 172.21.15.44:56124) 2023-09-10T01:08:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dc98a6700 1 -- 172.21.15.44:0/293811208 learned_addr learned my addr 172.21.15.44:0/293811208 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dca0a7700 1 --2- 172.21.15.44:0/293811208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc41005a0 0x7f1dc419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dca8a8700 1 --2- 172.21.15.44:0/293811208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dc4106660 0x7f1dc4196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dc98a6700 1 -- 172.21.15.44:0/293811208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc41005a0 msgr2=0x7f1dc419c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dc98a6700 1 --2- 172.21.15.44:0/293811208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc41005a0 0x7f1dc419c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dc98a6700 1 -- 172.21.15.44:0/293811208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dc4106660 msgr2=0x7f1dc4196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dc98a6700 1 --2- 172.21.15.44:0/293811208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dc4106660 0x7f1dc4196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dc98a6700 1 -- 172.21.15.44:0/293811208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc00095f0 con 0x7f1dc4100f60 2023-09-10T01:08:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dca8a8700 1 --2- 172.21.15.44:0/293811208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dc4106660 0x7f1dc4196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:08:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dc98a6700 1 --2- 172.21.15.44:0/293811208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4100f60 0x7f1dc419cb40 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1db400d950 tx=0x7f1db400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dbb7fe700 1 -- 172.21.15.44:0/293811208 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db4009940 con 0x7f1dc4100f60 2023-09-10T01:08:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dbb7fe700 1 -- 172.21.15.44:0/293811208 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1db4010460 con 0x7f1dc4100f60 2023-09-10T01:08:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dcc30b700 1 -- 172.21.15.44:0/293811208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc41970e0 con 0x7f1dc4100f60 2023-09-10T01:08:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dbb7fe700 1 -- 172.21.15.44:0/293811208 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db400f620 con 0x7f1dc4100f60 2023-09-10T01:08:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.783+0000 7f1dcc30b700 1 -- 172.21.15.44:0/293811208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc41a35f0 con 0x7f1dc4100f60 2023-09-10T01:08:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.786+0000 7f1dbb7fe700 1 -- 172.21.15.44:0/293811208 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1db40105d0 con 0x7f1dc4100f60 2023-09-10T01:08:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.786+0000 7f1dbb7fe700 1 --2- 172.21.15.44:0/293811208 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1db00a07b0 0x7f1db00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.786+0000 7f1dbb7fe700 1 -- 172.21.15.44:0/293811208 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1db40cc1f0 con 0x7f1dc4100f60 2023-09-10T01:08:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.786+0000 7f1dca0a7700 1 --2- 172.21.15.44:0/293811208 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1db00a07b0 0x7f1db00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.787+0000 7f1dca0a7700 1 --2- 172.21.15.44:0/293811208 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1db00a07b0 0x7f1db00a2c70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006010 tx=0x7f1dc0005c00 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.787+0000 7f1dd84d1b80 1 Processor -- start 2023-09-10T01:08:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.787+0000 7f1dd84d1b80 1 -- start start 2023-09-10T01:08:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dd84d1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a027785b0 0x557a02778990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dd84d1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a02772710 0x557a02772b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dd84d1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027730d0 0x557a0277f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dd84d1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a026c76d0 con 0x557a027730d0 2023-09-10T01:08:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dd84d1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a026c4ff0 con 0x557a027785b0 2023-09-10T01:08:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dd84d1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a026c08e0 con 0x557a02772710 2023-09-10T01:08:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dca8a8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027730d0 0x557a0277f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dc98a6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a02772710 0x557a02772b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dca8a8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027730d0 0x557a0277f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56134/0 (socket says 172.21.15.44:56134) 2023-09-10T01:08:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dca8a8700 1 -- 172.21.15.44:0/2409154935 learned_addr learned my addr 172.21.15.44:0/2409154935 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dca0a7700 1 --2- 172.21.15.44:0/2409154935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a027785b0 0x557a02778990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dca8a8700 1 -- 172.21.15.44:0/2409154935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a02772710 msgr2=0x557a02772b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dca8a8700 1 --2- 172.21.15.44:0/2409154935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a02772710 0x557a02772b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dca8a8700 1 -- 172.21.15.44:0/2409154935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a027785b0 msgr2=0x557a02778990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dca8a8700 1 --2- 172.21.15.44:0/2409154935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a027785b0 0x557a02778990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dca8a8700 1 -- 172.21.15.44:0/2409154935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a024ecd80 con 0x557a027730d0 2023-09-10T01:08:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.788+0000 7f1dca8a8700 1 --2- 172.21.15.44:0/2409154935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027730d0 0x557a0277f340 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc0093e0 tx=0x7f1dbc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de1273a9a2a516c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.789+0000 7f1db8d89700 1 -- 172.21.15.44:0/2409154935 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc00e070 con 0x557a027730d0 2023-09-10T01:08:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.789+0000 7f1db8d89700 1 -- 172.21.15.44:0/2409154935 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1dbc00f040 con 0x557a027730d0 2023-09-10T01:08:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.789+0000 7f1db8d89700 1 -- 172.21.15.44:0/2409154935 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc004800 con 0x557a027730d0 2023-09-10T01:08:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.789+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/2409154935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027730d0 msgr2=0x557a0277f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.789+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/2409154935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027730d0 0x557a0277f340 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc0093e0 tx=0x7f1dbc009920 comp rx=0 tx=0).stop 2023-09-10T01:08:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.789+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/2409154935 shutdown_connections 2023-09-10T01:08:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.789+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/2409154935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027730d0 0x557a0277f340 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.789+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/2409154935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a027785b0 0x557a02778990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.789+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/2409154935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a02772710 0x557a02772b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.790+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/2409154935 >> 172.21.15.44:0/2409154935 conn(0x557a026e3030 msgr2=0x557a02769560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.790+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/2409154935 shutdown_connections 2023-09-10T01:08:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.790+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/2409154935 wait complete. 2023-09-10T01:08:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.790+0000 7f1dd84d1b80 1 -- start start 2023-09-10T01:08:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.790+0000 7f1dd84d1b80 1 Processor -- start 2023-09-10T01:08:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.790+0000 7f1dd84d1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a02772710 0x557a0277efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.790+0000 7f1dd84d1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a027730d0 0x557a0277c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.790+0000 7f1dd84d1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027785b0 0x557a0277c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.790+0000 7f1dd84d1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0265eb10 con 0x557a027785b0 2023-09-10T01:08:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.790+0000 7f1dd84d1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0265e810 con 0x557a02772710 2023-09-10T01:08:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.790+0000 7f1dd84d1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a025428c0 con 0x557a027730d0 2023-09-10T01:08:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.790+0000 7f1dd84d1b80 10 client.0 _ll_register_callbacks cb 0x557a025a8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:08:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dca8a8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027785b0 0x557a0277c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dc98a6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a027730d0 0x557a0277c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dca8a8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027785b0 0x557a0277c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56140/0 (socket says 172.21.15.44:56140) 2023-09-10T01:08:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dca8a8700 1 -- 172.21.15.44:0/3867230422 learned_addr learned my addr 172.21.15.44:0/3867230422 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dc98a6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a027730d0 0x557a0277c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34788/0 (socket says 172.21.15.44:34788) 2023-09-10T01:08:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dca0a7700 1 --2- 172.21.15.44:0/3867230422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a02772710 0x557a0277efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dca8a8700 1 -- 172.21.15.44:0/3867230422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a027730d0 msgr2=0x557a0277c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dca8a8700 1 --2- 172.21.15.44:0/3867230422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a027730d0 0x557a0277c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dca8a8700 1 -- 172.21.15.44:0/3867230422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a02772710 msgr2=0x557a0277efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dca8a8700 1 --2- 172.21.15.44:0/3867230422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a02772710 0x557a0277efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dca8a8700 1 -- 172.21.15.44:0/3867230422 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dbc009090 con 0x557a027785b0 2023-09-10T01:08:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dca8a8700 1 --2- 172.21.15.44:0/3867230422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027785b0 0x557a0277c9e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc0093b0 tx=0x7f1dbc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683e1d836b1085af server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dadffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:08:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dadffb700 1 -- 172.21.15.44:0/3867230422 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc00e040 con 0x557a027785b0 2023-09-10T01:08:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dadffb700 1 -- 172.21.15.44:0/3867230422 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1dbc003c10 con 0x557a027785b0 2023-09-10T01:08:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/3867230422 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a02618490 con 0x557a027785b0 2023-09-10T01:08:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dadffb700 1 -- 172.21.15.44:0/3867230422 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dbc0095e0 con 0x557a027785b0 2023-09-10T01:08:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dd84d1b80 10 client.18906 fetch_fsmap learned FSMap version 111 2023-09-10T01:08:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.791+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/3867230422 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=4+}) v3 -- 0x557a024d5c00 con 0x557a027785b0 2023-09-10T01:08:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.792+0000 7f1dadffb700 1 -- 172.21.15.44:0/3867230422 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557a02618490 con 0x557a027785b0 2023-09-10T01:08:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.792+0000 7f1dadffb700 1 -- 172.21.15.44:0/3867230422 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 111) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1dbc015a90 con 0x557a027785b0 2023-09-10T01:08:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.792+0000 7f1dd84d1b80 10 client.18906 fetch_fsmap finished waiting for FSMap version 111 2023-09-10T01:08:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.792+0000 7f1dd84d1b80 10 client.18906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:08:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.792+0000 7f1dd84d1b80 20 client.18906 populate_metadata read hostname 'smithi044' 2023-09-10T01:08:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.792+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/3867230422 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557a027882e0 0x557a0278a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.792+0000 7f1dd84d1b80 4 client.18906 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:08:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.792+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/3867230422 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a026d5840 con 0x557a027882e0 2023-09-10T01:08:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.792+0000 7f1dca0a7700 1 --2- 172.21.15.44:0/3867230422 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557a027882e0 0x557a0278a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.793+0000 7f1dca0a7700 1 --2- 172.21.15.44:0/3867230422 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557a027882e0 0x557a0278a700 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fc60fb0894c573 server_cookie=3ea920e528b9b66a in_seq=0 out_seq=0 2023-09-10T01:08:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.793+0000 7f1dadffb700 10 client.18906 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:08:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.793+0000 7f1dadffb700 1 -- 172.21.15.44:0/3867230422 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557a026d5840 con 0x557a027882e0 2023-09-10T01:08:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.793+0000 7f1dadffb700 10 client.18906 handle_command_reply: tid=0 2023-09-10T01:08:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.793+0000 7f1dd84d1b80 1 client.18906 shutdown 2023-09-10T01:08:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.793+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/3867230422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027785b0 msgr2=0x557a0277c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.793+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/3867230422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027785b0 0x557a0277c9e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc0093b0 tx=0x7f1dbc0046b0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.793+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/3867230422 shutdown_connections 2023-09-10T01:08:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.793+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/3867230422 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557a027882e0 0x557a0278a700 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.793+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/3867230422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027785b0 0x557a0277c9e0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.793+0000 7f1dadffb700 0 client.18906 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:08:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.793+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/3867230422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a02772710 0x557a0277efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.793+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/3867230422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a027730d0 0x557a0277c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.793+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/3867230422 >> 172.21.15.44:0/3867230422 conn(0x557a026e3030 msgr2=0x557a027687a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.794+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/3867230422 shutdown_connections 2023-09-10T01:08:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.794+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/3867230422 wait complete. 2023-09-10T01:08:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.794+0000 7f1dd84d1b80 20 client.18906 trim_cache size 0 max 16384 2023-09-10T01:08:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.810+0000 7f1dd84d1b80 1 Processor -- start 2023-09-10T01:08:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.810+0000 7f1dd84d1b80 1 -- start start 2023-09-10T01:08:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dd84d1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a02772710 0x557a0277bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dd84d1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a027730d0 0x557a0277ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dd84d1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a027785b0 0x557a02777560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dd84d1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a02776f50 con 0x557a02772710 2023-09-10T01:08:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dd84d1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0276a330 con 0x557a027785b0 2023-09-10T01:08:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dd84d1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a027761c0 con 0x557a027730d0 2023-09-10T01:08:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dca0a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a02772710 0x557a0277bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dca8a8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a027785b0 0x557a02777560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dc98a6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a027730d0 0x557a0277ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dca0a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a02772710 0x557a0277bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56144/0 (socket says 172.21.15.44:56144) 2023-09-10T01:08:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dca0a7700 1 -- 172.21.15.44:0/1468158306 learned_addr learned my addr 172.21.15.44:0/1468158306 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dca0a7700 1 -- 172.21.15.44:0/1468158306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a027730d0 msgr2=0x557a0277ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dca0a7700 1 --2- 172.21.15.44:0/1468158306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a027730d0 0x557a0277ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dca0a7700 1 -- 172.21.15.44:0/1468158306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a027785b0 msgr2=0x557a02777560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dca0a7700 1 --2- 172.21.15.44:0/1468158306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a027785b0 0x557a02777560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dca0a7700 1 -- 172.21.15.44:0/1468158306 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a025131f0 con 0x557a02772710 2023-09-10T01:08:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dca8a8700 1 --2- 172.21.15.44:0/1468158306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a027785b0 0x557a02777560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:08:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.811+0000 7f1dca0a7700 1 --2- 172.21.15.44:0/1468158306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a02772710 0x557a0277bd90 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009fd0 tx=0x7f1dc0021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d6ad94e2e37b26 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.812+0000 7f1db958a700 1 -- 172.21.15.44:0/1468158306 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0028070 con 0x557a02772710 2023-09-10T01:08:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.812+0000 7f1db958a700 1 -- 172.21.15.44:0/1468158306 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1dc0029040 con 0x557a02772710 2023-09-10T01:08:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.812+0000 7f1db958a700 1 -- 172.21.15.44:0/1468158306 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0003cc0 con 0x557a02772710 2023-09-10T01:08:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.813+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/1468158306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a02772710 msgr2=0x557a0277bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.813+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/1468158306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a02772710 0x557a0277bd90 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009fd0 tx=0x7f1dc0021000 comp rx=0 tx=0).stop 2023-09-10T01:08:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.813+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/1468158306 shutdown_connections 2023-09-10T01:08:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.813+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/1468158306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a027785b0 0x557a02777560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.813+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/1468158306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a02772710 0x557a0277bd90 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.813+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/1468158306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a027730d0 0x557a0277ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.813+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/1468158306 >> 172.21.15.44:0/1468158306 conn(0x557a026e3030 msgr2=0x557a0278f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.813+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/1468158306 shutdown_connections 2023-09-10T01:08:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.813+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/1468158306 wait complete. 2023-09-10T01:08:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.813+0000 7f1dd84d1b80 1 -- start start 2023-09-10T01:08:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.813+0000 7f1dd84d1b80 1 Processor -- start 2023-09-10T01:08:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dd84d1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a02772710 0x557a02782fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dd84d1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027730d0 0x557a027834e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dd84d1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a027785b0 0x557a027db0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dd84d1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a027db630 con 0x557a027730d0 2023-09-10T01:08:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dd84d1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a027db770 con 0x557a02772710 2023-09-10T01:08:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dd84d1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a027db8b0 con 0x557a027785b0 2023-09-10T01:08:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dd84d1b80 10 client.0 _ll_register_callbacks cb 0x7f1dc000b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:08:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dc98a6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027730d0 0x557a027834e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dc98a6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027730d0 0x557a027834e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56152/0 (socket says 172.21.15.44:56152) 2023-09-10T01:08:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dc98a6700 1 -- 172.21.15.44:0/558280625 learned_addr learned my addr 172.21.15.44:0/558280625 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dca8a8700 1 --2- 172.21.15.44:0/558280625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a027785b0 0x557a027db0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dca0a7700 1 --2- 172.21.15.44:0/558280625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a02772710 0x557a02782fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dc98a6700 1 -- 172.21.15.44:0/558280625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a027785b0 msgr2=0x557a027db0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dc98a6700 1 --2- 172.21.15.44:0/558280625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a027785b0 0x557a027db0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dc98a6700 1 -- 172.21.15.44:0/558280625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a02772710 msgr2=0x557a02782fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dc98a6700 1 --2- 172.21.15.44:0/558280625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a02772710 0x557a02782fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dc98a6700 1 -- 172.21.15.44:0/558280625 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a024d5c00 con 0x557a027730d0 2023-09-10T01:08:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dca0a7700 1 --2- 172.21.15.44:0/558280625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a02772710 0x557a02782fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:08:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dc98a6700 1 --2- 172.21.15.44:0/558280625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027730d0 0x557a027834e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f1db4095d10 tx=0x7f1db400d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4617c4a86f7f56 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dadffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:08:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dadffb700 1 -- 172.21.15.44:0/558280625 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db40992a0 con 0x557a027730d0 2023-09-10T01:08:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dadffb700 1 -- 172.21.15.44:0/558280625 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1db409d0d0 con 0x557a027730d0 2023-09-10T01:08:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.814+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/558280625 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1dbc00f040 con 0x557a027730d0 2023-09-10T01:08:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.815+0000 7f1dadffb700 1 -- 172.21.15.44:0/558280625 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1db40a1310 con 0x557a027730d0 2023-09-10T01:08:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.815+0000 7f1dd84d1b80 10 client.18912 fetch_fsmap learned FSMap version 111 2023-09-10T01:08:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.815+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/558280625 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=4+}) v3 -- 0x7f1dc410dd20 con 0x557a027730d0 2023-09-10T01:08:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.815+0000 7f1dadffb700 1 -- 172.21.15.44:0/558280625 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1db4063030 con 0x557a027730d0 2023-09-10T01:08:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.815+0000 7f1dadffb700 1 -- 172.21.15.44:0/558280625 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 111) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1db409b040 con 0x557a027730d0 2023-09-10T01:08:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.815+0000 7f1dd84d1b80 10 client.18912 fetch_fsmap finished waiting for FSMap version 111 2023-09-10T01:08:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.815+0000 7f1dd84d1b80 10 client.18912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:08:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.815+0000 7f1dd84d1b80 20 client.18912 populate_metadata read hostname 'smithi044' 2023-09-10T01:08:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.815+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/558280625 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557a027dd0f0 0x557a027df510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.815+0000 7f1dd84d1b80 4 client.18912 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:08:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.815+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/558280625 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557a026b98c0 con 0x557a027dd0f0 2023-09-10T01:08:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.815+0000 7f1dca0a7700 1 --2- 172.21.15.44:0/558280625 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557a027dd0f0 0x557a027df510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.816+0000 7f1dadffb700 10 client.18912 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:08:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.816+0000 7f1dca0a7700 1 --2- 172.21.15.44:0/558280625 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557a027dd0f0 0x557a027df510 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55e984af546add7 server_cookie=5fe82a9886ea0434 in_seq=0 out_seq=0 2023-09-10T01:08:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.844+0000 7f1dadffb700 1 -- 172.21.15.44:0/558280625 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557a026b98c0 con 0x557a027dd0f0 2023-09-10T01:08:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.844+0000 7f1dadffb700 10 client.18912 handle_command_reply: tid=0 2023-09-10T01:08:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.844+0000 7f1dd84d1b80 1 client.18912 shutdown 2023-09-10T01:08:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.844+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/558280625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027730d0 msgr2=0x557a027834e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.844+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/558280625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027730d0 0x557a027834e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f1db4095d10 tx=0x7f1db400d8d0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.844+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/558280625 shutdown_connections 2023-09-10T01:08:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.844+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/558280625 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557a027dd0f0 0x557a027df510 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.844+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/558280625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a027785b0 0x557a027db0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.844+0000 7f1dadffb700 0 client.18912 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:08:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.844+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/558280625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a02772710 0x557a02782fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.844+0000 7f1dd84d1b80 1 --2- 172.21.15.44:0/558280625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a027730d0 0x557a027834e0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.844+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/558280625 >> 172.21.15.44:0/558280625 conn(0x557a026e3030 msgr2=0x557a0277bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.844+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/558280625 shutdown_connections 2023-09-10T01:08:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.844+0000 7f1dd84d1b80 1 -- 172.21.15.44:0/558280625 wait complete. 2023-09-10T01:08:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.844+0000 7f1dd84d1b80 20 client.18912 trim_cache size 0 max 16384 2023-09-10T01:08:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:08:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:08:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:08:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:08:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.845+0000 7f1dcc30b700 1 -- 172.21.15.44:0/293811208 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1db00a07b0 msgr2=0x7f1db00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.845+0000 7f1dcc30b700 1 --2- 172.21.15.44:0/293811208 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1db00a07b0 0x7f1db00a2c70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006010 tx=0x7f1dc0005c00 comp rx=0 tx=0).stop 2023-09-10T01:08:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.845+0000 7f1dcc30b700 1 -- 172.21.15.44:0/293811208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4100f60 msgr2=0x7f1dc419cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.845+0000 7f1dcc30b700 1 --2- 172.21.15.44:0/293811208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4100f60 0x7f1dc419cb40 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1db400d950 tx=0x7f1db400dc60 comp rx=0 tx=0).stop 2023-09-10T01:08:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.845+0000 7f1dcc30b700 1 -- 172.21.15.44:0/293811208 shutdown_connections 2023-09-10T01:08:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.845+0000 7f1dcc30b700 1 --2- 172.21.15.44:0/293811208 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1db00a07b0 0x7f1db00a2c70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.845+0000 7f1dcc30b700 1 --2- 172.21.15.44:0/293811208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dc4106660 0x7f1dc4196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.845+0000 7f1dcc30b700 1 --2- 172.21.15.44:0/293811208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc41005a0 0x7f1dc419c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.845+0000 7f1dcc30b700 1 --2- 172.21.15.44:0/293811208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dc4100f60 0x7f1dc419cb40 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.845+0000 7f1dcc30b700 1 -- 172.21.15.44:0/293811208 >> 172.21.15.44:0/293811208 conn(0x7f1dc40fc0b0 msgr2=0x7f1dc40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.845+0000 7f1dcc30b700 1 -- 172.21.15.44:0/293811208 shutdown_connections 2023-09-10T01:08:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:22.845+0000 7f1dcc30b700 1 -- 172.21.15.44:0/293811208 wait complete. 2023-09-10T01:08:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:22 smithi131 ceph-mon[103713]: pgmap v881: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2023-09-10T01:08:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:22 smithi044 ceph-mon[98676]: pgmap v881: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2023-09-10T01:08:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:22 smithi099 ceph-mon[102692]: pgmap v881: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2023-09-10T01:08:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:24 smithi131 ceph-mon[103713]: pgmap v882: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2023-09-10T01:08:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:24 smithi044 ceph-mon[98676]: pgmap v882: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2023-09-10T01:08:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:24 smithi099 ceph-mon[102692]: pgmap v882: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2023-09-10T01:08:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:26 smithi131 ceph-mon[103713]: pgmap v883: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2023-09-10T01:08:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:26 smithi044 ceph-mon[98676]: pgmap v883: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2023-09-10T01:08:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:26 smithi099 ceph-mon[102692]: pgmap v883: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2023-09-10T01:08:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:28 smithi131 ceph-mon[103713]: pgmap v884: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 999 KiB/s wr, 146 op/s 2023-09-10T01:08:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:28 smithi044 ceph-mon[98676]: pgmap v884: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 999 KiB/s wr, 146 op/s 2023-09-10T01:08:29.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:28 smithi099 ceph-mon[102692]: pgmap v884: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 999 KiB/s wr, 146 op/s 2023-09-10T01:08:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:30 smithi131 ceph-mon[103713]: pgmap v885: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 965 KiB/s wr, 141 op/s 2023-09-10T01:08:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:30 smithi044 ceph-mon[98676]: pgmap v885: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 965 KiB/s wr, 141 op/s 2023-09-10T01:08:31.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:30 smithi099 ceph-mon[102692]: pgmap v885: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 965 KiB/s wr, 141 op/s 2023-09-10T01:08:32.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:31 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:08:32.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:31 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:08:32.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:31 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:08:33.202 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:32 smithi099 ceph-mon[102692]: pgmap v886: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2023-09-10T01:08:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:32 smithi131 ceph-mon[103713]: pgmap v886: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2023-09-10T01:08:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:32 smithi044 ceph-mon[98676]: pgmap v886: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2023-09-10T01:08:35.264 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:34 smithi099 ceph-mon[102692]: pgmap v887: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 949 KiB/s wr, 138 op/s 2023-09-10T01:08:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:34 smithi131 ceph-mon[103713]: pgmap v887: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 949 KiB/s wr, 138 op/s 2023-09-10T01:08:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:34 smithi044 ceph-mon[98676]: pgmap v887: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 949 KiB/s wr, 138 op/s 2023-09-10T01:08:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.976+0000 7f17cb792700 1 -- 172.21.15.44:0/3303326986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17c4068920 msgr2=0x7f17c4068da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.976+0000 7f17cb792700 1 --2- 172.21.15.44:0/3303326986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17c4068920 0x7f17c4068da0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009990 tx=0x7f17b4009ca0 comp rx=0 tx=0).stop 2023-09-10T01:08:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.976+0000 7f17cb792700 1 -- 172.21.15.44:0/3303326986 shutdown_connections 2023-09-10T01:08:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.976+0000 7f17cb792700 1 --2- 172.21.15.44:0/3303326986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17c40692e0 0x7f17c410d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.976+0000 7f17cb792700 1 --2- 172.21.15.44:0/3303326986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17c4106620 0x7f17c4106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.976+0000 7f17cb792700 1 --2- 172.21.15.44:0/3303326986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17c4068920 0x7f17c4068da0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.976+0000 7f17cb792700 1 -- 172.21.15.44:0/3303326986 >> 172.21.15.44:0/3303326986 conn(0x7f17c4076600 msgr2=0x7f17c4076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.976+0000 7f17cb792700 1 -- 172.21.15.44:0/3303326986 shutdown_connections 2023-09-10T01:08:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.976+0000 7f17cb792700 1 -- 172.21.15.44:0/3303326986 wait complete. 2023-09-10T01:08:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.977+0000 7f17cb792700 1 Processor -- start 2023-09-10T01:08:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.977+0000 7f17cb792700 1 -- start start 2023-09-10T01:08:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.977+0000 7f17cb792700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17c4068920 0x7f17c419c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.977+0000 7f17cb792700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17c40692e0 0x7f17c419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.977+0000 7f17cb792700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17c4106620 0x7f17c4196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.977+0000 7f17cb792700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c4196ab0 con 0x7f17c4068920 2023-09-10T01:08:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.977+0000 7f17cb792700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c4196c20 con 0x7f17c4106620 2023-09-10T01:08:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.977+0000 7f17cb792700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c4196de0 con 0x7f17c40692e0 2023-09-10T01:08:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17c952e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17c4068920 0x7f17c419c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17c8d2d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17c40692e0 0x7f17c419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17c9d2f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17c4106620 0x7f17c4196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17c9d2f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17c4106620 0x7f17c4196440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57900/0 (socket says 172.21.15.44:57900) 2023-09-10T01:08:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17c8d2d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17c40692e0 0x7f17c419cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34812/0 (socket says 172.21.15.44:34812) 2023-09-10T01:08:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17c9d2f700 1 -- 172.21.15.44:0/2873654166 learned_addr learned my addr 172.21.15.44:0/2873654166 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17c9d2f700 1 -- 172.21.15.44:0/2873654166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17c40692e0 msgr2=0x7f17c419cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17c9d2f700 1 --2- 172.21.15.44:0/2873654166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17c40692e0 0x7f17c419cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17c9d2f700 1 -- 172.21.15.44:0/2873654166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17c4068920 msgr2=0x7f17c419c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17c9d2f700 1 --2- 172.21.15.44:0/2873654166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17c4068920 0x7f17c419c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17c9d2f700 1 -- 172.21.15.44:0/2873654166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b40095f0 con 0x7f17c4106620 2023-09-10T01:08:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17c9d2f700 1 --2- 172.21.15.44:0/2873654166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17c4106620 0x7f17c4196440 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00ef30 tx=0x7f17bc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17ba7fc700 1 -- 172.21.15.44:0/2873654166 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc018070 con 0x7f17c4106620 2023-09-10T01:08:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17cb792700 1 -- 172.21.15.44:0/2873654166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c4197060 con 0x7f17c4106620 2023-09-10T01:08:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17ba7fc700 1 -- 172.21.15.44:0/2873654166 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17bc00f040 con 0x7f17c4106620 2023-09-10T01:08:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17ba7fc700 1 -- 172.21.15.44:0/2873654166 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00cbc0 con 0x7f17c4106620 2023-09-10T01:08:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.978+0000 7f17cb792700 1 -- 172.21.15.44:0/2873654166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c41a35d0 con 0x7f17c4106620 2023-09-10T01:08:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.981+0000 7f17ba7fc700 1 -- 172.21.15.44:0/2873654166 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f17bc007500 con 0x7f17c4106620 2023-09-10T01:08:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.982+0000 7f17ba7fc700 1 --2- 172.21.15.44:0/2873654166 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f17b00a07b0 0x7f17b00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.982+0000 7f17ba7fc700 1 -- 172.21.15.44:0/2873654166 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f17bc0cbe70 con 0x7f17c4106620 2023-09-10T01:08:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.982+0000 7f17c952e700 1 --2- 172.21.15.44:0/2873654166 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f17b00a07b0 0x7f17b00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.982+0000 7f17d7958b80 1 Processor -- start 2023-09-10T01:08:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.982+0000 7f17d7958b80 1 -- start start 2023-09-10T01:08:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17d7958b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b58e1a0 0x55d99b58e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17d7958b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588300 0x55d99b588780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17d7958b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b588cc0 0x55d99b594f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17d7958b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99b4e16d0 con 0x55d99b58e1a0 2023-09-10T01:08:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17d7958b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99b4deff0 con 0x55d99b588300 2023-09-10T01:08:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17d7958b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99b4da8e0 con 0x55d99b588cc0 2023-09-10T01:08:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17c8d2d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588300 0x55d99b588780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17c8d2d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588300 0x55d99b588780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57910/0 (socket says 172.21.15.44:57910) 2023-09-10T01:08:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17c952e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b58e1a0 0x55d99b58e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17c8d2d700 1 -- 172.21.15.44:0/818460280 learned_addr learned my addr 172.21.15.44:0/818460280 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17c9d2f700 1 --2- 172.21.15.44:0/818460280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b588cc0 0x55d99b594f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17c952e700 1 --2- 172.21.15.44:0/2873654166 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f17b00a07b0 0x7f17b00a2c70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009c30 tx=0x7f17c0011440 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17c8d2d700 1 -- 172.21.15.44:0/818460280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b588cc0 msgr2=0x55d99b594f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17c8d2d700 1 --2- 172.21.15.44:0/818460280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b588cc0 0x55d99b594f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17c8d2d700 1 -- 172.21.15.44:0/818460280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b58e1a0 msgr2=0x55d99b58e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17c8d2d700 1 --2- 172.21.15.44:0/818460280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b58e1a0 0x55d99b58e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17c8d2d700 1 -- 172.21.15.44:0/818460280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d99b306d80 con 0x55d99b588300 2023-09-10T01:08:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17c952e700 1 --2- 172.21.15.44:0/818460280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b58e1a0 0x55d99b58e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:08:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.983+0000 7f17c8d2d700 1 --2- 172.21.15.44:0/818460280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588300 0x55d99b588780 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f17b4000c00 tx=0x7f17b4005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fab19b83d9ef5371 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.984+0000 7f17af7fe700 1 -- 172.21.15.44:0/818460280 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b401d070 con 0x55d99b588300 2023-09-10T01:08:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.984+0000 7f17af7fe700 1 -- 172.21.15.44:0/818460280 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17b401e040 con 0x55d99b588300 2023-09-10T01:08:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.984+0000 7f17af7fe700 1 -- 172.21.15.44:0/818460280 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b4004720 con 0x55d99b588300 2023-09-10T01:08:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.984+0000 7f17d7958b80 1 -- 172.21.15.44:0/818460280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588300 msgr2=0x55d99b588780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.984+0000 7f17d7958b80 1 --2- 172.21.15.44:0/818460280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588300 0x55d99b588780 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f17b4000c00 tx=0x7f17b4005bc0 comp rx=0 tx=0).stop 2023-09-10T01:08:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.984+0000 7f17d7958b80 1 -- 172.21.15.44:0/818460280 shutdown_connections 2023-09-10T01:08:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.984+0000 7f17d7958b80 1 --2- 172.21.15.44:0/818460280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b588cc0 0x55d99b594f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.984+0000 7f17d7958b80 1 --2- 172.21.15.44:0/818460280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b58e1a0 0x55d99b58e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.984+0000 7f17d7958b80 1 --2- 172.21.15.44:0/818460280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588300 0x55d99b588780 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.985+0000 7f17d7958b80 1 -- 172.21.15.44:0/818460280 >> 172.21.15.44:0/818460280 conn(0x55d99b4fd030 msgr2=0x55d99b583560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.985+0000 7f17d7958b80 1 -- 172.21.15.44:0/818460280 shutdown_connections 2023-09-10T01:08:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.985+0000 7f17d7958b80 1 -- 172.21.15.44:0/818460280 wait complete. 2023-09-10T01:08:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.985+0000 7f17d7958b80 1 -- start start 2023-09-10T01:08:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.985+0000 7f17d7958b80 1 Processor -- start 2023-09-10T01:08:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17d7958b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b594e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17d7958b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588cc0 0x55d99b592110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17d7958b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b58e1a0 0x55d99b592740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17d7958b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99b478b10 con 0x55d99b588300 2023-09-10T01:08:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17d7958b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99b478810 con 0x55d99b588cc0 2023-09-10T01:08:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17d7958b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99b35c8c0 con 0x55d99b58e1a0 2023-09-10T01:08:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17d7958b80 10 client.0 _ll_register_callbacks cb 0x55d99b3c2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:08:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17c9d2f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b58e1a0 0x55d99b592740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17c952e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b594e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17c9d2f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b58e1a0 0x55d99b592740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34828/0 (socket says 172.21.15.44:34828) 2023-09-10T01:08:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17c9d2f700 1 -- 172.21.15.44:0/658034407 learned_addr learned my addr 172.21.15.44:0/658034407 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17c8d2d700 1 --2- 172.21.15.44:0/658034407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588cc0 0x55d99b592110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17c9d2f700 1 -- 172.21.15.44:0/658034407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588cc0 msgr2=0x55d99b592110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17c9d2f700 1 --2- 172.21.15.44:0/658034407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588cc0 0x55d99b592110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17c9d2f700 1 -- 172.21.15.44:0/658034407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 msgr2=0x55d99b594e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17c9d2f700 1 --2- 172.21.15.44:0/658034407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b594e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17c9d2f700 1 -- 172.21.15.44:0/658034407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c0011b20 con 0x55d99b58e1a0 2023-09-10T01:08:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.986+0000 7f17c952e700 1 --2- 172.21.15.44:0/658034407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b594e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:08:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.987+0000 7f17c9d2f700 1 --2- 172.21.15.44:0/658034407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b58e1a0 0x55d99b592740 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00cdd0 tx=0x7f17bc098a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=753e09a422df75e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.987+0000 7f17acff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:08:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.987+0000 7f17acff9700 1 -- 172.21.15.44:0/658034407 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc098fb0 con 0x55d99b58e1a0 2023-09-10T01:08:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.987+0000 7f17acff9700 1 -- 172.21.15.44:0/658034407 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f17bc09cde0 con 0x55d99b58e1a0 2023-09-10T01:08:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.987+0000 7f17d7958b80 1 -- 172.21.15.44:0/658034407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d99b432490 con 0x55d99b58e1a0 2023-09-10T01:08:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.987+0000 7f17acff9700 1 -- 172.21.15.44:0/658034407 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17bc0a0f90 con 0x55d99b58e1a0 2023-09-10T01:08:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.987+0000 7f17d7958b80 10 client.28528 fetch_fsmap learned FSMap version 111 2023-09-10T01:08:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.987+0000 7f17d7958b80 1 -- 172.21.15.44:0/658034407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=4+}) v3 -- 0x7f17c410dd30 con 0x55d99b58e1a0 2023-09-10T01:08:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.987+0000 7f17acff9700 1 -- 172.21.15.44:0/658034407 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d99b432490 con 0x55d99b58e1a0 2023-09-10T01:08:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.987+0000 7f17acff9700 1 -- 172.21.15.44:0/658034407 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 111) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f17bc0a0b00 con 0x55d99b58e1a0 2023-09-10T01:08:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.987+0000 7f17d7958b80 10 client.28528 fetch_fsmap finished waiting for FSMap version 111 2023-09-10T01:08:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.987+0000 7f17d7958b80 10 client.28528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:08:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.987+0000 7f17d7958b80 20 client.28528 populate_metadata read hostname 'smithi044' 2023-09-10T01:08:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.987+0000 7f17d7958b80 1 --2- 172.21.15.44:0/658034407 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d99b59c3c0 0x55d99b59e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.988+0000 7f17d7958b80 4 client.28528 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:08:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.988+0000 7f17d7958b80 1 -- 172.21.15.44:0/658034407 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d99b4ef840 con 0x55d99b59c3c0 2023-09-10T01:08:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.988+0000 7f17c952e700 1 --2- 172.21.15.44:0/658034407 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d99b59c3c0 0x55d99b59e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.988+0000 7f17acff9700 10 client.28528 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:08:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.988+0000 7f17c952e700 1 --2- 172.21.15.44:0/658034407 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d99b59c3c0 0x55d99b59e7b0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17fec817d00a7934 server_cookie=311e06b7101e4b94 in_seq=0 out_seq=0 2023-09-10T01:08:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.989+0000 7f17acff9700 1 -- 172.21.15.44:0/658034407 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d99b4ef840 con 0x55d99b59c3c0 2023-09-10T01:08:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.989+0000 7f17acff9700 10 client.28528 handle_command_reply: tid=0 2023-09-10T01:08:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.989+0000 7f17d7958b80 1 client.28528 shutdown 2023-09-10T01:08:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.989+0000 7f17d7958b80 1 -- 172.21.15.44:0/658034407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b58e1a0 msgr2=0x55d99b592740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.989+0000 7f17d7958b80 1 --2- 172.21.15.44:0/658034407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b58e1a0 0x55d99b592740 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00cdd0 tx=0x7f17bc098a00 comp rx=0 tx=0).stop 2023-09-10T01:08:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.989+0000 7f17d7958b80 1 -- 172.21.15.44:0/658034407 shutdown_connections 2023-09-10T01:08:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.989+0000 7f17d7958b80 1 --2- 172.21.15.44:0/658034407 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d99b59c3c0 0x55d99b59e7b0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.989+0000 7f17d7958b80 1 --2- 172.21.15.44:0/658034407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b58e1a0 0x55d99b592740 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.989+0000 7f17d7958b80 1 --2- 172.21.15.44:0/658034407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b594e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.989+0000 7f17acff9700 0 client.28528 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:08:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.989+0000 7f17d7958b80 1 --2- 172.21.15.44:0/658034407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588cc0 0x55d99b592110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.989+0000 7f17d7958b80 1 -- 172.21.15.44:0/658034407 >> 172.21.15.44:0/658034407 conn(0x55d99b4fd030 msgr2=0x55d99b5827a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.989+0000 7f17d7958b80 1 -- 172.21.15.44:0/658034407 shutdown_connections 2023-09-10T01:08:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.989+0000 7f17d7958b80 1 -- 172.21.15.44:0/658034407 wait complete. 2023-09-10T01:08:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:35.989+0000 7f17d7958b80 20 client.28528 trim_cache size 0 max 16384 2023-09-10T01:08:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.005+0000 7f17d7958b80 1 Processor -- start 2023-09-10T01:08:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.005+0000 7f17d7958b80 1 -- start start 2023-09-10T01:08:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17d7958b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b58fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17d7958b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588cc0 0x55d99b5a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17d7958b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b58e1a0 0x55d99b5918f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17d7958b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99b58daa0 con 0x55d99b588300 2023-09-10T01:08:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17d7958b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99b5a3620 con 0x55d99b588cc0 2023-09-10T01:08:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17d7958b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99b590690 con 0x55d99b58e1a0 2023-09-10T01:08:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17c952e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b58fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17c8d2d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588cc0 0x55d99b5a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17c952e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b58fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56182/0 (socket says 172.21.15.44:56182) 2023-09-10T01:08:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17c952e700 1 -- 172.21.15.44:0/2778378793 learned_addr learned my addr 172.21.15.44:0/2778378793 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17c9d2f700 1 --2- 172.21.15.44:0/2778378793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b58e1a0 0x55d99b5918f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17c8d2d700 1 -- 172.21.15.44:0/2778378793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b58e1a0 msgr2=0x55d99b5918f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17c8d2d700 1 --2- 172.21.15.44:0/2778378793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b58e1a0 0x55d99b5918f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17c8d2d700 1 -- 172.21.15.44:0/2778378793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 msgr2=0x55d99b58fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17c8d2d700 1 --2- 172.21.15.44:0/2778378793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b58fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17c8d2d700 1 -- 172.21.15.44:0/2778378793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d99b2efc00 con 0x55d99b588cc0 2023-09-10T01:08:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17c9d2f700 1 --2- 172.21.15.44:0/2778378793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b58e1a0 0x55d99b5918f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:08:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.006+0000 7f17c952e700 1 --2- 172.21.15.44:0/2778378793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b58fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:08:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.007+0000 7f17c8d2d700 1 --2- 172.21.15.44:0/2778378793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588cc0 0x55d99b5a30e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009990 tx=0x7f17b4005a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd433c8bdd6e97eb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.007+0000 7f17affff700 1 -- 172.21.15.44:0/2778378793 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b401d040 con 0x55d99b588cc0 2023-09-10T01:08:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.007+0000 7f17affff700 1 -- 172.21.15.44:0/2778378793 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17b4031070 con 0x55d99b588cc0 2023-09-10T01:08:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.007+0000 7f17affff700 1 -- 172.21.15.44:0/2778378793 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b4004950 con 0x55d99b588cc0 2023-09-10T01:08:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.008+0000 7f17d7958b80 1 -- 172.21.15.44:0/2778378793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588cc0 msgr2=0x55d99b5a30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.008+0000 7f17d7958b80 1 --2- 172.21.15.44:0/2778378793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588cc0 0x55d99b5a30e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009990 tx=0x7f17b4005a50 comp rx=0 tx=0).stop 2023-09-10T01:08:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.008+0000 7f17d7958b80 1 -- 172.21.15.44:0/2778378793 shutdown_connections 2023-09-10T01:08:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.008+0000 7f17d7958b80 1 --2- 172.21.15.44:0/2778378793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b58e1a0 0x55d99b5918f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.008+0000 7f17d7958b80 1 --2- 172.21.15.44:0/2778378793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b58fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.008+0000 7f17d7958b80 1 --2- 172.21.15.44:0/2778378793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b588cc0 0x55d99b5a30e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.008+0000 7f17d7958b80 1 -- 172.21.15.44:0/2778378793 >> 172.21.15.44:0/2778378793 conn(0x55d99b4fd030 msgr2=0x55d99b58c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.008+0000 7f17d7958b80 1 -- 172.21.15.44:0/2778378793 shutdown_connections 2023-09-10T01:08:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.008+0000 7f17d7958b80 1 -- 172.21.15.44:0/2778378793 wait complete. 2023-09-10T01:08:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.008+0000 7f17d7958b80 1 -- start start 2023-09-10T01:08:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.008+0000 7f17d7958b80 1 Processor -- start 2023-09-10T01:08:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.008+0000 7f17d7958b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b58ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17d7958b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b588cc0 0x55d99b58b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17d7958b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b58e1a0 0x55d99b5939b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17d7958b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99b5a3620 con 0x55d99b588300 2023-09-10T01:08:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17d7958b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99b593f20 con 0x55d99b58e1a0 2023-09-10T01:08:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17d7958b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99b5940e0 con 0x55d99b588cc0 2023-09-10T01:08:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17d7958b80 10 client.0 _ll_register_callbacks cb 0x7f17bc098e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:08:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17c952e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b58ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17c952e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b58ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56188/0 (socket says 172.21.15.44:56188) 2023-09-10T01:08:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17c9d2f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b58e1a0 0x55d99b5939b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17c952e700 1 -- 172.21.15.44:0/1043973452 learned_addr learned my addr 172.21.15.44:0/1043973452 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17c8d2d700 1 --2- 172.21.15.44:0/1043973452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b588cc0 0x55d99b58b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17c9d2f700 1 -- 172.21.15.44:0/1043973452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b588cc0 msgr2=0x55d99b58b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17c9d2f700 1 --2- 172.21.15.44:0/1043973452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b588cc0 0x55d99b58b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17c9d2f700 1 -- 172.21.15.44:0/1043973452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 msgr2=0x55d99b58ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17c9d2f700 1 --2- 172.21.15.44:0/1043973452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b58ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17c9d2f700 1 -- 172.21.15.44:0/1043973452 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c000fa80 con 0x55d99b58e1a0 2023-09-10T01:08:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17c952e700 1 --2- 172.21.15.44:0/1043973452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b58ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:08:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17c8d2d700 1 --2- 172.21.15.44:0/1043973452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b588cc0 0x55d99b58b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:08:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17c9d2f700 1 --2- 172.21.15.44:0/1043973452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b58e1a0 0x55d99b5939b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00ef00 tx=0x7f17bc012480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e26c664cebb27200 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17acff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:08:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17acff9700 1 -- 172.21.15.44:0/1043973452 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc09cde0 con 0x55d99b58e1a0 2023-09-10T01:08:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17d7958b80 1 -- 172.21.15.44:0/1043973452 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17b401e040 con 0x55d99b58e1a0 2023-09-10T01:08:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.009+0000 7f17acff9700 1 -- 172.21.15.44:0/1043973452 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f17bc098fb0 con 0x55d99b58e1a0 2023-09-10T01:08:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.010+0000 7f17acff9700 1 -- 172.21.15.44:0/1043973452 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17bc0a0f90 con 0x55d99b58e1a0 2023-09-10T01:08:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.010+0000 7f17d7958b80 10 client.28592 fetch_fsmap learned FSMap version 111 2023-09-10T01:08:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.010+0000 7f17d7958b80 1 -- 172.21.15.44:0/1043973452 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=4+}) v3 -- 0x55d99b306d80 con 0x55d99b58e1a0 2023-09-10T01:08:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.010+0000 7f17acff9700 1 -- 172.21.15.44:0/1043973452 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17b401e040 con 0x55d99b58e1a0 2023-09-10T01:08:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.010+0000 7f17acff9700 1 -- 172.21.15.44:0/1043973452 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 111) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f17bc061070 con 0x55d99b58e1a0 2023-09-10T01:08:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.010+0000 7f17d7958b80 10 client.28592 fetch_fsmap finished waiting for FSMap version 111 2023-09-10T01:08:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.010+0000 7f17d7958b80 10 client.28592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:08:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.010+0000 7f17d7958b80 20 client.28592 populate_metadata read hostname 'smithi044' 2023-09-10T01:08:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.010+0000 7f17d7958b80 1 --2- 172.21.15.44:0/1043973452 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d99b607d20 0x55d99b60a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.010+0000 7f17d7958b80 4 client.28592 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:08:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.010+0000 7f17d7958b80 1 -- 172.21.15.44:0/1043973452 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d99b4d38c0 con 0x55d99b607d20 2023-09-10T01:08:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.010+0000 7f17c952e700 1 --2- 172.21.15.44:0/1043973452 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d99b607d20 0x55d99b60a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.011+0000 7f17acff9700 10 client.28592 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:08:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.011+0000 7f17c952e700 1 --2- 172.21.15.44:0/1043973452 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d99b607d20 0x55d99b60a140 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48f922e394dfd1b6 server_cookie=c19c523d754853b7 in_seq=0 out_seq=0 2023-09-10T01:08:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.023+0000 7f17acff9700 1 -- 172.21.15.44:0/1043973452 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d99b4d38c0 con 0x55d99b607d20 2023-09-10T01:08:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.023+0000 7f17acff9700 10 client.28592 handle_command_reply: tid=0 2023-09-10T01:08:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.023+0000 7f17d7958b80 1 client.28592 shutdown 2023-09-10T01:08:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17d7958b80 1 -- 172.21.15.44:0/1043973452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b58e1a0 msgr2=0x55d99b5939b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17d7958b80 1 --2- 172.21.15.44:0/1043973452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b58e1a0 0x55d99b5939b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00ef00 tx=0x7f17bc012480 comp rx=0 tx=0).stop 2023-09-10T01:08:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17d7958b80 1 -- 172.21.15.44:0/1043973452 shutdown_connections 2023-09-10T01:08:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17d7958b80 1 --2- 172.21.15.44:0/1043973452 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d99b607d20 0x55d99b60a140 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17d7958b80 1 --2- 172.21.15.44:0/1043973452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99b58e1a0 0x55d99b5939b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17acff9700 0 client.28592 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:08:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17d7958b80 1 --2- 172.21.15.44:0/1043973452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99b588300 0x55d99b58ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17d7958b80 1 --2- 172.21.15.44:0/1043973452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d99b588cc0 0x55d99b58b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17d7958b80 1 -- 172.21.15.44:0/1043973452 >> 172.21.15.44:0/1043973452 conn(0x55d99b4fd030 msgr2=0x55d99b5920d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17d7958b80 1 -- 172.21.15.44:0/1043973452 shutdown_connections 2023-09-10T01:08:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17d7958b80 1 -- 172.21.15.44:0/1043973452 wait complete. 2023-09-10T01:08:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17d7958b80 20 client.28592 trim_cache size 0 max 16384 2023-09-10T01:08:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:08:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:08:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:08:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:08:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17cb792700 1 -- 172.21.15.44:0/2873654166 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f17b00a07b0 msgr2=0x7f17b00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17cb792700 1 --2- 172.21.15.44:0/2873654166 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f17b00a07b0 0x7f17b00a2c70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009c30 tx=0x7f17c0011440 comp rx=0 tx=0).stop 2023-09-10T01:08:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17cb792700 1 -- 172.21.15.44:0/2873654166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17c4106620 msgr2=0x7f17c4196440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17cb792700 1 --2- 172.21.15.44:0/2873654166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17c4106620 0x7f17c4196440 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00ef30 tx=0x7f17bc00c3a0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17cb792700 1 -- 172.21.15.44:0/2873654166 shutdown_connections 2023-09-10T01:08:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17cb792700 1 --2- 172.21.15.44:0/2873654166 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f17b00a07b0 0x7f17b00a2c70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17cb792700 1 --2- 172.21.15.44:0/2873654166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17c4106620 0x7f17c4196440 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17cb792700 1 --2- 172.21.15.44:0/2873654166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17c4068920 0x7f17c419c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17cb792700 1 --2- 172.21.15.44:0/2873654166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17c40692e0 0x7f17c419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.024+0000 7f17cb792700 1 -- 172.21.15.44:0/2873654166 >> 172.21.15.44:0/2873654166 conn(0x7f17c4076600 msgr2=0x7f17c40fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.025+0000 7f17cb792700 1 -- 172.21.15.44:0/2873654166 shutdown_connections 2023-09-10T01:08:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:36.025+0000 7f17cb792700 1 -- 172.21.15.44:0/2873654166 wait complete. 2023-09-10T01:08:36.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:36 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:08:36.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:36 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:08:36.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:36 smithi099 ceph-mon[102692]: pgmap v888: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 200 op/s 2023-09-10T01:08:36.652 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:36 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:08:36.652 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:36 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:08:36.652 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:36 smithi131 ceph-mon[103713]: pgmap v888: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 200 op/s 2023-09-10T01:08:36.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:36 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:08:36.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:36 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:08:36.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:36 smithi044 ceph-mon[98676]: pgmap v888: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 200 op/s 2023-09-10T01:08:38.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:38 smithi131 ceph-mon[103713]: pgmap v889: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 888 KiB/s wr, 127 op/s 2023-09-10T01:08:38.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:38 smithi044 ceph-mon[98676]: pgmap v889: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 888 KiB/s wr, 127 op/s 2023-09-10T01:08:38.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:38 smithi099 ceph-mon[102692]: pgmap v889: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 888 KiB/s wr, 127 op/s 2023-09-10T01:08:40.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:40 smithi131 ceph-mon[103713]: pgmap v890: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 889 KiB/s wr, 127 op/s 2023-09-10T01:08:40.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:40 smithi044 ceph-mon[98676]: pgmap v890: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 889 KiB/s wr, 127 op/s 2023-09-10T01:08:40.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:40 smithi099 ceph-mon[102692]: pgmap v890: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 889 KiB/s wr, 127 op/s 2023-09-10T01:08:42.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:42 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:08:42.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:42 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:08:42.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:42 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:08:42.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:42 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:08:42.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:42 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:08:42.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:42 smithi131 ceph-mon[103713]: pgmap v891: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-09-10T01:08:42.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:42 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:08:42.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:42 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:08:42.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:42 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:08:42.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:42 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:08:42.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:42 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:08:42.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:42 smithi044 ceph-mon[98676]: pgmap v891: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-09-10T01:08:42.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:42 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:08:42.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:42 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:08:42.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:42 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:08:42.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:42 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:08:42.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:42 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:08:42.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:42 smithi099 ceph-mon[102692]: pgmap v891: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-09-10T01:08:44.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:44 smithi131 ceph-mon[103713]: pgmap v892: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 807 KiB/s wr, 118 op/s 2023-09-10T01:08:44.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:44 smithi044 ceph-mon[98676]: pgmap v892: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 807 KiB/s wr, 118 op/s 2023-09-10T01:08:44.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:44 smithi099 ceph-mon[102692]: pgmap v892: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 807 KiB/s wr, 118 op/s 2023-09-10T01:08:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:46 smithi131 ceph-mon[103713]: pgmap v893: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 198 op/s 2023-09-10T01:08:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:46 smithi044 ceph-mon[98676]: pgmap v893: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 198 op/s 2023-09-10T01:08:47.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:46 smithi099 ceph-mon[102692]: pgmap v893: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 198 op/s 2023-09-10T01:08:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.146+0000 7f0c0822c700 1 -- 172.21.15.44:0/3246626395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c000ffe70 msgr2=0x7f0c0010c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.146+0000 7f0c0822c700 1 --2- 172.21.15.44:0/3246626395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c000ffe70 0x7f0c0010c380 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800b0a0 tx=0x7f0bf800b3b0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.147+0000 7f0c0822c700 1 -- 172.21.15.44:0/3246626395 shutdown_connections 2023-09-10T01:08:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.147+0000 7f0c0822c700 1 --2- 172.21.15.44:0/3246626395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c000ffe70 0x7f0c0010c380 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.147+0000 7f0c0822c700 1 --2- 172.21.15.44:0/3246626395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c001044d0 0x7f0c001048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.147+0000 7f0c0822c700 1 --2- 172.21.15.44:0/3246626395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c000ff4b0 0x7f0c000ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.147+0000 7f0c0822c700 1 -- 172.21.15.44:0/3246626395 >> 172.21.15.44:0/3246626395 conn(0x7f0c00076600 msgr2=0x7f0c00076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.147+0000 7f0c0822c700 1 -- 172.21.15.44:0/3246626395 shutdown_connections 2023-09-10T01:08:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.147+0000 7f0c0822c700 1 -- 172.21.15.44:0/3246626395 wait complete. 2023-09-10T01:08:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.147+0000 7f0c0822c700 1 Processor -- start 2023-09-10T01:08:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.147+0000 7f0c0822c700 1 -- start start 2023-09-10T01:08:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c0822c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c000ff4b0 0x7f0c00197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c0822c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c000ffe70 0x7f0c00197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c0822c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c001044d0 0x7f0c001a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c0822c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c001a2af0 con 0x7f0c001044d0 2023-09-10T01:08:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c0822c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c001a2c60 con 0x7f0c000ffe70 2023-09-10T01:08:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c0822c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c001a2e20 con 0x7f0c000ff4b0 2023-09-10T01:08:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c067c9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c001044d0 0x7f0c001a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c05fc8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c000ff4b0 0x7f0c00197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c057c7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c000ffe70 0x7f0c00197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c067c9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c001044d0 0x7f0c001a2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56206/0 (socket says 172.21.15.44:56206) 2023-09-10T01:08:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c05fc8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c000ff4b0 0x7f0c00197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34852/0 (socket says 172.21.15.44:34852) 2023-09-10T01:08:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c057c7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c000ffe70 0x7f0c00197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57944/0 (socket says 172.21.15.44:57944) 2023-09-10T01:08:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c067c9700 1 -- 172.21.15.44:0/4191108460 learned_addr learned my addr 172.21.15.44:0/4191108460 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c067c9700 1 -- 172.21.15.44:0/4191108460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c000ff4b0 msgr2=0x7f0c00197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c067c9700 1 --2- 172.21.15.44:0/4191108460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c000ff4b0 0x7f0c00197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c067c9700 1 -- 172.21.15.44:0/4191108460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c000ffe70 msgr2=0x7f0c00197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c067c9700 1 --2- 172.21.15.44:0/4191108460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c000ffe70 0x7f0c00197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c067c9700 1 -- 172.21.15.44:0/4191108460 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf8009d00 con 0x7f0c001044d0 2023-09-10T01:08:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c057c7700 1 --2- 172.21.15.44:0/4191108460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c000ffe70 0x7f0c00197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:08:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.148+0000 7f0c067c9700 1 --2- 172.21.15.44:0/4191108460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c001044d0 0x7f0c001a2520 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800b780 tx=0x7f0bf8008f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.149+0000 7f0bf6ffd700 1 -- 172.21.15.44:0/4191108460 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800e040 con 0x7f0c001044d0 2023-09-10T01:08:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.149+0000 7f0bf6ffd700 1 -- 172.21.15.44:0/4191108460 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bf800be40 con 0x7f0c001044d0 2023-09-10T01:08:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.149+0000 7f0bf6ffd700 1 -- 172.21.15.44:0/4191108460 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80052a0 con 0x7f0c001044d0 2023-09-10T01:08:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.149+0000 7f0c0822c700 1 -- 172.21.15.44:0/4191108460 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c00195290 con 0x7f0c001044d0 2023-09-10T01:08:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.149+0000 7f0c0822c700 1 -- 172.21.15.44:0/4191108460 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c00195750 con 0x7f0c001044d0 2023-09-10T01:08:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.151+0000 7f0bf6ffd700 1 -- 172.21.15.44:0/4191108460 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f0bf801d470 con 0x7f0c001044d0 2023-09-10T01:08:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.152+0000 7f0bf6ffd700 1 --2- 172.21.15.44:0/4191108460 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0bec0a07b0 0x7f0bec0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.152+0000 7f0c05fc8700 1 --2- 172.21.15.44:0/4191108460 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0bec0a07b0 0x7f0bec0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.152+0000 7f0bf6ffd700 1 -- 172.21.15.44:0/4191108460 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f0bf80cd730 con 0x7f0c001044d0 2023-09-10T01:08:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.152+0000 7f0c05fc8700 1 --2- 172.21.15.44:0/4191108460 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0bec0a07b0 0x7f0bec0a2c70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0c000683c0 tx=0x7f0bfc009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.152+0000 7f0c143f2b80 1 Processor -- start 2023-09-10T01:08:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c143f2b80 1 -- start start 2023-09-10T01:08:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c143f2b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc3162b1a0 0x55fc3162b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c143f2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc31625300 0x55fc31625780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c143f2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 0x55fc31631f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c143f2b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3157e6d0 con 0x55fc3162b1a0 2023-09-10T01:08:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c143f2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3157bff0 con 0x55fc31625300 2023-09-10T01:08:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c143f2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc315778e0 con 0x55fc31625cc0 2023-09-10T01:08:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c05fc8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc3162b1a0 0x55fc3162b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c057c7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc31625300 0x55fc31625780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c067c9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 0x55fc31631f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c057c7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc31625300 0x55fc31625780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57952/0 (socket says 172.21.15.44:57952) 2023-09-10T01:08:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c05fc8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc3162b1a0 0x55fc3162b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56210/0 (socket says 172.21.15.44:56210) 2023-09-10T01:08:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c057c7700 1 -- 172.21.15.44:0/2280939816 learned_addr learned my addr 172.21.15.44:0/2280939816 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c05fc8700 1 -- 172.21.15.44:0/2280939816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 msgr2=0x55fc31631f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c05fc8700 1 --2- 172.21.15.44:0/2280939816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 0x55fc31631f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c05fc8700 1 -- 172.21.15.44:0/2280939816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc31625300 msgr2=0x55fc31625780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c05fc8700 1 --2- 172.21.15.44:0/2280939816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc31625300 0x55fc31625780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c05fc8700 1 -- 172.21.15.44:0/2280939816 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc313a3d80 con 0x55fc3162b1a0 2023-09-10T01:08:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c057c7700 1 --2- 172.21.15.44:0/2280939816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc31625300 0x55fc31625780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:08:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.153+0000 7f0c05fc8700 1 --2- 172.21.15.44:0/2280939816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc3162b1a0 0x55fc3162b580 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc011c80 tx=0x7f0bfc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e482230ca8f408 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.154+0000 7f0bebfff700 1 -- 172.21.15.44:0/2280939816 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc004220 con 0x55fc3162b1a0 2023-09-10T01:08:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.154+0000 7f0bebfff700 1 -- 172.21.15.44:0/2280939816 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bfc021070 con 0x55fc3162b1a0 2023-09-10T01:08:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.154+0000 7f0bebfff700 1 -- 172.21.15.44:0/2280939816 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc003a70 con 0x55fc3162b1a0 2023-09-10T01:08:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.154+0000 7f0c143f2b80 1 -- 172.21.15.44:0/2280939816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc3162b1a0 msgr2=0x55fc3162b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.154+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/2280939816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc3162b1a0 0x55fc3162b580 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc011c80 tx=0x7f0bfc014000 comp rx=0 tx=0).stop 2023-09-10T01:08:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.155+0000 7f0c143f2b80 1 -- 172.21.15.44:0/2280939816 shutdown_connections 2023-09-10T01:08:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.155+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/2280939816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 0x55fc31631f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.155+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/2280939816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc3162b1a0 0x55fc3162b580 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.155+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/2280939816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc31625300 0x55fc31625780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.155+0000 7f0c143f2b80 1 -- 172.21.15.44:0/2280939816 >> 172.21.15.44:0/2280939816 conn(0x55fc3159a030 msgr2=0x55fc31620560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.155+0000 7f0c143f2b80 1 -- 172.21.15.44:0/2280939816 shutdown_connections 2023-09-10T01:08:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.155+0000 7f0c143f2b80 1 -- 172.21.15.44:0/2280939816 wait complete. 2023-09-10T01:08:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.155+0000 7f0c143f2b80 1 -- start start 2023-09-10T01:08:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.155+0000 7f0c143f2b80 1 Processor -- start 2023-09-10T01:08:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.155+0000 7f0c143f2b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc31631e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.155+0000 7f0c143f2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 0x55fc3162f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c143f2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3162b1a0 0x55fc3162f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c143f2b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc31515b10 con 0x55fc31625300 2023-09-10T01:08:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c143f2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc31515810 con 0x55fc3162b1a0 2023-09-10T01:08:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c143f2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc313f98c0 con 0x55fc31625cc0 2023-09-10T01:08:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c05fc8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc31631e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c143f2b80 10 client.0 _ll_register_callbacks cb 0x55fc3145fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:08:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c05fc8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc31631e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56216/0 (socket says 172.21.15.44:56216) 2023-09-10T01:08:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c05fc8700 1 -- 172.21.15.44:0/1035587523 learned_addr learned my addr 172.21.15.44:0/1035587523 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c057c7700 1 --2- 172.21.15.44:0/1035587523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 0x55fc3162f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c067c9700 1 --2- 172.21.15.44:0/1035587523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3162b1a0 0x55fc3162f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c05fc8700 1 -- 172.21.15.44:0/1035587523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 msgr2=0x55fc3162f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c05fc8700 1 --2- 172.21.15.44:0/1035587523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 0x55fc3162f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c05fc8700 1 -- 172.21.15.44:0/1035587523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3162b1a0 msgr2=0x55fc3162f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c05fc8700 1 --2- 172.21.15.44:0/1035587523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3162b1a0 0x55fc3162f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c05fc8700 1 -- 172.21.15.44:0/1035587523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf0010e70 con 0x55fc31625300 2023-09-10T01:08:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c05fc8700 1 --2- 172.21.15.44:0/1035587523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc31631e40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00e980 tx=0x7f0bfc00e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef698b843700740 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0be97fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:08:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0be97fa700 1 -- 172.21.15.44:0/1035587523 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc026680 con 0x55fc31625300 2023-09-10T01:08:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0be97fa700 1 -- 172.21.15.44:0/1035587523 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f0bfc0267e0 con 0x55fc31625300 2023-09-10T01:08:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c143f2b80 1 -- 172.21.15.44:0/1035587523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc314cf490 con 0x55fc31625300 2023-09-10T01:08:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0be97fa700 1 -- 172.21.15.44:0/1035587523 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bfc00ee90 con 0x55fc31625300 2023-09-10T01:08:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.156+0000 7f0c143f2b80 10 client.18942 fetch_fsmap learned FSMap version 111 2023-09-10T01:08:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.157+0000 7f0c143f2b80 1 -- 172.21.15.44:0/1035587523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=4+}) v3 -- 0x7f0bfc017070 con 0x55fc31625300 2023-09-10T01:08:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.157+0000 7f0be97fa700 1 -- 172.21.15.44:0/1035587523 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fc314cf490 con 0x55fc31625300 2023-09-10T01:08:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.157+0000 7f0be97fa700 1 -- 172.21.15.44:0/1035587523 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 111) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0bfc030a40 con 0x55fc31625300 2023-09-10T01:08:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.157+0000 7f0c143f2b80 10 client.18942 fetch_fsmap finished waiting for FSMap version 111 2023-09-10T01:08:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.157+0000 7f0c143f2b80 10 client.18942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:08:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.157+0000 7f0c143f2b80 20 client.18942 populate_metadata read hostname 'smithi044' 2023-09-10T01:08:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.157+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/1035587523 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fc3163f380 0x55fc316417a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.157+0000 7f0c143f2b80 4 client.18942 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:08:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.157+0000 7f0c143f2b80 1 -- 172.21.15.44:0/1035587523 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc3158c840 con 0x55fc3163f380 2023-09-10T01:08:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.157+0000 7f0c057c7700 1 --2- 172.21.15.44:0/1035587523 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fc3163f380 0x55fc316417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.158+0000 7f0c057c7700 1 --2- 172.21.15.44:0/1035587523 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fc3163f380 0x55fc316417a0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5609d4836698175d server_cookie=c43f228e08f200d6 in_seq=0 out_seq=0 2023-09-10T01:08:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.158+0000 7f0be97fa700 10 client.18942 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:08:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.158+0000 7f0be97fa700 1 -- 172.21.15.44:0/1035587523 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fc3158c840 con 0x55fc3163f380 2023-09-10T01:08:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.158+0000 7f0be97fa700 10 client.18942 handle_command_reply: tid=0 2023-09-10T01:08:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.158+0000 7f0c143f2b80 1 client.18942 shutdown 2023-09-10T01:08:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.158+0000 7f0c143f2b80 1 -- 172.21.15.44:0/1035587523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 msgr2=0x55fc31631e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.158+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/1035587523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc31631e40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00e980 tx=0x7f0bfc00e9b0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.158+0000 7f0c143f2b80 1 -- 172.21.15.44:0/1035587523 shutdown_connections 2023-09-10T01:08:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.158+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/1035587523 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fc3163f380 0x55fc316417a0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.158+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/1035587523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3162b1a0 0x55fc3162f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.158+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/1035587523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc31631e40 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.158+0000 7f0be97fa700 0 client.18942 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:08:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.158+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/1035587523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 0x55fc3162f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.158+0000 7f0c143f2b80 1 -- 172.21.15.44:0/1035587523 >> 172.21.15.44:0/1035587523 conn(0x55fc3159a030 msgr2=0x55fc3161f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.158+0000 7f0c143f2b80 1 -- 172.21.15.44:0/1035587523 shutdown_connections 2023-09-10T01:08:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.158+0000 7f0c143f2b80 1 -- 172.21.15.44:0/1035587523 wait complete. 2023-09-10T01:08:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.159+0000 7f0c143f2b80 20 client.18942 trim_cache size 0 max 16384 2023-09-10T01:08:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.174+0000 7f0c143f2b80 1 Processor -- start 2023-09-10T01:08:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.174+0000 7f0c143f2b80 1 -- start start 2023-09-10T01:08:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.174+0000 7f0c143f2b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc316256e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.174+0000 7f0c143f2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 0x55fc31631970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.174+0000 7f0c143f2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3162b1a0 0x55fc31620090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.174+0000 7f0c143f2b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc31630740 con 0x55fc31625300 2023-09-10T01:08:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.174+0000 7f0c143f2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc316300c0 con 0x55fc3162b1a0 2023-09-10T01:08:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.174+0000 7f0c143f2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3162d700 con 0x55fc31625cc0 2023-09-10T01:08:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.174+0000 7f0c05fc8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc316256e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.174+0000 7f0c05fc8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc316256e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56224/0 (socket says 172.21.15.44:56224) 2023-09-10T01:08:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.174+0000 7f0c05fc8700 1 -- 172.21.15.44:0/2576002797 learned_addr learned my addr 172.21.15.44:0/2576002797 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.174+0000 7f0c067c9700 1 --2- 172.21.15.44:0/2576002797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3162b1a0 0x55fc31620090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.174+0000 7f0c057c7700 1 --2- 172.21.15.44:0/2576002797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 0x55fc31631970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.175+0000 7f0c067c9700 1 -- 172.21.15.44:0/2576002797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 msgr2=0x55fc31631970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.175+0000 7f0c067c9700 1 --2- 172.21.15.44:0/2576002797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 0x55fc31631970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.175+0000 7f0c067c9700 1 -- 172.21.15.44:0/2576002797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 msgr2=0x55fc316256e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.175+0000 7f0c067c9700 1 --2- 172.21.15.44:0/2576002797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc316256e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.175+0000 7f0c067c9700 1 -- 172.21.15.44:0/2576002797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc3138cc00 con 0x55fc3162b1a0 2023-09-10T01:08:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.175+0000 7f0c05fc8700 1 --2- 172.21.15.44:0/2576002797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc316256e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:08:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.175+0000 7f0c067c9700 1 --2- 172.21.15.44:0/2576002797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3162b1a0 0x55fc31620090 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80caa80 tx=0x7f0bf8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5b51ee2f52c8251 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.175+0000 7f0bf4ff9700 1 -- 172.21.15.44:0/2576002797 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8004c50 con 0x55fc3162b1a0 2023-09-10T01:08:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.175+0000 7f0bf4ff9700 1 -- 172.21.15.44:0/2576002797 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bf809a870 con 0x55fc3162b1a0 2023-09-10T01:08:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.175+0000 7f0bf4ff9700 1 -- 172.21.15.44:0/2576002797 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8096f80 con 0x55fc3162b1a0 2023-09-10T01:08:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.176+0000 7f0c143f2b80 1 -- 172.21.15.44:0/2576002797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3162b1a0 msgr2=0x55fc31620090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.176+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/2576002797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3162b1a0 0x55fc31620090 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80caa80 tx=0x7f0bf8009b80 comp rx=0 tx=0).stop 2023-09-10T01:08:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.176+0000 7f0c143f2b80 1 -- 172.21.15.44:0/2576002797 shutdown_connections 2023-09-10T01:08:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.176+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/2576002797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3162b1a0 0x55fc31620090 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.176+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/2576002797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc316256e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.176+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/2576002797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 0x55fc31631970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.176+0000 7f0c143f2b80 1 -- 172.21.15.44:0/2576002797 >> 172.21.15.44:0/2576002797 conn(0x55fc3159a030 msgr2=0x55fc3162a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.176+0000 7f0c143f2b80 1 -- 172.21.15.44:0/2576002797 shutdown_connections 2023-09-10T01:08:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.176+0000 7f0c143f2b80 1 -- 172.21.15.44:0/2576002797 wait complete. 2023-09-10T01:08:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.176+0000 7f0c143f2b80 1 -- start start 2023-09-10T01:08:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.176+0000 7f0c143f2b80 1 Processor -- start 2023-09-10T01:08:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c143f2b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc31634700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c143f2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 0x55fc31634c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c143f2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3162b1a0 0x55fc31692170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c143f2b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc316353e0 con 0x55fc31625300 2023-09-10T01:08:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c143f2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc316926e0 con 0x55fc3162b1a0 2023-09-10T01:08:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c143f2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc316928a0 con 0x55fc31625cc0 2023-09-10T01:08:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c143f2b80 10 client.0 _ll_register_callbacks cb 0x7f0bf0010d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:08:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c05fc8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc31634700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c057c7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 0x55fc31634c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c067c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3162b1a0 0x55fc31692170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c05fc8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc31634700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56230/0 (socket says 172.21.15.44:56230) 2023-09-10T01:08:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c05fc8700 1 -- 172.21.15.44:0/671747097 learned_addr learned my addr 172.21.15.44:0/671747097 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:08:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c05fc8700 1 -- 172.21.15.44:0/671747097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 msgr2=0x55fc31634c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c05fc8700 1 --2- 172.21.15.44:0/671747097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 0x55fc31634c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c05fc8700 1 -- 172.21.15.44:0/671747097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3162b1a0 msgr2=0x55fc31692170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:08:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c05fc8700 1 --2- 172.21.15.44:0/671747097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3162b1a0 0x55fc31692170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c05fc8700 1 -- 172.21.15.44:0/671747097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bfc017070 con 0x55fc31625300 2023-09-10T01:08:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0c05fc8700 1 --2- 172.21.15.44:0/671747097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc31634700 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc013010 tx=0x7f0bfc01daa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36776626d7ca3771 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:08:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.177+0000 7f0be97fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:08:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.178+0000 7f0be97fa700 1 -- 172.21.15.44:0/671747097 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc030850 con 0x55fc31625300 2023-09-10T01:08:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.178+0000 7f0be97fa700 1 -- 172.21.15.44:0/671747097 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f0bfc0309b0 con 0x55fc31625300 2023-09-10T01:08:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.178+0000 7f0c143f2b80 1 -- 172.21.15.44:0/671747097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bfc021070 con 0x55fc31625300 2023-09-10T01:08:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.178+0000 7f0be97fa700 1 -- 172.21.15.44:0/671747097 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bfc00ee90 con 0x55fc31625300 2023-09-10T01:08:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.178+0000 7f0c143f2b80 10 client.18948 fetch_fsmap learned FSMap version 111 2023-09-10T01:08:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.178+0000 7f0c143f2b80 1 -- 172.21.15.44:0/671747097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=4+}) v3 -- 0x7f0c00195750 con 0x55fc31625300 2023-09-10T01:08:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.178+0000 7f0be97fa700 1 -- 172.21.15.44:0/671747097 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bfc021070 con 0x55fc31625300 2023-09-10T01:08:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.178+0000 7f0be97fa700 1 -- 172.21.15.44:0/671747097 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 111) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0bfc0269a0 con 0x55fc31625300 2023-09-10T01:08:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.178+0000 7f0c143f2b80 10 client.18948 fetch_fsmap finished waiting for FSMap version 111 2023-09-10T01:08:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.178+0000 7f0c143f2b80 10 client.18948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:08:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.178+0000 7f0c143f2b80 20 client.18948 populate_metadata read hostname 'smithi044' 2023-09-10T01:08:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.178+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/671747097 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fc31693d50 0x55fc31696170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:08:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.178+0000 7f0c143f2b80 4 client.18948 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:08:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.178+0000 7f0c143f2b80 1 -- 172.21.15.44:0/671747097 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fc315708c0 con 0x55fc31693d50 2023-09-10T01:08:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.179+0000 7f0c057c7700 1 --2- 172.21.15.44:0/671747097 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fc31693d50 0x55fc31696170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:08:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.179+0000 7f0be97fa700 10 client.18948 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:08:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.179+0000 7f0c057c7700 1 --2- 172.21.15.44:0/671747097 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fc31693d50 0x55fc31696170 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0c1b2a39669ffd9 server_cookie=1c776788d47f6774 in_seq=0 out_seq=0 2023-09-10T01:08:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.188+0000 7f0be97fa700 1 -- 172.21.15.44:0/671747097 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fc315708c0 con 0x55fc31693d50 2023-09-10T01:08:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.188+0000 7f0be97fa700 10 client.18948 handle_command_reply: tid=0 2023-09-10T01:08:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.188+0000 7f0c143f2b80 1 client.18948 shutdown 2023-09-10T01:08:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.188+0000 7f0c143f2b80 1 -- 172.21.15.44:0/671747097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 msgr2=0x55fc31634700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.188+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/671747097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc31634700 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc013010 tx=0x7f0bfc01daa0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.188+0000 7f0c143f2b80 1 -- 172.21.15.44:0/671747097 shutdown_connections 2023-09-10T01:08:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.188+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/671747097 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fc31693d50 0x55fc31696170 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.188+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/671747097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3162b1a0 0x55fc31692170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.188+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/671747097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc31625300 0x55fc31634700 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.188+0000 7f0be97fa700 0 client.18948 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:08:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.188+0000 7f0c143f2b80 1 --2- 172.21.15.44:0/671747097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc31625cc0 0x55fc31634c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.188+0000 7f0c143f2b80 1 -- 172.21.15.44:0/671747097 >> 172.21.15.44:0/671747097 conn(0x55fc3159a030 msgr2=0x55fc3162a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.188+0000 7f0c143f2b80 1 -- 172.21.15.44:0/671747097 shutdown_connections 2023-09-10T01:08:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.188+0000 7f0c143f2b80 1 -- 172.21.15.44:0/671747097 wait complete. 2023-09-10T01:08:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.188+0000 7f0c143f2b80 20 client.18948 trim_cache size 0 max 16384 2023-09-10T01:08:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:08:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:08:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:08:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:08:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.189+0000 7f0c0822c700 1 -- 172.21.15.44:0/4191108460 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0bec0a07b0 msgr2=0x7f0bec0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.189+0000 7f0c0822c700 1 --2- 172.21.15.44:0/4191108460 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0bec0a07b0 0x7f0bec0a2c70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0c000683c0 tx=0x7f0bfc009260 comp rx=0 tx=0).stop 2023-09-10T01:08:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.189+0000 7f0c0822c700 1 -- 172.21.15.44:0/4191108460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c001044d0 msgr2=0x7f0c001a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:08:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.189+0000 7f0c0822c700 1 --2- 172.21.15.44:0/4191108460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c001044d0 0x7f0c001a2520 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800b780 tx=0x7f0bf8008f90 comp rx=0 tx=0).stop 2023-09-10T01:08:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.189+0000 7f0c0822c700 1 -- 172.21.15.44:0/4191108460 shutdown_connections 2023-09-10T01:08:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.189+0000 7f0c0822c700 1 --2- 172.21.15.44:0/4191108460 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0bec0a07b0 0x7f0bec0a2c70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.189+0000 7f0c0822c700 1 --2- 172.21.15.44:0/4191108460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c001044d0 0x7f0c001a2520 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.189+0000 7f0c0822c700 1 --2- 172.21.15.44:0/4191108460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c000ff4b0 0x7f0c00197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.189+0000 7f0c0822c700 1 --2- 172.21.15.44:0/4191108460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c000ffe70 0x7f0c00197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:08:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.189+0000 7f0c0822c700 1 -- 172.21.15.44:0/4191108460 >> 172.21.15.44:0/4191108460 conn(0x7f0c00076600 msgr2=0x7f0c000fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:08:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.189+0000 7f0c0822c700 1 -- 172.21.15.44:0/4191108460 shutdown_connections 2023-09-10T01:08:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:08:49.189+0000 7f0c0822c700 1 -- 172.21.15.44:0/4191108460 wait complete. 2023-09-10T01:08:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:48 smithi131 ceph-mon[103713]: pgmap v894: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 947 KiB/s wr, 137 op/s 2023-09-10T01:08:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:48 smithi044 ceph-mon[98676]: pgmap v894: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 947 KiB/s wr, 137 op/s 2023-09-10T01:08:49.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:48 smithi099 ceph-mon[102692]: pgmap v894: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 947 KiB/s wr, 137 op/s 2023-09-10T01:08:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:50 smithi131 ceph-mon[103713]: pgmap v895: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 947 KiB/s wr, 137 op/s 2023-09-10T01:08:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:50 smithi044 ceph-mon[98676]: pgmap v895: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 947 KiB/s wr, 137 op/s 2023-09-10T01:08:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:50 smithi099 ceph-mon[102692]: pgmap v895: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 947 KiB/s wr, 137 op/s 2023-09-10T01:08:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:52 smithi131 ceph-mon[103713]: pgmap v896: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T01:08:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:52 smithi044 ceph-mon[98676]: pgmap v896: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T01:08:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:52 smithi099 ceph-mon[102692]: pgmap v896: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T01:08:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:54 smithi131 ceph-mon[103713]: pgmap v897: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:08:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:54 smithi044 ceph-mon[98676]: pgmap v897: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:08:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:54 smithi099 ceph-mon[102692]: pgmap v897: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:08:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:56 smithi131 ceph-mon[103713]: pgmap v898: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 244 op/s 2023-09-10T01:08:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:56 smithi044 ceph-mon[98676]: pgmap v898: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 244 op/s 2023-09-10T01:08:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:56 smithi099 ceph-mon[102692]: pgmap v898: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 244 op/s 2023-09-10T01:08:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:08:58 smithi131 ceph-mon[103713]: pgmap v899: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:08:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:08:58 smithi044 ceph-mon[98676]: pgmap v899: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:08:59.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:08:58 smithi099 ceph-mon[102692]: pgmap v899: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:09:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:00 smithi131 ceph-mon[103713]: pgmap v900: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:09:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:00 smithi044 ceph-mon[98676]: pgmap v900: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:09:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:00 smithi099 ceph-mon[102692]: pgmap v900: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:09:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.325+0000 7f58fca02700 1 -- 172.21.15.44:0/1133661239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f80ff970 msgr2=0x7f58f80ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.325+0000 7f58fca02700 1 --2- 172.21.15.44:0/1133661239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f80ff970 0x7f58f80ffd50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009990 tx=0x7f58e8009ca0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.325+0000 7f58fca02700 1 -- 172.21.15.44:0/1133661239 shutdown_connections 2023-09-10T01:09:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.325+0000 7f58fca02700 1 --2- 172.21.15.44:0/1133661239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58f81047f0 0x7f58f810c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.325+0000 7f58fca02700 1 --2- 172.21.15.44:0/1133661239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f80ff970 0x7f58f80ffd50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.325+0000 7f58fca02700 1 --2- 172.21.15.44:0/1133661239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58f8100320 0x7f58f81040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.325+0000 7f58fca02700 1 -- 172.21.15.44:0/1133661239 >> 172.21.15.44:0/1133661239 conn(0x7f58f80773b0 msgr2=0x7f58f80777c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.325+0000 7f58fca02700 1 -- 172.21.15.44:0/1133661239 shutdown_connections 2023-09-10T01:09:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.325+0000 7f58fca02700 1 -- 172.21.15.44:0/1133661239 wait complete. 2023-09-10T01:09:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.326+0000 7f58fca02700 1 Processor -- start 2023-09-10T01:09:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.326+0000 7f58fca02700 1 -- start start 2023-09-10T01:09:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58fca02700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58f80ff970 0x7f58f819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58fca02700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58f8100320 0x7f58f819dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58fca02700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f81047f0 0x7f58f8198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58fca02700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f81995f0 con 0x7f58f80ff970 2023-09-10T01:09:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58fca02700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f8199760 con 0x7f58f8100320 2023-09-10T01:09:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58fca02700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f8199920 con 0x7f58f81047f0 2023-09-10T01:09:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58f659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58f80ff970 0x7f58f819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58f659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58f80ff970 0x7f58f819d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56242/0 (socket says 172.21.15.44:56242) 2023-09-10T01:09:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58f659c700 1 -- 172.21.15.44:0/2622042404 learned_addr learned my addr 172.21.15.44:0/2622042404 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58f6d9d700 1 --2- 172.21.15.44:0/2622042404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f81047f0 0x7f58f8198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58f5d9b700 1 --2- 172.21.15.44:0/2622042404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58f8100320 0x7f58f819dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58f659c700 1 -- 172.21.15.44:0/2622042404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f81047f0 msgr2=0x7f58f8198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58f659c700 1 --2- 172.21.15.44:0/2622042404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f81047f0 0x7f58f8198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58f659c700 1 -- 172.21.15.44:0/2622042404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58f8100320 msgr2=0x7f58f819dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58f659c700 1 --2- 172.21.15.44:0/2622042404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58f8100320 0x7f58f819dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58f659c700 1 -- 172.21.15.44:0/2622042404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e80095f0 con 0x7f58f80ff970 2023-09-10T01:09:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58f659c700 1 --2- 172.21.15.44:0/2622042404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58f80ff970 0x7f58f819d7a0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f58e8005740 tx=0x7f58e8005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.327+0000 7f58e77fe700 1 -- 172.21.15.44:0/2622042404 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e801d070 con 0x7f58f80ff970 2023-09-10T01:09:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.328+0000 7f58fca02700 1 -- 172.21.15.44:0/2622042404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f8199ba0 con 0x7f58f80ff970 2023-09-10T01:09:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.328+0000 7f58fca02700 1 -- 172.21.15.44:0/2622042404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f81a58f0 con 0x7f58f80ff970 2023-09-10T01:09:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.328+0000 7f58e77fe700 1 -- 172.21.15.44:0/2622042404 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58e8022470 con 0x7f58f80ff970 2023-09-10T01:09:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.328+0000 7f58e77fe700 1 -- 172.21.15.44:0/2622042404 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8017650 con 0x7f58f80ff970 2023-09-10T01:09:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.330+0000 7f58e77fe700 1 -- 172.21.15.44:0/2622042404 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f58e80228a0 con 0x7f58f80ff970 2023-09-10T01:09:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.331+0000 7f58e77fe700 1 --2- 172.21.15.44:0/2622042404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f58e00a07b0 0x7f58e00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.331+0000 7f58e77fe700 1 -- 172.21.15.44:0/2622042404 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f58e80cd530 con 0x7f58f80ff970 2023-09-10T01:09:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.331+0000 7f58f5d9b700 1 --2- 172.21.15.44:0/2622042404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f58e00a07b0 0x7f58e00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.332+0000 7f5908bc8b80 1 Processor -- start 2023-09-10T01:09:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.332+0000 7f5908bc8b80 1 -- start start 2023-09-10T01:09:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.332+0000 7f58f5d9b700 1 --2- 172.21.15.44:0/2622042404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f58e00a07b0 0x7f58e00a2c70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f58f810c7f0 tx=0x7f58ec009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.332+0000 7f5908bc8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afd39b0 0x56170afd3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.332+0000 7f5908bc8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afcdb10 0x56170afcdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.332+0000 7f5908bc8b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 0x56170afda740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.332+0000 7f5908bc8b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170af1e6d0 con 0x56170afce4d0 2023-09-10T01:09:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.332+0000 7f5908bc8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170af1bff0 con 0x56170afd39b0 2023-09-10T01:09:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.332+0000 7f5908bc8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170af178e0 con 0x56170afcdb10 2023-09-10T01:09:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.332+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afcdb10 0x56170afcdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.332+0000 7f58f659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afd39b0 0x56170afd3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.332+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afcdb10 0x56170afcdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34902/0 (socket says 172.21.15.44:34902) 2023-09-10T01:09:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.332+0000 7f58f5d9b700 1 -- 172.21.15.44:0/1961333857 learned_addr learned my addr 172.21.15.44:0/1961333857 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.332+0000 7f58f6d9d700 1 --2- 172.21.15.44:0/1961333857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 0x56170afda740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.332+0000 7f58f659c700 1 -- 172.21.15.44:0/1961333857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afcdb10 msgr2=0x56170afcdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.332+0000 7f58f659c700 1 --2- 172.21.15.44:0/1961333857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afcdb10 0x56170afcdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.333+0000 7f58f659c700 1 -- 172.21.15.44:0/1961333857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 msgr2=0x56170afda740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.333+0000 7f58f659c700 1 --2- 172.21.15.44:0/1961333857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 0x56170afda740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.333+0000 7f58f659c700 1 -- 172.21.15.44:0/1961333857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56170ad43d80 con 0x56170afd39b0 2023-09-10T01:09:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.333+0000 7f58f6d9d700 1 --2- 172.21.15.44:0/1961333857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 0x56170afda740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:09:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.333+0000 7f58f659c700 1 --2- 172.21.15.44:0/1961333857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afd39b0 0x56170afd3d90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f58e80ca880 tx=0x7f58e809a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa9f79cfed716255 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.333+0000 7f58e4ff9700 1 -- 172.21.15.44:0/1961333857 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e809ad70 con 0x56170afd39b0 2023-09-10T01:09:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.333+0000 7f58e4ff9700 1 -- 172.21.15.44:0/1961333857 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58e809a670 con 0x56170afd39b0 2023-09-10T01:09:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.333+0000 7f58e4ff9700 1 -- 172.21.15.44:0/1961333857 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e809e170 con 0x56170afd39b0 2023-09-10T01:09:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.334+0000 7f5908bc8b80 1 -- 172.21.15.44:0/1961333857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afd39b0 msgr2=0x56170afd3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.334+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/1961333857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afd39b0 0x56170afd3d90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f58e80ca880 tx=0x7f58e809a440 comp rx=0 tx=0).stop 2023-09-10T01:09:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.334+0000 7f5908bc8b80 1 -- 172.21.15.44:0/1961333857 shutdown_connections 2023-09-10T01:09:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.334+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/1961333857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 0x56170afda740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.334+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/1961333857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afd39b0 0x56170afd3d90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.334+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/1961333857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afcdb10 0x56170afcdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.334+0000 7f5908bc8b80 1 -- 172.21.15.44:0/1961333857 >> 172.21.15.44:0/1961333857 conn(0x56170af3a030 msgr2=0x56170afc0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.334+0000 7f5908bc8b80 1 -- 172.21.15.44:0/1961333857 shutdown_connections 2023-09-10T01:09:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.334+0000 7f5908bc8b80 1 -- 172.21.15.44:0/1961333857 wait complete. 2023-09-10T01:09:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.334+0000 7f5908bc8b80 1 -- start start 2023-09-10T01:09:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f5908bc8b80 1 Processor -- start 2023-09-10T01:09:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f5908bc8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afcdb10 0x56170afdd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f5908bc8b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 0x56170afdd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f5908bc8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afd39b0 0x56170afd75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f5908bc8b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170aeb5b10 con 0x56170afce4d0 2023-09-10T01:09:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f5908bc8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170aeb5810 con 0x56170afcdb10 2023-09-10T01:09:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f5908bc8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170ad998c0 con 0x56170afd39b0 2023-09-10T01:09:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f5908bc8b80 10 client.0 _ll_register_callbacks cb 0x56170adffa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:09:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 0x56170afdd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f58f659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afcdb10 0x56170afdd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f58f6d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afd39b0 0x56170afd75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f58f659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afcdb10 0x56170afdd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57996/0 (socket says 172.21.15.44:57996) 2023-09-10T01:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f58f659c700 1 -- 172.21.15.44:0/895243614 learned_addr learned my addr 172.21.15.44:0/895243614 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 0x56170afdd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56258/0 (socket says 172.21.15.44:56258) 2023-09-10T01:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f58f659c700 1 -- 172.21.15.44:0/895243614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afd39b0 msgr2=0x56170afd75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f58f659c700 1 --2- 172.21.15.44:0/895243614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afd39b0 0x56170afd75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f58f659c700 1 -- 172.21.15.44:0/895243614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 msgr2=0x56170afdd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.335+0000 7f58f659c700 1 --2- 172.21.15.44:0/895243614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 0x56170afdd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.336+0000 7f58f659c700 1 -- 172.21.15.44:0/895243614 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f0011230 con 0x56170afcdb10 2023-09-10T01:09:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.336+0000 7f58f659c700 1 --2- 172.21.15.44:0/895243614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afcdb10 0x56170afdd330 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f58e80ca930 tx=0x7f58e8022b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aa9bb5deff39cf3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.336+0000 7f58ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:09:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.336+0000 7f58ddffb700 1 -- 172.21.15.44:0/895243614 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e809e4a0 con 0x56170afcdb10 2023-09-10T01:09:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.336+0000 7f58ddffb700 1 -- 172.21.15.44:0/895243614 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f58e80a2650 con 0x56170afcdb10 2023-09-10T01:09:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.336+0000 7f5908bc8b80 1 -- 172.21.15.44:0/895243614 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56170ae6f490 con 0x56170afcdb10 2023-09-10T01:09:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.336+0000 7f58ddffb700 1 -- 172.21.15.44:0/895243614 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58e809ad70 con 0x56170afcdb10 2023-09-10T01:09:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.336+0000 7f5908bc8b80 10 client.28622 fetch_fsmap learned FSMap version 111 2023-09-10T01:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.336+0000 7f5908bc8b80 1 -- 172.21.15.44:0/895243614 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=4+}) v3 -- 0x7f58f81a58f0 con 0x56170afcdb10 2023-09-10T01:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.336+0000 7f58ddffb700 1 -- 172.21.15.44:0/895243614 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58e8081030 con 0x56170afcdb10 2023-09-10T01:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.336+0000 7f58ddffb700 1 -- 172.21.15.44:0/895243614 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 111) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f58e809a940 con 0x56170afcdb10 2023-09-10T01:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.337+0000 7f5908bc8b80 10 client.28622 fetch_fsmap finished waiting for FSMap version 111 2023-09-10T01:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.337+0000 7f5908bc8b80 10 client.28622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:09:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.337+0000 7f5908bc8b80 20 client.28622 populate_metadata read hostname 'smithi044' 2023-09-10T01:09:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.337+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/895243614 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56170afe72b0 0x56170afe96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.337+0000 7f5908bc8b80 4 client.28622 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:09:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.337+0000 7f5908bc8b80 1 -- 172.21.15.44:0/895243614 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56170af2c840 con 0x56170afe72b0 2023-09-10T01:09:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.337+0000 7f58f5d9b700 1 --2- 172.21.15.44:0/895243614 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56170afe72b0 0x56170afe96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.337+0000 7f58ddffb700 10 client.28622 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.337+0000 7f58f5d9b700 1 --2- 172.21.15.44:0/895243614 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56170afe72b0 0x56170afe96a0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7af4cb72cee06e9c server_cookie=3940c5b4b1e68a0 in_seq=0 out_seq=0 2023-09-10T01:09:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.338+0000 7f58ddffb700 1 -- 172.21.15.44:0/895243614 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56170af2c840 con 0x56170afe72b0 2023-09-10T01:09:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.338+0000 7f58ddffb700 10 client.28622 handle_command_reply: tid=0 2023-09-10T01:09:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.338+0000 7f5908bc8b80 1 client.28622 shutdown 2023-09-10T01:09:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.338+0000 7f5908bc8b80 1 -- 172.21.15.44:0/895243614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afcdb10 msgr2=0x56170afdd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.338+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/895243614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afcdb10 0x56170afdd330 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f58e80ca930 tx=0x7f58e8022b50 comp rx=0 tx=0).stop 2023-09-10T01:09:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.338+0000 7f5908bc8b80 1 -- 172.21.15.44:0/895243614 shutdown_connections 2023-09-10T01:09:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.338+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/895243614 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56170afe72b0 0x56170afe96a0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.338+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/895243614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afd39b0 0x56170afd75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.338+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/895243614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afcdb10 0x56170afdd330 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.338+0000 7f58ddffb700 0 client.28622 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.338+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/895243614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 0x56170afdd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.338+0000 7f5908bc8b80 1 -- 172.21.15.44:0/895243614 >> 172.21.15.44:0/895243614 conn(0x56170af3a030 msgr2=0x56170afbf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.338+0000 7f5908bc8b80 1 -- 172.21.15.44:0/895243614 shutdown_connections 2023-09-10T01:09:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.338+0000 7f5908bc8b80 1 -- 172.21.15.44:0/895243614 wait complete. 2023-09-10T01:09:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.338+0000 7f5908bc8b80 20 client.28622 trim_cache size 0 max 16384 2023-09-10T01:09:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.358+0000 7f5908bc8b80 1 Processor -- start 2023-09-10T01:09:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.358+0000 7f5908bc8b80 1 -- start start 2023-09-10T01:09:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.358+0000 7f5908bc8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afcdb10 0x56170afc0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.358+0000 7f5908bc8b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 0x56170afd2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.358+0000 7f5908bc8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afd39b0 0x56170afdcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.358+0000 7f5908bc8b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170afc1e40 con 0x56170afce4d0 2023-09-10T01:09:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.358+0000 7f5908bc8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170afd3480 con 0x56170afd39b0 2023-09-10T01:09:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.358+0000 7f5908bc8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170afc1930 con 0x56170afcdb10 2023-09-10T01:09:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.358+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 0x56170afd2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.359+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 0x56170afd2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56266/0 (socket says 172.21.15.44:56266) 2023-09-10T01:09:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.359+0000 7f58f5d9b700 1 -- 172.21.15.44:0/2432454714 learned_addr learned my addr 172.21.15.44:0/2432454714 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.359+0000 7f58f659c700 1 --2- 172.21.15.44:0/2432454714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afcdb10 0x56170afc0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.359+0000 7f58f6d9d700 1 --2- 172.21.15.44:0/2432454714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afd39b0 0x56170afdcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.359+0000 7f58f5d9b700 1 -- 172.21.15.44:0/2432454714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afcdb10 msgr2=0x56170afc0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.359+0000 7f58f5d9b700 1 --2- 172.21.15.44:0/2432454714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afcdb10 0x56170afc0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.359+0000 7f58f5d9b700 1 -- 172.21.15.44:0/2432454714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afd39b0 msgr2=0x56170afdcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.359+0000 7f58f5d9b700 1 --2- 172.21.15.44:0/2432454714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afd39b0 0x56170afdcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.359+0000 7f58f5d9b700 1 -- 172.21.15.44:0/2432454714 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56170ad2cc00 con 0x56170afce4d0 2023-09-10T01:09:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.359+0000 7f58f5d9b700 1 --2- 172.21.15.44:0/2432454714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 0x56170afd2f40 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f58ec013010 tx=0x7f58ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2e24ec963dbd2f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.359+0000 7f58e57fa700 1 -- 172.21.15.44:0/2432454714 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec022040 con 0x56170afce4d0 2023-09-10T01:09:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.359+0000 7f58e57fa700 1 -- 172.21.15.44:0/2432454714 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58ec023040 con 0x56170afce4d0 2023-09-10T01:09:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.359+0000 7f58e57fa700 1 -- 172.21.15.44:0/2432454714 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec003b50 con 0x56170afce4d0 2023-09-10T01:09:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.360+0000 7f5908bc8b80 1 -- 172.21.15.44:0/2432454714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 msgr2=0x56170afd2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.360+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/2432454714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 0x56170afd2f40 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f58ec013010 tx=0x7f58ec014000 comp rx=0 tx=0).stop 2023-09-10T01:09:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.360+0000 7f5908bc8b80 1 -- 172.21.15.44:0/2432454714 shutdown_connections 2023-09-10T01:09:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.360+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/2432454714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afd39b0 0x56170afdcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.360+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/2432454714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afcdb10 0x56170afc0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.360+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/2432454714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afce4d0 0x56170afd2f40 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.360+0000 7f5908bc8b80 1 -- 172.21.15.44:0/2432454714 >> 172.21.15.44:0/2432454714 conn(0x56170af3a030 msgr2=0x56170afd2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.360+0000 7f5908bc8b80 1 -- 172.21.15.44:0/2432454714 shutdown_connections 2023-09-10T01:09:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.360+0000 7f5908bc8b80 1 -- 172.21.15.44:0/2432454714 wait complete. 2023-09-10T01:09:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.361+0000 7f5908bc8b80 1 -- start start 2023-09-10T01:09:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.361+0000 7f5908bc8b80 1 Processor -- start 2023-09-10T01:09:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.361+0000 7f5908bc8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afcdb10 0x56170b0349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.361+0000 7f5908bc8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afce4d0 0x56170b034ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.361+0000 7f5908bc8b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afd39b0 0x56170b02efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.361+0000 7f5908bc8b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170b02f5e0 con 0x56170afd39b0 2023-09-10T01:09:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.361+0000 7f5908bc8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170b02f750 con 0x56170afcdb10 2023-09-10T01:09:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.361+0000 7f5908bc8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170b02f910 con 0x56170afce4d0 2023-09-10T01:09:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.361+0000 7f5908bc8b80 10 client.0 _ll_register_callbacks cb 0x7f58f0009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:09:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.361+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afce4d0 0x56170b034ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.361+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afce4d0 0x56170b034ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34922/0 (socket says 172.21.15.44:34922) 2023-09-10T01:09:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.361+0000 7f58f5d9b700 1 -- 172.21.15.44:0/3242027017 learned_addr learned my addr 172.21.15.44:0/3242027017 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.361+0000 7f58f659c700 1 --2- 172.21.15.44:0/3242027017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afcdb10 0x56170b0349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.361+0000 7f58f6d9d700 1 --2- 172.21.15.44:0/3242027017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afd39b0 0x56170b02efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.362+0000 7f58f5d9b700 1 -- 172.21.15.44:0/3242027017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afcdb10 msgr2=0x56170b0349b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.362+0000 7f58f5d9b700 1 --2- 172.21.15.44:0/3242027017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afcdb10 0x56170b0349b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.362+0000 7f58f5d9b700 1 -- 172.21.15.44:0/3242027017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afd39b0 msgr2=0x56170b02efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.362+0000 7f58f5d9b700 1 --2- 172.21.15.44:0/3242027017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afd39b0 0x56170b02efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.362+0000 7f58f5d9b700 1 -- 172.21.15.44:0/3242027017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ec017070 con 0x56170afce4d0 2023-09-10T01:09:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.362+0000 7f58f5d9b700 1 --2- 172.21.15.44:0/3242027017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afce4d0 0x56170b034ef0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f58ec02e040 tx=0x7f58ec0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abea805ab17aec3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.362+0000 7f58ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:09:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.362+0000 7f58ddffb700 1 -- 172.21.15.44:0/3242027017 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec022040 con 0x56170afce4d0 2023-09-10T01:09:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.362+0000 7f5908bc8b80 1 -- 172.21.15.44:0/3242027017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58e809a670 con 0x56170afce4d0 2023-09-10T01:09:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.362+0000 7f58ddffb700 1 -- 172.21.15.44:0/3242027017 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f58ec004040 con 0x56170afce4d0 2023-09-10T01:09:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.362+0000 7f58ddffb700 1 -- 172.21.15.44:0/3242027017 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58ec004200 con 0x56170afce4d0 2023-09-10T01:09:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.362+0000 7f5908bc8b80 10 client.28570 fetch_fsmap learned FSMap version 111 2023-09-10T01:09:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.362+0000 7f5908bc8b80 1 -- 172.21.15.44:0/3242027017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=4+}) v3 -- 0x56170ad6a1f0 con 0x56170afce4d0 2023-09-10T01:09:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.363+0000 7f58ddffb700 1 -- 172.21.15.44:0/3242027017 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58e809a670 con 0x56170afce4d0 2023-09-10T01:09:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.363+0000 7f58ddffb700 1 -- 172.21.15.44:0/3242027017 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 111) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f58ec032070 con 0x56170afce4d0 2023-09-10T01:09:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.363+0000 7f5908bc8b80 10 client.28570 fetch_fsmap finished waiting for FSMap version 111 2023-09-10T01:09:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.363+0000 7f5908bc8b80 10 client.28570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:09:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.363+0000 7f5908bc8b80 20 client.28570 populate_metadata read hostname 'smithi044' 2023-09-10T01:09:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.363+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/3242027017 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56170b03c8a0 0x56170b03ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.363+0000 7f5908bc8b80 4 client.28570 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:09:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.363+0000 7f5908bc8b80 1 -- 172.21.15.44:0/3242027017 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56170af108c0 con 0x56170b03c8a0 2023-09-10T01:09:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.363+0000 7f58f659c700 1 --2- 172.21.15.44:0/3242027017 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56170b03c8a0 0x56170b03ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.363+0000 7f58f659c700 1 --2- 172.21.15.44:0/3242027017 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56170b03c8a0 0x56170b03ecc0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3090ec5632d507f9 server_cookie=136f3955087d6728 in_seq=0 out_seq=0 2023-09-10T01:09:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:02.363+0000 7f58ddffb700 10 client.28570 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:02 smithi131 ceph-mon[103713]: pgmap v901: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 236 op/s 2023-09-10T01:09:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:02 smithi044 ceph-mon[98676]: pgmap v901: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 236 op/s 2023-09-10T01:09:03.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:02 smithi099 ceph-mon[102692]: pgmap v901: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 236 op/s 2023-09-10T01:09:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.426+0000 7f58ddffb700 1 -- 172.21.15.44:0/3242027017 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56170af108c0 con 0x56170b03c8a0 2023-09-10T01:09:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.426+0000 7f58ddffb700 10 client.28570 handle_command_reply: tid=0 2023-09-10T01:09:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.426+0000 7f5908bc8b80 1 client.28570 shutdown 2023-09-10T01:09:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.426+0000 7f5908bc8b80 1 -- 172.21.15.44:0/3242027017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afce4d0 msgr2=0x56170b034ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.426+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/3242027017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afce4d0 0x56170b034ef0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f58ec02e040 tx=0x7f58ec0057f0 comp rx=0 tx=0).stop 2023-09-10T01:09:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.426+0000 7f5908bc8b80 1 -- 172.21.15.44:0/3242027017 shutdown_connections 2023-09-10T01:09:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.426+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/3242027017 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56170b03c8a0 0x56170b03ecc0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.426+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/3242027017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56170afd39b0 0x56170b02efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.426+0000 7f58ddffb700 0 client.28570 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.426+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/3242027017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56170afcdb10 0x56170b0349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.426+0000 7f5908bc8b80 1 --2- 172.21.15.44:0/3242027017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56170afce4d0 0x56170b034ef0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.426+0000 7f5908bc8b80 1 -- 172.21.15.44:0/3242027017 >> 172.21.15.44:0/3242027017 conn(0x56170af3a030 msgr2=0x56170afccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.426+0000 7f5908bc8b80 1 -- 172.21.15.44:0/3242027017 shutdown_connections 2023-09-10T01:09:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.426+0000 7f5908bc8b80 1 -- 172.21.15.44:0/3242027017 wait complete. 2023-09-10T01:09:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.427+0000 7f5908bc8b80 20 client.28570 trim_cache size 0 max 16384 2023-09-10T01:09:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:09:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:09:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:09:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:09:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.427+0000 7f58fca02700 1 -- 172.21.15.44:0/2622042404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f58e00a07b0 msgr2=0x7f58e00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.427+0000 7f58fca02700 1 --2- 172.21.15.44:0/2622042404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f58e00a07b0 0x7f58e00a2c70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f58f810c7f0 tx=0x7f58ec009260 comp rx=0 tx=0).stop 2023-09-10T01:09:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.427+0000 7f58fca02700 1 -- 172.21.15.44:0/2622042404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58f80ff970 msgr2=0x7f58f819d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.427+0000 7f58fca02700 1 --2- 172.21.15.44:0/2622042404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58f80ff970 0x7f58f819d7a0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f58e8005740 tx=0x7f58e8005770 comp rx=0 tx=0).stop 2023-09-10T01:09:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.427+0000 7f58fca02700 1 -- 172.21.15.44:0/2622042404 shutdown_connections 2023-09-10T01:09:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.427+0000 7f58fca02700 1 --2- 172.21.15.44:0/2622042404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f58e00a07b0 0x7f58e00a2c70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.427+0000 7f58fca02700 1 --2- 172.21.15.44:0/2622042404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f81047f0 0x7f58f8198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.427+0000 7f58fca02700 1 --2- 172.21.15.44:0/2622042404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58f80ff970 0x7f58f819d7a0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.427+0000 7f58fca02700 1 --2- 172.21.15.44:0/2622042404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58f8100320 0x7f58f819dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.427+0000 7f58fca02700 1 -- 172.21.15.44:0/2622042404 >> 172.21.15.44:0/2622042404 conn(0x7f58f80773b0 msgr2=0x7f58f81030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.427+0000 7f58fca02700 1 -- 172.21.15.44:0/2622042404 shutdown_connections 2023-09-10T01:09:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:03.427+0000 7f58fca02700 1 -- 172.21.15.44:0/2622042404 wait complete. 2023-09-10T01:09:04.400 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: *** Caught signal (Segmentation fault) ** 2023-09-10T01:09:04.400 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: in thread 7f1d4d289700 thread_name:md_log_replay 2023-09-10T01:09:04.401 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:09:04.401 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f1d5de38cf0] 2023-09-10T01:09:04.401 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x5603d566030f] 2023-09-10T01:09:04.401 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5603d5662543] 2023-09-10T01:09:04.401 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 4: (MDLog::_replay_thread()+0x647) [0x5603d56cbe97] 2023-09-10T01:09:04.402 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5603d56d5381] 2023-09-10T01:09:04.402 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f1d5de2e1ca] 2023-09-10T01:09:04.402 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 7: clone() 2023-09-10T01:09:04.402 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 2023-09-10T01:09:04.140+0000 7f1d4d289700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:09:04.402 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: in thread 7f1d4d289700 thread_name:md_log_replay 2023-09-10T01:09:04.403 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 2023-09-10T01:09:04.403 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:09:04.403 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f1d5de38cf0] 2023-09-10T01:09:04.403 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x5603d566030f] 2023-09-10T01:09:04.403 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5603d5662543] 2023-09-10T01:09:04.404 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 4: (MDLog::_replay_thread()+0x647) [0x5603d56cbe97] 2023-09-10T01:09:04.404 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5603d56d5381] 2023-09-10T01:09:04.404 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f1d5de2e1ca] 2023-09-10T01:09:04.404 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 7: clone() 2023-09-10T01:09:04.404 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:09:04.404 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 2023-09-10T01:09:04.405 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 0> 2023-09-10T01:09:04.140+0000 7f1d4d289700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:09:04.405 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: in thread 7f1d4d289700 thread_name:md_log_replay 2023-09-10T01:09:04.405 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 2023-09-10T01:09:04.405 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:09:04.405 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f1d5de38cf0] 2023-09-10T01:09:04.405 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x5603d566030f] 2023-09-10T01:09:04.406 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5603d5662543] 2023-09-10T01:09:04.406 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 4: (MDLog::_replay_thread()+0x647) [0x5603d56cbe97] 2023-09-10T01:09:04.406 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5603d56d5381] 2023-09-10T01:09:04.406 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f1d5de2e1ca] 2023-09-10T01:09:04.406 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 7: clone() 2023-09-10T01:09:04.407 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:09:04.407 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 2023-09-10T01:09:04.407 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 0> 2023-09-10T01:09:04.140+0000 7f1d4d289700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:09:04.407 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: in thread 7f1d4d289700 thread_name:md_log_replay 2023-09-10T01:09:04.407 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 2023-09-10T01:09:04.408 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:09:04.408 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f1d5de38cf0] 2023-09-10T01:09:04.408 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x5603d566030f] 2023-09-10T01:09:04.408 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5603d5662543] 2023-09-10T01:09:04.408 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 4: (MDLog::_replay_thread()+0x647) [0x5603d56cbe97] 2023-09-10T01:09:04.408 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5603d56d5381] 2023-09-10T01:09:04.409 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f1d5de2e1ca] 2023-09-10T01:09:04.409 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 7: clone() 2023-09-10T01:09:04.409 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:09:04.409 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-c[150815]: 2023-09-10T01:09:04.795 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Main process exited, code=exited, status=139/n/a 2023-09-10T01:09:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-mon[103713]: pgmap v902: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-09-10T01:09:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:04 smithi131 ceph-mon[103713]: osdmap e140: 12 total, 12 up, 12 in 2023-09-10T01:09:05.296 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:04 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Failed with result 'exit-code'. 2023-09-10T01:09:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:04 smithi044 ceph-mon[98676]: pgmap v902: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-09-10T01:09:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:04 smithi044 ceph-mon[98676]: osdmap e140: 12 total, 12 up, 12 in 2023-09-10T01:09:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:04 smithi099 ceph-mon[102692]: pgmap v902: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-09-10T01:09:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:04 smithi099 ceph-mon[102692]: osdmap e140: 12 total, 12 up, 12 in 2023-09-10T01:09:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:06 smithi131 ceph-mon[103713]: pgmap v904: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:09:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:06 smithi044 ceph-mon[98676]: pgmap v904: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:09:07.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:06 smithi099 ceph-mon[102692]: pgmap v904: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:09:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:08 smithi131 ceph-mon[103713]: pgmap v905: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:09:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:08 smithi044 ceph-mon[98676]: pgmap v905: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:09:09.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:08 smithi099 ceph-mon[102692]: pgmap v905: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:09:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:10 smithi131 ceph-mon[103713]: pgmap v906: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:09:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:10 smithi131 ceph-mon[103713]: osdmap e141: 12 total, 12 up, 12 in 2023-09-10T01:09:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:10 smithi044 ceph-mon[98676]: pgmap v906: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:09:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:10 smithi044 ceph-mon[98676]: osdmap e141: 12 total, 12 up, 12 in 2023-09-10T01:09:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:10 smithi099 ceph-mon[102692]: pgmap v906: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:09:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:10 smithi099 ceph-mon[102692]: osdmap e141: 12 total, 12 up, 12 in 2023-09-10T01:09:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:12 smithi131 ceph-mon[103713]: pgmap v908: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2023-09-10T01:09:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:12 smithi044 ceph-mon[98676]: pgmap v908: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2023-09-10T01:09:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:12 smithi099 ceph-mon[102692]: pgmap v908: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2023-09-10T01:09:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:14 smithi131 ceph-mon[103713]: pgmap v909: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 220 op/s 2023-09-10T01:09:15.296 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:15 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Service RestartSec=10s expired, scheduling restart. 2023-09-10T01:09:15.296 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:15 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Scheduled restart job, restart counter is at 6. 2023-09-10T01:09:15.296 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:15 smithi131 systemd[1]: Stopped Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:09:15.296 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:15 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Start request repeated too quickly. 2023-09-10T01:09:15.296 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:15 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service: Failed with result 'exit-code'. 2023-09-10T01:09:15.297 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 10 01:09:15 smithi131 systemd[1]: Failed to start Ceph mds.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:09:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:14 smithi044 ceph-mon[98676]: pgmap v909: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 220 op/s 2023-09-10T01:09:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:14 smithi099 ceph-mon[102692]: pgmap v909: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 220 op/s 2023-09-10T01:09:16.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:15 smithi131 ceph-mon[103713]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2023-09-10T01:09:16.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:15 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:09:16.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:15 smithi044 ceph-mon[98676]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2023-09-10T01:09:16.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:15 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:09:16.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:15 smithi099 ceph-mon[102692]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2023-09-10T01:09:16.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:15 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:09:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.553+0000 7f7e99f65700 1 -- 172.21.15.44:0/3454438249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e940ff760 msgr2=0x7f7e940ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.553+0000 7f7e99f65700 1 --2- 172.21.15.44:0/3454438249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e940ff760 0x7f7e940ffbe0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7e880099c0 tx=0x7f7e88009cd0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.553+0000 7f7e99f65700 1 -- 172.21.15.44:0/3454438249 shutdown_connections 2023-09-10T01:09:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.553+0000 7f7e99f65700 1 --2- 172.21.15.44:0/3454438249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e94100120 0x7f7e9410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.553+0000 7f7e99f65700 1 --2- 172.21.15.44:0/3454438249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e941055b0 0x7f7e94105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.553+0000 7f7e99f65700 1 --2- 172.21.15.44:0/3454438249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e940ff760 0x7f7e940ffbe0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.553+0000 7f7e99f65700 1 -- 172.21.15.44:0/3454438249 >> 172.21.15.44:0/3454438249 conn(0x7f7e94074f30 msgr2=0x7f7e94075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.553+0000 7f7e99f65700 1 -- 172.21.15.44:0/3454438249 shutdown_connections 2023-09-10T01:09:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.553+0000 7f7e99f65700 1 -- 172.21.15.44:0/3454438249 wait complete. 2023-09-10T01:09:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.553+0000 7f7e99f65700 1 Processor -- start 2023-09-10T01:09:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.553+0000 7f7e99f65700 1 -- start start 2023-09-10T01:09:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e99f65700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e940ff760 0x7f7e941012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e99f65700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e94100120 0x7f7e941017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e99f65700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e941055b0 0x7f7e941a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e99f65700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e941a4c40 con 0x7f7e941055b0 2023-09-10T01:09:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e99f65700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e941a4d80 con 0x7f7e940ff760 2023-09-10T01:09:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e99f65700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e941a4ef0 con 0x7f7e94100120 2023-09-10T01:09:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e93fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e941055b0 0x7f7e941a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e92ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e94100120 0x7f7e941017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e93fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e941055b0 0x7f7e941a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56314/0 (socket says 172.21.15.44:56314) 2023-09-10T01:09:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e93fff700 1 -- 172.21.15.44:0/2252294305 learned_addr learned my addr 172.21.15.44:0/2252294305 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e937fe700 1 --2- 172.21.15.44:0/2252294305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e940ff760 0x7f7e941012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e93fff700 1 -- 172.21.15.44:0/2252294305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e94100120 msgr2=0x7f7e941017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e93fff700 1 --2- 172.21.15.44:0/2252294305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e94100120 0x7f7e941017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e93fff700 1 -- 172.21.15.44:0/2252294305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e940ff760 msgr2=0x7f7e941012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e93fff700 1 --2- 172.21.15.44:0/2252294305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e940ff760 0x7f7e941012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e93fff700 1 -- 172.21.15.44:0/2252294305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e880095f0 con 0x7f7e941055b0 2023-09-10T01:09:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e92ffd700 1 --2- 172.21.15.44:0/2252294305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e94100120 0x7f7e941017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:09:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.554+0000 7f7e93fff700 1 --2- 172.21.15.44:0/2252294305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e941055b0 0x7f7e941a4700 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00ef30 tx=0x7f7e8c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.555+0000 7f7e90ff9700 1 -- 172.21.15.44:0/2252294305 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c018070 con 0x7f7e941055b0 2023-09-10T01:09:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.555+0000 7f7e90ff9700 1 -- 172.21.15.44:0/2252294305 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e8c00f040 con 0x7f7e941055b0 2023-09-10T01:09:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.555+0000 7f7e99f65700 1 -- 172.21.15.44:0/2252294305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e941a5170 con 0x7f7e941055b0 2023-09-10T01:09:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.555+0000 7f7e90ff9700 1 -- 172.21.15.44:0/2252294305 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c00cbc0 con 0x7f7e941055b0 2023-09-10T01:09:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.555+0000 7f7e99f65700 1 -- 172.21.15.44:0/2252294305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e941a5600 con 0x7f7e941055b0 2023-09-10T01:09:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.558+0000 7f7e90ff9700 1 -- 172.21.15.44:0/2252294305 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f7e8c007500 con 0x7f7e941055b0 2023-09-10T01:09:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.558+0000 7f7e90ff9700 1 --2- 172.21.15.44:0/2252294305 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7e7c0a07b0 0x7f7e7c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.559+0000 7f7e937fe700 1 --2- 172.21.15.44:0/2252294305 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7e7c0a07b0 0x7f7e7c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.559+0000 7f7e90ff9700 1 -- 172.21.15.44:0/2252294305 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f7e8c0cbeb0 con 0x7f7e941055b0 2023-09-10T01:09:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.559+0000 7f7ea612bb80 1 Processor -- start 2023-09-10T01:09:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.559+0000 7f7e937fe700 1 --2- 172.21.15.44:0/2252294305 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7e7c0a07b0 0x7f7e7c0a2c70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009c60 tx=0x7f7e84009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.559+0000 7f7ea612bb80 1 -- start start 2023-09-10T01:09:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.559+0000 7f7ea612bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834e21a0 0x5605834e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.559+0000 7f7ea612bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834dc300 0x5605834dc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.559+0000 7f7ea612bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 0x5605834e8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.559+0000 7f7ea612bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605834356d0 con 0x5605834dc300 2023-09-10T01:09:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.559+0000 7f7ea612bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560583432ff0 con 0x5605834e21a0 2023-09-10T01:09:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.559+0000 7f7ea612bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56058342e8e0 con 0x5605834dccc0 2023-09-10T01:09:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.560+0000 7f7e937fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834e21a0 0x5605834e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.560+0000 7f7e92ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834dc300 0x5605834dc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.560+0000 7f7e93fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 0x5605834e8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.560+0000 7f7e93fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 0x5605834e8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:34972/0 (socket says 172.21.15.44:34972) 2023-09-10T01:09:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.560+0000 7f7e937fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834e21a0 0x5605834e2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58058/0 (socket says 172.21.15.44:58058) 2023-09-10T01:09:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.560+0000 7f7e93fff700 1 -- 172.21.15.44:0/1004653902 learned_addr learned my addr 172.21.15.44:0/1004653902 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.560+0000 7f7e93fff700 1 -- 172.21.15.44:0/1004653902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834e21a0 msgr2=0x5605834e2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.560+0000 7f7e93fff700 1 --2- 172.21.15.44:0/1004653902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834e21a0 0x5605834e2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.560+0000 7f7e93fff700 1 -- 172.21.15.44:0/1004653902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834dc300 msgr2=0x5605834dc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.560+0000 7f7e93fff700 1 --2- 172.21.15.44:0/1004653902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834dc300 0x5605834dc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.560+0000 7f7e93fff700 1 -- 172.21.15.44:0/1004653902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56058325ad80 con 0x5605834dccc0 2023-09-10T01:09:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.560+0000 7f7e937fe700 1 --2- 172.21.15.44:0/1004653902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834e21a0 0x5605834e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:09:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.560+0000 7f7e93fff700 1 --2- 172.21.15.44:0/1004653902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 0x5605834e8f30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0a0aa0 tx=0x7f7e8c098c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6145c2de50e39a31 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.560+0000 7f7e79ffb700 1 -- 172.21.15.44:0/1004653902 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0996f0 con 0x5605834dccc0 2023-09-10T01:09:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.560+0000 7f7e79ffb700 1 -- 172.21.15.44:0/1004653902 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e8c098ff0 con 0x5605834dccc0 2023-09-10T01:09:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.560+0000 7f7e79ffb700 1 -- 172.21.15.44:0/1004653902 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c099250 con 0x5605834dccc0 2023-09-10T01:09:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.561+0000 7f7ea612bb80 1 -- 172.21.15.44:0/1004653902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 msgr2=0x5605834e8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.561+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/1004653902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 0x5605834e8f30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0a0aa0 tx=0x7f7e8c098c20 comp rx=0 tx=0).stop 2023-09-10T01:09:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.561+0000 7f7ea612bb80 1 -- 172.21.15.44:0/1004653902 shutdown_connections 2023-09-10T01:09:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.561+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/1004653902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 0x5605834e8f30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.561+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/1004653902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834e21a0 0x5605834e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.561+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/1004653902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834dc300 0x5605834dc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.561+0000 7f7ea612bb80 1 -- 172.21.15.44:0/1004653902 >> 172.21.15.44:0/1004653902 conn(0x560583451030 msgr2=0x5605834d7560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.561+0000 7f7ea612bb80 1 -- 172.21.15.44:0/1004653902 shutdown_connections 2023-09-10T01:09:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.561+0000 7f7ea612bb80 1 -- 172.21.15.44:0/1004653902 wait complete. 2023-09-10T01:09:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.562+0000 7f7ea612bb80 1 -- start start 2023-09-10T01:09:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.562+0000 7f7ea612bb80 1 Processor -- start 2023-09-10T01:09:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.562+0000 7f7ea612bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dc300 0x5605834e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.562+0000 7f7ea612bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834dccc0 0x5605834e6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.562+0000 7f7ea612bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834e21a0 0x5605834e67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.562+0000 7f7ea612bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605833ccb10 con 0x5605834e21a0 2023-09-10T01:09:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.562+0000 7f7ea612bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605833cc810 con 0x5605834dccc0 2023-09-10T01:09:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.562+0000 7f7ea612bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605832b08c0 con 0x5605834dc300 2023-09-10T01:09:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.562+0000 7f7ea612bb80 10 client.0 _ll_register_callbacks cb 0x560583316a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:09:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.562+0000 7f7e93fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834e21a0 0x5605834e67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.562+0000 7f7e937fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dc300 0x5605834e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.562+0000 7f7e93fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834e21a0 0x5605834e67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56328/0 (socket says 172.21.15.44:56328) 2023-09-10T01:09:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.562+0000 7f7e93fff700 1 -- 172.21.15.44:0/1630271058 learned_addr learned my addr 172.21.15.44:0/1630271058 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.562+0000 7f7e92ffd700 1 --2- 172.21.15.44:0/1630271058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834dccc0 0x5605834e6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.563+0000 7f7e93fff700 1 -- 172.21.15.44:0/1630271058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dc300 msgr2=0x5605834e8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.563+0000 7f7e93fff700 1 --2- 172.21.15.44:0/1630271058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dc300 0x5605834e8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.563+0000 7f7e93fff700 1 -- 172.21.15.44:0/1630271058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834dccc0 msgr2=0x5605834e6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.563+0000 7f7e93fff700 1 --2- 172.21.15.44:0/1630271058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834dccc0 0x5605834e6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.563+0000 7f7e93fff700 1 -- 172.21.15.44:0/1630271058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e84017070 con 0x5605834e21a0 2023-09-10T01:09:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.563+0000 7f7e93fff700 1 --2- 172.21.15.44:0/1630271058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834e21a0 0x5605834e67d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c094d70 tx=0x7f7e8c004b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db02e48649ca5385 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.563+0000 7f7e72ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:09:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.563+0000 7f7e72ffd700 1 -- 172.21.15.44:0/1630271058 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c09ce20 con 0x5605834e21a0 2023-09-10T01:09:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.563+0000 7f7ea612bb80 1 -- 172.21.15.44:0/1630271058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560583386490 con 0x5605834e21a0 2023-09-10T01:09:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.563+0000 7f7e72ffd700 1 -- 172.21.15.44:0/1630271058 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f7e8c0a0fd0 con 0x5605834e21a0 2023-09-10T01:09:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.563+0000 7f7e72ffd700 1 -- 172.21.15.44:0/1630271058 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e8c00cd80 con 0x5605834e21a0 2023-09-10T01:09:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.563+0000 7f7ea612bb80 10 client.18978 fetch_fsmap learned FSMap version 112 2023-09-10T01:09:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.563+0000 7f7ea612bb80 1 -- 172.21.15.44:0/1630271058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=4+}) v3 -- 0x7f7e941a5600 con 0x5605834e21a0 2023-09-10T01:09:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.563+0000 7f7e72ffd700 1 -- 172.21.15.44:0/1630271058 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560583386490 con 0x5605834e21a0 2023-09-10T01:09:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.563+0000 7f7e72ffd700 1 -- 172.21.15.44:0/1630271058 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 112) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f7e8c0a0bd0 con 0x5605834e21a0 2023-09-10T01:09:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.563+0000 7f7ea612bb80 10 client.18978 fetch_fsmap finished waiting for FSMap version 112 2023-09-10T01:09:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.564+0000 7f7ea612bb80 10 client.18978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:09:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.564+0000 7f7ea612bb80 20 client.18978 populate_metadata read hostname 'smithi044' 2023-09-10T01:09:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.564+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/1630271058 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5605834f6020 0x5605834f8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.564+0000 7f7ea612bb80 4 client.18978 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:09:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.564+0000 7f7ea612bb80 1 -- 172.21.15.44:0/1630271058 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560583443840 con 0x5605834f6020 2023-09-10T01:09:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.564+0000 7f7e937fe700 1 --2- 172.21.15.44:0/1630271058 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5605834f6020 0x5605834f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.564+0000 7f7e72ffd700 10 client.18978 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.564+0000 7f7e937fe700 1 --2- 172.21.15.44:0/1630271058 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5605834f6020 0x5605834f8440 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af783b203f05386d server_cookie=e9ad9bb762e3355 in_seq=0 out_seq=0 2023-09-10T01:09:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.565+0000 7f7e72ffd700 1 -- 172.21.15.44:0/1630271058 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560583443840 con 0x5605834f6020 2023-09-10T01:09:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.565+0000 7f7e72ffd700 10 client.18978 handle_command_reply: tid=0 2023-09-10T01:09:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.565+0000 7f7ea612bb80 1 client.18978 shutdown 2023-09-10T01:09:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.565+0000 7f7ea612bb80 1 -- 172.21.15.44:0/1630271058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834e21a0 msgr2=0x5605834e67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.565+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/1630271058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834e21a0 0x5605834e67d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c094d70 tx=0x7f7e8c004b10 comp rx=0 tx=0).stop 2023-09-10T01:09:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.565+0000 7f7ea612bb80 1 -- 172.21.15.44:0/1630271058 shutdown_connections 2023-09-10T01:09:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.565+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/1630271058 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5605834f6020 0x5605834f8440 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.565+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/1630271058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834e21a0 0x5605834e67d0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.565+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/1630271058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dc300 0x5605834e8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.565+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/1630271058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834dccc0 0x5605834e6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.565+0000 7f7e72ffd700 0 client.18978 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.565+0000 7f7ea612bb80 1 -- 172.21.15.44:0/1630271058 >> 172.21.15.44:0/1630271058 conn(0x560583451030 msgr2=0x5605834d67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.565+0000 7f7ea612bb80 1 -- 172.21.15.44:0/1630271058 shutdown_connections 2023-09-10T01:09:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.565+0000 7f7ea612bb80 1 -- 172.21.15.44:0/1630271058 wait complete. 2023-09-10T01:09:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.565+0000 7f7ea612bb80 20 client.18978 trim_cache size 0 max 16384 2023-09-10T01:09:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.581+0000 7f7ea612bb80 1 Processor -- start 2023-09-10T01:09:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.581+0000 7f7ea612bb80 1 -- start start 2023-09-10T01:09:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.581+0000 7f7ea612bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834dc300 0x5605834ed530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.581+0000 7f7ea612bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 0x5605834e8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.581+0000 7f7ea612bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834e21a0 0x5605834d6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.581+0000 7f7ea612bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605834e1ab0 con 0x5605834dc300 2023-09-10T01:09:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.581+0000 7f7ea612bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605834eda70 con 0x5605834e21a0 2023-09-10T01:09:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.581+0000 7f7ea612bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605834dd0c0 con 0x5605834dccc0 2023-09-10T01:09:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.581+0000 7f7e937fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834dc300 0x5605834ed530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.581+0000 7f7e937fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834dc300 0x5605834ed530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56332/0 (socket says 172.21.15.44:56332) 2023-09-10T01:09:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.581+0000 7f7e937fe700 1 -- 172.21.15.44:0/3723026172 learned_addr learned my addr 172.21.15.44:0/3723026172 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.581+0000 7f7e92ffd700 1 --2- 172.21.15.44:0/3723026172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 0x5605834e8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.581+0000 7f7e93fff700 1 --2- 172.21.15.44:0/3723026172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834e21a0 0x5605834d6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.582+0000 7f7e937fe700 1 -- 172.21.15.44:0/3723026172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 msgr2=0x5605834e8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.582+0000 7f7e937fe700 1 --2- 172.21.15.44:0/3723026172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 0x5605834e8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.582+0000 7f7e937fe700 1 -- 172.21.15.44:0/3723026172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834e21a0 msgr2=0x5605834d6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.582+0000 7f7e937fe700 1 --2- 172.21.15.44:0/3723026172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834e21a0 0x5605834d6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.582+0000 7f7e937fe700 1 -- 172.21.15.44:0/3723026172 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560583243c00 con 0x5605834dc300 2023-09-10T01:09:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.582+0000 7f7e937fe700 1 --2- 172.21.15.44:0/3723026172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834dc300 0x5605834ed530 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f7e840097c0 tx=0x7f7e84014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb28f8ba712318c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.582+0000 7f7e7a7fc700 1 -- 172.21.15.44:0/3723026172 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84022030 con 0x5605834dc300 2023-09-10T01:09:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.582+0000 7f7e7a7fc700 1 -- 172.21.15.44:0/3723026172 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e84023040 con 0x5605834dc300 2023-09-10T01:09:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.582+0000 7f7e7a7fc700 1 -- 172.21.15.44:0/3723026172 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8402b5d0 con 0x5605834dc300 2023-09-10T01:09:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.583+0000 7f7ea612bb80 1 -- 172.21.15.44:0/3723026172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834dc300 msgr2=0x5605834ed530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.583+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/3723026172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834dc300 0x5605834ed530 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f7e840097c0 tx=0x7f7e84014000 comp rx=0 tx=0).stop 2023-09-10T01:09:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.583+0000 7f7ea612bb80 1 -- 172.21.15.44:0/3723026172 shutdown_connections 2023-09-10T01:09:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.583+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/3723026172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834e21a0 0x5605834d6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.583+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/3723026172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834dc300 0x5605834ed530 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.583+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/3723026172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 0x5605834e8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.583+0000 7f7ea612bb80 1 -- 172.21.15.44:0/3723026172 >> 172.21.15.44:0/3723026172 conn(0x560583451030 msgr2=0x5605834e0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.583+0000 7f7ea612bb80 1 -- 172.21.15.44:0/3723026172 shutdown_connections 2023-09-10T01:09:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.583+0000 7f7ea612bb80 1 -- 172.21.15.44:0/3723026172 wait complete. 2023-09-10T01:09:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.583+0000 7f7ea612bb80 1 -- start start 2023-09-10T01:09:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.583+0000 7f7ea612bb80 1 Processor -- start 2023-09-10T01:09:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7ea612bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834dc300 0x560583543510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7ea612bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 0x560583543a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7ea612bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834e21a0 0x56058353dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7ea612bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56058353e3f0 con 0x5605834e21a0 2023-09-10T01:09:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7ea612bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56058353e560 con 0x5605834dc300 2023-09-10T01:09:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7ea612bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56058353e720 con 0x5605834dccc0 2023-09-10T01:09:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7ea612bb80 10 client.0 _ll_register_callbacks cb 0x7f7e84005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:09:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7e93fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834e21a0 0x56058353dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7e92ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 0x560583543a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7e937fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834dc300 0x560583543510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7e93fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834e21a0 0x56058353dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56342/0 (socket says 172.21.15.44:56342) 2023-09-10T01:09:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7e93fff700 1 -- 172.21.15.44:0/2030153600 learned_addr learned my addr 172.21.15.44:0/2030153600 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7e93fff700 1 -- 172.21.15.44:0/2030153600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 msgr2=0x560583543a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7e93fff700 1 --2- 172.21.15.44:0/2030153600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 0x560583543a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7e93fff700 1 -- 172.21.15.44:0/2030153600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834dc300 msgr2=0x560583543510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7e93fff700 1 --2- 172.21.15.44:0/2030153600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834dc300 0x560583543510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7e93fff700 1 -- 172.21.15.44:0/2030153600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e84004160 con 0x5605834e21a0 2023-09-10T01:09:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7e937fe700 1 --2- 172.21.15.44:0/2030153600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834dc300 0x560583543510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:09:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7e93fff700 1 --2- 172.21.15.44:0/2030153600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834e21a0 0x56058353dda0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c004b80 tx=0x7f7e8c098b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3951f0637fb3255b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7e72ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:09:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7e72ffd700 1 -- 172.21.15.44:0/2030153600 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0a0d40 con 0x5605834e21a0 2023-09-10T01:09:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7ea612bb80 1 -- 172.21.15.44:0/2030153600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e8c098ff0 con 0x5605834e21a0 2023-09-10T01:09:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.584+0000 7f7e72ffd700 1 -- 172.21.15.44:0/2030153600 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f7e8c0a0fd0 con 0x5605834e21a0 2023-09-10T01:09:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.585+0000 7f7e72ffd700 1 -- 172.21.15.44:0/2030153600 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e8c098950 con 0x5605834e21a0 2023-09-10T01:09:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.585+0000 7f7ea612bb80 10 client.18984 fetch_fsmap learned FSMap version 112 2023-09-10T01:09:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.585+0000 7f7ea612bb80 1 -- 172.21.15.44:0/2030153600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=4+}) v3 -- 0x5605832811f0 con 0x5605834e21a0 2023-09-10T01:09:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.585+0000 7f7e72ffd700 1 -- 172.21.15.44:0/2030153600 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e8c098ff0 con 0x5605834e21a0 2023-09-10T01:09:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.585+0000 7f7e72ffd700 1 -- 172.21.15.44:0/2030153600 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 112) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f7e8c099250 con 0x5605834e21a0 2023-09-10T01:09:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.585+0000 7f7ea612bb80 10 client.18984 fetch_fsmap finished waiting for FSMap version 112 2023-09-10T01:09:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.585+0000 7f7ea612bb80 10 client.18984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:09:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.585+0000 7f7ea612bb80 20 client.18984 populate_metadata read hostname 'smithi044' 2023-09-10T01:09:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.585+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/2030153600 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56058354aea0 0x56058354d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.585+0000 7f7ea612bb80 4 client.18984 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:09:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.585+0000 7f7ea612bb80 1 -- 172.21.15.44:0/2030153600 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5605834278c0 con 0x56058354aea0 2023-09-10T01:09:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.585+0000 7f7e937fe700 1 --2- 172.21.15.44:0/2030153600 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56058354aea0 0x56058354d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.586+0000 7f7e937fe700 1 --2- 172.21.15.44:0/2030153600 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56058354aea0 0x56058354d290 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26c9df49839dd2d5 server_cookie=9bf9f8c0808ed4ae in_seq=0 out_seq=0 2023-09-10T01:09:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.586+0000 7f7e72ffd700 10 client.18984 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.598+0000 7f7e72ffd700 1 -- 172.21.15.44:0/2030153600 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5605834278c0 con 0x56058354aea0 2023-09-10T01:09:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.598+0000 7f7e72ffd700 10 client.18984 handle_command_reply: tid=0 2023-09-10T01:09:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.598+0000 7f7ea612bb80 1 client.18984 shutdown 2023-09-10T01:09:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.598+0000 7f7ea612bb80 1 -- 172.21.15.44:0/2030153600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834e21a0 msgr2=0x56058353dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.598+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/2030153600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834e21a0 0x56058353dda0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c004b80 tx=0x7f7e8c098b30 comp rx=0 tx=0).stop 2023-09-10T01:09:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.598+0000 7f7ea612bb80 1 -- 172.21.15.44:0/2030153600 shutdown_connections 2023-09-10T01:09:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.598+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/2030153600 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56058354aea0 0x56058354d290 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.598+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/2030153600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605834e21a0 0x56058353dda0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.598+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/2030153600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605834dc300 0x560583543510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.598+0000 7f7ea612bb80 1 --2- 172.21.15.44:0/2030153600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605834dccc0 0x560583543a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.598+0000 7f7ea612bb80 1 -- 172.21.15.44:0/2030153600 >> 172.21.15.44:0/2030153600 conn(0x560583451030 msgr2=0x5605834db2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.598+0000 7f7e72ffd700 0 client.18984 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.598+0000 7f7ea612bb80 1 -- 172.21.15.44:0/2030153600 shutdown_connections 2023-09-10T01:09:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.598+0000 7f7ea612bb80 1 -- 172.21.15.44:0/2030153600 wait complete. 2023-09-10T01:09:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.598+0000 7f7ea612bb80 20 client.18984 trim_cache size 0 max 16384 2023-09-10T01:09:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:09:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:09:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:09:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:09:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.599+0000 7f7e99f65700 1 -- 172.21.15.44:0/2252294305 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7e7c0a07b0 msgr2=0x7f7e7c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.599+0000 7f7e99f65700 1 --2- 172.21.15.44:0/2252294305 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7e7c0a07b0 0x7f7e7c0a2c70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009c60 tx=0x7f7e84009260 comp rx=0 tx=0).stop 2023-09-10T01:09:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.599+0000 7f7e99f65700 1 -- 172.21.15.44:0/2252294305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e941055b0 msgr2=0x7f7e941a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.599+0000 7f7e99f65700 1 --2- 172.21.15.44:0/2252294305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e941055b0 0x7f7e941a4700 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00ef30 tx=0x7f7e8c00c3a0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.599+0000 7f7e99f65700 1 -- 172.21.15.44:0/2252294305 shutdown_connections 2023-09-10T01:09:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.599+0000 7f7e99f65700 1 --2- 172.21.15.44:0/2252294305 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7e7c0a07b0 0x7f7e7c0a2c70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.599+0000 7f7e99f65700 1 --2- 172.21.15.44:0/2252294305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e941055b0 0x7f7e941a4700 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.599+0000 7f7e99f65700 1 --2- 172.21.15.44:0/2252294305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e940ff760 0x7f7e941012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.599+0000 7f7e99f65700 1 --2- 172.21.15.44:0/2252294305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e94100120 0x7f7e941017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.599+0000 7f7e99f65700 1 -- 172.21.15.44:0/2252294305 >> 172.21.15.44:0/2252294305 conn(0x7f7e94074f30 msgr2=0x7f7e94067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.599+0000 7f7e99f65700 1 -- 172.21.15.44:0/2252294305 shutdown_connections 2023-09-10T01:09:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:16.599+0000 7f7e99f65700 1 -- 172.21.15.44:0/2252294305 wait complete. 2023-09-10T01:09:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:16 smithi131 ceph-mon[103713]: pgmap v910: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2023-09-10T01:09:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:16 smithi044 ceph-mon[98676]: pgmap v910: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2023-09-10T01:09:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:16 smithi099 ceph-mon[102692]: pgmap v910: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2023-09-10T01:09:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:18 smithi131 ceph-mon[103713]: pgmap v911: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2023-09-10T01:09:19.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:18 smithi044 ceph-mon[98676]: pgmap v911: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2023-09-10T01:09:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:18 smithi099 ceph-mon[102692]: pgmap v911: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2023-09-10T01:09:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:20 smithi131 ceph-mon[103713]: pgmap v912: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2023-09-10T01:09:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:20 smithi044 ceph-mon[98676]: pgmap v912: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2023-09-10T01:09:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:20 smithi099 ceph-mon[102692]: pgmap v912: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2023-09-10T01:09:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:22 smithi131 ceph-mon[103713]: pgmap v913: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 167 op/s 2023-09-10T01:09:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:22 smithi044 ceph-mon[98676]: pgmap v913: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 167 op/s 2023-09-10T01:09:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:22 smithi099 ceph-mon[102692]: pgmap v913: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 167 op/s 2023-09-10T01:09:25.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:24 smithi131 ceph-mon[103713]: pgmap v914: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2023-09-10T01:09:25.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:24 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:09:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:24 smithi044 ceph-mon[98676]: pgmap v914: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2023-09-10T01:09:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:24 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:09:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:24 smithi099 ceph-mon[102692]: pgmap v914: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2023-09-10T01:09:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:24 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:09:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:26 smithi131 ceph-mon[103713]: pgmap v915: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 217 op/s 2023-09-10T01:09:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:26 smithi044 ceph-mon[98676]: pgmap v915: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 217 op/s 2023-09-10T01:09:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:26 smithi099 ceph-mon[102692]: pgmap v915: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 217 op/s 2023-09-10T01:09:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:28 smithi131 ceph-mon[103713]: pgmap v916: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 964 KiB/s wr, 139 op/s 2023-09-10T01:09:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:28 smithi044 ceph-mon[98676]: pgmap v916: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 964 KiB/s wr, 139 op/s 2023-09-10T01:09:29.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:28 smithi099 ceph-mon[102692]: pgmap v916: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 964 KiB/s wr, 139 op/s 2023-09-10T01:09:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.744+0000 7f1191eab700 1 -- 172.21.15.44:0/1162763339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f118c1055b0 msgr2=0x7f118c105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.744+0000 7f1191eab700 1 --2- 172.21.15.44:0/1162763339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f118c1055b0 0x7f118c105990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f117c00ba80 tx=0x7f117c00bd90 comp rx=0 tx=0).stop 2023-09-10T01:09:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.745+0000 7f1191eab700 1 -- 172.21.15.44:0/1162763339 shutdown_connections 2023-09-10T01:09:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.745+0000 7f1191eab700 1 --2- 172.21.15.44:0/1162763339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f118c100120 0x7f118c10c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.745+0000 7f1191eab700 1 --2- 172.21.15.44:0/1162763339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f118c1055b0 0x7f118c105990 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.745+0000 7f1191eab700 1 --2- 172.21.15.44:0/1162763339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f118c0ff760 0x7f118c0ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.745+0000 7f1191eab700 1 -- 172.21.15.44:0/1162763339 >> 172.21.15.44:0/1162763339 conn(0x7f118c074f30 msgr2=0x7f118c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.745+0000 7f1191eab700 1 -- 172.21.15.44:0/1162763339 shutdown_connections 2023-09-10T01:09:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.745+0000 7f1191eab700 1 -- 172.21.15.44:0/1162763339 wait complete. 2023-09-10T01:09:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.745+0000 7f1191eab700 1 Processor -- start 2023-09-10T01:09:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.745+0000 7f1191eab700 1 -- start start 2023-09-10T01:09:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.745+0000 7f1191eab700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f118c0ff760 0x7f118c194370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f1191eab700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f118c100120 0x7f118c1948b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f1191eab700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f118c1055b0 0x7f118c198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f1191eab700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c199560 con 0x7f118c1055b0 2023-09-10T01:09:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f1191eab700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c1996d0 con 0x7f118c0ff760 2023-09-10T01:09:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f1191eab700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c199890 con 0x7f118c100120 2023-09-10T01:09:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f118bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f118c1055b0 0x7f118c198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f118b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f118c0ff760 0x7f118c194370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f118b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f118c0ff760 0x7f118c194370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58092/0 (socket says 172.21.15.44:58092) 2023-09-10T01:09:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f118b7fe700 1 -- 172.21.15.44:0/3915277443 learned_addr learned my addr 172.21.15.44:0/3915277443 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f118bfff700 1 --2- 172.21.15.44:0/3915277443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f118c1055b0 0x7f118c198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56354/0 (socket says 172.21.15.44:56354) 2023-09-10T01:09:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f118affd700 1 --2- 172.21.15.44:0/3915277443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f118c100120 0x7f118c1948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f118b7fe700 1 -- 172.21.15.44:0/3915277443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f118c100120 msgr2=0x7f118c1948b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f118b7fe700 1 --2- 172.21.15.44:0/3915277443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f118c100120 0x7f118c1948b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f118b7fe700 1 -- 172.21.15.44:0/3915277443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f118c1055b0 msgr2=0x7f118c198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f118b7fe700 1 --2- 172.21.15.44:0/3915277443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f118c1055b0 0x7f118c198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f118b7fe700 1 -- 172.21.15.44:0/3915277443 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c00b730 con 0x7f118c0ff760 2023-09-10T01:09:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f118b7fe700 1 --2- 172.21.15.44:0/3915277443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f118c0ff760 0x7f118c194370 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f117c0147e0 tx=0x7f117c014810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.746+0000 7f1188ff9700 1 -- 172.21.15.44:0/3915277443 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c010050 con 0x7f118c0ff760 2023-09-10T01:09:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.747+0000 7f1188ff9700 1 -- 172.21.15.44:0/3915277443 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f117c0056d0 con 0x7f118c0ff760 2023-09-10T01:09:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.747+0000 7f1191eab700 1 -- 172.21.15.44:0/3915277443 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c199b10 con 0x7f118c0ff760 2023-09-10T01:09:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.747+0000 7f1188ff9700 1 -- 172.21.15.44:0/3915277443 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c01d3f0 con 0x7f118c0ff760 2023-09-10T01:09:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.747+0000 7f1191eab700 1 -- 172.21.15.44:0/3915277443 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c1a13b0 con 0x7f118c0ff760 2023-09-10T01:09:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.749+0000 7f1188ff9700 1 -- 172.21.15.44:0/3915277443 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f117c003680 con 0x7f118c0ff760 2023-09-10T01:09:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.750+0000 7f1188ff9700 1 --2- 172.21.15.44:0/3915277443 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f11740a0830 0x7f11740a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.750+0000 7f118affd700 1 --2- 172.21.15.44:0/3915277443 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f11740a0830 0x7f11740a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.750+0000 7f1188ff9700 1 -- 172.21.15.44:0/3915277443 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f117c0cf9e0 con 0x7f118c0ff760 2023-09-10T01:09:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.750+0000 7f118affd700 1 --2- 172.21.15.44:0/3915277443 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f11740a0830 0x7f11740a2cf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f118c195b90 tx=0x7f1180009310 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f119e071b80 1 Processor -- start 2023-09-10T01:09:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f119e071b80 1 -- start start 2023-09-10T01:09:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f119e071b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c541a0 0x5633c0c54580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f119e071b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4e300 0x5633c0c4e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f119e071b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c4ecc0 0x5633c0c5af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f119e071b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633c0ba76d0 con 0x5633c0c541a0 2023-09-10T01:09:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f119e071b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633c0ba4ff0 con 0x5633c0c4e300 2023-09-10T01:09:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f119e071b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633c0ba08e0 con 0x5633c0c4ecc0 2023-09-10T01:09:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f118b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c541a0 0x5633c0c54580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f118b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c541a0 0x5633c0c54580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56360/0 (socket says 172.21.15.44:56360) 2023-09-10T01:09:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f118b7fe700 1 -- 172.21.15.44:0/3548084604 learned_addr learned my addr 172.21.15.44:0/3548084604 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f118affd700 1 --2- 172.21.15.44:0/3548084604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4e300 0x5633c0c4e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f118bfff700 1 --2- 172.21.15.44:0/3548084604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c4ecc0 0x5633c0c5af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f118b7fe700 1 -- 172.21.15.44:0/3548084604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c4ecc0 msgr2=0x5633c0c5af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f118b7fe700 1 --2- 172.21.15.44:0/3548084604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c4ecc0 0x5633c0c5af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f118b7fe700 1 -- 172.21.15.44:0/3548084604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4e300 msgr2=0x5633c0c4e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f118b7fe700 1 --2- 172.21.15.44:0/3548084604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4e300 0x5633c0c4e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.751+0000 7f118b7fe700 1 -- 172.21.15.44:0/3548084604 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633c09ccd80 con 0x5633c0c541a0 2023-09-10T01:09:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.752+0000 7f118b7fe700 1 --2- 172.21.15.44:0/3548084604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c541a0 0x5633c0c54580 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f117c0a45d0 tx=0x7f117c09c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39df0e9c5ef50d66 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.752+0000 7f1171ffb700 1 -- 172.21.15.44:0/3548084604 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c0a0950 con 0x5633c0c541a0 2023-09-10T01:09:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.752+0000 7f1171ffb700 1 -- 172.21.15.44:0/3548084604 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f117c0a4b00 con 0x5633c0c541a0 2023-09-10T01:09:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.752+0000 7f1171ffb700 1 -- 172.21.15.44:0/3548084604 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c09c010 con 0x5633c0c541a0 2023-09-10T01:09:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.753+0000 7f119e071b80 1 -- 172.21.15.44:0/3548084604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c541a0 msgr2=0x5633c0c54580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.753+0000 7f119e071b80 1 --2- 172.21.15.44:0/3548084604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c541a0 0x5633c0c54580 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f117c0a45d0 tx=0x7f117c09c790 comp rx=0 tx=0).stop 2023-09-10T01:09:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.753+0000 7f119e071b80 1 -- 172.21.15.44:0/3548084604 shutdown_connections 2023-09-10T01:09:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.753+0000 7f119e071b80 1 --2- 172.21.15.44:0/3548084604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c4ecc0 0x5633c0c5af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.753+0000 7f119e071b80 1 --2- 172.21.15.44:0/3548084604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c541a0 0x5633c0c54580 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.753+0000 7f119e071b80 1 --2- 172.21.15.44:0/3548084604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4e300 0x5633c0c4e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.753+0000 7f119e071b80 1 -- 172.21.15.44:0/3548084604 >> 172.21.15.44:0/3548084604 conn(0x5633c0bc3030 msgr2=0x5633c0c49560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.753+0000 7f119e071b80 1 -- 172.21.15.44:0/3548084604 shutdown_connections 2023-09-10T01:09:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.753+0000 7f119e071b80 1 -- 172.21.15.44:0/3548084604 wait complete. 2023-09-10T01:09:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.753+0000 7f119e071b80 1 -- start start 2023-09-10T01:09:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.753+0000 7f119e071b80 1 Processor -- start 2023-09-10T01:09:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f119e071b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4e300 0x5633c0c5dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f119e071b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4ecc0 0x5633c0c5e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f119e071b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c541a0 0x5633c0c57ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f119e071b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633c0b3eb10 con 0x5633c0c4ecc0 2023-09-10T01:09:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f119e071b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633c0b3e810 con 0x5633c0c4e300 2023-09-10T01:09:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f119e071b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633c0a228c0 con 0x5633c0c541a0 2023-09-10T01:09:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f119e071b80 10 client.0 _ll_register_callbacks cb 0x5633c0a88a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:09:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f118affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4ecc0 0x5633c0c5e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f118b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4e300 0x5633c0c5dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f118affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4ecc0 0x5633c0c5e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56368/0 (socket says 172.21.15.44:56368) 2023-09-10T01:09:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f118affd700 1 -- 172.21.15.44:0/673006376 learned_addr learned my addr 172.21.15.44:0/673006376 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f118bfff700 1 --2- 172.21.15.44:0/673006376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c541a0 0x5633c0c57ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f118affd700 1 -- 172.21.15.44:0/673006376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c541a0 msgr2=0x5633c0c57ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f118affd700 1 --2- 172.21.15.44:0/673006376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c541a0 0x5633c0c57ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f118affd700 1 -- 172.21.15.44:0/673006376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4e300 msgr2=0x5633c0c5dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f118affd700 1 --2- 172.21.15.44:0/673006376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4e300 0x5633c0c5dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f118affd700 1 -- 172.21.15.44:0/673006376 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c1a13b0 con 0x5633c0c4ecc0 2023-09-10T01:09:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f118b7fe700 1 --2- 172.21.15.44:0/673006376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4e300 0x5633c0c5dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:09:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f118bfff700 1 --2- 172.21.15.44:0/673006376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c541a0 0x5633c0c57ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:09:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f118affd700 1 --2- 172.21.15.44:0/673006376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4ecc0 0x5633c0c5e190 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1180003900 tx=0x7f11800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93821bfe3826064 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f116affd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:09:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.754+0000 7f116affd700 1 -- 172.21.15.44:0/673006376 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180004a30 con 0x5633c0c4ecc0 2023-09-10T01:09:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.755+0000 7f116affd700 1 -- 172.21.15.44:0/673006376 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1180004b90 con 0x5633c0c4ecc0 2023-09-10T01:09:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.755+0000 7f119e071b80 1 -- 172.21.15.44:0/673006376 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633c0af8490 con 0x5633c0c4ecc0 2023-09-10T01:09:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.755+0000 7f116affd700 1 -- 172.21.15.44:0/673006376 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1180004d50 con 0x5633c0c4ecc0 2023-09-10T01:09:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.755+0000 7f119e071b80 10 client.18996 fetch_fsmap learned FSMap version 114 2023-09-10T01:09:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.755+0000 7f119e071b80 1 -- 172.21.15.44:0/673006376 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=4+}) v3 -- 0x5633c09b5c00 con 0x5633c0c4ecc0 2023-09-10T01:09:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.755+0000 7f116affd700 1 -- 172.21.15.44:0/673006376 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5633c0af8490 con 0x5633c0c4ecc0 2023-09-10T01:09:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.755+0000 7f116affd700 1 -- 172.21.15.44:0/673006376 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 114) v1 ==== 7347+0+0 (secure 0 0 0) 0x7f11800052c0 con 0x5633c0c4ecc0 2023-09-10T01:09:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.755+0000 7f119e071b80 10 client.18996 fetch_fsmap finished waiting for FSMap version 114 2023-09-10T01:09:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.755+0000 7f119e071b80 10 client.18996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:09:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.755+0000 7f119e071b80 20 client.18996 populate_metadata read hostname 'smithi044' 2023-09-10T01:09:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.755+0000 7f119e071b80 1 --2- 172.21.15.44:0/673006376 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5633c0c792d0 0x5633c0c7b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.756+0000 7f119e071b80 4 client.18996 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:09:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.756+0000 7f119e071b80 1 -- 172.21.15.44:0/673006376 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633c0bb5840 con 0x5633c0c792d0 2023-09-10T01:09:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.756+0000 7f118b7fe700 1 --2- 172.21.15.44:0/673006376 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5633c0c792d0 0x5633c0c7b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.756+0000 7f118b7fe700 1 --2- 172.21.15.44:0/673006376 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5633c0c792d0 0x5633c0c7b6f0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47126e9af9ee994e server_cookie=33fedab871985b77 in_seq=0 out_seq=0 2023-09-10T01:09:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.756+0000 7f116affd700 10 client.18996 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.757+0000 7f116affd700 1 -- 172.21.15.44:0/673006376 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5633c0bb5840 con 0x5633c0c792d0 2023-09-10T01:09:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.757+0000 7f116affd700 10 client.18996 handle_command_reply: tid=0 2023-09-10T01:09:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.757+0000 7f119e071b80 1 client.18996 shutdown 2023-09-10T01:09:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.757+0000 7f119e071b80 1 -- 172.21.15.44:0/673006376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4ecc0 msgr2=0x5633c0c5e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.757+0000 7f119e071b80 1 --2- 172.21.15.44:0/673006376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4ecc0 0x5633c0c5e190 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1180003900 tx=0x7f11800047c0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.757+0000 7f119e071b80 1 -- 172.21.15.44:0/673006376 shutdown_connections 2023-09-10T01:09:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.757+0000 7f119e071b80 1 --2- 172.21.15.44:0/673006376 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5633c0c792d0 0x5633c0c7b6f0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.757+0000 7f119e071b80 1 --2- 172.21.15.44:0/673006376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c541a0 0x5633c0c57ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.757+0000 7f119e071b80 1 --2- 172.21.15.44:0/673006376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4e300 0x5633c0c5dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.757+0000 7f116affd700 0 client.18996 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.757+0000 7f119e071b80 1 --2- 172.21.15.44:0/673006376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4ecc0 0x5633c0c5e190 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.757+0000 7f119e071b80 1 -- 172.21.15.44:0/673006376 >> 172.21.15.44:0/673006376 conn(0x5633c0bc3030 msgr2=0x5633c0c487a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.757+0000 7f119e071b80 1 -- 172.21.15.44:0/673006376 shutdown_connections 2023-09-10T01:09:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.757+0000 7f119e071b80 1 -- 172.21.15.44:0/673006376 wait complete. 2023-09-10T01:09:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.757+0000 7f119e071b80 20 client.18996 trim_cache size 0 max 16384 2023-09-10T01:09:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.775+0000 7f119e071b80 1 Processor -- start 2023-09-10T01:09:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.775+0000 7f119e071b80 1 -- start start 2023-09-10T01:09:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.775+0000 7f119e071b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c4e300 0x5633c0c539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.775+0000 7f119e071b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4ecc0 0x5633c0c4da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.775+0000 7f119e071b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c541a0 0x5633c0c48f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.775+0000 7f119e071b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633c0c40f50 con 0x5633c0c4ecc0 2023-09-10T01:09:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.775+0000 7f119e071b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633c0c767a0 con 0x5633c0c541a0 2023-09-10T01:09:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.775+0000 7f119e071b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633c0c4f580 con 0x5633c0c4e300 2023-09-10T01:09:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.775+0000 7f118affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4ecc0 0x5633c0c4da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.775+0000 7f118affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4ecc0 0x5633c0c4da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56376/0 (socket says 172.21.15.44:56376) 2023-09-10T01:09:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.775+0000 7f118affd700 1 -- 172.21.15.44:0/2765002081 learned_addr learned my addr 172.21.15.44:0/2765002081 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.775+0000 7f118bfff700 1 --2- 172.21.15.44:0/2765002081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c541a0 0x5633c0c48f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.775+0000 7f118b7fe700 1 --2- 172.21.15.44:0/2765002081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c4e300 0x5633c0c539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.776+0000 7f118affd700 1 -- 172.21.15.44:0/2765002081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c4e300 msgr2=0x5633c0c539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.776+0000 7f118affd700 1 --2- 172.21.15.44:0/2765002081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c4e300 0x5633c0c539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.776+0000 7f118affd700 1 -- 172.21.15.44:0/2765002081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c541a0 msgr2=0x5633c0c48f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.776+0000 7f118affd700 1 --2- 172.21.15.44:0/2765002081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c541a0 0x5633c0c48f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.776+0000 7f118affd700 1 -- 172.21.15.44:0/2765002081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633c09f31f0 con 0x5633c0c4ecc0 2023-09-10T01:09:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.776+0000 7f118bfff700 1 --2- 172.21.15.44:0/2765002081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c541a0 0x5633c0c48f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:09:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.776+0000 7f118b7fe700 1 --2- 172.21.15.44:0/2765002081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c4e300 0x5633c0c539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:09:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.776+0000 7f118affd700 1 --2- 172.21.15.44:0/2765002081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4ecc0 0x5633c0c4da00 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1180005620 tx=0x7f1180011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34931dae7ab7dae server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.776+0000 7f11727fc700 1 -- 172.21.15.44:0/2765002081 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180038070 con 0x5633c0c4ecc0 2023-09-10T01:09:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.776+0000 7f11727fc700 1 -- 172.21.15.44:0/2765002081 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1180009ec0 con 0x5633c0c4ecc0 2023-09-10T01:09:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.776+0000 7f11727fc700 1 -- 172.21.15.44:0/2765002081 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180034800 con 0x5633c0c4ecc0 2023-09-10T01:09:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.777+0000 7f119e071b80 1 -- 172.21.15.44:0/2765002081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4ecc0 msgr2=0x5633c0c4da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.777+0000 7f119e071b80 1 --2- 172.21.15.44:0/2765002081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4ecc0 0x5633c0c4da00 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1180005620 tx=0x7f1180011bb0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.777+0000 7f119e071b80 1 -- 172.21.15.44:0/2765002081 shutdown_connections 2023-09-10T01:09:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.777+0000 7f119e071b80 1 --2- 172.21.15.44:0/2765002081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c541a0 0x5633c0c48f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.777+0000 7f119e071b80 1 --2- 172.21.15.44:0/2765002081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c4e300 0x5633c0c539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.777+0000 7f119e071b80 1 --2- 172.21.15.44:0/2765002081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4ecc0 0x5633c0c4da00 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.777+0000 7f119e071b80 1 -- 172.21.15.44:0/2765002081 >> 172.21.15.44:0/2765002081 conn(0x5633c0bc3030 msgr2=0x5633c0c4d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.777+0000 7f119e071b80 1 -- 172.21.15.44:0/2765002081 shutdown_connections 2023-09-10T01:09:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.777+0000 7f119e071b80 1 -- 172.21.15.44:0/2765002081 wait complete. 2023-09-10T01:09:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.777+0000 7f119e071b80 1 -- start start 2023-09-10T01:09:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.777+0000 7f119e071b80 1 Processor -- start 2023-09-10T01:09:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f119e071b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4e300 0x5633c0cc1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f119e071b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4ecc0 0x5633c0cc1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f119e071b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c541a0 0x5633c0cc5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f119e071b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633c0cc56b0 con 0x5633c0c4e300 2023-09-10T01:09:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f119e071b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633c0cc5820 con 0x5633c0c4ecc0 2023-09-10T01:09:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f119e071b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633c0cc59e0 con 0x5633c0c541a0 2023-09-10T01:09:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f119e071b80 10 client.0 _ll_register_callbacks cb 0x7f1180004440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:09:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f118affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4ecc0 0x5633c0cc1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f118affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4ecc0 0x5633c0cc1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58120/0 (socket says 172.21.15.44:58120) 2023-09-10T01:09:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f118affd700 1 -- 172.21.15.44:0/2039814248 learned_addr learned my addr 172.21.15.44:0/2039814248 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f118bfff700 1 --2- 172.21.15.44:0/2039814248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c541a0 0x5633c0cc5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f118b7fe700 1 --2- 172.21.15.44:0/2039814248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4e300 0x5633c0cc1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f118affd700 1 -- 172.21.15.44:0/2039814248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c541a0 msgr2=0x5633c0cc5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f118affd700 1 --2- 172.21.15.44:0/2039814248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c541a0 0x5633c0cc5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f118affd700 1 -- 172.21.15.44:0/2039814248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4e300 msgr2=0x5633c0cc1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f118affd700 1 --2- 172.21.15.44:0/2039814248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4e300 0x5633c0cc1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f118affd700 1 -- 172.21.15.44:0/2039814248 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633c09ccd80 con 0x5633c0c4ecc0 2023-09-10T01:09:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f118bfff700 1 --2- 172.21.15.44:0/2039814248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c541a0 0x5633c0cc5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:09:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f118affd700 1 --2- 172.21.15.44:0/2039814248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4ecc0 0x5633c0cc1780 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1180005860 tx=0x7f118001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cc5cfdc0a8e15f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.778+0000 7f116affd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:09:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.779+0000 7f116affd700 1 -- 172.21.15.44:0/2039814248 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180038040 con 0x5633c0c4ecc0 2023-09-10T01:09:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.779+0000 7f116affd700 1 -- 172.21.15.44:0/2039814248 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1180034890 con 0x5633c0c4ecc0 2023-09-10T01:09:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.779+0000 7f119e071b80 1 -- 172.21.15.44:0/2039814248 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633c0b998c0 con 0x5633c0c4ecc0 2023-09-10T01:09:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.779+0000 7f116affd700 1 -- 172.21.15.44:0/2039814248 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1180009ec0 con 0x5633c0c4ecc0 2023-09-10T01:09:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.779+0000 7f119e071b80 10 client.28664 fetch_fsmap learned FSMap version 114 2023-09-10T01:09:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.779+0000 7f119e071b80 1 -- 172.21.15.44:0/2039814248 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=4+}) v3 -- 0x7f11840127f0 con 0x5633c0c4ecc0 2023-09-10T01:09:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.779+0000 7f116affd700 1 -- 172.21.15.44:0/2039814248 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5633c0b998c0 con 0x5633c0c4ecc0 2023-09-10T01:09:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.779+0000 7f116affd700 1 -- 172.21.15.44:0/2039814248 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 114) v1 ==== 7347+0+0 (secure 0 0 0) 0x7f1180027a50 con 0x5633c0c4ecc0 2023-09-10T01:09:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.779+0000 7f119e071b80 10 client.28664 fetch_fsmap finished waiting for FSMap version 114 2023-09-10T01:09:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.779+0000 7f119e071b80 10 client.28664 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:09:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.779+0000 7f119e071b80 20 client.28664 populate_metadata read hostname 'smithi044' 2023-09-10T01:09:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.779+0000 7f119e071b80 1 --2- 172.21.15.44:0/2039814248 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5633c0cd2510 0x5633c0cd4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.779+0000 7f119e071b80 4 client.28664 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:09:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.779+0000 7f119e071b80 1 -- 172.21.15.44:0/2039814248 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5633c0b5a050 con 0x5633c0cd2510 2023-09-10T01:09:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.779+0000 7f118b7fe700 1 --2- 172.21.15.44:0/2039814248 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5633c0cd2510 0x5633c0cd4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.780+0000 7f116affd700 10 client.28664 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.780+0000 7f118b7fe700 1 --2- 172.21.15.44:0/2039814248 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5633c0cd2510 0x5633c0cd4930 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29702fa362cf1d91 server_cookie=9fe049d77ca10e54 in_seq=0 out_seq=0 2023-09-10T01:09:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.791+0000 7f116affd700 1 -- 172.21.15.44:0/2039814248 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5633c0b5a050 con 0x5633c0cd2510 2023-09-10T01:09:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.791+0000 7f116affd700 10 client.28664 handle_command_reply: tid=0 2023-09-10T01:09:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.791+0000 7f119e071b80 1 client.28664 shutdown 2023-09-10T01:09:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.791+0000 7f119e071b80 1 -- 172.21.15.44:0/2039814248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4ecc0 msgr2=0x5633c0cc1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.791+0000 7f119e071b80 1 --2- 172.21.15.44:0/2039814248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4ecc0 0x5633c0cc1780 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1180005860 tx=0x7f118001ec90 comp rx=0 tx=0).stop 2023-09-10T01:09:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.791+0000 7f119e071b80 1 -- 172.21.15.44:0/2039814248 shutdown_connections 2023-09-10T01:09:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.791+0000 7f119e071b80 1 --2- 172.21.15.44:0/2039814248 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5633c0cd2510 0x5633c0cd4930 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.791+0000 7f119e071b80 1 --2- 172.21.15.44:0/2039814248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5633c0c541a0 0x5633c0cc5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.791+0000 7f116affd700 0 client.28664 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.791+0000 7f119e071b80 1 --2- 172.21.15.44:0/2039814248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633c0c4e300 0x5633c0cc1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.791+0000 7f119e071b80 1 --2- 172.21.15.44:0/2039814248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633c0c4ecc0 0x5633c0cc1780 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.791+0000 7f119e071b80 1 -- 172.21.15.44:0/2039814248 >> 172.21.15.44:0/2039814248 conn(0x5633c0bc3030 msgr2=0x5633c0c4db80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.792+0000 7f119e071b80 1 -- 172.21.15.44:0/2039814248 shutdown_connections 2023-09-10T01:09:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.792+0000 7f119e071b80 1 -- 172.21.15.44:0/2039814248 wait complete. 2023-09-10T01:09:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.792+0000 7f119e071b80 20 client.28664 trim_cache size 0 max 16384 2023-09-10T01:09:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:09:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:09:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:09:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:09:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.792+0000 7f1191eab700 1 -- 172.21.15.44:0/3915277443 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f11740a0830 msgr2=0x7f11740a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.792+0000 7f1191eab700 1 --2- 172.21.15.44:0/3915277443 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f11740a0830 0x7f11740a2cf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f118c195b90 tx=0x7f1180009310 comp rx=0 tx=0).stop 2023-09-10T01:09:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.792+0000 7f1191eab700 1 -- 172.21.15.44:0/3915277443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f118c0ff760 msgr2=0x7f118c194370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.792+0000 7f1191eab700 1 --2- 172.21.15.44:0/3915277443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f118c0ff760 0x7f118c194370 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f117c0147e0 tx=0x7f117c014810 comp rx=0 tx=0).stop 2023-09-10T01:09:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.792+0000 7f1191eab700 1 -- 172.21.15.44:0/3915277443 shutdown_connections 2023-09-10T01:09:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.792+0000 7f1191eab700 1 --2- 172.21.15.44:0/3915277443 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f11740a0830 0x7f11740a2cf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.792+0000 7f1191eab700 1 --2- 172.21.15.44:0/3915277443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f118c1055b0 0x7f118c198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.792+0000 7f1191eab700 1 --2- 172.21.15.44:0/3915277443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f118c0ff760 0x7f118c194370 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.792+0000 7f1191eab700 1 --2- 172.21.15.44:0/3915277443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f118c100120 0x7f118c1948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.792+0000 7f1191eab700 1 -- 172.21.15.44:0/3915277443 >> 172.21.15.44:0/3915277443 conn(0x7f118c074f30 msgr2=0x7f118c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.792+0000 7f1191eab700 1 -- 172.21.15.44:0/3915277443 shutdown_connections 2023-09-10T01:09:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:29.793+0000 7f1191eab700 1 -- 172.21.15.44:0/3915277443 wait complete. 2023-09-10T01:09:30.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:29 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:09:30.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:29 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:09:30.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:29 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:09:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:30 smithi131 ceph-mon[103713]: pgmap v917: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 964 KiB/s wr, 139 op/s 2023-09-10T01:09:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:30 smithi044 ceph-mon[98676]: pgmap v917: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 964 KiB/s wr, 139 op/s 2023-09-10T01:09:31.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:30 smithi099 ceph-mon[102692]: pgmap v917: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 964 KiB/s wr, 139 op/s 2023-09-10T01:09:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:32 smithi131 ceph-mon[103713]: pgmap v918: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 232 op/s 2023-09-10T01:09:33.988 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:32 smithi099 ceph-mon[102692]: pgmap v918: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 232 op/s 2023-09-10T01:09:33.988 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:32 smithi044 ceph-mon[98676]: pgmap v918: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 232 op/s 2023-09-10T01:09:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:34 smithi131 ceph-mon[103713]: pgmap v919: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:09:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:34 smithi044 ceph-mon[98676]: pgmap v919: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:09:35.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:34 smithi099 ceph-mon[102692]: pgmap v919: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:09:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:36 smithi131 ceph-mon[103713]: pgmap v920: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 259 op/s 2023-09-10T01:09:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:36 smithi044 ceph-mon[98676]: pgmap v920: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 259 op/s 2023-09-10T01:09:37.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:36 smithi099 ceph-mon[102692]: pgmap v920: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 259 op/s 2023-09-10T01:09:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:38 smithi131 ceph-mon[103713]: pgmap v921: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:09:39.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:38 smithi044 ceph-mon[98676]: pgmap v921: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:09:39.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:38 smithi099 ceph-mon[102692]: pgmap v921: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:09:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:40 smithi131 ceph-mon[103713]: pgmap v922: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:09:41.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:40 smithi044 ceph-mon[98676]: pgmap v922: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:09:41.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:40 smithi099 ceph-mon[102692]: pgmap v922: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:09:42.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:41 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:09:42.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:41 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:09:42.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:41 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:09:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.932+0000 7f87377d1700 1 -- 172.21.15.44:0/3766964461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87300ffd40 msgr2=0x7f873010c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.932+0000 7f87377d1700 1 --2- 172.21.15.44:0/3766964461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87300ffd40 0x7f873010c970 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f8720009990 tx=0x7f8720009ca0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.932+0000 7f87377d1700 1 -- 172.21.15.44:0/3766964461 shutdown_connections 2023-09-10T01:09:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.932+0000 7f87377d1700 1 --2- 172.21.15.44:0/3766964461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f873010cf40 0x7f873010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.932+0000 7f87377d1700 1 --2- 172.21.15.44:0/3766964461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87300ff390 0x7f87300ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.932+0000 7f87377d1700 1 --2- 172.21.15.44:0/3766964461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87300ffd40 0x7f873010c970 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.932+0000 7f87377d1700 1 -- 172.21.15.44:0/3766964461 >> 172.21.15.44:0/3766964461 conn(0x7f87300faf80 msgr2=0x7f87300fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.932+0000 7f87377d1700 1 -- 172.21.15.44:0/3766964461 shutdown_connections 2023-09-10T01:09:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.932+0000 7f87377d1700 1 -- 172.21.15.44:0/3766964461 wait complete. 2023-09-10T01:09:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.932+0000 7f87377d1700 1 Processor -- start 2023-09-10T01:09:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.932+0000 7f87377d1700 1 -- start start 2023-09-10T01:09:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f87377d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87300ff390 0x7f873019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f87377d1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87300ffd40 0x7f873019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f87377d1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f873010cf40 0x7f87301986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f87377d1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8730198e90 con 0x7f873010cf40 2023-09-10T01:09:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f87377d1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8730199000 con 0x7f87300ff390 2023-09-10T01:09:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f87377d1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87301991c0 con 0x7f87300ffd40 2023-09-10T01:09:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f8734d6c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87300ffd40 0x7f873019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f873556d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87300ff390 0x7f873019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f873556d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87300ff390 0x7f873019e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58134/0 (socket says 172.21.15.44:58134) 2023-09-10T01:09:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f8734d6c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87300ffd40 0x7f873019ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35046/0 (socket says 172.21.15.44:35046) 2023-09-10T01:09:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f873556d700 1 -- 172.21.15.44:0/1983381999 learned_addr learned my addr 172.21.15.44:0/1983381999 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f8735d6e700 1 --2- 172.21.15.44:0/1983381999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f873010cf40 0x7f87301986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f873556d700 1 -- 172.21.15.44:0/1983381999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87300ffd40 msgr2=0x7f873019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f873556d700 1 --2- 172.21.15.44:0/1983381999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87300ffd40 0x7f873019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f873556d700 1 -- 172.21.15.44:0/1983381999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f873010cf40 msgr2=0x7f87301986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f873556d700 1 --2- 172.21.15.44:0/1983381999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f873010cf40 0x7f87301986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f873556d700 1 -- 172.21.15.44:0/1983381999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87200095f0 con 0x7f87300ff390 2023-09-10T01:09:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.933+0000 7f8734d6c700 1 --2- 172.21.15.44:0/1983381999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87300ffd40 0x7f873019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:09:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.934+0000 7f873556d700 1 --2- 172.21.15.44:0/1983381999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87300ff390 0x7f873019e8f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f872c00e9a0 tx=0x7f872c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.934+0000 7f87267fc700 1 -- 172.21.15.44:0/1983381999 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c009790 con 0x7f87300ff390 2023-09-10T01:09:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.934+0000 7f87267fc700 1 -- 172.21.15.44:0/1983381999 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f872c010430 con 0x7f87300ff390 2023-09-10T01:09:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.934+0000 7f87267fc700 1 -- 172.21.15.44:0/1983381999 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c00cea0 con 0x7f87300ff390 2023-09-10T01:09:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.934+0000 7f87377d1700 1 -- 172.21.15.44:0/1983381999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87301994a0 con 0x7f87300ff390 2023-09-10T01:09:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.934+0000 7f87377d1700 1 -- 172.21.15.44:0/1983381999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87301a5870 con 0x7f87300ff390 2023-09-10T01:09:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.936+0000 7f87267fc700 1 -- 172.21.15.44:0/1983381999 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f872c0105a0 con 0x7f87300ff390 2023-09-10T01:09:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.937+0000 7f87267fc700 1 --2- 172.21.15.44:0/1983381999 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f871c0a0830 0x7f871c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.937+0000 7f8734d6c700 1 --2- 172.21.15.44:0/1983381999 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f871c0a0830 0x7f871c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.937+0000 7f87267fc700 1 -- 172.21.15.44:0/1983381999 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f872c014070 con 0x7f87300ff390 2023-09-10T01:09:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.937+0000 7f8734d6c700 1 --2- 172.21.15.44:0/1983381999 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f871c0a0830 0x7f871c0a2cf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f87200059d0 tx=0x7f8720005960 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f8743997b80 1 Processor -- start 2023-09-10T01:09:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f8743997b80 1 -- start start 2023-09-10T01:09:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f8743997b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e89142a1a0 0x55e89142a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f8743997b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e891424300 0x55e891424780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f8743997b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424cc0 0x55e891430f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f8743997b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89137d6d0 con 0x55e891424300 2023-09-10T01:09:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f8743997b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89137aff0 con 0x55e891424cc0 2023-09-10T01:09:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f8743997b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8913768e0 con 0x55e89142a1a0 2023-09-10T01:09:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f873556d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e89142a1a0 0x55e89142a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f8734d6c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e891424300 0x55e891424780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f873556d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e89142a1a0 0x55e89142a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35052/0 (socket says 172.21.15.44:35052) 2023-09-10T01:09:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f873556d700 1 -- 172.21.15.44:0/2334790639 learned_addr learned my addr 172.21.15.44:0/2334790639 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f8734d6c700 1 --2- 172.21.15.44:0/2334790639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e891424300 0x55e891424780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56404/0 (socket says 172.21.15.44:56404) 2023-09-10T01:09:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f8735d6e700 1 --2- 172.21.15.44:0/2334790639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424cc0 0x55e891430f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f873556d700 1 -- 172.21.15.44:0/2334790639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424cc0 msgr2=0x55e891430f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f873556d700 1 --2- 172.21.15.44:0/2334790639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424cc0 0x55e891430f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f873556d700 1 -- 172.21.15.44:0/2334790639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e891424300 msgr2=0x55e891424780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.939+0000 7f873556d700 1 --2- 172.21.15.44:0/2334790639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e891424300 0x55e891424780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.940+0000 7f873556d700 1 -- 172.21.15.44:0/2334790639 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8911a2d80 con 0x55e89142a1a0 2023-09-10T01:09:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.940+0000 7f8735d6e700 1 --2- 172.21.15.44:0/2334790639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424cc0 0x55e891430f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:09:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.940+0000 7f873556d700 1 --2- 172.21.15.44:0/2334790639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e89142a1a0 0x55e89142a580 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f872c0a10b0 tx=0x7f872c098ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e225de48bfb51490 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.940+0000 7f871b7fe700 1 -- 172.21.15.44:0/2334790639 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c09d170 con 0x55e89142a1a0 2023-09-10T01:09:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.940+0000 7f871b7fe700 1 -- 172.21.15.44:0/2334790639 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f872c099570 con 0x55e89142a1a0 2023-09-10T01:09:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.940+0000 7f871b7fe700 1 -- 172.21.15.44:0/2334790639 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c0cc320 con 0x55e89142a1a0 2023-09-10T01:09:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.941+0000 7f8743997b80 1 -- 172.21.15.44:0/2334790639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e89142a1a0 msgr2=0x55e89142a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.941+0000 7f8743997b80 1 --2- 172.21.15.44:0/2334790639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e89142a1a0 0x55e89142a580 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f872c0a10b0 tx=0x7f872c098ed0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.941+0000 7f8743997b80 1 -- 172.21.15.44:0/2334790639 shutdown_connections 2023-09-10T01:09:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.941+0000 7f8743997b80 1 --2- 172.21.15.44:0/2334790639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424cc0 0x55e891430f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.941+0000 7f8743997b80 1 --2- 172.21.15.44:0/2334790639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e89142a1a0 0x55e89142a580 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.941+0000 7f8743997b80 1 --2- 172.21.15.44:0/2334790639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e891424300 0x55e891424780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.941+0000 7f8743997b80 1 -- 172.21.15.44:0/2334790639 >> 172.21.15.44:0/2334790639 conn(0x55e891399030 msgr2=0x55e89141f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.941+0000 7f8743997b80 1 -- 172.21.15.44:0/2334790639 shutdown_connections 2023-09-10T01:09:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.941+0000 7f8743997b80 1 -- 172.21.15.44:0/2334790639 wait complete. 2023-09-10T01:09:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.941+0000 7f8743997b80 1 -- start start 2023-09-10T01:09:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.941+0000 7f8743997b80 1 Processor -- start 2023-09-10T01:09:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8743997b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e891424300 0x55e891430e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8743997b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424cc0 0x55e89142e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8743997b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e89142a1a0 0x55e89142e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8743997b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e891314b10 con 0x55e891424300 2023-09-10T01:09:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8743997b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e891314810 con 0x55e891424cc0 2023-09-10T01:09:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8743997b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8911f88c0 con 0x55e89142a1a0 2023-09-10T01:09:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8743997b80 10 client.0 _ll_register_callbacks cb 0x55e89125ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:09:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f873556d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e891424300 0x55e891430e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8735d6e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e89142a1a0 0x55e89142e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8734d6c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424cc0 0x55e89142e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8735d6e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e89142a1a0 0x55e89142e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35062/0 (socket says 172.21.15.44:35062) 2023-09-10T01:09:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8735d6e700 1 -- 172.21.15.44:0/3806901273 learned_addr learned my addr 172.21.15.44:0/3806901273 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8735d6e700 1 -- 172.21.15.44:0/3806901273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424cc0 msgr2=0x55e89142e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8735d6e700 1 --2- 172.21.15.44:0/3806901273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424cc0 0x55e89142e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8735d6e700 1 -- 172.21.15.44:0/3806901273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e891424300 msgr2=0x55e891430e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8735d6e700 1 --2- 172.21.15.44:0/3806901273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e891424300 0x55e891430e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8735d6e700 1 -- 172.21.15.44:0/3806901273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87301a5870 con 0x55e89142a1a0 2023-09-10T01:09:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8735d6e700 1 --2- 172.21.15.44:0/3806901273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e89142a1a0 0x55e89142e7b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8728009e20 tx=0x7f8728004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb460a1f7970df7c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.942+0000 7f8718ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:09:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.943+0000 7f8718ff9700 1 -- 172.21.15.44:0/3806901273 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728007500 con 0x55e89142a1a0 2023-09-10T01:09:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.943+0000 7f8718ff9700 1 -- 172.21.15.44:0/3806901273 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8728007660 con 0x55e89142a1a0 2023-09-10T01:09:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.943+0000 7f8743997b80 1 -- 172.21.15.44:0/3806901273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8912ce490 con 0x55e89142a1a0 2023-09-10T01:09:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.943+0000 7f8718ff9700 1 -- 172.21.15.44:0/3806901273 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8728007820 con 0x55e89142a1a0 2023-09-10T01:09:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.943+0000 7f8743997b80 10 client.28618 fetch_fsmap learned FSMap version 114 2023-09-10T01:09:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.943+0000 7f8743997b80 1 -- 172.21.15.44:0/3806901273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=4+}) v3 -- 0x55e89118bc00 con 0x55e89142a1a0 2023-09-10T01:09:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.943+0000 7f8718ff9700 1 -- 172.21.15.44:0/3806901273 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e8912ce490 con 0x55e89142a1a0 2023-09-10T01:09:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.943+0000 7f8718ff9700 1 -- 172.21.15.44:0/3806901273 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 114) v1 ==== 7347+0+0 (secure 0 0 0) 0x7f8728014070 con 0x55e89142a1a0 2023-09-10T01:09:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.943+0000 7f8743997b80 10 client.28618 fetch_fsmap finished waiting for FSMap version 114 2023-09-10T01:09:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.943+0000 7f8743997b80 10 client.28618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:09:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.943+0000 7f8743997b80 20 client.28618 populate_metadata read hostname 'smithi044' 2023-09-10T01:09:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.943+0000 7f8743997b80 1 --2- 172.21.15.44:0/3806901273 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e89143e4f0 0x55e891440910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.943+0000 7f8743997b80 4 client.28618 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:09:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.943+0000 7f8743997b80 1 -- 172.21.15.44:0/3806901273 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e89138b840 con 0x55e89143e4f0 2023-09-10T01:09:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.944+0000 7f873556d700 1 --2- 172.21.15.44:0/3806901273 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e89143e4f0 0x55e891440910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.944+0000 7f873556d700 1 --2- 172.21.15.44:0/3806901273 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e89143e4f0 0x55e891440910 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc3c7dda9de6e4b4 server_cookie=5b1860934e78c92d in_seq=0 out_seq=0 2023-09-10T01:09:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.944+0000 7f8718ff9700 10 client.28618 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.944+0000 7f8718ff9700 1 -- 172.21.15.44:0/3806901273 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e89138b840 con 0x55e89143e4f0 2023-09-10T01:09:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.944+0000 7f8718ff9700 10 client.28618 handle_command_reply: tid=0 2023-09-10T01:09:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.945+0000 7f8743997b80 1 client.28618 shutdown 2023-09-10T01:09:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.945+0000 7f8743997b80 1 -- 172.21.15.44:0/3806901273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e89142a1a0 msgr2=0x55e89142e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.945+0000 7f8743997b80 1 --2- 172.21.15.44:0/3806901273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e89142a1a0 0x55e89142e7b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8728009e20 tx=0x7f8728004320 comp rx=0 tx=0).stop 2023-09-10T01:09:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.945+0000 7f8743997b80 1 -- 172.21.15.44:0/3806901273 shutdown_connections 2023-09-10T01:09:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.945+0000 7f8743997b80 1 --2- 172.21.15.44:0/3806901273 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e89143e4f0 0x55e891440910 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.945+0000 7f8743997b80 1 --2- 172.21.15.44:0/3806901273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e89142a1a0 0x55e89142e7b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.945+0000 7f8743997b80 1 --2- 172.21.15.44:0/3806901273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e891424300 0x55e891430e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.945+0000 7f8718ff9700 0 client.28618 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.945+0000 7f8743997b80 1 --2- 172.21.15.44:0/3806901273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424cc0 0x55e89142e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.945+0000 7f8743997b80 1 -- 172.21.15.44:0/3806901273 >> 172.21.15.44:0/3806901273 conn(0x55e891399030 msgr2=0x55e89141e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.945+0000 7f8743997b80 1 -- 172.21.15.44:0/3806901273 shutdown_connections 2023-09-10T01:09:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.945+0000 7f8743997b80 1 -- 172.21.15.44:0/3806901273 wait complete. 2023-09-10T01:09:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.945+0000 7f8743997b80 20 client.28618 trim_cache size 0 max 16384 2023-09-10T01:09:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.966+0000 7f8743997b80 1 Processor -- start 2023-09-10T01:09:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.966+0000 7f8743997b80 1 -- start start 2023-09-10T01:09:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.966+0000 7f8743997b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424300 0x55e891428290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.966+0000 7f8743997b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e891424cc0 0x55e8914287d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.966+0000 7f8743997b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e89142a1a0 0x55e891429210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.966+0000 7f8743997b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e891427db0 con 0x55e89142a1a0 2023-09-10T01:09:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.966+0000 7f8743997b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8914252f0 con 0x55e891424300 2023-09-10T01:09:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.966+0000 7f8743997b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89143a300 con 0x55e891424cc0 2023-09-10T01:09:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.967+0000 7f873556d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424300 0x55e891428290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.967+0000 7f8734d6c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e891424cc0 0x55e8914287d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.967+0000 7f873556d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424300 0x55e891428290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58156/0 (socket says 172.21.15.44:58156) 2023-09-10T01:09:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.967+0000 7f8734d6c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e891424cc0 0x55e8914287d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35066/0 (socket says 172.21.15.44:35066) 2023-09-10T01:09:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.967+0000 7f873556d700 1 -- 172.21.15.44:0/1943482523 learned_addr learned my addr 172.21.15.44:0/1943482523 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.967+0000 7f8735d6e700 1 --2- 172.21.15.44:0/1943482523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e89142a1a0 0x55e891429210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.967+0000 7f873556d700 1 -- 172.21.15.44:0/1943482523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e891424cc0 msgr2=0x55e8914287d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.967+0000 7f873556d700 1 --2- 172.21.15.44:0/1943482523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e891424cc0 0x55e8914287d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.967+0000 7f873556d700 1 -- 172.21.15.44:0/1943482523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e89142a1a0 msgr2=0x55e891429210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.967+0000 7f873556d700 1 --2- 172.21.15.44:0/1943482523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e89142a1a0 0x55e891429210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.967+0000 7f873556d700 1 -- 172.21.15.44:0/1943482523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8911c91f0 con 0x55e891424300 2023-09-10T01:09:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.967+0000 7f873556d700 1 --2- 172.21.15.44:0/1943482523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424300 0x55e891428290 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f872c095a70 tx=0x7f872c0958b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5ec1dc8e9813ee6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.967+0000 7f871bfff700 1 -- 172.21.15.44:0/1943482523 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c09d420 con 0x55e891424300 2023-09-10T01:09:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.967+0000 7f871bfff700 1 -- 172.21.15.44:0/1943482523 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f872c0ace90 con 0x55e891424300 2023-09-10T01:09:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.967+0000 7f871bfff700 1 -- 172.21.15.44:0/1943482523 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c0997d0 con 0x55e891424300 2023-09-10T01:09:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.968+0000 7f8743997b80 1 -- 172.21.15.44:0/1943482523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424300 msgr2=0x55e891428290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.968+0000 7f8743997b80 1 --2- 172.21.15.44:0/1943482523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424300 0x55e891428290 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f872c095a70 tx=0x7f872c0958b0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.968+0000 7f8743997b80 1 -- 172.21.15.44:0/1943482523 shutdown_connections 2023-09-10T01:09:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.968+0000 7f8743997b80 1 --2- 172.21.15.44:0/1943482523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e89142a1a0 0x55e891429210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.968+0000 7f8743997b80 1 --2- 172.21.15.44:0/1943482523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e891424300 0x55e891428290 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.968+0000 7f8743997b80 1 --2- 172.21.15.44:0/1943482523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e891424cc0 0x55e8914287d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.968+0000 7f8743997b80 1 -- 172.21.15.44:0/1943482523 >> 172.21.15.44:0/1943482523 conn(0x55e891399030 msgr2=0x55e891439bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.968+0000 7f8743997b80 1 -- 172.21.15.44:0/1943482523 shutdown_connections 2023-09-10T01:09:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.968+0000 7f8743997b80 1 -- 172.21.15.44:0/1943482523 wait complete. 2023-09-10T01:09:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.969+0000 7f8743997b80 1 -- start start 2023-09-10T01:09:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.969+0000 7f8743997b80 1 Processor -- start 2023-09-10T01:09:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.969+0000 7f8743997b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e891424300 0x55e891433480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.969+0000 7f8743997b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e891424cc0 0x55e8914339c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.969+0000 7f8743997b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e89142a1a0 0x55e891433f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.969+0000 7f8743997b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e891435a50 con 0x55e891424cc0 2023-09-10T01:09:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.969+0000 7f8743997b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e891491300 con 0x55e89142a1a0 2023-09-10T01:09:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.969+0000 7f8743997b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e891491470 con 0x55e891424300 2023-09-10T01:09:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.969+0000 7f8743997b80 10 client.0 _ll_register_callbacks cb 0x7f8728003f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:09:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.969+0000 7f873556d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e891424300 0x55e891433480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.969+0000 7f873556d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e891424300 0x55e891433480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35070/0 (socket says 172.21.15.44:35070) 2023-09-10T01:09:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.969+0000 7f873556d700 1 -- 172.21.15.44:0/3040359801 learned_addr learned my addr 172.21.15.44:0/3040359801 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.969+0000 7f8735d6e700 1 --2- 172.21.15.44:0/3040359801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e89142a1a0 0x55e891433f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.969+0000 7f8734d6c700 1 --2- 172.21.15.44:0/3040359801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e891424cc0 0x55e8914339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.970+0000 7f873556d700 1 -- 172.21.15.44:0/3040359801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e89142a1a0 msgr2=0x55e891433f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.970+0000 7f873556d700 1 --2- 172.21.15.44:0/3040359801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e89142a1a0 0x55e891433f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.970+0000 7f873556d700 1 -- 172.21.15.44:0/3040359801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e891424cc0 msgr2=0x55e8914339c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.970+0000 7f873556d700 1 --2- 172.21.15.44:0/3040359801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e891424cc0 0x55e8914339c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.970+0000 7f873556d700 1 -- 172.21.15.44:0/3040359801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8911a2d80 con 0x55e891424300 2023-09-10T01:09:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.970+0000 7f873556d700 1 --2- 172.21.15.44:0/3040359801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e891424300 0x55e891433480 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f872c00c2d0 tx=0x7f872c0a1670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f32f108f613a608 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.970+0000 7f8718ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:09:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.970+0000 7f8718ff9700 1 -- 172.21.15.44:0/3040359801 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c0cc1b0 con 0x55e891424300 2023-09-10T01:09:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.970+0000 7f8718ff9700 1 -- 172.21.15.44:0/3040359801 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f872c0cc310 con 0x55e891424300 2023-09-10T01:09:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.970+0000 7f8743997b80 1 -- 172.21.15.44:0/3040359801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f872c099570 con 0x55e891424300 2023-09-10T01:09:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.970+0000 7f8718ff9700 1 -- 172.21.15.44:0/3040359801 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f872c098a60 con 0x55e891424300 2023-09-10T01:09:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.970+0000 7f8743997b80 10 client.28624 fetch_fsmap learned FSMap version 114 2023-09-10T01:09:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.970+0000 7f8743997b80 1 -- 172.21.15.44:0/3040359801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=4+}) v3 -- 0x55e8911a1250 con 0x55e891424300 2023-09-10T01:09:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.971+0000 7f8718ff9700 1 -- 172.21.15.44:0/3040359801 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f872c099570 con 0x55e891424300 2023-09-10T01:09:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.971+0000 7f8718ff9700 1 -- 172.21.15.44:0/3040359801 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 114) v1 ==== 7347+0+0 (secure 0 0 0) 0x7f872c0a1150 con 0x55e891424300 2023-09-10T01:09:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.971+0000 7f8743997b80 10 client.28624 fetch_fsmap finished waiting for FSMap version 114 2023-09-10T01:09:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.971+0000 7f8743997b80 10 client.28624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:09:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.971+0000 7f8743997b80 20 client.28624 populate_metadata read hostname 'smithi044' 2023-09-10T01:09:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.971+0000 7f8743997b80 1 --2- 172.21.15.44:0/3040359801 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e89143e4f0 0x55e891494740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.971+0000 7f8743997b80 4 client.28624 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:09:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.971+0000 7f8743997b80 1 -- 172.21.15.44:0/3040359801 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e89136f8c0 con 0x55e89143e4f0 2023-09-10T01:09:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.971+0000 7f8734d6c700 1 --2- 172.21.15.44:0/3040359801 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e89143e4f0 0x55e891494740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.971+0000 7f8718ff9700 10 client.28624 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.971+0000 7f8734d6c700 1 --2- 172.21.15.44:0/3040359801 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e89143e4f0 0x55e891494740 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf4177a5492917b6 server_cookie=f841d0fff0fd3ac9 in_seq=0 out_seq=0 2023-09-10T01:09:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.984+0000 7f8718ff9700 1 -- 172.21.15.44:0/3040359801 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e89136f8c0 con 0x55e89143e4f0 2023-09-10T01:09:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.984+0000 7f8718ff9700 10 client.28624 handle_command_reply: tid=0 2023-09-10T01:09:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.984+0000 7f8743997b80 1 client.28624 shutdown 2023-09-10T01:09:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.985+0000 7f8743997b80 1 -- 172.21.15.44:0/3040359801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e891424300 msgr2=0x55e891433480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.985+0000 7f8743997b80 1 --2- 172.21.15.44:0/3040359801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e891424300 0x55e891433480 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f872c00c2d0 tx=0x7f872c0a1670 comp rx=0 tx=0).stop 2023-09-10T01:09:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.985+0000 7f8743997b80 1 -- 172.21.15.44:0/3040359801 shutdown_connections 2023-09-10T01:09:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.985+0000 7f8743997b80 1 --2- 172.21.15.44:0/3040359801 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e89143e4f0 0x55e891494740 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.985+0000 7f8743997b80 1 --2- 172.21.15.44:0/3040359801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e89142a1a0 0x55e891433f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.985+0000 7f8718ff9700 0 client.28624 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.985+0000 7f8743997b80 1 --2- 172.21.15.44:0/3040359801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e891424300 0x55e891433480 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.985+0000 7f8743997b80 1 --2- 172.21.15.44:0/3040359801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e891424cc0 0x55e8914339c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.985+0000 7f8743997b80 1 -- 172.21.15.44:0/3040359801 >> 172.21.15.44:0/3040359801 conn(0x55e891399030 msgr2=0x55e891428290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.985+0000 7f8743997b80 1 -- 172.21.15.44:0/3040359801 shutdown_connections 2023-09-10T01:09:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.985+0000 7f8743997b80 1 -- 172.21.15.44:0/3040359801 wait complete. 2023-09-10T01:09:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.985+0000 7f8743997b80 20 client.28624 trim_cache size 0 max 16384 2023-09-10T01:09:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:09:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:09:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:09:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:09:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.985+0000 7f87377d1700 1 -- 172.21.15.44:0/1983381999 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f871c0a0830 msgr2=0x7f871c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.985+0000 7f87377d1700 1 --2- 172.21.15.44:0/1983381999 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f871c0a0830 0x7f871c0a2cf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f87200059d0 tx=0x7f8720005960 comp rx=0 tx=0).stop 2023-09-10T01:09:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.985+0000 7f87377d1700 1 -- 172.21.15.44:0/1983381999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87300ff390 msgr2=0x7f873019e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.985+0000 7f87377d1700 1 --2- 172.21.15.44:0/1983381999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87300ff390 0x7f873019e8f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f872c00e9a0 tx=0x7f872c00ecb0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.986+0000 7f87377d1700 1 -- 172.21.15.44:0/1983381999 shutdown_connections 2023-09-10T01:09:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.986+0000 7f87377d1700 1 --2- 172.21.15.44:0/1983381999 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f871c0a0830 0x7f871c0a2cf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.986+0000 7f87377d1700 1 --2- 172.21.15.44:0/1983381999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f873010cf40 0x7f87301986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.986+0000 7f87377d1700 1 --2- 172.21.15.44:0/1983381999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87300ff390 0x7f873019e8f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.986+0000 7f87377d1700 1 --2- 172.21.15.44:0/1983381999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87300ffd40 0x7f873019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.986+0000 7f87377d1700 1 -- 172.21.15.44:0/1983381999 >> 172.21.15.44:0/1983381999 conn(0x7f87300faf80 msgr2=0x7f873010e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.986+0000 7f87377d1700 1 -- 172.21.15.44:0/1983381999 shutdown_connections 2023-09-10T01:09:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:42.986+0000 7f87377d1700 1 -- 172.21.15.44:0/1983381999 wait complete. 2023-09-10T01:09:43.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:42 smithi131 ceph-mon[103713]: pgmap v923: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 283 op/s 2023-09-10T01:09:43.299 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:42 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:09:43.299 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:42 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:09:43.299 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:42 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:09:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:42 smithi044 ceph-mon[98676]: pgmap v923: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 283 op/s 2023-09-10T01:09:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:42 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:09:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:42 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:09:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:42 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:09:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:42 smithi099 ceph-mon[102692]: pgmap v923: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 283 op/s 2023-09-10T01:09:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:42 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:09:43.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:42 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:09:43.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:42 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:09:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:44 smithi131 ceph-mon[103713]: pgmap v924: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:09:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:44 smithi044 ceph-mon[98676]: pgmap v924: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:09:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:44 smithi099 ceph-mon[102692]: pgmap v924: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:09:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:46 smithi131 ceph-mon[103713]: pgmap v925: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 258 op/s 2023-09-10T01:09:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:46 smithi044 ceph-mon[98676]: pgmap v925: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 258 op/s 2023-09-10T01:09:47.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:46 smithi099 ceph-mon[102692]: pgmap v925: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 258 op/s 2023-09-10T01:09:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:48 smithi131 ceph-mon[103713]: pgmap v926: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:09:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:48 smithi044 ceph-mon[98676]: pgmap v926: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:09:49.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:48 smithi099 ceph-mon[102692]: pgmap v926: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:09:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:50 smithi131 ceph-mon[103713]: pgmap v927: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:09:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:50 smithi044 ceph-mon[98676]: pgmap v927: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:09:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:50 smithi099 ceph-mon[102692]: pgmap v927: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:09:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:52 smithi131 ceph-mon[103713]: pgmap v928: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 250 op/s 2023-09-10T01:09:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:52 smithi044 ceph-mon[98676]: pgmap v928: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 250 op/s 2023-09-10T01:09:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:52 smithi099 ceph-mon[102692]: pgmap v928: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 250 op/s 2023-09-10T01:09:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:54 smithi131 ceph-mon[103713]: pgmap v929: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:09:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:54 smithi044 ceph-mon[98676]: pgmap v929: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:09:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:54 smithi099 ceph-mon[102692]: pgmap v929: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:09:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.378+0000 7f0ccbd9d700 1 -- 172.21.15.44:0/3798801178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc40679f0 msgr2=0x7f0cc4067e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.378+0000 7f0ccbd9d700 1 --2- 172.21.15.44:0/3798801178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc40679f0 0x7f0cc4067e70 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40099c0 tx=0x7f0cb4009cd0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.378+0000 7f0ccbd9d700 1 -- 172.21.15.44:0/3798801178 shutdown_connections 2023-09-10T01:09:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.378+0000 7f0ccbd9d700 1 --2- 172.21.15.44:0/3798801178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cc40683b0 0x7f0cc410d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.378+0000 7f0ccbd9d700 1 --2- 172.21.15.44:0/3798801178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cc4106650 0x7f0cc4106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.378+0000 7f0ccbd9d700 1 --2- 172.21.15.44:0/3798801178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc40679f0 0x7f0cc4067e70 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.378+0000 7f0ccbd9d700 1 -- 172.21.15.44:0/3798801178 >> 172.21.15.44:0/3798801178 conn(0x7f0cc4074f30 msgr2=0x7f0cc4075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.378+0000 7f0ccbd9d700 1 -- 172.21.15.44:0/3798801178 shutdown_connections 2023-09-10T01:09:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.378+0000 7f0ccbd9d700 1 -- 172.21.15.44:0/3798801178 wait complete. 2023-09-10T01:09:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0ccbd9d700 1 Processor -- start 2023-09-10T01:09:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0ccbd9d700 1 -- start start 2023-09-10T01:09:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0ccbd9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cc40679f0 0x7f0cc4196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0ccbd9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc40683b0 0x7f0cc4196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0ccbd9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cc4106650 0x7f0cc419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0ccbd9d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc419a600 con 0x7f0cc40683b0 2023-09-10T01:09:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0ccbd9d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc419a770 con 0x7f0cc4106650 2023-09-10T01:09:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0ccbd9d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc419a930 con 0x7f0cc40679f0 2023-09-10T01:09:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0cc9338700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc40683b0 0x7f0cc4196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0cc9338700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc40683b0 0x7f0cc4196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56436/0 (socket says 172.21.15.44:56436) 2023-09-10T01:09:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0cc9338700 1 -- 172.21.15.44:0/2126648563 learned_addr learned my addr 172.21.15.44:0/2126648563 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0cc9b39700 1 --2- 172.21.15.44:0/2126648563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cc40679f0 0x7f0cc4196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0cca33a700 1 --2- 172.21.15.44:0/2126648563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cc4106650 0x7f0cc419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0cc9338700 1 -- 172.21.15.44:0/2126648563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cc40679f0 msgr2=0x7f0cc4196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0cc9338700 1 --2- 172.21.15.44:0/2126648563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cc40679f0 0x7f0cc4196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0cc9338700 1 -- 172.21.15.44:0/2126648563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cc4106650 msgr2=0x7f0cc419a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0cc9338700 1 --2- 172.21.15.44:0/2126648563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cc4106650 0x7f0cc419a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.379+0000 7f0cc9338700 1 -- 172.21.15.44:0/2126648563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb40095f0 con 0x7f0cc40683b0 2023-09-10T01:09:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.380+0000 7f0cc9338700 1 --2- 172.21.15.44:0/2126648563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc40683b0 0x7f0cc4196a40 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4005910 tx=0x7f0cb4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.380+0000 7f0cbaffd700 1 -- 172.21.15.44:0/2126648563 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb401d070 con 0x7f0cc40683b0 2023-09-10T01:09:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.380+0000 7f0ccbd9d700 1 -- 172.21.15.44:0/2126648563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc419abb0 con 0x7f0cc40683b0 2023-09-10T01:09:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.380+0000 7f0cbaffd700 1 -- 172.21.15.44:0/2126648563 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0cb4022470 con 0x7f0cc40683b0 2023-09-10T01:09:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.380+0000 7f0cbaffd700 1 -- 172.21.15.44:0/2126648563 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4017650 con 0x7f0cc40683b0 2023-09-10T01:09:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.380+0000 7f0ccbd9d700 1 -- 172.21.15.44:0/2126648563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc41a3570 con 0x7f0cc40683b0 2023-09-10T01:09:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.383+0000 7f0cbaffd700 1 -- 172.21.15.44:0/2126648563 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f0cb40225e0 con 0x7f0cc40683b0 2023-09-10T01:09:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.383+0000 7f0cbaffd700 1 --2- 172.21.15.44:0/2126648563 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0cb00a07b0 0x7f0cb00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.384+0000 7f0cbaffd700 1 -- 172.21.15.44:0/2126648563 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f0cb40cd390 con 0x7f0cc40683b0 2023-09-10T01:09:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.384+0000 7f0cc9b39700 1 --2- 172.21.15.44:0/2126648563 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0cb00a07b0 0x7f0cb00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.384+0000 7f0cc9b39700 1 --2- 172.21.15.44:0/2126648563 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0cb00a07b0 0x7f0cb00a2c70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009c60 tx=0x7f0cc0009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.384+0000 7f0cd7f63b80 1 Processor -- start 2023-09-10T01:09:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.384+0000 7f0cd7f63b80 1 -- start start 2023-09-10T01:09:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cd7f63b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea08901a0 0x562ea0890580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cd7f63b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea088a300 0x562ea088a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cd7f63b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088acc0 0x562ea0896f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cd7f63b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea07e36d0 con 0x562ea08901a0 2023-09-10T01:09:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cd7f63b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea07e0ff0 con 0x562ea088acc0 2023-09-10T01:09:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cd7f63b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea07dc8e0 con 0x562ea088a300 2023-09-10T01:09:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cc9b39700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea08901a0 0x562ea0890580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cc9338700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea088a300 0x562ea088a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cc9338700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea088a300 0x562ea088a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35094/0 (socket says 172.21.15.44:35094) 2023-09-10T01:09:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cc9338700 1 -- 172.21.15.44:0/745730059 learned_addr learned my addr 172.21.15.44:0/745730059 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cca33a700 1 --2- 172.21.15.44:0/745730059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088acc0 0x562ea0896f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cc9338700 1 -- 172.21.15.44:0/745730059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088acc0 msgr2=0x562ea0896f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cc9338700 1 --2- 172.21.15.44:0/745730059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088acc0 0x562ea0896f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cc9338700 1 -- 172.21.15.44:0/745730059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea08901a0 msgr2=0x562ea0890580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cc9338700 1 --2- 172.21.15.44:0/745730059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea08901a0 0x562ea0890580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cc9338700 1 -- 172.21.15.44:0/745730059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ea0608d80 con 0x562ea088a300 2023-09-10T01:09:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cca33a700 1 --2- 172.21.15.44:0/745730059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088acc0 0x562ea0896f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:09:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.385+0000 7f0cc9338700 1 --2- 172.21.15.44:0/745730059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea088a300 0x562ea088a780 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4096f40 tx=0x7f0cb4096940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee5405e0634f5aff server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.386+0000 7f0caffff700 1 -- 172.21.15.44:0/745730059 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb409abd0 con 0x562ea088a300 2023-09-10T01:09:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.386+0000 7f0caffff700 1 -- 172.21.15.44:0/745730059 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0cb409a4d0 con 0x562ea088a300 2023-09-10T01:09:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.386+0000 7f0caffff700 1 -- 172.21.15.44:0/745730059 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb40228c0 con 0x562ea088a300 2023-09-10T01:09:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.386+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/745730059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea088a300 msgr2=0x562ea088a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.386+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/745730059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea088a300 0x562ea088a780 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4096f40 tx=0x7f0cb4096940 comp rx=0 tx=0).stop 2023-09-10T01:09:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.387+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/745730059 shutdown_connections 2023-09-10T01:09:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.387+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/745730059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088acc0 0x562ea0896f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.387+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/745730059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea08901a0 0x562ea0890580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.387+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/745730059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea088a300 0x562ea088a780 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.387+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/745730059 >> 172.21.15.44:0/745730059 conn(0x562ea07ff030 msgr2=0x562ea0885560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.387+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/745730059 shutdown_connections 2023-09-10T01:09:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.387+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/745730059 wait complete. 2023-09-10T01:09:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.387+0000 7f0cd7f63b80 1 -- start start 2023-09-10T01:09:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.387+0000 7f0cd7f63b80 1 Processor -- start 2023-09-10T01:09:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.387+0000 7f0cd7f63b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea088a300 0x562ea0899b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.387+0000 7f0cd7f63b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088acc0 0x562ea089a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.387+0000 7f0cd7f63b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea08901a0 0x562ea0893df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.387+0000 7f0cd7f63b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea077ab10 con 0x562ea088a300 2023-09-10T01:09:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.387+0000 7f0cd7f63b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea077a810 con 0x562ea088acc0 2023-09-10T01:09:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.387+0000 7f0cd7f63b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea065e8c0 con 0x562ea08901a0 2023-09-10T01:09:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.388+0000 7f0cd7f63b80 10 client.0 _ll_register_callbacks cb 0x562ea06c4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:09:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.388+0000 7f0cc9338700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088acc0 0x562ea089a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.388+0000 7f0cc9338700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088acc0 0x562ea089a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58190/0 (socket says 172.21.15.44:58190) 2023-09-10T01:09:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.388+0000 7f0cc9338700 1 -- 172.21.15.44:0/3700521342 learned_addr learned my addr 172.21.15.44:0/3700521342 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.388+0000 7f0cca33a700 1 --2- 172.21.15.44:0/3700521342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea08901a0 0x562ea0893df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.388+0000 7f0cc9338700 1 -- 172.21.15.44:0/3700521342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea08901a0 msgr2=0x562ea0893df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.388+0000 7f0cc9338700 1 --2- 172.21.15.44:0/3700521342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea08901a0 0x562ea0893df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.388+0000 7f0cc9338700 1 -- 172.21.15.44:0/3700521342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea088a300 msgr2=0x562ea0899b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.388+0000 7f0cc9338700 1 --2- 172.21.15.44:0/3700521342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea088a300 0x562ea0899b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.388+0000 7f0cc9338700 1 -- 172.21.15.44:0/3700521342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cbc011230 con 0x562ea088acc0 2023-09-10T01:09:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.388+0000 7f0cca33a700 1 --2- 172.21.15.44:0/3700521342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea08901a0 0x562ea0893df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:09:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.388+0000 7f0cc9338700 1 --2- 172.21.15.44:0/3700521342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088acc0 0x562ea089a0a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4000c00 tx=0x7f0cb4009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e3d9c4304c73e1a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.388+0000 7f0cad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:09:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.388+0000 7f0cad7fa700 1 -- 172.21.15.44:0/3700521342 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb409abd0 con 0x562ea088acc0 2023-09-10T01:09:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.388+0000 7f0cad7fa700 1 -- 172.21.15.44:0/3700521342 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f0cb409e300 con 0x562ea088acc0 2023-09-10T01:09:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.388+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/3700521342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ea0734490 con 0x562ea088acc0 2023-09-10T01:09:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.389+0000 7f0cad7fa700 1 -- 172.21.15.44:0/3700521342 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cb409a730 con 0x562ea088acc0 2023-09-10T01:09:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.389+0000 7f0cd7f63b80 10 client.28691 fetch_fsmap learned FSMap version 114 2023-09-10T01:09:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.389+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/3700521342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=4+}) v3 -- 0x7f0cc41a3570 con 0x562ea088acc0 2023-09-10T01:09:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.389+0000 7f0cad7fa700 1 -- 172.21.15.44:0/3700521342 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562ea0734490 con 0x562ea088acc0 2023-09-10T01:09:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.389+0000 7f0cad7fa700 1 -- 172.21.15.44:0/3700521342 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 114) v1 ==== 7347+0+0 (secure 0 0 0) 0x7f0cb40a20b0 con 0x562ea088acc0 2023-09-10T01:09:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.389+0000 7f0cd7f63b80 10 client.28691 fetch_fsmap finished waiting for FSMap version 114 2023-09-10T01:09:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.389+0000 7f0cd7f63b80 10 client.28691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:09:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.389+0000 7f0cd7f63b80 20 client.28691 populate_metadata read hostname 'smithi044' 2023-09-10T01:09:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.389+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/3700521342 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ea08a3c20 0x562ea08a6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.389+0000 7f0cd7f63b80 4 client.28691 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:09:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.389+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/3700521342 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ea07f1840 con 0x562ea08a3c20 2023-09-10T01:09:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.389+0000 7f0cc9b39700 1 --2- 172.21.15.44:0/3700521342 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ea08a3c20 0x562ea08a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.390+0000 7f0cad7fa700 10 client.28691 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.390+0000 7f0cc9b39700 1 --2- 172.21.15.44:0/3700521342 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ea08a3c20 0x562ea08a6040 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d814a8db88649015 server_cookie=5ebdf9c219ba2181 in_seq=0 out_seq=0 2023-09-10T01:09:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.390+0000 7f0cad7fa700 1 -- 172.21.15.44:0/3700521342 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562ea07f1840 con 0x562ea08a3c20 2023-09-10T01:09:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.390+0000 7f0cad7fa700 10 client.28691 handle_command_reply: tid=0 2023-09-10T01:09:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.390+0000 7f0cd7f63b80 1 client.28691 shutdown 2023-09-10T01:09:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.390+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/3700521342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088acc0 msgr2=0x562ea089a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.390+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/3700521342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088acc0 0x562ea089a0a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4000c00 tx=0x7f0cb4009300 comp rx=0 tx=0).stop 2023-09-10T01:09:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.391+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/3700521342 shutdown_connections 2023-09-10T01:09:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.391+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/3700521342 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ea08a3c20 0x562ea08a6040 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.391+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/3700521342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea08901a0 0x562ea0893df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.391+0000 7f0cad7fa700 0 client.28691 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.391+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/3700521342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea088a300 0x562ea0899b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.391+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/3700521342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088acc0 0x562ea089a0a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.391+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/3700521342 >> 172.21.15.44:0/3700521342 conn(0x562ea07ff030 msgr2=0x562ea08847a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.391+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/3700521342 shutdown_connections 2023-09-10T01:09:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.391+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/3700521342 wait complete. 2023-09-10T01:09:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.391+0000 7f0cd7f63b80 20 client.28691 trim_cache size 0 max 16384 2023-09-10T01:09:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.408+0000 7f0cd7f63b80 1 Processor -- start 2023-09-10T01:09:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.408+0000 7f0cd7f63b80 1 -- start start 2023-09-10T01:09:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.408+0000 7f0cd7f63b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088a300 0x562ea088b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.408+0000 7f0cd7f63b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea088acc0 0x562ea0893da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.408+0000 7f0cd7f63b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea08901a0 0x562ea0884f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.408+0000 7f0cd7f63b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea08942e0 con 0x562ea088acc0 2023-09-10T01:09:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.408+0000 7f0cd7f63b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea08860d0 con 0x562ea088a300 2023-09-10T01:09:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.408+0000 7f0cd7f63b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea0886930 con 0x562ea08901a0 2023-09-10T01:09:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.408+0000 7f0cc9338700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea088acc0 0x562ea0893da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.408+0000 7f0cc9b39700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088a300 0x562ea088b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.408+0000 7f0cc9b39700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088a300 0x562ea088b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58196/0 (socket says 172.21.15.44:58196) 2023-09-10T01:09:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.408+0000 7f0cc9b39700 1 -- 172.21.15.44:0/3786807738 learned_addr learned my addr 172.21.15.44:0/3786807738 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.408+0000 7f0cca33a700 1 --2- 172.21.15.44:0/3786807738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea08901a0 0x562ea0884f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.409+0000 7f0cc9b39700 1 -- 172.21.15.44:0/3786807738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea08901a0 msgr2=0x562ea0884f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.409+0000 7f0cc9b39700 1 --2- 172.21.15.44:0/3786807738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea08901a0 0x562ea0884f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.409+0000 7f0cc9b39700 1 -- 172.21.15.44:0/3786807738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea088acc0 msgr2=0x562ea0893da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.409+0000 7f0cc9b39700 1 --2- 172.21.15.44:0/3786807738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea088acc0 0x562ea0893da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.409+0000 7f0cc9b39700 1 -- 172.21.15.44:0/3786807738 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ea05f1c00 con 0x562ea088a300 2023-09-10T01:09:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.409+0000 7f0cc9b39700 1 --2- 172.21.15.44:0/3786807738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088a300 0x562ea088b1b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0013010 tx=0x7f0cc0013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da86339b5736ee17 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.409+0000 7f0cb8ff9700 1 -- 172.21.15.44:0/3786807738 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0022040 con 0x562ea088a300 2023-09-10T01:09:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.409+0000 7f0cb8ff9700 1 -- 172.21.15.44:0/3786807738 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0cc0028420 con 0x562ea088a300 2023-09-10T01:09:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.409+0000 7f0cb8ff9700 1 -- 172.21.15.44:0/3786807738 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc001d670 con 0x562ea088a300 2023-09-10T01:09:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.410+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/3786807738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088a300 msgr2=0x562ea088b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.410+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/3786807738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088a300 0x562ea088b1b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0013010 tx=0x7f0cc0013040 comp rx=0 tx=0).stop 2023-09-10T01:09:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.410+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/3786807738 shutdown_connections 2023-09-10T01:09:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.410+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/3786807738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea08901a0 0x562ea0884f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.410+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/3786807738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea088a300 0x562ea088b1b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.410+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/3786807738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea088acc0 0x562ea0893da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.410+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/3786807738 >> 172.21.15.44:0/3786807738 conn(0x562ea07ff030 msgr2=0x562ea0893690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.410+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/3786807738 shutdown_connections 2023-09-10T01:09:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.410+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/3786807738 wait complete. 2023-09-10T01:09:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.410+0000 7f0cd7f63b80 1 -- start start 2023-09-10T01:09:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.410+0000 7f0cd7f63b80 1 Processor -- start 2023-09-10T01:09:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cd7f63b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea088a300 0x562ea08ebb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cd7f63b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea088acc0 0x562ea08ec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cd7f63b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea08901a0 0x562ea08f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cd7f63b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea08ec840 con 0x562ea088acc0 2023-09-10T01:09:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cd7f63b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea08f6f60 con 0x562ea08901a0 2023-09-10T01:09:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cd7f63b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea08f7120 con 0x562ea088a300 2023-09-10T01:09:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cd7f63b80 10 client.0 _ll_register_callbacks cb 0x7f0cc00096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:09:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cc9338700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea088acc0 0x562ea08ec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cc9338700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea088acc0 0x562ea08ec0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56464/0 (socket says 172.21.15.44:56464) 2023-09-10T01:09:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cc9338700 1 -- 172.21.15.44:0/920203278 learned_addr learned my addr 172.21.15.44:0/920203278 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:09:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cc9b39700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea088a300 0x562ea08ebb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cca33a700 1 --2- 172.21.15.44:0/920203278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea08901a0 0x562ea08f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cc9338700 1 -- 172.21.15.44:0/920203278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea088a300 msgr2=0x562ea08ebb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cc9338700 1 --2- 172.21.15.44:0/920203278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea088a300 0x562ea08ebb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cc9338700 1 -- 172.21.15.44:0/920203278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea08901a0 msgr2=0x562ea08f6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:09:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cc9338700 1 --2- 172.21.15.44:0/920203278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea08901a0 0x562ea08f6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cc9338700 1 -- 172.21.15.44:0/920203278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc0017070 con 0x562ea088acc0 2023-09-10T01:09:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cc9b39700 1 --2- 172.21.15.44:0/920203278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea088a300 0x562ea08ebb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:09:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cc9338700 1 --2- 172.21.15.44:0/920203278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea088acc0 0x562ea08ec0d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f0cb409a460 tx=0x7f0cb409e230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8a653e25d2adbc server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:09:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.411+0000 7f0cad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:09:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.412+0000 7f0cad7fa700 1 -- 172.21.15.44:0/920203278 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb409abd0 con 0x562ea088acc0 2023-09-10T01:09:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.412+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/920203278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cb409a4d0 con 0x562ea088acc0 2023-09-10T01:09:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.412+0000 7f0cad7fa700 1 -- 172.21.15.44:0/920203278 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f0cb40a24b0 con 0x562ea088acc0 2023-09-10T01:09:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.412+0000 7f0cad7fa700 1 -- 172.21.15.44:0/920203278 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cb409e300 con 0x562ea088acc0 2023-09-10T01:09:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.412+0000 7f0cd7f63b80 10 client.19035 fetch_fsmap learned FSMap version 114 2023-09-10T01:09:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.412+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/920203278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=4+}) v3 -- 0x562ea062f1f0 con 0x562ea088acc0 2023-09-10T01:09:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.412+0000 7f0cad7fa700 1 -- 172.21.15.44:0/920203278 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0cb409a4d0 con 0x562ea088acc0 2023-09-10T01:09:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.412+0000 7f0cad7fa700 1 -- 172.21.15.44:0/920203278 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 114) v1 ==== 7347+0+0 (secure 0 0 0) 0x7f0cb409dd80 con 0x562ea088acc0 2023-09-10T01:09:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.412+0000 7f0cd7f63b80 10 client.19035 fetch_fsmap finished waiting for FSMap version 114 2023-09-10T01:09:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.412+0000 7f0cd7f63b80 10 client.19035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:09:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.412+0000 7f0cd7f63b80 20 client.19035 populate_metadata read hostname 'smithi044' 2023-09-10T01:09:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.412+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/920203278 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ea08a3c20 0x562ea08fa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:09:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.412+0000 7f0cd7f63b80 4 client.19035 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:09:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.412+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/920203278 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562ea07d58c0 con 0x562ea08a3c20 2023-09-10T01:09:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.413+0000 7f0cc9b39700 1 --2- 172.21.15.44:0/920203278 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ea08a3c20 0x562ea08fa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:09:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.413+0000 7f0cad7fa700 10 client.19035 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.413+0000 7f0cc9b39700 1 --2- 172.21.15.44:0/920203278 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ea08a3c20 0x562ea08fa3e0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b46b117670e57602 server_cookie=b5e6ac6a1eeadc6a in_seq=0 out_seq=0 2023-09-10T01:09:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.426+0000 7f0cad7fa700 1 -- 172.21.15.44:0/920203278 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562ea07d58c0 con 0x562ea08a3c20 2023-09-10T01:09:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.426+0000 7f0cad7fa700 10 client.19035 handle_command_reply: tid=0 2023-09-10T01:09:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.426+0000 7f0cd7f63b80 1 client.19035 shutdown 2023-09-10T01:09:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.426+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/920203278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea088acc0 msgr2=0x562ea08ec0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.426+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/920203278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea088acc0 0x562ea08ec0d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f0cb409a460 tx=0x7f0cb409e230 comp rx=0 tx=0).stop 2023-09-10T01:09:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.426+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/920203278 shutdown_connections 2023-09-10T01:09:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.426+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/920203278 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ea08a3c20 0x562ea08fa3e0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.426+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/920203278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ea08901a0 0x562ea08f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.426+0000 7f0cad7fa700 0 client.19035 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:09:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.426+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/920203278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ea088a300 0x562ea08ebb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.426+0000 7f0cd7f63b80 1 --2- 172.21.15.44:0/920203278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ea088acc0 0x562ea08ec0d0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.426+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/920203278 >> 172.21.15.44:0/920203278 conn(0x562ea07ff030 msgr2=0x562ea08892f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.426+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/920203278 shutdown_connections 2023-09-10T01:09:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.426+0000 7f0cd7f63b80 1 -- 172.21.15.44:0/920203278 wait complete. 2023-09-10T01:09:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.426+0000 7f0cd7f63b80 20 client.19035 trim_cache size 0 max 16384 2023-09-10T01:09:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:09:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:09:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:09:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:09:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.427+0000 7f0ccbd9d700 1 -- 172.21.15.44:0/2126648563 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0cb00a07b0 msgr2=0x7f0cb00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.427+0000 7f0ccbd9d700 1 --2- 172.21.15.44:0/2126648563 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0cb00a07b0 0x7f0cb00a2c70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009c60 tx=0x7f0cc0009260 comp rx=0 tx=0).stop 2023-09-10T01:09:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.427+0000 7f0ccbd9d700 1 -- 172.21.15.44:0/2126648563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc40683b0 msgr2=0x7f0cc4196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:09:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.427+0000 7f0ccbd9d700 1 --2- 172.21.15.44:0/2126648563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc40683b0 0x7f0cc4196a40 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4005910 tx=0x7f0cb4009580 comp rx=0 tx=0).stop 2023-09-10T01:09:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.427+0000 7f0ccbd9d700 1 -- 172.21.15.44:0/2126648563 shutdown_connections 2023-09-10T01:09:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.427+0000 7f0ccbd9d700 1 --2- 172.21.15.44:0/2126648563 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0cb00a07b0 0x7f0cb00a2c70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.427+0000 7f0ccbd9d700 1 --2- 172.21.15.44:0/2126648563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cc4106650 0x7f0cc419a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.427+0000 7f0ccbd9d700 1 --2- 172.21.15.44:0/2126648563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cc40679f0 0x7f0cc4196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.427+0000 7f0ccbd9d700 1 --2- 172.21.15.44:0/2126648563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc40683b0 0x7f0cc4196a40 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:09:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.427+0000 7f0ccbd9d700 1 -- 172.21.15.44:0/2126648563 >> 172.21.15.44:0/2126648563 conn(0x7f0cc4074f30 msgr2=0x7f0cc40fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:09:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.427+0000 7f0ccbd9d700 1 -- 172.21.15.44:0/2126648563 shutdown_connections 2023-09-10T01:09:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:09:56.427+0000 7f0ccbd9d700 1 -- 172.21.15.44:0/2126648563 wait complete. 2023-09-10T01:09:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:56 smithi131 ceph-mon[103713]: pgmap v930: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-09-10T01:09:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:56 smithi044 ceph-mon[98676]: pgmap v930: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-09-10T01:09:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:56 smithi099 ceph-mon[102692]: pgmap v930: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-09-10T01:09:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:09:58 smithi131 ceph-mon[103713]: pgmap v931: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:09:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:09:58 smithi044 ceph-mon[98676]: pgmap v931: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:09:59.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:09:58 smithi099 ceph-mon[102692]: pgmap v931: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:10:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:00 smithi131 ceph-mon[103713]: pgmap v932: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:10:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:00 smithi131 ceph-mon[103713]: overall HEALTH_OK 2023-09-10T01:10:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:00 smithi044 ceph-mon[98676]: pgmap v932: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:10:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:00 smithi044 ceph-mon[98676]: overall HEALTH_OK 2023-09-10T01:10:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:00 smithi099 ceph-mon[102692]: pgmap v932: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:10:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:00 smithi099 ceph-mon[102692]: overall HEALTH_OK 2023-09-10T01:10:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:02 smithi131 ceph-mon[103713]: pgmap v933: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-09-10T01:10:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:02 smithi044 ceph-mon[98676]: pgmap v933: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-09-10T01:10:03.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:02 smithi099 ceph-mon[102692]: pgmap v933: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-09-10T01:10:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:04 smithi131 ceph-mon[103713]: pgmap v934: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 964 KiB/s wr, 138 op/s 2023-09-10T01:10:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:04 smithi044 ceph-mon[98676]: pgmap v934: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 964 KiB/s wr, 138 op/s 2023-09-10T01:10:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:04 smithi099 ceph-mon[102692]: pgmap v934: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 964 KiB/s wr, 138 op/s 2023-09-10T01:10:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:06 smithi131 ceph-mon[103713]: pgmap v935: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 228 op/s 2023-09-10T01:10:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:06 smithi044 ceph-mon[98676]: pgmap v935: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 228 op/s 2023-09-10T01:10:07.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:06 smithi099 ceph-mon[102692]: pgmap v935: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 228 op/s 2023-09-10T01:10:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:09 smithi131 ceph-mon[103713]: pgmap v936: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.0 MiB/s wr, 158 op/s 2023-09-10T01:10:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:09 smithi044 ceph-mon[98676]: pgmap v936: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.0 MiB/s wr, 158 op/s 2023-09-10T01:10:09.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:09 smithi099 ceph-mon[102692]: pgmap v936: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.0 MiB/s wr, 158 op/s 2023-09-10T01:10:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.577+0000 7f87d38c1700 1 -- 172.21.15.44:0/1318476646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87cc108fa0 msgr2=0x7f87cc109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.577+0000 7f87d38c1700 1 --2- 172.21.15.44:0/1318476646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87cc108fa0 0x7f87cc109380 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009990 tx=0x7f87c8009ca0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.577+0000 7f87d38c1700 1 -- 172.21.15.44:0/1318476646 shutdown_connections 2023-09-10T01:10:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.577+0000 7f87d38c1700 1 --2- 172.21.15.44:0/1318476646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87cc112550 0x7f87cc114940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.577+0000 7f87d38c1700 1 --2- 172.21.15.44:0/1318476646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87cc108fa0 0x7f87cc109380 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.577+0000 7f87d38c1700 1 --2- 172.21.15.44:0/1318476646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87cc1098c0 0x7f87cc112010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.577+0000 7f87d38c1700 1 -- 172.21.15.44:0/1318476646 >> 172.21.15.44:0/1318476646 conn(0x7f87cc06c6c0 msgr2=0x7f87cc06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.577+0000 7f87d38c1700 1 -- 172.21.15.44:0/1318476646 shutdown_connections 2023-09-10T01:10:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.577+0000 7f87d38c1700 1 -- 172.21.15.44:0/1318476646 wait complete. 2023-09-10T01:10:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.578+0000 7f87d38c1700 1 Processor -- start 2023-09-10T01:10:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.578+0000 7f87d38c1700 1 -- start start 2023-09-10T01:10:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.578+0000 7f87d38c1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87cc108fa0 0x7f87cc19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.578+0000 7f87d38c1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87cc1098c0 0x7f87cc19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.578+0000 7f87d38c1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87cc112550 0x7f87cc1a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.578+0000 7f87d38c1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc1a4090 con 0x7f87cc112550 2023-09-10T01:10:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.578+0000 7f87d38c1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc1a4200 con 0x7f87cc1098c0 2023-09-10T01:10:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.578+0000 7f87d38c1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc1a43c0 con 0x7f87cc108fa0 2023-09-10T01:10:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.578+0000 7f87d1e5e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87cc112550 0x7f87cc1a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.578+0000 7f87d165d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87cc108fa0 0x7f87cc19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.578+0000 7f87d1e5e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87cc112550 0x7f87cc1a3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56478/0 (socket says 172.21.15.44:56478) 2023-09-10T01:10:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.578+0000 7f87d165d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87cc108fa0 0x7f87cc19eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35126/0 (socket says 172.21.15.44:35126) 2023-09-10T01:10:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.579+0000 7f87d1e5e700 1 -- 172.21.15.44:0/2998415722 learned_addr learned my addr 172.21.15.44:0/2998415722 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.579+0000 7f87d0e5c700 1 --2- 172.21.15.44:0/2998415722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87cc1098c0 0x7f87cc19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.579+0000 7f87d1e5e700 1 -- 172.21.15.44:0/2998415722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87cc108fa0 msgr2=0x7f87cc19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.579+0000 7f87d1e5e700 1 --2- 172.21.15.44:0/2998415722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87cc108fa0 0x7f87cc19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.579+0000 7f87d1e5e700 1 -- 172.21.15.44:0/2998415722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87cc1098c0 msgr2=0x7f87cc19f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.579+0000 7f87d1e5e700 1 --2- 172.21.15.44:0/2998415722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87cc1098c0 0x7f87cc19f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.579+0000 7f87d1e5e700 1 -- 172.21.15.44:0/2998415722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c80095f0 con 0x7f87cc112550 2023-09-10T01:10:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.579+0000 7f87d1e5e700 1 --2- 172.21.15.44:0/2998415722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87cc112550 0x7f87cc1a3a50 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f87c400ef30 tx=0x7f87c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.579+0000 7f87c27fc700 1 -- 172.21.15.44:0/2998415722 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4004bb0 con 0x7f87cc112550 2023-09-10T01:10:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.579+0000 7f87c27fc700 1 -- 172.21.15.44:0/2998415722 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87c400f040 con 0x7f87cc112550 2023-09-10T01:10:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.579+0000 7f87d38c1700 1 -- 172.21.15.44:0/2998415722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc1a46a0 con 0x7f87cc112550 2023-09-10T01:10:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.579+0000 7f87c27fc700 1 -- 172.21.15.44:0/2998415722 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4004020 con 0x7f87cc112550 2023-09-10T01:10:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.579+0000 7f87d38c1700 1 -- 172.21.15.44:0/2998415722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc117850 con 0x7f87cc112550 2023-09-10T01:10:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.582+0000 7f87c27fc700 1 -- 172.21.15.44:0/2998415722 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f87c40041a0 con 0x7f87cc112550 2023-09-10T01:10:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.583+0000 7f87c27fc700 1 --2- 172.21.15.44:0/2998415722 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f87b80a07f0 0x7f87b80a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.583+0000 7f87c27fc700 1 -- 172.21.15.44:0/2998415722 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f87c40cfac0 con 0x7f87cc112550 2023-09-10T01:10:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.583+0000 7f87d165d700 1 --2- 172.21.15.44:0/2998415722 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f87b80a07f0 0x7f87b80a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.583+0000 7f87d165d700 1 --2- 172.21.15.44:0/2998415722 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f87b80a07f0 0x7f87b80a2cb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f87c800b2e0 tx=0x7f87c8005fb0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.584+0000 7f87dfa87b80 1 Processor -- start 2023-09-10T01:10:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87dfa87b80 1 -- start start 2023-09-10T01:10:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87dfa87b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc761a0 0x556f4bc76580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87dfa87b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 0x556f4bc70780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87dfa87b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc70cc0 0x556f4bc7cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87dfa87b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4bbc96d0 con 0x556f4bc70300 2023-09-10T01:10:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87dfa87b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4bbc6ff0 con 0x556f4bc761a0 2023-09-10T01:10:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87dfa87b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4bbc28e0 con 0x556f4bc70cc0 2023-09-10T01:10:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87d1e5e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc70cc0 0x556f4bc7cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87d0e5c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 0x556f4bc70780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87d1e5e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc70cc0 0x556f4bc7cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35138/0 (socket says 172.21.15.44:35138) 2023-09-10T01:10:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87d0e5c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 0x556f4bc70780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56486/0 (socket says 172.21.15.44:56486) 2023-09-10T01:10:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87d1e5e700 1 -- 172.21.15.44:0/3877096221 learned_addr learned my addr 172.21.15.44:0/3877096221 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87d165d700 1 --2- 172.21.15.44:0/3877096221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc761a0 0x556f4bc76580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87d1e5e700 1 -- 172.21.15.44:0/3877096221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc761a0 msgr2=0x556f4bc76580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87d1e5e700 1 --2- 172.21.15.44:0/3877096221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc761a0 0x556f4bc76580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87d1e5e700 1 -- 172.21.15.44:0/3877096221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 msgr2=0x556f4bc70780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87d1e5e700 1 --2- 172.21.15.44:0/3877096221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 0x556f4bc70780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.585+0000 7f87d1e5e700 1 -- 172.21.15.44:0/3877096221 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f4b9eed80 con 0x556f4bc70cc0 2023-09-10T01:10:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.586+0000 7f87d1e5e700 1 --2- 172.21.15.44:0/3877096221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc70cc0 0x556f4bc7cf30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f87c40996a0 tx=0x7f87c409c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=395198ae52b82c09 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.586+0000 7f87b77fe700 1 -- 172.21.15.44:0/3877096221 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c409d330 con 0x556f4bc70cc0 2023-09-10T01:10:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.586+0000 7f87b77fe700 1 -- 172.21.15.44:0/3877096221 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87c409cc30 con 0x556f4bc70cc0 2023-09-10T01:10:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.586+0000 7f87b77fe700 1 -- 172.21.15.44:0/3877096221 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c409c120 con 0x556f4bc70cc0 2023-09-10T01:10:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.587+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3877096221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc70cc0 msgr2=0x556f4bc7cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.587+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/3877096221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc70cc0 0x556f4bc7cf30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f87c40996a0 tx=0x7f87c409c840 comp rx=0 tx=0).stop 2023-09-10T01:10:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.587+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3877096221 shutdown_connections 2023-09-10T01:10:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.587+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/3877096221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc70cc0 0x556f4bc7cf30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.587+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/3877096221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc761a0 0x556f4bc76580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.587+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/3877096221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 0x556f4bc70780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.587+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3877096221 >> 172.21.15.44:0/3877096221 conn(0x556f4bbe5030 msgr2=0x556f4bc6b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.587+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3877096221 shutdown_connections 2023-09-10T01:10:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.587+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3877096221 wait complete. 2023-09-10T01:10:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.587+0000 7f87dfa87b80 1 -- start start 2023-09-10T01:10:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.588+0000 7f87dfa87b80 1 Processor -- start 2023-09-10T01:10:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.588+0000 7f87dfa87b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 0x556f4bc7ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.588+0000 7f87dfa87b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70cc0 0x556f4bc7a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.588+0000 7f87dfa87b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc761a0 0x556f4bc7a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.588+0000 7f87dfa87b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4bb60b10 con 0x556f4bc70300 2023-09-10T01:10:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.588+0000 7f87dfa87b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4bb60810 con 0x556f4bc70cc0 2023-09-10T01:10:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.588+0000 7f87dfa87b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4ba448c0 con 0x556f4bc761a0 2023-09-10T01:10:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.588+0000 7f87dfa87b80 10 client.0 _ll_register_callbacks cb 0x556f4baaaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:10:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.588+0000 7f87d165d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 0x556f4bc7ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.588+0000 7f87d165d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 0x556f4bc7ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56490/0 (socket says 172.21.15.44:56490) 2023-09-10T01:10:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.588+0000 7f87d165d700 1 -- 172.21.15.44:0/3589253149 learned_addr learned my addr 172.21.15.44:0/3589253149 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.588+0000 7f87d0e5c700 1 --2- 172.21.15.44:0/3589253149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70cc0 0x556f4bc7a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.589+0000 7f87d165d700 1 -- 172.21.15.44:0/3589253149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc761a0 msgr2=0x556f4bc7a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.589+0000 7f87d165d700 1 --2- 172.21.15.44:0/3589253149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc761a0 0x556f4bc7a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.589+0000 7f87d165d700 1 -- 172.21.15.44:0/3589253149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70cc0 msgr2=0x556f4bc7a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.589+0000 7f87d165d700 1 --2- 172.21.15.44:0/3589253149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70cc0 0x556f4bc7a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.589+0000 7f87d165d700 1 -- 172.21.15.44:0/3589253149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc117850 con 0x556f4bc70300 2023-09-10T01:10:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.589+0000 7f87d165d700 1 --2- 172.21.15.44:0/3589253149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 0x556f4bc7ce90 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f87c8015000 tx=0x7f87c8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a853ef279bbff1b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.589+0000 7f87b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:10:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.589+0000 7f87b4ff9700 1 -- 172.21.15.44:0/3589253149 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8005710 con 0x556f4bc70300 2023-09-10T01:10:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.589+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3589253149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f4bb1a490 con 0x556f4bc70300 2023-09-10T01:10:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.589+0000 7f87b4ff9700 1 -- 172.21.15.44:0/3589253149 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f87c800be40 con 0x556f4bc70300 2023-09-10T01:10:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.589+0000 7f87b4ff9700 1 -- 172.21.15.44:0/3589253149 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87c80398b0 con 0x556f4bc70300 2023-09-10T01:10:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.589+0000 7f87dfa87b80 10 client.19047 fetch_fsmap learned FSMap version 114 2023-09-10T01:10:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.589+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3589253149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=4+}) v3 -- 0x556f4b9d7c00 con 0x556f4bc70300 2023-09-10T01:10:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.590+0000 7f87b4ff9700 1 -- 172.21.15.44:0/3589253149 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556f4bb1a490 con 0x556f4bc70300 2023-09-10T01:10:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.590+0000 7f87b4ff9700 1 -- 172.21.15.44:0/3589253149 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 114) v1 ==== 7347+0+0 (secure 0 0 0) 0x7f87c802f070 con 0x556f4bc70300 2023-09-10T01:10:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.590+0000 7f87dfa87b80 10 client.19047 fetch_fsmap finished waiting for FSMap version 114 2023-09-10T01:10:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.590+0000 7f87dfa87b80 10 client.19047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:10:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.590+0000 7f87dfa87b80 20 client.19047 populate_metadata read hostname 'smithi044' 2023-09-10T01:10:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.590+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/3589253149 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556f4bc8a630 0x556f4bc8ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.590+0000 7f87dfa87b80 4 client.19047 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:10:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.590+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3589253149 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f4bbd7840 con 0x556f4bc8a630 2023-09-10T01:10:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.590+0000 7f87d0e5c700 1 --2- 172.21.15.44:0/3589253149 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556f4bc8a630 0x556f4bc8ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:09.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.591+0000 7f87d0e5c700 1 --2- 172.21.15.44:0/3589253149 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556f4bc8a630 0x556f4bc8ca50 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48f0c058c806347f server_cookie=b87617f8c06a2361 in_seq=0 out_seq=0 2023-09-10T01:10:09.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.591+0000 7f87b4ff9700 10 client.19047 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:10:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.591+0000 7f87b4ff9700 1 -- 172.21.15.44:0/3589253149 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556f4bbd7840 con 0x556f4bc8a630 2023-09-10T01:10:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.591+0000 7f87b4ff9700 10 client.19047 handle_command_reply: tid=0 2023-09-10T01:10:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.591+0000 7f87dfa87b80 1 client.19047 shutdown 2023-09-10T01:10:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.591+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3589253149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 msgr2=0x556f4bc7ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.591+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/3589253149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 0x556f4bc7ce90 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f87c8015000 tx=0x7f87c8009ca0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.592+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3589253149 shutdown_connections 2023-09-10T01:10:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.592+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/3589253149 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556f4bc8a630 0x556f4bc8ca50 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.592+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/3589253149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc761a0 0x556f4bc7a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.592+0000 7f87b4ff9700 0 client.19047 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:10:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.592+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/3589253149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 0x556f4bc7ce90 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.592+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/3589253149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70cc0 0x556f4bc7a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.592+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3589253149 >> 172.21.15.44:0/3589253149 conn(0x556f4bbe5030 msgr2=0x556f4bc6a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.592+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3589253149 shutdown_connections 2023-09-10T01:10:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.592+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3589253149 wait complete. 2023-09-10T01:10:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.592+0000 7f87dfa87b80 20 client.19047 trim_cache size 0 max 16384 2023-09-10T01:10:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.612+0000 7f87dfa87b80 1 Processor -- start 2023-09-10T01:10:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.612+0000 7f87dfa87b80 1 -- start start 2023-09-10T01:10:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.612+0000 7f87dfa87b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70300 0x556f4bc77d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.612+0000 7f87dfa87b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc70cc0 0x556f4bc7ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.612+0000 7f87dfa87b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc761a0 0x556f4bc798d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.612+0000 7f87dfa87b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4bc710a0 con 0x556f4bc761a0 2023-09-10T01:10:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.612+0000 7f87dfa87b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4bc7b6f0 con 0x556f4bc70300 2023-09-10T01:10:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.612+0000 7f87dfa87b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4bc6c270 con 0x556f4bc70cc0 2023-09-10T01:10:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.613+0000 7f87d1e5e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc761a0 0x556f4bc798d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.613+0000 7f87d0e5c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc70cc0 0x556f4bc7ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.613+0000 7f87d0e5c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc70cc0 0x556f4bc7ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35152/0 (socket says 172.21.15.44:35152) 2023-09-10T01:10:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.613+0000 7f87d1e5e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc761a0 0x556f4bc798d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56498/0 (socket says 172.21.15.44:56498) 2023-09-10T01:10:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.613+0000 7f87d0e5c700 1 -- 172.21.15.44:0/1718943301 learned_addr learned my addr 172.21.15.44:0/1718943301 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.613+0000 7f87d165d700 1 --2- 172.21.15.44:0/1718943301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70300 0x556f4bc77d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.613+0000 7f87d1e5e700 1 -- 172.21.15.44:0/1718943301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc70cc0 msgr2=0x556f4bc7ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.613+0000 7f87d1e5e700 1 --2- 172.21.15.44:0/1718943301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc70cc0 0x556f4bc7ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.613+0000 7f87d1e5e700 1 -- 172.21.15.44:0/1718943301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70300 msgr2=0x556f4bc77d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.613+0000 7f87d1e5e700 1 --2- 172.21.15.44:0/1718943301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70300 0x556f4bc77d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.613+0000 7f87d1e5e700 1 -- 172.21.15.44:0/1718943301 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f4ba151f0 con 0x556f4bc761a0 2023-09-10T01:10:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.613+0000 7f87d1e5e700 1 --2- 172.21.15.44:0/1718943301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc761a0 0x556f4bc798d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f87c40090a0 tx=0x7f87c401af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966ea7499b995d98 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.613+0000 7f87b7fff700 1 -- 172.21.15.44:0/1718943301 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c409d330 con 0x556f4bc761a0 2023-09-10T01:10:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.614+0000 7f87b7fff700 1 -- 172.21.15.44:0/1718943301 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87c40a0a60 con 0x556f4bc761a0 2023-09-10T01:10:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.614+0000 7f87b7fff700 1 -- 172.21.15.44:0/1718943301 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c409c3f0 con 0x556f4bc761a0 2023-09-10T01:10:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.614+0000 7f87dfa87b80 1 -- 172.21.15.44:0/1718943301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc761a0 msgr2=0x556f4bc798d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.614+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/1718943301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc761a0 0x556f4bc798d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f87c40090a0 tx=0x7f87c401af40 comp rx=0 tx=0).stop 2023-09-10T01:10:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.616+0000 7f87dfa87b80 1 -- 172.21.15.44:0/1718943301 shutdown_connections 2023-09-10T01:10:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.616+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/1718943301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc761a0 0x556f4bc798d0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.616+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/1718943301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70300 0x556f4bc77d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.616+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/1718943301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc70cc0 0x556f4bc7ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.616+0000 7f87dfa87b80 1 -- 172.21.15.44:0/1718943301 >> 172.21.15.44:0/1718943301 conn(0x556f4bbe5030 msgr2=0x556f4bc6b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.616+0000 7f87dfa87b80 1 -- 172.21.15.44:0/1718943301 shutdown_connections 2023-09-10T01:10:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.616+0000 7f87dfa87b80 1 -- 172.21.15.44:0/1718943301 wait complete. 2023-09-10T01:10:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.616+0000 7f87dfa87b80 1 -- start start 2023-09-10T01:10:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.616+0000 7f87dfa87b80 1 Processor -- start 2023-09-10T01:10:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.617+0000 7f87dfa87b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 0x556f4bc81130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.617+0000 7f87dfa87b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70cc0 0x556f4bc81670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.617+0000 7f87dfa87b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc761a0 0x556f4bc7f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.617+0000 7f87dfa87b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4bc7fd10 con 0x556f4bc70300 2023-09-10T01:10:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.617+0000 7f87dfa87b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4bc7fe80 con 0x556f4bc70cc0 2023-09-10T01:10:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.617+0000 7f87dfa87b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4bc80040 con 0x556f4bc761a0 2023-09-10T01:10:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.617+0000 7f87dfa87b80 10 client.0 _ll_register_callbacks cb 0x7f87c800ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:10:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.617+0000 7f87d0e5c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70cc0 0x556f4bc81670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.617+0000 7f87d0e5c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70cc0 0x556f4bc81670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58244/0 (socket says 172.21.15.44:58244) 2023-09-10T01:10:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.617+0000 7f87d0e5c700 1 -- 172.21.15.44:0/3216633573 learned_addr learned my addr 172.21.15.44:0/3216633573 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.617+0000 7f87d1e5e700 1 --2- 172.21.15.44:0/3216633573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc761a0 0x556f4bc7f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.617+0000 7f87d0e5c700 1 -- 172.21.15.44:0/3216633573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc761a0 msgr2=0x556f4bc7f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.617+0000 7f87d0e5c700 1 --2- 172.21.15.44:0/3216633573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc761a0 0x556f4bc7f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.617+0000 7f87d0e5c700 1 -- 172.21.15.44:0/3216633573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 msgr2=0x556f4bc81130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.617+0000 7f87d0e5c700 1 --2- 172.21.15.44:0/3216633573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 0x556f4bc81130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87d0e5c700 1 -- 172.21.15.44:0/3216633573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f4b9eed80 con 0x556f4bc70cc0 2023-09-10T01:10:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87d0e5c700 1 --2- 172.21.15.44:0/3216633573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70cc0 0x556f4bc81670 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x556f4bc6b060 tx=0x7f87bc009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77eda33f453c721e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:10:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87b4ff9700 1 -- 172.21.15.44:0/3216633573 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc017040 con 0x556f4bc70cc0 2023-09-10T01:10:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3216633573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87c409cc30 con 0x556f4bc70cc0 2023-09-10T01:10:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87b4ff9700 1 -- 172.21.15.44:0/3216633573 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f87bc004040 con 0x556f4bc70cc0 2023-09-10T01:10:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87b4ff9700 1 -- 172.21.15.44:0/3216633573 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87bc004200 con 0x556f4bc70cc0 2023-09-10T01:10:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87dfa87b80 10 client.28715 fetch_fsmap learned FSMap version 114 2023-09-10T01:10:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3216633573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=4+}) v3 -- 0x556f4b9ed250 con 0x556f4bc70cc0 2023-09-10T01:10:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87b4ff9700 1 -- 172.21.15.44:0/3216633573 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87c409cc30 con 0x556f4bc70cc0 2023-09-10T01:10:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87b4ff9700 1 -- 172.21.15.44:0/3216633573 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 114) v1 ==== 7347+0+0 (secure 0 0 0) 0x7f87bc012450 con 0x556f4bc70cc0 2023-09-10T01:10:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87dfa87b80 10 client.28715 fetch_fsmap finished waiting for FSMap version 114 2023-09-10T01:10:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87dfa87b80 10 client.28715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:10:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87dfa87b80 20 client.28715 populate_metadata read hostname 'smithi044' 2023-09-10T01:10:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/3216633573 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556f4bcd4b90 0x556f4bcd6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87dfa87b80 4 client.28715 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:10:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.618+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3216633573 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556f4bbbb8c0 con 0x556f4bcd4b90 2023-09-10T01:10:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.619+0000 7f87d165d700 1 --2- 172.21.15.44:0/3216633573 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556f4bcd4b90 0x556f4bcd6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.619+0000 7f87b4ff9700 10 client.28715 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:10:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.619+0000 7f87d165d700 1 --2- 172.21.15.44:0/3216633573 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556f4bcd4b90 0x556f4bcd6fb0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e435164521e314 server_cookie=8574955d3c241851 in_seq=0 out_seq=0 2023-09-10T01:10:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.628+0000 7f87b4ff9700 1 -- 172.21.15.44:0/3216633573 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556f4bbbb8c0 con 0x556f4bcd4b90 2023-09-10T01:10:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.628+0000 7f87b4ff9700 10 client.28715 handle_command_reply: tid=0 2023-09-10T01:10:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.628+0000 7f87dfa87b80 1 client.28715 shutdown 2023-09-10T01:10:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.628+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3216633573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70cc0 msgr2=0x556f4bc81670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.628+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/3216633573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70cc0 0x556f4bc81670 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x556f4bc6b060 tx=0x7f87bc009c60 comp rx=0 tx=0).stop 2023-09-10T01:10:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.628+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3216633573 shutdown_connections 2023-09-10T01:10:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.628+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/3216633573 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556f4bcd4b90 0x556f4bcd6fb0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.628+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/3216633573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4bc761a0 0x556f4bc7f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.628+0000 7f87b4ff9700 0 client.28715 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:10:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.628+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/3216633573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f4bc70300 0x556f4bc81130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.628+0000 7f87dfa87b80 1 --2- 172.21.15.44:0/3216633573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f4bc70cc0 0x556f4bc81670 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.628+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3216633573 >> 172.21.15.44:0/3216633573 conn(0x556f4bbe5030 msgr2=0x556f4bc6f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.628+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3216633573 shutdown_connections 2023-09-10T01:10:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.628+0000 7f87dfa87b80 1 -- 172.21.15.44:0/3216633573 wait complete. 2023-09-10T01:10:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.628+0000 7f87dfa87b80 20 client.28715 trim_cache size 0 max 16384 2023-09-10T01:10:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:10:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:10:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:10:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:10:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.629+0000 7f87d38c1700 1 -- 172.21.15.44:0/2998415722 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f87b80a07f0 msgr2=0x7f87b80a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.629+0000 7f87d38c1700 1 --2- 172.21.15.44:0/2998415722 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f87b80a07f0 0x7f87b80a2cb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f87c800b2e0 tx=0x7f87c8005fb0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.629+0000 7f87d38c1700 1 -- 172.21.15.44:0/2998415722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87cc112550 msgr2=0x7f87cc1a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.629+0000 7f87d38c1700 1 --2- 172.21.15.44:0/2998415722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87cc112550 0x7f87cc1a3a50 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f87c400ef30 tx=0x7f87c400c3a0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.629+0000 7f87d38c1700 1 -- 172.21.15.44:0/2998415722 shutdown_connections 2023-09-10T01:10:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.629+0000 7f87d38c1700 1 --2- 172.21.15.44:0/2998415722 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f87b80a07f0 0x7f87b80a2cb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.629+0000 7f87d38c1700 1 --2- 172.21.15.44:0/2998415722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87cc112550 0x7f87cc1a3a50 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.629+0000 7f87d38c1700 1 --2- 172.21.15.44:0/2998415722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87cc108fa0 0x7f87cc19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.629+0000 7f87d38c1700 1 --2- 172.21.15.44:0/2998415722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87cc1098c0 0x7f87cc19f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.629+0000 7f87d38c1700 1 -- 172.21.15.44:0/2998415722 >> 172.21.15.44:0/2998415722 conn(0x7f87cc06c6c0 msgr2=0x7f87cc109f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.631+0000 7f87d38c1700 1 -- 172.21.15.44:0/2998415722 shutdown_connections 2023-09-10T01:10:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:09.631+0000 7f87d38c1700 1 -- 172.21.15.44:0/2998415722 wait complete. 2023-09-10T01:10:10.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:10 smithi131 ceph-mon[103713]: osdmap e142: 12 total, 12 up, 12 in 2023-09-10T01:10:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:10 smithi044 ceph-mon[98676]: osdmap e142: 12 total, 12 up, 12 in 2023-09-10T01:10:10.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:10 smithi099 ceph-mon[102692]: osdmap e142: 12 total, 12 up, 12 in 2023-09-10T01:10:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:11 smithi131 ceph-mon[103713]: pgmap v938: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 190 op/s 2023-09-10T01:10:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:11 smithi044 ceph-mon[98676]: pgmap v938: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 190 op/s 2023-09-10T01:10:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:11 smithi099 ceph-mon[102692]: pgmap v938: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 190 op/s 2023-09-10T01:10:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:13 smithi131 ceph-mon[103713]: pgmap v939: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2023-09-10T01:10:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:13 smithi044 ceph-mon[98676]: pgmap v939: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2023-09-10T01:10:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:13 smithi099 ceph-mon[102692]: pgmap v939: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2023-09-10T01:10:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:15 smithi131 ceph-mon[103713]: pgmap v940: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2023-09-10T01:10:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:15 smithi044 ceph-mon[98676]: pgmap v940: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2023-09-10T01:10:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:15 smithi099 ceph-mon[102692]: pgmap v940: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2023-09-10T01:10:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:16 smithi131 ceph-mon[103713]: pgmap v941: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2023-09-10T01:10:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:16 smithi131 ceph-mon[103713]: osdmap e143: 12 total, 12 up, 12 in 2023-09-10T01:10:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:16 smithi044 ceph-mon[98676]: pgmap v941: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2023-09-10T01:10:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:16 smithi044 ceph-mon[98676]: osdmap e143: 12 total, 12 up, 12 in 2023-09-10T01:10:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:16 smithi099 ceph-mon[102692]: pgmap v941: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2023-09-10T01:10:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:16 smithi099 ceph-mon[102692]: osdmap e143: 12 total, 12 up, 12 in 2023-09-10T01:10:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:18 smithi131 ceph-mon[103713]: pgmap v943: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 226 op/s 2023-09-10T01:10:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:18 smithi044 ceph-mon[98676]: pgmap v943: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 226 op/s 2023-09-10T01:10:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:18 smithi099 ceph-mon[102692]: pgmap v943: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 226 op/s 2023-09-10T01:10:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:20 smithi131 ceph-mon[103713]: pgmap v944: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2023-09-10T01:10:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:20 smithi044 ceph-mon[98676]: pgmap v944: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2023-09-10T01:10:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:20 smithi099 ceph-mon[102692]: pgmap v944: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2023-09-10T01:10:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.769+0000 7fb85728e700 1 -- 172.21.15.44:0/2860834042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8500679f0 msgr2=0x7fb850067e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.769+0000 7fb85728e700 1 --2- 172.21.15.44:0/2860834042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8500679f0 0x7fb850067e70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb84000b0d0 tx=0x7fb84000b3e0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.769+0000 7fb85728e700 1 -- 172.21.15.44:0/2860834042 shutdown_connections 2023-09-10T01:10:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.769+0000 7fb85728e700 1 --2- 172.21.15.44:0/2860834042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8500683b0 0x7fb85010d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.769+0000 7fb85728e700 1 --2- 172.21.15.44:0/2860834042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb850106650 0x7fb850106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.769+0000 7fb85728e700 1 --2- 172.21.15.44:0/2860834042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8500679f0 0x7fb850067e70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.769+0000 7fb85728e700 1 -- 172.21.15.44:0/2860834042 >> 172.21.15.44:0/2860834042 conn(0x7fb850074f30 msgr2=0x7fb850075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.769+0000 7fb85728e700 1 -- 172.21.15.44:0/2860834042 shutdown_connections 2023-09-10T01:10:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.770+0000 7fb85728e700 1 -- 172.21.15.44:0/2860834042 wait complete. 2023-09-10T01:10:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.770+0000 7fb85728e700 1 Processor -- start 2023-09-10T01:10:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.770+0000 7fb85728e700 1 -- start start 2023-09-10T01:10:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85728e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8500679f0 0x7fb8500ff830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85728e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8500683b0 0x7fb8500ffd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85728e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb850106650 0x7fb8501002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85728e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8501a9190 con 0x7fb850106650 2023-09-10T01:10:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85728e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85019a880 con 0x7fb8500683b0 2023-09-10T01:10:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85728e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85019aa40 con 0x7fb8500679f0 2023-09-10T01:10:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85582b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb850106650 0x7fb8501002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85582b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb850106650 0x7fb8501002b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56522/0 (socket says 172.21.15.44:56522) 2023-09-10T01:10:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb854829700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8500683b0 0x7fb8500ffd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85582b700 1 -- 172.21.15.44:0/3689053476 learned_addr learned my addr 172.21.15.44:0/3689053476 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85502a700 1 --2- 172.21.15.44:0/3689053476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8500679f0 0x7fb8500ff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85582b700 1 -- 172.21.15.44:0/3689053476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8500679f0 msgr2=0x7fb8500ff830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85582b700 1 --2- 172.21.15.44:0/3689053476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8500679f0 0x7fb8500ff830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85582b700 1 -- 172.21.15.44:0/3689053476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8500683b0 msgr2=0x7fb8500ffd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85582b700 1 --2- 172.21.15.44:0/3689053476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8500683b0 0x7fb8500ffd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85582b700 1 -- 172.21.15.44:0/3689053476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb840009d00 con 0x7fb850106650 2023-09-10T01:10:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85502a700 1 --2- 172.21.15.44:0/3689053476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8500679f0 0x7fb8500ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:10:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb854829700 1 --2- 172.21.15.44:0/3689053476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8500683b0 0x7fb8500ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:10:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.771+0000 7fb85582b700 1 --2- 172.21.15.44:0/3689053476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb850106650 0x7fb8501002b0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fb84400e940 tx=0x7fb84400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.772+0000 7fb84e7fc700 1 -- 172.21.15.44:0/3689053476 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8440097b0 con 0x7fb850106650 2023-09-10T01:10:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.772+0000 7fb85728e700 1 -- 172.21.15.44:0/3689053476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85019ad20 con 0x7fb850106650 2023-09-10T01:10:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.772+0000 7fb84e7fc700 1 -- 172.21.15.44:0/3689053476 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb844005020 con 0x7fb850106650 2023-09-10T01:10:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.772+0000 7fb84e7fc700 1 -- 172.21.15.44:0/3689053476 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84400cb70 con 0x7fb850106650 2023-09-10T01:10:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.772+0000 7fb85728e700 1 -- 172.21.15.44:0/3689053476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85019b1e0 con 0x7fb850106650 2023-09-10T01:10:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.774+0000 7fb84e7fc700 1 -- 172.21.15.44:0/3689053476 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fb84400cd30 con 0x7fb850106650 2023-09-10T01:10:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.775+0000 7fb84e7fc700 1 --2- 172.21.15.44:0/3689053476 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb8380a07b0 0x7fb8380a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.775+0000 7fb84e7fc700 1 -- 172.21.15.44:0/3689053476 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb844014070 con 0x7fb850106650 2023-09-10T01:10:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.775+0000 7fb85502a700 1 --2- 172.21.15.44:0/3689053476 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb8380a07b0 0x7fb8380a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.776+0000 7fb85502a700 1 --2- 172.21.15.44:0/3689053476 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb8380a07b0 0x7fb8380a2c70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb848009fd0 tx=0x7fb848009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.776+0000 7fb863454b80 1 Processor -- start 2023-09-10T01:10:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.776+0000 7fb863454b80 1 -- start start 2023-09-10T01:10:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb863454b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a681a0 0x555bb8a68580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb863454b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a62300 0x555bb8a62780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb863454b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62cc0 0x555bb8a6ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb863454b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb89bb6d0 con 0x555bb8a62300 2023-09-10T01:10:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb863454b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb89b8ff0 con 0x555bb8a62cc0 2023-09-10T01:10:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb863454b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb89b48e0 con 0x555bb8a681a0 2023-09-10T01:10:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb85502a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a681a0 0x555bb8a68580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb854829700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a62300 0x555bb8a62780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb85502a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a681a0 0x555bb8a68580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35176/0 (socket says 172.21.15.44:35176) 2023-09-10T01:10:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb85502a700 1 -- 172.21.15.44:0/222880481 learned_addr learned my addr 172.21.15.44:0/222880481 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb85582b700 1 --2- 172.21.15.44:0/222880481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62cc0 0x555bb8a6ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb85502a700 1 -- 172.21.15.44:0/222880481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62cc0 msgr2=0x555bb8a6ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb85502a700 1 --2- 172.21.15.44:0/222880481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62cc0 0x555bb8a6ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb85502a700 1 -- 172.21.15.44:0/222880481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a62300 msgr2=0x555bb8a62780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb85502a700 1 --2- 172.21.15.44:0/222880481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a62300 0x555bb8a62780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb85502a700 1 -- 172.21.15.44:0/222880481 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bb87e0d80 con 0x555bb8a681a0 2023-09-10T01:10:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.777+0000 7fb85502a700 1 --2- 172.21.15.44:0/222880481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a681a0 0x555bb8a68580 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb84800e720 tx=0x7fb848014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=634fbfad04249d1f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.778+0000 7fb8377fe700 1 -- 172.21.15.44:0/222880481 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb848026600 con 0x555bb8a681a0 2023-09-10T01:10:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.778+0000 7fb8377fe700 1 -- 172.21.15.44:0/222880481 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb848026bf0 con 0x555bb8a681a0 2023-09-10T01:10:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.778+0000 7fb8377fe700 1 -- 172.21.15.44:0/222880481 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84801eb30 con 0x555bb8a681a0 2023-09-10T01:10:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.779+0000 7fb863454b80 1 -- 172.21.15.44:0/222880481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a681a0 msgr2=0x555bb8a68580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.779+0000 7fb863454b80 1 --2- 172.21.15.44:0/222880481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a681a0 0x555bb8a68580 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb84800e720 tx=0x7fb848014000 comp rx=0 tx=0).stop 2023-09-10T01:10:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.779+0000 7fb863454b80 1 -- 172.21.15.44:0/222880481 shutdown_connections 2023-09-10T01:10:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.779+0000 7fb863454b80 1 --2- 172.21.15.44:0/222880481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62cc0 0x555bb8a6ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.779+0000 7fb863454b80 1 --2- 172.21.15.44:0/222880481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a681a0 0x555bb8a68580 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.779+0000 7fb863454b80 1 --2- 172.21.15.44:0/222880481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a62300 0x555bb8a62780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.779+0000 7fb863454b80 1 -- 172.21.15.44:0/222880481 >> 172.21.15.44:0/222880481 conn(0x555bb89d7030 msgr2=0x555bb8a5d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.779+0000 7fb863454b80 1 -- 172.21.15.44:0/222880481 shutdown_connections 2023-09-10T01:10:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.779+0000 7fb863454b80 1 -- 172.21.15.44:0/222880481 wait complete. 2023-09-10T01:10:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.779+0000 7fb863454b80 1 -- start start 2023-09-10T01:10:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.779+0000 7fb863454b80 1 Processor -- start 2023-09-10T01:10:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb863454b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62300 0x555bb8a6ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb863454b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a62cc0 0x555bb8a6c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb863454b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a681a0 0x555bb8a6c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb863454b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb8952b10 con 0x555bb8a681a0 2023-09-10T01:10:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb863454b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb8952810 con 0x555bb8a62300 2023-09-10T01:10:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb863454b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb88368c0 con 0x555bb8a62cc0 2023-09-10T01:10:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb863454b80 10 client.0 _ll_register_callbacks cb 0x555bb889ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:10:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb85502a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62300 0x555bb8a6ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb854829700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a62cc0 0x555bb8a6c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb85502a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62300 0x555bb8a6ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58272/0 (socket says 172.21.15.44:58272) 2023-09-10T01:10:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb85502a700 1 -- 172.21.15.44:0/264907258 learned_addr learned my addr 172.21.15.44:0/264907258 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb85582b700 1 --2- 172.21.15.44:0/264907258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a681a0 0x555bb8a6c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb85502a700 1 -- 172.21.15.44:0/264907258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a62cc0 msgr2=0x555bb8a6c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb85502a700 1 --2- 172.21.15.44:0/264907258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a62cc0 0x555bb8a6c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb85502a700 1 -- 172.21.15.44:0/264907258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a681a0 msgr2=0x555bb8a6c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb85502a700 1 --2- 172.21.15.44:0/264907258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a681a0 0x555bb8a6c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.780+0000 7fb85502a700 1 -- 172.21.15.44:0/264907258 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb848017070 con 0x555bb8a62300 2023-09-10T01:10:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.781+0000 7fb85502a700 1 --2- 172.21.15.44:0/264907258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62300 0x555bb8a6ed70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb848005950 tx=0x7fb8480268c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c74624d27f21b93e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.781+0000 7fb834ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:10:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.781+0000 7fb834ff9700 1 -- 172.21.15.44:0/264907258 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84802c070 con 0x555bb8a62300 2023-09-10T01:10:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.781+0000 7fb834ff9700 1 -- 172.21.15.44:0/264907258 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb8480052d0 con 0x555bb8a62300 2023-09-10T01:10:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.781+0000 7fb863454b80 1 -- 172.21.15.44:0/264907258 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bb890c490 con 0x555bb8a62300 2023-09-10T01:10:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.781+0000 7fb834ff9700 1 -- 172.21.15.44:0/264907258 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb84801e570 con 0x555bb8a62300 2023-09-10T01:10:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.781+0000 7fb863454b80 10 client.28727 fetch_fsmap learned FSMap version 114 2023-09-10T01:10:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.781+0000 7fb863454b80 1 -- 172.21.15.44:0/264907258 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=4+}) v3 -- 0x555bb87c9c00 con 0x555bb8a62300 2023-09-10T01:10:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.781+0000 7fb834ff9700 1 -- 172.21.15.44:0/264907258 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555bb890c490 con 0x555bb8a62300 2023-09-10T01:10:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.781+0000 7fb834ff9700 1 -- 172.21.15.44:0/264907258 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 114) v1 ==== 7347+0+0 (secure 0 0 0) 0x7fb848024020 con 0x555bb8a62300 2023-09-10T01:10:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.781+0000 7fb863454b80 10 client.28727 fetch_fsmap finished waiting for FSMap version 114 2023-09-10T01:10:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.781+0000 7fb863454b80 10 client.28727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:10:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.781+0000 7fb863454b80 20 client.28727 populate_metadata read hostname 'smithi044' 2023-09-10T01:10:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.781+0000 7fb863454b80 1 --2- 172.21.15.44:0/264907258 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555bb8a7c280 0x555bb8a7e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.781+0000 7fb863454b80 4 client.28727 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:10:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.781+0000 7fb863454b80 1 -- 172.21.15.44:0/264907258 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bb89c9840 con 0x555bb8a7c280 2023-09-10T01:10:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.782+0000 7fb854829700 1 --2- 172.21.15.44:0/264907258 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555bb8a7c280 0x555bb8a7e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.782+0000 7fb854829700 1 --2- 172.21.15.44:0/264907258 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555bb8a7c280 0x555bb8a7e6a0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60f36e7517e57883 server_cookie=1101e5b9631aa738 in_seq=0 out_seq=0 2023-09-10T01:10:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.782+0000 7fb834ff9700 10 client.28727 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:10:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.782+0000 7fb834ff9700 1 -- 172.21.15.44:0/264907258 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555bb89c9840 con 0x555bb8a7c280 2023-09-10T01:10:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.782+0000 7fb834ff9700 10 client.28727 handle_command_reply: tid=0 2023-09-10T01:10:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.783+0000 7fb863454b80 1 client.28727 shutdown 2023-09-10T01:10:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.783+0000 7fb863454b80 1 -- 172.21.15.44:0/264907258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62300 msgr2=0x555bb8a6ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.783+0000 7fb863454b80 1 --2- 172.21.15.44:0/264907258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62300 0x555bb8a6ed70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb848005950 tx=0x7fb8480268c0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.783+0000 7fb863454b80 1 -- 172.21.15.44:0/264907258 shutdown_connections 2023-09-10T01:10:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.783+0000 7fb863454b80 1 --2- 172.21.15.44:0/264907258 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555bb8a7c280 0x555bb8a7e6a0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.783+0000 7fb863454b80 1 --2- 172.21.15.44:0/264907258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a681a0 0x555bb8a6c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.783+0000 7fb834ff9700 0 client.28727 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:10:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.783+0000 7fb863454b80 1 --2- 172.21.15.44:0/264907258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62300 0x555bb8a6ed70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.783+0000 7fb863454b80 1 --2- 172.21.15.44:0/264907258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a62cc0 0x555bb8a6c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.783+0000 7fb863454b80 1 -- 172.21.15.44:0/264907258 >> 172.21.15.44:0/264907258 conn(0x555bb89d7030 msgr2=0x555bb8a5c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.783+0000 7fb863454b80 1 -- 172.21.15.44:0/264907258 shutdown_connections 2023-09-10T01:10:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.783+0000 7fb863454b80 1 -- 172.21.15.44:0/264907258 wait complete. 2023-09-10T01:10:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.783+0000 7fb863454b80 20 client.28727 trim_cache size 0 max 16384 2023-09-10T01:10:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.801+0000 7fb863454b80 1 Processor -- start 2023-09-10T01:10:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.801+0000 7fb863454b80 1 -- start start 2023-09-10T01:10:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.801+0000 7fb863454b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a62300 0x555bb8a5e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.802+0000 7fb863454b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62cc0 0x555bb8a61a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.802+0000 7fb863454b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a681a0 0x555bb8a6e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.802+0000 7fb863454b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb8a6d0a0 con 0x555bb8a681a0 2023-09-10T01:10:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.802+0000 7fb863454b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb8a6d640 con 0x555bb8a62cc0 2023-09-10T01:10:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.802+0000 7fb863454b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb8a66da0 con 0x555bb8a62300 2023-09-10T01:10:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.802+0000 7fb854829700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62cc0 0x555bb8a61a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.802+0000 7fb854829700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62cc0 0x555bb8a61a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58280/0 (socket says 172.21.15.44:58280) 2023-09-10T01:10:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.802+0000 7fb854829700 1 -- 172.21.15.44:0/975855832 learned_addr learned my addr 172.21.15.44:0/975855832 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.802+0000 7fb85582b700 1 --2- 172.21.15.44:0/975855832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a681a0 0x555bb8a6e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.802+0000 7fb854829700 1 -- 172.21.15.44:0/975855832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a62300 msgr2=0x555bb8a5e930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:10:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.802+0000 7fb854829700 1 --2- 172.21.15.44:0/975855832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a62300 0x555bb8a5e930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.802+0000 7fb854829700 1 -- 172.21.15.44:0/975855832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a681a0 msgr2=0x555bb8a6e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.802+0000 7fb854829700 1 --2- 172.21.15.44:0/975855832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a681a0 0x555bb8a6e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.802+0000 7fb854829700 1 -- 172.21.15.44:0/975855832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bb88071f0 con 0x555bb8a62cc0 2023-09-10T01:10:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.802+0000 7fb854829700 1 --2- 172.21.15.44:0/975855832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62cc0 0x555bb8a61a00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb840007590 tx=0x7fb84000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=484781a2fc0332db server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.803+0000 7fb837fff700 1 -- 172.21.15.44:0/975855832 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb840021040 con 0x555bb8a62cc0 2023-09-10T01:10:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.803+0000 7fb837fff700 1 -- 172.21.15.44:0/975855832 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb840007b90 con 0x555bb8a62cc0 2023-09-10T01:10:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.803+0000 7fb837fff700 1 -- 172.21.15.44:0/975855832 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb840025730 con 0x555bb8a62cc0 2023-09-10T01:10:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.803+0000 7fb863454b80 1 -- 172.21.15.44:0/975855832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62cc0 msgr2=0x555bb8a61a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.803+0000 7fb863454b80 1 --2- 172.21.15.44:0/975855832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62cc0 0x555bb8a61a00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb840007590 tx=0x7fb84000b040 comp rx=0 tx=0).stop 2023-09-10T01:10:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.803+0000 7fb863454b80 1 -- 172.21.15.44:0/975855832 shutdown_connections 2023-09-10T01:10:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.803+0000 7fb863454b80 1 --2- 172.21.15.44:0/975855832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a681a0 0x555bb8a6e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.803+0000 7fb863454b80 1 --2- 172.21.15.44:0/975855832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a62300 0x555bb8a5e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.803+0000 7fb863454b80 1 --2- 172.21.15.44:0/975855832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a62cc0 0x555bb8a61a00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.803+0000 7fb863454b80 1 -- 172.21.15.44:0/975855832 >> 172.21.15.44:0/975855832 conn(0x555bb89d7030 msgr2=0x555bb8a612f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.803+0000 7fb863454b80 1 -- 172.21.15.44:0/975855832 shutdown_connections 2023-09-10T01:10:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.803+0000 7fb863454b80 1 -- 172.21.15.44:0/975855832 wait complete. 2023-09-10T01:10:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.804+0000 7fb863454b80 1 -- start start 2023-09-10T01:10:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.804+0000 7fb863454b80 1 Processor -- start 2023-09-10T01:10:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.804+0000 7fb863454b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a62300 0x555bb8a726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.804+0000 7fb863454b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a62cc0 0x555bb8a72c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.804+0000 7fb863454b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a681a0 0x555bb8ae00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.804+0000 7fb863454b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb8a73450 con 0x555bb8a62cc0 2023-09-10T01:10:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.804+0000 7fb863454b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb8ae0600 con 0x555bb8a681a0 2023-09-10T01:10:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.804+0000 7fb863454b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb8ae0740 con 0x555bb8a62300 2023-09-10T01:10:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.804+0000 7fb863454b80 10 client.0 _ll_register_callbacks cb 0x7fb848005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:10:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.804+0000 7fb85582b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a681a0 0x555bb8ae00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.804+0000 7fb85502a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a62300 0x555bb8a726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.804+0000 7fb85582b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a681a0 0x555bb8ae00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58286/0 (socket says 172.21.15.44:58286) 2023-09-10T01:10:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.804+0000 7fb85582b700 1 -- 172.21.15.44:0/3433089541 learned_addr learned my addr 172.21.15.44:0/3433089541 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.804+0000 7fb854829700 1 --2- 172.21.15.44:0/3433089541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a62cc0 0x555bb8a72c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.805+0000 7fb85502a700 1 -- 172.21.15.44:0/3433089541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a681a0 msgr2=0x555bb8ae00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.805+0000 7fb85502a700 1 --2- 172.21.15.44:0/3433089541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a681a0 0x555bb8ae00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.805+0000 7fb85502a700 1 -- 172.21.15.44:0/3433089541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a62cc0 msgr2=0x555bb8a72c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.805+0000 7fb85502a700 1 --2- 172.21.15.44:0/3433089541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a62cc0 0x555bb8a72c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.805+0000 7fb85502a700 1 -- 172.21.15.44:0/3433089541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85010dcd0 con 0x555bb8a62300 2023-09-10T01:10:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.805+0000 7fb854829700 1 --2- 172.21.15.44:0/3433089541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a62cc0 0x555bb8a72c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:10:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.805+0000 7fb85582b700 1 --2- 172.21.15.44:0/3433089541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a681a0 0x555bb8ae00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:10:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.805+0000 7fb85502a700 1 --2- 172.21.15.44:0/3433089541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a62300 0x555bb8a726e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb8480056d0 tx=0x7fb848026800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4db4f4fa0fae21d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.805+0000 7fb834ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:10:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.805+0000 7fb834ff9700 1 -- 172.21.15.44:0/3433089541 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84802c070 con 0x555bb8a62300 2023-09-10T01:10:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.805+0000 7fb863454b80 1 -- 172.21.15.44:0/3433089541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb848026bf0 con 0x555bb8a62300 2023-09-10T01:10:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.806+0000 7fb834ff9700 1 -- 172.21.15.44:0/3433089541 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb84800ed10 con 0x555bb8a62300 2023-09-10T01:10:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.806+0000 7fb834ff9700 1 -- 172.21.15.44:0/3433089541 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb848003a90 con 0x555bb8a62300 2023-09-10T01:10:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.806+0000 7fb863454b80 10 client.28672 fetch_fsmap learned FSMap version 114 2023-09-10T01:10:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.806+0000 7fb863454b80 1 -- 172.21.15.44:0/3433089541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=4+}) v3 -- 0x555bb87df250 con 0x555bb8a62300 2023-09-10T01:10:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.806+0000 7fb834ff9700 1 -- 172.21.15.44:0/3433089541 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb848026bf0 con 0x555bb8a62300 2023-09-10T01:10:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.806+0000 7fb834ff9700 1 -- 172.21.15.44:0/3433089541 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 114) v1 ==== 7347+0+0 (secure 0 0 0) 0x7fb848024050 con 0x555bb8a62300 2023-09-10T01:10:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.806+0000 7fb863454b80 10 client.28672 fetch_fsmap finished waiting for FSMap version 114 2023-09-10T01:10:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.806+0000 7fb863454b80 10 client.28672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:10:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.807+0000 7fb863454b80 20 client.28672 populate_metadata read hostname 'smithi044' 2023-09-10T01:10:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.807+0000 7fb863454b80 1 --2- 172.21.15.44:0/3433089541 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555bb8ae1a60 0x555bb8ae3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.807+0000 7fb863454b80 4 client.28672 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:10:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.807+0000 7fb863454b80 1 -- 172.21.15.44:0/3433089541 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555bb89ad8c0 con 0x555bb8ae1a60 2023-09-10T01:10:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.807+0000 7fb854829700 1 --2- 172.21.15.44:0/3433089541 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555bb8ae1a60 0x555bb8ae3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.807+0000 7fb834ff9700 10 client.28672 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:10:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.807+0000 7fb854829700 1 --2- 172.21.15.44:0/3433089541 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555bb8ae1a60 0x555bb8ae3e80 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522bcf612bc9efec server_cookie=b5c4869e1bb48b4f in_seq=0 out_seq=0 2023-09-10T01:10:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.819+0000 7fb834ff9700 1 -- 172.21.15.44:0/3433089541 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555bb89ad8c0 con 0x555bb8ae1a60 2023-09-10T01:10:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.819+0000 7fb834ff9700 10 client.28672 handle_command_reply: tid=0 2023-09-10T01:10:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.819+0000 7fb863454b80 1 client.28672 shutdown 2023-09-10T01:10:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.819+0000 7fb863454b80 1 -- 172.21.15.44:0/3433089541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a62300 msgr2=0x555bb8a726e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.819+0000 7fb863454b80 1 --2- 172.21.15.44:0/3433089541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a62300 0x555bb8a726e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb8480056d0 tx=0x7fb848026800 comp rx=0 tx=0).stop 2023-09-10T01:10:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.819+0000 7fb863454b80 1 -- 172.21.15.44:0/3433089541 shutdown_connections 2023-09-10T01:10:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.819+0000 7fb863454b80 1 --2- 172.21.15.44:0/3433089541 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555bb8ae1a60 0x555bb8ae3e80 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.819+0000 7fb863454b80 1 --2- 172.21.15.44:0/3433089541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb8a681a0 0x555bb8ae00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.819+0000 7fb834ff9700 0 client.28672 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:10:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.819+0000 7fb863454b80 1 --2- 172.21.15.44:0/3433089541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bb8a62300 0x555bb8a726e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.819+0000 7fb863454b80 1 --2- 172.21.15.44:0/3433089541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555bb8a62cc0 0x555bb8a72c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.819+0000 7fb863454b80 1 -- 172.21.15.44:0/3433089541 >> 172.21.15.44:0/3433089541 conn(0x555bb89d7030 msgr2=0x555bb8a61b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.819+0000 7fb863454b80 1 -- 172.21.15.44:0/3433089541 shutdown_connections 2023-09-10T01:10:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.819+0000 7fb863454b80 1 -- 172.21.15.44:0/3433089541 wait complete. 2023-09-10T01:10:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.819+0000 7fb863454b80 20 client.28672 trim_cache size 0 max 16384 2023-09-10T01:10:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:10:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:10:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:10:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:10:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.820+0000 7fb85728e700 1 -- 172.21.15.44:0/3689053476 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb8380a07b0 msgr2=0x7fb8380a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.820+0000 7fb85728e700 1 --2- 172.21.15.44:0/3689053476 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb8380a07b0 0x7fb8380a2c70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb848009fd0 tx=0x7fb848009260 comp rx=0 tx=0).stop 2023-09-10T01:10:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.820+0000 7fb85728e700 1 -- 172.21.15.44:0/3689053476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb850106650 msgr2=0x7fb8501002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.820+0000 7fb85728e700 1 --2- 172.21.15.44:0/3689053476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb850106650 0x7fb8501002b0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fb84400e940 tx=0x7fb84400ec50 comp rx=0 tx=0).stop 2023-09-10T01:10:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.820+0000 7fb85728e700 1 -- 172.21.15.44:0/3689053476 shutdown_connections 2023-09-10T01:10:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.820+0000 7fb85728e700 1 --2- 172.21.15.44:0/3689053476 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb8380a07b0 0x7fb8380a2c70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.820+0000 7fb85728e700 1 --2- 172.21.15.44:0/3689053476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb850106650 0x7fb8501002b0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.820+0000 7fb85728e700 1 --2- 172.21.15.44:0/3689053476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8500679f0 0x7fb8500ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.820+0000 7fb85728e700 1 --2- 172.21.15.44:0/3689053476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8500683b0 0x7fb8500ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.820+0000 7fb85728e700 1 -- 172.21.15.44:0/3689053476 >> 172.21.15.44:0/3689053476 conn(0x7fb850074f30 msgr2=0x7fb8500fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.820+0000 7fb85728e700 1 -- 172.21.15.44:0/3689053476 shutdown_connections 2023-09-10T01:10:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:22.820+0000 7fb85728e700 1 -- 172.21.15.44:0/3689053476 wait complete. 2023-09-10T01:10:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:22 smithi131 ceph-mon[103713]: pgmap v945: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:10:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:22 smithi044 ceph-mon[98676]: pgmap v945: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:10:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:22 smithi099 ceph-mon[102692]: pgmap v945: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:10:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:24 smithi131 ceph-mon[103713]: pgmap v946: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:10:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:24 smithi044 ceph-mon[98676]: pgmap v946: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:10:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:24 smithi099 ceph-mon[102692]: pgmap v946: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:10:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:26 smithi131 ceph-mon[103713]: pgmap v947: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:10:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:26 smithi044 ceph-mon[98676]: pgmap v947: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:10:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:26 smithi099 ceph-mon[102692]: pgmap v947: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:10:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:28 smithi131 ceph-mon[103713]: pgmap v948: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:10:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:28 smithi044 ceph-mon[98676]: pgmap v948: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:10:29.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:28 smithi099 ceph-mon[102692]: pgmap v948: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:10:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:30 smithi131 ceph-mon[103713]: pgmap v949: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:10:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:30 smithi044 ceph-mon[98676]: pgmap v949: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:10:31.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:30 smithi099 ceph-mon[102692]: pgmap v949: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-10T01:10:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:32 smithi131 ceph-mon[103713]: pgmap v950: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 223 op/s 2023-09-10T01:10:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:32 smithi044 ceph-mon[98676]: pgmap v950: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 223 op/s 2023-09-10T01:10:33.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:32 smithi099 ceph-mon[102692]: pgmap v950: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 223 op/s 2023-09-10T01:10:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:34 smithi131 ceph-mon[103713]: pgmap v951: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 150 op/s 2023-09-10T01:10:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:34 smithi044 ceph-mon[98676]: pgmap v951: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 150 op/s 2023-09-10T01:10:35.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:34 smithi099 ceph-mon[102692]: pgmap v951: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 150 op/s 2023-09-10T01:10:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.964+0000 7f7f145cc700 1 -- 172.21.15.44:0/731165407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f0c1044d0 msgr2=0x7f7f0c1048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.964+0000 7f7f145cc700 1 --2- 172.21.15.44:0/731165407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f0c1044d0 0x7f7f0c1048b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009990 tx=0x7f7f08009ca0 comp rx=0 tx=0).stop 2023-09-10T01:10:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.964+0000 7f7f145cc700 1 -- 172.21.15.44:0/731165407 shutdown_connections 2023-09-10T01:10:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.964+0000 7f7f145cc700 1 --2- 172.21.15.44:0/731165407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f0c0ffe70 0x7f7f0c10c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.964+0000 7f7f145cc700 1 --2- 172.21.15.44:0/731165407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f0c1044d0 0x7f7f0c1048b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.964+0000 7f7f145cc700 1 --2- 172.21.15.44:0/731165407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f0c0ff4b0 0x7f7f0c0ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.964+0000 7f7f145cc700 1 -- 172.21.15.44:0/731165407 >> 172.21.15.44:0/731165407 conn(0x7f7f0c076630 msgr2=0x7f7f0c076a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.964+0000 7f7f145cc700 1 -- 172.21.15.44:0/731165407 shutdown_connections 2023-09-10T01:10:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.964+0000 7f7f145cc700 1 -- 172.21.15.44:0/731165407 wait complete. 2023-09-10T01:10:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.965+0000 7f7f145cc700 1 Processor -- start 2023-09-10T01:10:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.965+0000 7f7f145cc700 1 -- start start 2023-09-10T01:10:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.965+0000 7f7f145cc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f0c0ff4b0 0x7f7f0c101110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.965+0000 7f7f145cc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f0c0ffe70 0x7f7f0c101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.965+0000 7f7f145cc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f0c1044d0 0x7f7f0c101b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.965+0000 7f7f145cc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c1a90f0 con 0x7f7f0c0ffe70 2023-09-10T01:10:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.965+0000 7f7f145cc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c1a9260 con 0x7f7f0c1044d0 2023-09-10T01:10:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.965+0000 7f7f145cc700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c19a820 con 0x7f7f0c0ff4b0 2023-09-10T01:10:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.965+0000 7f7f12368700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f0c0ff4b0 0x7f7f0c101110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.965+0000 7f7f11b67700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f0c0ffe70 0x7f7f0c101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.965+0000 7f7f12368700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f0c0ff4b0 0x7f7f0c101110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35208/0 (socket says 172.21.15.44:35208) 2023-09-10T01:10:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.965+0000 7f7f11b67700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f0c0ffe70 0x7f7f0c101650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56560/0 (socket says 172.21.15.44:56560) 2023-09-10T01:10:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.965+0000 7f7f12368700 1 -- 172.21.15.44:0/2911178438 learned_addr learned my addr 172.21.15.44:0/2911178438 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.965+0000 7f7f12b69700 1 --2- 172.21.15.44:0/2911178438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f0c1044d0 0x7f7f0c101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.966+0000 7f7f12368700 1 -- 172.21.15.44:0/2911178438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f0c1044d0 msgr2=0x7f7f0c101b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.966+0000 7f7f12368700 1 --2- 172.21.15.44:0/2911178438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f0c1044d0 0x7f7f0c101b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.966+0000 7f7f12368700 1 -- 172.21.15.44:0/2911178438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f0c0ffe70 msgr2=0x7f7f0c101650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.966+0000 7f7f12368700 1 --2- 172.21.15.44:0/2911178438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f0c0ffe70 0x7f7f0c101650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.966+0000 7f7f12368700 1 -- 172.21.15.44:0/2911178438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc009520 con 0x7f7f0c0ff4b0 2023-09-10T01:10:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.966+0000 7f7f11b67700 1 --2- 172.21.15.44:0/2911178438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f0c0ffe70 0x7f7f0c101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:10:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.966+0000 7f7f12368700 1 --2- 172.21.15.44:0/2911178438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f0c0ff4b0 0x7f7f0c101110 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7f08000c00 tx=0x7f7f080057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.966+0000 7f7f037fe700 1 -- 172.21.15.44:0/2911178438 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0801d070 con 0x7f7f0c0ff4b0 2023-09-10T01:10:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.966+0000 7f7f037fe700 1 -- 172.21.15.44:0/2911178438 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7f0800be70 con 0x7f7f0c0ff4b0 2023-09-10T01:10:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.966+0000 7f7f145cc700 1 -- 172.21.15.44:0/2911178438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f080095f0 con 0x7f7f0c0ff4b0 2023-09-10T01:10:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.966+0000 7f7f037fe700 1 -- 172.21.15.44:0/2911178438 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0800fdc0 con 0x7f7f0c0ff4b0 2023-09-10T01:10:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.966+0000 7f7f145cc700 1 -- 172.21.15.44:0/2911178438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0c19adc0 con 0x7f7f0c0ff4b0 2023-09-10T01:10:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.969+0000 7f7f037fe700 1 -- 172.21.15.44:0/2911178438 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f7f0800f560 con 0x7f7f0c0ff4b0 2023-09-10T01:10:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.970+0000 7f7f037fe700 1 --2- 172.21.15.44:0/2911178438 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7ef80a07b0 0x7f7ef80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.970+0000 7f7f037fe700 1 -- 172.21.15.44:0/2911178438 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f7f080cd6d0 con 0x7f7f0c0ff4b0 2023-09-10T01:10:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.970+0000 7f7f11b67700 1 --2- 172.21.15.44:0/2911178438 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7ef80a07b0 0x7f7ef80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f20792b80 1 Processor -- start 2023-09-10T01:10:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f20792b80 1 -- start start 2023-09-10T01:10:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f11b67700 1 --2- 172.21.15.44:0/2911178438 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7ef80a07b0 0x7f7ef80a2c70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009fd0 tx=0x7f7efc009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f20792b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a10558e1a0 0x55a10558e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f20792b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a105588300 0x55a105588780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f20792b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588cc0 0x55a105594f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f20792b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1054e16d0 con 0x55a105588cc0 2023-09-10T01:10:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f20792b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1054deff0 con 0x55a10558e1a0 2023-09-10T01:10:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f20792b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1054da8e0 con 0x55a105588300 2023-09-10T01:10:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f11b67700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a105588300 0x55a105588780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f12368700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a10558e1a0 0x55a10558e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f12b69700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588cc0 0x55a105594f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f11b67700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a105588300 0x55a105588780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35216/0 (socket says 172.21.15.44:35216) 2023-09-10T01:10:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f11b67700 1 -- 172.21.15.44:0/3278817666 learned_addr learned my addr 172.21.15.44:0/3278817666 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f11b67700 1 -- 172.21.15.44:0/3278817666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a10558e1a0 msgr2=0x55a10558e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f11b67700 1 --2- 172.21.15.44:0/3278817666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a10558e1a0 0x55a10558e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f11b67700 1 -- 172.21.15.44:0/3278817666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588cc0 msgr2=0x55a105594f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f11b67700 1 --2- 172.21.15.44:0/3278817666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588cc0 0x55a105594f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f11b67700 1 -- 172.21.15.44:0/3278817666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a105306d80 con 0x55a105588300 2023-09-10T01:10:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.971+0000 7f7f12368700 1 --2- 172.21.15.44:0/3278817666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a10558e1a0 0x55a10558e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:10:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.972+0000 7f7f11b67700 1 --2- 172.21.15.44:0/3278817666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a105588300 0x55a105588780 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00e6d0 tx=0x7f7efc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2067c197f60f20d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.972+0000 7f7f00ff9700 1 -- 172.21.15.44:0/3278817666 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc026630 con 0x55a105588300 2023-09-10T01:10:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.972+0000 7f7f00ff9700 1 -- 172.21.15.44:0/3278817666 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7efc021070 con 0x55a105588300 2023-09-10T01:10:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.972+0000 7f7f00ff9700 1 -- 172.21.15.44:0/3278817666 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc01d650 con 0x55a105588300 2023-09-10T01:10:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.973+0000 7f7f20792b80 1 -- 172.21.15.44:0/3278817666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a105588300 msgr2=0x55a105588780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.973+0000 7f7f20792b80 1 --2- 172.21.15.44:0/3278817666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a105588300 0x55a105588780 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00e6d0 tx=0x7f7efc014000 comp rx=0 tx=0).stop 2023-09-10T01:10:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.973+0000 7f7f20792b80 1 -- 172.21.15.44:0/3278817666 shutdown_connections 2023-09-10T01:10:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.973+0000 7f7f20792b80 1 --2- 172.21.15.44:0/3278817666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588cc0 0x55a105594f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.973+0000 7f7f20792b80 1 --2- 172.21.15.44:0/3278817666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a10558e1a0 0x55a10558e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.973+0000 7f7f20792b80 1 --2- 172.21.15.44:0/3278817666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a105588300 0x55a105588780 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.973+0000 7f7f20792b80 1 -- 172.21.15.44:0/3278817666 >> 172.21.15.44:0/3278817666 conn(0x55a1054fd030 msgr2=0x55a105583560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.973+0000 7f7f20792b80 1 -- 172.21.15.44:0/3278817666 shutdown_connections 2023-09-10T01:10:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.973+0000 7f7f20792b80 1 -- 172.21.15.44:0/3278817666 wait complete. 2023-09-10T01:10:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.973+0000 7f7f20792b80 1 -- start start 2023-09-10T01:10:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.973+0000 7f7f20792b80 1 Processor -- start 2023-09-10T01:10:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f20792b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 0x55a105594dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f20792b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a105588cc0 0x55a105592040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f20792b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a10558e1a0 0x55a105592620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f20792b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a105478b10 con 0x55a105588300 2023-09-10T01:10:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f20792b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a105478810 con 0x55a10558e1a0 2023-09-10T01:10:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f20792b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10535c8c0 con 0x55a105588cc0 2023-09-10T01:10:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f20792b80 10 client.0 _ll_register_callbacks cb 0x55a1053c2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:10:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f12368700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 0x55a105594dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f12368700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 0x55a105594dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56572/0 (socket says 172.21.15.44:56572) 2023-09-10T01:10:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f11b67700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a105588cc0 0x55a105592040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f12368700 1 -- 172.21.15.44:0/3586041889 learned_addr learned my addr 172.21.15.44:0/3586041889 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f12b69700 1 --2- 172.21.15.44:0/3586041889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a10558e1a0 0x55a105592620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f11b67700 1 -- 172.21.15.44:0/3586041889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a10558e1a0 msgr2=0x55a105592620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f11b67700 1 --2- 172.21.15.44:0/3586041889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a10558e1a0 0x55a105592620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f11b67700 1 -- 172.21.15.44:0/3586041889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 msgr2=0x55a105594dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f11b67700 1 --2- 172.21.15.44:0/3586041889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 0x55a105594dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f11b67700 1 -- 172.21.15.44:0/3586041889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0c19adc0 con 0x55a105588cc0 2023-09-10T01:10:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f12368700 1 --2- 172.21.15.44:0/3586041889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 0x55a105594dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:10:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f11b67700 1 --2- 172.21.15.44:0/3586041889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a105588cc0 0x55a105592040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00e6d0 tx=0x7f7efc01cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a968abc58833ef9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7ef5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:10:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7ef5ffb700 1 -- 172.21.15.44:0/3586041889 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc005040 con 0x55a105588cc0 2023-09-10T01:10:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7f20792b80 1 -- 172.21.15.44:0/3586041889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a105432490 con 0x55a105588cc0 2023-09-10T01:10:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.974+0000 7f7ef5ffb700 1 -- 172.21.15.44:0/3586041889 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f7efc0051a0 con 0x55a105588cc0 2023-09-10T01:10:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.975+0000 7f7ef5ffb700 1 -- 172.21.15.44:0/3586041889 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7efc005360 con 0x55a105588cc0 2023-09-10T01:10:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.975+0000 7f7f20792b80 10 client.28684 fetch_fsmap learned FSMap version 114 2023-09-10T01:10:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.975+0000 7f7f20792b80 1 -- 172.21.15.44:0/3586041889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=4+}) v3 -- 0x7f7efc017050 con 0x55a105588cc0 2023-09-10T01:10:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.975+0000 7f7ef5ffb700 1 -- 172.21.15.44:0/3586041889 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a105432490 con 0x55a105588cc0 2023-09-10T01:10:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.975+0000 7f7ef5ffb700 1 -- 172.21.15.44:0/3586041889 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 114) v1 ==== 7347+0+0 (secure 0 0 0) 0x7f7efc01c480 con 0x55a105588cc0 2023-09-10T01:10:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.975+0000 7f7f20792b80 10 client.28684 fetch_fsmap finished waiting for FSMap version 114 2023-09-10T01:10:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.975+0000 7f7f20792b80 10 client.28684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:10:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.975+0000 7f7f20792b80 20 client.28684 populate_metadata read hostname 'smithi044' 2023-09-10T01:10:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.975+0000 7f7f20792b80 1 --2- 172.21.15.44:0/3586041889 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a1055a2360 0x55a1055a4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.975+0000 7f7f20792b80 4 client.28684 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:10:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.975+0000 7f7f20792b80 1 -- 172.21.15.44:0/3586041889 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1054ef840 con 0x55a1055a2360 2023-09-10T01:10:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.975+0000 7f7f12368700 1 --2- 172.21.15.44:0/3586041889 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a1055a2360 0x55a1055a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.976+0000 7f7f12368700 1 --2- 172.21.15.44:0/3586041889 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a1055a2360 0x55a1055a4780 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d3f13ac9b4e4b32 server_cookie=e53130b1ec26e6af in_seq=0 out_seq=0 2023-09-10T01:10:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.976+0000 7f7ef5ffb700 10 client.28684 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:10:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.976+0000 7f7ef5ffb700 1 -- 172.21.15.44:0/3586041889 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a1054ef840 con 0x55a1055a2360 2023-09-10T01:10:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.976+0000 7f7ef5ffb700 10 client.28684 handle_command_reply: tid=0 2023-09-10T01:10:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.976+0000 7f7f20792b80 1 client.28684 shutdown 2023-09-10T01:10:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.976+0000 7f7f20792b80 1 -- 172.21.15.44:0/3586041889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a105588cc0 msgr2=0x55a105592040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.976+0000 7f7f20792b80 1 --2- 172.21.15.44:0/3586041889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a105588cc0 0x55a105592040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00e6d0 tx=0x7f7efc01cd30 comp rx=0 tx=0).stop 2023-09-10T01:10:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.977+0000 7f7f20792b80 1 -- 172.21.15.44:0/3586041889 shutdown_connections 2023-09-10T01:10:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.977+0000 7f7f20792b80 1 --2- 172.21.15.44:0/3586041889 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a1055a2360 0x55a1055a4780 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.977+0000 7f7f20792b80 1 --2- 172.21.15.44:0/3586041889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a10558e1a0 0x55a105592620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.977+0000 7f7f20792b80 1 --2- 172.21.15.44:0/3586041889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 0x55a105594dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.977+0000 7f7f20792b80 1 --2- 172.21.15.44:0/3586041889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a105588cc0 0x55a105592040 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.977+0000 7f7ef5ffb700 0 client.28684 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:10:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.977+0000 7f7f20792b80 1 -- 172.21.15.44:0/3586041889 >> 172.21.15.44:0/3586041889 conn(0x55a1054fd030 msgr2=0x55a1055827a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.977+0000 7f7f20792b80 1 -- 172.21.15.44:0/3586041889 shutdown_connections 2023-09-10T01:10:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.977+0000 7f7f20792b80 1 -- 172.21.15.44:0/3586041889 wait complete. 2023-09-10T01:10:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.977+0000 7f7f20792b80 20 client.28684 trim_cache size 0 max 16384 2023-09-10T01:10:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.996+0000 7f7f20792b80 1 Processor -- start 2023-09-10T01:10:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.996+0000 7f7f20792b80 1 -- start start 2023-09-10T01:10:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.996+0000 7f7f20792b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 0x55a105584930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.996+0000 7f7f20792b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a105588cc0 0x55a10558c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.996+0000 7f7f20792b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a10558e1a0 0x55a10558c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.996+0000 7f7f20792b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10557af90 con 0x55a105588300 2023-09-10T01:10:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.996+0000 7f7f20792b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a105592c30 con 0x55a105588cc0 2023-09-10T01:10:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.996+0000 7f7f20792b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10557ffc0 con 0x55a10558e1a0 2023-09-10T01:10:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.996+0000 7f7f12b69700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a10558e1a0 0x55a10558c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.997+0000 7f7f11b67700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a105588cc0 0x55a10558c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.997+0000 7f7f12b69700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a10558e1a0 0x55a10558c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35232/0 (socket says 172.21.15.44:35232) 2023-09-10T01:10:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.997+0000 7f7f12b69700 1 -- 172.21.15.44:0/1307067529 learned_addr learned my addr 172.21.15.44:0/1307067529 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.997+0000 7f7f11b67700 1 -- 172.21.15.44:0/1307067529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a10558e1a0 msgr2=0x55a10558c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.997+0000 7f7f11b67700 1 --2- 172.21.15.44:0/1307067529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a10558e1a0 0x55a10558c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.997+0000 7f7f11b67700 1 -- 172.21.15.44:0/1307067529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 msgr2=0x55a105584930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.997+0000 7f7f12368700 1 --2- 172.21.15.44:0/1307067529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 0x55a105584930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.997+0000 7f7f11b67700 1 --2- 172.21.15.44:0/1307067529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 0x55a105584930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.997+0000 7f7f11b67700 1 -- 172.21.15.44:0/1307067529 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1052efc00 con 0x55a105588cc0 2023-09-10T01:10:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.997+0000 7f7f12b69700 1 --2- 172.21.15.44:0/1307067529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a10558e1a0 0x55a10558c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:10:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.997+0000 7f7f12368700 1 --2- 172.21.15.44:0/1307067529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 0x55a105584930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:10:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.997+0000 7f7f11b67700 1 --2- 172.21.15.44:0/1307067529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a105588cc0 0x55a10558c1f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0114f0 tx=0x7f7efc01cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dab2b604aaea422 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.997+0000 7f7f017fa700 1 -- 172.21.15.44:0/1307067529 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc01cda0 con 0x55a105588cc0 2023-09-10T01:10:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.997+0000 7f7f017fa700 1 -- 172.21.15.44:0/1307067529 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7efc0092d0 con 0x55a105588cc0 2023-09-10T01:10:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.997+0000 7f7f017fa700 1 -- 172.21.15.44:0/1307067529 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc004930 con 0x55a105588cc0 2023-09-10T01:10:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.998+0000 7f7f20792b80 1 -- 172.21.15.44:0/1307067529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a105588cc0 msgr2=0x55a10558c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.998+0000 7f7f20792b80 1 --2- 172.21.15.44:0/1307067529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a105588cc0 0x55a10558c1f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0114f0 tx=0x7f7efc01cca0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.998+0000 7f7f20792b80 1 -- 172.21.15.44:0/1307067529 shutdown_connections 2023-09-10T01:10:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.998+0000 7f7f20792b80 1 --2- 172.21.15.44:0/1307067529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a10558e1a0 0x55a10558c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.998+0000 7f7f20792b80 1 --2- 172.21.15.44:0/1307067529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 0x55a105584930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.998+0000 7f7f20792b80 1 --2- 172.21.15.44:0/1307067529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a105588cc0 0x55a10558c1f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.998+0000 7f7f20792b80 1 -- 172.21.15.44:0/1307067529 >> 172.21.15.44:0/1307067529 conn(0x55a1054fd030 msgr2=0x55a10559e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.998+0000 7f7f20792b80 1 -- 172.21.15.44:0/1307067529 shutdown_connections 2023-09-10T01:10:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.998+0000 7f7f20792b80 1 -- 172.21.15.44:0/1307067529 wait complete. 2023-09-10T01:10:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f20792b80 1 -- start start 2023-09-10T01:10:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f20792b80 1 Processor -- start 2023-09-10T01:10:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f20792b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 0x55a1055986e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f20792b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a105588cc0 0x55a105598c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f20792b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a10558e1a0 0x55a1055f5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f20792b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a105599530 con 0x55a105588300 2023-09-10T01:10:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f20792b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1055f56b0 con 0x55a105588cc0 2023-09-10T01:10:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f20792b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1055f57f0 con 0x55a10558e1a0 2023-09-10T01:10:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f20792b80 10 client.0 _ll_register_callbacks cb 0x7f7f04009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:10:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f12368700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 0x55a1055986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f11b67700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a105588cc0 0x55a105598c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f12368700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 0x55a1055986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56586/0 (socket says 172.21.15.44:56586) 2023-09-10T01:10:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f11b67700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a105588cc0 0x55a105598c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58328/0 (socket says 172.21.15.44:58328) 2023-09-10T01:10:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f12368700 1 -- 172.21.15.44:0/4014721316 learned_addr learned my addr 172.21.15.44:0/4014721316 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f12b69700 1 --2- 172.21.15.44:0/4014721316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a10558e1a0 0x55a1055f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:36.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f11b67700 1 -- 172.21.15.44:0/4014721316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a10558e1a0 msgr2=0x55a1055f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:36.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f11b67700 1 --2- 172.21.15.44:0/4014721316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a10558e1a0 0x55a1055f5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:35.999+0000 7f7f11b67700 1 -- 172.21.15.44:0/4014721316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 msgr2=0x55a1055986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:36.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.000+0000 7f7f11b67700 1 --2- 172.21.15.44:0/4014721316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 0x55a1055986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.000+0000 7f7f11b67700 1 -- 172.21.15.44:0/4014721316 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f04009e00 con 0x55a105588cc0 2023-09-10T01:10:36.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.000+0000 7f7f12b69700 1 --2- 172.21.15.44:0/4014721316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a10558e1a0 0x55a1055f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:10:36.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.000+0000 7f7f11b67700 1 --2- 172.21.15.44:0/4014721316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a105588cc0 0x55a105598c20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55a10554d990 tx=0x7f7efc005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1018449d1248c16 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:36.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.000+0000 7f7ef5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:10:36.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.000+0000 7f7ef5ffb700 1 -- 172.21.15.44:0/4014721316 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc01cb30 con 0x55a105588cc0 2023-09-10T01:10:36.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.000+0000 7f7ef5ffb700 1 -- 172.21.15.44:0/4014721316 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f7efc01c3f0 con 0x55a105588cc0 2023-09-10T01:10:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.000+0000 7f7f20792b80 1 -- 172.21.15.44:0/4014721316 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7efc021070 con 0x55a105588cc0 2023-09-10T01:10:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.000+0000 7f7ef5ffb700 1 -- 172.21.15.44:0/4014721316 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7efc01c5f0 con 0x55a105588cc0 2023-09-10T01:10:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.000+0000 7f7f20792b80 10 client.28751 fetch_fsmap learned FSMap version 114 2023-09-10T01:10:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.000+0000 7f7f20792b80 1 -- 172.21.15.44:0/4014721316 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=4+}) v3 -- 0x7f7efc017050 con 0x55a105588cc0 2023-09-10T01:10:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.000+0000 7f7ef5ffb700 1 -- 172.21.15.44:0/4014721316 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7efc054020 con 0x55a105588cc0 2023-09-10T01:10:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.001+0000 7f7ef5ffb700 1 -- 172.21.15.44:0/4014721316 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 114) v1 ==== 7347+0+0 (secure 0 0 0) 0x7f7efc0384c0 con 0x55a105588cc0 2023-09-10T01:10:36.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.001+0000 7f7f20792b80 10 client.28751 fetch_fsmap finished waiting for FSMap version 114 2023-09-10T01:10:36.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.001+0000 7f7f20792b80 10 client.28751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:10:36.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.001+0000 7f7f20792b80 20 client.28751 populate_metadata read hostname 'smithi044' 2023-09-10T01:10:36.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.001+0000 7f7f20792b80 1 --2- 172.21.15.44:0/4014721316 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a1055f69e0 0x55a1055f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:36.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.001+0000 7f7f20792b80 4 client.28751 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:10:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.001+0000 7f7f20792b80 1 -- 172.21.15.44:0/4014721316 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a1054d38c0 con 0x55a1055f69e0 2023-09-10T01:10:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.001+0000 7f7f12368700 1 --2- 172.21.15.44:0/4014721316 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a1055f69e0 0x55a1055f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.001+0000 7f7ef5ffb700 10 client.28751 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:10:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.001+0000 7f7f12368700 1 --2- 172.21.15.44:0/4014721316 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a1055f69e0 0x55a1055f8e00 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d0ae89c1a30c930 server_cookie=580890f854aa723b in_seq=0 out_seq=0 2023-09-10T01:10:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.012+0000 7f7ef5ffb700 1 -- 172.21.15.44:0/4014721316 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a1054d38c0 con 0x55a1055f69e0 2023-09-10T01:10:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.012+0000 7f7ef5ffb700 10 client.28751 handle_command_reply: tid=0 2023-09-10T01:10:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.012+0000 7f7f20792b80 1 client.28751 shutdown 2023-09-10T01:10:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f20792b80 1 -- 172.21.15.44:0/4014721316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a105588cc0 msgr2=0x55a105598c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f20792b80 1 --2- 172.21.15.44:0/4014721316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a105588cc0 0x55a105598c20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55a10554d990 tx=0x7f7efc005770 comp rx=0 tx=0).stop 2023-09-10T01:10:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f20792b80 1 -- 172.21.15.44:0/4014721316 shutdown_connections 2023-09-10T01:10:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f20792b80 1 --2- 172.21.15.44:0/4014721316 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a1055f69e0 0x55a1055f8e00 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f20792b80 1 --2- 172.21.15.44:0/4014721316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a10558e1a0 0x55a1055f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7ef5ffb700 0 client.28751 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:10:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f20792b80 1 --2- 172.21.15.44:0/4014721316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a105588300 0x55a1055986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f20792b80 1 --2- 172.21.15.44:0/4014721316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a105588cc0 0x55a105598c20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f20792b80 1 -- 172.21.15.44:0/4014721316 >> 172.21.15.44:0/4014721316 conn(0x55a1054fd030 msgr2=0x55a10558c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:36.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f20792b80 1 -- 172.21.15.44:0/4014721316 shutdown_connections 2023-09-10T01:10:36.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f20792b80 1 -- 172.21.15.44:0/4014721316 wait complete. 2023-09-10T01:10:36.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f20792b80 20 client.28751 trim_cache size 0 max 16384 2023-09-10T01:10:36.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:10:36.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:10:36.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:10:36.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:10:36.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f145cc700 1 -- 172.21.15.44:0/2911178438 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7ef80a07b0 msgr2=0x7f7ef80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:36.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f145cc700 1 --2- 172.21.15.44:0/2911178438 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7ef80a07b0 0x7f7ef80a2c70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009fd0 tx=0x7f7efc009260 comp rx=0 tx=0).stop 2023-09-10T01:10:36.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f145cc700 1 -- 172.21.15.44:0/2911178438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f0c0ff4b0 msgr2=0x7f7f0c101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:36.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f145cc700 1 --2- 172.21.15.44:0/2911178438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f0c0ff4b0 0x7f7f0c101110 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7f08000c00 tx=0x7f7f080057a0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f145cc700 1 -- 172.21.15.44:0/2911178438 shutdown_connections 2023-09-10T01:10:36.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f145cc700 1 --2- 172.21.15.44:0/2911178438 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7ef80a07b0 0x7f7ef80a2c70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f145cc700 1 --2- 172.21.15.44:0/2911178438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f0c1044d0 0x7f7f0c101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f145cc700 1 --2- 172.21.15.44:0/2911178438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f0c0ff4b0 0x7f7f0c101110 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f145cc700 1 --2- 172.21.15.44:0/2911178438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f0c0ffe70 0x7f7f0c101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.013+0000 7f7f145cc700 1 -- 172.21.15.44:0/2911178438 >> 172.21.15.44:0/2911178438 conn(0x7f7f0c076630 msgr2=0x7f7f0c068a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.014+0000 7f7f145cc700 1 -- 172.21.15.44:0/2911178438 shutdown_connections 2023-09-10T01:10:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:36.014+0000 7f7f145cc700 1 -- 172.21.15.44:0/2911178438 wait complete. 2023-09-10T01:10:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:36 smithi131 ceph-mon[103713]: pgmap v952: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 223 op/s 2023-09-10T01:10:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:36 smithi044 ceph-mon[98676]: pgmap v952: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 223 op/s 2023-09-10T01:10:37.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:36 smithi099 ceph-mon[102692]: pgmap v952: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 223 op/s 2023-09-10T01:10:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:38 smithi131 ceph-mon[103713]: pgmap v953: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 965 KiB/s wr, 136 op/s 2023-09-10T01:10:39.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:38 smithi044 ceph-mon[98676]: pgmap v953: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 965 KiB/s wr, 136 op/s 2023-09-10T01:10:39.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:38 smithi099 ceph-mon[102692]: pgmap v953: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 965 KiB/s wr, 136 op/s 2023-09-10T01:10:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:40 smithi131 ceph-mon[103713]: pgmap v954: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 965 KiB/s wr, 136 op/s 2023-09-10T01:10:41.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:40 smithi044 ceph-mon[98676]: pgmap v954: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 965 KiB/s wr, 136 op/s 2023-09-10T01:10:41.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:40 smithi099 ceph-mon[102692]: pgmap v954: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 965 KiB/s wr, 136 op/s 2023-09-10T01:10:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:42 smithi131 ceph-mon[103713]: pgmap v955: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T01:10:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:42 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:10:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:42 smithi044 ceph-mon[98676]: pgmap v955: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T01:10:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:42 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:10:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:42 smithi099 ceph-mon[102692]: pgmap v955: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T01:10:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:42 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:10:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:44 smithi131 ceph-mon[103713]: pgmap v956: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-09-10T01:10:45.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:10:45.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:10:45.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:44 smithi044 ceph-mon[98676]: pgmap v956: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-09-10T01:10:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:10:45.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:10:45.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:44 smithi099 ceph-mon[102692]: pgmap v956: 129 pgs: 129 active+clean; 10 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-09-10T01:10:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:45.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:10:45.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:10:45.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:10:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:46 smithi131 ceph-mon[103713]: pgmap v957: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T01:10:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:46 smithi044 ceph-mon[98676]: pgmap v957: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T01:10:47.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:46 smithi099 ceph-mon[102692]: pgmap v957: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T01:10:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.149+0000 7f3f497b3700 1 -- 172.21.15.44:0/440585661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f441055b0 msgr2=0x7f3f44105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.149+0000 7f3f497b3700 1 --2- 172.21.15.44:0/440585661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f441055b0 0x7f3f44105990 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f3f340099c0 tx=0x7f3f34009cd0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.149+0000 7f3f497b3700 1 -- 172.21.15.44:0/440585661 shutdown_connections 2023-09-10T01:10:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.149+0000 7f3f497b3700 1 --2- 172.21.15.44:0/440585661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f44100050 0x7f3f4410c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.149+0000 7f3f497b3700 1 --2- 172.21.15.44:0/440585661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f441055b0 0x7f3f44105990 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.149+0000 7f3f497b3700 1 --2- 172.21.15.44:0/440585661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f440ff690 0x7f3f440ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.149+0000 7f3f497b3700 1 -- 172.21.15.44:0/440585661 >> 172.21.15.44:0/440585661 conn(0x7f3f44077430 msgr2=0x7f3f44077840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.149+0000 7f3f497b3700 1 -- 172.21.15.44:0/440585661 shutdown_connections 2023-09-10T01:10:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.149+0000 7f3f497b3700 1 -- 172.21.15.44:0/440585661 wait complete. 2023-09-10T01:10:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.150+0000 7f3f497b3700 1 Processor -- start 2023-09-10T01:10:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.150+0000 7f3f497b3700 1 -- start start 2023-09-10T01:10:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.150+0000 7f3f497b3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f440ff690 0x7f3f4419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.150+0000 7f3f497b3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f44100050 0x7f3f4419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.150+0000 7f3f497b3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f441055b0 0x7f3f44196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.150+0000 7f3f497b3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f441969d0 con 0x7f3f44100050 2023-09-10T01:10:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.150+0000 7f3f497b3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f44196b40 con 0x7f3f441055b0 2023-09-10T01:10:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.150+0000 7f3f497b3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f44196d00 con 0x7f3f440ff690 2023-09-10T01:10:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.150+0000 7f3f42ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f440ff690 0x7f3f4419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.150+0000 7f3f427fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f44100050 0x7f3f4419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.150+0000 7f3f42ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f440ff690 0x7f3f4419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35284/0 (socket says 172.21.15.44:35284) 2023-09-10T01:10:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.150+0000 7f3f42ffd700 1 -- 172.21.15.44:0/1671522810 learned_addr learned my addr 172.21.15.44:0/1671522810 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.150+0000 7f3f437fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f441055b0 0x7f3f44196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.151+0000 7f3f42ffd700 1 -- 172.21.15.44:0/1671522810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f441055b0 msgr2=0x7f3f44196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.151+0000 7f3f42ffd700 1 --2- 172.21.15.44:0/1671522810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f441055b0 0x7f3f44196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.151+0000 7f3f42ffd700 1 -- 172.21.15.44:0/1671522810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f44100050 msgr2=0x7f3f4419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.151+0000 7f3f42ffd700 1 --2- 172.21.15.44:0/1671522810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f44100050 0x7f3f4419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.151+0000 7f3f42ffd700 1 -- 172.21.15.44:0/1671522810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f340095f0 con 0x7f3f440ff690 2023-09-10T01:10:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.151+0000 7f3f42ffd700 1 --2- 172.21.15.44:0/1671522810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f440ff690 0x7f3f4419c500 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3f340098e0 tx=0x7f3f340052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.151+0000 7f3f2bfff700 1 -- 172.21.15.44:0/1671522810 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3401d070 con 0x7f3f440ff690 2023-09-10T01:10:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.151+0000 7f3f2bfff700 1 -- 172.21.15.44:0/1671522810 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f34005670 con 0x7f3f440ff690 2023-09-10T01:10:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.151+0000 7f3f497b3700 1 -- 172.21.15.44:0/1671522810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f44196f80 con 0x7f3f440ff690 2023-09-10T01:10:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.151+0000 7f3f2bfff700 1 -- 172.21.15.44:0/1671522810 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3400fb50 con 0x7f3f440ff690 2023-09-10T01:10:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.152+0000 7f3f497b3700 1 -- 172.21.15.44:0/1671522810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f441a34f0 con 0x7f3f440ff690 2023-09-10T01:10:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.154+0000 7f3f2bfff700 1 -- 172.21.15.44:0/1671522810 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f3f340036f0 con 0x7f3f440ff690 2023-09-10T01:10:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.155+0000 7f3f2bfff700 1 --2- 172.21.15.44:0/1671522810 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3f2c0a0830 0x7f3f2c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.155+0000 7f3f427fc700 1 --2- 172.21.15.44:0/1671522810 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3f2c0a0830 0x7f3f2c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.155+0000 7f3f2bfff700 1 -- 172.21.15.44:0/1671522810 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f3f340cd110 con 0x7f3f440ff690 2023-09-10T01:10:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.156+0000 7f3f427fc700 1 --2- 172.21.15.44:0/1671522810 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3f2c0a0830 0x7f3f2c0a2cf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3f44077cf0 tx=0x7f3f38009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.156+0000 7f3f55979b80 1 Processor -- start 2023-09-10T01:10:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.156+0000 7f3f55979b80 1 -- start start 2023-09-10T01:10:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.156+0000 7f3f55979b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 0x556a04e99580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.156+0000 7f3f55979b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04e93780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.156+0000 7f3f55979b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04e9ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.156+0000 7f3f55979b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a04dec6d0 con 0x556a04e991a0 2023-09-10T01:10:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.156+0000 7f3f55979b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a04de9ff0 con 0x556a04e93300 2023-09-10T01:10:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.156+0000 7f3f55979b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a04de58e0 con 0x556a04e93cc0 2023-09-10T01:10:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.157+0000 7f3f42ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 0x556a04e99580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.157+0000 7f3f427fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04e93780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.157+0000 7f3f427fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04e93780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58382/0 (socket says 172.21.15.44:58382) 2023-09-10T01:10:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.157+0000 7f3f427fc700 1 -- 172.21.15.44:0/436500726 learned_addr learned my addr 172.21.15.44:0/436500726 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.157+0000 7f3f437fe700 1 --2- 172.21.15.44:0/436500726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04e9ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.157+0000 7f3f427fc700 1 -- 172.21.15.44:0/436500726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 msgr2=0x556a04e9ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.157+0000 7f3f427fc700 1 --2- 172.21.15.44:0/436500726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04e9ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.157+0000 7f3f427fc700 1 -- 172.21.15.44:0/436500726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 msgr2=0x556a04e99580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.157+0000 7f3f427fc700 1 --2- 172.21.15.44:0/436500726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 0x556a04e99580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.157+0000 7f3f427fc700 1 -- 172.21.15.44:0/436500726 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a04c11d80 con 0x556a04e93300 2023-09-10T01:10:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.157+0000 7f3f42ffd700 1 --2- 172.21.15.44:0/436500726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 0x556a04e99580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:10:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.157+0000 7f3f427fc700 1 --2- 172.21.15.44:0/436500726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04e93780 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3f38011d50 tx=0x7f3f38014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d18c95d9835504ce server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.157+0000 7f3f297fa700 1 -- 172.21.15.44:0/436500726 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3800eb20 con 0x556a04e93300 2023-09-10T01:10:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.157+0000 7f3f297fa700 1 -- 172.21.15.44:0/436500726 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f38021070 con 0x556a04e93300 2023-09-10T01:10:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.157+0000 7f3f297fa700 1 -- 172.21.15.44:0/436500726 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3801d410 con 0x556a04e93300 2023-09-10T01:10:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.158+0000 7f3f55979b80 1 -- 172.21.15.44:0/436500726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 msgr2=0x556a04e93780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.158+0000 7f3f55979b80 1 --2- 172.21.15.44:0/436500726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04e93780 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3f38011d50 tx=0x7f3f38014000 comp rx=0 tx=0).stop 2023-09-10T01:10:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.158+0000 7f3f55979b80 1 -- 172.21.15.44:0/436500726 shutdown_connections 2023-09-10T01:10:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.158+0000 7f3f55979b80 1 --2- 172.21.15.44:0/436500726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04e9ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.158+0000 7f3f55979b80 1 --2- 172.21.15.44:0/436500726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 0x556a04e99580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.158+0000 7f3f55979b80 1 --2- 172.21.15.44:0/436500726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04e93780 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.158+0000 7f3f55979b80 1 -- 172.21.15.44:0/436500726 >> 172.21.15.44:0/436500726 conn(0x556a04e08030 msgr2=0x556a04e8e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.159+0000 7f3f55979b80 1 -- 172.21.15.44:0/436500726 shutdown_connections 2023-09-10T01:10:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.159+0000 7f3f55979b80 1 -- 172.21.15.44:0/436500726 wait complete. 2023-09-10T01:10:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.159+0000 7f3f55979b80 1 -- start start 2023-09-10T01:10:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.159+0000 7f3f55979b80 1 Processor -- start 2023-09-10T01:10:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.159+0000 7f3f55979b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04e9fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.159+0000 7f3f55979b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04e9d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.159+0000 7f3f55979b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 0x556a04e9d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.159+0000 7f3f55979b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a04d83b10 con 0x556a04e991a0 2023-09-10T01:10:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.159+0000 7f3f55979b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a04d83810 con 0x556a04e93300 2023-09-10T01:10:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.159+0000 7f3f55979b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a04c678c0 con 0x556a04e93cc0 2023-09-10T01:10:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.160+0000 7f3f55979b80 10 client.0 _ll_register_callbacks cb 0x556a04ccda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:10:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.160+0000 7f3f42ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04e9fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.160+0000 7f3f427fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04e9d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.160+0000 7f3f42ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04e9fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58388/0 (socket says 172.21.15.44:58388) 2023-09-10T01:10:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.160+0000 7f3f42ffd700 1 -- 172.21.15.44:0/277286111 learned_addr learned my addr 172.21.15.44:0/277286111 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.160+0000 7f3f437fe700 1 --2- 172.21.15.44:0/277286111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 0x556a04e9d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.160+0000 7f3f42ffd700 1 -- 172.21.15.44:0/277286111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 msgr2=0x556a04e9d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.160+0000 7f3f42ffd700 1 --2- 172.21.15.44:0/277286111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04e9d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.160+0000 7f3f42ffd700 1 -- 172.21.15.44:0/277286111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 msgr2=0x556a04e9d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.160+0000 7f3f42ffd700 1 --2- 172.21.15.44:0/277286111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 0x556a04e9d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.160+0000 7f3f42ffd700 1 -- 172.21.15.44:0/277286111 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f441a34f0 con 0x556a04e93300 2023-09-10T01:10:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.160+0000 7f3f42ffd700 1 --2- 172.21.15.44:0/277286111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04e9fe20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3f34005f50 tx=0x7f3f3409a9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7696b8bce81bdf85 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.160+0000 7f3f26ff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:10:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.160+0000 7f3f26ff5700 1 -- 172.21.15.44:0/277286111 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3409e080 con 0x556a04e93300 2023-09-10T01:10:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.160+0000 7f3f26ff5700 1 -- 172.21.15.44:0/277286111 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f3f340a2230 con 0x556a04e93300 2023-09-10T01:10:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.160+0000 7f3f55979b80 1 -- 172.21.15.44:0/277286111 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a04d3d490 con 0x556a04e93300 2023-09-10T01:10:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.161+0000 7f3f26ff5700 1 -- 172.21.15.44:0/277286111 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f3409a4b0 con 0x556a04e93300 2023-09-10T01:10:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.161+0000 7f3f55979b80 10 client.28763 fetch_fsmap learned FSMap version 114 2023-09-10T01:10:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.161+0000 7f3f55979b80 1 -- 172.21.15.44:0/277286111 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=4+}) v3 -- 0x7f3f38017070 con 0x556a04e93300 2023-09-10T01:10:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.161+0000 7f3f26ff5700 1 -- 172.21.15.44:0/277286111 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f340b8030 con 0x556a04e93300 2023-09-10T01:10:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.161+0000 7f3f26ff5700 1 -- 172.21.15.44:0/277286111 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 114) v1 ==== 7347+0+0 (secure 0 0 0) 0x7f3f340b4d20 con 0x556a04e93300 2023-09-10T01:10:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.161+0000 7f3f55979b80 10 client.28763 fetch_fsmap finished waiting for FSMap version 114 2023-09-10T01:10:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.161+0000 7f3f55979b80 10 client.28763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:10:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.161+0000 7f3f55979b80 20 client.28763 populate_metadata read hostname 'smithi044' 2023-09-10T01:10:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.161+0000 7f3f55979b80 1 --2- 172.21.15.44:0/277286111 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556a04ead130 0x556a04eaf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.161+0000 7f3f55979b80 4 client.28763 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:10:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.161+0000 7f3f55979b80 1 -- 172.21.15.44:0/277286111 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a04dfa840 con 0x556a04ead130 2023-09-10T01:10:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.161+0000 7f3f427fc700 1 --2- 172.21.15.44:0/277286111 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556a04ead130 0x556a04eaf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.162+0000 7f3f26ff5700 10 client.28763 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:10:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.162+0000 7f3f427fc700 1 --2- 172.21.15.44:0/277286111 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556a04ead130 0x556a04eaf550 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b48d88599db1fcc3 server_cookie=4b8ba7a24d27aeea in_seq=0 out_seq=0 2023-09-10T01:10:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.162+0000 7f3f26ff5700 1 -- 172.21.15.44:0/277286111 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556a04dfa840 con 0x556a04ead130 2023-09-10T01:10:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.162+0000 7f3f26ff5700 10 client.28763 handle_command_reply: tid=0 2023-09-10T01:10:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.162+0000 7f3f55979b80 1 client.28763 shutdown 2023-09-10T01:10:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.162+0000 7f3f55979b80 1 -- 172.21.15.44:0/277286111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 msgr2=0x556a04e9fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.162+0000 7f3f55979b80 1 --2- 172.21.15.44:0/277286111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04e9fe20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3f34005f50 tx=0x7f3f3409a9c0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.163+0000 7f3f55979b80 1 -- 172.21.15.44:0/277286111 shutdown_connections 2023-09-10T01:10:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.163+0000 7f3f55979b80 1 --2- 172.21.15.44:0/277286111 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556a04ead130 0x556a04eaf550 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.163+0000 7f3f55979b80 1 --2- 172.21.15.44:0/277286111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 0x556a04e9d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.163+0000 7f3f26ff5700 0 client.28763 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:10:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.163+0000 7f3f55979b80 1 --2- 172.21.15.44:0/277286111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04e9fe20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.163+0000 7f3f55979b80 1 --2- 172.21.15.44:0/277286111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04e9d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.163+0000 7f3f55979b80 1 -- 172.21.15.44:0/277286111 >> 172.21.15.44:0/277286111 conn(0x556a04e08030 msgr2=0x556a04e8d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.163+0000 7f3f55979b80 1 -- 172.21.15.44:0/277286111 shutdown_connections 2023-09-10T01:10:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.163+0000 7f3f55979b80 1 -- 172.21.15.44:0/277286111 wait complete. 2023-09-10T01:10:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.163+0000 7f3f55979b80 20 client.28763 trim_cache size 0 max 16384 2023-09-10T01:10:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.182+0000 7f3f55979b80 1 Processor -- start 2023-09-10T01:10:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.182+0000 7f3f55979b80 1 -- start start 2023-09-10T01:10:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.182+0000 7f3f55979b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04eaf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.182+0000 7f3f55979b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04ea4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.182+0000 7f3f55979b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 0x556a04e9f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.182+0000 7f3f55979b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a04e94580 con 0x556a04e991a0 2023-09-10T01:10:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.182+0000 7f3f55979b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a04e9e680 con 0x556a04e93300 2023-09-10T01:10:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.182+0000 7f3f55979b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a04ea9da0 con 0x556a04e93cc0 2023-09-10T01:10:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.182+0000 7f3f42ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04eaf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.182+0000 7f3f42ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04eaf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58396/0 (socket says 172.21.15.44:58396) 2023-09-10T01:10:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.183+0000 7f3f42ffd700 1 -- 172.21.15.44:0/3717620842 learned_addr learned my addr 172.21.15.44:0/3717620842 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.183+0000 7f3f427fc700 1 --2- 172.21.15.44:0/3717620842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04ea4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.183+0000 7f3f42ffd700 1 -- 172.21.15.44:0/3717620842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 msgr2=0x556a04ea4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.183+0000 7f3f42ffd700 1 --2- 172.21.15.44:0/3717620842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04ea4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.183+0000 7f3f42ffd700 1 -- 172.21.15.44:0/3717620842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 msgr2=0x556a04e9f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.183+0000 7f3f42ffd700 1 --2- 172.21.15.44:0/3717620842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 0x556a04e9f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.183+0000 7f3f42ffd700 1 -- 172.21.15.44:0/3717620842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a04bfac00 con 0x556a04e93300 2023-09-10T01:10:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.183+0000 7f3f42ffd700 1 --2- 172.21.15.44:0/3717620842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04eaf520 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009fd0 tx=0x7f3f34098800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df87bac191d0b7db server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.183+0000 7f3f29ffb700 1 -- 172.21.15.44:0/3717620842 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f340b81b0 con 0x556a04e93300 2023-09-10T01:10:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.183+0000 7f3f29ffb700 1 -- 172.21.15.44:0/3717620842 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f340b8040 con 0x556a04e93300 2023-09-10T01:10:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.183+0000 7f3f29ffb700 1 -- 172.21.15.44:0/3717620842 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3409e110 con 0x556a04e93300 2023-09-10T01:10:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.184+0000 7f3f55979b80 1 -- 172.21.15.44:0/3717620842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 msgr2=0x556a04eaf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.184+0000 7f3f55979b80 1 --2- 172.21.15.44:0/3717620842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04eaf520 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009fd0 tx=0x7f3f34098800 comp rx=0 tx=0).stop 2023-09-10T01:10:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.184+0000 7f3f55979b80 1 -- 172.21.15.44:0/3717620842 shutdown_connections 2023-09-10T01:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.184+0000 7f3f55979b80 1 --2- 172.21.15.44:0/3717620842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 0x556a04e9f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.184+0000 7f3f55979b80 1 --2- 172.21.15.44:0/3717620842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04eaf520 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.184+0000 7f3f55979b80 1 --2- 172.21.15.44:0/3717620842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04ea4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.184+0000 7f3f55979b80 1 -- 172.21.15.44:0/3717620842 >> 172.21.15.44:0/3717620842 conn(0x556a04e08030 msgr2=0x556a04e97250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.184+0000 7f3f55979b80 1 -- 172.21.15.44:0/3717620842 shutdown_connections 2023-09-10T01:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.184+0000 7f3f55979b80 1 -- 172.21.15.44:0/3717620842 wait complete. 2023-09-10T01:10:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.184+0000 7f3f55979b80 1 -- start start 2023-09-10T01:10:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f55979b80 1 Processor -- start 2023-09-10T01:10:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f55979b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04ef50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f55979b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04ef55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f55979b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 0x556a04efff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f55979b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a04f00470 con 0x556a04e991a0 2023-09-10T01:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f55979b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a04f005b0 con 0x556a04e93300 2023-09-10T01:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f55979b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a04f00770 con 0x556a04e93cc0 2023-09-10T01:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f55979b80 10 client.0 _ll_register_callbacks cb 0x7f3f34099dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f42ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04ef50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f427fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04ef55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f42ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04ef50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58402/0 (socket says 172.21.15.44:58402) 2023-09-10T01:10:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f42ffd700 1 -- 172.21.15.44:0/3879906330 learned_addr learned my addr 172.21.15.44:0/3879906330 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:10:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f427fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04ef55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35314/0 (socket says 172.21.15.44:35314) 2023-09-10T01:10:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f437fe700 1 --2- 172.21.15.44:0/3879906330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 0x556a04efff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f42ffd700 1 -- 172.21.15.44:0/3879906330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 msgr2=0x556a04ef55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f42ffd700 1 --2- 172.21.15.44:0/3879906330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04ef55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f42ffd700 1 -- 172.21.15.44:0/3879906330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 msgr2=0x556a04efff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:10:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f42ffd700 1 --2- 172.21.15.44:0/3879906330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 0x556a04efff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f42ffd700 1 -- 172.21.15.44:0/3879906330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f38017070 con 0x556a04e93300 2023-09-10T01:10:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f42ffd700 1 --2- 172.21.15.44:0/3879906330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04ef50a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3f3409a1e0 tx=0x7f3f340a22d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cb30abcca41ebda server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:10:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.185+0000 7f3f26ff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:10:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.186+0000 7f3f26ff5700 1 -- 172.21.15.44:0/3879906330 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3409e110 con 0x556a04e93300 2023-09-10T01:10:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.186+0000 7f3f26ff5700 1 -- 172.21.15.44:0/3879906330 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f3f3409a950 con 0x556a04e93300 2023-09-10T01:10:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.186+0000 7f3f55979b80 1 -- 172.21.15.44:0/3879906330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f38021070 con 0x556a04e93300 2023-09-10T01:10:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.186+0000 7f3f26ff5700 1 -- 172.21.15.44:0/3879906330 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f34099740 con 0x556a04e93300 2023-09-10T01:10:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.186+0000 7f3f55979b80 10 client.28769 fetch_fsmap learned FSMap version 114 2023-09-10T01:10:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.186+0000 7f3f55979b80 1 -- 172.21.15.44:0/3879906330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=4+}) v3 -- 0x556a04c381f0 con 0x556a04e93300 2023-09-10T01:10:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.186+0000 7f3f26ff5700 1 -- 172.21.15.44:0/3879906330 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f38021070 con 0x556a04e93300 2023-09-10T01:10:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.186+0000 7f3f26ff5700 1 -- 172.21.15.44:0/3879906330 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 114) v1 ==== 7347+0+0 (secure 0 0 0) 0x7f3f340cad20 con 0x556a04e93300 2023-09-10T01:10:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.186+0000 7f3f55979b80 10 client.28769 fetch_fsmap finished waiting for FSMap version 114 2023-09-10T01:10:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.186+0000 7f3f55979b80 10 client.28769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:10:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.186+0000 7f3f55979b80 20 client.28769 populate_metadata read hostname 'smithi044' 2023-09-10T01:10:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.186+0000 7f3f55979b80 1 --2- 172.21.15.44:0/3879906330 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556a04f01ad0 0x556a04f03ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:10:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.186+0000 7f3f55979b80 4 client.28769 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:10:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.186+0000 7f3f55979b80 1 -- 172.21.15.44:0/3879906330 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556a04dde8c0 con 0x556a04f01ad0 2023-09-10T01:10:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.186+0000 7f3f427fc700 1 --2- 172.21.15.44:0/3879906330 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556a04f01ad0 0x556a04f03ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:10:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.187+0000 7f3f26ff5700 10 client.28769 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:10:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.187+0000 7f3f427fc700 1 --2- 172.21.15.44:0/3879906330 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556a04f01ad0 0x556a04f03ef0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fdc6a4bc01d695e server_cookie=ed546f654f6ce96b in_seq=0 out_seq=0 2023-09-10T01:10:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.200+0000 7f3f26ff5700 1 -- 172.21.15.44:0/3879906330 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556a04dde8c0 con 0x556a04f01ad0 2023-09-10T01:10:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.200+0000 7f3f26ff5700 10 client.28769 handle_command_reply: tid=0 2023-09-10T01:10:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.200+0000 7f3f55979b80 1 client.28769 shutdown 2023-09-10T01:10:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.200+0000 7f3f55979b80 1 -- 172.21.15.44:0/3879906330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 msgr2=0x556a04ef50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.200+0000 7f3f55979b80 1 --2- 172.21.15.44:0/3879906330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04ef50a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3f3409a1e0 tx=0x7f3f340a22d0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.200+0000 7f3f55979b80 1 -- 172.21.15.44:0/3879906330 shutdown_connections 2023-09-10T01:10:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.200+0000 7f3f55979b80 1 --2- 172.21.15.44:0/3879906330 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x556a04f01ad0 0x556a04f03ef0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.200+0000 7f3f55979b80 1 --2- 172.21.15.44:0/3879906330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a04e991a0 0x556a04efff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.200+0000 7f3f26ff5700 0 client.28769 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:10:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.200+0000 7f3f55979b80 1 --2- 172.21.15.44:0/3879906330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a04e93300 0x556a04ef50a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.200+0000 7f3f55979b80 1 --2- 172.21.15.44:0/3879906330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a04e93cc0 0x556a04ef55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.200+0000 7f3f55979b80 1 -- 172.21.15.44:0/3879906330 >> 172.21.15.44:0/3879906330 conn(0x556a04e08030 msgr2=0x556a04e8e090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.200+0000 7f3f55979b80 1 -- 172.21.15.44:0/3879906330 shutdown_connections 2023-09-10T01:10:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.200+0000 7f3f55979b80 1 -- 172.21.15.44:0/3879906330 wait complete. 2023-09-10T01:10:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.201+0000 7f3f55979b80 20 client.28769 trim_cache size 0 max 16384 2023-09-10T01:10:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:10:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:10:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:10:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:10:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.201+0000 7f3f497b3700 1 -- 172.21.15.44:0/1671522810 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3f2c0a0830 msgr2=0x7f3f2c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.201+0000 7f3f497b3700 1 --2- 172.21.15.44:0/1671522810 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3f2c0a0830 0x7f3f2c0a2cf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3f44077cf0 tx=0x7f3f38009260 comp rx=0 tx=0).stop 2023-09-10T01:10:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.201+0000 7f3f497b3700 1 -- 172.21.15.44:0/1671522810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f440ff690 msgr2=0x7f3f4419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:10:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.201+0000 7f3f497b3700 1 --2- 172.21.15.44:0/1671522810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f440ff690 0x7f3f4419c500 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3f340098e0 tx=0x7f3f340052f0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.201+0000 7f3f497b3700 1 -- 172.21.15.44:0/1671522810 shutdown_connections 2023-09-10T01:10:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.201+0000 7f3f497b3700 1 --2- 172.21.15.44:0/1671522810 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3f2c0a0830 0x7f3f2c0a2cf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.201+0000 7f3f497b3700 1 --2- 172.21.15.44:0/1671522810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f441055b0 0x7f3f44196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.201+0000 7f3f497b3700 1 --2- 172.21.15.44:0/1671522810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f440ff690 0x7f3f4419c500 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.201+0000 7f3f497b3700 1 --2- 172.21.15.44:0/1671522810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f44100050 0x7f3f4419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:10:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.201+0000 7f3f497b3700 1 -- 172.21.15.44:0/1671522810 >> 172.21.15.44:0/1671522810 conn(0x7f3f44077430 msgr2=0x7f3f44103450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:10:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.202+0000 7f3f497b3700 1 -- 172.21.15.44:0/1671522810 shutdown_connections 2023-09-10T01:10:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:10:49.202+0000 7f3f497b3700 1 -- 172.21.15.44:0/1671522810 wait complete. 2023-09-10T01:10:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:48 smithi131 ceph-mon[103713]: pgmap v958: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:10:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:48 smithi044 ceph-mon[98676]: pgmap v958: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:10:49.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:48 smithi099 ceph-mon[102692]: pgmap v958: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:10:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:50 smithi131 ceph-mon[103713]: pgmap v959: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:10:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:50 smithi044 ceph-mon[98676]: pgmap v959: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:10:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:50 smithi099 ceph-mon[102692]: pgmap v959: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:10:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:52 smithi131 ceph-mon[103713]: pgmap v960: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 242 op/s 2023-09-10T01:10:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:52 smithi044 ceph-mon[98676]: pgmap v960: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 242 op/s 2023-09-10T01:10:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:52 smithi099 ceph-mon[102692]: pgmap v960: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 242 op/s 2023-09-10T01:10:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:54 smithi131 ceph-mon[103713]: pgmap v961: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:10:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:54 smithi044 ceph-mon[98676]: pgmap v961: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:10:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:54 smithi099 ceph-mon[102692]: pgmap v961: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:10:56.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:55 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:10:56.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:55 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:10:56.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:55 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:10:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:56 smithi131 ceph-mon[103713]: pgmap v962: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T01:10:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:56 smithi044 ceph-mon[98676]: pgmap v962: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T01:10:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:56 smithi099 ceph-mon[102692]: pgmap v962: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-09-10T01:10:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:10:58 smithi131 ceph-mon[103713]: pgmap v963: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-09-10T01:10:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:10:58 smithi044 ceph-mon[98676]: pgmap v963: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-09-10T01:10:59.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:10:58 smithi099 ceph-mon[102692]: pgmap v963: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-09-10T01:11:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:00 smithi131 ceph-mon[103713]: pgmap v964: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-09-10T01:11:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:00 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:11:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:00 smithi044 ceph-mon[98676]: pgmap v964: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-09-10T01:11:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:00 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:11:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:00 smithi099 ceph-mon[102692]: pgmap v964: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-09-10T01:11:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:00 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:11:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.325+0000 7f76390d7700 1 -- 172.21.15.44:0/2514704328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7634101470 msgr2=0x7f76340679f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.325+0000 7f76390d7700 1 --2- 172.21.15.44:0/2514704328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7634101470 0x7f76340679f0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f7624009990 tx=0x7f7624009ca0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.325+0000 7f76390d7700 1 -- 172.21.15.44:0/2514704328 shutdown_connections 2023-09-10T01:11:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.325+0000 7f76390d7700 1 --2- 172.21.15.44:0/2514704328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f763410cdd0 0x7f763410f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.325+0000 7f76390d7700 1 --2- 172.21.15.44:0/2514704328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7634101470 0x7f76340679f0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.325+0000 7f76390d7700 1 --2- 172.21.15.44:0/2514704328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7634067f30 0x7f76340683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.325+0000 7f76390d7700 1 -- 172.21.15.44:0/2514704328 >> 172.21.15.44:0/2514704328 conn(0x7f7634074f30 msgr2=0x7f7634075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.325+0000 7f76390d7700 1 -- 172.21.15.44:0/2514704328 shutdown_connections 2023-09-10T01:11:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.325+0000 7f76390d7700 1 -- 172.21.15.44:0/2514704328 wait complete. 2023-09-10T01:11:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f76390d7700 1 Processor -- start 2023-09-10T01:11:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f76390d7700 1 -- start start 2023-09-10T01:11:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f76390d7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7634067f30 0x7f7634198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f76390d7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7634101470 0x7f7634198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f76390d7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f763410cdd0 0x7f763419d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f76390d7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763419d930 con 0x7f7634067f30 2023-09-10T01:11:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f76390d7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763419daa0 con 0x7f7634101470 2023-09-10T01:11:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f76390d7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763419dc60 con 0x7f763410cdd0 2023-09-10T01:11:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f7632d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7634067f30 0x7f7634198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f7632d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7634067f30 0x7f7634198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56676/0 (socket says 172.21.15.44:56676) 2023-09-10T01:11:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f763359e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f763410cdd0 0x7f763419d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f7632d9d700 1 -- 172.21.15.44:0/3065814871 learned_addr learned my addr 172.21.15.44:0/3065814871 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f763259c700 1 --2- 172.21.15.44:0/3065814871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7634101470 0x7f7634198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f7632d9d700 1 -- 172.21.15.44:0/3065814871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f763410cdd0 msgr2=0x7f763419d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f7632d9d700 1 --2- 172.21.15.44:0/3065814871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f763410cdd0 0x7f763419d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f7632d9d700 1 -- 172.21.15.44:0/3065814871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7634101470 msgr2=0x7f7634198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f7632d9d700 1 --2- 172.21.15.44:0/3065814871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7634101470 0x7f7634198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.326+0000 7f7632d9d700 1 -- 172.21.15.44:0/3065814871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76240095f0 con 0x7f7634067f30 2023-09-10T01:11:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.327+0000 7f7632d9d700 1 --2- 172.21.15.44:0/3065814871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7634067f30 0x7f7634198780 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f7624005910 tx=0x7f76240047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.327+0000 7f761bfff700 1 -- 172.21.15.44:0/3065814871 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762401d070 con 0x7f7634067f30 2023-09-10T01:11:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.327+0000 7f76390d7700 1 -- 172.21.15.44:0/3065814871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763419dee0 con 0x7f7634067f30 2023-09-10T01:11:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.327+0000 7f761bfff700 1 -- 172.21.15.44:0/3065814871 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f762400bdb0 con 0x7f7634067f30 2023-09-10T01:11:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.327+0000 7f761bfff700 1 -- 172.21.15.44:0/3065814871 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762400fdf0 con 0x7f7634067f30 2023-09-10T01:11:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.327+0000 7f76390d7700 1 -- 172.21.15.44:0/3065814871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763419e1b0 con 0x7f7634067f30 2023-09-10T01:11:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.329+0000 7f761bfff700 1 -- 172.21.15.44:0/3065814871 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f762400b920 con 0x7f7634067f30 2023-09-10T01:11:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.330+0000 7f761bfff700 1 --2- 172.21.15.44:0/3065814871 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f761c0a07b0 0x7f761c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.330+0000 7f761bfff700 1 -- 172.21.15.44:0/3065814871 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f76240cd470 con 0x7f7634067f30 2023-09-10T01:11:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.330+0000 7f763259c700 1 --2- 172.21.15.44:0/3065814871 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f761c0a07b0 0x7f761c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.330+0000 7f763259c700 1 --2- 172.21.15.44:0/3065814871 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f761c0a07b0 0x7f761c0a2c70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7634199fa0 tx=0x7f7628009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.330+0000 7f764529db80 1 Processor -- start 2023-09-10T01:11:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f764529db80 1 -- start start 2023-09-10T01:11:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f764529db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291ac1a0 0x5573291ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f764529db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291a6300 0x5573291a6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f764529db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6cc0 0x5573291b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f764529db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573290ff6d0 con 0x5573291a6300 2023-09-10T01:11:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f764529db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573290fcff0 con 0x5573291a6cc0 2023-09-10T01:11:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f764529db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573290f88e0 con 0x5573291ac1a0 2023-09-10T01:11:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f763259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291a6300 0x5573291a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f7632d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291ac1a0 0x5573291ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f763259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291a6300 0x5573291a6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56686/0 (socket says 172.21.15.44:56686) 2023-09-10T01:11:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f763259c700 1 -- 172.21.15.44:0/2001308330 learned_addr learned my addr 172.21.15.44:0/2001308330 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f763359e700 1 --2- 172.21.15.44:0/2001308330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6cc0 0x5573291b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f7632d9d700 1 -- 172.21.15.44:0/2001308330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6cc0 msgr2=0x5573291b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f7632d9d700 1 --2- 172.21.15.44:0/2001308330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6cc0 0x5573291b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f7632d9d700 1 -- 172.21.15.44:0/2001308330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291a6300 msgr2=0x5573291a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f7632d9d700 1 --2- 172.21.15.44:0/2001308330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291a6300 0x5573291a6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f7632d9d700 1 -- 172.21.15.44:0/2001308330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557328f24d80 con 0x5573291ac1a0 2023-09-10T01:11:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f763259c700 1 --2- 172.21.15.44:0/2001308330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291a6300 0x5573291a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:11:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.331+0000 7f7632d9d700 1 --2- 172.21.15.44:0/2001308330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291ac1a0 0x5573291ac580 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f76240ca7c0 tx=0x7f7624096970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e5fcb1d0ca1924f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.332+0000 7f76197fa700 1 -- 172.21.15.44:0/2001308330 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624099f10 con 0x5573291ac1a0 2023-09-10T01:11:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.332+0000 7f76197fa700 1 -- 172.21.15.44:0/2001308330 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f762400bbd0 con 0x5573291ac1a0 2023-09-10T01:11:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.332+0000 7f76197fa700 1 -- 172.21.15.44:0/2001308330 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762400f560 con 0x5573291ac1a0 2023-09-10T01:11:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.332+0000 7f764529db80 1 -- 172.21.15.44:0/2001308330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291ac1a0 msgr2=0x5573291ac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.332+0000 7f764529db80 1 --2- 172.21.15.44:0/2001308330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291ac1a0 0x5573291ac580 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f76240ca7c0 tx=0x7f7624096970 comp rx=0 tx=0).stop 2023-09-10T01:11:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.333+0000 7f764529db80 1 -- 172.21.15.44:0/2001308330 shutdown_connections 2023-09-10T01:11:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.333+0000 7f764529db80 1 --2- 172.21.15.44:0/2001308330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6cc0 0x5573291b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.333+0000 7f764529db80 1 --2- 172.21.15.44:0/2001308330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291ac1a0 0x5573291ac580 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.333+0000 7f764529db80 1 --2- 172.21.15.44:0/2001308330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291a6300 0x5573291a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.333+0000 7f764529db80 1 -- 172.21.15.44:0/2001308330 >> 172.21.15.44:0/2001308330 conn(0x55732911b030 msgr2=0x5573291a1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.333+0000 7f764529db80 1 -- 172.21.15.44:0/2001308330 shutdown_connections 2023-09-10T01:11:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.333+0000 7f764529db80 1 -- 172.21.15.44:0/2001308330 wait complete. 2023-09-10T01:11:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.333+0000 7f764529db80 1 -- start start 2023-09-10T01:11:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.333+0000 7f764529db80 1 Processor -- start 2023-09-10T01:11:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.333+0000 7f764529db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6300 0x5573291b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.333+0000 7f764529db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6cc0 0x5573291b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.333+0000 7f764529db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291afde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.333+0000 7f764529db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557329096b10 con 0x5573291ac1a0 2023-09-10T01:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.333+0000 7f764529db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557329096810 con 0x5573291a6300 2023-09-10T01:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.333+0000 7f764529db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557328f7a8c0 con 0x5573291a6cc0 2023-09-10T01:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f764529db80 10 client.0 _ll_register_callbacks cb 0x557328fe0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f763359e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f763259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6cc0 0x5573291b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f7632d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6300 0x5573291b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f763359e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291afde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56694/0 (socket says 172.21.15.44:56694) 2023-09-10T01:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f7632d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6300 0x5573291b5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58430/0 (socket says 172.21.15.44:58430) 2023-09-10T01:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f763259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6cc0 0x5573291b60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35342/0 (socket says 172.21.15.44:35342) 2023-09-10T01:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f763359e700 1 -- 172.21.15.44:0/2120550949 learned_addr learned my addr 172.21.15.44:0/2120550949 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f763359e700 1 -- 172.21.15.44:0/2120550949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6cc0 msgr2=0x5573291b60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f763359e700 1 --2- 172.21.15.44:0/2120550949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6cc0 0x5573291b60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f763359e700 1 -- 172.21.15.44:0/2120550949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6300 msgr2=0x5573291b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f763359e700 1 --2- 172.21.15.44:0/2120550949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6300 0x5573291b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f763359e700 1 -- 172.21.15.44:0/2120550949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7628017070 con 0x5573291ac1a0 2023-09-10T01:11:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f763359e700 1 --2- 172.21.15.44:0/2120550949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291afde0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f762c011f60 tx=0x7f762c014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34791e32a5774b38 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f7616ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:11:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f7616ff5700 1 -- 172.21.15.44:0/2120550949 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c00e040 con 0x5573291ac1a0 2023-09-10T01:11:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f764529db80 1 -- 172.21.15.44:0/2120550949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557329050490 con 0x5573291ac1a0 2023-09-10T01:11:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f7616ff5700 1 -- 172.21.15.44:0/2120550949 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f762c014b20 con 0x5573291ac1a0 2023-09-10T01:11:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f7616ff5700 1 -- 172.21.15.44:0/2120550949 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) v2 ==== 24+0+0 (secure 0 0 0) 0x7f762c014ce0 con 0x5573291ac1a0 2023-09-10T01:11:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f764529db80 10 client.19107 fetch_fsmap learned FSMap version 116 2023-09-10T01:11:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.334+0000 7f764529db80 1 -- 172.21.15.44:0/2120550949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=4+}) v3 -- 0x7f763419e1b0 con 0x5573291ac1a0 2023-09-10T01:11:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.335+0000 7f7616ff5700 1 -- 172.21.15.44:0/2120550949 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557329050490 con 0x5573291ac1a0 2023-09-10T01:11:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.335+0000 7f7616ff5700 1 -- 172.21.15.44:0/2120550949 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 116) v1 ==== 7335+0+0 (secure 0 0 0) 0x7f762c01c020 con 0x5573291ac1a0 2023-09-10T01:11:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.335+0000 7f764529db80 10 client.19107 fetch_fsmap finished waiting for FSMap version 116 2023-09-10T01:11:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.335+0000 7f764529db80 10 client.19107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:11:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.335+0000 7f764529db80 20 client.19107 populate_metadata read hostname 'smithi044' 2023-09-10T01:11:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.335+0000 7f764529db80 1 --2- 172.21.15.44:0/2120550949 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5573291c01e0 0x5573291c2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.335+0000 7f764529db80 4 client.19107 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:11:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.335+0000 7f764529db80 1 -- 172.21.15.44:0/2120550949 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55732910d840 con 0x5573291c01e0 2023-09-10T01:11:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.335+0000 7f7632d9d700 1 --2- 172.21.15.44:0/2120550949 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5573291c01e0 0x5573291c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.336+0000 7f7632d9d700 1 --2- 172.21.15.44:0/2120550949 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5573291c01e0 0x5573291c2600 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=342df45b99542dc5 server_cookie=203f3876ac2b9a21 in_seq=0 out_seq=0 2023-09-10T01:11:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.336+0000 7f7616ff5700 10 client.19107 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.336+0000 7f7616ff5700 1 -- 172.21.15.44:0/2120550949 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55732910d840 con 0x5573291c01e0 2023-09-10T01:11:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.336+0000 7f7616ff5700 10 client.19107 handle_command_reply: tid=0 2023-09-10T01:11:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.336+0000 7f764529db80 1 client.19107 shutdown 2023-09-10T01:11:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.336+0000 7f764529db80 1 -- 172.21.15.44:0/2120550949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 msgr2=0x5573291afde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.336+0000 7f764529db80 1 --2- 172.21.15.44:0/2120550949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291afde0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f762c011f60 tx=0x7f762c014600 comp rx=0 tx=0).stop 2023-09-10T01:11:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.336+0000 7f764529db80 1 -- 172.21.15.44:0/2120550949 shutdown_connections 2023-09-10T01:11:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.336+0000 7f764529db80 1 --2- 172.21.15.44:0/2120550949 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5573291c01e0 0x5573291c2600 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.336+0000 7f764529db80 1 --2- 172.21.15.44:0/2120550949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291afde0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.336+0000 7f7616ff5700 0 client.19107 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.336+0000 7f764529db80 1 --2- 172.21.15.44:0/2120550949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6300 0x5573291b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.336+0000 7f764529db80 1 --2- 172.21.15.44:0/2120550949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6cc0 0x5573291b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.336+0000 7f764529db80 1 -- 172.21.15.44:0/2120550949 >> 172.21.15.44:0/2120550949 conn(0x55732911b030 msgr2=0x5573291a07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.336+0000 7f764529db80 1 -- 172.21.15.44:0/2120550949 shutdown_connections 2023-09-10T01:11:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.336+0000 7f764529db80 1 -- 172.21.15.44:0/2120550949 wait complete. 2023-09-10T01:11:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.337+0000 7f764529db80 20 client.19107 trim_cache size 0 max 16384 2023-09-10T01:11:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.353+0000 7f764529db80 1 Processor -- start 2023-09-10T01:11:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.353+0000 7f764529db80 1 -- start start 2023-09-10T01:11:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.353+0000 7f764529db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6300 0x5573291b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.353+0000 7f764529db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6cc0 0x5573291af350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.353+0000 7f764529db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291b5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.353+0000 7f764529db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573291a75c0 con 0x5573291ac1a0 2023-09-10T01:11:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.353+0000 7f764529db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573291b13c0 con 0x5573291a6cc0 2023-09-10T01:11:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.353+0000 7f764529db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573291b0ce0 con 0x5573291a6300 2023-09-10T01:11:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.353+0000 7f763359e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.353+0000 7f763359e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291b5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56702/0 (socket says 172.21.15.44:56702) 2023-09-10T01:11:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.353+0000 7f763259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6cc0 0x5573291af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.353+0000 7f763359e700 1 -- 172.21.15.44:0/3733335758 learned_addr learned my addr 172.21.15.44:0/3733335758 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.353+0000 7f7632d9d700 1 --2- 172.21.15.44:0/3733335758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6300 0x5573291b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.354+0000 7f763359e700 1 -- 172.21.15.44:0/3733335758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6300 msgr2=0x5573291b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.354+0000 7f763359e700 1 --2- 172.21.15.44:0/3733335758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6300 0x5573291b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.354+0000 7f763359e700 1 -- 172.21.15.44:0/3733335758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6cc0 msgr2=0x5573291af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.354+0000 7f763359e700 1 --2- 172.21.15.44:0/3733335758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6cc0 0x5573291af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.354+0000 7f763359e700 1 -- 172.21.15.44:0/3733335758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557328f0dc00 con 0x5573291ac1a0 2023-09-10T01:11:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.354+0000 7f763259c700 1 --2- 172.21.15.44:0/3733335758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6cc0 0x5573291af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:11:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.354+0000 7f763359e700 1 --2- 172.21.15.44:0/3733335758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291b5980 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f762c0061e0 tx=0x7f762c006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509c13472affd7f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.354+0000 7f7619ffb700 1 -- 172.21.15.44:0/3733335758 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c00e040 con 0x5573291ac1a0 2023-09-10T01:11:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.354+0000 7f7619ffb700 1 -- 172.21.15.44:0/3733335758 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f762c00f040 con 0x5573291ac1a0 2023-09-10T01:11:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.354+0000 7f7619ffb700 1 -- 172.21.15.44:0/3733335758 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c007830 con 0x5573291ac1a0 2023-09-10T01:11:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.355+0000 7f764529db80 1 -- 172.21.15.44:0/3733335758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 msgr2=0x5573291b5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.355+0000 7f764529db80 1 --2- 172.21.15.44:0/3733335758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291b5980 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f762c0061e0 tx=0x7f762c006000 comp rx=0 tx=0).stop 2023-09-10T01:11:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.355+0000 7f764529db80 1 -- 172.21.15.44:0/3733335758 shutdown_connections 2023-09-10T01:11:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.355+0000 7f764529db80 1 --2- 172.21.15.44:0/3733335758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291b5980 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.355+0000 7f764529db80 1 --2- 172.21.15.44:0/3733335758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6300 0x5573291b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.355+0000 7f764529db80 1 --2- 172.21.15.44:0/3733335758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6cc0 0x5573291af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.355+0000 7f764529db80 1 -- 172.21.15.44:0/3733335758 >> 172.21.15.44:0/3733335758 conn(0x55732911b030 msgr2=0x5573291c6f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.355+0000 7f764529db80 1 -- 172.21.15.44:0/3733335758 shutdown_connections 2023-09-10T01:11:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.355+0000 7f764529db80 1 -- 172.21.15.44:0/3733335758 wait complete. 2023-09-10T01:11:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.355+0000 7f764529db80 1 -- start start 2023-09-10T01:11:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f764529db80 1 Processor -- start 2023-09-10T01:11:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f764529db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6300 0x5573291ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f764529db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6cc0 0x5573291ba970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f764529db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291bb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f764529db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573291bbf40 con 0x5573291ac1a0 2023-09-10T01:11:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f764529db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573291bc0b0 con 0x5573291a6cc0 2023-09-10T01:11:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f764529db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573291bc270 con 0x5573291a6300 2023-09-10T01:11:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f764529db80 10 client.0 _ll_register_callbacks cb 0x7f762c0118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:11:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f7632d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6300 0x5573291ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f763359e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291bb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f7632d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6300 0x5573291ba430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35354/0 (socket says 172.21.15.44:35354) 2023-09-10T01:11:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f7632d9d700 1 -- 172.21.15.44:0/4191348557 learned_addr learned my addr 172.21.15.44:0/4191348557 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f763259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6cc0 0x5573291ba970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f7632d9d700 1 -- 172.21.15.44:0/4191348557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6cc0 msgr2=0x5573291ba970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f7632d9d700 1 --2- 172.21.15.44:0/4191348557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6cc0 0x5573291ba970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f7632d9d700 1 -- 172.21.15.44:0/4191348557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 msgr2=0x5573291bb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f7632d9d700 1 --2- 172.21.15.44:0/4191348557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291bb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f7632d9d700 1 -- 172.21.15.44:0/4191348557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7628011c90 con 0x5573291a6300 2023-09-10T01:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.356+0000 7f763359e700 1 --2- 172.21.15.44:0/4191348557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291bb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.357+0000 7f7632d9d700 1 --2- 172.21.15.44:0/4191348557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6300 0x5573291ba430 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f762409a2b0 tx=0x7f762400bd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c223725fc1c945bb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.357+0000 7f7616ff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.357+0000 7f7616ff5700 1 -- 172.21.15.44:0/4191348557 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762409e460 con 0x5573291a6300 2023-09-10T01:11:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.357+0000 7f764529db80 1 -- 172.21.15.44:0/4191348557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f762400bbd0 con 0x5573291a6300 2023-09-10T01:11:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.357+0000 7f7616ff5700 1 -- 172.21.15.44:0/4191348557 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f76240ade40 con 0x5573291a6300 2023-09-10T01:11:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.357+0000 7f7616ff5700 1 -- 172.21.15.44:0/4191348557 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7624099f10 con 0x5573291a6300 2023-09-10T01:11:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.357+0000 7f764529db80 10 client.28726 fetch_fsmap learned FSMap version 116 2023-09-10T01:11:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.357+0000 7f764529db80 1 -- 172.21.15.44:0/4191348557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=4+}) v3 -- 0x7f763419e1b0 con 0x5573291a6300 2023-09-10T01:11:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.357+0000 7f7616ff5700 1 -- 172.21.15.44:0/4191348557 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f762407f080 con 0x5573291a6300 2023-09-10T01:11:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.357+0000 7f7616ff5700 1 -- 172.21.15.44:0/4191348557 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 116) v1 ==== 7335+0+0 (secure 0 0 0) 0x7f762409a810 con 0x5573291a6300 2023-09-10T01:11:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.357+0000 7f764529db80 10 client.28726 fetch_fsmap finished waiting for FSMap version 116 2023-09-10T01:11:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.357+0000 7f764529db80 10 client.28726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:11:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.357+0000 7f764529db80 20 client.28726 populate_metadata read hostname 'smithi044' 2023-09-10T01:11:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.357+0000 7f764529db80 1 --2- 172.21.15.44:0/4191348557 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557329214f20 0x557329217340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.357+0000 7f764529db80 4 client.28726 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:11:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.357+0000 7f764529db80 1 -- 172.21.15.44:0/4191348557 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5573290f18c0 con 0x557329214f20 2023-09-10T01:11:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.358+0000 7f763259c700 1 --2- 172.21.15.44:0/4191348557 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557329214f20 0x557329217340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.358+0000 7f7616ff5700 10 client.28726 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:02.358+0000 7f763259c700 1 --2- 172.21.15.44:0/4191348557 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557329214f20 0x557329217340 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f92344af105b3946 server_cookie=9551578ec953e10b in_seq=0 out_seq=0 2023-09-10T01:11:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:02 smithi131 ceph-mon[103713]: pgmap v965: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 213 op/s 2023-09-10T01:11:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:02 smithi044 ceph-mon[98676]: pgmap v965: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 213 op/s 2023-09-10T01:11:03.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:02 smithi099 ceph-mon[102692]: pgmap v965: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 213 op/s 2023-09-10T01:11:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.428+0000 7f7616ff5700 1 -- 172.21.15.44:0/4191348557 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5573290f18c0 con 0x557329214f20 2023-09-10T01:11:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.428+0000 7f7616ff5700 10 client.28726 handle_command_reply: tid=0 2023-09-10T01:11:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.428+0000 7f764529db80 1 client.28726 shutdown 2023-09-10T01:11:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.428+0000 7f764529db80 1 -- 172.21.15.44:0/4191348557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6300 msgr2=0x5573291ba430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.428+0000 7f764529db80 1 --2- 172.21.15.44:0/4191348557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6300 0x5573291ba430 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f762409a2b0 tx=0x7f762400bd40 comp rx=0 tx=0).stop 2023-09-10T01:11:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.428+0000 7f764529db80 1 -- 172.21.15.44:0/4191348557 shutdown_connections 2023-09-10T01:11:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.428+0000 7f764529db80 1 --2- 172.21.15.44:0/4191348557 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557329214f20 0x557329217340 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.428+0000 7f764529db80 1 --2- 172.21.15.44:0/4191348557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573291ac1a0 0x5573291bb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.428+0000 7f764529db80 1 --2- 172.21.15.44:0/4191348557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573291a6300 0x5573291ba430 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.428+0000 7f764529db80 1 --2- 172.21.15.44:0/4191348557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573291a6cc0 0x5573291ba970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.428+0000 7f7616ff5700 0 client.28726 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.428+0000 7f764529db80 1 -- 172.21.15.44:0/4191348557 >> 172.21.15.44:0/4191348557 conn(0x55732911b030 msgr2=0x5573291b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.428+0000 7f764529db80 1 -- 172.21.15.44:0/4191348557 shutdown_connections 2023-09-10T01:11:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.428+0000 7f764529db80 1 -- 172.21.15.44:0/4191348557 wait complete. 2023-09-10T01:11:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.428+0000 7f764529db80 20 client.28726 trim_cache size 0 max 16384 2023-09-10T01:11:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:11:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:11:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:11:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:11:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.429+0000 7f76390d7700 1 -- 172.21.15.44:0/3065814871 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f761c0a07b0 msgr2=0x7f761c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.429+0000 7f76390d7700 1 --2- 172.21.15.44:0/3065814871 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f761c0a07b0 0x7f761c0a2c70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7634199fa0 tx=0x7f7628009260 comp rx=0 tx=0).stop 2023-09-10T01:11:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.429+0000 7f76390d7700 1 -- 172.21.15.44:0/3065814871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7634067f30 msgr2=0x7f7634198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.429+0000 7f76390d7700 1 --2- 172.21.15.44:0/3065814871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7634067f30 0x7f7634198780 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f7624005910 tx=0x7f76240047b0 comp rx=0 tx=0).stop 2023-09-10T01:11:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.429+0000 7f76390d7700 1 -- 172.21.15.44:0/3065814871 shutdown_connections 2023-09-10T01:11:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.429+0000 7f76390d7700 1 --2- 172.21.15.44:0/3065814871 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f761c0a07b0 0x7f761c0a2c70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.429+0000 7f76390d7700 1 --2- 172.21.15.44:0/3065814871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f763410cdd0 0x7f763419d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.429+0000 7f76390d7700 1 --2- 172.21.15.44:0/3065814871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7634067f30 0x7f7634198780 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.429+0000 7f76390d7700 1 --2- 172.21.15.44:0/3065814871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7634101470 0x7f7634198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.429+0000 7f76390d7700 1 -- 172.21.15.44:0/3065814871 >> 172.21.15.44:0/3065814871 conn(0x7f7634074f30 msgr2=0x7f76340fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.429+0000 7f76390d7700 1 -- 172.21.15.44:0/3065814871 shutdown_connections 2023-09-10T01:11:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:03.429+0000 7f76390d7700 1 -- 172.21.15.44:0/3065814871 wait complete. 2023-09-10T01:11:04.532 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: *** Caught signal (Segmentation fault) ** 2023-09-10T01:11:04.532 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: in thread 7fefd2fbd700 thread_name:md_log_replay 2023-09-10T01:11:04.532 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:11:04.533 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fefe3b6ccf0] 2023-09-10T01:11:04.533 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x55bc284c730f] 2023-09-10T01:11:04.533 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55bc284c9543] 2023-09-10T01:11:04.533 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 4: (MDLog::_replay_thread()+0x647) [0x55bc28532e97] 2023-09-10T01:11:04.533 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55bc2853c381] 2023-09-10T01:11:04.534 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fefe3b621ca] 2023-09-10T01:11:04.534 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 7: clone() 2023-09-10T01:11:04.534 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 2023-09-10T01:11:04.071+0000 7fefd2fbd700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:11:04.534 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: in thread 7fefd2fbd700 thread_name:md_log_replay 2023-09-10T01:11:04.534 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 2023-09-10T01:11:04.535 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:11:04.535 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fefe3b6ccf0] 2023-09-10T01:11:04.535 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x55bc284c730f] 2023-09-10T01:11:04.535 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55bc284c9543] 2023-09-10T01:11:04.535 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 4: (MDLog::_replay_thread()+0x647) [0x55bc28532e97] 2023-09-10T01:11:04.536 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55bc2853c381] 2023-09-10T01:11:04.536 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fefe3b621ca] 2023-09-10T01:11:04.536 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 7: clone() 2023-09-10T01:11:04.536 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:11:04.536 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 2023-09-10T01:11:04.536 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 0> 2023-09-10T01:11:04.071+0000 7fefd2fbd700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:11:04.537 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: in thread 7fefd2fbd700 thread_name:md_log_replay 2023-09-10T01:11:04.537 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 2023-09-10T01:11:04.537 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:11:04.537 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fefe3b6ccf0] 2023-09-10T01:11:04.537 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x55bc284c730f] 2023-09-10T01:11:04.538 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55bc284c9543] 2023-09-10T01:11:04.538 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 4: (MDLog::_replay_thread()+0x647) [0x55bc28532e97] 2023-09-10T01:11:04.538 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55bc2853c381] 2023-09-10T01:11:04.539 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fefe3b621ca] 2023-09-10T01:11:04.539 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 7: clone() 2023-09-10T01:11:04.539 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:11:04.539 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 2023-09-10T01:11:04.539 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: -3> 2023-09-10T01:11:04.071+0000 7fefd2fbd700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:11:04.540 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: in thread 7fefd2fbd700 thread_name:md_log_replay 2023-09-10T01:11:04.540 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 2023-09-10T01:11:04.540 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:11:04.540 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fefe3b6ccf0] 2023-09-10T01:11:04.540 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x55bc284c730f] 2023-09-10T01:11:04.541 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55bc284c9543] 2023-09-10T01:11:04.541 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 4: (MDLog::_replay_thread()+0x647) [0x55bc28532e97] 2023-09-10T01:11:04.541 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55bc2853c381] 2023-09-10T01:11:04.541 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fefe3b621ca] 2023-09-10T01:11:04.541 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 7: clone() 2023-09-10T01:11:04.541 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:11:04.542 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[187064]: 2023-09-10T01:11:04.542 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Main process exited, code=exited, status=139/n/a 2023-09-10T01:11:04.805 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:04 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Failed with result 'exit-code'. 2023-09-10T01:11:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:04 smithi131 ceph-mon[103713]: pgmap v966: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 909 KiB/s wr, 129 op/s 2023-09-10T01:11:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:04 smithi131 ceph-mon[103713]: osdmap e144: 12 total, 12 up, 12 in 2023-09-10T01:11:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-mon[98676]: pgmap v966: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 909 KiB/s wr, 129 op/s 2023-09-10T01:11:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:04 smithi044 ceph-mon[98676]: osdmap e144: 12 total, 12 up, 12 in 2023-09-10T01:11:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:04 smithi099 ceph-mon[102692]: pgmap v966: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 909 KiB/s wr, 129 op/s 2023-09-10T01:11:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:04 smithi099 ceph-mon[102692]: osdmap e144: 12 total, 12 up, 12 in 2023-09-10T01:11:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:06 smithi131 ceph-mon[103713]: pgmap v968: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2023-09-10T01:11:07.661 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:06 smithi099 ceph-mon[102692]: pgmap v968: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2023-09-10T01:11:07.662 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:06 smithi044 ceph-mon[98676]: pgmap v968: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2023-09-10T01:11:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:08 smithi131 ceph-mon[103713]: pgmap v969: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2023-09-10T01:11:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:08 smithi044 ceph-mon[98676]: pgmap v969: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2023-09-10T01:11:09.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:08 smithi099 ceph-mon[102692]: pgmap v969: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2023-09-10T01:11:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:10 smithi131 ceph-mon[103713]: pgmap v970: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2023-09-10T01:11:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:10 smithi131 ceph-mon[103713]: osdmap e145: 12 total, 12 up, 12 in 2023-09-10T01:11:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:10 smithi044 ceph-mon[98676]: pgmap v970: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2023-09-10T01:11:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:10 smithi044 ceph-mon[98676]: osdmap e145: 12 total, 12 up, 12 in 2023-09-10T01:11:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:10 smithi099 ceph-mon[102692]: pgmap v970: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2023-09-10T01:11:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:10 smithi099 ceph-mon[102692]: osdmap e145: 12 total, 12 up, 12 in 2023-09-10T01:11:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:12 smithi131 ceph-mon[103713]: pgmap v972: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 237 op/s 2023-09-10T01:11:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:12 smithi044 ceph-mon[98676]: pgmap v972: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 237 op/s 2023-09-10T01:11:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:12 smithi099 ceph-mon[102692]: pgmap v972: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 237 op/s 2023-09-10T01:11:15.132 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:14 smithi044 ceph-mon[98676]: pgmap v973: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 198 op/s 2023-09-10T01:11:15.132 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:14 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Service RestartSec=10s expired, scheduling restart. 2023-09-10T01:11:15.133 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:14 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Scheduled restart job, restart counter is at 2. 2023-09-10T01:11:15.133 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:14 smithi044 systemd[1]: Stopped Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:11:15.133 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:14 smithi044 systemd[1]: Starting Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:11:15.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:14 smithi131 ceph-mon[103713]: pgmap v973: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 198 op/s 2023-09-10T01:11:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:14 smithi099 ceph-mon[102692]: pgmap v973: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 198 op/s 2023-09-10T01:11:15.426 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:15 smithi044 podman[194032]: 2023-09-10T01:11:15.805 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:15 smithi044 bash[194032]: f8ed68646dd8a642ad9256eea6817b3938494a92139021abe9078fa267bfce0e 2023-09-10T01:11:15.805 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:15 smithi044 ceph-mds[194072]: -- 172.21.15.44:0/804023072 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d5053651e0 con 0x55d50533b000 2023-09-10T01:11:15.805 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:15 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: starting mds.d at 2023-09-10T01:11:15.806 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:11:15 smithi044 systemd[1]: Started Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:11:16.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:15 smithi131 ceph-mon[103713]: daemon mds.d restarted 2023-09-10T01:11:16.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:15 smithi131 ceph-mon[103713]: osdmap e146: 12 total, 12 up, 12 in 2023-09-10T01:11:16.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:15 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.44:6836/223304868,v1:172.21.15.44:6837/223304868] up:boot 2023-09-10T01:11:16.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:15 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:11:16.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:15 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T01:11:16.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:15 smithi044 ceph-mon[98676]: daemon mds.d restarted 2023-09-10T01:11:16.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:15 smithi044 ceph-mon[98676]: osdmap e146: 12 total, 12 up, 12 in 2023-09-10T01:11:16.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:15 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.44:6836/223304868,v1:172.21.15.44:6837/223304868] up:boot 2023-09-10T01:11:16.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:15 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:11:16.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:15 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T01:11:16.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:15 smithi099 ceph-mon[102692]: daemon mds.d restarted 2023-09-10T01:11:16.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:15 smithi099 ceph-mon[102692]: osdmap e146: 12 total, 12 up, 12 in 2023-09-10T01:11:16.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:15 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.44:6836/223304868,v1:172.21.15.44:6837/223304868] up:boot 2023-09-10T01:11:16.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:15 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:11:16.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:15 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T01:11:16.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.543+0000 7fa933421700 1 -- 172.21.15.44:0/2076093414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa92c1055b0 msgr2=0x7fa92c105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.543+0000 7fa933421700 1 --2- 172.21.15.44:0/2076093414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa92c1055b0 0x7fa92c105990 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa9280099c0 tx=0x7fa928009cd0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.543+0000 7fa933421700 1 -- 172.21.15.44:0/2076093414 shutdown_connections 2023-09-10T01:11:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.543+0000 7fa933421700 1 --2- 172.21.15.44:0/2076093414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa92c100120 0x7fa92c10c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.543+0000 7fa933421700 1 --2- 172.21.15.44:0/2076093414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa92c1055b0 0x7fa92c105990 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.543+0000 7fa933421700 1 --2- 172.21.15.44:0/2076093414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa92c0ff760 0x7fa92c0ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.543+0000 7fa933421700 1 -- 172.21.15.44:0/2076093414 >> 172.21.15.44:0/2076093414 conn(0x7fa92c074f30 msgr2=0x7fa92c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.543+0000 7fa933421700 1 -- 172.21.15.44:0/2076093414 shutdown_connections 2023-09-10T01:11:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.543+0000 7fa933421700 1 -- 172.21.15.44:0/2076093414 wait complete. 2023-09-10T01:11:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa933421700 1 Processor -- start 2023-09-10T01:11:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa933421700 1 -- start start 2023-09-10T01:11:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa933421700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa92c0ff760 0x7fa92c195430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa933421700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa92c100120 0x7fa92c195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa933421700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa92c1055b0 0x7fa92c199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa933421700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19a620 con 0x7fa92c100120 2023-09-10T01:11:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa933421700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19a790 con 0x7fa92c1055b0 2023-09-10T01:11:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa933421700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19a950 con 0x7fa92c0ff760 2023-09-10T01:11:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa9309bc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa92c100120 0x7fa92c195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa9311bd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa92c0ff760 0x7fa92c195430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa9309bc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa92c100120 0x7fa92c195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56744/0 (socket says 172.21.15.44:56744) 2023-09-10T01:11:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa9309bc700 1 -- 172.21.15.44:0/3180951276 learned_addr learned my addr 172.21.15.44:0/3180951276 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa9319be700 1 --2- 172.21.15.44:0/3180951276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa92c1055b0 0x7fa92c199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa9309bc700 1 -- 172.21.15.44:0/3180951276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa92c0ff760 msgr2=0x7fa92c195430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa9309bc700 1 --2- 172.21.15.44:0/3180951276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa92c0ff760 0x7fa92c195430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa9309bc700 1 -- 172.21.15.44:0/3180951276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa92c1055b0 msgr2=0x7fa92c199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa9309bc700 1 --2- 172.21.15.44:0/3180951276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa92c1055b0 0x7fa92c199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.544+0000 7fa9309bc700 1 -- 172.21.15.44:0/3180951276 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9280095f0 con 0x7fa92c100120 2023-09-10T01:11:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.545+0000 7fa9309bc700 1 --2- 172.21.15.44:0/3180951276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa92c100120 0x7fa92c195970 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00e9a0 tx=0x7fa91c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.545+0000 7fa9227fc700 1 -- 172.21.15.44:0/3180951276 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c009790 con 0x7fa92c100120 2023-09-10T01:11:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.545+0000 7fa9227fc700 1 -- 172.21.15.44:0/3180951276 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa91c010430 con 0x7fa92c100120 2023-09-10T01:11:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.545+0000 7fa933421700 1 -- 172.21.15.44:0/3180951276 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92c19ac30 con 0x7fa92c100120 2023-09-10T01:11:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.545+0000 7fa9227fc700 1 -- 172.21.15.44:0/3180951276 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c00cea0 con 0x7fa92c100120 2023-09-10T01:11:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.545+0000 7fa933421700 1 -- 172.21.15.44:0/3180951276 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c1a2470 con 0x7fa92c100120 2023-09-10T01:11:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.547+0000 7fa9227fc700 1 -- 172.21.15.44:0/3180951276 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa91c0105a0 con 0x7fa92c100120 2023-09-10T01:11:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.547+0000 7fa9227fc700 1 --2- 172.21.15.44:0/3180951276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa9180a07b0 0x7fa9180a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.548+0000 7fa9311bd700 1 --2- 172.21.15.44:0/3180951276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa9180a07b0 0x7fa9180a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.548+0000 7fa9227fc700 1 -- 172.21.15.44:0/3180951276 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa91c014070 con 0x7fa92c100120 2023-09-10T01:11:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.548+0000 7fa9311bd700 1 --2- 172.21.15.44:0/3180951276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa9180a07b0 0x7fa9180a2c70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa92800b580 tx=0x7fa928005c00 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.548+0000 7fa93f5e7b80 1 Processor -- start 2023-09-10T01:11:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.548+0000 7fa93f5e7b80 1 -- start start 2023-09-10T01:11:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.548+0000 7fa93f5e7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b8a3b0 0x564722b8a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.548+0000 7fa93f5e7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b84510 0x564722b84990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.548+0000 7fa93f5e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84ed0 0x564722b91140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.548+0000 7fa93f5e7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564722adb6d0 con 0x564722b84510 2023-09-10T01:11:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.548+0000 7fa93f5e7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564722ad8ff0 con 0x564722b8a3b0 2023-09-10T01:11:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.548+0000 7fa93f5e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564722ad48e0 con 0x564722b84ed0 2023-09-10T01:11:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.549+0000 7fa9309bc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b84510 0x564722b84990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.549+0000 7fa9309bc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b84510 0x564722b84990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56752/0 (socket says 172.21.15.44:56752) 2023-09-10T01:11:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.549+0000 7fa9311bd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b8a3b0 0x564722b8a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.549+0000 7fa9309bc700 1 -- 172.21.15.44:0/1470937144 learned_addr learned my addr 172.21.15.44:0/1470937144 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.549+0000 7fa9319be700 1 --2- 172.21.15.44:0/1470937144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84ed0 0x564722b91140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.549+0000 7fa9309bc700 1 -- 172.21.15.44:0/1470937144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84ed0 msgr2=0x564722b91140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.549+0000 7fa9309bc700 1 --2- 172.21.15.44:0/1470937144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84ed0 0x564722b91140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.549+0000 7fa9309bc700 1 -- 172.21.15.44:0/1470937144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b8a3b0 msgr2=0x564722b8a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.549+0000 7fa9309bc700 1 --2- 172.21.15.44:0/1470937144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b8a3b0 0x564722b8a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.549+0000 7fa9309bc700 1 -- 172.21.15.44:0/1470937144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564722900d80 con 0x564722b84510 2023-09-10T01:11:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.549+0000 7fa9309bc700 1 --2- 172.21.15.44:0/1470937144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b84510 0x564722b84990 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa91c096060 tx=0x7fa91c098f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e493341369eb15 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.549+0000 7fa9177fe700 1 -- 172.21.15.44:0/1470937144 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c099d30 con 0x564722b84510 2023-09-10T01:11:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.549+0000 7fa9177fe700 1 -- 172.21.15.44:0/1470937144 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa91c0995f0 con 0x564722b84510 2023-09-10T01:11:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.549+0000 7fa9177fe700 1 -- 172.21.15.44:0/1470937144 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0840b0 con 0x564722b84510 2023-09-10T01:11:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.550+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/1470937144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b84510 msgr2=0x564722b84990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.550+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/1470937144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b84510 0x564722b84990 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa91c096060 tx=0x7fa91c098f50 comp rx=0 tx=0).stop 2023-09-10T01:11:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.550+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/1470937144 shutdown_connections 2023-09-10T01:11:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.550+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/1470937144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84ed0 0x564722b91140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.550+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/1470937144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b8a3b0 0x564722b8a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.550+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/1470937144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b84510 0x564722b84990 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.550+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/1470937144 >> 172.21.15.44:0/1470937144 conn(0x564722af7030 msgr2=0x564722b7d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.550+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/1470937144 shutdown_connections 2023-09-10T01:11:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.550+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/1470937144 wait complete. 2023-09-10T01:11:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.550+0000 7fa93f5e7b80 1 -- start start 2023-09-10T01:11:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.550+0000 7fa93f5e7b80 1 Processor -- start 2023-09-10T01:11:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa93f5e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84510 0x564722b91080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa93f5e7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b84ed0 0x564722b8e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa93f5e7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b8a3b0 0x564722b8e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa93f5e7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564722a72b10 con 0x564722b8a3b0 2023-09-10T01:11:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa93f5e7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564722a72810 con 0x564722b84ed0 2023-09-10T01:11:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa93f5e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647229568c0 con 0x564722b84510 2023-09-10T01:11:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa93f5e7b80 10 client.0 _ll_register_callbacks cb 0x5647229bca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:11:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa9319be700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b8a3b0 0x564722b8e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa9311bd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84510 0x564722b91080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa9309bc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b84ed0 0x564722b8e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa9311bd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84510 0x564722b91080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35406/0 (socket says 172.21.15.44:35406) 2023-09-10T01:11:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa9311bd700 1 -- 172.21.15.44:0/1340711802 learned_addr learned my addr 172.21.15.44:0/1340711802 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa9319be700 1 -- 172.21.15.44:0/1340711802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84510 msgr2=0x564722b91080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa9319be700 1 --2- 172.21.15.44:0/1340711802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84510 0x564722b91080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa9319be700 1 -- 172.21.15.44:0/1340711802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b84ed0 msgr2=0x564722b8e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa9319be700 1 --2- 172.21.15.44:0/1340711802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b84ed0 0x564722b8e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa9319be700 1 -- 172.21.15.44:0/1340711802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c1a2470 con 0x564722b8a3b0 2023-09-10T01:11:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa9311bd700 1 --2- 172.21.15.44:0/1340711802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84510 0x564722b91080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:11:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa9319be700 1 --2- 172.21.15.44:0/1340711802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b8a3b0 0x564722b8e940 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa924012dc0 tx=0x7fa924014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a11c9a42353d372 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa914ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:11:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa914ff9700 1 -- 172.21.15.44:0/1340711802 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92400e040 con 0x564722b8a3b0 2023-09-10T01:11:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.551+0000 7fa914ff9700 1 -- 172.21.15.44:0/1340711802 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa924014a80 con 0x564722b8a3b0 2023-09-10T01:11:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.552+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/1340711802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564722a2c490 con 0x564722b8a3b0 2023-09-10T01:11:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.552+0000 7fa914ff9700 1 -- 172.21.15.44:0/1340711802 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa924014c40 con 0x564722b8a3b0 2023-09-10T01:11:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.552+0000 7fa93f5e7b80 10 client.19131 fetch_fsmap learned FSMap version 117 2023-09-10T01:11:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.552+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/1340711802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=117,monmap=4+}) v3 -- 0x5647228e9c00 con 0x564722b8a3b0 2023-09-10T01:11:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.552+0000 7fa914ff9700 1 -- 172.21.15.44:0/1340711802 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564722a2c490 con 0x564722b8a3b0 2023-09-10T01:11:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.552+0000 7fa914ff9700 1 -- 172.21.15.44:0/1340711802 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 117) v1 ==== 7335+0+0 (secure 0 0 0) 0x7fa92401c020 con 0x564722b8a3b0 2023-09-10T01:11:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.552+0000 7fa93f5e7b80 10 client.19131 fetch_fsmap finished waiting for FSMap version 117 2023-09-10T01:11:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.552+0000 7fa93f5e7b80 10 client.19131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:11:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.552+0000 7fa93f5e7b80 20 client.19131 populate_metadata read hostname 'smithi044' 2023-09-10T01:11:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.552+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/1340711802 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564722b9e770 0x564722ba0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.552+0000 7fa93f5e7b80 4 client.19131 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:11:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.552+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/1340711802 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564722ae9840 con 0x564722b9e770 2023-09-10T01:11:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.552+0000 7fa9311bd700 1 --2- 172.21.15.44:0/1340711802 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564722b9e770 0x564722ba0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.553+0000 7fa914ff9700 10 client.19131 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.553+0000 7fa9311bd700 1 --2- 172.21.15.44:0/1340711802 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564722b9e770 0x564722ba0b90 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4556a1675fd87e2 server_cookie=f3566ec391addac0 in_seq=0 out_seq=0 2023-09-10T01:11:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.553+0000 7fa914ff9700 1 -- 172.21.15.44:0/1340711802 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564722ae9840 con 0x564722b9e770 2023-09-10T01:11:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.553+0000 7fa914ff9700 10 client.19131 handle_command_reply: tid=0 2023-09-10T01:11:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.553+0000 7fa93f5e7b80 1 client.19131 shutdown 2023-09-10T01:11:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.553+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/1340711802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b8a3b0 msgr2=0x564722b8e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.553+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/1340711802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b8a3b0 0x564722b8e940 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa924012dc0 tx=0x7fa924014600 comp rx=0 tx=0).stop 2023-09-10T01:11:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.553+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/1340711802 shutdown_connections 2023-09-10T01:11:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.553+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/1340711802 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564722b9e770 0x564722ba0b90 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.553+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/1340711802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b8a3b0 0x564722b8e940 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.553+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/1340711802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84510 0x564722b91080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.553+0000 7fa914ff9700 0 client.19131 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.553+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/1340711802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b84ed0 0x564722b8e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.553+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/1340711802 >> 172.21.15.44:0/1340711802 conn(0x564722af7030 msgr2=0x564722b7c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.554+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/1340711802 shutdown_connections 2023-09-10T01:11:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.554+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/1340711802 wait complete. 2023-09-10T01:11:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.554+0000 7fa93f5e7b80 20 client.19131 trim_cache size 0 max 16384 2023-09-10T01:11:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.569+0000 7fa93f5e7b80 1 Processor -- start 2023-09-10T01:11:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.569+0000 7fa93f5e7b80 1 -- start start 2023-09-10T01:11:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.569+0000 7fa93f5e7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b84510 0x564722b8db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.569+0000 7fa93f5e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84ed0 0x564722b90b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.569+0000 7fa93f5e7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b8a3b0 0x564722b91080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.569+0000 7fa93f5e7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564722b7e320 con 0x564722b84510 2023-09-10T01:11:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.569+0000 7fa93f5e7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564722ba11f0 con 0x564722b8a3b0 2023-09-10T01:11:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.569+0000 7fa93f5e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564722b8c8e0 con 0x564722b84ed0 2023-09-10T01:11:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.569+0000 7fa9311bd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b84510 0x564722b8db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.569+0000 7fa9311bd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b84510 0x564722b8db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56764/0 (socket says 172.21.15.44:56764) 2023-09-10T01:11:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.569+0000 7fa9311bd700 1 -- 172.21.15.44:0/3456767821 learned_addr learned my addr 172.21.15.44:0/3456767821 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.569+0000 7fa9319be700 1 --2- 172.21.15.44:0/3456767821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b8a3b0 0x564722b91080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.569+0000 7fa9309bc700 1 --2- 172.21.15.44:0/3456767821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84ed0 0x564722b90b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.570+0000 7fa9311bd700 1 -- 172.21.15.44:0/3456767821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84ed0 msgr2=0x564722b90b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.570+0000 7fa9311bd700 1 --2- 172.21.15.44:0/3456767821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84ed0 0x564722b90b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.570+0000 7fa9311bd700 1 -- 172.21.15.44:0/3456767821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b8a3b0 msgr2=0x564722b91080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.570+0000 7fa9311bd700 1 --2- 172.21.15.44:0/3456767821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b8a3b0 0x564722b91080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.570+0000 7fa9311bd700 1 -- 172.21.15.44:0/3456767821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647229271f0 con 0x564722b84510 2023-09-10T01:11:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.570+0000 7fa9311bd700 1 --2- 172.21.15.44:0/3456767821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b84510 0x564722b8db40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa928000c00 tx=0x7fa928021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5c72362d2ba66b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.570+0000 7fa917fff700 1 -- 172.21.15.44:0/3456767821 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928028070 con 0x564722b84510 2023-09-10T01:11:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.570+0000 7fa917fff700 1 -- 172.21.15.44:0/3456767821 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa928029040 con 0x564722b84510 2023-09-10T01:11:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.570+0000 7fa917fff700 1 -- 172.21.15.44:0/3456767821 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928032510 con 0x564722b84510 2023-09-10T01:11:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/3456767821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b84510 msgr2=0x564722b8db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/3456767821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b84510 0x564722b8db40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa928000c00 tx=0x7fa928021000 comp rx=0 tx=0).stop 2023-09-10T01:11:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/3456767821 shutdown_connections 2023-09-10T01:11:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/3456767821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b8a3b0 0x564722b91080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/3456767821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b84510 0x564722b8db40 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/3456767821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84ed0 0x564722b90b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/3456767821 >> 172.21.15.44:0/3456767821 conn(0x564722af7030 msgr2=0x564722b8e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/3456767821 shutdown_connections 2023-09-10T01:11:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/3456767821 wait complete. 2023-09-10T01:11:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 -- start start 2023-09-10T01:11:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 Processor -- start 2023-09-10T01:11:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b84510 0x564722b93cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84ed0 0x564722b94200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b8a3b0 0x564722b94740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564722b95d10 con 0x564722b8a3b0 2023-09-10T01:11:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564722bf2570 con 0x564722b84510 2023-09-10T01:11:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564722bf26e0 con 0x564722b84ed0 2023-09-10T01:11:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa93f5e7b80 10 client.0 _ll_register_callbacks cb 0x7fa9240126f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:11:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.571+0000 7fa9319be700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b8a3b0 0x564722b94740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa9309bc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84ed0 0x564722b94200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa9319be700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b8a3b0 0x564722b94740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56774/0 (socket says 172.21.15.44:56774) 2023-09-10T01:11:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa9319be700 1 -- 172.21.15.44:0/3926608154 learned_addr learned my addr 172.21.15.44:0/3926608154 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa9311bd700 1 --2- 172.21.15.44:0/3926608154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b84510 0x564722b93cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa9319be700 1 -- 172.21.15.44:0/3926608154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84ed0 msgr2=0x564722b94200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa9319be700 1 --2- 172.21.15.44:0/3926608154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84ed0 0x564722b94200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa9319be700 1 -- 172.21.15.44:0/3926608154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b84510 msgr2=0x564722b93cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa9319be700 1 --2- 172.21.15.44:0/3926608154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b84510 0x564722b93cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa9319be700 1 -- 172.21.15.44:0/3926608154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c10cbc0 con 0x564722b8a3b0 2023-09-10T01:11:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa9319be700 1 --2- 172.21.15.44:0/3926608154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b8a3b0 0x564722b94740 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fa924009960 tx=0x7fa924009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d771b483765c566 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa914ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:11:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa914ff9700 1 -- 172.21.15.44:0/3926608154 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92400e070 con 0x564722b8a3b0 2023-09-10T01:11:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa914ff9700 1 -- 172.21.15.44:0/3926608154 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa9240049f0 con 0x564722b8a3b0 2023-09-10T01:11:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/3926608154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa91c0995f0 con 0x564722b8a3b0 2023-09-10T01:11:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa914ff9700 1 -- 172.21.15.44:0/3926608154 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa924004bb0 con 0x564722b8a3b0 2023-09-10T01:11:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa93f5e7b80 10 client.19137 fetch_fsmap learned FSMap version 117 2023-09-10T01:11:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/3926608154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=117,monmap=4+}) v3 -- 0x5647228ff250 con 0x564722b8a3b0 2023-09-10T01:11:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.572+0000 7fa914ff9700 1 -- 172.21.15.44:0/3926608154 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa91c0995f0 con 0x564722b8a3b0 2023-09-10T01:11:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.573+0000 7fa914ff9700 1 -- 172.21.15.44:0/3926608154 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 117) v1 ==== 7335+0+0 (secure 0 0 0) 0x7fa924014320 con 0x564722b8a3b0 2023-09-10T01:11:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.573+0000 7fa93f5e7b80 10 client.19137 fetch_fsmap finished waiting for FSMap version 117 2023-09-10T01:11:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.573+0000 7fa93f5e7b80 10 client.19137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:11:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.573+0000 7fa93f5e7b80 20 client.19137 populate_metadata read hostname 'smithi044' 2023-09-10T01:11:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.573+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/3926608154 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564722b827e0 0x564722b82bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.573+0000 7fa93f5e7b80 4 client.19137 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:11:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.573+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/3926608154 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564722acd8c0 con 0x564722b827e0 2023-09-10T01:11:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.573+0000 7fa9311bd700 1 --2- 172.21.15.44:0/3926608154 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564722b827e0 0x564722b82bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.573+0000 7fa914ff9700 10 client.19137 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.573+0000 7fa9311bd700 1 --2- 172.21.15.44:0/3926608154 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564722b827e0 0x564722b82bf0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd90d7672a1bae2e server_cookie=35335fdfaae9cef9 in_seq=0 out_seq=0 2023-09-10T01:11:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.598+0000 7fa914ff9700 1 -- 172.21.15.44:0/3926608154 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564722acd8c0 con 0x564722b827e0 2023-09-10T01:11:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.598+0000 7fa914ff9700 10 client.19137 handle_command_reply: tid=0 2023-09-10T01:11:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.598+0000 7fa93f5e7b80 1 client.19137 shutdown 2023-09-10T01:11:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.598+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/3926608154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b8a3b0 msgr2=0x564722b94740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.598+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/3926608154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b8a3b0 0x564722b94740 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fa924009960 tx=0x7fa924009510 comp rx=0 tx=0).stop 2023-09-10T01:11:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.598+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/3926608154 shutdown_connections 2023-09-10T01:11:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.598+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/3926608154 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564722b827e0 0x564722b82bf0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.598+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/3926608154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564722b8a3b0 0x564722b94740 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.598+0000 7fa914ff9700 0 client.19137 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.598+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/3926608154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564722b84510 0x564722b93cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.598+0000 7fa93f5e7b80 1 --2- 172.21.15.44:0/3926608154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564722b84ed0 0x564722b94200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.598+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/3926608154 >> 172.21.15.44:0/3926608154 conn(0x564722af7030 msgr2=0x564722b812f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.598+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/3926608154 shutdown_connections 2023-09-10T01:11:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.598+0000 7fa93f5e7b80 1 -- 172.21.15.44:0/3926608154 wait complete. 2023-09-10T01:11:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.599+0000 7fa93f5e7b80 20 client.19137 trim_cache size 0 max 16384 2023-09-10T01:11:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:11:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:11:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:11:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:11:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.599+0000 7fa933421700 1 -- 172.21.15.44:0/3180951276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa9180a07b0 msgr2=0x7fa9180a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.599+0000 7fa933421700 1 --2- 172.21.15.44:0/3180951276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa9180a07b0 0x7fa9180a2c70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa92800b580 tx=0x7fa928005c00 comp rx=0 tx=0).stop 2023-09-10T01:11:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.599+0000 7fa933421700 1 -- 172.21.15.44:0/3180951276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa92c100120 msgr2=0x7fa92c195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.599+0000 7fa933421700 1 --2- 172.21.15.44:0/3180951276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa92c100120 0x7fa92c195970 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00e9a0 tx=0x7fa91c00ecb0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.599+0000 7fa933421700 1 -- 172.21.15.44:0/3180951276 shutdown_connections 2023-09-10T01:11:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.599+0000 7fa933421700 1 --2- 172.21.15.44:0/3180951276 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa9180a07b0 0x7fa9180a2c70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.599+0000 7fa933421700 1 --2- 172.21.15.44:0/3180951276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa92c1055b0 0x7fa92c199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.599+0000 7fa933421700 1 --2- 172.21.15.44:0/3180951276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa92c0ff760 0x7fa92c195430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.599+0000 7fa933421700 1 --2- 172.21.15.44:0/3180951276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa92c100120 0x7fa92c195970 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.599+0000 7fa933421700 1 -- 172.21.15.44:0/3180951276 >> 172.21.15.44:0/3180951276 conn(0x7fa92c074f30 msgr2=0x7fa92c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.599+0000 7fa933421700 1 -- 172.21.15.44:0/3180951276 shutdown_connections 2023-09-10T01:11:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:16.599+0000 7fa933421700 1 -- 172.21.15.44:0/3180951276 wait complete. 2023-09-10T01:11:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:16 smithi131 ceph-mon[103713]: pgmap v974: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:11:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:16 smithi044 ceph-mon[98676]: pgmap v974: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:11:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:16 smithi099 ceph-mon[102692]: pgmap v974: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:11:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:18 smithi131 ceph-mon[103713]: pgmap v976: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 238 op/s 2023-09-10T01:11:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:18 smithi044 ceph-mon[98676]: pgmap v976: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 238 op/s 2023-09-10T01:11:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:18 smithi099 ceph-mon[102692]: pgmap v976: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 238 op/s 2023-09-10T01:11:20.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:19 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:11:20.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:19 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:11:20.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:19 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:11:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:20 smithi131 ceph-mon[103713]: pgmap v977: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 870 KiB/s wr, 121 op/s 2023-09-10T01:11:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:20 smithi044 ceph-mon[98676]: pgmap v977: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 870 KiB/s wr, 121 op/s 2023-09-10T01:11:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:20 smithi099 ceph-mon[102692]: pgmap v977: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 870 KiB/s wr, 121 op/s 2023-09-10T01:11:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:22 smithi131 ceph-mon[103713]: pgmap v978: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:11:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:22 smithi044 ceph-mon[98676]: pgmap v978: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:11:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:22 smithi099 ceph-mon[102692]: pgmap v978: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:11:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:24 smithi131 ceph-mon[103713]: pgmap v979: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:11:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:24 smithi044 ceph-mon[98676]: pgmap v979: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:11:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:24 smithi099 ceph-mon[102692]: pgmap v979: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:11:26.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:25 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:11:26.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:25 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:11:26.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:25 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:11:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:26 smithi131 ceph-mon[103713]: pgmap v980: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2023-09-10T01:11:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:26 smithi044 ceph-mon[98676]: pgmap v980: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2023-09-10T01:11:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:26 smithi099 ceph-mon[102692]: pgmap v980: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2023-09-10T01:11:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:28 smithi131 ceph-mon[103713]: pgmap v981: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 167 op/s 2023-09-10T01:11:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:28 smithi044 ceph-mon[98676]: pgmap v981: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 167 op/s 2023-09-10T01:11:29.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:28 smithi099 ceph-mon[102692]: pgmap v981: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 167 op/s 2023-09-10T01:11:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.751+0000 7faa5943a700 1 -- 172.21.15.44:0/2422440357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa541055b0 msgr2=0x7faa54105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.751+0000 7faa5943a700 1 --2- 172.21.15.44:0/2422440357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa541055b0 0x7faa54105990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faa44009920 tx=0x7faa44009c30 comp rx=0 tx=0).stop 2023-09-10T01:11:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.751+0000 7faa5943a700 1 -- 172.21.15.44:0/2422440357 shutdown_connections 2023-09-10T01:11:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.751+0000 7faa5943a700 1 --2- 172.21.15.44:0/2422440357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa54100120 0x7faa5410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.751+0000 7faa5943a700 1 --2- 172.21.15.44:0/2422440357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa541055b0 0x7faa54105990 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.751+0000 7faa5943a700 1 --2- 172.21.15.44:0/2422440357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa540ff760 0x7faa540ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.751+0000 7faa5943a700 1 -- 172.21.15.44:0/2422440357 >> 172.21.15.44:0/2422440357 conn(0x7faa54074f30 msgr2=0x7faa54075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.751+0000 7faa5943a700 1 -- 172.21.15.44:0/2422440357 shutdown_connections 2023-09-10T01:11:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.751+0000 7faa5943a700 1 -- 172.21.15.44:0/2422440357 wait complete. 2023-09-10T01:11:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa5943a700 1 Processor -- start 2023-09-10T01:11:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa5943a700 1 -- start start 2023-09-10T01:11:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa5943a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa540ff760 0x7faa541954c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa5943a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa54100120 0x7faa54195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa5943a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa541055b0 0x7faa54198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa5943a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa541995c0 con 0x7faa54100120 2023-09-10T01:11:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa5943a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa54199730 con 0x7faa540ff760 2023-09-10T01:11:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa5943a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa541998f0 con 0x7faa541055b0 2023-09-10T01:11:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa537fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa541055b0 0x7faa54198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa537fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa541055b0 0x7faa54198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35436/0 (socket says 172.21.15.44:35436) 2023-09-10T01:11:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa537fe700 1 -- 172.21.15.44:0/3431755200 learned_addr learned my addr 172.21.15.44:0/3431755200 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa527fc700 1 --2- 172.21.15.44:0/3431755200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa54100120 0x7faa54195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa52ffd700 1 --2- 172.21.15.44:0/3431755200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa540ff760 0x7faa541954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa537fe700 1 -- 172.21.15.44:0/3431755200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa540ff760 msgr2=0x7faa541954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa537fe700 1 --2- 172.21.15.44:0/3431755200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa540ff760 0x7faa541954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa537fe700 1 -- 172.21.15.44:0/3431755200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa54100120 msgr2=0x7faa54195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa537fe700 1 --2- 172.21.15.44:0/3431755200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa54100120 0x7faa54195a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.752+0000 7faa537fe700 1 -- 172.21.15.44:0/3431755200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa44009520 con 0x7faa541055b0 2023-09-10T01:11:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.753+0000 7faa527fc700 1 --2- 172.21.15.44:0/3431755200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa54100120 0x7faa54195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:11:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.753+0000 7faa537fe700 1 --2- 172.21.15.44:0/3431755200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa541055b0 0x7faa54198fc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00ef30 tx=0x7faa4c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.753+0000 7faa3bfff700 1 -- 172.21.15.44:0/3431755200 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c018070 con 0x7faa541055b0 2023-09-10T01:11:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.753+0000 7faa3bfff700 1 -- 172.21.15.44:0/3431755200 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa4c00f040 con 0x7faa541055b0 2023-09-10T01:11:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.753+0000 7faa3bfff700 1 -- 172.21.15.44:0/3431755200 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c00cbb0 con 0x7faa541055b0 2023-09-10T01:11:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.753+0000 7faa5943a700 1 -- 172.21.15.44:0/3431755200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa54199b70 con 0x7faa541055b0 2023-09-10T01:11:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.753+0000 7faa5943a700 1 -- 172.21.15.44:0/3431755200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa541a2530 con 0x7faa541055b0 2023-09-10T01:11:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.756+0000 7faa3bfff700 1 -- 172.21.15.44:0/3431755200 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7faa4c00cd90 con 0x7faa541055b0 2023-09-10T01:11:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.757+0000 7faa3bfff700 1 --2- 172.21.15.44:0/3431755200 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faa3c0a0830 0x7faa3c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.757+0000 7faa3bfff700 1 -- 172.21.15.44:0/3431755200 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 9452+0+0 (secure 0 0 0) 0x7faa4c0cbdb0 con 0x7faa541055b0 2023-09-10T01:11:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.757+0000 7faa52ffd700 1 --2- 172.21.15.44:0/3431755200 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faa3c0a0830 0x7faa3c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.757+0000 7faa52ffd700 1 --2- 172.21.15.44:0/3431755200 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faa3c0a0830 0x7faa3c0a2cf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faa440098f0 tx=0x7faa44019040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.757+0000 7faa65600b80 1 Processor -- start 2023-09-10T01:11:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa65600b80 1 -- start start 2023-09-10T01:11:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa65600b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 0x55b1ce9e4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa65600b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9de300 0x55b1ce9de780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa65600b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9decc0 0x55b1ce9eaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa65600b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce9376d0 con 0x55b1ce9decc0 2023-09-10T01:11:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa65600b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce934ff0 con 0x55b1ce9e41a0 2023-09-10T01:11:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa65600b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce9308e0 con 0x55b1ce9de300 2023-09-10T01:11:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa52ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 0x55b1ce9e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa537fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9decc0 0x55b1ce9eaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa52ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 0x55b1ce9e4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58532/0 (socket says 172.21.15.44:58532) 2023-09-10T01:11:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa527fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9de300 0x55b1ce9de780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa52ffd700 1 -- 172.21.15.44:0/1841034062 learned_addr learned my addr 172.21.15.44:0/1841034062 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa537fe700 1 -- 172.21.15.44:0/1841034062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9de300 msgr2=0x55b1ce9de780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa537fe700 1 --2- 172.21.15.44:0/1841034062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9de300 0x55b1ce9de780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa537fe700 1 -- 172.21.15.44:0/1841034062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 msgr2=0x55b1ce9e4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa537fe700 1 --2- 172.21.15.44:0/1841034062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 0x55b1ce9e4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa537fe700 1 -- 172.21.15.44:0/1841034062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1ce75cd80 con 0x55b1ce9decc0 2023-09-10T01:11:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa52ffd700 1 --2- 172.21.15.44:0/1841034062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 0x55b1ce9e4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:11:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.758+0000 7faa537fe700 1 --2- 172.21.15.44:0/1841034062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9decc0 0x55b1ce9eaf30 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0c9160 tx=0x7faa4c098b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9144fddf4bdd21a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.759+0000 7faa397fa700 1 -- 172.21.15.44:0/1841034062 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c099650 con 0x55b1ce9decc0 2023-09-10T01:11:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.759+0000 7faa397fa700 1 -- 172.21.15.44:0/1841034062 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa4c098f50 con 0x55b1ce9decc0 2023-09-10T01:11:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.759+0000 7faa397fa700 1 -- 172.21.15.44:0/1841034062 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c099410 con 0x55b1ce9decc0 2023-09-10T01:11:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.759+0000 7faa65600b80 1 -- 172.21.15.44:0/1841034062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9decc0 msgr2=0x55b1ce9eaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.759+0000 7faa65600b80 1 --2- 172.21.15.44:0/1841034062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9decc0 0x55b1ce9eaf30 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0c9160 tx=0x7faa4c098b60 comp rx=0 tx=0).stop 2023-09-10T01:11:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.759+0000 7faa65600b80 1 -- 172.21.15.44:0/1841034062 shutdown_connections 2023-09-10T01:11:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.759+0000 7faa65600b80 1 --2- 172.21.15.44:0/1841034062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9decc0 0x55b1ce9eaf30 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.759+0000 7faa65600b80 1 --2- 172.21.15.44:0/1841034062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 0x55b1ce9e4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.759+0000 7faa65600b80 1 --2- 172.21.15.44:0/1841034062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9de300 0x55b1ce9de780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.759+0000 7faa65600b80 1 -- 172.21.15.44:0/1841034062 >> 172.21.15.44:0/1841034062 conn(0x55b1ce953030 msgr2=0x55b1ce9d9560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.760+0000 7faa65600b80 1 -- 172.21.15.44:0/1841034062 shutdown_connections 2023-09-10T01:11:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.760+0000 7faa65600b80 1 -- 172.21.15.44:0/1841034062 wait complete. 2023-09-10T01:11:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.760+0000 7faa65600b80 1 -- start start 2023-09-10T01:11:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.760+0000 7faa65600b80 1 Processor -- start 2023-09-10T01:11:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.760+0000 7faa65600b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9de300 0x55b1ce9ede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.760+0000 7faa65600b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9decc0 0x55b1ce9ee360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.760+0000 7faa65600b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 0x55b1ce9e80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.760+0000 7faa65600b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce8ceb10 con 0x55b1ce9decc0 2023-09-10T01:11:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.760+0000 7faa65600b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce8ce810 con 0x55b1ce9e41a0 2023-09-10T01:11:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.760+0000 7faa65600b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce7b28c0 con 0x55b1ce9de300 2023-09-10T01:11:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.760+0000 7faa65600b80 10 client.0 _ll_register_callbacks cb 0x55b1ce818a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:11:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa52ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9de300 0x55b1ce9ede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa527fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9decc0 0x55b1ce9ee360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa52ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9de300 0x55b1ce9ede20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35448/0 (socket says 172.21.15.44:35448) 2023-09-10T01:11:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa52ffd700 1 -- 172.21.15.44:0/168212688 learned_addr learned my addr 172.21.15.44:0/168212688 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa537fe700 1 --2- 172.21.15.44:0/168212688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 0x55b1ce9e80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa52ffd700 1 -- 172.21.15.44:0/168212688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 msgr2=0x55b1ce9e80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa52ffd700 1 --2- 172.21.15.44:0/168212688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 0x55b1ce9e80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa52ffd700 1 -- 172.21.15.44:0/168212688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9decc0 msgr2=0x55b1ce9ee360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa52ffd700 1 --2- 172.21.15.44:0/168212688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9decc0 0x55b1ce9ee360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa52ffd700 1 -- 172.21.15.44:0/168212688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5410cbc0 con 0x55b1ce9de300 2023-09-10T01:11:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa537fe700 1 --2- 172.21.15.44:0/168212688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 0x55b1ce9e80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:11:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa52ffd700 1 --2- 172.21.15.44:0/168212688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9de300 0x55b1ce9ede20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faa4400f6e0 tx=0x7faa44009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45e4f5b55debf39c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa36ff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:11:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa36ff5700 1 -- 172.21.15.44:0/168212688 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4402c040 con 0x55b1ce9de300 2023-09-10T01:11:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa65600b80 1 -- 172.21.15.44:0/168212688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1ce888490 con 0x55b1ce9de300 2023-09-10T01:11:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa36ff5700 1 -- 172.21.15.44:0/168212688 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 9452+0+0 (secure 0 0 0) 0x7faa4400b970 con 0x55b1ce9de300 2023-09-10T01:11:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa36ff5700 1 -- 172.21.15.44:0/168212688 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa44005940 con 0x55b1ce9de300 2023-09-10T01:11:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa65600b80 10 client.28762 fetch_fsmap learned FSMap version 120 2023-09-10T01:11:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.761+0000 7faa65600b80 1 -- 172.21.15.44:0/168212688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x7faa541a2530 con 0x55b1ce9de300 2023-09-10T01:11:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.762+0000 7faa36ff5700 1 -- 172.21.15.44:0/168212688 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b1ce888490 con 0x55b1ce9de300 2023-09-10T01:11:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.762+0000 7faa36ff5700 1 -- 172.21.15.44:0/168212688 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7faa44039070 con 0x55b1ce9de300 2023-09-10T01:11:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.762+0000 7faa65600b80 10 client.28762 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:11:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.762+0000 7faa65600b80 10 client.28762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:11:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.762+0000 7faa65600b80 20 client.28762 populate_metadata read hostname 'smithi044' 2023-09-10T01:11:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.762+0000 7faa65600b80 1 --2- 172.21.15.44:0/168212688 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b1ce9f8320 0x55b1ce9fa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.762+0000 7faa65600b80 4 client.28762 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:11:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.762+0000 7faa65600b80 1 -- 172.21.15.44:0/168212688 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1ce945840 con 0x55b1ce9f8320 2023-09-10T01:11:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.762+0000 7faa527fc700 1 --2- 172.21.15.44:0/168212688 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b1ce9f8320 0x55b1ce9fa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.763+0000 7faa36ff5700 10 client.28762 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.763+0000 7faa527fc700 1 --2- 172.21.15.44:0/168212688 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b1ce9f8320 0x55b1ce9fa740 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ce9ed1a6ff5753f server_cookie=fea57a77c984a607 in_seq=0 out_seq=0 2023-09-10T01:11:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.763+0000 7faa36ff5700 1 -- 172.21.15.44:0/168212688 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b1ce945840 con 0x55b1ce9f8320 2023-09-10T01:11:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.763+0000 7faa36ff5700 10 client.28762 handle_command_reply: tid=0 2023-09-10T01:11:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.763+0000 7faa65600b80 1 client.28762 shutdown 2023-09-10T01:11:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.764+0000 7faa65600b80 1 -- 172.21.15.44:0/168212688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9de300 msgr2=0x55b1ce9ede20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.764+0000 7faa65600b80 1 --2- 172.21.15.44:0/168212688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9de300 0x55b1ce9ede20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faa4400f6e0 tx=0x7faa44009c30 comp rx=0 tx=0).stop 2023-09-10T01:11:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.764+0000 7faa65600b80 1 -- 172.21.15.44:0/168212688 shutdown_connections 2023-09-10T01:11:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.764+0000 7faa65600b80 1 --2- 172.21.15.44:0/168212688 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b1ce9f8320 0x55b1ce9fa740 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.764+0000 7faa65600b80 1 --2- 172.21.15.44:0/168212688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 0x55b1ce9e80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.764+0000 7faa36ff5700 0 client.28762 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.764+0000 7faa65600b80 1 --2- 172.21.15.44:0/168212688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9de300 0x55b1ce9ede20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.764+0000 7faa65600b80 1 --2- 172.21.15.44:0/168212688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9decc0 0x55b1ce9ee360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.764+0000 7faa65600b80 1 -- 172.21.15.44:0/168212688 >> 172.21.15.44:0/168212688 conn(0x55b1ce953030 msgr2=0x55b1ce9d87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.764+0000 7faa65600b80 1 -- 172.21.15.44:0/168212688 shutdown_connections 2023-09-10T01:11:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.764+0000 7faa65600b80 1 -- 172.21.15.44:0/168212688 wait complete. 2023-09-10T01:11:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.764+0000 7faa65600b80 20 client.28762 trim_cache size 0 max 16384 2023-09-10T01:11:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.782+0000 7faa65600b80 1 Processor -- start 2023-09-10T01:11:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.782+0000 7faa65600b80 1 -- start start 2023-09-10T01:11:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.782+0000 7faa65600b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9de300 0x55b1ce9e5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa65600b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9decc0 0x55b1ce9e7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa65600b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 0x55b1ce9e8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa65600b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce9e1e40 con 0x55b1ce9de300 2023-09-10T01:11:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa65600b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce9e8600 con 0x55b1ce9e41a0 2023-09-10T01:11:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa65600b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ce9d0ed0 con 0x55b1ce9decc0 2023-09-10T01:11:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa52ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9de300 0x55b1ce9e5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa527fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9decc0 0x55b1ce9e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa52ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9de300 0x55b1ce9e5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56806/0 (socket says 172.21.15.44:56806) 2023-09-10T01:11:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa52ffd700 1 -- 172.21.15.44:0/3822307589 learned_addr learned my addr 172.21.15.44:0/3822307589 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa537fe700 1 --2- 172.21.15.44:0/3822307589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 0x55b1ce9e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa527fc700 1 -- 172.21.15.44:0/3822307589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 msgr2=0x55b1ce9e8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa527fc700 1 --2- 172.21.15.44:0/3822307589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 0x55b1ce9e8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa527fc700 1 -- 172.21.15.44:0/3822307589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9de300 msgr2=0x55b1ce9e5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa527fc700 1 --2- 172.21.15.44:0/3822307589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9de300 0x55b1ce9e5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa527fc700 1 -- 172.21.15.44:0/3822307589 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1ce745c00 con 0x55b1ce9decc0 2023-09-10T01:11:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa527fc700 1 --2- 172.21.15.44:0/3822307589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9decc0 0x55b1ce9e7370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55b1ce9fb030 tx=0x7faa48009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c03b831a054d8930 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.783+0000 7faa39ffb700 1 -- 172.21.15.44:0/3822307589 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4801b040 con 0x55b1ce9decc0 2023-09-10T01:11:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.784+0000 7faa39ffb700 1 -- 172.21.15.44:0/3822307589 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa4801c040 con 0x55b1ce9decc0 2023-09-10T01:11:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.784+0000 7faa39ffb700 1 -- 172.21.15.44:0/3822307589 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa48016370 con 0x55b1ce9decc0 2023-09-10T01:11:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.784+0000 7faa65600b80 1 -- 172.21.15.44:0/3822307589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9decc0 msgr2=0x55b1ce9e7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.784+0000 7faa65600b80 1 --2- 172.21.15.44:0/3822307589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9decc0 0x55b1ce9e7370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55b1ce9fb030 tx=0x7faa48009310 comp rx=0 tx=0).stop 2023-09-10T01:11:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.784+0000 7faa65600b80 1 -- 172.21.15.44:0/3822307589 shutdown_connections 2023-09-10T01:11:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.784+0000 7faa65600b80 1 --2- 172.21.15.44:0/3822307589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9e41a0 0x55b1ce9e8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.784+0000 7faa65600b80 1 --2- 172.21.15.44:0/3822307589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9de300 0x55b1ce9e5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.784+0000 7faa65600b80 1 --2- 172.21.15.44:0/3822307589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9decc0 0x55b1ce9e7370 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.784+0000 7faa65600b80 1 -- 172.21.15.44:0/3822307589 >> 172.21.15.44:0/3822307589 conn(0x55b1ce953030 msgr2=0x55b1ce9e7980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.784+0000 7faa65600b80 1 -- 172.21.15.44:0/3822307589 shutdown_connections 2023-09-10T01:11:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.784+0000 7faa65600b80 1 -- 172.21.15.44:0/3822307589 wait complete. 2023-09-10T01:11:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.785+0000 7faa65600b80 1 -- start start 2023-09-10T01:11:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.785+0000 7faa65600b80 1 Processor -- start 2023-09-10T01:11:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.785+0000 7faa65600b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9de300 0x55b1cea45a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.785+0000 7faa65600b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9decc0 0x55b1cea45fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.785+0000 7faa65600b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9e41a0 0x55b1cea400c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.785+0000 7faa65600b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1cea407a0 con 0x55b1ce9de300 2023-09-10T01:11:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.785+0000 7faa65600b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1cea40910 con 0x55b1ce9decc0 2023-09-10T01:11:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.785+0000 7faa65600b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1cea40ad0 con 0x55b1ce9e41a0 2023-09-10T01:11:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.785+0000 7faa65600b80 10 client.0 _ll_register_callbacks cb 0x55b1ce818a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:11:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.785+0000 7faa52ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9de300 0x55b1cea45a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.785+0000 7faa52ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9de300 0x55b1cea45a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56812/0 (socket says 172.21.15.44:56812) 2023-09-10T01:11:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.785+0000 7faa527fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9decc0 0x55b1cea45fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.785+0000 7faa52ffd700 1 -- 172.21.15.44:0/1588258669 learned_addr learned my addr 172.21.15.44:0/1588258669 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.785+0000 7faa537fe700 1 --2- 172.21.15.44:0/1588258669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9e41a0 0x55b1cea400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.786+0000 7faa527fc700 1 -- 172.21.15.44:0/1588258669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9e41a0 msgr2=0x55b1cea400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.786+0000 7faa527fc700 1 --2- 172.21.15.44:0/1588258669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9e41a0 0x55b1cea400c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.786+0000 7faa527fc700 1 -- 172.21.15.44:0/1588258669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9de300 msgr2=0x55b1cea45a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.786+0000 7faa527fc700 1 --2- 172.21.15.44:0/1588258669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9de300 0x55b1cea45a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.786+0000 7faa527fc700 1 -- 172.21.15.44:0/1588258669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa48016be0 con 0x55b1ce9decc0 2023-09-10T01:11:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.786+0000 7faa527fc700 1 --2- 172.21.15.44:0/1588258669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9decc0 0x55b1cea45fa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55b1ce9e97d0 tx=0x7faa480255b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcf8263d1b4f554e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.786+0000 7faa36ff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:11:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.786+0000 7faa36ff5700 1 -- 172.21.15.44:0/1588258669 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa48004380 con 0x55b1ce9decc0 2023-09-10T01:11:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.786+0000 7faa65600b80 1 -- 172.21.15.44:0/1588258669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa4c098f50 con 0x55b1ce9decc0 2023-09-10T01:11:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.786+0000 7faa36ff5700 1 -- 172.21.15.44:0/1588258669 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 9452+0+0 (secure 0 0 0) 0x7faa480044e0 con 0x55b1ce9decc0 2023-09-10T01:11:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.786+0000 7faa36ff5700 1 -- 172.21.15.44:0/1588258669 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa48025020 con 0x55b1ce9decc0 2023-09-10T01:11:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.786+0000 7faa65600b80 10 client.28829 fetch_fsmap learned FSMap version 120 2023-09-10T01:11:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.786+0000 7faa65600b80 1 -- 172.21.15.44:0/1588258669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x55b1ce7831f0 con 0x55b1ce9decc0 2023-09-10T01:11:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.786+0000 7faa36ff5700 1 -- 172.21.15.44:0/1588258669 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa4c098f50 con 0x55b1ce9decc0 2023-09-10T01:11:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.786+0000 7faa36ff5700 1 -- 172.21.15.44:0/1588258669 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7faa48014ad0 con 0x55b1ce9decc0 2023-09-10T01:11:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.787+0000 7faa65600b80 10 client.28829 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:11:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.787+0000 7faa65600b80 10 client.28829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:11:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.787+0000 7faa65600b80 20 client.28829 populate_metadata read hostname 'smithi044' 2023-09-10T01:11:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.787+0000 7faa65600b80 1 --2- 172.21.15.44:0/1588258669 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b1ce9f8320 0x55b1cea4f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.787+0000 7faa65600b80 4 client.28829 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:11:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.787+0000 7faa65600b80 1 -- 172.21.15.44:0/1588258669 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b1ce9298c0 con 0x55b1ce9f8320 2023-09-10T01:11:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.787+0000 7faa52ffd700 1 --2- 172.21.15.44:0/1588258669 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b1ce9f8320 0x55b1cea4f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.787+0000 7faa36ff5700 10 client.28829 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.787+0000 7faa52ffd700 1 --2- 172.21.15.44:0/1588258669 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b1ce9f8320 0x55b1cea4f730 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba38876fc1a84295 server_cookie=68b12ab0ec109dcc in_seq=0 out_seq=0 2023-09-10T01:11:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.802+0000 7faa36ff5700 1 -- 172.21.15.44:0/1588258669 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b1ce9298c0 con 0x55b1ce9f8320 2023-09-10T01:11:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.802+0000 7faa36ff5700 10 client.28829 handle_command_reply: tid=0 2023-09-10T01:11:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.802+0000 7faa65600b80 1 client.28829 shutdown 2023-09-10T01:11:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.802+0000 7faa65600b80 1 -- 172.21.15.44:0/1588258669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9decc0 msgr2=0x55b1cea45fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.802+0000 7faa65600b80 1 --2- 172.21.15.44:0/1588258669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9decc0 0x55b1cea45fa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55b1ce9e97d0 tx=0x7faa480255b0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.802+0000 7faa65600b80 1 -- 172.21.15.44:0/1588258669 shutdown_connections 2023-09-10T01:11:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.802+0000 7faa65600b80 1 --2- 172.21.15.44:0/1588258669 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b1ce9f8320 0x55b1cea4f730 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.802+0000 7faa65600b80 1 --2- 172.21.15.44:0/1588258669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1ce9e41a0 0x55b1cea400c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.802+0000 7faa65600b80 1 --2- 172.21.15.44:0/1588258669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b1ce9de300 0x55b1cea45a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.802+0000 7faa36ff5700 0 client.28829 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.802+0000 7faa65600b80 1 --2- 172.21.15.44:0/1588258669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1ce9decc0 0x55b1cea45fa0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.802+0000 7faa65600b80 1 -- 172.21.15.44:0/1588258669 >> 172.21.15.44:0/1588258669 conn(0x55b1ce953030 msgr2=0x55b1ce9d7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.802+0000 7faa65600b80 1 -- 172.21.15.44:0/1588258669 shutdown_connections 2023-09-10T01:11:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.802+0000 7faa65600b80 1 -- 172.21.15.44:0/1588258669 wait complete. 2023-09-10T01:11:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.802+0000 7faa65600b80 20 client.28829 trim_cache size 0 max 16384 2023-09-10T01:11:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:11:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:11:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:11:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:11:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.803+0000 7faa5943a700 1 -- 172.21.15.44:0/3431755200 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faa3c0a0830 msgr2=0x7faa3c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.803+0000 7faa5943a700 1 --2- 172.21.15.44:0/3431755200 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faa3c0a0830 0x7faa3c0a2cf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faa440098f0 tx=0x7faa44019040 comp rx=0 tx=0).stop 2023-09-10T01:11:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.803+0000 7faa5943a700 1 -- 172.21.15.44:0/3431755200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa541055b0 msgr2=0x7faa54198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.803+0000 7faa5943a700 1 --2- 172.21.15.44:0/3431755200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa541055b0 0x7faa54198fc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00ef30 tx=0x7faa4c00c3a0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.803+0000 7faa5943a700 1 -- 172.21.15.44:0/3431755200 shutdown_connections 2023-09-10T01:11:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.803+0000 7faa5943a700 1 --2- 172.21.15.44:0/3431755200 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faa3c0a0830 0x7faa3c0a2cf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.803+0000 7faa5943a700 1 --2- 172.21.15.44:0/3431755200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa541055b0 0x7faa54198fc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.803+0000 7faa5943a700 1 --2- 172.21.15.44:0/3431755200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa540ff760 0x7faa541954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.803+0000 7faa5943a700 1 --2- 172.21.15.44:0/3431755200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa54100120 0x7faa54195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.803+0000 7faa5943a700 1 -- 172.21.15.44:0/3431755200 >> 172.21.15.44:0/3431755200 conn(0x7faa54074f30 msgr2=0x7faa540fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.803+0000 7faa5943a700 1 -- 172.21.15.44:0/3431755200 shutdown_connections 2023-09-10T01:11:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:29.803+0000 7faa5943a700 1 -- 172.21.15.44:0/3431755200 wait complete. 2023-09-10T01:11:30.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:29 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:11:30.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:29 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:11:30.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:29 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:11:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:30 smithi131 ceph-mon[103713]: pgmap v982: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2023-09-10T01:11:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:30 smithi044 ceph-mon[98676]: pgmap v982: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2023-09-10T01:11:31.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:30 smithi099 ceph-mon[102692]: pgmap v982: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2023-09-10T01:11:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:32 smithi131 ceph-mon[103713]: pgmap v983: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 236 op/s 2023-09-10T01:11:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:32 smithi044 ceph-mon[98676]: pgmap v983: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 236 op/s 2023-09-10T01:11:33.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:32 smithi099 ceph-mon[102692]: pgmap v983: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 236 op/s 2023-09-10T01:11:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:35 smithi131 ceph-mon[103713]: pgmap v984: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 152 op/s 2023-09-10T01:11:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:35 smithi044 ceph-mon[98676]: pgmap v984: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 152 op/s 2023-09-10T01:11:35.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:35 smithi099 ceph-mon[102692]: pgmap v984: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 152 op/s 2023-09-10T01:11:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:37 smithi131 ceph-mon[103713]: pgmap v985: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 226 op/s 2023-09-10T01:11:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:37 smithi044 ceph-mon[98676]: pgmap v985: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 226 op/s 2023-09-10T01:11:37.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:37 smithi099 ceph-mon[102692]: pgmap v985: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 226 op/s 2023-09-10T01:11:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:39 smithi131 ceph-mon[103713]: pgmap v986: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 148 op/s 2023-09-10T01:11:39.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:39 smithi044 ceph-mon[98676]: pgmap v986: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 148 op/s 2023-09-10T01:11:39.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:39 smithi099 ceph-mon[102692]: pgmap v986: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 148 op/s 2023-09-10T01:11:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:41 smithi131 ceph-mon[103713]: pgmap v987: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 148 op/s 2023-09-10T01:11:41.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:41 smithi044 ceph-mon[98676]: pgmap v987: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 148 op/s 2023-09-10T01:11:41.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:41 smithi099 ceph-mon[102692]: pgmap v987: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 148 op/s 2023-09-10T01:11:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.923+0000 7f6388017700 1 -- 172.21.15.44:0/4126385523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f638010d180 msgr2=0x7f638010f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.923+0000 7f6388017700 1 --2- 172.21.15.44:0/4126385523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f638010d180 0x7f638010f570 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f637800b0a0 tx=0x7f637800b3b0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.923+0000 7f6388017700 1 -- 172.21.15.44:0/4126385523 shutdown_connections 2023-09-10T01:11:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.923+0000 7f6388017700 1 --2- 172.21.15.44:0/4126385523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f638010d180 0x7f638010f570 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.923+0000 7f6388017700 1 --2- 172.21.15.44:0/4126385523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6380101490 0x7f63800fea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.923+0000 7f6388017700 1 --2- 172.21.15.44:0/4126385523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6380101900 0x7f63800fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.923+0000 7f6388017700 1 -- 172.21.15.44:0/4126385523 >> 172.21.15.44:0/4126385523 conn(0x7f63800769d0 msgr2=0x7f63800fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.923+0000 7f6388017700 1 -- 172.21.15.44:0/4126385523 shutdown_connections 2023-09-10T01:11:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.923+0000 7f6388017700 1 -- 172.21.15.44:0/4126385523 wait complete. 2023-09-10T01:11:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.923+0000 7f6388017700 1 Processor -- start 2023-09-10T01:11:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f6388017700 1 -- start start 2023-09-10T01:11:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f6388017700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6380101490 0x7f638019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f6388017700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6380101900 0x7f638019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f6388017700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f638010d180 0x7f63801986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f6388017700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6380198e90 con 0x7f6380101490 2023-09-10T01:11:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f6388017700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6380199000 con 0x7f638010d180 2023-09-10T01:11:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f6388017700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63801991c0 con 0x7f6380101900 2023-09-10T01:11:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f6385db3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6380101490 0x7f638019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f6385db3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6380101490 0x7f638019e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56826/0 (socket says 172.21.15.44:56826) 2023-09-10T01:11:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f6385db3700 1 -- 172.21.15.44:0/414619421 learned_addr learned my addr 172.21.15.44:0/414619421 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f63855b2700 1 --2- 172.21.15.44:0/414619421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6380101900 0x7f638019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f63865b4700 1 --2- 172.21.15.44:0/414619421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f638010d180 0x7f63801986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f6385db3700 1 -- 172.21.15.44:0/414619421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6380101900 msgr2=0x7f638019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f6385db3700 1 --2- 172.21.15.44:0/414619421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6380101900 0x7f638019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f6385db3700 1 -- 172.21.15.44:0/414619421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f638010d180 msgr2=0x7f63801986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f6385db3700 1 --2- 172.21.15.44:0/414619421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f638010d180 0x7f63801986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.924+0000 7f6385db3700 1 -- 172.21.15.44:0/414619421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6378009d00 con 0x7f6380101490 2023-09-10T01:11:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.925+0000 7f6385db3700 1 --2- 172.21.15.44:0/414619421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6380101490 0x7f638019e8f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f637c00e9a0 tx=0x7f637c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.925+0000 7f6376ffd700 1 -- 172.21.15.44:0/414619421 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c009790 con 0x7f6380101490 2023-09-10T01:11:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.925+0000 7f6388017700 1 -- 172.21.15.44:0/414619421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63801994a0 con 0x7f6380101490 2023-09-10T01:11:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.925+0000 7f6376ffd700 1 -- 172.21.15.44:0/414619421 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f637c010430 con 0x7f6380101490 2023-09-10T01:11:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.925+0000 7f6388017700 1 -- 172.21.15.44:0/414619421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63801a58c0 con 0x7f6380101490 2023-09-10T01:11:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.925+0000 7f6376ffd700 1 -- 172.21.15.44:0/414619421 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c00cea0 con 0x7f6380101490 2023-09-10T01:11:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.927+0000 7f6376ffd700 1 -- 172.21.15.44:0/414619421 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f637c0105a0 con 0x7f6380101490 2023-09-10T01:11:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.928+0000 7f6376ffd700 1 --2- 172.21.15.44:0/414619421 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f636c0a0830 0x7f636c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.928+0000 7f6376ffd700 1 -- 172.21.15.44:0/414619421 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f637c014070 con 0x7f6380101490 2023-09-10T01:11:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.928+0000 7f63855b2700 1 --2- 172.21.15.44:0/414619421 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f636c0a0830 0x7f636c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f63941ddb80 1 Processor -- start 2023-09-10T01:11:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f63855b2700 1 --2- 172.21.15.44:0/414619421 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f636c0a0830 0x7f636c0a2cf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f638019a100 tx=0x7f637000b410 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f63941ddb80 1 -- start start 2023-09-10T01:11:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f63941ddb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec8e1a0 0x56395ec8e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f63941ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec88300 0x56395ec88780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f63941ddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec88cc0 0x56395ec94f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f63941ddb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395ebe16d0 con 0x56395ec8e1a0 2023-09-10T01:11:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f63941ddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395ebdeff0 con 0x56395ec88cc0 2023-09-10T01:11:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f63941ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395ebda8e0 con 0x56395ec88300 2023-09-10T01:11:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f6385db3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec8e1a0 0x56395ec8e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f6385db3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec8e1a0 0x56395ec8e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56834/0 (socket says 172.21.15.44:56834) 2023-09-10T01:11:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f6385db3700 1 -- 172.21.15.44:0/4139442803 learned_addr learned my addr 172.21.15.44:0/4139442803 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f63855b2700 1 --2- 172.21.15.44:0/4139442803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec88300 0x56395ec88780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f63865b4700 1 --2- 172.21.15.44:0/4139442803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec88cc0 0x56395ec94f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f6385db3700 1 -- 172.21.15.44:0/4139442803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec88300 msgr2=0x56395ec88780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f6385db3700 1 --2- 172.21.15.44:0/4139442803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec88300 0x56395ec88780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f6385db3700 1 -- 172.21.15.44:0/4139442803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec88cc0 msgr2=0x56395ec94f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f6385db3700 1 --2- 172.21.15.44:0/4139442803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec88cc0 0x56395ec94f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.929+0000 7f6385db3700 1 -- 172.21.15.44:0/4139442803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56395ea06d80 con 0x56395ec8e1a0 2023-09-10T01:11:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.930+0000 7f6385db3700 1 --2- 172.21.15.44:0/4139442803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec8e1a0 0x56395ec8e580 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f637c0a10b0 tx=0x7f637c098ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b250385c986da3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.930+0000 7f636bfff700 1 -- 172.21.15.44:0/4139442803 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c09d170 con 0x56395ec8e1a0 2023-09-10T01:11:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.930+0000 7f636bfff700 1 -- 172.21.15.44:0/4139442803 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f637c099570 con 0x56395ec8e1a0 2023-09-10T01:11:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.930+0000 7f636bfff700 1 -- 172.21.15.44:0/4139442803 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c0997d0 con 0x56395ec8e1a0 2023-09-10T01:11:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.931+0000 7f63941ddb80 1 -- 172.21.15.44:0/4139442803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec8e1a0 msgr2=0x56395ec8e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.931+0000 7f63941ddb80 1 --2- 172.21.15.44:0/4139442803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec8e1a0 0x56395ec8e580 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f637c0a10b0 tx=0x7f637c098ed0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.931+0000 7f63941ddb80 1 -- 172.21.15.44:0/4139442803 shutdown_connections 2023-09-10T01:11:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.931+0000 7f63941ddb80 1 --2- 172.21.15.44:0/4139442803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec88cc0 0x56395ec94f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.931+0000 7f63941ddb80 1 --2- 172.21.15.44:0/4139442803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec8e1a0 0x56395ec8e580 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.931+0000 7f63941ddb80 1 --2- 172.21.15.44:0/4139442803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec88300 0x56395ec88780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.931+0000 7f63941ddb80 1 -- 172.21.15.44:0/4139442803 >> 172.21.15.44:0/4139442803 conn(0x56395ebfd030 msgr2=0x56395ec83560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.931+0000 7f63941ddb80 1 -- 172.21.15.44:0/4139442803 shutdown_connections 2023-09-10T01:11:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.931+0000 7f63941ddb80 1 -- 172.21.15.44:0/4139442803 wait complete. 2023-09-10T01:11:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.931+0000 7f63941ddb80 1 -- start start 2023-09-10T01:11:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.931+0000 7f63941ddb80 1 Processor -- start 2023-09-10T01:11:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f63941ddb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88300 0x56395ec94f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f63941ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec88cc0 0x56395ec92180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f63941ddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec8e1a0 0x56395ec928c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f63941ddb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395eb78b10 con 0x56395ec88300 2023-09-10T01:11:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f63941ddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395eb78810 con 0x56395ec8e1a0 2023-09-10T01:11:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f63941ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395ea5c8c0 con 0x56395ec88cc0 2023-09-10T01:11:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f63941ddb80 10 client.0 _ll_register_callbacks cb 0x56395eac2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:11:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f6385db3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88300 0x56395ec94f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f6385db3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88300 0x56395ec94f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56840/0 (socket says 172.21.15.44:56840) 2023-09-10T01:11:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f6385db3700 1 -- 172.21.15.44:0/3295628535 learned_addr learned my addr 172.21.15.44:0/3295628535 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f63855b2700 1 --2- 172.21.15.44:0/3295628535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec88cc0 0x56395ec92180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f63865b4700 1 --2- 172.21.15.44:0/3295628535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec8e1a0 0x56395ec928c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f6385db3700 1 -- 172.21.15.44:0/3295628535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec88cc0 msgr2=0x56395ec92180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f6385db3700 1 --2- 172.21.15.44:0/3295628535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec88cc0 0x56395ec92180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f6385db3700 1 -- 172.21.15.44:0/3295628535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec8e1a0 msgr2=0x56395ec928c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f6385db3700 1 --2- 172.21.15.44:0/3295628535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec8e1a0 0x56395ec928c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f6385db3700 1 -- 172.21.15.44:0/3295628535 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63801a58c0 con 0x56395ec88300 2023-09-10T01:11:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f6385db3700 1 --2- 172.21.15.44:0/3295628535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88300 0x56395ec94f00 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f637c0c9d90 tx=0x7f637c0cc3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b434ac670c2a5d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f63697fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:11:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f63697fa700 1 -- 172.21.15.44:0/3295628535 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c0aeea0 con 0x56395ec88300 2023-09-10T01:11:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f63697fa700 1 -- 172.21.15.44:0/3295628535 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f637c09d170 con 0x56395ec88300 2023-09-10T01:11:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.932+0000 7f63941ddb80 1 -- 172.21.15.44:0/3295628535 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56395eb32490 con 0x56395ec88300 2023-09-10T01:11:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.933+0000 7f63697fa700 1 -- 172.21.15.44:0/3295628535 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f637c09d3a0 con 0x56395ec88300 2023-09-10T01:11:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.933+0000 7f63941ddb80 10 client.19167 fetch_fsmap learned FSMap version 120 2023-09-10T01:11:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.933+0000 7f63941ddb80 1 -- 172.21.15.44:0/3295628535 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x56395e9efc00 con 0x56395ec88300 2023-09-10T01:11:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.933+0000 7f63697fa700 1 -- 172.21.15.44:0/3295628535 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56395eb32490 con 0x56395ec88300 2023-09-10T01:11:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.933+0000 7f63697fa700 1 -- 172.21.15.44:0/3295628535 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f637c0a1150 con 0x56395ec88300 2023-09-10T01:11:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.933+0000 7f63941ddb80 10 client.19167 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:11:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.933+0000 7f63941ddb80 10 client.19167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:11:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.933+0000 7f63941ddb80 20 client.19167 populate_metadata read hostname 'smithi044' 2023-09-10T01:11:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.933+0000 7f63941ddb80 1 --2- 172.21.15.44:0/3295628535 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56395eca1f40 0x56395eca4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.933+0000 7f63941ddb80 4 client.19167 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:11:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.933+0000 7f63941ddb80 1 -- 172.21.15.44:0/3295628535 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56395ebef840 con 0x56395eca1f40 2023-09-10T01:11:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.933+0000 7f63855b2700 1 --2- 172.21.15.44:0/3295628535 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56395eca1f40 0x56395eca4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.934+0000 7f63697fa700 10 client.19167 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.934+0000 7f63855b2700 1 --2- 172.21.15.44:0/3295628535 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56395eca1f40 0x56395eca4330 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17a20735be871181 server_cookie=ec4966d1dcd8f336 in_seq=0 out_seq=0 2023-09-10T01:11:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.934+0000 7f63697fa700 1 -- 172.21.15.44:0/3295628535 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56395ebef840 con 0x56395eca1f40 2023-09-10T01:11:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.934+0000 7f63697fa700 10 client.19167 handle_command_reply: tid=0 2023-09-10T01:11:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.934+0000 7f63941ddb80 1 client.19167 shutdown 2023-09-10T01:11:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.934+0000 7f63941ddb80 1 -- 172.21.15.44:0/3295628535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88300 msgr2=0x56395ec94f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.934+0000 7f63941ddb80 1 --2- 172.21.15.44:0/3295628535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88300 0x56395ec94f00 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f637c0c9d90 tx=0x7f637c0cc3e0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.935+0000 7f63941ddb80 1 -- 172.21.15.44:0/3295628535 shutdown_connections 2023-09-10T01:11:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.935+0000 7f63941ddb80 1 --2- 172.21.15.44:0/3295628535 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56395eca1f40 0x56395eca4330 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.935+0000 7f63941ddb80 1 --2- 172.21.15.44:0/3295628535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec8e1a0 0x56395ec928c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.935+0000 7f63941ddb80 1 --2- 172.21.15.44:0/3295628535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88300 0x56395ec94f00 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.935+0000 7f63941ddb80 1 --2- 172.21.15.44:0/3295628535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec88cc0 0x56395ec92180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.935+0000 7f63941ddb80 1 -- 172.21.15.44:0/3295628535 >> 172.21.15.44:0/3295628535 conn(0x56395ebfd030 msgr2=0x56395ec827a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.935+0000 7f63697fa700 0 client.19167 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.935+0000 7f63941ddb80 1 -- 172.21.15.44:0/3295628535 shutdown_connections 2023-09-10T01:11:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.935+0000 7f63941ddb80 1 -- 172.21.15.44:0/3295628535 wait complete. 2023-09-10T01:11:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.935+0000 7f63941ddb80 20 client.19167 trim_cache size 0 max 16384 2023-09-10T01:11:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.953+0000 7f63941ddb80 1 Processor -- start 2023-09-10T01:11:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.953+0000 7f63941ddb80 1 -- start start 2023-09-10T01:11:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.953+0000 7f63941ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec88300 0x56395ec8fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.953+0000 7f63941ddb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88cc0 0x56395ec94aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.953+0000 7f63941ddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec8e1a0 0x56395ec83090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.953+0000 7f63941ddb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395ec8bdb0 con 0x56395ec88cc0 2023-09-10T01:11:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.953+0000 7f63941ddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395ec8de00 con 0x56395ec8e1a0 2023-09-10T01:11:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.953+0000 7f63941ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395ec93760 con 0x56395ec88300 2023-09-10T01:11:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.953+0000 7f63855b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88cc0 0x56395ec94aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.953+0000 7f6385db3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec88300 0x56395ec8fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.953+0000 7f63855b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88cc0 0x56395ec94aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56850/0 (socket says 172.21.15.44:56850) 2023-09-10T01:11:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.953+0000 7f63855b2700 1 -- 172.21.15.44:0/1529355819 learned_addr learned my addr 172.21.15.44:0/1529355819 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.953+0000 7f63865b4700 1 --2- 172.21.15.44:0/1529355819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec8e1a0 0x56395ec83090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.954+0000 7f63855b2700 1 -- 172.21.15.44:0/1529355819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec88300 msgr2=0x56395ec8fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.954+0000 7f63855b2700 1 --2- 172.21.15.44:0/1529355819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec88300 0x56395ec8fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.954+0000 7f63855b2700 1 -- 172.21.15.44:0/1529355819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec8e1a0 msgr2=0x56395ec83090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.954+0000 7f63855b2700 1 --2- 172.21.15.44:0/1529355819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec8e1a0 0x56395ec83090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.954+0000 7f63855b2700 1 -- 172.21.15.44:0/1529355819 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56395ea2d1f0 con 0x56395ec88cc0 2023-09-10T01:11:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.954+0000 7f63855b2700 1 --2- 172.21.15.44:0/1529355819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88cc0 0x56395ec94aa0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f6380072cf0 tx=0x7f6370013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee59f0766c0fec61 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.954+0000 7f6374ff9700 1 -- 172.21.15.44:0/1529355819 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370024040 con 0x56395ec88cc0 2023-09-10T01:11:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.954+0000 7f6374ff9700 1 -- 172.21.15.44:0/1529355819 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6370015070 con 0x56395ec88cc0 2023-09-10T01:11:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.954+0000 7f6374ff9700 1 -- 172.21.15.44:0/1529355819 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370021b90 con 0x56395ec88cc0 2023-09-10T01:11:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.955+0000 7f63941ddb80 1 -- 172.21.15.44:0/1529355819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88cc0 msgr2=0x56395ec94aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.955+0000 7f63941ddb80 1 --2- 172.21.15.44:0/1529355819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88cc0 0x56395ec94aa0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f6380072cf0 tx=0x7f6370013000 comp rx=0 tx=0).stop 2023-09-10T01:11:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.955+0000 7f63941ddb80 1 -- 172.21.15.44:0/1529355819 shutdown_connections 2023-09-10T01:11:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.955+0000 7f63941ddb80 1 --2- 172.21.15.44:0/1529355819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec8e1a0 0x56395ec83090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.955+0000 7f63941ddb80 1 --2- 172.21.15.44:0/1529355819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec88300 0x56395ec8fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.955+0000 7f63941ddb80 1 --2- 172.21.15.44:0/1529355819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88cc0 0x56395ec94aa0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.955+0000 7f63941ddb80 1 -- 172.21.15.44:0/1529355819 >> 172.21.15.44:0/1529355819 conn(0x56395ebfd030 msgr2=0x56395ec8c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.955+0000 7f63941ddb80 1 -- 172.21.15.44:0/1529355819 shutdown_connections 2023-09-10T01:11:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.955+0000 7f63941ddb80 1 -- 172.21.15.44:0/1529355819 wait complete. 2023-09-10T01:11:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.955+0000 7f63941ddb80 1 -- start start 2023-09-10T01:11:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.955+0000 7f63941ddb80 1 Processor -- start 2023-09-10T01:11:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63941ddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec88300 0x56395ec98e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63941ddb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88cc0 0x56395ec99360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63941ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec8e1a0 0x56395ec97250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63941ddb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395ec978c0 con 0x56395ec88cc0 2023-09-10T01:11:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63941ddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395ec97a30 con 0x56395ec88300 2023-09-10T01:11:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63941ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395ec97bf0 con 0x56395ec8e1a0 2023-09-10T01:11:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63941ddb80 10 client.0 _ll_register_callbacks cb 0x7f637c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:11:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63855b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88cc0 0x56395ec99360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f6385db3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec88300 0x56395ec98e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63855b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88cc0 0x56395ec99360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56856/0 (socket says 172.21.15.44:56856) 2023-09-10T01:11:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63855b2700 1 -- 172.21.15.44:0/2264919603 learned_addr learned my addr 172.21.15.44:0/2264919603 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63865b4700 1 --2- 172.21.15.44:0/2264919603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec8e1a0 0x56395ec97250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63855b2700 1 -- 172.21.15.44:0/2264919603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec8e1a0 msgr2=0x56395ec97250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63855b2700 1 --2- 172.21.15.44:0/2264919603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec8e1a0 0x56395ec97250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63855b2700 1 -- 172.21.15.44:0/2264919603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec88300 msgr2=0x56395ec98e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63855b2700 1 --2- 172.21.15.44:0/2264919603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec88300 0x56395ec98e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63855b2700 1 -- 172.21.15.44:0/2264919603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6370010d30 con 0x56395ec88cc0 2023-09-10T01:11:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f6385db3700 1 --2- 172.21.15.44:0/2264919603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec88300 0x56395ec98e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:11:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63855b2700 1 --2- 172.21.15.44:0/2264919603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88cc0 0x56395ec99360 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f637002c040 tx=0x7f6370010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3425e916f93cde70 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63697fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:11:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.956+0000 7f63697fa700 1 -- 172.21.15.44:0/2264919603 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370024040 con 0x56395ec88cc0 2023-09-10T01:11:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.957+0000 7f63941ddb80 1 -- 172.21.15.44:0/2264919603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f637c099570 con 0x56395ec88cc0 2023-09-10T01:11:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.957+0000 7f63697fa700 1 -- 172.21.15.44:0/2264919603 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f6370017070 con 0x56395ec88cc0 2023-09-10T01:11:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.957+0000 7f63697fa700 1 -- 172.21.15.44:0/2264919603 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6370029b00 con 0x56395ec88cc0 2023-09-10T01:11:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.957+0000 7f63941ddb80 10 client.19173 fetch_fsmap learned FSMap version 120 2023-09-10T01:11:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.957+0000 7f63941ddb80 1 -- 172.21.15.44:0/2264919603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x56395ea05250 con 0x56395ec88cc0 2023-09-10T01:11:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.957+0000 7f63697fa700 1 -- 172.21.15.44:0/2264919603 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f637c099570 con 0x56395ec88cc0 2023-09-10T01:11:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.957+0000 7f63697fa700 1 -- 172.21.15.44:0/2264919603 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f6370005160 con 0x56395ec88cc0 2023-09-10T01:11:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.957+0000 7f63941ddb80 10 client.19173 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:11:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.957+0000 7f63941ddb80 10 client.19173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:11:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.957+0000 7f63941ddb80 20 client.19173 populate_metadata read hostname 'smithi044' 2023-09-10T01:11:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.957+0000 7f63941ddb80 1 --2- 172.21.15.44:0/2264919603 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56395ecf6d90 0x56395ecf91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.957+0000 7f63941ddb80 4 client.19173 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:11:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.957+0000 7f63941ddb80 1 -- 172.21.15.44:0/2264919603 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56395ebd38c0 con 0x56395ecf6d90 2023-09-10T01:11:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.957+0000 7f6385db3700 1 --2- 172.21.15.44:0/2264919603 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56395ecf6d90 0x56395ecf91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.958+0000 7f63697fa700 10 client.19173 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.958+0000 7f6385db3700 1 --2- 172.21.15.44:0/2264919603 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56395ecf6d90 0x56395ecf91b0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f071e20447976b93 server_cookie=a1e793cdcd95952a in_seq=0 out_seq=0 2023-09-10T01:11:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.970+0000 7f63697fa700 1 -- 172.21.15.44:0/2264919603 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56395ebd38c0 con 0x56395ecf6d90 2023-09-10T01:11:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.970+0000 7f63697fa700 10 client.19173 handle_command_reply: tid=0 2023-09-10T01:11:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.970+0000 7f63941ddb80 1 client.19173 shutdown 2023-09-10T01:11:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.970+0000 7f63941ddb80 1 -- 172.21.15.44:0/2264919603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88cc0 msgr2=0x56395ec99360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.970+0000 7f63941ddb80 1 --2- 172.21.15.44:0/2264919603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88cc0 0x56395ec99360 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f637002c040 tx=0x7f6370010a40 comp rx=0 tx=0).stop 2023-09-10T01:11:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.970+0000 7f63941ddb80 1 -- 172.21.15.44:0/2264919603 shutdown_connections 2023-09-10T01:11:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.970+0000 7f63941ddb80 1 --2- 172.21.15.44:0/2264919603 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56395ecf6d90 0x56395ecf91b0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.970+0000 7f63941ddb80 1 --2- 172.21.15.44:0/2264919603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56395ec8e1a0 0x56395ec97250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.970+0000 7f63697fa700 0 client.19173 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.970+0000 7f63941ddb80 1 --2- 172.21.15.44:0/2264919603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56395ec88300 0x56395ec98e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.970+0000 7f63941ddb80 1 --2- 172.21.15.44:0/2264919603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56395ec88cc0 0x56395ec99360 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.970+0000 7f63941ddb80 1 -- 172.21.15.44:0/2264919603 >> 172.21.15.44:0/2264919603 conn(0x56395ebfd030 msgr2=0x56395ec82fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.970+0000 7f63941ddb80 1 -- 172.21.15.44:0/2264919603 shutdown_connections 2023-09-10T01:11:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.970+0000 7f63941ddb80 1 -- 172.21.15.44:0/2264919603 wait complete. 2023-09-10T01:11:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.970+0000 7f63941ddb80 20 client.19173 trim_cache size 0 max 16384 2023-09-10T01:11:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:11:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:11:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:11:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:11:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.971+0000 7f6388017700 1 -- 172.21.15.44:0/414619421 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f636c0a0830 msgr2=0x7f636c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.971+0000 7f6388017700 1 --2- 172.21.15.44:0/414619421 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f636c0a0830 0x7f636c0a2cf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f638019a100 tx=0x7f637000b410 comp rx=0 tx=0).stop 2023-09-10T01:11:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.971+0000 7f6388017700 1 -- 172.21.15.44:0/414619421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6380101490 msgr2=0x7f638019e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.971+0000 7f6388017700 1 --2- 172.21.15.44:0/414619421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6380101490 0x7f638019e8f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f637c00e9a0 tx=0x7f637c00ecb0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.971+0000 7f6388017700 1 -- 172.21.15.44:0/414619421 shutdown_connections 2023-09-10T01:11:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.971+0000 7f6388017700 1 --2- 172.21.15.44:0/414619421 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f636c0a0830 0x7f636c0a2cf0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.971+0000 7f6388017700 1 --2- 172.21.15.44:0/414619421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f638010d180 0x7f63801986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.971+0000 7f6388017700 1 --2- 172.21.15.44:0/414619421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6380101490 0x7f638019e8f0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.971+0000 7f6388017700 1 --2- 172.21.15.44:0/414619421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6380101900 0x7f638019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.971+0000 7f6388017700 1 -- 172.21.15.44:0/414619421 >> 172.21.15.44:0/414619421 conn(0x7f63800769d0 msgr2=0x7f63800fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.971+0000 7f6388017700 1 -- 172.21.15.44:0/414619421 shutdown_connections 2023-09-10T01:11:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:42.971+0000 7f6388017700 1 -- 172.21.15.44:0/414619421 wait complete. 2023-09-10T01:11:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:43 smithi131 ceph-mon[103713]: pgmap v988: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 227 op/s 2023-09-10T01:11:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:43 smithi044 ceph-mon[98676]: pgmap v988: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 227 op/s 2023-09-10T01:11:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:43 smithi099 ceph-mon[102692]: pgmap v988: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 227 op/s 2023-09-10T01:11:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:45 smithi131 ceph-mon[103713]: pgmap v989: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 152 op/s 2023-09-10T01:11:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:45 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:11:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:45 smithi044 ceph-mon[98676]: pgmap v989: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 152 op/s 2023-09-10T01:11:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:45 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:11:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:45 smithi099 ceph-mon[102692]: pgmap v989: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 152 op/s 2023-09-10T01:11:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:45 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:11:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:47 smithi131 ceph-mon[103713]: pgmap v990: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 219 op/s 2023-09-10T01:11:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:47 smithi044 ceph-mon[98676]: pgmap v990: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 219 op/s 2023-09-10T01:11:47.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:47 smithi099 ceph-mon[102692]: pgmap v990: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 219 op/s 2023-09-10T01:11:48.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: pgmap v991: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2023-09-10T01:11:48.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T01:11:48.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.801 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.801 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:11:48.801 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.801 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.801 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.802 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.802 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.802 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.802 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T01:11:48.802 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T01:11:48.802 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:48 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: pgmap v991: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2023-09-10T01:11:48.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T01:11:48.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:11:48.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T01:11:48.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T01:11:48.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:48 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.862 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: pgmap v991: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2023-09-10T01:11:48.862 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.862 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.862 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.863 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.863 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.863 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.864 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.864 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.864 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T01:11:48.864 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.865 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.865 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:11:48.865 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.866 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.866 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.866 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.866 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.866 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:48.867 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T01:11:48.867 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T01:11:48.867 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:48.867 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:48 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:49.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:49 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:49 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:49 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:49 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:49 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:49 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:49 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:49 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:49 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T01:11:49.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:49 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T01:11:49.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:49 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:11:49.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:49 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:11:49.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:49 smithi131 ceph-mon[103713]: pgmap v992: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2023-09-10T01:11:49.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:49 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:49.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:49 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:49 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:49 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:49 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:49 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:49 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:49 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:49 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:49 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T01:11:49.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:49 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T01:11:49.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:49 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:11:49.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:49 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:11:49.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:49 smithi044 ceph-mon[98676]: pgmap v992: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2023-09-10T01:11:49.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:49 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:49.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:49 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:49 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:49 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:49 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:49 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:49 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:49 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:49 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:11:49.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:49 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T01:11:49.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:49 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T01:11:49.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:49 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:11:49.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:49 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:11:49.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:49 smithi099 ceph-mon[102692]: pgmap v992: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2023-09-10T01:11:49.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:49 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:11:50.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:50 smithi131 ceph-mon[103713]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-09-10T01:11:50.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:50 smithi044 ceph-mon[98676]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-09-10T01:11:50.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:50 smithi099 ceph-mon[102692]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-09-10T01:11:52.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:51 smithi131 ceph-mon[103713]: pgmap v993: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 217 op/s 2023-09-10T01:11:52.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:51 smithi044 ceph-mon[98676]: pgmap v993: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 217 op/s 2023-09-10T01:11:52.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:51 smithi099 ceph-mon[102692]: pgmap v993: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 217 op/s 2023-09-10T01:11:54.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:53 smithi131 ceph-mon[103713]: pgmap v994: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 927 KiB/s wr, 133 op/s 2023-09-10T01:11:54.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:53 smithi044 ceph-mon[98676]: pgmap v994: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 927 KiB/s wr, 133 op/s 2023-09-10T01:11:54.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:53 smithi099 ceph-mon[102692]: pgmap v994: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 927 KiB/s wr, 133 op/s 2023-09-10T01:11:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.112+0000 7fc42bdb9700 1 -- 172.21.15.44:0/2927627714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc424067f30 msgr2=0x7fc4240683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.112+0000 7fc42bdb9700 1 --2- 172.21.15.44:0/2927627714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc424067f30 0x7fc4240683b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc414009990 tx=0x7fc414009ca0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.113+0000 7fc42bdb9700 1 -- 172.21.15.44:0/2927627714 shutdown_connections 2023-09-10T01:11:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.113+0000 7fc42bdb9700 1 --2- 172.21.15.44:0/2927627714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc42410cdd0 0x7fc42410f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.113+0000 7fc42bdb9700 1 --2- 172.21.15.44:0/2927627714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc424101470 0x7fc4240679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.113+0000 7fc42bdb9700 1 --2- 172.21.15.44:0/2927627714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc424067f30 0x7fc4240683b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.113+0000 7fc42bdb9700 1 -- 172.21.15.44:0/2927627714 >> 172.21.15.44:0/2927627714 conn(0x7fc424074f30 msgr2=0x7fc424075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.113+0000 7fc42bdb9700 1 -- 172.21.15.44:0/2927627714 shutdown_connections 2023-09-10T01:11:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.113+0000 7fc42bdb9700 1 -- 172.21.15.44:0/2927627714 wait complete. 2023-09-10T01:11:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.113+0000 7fc42bdb9700 1 Processor -- start 2023-09-10T01:11:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.114+0000 7fc42bdb9700 1 -- start start 2023-09-10T01:11:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.114+0000 7fc42bdb9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc424067f30 0x7fc42419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.114+0000 7fc42bdb9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc424101470 0x7fc42419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.114+0000 7fc42bdb9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc42410cdd0 0x7fc424198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.114+0000 7fc42bdb9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc424198c90 con 0x7fc424067f30 2023-09-10T01:11:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.114+0000 7fc42bdb9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc424198e00 con 0x7fc424101470 2023-09-10T01:11:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.114+0000 7fc42bdb9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc424198fc0 con 0x7fc42410cdd0 2023-09-10T01:11:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.114+0000 7fc429b55700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc424067f30 0x7fc42419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.114+0000 7fc429b55700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc424067f30 0x7fc42419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56866/0 (socket says 172.21.15.44:56866) 2023-09-10T01:11:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.114+0000 7fc429b55700 1 -- 172.21.15.44:0/1798007801 learned_addr learned my addr 172.21.15.44:0/1798007801 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.114+0000 7fc42a356700 1 --2- 172.21.15.44:0/1798007801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc42410cdd0 0x7fc424198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.114+0000 7fc429354700 1 --2- 172.21.15.44:0/1798007801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc424101470 0x7fc42419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.114+0000 7fc42a356700 1 -- 172.21.15.44:0/1798007801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc424101470 msgr2=0x7fc42419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.114+0000 7fc42a356700 1 --2- 172.21.15.44:0/1798007801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc424101470 0x7fc42419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.115+0000 7fc42a356700 1 -- 172.21.15.44:0/1798007801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc424067f30 msgr2=0x7fc42419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.115+0000 7fc42a356700 1 --2- 172.21.15.44:0/1798007801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc424067f30 0x7fc42419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.115+0000 7fc42a356700 1 -- 172.21.15.44:0/1798007801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc420009520 con 0x7fc42410cdd0 2023-09-10T01:11:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.115+0000 7fc429b55700 1 --2- 172.21.15.44:0/1798007801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc424067f30 0x7fc42419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:11:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.115+0000 7fc42a356700 1 --2- 172.21.15.44:0/1798007801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc42410cdd0 0x7fc424198620 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00efc0 tx=0x7fc41c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.115+0000 7fc41affd700 1 -- 172.21.15.44:0/1798007801 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c018070 con 0x7fc42410cdd0 2023-09-10T01:11:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.115+0000 7fc41affd700 1 -- 172.21.15.44:0/1798007801 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc41c00f040 con 0x7fc42410cdd0 2023-09-10T01:11:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.115+0000 7fc42bdb9700 1 -- 172.21.15.44:0/1798007801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4140095f0 con 0x7fc42410cdd0 2023-09-10T01:11:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.115+0000 7fc41affd700 1 -- 172.21.15.44:0/1798007801 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00cba0 con 0x7fc42410cdd0 2023-09-10T01:11:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.115+0000 7fc42bdb9700 1 -- 172.21.15.44:0/1798007801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4241a55d0 con 0x7fc42410cdd0 2023-09-10T01:11:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.119+0000 7fc41affd700 1 -- 172.21.15.44:0/1798007801 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc41c003a00 con 0x7fc42410cdd0 2023-09-10T01:11:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.119+0000 7fc41affd700 1 --2- 172.21.15.44:0/1798007801 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc4100a0830 0x7fc4100a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.119+0000 7fc437f7fb80 1 Processor -- start 2023-09-10T01:11:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.119+0000 7fc41affd700 1 -- 172.21.15.44:0/1798007801 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc41c0cb880 con 0x7fc42410cdd0 2023-09-10T01:11:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.119+0000 7fc429b55700 1 --2- 172.21.15.44:0/1798007801 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc4100a0830 0x7fc4100a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc437f7fb80 1 -- start start 2023-09-10T01:11:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc429b55700 1 --2- 172.21.15.44:0/1798007801 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc4100a0830 0x7fc4100a2cf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc420009fd0 tx=0x7fc420009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc437f7fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d1589b0 0x55778d158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc437f7fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d152b10 0x55778d152f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc437f7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d1534d0 0x55778d15f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc437f7fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778d0a36d0 con 0x55778d152b10 2023-09-10T01:11:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc437f7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778d0a0ff0 con 0x55778d1534d0 2023-09-10T01:11:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc437f7fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778d09c8e0 con 0x55778d1589b0 2023-09-10T01:11:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc429354700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d152b10 0x55778d152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc429b55700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d1589b0 0x55778d158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc429354700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d152b10 0x55778d152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56876/0 (socket says 172.21.15.44:56876) 2023-09-10T01:11:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc429354700 1 -- 172.21.15.44:0/3489483673 learned_addr learned my addr 172.21.15.44:0/3489483673 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc42a356700 1 --2- 172.21.15.44:0/3489483673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d1534d0 0x55778d15f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc429b55700 1 -- 172.21.15.44:0/3489483673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d1534d0 msgr2=0x55778d15f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc429b55700 1 --2- 172.21.15.44:0/3489483673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d1534d0 0x55778d15f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc429b55700 1 -- 172.21.15.44:0/3489483673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d152b10 msgr2=0x55778d152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc429b55700 1 --2- 172.21.15.44:0/3489483673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d152b10 0x55778d152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc429b55700 1 -- 172.21.15.44:0/3489483673 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55778cec8d80 con 0x55778d1589b0 2023-09-10T01:11:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.120+0000 7fc429354700 1 --2- 172.21.15.44:0/3489483673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d152b10 0x55778d152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:11:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.121+0000 7fc429b55700 1 --2- 172.21.15.44:0/3489483673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d1589b0 0x55778d158d90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc420011e50 tx=0x7fc420014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0127073b4e24d45 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.121+0000 7fc40ffff700 1 -- 172.21.15.44:0/3489483673 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42001b630 con 0x55778d1589b0 2023-09-10T01:11:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.121+0000 7fc40ffff700 1 -- 172.21.15.44:0/3489483673 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc42001bc20 con 0x55778d1589b0 2023-09-10T01:11:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.121+0000 7fc40ffff700 1 -- 172.21.15.44:0/3489483673 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42001cbf0 con 0x55778d1589b0 2023-09-10T01:11:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.122+0000 7fc437f7fb80 1 -- 172.21.15.44:0/3489483673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d1589b0 msgr2=0x55778d158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.122+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/3489483673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d1589b0 0x55778d158d90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc420011e50 tx=0x7fc420014000 comp rx=0 tx=0).stop 2023-09-10T01:11:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.122+0000 7fc437f7fb80 1 -- 172.21.15.44:0/3489483673 shutdown_connections 2023-09-10T01:11:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.122+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/3489483673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d1534d0 0x55778d15f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.122+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/3489483673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d1589b0 0x55778d158d90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.122+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/3489483673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d152b10 0x55778d152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.122+0000 7fc437f7fb80 1 -- 172.21.15.44:0/3489483673 >> 172.21.15.44:0/3489483673 conn(0x55778d0bf030 msgr2=0x55778d145560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.122+0000 7fc437f7fb80 1 -- 172.21.15.44:0/3489483673 shutdown_connections 2023-09-10T01:11:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.122+0000 7fc437f7fb80 1 -- 172.21.15.44:0/3489483673 wait complete. 2023-09-10T01:11:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.122+0000 7fc437f7fb80 1 -- start start 2023-09-10T01:11:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.122+0000 7fc437f7fb80 1 Processor -- start 2023-09-10T01:11:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.123+0000 7fc437f7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d152b10 0x55778d15f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.123+0000 7fc437f7fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d1534d0 0x55778d15c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.123+0000 7fc437f7fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1589b0 0x55778d15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.123+0000 7fc437f7fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778d03ab10 con 0x55778d1589b0 2023-09-10T01:11:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.123+0000 7fc437f7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778d03a810 con 0x55778d152b10 2023-09-10T01:11:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.123+0000 7fc437f7fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778cf1e8c0 con 0x55778d1534d0 2023-09-10T01:11:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.123+0000 7fc437f7fb80 10 client.0 _ll_register_callbacks cb 0x55778cf84a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:11:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.123+0000 7fc429b55700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d152b10 0x55778d15f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.123+0000 7fc429b55700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d152b10 0x55778d15f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58620/0 (socket says 172.21.15.44:58620) 2023-09-10T01:11:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.123+0000 7fc429b55700 1 -- 172.21.15.44:0/19438338 learned_addr learned my addr 172.21.15.44:0/19438338 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.123+0000 7fc429354700 1 --2- 172.21.15.44:0/19438338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d1534d0 0x55778d15c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.123+0000 7fc42a356700 1 --2- 172.21.15.44:0/19438338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1589b0 0x55778d15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.124+0000 7fc429b55700 1 -- 172.21.15.44:0/19438338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d1534d0 msgr2=0x55778d15c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.124+0000 7fc429b55700 1 --2- 172.21.15.44:0/19438338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d1534d0 0x55778d15c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.124+0000 7fc429b55700 1 -- 172.21.15.44:0/19438338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1589b0 msgr2=0x55778d15cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.124+0000 7fc429b55700 1 --2- 172.21.15.44:0/19438338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1589b0 0x55778d15cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.124+0000 7fc429b55700 1 -- 172.21.15.44:0/19438338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42410fec0 con 0x55778d152b10 2023-09-10T01:11:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.124+0000 7fc429b55700 1 --2- 172.21.15.44:0/19438338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d152b10 0x55778d15f570 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc42001b900 tx=0x7fc42001b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a073b6a39800f3b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.124+0000 7fc40d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:11:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.124+0000 7fc40d7fa700 1 -- 172.21.15.44:0/19438338 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420005240 con 0x55778d152b10 2023-09-10T01:11:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.124+0000 7fc437f7fb80 1 -- 172.21.15.44:0/19438338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55778cff4490 con 0x55778d152b10 2023-09-10T01:11:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.124+0000 7fc40d7fa700 1 -- 172.21.15.44:0/19438338 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc4200053a0 con 0x55778d152b10 2023-09-10T01:11:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.124+0000 7fc40d7fa700 1 -- 172.21.15.44:0/19438338 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc420005560 con 0x55778d152b10 2023-09-10T01:11:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.124+0000 7fc437f7fb80 10 client.28859 fetch_fsmap learned FSMap version 120 2023-09-10T01:11:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.124+0000 7fc437f7fb80 1 -- 172.21.15.44:0/19438338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x7fc420017050 con 0x55778d152b10 2023-09-10T01:11:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.125+0000 7fc40d7fa700 1 -- 172.21.15.44:0/19438338 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55778cff4490 con 0x55778d152b10 2023-09-10T01:11:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.125+0000 7fc40d7fa700 1 -- 172.21.15.44:0/19438338 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fc420004710 con 0x55778d152b10 2023-09-10T01:11:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.125+0000 7fc437f7fb80 10 client.28859 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:11:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.125+0000 7fc437f7fb80 10 client.28859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:11:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.125+0000 7fc437f7fb80 20 client.28859 populate_metadata read hostname 'smithi044' 2023-09-10T01:11:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.125+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/19438338 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55778d175270 0x55778d177690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.125+0000 7fc437f7fb80 4 client.28859 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:11:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.125+0000 7fc437f7fb80 1 -- 172.21.15.44:0/19438338 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55778d0b1840 con 0x55778d175270 2023-09-10T01:11:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.125+0000 7fc429354700 1 --2- 172.21.15.44:0/19438338 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55778d175270 0x55778d177690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.125+0000 7fc429354700 1 --2- 172.21.15.44:0/19438338 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55778d175270 0x55778d177690 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be6c6dbd078281cc server_cookie=89b5912637c4bfc8 in_seq=0 out_seq=0 2023-09-10T01:11:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.125+0000 7fc40d7fa700 10 client.28859 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.127+0000 7fc40d7fa700 1 -- 172.21.15.44:0/19438338 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55778d0b1840 con 0x55778d175270 2023-09-10T01:11:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.127+0000 7fc40d7fa700 10 client.28859 handle_command_reply: tid=0 2023-09-10T01:11:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.127+0000 7fc437f7fb80 1 client.28859 shutdown 2023-09-10T01:11:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.127+0000 7fc437f7fb80 1 -- 172.21.15.44:0/19438338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d152b10 msgr2=0x55778d15f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.127+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/19438338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d152b10 0x55778d15f570 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc42001b900 tx=0x7fc42001b930 comp rx=0 tx=0).stop 2023-09-10T01:11:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.127+0000 7fc437f7fb80 1 -- 172.21.15.44:0/19438338 shutdown_connections 2023-09-10T01:11:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.127+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/19438338 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55778d175270 0x55778d177690 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.127+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/19438338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1589b0 0x55778d15cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.127+0000 7fc40d7fa700 0 client.28859 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.127+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/19438338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d152b10 0x55778d15f570 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.127+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/19438338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d1534d0 0x55778d15c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.127+0000 7fc437f7fb80 1 -- 172.21.15.44:0/19438338 >> 172.21.15.44:0/19438338 conn(0x55778d0bf030 msgr2=0x55778d1447a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.127+0000 7fc437f7fb80 1 -- 172.21.15.44:0/19438338 shutdown_connections 2023-09-10T01:11:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.127+0000 7fc437f7fb80 1 -- 172.21.15.44:0/19438338 wait complete. 2023-09-10T01:11:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.127+0000 7fc437f7fb80 20 client.28859 trim_cache size 0 max 16384 2023-09-10T01:11:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.148+0000 7fc437f7fb80 1 Processor -- start 2023-09-10T01:11:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.148+0000 7fc437f7fb80 1 -- start start 2023-09-10T01:11:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc437f7fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d152b10 0x55778d152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc437f7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d1534d0 0x55778d14e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc437f7fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1589b0 0x55778d15f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc437f7fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778d13f8a0 con 0x55778d1589b0 2023-09-10T01:11:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc437f7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778d1565c0 con 0x55778d1534d0 2023-09-10T01:11:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc437f7fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778d15de20 con 0x55778d152b10 2023-09-10T01:11:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc429b55700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d152b10 0x55778d152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc42a356700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1589b0 0x55778d15f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc429b55700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d152b10 0x55778d152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35538/0 (socket says 172.21.15.44:35538) 2023-09-10T01:11:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc429b55700 1 -- 172.21.15.44:0/4014504531 learned_addr learned my addr 172.21.15.44:0/4014504531 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc429354700 1 --2- 172.21.15.44:0/4014504531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d1534d0 0x55778d14e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc429b55700 1 -- 172.21.15.44:0/4014504531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d1534d0 msgr2=0x55778d14e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc429b55700 1 --2- 172.21.15.44:0/4014504531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d1534d0 0x55778d14e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc429b55700 1 -- 172.21.15.44:0/4014504531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1589b0 msgr2=0x55778d15f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc429b55700 1 --2- 172.21.15.44:0/4014504531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1589b0 0x55778d15f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc429b55700 1 -- 172.21.15.44:0/4014504531 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55778ceb1c00 con 0x55778d152b10 2023-09-10T01:11:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc429b55700 1 --2- 172.21.15.44:0/4014504531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d152b10 0x55778d152ef0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc4200115e0 tx=0x7fc42000ff50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=148238fb10cd403e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc418ff9700 1 -- 172.21.15.44:0/4014504531 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42001ba50 con 0x55778d152b10 2023-09-10T01:11:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc418ff9700 1 -- 172.21.15.44:0/4014504531 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc420005170 con 0x55778d152b10 2023-09-10T01:11:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.149+0000 7fc418ff9700 1 -- 172.21.15.44:0/4014504531 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42002e6c0 con 0x55778d152b10 2023-09-10T01:11:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.150+0000 7fc437f7fb80 1 -- 172.21.15.44:0/4014504531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d152b10 msgr2=0x55778d152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.150+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/4014504531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d152b10 0x55778d152ef0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc4200115e0 tx=0x7fc42000ff50 comp rx=0 tx=0).stop 2023-09-10T01:11:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.150+0000 7fc437f7fb80 1 -- 172.21.15.44:0/4014504531 shutdown_connections 2023-09-10T01:11:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.150+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/4014504531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1589b0 0x55778d15f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.150+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/4014504531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d152b10 0x55778d152ef0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.150+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/4014504531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d1534d0 0x55778d14e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.150+0000 7fc437f7fb80 1 -- 172.21.15.44:0/4014504531 >> 172.21.15.44:0/4014504531 conn(0x55778d0bf030 msgr2=0x55778d157930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.150+0000 7fc437f7fb80 1 -- 172.21.15.44:0/4014504531 shutdown_connections 2023-09-10T01:11:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.150+0000 7fc437f7fb80 1 -- 172.21.15.44:0/4014504531 wait complete. 2023-09-10T01:11:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc437f7fb80 1 -- start start 2023-09-10T01:11:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc437f7fb80 1 Processor -- start 2023-09-10T01:11:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc437f7fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d152b10 0x55778d14c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc437f7fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1534d0 0x55778d14c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc437f7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d1589b0 0x55778d14ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc437f7fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778d1c8140 con 0x55778d1534d0 2023-09-10T01:11:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc437f7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778d1c82b0 con 0x55778d1589b0 2023-09-10T01:11:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc437f7fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778d1c8470 con 0x55778d152b10 2023-09-10T01:11:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc437f7fb80 10 client.0 _ll_register_callbacks cb 0x7fc420005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:11:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc429354700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1534d0 0x55778d14c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc429b55700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d152b10 0x55778d14c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc429354700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1534d0 0x55778d14c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56896/0 (socket says 172.21.15.44:56896) 2023-09-10T01:11:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc429354700 1 -- 172.21.15.44:0/3387128186 learned_addr learned my addr 172.21.15.44:0/3387128186 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:11:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc42a356700 1 --2- 172.21.15.44:0/3387128186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d1589b0 0x55778d14ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc429354700 1 -- 172.21.15.44:0/3387128186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d152b10 msgr2=0x55778d14c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc429354700 1 --2- 172.21.15.44:0/3387128186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d152b10 0x55778d14c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc429354700 1 -- 172.21.15.44:0/3387128186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d1589b0 msgr2=0x55778d14ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:11:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc429354700 1 --2- 172.21.15.44:0/3387128186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d1589b0 0x55778d14ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.151+0000 7fc429354700 1 -- 172.21.15.44:0/3387128186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc420017050 con 0x55778d1534d0 2023-09-10T01:11:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.152+0000 7fc429354700 1 --2- 172.21.15.44:0/3387128186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1534d0 0x55778d14c8e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fc414005910 tx=0x7fc4140051e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8e62041b542904 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:11:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.152+0000 7fc40d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:11:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.152+0000 7fc40d7fa700 1 -- 172.21.15.44:0/3387128186 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41401d040 con 0x55778d1534d0 2023-09-10T01:11:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.152+0000 7fc437f7fb80 1 -- 172.21.15.44:0/3387128186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc42001bc20 con 0x55778d1534d0 2023-09-10T01:11:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.152+0000 7fc40d7fa700 1 -- 172.21.15.44:0/3387128186 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc414029cf0 con 0x55778d1534d0 2023-09-10T01:11:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.152+0000 7fc40d7fa700 1 -- 172.21.15.44:0/3387128186 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4140056d0 con 0x55778d1534d0 2023-09-10T01:11:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.152+0000 7fc437f7fb80 10 client.19191 fetch_fsmap learned FSMap version 120 2023-09-10T01:11:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.152+0000 7fc437f7fb80 1 -- 172.21.15.44:0/3387128186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x55778ceef1f0 con 0x55778d1534d0 2023-09-10T01:11:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.152+0000 7fc40d7fa700 1 -- 172.21.15.44:0/3387128186 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc42001bc20 con 0x55778d1534d0 2023-09-10T01:11:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.152+0000 7fc40d7fa700 1 -- 172.21.15.44:0/3387128186 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fc4140039b0 con 0x55778d1534d0 2023-09-10T01:11:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.152+0000 7fc437f7fb80 10 client.19191 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:11:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.152+0000 7fc437f7fb80 10 client.19191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:11:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.152+0000 7fc437f7fb80 20 client.19191 populate_metadata read hostname 'smithi044' 2023-09-10T01:11:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.152+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/3387128186 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55778d1c9ac0 0x55778d1cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:11:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.152+0000 7fc437f7fb80 4 client.19191 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:11:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.152+0000 7fc437f7fb80 1 -- 172.21.15.44:0/3387128186 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55778d0958c0 con 0x55778d1c9ac0 2023-09-10T01:11:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.153+0000 7fc429b55700 1 --2- 172.21.15.44:0/3387128186 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55778d1c9ac0 0x55778d1cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:11:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.153+0000 7fc429b55700 1 --2- 172.21.15.44:0/3387128186 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55778d1c9ac0 0x55778d1cbf00 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbf2dfc39cb9cf3b server_cookie=35283696f18fc679 in_seq=0 out_seq=0 2023-09-10T01:11:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.153+0000 7fc40d7fa700 10 client.19191 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.163+0000 7fc40d7fa700 1 -- 172.21.15.44:0/3387128186 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55778d0958c0 con 0x55778d1c9ac0 2023-09-10T01:11:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.163+0000 7fc40d7fa700 10 client.19191 handle_command_reply: tid=0 2023-09-10T01:11:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.163+0000 7fc437f7fb80 1 client.19191 shutdown 2023-09-10T01:11:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.163+0000 7fc437f7fb80 1 -- 172.21.15.44:0/3387128186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1534d0 msgr2=0x55778d14c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.163+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/3387128186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1534d0 0x55778d14c8e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fc414005910 tx=0x7fc4140051e0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.163+0000 7fc437f7fb80 1 -- 172.21.15.44:0/3387128186 shutdown_connections 2023-09-10T01:11:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.163+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/3387128186 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55778d1c9ac0 0x55778d1cbf00 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.163+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/3387128186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55778d1589b0 0x55778d14ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.163+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/3387128186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55778d152b10 0x55778d14c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.163+0000 7fc437f7fb80 1 --2- 172.21.15.44:0/3387128186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55778d1534d0 0x55778d14c8e0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.163+0000 7fc437f7fb80 1 -- 172.21.15.44:0/3387128186 >> 172.21.15.44:0/3387128186 conn(0x55778d0bf030 msgr2=0x55778d1492f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.163+0000 7fc40d7fa700 0 client.19191 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:11:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.163+0000 7fc437f7fb80 1 -- 172.21.15.44:0/3387128186 shutdown_connections 2023-09-10T01:11:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.163+0000 7fc437f7fb80 1 -- 172.21.15.44:0/3387128186 wait complete. 2023-09-10T01:11:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.163+0000 7fc437f7fb80 20 client.19191 trim_cache size 0 max 16384 2023-09-10T01:11:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:11:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:11:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:11:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:11:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.164+0000 7fc42bdb9700 1 -- 172.21.15.44:0/1798007801 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc4100a0830 msgr2=0x7fc4100a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.164+0000 7fc42bdb9700 1 --2- 172.21.15.44:0/1798007801 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc4100a0830 0x7fc4100a2cf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc420009fd0 tx=0x7fc420009260 comp rx=0 tx=0).stop 2023-09-10T01:11:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.164+0000 7fc42bdb9700 1 -- 172.21.15.44:0/1798007801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc42410cdd0 msgr2=0x7fc424198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:11:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.164+0000 7fc42bdb9700 1 --2- 172.21.15.44:0/1798007801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc42410cdd0 0x7fc424198620 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00efc0 tx=0x7fc41c00c3a0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.164+0000 7fc42bdb9700 1 -- 172.21.15.44:0/1798007801 shutdown_connections 2023-09-10T01:11:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.164+0000 7fc42bdb9700 1 --2- 172.21.15.44:0/1798007801 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc4100a0830 0x7fc4100a2cf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.164+0000 7fc42bdb9700 1 --2- 172.21.15.44:0/1798007801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc42410cdd0 0x7fc424198620 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.164+0000 7fc42bdb9700 1 --2- 172.21.15.44:0/1798007801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc424067f30 0x7fc42419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.164+0000 7fc42bdb9700 1 --2- 172.21.15.44:0/1798007801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc424101470 0x7fc42419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:11:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.164+0000 7fc42bdb9700 1 -- 172.21.15.44:0/1798007801 >> 172.21.15.44:0/1798007801 conn(0x7fc424074f30 msgr2=0x7fc4240fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:11:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.164+0000 7fc42bdb9700 1 -- 172.21.15.44:0/1798007801 shutdown_connections 2023-09-10T01:11:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:11:56.164+0000 7fc42bdb9700 1 -- 172.21.15.44:0/1798007801 wait complete. 2023-09-10T01:11:56.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:55 smithi131 ceph-mon[103713]: pgmap v995: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 927 KiB/s wr, 133 op/s 2023-09-10T01:11:56.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:55 smithi044 ceph-mon[98676]: pgmap v995: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 927 KiB/s wr, 133 op/s 2023-09-10T01:11:56.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:55 smithi099 ceph-mon[102692]: pgmap v995: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 927 KiB/s wr, 133 op/s 2023-09-10T01:11:58.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:57 smithi131 ceph-mon[103713]: pgmap v996: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 911 KiB/s wr, 130 op/s 2023-09-10T01:11:58.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:57 smithi044 ceph-mon[98676]: pgmap v996: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 911 KiB/s wr, 130 op/s 2023-09-10T01:11:58.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:57 smithi099 ceph-mon[102692]: pgmap v996: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 911 KiB/s wr, 130 op/s 2023-09-10T01:12:00.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:11:59 smithi131 ceph-mon[103713]: pgmap v997: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 911 KiB/s wr, 130 op/s 2023-09-10T01:12:00.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:11:59 smithi044 ceph-mon[98676]: pgmap v997: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 911 KiB/s wr, 130 op/s 2023-09-10T01:12:00.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:11:59 smithi099 ceph-mon[102692]: pgmap v997: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 911 KiB/s wr, 130 op/s 2023-09-10T01:12:02.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:01 smithi131 ceph-mon[103713]: pgmap v998: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 184 op/s 2023-09-10T01:12:02.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:01 smithi044 ceph-mon[98676]: pgmap v998: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 184 op/s 2023-09-10T01:12:02.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:01 smithi099 ceph-mon[102692]: pgmap v998: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 184 op/s 2023-09-10T01:12:04.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:03 smithi131 ceph-mon[103713]: pgmap v999: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 881 KiB/s wr, 127 op/s 2023-09-10T01:12:04.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:03 smithi044 ceph-mon[98676]: pgmap v999: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 881 KiB/s wr, 127 op/s 2023-09-10T01:12:04.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:03 smithi099 ceph-mon[102692]: pgmap v999: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 881 KiB/s wr, 127 op/s 2023-09-10T01:12:05.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:04 smithi131 ceph-mon[103713]: osdmap e147: 12 total, 12 up, 12 in 2023-09-10T01:12:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:04 smithi044 ceph-mon[98676]: osdmap e147: 12 total, 12 up, 12 in 2023-09-10T01:12:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:04 smithi099 ceph-mon[102692]: osdmap e147: 12 total, 12 up, 12 in 2023-09-10T01:12:06.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:05 smithi131 ceph-mon[103713]: pgmap v1001: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2023-09-10T01:12:06.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:05 smithi044 ceph-mon[98676]: pgmap v1001: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2023-09-10T01:12:06.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:05 smithi099 ceph-mon[102692]: pgmap v1001: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2023-09-10T01:12:08.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:07 smithi131 ceph-mon[103713]: pgmap v1002: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-09-10T01:12:08.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:07 smithi044 ceph-mon[98676]: pgmap v1002: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-09-10T01:12:08.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:07 smithi099 ceph-mon[102692]: pgmap v1002: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-09-10T01:12:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.279+0000 7f1c81dd5700 1 -- 172.21.15.44:0/3375910215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c7c0ff760 msgr2=0x7f1c7c0ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.279+0000 7f1c81dd5700 1 --2- 172.21.15.44:0/3375910215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c7c0ff760 0x7f1c7c0ffbe0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009990 tx=0x7f1c70009ca0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.279+0000 7f1c81dd5700 1 -- 172.21.15.44:0/3375910215 shutdown_connections 2023-09-10T01:12:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.279+0000 7f1c81dd5700 1 --2- 172.21.15.44:0/3375910215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c7c100120 0x7f1c7c10c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.279+0000 7f1c81dd5700 1 --2- 172.21.15.44:0/3375910215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c7c1055b0 0x7f1c7c105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.279+0000 7f1c81dd5700 1 --2- 172.21.15.44:0/3375910215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c7c0ff760 0x7f1c7c0ffbe0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.279+0000 7f1c81dd5700 1 -- 172.21.15.44:0/3375910215 >> 172.21.15.44:0/3375910215 conn(0x7f1c7c074f30 msgr2=0x7f1c7c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.280+0000 7f1c81dd5700 1 -- 172.21.15.44:0/3375910215 shutdown_connections 2023-09-10T01:12:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.280+0000 7f1c81dd5700 1 -- 172.21.15.44:0/3375910215 wait complete. 2023-09-10T01:12:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.280+0000 7f1c81dd5700 1 Processor -- start 2023-09-10T01:12:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.280+0000 7f1c81dd5700 1 -- start start 2023-09-10T01:12:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.280+0000 7f1c81dd5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c7c0ff760 0x7f1c7c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.280+0000 7f1c81dd5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c7c100120 0x7f1c7c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.280+0000 7f1c81dd5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c7c1055b0 0x7f1c7c1a2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.280+0000 7f1c81dd5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c1a2a50 con 0x7f1c7c100120 2023-09-10T01:12:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.280+0000 7f1c81dd5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c1a2bc0 con 0x7f1c7c0ff760 2023-09-10T01:12:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.280+0000 7f1c81dd5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c1a2d80 con 0x7f1c7c1055b0 2023-09-10T01:12:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.281+0000 7f1c7affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c7c100120 0x7f1c7c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.281+0000 7f1c7b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c7c0ff760 0x7f1c7c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.281+0000 7f1c7affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c7c100120 0x7f1c7c197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56910/0 (socket says 172.21.15.44:56910) 2023-09-10T01:12:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.281+0000 7f1c7affd700 1 -- 172.21.15.44:0/293416338 learned_addr learned my addr 172.21.15.44:0/293416338 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.281+0000 7f1c7bfff700 1 --2- 172.21.15.44:0/293416338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c7c1055b0 0x7f1c7c1a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.281+0000 7f1c7affd700 1 -- 172.21.15.44:0/293416338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c7c1055b0 msgr2=0x7f1c7c1a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.281+0000 7f1c7affd700 1 --2- 172.21.15.44:0/293416338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c7c1055b0 0x7f1c7c1a2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.281+0000 7f1c7affd700 1 -- 172.21.15.44:0/293416338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c7c0ff760 msgr2=0x7f1c7c1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.281+0000 7f1c7affd700 1 --2- 172.21.15.44:0/293416338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c7c0ff760 0x7f1c7c1975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.281+0000 7f1c7affd700 1 -- 172.21.15.44:0/293416338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c74009c40 con 0x7f1c7c100120 2023-09-10T01:12:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.281+0000 7f1c7affd700 1 --2- 172.21.15.44:0/293416338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c7c100120 0x7f1c7c197b20 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f1c70006010 tx=0x7f1c70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.281+0000 7f1c78ff9700 1 -- 172.21.15.44:0/293416338 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7001d070 con 0x7f1c7c100120 2023-09-10T01:12:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.281+0000 7f1c81dd5700 1 -- 172.21.15.44:0/293416338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c700095f0 con 0x7f1c7c100120 2023-09-10T01:12:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.281+0000 7f1c78ff9700 1 -- 172.21.15.44:0/293416338 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c70022470 con 0x7f1c7c100120 2023-09-10T01:12:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.281+0000 7f1c78ff9700 1 -- 172.21.15.44:0/293416338 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70017650 con 0x7f1c7c100120 2023-09-10T01:12:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.281+0000 7f1c81dd5700 1 -- 172.21.15.44:0/293416338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7c1a32f0 con 0x7f1c7c100120 2023-09-10T01:12:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.284+0000 7f1c78ff9700 1 -- 172.21.15.44:0/293416338 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1c700225e0 con 0x7f1c7c100120 2023-09-10T01:12:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.284+0000 7f1c78ff9700 1 --2- 172.21.15.44:0/293416338 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1c640a07b0 0x7f1c640a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.284+0000 7f1c7b7fe700 1 --2- 172.21.15.44:0/293416338 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1c640a07b0 0x7f1c640a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.284+0000 7f1c78ff9700 1 -- 172.21.15.44:0/293416338 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f1c700cd770 con 0x7f1c7c100120 2023-09-10T01:12:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.285+0000 7f1c7b7fe700 1 --2- 172.21.15.44:0/293416338 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1c640a07b0 0x7f1c640a2c70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009be0 tx=0x7f1c6c009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.285+0000 7f1c8df9bb80 1 Processor -- start 2023-09-10T01:12:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.285+0000 7f1c8df9bb80 1 -- start start 2023-09-10T01:12:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.285+0000 7f1c8df9bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860c51a0 0x555f860c5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.285+0000 7f1c8df9bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bf300 0x555f860bf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.285+0000 7f1c8df9bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bfcc0 0x555f860cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.285+0000 7f1c8df9bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f860186d0 con 0x555f860bf300 2023-09-10T01:12:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.285+0000 7f1c8df9bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f86015ff0 con 0x555f860bfcc0 2023-09-10T01:12:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.285+0000 7f1c8df9bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f860118e0 con 0x555f860c51a0 2023-09-10T01:12:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.286+0000 7f1c7affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bf300 0x555f860bf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.286+0000 7f1c7b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860c51a0 0x555f860c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.286+0000 7f1c7affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bf300 0x555f860bf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56918/0 (socket says 172.21.15.44:56918) 2023-09-10T01:12:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.286+0000 7f1c7affd700 1 -- 172.21.15.44:0/1298696536 learned_addr learned my addr 172.21.15.44:0/1298696536 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.286+0000 7f1c7bfff700 1 --2- 172.21.15.44:0/1298696536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bfcc0 0x555f860cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.286+0000 7f1c7affd700 1 -- 172.21.15.44:0/1298696536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860c51a0 msgr2=0x555f860c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.286+0000 7f1c7affd700 1 --2- 172.21.15.44:0/1298696536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860c51a0 0x555f860c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.286+0000 7f1c7affd700 1 -- 172.21.15.44:0/1298696536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bfcc0 msgr2=0x555f860cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.286+0000 7f1c7affd700 1 --2- 172.21.15.44:0/1298696536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bfcc0 0x555f860cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.286+0000 7f1c7affd700 1 -- 172.21.15.44:0/1298696536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f85e3dd80 con 0x555f860bf300 2023-09-10T01:12:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.286+0000 7f1c7affd700 1 --2- 172.21.15.44:0/1298696536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bf300 0x555f860bf780 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000b650 tx=0x7f1c70096cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9cade21c484cb9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.286+0000 7f1c61ffb700 1 -- 172.21.15.44:0/1298696536 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7009af60 con 0x555f860bf300 2023-09-10T01:12:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.286+0000 7f1c61ffb700 1 -- 172.21.15.44:0/1298696536 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c7009a860 con 0x555f860bf300 2023-09-10T01:12:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.286+0000 7f1c61ffb700 1 -- 172.21.15.44:0/1298696536 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7009ad20 con 0x555f860bf300 2023-09-10T01:12:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.287+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/1298696536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bf300 msgr2=0x555f860bf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.287+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/1298696536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bf300 0x555f860bf780 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000b650 tx=0x7f1c70096cd0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.287+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/1298696536 shutdown_connections 2023-09-10T01:12:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.287+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/1298696536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bfcc0 0x555f860cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.287+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/1298696536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860c51a0 0x555f860c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.287+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/1298696536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bf300 0x555f860bf780 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.287+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/1298696536 >> 172.21.15.44:0/1298696536 conn(0x555f86034030 msgr2=0x555f860ba560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.287+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/1298696536 shutdown_connections 2023-09-10T01:12:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.287+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/1298696536 wait complete. 2023-09-10T01:12:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.287+0000 7f1c8df9bb80 1 -- start start 2023-09-10T01:12:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.287+0000 7f1c8df9bb80 1 Processor -- start 2023-09-10T01:12:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c8df9bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bf300 0x555f860cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c8df9bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bfcc0 0x555f860cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c8df9bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860c51a0 0x555f860c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c8df9bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f85fafb10 con 0x555f860bfcc0 2023-09-10T01:12:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c8df9bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f85faf810 con 0x555f860bf300 2023-09-10T01:12:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c8df9bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f85e938c0 con 0x555f860c51a0 2023-09-10T01:12:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c8df9bb80 10 client.0 _ll_register_callbacks cb 0x555f85ef9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:12:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c7affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bfcc0 0x555f860cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c7b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bf300 0x555f860cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c7affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bfcc0 0x555f860cf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56924/0 (socket says 172.21.15.44:56924) 2023-09-10T01:12:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c7affd700 1 -- 172.21.15.44:0/4212934167 learned_addr learned my addr 172.21.15.44:0/4212934167 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c7bfff700 1 --2- 172.21.15.44:0/4212934167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860c51a0 0x555f860c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c7affd700 1 -- 172.21.15.44:0/4212934167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860c51a0 msgr2=0x555f860c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c7affd700 1 --2- 172.21.15.44:0/4212934167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860c51a0 0x555f860c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c7affd700 1 -- 172.21.15.44:0/4212934167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bf300 msgr2=0x555f860cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c7affd700 1 --2- 172.21.15.44:0/4212934167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bf300 0x555f860cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c7affd700 1 -- 172.21.15.44:0/4212934167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7c1a32f0 con 0x555f860bfcc0 2023-09-10T01:12:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c7b7fe700 1 --2- 172.21.15.44:0/4212934167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bf300 0x555f860cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:12:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c7bfff700 1 --2- 172.21.15.44:0/4212934167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860c51a0 0x555f860c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:12:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c7affd700 1 --2- 172.21.15.44:0/4212934167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bfcc0 0x555f860cf120 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1c700cab20 tx=0x7f1c70096560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14030bcfd7ed304e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.288+0000 7f1c5affd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:12:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.289+0000 7f1c5affd700 1 -- 172.21.15.44:0/4212934167 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c700a2840 con 0x555f860bfcc0 2023-09-10T01:12:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.289+0000 7f1c5affd700 1 -- 172.21.15.44:0/4212934167 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f1c700b0030 con 0x555f860bfcc0 2023-09-10T01:12:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.289+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/4212934167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f85f69490 con 0x555f860bfcc0 2023-09-10T01:12:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.289+0000 7f1c5affd700 1 -- 172.21.15.44:0/4212934167 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c7009aac0 con 0x555f860bfcc0 2023-09-10T01:12:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.289+0000 7f1c8df9bb80 10 client.19203 fetch_fsmap learned FSMap version 120 2023-09-10T01:12:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.289+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/4212934167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x555f85e26c00 con 0x555f860bfcc0 2023-09-10T01:12:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.289+0000 7f1c5affd700 1 -- 172.21.15.44:0/4212934167 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555f85f69490 con 0x555f860bfcc0 2023-09-10T01:12:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.289+0000 7f1c5affd700 1 -- 172.21.15.44:0/4212934167 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f1c7000fb10 con 0x555f860bfcc0 2023-09-10T01:12:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.289+0000 7f1c8df9bb80 10 client.19203 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:12:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.289+0000 7f1c8df9bb80 10 client.19203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:12:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.289+0000 7f1c8df9bb80 20 client.19203 populate_metadata read hostname 'smithi044' 2023-09-10T01:12:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.289+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/4212934167 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f860d8bb0 0x555f860dafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.289+0000 7f1c8df9bb80 4 client.19203 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:12:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.289+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/4212934167 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f86026840 con 0x555f860d8bb0 2023-09-10T01:12:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.289+0000 7f1c7b7fe700 1 --2- 172.21.15.44:0/4212934167 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f860d8bb0 0x555f860dafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.290+0000 7f1c5affd700 10 client.19203 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:12:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.290+0000 7f1c7b7fe700 1 --2- 172.21.15.44:0/4212934167 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f860d8bb0 0x555f860dafa0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e94a3be26c2940f server_cookie=4296d70e6fa79ef2 in_seq=0 out_seq=0 2023-09-10T01:12:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.290+0000 7f1c5affd700 1 -- 172.21.15.44:0/4212934167 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555f86026840 con 0x555f860d8bb0 2023-09-10T01:12:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.290+0000 7f1c5affd700 10 client.19203 handle_command_reply: tid=0 2023-09-10T01:12:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.290+0000 7f1c8df9bb80 1 client.19203 shutdown 2023-09-10T01:12:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.290+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/4212934167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bfcc0 msgr2=0x555f860cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.290+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/4212934167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bfcc0 0x555f860cf120 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1c700cab20 tx=0x7f1c70096560 comp rx=0 tx=0).stop 2023-09-10T01:12:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.291+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/4212934167 shutdown_connections 2023-09-10T01:12:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.291+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/4212934167 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f860d8bb0 0x555f860dafa0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.291+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/4212934167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860c51a0 0x555f860c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.291+0000 7f1c5affd700 0 client.19203 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:12:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.291+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/4212934167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bf300 0x555f860cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.291+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/4212934167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bfcc0 0x555f860cf120 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.291+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/4212934167 >> 172.21.15.44:0/4212934167 conn(0x555f86034030 msgr2=0x555f860b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.291+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/4212934167 shutdown_connections 2023-09-10T01:12:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.291+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/4212934167 wait complete. 2023-09-10T01:12:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.291+0000 7f1c8df9bb80 20 client.19203 trim_cache size 0 max 16384 2023-09-10T01:12:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.306+0000 7f1c8df9bb80 1 Processor -- start 2023-09-10T01:12:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.306+0000 7f1c8df9bb80 1 -- start start 2023-09-10T01:12:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.306+0000 7f1c8df9bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bf300 0x555f860c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.306+0000 7f1c8df9bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860bfcc0 0x555f860ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.306+0000 7f1c8df9bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860c51a0 0x555f860cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c8df9bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f860c0560 con 0x555f860c51a0 2023-09-10T01:12:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c8df9bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f860cf450 con 0x555f860bf300 2023-09-10T01:12:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c8df9bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f860c3da0 con 0x555f860bfcc0 2023-09-10T01:12:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c7bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860c51a0 0x555f860cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c7bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860c51a0 0x555f860cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56934/0 (socket says 172.21.15.44:56934) 2023-09-10T01:12:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c7bfff700 1 -- 172.21.15.44:0/1331606576 learned_addr learned my addr 172.21.15.44:0/1331606576 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c7b7fe700 1 --2- 172.21.15.44:0/1331606576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bf300 0x555f860c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c7affd700 1 --2- 172.21.15.44:0/1331606576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860bfcc0 0x555f860ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c7bfff700 1 -- 172.21.15.44:0/1331606576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860bfcc0 msgr2=0x555f860ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c7bfff700 1 --2- 172.21.15.44:0/1331606576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860bfcc0 0x555f860ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c7bfff700 1 -- 172.21.15.44:0/1331606576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bf300 msgr2=0x555f860c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c7bfff700 1 --2- 172.21.15.44:0/1331606576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bf300 0x555f860c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c7bfff700 1 -- 172.21.15.44:0/1331606576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f85e641f0 con 0x555f860c51a0 2023-09-10T01:12:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c7bfff700 1 --2- 172.21.15.44:0/1331606576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860c51a0 0x555f860cef10 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1c74007630 tx=0x7f1c74009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bd3d937537f6b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c627fc700 1 -- 172.21.15.44:0/1331606576 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7400e040 con 0x555f860c51a0 2023-09-10T01:12:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c627fc700 1 -- 172.21.15.44:0/1331606576 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c7400f040 con 0x555f860c51a0 2023-09-10T01:12:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.307+0000 7f1c627fc700 1 -- 172.21.15.44:0/1331606576 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c74005280 con 0x555f860c51a0 2023-09-10T01:12:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.308+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/1331606576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860c51a0 msgr2=0x555f860cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.308+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/1331606576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860c51a0 0x555f860cef10 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1c74007630 tx=0x7f1c74009920 comp rx=0 tx=0).stop 2023-09-10T01:12:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.308+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/1331606576 shutdown_connections 2023-09-10T01:12:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.308+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/1331606576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860c51a0 0x555f860cef10 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.308+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/1331606576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bf300 0x555f860c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.308+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/1331606576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860bfcc0 0x555f860ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.308+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/1331606576 >> 172.21.15.44:0/1331606576 conn(0x555f86034030 msgr2=0x555f860df8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.308+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/1331606576 shutdown_connections 2023-09-10T01:12:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.308+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/1331606576 wait complete. 2023-09-10T01:12:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.308+0000 7f1c8df9bb80 1 -- start start 2023-09-10T01:12:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.308+0000 7f1c8df9bb80 1 Processor -- start 2023-09-10T01:12:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c8df9bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bf300 0x555f860d4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c8df9bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bfcc0 0x555f860d52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c8df9bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860c51a0 0x555f860d3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c8df9bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f860d3850 con 0x555f860bf300 2023-09-10T01:12:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c8df9bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f860d39c0 con 0x555f860bfcc0 2023-09-10T01:12:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c8df9bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f860d3b80 con 0x555f860c51a0 2023-09-10T01:12:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c8df9bb80 10 client.0 _ll_register_callbacks cb 0x7f1c7400e280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:12:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c7b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bf300 0x555f860d4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c7b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bf300 0x555f860d4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56936/0 (socket says 172.21.15.44:56936) 2023-09-10T01:12:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c7b7fe700 1 -- 172.21.15.44:0/825265851 learned_addr learned my addr 172.21.15.44:0/825265851 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c7affd700 1 --2- 172.21.15.44:0/825265851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bfcc0 0x555f860d52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c7bfff700 1 --2- 172.21.15.44:0/825265851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860c51a0 0x555f860d3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c7b7fe700 1 -- 172.21.15.44:0/825265851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860c51a0 msgr2=0x555f860d3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c7b7fe700 1 --2- 172.21.15.44:0/825265851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860c51a0 0x555f860d3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c7b7fe700 1 -- 172.21.15.44:0/825265851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bfcc0 msgr2=0x555f860d52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c7b7fe700 1 --2- 172.21.15.44:0/825265851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bfcc0 0x555f860d52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c7b7fe700 1 -- 172.21.15.44:0/825265851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c740092e0 con 0x555f860bf300 2023-09-10T01:12:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c7b7fe700 1 --2- 172.21.15.44:0/825265851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bf300 0x555f860d4d60 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c005950 tx=0x7f1c6c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6fe088ee3dc2de server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c5affd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:12:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c5affd700 1 -- 172.21.15.44:0/825265851 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c024040 con 0x555f860bf300 2023-09-10T01:12:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c5affd700 1 -- 172.21.15.44:0/825265851 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f1c6c029900 con 0x555f860bf300 2023-09-10T01:12:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.309+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/825265851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c7009a860 con 0x555f860bf300 2023-09-10T01:12:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.310+0000 7f1c5affd700 1 -- 172.21.15.44:0/825265851 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c6c01f420 con 0x555f860bf300 2023-09-10T01:12:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.310+0000 7f1c8df9bb80 10 client.19209 fetch_fsmap learned FSMap version 120 2023-09-10T01:12:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.310+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/825265851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x555f85e3c250 con 0x555f860bf300 2023-09-10T01:12:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.310+0000 7f1c5affd700 1 -- 172.21.15.44:0/825265851 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c7009a860 con 0x555f860bf300 2023-09-10T01:12:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.310+0000 7f1c5affd700 1 -- 172.21.15.44:0/825265851 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f1c6c01fc70 con 0x555f860bf300 2023-09-10T01:12:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.310+0000 7f1c8df9bb80 10 client.19209 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:12:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.310+0000 7f1c8df9bb80 10 client.19209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:12:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.310+0000 7f1c8df9bb80 20 client.19209 populate_metadata read hostname 'smithi044' 2023-09-10T01:12:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.310+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/825265851 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f8612dc60 0x555f86130080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.310+0000 7f1c8df9bb80 4 client.19209 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:12:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.310+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/825265851 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555f8600a8c0 con 0x555f8612dc60 2023-09-10T01:12:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.310+0000 7f1c7affd700 1 --2- 172.21.15.44:0/825265851 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f8612dc60 0x555f86130080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.311+0000 7f1c5affd700 10 client.19209 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:12:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.311+0000 7f1c7affd700 1 --2- 172.21.15.44:0/825265851 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f8612dc60 0x555f86130080 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b78f028d3dcd1df server_cookie=f91bbb67b2ee35b9 in_seq=0 out_seq=0 2023-09-10T01:12:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.325+0000 7f1c5affd700 1 -- 172.21.15.44:0/825265851 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555f8600a8c0 con 0x555f8612dc60 2023-09-10T01:12:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.325+0000 7f1c5affd700 10 client.19209 handle_command_reply: tid=0 2023-09-10T01:12:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.325+0000 7f1c8df9bb80 1 client.19209 shutdown 2023-09-10T01:12:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.325+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/825265851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bf300 msgr2=0x555f860d4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.325+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/825265851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bf300 0x555f860d4d60 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c005950 tx=0x7f1c6c0047c0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.325+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/825265851 shutdown_connections 2023-09-10T01:12:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.325+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/825265851 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f8612dc60 0x555f86130080 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.325+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/825265851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f860c51a0 0x555f860d3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.325+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/825265851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f860bf300 0x555f860d4d60 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.325+0000 7f1c8df9bb80 1 --2- 172.21.15.44:0/825265851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f860bfcc0 0x555f860d52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.325+0000 7f1c5affd700 0 client.19209 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:12:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.325+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/825265851 >> 172.21.15.44:0/825265851 conn(0x555f86034030 msgr2=0x555f860c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.325+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/825265851 shutdown_connections 2023-09-10T01:12:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.325+0000 7f1c8df9bb80 1 -- 172.21.15.44:0/825265851 wait complete. 2023-09-10T01:12:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.325+0000 7f1c8df9bb80 20 client.19209 trim_cache size 0 max 16384 2023-09-10T01:12:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:12:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:12:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:12:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:12:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.326+0000 7f1c81dd5700 1 -- 172.21.15.44:0/293416338 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1c640a07b0 msgr2=0x7f1c640a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.326+0000 7f1c81dd5700 1 --2- 172.21.15.44:0/293416338 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1c640a07b0 0x7f1c640a2c70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009be0 tx=0x7f1c6c009260 comp rx=0 tx=0).stop 2023-09-10T01:12:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.326+0000 7f1c81dd5700 1 -- 172.21.15.44:0/293416338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c7c100120 msgr2=0x7f1c7c197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.326+0000 7f1c81dd5700 1 --2- 172.21.15.44:0/293416338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c7c100120 0x7f1c7c197b20 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f1c70006010 tx=0x7f1c70004730 comp rx=0 tx=0).stop 2023-09-10T01:12:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.326+0000 7f1c81dd5700 1 -- 172.21.15.44:0/293416338 shutdown_connections 2023-09-10T01:12:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.326+0000 7f1c81dd5700 1 --2- 172.21.15.44:0/293416338 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1c640a07b0 0x7f1c640a2c70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.326+0000 7f1c81dd5700 1 --2- 172.21.15.44:0/293416338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c7c1055b0 0x7f1c7c1a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.326+0000 7f1c81dd5700 1 --2- 172.21.15.44:0/293416338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c7c0ff760 0x7f1c7c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.326+0000 7f1c81dd5700 1 --2- 172.21.15.44:0/293416338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c7c100120 0x7f1c7c197b20 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.326+0000 7f1c81dd5700 1 -- 172.21.15.44:0/293416338 >> 172.21.15.44:0/293416338 conn(0x7f1c7c074f30 msgr2=0x7f1c7c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.326+0000 7f1c81dd5700 1 -- 172.21.15.44:0/293416338 shutdown_connections 2023-09-10T01:12:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:09.326+0000 7f1c81dd5700 1 -- 172.21.15.44:0/293416338 wait complete. 2023-09-10T01:12:10.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:09 smithi131 ceph-mon[103713]: pgmap v1003: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-09-10T01:12:10.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:09 smithi044 ceph-mon[98676]: pgmap v1003: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-09-10T01:12:10.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:09 smithi099 ceph-mon[102692]: pgmap v1003: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-09-10T01:12:12.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:11 smithi131 ceph-mon[103713]: pgmap v1004: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 197 op/s 2023-09-10T01:12:12.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:11 smithi131 ceph-mon[103713]: osdmap e148: 12 total, 12 up, 12 in 2023-09-10T01:12:12.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:11 smithi044 ceph-mon[98676]: pgmap v1004: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 197 op/s 2023-09-10T01:12:12.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:11 smithi044 ceph-mon[98676]: osdmap e148: 12 total, 12 up, 12 in 2023-09-10T01:12:12.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:11 smithi099 ceph-mon[102692]: pgmap v1004: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 197 op/s 2023-09-10T01:12:12.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:11 smithi099 ceph-mon[102692]: osdmap e148: 12 total, 12 up, 12 in 2023-09-10T01:12:14.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:13 smithi131 ceph-mon[103713]: pgmap v1006: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2023-09-10T01:12:14.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:13 smithi044 ceph-mon[98676]: pgmap v1006: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2023-09-10T01:12:14.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:13 smithi099 ceph-mon[102692]: pgmap v1006: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2023-09-10T01:12:16.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:15 smithi131 ceph-mon[103713]: pgmap v1007: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 197 op/s 2023-09-10T01:12:16.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:15 smithi044 ceph-mon[98676]: pgmap v1007: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 197 op/s 2023-09-10T01:12:16.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:15 smithi099 ceph-mon[102692]: pgmap v1007: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 197 op/s 2023-09-10T01:12:18.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:17 smithi131 ceph-mon[103713]: pgmap v1008: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 172 op/s 2023-09-10T01:12:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:17 smithi044 ceph-mon[98676]: pgmap v1008: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 172 op/s 2023-09-10T01:12:18.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:17 smithi099 ceph-mon[102692]: pgmap v1008: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 172 op/s 2023-09-10T01:12:20.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:19 smithi131 ceph-mon[103713]: pgmap v1009: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 172 op/s 2023-09-10T01:12:20.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:19 smithi044 ceph-mon[98676]: pgmap v1009: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 172 op/s 2023-09-10T01:12:20.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:19 smithi099 ceph-mon[102692]: pgmap v1009: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 172 op/s 2023-09-10T01:12:22.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:21 smithi131 ceph-mon[103713]: pgmap v1010: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2023-09-10T01:12:22.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:21 smithi044 ceph-mon[98676]: pgmap v1010: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2023-09-10T01:12:22.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:21 smithi099 ceph-mon[102692]: pgmap v1010: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2023-09-10T01:12:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.471+0000 7f99544ff700 1 -- 172.21.15.44:0/1037114527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c100120 msgr2=0x7f994c10c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.471+0000 7f99544ff700 1 --2- 172.21.15.44:0/1037114527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c100120 0x7f994c10c330 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9944009f90 tx=0x7f994400b330 comp rx=0 tx=0).stop 2023-09-10T01:12:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.472+0000 7f99544ff700 1 -- 172.21.15.44:0/1037114527 shutdown_connections 2023-09-10T01:12:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.472+0000 7f99544ff700 1 --2- 172.21.15.44:0/1037114527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c100120 0x7f994c10c330 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.472+0000 7f99544ff700 1 --2- 172.21.15.44:0/1037114527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c1055b0 0x7f994c105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.472+0000 7f99544ff700 1 --2- 172.21.15.44:0/1037114527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f994c0ff760 0x7f994c0ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.472+0000 7f99544ff700 1 -- 172.21.15.44:0/1037114527 >> 172.21.15.44:0/1037114527 conn(0x7f994c074f30 msgr2=0x7f994c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.472+0000 7f99544ff700 1 -- 172.21.15.44:0/1037114527 shutdown_connections 2023-09-10T01:12:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.472+0000 7f99544ff700 1 -- 172.21.15.44:0/1037114527 wait complete. 2023-09-10T01:12:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.472+0000 7f99544ff700 1 Processor -- start 2023-09-10T01:12:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.472+0000 7f99544ff700 1 -- start start 2023-09-10T01:12:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f99544ff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c0ff760 0x7f994c19c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f99544ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f994c100120 0x7f994c19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f99544ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c1055b0 0x7f994c196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f99544ff700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c196ac0 con 0x7f994c0ff760 2023-09-10T01:12:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f99544ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c196c30 con 0x7f994c1055b0 2023-09-10T01:12:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f99544ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c196df0 con 0x7f994c100120 2023-09-10T01:12:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f995229b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c0ff760 0x7f994c19c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f995229b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c0ff760 0x7f994c19c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56950/0 (socket says 172.21.15.44:56950) 2023-09-10T01:12:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f995229b700 1 -- 172.21.15.44:0/3278212373 learned_addr learned my addr 172.21.15.44:0/3278212373 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f9952a9c700 1 --2- 172.21.15.44:0/3278212373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c1055b0 0x7f994c196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f9951a9a700 1 --2- 172.21.15.44:0/3278212373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f994c100120 0x7f994c19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f9952a9c700 1 -- 172.21.15.44:0/3278212373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f994c100120 msgr2=0x7f994c19cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f9952a9c700 1 --2- 172.21.15.44:0/3278212373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f994c100120 0x7f994c19cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f9952a9c700 1 -- 172.21.15.44:0/3278212373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c0ff760 msgr2=0x7f994c19c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f9952a9c700 1 --2- 172.21.15.44:0/3278212373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c0ff760 0x7f994c19c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f9952a9c700 1 -- 172.21.15.44:0/3278212373 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9944009c40 con 0x7f994c1055b0 2023-09-10T01:12:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f995229b700 1 --2- 172.21.15.44:0/3278212373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c0ff760 0x7f994c19c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:12:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f9952a9c700 1 --2- 172.21.15.44:0/3278212373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c1055b0 0x7f994c196450 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f994400b6c0 tx=0x7f9944008aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.473+0000 7f99437fe700 1 -- 172.21.15.44:0/3278212373 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994400e040 con 0x7f994c1055b0 2023-09-10T01:12:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.474+0000 7f99437fe700 1 -- 172.21.15.44:0/3278212373 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f994400bbf0 con 0x7f994c1055b0 2023-09-10T01:12:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.474+0000 7f99544ff700 1 -- 172.21.15.44:0/3278212373 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f994c197070 con 0x7f994c1055b0 2023-09-10T01:12:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.474+0000 7f99437fe700 1 -- 172.21.15.44:0/3278212373 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944005040 con 0x7f994c1055b0 2023-09-10T01:12:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.474+0000 7f99544ff700 1 -- 172.21.15.44:0/3278212373 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994c1a35e0 con 0x7f994c1055b0 2023-09-10T01:12:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.476+0000 7f99437fe700 1 -- 172.21.15.44:0/3278212373 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f994401d470 con 0x7f994c1055b0 2023-09-10T01:12:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.477+0000 7f99437fe700 1 --2- 172.21.15.44:0/3278212373 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f99380a0830 0x7f99380a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.477+0000 7f99437fe700 1 -- 172.21.15.44:0/3278212373 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f99440cd4e0 con 0x7f994c1055b0 2023-09-10T01:12:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.477+0000 7f995229b700 1 --2- 172.21.15.44:0/3278212373 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f99380a0830 0x7f99380a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.478+0000 7f99606c5b80 1 Processor -- start 2023-09-10T01:12:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.478+0000 7f99606c5b80 1 -- start start 2023-09-10T01:12:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.478+0000 7f995229b700 1 --2- 172.21.15.44:0/3278212373 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f99380a0830 0x7f99380a2cf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9948011550 tx=0x7f9948009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.478+0000 7f99606c5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e91a0 0x55ecb31e9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.478+0000 7f99606c5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e3300 0x55ecb31e3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.478+0000 7f99606c5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31eff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.478+0000 7f99606c5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb313c6d0 con 0x55ecb31e3cc0 2023-09-10T01:12:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.478+0000 7f99606c5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb3139ff0 con 0x55ecb31e3300 2023-09-10T01:12:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.478+0000 7f99606c5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb31358e0 con 0x55ecb31e91a0 2023-09-10T01:12:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.478+0000 7f9951a9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e3300 0x55ecb31e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.479+0000 7f9951a9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e3300 0x55ecb31e3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58700/0 (socket says 172.21.15.44:58700) 2023-09-10T01:12:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.479+0000 7f9951a9a700 1 -- 172.21.15.44:0/3887666257 learned_addr learned my addr 172.21.15.44:0/3887666257 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.479+0000 7f9952a9c700 1 --2- 172.21.15.44:0/3887666257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31eff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.479+0000 7f995229b700 1 --2- 172.21.15.44:0/3887666257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e91a0 0x55ecb31e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.479+0000 7f9951a9a700 1 -- 172.21.15.44:0/3887666257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e91a0 msgr2=0x55ecb31e9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.479+0000 7f9951a9a700 1 --2- 172.21.15.44:0/3887666257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e91a0 0x55ecb31e9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.479+0000 7f9951a9a700 1 -- 172.21.15.44:0/3887666257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 msgr2=0x55ecb31eff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.479+0000 7f9951a9a700 1 --2- 172.21.15.44:0/3887666257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31eff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.479+0000 7f9951a9a700 1 -- 172.21.15.44:0/3887666257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecb2f61d80 con 0x55ecb31e3300 2023-09-10T01:12:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.479+0000 7f9952a9c700 1 --2- 172.21.15.44:0/3887666257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31eff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:12:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.479+0000 7f995229b700 1 --2- 172.21.15.44:0/3887666257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e91a0 0x55ecb31e9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:12:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.479+0000 7f9951a9a700 1 --2- 172.21.15.44:0/3887666257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e3300 0x55ecb31e3780 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f993c009d30 tx=0x7f993c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c8d8e3db0b536dd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.479+0000 7f9940ff9700 1 -- 172.21.15.44:0/3887666257 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c004250 con 0x55ecb31e3300 2023-09-10T01:12:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.479+0000 7f9940ff9700 1 -- 172.21.15.44:0/3887666257 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f993c004cc0 con 0x55ecb31e3300 2023-09-10T01:12:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.479+0000 7f9940ff9700 1 -- 172.21.15.44:0/3887666257 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c0054e0 con 0x55ecb31e3300 2023-09-10T01:12:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.480+0000 7f99606c5b80 1 -- 172.21.15.44:0/3887666257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e3300 msgr2=0x55ecb31e3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.480+0000 7f99606c5b80 1 --2- 172.21.15.44:0/3887666257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e3300 0x55ecb31e3780 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f993c009d30 tx=0x7f993c011720 comp rx=0 tx=0).stop 2023-09-10T01:12:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.480+0000 7f99606c5b80 1 -- 172.21.15.44:0/3887666257 shutdown_connections 2023-09-10T01:12:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.480+0000 7f99606c5b80 1 --2- 172.21.15.44:0/3887666257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31eff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.480+0000 7f99606c5b80 1 --2- 172.21.15.44:0/3887666257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e91a0 0x55ecb31e9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.480+0000 7f99606c5b80 1 --2- 172.21.15.44:0/3887666257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e3300 0x55ecb31e3780 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.480+0000 7f99606c5b80 1 -- 172.21.15.44:0/3887666257 >> 172.21.15.44:0/3887666257 conn(0x55ecb3158030 msgr2=0x55ecb31de560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.480+0000 7f99606c5b80 1 -- 172.21.15.44:0/3887666257 shutdown_connections 2023-09-10T01:12:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.480+0000 7f99606c5b80 1 -- 172.21.15.44:0/3887666257 wait complete. 2023-09-10T01:12:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.481+0000 7f99606c5b80 1 -- start start 2023-09-10T01:12:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.481+0000 7f99606c5b80 1 Processor -- start 2023-09-10T01:12:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.481+0000 7f99606c5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3300 0x55ecb31efd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.481+0000 7f99606c5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e3cc0 0x55ecb31ed040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.481+0000 7f99606c5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e91a0 0x55ecb31ed580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.481+0000 7f99606c5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb30d3b10 con 0x55ecb31e3300 2023-09-10T01:12:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.481+0000 7f99606c5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb30d3810 con 0x55ecb31e3cc0 2023-09-10T01:12:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.481+0000 7f99606c5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb2fb78c0 con 0x55ecb31e91a0 2023-09-10T01:12:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.481+0000 7f99606c5b80 10 client.0 _ll_register_callbacks cb 0x55ecb301da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:12:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.481+0000 7f995229b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3300 0x55ecb31efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.481+0000 7f9951a9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e3cc0 0x55ecb31ed040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.481+0000 7f995229b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3300 0x55ecb31efd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56964/0 (socket says 172.21.15.44:56964) 2023-09-10T01:12:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.481+0000 7f995229b700 1 -- 172.21.15.44:0/2996996633 learned_addr learned my addr 172.21.15.44:0/2996996633 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.482+0000 7f9952a9c700 1 --2- 172.21.15.44:0/2996996633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e91a0 0x55ecb31ed580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.482+0000 7f995229b700 1 -- 172.21.15.44:0/2996996633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e91a0 msgr2=0x55ecb31ed580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.482+0000 7f995229b700 1 --2- 172.21.15.44:0/2996996633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e91a0 0x55ecb31ed580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.482+0000 7f995229b700 1 -- 172.21.15.44:0/2996996633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e3cc0 msgr2=0x55ecb31ed040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.482+0000 7f995229b700 1 --2- 172.21.15.44:0/2996996633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e3cc0 0x55ecb31ed040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.482+0000 7f995229b700 1 -- 172.21.15.44:0/2996996633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994c1a35e0 con 0x55ecb31e3300 2023-09-10T01:12:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.482+0000 7f995229b700 1 --2- 172.21.15.44:0/2996996633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3300 0x55ecb31efd20 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f994800edf0 tx=0x7f99480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59aa5cef18ce8ca8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.482+0000 7f9935ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:12:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.482+0000 7f9935ffb700 1 -- 172.21.15.44:0/2996996633 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994801b830 con 0x55ecb31e3300 2023-09-10T01:12:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.482+0000 7f9935ffb700 1 -- 172.21.15.44:0/2996996633 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f994800ee20 con 0x55ecb31e3300 2023-09-10T01:12:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.482+0000 7f99606c5b80 1 -- 172.21.15.44:0/2996996633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecb308d490 con 0x55ecb31e3300 2023-09-10T01:12:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.482+0000 7f9935ffb700 1 -- 172.21.15.44:0/2996996633 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f994801b990 con 0x55ecb31e3300 2023-09-10T01:12:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.483+0000 7f99606c5b80 10 client.19221 fetch_fsmap learned FSMap version 120 2023-09-10T01:12:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.483+0000 7f99606c5b80 1 -- 172.21.15.44:0/2996996633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x7f993c009960 con 0x55ecb31e3300 2023-09-10T01:12:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.483+0000 7f9935ffb700 1 -- 172.21.15.44:0/2996996633 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ecb308d490 con 0x55ecb31e3300 2023-09-10T01:12:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.483+0000 7f9935ffb700 1 -- 172.21.15.44:0/2996996633 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f9948025b10 con 0x55ecb31e3300 2023-09-10T01:12:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.483+0000 7f99606c5b80 10 client.19221 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:12:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.483+0000 7f99606c5b80 10 client.19221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:12:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.483+0000 7f99606c5b80 20 client.19221 populate_metadata read hostname 'smithi044' 2023-09-10T01:12:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.483+0000 7f99606c5b80 1 --2- 172.21.15.44:0/2996996633 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ecb32016d0 0x55ecb3203af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.483+0000 7f99606c5b80 4 client.19221 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:12:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.483+0000 7f99606c5b80 1 -- 172.21.15.44:0/2996996633 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecb314a840 con 0x55ecb32016d0 2023-09-10T01:12:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.483+0000 7f9951a9a700 1 --2- 172.21.15.44:0/2996996633 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ecb32016d0 0x55ecb3203af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.484+0000 7f9935ffb700 10 client.19221 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:12:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.484+0000 7f9951a9a700 1 --2- 172.21.15.44:0/2996996633 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ecb32016d0 0x55ecb3203af0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b6fd85f8a0ebe15 server_cookie=d1f034681a43c5e5 in_seq=0 out_seq=0 2023-09-10T01:12:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.484+0000 7f9935ffb700 1 -- 172.21.15.44:0/2996996633 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ecb314a840 con 0x55ecb32016d0 2023-09-10T01:12:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.484+0000 7f9935ffb700 10 client.19221 handle_command_reply: tid=0 2023-09-10T01:12:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.484+0000 7f99606c5b80 1 client.19221 shutdown 2023-09-10T01:12:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.484+0000 7f99606c5b80 1 -- 172.21.15.44:0/2996996633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3300 msgr2=0x55ecb31efd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.484+0000 7f99606c5b80 1 --2- 172.21.15.44:0/2996996633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3300 0x55ecb31efd20 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f994800edf0 tx=0x7f99480047c0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.485+0000 7f99606c5b80 1 -- 172.21.15.44:0/2996996633 shutdown_connections 2023-09-10T01:12:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.485+0000 7f99606c5b80 1 --2- 172.21.15.44:0/2996996633 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ecb32016d0 0x55ecb3203af0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.485+0000 7f99606c5b80 1 --2- 172.21.15.44:0/2996996633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e91a0 0x55ecb31ed580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.485+0000 7f99606c5b80 1 --2- 172.21.15.44:0/2996996633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3300 0x55ecb31efd20 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.485+0000 7f99606c5b80 1 --2- 172.21.15.44:0/2996996633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e3cc0 0x55ecb31ed040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.485+0000 7f99606c5b80 1 -- 172.21.15.44:0/2996996633 >> 172.21.15.44:0/2996996633 conn(0x55ecb3158030 msgr2=0x55ecb31dd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.485+0000 7f9935ffb700 0 client.19221 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:12:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.485+0000 7f99606c5b80 1 -- 172.21.15.44:0/2996996633 shutdown_connections 2023-09-10T01:12:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.485+0000 7f99606c5b80 1 -- 172.21.15.44:0/2996996633 wait complete. 2023-09-10T01:12:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.485+0000 7f99606c5b80 20 client.19221 trim_cache size 0 max 16384 2023-09-10T01:12:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.506+0000 7f99606c5b80 1 Processor -- start 2023-09-10T01:12:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.509+0000 7f99606c5b80 1 -- start start 2023-09-10T01:12:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.509+0000 7f99606c5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e3300 0x55ecb31f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.509+0000 7f99606c5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31ef850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.509+0000 7f99606c5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e91a0 0x55ecb31e5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.509+0000 7f99606c5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb31fedc0 con 0x55ecb31e3cc0 2023-09-10T01:12:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.509+0000 7f99606c5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb31e4580 con 0x55ecb31e91a0 2023-09-10T01:12:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.509+0000 7f99606c5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb31dfd10 con 0x55ecb31e3300 2023-09-10T01:12:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.509+0000 7f9951a9a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31ef850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.509+0000 7f995229b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e3300 0x55ecb31f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.509+0000 7f9951a9a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31ef850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56974/0 (socket says 172.21.15.44:56974) 2023-09-10T01:12:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.509+0000 7f9951a9a700 1 -- 172.21.15.44:0/4164704235 learned_addr learned my addr 172.21.15.44:0/4164704235 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.509+0000 7f9952a9c700 1 --2- 172.21.15.44:0/4164704235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e91a0 0x55ecb31e5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.510+0000 7f9951a9a700 1 -- 172.21.15.44:0/4164704235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e3300 msgr2=0x55ecb31f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.510+0000 7f9951a9a700 1 --2- 172.21.15.44:0/4164704235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e3300 0x55ecb31f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.510+0000 7f9951a9a700 1 -- 172.21.15.44:0/4164704235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e91a0 msgr2=0x55ecb31e5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.510+0000 7f9951a9a700 1 --2- 172.21.15.44:0/4164704235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e91a0 0x55ecb31e5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.510+0000 7f9951a9a700 1 -- 172.21.15.44:0/4164704235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecb2f4ac00 con 0x55ecb31e3cc0 2023-09-10T01:12:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.510+0000 7f9952a9c700 1 --2- 172.21.15.44:0/4164704235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e91a0 0x55ecb31e5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:12:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.510+0000 7f9951a9a700 1 --2- 172.21.15.44:0/4164704235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31ef850 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x55ecb31de060 tx=0x7f993c005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77e404f8569fe7a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.510+0000 7f99417fa700 1 -- 172.21.15.44:0/4164704235 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c029070 con 0x55ecb31e3cc0 2023-09-10T01:12:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.510+0000 7f99417fa700 1 -- 172.21.15.44:0/4164704235 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f993c0044e0 con 0x55ecb31e3cc0 2023-09-10T01:12:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.510+0000 7f99417fa700 1 -- 172.21.15.44:0/4164704235 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c005070 con 0x55ecb31e3cc0 2023-09-10T01:12:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.511+0000 7f99606c5b80 1 -- 172.21.15.44:0/4164704235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 msgr2=0x55ecb31ef850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.511+0000 7f99606c5b80 1 --2- 172.21.15.44:0/4164704235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31ef850 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x55ecb31de060 tx=0x7f993c005000 comp rx=0 tx=0).stop 2023-09-10T01:12:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.511+0000 7f99606c5b80 1 -- 172.21.15.44:0/4164704235 shutdown_connections 2023-09-10T01:12:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.511+0000 7f99606c5b80 1 --2- 172.21.15.44:0/4164704235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e91a0 0x55ecb31e5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.511+0000 7f99606c5b80 1 --2- 172.21.15.44:0/4164704235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e3300 0x55ecb31f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.511+0000 7f99606c5b80 1 --2- 172.21.15.44:0/4164704235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31ef850 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.511+0000 7f99606c5b80 1 -- 172.21.15.44:0/4164704235 >> 172.21.15.44:0/4164704235 conn(0x55ecb3158030 msgr2=0x55ecb31de090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.511+0000 7f99606c5b80 1 -- 172.21.15.44:0/4164704235 shutdown_connections 2023-09-10T01:12:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.511+0000 7f99606c5b80 1 -- 172.21.15.44:0/4164704235 wait complete. 2023-09-10T01:12:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.511+0000 7f99606c5b80 1 -- start start 2023-09-10T01:12:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f99606c5b80 1 Processor -- start 2023-09-10T01:12:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f99606c5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e3300 0x55ecb31f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f99606c5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31f8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f99606c5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e91a0 0x55ecb31f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f99606c5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb31f70b0 con 0x55ecb31e3cc0 2023-09-10T01:12:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f99606c5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb31f7220 con 0x55ecb31e91a0 2023-09-10T01:12:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f99606c5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb31f73e0 con 0x55ecb31e3300 2023-09-10T01:12:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f99606c5b80 10 client.0 _ll_register_callbacks cb 0x7f994800e9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:12:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f9951a9a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f995229b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e3300 0x55ecb31f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f995229b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e3300 0x55ecb31f81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35628/0 (socket says 172.21.15.44:35628) 2023-09-10T01:12:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f9951a9a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31f8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56980/0 (socket says 172.21.15.44:56980) 2023-09-10T01:12:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f995229b700 1 -- 172.21.15.44:0/1993300835 learned_addr learned my addr 172.21.15.44:0/1993300835 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f995229b700 1 -- 172.21.15.44:0/1993300835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e91a0 msgr2=0x55ecb31f69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f995229b700 1 --2- 172.21.15.44:0/1993300835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e91a0 0x55ecb31f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f995229b700 1 -- 172.21.15.44:0/1993300835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 msgr2=0x55ecb31f8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f995229b700 1 --2- 172.21.15.44:0/1993300835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31f8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f995229b700 1 -- 172.21.15.44:0/1993300835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994c197070 con 0x55ecb31e3300 2023-09-10T01:12:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.512+0000 7f9951a9a700 1 --2- 172.21.15.44:0/1993300835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31f8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:12:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.513+0000 7f995229b700 1 --2- 172.21.15.44:0/1993300835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e3300 0x55ecb31f81e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9948009cf0 tx=0x7f9948004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a52630be480742e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.513+0000 7f9935ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:12:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.513+0000 7f9935ffb700 1 -- 172.21.15.44:0/1993300835 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994801ac60 con 0x55ecb31e3300 2023-09-10T01:12:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.513+0000 7f9935ffb700 1 -- 172.21.15.44:0/1993300835 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f994800e460 con 0x55ecb31e3300 2023-09-10T01:12:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.513+0000 7f99606c5b80 1 -- 172.21.15.44:0/1993300835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f993c004cc0 con 0x55ecb31e3300 2023-09-10T01:12:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.513+0000 7f9935ffb700 1 -- 172.21.15.44:0/1993300835 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f994801da80 con 0x55ecb31e3300 2023-09-10T01:12:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.513+0000 7f99606c5b80 10 client.28837 fetch_fsmap learned FSMap version 120 2023-09-10T01:12:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.513+0000 7f99606c5b80 1 -- 172.21.15.44:0/1993300835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x7f993c01d040 con 0x55ecb31e3300 2023-09-10T01:12:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.513+0000 7f9935ffb700 1 -- 172.21.15.44:0/1993300835 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f993c004cc0 con 0x55ecb31e3300 2023-09-10T01:12:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.514+0000 7f9935ffb700 1 -- 172.21.15.44:0/1993300835 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f994802d070 con 0x55ecb31e3300 2023-09-10T01:12:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.514+0000 7f99606c5b80 10 client.28837 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:12:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.514+0000 7f99606c5b80 10 client.28837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:12:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.514+0000 7f99606c5b80 20 client.28837 populate_metadata read hostname 'smithi044' 2023-09-10T01:12:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.514+0000 7f99606c5b80 1 --2- 172.21.15.44:0/1993300835 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ecb32567e0 0x55ecb3258c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.514+0000 7f99606c5b80 4 client.28837 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:12:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.514+0000 7f99606c5b80 1 -- 172.21.15.44:0/1993300835 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ecb312e8c0 con 0x55ecb32567e0 2023-09-10T01:12:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.514+0000 7f9951a9a700 1 --2- 172.21.15.44:0/1993300835 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ecb32567e0 0x55ecb3258c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.514+0000 7f9935ffb700 10 client.28837 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:12:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.514+0000 7f9951a9a700 1 --2- 172.21.15.44:0/1993300835 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ecb32567e0 0x55ecb3258c00 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=977c29afd33b83ce server_cookie=5062149eece8ac03 in_seq=0 out_seq=0 2023-09-10T01:12:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.526+0000 7f9935ffb700 1 -- 172.21.15.44:0/1993300835 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ecb312e8c0 con 0x55ecb32567e0 2023-09-10T01:12:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.526+0000 7f9935ffb700 10 client.28837 handle_command_reply: tid=0 2023-09-10T01:12:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.526+0000 7f99606c5b80 1 client.28837 shutdown 2023-09-10T01:12:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.526+0000 7f99606c5b80 1 -- 172.21.15.44:0/1993300835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e3300 msgr2=0x55ecb31f81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.526+0000 7f99606c5b80 1 --2- 172.21.15.44:0/1993300835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e3300 0x55ecb31f81e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9948009cf0 tx=0x7f9948004730 comp rx=0 tx=0).stop 2023-09-10T01:12:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.526+0000 7f99606c5b80 1 -- 172.21.15.44:0/1993300835 shutdown_connections 2023-09-10T01:12:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.526+0000 7f99606c5b80 1 --2- 172.21.15.44:0/1993300835 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ecb32567e0 0x55ecb3258c00 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.526+0000 7f99606c5b80 1 --2- 172.21.15.44:0/1993300835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ecb31e91a0 0x55ecb31f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.526+0000 7f9935ffb700 0 client.28837 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:12:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.526+0000 7f99606c5b80 1 --2- 172.21.15.44:0/1993300835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ecb31e3300 0x55ecb31f81e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.526+0000 7f99606c5b80 1 --2- 172.21.15.44:0/1993300835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecb31e3cc0 0x55ecb31f8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.526+0000 7f99606c5b80 1 -- 172.21.15.44:0/1993300835 >> 172.21.15.44:0/1993300835 conn(0x55ecb3158030 msgr2=0x55ecb31e22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.526+0000 7f99606c5b80 1 -- 172.21.15.44:0/1993300835 shutdown_connections 2023-09-10T01:12:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.526+0000 7f99606c5b80 1 -- 172.21.15.44:0/1993300835 wait complete. 2023-09-10T01:12:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.527+0000 7f99606c5b80 20 client.28837 trim_cache size 0 max 16384 2023-09-10T01:12:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:12:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:12:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:12:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:12:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.527+0000 7f99544ff700 1 -- 172.21.15.44:0/3278212373 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f99380a0830 msgr2=0x7f99380a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.527+0000 7f99544ff700 1 --2- 172.21.15.44:0/3278212373 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f99380a0830 0x7f99380a2cf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9948011550 tx=0x7f9948009260 comp rx=0 tx=0).stop 2023-09-10T01:12:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.527+0000 7f99544ff700 1 -- 172.21.15.44:0/3278212373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c1055b0 msgr2=0x7f994c196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.527+0000 7f99544ff700 1 --2- 172.21.15.44:0/3278212373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c1055b0 0x7f994c196450 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f994400b6c0 tx=0x7f9944008aa0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.527+0000 7f99544ff700 1 -- 172.21.15.44:0/3278212373 shutdown_connections 2023-09-10T01:12:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.527+0000 7f99544ff700 1 --2- 172.21.15.44:0/3278212373 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f99380a0830 0x7f99380a2cf0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.527+0000 7f99544ff700 1 --2- 172.21.15.44:0/3278212373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c1055b0 0x7f994c196450 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.527+0000 7f99544ff700 1 --2- 172.21.15.44:0/3278212373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c0ff760 0x7f994c19c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.527+0000 7f99544ff700 1 --2- 172.21.15.44:0/3278212373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f994c100120 0x7f994c19cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.527+0000 7f99544ff700 1 -- 172.21.15.44:0/3278212373 >> 172.21.15.44:0/3278212373 conn(0x7f994c074f30 msgr2=0x7f994c103200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.527+0000 7f99544ff700 1 -- 172.21.15.44:0/3278212373 shutdown_connections 2023-09-10T01:12:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:22.527+0000 7f99544ff700 1 -- 172.21.15.44:0/3278212373 wait complete. 2023-09-10T01:12:24.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:23 smithi131 ceph-mon[103713]: pgmap v1011: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1000 KiB/s wr, 142 op/s 2023-09-10T01:12:24.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:23 smithi044 ceph-mon[98676]: pgmap v1011: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1000 KiB/s wr, 142 op/s 2023-09-10T01:12:24.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:23 smithi099 ceph-mon[102692]: pgmap v1011: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1000 KiB/s wr, 142 op/s 2023-09-10T01:12:26.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:25 smithi131 ceph-mon[103713]: pgmap v1012: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 984 KiB/s wr, 139 op/s 2023-09-10T01:12:26.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:25 smithi044 ceph-mon[98676]: pgmap v1012: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 984 KiB/s wr, 139 op/s 2023-09-10T01:12:26.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:25 smithi099 ceph-mon[102692]: pgmap v1012: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 984 KiB/s wr, 139 op/s 2023-09-10T01:12:28.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:27 smithi131 ceph-mon[103713]: pgmap v1013: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 203 op/s 2023-09-10T01:12:28.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:27 smithi044 ceph-mon[98676]: pgmap v1013: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 203 op/s 2023-09-10T01:12:28.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:27 smithi099 ceph-mon[102692]: pgmap v1013: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 203 op/s 2023-09-10T01:12:30.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:29 smithi131 ceph-mon[103713]: pgmap v1014: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 931 KiB/s wr, 133 op/s 2023-09-10T01:12:30.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:29 smithi044 ceph-mon[98676]: pgmap v1014: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 931 KiB/s wr, 133 op/s 2023-09-10T01:12:30.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:29 smithi099 ceph-mon[102692]: pgmap v1014: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 931 KiB/s wr, 133 op/s 2023-09-10T01:12:32.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:31 smithi131 ceph-mon[103713]: pgmap v1015: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 195 op/s 2023-09-10T01:12:32.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:31 smithi044 ceph-mon[98676]: pgmap v1015: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 195 op/s 2023-09-10T01:12:32.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:31 smithi099 ceph-mon[102692]: pgmap v1015: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 195 op/s 2023-09-10T01:12:34.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:33 smithi131 ceph-mon[103713]: pgmap v1016: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 868 KiB/s wr, 124 op/s 2023-09-10T01:12:34.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:33 smithi044 ceph-mon[98676]: pgmap v1016: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 868 KiB/s wr, 124 op/s 2023-09-10T01:12:34.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:33 smithi099 ceph-mon[102692]: pgmap v1016: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 868 KiB/s wr, 124 op/s 2023-09-10T01:12:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.658+0000 7f1f59a55700 1 -- 172.21.15.44:0/2499737778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f54068f30 msgr2=0x7f1f5410d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.658+0000 7f1f59a55700 1 --2- 172.21.15.44:0/2499737778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f54068f30 0x7f1f5410d450 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00b0d0 tx=0x7f1f4c00b3e0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.658+0000 7f1f59a55700 1 -- 172.21.15.44:0/2499737778 shutdown_connections 2023-09-10T01:12:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.658+0000 7f1f59a55700 1 --2- 172.21.15.44:0/2499737778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f54068f30 0x7f1f5410d450 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.658+0000 7f1f59a55700 1 --2- 172.21.15.44:0/2499737778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f541066d0 0x7f1f54106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.658+0000 7f1f59a55700 1 --2- 172.21.15.44:0/2499737778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f54068570 0x7f1f540689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.658+0000 7f1f59a55700 1 -- 172.21.15.44:0/2499737778 >> 172.21.15.44:0/2499737778 conn(0x7f1f540773d0 msgr2=0x7f1f540777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.658+0000 7f1f59a55700 1 -- 172.21.15.44:0/2499737778 shutdown_connections 2023-09-10T01:12:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.658+0000 7f1f59a55700 1 -- 172.21.15.44:0/2499737778 wait complete. 2023-09-10T01:12:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.659+0000 7f1f59a55700 1 Processor -- start 2023-09-10T01:12:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.659+0000 7f1f59a55700 1 -- start start 2023-09-10T01:12:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.659+0000 7f1f59a55700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f54068570 0x7f1f5419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f59a55700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f54068f30 0x7f1f5419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f59a55700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f541066d0 0x7f1f54196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f59a55700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f54196c10 con 0x7f1f541066d0 2023-09-10T01:12:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f59a55700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f54196d80 con 0x7f1f54068f30 2023-09-10T01:12:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f59a55700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f54196f40 con 0x7f1f54068570 2023-09-10T01:12:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f52ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f54068570 0x7f1f5419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f52ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f54068570 0x7f1f5419c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35640/0 (socket says 172.21.15.44:35640) 2023-09-10T01:12:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f52ffd700 1 -- 172.21.15.44:0/1749569127 learned_addr learned my addr 172.21.15.44:0/1749569127 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f537fe700 1 --2- 172.21.15.44:0/1749569127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f541066d0 0x7f1f54196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f527fc700 1 --2- 172.21.15.44:0/1749569127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f54068f30 0x7f1f5419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f52ffd700 1 -- 172.21.15.44:0/1749569127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f54068f30 msgr2=0x7f1f5419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f52ffd700 1 --2- 172.21.15.44:0/1749569127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f54068f30 0x7f1f5419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f52ffd700 1 -- 172.21.15.44:0/1749569127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f541066d0 msgr2=0x7f1f54196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f52ffd700 1 --2- 172.21.15.44:0/1749569127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f541066d0 0x7f1f54196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f52ffd700 1 -- 172.21.15.44:0/1749569127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c009d00 con 0x7f1f54068570 2023-09-10T01:12:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f527fc700 1 --2- 172.21.15.44:0/1749569127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f54068f30 0x7f1f5419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:12:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.660+0000 7f1f52ffd700 1 --2- 172.21.15.44:0/1749569127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f54068570 0x7f1f5419c6e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400e9a0 tx=0x7f1f4400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.661+0000 7f1f58a53700 1 -- 172.21.15.44:0/1749569127 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44009790 con 0x7f1f54068570 2023-09-10T01:12:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.661+0000 7f1f58a53700 1 -- 172.21.15.44:0/1749569127 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f4400cbd0 con 0x7f1f54068570 2023-09-10T01:12:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.661+0000 7f1f58a53700 1 -- 172.21.15.44:0/1749569127 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44018680 con 0x7f1f54068570 2023-09-10T01:12:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.661+0000 7f1f59a55700 1 -- 172.21.15.44:0/1749569127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f54197220 con 0x7f1f54068570 2023-09-10T01:12:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.661+0000 7f1f59a55700 1 -- 172.21.15.44:0/1749569127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f541a3710 con 0x7f1f54068570 2023-09-10T01:12:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.664+0000 7f1f58a53700 1 -- 172.21.15.44:0/1749569127 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1f4400cd40 con 0x7f1f54068570 2023-09-10T01:12:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.664+0000 7f1f58a53700 1 --2- 172.21.15.44:0/1749569127 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1f3c0a07b0 0x7f1f3c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.664+0000 7f1f58a53700 1 -- 172.21.15.44:0/1749569127 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1f44014070 con 0x7f1f54068570 2023-09-10T01:12:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.664+0000 7f1f527fc700 1 --2- 172.21.15.44:0/1749569127 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1f3c0a07b0 0x7f1f3c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.665+0000 7f1f65c1bb80 1 Processor -- start 2023-09-10T01:12:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.665+0000 7f1f65c1bb80 1 -- start start 2023-09-10T01:12:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.665+0000 7f1f527fc700 1 --2- 172.21.15.44:0/1749569127 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1f3c0a07b0 0x7f1f3c0a2c70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1f48005fd0 tx=0x7f1f48005ee0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.665+0000 7f1f65c1bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce95270 0x55e8bce91300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.665+0000 7f1f65c1bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bce91840 0x55e8bce91cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.665+0000 7f1f65c1bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bcea3f60 0x55e8bcea6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.665+0000 7f1f65c1bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bcdea6d0 con 0x55e8bcea3f60 2023-09-10T01:12:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.665+0000 7f1f65c1bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bcde7ff0 con 0x55e8bce91840 2023-09-10T01:12:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.665+0000 7f1f65c1bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bcde38e0 con 0x55e8bce95270 2023-09-10T01:12:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.665+0000 7f1f52ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce95270 0x55e8bce91300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.665+0000 7f1f537fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bcea3f60 0x55e8bcea6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.665+0000 7f1f52ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce95270 0x55e8bce91300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35648/0 (socket says 172.21.15.44:35648) 2023-09-10T01:12:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.665+0000 7f1f537fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bcea3f60 0x55e8bcea6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57002/0 (socket says 172.21.15.44:57002) 2023-09-10T01:12:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.665+0000 7f1f52ffd700 1 -- 172.21.15.44:0/2714950951 learned_addr learned my addr 172.21.15.44:0/2714950951 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.666+0000 7f1f527fc700 1 --2- 172.21.15.44:0/2714950951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bce91840 0x55e8bce91cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.666+0000 7f1f537fe700 1 -- 172.21.15.44:0/2714950951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce95270 msgr2=0x55e8bce91300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.666+0000 7f1f537fe700 1 --2- 172.21.15.44:0/2714950951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce95270 0x55e8bce91300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.666+0000 7f1f537fe700 1 -- 172.21.15.44:0/2714950951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bce91840 msgr2=0x55e8bce91cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.666+0000 7f1f537fe700 1 --2- 172.21.15.44:0/2714950951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bce91840 0x55e8bce91cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.666+0000 7f1f537fe700 1 -- 172.21.15.44:0/2714950951 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8bcc0fd80 con 0x55e8bcea3f60 2023-09-10T01:12:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.666+0000 7f1f52ffd700 1 --2- 172.21.15.44:0/2714950951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce95270 0x55e8bce91300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:12:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.666+0000 7f1f537fe700 1 --2- 172.21.15.44:0/2714950951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bcea3f60 0x55e8bcea6350 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00b770 tx=0x7f1f4c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8192239f3f188a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.666+0000 7f1f39ffb700 1 -- 172.21.15.44:0/2714950951 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00e040 con 0x55e8bcea3f60 2023-09-10T01:12:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.666+0000 7f1f39ffb700 1 -- 172.21.15.44:0/2714950951 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f4c007b90 con 0x55e8bcea3f60 2023-09-10T01:12:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.666+0000 7f1f39ffb700 1 -- 172.21.15.44:0/2714950951 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c003a00 con 0x55e8bcea3f60 2023-09-10T01:12:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.667+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/2714950951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bcea3f60 msgr2=0x55e8bcea6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.667+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/2714950951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bcea3f60 0x55e8bcea6350 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00b770 tx=0x7f1f4c009b80 comp rx=0 tx=0).stop 2023-09-10T01:12:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.667+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/2714950951 shutdown_connections 2023-09-10T01:12:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.667+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/2714950951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bcea3f60 0x55e8bcea6350 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.667+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/2714950951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce95270 0x55e8bce91300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.667+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/2714950951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bce91840 0x55e8bce91cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.667+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/2714950951 >> 172.21.15.44:0/2714950951 conn(0x55e8bce06030 msgr2=0x55e8bce8c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.667+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/2714950951 shutdown_connections 2023-09-10T01:12:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.667+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/2714950951 wait complete. 2023-09-10T01:12:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.668+0000 7f1f65c1bb80 1 -- start start 2023-09-10T01:12:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.668+0000 7f1f65c1bb80 1 Processor -- start 2023-09-10T01:12:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.668+0000 7f1f65c1bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce91840 0x55e8bce9d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.668+0000 7f1f65c1bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bce95270 0x55e8bce9d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.668+0000 7f1f65c1bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bcea3f60 0x55e8bcea31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.668+0000 7f1f65c1bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bcd81b10 con 0x55e8bce91840 2023-09-10T01:12:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.668+0000 7f1f65c1bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bcd81810 con 0x55e8bce95270 2023-09-10T01:12:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.668+0000 7f1f65c1bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bcc658c0 con 0x55e8bcea3f60 2023-09-10T01:12:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.668+0000 7f1f65c1bb80 10 client.0 _ll_register_callbacks cb 0x55e8bcccba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:12:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.668+0000 7f1f52ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce91840 0x55e8bce9d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.668+0000 7f1f527fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bce95270 0x55e8bce9d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.668+0000 7f1f52ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce91840 0x55e8bce9d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57004/0 (socket says 172.21.15.44:57004) 2023-09-10T01:12:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.668+0000 7f1f52ffd700 1 -- 172.21.15.44:0/1064276491 learned_addr learned my addr 172.21.15.44:0/1064276491 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.668+0000 7f1f527fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bce95270 0x55e8bce9d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58746/0 (socket says 172.21.15.44:58746) 2023-09-10T01:12:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.668+0000 7f1f537fe700 1 --2- 172.21.15.44:0/1064276491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bcea3f60 0x55e8bcea31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.669+0000 7f1f52ffd700 1 -- 172.21.15.44:0/1064276491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bcea3f60 msgr2=0x55e8bcea31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.669+0000 7f1f52ffd700 1 --2- 172.21.15.44:0/1064276491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bcea3f60 0x55e8bcea31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.669+0000 7f1f52ffd700 1 -- 172.21.15.44:0/1064276491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bce95270 msgr2=0x55e8bce9d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.669+0000 7f1f52ffd700 1 --2- 172.21.15.44:0/1064276491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bce95270 0x55e8bce9d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.669+0000 7f1f52ffd700 1 -- 172.21.15.44:0/1064276491 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f541a3710 con 0x55e8bce91840 2023-09-10T01:12:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.669+0000 7f1f527fc700 1 --2- 172.21.15.44:0/1064276491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bce95270 0x55e8bce9d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:12:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.669+0000 7f1f52ffd700 1 --2- 172.21.15.44:0/1064276491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce91840 0x55e8bce9d4b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f1f440c99d0 tx=0x7f1f440990d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5829b7b6d547f49 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.669+0000 7f1f32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:12:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.669+0000 7f1f32ffd700 1 -- 172.21.15.44:0/1064276491 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44099680 con 0x55e8bce91840 2023-09-10T01:12:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.669+0000 7f1f32ffd700 1 -- 172.21.15.44:0/1064276491 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1f4409d4b0 con 0x55e8bce91840 2023-09-10T01:12:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.669+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/1064276491 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8bcd3b490 con 0x55e8bce91840 2023-09-10T01:12:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.669+0000 7f1f32ffd700 1 -- 172.21.15.44:0/1064276491 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f440a16f0 con 0x55e8bce91840 2023-09-10T01:12:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.669+0000 7f1f65c1bb80 10 client.19239 fetch_fsmap learned FSMap version 120 2023-09-10T01:12:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.669+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/1064276491 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x7f1f5410dce0 con 0x55e8bce91840 2023-09-10T01:12:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.670+0000 7f1f32ffd700 1 -- 172.21.15.44:0/1064276491 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e8bcd3b490 con 0x55e8bce91840 2023-09-10T01:12:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.670+0000 7f1f32ffd700 1 -- 172.21.15.44:0/1064276491 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f1f440a1260 con 0x55e8bce91840 2023-09-10T01:12:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.670+0000 7f1f65c1bb80 10 client.19239 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:12:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.670+0000 7f1f65c1bb80 10 client.19239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:12:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.670+0000 7f1f65c1bb80 20 client.19239 populate_metadata read hostname 'smithi044' 2023-09-10T01:12:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.670+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/1064276491 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e8bcebc3b0 0x55e8bcebe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.670+0000 7f1f65c1bb80 4 client.19239 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:12:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.670+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/1064276491 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8bcdf8840 con 0x55e8bcebc3b0 2023-09-10T01:12:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.670+0000 7f1f527fc700 1 --2- 172.21.15.44:0/1064276491 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e8bcebc3b0 0x55e8bcebe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.671+0000 7f1f32ffd700 10 client.19239 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:12:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.671+0000 7f1f527fc700 1 --2- 172.21.15.44:0/1064276491 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e8bcebc3b0 0x55e8bcebe7d0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88d3a230592dc9c4 server_cookie=6c259429fdb00951 in_seq=0 out_seq=0 2023-09-10T01:12:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.671+0000 7f1f32ffd700 1 -- 172.21.15.44:0/1064276491 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e8bcdf8840 con 0x55e8bcebc3b0 2023-09-10T01:12:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.671+0000 7f1f32ffd700 10 client.19239 handle_command_reply: tid=0 2023-09-10T01:12:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.671+0000 7f1f65c1bb80 1 client.19239 shutdown 2023-09-10T01:12:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.671+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/1064276491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce91840 msgr2=0x55e8bce9d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.671+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/1064276491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce91840 0x55e8bce9d4b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f1f440c99d0 tx=0x7f1f440990d0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.671+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/1064276491 shutdown_connections 2023-09-10T01:12:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.671+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/1064276491 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e8bcebc3b0 0x55e8bcebe7d0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.671+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/1064276491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bcea3f60 0x55e8bcea31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.671+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/1064276491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce91840 0x55e8bce9d4b0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.671+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/1064276491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bce95270 0x55e8bce9d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.672+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/1064276491 >> 172.21.15.44:0/1064276491 conn(0x55e8bce06030 msgr2=0x55e8bce8b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.672+0000 7f1f32ffd700 0 client.19239 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:12:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.672+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/1064276491 shutdown_connections 2023-09-10T01:12:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.672+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/1064276491 wait complete. 2023-09-10T01:12:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.672+0000 7f1f65c1bb80 20 client.19239 trim_cache size 0 max 16384 2023-09-10T01:12:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.689+0000 7f1f65c1bb80 1 Processor -- start 2023-09-10T01:12:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f65c1bb80 1 -- start start 2023-09-10T01:12:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f65c1bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce91840 0x55e8bce96390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f65c1bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce95270 0x55e8bce98650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f65c1bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bcea3f60 0x55e8bce975f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f65c1bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bce92580 con 0x55e8bce91840 2023-09-10T01:12:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f65c1bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bce88fc0 con 0x55e8bcea3f60 2023-09-10T01:12:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f65c1bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bce98b90 con 0x55e8bce95270 2023-09-10T01:12:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f52ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce91840 0x55e8bce96390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f52ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce91840 0x55e8bce96390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57012/0 (socket says 172.21.15.44:57012) 2023-09-10T01:12:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f52ffd700 1 -- 172.21.15.44:0/185127010 learned_addr learned my addr 172.21.15.44:0/185127010 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f527fc700 1 --2- 172.21.15.44:0/185127010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce95270 0x55e8bce98650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f537fe700 1 --2- 172.21.15.44:0/185127010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bcea3f60 0x55e8bce975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f52ffd700 1 -- 172.21.15.44:0/185127010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce95270 msgr2=0x55e8bce98650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f52ffd700 1 --2- 172.21.15.44:0/185127010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce95270 0x55e8bce98650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f52ffd700 1 -- 172.21.15.44:0/185127010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bcea3f60 msgr2=0x55e8bce975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f52ffd700 1 --2- 172.21.15.44:0/185127010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bcea3f60 0x55e8bce975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f52ffd700 1 -- 172.21.15.44:0/185127010 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8bcbf8c00 con 0x55e8bce91840 2023-09-10T01:12:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.690+0000 7f1f52ffd700 1 --2- 172.21.15.44:0/185127010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce91840 0x55e8bce96390 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1f440960f0 tx=0x7f1f44095ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1579890769acca server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.691+0000 7f1f3a7fc700 1 -- 172.21.15.44:0/185127010 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f440cc480 con 0x55e8bce91840 2023-09-10T01:12:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.691+0000 7f1f3a7fc700 1 -- 172.21.15.44:0/185127010 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f4409d4b0 con 0x55e8bce91840 2023-09-10T01:12:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.691+0000 7f1f3a7fc700 1 -- 172.21.15.44:0/185127010 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f440a16f0 con 0x55e8bce91840 2023-09-10T01:12:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.692+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/185127010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce91840 msgr2=0x55e8bce96390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.692+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/185127010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce91840 0x55e8bce96390 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1f440960f0 tx=0x7f1f44095ad0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.692+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/185127010 shutdown_connections 2023-09-10T01:12:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.692+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/185127010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bcea3f60 0x55e8bce975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.692+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/185127010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce91840 0x55e8bce96390 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.692+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/185127010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce95270 0x55e8bce98650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.692+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/185127010 >> 172.21.15.44:0/185127010 conn(0x55e8bce06030 msgr2=0x55e8bce96ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.692+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/185127010 shutdown_connections 2023-09-10T01:12:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.692+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/185127010 wait complete. 2023-09-10T01:12:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.692+0000 7f1f65c1bb80 1 -- start start 2023-09-10T01:12:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.692+0000 7f1f65c1bb80 1 Processor -- start 2023-09-10T01:12:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f65c1bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce91840 0x55e8bcea2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f65c1bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce95270 0x55e8bce9ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f65c1bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bcea3f60 0x55e8bcea0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f65c1bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bcea0e60 con 0x55e8bce95270 2023-09-10T01:12:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f65c1bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bcf0f200 con 0x55e8bcea3f60 2023-09-10T01:12:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f65c1bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bcf0f340 con 0x55e8bce91840 2023-09-10T01:12:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f65c1bb80 10 client.0 _ll_register_callbacks cb 0x7f1f440994e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:12:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f527fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce95270 0x55e8bce9ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f527fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce95270 0x55e8bce9ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57020/0 (socket says 172.21.15.44:57020) 2023-09-10T01:12:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f52ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce91840 0x55e8bcea2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f527fc700 1 -- 172.21.15.44:0/3484379303 learned_addr learned my addr 172.21.15.44:0/3484379303 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f537fe700 1 --2- 172.21.15.44:0/3484379303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bcea3f60 0x55e8bcea0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f527fc700 1 -- 172.21.15.44:0/3484379303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce91840 msgr2=0x55e8bcea2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f527fc700 1 --2- 172.21.15.44:0/3484379303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce91840 0x55e8bcea2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f527fc700 1 -- 172.21.15.44:0/3484379303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bcea3f60 msgr2=0x55e8bcea0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f527fc700 1 --2- 172.21.15.44:0/3484379303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bcea3f60 0x55e8bcea0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f527fc700 1 -- 172.21.15.44:0/3484379303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5410dce0 con 0x55e8bce95270 2023-09-10T01:12:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f537fe700 1 --2- 172.21.15.44:0/3484379303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bcea3f60 0x55e8bcea0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:12:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f527fc700 1 --2- 172.21.15.44:0/3484379303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce95270 0x55e8bce9ffe0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f1f48013040 tx=0x7f1f480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665d0e26b5257bdf server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.693+0000 7f1f32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:12:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.694+0000 7f1f32ffd700 1 -- 172.21.15.44:0/3484379303 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f48025070 con 0x55e8bce95270 2023-09-10T01:12:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.694+0000 7f1f32ffd700 1 -- 172.21.15.44:0/3484379303 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f1f48018040 con 0x55e8bce95270 2023-09-10T01:12:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.694+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/3484379303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8bce92140 con 0x55e8bce95270 2023-09-10T01:12:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.694+0000 7f1f32ffd700 1 -- 172.21.15.44:0/3484379303 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f48027550 con 0x55e8bce95270 2023-09-10T01:12:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.694+0000 7f1f65c1bb80 10 client.19245 fetch_fsmap learned FSMap version 120 2023-09-10T01:12:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.694+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/3484379303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x55e8bcc361f0 con 0x55e8bce95270 2023-09-10T01:12:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.694+0000 7f1f32ffd700 1 -- 172.21.15.44:0/3484379303 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e8bce92140 con 0x55e8bce95270 2023-09-10T01:12:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.694+0000 7f1f32ffd700 1 -- 172.21.15.44:0/3484379303 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f1f4802d030 con 0x55e8bce95270 2023-09-10T01:12:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.694+0000 7f1f65c1bb80 10 client.19245 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:12:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.694+0000 7f1f65c1bb80 10 client.19245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:12:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.694+0000 7f1f65c1bb80 20 client.19245 populate_metadata read hostname 'smithi044' 2023-09-10T01:12:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.694+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/3484379303 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e8bcf10b70 0x55e8bcf12f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.694+0000 7f1f65c1bb80 4 client.19245 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:12:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.694+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/3484379303 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f1f4c007b90 con 0x55e8bcf10b70 2023-09-10T01:12:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.695+0000 7f1f52ffd700 1 --2- 172.21.15.44:0/3484379303 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e8bcf10b70 0x55e8bcf12f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.695+0000 7f1f32ffd700 10 client.19245 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:12:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.695+0000 7f1f52ffd700 1 --2- 172.21.15.44:0/3484379303 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e8bcf10b70 0x55e8bcf12f90 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b5e4348e5ae8e4c server_cookie=18ef5e9d329c4d42 in_seq=0 out_seq=0 2023-09-10T01:12:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.705+0000 7f1f32ffd700 1 -- 172.21.15.44:0/3484379303 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f1f4c007b90 con 0x55e8bcf10b70 2023-09-10T01:12:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.705+0000 7f1f32ffd700 10 client.19245 handle_command_reply: tid=0 2023-09-10T01:12:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.705+0000 7f1f65c1bb80 1 client.19245 shutdown 2023-09-10T01:12:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.705+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/3484379303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce95270 msgr2=0x55e8bce9ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.705+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/3484379303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce95270 0x55e8bce9ffe0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f1f48013040 tx=0x7f1f480047c0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.705+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/3484379303 shutdown_connections 2023-09-10T01:12:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.705+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/3484379303 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e8bcf10b70 0x55e8bcf12f90 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.705+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/3484379303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8bcea3f60 0x55e8bcea0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.705+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/3484379303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8bce91840 0x55e8bcea2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.705+0000 7f1f65c1bb80 1 --2- 172.21.15.44:0/3484379303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8bce95270 0x55e8bce9ffe0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.705+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/3484379303 >> 172.21.15.44:0/3484379303 conn(0x55e8bce06030 msgr2=0x55e8bce96200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.705+0000 7f1f32ffd700 0 client.19245 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:12:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.705+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/3484379303 shutdown_connections 2023-09-10T01:12:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.706+0000 7f1f65c1bb80 1 -- 172.21.15.44:0/3484379303 wait complete. 2023-09-10T01:12:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.706+0000 7f1f65c1bb80 20 client.19245 trim_cache size 0 max 16384 2023-09-10T01:12:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:12:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:12:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:12:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:12:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.706+0000 7f1f59a55700 1 -- 172.21.15.44:0/1749569127 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1f3c0a07b0 msgr2=0x7f1f3c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.706+0000 7f1f59a55700 1 --2- 172.21.15.44:0/1749569127 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1f3c0a07b0 0x7f1f3c0a2c70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1f48005fd0 tx=0x7f1f48005ee0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.706+0000 7f1f59a55700 1 -- 172.21.15.44:0/1749569127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f54068570 msgr2=0x7f1f5419c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.706+0000 7f1f59a55700 1 --2- 172.21.15.44:0/1749569127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f54068570 0x7f1f5419c6e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400e9a0 tx=0x7f1f4400ecb0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.706+0000 7f1f59a55700 1 -- 172.21.15.44:0/1749569127 shutdown_connections 2023-09-10T01:12:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.706+0000 7f1f59a55700 1 --2- 172.21.15.44:0/1749569127 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f1f3c0a07b0 0x7f1f3c0a2c70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.706+0000 7f1f59a55700 1 --2- 172.21.15.44:0/1749569127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f541066d0 0x7f1f54196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.706+0000 7f1f59a55700 1 --2- 172.21.15.44:0/1749569127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f54068570 0x7f1f5419c6e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.706+0000 7f1f59a55700 1 --2- 172.21.15.44:0/1749569127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f54068f30 0x7f1f5419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.706+0000 7f1f59a55700 1 -- 172.21.15.44:0/1749569127 >> 172.21.15.44:0/1749569127 conn(0x7f1f540773d0 msgr2=0x7f1f540fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.706+0000 7f1f59a55700 1 -- 172.21.15.44:0/1749569127 shutdown_connections 2023-09-10T01:12:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:35.706+0000 7f1f59a55700 1 -- 172.21.15.44:0/1749569127 wait complete. 2023-09-10T01:12:36.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:35 smithi131 ceph-mon[103713]: pgmap v1017: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 868 KiB/s wr, 124 op/s 2023-09-10T01:12:36.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:35 smithi044 ceph-mon[98676]: pgmap v1017: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 868 KiB/s wr, 124 op/s 2023-09-10T01:12:36.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:35 smithi099 ceph-mon[102692]: pgmap v1017: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 868 KiB/s wr, 124 op/s 2023-09-10T01:12:38.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:37 smithi131 ceph-mon[103713]: pgmap v1018: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:12:38.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:37 smithi044 ceph-mon[98676]: pgmap v1018: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:12:38.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:37 smithi099 ceph-mon[102692]: pgmap v1018: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:12:40.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:39 smithi131 ceph-mon[103713]: pgmap v1019: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 902 KiB/s wr, 128 op/s 2023-09-10T01:12:40.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:39 smithi044 ceph-mon[98676]: pgmap v1019: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 902 KiB/s wr, 128 op/s 2023-09-10T01:12:40.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:39 smithi099 ceph-mon[102692]: pgmap v1019: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 902 KiB/s wr, 128 op/s 2023-09-10T01:12:42.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:41 smithi131 ceph-mon[103713]: pgmap v1020: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 195 op/s 2023-09-10T01:12:42.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:41 smithi044 ceph-mon[98676]: pgmap v1020: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 195 op/s 2023-09-10T01:12:42.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:41 smithi099 ceph-mon[102692]: pgmap v1020: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 195 op/s 2023-09-10T01:12:44.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:43 smithi131 ceph-mon[103713]: pgmap v1021: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 930 KiB/s wr, 133 op/s 2023-09-10T01:12:44.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:43 smithi044 ceph-mon[98676]: pgmap v1021: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 930 KiB/s wr, 133 op/s 2023-09-10T01:12:44.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:43 smithi099 ceph-mon[102692]: pgmap v1021: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 930 KiB/s wr, 133 op/s 2023-09-10T01:12:46.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:45 smithi131 ceph-mon[103713]: pgmap v1022: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 931 KiB/s wr, 133 op/s 2023-09-10T01:12:46.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:45 smithi044 ceph-mon[98676]: pgmap v1022: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 931 KiB/s wr, 133 op/s 2023-09-10T01:12:46.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:45 smithi099 ceph-mon[102692]: pgmap v1022: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 931 KiB/s wr, 133 op/s 2023-09-10T01:12:48.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:47 smithi131 ceph-mon[103713]: pgmap v1023: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 201 op/s 2023-09-10T01:12:48.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:47 smithi044 ceph-mon[98676]: pgmap v1023: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 201 op/s 2023-09-10T01:12:48.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:47 smithi099 ceph-mon[102692]: pgmap v1023: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 201 op/s 2023-09-10T01:12:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.833+0000 7f291b4b6700 1 -- 172.21.15.44:0/2622769021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29140687e0 msgr2=0x7f2914068bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.833+0000 7f291b4b6700 1 --2- 172.21.15.44:0/2622769021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29140687e0 0x7f2914068bc0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f29100099c0 tx=0x7f2910009cd0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.833+0000 7f291b4b6700 1 -- 172.21.15.44:0/2622769021 shutdown_connections 2023-09-10T01:12:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.833+0000 7f291b4b6700 1 --2- 172.21.15.44:0/2622769021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2914105940 0x7f291410d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.833+0000 7f291b4b6700 1 --2- 172.21.15.44:0/2622769021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29140687e0 0x7f2914068bc0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.833+0000 7f291b4b6700 1 --2- 172.21.15.44:0/2622769021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2914069190 0x7f2914105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.833+0000 7f291b4b6700 1 -- 172.21.15.44:0/2622769021 >> 172.21.15.44:0/2622769021 conn(0x7f29140773d0 msgr2=0x7f29140777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.834+0000 7f291b4b6700 1 -- 172.21.15.44:0/2622769021 shutdown_connections 2023-09-10T01:12:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.834+0000 7f291b4b6700 1 -- 172.21.15.44:0/2622769021 wait complete. 2023-09-10T01:12:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.834+0000 7f291b4b6700 1 Processor -- start 2023-09-10T01:12:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.834+0000 7f291b4b6700 1 -- start start 2023-09-10T01:12:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.834+0000 7f291b4b6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29140687e0 0x7f29140ff940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.834+0000 7f291b4b6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2914105940 0x7f29140ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.834+0000 7f291b4b6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29141a2740 0x7f29141a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.834+0000 7f291b4b6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29141a5070 con 0x7f2914105940 2023-09-10T01:12:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.834+0000 7f291b4b6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29141a51b0 con 0x7f29141a2740 2023-09-10T01:12:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.834+0000 7f291b4b6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29141a5320 con 0x7f29140687e0 2023-09-10T01:12:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f2918a51700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2914105940 0x7f29140ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f2919252700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29140687e0 0x7f29140ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f2919252700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29140687e0 0x7f29140ff940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35680/0 (socket says 172.21.15.44:35680) 2023-09-10T01:12:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f2918a51700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2914105940 0x7f29140ffe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57032/0 (socket says 172.21.15.44:57032) 2023-09-10T01:12:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f2919252700 1 -- 172.21.15.44:0/3912236404 learned_addr learned my addr 172.21.15.44:0/3912236404 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f2919a53700 1 --2- 172.21.15.44:0/3912236404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29141a2740 0x7f29141a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f2919252700 1 -- 172.21.15.44:0/3912236404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29141a2740 msgr2=0x7f29141a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f2919252700 1 --2- 172.21.15.44:0/3912236404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29141a2740 0x7f29141a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f2919252700 1 -- 172.21.15.44:0/3912236404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2914105940 msgr2=0x7f29140ffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f2919252700 1 --2- 172.21.15.44:0/3912236404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2914105940 0x7f29140ffe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f2919252700 1 -- 172.21.15.44:0/3912236404 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29100095f0 con 0x7f29140687e0 2023-09-10T01:12:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f2918a51700 1 --2- 172.21.15.44:0/3912236404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2914105940 0x7f29140ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:12:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f2919252700 1 --2- 172.21.15.44:0/3912236404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29140687e0 0x7f29140ff940 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f29100059d0 tx=0x7f291000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f290a7fc700 1 -- 172.21.15.44:0/3912236404 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291001d070 con 0x7f29140687e0 2023-09-10T01:12:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f290a7fc700 1 -- 172.21.15.44:0/3912236404 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2910022470 con 0x7f29140687e0 2023-09-10T01:12:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f291b4b6700 1 -- 172.21.15.44:0/3912236404 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29141a55a0 con 0x7f29140687e0 2023-09-10T01:12:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f290a7fc700 1 -- 172.21.15.44:0/3912236404 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910017670 con 0x7f29140687e0 2023-09-10T01:12:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.835+0000 7f291b4b6700 1 -- 172.21.15.44:0/3912236404 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29141a59c0 con 0x7f29140687e0 2023-09-10T01:12:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.838+0000 7f290a7fc700 1 -- 172.21.15.44:0/3912236404 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f29100225e0 con 0x7f29140687e0 2023-09-10T01:12:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.839+0000 7f290a7fc700 1 --2- 172.21.15.44:0/3912236404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29000a07b0 0x7f29000a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.839+0000 7f2918a51700 1 --2- 172.21.15.44:0/3912236404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29000a07b0 0x7f29000a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.839+0000 7f290a7fc700 1 -- 172.21.15.44:0/3912236404 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f29100cd450 con 0x7f29140687e0 2023-09-10T01:12:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f2918a51700 1 --2- 172.21.15.44:0/3912236404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29000a07b0 0x7f29000a2c70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2914101160 tx=0x7f2904009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f292767cb80 1 Processor -- start 2023-09-10T01:12:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f292767cb80 1 -- start start 2023-09-10T01:12:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f292767cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf651a0 0x555a3cf65580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f292767cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf5f300 0x555a3cf5f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f292767cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cf6bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f292767cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3ceb86d0 con 0x555a3cf5fcc0 2023-09-10T01:12:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f292767cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3ceb5ff0 con 0x555a3cf5f300 2023-09-10T01:12:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f292767cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3ceb18e0 con 0x555a3cf651a0 2023-09-10T01:12:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f2919a53700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cf6bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f2919252700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf651a0 0x555a3cf65580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f2919252700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf651a0 0x555a3cf65580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35688/0 (socket says 172.21.15.44:35688) 2023-09-10T01:12:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f2918a51700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf5f300 0x555a3cf5f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f2919252700 1 -- 172.21.15.44:0/1705434024 learned_addr learned my addr 172.21.15.44:0/1705434024 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f2919a53700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cf6bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57042/0 (socket says 172.21.15.44:57042) 2023-09-10T01:12:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f2919252700 1 -- 172.21.15.44:0/1705434024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf5f300 msgr2=0x555a3cf5f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f2919252700 1 --2- 172.21.15.44:0/1705434024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf5f300 0x555a3cf5f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f2919252700 1 -- 172.21.15.44:0/1705434024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 msgr2=0x555a3cf6bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.840+0000 7f2919252700 1 --2- 172.21.15.44:0/1705434024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cf6bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.841+0000 7f2919252700 1 -- 172.21.15.44:0/1705434024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a3ccddd80 con 0x555a3cf651a0 2023-09-10T01:12:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.841+0000 7f2919a53700 1 --2- 172.21.15.44:0/1705434024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cf6bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:12:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.841+0000 7f2919252700 1 --2- 172.21.15.44:0/1705434024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf651a0 0x555a3cf65580 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2910096a90 tx=0x7f2910099f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fffc31de0c88c36 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.841+0000 7f28ff7fe700 1 -- 172.21.15.44:0/1705434024 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291009a590 con 0x555a3cf651a0 2023-09-10T01:12:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.841+0000 7f28ff7fe700 1 -- 172.21.15.44:0/1705434024 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f291009e3c0 con 0x555a3cf651a0 2023-09-10T01:12:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.841+0000 7f28ff7fe700 1 -- 172.21.15.44:0/1705434024 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291000fa90 con 0x555a3cf651a0 2023-09-10T01:12:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 -- 172.21.15.44:0/1705434024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf651a0 msgr2=0x555a3cf65580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 --2- 172.21.15.44:0/1705434024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf651a0 0x555a3cf65580 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2910096a90 tx=0x7f2910099f70 comp rx=0 tx=0).stop 2023-09-10T01:12:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 -- 172.21.15.44:0/1705434024 shutdown_connections 2023-09-10T01:12:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 --2- 172.21.15.44:0/1705434024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cf6bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 --2- 172.21.15.44:0/1705434024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf651a0 0x555a3cf65580 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 --2- 172.21.15.44:0/1705434024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf5f300 0x555a3cf5f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 -- 172.21.15.44:0/1705434024 >> 172.21.15.44:0/1705434024 conn(0x555a3ced4030 msgr2=0x555a3cf5a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 -- 172.21.15.44:0/1705434024 shutdown_connections 2023-09-10T01:12:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 -- 172.21.15.44:0/1705434024 wait complete. 2023-09-10T01:12:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 -- start start 2023-09-10T01:12:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 Processor -- start 2023-09-10T01:12:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5f300 0x555a3cf6ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cf6f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf651a0 0x555a3cf68e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3ce4fb10 con 0x555a3cf5fcc0 2023-09-10T01:12:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3ce4f810 con 0x555a3cf651a0 2023-09-10T01:12:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.842+0000 7f292767cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3cd338c0 con 0x555a3cf5f300 2023-09-10T01:12:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f292767cb80 10 client.0 _ll_register_callbacks cb 0x555a3cd99a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:12:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f2918a51700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cf6f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f2919252700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5f300 0x555a3cf6ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f2918a51700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cf6f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57046/0 (socket says 172.21.15.44:57046) 2023-09-10T01:12:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f2918a51700 1 -- 172.21.15.44:0/1879549608 learned_addr learned my addr 172.21.15.44:0/1879549608 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f2919a53700 1 --2- 172.21.15.44:0/1879549608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf651a0 0x555a3cf68e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f2918a51700 1 -- 172.21.15.44:0/1879549608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5f300 msgr2=0x555a3cf6ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f2918a51700 1 --2- 172.21.15.44:0/1879549608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5f300 0x555a3cf6ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f2918a51700 1 -- 172.21.15.44:0/1879549608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf651a0 msgr2=0x555a3cf68e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f2918a51700 1 --2- 172.21.15.44:0/1879549608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf651a0 0x555a3cf68e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f2918a51700 1 -- 172.21.15.44:0/1879549608 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290c011230 con 0x555a3cf5fcc0 2023-09-10T01:12:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f2919a53700 1 --2- 172.21.15.44:0/1879549608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf651a0 0x555a3cf68e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:12:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f2918a51700 1 --2- 172.21.15.44:0/1879549608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cf6f100 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f29040037c0 tx=0x7f29040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6c444fa845cd8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f28fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:12:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f28fcff9700 1 -- 172.21.15.44:0/1879549608 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290401e6b0 con 0x555a3cf5fcc0 2023-09-10T01:12:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f28fcff9700 1 -- 172.21.15.44:0/1879549608 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f290401e810 con 0x555a3cf5fcc0 2023-09-10T01:12:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.843+0000 7f292767cb80 1 -- 172.21.15.44:0/1879549608 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a3ce09490 con 0x555a3cf5fcc0 2023-09-10T01:12:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.844+0000 7f28fcff9700 1 -- 172.21.15.44:0/1879549608 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f290401e9d0 con 0x555a3cf5fcc0 2023-09-10T01:12:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.844+0000 7f292767cb80 10 client.19257 fetch_fsmap learned FSMap version 120 2023-09-10T01:12:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.844+0000 7f292767cb80 1 -- 172.21.15.44:0/1879549608 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x7f29141a59c0 con 0x555a3cf5fcc0 2023-09-10T01:12:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.844+0000 7f28fcff9700 1 -- 172.21.15.44:0/1879549608 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555a3ce09490 con 0x555a3cf5fcc0 2023-09-10T01:12:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.844+0000 7f28fcff9700 1 -- 172.21.15.44:0/1879549608 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f2904028a90 con 0x555a3cf5fcc0 2023-09-10T01:12:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.844+0000 7f292767cb80 10 client.19257 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:12:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.844+0000 7f292767cb80 10 client.19257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:12:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.844+0000 7f292767cb80 20 client.19257 populate_metadata read hostname 'smithi044' 2023-09-10T01:12:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.844+0000 7f292767cb80 1 --2- 172.21.15.44:0/1879549608 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555a3cf791a0 0x555a3cf7b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.844+0000 7f292767cb80 4 client.19257 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:12:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.844+0000 7f292767cb80 1 -- 172.21.15.44:0/1879549608 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a3cec6840 con 0x555a3cf791a0 2023-09-10T01:12:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.844+0000 7f2919252700 1 --2- 172.21.15.44:0/1879549608 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555a3cf791a0 0x555a3cf7b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.845+0000 7f28fcff9700 10 client.19257 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:12:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.845+0000 7f2919252700 1 --2- 172.21.15.44:0/1879549608 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555a3cf791a0 0x555a3cf7b5c0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c3ebd5ca51bd73 server_cookie=ec42aac268aaa834 in_seq=0 out_seq=0 2023-09-10T01:12:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.845+0000 7f28fcff9700 1 -- 172.21.15.44:0/1879549608 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555a3cec6840 con 0x555a3cf791a0 2023-09-10T01:12:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.845+0000 7f28fcff9700 10 client.19257 handle_command_reply: tid=0 2023-09-10T01:12:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.845+0000 7f292767cb80 1 client.19257 shutdown 2023-09-10T01:12:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.845+0000 7f292767cb80 1 -- 172.21.15.44:0/1879549608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 msgr2=0x555a3cf6f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.845+0000 7f292767cb80 1 --2- 172.21.15.44:0/1879549608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cf6f100 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f29040037c0 tx=0x7f29040047c0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.845+0000 7f292767cb80 1 -- 172.21.15.44:0/1879549608 shutdown_connections 2023-09-10T01:12:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.845+0000 7f292767cb80 1 --2- 172.21.15.44:0/1879549608 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555a3cf791a0 0x555a3cf7b5c0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.845+0000 7f292767cb80 1 --2- 172.21.15.44:0/1879549608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf651a0 0x555a3cf68e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.845+0000 7f292767cb80 1 --2- 172.21.15.44:0/1879549608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5f300 0x555a3cf6ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.845+0000 7f292767cb80 1 --2- 172.21.15.44:0/1879549608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cf6f100 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.845+0000 7f292767cb80 1 -- 172.21.15.44:0/1879549608 >> 172.21.15.44:0/1879549608 conn(0x555a3ced4030 msgr2=0x555a3cf597a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.845+0000 7f28fcff9700 0 client.19257 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:12:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.845+0000 7f292767cb80 1 -- 172.21.15.44:0/1879549608 shutdown_connections 2023-09-10T01:12:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.846+0000 7f292767cb80 1 -- 172.21.15.44:0/1879549608 wait complete. 2023-09-10T01:12:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.846+0000 7f292767cb80 20 client.19257 trim_cache size 0 max 16384 2023-09-10T01:12:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.863+0000 7f292767cb80 1 Processor -- start 2023-09-10T01:12:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.863+0000 7f292767cb80 1 -- start start 2023-09-10T01:12:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.863+0000 7f292767cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf5f300 0x555a3cf648f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.863+0000 7f292767cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5fcc0 0x555a3cf67cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.863+0000 7f292767cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf651a0 0x555a3cf59f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.863+0000 7f292767cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3cf5bd10 con 0x555a3cf651a0 2023-09-10T01:12:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.863+0000 7f292767cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3cf5b140 con 0x555a3cf5f300 2023-09-10T01:12:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.863+0000 7f292767cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3cf76780 con 0x555a3cf5fcc0 2023-09-10T01:12:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.863+0000 7f2919a53700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf651a0 0x555a3cf59f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.864+0000 7f2918a51700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5fcc0 0x555a3cf67cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.864+0000 7f2919252700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf5f300 0x555a3cf648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.864+0000 7f2918a51700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5fcc0 0x555a3cf67cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35704/0 (socket says 172.21.15.44:35704) 2023-09-10T01:12:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.864+0000 7f2918a51700 1 -- 172.21.15.44:0/1766798921 learned_addr learned my addr 172.21.15.44:0/1766798921 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.864+0000 7f2919a53700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf651a0 0x555a3cf59f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57056/0 (socket says 172.21.15.44:57056) 2023-09-10T01:12:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.864+0000 7f2918a51700 1 -- 172.21.15.44:0/1766798921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf5f300 msgr2=0x555a3cf648f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.864+0000 7f2918a51700 1 --2- 172.21.15.44:0/1766798921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf5f300 0x555a3cf648f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.864+0000 7f2918a51700 1 -- 172.21.15.44:0/1766798921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf651a0 msgr2=0x555a3cf59f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.864+0000 7f2918a51700 1 --2- 172.21.15.44:0/1766798921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf651a0 0x555a3cf59f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.864+0000 7f2918a51700 1 -- 172.21.15.44:0/1766798921 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a3cf68980 con 0x555a3cf5fcc0 2023-09-10T01:12:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.864+0000 7f2919252700 1 --2- 172.21.15.44:0/1766798921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf5f300 0x555a3cf648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:12:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.864+0000 7f2919a53700 1 --2- 172.21.15.44:0/1766798921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf651a0 0x555a3cf59f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:12:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.864+0000 7f2918a51700 1 --2- 172.21.15.44:0/1766798921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5fcc0 0x555a3cf67cd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2904013010 tx=0x7f2904004650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad46ad4b411e379b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.864+0000 7f28fffff700 1 -- 172.21.15.44:0/1766798921 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29040285f0 con 0x555a3cf5fcc0 2023-09-10T01:12:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.864+0000 7f28fffff700 1 -- 172.21.15.44:0/1766798921 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2904028be0 con 0x555a3cf5fcc0 2023-09-10T01:12:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.864+0000 7f28fffff700 1 -- 172.21.15.44:0/1766798921 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290401eca0 con 0x555a3cf5fcc0 2023-09-10T01:12:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.865+0000 7f292767cb80 1 -- 172.21.15.44:0/1766798921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5fcc0 msgr2=0x555a3cf67cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.865+0000 7f292767cb80 1 --2- 172.21.15.44:0/1766798921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5fcc0 0x555a3cf67cd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2904013010 tx=0x7f2904004650 comp rx=0 tx=0).stop 2023-09-10T01:12:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.865+0000 7f292767cb80 1 -- 172.21.15.44:0/1766798921 shutdown_connections 2023-09-10T01:12:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.865+0000 7f292767cb80 1 --2- 172.21.15.44:0/1766798921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf651a0 0x555a3cf59f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.865+0000 7f292767cb80 1 --2- 172.21.15.44:0/1766798921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf5f300 0x555a3cf648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.865+0000 7f292767cb80 1 --2- 172.21.15.44:0/1766798921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5fcc0 0x555a3cf67cd0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.865+0000 7f292767cb80 1 -- 172.21.15.44:0/1766798921 >> 172.21.15.44:0/1766798921 conn(0x555a3ced4030 msgr2=0x555a3cf675c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.865+0000 7f292767cb80 1 -- 172.21.15.44:0/1766798921 shutdown_connections 2023-09-10T01:12:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.865+0000 7f292767cb80 1 -- 172.21.15.44:0/1766798921 wait complete. 2023-09-10T01:12:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.865+0000 7f292767cb80 1 -- start start 2023-09-10T01:12:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f292767cb80 1 Processor -- start 2023-09-10T01:12:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f292767cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5f300 0x555a3cfc56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f292767cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cfc5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f292767cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf651a0 0x555a3cf6e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f292767cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3cf6e760 con 0x555a3cf5fcc0 2023-09-10T01:12:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f292767cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3cf6e8d0 con 0x555a3cf651a0 2023-09-10T01:12:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f292767cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3cf6ea90 con 0x555a3cf5f300 2023-09-10T01:12:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f292767cb80 10 client.0 _ll_register_callbacks cb 0x7f29040045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:12:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f2919252700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5f300 0x555a3cfc56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f2918a51700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cfc5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f2919252700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5f300 0x555a3cfc56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35708/0 (socket says 172.21.15.44:35708) 2023-09-10T01:12:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f2919252700 1 -- 172.21.15.44:0/907118836 learned_addr learned my addr 172.21.15.44:0/907118836 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:12:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f2918a51700 1 --2- 172.21.15.44:0/907118836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cfc5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57060/0 (socket says 172.21.15.44:57060) 2023-09-10T01:12:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f2919a53700 1 --2- 172.21.15.44:0/907118836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf651a0 0x555a3cf6e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f2919252700 1 -- 172.21.15.44:0/907118836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf651a0 msgr2=0x555a3cf6e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f2919252700 1 --2- 172.21.15.44:0/907118836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf651a0 0x555a3cf6e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f2919252700 1 -- 172.21.15.44:0/907118836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 msgr2=0x555a3cfc5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:12:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f2919252700 1 --2- 172.21.15.44:0/907118836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cfc5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f2919252700 1 -- 172.21.15.44:0/907118836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a3ccddd80 con 0x555a3cf5f300 2023-09-10T01:12:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.866+0000 7f2918a51700 1 --2- 172.21.15.44:0/907118836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cfc5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:12:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.867+0000 7f2919252700 1 --2- 172.21.15.44:0/907118836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5f300 0x555a3cfc56f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f29100cada0 tx=0x7f291001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b78fab26141eb476 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:12:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.867+0000 7f28fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:12:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.867+0000 7f28fcff9700 1 -- 172.21.15.44:0/907118836 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910099ef0 con 0x555a3cf5f300 2023-09-10T01:12:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.867+0000 7f28fcff9700 1 -- 172.21.15.44:0/907118836 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f29100a2570 con 0x555a3cf5f300 2023-09-10T01:12:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.867+0000 7f292767cb80 1 -- 172.21.15.44:0/907118836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f291009e3c0 con 0x555a3cf5f300 2023-09-10T01:12:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.867+0000 7f28fcff9700 1 -- 172.21.15.44:0/907118836 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29100850b0 con 0x555a3cf5f300 2023-09-10T01:12:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.867+0000 7f292767cb80 10 client.28873 fetch_fsmap learned FSMap version 120 2023-09-10T01:12:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.867+0000 7f292767cb80 1 -- 172.21.15.44:0/907118836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x7f290c007690 con 0x555a3cf5f300 2023-09-10T01:12:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.867+0000 7f28fcff9700 1 -- 172.21.15.44:0/907118836 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f291009e3c0 con 0x555a3cf5f300 2023-09-10T01:12:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.867+0000 7f28fcff9700 1 -- 172.21.15.44:0/907118836 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f29100228c0 con 0x555a3cf5f300 2023-09-10T01:12:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.867+0000 7f292767cb80 10 client.28873 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:12:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.867+0000 7f292767cb80 10 client.28873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:12:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.867+0000 7f292767cb80 20 client.28873 populate_metadata read hostname 'smithi044' 2023-09-10T01:12:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.867+0000 7f292767cb80 1 --2- 172.21.15.44:0/907118836 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555a3cfcd040 0x555a3cfcf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:12:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.867+0000 7f292767cb80 4 client.28873 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:12:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.867+0000 7f292767cb80 1 -- 172.21.15.44:0/907118836 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555a3ceaa8c0 con 0x555a3cfcd040 2023-09-10T01:12:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.868+0000 7f2918a51700 1 --2- 172.21.15.44:0/907118836 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555a3cfcd040 0x555a3cfcf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:12:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.868+0000 7f28fcff9700 10 client.28873 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:12:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.868+0000 7f2918a51700 1 --2- 172.21.15.44:0/907118836 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555a3cfcd040 0x555a3cfcf430 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2897691bb68e2162 server_cookie=d6be29f2e272e601 in_seq=0 out_seq=0 2023-09-10T01:12:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.877+0000 7f28fcff9700 1 -- 172.21.15.44:0/907118836 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555a3ceaa8c0 con 0x555a3cfcd040 2023-09-10T01:12:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.877+0000 7f28fcff9700 10 client.28873 handle_command_reply: tid=0 2023-09-10T01:12:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.877+0000 7f292767cb80 1 client.28873 shutdown 2023-09-10T01:12:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.878+0000 7f292767cb80 1 -- 172.21.15.44:0/907118836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5f300 msgr2=0x555a3cfc56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.878+0000 7f292767cb80 1 --2- 172.21.15.44:0/907118836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5f300 0x555a3cfc56f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f29100cada0 tx=0x7f291001a040 comp rx=0 tx=0).stop 2023-09-10T01:12:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.878+0000 7f292767cb80 1 -- 172.21.15.44:0/907118836 shutdown_connections 2023-09-10T01:12:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.878+0000 7f292767cb80 1 --2- 172.21.15.44:0/907118836 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555a3cfcd040 0x555a3cfcf430 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.878+0000 7f292767cb80 1 --2- 172.21.15.44:0/907118836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a3cf651a0 0x555a3cf6e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.878+0000 7f292767cb80 1 --2- 172.21.15.44:0/907118836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a3cf5f300 0x555a3cfc56f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.878+0000 7f28fcff9700 0 client.28873 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:12:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.878+0000 7f292767cb80 1 --2- 172.21.15.44:0/907118836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a3cf5fcc0 0x555a3cfc5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.878+0000 7f292767cb80 1 -- 172.21.15.44:0/907118836 >> 172.21.15.44:0/907118836 conn(0x555a3ced4030 msgr2=0x555a3cf5eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.878+0000 7f292767cb80 1 -- 172.21.15.44:0/907118836 shutdown_connections 2023-09-10T01:12:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.878+0000 7f292767cb80 1 -- 172.21.15.44:0/907118836 wait complete. 2023-09-10T01:12:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.878+0000 7f292767cb80 20 client.28873 trim_cache size 0 max 16384 2023-09-10T01:12:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:12:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:12:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:12:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:12:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.879+0000 7f291b4b6700 1 -- 172.21.15.44:0/3912236404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29000a07b0 msgr2=0x7f29000a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.879+0000 7f291b4b6700 1 --2- 172.21.15.44:0/3912236404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29000a07b0 0x7f29000a2c70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2914101160 tx=0x7f2904009260 comp rx=0 tx=0).stop 2023-09-10T01:12:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.879+0000 7f291b4b6700 1 -- 172.21.15.44:0/3912236404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29140687e0 msgr2=0x7f29140ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:12:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.879+0000 7f291b4b6700 1 --2- 172.21.15.44:0/3912236404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29140687e0 0x7f29140ff940 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f29100059d0 tx=0x7f291000f690 comp rx=0 tx=0).stop 2023-09-10T01:12:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.879+0000 7f291b4b6700 1 -- 172.21.15.44:0/3912236404 shutdown_connections 2023-09-10T01:12:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.879+0000 7f291b4b6700 1 --2- 172.21.15.44:0/3912236404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29000a07b0 0x7f29000a2c70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.879+0000 7f291b4b6700 1 --2- 172.21.15.44:0/3912236404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29141a2740 0x7f29141a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.879+0000 7f291b4b6700 1 --2- 172.21.15.44:0/3912236404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29140687e0 0x7f29140ff940 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.879+0000 7f291b4b6700 1 --2- 172.21.15.44:0/3912236404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2914105940 0x7f29140ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:12:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.879+0000 7f291b4b6700 1 -- 172.21.15.44:0/3912236404 >> 172.21.15.44:0/3912236404 conn(0x7f29140773d0 msgr2=0x7f29140fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:12:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.879+0000 7f291b4b6700 1 -- 172.21.15.44:0/3912236404 shutdown_connections 2023-09-10T01:12:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:12:48.879+0000 7f291b4b6700 1 -- 172.21.15.44:0/3912236404 wait complete. 2023-09-10T01:12:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:48 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:12:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:48 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:12:49.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:48 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:12:50.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:49 smithi131 ceph-mon[103713]: pgmap v1024: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 938 KiB/s wr, 134 op/s 2023-09-10T01:12:50.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:49 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:12:50.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:49 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:12:50.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:49 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:12:50.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:49 smithi044 ceph-mon[98676]: pgmap v1024: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 938 KiB/s wr, 134 op/s 2023-09-10T01:12:50.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:49 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:12:50.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:49 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:12:50.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:49 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:12:50.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:49 smithi099 ceph-mon[102692]: pgmap v1024: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 938 KiB/s wr, 134 op/s 2023-09-10T01:12:50.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:49 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:12:50.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:49 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:12:50.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:49 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:12:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:50 smithi131 ceph-mon[103713]: pgmap v1025: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.0 MiB/s wr, 147 op/s 2023-09-10T01:12:51.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:50 smithi131 ceph-mon[103713]: pgmap v1026: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 643 KiB/s wr, 91 op/s 2023-09-10T01:12:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:50 smithi044 ceph-mon[98676]: pgmap v1025: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.0 MiB/s wr, 147 op/s 2023-09-10T01:12:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:50 smithi044 ceph-mon[98676]: pgmap v1026: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 643 KiB/s wr, 91 op/s 2023-09-10T01:12:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:50 smithi099 ceph-mon[102692]: pgmap v1025: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.0 MiB/s wr, 147 op/s 2023-09-10T01:12:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:50 smithi099 ceph-mon[102692]: pgmap v1026: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 643 KiB/s wr, 91 op/s 2023-09-10T01:12:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:52 smithi131 ceph-mon[103713]: pgmap v1027: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:12:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:52 smithi044 ceph-mon[98676]: pgmap v1027: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:12:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:52 smithi099 ceph-mon[102692]: pgmap v1027: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:12:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:54 smithi131 ceph-mon[103713]: pgmap v1028: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:12:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:54 smithi044 ceph-mon[98676]: pgmap v1028: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:12:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:54 smithi099 ceph-mon[102692]: pgmap v1028: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:12:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:56 smithi131 ceph-mon[103713]: pgmap v1029: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 206 op/s 2023-09-10T01:12:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:56 smithi044 ceph-mon[98676]: pgmap v1029: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 206 op/s 2023-09-10T01:12:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:56 smithi099 ceph-mon[102692]: pgmap v1029: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 206 op/s 2023-09-10T01:12:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:12:58 smithi131 ceph-mon[103713]: pgmap v1030: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 206 op/s 2023-09-10T01:12:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:12:58 smithi044 ceph-mon[98676]: pgmap v1030: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 206 op/s 2023-09-10T01:12:59.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:12:58 smithi099 ceph-mon[102692]: pgmap v1030: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 206 op/s 2023-09-10T01:13:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:00 smithi131 ceph-mon[103713]: pgmap v1031: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 184 op/s 2023-09-10T01:13:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:00 smithi044 ceph-mon[98676]: pgmap v1031: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 184 op/s 2023-09-10T01:13:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:00 smithi099 ceph-mon[102692]: pgmap v1031: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 184 op/s 2023-09-10T01:13:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.024+0000 7f64414fb700 1 -- 172.21.15.44:0/521399218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c0ff830 msgr2=0x7f643c103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.024+0000 7f64414fb700 1 --2- 172.21.15.44:0/521399218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c0ff830 0x7f643c103870 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6430009990 tx=0x7f6430009ca0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.024+0000 7f64414fb700 1 -- 172.21.15.44:0/521399218 shutdown_connections 2023-09-10T01:13:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.024+0000 7f64414fb700 1 --2- 172.21.15.44:0/521399218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f643c103fa0 0x7f643c10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.024+0000 7f64414fb700 1 --2- 172.21.15.44:0/521399218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f643c0fee80 0x7f643c0ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.024+0000 7f64414fb700 1 --2- 172.21.15.44:0/521399218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c0ff830 0x7f643c103870 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.024+0000 7f64414fb700 1 -- 172.21.15.44:0/521399218 >> 172.21.15.44:0/521399218 conn(0x7f643c0769d0 msgr2=0x7f643c0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.024+0000 7f64414fb700 1 -- 172.21.15.44:0/521399218 shutdown_connections 2023-09-10T01:13:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.024+0000 7f64414fb700 1 -- 172.21.15.44:0/521399218 wait complete. 2023-09-10T01:13:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.024+0000 7f64414fb700 1 Processor -- start 2023-09-10T01:13:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.025+0000 7f64414fb700 1 -- start start 2023-09-10T01:13:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.025+0000 7f64414fb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f643c0fee80 0x7f643c194380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.025+0000 7f64414fb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c103fa0 0x7f643c1948c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.025+0000 7f64414fb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f643c1a2750 0x7f643c1a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.025+0000 7f64414fb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c1a5080 con 0x7f643c1a2750 2023-09-10T01:13:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.025+0000 7f64414fb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c1a51c0 con 0x7f643c103fa0 2023-09-10T01:13:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.025+0000 7f64414fb700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c1a5330 con 0x7f643c0fee80 2023-09-10T01:13:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.025+0000 7f643affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f643c0fee80 0x7f643c194380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.025+0000 7f643affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f643c0fee80 0x7f643c194380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35722/0 (socket says 172.21.15.44:35722) 2023-09-10T01:13:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.025+0000 7f643affd700 1 -- 172.21.15.44:0/2810369907 learned_addr learned my addr 172.21.15.44:0/2810369907 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.025+0000 7f643a7fc700 1 --2- 172.21.15.44:0/2810369907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c103fa0 0x7f643c1948c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.025+0000 7f643b7fe700 1 --2- 172.21.15.44:0/2810369907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f643c1a2750 0x7f643c1a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.026+0000 7f643affd700 1 -- 172.21.15.44:0/2810369907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c103fa0 msgr2=0x7f643c1948c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.026+0000 7f643affd700 1 --2- 172.21.15.44:0/2810369907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c103fa0 0x7f643c1948c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.026+0000 7f643affd700 1 -- 172.21.15.44:0/2810369907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f643c1a2750 msgr2=0x7f643c1a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.026+0000 7f643affd700 1 --2- 172.21.15.44:0/2810369907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f643c1a2750 0x7f643c1a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.026+0000 7f643affd700 1 -- 172.21.15.44:0/2810369907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6434009c40 con 0x7f643c0fee80 2023-09-10T01:13:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.026+0000 7f643a7fc700 1 --2- 172.21.15.44:0/2810369907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c103fa0 0x7f643c1948c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.026+0000 7f643affd700 1 --2- 172.21.15.44:0/2810369907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f643c0fee80 0x7f643c194380 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f642c00eab0 tx=0x7f642c00edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.026+0000 7f6423fff700 1 -- 172.21.15.44:0/2810369907 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c00a800 con 0x7f643c0fee80 2023-09-10T01:13:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.026+0000 7f64414fb700 1 -- 172.21.15.44:0/2810369907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64300095f0 con 0x7f643c0fee80 2023-09-10T01:13:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.026+0000 7f64414fb700 1 -- 172.21.15.44:0/2810369907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c1a59b0 con 0x7f643c0fee80 2023-09-10T01:13:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.026+0000 7f6423fff700 1 -- 172.21.15.44:0/2810369907 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f642c00ccd0 con 0x7f643c0fee80 2023-09-10T01:13:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.026+0000 7f6423fff700 1 -- 172.21.15.44:0/2810369907 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c0155f0 con 0x7f643c0fee80 2023-09-10T01:13:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.029+0000 7f6423fff700 1 -- 172.21.15.44:0/2810369907 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f642c0163e0 con 0x7f643c0fee80 2023-09-10T01:13:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.029+0000 7f6423fff700 1 --2- 172.21.15.44:0/2810369907 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f64240a07b0 0x7f64240a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.029+0000 7f6423fff700 1 -- 172.21.15.44:0/2810369907 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f642c0cc540 con 0x7f643c0fee80 2023-09-10T01:13:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.029+0000 7f643a7fc700 1 --2- 172.21.15.44:0/2810369907 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f64240a07b0 0x7f64240a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.030+0000 7f644d6c1b80 1 Processor -- start 2023-09-10T01:13:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.030+0000 7f643a7fc700 1 --2- 172.21.15.44:0/2810369907 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f64240a07b0 0x7f64240a2c70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f643000b2e0 tx=0x7f6430004500 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.030+0000 7f644d6c1b80 1 -- start start 2023-09-10T01:13:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.030+0000 7f644d6c1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d75161a0 0x5562d7516580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.030+0000 7f644d6c1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510300 0x5562d7510780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.030+0000 7f644d6c1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d751cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.030+0000 7f644d6c1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d74696d0 con 0x5562d7510cc0 2023-09-10T01:13:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.030+0000 7f644d6c1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d7466ff0 con 0x5562d7510300 2023-09-10T01:13:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.030+0000 7f644d6c1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d74628e0 con 0x5562d75161a0 2023-09-10T01:13:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.030+0000 7f643b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d751cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.030+0000 7f643affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d75161a0 0x5562d7516580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.030+0000 7f643a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510300 0x5562d7510780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.030+0000 7f643b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d751cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57084/0 (socket says 172.21.15.44:57084) 2023-09-10T01:13:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.030+0000 7f643b7fe700 1 -- 172.21.15.44:0/850671542 learned_addr learned my addr 172.21.15.44:0/850671542 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.031+0000 7f643b7fe700 1 -- 172.21.15.44:0/850671542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d75161a0 msgr2=0x5562d7516580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.031+0000 7f643b7fe700 1 --2- 172.21.15.44:0/850671542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d75161a0 0x5562d7516580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.031+0000 7f643b7fe700 1 -- 172.21.15.44:0/850671542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510300 msgr2=0x5562d7510780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.031+0000 7f643b7fe700 1 --2- 172.21.15.44:0/850671542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510300 0x5562d7510780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.031+0000 7f643b7fe700 1 -- 172.21.15.44:0/850671542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562d728ed80 con 0x5562d7510cc0 2023-09-10T01:13:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.031+0000 7f643a7fc700 1 --2- 172.21.15.44:0/850671542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510300 0x5562d7510780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.032+0000 7f643b7fe700 1 --2- 172.21.15.44:0/850671542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d751cf30 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f643400e730 tx=0x7f6434009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f6e8c83c81b2eb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.032+0000 7f64217fa700 1 -- 172.21.15.44:0/850671542 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643400e040 con 0x5562d7510cc0 2023-09-10T01:13:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.032+0000 7f64217fa700 1 -- 172.21.15.44:0/850671542 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f643400f040 con 0x5562d7510cc0 2023-09-10T01:13:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.032+0000 7f64217fa700 1 -- 172.21.15.44:0/850671542 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434007500 con 0x5562d7510cc0 2023-09-10T01:13:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.033+0000 7f644d6c1b80 1 -- 172.21.15.44:0/850671542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 msgr2=0x5562d751cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.033+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/850671542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d751cf30 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f643400e730 tx=0x7f6434009920 comp rx=0 tx=0).stop 2023-09-10T01:13:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.033+0000 7f644d6c1b80 1 -- 172.21.15.44:0/850671542 shutdown_connections 2023-09-10T01:13:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.033+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/850671542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d751cf30 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.033+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/850671542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d75161a0 0x5562d7516580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.033+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/850671542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510300 0x5562d7510780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.033+0000 7f644d6c1b80 1 -- 172.21.15.44:0/850671542 >> 172.21.15.44:0/850671542 conn(0x5562d7485030 msgr2=0x5562d750b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.033+0000 7f644d6c1b80 1 -- 172.21.15.44:0/850671542 shutdown_connections 2023-09-10T01:13:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.033+0000 7f644d6c1b80 1 -- 172.21.15.44:0/850671542 wait complete. 2023-09-10T01:13:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.033+0000 7f644d6c1b80 1 -- start start 2023-09-10T01:13:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.033+0000 7f644d6c1b80 1 Processor -- start 2023-09-10T01:13:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.033+0000 7f644d6c1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d7510300 0x5562d751cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f644d6c1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d7519fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f644d6c1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d75161a0 0x5562d751a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f644d6c1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d7400b10 con 0x5562d7510cc0 2023-09-10T01:13:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f644d6c1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d7400810 con 0x5562d75161a0 2023-09-10T01:13:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f644d6c1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d72e48c0 con 0x5562d7510300 2023-09-10T01:13:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f644d6c1b80 10 client.0 _ll_register_callbacks cb 0x5562d734aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:13:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f643a7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d7519fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f643a7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d7519fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57088/0 (socket says 172.21.15.44:57088) 2023-09-10T01:13:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f643affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d7510300 0x5562d751cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f643a7fc700 1 -- 172.21.15.44:0/3859090684 learned_addr learned my addr 172.21.15.44:0/3859090684 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f643b7fe700 1 --2- 172.21.15.44:0/3859090684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d75161a0 0x5562d751a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f643b7fe700 1 -- 172.21.15.44:0/3859090684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d7510300 msgr2=0x5562d751cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f643b7fe700 1 --2- 172.21.15.44:0/3859090684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d7510300 0x5562d751cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f643b7fe700 1 -- 172.21.15.44:0/3859090684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 msgr2=0x5562d7519fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f643b7fe700 1 --2- 172.21.15.44:0/3859090684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d7519fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f643b7fe700 1 -- 172.21.15.44:0/3859090684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c10c390 con 0x5562d75161a0 2023-09-10T01:13:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f643a7fc700 1 --2- 172.21.15.44:0/3859090684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d7519fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f643b7fe700 1 --2- 172.21.15.44:0/3859090684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d75161a0 0x5562d751a560 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f643400e730 tx=0x7f6434005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=131807d2d0f3913b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f641eff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:13:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.034+0000 7f641eff5700 1 -- 172.21.15.44:0/3859090684 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643400e040 con 0x5562d75161a0 2023-09-10T01:13:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.035+0000 7f641eff5700 1 -- 172.21.15.44:0/3859090684 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f64340048d0 con 0x5562d75161a0 2023-09-10T01:13:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.035+0000 7f644d6c1b80 1 -- 172.21.15.44:0/3859090684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562d73ba490 con 0x5562d75161a0 2023-09-10T01:13:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.035+0000 7f641eff5700 1 -- 172.21.15.44:0/3859090684 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6434003c00 con 0x5562d75161a0 2023-09-10T01:13:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.035+0000 7f644d6c1b80 10 client.28943 fetch_fsmap learned FSMap version 120 2023-09-10T01:13:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.035+0000 7f644d6c1b80 1 -- 172.21.15.44:0/3859090684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x7f643400e410 con 0x5562d75161a0 2023-09-10T01:13:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.035+0000 7f641eff5700 1 -- 172.21.15.44:0/3859090684 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5562d73ba490 con 0x5562d75161a0 2023-09-10T01:13:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.035+0000 7f641eff5700 1 -- 172.21.15.44:0/3859090684 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f6434015450 con 0x5562d75161a0 2023-09-10T01:13:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.035+0000 7f644d6c1b80 10 client.28943 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:13:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.035+0000 7f644d6c1b80 10 client.28943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:13:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.035+0000 7f644d6c1b80 20 client.28943 populate_metadata read hostname 'smithi044' 2023-09-10T01:13:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.035+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/3859090684 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5562d752a330 0x5562d752c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.035+0000 7f644d6c1b80 4 client.28943 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:13:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.035+0000 7f644d6c1b80 1 -- 172.21.15.44:0/3859090684 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562d7477840 con 0x5562d752a330 2023-09-10T01:13:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.035+0000 7f643affd700 1 --2- 172.21.15.44:0/3859090684 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5562d752a330 0x5562d752c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.036+0000 7f641eff5700 10 client.28943 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.036+0000 7f643affd700 1 --2- 172.21.15.44:0/3859090684 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5562d752a330 0x5562d752c750 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=120b907622447845 server_cookie=edeef8879044f3dd in_seq=0 out_seq=0 2023-09-10T01:13:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.037+0000 7f641eff5700 1 -- 172.21.15.44:0/3859090684 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5562d7477840 con 0x5562d752a330 2023-09-10T01:13:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.037+0000 7f641eff5700 10 client.28943 handle_command_reply: tid=0 2023-09-10T01:13:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.037+0000 7f644d6c1b80 1 client.28943 shutdown 2023-09-10T01:13:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.037+0000 7f644d6c1b80 1 -- 172.21.15.44:0/3859090684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d75161a0 msgr2=0x5562d751a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.037+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/3859090684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d75161a0 0x5562d751a560 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f643400e730 tx=0x7f6434005b90 comp rx=0 tx=0).stop 2023-09-10T01:13:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.037+0000 7f644d6c1b80 1 -- 172.21.15.44:0/3859090684 shutdown_connections 2023-09-10T01:13:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.037+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/3859090684 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5562d752a330 0x5562d752c750 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.037+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/3859090684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d75161a0 0x5562d751a560 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.037+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/3859090684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d7510300 0x5562d751cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.037+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/3859090684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d7519fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.037+0000 7f644d6c1b80 1 -- 172.21.15.44:0/3859090684 >> 172.21.15.44:0/3859090684 conn(0x5562d7485030 msgr2=0x5562d750a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.037+0000 7f641eff5700 0 client.28943 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.037+0000 7f644d6c1b80 1 -- 172.21.15.44:0/3859090684 shutdown_connections 2023-09-10T01:13:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.037+0000 7f644d6c1b80 1 -- 172.21.15.44:0/3859090684 wait complete. 2023-09-10T01:13:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.037+0000 7f644d6c1b80 20 client.28943 trim_cache size 0 max 16384 2023-09-10T01:13:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.056+0000 7f644d6c1b80 1 Processor -- start 2023-09-10T01:13:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.056+0000 7f644d6c1b80 1 -- start start 2023-09-10T01:13:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.056+0000 7f644d6c1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510300 0x5562d7521530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.056+0000 7f644d6c1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d751c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.056+0000 7f644d6c1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d75161a0 0x5562d750af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.056+0000 7f644d6c1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d7518560 con 0x5562d7510cc0 2023-09-10T01:13:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.056+0000 7f644d6c1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d7514d20 con 0x5562d7510300 2023-09-10T01:13:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.056+0000 7f644d6c1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d751b330 con 0x5562d75161a0 2023-09-10T01:13:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.056+0000 7f643a7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d751c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.056+0000 7f643affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510300 0x5562d7521530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.056+0000 7f643a7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d751c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57096/0 (socket says 172.21.15.44:57096) 2023-09-10T01:13:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.056+0000 7f643b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d75161a0 0x5562d750af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.056+0000 7f643a7fc700 1 -- 172.21.15.44:0/243616368 learned_addr learned my addr 172.21.15.44:0/243616368 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.057+0000 7f643affd700 1 -- 172.21.15.44:0/243616368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d75161a0 msgr2=0x5562d750af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.057+0000 7f643affd700 1 --2- 172.21.15.44:0/243616368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d75161a0 0x5562d750af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.057+0000 7f643affd700 1 -- 172.21.15.44:0/243616368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 msgr2=0x5562d751c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.057+0000 7f643affd700 1 --2- 172.21.15.44:0/243616368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d751c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.057+0000 7f643affd700 1 -- 172.21.15.44:0/243616368 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562d7277c00 con 0x5562d7510300 2023-09-10T01:13:02.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.057+0000 7f643b7fe700 1 --2- 172.21.15.44:0/243616368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d75161a0 0x5562d750af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:02.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.057+0000 7f643a7fc700 1 --2- 172.21.15.44:0/243616368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d751c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:02.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.057+0000 7f643affd700 1 --2- 172.21.15.44:0/243616368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510300 0x5562d7521530 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f642c0c9890 tx=0x7f642c0cb040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ec8448e40f9d274 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:02.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.057+0000 7f6421ffb700 1 -- 172.21.15.44:0/243616368 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c0995f0 con 0x5562d7510300 2023-09-10T01:13:02.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.057+0000 7f6421ffb700 1 -- 172.21.15.44:0/243616368 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f642c09d420 con 0x5562d7510300 2023-09-10T01:13:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.057+0000 7f6421ffb700 1 -- 172.21.15.44:0/243616368 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c0840b0 con 0x5562d7510300 2023-09-10T01:13:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.058+0000 7f644d6c1b80 1 -- 172.21.15.44:0/243616368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510300 msgr2=0x5562d7521530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.058+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/243616368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510300 0x5562d7521530 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f642c0c9890 tx=0x7f642c0cb040 comp rx=0 tx=0).stop 2023-09-10T01:13:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.058+0000 7f644d6c1b80 1 -- 172.21.15.44:0/243616368 shutdown_connections 2023-09-10T01:13:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.058+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/243616368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d75161a0 0x5562d750af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.058+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/243616368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510300 0x5562d7521530 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.058+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/243616368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d7510cc0 0x5562d751c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.058+0000 7f644d6c1b80 1 -- 172.21.15.44:0/243616368 >> 172.21.15.44:0/243616368 conn(0x5562d7485030 msgr2=0x5562d751b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.058+0000 7f644d6c1b80 1 -- 172.21.15.44:0/243616368 shutdown_connections 2023-09-10T01:13:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.058+0000 7f644d6c1b80 1 -- 172.21.15.44:0/243616368 wait complete. 2023-09-10T01:13:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.058+0000 7f644d6c1b80 1 -- start start 2023-09-10T01:13:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f644d6c1b80 1 Processor -- start 2023-09-10T01:13:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f644d6c1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d7510300 0x5562d75722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f644d6c1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510cc0 0x5562d7572820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f644d6c1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d75161a0 0x5562d757d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f644d6c1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d7572ef0 con 0x5562d75161a0 2023-09-10T01:13:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f644d6c1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d757d6b0 con 0x5562d7510cc0 2023-09-10T01:13:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f644d6c1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d757d7f0 con 0x5562d7510300 2023-09-10T01:13:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f644d6c1b80 10 client.0 _ll_register_callbacks cb 0x7f6430005700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:13:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f643affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d7510300 0x5562d75722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f643a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510cc0 0x5562d7572820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f643affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d7510300 0x5562d75722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35750/0 (socket says 172.21.15.44:35750) 2023-09-10T01:13:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f643b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d75161a0 0x5562d757d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f643affd700 1 -- 172.21.15.44:0/1505368540 learned_addr learned my addr 172.21.15.44:0/1505368540 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f643b7fe700 1 -- 172.21.15.44:0/1505368540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d7510300 msgr2=0x5562d75722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f643b7fe700 1 --2- 172.21.15.44:0/1505368540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d7510300 0x5562d75722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f643b7fe700 1 -- 172.21.15.44:0/1505368540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510cc0 msgr2=0x5562d7572820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f643b7fe700 1 --2- 172.21.15.44:0/1505368540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510cc0 0x5562d7572820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f643b7fe700 1 -- 172.21.15.44:0/1505368540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643000f8d0 con 0x5562d75161a0 2023-09-10T01:13:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f643affd700 1 --2- 172.21.15.44:0/1505368540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d7510300 0x5562d75722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.059+0000 7f643a7fc700 1 --2- 172.21.15.44:0/1505368540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510cc0 0x5562d7572820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.060+0000 7f643b7fe700 1 --2- 172.21.15.44:0/1505368540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d75161a0 0x5562d757d170 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f643400e700 tx=0x7f643400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b074ccc2c68dbb8f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.060+0000 7f641eff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:13:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.060+0000 7f641eff5700 1 -- 172.21.15.44:0/1505368540 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434015bd0 con 0x5562d75161a0 2023-09-10T01:13:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.060+0000 7f644d6c1b80 1 -- 172.21.15.44:0/1505368540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f643400f040 con 0x5562d75161a0 2023-09-10T01:13:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.060+0000 7f641eff5700 1 -- 172.21.15.44:0/1505368540 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f6434003a00 con 0x5562d75161a0 2023-09-10T01:13:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.060+0000 7f641eff5700 1 -- 172.21.15.44:0/1505368540 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6434003bc0 con 0x5562d75161a0 2023-09-10T01:13:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.060+0000 7f644d6c1b80 10 client.19281 fetch_fsmap learned FSMap version 120 2023-09-10T01:13:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.060+0000 7f644d6c1b80 1 -- 172.21.15.44:0/1505368540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x7f643400e410 con 0x5562d75161a0 2023-09-10T01:13:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.060+0000 7f641eff5700 1 -- 172.21.15.44:0/1505368540 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f643400f040 con 0x5562d75161a0 2023-09-10T01:13:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.060+0000 7f641eff5700 1 -- 172.21.15.44:0/1505368540 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f6434007af0 con 0x5562d75161a0 2023-09-10T01:13:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.060+0000 7f644d6c1b80 10 client.19281 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:13:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.060+0000 7f644d6c1b80 10 client.19281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:13:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.060+0000 7f644d6c1b80 20 client.19281 populate_metadata read hostname 'smithi044' 2023-09-10T01:13:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.060+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/1505368540 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5562d757eb90 0x5562d7580f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.060+0000 7f644d6c1b80 4 client.19281 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:13:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.060+0000 7f644d6c1b80 1 -- 172.21.15.44:0/1505368540 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5562d745b8c0 con 0x5562d757eb90 2023-09-10T01:13:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.061+0000 7f643affd700 1 --2- 172.21.15.44:0/1505368540 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5562d757eb90 0x5562d7580f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.061+0000 7f643affd700 1 --2- 172.21.15.44:0/1505368540 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5562d757eb90 0x5562d7580f80 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f689d611cc6c48e server_cookie=7f2312e032ce210d in_seq=0 out_seq=0 2023-09-10T01:13:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.061+0000 7f641eff5700 10 client.19281 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:02.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.077+0000 7f641eff5700 1 -- 172.21.15.44:0/1505368540 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5562d745b8c0 con 0x5562d757eb90 2023-09-10T01:13:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.077+0000 7f641eff5700 10 client.19281 handle_command_reply: tid=0 2023-09-10T01:13:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.077+0000 7f644d6c1b80 1 client.19281 shutdown 2023-09-10T01:13:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.078+0000 7f644d6c1b80 1 -- 172.21.15.44:0/1505368540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d75161a0 msgr2=0x5562d757d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.078+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/1505368540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d75161a0 0x5562d757d170 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f643400e700 tx=0x7f643400eda0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.078+0000 7f644d6c1b80 1 -- 172.21.15.44:0/1505368540 shutdown_connections 2023-09-10T01:13:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.078+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/1505368540 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5562d757eb90 0x5562d7580f80 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.078+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/1505368540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5562d75161a0 0x5562d757d170 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.078+0000 7f641eff5700 0 client.19281 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.078+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/1505368540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562d7510300 0x5562d75722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.078+0000 7f644d6c1b80 1 --2- 172.21.15.44:0/1505368540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562d7510cc0 0x5562d7572820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.078+0000 7f644d6c1b80 1 -- 172.21.15.44:0/1505368540 >> 172.21.15.44:0/1505368540 conn(0x5562d7485030 msgr2=0x5562d750fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.078+0000 7f644d6c1b80 1 -- 172.21.15.44:0/1505368540 shutdown_connections 2023-09-10T01:13:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.078+0000 7f644d6c1b80 1 -- 172.21.15.44:0/1505368540 wait complete. 2023-09-10T01:13:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.078+0000 7f644d6c1b80 20 client.19281 trim_cache size 0 max 16384 2023-09-10T01:13:02.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:13:02.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:13:02.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:13:02.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:13:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.079+0000 7f64414fb700 1 -- 172.21.15.44:0/2810369907 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f64240a07b0 msgr2=0x7f64240a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.079+0000 7f64414fb700 1 --2- 172.21.15.44:0/2810369907 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f64240a07b0 0x7f64240a2c70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f643000b2e0 tx=0x7f6430004500 comp rx=0 tx=0).stop 2023-09-10T01:13:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.079+0000 7f64414fb700 1 -- 172.21.15.44:0/2810369907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f643c0fee80 msgr2=0x7f643c194380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.079+0000 7f64414fb700 1 --2- 172.21.15.44:0/2810369907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f643c0fee80 0x7f643c194380 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f642c00eab0 tx=0x7f642c00edc0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.079+0000 7f64414fb700 1 -- 172.21.15.44:0/2810369907 shutdown_connections 2023-09-10T01:13:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.079+0000 7f64414fb700 1 --2- 172.21.15.44:0/2810369907 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f64240a07b0 0x7f64240a2c70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.079+0000 7f64414fb700 1 --2- 172.21.15.44:0/2810369907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f643c1a2750 0x7f643c1a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.079+0000 7f64414fb700 1 --2- 172.21.15.44:0/2810369907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f643c0fee80 0x7f643c194380 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.079+0000 7f64414fb700 1 --2- 172.21.15.44:0/2810369907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c103fa0 0x7f643c1948c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.079+0000 7f64414fb700 1 -- 172.21.15.44:0/2810369907 >> 172.21.15.44:0/2810369907 conn(0x7f643c0769d0 msgr2=0x7f643c1036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.079+0000 7f64414fb700 1 -- 172.21.15.44:0/2810369907 shutdown_connections 2023-09-10T01:13:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:02.079+0000 7f64414fb700 1 -- 172.21.15.44:0/2810369907 wait complete. 2023-09-10T01:13:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:02 smithi131 ceph-mon[103713]: pgmap v1032: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 234 op/s 2023-09-10T01:13:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:02 smithi044 ceph-mon[98676]: pgmap v1032: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 234 op/s 2023-09-10T01:13:03.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:02 smithi099 ceph-mon[102692]: pgmap v1032: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 234 op/s 2023-09-10T01:13:05.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:04 smithi131 ceph-mon[103713]: pgmap v1033: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2023-09-10T01:13:05.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:04 smithi131 ceph-mon[103713]: osdmap e149: 12 total, 12 up, 12 in 2023-09-10T01:13:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:04 smithi044 ceph-mon[98676]: pgmap v1033: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2023-09-10T01:13:05.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:04 smithi044 ceph-mon[98676]: osdmap e149: 12 total, 12 up, 12 in 2023-09-10T01:13:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:04 smithi099 ceph-mon[102692]: pgmap v1033: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2023-09-10T01:13:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:04 smithi099 ceph-mon[102692]: osdmap e149: 12 total, 12 up, 12 in 2023-09-10T01:13:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:06 smithi131 ceph-mon[103713]: pgmap v1035: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-09-10T01:13:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:06 smithi044 ceph-mon[98676]: pgmap v1035: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-09-10T01:13:07.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:06 smithi099 ceph-mon[102692]: pgmap v1035: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-09-10T01:13:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:08 smithi131 ceph-mon[103713]: pgmap v1036: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-09-10T01:13:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:08 smithi044 ceph-mon[98676]: pgmap v1036: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-09-10T01:13:09.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:08 smithi099 ceph-mon[102692]: pgmap v1036: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-09-10T01:13:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:10 smithi131 ceph-mon[103713]: pgmap v1037: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-09-10T01:13:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:10 smithi131 ceph-mon[103713]: osdmap e150: 12 total, 12 up, 12 in 2023-09-10T01:13:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:10 smithi044 ceph-mon[98676]: pgmap v1037: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-09-10T01:13:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:10 smithi044 ceph-mon[98676]: osdmap e150: 12 total, 12 up, 12 in 2023-09-10T01:13:11.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:10 smithi099 ceph-mon[102692]: pgmap v1037: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-09-10T01:13:11.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:10 smithi099 ceph-mon[102692]: osdmap e150: 12 total, 12 up, 12 in 2023-09-10T01:13:13.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:12 smithi131 ceph-mon[103713]: pgmap v1039: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 227 op/s 2023-09-10T01:13:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:12 smithi044 ceph-mon[98676]: pgmap v1039: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 227 op/s 2023-09-10T01:13:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:12 smithi099 ceph-mon[102692]: pgmap v1039: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 227 op/s 2023-09-10T01:13:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.216+0000 7fa4edae6700 1 -- 172.21.15.44:0/1693950272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4e8068920 msgr2=0x7fa4e8068da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.216+0000 7fa4edae6700 1 --2- 172.21.15.44:0/1693950272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4e8068920 0x7fa4e8068da0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009990 tx=0x7fa4dc009ca0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.216+0000 7fa4edae6700 1 -- 172.21.15.44:0/1693950272 shutdown_connections 2023-09-10T01:13:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.216+0000 7fa4edae6700 1 --2- 172.21.15.44:0/1693950272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4e80692e0 0x7fa4e810d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.216+0000 7fa4edae6700 1 --2- 172.21.15.44:0/1693950272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4e8106620 0x7fa4e8106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.216+0000 7fa4edae6700 1 --2- 172.21.15.44:0/1693950272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4e8068920 0x7fa4e8068da0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.216+0000 7fa4edae6700 1 -- 172.21.15.44:0/1693950272 >> 172.21.15.44:0/1693950272 conn(0x7fa4e8076600 msgr2=0x7fa4e8076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.216+0000 7fa4edae6700 1 -- 172.21.15.44:0/1693950272 shutdown_connections 2023-09-10T01:13:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.216+0000 7fa4edae6700 1 -- 172.21.15.44:0/1693950272 wait complete. 2023-09-10T01:13:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.217+0000 7fa4edae6700 1 Processor -- start 2023-09-10T01:13:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.217+0000 7fa4edae6700 1 -- start start 2023-09-10T01:13:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.217+0000 7fa4edae6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4e8068920 0x7fa4e819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.217+0000 7fa4edae6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4e80692e0 0x7fa4e819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.217+0000 7fa4edae6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4e8106620 0x7fa4e8196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.217+0000 7fa4edae6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e8196ac0 con 0x7fa4e8068920 2023-09-10T01:13:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.217+0000 7fa4edae6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e8196c30 con 0x7fa4e8106620 2023-09-10T01:13:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.217+0000 7fa4edae6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e8196df0 con 0x7fa4e80692e0 2023-09-10T01:13:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.217+0000 7fa4e77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4e8068920 0x7fa4e819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.217+0000 7fa4e7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4e8106620 0x7fa4e8196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.217+0000 7fa4e77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4e8068920 0x7fa4e819c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57114/0 (socket says 172.21.15.44:57114) 2023-09-10T01:13:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.217+0000 7fa4e77fe700 1 -- 172.21.15.44:0/3727009558 learned_addr learned my addr 172.21.15.44:0/3727009558 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.217+0000 7fa4e6ffd700 1 --2- 172.21.15.44:0/3727009558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4e80692e0 0x7fa4e819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.218+0000 7fa4e7fff700 1 -- 172.21.15.44:0/3727009558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4e80692e0 msgr2=0x7fa4e819cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.218+0000 7fa4e7fff700 1 --2- 172.21.15.44:0/3727009558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4e80692e0 0x7fa4e819cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.218+0000 7fa4e7fff700 1 -- 172.21.15.44:0/3727009558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4e8068920 msgr2=0x7fa4e819c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.218+0000 7fa4e7fff700 1 --2- 172.21.15.44:0/3727009558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4e8068920 0x7fa4e819c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.218+0000 7fa4e7fff700 1 -- 172.21.15.44:0/3727009558 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e0009c40 con 0x7fa4e8106620 2023-09-10T01:13:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.218+0000 7fa4e77fe700 1 --2- 172.21.15.44:0/3727009558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4e8068920 0x7fa4e819c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.218+0000 7fa4e7fff700 1 --2- 172.21.15.44:0/3727009558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4e8106620 0x7fa4e8196450 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000efb0 tx=0x7fa4e000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.218+0000 7fa4e4ff9700 1 -- 172.21.15.44:0/3727009558 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0003a00 con 0x7fa4e8106620 2023-09-10T01:13:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.218+0000 7fa4edae6700 1 -- 172.21.15.44:0/3727009558 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4dc0095f0 con 0x7fa4e8106620 2023-09-10T01:13:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.218+0000 7fa4e4ff9700 1 -- 172.21.15.44:0/3727009558 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa4e000f040 con 0x7fa4e8106620 2023-09-10T01:13:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.218+0000 7fa4e4ff9700 1 -- 172.21.15.44:0/3727009558 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00044d0 con 0x7fa4e8106620 2023-09-10T01:13:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.218+0000 7fa4edae6700 1 -- 172.21.15.44:0/3727009558 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e80ff470 con 0x7fa4e8106620 2023-09-10T01:13:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.221+0000 7fa4e4ff9700 1 -- 172.21.15.44:0/3727009558 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa4e000cbc0 con 0x7fa4e8106620 2023-09-10T01:13:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.222+0000 7fa4e4ff9700 1 --2- 172.21.15.44:0/3727009558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4d00a07b0 0x7fa4d00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.222+0000 7fa4e4ff9700 1 -- 172.21.15.44:0/3727009558 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa4e00cbfe0 con 0x7fa4e8106620 2023-09-10T01:13:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.222+0000 7fa4e77fe700 1 --2- 172.21.15.44:0/3727009558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4d00a07b0 0x7fa4d00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.222+0000 7fa4f9cacb80 1 Processor -- start 2023-09-10T01:13:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.222+0000 7fa4f9cacb80 1 -- start start 2023-09-10T01:13:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.222+0000 7fa4f9cacb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622005161a0 0x562200516580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.222+0000 7fa4f9cacb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510300 0x562200510780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.222+0000 7fa4f9cacb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510cc0 0x56220051cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.222+0000 7fa4f9cacb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622004696d0 con 0x5622005161a0 2023-09-10T01:13:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.222+0000 7fa4f9cacb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562200466ff0 con 0x562200510cc0 2023-09-10T01:13:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.222+0000 7fa4f9cacb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622004628e0 con 0x562200510300 2023-09-10T01:13:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.222+0000 7fa4e77fe700 1 --2- 172.21.15.44:0/3727009558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4d00a07b0 0x7fa4d00a2c70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8011550 tx=0x7fa4d8009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.223+0000 7fa4e77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622005161a0 0x562200516580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.223+0000 7fa4e6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510300 0x562200510780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.223+0000 7fa4e77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622005161a0 0x562200516580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57122/0 (socket says 172.21.15.44:57122) 2023-09-10T01:13:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.223+0000 7fa4e77fe700 1 -- 172.21.15.44:0/2432930909 learned_addr learned my addr 172.21.15.44:0/2432930909 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.223+0000 7fa4e7fff700 1 --2- 172.21.15.44:0/2432930909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510cc0 0x56220051cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.223+0000 7fa4e6ffd700 1 -- 172.21.15.44:0/2432930909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510cc0 msgr2=0x56220051cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.223+0000 7fa4e6ffd700 1 --2- 172.21.15.44:0/2432930909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510cc0 0x56220051cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.223+0000 7fa4e6ffd700 1 -- 172.21.15.44:0/2432930909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622005161a0 msgr2=0x562200516580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.223+0000 7fa4e6ffd700 1 --2- 172.21.15.44:0/2432930909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622005161a0 0x562200516580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.223+0000 7fa4e6ffd700 1 -- 172.21.15.44:0/2432930909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56220028ed80 con 0x562200510300 2023-09-10T01:13:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.223+0000 7fa4e77fe700 1 --2- 172.21.15.44:0/2432930909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622005161a0 0x562200516580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.223+0000 7fa4e6ffd700 1 --2- 172.21.15.44:0/2432930909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510300 0x562200510780 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc005910 tx=0x7fa4dc005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1055463c5bd0836 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.223+0000 7fa4cdffb700 1 -- 172.21.15.44:0/2432930909 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc01d070 con 0x562200510300 2023-09-10T01:13:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.223+0000 7fa4cdffb700 1 -- 172.21.15.44:0/2432930909 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa4dc01e040 con 0x562200510300 2023-09-10T01:13:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.223+0000 7fa4cdffb700 1 -- 172.21.15.44:0/2432930909 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc004760 con 0x562200510300 2023-09-10T01:13:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.224+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/2432930909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510300 msgr2=0x562200510780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.224+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/2432930909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510300 0x562200510780 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc005910 tx=0x7fa4dc005f00 comp rx=0 tx=0).stop 2023-09-10T01:13:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.224+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/2432930909 shutdown_connections 2023-09-10T01:13:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.224+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/2432930909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510cc0 0x56220051cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.224+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/2432930909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622005161a0 0x562200516580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.224+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/2432930909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510300 0x562200510780 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.224+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/2432930909 >> 172.21.15.44:0/2432930909 conn(0x562200485030 msgr2=0x56220050b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.224+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/2432930909 shutdown_connections 2023-09-10T01:13:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.224+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/2432930909 wait complete. 2023-09-10T01:13:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.225+0000 7fa4f9cacb80 1 -- start start 2023-09-10T01:13:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.225+0000 7fa4f9cacb80 1 Processor -- start 2023-09-10T01:13:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.225+0000 7fa4f9cacb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510300 0x56220051cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.225+0000 7fa4f9cacb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562200510cc0 0x56220051a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.225+0000 7fa4f9cacb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622005161a0 0x56220051a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.225+0000 7fa4f9cacb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562200400b10 con 0x562200510cc0 2023-09-10T01:13:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.225+0000 7fa4f9cacb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562200400810 con 0x5622005161a0 2023-09-10T01:13:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.225+0000 7fa4f9cacb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622002e48c0 con 0x562200510300 2023-09-10T01:13:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.225+0000 7fa4f9cacb80 10 client.0 _ll_register_callbacks cb 0x56220034aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:13:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.225+0000 7fa4e6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562200510cc0 0x56220051a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.225+0000 7fa4e77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510300 0x56220051cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.225+0000 7fa4e77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510300 0x56220051cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35778/0 (socket says 172.21.15.44:35778) 2023-09-10T01:13:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.225+0000 7fa4e77fe700 1 -- 172.21.15.44:0/180308773 learned_addr learned my addr 172.21.15.44:0/180308773 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.225+0000 7fa4e6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562200510cc0 0x56220051a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57130/0 (socket says 172.21.15.44:57130) 2023-09-10T01:13:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.225+0000 7fa4e7fff700 1 --2- 172.21.15.44:0/180308773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622005161a0 0x56220051a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.226+0000 7fa4e77fe700 1 -- 172.21.15.44:0/180308773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622005161a0 msgr2=0x56220051a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.226+0000 7fa4e77fe700 1 --2- 172.21.15.44:0/180308773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622005161a0 0x56220051a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.226+0000 7fa4e77fe700 1 -- 172.21.15.44:0/180308773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562200510cc0 msgr2=0x56220051a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.226+0000 7fa4e77fe700 1 --2- 172.21.15.44:0/180308773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562200510cc0 0x56220051a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.226+0000 7fa4e77fe700 1 -- 172.21.15.44:0/180308773 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d8017070 con 0x562200510300 2023-09-10T01:13:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.226+0000 7fa4e6ffd700 1 --2- 172.21.15.44:0/180308773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562200510cc0 0x56220051a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.226+0000 7fa4e7fff700 1 --2- 172.21.15.44:0/180308773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622005161a0 0x56220051a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.226+0000 7fa4e77fe700 1 --2- 172.21.15.44:0/180308773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510300 0x56220051cdc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8011ca0 tx=0x7fa4d80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a95d6e95c4c061b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.226+0000 7fa4c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:13:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.226+0000 7fa4c6ffd700 1 -- 172.21.15.44:0/180308773 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d801a5b0 con 0x562200510300 2023-09-10T01:13:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.226+0000 7fa4c6ffd700 1 -- 172.21.15.44:0/180308773 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa4d801a710 con 0x562200510300 2023-09-10T01:13:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.226+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/180308773 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622003ba490 con 0x562200510300 2023-09-10T01:13:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.226+0000 7fa4c6ffd700 1 -- 172.21.15.44:0/180308773 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4d801a8d0 con 0x562200510300 2023-09-10T01:13:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.226+0000 7fa4f9cacb80 10 client.28903 fetch_fsmap learned FSMap version 120 2023-09-10T01:13:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.226+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/180308773 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x7fa4e810dd30 con 0x562200510300 2023-09-10T01:13:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.227+0000 7fa4c6ffd700 1 -- 172.21.15.44:0/180308773 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5622003ba490 con 0x562200510300 2023-09-10T01:13:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.227+0000 7fa4c6ffd700 1 -- 172.21.15.44:0/180308773 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fa4d8028970 con 0x562200510300 2023-09-10T01:13:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.227+0000 7fa4f9cacb80 10 client.28903 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:13:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.227+0000 7fa4f9cacb80 10 client.28903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:13:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.227+0000 7fa4f9cacb80 20 client.28903 populate_metadata read hostname 'smithi044' 2023-09-10T01:13:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.227+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/180308773 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56220052a480 0x56220052c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.227+0000 7fa4f9cacb80 4 client.28903 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:13:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.227+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/180308773 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562200477840 con 0x56220052a480 2023-09-10T01:13:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.227+0000 7fa4e6ffd700 1 --2- 172.21.15.44:0/180308773 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56220052a480 0x56220052c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.228+0000 7fa4c6ffd700 10 client.28903 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.228+0000 7fa4e6ffd700 1 --2- 172.21.15.44:0/180308773 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56220052a480 0x56220052c8a0 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dc6935878ceaa57 server_cookie=d5c14462afdfd95e in_seq=0 out_seq=0 2023-09-10T01:13:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.228+0000 7fa4c6ffd700 1 -- 172.21.15.44:0/180308773 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562200477840 con 0x56220052a480 2023-09-10T01:13:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.228+0000 7fa4c6ffd700 10 client.28903 handle_command_reply: tid=0 2023-09-10T01:13:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.228+0000 7fa4f9cacb80 1 client.28903 shutdown 2023-09-10T01:13:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.228+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/180308773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510300 msgr2=0x56220051cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.228+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/180308773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510300 0x56220051cdc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8011ca0 tx=0x7fa4d80047c0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.228+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/180308773 shutdown_connections 2023-09-10T01:13:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.228+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/180308773 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56220052a480 0x56220052c8a0 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.228+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/180308773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622005161a0 0x56220051a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.228+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/180308773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510300 0x56220051cdc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.228+0000 7fa4c6ffd700 0 client.28903 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.228+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/180308773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562200510cc0 0x56220051a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.228+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/180308773 >> 172.21.15.44:0/180308773 conn(0x562200485030 msgr2=0x56220050a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.228+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/180308773 shutdown_connections 2023-09-10T01:13:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.229+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/180308773 wait complete. 2023-09-10T01:13:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.229+0000 7fa4f9cacb80 20 client.28903 trim_cache size 0 max 16384 2023-09-10T01:13:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.248+0000 7fa4f9cacb80 1 Processor -- start 2023-09-10T01:13:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.248+0000 7fa4f9cacb80 1 -- start start 2023-09-10T01:13:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4f9cacb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562200510300 0x562200521490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4f9cacb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510cc0 0x56220051c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4f9cacb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5622005161a0 0x562200514250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4f9cacb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562200511340 con 0x562200510300 2023-09-10T01:13:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4f9cacb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220050c930 con 0x562200510cc0 2023-09-10T01:13:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4f9cacb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220050cd10 con 0x5622005161a0 2023-09-10T01:13:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4e77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562200510300 0x562200521490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4e6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510cc0 0x56220051c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4e77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562200510300 0x562200521490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57136/0 (socket says 172.21.15.44:57136) 2023-09-10T01:13:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4e7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5622005161a0 0x562200514250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4e77fe700 1 -- 172.21.15.44:0/1491184564 learned_addr learned my addr 172.21.15.44:0/1491184564 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4e6ffd700 1 -- 172.21.15.44:0/1491184564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5622005161a0 msgr2=0x562200514250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4e6ffd700 1 --2- 172.21.15.44:0/1491184564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5622005161a0 0x562200514250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4e6ffd700 1 -- 172.21.15.44:0/1491184564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562200510300 msgr2=0x562200521490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4e6ffd700 1 --2- 172.21.15.44:0/1491184564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562200510300 0x562200521490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4e6ffd700 1 -- 172.21.15.44:0/1491184564 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562200277c00 con 0x562200510cc0 2023-09-10T01:13:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4e77fe700 1 --2- 172.21.15.44:0/1491184564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562200510300 0x562200521490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.249+0000 7fa4e6ffd700 1 --2- 172.21.15.44:0/1491184564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510cc0 0x56220051c960 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc005b40 tx=0x7fa4dc009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dfde15d55b7c702 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.250+0000 7fa4ce7fc700 1 -- 172.21.15.44:0/1491184564 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc01d040 con 0x562200510cc0 2023-09-10T01:13:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.250+0000 7fa4ce7fc700 1 -- 172.21.15.44:0/1491184564 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa4dc00bb30 con 0x562200510cc0 2023-09-10T01:13:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.250+0000 7fa4ce7fc700 1 -- 172.21.15.44:0/1491184564 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc004830 con 0x562200510cc0 2023-09-10T01:13:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.250+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/1491184564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510cc0 msgr2=0x56220051c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.250+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/1491184564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510cc0 0x56220051c960 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc005b40 tx=0x7fa4dc009ca0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.250+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/1491184564 shutdown_connections 2023-09-10T01:13:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.250+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/1491184564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5622005161a0 0x562200514250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.250+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/1491184564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562200510300 0x562200521490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.251+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/1491184564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510cc0 0x56220051c960 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.251+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/1491184564 >> 172.21.15.44:0/1491184564 conn(0x562200485030 msgr2=0x562200525bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.251+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/1491184564 shutdown_connections 2023-09-10T01:13:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.251+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/1491184564 wait complete. 2023-09-10T01:13:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.251+0000 7fa4f9cacb80 1 -- start start 2023-09-10T01:13:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.251+0000 7fa4f9cacb80 1 Processor -- start 2023-09-10T01:13:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.251+0000 7fa4f9cacb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510300 0x5622005215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.251+0000 7fa4f9cacb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510cc0 0x56220051d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.251+0000 7fa4f9cacb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622005161a0 0x56220051d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.251+0000 7fa4f9cacb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220051dfb0 con 0x5622005161a0 2023-09-10T01:13:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.251+0000 7fa4f9cacb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220051e120 con 0x562200510300 2023-09-10T01:13:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.251+0000 7fa4f9cacb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220057d280 con 0x562200510cc0 2023-09-10T01:13:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.251+0000 7fa4f9cacb80 10 client.0 _ll_register_callbacks cb 0x7fa4d800eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:13:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4e6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510cc0 0x56220051d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4e77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510300 0x5622005215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4e6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510cc0 0x56220051d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35794/0 (socket says 172.21.15.44:35794) 2023-09-10T01:13:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4e77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510300 0x5622005215d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58882/0 (socket says 172.21.15.44:58882) 2023-09-10T01:13:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4e6ffd700 1 -- 172.21.15.44:0/2958001806 learned_addr learned my addr 172.21.15.44:0/2958001806 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4e7fff700 1 --2- 172.21.15.44:0/2958001806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622005161a0 0x56220051d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4e77fe700 1 -- 172.21.15.44:0/2958001806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510cc0 msgr2=0x56220051d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4e77fe700 1 --2- 172.21.15.44:0/2958001806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510cc0 0x56220051d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4e77fe700 1 -- 172.21.15.44:0/2958001806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622005161a0 msgr2=0x56220051d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4e77fe700 1 --2- 172.21.15.44:0/2958001806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622005161a0 0x56220051d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4e77fe700 1 -- 172.21.15.44:0/2958001806 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e810dd30 con 0x562200510300 2023-09-10T01:13:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4e6ffd700 1 --2- 172.21.15.44:0/2958001806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510cc0 0x56220051d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4e77fe700 1 --2- 172.21.15.44:0/2958001806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510300 0x5622005215d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8028d20 tx=0x7fa4d80289f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a620f27ce4f4b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:13:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4c6ffd700 1 -- 172.21.15.44:0/2958001806 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8028650 con 0x562200510300 2023-09-10T01:13:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/2958001806 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4dc01e040 con 0x562200510300 2023-09-10T01:13:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.252+0000 7fa4c6ffd700 1 -- 172.21.15.44:0/2958001806 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa4d80287b0 con 0x562200510300 2023-09-10T01:13:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.253+0000 7fa4c6ffd700 1 -- 172.21.15.44:0/2958001806 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4d8004830 con 0x562200510300 2023-09-10T01:13:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.253+0000 7fa4f9cacb80 10 client.28967 fetch_fsmap learned FSMap version 120 2023-09-10T01:13:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.253+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/2958001806 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x56220028ed80 con 0x562200510300 2023-09-10T01:13:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.253+0000 7fa4c6ffd700 1 -- 172.21.15.44:0/2958001806 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa4dc01e040 con 0x562200510300 2023-09-10T01:13:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.253+0000 7fa4c6ffd700 1 -- 172.21.15.44:0/2958001806 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fa4d80054f0 con 0x562200510300 2023-09-10T01:13:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.253+0000 7fa4f9cacb80 10 client.28967 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:13:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.253+0000 7fa4f9cacb80 10 client.28967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:13:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.253+0000 7fa4f9cacb80 20 client.28967 populate_metadata read hostname 'smithi044' 2023-09-10T01:13:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.253+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/2958001806 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56220057e8a0 0x562200580cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.253+0000 7fa4f9cacb80 4 client.28967 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:13:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.253+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/2958001806 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56220045b8c0 con 0x56220057e8a0 2023-09-10T01:13:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.253+0000 7fa4e6ffd700 1 --2- 172.21.15.44:0/2958001806 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56220057e8a0 0x562200580cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.253+0000 7fa4c6ffd700 10 client.28967 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.253+0000 7fa4e6ffd700 1 --2- 172.21.15.44:0/2958001806 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56220057e8a0 0x562200580cc0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=841a1b888ba52201 server_cookie=52a48a6a9665dda7 in_seq=0 out_seq=0 2023-09-10T01:13:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.273+0000 7fa4c6ffd700 1 -- 172.21.15.44:0/2958001806 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56220045b8c0 con 0x56220057e8a0 2023-09-10T01:13:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.273+0000 7fa4c6ffd700 10 client.28967 handle_command_reply: tid=0 2023-09-10T01:13:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.273+0000 7fa4f9cacb80 1 client.28967 shutdown 2023-09-10T01:13:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.273+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/2958001806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510300 msgr2=0x5622005215d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.273+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/2958001806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510300 0x5622005215d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8028d20 tx=0x7fa4d80289f0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/2958001806 shutdown_connections 2023-09-10T01:13:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/2958001806 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56220057e8a0 0x562200580cc0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/2958001806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622005161a0 0x56220051d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4c6ffd700 0 client.28967 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/2958001806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562200510300 0x5622005215d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4f9cacb80 1 --2- 172.21.15.44:0/2958001806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562200510cc0 0x56220051d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/2958001806 >> 172.21.15.44:0/2958001806 conn(0x562200485030 msgr2=0x56220050b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/2958001806 shutdown_connections 2023-09-10T01:13:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4f9cacb80 1 -- 172.21.15.44:0/2958001806 wait complete. 2023-09-10T01:13:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4f9cacb80 20 client.28967 trim_cache size 0 max 16384 2023-09-10T01:13:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:13:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:13:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:13:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:13:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4edae6700 1 -- 172.21.15.44:0/3727009558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4d00a07b0 msgr2=0x7fa4d00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4edae6700 1 --2- 172.21.15.44:0/3727009558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4d00a07b0 0x7fa4d00a2c70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8011550 tx=0x7fa4d8009260 comp rx=0 tx=0).stop 2023-09-10T01:13:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4edae6700 1 -- 172.21.15.44:0/3727009558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4e8106620 msgr2=0x7fa4e8196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4edae6700 1 --2- 172.21.15.44:0/3727009558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4e8106620 0x7fa4e8196450 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000efb0 tx=0x7fa4e000c3a0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4edae6700 1 -- 172.21.15.44:0/3727009558 shutdown_connections 2023-09-10T01:13:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4edae6700 1 --2- 172.21.15.44:0/3727009558 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa4d00a07b0 0x7fa4d00a2c70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4edae6700 1 --2- 172.21.15.44:0/3727009558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4e8106620 0x7fa4e8196450 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4edae6700 1 --2- 172.21.15.44:0/3727009558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4e8068920 0x7fa4e819c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4edae6700 1 --2- 172.21.15.44:0/3727009558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4e80692e0 0x7fa4e819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4edae6700 1 -- 172.21.15.44:0/3727009558 >> 172.21.15.44:0/3727009558 conn(0x7fa4e8076600 msgr2=0x7fa4e80fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.274+0000 7fa4edae6700 1 -- 172.21.15.44:0/3727009558 shutdown_connections 2023-09-10T01:13:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:15.275+0000 7fa4edae6700 1 -- 172.21.15.44:0/3727009558 wait complete. 2023-09-10T01:13:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:15 smithi131 ceph-mon[103713]: pgmap v1040: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 188 op/s 2023-09-10T01:13:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:15 smithi044 ceph-mon[98676]: pgmap v1040: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 188 op/s 2023-09-10T01:13:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:15 smithi099 ceph-mon[102692]: pgmap v1040: 129 pgs: 129 active+clean; 10 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 188 op/s 2023-09-10T01:13:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:17 smithi131 ceph-mon[103713]: pgmap v1041: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:13:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:17 smithi044 ceph-mon[98676]: pgmap v1041: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:13:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:17 smithi099 ceph-mon[102692]: pgmap v1041: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:13:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:19 smithi131 ceph-mon[103713]: pgmap v1042: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:13:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:19 smithi044 ceph-mon[98676]: pgmap v1042: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:13:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:19 smithi099 ceph-mon[102692]: pgmap v1042: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:13:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:21 smithi131 ceph-mon[103713]: pgmap v1043: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:13:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:21 smithi044 ceph-mon[98676]: pgmap v1043: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:13:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:21 smithi099 ceph-mon[102692]: pgmap v1043: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-09-10T01:13:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:23 smithi131 ceph-mon[103713]: pgmap v1044: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2023-09-10T01:13:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:23 smithi044 ceph-mon[98676]: pgmap v1044: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2023-09-10T01:13:23.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:23 smithi099 ceph-mon[102692]: pgmap v1044: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2023-09-10T01:13:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:25 smithi131 ceph-mon[103713]: pgmap v1045: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 157 op/s 2023-09-10T01:13:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:25 smithi044 ceph-mon[98676]: pgmap v1045: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 157 op/s 2023-09-10T01:13:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:25 smithi099 ceph-mon[102692]: pgmap v1045: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 157 op/s 2023-09-10T01:13:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:27 smithi131 ceph-mon[103713]: pgmap v1046: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 223 op/s 2023-09-10T01:13:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:27 smithi044 ceph-mon[98676]: pgmap v1046: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 223 op/s 2023-09-10T01:13:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:27 smithi099 ceph-mon[102692]: pgmap v1046: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 223 op/s 2023-09-10T01:13:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.398+0000 7fda783e3700 1 -- 172.21.15.44:0/1728861863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda70101470 msgr2=0x7fda700679f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.398+0000 7fda783e3700 1 --2- 172.21.15.44:0/1728861863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda70101470 0x7fda700679f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009990 tx=0x7fda6c009ca0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.399+0000 7fda783e3700 1 -- 172.21.15.44:0/1728861863 shutdown_connections 2023-09-10T01:13:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.399+0000 7fda783e3700 1 --2- 172.21.15.44:0/1728861863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda7010cdd0 0x7fda7010f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.399+0000 7fda783e3700 1 --2- 172.21.15.44:0/1728861863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda70101470 0x7fda700679f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.399+0000 7fda783e3700 1 --2- 172.21.15.44:0/1728861863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda70067f30 0x7fda700683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.399+0000 7fda783e3700 1 -- 172.21.15.44:0/1728861863 >> 172.21.15.44:0/1728861863 conn(0x7fda70074f30 msgr2=0x7fda70075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.399+0000 7fda783e3700 1 -- 172.21.15.44:0/1728861863 shutdown_connections 2023-09-10T01:13:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.399+0000 7fda783e3700 1 -- 172.21.15.44:0/1728861863 wait complete. 2023-09-10T01:13:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.399+0000 7fda783e3700 1 Processor -- start 2023-09-10T01:13:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.399+0000 7fda783e3700 1 -- start start 2023-09-10T01:13:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda783e3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda70067f30 0x7fda7019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda783e3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda70101470 0x7fda7019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda783e3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda7010cdd0 0x7fda701985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda783e3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda70198c60 con 0x7fda7010cdd0 2023-09-10T01:13:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda783e3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda70198dd0 con 0x7fda70101470 2023-09-10T01:13:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda783e3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda70198f90 con 0x7fda70067f30 2023-09-10T01:13:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda7617f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda70067f30 0x7fda7019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda7617f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda70067f30 0x7fda7019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35804/0 (socket says 172.21.15.44:35804) 2023-09-10T01:13:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda76980700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda7010cdd0 0x7fda701985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda7617f700 1 -- 172.21.15.44:0/1344877995 learned_addr learned my addr 172.21.15.44:0/1344877995 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda7597e700 1 --2- 172.21.15.44:0/1344877995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda70101470 0x7fda7019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda76980700 1 -- 172.21.15.44:0/1344877995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda70067f30 msgr2=0x7fda7019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda76980700 1 --2- 172.21.15.44:0/1344877995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda70067f30 0x7fda7019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda76980700 1 -- 172.21.15.44:0/1344877995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda70101470 msgr2=0x7fda7019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda76980700 1 --2- 172.21.15.44:0/1344877995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda70101470 0x7fda7019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda76980700 1 -- 172.21.15.44:0/1344877995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c0095f0 con 0x7fda7010cdd0 2023-09-10T01:13:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda7617f700 1 --2- 172.21.15.44:0/1344877995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda70067f30 0x7fda7019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.400+0000 7fda76980700 1 --2- 172.21.15.44:0/1344877995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda7010cdd0 0x7fda701985f0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fda6800efc0 tx=0x7fda6800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.401+0000 7fda677fe700 1 -- 172.21.15.44:0/1344877995 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68018070 con 0x7fda7010cdd0 2023-09-10T01:13:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.401+0000 7fda783e3700 1 -- 172.21.15.44:0/1344877995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda70199210 con 0x7fda7010cdd0 2023-09-10T01:13:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.401+0000 7fda677fe700 1 -- 172.21.15.44:0/1344877995 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda6800f040 con 0x7fda7010cdd0 2023-09-10T01:13:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.401+0000 7fda677fe700 1 -- 172.21.15.44:0/1344877995 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6800cba0 con 0x7fda7010cdd0 2023-09-10T01:13:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.401+0000 7fda783e3700 1 -- 172.21.15.44:0/1344877995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda701a5780 con 0x7fda7010cdd0 2023-09-10T01:13:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.404+0000 7fda677fe700 1 -- 172.21.15.44:0/1344877995 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fda68003a00 con 0x7fda7010cdd0 2023-09-10T01:13:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.404+0000 7fda845a9b80 1 Processor -- start 2023-09-10T01:13:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.404+0000 7fda845a9b80 1 -- start start 2023-09-10T01:13:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.404+0000 7fda845a9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56219328a1a0 0x56219328a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.404+0000 7fda845a9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x562193284780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.404+0000 7fda845a9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284cc0 0x562193290f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.404+0000 7fda845a9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621931dd6d0 con 0x562193284300 2023-09-10T01:13:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.404+0000 7fda845a9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621931daff0 con 0x562193284cc0 2023-09-10T01:13:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.404+0000 7fda845a9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621931d68e0 con 0x56219328a1a0 2023-09-10T01:13:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.405+0000 7fda7617f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x562193284780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.405+0000 7fda7617f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x562193284780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57164/0 (socket says 172.21.15.44:57164) 2023-09-10T01:13:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.405+0000 7fda7617f700 1 -- 172.21.15.44:0/2832987986 learned_addr learned my addr 172.21.15.44:0/2832987986 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.405+0000 7fda7597e700 1 --2- 172.21.15.44:0/2832987986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56219328a1a0 0x56219328a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.405+0000 7fda677fe700 1 --2- 172.21.15.44:0/1344877995 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda5c0a07b0 0x7fda5c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.405+0000 7fda7597e700 1 --2- 172.21.15.44:0/2832987986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284cc0 0x562193290f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.405+0000 7fda76980700 1 --2- 172.21.15.44:0/1344877995 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda5c0a07b0 0x7fda5c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.405+0000 7fda7617f700 1 -- 172.21.15.44:0/2832987986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56219328a1a0 msgr2=0x56219328a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.405+0000 7fda7617f700 1 --2- 172.21.15.44:0/2832987986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56219328a1a0 0x56219328a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.405+0000 7fda7617f700 1 -- 172.21.15.44:0/2832987986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284cc0 msgr2=0x562193290f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.405+0000 7fda7617f700 1 --2- 172.21.15.44:0/2832987986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284cc0 0x562193290f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.405+0000 7fda7617f700 1 -- 172.21.15.44:0/2832987986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562193002d80 con 0x562193284300 2023-09-10T01:13:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.405+0000 7fda677fe700 1 -- 172.21.15.44:0/1344877995 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fda680cbe40 con 0x7fda7010cdd0 2023-09-10T01:13:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.405+0000 7fda7617f700 1 --2- 172.21.15.44:0/2832987986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x562193284780 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fda6c005c70 tx=0x7fda6c004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00881061c7e0f14 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.405+0000 7fda64d89700 1 -- 172.21.15.44:0/2832987986 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c01d050 con 0x562193284300 2023-09-10T01:13:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.405+0000 7fda76980700 1 --2- 172.21.15.44:0/1344877995 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda5c0a07b0 0x7fda5c0a2c70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fda680c9790 tx=0x7fda6808e910 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.406+0000 7fda64d89700 1 -- 172.21.15.44:0/2832987986 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda6c01e040 con 0x562193284300 2023-09-10T01:13:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.406+0000 7fda64d89700 1 -- 172.21.15.44:0/2832987986 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c004c90 con 0x562193284300 2023-09-10T01:13:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.407+0000 7fda845a9b80 1 -- 172.21.15.44:0/2832987986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 msgr2=0x562193284780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.407+0000 7fda845a9b80 1 --2- 172.21.15.44:0/2832987986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x562193284780 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fda6c005c70 tx=0x7fda6c004500 comp rx=0 tx=0).stop 2023-09-10T01:13:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.408+0000 7fda845a9b80 1 -- 172.21.15.44:0/2832987986 shutdown_connections 2023-09-10T01:13:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.408+0000 7fda845a9b80 1 --2- 172.21.15.44:0/2832987986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284cc0 0x562193290f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.408+0000 7fda845a9b80 1 --2- 172.21.15.44:0/2832987986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56219328a1a0 0x56219328a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.408+0000 7fda845a9b80 1 --2- 172.21.15.44:0/2832987986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x562193284780 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.408+0000 7fda845a9b80 1 -- 172.21.15.44:0/2832987986 >> 172.21.15.44:0/2832987986 conn(0x5621931f9030 msgr2=0x56219327f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:28.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.408+0000 7fda845a9b80 1 -- 172.21.15.44:0/2832987986 shutdown_connections 2023-09-10T01:13:28.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.408+0000 7fda845a9b80 1 -- 172.21.15.44:0/2832987986 wait complete. 2023-09-10T01:13:28.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.408+0000 7fda845a9b80 1 -- start start 2023-09-10T01:13:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.408+0000 7fda845a9b80 1 Processor -- start 2023-09-10T01:13:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.408+0000 7fda845a9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284300 0x562193290cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.408+0000 7fda845a9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562193284cc0 0x56219328e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.408+0000 7fda845a9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56219328a1a0 0x56219328e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.408+0000 7fda845a9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562193174b10 con 0x56219328a1a0 2023-09-10T01:13:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.408+0000 7fda845a9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562193174810 con 0x562193284300 2023-09-10T01:13:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.408+0000 7fda845a9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621930588c0 con 0x562193284cc0 2023-09-10T01:13:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.408+0000 7fda845a9b80 10 client.0 _ll_register_callbacks cb 0x5621930bea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:13:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.409+0000 7fda7597e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284300 0x562193290cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.409+0000 7fda7617f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562193284cc0 0x56219328e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.409+0000 7fda7597e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56219328a1a0 0x56219328e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.409+0000 7fda7617f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562193284cc0 0x56219328e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35820/0 (socket says 172.21.15.44:35820) 2023-09-10T01:13:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.409+0000 7fda7617f700 1 -- 172.21.15.44:0/961065001 learned_addr learned my addr 172.21.15.44:0/961065001 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.409+0000 7fda7597e700 1 -- 172.21.15.44:0/961065001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562193284cc0 msgr2=0x56219328e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.409+0000 7fda7597e700 1 --2- 172.21.15.44:0/961065001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562193284cc0 0x56219328e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.409+0000 7fda7597e700 1 -- 172.21.15.44:0/961065001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284300 msgr2=0x562193290cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.409+0000 7fda7597e700 1 --2- 172.21.15.44:0/961065001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284300 0x562193290cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.409+0000 7fda7597e700 1 -- 172.21.15.44:0/961065001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7010fec0 con 0x56219328a1a0 2023-09-10T01:13:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.409+0000 7fda7617f700 1 --2- 172.21.15.44:0/961065001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562193284cc0 0x56219328e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.409+0000 7fda7597e700 1 --2- 172.21.15.44:0/961065001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56219328a1a0 0x56219328e550 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fda7010d5f0 tx=0x7fda60009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4849d4d0ad7ac35 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.409+0000 7fda59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:13:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.409+0000 7fda59ffb700 1 -- 172.21.15.44:0/961065001 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60016070 con 0x56219328a1a0 2023-09-10T01:13:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.410+0000 7fda59ffb700 1 -- 172.21.15.44:0/961065001 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fda60004260 con 0x56219328a1a0 2023-09-10T01:13:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.410+0000 7fda845a9b80 1 -- 172.21.15.44:0/961065001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56219312e490 con 0x56219328a1a0 2023-09-10T01:13:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.410+0000 7fda59ffb700 1 -- 172.21.15.44:0/961065001 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda60004420 con 0x56219328a1a0 2023-09-10T01:13:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.410+0000 7fda845a9b80 10 client.19311 fetch_fsmap learned FSMap version 120 2023-09-10T01:13:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.410+0000 7fda845a9b80 1 -- 172.21.15.44:0/961065001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x562192febc00 con 0x56219328a1a0 2023-09-10T01:13:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.410+0000 7fda59ffb700 1 -- 172.21.15.44:0/961065001 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56219312e490 con 0x56219328a1a0 2023-09-10T01:13:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.410+0000 7fda59ffb700 1 -- 172.21.15.44:0/961065001 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fda60003680 con 0x56219328a1a0 2023-09-10T01:13:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.410+0000 7fda845a9b80 10 client.19311 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:13:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.410+0000 7fda845a9b80 10 client.19311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:13:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.410+0000 7fda845a9b80 20 client.19311 populate_metadata read hostname 'smithi044' 2023-09-10T01:13:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.410+0000 7fda845a9b80 1 --2- 172.21.15.44:0/961065001 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56219329e3d0 0x5621932a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.410+0000 7fda845a9b80 4 client.19311 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:13:28.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.410+0000 7fda845a9b80 1 -- 172.21.15.44:0/961065001 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621931eb840 con 0x56219329e3d0 2023-09-10T01:13:28.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.410+0000 7fda7617f700 1 --2- 172.21.15.44:0/961065001 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56219329e3d0 0x5621932a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.411+0000 7fda7617f700 1 --2- 172.21.15.44:0/961065001 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56219329e3d0 0x5621932a07f0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cdc1f877bc82f0 server_cookie=bc7bba9bcc44939f in_seq=0 out_seq=0 2023-09-10T01:13:28.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.411+0000 7fda59ffb700 10 client.19311 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:28.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.411+0000 7fda59ffb700 1 -- 172.21.15.44:0/961065001 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5621931eb840 con 0x56219329e3d0 2023-09-10T01:13:28.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.411+0000 7fda59ffb700 10 client.19311 handle_command_reply: tid=0 2023-09-10T01:13:28.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.411+0000 7fda845a9b80 1 client.19311 shutdown 2023-09-10T01:13:28.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.412+0000 7fda845a9b80 1 -- 172.21.15.44:0/961065001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56219328a1a0 msgr2=0x56219328e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:28.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.412+0000 7fda845a9b80 1 --2- 172.21.15.44:0/961065001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56219328a1a0 0x56219328e550 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fda7010d5f0 tx=0x7fda60009260 comp rx=0 tx=0).stop 2023-09-10T01:13:28.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.412+0000 7fda845a9b80 1 -- 172.21.15.44:0/961065001 shutdown_connections 2023-09-10T01:13:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.412+0000 7fda845a9b80 1 --2- 172.21.15.44:0/961065001 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56219329e3d0 0x5621932a07f0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.412+0000 7fda845a9b80 1 --2- 172.21.15.44:0/961065001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56219328a1a0 0x56219328e550 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.412+0000 7fda845a9b80 1 --2- 172.21.15.44:0/961065001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284300 0x562193290cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.412+0000 7fda59ffb700 0 client.19311 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.412+0000 7fda845a9b80 1 --2- 172.21.15.44:0/961065001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562193284cc0 0x56219328e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.412+0000 7fda845a9b80 1 -- 172.21.15.44:0/961065001 >> 172.21.15.44:0/961065001 conn(0x5621931f9030 msgr2=0x56219327e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.412+0000 7fda845a9b80 1 -- 172.21.15.44:0/961065001 shutdown_connections 2023-09-10T01:13:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.412+0000 7fda845a9b80 1 -- 172.21.15.44:0/961065001 wait complete. 2023-09-10T01:13:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.412+0000 7fda845a9b80 20 client.19311 trim_cache size 0 max 16384 2023-09-10T01:13:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.427+0000 7fda845a9b80 1 Processor -- start 2023-09-10T01:13:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.427+0000 7fda845a9b80 1 -- start start 2023-09-10T01:13:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.427+0000 7fda845a9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x562193295530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda845a9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562193284cc0 0x562193283a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda845a9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56219328a1a0 0x562193290820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda845a9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56219327bfc0 con 0x562193284300 2023-09-10T01:13:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda845a9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621932912d0 con 0x56219328a1a0 2023-09-10T01:13:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda845a9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56219329ac90 con 0x562193284cc0 2023-09-10T01:13:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda7597e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x562193295530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda7597e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x562193295530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57176/0 (socket says 172.21.15.44:57176) 2023-09-10T01:13:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda7597e700 1 -- 172.21.15.44:0/4145775245 learned_addr learned my addr 172.21.15.44:0/4145775245 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda7617f700 1 --2- 172.21.15.44:0/4145775245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562193284cc0 0x562193283a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda7597e700 1 --2- 172.21.15.44:0/4145775245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56219328a1a0 0x562193290820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda7597e700 1 -- 172.21.15.44:0/4145775245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562193284cc0 msgr2=0x562193283a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda7597e700 1 --2- 172.21.15.44:0/4145775245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562193284cc0 0x562193283a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda7597e700 1 -- 172.21.15.44:0/4145775245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56219328a1a0 msgr2=0x562193290820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda7597e700 1 --2- 172.21.15.44:0/4145775245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56219328a1a0 0x562193290820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda7597e700 1 -- 172.21.15.44:0/4145775245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621930291f0 con 0x562193284300 2023-09-10T01:13:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda7597e700 1 --2- 172.21.15.44:0/4145775245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x562193295530 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fda600115e0 tx=0x7fda60003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210c4404d45a9bff server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda6558a700 1 -- 172.21.15.44:0/4145775245 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60016070 con 0x562193284300 2023-09-10T01:13:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda6558a700 1 -- 172.21.15.44:0/4145775245 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda60003880 con 0x562193284300 2023-09-10T01:13:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.428+0000 7fda6558a700 1 -- 172.21.15.44:0/4145775245 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6000fb00 con 0x562193284300 2023-09-10T01:13:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.429+0000 7fda845a9b80 1 -- 172.21.15.44:0/4145775245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 msgr2=0x562193295530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.429+0000 7fda845a9b80 1 --2- 172.21.15.44:0/4145775245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x562193295530 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fda600115e0 tx=0x7fda60003d80 comp rx=0 tx=0).stop 2023-09-10T01:13:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.429+0000 7fda845a9b80 1 -- 172.21.15.44:0/4145775245 shutdown_connections 2023-09-10T01:13:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.429+0000 7fda845a9b80 1 --2- 172.21.15.44:0/4145775245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56219328a1a0 0x562193290820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.429+0000 7fda845a9b80 1 --2- 172.21.15.44:0/4145775245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x562193295530 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.429+0000 7fda845a9b80 1 --2- 172.21.15.44:0/4145775245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562193284cc0 0x562193283a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.429+0000 7fda845a9b80 1 -- 172.21.15.44:0/4145775245 >> 172.21.15.44:0/4145775245 conn(0x5621931f9030 msgr2=0x5621932832f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:28.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.429+0000 7fda845a9b80 1 -- 172.21.15.44:0/4145775245 shutdown_connections 2023-09-10T01:13:28.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.429+0000 7fda845a9b80 1 -- 172.21.15.44:0/4145775245 wait complete. 2023-09-10T01:13:28.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.430+0000 7fda845a9b80 1 -- start start 2023-09-10T01:13:28.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.430+0000 7fda845a9b80 1 Processor -- start 2023-09-10T01:13:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.430+0000 7fda845a9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x5621932929e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.430+0000 7fda845a9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284cc0 0x562193292f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.430+0000 7fda845a9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56219328a1a0 0x562193293460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.430+0000 7fda845a9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621932f11c0 con 0x562193284300 2023-09-10T01:13:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.430+0000 7fda845a9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621932f1300 con 0x562193284cc0 2023-09-10T01:13:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.430+0000 7fda845a9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621932f1470 con 0x56219328a1a0 2023-09-10T01:13:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.430+0000 7fda845a9b80 10 client.0 _ll_register_callbacks cb 0x5621930bea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:13:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.430+0000 7fda7597e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x5621932929e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.430+0000 7fda7617f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284cc0 0x562193292f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda7597e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x5621932929e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57182/0 (socket says 172.21.15.44:57182) 2023-09-10T01:13:28.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda7597e700 1 -- 172.21.15.44:0/1410322662 learned_addr learned my addr 172.21.15.44:0/1410322662 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:28.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda7617f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284cc0 0x562193292f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58924/0 (socket says 172.21.15.44:58924) 2023-09-10T01:13:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda7597e700 1 --2- 172.21.15.44:0/1410322662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56219328a1a0 0x562193293460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda7597e700 1 -- 172.21.15.44:0/1410322662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56219328a1a0 msgr2=0x562193293460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda7597e700 1 --2- 172.21.15.44:0/1410322662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56219328a1a0 0x562193293460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda7597e700 1 -- 172.21.15.44:0/1410322662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284cc0 msgr2=0x562193292f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda7597e700 1 --2- 172.21.15.44:0/1410322662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284cc0 0x562193292f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda7597e700 1 -- 172.21.15.44:0/1410322662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562192febc00 con 0x562193284300 2023-09-10T01:13:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda7617f700 1 --2- 172.21.15.44:0/1410322662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284cc0 0x562193292f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda7597e700 1 --2- 172.21.15.44:0/1410322662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x5621932929e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fda60009ce0 tx=0x7fda60011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275e572027db3aee server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:13:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda59ffb700 1 -- 172.21.15.44:0/1410322662 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60016070 con 0x562193284300 2023-09-10T01:13:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda59ffb700 1 -- 172.21.15.44:0/1410322662 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fda60000e40 con 0x562193284300 2023-09-10T01:13:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda845a9b80 1 -- 172.21.15.44:0/1410322662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda6c01e040 con 0x562193284300 2023-09-10T01:13:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda59ffb700 1 -- 172.21.15.44:0/1410322662 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda60005720 con 0x562193284300 2023-09-10T01:13:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.431+0000 7fda845a9b80 10 client.19317 fetch_fsmap learned FSMap version 120 2023-09-10T01:13:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.432+0000 7fda845a9b80 1 -- 172.21.15.44:0/1410322662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x562193001250 con 0x562193284300 2023-09-10T01:13:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.432+0000 7fda59ffb700 1 -- 172.21.15.44:0/1410322662 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda6c01e040 con 0x562193284300 2023-09-10T01:13:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.432+0000 7fda59ffb700 1 -- 172.21.15.44:0/1410322662 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fda6002a070 con 0x562193284300 2023-09-10T01:13:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.432+0000 7fda845a9b80 10 client.19317 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:13:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.432+0000 7fda845a9b80 10 client.19317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:13:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.432+0000 7fda845a9b80 20 client.19317 populate_metadata read hostname 'smithi044' 2023-09-10T01:13:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.432+0000 7fda845a9b80 1 --2- 172.21.15.44:0/1410322662 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5621932f2890 0x5621932f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.432+0000 7fda845a9b80 4 client.19317 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:13:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.432+0000 7fda845a9b80 1 -- 172.21.15.44:0/1410322662 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5621931cf8c0 con 0x5621932f2890 2023-09-10T01:13:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.432+0000 7fda7617f700 1 --2- 172.21.15.44:0/1410322662 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5621932f2890 0x5621932f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.433+0000 7fda7617f700 1 --2- 172.21.15.44:0/1410322662 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5621932f2890 0x5621932f4cb0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4c2c03aa8aceb23 server_cookie=5d0ff50090fdbb59 in_seq=0 out_seq=0 2023-09-10T01:13:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.433+0000 7fda59ffb700 10 client.19317 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.443+0000 7fda59ffb700 1 -- 172.21.15.44:0/1410322662 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5621931cf8c0 con 0x5621932f2890 2023-09-10T01:13:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.443+0000 7fda59ffb700 10 client.19317 handle_command_reply: tid=0 2023-09-10T01:13:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.443+0000 7fda845a9b80 1 client.19317 shutdown 2023-09-10T01:13:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.443+0000 7fda845a9b80 1 -- 172.21.15.44:0/1410322662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 msgr2=0x5621932929e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.443+0000 7fda845a9b80 1 --2- 172.21.15.44:0/1410322662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x5621932929e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fda60009ce0 tx=0x7fda60011440 comp rx=0 tx=0).stop 2023-09-10T01:13:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.443+0000 7fda845a9b80 1 -- 172.21.15.44:0/1410322662 shutdown_connections 2023-09-10T01:13:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.443+0000 7fda845a9b80 1 --2- 172.21.15.44:0/1410322662 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5621932f2890 0x5621932f4cb0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.443+0000 7fda845a9b80 1 --2- 172.21.15.44:0/1410322662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56219328a1a0 0x562193293460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.443+0000 7fda59ffb700 0 client.19317 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.443+0000 7fda845a9b80 1 --2- 172.21.15.44:0/1410322662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562193284300 0x5621932929e0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.443+0000 7fda845a9b80 1 --2- 172.21.15.44:0/1410322662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562193284cc0 0x562193292f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.443+0000 7fda845a9b80 1 -- 172.21.15.44:0/1410322662 >> 172.21.15.44:0/1410322662 conn(0x5621931f9030 msgr2=0x562193283b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.443+0000 7fda845a9b80 1 -- 172.21.15.44:0/1410322662 shutdown_connections 2023-09-10T01:13:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.443+0000 7fda845a9b80 1 -- 172.21.15.44:0/1410322662 wait complete. 2023-09-10T01:13:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.444+0000 7fda845a9b80 20 client.19317 trim_cache size 0 max 16384 2023-09-10T01:13:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:13:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:13:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:13:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:13:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.444+0000 7fda783e3700 1 -- 172.21.15.44:0/1344877995 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda5c0a07b0 msgr2=0x7fda5c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.444+0000 7fda783e3700 1 --2- 172.21.15.44:0/1344877995 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda5c0a07b0 0x7fda5c0a2c70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fda680c9790 tx=0x7fda6808e910 comp rx=0 tx=0).stop 2023-09-10T01:13:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.444+0000 7fda783e3700 1 -- 172.21.15.44:0/1344877995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda7010cdd0 msgr2=0x7fda701985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.444+0000 7fda783e3700 1 --2- 172.21.15.44:0/1344877995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda7010cdd0 0x7fda701985f0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fda6800efc0 tx=0x7fda6800c3a0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.444+0000 7fda783e3700 1 -- 172.21.15.44:0/1344877995 shutdown_connections 2023-09-10T01:13:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.444+0000 7fda783e3700 1 --2- 172.21.15.44:0/1344877995 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda5c0a07b0 0x7fda5c0a2c70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.444+0000 7fda783e3700 1 --2- 172.21.15.44:0/1344877995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda7010cdd0 0x7fda701985f0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.444+0000 7fda783e3700 1 --2- 172.21.15.44:0/1344877995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda70067f30 0x7fda7019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.444+0000 7fda783e3700 1 --2- 172.21.15.44:0/1344877995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda70101470 0x7fda7019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.444+0000 7fda783e3700 1 -- 172.21.15.44:0/1344877995 >> 172.21.15.44:0/1344877995 conn(0x7fda70074f30 msgr2=0x7fda700fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.444+0000 7fda783e3700 1 -- 172.21.15.44:0/1344877995 shutdown_connections 2023-09-10T01:13:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:28.444+0000 7fda783e3700 1 -- 172.21.15.44:0/1344877995 wait complete. 2023-09-10T01:13:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:29 smithi131 ceph-mon[103713]: pgmap v1047: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 149 op/s 2023-09-10T01:13:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:29 smithi044 ceph-mon[98676]: pgmap v1047: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 149 op/s 2023-09-10T01:13:29.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:29 smithi099 ceph-mon[102692]: pgmap v1047: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 149 op/s 2023-09-10T01:13:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:31 smithi131 ceph-mon[103713]: pgmap v1048: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 149 op/s 2023-09-10T01:13:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:31 smithi044 ceph-mon[98676]: pgmap v1048: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 149 op/s 2023-09-10T01:13:31.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:31 smithi099 ceph-mon[102692]: pgmap v1048: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 149 op/s 2023-09-10T01:13:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:33 smithi131 ceph-mon[103713]: pgmap v1049: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 230 op/s 2023-09-10T01:13:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:33 smithi044 ceph-mon[98676]: pgmap v1049: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 230 op/s 2023-09-10T01:13:33.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:33 smithi099 ceph-mon[102692]: pgmap v1049: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 230 op/s 2023-09-10T01:13:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:35 smithi131 ceph-mon[103713]: pgmap v1050: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2023-09-10T01:13:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:35 smithi044 ceph-mon[98676]: pgmap v1050: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2023-09-10T01:13:35.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:35 smithi099 ceph-mon[102692]: pgmap v1050: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2023-09-10T01:13:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:37 smithi131 ceph-mon[103713]: pgmap v1051: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 212 op/s 2023-09-10T01:13:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:37 smithi044 ceph-mon[98676]: pgmap v1051: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 212 op/s 2023-09-10T01:13:37.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:37 smithi099 ceph-mon[102692]: pgmap v1051: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 212 op/s 2023-09-10T01:13:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:39 smithi131 ceph-mon[103713]: pgmap v1052: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 146 op/s 2023-09-10T01:13:39.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:39 smithi044 ceph-mon[98676]: pgmap v1052: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 146 op/s 2023-09-10T01:13:39.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:39 smithi099 ceph-mon[102692]: pgmap v1052: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 146 op/s 2023-09-10T01:13:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:41 smithi131 ceph-mon[103713]: pgmap v1053: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 146 op/s 2023-09-10T01:13:41.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:41 smithi044 ceph-mon[98676]: pgmap v1053: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 146 op/s 2023-09-10T01:13:41.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:41 smithi099 ceph-mon[102692]: pgmap v1053: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 146 op/s 2023-09-10T01:13:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.561+0000 7fa88c035700 1 -- 172.21.15.44:0/2809506519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8840ff690 msgr2=0x7fa8840ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.561+0000 7fa88c035700 1 --2- 172.21.15.44:0/2809506519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8840ff690 0x7fa8840ffb10 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa8780099c0 tx=0x7fa878009cd0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.561+0000 7fa88c035700 1 -- 172.21.15.44:0/2809506519 shutdown_connections 2023-09-10T01:13:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.561+0000 7fa88c035700 1 --2- 172.21.15.44:0/2809506519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa884100050 0x7fa88410c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.561+0000 7fa88c035700 1 --2- 172.21.15.44:0/2809506519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8841055b0 0x7fa884105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.561+0000 7fa88c035700 1 --2- 172.21.15.44:0/2809506519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8840ff690 0x7fa8840ffb10 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.561+0000 7fa88c035700 1 -- 172.21.15.44:0/2809506519 >> 172.21.15.44:0/2809506519 conn(0x7fa884077430 msgr2=0x7fa884077840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:41.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.561+0000 7fa88c035700 1 -- 172.21.15.44:0/2809506519 shutdown_connections 2023-09-10T01:13:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.561+0000 7fa88c035700 1 -- 172.21.15.44:0/2809506519 wait complete. 2023-09-10T01:13:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa88c035700 1 Processor -- start 2023-09-10T01:13:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa88c035700 1 -- start start 2023-09-10T01:13:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa88c035700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8840ff690 0x7fa88419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa88c035700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa884100050 0x7fa88419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa88c035700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8841055b0 0x7fa884198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa88c035700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa884198c80 con 0x7fa8840ff690 2023-09-10T01:13:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa88c035700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa884198df0 con 0x7fa8841055b0 2023-09-10T01:13:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa88c035700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa884198fb0 con 0x7fa884100050 2023-09-10T01:13:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa889dd1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8840ff690 0x7fa88419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa889dd1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8840ff690 0x7fa88419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57196/0 (socket says 172.21.15.44:57196) 2023-09-10T01:13:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa889dd1700 1 -- 172.21.15.44:0/925087373 learned_addr learned my addr 172.21.15.44:0/925087373 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa88a5d2700 1 --2- 172.21.15.44:0/925087373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8841055b0 0x7fa884198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa8895d0700 1 --2- 172.21.15.44:0/925087373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa884100050 0x7fa88419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa889dd1700 1 -- 172.21.15.44:0/925087373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa884100050 msgr2=0x7fa88419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa889dd1700 1 --2- 172.21.15.44:0/925087373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa884100050 0x7fa88419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa889dd1700 1 -- 172.21.15.44:0/925087373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8841055b0 msgr2=0x7fa884198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa889dd1700 1 --2- 172.21.15.44:0/925087373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8841055b0 0x7fa884198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa889dd1700 1 -- 172.21.15.44:0/925087373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8780095f0 con 0x7fa8840ff690 2023-09-10T01:13:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.562+0000 7fa889dd1700 1 --2- 172.21.15.44:0/925087373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8840ff690 0x7fa88419e7b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00ef10 tx=0x7fa87c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.563+0000 7fa882ffd700 1 -- 172.21.15.44:0/925087373 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c00c9c0 con 0x7fa8840ff690 2023-09-10T01:13:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.563+0000 7fa882ffd700 1 -- 172.21.15.44:0/925087373 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa87c00f040 con 0x7fa8840ff690 2023-09-10T01:13:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.563+0000 7fa88c035700 1 -- 172.21.15.44:0/925087373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa884199230 con 0x7fa8840ff690 2023-09-10T01:13:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.563+0000 7fa882ffd700 1 -- 172.21.15.44:0/925087373 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c014480 con 0x7fa8840ff690 2023-09-10T01:13:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.563+0000 7fa88c035700 1 -- 172.21.15.44:0/925087373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8841a57a0 con 0x7fa8840ff690 2023-09-10T01:13:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.566+0000 7fa882ffd700 1 -- 172.21.15.44:0/925087373 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa87c007690 con 0x7fa8840ff690 2023-09-10T01:13:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.566+0000 7fa8981fbb80 1 Processor -- start 2023-09-10T01:13:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa8981fbb80 1 -- start start 2023-09-10T01:13:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa882ffd700 1 --2- 172.21.15.44:0/925087373 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa86c0a0830 0x7fa86c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa8981fbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec81a0 0x557c3fec8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa88a5d2700 1 --2- 172.21.15.44:0/925087373 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa86c0a0830 0x7fa86c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa8981fbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fec2300 0x557c3fec2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa882ffd700 1 -- 172.21.15.44:0/925087373 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa87c0cc5b0 con 0x7fa8840ff690 2023-09-10T01:13:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa8981fbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2cc0 0x557c3fecef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa8981fbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3fe1b6d0 con 0x557c3fec2cc0 2023-09-10T01:13:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa8981fbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3fe18ff0 con 0x557c3fec81a0 2023-09-10T01:13:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa8981fbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3fe148e0 con 0x557c3fec2300 2023-09-10T01:13:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa88a5d2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2cc0 0x557c3fecef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa88a5d2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2cc0 0x557c3fecef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57204/0 (socket says 172.21.15.44:57204) 2023-09-10T01:13:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa88a5d2700 1 -- 172.21.15.44:0/3675984434 learned_addr learned my addr 172.21.15.44:0/3675984434 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa8895d0700 1 --2- 172.21.15.44:0/3675984434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fec2300 0x557c3fec2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa889dd1700 1 --2- 172.21.15.44:0/3675984434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec81a0 0x557c3fec8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa88a5d2700 1 -- 172.21.15.44:0/3675984434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fec2300 msgr2=0x557c3fec2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa88a5d2700 1 --2- 172.21.15.44:0/3675984434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fec2300 0x557c3fec2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa88a5d2700 1 -- 172.21.15.44:0/3675984434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec81a0 msgr2=0x557c3fec8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa88a5d2700 1 --2- 172.21.15.44:0/3675984434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec81a0 0x557c3fec8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.567+0000 7fa88a5d2700 1 -- 172.21.15.44:0/3675984434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c3fc40d80 con 0x557c3fec2cc0 2023-09-10T01:13:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.568+0000 7fa88a5d2700 1 --2- 172.21.15.44:0/3675984434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2cc0 0x557c3fecef30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fa874013830 tx=0x7fa874013b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89958d034c9dcd9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.568+0000 7fa86bfff700 1 -- 172.21.15.44:0/3675984434 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8740132e0 con 0x557c3fec2cc0 2023-09-10T01:13:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.568+0000 7fa86bfff700 1 -- 172.21.15.44:0/3675984434 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa87400f970 con 0x557c3fec2cc0 2023-09-10T01:13:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.568+0000 7fa86bfff700 1 -- 172.21.15.44:0/3675984434 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874010960 con 0x557c3fec2cc0 2023-09-10T01:13:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.569+0000 7fa8981fbb80 1 -- 172.21.15.44:0/3675984434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2cc0 msgr2=0x557c3fecef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.569+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/3675984434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2cc0 0x557c3fecef30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fa874013830 tx=0x7fa874013b40 comp rx=0 tx=0).stop 2023-09-10T01:13:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.569+0000 7fa8981fbb80 1 -- 172.21.15.44:0/3675984434 shutdown_connections 2023-09-10T01:13:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.569+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/3675984434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2cc0 0x557c3fecef30 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.569+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/3675984434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec81a0 0x557c3fec8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.569+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/3675984434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fec2300 0x557c3fec2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.569+0000 7fa8981fbb80 1 -- 172.21.15.44:0/3675984434 >> 172.21.15.44:0/3675984434 conn(0x557c3fe37030 msgr2=0x557c3febd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.569+0000 7fa8981fbb80 1 -- 172.21.15.44:0/3675984434 shutdown_connections 2023-09-10T01:13:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.569+0000 7fa8981fbb80 1 -- 172.21.15.44:0/3675984434 wait complete. 2023-09-10T01:13:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.569+0000 7fa8981fbb80 1 -- start start 2023-09-10T01:13:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.569+0000 7fa8981fbb80 1 Processor -- start 2023-09-10T01:13:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8981fbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec2300 0x557c3feced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8981fbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2cc0 0x557c3fecc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8981fbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fec81a0 0x557c3fecc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8981fbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3fdb2b10 con 0x557c3fec2cc0 2023-09-10T01:13:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8981fbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3fdb2810 con 0x557c3fec2300 2023-09-10T01:13:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8981fbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3fc968c0 con 0x557c3fec81a0 2023-09-10T01:13:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8981fbb80 10 client.0 _ll_register_callbacks cb 0x557c3fcfca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:13:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8895d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2cc0 0x557c3fecc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8895d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2cc0 0x557c3fecc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57212/0 (socket says 172.21.15.44:57212) 2023-09-10T01:13:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8895d0700 1 -- 172.21.15.44:0/649435612 learned_addr learned my addr 172.21.15.44:0/649435612 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa88a5d2700 1 --2- 172.21.15.44:0/649435612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fec81a0 0x557c3fecc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa889dd1700 1 --2- 172.21.15.44:0/649435612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec2300 0x557c3feced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8895d0700 1 -- 172.21.15.44:0/649435612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fec81a0 msgr2=0x557c3fecc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8895d0700 1 --2- 172.21.15.44:0/649435612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fec81a0 0x557c3fecc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8895d0700 1 -- 172.21.15.44:0/649435612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec2300 msgr2=0x557c3feced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8895d0700 1 --2- 172.21.15.44:0/649435612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec2300 0x557c3feced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8895d0700 1 -- 172.21.15.44:0/649435612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa874013460 con 0x557c3fec2cc0 2023-09-10T01:13:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8895d0700 1 --2- 172.21.15.44:0/649435612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2cc0 0x557c3fecc020 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa8780099c0 tx=0x7fa878005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd256f5c14d655a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8697fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:13:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8697fa700 1 -- 172.21.15.44:0/649435612 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8780054d0 con 0x557c3fec2cc0 2023-09-10T01:13:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8697fa700 1 -- 172.21.15.44:0/649435612 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa878005630 con 0x557c3fec2cc0 2023-09-10T01:13:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.570+0000 7fa8981fbb80 1 -- 172.21.15.44:0/649435612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c3fd6c490 con 0x557c3fec2cc0 2023-09-10T01:13:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.571+0000 7fa8697fa700 1 -- 172.21.15.44:0/649435612 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa878003760 con 0x557c3fec2cc0 2023-09-10T01:13:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.571+0000 7fa8981fbb80 10 client.19329 fetch_fsmap learned FSMap version 120 2023-09-10T01:13:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.571+0000 7fa8981fbb80 1 -- 172.21.15.44:0/649435612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x557c3fc29c00 con 0x557c3fec2cc0 2023-09-10T01:13:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.571+0000 7fa8697fa700 1 -- 172.21.15.44:0/649435612 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557c3fd6c490 con 0x557c3fec2cc0 2023-09-10T01:13:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.571+0000 7fa8697fa700 1 -- 172.21.15.44:0/649435612 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fa87800f850 con 0x557c3fec2cc0 2023-09-10T01:13:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.571+0000 7fa8981fbb80 10 client.19329 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:13:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.571+0000 7fa8981fbb80 10 client.19329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:13:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.571+0000 7fa8981fbb80 20 client.19329 populate_metadata read hostname 'smithi044' 2023-09-10T01:13:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.571+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/649435612 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557c3fedc340 0x557c3fede760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.571+0000 7fa8981fbb80 4 client.19329 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:13:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.571+0000 7fa8981fbb80 1 -- 172.21.15.44:0/649435612 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c3fe29840 con 0x557c3fedc340 2023-09-10T01:13:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.572+0000 7fa889dd1700 1 --2- 172.21.15.44:0/649435612 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557c3fedc340 0x557c3fede760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.572+0000 7fa88a5d2700 1 --2- 172.21.15.44:0/925087373 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa86c0a0830 0x7fa86c0a2cf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa87400ffd0 tx=0x7fa874009c90 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.572+0000 7fa8697fa700 10 client.19329 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.572+0000 7fa889dd1700 1 --2- 172.21.15.44:0/649435612 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557c3fedc340 0x557c3fede760 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3109d34bb3f76bec server_cookie=269b35c815638eb6 in_seq=0 out_seq=0 2023-09-10T01:13:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.573+0000 7fa8697fa700 1 -- 172.21.15.44:0/649435612 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557c3fe29840 con 0x557c3fedc340 2023-09-10T01:13:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.573+0000 7fa8697fa700 10 client.19329 handle_command_reply: tid=0 2023-09-10T01:13:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.573+0000 7fa8981fbb80 1 client.19329 shutdown 2023-09-10T01:13:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.573+0000 7fa8981fbb80 1 -- 172.21.15.44:0/649435612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2cc0 msgr2=0x557c3fecc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.573+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/649435612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2cc0 0x557c3fecc020 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa8780099c0 tx=0x7fa878005940 comp rx=0 tx=0).stop 2023-09-10T01:13:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.573+0000 7fa8981fbb80 1 -- 172.21.15.44:0/649435612 shutdown_connections 2023-09-10T01:13:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.573+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/649435612 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557c3fedc340 0x557c3fede760 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.573+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/649435612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fec81a0 0x557c3fecc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.573+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/649435612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec2300 0x557c3feced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.573+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/649435612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2cc0 0x557c3fecc020 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.573+0000 7fa8697fa700 0 client.19329 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.573+0000 7fa8981fbb80 1 -- 172.21.15.44:0/649435612 >> 172.21.15.44:0/649435612 conn(0x557c3fe37030 msgr2=0x557c3febc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.573+0000 7fa8981fbb80 1 -- 172.21.15.44:0/649435612 shutdown_connections 2023-09-10T01:13:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.573+0000 7fa8981fbb80 1 -- 172.21.15.44:0/649435612 wait complete. 2023-09-10T01:13:41.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.573+0000 7fa8981fbb80 20 client.19329 trim_cache size 0 max 16384 2023-09-10T01:13:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa8981fbb80 1 Processor -- start 2023-09-10T01:13:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa8981fbb80 1 -- start start 2023-09-10T01:13:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa8981fbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fe37030 0x557c3fec79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa8981fbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fec2300 0x557c3fec1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa8981fbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec2cc0 0x557c3fece830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa8981fbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3fec3580 con 0x557c3fe37030 2023-09-10T01:13:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa8981fbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3feccb40 con 0x557c3fec2cc0 2023-09-10T01:13:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa8981fbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3feb9fc0 con 0x557c3fec2300 2023-09-10T01:13:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa889dd1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fe37030 0x557c3fec79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa889dd1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fe37030 0x557c3fec79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57218/0 (socket says 172.21.15.44:57218) 2023-09-10T01:13:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa889dd1700 1 -- 172.21.15.44:0/727949595 learned_addr learned my addr 172.21.15.44:0/727949595 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa8895d0700 1 --2- 172.21.15.44:0/727949595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fec2300 0x557c3fec1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa88a5d2700 1 --2- 172.21.15.44:0/727949595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec2cc0 0x557c3fece830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa889dd1700 1 -- 172.21.15.44:0/727949595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fec2300 msgr2=0x557c3fec1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa889dd1700 1 --2- 172.21.15.44:0/727949595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fec2300 0x557c3fec1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa889dd1700 1 -- 172.21.15.44:0/727949595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec2cc0 msgr2=0x557c3fece830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa889dd1700 1 --2- 172.21.15.44:0/727949595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec2cc0 0x557c3fece830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.589+0000 7fa889dd1700 1 -- 172.21.15.44:0/727949595 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c3fc671f0 con 0x557c3fe37030 2023-09-10T01:13:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.590+0000 7fa889dd1700 1 --2- 172.21.15.44:0/727949595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fe37030 0x557c3fec79d0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa87c096070 tx=0x7fa87c016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23ff435a07a011 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.590+0000 7fa880ff9700 1 -- 172.21.15.44:0/727949595 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c099600 con 0x557c3fe37030 2023-09-10T01:13:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.590+0000 7fa880ff9700 1 -- 172.21.15.44:0/727949595 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa87c09d430 con 0x557c3fe37030 2023-09-10T01:13:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.590+0000 7fa880ff9700 1 -- 172.21.15.44:0/727949595 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c0840b0 con 0x557c3fe37030 2023-09-10T01:13:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.591+0000 7fa8981fbb80 1 -- 172.21.15.44:0/727949595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fe37030 msgr2=0x557c3fec79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.591+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/727949595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fe37030 0x557c3fec79d0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa87c096070 tx=0x7fa87c016040 comp rx=0 tx=0).stop 2023-09-10T01:13:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.591+0000 7fa8981fbb80 1 -- 172.21.15.44:0/727949595 shutdown_connections 2023-09-10T01:13:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.591+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/727949595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec2cc0 0x557c3fece830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.591+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/727949595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fe37030 0x557c3fec79d0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.591+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/727949595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fec2300 0x557c3fec1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.591+0000 7fa8981fbb80 1 -- 172.21.15.44:0/727949595 >> 172.21.15.44:0/727949595 conn(0x557c3fec6910 msgr2=0x557c3fec12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.591+0000 7fa8981fbb80 1 -- 172.21.15.44:0/727949595 shutdown_connections 2023-09-10T01:13:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.591+0000 7fa8981fbb80 1 -- 172.21.15.44:0/727949595 wait complete. 2023-09-10T01:13:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.591+0000 7fa8981fbb80 1 -- start start 2023-09-10T01:13:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.591+0000 7fa8981fbb80 1 Processor -- start 2023-09-10T01:13:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8981fbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fe37030 0x557c3fed3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8981fbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2300 0x557c3fed1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8981fbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec2cc0 0x557c3fed1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8981fbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3fed24b0 con 0x557c3fec2300 2023-09-10T01:13:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8981fbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3fed2620 con 0x557c3fec2cc0 2023-09-10T01:13:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8981fbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3fed27e0 con 0x557c3fe37030 2023-09-10T01:13:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8981fbb80 10 client.0 _ll_register_callbacks cb 0x7fa87800baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:13:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8895d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2300 0x557c3fed1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa889dd1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fe37030 0x557c3fed3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8895d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2300 0x557c3fed1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57226/0 (socket says 172.21.15.44:57226) 2023-09-10T01:13:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8895d0700 1 -- 172.21.15.44:0/1585029802 learned_addr learned my addr 172.21.15.44:0/1585029802 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa88a5d2700 1 --2- 172.21.15.44:0/1585029802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec2cc0 0x557c3fed1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8895d0700 1 -- 172.21.15.44:0/1585029802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fe37030 msgr2=0x557c3fed3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8895d0700 1 --2- 172.21.15.44:0/1585029802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fe37030 0x557c3fed3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8895d0700 1 -- 172.21.15.44:0/1585029802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec2cc0 msgr2=0x557c3fed1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8895d0700 1 --2- 172.21.15.44:0/1585029802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec2cc0 0x557c3fed1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8895d0700 1 -- 172.21.15.44:0/1585029802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8841a57a0 con 0x557c3fec2300 2023-09-10T01:13:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8895d0700 1 --2- 172.21.15.44:0/1585029802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2300 0x557c3fed1940 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa878006010 tx=0x7fa878023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db417ebe64cd8ff5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8697fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:13:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.592+0000 7fa8697fa700 1 -- 172.21.15.44:0/1585029802 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8780056e0 con 0x557c3fec2300 2023-09-10T01:13:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.593+0000 7fa8697fa700 1 -- 172.21.15.44:0/1585029802 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa87800f460 con 0x557c3fec2300 2023-09-10T01:13:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.593+0000 7fa8981fbb80 1 -- 172.21.15.44:0/1585029802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa87400f970 con 0x557c3fec2300 2023-09-10T01:13:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.593+0000 7fa8697fa700 1 -- 172.21.15.44:0/1585029802 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa87800f640 con 0x557c3fec2300 2023-09-10T01:13:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.593+0000 7fa8981fbb80 10 client.19335 fetch_fsmap learned FSMap version 120 2023-09-10T01:13:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.593+0000 7fa8981fbb80 1 -- 172.21.15.44:0/1585029802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x557c3fc3f250 con 0x557c3fec2300 2023-09-10T01:13:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.593+0000 7fa8697fa700 1 -- 172.21.15.44:0/1585029802 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa87400f970 con 0x557c3fec2300 2023-09-10T01:13:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.593+0000 7fa8697fa700 1 -- 172.21.15.44:0/1585029802 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fa87800b260 con 0x557c3fec2300 2023-09-10T01:13:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.593+0000 7fa8981fbb80 10 client.19335 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:13:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.593+0000 7fa8981fbb80 10 client.19335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:13:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.593+0000 7fa8981fbb80 20 client.19335 populate_metadata read hostname 'smithi044' 2023-09-10T01:13:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.593+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/1585029802 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557c3fec81a0 0x557c3ff33340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.593+0000 7fa8981fbb80 4 client.19335 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:13:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.593+0000 7fa8981fbb80 1 -- 172.21.15.44:0/1585029802 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c3fe0d8c0 con 0x557c3fec81a0 2023-09-10T01:13:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.593+0000 7fa889dd1700 1 --2- 172.21.15.44:0/1585029802 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557c3fec81a0 0x557c3ff33340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.594+0000 7fa889dd1700 1 --2- 172.21.15.44:0/1585029802 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557c3fec81a0 0x557c3ff33340 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=414fbff9b04b7436 server_cookie=718e71bdb7629600 in_seq=0 out_seq=0 2023-09-10T01:13:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.594+0000 7fa8697fa700 10 client.19335 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.611+0000 7fa8697fa700 1 -- 172.21.15.44:0/1585029802 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c3fe0d8c0 con 0x557c3fec81a0 2023-09-10T01:13:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.611+0000 7fa8697fa700 10 client.19335 handle_command_reply: tid=0 2023-09-10T01:13:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.611+0000 7fa8981fbb80 1 client.19335 shutdown 2023-09-10T01:13:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.611+0000 7fa8981fbb80 1 -- 172.21.15.44:0/1585029802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2300 msgr2=0x557c3fed1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.611+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/1585029802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2300 0x557c3fed1940 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa878006010 tx=0x7fa878023000 comp rx=0 tx=0).stop 2023-09-10T01:13:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.611+0000 7fa8981fbb80 1 -- 172.21.15.44:0/1585029802 shutdown_connections 2023-09-10T01:13:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.611+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/1585029802 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557c3fec81a0 0x557c3ff33340 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.611+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/1585029802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c3fec2cc0 0x557c3fed1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.611+0000 7fa8697fa700 0 client.19335 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.611+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/1585029802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c3fe37030 0x557c3fed3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.611+0000 7fa8981fbb80 1 --2- 172.21.15.44:0/1585029802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557c3fec2300 0x557c3fed1940 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.611+0000 7fa8981fbb80 1 -- 172.21.15.44:0/1585029802 >> 172.21.15.44:0/1585029802 conn(0x557c3fec6910 msgr2=0x557c3fec1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.612+0000 7fa8981fbb80 1 -- 172.21.15.44:0/1585029802 shutdown_connections 2023-09-10T01:13:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.612+0000 7fa8981fbb80 1 -- 172.21.15.44:0/1585029802 wait complete. 2023-09-10T01:13:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.612+0000 7fa8981fbb80 20 client.19335 trim_cache size 0 max 16384 2023-09-10T01:13:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:13:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:13:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:13:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:13:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.612+0000 7fa88c035700 1 -- 172.21.15.44:0/925087373 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa86c0a0830 msgr2=0x7fa86c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.612+0000 7fa88c035700 1 --2- 172.21.15.44:0/925087373 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa86c0a0830 0x7fa86c0a2cf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa87400ffd0 tx=0x7fa874009c90 comp rx=0 tx=0).stop 2023-09-10T01:13:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.612+0000 7fa88c035700 1 -- 172.21.15.44:0/925087373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8840ff690 msgr2=0x7fa88419e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.612+0000 7fa88c035700 1 --2- 172.21.15.44:0/925087373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8840ff690 0x7fa88419e7b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00ef10 tx=0x7fa87c00c3a0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.612+0000 7fa88c035700 1 -- 172.21.15.44:0/925087373 shutdown_connections 2023-09-10T01:13:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.612+0000 7fa88c035700 1 --2- 172.21.15.44:0/925087373 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa86c0a0830 0x7fa86c0a2cf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.612+0000 7fa88c035700 1 --2- 172.21.15.44:0/925087373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8841055b0 0x7fa884198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.612+0000 7fa88c035700 1 --2- 172.21.15.44:0/925087373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8840ff690 0x7fa88419e7b0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.612+0000 7fa88c035700 1 --2- 172.21.15.44:0/925087373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa884100050 0x7fa88419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.612+0000 7fa88c035700 1 -- 172.21.15.44:0/925087373 >> 172.21.15.44:0/925087373 conn(0x7fa884077430 msgr2=0x7fa8840fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.612+0000 7fa88c035700 1 -- 172.21.15.44:0/925087373 shutdown_connections 2023-09-10T01:13:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:41.612+0000 7fa88c035700 1 -- 172.21.15.44:0/925087373 wait complete. 2023-09-10T01:13:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:43 smithi131 ceph-mon[103713]: pgmap v1054: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 232 op/s 2023-09-10T01:13:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:43 smithi044 ceph-mon[98676]: pgmap v1054: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 232 op/s 2023-09-10T01:13:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:43 smithi099 ceph-mon[102692]: pgmap v1054: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 232 op/s 2023-09-10T01:13:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:45 smithi131 ceph-mon[103713]: pgmap v1055: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-09-10T01:13:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:45 smithi044 ceph-mon[98676]: pgmap v1055: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-09-10T01:13:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:45 smithi099 ceph-mon[102692]: pgmap v1055: 129 pgs: 129 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-09-10T01:13:47.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:47 smithi131 ceph-mon[103713]: pgmap v1056: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 212 op/s 2023-09-10T01:13:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:47 smithi044 ceph-mon[98676]: pgmap v1056: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 212 op/s 2023-09-10T01:13:47.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:47 smithi099 ceph-mon[102692]: pgmap v1056: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 212 op/s 2023-09-10T01:13:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:49 smithi131 ceph-mon[103713]: pgmap v1057: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2023-09-10T01:13:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:49 smithi044 ceph-mon[98676]: pgmap v1057: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2023-09-10T01:13:49.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:49 smithi099 ceph-mon[102692]: pgmap v1057: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2023-09-10T01:13:50.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:50 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:13:50.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:50 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:13:50.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:50 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:13:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:51 smithi131 ceph-mon[103713]: pgmap v1058: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2023-09-10T01:13:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:51 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:51 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:51 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:51 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:51 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:51 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:51 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:13:51.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:51 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:13:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:51 smithi044 ceph-mon[98676]: pgmap v1058: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2023-09-10T01:13:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:51 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:51 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:51 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:51 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:51 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:51 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:51 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:13:51.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:51 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:13:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:51 smithi099 ceph-mon[102692]: pgmap v1058: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2023-09-10T01:13:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:51 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:51 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:51 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:51 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:51 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:51 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:51.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:51 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:13:51.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:51 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:13:52.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:52 smithi044 ceph-mon[98676]: pgmap v1059: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 226 op/s 2023-09-10T01:13:52.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:52 smithi044 ceph-mon[98676]: pgmap v1060: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2023-09-10T01:13:52.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:52 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:52.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:52 smithi099 ceph-mon[102692]: pgmap v1059: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 226 op/s 2023-09-10T01:13:52.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:52 smithi099 ceph-mon[102692]: pgmap v1060: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2023-09-10T01:13:52.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:52 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:52.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:52 smithi131 ceph-mon[103713]: pgmap v1059: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 226 op/s 2023-09-10T01:13:52.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:52 smithi131 ceph-mon[103713]: pgmap v1060: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2023-09-10T01:13:52.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:52 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:13:54.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:54 smithi044 ceph-mon[98676]: pgmap v1061: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2023-09-10T01:13:54.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:54 smithi099 ceph-mon[102692]: pgmap v1061: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2023-09-10T01:13:54.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:54 smithi131 ceph-mon[103713]: pgmap v1061: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2023-09-10T01:13:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.745+0000 7f720b0b0700 1 -- 172.21.15.44:0/4164067360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040688c0 msgr2=0x7f7204068d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.745+0000 7f720b0b0700 1 --2- 172.21.15.44:0/4164067360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040688c0 0x7f7204068d40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f71f00099c0 tx=0x7f71f0009cd0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.745+0000 7f720b0b0700 1 -- 172.21.15.44:0/4164067360 shutdown_connections 2023-09-10T01:13:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.745+0000 7f720b0b0700 1 --2- 172.21.15.44:0/4164067360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7204069280 0x7f720410d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.745+0000 7f720b0b0700 1 --2- 172.21.15.44:0/4164067360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7204106710 0x7f7204106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.745+0000 7f720b0b0700 1 --2- 172.21.15.44:0/4164067360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040688c0 0x7f7204068d40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.745+0000 7f720b0b0700 1 -- 172.21.15.44:0/4164067360 >> 172.21.15.44:0/4164067360 conn(0x7f72040773b0 msgr2=0x7f72040777c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.745+0000 7f720b0b0700 1 -- 172.21.15.44:0/4164067360 shutdown_connections 2023-09-10T01:13:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.745+0000 7f720b0b0700 1 -- 172.21.15.44:0/4164067360 wait complete. 2023-09-10T01:13:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.746+0000 7f720b0b0700 1 Processor -- start 2023-09-10T01:13:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.746+0000 7f720b0b0700 1 -- start start 2023-09-10T01:13:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f720b0b0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72040688c0 0x7f720419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f720b0b0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7204069280 0x7f720419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f720b0b0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7204106710 0x7f7204196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f720b0b0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7204196b70 con 0x7f7204106710 2023-09-10T01:13:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f720b0b0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7204196ce0 con 0x7f7204069280 2023-09-10T01:13:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f720b0b0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7204196ea0 con 0x7f72040688c0 2023-09-10T01:13:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f7208e4c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72040688c0 0x7f720419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f720964d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7204106710 0x7f7204196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f7208e4c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72040688c0 0x7f720419c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35888/0 (socket says 172.21.15.44:35888) 2023-09-10T01:13:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f7208e4c700 1 -- 172.21.15.44:0/3995482626 learned_addr learned my addr 172.21.15.44:0/3995482626 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f720964d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7204106710 0x7f7204196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57240/0 (socket says 172.21.15.44:57240) 2023-09-10T01:13:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f7203fff700 1 --2- 172.21.15.44:0/3995482626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7204069280 0x7f720419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f720964d700 1 -- 172.21.15.44:0/3995482626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72040688c0 msgr2=0x7f720419c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f720964d700 1 --2- 172.21.15.44:0/3995482626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72040688c0 0x7f720419c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f720964d700 1 -- 172.21.15.44:0/3995482626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7204069280 msgr2=0x7f720419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f720964d700 1 --2- 172.21.15.44:0/3995482626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7204069280 0x7f720419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f720964d700 1 -- 172.21.15.44:0/3995482626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f00095f0 con 0x7f7204106710 2023-09-10T01:13:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f7203fff700 1 --2- 172.21.15.44:0/3995482626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7204069280 0x7f720419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.747+0000 7f720964d700 1 --2- 172.21.15.44:0/3995482626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7204106710 0x7f7204196500 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00efc0 tx=0x7f71fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.748+0000 7f7201ffb700 1 -- 172.21.15.44:0/3995482626 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc018070 con 0x7f7204106710 2023-09-10T01:13:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.748+0000 7f720b0b0700 1 -- 172.21.15.44:0/3995482626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7204197120 con 0x7f7204106710 2023-09-10T01:13:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.748+0000 7f7201ffb700 1 -- 172.21.15.44:0/3995482626 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71fc00f040 con 0x7f7204106710 2023-09-10T01:13:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.748+0000 7f7201ffb700 1 -- 172.21.15.44:0/3995482626 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc00cba0 con 0x7f7204106710 2023-09-10T01:13:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.748+0000 7f720b0b0700 1 -- 172.21.15.44:0/3995482626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72040ff8e0 con 0x7f7204106710 2023-09-10T01:13:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.750+0000 7f7201ffb700 1 -- 172.21.15.44:0/3995482626 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f71fc003a00 con 0x7f7204106710 2023-09-10T01:13:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.751+0000 7f7201ffb700 1 --2- 172.21.15.44:0/3995482626 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f71f40a07b0 0x7f71f40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.751+0000 7f7201ffb700 1 -- 172.21.15.44:0/3995482626 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f71fc0cbe80 con 0x7f7204106710 2023-09-10T01:13:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.751+0000 7f7208e4c700 1 --2- 172.21.15.44:0/3995482626 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f71f40a07b0 0x7f71f40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.752+0000 7f7208e4c700 1 --2- 172.21.15.44:0/3995482626 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f71f40a07b0 0x7f71f40a2c70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009ca0 tx=0x7f71f8009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.752+0000 7f7217276b80 1 Processor -- start 2023-09-10T01:13:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f7217276b80 1 -- start start 2023-09-10T01:13:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f7217276b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be4131a0 0x5623be413580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f7217276b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be40d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f7217276b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be40dcc0 0x5623be419f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f7217276b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623be3666d0 con 0x5623be40d300 2023-09-10T01:13:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f7217276b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623be363ff0 con 0x5623be40dcc0 2023-09-10T01:13:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f7217276b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623be35f8e0 con 0x5623be4131a0 2023-09-10T01:13:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f7203fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be40d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f7208e4c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be4131a0 0x5623be413580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f7203fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be40d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57248/0 (socket says 172.21.15.44:57248) 2023-09-10T01:13:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f7203fff700 1 -- 172.21.15.44:0/3075887806 learned_addr learned my addr 172.21.15.44:0/3075887806 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f720964d700 1 --2- 172.21.15.44:0/3075887806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be40dcc0 0x5623be419f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f7208e4c700 1 -- 172.21.15.44:0/3075887806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be40dcc0 msgr2=0x5623be419f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f7208e4c700 1 --2- 172.21.15.44:0/3075887806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be40dcc0 0x5623be419f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f7208e4c700 1 -- 172.21.15.44:0/3075887806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 msgr2=0x5623be40d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f7208e4c700 1 --2- 172.21.15.44:0/3075887806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be40d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.753+0000 7f7208e4c700 1 -- 172.21.15.44:0/3075887806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623be18bd80 con 0x5623be4131a0 2023-09-10T01:13:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.754+0000 7f7208e4c700 1 --2- 172.21.15.44:0/3075887806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be4131a0 0x5623be413580 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f71f8011ab0 tx=0x7f71f8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f56d2465cb32d44 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.754+0000 7f71eeffd700 1 -- 172.21.15.44:0/3075887806 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f800eb70 con 0x5623be4131a0 2023-09-10T01:13:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.754+0000 7f71eeffd700 1 -- 172.21.15.44:0/3075887806 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71f8021070 con 0x5623be4131a0 2023-09-10T01:13:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.754+0000 7f71eeffd700 1 -- 172.21.15.44:0/3075887806 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f801c630 con 0x5623be4131a0 2023-09-10T01:13:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.755+0000 7f7217276b80 1 -- 172.21.15.44:0/3075887806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be4131a0 msgr2=0x5623be413580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.755+0000 7f7217276b80 1 --2- 172.21.15.44:0/3075887806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be4131a0 0x5623be413580 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f71f8011ab0 tx=0x7f71f8014000 comp rx=0 tx=0).stop 2023-09-10T01:13:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.755+0000 7f7217276b80 1 -- 172.21.15.44:0/3075887806 shutdown_connections 2023-09-10T01:13:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.755+0000 7f7217276b80 1 --2- 172.21.15.44:0/3075887806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be40dcc0 0x5623be419f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.755+0000 7f7217276b80 1 --2- 172.21.15.44:0/3075887806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be4131a0 0x5623be413580 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.755+0000 7f7217276b80 1 --2- 172.21.15.44:0/3075887806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be40d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.755+0000 7f7217276b80 1 -- 172.21.15.44:0/3075887806 >> 172.21.15.44:0/3075887806 conn(0x5623be382030 msgr2=0x5623be408560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.755+0000 7f7217276b80 1 -- 172.21.15.44:0/3075887806 shutdown_connections 2023-09-10T01:13:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.755+0000 7f7217276b80 1 -- 172.21.15.44:0/3075887806 wait complete. 2023-09-10T01:13:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.755+0000 7f7217276b80 1 -- start start 2023-09-10T01:13:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.755+0000 7f7217276b80 1 Processor -- start 2023-09-10T01:13:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f7217276b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be419d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f7217276b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be40dcc0 0x5623be4170b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f7217276b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be4131a0 0x5623be417650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f7217276b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623be2fdb10 con 0x5623be40d300 2023-09-10T01:13:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f7217276b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623be2fd810 con 0x5623be40dcc0 2023-09-10T01:13:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f7217276b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623be1e18c0 con 0x5623be4131a0 2023-09-10T01:13:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f7217276b80 10 client.0 _ll_register_callbacks cb 0x5623be247a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:13:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f7208e4c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be419d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f720964d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be4131a0 0x5623be417650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f7208e4c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be419d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57252/0 (socket says 172.21.15.44:57252) 2023-09-10T01:13:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f7208e4c700 1 -- 172.21.15.44:0/332370189 learned_addr learned my addr 172.21.15.44:0/332370189 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f7203fff700 1 --2- 172.21.15.44:0/332370189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be40dcc0 0x5623be4170b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f720964d700 1 -- 172.21.15.44:0/332370189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be40dcc0 msgr2=0x5623be4170b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f720964d700 1 --2- 172.21.15.44:0/332370189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be40dcc0 0x5623be4170b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f720964d700 1 -- 172.21.15.44:0/332370189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 msgr2=0x5623be419d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f720964d700 1 --2- 172.21.15.44:0/332370189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be419d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f720964d700 1 -- 172.21.15.44:0/332370189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f8017070 con 0x5623be4131a0 2023-09-10T01:13:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f7208e4c700 1 --2- 172.21.15.44:0/332370189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be419d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:13:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f720964d700 1 --2- 172.21.15.44:0/332370189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be4131a0 0x5623be417650 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0c91d0 tx=0x7f71fc01b320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cfa3bea4d01727 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.756+0000 7f71e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:13:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.757+0000 7f71e3fff700 1 -- 172.21.15.44:0/332370189 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc09cdf0 con 0x5623be4131a0 2023-09-10T01:13:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.757+0000 7f71e3fff700 1 -- 172.21.15.44:0/332370189 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f71fc0a0fa0 con 0x5623be4131a0 2023-09-10T01:13:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.757+0000 7f7217276b80 1 -- 172.21.15.44:0/332370189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623be2b7490 con 0x5623be4131a0 2023-09-10T01:13:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.757+0000 7f71e3fff700 1 -- 172.21.15.44:0/332370189 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71fc099220 con 0x5623be4131a0 2023-09-10T01:13:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.757+0000 7f7217276b80 10 client.28957 fetch_fsmap learned FSMap version 120 2023-09-10T01:13:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.757+0000 7f7217276b80 1 -- 172.21.15.44:0/332370189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x5623be174c00 con 0x5623be4131a0 2023-09-10T01:13:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.757+0000 7f71e3fff700 1 -- 172.21.15.44:0/332370189 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5623be2b7490 con 0x5623be4131a0 2023-09-10T01:13:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.757+0000 7f71e3fff700 1 -- 172.21.15.44:0/332370189 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f71fc093bd0 con 0x5623be4131a0 2023-09-10T01:13:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.757+0000 7f7217276b80 10 client.28957 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:13:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.757+0000 7f7217276b80 10 client.28957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:13:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.757+0000 7f7217276b80 20 client.28957 populate_metadata read hostname 'smithi044' 2023-09-10T01:13:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.757+0000 7f7217276b80 1 --2- 172.21.15.44:0/332370189 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623be4272a0 0x5623be429690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.757+0000 7f7217276b80 4 client.28957 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:13:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.757+0000 7f7217276b80 1 -- 172.21.15.44:0/332370189 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623be374840 con 0x5623be4272a0 2023-09-10T01:13:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.758+0000 7f7208e4c700 1 --2- 172.21.15.44:0/332370189 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623be4272a0 0x5623be429690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.758+0000 7f71e3fff700 10 client.28957 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.758+0000 7f7208e4c700 1 --2- 172.21.15.44:0/332370189 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623be4272a0 0x5623be429690 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8375e93c567171bb server_cookie=5c39340c6e2cd64f in_seq=0 out_seq=0 2023-09-10T01:13:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.758+0000 7f71e3fff700 1 -- 172.21.15.44:0/332370189 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5623be374840 con 0x5623be4272a0 2023-09-10T01:13:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.758+0000 7f71e3fff700 10 client.28957 handle_command_reply: tid=0 2023-09-10T01:13:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.758+0000 7f7217276b80 1 client.28957 shutdown 2023-09-10T01:13:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.759+0000 7f7217276b80 1 -- 172.21.15.44:0/332370189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be4131a0 msgr2=0x5623be417650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.759+0000 7f7217276b80 1 --2- 172.21.15.44:0/332370189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be4131a0 0x5623be417650 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0c91d0 tx=0x7f71fc01b320 comp rx=0 tx=0).stop 2023-09-10T01:13:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.759+0000 7f7217276b80 1 -- 172.21.15.44:0/332370189 shutdown_connections 2023-09-10T01:13:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.759+0000 7f7217276b80 1 --2- 172.21.15.44:0/332370189 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623be4272a0 0x5623be429690 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.759+0000 7f7217276b80 1 --2- 172.21.15.44:0/332370189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be4131a0 0x5623be417650 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.759+0000 7f71e3fff700 0 client.28957 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.759+0000 7f7217276b80 1 --2- 172.21.15.44:0/332370189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be419d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.759+0000 7f7217276b80 1 --2- 172.21.15.44:0/332370189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be40dcc0 0x5623be4170b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.759+0000 7f7217276b80 1 -- 172.21.15.44:0/332370189 >> 172.21.15.44:0/332370189 conn(0x5623be382030 msgr2=0x5623be4077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.759+0000 7f7217276b80 1 -- 172.21.15.44:0/332370189 shutdown_connections 2023-09-10T01:13:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.759+0000 7f7217276b80 1 -- 172.21.15.44:0/332370189 wait complete. 2023-09-10T01:13:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.759+0000 7f7217276b80 20 client.28957 trim_cache size 0 max 16384 2023-09-10T01:13:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.776+0000 7f7217276b80 1 Processor -- start 2023-09-10T01:13:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.776+0000 7f7217276b80 1 -- start start 2023-09-10T01:13:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f7217276b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be41e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f7217276b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be40dcc0 0x5623be4198c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f7217276b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be4131a0 0x5623be406f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f7217276b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623be374840 con 0x5623be40d300 2023-09-10T01:13:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f7217276b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623be2b7490 con 0x5623be4131a0 2023-09-10T01:13:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f7217276b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623be410db0 con 0x5623be40dcc0 2023-09-10T01:13:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f7208e4c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be41e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f7208e4c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be41e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57260/0 (socket says 172.21.15.44:57260) 2023-09-10T01:13:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f7208e4c700 1 -- 172.21.15.44:0/1985348430 learned_addr learned my addr 172.21.15.44:0/1985348430 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f7203fff700 1 --2- 172.21.15.44:0/1985348430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be40dcc0 0x5623be4198c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f720964d700 1 --2- 172.21.15.44:0/1985348430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be4131a0 0x5623be406f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f7208e4c700 1 -- 172.21.15.44:0/1985348430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be40dcc0 msgr2=0x5623be4198c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f7208e4c700 1 --2- 172.21.15.44:0/1985348430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be40dcc0 0x5623be4198c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f7208e4c700 1 -- 172.21.15.44:0/1985348430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be4131a0 msgr2=0x5623be406f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f7208e4c700 1 --2- 172.21.15.44:0/1985348430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be4131a0 0x5623be406f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f7208e4c700 1 -- 172.21.15.44:0/1985348430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623be1b21f0 con 0x5623be40d300 2023-09-10T01:13:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.777+0000 7f7208e4c700 1 --2- 172.21.15.44:0/1985348430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be41e530 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f71f8027d50 tx=0x7f71f8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83c9c69a591ecc1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.778+0000 7f71ef7fe700 1 -- 172.21.15.44:0/1985348430 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f802c040 con 0x5623be40d300 2023-09-10T01:13:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.778+0000 7f71ef7fe700 1 -- 172.21.15.44:0/1985348430 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71f800ee80 con 0x5623be40d300 2023-09-10T01:13:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.778+0000 7f71ef7fe700 1 -- 172.21.15.44:0/1985348430 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8031c80 con 0x5623be40d300 2023-09-10T01:13:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.778+0000 7f7217276b80 1 -- 172.21.15.44:0/1985348430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 msgr2=0x5623be41e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.778+0000 7f7217276b80 1 --2- 172.21.15.44:0/1985348430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be41e530 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f71f8027d50 tx=0x7f71f8011b60 comp rx=0 tx=0).stop 2023-09-10T01:13:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.778+0000 7f7217276b80 1 -- 172.21.15.44:0/1985348430 shutdown_connections 2023-09-10T01:13:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.778+0000 7f7217276b80 1 --2- 172.21.15.44:0/1985348430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be4131a0 0x5623be406f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.778+0000 7f7217276b80 1 --2- 172.21.15.44:0/1985348430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be41e530 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.778+0000 7f7217276b80 1 --2- 172.21.15.44:0/1985348430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be40dcc0 0x5623be4198c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.778+0000 7f7217276b80 1 -- 172.21.15.44:0/1985348430 >> 172.21.15.44:0/1985348430 conn(0x5623be382030 msgr2=0x5623be408090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.779+0000 7f7217276b80 1 -- 172.21.15.44:0/1985348430 shutdown_connections 2023-09-10T01:13:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.779+0000 7f7217276b80 1 -- 172.21.15.44:0/1985348430 wait complete. 2023-09-10T01:13:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.779+0000 7f7217276b80 1 -- start start 2023-09-10T01:13:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.779+0000 7f7217276b80 1 Processor -- start 2023-09-10T01:13:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.779+0000 7f7217276b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be46f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.779+0000 7f7217276b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be40dcc0 0x5623be46f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.779+0000 7f7217276b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be4131a0 0x5623be4730f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.779+0000 7f7217276b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623be4736a0 con 0x5623be40d300 2023-09-10T01:13:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.779+0000 7f7217276b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623be473810 con 0x5623be4131a0 2023-09-10T01:13:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.779+0000 7f7217276b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623be4739d0 con 0x5623be40dcc0 2023-09-10T01:13:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.779+0000 7f7217276b80 10 client.0 _ll_register_callbacks cb 0x7f71fc098920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:13:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.780+0000 7f7208e4c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be46f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.780+0000 7f7203fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be40dcc0 0x5623be46f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.780+0000 7f7203fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be40dcc0 0x5623be46f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35918/0 (socket says 172.21.15.44:35918) 2023-09-10T01:13:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.780+0000 7f7203fff700 1 -- 172.21.15.44:0/637124433 learned_addr learned my addr 172.21.15.44:0/637124433 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:13:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.780+0000 7f720964d700 1 --2- 172.21.15.44:0/637124433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be4131a0 0x5623be4730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.780+0000 7f7208e4c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be46f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57266/0 (socket says 172.21.15.44:57266) 2023-09-10T01:13:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.780+0000 7f7203fff700 1 -- 172.21.15.44:0/637124433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be4131a0 msgr2=0x5623be4730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.780+0000 7f7203fff700 1 --2- 172.21.15.44:0/637124433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be4131a0 0x5623be4730f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.780+0000 7f7203fff700 1 -- 172.21.15.44:0/637124433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 msgr2=0x5623be46f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:13:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.780+0000 7f7203fff700 1 --2- 172.21.15.44:0/637124433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be46f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.780+0000 7f7203fff700 1 -- 172.21.15.44:0/637124433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623be18bd80 con 0x5623be40dcc0 2023-09-10T01:13:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.780+0000 7f7203fff700 1 --2- 172.21.15.44:0/637124433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be40dcc0 0x5623be46f700 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009fd0 tx=0x7f71f0005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0f0027b36a90f5c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:13:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.780+0000 7f71e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:13:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.780+0000 7f71e3fff700 1 -- 172.21.15.44:0/637124433 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0003b70 con 0x5623be40dcc0 2023-09-10T01:13:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.780+0000 7f71e3fff700 1 -- 172.21.15.44:0/637124433 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f71f0003cd0 con 0x5623be40dcc0 2023-09-10T01:13:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.780+0000 7f7217276b80 1 -- 172.21.15.44:0/637124433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623be3588c0 con 0x5623be40dcc0 2023-09-10T01:13:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.781+0000 7f71e3fff700 1 -- 172.21.15.44:0/637124433 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71f0003e90 con 0x5623be40dcc0 2023-09-10T01:13:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.781+0000 7f7217276b80 10 client.28963 fetch_fsmap learned FSMap version 120 2023-09-10T01:13:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.781+0000 7f7217276b80 1 -- 172.21.15.44:0/637124433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x5623be18a250 con 0x5623be40dcc0 2023-09-10T01:13:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.781+0000 7f71e3fff700 1 -- 172.21.15.44:0/637124433 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5623be3588c0 con 0x5623be40dcc0 2023-09-10T01:13:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.781+0000 7f71e3fff700 1 -- 172.21.15.44:0/637124433 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f71f00099f0 con 0x5623be40dcc0 2023-09-10T01:13:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.781+0000 7f7217276b80 10 client.28963 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:13:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.781+0000 7f7217276b80 10 client.28963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:13:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.781+0000 7f7217276b80 20 client.28963 populate_metadata read hostname 'smithi044' 2023-09-10T01:13:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.781+0000 7f7217276b80 1 --2- 172.21.15.44:0/637124433 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623be47c260 0x5623be47e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:13:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.781+0000 7f7217276b80 4 client.28963 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:13:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.781+0000 7f7217276b80 1 -- 172.21.15.44:0/637124433 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5623be319050 con 0x5623be47c260 2023-09-10T01:13:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.781+0000 7f7208e4c700 1 --2- 172.21.15.44:0/637124433 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623be47c260 0x5623be47e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:13:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.782+0000 7f71e3fff700 10 client.28963 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.782+0000 7f7208e4c700 1 --2- 172.21.15.44:0/637124433 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623be47c260 0x5623be47e680 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95f6b9251a11d910 server_cookie=7b2b829ede6ce58c in_seq=0 out_seq=0 2023-09-10T01:13:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.786+0000 7f71e3fff700 1 -- 172.21.15.44:0/637124433 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5623be319050 con 0x5623be47c260 2023-09-10T01:13:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.786+0000 7f71e3fff700 10 client.28963 handle_command_reply: tid=0 2023-09-10T01:13:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.786+0000 7f7217276b80 1 client.28963 shutdown 2023-09-10T01:13:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.787+0000 7f7217276b80 1 -- 172.21.15.44:0/637124433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be40dcc0 msgr2=0x5623be46f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.787+0000 7f7217276b80 1 --2- 172.21.15.44:0/637124433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be40dcc0 0x5623be46f700 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009fd0 tx=0x7f71f0005800 comp rx=0 tx=0).stop 2023-09-10T01:13:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.787+0000 7f7217276b80 1 -- 172.21.15.44:0/637124433 shutdown_connections 2023-09-10T01:13:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.787+0000 7f7217276b80 1 --2- 172.21.15.44:0/637124433 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623be47c260 0x5623be47e680 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.787+0000 7f7217276b80 1 --2- 172.21.15.44:0/637124433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623be4131a0 0x5623be4730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.787+0000 7f7217276b80 1 --2- 172.21.15.44:0/637124433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623be40d300 0x5623be46f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.787+0000 7f71e3fff700 0 client.28963 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:13:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.787+0000 7f7217276b80 1 --2- 172.21.15.44:0/637124433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623be40dcc0 0x5623be46f700 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.787+0000 7f7217276b80 1 -- 172.21.15.44:0/637124433 >> 172.21.15.44:0/637124433 conn(0x5623be382030 msgr2=0x5623be40c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.787+0000 7f7217276b80 1 -- 172.21.15.44:0/637124433 shutdown_connections 2023-09-10T01:13:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.787+0000 7f7217276b80 1 -- 172.21.15.44:0/637124433 wait complete. 2023-09-10T01:13:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.787+0000 7f7217276b80 20 client.28963 trim_cache size 0 max 16384 2023-09-10T01:13:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:13:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:13:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:13:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:13:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.787+0000 7f720b0b0700 1 -- 172.21.15.44:0/3995482626 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f71f40a07b0 msgr2=0x7f71f40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.787+0000 7f720b0b0700 1 --2- 172.21.15.44:0/3995482626 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f71f40a07b0 0x7f71f40a2c70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009ca0 tx=0x7f71f8009260 comp rx=0 tx=0).stop 2023-09-10T01:13:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.787+0000 7f720b0b0700 1 -- 172.21.15.44:0/3995482626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7204106710 msgr2=0x7f7204196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:13:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.787+0000 7f720b0b0700 1 --2- 172.21.15.44:0/3995482626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7204106710 0x7f7204196500 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00efc0 tx=0x7f71fc00c3a0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.788+0000 7f720b0b0700 1 -- 172.21.15.44:0/3995482626 shutdown_connections 2023-09-10T01:13:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.788+0000 7f720b0b0700 1 --2- 172.21.15.44:0/3995482626 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f71f40a07b0 0x7f71f40a2c70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.788+0000 7f720b0b0700 1 --2- 172.21.15.44:0/3995482626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7204106710 0x7f7204196500 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.788+0000 7f720b0b0700 1 --2- 172.21.15.44:0/3995482626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72040688c0 0x7f720419c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.788+0000 7f720b0b0700 1 --2- 172.21.15.44:0/3995482626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7204069280 0x7f720419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:13:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.788+0000 7f720b0b0700 1 -- 172.21.15.44:0/3995482626 >> 172.21.15.44:0/3995482626 conn(0x7f72040773b0 msgr2=0x7f72040fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:13:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.788+0000 7f720b0b0700 1 -- 172.21.15.44:0/3995482626 shutdown_connections 2023-09-10T01:13:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:13:54.788+0000 7f720b0b0700 1 -- 172.21.15.44:0/3995482626 wait complete. 2023-09-10T01:13:56.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:56 smithi044 ceph-mon[98676]: pgmap v1062: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2023-09-10T01:13:56.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:56 smithi099 ceph-mon[102692]: pgmap v1062: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2023-09-10T01:13:56.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:56 smithi131 ceph-mon[103713]: pgmap v1062: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2023-09-10T01:13:58.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:13:58 smithi044 ceph-mon[98676]: pgmap v1063: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 172 op/s 2023-09-10T01:13:58.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:13:58 smithi099 ceph-mon[102692]: pgmap v1063: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 172 op/s 2023-09-10T01:13:58.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:13:58 smithi131 ceph-mon[103713]: pgmap v1063: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 172 op/s 2023-09-10T01:14:00.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:00 smithi044 ceph-mon[98676]: pgmap v1064: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 172 op/s 2023-09-10T01:14:00.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:00 smithi099 ceph-mon[102692]: pgmap v1064: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 172 op/s 2023-09-10T01:14:00.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:00 smithi131 ceph-mon[103713]: pgmap v1064: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 172 op/s 2023-09-10T01:14:02.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:01 smithi131 ceph-mon[103713]: pgmap v1065: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2023-09-10T01:14:02.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:01 smithi044 ceph-mon[98676]: pgmap v1065: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2023-09-10T01:14:02.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:01 smithi099 ceph-mon[102692]: pgmap v1065: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2023-09-10T01:14:03.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:03 smithi131 ceph-mon[103713]: pgmap v1066: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 958 KiB/s wr, 135 op/s 2023-09-10T01:14:03.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:03 smithi044 ceph-mon[98676]: pgmap v1066: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 958 KiB/s wr, 135 op/s 2023-09-10T01:14:03.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:03 smithi099 ceph-mon[102692]: pgmap v1066: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 958 KiB/s wr, 135 op/s 2023-09-10T01:14:05.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:05 smithi131 ceph-mon[103713]: pgmap v1067: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 211 op/s 2023-09-10T01:14:05.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:05 smithi044 ceph-mon[98676]: pgmap v1067: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 211 op/s 2023-09-10T01:14:05.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:05 smithi099 ceph-mon[102692]: pgmap v1067: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 211 op/s 2023-09-10T01:14:07.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:07 smithi131 ceph-mon[103713]: pgmap v1068: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 997 KiB/s wr, 152 op/s 2023-09-10T01:14:07.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:07 smithi044 ceph-mon[98676]: pgmap v1068: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 997 KiB/s wr, 152 op/s 2023-09-10T01:14:07.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:07 smithi099 ceph-mon[102692]: pgmap v1068: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 997 KiB/s wr, 152 op/s 2023-09-10T01:14:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.922+0000 7f2da5b54700 1 -- 172.21.15.44:0/225886023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2da00689f0 msgr2=0x7f2da0068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.922+0000 7f2da5b54700 1 --2- 172.21.15.44:0/225886023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2da00689f0 0x7f2da0068e50 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2d9401a9a0 tx=0x7f2d9401acb0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.922+0000 7f2da5b54700 1 -- 172.21.15.44:0/225886023 shutdown_connections 2023-09-10T01:14:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.922+0000 7f2da5b54700 1 --2- 172.21.15.44:0/225886023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da0103020 0x7f2da01034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.922+0000 7f2da5b54700 1 --2- 172.21.15.44:0/225886023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2da0105560 0x7f2da00684b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.922+0000 7f2da5b54700 1 --2- 172.21.15.44:0/225886023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2da00689f0 0x7f2da0068e50 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.922+0000 7f2da5b54700 1 -- 172.21.15.44:0/225886023 >> 172.21.15.44:0/225886023 conn(0x7f2da0077430 msgr2=0x7f2da0077840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.923+0000 7f2da5b54700 1 -- 172.21.15.44:0/225886023 shutdown_connections 2023-09-10T01:14:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.923+0000 7f2da5b54700 1 -- 172.21.15.44:0/225886023 wait complete. 2023-09-10T01:14:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.923+0000 7f2da5b54700 1 Processor -- start 2023-09-10T01:14:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.923+0000 7f2da5b54700 1 -- start start 2023-09-10T01:14:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2da5b54700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da00689f0 0x7f2da010f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2da5b54700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2da0103020 0x7f2da010fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2da5b54700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2da0105560 0x7f2da01a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2da5b54700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da01a0910 con 0x7f2da0103020 2023-09-10T01:14:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2da5b54700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da01a0a50 con 0x7f2da0105560 2023-09-10T01:14:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2da5b54700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da01a0b90 con 0x7f2da00689f0 2023-09-10T01:14:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2d9effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2da0103020 0x7f2da010fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2d9f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da00689f0 0x7f2da010f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2d9effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2da0103020 0x7f2da010fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57282/0 (socket says 172.21.15.44:57282) 2023-09-10T01:14:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2d9f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da00689f0 0x7f2da010f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35930/0 (socket says 172.21.15.44:35930) 2023-09-10T01:14:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2d9effd700 1 -- 172.21.15.44:0/1111876047 learned_addr learned my addr 172.21.15.44:0/1111876047 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2d9ffff700 1 --2- 172.21.15.44:0/1111876047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2da0105560 0x7f2da01a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2d9f7fe700 1 -- 172.21.15.44:0/1111876047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2da0105560 msgr2=0x7f2da01a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2d9f7fe700 1 --2- 172.21.15.44:0/1111876047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2da0105560 0x7f2da01a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2d9f7fe700 1 -- 172.21.15.44:0/1111876047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2da0103020 msgr2=0x7f2da010fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2d9f7fe700 1 --2- 172.21.15.44:0/1111876047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2da0103020 0x7f2da010fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2d9f7fe700 1 -- 172.21.15.44:0/1111876047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d98009c40 con 0x7f2da00689f0 2023-09-10T01:14:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2d9effd700 1 --2- 172.21.15.44:0/1111876047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2da0103020 0x7f2da010fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:14:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2d9ffff700 1 --2- 172.21.15.44:0/1111876047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2da0105560 0x7f2da01a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:14:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.924+0000 7f2d9f7fe700 1 --2- 172.21.15.44:0/1111876047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da00689f0 0x7f2da010f500 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000e9f0 tx=0x7f2d9000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.925+0000 7f2d9cff9700 1 -- 172.21.15.44:0/1111876047 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9000cb70 con 0x7f2da00689f0 2023-09-10T01:14:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.925+0000 7f2da5b54700 1 -- 172.21.15.44:0/1111876047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d90009520 con 0x7f2da00689f0 2023-09-10T01:14:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.925+0000 7f2d9cff9700 1 -- 172.21.15.44:0/1111876047 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d9000ee90 con 0x7f2da00689f0 2023-09-10T01:14:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.925+0000 7f2d9cff9700 1 -- 172.21.15.44:0/1111876047 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90003bc0 con 0x7f2da00689f0 2023-09-10T01:14:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.925+0000 7f2da5b54700 1 -- 172.21.15.44:0/1111876047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9401a600 con 0x7f2da00689f0 2023-09-10T01:14:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.927+0000 7f2d9cff9700 1 -- 172.21.15.44:0/1111876047 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f2d90003d60 con 0x7f2da00689f0 2023-09-10T01:14:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.928+0000 7f2d9cff9700 1 --2- 172.21.15.44:0/1111876047 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2d880a07b0 0x7f2d880a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.928+0000 7f2d9cff9700 1 -- 172.21.15.44:0/1111876047 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2d90014070 con 0x7f2da00689f0 2023-09-10T01:14:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.928+0000 7f2d9effd700 1 --2- 172.21.15.44:0/1111876047 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2d880a07b0 0x7f2d880a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.929+0000 7f2db1d1ab80 1 Processor -- start 2023-09-10T01:14:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.929+0000 7f2db1d1ab80 1 -- start start 2023-09-10T01:14:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.929+0000 7f2db1d1ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559bec1a0 0x562559bec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.929+0000 7f2db1d1ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6300 0x562559be6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.929+0000 7f2db1d1ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6cc0 0x562559bf2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.929+0000 7f2db1d1ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559b3f6d0 con 0x562559bec1a0 2023-09-10T01:14:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.929+0000 7f2db1d1ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559b3cff0 con 0x562559be6300 2023-09-10T01:14:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.929+0000 7f2db1d1ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559b388e0 con 0x562559be6cc0 2023-09-10T01:14:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.929+0000 7f2d9effd700 1 --2- 172.21.15.44:0/1111876047 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2d880a07b0 0x7f2d880a2c70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2da0077db0 tx=0x7f2d9402a040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.929+0000 7f2d9f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559bec1a0 0x562559bec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.929+0000 7f2d9effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6300 0x562559be6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.929+0000 7f2d9ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6cc0 0x562559bf2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.929+0000 7f2d9f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559bec1a0 0x562559bec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57288/0 (socket says 172.21.15.44:57288) 2023-09-10T01:14:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.929+0000 7f2d9ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6cc0 0x562559bf2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35942/0 (socket says 172.21.15.44:35942) 2023-09-10T01:14:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.929+0000 7f2d9f7fe700 1 -- 172.21.15.44:0/2089498179 learned_addr learned my addr 172.21.15.44:0/2089498179 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.930+0000 7f2d9effd700 1 -- 172.21.15.44:0/2089498179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6cc0 msgr2=0x562559bf2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.930+0000 7f2d9effd700 1 --2- 172.21.15.44:0/2089498179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6cc0 0x562559bf2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.930+0000 7f2d9effd700 1 -- 172.21.15.44:0/2089498179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559bec1a0 msgr2=0x562559bec580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.930+0000 7f2d9effd700 1 --2- 172.21.15.44:0/2089498179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559bec1a0 0x562559bec580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.930+0000 7f2d9effd700 1 -- 172.21.15.44:0/2089498179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562559964d80 con 0x562559be6300 2023-09-10T01:14:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.930+0000 7f2d9effd700 1 --2- 172.21.15.44:0/2089498179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6300 0x562559be6780 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2d9401c2f0 tx=0x7f2d94005050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b72b6301a6376da8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.930+0000 7f2d85ffb700 1 -- 172.21.15.44:0/2089498179 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9402d040 con 0x562559be6300 2023-09-10T01:14:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.930+0000 7f2d85ffb700 1 -- 172.21.15.44:0/2089498179 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d94005550 con 0x562559be6300 2023-09-10T01:14:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.930+0000 7f2d85ffb700 1 -- 172.21.15.44:0/2089498179 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9403ed00 con 0x562559be6300 2023-09-10T01:14:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.931+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/2089498179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6300 msgr2=0x562559be6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.931+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/2089498179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6300 0x562559be6780 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2d9401c2f0 tx=0x7f2d94005050 comp rx=0 tx=0).stop 2023-09-10T01:14:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.931+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/2089498179 shutdown_connections 2023-09-10T01:14:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.931+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/2089498179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6cc0 0x562559bf2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.931+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/2089498179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559bec1a0 0x562559bec580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.931+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/2089498179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6300 0x562559be6780 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.931+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/2089498179 >> 172.21.15.44:0/2089498179 conn(0x562559b5b030 msgr2=0x562559be1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.931+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/2089498179 shutdown_connections 2023-09-10T01:14:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.931+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/2089498179 wait complete. 2023-09-10T01:14:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.931+0000 7f2db1d1ab80 1 -- start start 2023-09-10T01:14:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.931+0000 7f2db1d1ab80 1 Processor -- start 2023-09-10T01:14:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2db1d1ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6300 0x562559bf2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2db1d1ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6cc0 0x562559bf0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2db1d1ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559bec1a0 0x562559bf0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2db1d1ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559ad6b10 con 0x562559bec1a0 2023-09-10T01:14:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2db1d1ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559ad6810 con 0x562559be6cc0 2023-09-10T01:14:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2db1d1ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625599ba8c0 con 0x562559be6300 2023-09-10T01:14:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2db1d1ab80 10 client.0 _ll_register_callbacks cb 0x562559a20a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:14:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2d9f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6300 0x562559bf2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2d9f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6300 0x562559bf2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35944/0 (socket says 172.21.15.44:35944) 2023-09-10T01:14:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2d9f7fe700 1 -- 172.21.15.44:0/409644760 learned_addr learned my addr 172.21.15.44:0/409644760 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2d9effd700 1 --2- 172.21.15.44:0/409644760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6cc0 0x562559bf0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2d9ffff700 1 --2- 172.21.15.44:0/409644760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559bec1a0 0x562559bf0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2d9effd700 1 -- 172.21.15.44:0/409644760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6300 msgr2=0x562559bf2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2d9effd700 1 --2- 172.21.15.44:0/409644760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6300 0x562559bf2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2d9effd700 1 -- 172.21.15.44:0/409644760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559bec1a0 msgr2=0x562559bf0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2d9effd700 1 --2- 172.21.15.44:0/409644760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559bec1a0 0x562559bf0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2d9effd700 1 -- 172.21.15.44:0/409644760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da01144f0 con 0x562559be6cc0 2023-09-10T01:14:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2d9f7fe700 1 --2- 172.21.15.44:0/409644760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6300 0x562559bf2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:14:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.932+0000 7f2d9ffff700 1 --2- 172.21.15.44:0/409644760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559bec1a0 0x562559bf0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:14:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.933+0000 7f2d9effd700 1 --2- 172.21.15.44:0/409644760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6cc0 0x562559bf0170 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2d94005f50 tx=0x7f2d94005340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaa5ec963c0ae82f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.933+0000 7f2d7effd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:14:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.933+0000 7f2d7effd700 1 -- 172.21.15.44:0/409644760 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9402d040 con 0x562559be6cc0 2023-09-10T01:14:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.933+0000 7f2d7effd700 1 -- 172.21.15.44:0/409644760 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2d9403e940 con 0x562559be6cc0 2023-09-10T01:14:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.933+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/409644760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562559a90490 con 0x562559be6cc0 2023-09-10T01:14:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.933+0000 7f2d7effd700 1 -- 172.21.15.44:0/409644760 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d9403eb00 con 0x562559be6cc0 2023-09-10T01:14:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.933+0000 7f2db1d1ab80 10 client.29033 fetch_fsmap learned FSMap version 120 2023-09-10T01:14:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.933+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/409644760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x56255994dc00 con 0x562559be6cc0 2023-09-10T01:14:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.933+0000 7f2d7effd700 1 -- 172.21.15.44:0/409644760 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562559a90490 con 0x562559be6cc0 2023-09-10T01:14:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.933+0000 7f2d7effd700 1 -- 172.21.15.44:0/409644760 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f2d94045070 con 0x562559be6cc0 2023-09-10T01:14:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.933+0000 7f2db1d1ab80 10 client.29033 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:14:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.933+0000 7f2db1d1ab80 10 client.29033 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:14:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.933+0000 7f2db1d1ab80 20 client.29033 populate_metadata read hostname 'smithi044' 2023-09-10T01:14:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.933+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/409644760 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562559c10f20 0x562559c13310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.933+0000 7f2db1d1ab80 4 client.29033 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:14:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.933+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/409644760 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562559b4d840 con 0x562559c10f20 2023-09-10T01:14:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.934+0000 7f2d9f7fe700 1 --2- 172.21.15.44:0/409644760 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562559c10f20 0x562559c13310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.934+0000 7f2d9f7fe700 1 --2- 172.21.15.44:0/409644760 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562559c10f20 0x562559c13310 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e9a84f48af20c9 server_cookie=f22a1d600d386849 in_seq=0 out_seq=0 2023-09-10T01:14:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.934+0000 7f2d7effd700 10 client.29033 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:14:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.935+0000 7f2d7effd700 1 -- 172.21.15.44:0/409644760 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562559b4d840 con 0x562559c10f20 2023-09-10T01:14:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.935+0000 7f2d7effd700 10 client.29033 handle_command_reply: tid=0 2023-09-10T01:14:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.935+0000 7f2db1d1ab80 1 client.29033 shutdown 2023-09-10T01:14:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.935+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/409644760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6cc0 msgr2=0x562559bf0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.935+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/409644760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6cc0 0x562559bf0170 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2d94005f50 tx=0x7f2d94005340 comp rx=0 tx=0).stop 2023-09-10T01:14:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.935+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/409644760 shutdown_connections 2023-09-10T01:14:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.935+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/409644760 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562559c10f20 0x562559c13310 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.935+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/409644760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559bec1a0 0x562559bf0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.935+0000 7f2d7effd700 0 client.29033 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:14:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.935+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/409644760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6300 0x562559bf2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.935+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/409644760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6cc0 0x562559bf0170 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.935+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/409644760 >> 172.21.15.44:0/409644760 conn(0x562559b5b030 msgr2=0x562559be07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.935+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/409644760 shutdown_connections 2023-09-10T01:14:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.935+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/409644760 wait complete. 2023-09-10T01:14:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.935+0000 7f2db1d1ab80 20 client.29033 trim_cache size 0 max 16384 2023-09-10T01:14:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.953+0000 7f2db1d1ab80 1 Processor -- start 2023-09-10T01:14:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2db1d1ab80 1 -- start start 2023-09-10T01:14:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2db1d1ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559be6300 0x562559be6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2db1d1ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6cc0 0x562559c084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2db1d1ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559bec1a0 0x562559bf2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2db1d1ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559c089e0 con 0x562559be6300 2023-09-10T01:14:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2db1d1ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559bf1390 con 0x562559be6cc0 2023-09-10T01:14:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2db1d1ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559be72f0 con 0x562559bec1a0 2023-09-10T01:14:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2d9f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559be6300 0x562559be6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2d9ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559bec1a0 0x562559bf2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2d9f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559be6300 0x562559be6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57302/0 (socket says 172.21.15.44:57302) 2023-09-10T01:14:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2d9f7fe700 1 -- 172.21.15.44:0/2533165267 learned_addr learned my addr 172.21.15.44:0/2533165267 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2d9f7fe700 1 -- 172.21.15.44:0/2533165267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559bec1a0 msgr2=0x562559bf2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2d9effd700 1 --2- 172.21.15.44:0/2533165267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6cc0 0x562559c084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2d9f7fe700 1 --2- 172.21.15.44:0/2533165267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559bec1a0 0x562559bf2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2d9f7fe700 1 -- 172.21.15.44:0/2533165267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6cc0 msgr2=0x562559c084a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2d9f7fe700 1 --2- 172.21.15.44:0/2533165267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6cc0 0x562559c084a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2d9f7fe700 1 -- 172.21.15.44:0/2533165267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56255998b1f0 con 0x562559be6300 2023-09-10T01:14:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2d9ffff700 1 --2- 172.21.15.44:0/2533165267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559bec1a0 0x562559bf2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:14:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.954+0000 7f2d9effd700 1 --2- 172.21.15.44:0/2533165267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6cc0 0x562559c084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:14:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.955+0000 7f2d9f7fe700 1 --2- 172.21.15.44:0/2533165267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559be6300 0x562559be6720 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009230 tx=0x7f2d9000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3120358faef4e31 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.955+0000 7f2d867fc700 1 -- 172.21.15.44:0/2533165267 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90099680 con 0x562559be6300 2023-09-10T01:14:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.955+0000 7f2d867fc700 1 -- 172.21.15.44:0/2533165267 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d9009d4b0 con 0x562559be6300 2023-09-10T01:14:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.955+0000 7f2d867fc700 1 -- 172.21.15.44:0/2533165267 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d900998e0 con 0x562559be6300 2023-09-10T01:14:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.956+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/2533165267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559be6300 msgr2=0x562559be6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.956+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/2533165267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559be6300 0x562559be6720 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009230 tx=0x7f2d9000cf60 comp rx=0 tx=0).stop 2023-09-10T01:14:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.956+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/2533165267 shutdown_connections 2023-09-10T01:14:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.956+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/2533165267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559bec1a0 0x562559bf2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.956+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/2533165267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559be6300 0x562559be6720 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.956+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/2533165267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559be6cc0 0x562559c084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.956+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/2533165267 >> 172.21.15.44:0/2533165267 conn(0x562559b5b030 msgr2=0x562559beb160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.956+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/2533165267 shutdown_connections 2023-09-10T01:14:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.956+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/2533165267 wait complete. 2023-09-10T01:14:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.956+0000 7f2db1d1ab80 1 -- start start 2023-09-10T01:14:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.956+0000 7f2db1d1ab80 1 Processor -- start 2023-09-10T01:14:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2db1d1ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6300 0x562559befda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2db1d1ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559be6cc0 0x562559bee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2db1d1ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559bec1a0 0x562559beec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2db1d1ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559bf04b0 con 0x562559be6cc0 2023-09-10T01:14:07.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2db1d1ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559bef150 con 0x562559bec1a0 2023-09-10T01:14:07.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2db1d1ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559bef310 con 0x562559be6300 2023-09-10T01:14:07.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2db1d1ab80 10 client.0 _ll_register_callbacks cb 0x7f2d940039a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:14:07.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2d9f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6300 0x562559befda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2d9effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559be6cc0 0x562559bee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2d9f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6300 0x562559befda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:35958/0 (socket says 172.21.15.44:35958) 2023-09-10T01:14:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2d9f7fe700 1 -- 172.21.15.44:0/405335724 learned_addr learned my addr 172.21.15.44:0/405335724 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2d9ffff700 1 --2- 172.21.15.44:0/405335724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559bec1a0 0x562559beec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2d9effd700 1 -- 172.21.15.44:0/405335724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6300 msgr2=0x562559befda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2d9effd700 1 --2- 172.21.15.44:0/405335724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6300 0x562559befda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2d9effd700 1 -- 172.21.15.44:0/405335724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559bec1a0 msgr2=0x562559beec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2d9effd700 1 --2- 172.21.15.44:0/405335724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559bec1a0 0x562559beec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2d9effd700 1 -- 172.21.15.44:0/405335724 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d98003c90 con 0x562559be6cc0 2023-09-10T01:14:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2d9ffff700 1 --2- 172.21.15.44:0/405335724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559bec1a0 0x562559beec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:14:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.957+0000 7f2d9f7fe700 1 --2- 172.21.15.44:0/405335724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6300 0x562559befda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:14:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.958+0000 7f2d9effd700 1 --2- 172.21.15.44:0/405335724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559be6cc0 0x562559bee6d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x562559ad6890 tx=0x7f2d9401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e232113b01632f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.958+0000 7f2d7effd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:14:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.958+0000 7f2d7effd700 1 -- 172.21.15.44:0/405335724 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9402d040 con 0x562559be6cc0 2023-09-10T01:14:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.958+0000 7f2d7effd700 1 -- 172.21.15.44:0/405335724 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2d9403ea20 con 0x562559be6cc0 2023-09-10T01:14:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.958+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/405335724 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d94005550 con 0x562559be6cc0 2023-09-10T01:14:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.958+0000 7f2d7effd700 1 -- 172.21.15.44:0/405335724 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d94047d40 con 0x562559be6cc0 2023-09-10T01:14:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.958+0000 7f2db1d1ab80 10 client.19371 fetch_fsmap learned FSMap version 120 2023-09-10T01:14:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.958+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/405335724 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x7f2d9401a600 con 0x562559be6cc0 2023-09-10T01:14:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.958+0000 7f2d7effd700 1 -- 172.21.15.44:0/405335724 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d94005550 con 0x562559be6cc0 2023-09-10T01:14:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.958+0000 7f2d7effd700 1 -- 172.21.15.44:0/405335724 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f2d94045070 con 0x562559be6cc0 2023-09-10T01:14:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.958+0000 7f2db1d1ab80 10 client.19371 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:14:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.959+0000 7f2db1d1ab80 10 client.19371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:14:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.959+0000 7f2db1d1ab80 20 client.19371 populate_metadata read hostname 'smithi044' 2023-09-10T01:14:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.959+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/405335724 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562559c10f20 0x562559c67b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.959+0000 7f2db1d1ab80 4 client.19371 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:14:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.959+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/405335724 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562559b318c0 con 0x562559c10f20 2023-09-10T01:14:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.959+0000 7f2d9f7fe700 1 --2- 172.21.15.44:0/405335724 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562559c10f20 0x562559c67b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.959+0000 7f2d7effd700 10 client.19371 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:14:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.959+0000 7f2d9f7fe700 1 --2- 172.21.15.44:0/405335724 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562559c10f20 0x562559c67b10 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c610fdcdf2e123d server_cookie=5bfcb24d664a97db in_seq=0 out_seq=0 2023-09-10T01:14:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.971+0000 7f2d7effd700 1 -- 172.21.15.44:0/405335724 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562559b318c0 con 0x562559c10f20 2023-09-10T01:14:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.971+0000 7f2d7effd700 10 client.19371 handle_command_reply: tid=0 2023-09-10T01:14:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.971+0000 7f2db1d1ab80 1 client.19371 shutdown 2023-09-10T01:14:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.971+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/405335724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559be6cc0 msgr2=0x562559bee6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.971+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/405335724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559be6cc0 0x562559bee6d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x562559ad6890 tx=0x7f2d9401c420 comp rx=0 tx=0).stop 2023-09-10T01:14:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.971+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/405335724 shutdown_connections 2023-09-10T01:14:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.971+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/405335724 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562559c10f20 0x562559c67b10 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.971+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/405335724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559bec1a0 0x562559beec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.971+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/405335724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562559be6300 0x562559befda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.971+0000 7f2db1d1ab80 1 --2- 172.21.15.44:0/405335724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562559be6cc0 0x562559bee6d0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.971+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/405335724 >> 172.21.15.44:0/405335724 conn(0x562559b5b030 msgr2=0x562559bea1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.971+0000 7f2d7effd700 0 client.19371 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:14:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.971+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/405335724 shutdown_connections 2023-09-10T01:14:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.971+0000 7f2db1d1ab80 1 -- 172.21.15.44:0/405335724 wait complete. 2023-09-10T01:14:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.972+0000 7f2db1d1ab80 20 client.19371 trim_cache size 0 max 16384 2023-09-10T01:14:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:14:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:14:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:14:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:14:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.972+0000 7f2da5b54700 1 -- 172.21.15.44:0/1111876047 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2d880a07b0 msgr2=0x7f2d880a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.972+0000 7f2da5b54700 1 --2- 172.21.15.44:0/1111876047 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2d880a07b0 0x7f2d880a2c70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2da0077db0 tx=0x7f2d9402a040 comp rx=0 tx=0).stop 2023-09-10T01:14:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.972+0000 7f2da5b54700 1 -- 172.21.15.44:0/1111876047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da00689f0 msgr2=0x7f2da010f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.972+0000 7f2da5b54700 1 --2- 172.21.15.44:0/1111876047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da00689f0 0x7f2da010f500 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000e9f0 tx=0x7f2d9000ed00 comp rx=0 tx=0).stop 2023-09-10T01:14:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.972+0000 7f2da5b54700 1 -- 172.21.15.44:0/1111876047 shutdown_connections 2023-09-10T01:14:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.972+0000 7f2da5b54700 1 --2- 172.21.15.44:0/1111876047 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2d880a07b0 0x7f2d880a2c70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.972+0000 7f2da5b54700 1 --2- 172.21.15.44:0/1111876047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2da0105560 0x7f2da01a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.972+0000 7f2da5b54700 1 --2- 172.21.15.44:0/1111876047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da00689f0 0x7f2da010f500 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.972+0000 7f2da5b54700 1 --2- 172.21.15.44:0/1111876047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2da0103020 0x7f2da010fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.972+0000 7f2da5b54700 1 -- 172.21.15.44:0/1111876047 >> 172.21.15.44:0/1111876047 conn(0x7f2da0077430 msgr2=0x7f2da00fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.972+0000 7f2da5b54700 1 -- 172.21.15.44:0/1111876047 shutdown_connections 2023-09-10T01:14:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:07.972+0000 7f2da5b54700 1 -- 172.21.15.44:0/1111876047 wait complete. 2023-09-10T01:14:09.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:09 smithi131 ceph-mon[103713]: osdmap e151: 12 total, 12 up, 12 in 2023-09-10T01:14:09.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:09 smithi131 ceph-mon[103713]: pgmap v1070: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2023-09-10T01:14:09.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:09 smithi044 ceph-mon[98676]: osdmap e151: 12 total, 12 up, 12 in 2023-09-10T01:14:09.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:09 smithi044 ceph-mon[98676]: pgmap v1070: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2023-09-10T01:14:09.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:09 smithi099 ceph-mon[102692]: osdmap e151: 12 total, 12 up, 12 in 2023-09-10T01:14:09.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:09 smithi099 ceph-mon[102692]: pgmap v1070: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2023-09-10T01:14:12.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:11 smithi131 ceph-mon[103713]: pgmap v1071: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2023-09-10T01:14:12.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:11 smithi044 ceph-mon[98676]: pgmap v1071: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2023-09-10T01:14:12.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:11 smithi099 ceph-mon[102692]: pgmap v1071: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2023-09-10T01:14:13.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:13 smithi131 ceph-mon[103713]: pgmap v1072: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2023-09-10T01:14:13.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:13 smithi044 ceph-mon[98676]: pgmap v1072: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2023-09-10T01:14:13.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:13 smithi099 ceph-mon[102692]: pgmap v1072: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2023-09-10T01:14:15.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:15 smithi131 ceph-mon[103713]: pgmap v1073: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 171 op/s 2023-09-10T01:14:15.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:15 smithi044 ceph-mon[98676]: pgmap v1073: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 171 op/s 2023-09-10T01:14:15.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:15 smithi099 ceph-mon[102692]: pgmap v1073: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 171 op/s 2023-09-10T01:14:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:16 smithi131 ceph-mon[103713]: osdmap e152: 12 total, 12 up, 12 in 2023-09-10T01:14:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:16 smithi044 ceph-mon[98676]: osdmap e152: 12 total, 12 up, 12 in 2023-09-10T01:14:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:16 smithi099 ceph-mon[102692]: osdmap e152: 12 total, 12 up, 12 in 2023-09-10T01:14:18.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:17 smithi131 ceph-mon[103713]: pgmap v1075: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2023-09-10T01:14:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:17 smithi044 ceph-mon[98676]: pgmap v1075: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2023-09-10T01:14:18.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:17 smithi099 ceph-mon[102692]: pgmap v1075: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2023-09-10T01:14:19.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:19 smithi131 ceph-mon[103713]: pgmap v1076: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-09-10T01:14:19.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:19 smithi044 ceph-mon[98676]: pgmap v1076: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-09-10T01:14:19.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:19 smithi099 ceph-mon[102692]: pgmap v1076: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-09-10T01:14:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.088+0000 7fcb5d8e0700 1 -- 172.21.15.44:0/3030880582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb58106690 msgr2=0x7fcb58106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.088+0000 7fcb5d8e0700 1 --2- 172.21.15.44:0/3030880582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb58106690 0x7fcb58106a70 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fcb480099c0 tx=0x7fcb48009cd0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.088+0000 7fcb5d8e0700 1 -- 172.21.15.44:0/3030880582 shutdown_connections 2023-09-10T01:14:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.088+0000 7fcb5d8e0700 1 --2- 172.21.15.44:0/3030880582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb58100f90 0x7fcb5810d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.088+0000 7fcb5d8e0700 1 --2- 172.21.15.44:0/3030880582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb58106690 0x7fcb58106a70 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.088+0000 7fcb5d8e0700 1 --2- 172.21.15.44:0/3030880582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb581005d0 0x7fcb58100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.088+0000 7fcb5d8e0700 1 -- 172.21.15.44:0/3030880582 >> 172.21.15.44:0/3030880582 conn(0x7fcb580fc0c0 msgr2=0x7fcb580fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.088+0000 7fcb5d8e0700 1 -- 172.21.15.44:0/3030880582 shutdown_connections 2023-09-10T01:14:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.088+0000 7fcb5d8e0700 1 -- 172.21.15.44:0/3030880582 wait complete. 2023-09-10T01:14:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb5d8e0700 1 Processor -- start 2023-09-10T01:14:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb5d8e0700 1 -- start start 2023-09-10T01:14:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb5d8e0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb581005d0 0x7fcb58072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb5d8e0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb58100f90 0x7fcb58073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb5d8e0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb58106690 0x7fcb581a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb5d8e0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581a4c30 con 0x7fcb581005d0 2023-09-10T01:14:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb5d8e0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581a4d70 con 0x7fcb58100f90 2023-09-10T01:14:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb5d8e0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581a4ee0 con 0x7fcb58106690 2023-09-10T01:14:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb56ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb581005d0 0x7fcb58072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb56ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb581005d0 0x7fcb58072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57322/0 (socket says 172.21.15.44:57322) 2023-09-10T01:14:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb56ffd700 1 -- 172.21.15.44:0/1144164572 learned_addr learned my addr 172.21.15.44:0/1144164572 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb567fc700 1 --2- 172.21.15.44:0/1144164572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb58100f90 0x7fcb58073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb577fe700 1 --2- 172.21.15.44:0/1144164572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb58106690 0x7fcb581a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb56ffd700 1 -- 172.21.15.44:0/1144164572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb58106690 msgr2=0x7fcb581a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb56ffd700 1 --2- 172.21.15.44:0/1144164572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb58106690 0x7fcb581a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb56ffd700 1 -- 172.21.15.44:0/1144164572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb58100f90 msgr2=0x7fcb58073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb56ffd700 1 --2- 172.21.15.44:0/1144164572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb58100f90 0x7fcb58073020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb56ffd700 1 -- 172.21.15.44:0/1144164572 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb480095f0 con 0x7fcb581005d0 2023-09-10T01:14:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.089+0000 7fcb567fc700 1 --2- 172.21.15.44:0/1144164572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb58100f90 0x7fcb58073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:14:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.090+0000 7fcb56ffd700 1 --2- 172.21.15.44:0/1144164572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb581005d0 0x7fcb58072ae0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcb48005910 tx=0x7fcb48009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.090+0000 7fcb5c8de700 1 -- 172.21.15.44:0/1144164572 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4801d070 con 0x7fcb581005d0 2023-09-10T01:14:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.090+0000 7fcb5c8de700 1 -- 172.21.15.44:0/1144164572 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb48022470 con 0x7fcb581005d0 2023-09-10T01:14:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.090+0000 7fcb5d8e0700 1 -- 172.21.15.44:0/1144164572 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb581a5160 con 0x7fcb581005d0 2023-09-10T01:14:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.090+0000 7fcb5c8de700 1 -- 172.21.15.44:0/1144164572 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48017650 con 0x7fcb581005d0 2023-09-10T01:14:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.090+0000 7fcb5d8e0700 1 -- 172.21.15.44:0/1144164572 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb581a5580 con 0x7fcb581005d0 2023-09-10T01:14:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.092+0000 7fcb5c8de700 1 -- 172.21.15.44:0/1144164572 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fcb480225e0 con 0x7fcb581005d0 2023-09-10T01:14:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.093+0000 7fcb5c8de700 1 --2- 172.21.15.44:0/1144164572 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcb440a07b0 0x7fcb440a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.093+0000 7fcb5c8de700 1 -- 172.21.15.44:0/1144164572 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fcb480cd2a0 con 0x7fcb581005d0 2023-09-10T01:14:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.093+0000 7fcb567fc700 1 --2- 172.21.15.44:0/1144164572 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcb440a07b0 0x7fcb440a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.093+0000 7fcb567fc700 1 --2- 172.21.15.44:0/1144164572 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcb440a07b0 0x7fcb440a2c70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcb58074300 tx=0x7fcb40009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.093+0000 7fcb69aa6b80 1 Processor -- start 2023-09-10T01:14:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.093+0000 7fcb69aa6b80 1 -- start start 2023-09-10T01:14:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.093+0000 7fcb69aa6b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d60a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.093+0000 7fcb69aa6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604300 0x56439d604780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.093+0000 7fcb69aa6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604cc0 0x56439d610f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.093+0000 7fcb69aa6b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439d55d6d0 con 0x56439d60a1a0 2023-09-10T01:14:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.093+0000 7fcb69aa6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439d55aff0 con 0x56439d604cc0 2023-09-10T01:14:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.093+0000 7fcb69aa6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439d5568e0 con 0x56439d604300 2023-09-10T01:14:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.094+0000 7fcb56ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d60a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.094+0000 7fcb56ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d60a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57330/0 (socket says 172.21.15.44:57330) 2023-09-10T01:14:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.094+0000 7fcb56ffd700 1 -- 172.21.15.44:0/2334695499 learned_addr learned my addr 172.21.15.44:0/2334695499 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.094+0000 7fcb567fc700 1 --2- 172.21.15.44:0/2334695499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604300 0x56439d604780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.094+0000 7fcb577fe700 1 --2- 172.21.15.44:0/2334695499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604cc0 0x56439d610f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.094+0000 7fcb56ffd700 1 -- 172.21.15.44:0/2334695499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604300 msgr2=0x56439d604780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.094+0000 7fcb56ffd700 1 --2- 172.21.15.44:0/2334695499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604300 0x56439d604780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.094+0000 7fcb56ffd700 1 -- 172.21.15.44:0/2334695499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604cc0 msgr2=0x56439d610f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.094+0000 7fcb56ffd700 1 --2- 172.21.15.44:0/2334695499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604cc0 0x56439d610f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.094+0000 7fcb56ffd700 1 -- 172.21.15.44:0/2334695499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56439d382d80 con 0x56439d60a1a0 2023-09-10T01:14:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.094+0000 7fcb567fc700 1 --2- 172.21.15.44:0/2334695499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604300 0x56439d604780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:14:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.094+0000 7fcb56ffd700 1 --2- 172.21.15.44:0/2334695499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d60a580 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcb480a1e90 tx=0x7fcb48096850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6d92c687cd736 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.094+0000 7fcb4dffb700 1 -- 172.21.15.44:0/2334695499 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480cd130 con 0x56439d60a1a0 2023-09-10T01:14:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.094+0000 7fcb4dffb700 1 -- 172.21.15.44:0/2334695499 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb4809aae0 con 0x56439d60a1a0 2023-09-10T01:14:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.094+0000 7fcb4dffb700 1 -- 172.21.15.44:0/2334695499 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480a00c0 con 0x56439d60a1a0 2023-09-10T01:14:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.095+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2334695499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 msgr2=0x56439d60a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.095+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/2334695499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d60a580 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcb480a1e90 tx=0x7fcb48096850 comp rx=0 tx=0).stop 2023-09-10T01:14:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.095+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2334695499 shutdown_connections 2023-09-10T01:14:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.095+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/2334695499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604cc0 0x56439d610f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.095+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/2334695499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d60a580 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.095+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/2334695499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604300 0x56439d604780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.095+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2334695499 >> 172.21.15.44:0/2334695499 conn(0x56439d579030 msgr2=0x56439d5ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.095+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2334695499 shutdown_connections 2023-09-10T01:14:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.095+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2334695499 wait complete. 2023-09-10T01:14:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.095+0000 7fcb69aa6b80 1 -- start start 2023-09-10T01:14:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.095+0000 7fcb69aa6b80 1 Processor -- start 2023-09-10T01:14:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb69aa6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604300 0x56439d613b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb69aa6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604cc0 0x56439d6140a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb69aa6b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d60ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb69aa6b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439d4f4b10 con 0x56439d60a1a0 2023-09-10T01:14:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb69aa6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439d4f4810 con 0x56439d604cc0 2023-09-10T01:14:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb69aa6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439d3d88c0 con 0x56439d604300 2023-09-10T01:14:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb69aa6b80 10 client.0 _ll_register_callbacks cb 0x56439d43ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:14:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d60ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb56ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604300 0x56439d613b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d60ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57340/0 (socket says 172.21.15.44:57340) 2023-09-10T01:14:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb577fe700 1 -- 172.21.15.44:0/2658538014 learned_addr learned my addr 172.21.15.44:0/2658538014 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb567fc700 1 --2- 172.21.15.44:0/2658538014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604cc0 0x56439d6140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb577fe700 1 -- 172.21.15.44:0/2658538014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604300 msgr2=0x56439d613b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb577fe700 1 --2- 172.21.15.44:0/2658538014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604300 0x56439d613b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb577fe700 1 -- 172.21.15.44:0/2658538014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604cc0 msgr2=0x56439d6140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb577fe700 1 --2- 172.21.15.44:0/2658538014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604cc0 0x56439d6140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb577fe700 1 -- 172.21.15.44:0/2658538014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb581a5580 con 0x56439d60a1a0 2023-09-10T01:14:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb577fe700 1 --2- 172.21.15.44:0/2658538014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d60ddb0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009790 tx=0x7fcb50007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5deb130d70b4d156 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:14:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb36ffd700 1 -- 172.21.15.44:0/2658538014 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5000e040 con 0x56439d60a1a0 2023-09-10T01:14:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2658538014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56439d4ae490 con 0x56439d60a1a0 2023-09-10T01:14:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.096+0000 7fcb36ffd700 1 -- 172.21.15.44:0/2658538014 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(152..152 src has 1..152) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fcb50015860 con 0x56439d60a1a0 2023-09-10T01:14:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.097+0000 7fcb36ffd700 1 -- 172.21.15.44:0/2658538014 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb50015a20 con 0x56439d60a1a0 2023-09-10T01:14:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.097+0000 7fcb69aa6b80 10 client.19383 fetch_fsmap learned FSMap version 120 2023-09-10T01:14:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.097+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2658538014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x56439d36bc00 con 0x56439d60a1a0 2023-09-10T01:14:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.097+0000 7fcb36ffd700 1 -- 172.21.15.44:0/2658538014 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56439d4ae490 con 0x56439d60a1a0 2023-09-10T01:14:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.097+0000 7fcb36ffd700 1 -- 172.21.15.44:0/2658538014 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fcb50016ce0 con 0x56439d60a1a0 2023-09-10T01:14:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.097+0000 7fcb69aa6b80 10 client.19383 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:14:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.097+0000 7fcb69aa6b80 10 client.19383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:14:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.097+0000 7fcb69aa6b80 20 client.19383 populate_metadata read hostname 'smithi044' 2023-09-10T01:14:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.097+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/2658538014 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56439d61e0f0 0x56439d620510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.097+0000 7fcb69aa6b80 4 client.19383 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:14:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.097+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2658538014 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56439d56b840 con 0x56439d61e0f0 2023-09-10T01:14:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.097+0000 7fcb56ffd700 1 --2- 172.21.15.44:0/2658538014 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56439d61e0f0 0x56439d620510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.098+0000 7fcb36ffd700 10 client.19383 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:14:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.098+0000 7fcb56ffd700 1 --2- 172.21.15.44:0/2658538014 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56439d61e0f0 0x56439d620510 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3be4f3e278fc27b5 server_cookie=cd569497a913dd68 in_seq=0 out_seq=0 2023-09-10T01:14:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.098+0000 7fcb36ffd700 1 -- 172.21.15.44:0/2658538014 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56439d56b840 con 0x56439d61e0f0 2023-09-10T01:14:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.098+0000 7fcb36ffd700 10 client.19383 handle_command_reply: tid=0 2023-09-10T01:14:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.098+0000 7fcb69aa6b80 1 client.19383 shutdown 2023-09-10T01:14:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.098+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2658538014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 msgr2=0x56439d60ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.098+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/2658538014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d60ddb0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009790 tx=0x7fcb50007c30 comp rx=0 tx=0).stop 2023-09-10T01:14:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.098+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2658538014 shutdown_connections 2023-09-10T01:14:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.098+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/2658538014 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56439d61e0f0 0x56439d620510 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.098+0000 7fcb36ffd700 0 client.19383 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:14:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.098+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/2658538014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d60ddb0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.098+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/2658538014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604300 0x56439d613b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.098+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/2658538014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604cc0 0x56439d6140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.098+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2658538014 >> 172.21.15.44:0/2658538014 conn(0x56439d579030 msgr2=0x56439d5fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:21.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.098+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2658538014 shutdown_connections 2023-09-10T01:14:21.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.098+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2658538014 wait complete. 2023-09-10T01:14:21.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.099+0000 7fcb69aa6b80 20 client.19383 trim_cache size 0 max 16384 2023-09-10T01:14:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.113+0000 7fcb69aa6b80 1 Processor -- start 2023-09-10T01:14:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.113+0000 7fcb69aa6b80 1 -- start start 2023-09-10T01:14:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb69aa6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604300 0x56439d614060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb69aa6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604cc0 0x56439d60d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb69aa6b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d608ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb69aa6b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439d5f98a0 con 0x56439d60a1a0 2023-09-10T01:14:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb69aa6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439d6050a0 con 0x56439d604300 2023-09-10T01:14:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb69aa6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439d6055b0 con 0x56439d604cc0 2023-09-10T01:14:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d608ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb56ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604300 0x56439d614060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb56ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604300 0x56439d614060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:59084/0 (socket says 172.21.15.44:59084) 2023-09-10T01:14:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb56ffd700 1 -- 172.21.15.44:0/2892610024 learned_addr learned my addr 172.21.15.44:0/2892610024 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb567fc700 1 --2- 172.21.15.44:0/2892610024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604cc0 0x56439d60d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb56ffd700 1 -- 172.21.15.44:0/2892610024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604cc0 msgr2=0x56439d60d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb56ffd700 1 --2- 172.21.15.44:0/2892610024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604cc0 0x56439d60d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb56ffd700 1 -- 172.21.15.44:0/2892610024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 msgr2=0x56439d608ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb56ffd700 1 --2- 172.21.15.44:0/2892610024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d608ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb56ffd700 1 -- 172.21.15.44:0/2892610024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56439d3a91f0 con 0x56439d604300 2023-09-10T01:14:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb577fe700 1 --2- 172.21.15.44:0/2892610024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d608ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:14:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb56ffd700 1 --2- 172.21.15.44:0/2892610024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604300 0x56439d614060 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcb480cabf0 tx=0x7fcb48022ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69008f2da36f5f9a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.114+0000 7fcb4e7fc700 1 -- 172.21.15.44:0/2892610024 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4809a3e0 con 0x56439d604300 2023-09-10T01:14:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.115+0000 7fcb4e7fc700 1 -- 172.21.15.44:0/2892610024 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb4809e210 con 0x56439d604300 2023-09-10T01:14:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.115+0000 7fcb4e7fc700 1 -- 172.21.15.44:0/2892610024 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480a0310 con 0x56439d604300 2023-09-10T01:14:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.115+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2892610024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604300 msgr2=0x56439d614060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.115+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/2892610024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604300 0x56439d614060 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcb480cabf0 tx=0x7fcb48022ec0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.115+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2892610024 shutdown_connections 2023-09-10T01:14:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.115+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/2892610024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d608ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.115+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/2892610024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604300 0x56439d614060 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.115+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/2892610024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604cc0 0x56439d60d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.115+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2892610024 >> 172.21.15.44:0/2892610024 conn(0x56439d579030 msgr2=0x56439d613950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.115+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2892610024 shutdown_connections 2023-09-10T01:14:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.115+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/2892610024 wait complete. 2023-09-10T01:14:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.115+0000 7fcb69aa6b80 1 -- start start 2023-09-10T01:14:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.115+0000 7fcb69aa6b80 1 Processor -- start 2023-09-10T01:14:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb69aa6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604300 0x56439d617ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb69aa6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604cc0 0x56439d618220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb69aa6b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d618760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb69aa6b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439d671000 con 0x56439d60a1a0 2023-09-10T01:14:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb69aa6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439d671170 con 0x56439d604300 2023-09-10T01:14:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb69aa6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439d671330 con 0x56439d604cc0 2023-09-10T01:14:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb69aa6b80 10 client.0 _ll_register_callbacks cb 0x7fcb50007590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:14:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d618760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d618760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57354/0 (socket says 172.21.15.44:57354) 2023-09-10T01:14:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb567fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604cc0 0x56439d618220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb577fe700 1 -- 172.21.15.44:0/3018760607 learned_addr learned my addr 172.21.15.44:0/3018760607 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb56ffd700 1 --2- 172.21.15.44:0/3018760607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604300 0x56439d617ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb577fe700 1 -- 172.21.15.44:0/3018760607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604cc0 msgr2=0x56439d618220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb577fe700 1 --2- 172.21.15.44:0/3018760607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604cc0 0x56439d618220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb577fe700 1 -- 172.21.15.44:0/3018760607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604300 msgr2=0x56439d617ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb577fe700 1 --2- 172.21.15.44:0/3018760607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604300 0x56439d617ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb577fe700 1 -- 172.21.15.44:0/3018760607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56439d36bc00 con 0x56439d60a1a0 2023-09-10T01:14:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb56ffd700 1 --2- 172.21.15.44:0/3018760607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604300 0x56439d617ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:14:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb577fe700 1 --2- 172.21.15.44:0/3018760607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d618760 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009790 tx=0x7fcb500099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5be16d78ea64c87 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:14:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.116+0000 7fcb36ffd700 1 -- 172.21.15.44:0/3018760607 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5000e070 con 0x56439d60a1a0 2023-09-10T01:14:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.117+0000 7fcb36ffd700 1 -- 172.21.15.44:0/3018760607 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(152..152 src has 1..152) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fcb5001f920 con 0x56439d60a1a0 2023-09-10T01:14:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.117+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/3018760607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb4809aae0 con 0x56439d60a1a0 2023-09-10T01:14:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.117+0000 7fcb36ffd700 1 -- 172.21.15.44:0/3018760607 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb5001fae0 con 0x56439d60a1a0 2023-09-10T01:14:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.117+0000 7fcb69aa6b80 10 client.19389 fetch_fsmap learned FSMap version 120 2023-09-10T01:14:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.117+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/3018760607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x56439d382d80 con 0x56439d60a1a0 2023-09-10T01:14:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.117+0000 7fcb36ffd700 1 -- 172.21.15.44:0/3018760607 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb4809aae0 con 0x56439d60a1a0 2023-09-10T01:14:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.117+0000 7fcb36ffd700 1 -- 172.21.15.44:0/3018760607 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fcb500050a0 con 0x56439d60a1a0 2023-09-10T01:14:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.117+0000 7fcb69aa6b80 10 client.19389 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:14:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.117+0000 7fcb69aa6b80 10 client.19389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:14:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.117+0000 7fcb69aa6b80 20 client.19389 populate_metadata read hostname 'smithi044' 2023-09-10T01:14:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.117+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/3018760607 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56439d61e0f0 0x56439d6749f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.117+0000 7fcb69aa6b80 4 client.19389 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:14:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.117+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/3018760607 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56439d54f8c0 con 0x56439d61e0f0 2023-09-10T01:14:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.117+0000 7fcb56ffd700 1 --2- 172.21.15.44:0/3018760607 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56439d61e0f0 0x56439d6749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.118+0000 7fcb36ffd700 10 client.19389 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:14:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.118+0000 7fcb56ffd700 1 --2- 172.21.15.44:0/3018760607 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56439d61e0f0 0x56439d6749f0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58c51b2f85e48be2 server_cookie=144352fc4eef12e in_seq=0 out_seq=0 2023-09-10T01:14:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.131+0000 7fcb36ffd700 1 -- 172.21.15.44:0/3018760607 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56439d54f8c0 con 0x56439d61e0f0 2023-09-10T01:14:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.131+0000 7fcb36ffd700 10 client.19389 handle_command_reply: tid=0 2023-09-10T01:14:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.131+0000 7fcb69aa6b80 1 client.19389 shutdown 2023-09-10T01:14:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.131+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/3018760607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 msgr2=0x56439d618760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.131+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/3018760607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d618760 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009790 tx=0x7fcb500099b0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/3018760607 shutdown_connections 2023-09-10T01:14:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/3018760607 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56439d61e0f0 0x56439d6749f0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/3018760607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56439d60a1a0 0x56439d618760 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb36ffd700 0 client.19389 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:14:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/3018760607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439d604300 0x56439d617ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb69aa6b80 1 --2- 172.21.15.44:0/3018760607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56439d604cc0 0x56439d618220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/3018760607 >> 172.21.15.44:0/3018760607 conn(0x56439d579030 msgr2=0x56439d608ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/3018760607 shutdown_connections 2023-09-10T01:14:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb69aa6b80 1 -- 172.21.15.44:0/3018760607 wait complete. 2023-09-10T01:14:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb69aa6b80 20 client.19389 trim_cache size 0 max 16384 2023-09-10T01:14:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:14:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:14:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:14:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:14:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb5d8e0700 1 -- 172.21.15.44:0/1144164572 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcb440a07b0 msgr2=0x7fcb440a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb5d8e0700 1 --2- 172.21.15.44:0/1144164572 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcb440a07b0 0x7fcb440a2c70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcb58074300 tx=0x7fcb40009260 comp rx=0 tx=0).stop 2023-09-10T01:14:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb5d8e0700 1 -- 172.21.15.44:0/1144164572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb581005d0 msgr2=0x7fcb58072ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb5d8e0700 1 --2- 172.21.15.44:0/1144164572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb581005d0 0x7fcb58072ae0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcb48005910 tx=0x7fcb48009580 comp rx=0 tx=0).stop 2023-09-10T01:14:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb5d8e0700 1 -- 172.21.15.44:0/1144164572 shutdown_connections 2023-09-10T01:14:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb5d8e0700 1 --2- 172.21.15.44:0/1144164572 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fcb440a07b0 0x7fcb440a2c70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb5d8e0700 1 --2- 172.21.15.44:0/1144164572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb58106690 0x7fcb581a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb5d8e0700 1 --2- 172.21.15.44:0/1144164572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb581005d0 0x7fcb58072ae0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb5d8e0700 1 --2- 172.21.15.44:0/1144164572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb58100f90 0x7fcb58073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb5d8e0700 1 -- 172.21.15.44:0/1144164572 >> 172.21.15.44:0/1144164572 conn(0x7fcb580fc0c0 msgr2=0x7fcb580fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.132+0000 7fcb5d8e0700 1 -- 172.21.15.44:0/1144164572 shutdown_connections 2023-09-10T01:14:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:21.133+0000 7fcb5d8e0700 1 -- 172.21.15.44:0/1144164572 wait complete. 2023-09-10T01:14:22.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:21 smithi131 ceph-mon[103713]: pgmap v1077: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2023-09-10T01:14:22.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:21 smithi044 ceph-mon[98676]: pgmap v1077: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2023-09-10T01:14:22.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:21 smithi099 ceph-mon[102692]: pgmap v1077: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2023-09-10T01:14:23.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:23 smithi131 ceph-mon[103713]: pgmap v1078: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2023-09-10T01:14:23.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:23 smithi044 ceph-mon[98676]: pgmap v1078: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2023-09-10T01:14:23.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:23 smithi099 ceph-mon[102692]: pgmap v1078: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2023-09-10T01:14:25.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:25 smithi131 ceph-mon[103713]: pgmap v1079: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1012 KiB/s wr, 146 op/s 2023-09-10T01:14:25.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:25 smithi044 ceph-mon[98676]: pgmap v1079: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1012 KiB/s wr, 146 op/s 2023-09-10T01:14:25.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:25 smithi099 ceph-mon[102692]: pgmap v1079: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1012 KiB/s wr, 146 op/s 2023-09-10T01:14:27.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:27 smithi131 ceph-mon[103713]: pgmap v1080: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 918 KiB/s wr, 131 op/s 2023-09-10T01:14:27.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:27 smithi044 ceph-mon[98676]: pgmap v1080: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 918 KiB/s wr, 131 op/s 2023-09-10T01:14:27.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:27 smithi099 ceph-mon[102692]: pgmap v1080: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 918 KiB/s wr, 131 op/s 2023-09-10T01:14:29.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:29 smithi131 ceph-mon[103713]: pgmap v1081: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 849 KiB/s wr, 122 op/s 2023-09-10T01:14:29.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:29 smithi044 ceph-mon[98676]: pgmap v1081: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 849 KiB/s wr, 122 op/s 2023-09-10T01:14:29.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:29 smithi099 ceph-mon[102692]: pgmap v1081: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 849 KiB/s wr, 122 op/s 2023-09-10T01:14:32.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:31 smithi131 ceph-mon[103713]: pgmap v1082: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:14:32.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:31 smithi044 ceph-mon[98676]: pgmap v1082: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:14:32.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:31 smithi099 ceph-mon[102692]: pgmap v1082: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:14:33.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:33 smithi131 ceph-mon[103713]: pgmap v1083: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 823 KiB/s wr, 118 op/s 2023-09-10T01:14:33.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:33 smithi044 ceph-mon[98676]: pgmap v1083: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 823 KiB/s wr, 118 op/s 2023-09-10T01:14:33.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:33 smithi099 ceph-mon[102692]: pgmap v1083: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 823 KiB/s wr, 118 op/s 2023-09-10T01:14:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.272+0000 7f18be359700 1 -- 172.21.15.44:0/2941374711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b81054b0 msgr2=0x7f18b80679f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.272+0000 7f18be359700 1 --2- 172.21.15.44:0/2941374711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b81054b0 0x7f18b80679f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009990 tx=0x7f18a8009ca0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.272+0000 7f18be359700 1 -- 172.21.15.44:0/2941374711 shutdown_connections 2023-09-10T01:14:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.272+0000 7f18be359700 1 --2- 172.21.15.44:0/2941374711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b8068610 0x7f18b8103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.272+0000 7f18be359700 1 --2- 172.21.15.44:0/2941374711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b81054b0 0x7f18b80679f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.272+0000 7f18be359700 1 --2- 172.21.15.44:0/2941374711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18b8105920 0x7f18b8067f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.272+0000 7f18be359700 1 -- 172.21.15.44:0/2941374711 >> 172.21.15.44:0/2941374711 conn(0x7f18b8074f30 msgr2=0x7f18b8075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.272+0000 7f18be359700 1 -- 172.21.15.44:0/2941374711 shutdown_connections 2023-09-10T01:14:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.272+0000 7f18be359700 1 -- 172.21.15.44:0/2941374711 wait complete. 2023-09-10T01:14:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.272+0000 7f18be359700 1 Processor -- start 2023-09-10T01:14:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.272+0000 7f18be359700 1 -- start start 2023-09-10T01:14:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18be359700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b81054b0 0x7f18b81a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18be359700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b8105920 0x7f18b81a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18be359700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18b819ca00 0x7f18b819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18be359700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b819d480 con 0x7f18b819ca00 2023-09-10T01:14:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18be359700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b819d5c0 con 0x7f18b8105920 2023-09-10T01:14:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18be359700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b819d780 con 0x7f18b81054b0 2023-09-10T01:14:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18b7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b81054b0 0x7f18b81a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18b77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b8105920 0x7f18b81a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18b7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b81054b0 0x7f18b81a2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36014/0 (socket says 172.21.15.44:36014) 2023-09-10T01:14:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18b7fff700 1 -- 172.21.15.44:0/1337889257 learned_addr learned my addr 172.21.15.44:0/1337889257 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18bc8f6700 1 --2- 172.21.15.44:0/1337889257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18b819ca00 0x7f18b819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18b7fff700 1 -- 172.21.15.44:0/1337889257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b8105920 msgr2=0x7f18b81a30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18b7fff700 1 --2- 172.21.15.44:0/1337889257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b8105920 0x7f18b81a30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18b7fff700 1 -- 172.21.15.44:0/1337889257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18b819ca00 msgr2=0x7f18b819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18b7fff700 1 --2- 172.21.15.44:0/1337889257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18b819ca00 0x7f18b819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18b7fff700 1 -- 172.21.15.44:0/1337889257 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a80095f0 con 0x7f18b81054b0 2023-09-10T01:14:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.273+0000 7f18b7fff700 1 --2- 172.21.15.44:0/1337889257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b81054b0 0x7f18b81a2ba0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f18a8005770 tx=0x7f18a80057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.274+0000 7f18b57fa700 1 -- 172.21.15.44:0/1337889257 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a801d070 con 0x7f18b81054b0 2023-09-10T01:14:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.274+0000 7f18be359700 1 -- 172.21.15.44:0/1337889257 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b81a99b0 con 0x7f18b81054b0 2023-09-10T01:14:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.274+0000 7f18b57fa700 1 -- 172.21.15.44:0/1337889257 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18a8021410 con 0x7f18b81054b0 2023-09-10T01:14:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.274+0000 7f18b57fa700 1 -- 172.21.15.44:0/1337889257 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8017650 con 0x7f18b81054b0 2023-09-10T01:14:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.274+0000 7f18be359700 1 -- 172.21.15.44:0/1337889257 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b81a9f10 con 0x7f18b81054b0 2023-09-10T01:14:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.276+0000 7f18b57fa700 1 -- 172.21.15.44:0/1337889257 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f18a80217f0 con 0x7f18b81054b0 2023-09-10T01:14:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.277+0000 7f18b57fa700 1 --2- 172.21.15.44:0/1337889257 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f18a00a07b0 0x7f18a00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.277+0000 7f18b57fa700 1 -- 172.21.15.44:0/1337889257 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f18a80cd6a0 con 0x7f18b81054b0 2023-09-10T01:14:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.277+0000 7f18b77fe700 1 --2- 172.21.15.44:0/1337889257 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f18a00a07b0 0x7f18a00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.277+0000 7f18b77fe700 1 --2- 172.21.15.44:0/1337889257 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f18a00a07b0 0x7f18a00a2c70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f18b80fca80 tx=0x7f18ac009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.278+0000 7f18ca51fb80 1 Processor -- start 2023-09-10T01:14:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.278+0000 7f18ca51fb80 1 -- start start 2023-09-10T01:14:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.278+0000 7f18ca51fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae53b1a0 0x5598ae53b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.278+0000 7f18ca51fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535300 0x5598ae535780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.278+0000 7f18ca51fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae535cc0 0x5598ae541f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.278+0000 7f18ca51fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ae48e6d0 con 0x5598ae53b1a0 2023-09-10T01:14:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.278+0000 7f18ca51fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ae48bff0 con 0x5598ae535300 2023-09-10T01:14:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.278+0000 7f18ca51fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ae4878e0 con 0x5598ae535cc0 2023-09-10T01:14:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.278+0000 7f18b7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae53b1a0 0x5598ae53b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.278+0000 7f18b7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae53b1a0 0x5598ae53b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57372/0 (socket says 172.21.15.44:57372) 2023-09-10T01:14:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.278+0000 7f18bc8f6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae535cc0 0x5598ae541f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.278+0000 7f18b7fff700 1 -- 172.21.15.44:0/1707579459 learned_addr learned my addr 172.21.15.44:0/1707579459 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.278+0000 7f18b77fe700 1 --2- 172.21.15.44:0/1707579459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535300 0x5598ae535780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.279+0000 7f18b7fff700 1 -- 172.21.15.44:0/1707579459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae535cc0 msgr2=0x5598ae541f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.279+0000 7f18b7fff700 1 --2- 172.21.15.44:0/1707579459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae535cc0 0x5598ae541f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.279+0000 7f18b7fff700 1 -- 172.21.15.44:0/1707579459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535300 msgr2=0x5598ae535780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.279+0000 7f18b7fff700 1 --2- 172.21.15.44:0/1707579459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535300 0x5598ae535780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.279+0000 7f18b7fff700 1 -- 172.21.15.44:0/1707579459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598ae2b3d80 con 0x5598ae53b1a0 2023-09-10T01:14:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.279+0000 7f18b77fe700 1 --2- 172.21.15.44:0/1707579459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535300 0x5598ae535780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:14:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.279+0000 7f18bc8f6700 1 --2- 172.21.15.44:0/1707579459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae535cc0 0x5598ae541f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:14:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.279+0000 7f18b7fff700 1 --2- 172.21.15.44:0/1707579459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae53b1a0 0x5598ae53b580 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f18a80caa50 tx=0x7f18a809a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932cbea0707d9e67 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.279+0000 7f189e7fc700 1 -- 172.21.15.44:0/1707579459 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a809a7e0 con 0x5598ae53b1a0 2023-09-10T01:14:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.279+0000 7f189e7fc700 1 -- 172.21.15.44:0/1707579459 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18a80cc070 con 0x5598ae53b1a0 2023-09-10T01:14:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.279+0000 7f189e7fc700 1 -- 172.21.15.44:0/1707579459 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a809aa40 con 0x5598ae53b1a0 2023-09-10T01:14:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.280+0000 7f18ca51fb80 1 -- 172.21.15.44:0/1707579459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae53b1a0 msgr2=0x5598ae53b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.280+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/1707579459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae53b1a0 0x5598ae53b580 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f18a80caa50 tx=0x7f18a809a320 comp rx=0 tx=0).stop 2023-09-10T01:14:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.280+0000 7f18ca51fb80 1 -- 172.21.15.44:0/1707579459 shutdown_connections 2023-09-10T01:14:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.280+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/1707579459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae535cc0 0x5598ae541f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.280+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/1707579459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae53b1a0 0x5598ae53b580 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.280+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/1707579459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535300 0x5598ae535780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.280+0000 7f18ca51fb80 1 -- 172.21.15.44:0/1707579459 >> 172.21.15.44:0/1707579459 conn(0x5598ae4aa030 msgr2=0x5598ae530560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.280+0000 7f18ca51fb80 1 -- 172.21.15.44:0/1707579459 shutdown_connections 2023-09-10T01:14:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.280+0000 7f18ca51fb80 1 -- 172.21.15.44:0/1707579459 wait complete. 2023-09-10T01:14:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.280+0000 7f18ca51fb80 1 -- start start 2023-09-10T01:14:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18ca51fb80 1 Processor -- start 2023-09-10T01:14:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18ca51fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae535300 0x5598ae544c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18ca51fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535cc0 0x5598ae545150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18ca51fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae53b1a0 0x5598ae53ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18ca51fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ae425b10 con 0x5598ae535300 2023-09-10T01:14:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18ca51fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ae425810 con 0x5598ae535cc0 2023-09-10T01:14:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18ca51fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ae3098c0 con 0x5598ae53b1a0 2023-09-10T01:14:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18ca51fb80 10 client.0 _ll_register_callbacks cb 0x5598ae36fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:14:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18b7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae535300 0x5598ae544c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18b7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae535300 0x5598ae544c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57378/0 (socket says 172.21.15.44:57378) 2023-09-10T01:14:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18b7fff700 1 -- 172.21.15.44:0/4463071 learned_addr learned my addr 172.21.15.44:0/4463071 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18b77fe700 1 --2- 172.21.15.44:0/4463071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535cc0 0x5598ae545150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18bc8f6700 1 --2- 172.21.15.44:0/4463071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae53b1a0 0x5598ae53ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18b7fff700 1 -- 172.21.15.44:0/4463071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae53b1a0 msgr2=0x5598ae53ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18b7fff700 1 --2- 172.21.15.44:0/4463071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae53b1a0 0x5598ae53ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18b7fff700 1 -- 172.21.15.44:0/4463071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535cc0 msgr2=0x5598ae545150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18b7fff700 1 --2- 172.21.15.44:0/4463071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535cc0 0x5598ae545150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.281+0000 7f18b7fff700 1 -- 172.21.15.44:0/4463071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ac017070 con 0x5598ae535300 2023-09-10T01:14:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18b77fe700 1 --2- 172.21.15.44:0/4463071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535cc0 0x5598ae545150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:14:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18bc8f6700 1 --2- 172.21.15.44:0/4463071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae53b1a0 0x5598ae53ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:14:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18b7fff700 1 --2- 172.21.15.44:0/4463071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae535300 0x5598ae544c10 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f18a80cd980 tx=0x7f18a800b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5e437839c790b4 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18977fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:14:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18977fe700 1 -- 172.21.15.44:0/4463071 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a809e610 con 0x5598ae535300 2023-09-10T01:14:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18977fe700 1 -- 172.21.15.44:0/4463071 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(152..152 src has 1..152) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f18a80a27c0 con 0x5598ae535300 2023-09-10T01:14:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18ca51fb80 1 -- 172.21.15.44:0/4463071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598ae3df490 con 0x5598ae535300 2023-09-10T01:14:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18977fe700 1 -- 172.21.15.44:0/4463071 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18a809aa40 con 0x5598ae535300 2023-09-10T01:14:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18ca51fb80 10 client.19401 fetch_fsmap learned FSMap version 120 2023-09-10T01:14:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18ca51fb80 1 -- 172.21.15.44:0/4463071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x7f18b81a9f10 con 0x5598ae535300 2023-09-10T01:14:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18977fe700 1 -- 172.21.15.44:0/4463071 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5598ae3df490 con 0x5598ae535300 2023-09-10T01:14:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18977fe700 1 -- 172.21.15.44:0/4463071 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f18a80cd260 con 0x5598ae535300 2023-09-10T01:14:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18ca51fb80 10 client.19401 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:14:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18ca51fb80 10 client.19401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:14:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18ca51fb80 20 client.19401 populate_metadata read hostname 'smithi044' 2023-09-10T01:14:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/4463071 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5598ae54ec50 0x5598ae551070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18ca51fb80 4 client.19401 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:14:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.282+0000 7f18ca51fb80 1 -- 172.21.15.44:0/4463071 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598ae49c840 con 0x5598ae54ec50 2023-09-10T01:14:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.283+0000 7f18b77fe700 1 --2- 172.21.15.44:0/4463071 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5598ae54ec50 0x5598ae551070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.283+0000 7f18977fe700 10 client.19401 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:14:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.283+0000 7f18b77fe700 1 --2- 172.21.15.44:0/4463071 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5598ae54ec50 0x5598ae551070 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61614194eae585f server_cookie=acd24ec8931e1deb in_seq=0 out_seq=0 2023-09-10T01:14:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.284+0000 7f18977fe700 1 -- 172.21.15.44:0/4463071 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5598ae49c840 con 0x5598ae54ec50 2023-09-10T01:14:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.284+0000 7f18977fe700 10 client.19401 handle_command_reply: tid=0 2023-09-10T01:14:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.284+0000 7f18ca51fb80 1 client.19401 shutdown 2023-09-10T01:14:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.284+0000 7f18ca51fb80 1 -- 172.21.15.44:0/4463071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae535300 msgr2=0x5598ae544c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.284+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/4463071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae535300 0x5598ae544c10 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f18a80cd980 tx=0x7f18a800b490 comp rx=0 tx=0).stop 2023-09-10T01:14:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.284+0000 7f18ca51fb80 1 -- 172.21.15.44:0/4463071 shutdown_connections 2023-09-10T01:14:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.284+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/4463071 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5598ae54ec50 0x5598ae551070 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.284+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/4463071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae53b1a0 0x5598ae53ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.284+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/4463071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae535300 0x5598ae544c10 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.284+0000 7f18977fe700 0 client.19401 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:14:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.284+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/4463071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535cc0 0x5598ae545150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.284+0000 7f18ca51fb80 1 -- 172.21.15.44:0/4463071 >> 172.21.15.44:0/4463071 conn(0x5598ae4aa030 msgr2=0x5598ae52f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.284+0000 7f18ca51fb80 1 -- 172.21.15.44:0/4463071 shutdown_connections 2023-09-10T01:14:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.284+0000 7f18ca51fb80 1 -- 172.21.15.44:0/4463071 wait complete. 2023-09-10T01:14:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.284+0000 7f18ca51fb80 20 client.19401 trim_cache size 0 max 16384 2023-09-10T01:14:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.302+0000 7f18ca51fb80 1 Processor -- start 2023-09-10T01:14:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.302+0000 7f18ca51fb80 1 -- start start 2023-09-10T01:14:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.302+0000 7f18ca51fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae535300 0x5598ae53eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f18ca51fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535cc0 0x5598ae53a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f18ca51fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae53b1a0 0x5598ae53a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f18ca51fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ae53bc60 con 0x5598ae53b1a0 2023-09-10T01:14:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f18ca51fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ae536410 con 0x5598ae535cc0 2023-09-10T01:14:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f18ca51fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ae539d50 con 0x5598ae535300 2023-09-10T01:14:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f18b7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae535300 0x5598ae53eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f18bc8f6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae53b1a0 0x5598ae53a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f18b7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae535300 0x5598ae53eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36036/0 (socket says 172.21.15.44:36036) 2023-09-10T01:14:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f18b7fff700 1 -- 172.21.15.44:0/3007521668 learned_addr learned my addr 172.21.15.44:0/3007521668 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f18b77fe700 1 --2- 172.21.15.44:0/3007521668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535cc0 0x5598ae53a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f18b7fff700 1 -- 172.21.15.44:0/3007521668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535cc0 msgr2=0x5598ae53a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f18b7fff700 1 --2- 172.21.15.44:0/3007521668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535cc0 0x5598ae53a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f18b7fff700 1 -- 172.21.15.44:0/3007521668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae53b1a0 msgr2=0x5598ae53a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f18b7fff700 1 --2- 172.21.15.44:0/3007521668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae53b1a0 0x5598ae53a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f18b7fff700 1 -- 172.21.15.44:0/3007521668 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598ae29cc00 con 0x5598ae535300 2023-09-10T01:14:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f18b7fff700 1 --2- 172.21.15.44:0/3007521668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae535300 0x5598ae53eec0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f18a8097250 tx=0x7f18a809a320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c66ba771f1d118f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.303+0000 7f189effd700 1 -- 172.21.15.44:0/3007521668 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8096980 con 0x5598ae535300 2023-09-10T01:14:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.304+0000 7f189effd700 1 -- 172.21.15.44:0/3007521668 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18a8096720 con 0x5598ae535300 2023-09-10T01:14:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.304+0000 7f189effd700 1 -- 172.21.15.44:0/3007521668 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a80a2330 con 0x5598ae535300 2023-09-10T01:14:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.304+0000 7f18ca51fb80 1 -- 172.21.15.44:0/3007521668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae535300 msgr2=0x5598ae53eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.304+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/3007521668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae535300 0x5598ae53eec0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f18a8097250 tx=0x7f18a809a320 comp rx=0 tx=0).stop 2023-09-10T01:14:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.304+0000 7f18ca51fb80 1 -- 172.21.15.44:0/3007521668 shutdown_connections 2023-09-10T01:14:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.304+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/3007521668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae53b1a0 0x5598ae53a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.304+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/3007521668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae535300 0x5598ae53eec0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.304+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/3007521668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535cc0 0x5598ae53a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.304+0000 7f18ca51fb80 1 -- 172.21.15.44:0/3007521668 >> 172.21.15.44:0/3007521668 conn(0x5598ae4aa030 msgr2=0x5598ae53e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:34.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.304+0000 7f18ca51fb80 1 -- 172.21.15.44:0/3007521668 shutdown_connections 2023-09-10T01:14:34.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.304+0000 7f18ca51fb80 1 -- 172.21.15.44:0/3007521668 wait complete. 2023-09-10T01:14:34.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.305+0000 7f18ca51fb80 1 -- start start 2023-09-10T01:14:34.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.305+0000 7f18ca51fb80 1 Processor -- start 2023-09-10T01:14:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.305+0000 7f18ca51fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535300 0x5598ae59be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.305+0000 7f18ca51fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae535cc0 0x5598ae59c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.305+0000 7f18ca51fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae53b1a0 0x5598ae596a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.305+0000 7f18ca51fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ae596ff0 con 0x5598ae535cc0 2023-09-10T01:14:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.305+0000 7f18ca51fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ae597130 con 0x5598ae535300 2023-09-10T01:14:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.305+0000 7f18ca51fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ae5972f0 con 0x5598ae53b1a0 2023-09-10T01:14:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.305+0000 7f18ca51fb80 10 client.0 _ll_register_callbacks cb 0x5598ae36fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:14:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.305+0000 7f18b77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae535cc0 0x5598ae59c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.305+0000 7f18b7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535300 0x5598ae59be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.305+0000 7f18b77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae535cc0 0x5598ae59c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57394/0 (socket says 172.21.15.44:57394) 2023-09-10T01:14:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.305+0000 7f18b77fe700 1 -- 172.21.15.44:0/2836816785 learned_addr learned my addr 172.21.15.44:0/2836816785 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.305+0000 7f18bc8f6700 1 --2- 172.21.15.44:0/2836816785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae53b1a0 0x5598ae596a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.306+0000 7f18b77fe700 1 -- 172.21.15.44:0/2836816785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae53b1a0 msgr2=0x5598ae596a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.306+0000 7f18b77fe700 1 --2- 172.21.15.44:0/2836816785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae53b1a0 0x5598ae596a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.306+0000 7f18b77fe700 1 -- 172.21.15.44:0/2836816785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535300 msgr2=0x5598ae59be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.306+0000 7f18b77fe700 1 --2- 172.21.15.44:0/2836816785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535300 0x5598ae59be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.306+0000 7f18b77fe700 1 -- 172.21.15.44:0/2836816785 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b81a9f10 con 0x5598ae535cc0 2023-09-10T01:14:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.306+0000 7f18b7fff700 1 --2- 172.21.15.44:0/2836816785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535300 0x5598ae59be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:14:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.306+0000 7f18b77fe700 1 --2- 172.21.15.44:0/2836816785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae535cc0 0x5598ae59c370 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f18ac005950 tx=0x7f18ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f79d78652837880 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.306+0000 7f18977fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:14:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.306+0000 7f18977fe700 1 -- 172.21.15.44:0/2836816785 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac024040 con 0x5598ae535cc0 2023-09-10T01:14:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.306+0000 7f18977fe700 1 -- 172.21.15.44:0/2836816785 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(152..152 src has 1..152) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f18ac01fb90 con 0x5598ae535cc0 2023-09-10T01:14:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.306+0000 7f18ca51fb80 1 -- 172.21.15.44:0/2836816785 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18a80cc070 con 0x5598ae535cc0 2023-09-10T01:14:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.306+0000 7f18977fe700 1 -- 172.21.15.44:0/2836816785 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18ac028730 con 0x5598ae535cc0 2023-09-10T01:14:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.306+0000 7f18ca51fb80 10 client.19407 fetch_fsmap learned FSMap version 120 2023-09-10T01:14:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.306+0000 7f18ca51fb80 1 -- 172.21.15.44:0/2836816785 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x5598ae2da1f0 con 0x5598ae535cc0 2023-09-10T01:14:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.306+0000 7f18977fe700 1 -- 172.21.15.44:0/2836816785 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18a80cc070 con 0x5598ae535cc0 2023-09-10T01:14:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.307+0000 7f18977fe700 1 -- 172.21.15.44:0/2836816785 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f18ac028af0 con 0x5598ae535cc0 2023-09-10T01:14:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.307+0000 7f18ca51fb80 10 client.19407 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:14:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.307+0000 7f18ca51fb80 10 client.19407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:14:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.307+0000 7f18ca51fb80 20 client.19407 populate_metadata read hostname 'smithi044' 2023-09-10T01:14:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.307+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/2836816785 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5598ae5a3c30 0x5598ae5a6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.307+0000 7f18ca51fb80 4 client.19407 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:14:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.307+0000 7f18ca51fb80 1 -- 172.21.15.44:0/2836816785 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5598ae4808c0 con 0x5598ae5a3c30 2023-09-10T01:14:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.307+0000 7f18b7fff700 1 --2- 172.21.15.44:0/2836816785 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5598ae5a3c30 0x5598ae5a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.307+0000 7f18b7fff700 1 --2- 172.21.15.44:0/2836816785 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5598ae5a3c30 0x5598ae5a6050 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efb1910f26926356 server_cookie=51fb6a7c5a2bd82e in_seq=0 out_seq=0 2023-09-10T01:14:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.307+0000 7f18977fe700 10 client.19407 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:14:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.324+0000 7f18977fe700 1 -- 172.21.15.44:0/2836816785 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5598ae4808c0 con 0x5598ae5a3c30 2023-09-10T01:14:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.324+0000 7f18977fe700 10 client.19407 handle_command_reply: tid=0 2023-09-10T01:14:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.324+0000 7f18ca51fb80 1 client.19407 shutdown 2023-09-10T01:14:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.324+0000 7f18ca51fb80 1 -- 172.21.15.44:0/2836816785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae535cc0 msgr2=0x5598ae59c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.324+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/2836816785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae535cc0 0x5598ae59c370 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f18ac005950 tx=0x7f18ac0047c0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.324+0000 7f18ca51fb80 1 -- 172.21.15.44:0/2836816785 shutdown_connections 2023-09-10T01:14:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.324+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/2836816785 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5598ae5a3c30 0x5598ae5a6050 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.324+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/2836816785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598ae53b1a0 0x5598ae596a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.324+0000 7f18977fe700 0 client.19407 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:14:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.324+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/2836816785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598ae535300 0x5598ae59be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.324+0000 7f18ca51fb80 1 --2- 172.21.15.44:0/2836816785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ae535cc0 0x5598ae59c370 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.324+0000 7f18ca51fb80 1 -- 172.21.15.44:0/2836816785 >> 172.21.15.44:0/2836816785 conn(0x5598ae4aa030 msgr2=0x5598ae53a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.324+0000 7f18ca51fb80 1 -- 172.21.15.44:0/2836816785 shutdown_connections 2023-09-10T01:14:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.324+0000 7f18ca51fb80 1 -- 172.21.15.44:0/2836816785 wait complete. 2023-09-10T01:14:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.325+0000 7f18ca51fb80 20 client.19407 trim_cache size 0 max 16384 2023-09-10T01:14:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:14:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:14:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:14:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:14:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.325+0000 7f18be359700 1 -- 172.21.15.44:0/1337889257 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f18a00a07b0 msgr2=0x7f18a00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.325+0000 7f18be359700 1 --2- 172.21.15.44:0/1337889257 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f18a00a07b0 0x7f18a00a2c70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f18b80fca80 tx=0x7f18ac009260 comp rx=0 tx=0).stop 2023-09-10T01:14:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.325+0000 7f18be359700 1 -- 172.21.15.44:0/1337889257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b81054b0 msgr2=0x7f18b81a2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.325+0000 7f18be359700 1 --2- 172.21.15.44:0/1337889257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b81054b0 0x7f18b81a2ba0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f18a8005770 tx=0x7f18a80057a0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.325+0000 7f18be359700 1 -- 172.21.15.44:0/1337889257 shutdown_connections 2023-09-10T01:14:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.325+0000 7f18be359700 1 --2- 172.21.15.44:0/1337889257 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f18a00a07b0 0x7f18a00a2c70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.325+0000 7f18be359700 1 --2- 172.21.15.44:0/1337889257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18b819ca00 0x7f18b819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.325+0000 7f18be359700 1 --2- 172.21.15.44:0/1337889257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b81054b0 0x7f18b81a2ba0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.325+0000 7f18be359700 1 --2- 172.21.15.44:0/1337889257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b8105920 0x7f18b81a30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.325+0000 7f18be359700 1 -- 172.21.15.44:0/1337889257 >> 172.21.15.44:0/1337889257 conn(0x7f18b8074f30 msgr2=0x7f18b8101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.325+0000 7f18be359700 1 -- 172.21.15.44:0/1337889257 shutdown_connections 2023-09-10T01:14:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:34.325+0000 7f18be359700 1 -- 172.21.15.44:0/1337889257 wait complete. 2023-09-10T01:14:35.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:35 smithi131 ceph-mon[103713]: pgmap v1084: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2023-09-10T01:14:35.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:35 smithi044 ceph-mon[98676]: pgmap v1084: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2023-09-10T01:14:35.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:35 smithi099 ceph-mon[102692]: pgmap v1084: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2023-09-10T01:14:37.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:37 smithi131 ceph-mon[103713]: pgmap v1085: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 881 KiB/s wr, 127 op/s 2023-09-10T01:14:37.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:37 smithi044 ceph-mon[98676]: pgmap v1085: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 881 KiB/s wr, 127 op/s 2023-09-10T01:14:37.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:37 smithi099 ceph-mon[102692]: pgmap v1085: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 881 KiB/s wr, 127 op/s 2023-09-10T01:14:39.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:39 smithi131 ceph-mon[103713]: pgmap v1086: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 876 KiB/s wr, 125 op/s 2023-09-10T01:14:39.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:39 smithi044 ceph-mon[98676]: pgmap v1086: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 876 KiB/s wr, 125 op/s 2023-09-10T01:14:39.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:39 smithi099 ceph-mon[102692]: pgmap v1086: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 876 KiB/s wr, 125 op/s 2023-09-10T01:14:42.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:41 smithi131 ceph-mon[103713]: pgmap v1087: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:14:42.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:41 smithi044 ceph-mon[98676]: pgmap v1087: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:14:42.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:41 smithi099 ceph-mon[102692]: pgmap v1087: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:14:43.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:43 smithi131 ceph-mon[103713]: pgmap v1088: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 148 op/s 2023-09-10T01:14:43.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:43 smithi044 ceph-mon[98676]: pgmap v1088: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 148 op/s 2023-09-10T01:14:43.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:43 smithi099 ceph-mon[102692]: pgmap v1088: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 148 op/s 2023-09-10T01:14:45.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:45 smithi131 ceph-mon[103713]: pgmap v1089: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T01:14:45.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:45 smithi044 ceph-mon[98676]: pgmap v1089: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T01:14:45.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:45 smithi099 ceph-mon[102692]: pgmap v1089: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-09-10T01:14:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.456+0000 7f59c16b9700 1 -- 172.21.15.44:0/1770824049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59bc100120 msgr2=0x7f59bc10c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.456+0000 7f59c16b9700 1 --2- 172.21.15.44:0/1770824049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59bc100120 0x7f59bc10c330 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f59b400b0d0 tx=0x7f59b400b3e0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.456+0000 7f59c16b9700 1 -- 172.21.15.44:0/1770824049 shutdown_connections 2023-09-10T01:14:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.456+0000 7f59c16b9700 1 --2- 172.21.15.44:0/1770824049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59bc100120 0x7f59bc10c330 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.456+0000 7f59c16b9700 1 --2- 172.21.15.44:0/1770824049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59bc1055b0 0x7f59bc105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.456+0000 7f59c16b9700 1 --2- 172.21.15.44:0/1770824049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc0ff760 0x7f59bc0ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.456+0000 7f59c16b9700 1 -- 172.21.15.44:0/1770824049 >> 172.21.15.44:0/1770824049 conn(0x7f59bc074f30 msgr2=0x7f59bc075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.456+0000 7f59c16b9700 1 -- 172.21.15.44:0/1770824049 shutdown_connections 2023-09-10T01:14:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.456+0000 7f59c16b9700 1 -- 172.21.15.44:0/1770824049 wait complete. 2023-09-10T01:14:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.457+0000 7f59c16b9700 1 Processor -- start 2023-09-10T01:14:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.457+0000 7f59c16b9700 1 -- start start 2023-09-10T01:14:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.457+0000 7f59c16b9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc0ff760 0x7f59bc19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.457+0000 7f59c16b9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59bc100120 0x7f59bc19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.457+0000 7f59c16b9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59bc1055b0 0x7f59bc196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.457+0000 7f59c16b9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc196b70 con 0x7f59bc1055b0 2023-09-10T01:14:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.457+0000 7f59c16b9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc196ce0 con 0x7f59bc100120 2023-09-10T01:14:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.457+0000 7f59c16b9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc196ea0 con 0x7f59bc0ff760 2023-09-10T01:14:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59baffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc0ff760 0x7f59bc19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59ba7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59bc100120 0x7f59bc19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59ba7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59bc100120 0x7f59bc19cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:59146/0 (socket says 172.21.15.44:59146) 2023-09-10T01:14:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59baffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc0ff760 0x7f59bc19c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36054/0 (socket says 172.21.15.44:36054) 2023-09-10T01:14:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59ba7fc700 1 -- 172.21.15.44:0/1206298203 learned_addr learned my addr 172.21.15.44:0/1206298203 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59bb7fe700 1 --2- 172.21.15.44:0/1206298203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59bc1055b0 0x7f59bc196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59baffd700 1 -- 172.21.15.44:0/1206298203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59bc100120 msgr2=0x7f59bc19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59baffd700 1 --2- 172.21.15.44:0/1206298203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59bc100120 0x7f59bc19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59baffd700 1 -- 172.21.15.44:0/1206298203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59bc1055b0 msgr2=0x7f59bc196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59baffd700 1 --2- 172.21.15.44:0/1206298203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59bc1055b0 0x7f59bc196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59baffd700 1 -- 172.21.15.44:0/1206298203 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b4009d00 con 0x7f59bc0ff760 2023-09-10T01:14:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59ba7fc700 1 --2- 172.21.15.44:0/1206298203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59bc100120 0x7f59bc19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:14:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59baffd700 1 --2- 172.21.15.44:0/1206298203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc0ff760 0x7f59bc19c6a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00d950 tx=0x7f59ac00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59a3fff700 1 -- 172.21.15.44:0/1206298203 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac00f890 con 0x7f59bc0ff760 2023-09-10T01:14:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59c16b9700 1 -- 172.21.15.44:0/1206298203 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc197180 con 0x7f59bc0ff760 2023-09-10T01:14:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59a3fff700 1 -- 172.21.15.44:0/1206298203 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59ac00fe80 con 0x7f59bc0ff760 2023-09-10T01:14:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59a3fff700 1 -- 172.21.15.44:0/1206298203 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac00e3b0 con 0x7f59bc0ff760 2023-09-10T01:14:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.458+0000 7f59c16b9700 1 -- 172.21.15.44:0/1206298203 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc1a3690 con 0x7f59bc0ff760 2023-09-10T01:14:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.462+0000 7f59a3fff700 1 -- 172.21.15.44:0/1206298203 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f59ac00f9f0 con 0x7f59bc0ff760 2023-09-10T01:14:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.462+0000 7f59a3fff700 1 --2- 172.21.15.44:0/1206298203 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f59a40a07b0 0x7f59a40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.462+0000 7f59a3fff700 1 -- 172.21.15.44:0/1206298203 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f59ac0cc0a0 con 0x7f59bc0ff760 2023-09-10T01:14:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.462+0000 7f59ba7fc700 1 --2- 172.21.15.44:0/1206298203 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f59a40a07b0 0x7f59a40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.463+0000 7f59ba7fc700 1 --2- 172.21.15.44:0/1206298203 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f59a40a07b0 0x7f59a40a2c70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f59bc197970 tx=0x7f59b0009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.463+0000 7f59cd87fb80 1 Processor -- start 2023-09-10T01:14:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.463+0000 7f59cd87fb80 1 -- start start 2023-09-10T01:14:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.463+0000 7f59cd87fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa221a0 0x56327fa22580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.463+0000 7f59cd87fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1c300 0x56327fa1c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.463+0000 7f59cd87fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa1ccc0 0x56327fa28f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59cd87fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327f9756d0 con 0x56327fa1ccc0 2023-09-10T01:14:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59cd87fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327f972ff0 con 0x56327fa1c300 2023-09-10T01:14:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59cd87fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327f96e8e0 con 0x56327fa221a0 2023-09-10T01:14:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59baffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa221a0 0x56327fa22580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59baffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa221a0 0x56327fa22580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36062/0 (socket says 172.21.15.44:36062) 2023-09-10T01:14:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59baffd700 1 -- 172.21.15.44:0/3613905441 learned_addr learned my addr 172.21.15.44:0/3613905441 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59ba7fc700 1 --2- 172.21.15.44:0/3613905441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1c300 0x56327fa1c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59bb7fe700 1 --2- 172.21.15.44:0/3613905441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa1ccc0 0x56327fa28f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59ba7fc700 1 -- 172.21.15.44:0/3613905441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa221a0 msgr2=0x56327fa22580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59ba7fc700 1 --2- 172.21.15.44:0/3613905441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa221a0 0x56327fa22580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59ba7fc700 1 -- 172.21.15.44:0/3613905441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa1ccc0 msgr2=0x56327fa28f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59ba7fc700 1 --2- 172.21.15.44:0/3613905441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa1ccc0 0x56327fa28f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59ba7fc700 1 -- 172.21.15.44:0/3613905441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56327f79ad80 con 0x56327fa1c300 2023-09-10T01:14:47.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59baffd700 1 --2- 172.21.15.44:0/3613905441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa221a0 0x56327fa22580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:14:47.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59ba7fc700 1 --2- 172.21.15.44:0/3613905441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1c300 0x56327fa1c780 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f59b0011c70 tx=0x7f59b0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8546098f5d254495 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:47.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59a17fa700 1 -- 172.21.15.44:0/3613905441 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0004830 con 0x56327fa1c300 2023-09-10T01:14:47.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59a17fa700 1 -- 172.21.15.44:0/3613905441 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59b0021070 con 0x56327fa1c300 2023-09-10T01:14:47.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.464+0000 7f59a17fa700 1 -- 172.21.15.44:0/3613905441 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b00266e0 con 0x56327fa1c300 2023-09-10T01:14:47.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.465+0000 7f59cd87fb80 1 -- 172.21.15.44:0/3613905441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1c300 msgr2=0x56327fa1c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:47.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.465+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/3613905441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1c300 0x56327fa1c780 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f59b0011c70 tx=0x7f59b0014000 comp rx=0 tx=0).stop 2023-09-10T01:14:47.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.465+0000 7f59cd87fb80 1 -- 172.21.15.44:0/3613905441 shutdown_connections 2023-09-10T01:14:47.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.465+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/3613905441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa1ccc0 0x56327fa28f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.465+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/3613905441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa221a0 0x56327fa22580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.465+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/3613905441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1c300 0x56327fa1c780 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.465+0000 7f59cd87fb80 1 -- 172.21.15.44:0/3613905441 >> 172.21.15.44:0/3613905441 conn(0x56327f991030 msgr2=0x56327fa17560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.465+0000 7f59cd87fb80 1 -- 172.21.15.44:0/3613905441 shutdown_connections 2023-09-10T01:14:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.466+0000 7f59cd87fb80 1 -- 172.21.15.44:0/3613905441 wait complete. 2023-09-10T01:14:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.466+0000 7f59cd87fb80 1 -- start start 2023-09-10T01:14:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.466+0000 7f59cd87fb80 1 Processor -- start 2023-09-10T01:14:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.466+0000 7f59cd87fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa28ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.466+0000 7f59cd87fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 0x56327fa26170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.466+0000 7f59cd87fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 0x56327fa26860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.466+0000 7f59cd87fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327f90cb10 con 0x56327fa221a0 2023-09-10T01:14:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.466+0000 7f59cd87fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327f90c810 con 0x56327fa1ccc0 2023-09-10T01:14:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.466+0000 7f59cd87fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327f7f08c0 con 0x56327fa1c300 2023-09-10T01:14:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.466+0000 7f59cd87fb80 10 client.0 _ll_register_callbacks cb 0x56327f856a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:14:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.466+0000 7f59baffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa28ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.466+0000 7f59ba7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 0x56327fa26170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.466+0000 7f59baffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa28ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36068/0 (socket says 172.21.15.44:36068) 2023-09-10T01:14:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.466+0000 7f59ba7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 0x56327fa26170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:59160/0 (socket says 172.21.15.44:59160) 2023-09-10T01:14:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.466+0000 7f59baffd700 1 -- 172.21.15.44:0/1755632529 learned_addr learned my addr 172.21.15.44:0/1755632529 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.467+0000 7f59bb7fe700 1 --2- 172.21.15.44:0/1755632529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 0x56327fa26860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.467+0000 7f59baffd700 1 -- 172.21.15.44:0/1755632529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 msgr2=0x56327fa26170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.467+0000 7f59baffd700 1 --2- 172.21.15.44:0/1755632529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 0x56327fa26170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.467+0000 7f59baffd700 1 -- 172.21.15.44:0/1755632529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 msgr2=0x56327fa26860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.467+0000 7f59baffd700 1 --2- 172.21.15.44:0/1755632529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 0x56327fa26860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.467+0000 7f59baffd700 1 -- 172.21.15.44:0/1755632529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc1a3690 con 0x56327fa1c300 2023-09-10T01:14:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.467+0000 7f59ba7fc700 1 --2- 172.21.15.44:0/1755632529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 0x56327fa26170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:14:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.467+0000 7f59baffd700 1 --2- 172.21.15.44:0/1755632529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa28ef0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f59ac005950 tx=0x7f59ac010750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b77dba1c60bcdd8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.467+0000 7f599eff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:14:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.467+0000 7f599eff5700 1 -- 172.21.15.44:0/1755632529 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac099550 con 0x56327fa1c300 2023-09-10T01:14:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.467+0000 7f599eff5700 1 -- 172.21.15.44:0/1755632529 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(152..152 src has 1..152) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f59ac0990f0 con 0x56327fa1c300 2023-09-10T01:14:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.467+0000 7f59cd87fb80 1 -- 172.21.15.44:0/1755632529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56327f8c6490 con 0x56327fa1c300 2023-09-10T01:14:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.467+0000 7f599eff5700 1 -- 172.21.15.44:0/1755632529 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59ac09cf20 con 0x56327fa1c300 2023-09-10T01:14:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.467+0000 7f59cd87fb80 10 client.29029 fetch_fsmap learned FSMap version 120 2023-09-10T01:14:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.467+0000 7f59cd87fb80 1 -- 172.21.15.44:0/1755632529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x7f59b0017070 con 0x56327fa1c300 2023-09-10T01:14:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.468+0000 7f599eff5700 1 -- 172.21.15.44:0/1755632529 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56327f8c6490 con 0x56327fa1c300 2023-09-10T01:14:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.468+0000 7f599eff5700 1 -- 172.21.15.44:0/1755632529 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f59ac0a0cd0 con 0x56327fa1c300 2023-09-10T01:14:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.468+0000 7f59cd87fb80 10 client.29029 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:14:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.468+0000 7f59cd87fb80 10 client.29029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:14:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.468+0000 7f59cd87fb80 20 client.29029 populate_metadata read hostname 'smithi044' 2023-09-10T01:14:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.468+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/1755632529 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56327fa36360 0x56327fa38780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.468+0000 7f59cd87fb80 4 client.29029 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:14:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.468+0000 7f59cd87fb80 1 -- 172.21.15.44:0/1755632529 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56327f983840 con 0x56327fa36360 2023-09-10T01:14:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.468+0000 7f59ba7fc700 1 --2- 172.21.15.44:0/1755632529 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56327fa36360 0x56327fa38780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.469+0000 7f599eff5700 10 client.29029 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:14:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.469+0000 7f59ba7fc700 1 --2- 172.21.15.44:0/1755632529 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56327fa36360 0x56327fa38780 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f7bb09f65230e8d server_cookie=b809f73b4c67b3ab in_seq=0 out_seq=0 2023-09-10T01:14:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.469+0000 7f599eff5700 1 -- 172.21.15.44:0/1755632529 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56327f983840 con 0x56327fa36360 2023-09-10T01:14:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.469+0000 7f599eff5700 10 client.29029 handle_command_reply: tid=0 2023-09-10T01:14:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.469+0000 7f59cd87fb80 1 client.29029 shutdown 2023-09-10T01:14:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.469+0000 7f59cd87fb80 1 -- 172.21.15.44:0/1755632529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 msgr2=0x56327fa28ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.469+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/1755632529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa28ef0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f59ac005950 tx=0x7f59ac010750 comp rx=0 tx=0).stop 2023-09-10T01:14:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.470+0000 7f59cd87fb80 1 -- 172.21.15.44:0/1755632529 shutdown_connections 2023-09-10T01:14:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.470+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/1755632529 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56327fa36360 0x56327fa38780 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.470+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/1755632529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 0x56327fa26860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.470+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/1755632529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa28ef0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.470+0000 7f599eff5700 0 client.29029 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:14:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.470+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/1755632529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 0x56327fa26170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.470+0000 7f59cd87fb80 1 -- 172.21.15.44:0/1755632529 >> 172.21.15.44:0/1755632529 conn(0x56327f991030 msgr2=0x56327fa167a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.470+0000 7f59cd87fb80 1 -- 172.21.15.44:0/1755632529 shutdown_connections 2023-09-10T01:14:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.470+0000 7f59cd87fb80 1 -- 172.21.15.44:0/1755632529 wait complete. 2023-09-10T01:14:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.470+0000 7f59cd87fb80 20 client.29029 trim_cache size 0 max 16384 2023-09-10T01:14:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.487+0000 7f59cd87fb80 1 Processor -- start 2023-09-10T01:14:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.487+0000 7f59cd87fb80 1 -- start start 2023-09-10T01:14:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.487+0000 7f59cd87fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa23d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.487+0000 7f59cd87fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 0x56327fa25400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.487+0000 7f59cd87fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 0x56327fa28a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.487+0000 7f59cd87fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327fa1d5f0 con 0x56327fa221a0 2023-09-10T01:14:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.487+0000 7f59cd87fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327fa18960 con 0x56327fa1ccc0 2023-09-10T01:14:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.487+0000 7f59cd87fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327fa26e70 con 0x56327fa1c300 2023-09-10T01:14:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.487+0000 7f59bb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 0x56327fa28a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.487+0000 7f59baffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa23d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.487+0000 7f59baffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa23d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36076/0 (socket says 172.21.15.44:36076) 2023-09-10T01:14:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.487+0000 7f59bb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 0x56327fa28a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57428/0 (socket says 172.21.15.44:57428) 2023-09-10T01:14:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.487+0000 7f59baffd700 1 -- 172.21.15.44:0/3276695947 learned_addr learned my addr 172.21.15.44:0/3276695947 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.488+0000 7f59ba7fc700 1 --2- 172.21.15.44:0/3276695947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 0x56327fa25400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.488+0000 7f59bb7fe700 1 -- 172.21.15.44:0/3276695947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 msgr2=0x56327fa23d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.488+0000 7f59bb7fe700 1 --2- 172.21.15.44:0/3276695947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa23d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.488+0000 7f59bb7fe700 1 -- 172.21.15.44:0/3276695947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 msgr2=0x56327fa25400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.488+0000 7f59bb7fe700 1 --2- 172.21.15.44:0/3276695947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 0x56327fa25400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.488+0000 7f59bb7fe700 1 -- 172.21.15.44:0/3276695947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56327f783c00 con 0x56327fa221a0 2023-09-10T01:14:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.488+0000 7f59baffd700 1 --2- 172.21.15.44:0/3276695947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa23d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:14:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.488+0000 7f59bb7fe700 1 --2- 172.21.15.44:0/3276695947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 0x56327fa28a20 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f59bc19d3a0 tx=0x7f59b40093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef064dcf51ba6dd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.488+0000 7f59a1ffb700 1 -- 172.21.15.44:0/3276695947 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4021070 con 0x56327fa221a0 2023-09-10T01:14:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.488+0000 7f59a1ffb700 1 -- 172.21.15.44:0/3276695947 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59b40094a0 con 0x56327fa221a0 2023-09-10T01:14:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.488+0000 7f59a1ffb700 1 -- 172.21.15.44:0/3276695947 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b400e040 con 0x56327fa221a0 2023-09-10T01:14:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.489+0000 7f59cd87fb80 1 -- 172.21.15.44:0/3276695947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 msgr2=0x56327fa28a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.489+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/3276695947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 0x56327fa28a20 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f59bc19d3a0 tx=0x7f59b40093b0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.489+0000 7f59cd87fb80 1 -- 172.21.15.44:0/3276695947 shutdown_connections 2023-09-10T01:14:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.489+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/3276695947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 0x56327fa28a20 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.489+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/3276695947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa23d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.489+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/3276695947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 0x56327fa25400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.489+0000 7f59cd87fb80 1 -- 172.21.15.44:0/3276695947 >> 172.21.15.44:0/3276695947 conn(0x56327f991030 msgr2=0x56327fa17090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.489+0000 7f59cd87fb80 1 -- 172.21.15.44:0/3276695947 shutdown_connections 2023-09-10T01:14:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.489+0000 7f59cd87fb80 1 -- 172.21.15.44:0/3276695947 wait complete. 2023-09-10T01:14:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.489+0000 7f59cd87fb80 1 -- start start 2023-09-10T01:14:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.489+0000 7f59cd87fb80 1 Processor -- start 2023-09-10T01:14:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59cd87fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa2bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59cd87fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 0x56327fa2c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59cd87fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 0x56327fa2c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59cd87fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327fa8a1a0 con 0x56327fa221a0 2023-09-10T01:14:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59cd87fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327fa8a2e0 con 0x56327fa1ccc0 2023-09-10T01:14:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59cd87fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327fa8a420 con 0x56327fa1c300 2023-09-10T01:14:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59cd87fb80 10 client.0 _ll_register_callbacks cb 0x56327f856a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:14:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59baffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa2bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59baffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa2bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36080/0 (socket says 172.21.15.44:36080) 2023-09-10T01:14:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59baffd700 1 -- 172.21.15.44:0/2699137579 learned_addr learned my addr 172.21.15.44:0/2699137579 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:14:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59bb7fe700 1 --2- 172.21.15.44:0/2699137579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 0x56327fa2c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59ba7fc700 1 --2- 172.21.15.44:0/2699137579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 0x56327fa2c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59baffd700 1 -- 172.21.15.44:0/2699137579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 msgr2=0x56327fa2c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59baffd700 1 --2- 172.21.15.44:0/2699137579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 0x56327fa2c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59baffd700 1 -- 172.21.15.44:0/2699137579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 msgr2=0x56327fa2c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:14:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59baffd700 1 --2- 172.21.15.44:0/2699137579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 0x56327fa2c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59baffd700 1 -- 172.21.15.44:0/2699137579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b0017070 con 0x56327fa1c300 2023-09-10T01:14:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.490+0000 7f59baffd700 1 --2- 172.21.15.44:0/2699137579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa2bb60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f59ac098df0 tx=0x7f59ac099600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42ed024820aecc0f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:14:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.491+0000 7f599eff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:14:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.491+0000 7f599eff5700 1 -- 172.21.15.44:0/2699137579 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac014050 con 0x56327fa1c300 2023-09-10T01:14:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.491+0000 7f59cd87fb80 1 -- 172.21.15.44:0/2699137579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59b0021070 con 0x56327fa1c300 2023-09-10T01:14:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.491+0000 7f599eff5700 1 -- 172.21.15.44:0/2699137579 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(152..152 src has 1..152) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f59ac0990f0 con 0x56327fa1c300 2023-09-10T01:14:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.491+0000 7f599eff5700 1 -- 172.21.15.44:0/2699137579 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59ac09cf20 con 0x56327fa1c300 2023-09-10T01:14:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.491+0000 7f59cd87fb80 10 client.29035 fetch_fsmap learned FSMap version 120 2023-09-10T01:14:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.491+0000 7f59cd87fb80 1 -- 172.21.15.44:0/2699137579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x7f59bc0679f0 con 0x56327fa1c300 2023-09-10T01:14:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.491+0000 7f599eff5700 1 -- 172.21.15.44:0/2699137579 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59b0021070 con 0x56327fa1c300 2023-09-10T01:14:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.491+0000 7f599eff5700 1 -- 172.21.15.44:0/2699137579 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f59ac0985e0 con 0x56327fa1c300 2023-09-10T01:14:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.491+0000 7f59cd87fb80 10 client.29035 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:14:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.491+0000 7f59cd87fb80 10 client.29035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:14:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.491+0000 7f59cd87fb80 20 client.29035 populate_metadata read hostname 'smithi044' 2023-09-10T01:14:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.491+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/2699137579 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56327fa8b6a0 0x56327fa8da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:14:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.491+0000 7f59cd87fb80 4 client.29035 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:14:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.491+0000 7f59cd87fb80 1 -- 172.21.15.44:0/2699137579 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56327f9678c0 con 0x56327fa8b6a0 2023-09-10T01:14:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.492+0000 7f59ba7fc700 1 --2- 172.21.15.44:0/2699137579 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56327fa8b6a0 0x56327fa8da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:14:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.492+0000 7f59ba7fc700 1 --2- 172.21.15.44:0/2699137579 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56327fa8b6a0 0x56327fa8da90 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f00f84f55589d7d8 server_cookie=c054f08b0fa7ff0 in_seq=0 out_seq=0 2023-09-10T01:14:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.492+0000 7f599eff5700 10 client.29035 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:14:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.506+0000 7f599eff5700 1 -- 172.21.15.44:0/2699137579 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56327f9678c0 con 0x56327fa8b6a0 2023-09-10T01:14:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.506+0000 7f599eff5700 10 client.29035 handle_command_reply: tid=0 2023-09-10T01:14:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.506+0000 7f59cd87fb80 1 client.29035 shutdown 2023-09-10T01:14:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.507+0000 7f59cd87fb80 1 -- 172.21.15.44:0/2699137579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 msgr2=0x56327fa2bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.507+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/2699137579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa2bb60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f59ac098df0 tx=0x7f59ac099600 comp rx=0 tx=0).stop 2023-09-10T01:14:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.507+0000 7f59cd87fb80 1 -- 172.21.15.44:0/2699137579 shutdown_connections 2023-09-10T01:14:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.507+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/2699137579 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56327fa8b6a0 0x56327fa8da90 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.507+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/2699137579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56327fa221a0 0x56327fa2c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.507+0000 7f599eff5700 0 client.29035 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:14:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.507+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/2699137579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56327fa1c300 0x56327fa2bb60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.507+0000 7f59cd87fb80 1 --2- 172.21.15.44:0/2699137579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56327fa1ccc0 0x56327fa2c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.507+0000 7f59cd87fb80 1 -- 172.21.15.44:0/2699137579 >> 172.21.15.44:0/2699137579 conn(0x56327f991030 msgr2=0x56327fa259e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.507+0000 7f59cd87fb80 1 -- 172.21.15.44:0/2699137579 shutdown_connections 2023-09-10T01:14:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.507+0000 7f59cd87fb80 1 -- 172.21.15.44:0/2699137579 wait complete. 2023-09-10T01:14:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.507+0000 7f59cd87fb80 20 client.29035 trim_cache size 0 max 16384 2023-09-10T01:14:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:14:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:14:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:14:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:14:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.508+0000 7f59c16b9700 1 -- 172.21.15.44:0/1206298203 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f59a40a07b0 msgr2=0x7f59a40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.508+0000 7f59c16b9700 1 --2- 172.21.15.44:0/1206298203 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f59a40a07b0 0x7f59a40a2c70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f59bc197970 tx=0x7f59b0009260 comp rx=0 tx=0).stop 2023-09-10T01:14:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.508+0000 7f59c16b9700 1 -- 172.21.15.44:0/1206298203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc0ff760 msgr2=0x7f59bc19c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:14:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.508+0000 7f59c16b9700 1 --2- 172.21.15.44:0/1206298203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc0ff760 0x7f59bc19c6a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00d950 tx=0x7f59ac00dc60 comp rx=0 tx=0).stop 2023-09-10T01:14:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.508+0000 7f59c16b9700 1 -- 172.21.15.44:0/1206298203 shutdown_connections 2023-09-10T01:14:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.508+0000 7f59c16b9700 1 --2- 172.21.15.44:0/1206298203 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f59a40a07b0 0x7f59a40a2c70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.508+0000 7f59c16b9700 1 --2- 172.21.15.44:0/1206298203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59bc1055b0 0x7f59bc196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.508+0000 7f59c16b9700 1 --2- 172.21.15.44:0/1206298203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc0ff760 0x7f59bc19c6a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.508+0000 7f59c16b9700 1 --2- 172.21.15.44:0/1206298203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59bc100120 0x7f59bc19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:14:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.508+0000 7f59c16b9700 1 -- 172.21.15.44:0/1206298203 >> 172.21.15.44:0/1206298203 conn(0x7f59bc074f30 msgr2=0x7f59bc0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:14:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.508+0000 7f59c16b9700 1 -- 172.21.15.44:0/1206298203 shutdown_connections 2023-09-10T01:14:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:14:47.508+0000 7f59c16b9700 1 -- 172.21.15.44:0/1206298203 wait complete. 2023-09-10T01:14:47.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:47 smithi131 ceph-mon[103713]: pgmap v1090: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-09-10T01:14:47.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:47 smithi044 ceph-mon[98676]: pgmap v1090: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-09-10T01:14:47.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:47 smithi099 ceph-mon[102692]: pgmap v1090: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-09-10T01:14:49.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:49 smithi131 ceph-mon[103713]: pgmap v1091: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 149 op/s 2023-09-10T01:14:49.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:49 smithi044 ceph-mon[98676]: pgmap v1091: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 149 op/s 2023-09-10T01:14:49.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:49 smithi099 ceph-mon[102692]: pgmap v1091: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 149 op/s 2023-09-10T01:14:52.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:51 smithi131 ceph-mon[103713]: pgmap v1092: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2023-09-10T01:14:52.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:51 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:14:52.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:51 smithi044 ceph-mon[98676]: pgmap v1092: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2023-09-10T01:14:52.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:51 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:14:52.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:51 smithi099 ceph-mon[102692]: pgmap v1092: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2023-09-10T01:14:52.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:51 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:14:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:52 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:14:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:52 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:14:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:52 smithi131 ceph-mon[103713]: pgmap v1093: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2023-09-10T01:14:53.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:52 smithi131 ceph-mon[103713]: pgmap v1094: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2023-09-10T01:14:53.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:52 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:14:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:52 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:14:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:52 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:14:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:52 smithi044 ceph-mon[98676]: pgmap v1093: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2023-09-10T01:14:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:52 smithi044 ceph-mon[98676]: pgmap v1094: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2023-09-10T01:14:53.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:52 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:14:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:52 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:14:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:52 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:14:53.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:52 smithi099 ceph-mon[102692]: pgmap v1093: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2023-09-10T01:14:53.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:52 smithi099 ceph-mon[102692]: pgmap v1094: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2023-09-10T01:14:53.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:52 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:14:55.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:55 smithi131 ceph-mon[103713]: pgmap v1095: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 626 KiB/s wr, 91 op/s 2023-09-10T01:14:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:55 smithi044 ceph-mon[98676]: pgmap v1095: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 626 KiB/s wr, 91 op/s 2023-09-10T01:14:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:55 smithi099 ceph-mon[102692]: pgmap v1095: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 626 KiB/s wr, 91 op/s 2023-09-10T01:14:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:56 smithi131 ceph-mon[103713]: pgmap v1096: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 163 op/s 2023-09-10T01:14:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:56 smithi044 ceph-mon[98676]: pgmap v1096: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 163 op/s 2023-09-10T01:14:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:56 smithi099 ceph-mon[102692]: pgmap v1096: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 163 op/s 2023-09-10T01:14:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:14:59 smithi131 ceph-mon[103713]: pgmap v1097: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 163 op/s 2023-09-10T01:14:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:14:59 smithi044 ceph-mon[98676]: pgmap v1097: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 163 op/s 2023-09-10T01:14:59.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:14:59 smithi099 ceph-mon[102692]: pgmap v1097: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 163 op/s 2023-09-10T01:15:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.632+0000 7fe464c8b700 1 -- 172.21.15.44:0/3156759164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe460106680 msgr2=0x7fe460106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.632+0000 7fe464c8b700 1 --2- 172.21.15.44:0/3156759164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe460106680 0x7fe460106a60 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fe4500099c0 tx=0x7fe450009cd0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.633+0000 7fe464c8b700 1 -- 172.21.15.44:0/3156759164 shutdown_connections 2023-09-10T01:15:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.633+0000 7fe464c8b700 1 --2- 172.21.15.44:0/3156759164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe460100f80 0x7fe46010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.633+0000 7fe464c8b700 1 --2- 172.21.15.44:0/3156759164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe460106680 0x7fe460106a60 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.633+0000 7fe464c8b700 1 --2- 172.21.15.44:0/3156759164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4601005c0 0x7fe460100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.633+0000 7fe464c8b700 1 -- 172.21.15.44:0/3156759164 >> 172.21.15.44:0/3156759164 conn(0x7fe4600fc0b0 msgr2=0x7fe4600fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.633+0000 7fe464c8b700 1 -- 172.21.15.44:0/3156759164 shutdown_connections 2023-09-10T01:15:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.633+0000 7fe464c8b700 1 -- 172.21.15.44:0/3156759164 wait complete. 2023-09-10T01:15:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.633+0000 7fe464c8b700 1 Processor -- start 2023-09-10T01:15:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.633+0000 7fe464c8b700 1 -- start start 2023-09-10T01:15:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe464c8b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4601005c0 0x7fe460072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe464c8b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe460100f80 0x7fe460073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe464c8b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe460106680 0x7fe4601a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe464c8b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4601a4c80 con 0x7fe4601005c0 2023-09-10T01:15:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe464c8b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4601a4dc0 con 0x7fe460106680 2023-09-10T01:15:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe464c8b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4601a4f30 con 0x7fe460100f80 2023-09-10T01:15:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe45e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4601005c0 0x7fe460072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe45e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4601005c0 0x7fe460072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57444/0 (socket says 172.21.15.44:57444) 2023-09-10T01:15:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe45e59c700 1 -- 172.21.15.44:0/250522406 learned_addr learned my addr 172.21.15.44:0/250522406 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe45ed9d700 1 --2- 172.21.15.44:0/250522406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe460106680 0x7fe4601a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe45dd9b700 1 --2- 172.21.15.44:0/250522406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe460100f80 0x7fe460073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe45e59c700 1 -- 172.21.15.44:0/250522406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe460100f80 msgr2=0x7fe460073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe45e59c700 1 --2- 172.21.15.44:0/250522406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe460100f80 0x7fe460073020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe45e59c700 1 -- 172.21.15.44:0/250522406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe460106680 msgr2=0x7fe4601a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe45e59c700 1 --2- 172.21.15.44:0/250522406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe460106680 0x7fe4601a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe45e59c700 1 -- 172.21.15.44:0/250522406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4500095f0 con 0x7fe4601005c0 2023-09-10T01:15:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe45ed9d700 1 --2- 172.21.15.44:0/250522406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe460106680 0x7fe4601a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:15:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.634+0000 7fe45dd9b700 1 --2- 172.21.15.44:0/250522406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe460100f80 0x7fe460073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:15:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.635+0000 7fe45e59c700 1 --2- 172.21.15.44:0/250522406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4601005c0 0x7fe460072ae0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fe450005910 tx=0x7fe450009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.635+0000 7fe44f7fe700 1 -- 172.21.15.44:0/250522406 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45001d070 con 0x7fe4601005c0 2023-09-10T01:15:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.635+0000 7fe44f7fe700 1 -- 172.21.15.44:0/250522406 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe450022470 con 0x7fe4601005c0 2023-09-10T01:15:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.635+0000 7fe464c8b700 1 -- 172.21.15.44:0/250522406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4601a51b0 con 0x7fe4601005c0 2023-09-10T01:15:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.635+0000 7fe44f7fe700 1 -- 172.21.15.44:0/250522406 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450017650 con 0x7fe4601005c0 2023-09-10T01:15:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.635+0000 7fe464c8b700 1 -- 172.21.15.44:0/250522406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4601a55d0 con 0x7fe4601005c0 2023-09-10T01:15:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.637+0000 7fe44f7fe700 1 -- 172.21.15.44:0/250522406 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fe4500225e0 con 0x7fe4601005c0 2023-09-10T01:15:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.638+0000 7fe44f7fe700 1 --2- 172.21.15.44:0/250522406 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4480a07b0 0x7fe4480a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.638+0000 7fe44f7fe700 1 -- 172.21.15.44:0/250522406 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fe4500cd3f0 con 0x7fe4601005c0 2023-09-10T01:15:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.638+0000 7fe45dd9b700 1 --2- 172.21.15.44:0/250522406 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4480a07b0 0x7fe4480a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.638+0000 7fe45dd9b700 1 --2- 172.21.15.44:0/250522406 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4480a07b0 0x7fe4480a2c70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe460074300 tx=0x7fe454009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.639+0000 7fe470e51b80 1 Processor -- start 2023-09-10T01:15:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.639+0000 7fe470e51b80 1 -- start start 2023-09-10T01:15:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.639+0000 7fe470e51b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf169e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.639+0000 7fe470e51b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf1698780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.639+0000 7fe470e51b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf16a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.639+0000 7fe470e51b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf15f16d0 con 0x562bf1698300 2023-09-10T01:15:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.639+0000 7fe470e51b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf15eeff0 con 0x562bf169e1a0 2023-09-10T01:15:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.639+0000 7fe470e51b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf15ea8e0 con 0x562bf1698cc0 2023-09-10T01:15:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.639+0000 7fe45dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf1698780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.639+0000 7fe45dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf1698780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57454/0 (socket says 172.21.15.44:57454) 2023-09-10T01:15:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.639+0000 7fe45dd9b700 1 -- 172.21.15.44:0/269024558 learned_addr learned my addr 172.21.15.44:0/269024558 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.639+0000 7fe45e59c700 1 --2- 172.21.15.44:0/269024558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf169e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.639+0000 7fe45ed9d700 1 --2- 172.21.15.44:0/269024558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf16a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.640+0000 7fe45dd9b700 1 -- 172.21.15.44:0/269024558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 msgr2=0x562bf16a4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.640+0000 7fe45dd9b700 1 --2- 172.21.15.44:0/269024558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf16a4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.640+0000 7fe45dd9b700 1 -- 172.21.15.44:0/269024558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 msgr2=0x562bf169e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.640+0000 7fe45dd9b700 1 --2- 172.21.15.44:0/269024558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf169e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.640+0000 7fe45dd9b700 1 -- 172.21.15.44:0/269024558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bf1416d80 con 0x562bf1698300 2023-09-10T01:15:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.640+0000 7fe45e59c700 1 --2- 172.21.15.44:0/269024558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf169e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:15:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.640+0000 7fe45dd9b700 1 --2- 172.21.15.44:0/269024558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf1698780 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fe4540043c0 tx=0x7fe454014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a007249a69ce13da server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.640+0000 7fe44cff9700 1 -- 172.21.15.44:0/269024558 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454003d10 con 0x562bf1698300 2023-09-10T01:15:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.640+0000 7fe44cff9700 1 -- 172.21.15.44:0/269024558 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe454003e70 con 0x562bf1698300 2023-09-10T01:15:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.640+0000 7fe44cff9700 1 -- 172.21.15.44:0/269024558 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45400e730 con 0x562bf1698300 2023-09-10T01:15:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.641+0000 7fe470e51b80 1 -- 172.21.15.44:0/269024558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 msgr2=0x562bf1698780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.641+0000 7fe470e51b80 1 --2- 172.21.15.44:0/269024558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf1698780 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fe4540043c0 tx=0x7fe454014000 comp rx=0 tx=0).stop 2023-09-10T01:15:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.641+0000 7fe470e51b80 1 -- 172.21.15.44:0/269024558 shutdown_connections 2023-09-10T01:15:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.641+0000 7fe470e51b80 1 --2- 172.21.15.44:0/269024558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf16a4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.641+0000 7fe470e51b80 1 --2- 172.21.15.44:0/269024558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf169e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.641+0000 7fe470e51b80 1 --2- 172.21.15.44:0/269024558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf1698780 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.641+0000 7fe470e51b80 1 -- 172.21.15.44:0/269024558 >> 172.21.15.44:0/269024558 conn(0x562bf160d030 msgr2=0x562bf1693560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.641+0000 7fe470e51b80 1 -- 172.21.15.44:0/269024558 shutdown_connections 2023-09-10T01:15:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.641+0000 7fe470e51b80 1 -- 172.21.15.44:0/269024558 wait complete. 2023-09-10T01:15:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.641+0000 7fe470e51b80 1 -- start start 2023-09-10T01:15:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.641+0000 7fe470e51b80 1 Processor -- start 2023-09-10T01:15:00.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe470e51b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf16a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe470e51b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf16a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe470e51b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf16a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe470e51b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf1588b10 con 0x562bf1698300 2023-09-10T01:15:00.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe470e51b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf1588810 con 0x562bf169e1a0 2023-09-10T01:15:00.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe470e51b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf146c8c0 con 0x562bf1698cc0 2023-09-10T01:15:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe470e51b80 10 client.0 _ll_register_callbacks cb 0x562bf14d2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:15:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe45e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf16a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe45e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf16a4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57458/0 (socket says 172.21.15.44:57458) 2023-09-10T01:15:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe45e59c700 1 -- 172.21.15.44:0/1928126155 learned_addr learned my addr 172.21.15.44:0/1928126155 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe45dd9b700 1 --2- 172.21.15.44:0/1928126155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf16a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe45ed9d700 1 --2- 172.21.15.44:0/1928126155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf16a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe45dd9b700 1 -- 172.21.15.44:0/1928126155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 msgr2=0x562bf16a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe45dd9b700 1 --2- 172.21.15.44:0/1928126155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf16a26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe45dd9b700 1 -- 172.21.15.44:0/1928126155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 msgr2=0x562bf16a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe45dd9b700 1 --2- 172.21.15.44:0/1928126155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf16a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe45dd9b700 1 -- 172.21.15.44:0/1928126155 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4601a55d0 con 0x562bf1698cc0 2023-09-10T01:15:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe45e59c700 1 --2- 172.21.15.44:0/1928126155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf16a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:15:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe45dd9b700 1 --2- 172.21.15.44:0/1928126155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf16a20c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe4540043c0 tx=0x7fe45401e620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1814947d7e27e9d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe445ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:15:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.642+0000 7fe445ffb700 1 -- 172.21.15.44:0/1928126155 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45402b070 con 0x562bf1698cc0 2023-09-10T01:15:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.643+0000 7fe470e51b80 1 -- 172.21.15.44:0/1928126155 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bf1542490 con 0x562bf1698cc0 2023-09-10T01:15:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.643+0000 7fe445ffb700 1 -- 172.21.15.44:0/1928126155 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(152..152 src has 1..152) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fe45401e9a0 con 0x562bf1698cc0 2023-09-10T01:15:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.643+0000 7fe445ffb700 1 -- 172.21.15.44:0/1928126155 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe45401eb60 con 0x562bf1698cc0 2023-09-10T01:15:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.643+0000 7fe470e51b80 10 client.29047 fetch_fsmap learned FSMap version 120 2023-09-10T01:15:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.643+0000 7fe470e51b80 1 -- 172.21.15.44:0/1928126155 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x7fe454017070 con 0x562bf1698cc0 2023-09-10T01:15:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.643+0000 7fe445ffb700 1 -- 172.21.15.44:0/1928126155 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562bf1542490 con 0x562bf1698cc0 2023-09-10T01:15:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.643+0000 7fe445ffb700 1 -- 172.21.15.44:0/1928126155 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fe454024020 con 0x562bf1698cc0 2023-09-10T01:15:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.643+0000 7fe470e51b80 10 client.29047 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:15:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.643+0000 7fe470e51b80 10 client.29047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:15:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.643+0000 7fe470e51b80 20 client.29047 populate_metadata read hostname 'smithi044' 2023-09-10T01:15:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.643+0000 7fe470e51b80 1 --2- 172.21.15.44:0/1928126155 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562bf16b2290 0x562bf16b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.643+0000 7fe470e51b80 4 client.29047 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:15:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.643+0000 7fe470e51b80 1 -- 172.21.15.44:0/1928126155 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bf15ff840 con 0x562bf16b2290 2023-09-10T01:15:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.643+0000 7fe45e59c700 1 --2- 172.21.15.44:0/1928126155 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562bf16b2290 0x562bf16b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.644+0000 7fe445ffb700 10 client.29047 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.644+0000 7fe45e59c700 1 --2- 172.21.15.44:0/1928126155 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562bf16b2290 0x562bf16b46b0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b52c488a0810369 server_cookie=ea1e45b32a6e8106 in_seq=0 out_seq=0 2023-09-10T01:15:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.644+0000 7fe445ffb700 1 -- 172.21.15.44:0/1928126155 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562bf15ff840 con 0x562bf16b2290 2023-09-10T01:15:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.644+0000 7fe445ffb700 10 client.29047 handle_command_reply: tid=0 2023-09-10T01:15:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.644+0000 7fe470e51b80 1 client.29047 shutdown 2023-09-10T01:15:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.645+0000 7fe470e51b80 1 -- 172.21.15.44:0/1928126155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 msgr2=0x562bf16a20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.645+0000 7fe470e51b80 1 --2- 172.21.15.44:0/1928126155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf16a20c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe4540043c0 tx=0x7fe45401e620 comp rx=0 tx=0).stop 2023-09-10T01:15:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.645+0000 7fe470e51b80 1 -- 172.21.15.44:0/1928126155 shutdown_connections 2023-09-10T01:15:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.645+0000 7fe470e51b80 1 --2- 172.21.15.44:0/1928126155 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562bf16b2290 0x562bf16b46b0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.645+0000 7fe470e51b80 1 --2- 172.21.15.44:0/1928126155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf16a26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.645+0000 7fe470e51b80 1 --2- 172.21.15.44:0/1928126155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf16a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.645+0000 7fe470e51b80 1 --2- 172.21.15.44:0/1928126155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf16a20c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.645+0000 7fe445ffb700 0 client.29047 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.645+0000 7fe470e51b80 1 -- 172.21.15.44:0/1928126155 >> 172.21.15.44:0/1928126155 conn(0x562bf160d030 msgr2=0x562bf16927a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:00.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.645+0000 7fe470e51b80 1 -- 172.21.15.44:0/1928126155 shutdown_connections 2023-09-10T01:15:00.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.645+0000 7fe470e51b80 1 -- 172.21.15.44:0/1928126155 wait complete. 2023-09-10T01:15:00.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.645+0000 7fe470e51b80 20 client.29047 trim_cache size 0 max 16384 2023-09-10T01:15:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.661+0000 7fe470e51b80 1 Processor -- start 2023-09-10T01:15:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.661+0000 7fe470e51b80 1 -- start start 2023-09-10T01:15:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.661+0000 7fe470e51b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf16adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.661+0000 7fe470e51b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf169c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.661+0000 7fe470e51b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf169c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.661+0000 7fe470e51b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf169ccc0 con 0x562bf1698300 2023-09-10T01:15:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.661+0000 7fe470e51b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf1699460 con 0x562bf169e1a0 2023-09-10T01:15:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.661+0000 7fe470e51b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf16ae1c0 con 0x562bf1698cc0 2023-09-10T01:15:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.662+0000 7fe45e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf16adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.662+0000 7fe45dd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf169c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.662+0000 7fe45dd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf169c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36118/0 (socket says 172.21.15.44:36118) 2023-09-10T01:15:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.662+0000 7fe45dd9b700 1 -- 172.21.15.44:0/6380316 learned_addr learned my addr 172.21.15.44:0/6380316 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.662+0000 7fe45ed9d700 1 --2- 172.21.15.44:0/6380316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf169c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.662+0000 7fe45e59c700 1 -- 172.21.15.44:0/6380316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 msgr2=0x562bf169c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.662+0000 7fe45e59c700 1 --2- 172.21.15.44:0/6380316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf169c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.662+0000 7fe45e59c700 1 -- 172.21.15.44:0/6380316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 msgr2=0x562bf169c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.662+0000 7fe45e59c700 1 --2- 172.21.15.44:0/6380316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf169c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.662+0000 7fe45e59c700 1 -- 172.21.15.44:0/6380316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bf13ffc00 con 0x562bf1698300 2023-09-10T01:15:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.662+0000 7fe45e59c700 1 --2- 172.21.15.44:0/6380316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf16adc80 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fe4500ca7f0 tx=0x7fe450005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd25661c1017e92 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.662+0000 7fe44d7fa700 1 -- 172.21.15.44:0/6380316 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45009e170 con 0x562bf1698300 2023-09-10T01:15:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.662+0000 7fe44d7fa700 1 -- 172.21.15.44:0/6380316 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4500cae90 con 0x562bf1698300 2023-09-10T01:15:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.662+0000 7fe44d7fa700 1 -- 172.21.15.44:0/6380316 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4500989f0 con 0x562bf1698300 2023-09-10T01:15:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.663+0000 7fe470e51b80 1 -- 172.21.15.44:0/6380316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 msgr2=0x562bf16adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.663+0000 7fe470e51b80 1 --2- 172.21.15.44:0/6380316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf16adc80 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fe4500ca7f0 tx=0x7fe450005c60 comp rx=0 tx=0).stop 2023-09-10T01:15:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.663+0000 7fe470e51b80 1 -- 172.21.15.44:0/6380316 shutdown_connections 2023-09-10T01:15:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.663+0000 7fe470e51b80 1 --2- 172.21.15.44:0/6380316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf169c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.663+0000 7fe470e51b80 1 --2- 172.21.15.44:0/6380316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf16adc80 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.663+0000 7fe470e51b80 1 --2- 172.21.15.44:0/6380316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf169c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.663+0000 7fe470e51b80 1 -- 172.21.15.44:0/6380316 >> 172.21.15.44:0/6380316 conn(0x562bf160d030 msgr2=0x562bf16b9010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.663+0000 7fe470e51b80 1 -- 172.21.15.44:0/6380316 shutdown_connections 2023-09-10T01:15:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.663+0000 7fe470e51b80 1 -- 172.21.15.44:0/6380316 wait complete. 2023-09-10T01:15:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.664+0000 7fe470e51b80 1 -- start start 2023-09-10T01:15:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.664+0000 7fe470e51b80 1 Processor -- start 2023-09-10T01:15:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.664+0000 7fe470e51b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf16a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.664+0000 7fe470e51b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf16a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.664+0000 7fe470e51b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf16a7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.664+0000 7fe470e51b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf17050c0 con 0x562bf1698300 2023-09-10T01:15:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.664+0000 7fe470e51b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf1705200 con 0x562bf169e1a0 2023-09-10T01:15:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.664+0000 7fe470e51b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf1705340 con 0x562bf1698cc0 2023-09-10T01:15:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.664+0000 7fe470e51b80 10 client.0 _ll_register_callbacks cb 0x7fe450022ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:15:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.664+0000 7fe45dd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf16a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.664+0000 7fe45e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf16a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.664+0000 7fe45dd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf16a70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36124/0 (socket says 172.21.15.44:36124) 2023-09-10T01:15:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.664+0000 7fe45dd9b700 1 -- 172.21.15.44:0/2085809645 learned_addr learned my addr 172.21.15.44:0/2085809645 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.665+0000 7fe45ed9d700 1 --2- 172.21.15.44:0/2085809645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf16a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.665+0000 7fe45dd9b700 1 -- 172.21.15.44:0/2085809645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 msgr2=0x562bf16a7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.665+0000 7fe45dd9b700 1 --2- 172.21.15.44:0/2085809645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf16a7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.665+0000 7fe45dd9b700 1 -- 172.21.15.44:0/2085809645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 msgr2=0x562bf16a6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.665+0000 7fe45dd9b700 1 --2- 172.21.15.44:0/2085809645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf16a6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.665+0000 7fe45dd9b700 1 -- 172.21.15.44:0/2085809645 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4601a51b0 con 0x562bf1698cc0 2023-09-10T01:15:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.665+0000 7fe45dd9b700 1 --2- 172.21.15.44:0/2085809645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf16a70c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe4540043c0 tx=0x7fe45401ebd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c919e1a260056a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.665+0000 7fe445ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:15:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.665+0000 7fe445ffb700 1 -- 172.21.15.44:0/2085809645 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45402b070 con 0x562bf1698cc0 2023-09-10T01:15:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.665+0000 7fe470e51b80 1 -- 172.21.15.44:0/2085809645 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe454003e70 con 0x562bf1698cc0 2023-09-10T01:15:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.665+0000 7fe445ffb700 1 -- 172.21.15.44:0/2085809645 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(152..152 src has 1..152) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fe454026a20 con 0x562bf1698cc0 2023-09-10T01:15:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.665+0000 7fe445ffb700 1 -- 172.21.15.44:0/2085809645 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe45401ed80 con 0x562bf1698cc0 2023-09-10T01:15:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.665+0000 7fe470e51b80 10 client.29053 fetch_fsmap learned FSMap version 120 2023-09-10T01:15:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.666+0000 7fe470e51b80 1 -- 172.21.15.44:0/2085809645 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x562bf143d1f0 con 0x562bf1698cc0 2023-09-10T01:15:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.666+0000 7fe445ffb700 1 -- 172.21.15.44:0/2085809645 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe454003e70 con 0x562bf1698cc0 2023-09-10T01:15:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.666+0000 7fe445ffb700 1 -- 172.21.15.44:0/2085809645 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fe454024070 con 0x562bf1698cc0 2023-09-10T01:15:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.666+0000 7fe470e51b80 10 client.29053 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:15:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.666+0000 7fe470e51b80 10 client.29053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:15:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.666+0000 7fe470e51b80 20 client.29053 populate_metadata read hostname 'smithi044' 2023-09-10T01:15:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.666+0000 7fe470e51b80 1 --2- 172.21.15.44:0/2085809645 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562bf1706690 0x562bf1708ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.666+0000 7fe470e51b80 4 client.29053 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:15:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.666+0000 7fe470e51b80 1 -- 172.21.15.44:0/2085809645 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562bf15e38c0 con 0x562bf1706690 2023-09-10T01:15:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.666+0000 7fe45e59c700 1 --2- 172.21.15.44:0/2085809645 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562bf1706690 0x562bf1708ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.667+0000 7fe445ffb700 10 client.29053 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:00.667+0000 7fe45e59c700 1 --2- 172.21.15.44:0/2085809645 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562bf1706690 0x562bf1708ab0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cb9f9acb8867499 server_cookie=c24b371726f2096f in_seq=0 out_seq=0 2023-09-10T01:15:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:01 smithi131 ceph-mon[103713]: pgmap v1098: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 510 KiB/s wr, 73 op/s 2023-09-10T01:15:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:01 smithi044 ceph-mon[98676]: pgmap v1098: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 510 KiB/s wr, 73 op/s 2023-09-10T01:15:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:01 smithi099 ceph-mon[102692]: pgmap v1098: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 510 KiB/s wr, 73 op/s 2023-09-10T01:15:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:03 smithi131 ceph-mon[103713]: pgmap v1099: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 981 KiB/s wr, 140 op/s 2023-09-10T01:15:03.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:03 smithi044 ceph-mon[98676]: pgmap v1099: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 981 KiB/s wr, 140 op/s 2023-09-10T01:15:03.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:03 smithi099 ceph-mon[102692]: pgmap v1099: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 981 KiB/s wr, 140 op/s 2023-09-10T01:15:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.441+0000 7fe445ffb700 1 -- 172.21.15.44:0/2085809645 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562bf15e38c0 con 0x562bf1706690 2023-09-10T01:15:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.441+0000 7fe445ffb700 10 client.29053 handle_command_reply: tid=0 2023-09-10T01:15:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.441+0000 7fe470e51b80 1 client.29053 shutdown 2023-09-10T01:15:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.441+0000 7fe470e51b80 1 -- 172.21.15.44:0/2085809645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 msgr2=0x562bf16a70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.441+0000 7fe470e51b80 1 --2- 172.21.15.44:0/2085809645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf16a70c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe4540043c0 tx=0x7fe45401ebd0 comp rx=0 tx=0).stop 2023-09-10T01:15:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.441+0000 7fe470e51b80 1 -- 172.21.15.44:0/2085809645 shutdown_connections 2023-09-10T01:15:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.441+0000 7fe470e51b80 1 --2- 172.21.15.44:0/2085809645 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562bf1706690 0x562bf1708ab0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.441+0000 7fe470e51b80 1 --2- 172.21.15.44:0/2085809645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bf169e1a0 0x562bf16a7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.441+0000 7fe470e51b80 1 --2- 172.21.15.44:0/2085809645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562bf1698300 0x562bf16a6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.441+0000 7fe445ffb700 0 client.29053 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.441+0000 7fe470e51b80 1 --2- 172.21.15.44:0/2085809645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bf1698cc0 0x562bf16a70c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.441+0000 7fe470e51b80 1 -- 172.21.15.44:0/2085809645 >> 172.21.15.44:0/2085809645 conn(0x562bf160d030 msgr2=0x562bf169d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.441+0000 7fe470e51b80 1 -- 172.21.15.44:0/2085809645 shutdown_connections 2023-09-10T01:15:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.441+0000 7fe470e51b80 1 -- 172.21.15.44:0/2085809645 wait complete. 2023-09-10T01:15:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.442+0000 7fe470e51b80 20 client.29053 trim_cache size 0 max 16384 2023-09-10T01:15:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:15:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:15:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:15:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:15:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.446+0000 7fe464c8b700 1 -- 172.21.15.44:0/250522406 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4480a07b0 msgr2=0x7fe4480a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.446+0000 7fe464c8b700 1 --2- 172.21.15.44:0/250522406 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4480a07b0 0x7fe4480a2c70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe460074300 tx=0x7fe454009260 comp rx=0 tx=0).stop 2023-09-10T01:15:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.446+0000 7fe464c8b700 1 -- 172.21.15.44:0/250522406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4601005c0 msgr2=0x7fe460072ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.446+0000 7fe464c8b700 1 --2- 172.21.15.44:0/250522406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4601005c0 0x7fe460072ae0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fe450005910 tx=0x7fe450009580 comp rx=0 tx=0).stop 2023-09-10T01:15:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.447+0000 7fe464c8b700 1 -- 172.21.15.44:0/250522406 shutdown_connections 2023-09-10T01:15:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.447+0000 7fe464c8b700 1 --2- 172.21.15.44:0/250522406 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe4480a07b0 0x7fe4480a2c70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.447+0000 7fe464c8b700 1 --2- 172.21.15.44:0/250522406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe460106680 0x7fe4601a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.447+0000 7fe464c8b700 1 --2- 172.21.15.44:0/250522406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4601005c0 0x7fe460072ae0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.447+0000 7fe464c8b700 1 --2- 172.21.15.44:0/250522406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe460100f80 0x7fe460073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.447+0000 7fe464c8b700 1 -- 172.21.15.44:0/250522406 >> 172.21.15.44:0/250522406 conn(0x7fe4600fc0b0 msgr2=0x7fe4601042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.447+0000 7fe464c8b700 1 -- 172.21.15.44:0/250522406 shutdown_connections 2023-09-10T01:15:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:03.447+0000 7fe464c8b700 1 -- 172.21.15.44:0/250522406 wait complete. 2023-09-10T01:15:04.559 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: *** Caught signal (Segmentation fault) ** 2023-09-10T01:15:04.559 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: in thread 7ff556a24700 thread_name:md_log_replay 2023-09-10T01:15:04.560 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:15:04.560 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7ff5675d3cf0] 2023-09-10T01:15:04.560 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x55d50382e30f] 2023-09-10T01:15:04.560 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55d503830543] 2023-09-10T01:15:04.560 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 4: (MDLog::_replay_thread()+0x647) [0x55d503899e97] 2023-09-10T01:15:04.561 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55d5038a3381] 2023-09-10T01:15:04.561 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff5675c91ca] 2023-09-10T01:15:04.561 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 7: clone() 2023-09-10T01:15:04.562 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 2023-09-10T01:15:04.293+0000 7ff556a24700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:15:04.562 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: in thread 7ff556a24700 thread_name:md_log_replay 2023-09-10T01:15:04.562 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 2023-09-10T01:15:04.562 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:15:04.563 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7ff5675d3cf0] 2023-09-10T01:15:04.563 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x55d50382e30f] 2023-09-10T01:15:04.563 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55d503830543] 2023-09-10T01:15:04.563 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 4: (MDLog::_replay_thread()+0x647) [0x55d503899e97] 2023-09-10T01:15:04.564 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55d5038a3381] 2023-09-10T01:15:04.564 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff5675c91ca] 2023-09-10T01:15:04.564 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 7: clone() 2023-09-10T01:15:04.564 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:15:04.565 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 2023-09-10T01:15:04.565 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 0> 2023-09-10T01:15:04.293+0000 7ff556a24700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:15:04.565 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: in thread 7ff556a24700 thread_name:md_log_replay 2023-09-10T01:15:04.565 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 2023-09-10T01:15:04.566 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:15:04.566 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7ff5675d3cf0] 2023-09-10T01:15:04.566 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x55d50382e30f] 2023-09-10T01:15:04.566 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55d503830543] 2023-09-10T01:15:04.567 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 4: (MDLog::_replay_thread()+0x647) [0x55d503899e97] 2023-09-10T01:15:04.567 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55d5038a3381] 2023-09-10T01:15:04.567 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff5675c91ca] 2023-09-10T01:15:04.567 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 7: clone() 2023-09-10T01:15:04.567 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:15:04.567 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 2023-09-10T01:15:04.568 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 0> 2023-09-10T01:15:04.293+0000 7ff556a24700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:15:04.568 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: in thread 7ff556a24700 thread_name:md_log_replay 2023-09-10T01:15:04.568 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 2023-09-10T01:15:04.568 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:15:04.568 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7ff5675d3cf0] 2023-09-10T01:15:04.569 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x55d50382e30f] 2023-09-10T01:15:04.569 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55d503830543] 2023-09-10T01:15:04.569 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 4: (MDLog::_replay_thread()+0x647) [0x55d503899e97] 2023-09-10T01:15:04.569 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55d5038a3381] 2023-09-10T01:15:04.569 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff5675c91ca] 2023-09-10T01:15:04.570 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 7: clone() 2023-09-10T01:15:04.570 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:15:04.570 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[194048]: 2023-09-10T01:15:04.570 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:04 smithi044 ceph-mon[98676]: pgmap v1100: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 817 KiB/s wr, 117 op/s 2023-09-10T01:15:04.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:04 smithi131 ceph-mon[103713]: pgmap v1100: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 817 KiB/s wr, 117 op/s 2023-09-10T01:15:04.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:04 smithi099 ceph-mon[102692]: pgmap v1100: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 817 KiB/s wr, 117 op/s 2023-09-10T01:15:05.305 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:04 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Main process exited, code=exited, status=139/n/a 2023-09-10T01:15:05.305 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:05 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Failed with result 'exit-code'. 2023-09-10T01:15:05.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:05 smithi131 ceph-mon[103713]: osdmap e153: 12 total, 12 up, 12 in 2023-09-10T01:15:05.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:05 smithi044 ceph-mon[98676]: osdmap e153: 12 total, 12 up, 12 in 2023-09-10T01:15:05.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:05 smithi099 ceph-mon[102692]: osdmap e153: 12 total, 12 up, 12 in 2023-09-10T01:15:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:06 smithi131 ceph-mon[103713]: pgmap v1102: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2023-09-10T01:15:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:06 smithi044 ceph-mon[98676]: pgmap v1102: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2023-09-10T01:15:07.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:06 smithi099 ceph-mon[102692]: pgmap v1102: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2023-09-10T01:15:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:09 smithi131 ceph-mon[103713]: pgmap v1103: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2023-09-10T01:15:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:09 smithi044 ceph-mon[98676]: pgmap v1103: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2023-09-10T01:15:09.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:09 smithi099 ceph-mon[102692]: pgmap v1103: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2023-09-10T01:15:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:11 smithi131 ceph-mon[103713]: pgmap v1104: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2023-09-10T01:15:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:11 smithi131 ceph-mon[103713]: osdmap e154: 12 total, 12 up, 12 in 2023-09-10T01:15:11.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:11 smithi044 ceph-mon[98676]: pgmap v1104: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2023-09-10T01:15:11.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:11 smithi044 ceph-mon[98676]: osdmap e154: 12 total, 12 up, 12 in 2023-09-10T01:15:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:11 smithi099 ceph-mon[102692]: pgmap v1104: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2023-09-10T01:15:11.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:11 smithi099 ceph-mon[102692]: osdmap e154: 12 total, 12 up, 12 in 2023-09-10T01:15:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:13 smithi131 ceph-mon[103713]: pgmap v1106: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 236 op/s 2023-09-10T01:15:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:13 smithi044 ceph-mon[98676]: pgmap v1106: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 236 op/s 2023-09-10T01:15:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:13 smithi099 ceph-mon[102692]: pgmap v1106: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 236 op/s 2023-09-10T01:15:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:15 smithi131 ceph-mon[103713]: pgmap v1107: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 197 op/s 2023-09-10T01:15:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:15 smithi044 ceph-mon[98676]: pgmap v1107: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 197 op/s 2023-09-10T01:15:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:15 smithi099 ceph-mon[102692]: pgmap v1107: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 197 op/s 2023-09-10T01:15:15.805 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:15 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Service RestartSec=10s expired, scheduling restart. 2023-09-10T01:15:15.806 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:15 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Scheduled restart job, restart counter is at 3. 2023-09-10T01:15:15.806 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:15 smithi044 systemd[1]: Stopped Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:15:15.806 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:15 smithi044 systemd[1]: Starting Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:15:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.561+0000 7efc6c687700 1 -- 172.21.15.44:0/4132863535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc641055b0 msgr2=0x7efc64105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.561+0000 7efc6c687700 1 --2- 172.21.15.44:0/4132863535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc641055b0 0x7efc64105990 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7efc600099c0 tx=0x7efc60009cd0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.561+0000 7efc6c687700 1 -- 172.21.15.44:0/4132863535 shutdown_connections 2023-09-10T01:15:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.561+0000 7efc6c687700 1 --2- 172.21.15.44:0/4132863535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc64100120 0x7efc6410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.561+0000 7efc6c687700 1 --2- 172.21.15.44:0/4132863535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc641055b0 0x7efc64105990 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.561+0000 7efc6c687700 1 --2- 172.21.15.44:0/4132863535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc640ff760 0x7efc640ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.561+0000 7efc6c687700 1 -- 172.21.15.44:0/4132863535 >> 172.21.15.44:0/4132863535 conn(0x7efc64074f30 msgr2=0x7efc64075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.561+0000 7efc6c687700 1 -- 172.21.15.44:0/4132863535 shutdown_connections 2023-09-10T01:15:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.561+0000 7efc6c687700 1 -- 172.21.15.44:0/4132863535 wait complete. 2023-09-10T01:15:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.562+0000 7efc6c687700 1 Processor -- start 2023-09-10T01:15:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.562+0000 7efc6c687700 1 -- start start 2023-09-10T01:15:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.562+0000 7efc6c687700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc640ff760 0x7efc64195390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.562+0000 7efc6c687700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc64100120 0x7efc641958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.562+0000 7efc6c687700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc641055b0 0x7efc64199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.562+0000 7efc6c687700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6419a580 con 0x7efc640ff760 2023-09-10T01:15:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.562+0000 7efc6c687700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6419a6f0 con 0x7efc64100120 2023-09-10T01:15:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.562+0000 7efc6c687700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6419a8b0 con 0x7efc641055b0 2023-09-10T01:15:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.562+0000 7efc6a423700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc640ff760 0x7efc64195390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.562+0000 7efc6a423700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc640ff760 0x7efc64195390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57486/0 (socket says 172.21.15.44:57486) 2023-09-10T01:15:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.562+0000 7efc6a423700 1 -- 172.21.15.44:0/1163092480 learned_addr learned my addr 172.21.15.44:0/1163092480 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.562+0000 7efc69c22700 1 --2- 172.21.15.44:0/1163092480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc64100120 0x7efc641958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.562+0000 7efc6ac24700 1 --2- 172.21.15.44:0/1163092480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc641055b0 0x7efc64199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.563+0000 7efc6a423700 1 -- 172.21.15.44:0/1163092480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc641055b0 msgr2=0x7efc64199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.563+0000 7efc6a423700 1 --2- 172.21.15.44:0/1163092480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc641055b0 0x7efc64199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.563+0000 7efc6a423700 1 -- 172.21.15.44:0/1163092480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc64100120 msgr2=0x7efc641958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.563+0000 7efc6a423700 1 --2- 172.21.15.44:0/1163092480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc64100120 0x7efc641958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.563+0000 7efc6a423700 1 -- 172.21.15.44:0/1163092480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc600095f0 con 0x7efc640ff760 2023-09-10T01:15:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.563+0000 7efc6a423700 1 --2- 172.21.15.44:0/1163092480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc640ff760 0x7efc64195390 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7efc60009fd0 tx=0x7efc60005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.563+0000 7efc5b7fe700 1 -- 172.21.15.44:0/1163092480 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6000fb20 con 0x7efc640ff760 2023-09-10T01:15:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.563+0000 7efc5b7fe700 1 -- 172.21.15.44:0/1163092480 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc60004ca0 con 0x7efc640ff760 2023-09-10T01:15:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.563+0000 7efc6c687700 1 -- 172.21.15.44:0/1163092480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc6419ab30 con 0x7efc640ff760 2023-09-10T01:15:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.563+0000 7efc5b7fe700 1 -- 172.21.15.44:0/1163092480 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc600055b0 con 0x7efc640ff760 2023-09-10T01:15:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.563+0000 7efc6c687700 1 -- 172.21.15.44:0/1163092480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc64067bd0 con 0x7efc640ff760 2023-09-10T01:15:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.565+0000 7efc5b7fe700 1 -- 172.21.15.44:0/1163092480 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7efc6000fc80 con 0x7efc640ff760 2023-09-10T01:15:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.566+0000 7efc5b7fe700 1 --2- 172.21.15.44:0/1163092480 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efc500a07b0 0x7efc500a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.566+0000 7efc5b7fe700 1 -- 172.21.15.44:0/1163092480 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 9540+0+0 (secure 0 0 0) 0x7efc600cfb50 con 0x7efc640ff760 2023-09-10T01:15:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.566+0000 7efc69c22700 1 --2- 172.21.15.44:0/1163092480 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efc500a07b0 0x7efc500a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.566+0000 7efc69c22700 1 --2- 172.21.15.44:0/1163092480 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efc500a07b0 0x7efc500a2c70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efc64196bb0 tx=0x7efc5400a380 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc7884db80 1 Processor -- start 2023-09-10T01:15:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc7884db80 1 -- start start 2023-09-10T01:15:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc7884db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdec11a0 0x55e6bdec1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc7884db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdebb300 0x55e6bdebb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc7884db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdebbcc0 0x55e6bdec7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc7884db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bde146d0 con 0x55e6bdebbcc0 2023-09-10T01:15:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc7884db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bde11ff0 con 0x55e6bdec11a0 2023-09-10T01:15:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc7884db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bde0d8e0 con 0x55e6bdebb300 2023-09-10T01:15:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc6ac24700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdebbcc0 0x55e6bdec7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc6ac24700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdebbcc0 0x55e6bdec7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57498/0 (socket says 172.21.15.44:57498) 2023-09-10T01:15:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc6ac24700 1 -- 172.21.15.44:0/2174502321 learned_addr learned my addr 172.21.15.44:0/2174502321 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc6a423700 1 --2- 172.21.15.44:0/2174502321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdec11a0 0x55e6bdec1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc69c22700 1 --2- 172.21.15.44:0/2174502321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdebb300 0x55e6bdebb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc6ac24700 1 -- 172.21.15.44:0/2174502321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdebb300 msgr2=0x55e6bdebb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc6ac24700 1 --2- 172.21.15.44:0/2174502321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdebb300 0x55e6bdebb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc6ac24700 1 -- 172.21.15.44:0/2174502321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdec11a0 msgr2=0x55e6bdec1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.567+0000 7efc6ac24700 1 --2- 172.21.15.44:0/2174502321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdec11a0 0x55e6bdec1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.568+0000 7efc6ac24700 1 -- 172.21.15.44:0/2174502321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6bdc39d80 con 0x55e6bdebbcc0 2023-09-10T01:15:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.568+0000 7efc6ac24700 1 --2- 172.21.15.44:0/2174502321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdebbcc0 0x55e6bdec7f30 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7efc5c011680 tx=0x7efc5c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4e8bec805135cc server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.568+0000 7efc58ff9700 1 -- 172.21.15.44:0/2174502321 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c00e040 con 0x55e6bdebbcc0 2023-09-10T01:15:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.568+0000 7efc58ff9700 1 -- 172.21.15.44:0/2174502321 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc5c00f040 con 0x55e6bdebbcc0 2023-09-10T01:15:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.568+0000 7efc58ff9700 1 -- 172.21.15.44:0/2174502321 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c004730 con 0x55e6bdebbcc0 2023-09-10T01:15:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.568+0000 7efc7884db80 1 -- 172.21.15.44:0/2174502321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdebbcc0 msgr2=0x55e6bdec7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.568+0000 7efc7884db80 1 --2- 172.21.15.44:0/2174502321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdebbcc0 0x55e6bdec7f30 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7efc5c011680 tx=0x7efc5c009920 comp rx=0 tx=0).stop 2023-09-10T01:15:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.569+0000 7efc7884db80 1 -- 172.21.15.44:0/2174502321 shutdown_connections 2023-09-10T01:15:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.569+0000 7efc7884db80 1 --2- 172.21.15.44:0/2174502321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdebbcc0 0x55e6bdec7f30 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.569+0000 7efc7884db80 1 --2- 172.21.15.44:0/2174502321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdec11a0 0x55e6bdec1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.569+0000 7efc7884db80 1 --2- 172.21.15.44:0/2174502321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdebb300 0x55e6bdebb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.569+0000 7efc7884db80 1 -- 172.21.15.44:0/2174502321 >> 172.21.15.44:0/2174502321 conn(0x55e6bde30030 msgr2=0x55e6bdeb6560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.569+0000 7efc7884db80 1 -- 172.21.15.44:0/2174502321 shutdown_connections 2023-09-10T01:15:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.569+0000 7efc7884db80 1 -- 172.21.15.44:0/2174502321 wait complete. 2023-09-10T01:15:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.569+0000 7efc7884db80 1 -- start start 2023-09-10T01:15:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.569+0000 7efc7884db80 1 Processor -- start 2023-09-10T01:15:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.569+0000 7efc7884db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebb300 0x55e6bdec7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.569+0000 7efc7884db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdebbcc0 0x55e6bdec4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.569+0000 7efc7884db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdec11a0 0x55e6bdec54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.569+0000 7efc7884db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bddabb10 con 0x55e6bdec11a0 2023-09-10T01:15:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.569+0000 7efc7884db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bddab810 con 0x55e6bdebb300 2023-09-10T01:15:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.569+0000 7efc7884db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bdc8f8c0 con 0x55e6bdebbcc0 2023-09-10T01:15:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.569+0000 7efc7884db80 10 client.0 _ll_register_callbacks cb 0x55e6bdcf5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:15:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc6ac24700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdec11a0 0x55e6bdec54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc6a423700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebb300 0x55e6bdec7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc6a423700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebb300 0x55e6bdec7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:59240/0 (socket says 172.21.15.44:59240) 2023-09-10T01:15:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc6a423700 1 -- 172.21.15.44:0/728900274 learned_addr learned my addr 172.21.15.44:0/728900274 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc69c22700 1 --2- 172.21.15.44:0/728900274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdebbcc0 0x55e6bdec4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc6a423700 1 -- 172.21.15.44:0/728900274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdebbcc0 msgr2=0x55e6bdec4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc6a423700 1 --2- 172.21.15.44:0/728900274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdebbcc0 0x55e6bdec4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc6a423700 1 -- 172.21.15.44:0/728900274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdec11a0 msgr2=0x55e6bdec54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc6a423700 1 --2- 172.21.15.44:0/728900274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdec11a0 0x55e6bdec54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc6a423700 1 -- 172.21.15.44:0/728900274 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5c011330 con 0x55e6bdebb300 2023-09-10T01:15:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc6ac24700 1 --2- 172.21.15.44:0/728900274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdec11a0 0x55e6bdec54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:15:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc6a423700 1 --2- 172.21.15.44:0/728900274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebb300 0x55e6bdec7d70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efc600095c0 tx=0x7efc6009c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae4ca8ca9c8b2a03 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:15:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc4dffb700 1 -- 172.21.15.44:0/728900274 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6009cc90 con 0x55e6bdebb300 2023-09-10T01:15:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc4dffb700 1 -- 172.21.15.44:0/728900274 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 9540+0+0 (secure 0 0 0) 0x7efc600a0ac0 con 0x55e6bdebb300 2023-09-10T01:15:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc7884db80 1 -- 172.21.15.44:0/728900274 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6bdd65490 con 0x55e6bdebb300 2023-09-10T01:15:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc4dffb700 1 -- 172.21.15.44:0/728900274 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc600a4c70 con 0x55e6bdebb300 2023-09-10T01:15:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc7884db80 10 client.29117 fetch_fsmap learned FSMap version 120 2023-09-10T01:15:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.570+0000 7efc7884db80 1 -- 172.21.15.44:0/728900274 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x55e6bdc22c00 con 0x55e6bdebb300 2023-09-10T01:15:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.571+0000 7efc4dffb700 1 -- 172.21.15.44:0/728900274 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e6bdd65490 con 0x55e6bdebb300 2023-09-10T01:15:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.571+0000 7efc4dffb700 1 -- 172.21.15.44:0/728900274 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7efc60098a80 con 0x55e6bdebb300 2023-09-10T01:15:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.571+0000 7efc7884db80 10 client.29117 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:15:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.571+0000 7efc7884db80 10 client.29117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:15:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.571+0000 7efc7884db80 20 client.29117 populate_metadata read hostname 'smithi044' 2023-09-10T01:15:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.571+0000 7efc7884db80 1 --2- 172.21.15.44:0/728900274 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e6bded4ff0 0x55e6bded7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.571+0000 7efc7884db80 4 client.29117 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:15:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.571+0000 7efc7884db80 1 -- 172.21.15.44:0/728900274 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6bde22840 con 0x55e6bded4ff0 2023-09-10T01:15:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.571+0000 7efc69c22700 1 --2- 172.21.15.44:0/728900274 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e6bded4ff0 0x55e6bded7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.571+0000 7efc4dffb700 10 client.29117 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.571+0000 7efc69c22700 1 --2- 172.21.15.44:0/728900274 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e6bded4ff0 0x55e6bded7410 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd427cd0bb1add55 server_cookie=b29bbdf3d33b88ee in_seq=0 out_seq=0 2023-09-10T01:15:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.572+0000 7efc4dffb700 1 -- 172.21.15.44:0/728900274 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e6bde22840 con 0x55e6bded4ff0 2023-09-10T01:15:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.572+0000 7efc4dffb700 10 client.29117 handle_command_reply: tid=0 2023-09-10T01:15:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.572+0000 7efc7884db80 1 client.29117 shutdown 2023-09-10T01:15:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.572+0000 7efc7884db80 1 -- 172.21.15.44:0/728900274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebb300 msgr2=0x55e6bdec7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.572+0000 7efc7884db80 1 --2- 172.21.15.44:0/728900274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebb300 0x55e6bdec7d70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efc600095c0 tx=0x7efc6009c8c0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.572+0000 7efc7884db80 1 -- 172.21.15.44:0/728900274 shutdown_connections 2023-09-10T01:15:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.572+0000 7efc7884db80 1 --2- 172.21.15.44:0/728900274 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e6bded4ff0 0x55e6bded7410 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.572+0000 7efc7884db80 1 --2- 172.21.15.44:0/728900274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdec11a0 0x55e6bdec54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.572+0000 7efc4dffb700 0 client.29117 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.572+0000 7efc7884db80 1 --2- 172.21.15.44:0/728900274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebb300 0x55e6bdec7d70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.572+0000 7efc7884db80 1 --2- 172.21.15.44:0/728900274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdebbcc0 0x55e6bdec4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.572+0000 7efc7884db80 1 -- 172.21.15.44:0/728900274 >> 172.21.15.44:0/728900274 conn(0x55e6bde30030 msgr2=0x55e6bdeb57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.572+0000 7efc7884db80 1 -- 172.21.15.44:0/728900274 shutdown_connections 2023-09-10T01:15:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.572+0000 7efc7884db80 1 -- 172.21.15.44:0/728900274 wait complete. 2023-09-10T01:15:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.572+0000 7efc7884db80 20 client.29117 trim_cache size 0 max 16384 2023-09-10T01:15:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc7884db80 1 Processor -- start 2023-09-10T01:15:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc7884db80 1 -- start start 2023-09-10T01:15:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc7884db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdebb300 0x55e6bdec06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc7884db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebbcc0 0x55e6bdecc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc7884db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdec11a0 0x55e6bdec78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc7884db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bdd65490 con 0x55e6bdec11a0 2023-09-10T01:15:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc7884db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bdebc140 con 0x55e6bdebbcc0 2023-09-10T01:15:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc7884db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bdeb08a0 con 0x55e6bdebb300 2023-09-10T01:15:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc69c22700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebbcc0 0x55e6bdecc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc6ac24700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdec11a0 0x55e6bdec78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc69c22700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebbcc0 0x55e6bdecc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:59250/0 (socket says 172.21.15.44:59250) 2023-09-10T01:15:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc69c22700 1 -- 172.21.15.44:0/3010727218 learned_addr learned my addr 172.21.15.44:0/3010727218 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc6a423700 1 --2- 172.21.15.44:0/3010727218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdebb300 0x55e6bdec06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc69c22700 1 -- 172.21.15.44:0/3010727218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdebb300 msgr2=0x55e6bdec06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc69c22700 1 --2- 172.21.15.44:0/3010727218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdebb300 0x55e6bdec06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc69c22700 1 -- 172.21.15.44:0/3010727218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdec11a0 msgr2=0x55e6bdec78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc69c22700 1 --2- 172.21.15.44:0/3010727218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdec11a0 0x55e6bdec78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.588+0000 7efc69c22700 1 -- 172.21.15.44:0/3010727218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6bdc601f0 con 0x55e6bdebbcc0 2023-09-10T01:15:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.589+0000 7efc69c22700 1 --2- 172.21.15.44:0/3010727218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebbcc0 0x55e6bdecc520 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55e6bdeb6060 tx=0x7efc54012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14bb4be155f8aa9a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.589+0000 7efc597fa700 1 -- 172.21.15.44:0/3010727218 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc54023040 con 0x55e6bdebbcc0 2023-09-10T01:15:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.589+0000 7efc597fa700 1 -- 172.21.15.44:0/3010727218 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc540285a0 con 0x55e6bdebbcc0 2023-09-10T01:15:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.589+0000 7efc597fa700 1 -- 172.21.15.44:0/3010727218 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc54010690 con 0x55e6bdebbcc0 2023-09-10T01:15:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.589+0000 7efc7884db80 1 -- 172.21.15.44:0/3010727218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebbcc0 msgr2=0x55e6bdecc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.589+0000 7efc7884db80 1 --2- 172.21.15.44:0/3010727218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebbcc0 0x55e6bdecc520 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55e6bdeb6060 tx=0x7efc54012000 comp rx=0 tx=0).stop 2023-09-10T01:15:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc7884db80 1 -- 172.21.15.44:0/3010727218 shutdown_connections 2023-09-10T01:15:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc7884db80 1 --2- 172.21.15.44:0/3010727218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdec11a0 0x55e6bdec78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc7884db80 1 --2- 172.21.15.44:0/3010727218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdebb300 0x55e6bdec06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc7884db80 1 --2- 172.21.15.44:0/3010727218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebbcc0 0x55e6bdecc520 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc7884db80 1 -- 172.21.15.44:0/3010727218 >> 172.21.15.44:0/3010727218 conn(0x55e6bde30030 msgr2=0x55e6bdebab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc7884db80 1 -- 172.21.15.44:0/3010727218 shutdown_connections 2023-09-10T01:15:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc7884db80 1 -- 172.21.15.44:0/3010727218 wait complete. 2023-09-10T01:15:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc7884db80 1 -- start start 2023-09-10T01:15:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc7884db80 1 Processor -- start 2023-09-10T01:15:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc7884db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebb300 0x55e6bdecc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc7884db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdebbcc0 0x55e6bdecc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc7884db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdec11a0 0x55e6bdeca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc7884db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bdecac70 con 0x55e6bdebbcc0 2023-09-10T01:15:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc7884db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bdecade0 con 0x55e6bdebb300 2023-09-10T01:15:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc7884db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6bdecafa0 con 0x55e6bdec11a0 2023-09-10T01:15:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc7884db80 10 client.0 _ll_register_callbacks cb 0x7efc6009ca60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:15:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc69c22700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdebbcc0 0x55e6bdecc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc69c22700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdebbcc0 0x55e6bdecc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57516/0 (socket says 172.21.15.44:57516) 2023-09-10T01:15:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc69c22700 1 -- 172.21.15.44:0/2811074969 learned_addr learned my addr 172.21.15.44:0/2811074969 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc6a423700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebb300 0x55e6bdecc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.590+0000 7efc6ac24700 1 --2- 172.21.15.44:0/2811074969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdec11a0 0x55e6bdeca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc6a423700 1 -- 172.21.15.44:0/2811074969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdec11a0 msgr2=0x55e6bdeca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc6a423700 1 --2- 172.21.15.44:0/2811074969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdec11a0 0x55e6bdeca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc6a423700 1 -- 172.21.15.44:0/2811074969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdebbcc0 msgr2=0x55e6bdecc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc6a423700 1 --2- 172.21.15.44:0/2811074969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdebbcc0 0x55e6bdecc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc6a423700 1 -- 172.21.15.44:0/2811074969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc540040b0 con 0x55e6bdebb300 2023-09-10T01:15:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc6a423700 1 --2- 172.21.15.44:0/2811074969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebb300 0x55e6bdecc140 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efc6009cc20 tx=0x7efc60005790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cd5f42e31ce7b4d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:15:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc4dffb700 1 -- 172.21.15.44:0/2811074969 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6009c380 con 0x55e6bdebb300 2023-09-10T01:15:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc4dffb700 1 -- 172.21.15.44:0/2811074969 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 9540+0+0 (secure 0 0 0) 0x7efc600a0ac0 con 0x55e6bdebb300 2023-09-10T01:15:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc7884db80 1 -- 172.21.15.44:0/2811074969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc5c00f040 con 0x55e6bdebb300 2023-09-10T01:15:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc4dffb700 1 -- 172.21.15.44:0/2811074969 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc6009cc90 con 0x55e6bdebb300 2023-09-10T01:15:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc7884db80 10 client.29123 fetch_fsmap learned FSMap version 120 2023-09-10T01:15:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc7884db80 1 -- 172.21.15.44:0/2811074969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=4+}) v3 -- 0x55e6bdc38250 con 0x55e6bdebb300 2023-09-10T01:15:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc4dffb700 1 -- 172.21.15.44:0/2811074969 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc5c00f040 con 0x55e6bdebb300 2023-09-10T01:15:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc4dffb700 1 -- 172.21.15.44:0/2811074969 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 120) v1 ==== 7315+0+0 (secure 0 0 0) 0x7efc600b7070 con 0x55e6bdebb300 2023-09-10T01:15:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc7884db80 10 client.29123 fetch_fsmap finished waiting for FSMap version 120 2023-09-10T01:15:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc7884db80 10 client.29123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:15:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.591+0000 7efc7884db80 20 client.29123 populate_metadata read hostname 'smithi044' 2023-09-10T01:15:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.592+0000 7efc7884db80 1 --2- 172.21.15.44:0/2811074969 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e6bdf29b90 0x55e6bdf2bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.592+0000 7efc7884db80 4 client.29123 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:15:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.592+0000 7efc7884db80 1 -- 172.21.15.44:0/2811074969 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e6bde068c0 con 0x55e6bdf29b90 2023-09-10T01:15:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.592+0000 7efc69c22700 1 --2- 172.21.15.44:0/2811074969 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e6bdf29b90 0x55e6bdf2bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.592+0000 7efc4dffb700 10 client.29123 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.592+0000 7efc69c22700 1 --2- 172.21.15.44:0/2811074969 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e6bdf29b90 0x55e6bdf2bfb0 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34c7d15f01e6dbe6 server_cookie=5ce3c189e5e0f5c5 in_seq=0 out_seq=0 2023-09-10T01:15:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.607+0000 7efc4dffb700 1 -- 172.21.15.44:0/2811074969 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e6bde068c0 con 0x55e6bdf29b90 2023-09-10T01:15:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.607+0000 7efc4dffb700 10 client.29123 handle_command_reply: tid=0 2023-09-10T01:15:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.607+0000 7efc7884db80 1 client.29123 shutdown 2023-09-10T01:15:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc7884db80 1 -- 172.21.15.44:0/2811074969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebb300 msgr2=0x55e6bdecc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc7884db80 1 --2- 172.21.15.44:0/2811074969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebb300 0x55e6bdecc140 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efc6009cc20 tx=0x7efc60005790 comp rx=0 tx=0).stop 2023-09-10T01:15:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc7884db80 1 -- 172.21.15.44:0/2811074969 shutdown_connections 2023-09-10T01:15:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc7884db80 1 --2- 172.21.15.44:0/2811074969 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e6bdf29b90 0x55e6bdf2bfb0 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc7884db80 1 --2- 172.21.15.44:0/2811074969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6bdec11a0 0x55e6bdeca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc7884db80 1 --2- 172.21.15.44:0/2811074969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6bdebb300 0x55e6bdecc140 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc7884db80 1 --2- 172.21.15.44:0/2811074969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6bdebbcc0 0x55e6bdecc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc4dffb700 0 client.29123 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc7884db80 1 -- 172.21.15.44:0/2811074969 >> 172.21.15.44:0/2811074969 conn(0x55e6bde30030 msgr2=0x55e6bdeb4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc7884db80 1 -- 172.21.15.44:0/2811074969 shutdown_connections 2023-09-10T01:15:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc7884db80 1 -- 172.21.15.44:0/2811074969 wait complete. 2023-09-10T01:15:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc7884db80 20 client.29123 trim_cache size 0 max 16384 2023-09-10T01:15:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:15:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:15:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:15:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:15:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc6c687700 1 -- 172.21.15.44:0/1163092480 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efc500a07b0 msgr2=0x7efc500a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc6c687700 1 --2- 172.21.15.44:0/1163092480 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efc500a07b0 0x7efc500a2c70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efc64196bb0 tx=0x7efc5400a380 comp rx=0 tx=0).stop 2023-09-10T01:15:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc6c687700 1 -- 172.21.15.44:0/1163092480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc640ff760 msgr2=0x7efc64195390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc6c687700 1 --2- 172.21.15.44:0/1163092480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc640ff760 0x7efc64195390 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7efc60009fd0 tx=0x7efc60005e80 comp rx=0 tx=0).stop 2023-09-10T01:15:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc6c687700 1 -- 172.21.15.44:0/1163092480 shutdown_connections 2023-09-10T01:15:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc6c687700 1 --2- 172.21.15.44:0/1163092480 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7efc500a07b0 0x7efc500a2c70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc6c687700 1 --2- 172.21.15.44:0/1163092480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc641055b0 0x7efc64199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc6c687700 1 --2- 172.21.15.44:0/1163092480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc640ff760 0x7efc64195390 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.608+0000 7efc6c687700 1 --2- 172.21.15.44:0/1163092480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc64100120 0x7efc641958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.609+0000 7efc6c687700 1 -- 172.21.15.44:0/1163092480 >> 172.21.15.44:0/1163092480 conn(0x7efc64074f30 msgr2=0x7efc640fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.609+0000 7efc6c687700 1 -- 172.21.15.44:0/1163092480 shutdown_connections 2023-09-10T01:15:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:16.609+0000 7efc6c687700 1 -- 172.21.15.44:0/1163092480 wait complete. 2023-09-10T01:15:17.262 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:17 smithi044 podman[196163]: 2023-09-10T01:15:17.262 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:16 smithi044 ceph-mon[98676]: pgmap v1108: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:15:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:16 smithi131 ceph-mon[103713]: pgmap v1108: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:15:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:16 smithi099 ceph-mon[102692]: pgmap v1108: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2023-09-10T01:15:17.805 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:17 smithi044 bash[196163]: 85e68be07a22434f9373043ccc9f493804d32c7f6ff47200b4c2ea01623c1132 2023-09-10T01:15:17.805 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:17 smithi044 ceph-mds[196253]: -- 172.21.15.44:0/4220248448 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c3cbd7b1e0 con 0x55c3cbd50c00 2023-09-10T01:15:17.805 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:17 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: starting mds.d at 2023-09-10T01:15:17.806 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:15:17 smithi044 systemd[1]: Started Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:15:18.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:17 smithi131 ceph-mon[103713]: daemon mds.d restarted 2023-09-10T01:15:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:17 smithi044 ceph-mon[98676]: daemon mds.d restarted 2023-09-10T01:15:18.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:17 smithi099 ceph-mon[102692]: daemon mds.d restarted 2023-09-10T01:15:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:18 smithi131 ceph-mon[103713]: osdmap e155: 12 total, 12 up, 12 in 2023-09-10T01:15:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:18 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.44:6836/535338380,v1:172.21.15.44:6837/535338380] up:boot 2023-09-10T01:15:19.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:18 smithi131 ceph-mon[103713]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:15:19.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:18 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T01:15:19.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:18 smithi131 ceph-mon[103713]: pgmap v1110: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 199 op/s 2023-09-10T01:15:19.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:18 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:15:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:18 smithi044 ceph-mon[98676]: osdmap e155: 12 total, 12 up, 12 in 2023-09-10T01:15:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:18 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.44:6836/535338380,v1:172.21.15.44:6837/535338380] up:boot 2023-09-10T01:15:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:18 smithi044 ceph-mon[98676]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:15:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:18 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T01:15:19.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:18 smithi044 ceph-mon[98676]: pgmap v1110: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 199 op/s 2023-09-10T01:15:19.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:18 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:15:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:18 smithi099 ceph-mon[102692]: osdmap e155: 12 total, 12 up, 12 in 2023-09-10T01:15:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:18 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.44:6836/535338380,v1:172.21.15.44:6837/535338380] up:boot 2023-09-10T01:15:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:18 smithi099 ceph-mon[102692]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:15:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:18 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T01:15:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:18 smithi099 ceph-mon[102692]: pgmap v1110: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 199 op/s 2023-09-10T01:15:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:18 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:15:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:21 smithi131 ceph-mon[103713]: pgmap v1111: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 634 KiB/s wr, 89 op/s 2023-09-10T01:15:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:21 smithi044 ceph-mon[98676]: pgmap v1111: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 634 KiB/s wr, 89 op/s 2023-09-10T01:15:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:21 smithi099 ceph-mon[102692]: pgmap v1111: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 634 KiB/s wr, 89 op/s 2023-09-10T01:15:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:23 smithi131 ceph-mon[103713]: pgmap v1112: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 154 op/s 2023-09-10T01:15:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:23 smithi044 ceph-mon[98676]: pgmap v1112: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 154 op/s 2023-09-10T01:15:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:23 smithi099 ceph-mon[102692]: pgmap v1112: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 154 op/s 2023-09-10T01:15:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:25 smithi131 ceph-mon[103713]: pgmap v1113: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 154 op/s 2023-09-10T01:15:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:25 smithi044 ceph-mon[98676]: pgmap v1113: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 154 op/s 2023-09-10T01:15:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:25 smithi099 ceph-mon[102692]: pgmap v1113: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 154 op/s 2023-09-10T01:15:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:26 smithi131 ceph-mon[103713]: pgmap v1114: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 987 KiB/s wr, 143 op/s 2023-09-10T01:15:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:26 smithi044 ceph-mon[98676]: pgmap v1114: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 987 KiB/s wr, 143 op/s 2023-09-10T01:15:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:26 smithi099 ceph-mon[102692]: pgmap v1114: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 987 KiB/s wr, 143 op/s 2023-09-10T01:15:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:29 smithi131 ceph-mon[103713]: pgmap v1115: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 986 KiB/s wr, 143 op/s 2023-09-10T01:15:29.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:29 smithi044 ceph-mon[98676]: pgmap v1115: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 986 KiB/s wr, 143 op/s 2023-09-10T01:15:29.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:29 smithi099 ceph-mon[102692]: pgmap v1115: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 986 KiB/s wr, 143 op/s 2023-09-10T01:15:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.742+0000 7f915dadb700 1 -- 172.21.15.44:0/1539617676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91580ff760 msgr2=0x7f91580ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.742+0000 7f915dadb700 1 --2- 172.21.15.44:0/1539617676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91580ff760 0x7f91580ffbe0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f914c0099c0 tx=0x7f914c009cd0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.742+0000 7f915dadb700 1 -- 172.21.15.44:0/1539617676 shutdown_connections 2023-09-10T01:15:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.742+0000 7f915dadb700 1 --2- 172.21.15.44:0/1539617676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158100120 0x7f915810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.742+0000 7f915dadb700 1 --2- 172.21.15.44:0/1539617676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91581055b0 0x7f9158105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.742+0000 7f915dadb700 1 --2- 172.21.15.44:0/1539617676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91580ff760 0x7f91580ffbe0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.742+0000 7f915dadb700 1 -- 172.21.15.44:0/1539617676 >> 172.21.15.44:0/1539617676 conn(0x7f9158074f30 msgr2=0x7f9158075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.742+0000 7f915dadb700 1 -- 172.21.15.44:0/1539617676 shutdown_connections 2023-09-10T01:15:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.742+0000 7f915dadb700 1 -- 172.21.15.44:0/1539617676 wait complete. 2023-09-10T01:15:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.742+0000 7f915dadb700 1 Processor -- start 2023-09-10T01:15:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.742+0000 7f915dadb700 1 -- start start 2023-09-10T01:15:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f915dadb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91580ff760 0x7f9158197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f915dadb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158100120 0x7f9158197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f915dadb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91581055b0 0x7f91581a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f915dadb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91581a29f0 con 0x7f91580ff760 2023-09-10T01:15:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f915dadb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91581a2b60 con 0x7f9158100120 2023-09-10T01:15:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f915dadb700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91581a2d20 con 0x7f91581055b0 2023-09-10T01:15:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f91577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91580ff760 0x7f9158197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f91577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91580ff760 0x7f9158197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57554/0 (socket says 172.21.15.44:57554) 2023-09-10T01:15:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f91577fe700 1 -- 172.21.15.44:0/3382490879 learned_addr learned my addr 172.21.15.44:0/3382490879 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f9157fff700 1 --2- 172.21.15.44:0/3382490879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91581055b0 0x7f91581a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f9156ffd700 1 --2- 172.21.15.44:0/3382490879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158100120 0x7f9158197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f91577fe700 1 -- 172.21.15.44:0/3382490879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91581055b0 msgr2=0x7f91581a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f91577fe700 1 --2- 172.21.15.44:0/3382490879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91581055b0 0x7f91581a2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f91577fe700 1 -- 172.21.15.44:0/3382490879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158100120 msgr2=0x7f9158197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f91577fe700 1 --2- 172.21.15.44:0/3382490879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158100120 0x7f9158197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f91577fe700 1 -- 172.21.15.44:0/3382490879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914c0095f0 con 0x7f91580ff760 2023-09-10T01:15:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.743+0000 7f91577fe700 1 --2- 172.21.15.44:0/3382490879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91580ff760 0x7f9158197580 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f914800e9a0 tx=0x7f914800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.744+0000 7f9154ff9700 1 -- 172.21.15.44:0/3382490879 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148009790 con 0x7f91580ff760 2023-09-10T01:15:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.744+0000 7f915dadb700 1 -- 172.21.15.44:0/3382490879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91581a2fa0 con 0x7f91580ff760 2023-09-10T01:15:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.744+0000 7f9154ff9700 1 -- 172.21.15.44:0/3382490879 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f914800cbd0 con 0x7f91580ff760 2023-09-10T01:15:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.744+0000 7f9154ff9700 1 -- 172.21.15.44:0/3382490879 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148018680 con 0x7f91580ff760 2023-09-10T01:15:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.744+0000 7f915dadb700 1 -- 172.21.15.44:0/3382490879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91581a34d0 con 0x7f91580ff760 2023-09-10T01:15:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.747+0000 7f9154ff9700 1 -- 172.21.15.44:0/3382490879 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f914800cd40 con 0x7f91580ff760 2023-09-10T01:15:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.747+0000 7f9154ff9700 1 --2- 172.21.15.44:0/3382490879 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f91400a07b0 0x7f91400a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.748+0000 7f9169ca1b80 1 Processor -- start 2023-09-10T01:15:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.748+0000 7f9154ff9700 1 -- 172.21.15.44:0/3382490879 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f9148014070 con 0x7f91580ff760 2023-09-10T01:15:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.748+0000 7f9156ffd700 1 --2- 172.21.15.44:0/3382490879 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f91400a07b0 0x7f91400a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.748+0000 7f9169ca1b80 1 -- start start 2023-09-10T01:15:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.748+0000 7f9169ca1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383c41a0 0x5604383c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.748+0000 7f9169ca1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383be300 0x5604383be780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.748+0000 7f9169ca1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383becc0 0x5604383caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.748+0000 7f9169ca1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604383176d0 con 0x5604383becc0 2023-09-10T01:15:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.748+0000 7f9169ca1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560438314ff0 con 0x5604383c41a0 2023-09-10T01:15:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.748+0000 7f9169ca1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604383108e0 con 0x5604383be300 2023-09-10T01:15:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.748+0000 7f9157fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383becc0 0x5604383caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.748+0000 7f9157fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383becc0 0x5604383caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57564/0 (socket says 172.21.15.44:57564) 2023-09-10T01:15:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.748+0000 7f9157fff700 1 -- 172.21.15.44:0/4144324571 learned_addr learned my addr 172.21.15.44:0/4144324571 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.748+0000 7f9156ffd700 1 --2- 172.21.15.44:0/4144324571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383be300 0x5604383be780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.748+0000 7f9156ffd700 1 --2- 172.21.15.44:0/3382490879 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f91400a07b0 0x7f91400a2c70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f91580684b0 tx=0x7f914c01a040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.748+0000 7f91577fe700 1 --2- 172.21.15.44:0/4144324571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383c41a0 0x5604383c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.749+0000 7f9157fff700 1 -- 172.21.15.44:0/4144324571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383be300 msgr2=0x5604383be780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.749+0000 7f9157fff700 1 --2- 172.21.15.44:0/4144324571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383be300 0x5604383be780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.749+0000 7f9157fff700 1 -- 172.21.15.44:0/4144324571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383c41a0 msgr2=0x5604383c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.749+0000 7f9157fff700 1 --2- 172.21.15.44:0/4144324571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383c41a0 0x5604383c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.749+0000 7f9157fff700 1 -- 172.21.15.44:0/4144324571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56043813cd80 con 0x5604383becc0 2023-09-10T01:15:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.749+0000 7f91577fe700 1 --2- 172.21.15.44:0/4144324571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383c41a0 0x5604383c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:15:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.749+0000 7f9157fff700 1 --2- 172.21.15.44:0/4144324571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383becc0 0x5604383caf30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f91500098f0 tx=0x7f9150009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92b66c8bc275c3b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.749+0000 7f913dffb700 1 -- 172.21.15.44:0/4144324571 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150019070 con 0x5604383becc0 2023-09-10T01:15:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.749+0000 7f913dffb700 1 -- 172.21.15.44:0/4144324571 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f915000f040 con 0x5604383becc0 2023-09-10T01:15:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.749+0000 7f913dffb700 1 -- 172.21.15.44:0/4144324571 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91500079f0 con 0x5604383becc0 2023-09-10T01:15:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.750+0000 7f9169ca1b80 1 -- 172.21.15.44:0/4144324571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383becc0 msgr2=0x5604383caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.750+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/4144324571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383becc0 0x5604383caf30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f91500098f0 tx=0x7f9150009c00 comp rx=0 tx=0).stop 2023-09-10T01:15:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.750+0000 7f9169ca1b80 1 -- 172.21.15.44:0/4144324571 shutdown_connections 2023-09-10T01:15:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.750+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/4144324571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383becc0 0x5604383caf30 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.750+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/4144324571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383c41a0 0x5604383c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.750+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/4144324571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383be300 0x5604383be780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.750+0000 7f9169ca1b80 1 -- 172.21.15.44:0/4144324571 >> 172.21.15.44:0/4144324571 conn(0x560438333030 msgr2=0x5604383b9560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.750+0000 7f9169ca1b80 1 -- 172.21.15.44:0/4144324571 shutdown_connections 2023-09-10T01:15:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.750+0000 7f9169ca1b80 1 -- 172.21.15.44:0/4144324571 wait complete. 2023-09-10T01:15:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.750+0000 7f9169ca1b80 1 -- start start 2023-09-10T01:15:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.750+0000 7f9169ca1b80 1 Processor -- start 2023-09-10T01:15:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9169ca1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383be300 0x5604383cad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9169ca1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383becc0 0x5604383c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9169ca1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383c41a0 0x5604383c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9169ca1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604382aeb10 con 0x5604383becc0 2023-09-10T01:15:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9169ca1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604382ae810 con 0x5604383be300 2023-09-10T01:15:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9169ca1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604381928c0 con 0x5604383c41a0 2023-09-10T01:15:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9169ca1b80 10 client.0 _ll_register_callbacks cb 0x5604381f8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:15:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9156ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383becc0 0x5604383c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9156ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383becc0 0x5604383c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57570/0 (socket says 172.21.15.44:57570) 2023-09-10T01:15:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9156ffd700 1 -- 172.21.15.44:0/510750018 learned_addr learned my addr 172.21.15.44:0/510750018 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f91577fe700 1 --2- 172.21.15.44:0/510750018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383be300 0x5604383cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9157fff700 1 --2- 172.21.15.44:0/510750018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383c41a0 0x5604383c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9156ffd700 1 -- 172.21.15.44:0/510750018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383c41a0 msgr2=0x5604383c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9156ffd700 1 --2- 172.21.15.44:0/510750018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383c41a0 0x5604383c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9156ffd700 1 -- 172.21.15.44:0/510750018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383be300 msgr2=0x5604383cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9156ffd700 1 --2- 172.21.15.44:0/510750018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383be300 0x5604383cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9156ffd700 1 -- 172.21.15.44:0/510750018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9150003c50 con 0x5604383becc0 2023-09-10T01:15:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9157fff700 1 --2- 172.21.15.44:0/510750018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383c41a0 0x5604383c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:15:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9156ffd700 1 --2- 172.21.15.44:0/510750018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383becc0 0x5604383c7f80 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f914c0099c0 tx=0x7f914c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e37eb9859ba8ad3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.751+0000 7f9136ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:15:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.752+0000 7f9136ffd700 1 -- 172.21.15.44:0/510750018 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c01d040 con 0x5604383becc0 2023-09-10T01:15:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.752+0000 7f9136ffd700 1 -- 172.21.15.44:0/510750018 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f914c00f870 con 0x5604383becc0 2023-09-10T01:15:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.752+0000 7f9169ca1b80 1 -- 172.21.15.44:0/510750018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560438268490 con 0x5604383becc0 2023-09-10T01:15:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.752+0000 7f9136ffd700 1 -- 172.21.15.44:0/510750018 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f914c00fa70 con 0x5604383becc0 2023-09-10T01:15:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.752+0000 7f9169ca1b80 10 client.19479 fetch_fsmap learned FSMap version 122 2023-09-10T01:15:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.752+0000 7f9169ca1b80 1 -- 172.21.15.44:0/510750018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=4+}) v3 -- 0x560438125c00 con 0x5604383becc0 2023-09-10T01:15:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.752+0000 7f9136ffd700 1 -- 172.21.15.44:0/510750018 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560438268490 con 0x5604383becc0 2023-09-10T01:15:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.752+0000 7f9136ffd700 1 -- 172.21.15.44:0/510750018 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 122) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f914c02ebe0 con 0x5604383becc0 2023-09-10T01:15:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.752+0000 7f9169ca1b80 10 client.19479 fetch_fsmap finished waiting for FSMap version 122 2023-09-10T01:15:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.752+0000 7f9169ca1b80 10 client.19479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:15:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.752+0000 7f9169ca1b80 20 client.19479 populate_metadata read hostname 'smithi044' 2023-09-10T01:15:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.752+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/510750018 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5604383d8380 0x5604383da7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.752+0000 7f9169ca1b80 4 client.19479 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:15:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.752+0000 7f9169ca1b80 1 -- 172.21.15.44:0/510750018 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560438325840 con 0x5604383d8380 2023-09-10T01:15:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.753+0000 7f91577fe700 1 --2- 172.21.15.44:0/510750018 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5604383d8380 0x5604383da7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.753+0000 7f9136ffd700 10 client.19479 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.753+0000 7f91577fe700 1 --2- 172.21.15.44:0/510750018 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5604383d8380 0x5604383da7a0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0647ce128f6a249 server_cookie=a45d3bdeb898df9a in_seq=0 out_seq=0 2023-09-10T01:15:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.754+0000 7f9136ffd700 1 -- 172.21.15.44:0/510750018 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560438325840 con 0x5604383d8380 2023-09-10T01:15:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.754+0000 7f9136ffd700 10 client.19479 handle_command_reply: tid=0 2023-09-10T01:15:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.754+0000 7f9169ca1b80 1 client.19479 shutdown 2023-09-10T01:15:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.754+0000 7f9169ca1b80 1 -- 172.21.15.44:0/510750018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383becc0 msgr2=0x5604383c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.754+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/510750018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383becc0 0x5604383c7f80 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f914c0099c0 tx=0x7f914c009cd0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.754+0000 7f9169ca1b80 1 -- 172.21.15.44:0/510750018 shutdown_connections 2023-09-10T01:15:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.754+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/510750018 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5604383d8380 0x5604383da7a0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.754+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/510750018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383c41a0 0x5604383c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.754+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/510750018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383be300 0x5604383cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.754+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/510750018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383becc0 0x5604383c7f80 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.754+0000 7f9136ffd700 0 client.19479 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.754+0000 7f9169ca1b80 1 -- 172.21.15.44:0/510750018 >> 172.21.15.44:0/510750018 conn(0x560438333030 msgr2=0x5604383b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.754+0000 7f9169ca1b80 1 -- 172.21.15.44:0/510750018 shutdown_connections 2023-09-10T01:15:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.754+0000 7f9169ca1b80 1 -- 172.21.15.44:0/510750018 wait complete. 2023-09-10T01:15:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.754+0000 7f9169ca1b80 20 client.19479 trim_cache size 0 max 16384 2023-09-10T01:15:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.771+0000 7f9169ca1b80 1 Processor -- start 2023-09-10T01:15:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.771+0000 7f9169ca1b80 1 -- start start 2023-09-10T01:15:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.771+0000 7f9169ca1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383be300 0x5604383c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.771+0000 7f9169ca1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383becc0 0x5604383ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.771+0000 7f9169ca1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383c41a0 0x5604383b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.771+0000 7f9169ca1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604383b38a0 con 0x5604383be300 2023-09-10T01:15:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.771+0000 7f9169ca1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604383b5fc0 con 0x5604383becc0 2023-09-10T01:15:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.771+0000 7f9169ca1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604383bf580 con 0x5604383c41a0 2023-09-10T01:15:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.772+0000 7f91577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383be300 0x5604383c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.772+0000 7f91577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383be300 0x5604383c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57576/0 (socket says 172.21.15.44:57576) 2023-09-10T01:15:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.772+0000 7f91577fe700 1 -- 172.21.15.44:0/2911892911 learned_addr learned my addr 172.21.15.44:0/2911892911 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.772+0000 7f9157fff700 1 --2- 172.21.15.44:0/2911892911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383c41a0 0x5604383b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.772+0000 7f9156ffd700 1 --2- 172.21.15.44:0/2911892911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383becc0 0x5604383ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.772+0000 7f91577fe700 1 -- 172.21.15.44:0/2911892911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383c41a0 msgr2=0x5604383b8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.772+0000 7f91577fe700 1 --2- 172.21.15.44:0/2911892911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383c41a0 0x5604383b8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.772+0000 7f91577fe700 1 -- 172.21.15.44:0/2911892911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383becc0 msgr2=0x5604383ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.772+0000 7f91577fe700 1 --2- 172.21.15.44:0/2911892911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383becc0 0x5604383ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.772+0000 7f91577fe700 1 -- 172.21.15.44:0/2911892911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604381631f0 con 0x5604383be300 2023-09-10T01:15:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.772+0000 7f91577fe700 1 --2- 172.21.15.44:0/2911892911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383be300 0x5604383c6e40 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f91480c9fc0 tx=0x7f914801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e293e6512f212f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.772+0000 7f913e7fc700 1 -- 172.21.15.44:0/2911892911 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914809d550 con 0x5604383be300 2023-09-10T01:15:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.772+0000 7f913e7fc700 1 -- 172.21.15.44:0/2911892911 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f91480a1790 con 0x5604383be300 2023-09-10T01:15:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.772+0000 7f913e7fc700 1 -- 172.21.15.44:0/2911892911 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148099980 con 0x5604383be300 2023-09-10T01:15:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.773+0000 7f9169ca1b80 1 -- 172.21.15.44:0/2911892911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383be300 msgr2=0x5604383c6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.773+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/2911892911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383be300 0x5604383c6e40 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f91480c9fc0 tx=0x7f914801a040 comp rx=0 tx=0).stop 2023-09-10T01:15:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.773+0000 7f9169ca1b80 1 -- 172.21.15.44:0/2911892911 shutdown_connections 2023-09-10T01:15:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.773+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/2911892911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383c41a0 0x5604383b8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.773+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/2911892911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383be300 0x5604383c6e40 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.773+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/2911892911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383becc0 0x5604383ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.773+0000 7f9169ca1b80 1 -- 172.21.15.44:0/2911892911 >> 172.21.15.44:0/2911892911 conn(0x560438333030 msgr2=0x5604383c6730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.773+0000 7f9169ca1b80 1 -- 172.21.15.44:0/2911892911 shutdown_connections 2023-09-10T01:15:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.773+0000 7f9169ca1b80 1 -- 172.21.15.44:0/2911892911 wait complete. 2023-09-10T01:15:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.773+0000 7f9169ca1b80 1 -- start start 2023-09-10T01:15:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.773+0000 7f9169ca1b80 1 Processor -- start 2023-09-10T01:15:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f9169ca1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383be300 0x5604383cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f9169ca1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383becc0 0x5604383cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f9169ca1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383c41a0 0x5604383cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f9169ca1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604383ce1a0 con 0x5604383be300 2023-09-10T01:15:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f9169ca1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604383ce310 con 0x5604383becc0 2023-09-10T01:15:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f9169ca1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604383ce4d0 con 0x5604383c41a0 2023-09-10T01:15:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f9169ca1b80 10 client.0 _ll_register_callbacks cb 0x5604381f8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:15:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f91577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383be300 0x5604383cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f91577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383be300 0x5604383cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57582/0 (socket says 172.21.15.44:57582) 2023-09-10T01:15:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f91577fe700 1 -- 172.21.15.44:0/362378495 learned_addr learned my addr 172.21.15.44:0/362378495 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f9156ffd700 1 --2- 172.21.15.44:0/362378495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383becc0 0x5604383cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f9157fff700 1 --2- 172.21.15.44:0/362378495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383c41a0 0x5604383cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f91577fe700 1 -- 172.21.15.44:0/362378495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383c41a0 msgr2=0x5604383cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f91577fe700 1 --2- 172.21.15.44:0/362378495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383c41a0 0x5604383cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f91577fe700 1 -- 172.21.15.44:0/362378495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383becc0 msgr2=0x5604383cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f91577fe700 1 --2- 172.21.15.44:0/362378495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383becc0 0x5604383cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.774+0000 7f91577fe700 1 -- 172.21.15.44:0/362378495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91581a34d0 con 0x5604383be300 2023-09-10T01:15:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.775+0000 7f91577fe700 1 --2- 172.21.15.44:0/362378495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383be300 0x5604383cf3e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f91480c9a70 tx=0x7f91480c9960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fcb63b484d23ca7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.775+0000 7f9136ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:15:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.775+0000 7f9136ffd700 1 -- 172.21.15.44:0/362378495 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91480cc1f0 con 0x5604383be300 2023-09-10T01:15:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.775+0000 7f9136ffd700 1 -- 172.21.15.44:0/362378495 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f91480cc350 con 0x5604383be300 2023-09-10T01:15:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.775+0000 7f9169ca1b80 1 -- 172.21.15.44:0/362378495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f915000f040 con 0x5604383be300 2023-09-10T01:15:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.775+0000 7f9136ffd700 1 -- 172.21.15.44:0/362378495 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91480652a0 con 0x5604383be300 2023-09-10T01:15:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.775+0000 7f9169ca1b80 10 client.19485 fetch_fsmap learned FSMap version 122 2023-09-10T01:15:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.775+0000 7f9169ca1b80 1 -- 172.21.15.44:0/362378495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=4+}) v3 -- 0x56043813b250 con 0x5604383be300 2023-09-10T01:15:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.775+0000 7f9136ffd700 1 -- 172.21.15.44:0/362378495 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f915000f040 con 0x5604383be300 2023-09-10T01:15:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.775+0000 7f9136ffd700 1 -- 172.21.15.44:0/362378495 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 122) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f9148099980 con 0x5604383be300 2023-09-10T01:15:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.776+0000 7f9169ca1b80 10 client.19485 fetch_fsmap finished waiting for FSMap version 122 2023-09-10T01:15:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.776+0000 7f9169ca1b80 10 client.19485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:15:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.776+0000 7f9169ca1b80 20 client.19485 populate_metadata read hostname 'smithi044' 2023-09-10T01:15:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.776+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/362378495 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56043842d2b0 0x56043842f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.776+0000 7f9169ca1b80 4 client.19485 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:15:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.776+0000 7f9169ca1b80 1 -- 172.21.15.44:0/362378495 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5604383098c0 con 0x56043842d2b0 2023-09-10T01:15:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.776+0000 7f9156ffd700 1 --2- 172.21.15.44:0/362378495 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56043842d2b0 0x56043842f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.776+0000 7f9136ffd700 10 client.19485 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.776+0000 7f9156ffd700 1 --2- 172.21.15.44:0/362378495 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56043842d2b0 0x56043842f6d0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b78c1a293fe595f6 server_cookie=f25f392149c672ef in_seq=0 out_seq=0 2023-09-10T01:15:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.792+0000 7f9136ffd700 1 -- 172.21.15.44:0/362378495 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5604383098c0 con 0x56043842d2b0 2023-09-10T01:15:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.792+0000 7f9136ffd700 10 client.19485 handle_command_reply: tid=0 2023-09-10T01:15:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.792+0000 7f9169ca1b80 1 client.19485 shutdown 2023-09-10T01:15:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.792+0000 7f9169ca1b80 1 -- 172.21.15.44:0/362378495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383be300 msgr2=0x5604383cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.792+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/362378495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383be300 0x5604383cf3e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f91480c9a70 tx=0x7f91480c9960 comp rx=0 tx=0).stop 2023-09-10T01:15:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.792+0000 7f9169ca1b80 1 -- 172.21.15.44:0/362378495 shutdown_connections 2023-09-10T01:15:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.792+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/362378495 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56043842d2b0 0x56043842f6d0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.792+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/362378495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5604383c41a0 0x5604383cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.792+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/362378495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604383be300 0x5604383cf3e0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.792+0000 7f9136ffd700 0 client.19485 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.792+0000 7f9169ca1b80 1 --2- 172.21.15.44:0/362378495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604383becc0 0x5604383cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.792+0000 7f9169ca1b80 1 -- 172.21.15.44:0/362378495 >> 172.21.15.44:0/362378495 conn(0x560438333030 msgr2=0x5604383bdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.792+0000 7f9169ca1b80 1 -- 172.21.15.44:0/362378495 shutdown_connections 2023-09-10T01:15:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.792+0000 7f9169ca1b80 1 -- 172.21.15.44:0/362378495 wait complete. 2023-09-10T01:15:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.792+0000 7f9169ca1b80 20 client.19485 trim_cache size 0 max 16384 2023-09-10T01:15:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:15:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:15:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:15:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:15:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.793+0000 7f915dadb700 1 -- 172.21.15.44:0/3382490879 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f91400a07b0 msgr2=0x7f91400a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.793+0000 7f915dadb700 1 --2- 172.21.15.44:0/3382490879 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f91400a07b0 0x7f91400a2c70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f91580684b0 tx=0x7f914c01a040 comp rx=0 tx=0).stop 2023-09-10T01:15:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.793+0000 7f915dadb700 1 -- 172.21.15.44:0/3382490879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91580ff760 msgr2=0x7f9158197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.793+0000 7f915dadb700 1 --2- 172.21.15.44:0/3382490879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91580ff760 0x7f9158197580 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f914800e9a0 tx=0x7f914800ecb0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.793+0000 7f915dadb700 1 -- 172.21.15.44:0/3382490879 shutdown_connections 2023-09-10T01:15:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.793+0000 7f915dadb700 1 --2- 172.21.15.44:0/3382490879 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f91400a07b0 0x7f91400a2c70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.793+0000 7f915dadb700 1 --2- 172.21.15.44:0/3382490879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91581055b0 0x7f91581a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.793+0000 7f915dadb700 1 --2- 172.21.15.44:0/3382490879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91580ff760 0x7f9158197580 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.793+0000 7f915dadb700 1 --2- 172.21.15.44:0/3382490879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158100120 0x7f9158197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.793+0000 7f915dadb700 1 -- 172.21.15.44:0/3382490879 >> 172.21.15.44:0/3382490879 conn(0x7f9158074f30 msgr2=0x7f91580fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.793+0000 7f915dadb700 1 -- 172.21.15.44:0/3382490879 shutdown_connections 2023-09-10T01:15:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:29.793+0000 7f915dadb700 1 -- 172.21.15.44:0/3382490879 wait complete. 2023-09-10T01:15:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:31 smithi131 ceph-mon[103713]: pgmap v1116: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 822 KiB/s wr, 119 op/s 2023-09-10T01:15:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:31 smithi044 ceph-mon[98676]: pgmap v1116: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 822 KiB/s wr, 119 op/s 2023-09-10T01:15:31.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:31 smithi099 ceph-mon[102692]: pgmap v1116: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 822 KiB/s wr, 119 op/s 2023-09-10T01:15:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:33 smithi131 ceph-mon[103713]: pgmap v1117: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2023-09-10T01:15:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:33 smithi044 ceph-mon[98676]: pgmap v1117: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2023-09-10T01:15:33.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:33 smithi099 ceph-mon[102692]: pgmap v1117: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2023-09-10T01:15:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:35 smithi131 ceph-mon[103713]: pgmap v1118: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 880 KiB/s wr, 126 op/s 2023-09-10T01:15:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:35 smithi044 ceph-mon[98676]: pgmap v1118: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 880 KiB/s wr, 126 op/s 2023-09-10T01:15:35.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:35 smithi099 ceph-mon[102692]: pgmap v1118: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 880 KiB/s wr, 126 op/s 2023-09-10T01:15:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:36 smithi131 ceph-mon[103713]: pgmap v1119: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 183 op/s 2023-09-10T01:15:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:36 smithi044 ceph-mon[98676]: pgmap v1119: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 183 op/s 2023-09-10T01:15:37.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:36 smithi099 ceph-mon[102692]: pgmap v1119: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 183 op/s 2023-09-10T01:15:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:39 smithi131 ceph-mon[103713]: pgmap v1120: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 870 KiB/s wr, 125 op/s 2023-09-10T01:15:39.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:39 smithi044 ceph-mon[98676]: pgmap v1120: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 870 KiB/s wr, 125 op/s 2023-09-10T01:15:39.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:39 smithi099 ceph-mon[102692]: pgmap v1120: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 870 KiB/s wr, 125 op/s 2023-09-10T01:15:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:41 smithi131 ceph-mon[103713]: pgmap v1121: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 870 KiB/s wr, 125 op/s 2023-09-10T01:15:41.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:41 smithi044 ceph-mon[98676]: pgmap v1121: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 870 KiB/s wr, 125 op/s 2023-09-10T01:15:41.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:41 smithi099 ceph-mon[102692]: pgmap v1121: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 870 KiB/s wr, 125 op/s 2023-09-10T01:15:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.934+0000 7fa62d590700 1 -- 172.21.15.44:0/446001396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6281055b0 msgr2=0x7fa628105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.934+0000 7fa62d590700 1 --2- 172.21.15.44:0/446001396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6281055b0 0x7fa628105990 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fa6180099c0 tx=0x7fa618009cd0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.934+0000 7fa62d590700 1 -- 172.21.15.44:0/446001396 shutdown_connections 2023-09-10T01:15:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.934+0000 7fa62d590700 1 --2- 172.21.15.44:0/446001396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa628100120 0x7fa62810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.934+0000 7fa62d590700 1 --2- 172.21.15.44:0/446001396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6281055b0 0x7fa628105990 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.934+0000 7fa62d590700 1 --2- 172.21.15.44:0/446001396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6280ff760 0x7fa6280ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.934+0000 7fa62d590700 1 -- 172.21.15.44:0/446001396 >> 172.21.15.44:0/446001396 conn(0x7fa628074f30 msgr2=0x7fa628075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.934+0000 7fa62d590700 1 -- 172.21.15.44:0/446001396 shutdown_connections 2023-09-10T01:15:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.934+0000 7fa62d590700 1 -- 172.21.15.44:0/446001396 wait complete. 2023-09-10T01:15:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.935+0000 7fa62d590700 1 Processor -- start 2023-09-10T01:15:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.935+0000 7fa62d590700 1 -- start start 2023-09-10T01:15:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.935+0000 7fa62d590700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6280ff760 0x7fa628197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.935+0000 7fa62d590700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa628100120 0x7fa628197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.935+0000 7fa62d590700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6281055b0 0x7fa6281a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.935+0000 7fa62d590700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6281a29d0 con 0x7fa6281055b0 2023-09-10T01:15:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.935+0000 7fa62d590700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6281a2b40 con 0x7fa6280ff760 2023-09-10T01:15:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.935+0000 7fa62d590700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6281a2d00 con 0x7fa628100120 2023-09-10T01:15:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa6267fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa628100120 0x7fa628197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa6277fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6281055b0 0x7fa6281a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa6267fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa628100120 0x7fa628197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36248/0 (socket says 172.21.15.44:36248) 2023-09-10T01:15:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa626ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6280ff760 0x7fa628197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa6267fc700 1 -- 172.21.15.44:0/3653181964 learned_addr learned my addr 172.21.15.44:0/3653181964 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa6267fc700 1 -- 172.21.15.44:0/3653181964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6280ff760 msgr2=0x7fa628197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa6267fc700 1 --2- 172.21.15.44:0/3653181964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6280ff760 0x7fa628197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa6267fc700 1 -- 172.21.15.44:0/3653181964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6281055b0 msgr2=0x7fa6281a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa6267fc700 1 --2- 172.21.15.44:0/3653181964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6281055b0 0x7fa6281a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa6267fc700 1 -- 172.21.15.44:0/3653181964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6180095f0 con 0x7fa628100120 2023-09-10T01:15:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa6277fe700 1 --2- 172.21.15.44:0/3653181964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6281055b0 0x7fa6281a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:15:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa6267fc700 1 --2- 172.21.15.44:0/3653181964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa628100120 0x7fa628197aa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00e9a0 tx=0x7fa61c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa60ffff700 1 -- 172.21.15.44:0/3653181964 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c009790 con 0x7fa628100120 2023-09-10T01:15:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa60ffff700 1 -- 172.21.15.44:0/3653181964 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa61c010430 con 0x7fa628100120 2023-09-10T01:15:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa62d590700 1 -- 172.21.15.44:0/3653181964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6281a2fe0 con 0x7fa628100120 2023-09-10T01:15:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa60ffff700 1 -- 172.21.15.44:0/3653181964 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c00cea0 con 0x7fa628100120 2023-09-10T01:15:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.936+0000 7fa62d590700 1 -- 172.21.15.44:0/3653181964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6281a3460 con 0x7fa628100120 2023-09-10T01:15:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.939+0000 7fa60ffff700 1 -- 172.21.15.44:0/3653181964 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa61c0105a0 con 0x7fa628100120 2023-09-10T01:15:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.940+0000 7fa60ffff700 1 --2- 172.21.15.44:0/3653181964 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa6100a07b0 0x7fa6100a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.940+0000 7fa639756b80 1 Processor -- start 2023-09-10T01:15:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.940+0000 7fa626ffd700 1 --2- 172.21.15.44:0/3653181964 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa6100a07b0 0x7fa6100a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.940+0000 7fa60ffff700 1 -- 172.21.15.44:0/3653181964 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa61c014070 con 0x7fa628100120 2023-09-10T01:15:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.940+0000 7fa639756b80 1 -- start start 2023-09-10T01:15:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.940+0000 7fa639756b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c3c270 0x55c278c38300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.940+0000 7fa639756b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c38840 0x55c278c38cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.940+0000 7fa639756b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c4af60 0x55c278c4d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.940+0000 7fa639756b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c278b916d0 con 0x55c278c4af60 2023-09-10T01:15:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.940+0000 7fa639756b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c278b8eff0 con 0x55c278c38840 2023-09-10T01:15:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.941+0000 7fa639756b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c278b8a8e0 con 0x55c278c3c270 2023-09-10T01:15:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.941+0000 7fa626ffd700 1 --2- 172.21.15.44:0/3653181964 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa6100a07b0 0x7fa6100a2c70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa61800b580 tx=0x7fa618005c00 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.941+0000 7fa626ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c3c270 0x55c278c38300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.941+0000 7fa626ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c3c270 0x55c278c38300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36254/0 (socket says 172.21.15.44:36254) 2023-09-10T01:15:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.941+0000 7fa626ffd700 1 -- 172.21.15.44:0/4170562087 learned_addr learned my addr 172.21.15.44:0/4170562087 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.941+0000 7fa6267fc700 1 --2- 172.21.15.44:0/4170562087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c38840 0x55c278c38cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.941+0000 7fa6277fe700 1 --2- 172.21.15.44:0/4170562087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c4af60 0x55c278c4d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.941+0000 7fa6267fc700 1 -- 172.21.15.44:0/4170562087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c3c270 msgr2=0x55c278c38300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.941+0000 7fa6267fc700 1 --2- 172.21.15.44:0/4170562087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c3c270 0x55c278c38300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.941+0000 7fa6267fc700 1 -- 172.21.15.44:0/4170562087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c4af60 msgr2=0x55c278c4d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.941+0000 7fa6267fc700 1 --2- 172.21.15.44:0/4170562087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c4af60 0x55c278c4d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.941+0000 7fa6267fc700 1 -- 172.21.15.44:0/4170562087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2789b6d80 con 0x55c278c38840 2023-09-10T01:15:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.941+0000 7fa6267fc700 1 --2- 172.21.15.44:0/4170562087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c38840 0x55c278c38cc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0c9860 tx=0x7fa61c098f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f866d953eef49f37 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.941+0000 7fa60d7fa700 1 -- 172.21.15.44:0/4170562087 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0995c0 con 0x55c278c38840 2023-09-10T01:15:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.942+0000 7fa60d7fa700 1 -- 172.21.15.44:0/4170562087 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa61c09d3f0 con 0x55c278c38840 2023-09-10T01:15:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.942+0000 7fa60d7fa700 1 -- 172.21.15.44:0/4170562087 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0840b0 con 0x55c278c38840 2023-09-10T01:15:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.942+0000 7fa639756b80 1 -- 172.21.15.44:0/4170562087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c38840 msgr2=0x55c278c38cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.942+0000 7fa639756b80 1 --2- 172.21.15.44:0/4170562087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c38840 0x55c278c38cc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0c9860 tx=0x7fa61c098f20 comp rx=0 tx=0).stop 2023-09-10T01:15:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.942+0000 7fa639756b80 1 -- 172.21.15.44:0/4170562087 shutdown_connections 2023-09-10T01:15:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.942+0000 7fa639756b80 1 --2- 172.21.15.44:0/4170562087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c4af60 0x55c278c4d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.942+0000 7fa639756b80 1 --2- 172.21.15.44:0/4170562087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c3c270 0x55c278c38300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.942+0000 7fa639756b80 1 --2- 172.21.15.44:0/4170562087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c38840 0x55c278c38cc0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.942+0000 7fa639756b80 1 -- 172.21.15.44:0/4170562087 >> 172.21.15.44:0/4170562087 conn(0x55c278bad030 msgr2=0x55c278c33560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.942+0000 7fa639756b80 1 -- 172.21.15.44:0/4170562087 shutdown_connections 2023-09-10T01:15:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.943+0000 7fa639756b80 1 -- 172.21.15.44:0/4170562087 wait complete. 2023-09-10T01:15:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.943+0000 7fa639756b80 1 -- start start 2023-09-10T01:15:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.943+0000 7fa639756b80 1 Processor -- start 2023-09-10T01:15:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.943+0000 7fa639756b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c38840 0x55c278c444c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.943+0000 7fa639756b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c3c270 0x55c278c44a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.943+0000 7fa639756b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c4af60 0x55c278c4a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.943+0000 7fa639756b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c278b28b10 con 0x55c278c38840 2023-09-10T01:15:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.943+0000 7fa639756b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c278b28810 con 0x55c278c3c270 2023-09-10T01:15:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.943+0000 7fa639756b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c278a0c8c0 con 0x55c278c4af60 2023-09-10T01:15:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.943+0000 7fa639756b80 10 client.0 _ll_register_callbacks cb 0x55c278a72a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:15:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa626ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c38840 0x55c278c444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa626ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c38840 0x55c278c444c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57610/0 (socket says 172.21.15.44:57610) 2023-09-10T01:15:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa626ffd700 1 -- 172.21.15.44:0/842543838 learned_addr learned my addr 172.21.15.44:0/842543838 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa6267fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c3c270 0x55c278c44a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa6277fe700 1 --2- 172.21.15.44:0/842543838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c4af60 0x55c278c4a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa6267fc700 1 -- 172.21.15.44:0/842543838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c4af60 msgr2=0x55c278c4a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa6267fc700 1 --2- 172.21.15.44:0/842543838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c4af60 0x55c278c4a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa6267fc700 1 -- 172.21.15.44:0/842543838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c38840 msgr2=0x55c278c444c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa6267fc700 1 --2- 172.21.15.44:0/842543838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c38840 0x55c278c444c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa6267fc700 1 -- 172.21.15.44:0/842543838 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6281a3460 con 0x55c278c3c270 2023-09-10T01:15:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa626ffd700 1 --2- 172.21.15.44:0/842543838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c38840 0x55c278c444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:15:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa6267fc700 1 --2- 172.21.15.44:0/842543838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c3c270 0x55c278c44a00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa61c097910 tx=0x7fa61c010bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7decd1d995d9006e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa60aff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:15:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa60aff5700 1 -- 172.21.15.44:0/842543838 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0a1630 con 0x55c278c3c270 2023-09-10T01:15:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa60aff5700 1 -- 172.21.15.44:0/842543838 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa61c084040 con 0x55c278c3c270 2023-09-10T01:15:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa639756b80 1 -- 172.21.15.44:0/842543838 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c278ae2490 con 0x55c278c3c270 2023-09-10T01:15:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa60aff5700 1 -- 172.21.15.44:0/842543838 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa61c084200 con 0x55c278c3c270 2023-09-10T01:15:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa639756b80 10 client.29159 fetch_fsmap learned FSMap version 122 2023-09-10T01:15:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.944+0000 7fa639756b80 1 -- 172.21.15.44:0/842543838 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=4+}) v3 -- 0x55c27899fc00 con 0x55c278c3c270 2023-09-10T01:15:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.945+0000 7fa60aff5700 1 -- 172.21.15.44:0/842543838 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c278ae2490 con 0x55c278c3c270 2023-09-10T01:15:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.945+0000 7fa60aff5700 1 -- 172.21.15.44:0/842543838 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 122) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fa61c0a11a0 con 0x55c278c3c270 2023-09-10T01:15:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.945+0000 7fa639756b80 10 client.29159 fetch_fsmap finished waiting for FSMap version 122 2023-09-10T01:15:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.945+0000 7fa639756b80 10 client.29159 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:15:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.945+0000 7fa639756b80 20 client.29159 populate_metadata read hostname 'smithi044' 2023-09-10T01:15:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.945+0000 7fa639756b80 1 --2- 172.21.15.44:0/842543838 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c278c630a0 0x55c278c65490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.945+0000 7fa639756b80 4 client.29159 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:15:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.945+0000 7fa639756b80 1 -- 172.21.15.44:0/842543838 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c278b9f840 con 0x55c278c630a0 2023-09-10T01:15:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.945+0000 7fa626ffd700 1 --2- 172.21.15.44:0/842543838 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c278c630a0 0x55c278c65490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.945+0000 7fa60aff5700 10 client.29159 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.946+0000 7fa626ffd700 1 --2- 172.21.15.44:0/842543838 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c278c630a0 0x55c278c65490 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f130d6e6b619dbaa server_cookie=5f21dc59538ce68d in_seq=0 out_seq=0 2023-09-10T01:15:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.946+0000 7fa60aff5700 1 -- 172.21.15.44:0/842543838 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c278b9f840 con 0x55c278c630a0 2023-09-10T01:15:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.946+0000 7fa60aff5700 10 client.29159 handle_command_reply: tid=0 2023-09-10T01:15:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.946+0000 7fa639756b80 1 client.29159 shutdown 2023-09-10T01:15:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.946+0000 7fa639756b80 1 -- 172.21.15.44:0/842543838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c3c270 msgr2=0x55c278c44a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.946+0000 7fa639756b80 1 --2- 172.21.15.44:0/842543838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c3c270 0x55c278c44a00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa61c097910 tx=0x7fa61c010bf0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.946+0000 7fa639756b80 1 -- 172.21.15.44:0/842543838 shutdown_connections 2023-09-10T01:15:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.946+0000 7fa639756b80 1 --2- 172.21.15.44:0/842543838 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c278c630a0 0x55c278c65490 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.946+0000 7fa639756b80 1 --2- 172.21.15.44:0/842543838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c4af60 0x55c278c4a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.946+0000 7fa60aff5700 0 client.29159 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.946+0000 7fa639756b80 1 --2- 172.21.15.44:0/842543838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c38840 0x55c278c444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.946+0000 7fa639756b80 1 --2- 172.21.15.44:0/842543838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c3c270 0x55c278c44a00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.946+0000 7fa639756b80 1 -- 172.21.15.44:0/842543838 >> 172.21.15.44:0/842543838 conn(0x55c278bad030 msgr2=0x55c278c327a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.946+0000 7fa639756b80 1 -- 172.21.15.44:0/842543838 shutdown_connections 2023-09-10T01:15:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.946+0000 7fa639756b80 1 -- 172.21.15.44:0/842543838 wait complete. 2023-09-10T01:15:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.947+0000 7fa639756b80 20 client.29159 trim_cache size 0 max 16384 2023-09-10T01:15:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.966+0000 7fa639756b80 1 Processor -- start 2023-09-10T01:15:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.966+0000 7fa639756b80 1 -- start start 2023-09-10T01:15:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.966+0000 7fa639756b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c38840 0x55c278c3d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.966+0000 7fa639756b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c3c270 0x55c278c3e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.966+0000 7fa639756b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c4af60 0x55c278c32f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.966+0000 7fa639756b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c278c40de0 con 0x55c278c38840 2023-09-10T01:15:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.966+0000 7fa639756b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c278c4a840 con 0x55c278c3c270 2023-09-10T01:15:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.966+0000 7fa639756b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c278c34bb0 con 0x55c278c4af60 2023-09-10T01:15:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.967+0000 7fa626ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c38840 0x55c278c3d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.967+0000 7fa6277fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c4af60 0x55c278c32f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.967+0000 7fa6277fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c4af60 0x55c278c32f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36272/0 (socket says 172.21.15.44:36272) 2023-09-10T01:15:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.967+0000 7fa6277fe700 1 -- 172.21.15.44:0/3429435550 learned_addr learned my addr 172.21.15.44:0/3429435550 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.967+0000 7fa6267fc700 1 --2- 172.21.15.44:0/3429435550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c3c270 0x55c278c3e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.967+0000 7fa626ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c38840 0x55c278c3d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57618/0 (socket says 172.21.15.44:57618) 2023-09-10T01:15:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.967+0000 7fa6277fe700 1 -- 172.21.15.44:0/3429435550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c3c270 msgr2=0x55c278c3e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.967+0000 7fa6277fe700 1 --2- 172.21.15.44:0/3429435550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c3c270 0x55c278c3e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.967+0000 7fa6277fe700 1 -- 172.21.15.44:0/3429435550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c38840 msgr2=0x55c278c3d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.967+0000 7fa6277fe700 1 --2- 172.21.15.44:0/3429435550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c38840 0x55c278c3d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.967+0000 7fa6277fe700 1 -- 172.21.15.44:0/3429435550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2789dd1f0 con 0x55c278c4af60 2023-09-10T01:15:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.967+0000 7fa626ffd700 1 --2- 172.21.15.44:0/3429435550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c38840 0x55c278c3d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:15:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.967+0000 7fa6277fe700 1 --2- 172.21.15.44:0/3429435550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c4af60 0x55c278c32f20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa62000ea90 tx=0x7fa620009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a37de970b0dcc0e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.967+0000 7fa60dffb700 1 -- 172.21.15.44:0/3429435550 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62000eeb0 con 0x55c278c4af60 2023-09-10T01:15:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.967+0000 7fa60dffb700 1 -- 172.21.15.44:0/3429435550 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa62000f040 con 0x55c278c4af60 2023-09-10T01:15:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.967+0000 7fa60dffb700 1 -- 172.21.15.44:0/3429435550 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620007ce0 con 0x55c278c4af60 2023-09-10T01:15:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.968+0000 7fa639756b80 1 -- 172.21.15.44:0/3429435550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c4af60 msgr2=0x55c278c32f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.968+0000 7fa639756b80 1 --2- 172.21.15.44:0/3429435550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c4af60 0x55c278c32f20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa62000ea90 tx=0x7fa620009920 comp rx=0 tx=0).stop 2023-09-10T01:15:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.968+0000 7fa639756b80 1 -- 172.21.15.44:0/3429435550 shutdown_connections 2023-09-10T01:15:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.968+0000 7fa639756b80 1 --2- 172.21.15.44:0/3429435550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c4af60 0x55c278c32f20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.968+0000 7fa639756b80 1 --2- 172.21.15.44:0/3429435550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c38840 0x55c278c3d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.968+0000 7fa639756b80 1 --2- 172.21.15.44:0/3429435550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c3c270 0x55c278c3e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.968+0000 7fa639756b80 1 -- 172.21.15.44:0/3429435550 >> 172.21.15.44:0/3429435550 conn(0x55c278bad030 msgr2=0x55c278c3df40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.968+0000 7fa639756b80 1 -- 172.21.15.44:0/3429435550 shutdown_connections 2023-09-10T01:15:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.968+0000 7fa639756b80 1 -- 172.21.15.44:0/3429435550 wait complete. 2023-09-10T01:15:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.969+0000 7fa639756b80 1 -- start start 2023-09-10T01:15:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.969+0000 7fa639756b80 1 Processor -- start 2023-09-10T01:15:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.969+0000 7fa639756b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c38840 0x55c278c47d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.969+0000 7fa639756b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c3c270 0x55c278c48250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.969+0000 7fa639756b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c4af60 0x55c278cb5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.969+0000 7fa639756b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c278c489a0 con 0x55c278c4af60 2023-09-10T01:15:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.969+0000 7fa639756b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c278cb6100 con 0x55c278c38840 2023-09-10T01:15:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.969+0000 7fa639756b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c278cb62c0 con 0x55c278c3c270 2023-09-10T01:15:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.969+0000 7fa639756b80 10 client.0 _ll_register_callbacks cb 0x7fa620009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:15:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.969+0000 7fa6267fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c3c270 0x55c278c48250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.969+0000 7fa626ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c38840 0x55c278c47d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.969+0000 7fa6267fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c3c270 0x55c278c48250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36276/0 (socket says 172.21.15.44:36276) 2023-09-10T01:15:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.969+0000 7fa6267fc700 1 -- 172.21.15.44:0/2942785382 learned_addr learned my addr 172.21.15.44:0/2942785382 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.970+0000 7fa6277fe700 1 --2- 172.21.15.44:0/2942785382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c4af60 0x55c278cb5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.970+0000 7fa6267fc700 1 -- 172.21.15.44:0/2942785382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c38840 msgr2=0x55c278c47d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.970+0000 7fa6267fc700 1 --2- 172.21.15.44:0/2942785382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c38840 0x55c278c47d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.970+0000 7fa6267fc700 1 -- 172.21.15.44:0/2942785382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c4af60 msgr2=0x55c278cb5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.970+0000 7fa6267fc700 1 --2- 172.21.15.44:0/2942785382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c4af60 0x55c278cb5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.970+0000 7fa6267fc700 1 -- 172.21.15.44:0/2942785382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62810cbc0 con 0x55c278c3c270 2023-09-10T01:15:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.970+0000 7fa6277fe700 1 --2- 172.21.15.44:0/2942785382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c4af60 0x55c278cb5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:15:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.970+0000 7fa6267fc700 1 --2- 172.21.15.44:0/2942785382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c3c270 0x55c278c48250 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa61c099550 tx=0x7fa61c098cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e713e9c47c64df77 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.970+0000 7fa60aff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:15:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.970+0000 7fa60aff5700 1 -- 172.21.15.44:0/2942785382 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0a1630 con 0x55c278c3c270 2023-09-10T01:15:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.970+0000 7fa639756b80 1 -- 172.21.15.44:0/2942785382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa61c09d3f0 con 0x55c278c3c270 2023-09-10T01:15:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.970+0000 7fa60aff5700 1 -- 172.21.15.44:0/2942785382 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa61c0995c0 con 0x55c278c3c270 2023-09-10T01:15:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.970+0000 7fa60aff5700 1 -- 172.21.15.44:0/2942785382 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa61c0650c0 con 0x55c278c3c270 2023-09-10T01:15:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.970+0000 7fa639756b80 10 client.29113 fetch_fsmap learned FSMap version 122 2023-09-10T01:15:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.970+0000 7fa639756b80 1 -- 172.21.15.44:0/2942785382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=4+}) v3 -- 0x7fa62000e740 con 0x55c278c3c270 2023-09-10T01:15:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.971+0000 7fa60aff5700 1 -- 172.21.15.44:0/2942785382 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa61c09d3f0 con 0x55c278c3c270 2023-09-10T01:15:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.971+0000 7fa60aff5700 1 -- 172.21.15.44:0/2942785382 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 122) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fa61c0b1bc0 con 0x55c278c3c270 2023-09-10T01:15:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.971+0000 7fa639756b80 10 client.29113 fetch_fsmap finished waiting for FSMap version 122 2023-09-10T01:15:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.971+0000 7fa639756b80 10 client.29113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:15:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.971+0000 7fa639756b80 20 client.29113 populate_metadata read hostname 'smithi044' 2023-09-10T01:15:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.971+0000 7fa639756b80 1 --2- 172.21.15.44:0/2942785382 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c278c630a0 0x55c278cb9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.971+0000 7fa639756b80 4 client.29113 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:15:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.971+0000 7fa639756b80 1 -- 172.21.15.44:0/2942785382 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c278b838c0 con 0x55c278c630a0 2023-09-10T01:15:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.971+0000 7fa626ffd700 1 --2- 172.21.15.44:0/2942785382 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c278c630a0 0x55c278cb9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.971+0000 7fa60aff5700 10 client.29113 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.971+0000 7fa626ffd700 1 --2- 172.21.15.44:0/2942785382 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c278c630a0 0x55c278cb9660 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2dbeebebec184de server_cookie=b1e9bf5c0a802847 in_seq=0 out_seq=0 2023-09-10T01:15:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.980+0000 7fa60aff5700 1 -- 172.21.15.44:0/2942785382 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c278b838c0 con 0x55c278c630a0 2023-09-10T01:15:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.980+0000 7fa60aff5700 10 client.29113 handle_command_reply: tid=0 2023-09-10T01:15:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.980+0000 7fa639756b80 1 client.29113 shutdown 2023-09-10T01:15:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa639756b80 1 -- 172.21.15.44:0/2942785382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c3c270 msgr2=0x55c278c48250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa639756b80 1 --2- 172.21.15.44:0/2942785382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c3c270 0x55c278c48250 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa61c099550 tx=0x7fa61c098cc0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa639756b80 1 -- 172.21.15.44:0/2942785382 shutdown_connections 2023-09-10T01:15:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa639756b80 1 --2- 172.21.15.44:0/2942785382 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c278c630a0 0x55c278cb9660 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa639756b80 1 --2- 172.21.15.44:0/2942785382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c278c4af60 0x55c278cb5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa60aff5700 0 client.29113 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa639756b80 1 --2- 172.21.15.44:0/2942785382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c278c38840 0x55c278c47d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa639756b80 1 --2- 172.21.15.44:0/2942785382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c278c3c270 0x55c278c48250 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa639756b80 1 -- 172.21.15.44:0/2942785382 >> 172.21.15.44:0/2942785382 conn(0x55c278bad030 msgr2=0x55c278c3e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa639756b80 1 -- 172.21.15.44:0/2942785382 shutdown_connections 2023-09-10T01:15:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa639756b80 1 -- 172.21.15.44:0/2942785382 wait complete. 2023-09-10T01:15:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa639756b80 20 client.29113 trim_cache size 0 max 16384 2023-09-10T01:15:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:15:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:15:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:15:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:15:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa62d590700 1 -- 172.21.15.44:0/3653181964 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa6100a07b0 msgr2=0x7fa6100a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa62d590700 1 --2- 172.21.15.44:0/3653181964 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa6100a07b0 0x7fa6100a2c70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa61800b580 tx=0x7fa618005c00 comp rx=0 tx=0).stop 2023-09-10T01:15:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa62d590700 1 -- 172.21.15.44:0/3653181964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa628100120 msgr2=0x7fa628197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa62d590700 1 --2- 172.21.15.44:0/3653181964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa628100120 0x7fa628197aa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00e9a0 tx=0x7fa61c00ecb0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa62d590700 1 -- 172.21.15.44:0/3653181964 shutdown_connections 2023-09-10T01:15:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa62d590700 1 --2- 172.21.15.44:0/3653181964 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa6100a07b0 0x7fa6100a2c70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.981+0000 7fa62d590700 1 --2- 172.21.15.44:0/3653181964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6281055b0 0x7fa6281a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.982+0000 7fa62d590700 1 --2- 172.21.15.44:0/3653181964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6280ff760 0x7fa628197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.982+0000 7fa62d590700 1 --2- 172.21.15.44:0/3653181964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa628100120 0x7fa628197aa0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.982+0000 7fa62d590700 1 -- 172.21.15.44:0/3653181964 >> 172.21.15.44:0/3653181964 conn(0x7fa628074f30 msgr2=0x7fa6280fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.982+0000 7fa62d590700 1 -- 172.21.15.44:0/3653181964 shutdown_connections 2023-09-10T01:15:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:42.982+0000 7fa62d590700 1 -- 172.21.15.44:0/3653181964 wait complete. 2023-09-10T01:15:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:43 smithi131 ceph-mon[103713]: pgmap v1122: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2023-09-10T01:15:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:43 smithi044 ceph-mon[98676]: pgmap v1122: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2023-09-10T01:15:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:43 smithi099 ceph-mon[102692]: pgmap v1122: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2023-09-10T01:15:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:45 smithi044 ceph-mon[98676]: pgmap v1123: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 876 KiB/s wr, 125 op/s 2023-09-10T01:15:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:45 smithi099 ceph-mon[102692]: pgmap v1123: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 876 KiB/s wr, 125 op/s 2023-09-10T01:15:45.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:45 smithi131 ceph-mon[103713]: pgmap v1123: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 876 KiB/s wr, 125 op/s 2023-09-10T01:15:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:46 smithi131 ceph-mon[103713]: pgmap v1124: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 203 op/s 2023-09-10T01:15:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:46 smithi044 ceph-mon[98676]: pgmap v1124: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 203 op/s 2023-09-10T01:15:47.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:46 smithi099 ceph-mon[102692]: pgmap v1124: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 203 op/s 2023-09-10T01:15:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:49 smithi131 ceph-mon[103713]: pgmap v1125: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2023-09-10T01:15:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:49 smithi044 ceph-mon[98676]: pgmap v1125: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2023-09-10T01:15:49.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:49 smithi099 ceph-mon[102692]: pgmap v1125: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2023-09-10T01:15:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:51 smithi131 ceph-mon[103713]: pgmap v1126: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2023-09-10T01:15:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:51 smithi044 ceph-mon[98676]: pgmap v1126: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2023-09-10T01:15:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:51 smithi099 ceph-mon[102692]: pgmap v1126: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2023-09-10T01:15:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:53 smithi044 ceph-mon[98676]: pgmap v1127: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2023-09-10T01:15:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:53 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:15:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:53 smithi099 ceph-mon[102692]: pgmap v1127: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2023-09-10T01:15:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:53 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:15:53.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:53 smithi131 ceph-mon[103713]: pgmap v1127: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2023-09-10T01:15:53.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:53 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:15:54.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:54 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:15:54.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:54 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:15:54.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:54 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:15:54.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:54 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:15:54.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:54 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:15:54.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:54 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:15:54.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:54 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:15:54.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:54 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:15:54.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:54 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:15:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:55 smithi044 ceph-mon[98676]: pgmap v1128: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2023-09-10T01:15:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:55 smithi044 ceph-mon[98676]: pgmap v1129: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:15:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:55 smithi099 ceph-mon[102692]: pgmap v1128: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2023-09-10T01:15:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:55 smithi099 ceph-mon[102692]: pgmap v1129: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:15:55.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:55 smithi131 ceph-mon[103713]: pgmap v1128: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2023-09-10T01:15:55.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:55 smithi131 ceph-mon[103713]: pgmap v1129: 129 pgs: 129 active+clean; 11 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:15:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.109+0000 7fc37299a700 1 -- 172.21.15.44:0/2116373375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc36c0729d0 msgr2=0x7fc36c072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.109+0000 7fc37299a700 1 --2- 172.21.15.44:0/2116373375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc36c0729d0 0x7fc36c072db0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009990 tx=0x7fc35c009ca0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.110+0000 7fc37299a700 1 -- 172.21.15.44:0/2116373375 shutdown_connections 2023-09-10T01:15:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.110+0000 7fc37299a700 1 --2- 172.21.15.44:0/2116373375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c10ce10 0x7fc36c10f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.110+0000 7fc37299a700 1 --2- 172.21.15.44:0/2116373375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc36c0729d0 0x7fc36c072db0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.110+0000 7fc37299a700 1 --2- 172.21.15.44:0/2116373375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc36c0732f0 0x7fc36c10c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.110+0000 7fc37299a700 1 -- 172.21.15.44:0/2116373375 >> 172.21.15.44:0/2116373375 conn(0x7fc36c0fc060 msgr2=0x7fc36c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.110+0000 7fc37299a700 1 -- 172.21.15.44:0/2116373375 shutdown_connections 2023-09-10T01:15:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.110+0000 7fc37299a700 1 -- 172.21.15.44:0/2116373375 wait complete. 2023-09-10T01:15:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.110+0000 7fc37299a700 1 Processor -- start 2023-09-10T01:15:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.110+0000 7fc37299a700 1 -- start start 2023-09-10T01:15:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc37299a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c0729d0 0x7fc36c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc37299a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc36c0732f0 0x7fc36c198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc37299a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc36c10ce10 0x7fc36c19d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc37299a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c19d950 con 0x7fc36c10ce10 2023-09-10T01:15:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc37299a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c19dac0 con 0x7fc36c0729d0 2023-09-10T01:15:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc37299a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c19dc80 con 0x7fc36c0732f0 2023-09-10T01:15:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc370f37700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc36c10ce10 0x7fc36c19d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc36bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c0729d0 0x7fc36c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc36bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c0729d0 0x7fc36c1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:59376/0 (socket says 172.21.15.44:59376) 2023-09-10T01:15:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc370f37700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc36c10ce10 0x7fc36c19d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57638/0 (socket says 172.21.15.44:57638) 2023-09-10T01:15:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc36bfff700 1 -- 172.21.15.44:0/1363757164 learned_addr learned my addr 172.21.15.44:0/1363757164 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc36b7fe700 1 --2- 172.21.15.44:0/1363757164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc36c0732f0 0x7fc36c198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc36bfff700 1 -- 172.21.15.44:0/1363757164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc36c0732f0 msgr2=0x7fc36c198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc36bfff700 1 --2- 172.21.15.44:0/1363757164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc36c0732f0 0x7fc36c198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc36bfff700 1 -- 172.21.15.44:0/1363757164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc36c10ce10 msgr2=0x7fc36c19d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc36bfff700 1 --2- 172.21.15.44:0/1363757164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc36c10ce10 0x7fc36c19d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc36bfff700 1 -- 172.21.15.44:0/1363757164 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c0095f0 con 0x7fc36c0729d0 2023-09-10T01:15:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc370f37700 1 --2- 172.21.15.44:0/1363757164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc36c10ce10 0x7fc36c19d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:15:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.111+0000 7fc36bfff700 1 --2- 172.21.15.44:0/1363757164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c0729d0 0x7fc36c1987a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc35c005910 tx=0x7fc35c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.112+0000 7fc3697fa700 1 -- 172.21.15.44:0/1363757164 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c01d070 con 0x7fc36c0729d0 2023-09-10T01:15:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.112+0000 7fc3697fa700 1 -- 172.21.15.44:0/1363757164 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc35c021410 con 0x7fc36c0729d0 2023-09-10T01:15:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.112+0000 7fc37299a700 1 -- 172.21.15.44:0/1363757164 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36c19df00 con 0x7fc36c0729d0 2023-09-10T01:15:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.112+0000 7fc3697fa700 1 -- 172.21.15.44:0/1363757164 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c017650 con 0x7fc36c0729d0 2023-09-10T01:15:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.112+0000 7fc37299a700 1 -- 172.21.15.44:0/1363757164 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c19e1d0 con 0x7fc36c0729d0 2023-09-10T01:15:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.114+0000 7fc3697fa700 1 -- 172.21.15.44:0/1363757164 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc35c021580 con 0x7fc36c0729d0 2023-09-10T01:15:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.115+0000 7fc3697fa700 1 --2- 172.21.15.44:0/1363757164 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc3540a07b0 0x7fc3540a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.115+0000 7fc3697fa700 1 -- 172.21.15.44:0/1363757164 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc35c0cd480 con 0x7fc36c0729d0 2023-09-10T01:15:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.115+0000 7fc36b7fe700 1 --2- 172.21.15.44:0/1363757164 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc3540a07b0 0x7fc3540a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.116+0000 7fc37eb60b80 1 Processor -- start 2023-09-10T01:15:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.116+0000 7fc37eb60b80 1 -- start start 2023-09-10T01:15:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.116+0000 7fc36b7fe700 1 --2- 172.21.15.44:0/1363757164 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc3540a07b0 0x7fc3540a2c70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc36c199fc0 tx=0x7fc360009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.116+0000 7fc37eb60b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e15580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.116+0000 7fc37eb60b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0f300 0x55c238e0f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.116+0000 7fc37eb60b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0fcc0 0x55c238e24740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.116+0000 7fc37eb60b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c238d686d0 con 0x55c238e0fcc0 2023-09-10T01:15:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.116+0000 7fc37eb60b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c238d65ff0 con 0x55c238e0f300 2023-09-10T01:15:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.116+0000 7fc37eb60b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c238d618e0 con 0x55c238e151a0 2023-09-10T01:15:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.116+0000 7fc370f37700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0fcc0 0x55c238e24740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.116+0000 7fc370f37700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0fcc0 0x55c238e24740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57648/0 (socket says 172.21.15.44:57648) 2023-09-10T01:15:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.116+0000 7fc370f37700 1 -- 172.21.15.44:0/2324691764 learned_addr learned my addr 172.21.15.44:0/2324691764 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.116+0000 7fc36bfff700 1 --2- 172.21.15.44:0/2324691764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e15580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.116+0000 7fc36b7fe700 1 --2- 172.21.15.44:0/2324691764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0f300 0x55c238e0f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.117+0000 7fc370f37700 1 -- 172.21.15.44:0/2324691764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 msgr2=0x55c238e15580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.117+0000 7fc370f37700 1 --2- 172.21.15.44:0/2324691764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e15580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.117+0000 7fc370f37700 1 -- 172.21.15.44:0/2324691764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0f300 msgr2=0x55c238e0f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.117+0000 7fc370f37700 1 --2- 172.21.15.44:0/2324691764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0f300 0x55c238e0f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.117+0000 7fc370f37700 1 -- 172.21.15.44:0/2324691764 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c238b8dd80 con 0x55c238e0fcc0 2023-09-10T01:15:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.117+0000 7fc370f37700 1 --2- 172.21.15.44:0/2324691764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0fcc0 0x55c238e24740 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc364003e40 tx=0x7fc364009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60c0f95eb799eff server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.117+0000 7fc3527fc700 1 -- 172.21.15.44:0/2324691764 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3640092d0 con 0x55c238e0fcc0 2023-09-10T01:15:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.117+0000 7fc3527fc700 1 -- 172.21.15.44:0/2324691764 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc36400f040 con 0x55c238e0fcc0 2023-09-10T01:15:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.117+0000 7fc3527fc700 1 -- 172.21.15.44:0/2324691764 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36400e710 con 0x55c238e0fcc0 2023-09-10T01:15:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.118+0000 7fc37eb60b80 1 -- 172.21.15.44:0/2324691764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0fcc0 msgr2=0x55c238e24740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.118+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/2324691764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0fcc0 0x55c238e24740 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc364003e40 tx=0x7fc364009920 comp rx=0 tx=0).stop 2023-09-10T01:15:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.118+0000 7fc37eb60b80 1 -- 172.21.15.44:0/2324691764 shutdown_connections 2023-09-10T01:15:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.118+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/2324691764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0fcc0 0x55c238e24740 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.118+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/2324691764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e15580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.118+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/2324691764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0f300 0x55c238e0f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.118+0000 7fc37eb60b80 1 -- 172.21.15.44:0/2324691764 >> 172.21.15.44:0/2324691764 conn(0x55c238d84030 msgr2=0x55c238e0a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.118+0000 7fc37eb60b80 1 -- 172.21.15.44:0/2324691764 shutdown_connections 2023-09-10T01:15:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.118+0000 7fc37eb60b80 1 -- 172.21.15.44:0/2324691764 wait complete. 2023-09-10T01:15:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.118+0000 7fc37eb60b80 1 -- start start 2023-09-10T01:15:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc37eb60b80 1 Processor -- start 2023-09-10T01:15:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc37eb60b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0f300 0x55c238e24650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc37eb60b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0fcc0 0x55c238e21940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc37eb60b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e21fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc37eb60b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c238cffb10 con 0x55c238e0fcc0 2023-09-10T01:15:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc37eb60b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c238cff810 con 0x55c238e0f300 2023-09-10T01:15:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc37eb60b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c238be38c0 con 0x55c238e151a0 2023-09-10T01:15:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc37eb60b80 10 client.0 _ll_register_callbacks cb 0x55c238c49a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:15:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc36b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0fcc0 0x55c238e21940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc36b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0fcc0 0x55c238e21940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57652/0 (socket says 172.21.15.44:57652) 2023-09-10T01:15:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc36b7fe700 1 -- 172.21.15.44:0/2163118000 learned_addr learned my addr 172.21.15.44:0/2163118000 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc370f37700 1 --2- 172.21.15.44:0/2163118000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e21fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc36b7fe700 1 -- 172.21.15.44:0/2163118000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 msgr2=0x55c238e21fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc36b7fe700 1 --2- 172.21.15.44:0/2163118000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e21fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc36b7fe700 1 -- 172.21.15.44:0/2163118000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0f300 msgr2=0x55c238e24650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:15:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc36b7fe700 1 --2- 172.21.15.44:0/2163118000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0f300 0x55c238e24650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc36b7fe700 1 -- 172.21.15.44:0/2163118000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc364003b20 con 0x55c238e0fcc0 2023-09-10T01:15:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.119+0000 7fc36b7fe700 1 --2- 172.21.15.44:0/2163118000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0fcc0 0x55c238e21940 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc360004520 tx=0x7fc3600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff979726743150c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.120+0000 7fc34b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:15:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.120+0000 7fc34b7fe700 1 -- 172.21.15.44:0/2163118000 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360022070 con 0x55c238e0fcc0 2023-09-10T01:15:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.120+0000 7fc34b7fe700 1 -- 172.21.15.44:0/2163118000 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc360004550 con 0x55c238e0fcc0 2023-09-10T01:15:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.120+0000 7fc37eb60b80 1 -- 172.21.15.44:0/2163118000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c238cb9490 con 0x55c238e0fcc0 2023-09-10T01:15:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.120+0000 7fc34b7fe700 1 -- 172.21.15.44:0/2163118000 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc360003e20 con 0x55c238e0fcc0 2023-09-10T01:15:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.120+0000 7fc37eb60b80 10 client.19515 fetch_fsmap learned FSMap version 122 2023-09-10T01:15:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.120+0000 7fc37eb60b80 1 -- 172.21.15.44:0/2163118000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=4+}) v3 -- 0x55c238b76c00 con 0x55c238e0fcc0 2023-09-10T01:15:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.120+0000 7fc34b7fe700 1 -- 172.21.15.44:0/2163118000 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c238cb9490 con 0x55c238e0fcc0 2023-09-10T01:15:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.120+0000 7fc34b7fe700 1 -- 172.21.15.44:0/2163118000 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 122) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fc36001c5f0 con 0x55c238e0fcc0 2023-09-10T01:15:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.120+0000 7fc37eb60b80 10 client.19515 fetch_fsmap finished waiting for FSMap version 122 2023-09-10T01:15:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.120+0000 7fc37eb60b80 10 client.19515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:15:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.120+0000 7fc37eb60b80 20 client.19515 populate_metadata read hostname 'smithi044' 2023-09-10T01:15:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.120+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/2163118000 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c238e31d40 0x55c238e34180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.120+0000 7fc37eb60b80 4 client.19515 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:15:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.120+0000 7fc37eb60b80 1 -- 172.21.15.44:0/2163118000 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c238d76840 con 0x55c238e31d40 2023-09-10T01:15:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.121+0000 7fc36bfff700 1 --2- 172.21.15.44:0/2163118000 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c238e31d40 0x55c238e34180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.121+0000 7fc34b7fe700 10 client.19515 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.121+0000 7fc36bfff700 1 --2- 172.21.15.44:0/2163118000 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c238e31d40 0x55c238e34180 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a1d4a9e49e4749 server_cookie=2fc65456cc7d9a74 in_seq=0 out_seq=0 2023-09-10T01:15:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.122+0000 7fc34b7fe700 1 -- 172.21.15.44:0/2163118000 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c238d76840 con 0x55c238e31d40 2023-09-10T01:15:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.122+0000 7fc34b7fe700 10 client.19515 handle_command_reply: tid=0 2023-09-10T01:15:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.122+0000 7fc37eb60b80 1 client.19515 shutdown 2023-09-10T01:15:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.122+0000 7fc37eb60b80 1 -- 172.21.15.44:0/2163118000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0fcc0 msgr2=0x55c238e21940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.122+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/2163118000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0fcc0 0x55c238e21940 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc360004520 tx=0x7fc3600047c0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.122+0000 7fc37eb60b80 1 -- 172.21.15.44:0/2163118000 shutdown_connections 2023-09-10T01:15:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.122+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/2163118000 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c238e31d40 0x55c238e34180 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.122+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/2163118000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e21fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.122+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/2163118000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0f300 0x55c238e24650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.122+0000 7fc34b7fe700 0 client.19515 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.122+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/2163118000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0fcc0 0x55c238e21940 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.122+0000 7fc37eb60b80 1 -- 172.21.15.44:0/2163118000 >> 172.21.15.44:0/2163118000 conn(0x55c238d84030 msgr2=0x55c238e097a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.122+0000 7fc37eb60b80 1 -- 172.21.15.44:0/2163118000 shutdown_connections 2023-09-10T01:15:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.122+0000 7fc37eb60b80 1 -- 172.21.15.44:0/2163118000 wait complete. 2023-09-10T01:15:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.122+0000 7fc37eb60b80 20 client.19515 trim_cache size 0 max 16384 2023-09-10T01:15:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.143+0000 7fc37eb60b80 1 Processor -- start 2023-09-10T01:15:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.143+0000 7fc37eb60b80 1 -- start start 2023-09-10T01:15:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.143+0000 7fc37eb60b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0f300 0x55c238e28d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.143+0000 7fc37eb60b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0fcc0 0x55c238e21730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.143+0000 7fc37eb60b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e0ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.143+0000 7fc37eb60b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c238cb94c0 con 0x55c238e0f300 2023-09-10T01:15:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.143+0000 7fc37eb60b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c238e105b0 con 0x55c238e0fcc0 2023-09-10T01:15:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.143+0000 7fc37eb60b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c238e0bd90 con 0x55c238e151a0 2023-09-10T01:15:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.143+0000 7fc36bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0f300 0x55c238e28d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.143+0000 7fc36bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0f300 0x55c238e28d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57658/0 (socket says 172.21.15.44:57658) 2023-09-10T01:15:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.143+0000 7fc36bfff700 1 -- 172.21.15.44:0/1770955127 learned_addr learned my addr 172.21.15.44:0/1770955127 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.143+0000 7fc370f37700 1 --2- 172.21.15.44:0/1770955127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e0ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.143+0000 7fc36b7fe700 1 --2- 172.21.15.44:0/1770955127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0fcc0 0x55c238e21730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.144+0000 7fc36bfff700 1 -- 172.21.15.44:0/1770955127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 msgr2=0x55c238e0ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.144+0000 7fc36bfff700 1 --2- 172.21.15.44:0/1770955127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e0ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.144+0000 7fc36bfff700 1 -- 172.21.15.44:0/1770955127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0fcc0 msgr2=0x55c238e21730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.144+0000 7fc36bfff700 1 --2- 172.21.15.44:0/1770955127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0fcc0 0x55c238e21730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.144+0000 7fc36bfff700 1 -- 172.21.15.44:0/1770955127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c238bb41f0 con 0x55c238e0f300 2023-09-10T01:15:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.144+0000 7fc36bfff700 1 --2- 172.21.15.44:0/1770955127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0f300 0x55c238e28d40 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0ca8e0 tx=0x7fc35c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e14579e1ac3186 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.144+0000 7fc352ffd700 1 -- 172.21.15.44:0/1770955127 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c09a620 con 0x55c238e0f300 2023-09-10T01:15:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.144+0000 7fc352ffd700 1 -- 172.21.15.44:0/1770955127 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc35c09e450 con 0x55c238e0f300 2023-09-10T01:15:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.144+0000 7fc352ffd700 1 -- 172.21.15.44:0/1770955127 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0cd2a0 con 0x55c238e0f300 2023-09-10T01:15:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.145+0000 7fc37eb60b80 1 -- 172.21.15.44:0/1770955127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0f300 msgr2=0x55c238e28d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.145+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/1770955127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0f300 0x55c238e28d40 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0ca8e0 tx=0x7fc35c009ca0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.145+0000 7fc37eb60b80 1 -- 172.21.15.44:0/1770955127 shutdown_connections 2023-09-10T01:15:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.145+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/1770955127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e0ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.145+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/1770955127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0f300 0x55c238e28d40 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.145+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/1770955127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0fcc0 0x55c238e21730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.145+0000 7fc37eb60b80 1 -- 172.21.15.44:0/1770955127 >> 172.21.15.44:0/1770955127 conn(0x55c238d84030 msgr2=0x55c238e0e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.145+0000 7fc37eb60b80 1 -- 172.21.15.44:0/1770955127 shutdown_connections 2023-09-10T01:15:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.145+0000 7fc37eb60b80 1 -- 172.21.15.44:0/1770955127 wait complete. 2023-09-10T01:15:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.145+0000 7fc37eb60b80 1 -- start start 2023-09-10T01:15:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.145+0000 7fc37eb60b80 1 Processor -- start 2023-09-10T01:15:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc37eb60b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0f300 0x55c238e28870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc37eb60b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0fcc0 0x55c238e26d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc37eb60b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e27260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc37eb60b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c238e29140 con 0x55c238e0f300 2023-09-10T01:15:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc37eb60b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c238e277a0 con 0x55c238e0fcc0 2023-09-10T01:15:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc37eb60b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c238e27960 con 0x55c238e151a0 2023-09-10T01:15:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc37eb60b80 10 client.0 _ll_register_callbacks cb 0x55c238c49a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:15:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc36bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0f300 0x55c238e28870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc36bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0f300 0x55c238e28870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57664/0 (socket says 172.21.15.44:57664) 2023-09-10T01:15:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc36bfff700 1 -- 172.21.15.44:0/3809051379 learned_addr learned my addr 172.21.15.44:0/3809051379 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:15:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc36b7fe700 1 --2- 172.21.15.44:0/3809051379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0fcc0 0x55c238e26d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc370f37700 1 --2- 172.21.15.44:0/3809051379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e27260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc36bfff700 1 -- 172.21.15.44:0/3809051379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 msgr2=0x55c238e27260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc36bfff700 1 --2- 172.21.15.44:0/3809051379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e27260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc36bfff700 1 -- 172.21.15.44:0/3809051379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0fcc0 msgr2=0x55c238e26d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:15:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc36bfff700 1 --2- 172.21.15.44:0/3809051379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0fcc0 0x55c238e26d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc36bfff700 1 -- 172.21.15.44:0/3809051379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c19e1d0 con 0x55c238e0f300 2023-09-10T01:15:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc36b7fe700 1 --2- 172.21.15.44:0/3809051379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0fcc0 0x55c238e26d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:15:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc370f37700 1 --2- 172.21.15.44:0/3809051379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e27260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:15:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc36bfff700 1 --2- 172.21.15.44:0/3809051379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0f300 0x55c238e28870 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0cadb0 tx=0x7fc35c09ae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab368b69e71e4774 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:15:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc34b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:15:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.146+0000 7fc34b7fe700 1 -- 172.21.15.44:0/3809051379 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0b6070 con 0x55c238e0f300 2023-09-10T01:15:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.147+0000 7fc34b7fe700 1 -- 172.21.15.44:0/3809051379 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fc35c0a0040 con 0x55c238e0f300 2023-09-10T01:15:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.147+0000 7fc37eb60b80 1 -- 172.21.15.44:0/3809051379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc36400f040 con 0x55c238e0f300 2023-09-10T01:15:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.147+0000 7fc34b7fe700 1 -- 172.21.15.44:0/3809051379 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc35c0cd150 con 0x55c238e0f300 2023-09-10T01:15:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.147+0000 7fc37eb60b80 10 client.19521 fetch_fsmap learned FSMap version 122 2023-09-10T01:15:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.147+0000 7fc37eb60b80 1 -- 172.21.15.44:0/3809051379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=4+}) v3 -- 0x55c238b8c250 con 0x55c238e0f300 2023-09-10T01:15:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.147+0000 7fc34b7fe700 1 -- 172.21.15.44:0/3809051379 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc36400f040 con 0x55c238e0f300 2023-09-10T01:15:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.147+0000 7fc34b7fe700 1 -- 172.21.15.44:0/3809051379 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 122) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fc35c09e0e0 con 0x55c238e0f300 2023-09-10T01:15:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.147+0000 7fc37eb60b80 10 client.19521 fetch_fsmap finished waiting for FSMap version 122 2023-09-10T01:15:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.147+0000 7fc37eb60b80 10 client.19521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:15:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.147+0000 7fc37eb60b80 20 client.19521 populate_metadata read hostname 'smithi044' 2023-09-10T01:15:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.147+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/3809051379 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c238e86a70 0x55c238e88e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:15:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.147+0000 7fc37eb60b80 4 client.19521 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:15:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.147+0000 7fc37eb60b80 1 -- 172.21.15.44:0/3809051379 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c238d5a8c0 con 0x55c238e86a70 2023-09-10T01:15:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.147+0000 7fc36b7fe700 1 --2- 172.21.15.44:0/3809051379 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c238e86a70 0x55c238e88e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:15:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.148+0000 7fc34b7fe700 10 client.19521 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.148+0000 7fc36b7fe700 1 --2- 172.21.15.44:0/3809051379 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c238e86a70 0x55c238e88e60 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab7843181b1b6a03 server_cookie=eec9ebe2da171fb1 in_seq=0 out_seq=0 2023-09-10T01:15:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.159+0000 7fc34b7fe700 1 -- 172.21.15.44:0/3809051379 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c238d5a8c0 con 0x55c238e86a70 2023-09-10T01:15:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.159+0000 7fc34b7fe700 10 client.19521 handle_command_reply: tid=0 2023-09-10T01:15:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.159+0000 7fc37eb60b80 1 client.19521 shutdown 2023-09-10T01:15:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.159+0000 7fc37eb60b80 1 -- 172.21.15.44:0/3809051379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0f300 msgr2=0x55c238e28870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.159+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/3809051379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0f300 0x55c238e28870 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0cadb0 tx=0x7fc35c09ae00 comp rx=0 tx=0).stop 2023-09-10T01:15:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.159+0000 7fc37eb60b80 1 -- 172.21.15.44:0/3809051379 shutdown_connections 2023-09-10T01:15:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.159+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/3809051379 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c238e86a70 0x55c238e88e60 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.159+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/3809051379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c238e151a0 0x55c238e27260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.159+0000 7fc34b7fe700 0 client.19521 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:15:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.159+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/3809051379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c238e0f300 0x55c238e28870 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.159+0000 7fc37eb60b80 1 --2- 172.21.15.44:0/3809051379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c238e0fcc0 0x55c238e26d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.159+0000 7fc37eb60b80 1 -- 172.21.15.44:0/3809051379 >> 172.21.15.44:0/3809051379 conn(0x55c238d84030 msgr2=0x55c238e0eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.159+0000 7fc37eb60b80 1 -- 172.21.15.44:0/3809051379 shutdown_connections 2023-09-10T01:15:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.159+0000 7fc37eb60b80 1 -- 172.21.15.44:0/3809051379 wait complete. 2023-09-10T01:15:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.159+0000 7fc37eb60b80 20 client.19521 trim_cache size 0 max 16384 2023-09-10T01:15:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:15:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:15:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:15:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:15:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.160+0000 7fc37299a700 1 -- 172.21.15.44:0/1363757164 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc3540a07b0 msgr2=0x7fc3540a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.160+0000 7fc37299a700 1 --2- 172.21.15.44:0/1363757164 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc3540a07b0 0x7fc3540a2c70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc36c199fc0 tx=0x7fc360009260 comp rx=0 tx=0).stop 2023-09-10T01:15:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.160+0000 7fc37299a700 1 -- 172.21.15.44:0/1363757164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c0729d0 msgr2=0x7fc36c1987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:15:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.160+0000 7fc37299a700 1 --2- 172.21.15.44:0/1363757164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c0729d0 0x7fc36c1987a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc35c005910 tx=0x7fc35c0047b0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.160+0000 7fc37299a700 1 -- 172.21.15.44:0/1363757164 shutdown_connections 2023-09-10T01:15:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.160+0000 7fc37299a700 1 --2- 172.21.15.44:0/1363757164 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc3540a07b0 0x7fc3540a2c70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.160+0000 7fc37299a700 1 --2- 172.21.15.44:0/1363757164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc36c10ce10 0x7fc36c19d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.160+0000 7fc37299a700 1 --2- 172.21.15.44:0/1363757164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c0729d0 0x7fc36c1987a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.160+0000 7fc37299a700 1 --2- 172.21.15.44:0/1363757164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc36c0732f0 0x7fc36c198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:15:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.160+0000 7fc37299a700 1 -- 172.21.15.44:0/1363757164 >> 172.21.15.44:0/1363757164 conn(0x7fc36c0fc060 msgr2=0x7fc36c10efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:15:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.160+0000 7fc37299a700 1 -- 172.21.15.44:0/1363757164 shutdown_connections 2023-09-10T01:15:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:15:56.160+0000 7fc37299a700 1 -- 172.21.15.44:0/1363757164 wait complete. 2023-09-10T01:15:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:57 smithi044 ceph-mon[98676]: pgmap v1130: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-09-10T01:15:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:57 smithi099 ceph-mon[102692]: pgmap v1130: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-09-10T01:15:57.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:57 smithi131 ceph-mon[103713]: pgmap v1130: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-09-10T01:15:59.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:15:59 smithi099 ceph-mon[102692]: pgmap v1131: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-09-10T01:15:59.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:15:59 smithi131 ceph-mon[103713]: pgmap v1131: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-09-10T01:15:59.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:15:59 smithi044 ceph-mon[98676]: pgmap v1131: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-09-10T01:16:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:01 smithi099 ceph-mon[102692]: pgmap v1132: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-09-10T01:16:01.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:01 smithi131 ceph-mon[103713]: pgmap v1132: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-09-10T01:16:01.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:01 smithi044 ceph-mon[98676]: pgmap v1132: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-09-10T01:16:03.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:03 smithi099 ceph-mon[102692]: pgmap v1133: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 182 op/s 2023-09-10T01:16:03.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:03 smithi131 ceph-mon[103713]: pgmap v1133: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 182 op/s 2023-09-10T01:16:03.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:03 smithi044 ceph-mon[98676]: pgmap v1133: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 182 op/s 2023-09-10T01:16:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:05 smithi099 ceph-mon[102692]: pgmap v1134: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 173 op/s 2023-09-10T01:16:05.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:05 smithi131 ceph-mon[103713]: pgmap v1134: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 173 op/s 2023-09-10T01:16:05.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:05 smithi044 ceph-mon[98676]: pgmap v1134: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 173 op/s 2023-09-10T01:16:07.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:07 smithi099 ceph-mon[102692]: pgmap v1135: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 237 op/s 2023-09-10T01:16:07.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:07 smithi131 ceph-mon[103713]: pgmap v1135: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 237 op/s 2023-09-10T01:16:07.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:07 smithi044 ceph-mon[98676]: pgmap v1135: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 237 op/s 2023-09-10T01:16:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.330+0000 7f99c45fb700 1 -- 172.21.15.44:0/4119105109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99bc1055b0 msgr2=0x7f99bc105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.330+0000 7f99c45fb700 1 --2- 172.21.15.44:0/4119105109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99bc1055b0 0x7f99bc105990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009990 tx=0x7f99b8009ca0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.330+0000 7f99c45fb700 1 -- 172.21.15.44:0/4119105109 shutdown_connections 2023-09-10T01:16:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.330+0000 7f99c45fb700 1 --2- 172.21.15.44:0/4119105109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99bc100120 0x7f99bc10c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.330+0000 7f99c45fb700 1 --2- 172.21.15.44:0/4119105109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99bc1055b0 0x7f99bc105990 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.330+0000 7f99c45fb700 1 --2- 172.21.15.44:0/4119105109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99bc0ff760 0x7f99bc0ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.330+0000 7f99c45fb700 1 -- 172.21.15.44:0/4119105109 >> 172.21.15.44:0/4119105109 conn(0x7f99bc074f30 msgr2=0x7f99bc075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.330+0000 7f99c45fb700 1 -- 172.21.15.44:0/4119105109 shutdown_connections 2023-09-10T01:16:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.330+0000 7f99c45fb700 1 -- 172.21.15.44:0/4119105109 wait complete. 2023-09-10T01:16:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.330+0000 7f99c45fb700 1 Processor -- start 2023-09-10T01:16:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.330+0000 7f99c45fb700 1 -- start start 2023-09-10T01:16:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c45fb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99bc0ff760 0x7f99bc114600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c45fb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99bc100120 0x7f99bc114b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c45fb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99bc1055b0 0x7f99bc11c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c45fb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc10f740 con 0x7f99bc0ff760 2023-09-10T01:16:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c45fb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc10f8b0 con 0x7f99bc100120 2023-09-10T01:16:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c45fb700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc10fa70 con 0x7f99bc1055b0 2023-09-10T01:16:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c2397700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99bc0ff760 0x7f99bc114600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c1b96700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99bc100120 0x7f99bc114b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c2397700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99bc0ff760 0x7f99bc114600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57678/0 (socket says 172.21.15.44:57678) 2023-09-10T01:16:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c2397700 1 -- 172.21.15.44:0/1566576269 learned_addr learned my addr 172.21.15.44:0/1566576269 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c2b98700 1 --2- 172.21.15.44:0/1566576269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99bc1055b0 0x7f99bc11c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c1b96700 1 -- 172.21.15.44:0/1566576269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99bc1055b0 msgr2=0x7f99bc11c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c1b96700 1 --2- 172.21.15.44:0/1566576269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99bc1055b0 0x7f99bc11c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c1b96700 1 -- 172.21.15.44:0/1566576269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99bc0ff760 msgr2=0x7f99bc114600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c1b96700 1 --2- 172.21.15.44:0/1566576269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99bc0ff760 0x7f99bc114600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c1b96700 1 -- 172.21.15.44:0/1566576269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b4009c40 con 0x7f99bc100120 2023-09-10T01:16:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c2397700 1 --2- 172.21.15.44:0/1566576269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99bc0ff760 0x7f99bc114600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:16:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.331+0000 7f99c1b96700 1 --2- 172.21.15.44:0/1566576269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99bc100120 0x7f99bc114b60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00e9f0 tx=0x7f99ac00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.332+0000 7f99b37fe700 1 -- 172.21.15.44:0/1566576269 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac00c9c0 con 0x7f99bc100120 2023-09-10T01:16:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.332+0000 7f99b37fe700 1 -- 172.21.15.44:0/1566576269 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99ac010430 con 0x7f99bc100120 2023-09-10T01:16:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.332+0000 7f99b37fe700 1 -- 172.21.15.44:0/1566576269 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac018680 con 0x7f99bc100120 2023-09-10T01:16:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.332+0000 7f99c45fb700 1 -- 172.21.15.44:0/1566576269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b80095f0 con 0x7f99bc100120 2023-09-10T01:16:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.332+0000 7f99c45fb700 1 -- 172.21.15.44:0/1566576269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99bc110020 con 0x7f99bc100120 2023-09-10T01:16:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.334+0000 7f99b37fe700 1 -- 172.21.15.44:0/1566576269 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f99ac0105a0 con 0x7f99bc100120 2023-09-10T01:16:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.335+0000 7f99b37fe700 1 --2- 172.21.15.44:0/1566576269 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f99a80a0830 0x7f99a80a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.335+0000 7f99b37fe700 1 -- 172.21.15.44:0/1566576269 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f99ac014070 con 0x7f99bc100120 2023-09-10T01:16:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.335+0000 7f99c2397700 1 --2- 172.21.15.44:0/1566576269 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f99a80a0830 0x7f99a80a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.336+0000 7f99c2397700 1 --2- 172.21.15.44:0/1566576269 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f99a80a0830 0x7f99a80a2cf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b2e0 tx=0x7f99b8005960 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.336+0000 7f99d07c1b80 1 Processor -- start 2023-09-10T01:16:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.336+0000 7f99d07c1b80 1 -- start start 2023-09-10T01:16:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99d07c1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae7b1a0 0x55f19ae7b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99d07c1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae75300 0x55f19ae75780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99d07c1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae75cc0 0x55f19ae81f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99d07c1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f19adce6d0 con 0x55f19ae7b1a0 2023-09-10T01:16:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99d07c1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f19adcbff0 con 0x55f19ae75300 2023-09-10T01:16:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99d07c1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f19adc78e0 con 0x55f19ae75cc0 2023-09-10T01:16:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99c2397700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae7b1a0 0x55f19ae7b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99c2b98700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae75cc0 0x55f19ae81f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99c2397700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae7b1a0 0x55f19ae7b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57686/0 (socket says 172.21.15.44:57686) 2023-09-10T01:16:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99c2b98700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae75cc0 0x55f19ae81f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36338/0 (socket says 172.21.15.44:36338) 2023-09-10T01:16:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99c2397700 1 -- 172.21.15.44:0/3203894958 learned_addr learned my addr 172.21.15.44:0/3203894958 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99c1b96700 1 --2- 172.21.15.44:0/3203894958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae75300 0x55f19ae75780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99c2397700 1 -- 172.21.15.44:0/3203894958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae75cc0 msgr2=0x55f19ae81f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99c2397700 1 --2- 172.21.15.44:0/3203894958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae75cc0 0x55f19ae81f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99c2397700 1 -- 172.21.15.44:0/3203894958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae75300 msgr2=0x55f19ae75780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99c2397700 1 --2- 172.21.15.44:0/3203894958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae75300 0x55f19ae75780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99c2397700 1 -- 172.21.15.44:0/3203894958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f19abf3d80 con 0x55f19ae7b1a0 2023-09-10T01:16:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99c1b96700 1 --2- 172.21.15.44:0/3203894958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae75300 0x55f19ae75780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:16:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99c2397700 1 --2- 172.21.15.44:0/3203894958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae7b1a0 0x55f19ae7b580 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f99b8000c00 tx=0x7f99b8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9a097f4366d08c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.337+0000 7f99b0ff9700 1 -- 172.21.15.44:0/3203894958 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8004990 con 0x55f19ae7b1a0 2023-09-10T01:16:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.338+0000 7f99b0ff9700 1 -- 172.21.15.44:0/3203894958 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99b801e040 con 0x55f19ae7b1a0 2023-09-10T01:16:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.338+0000 7f99b0ff9700 1 -- 172.21.15.44:0/3203894958 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b800fdc0 con 0x55f19ae7b1a0 2023-09-10T01:16:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.338+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3203894958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae7b1a0 msgr2=0x55f19ae7b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.338+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/3203894958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae7b1a0 0x55f19ae7b580 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f99b8000c00 tx=0x7f99b8020000 comp rx=0 tx=0).stop 2023-09-10T01:16:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.338+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3203894958 shutdown_connections 2023-09-10T01:16:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.338+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/3203894958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae75cc0 0x55f19ae81f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.338+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/3203894958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae7b1a0 0x55f19ae7b580 secure :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f99b8000c00 tx=0x7f99b8020000 comp rx=0 tx=0).stop 2023-09-10T01:16:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.338+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/3203894958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae75300 0x55f19ae75780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.338+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3203894958 >> 172.21.15.44:0/3203894958 conn(0x55f19adea030 msgr2=0x55f19ae70560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.340+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3203894958 shutdown_connections 2023-09-10T01:16:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.340+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3203894958 wait complete. 2023-09-10T01:16:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.340+0000 7f99d07c1b80 1 -- start start 2023-09-10T01:16:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.340+0000 7f99d07c1b80 1 Processor -- start 2023-09-10T01:16:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99d07c1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae75300 0x55f19ae7f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99d07c1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae75cc0 0x55f19ae7f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99d07c1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae7fd40 0x55f19ae8b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99d07c1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f19ad65b10 con 0x55f19ae75300 2023-09-10T01:16:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99d07c1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f19ad65810 con 0x55f19ae75cc0 2023-09-10T01:16:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99d07c1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f19ac498c0 con 0x55f19ae7fd40 2023-09-10T01:16:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99d07c1b80 10 client.0 _ll_register_callbacks cb 0x55f19acafa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:16:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99c2397700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae75300 0x55f19ae7f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99c2397700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae75300 0x55f19ae7f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57692/0 (socket says 172.21.15.44:57692) 2023-09-10T01:16:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99c2397700 1 -- 172.21.15.44:0/3189014327 learned_addr learned my addr 172.21.15.44:0/3189014327 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99c2b98700 1 --2- 172.21.15.44:0/3189014327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae7fd40 0x55f19ae8b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99c1b96700 1 --2- 172.21.15.44:0/3189014327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae75cc0 0x55f19ae7f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99c2397700 1 -- 172.21.15.44:0/3189014327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae7fd40 msgr2=0x55f19ae8b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99c2397700 1 --2- 172.21.15.44:0/3189014327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae7fd40 0x55f19ae8b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99c2397700 1 -- 172.21.15.44:0/3189014327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae75cc0 msgr2=0x55f19ae7f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99c2397700 1 --2- 172.21.15.44:0/3189014327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae75cc0 0x55f19ae7f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99c2397700 1 -- 172.21.15.44:0/3189014327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99bc110020 con 0x55f19ae75300 2023-09-10T01:16:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.341+0000 7f99c2b98700 1 --2- 172.21.15.44:0/3189014327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae7fd40 0x55f19ae8b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:16:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.342+0000 7f99c2397700 1 --2- 172.21.15.44:0/3189014327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae75300 0x55f19ae7f2c0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f99b8005d10 tx=0x7f99b8004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f6fd631112b81a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.342+0000 7f99a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:16:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.342+0000 7f99a5ffb700 1 -- 172.21.15.44:0/3189014327 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b800f9a0 con 0x55f19ae75300 2023-09-10T01:16:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.342+0000 7f99a5ffb700 1 -- 172.21.15.44:0/3189014327 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(156..156 src has 1..156) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f99b800fb00 con 0x55f19ae75300 2023-09-10T01:16:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.342+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3189014327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f19ad1f490 con 0x55f19ae75300 2023-09-10T01:16:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.342+0000 7f99a5ffb700 1 -- 172.21.15.44:0/3189014327 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99b800fcc0 con 0x55f19ae75300 2023-09-10T01:16:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.342+0000 7f99d07c1b80 10 client.19533 fetch_fsmap learned FSMap version 122 2023-09-10T01:16:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.342+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3189014327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=4+}) v3 -- 0x7f99bc10cbc0 con 0x55f19ae75300 2023-09-10T01:16:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.342+0000 7f99a5ffb700 1 -- 172.21.15.44:0/3189014327 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f19ad1f490 con 0x55f19ae75300 2023-09-10T01:16:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.343+0000 7f99a5ffb700 1 -- 172.21.15.44:0/3189014327 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 122) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f99b80384c0 con 0x55f19ae75300 2023-09-10T01:16:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.343+0000 7f99d07c1b80 10 client.19533 fetch_fsmap finished waiting for FSMap version 122 2023-09-10T01:16:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.343+0000 7f99d07c1b80 10 client.19533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:16:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.343+0000 7f99d07c1b80 20 client.19533 populate_metadata read hostname 'smithi044' 2023-09-10T01:16:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.343+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/3189014327 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f19ae8f4c0 0x55f19ae918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.343+0000 7f99d07c1b80 4 client.19533 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:16:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.343+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3189014327 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f19addc840 con 0x55f19ae8f4c0 2023-09-10T01:16:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.343+0000 7f99c1b96700 1 --2- 172.21.15.44:0/3189014327 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f19ae8f4c0 0x55f19ae918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.343+0000 7f99a5ffb700 10 client.19533 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:16:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.343+0000 7f99c1b96700 1 --2- 172.21.15.44:0/3189014327 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f19ae8f4c0 0x55f19ae918b0 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d28b58782a991b3 server_cookie=db4ac620a70399f8 in_seq=0 out_seq=0 2023-09-10T01:16:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.344+0000 7f99a5ffb700 1 -- 172.21.15.44:0/3189014327 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f19addc840 con 0x55f19ae8f4c0 2023-09-10T01:16:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.344+0000 7f99a5ffb700 10 client.19533 handle_command_reply: tid=0 2023-09-10T01:16:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.344+0000 7f99d07c1b80 1 client.19533 shutdown 2023-09-10T01:16:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.344+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3189014327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae75300 msgr2=0x55f19ae7f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.344+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/3189014327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae75300 0x55f19ae7f2c0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f99b8005d10 tx=0x7f99b8004a20 comp rx=0 tx=0).stop 2023-09-10T01:16:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.344+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3189014327 shutdown_connections 2023-09-10T01:16:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.344+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/3189014327 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f19ae8f4c0 0x55f19ae918b0 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.344+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/3189014327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae7fd40 0x55f19ae8b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.344+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/3189014327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae75300 0x55f19ae7f2c0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.344+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/3189014327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae75cc0 0x55f19ae7f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.344+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3189014327 >> 172.21.15.44:0/3189014327 conn(0x55f19adea030 msgr2=0x55f19ae6f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.344+0000 7f99a5ffb700 0 client.19533 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:16:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.344+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3189014327 shutdown_connections 2023-09-10T01:16:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.344+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3189014327 wait complete. 2023-09-10T01:16:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.344+0000 7f99d07c1b80 20 client.19533 trim_cache size 0 max 16384 2023-09-10T01:16:09.378 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:09 smithi099 ceph-mon[102692]: pgmap v1136: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2023-09-10T01:16:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.365+0000 7f99d07c1b80 1 Processor -- start 2023-09-10T01:16:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.365+0000 7f99d07c1b80 1 -- start start 2023-09-10T01:16:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.365+0000 7f99d07c1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae75cc0 0x55f19ae7cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.365+0000 7f99d07c1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae7fd40 0x55f19ae862d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.365+0000 7f99d07c1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae79c20 0x55f19ae8aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.365+0000 7f99d07c1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f19ae820e0 con 0x55f19ae75cc0 2023-09-10T01:16:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.365+0000 7f99d07c1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f19ae7fba0 con 0x55f19ae79c20 2023-09-10T01:16:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.365+0000 7f99d07c1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f19ae6cfc0 con 0x55f19ae7fd40 2023-09-10T01:16:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.365+0000 7f99c2397700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae75cc0 0x55f19ae7cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.365+0000 7f99c2397700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae75cc0 0x55f19ae7cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57700/0 (socket says 172.21.15.44:57700) 2023-09-10T01:16:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.365+0000 7f99c1b96700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae7fd40 0x55f19ae862d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.365+0000 7f99c2397700 1 -- 172.21.15.44:0/3802326138 learned_addr learned my addr 172.21.15.44:0/3802326138 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.365+0000 7f99c2b98700 1 --2- 172.21.15.44:0/3802326138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae79c20 0x55f19ae8aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.366+0000 7f99c2397700 1 -- 172.21.15.44:0/3802326138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae7fd40 msgr2=0x55f19ae862d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.366+0000 7f99c2397700 1 --2- 172.21.15.44:0/3802326138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae7fd40 0x55f19ae862d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.366+0000 7f99c2397700 1 -- 172.21.15.44:0/3802326138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae79c20 msgr2=0x55f19ae8aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.366+0000 7f99c2397700 1 --2- 172.21.15.44:0/3802326138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae79c20 0x55f19ae8aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.366+0000 7f99c2397700 1 -- 172.21.15.44:0/3802326138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f19abdcc00 con 0x55f19ae75cc0 2023-09-10T01:16:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.366+0000 7f99c1b96700 1 --2- 172.21.15.44:0/3802326138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae7fd40 0x55f19ae862d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:16:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.366+0000 7f99c2397700 1 --2- 172.21.15.44:0/3802326138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae75cc0 0x55f19ae7cdf0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f99b8006010 tx=0x7f99b8005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217f457a4f5828ec server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.366+0000 7f99b17fa700 1 -- 172.21.15.44:0/3802326138 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b80047e0 con 0x55f19ae75cc0 2023-09-10T01:16:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.366+0000 7f99b17fa700 1 -- 172.21.15.44:0/3802326138 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99b8038b90 con 0x55f19ae75cc0 2023-09-10T01:16:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.366+0000 7f99b17fa700 1 -- 172.21.15.44:0/3802326138 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b802f6a0 con 0x55f19ae75cc0 2023-09-10T01:16:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.367+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3802326138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae75cc0 msgr2=0x55f19ae7cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.367+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/3802326138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae75cc0 0x55f19ae7cdf0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f99b8006010 tx=0x7f99b8005be0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.369+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3802326138 shutdown_connections 2023-09-10T01:16:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.369+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/3802326138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae79c20 0x55f19ae8aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.369+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/3802326138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae75cc0 0x55f19ae7cdf0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.369+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/3802326138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae7fd40 0x55f19ae862d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.369+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3802326138 >> 172.21.15.44:0/3802326138 conn(0x55f19adea030 msgr2=0x55f19ae79510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.369+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3802326138 shutdown_connections 2023-09-10T01:16:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.369+0000 7f99d07c1b80 1 -- 172.21.15.44:0/3802326138 wait complete. 2023-09-10T01:16:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.369+0000 7f99d07c1b80 1 -- start start 2023-09-10T01:16:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.369+0000 7f99d07c1b80 1 Processor -- start 2023-09-10T01:16:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99d07c1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae75cc0 0x55f19ae86280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99d07c1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae7fd40 0x55f19ae76b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99d07c1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae77090 0x55f19ae77510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99d07c1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f19ae77aa0 con 0x55f19ae77090 2023-09-10T01:16:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99d07c1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f19ae77c10 con 0x55f19ae7fd40 2023-09-10T01:16:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99d07c1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f19ae77dd0 con 0x55f19ae75cc0 2023-09-10T01:16:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99d07c1b80 10 client.0 _ll_register_callbacks cb 0x7f99ac005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:16:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99c2397700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae75cc0 0x55f19ae86280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99c2397700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae75cc0 0x55f19ae86280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36356/0 (socket says 172.21.15.44:36356) 2023-09-10T01:16:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99c1b96700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae7fd40 0x55f19ae76b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99c2397700 1 -- 172.21.15.44:0/1227641431 learned_addr learned my addr 172.21.15.44:0/1227641431 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99c2b98700 1 --2- 172.21.15.44:0/1227641431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae77090 0x55f19ae77510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99c2397700 1 -- 172.21.15.44:0/1227641431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae7fd40 msgr2=0x55f19ae76b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99c2397700 1 --2- 172.21.15.44:0/1227641431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae7fd40 0x55f19ae76b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99c2397700 1 -- 172.21.15.44:0/1227641431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae77090 msgr2=0x55f19ae77510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99c2397700 1 --2- 172.21.15.44:0/1227641431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae77090 0x55f19ae77510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99c2397700 1 -- 172.21.15.44:0/1227641431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b80095f0 con 0x55f19ae75cc0 2023-09-10T01:16:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99c1b96700 1 --2- 172.21.15.44:0/1227641431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae7fd40 0x55f19ae76b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:16:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99c2397700 1 --2- 172.21.15.44:0/1227641431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae75cc0 0x55f19ae86280 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b650 tx=0x7f99b8038430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28d0a882128be8be server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.370+0000 7f99a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:16:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.371+0000 7f99a5ffb700 1 -- 172.21.15.44:0/1227641431 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b800fd30 con 0x55f19ae75cc0 2023-09-10T01:16:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.371+0000 7f99d07c1b80 1 -- 172.21.15.44:0/1227641431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99b801e040 con 0x55f19ae75cc0 2023-09-10T01:16:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.371+0000 7f99a5ffb700 1 -- 172.21.15.44:0/1227641431 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(156..156 src has 1..156) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f99b8038e40 con 0x55f19ae75cc0 2023-09-10T01:16:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.371+0000 7f99a5ffb700 1 -- 172.21.15.44:0/1227641431 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99b8038660 con 0x55f19ae75cc0 2023-09-10T01:16:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.371+0000 7f99d07c1b80 10 client.29149 fetch_fsmap learned FSMap version 122 2023-09-10T01:16:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.371+0000 7f99d07c1b80 1 -- 172.21.15.44:0/1227641431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=4+}) v3 -- 0x7f99bc10cbc0 con 0x55f19ae75cc0 2023-09-10T01:16:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.371+0000 7f99a5ffb700 1 -- 172.21.15.44:0/1227641431 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99b801e040 con 0x55f19ae75cc0 2023-09-10T01:16:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.371+0000 7f99a5ffb700 1 -- 172.21.15.44:0/1227641431 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 122) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f99b804a540 con 0x55f19ae75cc0 2023-09-10T01:16:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.371+0000 7f99d07c1b80 10 client.29149 fetch_fsmap finished waiting for FSMap version 122 2023-09-10T01:16:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.371+0000 7f99d07c1b80 10 client.29149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:16:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.371+0000 7f99d07c1b80 20 client.29149 populate_metadata read hostname 'smithi044' 2023-09-10T01:16:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.371+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/1227641431 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f19aee39d0 0x55f19aee5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.371+0000 7f99d07c1b80 4 client.29149 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:16:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.371+0000 7f99d07c1b80 1 -- 172.21.15.44:0/1227641431 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f19adc08c0 con 0x55f19aee39d0 2023-09-10T01:16:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.372+0000 7f99c1b96700 1 --2- 172.21.15.44:0/1227641431 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f19aee39d0 0x55f19aee5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.372+0000 7f99a5ffb700 10 client.29149 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:16:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.372+0000 7f99c1b96700 1 --2- 172.21.15.44:0/1227641431 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f19aee39d0 0x55f19aee5dc0 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10189e0e3fff6ddb server_cookie=2a603fda007fb1c8 in_seq=0 out_seq=0 2023-09-10T01:16:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.381+0000 7f99a5ffb700 1 -- 172.21.15.44:0/1227641431 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f19adc08c0 con 0x55f19aee39d0 2023-09-10T01:16:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.381+0000 7f99a5ffb700 10 client.29149 handle_command_reply: tid=0 2023-09-10T01:16:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.381+0000 7f99d07c1b80 1 client.29149 shutdown 2023-09-10T01:16:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.381+0000 7f99d07c1b80 1 -- 172.21.15.44:0/1227641431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae75cc0 msgr2=0x55f19ae86280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.381+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/1227641431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae75cc0 0x55f19ae86280 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b650 tx=0x7f99b8038430 comp rx=0 tx=0).stop 2023-09-10T01:16:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99d07c1b80 1 -- 172.21.15.44:0/1227641431 shutdown_connections 2023-09-10T01:16:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/1227641431 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f19aee39d0 0x55f19aee5dc0 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/1227641431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f19ae77090 0x55f19ae77510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99a5ffb700 0 client.29149 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:16:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/1227641431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f19ae75cc0 0x55f19ae86280 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99d07c1b80 1 --2- 172.21.15.44:0/1227641431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19ae7fd40 0x55f19ae76b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99d07c1b80 1 -- 172.21.15.44:0/1227641431 >> 172.21.15.44:0/1227641431 conn(0x55f19adea030 msgr2=0x55f19ae742f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99d07c1b80 1 -- 172.21.15.44:0/1227641431 shutdown_connections 2023-09-10T01:16:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99d07c1b80 1 -- 172.21.15.44:0/1227641431 wait complete. 2023-09-10T01:16:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99d07c1b80 20 client.29149 trim_cache size 0 max 16384 2023-09-10T01:16:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:16:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:16:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:16:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:16:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99c45fb700 1 -- 172.21.15.44:0/1566576269 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f99a80a0830 msgr2=0x7f99a80a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99c45fb700 1 --2- 172.21.15.44:0/1566576269 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f99a80a0830 0x7f99a80a2cf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b2e0 tx=0x7f99b8005960 comp rx=0 tx=0).stop 2023-09-10T01:16:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99c45fb700 1 -- 172.21.15.44:0/1566576269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99bc100120 msgr2=0x7f99bc114b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99c45fb700 1 --2- 172.21.15.44:0/1566576269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99bc100120 0x7f99bc114b60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00e9f0 tx=0x7f99ac00ed00 comp rx=0 tx=0).stop 2023-09-10T01:16:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99c45fb700 1 -- 172.21.15.44:0/1566576269 shutdown_connections 2023-09-10T01:16:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99c45fb700 1 --2- 172.21.15.44:0/1566576269 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f99a80a0830 0x7f99a80a2cf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99c45fb700 1 --2- 172.21.15.44:0/1566576269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99bc1055b0 0x7f99bc11c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99c45fb700 1 --2- 172.21.15.44:0/1566576269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99bc0ff760 0x7f99bc114600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.382+0000 7f99c45fb700 1 --2- 172.21.15.44:0/1566576269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99bc100120 0x7f99bc114b60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.383+0000 7f99c45fb700 1 -- 172.21.15.44:0/1566576269 >> 172.21.15.44:0/1566576269 conn(0x7f99bc074f30 msgr2=0x7f99bc0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.383+0000 7f99c45fb700 1 -- 172.21.15.44:0/1566576269 shutdown_connections 2023-09-10T01:16:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:09.383+0000 7f99c45fb700 1 -- 172.21.15.44:0/1566576269 wait complete. 2023-09-10T01:16:09.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:09 smithi131 ceph-mon[103713]: pgmap v1136: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2023-09-10T01:16:09.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:09 smithi044 ceph-mon[98676]: pgmap v1136: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2023-09-10T01:16:10.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:10 smithi099 ceph-mon[102692]: osdmap e156: 12 total, 12 up, 12 in 2023-09-10T01:16:10.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:10 smithi131 ceph-mon[103713]: osdmap e156: 12 total, 12 up, 12 in 2023-09-10T01:16:10.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:10 smithi044 ceph-mon[98676]: osdmap e156: 12 total, 12 up, 12 in 2023-09-10T01:16:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:11 smithi099 ceph-mon[102692]: pgmap v1138: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-09-10T01:16:11.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:11 smithi131 ceph-mon[103713]: pgmap v1138: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-09-10T01:16:11.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:11 smithi044 ceph-mon[98676]: pgmap v1138: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-09-10T01:16:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:13 smithi099 ceph-mon[102692]: pgmap v1139: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:16:13.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:13 smithi131 ceph-mon[103713]: pgmap v1139: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:16:13.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:13 smithi044 ceph-mon[98676]: pgmap v1139: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:16:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:15 smithi099 ceph-mon[102692]: pgmap v1140: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:16:15.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:15 smithi131 ceph-mon[103713]: pgmap v1140: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:16:15.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:15 smithi044 ceph-mon[98676]: pgmap v1140: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:16:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:16 smithi131 ceph-mon[103713]: pgmap v1141: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2023-09-10T01:16:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:16 smithi131 ceph-mon[103713]: osdmap e157: 12 total, 12 up, 12 in 2023-09-10T01:16:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:16 smithi044 ceph-mon[98676]: pgmap v1141: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2023-09-10T01:16:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:16 smithi044 ceph-mon[98676]: osdmap e157: 12 total, 12 up, 12 in 2023-09-10T01:16:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:16 smithi099 ceph-mon[102692]: pgmap v1141: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2023-09-10T01:16:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:16 smithi099 ceph-mon[102692]: osdmap e157: 12 total, 12 up, 12 in 2023-09-10T01:16:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:18 smithi131 ceph-mon[103713]: pgmap v1143: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2023-09-10T01:16:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:18 smithi044 ceph-mon[98676]: pgmap v1143: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2023-09-10T01:16:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:18 smithi099 ceph-mon[102692]: pgmap v1143: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2023-09-10T01:16:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:20 smithi131 ceph-mon[103713]: pgmap v1144: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2023-09-10T01:16:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:20 smithi044 ceph-mon[98676]: pgmap v1144: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2023-09-10T01:16:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:20 smithi099 ceph-mon[102692]: pgmap v1144: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2023-09-10T01:16:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.508+0000 7fc398e74700 1 -- 172.21.15.44:0/2675072081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc394106650 msgr2=0x7fc394106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.508+0000 7fc398e74700 1 --2- 172.21.15.44:0/2675072081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc394106650 0x7fc394106a30 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc3840099c0 tx=0x7fc384009cd0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.509+0000 7fc398e74700 1 -- 172.21.15.44:0/2675072081 shutdown_connections 2023-09-10T01:16:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.509+0000 7fc398e74700 1 --2- 172.21.15.44:0/2675072081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc394100f50 0x7fc39410d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.509+0000 7fc398e74700 1 --2- 172.21.15.44:0/2675072081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc394106650 0x7fc394106a30 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.509+0000 7fc398e74700 1 --2- 172.21.15.44:0/2675072081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc394100590 0x7fc394100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.509+0000 7fc398e74700 1 -- 172.21.15.44:0/2675072081 >> 172.21.15.44:0/2675072081 conn(0x7fc3940fc060 msgr2=0x7fc3940fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.509+0000 7fc398e74700 1 -- 172.21.15.44:0/2675072081 shutdown_connections 2023-09-10T01:16:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.509+0000 7fc398e74700 1 -- 172.21.15.44:0/2675072081 wait complete. 2023-09-10T01:16:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.509+0000 7fc398e74700 1 Processor -- start 2023-09-10T01:16:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.509+0000 7fc398e74700 1 -- start start 2023-09-10T01:16:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc398e74700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc394100590 0x7fc39419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc398e74700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc394100f50 0x7fc39419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc398e74700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc394106650 0x7fc394196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc398e74700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc394196a00 con 0x7fc394100f50 2023-09-10T01:16:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc398e74700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc394196b70 con 0x7fc394100590 2023-09-10T01:16:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc398e74700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc394196d30 con 0x7fc394106650 2023-09-10T01:16:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc391d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc394100f50 0x7fc39419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc391d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc394100f50 0x7fc39419ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57722/0 (socket says 172.21.15.44:57722) 2023-09-10T01:16:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc391d9b700 1 -- 172.21.15.44:0/2459126900 learned_addr learned my addr 172.21.15.44:0/2459126900 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc39259c700 1 --2- 172.21.15.44:0/2459126900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc394100590 0x7fc39419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc392d9d700 1 --2- 172.21.15.44:0/2459126900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc394106650 0x7fc394196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc391d9b700 1 -- 172.21.15.44:0/2459126900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc394106650 msgr2=0x7fc394196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc391d9b700 1 --2- 172.21.15.44:0/2459126900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc394106650 0x7fc394196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc391d9b700 1 -- 172.21.15.44:0/2459126900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc394100590 msgr2=0x7fc39419c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc391d9b700 1 --2- 172.21.15.44:0/2459126900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc394100590 0x7fc39419c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc391d9b700 1 -- 172.21.15.44:0/2459126900 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3840095f0 con 0x7fc394100f50 2023-09-10T01:16:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc39259c700 1 --2- 172.21.15.44:0/2459126900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc394100590 0x7fc39419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:16:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.510+0000 7fc391d9b700 1 --2- 172.21.15.44:0/2459126900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc394100f50 0x7fc39419ca70 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc38800e9a0 tx=0x7fc38800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.511+0000 7fc3837fe700 1 -- 172.21.15.44:0/2459126900 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388009790 con 0x7fc394100f50 2023-09-10T01:16:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.511+0000 7fc3837fe700 1 -- 172.21.15.44:0/2459126900 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc38800ce20 con 0x7fc394100f50 2023-09-10T01:16:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.511+0000 7fc398e74700 1 -- 172.21.15.44:0/2459126900 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc394197010 con 0x7fc394100f50 2023-09-10T01:16:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.511+0000 7fc3837fe700 1 -- 172.21.15.44:0/2459126900 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388018420 con 0x7fc394100f50 2023-09-10T01:16:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.511+0000 7fc398e74700 1 -- 172.21.15.44:0/2459126900 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc394072a10 con 0x7fc394100f50 2023-09-10T01:16:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.513+0000 7fc3837fe700 1 -- 172.21.15.44:0/2459126900 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc38800c990 con 0x7fc394100f50 2023-09-10T01:16:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.514+0000 7fc3837fe700 1 --2- 172.21.15.44:0/2459126900 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc37c0a0830 0x7fc37c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.514+0000 7fc3837fe700 1 -- 172.21.15.44:0/2459126900 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fc388014070 con 0x7fc394100f50 2023-09-10T01:16:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.514+0000 7fc39259c700 1 --2- 172.21.15.44:0/2459126900 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc37c0a0830 0x7fc37c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.514+0000 7fc39259c700 1 --2- 172.21.15.44:0/2459126900 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc37c0a0830 0x7fc37c0a2cf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc384006010 tx=0x7fc38401a040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.515+0000 7fc3a503ab80 1 Processor -- start 2023-09-10T01:16:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.515+0000 7fc3a503ab80 1 -- start start 2023-09-10T01:16:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.515+0000 7fc3a503ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcba51a0 0x55e8fcba5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.515+0000 7fc3a503ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9f300 0x55e8fcb9f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.515+0000 7fc3a503ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcbabf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.515+0000 7fc3a503ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fcaf86d0 con 0x55e8fcb9f300 2023-09-10T01:16:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.515+0000 7fc3a503ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fcaf5ff0 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.515+0000 7fc3a503ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fcaf18e0 con 0x55e8fcba51a0 2023-09-10T01:16:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.515+0000 7fc391d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9f300 0x55e8fcb9f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.515+0000 7fc39259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcba51a0 0x55e8fcba5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.515+0000 7fc391d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9f300 0x55e8fcb9f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57730/0 (socket says 172.21.15.44:57730) 2023-09-10T01:16:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.515+0000 7fc391d9b700 1 -- 172.21.15.44:0/1846672785 learned_addr learned my addr 172.21.15.44:0/1846672785 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.515+0000 7fc39259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcba51a0 0x55e8fcba5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36378/0 (socket says 172.21.15.44:36378) 2023-09-10T01:16:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.516+0000 7fc392d9d700 1 --2- 172.21.15.44:0/1846672785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcbabf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.516+0000 7fc391d9b700 1 -- 172.21.15.44:0/1846672785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcba51a0 msgr2=0x55e8fcba5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.516+0000 7fc391d9b700 1 --2- 172.21.15.44:0/1846672785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcba51a0 0x55e8fcba5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.516+0000 7fc391d9b700 1 -- 172.21.15.44:0/1846672785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcb9fcc0 msgr2=0x55e8fcbabf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.516+0000 7fc391d9b700 1 --2- 172.21.15.44:0/1846672785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcbabf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.516+0000 7fc391d9b700 1 -- 172.21.15.44:0/1846672785 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8fc91dd80 con 0x55e8fcb9f300 2023-09-10T01:16:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.516+0000 7fc391d9b700 1 --2- 172.21.15.44:0/1846672785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9f300 0x55e8fcb9f780 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc3880a1020 tx=0x7fc388098e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fe79655db174dc server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.516+0000 7fc380ff9700 1 -- 172.21.15.44:0/1846672785 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388099be0 con 0x55e8fcb9f300 2023-09-10T01:16:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.516+0000 7fc380ff9700 1 -- 172.21.15.44:0/1846672785 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc3880994e0 con 0x55e8fcb9f300 2023-09-10T01:16:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.516+0000 7fc380ff9700 1 -- 172.21.15.44:0/1846672785 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880840b0 con 0x55e8fcb9f300 2023-09-10T01:16:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.517+0000 7fc3a503ab80 1 -- 172.21.15.44:0/1846672785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9f300 msgr2=0x55e8fcb9f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.517+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/1846672785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9f300 0x55e8fcb9f780 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc3880a1020 tx=0x7fc388098e40 comp rx=0 tx=0).stop 2023-09-10T01:16:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.517+0000 7fc3a503ab80 1 -- 172.21.15.44:0/1846672785 shutdown_connections 2023-09-10T01:16:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.517+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/1846672785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcbabf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.517+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/1846672785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcba51a0 0x55e8fcba5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.517+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/1846672785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9f300 0x55e8fcb9f780 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.517+0000 7fc3a503ab80 1 -- 172.21.15.44:0/1846672785 >> 172.21.15.44:0/1846672785 conn(0x55e8fcb14030 msgr2=0x55e8fcb9a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.517+0000 7fc3a503ab80 1 -- 172.21.15.44:0/1846672785 shutdown_connections 2023-09-10T01:16:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.517+0000 7fc3a503ab80 1 -- 172.21.15.44:0/1846672785 wait complete. 2023-09-10T01:16:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.517+0000 7fc3a503ab80 1 -- start start 2023-09-10T01:16:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.517+0000 7fc3a503ab80 1 Processor -- start 2023-09-10T01:16:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc3a503ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcb9f300 0x55e8fcbabeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc3a503ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcba90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc3a503ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcba51a0 0x55e8fcba9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc3a503ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fca8fb10 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc3a503ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fca8f810 con 0x55e8fcb9f300 2023-09-10T01:16:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc3a503ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fc9738c0 con 0x55e8fcba51a0 2023-09-10T01:16:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc3a503ab80 10 client.0 _ll_register_callbacks cb 0x55e8fc9d9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:16:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc391d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcba90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc391d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcba90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57736/0 (socket says 172.21.15.44:57736) 2023-09-10T01:16:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc391d9b700 1 -- 172.21.15.44:0/2978701092 learned_addr learned my addr 172.21.15.44:0/2978701092 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc39259c700 1 --2- 172.21.15.44:0/2978701092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcb9f300 0x55e8fcbabeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc392d9d700 1 --2- 172.21.15.44:0/2978701092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcba51a0 0x55e8fcba9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc391d9b700 1 -- 172.21.15.44:0/2978701092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcba51a0 msgr2=0x55e8fcba9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc391d9b700 1 --2- 172.21.15.44:0/2978701092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcba51a0 0x55e8fcba9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc391d9b700 1 -- 172.21.15.44:0/2978701092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcb9f300 msgr2=0x55e8fcbabeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc391d9b700 1 --2- 172.21.15.44:0/2978701092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcb9f300 0x55e8fcbabeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc391d9b700 1 -- 172.21.15.44:0/2978701092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc394072a10 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc391d9b700 1 --2- 172.21.15.44:0/2978701092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcba90c0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc3880959e0 tx=0x7fc388010a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c5ba280d6ba831 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.518+0000 7fc379ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:16:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.519+0000 7fc379ffb700 1 -- 172.21.15.44:0/2978701092 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38809d310 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.519+0000 7fc3a503ab80 1 -- 172.21.15.44:0/2978701092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8fca49490 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.519+0000 7fc379ffb700 1 -- 172.21.15.44:0/2978701092 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fc3880a1550 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.519+0000 7fc379ffb700 1 -- 172.21.15.44:0/2978701092 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc388099be0 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.519+0000 7fc3a503ab80 10 client.19551 fetch_fsmap learned FSMap version 122 2023-09-10T01:16:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.519+0000 7fc3a503ab80 1 -- 172.21.15.44:0/2978701092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=4+}) v3 -- 0x55e8fc906c00 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.519+0000 7fc379ffb700 1 -- 172.21.15.44:0/2978701092 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e8fca49490 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.519+0000 7fc379ffb700 1 -- 172.21.15.44:0/2978701092 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 122) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fc3880b72c0 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.519+0000 7fc3a503ab80 10 client.19551 fetch_fsmap finished waiting for FSMap version 122 2023-09-10T01:16:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.519+0000 7fc3a503ab80 10 client.19551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:16:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.519+0000 7fc3a503ab80 20 client.19551 populate_metadata read hostname 'smithi044' 2023-09-10T01:16:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.519+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/2978701092 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e8fcbb8ee0 0x55e8fcbbb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.519+0000 7fc3a503ab80 4 client.19551 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:16:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.519+0000 7fc3a503ab80 1 -- 172.21.15.44:0/2978701092 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8fcb06840 con 0x55e8fcbb8ee0 2023-09-10T01:16:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.519+0000 7fc39259c700 1 --2- 172.21.15.44:0/2978701092 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e8fcbb8ee0 0x55e8fcbbb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.520+0000 7fc379ffb700 10 client.19551 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:16:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.520+0000 7fc39259c700 1 --2- 172.21.15.44:0/2978701092 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e8fcbb8ee0 0x55e8fcbbb2d0 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=445fd3cbcd15a80e server_cookie=3c0f435a2be18891 in_seq=0 out_seq=0 2023-09-10T01:16:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.520+0000 7fc379ffb700 1 -- 172.21.15.44:0/2978701092 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e8fcb06840 con 0x55e8fcbb8ee0 2023-09-10T01:16:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.520+0000 7fc379ffb700 10 client.19551 handle_command_reply: tid=0 2023-09-10T01:16:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.521+0000 7fc3a503ab80 1 client.19551 shutdown 2023-09-10T01:16:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.521+0000 7fc3a503ab80 1 -- 172.21.15.44:0/2978701092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9fcc0 msgr2=0x55e8fcba90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.521+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/2978701092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcba90c0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc3880959e0 tx=0x7fc388010a10 comp rx=0 tx=0).stop 2023-09-10T01:16:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.521+0000 7fc3a503ab80 1 -- 172.21.15.44:0/2978701092 shutdown_connections 2023-09-10T01:16:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.521+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/2978701092 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e8fcbb8ee0 0x55e8fcbbb2d0 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.521+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/2978701092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcba51a0 0x55e8fcba9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.521+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/2978701092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcb9f300 0x55e8fcbabeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.521+0000 7fc379ffb700 0 client.19551 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:16:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.521+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/2978701092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcba90c0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.521+0000 7fc3a503ab80 1 -- 172.21.15.44:0/2978701092 >> 172.21.15.44:0/2978701092 conn(0x55e8fcb14030 msgr2=0x55e8fcb997a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.521+0000 7fc3a503ab80 1 -- 172.21.15.44:0/2978701092 shutdown_connections 2023-09-10T01:16:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.521+0000 7fc3a503ab80 1 -- 172.21.15.44:0/2978701092 wait complete. 2023-09-10T01:16:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.521+0000 7fc3a503ab80 20 client.19551 trim_cache size 0 max 16384 2023-09-10T01:16:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.540+0000 7fc3a503ab80 1 Processor -- start 2023-09-10T01:16:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.540+0000 7fc3a503ab80 1 -- start start 2023-09-10T01:16:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.540+0000 7fc3a503ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcb9f300 0x55e8fcba8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.540+0000 7fc3a503ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcbab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.540+0000 7fc3a503ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcba51a0 0x55e8fcba3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.540+0000 7fc3a503ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fcba9440 con 0x55e8fcba51a0 2023-09-10T01:16:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.540+0000 7fc3a503ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fcba8ea0 con 0x55e8fcb9f300 2023-09-10T01:16:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.540+0000 7fc3a503ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fcba3d10 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.541+0000 7fc392d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcba51a0 0x55e8fcba3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.541+0000 7fc391d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcbab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.541+0000 7fc392d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcba51a0 0x55e8fcba3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57744/0 (socket says 172.21.15.44:57744) 2023-09-10T01:16:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.541+0000 7fc392d9d700 1 -- 172.21.15.44:0/4015672351 learned_addr learned my addr 172.21.15.44:0/4015672351 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.541+0000 7fc39259c700 1 --2- 172.21.15.44:0/4015672351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcb9f300 0x55e8fcba8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.541+0000 7fc39259c700 1 -- 172.21.15.44:0/4015672351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcb9fcc0 msgr2=0x55e8fcbab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.541+0000 7fc39259c700 1 --2- 172.21.15.44:0/4015672351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcbab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.541+0000 7fc39259c700 1 -- 172.21.15.44:0/4015672351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcba51a0 msgr2=0x55e8fcba3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.541+0000 7fc39259c700 1 --2- 172.21.15.44:0/4015672351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcba51a0 0x55e8fcba3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.541+0000 7fc39259c700 1 -- 172.21.15.44:0/4015672351 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8fc9441f0 con 0x55e8fcb9f300 2023-09-10T01:16:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.541+0000 7fc39259c700 1 --2- 172.21.15.44:0/4015672351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcb9f300 0x55e8fcba8960 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc384009990 tx=0x7fc384022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63c49a7771e17ba0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.541+0000 7fc3817fa700 1 -- 172.21.15.44:0/4015672351 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38402c040 con 0x55e8fcb9f300 2023-09-10T01:16:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.541+0000 7fc3817fa700 1 -- 172.21.15.44:0/4015672351 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc38402d040 con 0x55e8fcb9f300 2023-09-10T01:16:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.541+0000 7fc3817fa700 1 -- 172.21.15.44:0/4015672351 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384003b50 con 0x55e8fcb9f300 2023-09-10T01:16:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.542+0000 7fc3a503ab80 1 -- 172.21.15.44:0/4015672351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcb9f300 msgr2=0x55e8fcba8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.542+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/4015672351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcb9f300 0x55e8fcba8960 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc384009990 tx=0x7fc384022000 comp rx=0 tx=0).stop 2023-09-10T01:16:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.542+0000 7fc3a503ab80 1 -- 172.21.15.44:0/4015672351 shutdown_connections 2023-09-10T01:16:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.542+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/4015672351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcba51a0 0x55e8fcba3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.542+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/4015672351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcb9f300 0x55e8fcba8960 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.542+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/4015672351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcbab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.542+0000 7fc3a503ab80 1 -- 172.21.15.44:0/4015672351 >> 172.21.15.44:0/4015672351 conn(0x55e8fcb14030 msgr2=0x55e8fcbbfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.542+0000 7fc3a503ab80 1 -- 172.21.15.44:0/4015672351 shutdown_connections 2023-09-10T01:16:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.542+0000 7fc3a503ab80 1 -- 172.21.15.44:0/4015672351 wait complete. 2023-09-10T01:16:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc3a503ab80 1 -- start start 2023-09-10T01:16:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc3a503ab80 1 Processor -- start 2023-09-10T01:16:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc3a503ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcb9f300 0x55e8fcbafe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc3a503ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcbb0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc3a503ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcba51a0 0x55e8fcbae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc3a503ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fcbae8c0 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc3a503ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fcbaea30 con 0x55e8fcba51a0 2023-09-10T01:16:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc3a503ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fcbaebf0 con 0x55e8fcb9f300 2023-09-10T01:16:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc3a503ab80 10 client.0 _ll_register_callbacks cb 0x7fc38400bc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:16:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc391d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcbb0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc391d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcbb0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57748/0 (socket says 172.21.15.44:57748) 2023-09-10T01:16:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc39259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcb9f300 0x55e8fcbafe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc391d9b700 1 -- 172.21.15.44:0/1851801883 learned_addr learned my addr 172.21.15.44:0/1851801883 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc391d9b700 1 -- 172.21.15.44:0/1851801883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcb9f300 msgr2=0x55e8fcbafe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc391d9b700 1 --2- 172.21.15.44:0/1851801883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcb9f300 0x55e8fcbafe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc392d9d700 1 --2- 172.21.15.44:0/1851801883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcba51a0 0x55e8fcbae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc391d9b700 1 -- 172.21.15.44:0/1851801883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcba51a0 msgr2=0x55e8fcbae2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc391d9b700 1 --2- 172.21.15.44:0/1851801883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcba51a0 0x55e8fcbae2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.543+0000 7fc391d9b700 1 -- 172.21.15.44:0/1851801883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39410dd10 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.544+0000 7fc391d9b700 1 --2- 172.21.15.44:0/1851801883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcbb0370 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fc38800c2d0 tx=0x7fc3880990d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7763380d485edf1c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.544+0000 7fc379ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:16:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.544+0000 7fc379ffb700 1 -- 172.21.15.44:0/1851801883 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388098e40 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.544+0000 7fc379ffb700 1 -- 172.21.15.44:0/1851801883 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fc388099be0 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.544+0000 7fc3a503ab80 1 -- 172.21.15.44:0/1851801883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3880994e0 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.544+0000 7fc379ffb700 1 -- 172.21.15.44:0/1851801883 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3880a1550 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.544+0000 7fc3a503ab80 10 client.19557 fetch_fsmap learned FSMap version 122 2023-09-10T01:16:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.544+0000 7fc3a503ab80 1 -- 172.21.15.44:0/1851801883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=4+}) v3 -- 0x55e8fc91c250 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.544+0000 7fc379ffb700 1 -- 172.21.15.44:0/1851801883 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc3880994e0 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.544+0000 7fc379ffb700 1 -- 172.21.15.44:0/1851801883 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 122) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fc388023d40 con 0x55e8fcb9fcc0 2023-09-10T01:16:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.544+0000 7fc3a503ab80 10 client.19557 fetch_fsmap finished waiting for FSMap version 122 2023-09-10T01:16:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.544+0000 7fc3a503ab80 10 client.19557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:16:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.544+0000 7fc3a503ab80 20 client.19557 populate_metadata read hostname 'smithi044' 2023-09-10T01:16:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.544+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/1851801883 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e8fcc0da00 0x55e8fcc0fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.544+0000 7fc3a503ab80 4 client.19557 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:16:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.544+0000 7fc3a503ab80 1 -- 172.21.15.44:0/1851801883 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e8fcaea8c0 con 0x55e8fcc0da00 2023-09-10T01:16:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.545+0000 7fc39259c700 1 --2- 172.21.15.44:0/1851801883 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e8fcc0da00 0x55e8fcc0fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.545+0000 7fc379ffb700 10 client.19557 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:16:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.545+0000 7fc39259c700 1 --2- 172.21.15.44:0/1851801883 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e8fcc0da00 0x55e8fcc0fe20 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9d8ecd399e76b0c server_cookie=4dfbb3fe57de99a8 in_seq=0 out_seq=0 2023-09-10T01:16:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.555+0000 7fc379ffb700 1 -- 172.21.15.44:0/1851801883 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e8fcaea8c0 con 0x55e8fcc0da00 2023-09-10T01:16:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.555+0000 7fc379ffb700 10 client.19557 handle_command_reply: tid=0 2023-09-10T01:16:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.555+0000 7fc3a503ab80 1 client.19557 shutdown 2023-09-10T01:16:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.555+0000 7fc3a503ab80 1 -- 172.21.15.44:0/1851801883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9fcc0 msgr2=0x55e8fcbb0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.555+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/1851801883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcbb0370 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fc38800c2d0 tx=0x7fc3880990d0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.555+0000 7fc3a503ab80 1 -- 172.21.15.44:0/1851801883 shutdown_connections 2023-09-10T01:16:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.555+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/1851801883 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e8fcc0da00 0x55e8fcc0fe20 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.555+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/1851801883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8fcba51a0 0x55e8fcbae2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.555+0000 7fc379ffb700 0 client.19557 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:16:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.555+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/1851801883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fcb9f300 0x55e8fcbafe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.555+0000 7fc3a503ab80 1 --2- 172.21.15.44:0/1851801883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8fcb9fcc0 0x55e8fcbb0370 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.555+0000 7fc3a503ab80 1 -- 172.21.15.44:0/1851801883 >> 172.21.15.44:0/1851801883 conn(0x55e8fcb14030 msgr2=0x55e8fcba3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.555+0000 7fc3a503ab80 1 -- 172.21.15.44:0/1851801883 shutdown_connections 2023-09-10T01:16:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.556+0000 7fc3a503ab80 1 -- 172.21.15.44:0/1851801883 wait complete. 2023-09-10T01:16:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.556+0000 7fc3a503ab80 20 client.19557 trim_cache size 0 max 16384 2023-09-10T01:16:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:16:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:16:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:16:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:16:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.556+0000 7fc398e74700 1 -- 172.21.15.44:0/2459126900 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc37c0a0830 msgr2=0x7fc37c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.556+0000 7fc398e74700 1 --2- 172.21.15.44:0/2459126900 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc37c0a0830 0x7fc37c0a2cf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc384006010 tx=0x7fc38401a040 comp rx=0 tx=0).stop 2023-09-10T01:16:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.556+0000 7fc398e74700 1 -- 172.21.15.44:0/2459126900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc394100f50 msgr2=0x7fc39419ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.556+0000 7fc398e74700 1 --2- 172.21.15.44:0/2459126900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc394100f50 0x7fc39419ca70 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc38800e9a0 tx=0x7fc38800ecb0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.556+0000 7fc398e74700 1 -- 172.21.15.44:0/2459126900 shutdown_connections 2023-09-10T01:16:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.556+0000 7fc398e74700 1 --2- 172.21.15.44:0/2459126900 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc37c0a0830 0x7fc37c0a2cf0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.556+0000 7fc398e74700 1 --2- 172.21.15.44:0/2459126900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc394106650 0x7fc394196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.556+0000 7fc398e74700 1 --2- 172.21.15.44:0/2459126900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc394100590 0x7fc39419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.556+0000 7fc398e74700 1 --2- 172.21.15.44:0/2459126900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc394100f50 0x7fc39419ca70 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.556+0000 7fc398e74700 1 -- 172.21.15.44:0/2459126900 >> 172.21.15.44:0/2459126900 conn(0x7fc3940fc060 msgr2=0x7fc3940fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.556+0000 7fc398e74700 1 -- 172.21.15.44:0/2459126900 shutdown_connections 2023-09-10T01:16:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:22.556+0000 7fc398e74700 1 -- 172.21.15.44:0/2459126900 wait complete. 2023-09-10T01:16:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:22 smithi131 ceph-mon[103713]: pgmap v1145: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2023-09-10T01:16:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:22 smithi044 ceph-mon[98676]: pgmap v1145: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2023-09-10T01:16:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:22 smithi099 ceph-mon[102692]: pgmap v1145: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2023-09-10T01:16:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:24 smithi131 ceph-mon[103713]: pgmap v1146: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2023-09-10T01:16:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:24 smithi044 ceph-mon[98676]: pgmap v1146: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2023-09-10T01:16:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:24 smithi099 ceph-mon[102692]: pgmap v1146: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2023-09-10T01:16:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:26 smithi131 ceph-mon[103713]: pgmap v1147: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2023-09-10T01:16:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:26 smithi044 ceph-mon[98676]: pgmap v1147: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2023-09-10T01:16:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:26 smithi099 ceph-mon[102692]: pgmap v1147: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2023-09-10T01:16:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:28 smithi131 ceph-mon[103713]: pgmap v1148: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 148 op/s 2023-09-10T01:16:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:28 smithi044 ceph-mon[98676]: pgmap v1148: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 148 op/s 2023-09-10T01:16:29.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:28 smithi099 ceph-mon[102692]: pgmap v1148: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 148 op/s 2023-09-10T01:16:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:30 smithi131 ceph-mon[103713]: pgmap v1149: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1004 KiB/s wr, 142 op/s 2023-09-10T01:16:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:30 smithi044 ceph-mon[98676]: pgmap v1149: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1004 KiB/s wr, 142 op/s 2023-09-10T01:16:31.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:30 smithi099 ceph-mon[102692]: pgmap v1149: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1004 KiB/s wr, 142 op/s 2023-09-10T01:16:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:32 smithi131 ceph-mon[103713]: pgmap v1150: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2023-09-10T01:16:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:32 smithi044 ceph-mon[98676]: pgmap v1150: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2023-09-10T01:16:33.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:32 smithi099 ceph-mon[102692]: pgmap v1150: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2023-09-10T01:16:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:34 smithi131 ceph-mon[103713]: pgmap v1151: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 978 KiB/s wr, 139 op/s 2023-09-10T01:16:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:34 smithi044 ceph-mon[98676]: pgmap v1151: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 978 KiB/s wr, 139 op/s 2023-09-10T01:16:35.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:34 smithi099 ceph-mon[102692]: pgmap v1151: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 978 KiB/s wr, 139 op/s 2023-09-10T01:16:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.676+0000 7f44cf5bf700 1 -- 172.21.15.44:0/3229545467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c8106670 msgr2=0x7f44c8106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.676+0000 7f44cf5bf700 1 --2- 172.21.15.44:0/3229545467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c8106670 0x7f44c8106a50 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f44c40099c0 tx=0x7f44c4009cd0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.676+0000 7f44cf5bf700 1 -- 172.21.15.44:0/3229545467 shutdown_connections 2023-09-10T01:16:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.676+0000 7f44cf5bf700 1 --2- 172.21.15.44:0/3229545467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44c8100f70 0x7f44c810d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.676+0000 7f44cf5bf700 1 --2- 172.21.15.44:0/3229545467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c8106670 0x7f44c8106a50 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.676+0000 7f44cf5bf700 1 --2- 172.21.15.44:0/3229545467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44c81005b0 0x7f44c8100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.676+0000 7f44cf5bf700 1 -- 172.21.15.44:0/3229545467 >> 172.21.15.44:0/3229545467 conn(0x7f44c80fc060 msgr2=0x7f44c80fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.676+0000 7f44cf5bf700 1 -- 172.21.15.44:0/3229545467 shutdown_connections 2023-09-10T01:16:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.676+0000 7f44cf5bf700 1 -- 172.21.15.44:0/3229545467 wait complete. 2023-09-10T01:16:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.677+0000 7f44cf5bf700 1 Processor -- start 2023-09-10T01:16:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.677+0000 7f44cf5bf700 1 -- start start 2023-09-10T01:16:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.677+0000 7f44cf5bf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44c81005b0 0x7f44c81964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.677+0000 7f44cf5bf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c8100f70 0x7f44c8196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.677+0000 7f44cf5bf700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44c8106670 0x7f44c8199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.677+0000 7f44cf5bf700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c819a5f0 con 0x7f44c8100f70 2023-09-10T01:16:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.677+0000 7f44cf5bf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c819a760 con 0x7f44c81005b0 2023-09-10T01:16:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.677+0000 7f44cf5bf700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c819a920 con 0x7f44c8106670 2023-09-10T01:16:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.677+0000 7f44ccb5a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c8100f70 0x7f44c8196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.677+0000 7f44cd35b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44c81005b0 0x7f44c81964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.677+0000 7f44ccb5a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c8100f70 0x7f44c8196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57760/0 (socket says 172.21.15.44:57760) 2023-09-10T01:16:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.677+0000 7f44ccb5a700 1 -- 172.21.15.44:0/3629557273 learned_addr learned my addr 172.21.15.44:0/3629557273 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.677+0000 7f44cdb5c700 1 --2- 172.21.15.44:0/3629557273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44c8106670 0x7f44c8199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.678+0000 7f44ccb5a700 1 -- 172.21.15.44:0/3629557273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44c8106670 msgr2=0x7f44c8199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.678+0000 7f44ccb5a700 1 --2- 172.21.15.44:0/3629557273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44c8106670 0x7f44c8199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.678+0000 7f44ccb5a700 1 -- 172.21.15.44:0/3629557273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44c81005b0 msgr2=0x7f44c81964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.678+0000 7f44ccb5a700 1 --2- 172.21.15.44:0/3629557273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44c81005b0 0x7f44c81964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.678+0000 7f44ccb5a700 1 -- 172.21.15.44:0/3629557273 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c40095f0 con 0x7f44c8100f70 2023-09-10T01:16:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.678+0000 7f44cd35b700 1 --2- 172.21.15.44:0/3629557273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44c81005b0 0x7f44c81964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:16:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.678+0000 7f44ccb5a700 1 --2- 172.21.15.44:0/3629557273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c8100f70 0x7f44c8196a30 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f44b800e9a0 tx=0x7f44b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.678+0000 7f44be7fc700 1 -- 172.21.15.44:0/3629557273 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8009790 con 0x7f44c8100f70 2023-09-10T01:16:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.678+0000 7f44be7fc700 1 -- 172.21.15.44:0/3629557273 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f44b8010430 con 0x7f44c8100f70 2023-09-10T01:16:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.678+0000 7f44cf5bf700 1 -- 172.21.15.44:0/3629557273 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c819ac00 con 0x7f44c8100f70 2023-09-10T01:16:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.678+0000 7f44be7fc700 1 -- 172.21.15.44:0/3629557273 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b800cea0 con 0x7f44c8100f70 2023-09-10T01:16:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.678+0000 7f44cf5bf700 1 -- 172.21.15.44:0/3629557273 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c81a3560 con 0x7f44c8100f70 2023-09-10T01:16:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.680+0000 7f44be7fc700 1 -- 172.21.15.44:0/3629557273 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f44b80105a0 con 0x7f44c8100f70 2023-09-10T01:16:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.681+0000 7f44be7fc700 1 --2- 172.21.15.44:0/3629557273 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44b40a07b0 0x7f44b40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.681+0000 7f44cd35b700 1 --2- 172.21.15.44:0/3629557273 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44b40a07b0 0x7f44b40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.681+0000 7f44be7fc700 1 -- 172.21.15.44:0/3629557273 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f44b8014070 con 0x7f44c8100f70 2023-09-10T01:16:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.681+0000 7f44cd35b700 1 --2- 172.21.15.44:0/3629557273 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44b40a07b0 0x7f44b40a2c70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f44c400b580 tx=0x7f44c401a040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44db785b80 1 Processor -- start 2023-09-10T01:16:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44db785b80 1 -- start start 2023-09-10T01:16:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44db785b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f23885b0 0x55f2f2388990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44db785b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f2382710 0x55f2f2382b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44db785b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23830d0 0x55f2f238f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44db785b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f22d76d0 con 0x55f2f2382710 2023-09-10T01:16:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44db785b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f22d4ff0 con 0x55f2f23885b0 2023-09-10T01:16:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44db785b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f22d08e0 con 0x55f2f23830d0 2023-09-10T01:16:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44cd35b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f23885b0 0x55f2f2388990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44cd35b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f23885b0 0x55f2f2388990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:59506/0 (socket says 172.21.15.44:59506) 2023-09-10T01:16:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44cd35b700 1 -- 172.21.15.44:0/2677808524 learned_addr learned my addr 172.21.15.44:0/2677808524 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44ccb5a700 1 --2- 172.21.15.44:0/2677808524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f2382710 0x55f2f2382b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44cdb5c700 1 --2- 172.21.15.44:0/2677808524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23830d0 0x55f2f238f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44cd35b700 1 -- 172.21.15.44:0/2677808524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23830d0 msgr2=0x55f2f238f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44cd35b700 1 --2- 172.21.15.44:0/2677808524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23830d0 0x55f2f238f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44cd35b700 1 -- 172.21.15.44:0/2677808524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f2382710 msgr2=0x55f2f2382b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44cd35b700 1 --2- 172.21.15.44:0/2677808524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f2382710 0x55f2f2382b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44cd35b700 1 -- 172.21.15.44:0/2677808524 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2f20fcd80 con 0x55f2f23885b0 2023-09-10T01:16:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.682+0000 7f44ccb5a700 1 --2- 172.21.15.44:0/2677808524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f2382710 0x55f2f2382b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:16:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.683+0000 7f44cd35b700 1 --2- 172.21.15.44:0/2677808524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f23885b0 0x55f2f2388990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f44c40099c0 tx=0x7f44c401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4353d4fedc3b18b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.683+0000 7f44b37fe700 1 -- 172.21.15.44:0/2677808524 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c40056a0 con 0x55f2f23885b0 2023-09-10T01:16:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.683+0000 7f44b37fe700 1 -- 172.21.15.44:0/2677808524 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f44c401e040 con 0x55f2f23885b0 2023-09-10T01:16:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.683+0000 7f44b37fe700 1 -- 172.21.15.44:0/2677808524 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c400f900 con 0x55f2f23885b0 2023-09-10T01:16:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.684+0000 7f44db785b80 1 -- 172.21.15.44:0/2677808524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f23885b0 msgr2=0x55f2f2388990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.684+0000 7f44db785b80 1 --2- 172.21.15.44:0/2677808524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f23885b0 0x55f2f2388990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f44c40099c0 tx=0x7f44c401f000 comp rx=0 tx=0).stop 2023-09-10T01:16:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.684+0000 7f44db785b80 1 -- 172.21.15.44:0/2677808524 shutdown_connections 2023-09-10T01:16:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.684+0000 7f44db785b80 1 --2- 172.21.15.44:0/2677808524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23830d0 0x55f2f238f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.684+0000 7f44db785b80 1 --2- 172.21.15.44:0/2677808524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f23885b0 0x55f2f2388990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.684+0000 7f44db785b80 1 --2- 172.21.15.44:0/2677808524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f2382710 0x55f2f2382b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.684+0000 7f44db785b80 1 -- 172.21.15.44:0/2677808524 >> 172.21.15.44:0/2677808524 conn(0x55f2f22f3030 msgr2=0x55f2f2379560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.684+0000 7f44db785b80 1 -- 172.21.15.44:0/2677808524 shutdown_connections 2023-09-10T01:16:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.684+0000 7f44db785b80 1 -- 172.21.15.44:0/2677808524 wait complete. 2023-09-10T01:16:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.684+0000 7f44db785b80 1 -- start start 2023-09-10T01:16:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.684+0000 7f44db785b80 1 Processor -- start 2023-09-10T01:16:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44db785b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 0x55f2f238eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44db785b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23830d0 0x55f2f23814f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44db785b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23885b0 0x55f2f2381a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44db785b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f226eb10 con 0x55f2f23885b0 2023-09-10T01:16:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44db785b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f226e810 con 0x55f2f2382710 2023-09-10T01:16:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44db785b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f21528c0 con 0x55f2f23830d0 2023-09-10T01:16:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44db785b80 10 client.0 _ll_register_callbacks cb 0x55f2f21b8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:16:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44cdb5c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23885b0 0x55f2f2381a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44ccb5a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23830d0 0x55f2f23814f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44cd35b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 0x55f2f238eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44cdb5c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23885b0 0x55f2f2381a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57776/0 (socket says 172.21.15.44:57776) 2023-09-10T01:16:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44cdb5c700 1 -- 172.21.15.44:0/2190421372 learned_addr learned my addr 172.21.15.44:0/2190421372 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44cd35b700 1 -- 172.21.15.44:0/2190421372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23830d0 msgr2=0x55f2f23814f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44cd35b700 1 --2- 172.21.15.44:0/2190421372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23830d0 0x55f2f23814f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44cd35b700 1 -- 172.21.15.44:0/2190421372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23885b0 msgr2=0x55f2f2381a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44cd35b700 1 --2- 172.21.15.44:0/2190421372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23885b0 0x55f2f2381a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44cd35b700 1 -- 172.21.15.44:0/2190421372 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c81a3560 con 0x55f2f2382710 2023-09-10T01:16:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44cdb5c700 1 --2- 172.21.15.44:0/2190421372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23885b0 0x55f2f2381a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:16:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44cd35b700 1 --2- 172.21.15.44:0/2190421372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 0x55f2f238eeb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f44c4005d10 tx=0x7f44c4005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e4b48d7880a59a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:16:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44b0ff9700 1 -- 172.21.15.44:0/2190421372 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c400f670 con 0x55f2f2382710 2023-09-10T01:16:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44b0ff9700 1 -- 172.21.15.44:0/2190421372 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f44c400f7d0 con 0x55f2f2382710 2023-09-10T01:16:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.685+0000 7f44db785b80 1 -- 172.21.15.44:0/2190421372 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2f2228490 con 0x55f2f2382710 2023-09-10T01:16:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.686+0000 7f44b0ff9700 1 -- 172.21.15.44:0/2190421372 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44c400f990 con 0x55f2f2382710 2023-09-10T01:16:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.686+0000 7f44db785b80 10 client.29225 fetch_fsmap learned FSMap version 122 2023-09-10T01:16:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.686+0000 7f44db785b80 1 -- 172.21.15.44:0/2190421372 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=4+}) v3 -- 0x7f44c810dd30 con 0x55f2f2382710 2023-09-10T01:16:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.686+0000 7f44b0ff9700 1 -- 172.21.15.44:0/2190421372 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f2f2228490 con 0x55f2f2382710 2023-09-10T01:16:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.686+0000 7f44b0ff9700 1 -- 172.21.15.44:0/2190421372 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 122) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f44c40388f0 con 0x55f2f2382710 2023-09-10T01:16:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.686+0000 7f44db785b80 10 client.29225 fetch_fsmap finished waiting for FSMap version 122 2023-09-10T01:16:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.686+0000 7f44db785b80 10 client.29225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:16:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.686+0000 7f44db785b80 20 client.29225 populate_metadata read hostname 'smithi044' 2023-09-10T01:16:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.686+0000 7f44db785b80 1 --2- 172.21.15.44:0/2190421372 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2f23a08b0 0x55f2f23a2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.686+0000 7f44db785b80 4 client.29225 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:16:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.686+0000 7f44db785b80 1 -- 172.21.15.44:0/2190421372 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2f22e5840 con 0x55f2f23a08b0 2023-09-10T01:16:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.687+0000 7f44ccb5a700 1 --2- 172.21.15.44:0/2190421372 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2f23a08b0 0x55f2f23a2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.687+0000 7f44b0ff9700 10 client.29225 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:16:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.687+0000 7f44ccb5a700 1 --2- 172.21.15.44:0/2190421372 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2f23a08b0 0x55f2f23a2ca0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80847d7bf284c791 server_cookie=87224585197d7d9a in_seq=0 out_seq=0 2023-09-10T01:16:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.687+0000 7f44b0ff9700 1 -- 172.21.15.44:0/2190421372 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f2f22e5840 con 0x55f2f23a08b0 2023-09-10T01:16:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.687+0000 7f44b0ff9700 10 client.29225 handle_command_reply: tid=0 2023-09-10T01:16:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.687+0000 7f44db785b80 1 client.29225 shutdown 2023-09-10T01:16:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.688+0000 7f44db785b80 1 -- 172.21.15.44:0/2190421372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 msgr2=0x55f2f238eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.688+0000 7f44db785b80 1 --2- 172.21.15.44:0/2190421372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 0x55f2f238eeb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f44c4005d10 tx=0x7f44c4005750 comp rx=0 tx=0).stop 2023-09-10T01:16:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.688+0000 7f44db785b80 1 -- 172.21.15.44:0/2190421372 shutdown_connections 2023-09-10T01:16:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.688+0000 7f44db785b80 1 --2- 172.21.15.44:0/2190421372 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2f23a08b0 0x55f2f23a2ca0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.688+0000 7f44db785b80 1 --2- 172.21.15.44:0/2190421372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23885b0 0x55f2f2381a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.688+0000 7f44b0ff9700 0 client.29225 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:16:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.688+0000 7f44db785b80 1 --2- 172.21.15.44:0/2190421372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 0x55f2f238eeb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.688+0000 7f44db785b80 1 --2- 172.21.15.44:0/2190421372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23830d0 0x55f2f23814f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.688+0000 7f44db785b80 1 -- 172.21.15.44:0/2190421372 >> 172.21.15.44:0/2190421372 conn(0x55f2f22f3030 msgr2=0x55f2f23787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.688+0000 7f44db785b80 1 -- 172.21.15.44:0/2190421372 shutdown_connections 2023-09-10T01:16:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.688+0000 7f44db785b80 1 -- 172.21.15.44:0/2190421372 wait complete. 2023-09-10T01:16:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.688+0000 7f44db785b80 20 client.29225 trim_cache size 0 max 16384 2023-09-10T01:16:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44db785b80 1 Processor -- start 2023-09-10T01:16:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44db785b80 1 -- start start 2023-09-10T01:16:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44db785b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 0x55f2f238c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44db785b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23830d0 0x55f2f2378f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44db785b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23885b0 0x55f2f2379460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44db785b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f2389620 con 0x55f2f23830d0 2023-09-10T01:16:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44db785b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f2388d20 con 0x55f2f2382710 2023-09-10T01:16:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44db785b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f237a320 con 0x55f2f23885b0 2023-09-10T01:16:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44ccb5a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23830d0 0x55f2f2378f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44ccb5a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23830d0 0x55f2f2378f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57782/0 (socket says 172.21.15.44:57782) 2023-09-10T01:16:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44ccb5a700 1 -- 172.21.15.44:0/523362120 learned_addr learned my addr 172.21.15.44:0/523362120 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44cdb5c700 1 --2- 172.21.15.44:0/523362120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23885b0 0x55f2f2379460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44cd35b700 1 --2- 172.21.15.44:0/523362120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 0x55f2f238c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44ccb5a700 1 -- 172.21.15.44:0/523362120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23885b0 msgr2=0x55f2f2379460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44ccb5a700 1 --2- 172.21.15.44:0/523362120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23885b0 0x55f2f2379460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44ccb5a700 1 -- 172.21.15.44:0/523362120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 msgr2=0x55f2f238c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44ccb5a700 1 --2- 172.21.15.44:0/523362120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 0x55f2f238c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44ccb5a700 1 -- 172.21.15.44:0/523362120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2f20e5c00 con 0x55f2f23830d0 2023-09-10T01:16:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.705+0000 7f44cd35b700 1 --2- 172.21.15.44:0/523362120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 0x55f2f238c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:16:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.706+0000 7f44ccb5a700 1 --2- 172.21.15.44:0/523362120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23830d0 0x55f2f2378f20 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f44b80c9e90 tx=0x7f44b800c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb7f65f6e42d78a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.706+0000 7f44b3fff700 1 -- 172.21.15.44:0/523362120 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b80995f0 con 0x55f2f23830d0 2023-09-10T01:16:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.706+0000 7f44b3fff700 1 -- 172.21.15.44:0/523362120 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f44b809d420 con 0x55f2f23830d0 2023-09-10T01:16:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.706+0000 7f44b3fff700 1 -- 172.21.15.44:0/523362120 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b80840d0 con 0x55f2f23830d0 2023-09-10T01:16:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 -- 172.21.15.44:0/523362120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23830d0 msgr2=0x55f2f2378f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 --2- 172.21.15.44:0/523362120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23830d0 0x55f2f2378f20 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f44b80c9e90 tx=0x7f44b800c480 comp rx=0 tx=0).stop 2023-09-10T01:16:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 -- 172.21.15.44:0/523362120 shutdown_connections 2023-09-10T01:16:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 --2- 172.21.15.44:0/523362120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23885b0 0x55f2f2379460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 --2- 172.21.15.44:0/523362120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 0x55f2f238c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 --2- 172.21.15.44:0/523362120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23830d0 0x55f2f2378f20 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 -- 172.21.15.44:0/523362120 >> 172.21.15.44:0/523362120 conn(0x55f2f22f3030 msgr2=0x55f2f23817b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 -- 172.21.15.44:0/523362120 shutdown_connections 2023-09-10T01:16:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 -- 172.21.15.44:0/523362120 wait complete. 2023-09-10T01:16:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 -- start start 2023-09-10T01:16:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 Processor -- start 2023-09-10T01:16:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 0x55f2f23e85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23830d0 0x55f2f23e8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23885b0 0x55f2f23f3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f23f3980 con 0x55f2f23830d0 2023-09-10T01:16:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f23f3ac0 con 0x55f2f2382710 2023-09-10T01:16:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f23f3c00 con 0x55f2f23885b0 2023-09-10T01:16:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44db785b80 10 client.0 _ll_register_callbacks cb 0x7f44b8095270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:16:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44ccb5a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23830d0 0x55f2f23e8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44ccb5a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23830d0 0x55f2f23e8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57788/0 (socket says 172.21.15.44:57788) 2023-09-10T01:16:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44ccb5a700 1 -- 172.21.15.44:0/3857020230 learned_addr learned my addr 172.21.15.44:0/3857020230 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.707+0000 7f44cdb5c700 1 --2- 172.21.15.44:0/3857020230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23885b0 0x55f2f23f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.708+0000 7f44cd35b700 1 --2- 172.21.15.44:0/3857020230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 0x55f2f23e85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.708+0000 7f44ccb5a700 1 -- 172.21.15.44:0/3857020230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23885b0 msgr2=0x55f2f23f3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.708+0000 7f44ccb5a700 1 --2- 172.21.15.44:0/3857020230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23885b0 0x55f2f23f3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.708+0000 7f44ccb5a700 1 -- 172.21.15.44:0/3857020230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 msgr2=0x55f2f23e85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.708+0000 7f44ccb5a700 1 --2- 172.21.15.44:0/3857020230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 0x55f2f23e85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.708+0000 7f44ccb5a700 1 -- 172.21.15.44:0/3857020230 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c819ac00 con 0x55f2f23830d0 2023-09-10T01:16:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.708+0000 7f44ccb5a700 1 --2- 172.21.15.44:0/3857020230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23830d0 0x55f2f23e8af0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f44b8099580 tx=0x7f44b8095930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe57df91fcd22d3b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.708+0000 7f44b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:16:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.708+0000 7f44b0ff9700 1 -- 172.21.15.44:0/3857020230 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b80995f0 con 0x55f2f23830d0 2023-09-10T01:16:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.708+0000 7f44db785b80 1 -- 172.21.15.44:0/3857020230 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44c401e040 con 0x55f2f23830d0 2023-09-10T01:16:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.708+0000 7f44b0ff9700 1 -- 172.21.15.44:0/3857020230 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f44b809d120 con 0x55f2f23830d0 2023-09-10T01:16:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.708+0000 7f44b0ff9700 1 -- 172.21.15.44:0/3857020230 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44b80840c0 con 0x55f2f23830d0 2023-09-10T01:16:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.708+0000 7f44db785b80 10 client.19575 fetch_fsmap learned FSMap version 122 2023-09-10T01:16:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.708+0000 7f44db785b80 1 -- 172.21.15.44:0/3857020230 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=4+}) v3 -- 0x55f2f21231f0 con 0x55f2f23830d0 2023-09-10T01:16:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.708+0000 7f44b0ff9700 1 -- 172.21.15.44:0/3857020230 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f44c401e040 con 0x55f2f23830d0 2023-09-10T01:16:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.709+0000 7f44b0ff9700 1 -- 172.21.15.44:0/3857020230 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 122) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f44b8099950 con 0x55f2f23830d0 2023-09-10T01:16:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.709+0000 7f44db785b80 10 client.19575 fetch_fsmap finished waiting for FSMap version 122 2023-09-10T01:16:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.709+0000 7f44db785b80 10 client.19575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:16:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.709+0000 7f44db785b80 20 client.19575 populate_metadata read hostname 'smithi044' 2023-09-10T01:16:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.709+0000 7f44db785b80 1 --2- 172.21.15.44:0/3857020230 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2f23f5050 0x55f2f23f7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.709+0000 7f44db785b80 4 client.19575 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:16:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.709+0000 7f44db785b80 1 -- 172.21.15.44:0/3857020230 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f2f22c98c0 con 0x55f2f23f5050 2023-09-10T01:16:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.709+0000 7f44cd35b700 1 --2- 172.21.15.44:0/3857020230 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2f23f5050 0x55f2f23f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.709+0000 7f44b0ff9700 10 client.19575 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:16:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.709+0000 7f44cd35b700 1 --2- 172.21.15.44:0/3857020230 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2f23f5050 0x55f2f23f7470 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c593817217e41c1b server_cookie=b83b14977ca04475 in_seq=0 out_seq=0 2023-09-10T01:16:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.719+0000 7f44b0ff9700 1 -- 172.21.15.44:0/3857020230 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f2f22c98c0 con 0x55f2f23f5050 2023-09-10T01:16:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.719+0000 7f44b0ff9700 10 client.19575 handle_command_reply: tid=0 2023-09-10T01:16:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.719+0000 7f44db785b80 1 client.19575 shutdown 2023-09-10T01:16:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.719+0000 7f44db785b80 1 -- 172.21.15.44:0/3857020230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23830d0 msgr2=0x55f2f23e8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.719+0000 7f44db785b80 1 --2- 172.21.15.44:0/3857020230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23830d0 0x55f2f23e8af0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f44b8099580 tx=0x7f44b8095930 comp rx=0 tx=0).stop 2023-09-10T01:16:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.719+0000 7f44db785b80 1 -- 172.21.15.44:0/3857020230 shutdown_connections 2023-09-10T01:16:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.719+0000 7f44db785b80 1 --2- 172.21.15.44:0/3857020230 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2f23f5050 0x55f2f23f7470 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.719+0000 7f44db785b80 1 --2- 172.21.15.44:0/3857020230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f23885b0 0x55f2f23f3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.719+0000 7f44db785b80 1 --2- 172.21.15.44:0/3857020230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2f2382710 0x55f2f23e85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.719+0000 7f44db785b80 1 --2- 172.21.15.44:0/3857020230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2f23830d0 0x55f2f23e8af0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.719+0000 7f44b0ff9700 0 client.19575 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:16:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.719+0000 7f44db785b80 1 -- 172.21.15.44:0/3857020230 >> 172.21.15.44:0/3857020230 conn(0x55f2f22f3030 msgr2=0x55f2f238b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.719+0000 7f44db785b80 1 -- 172.21.15.44:0/3857020230 shutdown_connections 2023-09-10T01:16:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.719+0000 7f44db785b80 1 -- 172.21.15.44:0/3857020230 wait complete. 2023-09-10T01:16:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.720+0000 7f44db785b80 20 client.19575 trim_cache size 0 max 16384 2023-09-10T01:16:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:16:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:16:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:16:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:16:35.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.720+0000 7f44cf5bf700 1 -- 172.21.15.44:0/3629557273 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44b40a07b0 msgr2=0x7f44b40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:35.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.720+0000 7f44cf5bf700 1 --2- 172.21.15.44:0/3629557273 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44b40a07b0 0x7f44b40a2c70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f44c400b580 tx=0x7f44c401a040 comp rx=0 tx=0).stop 2023-09-10T01:16:35.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.720+0000 7f44cf5bf700 1 -- 172.21.15.44:0/3629557273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c8100f70 msgr2=0x7f44c8196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:35.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.720+0000 7f44cf5bf700 1 --2- 172.21.15.44:0/3629557273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c8100f70 0x7f44c8196a30 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f44b800e9a0 tx=0x7f44b800ecb0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.720+0000 7f44cf5bf700 1 -- 172.21.15.44:0/3629557273 shutdown_connections 2023-09-10T01:16:35.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.720+0000 7f44cf5bf700 1 --2- 172.21.15.44:0/3629557273 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44b40a07b0 0x7f44b40a2c70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.720+0000 7f44cf5bf700 1 --2- 172.21.15.44:0/3629557273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44c8106670 0x7f44c8199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.720+0000 7f44cf5bf700 1 --2- 172.21.15.44:0/3629557273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44c81005b0 0x7f44c81964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.720+0000 7f44cf5bf700 1 --2- 172.21.15.44:0/3629557273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c8100f70 0x7f44c8196a30 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.720+0000 7f44cf5bf700 1 -- 172.21.15.44:0/3629557273 >> 172.21.15.44:0/3629557273 conn(0x7f44c80fc060 msgr2=0x7f44c81042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.720+0000 7f44cf5bf700 1 -- 172.21.15.44:0/3629557273 shutdown_connections 2023-09-10T01:16:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:35.720+0000 7f44cf5bf700 1 -- 172.21.15.44:0/3629557273 wait complete. 2023-09-10T01:16:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:36 smithi131 ceph-mon[103713]: pgmap v1152: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:16:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:36 smithi044 ceph-mon[98676]: pgmap v1152: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:16:37.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:36 smithi099 ceph-mon[102692]: pgmap v1152: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2023-09-10T01:16:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:39 smithi131 ceph-mon[103713]: pgmap v1153: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 918 KiB/s wr, 130 op/s 2023-09-10T01:16:39.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:39 smithi044 ceph-mon[98676]: pgmap v1153: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 918 KiB/s wr, 130 op/s 2023-09-10T01:16:39.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:39 smithi099 ceph-mon[102692]: pgmap v1153: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 918 KiB/s wr, 130 op/s 2023-09-10T01:16:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:41 smithi131 ceph-mon[103713]: pgmap v1154: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 918 KiB/s wr, 130 op/s 2023-09-10T01:16:41.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:41 smithi044 ceph-mon[98676]: pgmap v1154: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 918 KiB/s wr, 130 op/s 2023-09-10T01:16:41.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:41 smithi099 ceph-mon[102692]: pgmap v1154: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 918 KiB/s wr, 130 op/s 2023-09-10T01:16:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:43 smithi131 ceph-mon[103713]: pgmap v1155: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:16:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:43 smithi044 ceph-mon[98676]: pgmap v1155: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:16:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:43 smithi099 ceph-mon[102692]: pgmap v1155: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-10T01:16:45.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:45 smithi131 ceph-mon[103713]: pgmap v1156: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 874 KiB/s wr, 125 op/s 2023-09-10T01:16:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:45 smithi044 ceph-mon[98676]: pgmap v1156: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 874 KiB/s wr, 125 op/s 2023-09-10T01:16:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:45 smithi099 ceph-mon[102692]: pgmap v1156: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 874 KiB/s wr, 125 op/s 2023-09-10T01:16:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:47 smithi131 ceph-mon[103713]: pgmap v1157: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 199 op/s 2023-09-10T01:16:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:47 smithi044 ceph-mon[98676]: pgmap v1157: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 199 op/s 2023-09-10T01:16:47.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:47 smithi099 ceph-mon[102692]: pgmap v1157: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 199 op/s 2023-09-10T01:16:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.837+0000 7fda9e176700 1 -- 172.21.15.44:0/533102336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda980679f0 msgr2=0x7fda98067e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.837+0000 7fda9e176700 1 --2- 172.21.15.44:0/533102336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda980679f0 0x7fda98067e70 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0099c0 tx=0x7fda8c009cd0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.837+0000 7fda9e176700 1 -- 172.21.15.44:0/533102336 shutdown_connections 2023-09-10T01:16:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.837+0000 7fda9e176700 1 --2- 172.21.15.44:0/533102336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda980683b0 0x7fda9810d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.837+0000 7fda9e176700 1 --2- 172.21.15.44:0/533102336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda98106650 0x7fda98106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.837+0000 7fda9e176700 1 --2- 172.21.15.44:0/533102336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda980679f0 0x7fda98067e70 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.837+0000 7fda9e176700 1 -- 172.21.15.44:0/533102336 >> 172.21.15.44:0/533102336 conn(0x7fda98074f30 msgr2=0x7fda98075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.837+0000 7fda9e176700 1 -- 172.21.15.44:0/533102336 shutdown_connections 2023-09-10T01:16:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.837+0000 7fda9e176700 1 -- 172.21.15.44:0/533102336 wait complete. 2023-09-10T01:16:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda9e176700 1 Processor -- start 2023-09-10T01:16:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda9e176700 1 -- start start 2023-09-10T01:16:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda9e176700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda980679f0 0x7fda98196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda9e176700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda980683b0 0x7fda98196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda9e176700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda98106650 0x7fda9819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda9e176700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9819a600 con 0x7fda980683b0 2023-09-10T01:16:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda9e176700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9819a770 con 0x7fda980679f0 2023-09-10T01:16:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda9e176700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9819a930 con 0x7fda98106650 2023-09-10T01:16:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda96ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda980683b0 0x7fda98196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda97fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda98106650 0x7fda9819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda96ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda980683b0 0x7fda98196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57802/0 (socket says 172.21.15.44:57802) 2023-09-10T01:16:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda96ffd700 1 -- 172.21.15.44:0/3865796211 learned_addr learned my addr 172.21.15.44:0/3865796211 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda977fe700 1 --2- 172.21.15.44:0/3865796211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda980679f0 0x7fda98196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda96ffd700 1 -- 172.21.15.44:0/3865796211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda98106650 msgr2=0x7fda9819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda96ffd700 1 --2- 172.21.15.44:0/3865796211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda98106650 0x7fda9819a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda96ffd700 1 -- 172.21.15.44:0/3865796211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda980679f0 msgr2=0x7fda98196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda96ffd700 1 --2- 172.21.15.44:0/3865796211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda980679f0 0x7fda98196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda96ffd700 1 -- 172.21.15.44:0/3865796211 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c0095f0 con 0x7fda980683b0 2023-09-10T01:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.838+0000 7fda97fff700 1 --2- 172.21.15.44:0/3865796211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda98106650 0x7fda9819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.839+0000 7fda96ffd700 1 --2- 172.21.15.44:0/3865796211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda980683b0 0x7fda98196a40 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fda8c005910 tx=0x7fda8c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.839+0000 7fda94ff9700 1 -- 172.21.15.44:0/3865796211 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c01d070 con 0x7fda980683b0 2023-09-10T01:16:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.839+0000 7fda9e176700 1 -- 172.21.15.44:0/3865796211 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9819abb0 con 0x7fda980683b0 2023-09-10T01:16:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.839+0000 7fda94ff9700 1 -- 172.21.15.44:0/3865796211 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda8c022470 con 0x7fda980683b0 2023-09-10T01:16:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.839+0000 7fda94ff9700 1 -- 172.21.15.44:0/3865796211 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c017650 con 0x7fda980683b0 2023-09-10T01:16:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.839+0000 7fda9e176700 1 -- 172.21.15.44:0/3865796211 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9819ae80 con 0x7fda980683b0 2023-09-10T01:16:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.842+0000 7fda94ff9700 1 -- 172.21.15.44:0/3865796211 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fda8c0225e0 con 0x7fda980683b0 2023-09-10T01:16:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.842+0000 7fda94ff9700 1 --2- 172.21.15.44:0/3865796211 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda800a07b0 0x7fda800a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.842+0000 7fda977fe700 1 --2- 172.21.15.44:0/3865796211 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda800a07b0 0x7fda800a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.842+0000 7fda94ff9700 1 -- 172.21.15.44:0/3865796211 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fda8c0cd3f0 con 0x7fda980683b0 2023-09-10T01:16:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fdaaa33cb80 1 Processor -- start 2023-09-10T01:16:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fdaaa33cb80 1 -- start start 2023-09-10T01:16:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fda977fe700 1 --2- 172.21.15.44:0/3865796211 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda800a07b0 0x7fda800a2c70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fda88009c60 tx=0x7fda88009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fdaaa33cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad99501a0 0x560ad9950580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fdaaa33cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad994a300 0x560ad994a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fdaaa33cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad994acc0 0x560ad9956f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fdaaa33cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad98a36d0 con 0x560ad99501a0 2023-09-10T01:16:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fdaaa33cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad98a0ff0 con 0x560ad994a300 2023-09-10T01:16:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fdaaa33cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad989c8e0 con 0x560ad994acc0 2023-09-10T01:16:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fda977fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad99501a0 0x560ad9950580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fda977fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad99501a0 0x560ad9950580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57808/0 (socket says 172.21.15.44:57808) 2023-09-10T01:16:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fda977fe700 1 -- 172.21.15.44:0/4059526312 learned_addr learned my addr 172.21.15.44:0/4059526312 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fda97fff700 1 --2- 172.21.15.44:0/4059526312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad994acc0 0x560ad9956f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fda96ffd700 1 --2- 172.21.15.44:0/4059526312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad994a300 0x560ad994a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fda977fe700 1 -- 172.21.15.44:0/4059526312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad994acc0 msgr2=0x560ad9956f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fda977fe700 1 --2- 172.21.15.44:0/4059526312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad994acc0 0x560ad9956f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fda977fe700 1 -- 172.21.15.44:0/4059526312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad994a300 msgr2=0x560ad994a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fda977fe700 1 --2- 172.21.15.44:0/4059526312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad994a300 0x560ad994a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.843+0000 7fda977fe700 1 -- 172.21.15.44:0/4059526312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ad96c8d80 con 0x560ad99501a0 2023-09-10T01:16:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.844+0000 7fda977fe700 1 --2- 172.21.15.44:0/4059526312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad99501a0 0x560ad9950580 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fda8800e4e0 tx=0x7fda88014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff585d9e87774505 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.844+0000 7fda7dffb700 1 -- 172.21.15.44:0/4059526312 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8800ee00 con 0x560ad99501a0 2023-09-10T01:16:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.844+0000 7fda7dffb700 1 -- 172.21.15.44:0/4059526312 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda88021070 con 0x560ad99501a0 2023-09-10T01:16:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.844+0000 7fda7dffb700 1 -- 172.21.15.44:0/4059526312 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8801d740 con 0x560ad99501a0 2023-09-10T01:16:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.845+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/4059526312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad99501a0 msgr2=0x560ad9950580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.845+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/4059526312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad99501a0 0x560ad9950580 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fda8800e4e0 tx=0x7fda88014000 comp rx=0 tx=0).stop 2023-09-10T01:16:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.845+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/4059526312 shutdown_connections 2023-09-10T01:16:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.845+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/4059526312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad994acc0 0x560ad9956f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.845+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/4059526312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad99501a0 0x560ad9950580 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.845+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/4059526312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad994a300 0x560ad994a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.845+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/4059526312 >> 172.21.15.44:0/4059526312 conn(0x560ad98bf030 msgr2=0x560ad9945560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.845+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/4059526312 shutdown_connections 2023-09-10T01:16:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.845+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/4059526312 wait complete. 2023-09-10T01:16:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.845+0000 7fdaaa33cb80 1 -- start start 2023-09-10T01:16:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.845+0000 7fdaaa33cb80 1 Processor -- start 2023-09-10T01:16:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fdaaa33cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad994a300 0x560ad9956da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fdaaa33cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad994acc0 0x560ad9954130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fdaaa33cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad99501a0 0x560ad99546f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fdaaa33cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad983ab10 con 0x560ad994a300 2023-09-10T01:16:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fdaaa33cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad983a810 con 0x560ad994acc0 2023-09-10T01:16:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fdaaa33cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad971e8c0 con 0x560ad99501a0 2023-09-10T01:16:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fdaaa33cb80 10 client.0 _ll_register_callbacks cb 0x560ad9784a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:16:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fda977fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad994a300 0x560ad9956da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fda977fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad994a300 0x560ad9956da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57814/0 (socket says 172.21.15.44:57814) 2023-09-10T01:16:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fda977fe700 1 -- 172.21.15.44:0/1345004317 learned_addr learned my addr 172.21.15.44:0/1345004317 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fda97fff700 1 --2- 172.21.15.44:0/1345004317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad99501a0 0x560ad99546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fda977fe700 1 -- 172.21.15.44:0/1345004317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad99501a0 msgr2=0x560ad99546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fda977fe700 1 --2- 172.21.15.44:0/1345004317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad99501a0 0x560ad99546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fda977fe700 1 -- 172.21.15.44:0/1345004317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad994acc0 msgr2=0x560ad9954130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fda977fe700 1 --2- 172.21.15.44:0/1345004317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad994acc0 0x560ad9954130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fda977fe700 1 -- 172.21.15.44:0/1345004317 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda88017070 con 0x560ad994a300 2023-09-10T01:16:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fda977fe700 1 --2- 172.21.15.44:0/1345004317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad994a300 0x560ad9956da0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fda88005d90 tx=0x7fda8800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97678a21bfd8670 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.846+0000 7fda76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:16:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.847+0000 7fda76ffd700 1 -- 172.21.15.44:0/1345004317 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda88005390 con 0x560ad994a300 2023-09-10T01:16:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.847+0000 7fda76ffd700 1 -- 172.21.15.44:0/1345004317 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fda880054f0 con 0x560ad994a300 2023-09-10T01:16:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.847+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1345004317 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ad97f4490 con 0x560ad994a300 2023-09-10T01:16:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.847+0000 7fda76ffd700 1 -- 172.21.15.44:0/1345004317 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda880056b0 con 0x560ad994a300 2023-09-10T01:16:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.847+0000 7fdaaa33cb80 10 client.19587 fetch_fsmap learned FSMap version 122 2023-09-10T01:16:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.847+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1345004317 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=4+}) v3 -- 0x560ad96b1c00 con 0x560ad994a300 2023-09-10T01:16:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.847+0000 7fda76ffd700 1 -- 172.21.15.44:0/1345004317 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560ad97f4490 con 0x560ad994a300 2023-09-10T01:16:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.847+0000 7fda76ffd700 1 -- 172.21.15.44:0/1345004317 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 122) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fda8801c710 con 0x560ad994a300 2023-09-10T01:16:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.847+0000 7fdaaa33cb80 10 client.19587 fetch_fsmap finished waiting for FSMap version 122 2023-09-10T01:16:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.847+0000 7fdaaa33cb80 10 client.19587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:16:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.847+0000 7fdaaa33cb80 20 client.19587 populate_metadata read hostname 'smithi044' 2023-09-10T01:16:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.847+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/1345004317 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560ad99642e0 0x560ad99666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.847+0000 7fdaaa33cb80 4 client.19587 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:16:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.847+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1345004317 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ad98b1840 con 0x560ad99642e0 2023-09-10T01:16:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.848+0000 7fda96ffd700 1 --2- 172.21.15.44:0/1345004317 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560ad99642e0 0x560ad99666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.848+0000 7fda76ffd700 10 client.19587 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:16:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.848+0000 7fda96ffd700 1 --2- 172.21.15.44:0/1345004317 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560ad99642e0 0x560ad99666d0 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=513a7607e54d903b server_cookie=b24a1e7ef70a91c8 in_seq=0 out_seq=0 2023-09-10T01:16:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.849+0000 7fda76ffd700 1 -- 172.21.15.44:0/1345004317 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560ad98b1840 con 0x560ad99642e0 2023-09-10T01:16:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.849+0000 7fda76ffd700 10 client.19587 handle_command_reply: tid=0 2023-09-10T01:16:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.849+0000 7fdaaa33cb80 1 client.19587 shutdown 2023-09-10T01:16:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.849+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1345004317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad994a300 msgr2=0x560ad9956da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.849+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/1345004317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad994a300 0x560ad9956da0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fda88005d90 tx=0x7fda8800ed10 comp rx=0 tx=0).stop 2023-09-10T01:16:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.849+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1345004317 shutdown_connections 2023-09-10T01:16:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.849+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/1345004317 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560ad99642e0 0x560ad99666d0 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.849+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/1345004317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad99501a0 0x560ad99546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.849+0000 7fda76ffd700 0 client.19587 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:16:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.849+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/1345004317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad994a300 0x560ad9956da0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.849+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/1345004317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad994acc0 0x560ad9954130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.849+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1345004317 >> 172.21.15.44:0/1345004317 conn(0x560ad98bf030 msgr2=0x560ad99447a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.849+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1345004317 shutdown_connections 2023-09-10T01:16:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.849+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1345004317 wait complete. 2023-09-10T01:16:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.849+0000 7fdaaa33cb80 20 client.19587 trim_cache size 0 max 16384 2023-09-10T01:16:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.865+0000 7fdaaa33cb80 1 Processor -- start 2023-09-10T01:16:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.865+0000 7fdaaa33cb80 1 -- start start 2023-09-10T01:16:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.865+0000 7fdaaa33cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad994a300 0x560ad99568d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.865+0000 7fdaaa33cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad994acc0 0x560ad994e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.865+0000 7fdaaa33cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad99501a0 0x560ad994e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.865+0000 7fdaaa33cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad9950960 con 0x560ad994a300 2023-09-10T01:16:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.865+0000 7fdaaa33cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad994fae0 con 0x560ad99501a0 2023-09-10T01:16:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.865+0000 7fdaaa33cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad9952690 con 0x560ad994acc0 2023-09-10T01:16:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.865+0000 7fda977fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad994a300 0x560ad99568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.865+0000 7fda977fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad994a300 0x560ad99568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57822/0 (socket says 172.21.15.44:57822) 2023-09-10T01:16:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.865+0000 7fda96ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad994acc0 0x560ad994e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.865+0000 7fda977fe700 1 -- 172.21.15.44:0/1598677862 learned_addr learned my addr 172.21.15.44:0/1598677862 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.865+0000 7fda97fff700 1 --2- 172.21.15.44:0/1598677862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad99501a0 0x560ad994e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.866+0000 7fda977fe700 1 -- 172.21.15.44:0/1598677862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad994acc0 msgr2=0x560ad994e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.866+0000 7fda977fe700 1 --2- 172.21.15.44:0/1598677862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad994acc0 0x560ad994e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.866+0000 7fda977fe700 1 -- 172.21.15.44:0/1598677862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad99501a0 msgr2=0x560ad994e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.866+0000 7fda977fe700 1 --2- 172.21.15.44:0/1598677862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad99501a0 0x560ad994e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.866+0000 7fda977fe700 1 -- 172.21.15.44:0/1598677862 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ad96ef1f0 con 0x560ad994a300 2023-09-10T01:16:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.866+0000 7fda97fff700 1 --2- 172.21.15.44:0/1598677862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad99501a0 0x560ad994e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:16:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.866+0000 7fda977fe700 1 --2- 172.21.15.44:0/1598677862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad994a300 0x560ad99568d0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fda88011b30 tx=0x7fda8800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605850d60ae68aee server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.866+0000 7fda7e7fc700 1 -- 172.21.15.44:0/1598677862 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda880092d0 con 0x560ad994a300 2023-09-10T01:16:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.866+0000 7fda7e7fc700 1 -- 172.21.15.44:0/1598677862 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda8801cc90 con 0x560ad994a300 2023-09-10T01:16:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.866+0000 7fda7e7fc700 1 -- 172.21.15.44:0/1598677862 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda88003680 con 0x560ad994a300 2023-09-10T01:16:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.867+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1598677862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad994a300 msgr2=0x560ad99568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.867+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/1598677862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad994a300 0x560ad99568d0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fda88011b30 tx=0x7fda8800ec70 comp rx=0 tx=0).stop 2023-09-10T01:16:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.867+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1598677862 shutdown_connections 2023-09-10T01:16:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.867+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/1598677862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad99501a0 0x560ad994e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.867+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/1598677862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad994a300 0x560ad99568d0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.867+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/1598677862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad994acc0 0x560ad994e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.867+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1598677862 >> 172.21.15.44:0/1598677862 conn(0x560ad98bf030 msgr2=0x560ad99606a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.867+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1598677862 shutdown_connections 2023-09-10T01:16:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.867+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1598677862 wait complete. 2023-09-10T01:16:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.867+0000 7fdaaa33cb80 1 -- start start 2023-09-10T01:16:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.867+0000 7fdaaa33cb80 1 Processor -- start 2023-09-10T01:16:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.867+0000 7fdaaa33cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad994a300 0x560ad9959700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.867+0000 7fdaaa33cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad994acc0 0x560ad9959c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.867+0000 7fdaaa33cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad99501a0 0x560ad99b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fdaaa33cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad99b7620 con 0x560ad99501a0 2023-09-10T01:16:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fdaaa33cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad99b7760 con 0x560ad994acc0 2023-09-10T01:16:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fdaaa33cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad99b78f0 con 0x560ad994a300 2023-09-10T01:16:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fdaaa33cb80 10 client.0 _ll_register_callbacks cb 0x7fda8c022ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:16:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fda977fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad994a300 0x560ad9959700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fda97fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad99501a0 0x560ad99b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fda977fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad994a300 0x560ad9959700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36478/0 (socket says 172.21.15.44:36478) 2023-09-10T01:16:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fda977fe700 1 -- 172.21.15.44:0/1435586462 learned_addr learned my addr 172.21.15.44:0/1435586462 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:16:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fda96ffd700 1 --2- 172.21.15.44:0/1435586462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad994acc0 0x560ad9959c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fda97fff700 1 -- 172.21.15.44:0/1435586462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad994a300 msgr2=0x560ad9959700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fda97fff700 1 --2- 172.21.15.44:0/1435586462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad994a300 0x560ad9959700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fda97fff700 1 -- 172.21.15.44:0/1435586462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad994acc0 msgr2=0x560ad9959c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:16:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fda97fff700 1 --2- 172.21.15.44:0/1435586462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad994acc0 0x560ad9959c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fda97fff700 1 -- 172.21.15.44:0/1435586462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ad96b1c00 con 0x560ad99501a0 2023-09-10T01:16:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fda96ffd700 1 --2- 172.21.15.44:0/1435586462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad994acc0 0x560ad9959c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:16:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fda97fff700 1 --2- 172.21.15.44:0/1435586462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad99501a0 0x560ad99b70e0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fda9001bec0 tx=0x7fda9000e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875c9d2d85360e0b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:16:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fda76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:16:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.868+0000 7fda76ffd700 1 -- 172.21.15.44:0/1435586462 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90020070 con 0x560ad99501a0 2023-09-10T01:16:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.869+0000 7fda76ffd700 1 -- 172.21.15.44:0/1435586462 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fda9000ee80 con 0x560ad99501a0 2023-09-10T01:16:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.869+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1435586462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda88021070 con 0x560ad99501a0 2023-09-10T01:16:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.869+0000 7fda76ffd700 1 -- 172.21.15.44:0/1435586462 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda90004bb0 con 0x560ad99501a0 2023-09-10T01:16:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.869+0000 7fdaaa33cb80 10 client.19593 fetch_fsmap learned FSMap version 122 2023-09-10T01:16:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.869+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1435586462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=4+}) v3 -- 0x560ad96c7250 con 0x560ad99501a0 2023-09-10T01:16:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.869+0000 7fda76ffd700 1 -- 172.21.15.44:0/1435586462 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda88021070 con 0x560ad99501a0 2023-09-10T01:16:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.869+0000 7fda76ffd700 1 -- 172.21.15.44:0/1435586462 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 122) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fda90005040 con 0x560ad99501a0 2023-09-10T01:16:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.869+0000 7fdaaa33cb80 10 client.19593 fetch_fsmap finished waiting for FSMap version 122 2023-09-10T01:16:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.869+0000 7fdaaa33cb80 10 client.19593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:16:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.869+0000 7fdaaa33cb80 20 client.19593 populate_metadata read hostname 'smithi044' 2023-09-10T01:16:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.869+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/1435586462 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560ad99b90b0 0x560ad99bb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:16:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.869+0000 7fdaaa33cb80 4 client.19593 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:16:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.869+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1435586462 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560ad98958c0 con 0x560ad99b90b0 2023-09-10T01:16:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.869+0000 7fda977fe700 1 --2- 172.21.15.44:0/1435586462 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560ad99b90b0 0x560ad99bb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:16:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.870+0000 7fda76ffd700 10 client.19593 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:16:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.870+0000 7fda977fe700 1 --2- 172.21.15.44:0/1435586462 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560ad99b90b0 0x560ad99bb4d0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c08a0f20cda956e1 server_cookie=e3b04ea089369795 in_seq=0 out_seq=0 2023-09-10T01:16:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.881+0000 7fda76ffd700 1 -- 172.21.15.44:0/1435586462 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560ad98958c0 con 0x560ad99b90b0 2023-09-10T01:16:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.881+0000 7fda76ffd700 10 client.19593 handle_command_reply: tid=0 2023-09-10T01:16:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.881+0000 7fdaaa33cb80 1 client.19593 shutdown 2023-09-10T01:16:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.881+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1435586462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad99501a0 msgr2=0x560ad99b70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.881+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/1435586462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad99501a0 0x560ad99b70e0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fda9001bec0 tx=0x7fda9000e8b0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.881+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1435586462 shutdown_connections 2023-09-10T01:16:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.881+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/1435586462 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560ad99b90b0 0x560ad99bb4d0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.881+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/1435586462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ad99501a0 0x560ad99b70e0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.881+0000 7fda76ffd700 0 client.19593 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:16:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.881+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/1435586462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ad994a300 0x560ad9959700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.881+0000 7fdaaa33cb80 1 --2- 172.21.15.44:0/1435586462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ad994acc0 0x560ad9959c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.881+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1435586462 >> 172.21.15.44:0/1435586462 conn(0x560ad98bf030 msgr2=0x560ad994e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.881+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1435586462 shutdown_connections 2023-09-10T01:16:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.881+0000 7fdaaa33cb80 1 -- 172.21.15.44:0/1435586462 wait complete. 2023-09-10T01:16:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.882+0000 7fdaaa33cb80 20 client.19593 trim_cache size 0 max 16384 2023-09-10T01:16:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:16:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:16:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:16:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:16:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.882+0000 7fda9e176700 1 -- 172.21.15.44:0/3865796211 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda800a07b0 msgr2=0x7fda800a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.882+0000 7fda9e176700 1 --2- 172.21.15.44:0/3865796211 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda800a07b0 0x7fda800a2c70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fda88009c60 tx=0x7fda88009260 comp rx=0 tx=0).stop 2023-09-10T01:16:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.882+0000 7fda9e176700 1 -- 172.21.15.44:0/3865796211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda980683b0 msgr2=0x7fda98196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:16:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.882+0000 7fda9e176700 1 --2- 172.21.15.44:0/3865796211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda980683b0 0x7fda98196a40 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fda8c005910 tx=0x7fda8c009580 comp rx=0 tx=0).stop 2023-09-10T01:16:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.882+0000 7fda9e176700 1 -- 172.21.15.44:0/3865796211 shutdown_connections 2023-09-10T01:16:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.882+0000 7fda9e176700 1 --2- 172.21.15.44:0/3865796211 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda800a07b0 0x7fda800a2c70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.882+0000 7fda9e176700 1 --2- 172.21.15.44:0/3865796211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda98106650 0x7fda9819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.882+0000 7fda9e176700 1 --2- 172.21.15.44:0/3865796211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda980679f0 0x7fda98196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.882+0000 7fda9e176700 1 --2- 172.21.15.44:0/3865796211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda980683b0 0x7fda98196a40 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:16:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.882+0000 7fda9e176700 1 -- 172.21.15.44:0/3865796211 >> 172.21.15.44:0/3865796211 conn(0x7fda98074f30 msgr2=0x7fda980fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:16:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.882+0000 7fda9e176700 1 -- 172.21.15.44:0/3865796211 shutdown_connections 2023-09-10T01:16:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:16:48.882+0000 7fda9e176700 1 -- 172.21.15.44:0/3865796211 wait complete. 2023-09-10T01:16:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:49 smithi131 ceph-mon[103713]: pgmap v1158: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 938 KiB/s wr, 134 op/s 2023-09-10T01:16:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:49 smithi044 ceph-mon[98676]: pgmap v1158: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 938 KiB/s wr, 134 op/s 2023-09-10T01:16:49.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:49 smithi099 ceph-mon[102692]: pgmap v1158: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 938 KiB/s wr, 134 op/s 2023-09-10T01:16:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:51 smithi131 ceph-mon[103713]: pgmap v1159: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 938 KiB/s wr, 134 op/s 2023-09-10T01:16:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:51 smithi044 ceph-mon[98676]: pgmap v1159: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 938 KiB/s wr, 134 op/s 2023-09-10T01:16:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:51 smithi099 ceph-mon[102692]: pgmap v1159: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 938 KiB/s wr, 134 op/s 2023-09-10T01:16:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:53 smithi131 ceph-mon[103713]: pgmap v1160: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:16:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:53 smithi044 ceph-mon[98676]: pgmap v1160: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:16:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:53 smithi099 ceph-mon[102692]: pgmap v1160: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-09-10T01:16:54.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:54 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:16:54.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:54 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:16:54.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:54 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:16:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:55 smithi131 ceph-mon[103713]: pgmap v1161: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 919 KiB/s wr, 131 op/s 2023-09-10T01:16:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:55 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:16:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:55 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:16:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:55 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:16:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:55 smithi044 ceph-mon[98676]: pgmap v1161: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 919 KiB/s wr, 131 op/s 2023-09-10T01:16:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:55 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:16:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:55 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:16:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:55 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:16:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:55 smithi099 ceph-mon[102692]: pgmap v1161: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 919 KiB/s wr, 131 op/s 2023-09-10T01:16:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:55 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:16:55.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:55 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:16:55.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:55 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:16:56.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:56 smithi131 ceph-mon[103713]: pgmap v1162: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1005 KiB/s wr, 143 op/s 2023-09-10T01:16:56.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:56 smithi131 ceph-mon[103713]: pgmap v1163: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 528 KiB/s wr, 76 op/s 2023-09-10T01:16:56.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:56 smithi044 ceph-mon[98676]: pgmap v1162: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1005 KiB/s wr, 143 op/s 2023-09-10T01:16:56.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:56 smithi044 ceph-mon[98676]: pgmap v1163: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 528 KiB/s wr, 76 op/s 2023-09-10T01:16:56.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:56 smithi099 ceph-mon[102692]: pgmap v1162: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1005 KiB/s wr, 143 op/s 2023-09-10T01:16:56.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:56 smithi099 ceph-mon[102692]: pgmap v1163: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 528 KiB/s wr, 76 op/s 2023-09-10T01:16:58.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:16:58 smithi131 ceph-mon[103713]: pgmap v1164: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:16:58.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:16:58 smithi044 ceph-mon[98676]: pgmap v1164: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:16:58.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:16:58 smithi099 ceph-mon[102692]: pgmap v1164: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:17:00.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:00 smithi131 ceph-mon[103713]: pgmap v1165: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:17:00.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:00 smithi044 ceph-mon[98676]: pgmap v1165: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:17:00.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:00 smithi099 ceph-mon[102692]: pgmap v1165: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2023-09-10T01:17:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.008+0000 7f9effff7700 1 -- 172.21.15.44:0/3655958410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef80ff760 msgr2=0x7f9ef80ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.008+0000 7f9effff7700 1 --2- 172.21.15.44:0/3655958410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef80ff760 0x7f9ef80ffbe0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f9ef40099c0 tx=0x7f9ef4009cd0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.008+0000 7f9effff7700 1 -- 172.21.15.44:0/3655958410 shutdown_connections 2023-09-10T01:17:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.008+0000 7f9effff7700 1 --2- 172.21.15.44:0/3655958410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef8100120 0x7f9ef810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.008+0000 7f9effff7700 1 --2- 172.21.15.44:0/3655958410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ef81055b0 0x7f9ef8105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.008+0000 7f9effff7700 1 --2- 172.21.15.44:0/3655958410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef80ff760 0x7f9ef80ffbe0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.008+0000 7f9effff7700 1 -- 172.21.15.44:0/3655958410 >> 172.21.15.44:0/3655958410 conn(0x7f9ef8074f30 msgr2=0x7f9ef8075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.008+0000 7f9effff7700 1 -- 172.21.15.44:0/3655958410 shutdown_connections 2023-09-10T01:17:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.008+0000 7f9effff7700 1 -- 172.21.15.44:0/3655958410 wait complete. 2023-09-10T01:17:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.008+0000 7f9effff7700 1 Processor -- start 2023-09-10T01:17:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.008+0000 7f9effff7700 1 -- start start 2023-09-10T01:17:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9effff7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ef80ff760 0x7f9ef8195440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9effff7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef8100120 0x7f9ef8195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9effff7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef81055b0 0x7f9ef8199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9effff7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef819a630 con 0x7f9ef81055b0 2023-09-10T01:17:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9effff7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef819a7a0 con 0x7f9ef8100120 2023-09-10T01:17:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9effff7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef819a960 con 0x7f9ef80ff760 2023-09-10T01:17:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9efe594700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef81055b0 0x7f9ef8199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9efdd93700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ef80ff760 0x7f9ef8195440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9efe594700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef81055b0 0x7f9ef8199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57846/0 (socket says 172.21.15.44:57846) 2023-09-10T01:17:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9efd592700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef8100120 0x7f9ef8195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9efe594700 1 -- 172.21.15.44:0/65399755 learned_addr learned my addr 172.21.15.44:0/65399755 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9efe594700 1 -- 172.21.15.44:0/65399755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ef80ff760 msgr2=0x7f9ef8195440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9efe594700 1 --2- 172.21.15.44:0/65399755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ef80ff760 0x7f9ef8195440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9efe594700 1 -- 172.21.15.44:0/65399755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef8100120 msgr2=0x7f9ef8195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9efe594700 1 --2- 172.21.15.44:0/65399755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef8100120 0x7f9ef8195980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9efe594700 1 -- 172.21.15.44:0/65399755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef40095f0 con 0x7f9ef81055b0 2023-09-10T01:17:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9efe594700 1 --2- 172.21.15.44:0/65399755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef81055b0 0x7f9ef8199ff0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000ef30 tx=0x7f9ef000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.009+0000 7f9eeeffd700 1 -- 172.21.15.44:0/65399755 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0018070 con 0x7f9ef81055b0 2023-09-10T01:17:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.010+0000 7f9effff7700 1 -- 172.21.15.44:0/65399755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef819abe0 con 0x7f9ef81055b0 2023-09-10T01:17:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.010+0000 7f9eeeffd700 1 -- 172.21.15.44:0/65399755 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ef000f040 con 0x7f9ef81055b0 2023-09-10T01:17:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.010+0000 7f9eeeffd700 1 -- 172.21.15.44:0/65399755 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef000cbc0 con 0x7f9ef81055b0 2023-09-10T01:17:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.010+0000 7f9effff7700 1 -- 172.21.15.44:0/65399755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef81a2480 con 0x7f9ef81055b0 2023-09-10T01:17:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.012+0000 7f9eeeffd700 1 -- 172.21.15.44:0/65399755 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f9ef000cd80 con 0x7f9ef81055b0 2023-09-10T01:17:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.012+0000 7f9eeeffd700 1 --2- 172.21.15.44:0/65399755 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9ee40a07b0 0x7f9ee40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.012+0000 7f9eeeffd700 1 -- 172.21.15.44:0/65399755 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f9ef00cbe90 con 0x7f9ef81055b0 2023-09-10T01:17:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.012+0000 7f9efdd93700 1 --2- 172.21.15.44:0/65399755 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9ee40a07b0 0x7f9ee40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.013+0000 7f9efdd93700 1 --2- 172.21.15.44:0/65399755 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9ee40a07b0 0x7f9ee40a2c70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8009c60 tx=0x7f9ee8009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.013+0000 7f9f0c1bdb80 1 Processor -- start 2023-09-10T01:17:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.013+0000 7f9f0c1bdb80 1 -- start start 2023-09-10T01:17:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.013+0000 7f9f0c1bdb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ceec1a0 0x56161ceec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.013+0000 7f9f0c1bdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6300 0x56161cee6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.013+0000 7f9f0c1bdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6cc0 0x56161cef2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.013+0000 7f9f0c1bdb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161ce3f6d0 con 0x56161ceec1a0 2023-09-10T01:17:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.013+0000 7f9f0c1bdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161ce3cff0 con 0x56161cee6cc0 2023-09-10T01:17:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.013+0000 7f9f0c1bdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161ce388e0 con 0x56161cee6300 2023-09-10T01:17:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.013+0000 7f9efdd93700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ceec1a0 0x56161ceec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.013+0000 7f9efdd93700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ceec1a0 0x56161ceec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57850/0 (socket says 172.21.15.44:57850) 2023-09-10T01:17:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.014+0000 7f9efdd93700 1 -- 172.21.15.44:0/1789557582 learned_addr learned my addr 172.21.15.44:0/1789557582 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.014+0000 7f9efd592700 1 --2- 172.21.15.44:0/1789557582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6300 0x56161cee6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.014+0000 7f9efe594700 1 --2- 172.21.15.44:0/1789557582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6cc0 0x56161cef2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.014+0000 7f9efdd93700 1 -- 172.21.15.44:0/1789557582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6300 msgr2=0x56161cee6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.014+0000 7f9efdd93700 1 --2- 172.21.15.44:0/1789557582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6300 0x56161cee6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.014+0000 7f9efdd93700 1 -- 172.21.15.44:0/1789557582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6cc0 msgr2=0x56161cef2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.014+0000 7f9efdd93700 1 --2- 172.21.15.44:0/1789557582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6cc0 0x56161cef2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.014+0000 7f9efdd93700 1 -- 172.21.15.44:0/1789557582 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56161cc64d80 con 0x56161ceec1a0 2023-09-10T01:17:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.014+0000 7f9efe594700 1 --2- 172.21.15.44:0/1789557582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6cc0 0x56161cef2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:17:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.014+0000 7f9efdd93700 1 --2- 172.21.15.44:0/1789557582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ceec1a0 0x56161ceec580 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800e4e0 tx=0x7f9ee8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c91dcfac66931c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.014+0000 7f9ee3fff700 1 -- 172.21.15.44:0/1789557582 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee800ee00 con 0x56161ceec1a0 2023-09-10T01:17:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.014+0000 7f9ee3fff700 1 -- 172.21.15.44:0/1789557582 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ee8021070 con 0x56161ceec1a0 2023-09-10T01:17:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.014+0000 7f9ee3fff700 1 -- 172.21.15.44:0/1789557582 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee801d740 con 0x56161ceec1a0 2023-09-10T01:17:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.015+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/1789557582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ceec1a0 msgr2=0x56161ceec580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.015+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/1789557582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ceec1a0 0x56161ceec580 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800e4e0 tx=0x7f9ee8014000 comp rx=0 tx=0).stop 2023-09-10T01:17:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.015+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/1789557582 shutdown_connections 2023-09-10T01:17:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.015+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/1789557582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6cc0 0x56161cef2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.015+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/1789557582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ceec1a0 0x56161ceec580 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.015+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/1789557582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6300 0x56161cee6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.015+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/1789557582 >> 172.21.15.44:0/1789557582 conn(0x56161ce5b030 msgr2=0x56161cee1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.015+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/1789557582 shutdown_connections 2023-09-10T01:17:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.015+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/1789557582 wait complete. 2023-09-10T01:17:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.015+0000 7f9f0c1bdb80 1 -- start start 2023-09-10T01:17:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.015+0000 7f9f0c1bdb80 1 Processor -- start 2023-09-10T01:17:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9f0c1bdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6300 0x56161cef2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9f0c1bdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6cc0 0x56161cef0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9f0c1bdb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ceec1a0 0x56161cef0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9f0c1bdb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161cdd6b10 con 0x56161ceec1a0 2023-09-10T01:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9f0c1bdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161cdd6810 con 0x56161cee6cc0 2023-09-10T01:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9f0c1bdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161ccba8c0 con 0x56161cee6300 2023-09-10T01:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9f0c1bdb80 10 client.0 _ll_register_callbacks cb 0x56161cd20a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9efe594700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ceec1a0 0x56161cef0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9efdd93700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6300 0x56161cef2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9efe594700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ceec1a0 0x56161cef0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57860/0 (socket says 172.21.15.44:57860) 2023-09-10T01:17:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9efe594700 1 -- 172.21.15.44:0/3216457936 learned_addr learned my addr 172.21.15.44:0/3216457936 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9efd592700 1 --2- 172.21.15.44:0/3216457936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6cc0 0x56161cef0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9efe594700 1 -- 172.21.15.44:0/3216457936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6300 msgr2=0x56161cef2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9efe594700 1 --2- 172.21.15.44:0/3216457936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6300 0x56161cef2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9efe594700 1 -- 172.21.15.44:0/3216457936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6cc0 msgr2=0x56161cef0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9efe594700 1 --2- 172.21.15.44:0/3216457936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6cc0 0x56161cef0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9efe594700 1 -- 172.21.15.44:0/3216457936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef81a2480 con 0x56161ceec1a0 2023-09-10T01:17:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9efd592700 1 --2- 172.21.15.44:0/3216457936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6cc0 0x56161cef0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:17:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9efe594700 1 --2- 172.21.15.44:0/3216457936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ceec1a0 0x56161cef0560 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9ef001b000 tx=0x7f9ef0098c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4769d107ca8e9284 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9ee17fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:17:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9ee17fa700 1 -- 172.21.15.44:0/3216457936 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00986c0 con 0x56161ceec1a0 2023-09-10T01:17:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9ee17fa700 1 -- 172.21.15.44:0/3216457936 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f9ef00b8040 con 0x56161ceec1a0 2023-09-10T01:17:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.016+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/3216457936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56161cd90490 con 0x56161ceec1a0 2023-09-10T01:17:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.017+0000 7f9ee17fa700 1 -- 172.21.15.44:0/3216457936 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ef0098fd0 con 0x56161ceec1a0 2023-09-10T01:17:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.017+0000 7f9f0c1bdb80 10 client.19605 fetch_fsmap learned FSMap version 122 2023-09-10T01:17:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.017+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/3216457936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=4+}) v3 -- 0x7f9ee8017070 con 0x56161ceec1a0 2023-09-10T01:17:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.017+0000 7f9ee17fa700 1 -- 172.21.15.44:0/3216457936 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56161cd90490 con 0x56161ceec1a0 2023-09-10T01:17:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.017+0000 7f9ee17fa700 1 -- 172.21.15.44:0/3216457936 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 122) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f9ef00a0b20 con 0x56161ceec1a0 2023-09-10T01:17:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.017+0000 7f9f0c1bdb80 10 client.19605 fetch_fsmap finished waiting for FSMap version 122 2023-09-10T01:17:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.017+0000 7f9f0c1bdb80 10 client.19605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:17:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.017+0000 7f9f0c1bdb80 20 client.19605 populate_metadata read hostname 'smithi044' 2023-09-10T01:17:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.017+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/3216457936 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56161cf001e0 0x56161cf02600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.017+0000 7f9f0c1bdb80 4 client.19605 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:17:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.017+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/3216457936 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56161ce4d840 con 0x56161cf001e0 2023-09-10T01:17:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.017+0000 7f9efdd93700 1 --2- 172.21.15.44:0/3216457936 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56161cf001e0 0x56161cf02600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.018+0000 7f9ee17fa700 10 client.19605 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.018+0000 7f9efdd93700 1 --2- 172.21.15.44:0/3216457936 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56161cf001e0 0x56161cf02600 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d10152d617e302e server_cookie=19ff273a94d4a2ba in_seq=0 out_seq=0 2023-09-10T01:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.018+0000 7f9ee17fa700 1 -- 172.21.15.44:0/3216457936 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56161ce4d840 con 0x56161cf001e0 2023-09-10T01:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.018+0000 7f9ee17fa700 10 client.19605 handle_command_reply: tid=0 2023-09-10T01:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.019+0000 7f9f0c1bdb80 1 client.19605 shutdown 2023-09-10T01:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.019+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/3216457936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ceec1a0 msgr2=0x56161cef0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.019+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/3216457936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ceec1a0 0x56161cef0560 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9ef001b000 tx=0x7f9ef0098c30 comp rx=0 tx=0).stop 2023-09-10T01:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.019+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/3216457936 shutdown_connections 2023-09-10T01:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.019+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/3216457936 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56161cf001e0 0x56161cf02600 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.019+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/3216457936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ceec1a0 0x56161cef0560 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.019+0000 7f9ee17fa700 0 client.19605 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.019+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/3216457936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6300 0x56161cef2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.019+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/3216457936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6cc0 0x56161cef0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.019+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/3216457936 >> 172.21.15.44:0/3216457936 conn(0x56161ce5b030 msgr2=0x56161cee07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.019+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/3216457936 shutdown_connections 2023-09-10T01:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.019+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/3216457936 wait complete. 2023-09-10T01:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.019+0000 7f9f0c1bdb80 20 client.19605 trim_cache size 0 max 16384 2023-09-10T01:17:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9f0c1bdb80 1 Processor -- start 2023-09-10T01:17:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9f0c1bdb80 1 -- start start 2023-09-10T01:17:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9f0c1bdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161ce5b030 0x56161ceeb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9f0c1bdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6300 0x56161cef7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9f0c1bdb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161cee6cc0 0x56161cee0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9f0c1bdb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161ced8e00 con 0x56161cee6cc0 2023-09-10T01:17:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9f0c1bdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161cef3300 con 0x56161ce5b030 2023-09-10T01:17:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9f0c1bdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161ce4d840 con 0x56161cee6300 2023-09-10T01:17:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9efdd93700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161ce5b030 0x56161ceeb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9efe594700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161cee6cc0 0x56161cee0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9efdd93700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161ce5b030 0x56161ceeb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:59604/0 (socket says 172.21.15.44:59604) 2023-09-10T01:17:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9efdd93700 1 -- 172.21.15.44:0/4227034684 learned_addr learned my addr 172.21.15.44:0/4227034684 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9efd592700 1 --2- 172.21.15.44:0/4227034684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6300 0x56161cef7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9efe594700 1 -- 172.21.15.44:0/4227034684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6300 msgr2=0x56161cef7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9efe594700 1 --2- 172.21.15.44:0/4227034684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6300 0x56161cef7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9efe594700 1 -- 172.21.15.44:0/4227034684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161ce5b030 msgr2=0x56161ceeb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9efe594700 1 --2- 172.21.15.44:0/4227034684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161ce5b030 0x56161ceeb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9efe594700 1 -- 172.21.15.44:0/4227034684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56161cc4dc00 con 0x56161cee6cc0 2023-09-10T01:17:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9efdd93700 1 --2- 172.21.15.44:0/4227034684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161ce5b030 0x56161ceeb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:17:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.034+0000 7f9efe594700 1 --2- 172.21.15.44:0/4227034684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161cee6cc0 0x56161cee0f20 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00090a0 tx=0x7f9ef0095440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66895f286f68eca1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.035+0000 7f9eecff9700 1 -- 172.21.15.44:0/4227034684 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00b1200 con 0x56161cee6cc0 2023-09-10T01:17:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.035+0000 7f9eecff9700 1 -- 172.21.15.44:0/4227034684 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ef0098fd0 con 0x56161cee6cc0 2023-09-10T01:17:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.035+0000 7f9eecff9700 1 -- 172.21.15.44:0/4227034684 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00a0fb0 con 0x56161cee6cc0 2023-09-10T01:17:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.035+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/4227034684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161cee6cc0 msgr2=0x56161cee0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.035+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/4227034684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161cee6cc0 0x56161cee0f20 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00090a0 tx=0x7f9ef0095440 comp rx=0 tx=0).stop 2023-09-10T01:17:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.035+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/4227034684 shutdown_connections 2023-09-10T01:17:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.035+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/4227034684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161cee6cc0 0x56161cee0f20 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.035+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/4227034684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161ce5b030 0x56161ceeb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.035+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/4227034684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6300 0x56161cef7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.035+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/4227034684 >> 172.21.15.44:0/4227034684 conn(0x56161ceea910 msgr2=0x56161cee5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.035+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/4227034684 shutdown_connections 2023-09-10T01:17:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.035+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/4227034684 wait complete. 2023-09-10T01:17:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9f0c1bdb80 1 -- start start 2023-09-10T01:17:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9f0c1bdb80 1 Processor -- start 2023-09-10T01:17:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9f0c1bdb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ce5b030 0x56161cf4f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9f0c1bdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6300 0x56161cf4fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9f0c1bdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6cc0 0x56161cf49fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9f0c1bdb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161cf4a6c0 con 0x56161ce5b030 2023-09-10T01:17:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9f0c1bdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161cf4a830 con 0x56161cee6300 2023-09-10T01:17:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9f0c1bdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161cf4a9f0 con 0x56161cee6cc0 2023-09-10T01:17:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9f0c1bdb80 10 client.0 _ll_register_callbacks cb 0x7f9ef0098b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:17:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9efdd93700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ce5b030 0x56161cf4f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9efdd93700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ce5b030 0x56161cf4f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57870/0 (socket says 172.21.15.44:57870) 2023-09-10T01:17:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9efdd93700 1 -- 172.21.15.44:0/46690422 learned_addr learned my addr 172.21.15.44:0/46690422 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9efd592700 1 --2- 172.21.15.44:0/46690422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6300 0x56161cf4fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9efe594700 1 --2- 172.21.15.44:0/46690422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6cc0 0x56161cf49fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9efdd93700 1 -- 172.21.15.44:0/46690422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6cc0 msgr2=0x56161cf49fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9efdd93700 1 --2- 172.21.15.44:0/46690422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6cc0 0x56161cf49fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9efdd93700 1 -- 172.21.15.44:0/46690422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6300 msgr2=0x56161cf4fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9efdd93700 1 --2- 172.21.15.44:0/46690422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6300 0x56161cf4fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.036+0000 7f9efdd93700 1 -- 172.21.15.44:0/46690422 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee802a040 con 0x56161ce5b030 2023-09-10T01:17:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.037+0000 7f9efdd93700 1 --2- 172.21.15.44:0/46690422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ce5b030 0x56161cf4f9b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8026000 tx=0x7f9ee800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409dc480692995e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.037+0000 7f9ee17fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:17:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.037+0000 7f9ee17fa700 1 -- 172.21.15.44:0/46690422 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8033a90 con 0x56161ce5b030 2023-09-10T01:17:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.037+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/46690422 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ee8021070 con 0x56161ce5b030 2023-09-10T01:17:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.037+0000 7f9ee17fa700 1 -- 172.21.15.44:0/46690422 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f9ee8033bf0 con 0x56161ce5b030 2023-09-10T01:17:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.040+0000 7f9ee17fa700 1 -- 172.21.15.44:0/46690422 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ee8033db0 con 0x56161ce5b030 2023-09-10T01:17:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.040+0000 7f9f0c1bdb80 10 client.19611 fetch_fsmap learned FSMap version 122 2023-09-10T01:17:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.040+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/46690422 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=4+}) v3 -- 0x7f9ee8017070 con 0x56161ce5b030 2023-09-10T01:17:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.040+0000 7f9ee17fa700 1 -- 172.21.15.44:0/46690422 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ee8021070 con 0x56161ce5b030 2023-09-10T01:17:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.040+0000 7f9ee17fa700 1 -- 172.21.15.44:0/46690422 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 122) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f9ee803a060 con 0x56161ce5b030 2023-09-10T01:17:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.040+0000 7f9f0c1bdb80 10 client.19611 fetch_fsmap finished waiting for FSMap version 122 2023-09-10T01:17:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.040+0000 7f9f0c1bdb80 10 client.19611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:17:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.040+0000 7f9f0c1bdb80 20 client.19611 populate_metadata read hostname 'smithi044' 2023-09-10T01:17:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.040+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/46690422 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56161ceec1a0 0x56161cf59790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.040+0000 7f9f0c1bdb80 4 client.19611 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:17:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.040+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/46690422 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56161ce318c0 con 0x56161ceec1a0 2023-09-10T01:17:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.040+0000 7f9efd592700 1 --2- 172.21.15.44:0/46690422 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56161ceec1a0 0x56161cf59790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.041+0000 7f9ee17fa700 10 client.19611 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:02.041+0000 7f9efd592700 1 --2- 172.21.15.44:0/46690422 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56161ceec1a0 0x56161cf59790 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=400d661edf22689d server_cookie=36759052e6d6ec27 in_seq=0 out_seq=0 2023-09-10T01:17:02.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:02 smithi131 ceph-mon[103713]: pgmap v1166: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:17:02.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:02 smithi044 ceph-mon[98676]: pgmap v1166: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:17:02.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:02 smithi099 ceph-mon[102692]: pgmap v1166: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:17:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.434+0000 7f9ee17fa700 1 -- 172.21.15.44:0/46690422 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56161ce318c0 con 0x56161ceec1a0 2023-09-10T01:17:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.434+0000 7f9ee17fa700 10 client.19611 handle_command_reply: tid=0 2023-09-10T01:17:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.434+0000 7f9f0c1bdb80 1 client.19611 shutdown 2023-09-10T01:17:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.434+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/46690422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ce5b030 msgr2=0x56161cf4f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.434+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/46690422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ce5b030 0x56161cf4f9b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8026000 tx=0x7f9ee800ead0 comp rx=0 tx=0).stop 2023-09-10T01:17:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.434+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/46690422 shutdown_connections 2023-09-10T01:17:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.434+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/46690422 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56161ceec1a0 0x56161cf59790 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.434+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/46690422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56161cee6cc0 0x56161cf49fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.434+0000 7f9ee17fa700 0 client.19611 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.434+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/46690422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56161ce5b030 0x56161cf4f9b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.434+0000 7f9f0c1bdb80 1 --2- 172.21.15.44:0/46690422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56161cee6300 0x56161cf4fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.434+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/46690422 >> 172.21.15.44:0/46690422 conn(0x56161ceea910 msgr2=0x56161cee5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.434+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/46690422 shutdown_connections 2023-09-10T01:17:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.434+0000 7f9f0c1bdb80 1 -- 172.21.15.44:0/46690422 wait complete. 2023-09-10T01:17:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.434+0000 7f9f0c1bdb80 20 client.19611 trim_cache size 0 max 16384 2023-09-10T01:17:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:17:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:17:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:17:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:17:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.435+0000 7f9effff7700 1 -- 172.21.15.44:0/65399755 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9ee40a07b0 msgr2=0x7f9ee40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.435+0000 7f9effff7700 1 --2- 172.21.15.44:0/65399755 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9ee40a07b0 0x7f9ee40a2c70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8009c60 tx=0x7f9ee8009260 comp rx=0 tx=0).stop 2023-09-10T01:17:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.435+0000 7f9effff7700 1 -- 172.21.15.44:0/65399755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef81055b0 msgr2=0x7f9ef8199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.435+0000 7f9effff7700 1 --2- 172.21.15.44:0/65399755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef81055b0 0x7f9ef8199ff0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000ef30 tx=0x7f9ef000c3a0 comp rx=0 tx=0).stop 2023-09-10T01:17:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.435+0000 7f9effff7700 1 -- 172.21.15.44:0/65399755 shutdown_connections 2023-09-10T01:17:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.435+0000 7f9effff7700 1 --2- 172.21.15.44:0/65399755 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9ee40a07b0 0x7f9ee40a2c70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.435+0000 7f9effff7700 1 --2- 172.21.15.44:0/65399755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef81055b0 0x7f9ef8199ff0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.435+0000 7f9effff7700 1 --2- 172.21.15.44:0/65399755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ef80ff760 0x7f9ef8195440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.435+0000 7f9effff7700 1 --2- 172.21.15.44:0/65399755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef8100120 0x7f9ef8195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.435+0000 7f9effff7700 1 -- 172.21.15.44:0/65399755 >> 172.21.15.44:0/65399755 conn(0x7f9ef8074f30 msgr2=0x7f9ef80fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.435+0000 7f9effff7700 1 -- 172.21.15.44:0/65399755 shutdown_connections 2023-09-10T01:17:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:03.435+0000 7f9effff7700 1 -- 172.21.15.44:0/65399755 wait complete. 2023-09-10T01:17:04.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:04 smithi131 ceph-mon[103713]: pgmap v1167: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:17:04.305 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: *** Caught signal (Segmentation fault) ** 2023-09-10T01:17:04.306 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: in thread 7f934da6e700 thread_name:md_log_replay 2023-09-10T01:17:04.306 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:17:04.306 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f935e61dcf0] 2023-09-10T01:17:04.306 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x55c3cab4b30f] 2023-09-10T01:17:04.307 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55c3cab4d543] 2023-09-10T01:17:04.307 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 4: (MDLog::_replay_thread()+0x647) [0x55c3cabb6e97] 2023-09-10T01:17:04.307 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55c3cabc0381] 2023-09-10T01:17:04.307 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f935e6131ca] 2023-09-10T01:17:04.307 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 7: clone() 2023-09-10T01:17:04.308 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 2023-09-10T01:17:04.189+0000 7f934da6e700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:17:04.308 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: in thread 7f934da6e700 thread_name:md_log_replay 2023-09-10T01:17:04.308 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 2023-09-10T01:17:04.308 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:17:04.308 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f935e61dcf0] 2023-09-10T01:17:04.309 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x55c3cab4b30f] 2023-09-10T01:17:04.309 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55c3cab4d543] 2023-09-10T01:17:04.309 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 4: (MDLog::_replay_thread()+0x647) [0x55c3cabb6e97] 2023-09-10T01:17:04.309 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55c3cabc0381] 2023-09-10T01:17:04.309 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f935e6131ca] 2023-09-10T01:17:04.309 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 7: clone() 2023-09-10T01:17:04.310 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:17:04.310 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 2023-09-10T01:17:04.310 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 0> 2023-09-10T01:17:04.189+0000 7f934da6e700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:17:04.310 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: in thread 7f934da6e700 thread_name:md_log_replay 2023-09-10T01:17:04.311 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 2023-09-10T01:17:04.311 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:17:04.311 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f935e61dcf0] 2023-09-10T01:17:04.311 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x55c3cab4b30f] 2023-09-10T01:17:04.311 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-mon[98676]: pgmap v1167: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:17:04.312 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55c3cab4d543] 2023-09-10T01:17:04.312 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 4: (MDLog::_replay_thread()+0x647) [0x55c3cabb6e97] 2023-09-10T01:17:04.312 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55c3cabc0381] 2023-09-10T01:17:04.313 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f935e6131ca] 2023-09-10T01:17:04.313 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 7: clone() 2023-09-10T01:17:04.313 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:17:04.313 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 2023-09-10T01:17:04.313 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 0> 2023-09-10T01:17:04.189+0000 7f934da6e700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:17:04.313 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: in thread 7f934da6e700 thread_name:md_log_replay 2023-09-10T01:17:04.314 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 2023-09-10T01:17:04.314 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:17:04.314 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f935e61dcf0] 2023-09-10T01:17:04.314 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 2: /usr/bin/ceph-mds(+0x4fd30f) [0x55c3cab4b30f] 2023-09-10T01:17:04.314 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55c3cab4d543] 2023-09-10T01:17:04.315 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 4: (MDLog::_replay_thread()+0x647) [0x55c3cabb6e97] 2023-09-10T01:17:04.315 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55c3cabc0381] 2023-09-10T01:17:04.315 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f935e6131ca] 2023-09-10T01:17:04.315 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 7: clone() 2023-09-10T01:17:04.315 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:17:04.316 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[196228]: 2023-09-10T01:17:04.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:04 smithi099 ceph-mon[102692]: pgmap v1167: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2023-09-10T01:17:05.053 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:04 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Main process exited, code=exited, status=139/n/a 2023-09-10T01:17:05.053 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:05 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Failed with result 'exit-code'. 2023-09-10T01:17:05.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:05 smithi044 ceph-mon[98676]: osdmap e158: 12 total, 12 up, 12 in 2023-09-10T01:17:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:05 smithi099 ceph-mon[102692]: osdmap e158: 12 total, 12 up, 12 in 2023-09-10T01:17:05.555 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:05 smithi131 ceph-mon[103713]: osdmap e158: 12 total, 12 up, 12 in 2023-09-10T01:17:06.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:06 smithi044 ceph-mon[98676]: pgmap v1169: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2023-09-10T01:17:06.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:06 smithi099 ceph-mon[102692]: pgmap v1169: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2023-09-10T01:17:06.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:06 smithi131 ceph-mon[103713]: pgmap v1169: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2023-09-10T01:17:08.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:08 smithi099 ceph-mon[102692]: pgmap v1170: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 171 op/s 2023-09-10T01:17:08.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:08 smithi131 ceph-mon[103713]: pgmap v1170: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 171 op/s 2023-09-10T01:17:08.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:08 smithi044 ceph-mon[98676]: pgmap v1170: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 171 op/s 2023-09-10T01:17:10.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:10 smithi099 ceph-mon[102692]: pgmap v1171: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 171 op/s 2023-09-10T01:17:10.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:10 smithi131 ceph-mon[103713]: pgmap v1171: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 171 op/s 2023-09-10T01:17:10.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:10 smithi044 ceph-mon[98676]: pgmap v1171: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 171 op/s 2023-09-10T01:17:12.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:11 smithi131 ceph-mon[103713]: pgmap v1172: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2023-09-10T01:17:12.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:11 smithi131 ceph-mon[103713]: osdmap e159: 12 total, 12 up, 12 in 2023-09-10T01:17:12.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:11 smithi044 ceph-mon[98676]: pgmap v1172: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2023-09-10T01:17:12.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:11 smithi044 ceph-mon[98676]: osdmap e159: 12 total, 12 up, 12 in 2023-09-10T01:17:12.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:11 smithi099 ceph-mon[102692]: pgmap v1172: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2023-09-10T01:17:12.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:11 smithi099 ceph-mon[102692]: osdmap e159: 12 total, 12 up, 12 in 2023-09-10T01:17:14.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:13 smithi131 ceph-mon[103713]: pgmap v1174: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:17:14.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:13 smithi044 ceph-mon[98676]: pgmap v1174: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:17:14.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:13 smithi099 ceph-mon[102692]: pgmap v1174: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-09-10T01:17:15.390 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:15 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Service RestartSec=10s expired, scheduling restart. 2023-09-10T01:17:15.390 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:15 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Scheduled restart job, restart counter is at 4. 2023-09-10T01:17:15.391 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:15 smithi044 systemd[1]: Stopped Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:17:15.391 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:15 smithi044 systemd[1]: Starting Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:17:15.806 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:15 smithi044 podman[197093]: 2023-09-10T01:17:16.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:15 smithi131 ceph-mon[103713]: pgmap v1175: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2023-09-10T01:17:16.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:15 smithi131 ceph-mon[103713]: daemon mds.d restarted 2023-09-10T01:17:16.305 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:15 smithi044 bash[197093]: 352911fbb6e464631652023546aeaf45ecc3b4918e18166226e384ab567cec5b 2023-09-10T01:17:16.305 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:15 smithi044 ceph-mds[197134]: -- 172.21.15.44:0/2900704114 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ef0f7191e0 con 0x55ef0f6ee800 2023-09-10T01:17:16.305 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:15 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: starting mds.d at 2023-09-10T01:17:16.305 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:17:15 smithi044 systemd[1]: Started Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:17:16.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:15 smithi044 ceph-mon[98676]: pgmap v1175: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2023-09-10T01:17:16.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:15 smithi044 ceph-mon[98676]: daemon mds.d restarted 2023-09-10T01:17:16.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:15 smithi099 ceph-mon[102692]: pgmap v1175: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2023-09-10T01:17:16.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:15 smithi099 ceph-mon[102692]: daemon mds.d restarted 2023-09-10T01:17:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.571+0000 7feb89489700 1 -- 172.21.15.44:0/237630458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb840ff760 msgr2=0x7feb840ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.571+0000 7feb89489700 1 --2- 172.21.15.44:0/237630458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb840ff760 0x7feb840ffbe0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7feb780099c0 tx=0x7feb78009cd0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.571+0000 7feb89489700 1 -- 172.21.15.44:0/237630458 shutdown_connections 2023-09-10T01:17:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.571+0000 7feb89489700 1 --2- 172.21.15.44:0/237630458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84100120 0x7feb8410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.571+0000 7feb89489700 1 --2- 172.21.15.44:0/237630458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841055b0 0x7feb84105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.571+0000 7feb89489700 1 --2- 172.21.15.44:0/237630458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb840ff760 0x7feb840ffbe0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.571+0000 7feb89489700 1 -- 172.21.15.44:0/237630458 >> 172.21.15.44:0/237630458 conn(0x7feb84074f30 msgr2=0x7feb84075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.571+0000 7feb89489700 1 -- 172.21.15.44:0/237630458 shutdown_connections 2023-09-10T01:17:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.571+0000 7feb89489700 1 -- 172.21.15.44:0/237630458 wait complete. 2023-09-10T01:17:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.572+0000 7feb89489700 1 Processor -- start 2023-09-10T01:17:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.572+0000 7feb89489700 1 -- start start 2023-09-10T01:17:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb89489700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb840ff760 0x7feb841975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb89489700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84100120 0x7feb84197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb89489700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841055b0 0x7feb841a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb89489700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb841a2a30 con 0x7feb841055b0 2023-09-10T01:17:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb89489700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb841a2ba0 con 0x7feb84100120 2023-09-10T01:17:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb89489700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb841a2d60 con 0x7feb840ff760 2023-09-10T01:17:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb837fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841055b0 0x7feb841a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb827fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84100120 0x7feb84197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb837fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841055b0 0x7feb841a2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57914/0 (socket says 172.21.15.44:57914) 2023-09-10T01:17:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb837fe700 1 -- 172.21.15.44:0/530023404 learned_addr learned my addr 172.21.15.44:0/530023404 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb82ffd700 1 --2- 172.21.15.44:0/530023404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb840ff760 0x7feb841975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb827fc700 1 -- 172.21.15.44:0/530023404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb840ff760 msgr2=0x7feb841975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb827fc700 1 --2- 172.21.15.44:0/530023404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb840ff760 0x7feb841975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb827fc700 1 -- 172.21.15.44:0/530023404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841055b0 msgr2=0x7feb841a2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb827fc700 1 --2- 172.21.15.44:0/530023404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841055b0 0x7feb841a2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb827fc700 1 -- 172.21.15.44:0/530023404 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb74009520 con 0x7feb84100120 2023-09-10T01:17:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb82ffd700 1 --2- 172.21.15.44:0/530023404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb840ff760 0x7feb841975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:17:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.573+0000 7feb837fe700 1 --2- 172.21.15.44:0/530023404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841055b0 0x7feb841a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:17:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.574+0000 7feb827fc700 1 --2- 172.21.15.44:0/530023404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84100120 0x7feb84197b00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7feb78000c00 tx=0x7feb7800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.574+0000 7feb6bfff700 1 -- 172.21.15.44:0/530023404 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7801d070 con 0x7feb84100120 2023-09-10T01:17:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.574+0000 7feb89489700 1 -- 172.21.15.44:0/530023404 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb780095f0 con 0x7feb84100120 2023-09-10T01:17:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.574+0000 7feb6bfff700 1 -- 172.21.15.44:0/530023404 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb78005050 con 0x7feb84100120 2023-09-10T01:17:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.574+0000 7feb6bfff700 1 -- 172.21.15.44:0/530023404 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb78004970 con 0x7feb84100120 2023-09-10T01:17:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.574+0000 7feb89489700 1 -- 172.21.15.44:0/530023404 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb841a3280 con 0x7feb84100120 2023-09-10T01:17:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.577+0000 7feb6bfff700 1 -- 172.21.15.44:0/530023404 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7feb780054c0 con 0x7feb84100120 2023-09-10T01:17:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.578+0000 7feb6bfff700 1 --2- 172.21.15.44:0/530023404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb6c0a07b0 0x7feb6c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.578+0000 7feb6bfff700 1 -- 172.21.15.44:0/530023404 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 9452+0+0 (secure 0 0 0) 0x7feb780ccf20 con 0x7feb84100120 2023-09-10T01:17:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.578+0000 7feb82ffd700 1 --2- 172.21.15.44:0/530023404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb6c0a07b0 0x7feb6c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.579+0000 7feb9564fb80 1 Processor -- start 2023-09-10T01:17:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.579+0000 7feb9564fb80 1 -- start start 2023-09-10T01:17:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.579+0000 7feb82ffd700 1 --2- 172.21.15.44:0/530023404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb6c0a07b0 0x7feb6c0a2c70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7feb74011630 tx=0x7feb74009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.579+0000 7feb9564fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba45990 0x55c6bba45d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.579+0000 7feb9564fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38300 0x55c6bba38780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.579+0000 7feb9564fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba38cc0 0x55c6bba4d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.579+0000 7feb9564fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bb9916d0 con 0x55c6bba38cc0 2023-09-10T01:17:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.579+0000 7feb9564fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bb98eff0 con 0x55c6bba45990 2023-09-10T01:17:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.579+0000 7feb9564fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bb98a8e0 con 0x55c6bba38300 2023-09-10T01:17:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.579+0000 7feb82ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba45990 0x55c6bba45d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.579+0000 7feb837fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba38cc0 0x55c6bba4d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.579+0000 7feb837fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba38cc0 0x55c6bba4d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57920/0 (socket says 172.21.15.44:57920) 2023-09-10T01:17:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.579+0000 7feb82ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba45990 0x55c6bba45d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:59658/0 (socket says 172.21.15.44:59658) 2023-09-10T01:17:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.579+0000 7feb837fe700 1 -- 172.21.15.44:0/3385067379 learned_addr learned my addr 172.21.15.44:0/3385067379 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.580+0000 7feb82ffd700 1 -- 172.21.15.44:0/3385067379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38300 msgr2=0x55c6bba38780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.580+0000 7feb82ffd700 1 --2- 172.21.15.44:0/3385067379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38300 0x55c6bba38780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.580+0000 7feb82ffd700 1 -- 172.21.15.44:0/3385067379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba38cc0 msgr2=0x55c6bba4d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.580+0000 7feb82ffd700 1 --2- 172.21.15.44:0/3385067379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba38cc0 0x55c6bba4d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.580+0000 7feb82ffd700 1 -- 172.21.15.44:0/3385067379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6bb7b6d80 con 0x55c6bba45990 2023-09-10T01:17:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.580+0000 7feb82ffd700 1 --2- 172.21.15.44:0/3385067379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba45990 0x55c6bba45d70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7feb7401d5e0 tx=0x7feb74014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbabb4fabd41e150 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.580+0000 7feb697fa700 1 -- 172.21.15.44:0/3385067379 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7401dea0 con 0x55c6bba45990 2023-09-10T01:17:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.580+0000 7feb697fa700 1 -- 172.21.15.44:0/3385067379 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb74027b00 con 0x55c6bba45990 2023-09-10T01:17:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.580+0000 7feb697fa700 1 -- 172.21.15.44:0/3385067379 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7401eb30 con 0x55c6bba45990 2023-09-10T01:17:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.581+0000 7feb9564fb80 1 -- 172.21.15.44:0/3385067379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba45990 msgr2=0x55c6bba45d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.581+0000 7feb9564fb80 1 --2- 172.21.15.44:0/3385067379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba45990 0x55c6bba45d70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7feb7401d5e0 tx=0x7feb74014000 comp rx=0 tx=0).stop 2023-09-10T01:17:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.581+0000 7feb9564fb80 1 -- 172.21.15.44:0/3385067379 shutdown_connections 2023-09-10T01:17:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.581+0000 7feb9564fb80 1 --2- 172.21.15.44:0/3385067379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba38cc0 0x55c6bba4d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.581+0000 7feb9564fb80 1 --2- 172.21.15.44:0/3385067379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba45990 0x55c6bba45d70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.581+0000 7feb9564fb80 1 --2- 172.21.15.44:0/3385067379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38300 0x55c6bba38780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.581+0000 7feb9564fb80 1 -- 172.21.15.44:0/3385067379 >> 172.21.15.44:0/3385067379 conn(0x55c6bb9ad030 msgr2=0x55c6bba33240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.581+0000 7feb9564fb80 1 -- 172.21.15.44:0/3385067379 shutdown_connections 2023-09-10T01:17:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.581+0000 7feb9564fb80 1 -- 172.21.15.44:0/3385067379 wait complete. 2023-09-10T01:17:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.581+0000 7feb9564fb80 1 -- start start 2023-09-10T01:17:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.582+0000 7feb9564fb80 1 Processor -- start 2023-09-10T01:17:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.582+0000 7feb9564fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba38300 0x55c6bba4d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.582+0000 7feb9564fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38cc0 0x55c6bba4a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.582+0000 7feb9564fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba45990 0x55c6bba4af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.582+0000 7feb9564fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bb928b10 con 0x55c6bba38300 2023-09-10T01:17:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.582+0000 7feb9564fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bb928810 con 0x55c6bba45990 2023-09-10T01:17:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.582+0000 7feb9564fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bb80c8c0 con 0x55c6bba38cc0 2023-09-10T01:17:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.582+0000 7feb9564fb80 10 client.0 _ll_register_callbacks cb 0x55c6bb872a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:17:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.582+0000 7feb827fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38cc0 0x55c6bba4a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.582+0000 7feb82ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba38300 0x55c6bba4d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.582+0000 7feb827fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38cc0 0x55c6bba4a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36576/0 (socket says 172.21.15.44:36576) 2023-09-10T01:17:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.582+0000 7feb82ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba38300 0x55c6bba4d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57924/0 (socket says 172.21.15.44:57924) 2023-09-10T01:17:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.582+0000 7feb827fc700 1 -- 172.21.15.44:0/107333401 learned_addr learned my addr 172.21.15.44:0/107333401 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.582+0000 7feb837fe700 1 --2- 172.21.15.44:0/107333401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba45990 0x55c6bba4af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.583+0000 7feb82ffd700 1 -- 172.21.15.44:0/107333401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38cc0 msgr2=0x55c6bba4a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.583+0000 7feb82ffd700 1 --2- 172.21.15.44:0/107333401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38cc0 0x55c6bba4a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.583+0000 7feb82ffd700 1 -- 172.21.15.44:0/107333401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba45990 msgr2=0x55c6bba4af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.583+0000 7feb82ffd700 1 --2- 172.21.15.44:0/107333401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba45990 0x55c6bba4af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.583+0000 7feb82ffd700 1 -- 172.21.15.44:0/107333401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb74017070 con 0x55c6bba38300 2023-09-10T01:17:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.583+0000 7feb827fc700 1 --2- 172.21.15.44:0/107333401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38cc0 0x55c6bba4a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:17:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.583+0000 7feb82ffd700 1 --2- 172.21.15.44:0/107333401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba38300 0x55c6bba4d690 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7feb74009840 tx=0x7feb740279c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dc725a023fe0ac server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.583+0000 7feb66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:17:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.583+0000 7feb66ff5700 1 -- 172.21.15.44:0/107333401 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74005230 con 0x55c6bba38300 2023-09-10T01:17:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.583+0000 7feb66ff5700 1 -- 172.21.15.44:0/107333401 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(160..160 src has 1..160) v4 ==== 9452+0+0 (secure 0 0 0) 0x7feb74005390 con 0x55c6bba38300 2023-09-10T01:17:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.583+0000 7feb9564fb80 1 -- 172.21.15.44:0/107333401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6bb8e2490 con 0x55c6bba38300 2023-09-10T01:17:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.583+0000 7feb66ff5700 1 -- 172.21.15.44:0/107333401 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb74005550 con 0x55c6bba38300 2023-09-10T01:17:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.583+0000 7feb9564fb80 10 client.19629 fetch_fsmap learned FSMap version 123 2023-09-10T01:17:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.583+0000 7feb9564fb80 1 -- 172.21.15.44:0/107333401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=4+}) v3 -- 0x55c6bb79fc00 con 0x55c6bba38300 2023-09-10T01:17:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.584+0000 7feb66ff5700 1 -- 172.21.15.44:0/107333401 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c6bb8e2490 con 0x55c6bba38300 2023-09-10T01:17:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.584+0000 7feb66ff5700 1 -- 172.21.15.44:0/107333401 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 123) v1 ==== 7315+0+0 (secure 0 0 0) 0x7feb74004a50 con 0x55c6bba38300 2023-09-10T01:17:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.584+0000 7feb9564fb80 10 client.19629 fetch_fsmap finished waiting for FSMap version 123 2023-09-10T01:17:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.584+0000 7feb9564fb80 10 client.19629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:17:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.584+0000 7feb9564fb80 20 client.19629 populate_metadata read hostname 'smithi044' 2023-09-10T01:17:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.584+0000 7feb9564fb80 1 --2- 172.21.15.44:0/107333401 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c6bba5aa60 0x55c6bba5ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.584+0000 7feb9564fb80 4 client.19629 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:17:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.584+0000 7feb9564fb80 1 -- 172.21.15.44:0/107333401 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6bb99f840 con 0x55c6bba5aa60 2023-09-10T01:17:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.584+0000 7feb827fc700 1 --2- 172.21.15.44:0/107333401 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c6bba5aa60 0x55c6bba5ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.584+0000 7feb66ff5700 10 client.19629 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.585+0000 7feb827fc700 1 --2- 172.21.15.44:0/107333401 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c6bba5aa60 0x55c6bba5ce80 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa3f6e302bbb25b2 server_cookie=7bd2190e4d8f70f2 in_seq=0 out_seq=0 2023-09-10T01:17:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.585+0000 7feb66ff5700 1 -- 172.21.15.44:0/107333401 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c6bb99f840 con 0x55c6bba5aa60 2023-09-10T01:17:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.585+0000 7feb66ff5700 10 client.19629 handle_command_reply: tid=0 2023-09-10T01:17:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.585+0000 7feb9564fb80 1 client.19629 shutdown 2023-09-10T01:17:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.585+0000 7feb9564fb80 1 -- 172.21.15.44:0/107333401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba38300 msgr2=0x55c6bba4d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.585+0000 7feb9564fb80 1 --2- 172.21.15.44:0/107333401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba38300 0x55c6bba4d690 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7feb74009840 tx=0x7feb740279c0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.585+0000 7feb9564fb80 1 -- 172.21.15.44:0/107333401 shutdown_connections 2023-09-10T01:17:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.585+0000 7feb9564fb80 1 --2- 172.21.15.44:0/107333401 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c6bba5aa60 0x55c6bba5ce80 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.585+0000 7feb9564fb80 1 --2- 172.21.15.44:0/107333401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba45990 0x55c6bba4af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.585+0000 7feb9564fb80 1 --2- 172.21.15.44:0/107333401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba38300 0x55c6bba4d690 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.585+0000 7feb9564fb80 1 --2- 172.21.15.44:0/107333401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38cc0 0x55c6bba4a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.585+0000 7feb9564fb80 1 -- 172.21.15.44:0/107333401 >> 172.21.15.44:0/107333401 conn(0x55c6bb9ad030 msgr2=0x55c6bba327a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.585+0000 7feb66ff5700 0 client.19629 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.586+0000 7feb9564fb80 1 -- 172.21.15.44:0/107333401 shutdown_connections 2023-09-10T01:17:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.586+0000 7feb9564fb80 1 -- 172.21.15.44:0/107333401 wait complete. 2023-09-10T01:17:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.586+0000 7feb9564fb80 20 client.19629 trim_cache size 0 max 16384 2023-09-10T01:17:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.601+0000 7feb9564fb80 1 Processor -- start 2023-09-10T01:17:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.601+0000 7feb9564fb80 1 -- start start 2023-09-10T01:17:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.601+0000 7feb9564fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38300 0x55c6bba5ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.601+0000 7feb9564fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba38cc0 0x55c6bba4d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.601+0000 7feb9564fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba45990 0x55c6bba44d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.601+0000 7feb9564fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bba2ffc0 con 0x55c6bba45990 2023-09-10T01:17:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.601+0000 7feb9564fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bba4b590 con 0x55c6bba38cc0 2023-09-10T01:17:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.601+0000 7feb9564fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bba34df0 con 0x55c6bba38300 2023-09-10T01:17:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.601+0000 7feb82ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38300 0x55c6bba5ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.601+0000 7feb837fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba45990 0x55c6bba44d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.601+0000 7feb82ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38300 0x55c6bba5ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36582/0 (socket says 172.21.15.44:36582) 2023-09-10T01:17:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.601+0000 7feb82ffd700 1 -- 172.21.15.44:0/3150977173 learned_addr learned my addr 172.21.15.44:0/3150977173 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.601+0000 7feb827fc700 1 --2- 172.21.15.44:0/3150977173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba38cc0 0x55c6bba4d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.602+0000 7feb82ffd700 1 -- 172.21.15.44:0/3150977173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba38cc0 msgr2=0x55c6bba4d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.602+0000 7feb82ffd700 1 --2- 172.21.15.44:0/3150977173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba38cc0 0x55c6bba4d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.602+0000 7feb82ffd700 1 -- 172.21.15.44:0/3150977173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba45990 msgr2=0x55c6bba44d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.602+0000 7feb82ffd700 1 --2- 172.21.15.44:0/3150977173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba45990 0x55c6bba44d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.602+0000 7feb82ffd700 1 -- 172.21.15.44:0/3150977173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6bb7dd1f0 con 0x55c6bba38300 2023-09-10T01:17:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.602+0000 7feb837fe700 1 --2- 172.21.15.44:0/3150977173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba45990 0x55c6bba44d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:17:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.602+0000 7feb82ffd700 1 --2- 172.21.15.44:0/3150977173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38300 0x55c6bba5ca70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7feb74009ee0 tx=0x7feb74030440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92fb327c8774afcc server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.602+0000 7feb69ffb700 1 -- 172.21.15.44:0/3150977173 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb740048d0 con 0x55c6bba38300 2023-09-10T01:17:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.602+0000 7feb69ffb700 1 -- 172.21.15.44:0/3150977173 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb7401dbd0 con 0x55c6bba38300 2023-09-10T01:17:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.602+0000 7feb69ffb700 1 -- 172.21.15.44:0/3150977173 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74004f80 con 0x55c6bba38300 2023-09-10T01:17:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.603+0000 7feb9564fb80 1 -- 172.21.15.44:0/3150977173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38300 msgr2=0x55c6bba5ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.603+0000 7feb9564fb80 1 --2- 172.21.15.44:0/3150977173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38300 0x55c6bba5ca70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7feb74009ee0 tx=0x7feb74030440 comp rx=0 tx=0).stop 2023-09-10T01:17:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.603+0000 7feb9564fb80 1 -- 172.21.15.44:0/3150977173 shutdown_connections 2023-09-10T01:17:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.603+0000 7feb9564fb80 1 --2- 172.21.15.44:0/3150977173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba45990 0x55c6bba44d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.603+0000 7feb9564fb80 1 --2- 172.21.15.44:0/3150977173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38300 0x55c6bba5ca70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.603+0000 7feb9564fb80 1 --2- 172.21.15.44:0/3150977173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba38cc0 0x55c6bba4d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.603+0000 7feb9564fb80 1 -- 172.21.15.44:0/3150977173 >> 172.21.15.44:0/3150977173 conn(0x55c6bb9ad030 msgr2=0x55c6bba56420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.603+0000 7feb9564fb80 1 -- 172.21.15.44:0/3150977173 shutdown_connections 2023-09-10T01:17:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.603+0000 7feb9564fb80 1 -- 172.21.15.44:0/3150977173 wait complete. 2023-09-10T01:17:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.603+0000 7feb9564fb80 1 -- start start 2023-09-10T01:17:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.603+0000 7feb9564fb80 1 Processor -- start 2023-09-10T01:17:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb9564fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba38300 0x55c6bba51ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb9564fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38cc0 0x55c6bba4fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb9564fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba45990 0x55c6bba503f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb9564fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bba50b00 con 0x55c6bba45990 2023-09-10T01:17:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb9564fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bbaad850 con 0x55c6bba38300 2023-09-10T01:17:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb9564fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bbaad990 con 0x55c6bba38cc0 2023-09-10T01:17:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb9564fb80 10 client.0 _ll_register_callbacks cb 0x7feb7c007da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:17:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb827fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38cc0 0x55c6bba4fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb837fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba45990 0x55c6bba503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb827fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38cc0 0x55c6bba4fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36586/0 (socket says 172.21.15.44:36586) 2023-09-10T01:17:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb827fc700 1 -- 172.21.15.44:0/1122380858 learned_addr learned my addr 172.21.15.44:0/1122380858 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb827fc700 1 -- 172.21.15.44:0/1122380858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba38300 msgr2=0x55c6bba51ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb827fc700 1 --2- 172.21.15.44:0/1122380858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba38300 0x55c6bba51ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb827fc700 1 -- 172.21.15.44:0/1122380858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba45990 msgr2=0x55c6bba503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb827fc700 1 --2- 172.21.15.44:0/1122380858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba45990 0x55c6bba503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb827fc700 1 -- 172.21.15.44:0/1122380858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7c003b30 con 0x55c6bba38cc0 2023-09-10T01:17:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb837fe700 1 --2- 172.21.15.44:0/1122380858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba45990 0x55c6bba503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:17:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb827fc700 1 --2- 172.21.15.44:0/1122380858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38cc0 0x55c6bba4fcb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7feb78009990 tx=0x7feb780051c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bad774e9c31e567c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.604+0000 7feb66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:17:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.605+0000 7feb66ff5700 1 -- 172.21.15.44:0/1122380858 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb780997b0 con 0x55c6bba38cc0 2023-09-10T01:17:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.605+0000 7feb66ff5700 1 -- 172.21.15.44:0/1122380858 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(160..160 src has 1..160) v4 ==== 9452+0+0 (secure 0 0 0) 0x7feb7809a0c0 con 0x55c6bba38cc0 2023-09-10T01:17:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.605+0000 7feb9564fb80 1 -- 172.21.15.44:0/1122380858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb74027b00 con 0x55c6bba38cc0 2023-09-10T01:17:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.605+0000 7feb66ff5700 1 -- 172.21.15.44:0/1122380858 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb7809def0 con 0x55c6bba38cc0 2023-09-10T01:17:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.605+0000 7feb9564fb80 10 client.29242 fetch_fsmap learned FSMap version 123 2023-09-10T01:17:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.605+0000 7feb9564fb80 1 -- 172.21.15.44:0/1122380858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=4+}) v3 -- 0x55c6bb79fc00 con 0x55c6bba38cc0 2023-09-10T01:17:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.605+0000 7feb66ff5700 1 -- 172.21.15.44:0/1122380858 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb74027b00 con 0x55c6bba38cc0 2023-09-10T01:17:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.605+0000 7feb66ff5700 1 -- 172.21.15.44:0/1122380858 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 123) v1 ==== 7315+0+0 (secure 0 0 0) 0x7feb780b2180 con 0x55c6bba38cc0 2023-09-10T01:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.605+0000 7feb9564fb80 10 client.29242 fetch_fsmap finished waiting for FSMap version 123 2023-09-10T01:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.605+0000 7feb9564fb80 10 client.29242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.605+0000 7feb9564fb80 20 client.29242 populate_metadata read hostname 'smithi044' 2023-09-10T01:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.605+0000 7feb9564fb80 1 --2- 172.21.15.44:0/1122380858 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c6bbaaedb0 0x55c6bbab11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.605+0000 7feb9564fb80 4 client.29242 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:17:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.605+0000 7feb9564fb80 1 -- 172.21.15.44:0/1122380858 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c6bb9838c0 con 0x55c6bbaaedb0 2023-09-10T01:17:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.605+0000 7feb82ffd700 1 --2- 172.21.15.44:0/1122380858 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c6bbaaedb0 0x55c6bbab11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.606+0000 7feb66ff5700 10 client.29242 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.606+0000 7feb82ffd700 1 --2- 172.21.15.44:0/1122380858 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c6bbaaedb0 0x55c6bbab11d0 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=656b1dfaa14f8a2f server_cookie=ec54cde0b5e128f2 in_seq=0 out_seq=0 2023-09-10T01:17:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.620+0000 7feb66ff5700 1 -- 172.21.15.44:0/1122380858 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c6bb9838c0 con 0x55c6bbaaedb0 2023-09-10T01:17:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.620+0000 7feb66ff5700 10 client.29242 handle_command_reply: tid=0 2023-09-10T01:17:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.620+0000 7feb9564fb80 1 client.29242 shutdown 2023-09-10T01:17:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.620+0000 7feb9564fb80 1 -- 172.21.15.44:0/1122380858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38cc0 msgr2=0x55c6bba4fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.620+0000 7feb9564fb80 1 --2- 172.21.15.44:0/1122380858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38cc0 0x55c6bba4fcb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7feb78009990 tx=0x7feb780051c0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.620+0000 7feb9564fb80 1 -- 172.21.15.44:0/1122380858 shutdown_connections 2023-09-10T01:17:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.620+0000 7feb9564fb80 1 --2- 172.21.15.44:0/1122380858 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c6bbaaedb0 0x55c6bbab11d0 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.620+0000 7feb9564fb80 1 --2- 172.21.15.44:0/1122380858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6bba45990 0x55c6bba503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.620+0000 7feb9564fb80 1 --2- 172.21.15.44:0/1122380858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c6bba38300 0x55c6bba51ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.620+0000 7feb66ff5700 0 client.29242 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.620+0000 7feb9564fb80 1 --2- 172.21.15.44:0/1122380858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6bba38cc0 0x55c6bba4fcb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.620+0000 7feb9564fb80 1 -- 172.21.15.44:0/1122380858 >> 172.21.15.44:0/1122380858 conn(0x55c6bb9ad030 msgr2=0x55c6bba5ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.620+0000 7feb9564fb80 1 -- 172.21.15.44:0/1122380858 shutdown_connections 2023-09-10T01:17:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.620+0000 7feb9564fb80 1 -- 172.21.15.44:0/1122380858 wait complete. 2023-09-10T01:17:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.620+0000 7feb9564fb80 20 client.29242 trim_cache size 0 max 16384 2023-09-10T01:17:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:17:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:17:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:17:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:17:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.621+0000 7feb89489700 1 -- 172.21.15.44:0/530023404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb6c0a07b0 msgr2=0x7feb6c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.621+0000 7feb89489700 1 --2- 172.21.15.44:0/530023404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb6c0a07b0 0x7feb6c0a2c70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7feb74011630 tx=0x7feb74009260 comp rx=0 tx=0).stop 2023-09-10T01:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.621+0000 7feb89489700 1 -- 172.21.15.44:0/530023404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84100120 msgr2=0x7feb84197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.621+0000 7feb89489700 1 --2- 172.21.15.44:0/530023404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84100120 0x7feb84197b00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7feb78000c00 tx=0x7feb7800f690 comp rx=0 tx=0).stop 2023-09-10T01:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.621+0000 7feb89489700 1 -- 172.21.15.44:0/530023404 shutdown_connections 2023-09-10T01:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.621+0000 7feb89489700 1 --2- 172.21.15.44:0/530023404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb6c0a07b0 0x7feb6c0a2c70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.621+0000 7feb89489700 1 --2- 172.21.15.44:0/530023404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841055b0 0x7feb841a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.621+0000 7feb89489700 1 --2- 172.21.15.44:0/530023404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb840ff760 0x7feb841975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.621+0000 7feb89489700 1 --2- 172.21.15.44:0/530023404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84100120 0x7feb84197b00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.621+0000 7feb89489700 1 -- 172.21.15.44:0/530023404 >> 172.21.15.44:0/530023404 conn(0x7feb84074f30 msgr2=0x7feb840fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.621+0000 7feb89489700 1 -- 172.21.15.44:0/530023404 shutdown_connections 2023-09-10T01:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:16.621+0000 7feb89489700 1 -- 172.21.15.44:0/530023404 wait complete. 2023-09-10T01:17:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:17 smithi131 ceph-mon[103713]: osdmap e160: 12 total, 12 up, 12 in 2023-09-10T01:17:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:17 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.44:6836/3498266123,v1:172.21.15.44:6837/3498266123] up:boot 2023-09-10T01:17:17.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:17 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:17:17.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:17 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T01:17:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:17 smithi044 ceph-mon[98676]: osdmap e160: 12 total, 12 up, 12 in 2023-09-10T01:17:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:17 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.44:6836/3498266123,v1:172.21.15.44:6837/3498266123] up:boot 2023-09-10T01:17:17.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:17 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:17:17.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:17 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T01:17:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:17 smithi099 ceph-mon[102692]: osdmap e160: 12 total, 12 up, 12 in 2023-09-10T01:17:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:17 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.44:6836/3498266123,v1:172.21.15.44:6837/3498266123] up:boot 2023-09-10T01:17:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:17 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:17:17.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:17 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T01:17:18.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:18 smithi131 ceph-mon[103713]: pgmap v1177: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 206 op/s 2023-09-10T01:17:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:18 smithi044 ceph-mon[98676]: pgmap v1177: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 206 op/s 2023-09-10T01:17:18.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:18 smithi099 ceph-mon[102692]: pgmap v1177: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 206 op/s 2023-09-10T01:17:20.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:20 smithi131 ceph-mon[103713]: pgmap v1178: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 731 KiB/s wr, 102 op/s 2023-09-10T01:17:20.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:20 smithi044 ceph-mon[98676]: pgmap v1178: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 731 KiB/s wr, 102 op/s 2023-09-10T01:17:20.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:20 smithi099 ceph-mon[102692]: pgmap v1178: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 731 KiB/s wr, 102 op/s 2023-09-10T01:17:22.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:22 smithi131 ceph-mon[103713]: pgmap v1179: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 20 MiB/s wr, 121 op/s 2023-09-10T01:17:22.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:22 smithi044 ceph-mon[98676]: pgmap v1179: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 20 MiB/s wr, 121 op/s 2023-09-10T01:17:22.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:22 smithi099 ceph-mon[102692]: pgmap v1179: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 20 MiB/s wr, 121 op/s 2023-09-10T01:17:24.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:24 smithi131 ceph-mon[103713]: pgmap v1180: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 19 MiB/s wr, 115 op/s 2023-09-10T01:17:24.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:24 smithi044 ceph-mon[98676]: pgmap v1180: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 19 MiB/s wr, 115 op/s 2023-09-10T01:17:24.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:24 smithi099 ceph-mon[102692]: pgmap v1180: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 19 MiB/s wr, 115 op/s 2023-09-10T01:17:26.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:26 smithi131 ceph-mon[103713]: pgmap v1181: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 19 MiB/s wr, 115 op/s 2023-09-10T01:17:26.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:26 smithi044 ceph-mon[98676]: pgmap v1181: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 19 MiB/s wr, 115 op/s 2023-09-10T01:17:26.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:26 smithi099 ceph-mon[102692]: pgmap v1181: 129 pgs: 129 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 19 MiB/s wr, 115 op/s 2023-09-10T01:17:28.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:28 smithi131 ceph-mon[103713]: pgmap v1182: 129 pgs: 129 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 39 MiB/s wr, 51 op/s 2023-09-10T01:17:28.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:28 smithi044 ceph-mon[98676]: pgmap v1182: 129 pgs: 129 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 39 MiB/s wr, 51 op/s 2023-09-10T01:17:28.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:28 smithi099 ceph-mon[102692]: pgmap v1182: 129 pgs: 129 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 39 MiB/s wr, 51 op/s 2023-09-10T01:17:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.753+0000 7f355298e700 1 -- 172.21.15.44:0/772166040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c1055b0 msgr2=0x7f354c105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.753+0000 7f355298e700 1 --2- 172.21.15.44:0/772166040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c1055b0 0x7f354c105990 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f353c0099c0 tx=0x7f353c009cd0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.753+0000 7f355298e700 1 -- 172.21.15.44:0/772166040 shutdown_connections 2023-09-10T01:17:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.754+0000 7f355298e700 1 --2- 172.21.15.44:0/772166040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f354c100120 0x7f354c10c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.754+0000 7f355298e700 1 --2- 172.21.15.44:0/772166040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c1055b0 0x7f354c105990 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.754+0000 7f355298e700 1 --2- 172.21.15.44:0/772166040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f354c0ff760 0x7f354c0ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.754+0000 7f355298e700 1 -- 172.21.15.44:0/772166040 >> 172.21.15.44:0/772166040 conn(0x7f354c074f30 msgr2=0x7f354c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.754+0000 7f355298e700 1 -- 172.21.15.44:0/772166040 shutdown_connections 2023-09-10T01:17:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.754+0000 7f355298e700 1 -- 172.21.15.44:0/772166040 wait complete. 2023-09-10T01:17:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.754+0000 7f355298e700 1 Processor -- start 2023-09-10T01:17:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.754+0000 7f355298e700 1 -- start start 2023-09-10T01:17:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.754+0000 7f355298e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c0ff760 0x7f354c197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.754+0000 7f355298e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f354c100120 0x7f354c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f355298e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f354c1055b0 0x7f354c1a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f355298e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c1a29d0 con 0x7f354c0ff760 2023-09-10T01:17:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f355298e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c1a2b40 con 0x7f354c1055b0 2023-09-10T01:17:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f355298e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c1a2d00 con 0x7f354c100120 2023-09-10T01:17:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f354bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c0ff760 0x7f354c197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f354bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c0ff760 0x7f354c197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57950/0 (socket says 172.21.15.44:57950) 2023-09-10T01:17:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f354bfff700 1 -- 172.21.15.44:0/376769462 learned_addr learned my addr 172.21.15.44:0/376769462 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f3543fff700 1 --2- 172.21.15.44:0/376769462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f354c100120 0x7f354c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f3550f2b700 1 --2- 172.21.15.44:0/376769462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f354c1055b0 0x7f354c1a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f354bfff700 1 -- 172.21.15.44:0/376769462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f354c100120 msgr2=0x7f354c197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f354bfff700 1 --2- 172.21.15.44:0/376769462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f354c100120 0x7f354c197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f354bfff700 1 -- 172.21.15.44:0/376769462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f354c1055b0 msgr2=0x7f354c1a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f354bfff700 1 --2- 172.21.15.44:0/376769462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f354c1055b0 0x7f354c1a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f354bfff700 1 -- 172.21.15.44:0/376769462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353c0095f0 con 0x7f354c0ff760 2023-09-10T01:17:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f3550f2b700 1 --2- 172.21.15.44:0/376769462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f354c1055b0 0x7f354c1a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:17:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f3543fff700 1 --2- 172.21.15.44:0/376769462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f354c100120 0x7f354c197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:17:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f354bfff700 1 --2- 172.21.15.44:0/376769462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c0ff760 0x7f354c197560 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f353c005910 tx=0x7f353c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f3549ffb700 1 -- 172.21.15.44:0/376769462 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c01d070 con 0x7f354c0ff760 2023-09-10T01:17:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f3549ffb700 1 -- 172.21.15.44:0/376769462 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f353c0054e0 con 0x7f354c0ff760 2023-09-10T01:17:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f355298e700 1 -- 172.21.15.44:0/376769462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354c195230 con 0x7f354c0ff760 2023-09-10T01:17:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.755+0000 7f3549ffb700 1 -- 172.21.15.44:0/376769462 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c004a90 con 0x7f354c0ff760 2023-09-10T01:17:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.756+0000 7f355298e700 1 -- 172.21.15.44:0/376769462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c195730 con 0x7f354c0ff760 2023-09-10T01:17:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.758+0000 7f3549ffb700 1 -- 172.21.15.44:0/376769462 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f353c00fb80 con 0x7f354c0ff760 2023-09-10T01:17:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.758+0000 7f3549ffb700 1 --2- 172.21.15.44:0/376769462 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f35380a0830 0x7f35380a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.759+0000 7f3549ffb700 1 -- 172.21.15.44:0/376769462 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f353c0cd050 con 0x7f354c0ff760 2023-09-10T01:17:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.759+0000 7f3543fff700 1 --2- 172.21.15.44:0/376769462 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f35380a0830 0x7f35380a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.759+0000 7f3543fff700 1 --2- 172.21.15.44:0/376769462 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f35380a0830 0x7f35380a2cf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f354c0684b0 tx=0x7f3534009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f355eb54b80 1 Processor -- start 2023-09-10T01:17:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f355eb54b80 1 -- start start 2023-09-10T01:17:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f355eb54b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f231a0 0x5594a2f23580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f355eb54b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f1d300 0x5594a2f1d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f355eb54b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f1dcc0 0x5594a2f29f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f355eb54b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a2e766d0 con 0x5594a2f231a0 2023-09-10T01:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f355eb54b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a2e73ff0 con 0x5594a2f1dcc0 2023-09-10T01:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f355eb54b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a2e6f8e0 con 0x5594a2f1d300 2023-09-10T01:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f354bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f231a0 0x5594a2f23580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f354bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f231a0 0x5594a2f23580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57958/0 (socket says 172.21.15.44:57958) 2023-09-10T01:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f354bfff700 1 -- 172.21.15.44:0/1097177835 learned_addr learned my addr 172.21.15.44:0/1097177835 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f3543fff700 1 --2- 172.21.15.44:0/1097177835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f1d300 0x5594a2f1d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f3550f2b700 1 --2- 172.21.15.44:0/1097177835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f1dcc0 0x5594a2f29f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f354bfff700 1 -- 172.21.15.44:0/1097177835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f1d300 msgr2=0x5594a2f1d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f354bfff700 1 --2- 172.21.15.44:0/1097177835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f1d300 0x5594a2f1d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f354bfff700 1 -- 172.21.15.44:0/1097177835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f1dcc0 msgr2=0x5594a2f29f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.760+0000 7f354bfff700 1 --2- 172.21.15.44:0/1097177835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f1dcc0 0x5594a2f29f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.761+0000 7f354bfff700 1 -- 172.21.15.44:0/1097177835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594a2c9bd80 con 0x5594a2f231a0 2023-09-10T01:17:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.761+0000 7f3550f2b700 1 --2- 172.21.15.44:0/1097177835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f1dcc0 0x5594a2f29f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:17:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.761+0000 7f354bfff700 1 --2- 172.21.15.44:0/1097177835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f231a0 0x5594a2f23580 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f353c0966f0 tx=0x7f353c099e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe1f93299dab8c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.761+0000 7f35427fc700 1 -- 172.21.15.44:0/1097177835 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c0a00f0 con 0x5594a2f231a0 2023-09-10T01:17:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.761+0000 7f35427fc700 1 -- 172.21.15.44:0/1097177835 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f353c09e020 con 0x5594a2f231a0 2023-09-10T01:17:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.761+0000 7f35427fc700 1 -- 172.21.15.44:0/1097177835 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c004e80 con 0x5594a2f231a0 2023-09-10T01:17:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.762+0000 7f355eb54b80 1 -- 172.21.15.44:0/1097177835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f231a0 msgr2=0x5594a2f23580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.762+0000 7f355eb54b80 1 --2- 172.21.15.44:0/1097177835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f231a0 0x5594a2f23580 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f353c0966f0 tx=0x7f353c099e40 comp rx=0 tx=0).stop 2023-09-10T01:17:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.762+0000 7f355eb54b80 1 -- 172.21.15.44:0/1097177835 shutdown_connections 2023-09-10T01:17:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.762+0000 7f355eb54b80 1 --2- 172.21.15.44:0/1097177835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f1dcc0 0x5594a2f29f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.762+0000 7f355eb54b80 1 --2- 172.21.15.44:0/1097177835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f231a0 0x5594a2f23580 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.762+0000 7f355eb54b80 1 --2- 172.21.15.44:0/1097177835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f1d300 0x5594a2f1d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.762+0000 7f355eb54b80 1 -- 172.21.15.44:0/1097177835 >> 172.21.15.44:0/1097177835 conn(0x5594a2e92030 msgr2=0x5594a2f18560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.762+0000 7f355eb54b80 1 -- 172.21.15.44:0/1097177835 shutdown_connections 2023-09-10T01:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.762+0000 7f355eb54b80 1 -- 172.21.15.44:0/1097177835 wait complete. 2023-09-10T01:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.762+0000 7f355eb54b80 1 -- start start 2023-09-10T01:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.762+0000 7f355eb54b80 1 Processor -- start 2023-09-10T01:17:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f355eb54b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f1d300 0x5594a2f2cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f355eb54b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1dcc0 0x5594a2f2d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f355eb54b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f231a0 0x5594a2f26de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f355eb54b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a2e0db10 con 0x5594a2f1dcc0 2023-09-10T01:17:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f355eb54b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a2e0d810 con 0x5594a2f1d300 2023-09-10T01:17:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f355eb54b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a2cf18c0 con 0x5594a2f231a0 2023-09-10T01:17:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f355eb54b80 10 client.0 _ll_register_callbacks cb 0x5594a2d57a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:17:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f3543fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1dcc0 0x5594a2f2d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f3550f2b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f231a0 0x5594a2f26de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f3543fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1dcc0 0x5594a2f2d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57966/0 (socket says 172.21.15.44:57966) 2023-09-10T01:17:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f3543fff700 1 -- 172.21.15.44:0/19785544 learned_addr learned my addr 172.21.15.44:0/19785544 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f3543fff700 1 -- 172.21.15.44:0/19785544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f231a0 msgr2=0x5594a2f26de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f3543fff700 1 --2- 172.21.15.44:0/19785544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f231a0 0x5594a2f26de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f3543fff700 1 -- 172.21.15.44:0/19785544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f1d300 msgr2=0x5594a2f2cb90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:17:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f3543fff700 1 --2- 172.21.15.44:0/19785544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f1d300 0x5594a2f2cb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f3543fff700 1 -- 172.21.15.44:0/19785544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c195730 con 0x5594a2f1dcc0 2023-09-10T01:17:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f3543fff700 1 --2- 172.21.15.44:0/19785544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1dcc0 0x5594a2f2d0d0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f35340039b0 tx=0x7f35340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6719843bdc0c5db2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.763+0000 7f352b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:17:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.764+0000 7f352b7fe700 1 -- 172.21.15.44:0/19785544 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353401fca0 con 0x5594a2f1dcc0 2023-09-10T01:17:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.764+0000 7f352b7fe700 1 -- 172.21.15.44:0/19785544 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(160..160 src has 1..160) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f3534003e30 con 0x5594a2f1dcc0 2023-09-10T01:17:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.764+0000 7f355eb54b80 1 -- 172.21.15.44:0/19785544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594a2dc7490 con 0x5594a2f1dcc0 2023-09-10T01:17:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.764+0000 7f352b7fe700 1 -- 172.21.15.44:0/19785544 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7f353401fe00 con 0x5594a2f1dcc0 2023-09-10T01:17:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.764+0000 7f355eb54b80 10 client.19647 fetch_fsmap learned FSMap version 123 2023-09-10T01:17:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.764+0000 7f355eb54b80 1 -- 172.21.15.44:0/19785544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=4+}) v3 -- 0x5594a2c84c00 con 0x5594a2f1dcc0 2023-09-10T01:17:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.764+0000 7f352b7fe700 1 -- 172.21.15.44:0/19785544 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5594a2dc7490 con 0x5594a2f1dcc0 2023-09-10T01:17:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.764+0000 7f352b7fe700 1 -- 172.21.15.44:0/19785544 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 123) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f35340273e0 con 0x5594a2f1dcc0 2023-09-10T01:17:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.764+0000 7f355eb54b80 10 client.19647 fetch_fsmap finished waiting for FSMap version 123 2023-09-10T01:17:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.764+0000 7f355eb54b80 10 client.19647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:17:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.764+0000 7f355eb54b80 20 client.19647 populate_metadata read hostname 'smithi044' 2023-09-10T01:17:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.764+0000 7f355eb54b80 1 --2- 172.21.15.44:0/19785544 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5594a2f370f0 0x5594a2f39510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.764+0000 7f355eb54b80 4 client.19647 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:17:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.764+0000 7f355eb54b80 1 -- 172.21.15.44:0/19785544 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594a2e84840 con 0x5594a2f370f0 2023-09-10T01:17:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.764+0000 7f354bfff700 1 --2- 172.21.15.44:0/19785544 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5594a2f370f0 0x5594a2f39510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.765+0000 7f352b7fe700 10 client.19647 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.765+0000 7f354bfff700 1 --2- 172.21.15.44:0/19785544 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5594a2f370f0 0x5594a2f39510 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a25e89e34ff1843 server_cookie=7035cac1764f5bc7 in_seq=0 out_seq=0 2023-09-10T01:17:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.765+0000 7f352b7fe700 1 -- 172.21.15.44:0/19785544 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5594a2e84840 con 0x5594a2f370f0 2023-09-10T01:17:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.765+0000 7f352b7fe700 10 client.19647 handle_command_reply: tid=0 2023-09-10T01:17:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.766+0000 7f355eb54b80 1 client.19647 shutdown 2023-09-10T01:17:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.766+0000 7f355eb54b80 1 -- 172.21.15.44:0/19785544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1dcc0 msgr2=0x5594a2f2d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.766+0000 7f355eb54b80 1 --2- 172.21.15.44:0/19785544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1dcc0 0x5594a2f2d0d0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f35340039b0 tx=0x7f35340047c0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.766+0000 7f355eb54b80 1 -- 172.21.15.44:0/19785544 shutdown_connections 2023-09-10T01:17:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.766+0000 7f355eb54b80 1 --2- 172.21.15.44:0/19785544 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5594a2f370f0 0x5594a2f39510 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.766+0000 7f355eb54b80 1 --2- 172.21.15.44:0/19785544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f231a0 0x5594a2f26de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.766+0000 7f355eb54b80 1 --2- 172.21.15.44:0/19785544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f1d300 0x5594a2f2cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.766+0000 7f352b7fe700 0 client.19647 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.766+0000 7f355eb54b80 1 --2- 172.21.15.44:0/19785544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1dcc0 0x5594a2f2d0d0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.766+0000 7f355eb54b80 1 -- 172.21.15.44:0/19785544 >> 172.21.15.44:0/19785544 conn(0x5594a2e92030 msgr2=0x5594a2f177a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.766+0000 7f355eb54b80 1 -- 172.21.15.44:0/19785544 shutdown_connections 2023-09-10T01:17:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.766+0000 7f355eb54b80 1 -- 172.21.15.44:0/19785544 wait complete. 2023-09-10T01:17:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.766+0000 7f355eb54b80 20 client.19647 trim_cache size 0 max 16384 2023-09-10T01:17:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.789+0000 7f355eb54b80 1 Processor -- start 2023-09-10T01:17:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.789+0000 7f355eb54b80 1 -- start start 2023-09-10T01:17:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.789+0000 7f355eb54b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f1d300 0x5594a2f17f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.789+0000 7f355eb54b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1dcc0 0x5594a2f18460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.789+0000 7f355eb54b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f231a0 0x5594a2f21ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.789+0000 7f355eb54b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a2f34750 con 0x5594a2f1dcc0 2023-09-10T01:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.789+0000 7f355eb54b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a2f1e0a0 con 0x5594a2f1d300 2023-09-10T01:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.789+0000 7f355eb54b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a2f22e00 con 0x5594a2f231a0 2023-09-10T01:17:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.790+0000 7f3543fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1dcc0 0x5594a2f18460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.790+0000 7f3550f2b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f231a0 0x5594a2f21ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.790+0000 7f3543fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1dcc0 0x5594a2f18460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57976/0 (socket says 172.21.15.44:57976) 2023-09-10T01:17:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.790+0000 7f3543fff700 1 -- 172.21.15.44:0/3373476471 learned_addr learned my addr 172.21.15.44:0/3373476471 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.790+0000 7f354bfff700 1 --2- 172.21.15.44:0/3373476471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f1d300 0x5594a2f17f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.790+0000 7f3543fff700 1 -- 172.21.15.44:0/3373476471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f231a0 msgr2=0x5594a2f21ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.790+0000 7f3543fff700 1 --2- 172.21.15.44:0/3373476471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f231a0 0x5594a2f21ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.790+0000 7f3543fff700 1 -- 172.21.15.44:0/3373476471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f1d300 msgr2=0x5594a2f17f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.790+0000 7f3543fff700 1 --2- 172.21.15.44:0/3373476471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f1d300 0x5594a2f17f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.790+0000 7f3543fff700 1 -- 172.21.15.44:0/3373476471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594a2cc21f0 con 0x5594a2f1dcc0 2023-09-10T01:17:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.790+0000 7f3543fff700 1 --2- 172.21.15.44:0/3373476471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1dcc0 0x5594a2f18460 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f3534000c00 tx=0x7f3534009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9336c4eb966e06 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.791+0000 7f3542ffd700 1 -- 172.21.15.44:0/3373476471 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35340275e0 con 0x5594a2f1dcc0 2023-09-10T01:17:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.791+0000 7f3542ffd700 1 -- 172.21.15.44:0/3373476471 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3534003e30 con 0x5594a2f1dcc0 2023-09-10T01:17:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.791+0000 7f3542ffd700 1 -- 172.21.15.44:0/3373476471 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35340366a0 con 0x5594a2f1dcc0 2023-09-10T01:17:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.791+0000 7f355eb54b80 1 -- 172.21.15.44:0/3373476471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1dcc0 msgr2=0x5594a2f18460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.791+0000 7f355eb54b80 1 --2- 172.21.15.44:0/3373476471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1dcc0 0x5594a2f18460 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f3534000c00 tx=0x7f3534009390 comp rx=0 tx=0).stop 2023-09-10T01:17:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.792+0000 7f355eb54b80 1 -- 172.21.15.44:0/3373476471 shutdown_connections 2023-09-10T01:17:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.792+0000 7f355eb54b80 1 --2- 172.21.15.44:0/3373476471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f231a0 0x5594a2f21ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.792+0000 7f355eb54b80 1 --2- 172.21.15.44:0/3373476471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f1d300 0x5594a2f17f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.792+0000 7f355eb54b80 1 --2- 172.21.15.44:0/3373476471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1dcc0 0x5594a2f18460 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.792+0000 7f355eb54b80 1 -- 172.21.15.44:0/3373476471 >> 172.21.15.44:0/3373476471 conn(0x5594a2e92030 msgr2=0x5594a2f276b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.792+0000 7f355eb54b80 1 -- 172.21.15.44:0/3373476471 shutdown_connections 2023-09-10T01:17:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.792+0000 7f355eb54b80 1 -- 172.21.15.44:0/3373476471 wait complete. 2023-09-10T01:17:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.792+0000 7f355eb54b80 1 -- start start 2023-09-10T01:17:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.793+0000 7f355eb54b80 1 Processor -- start 2023-09-10T01:17:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.793+0000 7f355eb54b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1d300 0x5594a2f7f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.793+0000 7f355eb54b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f1dcc0 0x5594a2f7f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.793+0000 7f355eb54b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f231a0 0x5594a2f85f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.793+0000 7f355eb54b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a2f7fd50 con 0x5594a2f1d300 2023-09-10T01:17:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.793+0000 7f355eb54b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a2f86470 con 0x5594a2f231a0 2023-09-10T01:17:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.793+0000 7f355eb54b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a2f86630 con 0x5594a2f1dcc0 2023-09-10T01:17:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.793+0000 7f355eb54b80 10 client.0 _ll_register_callbacks cb 0x7f3534004620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:17:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.793+0000 7f354bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1d300 0x5594a2f7f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.793+0000 7f354bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1d300 0x5594a2f7f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57978/0 (socket says 172.21.15.44:57978) 2023-09-10T01:17:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.793+0000 7f354bfff700 1 -- 172.21.15.44:0/1117771047 learned_addr learned my addr 172.21.15.44:0/1117771047 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.794+0000 7f354bfff700 1 -- 172.21.15.44:0/1117771047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f1dcc0 msgr2=0x5594a2f7f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.794+0000 7f354bfff700 1 --2- 172.21.15.44:0/1117771047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f1dcc0 0x5594a2f7f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.794+0000 7f354bfff700 1 -- 172.21.15.44:0/1117771047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f231a0 msgr2=0x5594a2f85f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:17:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.794+0000 7f354bfff700 1 --2- 172.21.15.44:0/1117771047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f231a0 0x5594a2f85f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.794+0000 7f354bfff700 1 -- 172.21.15.44:0/1117771047 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594a2c84c00 con 0x5594a2f1d300 2023-09-10T01:17:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.794+0000 7f354bfff700 1 --2- 172.21.15.44:0/1117771047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1d300 0x5594a2f7f0c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f353c099ef0 tx=0x7f353c09a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9879bf48603fdac server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.794+0000 7f352b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:17:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.794+0000 7f352b7fe700 1 -- 172.21.15.44:0/1117771047 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c0b8170 con 0x5594a2f1d300 2023-09-10T01:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.794+0000 7f352b7fe700 1 -- 172.21.15.44:0/1117771047 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(160..160 src has 1..160) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f353c0b2070 con 0x5594a2f1d300 2023-09-10T01:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.794+0000 7f355eb54b80 1 -- 172.21.15.44:0/1117771047 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f353c09e020 con 0x5594a2f1d300 2023-09-10T01:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.795+0000 7f352b7fe700 1 -- 172.21.15.44:0/1117771047 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7f353c0a0040 con 0x5594a2f1d300 2023-09-10T01:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.795+0000 7f355eb54b80 10 client.19653 fetch_fsmap learned FSMap version 123 2023-09-10T01:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.795+0000 7f355eb54b80 1 -- 172.21.15.44:0/1117771047 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=4+}) v3 -- 0x5594a2c9a250 con 0x5594a2f1d300 2023-09-10T01:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.795+0000 7f352b7fe700 1 -- 172.21.15.44:0/1117771047 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f353c09e020 con 0x5594a2f1d300 2023-09-10T01:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.795+0000 7f352b7fe700 1 -- 172.21.15.44:0/1117771047 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 123) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f353c09a4e0 con 0x5594a2f1d300 2023-09-10T01:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.795+0000 7f355eb54b80 10 client.19653 fetch_fsmap finished waiting for FSMap version 123 2023-09-10T01:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.795+0000 7f355eb54b80 10 client.19653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.795+0000 7f355eb54b80 20 client.19653 populate_metadata read hostname 'smithi044' 2023-09-10T01:17:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.795+0000 7f355eb54b80 1 --2- 172.21.15.44:0/1117771047 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5594a2f9c9e0 0x5594a2f9ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.795+0000 7f355eb54b80 4 client.19653 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:17:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.795+0000 7f355eb54b80 1 -- 172.21.15.44:0/1117771047 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5594a2e688c0 con 0x5594a2f9c9e0 2023-09-10T01:17:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.796+0000 7f3543fff700 1 --2- 172.21.15.44:0/1117771047 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5594a2f9c9e0 0x5594a2f9ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.796+0000 7f3543fff700 1 --2- 172.21.15.44:0/1117771047 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5594a2f9c9e0 0x5594a2f9ee00 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bd80393f647192d server_cookie=88b9eca1a9d762f9 in_seq=0 out_seq=0 2023-09-10T01:17:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.796+0000 7f352b7fe700 10 client.19653 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.822+0000 7f352b7fe700 1 -- 172.21.15.44:0/1117771047 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5594a2e688c0 con 0x5594a2f9c9e0 2023-09-10T01:17:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.822+0000 7f352b7fe700 10 client.19653 handle_command_reply: tid=0 2023-09-10T01:17:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.822+0000 7f355eb54b80 1 client.19653 shutdown 2023-09-10T01:17:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.822+0000 7f355eb54b80 1 -- 172.21.15.44:0/1117771047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1d300 msgr2=0x5594a2f7f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.822+0000 7f355eb54b80 1 --2- 172.21.15.44:0/1117771047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1d300 0x5594a2f7f0c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f353c099ef0 tx=0x7f353c09a370 comp rx=0 tx=0).stop 2023-09-10T01:17:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355eb54b80 1 -- 172.21.15.44:0/1117771047 shutdown_connections 2023-09-10T01:17:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355eb54b80 1 --2- 172.21.15.44:0/1117771047 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5594a2f9c9e0 0x5594a2f9ee00 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355eb54b80 1 --2- 172.21.15.44:0/1117771047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594a2f231a0 0x5594a2f85f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355eb54b80 1 --2- 172.21.15.44:0/1117771047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a2f1d300 0x5594a2f7f0c0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f352b7fe700 0 client.19653 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355eb54b80 1 --2- 172.21.15.44:0/1117771047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5594a2f1dcc0 0x5594a2f7f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355eb54b80 1 -- 172.21.15.44:0/1117771047 >> 172.21.15.44:0/1117771047 conn(0x5594a2e92030 msgr2=0x5594a2f1c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355eb54b80 1 -- 172.21.15.44:0/1117771047 shutdown_connections 2023-09-10T01:17:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355eb54b80 1 -- 172.21.15.44:0/1117771047 wait complete. 2023-09-10T01:17:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355eb54b80 20 client.19653 trim_cache size 0 max 16384 2023-09-10T01:17:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:17:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:17:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:17:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:17:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355298e700 1 -- 172.21.15.44:0/376769462 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f35380a0830 msgr2=0x7f35380a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355298e700 1 --2- 172.21.15.44:0/376769462 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f35380a0830 0x7f35380a2cf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f354c0684b0 tx=0x7f3534009260 comp rx=0 tx=0).stop 2023-09-10T01:17:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355298e700 1 -- 172.21.15.44:0/376769462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c0ff760 msgr2=0x7f354c197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355298e700 1 --2- 172.21.15.44:0/376769462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c0ff760 0x7f354c197560 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f353c005910 tx=0x7f353c00f690 comp rx=0 tx=0).stop 2023-09-10T01:17:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355298e700 1 -- 172.21.15.44:0/376769462 shutdown_connections 2023-09-10T01:17:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355298e700 1 --2- 172.21.15.44:0/376769462 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f35380a0830 0x7f35380a2cf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355298e700 1 --2- 172.21.15.44:0/376769462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f354c1055b0 0x7f354c1a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355298e700 1 --2- 172.21.15.44:0/376769462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f354c0ff760 0x7f354c197560 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355298e700 1 --2- 172.21.15.44:0/376769462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f354c100120 0x7f354c197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.823+0000 7f355298e700 1 -- 172.21.15.44:0/376769462 >> 172.21.15.44:0/376769462 conn(0x7f354c074f30 msgr2=0x7f354c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.824+0000 7f355298e700 1 -- 172.21.15.44:0/376769462 shutdown_connections 2023-09-10T01:17:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:29.824+0000 7f355298e700 1 -- 172.21.15.44:0/376769462 wait complete. 2023-09-10T01:17:30.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:30 smithi131 ceph-mon[103713]: pgmap v1183: 129 pgs: 129 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 34 MiB/s wr, 45 op/s 2023-09-10T01:17:30.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:30 smithi044 ceph-mon[98676]: pgmap v1183: 129 pgs: 129 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 34 MiB/s wr, 45 op/s 2023-09-10T01:17:30.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:30 smithi099 ceph-mon[102692]: pgmap v1183: 129 pgs: 129 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 34 MiB/s wr, 45 op/s 2023-09-10T01:17:32.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:32 smithi131 ceph-mon[103713]: pgmap v1184: 129 pgs: 129 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 53 MiB/s wr, 64 op/s 2023-09-10T01:17:32.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:32 smithi044 ceph-mon[98676]: pgmap v1184: 129 pgs: 129 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 53 MiB/s wr, 64 op/s 2023-09-10T01:17:32.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:32 smithi099 ceph-mon[102692]: pgmap v1184: 129 pgs: 129 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 53 MiB/s wr, 64 op/s 2023-09-10T01:17:34.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:34 smithi131 ceph-mon[103713]: pgmap v1185: 129 pgs: 129 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 37 MiB/s wr, 36 op/s 2023-09-10T01:17:34.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:34 smithi044 ceph-mon[98676]: pgmap v1185: 129 pgs: 129 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 37 MiB/s wr, 36 op/s 2023-09-10T01:17:34.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:34 smithi099 ceph-mon[102692]: pgmap v1185: 129 pgs: 129 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 37 MiB/s wr, 36 op/s 2023-09-10T01:17:36.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:36 smithi131 ceph-mon[103713]: pgmap v1186: 129 pgs: 129 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 37 MiB/s wr, 36 op/s 2023-09-10T01:17:36.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:36 smithi044 ceph-mon[98676]: pgmap v1186: 129 pgs: 129 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 37 MiB/s wr, 36 op/s 2023-09-10T01:17:36.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:36 smithi099 ceph-mon[102692]: pgmap v1186: 129 pgs: 129 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 37 MiB/s wr, 36 op/s 2023-09-10T01:17:38.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:38 smithi131 ceph-mon[103713]: pgmap v1187: 129 pgs: 129 active+clean; 12 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 56 MiB/s wr, 53 op/s 2023-09-10T01:17:38.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:38 smithi044 ceph-mon[98676]: pgmap v1187: 129 pgs: 129 active+clean; 12 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 56 MiB/s wr, 53 op/s 2023-09-10T01:17:38.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:38 smithi099 ceph-mon[102692]: pgmap v1187: 129 pgs: 129 active+clean; 12 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 56 MiB/s wr, 53 op/s 2023-09-10T01:17:40.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:40 smithi131 ceph-mon[103713]: pgmap v1188: 129 pgs: 129 active+clean; 12 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 37 MiB/s wr, 36 op/s 2023-09-10T01:17:40.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:40 smithi044 ceph-mon[98676]: pgmap v1188: 129 pgs: 129 active+clean; 12 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 37 MiB/s wr, 36 op/s 2023-09-10T01:17:40.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:40 smithi099 ceph-mon[102692]: pgmap v1188: 129 pgs: 129 active+clean; 12 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 37 MiB/s wr, 36 op/s 2023-09-10T01:17:42.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:42 smithi044 ceph-mon[98676]: pgmap v1189: 129 pgs: 129 active+clean; 12 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 56 MiB/s wr, 54 op/s 2023-09-10T01:17:42.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:42 smithi099 ceph-mon[102692]: pgmap v1189: 129 pgs: 129 active+clean; 12 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 56 MiB/s wr, 54 op/s 2023-09-10T01:17:42.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:42 smithi131 ceph-mon[103713]: pgmap v1189: 129 pgs: 129 active+clean; 12 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 56 MiB/s wr, 54 op/s 2023-09-10T01:17:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.953+0000 7f7fbc74d700 1 -- 172.21.15.44:0/1621509248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fb41005b0 msgr2=0x7f7fb4100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.953+0000 7f7fbc74d700 1 --2- 172.21.15.44:0/1621509248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fb41005b0 0x7f7fb4100a30 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40099c0 tx=0x7f7fa4009cd0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.953+0000 7f7fbc74d700 1 -- 172.21.15.44:0/1621509248 shutdown_connections 2023-09-10T01:17:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.953+0000 7f7fbc74d700 1 --2- 172.21.15.44:0/1621509248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fb4100f70 0x7f7fb410d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.953+0000 7f7fbc74d700 1 --2- 172.21.15.44:0/1621509248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb4106670 0x7f7fb4106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.953+0000 7f7fbc74d700 1 --2- 172.21.15.44:0/1621509248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fb41005b0 0x7f7fb4100a30 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.953+0000 7f7fbc74d700 1 -- 172.21.15.44:0/1621509248 >> 172.21.15.44:0/1621509248 conn(0x7f7fb40fc060 msgr2=0x7f7fb40fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.953+0000 7f7fbc74d700 1 -- 172.21.15.44:0/1621509248 shutdown_connections 2023-09-10T01:17:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.953+0000 7f7fbc74d700 1 -- 172.21.15.44:0/1621509248 wait complete. 2023-09-10T01:17:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.954+0000 7f7fbc74d700 1 Processor -- start 2023-09-10T01:17:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.954+0000 7f7fbc74d700 1 -- start start 2023-09-10T01:17:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.954+0000 7f7fbc74d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fb41005b0 0x7f7fb4072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.954+0000 7f7fbc74d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb4100f70 0x7f7fb4073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.954+0000 7f7fbc74d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fb4106670 0x7f7fb41a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.954+0000 7f7fbc74d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb41a4c50 con 0x7f7fb4106670 2023-09-10T01:17:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.954+0000 7f7fbc74d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb41a4d90 con 0x7f7fb41005b0 2023-09-10T01:17:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.954+0000 7f7fbc74d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb41a4f00 con 0x7f7fb4100f70 2023-09-10T01:17:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.955+0000 7f7fbacea700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fb4106670 0x7f7fb41a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.955+0000 7f7fbacea700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fb4106670 0x7f7fb41a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57992/0 (socket says 172.21.15.44:57992) 2023-09-10T01:17:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.955+0000 7f7fbacea700 1 -- 172.21.15.44:0/4293333983 learned_addr learned my addr 172.21.15.44:0/4293333983 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.955+0000 7f7fbacea700 1 -- 172.21.15.44:0/4293333983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb4100f70 msgr2=0x7f7fb4073020 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-10T01:17:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.955+0000 7f7fbacea700 1 --2- 172.21.15.44:0/4293333983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb4100f70 0x7f7fb4073020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.955+0000 7f7fbacea700 1 -- 172.21.15.44:0/4293333983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fb41005b0 msgr2=0x7f7fb4072ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-10T01:17:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.955+0000 7f7fbacea700 1 --2- 172.21.15.44:0/4293333983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fb41005b0 0x7f7fb4072ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.955+0000 7f7fbacea700 1 -- 172.21.15.44:0/4293333983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa40095f0 con 0x7f7fb4106670 2023-09-10T01:17:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.955+0000 7f7fbacea700 1 --2- 172.21.15.44:0/4293333983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fb4106670 0x7f7fb41a4710 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00ef30 tx=0x7f7fac00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.955+0000 7f7fab7fe700 1 -- 172.21.15.44:0/4293333983 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac018070 con 0x7f7fb4106670 2023-09-10T01:17:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.956+0000 7f7fbc74d700 1 -- 172.21.15.44:0/4293333983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb41a5180 con 0x7f7fb4106670 2023-09-10T01:17:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.956+0000 7f7fbc74d700 1 -- 172.21.15.44:0/4293333983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb41a5690 con 0x7f7fb4106670 2023-09-10T01:17:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.958+0000 7f7fab7fe700 1 -- 172.21.15.44:0/4293333983 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7fac00f040 con 0x7f7fb4106670 2023-09-10T01:17:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.958+0000 7f7fab7fe700 1 -- 172.21.15.44:0/4293333983 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac003d70 con 0x7f7fb4106670 2023-09-10T01:17:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.958+0000 7f7fab7fe700 1 -- 172.21.15.44:0/4293333983 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f7fac003fb0 con 0x7f7fb4106670 2023-09-10T01:17:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.959+0000 7f7fab7fe700 1 --2- 172.21.15.44:0/4293333983 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7fa00a0830 0x7f7fa00a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.959+0000 7f7fab7fe700 1 -- 172.21.15.44:0/4293333983 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7fac0ccd30 con 0x7f7fb4106670 2023-09-10T01:17:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.959+0000 7f7fba4e9700 1 --2- 172.21.15.44:0/4293333983 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7fa00a0830 0x7f7fa00a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.960+0000 7f7fba4e9700 1 --2- 172.21.15.44:0/4293333983 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7fa00a0830 0x7f7fa00a2cf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009af0 tx=0x7f7fb0009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.960+0000 7f7fc8913b80 1 Processor -- start 2023-09-10T01:17:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fc8913b80 1 -- start start 2023-09-10T01:17:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fc8913b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c6f1a0 0x55f059c6f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fc8913b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c69300 0x55f059c69780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fc8913b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c69cc0 0x55f059c75f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fc8913b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f059bc26d0 con 0x55f059c6f1a0 2023-09-10T01:17:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fc8913b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f059bbfff0 con 0x55f059c69cc0 2023-09-10T01:17:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fc8913b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f059bbb8e0 con 0x55f059c69300 2023-09-10T01:17:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fba4e9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c6f1a0 0x55f059c6f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fb9ce8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c69300 0x55f059c69780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fb9ce8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c69300 0x55f059c69780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36652/0 (socket says 172.21.15.44:36652) 2023-09-10T01:17:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fba4e9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c6f1a0 0x55f059c6f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58000/0 (socket says 172.21.15.44:58000) 2023-09-10T01:17:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fb9ce8700 1 -- 172.21.15.44:0/2310307550 learned_addr learned my addr 172.21.15.44:0/2310307550 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fbacea700 1 --2- 172.21.15.44:0/2310307550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c69cc0 0x55f059c75f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fb9ce8700 1 -- 172.21.15.44:0/2310307550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c69cc0 msgr2=0x55f059c75f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fb9ce8700 1 --2- 172.21.15.44:0/2310307550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c69cc0 0x55f059c75f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fb9ce8700 1 -- 172.21.15.44:0/2310307550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c6f1a0 msgr2=0x55f059c6f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fb9ce8700 1 --2- 172.21.15.44:0/2310307550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c6f1a0 0x55f059c6f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.961+0000 7f7fb9ce8700 1 -- 172.21.15.44:0/2310307550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0599e7d80 con 0x55f059c69300 2023-09-10T01:17:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.962+0000 7f7fb9ce8700 1 --2- 172.21.15.44:0/2310307550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c69300 0x55f059c69780 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4000c00 tx=0x7f7fa4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eca5205c674e751d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.962+0000 7f7fa8ff9700 1 -- 172.21.15.44:0/2310307550 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa401d070 con 0x55f059c69300 2023-09-10T01:17:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.962+0000 7f7fa8ff9700 1 -- 172.21.15.44:0/2310307550 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7fa40044d0 con 0x55f059c69300 2023-09-10T01:17:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.962+0000 7f7fa8ff9700 1 -- 172.21.15.44:0/2310307550 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4003760 con 0x55f059c69300 2023-09-10T01:17:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.963+0000 7f7fc8913b80 1 -- 172.21.15.44:0/2310307550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c69300 msgr2=0x55f059c69780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.963+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/2310307550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c69300 0x55f059c69780 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4000c00 tx=0x7f7fa4009580 comp rx=0 tx=0).stop 2023-09-10T01:17:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.963+0000 7f7fc8913b80 1 -- 172.21.15.44:0/2310307550 shutdown_connections 2023-09-10T01:17:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.963+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/2310307550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c69cc0 0x55f059c75f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.963+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/2310307550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c6f1a0 0x55f059c6f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.963+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/2310307550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c69300 0x55f059c69780 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.963+0000 7f7fc8913b80 1 -- 172.21.15.44:0/2310307550 >> 172.21.15.44:0/2310307550 conn(0x55f059bde030 msgr2=0x55f059c64560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.963+0000 7f7fc8913b80 1 -- 172.21.15.44:0/2310307550 shutdown_connections 2023-09-10T01:17:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.963+0000 7f7fc8913b80 1 -- 172.21.15.44:0/2310307550 wait complete. 2023-09-10T01:17:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.963+0000 7f7fc8913b80 1 -- start start 2023-09-10T01:17:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.963+0000 7f7fc8913b80 1 Processor -- start 2023-09-10T01:17:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fc8913b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c69300 0x55f059c75e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fc8913b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69cc0 0x55f059c730e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fc8913b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c6f1a0 0x55f059c736f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fc8913b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f059b59b10 con 0x55f059c69cc0 2023-09-10T01:17:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fc8913b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f059b59810 con 0x55f059c69300 2023-09-10T01:17:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fc8913b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f059a3d8c0 con 0x55f059c6f1a0 2023-09-10T01:17:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fc8913b80 10 client.0 _ll_register_callbacks cb 0x55f059aa3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:17:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fb9ce8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69cc0 0x55f059c730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fba4e9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c69300 0x55f059c75e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fb9ce8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69cc0 0x55f059c730e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58008/0 (socket says 172.21.15.44:58008) 2023-09-10T01:17:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fb9ce8700 1 -- 172.21.15.44:0/854978928 learned_addr learned my addr 172.21.15.44:0/854978928 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fbacea700 1 --2- 172.21.15.44:0/854978928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c6f1a0 0x55f059c736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fba4e9700 1 -- 172.21.15.44:0/854978928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c6f1a0 msgr2=0x55f059c736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fba4e9700 1 --2- 172.21.15.44:0/854978928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c6f1a0 0x55f059c736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fba4e9700 1 -- 172.21.15.44:0/854978928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69cc0 msgr2=0x55f059c730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fba4e9700 1 --2- 172.21.15.44:0/854978928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69cc0 0x55f059c730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fba4e9700 1 -- 172.21.15.44:0/854978928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb410dd30 con 0x55f059c69300 2023-09-10T01:17:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fb9ce8700 1 --2- 172.21.15.44:0/854978928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69cc0 0x55f059c730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:17:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7fba4e9700 1 --2- 172.21.15.44:0/854978928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c69300 0x55f059c75e60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7fb001d6b0 tx=0x7f7fb00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89b4b36ac60db514 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.964+0000 7f7f9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:17:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.965+0000 7f7f9dffb700 1 -- 172.21.15.44:0/854978928 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0004a30 con 0x55f059c69300 2023-09-10T01:17:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.965+0000 7f7f9dffb700 1 -- 172.21.15.44:0/854978928 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(160..160 src has 1..160) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7fb000ee70 con 0x55f059c69300 2023-09-10T01:17:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.965+0000 7f7fc8913b80 1 -- 172.21.15.44:0/854978928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f059b13490 con 0x55f059c69300 2023-09-10T01:17:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.965+0000 7f7f9dffb700 1 -- 172.21.15.44:0/854978928 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fb0004b90 con 0x55f059c69300 2023-09-10T01:17:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.965+0000 7f7fc8913b80 10 client.29297 fetch_fsmap learned FSMap version 123 2023-09-10T01:17:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.965+0000 7f7fc8913b80 1 -- 172.21.15.44:0/854978928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=4+}) v3 -- 0x55f0599d0c00 con 0x55f059c69300 2023-09-10T01:17:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.965+0000 7f7f9dffb700 1 -- 172.21.15.44:0/854978928 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f059b13490 con 0x55f059c69300 2023-09-10T01:17:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.965+0000 7f7f9dffb700 1 -- 172.21.15.44:0/854978928 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 123) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f7fb0018070 con 0x55f059c69300 2023-09-10T01:17:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.965+0000 7f7fc8913b80 10 client.29297 fetch_fsmap finished waiting for FSMap version 123 2023-09-10T01:17:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.965+0000 7f7fc8913b80 10 client.29297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:17:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.965+0000 7f7fc8913b80 20 client.29297 populate_metadata read hostname 'smithi044' 2023-09-10T01:17:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.965+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/854978928 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f059c83530 0x55f059c85950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.965+0000 7f7fc8913b80 4 client.29297 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:17:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.965+0000 7f7fc8913b80 1 -- 172.21.15.44:0/854978928 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f059bd0840 con 0x55f059c83530 2023-09-10T01:17:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.966+0000 7f7fb9ce8700 1 --2- 172.21.15.44:0/854978928 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f059c83530 0x55f059c85950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.967+0000 7f7fb9ce8700 1 --2- 172.21.15.44:0/854978928 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f059c83530 0x55f059c85950 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecb18bd745e53612 server_cookie=1db17cb2682053c in_seq=0 out_seq=0 2023-09-10T01:17:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.967+0000 7f7f9dffb700 10 client.29297 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.968+0000 7f7f9dffb700 1 -- 172.21.15.44:0/854978928 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f059bd0840 con 0x55f059c83530 2023-09-10T01:17:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.968+0000 7f7f9dffb700 10 client.29297 handle_command_reply: tid=0 2023-09-10T01:17:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.968+0000 7f7fc8913b80 1 client.29297 shutdown 2023-09-10T01:17:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.968+0000 7f7fc8913b80 1 -- 172.21.15.44:0/854978928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c69300 msgr2=0x55f059c75e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.968+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/854978928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c69300 0x55f059c75e60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7fb001d6b0 tx=0x7f7fb00047c0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.968+0000 7f7fc8913b80 1 -- 172.21.15.44:0/854978928 shutdown_connections 2023-09-10T01:17:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.968+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/854978928 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f059c83530 0x55f059c85950 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.968+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/854978928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c6f1a0 0x55f059c736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.968+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/854978928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c69300 0x55f059c75e60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.968+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/854978928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69cc0 0x55f059c730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.968+0000 7f7fc8913b80 1 -- 172.21.15.44:0/854978928 >> 172.21.15.44:0/854978928 conn(0x55f059bde030 msgr2=0x55f059c637a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.968+0000 7f7f9dffb700 0 client.29297 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.968+0000 7f7fc8913b80 1 -- 172.21.15.44:0/854978928 shutdown_connections 2023-09-10T01:17:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.969+0000 7f7fc8913b80 1 -- 172.21.15.44:0/854978928 wait complete. 2023-09-10T01:17:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.969+0000 7f7fc8913b80 20 client.29297 trim_cache size 0 max 16384 2023-09-10T01:17:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.987+0000 7f7fc8913b80 1 Processor -- start 2023-09-10T01:17:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.987+0000 7f7fc8913b80 1 -- start start 2023-09-10T01:17:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.987+0000 7f7fc8913b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69300 0x55f059c7a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.987+0000 7f7fc8913b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c69cc0 0x55f059c75990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.987+0000 7f7fc8913b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c6f1a0 0x55f059c6bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.987+0000 7f7fc8913b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f059c65930 con 0x55f059c69300 2023-09-10T01:17:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.987+0000 7f7fc8913b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f059c74540 con 0x55f059c6f1a0 2023-09-10T01:17:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.987+0000 7f7fc8913b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f059c7aa60 con 0x55f059c69cc0 2023-09-10T01:17:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.987+0000 7f7fb9ce8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c69cc0 0x55f059c75990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.987+0000 7f7fba4e9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69300 0x55f059c7a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.987+0000 7f7fb9ce8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c69cc0 0x55f059c75990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36666/0 (socket says 172.21.15.44:36666) 2023-09-10T01:17:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.987+0000 7f7fb9ce8700 1 -- 172.21.15.44:0/3828812173 learned_addr learned my addr 172.21.15.44:0/3828812173 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.987+0000 7f7fbacea700 1 --2- 172.21.15.44:0/3828812173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c6f1a0 0x55f059c6bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.987+0000 7f7fb9ce8700 1 -- 172.21.15.44:0/3828812173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c6f1a0 msgr2=0x55f059c6bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.987+0000 7f7fb9ce8700 1 --2- 172.21.15.44:0/3828812173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c6f1a0 0x55f059c6bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.988+0000 7f7fb9ce8700 1 -- 172.21.15.44:0/3828812173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69300 msgr2=0x55f059c7a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.988+0000 7f7fb9ce8700 1 --2- 172.21.15.44:0/3828812173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69300 0x55f059c7a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.988+0000 7f7fb9ce8700 1 -- 172.21.15.44:0/3828812173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f059a0e1f0 con 0x55f059c69cc0 2023-09-10T01:17:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.988+0000 7f7fb9ce8700 1 --2- 172.21.15.44:0/3828812173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c69cc0 0x55f059c75990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009990 tx=0x7f7fa4005900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4cd8cb7f270a26e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.988+0000 7f7fa97fa700 1 -- 172.21.15.44:0/3828812173 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa401d040 con 0x55f059c69cc0 2023-09-10T01:17:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.988+0000 7f7fa97fa700 1 -- 172.21.15.44:0/3828812173 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7fa401e040 con 0x55f059c69cc0 2023-09-10T01:17:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.988+0000 7f7fa97fa700 1 -- 172.21.15.44:0/3828812173 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4004d20 con 0x55f059c69cc0 2023-09-10T01:17:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.989+0000 7f7fc8913b80 1 -- 172.21.15.44:0/3828812173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c69cc0 msgr2=0x55f059c75990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.989+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/3828812173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c69cc0 0x55f059c75990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009990 tx=0x7f7fa4005900 comp rx=0 tx=0).stop 2023-09-10T01:17:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.989+0000 7f7fc8913b80 1 -- 172.21.15.44:0/3828812173 shutdown_connections 2023-09-10T01:17:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.989+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/3828812173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c6f1a0 0x55f059c6bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.989+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/3828812173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69300 0x55f059c7a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.989+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/3828812173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c69cc0 0x55f059c75990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.989+0000 7f7fc8913b80 1 -- 172.21.15.44:0/3828812173 >> 172.21.15.44:0/3828812173 conn(0x55f059bde030 msgr2=0x55f059c6e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.989+0000 7f7fc8913b80 1 -- 172.21.15.44:0/3828812173 shutdown_connections 2023-09-10T01:17:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.989+0000 7f7fc8913b80 1 -- 172.21.15.44:0/3828812173 wait complete. 2023-09-10T01:17:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.989+0000 7f7fc8913b80 1 -- start start 2023-09-10T01:17:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.989+0000 7f7fc8913b80 1 Processor -- start 2023-09-10T01:17:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fc8913b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c69300 0x55f059c71890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fc8913b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69cc0 0x55f059c71dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fc8913b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c6f1a0 0x55f059c72310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fc8913b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f059c729e0 con 0x55f059c69cc0 2023-09-10T01:17:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fc8913b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f059c72b50 con 0x55f059c69300 2023-09-10T01:17:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fc8913b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f059c72d10 con 0x55f059c6f1a0 2023-09-10T01:17:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fc8913b80 10 client.0 _ll_register_callbacks cb 0x7f7fb000ecd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:17:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fb9ce8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69cc0 0x55f059c71dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fb9ce8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69cc0 0x55f059c71dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58022/0 (socket says 172.21.15.44:58022) 2023-09-10T01:17:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fb9ce8700 1 -- 172.21.15.44:0/723947528 learned_addr learned my addr 172.21.15.44:0/723947528 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fbacea700 1 --2- 172.21.15.44:0/723947528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c6f1a0 0x55f059c72310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fba4e9700 1 --2- 172.21.15.44:0/723947528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c69300 0x55f059c71890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fb9ce8700 1 -- 172.21.15.44:0/723947528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c6f1a0 msgr2=0x55f059c72310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fb9ce8700 1 --2- 172.21.15.44:0/723947528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c6f1a0 0x55f059c72310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fb9ce8700 1 -- 172.21.15.44:0/723947528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c69300 msgr2=0x55f059c71890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fb9ce8700 1 --2- 172.21.15.44:0/723947528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c69300 0x55f059c71890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fb9ce8700 1 -- 172.21.15.44:0/723947528 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0599e7d80 con 0x55f059c69cc0 2023-09-10T01:17:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fbacea700 1 --2- 172.21.15.44:0/723947528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c6f1a0 0x55f059c72310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:17:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fb9ce8700 1 --2- 172.21.15.44:0/723947528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69cc0 0x55f059c71dd0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x55f059c80b80 tx=0x7f7fa400b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a65e48185a9d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7f9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:17:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7f9dffb700 1 -- 172.21.15.44:0/723947528 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa401d040 con 0x55f059c69cc0 2023-09-10T01:17:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7fc8913b80 1 -- 172.21.15.44:0/723947528 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fa40044d0 con 0x55f059c69cc0 2023-09-10T01:17:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.990+0000 7f7f9dffb700 1 -- 172.21.15.44:0/723947528 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(160..160 src has 1..160) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7fa4005450 con 0x55f059c69cc0 2023-09-10T01:17:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.991+0000 7f7f9dffb700 1 -- 172.21.15.44:0/723947528 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fa4005610 con 0x55f059c69cc0 2023-09-10T01:17:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.991+0000 7f7fc8913b80 10 client.19671 fetch_fsmap learned FSMap version 123 2023-09-10T01:17:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.991+0000 7f7fc8913b80 1 -- 172.21.15.44:0/723947528 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=4+}) v3 -- 0x55f0599e6250 con 0x55f059c69cc0 2023-09-10T01:17:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.991+0000 7f7f9dffb700 1 -- 172.21.15.44:0/723947528 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7fa40044d0 con 0x55f059c69cc0 2023-09-10T01:17:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.991+0000 7f7f9dffb700 1 -- 172.21.15.44:0/723947528 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 123) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f7fa4042070 con 0x55f059c69cc0 2023-09-10T01:17:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.991+0000 7f7fc8913b80 10 client.19671 fetch_fsmap finished waiting for FSMap version 123 2023-09-10T01:17:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.991+0000 7f7fc8913b80 10 client.19671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:17:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.991+0000 7f7fc8913b80 20 client.19671 populate_metadata read hostname 'smithi044' 2023-09-10T01:17:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.991+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/723947528 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f059ce0410 0x55f059ce2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.991+0000 7f7fc8913b80 4 client.19671 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:17:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.991+0000 7f7fc8913b80 1 -- 172.21.15.44:0/723947528 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f059bb48c0 con 0x55f059ce0410 2023-09-10T01:17:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.991+0000 7f7fba4e9700 1 --2- 172.21.15.44:0/723947528 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f059ce0410 0x55f059ce2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.992+0000 7f7fba4e9700 1 --2- 172.21.15.44:0/723947528 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f059ce0410 0x55f059ce2830 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c175535cf5b0b24 server_cookie=2c352b01fc612539 in_seq=0 out_seq=0 2023-09-10T01:17:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:42.992+0000 7f7f9dffb700 10 client.19671 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.003+0000 7f7f9dffb700 1 -- 172.21.15.44:0/723947528 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f059bb48c0 con 0x55f059ce0410 2023-09-10T01:17:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.003+0000 7f7f9dffb700 10 client.19671 handle_command_reply: tid=0 2023-09-10T01:17:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.003+0000 7f7fc8913b80 1 client.19671 shutdown 2023-09-10T01:17:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.003+0000 7f7fc8913b80 1 -- 172.21.15.44:0/723947528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69cc0 msgr2=0x55f059c71dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.003+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/723947528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69cc0 0x55f059c71dd0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x55f059c80b80 tx=0x7f7fa400b5c0 comp rx=0 tx=0).stop 2023-09-10T01:17:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.003+0000 7f7fc8913b80 1 -- 172.21.15.44:0/723947528 shutdown_connections 2023-09-10T01:17:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.003+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/723947528 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f059ce0410 0x55f059ce2830 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.003+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/723947528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f059c6f1a0 0x55f059c72310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.003+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/723947528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f059c69300 0x55f059c71890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.003+0000 7f7f9dffb700 0 client.19671 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.003+0000 7f7fc8913b80 1 --2- 172.21.15.44:0/723947528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f059c69cc0 0x55f059c71dd0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.003+0000 7f7fc8913b80 1 -- 172.21.15.44:0/723947528 >> 172.21.15.44:0/723947528 conn(0x55f059bde030 msgr2=0x55f059c6e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.004+0000 7f7fc8913b80 1 -- 172.21.15.44:0/723947528 shutdown_connections 2023-09-10T01:17:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.004+0000 7f7fc8913b80 1 -- 172.21.15.44:0/723947528 wait complete. 2023-09-10T01:17:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.004+0000 7f7fc8913b80 20 client.19671 trim_cache size 0 max 16384 2023-09-10T01:17:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:17:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:17:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:17:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:17:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.004+0000 7f7fbc74d700 1 -- 172.21.15.44:0/4293333983 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7fa00a0830 msgr2=0x7f7fa00a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.004+0000 7f7fbc74d700 1 --2- 172.21.15.44:0/4293333983 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7fa00a0830 0x7f7fa00a2cf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009af0 tx=0x7f7fb0009260 comp rx=0 tx=0).stop 2023-09-10T01:17:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.004+0000 7f7fbc74d700 1 -- 172.21.15.44:0/4293333983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fb4106670 msgr2=0x7f7fb41a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.004+0000 7f7fbc74d700 1 --2- 172.21.15.44:0/4293333983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fb4106670 0x7f7fb41a4710 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00ef30 tx=0x7f7fac00c450 comp rx=0 tx=0).stop 2023-09-10T01:17:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.004+0000 7f7fbc74d700 1 -- 172.21.15.44:0/4293333983 shutdown_connections 2023-09-10T01:17:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.004+0000 7f7fbc74d700 1 --2- 172.21.15.44:0/4293333983 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7fa00a0830 0x7f7fa00a2cf0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.004+0000 7f7fbc74d700 1 --2- 172.21.15.44:0/4293333983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fb4106670 0x7f7fb41a4710 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.004+0000 7f7fbc74d700 1 --2- 172.21.15.44:0/4293333983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fb41005b0 0x7f7fb4072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.004+0000 7f7fbc74d700 1 --2- 172.21.15.44:0/4293333983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb4100f70 0x7f7fb4073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.004+0000 7f7fbc74d700 1 -- 172.21.15.44:0/4293333983 >> 172.21.15.44:0/4293333983 conn(0x7f7fb40fc060 msgr2=0x7f7fb40fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.005+0000 7f7fbc74d700 1 -- 172.21.15.44:0/4293333983 shutdown_connections 2023-09-10T01:17:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:43.005+0000 7f7fbc74d700 1 -- 172.21.15.44:0/4293333983 wait complete. 2023-09-10T01:17:44.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:44 smithi044 ceph-mon[98676]: pgmap v1190: 129 pgs: 129 active+clean; 12 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 37 MiB/s wr, 35 op/s 2023-09-10T01:17:44.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:44 smithi099 ceph-mon[102692]: pgmap v1190: 129 pgs: 129 active+clean; 12 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 37 MiB/s wr, 35 op/s 2023-09-10T01:17:44.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:44 smithi131 ceph-mon[103713]: pgmap v1190: 129 pgs: 129 active+clean; 12 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 37 MiB/s wr, 35 op/s 2023-09-10T01:17:46.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:46 smithi044 ceph-mon[98676]: pgmap v1191: 129 pgs: 129 active+clean; 12 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 37 MiB/s wr, 35 op/s 2023-09-10T01:17:46.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:46 smithi099 ceph-mon[102692]: pgmap v1191: 129 pgs: 129 active+clean; 12 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 37 MiB/s wr, 35 op/s 2023-09-10T01:17:46.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:46 smithi131 ceph-mon[103713]: pgmap v1191: 129 pgs: 129 active+clean; 12 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 37 MiB/s wr, 35 op/s 2023-09-10T01:17:48.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:48 smithi099 ceph-mon[102692]: pgmap v1192: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 56 MiB/s wr, 55 op/s 2023-09-10T01:17:48.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:48 smithi131 ceph-mon[103713]: pgmap v1192: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 56 MiB/s wr, 55 op/s 2023-09-10T01:17:48.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:48 smithi044 ceph-mon[98676]: pgmap v1192: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 56 MiB/s wr, 55 op/s 2023-09-10T01:17:50.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:50 smithi099 ceph-mon[102692]: pgmap v1193: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 37 MiB/s wr, 37 op/s 2023-09-10T01:17:50.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:50 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:17:50.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:50 smithi131 ceph-mon[103713]: pgmap v1193: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 37 MiB/s wr, 37 op/s 2023-09-10T01:17:50.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:50 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:17:50.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:50 smithi044 ceph-mon[98676]: pgmap v1193: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 37 MiB/s wr, 37 op/s 2023-09-10T01:17:50.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:50 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:17:52.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:52 smithi099 ceph-mon[102692]: pgmap v1194: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 56 MiB/s wr, 55 op/s 2023-09-10T01:17:52.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:52 smithi131 ceph-mon[103713]: pgmap v1194: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 56 MiB/s wr, 55 op/s 2023-09-10T01:17:52.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:52 smithi044 ceph-mon[98676]: pgmap v1194: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 56 MiB/s wr, 55 op/s 2023-09-10T01:17:54.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:54 smithi099 ceph-mon[102692]: pgmap v1195: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 37 MiB/s wr, 37 op/s 2023-09-10T01:17:54.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:54 smithi131 ceph-mon[103713]: pgmap v1195: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 37 MiB/s wr, 37 op/s 2023-09-10T01:17:54.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:54 smithi044 ceph-mon[98676]: pgmap v1195: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 37 MiB/s wr, 37 op/s 2023-09-10T01:17:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:55 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:17:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:55 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:17:55.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:55 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:17:55.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:55 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:17:55.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:55 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:17:55.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:55 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:17:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.145+0000 7f2f88afb700 1 -- 172.21.15.44:0/4103297638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f840679f0 msgr2=0x7f2f84067e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.145+0000 7f2f88afb700 1 --2- 172.21.15.44:0/4103297638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f840679f0 0x7f2f84067e70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f2f780099c0 tx=0x7f2f78009cd0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.145+0000 7f2f88afb700 1 -- 172.21.15.44:0/4103297638 shutdown_connections 2023-09-10T01:17:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.145+0000 7f2f88afb700 1 --2- 172.21.15.44:0/4103297638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f840683b0 0x7f2f8410d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.145+0000 7f2f88afb700 1 --2- 172.21.15.44:0/4103297638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f84106650 0x7f2f84106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.145+0000 7f2f88afb700 1 --2- 172.21.15.44:0/4103297638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f840679f0 0x7f2f84067e70 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.145+0000 7f2f88afb700 1 -- 172.21.15.44:0/4103297638 >> 172.21.15.44:0/4103297638 conn(0x7f2f84074f30 msgr2=0x7f2f84075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.145+0000 7f2f88afb700 1 -- 172.21.15.44:0/4103297638 shutdown_connections 2023-09-10T01:17:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.145+0000 7f2f88afb700 1 -- 172.21.15.44:0/4103297638 wait complete. 2023-09-10T01:17:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.146+0000 7f2f88afb700 1 Processor -- start 2023-09-10T01:17:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.146+0000 7f2f88afb700 1 -- start start 2023-09-10T01:17:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.146+0000 7f2f88afb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f840679f0 0x7f2f840ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.146+0000 7f2f88afb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f840683b0 0x7f2f840fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.146+0000 7f2f88afb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f84106650 0x7f2f841a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.146+0000 7f2f88afb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f841a4ce0 con 0x7f2f840679f0 2023-09-10T01:17:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.146+0000 7f2f88afb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f841a4e20 con 0x7f2f840683b0 2023-09-10T01:17:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.146+0000 7f2f88afb700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f841a4f90 con 0x7f2f84106650 2023-09-10T01:17:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.147+0000 7f2f8259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f840679f0 0x7f2f840ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.147+0000 7f2f81d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f840683b0 0x7f2f840fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.147+0000 7f2f8259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f840679f0 0x7f2f840ffa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58034/0 (socket says 172.21.15.44:58034) 2023-09-10T01:17:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.147+0000 7f2f8259c700 1 -- 172.21.15.44:0/3245814750 learned_addr learned my addr 172.21.15.44:0/3245814750 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.147+0000 7f2f82d9d700 1 --2- 172.21.15.44:0/3245814750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f84106650 0x7f2f841a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.147+0000 7f2f81d9b700 1 -- 172.21.15.44:0/3245814750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f84106650 msgr2=0x7f2f841a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.147+0000 7f2f81d9b700 1 --2- 172.21.15.44:0/3245814750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f84106650 0x7f2f841a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.147+0000 7f2f81d9b700 1 -- 172.21.15.44:0/3245814750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f840679f0 msgr2=0x7f2f840ffa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.147+0000 7f2f81d9b700 1 --2- 172.21.15.44:0/3245814750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f840679f0 0x7f2f840ffa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.147+0000 7f2f81d9b700 1 -- 172.21.15.44:0/3245814750 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c009c40 con 0x7f2f840683b0 2023-09-10T01:17:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.147+0000 7f2f8259c700 1 --2- 172.21.15.44:0/3245814750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f840679f0 0x7f2f840ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:17:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.147+0000 7f2f81d9b700 1 --2- 172.21.15.44:0/3245814750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f840683b0 0x7f2f840fff50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009fd0 tx=0x7f2f78009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.147+0000 7f2f737fe700 1 -- 172.21.15.44:0/3245814750 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7801d070 con 0x7f2f840683b0 2023-09-10T01:17:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.147+0000 7f2f88afb700 1 -- 172.21.15.44:0/3245814750 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f74009520 con 0x7f2f840683b0 2023-09-10T01:17:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.148+0000 7f2f88afb700 1 -- 172.21.15.44:0/3245814750 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f780095f0 con 0x7f2f840683b0 2023-09-10T01:17:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.148+0000 7f2f737fe700 1 -- 172.21.15.44:0/3245814750 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f780059a0 con 0x7f2f840683b0 2023-09-10T01:17:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.148+0000 7f2f737fe700 1 -- 172.21.15.44:0/3245814750 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78022cd0 con 0x7f2f840683b0 2023-09-10T01:17:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.150+0000 7f2f737fe700 1 -- 172.21.15.44:0/3245814750 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f2f78022470 con 0x7f2f840683b0 2023-09-10T01:17:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.151+0000 7f2f737fe700 1 --2- 172.21.15.44:0/3245814750 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2f6c0a07b0 0x7f2f6c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.151+0000 7f2f737fe700 1 -- 172.21.15.44:0/3245814750 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f2f780cd130 con 0x7f2f840683b0 2023-09-10T01:17:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.151+0000 7f2f8259c700 1 --2- 172.21.15.44:0/3245814750 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2f6c0a07b0 0x7f2f6c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.151+0000 7f2f8259c700 1 --2- 172.21.15.44:0/3245814750 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2f6c0a07b0 0x7f2f6c0a2c70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2f740116b0 tx=0x7f2f74009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f94cc1b80 1 Processor -- start 2023-09-10T01:17:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f94cc1b80 1 -- start start 2023-09-10T01:17:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f94cc1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac685421a0 0x55ac68542580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f94cc1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac6853c300 0x55ac6853c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f94cc1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853ccc0 0x55ac68548f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f94cc1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac684956d0 con 0x55ac6853ccc0 2023-09-10T01:17:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f94cc1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac68492ff0 con 0x55ac685421a0 2023-09-10T01:17:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f94cc1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6848e8e0 con 0x55ac6853c300 2023-09-10T01:17:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f82d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853ccc0 0x55ac68548f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f82d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853ccc0 0x55ac68548f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58046/0 (socket says 172.21.15.44:58046) 2023-09-10T01:17:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f82d9d700 1 -- 172.21.15.44:0/998834954 learned_addr learned my addr 172.21.15.44:0/998834954 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f8259c700 1 --2- 172.21.15.44:0/998834954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac685421a0 0x55ac68542580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f81d9b700 1 --2- 172.21.15.44:0/998834954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac6853c300 0x55ac6853c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f81d9b700 1 -- 172.21.15.44:0/998834954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac685421a0 msgr2=0x55ac68542580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f81d9b700 1 --2- 172.21.15.44:0/998834954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac685421a0 0x55ac68542580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f81d9b700 1 -- 172.21.15.44:0/998834954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853ccc0 msgr2=0x55ac68548f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f81d9b700 1 --2- 172.21.15.44:0/998834954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853ccc0 0x55ac68548f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.152+0000 7f2f81d9b700 1 -- 172.21.15.44:0/998834954 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac682bad80 con 0x55ac6853c300 2023-09-10T01:17:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.153+0000 7f2f82d9d700 1 --2- 172.21.15.44:0/998834954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853ccc0 0x55ac68548f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:17:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.153+0000 7f2f8259c700 1 --2- 172.21.15.44:0/998834954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac685421a0 0x55ac68542580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:17:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.153+0000 7f2f81d9b700 1 --2- 172.21.15.44:0/998834954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac6853c300 0x55ac6853c780 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2f780ca4e0 tx=0x7f2f78099e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd616ad2d7ebd81 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.153+0000 7f2f70ff9700 1 -- 172.21.15.44:0/998834954 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7809a2d0 con 0x55ac6853c300 2023-09-10T01:17:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.153+0000 7f2f70ff9700 1 -- 172.21.15.44:0/998834954 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f7809e100 con 0x55ac6853c300 2023-09-10T01:17:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.153+0000 7f2f70ff9700 1 -- 172.21.15.44:0/998834954 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78099a20 con 0x55ac6853c300 2023-09-10T01:17:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.154+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/998834954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac6853c300 msgr2=0x55ac6853c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.154+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/998834954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac6853c300 0x55ac6853c780 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2f780ca4e0 tx=0x7f2f78099e10 comp rx=0 tx=0).stop 2023-09-10T01:17:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.154+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/998834954 shutdown_connections 2023-09-10T01:17:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.154+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/998834954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853ccc0 0x55ac68548f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.154+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/998834954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac685421a0 0x55ac68542580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.154+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/998834954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac6853c300 0x55ac6853c780 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.154+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/998834954 >> 172.21.15.44:0/998834954 conn(0x55ac684b1030 msgr2=0x55ac68537560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.154+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/998834954 shutdown_connections 2023-09-10T01:17:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.154+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/998834954 wait complete. 2023-09-10T01:17:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.154+0000 7f2f94cc1b80 1 -- start start 2023-09-10T01:17:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.154+0000 7f2f94cc1b80 1 Processor -- start 2023-09-10T01:17:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f94cc1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853c300 0x55ac6854bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f94cc1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac6853ccc0 0x55ac6854c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f94cc1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 0x55ac68545d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f94cc1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6842cb10 con 0x55ac6853c300 2023-09-10T01:17:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f94cc1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6842c810 con 0x55ac6853ccc0 2023-09-10T01:17:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f94cc1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac683108c0 con 0x55ac685421a0 2023-09-10T01:17:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f94cc1b80 10 client.0 _ll_register_callbacks cb 0x55ac68376a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:17:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f8259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853c300 0x55ac6854bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f8259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853c300 0x55ac6854bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58048/0 (socket says 172.21.15.44:58048) 2023-09-10T01:17:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f8259c700 1 -- 172.21.15.44:0/985275350 learned_addr learned my addr 172.21.15.44:0/985275350 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f82d9d700 1 --2- 172.21.15.44:0/985275350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 0x55ac68545d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f81d9b700 1 --2- 172.21.15.44:0/985275350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac6853ccc0 0x55ac6854c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f8259c700 1 -- 172.21.15.44:0/985275350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 msgr2=0x55ac68545d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f8259c700 1 --2- 172.21.15.44:0/985275350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 0x55ac68545d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f8259c700 1 -- 172.21.15.44:0/985275350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac6853ccc0 msgr2=0x55ac6854c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f8259c700 1 --2- 172.21.15.44:0/985275350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac6853ccc0 0x55ac6854c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f8259c700 1 -- 172.21.15.44:0/985275350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7c00e410 con 0x55ac6853c300 2023-09-10T01:17:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f8259c700 1 --2- 172.21.15.44:0/985275350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853c300 0x55ac6854bb30 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400fc30 tx=0x7f2f740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6955c9d15bea7c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:17:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.155+0000 7f2f69ffb700 1 -- 172.21.15.44:0/985275350 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f74022070 con 0x55ac6853c300 2023-09-10T01:17:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.156+0000 7f2f69ffb700 1 -- 172.21.15.44:0/985275350 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(160..160 src has 1..160) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f2f7401d800 con 0x55ac6853c300 2023-09-10T01:17:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.156+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/985275350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac683e6490 con 0x55ac6853c300 2023-09-10T01:17:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.156+0000 7f2f69ffb700 1 -- 172.21.15.44:0/985275350 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f7401d9c0 con 0x55ac6853c300 2023-09-10T01:17:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.156+0000 7f2f94cc1b80 10 client.19683 fetch_fsmap learned FSMap version 125 2023-09-10T01:17:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.156+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/985275350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=4+}) v3 -- 0x7f2f74017050 con 0x55ac6853c300 2023-09-10T01:17:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.156+0000 7f2f69ffb700 1 -- 172.21.15.44:0/985275350 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ac683e6490 con 0x55ac6853c300 2023-09-10T01:17:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.156+0000 7f2f69ffb700 1 -- 172.21.15.44:0/985275350 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 125) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f2f74027cc0 con 0x55ac6853c300 2023-09-10T01:17:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.156+0000 7f2f94cc1b80 10 client.19683 fetch_fsmap finished waiting for FSMap version 125 2023-09-10T01:17:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.156+0000 7f2f94cc1b80 10 client.19683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:17:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.156+0000 7f2f94cc1b80 20 client.19683 populate_metadata read hostname 'smithi044' 2023-09-10T01:17:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.156+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/985275350 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ac68556110 0x55ac68558530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.156+0000 7f2f94cc1b80 4 client.19683 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:17:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.156+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/985275350 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac684a3840 con 0x55ac68556110 2023-09-10T01:17:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.156+0000 7f2f81d9b700 1 --2- 172.21.15.44:0/985275350 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ac68556110 0x55ac68558530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.157+0000 7f2f81d9b700 1 --2- 172.21.15.44:0/985275350 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ac68556110 0x55ac68558530 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d3c077b4b30147e server_cookie=7283c9800a367cb5 in_seq=0 out_seq=0 2023-09-10T01:17:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.157+0000 7f2f69ffb700 10 client.19683 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.157+0000 7f2f69ffb700 1 -- 172.21.15.44:0/985275350 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ac684a3840 con 0x55ac68556110 2023-09-10T01:17:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.157+0000 7f2f69ffb700 10 client.19683 handle_command_reply: tid=0 2023-09-10T01:17:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.157+0000 7f2f94cc1b80 1 client.19683 shutdown 2023-09-10T01:17:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.157+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/985275350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853c300 msgr2=0x55ac6854bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.157+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/985275350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853c300 0x55ac6854bb30 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400fc30 tx=0x7f2f740047c0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.158+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/985275350 shutdown_connections 2023-09-10T01:17:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.158+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/985275350 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ac68556110 0x55ac68558530 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.158+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/985275350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 0x55ac68545d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.158+0000 7f2f69ffb700 0 client.19683 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.158+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/985275350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853c300 0x55ac6854bb30 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.158+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/985275350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac6853ccc0 0x55ac6854c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.158+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/985275350 >> 172.21.15.44:0/985275350 conn(0x55ac684b1030 msgr2=0x55ac685367a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.158+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/985275350 shutdown_connections 2023-09-10T01:17:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.158+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/985275350 wait complete. 2023-09-10T01:17:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.158+0000 7f2f94cc1b80 20 client.19683 trim_cache size 0 max 16384 2023-09-10T01:17:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.175+0000 7f2f94cc1b80 1 Processor -- start 2023-09-10T01:17:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.175+0000 7f2f94cc1b80 1 -- start start 2023-09-10T01:17:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.175+0000 7f2f94cc1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853c300 0x55ac68541510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.175+0000 7f2f94cc1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac6853ccc0 0x55ac6854b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.175+0000 7f2f94cc1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 0x55ac6854bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.175+0000 7f2f94cc1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6853d580 con 0x55ac6853c300 2023-09-10T01:17:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.175+0000 7f2f94cc1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac68547360 con 0x55ac6853ccc0 2023-09-10T01:17:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.175+0000 7f2f94cc1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac68533fc0 con 0x55ac685421a0 2023-09-10T01:17:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.175+0000 7f2f8259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853c300 0x55ac68541510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.176+0000 7f2f8259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853c300 0x55ac68541510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58056/0 (socket says 172.21.15.44:58056) 2023-09-10T01:17:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.176+0000 7f2f8259c700 1 -- 172.21.15.44:0/1522779854 learned_addr learned my addr 172.21.15.44:0/1522779854 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.176+0000 7f2f82d9d700 1 --2- 172.21.15.44:0/1522779854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 0x55ac6854bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.176+0000 7f2f81d9b700 1 --2- 172.21.15.44:0/1522779854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac6853ccc0 0x55ac6854b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.176+0000 7f2f8259c700 1 -- 172.21.15.44:0/1522779854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 msgr2=0x55ac6854bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.176+0000 7f2f8259c700 1 --2- 172.21.15.44:0/1522779854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 0x55ac6854bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.176+0000 7f2f8259c700 1 -- 172.21.15.44:0/1522779854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac6853ccc0 msgr2=0x55ac6854b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.176+0000 7f2f8259c700 1 --2- 172.21.15.44:0/1522779854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac6853ccc0 0x55ac6854b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.176+0000 7f2f8259c700 1 -- 172.21.15.44:0/1522779854 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f780095f0 con 0x55ac6853c300 2023-09-10T01:17:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.176+0000 7f2f8259c700 1 --2- 172.21.15.44:0/1522779854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853c300 0x55ac68541510 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009350 tx=0x7f2f74009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7803718b741af82c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.176+0000 7f2f717fa700 1 -- 172.21.15.44:0/1522779854 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f74022070 con 0x55ac6853c300 2023-09-10T01:17:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.176+0000 7f2f717fa700 1 -- 172.21.15.44:0/1522779854 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f74026e40 con 0x55ac6853c300 2023-09-10T01:17:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.176+0000 7f2f717fa700 1 -- 172.21.15.44:0/1522779854 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7401dde0 con 0x55ac6853c300 2023-09-10T01:17:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.177+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/1522779854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853c300 msgr2=0x55ac68541510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.177+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/1522779854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853c300 0x55ac68541510 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009350 tx=0x7f2f74009380 comp rx=0 tx=0).stop 2023-09-10T01:17:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.177+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/1522779854 shutdown_connections 2023-09-10T01:17:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.177+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/1522779854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 0x55ac6854bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.177+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/1522779854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853c300 0x55ac68541510 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.177+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/1522779854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac6853ccc0 0x55ac6854b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.177+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/1522779854 >> 172.21.15.44:0/1522779854 conn(0x55ac684b1030 msgr2=0x55ac6853bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.177+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/1522779854 shutdown_connections 2023-09-10T01:17:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.177+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/1522779854 wait complete. 2023-09-10T01:17:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.178+0000 7f2f94cc1b80 1 -- start start 2023-09-10T01:17:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.178+0000 7f2f94cc1b80 1 Processor -- start 2023-09-10T01:17:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.178+0000 7f2f94cc1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac6853c300 0x55ac685a3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.178+0000 7f2f94cc1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853ccc0 0x55ac685a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.178+0000 7f2f94cc1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 0x55ac6859dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.178+0000 7f2f94cc1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6859e1a0 con 0x55ac6853ccc0 2023-09-10T01:17:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.178+0000 7f2f94cc1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6859e310 con 0x55ac6853c300 2023-09-10T01:17:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.178+0000 7f2f94cc1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6859e4d0 con 0x55ac685421a0 2023-09-10T01:17:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.178+0000 7f2f94cc1b80 10 client.0 _ll_register_callbacks cb 0x55ac68376a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:17:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.178+0000 7f2f81d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853ccc0 0x55ac685a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.178+0000 7f2f82d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 0x55ac6859dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.178+0000 7f2f82d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 0x55ac6859dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36716/0 (socket says 172.21.15.44:36716) 2023-09-10T01:17:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.178+0000 7f2f81d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853ccc0 0x55ac685a3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58064/0 (socket says 172.21.15.44:58064) 2023-09-10T01:17:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.178+0000 7f2f82d9d700 1 -- 172.21.15.44:0/1889132408 learned_addr learned my addr 172.21.15.44:0/1889132408 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:17:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.178+0000 7f2f8259c700 1 --2- 172.21.15.44:0/1889132408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac6853c300 0x55ac685a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.179+0000 7f2f82d9d700 1 -- 172.21.15.44:0/1889132408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac6853c300 msgr2=0x55ac685a3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.179+0000 7f2f82d9d700 1 --2- 172.21.15.44:0/1889132408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac6853c300 0x55ac685a3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.179+0000 7f2f82d9d700 1 -- 172.21.15.44:0/1889132408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853ccc0 msgr2=0x55ac685a3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:17:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.179+0000 7f2f82d9d700 1 --2- 172.21.15.44:0/1889132408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853ccc0 0x55ac685a3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.179+0000 7f2f82d9d700 1 -- 172.21.15.44:0/1889132408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f74017050 con 0x55ac685421a0 2023-09-10T01:17:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.179+0000 7f2f82d9d700 1 --2- 172.21.15.44:0/1889132408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 0x55ac6859dbc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00efa0 tx=0x7f2f7c00efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30847e02d2c8aaef server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:17:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.179+0000 7f2f69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:17:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.179+0000 7f2f69ffb700 1 -- 172.21.15.44:0/1889132408 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c00e040 con 0x55ac685421a0 2023-09-10T01:17:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.179+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/1889132408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f7809e100 con 0x55ac685421a0 2023-09-10T01:17:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.179+0000 7f2f69ffb700 1 -- 172.21.15.44:0/1889132408 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(160..160 src has 1..160) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f2f7c021630 con 0x55ac685421a0 2023-09-10T01:17:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.179+0000 7f2f69ffb700 1 -- 172.21.15.44:0/1889132408 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f7c0093e0 con 0x55ac685421a0 2023-09-10T01:17:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.179+0000 7f2f94cc1b80 10 client.29290 fetch_fsmap learned FSMap version 125 2023-09-10T01:17:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.179+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/1889132408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=4+}) v3 -- 0x55ac682a3c00 con 0x55ac685421a0 2023-09-10T01:17:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.179+0000 7f2f69ffb700 1 -- 172.21.15.44:0/1889132408 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f7809e100 con 0x55ac685421a0 2023-09-10T01:17:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.180+0000 7f2f69ffb700 1 -- 172.21.15.44:0/1889132408 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 125) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f2f7c017840 con 0x55ac685421a0 2023-09-10T01:17:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.180+0000 7f2f94cc1b80 10 client.29290 fetch_fsmap finished waiting for FSMap version 125 2023-09-10T01:17:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.180+0000 7f2f94cc1b80 10 client.29290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:17:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.180+0000 7f2f94cc1b80 20 client.29290 populate_metadata read hostname 'smithi044' 2023-09-10T01:17:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.180+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/1889132408 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ac685ab8a0 0x55ac685adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:17:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.180+0000 7f2f94cc1b80 4 client.29290 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:17:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.180+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/1889132408 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ac684878c0 con 0x55ac685ab8a0 2023-09-10T01:17:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.180+0000 7f2f8259c700 1 --2- 172.21.15.44:0/1889132408 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ac685ab8a0 0x55ac685adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:17:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.180+0000 7f2f8259c700 1 --2- 172.21.15.44:0/1889132408 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ac685ab8a0 0x55ac685adcc0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcfed10c86a4a22a server_cookie=6fea27cb2c0e1cb8 in_seq=0 out_seq=0 2023-09-10T01:17:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.180+0000 7f2f69ffb700 10 client.29290 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.193+0000 7f2f69ffb700 1 -- 172.21.15.44:0/1889132408 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ac684878c0 con 0x55ac685ab8a0 2023-09-10T01:17:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.193+0000 7f2f69ffb700 10 client.29290 handle_command_reply: tid=0 2023-09-10T01:17:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.193+0000 7f2f94cc1b80 1 client.29290 shutdown 2023-09-10T01:17:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.193+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/1889132408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 msgr2=0x55ac6859dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.194+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/1889132408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 0x55ac6859dbc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00efa0 tx=0x7f2f7c00efd0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.194+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/1889132408 shutdown_connections 2023-09-10T01:17:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.194+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/1889132408 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ac685ab8a0 0x55ac685adcc0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.194+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/1889132408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac685421a0 0x55ac6859dbc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.194+0000 7f2f69ffb700 0 client.29290 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:17:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.194+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/1889132408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac6853c300 0x55ac685a3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.194+0000 7f2f94cc1b80 1 --2- 172.21.15.44:0/1889132408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ac6853ccc0 0x55ac685a3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.194+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/1889132408 >> 172.21.15.44:0/1889132408 conn(0x55ac684b1030 msgr2=0x55ac6853bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.194+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/1889132408 shutdown_connections 2023-09-10T01:17:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.194+0000 7f2f94cc1b80 1 -- 172.21.15.44:0/1889132408 wait complete. 2023-09-10T01:17:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.194+0000 7f2f94cc1b80 20 client.29290 trim_cache size 0 max 16384 2023-09-10T01:17:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:17:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:17:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:17:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:17:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.196+0000 7f2f88afb700 1 -- 172.21.15.44:0/3245814750 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2f6c0a07b0 msgr2=0x7f2f6c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.196+0000 7f2f88afb700 1 --2- 172.21.15.44:0/3245814750 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2f6c0a07b0 0x7f2f6c0a2c70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2f740116b0 tx=0x7f2f74009260 comp rx=0 tx=0).stop 2023-09-10T01:17:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.196+0000 7f2f88afb700 1 -- 172.21.15.44:0/3245814750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f840683b0 msgr2=0x7f2f840fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:17:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.196+0000 7f2f88afb700 1 --2- 172.21.15.44:0/3245814750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f840683b0 0x7f2f840fff50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009fd0 tx=0x7f2f78009580 comp rx=0 tx=0).stop 2023-09-10T01:17:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.196+0000 7f2f88afb700 1 -- 172.21.15.44:0/3245814750 shutdown_connections 2023-09-10T01:17:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.196+0000 7f2f88afb700 1 --2- 172.21.15.44:0/3245814750 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2f6c0a07b0 0x7f2f6c0a2c70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.196+0000 7f2f88afb700 1 --2- 172.21.15.44:0/3245814750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f84106650 0x7f2f841a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.196+0000 7f2f88afb700 1 --2- 172.21.15.44:0/3245814750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f840679f0 0x7f2f840ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.196+0000 7f2f88afb700 1 --2- 172.21.15.44:0/3245814750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f840683b0 0x7f2f840fff50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:17:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.196+0000 7f2f88afb700 1 -- 172.21.15.44:0/3245814750 >> 172.21.15.44:0/3245814750 conn(0x7f2f84074f30 msgr2=0x7f2f840fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:17:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.196+0000 7f2f88afb700 1 -- 172.21.15.44:0/3245814750 shutdown_connections 2023-09-10T01:17:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:17:56.197+0000 7f2f88afb700 1 -- 172.21.15.44:0/3245814750 wait complete. 2023-09-10T01:17:56.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:56 smithi099 ceph-mon[102692]: pgmap v1196: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 37 MiB/s wr, 37 op/s 2023-09-10T01:17:56.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:56 smithi131 ceph-mon[103713]: pgmap v1196: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 37 MiB/s wr, 37 op/s 2023-09-10T01:17:56.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:56 smithi044 ceph-mon[98676]: pgmap v1196: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 37 MiB/s wr, 37 op/s 2023-09-10T01:17:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:57 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:17:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:57 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:17:57.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:57 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:17:57.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:57 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:17:57.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:57 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:17:57.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:57 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:17:57.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:57 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:17:57.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:57 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:17:57.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:57 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:17:58.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:58 smithi099 ceph-mon[102692]: pgmap v1197: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 57 MiB/s wr, 56 op/s 2023-09-10T01:17:58.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:17:58 smithi099 ceph-mon[102692]: pgmap v1198: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 46 MiB/s wr, 43 op/s 2023-09-10T01:17:58.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:58 smithi131 ceph-mon[103713]: pgmap v1197: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 57 MiB/s wr, 56 op/s 2023-09-10T01:17:58.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:17:58 smithi131 ceph-mon[103713]: pgmap v1198: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 46 MiB/s wr, 43 op/s 2023-09-10T01:17:58.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:58 smithi044 ceph-mon[98676]: pgmap v1197: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 57 MiB/s wr, 56 op/s 2023-09-10T01:17:58.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:17:58 smithi044 ceph-mon[98676]: pgmap v1198: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 46 MiB/s wr, 43 op/s 2023-09-10T01:18:00.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:00 smithi099 ceph-mon[102692]: pgmap v1199: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 46 MiB/s wr, 43 op/s 2023-09-10T01:18:00.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:00 smithi131 ceph-mon[103713]: pgmap v1199: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 46 MiB/s wr, 43 op/s 2023-09-10T01:18:00.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:00 smithi044 ceph-mon[98676]: pgmap v1199: 129 pgs: 129 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 46 MiB/s wr, 43 op/s 2023-09-10T01:18:02.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:02 smithi099 ceph-mon[102692]: pgmap v1200: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 45 MiB/s wr, 44 op/s 2023-09-10T01:18:02.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:02 smithi131 ceph-mon[103713]: pgmap v1200: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 45 MiB/s wr, 44 op/s 2023-09-10T01:18:02.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:02 smithi044 ceph-mon[98676]: pgmap v1200: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 45 MiB/s wr, 44 op/s 2023-09-10T01:18:04.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:04 smithi099 ceph-mon[102692]: pgmap v1201: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 45 MiB/s wr, 44 op/s 2023-09-10T01:18:04.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:04 smithi131 ceph-mon[103713]: pgmap v1201: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 45 MiB/s wr, 44 op/s 2023-09-10T01:18:04.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:04 smithi044 ceph-mon[98676]: pgmap v1201: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 45 MiB/s wr, 44 op/s 2023-09-10T01:18:06.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:06 smithi099 ceph-mon[102692]: pgmap v1202: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 45 MiB/s wr, 44 op/s 2023-09-10T01:18:06.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:06 smithi131 ceph-mon[103713]: pgmap v1202: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 45 MiB/s wr, 44 op/s 2023-09-10T01:18:06.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:06 smithi044 ceph-mon[98676]: pgmap v1202: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 45 MiB/s wr, 44 op/s 2023-09-10T01:18:08.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:08 smithi131 ceph-mon[103713]: pgmap v1203: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 44 MiB/s wr, 67 op/s 2023-09-10T01:18:08.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:08 smithi044 ceph-mon[98676]: pgmap v1203: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 44 MiB/s wr, 67 op/s 2023-09-10T01:18:08.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:08 smithi099 ceph-mon[102692]: pgmap v1203: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 44 MiB/s wr, 67 op/s 2023-09-10T01:18:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.382+0000 7f7d7a9e7700 1 -- 172.21.15.44:0/3705484962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d74108220 msgr2=0x7f7d7410b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.382+0000 7f7d7a9e7700 1 --2- 172.21.15.44:0/3705484962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d74108220 0x7f7d7410b830 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7d64007570 tx=0x7f7d64007880 comp rx=0 tx=0).stop 2023-09-10T01:18:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.383+0000 7f7d7a9e7700 1 -- 172.21.15.44:0/3705484962 shutdown_connections 2023-09-10T01:18:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.383+0000 7f7d7a9e7700 1 --2- 172.21.15.44:0/3705484962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d7410bf10 0x7f7d740788f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.383+0000 7f7d7a9e7700 1 --2- 172.21.15.44:0/3705484962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d74107870 0x7f7d74107c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.383+0000 7f7d7a9e7700 1 --2- 172.21.15.44:0/3705484962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d74108220 0x7f7d7410b830 secure :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7d64007570 tx=0x7f7d64007880 comp rx=0 tx=0).stop 2023-09-10T01:18:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.383+0000 7f7d7a9e7700 1 -- 172.21.15.44:0/3705484962 >> 172.21.15.44:0/3705484962 conn(0x7f7d7406c6c0 msgr2=0x7f7d7406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.383+0000 7f7d7a9e7700 1 -- 172.21.15.44:0/3705484962 shutdown_connections 2023-09-10T01:18:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.383+0000 7f7d7a9e7700 1 -- 172.21.15.44:0/3705484962 wait complete. 2023-09-10T01:18:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.383+0000 7f7d7a9e7700 1 Processor -- start 2023-09-10T01:18:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.385+0000 7f7d7a9e7700 1 -- start start 2023-09-10T01:18:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.386+0000 7f7d7a9e7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d74107870 0x7f7d74083150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.386+0000 7f7d7a9e7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d7410bf10 0x7f7d74081210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.386+0000 7f7d7a9e7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d74081750 0x7f7d74081c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.386+0000 7f7d7a9e7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d741ba850 con 0x7f7d7410bf10 2023-09-10T01:18:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.386+0000 7f7d7a9e7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d741ba9c0 con 0x7f7d74107870 2023-09-10T01:18:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.386+0000 7f7d7a9e7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d741bab30 con 0x7f7d74081750 2023-09-10T01:18:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.388+0000 7f7d73fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d74107870 0x7f7d74083150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.389+0000 7f7d73fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d74107870 0x7f7d74083150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:59818/0 (socket says 172.21.15.44:59818) 2023-09-10T01:18:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.389+0000 7f7d73fff700 1 -- 172.21.15.44:0/2720390679 learned_addr learned my addr 172.21.15.44:0/2720390679 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.389+0000 7f7d73fff700 1 -- 172.21.15.44:0/2720390679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d74081750 msgr2=0x7f7d74081c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.389+0000 7f7d73fff700 1 --2- 172.21.15.44:0/2720390679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d74081750 0x7f7d74081c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.389+0000 7f7d73fff700 1 -- 172.21.15.44:0/2720390679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d7410bf10 msgr2=0x7f7d74081210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.389+0000 7f7d73fff700 1 --2- 172.21.15.44:0/2720390679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d7410bf10 0x7f7d74081210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.389+0000 7f7d73fff700 1 -- 172.21.15.44:0/2720390679 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d64007220 con 0x7f7d74107870 2023-09-10T01:18:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.389+0000 7f7d73fff700 1 --2- 172.21.15.44:0/2720390679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d74107870 0x7f7d74083150 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800e9a0 tx=0x7f7d6800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.389+0000 7f7d717fa700 1 -- 172.21.15.44:0/2720390679 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d68009790 con 0x7f7d74107870 2023-09-10T01:18:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.390+0000 7f7d717fa700 1 -- 172.21.15.44:0/2720390679 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d68010430 con 0x7f7d74107870 2023-09-10T01:18:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.390+0000 7f7d717fa700 1 -- 172.21.15.44:0/2720390679 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d68018680 con 0x7f7d74107870 2023-09-10T01:18:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.391+0000 7f7d7a9e7700 1 -- 172.21.15.44:0/2720390679 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d741bae10 con 0x7f7d74107870 2023-09-10T01:18:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.391+0000 7f7d7a9e7700 1 -- 172.21.15.44:0/2720390679 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d741bb2a0 con 0x7f7d74107870 2023-09-10T01:18:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.394+0000 7f7d717fa700 1 -- 172.21.15.44:0/2720390679 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f7d6800cb50 con 0x7f7d74107870 2023-09-10T01:18:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.395+0000 7f7d717fa700 1 --2- 172.21.15.44:0/2720390679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7d5c0a0830 0x7f7d5c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.395+0000 7f7d717fa700 1 -- 172.21.15.44:0/2720390679 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f7d68014070 con 0x7f7d74107870 2023-09-10T01:18:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.400+0000 7f7d737fe700 1 --2- 172.21.15.44:0/2720390679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7d5c0a0830 0x7f7d5c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.400+0000 7f7d86badb80 1 Processor -- start 2023-09-10T01:18:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.400+0000 7f7d86badb80 1 -- start start 2023-09-10T01:18:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.400+0000 7f7d86badb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc976421a0 0x55dc97642580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.400+0000 7f7d86badb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763c300 0x55dc9763c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.400+0000 7f7d86badb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc9763ccc0 0x55dc97648f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.400+0000 7f7d86badb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc975956d0 con 0x55dc9763c300 2023-09-10T01:18:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.400+0000 7f7d86badb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc97592ff0 con 0x55dc9763ccc0 2023-09-10T01:18:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.400+0000 7f7d86badb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc9758e8e0 con 0x55dc976421a0 2023-09-10T01:18:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.401+0000 7f7d73fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc976421a0 0x55dc97642580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.401+0000 7f7d73fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc976421a0 0x55dc97642580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36736/0 (socket says 172.21.15.44:36736) 2023-09-10T01:18:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.401+0000 7f7d73fff700 1 -- 172.21.15.44:0/180052639 learned_addr learned my addr 172.21.15.44:0/180052639 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.403+0000 7f7d737fe700 1 --2- 172.21.15.44:0/180052639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763c300 0x55dc9763c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.403+0000 7f7d737fe700 1 --2- 172.21.15.44:0/2720390679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7d5c0a0830 0x7f7d5c0a2cf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7d7410d2e0 tx=0x7f7d64005ab0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.407+0000 7f7d737fe700 1 -- 172.21.15.44:0/180052639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc976421a0 msgr2=0x55dc97642580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.407+0000 7f7d737fe700 1 --2- 172.21.15.44:0/180052639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc976421a0 0x55dc97642580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.407+0000 7f7d737fe700 1 -- 172.21.15.44:0/180052639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc9763ccc0 msgr2=0x55dc97648f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.407+0000 7f7d737fe700 1 --2- 172.21.15.44:0/180052639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc9763ccc0 0x55dc97648f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.407+0000 7f7d737fe700 1 -- 172.21.15.44:0/180052639 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc973bad80 con 0x55dc9763c300 2023-09-10T01:18:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.408+0000 7f7d737fe700 1 --2- 172.21.15.44:0/180052639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763c300 0x55dc9763c780 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f7d7406d4d0 tx=0x7f7d6401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea33ed1f455d2a54 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.408+0000 7f7d5a7fc700 1 -- 172.21.15.44:0/180052639 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6402b070 con 0x55dc9763c300 2023-09-10T01:18:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.408+0000 7f7d5a7fc700 1 -- 172.21.15.44:0/180052639 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d64007df0 con 0x55dc9763c300 2023-09-10T01:18:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.408+0000 7f7d5a7fc700 1 -- 172.21.15.44:0/180052639 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6400f040 con 0x55dc9763c300 2023-09-10T01:18:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.409+0000 7f7d86badb80 1 -- 172.21.15.44:0/180052639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763c300 msgr2=0x55dc9763c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.409+0000 7f7d86badb80 1 --2- 172.21.15.44:0/180052639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763c300 0x55dc9763c780 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f7d7406d4d0 tx=0x7f7d6401a000 comp rx=0 tx=0).stop 2023-09-10T01:18:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.413+0000 7f7d86badb80 1 -- 172.21.15.44:0/180052639 shutdown_connections 2023-09-10T01:18:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.413+0000 7f7d86badb80 1 --2- 172.21.15.44:0/180052639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc9763ccc0 0x55dc97648f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.413+0000 7f7d86badb80 1 --2- 172.21.15.44:0/180052639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc976421a0 0x55dc97642580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.413+0000 7f7d86badb80 1 --2- 172.21.15.44:0/180052639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763c300 0x55dc9763c780 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.413+0000 7f7d86badb80 1 -- 172.21.15.44:0/180052639 >> 172.21.15.44:0/180052639 conn(0x55dc975b1030 msgr2=0x55dc97637560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.417+0000 7f7d86badb80 1 -- 172.21.15.44:0/180052639 shutdown_connections 2023-09-10T01:18:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.417+0000 7f7d86badb80 1 -- 172.21.15.44:0/180052639 wait complete. 2023-09-10T01:18:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.417+0000 7f7d86badb80 1 -- start start 2023-09-10T01:18:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.418+0000 7f7d86badb80 1 Processor -- start 2023-09-10T01:18:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.418+0000 7f7d86badb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc9763c300 0x55dc97648ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.418+0000 7f7d86badb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763ccc0 0x55dc97646170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.418+0000 7f7d86badb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc976421a0 0x55dc97646860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.418+0000 7f7d86badb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc9752cb10 con 0x55dc9763ccc0 2023-09-10T01:18:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.418+0000 7f7d86badb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc9752c810 con 0x55dc976421a0 2023-09-10T01:18:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.418+0000 7f7d86badb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc974108c0 con 0x55dc9763c300 2023-09-10T01:18:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.418+0000 7f7d86badb80 10 client.0 _ll_register_callbacks cb 0x55dc97476a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:18:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.424+0000 7f7d737fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763ccc0 0x55dc97646170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.424+0000 7f7d737fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763ccc0 0x55dc97646170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58092/0 (socket says 172.21.15.44:58092) 2023-09-10T01:18:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.424+0000 7f7d737fe700 1 -- 172.21.15.44:0/3179976226 learned_addr learned my addr 172.21.15.44:0/3179976226 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.424+0000 7f7d737fe700 1 -- 172.21.15.44:0/3179976226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc9763c300 msgr2=0x55dc97648ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:18:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.424+0000 7f7d737fe700 1 --2- 172.21.15.44:0/3179976226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc9763c300 0x55dc97648ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.424+0000 7f7d737fe700 1 -- 172.21.15.44:0/3179976226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc976421a0 msgr2=0x55dc97646860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:18:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.424+0000 7f7d737fe700 1 --2- 172.21.15.44:0/3179976226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc976421a0 0x55dc97646860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.424+0000 7f7d737fe700 1 -- 172.21.15.44:0/3179976226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d740791b0 con 0x55dc9763ccc0 2023-09-10T01:18:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.424+0000 7f7d737fe700 1 --2- 172.21.15.44:0/3179976226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763ccc0 0x55dc97646170 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400e010 tx=0x7f7d6400caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a5401c9f95e01f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.424+0000 7f7d537fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:18:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.425+0000 7f7d537fe700 1 -- 172.21.15.44:0/3179976226 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6402b040 con 0x55dc9763ccc0 2023-09-10T01:18:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.425+0000 7f7d537fe700 1 -- 172.21.15.44:0/3179976226 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(161..161 src has 1..161) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f7d6400a550 con 0x55dc9763ccc0 2023-09-10T01:18:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.425+0000 7f7d86badb80 1 -- 172.21.15.44:0/3179976226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc974e6490 con 0x55dc9763ccc0 2023-09-10T01:18:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.425+0000 7f7d537fe700 1 -- 172.21.15.44:0/3179976226 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d6400f040 con 0x55dc9763ccc0 2023-09-10T01:18:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.425+0000 7f7d86badb80 10 client.19698 fetch_fsmap learned FSMap version 125 2023-09-10T01:18:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.425+0000 7f7d86badb80 1 -- 172.21.15.44:0/3179976226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=4+}) v3 -- 0x55dc973a3c00 con 0x55dc9763ccc0 2023-09-10T01:18:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.425+0000 7f7d537fe700 1 -- 172.21.15.44:0/3179976226 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dc974e6490 con 0x55dc9763ccc0 2023-09-10T01:18:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.425+0000 7f7d537fe700 1 -- 172.21.15.44:0/3179976226 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 125) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f7d64008560 con 0x55dc9763ccc0 2023-09-10T01:18:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.425+0000 7f7d86badb80 10 client.19698 fetch_fsmap finished waiting for FSMap version 125 2023-09-10T01:18:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.425+0000 7f7d86badb80 10 client.19698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:18:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.425+0000 7f7d86badb80 20 client.19698 populate_metadata read hostname 'smithi044' 2023-09-10T01:18:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.425+0000 7f7d86badb80 1 --2- 172.21.15.44:0/3179976226 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55dc976560d0 0x55dc97658570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.425+0000 7f7d86badb80 4 client.19698 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:18:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.425+0000 7f7d86badb80 1 -- 172.21.15.44:0/3179976226 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc975a3840 con 0x55dc976560d0 2023-09-10T01:18:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.429+0000 7f7d73fff700 1 --2- 172.21.15.44:0/3179976226 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55dc976560d0 0x55dc97658570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.431+0000 7f7d537fe700 10 client.19698 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:18:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.431+0000 7f7d73fff700 1 --2- 172.21.15.44:0/3179976226 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55dc976560d0 0x55dc97658570 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a3aae3e3c20c689 server_cookie=48b084ef570e0b3e in_seq=0 out_seq=0 2023-09-10T01:18:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.431+0000 7f7d537fe700 1 -- 172.21.15.44:0/3179976226 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dc975a3840 con 0x55dc976560d0 2023-09-10T01:18:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.431+0000 7f7d537fe700 10 client.19698 handle_command_reply: tid=0 2023-09-10T01:18:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.431+0000 7f7d86badb80 1 client.19698 shutdown 2023-09-10T01:18:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.433+0000 7f7d86badb80 1 -- 172.21.15.44:0/3179976226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763ccc0 msgr2=0x55dc97646170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.433+0000 7f7d86badb80 1 --2- 172.21.15.44:0/3179976226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763ccc0 0x55dc97646170 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400e010 tx=0x7f7d6400caf0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.438+0000 7f7d86badb80 1 -- 172.21.15.44:0/3179976226 shutdown_connections 2023-09-10T01:18:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.438+0000 7f7d86badb80 1 --2- 172.21.15.44:0/3179976226 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55dc976560d0 0x55dc97658570 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.438+0000 7f7d86badb80 1 --2- 172.21.15.44:0/3179976226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc976421a0 0x55dc97646860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.438+0000 7f7d86badb80 1 --2- 172.21.15.44:0/3179976226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc9763c300 0x55dc97648ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.438+0000 7f7d86badb80 1 --2- 172.21.15.44:0/3179976226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763ccc0 0x55dc97646170 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.438+0000 7f7d86badb80 1 -- 172.21.15.44:0/3179976226 >> 172.21.15.44:0/3179976226 conn(0x55dc975b1030 msgr2=0x55dc976367a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.438+0000 7f7d537fe700 0 client.19698 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:18:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.439+0000 7f7d86badb80 1 -- 172.21.15.44:0/3179976226 shutdown_connections 2023-09-10T01:18:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.447+0000 7f7d86badb80 1 -- 172.21.15.44:0/3179976226 wait complete. 2023-09-10T01:18:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.454+0000 7f7d86badb80 20 client.19698 trim_cache size 0 max 16384 2023-09-10T01:18:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.485+0000 7f7d86badb80 1 Processor -- start 2023-09-10T01:18:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.486+0000 7f7d86badb80 1 -- start start 2023-09-10T01:18:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.486+0000 7f7d86badb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc9763c300 0x55dc97643e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.486+0000 7f7d86badb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc9763ccc0 0x55dc97646770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.486+0000 7f7d86badb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc976421a0 0x55dc9764ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.486+0000 7f7d86badb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc97641a40 con 0x55dc976421a0 2023-09-10T01:18:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.486+0000 7f7d86badb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc97647600 con 0x55dc9763ccc0 2023-09-10T01:18:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.486+0000 7f7d86badb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc97638210 con 0x55dc9763c300 2023-09-10T01:18:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.486+0000 7f7d73fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc9763c300 0x55dc97643e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:09.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.489+0000 7f7d73fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc9763c300 0x55dc97643e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36748/0 (socket says 172.21.15.44:36748) 2023-09-10T01:18:09.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.489+0000 7f7d73fff700 1 -- 172.21.15.44:0/3189434059 learned_addr learned my addr 172.21.15.44:0/3189434059 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.492+0000 7f7d737fe700 1 --2- 172.21.15.44:0/3189434059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc9763ccc0 0x55dc97646770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.493+0000 7f7d73fff700 1 -- 172.21.15.44:0/3189434059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc9763ccc0 msgr2=0x55dc97646770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.493+0000 7f7d73fff700 1 --2- 172.21.15.44:0/3189434059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc9763ccc0 0x55dc97646770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.493+0000 7f7d73fff700 1 -- 172.21.15.44:0/3189434059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc976421a0 msgr2=0x55dc9764ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.493+0000 7f7d73fff700 1 --2- 172.21.15.44:0/3189434059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc976421a0 0x55dc9764ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.493+0000 7f7d73fff700 1 -- 172.21.15.44:0/3189434059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc973e11f0 con 0x55dc9763c300 2023-09-10T01:18:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.493+0000 7f7d73fff700 1 --2- 172.21.15.44:0/3189434059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc9763c300 0x55dc97643e30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7d68099600 tx=0x7f7d680b9040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78f2e0403ef79874 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.493+0000 7f7d5affd700 1 -- 172.21.15.44:0/3189434059 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6809d730 con 0x55dc9763c300 2023-09-10T01:18:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.494+0000 7f7d5affd700 1 -- 172.21.15.44:0/3189434059 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d680a1970 con 0x55dc9763c300 2023-09-10T01:18:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.494+0000 7f7d5affd700 1 -- 172.21.15.44:0/3189434059 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d68010690 con 0x55dc9763c300 2023-09-10T01:18:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.494+0000 7f7d86badb80 1 -- 172.21.15.44:0/3189434059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc9763c300 msgr2=0x55dc97643e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.494+0000 7f7d86badb80 1 --2- 172.21.15.44:0/3189434059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc9763c300 0x55dc97643e30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7d68099600 tx=0x7f7d680b9040 comp rx=0 tx=0).stop 2023-09-10T01:18:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.496+0000 7f7d86badb80 1 -- 172.21.15.44:0/3189434059 shutdown_connections 2023-09-10T01:18:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.496+0000 7f7d86badb80 1 --2- 172.21.15.44:0/3189434059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc976421a0 0x55dc9764ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.496+0000 7f7d86badb80 1 --2- 172.21.15.44:0/3189434059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc9763c300 0x55dc97643e30 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.496+0000 7f7d86badb80 1 --2- 172.21.15.44:0/3189434059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc9763ccc0 0x55dc97646770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.496+0000 7f7d86badb80 1 -- 172.21.15.44:0/3189434059 >> 172.21.15.44:0/3189434059 conn(0x55dc975b1030 msgr2=0x55dc97637150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.513+0000 7f7d86badb80 1 -- 172.21.15.44:0/3189434059 shutdown_connections 2023-09-10T01:18:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.520+0000 7f7d86badb80 1 -- 172.21.15.44:0/3189434059 wait complete. 2023-09-10T01:18:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.520+0000 7f7d86badb80 1 -- start start 2023-09-10T01:18:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.520+0000 7f7d86badb80 1 Processor -- start 2023-09-10T01:18:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.521+0000 7f7d86badb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763c300 0x55dc9769e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.521+0000 7f7d86badb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc976421a0 0x55dc9769e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.521+0000 7f7d86badb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc976a30f0 0x55dc976a3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.521+0000 7f7d86badb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc976a3ab0 con 0x55dc9763c300 2023-09-10T01:18:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.521+0000 7f7d86badb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc976a3c20 con 0x55dc976421a0 2023-09-10T01:18:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.521+0000 7f7d86badb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc976a3de0 con 0x55dc976a30f0 2023-09-10T01:18:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.521+0000 7f7d86badb80 10 client.0 _ll_register_callbacks cb 0x7f7d640058e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:18:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.524+0000 7f7d78f84700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc976a30f0 0x55dc976a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.524+0000 7f7d78f84700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc976a30f0 0x55dc976a3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36758/0 (socket says 172.21.15.44:36758) 2023-09-10T01:18:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.524+0000 7f7d78f84700 1 -- 172.21.15.44:0/1510204462 learned_addr learned my addr 172.21.15.44:0/1510204462 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.524+0000 7f7d78f84700 1 -- 172.21.15.44:0/1510204462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc976421a0 msgr2=0x55dc9769e680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:18:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.524+0000 7f7d78f84700 1 --2- 172.21.15.44:0/1510204462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc976421a0 0x55dc9769e680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.524+0000 7f7d78f84700 1 -- 172.21.15.44:0/1510204462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763c300 msgr2=0x55dc9769e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.524+0000 7f7d78f84700 1 --2- 172.21.15.44:0/1510204462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763c300 0x55dc9769e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.524+0000 7f7d78f84700 1 -- 172.21.15.44:0/1510204462 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d741bb2a0 con 0x55dc976a30f0 2023-09-10T01:18:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.525+0000 7f7d78f84700 1 --2- 172.21.15.44:0/1510204462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc976a30f0 0x55dc976a3570 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00eb00 tx=0x7f7d6c00eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c9a3fe3b34882e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.525+0000 7f7d537fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:18:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.525+0000 7f7d537fe700 1 -- 172.21.15.44:0/1510204462 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c018070 con 0x55dc976a30f0 2023-09-10T01:18:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.525+0000 7f7d537fe700 1 -- 172.21.15.44:0/1510204462 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(161..161 src has 1..161) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f7d6c0092e0 con 0x55dc976a30f0 2023-09-10T01:18:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.525+0000 7f7d86badb80 1 -- 172.21.15.44:0/1510204462 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d64007df0 con 0x55dc976a30f0 2023-09-10T01:18:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.526+0000 7f7d537fe700 1 -- 172.21.15.44:0/1510204462 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d6c010040 con 0x55dc976a30f0 2023-09-10T01:18:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.528+0000 7f7d86badb80 10 client.29302 fetch_fsmap learned FSMap version 125 2023-09-10T01:18:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.528+0000 7f7d86badb80 1 -- 172.21.15.44:0/1510204462 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=4+}) v3 -- 0x55dc973b9250 con 0x55dc976a30f0 2023-09-10T01:18:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.533+0000 7f7d537fe700 1 -- 172.21.15.44:0/1510204462 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d64007df0 con 0x55dc976a30f0 2023-09-10T01:18:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.533+0000 7f7d537fe700 1 -- 172.21.15.44:0/1510204462 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 125) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f7d6c01b020 con 0x55dc976a30f0 2023-09-10T01:18:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.534+0000 7f7d86badb80 10 client.29302 fetch_fsmap finished waiting for FSMap version 125 2023-09-10T01:18:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.534+0000 7f7d86badb80 10 client.29302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:18:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.534+0000 7f7d86badb80 20 client.29302 populate_metadata read hostname 'smithi044' 2023-09-10T01:18:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.534+0000 7f7d86badb80 1 --2- 172.21.15.44:0/1510204462 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55dc976ab610 0x55dc976ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.534+0000 7f7d86badb80 4 client.29302 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:18:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.534+0000 7f7d86badb80 1 -- 172.21.15.44:0/1510204462 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dc975878c0 con 0x55dc976ab610 2023-09-10T01:18:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.542+0000 7f7d73fff700 1 --2- 172.21.15.44:0/1510204462 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55dc976ab610 0x55dc976ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.558+0000 7f7d537fe700 10 client.29302 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:18:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.558+0000 7f7d73fff700 1 --2- 172.21.15.44:0/1510204462 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55dc976ab610 0x55dc976ada30 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfd5d3885f10d0ec server_cookie=d6264e7e01409693 in_seq=0 out_seq=0 2023-09-10T01:18:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.619+0000 7f7d537fe700 1 -- 172.21.15.44:0/1510204462 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dc975878c0 con 0x55dc976ab610 2023-09-10T01:18:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.620+0000 7f7d537fe700 10 client.29302 handle_command_reply: tid=0 2023-09-10T01:18:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.620+0000 7f7d86badb80 1 client.29302 shutdown 2023-09-10T01:18:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.622+0000 7f7d86badb80 1 -- 172.21.15.44:0/1510204462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc976a30f0 msgr2=0x55dc976a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.622+0000 7f7d86badb80 1 --2- 172.21.15.44:0/1510204462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc976a30f0 0x55dc976a3570 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00eb00 tx=0x7f7d6c00eec0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.622+0000 7f7d86badb80 1 -- 172.21.15.44:0/1510204462 shutdown_connections 2023-09-10T01:18:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.622+0000 7f7d86badb80 1 --2- 172.21.15.44:0/1510204462 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55dc976ab610 0x55dc976ada30 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.622+0000 7f7d86badb80 1 --2- 172.21.15.44:0/1510204462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc976a30f0 0x55dc976a3570 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.622+0000 7f7d86badb80 1 --2- 172.21.15.44:0/1510204462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dc9763c300 0x55dc9769e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.622+0000 7f7d86badb80 1 --2- 172.21.15.44:0/1510204462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc976421a0 0x55dc9769e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.622+0000 7f7d86badb80 1 -- 172.21.15.44:0/1510204462 >> 172.21.15.44:0/1510204462 conn(0x55dc975b1030 msgr2=0x55dc97645a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.622+0000 7f7d537fe700 0 client.29302 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:18:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.622+0000 7f7d86badb80 1 -- 172.21.15.44:0/1510204462 shutdown_connections 2023-09-10T01:18:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.622+0000 7f7d86badb80 1 -- 172.21.15.44:0/1510204462 wait complete. 2023-09-10T01:18:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.623+0000 7f7d86badb80 20 client.29302 trim_cache size 0 max 16384 2023-09-10T01:18:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:18:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:18:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:18:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:18:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.632+0000 7f7d7a9e7700 1 -- 172.21.15.44:0/2720390679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7d5c0a0830 msgr2=0x7f7d5c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.632+0000 7f7d7a9e7700 1 --2- 172.21.15.44:0/2720390679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7d5c0a0830 0x7f7d5c0a2cf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7d7410d2e0 tx=0x7f7d64005ab0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.632+0000 7f7d7a9e7700 1 -- 172.21.15.44:0/2720390679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d74107870 msgr2=0x7f7d74083150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.632+0000 7f7d7a9e7700 1 --2- 172.21.15.44:0/2720390679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d74107870 0x7f7d74083150 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800e9a0 tx=0x7f7d6800ed60 comp rx=0 tx=0).stop 2023-09-10T01:18:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.634+0000 7f7d7a9e7700 1 -- 172.21.15.44:0/2720390679 shutdown_connections 2023-09-10T01:18:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.634+0000 7f7d7a9e7700 1 --2- 172.21.15.44:0/2720390679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7d5c0a0830 0x7f7d5c0a2cf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.634+0000 7f7d7a9e7700 1 --2- 172.21.15.44:0/2720390679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d74081750 0x7f7d74081c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.634+0000 7f7d7a9e7700 1 --2- 172.21.15.44:0/2720390679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d74107870 0x7f7d74083150 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.634+0000 7f7d7a9e7700 1 --2- 172.21.15.44:0/2720390679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d7410bf10 0x7f7d74081210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.634+0000 7f7d7a9e7700 1 -- 172.21.15.44:0/2720390679 >> 172.21.15.44:0/2720390679 conn(0x7f7d7406c6c0 msgr2=0x7f7d74070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.649+0000 7f7d7a9e7700 1 -- 172.21.15.44:0/2720390679 shutdown_connections 2023-09-10T01:18:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:09.651+0000 7f7d7a9e7700 1 -- 172.21.15.44:0/2720390679 wait complete. 2023-09-10T01:18:10.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:10 smithi131 ceph-mon[103713]: pgmap v1204: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 36 MiB/s wr, 56 op/s 2023-09-10T01:18:10.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:10 smithi131 ceph-mon[103713]: osdmap e161: 12 total, 12 up, 12 in 2023-09-10T01:18:10.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:10 smithi044 ceph-mon[98676]: pgmap v1204: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 36 MiB/s wr, 56 op/s 2023-09-10T01:18:10.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:10 smithi044 ceph-mon[98676]: osdmap e161: 12 total, 12 up, 12 in 2023-09-10T01:18:10.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:10 smithi099 ceph-mon[102692]: pgmap v1204: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 36 MiB/s wr, 56 op/s 2023-09-10T01:18:10.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:10 smithi099 ceph-mon[102692]: osdmap e161: 12 total, 12 up, 12 in 2023-09-10T01:18:12.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:12 smithi131 ceph-mon[103713]: pgmap v1206: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 35 MiB/s wr, 198 op/s 2023-09-10T01:18:12.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:12 smithi044 ceph-mon[98676]: pgmap v1206: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 35 MiB/s wr, 198 op/s 2023-09-10T01:18:12.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:12 smithi099 ceph-mon[102692]: pgmap v1206: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 35 MiB/s wr, 198 op/s 2023-09-10T01:18:14.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:14 smithi131 ceph-mon[103713]: pgmap v1207: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 35 MiB/s wr, 198 op/s 2023-09-10T01:18:14.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:14 smithi044 ceph-mon[98676]: pgmap v1207: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 35 MiB/s wr, 198 op/s 2023-09-10T01:18:14.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:14 smithi099 ceph-mon[102692]: pgmap v1207: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 35 MiB/s wr, 198 op/s 2023-09-10T01:18:14.908 INFO:teuthology.orchestra.run.smithi044.stderr:set primary keys... 2023-09-10T01:18:16.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:16 smithi131 ceph-mon[103713]: pgmap v1208: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 35 MiB/s wr, 198 op/s 2023-09-10T01:18:16.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:16 smithi131 ceph-mon[103713]: osdmap e162: 12 total, 12 up, 12 in 2023-09-10T01:18:16.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:16 smithi044 ceph-mon[98676]: pgmap v1208: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 35 MiB/s wr, 198 op/s 2023-09-10T01:18:16.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:16 smithi044 ceph-mon[98676]: osdmap e162: 12 total, 12 up, 12 in 2023-09-10T01:18:16.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:16 smithi099 ceph-mon[102692]: pgmap v1208: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 35 MiB/s wr, 198 op/s 2023-09-10T01:18:16.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:16 smithi099 ceph-mon[102692]: osdmap e162: 12 total, 12 up, 12 in 2023-09-10T01:18:18.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:18 smithi131 ceph-mon[103713]: pgmap v1210: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 16 MiB/s wr, 770 op/s 2023-09-10T01:18:18.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:18 smithi044 ceph-mon[98676]: pgmap v1210: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 16 MiB/s wr, 770 op/s 2023-09-10T01:18:18.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:18 smithi099 ceph-mon[102692]: pgmap v1210: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 16 MiB/s wr, 770 op/s 2023-09-10T01:18:20.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:20 smithi131 ceph-mon[103713]: pgmap v1211: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 14 MiB/s wr, 678 op/s 2023-09-10T01:18:20.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:20 smithi044 ceph-mon[98676]: pgmap v1211: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 14 MiB/s wr, 678 op/s 2023-09-10T01:18:20.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:20 smithi099 ceph-mon[102692]: pgmap v1211: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 14 MiB/s wr, 678 op/s 2023-09-10T01:18:22.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:22 smithi131 ceph-mon[103713]: pgmap v1212: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 5.8 MiB/s wr, 877 op/s 2023-09-10T01:18:22.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:22 smithi131 ceph-mon[103713]: osdmap e163: 12 total, 12 up, 12 in 2023-09-10T01:18:22.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:22 smithi044 ceph-mon[98676]: pgmap v1212: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 5.8 MiB/s wr, 877 op/s 2023-09-10T01:18:22.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:22 smithi044 ceph-mon[98676]: osdmap e163: 12 total, 12 up, 12 in 2023-09-10T01:18:22.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:22 smithi099 ceph-mon[102692]: pgmap v1212: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 5.8 MiB/s wr, 877 op/s 2023-09-10T01:18:22.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:22 smithi099 ceph-mon[102692]: osdmap e163: 12 total, 12 up, 12 in 2023-09-10T01:18:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.793+0000 7fc786a0d700 1 -- 172.21.15.44:0/2270888503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc780105920 msgr2=0x7fc780067f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.793+0000 7fc786a0d700 1 --2- 172.21.15.44:0/2270888503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc780105920 0x7fc780067f30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc7740098f0 tx=0x7fc774009c00 comp rx=0 tx=0).stop 2023-09-10T01:18:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.793+0000 7fc786a0d700 1 -- 172.21.15.44:0/2270888503 shutdown_connections 2023-09-10T01:18:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.793+0000 7fc786a0d700 1 --2- 172.21.15.44:0/2270888503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc780068610 0x7fc780103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.793+0000 7fc786a0d700 1 --2- 172.21.15.44:0/2270888503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7801054b0 0x7fc7800679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.793+0000 7fc786a0d700 1 --2- 172.21.15.44:0/2270888503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc780105920 0x7fc780067f30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.793+0000 7fc786a0d700 1 -- 172.21.15.44:0/2270888503 >> 172.21.15.44:0/2270888503 conn(0x7fc780074f30 msgr2=0x7fc780075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.793+0000 7fc786a0d700 1 -- 172.21.15.44:0/2270888503 shutdown_connections 2023-09-10T01:18:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.793+0000 7fc786a0d700 1 -- 172.21.15.44:0/2270888503 wait complete. 2023-09-10T01:18:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.794+0000 7fc786a0d700 1 Processor -- start 2023-09-10T01:18:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.794+0000 7fc786a0d700 1 -- start start 2023-09-10T01:18:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc786a0d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7801054b0 0x7fc7801a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc786a0d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc780105920 0x7fc7801a31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc786a0d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc78019cac0 0x7fc78019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc786a0d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78019d540 con 0x7fc78019cac0 2023-09-10T01:18:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc786a0d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78019d680 con 0x7fc780105920 2023-09-10T01:18:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc786a0d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78019d840 con 0x7fc7801054b0 2023-09-10T01:18:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc784faa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc78019cac0 0x7fc78019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc784faa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc78019cac0 0x7fc78019cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58116/0 (socket says 172.21.15.44:58116) 2023-09-10T01:18:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc784faa700 1 -- 172.21.15.44:0/1003091844 learned_addr learned my addr 172.21.15.44:0/1003091844 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc77f7fe700 1 --2- 172.21.15.44:0/1003091844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc780105920 0x7fc7801a31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc77ffff700 1 --2- 172.21.15.44:0/1003091844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7801054b0 0x7fc7801a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc784faa700 1 -- 172.21.15.44:0/1003091844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7801054b0 msgr2=0x7fc7801a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc784faa700 1 --2- 172.21.15.44:0/1003091844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7801054b0 0x7fc7801a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc784faa700 1 -- 172.21.15.44:0/1003091844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc780105920 msgr2=0x7fc7801a31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc784faa700 1 --2- 172.21.15.44:0/1003091844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc780105920 0x7fc7801a31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc784faa700 1 -- 172.21.15.44:0/1003091844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc774009520 con 0x7fc78019cac0 2023-09-10T01:18:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.795+0000 7fc77f7fe700 1 --2- 172.21.15.44:0/1003091844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc780105920 0x7fc7801a31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:18:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.796+0000 7fc784faa700 1 --2- 172.21.15.44:0/1003091844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc78019cac0 0x7fc78019cf70 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fc77800ef30 tx=0x7fc77800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.796+0000 7fc77d7fa700 1 -- 172.21.15.44:0/1003091844 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778018070 con 0x7fc78019cac0 2023-09-10T01:18:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.796+0000 7fc786a0d700 1 -- 172.21.15.44:0/1003091844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7801a9ad0 con 0x7fc78019cac0 2023-09-10T01:18:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.796+0000 7fc77d7fa700 1 -- 172.21.15.44:0/1003091844 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc77800f040 con 0x7fc78019cac0 2023-09-10T01:18:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.796+0000 7fc77d7fa700 1 -- 172.21.15.44:0/1003091844 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77800cbb0 con 0x7fc78019cac0 2023-09-10T01:18:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.796+0000 7fc786a0d700 1 -- 172.21.15.44:0/1003091844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7801a9fb0 con 0x7fc78019cac0 2023-09-10T01:18:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.798+0000 7fc77d7fa700 1 -- 172.21.15.44:0/1003091844 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc77800cd70 con 0x7fc78019cac0 2023-09-10T01:18:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.799+0000 7fc77d7fa700 1 --2- 172.21.15.44:0/1003091844 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc7680a0830 0x7fc7680a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.799+0000 7fc77d7fa700 1 -- 172.21.15.44:0/1003091844 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fc7780cbe00 con 0x7fc78019cac0 2023-09-10T01:18:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.800+0000 7fc77ffff700 1 --2- 172.21.15.44:0/1003091844 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc7680a0830 0x7fc7680a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.801+0000 7fc792bd3b80 1 Processor -- start 2023-09-10T01:18:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.801+0000 7fc77ffff700 1 --2- 172.21.15.44:0/1003091844 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc7680a0830 0x7fc7680a2cf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc770005fd0 tx=0x7fc770009310 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.801+0000 7fc792bd3b80 1 -- start start 2023-09-10T01:18:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.801+0000 7fc792bd3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 0x55d14cdba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.801+0000 7fc792bd3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4300 0x55d14cdb4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.801+0000 7fc792bd3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 0x55d14cdc0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.801+0000 7fc792bd3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d14cd0d6d0 con 0x55d14cdb4300 2023-09-10T01:18:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.801+0000 7fc792bd3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d14cd0aff0 con 0x55d14cdba1a0 2023-09-10T01:18:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.801+0000 7fc792bd3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d14cd068e0 con 0x55d14cdb4cc0 2023-09-10T01:18:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.801+0000 7fc77f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4300 0x55d14cdb4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.801+0000 7fc77f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4300 0x55d14cdb4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58126/0 (socket says 172.21.15.44:58126) 2023-09-10T01:18:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.801+0000 7fc77ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 0x55d14cdba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.801+0000 7fc77f7fe700 1 -- 172.21.15.44:0/3838113198 learned_addr learned my addr 172.21.15.44:0/3838113198 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.801+0000 7fc784faa700 1 --2- 172.21.15.44:0/3838113198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 0x55d14cdc0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.801+0000 7fc77ffff700 1 -- 172.21.15.44:0/3838113198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 msgr2=0x55d14cdc0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.801+0000 7fc77ffff700 1 --2- 172.21.15.44:0/3838113198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 0x55d14cdc0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.802+0000 7fc77ffff700 1 -- 172.21.15.44:0/3838113198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4300 msgr2=0x55d14cdb4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.802+0000 7fc77ffff700 1 --2- 172.21.15.44:0/3838113198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4300 0x55d14cdb4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.802+0000 7fc77ffff700 1 -- 172.21.15.44:0/3838113198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d14cb32d80 con 0x55d14cdba1a0 2023-09-10T01:18:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.802+0000 7fc784faa700 1 --2- 172.21.15.44:0/3838113198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 0x55d14cdc0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:18:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.802+0000 7fc77ffff700 1 --2- 172.21.15.44:0/3838113198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 0x55d14cdba580 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc7700117d0 tx=0x7fc770014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91949dd821bd894f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.802+0000 7fc7667fc700 1 -- 172.21.15.44:0/3838113198 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770004830 con 0x55d14cdba1a0 2023-09-10T01:18:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.802+0000 7fc7667fc700 1 -- 172.21.15.44:0/3838113198 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc770004e20 con 0x55d14cdba1a0 2023-09-10T01:18:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.802+0000 7fc7667fc700 1 -- 172.21.15.44:0/3838113198 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7700038c0 con 0x55d14cdba1a0 2023-09-10T01:18:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.803+0000 7fc792bd3b80 1 -- 172.21.15.44:0/3838113198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 msgr2=0x55d14cdba580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.803+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/3838113198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 0x55d14cdba580 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc7700117d0 tx=0x7fc770014000 comp rx=0 tx=0).stop 2023-09-10T01:18:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.803+0000 7fc792bd3b80 1 -- 172.21.15.44:0/3838113198 shutdown_connections 2023-09-10T01:18:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.803+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/3838113198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 0x55d14cdc0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.803+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/3838113198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 0x55d14cdba580 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.803+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/3838113198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4300 0x55d14cdb4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.803+0000 7fc792bd3b80 1 -- 172.21.15.44:0/3838113198 >> 172.21.15.44:0/3838113198 conn(0x55d14cd29030 msgr2=0x55d14cdaf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.803+0000 7fc792bd3b80 1 -- 172.21.15.44:0/3838113198 shutdown_connections 2023-09-10T01:18:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.803+0000 7fc792bd3b80 1 -- 172.21.15.44:0/3838113198 wait complete. 2023-09-10T01:18:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.803+0000 7fc792bd3b80 1 -- start start 2023-09-10T01:18:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.804+0000 7fc792bd3b80 1 Processor -- start 2023-09-10T01:18:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.804+0000 7fc792bd3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdb4300 0x55d14cdc0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.804+0000 7fc792bd3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 0x55d14cdbe080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.804+0000 7fc792bd3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdba1a0 0x55d14cdbe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.804+0000 7fc792bd3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d14cca4b10 con 0x55d14cdba1a0 2023-09-10T01:18:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.804+0000 7fc792bd3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d14cca4810 con 0x55d14cdb4300 2023-09-10T01:18:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.804+0000 7fc792bd3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d14cb888c0 con 0x55d14cdb4cc0 2023-09-10T01:18:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.804+0000 7fc792bd3b80 10 client.0 _ll_register_callbacks cb 0x55d14cbeea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:18:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.804+0000 7fc77f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 0x55d14cdbe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.804+0000 7fc77f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 0x55d14cdbe080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36782/0 (socket says 172.21.15.44:36782) 2023-09-10T01:18:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.804+0000 7fc77ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdb4300 0x55d14cdc0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.804+0000 7fc77f7fe700 1 -- 172.21.15.44:0/2386945587 learned_addr learned my addr 172.21.15.44:0/2386945587 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.804+0000 7fc784faa700 1 --2- 172.21.15.44:0/2386945587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdba1a0 0x55d14cdbe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.804+0000 7fc77f7fe700 1 -- 172.21.15.44:0/2386945587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdb4300 msgr2=0x55d14cdc0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.804+0000 7fc77f7fe700 1 --2- 172.21.15.44:0/2386945587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdb4300 0x55d14cdc0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.805+0000 7fc77f7fe700 1 -- 172.21.15.44:0/2386945587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdba1a0 msgr2=0x55d14cdbe690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.805+0000 7fc77f7fe700 1 --2- 172.21.15.44:0/2386945587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdba1a0 0x55d14cdbe690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.805+0000 7fc77f7fe700 1 -- 172.21.15.44:0/2386945587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc770017070 con 0x55d14cdb4cc0 2023-09-10T01:18:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.805+0000 7fc77ffff700 1 --2- 172.21.15.44:0/2386945587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdb4300 0x55d14cdc0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:18:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.805+0000 7fc77f7fe700 1 --2- 172.21.15.44:0/2386945587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 0x55d14cdbe080 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc774009230 tx=0x7fc77400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60b5cc0a7bf7c455 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.805+0000 7fc75f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:18:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.805+0000 7fc75f7fe700 1 -- 172.21.15.44:0/2386945587 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77401c040 con 0x55d14cdb4cc0 2023-09-10T01:18:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.805+0000 7fc75f7fe700 1 -- 172.21.15.44:0/2386945587 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fc774005040 con 0x55d14cdb4cc0 2023-09-10T01:18:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.805+0000 7fc792bd3b80 1 -- 172.21.15.44:0/2386945587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d14cc5e490 con 0x55d14cdb4cc0 2023-09-10T01:18:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.805+0000 7fc75f7fe700 1 -- 172.21.15.44:0/2386945587 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc77400bec0 con 0x55d14cdb4cc0 2023-09-10T01:18:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.805+0000 7fc792bd3b80 10 client.29314 fetch_fsmap learned FSMap version 125 2023-09-10T01:18:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.805+0000 7fc792bd3b80 1 -- 172.21.15.44:0/2386945587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=4+}) v3 -- 0x55d14cb1bc00 con 0x55d14cdb4cc0 2023-09-10T01:18:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.805+0000 7fc75f7fe700 1 -- 172.21.15.44:0/2386945587 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d14cc5e490 con 0x55d14cdb4cc0 2023-09-10T01:18:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.806+0000 7fc75f7fe700 1 -- 172.21.15.44:0/2386945587 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 125) v1 ==== 7331+0+0 (secure 0 0 0) 0x7fc774015020 con 0x55d14cdb4cc0 2023-09-10T01:18:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.806+0000 7fc792bd3b80 10 client.29314 fetch_fsmap finished waiting for FSMap version 125 2023-09-10T01:18:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.806+0000 7fc792bd3b80 10 client.29314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:18:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.806+0000 7fc792bd3b80 20 client.29314 populate_metadata read hostname 'smithi044' 2023-09-10T01:18:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.806+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/2386945587 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d14cdce5b0 0x55d14cdd09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.806+0000 7fc792bd3b80 4 client.29314 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:18:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.806+0000 7fc792bd3b80 1 -- 172.21.15.44:0/2386945587 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d14cd1b840 con 0x55d14cdce5b0 2023-09-10T01:18:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.806+0000 7fc77ffff700 1 --2- 172.21.15.44:0/2386945587 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d14cdce5b0 0x55d14cdd09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.806+0000 7fc75f7fe700 10 client.29314 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:18:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.806+0000 7fc77ffff700 1 --2- 172.21.15.44:0/2386945587 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d14cdce5b0 0x55d14cdd09d0 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d5b9f12bee4cf63 server_cookie=26be0ae972854f09 in_seq=0 out_seq=0 2023-09-10T01:18:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.807+0000 7fc75f7fe700 1 -- 172.21.15.44:0/2386945587 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d14cd1b840 con 0x55d14cdce5b0 2023-09-10T01:18:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.807+0000 7fc75f7fe700 10 client.29314 handle_command_reply: tid=0 2023-09-10T01:18:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.807+0000 7fc792bd3b80 1 client.29314 shutdown 2023-09-10T01:18:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.807+0000 7fc792bd3b80 1 -- 172.21.15.44:0/2386945587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 msgr2=0x55d14cdbe080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.807+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/2386945587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 0x55d14cdbe080 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc774009230 tx=0x7fc77400b790 comp rx=0 tx=0).stop 2023-09-10T01:18:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.807+0000 7fc792bd3b80 1 -- 172.21.15.44:0/2386945587 shutdown_connections 2023-09-10T01:18:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.807+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/2386945587 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d14cdce5b0 0x55d14cdd09d0 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.807+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/2386945587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdba1a0 0x55d14cdbe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.807+0000 7fc75f7fe700 0 client.29314 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:18:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.807+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/2386945587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdb4300 0x55d14cdc0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.807+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/2386945587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 0x55d14cdbe080 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.807+0000 7fc792bd3b80 1 -- 172.21.15.44:0/2386945587 >> 172.21.15.44:0/2386945587 conn(0x55d14cd29030 msgr2=0x55d14cdae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.807+0000 7fc792bd3b80 1 -- 172.21.15.44:0/2386945587 shutdown_connections 2023-09-10T01:18:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.807+0000 7fc792bd3b80 1 -- 172.21.15.44:0/2386945587 wait complete. 2023-09-10T01:18:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.808+0000 7fc792bd3b80 20 client.29314 trim_cache size 0 max 16384 2023-09-10T01:18:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.828+0000 7fc792bd3b80 1 Processor -- start 2023-09-10T01:18:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc792bd3b80 1 -- start start 2023-09-10T01:18:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc792bd3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4300 0x55d14cdc09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc792bd3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 0x55d14cdaef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc792bd3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 0x55d14cdaf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc792bd3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d14cdb5580 con 0x55d14cdb4300 2023-09-10T01:18:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc792bd3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d14cdd1030 con 0x55d14cdba1a0 2023-09-10T01:18:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc792bd3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d14cdbc5e0 con 0x55d14cdb4cc0 2023-09-10T01:18:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc77ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4300 0x55d14cdc09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc77f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 0x55d14cdaef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc77ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4300 0x55d14cdc09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58138/0 (socket says 172.21.15.44:58138) 2023-09-10T01:18:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc77ffff700 1 -- 172.21.15.44:0/81412098 learned_addr learned my addr 172.21.15.44:0/81412098 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc784faa700 1 --2- 172.21.15.44:0/81412098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 0x55d14cdaf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc77ffff700 1 -- 172.21.15.44:0/81412098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 msgr2=0x55d14cdaef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc77ffff700 1 --2- 172.21.15.44:0/81412098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 0x55d14cdaef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc77ffff700 1 -- 172.21.15.44:0/81412098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 msgr2=0x55d14cdaf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc77ffff700 1 --2- 172.21.15.44:0/81412098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 0x55d14cdaf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc77ffff700 1 -- 172.21.15.44:0/81412098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d14cb591f0 con 0x55d14cdb4300 2023-09-10T01:18:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.829+0000 7fc784faa700 1 --2- 172.21.15.44:0/81412098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 0x55d14cdaf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:18:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.830+0000 7fc77ffff700 1 --2- 172.21.15.44:0/81412098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4300 0x55d14cdc09a0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x55d14cdd12a0 tx=0x7fc770017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b684a1e0212f3328 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.830+0000 7fc766ffd700 1 -- 172.21.15.44:0/81412098 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7700052a0 con 0x55d14cdb4300 2023-09-10T01:18:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.830+0000 7fc766ffd700 1 -- 172.21.15.44:0/81412098 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc77002d040 con 0x55d14cdb4300 2023-09-10T01:18:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.830+0000 7fc766ffd700 1 -- 172.21.15.44:0/81412098 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770004930 con 0x55d14cdb4300 2023-09-10T01:18:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.831+0000 7fc792bd3b80 1 -- 172.21.15.44:0/81412098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4300 msgr2=0x55d14cdc09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.831+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/81412098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4300 0x55d14cdc09a0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x55d14cdd12a0 tx=0x7fc770017000 comp rx=0 tx=0).stop 2023-09-10T01:18:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.831+0000 7fc792bd3b80 1 -- 172.21.15.44:0/81412098 shutdown_connections 2023-09-10T01:18:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.831+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/81412098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 0x55d14cdaf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.831+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/81412098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4300 0x55d14cdc09a0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.831+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/81412098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4cc0 0x55d14cdaef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.831+0000 7fc792bd3b80 1 -- 172.21.15.44:0/81412098 >> 172.21.15.44:0/81412098 conn(0x55d14cd29030 msgr2=0x55d14cdb9070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.831+0000 7fc792bd3b80 1 -- 172.21.15.44:0/81412098 shutdown_connections 2023-09-10T01:18:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.831+0000 7fc792bd3b80 1 -- 172.21.15.44:0/81412098 wait complete. 2023-09-10T01:18:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.831+0000 7fc792bd3b80 1 -- start start 2023-09-10T01:18:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.832+0000 7fc792bd3b80 1 Processor -- start 2023-09-10T01:18:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.832+0000 7fc792bd3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4300 0x55d14cdc5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.832+0000 7fc792bd3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4cc0 0x55d14cdc3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.832+0000 7fc792bd3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 0x55d14cdc3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.832+0000 7fc792bd3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d14cdc4520 con 0x55d14cdb4cc0 2023-09-10T01:18:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.832+0000 7fc792bd3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d14cdc4690 con 0x55d14cdba1a0 2023-09-10T01:18:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.832+0000 7fc792bd3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d14ce223b0 con 0x55d14cdb4300 2023-09-10T01:18:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.832+0000 7fc792bd3b80 10 client.0 _ll_register_callbacks cb 0x7fc77400f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:18:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.832+0000 7fc77f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4cc0 0x55d14cdc3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.832+0000 7fc77ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4300 0x55d14cdc5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.832+0000 7fc77f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4cc0 0x55d14cdc3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58146/0 (socket says 172.21.15.44:58146) 2023-09-10T01:18:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.832+0000 7fc77ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4300 0x55d14cdc5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36794/0 (socket says 172.21.15.44:36794) 2023-09-10T01:18:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.832+0000 7fc77f7fe700 1 -- 172.21.15.44:0/1338652055 learned_addr learned my addr 172.21.15.44:0/1338652055 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.833+0000 7fc784faa700 1 --2- 172.21.15.44:0/1338652055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 0x55d14cdc3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.833+0000 7fc77f7fe700 1 -- 172.21.15.44:0/1338652055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4300 msgr2=0x55d14cdc5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.833+0000 7fc77f7fe700 1 --2- 172.21.15.44:0/1338652055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4300 0x55d14cdc5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.833+0000 7fc77f7fe700 1 -- 172.21.15.44:0/1338652055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 msgr2=0x55d14cdc3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.833+0000 7fc77f7fe700 1 --2- 172.21.15.44:0/1338652055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 0x55d14cdc3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.833+0000 7fc77f7fe700 1 -- 172.21.15.44:0/1338652055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d14cb32d80 con 0x55d14cdb4cc0 2023-09-10T01:18:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.833+0000 7fc784faa700 1 --2- 172.21.15.44:0/1338652055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 0x55d14cdc3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:18:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.833+0000 7fc77ffff700 1 --2- 172.21.15.44:0/1338652055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4300 0x55d14cdc5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:18:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.833+0000 7fc77f7fe700 1 --2- 172.21.15.44:0/1338652055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4cc0 0x55d14cdc3960 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x55d14cdbf630 tx=0x7fc774009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549a14b24d22b07a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.833+0000 7fc75f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:18:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.833+0000 7fc75f7fe700 1 -- 172.21.15.44:0/1338652055 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77401c040 con 0x55d14cdb4cc0 2023-09-10T01:18:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.833+0000 7fc75f7fe700 1 -- 172.21.15.44:0/1338652055 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fc774000e40 con 0x55d14cdb4cc0 2023-09-10T01:18:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.833+0000 7fc792bd3b80 1 -- 172.21.15.44:0/1338652055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc770004e20 con 0x55d14cdb4cc0 2023-09-10T01:18:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.834+0000 7fc75f7fe700 1 -- 172.21.15.44:0/1338652055 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc77400bec0 con 0x55d14cdb4cc0 2023-09-10T01:18:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.834+0000 7fc792bd3b80 10 client.19716 fetch_fsmap learned FSMap version 125 2023-09-10T01:18:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.834+0000 7fc792bd3b80 1 -- 172.21.15.44:0/1338652055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=4+}) v3 -- 0x55d14cb31250 con 0x55d14cdb4cc0 2023-09-10T01:18:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.834+0000 7fc75f7fe700 1 -- 172.21.15.44:0/1338652055 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc770004e20 con 0x55d14cdb4cc0 2023-09-10T01:18:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.834+0000 7fc75f7fe700 1 -- 172.21.15.44:0/1338652055 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 125) v1 ==== 7331+0+0 (secure 0 0 0) 0x7fc774009c70 con 0x55d14cdb4cc0 2023-09-10T01:18:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.834+0000 7fc792bd3b80 10 client.19716 fetch_fsmap finished waiting for FSMap version 125 2023-09-10T01:18:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.834+0000 7fc792bd3b80 10 client.19716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:18:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.834+0000 7fc792bd3b80 20 client.19716 populate_metadata read hostname 'smithi044' 2023-09-10T01:18:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.834+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/1338652055 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d14ce23720 0x55d14ce25b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.834+0000 7fc792bd3b80 4 client.19716 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:18:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.834+0000 7fc792bd3b80 1 -- 172.21.15.44:0/1338652055 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d14ccff8c0 con 0x55d14ce23720 2023-09-10T01:18:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.834+0000 7fc77ffff700 1 --2- 172.21.15.44:0/1338652055 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d14ce23720 0x55d14ce25b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.835+0000 7fc75f7fe700 10 client.19716 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:18:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.835+0000 7fc77ffff700 1 --2- 172.21.15.44:0/1338652055 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d14ce23720 0x55d14ce25b40 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecc96bb18458bad4 server_cookie=61546ef85732a7c4 in_seq=0 out_seq=0 2023-09-10T01:18:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.857+0000 7fc75f7fe700 1 -- 172.21.15.44:0/1338652055 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d14ccff8c0 con 0x55d14ce23720 2023-09-10T01:18:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.857+0000 7fc75f7fe700 10 client.19716 handle_command_reply: tid=0 2023-09-10T01:18:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.857+0000 7fc792bd3b80 1 client.19716 shutdown 2023-09-10T01:18:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.857+0000 7fc792bd3b80 1 -- 172.21.15.44:0/1338652055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4cc0 msgr2=0x55d14cdc3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.857+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/1338652055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4cc0 0x55d14cdc3960 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x55d14cdbf630 tx=0x7fc774009fd0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.857+0000 7fc792bd3b80 1 -- 172.21.15.44:0/1338652055 shutdown_connections 2023-09-10T01:18:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.857+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/1338652055 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d14ce23720 0x55d14ce25b40 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.857+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/1338652055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d14cdba1a0 0x55d14cdc3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.857+0000 7fc75f7fe700 0 client.19716 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:18:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.857+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/1338652055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d14cdb4300 0x55d14cdc5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.857+0000 7fc792bd3b80 1 --2- 172.21.15.44:0/1338652055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d14cdb4cc0 0x55d14cdc3960 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.857+0000 7fc792bd3b80 1 -- 172.21.15.44:0/1338652055 >> 172.21.15.44:0/1338652055 conn(0x55d14cd29030 msgr2=0x55d14cdbc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.857+0000 7fc792bd3b80 1 -- 172.21.15.44:0/1338652055 shutdown_connections 2023-09-10T01:18:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.857+0000 7fc792bd3b80 1 -- 172.21.15.44:0/1338652055 wait complete. 2023-09-10T01:18:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.858+0000 7fc792bd3b80 20 client.19716 trim_cache size 0 max 16384 2023-09-10T01:18:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:18:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:18:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:18:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:18:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.858+0000 7fc786a0d700 1 -- 172.21.15.44:0/1003091844 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc7680a0830 msgr2=0x7fc7680a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.858+0000 7fc786a0d700 1 --2- 172.21.15.44:0/1003091844 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc7680a0830 0x7fc7680a2cf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc770005fd0 tx=0x7fc770009310 comp rx=0 tx=0).stop 2023-09-10T01:18:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.858+0000 7fc786a0d700 1 -- 172.21.15.44:0/1003091844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc78019cac0 msgr2=0x7fc78019cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.858+0000 7fc786a0d700 1 --2- 172.21.15.44:0/1003091844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc78019cac0 0x7fc78019cf70 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fc77800ef30 tx=0x7fc77800c3a0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.858+0000 7fc786a0d700 1 -- 172.21.15.44:0/1003091844 shutdown_connections 2023-09-10T01:18:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.858+0000 7fc786a0d700 1 --2- 172.21.15.44:0/1003091844 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc7680a0830 0x7fc7680a2cf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.858+0000 7fc786a0d700 1 --2- 172.21.15.44:0/1003091844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc78019cac0 0x7fc78019cf70 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.858+0000 7fc786a0d700 1 --2- 172.21.15.44:0/1003091844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7801054b0 0x7fc7801a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.858+0000 7fc786a0d700 1 --2- 172.21.15.44:0/1003091844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc780105920 0x7fc7801a31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.858+0000 7fc786a0d700 1 -- 172.21.15.44:0/1003091844 >> 172.21.15.44:0/1003091844 conn(0x7fc780074f30 msgr2=0x7fc780101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.858+0000 7fc786a0d700 1 -- 172.21.15.44:0/1003091844 shutdown_connections 2023-09-10T01:18:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:22.858+0000 7fc786a0d700 1 -- 172.21.15.44:0/1003091844 wait complete. 2023-09-10T01:18:24.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:24 smithi131 ceph-mon[103713]: pgmap v1214: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 7.2 MiB/s wr, 1.10k op/s 2023-09-10T01:18:24.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:24 smithi044 ceph-mon[98676]: pgmap v1214: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 7.2 MiB/s wr, 1.10k op/s 2023-09-10T01:18:24.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:24 smithi099 ceph-mon[102692]: pgmap v1214: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 7.2 MiB/s wr, 1.10k op/s 2023-09-10T01:18:26.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:26 smithi131 ceph-mon[103713]: pgmap v1215: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 6.9 MiB/s wr, 504 op/s 2023-09-10T01:18:26.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:26 smithi044 ceph-mon[98676]: pgmap v1215: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 6.9 MiB/s wr, 504 op/s 2023-09-10T01:18:26.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:26 smithi099 ceph-mon[102692]: pgmap v1215: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 6.9 MiB/s wr, 504 op/s 2023-09-10T01:18:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:27 smithi131 ceph-mon[103713]: osdmap e164: 12 total, 12 up, 12 in 2023-09-10T01:18:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:27 smithi044 ceph-mon[98676]: osdmap e164: 12 total, 12 up, 12 in 2023-09-10T01:18:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:27 smithi099 ceph-mon[102692]: osdmap e164: 12 total, 12 up, 12 in 2023-09-10T01:18:28.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:28 smithi131 ceph-mon[103713]: pgmap v1217: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 MiB/s rd, 13 MiB/s wr, 1.15k op/s 2023-09-10T01:18:28.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:28 smithi044 ceph-mon[98676]: pgmap v1217: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 MiB/s rd, 13 MiB/s wr, 1.15k op/s 2023-09-10T01:18:28.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:28 smithi099 ceph-mon[102692]: pgmap v1217: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 MiB/s rd, 13 MiB/s wr, 1.15k op/s 2023-09-10T01:18:30.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:30 smithi131 ceph-mon[103713]: pgmap v1218: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 6.0 MiB/s wr, 628 op/s 2023-09-10T01:18:30.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:30 smithi044 ceph-mon[98676]: pgmap v1218: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 6.0 MiB/s wr, 628 op/s 2023-09-10T01:18:30.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:30 smithi099 ceph-mon[102692]: pgmap v1218: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 6.0 MiB/s wr, 628 op/s 2023-09-10T01:18:32.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:32 smithi131 ceph-mon[103713]: pgmap v1219: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s rd, 15 MiB/s wr, 875 op/s 2023-09-10T01:18:32.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:32 smithi044 ceph-mon[98676]: pgmap v1219: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s rd, 15 MiB/s wr, 875 op/s 2023-09-10T01:18:32.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:32 smithi099 ceph-mon[102692]: pgmap v1219: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s rd, 15 MiB/s wr, 875 op/s 2023-09-10T01:18:34.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:34 smithi131 ceph-mon[103713]: pgmap v1220: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 14 MiB/s wr, 804 op/s 2023-09-10T01:18:34.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:34 smithi044 ceph-mon[98676]: pgmap v1220: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 14 MiB/s wr, 804 op/s 2023-09-10T01:18:34.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:34 smithi099 ceph-mon[102692]: pgmap v1220: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 14 MiB/s wr, 804 op/s 2023-09-10T01:18:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:35.998+0000 7f947ee31700 1 -- 172.21.15.44:0/1665713467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94780ff4d0 msgr2=0x7f94780ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:35.998+0000 7f947ee31700 1 --2- 172.21.15.44:0/1665713467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94780ff4d0 0x7f94780ff950 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f94640099c0 tx=0x7f9464009cd0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:35.998+0000 7f947ee31700 1 -- 172.21.15.44:0/1665713467 shutdown_connections 2023-09-10T01:18:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:35.998+0000 7f947ee31700 1 --2- 172.21.15.44:0/1665713467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94780ffe90 0x7f947810c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:35.998+0000 7f947ee31700 1 --2- 172.21.15.44:0/1665713467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9478105590 0x7f9478105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:35.998+0000 7f947ee31700 1 --2- 172.21.15.44:0/1665713467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94780ff4d0 0x7f94780ff950 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:35.998+0000 7f947ee31700 1 -- 172.21.15.44:0/1665713467 >> 172.21.15.44:0/1665713467 conn(0x7f94780faf60 msgr2=0x7f94780fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:35.998+0000 7f947ee31700 1 -- 172.21.15.44:0/1665713467 shutdown_connections 2023-09-10T01:18:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:35.998+0000 7f947ee31700 1 -- 172.21.15.44:0/1665713467 wait complete. 2023-09-10T01:18:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:35.999+0000 7f947ee31700 1 Processor -- start 2023-09-10T01:18:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:35.999+0000 7f947ee31700 1 -- start start 2023-09-10T01:18:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f947ee31700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94780ff4d0 0x7f94781975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f947ee31700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94780ffe90 0x7f9478197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f947ee31700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9478105590 0x7f94781a2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f947ee31700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781a2a20 con 0x7f94780ffe90 2023-09-10T01:18:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f947ee31700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781a2b90 con 0x7f94780ff4d0 2023-09-10T01:18:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f947ee31700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781a2d50 con 0x7f9478105590 2023-09-10T01:18:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f9477fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94780ffe90 0x7f9478197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f947d3ce700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9478105590 0x7f94781a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f9477fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94780ffe90 0x7f9478197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58160/0 (socket says 172.21.15.44:58160) 2023-09-10T01:18:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f9477fff700 1 -- 172.21.15.44:0/146677492 learned_addr learned my addr 172.21.15.44:0/146677492 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f947cbcd700 1 --2- 172.21.15.44:0/146677492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94780ff4d0 0x7f94781975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f947d3ce700 1 -- 172.21.15.44:0/146677492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94780ff4d0 msgr2=0x7f94781975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f947d3ce700 1 --2- 172.21.15.44:0/146677492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94780ff4d0 0x7f94781975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f947d3ce700 1 -- 172.21.15.44:0/146677492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94780ffe90 msgr2=0x7f9478197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f947d3ce700 1 --2- 172.21.15.44:0/146677492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94780ffe90 0x7f9478197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f947d3ce700 1 -- 172.21.15.44:0/146677492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94640095f0 con 0x7f9478105590 2023-09-10T01:18:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f947cbcd700 1 --2- 172.21.15.44:0/146677492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94780ff4d0 0x7f94781975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:18:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f9477fff700 1 --2- 172.21.15.44:0/146677492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94780ffe90 0x7f9478197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:18:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.000+0000 7f947d3ce700 1 --2- 172.21.15.44:0/146677492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9478105590 0x7f94781a2450 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f947000efc0 tx=0x7f947000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.001+0000 7f9475ffb700 1 -- 172.21.15.44:0/146677492 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470018070 con 0x7f9478105590 2023-09-10T01:18:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.001+0000 7f947ee31700 1 -- 172.21.15.44:0/146677492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94781a2fd0 con 0x7f9478105590 2023-09-10T01:18:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.001+0000 7f9475ffb700 1 -- 172.21.15.44:0/146677492 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f947000f040 con 0x7f9478105590 2023-09-10T01:18:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.001+0000 7f9475ffb700 1 -- 172.21.15.44:0/146677492 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947000cc50 con 0x7f9478105590 2023-09-10T01:18:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.001+0000 7f947ee31700 1 -- 172.21.15.44:0/146677492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94781a34b0 con 0x7f9478105590 2023-09-10T01:18:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.004+0000 7f9475ffb700 1 -- 172.21.15.44:0/146677492 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f9470003a00 con 0x7f9478105590 2023-09-10T01:18:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.004+0000 7f9475ffb700 1 --2- 172.21.15.44:0/146677492 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f94680a07b0 0x7f94680a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.005+0000 7f9475ffb700 1 -- 172.21.15.44:0/146677492 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 9480+0+0 (secure 0 0 0) 0x7f94700cbf00 con 0x7f9478105590 2023-09-10T01:18:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.005+0000 7f948aff7b80 1 Processor -- start 2023-09-10T01:18:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.005+0000 7f948aff7b80 1 -- start start 2023-09-10T01:18:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.005+0000 7f947cbcd700 1 --2- 172.21.15.44:0/146677492 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f94680a07b0 0x7f94680a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.005+0000 7f948aff7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ea990 0x5601515ead70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.005+0000 7f948aff7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 0x5601515dd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.005+0000 7f948aff7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ddcc0 0x5601515f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.005+0000 7f948aff7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601515366d0 con 0x5601515ddcc0 2023-09-10T01:18:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.005+0000 7f948aff7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560151533ff0 con 0x5601515ea990 2023-09-10T01:18:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.005+0000 7f948aff7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56015152f8e0 con 0x5601515dd300 2023-09-10T01:18:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.005+0000 7f947d3ce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ddcc0 0x5601515f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.005+0000 7f947d3ce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ddcc0 0x5601515f26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58168/0 (socket says 172.21.15.44:58168) 2023-09-10T01:18:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.005+0000 7f947d3ce700 1 -- 172.21.15.44:0/4065934748 learned_addr learned my addr 172.21.15.44:0/4065934748 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.005+0000 7f947cbcd700 1 --2- 172.21.15.44:0/4065934748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ea990 0x5601515ead70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.005+0000 7f9477fff700 1 --2- 172.21.15.44:0/4065934748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 0x5601515dd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.006+0000 7f947cbcd700 1 --2- 172.21.15.44:0/146677492 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f94680a07b0 0x7f94680a2c70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f946c009ce0 tx=0x7f946c009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.006+0000 7f947d3ce700 1 -- 172.21.15.44:0/4065934748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 msgr2=0x5601515dd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.006+0000 7f947d3ce700 1 --2- 172.21.15.44:0/4065934748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 0x5601515dd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.006+0000 7f947d3ce700 1 -- 172.21.15.44:0/4065934748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ea990 msgr2=0x5601515ead70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.006+0000 7f947d3ce700 1 --2- 172.21.15.44:0/4065934748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ea990 0x5601515ead70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.006+0000 7f947d3ce700 1 -- 172.21.15.44:0/4065934748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56015135bd80 con 0x5601515ddcc0 2023-09-10T01:18:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.006+0000 7f947cbcd700 1 --2- 172.21.15.44:0/4065934748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ea990 0x5601515ead70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:18:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.006+0000 7f947d3ce700 1 --2- 172.21.15.44:0/4065934748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ddcc0 0x5601515f26d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f9470095af0 tx=0x7f9470098f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408e84af9719720c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.007+0000 7f9462ffd700 1 -- 172.21.15.44:0/4065934748 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947009c030 con 0x5601515ddcc0 2023-09-10T01:18:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.007+0000 7f9462ffd700 1 -- 172.21.15.44:0/4065934748 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f94700092e0 con 0x5601515ddcc0 2023-09-10T01:18:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.007+0000 7f9462ffd700 1 -- 172.21.15.44:0/4065934748 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947009ceb0 con 0x5601515ddcc0 2023-09-10T01:18:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.008+0000 7f948aff7b80 1 -- 172.21.15.44:0/4065934748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ddcc0 msgr2=0x5601515f26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.008+0000 7f948aff7b80 1 --2- 172.21.15.44:0/4065934748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ddcc0 0x5601515f26d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f9470095af0 tx=0x7f9470098f20 comp rx=0 tx=0).stop 2023-09-10T01:18:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.008+0000 7f948aff7b80 1 -- 172.21.15.44:0/4065934748 shutdown_connections 2023-09-10T01:18:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.008+0000 7f948aff7b80 1 --2- 172.21.15.44:0/4065934748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ddcc0 0x5601515f26d0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.008+0000 7f948aff7b80 1 --2- 172.21.15.44:0/4065934748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ea990 0x5601515ead70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.008+0000 7f948aff7b80 1 --2- 172.21.15.44:0/4065934748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 0x5601515dd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.008+0000 7f948aff7b80 1 -- 172.21.15.44:0/4065934748 >> 172.21.15.44:0/4065934748 conn(0x560151552030 msgr2=0x5601515d8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.008+0000 7f948aff7b80 1 -- 172.21.15.44:0/4065934748 shutdown_connections 2023-09-10T01:18:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.008+0000 7f948aff7b80 1 -- 172.21.15.44:0/4065934748 wait complete. 2023-09-10T01:18:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.009+0000 7f948aff7b80 1 -- start start 2023-09-10T01:18:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.009+0000 7f948aff7b80 1 Processor -- start 2023-09-10T01:18:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f948aff7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 0x5601515f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f948aff7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ddcc0 0x5601515ef8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f948aff7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ea990 0x5601515efff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f948aff7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601514cdb10 con 0x5601515ddcc0 2023-09-10T01:18:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f948aff7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601514cd810 con 0x5601515ea990 2023-09-10T01:18:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f948aff7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601513b18c0 con 0x5601515dd300 2023-09-10T01:18:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f948aff7b80 10 client.0 _ll_register_callbacks cb 0x560151417a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:18:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f947cbcd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 0x5601515f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f947cbcd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 0x5601515f26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36822/0 (socket says 172.21.15.44:36822) 2023-09-10T01:18:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f947cbcd700 1 -- 172.21.15.44:0/1992792968 learned_addr learned my addr 172.21.15.44:0/1992792968 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f9477fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ddcc0 0x5601515ef8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f947d3ce700 1 --2- 172.21.15.44:0/1992792968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ea990 0x5601515efff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f947cbcd700 1 -- 172.21.15.44:0/1992792968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ea990 msgr2=0x5601515efff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f947cbcd700 1 --2- 172.21.15.44:0/1992792968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ea990 0x5601515efff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f947cbcd700 1 -- 172.21.15.44:0/1992792968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ddcc0 msgr2=0x5601515ef8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f947cbcd700 1 --2- 172.21.15.44:0/1992792968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ddcc0 0x5601515ef8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.010+0000 7f947cbcd700 1 -- 172.21.15.44:0/1992792968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c010c50 con 0x5601515dd300 2023-09-10T01:18:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.011+0000 7f947cbcd700 1 --2- 172.21.15.44:0/1992792968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 0x5601515f26a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f946c011c20 tx=0x7f946c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ece2d2d48b77740c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.011+0000 7f9457fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:18:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.011+0000 7f9457fff700 1 -- 172.21.15.44:0/1992792968 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c022070 con 0x5601515dd300 2023-09-10T01:18:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.011+0000 7f9457fff700 1 -- 172.21.15.44:0/1992792968 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(164..164 src has 1..164) v4 ==== 9480+0+0 (secure 0 0 0) 0x7f946c017050 con 0x5601515dd300 2023-09-10T01:18:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.011+0000 7f948aff7b80 1 -- 172.21.15.44:0/1992792968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560151487490 con 0x5601515dd300 2023-09-10T01:18:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.011+0000 7f9457fff700 1 -- 172.21.15.44:0/1992792968 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f946c010e80 con 0x5601515dd300 2023-09-10T01:18:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.011+0000 7f948aff7b80 10 client.29332 fetch_fsmap learned FSMap version 125 2023-09-10T01:18:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.011+0000 7f948aff7b80 1 -- 172.21.15.44:0/1992792968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=4+}) v3 -- 0x7f94781a34b0 con 0x5601515dd300 2023-09-10T01:18:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.011+0000 7f9457fff700 1 -- 172.21.15.44:0/1992792968 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560151487490 con 0x5601515dd300 2023-09-10T01:18:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.011+0000 7f9457fff700 1 -- 172.21.15.44:0/1992792968 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 125) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f946c027bf0 con 0x5601515dd300 2023-09-10T01:18:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.011+0000 7f948aff7b80 10 client.29332 fetch_fsmap finished waiting for FSMap version 125 2023-09-10T01:18:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.011+0000 7f948aff7b80 10 client.29332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:18:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.012+0000 7f948aff7b80 20 client.29332 populate_metadata read hostname 'smithi044' 2023-09-10T01:18:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.012+0000 7f948aff7b80 1 --2- 172.21.15.44:0/1992792968 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5601515ffd20 0x560151602140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.012+0000 7f948aff7b80 4 client.29332 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:18:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.012+0000 7f948aff7b80 1 -- 172.21.15.44:0/1992792968 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560151544840 con 0x5601515ffd20 2023-09-10T01:18:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.012+0000 7f9477fff700 1 --2- 172.21.15.44:0/1992792968 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5601515ffd20 0x560151602140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.012+0000 7f9457fff700 10 client.29332 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:18:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.012+0000 7f9477fff700 1 --2- 172.21.15.44:0/1992792968 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5601515ffd20 0x560151602140 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35b6539a537cf48e server_cookie=44029583fb431084 in_seq=0 out_seq=0 2023-09-10T01:18:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.013+0000 7f9457fff700 1 -- 172.21.15.44:0/1992792968 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560151544840 con 0x5601515ffd20 2023-09-10T01:18:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.013+0000 7f9457fff700 10 client.29332 handle_command_reply: tid=0 2023-09-10T01:18:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.013+0000 7f948aff7b80 1 client.29332 shutdown 2023-09-10T01:18:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.013+0000 7f948aff7b80 1 -- 172.21.15.44:0/1992792968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 msgr2=0x5601515f26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.013+0000 7f948aff7b80 1 --2- 172.21.15.44:0/1992792968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 0x5601515f26a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f946c011c20 tx=0x7f946c0047c0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.013+0000 7f948aff7b80 1 -- 172.21.15.44:0/1992792968 shutdown_connections 2023-09-10T01:18:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.013+0000 7f948aff7b80 1 --2- 172.21.15.44:0/1992792968 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5601515ffd20 0x560151602140 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.013+0000 7f948aff7b80 1 --2- 172.21.15.44:0/1992792968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ea990 0x5601515efff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.013+0000 7f948aff7b80 1 --2- 172.21.15.44:0/1992792968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 0x5601515f26a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.013+0000 7f948aff7b80 1 --2- 172.21.15.44:0/1992792968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ddcc0 0x5601515ef8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.013+0000 7f948aff7b80 1 -- 172.21.15.44:0/1992792968 >> 172.21.15.44:0/1992792968 conn(0x560151552030 msgr2=0x5601515d77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.013+0000 7f9457fff700 0 client.29332 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:18:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.013+0000 7f948aff7b80 1 -- 172.21.15.44:0/1992792968 shutdown_connections 2023-09-10T01:18:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.013+0000 7f948aff7b80 1 -- 172.21.15.44:0/1992792968 wait complete. 2023-09-10T01:18:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.014+0000 7f948aff7b80 20 client.29332 trim_cache size 0 max 16384 2023-09-10T01:18:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.036+0000 7f948aff7b80 1 Processor -- start 2023-09-10T01:18:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.036+0000 7f948aff7b80 1 -- start start 2023-09-10T01:18:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.036+0000 7f948aff7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515dd300 0x5601515f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.036+0000 7f948aff7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515ddcc0 0x5601515dca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.036+0000 7f948aff7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ea990 0x5601515d7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.036+0000 7f948aff7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601515d9930 con 0x5601515ea990 2023-09-10T01:18:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.036+0000 7f948aff7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601515d28a0 con 0x5601515dd300 2023-09-10T01:18:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.036+0000 7f948aff7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601515fd230 con 0x5601515ddcc0 2023-09-10T01:18:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.036+0000 7f947d3ce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ea990 0x5601515d7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.036+0000 7f947d3ce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ea990 0x5601515d7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58182/0 (socket says 172.21.15.44:58182) 2023-09-10T01:18:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.036+0000 7f947d3ce700 1 -- 172.21.15.44:0/161409235 learned_addr learned my addr 172.21.15.44:0/161409235 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.036+0000 7f947cbcd700 1 --2- 172.21.15.44:0/161409235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515dd300 0x5601515f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.036+0000 7f947d3ce700 1 -- 172.21.15.44:0/161409235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515ddcc0 msgr2=0x5601515dca00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:18:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.036+0000 7f947d3ce700 1 --2- 172.21.15.44:0/161409235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515ddcc0 0x5601515dca00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.037+0000 7f947d3ce700 1 -- 172.21.15.44:0/161409235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515dd300 msgr2=0x5601515f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.037+0000 7f947d3ce700 1 --2- 172.21.15.44:0/161409235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515dd300 0x5601515f51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.037+0000 7f947d3ce700 1 -- 172.21.15.44:0/161409235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560151344c00 con 0x5601515ea990 2023-09-10T01:18:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.037+0000 7f947cbcd700 1 --2- 172.21.15.44:0/161409235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515dd300 0x5601515f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:18:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.037+0000 7f947d3ce700 1 --2- 172.21.15.44:0/161409235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ea990 0x5601515d7f20 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f94700c9340 tx=0x7f9470009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5100387f86faed0a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.037+0000 7f94637fe700 1 -- 172.21.15.44:0/161409235 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947009c040 con 0x5601515ea990 2023-09-10T01:18:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.037+0000 7f94637fe700 1 -- 172.21.15.44:0/161409235 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f947001b320 con 0x5601515ea990 2023-09-10T01:18:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.037+0000 7f94637fe700 1 -- 172.21.15.44:0/161409235 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470099080 con 0x5601515ea990 2023-09-10T01:18:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.038+0000 7f948aff7b80 1 -- 172.21.15.44:0/161409235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ea990 msgr2=0x5601515d7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.038+0000 7f948aff7b80 1 --2- 172.21.15.44:0/161409235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ea990 0x5601515d7f20 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f94700c9340 tx=0x7f9470009f90 comp rx=0 tx=0).stop 2023-09-10T01:18:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.038+0000 7f948aff7b80 1 -- 172.21.15.44:0/161409235 shutdown_connections 2023-09-10T01:18:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.038+0000 7f948aff7b80 1 --2- 172.21.15.44:0/161409235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ea990 0x5601515d7f20 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.038+0000 7f948aff7b80 1 --2- 172.21.15.44:0/161409235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515dd300 0x5601515f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.038+0000 7f948aff7b80 1 --2- 172.21.15.44:0/161409235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515ddcc0 0x5601515dca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.038+0000 7f948aff7b80 1 -- 172.21.15.44:0/161409235 >> 172.21.15.44:0/161409235 conn(0x560151552030 msgr2=0x5601515dc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.038+0000 7f948aff7b80 1 -- 172.21.15.44:0/161409235 shutdown_connections 2023-09-10T01:18:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.038+0000 7f948aff7b80 1 -- 172.21.15.44:0/161409235 wait complete. 2023-09-10T01:18:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.038+0000 7f948aff7b80 1 -- start start 2023-09-10T01:18:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.039+0000 7f948aff7b80 1 Processor -- start 2023-09-10T01:18:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.039+0000 7f948aff7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 0x56015164c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.039+0000 7f948aff7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ddcc0 0x56015164cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.039+0000 7f948aff7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ea990 0x5601515f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.039+0000 7f948aff7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601515f4db0 con 0x5601515ea990 2023-09-10T01:18:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.039+0000 7f948aff7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601515f4f20 con 0x5601515ddcc0 2023-09-10T01:18:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.039+0000 7f948aff7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601515f50e0 con 0x5601515dd300 2023-09-10T01:18:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.039+0000 7f948aff7b80 10 client.0 _ll_register_callbacks cb 0x7f946c0117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:18:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.039+0000 7f947cbcd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 0x56015164c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.039+0000 7f947cbcd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 0x56015164c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36834/0 (socket says 172.21.15.44:36834) 2023-09-10T01:18:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.039+0000 7f947cbcd700 1 -- 172.21.15.44:0/3711263161 learned_addr learned my addr 172.21.15.44:0/3711263161 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.039+0000 7f947d3ce700 1 --2- 172.21.15.44:0/3711263161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ea990 0x5601515f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.040+0000 7f9477fff700 1 --2- 172.21.15.44:0/3711263161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ddcc0 0x56015164cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.040+0000 7f947cbcd700 1 -- 172.21.15.44:0/3711263161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ddcc0 msgr2=0x56015164cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.040+0000 7f947cbcd700 1 --2- 172.21.15.44:0/3711263161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ddcc0 0x56015164cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.040+0000 7f947cbcd700 1 -- 172.21.15.44:0/3711263161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ea990 msgr2=0x5601515f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.040+0000 7f947cbcd700 1 --2- 172.21.15.44:0/3711263161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ea990 0x5601515f47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.040+0000 7f947cbcd700 1 -- 172.21.15.44:0/3711263161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56015135bd80 con 0x5601515dd300 2023-09-10T01:18:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.040+0000 7f9477fff700 1 --2- 172.21.15.44:0/3711263161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ddcc0 0x56015164cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:18:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.040+0000 7f947cbcd700 1 --2- 172.21.15.44:0/3711263161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 0x56015164c660 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f946c026530 tx=0x7f946c026560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5bcabef1488ce49 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.040+0000 7f9457fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:18:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.040+0000 7f9457fff700 1 -- 172.21.15.44:0/3711263161 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c022040 con 0x5601515dd300 2023-09-10T01:18:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.040+0000 7f9457fff700 1 -- 172.21.15.44:0/3711263161 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(164..164 src has 1..164) v4 ==== 9480+0+0 (secure 0 0 0) 0x7f946c017040 con 0x5601515dd300 2023-09-10T01:18:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.040+0000 7f948aff7b80 1 -- 172.21.15.44:0/3711263161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94700092e0 con 0x5601515dd300 2023-09-10T01:18:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.041+0000 7f9457fff700 1 -- 172.21.15.44:0/3711263161 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f946c010a40 con 0x5601515dd300 2023-09-10T01:18:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.041+0000 7f948aff7b80 10 client.29335 fetch_fsmap learned FSMap version 125 2023-09-10T01:18:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.041+0000 7f948aff7b80 1 -- 172.21.15.44:0/3711263161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=4+}) v3 -- 0x5601513821f0 con 0x5601515dd300 2023-09-10T01:18:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.041+0000 7f9457fff700 1 -- 172.21.15.44:0/3711263161 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f94700092e0 con 0x5601515dd300 2023-09-10T01:18:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.041+0000 7f9457fff700 1 -- 172.21.15.44:0/3711263161 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 125) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f946c035440 con 0x5601515dd300 2023-09-10T01:18:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.041+0000 7f948aff7b80 10 client.29335 fetch_fsmap finished waiting for FSMap version 125 2023-09-10T01:18:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.041+0000 7f948aff7b80 10 client.29335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:18:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.041+0000 7f948aff7b80 20 client.29335 populate_metadata read hostname 'smithi044' 2023-09-10T01:18:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.041+0000 7f948aff7b80 1 --2- 172.21.15.44:0/3711263161 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5601516545a0 0x5601516569c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.041+0000 7f948aff7b80 4 client.29335 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:18:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.041+0000 7f948aff7b80 1 -- 172.21.15.44:0/3711263161 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5601515288c0 con 0x5601516545a0 2023-09-10T01:18:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.041+0000 7f9477fff700 1 --2- 172.21.15.44:0/3711263161 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5601516545a0 0x5601516569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.042+0000 7f9457fff700 10 client.29335 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:18:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.042+0000 7f9477fff700 1 --2- 172.21.15.44:0/3711263161 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5601516545a0 0x5601516569c0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=477aab786f385c7 server_cookie=38d44e42116b22ab in_seq=0 out_seq=0 2023-09-10T01:18:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.050+0000 7f9457fff700 1 -- 172.21.15.44:0/3711263161 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5601515288c0 con 0x5601516545a0 2023-09-10T01:18:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.050+0000 7f9457fff700 10 client.29335 handle_command_reply: tid=0 2023-09-10T01:18:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.050+0000 7f948aff7b80 1 client.29335 shutdown 2023-09-10T01:18:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.050+0000 7f948aff7b80 1 -- 172.21.15.44:0/3711263161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 msgr2=0x56015164c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.050+0000 7f948aff7b80 1 --2- 172.21.15.44:0/3711263161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 0x56015164c660 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f946c026530 tx=0x7f946c026560 comp rx=0 tx=0).stop 2023-09-10T01:18:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.050+0000 7f948aff7b80 1 -- 172.21.15.44:0/3711263161 shutdown_connections 2023-09-10T01:18:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.050+0000 7f948aff7b80 1 --2- 172.21.15.44:0/3711263161 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5601516545a0 0x5601516569c0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.050+0000 7f948aff7b80 1 --2- 172.21.15.44:0/3711263161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5601515ea990 0x5601515f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.050+0000 7f9457fff700 0 client.29335 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:18:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.050+0000 7f948aff7b80 1 --2- 172.21.15.44:0/3711263161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5601515dd300 0x56015164c660 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.050+0000 7f948aff7b80 1 --2- 172.21.15.44:0/3711263161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601515ddcc0 0x56015164cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.050+0000 7f948aff7b80 1 -- 172.21.15.44:0/3711263161 >> 172.21.15.44:0/3711263161 conn(0x560151552030 msgr2=0x5601515dcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:36.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.050+0000 7f948aff7b80 1 -- 172.21.15.44:0/3711263161 shutdown_connections 2023-09-10T01:18:36.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.050+0000 7f948aff7b80 1 -- 172.21.15.44:0/3711263161 wait complete. 2023-09-10T01:18:36.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.050+0000 7f948aff7b80 20 client.29335 trim_cache size 0 max 16384 2023-09-10T01:18:36.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:18:36.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:18:36.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:18:36.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:18:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.051+0000 7f947ee31700 1 -- 172.21.15.44:0/146677492 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f94680a07b0 msgr2=0x7f94680a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.051+0000 7f947ee31700 1 --2- 172.21.15.44:0/146677492 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f94680a07b0 0x7f94680a2c70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f946c009ce0 tx=0x7f946c009260 comp rx=0 tx=0).stop 2023-09-10T01:18:36.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.051+0000 7f947ee31700 1 -- 172.21.15.44:0/146677492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9478105590 msgr2=0x7f94781a2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:36.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.051+0000 7f947ee31700 1 --2- 172.21.15.44:0/146677492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9478105590 0x7f94781a2450 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f947000efc0 tx=0x7f947000c3a0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.051+0000 7f947ee31700 1 -- 172.21.15.44:0/146677492 shutdown_connections 2023-09-10T01:18:36.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.051+0000 7f947ee31700 1 --2- 172.21.15.44:0/146677492 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f94680a07b0 0x7f94680a2c70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.051+0000 7f947ee31700 1 --2- 172.21.15.44:0/146677492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9478105590 0x7f94781a2450 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.051+0000 7f947ee31700 1 --2- 172.21.15.44:0/146677492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94780ff4d0 0x7f94781975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.051+0000 7f947ee31700 1 --2- 172.21.15.44:0/146677492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94780ffe90 0x7f9478197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.051+0000 7f947ee31700 1 -- 172.21.15.44:0/146677492 >> 172.21.15.44:0/146677492 conn(0x7f94780faf60 msgr2=0x7f94780fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.051+0000 7f947ee31700 1 -- 172.21.15.44:0/146677492 shutdown_connections 2023-09-10T01:18:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:36.051+0000 7f947ee31700 1 -- 172.21.15.44:0/146677492 wait complete. 2023-09-10T01:18:36.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:36 smithi131 ceph-mon[103713]: pgmap v1221: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 14 MiB/s wr, 804 op/s 2023-09-10T01:18:36.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:36 smithi044 ceph-mon[98676]: pgmap v1221: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 14 MiB/s wr, 804 op/s 2023-09-10T01:18:36.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:36 smithi099 ceph-mon[102692]: pgmap v1221: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 14 MiB/s wr, 804 op/s 2023-09-10T01:18:38.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:38 smithi131 ceph-mon[103713]: pgmap v1222: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 22 MiB/s wr, 380 op/s 2023-09-10T01:18:38.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:38 smithi044 ceph-mon[98676]: pgmap v1222: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 22 MiB/s wr, 380 op/s 2023-09-10T01:18:38.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:38 smithi099 ceph-mon[102692]: pgmap v1222: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 22 MiB/s wr, 380 op/s 2023-09-10T01:18:40.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:40 smithi131 ceph-mon[103713]: pgmap v1223: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 18 MiB/s wr, 323 op/s 2023-09-10T01:18:40.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:40 smithi044 ceph-mon[98676]: pgmap v1223: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 18 MiB/s wr, 323 op/s 2023-09-10T01:18:40.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:40 smithi099 ceph-mon[102692]: pgmap v1223: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 18 MiB/s wr, 323 op/s 2023-09-10T01:18:42.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:42 smithi131 ceph-mon[103713]: pgmap v1224: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 24 MiB/s wr, 607 op/s 2023-09-10T01:18:42.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:42 smithi044 ceph-mon[98676]: pgmap v1224: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 24 MiB/s wr, 607 op/s 2023-09-10T01:18:42.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:42 smithi099 ceph-mon[102692]: pgmap v1224: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 24 MiB/s wr, 607 op/s 2023-09-10T01:18:44.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:44 smithi044 ceph-mon[98676]: pgmap v1225: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 17 MiB/s wr, 356 op/s 2023-09-10T01:18:44.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:44 smithi099 ceph-mon[102692]: pgmap v1225: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 17 MiB/s wr, 356 op/s 2023-09-10T01:18:44.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:44 smithi131 ceph-mon[103713]: pgmap v1225: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 17 MiB/s wr, 356 op/s 2023-09-10T01:18:46.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:46 smithi044 ceph-mon[98676]: pgmap v1226: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 17 MiB/s wr, 356 op/s 2023-09-10T01:18:46.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:46 smithi099 ceph-mon[102692]: pgmap v1226: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 17 MiB/s wr, 356 op/s 2023-09-10T01:18:46.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:46 smithi131 ceph-mon[103713]: pgmap v1226: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 17 MiB/s wr, 356 op/s 2023-09-10T01:18:48.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:48 smithi044 ceph-mon[98676]: pgmap v1227: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 22 MiB/s wr, 728 op/s 2023-09-10T01:18:48.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:48 smithi099 ceph-mon[102692]: pgmap v1227: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 22 MiB/s wr, 728 op/s 2023-09-10T01:18:48.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:48 smithi131 ceph-mon[103713]: pgmap v1227: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 22 MiB/s wr, 728 op/s 2023-09-10T01:18:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.231+0000 7f81f4ac8700 1 -- 172.21.15.44:0/1783640366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81f00ff760 msgr2=0x7f81f00ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.231+0000 7f81f4ac8700 1 --2- 172.21.15.44:0/1783640366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81f00ff760 0x7f81f00ffbe0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f81e40099c0 tx=0x7f81e4009cd0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.231+0000 7f81f4ac8700 1 -- 172.21.15.44:0/1783640366 shutdown_connections 2023-09-10T01:18:49.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.231+0000 7f81f4ac8700 1 --2- 172.21.15.44:0/1783640366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f0100120 0x7f81f010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.231+0000 7f81f4ac8700 1 --2- 172.21.15.44:0/1783640366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81f01055b0 0x7f81f0105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.231+0000 7f81f4ac8700 1 --2- 172.21.15.44:0/1783640366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81f00ff760 0x7f81f00ffbe0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.231+0000 7f81f4ac8700 1 -- 172.21.15.44:0/1783640366 >> 172.21.15.44:0/1783640366 conn(0x7f81f0074f30 msgr2=0x7f81f0075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:49.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.231+0000 7f81f4ac8700 1 -- 172.21.15.44:0/1783640366 shutdown_connections 2023-09-10T01:18:49.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.231+0000 7f81f4ac8700 1 -- 172.21.15.44:0/1783640366 wait complete. 2023-09-10T01:18:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.232+0000 7f81f4ac8700 1 Processor -- start 2023-09-10T01:18:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.232+0000 7f81f4ac8700 1 -- start start 2023-09-10T01:18:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.232+0000 7f81f4ac8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f00ff760 0x7f81f0195400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.232+0000 7f81f4ac8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81f0100120 0x7f81f0195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.232+0000 7f81f4ac8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81f01055b0 0x7f81f0199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.232+0000 7f81f4ac8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f019a5f0 con 0x7f81f01055b0 2023-09-10T01:18:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.232+0000 7f81f4ac8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f019a760 con 0x7f81f0100120 2023-09-10T01:18:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.232+0000 7f81f4ac8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f019a920 con 0x7f81f00ff760 2023-09-10T01:18:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.232+0000 7f81eed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81f01055b0 0x7f81f0199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.232+0000 7f81eed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81f01055b0 0x7f81f0199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58200/0 (socket says 172.21.15.44:58200) 2023-09-10T01:18:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.232+0000 7f81eed9d700 1 -- 172.21.15.44:0/2404834749 learned_addr learned my addr 172.21.15.44:0/2404834749 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.232+0000 7f81edd9b700 1 --2- 172.21.15.44:0/2404834749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81f0100120 0x7f81f0195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.232+0000 7f81ee59c700 1 --2- 172.21.15.44:0/2404834749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f00ff760 0x7f81f0195400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.233+0000 7f81eed9d700 1 -- 172.21.15.44:0/2404834749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f00ff760 msgr2=0x7f81f0195400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.233+0000 7f81eed9d700 1 --2- 172.21.15.44:0/2404834749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f00ff760 0x7f81f0195400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.233+0000 7f81eed9d700 1 -- 172.21.15.44:0/2404834749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81f0100120 msgr2=0x7f81f0195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.233+0000 7f81eed9d700 1 --2- 172.21.15.44:0/2404834749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81f0100120 0x7f81f0195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.233+0000 7f81eed9d700 1 -- 172.21.15.44:0/2404834749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e40095f0 con 0x7f81f01055b0 2023-09-10T01:18:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.233+0000 7f81edd9b700 1 --2- 172.21.15.44:0/2404834749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81f0100120 0x7f81f0195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:18:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.233+0000 7f81eed9d700 1 --2- 172.21.15.44:0/2404834749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81f01055b0 0x7f81f0199fb0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f81e800ef30 tx=0x7f81e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.233+0000 7f81df7fe700 1 -- 172.21.15.44:0/2404834749 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e8018070 con 0x7f81f01055b0 2023-09-10T01:18:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.233+0000 7f81df7fe700 1 -- 172.21.15.44:0/2404834749 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f81e800f040 con 0x7f81f01055b0 2023-09-10T01:18:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.233+0000 7f81f4ac8700 1 -- 172.21.15.44:0/2404834749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f019aba0 con 0x7f81f01055b0 2023-09-10T01:18:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.233+0000 7f81df7fe700 1 -- 172.21.15.44:0/2404834749 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e800cbc0 con 0x7f81f01055b0 2023-09-10T01:18:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.233+0000 7f81f4ac8700 1 -- 172.21.15.44:0/2404834749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f01a2440 con 0x7f81f01055b0 2023-09-10T01:18:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.236+0000 7f81df7fe700 1 -- 172.21.15.44:0/2404834749 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f81e8007500 con 0x7f81f01055b0 2023-09-10T01:18:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.237+0000 7f81df7fe700 1 --2- 172.21.15.44:0/2404834749 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f81d80a07b0 0x7f81d80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.237+0000 7f8200c8eb80 1 Processor -- start 2023-09-10T01:18:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.237+0000 7f81df7fe700 1 -- 172.21.15.44:0/2404834749 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 9480+0+0 (secure 0 0 0) 0x7f81e80cbe70 con 0x7f81f01055b0 2023-09-10T01:18:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.237+0000 7f81ee59c700 1 --2- 172.21.15.44:0/2404834749 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f81d80a07b0 0x7f81d80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.237+0000 7f8200c8eb80 1 -- start start 2023-09-10T01:18:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.237+0000 7f8200c8eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915300 0x55d89b9156e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.237+0000 7f8200c8eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915c20 0x55d89b923980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.237+0000 7f8200c8eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b923ec0 0x55d89b9262b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.237+0000 7f8200c8eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89b86e6d0 con 0x55d89b915300 2023-09-10T01:18:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.237+0000 7f8200c8eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89b86bff0 con 0x55d89b923ec0 2023-09-10T01:18:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.237+0000 7f8200c8eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89b8678e0 con 0x55d89b915c20 2023-09-10T01:18:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.238+0000 7f81ee59c700 1 --2- 172.21.15.44:0/2404834749 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f81d80a07b0 0x7f81d80a2c70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009c60 tx=0x7f81e0009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.238+0000 7f81ee59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915300 0x55d89b9156e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.238+0000 7f81ee59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915300 0x55d89b9156e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58204/0 (socket says 172.21.15.44:58204) 2023-09-10T01:18:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.238+0000 7f81ee59c700 1 -- 172.21.15.44:0/3962291976 learned_addr learned my addr 172.21.15.44:0/3962291976 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.238+0000 7f81edd9b700 1 --2- 172.21.15.44:0/3962291976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915c20 0x55d89b923980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.238+0000 7f81eed9d700 1 --2- 172.21.15.44:0/3962291976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b923ec0 0x55d89b9262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.238+0000 7f81ee59c700 1 -- 172.21.15.44:0/3962291976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915c20 msgr2=0x55d89b923980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.238+0000 7f81ee59c700 1 --2- 172.21.15.44:0/3962291976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915c20 0x55d89b923980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.238+0000 7f81ee59c700 1 -- 172.21.15.44:0/3962291976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b923ec0 msgr2=0x55d89b9262b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.238+0000 7f81ee59c700 1 --2- 172.21.15.44:0/3962291976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b923ec0 0x55d89b9262b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.238+0000 7f81ee59c700 1 -- 172.21.15.44:0/3962291976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d89b693d80 con 0x55d89b915300 2023-09-10T01:18:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.238+0000 7f81eed9d700 1 --2- 172.21.15.44:0/3962291976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b923ec0 0x55d89b9262b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:18:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.238+0000 7f81ee59c700 1 --2- 172.21.15.44:0/3962291976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915300 0x55d89b9156e0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f81e000e4e0 tx=0x7f81e0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda2e9238e6854a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.238+0000 7f81dcff9700 1 -- 172.21.15.44:0/3962291976 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e000ee80 con 0x55d89b915300 2023-09-10T01:18:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.238+0000 7f81dcff9700 1 -- 172.21.15.44:0/3962291976 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f81e0021070 con 0x55d89b915300 2023-09-10T01:18:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.238+0000 7f81dcff9700 1 -- 172.21.15.44:0/3962291976 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e0025610 con 0x55d89b915300 2023-09-10T01:18:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.239+0000 7f8200c8eb80 1 -- 172.21.15.44:0/3962291976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915300 msgr2=0x55d89b9156e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.239+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/3962291976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915300 0x55d89b9156e0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f81e000e4e0 tx=0x7f81e0014000 comp rx=0 tx=0).stop 2023-09-10T01:18:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.239+0000 7f8200c8eb80 1 -- 172.21.15.44:0/3962291976 shutdown_connections 2023-09-10T01:18:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.239+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/3962291976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b923ec0 0x55d89b9262b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.239+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/3962291976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915300 0x55d89b9156e0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.239+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/3962291976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915c20 0x55d89b923980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.239+0000 7f8200c8eb80 1 -- 172.21.15.44:0/3962291976 >> 172.21.15.44:0/3962291976 conn(0x55d89b88a030 msgr2=0x55d89b910560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.239+0000 7f8200c8eb80 1 -- 172.21.15.44:0/3962291976 shutdown_connections 2023-09-10T01:18:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.239+0000 7f8200c8eb80 1 -- 172.21.15.44:0/3962291976 wait complete. 2023-09-10T01:18:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.240+0000 7f8200c8eb80 1 -- start start 2023-09-10T01:18:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.240+0000 7f8200c8eb80 1 Processor -- start 2023-09-10T01:18:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.240+0000 7f8200c8eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915300 0x55d89b923830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.240+0000 7f8200c8eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b915c20 0x55d89b920a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.240+0000 7f8200c8eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b923ec0 0x55d89b920fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.240+0000 7f8200c8eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89b805b10 con 0x55d89b923ec0 2023-09-10T01:18:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.240+0000 7f8200c8eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89b805810 con 0x55d89b915c20 2023-09-10T01:18:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.240+0000 7f8200c8eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89b6e98c0 con 0x55d89b915300 2023-09-10T01:18:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.240+0000 7f8200c8eb80 10 client.0 _ll_register_callbacks cb 0x55d89b74fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:18:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.240+0000 7f81eed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b923ec0 0x55d89b920fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.240+0000 7f81ee59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915300 0x55d89b923830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.241+0000 7f81eed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b923ec0 0x55d89b920fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58212/0 (socket says 172.21.15.44:58212) 2023-09-10T01:18:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.241+0000 7f81eed9d700 1 -- 172.21.15.44:0/3961283395 learned_addr learned my addr 172.21.15.44:0/3961283395 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.241+0000 7f81edd9b700 1 --2- 172.21.15.44:0/3961283395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b915c20 0x55d89b920a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.241+0000 7f81ee59c700 1 -- 172.21.15.44:0/3961283395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b915c20 msgr2=0x55d89b920a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.241+0000 7f81ee59c700 1 --2- 172.21.15.44:0/3961283395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b915c20 0x55d89b920a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.241+0000 7f81ee59c700 1 -- 172.21.15.44:0/3961283395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b923ec0 msgr2=0x55d89b920fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.241+0000 7f81ee59c700 1 --2- 172.21.15.44:0/3961283395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b923ec0 0x55d89b920fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.241+0000 7f81ee59c700 1 -- 172.21.15.44:0/3961283395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e0017070 con 0x55d89b915300 2023-09-10T01:18:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.241+0000 7f81eed9d700 1 --2- 172.21.15.44:0/3961283395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b923ec0 0x55d89b920fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:18:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.241+0000 7f81ee59c700 1 --2- 172.21.15.44:0/3961283395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915300 0x55d89b923830 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f81e000e4b0 tx=0x7f81e000edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fad574513c3d277d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.241+0000 7f81d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:18:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.241+0000 7f81d5ffb700 1 -- 172.21.15.44:0/3961283395 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e00052d0 con 0x55d89b915300 2023-09-10T01:18:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.241+0000 7f81d5ffb700 1 -- 172.21.15.44:0/3961283395 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(164..164 src has 1..164) v4 ==== 9480+0+0 (secure 0 0 0) 0x7f81e001de40 con 0x55d89b915300 2023-09-10T01:18:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.241+0000 7f8200c8eb80 1 -- 172.21.15.44:0/3961283395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d89b7bf490 con 0x55d89b915300 2023-09-10T01:18:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.242+0000 7f81d5ffb700 1 -- 172.21.15.44:0/3961283395 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81e0005430 con 0x55d89b915300 2023-09-10T01:18:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.242+0000 7f8200c8eb80 10 client.29347 fetch_fsmap learned FSMap version 125 2023-09-10T01:18:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.242+0000 7f8200c8eb80 1 -- 172.21.15.44:0/3961283395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=4+}) v3 -- 0x55d89b67cc00 con 0x55d89b915300 2023-09-10T01:18:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.242+0000 7f81d5ffb700 1 -- 172.21.15.44:0/3961283395 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d89b7bf490 con 0x55d89b915300 2023-09-10T01:18:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.242+0000 7f81d5ffb700 1 -- 172.21.15.44:0/3961283395 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 125) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f81e0004980 con 0x55d89b915300 2023-09-10T01:18:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.242+0000 7f8200c8eb80 10 client.29347 fetch_fsmap finished waiting for FSMap version 125 2023-09-10T01:18:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.242+0000 7f8200c8eb80 10 client.29347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:18:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.242+0000 7f8200c8eb80 20 client.29347 populate_metadata read hostname 'smithi044' 2023-09-10T01:18:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.242+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/3961283395 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d89b933720 0x55d89b935b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.242+0000 7f8200c8eb80 4 client.29347 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:18:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.242+0000 7f8200c8eb80 1 -- 172.21.15.44:0/3961283395 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d89b87c840 con 0x55d89b933720 2023-09-10T01:18:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.242+0000 7f81edd9b700 1 --2- 172.21.15.44:0/3961283395 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d89b933720 0x55d89b935b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.243+0000 7f81d5ffb700 10 client.29347 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:18:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.243+0000 7f81edd9b700 1 --2- 172.21.15.44:0/3961283395 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d89b933720 0x55d89b935b40 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b5866f7644030cf server_cookie=84db7f54b99d5fbf in_seq=0 out_seq=0 2023-09-10T01:18:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.243+0000 7f81d5ffb700 1 -- 172.21.15.44:0/3961283395 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d89b87c840 con 0x55d89b933720 2023-09-10T01:18:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.243+0000 7f81d5ffb700 10 client.29347 handle_command_reply: tid=0 2023-09-10T01:18:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.243+0000 7f8200c8eb80 1 client.29347 shutdown 2023-09-10T01:18:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.243+0000 7f8200c8eb80 1 -- 172.21.15.44:0/3961283395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915300 msgr2=0x55d89b923830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.243+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/3961283395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915300 0x55d89b923830 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f81e000e4b0 tx=0x7f81e000edb0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.244+0000 7f8200c8eb80 1 -- 172.21.15.44:0/3961283395 shutdown_connections 2023-09-10T01:18:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.244+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/3961283395 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d89b933720 0x55d89b935b40 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.244+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/3961283395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b923ec0 0x55d89b920fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.244+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/3961283395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915300 0x55d89b923830 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.244+0000 7f81d5ffb700 0 client.29347 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:18:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.244+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/3961283395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b915c20 0x55d89b920a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.244+0000 7f8200c8eb80 1 -- 172.21.15.44:0/3961283395 >> 172.21.15.44:0/3961283395 conn(0x55d89b88a030 msgr2=0x55d89b90f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.244+0000 7f8200c8eb80 1 -- 172.21.15.44:0/3961283395 shutdown_connections 2023-09-10T01:18:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.244+0000 7f8200c8eb80 1 -- 172.21.15.44:0/3961283395 wait complete. 2023-09-10T01:18:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.244+0000 7f8200c8eb80 20 client.29347 trim_cache size 0 max 16384 2023-09-10T01:18:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.262+0000 7f8200c8eb80 1 Processor -- start 2023-09-10T01:18:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.262+0000 7f8200c8eb80 1 -- start start 2023-09-10T01:18:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.262+0000 7f8200c8eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b915300 0x55d89b920240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.262+0000 7f8200c8eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915c20 0x55d89b920780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.262+0000 7f8200c8eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b923ec0 0x55d89b90ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.262+0000 7f8200c8eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89b9219c0 con 0x55d89b915c20 2023-09-10T01:18:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.262+0000 7f8200c8eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89b91efe0 con 0x55d89b915300 2023-09-10T01:18:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.262+0000 7f8200c8eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89b91c8f0 con 0x55d89b923ec0 2023-09-10T01:18:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.262+0000 7f81edd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915c20 0x55d89b920780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.262+0000 7f81edd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915c20 0x55d89b920780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58218/0 (socket says 172.21.15.44:58218) 2023-09-10T01:18:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.262+0000 7f81edd9b700 1 -- 172.21.15.44:0/1397432558 learned_addr learned my addr 172.21.15.44:0/1397432558 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.262+0000 7f81eed9d700 1 --2- 172.21.15.44:0/1397432558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b923ec0 0x55d89b90ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.262+0000 7f81ee59c700 1 --2- 172.21.15.44:0/1397432558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b915300 0x55d89b920240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.263+0000 7f81edd9b700 1 -- 172.21.15.44:0/1397432558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b923ec0 msgr2=0x55d89b90ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.263+0000 7f81edd9b700 1 --2- 172.21.15.44:0/1397432558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b923ec0 0x55d89b90ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.263+0000 7f81edd9b700 1 -- 172.21.15.44:0/1397432558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b915300 msgr2=0x55d89b920240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.263+0000 7f81edd9b700 1 --2- 172.21.15.44:0/1397432558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b915300 0x55d89b920240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.263+0000 7f81edd9b700 1 -- 172.21.15.44:0/1397432558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d89b6ba1f0 con 0x55d89b915c20 2023-09-10T01:18:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.263+0000 7f81ee59c700 1 --2- 172.21.15.44:0/1397432558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b915300 0x55d89b920240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:18:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.263+0000 7f81edd9b700 1 --2- 172.21.15.44:0/1397432558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915c20 0x55d89b920780 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f81e40098e0 tx=0x7f81e401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab311047978e7e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.263+0000 7f81dd7fa700 1 -- 172.21.15.44:0/1397432558 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4027c70 con 0x55d89b915c20 2023-09-10T01:18:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.263+0000 7f81dd7fa700 1 -- 172.21.15.44:0/1397432558 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f81e401e040 con 0x55d89b915c20 2023-09-10T01:18:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.263+0000 7f81dd7fa700 1 -- 172.21.15.44:0/1397432558 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4005540 con 0x55d89b915c20 2023-09-10T01:18:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.264+0000 7f8200c8eb80 1 -- 172.21.15.44:0/1397432558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915c20 msgr2=0x55d89b920780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.264+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/1397432558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915c20 0x55d89b920780 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f81e40098e0 tx=0x7f81e401a040 comp rx=0 tx=0).stop 2023-09-10T01:18:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.264+0000 7f8200c8eb80 1 -- 172.21.15.44:0/1397432558 shutdown_connections 2023-09-10T01:18:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.264+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/1397432558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b923ec0 0x55d89b90ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.264+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/1397432558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b915300 0x55d89b920240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.264+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/1397432558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915c20 0x55d89b920780 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.264+0000 7f8200c8eb80 1 -- 172.21.15.44:0/1397432558 >> 172.21.15.44:0/1397432558 conn(0x55d89b88a030 msgr2=0x55d89b914b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.264+0000 7f8200c8eb80 1 -- 172.21.15.44:0/1397432558 shutdown_connections 2023-09-10T01:18:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.264+0000 7f8200c8eb80 1 -- 172.21.15.44:0/1397432558 wait complete. 2023-09-10T01:18:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.264+0000 7f8200c8eb80 1 -- start start 2023-09-10T01:18:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.264+0000 7f8200c8eb80 1 Processor -- start 2023-09-10T01:18:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f8200c8eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915300 0x55d89b928b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f8200c8eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915c20 0x55d89b929060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f8200c8eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b923ec0 0x55d89b986510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f8200c8eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89b9297f0 con 0x55d89b915300 2023-09-10T01:18:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f8200c8eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89b986a50 con 0x55d89b923ec0 2023-09-10T01:18:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f8200c8eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89b986bc0 con 0x55d89b915c20 2023-09-10T01:18:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f8200c8eb80 10 client.0 _ll_register_callbacks cb 0x55d89b74fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:18:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f81ee59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915300 0x55d89b928b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f81edd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915c20 0x55d89b929060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f81edd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915c20 0x55d89b929060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36874/0 (socket says 172.21.15.44:36874) 2023-09-10T01:18:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f81edd9b700 1 -- 172.21.15.44:0/427086202 learned_addr learned my addr 172.21.15.44:0/427086202 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:18:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f81ee59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915300 0x55d89b928b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58222/0 (socket says 172.21.15.44:58222) 2023-09-10T01:18:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f81eed9d700 1 --2- 172.21.15.44:0/427086202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b923ec0 0x55d89b986510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f81edd9b700 1 -- 172.21.15.44:0/427086202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b923ec0 msgr2=0x55d89b986510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f81edd9b700 1 --2- 172.21.15.44:0/427086202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b923ec0 0x55d89b986510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f81edd9b700 1 -- 172.21.15.44:0/427086202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915300 msgr2=0x55d89b928b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:18:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f81edd9b700 1 --2- 172.21.15.44:0/427086202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915300 0x55d89b928b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f81edd9b700 1 -- 172.21.15.44:0/427086202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d89b67cc00 con 0x55d89b915c20 2023-09-10T01:18:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.265+0000 7f81ee59c700 1 --2- 172.21.15.44:0/427086202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915300 0x55d89b928b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:18:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.266+0000 7f81edd9b700 1 --2- 172.21.15.44:0/427086202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915c20 0x55d89b929060 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f81e400a010 tx=0x7f81e4028a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=592c4c13f1bd9bd9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:18:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.266+0000 7f81d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:18:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.266+0000 7f81d5ffb700 1 -- 172.21.15.44:0/427086202 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e40053c0 con 0x55d89b915c20 2023-09-10T01:18:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.266+0000 7f81d5ffb700 1 -- 172.21.15.44:0/427086202 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(164..164 src has 1..164) v4 ==== 9480+0+0 (secure 0 0 0) 0x7f81e4005520 con 0x55d89b915c20 2023-09-10T01:18:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.266+0000 7f8200c8eb80 1 -- 172.21.15.44:0/427086202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81e0021070 con 0x55d89b915c20 2023-09-10T01:18:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.266+0000 7f81d5ffb700 1 -- 172.21.15.44:0/427086202 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81e40056e0 con 0x55d89b915c20 2023-09-10T01:18:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.266+0000 7f8200c8eb80 10 client.29353 fetch_fsmap learned FSMap version 125 2023-09-10T01:18:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.266+0000 7f8200c8eb80 1 -- 172.21.15.44:0/427086202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=4+}) v3 -- 0x7f81f010cbc0 con 0x55d89b915c20 2023-09-10T01:18:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.266+0000 7f81d5ffb700 1 -- 172.21.15.44:0/427086202 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f81e4043080 con 0x55d89b915c20 2023-09-10T01:18:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.266+0000 7f81d5ffb700 1 -- 172.21.15.44:0/427086202 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 125) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f81e4003be0 con 0x55d89b915c20 2023-09-10T01:18:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.266+0000 7f8200c8eb80 10 client.29353 fetch_fsmap finished waiting for FSMap version 125 2023-09-10T01:18:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.266+0000 7f8200c8eb80 10 client.29353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:18:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.266+0000 7f8200c8eb80 20 client.29353 populate_metadata read hostname 'smithi044' 2023-09-10T01:18:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.266+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/427086202 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d89b988010 0x55d89b98a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:18:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.266+0000 7f8200c8eb80 4 client.29353 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:18:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.266+0000 7f8200c8eb80 1 -- 172.21.15.44:0/427086202 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d89b8608c0 con 0x55d89b988010 2023-09-10T01:18:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.267+0000 7f81ee59c700 1 --2- 172.21.15.44:0/427086202 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d89b988010 0x55d89b98a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:18:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.267+0000 7f81d5ffb700 10 client.29353 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:18:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.267+0000 7f81ee59c700 1 --2- 172.21.15.44:0/427086202 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d89b988010 0x55d89b98a430 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=693263c631a0dd62 server_cookie=43e95650889f7107 in_seq=0 out_seq=0 2023-09-10T01:18:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.274+0000 7f81d5ffb700 1 -- 172.21.15.44:0/427086202 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d89b8608c0 con 0x55d89b988010 2023-09-10T01:18:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.274+0000 7f81d5ffb700 10 client.29353 handle_command_reply: tid=0 2023-09-10T01:18:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.275+0000 7f8200c8eb80 1 client.29353 shutdown 2023-09-10T01:18:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.275+0000 7f8200c8eb80 1 -- 172.21.15.44:0/427086202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915c20 msgr2=0x55d89b929060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.275+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/427086202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915c20 0x55d89b929060 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f81e400a010 tx=0x7f81e4028a80 comp rx=0 tx=0).stop 2023-09-10T01:18:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.275+0000 7f8200c8eb80 1 -- 172.21.15.44:0/427086202 shutdown_connections 2023-09-10T01:18:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.275+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/427086202 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d89b988010 0x55d89b98a430 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.275+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/427086202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d89b923ec0 0x55d89b986510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.275+0000 7f81d5ffb700 0 client.29353 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:18:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.275+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/427086202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d89b915300 0x55d89b928b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.275+0000 7f8200c8eb80 1 --2- 172.21.15.44:0/427086202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d89b915c20 0x55d89b929060 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.275+0000 7f8200c8eb80 1 -- 172.21.15.44:0/427086202 >> 172.21.15.44:0/427086202 conn(0x55d89b88a030 msgr2=0x55d89b92f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.275+0000 7f8200c8eb80 1 -- 172.21.15.44:0/427086202 shutdown_connections 2023-09-10T01:18:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.275+0000 7f8200c8eb80 1 -- 172.21.15.44:0/427086202 wait complete. 2023-09-10T01:18:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.275+0000 7f8200c8eb80 20 client.29353 trim_cache size 0 max 16384 2023-09-10T01:18:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:18:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:18:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:18:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:18:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.276+0000 7f81f4ac8700 1 -- 172.21.15.44:0/2404834749 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f81d80a07b0 msgr2=0x7f81d80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.276+0000 7f81f4ac8700 1 --2- 172.21.15.44:0/2404834749 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f81d80a07b0 0x7f81d80a2c70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009c60 tx=0x7f81e0009260 comp rx=0 tx=0).stop 2023-09-10T01:18:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.276+0000 7f81f4ac8700 1 -- 172.21.15.44:0/2404834749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81f01055b0 msgr2=0x7f81f0199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:18:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.276+0000 7f81f4ac8700 1 --2- 172.21.15.44:0/2404834749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81f01055b0 0x7f81f0199fb0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f81e800ef30 tx=0x7f81e800c3a0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.276+0000 7f81f4ac8700 1 -- 172.21.15.44:0/2404834749 shutdown_connections 2023-09-10T01:18:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.276+0000 7f81f4ac8700 1 --2- 172.21.15.44:0/2404834749 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f81d80a07b0 0x7f81d80a2c70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.276+0000 7f81f4ac8700 1 --2- 172.21.15.44:0/2404834749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81f01055b0 0x7f81f0199fb0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.276+0000 7f81f4ac8700 1 --2- 172.21.15.44:0/2404834749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f00ff760 0x7f81f0195400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.276+0000 7f81f4ac8700 1 --2- 172.21.15.44:0/2404834749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81f0100120 0x7f81f0195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:18:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.276+0000 7f81f4ac8700 1 -- 172.21.15.44:0/2404834749 >> 172.21.15.44:0/2404834749 conn(0x7f81f0074f30 msgr2=0x7f81f00fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:18:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.276+0000 7f81f4ac8700 1 -- 172.21.15.44:0/2404834749 shutdown_connections 2023-09-10T01:18:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:18:49.276+0000 7f81f4ac8700 1 -- 172.21.15.44:0/2404834749 wait complete. 2023-09-10T01:18:50.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:50 smithi044 ceph-mon[98676]: pgmap v1228: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 61 MiB/s rd, 11 MiB/s wr, 657 op/s 2023-09-10T01:18:50.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:50 smithi099 ceph-mon[102692]: pgmap v1228: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 61 MiB/s rd, 11 MiB/s wr, 657 op/s 2023-09-10T01:18:50.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:50 smithi131 ceph-mon[103713]: pgmap v1228: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 61 MiB/s rd, 11 MiB/s wr, 657 op/s 2023-09-10T01:18:52.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:52 smithi099 ceph-mon[102692]: pgmap v1229: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 17 MiB/s wr, 975 op/s 2023-09-10T01:18:52.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:52 smithi131 ceph-mon[103713]: pgmap v1229: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 17 MiB/s wr, 975 op/s 2023-09-10T01:18:52.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:52 smithi044 ceph-mon[98676]: pgmap v1229: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 17 MiB/s wr, 975 op/s 2023-09-10T01:18:54.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:54 smithi099 ceph-mon[102692]: pgmap v1230: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 11 MiB/s wr, 691 op/s 2023-09-10T01:18:54.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:54 smithi131 ceph-mon[103713]: pgmap v1230: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 11 MiB/s wr, 691 op/s 2023-09-10T01:18:54.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:54 smithi044 ceph-mon[98676]: pgmap v1230: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 11 MiB/s wr, 691 op/s 2023-09-10T01:18:56.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:56 smithi099 ceph-mon[102692]: pgmap v1231: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 11 MiB/s wr, 691 op/s 2023-09-10T01:18:56.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:56 smithi131 ceph-mon[103713]: pgmap v1231: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 11 MiB/s wr, 691 op/s 2023-09-10T01:18:56.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:56 smithi044 ceph-mon[98676]: pgmap v1231: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 11 MiB/s wr, 691 op/s 2023-09-10T01:18:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:57 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:18:57.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:57 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:18:57.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:57 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:18:58.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:58 smithi099 ceph-mon[102692]: pgmap v1232: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 23 MiB/s wr, 739 op/s 2023-09-10T01:18:58.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:58 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:18:58.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:58 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:18:58.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:58 smithi099 ceph-mon[102692]: pgmap v1233: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 20 MiB/s wr, 402 op/s 2023-09-10T01:18:58.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:58 smithi099 ceph-mon[102692]: pgmap v1234: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 24 MiB/s wr, 492 op/s 2023-09-10T01:18:58.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:58 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:18:58.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:58 smithi131 ceph-mon[103713]: pgmap v1232: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 23 MiB/s wr, 739 op/s 2023-09-10T01:18:58.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:58 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:18:58.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:58 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:18:58.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:58 smithi131 ceph-mon[103713]: pgmap v1233: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 20 MiB/s wr, 402 op/s 2023-09-10T01:18:58.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:58 smithi131 ceph-mon[103713]: pgmap v1234: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 24 MiB/s wr, 492 op/s 2023-09-10T01:18:58.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:58 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:18:58.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:58 smithi044 ceph-mon[98676]: pgmap v1232: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 23 MiB/s wr, 739 op/s 2023-09-10T01:18:58.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:58 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:18:58.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:58 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:18:58.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:58 smithi044 ceph-mon[98676]: pgmap v1233: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 20 MiB/s wr, 402 op/s 2023-09-10T01:18:58.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:58 smithi044 ceph-mon[98676]: pgmap v1234: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 24 MiB/s wr, 492 op/s 2023-09-10T01:18:58.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:58 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:18:59.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:18:59 smithi131 ceph-mon[103713]: pgmap v1235: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 16 MiB/s wr, 64 op/s 2023-09-10T01:18:59.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:18:59 smithi044 ceph-mon[98676]: pgmap v1235: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 16 MiB/s wr, 64 op/s 2023-09-10T01:18:59.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:18:59 smithi099 ceph-mon[102692]: pgmap v1235: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 16 MiB/s wr, 64 op/s 2023-09-10T01:19:02.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:02 smithi131 ceph-mon[103713]: pgmap v1236: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 31 MiB/s wr, 124 op/s 2023-09-10T01:19:02.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:02 smithi044 ceph-mon[98676]: pgmap v1236: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 31 MiB/s wr, 124 op/s 2023-09-10T01:19:02.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:02 smithi099 ceph-mon[102692]: pgmap v1236: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 31 MiB/s wr, 124 op/s 2023-09-10T01:19:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.613+0000 7f17092e1700 1 -- 172.21.15.44:0/1674241923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17041055b0 msgr2=0x7f1704105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.613+0000 7f17092e1700 1 --2- 172.21.15.44:0/1674241923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17041055b0 0x7f1704105990 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f16f40099c0 tx=0x7f16f4009cd0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.614+0000 7f17092e1700 1 -- 172.21.15.44:0/1674241923 shutdown_connections 2023-09-10T01:19:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.614+0000 7f17092e1700 1 --2- 172.21.15.44:0/1674241923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1704100120 0x7f170410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.614+0000 7f17092e1700 1 --2- 172.21.15.44:0/1674241923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17041055b0 0x7f1704105990 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.614+0000 7f17092e1700 1 --2- 172.21.15.44:0/1674241923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17040ff760 0x7f17040ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.614+0000 7f17092e1700 1 -- 172.21.15.44:0/1674241923 >> 172.21.15.44:0/1674241923 conn(0x7f1704074f30 msgr2=0x7f1704075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.614+0000 7f17092e1700 1 -- 172.21.15.44:0/1674241923 shutdown_connections 2023-09-10T01:19:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.614+0000 7f17092e1700 1 -- 172.21.15.44:0/1674241923 wait complete. 2023-09-10T01:19:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.614+0000 7f17092e1700 1 Processor -- start 2023-09-10T01:19:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f17092e1700 1 -- start start 2023-09-10T01:19:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f17092e1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17040ff760 0x7f1704197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f17092e1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1704100120 0x7f1704197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f17092e1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17041055b0 0x7f17041a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f17092e1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17041a29b0 con 0x7f1704100120 2023-09-10T01:19:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f17092e1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17041a2b20 con 0x7f17041055b0 2023-09-10T01:19:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f17092e1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17041a2ce0 con 0x7f17040ff760 2023-09-10T01:19:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f17027fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1704100120 0x7f1704197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f17027fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1704100120 0x7f1704197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58240/0 (socket says 172.21.15.44:58240) 2023-09-10T01:19:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f17027fc700 1 -- 172.21.15.44:0/2897360182 learned_addr learned my addr 172.21.15.44:0/2897360182 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f1702ffd700 1 --2- 172.21.15.44:0/2897360182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17040ff760 0x7f1704197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f17037fe700 1 --2- 172.21.15.44:0/2897360182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17041055b0 0x7f17041a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f17027fc700 1 -- 172.21.15.44:0/2897360182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17040ff760 msgr2=0x7f1704197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f17027fc700 1 --2- 172.21.15.44:0/2897360182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17040ff760 0x7f1704197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f17027fc700 1 -- 172.21.15.44:0/2897360182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17041055b0 msgr2=0x7f17041a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f17027fc700 1 --2- 172.21.15.44:0/2897360182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17041055b0 0x7f17041a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.615+0000 7f17027fc700 1 -- 172.21.15.44:0/2897360182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f40095f0 con 0x7f1704100120 2023-09-10T01:19:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.616+0000 7f17037fe700 1 --2- 172.21.15.44:0/2897360182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17041055b0 0x7f17041a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:19:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.616+0000 7f17027fc700 1 --2- 172.21.15.44:0/2897360182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1704100120 0x7f1704197a80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f16f800e9a0 tx=0x7f16f800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.616+0000 7f16ebfff700 1 -- 172.21.15.44:0/2897360182 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f8009790 con 0x7f1704100120 2023-09-10T01:19:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.616+0000 7f16ebfff700 1 -- 172.21.15.44:0/2897360182 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16f800cbd0 con 0x7f1704100120 2023-09-10T01:19:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.616+0000 7f16ebfff700 1 -- 172.21.15.44:0/2897360182 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f8018680 con 0x7f1704100120 2023-09-10T01:19:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.616+0000 7f17092e1700 1 -- 172.21.15.44:0/2897360182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17041951b0 con 0x7f1704100120 2023-09-10T01:19:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.616+0000 7f17092e1700 1 -- 172.21.15.44:0/2897360182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17041956c0 con 0x7f1704100120 2023-09-10T01:19:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.619+0000 7f16ebfff700 1 -- 172.21.15.44:0/2897360182 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f16f800cd40 con 0x7f1704100120 2023-09-10T01:19:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.619+0000 7f16ebfff700 1 --2- 172.21.15.44:0/2897360182 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f16ec0a0900 0x7f16ec0a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.619+0000 7f1702ffd700 1 --2- 172.21.15.44:0/2897360182 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f16ec0a0900 0x7f16ec0a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.620+0000 7f16ebfff700 1 -- 172.21.15.44:0/2897360182 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 9480+0+0 (secure 0 0 0) 0x7f16f8014070 con 0x7f1704100120 2023-09-10T01:19:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.620+0000 7f1702ffd700 1 --2- 172.21.15.44:0/2897360182 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f16ec0a0900 0x7f16ec0a2dc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f16f400b580 tx=0x7f16f401a040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.620+0000 7f17154a7b80 1 Processor -- start 2023-09-10T01:19:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.620+0000 7f17154a7b80 1 -- start start 2023-09-10T01:19:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.620+0000 7f17154a7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18af5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.620+0000 7f17154a7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aef300 0x55ed18aef780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.620+0000 7f17154a7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aefcc0 0x55ed18afbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f17154a7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed18a486d0 con 0x55ed18af51a0 2023-09-10T01:19:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f17154a7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed18a45ff0 con 0x55ed18aefcc0 2023-09-10T01:19:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f17154a7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed18a418e0 con 0x55ed18aef300 2023-09-10T01:19:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f1702ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18af5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f1702ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18af5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58246/0 (socket says 172.21.15.44:58246) 2023-09-10T01:19:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f17027fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aef300 0x55ed18aef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f1702ffd700 1 -- 172.21.15.44:0/704236433 learned_addr learned my addr 172.21.15.44:0/704236433 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f17037fe700 1 --2- 172.21.15.44:0/704236433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aefcc0 0x55ed18afbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f17027fc700 1 -- 172.21.15.44:0/704236433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aefcc0 msgr2=0x55ed18afbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f17027fc700 1 --2- 172.21.15.44:0/704236433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aefcc0 0x55ed18afbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f17027fc700 1 -- 172.21.15.44:0/704236433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 msgr2=0x55ed18af5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f17027fc700 1 --2- 172.21.15.44:0/704236433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18af5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f17027fc700 1 -- 172.21.15.44:0/704236433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed1886dd80 con 0x55ed18aef300 2023-09-10T01:19:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f1702ffd700 1 --2- 172.21.15.44:0/704236433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18af5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:19:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f17027fc700 1 --2- 172.21.15.44:0/704236433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aef300 0x55ed18aef780 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f16f80c9910 tx=0x7f16f8098fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8ec67b56d6bac6f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f16e97fa700 1 -- 172.21.15.44:0/704236433 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f809d4a0 con 0x55ed18aef300 2023-09-10T01:19:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f16e97fa700 1 -- 172.21.15.44:0/704236433 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16f80a16e0 con 0x55ed18aef300 2023-09-10T01:19:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.621+0000 7f16e97fa700 1 -- 172.21.15.44:0/704236433 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f8098b60 con 0x55ed18aef300 2023-09-10T01:19:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.622+0000 7f17154a7b80 1 -- 172.21.15.44:0/704236433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aef300 msgr2=0x55ed18aef780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.622+0000 7f17154a7b80 1 --2- 172.21.15.44:0/704236433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aef300 0x55ed18aef780 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f16f80c9910 tx=0x7f16f8098fd0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.622+0000 7f17154a7b80 1 -- 172.21.15.44:0/704236433 shutdown_connections 2023-09-10T01:19:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.622+0000 7f17154a7b80 1 --2- 172.21.15.44:0/704236433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aefcc0 0x55ed18afbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.622+0000 7f17154a7b80 1 --2- 172.21.15.44:0/704236433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18af5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.622+0000 7f17154a7b80 1 --2- 172.21.15.44:0/704236433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aef300 0x55ed18aef780 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.622+0000 7f17154a7b80 1 -- 172.21.15.44:0/704236433 >> 172.21.15.44:0/704236433 conn(0x55ed18a64030 msgr2=0x55ed18aea560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.622+0000 7f17154a7b80 1 -- 172.21.15.44:0/704236433 shutdown_connections 2023-09-10T01:19:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.622+0000 7f17154a7b80 1 -- 172.21.15.44:0/704236433 wait complete. 2023-09-10T01:19:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.623+0000 7f17154a7b80 1 -- start start 2023-09-10T01:19:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.623+0000 7f17154a7b80 1 Processor -- start 2023-09-10T01:19:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.623+0000 7f17154a7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aef300 0x55ed18afbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.623+0000 7f17154a7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aefcc0 0x55ed18af9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.623+0000 7f17154a7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18af97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.623+0000 7f17154a7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed189dfb10 con 0x55ed18af51a0 2023-09-10T01:19:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.623+0000 7f17154a7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed189df810 con 0x55ed18aef300 2023-09-10T01:19:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.623+0000 7f17154a7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed188c38c0 con 0x55ed18aefcc0 2023-09-10T01:19:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.623+0000 7f17154a7b80 10 client.0 _ll_register_callbacks cb 0x55ed18929a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:19:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.623+0000 7f17037fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18af97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.623+0000 7f17037fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18af97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58254/0 (socket says 172.21.15.44:58254) 2023-09-10T01:19:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.623+0000 7f17037fe700 1 -- 172.21.15.44:0/2893165748 learned_addr learned my addr 172.21.15.44:0/2893165748 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.624+0000 7f1702ffd700 1 --2- 172.21.15.44:0/2893165748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aef300 0x55ed18afbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.624+0000 7f17037fe700 1 -- 172.21.15.44:0/2893165748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aefcc0 msgr2=0x55ed18af9150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:19:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.624+0000 7f17037fe700 1 --2- 172.21.15.44:0/2893165748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aefcc0 0x55ed18af9150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.624+0000 7f17037fe700 1 -- 172.21.15.44:0/2893165748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aef300 msgr2=0x55ed18afbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.624+0000 7f17037fe700 1 --2- 172.21.15.44:0/2893165748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aef300 0x55ed18afbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.624+0000 7f17037fe700 1 -- 172.21.15.44:0/2893165748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170410cbc0 con 0x55ed18af51a0 2023-09-10T01:19:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.624+0000 7f1702ffd700 1 --2- 172.21.15.44:0/2893165748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aef300 0x55ed18afbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:19:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.624+0000 7f17037fe700 1 --2- 172.21.15.44:0/2893165748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18af97d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f16fc007aa0 tx=0x7f16fc007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0265beacc2338ee server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.624+0000 7f16e6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:19:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.624+0000 7f16e6ff5700 1 -- 172.21.15.44:0/2893165748 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc012b60 con 0x55ed18af51a0 2023-09-10T01:19:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.624+0000 7f16e6ff5700 1 -- 172.21.15.44:0/2893165748 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(164..164 src has 1..164) v4 ==== 9480+0+0 (secure 0 0 0) 0x7f16fc012cc0 con 0x55ed18af51a0 2023-09-10T01:19:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.624+0000 7f17154a7b80 1 -- 172.21.15.44:0/2893165748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed18999490 con 0x55ed18af51a0 2023-09-10T01:19:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.624+0000 7f16e6ff5700 1 -- 172.21.15.44:0/2893165748 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16fc012e80 con 0x55ed18af51a0 2023-09-10T01:19:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.624+0000 7f17154a7b80 10 client.19764 fetch_fsmap learned FSMap version 125 2023-09-10T01:19:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.624+0000 7f17154a7b80 1 -- 172.21.15.44:0/2893165748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=4+}) v3 -- 0x7f17041956c0 con 0x55ed18af51a0 2023-09-10T01:19:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.625+0000 7f16e6ff5700 1 -- 172.21.15.44:0/2893165748 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ed18999490 con 0x55ed18af51a0 2023-09-10T01:19:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.625+0000 7f16e6ff5700 1 -- 172.21.15.44:0/2893165748 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 125) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f16fc014040 con 0x55ed18af51a0 2023-09-10T01:19:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.625+0000 7f17154a7b80 10 client.19764 fetch_fsmap finished waiting for FSMap version 125 2023-09-10T01:19:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.625+0000 7f17154a7b80 10 client.19764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:19:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.625+0000 7f17154a7b80 20 client.19764 populate_metadata read hostname 'smithi044' 2023-09-10T01:19:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.625+0000 7f17154a7b80 1 --2- 172.21.15.44:0/2893165748 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ed18b095c0 0x55ed18b0b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.625+0000 7f17154a7b80 4 client.19764 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:19:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.625+0000 7f17154a7b80 1 -- 172.21.15.44:0/2893165748 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed18a56840 con 0x55ed18b095c0 2023-09-10T01:19:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.625+0000 7f1702ffd700 1 --2- 172.21.15.44:0/2893165748 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ed18b095c0 0x55ed18b0b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.626+0000 7f16e6ff5700 10 client.19764 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.626+0000 7f1702ffd700 1 --2- 172.21.15.44:0/2893165748 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ed18b095c0 0x55ed18b0b9e0 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a5b49f0753b265c server_cookie=af138df59d4e6a3d in_seq=0 out_seq=0 2023-09-10T01:19:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.627+0000 7f16e6ff5700 1 -- 172.21.15.44:0/2893165748 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ed18a56840 con 0x55ed18b095c0 2023-09-10T01:19:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.627+0000 7f16e6ff5700 10 client.19764 handle_command_reply: tid=0 2023-09-10T01:19:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.627+0000 7f17154a7b80 1 client.19764 shutdown 2023-09-10T01:19:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.627+0000 7f17154a7b80 1 -- 172.21.15.44:0/2893165748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 msgr2=0x55ed18af97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.627+0000 7f17154a7b80 1 --2- 172.21.15.44:0/2893165748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18af97d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f16fc007aa0 tx=0x7f16fc007e60 comp rx=0 tx=0).stop 2023-09-10T01:19:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.627+0000 7f17154a7b80 1 -- 172.21.15.44:0/2893165748 shutdown_connections 2023-09-10T01:19:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.627+0000 7f17154a7b80 1 --2- 172.21.15.44:0/2893165748 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ed18b095c0 0x55ed18b0b9e0 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.627+0000 7f17154a7b80 1 --2- 172.21.15.44:0/2893165748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18af97d0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.627+0000 7f16e6ff5700 0 client.19764 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.627+0000 7f17154a7b80 1 --2- 172.21.15.44:0/2893165748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aef300 0x55ed18afbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.627+0000 7f17154a7b80 1 --2- 172.21.15.44:0/2893165748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aefcc0 0x55ed18af9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.627+0000 7f17154a7b80 1 -- 172.21.15.44:0/2893165748 >> 172.21.15.44:0/2893165748 conn(0x55ed18a64030 msgr2=0x55ed18ae97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.627+0000 7f17154a7b80 1 -- 172.21.15.44:0/2893165748 shutdown_connections 2023-09-10T01:19:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.627+0000 7f17154a7b80 1 -- 172.21.15.44:0/2893165748 wait complete. 2023-09-10T01:19:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.627+0000 7f17154a7b80 20 client.19764 trim_cache size 0 max 16384 2023-09-10T01:19:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f17154a7b80 1 Processor -- start 2023-09-10T01:19:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f17154a7b80 1 -- start start 2023-09-10T01:19:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f17154a7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aef300 0x55ed18b00530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f17154a7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aefcc0 0x55ed18aea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f17154a7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18aea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f17154a7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed18aeab10 con 0x55ed18af51a0 2023-09-10T01:19:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f17154a7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed18ae6ff0 con 0x55ed18aefcc0 2023-09-10T01:19:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f17154a7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed18af0580 con 0x55ed18aef300 2023-09-10T01:19:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f17037fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18aea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f1702ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aef300 0x55ed18b00530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f17037fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18aea5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58262/0 (socket says 172.21.15.44:58262) 2023-09-10T01:19:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f17037fe700 1 -- 172.21.15.44:0/1567656572 learned_addr learned my addr 172.21.15.44:0/1567656572 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f17027fc700 1 --2- 172.21.15.44:0/1567656572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aefcc0 0x55ed18aea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f1702ffd700 1 -- 172.21.15.44:0/1567656572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aefcc0 msgr2=0x55ed18aea090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f1702ffd700 1 --2- 172.21.15.44:0/1567656572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aefcc0 0x55ed18aea090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f1702ffd700 1 -- 172.21.15.44:0/1567656572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 msgr2=0x55ed18aea5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f1702ffd700 1 --2- 172.21.15.44:0/1567656572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18aea5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.645+0000 7f1702ffd700 1 -- 172.21.15.44:0/1567656572 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed18856c00 con 0x55ed18aef300 2023-09-10T01:19:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.646+0000 7f1702ffd700 1 --2- 172.21.15.44:0/1567656572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aef300 0x55ed18b00530 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55ed18b0c290 tx=0x7f16f401e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb465e0ea47cb138 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.646+0000 7f16e9ffb700 1 -- 172.21.15.44:0/1567656572 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f401d040 con 0x55ed18aef300 2023-09-10T01:19:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.646+0000 7f16e9ffb700 1 -- 172.21.15.44:0/1567656572 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16f4029040 con 0x55ed18aef300 2023-09-10T01:19:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.646+0000 7f16e9ffb700 1 -- 172.21.15.44:0/1567656572 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f400f820 con 0x55ed18aef300 2023-09-10T01:19:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.647+0000 7f17154a7b80 1 -- 172.21.15.44:0/1567656572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aef300 msgr2=0x55ed18b00530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.647+0000 7f17154a7b80 1 --2- 172.21.15.44:0/1567656572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aef300 0x55ed18b00530 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55ed18b0c290 tx=0x7f16f401e000 comp rx=0 tx=0).stop 2023-09-10T01:19:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.647+0000 7f17154a7b80 1 -- 172.21.15.44:0/1567656572 shutdown_connections 2023-09-10T01:19:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.647+0000 7f17154a7b80 1 --2- 172.21.15.44:0/1567656572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18aea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.647+0000 7f17154a7b80 1 --2- 172.21.15.44:0/1567656572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aef300 0x55ed18b00530 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.647+0000 7f17154a7b80 1 --2- 172.21.15.44:0/1567656572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aefcc0 0x55ed18aea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.647+0000 7f17154a7b80 1 -- 172.21.15.44:0/1567656572 >> 172.21.15.44:0/1567656572 conn(0x55ed18a64030 msgr2=0x55ed18af3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.647+0000 7f17154a7b80 1 -- 172.21.15.44:0/1567656572 shutdown_connections 2023-09-10T01:19:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.647+0000 7f17154a7b80 1 -- 172.21.15.44:0/1567656572 wait complete. 2023-09-10T01:19:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.647+0000 7f17154a7b80 1 -- start start 2023-09-10T01:19:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.647+0000 7f17154a7b80 1 Processor -- start 2023-09-10T01:19:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17154a7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aef300 0x55ed18b55820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17154a7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aefcc0 0x55ed18b55d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17154a7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18b51380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17154a7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed18b51980 con 0x55ed18af51a0 2023-09-10T01:19:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17154a7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed18b51af0 con 0x55ed18aef300 2023-09-10T01:19:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17154a7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed18b51cb0 con 0x55ed18aefcc0 2023-09-10T01:19:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17154a7b80 10 client.0 _ll_register_callbacks cb 0x7f16fc007470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:19:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17027fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aefcc0 0x55ed18b55d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17027fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aefcc0 0x55ed18b55d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:36916/0 (socket says 172.21.15.44:36916) 2023-09-10T01:19:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17027fc700 1 -- 172.21.15.44:0/4102564229 learned_addr learned my addr 172.21.15.44:0/4102564229 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17037fe700 1 --2- 172.21.15.44:0/4102564229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18b51380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f1702ffd700 1 --2- 172.21.15.44:0/4102564229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aef300 0x55ed18b55820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17027fc700 1 -- 172.21.15.44:0/4102564229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aef300 msgr2=0x55ed18b55820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17027fc700 1 --2- 172.21.15.44:0/4102564229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aef300 0x55ed18b55820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17027fc700 1 -- 172.21.15.44:0/4102564229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 msgr2=0x55ed18b51380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17027fc700 1 --2- 172.21.15.44:0/4102564229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18b51380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17027fc700 1 -- 172.21.15.44:0/4102564229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f400bc50 con 0x55ed18aefcc0 2023-09-10T01:19:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f1702ffd700 1 --2- 172.21.15.44:0/4102564229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aef300 0x55ed18b55820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:19:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17027fc700 1 --2- 172.21.15.44:0/4102564229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aefcc0 0x55ed18b55d60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f16f800c5b0 tx=0x7f16f8099440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e833d563a54c885 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f16e6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:19:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f16e6ff5700 1 -- 172.21.15.44:0/4102564229 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f809d4a0 con 0x55ed18aefcc0 2023-09-10T01:19:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f16e6ff5700 1 -- 172.21.15.44:0/4102564229 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(164..164 src has 1..164) v4 ==== 9480+0+0 (secure 0 0 0) 0x7f16f80998d0 con 0x55ed18aefcc0 2023-09-10T01:19:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.648+0000 7f17154a7b80 1 -- 172.21.15.44:0/4102564229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16f80a16e0 con 0x55ed18aefcc0 2023-09-10T01:19:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.649+0000 7f16e6ff5700 1 -- 172.21.15.44:0/4102564229 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16f8099a90 con 0x55ed18aefcc0 2023-09-10T01:19:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.649+0000 7f17154a7b80 10 client.29368 fetch_fsmap learned FSMap version 125 2023-09-10T01:19:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.649+0000 7f17154a7b80 1 -- 172.21.15.44:0/4102564229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=4+}) v3 -- 0x55ed188941f0 con 0x55ed18aefcc0 2023-09-10T01:19:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.649+0000 7f16e6ff5700 1 -- 172.21.15.44:0/4102564229 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16f80a16e0 con 0x55ed18aefcc0 2023-09-10T01:19:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.649+0000 7f16e6ff5700 1 -- 172.21.15.44:0/4102564229 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 125) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f16f80954f0 con 0x55ed18aefcc0 2023-09-10T01:19:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.649+0000 7f17154a7b80 10 client.29368 fetch_fsmap finished waiting for FSMap version 125 2023-09-10T01:19:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.649+0000 7f17154a7b80 10 client.29368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:19:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.649+0000 7f17154a7b80 20 client.29368 populate_metadata read hostname 'smithi044' 2023-09-10T01:19:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.649+0000 7f17154a7b80 1 --2- 172.21.15.44:0/4102564229 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ed18b5e420 0x55ed18b60840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.649+0000 7f17154a7b80 4 client.29368 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:19:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.649+0000 7f17154a7b80 1 -- 172.21.15.44:0/4102564229 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ed18a3a8c0 con 0x55ed18b5e420 2023-09-10T01:19:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.649+0000 7f1702ffd700 1 --2- 172.21.15.44:0/4102564229 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ed18b5e420 0x55ed18b60840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.650+0000 7f16e6ff5700 10 client.29368 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:02.650+0000 7f1702ffd700 1 --2- 172.21.15.44:0/4102564229 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ed18b5e420 0x55ed18b60840 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34605ddaf3e2eb54 server_cookie=9c9b5f54c6b479c1 in_seq=0 out_seq=0 2023-09-10T01:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.436+0000 7f16e6ff5700 1 -- 172.21.15.44:0/4102564229 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ed18a3a8c0 con 0x55ed18b5e420 2023-09-10T01:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.436+0000 7f16e6ff5700 10 client.29368 handle_command_reply: tid=0 2023-09-10T01:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.436+0000 7f17154a7b80 1 client.29368 shutdown 2023-09-10T01:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.436+0000 7f17154a7b80 1 -- 172.21.15.44:0/4102564229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aefcc0 msgr2=0x55ed18b55d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.436+0000 7f17154a7b80 1 --2- 172.21.15.44:0/4102564229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aefcc0 0x55ed18b55d60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f16f800c5b0 tx=0x7f16f8099440 comp rx=0 tx=0).stop 2023-09-10T01:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17154a7b80 1 -- 172.21.15.44:0/4102564229 shutdown_connections 2023-09-10T01:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17154a7b80 1 --2- 172.21.15.44:0/4102564229 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ed18b5e420 0x55ed18b60840 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17154a7b80 1 --2- 172.21.15.44:0/4102564229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed18af51a0 0x55ed18b51380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17154a7b80 1 --2- 172.21.15.44:0/4102564229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed18aef300 0x55ed18b55820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f16e6ff5700 0 client.29368 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17154a7b80 1 --2- 172.21.15.44:0/4102564229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed18aefcc0 0x55ed18b55d60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17154a7b80 1 -- 172.21.15.44:0/4102564229 >> 172.21.15.44:0/4102564229 conn(0x55ed18a64030 msgr2=0x55ed18af3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17154a7b80 1 -- 172.21.15.44:0/4102564229 shutdown_connections 2023-09-10T01:19:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17154a7b80 1 -- 172.21.15.44:0/4102564229 wait complete. 2023-09-10T01:19:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17154a7b80 20 client.29368 trim_cache size 0 max 16384 2023-09-10T01:19:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:19:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:19:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:19:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:19:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17092e1700 1 -- 172.21.15.44:0/2897360182 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f16ec0a0900 msgr2=0x7f16ec0a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17092e1700 1 --2- 172.21.15.44:0/2897360182 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f16ec0a0900 0x7f16ec0a2dc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f16f400b580 tx=0x7f16f401a040 comp rx=0 tx=0).stop 2023-09-10T01:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17092e1700 1 -- 172.21.15.44:0/2897360182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1704100120 msgr2=0x7f1704197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17092e1700 1 --2- 172.21.15.44:0/2897360182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1704100120 0x7f1704197a80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f16f800e9a0 tx=0x7f16f800ed60 comp rx=0 tx=0).stop 2023-09-10T01:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17092e1700 1 -- 172.21.15.44:0/2897360182 shutdown_connections 2023-09-10T01:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17092e1700 1 --2- 172.21.15.44:0/2897360182 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f16ec0a0900 0x7f16ec0a2dc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17092e1700 1 --2- 172.21.15.44:0/2897360182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17041055b0 0x7f17041a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17092e1700 1 --2- 172.21.15.44:0/2897360182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17040ff760 0x7f1704197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17092e1700 1 --2- 172.21.15.44:0/2897360182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1704100120 0x7f1704197a80 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.437+0000 7f17092e1700 1 -- 172.21.15.44:0/2897360182 >> 172.21.15.44:0/2897360182 conn(0x7f1704074f30 msgr2=0x7f17040fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.438+0000 7f17092e1700 1 -- 172.21.15.44:0/2897360182 shutdown_connections 2023-09-10T01:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:03.438+0000 7f17092e1700 1 -- 172.21.15.44:0/2897360182 wait complete. 2023-09-10T01:19:03.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:03 smithi131 ceph-mon[103713]: pgmap v1237: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 31 MiB/s wr, 124 op/s 2023-09-10T01:19:03.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:03 smithi044 ceph-mon[98676]: pgmap v1237: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 31 MiB/s wr, 124 op/s 2023-09-10T01:19:03.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:03 smithi099 ceph-mon[102692]: pgmap v1237: 129 pgs: 129 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 31 MiB/s wr, 124 op/s 2023-09-10T01:19:04.561 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: *** Caught signal (Segmentation fault) ** 2023-09-10T01:19:04.562 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: in thread 7fd74c4d2700 thread_name:md_log_replay 2023-09-10T01:19:04.562 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:19:04.562 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fd75d081cf0] 2023-09-10T01:19:04.562 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x55ef0e4e52f2] 2023-09-10T01:19:04.562 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55ef0e4e7543] 2023-09-10T01:19:04.563 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 4: (MDLog::_replay_thread()+0x647) [0x55ef0e550e97] 2023-09-10T01:19:04.563 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55ef0e55a381] 2023-09-10T01:19:04.563 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fd75d0771ca] 2023-09-10T01:19:04.563 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 7: clone() 2023-09-10T01:19:04.563 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 2023-09-10T01:19:04.230+0000 7fd74c4d2700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:19:04.564 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: in thread 7fd74c4d2700 thread_name:md_log_replay 2023-09-10T01:19:04.564 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 2023-09-10T01:19:04.564 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:19:04.564 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fd75d081cf0] 2023-09-10T01:19:04.564 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x55ef0e4e52f2] 2023-09-10T01:19:04.565 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55ef0e4e7543] 2023-09-10T01:19:04.565 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 4: (MDLog::_replay_thread()+0x647) [0x55ef0e550e97] 2023-09-10T01:19:04.565 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55ef0e55a381] 2023-09-10T01:19:04.565 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fd75d0771ca] 2023-09-10T01:19:04.565 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 7: clone() 2023-09-10T01:19:04.566 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:19:04.566 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 2023-09-10T01:19:04.566 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 0> 2023-09-10T01:19:04.230+0000 7fd74c4d2700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:19:04.566 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: in thread 7fd74c4d2700 thread_name:md_log_replay 2023-09-10T01:19:04.566 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 2023-09-10T01:19:04.567 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:19:04.567 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fd75d081cf0] 2023-09-10T01:19:04.567 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x55ef0e4e52f2] 2023-09-10T01:19:04.567 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55ef0e4e7543] 2023-09-10T01:19:04.567 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 4: (MDLog::_replay_thread()+0x647) [0x55ef0e550e97] 2023-09-10T01:19:04.568 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55ef0e55a381] 2023-09-10T01:19:04.568 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fd75d0771ca] 2023-09-10T01:19:04.568 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 7: clone() 2023-09-10T01:19:04.568 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:19:04.568 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 2023-09-10T01:19:04.568 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 0> 2023-09-10T01:19:04.230+0000 7fd74c4d2700 -1 *** Caught signal (Segmentation fault) ** 2023-09-10T01:19:04.569 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: in thread 7fd74c4d2700 thread_name:md_log_replay 2023-09-10T01:19:04.569 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 2023-09-10T01:19:04.569 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-10T01:19:04.569 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fd75d081cf0] 2023-09-10T01:19:04.569 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 2: /usr/bin/ceph-mds(+0x4fd2f2) [0x55ef0e4e52f2] 2023-09-10T01:19:04.570 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55ef0e4e7543] 2023-09-10T01:19:04.570 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 4: (MDLog::_replay_thread()+0x647) [0x55ef0e550e97] 2023-09-10T01:19:04.570 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55ef0e55a381] 2023-09-10T01:19:04.570 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fd75d0771ca] 2023-09-10T01:19:04.570 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 7: clone() 2023-09-10T01:19:04.570 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-09-10T01:19:04.571 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[197109]: 2023-09-10T01:19:05.071 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:05 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Main process exited, code=exited, status=139/n/a 2023-09-10T01:19:05.462 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:05 smithi044 ceph-mon[98676]: osdmap e165: 12 total, 12 up, 12 in 2023-09-10T01:19:05.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:05 smithi131 ceph-mon[103713]: osdmap e165: 12 total, 12 up, 12 in 2023-09-10T01:19:05.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:05 smithi099 ceph-mon[102692]: osdmap e165: 12 total, 12 up, 12 in 2023-09-10T01:19:05.805 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:05 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Failed with result 'exit-code'. 2023-09-10T01:19:07.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:07 smithi131 ceph-mon[103713]: pgmap v1239: 129 pgs: 7 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 114 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 30 MiB/s wr, 136 op/s 2023-09-10T01:19:07.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:07 smithi044 ceph-mon[98676]: pgmap v1239: 129 pgs: 7 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 114 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 30 MiB/s wr, 136 op/s 2023-09-10T01:19:07.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:07 smithi099 ceph-mon[102692]: pgmap v1239: 129 pgs: 7 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 114 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 30 MiB/s wr, 136 op/s 2023-09-10T01:19:09.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:09 smithi131 ceph-mon[103713]: pgmap v1240: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 25 MiB/s wr, 124 op/s 2023-09-10T01:19:09.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:09 smithi044 ceph-mon[98676]: pgmap v1240: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 25 MiB/s wr, 124 op/s 2023-09-10T01:19:09.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:09 smithi099 ceph-mon[102692]: pgmap v1240: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 25 MiB/s wr, 124 op/s 2023-09-10T01:19:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:11 smithi131 ceph-mon[103713]: pgmap v1241: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 25 MiB/s wr, 124 op/s 2023-09-10T01:19:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:11 smithi044 ceph-mon[98676]: pgmap v1241: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 25 MiB/s wr, 124 op/s 2023-09-10T01:19:11.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:11 smithi099 ceph-mon[102692]: pgmap v1241: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 25 MiB/s wr, 124 op/s 2023-09-10T01:19:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:13 smithi131 ceph-mon[103713]: pgmap v1242: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 143 op/s 2023-09-10T01:19:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:13 smithi044 ceph-mon[98676]: pgmap v1242: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 143 op/s 2023-09-10T01:19:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:13 smithi099 ceph-mon[102692]: pgmap v1242: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 143 op/s 2023-09-10T01:19:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:15 smithi131 ceph-mon[103713]: pgmap v1243: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 143 op/s 2023-09-10T01:19:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:15 smithi044 ceph-mon[98676]: pgmap v1243: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 143 op/s 2023-09-10T01:19:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:15 smithi099 ceph-mon[102692]: pgmap v1243: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 143 op/s 2023-09-10T01:19:15.946 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:15 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Service RestartSec=10s expired, scheduling restart. 2023-09-10T01:19:15.947 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:15 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service: Scheduled restart job, restart counter is at 5. 2023-09-10T01:19:15.947 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:15 smithi044 systemd[1]: Stopped Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:19:15.947 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:15 smithi044 systemd[1]: Starting Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:19:16.264 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:15 smithi044 podman[198049]: 2023-09-10T01:19:16.517 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:16 smithi044 bash[198049]: c5a67fb31ad7a6c2ff1c657a13a2844d99980f84dc8fe3aa907eab4905c6e8ab 2023-09-10T01:19:16.518 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:16 smithi044 systemd[1]: Started Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:19:16.518 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:16 smithi044 ceph-mds[198088]: -- 172.21.15.44:0/2679887972 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56159f2ab1e0 con 0x56159f281000 2023-09-10T01:19:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.572+0000 7ff1e1dfe700 1 -- 172.21.15.44:0/1840306957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1dc106650 msgr2=0x7ff1dc106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.572+0000 7ff1e1dfe700 1 --2- 172.21.15.44:0/1840306957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1dc106650 0x7ff1dc106a30 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0099c0 tx=0x7ff1cc009cd0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.572+0000 7ff1e1dfe700 1 -- 172.21.15.44:0/1840306957 shutdown_connections 2023-09-10T01:19:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.572+0000 7ff1e1dfe700 1 --2- 172.21.15.44:0/1840306957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1dc0683b0 0x7ff1dc10d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.572+0000 7ff1e1dfe700 1 --2- 172.21.15.44:0/1840306957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1dc106650 0x7ff1dc106a30 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.572+0000 7ff1e1dfe700 1 --2- 172.21.15.44:0/1840306957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1dc0679f0 0x7ff1dc067e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.572+0000 7ff1e1dfe700 1 -- 172.21.15.44:0/1840306957 >> 172.21.15.44:0/1840306957 conn(0x7ff1dc074f30 msgr2=0x7ff1dc075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.572+0000 7ff1e1dfe700 1 -- 172.21.15.44:0/1840306957 shutdown_connections 2023-09-10T01:19:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.572+0000 7ff1e1dfe700 1 -- 172.21.15.44:0/1840306957 wait complete. 2023-09-10T01:19:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.573+0000 7ff1e1dfe700 1 Processor -- start 2023-09-10T01:19:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.573+0000 7ff1e1dfe700 1 -- start start 2023-09-10T01:19:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.573+0000 7ff1e1dfe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1dc0679f0 0x7ff1dc19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.573+0000 7ff1e1dfe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1dc0683b0 0x7ff1dc19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.573+0000 7ff1e1dfe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1dc106650 0x7ff1dc196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.573+0000 7ff1e1dfe700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc196a70 con 0x7ff1dc0683b0 2023-09-10T01:19:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.573+0000 7ff1e1dfe700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc196be0 con 0x7ff1dc106650 2023-09-10T01:19:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.573+0000 7ff1e1dfe700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc196da0 con 0x7ff1dc0679f0 2023-09-10T01:19:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1dbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1dc106650 0x7ff1dc196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1daffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1dc0683b0 0x7ff1dc19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1dbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1dc106650 0x7ff1dc196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60026/0 (socket says 172.21.15.44:60026) 2023-09-10T01:19:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1daffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1dc0683b0 0x7ff1dc19caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58288/0 (socket says 172.21.15.44:58288) 2023-09-10T01:19:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1dbfff700 1 -- 172.21.15.44:0/3587447715 learned_addr learned my addr 172.21.15.44:0/3587447715 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1db7fe700 1 --2- 172.21.15.44:0/3587447715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1dc0679f0 0x7ff1dc19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1dbfff700 1 -- 172.21.15.44:0/3587447715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1dc0679f0 msgr2=0x7ff1dc19c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1dbfff700 1 --2- 172.21.15.44:0/3587447715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1dc0679f0 0x7ff1dc19c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1dbfff700 1 -- 172.21.15.44:0/3587447715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1dc0683b0 msgr2=0x7ff1dc19caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1dbfff700 1 --2- 172.21.15.44:0/3587447715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1dc0683b0 0x7ff1dc19caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1dbfff700 1 -- 172.21.15.44:0/3587447715 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc0095f0 con 0x7ff1dc106650 2023-09-10T01:19:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1dbfff700 1 --2- 172.21.15.44:0/3587447715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1dc106650 0x7ff1dc196410 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400ef30 tx=0x7ff1d400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1d8ff9700 1 -- 172.21.15.44:0/3587447715 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4018070 con 0x7ff1dc106650 2023-09-10T01:19:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1d8ff9700 1 -- 172.21.15.44:0/3587447715 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1d400f040 con 0x7ff1dc106650 2023-09-10T01:19:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1e1dfe700 1 -- 172.21.15.44:0/3587447715 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1dc197020 con 0x7ff1dc106650 2023-09-10T01:19:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1d8ff9700 1 -- 172.21.15.44:0/3587447715 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d400cbc0 con 0x7ff1dc106650 2023-09-10T01:19:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.574+0000 7ff1e1dfe700 1 -- 172.21.15.44:0/3587447715 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc1a35a0 con 0x7ff1dc106650 2023-09-10T01:19:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.577+0000 7ff1d8ff9700 1 -- 172.21.15.44:0/3587447715 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ff1d400cd80 con 0x7ff1dc106650 2023-09-10T01:19:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.577+0000 7ff1d8ff9700 1 --2- 172.21.15.44:0/3587447715 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1c40a0830 0x7ff1c40a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.577+0000 7ff1d8ff9700 1 -- 172.21.15.44:0/3587447715 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff1d40cbda0 con 0x7ff1dc106650 2023-09-10T01:19:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.577+0000 7ff1db7fe700 1 --2- 172.21.15.44:0/3587447715 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1c40a0830 0x7ff1c40a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.578+0000 7ff1db7fe700 1 --2- 172.21.15.44:0/3587447715 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1c40a0830 0x7ff1c40a2cf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc006010 tx=0x7ff1cc005c00 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.578+0000 7ff1edfc4b80 1 Processor -- start 2023-09-10T01:19:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.578+0000 7ff1edfc4b80 1 -- start start 2023-09-10T01:19:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1edfc4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326b21a0 0x55d9326b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1edfc4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326ac300 0x55d9326ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1edfc4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326accc0 0x55d9326b8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1edfc4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9326056d0 con 0x55d9326b21a0 2023-09-10T01:19:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1edfc4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d932602ff0 con 0x55d9326accc0 2023-09-10T01:19:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1edfc4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9325fe8e0 con 0x55d9326ac300 2023-09-10T01:19:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1db7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326b21a0 0x55d9326b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1db7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326b21a0 0x55d9326b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58294/0 (socket says 172.21.15.44:58294) 2023-09-10T01:19:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1db7fe700 1 -- 172.21.15.44:0/3763378021 learned_addr learned my addr 172.21.15.44:0/3763378021 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1dbfff700 1 --2- 172.21.15.44:0/3763378021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326accc0 0x55d9326b8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1daffd700 1 --2- 172.21.15.44:0/3763378021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326ac300 0x55d9326ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1db7fe700 1 -- 172.21.15.44:0/3763378021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326ac300 msgr2=0x55d9326ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1db7fe700 1 --2- 172.21.15.44:0/3763378021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326ac300 0x55d9326ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1db7fe700 1 -- 172.21.15.44:0/3763378021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326accc0 msgr2=0x55d9326b8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1db7fe700 1 --2- 172.21.15.44:0/3763378021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326accc0 0x55d9326b8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1db7fe700 1 -- 172.21.15.44:0/3763378021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d93242ad80 con 0x55d9326b21a0 2023-09-10T01:19:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.579+0000 7ff1db7fe700 1 --2- 172.21.15.44:0/3763378021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326b21a0 0x55d9326b2580 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc015000 tx=0x7ff1cc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e435642f49bfb88 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.580+0000 7ff1c1ffb700 1 -- 172.21.15.44:0/3763378021 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc028070 con 0x55d9326b21a0 2023-09-10T01:19:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.580+0000 7ff1c1ffb700 1 -- 172.21.15.44:0/3763378021 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1cc01e040 con 0x55d9326b21a0 2023-09-10T01:19:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.580+0000 7ff1c1ffb700 1 -- 172.21.15.44:0/3763378021 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc003ea0 con 0x55d9326b21a0 2023-09-10T01:19:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.580+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/3763378021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326b21a0 msgr2=0x55d9326b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.580+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/3763378021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326b21a0 0x55d9326b2580 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc015000 tx=0x7ff1cc01f000 comp rx=0 tx=0).stop 2023-09-10T01:19:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.580+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/3763378021 shutdown_connections 2023-09-10T01:19:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.580+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/3763378021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326accc0 0x55d9326b8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.580+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/3763378021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326b21a0 0x55d9326b2580 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.580+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/3763378021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326ac300 0x55d9326ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.580+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/3763378021 >> 172.21.15.44:0/3763378021 conn(0x55d932621030 msgr2=0x55d9326a7560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.580+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/3763378021 shutdown_connections 2023-09-10T01:19:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.580+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/3763378021 wait complete. 2023-09-10T01:19:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.581+0000 7ff1edfc4b80 1 -- start start 2023-09-10T01:19:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.581+0000 7ff1edfc4b80 1 Processor -- start 2023-09-10T01:19:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.581+0000 7ff1edfc4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 0x55d9326b8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.581+0000 7ff1edfc4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326accc0 0x55d9326b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.581+0000 7ff1edfc4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326b21a0 0x55d9326b6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.581+0000 7ff1edfc4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d93259cb10 con 0x55d9326accc0 2023-09-10T01:19:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.581+0000 7ff1edfc4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d93259c810 con 0x55d9326ac300 2023-09-10T01:19:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.581+0000 7ff1edfc4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9324808c0 con 0x55d9326b21a0 2023-09-10T01:19:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.581+0000 7ff1edfc4b80 10 client.0 _ll_register_callbacks cb 0x55d9324e6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:19:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.582+0000 7ff1daffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326accc0 0x55d9326b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.582+0000 7ff1daffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326accc0 0x55d9326b60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58302/0 (socket says 172.21.15.44:58302) 2023-09-10T01:19:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.582+0000 7ff1daffd700 1 -- 172.21.15.44:0/1691686642 learned_addr learned my addr 172.21.15.44:0/1691686642 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.582+0000 7ff1db7fe700 1 --2- 172.21.15.44:0/1691686642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 0x55d9326b8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.582+0000 7ff1dbfff700 1 --2- 172.21.15.44:0/1691686642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326b21a0 0x55d9326b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.582+0000 7ff1daffd700 1 -- 172.21.15.44:0/1691686642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326b21a0 msgr2=0x55d9326b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.582+0000 7ff1daffd700 1 --2- 172.21.15.44:0/1691686642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326b21a0 0x55d9326b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.582+0000 7ff1daffd700 1 -- 172.21.15.44:0/1691686642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 msgr2=0x55d9326b8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.582+0000 7ff1daffd700 1 --2- 172.21.15.44:0/1691686642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 0x55d9326b8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.582+0000 7ff1daffd700 1 -- 172.21.15.44:0/1691686642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc10dcd0 con 0x55d9326accc0 2023-09-10T01:19:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.582+0000 7ff1db7fe700 1 --2- 172.21.15.44:0/1691686642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 0x55d9326b8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:19:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.582+0000 7ff1daffd700 1 --2- 172.21.15.44:0/1691686642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326accc0 0x55d9326b60b0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0012840 tx=0x7ff1d0012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38134c369594a812 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.582+0000 7ff1baffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:19:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.582+0000 7ff1baffd700 1 -- 172.21.15.44:0/1691686642 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0017070 con 0x55d9326accc0 2023-09-10T01:19:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.582+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/1691686642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d932556490 con 0x55d9326accc0 2023-09-10T01:19:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.582+0000 7ff1baffd700 1 -- 172.21.15.44:0/1691686642 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(166..166 src has 1..166) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff1d0004830 con 0x55d9326accc0 2023-09-10T01:19:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.583+0000 7ff1baffd700 1 -- 172.21.15.44:0/1691686642 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1d0004a30 con 0x55d9326accc0 2023-09-10T01:19:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.583+0000 7ff1edfc4b80 10 client.19788 fetch_fsmap learned FSMap version 125 2023-09-10T01:19:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.583+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/1691686642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=4+}) v3 -- 0x55d932413c00 con 0x55d9326accc0 2023-09-10T01:19:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.583+0000 7ff1baffd700 1 -- 172.21.15.44:0/1691686642 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d932556490 con 0x55d9326accc0 2023-09-10T01:19:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.583+0000 7ff1baffd700 1 -- 172.21.15.44:0/1691686642 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 125) v1 ==== 7331+0+0 (secure 0 0 0) 0x7ff1d001c630 con 0x55d9326accc0 2023-09-10T01:19:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.583+0000 7ff1edfc4b80 10 client.19788 fetch_fsmap finished waiting for FSMap version 125 2023-09-10T01:19:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.583+0000 7ff1edfc4b80 10 client.19788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:19:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.583+0000 7ff1edfc4b80 20 client.19788 populate_metadata read hostname 'smithi044' 2023-09-10T01:19:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.583+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/1691686642 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d9326c64b0 0x55d9326c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.583+0000 7ff1edfc4b80 4 client.19788 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:19:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.583+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/1691686642 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d932613840 con 0x55d9326c64b0 2023-09-10T01:19:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.583+0000 7ff1db7fe700 1 --2- 172.21.15.44:0/1691686642 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d9326c64b0 0x55d9326c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.584+0000 7ff1baffd700 10 client.19788 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.584+0000 7ff1db7fe700 1 --2- 172.21.15.44:0/1691686642 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d9326c64b0 0x55d9326c88d0 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eb1a8afc8ad3403 server_cookie=f25c091d2748441e in_seq=0 out_seq=0 2023-09-10T01:19:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.584+0000 7ff1baffd700 1 -- 172.21.15.44:0/1691686642 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d932613840 con 0x55d9326c64b0 2023-09-10T01:19:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.584+0000 7ff1baffd700 10 client.19788 handle_command_reply: tid=0 2023-09-10T01:19:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.584+0000 7ff1edfc4b80 1 client.19788 shutdown 2023-09-10T01:19:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.584+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/1691686642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326accc0 msgr2=0x55d9326b60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.584+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/1691686642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326accc0 0x55d9326b60b0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0012840 tx=0x7ff1d0012b50 comp rx=0 tx=0).stop 2023-09-10T01:19:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.584+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/1691686642 shutdown_connections 2023-09-10T01:19:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.584+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/1691686642 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d9326c64b0 0x55d9326c88d0 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.584+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/1691686642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326b21a0 0x55d9326b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.584+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/1691686642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 0x55d9326b8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.584+0000 7ff1baffd700 0 client.19788 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.584+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/1691686642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326accc0 0x55d9326b60b0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.584+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/1691686642 >> 172.21.15.44:0/1691686642 conn(0x55d932621030 msgr2=0x55d9326a67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.585+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/1691686642 shutdown_connections 2023-09-10T01:19:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.585+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/1691686642 wait complete. 2023-09-10T01:19:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.585+0000 7ff1edfc4b80 20 client.19788 trim_cache size 0 max 16384 2023-09-10T01:19:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.602+0000 7ff1edfc4b80 1 Processor -- start 2023-09-10T01:19:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.602+0000 7ff1edfc4b80 1 -- start start 2023-09-10T01:19:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.602+0000 7ff1edfc4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 0x55d9326b0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.602+0000 7ff1edfc4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326accc0 0x55d9326b0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.602+0000 7ff1edfc4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326b21a0 0x55d9326b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.602+0000 7ff1edfc4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9326c39f0 con 0x55d9326b21a0 2023-09-10T01:19:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.602+0000 7ff1edfc4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9326afdb0 con 0x55d9326ac300 2023-09-10T01:19:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.602+0000 7ff1edfc4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9326b1e00 con 0x55d9326accc0 2023-09-10T01:19:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.603+0000 7ff1dbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326b21a0 0x55d9326b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.603+0000 7ff1db7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 0x55d9326b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.603+0000 7ff1dbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326b21a0 0x55d9326b11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58312/0 (socket says 172.21.15.44:58312) 2023-09-10T01:19:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.603+0000 7ff1dbfff700 1 -- 172.21.15.44:0/1964148384 learned_addr learned my addr 172.21.15.44:0/1964148384 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.603+0000 7ff1daffd700 1 --2- 172.21.15.44:0/1964148384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326accc0 0x55d9326b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.603+0000 7ff1dbfff700 1 -- 172.21.15.44:0/1964148384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326accc0 msgr2=0x55d9326b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.603+0000 7ff1dbfff700 1 --2- 172.21.15.44:0/1964148384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326accc0 0x55d9326b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.603+0000 7ff1dbfff700 1 -- 172.21.15.44:0/1964148384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 msgr2=0x55d9326b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.603+0000 7ff1dbfff700 1 --2- 172.21.15.44:0/1964148384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 0x55d9326b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.603+0000 7ff1dbfff700 1 -- 172.21.15.44:0/1964148384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9324511f0 con 0x55d9326b21a0 2023-09-10T01:19:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.603+0000 7ff1dbfff700 1 --2- 172.21.15.44:0/1964148384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326b21a0 0x55d9326b11d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4098e20 tx=0x7ff1d4098840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58433daa6bff73eb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.603+0000 7ff1c27fc700 1 -- 172.21.15.44:0/1964148384 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4098ee0 con 0x55d9326b21a0 2023-09-10T01:19:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.603+0000 7ff1c27fc700 1 -- 172.21.15.44:0/1964148384 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1d409cd10 con 0x55d9326b21a0 2023-09-10T01:19:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.603+0000 7ff1c27fc700 1 -- 172.21.15.44:0/1964148384 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4099140 con 0x55d9326b21a0 2023-09-10T01:19:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.604+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/1964148384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326b21a0 msgr2=0x55d9326b11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.604+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/1964148384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326b21a0 0x55d9326b11d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4098e20 tx=0x7ff1d4098840 comp rx=0 tx=0).stop 2023-09-10T01:19:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.604+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/1964148384 shutdown_connections 2023-09-10T01:19:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.604+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/1964148384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326b21a0 0x55d9326b11d0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.604+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/1964148384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 0x55d9326b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.604+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/1964148384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326accc0 0x55d9326b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.604+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/1964148384 >> 172.21.15.44:0/1964148384 conn(0x55d932621030 msgr2=0x55d9326c1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.604+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/1964148384 shutdown_connections 2023-09-10T01:19:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.604+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/1964148384 wait complete. 2023-09-10T01:19:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.604+0000 7ff1edfc4b80 1 -- start start 2023-09-10T01:19:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.604+0000 7ff1edfc4b80 1 Processor -- start 2023-09-10T01:19:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1edfc4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 0x55d9326bb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1edfc4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326accc0 0x55d9326bbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1edfc4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326b21a0 0x55d932719300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1edfc4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d932719840 con 0x55d9326accc0 2023-09-10T01:19:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1edfc4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d932719980 con 0x55d9326ac300 2023-09-10T01:19:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1edfc4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d932719ac0 con 0x55d9326b21a0 2023-09-10T01:19:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1edfc4b80 10 client.0 _ll_register_callbacks cb 0x7ff1d0009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:19:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1db7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 0x55d9326bb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1db7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 0x55d9326bb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60054/0 (socket says 172.21.15.44:60054) 2023-09-10T01:19:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1daffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326accc0 0x55d9326bbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1db7fe700 1 -- 172.21.15.44:0/2142204955 learned_addr learned my addr 172.21.15.44:0/2142204955 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1dbfff700 1 --2- 172.21.15.44:0/2142204955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326b21a0 0x55d932719300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1db7fe700 1 -- 172.21.15.44:0/2142204955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326b21a0 msgr2=0x55d932719300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1db7fe700 1 --2- 172.21.15.44:0/2142204955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326b21a0 0x55d932719300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1db7fe700 1 -- 172.21.15.44:0/2142204955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326accc0 msgr2=0x55d9326bbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1db7fe700 1 --2- 172.21.15.44:0/2142204955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326accc0 0x55d9326bbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1db7fe700 1 -- 172.21.15.44:0/2142204955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc1a35a0 con 0x55d9326ac300 2023-09-10T01:19:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1db7fe700 1 --2- 172.21.15.44:0/2142204955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 0x55d9326bb610 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc02f040 tx=0x7ff1cc00be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bda49ab98cc4b77c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.605+0000 7ff1baffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:19:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.606+0000 7ff1baffd700 1 -- 172.21.15.44:0/2142204955 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc028040 con 0x55d9326ac300 2023-09-10T01:19:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.606+0000 7ff1baffd700 1 -- 172.21.15.44:0/2142204955 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(166..166 src has 1..166) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ff1cc0379e0 con 0x55d9326ac300 2023-09-10T01:19:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.606+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/2142204955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1cc01e040 con 0x55d9326ac300 2023-09-10T01:19:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.606+0000 7ff1baffd700 1 -- 172.21.15.44:0/2142204955 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1cc003b70 con 0x55d9326ac300 2023-09-10T01:19:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.606+0000 7ff1edfc4b80 10 client.29414 fetch_fsmap learned FSMap version 125 2023-09-10T01:19:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.606+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/2142204955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=4+}) v3 -- 0x55d932429250 con 0x55d9326ac300 2023-09-10T01:19:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.606+0000 7ff1baffd700 1 -- 172.21.15.44:0/2142204955 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1cc01e040 con 0x55d9326ac300 2023-09-10T01:19:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.606+0000 7ff1baffd700 1 -- 172.21.15.44:0/2142204955 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 125) v1 ==== 7331+0+0 (secure 0 0 0) 0x7ff1cc00f6f0 con 0x55d9326ac300 2023-09-10T01:19:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.606+0000 7ff1edfc4b80 10 client.29414 fetch_fsmap finished waiting for FSMap version 125 2023-09-10T01:19:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.606+0000 7ff1edfc4b80 10 client.29414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:19:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.606+0000 7ff1edfc4b80 20 client.29414 populate_metadata read hostname 'smithi044' 2023-09-10T01:19:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.606+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/2142204955 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d93271ae10 0x55d93271d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.606+0000 7ff1edfc4b80 4 client.29414 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:19:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.606+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/2142204955 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d9325f78c0 con 0x55d93271ae10 2023-09-10T01:19:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.606+0000 7ff1daffd700 1 --2- 172.21.15.44:0/2142204955 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d93271ae10 0x55d93271d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.607+0000 7ff1baffd700 10 client.29414 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.607+0000 7ff1daffd700 1 --2- 172.21.15.44:0/2142204955 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d93271ae10 0x55d93271d230 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33835f931de49ee server_cookie=f4f139edbe63562f in_seq=0 out_seq=0 2023-09-10T01:19:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.621+0000 7ff1baffd700 1 -- 172.21.15.44:0/2142204955 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d9325f78c0 con 0x55d93271ae10 2023-09-10T01:19:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.621+0000 7ff1baffd700 10 client.29414 handle_command_reply: tid=0 2023-09-10T01:19:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.621+0000 7ff1edfc4b80 1 client.29414 shutdown 2023-09-10T01:19:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/2142204955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 msgr2=0x55d9326bb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/2142204955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 0x55d9326bb610 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc02f040 tx=0x7ff1cc00be20 comp rx=0 tx=0).stop 2023-09-10T01:19:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/2142204955 shutdown_connections 2023-09-10T01:19:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/2142204955 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d93271ae10 0x55d93271d230 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/2142204955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9326b21a0 0x55d932719300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/2142204955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9326ac300 0x55d9326bb610 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1edfc4b80 1 --2- 172.21.15.44:0/2142204955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9326accc0 0x55d9326bbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1baffd700 0 client.29414 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/2142204955 >> 172.21.15.44:0/2142204955 conn(0x55d932621030 msgr2=0x55d9326b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/2142204955 shutdown_connections 2023-09-10T01:19:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1edfc4b80 1 -- 172.21.15.44:0/2142204955 wait complete. 2023-09-10T01:19:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1edfc4b80 20 client.29414 trim_cache size 0 max 16384 2023-09-10T01:19:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:19:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:19:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:19:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:19:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1e1dfe700 1 -- 172.21.15.44:0/3587447715 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1c40a0830 msgr2=0x7ff1c40a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1e1dfe700 1 --2- 172.21.15.44:0/3587447715 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1c40a0830 0x7ff1c40a2cf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc006010 tx=0x7ff1cc005c00 comp rx=0 tx=0).stop 2023-09-10T01:19:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1e1dfe700 1 -- 172.21.15.44:0/3587447715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1dc106650 msgr2=0x7ff1dc196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1e1dfe700 1 --2- 172.21.15.44:0/3587447715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1dc106650 0x7ff1dc196410 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400ef30 tx=0x7ff1d400c3a0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1e1dfe700 1 -- 172.21.15.44:0/3587447715 shutdown_connections 2023-09-10T01:19:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1e1dfe700 1 --2- 172.21.15.44:0/3587447715 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ff1c40a0830 0x7ff1c40a2cf0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1e1dfe700 1 --2- 172.21.15.44:0/3587447715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1dc106650 0x7ff1dc196410 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1e1dfe700 1 --2- 172.21.15.44:0/3587447715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1dc0679f0 0x7ff1dc19c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1e1dfe700 1 --2- 172.21.15.44:0/3587447715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1dc0683b0 0x7ff1dc19caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.622+0000 7ff1e1dfe700 1 -- 172.21.15.44:0/3587447715 >> 172.21.15.44:0/3587447715 conn(0x7ff1dc074f30 msgr2=0x7ff1dc104250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.623+0000 7ff1e1dfe700 1 -- 172.21.15.44:0/3587447715 shutdown_connections 2023-09-10T01:19:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:16.623+0000 7ff1e1dfe700 1 -- 172.21.15.44:0/3587447715 wait complete. 2023-09-10T01:19:16.805 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:19:16 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[198064]: starting mds.d at 2023-09-10T01:19:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:17 smithi131 ceph-mon[103713]: pgmap v1244: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 26 MiB/s wr, 165 op/s 2023-09-10T01:19:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:17 smithi131 ceph-mon[103713]: osdmap e166: 12 total, 12 up, 12 in 2023-09-10T01:19:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:17 smithi131 ceph-mon[103713]: daemon mds.d restarted 2023-09-10T01:19:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:17 smithi044 ceph-mon[98676]: pgmap v1244: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 26 MiB/s wr, 165 op/s 2023-09-10T01:19:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:17 smithi044 ceph-mon[98676]: osdmap e166: 12 total, 12 up, 12 in 2023-09-10T01:19:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:17 smithi044 ceph-mon[98676]: daemon mds.d restarted 2023-09-10T01:19:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:17 smithi099 ceph-mon[102692]: pgmap v1244: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 26 MiB/s wr, 165 op/s 2023-09-10T01:19:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:17 smithi099 ceph-mon[102692]: osdmap e166: 12 total, 12 up, 12 in 2023-09-10T01:19:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:17 smithi099 ceph-mon[102692]: daemon mds.d restarted 2023-09-10T01:19:18.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:18 smithi131 ceph-mon[103713]: osdmap e167: 12 total, 12 up, 12 in 2023-09-10T01:19:18.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:18 smithi131 ceph-mon[103713]: mds.? [v2:172.21.15.44:6836/613045926,v1:172.21.15.44:6837/613045926] up:boot 2023-09-10T01:19:18.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:18 smithi131 ceph-mon[103713]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:19:18.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:18 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T01:19:18.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:18 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:19:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:18 smithi044 ceph-mon[98676]: osdmap e167: 12 total, 12 up, 12 in 2023-09-10T01:19:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:18 smithi044 ceph-mon[98676]: mds.? [v2:172.21.15.44:6836/613045926,v1:172.21.15.44:6837/613045926] up:boot 2023-09-10T01:19:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:18 smithi044 ceph-mon[98676]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:19:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:18 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T01:19:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:18 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:19:18.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:18 smithi099 ceph-mon[102692]: osdmap e167: 12 total, 12 up, 12 in 2023-09-10T01:19:18.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:18 smithi099 ceph-mon[102692]: mds.? [v2:172.21.15.44:6836/613045926,v1:172.21.15.44:6837/613045926] up:boot 2023-09-10T01:19:18.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:18 smithi099 ceph-mon[102692]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:19:18.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:18 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-10T01:19:18.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:18 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:19:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:19 smithi131 ceph-mon[103713]: pgmap v1247: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 25 MiB/s wr, 165 op/s 2023-09-10T01:19:19.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:19 smithi044 ceph-mon[98676]: pgmap v1247: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 25 MiB/s wr, 165 op/s 2023-09-10T01:19:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:19 smithi099 ceph-mon[102692]: pgmap v1247: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 25 MiB/s wr, 165 op/s 2023-09-10T01:19:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:21 smithi131 ceph-mon[103713]: pgmap v1248: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 74 op/s 2023-09-10T01:19:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:21 smithi044 ceph-mon[98676]: pgmap v1248: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 74 op/s 2023-09-10T01:19:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:21 smithi099 ceph-mon[102692]: pgmap v1248: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 74 op/s 2023-09-10T01:19:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:23 smithi131 ceph-mon[103713]: pgmap v1249: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 24 MiB/s wr, 154 op/s 2023-09-10T01:19:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:23 smithi044 ceph-mon[98676]: pgmap v1249: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 24 MiB/s wr, 154 op/s 2023-09-10T01:19:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:23 smithi099 ceph-mon[102692]: pgmap v1249: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 24 MiB/s wr, 154 op/s 2023-09-10T01:19:25.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:25 smithi044 ceph-mon[98676]: pgmap v1250: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 107 op/s 2023-09-10T01:19:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:25 smithi099 ceph-mon[102692]: pgmap v1250: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 107 op/s 2023-09-10T01:19:25.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:25 smithi131 ceph-mon[103713]: pgmap v1250: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 107 op/s 2023-09-10T01:19:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:27 smithi044 ceph-mon[98676]: pgmap v1251: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 18 MiB/s wr, 113 op/s 2023-09-10T01:19:27.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:27 smithi099 ceph-mon[102692]: pgmap v1251: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 18 MiB/s wr, 113 op/s 2023-09-10T01:19:27.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:27 smithi131 ceph-mon[103713]: pgmap v1251: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 18 MiB/s wr, 113 op/s 2023-09-10T01:19:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:29 smithi044 ceph-mon[98676]: pgmap v1252: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 118 op/s 2023-09-10T01:19:29.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:29 smithi099 ceph-mon[102692]: pgmap v1252: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 118 op/s 2023-09-10T01:19:29.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:29 smithi131 ceph-mon[103713]: pgmap v1252: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 118 op/s 2023-09-10T01:19:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.772+0000 7f7597dfe700 1 -- 172.21.15.44:0/157866819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75900ff5f0 msgr2=0x7f75900ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.773+0000 7f7597dfe700 1 --2- 172.21.15.44:0/157866819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75900ff5f0 0x7f75900ffa70 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f758c009990 tx=0x7f758c009ca0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.773+0000 7f7597dfe700 1 -- 172.21.15.44:0/157866819 shutdown_connections 2023-09-10T01:19:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.773+0000 7f7597dfe700 1 --2- 172.21.15.44:0/157866819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75900fffb0 0x7f759010c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.773+0000 7f7597dfe700 1 --2- 172.21.15.44:0/157866819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75901054c0 0x7f75901058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.773+0000 7f7597dfe700 1 --2- 172.21.15.44:0/157866819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75900ff5f0 0x7f75900ffa70 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.773+0000 7f7597dfe700 1 -- 172.21.15.44:0/157866819 >> 172.21.15.44:0/157866819 conn(0x7f7590074f10 msgr2=0x7f7590075320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.773+0000 7f7597dfe700 1 -- 172.21.15.44:0/157866819 shutdown_connections 2023-09-10T01:19:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.773+0000 7f7597dfe700 1 -- 172.21.15.44:0/157866819 wait complete. 2023-09-10T01:19:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f7597dfe700 1 Processor -- start 2023-09-10T01:19:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f7597dfe700 1 -- start start 2023-09-10T01:19:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f7597dfe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75900ff5f0 0x7f7590196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f7597dfe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75900fffb0 0x7f7590196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f7597dfe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75901054c0 0x7f759019afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f7597dfe700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759019b5e0 con 0x7f75900fffb0 2023-09-10T01:19:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f7597dfe700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759019b750 con 0x7f75901054c0 2023-09-10T01:19:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f7597dfe700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759019b910 con 0x7f75900ff5f0 2023-09-10T01:19:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f759639b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75901054c0 0x7f759019afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f759639b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75901054c0 0x7f759019afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60082/0 (socket says 172.21.15.44:60082) 2023-09-10T01:19:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f759639b700 1 -- 172.21.15.44:0/2412030770 learned_addr learned my addr 172.21.15.44:0/2412030770 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f7595399700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75900fffb0 0x7f7590196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f7595b9a700 1 --2- 172.21.15.44:0/2412030770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75900ff5f0 0x7f7590196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f759639b700 1 -- 172.21.15.44:0/2412030770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75900ff5f0 msgr2=0x7f7590196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f759639b700 1 --2- 172.21.15.44:0/2412030770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75900ff5f0 0x7f7590196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f759639b700 1 -- 172.21.15.44:0/2412030770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75900fffb0 msgr2=0x7f7590196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f759639b700 1 --2- 172.21.15.44:0/2412030770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75900fffb0 0x7f7590196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.774+0000 7f759639b700 1 -- 172.21.15.44:0/2412030770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c0095f0 con 0x7f75901054c0 2023-09-10T01:19:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.775+0000 7f759639b700 1 --2- 172.21.15.44:0/2412030770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75901054c0 0x7f759019afa0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f758800ef30 tx=0x7f758800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.775+0000 7f7586ffd700 1 -- 172.21.15.44:0/2412030770 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588018070 con 0x7f75901054c0 2023-09-10T01:19:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.775+0000 7f7597dfe700 1 -- 172.21.15.44:0/2412030770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759019bb90 con 0x7f75901054c0 2023-09-10T01:19:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.775+0000 7f7586ffd700 1 -- 172.21.15.44:0/2412030770 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f758800f040 con 0x7f75901054c0 2023-09-10T01:19:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.775+0000 7f7586ffd700 1 -- 172.21.15.44:0/2412030770 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758800cbc0 con 0x7f75901054c0 2023-09-10T01:19:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.775+0000 7f7597dfe700 1 -- 172.21.15.44:0/2412030770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759010cd50 con 0x7f75901054c0 2023-09-10T01:19:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.778+0000 7f7586ffd700 1 -- 172.21.15.44:0/2412030770 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f7588007500 con 0x7f75901054c0 2023-09-10T01:19:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.778+0000 7f7586ffd700 1 --2- 172.21.15.44:0/2412030770 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f757c0a07b0 0x7f757c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.779+0000 7f7586ffd700 1 -- 172.21.15.44:0/2412030770 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f75880cbe50 con 0x7f75901054c0 2023-09-10T01:19:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.779+0000 7f7595b9a700 1 --2- 172.21.15.44:0/2412030770 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f757c0a07b0 0x7f757c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.779+0000 7f7595b9a700 1 --2- 172.21.15.44:0/2412030770 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f757c0a07b0 0x7f757c0a2c70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7580009be0 tx=0x7f7580009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.779+0000 7f75a3fc4b80 1 Processor -- start 2023-09-10T01:19:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.779+0000 7f75a3fc4b80 1 -- start start 2023-09-10T01:19:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f75a3fc4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f6e1a0 0x564b75f6e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f75a3fc4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f68300 0x564b75f68780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f75a3fc4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75f74f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f75a3fc4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b75ec16d0 con 0x564b75f68300 2023-09-10T01:19:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f75a3fc4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b75ebeff0 con 0x564b75f6e1a0 2023-09-10T01:19:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f75a3fc4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b75eba8e0 con 0x564b75f68cc0 2023-09-10T01:19:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f7595b9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f6e1a0 0x564b75f6e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f7595b9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f6e1a0 0x564b75f6e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60090/0 (socket says 172.21.15.44:60090) 2023-09-10T01:19:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f7595b9a700 1 -- 172.21.15.44:0/3977884378 learned_addr learned my addr 172.21.15.44:0/3977884378 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f7595399700 1 --2- 172.21.15.44:0/3977884378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f68300 0x564b75f68780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f759639b700 1 --2- 172.21.15.44:0/3977884378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75f74f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f7595b9a700 1 -- 172.21.15.44:0/3977884378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 msgr2=0x564b75f74f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f7595b9a700 1 --2- 172.21.15.44:0/3977884378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75f74f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f7595b9a700 1 -- 172.21.15.44:0/3977884378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f68300 msgr2=0x564b75f68780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f7595b9a700 1 --2- 172.21.15.44:0/3977884378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f68300 0x564b75f68780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f7595b9a700 1 -- 172.21.15.44:0/3977884378 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b75ce6d80 con 0x564b75f6e1a0 2023-09-10T01:19:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.780+0000 7f7595b9a700 1 --2- 172.21.15.44:0/3977884378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f6e1a0 0x564b75f6e580 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7580011d70 tx=0x7f7580014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8089e3acb75f64c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.781+0000 7f757bfff700 1 -- 172.21.15.44:0/3977884378 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580011da0 con 0x564b75f6e1a0 2023-09-10T01:19:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.781+0000 7f757bfff700 1 -- 172.21.15.44:0/3977884378 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7580004020 con 0x564b75f6e1a0 2023-09-10T01:19:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.781+0000 7f757bfff700 1 -- 172.21.15.44:0/3977884378 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580003ba0 con 0x564b75f6e1a0 2023-09-10T01:19:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.782+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/3977884378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f6e1a0 msgr2=0x564b75f6e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.782+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/3977884378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f6e1a0 0x564b75f6e580 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7580011d70 tx=0x7f7580014000 comp rx=0 tx=0).stop 2023-09-10T01:19:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.782+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/3977884378 shutdown_connections 2023-09-10T01:19:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.782+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/3977884378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75f74f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.782+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/3977884378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f6e1a0 0x564b75f6e580 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.782+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/3977884378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f68300 0x564b75f68780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.782+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/3977884378 >> 172.21.15.44:0/3977884378 conn(0x564b75edd030 msgr2=0x564b75f63560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.782+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/3977884378 shutdown_connections 2023-09-10T01:19:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.782+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/3977884378 wait complete. 2023-09-10T01:19:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.782+0000 7f75a3fc4b80 1 -- start start 2023-09-10T01:19:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.782+0000 7f75a3fc4b80 1 Processor -- start 2023-09-10T01:19:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f75a3fc4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f68300 0x564b75f74d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f75a3fc4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75f72090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f75a3fc4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f6e1a0 0x564b75f72610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f75a3fc4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b75e58b10 con 0x564b75f6e1a0 2023-09-10T01:19:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f75a3fc4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b75e58810 con 0x564b75f68300 2023-09-10T01:19:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f75a3fc4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b75d3c8c0 con 0x564b75f68cc0 2023-09-10T01:19:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f75a3fc4b80 10 client.0 _ll_register_callbacks cb 0x564b75da2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:19:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f7595b9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f68300 0x564b75f74d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f759639b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f6e1a0 0x564b75f72610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f7595399700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75f72090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f759639b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f6e1a0 0x564b75f72610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58360/0 (socket says 172.21.15.44:58360) 2023-09-10T01:19:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f759639b700 1 -- 172.21.15.44:0/55867813 learned_addr learned my addr 172.21.15.44:0/55867813 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f7595b9a700 1 -- 172.21.15.44:0/55867813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 msgr2=0x564b75f72090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f7595b9a700 1 --2- 172.21.15.44:0/55867813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75f72090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f7595b9a700 1 -- 172.21.15.44:0/55867813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f6e1a0 msgr2=0x564b75f72610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f7595b9a700 1 --2- 172.21.15.44:0/55867813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f6e1a0 0x564b75f72610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f7595b9a700 1 -- 172.21.15.44:0/55867813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7580017070 con 0x564b75f68300 2023-09-10T01:19:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.783+0000 7f7595399700 1 --2- 172.21.15.44:0/55867813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75f72090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:19:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.784+0000 7f7595b9a700 1 --2- 172.21.15.44:0/55867813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f68300 0x564b75f74d70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7580005950 tx=0x7f75800045c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9e2cd726ab45a2c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.784+0000 7f75797fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:19:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.784+0000 7f75797fa700 1 -- 172.21.15.44:0/55867813 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580005460 con 0x564b75f68300 2023-09-10T01:19:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.784+0000 7f75797fa700 1 -- 172.21.15.44:0/55867813 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(167..167 src has 1..167) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7580011da0 con 0x564b75f68300 2023-09-10T01:19:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.784+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/55867813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b75e12490 con 0x564b75f68300 2023-09-10T01:19:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.784+0000 7f75797fa700 1 -- 172.21.15.44:0/55867813 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75800055c0 con 0x564b75f68300 2023-09-10T01:19:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.784+0000 7f75a3fc4b80 10 client.29426 fetch_fsmap learned FSMap version 127 2023-09-10T01:19:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.784+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/55867813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x564b75ccfc00 con 0x564b75f68300 2023-09-10T01:19:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.784+0000 7f75797fa700 1 -- 172.21.15.44:0/55867813 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564b75e12490 con 0x564b75f68300 2023-09-10T01:19:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.785+0000 7f75797fa700 1 -- 172.21.15.44:0/55867813 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 127) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f7580036070 con 0x564b75f68300 2023-09-10T01:19:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.785+0000 7f75a3fc4b80 10 client.29426 fetch_fsmap finished waiting for FSMap version 127 2023-09-10T01:19:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.785+0000 7f75a3fc4b80 10 client.29426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:19:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.785+0000 7f75a3fc4b80 20 client.29426 populate_metadata read hostname 'smithi044' 2023-09-10T01:19:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.785+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/55867813 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564b75f82240 0x564b75f84660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.785+0000 7f75a3fc4b80 4 client.29426 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:19:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.785+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/55867813 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b75ecf840 con 0x564b75f82240 2023-09-10T01:19:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.785+0000 7f7595399700 1 --2- 172.21.15.44:0/55867813 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564b75f82240 0x564b75f84660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.785+0000 7f75797fa700 10 client.29426 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.785+0000 7f7595399700 1 --2- 172.21.15.44:0/55867813 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564b75f82240 0x564b75f84660 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bdcb1505c88c72f server_cookie=298ea0827f5ddb26 in_seq=0 out_seq=0 2023-09-10T01:19:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.786+0000 7f75797fa700 1 -- 172.21.15.44:0/55867813 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564b75ecf840 con 0x564b75f82240 2023-09-10T01:19:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.786+0000 7f75797fa700 10 client.29426 handle_command_reply: tid=0 2023-09-10T01:19:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.786+0000 7f75a3fc4b80 1 client.29426 shutdown 2023-09-10T01:19:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.786+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/55867813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f68300 msgr2=0x564b75f74d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.786+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/55867813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f68300 0x564b75f74d70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7580005950 tx=0x7f75800045c0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.786+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/55867813 shutdown_connections 2023-09-10T01:19:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.786+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/55867813 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564b75f82240 0x564b75f84660 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.786+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/55867813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f6e1a0 0x564b75f72610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.786+0000 7f75797fa700 0 client.29426 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.786+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/55867813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f68300 0x564b75f74d70 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.786+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/55867813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75f72090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.786+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/55867813 >> 172.21.15.44:0/55867813 conn(0x564b75edd030 msgr2=0x564b75f627a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.786+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/55867813 shutdown_connections 2023-09-10T01:19:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.786+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/55867813 wait complete. 2023-09-10T01:19:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.786+0000 7f75a3fc4b80 20 client.29426 trim_cache size 0 max 16384 2023-09-10T01:19:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.806+0000 7f75a3fc4b80 1 Processor -- start 2023-09-10T01:19:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.806+0000 7f75a3fc4b80 1 -- start start 2023-09-10T01:19:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f75a3fc4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f68300 0x564b75f79530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f75a3fc4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75f748a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f75a3fc4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f6e1a0 0x564b75f6d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f75a3fc4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b75f6de30 con 0x564b75f68300 2023-09-10T01:19:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f75a3fc4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b75f73670 con 0x564b75f6e1a0 2023-09-10T01:19:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f75a3fc4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b75f79aa0 con 0x564b75f68cc0 2023-09-10T01:19:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f7595b9a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f68300 0x564b75f79530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f7595b9a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f68300 0x564b75f79530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58364/0 (socket says 172.21.15.44:58364) 2023-09-10T01:19:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f7595b9a700 1 -- 172.21.15.44:0/185953926 learned_addr learned my addr 172.21.15.44:0/185953926 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f7595399700 1 --2- 172.21.15.44:0/185953926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75f748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f759639b700 1 --2- 172.21.15.44:0/185953926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f6e1a0 0x564b75f6d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f7595399700 1 -- 172.21.15.44:0/185953926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f6e1a0 msgr2=0x564b75f6d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f7595399700 1 --2- 172.21.15.44:0/185953926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f6e1a0 0x564b75f6d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f7595399700 1 -- 172.21.15.44:0/185953926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f68300 msgr2=0x564b75f79530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f7595399700 1 --2- 172.21.15.44:0/185953926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f68300 0x564b75f79530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f7595399700 1 -- 172.21.15.44:0/185953926 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b75d0d1f0 con 0x564b75f68cc0 2023-09-10T01:19:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f759639b700 1 --2- 172.21.15.44:0/185953926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f6e1a0 0x564b75f6d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:19:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.807+0000 7f7595399700 1 --2- 172.21.15.44:0/185953926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75f748a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f758c000c00 tx=0x7f758c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f6203f63a68767 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.808+0000 7f7584ff9700 1 -- 172.21.15.44:0/185953926 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c01d070 con 0x564b75f68cc0 2023-09-10T01:19:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.808+0000 7f7584ff9700 1 -- 172.21.15.44:0/185953926 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f758c01e040 con 0x564b75f68cc0 2023-09-10T01:19:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.808+0000 7f7584ff9700 1 -- 172.21.15.44:0/185953926 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c004e50 con 0x564b75f68cc0 2023-09-10T01:19:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.808+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/185953926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 msgr2=0x564b75f748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.808+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/185953926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75f748a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f758c000c00 tx=0x7f758c009310 comp rx=0 tx=0).stop 2023-09-10T01:19:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.809+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/185953926 shutdown_connections 2023-09-10T01:19:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.809+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/185953926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f6e1a0 0x564b75f6d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.809+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/185953926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f68300 0x564b75f79530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.809+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/185953926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75f748a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.809+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/185953926 >> 172.21.15.44:0/185953926 conn(0x564b75edd030 msgr2=0x564b75f6c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.809+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/185953926 shutdown_connections 2023-09-10T01:19:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.809+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/185953926 wait complete. 2023-09-10T01:19:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.809+0000 7f75a3fc4b80 1 -- start start 2023-09-10T01:19:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.809+0000 7f75a3fc4b80 1 Processor -- start 2023-09-10T01:19:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.809+0000 7f75a3fc4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f68300 0x564b75fca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.809+0000 7f75a3fc4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75fca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.809+0000 7f75a3fc4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f6e1a0 0x564b75fd5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f75a3fc4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b75fd5580 con 0x564b75f6e1a0 2023-09-10T01:19:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f75a3fc4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b75fd56c0 con 0x564b75f68300 2023-09-10T01:19:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f75a3fc4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b75fd5880 con 0x564b75f68cc0 2023-09-10T01:19:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f75a3fc4b80 10 client.0 _ll_register_callbacks cb 0x7f7580005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:19:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f7595b9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f68300 0x564b75fca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f7595b9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f68300 0x564b75fca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60110/0 (socket says 172.21.15.44:60110) 2023-09-10T01:19:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f7595b9a700 1 -- 172.21.15.44:0/3559348336 learned_addr learned my addr 172.21.15.44:0/3559348336 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f7595399700 1 --2- 172.21.15.44:0/3559348336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75fca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f7595b9a700 1 -- 172.21.15.44:0/3559348336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 msgr2=0x564b75fca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f7595b9a700 1 --2- 172.21.15.44:0/3559348336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75fca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f7595b9a700 1 -- 172.21.15.44:0/3559348336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f6e1a0 msgr2=0x564b75fd5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f7595b9a700 1 --2- 172.21.15.44:0/3559348336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f6e1a0 0x564b75fd5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f7595b9a700 1 -- 172.21.15.44:0/3559348336 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759010cd50 con 0x564b75f68300 2023-09-10T01:19:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f7595b9a700 1 --2- 172.21.15.44:0/3559348336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f68300 0x564b75fca1b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7580004590 tx=0x7f75800117d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30a727ef99c1ceef server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f75797fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:19:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f75797fa700 1 -- 172.21.15.44:0/3559348336 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580004630 con 0x564b75f68300 2023-09-10T01:19:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f75797fa700 1 -- 172.21.15.44:0/3559348336 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(167..167 src has 1..167) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7580003680 con 0x564b75f68300 2023-09-10T01:19:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.810+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/3559348336 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7580004020 con 0x564b75f68300 2023-09-10T01:19:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.811+0000 7f75797fa700 1 -- 172.21.15.44:0/3559348336 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7580003840 con 0x564b75f68300 2023-09-10T01:19:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.811+0000 7f75a3fc4b80 10 client.29432 fetch_fsmap learned FSMap version 127 2023-09-10T01:19:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.811+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/3559348336 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x7f75900679f0 con 0x564b75f68300 2023-09-10T01:19:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.811+0000 7f75797fa700 1 -- 172.21.15.44:0/3559348336 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7580004020 con 0x564b75f68300 2023-09-10T01:19:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.811+0000 7f75797fa700 1 -- 172.21.15.44:0/3559348336 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 127) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f7580036070 con 0x564b75f68300 2023-09-10T01:19:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.811+0000 7f75a3fc4b80 10 client.29432 fetch_fsmap finished waiting for FSMap version 127 2023-09-10T01:19:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.811+0000 7f75a3fc4b80 10 client.29432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:19:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.811+0000 7f75a3fc4b80 20 client.29432 populate_metadata read hostname 'smithi044' 2023-09-10T01:19:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.811+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/3559348336 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564b75fd6cb0 0x564b75fd90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.811+0000 7f75a3fc4b80 4 client.29432 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:19:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.811+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/3559348336 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564b75eb38c0 con 0x564b75fd6cb0 2023-09-10T01:19:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.811+0000 7f7595399700 1 --2- 172.21.15.44:0/3559348336 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564b75fd6cb0 0x564b75fd90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.812+0000 7f75797fa700 10 client.29432 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.812+0000 7f7595399700 1 --2- 172.21.15.44:0/3559348336 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564b75fd6cb0 0x564b75fd90d0 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b2876340e302b22 server_cookie=bb2583dc4b54c065 in_seq=0 out_seq=0 2023-09-10T01:19:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.826+0000 7f75797fa700 1 -- 172.21.15.44:0/3559348336 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564b75eb38c0 con 0x564b75fd6cb0 2023-09-10T01:19:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.826+0000 7f75797fa700 10 client.29432 handle_command_reply: tid=0 2023-09-10T01:19:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.826+0000 7f75a3fc4b80 1 client.29432 shutdown 2023-09-10T01:19:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.826+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/3559348336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f68300 msgr2=0x564b75fca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.826+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/3559348336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f68300 0x564b75fca1b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7580004590 tx=0x7f75800117d0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.826+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/3559348336 shutdown_connections 2023-09-10T01:19:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.826+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/3559348336 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564b75fd6cb0 0x564b75fd90d0 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.826+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/3559348336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b75f6e1a0 0x564b75fd5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.826+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/3559348336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b75f68300 0x564b75fca1b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.826+0000 7f75a3fc4b80 1 --2- 172.21.15.44:0/3559348336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564b75f68cc0 0x564b75fca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.826+0000 7f75797fa700 0 client.29432 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.826+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/3559348336 >> 172.21.15.44:0/3559348336 conn(0x564b75edd030 msgr2=0x564b75f6d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.826+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/3559348336 shutdown_connections 2023-09-10T01:19:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.826+0000 7f75a3fc4b80 1 -- 172.21.15.44:0/3559348336 wait complete. 2023-09-10T01:19:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.826+0000 7f75a3fc4b80 20 client.29432 trim_cache size 0 max 16384 2023-09-10T01:19:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:19:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:19:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:19:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:19:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.827+0000 7f7597dfe700 1 -- 172.21.15.44:0/2412030770 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f757c0a07b0 msgr2=0x7f757c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.827+0000 7f7597dfe700 1 --2- 172.21.15.44:0/2412030770 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f757c0a07b0 0x7f757c0a2c70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7580009be0 tx=0x7f7580009260 comp rx=0 tx=0).stop 2023-09-10T01:19:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.827+0000 7f7597dfe700 1 -- 172.21.15.44:0/2412030770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75901054c0 msgr2=0x7f759019afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.827+0000 7f7597dfe700 1 --2- 172.21.15.44:0/2412030770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75901054c0 0x7f759019afa0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f758800ef30 tx=0x7f758800c3a0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.827+0000 7f7597dfe700 1 -- 172.21.15.44:0/2412030770 shutdown_connections 2023-09-10T01:19:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.827+0000 7f7597dfe700 1 --2- 172.21.15.44:0/2412030770 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f757c0a07b0 0x7f757c0a2c70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.827+0000 7f7597dfe700 1 --2- 172.21.15.44:0/2412030770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75901054c0 0x7f759019afa0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.827+0000 7f7597dfe700 1 --2- 172.21.15.44:0/2412030770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75900ff5f0 0x7f7590196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.827+0000 7f7597dfe700 1 --2- 172.21.15.44:0/2412030770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75900fffb0 0x7f7590196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.827+0000 7f7597dfe700 1 -- 172.21.15.44:0/2412030770 >> 172.21.15.44:0/2412030770 conn(0x7f7590074f10 msgr2=0x7f75900fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.831+0000 7f7597dfe700 1 -- 172.21.15.44:0/2412030770 shutdown_connections 2023-09-10T01:19:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:29.831+0000 7f7597dfe700 1 -- 172.21.15.44:0/2412030770 wait complete. 2023-09-10T01:19:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:31 smithi044 ceph-mon[98676]: pgmap v1253: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 99 op/s 2023-09-10T01:19:31.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:31 smithi099 ceph-mon[102692]: pgmap v1253: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 99 op/s 2023-09-10T01:19:31.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:31 smithi131 ceph-mon[103713]: pgmap v1253: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 99 op/s 2023-09-10T01:19:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:33 smithi044 ceph-mon[98676]: pgmap v1254: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 23 MiB/s wr, 147 op/s 2023-09-10T01:19:33.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:33 smithi099 ceph-mon[102692]: pgmap v1254: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 23 MiB/s wr, 147 op/s 2023-09-10T01:19:33.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:33 smithi131 ceph-mon[103713]: pgmap v1254: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 23 MiB/s wr, 147 op/s 2023-09-10T01:19:35.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:35 smithi099 ceph-mon[102692]: pgmap v1255: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 93 op/s 2023-09-10T01:19:35.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:35 smithi131 ceph-mon[103713]: pgmap v1255: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 93 op/s 2023-09-10T01:19:35.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:35 smithi044 ceph-mon[98676]: pgmap v1255: 129 pgs: 129 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 93 op/s 2023-09-10T01:19:37.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:37 smithi099 ceph-mon[102692]: pgmap v1256: 129 pgs: 129 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 22 MiB/s wr, 137 op/s 2023-09-10T01:19:37.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:37 smithi131 ceph-mon[103713]: pgmap v1256: 129 pgs: 129 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 22 MiB/s wr, 137 op/s 2023-09-10T01:19:37.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:37 smithi044 ceph-mon[98676]: pgmap v1256: 129 pgs: 129 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 22 MiB/s wr, 137 op/s 2023-09-10T01:19:39.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:39 smithi099 ceph-mon[102692]: pgmap v1257: 129 pgs: 129 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 17 MiB/s wr, 108 op/s 2023-09-10T01:19:39.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:39 smithi131 ceph-mon[103713]: pgmap v1257: 129 pgs: 129 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 17 MiB/s wr, 108 op/s 2023-09-10T01:19:39.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:39 smithi044 ceph-mon[98676]: pgmap v1257: 129 pgs: 129 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 17 MiB/s wr, 108 op/s 2023-09-10T01:19:41.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:41 smithi099 ceph-mon[102692]: pgmap v1258: 129 pgs: 129 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 95 op/s 2023-09-10T01:19:41.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:41 smithi131 ceph-mon[103713]: pgmap v1258: 129 pgs: 129 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 95 op/s 2023-09-10T01:19:41.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:41 smithi044 ceph-mon[98676]: pgmap v1258: 129 pgs: 129 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 95 op/s 2023-09-10T01:19:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.974+0000 7f80f6b80700 1 -- 172.21.15.44:0/1414290395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80f00fee80 msgr2=0x7f80f00ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.974+0000 7f80f6b80700 1 --2- 172.21.15.44:0/1414290395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80f00fee80 0x7f80f00ff260 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009990 tx=0x7f80e4009ca0 comp rx=0 tx=0).stop 2023-09-10T01:19:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.974+0000 7f80f6b80700 1 -- 172.21.15.44:0/1414290395 shutdown_connections 2023-09-10T01:19:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.975+0000 7f80f6b80700 1 --2- 172.21.15.44:0/1414290395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80f0103fa0 0x7f80f010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.975+0000 7f80f6b80700 1 --2- 172.21.15.44:0/1414290395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80f00fee80 0x7f80f00ff260 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.975+0000 7f80f6b80700 1 --2- 172.21.15.44:0/1414290395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80f00ff830 0x7f80f0103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.975+0000 7f80f6b80700 1 -- 172.21.15.44:0/1414290395 >> 172.21.15.44:0/1414290395 conn(0x7f80f00769d0 msgr2=0x7f80f00fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.975+0000 7f80f6b80700 1 -- 172.21.15.44:0/1414290395 shutdown_connections 2023-09-10T01:19:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.975+0000 7f80f6b80700 1 -- 172.21.15.44:0/1414290395 wait complete. 2023-09-10T01:19:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.975+0000 7f80f6b80700 1 Processor -- start 2023-09-10T01:19:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.975+0000 7f80f6b80700 1 -- start start 2023-09-10T01:19:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f6b80700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80f00fee80 0x7f80f0197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f6b80700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80f0103fa0 0x7f80f0197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f6b80700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80f01a04f0 0x7f80f01a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f6b80700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f01a2eb0 con 0x7f80f00fee80 2023-09-10T01:19:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f6b80700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f01a3020 con 0x7f80f01a04f0 2023-09-10T01:19:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f6b80700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f01a31e0 con 0x7f80f0103fa0 2023-09-10T01:19:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f491c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80f00fee80 0x7f80f0197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80effff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80f0103fa0 0x7f80f0197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f491c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80f00fee80 0x7f80f0197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58384/0 (socket says 172.21.15.44:58384) 2023-09-10T01:19:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f491c700 1 -- 172.21.15.44:0/2319508191 learned_addr learned my addr 172.21.15.44:0/2319508191 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80effff700 1 --2- 172.21.15.44:0/2319508191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80f0103fa0 0x7f80f0197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37036/0 (socket says 172.21.15.44:37036) 2023-09-10T01:19:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f511d700 1 --2- 172.21.15.44:0/2319508191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80f01a04f0 0x7f80f01a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f491c700 1 -- 172.21.15.44:0/2319508191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80f0103fa0 msgr2=0x7f80f0197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f491c700 1 --2- 172.21.15.44:0/2319508191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80f0103fa0 0x7f80f0197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f491c700 1 -- 172.21.15.44:0/2319508191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80f01a04f0 msgr2=0x7f80f01a28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f491c700 1 --2- 172.21.15.44:0/2319508191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80f01a04f0 0x7f80f01a28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f491c700 1 -- 172.21.15.44:0/2319508191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e40095f0 con 0x7f80f00fee80 2023-09-10T01:19:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f511d700 1 --2- 172.21.15.44:0/2319508191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80f01a04f0 0x7f80f01a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:19:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.976+0000 7f80f491c700 1 --2- 172.21.15.44:0/2319508191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80f00fee80 0x7f80f0197660 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f80e4005770 tx=0x7f80e40057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.977+0000 7f80edffb700 1 -- 172.21.15.44:0/2319508191 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e401d070 con 0x7f80f00fee80 2023-09-10T01:19:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.977+0000 7f80edffb700 1 -- 172.21.15.44:0/2319508191 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80e4004040 con 0x7f80f00fee80 2023-09-10T01:19:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.977+0000 7f80edffb700 1 -- 172.21.15.44:0/2319508191 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e400fca0 con 0x7f80f00fee80 2023-09-10T01:19:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.977+0000 7f80f6b80700 1 -- 172.21.15.44:0/2319508191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f01a3460 con 0x7f80f00fee80 2023-09-10T01:19:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.977+0000 7f80f6b80700 1 -- 172.21.15.44:0/2319508191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f01a3830 con 0x7f80f00fee80 2023-09-10T01:19:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.979+0000 7f80edffb700 1 -- 172.21.15.44:0/2319508191 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f80e400f460 con 0x7f80f00fee80 2023-09-10T01:19:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.980+0000 7f80edffb700 1 --2- 172.21.15.44:0/2319508191 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80d80a07e0 0x7f80d80a2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.980+0000 7f80edffb700 1 -- 172.21.15.44:0/2319508191 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f80e40cd2f0 con 0x7f80f00fee80 2023-09-10T01:19:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.980+0000 7f80effff700 1 --2- 172.21.15.44:0/2319508191 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80d80a07e0 0x7f80d80a2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.980+0000 7f8102d46b80 1 Processor -- start 2023-09-10T01:19:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.980+0000 7f8102d46b80 1 -- start start 2023-09-10T01:19:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.980+0000 7f80effff700 1 --2- 172.21.15.44:0/2319508191 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80d80a07e0 0x7f80d80a2ca0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f80f010d470 tx=0x7f80e000a380 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.981+0000 7f8102d46b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc091a0 0x55f5dbc09580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.981+0000 7f8102d46b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc03300 0x55f5dbc03780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.981+0000 7f8102d46b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 0x55f5dbc0ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.981+0000 7f8102d46b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dbb5c6d0 con 0x55f5dbc03300 2023-09-10T01:19:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.981+0000 7f8102d46b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dbb59ff0 con 0x55f5dbc091a0 2023-09-10T01:19:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.981+0000 7f8102d46b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dbb558e0 con 0x55f5dbc03cc0 2023-09-10T01:19:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.981+0000 7f80effff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc03300 0x55f5dbc03780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.981+0000 7f80effff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc03300 0x55f5dbc03780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58394/0 (socket says 172.21.15.44:58394) 2023-09-10T01:19:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.981+0000 7f80effff700 1 -- 172.21.15.44:0/530140884 learned_addr learned my addr 172.21.15.44:0/530140884 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.981+0000 7f80f511d700 1 --2- 172.21.15.44:0/530140884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 0x55f5dbc0ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.981+0000 7f80effff700 1 -- 172.21.15.44:0/530140884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 msgr2=0x55f5dbc0ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.981+0000 7f80effff700 1 --2- 172.21.15.44:0/530140884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 0x55f5dbc0ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.981+0000 7f80effff700 1 -- 172.21.15.44:0/530140884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc091a0 msgr2=0x55f5dbc09580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.981+0000 7f80effff700 1 --2- 172.21.15.44:0/530140884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc091a0 0x55f5dbc09580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.981+0000 7f80effff700 1 -- 172.21.15.44:0/530140884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5db981d80 con 0x55f5dbc03300 2023-09-10T01:19:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.981+0000 7f80effff700 1 --2- 172.21.15.44:0/530140884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc03300 0x55f5dbc03780 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f80e000fe70 tx=0x7f80e0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728cd55cade5bffb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.982+0000 7f80d6ffd700 1 -- 172.21.15.44:0/530140884 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0003a90 con 0x55f5dbc03300 2023-09-10T01:19:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.982+0000 7f80d6ffd700 1 -- 172.21.15.44:0/530140884 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80e0003bf0 con 0x55f5dbc03300 2023-09-10T01:19:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.982+0000 7f80d6ffd700 1 -- 172.21.15.44:0/530140884 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e001d400 con 0x55f5dbc03300 2023-09-10T01:19:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.983+0000 7f8102d46b80 1 -- 172.21.15.44:0/530140884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc03300 msgr2=0x55f5dbc03780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.983+0000 7f8102d46b80 1 --2- 172.21.15.44:0/530140884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc03300 0x55f5dbc03780 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f80e000fe70 tx=0x7f80e0012000 comp rx=0 tx=0).stop 2023-09-10T01:19:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.983+0000 7f8102d46b80 1 -- 172.21.15.44:0/530140884 shutdown_connections 2023-09-10T01:19:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.983+0000 7f8102d46b80 1 --2- 172.21.15.44:0/530140884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 0x55f5dbc0ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.983+0000 7f8102d46b80 1 --2- 172.21.15.44:0/530140884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc091a0 0x55f5dbc09580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.983+0000 7f8102d46b80 1 --2- 172.21.15.44:0/530140884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc03300 0x55f5dbc03780 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.983+0000 7f8102d46b80 1 -- 172.21.15.44:0/530140884 >> 172.21.15.44:0/530140884 conn(0x55f5dbb78030 msgr2=0x55f5dbbfe560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.983+0000 7f8102d46b80 1 -- 172.21.15.44:0/530140884 shutdown_connections 2023-09-10T01:19:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.983+0000 7f8102d46b80 1 -- 172.21.15.44:0/530140884 wait complete. 2023-09-10T01:19:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.983+0000 7f8102d46b80 1 -- start start 2023-09-10T01:19:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.984+0000 7f8102d46b80 1 Processor -- start 2023-09-10T01:19:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.984+0000 7f8102d46b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc03300 0x55f5dbc0fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.984+0000 7f8102d46b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 0x55f5dbc0d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.984+0000 7f8102d46b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc091a0 0x55f5dbc0d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.984+0000 7f8102d46b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dbaf3b10 con 0x55f5dbc03300 2023-09-10T01:19:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.984+0000 7f8102d46b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dbaf3810 con 0x55f5dbc091a0 2023-09-10T01:19:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.984+0000 7f8102d46b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5db9d78c0 con 0x55f5dbc03cc0 2023-09-10T01:19:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.984+0000 7f8102d46b80 10 client.0 _ll_register_callbacks cb 0x55f5dba3da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:19:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.984+0000 7f80effff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 0x55f5dbc0d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.984+0000 7f80effff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 0x55f5dbc0d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37048/0 (socket says 172.21.15.44:37048) 2023-09-10T01:19:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.984+0000 7f80effff700 1 -- 172.21.15.44:0/26770324 learned_addr learned my addr 172.21.15.44:0/26770324 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.984+0000 7f80f511d700 1 --2- 172.21.15.44:0/26770324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc091a0 0x55f5dbc0d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.984+0000 7f80f491c700 1 --2- 172.21.15.44:0/26770324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc03300 0x55f5dbc0fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.985+0000 7f80effff700 1 -- 172.21.15.44:0/26770324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc091a0 msgr2=0x55f5dbc0d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.985+0000 7f80effff700 1 --2- 172.21.15.44:0/26770324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc091a0 0x55f5dbc0d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.985+0000 7f80effff700 1 -- 172.21.15.44:0/26770324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc03300 msgr2=0x55f5dbc0fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.985+0000 7f80effff700 1 --2- 172.21.15.44:0/26770324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc03300 0x55f5dbc0fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.985+0000 7f80effff700 1 -- 172.21.15.44:0/26770324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e000fb20 con 0x55f5dbc03cc0 2023-09-10T01:19:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.985+0000 7f80f511d700 1 --2- 172.21.15.44:0/26770324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc091a0 0x55f5dbc0d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:19:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.985+0000 7f80effff700 1 --2- 172.21.15.44:0/26770324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 0x55f5dbc0d180 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f80e0003f90 tx=0x7f80e000a9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9088942bab875e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.985+0000 7f80cffff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:19:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.985+0000 7f80cffff700 1 -- 172.21.15.44:0/26770324 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e002b070 con 0x55f5dbc03cc0 2023-09-10T01:19:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.985+0000 7f8102d46b80 1 -- 172.21.15.44:0/26770324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5dbaad490 con 0x55f5dbc03cc0 2023-09-10T01:19:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.985+0000 7f80cffff700 1 -- 172.21.15.44:0/26770324 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(167..167 src has 1..167) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f80e0010b50 con 0x55f5dbc03cc0 2023-09-10T01:19:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.985+0000 7f80cffff700 1 -- 172.21.15.44:0/26770324 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80e0010d10 con 0x55f5dbc03cc0 2023-09-10T01:19:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.985+0000 7f8102d46b80 10 client.29419 fetch_fsmap learned FSMap version 127 2023-09-10T01:19:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.985+0000 7f8102d46b80 1 -- 172.21.15.44:0/26770324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x55f5db96ac00 con 0x55f5dbc03cc0 2023-09-10T01:19:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.986+0000 7f80cffff700 1 -- 172.21.15.44:0/26770324 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f5dbaad490 con 0x55f5dbc03cc0 2023-09-10T01:19:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.986+0000 7f80cffff700 1 -- 172.21.15.44:0/26770324 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 127) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f80e001db50 con 0x55f5dbc03cc0 2023-09-10T01:19:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.986+0000 7f8102d46b80 10 client.29419 fetch_fsmap finished waiting for FSMap version 127 2023-09-10T01:19:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.986+0000 7f8102d46b80 10 client.29419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:19:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.986+0000 7f8102d46b80 20 client.29419 populate_metadata read hostname 'smithi044' 2023-09-10T01:19:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.986+0000 7f8102d46b80 1 --2- 172.21.15.44:0/26770324 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f5dbc1d4c0 0x55f5dbc1f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.986+0000 7f8102d46b80 4 client.29419 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:19:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.986+0000 7f8102d46b80 1 -- 172.21.15.44:0/26770324 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5dbb6a840 con 0x55f5dbc1d4c0 2023-09-10T01:19:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.986+0000 7f80f491c700 1 --2- 172.21.15.44:0/26770324 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f5dbc1d4c0 0x55f5dbc1f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.986+0000 7f80cffff700 10 client.29419 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.986+0000 7f80f491c700 1 --2- 172.21.15.44:0/26770324 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f5dbc1d4c0 0x55f5dbc1f8e0 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c194de0022a5830e server_cookie=1a4bce8f649d809b in_seq=0 out_seq=0 2023-09-10T01:19:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.987+0000 7f80cffff700 1 -- 172.21.15.44:0/26770324 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f5dbb6a840 con 0x55f5dbc1d4c0 2023-09-10T01:19:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.987+0000 7f80cffff700 10 client.29419 handle_command_reply: tid=0 2023-09-10T01:19:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.987+0000 7f8102d46b80 1 client.29419 shutdown 2023-09-10T01:19:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.987+0000 7f8102d46b80 1 -- 172.21.15.44:0/26770324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 msgr2=0x55f5dbc0d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.987+0000 7f8102d46b80 1 --2- 172.21.15.44:0/26770324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 0x55f5dbc0d180 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f80e0003f90 tx=0x7f80e000a9b0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.987+0000 7f8102d46b80 1 -- 172.21.15.44:0/26770324 shutdown_connections 2023-09-10T01:19:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.987+0000 7f8102d46b80 1 --2- 172.21.15.44:0/26770324 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f5dbc1d4c0 0x55f5dbc1f8e0 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.987+0000 7f8102d46b80 1 --2- 172.21.15.44:0/26770324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc091a0 0x55f5dbc0d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.987+0000 7f8102d46b80 1 --2- 172.21.15.44:0/26770324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc03300 0x55f5dbc0fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.987+0000 7f8102d46b80 1 --2- 172.21.15.44:0/26770324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 0x55f5dbc0d180 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.987+0000 7f80cffff700 0 client.29419 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.987+0000 7f8102d46b80 1 -- 172.21.15.44:0/26770324 >> 172.21.15.44:0/26770324 conn(0x55f5dbb78030 msgr2=0x55f5dbbfd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.987+0000 7f8102d46b80 1 -- 172.21.15.44:0/26770324 shutdown_connections 2023-09-10T01:19:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.987+0000 7f8102d46b80 1 -- 172.21.15.44:0/26770324 wait complete. 2023-09-10T01:19:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:42.988+0000 7f8102d46b80 20 client.29419 trim_cache size 0 max 16384 2023-09-10T01:19:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.009+0000 7f8102d46b80 1 Processor -- start 2023-09-10T01:19:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.009+0000 7f8102d46b80 1 -- start start 2023-09-10T01:19:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.009+0000 7f8102d46b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03300 0x55f5dbc0ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f8102d46b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc03cc0 0x55f5dbbfdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f8102d46b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc091a0 0x55f5dbbfe460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f8102d46b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dbaad490 con 0x55f5dbc091a0 2023-09-10T01:19:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f8102d46b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dbbf88a0 con 0x55f5dbc03cc0 2023-09-10T01:19:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f8102d46b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dbbfafc0 con 0x55f5dbc03300 2023-09-10T01:19:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f80f511d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc091a0 0x55f5dbbfe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f80f491c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03300 0x55f5dbc0ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f80f511d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc091a0 0x55f5dbbfe460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58410/0 (socket says 172.21.15.44:58410) 2023-09-10T01:19:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f80f511d700 1 -- 172.21.15.44:0/1150364279 learned_addr learned my addr 172.21.15.44:0/1150364279 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f80f491c700 1 --2- 172.21.15.44:0/1150364279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03300 0x55f5dbc0ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37056/0 (socket says 172.21.15.44:37056) 2023-09-10T01:19:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f80effff700 1 --2- 172.21.15.44:0/1150364279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc03cc0 0x55f5dbbfdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f80f491c700 1 -- 172.21.15.44:0/1150364279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc03cc0 msgr2=0x55f5dbbfdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f80f491c700 1 --2- 172.21.15.44:0/1150364279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc03cc0 0x55f5dbbfdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f80f491c700 1 -- 172.21.15.44:0/1150364279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc091a0 msgr2=0x55f5dbbfe460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f80f491c700 1 --2- 172.21.15.44:0/1150364279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc091a0 0x55f5dbbfe460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f80f491c700 1 -- 172.21.15.44:0/1150364279 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5db9a81f0 con 0x55f5dbc03300 2023-09-10T01:19:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f80f511d700 1 --2- 172.21.15.44:0/1150364279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc091a0 0x55f5dbbfe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:19:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.010+0000 7f80f491c700 1 --2- 172.21.15.44:0/1150364279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03300 0x55f5dbc0ae60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f80e40ca6a0 tx=0x7f80e40a0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9965de8f78582cdf server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.011+0000 7f80d77fe700 1 -- 172.21.15.44:0/1150364279 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e40cd150 con 0x55f5dbc03300 2023-09-10T01:19:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.011+0000 7f80d77fe700 1 -- 172.21.15.44:0/1150364279 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80e409ab90 con 0x55f5dbc03300 2023-09-10T01:19:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.011+0000 7f80d77fe700 1 -- 172.21.15.44:0/1150364279 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4098660 con 0x55f5dbc03300 2023-09-10T01:19:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.011+0000 7f8102d46b80 1 -- 172.21.15.44:0/1150364279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03300 msgr2=0x55f5dbc0ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.011+0000 7f8102d46b80 1 --2- 172.21.15.44:0/1150364279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03300 0x55f5dbc0ae60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f80e40ca6a0 tx=0x7f80e40a0040 comp rx=0 tx=0).stop 2023-09-10T01:19:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.011+0000 7f8102d46b80 1 -- 172.21.15.44:0/1150364279 shutdown_connections 2023-09-10T01:19:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.011+0000 7f8102d46b80 1 --2- 172.21.15.44:0/1150364279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc091a0 0x55f5dbbfe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.011+0000 7f8102d46b80 1 --2- 172.21.15.44:0/1150364279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03300 0x55f5dbc0ae60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.011+0000 7f8102d46b80 1 --2- 172.21.15.44:0/1150364279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc03cc0 0x55f5dbbfdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.011+0000 7f8102d46b80 1 -- 172.21.15.44:0/1150364279 >> 172.21.15.44:0/1150364279 conn(0x55f5dbb78030 msgr2=0x55f5dbc07290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.011+0000 7f8102d46b80 1 -- 172.21.15.44:0/1150364279 shutdown_connections 2023-09-10T01:19:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.011+0000 7f8102d46b80 1 -- 172.21.15.44:0/1150364279 wait complete. 2023-09-10T01:19:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.012+0000 7f8102d46b80 1 -- start start 2023-09-10T01:19:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.012+0000 7f8102d46b80 1 Processor -- start 2023-09-10T01:19:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.012+0000 7f8102d46b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc03300 0x55f5dbc12700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.012+0000 7f8102d46b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 0x55f5dbc12c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.012+0000 7f8102d46b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc091a0 0x55f5dbc6a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.012+0000 7f8102d46b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dbc6a830 con 0x55f5dbc091a0 2023-09-10T01:19:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.012+0000 7f8102d46b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dbc6a9a0 con 0x55f5dbc03300 2023-09-10T01:19:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.012+0000 7f8102d46b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dbc6ab60 con 0x55f5dbc03cc0 2023-09-10T01:19:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.012+0000 7f8102d46b80 10 client.0 _ll_register_callbacks cb 0x7f80e8009730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:19:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.012+0000 7f80effff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 0x55f5dbc12c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.012+0000 7f80effff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 0x55f5dbc12c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37064/0 (socket says 172.21.15.44:37064) 2023-09-10T01:19:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.012+0000 7f80f491c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc03300 0x55f5dbc12700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.012+0000 7f80effff700 1 -- 172.21.15.44:0/1567966607 learned_addr learned my addr 172.21.15.44:0/1567966607 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.012+0000 7f80f511d700 1 --2- 172.21.15.44:0/1567966607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc091a0 0x55f5dbc6a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.013+0000 7f80f491c700 1 -- 172.21.15.44:0/1567966607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 msgr2=0x55f5dbc12c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.013+0000 7f80f491c700 1 --2- 172.21.15.44:0/1567966607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 0x55f5dbc12c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.013+0000 7f80f491c700 1 -- 172.21.15.44:0/1567966607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc091a0 msgr2=0x55f5dbc6a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.013+0000 7f80f491c700 1 --2- 172.21.15.44:0/1567966607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc091a0 0x55f5dbc6a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.013+0000 7f80f491c700 1 -- 172.21.15.44:0/1567966607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e8009e70 con 0x55f5dbc03300 2023-09-10T01:19:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.013+0000 7f80effff700 1 --2- 172.21.15.44:0/1567966607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 0x55f5dbc12c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:19:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.013+0000 7f80f491c700 1 --2- 172.21.15.44:0/1567966607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc03300 0x55f5dbc12700 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f80e400c010 tx=0x7f80e4005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c427907972614951 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.013+0000 7f80cffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:19:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.013+0000 7f80cffff700 1 -- 172.21.15.44:0/1567966607 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e40a2470 con 0x55f5dbc03300 2023-09-10T01:19:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.013+0000 7f8102d46b80 1 -- 172.21.15.44:0/1567966607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80e0003bf0 con 0x55f5dbc03300 2023-09-10T01:19:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.013+0000 7f80cffff700 1 -- 172.21.15.44:0/1567966607 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(167..167 src has 1..167) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f80e409a490 con 0x55f5dbc03300 2023-09-10T01:19:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.013+0000 7f80cffff700 1 -- 172.21.15.44:0/1567966607 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80e40985d0 con 0x55f5dbc03300 2023-09-10T01:19:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.013+0000 7f8102d46b80 10 client.29444 fetch_fsmap learned FSMap version 127 2023-09-10T01:19:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.013+0000 7f8102d46b80 1 -- 172.21.15.44:0/1567966607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x7f80f01a3830 con 0x55f5dbc03300 2023-09-10T01:19:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.013+0000 7f80cffff700 1 -- 172.21.15.44:0/1567966607 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80e0003bf0 con 0x55f5dbc03300 2023-09-10T01:19:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.014+0000 7f80cffff700 1 -- 172.21.15.44:0/1567966607 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 127) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f80e40add20 con 0x55f5dbc03300 2023-09-10T01:19:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.014+0000 7f8102d46b80 10 client.29444 fetch_fsmap finished waiting for FSMap version 127 2023-09-10T01:19:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.014+0000 7f8102d46b80 10 client.29444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:19:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.014+0000 7f8102d46b80 20 client.29444 populate_metadata read hostname 'smithi044' 2023-09-10T01:19:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.014+0000 7f8102d46b80 1 --2- 172.21.15.44:0/1567966607 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f5dbc71fa0 0x55f5dbc743c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.014+0000 7f8102d46b80 4 client.29444 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:19:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.014+0000 7f8102d46b80 1 -- 172.21.15.44:0/1567966607 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f5dbb4e8c0 con 0x55f5dbc71fa0 2023-09-10T01:19:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.014+0000 7f80effff700 1 --2- 172.21.15.44:0/1567966607 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f5dbc71fa0 0x55f5dbc743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.014+0000 7f80effff700 1 --2- 172.21.15.44:0/1567966607 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f5dbc71fa0 0x55f5dbc743c0 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac149e504afb70d3 server_cookie=b5d96eda5496a262 in_seq=0 out_seq=0 2023-09-10T01:19:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.014+0000 7f80cffff700 10 client.29444 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.026+0000 7f80cffff700 1 -- 172.21.15.44:0/1567966607 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f5dbb4e8c0 con 0x55f5dbc71fa0 2023-09-10T01:19:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.026+0000 7f80cffff700 10 client.29444 handle_command_reply: tid=0 2023-09-10T01:19:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.026+0000 7f8102d46b80 1 client.29444 shutdown 2023-09-10T01:19:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.026+0000 7f8102d46b80 1 -- 172.21.15.44:0/1567966607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc03300 msgr2=0x55f5dbc12700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.026+0000 7f8102d46b80 1 --2- 172.21.15.44:0/1567966607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc03300 0x55f5dbc12700 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f80e400c010 tx=0x7f80e4005c00 comp rx=0 tx=0).stop 2023-09-10T01:19:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.026+0000 7f8102d46b80 1 -- 172.21.15.44:0/1567966607 shutdown_connections 2023-09-10T01:19:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.026+0000 7f8102d46b80 1 --2- 172.21.15.44:0/1567966607 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f5dbc71fa0 0x55f5dbc743c0 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.026+0000 7f8102d46b80 1 --2- 172.21.15.44:0/1567966607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5dbc091a0 0x55f5dbc6a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.026+0000 7f8102d46b80 1 --2- 172.21.15.44:0/1567966607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5dbc03300 0x55f5dbc12700 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.026+0000 7f80cffff700 0 client.29444 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.026+0000 7f8102d46b80 1 --2- 172.21.15.44:0/1567966607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f5dbc03cc0 0x55f5dbc12c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.026+0000 7f8102d46b80 1 -- 172.21.15.44:0/1567966607 >> 172.21.15.44:0/1567966607 conn(0x55f5dbb78030 msgr2=0x55f5dbc022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.026+0000 7f8102d46b80 1 -- 172.21.15.44:0/1567966607 shutdown_connections 2023-09-10T01:19:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.026+0000 7f8102d46b80 1 -- 172.21.15.44:0/1567966607 wait complete. 2023-09-10T01:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.026+0000 7f8102d46b80 20 client.29444 trim_cache size 0 max 16384 2023-09-10T01:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:19:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.027+0000 7f80f6b80700 1 -- 172.21.15.44:0/2319508191 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80d80a07e0 msgr2=0x7f80d80a2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.027+0000 7f80f6b80700 1 --2- 172.21.15.44:0/2319508191 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80d80a07e0 0x7f80d80a2ca0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f80f010d470 tx=0x7f80e000a380 comp rx=0 tx=0).stop 2023-09-10T01:19:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.027+0000 7f80f6b80700 1 -- 172.21.15.44:0/2319508191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80f00fee80 msgr2=0x7f80f0197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.027+0000 7f80f6b80700 1 --2- 172.21.15.44:0/2319508191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80f00fee80 0x7f80f0197660 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f80e4005770 tx=0x7f80e40057a0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.027+0000 7f80f6b80700 1 -- 172.21.15.44:0/2319508191 shutdown_connections 2023-09-10T01:19:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.027+0000 7f80f6b80700 1 --2- 172.21.15.44:0/2319508191 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f80d80a07e0 0x7f80d80a2ca0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.027+0000 7f80f6b80700 1 --2- 172.21.15.44:0/2319508191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80f01a04f0 0x7f80f01a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.027+0000 7f80f6b80700 1 --2- 172.21.15.44:0/2319508191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80f00fee80 0x7f80f0197660 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.027+0000 7f80f6b80700 1 --2- 172.21.15.44:0/2319508191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80f0103fa0 0x7f80f0197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.027+0000 7f80f6b80700 1 -- 172.21.15.44:0/2319508191 >> 172.21.15.44:0/2319508191 conn(0x7f80f00769d0 msgr2=0x7f80f0103650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.027+0000 7f80f6b80700 1 -- 172.21.15.44:0/2319508191 shutdown_connections 2023-09-10T01:19:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:43.027+0000 7f80f6b80700 1 -- 172.21.15.44:0/2319508191 wait complete. 2023-09-10T01:19:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:43 smithi099 ceph-mon[102692]: pgmap v1259: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 23 MiB/s wr, 143 op/s 2023-09-10T01:19:43.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:43 smithi131 ceph-mon[103713]: pgmap v1259: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 23 MiB/s wr, 143 op/s 2023-09-10T01:19:43.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:43 smithi044 ceph-mon[98676]: pgmap v1259: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 23 MiB/s wr, 143 op/s 2023-09-10T01:19:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:45 smithi099 ceph-mon[102692]: pgmap v1260: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 96 op/s 2023-09-10T01:19:45.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:45 smithi131 ceph-mon[103713]: pgmap v1260: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 96 op/s 2023-09-10T01:19:45.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:45 smithi044 ceph-mon[98676]: pgmap v1260: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 96 op/s 2023-09-10T01:19:47.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:47 smithi099 ceph-mon[102692]: pgmap v1261: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 21 MiB/s wr, 131 op/s 2023-09-10T01:19:47.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:47 smithi131 ceph-mon[103713]: pgmap v1261: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 21 MiB/s wr, 131 op/s 2023-09-10T01:19:47.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:47 smithi044 ceph-mon[98676]: pgmap v1261: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 21 MiB/s wr, 131 op/s 2023-09-10T01:19:49.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:49 smithi099 ceph-mon[102692]: pgmap v1262: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 102 op/s 2023-09-10T01:19:49.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:49 smithi131 ceph-mon[103713]: pgmap v1262: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 102 op/s 2023-09-10T01:19:49.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:49 smithi044 ceph-mon[98676]: pgmap v1262: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 102 op/s 2023-09-10T01:19:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:51 smithi099 ceph-mon[102692]: pgmap v1263: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 99 op/s 2023-09-10T01:19:51.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:51 smithi131 ceph-mon[103713]: pgmap v1263: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 99 op/s 2023-09-10T01:19:51.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:51 smithi044 ceph-mon[98676]: pgmap v1263: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 99 op/s 2023-09-10T01:19:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:53 smithi099 ceph-mon[102692]: pgmap v1264: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 24 MiB/s wr, 146 op/s 2023-09-10T01:19:53.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:53 smithi131 ceph-mon[103713]: pgmap v1264: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 24 MiB/s wr, 146 op/s 2023-09-10T01:19:53.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:53 smithi044 ceph-mon[98676]: pgmap v1264: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 24 MiB/s wr, 146 op/s 2023-09-10T01:19:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:55 smithi099 ceph-mon[102692]: pgmap v1265: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 16 MiB/s wr, 98 op/s 2023-09-10T01:19:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:55 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:19:55.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:55 smithi131 ceph-mon[103713]: pgmap v1265: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 16 MiB/s wr, 98 op/s 2023-09-10T01:19:55.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:55 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:19:55.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:55 smithi044 ceph-mon[98676]: pgmap v1265: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 16 MiB/s wr, 98 op/s 2023-09-10T01:19:55.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:55 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:19:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.195+0000 7fbe366d2700 1 -- 172.21.15.44:0/2089755981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe30104420 msgr2=0x7fbe30104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.195+0000 7fbe366d2700 1 --2- 172.21.15.44:0/2089755981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe30104420 0x7fbe30104800 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000dd20 tx=0x7fbe2000f330 comp rx=0 tx=0).stop 2023-09-10T01:19:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.196+0000 7fbe366d2700 1 -- 172.21.15.44:0/2089755981 shutdown_connections 2023-09-10T01:19:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.196+0000 7fbe366d2700 1 --2- 172.21.15.44:0/2089755981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe300ffe70 0x7fbe3010c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.196+0000 7fbe366d2700 1 --2- 172.21.15.44:0/2089755981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe30104420 0x7fbe30104800 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.196+0000 7fbe366d2700 1 --2- 172.21.15.44:0/2089755981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe300ff4b0 0x7fbe300ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.196+0000 7fbe366d2700 1 -- 172.21.15.44:0/2089755981 >> 172.21.15.44:0/2089755981 conn(0x7fbe30076600 msgr2=0x7fbe30076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.196+0000 7fbe366d2700 1 -- 172.21.15.44:0/2089755981 shutdown_connections 2023-09-10T01:19:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.196+0000 7fbe366d2700 1 -- 172.21.15.44:0/2089755981 wait complete. 2023-09-10T01:19:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.196+0000 7fbe366d2700 1 Processor -- start 2023-09-10T01:19:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.196+0000 7fbe366d2700 1 -- start start 2023-09-10T01:19:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.197+0000 7fbe366d2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe300ff4b0 0x7fbe3019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.197+0000 7fbe366d2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe300ffe70 0x7fbe3019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.197+0000 7fbe366d2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe30104420 0x7fbe30194430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.197+0000 7fbe366d2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30194a90 con 0x7fbe300ff4b0 2023-09-10T01:19:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.197+0000 7fbe366d2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30194c00 con 0x7fbe300ffe70 2023-09-10T01:19:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.197+0000 7fbe366d2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30194dc0 con 0x7fbe30104420 2023-09-10T01:19:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.198+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe300ff4b0 0x7fbe3019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.198+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe300ff4b0 0x7fbe3019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58422/0 (socket says 172.21.15.44:58422) 2023-09-10T01:19:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.198+0000 7fbe2ffff700 1 -- 172.21.15.44:0/1715561989 learned_addr learned my addr 172.21.15.44:0/1715561989 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.198+0000 7fbe2ffff700 1 -- 172.21.15.44:0/1715561989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe30104420 msgr2=0x7fbe30194430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T01:19:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.198+0000 7fbe2ffff700 1 --2- 172.21.15.44:0/1715561989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe30104420 0x7fbe30194430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.198+0000 7fbe2ffff700 1 -- 172.21.15.44:0/1715561989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe300ffe70 msgr2=0x7fbe3019aaa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T01:19:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.198+0000 7fbe2ffff700 1 --2- 172.21.15.44:0/1715561989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe300ffe70 0x7fbe3019aaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.198+0000 7fbe2ffff700 1 -- 172.21.15.44:0/1715561989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2000da00 con 0x7fbe300ff4b0 2023-09-10T01:19:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.198+0000 7fbe2ffff700 1 --2- 172.21.15.44:0/1715561989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe300ff4b0 0x7fbe3019a560 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fbe20006010 tx=0x7fbe20005490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.199+0000 7fbe2d7fa700 1 -- 172.21.15.44:0/1715561989 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20015070 con 0x7fbe300ff4b0 2023-09-10T01:19:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.199+0000 7fbe2d7fa700 1 -- 172.21.15.44:0/1715561989 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe20003b10 con 0x7fbe300ff4b0 2023-09-10T01:19:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.199+0000 7fbe366d2700 1 -- 172.21.15.44:0/1715561989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe30195040 con 0x7fbe300ff4b0 2023-09-10T01:19:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.199+0000 7fbe2d7fa700 1 -- 172.21.15.44:0/1715561989 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20018950 con 0x7fbe300ff4b0 2023-09-10T01:19:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.200+0000 7fbe366d2700 1 -- 172.21.15.44:0/1715561989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe301b2560 con 0x7fbe300ff4b0 2023-09-10T01:19:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.202+0000 7fbe2d7fa700 1 -- 172.21.15.44:0/1715561989 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fbe20003c80 con 0x7fbe300ff4b0 2023-09-10T01:19:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.203+0000 7fbe2d7fa700 1 --2- 172.21.15.44:0/1715561989 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbe180a08b0 0x7fbe180a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.203+0000 7fbe2d7fa700 1 -- 172.21.15.44:0/1715561989 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fbe200cd010 con 0x7fbe300ff4b0 2023-09-10T01:19:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.203+0000 7fbe2f7fe700 1 --2- 172.21.15.44:0/1715561989 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbe180a08b0 0x7fbe180a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.203+0000 7fbe2f7fe700 1 --2- 172.21.15.44:0/1715561989 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbe180a08b0 0x7fbe180a2d70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbe30195da0 tx=0x7fbe24006c60 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.204+0000 7fbe42898b80 1 Processor -- start 2023-09-10T01:19:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.206+0000 7fbe42898b80 1 -- start start 2023-09-10T01:19:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.206+0000 7fbe42898b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f81a0 0x56177d1f8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.206+0000 7fbe42898b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1f2300 0x56177d1f2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.206+0000 7fbe42898b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1f2cc0 0x56177d1fef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.206+0000 7fbe42898b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d14b6d0 con 0x56177d1f2300 2023-09-10T01:19:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.206+0000 7fbe42898b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d148ff0 con 0x56177d1f2cc0 2023-09-10T01:19:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.206+0000 7fbe42898b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d1448e0 con 0x56177d1f81a0 2023-09-10T01:19:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.206+0000 7fbe34c6f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1f2cc0 0x56177d1fef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.206+0000 7fbe34c6f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1f2cc0 0x56177d1fef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60166/0 (socket says 172.21.15.44:60166) 2023-09-10T01:19:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.206+0000 7fbe34c6f700 1 -- 172.21.15.44:0/562218821 learned_addr learned my addr 172.21.15.44:0/562218821 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.206+0000 7fbe34c6f700 1 -- 172.21.15.44:0/562218821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f81a0 msgr2=0x56177d1f8580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:19:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.206+0000 7fbe34c6f700 1 --2- 172.21.15.44:0/562218821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f81a0 0x56177d1f8580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.207+0000 7fbe34c6f700 1 -- 172.21.15.44:0/562218821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1f2300 msgr2=0x56177d1f2780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:19:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.207+0000 7fbe34c6f700 1 --2- 172.21.15.44:0/562218821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1f2300 0x56177d1f2780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.207+0000 7fbe34c6f700 1 -- 172.21.15.44:0/562218821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56177cf70d80 con 0x56177d1f2cc0 2023-09-10T01:19:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.207+0000 7fbe34c6f700 1 --2- 172.21.15.44:0/562218821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1f2cc0 0x56177d1fef30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009d00 tx=0x7fbe2800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd12fc3e7e944869 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.207+0000 7fbe167fc700 1 -- 172.21.15.44:0/562218821 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2800e070 con 0x56177d1f2cc0 2023-09-10T01:19:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.207+0000 7fbe167fc700 1 -- 172.21.15.44:0/562218821 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe28009e50 con 0x56177d1f2cc0 2023-09-10T01:19:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.207+0000 7fbe167fc700 1 -- 172.21.15.44:0/562218821 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28011390 con 0x56177d1f2cc0 2023-09-10T01:19:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.208+0000 7fbe42898b80 1 -- 172.21.15.44:0/562218821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1f2cc0 msgr2=0x56177d1fef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.208+0000 7fbe42898b80 1 --2- 172.21.15.44:0/562218821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1f2cc0 0x56177d1fef30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009d00 tx=0x7fbe2800b330 comp rx=0 tx=0).stop 2023-09-10T01:19:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.208+0000 7fbe42898b80 1 -- 172.21.15.44:0/562218821 shutdown_connections 2023-09-10T01:19:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.208+0000 7fbe42898b80 1 --2- 172.21.15.44:0/562218821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1f2cc0 0x56177d1fef30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.208+0000 7fbe42898b80 1 --2- 172.21.15.44:0/562218821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f81a0 0x56177d1f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.208+0000 7fbe42898b80 1 --2- 172.21.15.44:0/562218821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1f2300 0x56177d1f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.208+0000 7fbe42898b80 1 -- 172.21.15.44:0/562218821 >> 172.21.15.44:0/562218821 conn(0x56177d167030 msgr2=0x56177d1ed560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.209+0000 7fbe42898b80 1 -- 172.21.15.44:0/562218821 shutdown_connections 2023-09-10T01:19:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.209+0000 7fbe42898b80 1 -- 172.21.15.44:0/562218821 wait complete. 2023-09-10T01:19:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.209+0000 7fbe42898b80 1 -- start start 2023-09-10T01:19:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe42898b80 1 Processor -- start 2023-09-10T01:19:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe42898b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f2cc0 0x56177d1fc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe42898b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1ff010 0x56177d1fc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe42898b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1fcee0 0x56177d2085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe42898b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d0e2b10 con 0x56177d1fcee0 2023-09-10T01:19:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe42898b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d0e2810 con 0x56177d1ff010 2023-09-10T01:19:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe42898b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177cfc68c0 con 0x56177d1f2cc0 2023-09-10T01:19:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe42898b80 10 client.0 _ll_register_callbacks cb 0x56177d02ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:19:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f2cc0 0x56177d1fc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe34c6f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1fcee0 0x56177d2085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f2cc0 0x56177d1fc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37078/0 (socket says 172.21.15.44:37078) 2023-09-10T01:19:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe2ffff700 1 -- 172.21.15.44:0/1830052884 learned_addr learned my addr 172.21.15.44:0/1830052884 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe2f7fe700 1 --2- 172.21.15.44:0/1830052884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1ff010 0x56177d1fc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe2ffff700 1 -- 172.21.15.44:0/1830052884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1ff010 msgr2=0x56177d1fc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe2ffff700 1 --2- 172.21.15.44:0/1830052884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1ff010 0x56177d1fc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe2ffff700 1 -- 172.21.15.44:0/1830052884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1fcee0 msgr2=0x56177d2085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe2ffff700 1 --2- 172.21.15.44:0/1830052884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1fcee0 0x56177d2085e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe2ffff700 1 -- 172.21.15.44:0/1830052884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe280099b0 con 0x56177d1f2cc0 2023-09-10T01:19:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.210+0000 7fbe34c6f700 1 --2- 172.21.15.44:0/1830052884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1fcee0 0x56177d2085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:19:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.211+0000 7fbe2ffff700 1 --2- 172.21.15.44:0/1830052884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f2cc0 0x56177d1fc300 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbe200ca3c0 tx=0x7fbe20099f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ba462ca1c128c0b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.211+0000 7fbe0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:19:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.211+0000 7fbe0f7fe700 1 -- 172.21.15.44:0/1830052884 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2009a5b0 con 0x56177d1f2cc0 2023-09-10T01:19:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.211+0000 7fbe0f7fe700 1 -- 172.21.15.44:0/1830052884 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(167..167 src has 1..167) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fbe20018260 con 0x56177d1f2cc0 2023-09-10T01:19:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.211+0000 7fbe42898b80 1 -- 172.21.15.44:0/1830052884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56177d09c490 con 0x56177d1f2cc0 2023-09-10T01:19:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.211+0000 7fbe0f7fe700 1 -- 172.21.15.44:0/1830052884 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe20099ab0 con 0x56177d1f2cc0 2023-09-10T01:19:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.211+0000 7fbe42898b80 10 client.29431 fetch_fsmap learned FSMap version 128 2023-09-10T01:19:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.211+0000 7fbe42898b80 1 -- 172.21.15.44:0/1830052884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=128,monmap=4+}) v3 -- 0x56177cf59c00 con 0x56177d1f2cc0 2023-09-10T01:19:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.211+0000 7fbe0f7fe700 1 -- 172.21.15.44:0/1830052884 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56177d09c490 con 0x56177d1f2cc0 2023-09-10T01:19:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.211+0000 7fbe0f7fe700 1 -- 172.21.15.44:0/1830052884 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 128) v1 ==== 7323+0+0 (secure 0 0 0) 0x7fbe20099640 con 0x56177d1f2cc0 2023-09-10T01:19:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.212+0000 7fbe42898b80 10 client.29431 fetch_fsmap finished waiting for FSMap version 128 2023-09-10T01:19:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.212+0000 7fbe42898b80 10 client.29431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:19:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.212+0000 7fbe42898b80 20 client.29431 populate_metadata read hostname 'smithi044' 2023-09-10T01:19:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.212+0000 7fbe42898b80 1 --2- 172.21.15.44:0/1830052884 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56177d20c8b0 0x56177d20eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.212+0000 7fbe42898b80 4 client.29431 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:19:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.212+0000 7fbe42898b80 1 -- 172.21.15.44:0/1830052884 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56177d159840 con 0x56177d20c8b0 2023-09-10T01:19:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.212+0000 7fbe2f7fe700 1 --2- 172.21.15.44:0/1830052884 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56177d20c8b0 0x56177d20eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.212+0000 7fbe0f7fe700 10 client.29431 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.212+0000 7fbe2f7fe700 1 --2- 172.21.15.44:0/1830052884 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56177d20c8b0 0x56177d20eca0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca6db935a9814a18 server_cookie=b641fe29f425be48 in_seq=0 out_seq=0 2023-09-10T01:19:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.213+0000 7fbe0f7fe700 1 -- 172.21.15.44:0/1830052884 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56177d159840 con 0x56177d20c8b0 2023-09-10T01:19:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.213+0000 7fbe0f7fe700 10 client.29431 handle_command_reply: tid=0 2023-09-10T01:19:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.213+0000 7fbe42898b80 1 client.29431 shutdown 2023-09-10T01:19:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.213+0000 7fbe42898b80 1 -- 172.21.15.44:0/1830052884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f2cc0 msgr2=0x56177d1fc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.213+0000 7fbe42898b80 1 --2- 172.21.15.44:0/1830052884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f2cc0 0x56177d1fc300 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbe200ca3c0 tx=0x7fbe20099f20 comp rx=0 tx=0).stop 2023-09-10T01:19:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.213+0000 7fbe42898b80 1 -- 172.21.15.44:0/1830052884 shutdown_connections 2023-09-10T01:19:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.213+0000 7fbe42898b80 1 --2- 172.21.15.44:0/1830052884 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56177d20c8b0 0x56177d20eca0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.213+0000 7fbe42898b80 1 --2- 172.21.15.44:0/1830052884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1fcee0 0x56177d2085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.213+0000 7fbe42898b80 1 --2- 172.21.15.44:0/1830052884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f2cc0 0x56177d1fc300 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.213+0000 7fbe42898b80 1 --2- 172.21.15.44:0/1830052884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1ff010 0x56177d1fc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.213+0000 7fbe42898b80 1 -- 172.21.15.44:0/1830052884 >> 172.21.15.44:0/1830052884 conn(0x56177d167030 msgr2=0x56177d1ec7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.213+0000 7fbe0f7fe700 0 client.29431 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.213+0000 7fbe42898b80 1 -- 172.21.15.44:0/1830052884 shutdown_connections 2023-09-10T01:19:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.213+0000 7fbe42898b80 1 -- 172.21.15.44:0/1830052884 wait complete. 2023-09-10T01:19:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.214+0000 7fbe42898b80 20 client.29431 trim_cache size 0 max 16384 2023-09-10T01:19:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.232+0000 7fbe42898b80 1 Processor -- start 2023-09-10T01:19:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.232+0000 7fbe42898b80 1 -- start start 2023-09-10T01:19:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.233+0000 7fbe42898b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1f2cc0 0x56177d1f9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.233+0000 7fbe42898b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1fcee0 0x56177d1ed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.233+0000 7fbe42898b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1ff010 0x56177d1ed5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.233+0000 7fbe42898b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d1edb10 con 0x56177d1f2cc0 2023-09-10T01:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.233+0000 7fbe42898b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d159870 con 0x56177d1fcee0 2023-09-10T01:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.233+0000 7fbe42898b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d1f3140 con 0x56177d1ff010 2023-09-10T01:19:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.233+0000 7fbe34c6f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1ff010 0x56177d1ed5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.233+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1f2cc0 0x56177d1f9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.233+0000 7fbe2f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1fcee0 0x56177d1ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.233+0000 7fbe34c6f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1ff010 0x56177d1ed5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37090/0 (socket says 172.21.15.44:37090) 2023-09-10T01:19:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.233+0000 7fbe34c6f700 1 -- 172.21.15.44:0/581416513 learned_addr learned my addr 172.21.15.44:0/581416513 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.233+0000 7fbe34c6f700 1 -- 172.21.15.44:0/581416513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1fcee0 msgr2=0x56177d1ed090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.233+0000 7fbe34c6f700 1 --2- 172.21.15.44:0/581416513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1fcee0 0x56177d1ed090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.233+0000 7fbe34c6f700 1 -- 172.21.15.44:0/581416513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1f2cc0 msgr2=0x56177d1f9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.233+0000 7fbe34c6f700 1 --2- 172.21.15.44:0/581416513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1f2cc0 0x56177d1f9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.233+0000 7fbe34c6f700 1 -- 172.21.15.44:0/581416513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56177d2138d0 con 0x56177d1ff010 2023-09-10T01:19:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.234+0000 7fbe34c6f700 1 --2- 172.21.15.44:0/581416513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1ff010 0x56177d1ed5d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009ca0 tx=0x7fbe28009470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14a9496b9981926b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.234+0000 7fbe16ffd700 1 -- 172.21.15.44:0/581416513 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2800e040 con 0x56177d1ff010 2023-09-10T01:19:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.234+0000 7fbe16ffd700 1 -- 172.21.15.44:0/581416513 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe280125e0 con 0x56177d1ff010 2023-09-10T01:19:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.234+0000 7fbe16ffd700 1 -- 172.21.15.44:0/581416513 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280043a0 con 0x56177d1ff010 2023-09-10T01:19:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.235+0000 7fbe42898b80 1 -- 172.21.15.44:0/581416513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1ff010 msgr2=0x56177d1ed5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.235+0000 7fbe42898b80 1 --2- 172.21.15.44:0/581416513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1ff010 0x56177d1ed5d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009ca0 tx=0x7fbe28009470 comp rx=0 tx=0).stop 2023-09-10T01:19:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.235+0000 7fbe42898b80 1 -- 172.21.15.44:0/581416513 shutdown_connections 2023-09-10T01:19:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.235+0000 7fbe42898b80 1 --2- 172.21.15.44:0/581416513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1ff010 0x56177d1ed5d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.235+0000 7fbe42898b80 1 --2- 172.21.15.44:0/581416513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1f2cc0 0x56177d1f9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.235+0000 7fbe42898b80 1 --2- 172.21.15.44:0/581416513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1fcee0 0x56177d1ed090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.235+0000 7fbe42898b80 1 -- 172.21.15.44:0/581416513 >> 172.21.15.44:0/581416513 conn(0x56177d167030 msgr2=0x56177d1f6290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.235+0000 7fbe42898b80 1 -- 172.21.15.44:0/581416513 shutdown_connections 2023-09-10T01:19:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.235+0000 7fbe42898b80 1 -- 172.21.15.44:0/581416513 wait complete. 2023-09-10T01:19:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.235+0000 7fbe42898b80 1 -- start start 2023-09-10T01:19:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.235+0000 7fbe42898b80 1 Processor -- start 2023-09-10T01:19:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe42898b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f2cc0 0x56177d203720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe42898b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1fcee0 0x56177d201510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe42898b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1ff010 0x56177d201b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe42898b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d202270 con 0x56177d1fcee0 2023-09-10T01:19:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe42898b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d2023e0 con 0x56177d1ff010 2023-09-10T01:19:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe42898b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d25f730 con 0x56177d1f2cc0 2023-09-10T01:19:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe42898b80 10 client.0 _ll_register_callbacks cb 0x56177d02ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:19:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f2cc0 0x56177d203720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f2cc0 0x56177d203720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37092/0 (socket says 172.21.15.44:37092) 2023-09-10T01:19:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe2ffff700 1 -- 172.21.15.44:0/694107920 learned_addr learned my addr 172.21.15.44:0/694107920 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:19:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe34c6f700 1 --2- 172.21.15.44:0/694107920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1ff010 0x56177d201b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe2ffff700 1 -- 172.21.15.44:0/694107920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1ff010 msgr2=0x56177d201b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe2ffff700 1 --2- 172.21.15.44:0/694107920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1ff010 0x56177d201b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe2ffff700 1 -- 172.21.15.44:0/694107920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1fcee0 msgr2=0x56177d201510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:19:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe2ffff700 1 --2- 172.21.15.44:0/694107920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1fcee0 0x56177d201510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe2ffff700 1 -- 172.21.15.44:0/694107920 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56177cf70d80 con 0x56177d1f2cc0 2023-09-10T01:19:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe2ffff700 1 --2- 172.21.15.44:0/694107920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f2cc0 0x56177d203720 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbe2009a0e0 tx=0x7fbe20005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75a116c66f39e9c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:19:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.236+0000 7fbe0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:19:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.237+0000 7fbe0f7fe700 1 -- 172.21.15.44:0/694107920 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2007c070 con 0x56177d1f2cc0 2023-09-10T01:19:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.237+0000 7fbe0f7fe700 1 -- 172.21.15.44:0/694107920 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(167..167 src has 1..167) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fbe20018260 con 0x56177d1f2cc0 2023-09-10T01:19:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.237+0000 7fbe42898b80 1 -- 172.21.15.44:0/694107920 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe28009e50 con 0x56177d1f2cc0 2023-09-10T01:19:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.237+0000 7fbe0f7fe700 1 -- 172.21.15.44:0/694107920 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe200ca7f0 con 0x56177d1f2cc0 2023-09-10T01:19:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.237+0000 7fbe42898b80 10 client.29437 fetch_fsmap learned FSMap version 128 2023-09-10T01:19:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.237+0000 7fbe42898b80 1 -- 172.21.15.44:0/694107920 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=128,monmap=4+}) v3 -- 0x56177cf971f0 con 0x56177d1f2cc0 2023-09-10T01:19:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.237+0000 7fbe0f7fe700 1 -- 172.21.15.44:0/694107920 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe28009e50 con 0x56177d1f2cc0 2023-09-10T01:19:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.237+0000 7fbe0f7fe700 1 -- 172.21.15.44:0/694107920 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 128) v1 ==== 7323+0+0 (secure 0 0 0) 0x7fbe2007f020 con 0x56177d1f2cc0 2023-09-10T01:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.237+0000 7fbe42898b80 10 client.29437 fetch_fsmap finished waiting for FSMap version 128 2023-09-10T01:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.237+0000 7fbe42898b80 10 client.29437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.237+0000 7fbe42898b80 20 client.29437 populate_metadata read hostname 'smithi044' 2023-09-10T01:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.237+0000 7fbe42898b80 1 --2- 172.21.15.44:0/694107920 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56177d260990 0x56177d262d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.237+0000 7fbe42898b80 4 client.29437 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:19:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.237+0000 7fbe42898b80 1 -- 172.21.15.44:0/694107920 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56177d13d8c0 con 0x56177d260990 2023-09-10T01:19:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.237+0000 7fbe2f7fe700 1 --2- 172.21.15.44:0/694107920 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56177d260990 0x56177d262d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:19:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.238+0000 7fbe0f7fe700 10 client.29437 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.238+0000 7fbe2f7fe700 1 --2- 172.21.15.44:0/694107920 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56177d260990 0x56177d262d80 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf265e576360dd0e server_cookie=4a1f6a000053f6a1 in_seq=0 out_seq=0 2023-09-10T01:19:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.256+0000 7fbe0f7fe700 1 -- 172.21.15.44:0/694107920 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56177d13d8c0 con 0x56177d260990 2023-09-10T01:19:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.256+0000 7fbe0f7fe700 10 client.29437 handle_command_reply: tid=0 2023-09-10T01:19:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.256+0000 7fbe42898b80 1 client.29437 shutdown 2023-09-10T01:19:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.256+0000 7fbe42898b80 1 -- 172.21.15.44:0/694107920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f2cc0 msgr2=0x56177d203720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.256+0000 7fbe42898b80 1 --2- 172.21.15.44:0/694107920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f2cc0 0x56177d203720 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbe2009a0e0 tx=0x7fbe20005b00 comp rx=0 tx=0).stop 2023-09-10T01:19:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.256+0000 7fbe42898b80 1 -- 172.21.15.44:0/694107920 shutdown_connections 2023-09-10T01:19:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.256+0000 7fbe42898b80 1 --2- 172.21.15.44:0/694107920 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56177d260990 0x56177d262d80 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.256+0000 7fbe42898b80 1 --2- 172.21.15.44:0/694107920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56177d1ff010 0x56177d201b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.256+0000 7fbe42898b80 1 --2- 172.21.15.44:0/694107920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177d1f2cc0 0x56177d203720 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.256+0000 7fbe0f7fe700 0 client.29437 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:19:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.256+0000 7fbe42898b80 1 --2- 172.21.15.44:0/694107920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56177d1fcee0 0x56177d201510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.256+0000 7fbe42898b80 1 -- 172.21.15.44:0/694107920 >> 172.21.15.44:0/694107920 conn(0x56177d167030 msgr2=0x56177d1f6290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.257+0000 7fbe42898b80 1 -- 172.21.15.44:0/694107920 shutdown_connections 2023-09-10T01:19:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.257+0000 7fbe42898b80 1 -- 172.21.15.44:0/694107920 wait complete. 2023-09-10T01:19:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.257+0000 7fbe42898b80 20 client.29437 trim_cache size 0 max 16384 2023-09-10T01:19:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:19:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:19:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:19:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:19:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.257+0000 7fbe366d2700 1 -- 172.21.15.44:0/1715561989 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbe180a08b0 msgr2=0x7fbe180a2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.257+0000 7fbe366d2700 1 --2- 172.21.15.44:0/1715561989 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbe180a08b0 0x7fbe180a2d70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbe30195da0 tx=0x7fbe24006c60 comp rx=0 tx=0).stop 2023-09-10T01:19:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.257+0000 7fbe366d2700 1 -- 172.21.15.44:0/1715561989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe300ff4b0 msgr2=0x7fbe3019a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:19:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.257+0000 7fbe366d2700 1 --2- 172.21.15.44:0/1715561989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe300ff4b0 0x7fbe3019a560 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fbe20006010 tx=0x7fbe20005490 comp rx=0 tx=0).stop 2023-09-10T01:19:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.257+0000 7fbe366d2700 1 -- 172.21.15.44:0/1715561989 shutdown_connections 2023-09-10T01:19:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.257+0000 7fbe366d2700 1 --2- 172.21.15.44:0/1715561989 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbe180a08b0 0x7fbe180a2d70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.257+0000 7fbe366d2700 1 --2- 172.21.15.44:0/1715561989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe30104420 0x7fbe30194430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.257+0000 7fbe366d2700 1 --2- 172.21.15.44:0/1715561989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe300ff4b0 0x7fbe3019a560 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.257+0000 7fbe366d2700 1 --2- 172.21.15.44:0/1715561989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe300ffe70 0x7fbe3019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:19:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.257+0000 7fbe366d2700 1 -- 172.21.15.44:0/1715561989 >> 172.21.15.44:0/1715561989 conn(0x7fbe30076600 msgr2=0x7fbe300fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:19:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.258+0000 7fbe366d2700 1 -- 172.21.15.44:0/1715561989 shutdown_connections 2023-09-10T01:19:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:19:56.258+0000 7fbe366d2700 1 -- 172.21.15.44:0/1715561989 wait complete. 2023-09-10T01:19:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:57 smithi099 ceph-mon[102692]: pgmap v1266: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 22 MiB/s wr, 140 op/s 2023-09-10T01:19:57.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:57 smithi131 ceph-mon[103713]: pgmap v1266: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 22 MiB/s wr, 140 op/s 2023-09-10T01:19:57.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:57 smithi044 ceph-mon[98676]: pgmap v1266: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 22 MiB/s wr, 140 op/s 2023-09-10T01:19:58.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:58 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:19:58.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:58 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:19:58.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:58 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:19:59.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:59 smithi099 ceph-mon[102692]: pgmap v1267: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 113 op/s 2023-09-10T01:19:59.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:59 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:19:59.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:59 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:19:59.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:19:59 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:19:59.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:59 smithi131 ceph-mon[103713]: pgmap v1267: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 113 op/s 2023-09-10T01:19:59.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:59 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:19:59.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:59 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:19:59.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:19:59 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:19:59.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:59 smithi044 ceph-mon[98676]: pgmap v1267: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 113 op/s 2023-09-10T01:19:59.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:59 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:19:59.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:59 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:19:59.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:19:59 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:20:00.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:00 smithi099 ceph-mon[102692]: pgmap v1268: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 102 op/s 2023-09-10T01:20:00.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:00 smithi099 ceph-mon[102692]: pgmap v1269: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 20 MiB/s wr, 124 op/s 2023-09-10T01:20:00.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:00 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:20:00.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:00 smithi099 ceph-mon[102692]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2023-09-10T01:20:00.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:00 smithi099 ceph-mon[102692]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2023-09-10T01:20:00.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:00 smithi099 ceph-mon[102692]: daemon mds.c on smithi131 is in error state 2023-09-10T01:20:00.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:00 smithi131 ceph-mon[103713]: pgmap v1268: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 102 op/s 2023-09-10T01:20:00.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:00 smithi131 ceph-mon[103713]: pgmap v1269: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 20 MiB/s wr, 124 op/s 2023-09-10T01:20:00.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:00 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:20:00.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:00 smithi131 ceph-mon[103713]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2023-09-10T01:20:00.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:00 smithi131 ceph-mon[103713]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2023-09-10T01:20:00.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:00 smithi131 ceph-mon[103713]: daemon mds.c on smithi131 is in error state 2023-09-10T01:20:00.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:00 smithi044 ceph-mon[98676]: pgmap v1268: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 102 op/s 2023-09-10T01:20:00.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:00 smithi044 ceph-mon[98676]: pgmap v1269: 129 pgs: 129 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 20 MiB/s wr, 124 op/s 2023-09-10T01:20:00.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:00 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:20:00.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:00 smithi044 ceph-mon[98676]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2023-09-10T01:20:00.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:00 smithi044 ceph-mon[98676]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2023-09-10T01:20:00.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:00 smithi044 ceph-mon[98676]: daemon mds.c on smithi131 is in error state 2023-09-10T01:20:02.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:02 smithi099 ceph-mon[102692]: pgmap v1270: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 123 op/s 2023-09-10T01:20:02.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:02 smithi131 ceph-mon[103713]: pgmap v1270: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 123 op/s 2023-09-10T01:20:02.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:02 smithi044 ceph-mon[98676]: pgmap v1270: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 123 op/s 2023-09-10T01:20:04.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:04 smithi099 ceph-mon[102692]: pgmap v1271: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 123 op/s 2023-09-10T01:20:04.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:04 smithi131 ceph-mon[103713]: pgmap v1271: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 123 op/s 2023-09-10T01:20:04.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:04 smithi044 ceph-mon[98676]: pgmap v1271: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 123 op/s 2023-09-10T01:20:05.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:05 smithi131 ceph-mon[103713]: osdmap e168: 12 total, 12 up, 12 in 2023-09-10T01:20:05.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:05 smithi044 ceph-mon[98676]: osdmap e168: 12 total, 12 up, 12 in 2023-09-10T01:20:05.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:05 smithi099 ceph-mon[102692]: osdmap e168: 12 total, 12 up, 12 in 2023-09-10T01:20:06.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:06 smithi131 ceph-mon[103713]: pgmap v1273: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 75 op/s 2023-09-10T01:20:06.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:06 smithi044 ceph-mon[98676]: pgmap v1273: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 75 op/s 2023-09-10T01:20:06.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:06 smithi099 ceph-mon[102692]: pgmap v1273: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 75 op/s 2023-09-10T01:20:08.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:08 smithi131 ceph-mon[103713]: pgmap v1274: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 21 MiB/s wr, 158 op/s 2023-09-10T01:20:08.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:08 smithi044 ceph-mon[98676]: pgmap v1274: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 21 MiB/s wr, 158 op/s 2023-09-10T01:20:08.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:08 smithi099 ceph-mon[102692]: pgmap v1274: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 21 MiB/s wr, 158 op/s 2023-09-10T01:20:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.386+0000 7f5f7afac700 1 -- 172.21.15.44:0/3351735134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f74100120 msgr2=0x7f5f7410c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.386+0000 7f5f7afac700 1 --2- 172.21.15.44:0/3351735134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f74100120 0x7f5f7410c330 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00b0d0 tx=0x7f5f6c00b3e0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.386+0000 7f5f7afac700 1 -- 172.21.15.44:0/3351735134 shutdown_connections 2023-09-10T01:20:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.386+0000 7f5f7afac700 1 --2- 172.21.15.44:0/3351735134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f74100120 0x7f5f7410c330 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.386+0000 7f5f7afac700 1 --2- 172.21.15.44:0/3351735134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f741055b0 0x7f5f74105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.386+0000 7f5f7afac700 1 --2- 172.21.15.44:0/3351735134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f740ff760 0x7f5f740ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.386+0000 7f5f7afac700 1 -- 172.21.15.44:0/3351735134 >> 172.21.15.44:0/3351735134 conn(0x7f5f74074f30 msgr2=0x7f5f74075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.386+0000 7f5f7afac700 1 -- 172.21.15.44:0/3351735134 shutdown_connections 2023-09-10T01:20:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.386+0000 7f5f7afac700 1 -- 172.21.15.44:0/3351735134 wait complete. 2023-09-10T01:20:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.387+0000 7f5f7afac700 1 Processor -- start 2023-09-10T01:20:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.387+0000 7f5f7afac700 1 -- start start 2023-09-10T01:20:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f7afac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f740ff760 0x7f5f7419d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f7afac700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f74100120 0x7f5f7419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f7afac700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f741055b0 0x7f5f741a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f7afac700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f741a1f60 con 0x7f5f74100120 2023-09-10T01:20:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f7afac700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f741a20d0 con 0x7f5f740ff760 2023-09-10T01:20:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f7afac700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f741a2290 con 0x7f5f741055b0 2023-09-10T01:20:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f73fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f74100120 0x7f5f7419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f73fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f74100120 0x7f5f7419d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58456/0 (socket says 172.21.15.44:58456) 2023-09-10T01:20:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f73fff700 1 -- 172.21.15.44:0/2753996704 learned_addr learned my addr 172.21.15.44:0/2753996704 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f78d48700 1 --2- 172.21.15.44:0/2753996704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f740ff760 0x7f5f7419d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f79549700 1 --2- 172.21.15.44:0/2753996704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f741055b0 0x7f5f741a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f73fff700 1 -- 172.21.15.44:0/2753996704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f741055b0 msgr2=0x7f5f741a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f73fff700 1 --2- 172.21.15.44:0/2753996704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f741055b0 0x7f5f741a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f73fff700 1 -- 172.21.15.44:0/2753996704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f740ff760 msgr2=0x7f5f7419d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f73fff700 1 --2- 172.21.15.44:0/2753996704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f740ff760 0x7f5f7419d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f73fff700 1 -- 172.21.15.44:0/2753996704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c009d00 con 0x7f5f74100120 2023-09-10T01:20:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f78d48700 1 --2- 172.21.15.44:0/2753996704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f740ff760 0x7f5f7419d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:20:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.388+0000 7f5f73fff700 1 --2- 172.21.15.44:0/2753996704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f74100120 0x7f5f7419d7d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e9a0 tx=0x7f5f6000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.389+0000 7f5f71ffb700 1 -- 172.21.15.44:0/2753996704 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60009790 con 0x7f5f74100120 2023-09-10T01:20:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.389+0000 7f5f71ffb700 1 -- 172.21.15.44:0/2753996704 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f60010430 con 0x7f5f74100120 2023-09-10T01:20:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.389+0000 7f5f7afac700 1 -- 172.21.15.44:0/2753996704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f741a2570 con 0x7f5f74100120 2023-09-10T01:20:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.389+0000 7f5f71ffb700 1 -- 172.21.15.44:0/2753996704 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6000cea0 con 0x7f5f74100120 2023-09-10T01:20:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.389+0000 7f5f7afac700 1 -- 172.21.15.44:0/2753996704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f741a9db0 con 0x7f5f74100120 2023-09-10T01:20:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.391+0000 7f5f71ffb700 1 -- 172.21.15.44:0/2753996704 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f5f600105a0 con 0x7f5f74100120 2023-09-10T01:20:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.392+0000 7f5f71ffb700 1 --2- 172.21.15.44:0/2753996704 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5f640a07b0 0x7f5f640a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.392+0000 7f5f78d48700 1 --2- 172.21.15.44:0/2753996704 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5f640a07b0 0x7f5f640a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.393+0000 7f5f71ffb700 1 -- 172.21.15.44:0/2753996704 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f5f60014070 con 0x7f5f74100120 2023-09-10T01:20:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.393+0000 7f5f78d48700 1 --2- 172.21.15.44:0/2753996704 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5f640a07b0 0x7f5f640a2c70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5f68005950 tx=0x7f5f6800b410 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.394+0000 7f5f87172b80 1 Processor -- start 2023-09-10T01:20:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.394+0000 7f5f87172b80 1 -- start start 2023-09-10T01:20:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f87172b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834faa1a0 0x55c834faa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f87172b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4300 0x55c834fa4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f87172b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834fa4cc0 0x55c834fb0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f87172b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c834efd6d0 con 0x55c834fa4300 2023-09-10T01:20:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f87172b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c834efaff0 con 0x55c834faa1a0 2023-09-10T01:20:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f87172b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c834ef68e0 con 0x55c834fa4cc0 2023-09-10T01:20:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f73fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4300 0x55c834fa4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f73fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4300 0x55c834fa4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58464/0 (socket says 172.21.15.44:58464) 2023-09-10T01:20:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f73fff700 1 -- 172.21.15.44:0/2800269723 learned_addr learned my addr 172.21.15.44:0/2800269723 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f79549700 1 --2- 172.21.15.44:0/2800269723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834fa4cc0 0x55c834fb0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f78d48700 1 --2- 172.21.15.44:0/2800269723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834faa1a0 0x55c834faa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f73fff700 1 -- 172.21.15.44:0/2800269723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834fa4cc0 msgr2=0x55c834fb0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f73fff700 1 --2- 172.21.15.44:0/2800269723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834fa4cc0 0x55c834fb0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f73fff700 1 -- 172.21.15.44:0/2800269723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834faa1a0 msgr2=0x55c834faa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f73fff700 1 --2- 172.21.15.44:0/2800269723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834faa1a0 0x55c834faa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f73fff700 1 -- 172.21.15.44:0/2800269723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c834d22d80 con 0x55c834fa4300 2023-09-10T01:20:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f79549700 1 --2- 172.21.15.44:0/2800269723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834fa4cc0 0x55c834fb0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:20:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f73fff700 1 --2- 172.21.15.44:0/2800269723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4300 0x55c834fa4780 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f5f60096060 tx=0x7f5f60098f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b63375cb02bbbc server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.395+0000 7f5f5effd700 1 -- 172.21.15.44:0/2800269723 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60099d30 con 0x55c834fa4300 2023-09-10T01:20:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.396+0000 7f5f5effd700 1 -- 172.21.15.44:0/2800269723 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f600995f0 con 0x55c834fa4300 2023-09-10T01:20:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.396+0000 7f5f5effd700 1 -- 172.21.15.44:0/2800269723 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f600cc170 con 0x55c834fa4300 2023-09-10T01:20:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.396+0000 7f5f87172b80 1 -- 172.21.15.44:0/2800269723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4300 msgr2=0x55c834fa4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.396+0000 7f5f87172b80 1 --2- 172.21.15.44:0/2800269723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4300 0x55c834fa4780 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f5f60096060 tx=0x7f5f60098f50 comp rx=0 tx=0).stop 2023-09-10T01:20:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.396+0000 7f5f87172b80 1 -- 172.21.15.44:0/2800269723 shutdown_connections 2023-09-10T01:20:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.396+0000 7f5f87172b80 1 --2- 172.21.15.44:0/2800269723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834fa4cc0 0x55c834fb0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.396+0000 7f5f87172b80 1 --2- 172.21.15.44:0/2800269723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834faa1a0 0x55c834faa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.396+0000 7f5f87172b80 1 --2- 172.21.15.44:0/2800269723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4300 0x55c834fa4780 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.396+0000 7f5f87172b80 1 -- 172.21.15.44:0/2800269723 >> 172.21.15.44:0/2800269723 conn(0x55c834f19030 msgr2=0x55c834f9f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.396+0000 7f5f87172b80 1 -- 172.21.15.44:0/2800269723 shutdown_connections 2023-09-10T01:20:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.396+0000 7f5f87172b80 1 -- 172.21.15.44:0/2800269723 wait complete. 2023-09-10T01:20:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.397+0000 7f5f87172b80 1 -- start start 2023-09-10T01:20:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.397+0000 7f5f87172b80 1 Processor -- start 2023-09-10T01:20:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.397+0000 7f5f87172b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4300 0x55c834fb0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.397+0000 7f5f87172b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834fa4cc0 0x55c834fae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.397+0000 7f5f87172b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834faa1a0 0x55c834fae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.397+0000 7f5f87172b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c834e94b10 con 0x55c834fa4300 2023-09-10T01:20:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.397+0000 7f5f87172b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c834e94810 con 0x55c834fa4cc0 2023-09-10T01:20:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.397+0000 7f5f87172b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c834d788c0 con 0x55c834faa1a0 2023-09-10T01:20:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.397+0000 7f5f87172b80 10 client.0 _ll_register_callbacks cb 0x55c834ddea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:20:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.397+0000 7f5f78d48700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4300 0x55c834fb0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.397+0000 7f5f78d48700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4300 0x55c834fb0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58468/0 (socket says 172.21.15.44:58468) 2023-09-10T01:20:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.397+0000 7f5f78d48700 1 -- 172.21.15.44:0/1603696800 learned_addr learned my addr 172.21.15.44:0/1603696800 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.397+0000 7f5f79549700 1 --2- 172.21.15.44:0/1603696800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834faa1a0 0x55c834fae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.397+0000 7f5f73fff700 1 --2- 172.21.15.44:0/1603696800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834fa4cc0 0x55c834fae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.398+0000 7f5f78d48700 1 -- 172.21.15.44:0/1603696800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834faa1a0 msgr2=0x55c834fae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.398+0000 7f5f78d48700 1 --2- 172.21.15.44:0/1603696800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834faa1a0 0x55c834fae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.398+0000 7f5f78d48700 1 -- 172.21.15.44:0/1603696800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834fa4cc0 msgr2=0x55c834fae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.398+0000 7f5f78d48700 1 --2- 172.21.15.44:0/1603696800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834fa4cc0 0x55c834fae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.398+0000 7f5f78d48700 1 -- 172.21.15.44:0/1603696800 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7410cbc0 con 0x55c834fa4300 2023-09-10T01:20:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.398+0000 7f5f73fff700 1 --2- 172.21.15.44:0/1603696800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834fa4cc0 0x55c834fae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:20:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.398+0000 7f5f78d48700 1 --2- 172.21.15.44:0/1603696800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4300 0x55c834fb0e80 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f5f680119d0 tx=0x7f5f680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c88e682cd95e2d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.398+0000 7f5f53fff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:20:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.398+0000 7f5f53fff700 1 -- 172.21.15.44:0/1603696800 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6801f840 con 0x55c834fa4300 2023-09-10T01:20:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.398+0000 7f5f53fff700 1 -- 172.21.15.44:0/1603696800 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(168..168 src has 1..168) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f5f68017070 con 0x55c834fa4300 2023-09-10T01:20:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.398+0000 7f5f87172b80 1 -- 172.21.15.44:0/1603696800 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c834e4e490 con 0x55c834fa4300 2023-09-10T01:20:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.398+0000 7f5f53fff700 1 -- 172.21.15.44:0/1603696800 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f6801f9a0 con 0x55c834fa4300 2023-09-10T01:20:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.398+0000 7f5f87172b80 10 client.19848 fetch_fsmap learned FSMap version 130 2023-09-10T01:20:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.398+0000 7f5f87172b80 1 -- 172.21.15.44:0/1603696800 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=130,monmap=4+}) v3 -- 0x7f5f741a9db0 con 0x55c834fa4300 2023-09-10T01:20:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.398+0000 7f5f53fff700 1 -- 172.21.15.44:0/1603696800 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c834e4e490 con 0x55c834fa4300 2023-09-10T01:20:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.399+0000 7f5f53fff700 1 -- 172.21.15.44:0/1603696800 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 130) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f5f68028b30 con 0x55c834fa4300 2023-09-10T01:20:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.399+0000 7f5f87172b80 10 client.19848 fetch_fsmap finished waiting for FSMap version 130 2023-09-10T01:20:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.399+0000 7f5f87172b80 10 client.19848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:20:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.399+0000 7f5f87172b80 20 client.19848 populate_metadata read hostname 'smithi044' 2023-09-10T01:20:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.399+0000 7f5f87172b80 1 --2- 172.21.15.44:0/1603696800 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c834fbe4b0 0x55c834fc08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.399+0000 7f5f87172b80 4 client.19848 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:20:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.399+0000 7f5f87172b80 1 -- 172.21.15.44:0/1603696800 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c834f0b840 con 0x55c834fbe4b0 2023-09-10T01:20:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.399+0000 7f5f73fff700 1 --2- 172.21.15.44:0/1603696800 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c834fbe4b0 0x55c834fc08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.399+0000 7f5f53fff700 10 client.19848 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.399+0000 7f5f73fff700 1 --2- 172.21.15.44:0/1603696800 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c834fbe4b0 0x55c834fc08d0 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=221df242afa0b504 server_cookie=ef4a202e19e89ee2 in_seq=0 out_seq=0 2023-09-10T01:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.400+0000 7f5f53fff700 1 -- 172.21.15.44:0/1603696800 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c834f0b840 con 0x55c834fbe4b0 2023-09-10T01:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.400+0000 7f5f53fff700 10 client.19848 handle_command_reply: tid=0 2023-09-10T01:20:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.400+0000 7f5f87172b80 1 client.19848 shutdown 2023-09-10T01:20:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.400+0000 7f5f87172b80 1 -- 172.21.15.44:0/1603696800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4300 msgr2=0x55c834fb0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.400+0000 7f5f87172b80 1 --2- 172.21.15.44:0/1603696800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4300 0x55c834fb0e80 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f5f680119d0 tx=0x7f5f680047c0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.400+0000 7f5f87172b80 1 -- 172.21.15.44:0/1603696800 shutdown_connections 2023-09-10T01:20:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.400+0000 7f5f87172b80 1 --2- 172.21.15.44:0/1603696800 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c834fbe4b0 0x55c834fc08d0 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.400+0000 7f5f87172b80 1 --2- 172.21.15.44:0/1603696800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834faa1a0 0x55c834fae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.400+0000 7f5f53fff700 0 client.19848 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:20:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.400+0000 7f5f87172b80 1 --2- 172.21.15.44:0/1603696800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4300 0x55c834fb0e80 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.400+0000 7f5f87172b80 1 --2- 172.21.15.44:0/1603696800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834fa4cc0 0x55c834fae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.400+0000 7f5f87172b80 1 -- 172.21.15.44:0/1603696800 >> 172.21.15.44:0/1603696800 conn(0x55c834f19030 msgr2=0x55c834f9e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.400+0000 7f5f87172b80 1 -- 172.21.15.44:0/1603696800 shutdown_connections 2023-09-10T01:20:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.400+0000 7f5f87172b80 1 -- 172.21.15.44:0/1603696800 wait complete. 2023-09-10T01:20:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.401+0000 7f5f87172b80 20 client.19848 trim_cache size 0 max 16384 2023-09-10T01:20:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.420+0000 7f5f87172b80 1 Processor -- start 2023-09-10T01:20:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.420+0000 7f5f87172b80 1 -- start start 2023-09-10T01:20:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.420+0000 7f5f87172b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834fa4300 0x55c834fb0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.420+0000 7f5f87172b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4cc0 0x55c834fb0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.420+0000 7f5f87172b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834faa1a0 0x55c834fa8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.420+0000 7f5f87172b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c834fa7db0 con 0x55c834fa4cc0 2023-09-10T01:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.420+0000 7f5f87172b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c834f9bfc0 con 0x55c834faa1a0 2023-09-10T01:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.420+0000 7f5f87172b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c834fa5560 con 0x55c834fa4300 2023-09-10T01:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.421+0000 7f5f73fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4cc0 0x55c834fb0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.421+0000 7f5f73fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4cc0 0x55c834fb0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58478/0 (socket says 172.21.15.44:58478) 2023-09-10T01:20:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.421+0000 7f5f73fff700 1 -- 172.21.15.44:0/1575204665 learned_addr learned my addr 172.21.15.44:0/1575204665 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.421+0000 7f5f78d48700 1 --2- 172.21.15.44:0/1575204665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834fa4300 0x55c834fb0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.421+0000 7f5f79549700 1 --2- 172.21.15.44:0/1575204665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834faa1a0 0x55c834fa8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.421+0000 7f5f73fff700 1 -- 172.21.15.44:0/1575204665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834fa4300 msgr2=0x55c834fb0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.421+0000 7f5f73fff700 1 --2- 172.21.15.44:0/1575204665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834fa4300 0x55c834fb0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.421+0000 7f5f73fff700 1 -- 172.21.15.44:0/1575204665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834faa1a0 msgr2=0x55c834fa8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.421+0000 7f5f73fff700 1 --2- 172.21.15.44:0/1575204665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834faa1a0 0x55c834fa8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.421+0000 7f5f73fff700 1 -- 172.21.15.44:0/1575204665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c834d0bc00 con 0x55c834fa4cc0 2023-09-10T01:20:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.421+0000 7f5f73fff700 1 --2- 172.21.15.44:0/1575204665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4cc0 0x55c834fb0e80 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f5f600992f0 tx=0x7f5f60010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec0206cf423f75c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.421+0000 7f5f5f7fe700 1 -- 172.21.15.44:0/1575204665 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f600a1660 con 0x55c834fa4cc0 2023-09-10T01:20:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.422+0000 7f5f5f7fe700 1 -- 172.21.15.44:0/1575204665 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f60097990 con 0x55c834fa4cc0 2023-09-10T01:20:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.422+0000 7f5f5f7fe700 1 -- 172.21.15.44:0/1575204665 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f600cc170 con 0x55c834fa4cc0 2023-09-10T01:20:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.422+0000 7f5f87172b80 1 -- 172.21.15.44:0/1575204665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4cc0 msgr2=0x55c834fb0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.422+0000 7f5f87172b80 1 --2- 172.21.15.44:0/1575204665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4cc0 0x55c834fb0e80 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f5f600992f0 tx=0x7f5f60010b80 comp rx=0 tx=0).stop 2023-09-10T01:20:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.422+0000 7f5f87172b80 1 -- 172.21.15.44:0/1575204665 shutdown_connections 2023-09-10T01:20:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.422+0000 7f5f87172b80 1 --2- 172.21.15.44:0/1575204665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834faa1a0 0x55c834fa8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.422+0000 7f5f87172b80 1 --2- 172.21.15.44:0/1575204665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834fa4300 0x55c834fb0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.422+0000 7f5f87172b80 1 --2- 172.21.15.44:0/1575204665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4cc0 0x55c834fb0e80 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.422+0000 7f5f87172b80 1 -- 172.21.15.44:0/1575204665 >> 172.21.15.44:0/1575204665 conn(0x55c834f19030 msgr2=0x55c834fa9140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.422+0000 7f5f87172b80 1 -- 172.21.15.44:0/1575204665 shutdown_connections 2023-09-10T01:20:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.422+0000 7f5f87172b80 1 -- 172.21.15.44:0/1575204665 wait complete. 2023-09-10T01:20:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.423+0000 7f5f87172b80 1 -- start start 2023-09-10T01:20:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.423+0000 7f5f87172b80 1 Processor -- start 2023-09-10T01:20:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.423+0000 7f5f87172b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834fa4300 0x55c834fb3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.423+0000 7f5f87172b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4cc0 0x55c834fb3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.423+0000 7f5f87172b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834faa1a0 0x55c8350112e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.423+0000 7f5f87172b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c835011820 con 0x55c834fa4cc0 2023-09-10T01:20:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.423+0000 7f5f87172b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c835011960 con 0x55c834fa4300 2023-09-10T01:20:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.423+0000 7f5f87172b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c835011aa0 con 0x55c834faa1a0 2023-09-10T01:20:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.423+0000 7f5f73fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4cc0 0x55c834fb3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.423+0000 7f5f87172b80 10 client.0 _ll_register_callbacks cb 0x7f5f68011580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:20:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.423+0000 7f5f73fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4cc0 0x55c834fb3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58484/0 (socket says 172.21.15.44:58484) 2023-09-10T01:20:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.423+0000 7f5f73fff700 1 -- 172.21.15.44:0/2429248804 learned_addr learned my addr 172.21.15.44:0/2429248804 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.423+0000 7f5f78d48700 1 --2- 172.21.15.44:0/2429248804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834fa4300 0x55c834fb3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.423+0000 7f5f79549700 1 --2- 172.21.15.44:0/2429248804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834faa1a0 0x55c8350112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.424+0000 7f5f78d48700 1 -- 172.21.15.44:0/2429248804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834faa1a0 msgr2=0x55c8350112e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.424+0000 7f5f78d48700 1 --2- 172.21.15.44:0/2429248804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834faa1a0 0x55c8350112e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.424+0000 7f5f78d48700 1 -- 172.21.15.44:0/2429248804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4cc0 msgr2=0x55c834fb3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.424+0000 7f5f78d48700 1 --2- 172.21.15.44:0/2429248804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4cc0 0x55c834fb3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.424+0000 7f5f78d48700 1 -- 172.21.15.44:0/2429248804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c834d22d80 con 0x55c834fa4300 2023-09-10T01:20:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.424+0000 7f5f73fff700 1 --2- 172.21.15.44:0/2429248804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4cc0 0x55c834fb3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:20:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.424+0000 7f5f78d48700 1 --2- 172.21.15.44:0/2429248804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834fa4300 0x55c834fb3700 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5f68028e90 tx=0x7f5f6801fda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d477beb2b16139 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.424+0000 7f5f53fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:20:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.424+0000 7f5f53fff700 1 -- 172.21.15.44:0/2429248804 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f68004220 con 0x55c834fa4300 2023-09-10T01:20:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.424+0000 7f5f53fff700 1 -- 172.21.15.44:0/2429248804 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(168..168 src has 1..168) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f5f68017070 con 0x55c834fa4300 2023-09-10T01:20:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.424+0000 7f5f87172b80 1 -- 172.21.15.44:0/2429248804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f600995f0 con 0x55c834fa4300 2023-09-10T01:20:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.424+0000 7f5f53fff700 1 -- 172.21.15.44:0/2429248804 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f68011d90 con 0x55c834fa4300 2023-09-10T01:20:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.424+0000 7f5f87172b80 10 client.29471 fetch_fsmap learned FSMap version 130 2023-09-10T01:20:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.424+0000 7f5f87172b80 1 -- 172.21.15.44:0/2429248804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=130,monmap=4+}) v3 -- 0x55c834d491f0 con 0x55c834fa4300 2023-09-10T01:20:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.425+0000 7f5f53fff700 1 -- 172.21.15.44:0/2429248804 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f600995f0 con 0x55c834fa4300 2023-09-10T01:20:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.425+0000 7f5f53fff700 1 -- 172.21.15.44:0/2429248804 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 130) v1 ==== 7315+0+0 (secure 0 0 0) 0x7f5f68003b10 con 0x55c834fa4300 2023-09-10T01:20:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.425+0000 7f5f87172b80 10 client.29471 fetch_fsmap finished waiting for FSMap version 130 2023-09-10T01:20:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.425+0000 7f5f87172b80 10 client.29471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:20:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.425+0000 7f5f87172b80 20 client.29471 populate_metadata read hostname 'smithi044' 2023-09-10T01:20:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.425+0000 7f5f87172b80 1 --2- 172.21.15.44:0/2429248804 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c835013220 0x55c835015640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.425+0000 7f5f87172b80 4 client.29471 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:20:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.425+0000 7f5f87172b80 1 -- 172.21.15.44:0/2429248804 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c834eef8c0 con 0x55c835013220 2023-09-10T01:20:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.425+0000 7f5f73fff700 1 --2- 172.21.15.44:0/2429248804 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c835013220 0x55c835015640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.425+0000 7f5f73fff700 1 --2- 172.21.15.44:0/2429248804 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c835013220 0x55c835015640 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a23d06882016cd3 server_cookie=6d175ce77d4d0990 in_seq=0 out_seq=0 2023-09-10T01:20:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.425+0000 7f5f53fff700 10 client.29471 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:20:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.449+0000 7f5f53fff700 1 -- 172.21.15.44:0/2429248804 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c834eef8c0 con 0x55c835013220 2023-09-10T01:20:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.449+0000 7f5f53fff700 10 client.29471 handle_command_reply: tid=0 2023-09-10T01:20:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.449+0000 7f5f87172b80 1 client.29471 shutdown 2023-09-10T01:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.450+0000 7f5f87172b80 1 -- 172.21.15.44:0/2429248804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834fa4300 msgr2=0x55c834fb3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.450+0000 7f5f87172b80 1 --2- 172.21.15.44:0/2429248804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834fa4300 0x55c834fb3700 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5f68028e90 tx=0x7f5f6801fda0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.450+0000 7f5f87172b80 1 -- 172.21.15.44:0/2429248804 shutdown_connections 2023-09-10T01:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.450+0000 7f5f87172b80 1 --2- 172.21.15.44:0/2429248804 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c835013220 0x55c835015640 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.450+0000 7f5f87172b80 1 --2- 172.21.15.44:0/2429248804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c834faa1a0 0x55c8350112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.450+0000 7f5f87172b80 1 --2- 172.21.15.44:0/2429248804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c834fa4300 0x55c834fb3700 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.450+0000 7f5f87172b80 1 --2- 172.21.15.44:0/2429248804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c834fa4cc0 0x55c834fb3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.450+0000 7f5f53fff700 0 client.29471 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.450+0000 7f5f87172b80 1 -- 172.21.15.44:0/2429248804 >> 172.21.15.44:0/2429248804 conn(0x55c834f19030 msgr2=0x55c834fa9140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.450+0000 7f5f87172b80 1 -- 172.21.15.44:0/2429248804 shutdown_connections 2023-09-10T01:20:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.450+0000 7f5f87172b80 1 -- 172.21.15.44:0/2429248804 wait complete. 2023-09-10T01:20:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.450+0000 7f5f87172b80 20 client.29471 trim_cache size 0 max 16384 2023-09-10T01:20:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:20:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:20:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:20:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:20:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.450+0000 7f5f7afac700 1 -- 172.21.15.44:0/2753996704 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5f640a07b0 msgr2=0x7f5f640a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.450+0000 7f5f7afac700 1 --2- 172.21.15.44:0/2753996704 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5f640a07b0 0x7f5f640a2c70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5f68005950 tx=0x7f5f6800b410 comp rx=0 tx=0).stop 2023-09-10T01:20:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.450+0000 7f5f7afac700 1 -- 172.21.15.44:0/2753996704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f74100120 msgr2=0x7f5f7419d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.450+0000 7f5f7afac700 1 --2- 172.21.15.44:0/2753996704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f74100120 0x7f5f7419d7d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e9a0 tx=0x7f5f6000ecb0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.451+0000 7f5f7afac700 1 -- 172.21.15.44:0/2753996704 shutdown_connections 2023-09-10T01:20:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.451+0000 7f5f7afac700 1 --2- 172.21.15.44:0/2753996704 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5f640a07b0 0x7f5f640a2c70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.451+0000 7f5f7afac700 1 --2- 172.21.15.44:0/2753996704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f741055b0 0x7f5f741a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.451+0000 7f5f7afac700 1 --2- 172.21.15.44:0/2753996704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f740ff760 0x7f5f7419d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.451+0000 7f5f7afac700 1 --2- 172.21.15.44:0/2753996704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f74100120 0x7f5f7419d7d0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.451+0000 7f5f7afac700 1 -- 172.21.15.44:0/2753996704 >> 172.21.15.44:0/2753996704 conn(0x7f5f74074f30 msgr2=0x7f5f740679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.451+0000 7f5f7afac700 1 -- 172.21.15.44:0/2753996704 shutdown_connections 2023-09-10T01:20:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:09.451+0000 7f5f7afac700 1 -- 172.21.15.44:0/2753996704 wait complete. 2023-09-10T01:20:10.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:10 smithi131 ceph-mon[103713]: pgmap v1275: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 17 MiB/s wr, 126 op/s 2023-09-10T01:20:10.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:10 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:20:10.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:10 smithi044 ceph-mon[98676]: pgmap v1275: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 17 MiB/s wr, 126 op/s 2023-09-10T01:20:10.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:10 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:20:10.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:10 smithi099 ceph-mon[102692]: pgmap v1275: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 17 MiB/s wr, 126 op/s 2023-09-10T01:20:10.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:10 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:20:12.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:12 smithi131 ceph-mon[103713]: pgmap v1276: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 17 MiB/s wr, 130 op/s 2023-09-10T01:20:12.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:12 smithi044 ceph-mon[98676]: pgmap v1276: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 17 MiB/s wr, 130 op/s 2023-09-10T01:20:12.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:12 smithi099 ceph-mon[102692]: pgmap v1276: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 17 MiB/s wr, 130 op/s 2023-09-10T01:20:14.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:14 smithi131 ceph-mon[103713]: pgmap v1277: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 17 MiB/s wr, 130 op/s 2023-09-10T01:20:14.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:14 smithi044 ceph-mon[98676]: pgmap v1277: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 17 MiB/s wr, 130 op/s 2023-09-10T01:20:14.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:14 smithi099 ceph-mon[102692]: pgmap v1277: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 17 MiB/s wr, 130 op/s 2023-09-10T01:20:15.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:15 smithi131 ceph-mon[103713]: pgmap v1278: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 16 MiB/s wr, 123 op/s 2023-09-10T01:20:15.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:15 smithi044 ceph-mon[98676]: pgmap v1278: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 16 MiB/s wr, 123 op/s 2023-09-10T01:20:15.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:15 smithi099 ceph-mon[102692]: pgmap v1278: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 16 MiB/s wr, 123 op/s 2023-09-10T01:20:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:17 smithi131 ceph-mon[103713]: osdmap e169: 12 total, 12 up, 12 in 2023-09-10T01:20:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:17 smithi044 ceph-mon[98676]: osdmap e169: 12 total, 12 up, 12 in 2023-09-10T01:20:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:17 smithi099 ceph-mon[102692]: osdmap e169: 12 total, 12 up, 12 in 2023-09-10T01:20:18.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:18 smithi131 ceph-mon[103713]: pgmap v1280: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 19 MiB/s wr, 120 op/s 2023-09-10T01:20:18.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:18 smithi044 ceph-mon[98676]: pgmap v1280: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 19 MiB/s wr, 120 op/s 2023-09-10T01:20:18.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:18 smithi099 ceph-mon[102692]: pgmap v1280: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 19 MiB/s wr, 120 op/s 2023-09-10T01:20:20.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:20 smithi131 ceph-mon[103713]: pgmap v1281: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 19 MiB/s wr, 120 op/s 2023-09-10T01:20:20.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:20 smithi044 ceph-mon[98676]: pgmap v1281: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 19 MiB/s wr, 120 op/s 2023-09-10T01:20:20.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:20 smithi099 ceph-mon[102692]: pgmap v1281: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 19 MiB/s wr, 120 op/s 2023-09-10T01:20:22.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:22 smithi131 ceph-mon[103713]: pgmap v1282: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 19 MiB/s wr, 121 op/s 2023-09-10T01:20:22.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:22 smithi044 ceph-mon[98676]: pgmap v1282: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 19 MiB/s wr, 121 op/s 2023-09-10T01:20:22.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:22 smithi099 ceph-mon[102692]: pgmap v1282: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 19 MiB/s wr, 121 op/s 2023-09-10T01:20:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.581+0000 7f35e1eed700 1 -- 172.21.15.44:0/801174904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35dc100f90 msgr2=0x7f35dc10d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.581+0000 7f35e1eed700 1 --2- 172.21.15.44:0/801174904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35dc100f90 0x7f35dc10d450 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f35d400b0d0 tx=0x7f35d400b3e0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.581+0000 7f35e1eed700 1 -- 172.21.15.44:0/801174904 shutdown_connections 2023-09-10T01:20:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.581+0000 7f35e1eed700 1 --2- 172.21.15.44:0/801174904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35dc100f90 0x7f35dc10d450 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.581+0000 7f35e1eed700 1 --2- 172.21.15.44:0/801174904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35dc106690 0x7f35dc106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.581+0000 7f35e1eed700 1 --2- 172.21.15.44:0/801174904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35dc1005d0 0x7f35dc100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.581+0000 7f35e1eed700 1 -- 172.21.15.44:0/801174904 >> 172.21.15.44:0/801174904 conn(0x7f35dc0fc0c0 msgr2=0x7f35dc0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.581+0000 7f35e1eed700 1 -- 172.21.15.44:0/801174904 shutdown_connections 2023-09-10T01:20:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.581+0000 7f35e1eed700 1 -- 172.21.15.44:0/801174904 wait complete. 2023-09-10T01:20:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.582+0000 7f35e1eed700 1 Processor -- start 2023-09-10T01:20:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.582+0000 7f35e1eed700 1 -- start start 2023-09-10T01:20:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.582+0000 7f35e1eed700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35dc1005d0 0x7f35dc196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.582+0000 7f35e1eed700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35dc100f90 0x7f35dc196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.582+0000 7f35e1eed700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35dc106690 0x7f35dc19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.582+0000 7f35e1eed700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc19b900 con 0x7f35dc106690 2023-09-10T01:20:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.582+0000 7f35e1eed700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc19ba70 con 0x7f35dc100f90 2023-09-10T01:20:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.582+0000 7f35e1eed700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc19bc30 con 0x7f35dc1005d0 2023-09-10T01:20:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35db7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35dc1005d0 0x7f35dc196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35dbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35dc106690 0x7f35dc19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35db7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35dc1005d0 0x7f35dc196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37146/0 (socket says 172.21.15.44:37146) 2023-09-10T01:20:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35db7fe700 1 -- 172.21.15.44:0/2014045204 learned_addr learned my addr 172.21.15.44:0/2014045204 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35dbfff700 1 --2- 172.21.15.44:0/2014045204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35dc106690 0x7f35dc19b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58498/0 (socket says 172.21.15.44:58498) 2023-09-10T01:20:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35daffd700 1 --2- 172.21.15.44:0/2014045204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35dc100f90 0x7f35dc196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35db7fe700 1 -- 172.21.15.44:0/2014045204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35dc100f90 msgr2=0x7f35dc196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35db7fe700 1 --2- 172.21.15.44:0/2014045204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35dc100f90 0x7f35dc196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35db7fe700 1 -- 172.21.15.44:0/2014045204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35dc106690 msgr2=0x7f35dc19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35db7fe700 1 --2- 172.21.15.44:0/2014045204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35dc106690 0x7f35dc19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35db7fe700 1 -- 172.21.15.44:0/2014045204 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc009520 con 0x7f35dc1005d0 2023-09-10T01:20:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35dbfff700 1 --2- 172.21.15.44:0/2014045204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35dc106690 0x7f35dc19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:20:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35db7fe700 1 --2- 172.21.15.44:0/2014045204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35dc1005d0 0x7f35dc196750 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00eb30 tx=0x7f35cc00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35d8ff9700 1 -- 172.21.15.44:0/2014045204 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc009880 con 0x7f35dc1005d0 2023-09-10T01:20:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35e1eed700 1 -- 172.21.15.44:0/2014045204 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d4009d00 con 0x7f35dc1005d0 2023-09-10T01:20:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35d8ff9700 1 -- 172.21.15.44:0/2014045204 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35cc0044b0 con 0x7f35dc1005d0 2023-09-10T01:20:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35d8ff9700 1 -- 172.21.15.44:0/2014045204 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc003df0 con 0x7f35dc1005d0 2023-09-10T01:20:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.583+0000 7f35e1eed700 1 -- 172.21.15.44:0/2014045204 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc1a35e0 con 0x7f35dc1005d0 2023-09-10T01:20:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.586+0000 7f35d8ff9700 1 -- 172.21.15.44:0/2014045204 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f35cc003f50 con 0x7f35dc1005d0 2023-09-10T01:20:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.587+0000 7f35d8ff9700 1 --2- 172.21.15.44:0/2014045204 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f35c40a07b0 0x7f35c40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.587+0000 7f35d8ff9700 1 -- 172.21.15.44:0/2014045204 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f35cc014070 con 0x7f35dc1005d0 2023-09-10T01:20:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35ee0b3b80 1 Processor -- start 2023-09-10T01:20:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35ee0b3b80 1 -- start start 2023-09-10T01:20:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35ee0b3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c791a0 0x55aeb7c79580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35ee0b3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c73300 0x55aeb7c73780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35ee0b3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c7ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35ee0b3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7bcc6d0 con 0x55aeb7c791a0 2023-09-10T01:20:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35ee0b3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7bc9ff0 con 0x55aeb7c73cc0 2023-09-10T01:20:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35ee0b3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7bc58e0 con 0x55aeb7c73300 2023-09-10T01:20:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35db7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c791a0 0x55aeb7c79580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35db7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c791a0 0x55aeb7c79580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58504/0 (socket says 172.21.15.44:58504) 2023-09-10T01:20:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35db7fe700 1 -- 172.21.15.44:0/1680717989 learned_addr learned my addr 172.21.15.44:0/1680717989 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35dbfff700 1 --2- 172.21.15.44:0/1680717989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c7ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35daffd700 1 --2- 172.21.15.44:0/2014045204 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f35c40a07b0 0x7f35c40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35db7fe700 1 -- 172.21.15.44:0/1680717989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c73300 msgr2=0x55aeb7c73780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35daffd700 1 --2- 172.21.15.44:0/1680717989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c73300 0x55aeb7c73780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35db7fe700 1 --2- 172.21.15.44:0/1680717989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c73300 0x55aeb7c73780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35db7fe700 1 -- 172.21.15.44:0/1680717989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73cc0 msgr2=0x55aeb7c7ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.588+0000 7f35db7fe700 1 --2- 172.21.15.44:0/1680717989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c7ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.589+0000 7f35db7fe700 1 -- 172.21.15.44:0/1680717989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aeb79f1d80 con 0x55aeb7c791a0 2023-09-10T01:20:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.589+0000 7f35db7fe700 1 --2- 172.21.15.44:0/1680717989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c791a0 0x55aeb7c79580 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f35cc096210 tx=0x7f35cc099100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a90e613d3c7e0d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.589+0000 7f35c1ffb700 1 -- 172.21.15.44:0/1680717989 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc00cc90 con 0x55aeb7c791a0 2023-09-10T01:20:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.589+0000 7f35daffd700 1 --2- 172.21.15.44:0/2014045204 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f35c40a07b0 0x7f35c40a2c70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55aeb7c6e060 tx=0x7f35d0009af0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.589+0000 7f35c1ffb700 1 -- 172.21.15.44:0/1680717989 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35cc0997a0 con 0x55aeb7c791a0 2023-09-10T01:20:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.589+0000 7f35c1ffb700 1 -- 172.21.15.44:0/1680717989 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc095eb0 con 0x55aeb7c791a0 2023-09-10T01:20:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.590+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/1680717989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c791a0 msgr2=0x55aeb7c79580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.590+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/1680717989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c791a0 0x55aeb7c79580 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f35cc096210 tx=0x7f35cc099100 comp rx=0 tx=0).stop 2023-09-10T01:20:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.590+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/1680717989 shutdown_connections 2023-09-10T01:20:22.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.590+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/1680717989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c7ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.590+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/1680717989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c791a0 0x55aeb7c79580 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.590+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/1680717989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c73300 0x55aeb7c73780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.590+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/1680717989 >> 172.21.15.44:0/1680717989 conn(0x55aeb7be8030 msgr2=0x55aeb7c6e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:22.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.590+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/1680717989 shutdown_connections 2023-09-10T01:20:22.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.590+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/1680717989 wait complete. 2023-09-10T01:20:22.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.590+0000 7f35ee0b3b80 1 -- start start 2023-09-10T01:20:22.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35ee0b3b80 1 Processor -- start 2023-09-10T01:20:22.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35ee0b3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73300 0x55aeb7c7fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35ee0b3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c7d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35ee0b3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c791a0 0x55aeb7c7d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35ee0b3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7b63b10 con 0x55aeb7c73cc0 2023-09-10T01:20:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35ee0b3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7b63810 con 0x55aeb7c73300 2023-09-10T01:20:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35ee0b3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7a478c0 con 0x55aeb7c791a0 2023-09-10T01:20:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35ee0b3b80 10 client.0 _ll_register_callbacks cb 0x55aeb7aada20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:20:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35daffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c7d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35daffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c7d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58512/0 (socket says 172.21.15.44:58512) 2023-09-10T01:20:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35daffd700 1 -- 172.21.15.44:0/2359361458 learned_addr learned my addr 172.21.15.44:0/2359361458 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35dbfff700 1 --2- 172.21.15.44:0/2359361458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c791a0 0x55aeb7c7d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35db7fe700 1 --2- 172.21.15.44:0/2359361458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73300 0x55aeb7c7fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35daffd700 1 -- 172.21.15.44:0/2359361458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c791a0 msgr2=0x55aeb7c7d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35daffd700 1 --2- 172.21.15.44:0/2359361458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c791a0 0x55aeb7c7d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35daffd700 1 -- 172.21.15.44:0/2359361458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73300 msgr2=0x55aeb7c7fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35daffd700 1 --2- 172.21.15.44:0/2359361458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73300 0x55aeb7c7fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35daffd700 1 -- 172.21.15.44:0/2359361458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc1a35e0 con 0x55aeb7c73cc0 2023-09-10T01:20:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.591+0000 7f35db7fe700 1 --2- 172.21.15.44:0/2359361458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73300 0x55aeb7c7fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:20:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.592+0000 7f35daffd700 1 --2- 172.21.15.44:0/2359361458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c7d0e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f35d0011d80 tx=0x7f35d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99eea9fc1ffbc1ee server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.592+0000 7f35baffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:20:22.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.592+0000 7f35baffd700 1 -- 172.21.15.44:0/2359361458 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0012bc0 con 0x55aeb7c73cc0 2023-09-10T01:20:22.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.592+0000 7f35baffd700 1 -- 172.21.15.44:0/2359361458 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f35d0012d20 con 0x55aeb7c73cc0 2023-09-10T01:20:22.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.592+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/2359361458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aeb7b1d490 con 0x55aeb7c73cc0 2023-09-10T01:20:22.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.592+0000 7f35baffd700 1 -- 172.21.15.44:0/2359361458 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35d0020460 con 0x55aeb7c73cc0 2023-09-10T01:20:22.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.592+0000 7f35ee0b3b80 10 client.19866 fetch_fsmap learned FSMap version 131 2023-09-10T01:20:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.592+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/2359361458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=4+}) v3 -- 0x55aeb79dac00 con 0x55aeb7c73cc0 2023-09-10T01:20:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.592+0000 7f35baffd700 1 -- 172.21.15.44:0/2359361458 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aeb7b1d490 con 0x55aeb7c73cc0 2023-09-10T01:20:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.593+0000 7f35baffd700 1 -- 172.21.15.44:0/2359361458 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 131) v1 ==== 7319+0+0 (secure 0 0 0) 0x7f35d0029410 con 0x55aeb7c73cc0 2023-09-10T01:20:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.593+0000 7f35ee0b3b80 10 client.19866 fetch_fsmap finished waiting for FSMap version 131 2023-09-10T01:20:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.593+0000 7f35ee0b3b80 10 client.19866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:20:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.593+0000 7f35ee0b3b80 20 client.19866 populate_metadata read hostname 'smithi044' 2023-09-10T01:20:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.593+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/2359361458 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aeb7c8d510 0x55aeb7c8f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.593+0000 7f35ee0b3b80 4 client.19866 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:20:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.593+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/2359361458 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aeb7bda840 con 0x55aeb7c8d510 2023-09-10T01:20:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.593+0000 7f35db7fe700 1 --2- 172.21.15.44:0/2359361458 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aeb7c8d510 0x55aeb7c8f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.593+0000 7f35baffd700 10 client.19866 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:20:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.593+0000 7f35db7fe700 1 --2- 172.21.15.44:0/2359361458 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aeb7c8d510 0x55aeb7c8f930 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8818e4471404a8ab server_cookie=68ef0abe279a08cd in_seq=0 out_seq=0 2023-09-10T01:20:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.594+0000 7f35baffd700 1 -- 172.21.15.44:0/2359361458 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aeb7bda840 con 0x55aeb7c8d510 2023-09-10T01:20:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.594+0000 7f35baffd700 10 client.19866 handle_command_reply: tid=0 2023-09-10T01:20:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.594+0000 7f35ee0b3b80 1 client.19866 shutdown 2023-09-10T01:20:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.594+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/2359361458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c73cc0 msgr2=0x55aeb7c7d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.594+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/2359361458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c7d0e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f35d0011d80 tx=0x7f35d0004750 comp rx=0 tx=0).stop 2023-09-10T01:20:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.594+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/2359361458 shutdown_connections 2023-09-10T01:20:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.594+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/2359361458 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aeb7c8d510 0x55aeb7c8f930 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.594+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/2359361458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c791a0 0x55aeb7c7d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.594+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/2359361458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73300 0x55aeb7c7fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.594+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/2359361458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c7d0e0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.594+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/2359361458 >> 172.21.15.44:0/2359361458 conn(0x55aeb7be8030 msgr2=0x55aeb7c6d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.594+0000 7f35baffd700 0 client.19866 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:20:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.594+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/2359361458 shutdown_connections 2023-09-10T01:20:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.594+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/2359361458 wait complete. 2023-09-10T01:20:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.594+0000 7f35ee0b3b80 20 client.19866 trim_cache size 0 max 16384 2023-09-10T01:20:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.613+0000 7f35ee0b3b80 1 Processor -- start 2023-09-10T01:20:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.613+0000 7f35ee0b3b80 1 -- start start 2023-09-10T01:20:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.613+0000 7f35ee0b3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c73300 0x55aeb7c7fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.613+0000 7f35ee0b3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c6e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.613+0000 7f35ee0b3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c791a0 0x55aeb7c6e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.613+0000 7f35ee0b3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7bda840 con 0x55aeb7c791a0 2023-09-10T01:20:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.613+0000 7f35ee0b3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7b1d490 con 0x55aeb7c73cc0 2023-09-10T01:20:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.613+0000 7f35ee0b3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7c8ab40 con 0x55aeb7c73300 2023-09-10T01:20:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.613+0000 7f35dbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c791a0 0x55aeb7c6e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.613+0000 7f35dbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c791a0 0x55aeb7c6e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58522/0 (socket says 172.21.15.44:58522) 2023-09-10T01:20:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.613+0000 7f35dbfff700 1 -- 172.21.15.44:0/671203583 learned_addr learned my addr 172.21.15.44:0/671203583 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.613+0000 7f35db7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c73300 0x55aeb7c7fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.614+0000 7f35daffd700 1 --2- 172.21.15.44:0/671203583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c6e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.614+0000 7f35dbfff700 1 -- 172.21.15.44:0/671203583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c73300 msgr2=0x55aeb7c7fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.614+0000 7f35dbfff700 1 --2- 172.21.15.44:0/671203583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c73300 0x55aeb7c7fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.614+0000 7f35dbfff700 1 -- 172.21.15.44:0/671203583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73cc0 msgr2=0x55aeb7c6e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.614+0000 7f35dbfff700 1 --2- 172.21.15.44:0/671203583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c6e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.614+0000 7f35dbfff700 1 -- 172.21.15.44:0/671203583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aeb7a181f0 con 0x55aeb7c791a0 2023-09-10T01:20:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.614+0000 7f35daffd700 1 --2- 172.21.15.44:0/671203583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c6e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:20:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.614+0000 7f35dbfff700 1 --2- 172.21.15.44:0/671203583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c791a0 0x55aeb7c6e5d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009390 tx=0x7f35d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a253f26d61548e34 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.614+0000 7f35c27fc700 1 -- 172.21.15.44:0/671203583 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4021070 con 0x55aeb7c791a0 2023-09-10T01:20:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.614+0000 7f35c27fc700 1 -- 172.21.15.44:0/671203583 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35d4009470 con 0x55aeb7c791a0 2023-09-10T01:20:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.614+0000 7f35c27fc700 1 -- 172.21.15.44:0/671203583 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d400e040 con 0x55aeb7c791a0 2023-09-10T01:20:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.615+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/671203583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c791a0 msgr2=0x55aeb7c6e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.615+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/671203583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c791a0 0x55aeb7c6e5d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009390 tx=0x7f35d400b040 comp rx=0 tx=0).stop 2023-09-10T01:20:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.615+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/671203583 shutdown_connections 2023-09-10T01:20:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.615+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/671203583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c791a0 0x55aeb7c6e5d0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.615+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/671203583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c73300 0x55aeb7c7fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.615+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/671203583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c6e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.615+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/671203583 >> 172.21.15.44:0/671203583 conn(0x55aeb7be8030 msgr2=0x55aeb7c780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.615+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/671203583 shutdown_connections 2023-09-10T01:20:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.615+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/671203583 wait complete. 2023-09-10T01:20:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.615+0000 7f35ee0b3b80 1 -- start start 2023-09-10T01:20:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.616+0000 7f35ee0b3b80 1 Processor -- start 2023-09-10T01:20:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.616+0000 7f35ee0b3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c73300 0x55aeb7c82510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.616+0000 7f35ee0b3b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c82a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.616+0000 7f35ee0b3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c791a0 0x55aeb7ce0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.616+0000 7f35ee0b3b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7c83340 con 0x55aeb7c73cc0 2023-09-10T01:20:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.616+0000 7f35ee0b3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7ce08a0 con 0x55aeb7c791a0 2023-09-10T01:20:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.616+0000 7f35ee0b3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7ce0a10 con 0x55aeb7c73300 2023-09-10T01:20:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.616+0000 7f35ee0b3b80 10 client.0 _ll_register_callbacks cb 0x7f35d0011b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:20:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.616+0000 7f35daffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c82a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.616+0000 7f35db7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c73300 0x55aeb7c82510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.616+0000 7f35daffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c82a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58526/0 (socket says 172.21.15.44:58526) 2023-09-10T01:20:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.616+0000 7f35daffd700 1 -- 172.21.15.44:0/1746224435 learned_addr learned my addr 172.21.15.44:0/1746224435 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.616+0000 7f35dbfff700 1 --2- 172.21.15.44:0/1746224435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c791a0 0x55aeb7ce0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35db7fe700 1 -- 172.21.15.44:0/1746224435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c791a0 msgr2=0x55aeb7ce0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35db7fe700 1 --2- 172.21.15.44:0/1746224435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c791a0 0x55aeb7ce0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35db7fe700 1 -- 172.21.15.44:0/1746224435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c73cc0 msgr2=0x55aeb7c82a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35db7fe700 1 --2- 172.21.15.44:0/1746224435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c82a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35db7fe700 1 -- 172.21.15.44:0/1746224435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc10dd50 con 0x55aeb7c73300 2023-09-10T01:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35dbfff700 1 --2- 172.21.15.44:0/1746224435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c791a0 0x55aeb7ce0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35daffd700 1 --2- 172.21.15.44:0/1746224435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c82a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35db7fe700 1 --2- 172.21.15.44:0/1746224435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c73300 0x55aeb7c82510 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f35cc09a050 tx=0x7f35cc004620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69837e3cc99daad server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35baffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35baffd700 1 -- 172.21.15.44:0/1746224435 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0b7040 con 0x55aeb7c73300 2023-09-10T01:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/1746224435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35cc0997a0 con 0x55aeb7c73300 2023-09-10T01:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35baffd700 1 -- 172.21.15.44:0/1746224435 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f35cc0b6070 con 0x55aeb7c73300 2023-09-10T01:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35baffd700 1 -- 172.21.15.44:0/1746224435 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35cc004200 con 0x55aeb7c73300 2023-09-10T01:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35ee0b3b80 10 client.29470 fetch_fsmap learned FSMap version 131 2023-09-10T01:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/1746224435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=4+}) v3 -- 0x55aeb79f0250 con 0x55aeb7c73300 2023-09-10T01:20:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35baffd700 1 -- 172.21.15.44:0/1746224435 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35cc07f080 con 0x55aeb7c73300 2023-09-10T01:20:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.617+0000 7f35baffd700 1 -- 172.21.15.44:0/1746224435 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 131) v1 ==== 7319+0+0 (secure 0 0 0) 0x7f35cc098c90 con 0x55aeb7c73300 2023-09-10T01:20:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.618+0000 7f35ee0b3b80 10 client.29470 fetch_fsmap finished waiting for FSMap version 131 2023-09-10T01:20:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.618+0000 7f35ee0b3b80 10 client.29470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:20:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.618+0000 7f35ee0b3b80 20 client.29470 populate_metadata read hostname 'smithi044' 2023-09-10T01:20:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.618+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/1746224435 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aeb7ce1e10 0x55aeb7ce4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.618+0000 7f35ee0b3b80 4 client.29470 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:20:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.618+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/1746224435 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aeb7bbe8c0 con 0x55aeb7ce1e10 2023-09-10T01:20:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.618+0000 7f35daffd700 1 --2- 172.21.15.44:0/1746224435 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aeb7ce1e10 0x55aeb7ce4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.618+0000 7f35baffd700 10 client.29470 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:20:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.618+0000 7f35daffd700 1 --2- 172.21.15.44:0/1746224435 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aeb7ce1e10 0x55aeb7ce4230 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ced4376461d9044 server_cookie=3cbeceb032ecd221 in_seq=0 out_seq=0 2023-09-10T01:20:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.633+0000 7f35baffd700 1 -- 172.21.15.44:0/1746224435 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aeb7bbe8c0 con 0x55aeb7ce1e10 2023-09-10T01:20:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.633+0000 7f35baffd700 10 client.29470 handle_command_reply: tid=0 2023-09-10T01:20:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.633+0000 7f35ee0b3b80 1 client.29470 shutdown 2023-09-10T01:20:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/1746224435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c73300 msgr2=0x55aeb7c82510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:22.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/1746224435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c73300 0x55aeb7c82510 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f35cc09a050 tx=0x7f35cc004620 comp rx=0 tx=0).stop 2023-09-10T01:20:22.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/1746224435 shutdown_connections 2023-09-10T01:20:22.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/1746224435 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aeb7ce1e10 0x55aeb7ce4230 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/1746224435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aeb7c791a0 0x55aeb7ce0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/1746224435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aeb7c73300 0x55aeb7c82510 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35ee0b3b80 1 --2- 172.21.15.44:0/1746224435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aeb7c73cc0 0x55aeb7c82a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35baffd700 0 client.29470 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:20:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/1746224435 >> 172.21.15.44:0/1746224435 conn(0x55aeb7be8030 msgr2=0x55aeb7c780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/1746224435 shutdown_connections 2023-09-10T01:20:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35ee0b3b80 1 -- 172.21.15.44:0/1746224435 wait complete. 2023-09-10T01:20:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35ee0b3b80 20 client.29470 trim_cache size 0 max 16384 2023-09-10T01:20:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:20:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:20:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:20:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:20:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35e1eed700 1 -- 172.21.15.44:0/2014045204 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f35c40a07b0 msgr2=0x7f35c40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35e1eed700 1 --2- 172.21.15.44:0/2014045204 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f35c40a07b0 0x7f35c40a2c70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55aeb7c6e060 tx=0x7f35d0009af0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35e1eed700 1 -- 172.21.15.44:0/2014045204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35dc1005d0 msgr2=0x7f35dc196750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35e1eed700 1 --2- 172.21.15.44:0/2014045204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35dc1005d0 0x7f35dc196750 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00eb30 tx=0x7f35cc00ee40 comp rx=0 tx=0).stop 2023-09-10T01:20:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35e1eed700 1 -- 172.21.15.44:0/2014045204 shutdown_connections 2023-09-10T01:20:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35e1eed700 1 --2- 172.21.15.44:0/2014045204 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f35c40a07b0 0x7f35c40a2c70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35e1eed700 1 --2- 172.21.15.44:0/2014045204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35dc106690 0x7f35dc19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35e1eed700 1 --2- 172.21.15.44:0/2014045204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35dc1005d0 0x7f35dc196750 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35e1eed700 1 --2- 172.21.15.44:0/2014045204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35dc100f90 0x7f35dc196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.634+0000 7f35e1eed700 1 -- 172.21.15.44:0/2014045204 >> 172.21.15.44:0/2014045204 conn(0x7f35dc0fc0c0 msgr2=0x7f35dc0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.635+0000 7f35e1eed700 1 -- 172.21.15.44:0/2014045204 shutdown_connections 2023-09-10T01:20:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:22.635+0000 7f35e1eed700 1 -- 172.21.15.44:0/2014045204 wait complete. 2023-09-10T01:20:23.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:23 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:20:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:23 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:20:23.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:23 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:20:24.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:24 smithi044 ceph-mon[98676]: pgmap v1283: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 19 MiB/s wr, 121 op/s 2023-09-10T01:20:24.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:24 smithi099 ceph-mon[102692]: pgmap v1283: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 19 MiB/s wr, 121 op/s 2023-09-10T01:20:24.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:24 smithi131 ceph-mon[103713]: pgmap v1283: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 19 MiB/s wr, 121 op/s 2023-09-10T01:20:26.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:26 smithi044 ceph-mon[98676]: pgmap v1284: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 19 MiB/s wr, 121 op/s 2023-09-10T01:20:26.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:26 smithi099 ceph-mon[102692]: pgmap v1284: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 19 MiB/s wr, 121 op/s 2023-09-10T01:20:26.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:26 smithi131 ceph-mon[103713]: pgmap v1284: 129 pgs: 129 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 19 MiB/s wr, 121 op/s 2023-09-10T01:20:28.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:28 smithi099 ceph-mon[102692]: pgmap v1285: 129 pgs: 129 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 17 MiB/s wr, 112 op/s 2023-09-10T01:20:28.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:28 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:20:28.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:28 smithi131 ceph-mon[103713]: pgmap v1285: 129 pgs: 129 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 17 MiB/s wr, 112 op/s 2023-09-10T01:20:28.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:28 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:20:28.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:28 smithi044 ceph-mon[98676]: pgmap v1285: 129 pgs: 129 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 17 MiB/s wr, 112 op/s 2023-09-10T01:20:28.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:28 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:20:30.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:30 smithi099 ceph-mon[102692]: pgmap v1286: 129 pgs: 129 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 99 op/s 2023-09-10T01:20:30.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:30 smithi131 ceph-mon[103713]: pgmap v1286: 129 pgs: 129 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 99 op/s 2023-09-10T01:20:30.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:30 smithi044 ceph-mon[98676]: pgmap v1286: 129 pgs: 129 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 99 op/s 2023-09-10T01:20:32.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:32 smithi099 ceph-mon[102692]: pgmap v1287: 129 pgs: 129 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 23 MiB/s wr, 148 op/s 2023-09-10T01:20:32.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:32 smithi131 ceph-mon[103713]: pgmap v1287: 129 pgs: 129 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 23 MiB/s wr, 148 op/s 2023-09-10T01:20:32.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:32 smithi044 ceph-mon[98676]: pgmap v1287: 129 pgs: 129 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 23 MiB/s wr, 148 op/s 2023-09-10T01:20:34.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:34 smithi099 ceph-mon[102692]: pgmap v1288: 129 pgs: 129 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 97 op/s 2023-09-10T01:20:34.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:34 smithi131 ceph-mon[103713]: pgmap v1288: 129 pgs: 129 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 97 op/s 2023-09-10T01:20:34.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:34 smithi044 ceph-mon[98676]: pgmap v1288: 129 pgs: 129 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 97 op/s 2023-09-10T01:20:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.770+0000 7f4a6454b700 1 -- 172.21.15.44:0/1881499329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c0ff760 msgr2=0x7f4a5c0ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.770+0000 7f4a6454b700 1 --2- 172.21.15.44:0/1881499329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c0ff760 0x7f4a5c0ffbe0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0099c0 tx=0x7f4a4c009cd0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.770+0000 7f4a6454b700 1 -- 172.21.15.44:0/1881499329 shutdown_connections 2023-09-10T01:20:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.770+0000 7f4a6454b700 1 --2- 172.21.15.44:0/1881499329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a5c100120 0x7f4a5c10c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.770+0000 7f4a6454b700 1 --2- 172.21.15.44:0/1881499329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a5c1055b0 0x7f4a5c105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.770+0000 7f4a6454b700 1 --2- 172.21.15.44:0/1881499329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c0ff760 0x7f4a5c0ffbe0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.770+0000 7f4a6454b700 1 -- 172.21.15.44:0/1881499329 >> 172.21.15.44:0/1881499329 conn(0x7f4a5c074f30 msgr2=0x7f4a5c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.770+0000 7f4a6454b700 1 -- 172.21.15.44:0/1881499329 shutdown_connections 2023-09-10T01:20:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.770+0000 7f4a6454b700 1 -- 172.21.15.44:0/1881499329 wait complete. 2023-09-10T01:20:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.771+0000 7f4a6454b700 1 Processor -- start 2023-09-10T01:20:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.771+0000 7f4a6454b700 1 -- start start 2023-09-10T01:20:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.771+0000 7f4a6454b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a5c0ff760 0x7f4a5c19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.771+0000 7f4a6454b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c100120 0x7f4a5c19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.771+0000 7f4a6454b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a5c1055b0 0x7f4a5c196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.771+0000 7f4a6454b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c196a80 con 0x7f4a5c100120 2023-09-10T01:20:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.771+0000 7f4a6454b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c196bf0 con 0x7f4a5c0ff760 2023-09-10T01:20:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.771+0000 7f4a6454b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c196db0 con 0x7f4a5c1055b0 2023-09-10T01:20:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a61ae6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c100120 0x7f4a5c19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a622e7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a5c0ff760 0x7f4a5c19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a61ae6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c100120 0x7f4a5c19caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58540/0 (socket says 172.21.15.44:58540) 2023-09-10T01:20:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a61ae6700 1 -- 172.21.15.44:0/927643923 learned_addr learned my addr 172.21.15.44:0/927643923 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a62ae8700 1 --2- 172.21.15.44:0/927643923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a5c1055b0 0x7f4a5c196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a61ae6700 1 -- 172.21.15.44:0/927643923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a5c1055b0 msgr2=0x7f4a5c196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a61ae6700 1 --2- 172.21.15.44:0/927643923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a5c1055b0 0x7f4a5c196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a61ae6700 1 -- 172.21.15.44:0/927643923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a5c0ff760 msgr2=0x7f4a5c19c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a61ae6700 1 --2- 172.21.15.44:0/927643923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a5c0ff760 0x7f4a5c19c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a61ae6700 1 -- 172.21.15.44:0/927643923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c0095f0 con 0x7f4a5c100120 2023-09-10T01:20:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a622e7700 1 --2- 172.21.15.44:0/927643923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a5c0ff760 0x7f4a5c19c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:20:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a62ae8700 1 --2- 172.21.15.44:0/927643923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a5c1055b0 0x7f4a5c196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:20:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a61ae6700 1 --2- 172.21.15.44:0/927643923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c100120 0x7f4a5c19caf0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009fd0 tx=0x7f4a4c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a537fe700 1 -- 172.21.15.44:0/927643923 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c01d070 con 0x7f4a5c100120 2023-09-10T01:20:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a537fe700 1 -- 172.21.15.44:0/927643923 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a4c005030 con 0x7f4a5c100120 2023-09-10T01:20:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a6454b700 1 -- 172.21.15.44:0/927643923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5c197030 con 0x7f4a5c100120 2023-09-10T01:20:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a537fe700 1 -- 172.21.15.44:0/927643923 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c004990 con 0x7f4a5c100120 2023-09-10T01:20:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.772+0000 7f4a6454b700 1 -- 172.21.15.44:0/927643923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5c1a35a0 con 0x7f4a5c100120 2023-09-10T01:20:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.776+0000 7f4a537fe700 1 -- 172.21.15.44:0/927643923 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f4a4c0051a0 con 0x7f4a5c100120 2023-09-10T01:20:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.776+0000 7f4a537fe700 1 --2- 172.21.15.44:0/927643923 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4a480a07b0 0x7f4a480a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.777+0000 7f4a537fe700 1 -- 172.21.15.44:0/927643923 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f4a4c0cd100 con 0x7f4a5c100120 2023-09-10T01:20:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.777+0000 7f4a622e7700 1 --2- 172.21.15.44:0/927643923 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4a480a07b0 0x7f4a480a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.777+0000 7f4a70711b80 1 Processor -- start 2023-09-10T01:20:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.777+0000 7f4a70711b80 1 -- start start 2023-09-10T01:20:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.777+0000 7f4a70711b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585fc5b0 0x562b585fc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.777+0000 7f4a70711b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585f6710 0x562b585f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.777+0000 7f4a70711b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b585f70d0 0x562b58603340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.777+0000 7f4a70711b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5854b6d0 con 0x562b585f6710 2023-09-10T01:20:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.777+0000 7f4a70711b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b58548ff0 con 0x562b585f70d0 2023-09-10T01:20:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.777+0000 7f4a70711b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b585448e0 con 0x562b585fc5b0 2023-09-10T01:20:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.778+0000 7f4a622e7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585fc5b0 0x562b585fc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.778+0000 7f4a62ae8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b585f70d0 0x562b58603340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.778+0000 7f4a62ae8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b585f70d0 0x562b58603340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60288/0 (socket says 172.21.15.44:60288) 2023-09-10T01:20:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.778+0000 7f4a622e7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585fc5b0 0x562b585fc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37196/0 (socket says 172.21.15.44:37196) 2023-09-10T01:20:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.778+0000 7f4a62ae8700 1 -- 172.21.15.44:0/2276341709 learned_addr learned my addr 172.21.15.44:0/2276341709 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.778+0000 7f4a622e7700 1 --2- 172.21.15.44:0/927643923 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4a480a07b0 0x7f4a480a2c70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009fd0 tx=0x7f4a58009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.778+0000 7f4a62ae8700 1 -- 172.21.15.44:0/2276341709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585fc5b0 msgr2=0x562b585fc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.778+0000 7f4a62ae8700 1 --2- 172.21.15.44:0/2276341709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585fc5b0 0x562b585fc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.778+0000 7f4a62ae8700 1 -- 172.21.15.44:0/2276341709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585f6710 msgr2=0x562b585f6b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:20:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.778+0000 7f4a62ae8700 1 --2- 172.21.15.44:0/2276341709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585f6710 0x562b585f6b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.778+0000 7f4a62ae8700 1 -- 172.21.15.44:0/2276341709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b58370d80 con 0x562b585f70d0 2023-09-10T01:20:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.778+0000 7f4a622e7700 1 --2- 172.21.15.44:0/2276341709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585fc5b0 0x562b585fc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:20:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.778+0000 7f4a62ae8700 1 --2- 172.21.15.44:0/2276341709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b585f70d0 0x562b58603340 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009790 tx=0x7f4a54007b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ba64d28f78f9d0c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:35.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.778+0000 7f4a50ff9700 1 -- 172.21.15.44:0/2276341709 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54003c00 con 0x562b585f70d0 2023-09-10T01:20:35.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.778+0000 7f4a50ff9700 1 -- 172.21.15.44:0/2276341709 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a5400f040 con 0x562b585f70d0 2023-09-10T01:20:35.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.778+0000 7f4a50ff9700 1 -- 172.21.15.44:0/2276341709 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54004320 con 0x562b585f70d0 2023-09-10T01:20:35.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.779+0000 7f4a70711b80 1 -- 172.21.15.44:0/2276341709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b585f70d0 msgr2=0x562b58603340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:35.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.779+0000 7f4a70711b80 1 --2- 172.21.15.44:0/2276341709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b585f70d0 0x562b58603340 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009790 tx=0x7f4a54007b80 comp rx=0 tx=0).stop 2023-09-10T01:20:35.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.779+0000 7f4a70711b80 1 -- 172.21.15.44:0/2276341709 shutdown_connections 2023-09-10T01:20:35.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.779+0000 7f4a70711b80 1 --2- 172.21.15.44:0/2276341709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b585f70d0 0x562b58603340 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.779+0000 7f4a70711b80 1 --2- 172.21.15.44:0/2276341709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585fc5b0 0x562b585fc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.779+0000 7f4a70711b80 1 --2- 172.21.15.44:0/2276341709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585f6710 0x562b585f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.779+0000 7f4a70711b80 1 -- 172.21.15.44:0/2276341709 >> 172.21.15.44:0/2276341709 conn(0x562b58567030 msgr2=0x562b585ed560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.780+0000 7f4a70711b80 1 -- 172.21.15.44:0/2276341709 shutdown_connections 2023-09-10T01:20:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.780+0000 7f4a70711b80 1 -- 172.21.15.44:0/2276341709 wait complete. 2023-09-10T01:20:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.780+0000 7f4a70711b80 1 -- start start 2023-09-10T01:20:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.780+0000 7f4a70711b80 1 Processor -- start 2023-09-10T01:20:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a70711b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 0x562b58603330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a70711b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585fc5b0 0x562b58600620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a70711b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 0x562b5860c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a70711b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b584e2b10 con 0x562b585fc5b0 2023-09-10T01:20:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a70711b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b584e2810 con 0x562b58600cc0 2023-09-10T01:20:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a70711b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b583c68c0 con 0x562b585f70d0 2023-09-10T01:20:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a70711b80 10 client.0 _ll_register_callbacks cb 0x562b5842ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:20:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a622e7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 0x562b58603330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a62ae8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 0x562b5860c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a622e7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 0x562b58603330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37200/0 (socket says 172.21.15.44:37200) 2023-09-10T01:20:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a62ae8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 0x562b5860c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60292/0 (socket says 172.21.15.44:60292) 2023-09-10T01:20:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a622e7700 1 -- 172.21.15.44:0/4253425009 learned_addr learned my addr 172.21.15.44:0/4253425009 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a62ae8700 1 -- 172.21.15.44:0/4253425009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 msgr2=0x562b58603330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a62ae8700 1 --2- 172.21.15.44:0/4253425009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 0x562b58603330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a62ae8700 1 -- 172.21.15.44:0/4253425009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585fc5b0 msgr2=0x562b58600620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:20:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a62ae8700 1 --2- 172.21.15.44:0/4253425009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585fc5b0 0x562b58600620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.781+0000 7f4a62ae8700 1 -- 172.21.15.44:0/4253425009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a58017070 con 0x562b58600cc0 2023-09-10T01:20:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a622e7700 1 --2- 172.21.15.44:0/4253425009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 0x562b58603330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:20:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a62ae8700 1 --2- 172.21.15.44:0/4253425009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 0x562b5860c9b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009550 tx=0x7f4a54003cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b35558e0834bf62 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:20:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a45ffb700 1 -- 172.21.15.44:0/4253425009 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54015a60 con 0x562b58600cc0 2023-09-10T01:20:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a45ffb700 1 -- 172.21.15.44:0/4253425009 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f4a54004000 con 0x562b58600cc0 2023-09-10T01:20:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a70711b80 1 -- 172.21.15.44:0/4253425009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b5849c490 con 0x562b58600cc0 2023-09-10T01:20:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a45ffb700 1 -- 172.21.15.44:0/4253425009 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a54015d40 con 0x562b58600cc0 2023-09-10T01:20:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a70711b80 10 client.29501 fetch_fsmap learned FSMap version 132 2023-09-10T01:20:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a70711b80 1 -- 172.21.15.44:0/4253425009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=4+}) v3 -- 0x7f4a54007500 con 0x562b58600cc0 2023-09-10T01:20:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a45ffb700 1 -- 172.21.15.44:0/4253425009 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562b5849c490 con 0x562b58600cc0 2023-09-10T01:20:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a45ffb700 1 -- 172.21.15.44:0/4253425009 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 132) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f4a5401c070 con 0x562b58600cc0 2023-09-10T01:20:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a70711b80 10 client.29501 fetch_fsmap finished waiting for FSMap version 132 2023-09-10T01:20:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a70711b80 10 client.29501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:20:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a70711b80 20 client.29501 populate_metadata read hostname 'smithi044' 2023-09-10T01:20:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a70711b80 1 --2- 172.21.15.44:0/4253425009 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562b58610c70 0x562b58613090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a70711b80 4 client.29501 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:20:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.782+0000 7f4a70711b80 1 -- 172.21.15.44:0/4253425009 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b58559840 con 0x562b58610c70 2023-09-10T01:20:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.783+0000 7f4a622e7700 1 --2- 172.21.15.44:0/4253425009 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562b58610c70 0x562b58613090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.783+0000 7f4a45ffb700 10 client.29501 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:20:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.783+0000 7f4a622e7700 1 --2- 172.21.15.44:0/4253425009 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562b58610c70 0x562b58613090 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abb33f7a5b4ec639 server_cookie=6f653c4a811f478b in_seq=0 out_seq=0 2023-09-10T01:20:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.784+0000 7f4a45ffb700 1 -- 172.21.15.44:0/4253425009 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562b58559840 con 0x562b58610c70 2023-09-10T01:20:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.784+0000 7f4a45ffb700 10 client.29501 handle_command_reply: tid=0 2023-09-10T01:20:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.784+0000 7f4a70711b80 1 client.29501 shutdown 2023-09-10T01:20:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.784+0000 7f4a70711b80 1 -- 172.21.15.44:0/4253425009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 msgr2=0x562b5860c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.784+0000 7f4a70711b80 1 --2- 172.21.15.44:0/4253425009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 0x562b5860c9b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009550 tx=0x7f4a54003cb0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.784+0000 7f4a70711b80 1 -- 172.21.15.44:0/4253425009 shutdown_connections 2023-09-10T01:20:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.784+0000 7f4a70711b80 1 --2- 172.21.15.44:0/4253425009 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562b58610c70 0x562b58613090 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.784+0000 7f4a70711b80 1 --2- 172.21.15.44:0/4253425009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 0x562b5860c9b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.784+0000 7f4a70711b80 1 --2- 172.21.15.44:0/4253425009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 0x562b58603330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.784+0000 7f4a45ffb700 0 client.29501 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:20:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.784+0000 7f4a70711b80 1 --2- 172.21.15.44:0/4253425009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585fc5b0 0x562b58600620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.784+0000 7f4a70711b80 1 -- 172.21.15.44:0/4253425009 >> 172.21.15.44:0/4253425009 conn(0x562b58567030 msgr2=0x562b585ec7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.785+0000 7f4a70711b80 1 -- 172.21.15.44:0/4253425009 shutdown_connections 2023-09-10T01:20:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.785+0000 7f4a70711b80 1 -- 172.21.15.44:0/4253425009 wait complete. 2023-09-10T01:20:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.785+0000 7f4a70711b80 20 client.29501 trim_cache size 0 max 16384 2023-09-10T01:20:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.805+0000 7f4a70711b80 1 Processor -- start 2023-09-10T01:20:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a70711b80 1 -- start start 2023-09-10T01:20:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a70711b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 0x562b585ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a70711b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 0x562b585fdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a70711b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585ed7a0 0x562b585fb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a70711b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b585fb270 con 0x562b585ed7a0 2023-09-10T01:20:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a70711b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b585eed10 con 0x562b58600cc0 2023-09-10T01:20:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a70711b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b585eea60 con 0x562b585f70d0 2023-09-10T01:20:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a62ae8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585ed7a0 0x562b585fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a62ae8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585ed7a0 0x562b585fb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58560/0 (socket says 172.21.15.44:58560) 2023-09-10T01:20:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a62ae8700 1 -- 172.21.15.44:0/1273999924 learned_addr learned my addr 172.21.15.44:0/1273999924 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a622e7700 1 --2- 172.21.15.44:0/1273999924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 0x562b585ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a61ae6700 1 --2- 172.21.15.44:0/1273999924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 0x562b585fdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a622e7700 1 -- 172.21.15.44:0/1273999924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 msgr2=0x562b585fdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a622e7700 1 --2- 172.21.15.44:0/1273999924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 0x562b585fdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a622e7700 1 -- 172.21.15.44:0/1273999924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585ed7a0 msgr2=0x562b585fb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a622e7700 1 --2- 172.21.15.44:0/1273999924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585ed7a0 0x562b585fb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.806+0000 7f4a622e7700 1 -- 172.21.15.44:0/1273999924 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b58359c00 con 0x562b585f70d0 2023-09-10T01:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.807+0000 7f4a622e7700 1 --2- 172.21.15.44:0/1273999924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 0x562b585ffde0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x562b58613670 tx=0x7f4a58014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0aabb920f594452 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.807+0000 7f4a517fa700 1 -- 172.21.15.44:0/1273999924 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5802d070 con 0x562b585f70d0 2023-09-10T01:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.807+0000 7f4a517fa700 1 -- 172.21.15.44:0/1273999924 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a580293a0 con 0x562b585f70d0 2023-09-10T01:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.807+0000 7f4a517fa700 1 -- 172.21.15.44:0/1273999924 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a58024040 con 0x562b585f70d0 2023-09-10T01:20:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.808+0000 7f4a70711b80 1 -- 172.21.15.44:0/1273999924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 msgr2=0x562b585ffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.808+0000 7f4a70711b80 1 --2- 172.21.15.44:0/1273999924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 0x562b585ffde0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x562b58613670 tx=0x7f4a58014000 comp rx=0 tx=0).stop 2023-09-10T01:20:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.808+0000 7f4a70711b80 1 -- 172.21.15.44:0/1273999924 shutdown_connections 2023-09-10T01:20:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.808+0000 7f4a70711b80 1 --2- 172.21.15.44:0/1273999924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585ed7a0 0x562b585fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.808+0000 7f4a70711b80 1 --2- 172.21.15.44:0/1273999924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 0x562b585ffde0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.808+0000 7f4a70711b80 1 --2- 172.21.15.44:0/1273999924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 0x562b585fdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.808+0000 7f4a70711b80 1 -- 172.21.15.44:0/1273999924 >> 172.21.15.44:0/1273999924 conn(0x562b58567030 msgr2=0x562b585ed090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.808+0000 7f4a70711b80 1 -- 172.21.15.44:0/1273999924 shutdown_connections 2023-09-10T01:20:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.808+0000 7f4a70711b80 1 -- 172.21.15.44:0/1273999924 wait complete. 2023-09-10T01:20:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.808+0000 7f4a70711b80 1 -- start start 2023-09-10T01:20:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.808+0000 7f4a70711b80 1 Processor -- start 2023-09-10T01:20:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a70711b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585ed7a0 0x562b58607680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a70711b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 0x562b58605980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a70711b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 0x562b58605ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a70711b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b58606400 con 0x562b585ed7a0 2023-09-10T01:20:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a70711b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b58606570 con 0x562b58600cc0 2023-09-10T01:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a70711b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b58606730 con 0x562b585f70d0 2023-09-10T01:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a70711b80 10 client.0 _ll_register_callbacks cb 0x7f4a5800e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a622e7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585ed7a0 0x562b58607680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a622e7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585ed7a0 0x562b58607680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58562/0 (socket says 172.21.15.44:58562) 2023-09-10T01:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a622e7700 1 -- 172.21.15.44:0/585063096 learned_addr learned my addr 172.21.15.44:0/585063096 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a62ae8700 1 --2- 172.21.15.44:0/585063096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 0x562b58605ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a622e7700 1 -- 172.21.15.44:0/585063096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 msgr2=0x562b58605980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a622e7700 1 --2- 172.21.15.44:0/585063096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 0x562b58605980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a622e7700 1 -- 172.21.15.44:0/585063096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 msgr2=0x562b58605ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a622e7700 1 --2- 172.21.15.44:0/585063096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 0x562b58605ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a622e7700 1 -- 172.21.15.44:0/585063096 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5800e9a0 con 0x562b585ed7a0 2023-09-10T01:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.809+0000 7f4a62ae8700 1 --2- 172.21.15.44:0/585063096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 0x562b58605ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.810+0000 7f4a622e7700 1 --2- 172.21.15.44:0/585063096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585ed7a0 0x562b58607680 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4a58011b60 tx=0x7f4a58029510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cdace4e0ed39682 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.810+0000 7f4a45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.810+0000 7f4a45ffb700 1 -- 172.21.15.44:0/585063096 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a58024070 con 0x562b585ed7a0 2023-09-10T01:20:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.810+0000 7f4a45ffb700 1 -- 172.21.15.44:0/585063096 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f4a58029900 con 0x562b585ed7a0 2023-09-10T01:20:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.810+0000 7f4a70711b80 1 -- 172.21.15.44:0/585063096 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a5400f040 con 0x562b585ed7a0 2023-09-10T01:20:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.810+0000 7f4a45ffb700 1 -- 172.21.15.44:0/585063096 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a5802d040 con 0x562b585ed7a0 2023-09-10T01:20:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.810+0000 7f4a70711b80 10 client.19884 fetch_fsmap learned FSMap version 132 2023-09-10T01:20:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.810+0000 7f4a70711b80 1 -- 172.21.15.44:0/585063096 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=4+}) v3 -- 0x562b583971f0 con 0x562b585ed7a0 2023-09-10T01:20:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.810+0000 7f4a45ffb700 1 -- 172.21.15.44:0/585063096 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a5400f040 con 0x562b585ed7a0 2023-09-10T01:20:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.810+0000 7f4a45ffb700 1 -- 172.21.15.44:0/585063096 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 132) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f4a58032b20 con 0x562b585ed7a0 2023-09-10T01:20:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.810+0000 7f4a70711b80 10 client.19884 fetch_fsmap finished waiting for FSMap version 132 2023-09-10T01:20:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.810+0000 7f4a70711b80 10 client.19884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:20:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.811+0000 7f4a70711b80 20 client.19884 populate_metadata read hostname 'smithi044' 2023-09-10T01:20:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.811+0000 7f4a70711b80 1 --2- 172.21.15.44:0/585063096 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562b58665dc0 0x562b586681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.811+0000 7f4a70711b80 4 client.19884 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.811+0000 7f4a70711b80 1 -- 172.21.15.44:0/585063096 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562b5853d8c0 con 0x562b58665dc0 2023-09-10T01:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.811+0000 7f4a61ae6700 1 --2- 172.21.15.44:0/585063096 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562b58665dc0 0x562b586681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.811+0000 7f4a45ffb700 10 client.19884 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.811+0000 7f4a61ae6700 1 --2- 172.21.15.44:0/585063096 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562b58665dc0 0x562b586681e0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=155f3c46e9601919 server_cookie=408a9fa617c560c9 in_seq=0 out_seq=0 2023-09-10T01:20:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.823+0000 7f4a45ffb700 1 -- 172.21.15.44:0/585063096 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562b5853d8c0 con 0x562b58665dc0 2023-09-10T01:20:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.823+0000 7f4a45ffb700 10 client.19884 handle_command_reply: tid=0 2023-09-10T01:20:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.823+0000 7f4a70711b80 1 client.19884 shutdown 2023-09-10T01:20:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.823+0000 7f4a70711b80 1 -- 172.21.15.44:0/585063096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585ed7a0 msgr2=0x562b58607680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.823+0000 7f4a70711b80 1 --2- 172.21.15.44:0/585063096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585ed7a0 0x562b58607680 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4a58011b60 tx=0x7f4a58029510 comp rx=0 tx=0).stop 2023-09-10T01:20:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.823+0000 7f4a70711b80 1 -- 172.21.15.44:0/585063096 shutdown_connections 2023-09-10T01:20:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.823+0000 7f4a70711b80 1 --2- 172.21.15.44:0/585063096 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562b58665dc0 0x562b586681e0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.823+0000 7f4a70711b80 1 --2- 172.21.15.44:0/585063096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b58600cc0 0x562b58605ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.823+0000 7f4a70711b80 1 --2- 172.21.15.44:0/585063096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b585ed7a0 0x562b58607680 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.823+0000 7f4a70711b80 1 --2- 172.21.15.44:0/585063096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b585f70d0 0x562b58605980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.823+0000 7f4a70711b80 1 -- 172.21.15.44:0/585063096 >> 172.21.15.44:0/585063096 conn(0x562b58567030 msgr2=0x562b585fb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.823+0000 7f4a45ffb700 0 client.19884 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:20:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.823+0000 7f4a70711b80 1 -- 172.21.15.44:0/585063096 shutdown_connections 2023-09-10T01:20:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.823+0000 7f4a70711b80 1 -- 172.21.15.44:0/585063096 wait complete. 2023-09-10T01:20:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.823+0000 7f4a70711b80 20 client.19884 trim_cache size 0 max 16384 2023-09-10T01:20:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:20:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:20:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:20:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:20:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.824+0000 7f4a6454b700 1 -- 172.21.15.44:0/927643923 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4a480a07b0 msgr2=0x7f4a480a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.824+0000 7f4a6454b700 1 --2- 172.21.15.44:0/927643923 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4a480a07b0 0x7f4a480a2c70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009fd0 tx=0x7f4a58009260 comp rx=0 tx=0).stop 2023-09-10T01:20:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.824+0000 7f4a6454b700 1 -- 172.21.15.44:0/927643923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c100120 msgr2=0x7f4a5c19caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.824+0000 7f4a6454b700 1 --2- 172.21.15.44:0/927643923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c100120 0x7f4a5c19caf0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009fd0 tx=0x7f4a4c00f690 comp rx=0 tx=0).stop 2023-09-10T01:20:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.824+0000 7f4a6454b700 1 -- 172.21.15.44:0/927643923 shutdown_connections 2023-09-10T01:20:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.824+0000 7f4a6454b700 1 --2- 172.21.15.44:0/927643923 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4a480a07b0 0x7f4a480a2c70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.824+0000 7f4a6454b700 1 --2- 172.21.15.44:0/927643923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a5c1055b0 0x7f4a5c196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.824+0000 7f4a6454b700 1 --2- 172.21.15.44:0/927643923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a5c0ff760 0x7f4a5c19c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.824+0000 7f4a6454b700 1 --2- 172.21.15.44:0/927643923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a5c100120 0x7f4a5c19caf0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.824+0000 7f4a6454b700 1 -- 172.21.15.44:0/927643923 >> 172.21.15.44:0/927643923 conn(0x7f4a5c074f30 msgr2=0x7f4a5c103200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.824+0000 7f4a6454b700 1 -- 172.21.15.44:0/927643923 shutdown_connections 2023-09-10T01:20:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:35.824+0000 7f4a6454b700 1 -- 172.21.15.44:0/927643923 wait complete. 2023-09-10T01:20:36.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:36 smithi099 ceph-mon[102692]: pgmap v1289: 129 pgs: 129 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 97 op/s 2023-09-10T01:20:36.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:36 smithi131 ceph-mon[103713]: pgmap v1289: 129 pgs: 129 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 97 op/s 2023-09-10T01:20:36.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:36 smithi044 ceph-mon[98676]: pgmap v1289: 129 pgs: 129 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 97 op/s 2023-09-10T01:20:38.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:38 smithi099 ceph-mon[102692]: pgmap v1290: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 23 MiB/s wr, 146 op/s 2023-09-10T01:20:38.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:38 smithi131 ceph-mon[103713]: pgmap v1290: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 23 MiB/s wr, 146 op/s 2023-09-10T01:20:38.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:38 smithi044 ceph-mon[98676]: pgmap v1290: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 23 MiB/s wr, 146 op/s 2023-09-10T01:20:40.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:40 smithi099 ceph-mon[102692]: pgmap v1291: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 98 op/s 2023-09-10T01:20:40.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:40 smithi131 ceph-mon[103713]: pgmap v1291: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 98 op/s 2023-09-10T01:20:40.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:40 smithi044 ceph-mon[98676]: pgmap v1291: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 98 op/s 2023-09-10T01:20:42.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:42 smithi099 ceph-mon[102692]: pgmap v1292: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 20 MiB/s wr, 138 op/s 2023-09-10T01:20:42.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:42 smithi131 ceph-mon[103713]: pgmap v1292: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 20 MiB/s wr, 138 op/s 2023-09-10T01:20:42.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:42 smithi044 ceph-mon[98676]: pgmap v1292: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 20 MiB/s wr, 138 op/s 2023-09-10T01:20:44.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:44 smithi099 ceph-mon[102692]: pgmap v1293: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 89 op/s 2023-09-10T01:20:44.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:44 smithi131 ceph-mon[103713]: pgmap v1293: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 89 op/s 2023-09-10T01:20:44.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:44 smithi044 ceph-mon[98676]: pgmap v1293: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 89 op/s 2023-09-10T01:20:46.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:46 smithi099 ceph-mon[102692]: pgmap v1294: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 89 op/s 2023-09-10T01:20:46.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:46 smithi131 ceph-mon[103713]: pgmap v1294: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 89 op/s 2023-09-10T01:20:46.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:46 smithi044 ceph-mon[98676]: pgmap v1294: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 89 op/s 2023-09-10T01:20:48.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:48 smithi099 ceph-mon[102692]: pgmap v1295: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 20 MiB/s wr, 139 op/s 2023-09-10T01:20:48.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:48 smithi131 ceph-mon[103713]: pgmap v1295: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 20 MiB/s wr, 139 op/s 2023-09-10T01:20:48.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:48 smithi044 ceph-mon[98676]: pgmap v1295: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 20 MiB/s wr, 139 op/s 2023-09-10T01:20:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.973+0000 7facc1401700 1 -- 172.21.15.44:0/1871147895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facbc0ff610 msgr2=0x7facbc10d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.973+0000 7facc1401700 1 --2- 172.21.15.44:0/1871147895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facbc0ff610 0x7facbc10d2b0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7facb4009f90 tx=0x7facb400b330 comp rx=0 tx=0).stop 2023-09-10T01:20:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.973+0000 7facc1401700 1 -- 172.21.15.44:0/1871147895 shutdown_connections 2023-09-10T01:20:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.973+0000 7facc1401700 1 --2- 172.21.15.44:0/1871147895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facbc0ff610 0x7facbc10d2b0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.973+0000 7facc1401700 1 --2- 172.21.15.44:0/1871147895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facbc106530 0x7facbc106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.973+0000 7facc1401700 1 --2- 172.21.15.44:0/1871147895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facbc0fec50 0x7facbc0ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.973+0000 7facc1401700 1 -- 172.21.15.44:0/1871147895 >> 172.21.15.44:0/1871147895 conn(0x7facbc075760 msgr2=0x7facbc075b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.973+0000 7facc1401700 1 -- 172.21.15.44:0/1871147895 shutdown_connections 2023-09-10T01:20:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.973+0000 7facc1401700 1 -- 172.21.15.44:0/1871147895 wait complete. 2023-09-10T01:20:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.974+0000 7facc1401700 1 Processor -- start 2023-09-10T01:20:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.974+0000 7facc1401700 1 -- start start 2023-09-10T01:20:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.974+0000 7facc1401700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facbc0fec50 0x7facbc0ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.974+0000 7facc1401700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facbc0ff610 0x7facbc100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.974+0000 7facc1401700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facbc106530 0x7facbc1a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.974+0000 7facc1401700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facbc100ba0 con 0x7facbc0fec50 2023-09-10T01:20:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.974+0000 7facc1401700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facbc1a4c70 con 0x7facbc106530 2023-09-10T01:20:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.974+0000 7facc1401700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facbc1a4db0 con 0x7facbc0ff610 2023-09-10T01:20:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.974+0000 7facbaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facbc0fec50 0x7facbc0ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.974+0000 7facbaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facbc0fec50 0x7facbc0ffe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58574/0 (socket says 172.21.15.44:58574) 2023-09-10T01:20:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.975+0000 7facbaffd700 1 -- 172.21.15.44:0/1164797869 learned_addr learned my addr 172.21.15.44:0/1164797869 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.975+0000 7facbb7fe700 1 --2- 172.21.15.44:0/1164797869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facbc106530 0x7facbc1a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.975+0000 7facba7fc700 1 --2- 172.21.15.44:0/1164797869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facbc0ff610 0x7facbc100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.975+0000 7facbaffd700 1 -- 172.21.15.44:0/1164797869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facbc0ff610 msgr2=0x7facbc100370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.975+0000 7facbaffd700 1 --2- 172.21.15.44:0/1164797869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facbc0ff610 0x7facbc100370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.975+0000 7facbaffd700 1 -- 172.21.15.44:0/1164797869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facbc106530 msgr2=0x7facbc1a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.975+0000 7facbaffd700 1 --2- 172.21.15.44:0/1164797869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facbc106530 0x7facbc1a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.975+0000 7facbaffd700 1 -- 172.21.15.44:0/1164797869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb4009c40 con 0x7facbc0fec50 2023-09-10T01:20:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.975+0000 7facbaffd700 1 --2- 172.21.15.44:0/1164797869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facbc0fec50 0x7facbc0ffe30 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7facac00e9f0 tx=0x7facac00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.975+0000 7faca3fff700 1 -- 172.21.15.44:0/1164797869 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac004220 con 0x7facbc0fec50 2023-09-10T01:20:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.975+0000 7faca3fff700 1 -- 172.21.15.44:0/1164797869 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facac004380 con 0x7facbc0fec50 2023-09-10T01:20:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.975+0000 7faca3fff700 1 -- 172.21.15.44:0/1164797869 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac003a80 con 0x7facbc0fec50 2023-09-10T01:20:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.975+0000 7facc1401700 1 -- 172.21.15.44:0/1164797869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facbc1a4fb0 con 0x7facbc0fec50 2023-09-10T01:20:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.975+0000 7facc1401700 1 -- 172.21.15.44:0/1164797869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facbc1a5440 con 0x7facbc0fec50 2023-09-10T01:20:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.978+0000 7faca3fff700 1 -- 172.21.15.44:0/1164797869 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7facac003c20 con 0x7facbc0fec50 2023-09-10T01:20:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.979+0000 7faca3fff700 1 --2- 172.21.15.44:0/1164797869 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faca40a07b0 0x7faca40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.979+0000 7facba7fc700 1 --2- 172.21.15.44:0/1164797869 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faca40a07b0 0x7faca40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.979+0000 7faca3fff700 1 -- 172.21.15.44:0/1164797869 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 9540+0+0 (secure 0 0 0) 0x7facac014070 con 0x7facbc0fec50 2023-09-10T01:20:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7faccd5c7b80 1 Processor -- start 2023-09-10T01:20:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7faccd5c7b80 1 -- start start 2023-09-10T01:20:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7facba7fc700 1 --2- 172.21.15.44:0/1164797869 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faca40a07b0 0x7faca40a2c70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7facbc101620 tx=0x7facb0009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7faccd5c7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aeb9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7faccd5c7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb9bb0 0x555d3aeb3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7faccd5c7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb39a0 0x555d3aeb3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7faccd5c7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d3ae0c6d0 con 0x555d3aeb9290 2023-09-10T01:20:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7faccd5c7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d3ae09ff0 con 0x555d3aeb39a0 2023-09-10T01:20:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7faccd5c7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d3ae058e0 con 0x555d3aeb9bb0 2023-09-10T01:20:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7facbaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aeb9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7facbaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aeb9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58582/0 (socket says 172.21.15.44:58582) 2023-09-10T01:20:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7facbaffd700 1 -- 172.21.15.44:0/2576139718 learned_addr learned my addr 172.21.15.44:0/2576139718 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7facba7fc700 1 --2- 172.21.15.44:0/2576139718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb9bb0 0x555d3aeb3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7facbb7fe700 1 --2- 172.21.15.44:0/2576139718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb39a0 0x555d3aeb3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7facbaffd700 1 -- 172.21.15.44:0/2576139718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb9bb0 msgr2=0x555d3aeb3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7facbaffd700 1 --2- 172.21.15.44:0/2576139718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb9bb0 0x555d3aeb3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7facbaffd700 1 -- 172.21.15.44:0/2576139718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb39a0 msgr2=0x555d3aeb3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7facbaffd700 1 --2- 172.21.15.44:0/2576139718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb39a0 0x555d3aeb3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.980+0000 7facbaffd700 1 -- 172.21.15.44:0/2576139718 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d3ac31d80 con 0x555d3aeb9290 2023-09-10T01:20:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.981+0000 7facbaffd700 1 --2- 172.21.15.44:0/2576139718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aeb9670 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7facac0a10a0 tx=0x7facac099460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fb60e8b142af9c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.981+0000 7faca17fa700 1 -- 172.21.15.44:0/2576139718 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac09d170 con 0x555d3aeb9290 2023-09-10T01:20:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.981+0000 7faca17fa700 1 -- 172.21.15.44:0/2576139718 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facac099560 con 0x555d3aeb9290 2023-09-10T01:20:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.981+0000 7faca17fa700 1 -- 172.21.15.44:0/2576139718 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac0cc170 con 0x555d3aeb9290 2023-09-10T01:20:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.982+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2576139718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 msgr2=0x555d3aeb9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.982+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/2576139718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aeb9670 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7facac0a10a0 tx=0x7facac099460 comp rx=0 tx=0).stop 2023-09-10T01:20:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.982+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2576139718 shutdown_connections 2023-09-10T01:20:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.982+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/2576139718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb39a0 0x555d3aeb3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.982+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/2576139718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aeb9670 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.982+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/2576139718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb9bb0 0x555d3aeb3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.982+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2576139718 >> 172.21.15.44:0/2576139718 conn(0x555d3ae28030 msgr2=0x555d3aeae6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.982+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2576139718 shutdown_connections 2023-09-10T01:20:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.982+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2576139718 wait complete. 2023-09-10T01:20:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.982+0000 7faccd5c7b80 1 -- start start 2023-09-10T01:20:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7faccd5c7b80 1 Processor -- start 2023-09-10T01:20:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7faccd5c7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb39a0 0x555d3aebfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7faccd5c7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aebce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7faccd5c7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb9bb0 0x555d3aebd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7faccd5c7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d3ada3b10 con 0x555d3aeb9290 2023-09-10T01:20:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7faccd5c7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d3ada3810 con 0x555d3aeb39a0 2023-09-10T01:20:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7faccd5c7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d3ac878c0 con 0x555d3aeb9bb0 2023-09-10T01:20:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7faccd5c7b80 10 client.0 _ll_register_callbacks cb 0x555d3aceda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:20:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7facbaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb39a0 0x555d3aebfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7facba7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aebce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7facbaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb39a0 0x555d3aebfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60328/0 (socket says 172.21.15.44:60328) 2023-09-10T01:20:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7facbaffd700 1 -- 172.21.15.44:0/2325206863 learned_addr learned my addr 172.21.15.44:0/2325206863 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7facba7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aebce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58590/0 (socket says 172.21.15.44:58590) 2023-09-10T01:20:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7facbb7fe700 1 --2- 172.21.15.44:0/2325206863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb9bb0 0x555d3aebd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7facbaffd700 1 -- 172.21.15.44:0/2325206863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb9bb0 msgr2=0x555d3aebd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7facbaffd700 1 --2- 172.21.15.44:0/2325206863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb9bb0 0x555d3aebd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7facbaffd700 1 -- 172.21.15.44:0/2325206863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 msgr2=0x555d3aebce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7facbaffd700 1 --2- 172.21.15.44:0/2325206863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aebce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.983+0000 7facbaffd700 1 -- 172.21.15.44:0/2325206863 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facbc1a5440 con 0x555d3aeb39a0 2023-09-10T01:20:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.984+0000 7facbaffd700 1 --2- 172.21.15.44:0/2325206863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb39a0 0x555d3aebfbb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7facac00e9c0 tx=0x7facac0959d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd326447b9e54832 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.984+0000 7fac9eff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:20:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.984+0000 7fac9eff5700 1 -- 172.21.15.44:0/2325206863 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac0cc380 con 0x555d3aeb39a0 2023-09-10T01:20:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.984+0000 7fac9eff5700 1 -- 172.21.15.44:0/2325206863 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 9540+0+0 (secure 0 0 0) 0x7facac09d170 con 0x555d3aeb39a0 2023-09-10T01:20:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.984+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2325206863 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d3ad5d490 con 0x555d3aeb39a0 2023-09-10T01:20:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.984+0000 7fac9eff5700 1 -- 172.21.15.44:0/2325206863 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7facac0a15d0 con 0x555d3aeb39a0 2023-09-10T01:20:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.984+0000 7faccd5c7b80 10 client.29519 fetch_fsmap learned FSMap version 132 2023-09-10T01:20:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.984+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2325206863 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=4+}) v3 -- 0x555d3ac1ac00 con 0x555d3aeb39a0 2023-09-10T01:20:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.984+0000 7fac9eff5700 1 -- 172.21.15.44:0/2325206863 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555d3ad5d490 con 0x555d3aeb39a0 2023-09-10T01:20:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.984+0000 7fac9eff5700 1 -- 172.21.15.44:0/2325206863 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 132) v1 ==== 7331+0+0 (secure 0 0 0) 0x7facac0a1140 con 0x555d3aeb39a0 2023-09-10T01:20:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.984+0000 7faccd5c7b80 10 client.29519 fetch_fsmap finished waiting for FSMap version 132 2023-09-10T01:20:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.984+0000 7faccd5c7b80 10 client.29519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:20:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.984+0000 7faccd5c7b80 20 client.29519 populate_metadata read hostname 'smithi044' 2023-09-10T01:20:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.984+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/2325206863 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555d3aecd240 0x555d3aecf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.985+0000 7faccd5c7b80 4 client.29519 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:20:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.985+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2325206863 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d3ae1a840 con 0x555d3aecd240 2023-09-10T01:20:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.985+0000 7facba7fc700 1 --2- 172.21.15.44:0/2325206863 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555d3aecd240 0x555d3aecf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.985+0000 7fac9eff5700 10 client.29519 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:20:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.985+0000 7facba7fc700 1 --2- 172.21.15.44:0/2325206863 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555d3aecd240 0x555d3aecf630 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2845eb2c6b8e6cf9 server_cookie=eaa2ca7720ad2e36 in_seq=0 out_seq=0 2023-09-10T01:20:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.986+0000 7fac9eff5700 1 -- 172.21.15.44:0/2325206863 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555d3ae1a840 con 0x555d3aecd240 2023-09-10T01:20:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.986+0000 7fac9eff5700 10 client.29519 handle_command_reply: tid=0 2023-09-10T01:20:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.986+0000 7faccd5c7b80 1 client.29519 shutdown 2023-09-10T01:20:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.986+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2325206863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb39a0 msgr2=0x555d3aebfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.986+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/2325206863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb39a0 0x555d3aebfbb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7facac00e9c0 tx=0x7facac0959d0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.986+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2325206863 shutdown_connections 2023-09-10T01:20:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.986+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/2325206863 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555d3aecd240 0x555d3aecf630 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.986+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/2325206863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb9bb0 0x555d3aebd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.986+0000 7fac9eff5700 0 client.29519 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:20:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.986+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/2325206863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb39a0 0x555d3aebfbb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.986+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/2325206863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aebce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.986+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2325206863 >> 172.21.15.44:0/2325206863 conn(0x555d3ae28030 msgr2=0x555d3aead070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.986+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2325206863 shutdown_connections 2023-09-10T01:20:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.986+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2325206863 wait complete. 2023-09-10T01:20:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:48.986+0000 7faccd5c7b80 20 client.29519 trim_cache size 0 max 16384 2023-09-10T01:20:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.003+0000 7faccd5c7b80 1 Processor -- start 2023-09-10T01:20:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.003+0000 7faccd5c7b80 1 -- start start 2023-09-10T01:20:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.003+0000 7faccd5c7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb39a0 0x555d3aeb3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.003+0000 7faccd5c7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb9290 0x555d3aeb87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.003+0000 7faccd5c7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9bb0 0x555d3aec4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.003+0000 7faccd5c7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d3aeaaa10 con 0x555d3aeb9bb0 2023-09-10T01:20:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.003+0000 7faccd5c7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d3aec4b60 con 0x555d3aeb9290 2023-09-10T01:20:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.003+0000 7faccd5c7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d3aebb390 con 0x555d3aeb39a0 2023-09-10T01:20:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.004+0000 7facbb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9bb0 0x555d3aec4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.004+0000 7facba7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb9290 0x555d3aeb87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.004+0000 7facbaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb39a0 0x555d3aeb3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.004+0000 7facba7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb9290 0x555d3aeb87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60338/0 (socket says 172.21.15.44:60338) 2023-09-10T01:20:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.004+0000 7facba7fc700 1 -- 172.21.15.44:0/2858653491 learned_addr learned my addr 172.21.15.44:0/2858653491 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.004+0000 7facba7fc700 1 -- 172.21.15.44:0/2858653491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb39a0 msgr2=0x555d3aeb3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.004+0000 7facba7fc700 1 --2- 172.21.15.44:0/2858653491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb39a0 0x555d3aeb3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.004+0000 7facba7fc700 1 -- 172.21.15.44:0/2858653491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9bb0 msgr2=0x555d3aec4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.004+0000 7facba7fc700 1 --2- 172.21.15.44:0/2858653491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9bb0 0x555d3aec4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.004+0000 7facba7fc700 1 -- 172.21.15.44:0/2858653491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d3ac581f0 con 0x555d3aeb9290 2023-09-10T01:20:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.004+0000 7facbb7fe700 1 --2- 172.21.15.44:0/2858653491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9bb0 0x555d3aec4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:20:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.004+0000 7facba7fc700 1 --2- 172.21.15.44:0/2858653491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb9290 0x555d3aeb87a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7facb00097c0 tx=0x7facb0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeae7adbc751de8f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.004+0000 7faca1ffb700 1 -- 172.21.15.44:0/2858653491 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0022070 con 0x555d3aeb9290 2023-09-10T01:20:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.004+0000 7faca1ffb700 1 -- 172.21.15.44:0/2858653491 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facb0023040 con 0x555d3aeb9290 2023-09-10T01:20:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.004+0000 7faca1ffb700 1 -- 172.21.15.44:0/2858653491 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0003bd0 con 0x555d3aeb9290 2023-09-10T01:20:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.005+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2858653491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb9290 msgr2=0x555d3aeb87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.005+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/2858653491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb9290 0x555d3aeb87a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7facb00097c0 tx=0x7facb0014000 comp rx=0 tx=0).stop 2023-09-10T01:20:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.005+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2858653491 shutdown_connections 2023-09-10T01:20:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.005+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/2858653491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9bb0 0x555d3aec4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.005+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/2858653491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb39a0 0x555d3aeb3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.005+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/2858653491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb9290 0x555d3aeb87a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.005+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2858653491 >> 172.21.15.44:0/2858653491 conn(0x555d3ae28030 msgr2=0x555d3aeba060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.005+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2858653491 shutdown_connections 2023-09-10T01:20:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.005+0000 7faccd5c7b80 1 -- 172.21.15.44:0/2858653491 wait complete. 2023-09-10T01:20:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.006+0000 7faccd5c7b80 1 -- start start 2023-09-10T01:20:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.006+0000 7faccd5c7b80 1 Processor -- start 2023-09-10T01:20:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.006+0000 7faccd5c7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb39a0 0x555d3aec44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.006+0000 7faccd5c7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aec2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.006+0000 7faccd5c7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb9bb0 0x555d3aec2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.006+0000 7faccd5c7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d3aec34b0 con 0x555d3aeb9290 2023-09-10T01:20:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.006+0000 7faccd5c7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d3aec3620 con 0x555d3aeb9bb0 2023-09-10T01:20:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.006+0000 7faccd5c7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d3af21070 con 0x555d3aeb39a0 2023-09-10T01:20:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.006+0000 7faccd5c7b80 10 client.0 _ll_register_callbacks cb 0x7facac0af070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:20:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.006+0000 7facba7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aec2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.006+0000 7facbaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb39a0 0x555d3aec44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.006+0000 7facbaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb39a0 0x555d3aec44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37252/0 (socket says 172.21.15.44:37252) 2023-09-10T01:20:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.006+0000 7facba7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aec2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58604/0 (socket says 172.21.15.44:58604) 2023-09-10T01:20:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.006+0000 7facbaffd700 1 -- 172.21.15.44:0/1239913219 learned_addr learned my addr 172.21.15.44:0/1239913219 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:20:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.006+0000 7facbb7fe700 1 --2- 172.21.15.44:0/1239913219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb9bb0 0x555d3aec2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.007+0000 7facba7fc700 1 -- 172.21.15.44:0/1239913219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb39a0 msgr2=0x555d3aec44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.007+0000 7facba7fc700 1 --2- 172.21.15.44:0/1239913219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb39a0 0x555d3aec44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.007+0000 7facba7fc700 1 -- 172.21.15.44:0/1239913219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb9bb0 msgr2=0x555d3aec2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:20:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.007+0000 7facba7fc700 1 --2- 172.21.15.44:0/1239913219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb9bb0 0x555d3aec2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.007+0000 7facba7fc700 1 -- 172.21.15.44:0/1239913219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb0017070 con 0x555d3aeb9290 2023-09-10T01:20:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.007+0000 7facbb7fe700 1 --2- 172.21.15.44:0/1239913219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb9bb0 0x555d3aec2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:20:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.007+0000 7facba7fc700 1 --2- 172.21.15.44:0/1239913219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aec2840 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x555d3aecd120 tx=0x7facb002bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7ffa2f1a05b3a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:20:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.007+0000 7fac9eff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:20:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.007+0000 7fac9eff5700 1 -- 172.21.15.44:0/1239913219 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0022050 con 0x555d3aeb9290 2023-09-10T01:20:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.007+0000 7faccd5c7b80 1 -- 172.21.15.44:0/1239913219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facac099560 con 0x555d3aeb9290 2023-09-10T01:20:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.007+0000 7fac9eff5700 1 -- 172.21.15.44:0/1239913219 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 9540+0+0 (secure 0 0 0) 0x7facb002b4c0 con 0x555d3aeb9290 2023-09-10T01:20:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.007+0000 7fac9eff5700 1 -- 172.21.15.44:0/1239913219 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7facb00037b0 con 0x555d3aeb9290 2023-09-10T01:20:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.007+0000 7faccd5c7b80 10 client.19902 fetch_fsmap learned FSMap version 132 2023-09-10T01:20:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.007+0000 7faccd5c7b80 1 -- 172.21.15.44:0/1239913219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=4+}) v3 -- 0x555d3ac30250 con 0x555d3aeb9290 2023-09-10T01:20:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.008+0000 7fac9eff5700 1 -- 172.21.15.44:0/1239913219 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facac099560 con 0x555d3aeb9290 2023-09-10T01:20:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.008+0000 7fac9eff5700 1 -- 172.21.15.44:0/1239913219 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 132) v1 ==== 7331+0+0 (secure 0 0 0) 0x7facb00337d0 con 0x555d3aeb9290 2023-09-10T01:20:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.008+0000 7faccd5c7b80 10 client.19902 fetch_fsmap finished waiting for FSMap version 132 2023-09-10T01:20:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.008+0000 7faccd5c7b80 10 client.19902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:20:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.008+0000 7faccd5c7b80 20 client.19902 populate_metadata read hostname 'smithi044' 2023-09-10T01:20:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.008+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/1239913219 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555d3af22810 0x555d3af24c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:20:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.008+0000 7faccd5c7b80 4 client.19902 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:20:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.008+0000 7faccd5c7b80 1 -- 172.21.15.44:0/1239913219 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555d3adfe8c0 con 0x555d3af22810 2023-09-10T01:20:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.008+0000 7facbaffd700 1 --2- 172.21.15.44:0/1239913219 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555d3af22810 0x555d3af24c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:20:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.009+0000 7facbaffd700 1 --2- 172.21.15.44:0/1239913219 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555d3af22810 0x555d3af24c30 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1eb9c727f2d9ae2 server_cookie=e45392f644601973 in_seq=0 out_seq=0 2023-09-10T01:20:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.009+0000 7fac9eff5700 10 client.19902 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:20:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.030+0000 7fac9eff5700 1 -- 172.21.15.44:0/1239913219 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555d3adfe8c0 con 0x555d3af22810 2023-09-10T01:20:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.030+0000 7fac9eff5700 10 client.19902 handle_command_reply: tid=0 2023-09-10T01:20:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.030+0000 7faccd5c7b80 1 client.19902 shutdown 2023-09-10T01:20:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.030+0000 7faccd5c7b80 1 -- 172.21.15.44:0/1239913219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 msgr2=0x555d3aec2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.030+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/1239913219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aec2840 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x555d3aecd120 tx=0x7facb002bd10 comp rx=0 tx=0).stop 2023-09-10T01:20:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.030+0000 7faccd5c7b80 1 -- 172.21.15.44:0/1239913219 shutdown_connections 2023-09-10T01:20:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.030+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/1239913219 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555d3af22810 0x555d3af24c30 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.030+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/1239913219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d3aeb9bb0 0x555d3aec2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.030+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/1239913219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d3aeb39a0 0x555d3aec44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.030+0000 7fac9eff5700 0 client.19902 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:20:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.030+0000 7faccd5c7b80 1 --2- 172.21.15.44:0/1239913219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d3aeb9290 0x555d3aec2840 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.030+0000 7faccd5c7b80 1 -- 172.21.15.44:0/1239913219 >> 172.21.15.44:0/1239913219 conn(0x555d3ae28030 msgr2=0x555d3aeba8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.030+0000 7faccd5c7b80 1 -- 172.21.15.44:0/1239913219 shutdown_connections 2023-09-10T01:20:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.030+0000 7faccd5c7b80 1 -- 172.21.15.44:0/1239913219 wait complete. 2023-09-10T01:20:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.030+0000 7faccd5c7b80 20 client.19902 trim_cache size 0 max 16384 2023-09-10T01:20:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:20:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:20:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:20:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:20:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.031+0000 7facc1401700 1 -- 172.21.15.44:0/1164797869 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faca40a07b0 msgr2=0x7faca40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.031+0000 7facc1401700 1 --2- 172.21.15.44:0/1164797869 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faca40a07b0 0x7faca40a2c70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7facbc101620 tx=0x7facb0009260 comp rx=0 tx=0).stop 2023-09-10T01:20:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.031+0000 7facc1401700 1 -- 172.21.15.44:0/1164797869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facbc0fec50 msgr2=0x7facbc0ffe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:20:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.031+0000 7facc1401700 1 --2- 172.21.15.44:0/1164797869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facbc0fec50 0x7facbc0ffe30 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7facac00e9f0 tx=0x7facac00ed00 comp rx=0 tx=0).stop 2023-09-10T01:20:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.031+0000 7facc1401700 1 -- 172.21.15.44:0/1164797869 shutdown_connections 2023-09-10T01:20:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.031+0000 7facc1401700 1 --2- 172.21.15.44:0/1164797869 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7faca40a07b0 0x7faca40a2c70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.031+0000 7facc1401700 1 --2- 172.21.15.44:0/1164797869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facbc106530 0x7facbc1a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.031+0000 7facc1401700 1 --2- 172.21.15.44:0/1164797869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facbc0fec50 0x7facbc0ffe30 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.031+0000 7facc1401700 1 --2- 172.21.15.44:0/1164797869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facbc0ff610 0x7facbc100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:20:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.031+0000 7facc1401700 1 -- 172.21.15.44:0/1164797869 >> 172.21.15.44:0/1164797869 conn(0x7facbc075760 msgr2=0x7facbc104020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:20:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.031+0000 7facc1401700 1 -- 172.21.15.44:0/1164797869 shutdown_connections 2023-09-10T01:20:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:20:49.031+0000 7facc1401700 1 -- 172.21.15.44:0/1164797869 wait complete. 2023-09-10T01:20:50.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:50 smithi099 ceph-mon[102692]: pgmap v1296: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 90 op/s 2023-09-10T01:20:50.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:50 smithi131 ceph-mon[103713]: pgmap v1296: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 90 op/s 2023-09-10T01:20:50.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:50 smithi044 ceph-mon[98676]: pgmap v1296: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 90 op/s 2023-09-10T01:20:52.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:52 smithi099 ceph-mon[102692]: pgmap v1297: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 20 MiB/s wr, 131 op/s 2023-09-10T01:20:52.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:52 smithi131 ceph-mon[103713]: pgmap v1297: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 20 MiB/s wr, 131 op/s 2023-09-10T01:20:52.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:52 smithi044 ceph-mon[98676]: pgmap v1297: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 20 MiB/s wr, 131 op/s 2023-09-10T01:20:54.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:54 smithi099 ceph-mon[102692]: pgmap v1298: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 15 MiB/s wr, 90 op/s 2023-09-10T01:20:54.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:54 smithi131 ceph-mon[103713]: pgmap v1298: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 15 MiB/s wr, 90 op/s 2023-09-10T01:20:54.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:54 smithi044 ceph-mon[98676]: pgmap v1298: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 15 MiB/s wr, 90 op/s 2023-09-10T01:20:56.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:56 smithi099 ceph-mon[102692]: pgmap v1299: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 15 MiB/s wr, 90 op/s 2023-09-10T01:20:56.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:56 smithi131 ceph-mon[103713]: pgmap v1299: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 15 MiB/s wr, 90 op/s 2023-09-10T01:20:56.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:56 smithi044 ceph-mon[98676]: pgmap v1299: 129 pgs: 129 active+clean; 16 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 15 MiB/s wr, 90 op/s 2023-09-10T01:20:58.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:58 smithi131 ceph-mon[103713]: pgmap v1300: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 21 MiB/s wr, 133 op/s 2023-09-10T01:20:58.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:58 smithi044 ceph-mon[98676]: pgmap v1300: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 21 MiB/s wr, 133 op/s 2023-09-10T01:20:58.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:58 smithi099 ceph-mon[102692]: pgmap v1300: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 21 MiB/s wr, 133 op/s 2023-09-10T01:20:59.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:20:59 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:20:59.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:20:59 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:20:59.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:20:59 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:21:00.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:00 smithi131 ceph-mon[103713]: pgmap v1301: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 14 MiB/s wr, 83 op/s 2023-09-10T01:21:00.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:00 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:21:00.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:00 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:21:00.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:00 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:21:00.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:00 smithi044 ceph-mon[98676]: pgmap v1301: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 14 MiB/s wr, 83 op/s 2023-09-10T01:21:00.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:00 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:21:00.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:00 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:21:00.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:00 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:21:00.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:00 smithi099 ceph-mon[102692]: pgmap v1301: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 14 MiB/s wr, 83 op/s 2023-09-10T01:21:00.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:00 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:21:00.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:00 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:21:00.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:00 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:21:01.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:01 smithi131 ceph-mon[103713]: pgmap v1302: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 15 MiB/s wr, 90 op/s 2023-09-10T01:21:01.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:01 smithi131 ceph-mon[103713]: pgmap v1303: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.1 MiB/s wr, 57 op/s 2023-09-10T01:21:01.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:01 smithi044 ceph-mon[98676]: pgmap v1302: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 15 MiB/s wr, 90 op/s 2023-09-10T01:21:01.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:01 smithi044 ceph-mon[98676]: pgmap v1303: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.1 MiB/s wr, 57 op/s 2023-09-10T01:21:01.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:01 smithi099 ceph-mon[102692]: pgmap v1302: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 15 MiB/s wr, 90 op/s 2023-09-10T01:21:01.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:01 smithi099 ceph-mon[102692]: pgmap v1303: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.1 MiB/s wr, 57 op/s 2023-09-10T01:21:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.156+0000 7f5b4b535700 1 -- 172.21.15.44:0/3819413705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b440679f0 msgr2=0x7f5b44067e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.156+0000 7f5b4b535700 1 --2- 172.21.15.44:0/3819413705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b440679f0 0x7f5b44067e70 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f5b400099c0 tx=0x7f5b40009cd0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.156+0000 7f5b4b535700 1 -- 172.21.15.44:0/3819413705 shutdown_connections 2023-09-10T01:21:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.156+0000 7f5b4b535700 1 --2- 172.21.15.44:0/3819413705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b440683b0 0x7f5b4410d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.156+0000 7f5b4b535700 1 --2- 172.21.15.44:0/3819413705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b44106650 0x7f5b44106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.156+0000 7f5b4b535700 1 --2- 172.21.15.44:0/3819413705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b440679f0 0x7f5b44067e70 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.156+0000 7f5b4b535700 1 -- 172.21.15.44:0/3819413705 >> 172.21.15.44:0/3819413705 conn(0x7f5b44074f30 msgr2=0x7f5b44075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.156+0000 7f5b4b535700 1 -- 172.21.15.44:0/3819413705 shutdown_connections 2023-09-10T01:21:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.156+0000 7f5b4b535700 1 -- 172.21.15.44:0/3819413705 wait complete. 2023-09-10T01:21:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.157+0000 7f5b4b535700 1 Processor -- start 2023-09-10T01:21:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.157+0000 7f5b4b535700 1 -- start start 2023-09-10T01:21:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.157+0000 7f5b4b535700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b440679f0 0x7f5b4419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.157+0000 7f5b4b535700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b440683b0 0x7f5b4419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.157+0000 7f5b4b535700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b44106650 0x7f5b44196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.157+0000 7f5b4b535700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b44196bb0 con 0x7f5b44106650 2023-09-10T01:21:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.157+0000 7f5b4b535700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b44196d20 con 0x7f5b440679f0 2023-09-10T01:21:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.157+0000 7f5b4b535700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b44196ee0 con 0x7f5b440683b0 2023-09-10T01:21:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.157+0000 7f5b49ad2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b44106650 0x7f5b44196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.158+0000 7f5b49ad2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b44106650 0x7f5b44196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58620/0 (socket says 172.21.15.44:58620) 2023-09-10T01:21:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.158+0000 7f5b48ad0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b440683b0 0x7f5b4419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.158+0000 7f5b49ad2700 1 -- 172.21.15.44:0/1875640861 learned_addr learned my addr 172.21.15.44:0/1875640861 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.158+0000 7f5b492d1700 1 --2- 172.21.15.44:0/1875640861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b440679f0 0x7f5b4419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.158+0000 7f5b49ad2700 1 -- 172.21.15.44:0/1875640861 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b440683b0 msgr2=0x7f5b4419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.158+0000 7f5b49ad2700 1 --2- 172.21.15.44:0/1875640861 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b440683b0 0x7f5b4419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.158+0000 7f5b49ad2700 1 -- 172.21.15.44:0/1875640861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b440679f0 msgr2=0x7f5b4419c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.158+0000 7f5b49ad2700 1 --2- 172.21.15.44:0/1875640861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b440679f0 0x7f5b4419c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.158+0000 7f5b49ad2700 1 -- 172.21.15.44:0/1875640861 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b38009520 con 0x7f5b44106650 2023-09-10T01:21:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.158+0000 7f5b48ad0700 1 --2- 172.21.15.44:0/1875640861 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b440683b0 0x7f5b4419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:21:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.158+0000 7f5b49ad2700 1 --2- 172.21.15.44:0/1875640861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b44106650 0x7f5b44196540 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c007ee0 tx=0x7f5b3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.158+0000 7f5b367fc700 1 -- 172.21.15.44:0/1875640861 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c017070 con 0x7f5b44106650 2023-09-10T01:21:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.158+0000 7f5b367fc700 1 -- 172.21.15.44:0/1875640861 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b3c00f040 con 0x7f5b44106650 2023-09-10T01:21:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.158+0000 7f5b4b535700 1 -- 172.21.15.44:0/1875640861 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b400095f0 con 0x7f5b44106650 2023-09-10T01:21:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.158+0000 7f5b367fc700 1 -- 172.21.15.44:0/1875640861 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c005570 con 0x7f5b44106650 2023-09-10T01:21:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.158+0000 7f5b4b535700 1 -- 172.21.15.44:0/1875640861 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b441a34f0 con 0x7f5b44106650 2023-09-10T01:21:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.161+0000 7f5b367fc700 1 -- 172.21.15.44:0/1875640861 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f5b3c00db20 con 0x7f5b44106650 2023-09-10T01:21:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.161+0000 7f5b367fc700 1 --2- 172.21.15.44:0/1875640861 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5b300a07b0 0x7f5b300a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.161+0000 7f5b367fc700 1 -- 172.21.15.44:0/1875640861 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f5b3c0cbb50 con 0x7f5b44106650 2023-09-10T01:21:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.161+0000 7f5b492d1700 1 --2- 172.21.15.44:0/1875640861 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5b300a07b0 0x7f5b300a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.162+0000 7f5b492d1700 1 --2- 172.21.15.44:0/1875640861 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5b300a07b0 0x7f5b300a2c70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009d70 tx=0x7f5b38009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.162+0000 7f5b576fbb80 1 Processor -- start 2023-09-10T01:21:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.162+0000 7f5b576fbb80 1 -- start start 2023-09-10T01:21:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.162+0000 7f5b576fbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac8b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.162+0000 7f5b576fbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85300 0x55ee3ac85780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.162+0000 7f5b576fbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac91f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.162+0000 7f5b576fbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3abde6d0 con 0x55ee3ac85cc0 2023-09-10T01:21:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.162+0000 7f5b576fbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3abdbff0 con 0x55ee3ac8b1a0 2023-09-10T01:21:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.162+0000 7f5b576fbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3abd78e0 con 0x55ee3ac85300 2023-09-10T01:21:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.162+0000 7f5b48ad0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85300 0x55ee3ac85780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.162+0000 7f5b49ad2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac91f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.162+0000 7f5b48ad0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85300 0x55ee3ac85780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37276/0 (socket says 172.21.15.44:37276) 2023-09-10T01:21:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.162+0000 7f5b48ad0700 1 -- 172.21.15.44:0/3323490415 learned_addr learned my addr 172.21.15.44:0/3323490415 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.162+0000 7f5b492d1700 1 --2- 172.21.15.44:0/3323490415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac8b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.163+0000 7f5b48ad0700 1 -- 172.21.15.44:0/3323490415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 msgr2=0x55ee3ac8b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.163+0000 7f5b48ad0700 1 --2- 172.21.15.44:0/3323490415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac8b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.163+0000 7f5b48ad0700 1 -- 172.21.15.44:0/3323490415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85cc0 msgr2=0x55ee3ac91f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.163+0000 7f5b48ad0700 1 --2- 172.21.15.44:0/3323490415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac91f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.163+0000 7f5b48ad0700 1 -- 172.21.15.44:0/3323490415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee3aa03d80 con 0x55ee3ac85300 2023-09-10T01:21:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.163+0000 7f5b48ad0700 1 --2- 172.21.15.44:0/3323490415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85300 0x55ee3ac85780 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000b580 tx=0x7f5b40005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a0f2fce8ddc42a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.163+0000 7f5b2f7fe700 1 -- 172.21.15.44:0/3323490415 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4000be40 con 0x55ee3ac85300 2023-09-10T01:21:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.163+0000 7f5b2f7fe700 1 -- 172.21.15.44:0/3323490415 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b4001e040 con 0x55ee3ac85300 2023-09-10T01:21:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.163+0000 7f5b2f7fe700 1 -- 172.21.15.44:0/3323490415 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40003dd0 con 0x55ee3ac85300 2023-09-10T01:21:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.164+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3323490415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85300 msgr2=0x55ee3ac85780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.164+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/3323490415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85300 0x55ee3ac85780 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000b580 tx=0x7f5b40005c90 comp rx=0 tx=0).stop 2023-09-10T01:21:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.164+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3323490415 shutdown_connections 2023-09-10T01:21:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.164+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/3323490415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac91f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.164+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/3323490415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac8b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.164+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/3323490415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85300 0x55ee3ac85780 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.164+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3323490415 >> 172.21.15.44:0/3323490415 conn(0x55ee3abfa030 msgr2=0x55ee3ac80560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.164+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3323490415 shutdown_connections 2023-09-10T01:21:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.164+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3323490415 wait complete. 2023-09-10T01:21:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.164+0000 7f5b576fbb80 1 -- start start 2023-09-10T01:21:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.164+0000 7f5b576fbb80 1 Processor -- start 2023-09-10T01:21:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.164+0000 7f5b576fbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85300 0x55ee3ac91e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b576fbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac8f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b576fbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac8f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b576fbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3ab75b10 con 0x55ee3ac85cc0 2023-09-10T01:21:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b576fbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3ab75810 con 0x55ee3ac8b1a0 2023-09-10T01:21:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b576fbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3aa598c0 con 0x55ee3ac85300 2023-09-10T01:21:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b576fbb80 10 client.0 _ll_register_callbacks cb 0x55ee3aabfa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:21:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b48ad0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac8f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b492d1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85300 0x55ee3ac91e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b48ad0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac8f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58632/0 (socket says 172.21.15.44:58632) 2023-09-10T01:21:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b48ad0700 1 -- 172.21.15.44:0/3326328554 learned_addr learned my addr 172.21.15.44:0/3326328554 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b49ad2700 1 --2- 172.21.15.44:0/3326328554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac8f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b492d1700 1 -- 172.21.15.44:0/3326328554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 msgr2=0x55ee3ac8f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b492d1700 1 --2- 172.21.15.44:0/3326328554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac8f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b492d1700 1 -- 172.21.15.44:0/3326328554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85cc0 msgr2=0x55ee3ac8f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b492d1700 1 --2- 172.21.15.44:0/3326328554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac8f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b492d1700 1 -- 172.21.15.44:0/3326328554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4410dcd0 con 0x55ee3ac85300 2023-09-10T01:21:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b49ad2700 1 --2- 172.21.15.44:0/3326328554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac8f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:21:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b492d1700 1 --2- 172.21.15.44:0/3326328554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85300 0x55ee3ac91e60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009870 tx=0x7f5b380047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80758b6bc8549ee2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:21:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b2cff9700 1 -- 172.21.15.44:0/3326328554 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b38009eb0 con 0x55ee3ac85300 2023-09-10T01:21:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3326328554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee3ab2f490 con 0x55ee3ac85300 2023-09-10T01:21:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.165+0000 7f5b2cff9700 1 -- 172.21.15.44:0/3326328554 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f5b3800e810 con 0x55ee3ac85300 2023-09-10T01:21:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.166+0000 7f5b2cff9700 1 -- 172.21.15.44:0/3326328554 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b3800e9d0 con 0x55ee3ac85300 2023-09-10T01:21:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.166+0000 7f5b576fbb80 10 client.29515 fetch_fsmap learned FSMap version 132 2023-09-10T01:21:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.166+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3326328554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=4+}) v3 -- 0x55ee3a9ecc00 con 0x55ee3ac85300 2023-09-10T01:21:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.166+0000 7f5b2cff9700 1 -- 172.21.15.44:0/3326328554 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ee3ab2f490 con 0x55ee3ac85300 2023-09-10T01:21:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.166+0000 7f5b2cff9700 1 -- 172.21.15.44:0/3326328554 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 132) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f5b38028bf0 con 0x55ee3ac85300 2023-09-10T01:21:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.166+0000 7f5b576fbb80 10 client.29515 fetch_fsmap finished waiting for FSMap version 132 2023-09-10T01:21:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.166+0000 7f5b576fbb80 10 client.29515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:21:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.166+0000 7f5b576fbb80 20 client.29515 populate_metadata read hostname 'smithi044' 2023-09-10T01:21:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.166+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/3326328554 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ee3ac9f410 0x55ee3aca1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.166+0000 7f5b576fbb80 4 client.29515 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:21:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.166+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3326328554 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee3abec840 con 0x55ee3ac9f410 2023-09-10T01:21:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.166+0000 7f5b48ad0700 1 --2- 172.21.15.44:0/3326328554 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ee3ac9f410 0x55ee3aca1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.167+0000 7f5b2cff9700 10 client.29515 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.167+0000 7f5b48ad0700 1 --2- 172.21.15.44:0/3326328554 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ee3ac9f410 0x55ee3aca1830 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20353113877dce79 server_cookie=7fe7633960aae690 in_seq=0 out_seq=0 2023-09-10T01:21:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.167+0000 7f5b2cff9700 1 -- 172.21.15.44:0/3326328554 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ee3abec840 con 0x55ee3ac9f410 2023-09-10T01:21:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.167+0000 7f5b2cff9700 10 client.29515 handle_command_reply: tid=0 2023-09-10T01:21:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.167+0000 7f5b576fbb80 1 client.29515 shutdown 2023-09-10T01:21:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.167+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3326328554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85300 msgr2=0x55ee3ac91e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.167+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/3326328554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85300 0x55ee3ac91e60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009870 tx=0x7f5b380047c0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.168+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3326328554 shutdown_connections 2023-09-10T01:21:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.168+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/3326328554 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ee3ac9f410 0x55ee3aca1830 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.168+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/3326328554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac8f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.168+0000 7f5b2cff9700 0 client.29515 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.168+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/3326328554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85300 0x55ee3ac91e60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.168+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/3326328554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac8f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.168+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3326328554 >> 172.21.15.44:0/3326328554 conn(0x55ee3abfa030 msgr2=0x55ee3ac7f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.168+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3326328554 shutdown_connections 2023-09-10T01:21:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.168+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3326328554 wait complete. 2023-09-10T01:21:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.168+0000 7f5b576fbb80 20 client.29515 trim_cache size 0 max 16384 2023-09-10T01:21:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b576fbb80 1 Processor -- start 2023-09-10T01:21:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b576fbb80 1 -- start start 2023-09-10T01:21:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b576fbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85300 0x55ee3ac8f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b576fbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac96460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b576fbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac91990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b576fbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3ac89d00 con 0x55ee3ac85300 2023-09-10T01:21:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b576fbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3ac969a0 con 0x55ee3ac8b1a0 2023-09-10T01:21:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b576fbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3ac8fc80 con 0x55ee3ac85cc0 2023-09-10T01:21:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b492d1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85300 0x55ee3ac8f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b48ad0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac96460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b492d1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85300 0x55ee3ac8f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58638/0 (socket says 172.21.15.44:58638) 2023-09-10T01:21:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b492d1700 1 -- 172.21.15.44:0/3231733148 learned_addr learned my addr 172.21.15.44:0/3231733148 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b49ad2700 1 --2- 172.21.15.44:0/3231733148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac91990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b48ad0700 1 -- 172.21.15.44:0/3231733148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 msgr2=0x55ee3ac91990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b48ad0700 1 --2- 172.21.15.44:0/3231733148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac91990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b48ad0700 1 -- 172.21.15.44:0/3231733148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85300 msgr2=0x55ee3ac8f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b48ad0700 1 --2- 172.21.15.44:0/3231733148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85300 0x55ee3ac8f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b48ad0700 1 -- 172.21.15.44:0/3231733148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee3aa2a1f0 con 0x55ee3ac85cc0 2023-09-10T01:21:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b49ad2700 1 --2- 172.21.15.44:0/3231733148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac91990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:21:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.183+0000 7f5b492d1700 1 --2- 172.21.15.44:0/3231733148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85300 0x55ee3ac8f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:21:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.184+0000 7f5b48ad0700 1 --2- 172.21.15.44:0/3231733148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac96460 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009350 tx=0x7f5b4001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20e2294abc81b410 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.184+0000 7f5b2ffff700 1 -- 172.21.15.44:0/3231733148 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b400059c0 con 0x55ee3ac85cc0 2023-09-10T01:21:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.184+0000 7f5b2ffff700 1 -- 172.21.15.44:0/3231733148 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b40031150 con 0x55ee3ac85cc0 2023-09-10T01:21:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.184+0000 7f5b2ffff700 1 -- 172.21.15.44:0/3231733148 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40003760 con 0x55ee3ac85cc0 2023-09-10T01:21:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.184+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3231733148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85cc0 msgr2=0x55ee3ac96460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.184+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/3231733148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac96460 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009350 tx=0x7f5b4001a040 comp rx=0 tx=0).stop 2023-09-10T01:21:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.184+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3231733148 shutdown_connections 2023-09-10T01:21:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.184+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/3231733148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac91990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.184+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/3231733148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac85300 0x55ee3ac8f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.184+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/3231733148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac96460 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.184+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3231733148 >> 172.21.15.44:0/3231733148 conn(0x55ee3abfa030 msgr2=0x55ee3ac8e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.184+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3231733148 shutdown_connections 2023-09-10T01:21:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.184+0000 7f5b576fbb80 1 -- 172.21.15.44:0/3231733148 wait complete. 2023-09-10T01:21:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.185+0000 7f5b576fbb80 1 -- start start 2023-09-10T01:21:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.185+0000 7f5b576fbb80 1 Processor -- start 2023-09-10T01:21:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.185+0000 7f5b576fbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac85300 0x55ee3ac92440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.185+0000 7f5b576fbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac92980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.185+0000 7f5b576fbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac92ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.185+0000 7f5b576fbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3acf2310 con 0x55ee3ac8b1a0 2023-09-10T01:21:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.185+0000 7f5b576fbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3acf2450 con 0x55ee3ac85300 2023-09-10T01:21:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.185+0000 7f5b576fbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3acf2610 con 0x55ee3ac85cc0 2023-09-10T01:21:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.185+0000 7f5b576fbb80 10 client.0 _ll_register_callbacks cb 0x7f5b3801e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:21:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.185+0000 7f5b49ad2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac92ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.185+0000 7f5b49ad2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac92ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58648/0 (socket says 172.21.15.44:58648) 2023-09-10T01:21:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.185+0000 7f5b49ad2700 1 -- 172.21.15.44:0/2119256659 learned_addr learned my addr 172.21.15.44:0/2119256659 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.185+0000 7f5b48ad0700 1 --2- 172.21.15.44:0/2119256659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac92980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.186+0000 7f5b492d1700 1 --2- 172.21.15.44:0/2119256659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac85300 0x55ee3ac92440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.186+0000 7f5b49ad2700 1 -- 172.21.15.44:0/2119256659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85cc0 msgr2=0x55ee3ac92980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.186+0000 7f5b49ad2700 1 --2- 172.21.15.44:0/2119256659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac92980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.186+0000 7f5b49ad2700 1 -- 172.21.15.44:0/2119256659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac85300 msgr2=0x55ee3ac92440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.186+0000 7f5b49ad2700 1 --2- 172.21.15.44:0/2119256659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac85300 0x55ee3ac92440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.186+0000 7f5b49ad2700 1 -- 172.21.15.44:0/2119256659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee3a9ecc00 con 0x55ee3ac8b1a0 2023-09-10T01:21:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.186+0000 7f5b492d1700 1 --2- 172.21.15.44:0/2119256659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac85300 0x55ee3ac92440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:21:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.186+0000 7f5b49ad2700 1 --2- 172.21.15.44:0/2119256659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac92ec0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c0c94a0 tx=0x7f5b3c004fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa258af004f6cc19 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.186+0000 7f5b2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:21:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.186+0000 7f5b2cff9700 1 -- 172.21.15.44:0/2119256659 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c0b0070 con 0x55ee3ac8b1a0 2023-09-10T01:21:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.186+0000 7f5b2cff9700 1 -- 172.21.15.44:0/2119256659 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f5b3c00ddd0 con 0x55ee3ac8b1a0 2023-09-10T01:21:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.186+0000 7f5b576fbb80 1 -- 172.21.15.44:0/2119256659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b4001e040 con 0x55ee3ac8b1a0 2023-09-10T01:21:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.186+0000 7f5b2cff9700 1 -- 172.21.15.44:0/2119256659 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b3c099300 con 0x55ee3ac8b1a0 2023-09-10T01:21:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.186+0000 7f5b576fbb80 10 client.19920 fetch_fsmap learned FSMap version 132 2023-09-10T01:21:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.186+0000 7f5b576fbb80 1 -- 172.21.15.44:0/2119256659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=4+}) v3 -- 0x55ee3aa03d80 con 0x55ee3ac8b1a0 2023-09-10T01:21:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.186+0000 7f5b2cff9700 1 -- 172.21.15.44:0/2119256659 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b3c062070 con 0x55ee3ac8b1a0 2023-09-10T01:21:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.187+0000 7f5b2cff9700 1 -- 172.21.15.44:0/2119256659 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 132) v1 ==== 7331+0+0 (secure 0 0 0) 0x7f5b3c0980f0 con 0x55ee3ac8b1a0 2023-09-10T01:21:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.187+0000 7f5b576fbb80 10 client.19920 fetch_fsmap finished waiting for FSMap version 132 2023-09-10T01:21:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.187+0000 7f5b576fbb80 10 client.19920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:21:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.187+0000 7f5b576fbb80 20 client.19920 populate_metadata read hostname 'smithi044' 2023-09-10T01:21:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.187+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/2119256659 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ee3acf3cb0 0x55ee3acf60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.187+0000 7f5b576fbb80 4 client.19920 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:21:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.187+0000 7f5b576fbb80 1 -- 172.21.15.44:0/2119256659 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ee3abd08c0 con 0x55ee3acf3cb0 2023-09-10T01:21:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.187+0000 7f5b492d1700 1 --2- 172.21.15.44:0/2119256659 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ee3acf3cb0 0x55ee3acf60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.187+0000 7f5b2cff9700 10 client.19920 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.187+0000 7f5b492d1700 1 --2- 172.21.15.44:0/2119256659 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ee3acf3cb0 0x55ee3acf60d0 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f94c0c447a466b27 server_cookie=b25881cbd803ad19 in_seq=0 out_seq=0 2023-09-10T01:21:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.205+0000 7f5b2cff9700 1 -- 172.21.15.44:0/2119256659 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ee3abd08c0 con 0x55ee3acf3cb0 2023-09-10T01:21:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.206+0000 7f5b2cff9700 10 client.19920 handle_command_reply: tid=0 2023-09-10T01:21:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.206+0000 7f5b576fbb80 1 client.19920 shutdown 2023-09-10T01:21:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.206+0000 7f5b576fbb80 1 -- 172.21.15.44:0/2119256659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac8b1a0 msgr2=0x55ee3ac92ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.206+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/2119256659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac92ec0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c0c94a0 tx=0x7f5b3c004fc0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.206+0000 7f5b576fbb80 1 -- 172.21.15.44:0/2119256659 shutdown_connections 2023-09-10T01:21:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.206+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/2119256659 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55ee3acf3cb0 0x55ee3acf60d0 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.206+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/2119256659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee3ac8b1a0 0x55ee3ac92ec0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.206+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/2119256659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee3ac85300 0x55ee3ac92440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.206+0000 7f5b576fbb80 1 --2- 172.21.15.44:0/2119256659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee3ac85cc0 0x55ee3ac92980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.206+0000 7f5b2cff9700 0 client.19920 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.206+0000 7f5b576fbb80 1 -- 172.21.15.44:0/2119256659 >> 172.21.15.44:0/2119256659 conn(0x55ee3abfa030 msgr2=0x55ee3ac84b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.206+0000 7f5b576fbb80 1 -- 172.21.15.44:0/2119256659 shutdown_connections 2023-09-10T01:21:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.206+0000 7f5b576fbb80 1 -- 172.21.15.44:0/2119256659 wait complete. 2023-09-10T01:21:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.206+0000 7f5b576fbb80 20 client.19920 trim_cache size 0 max 16384 2023-09-10T01:21:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:21:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:21:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:21:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:21:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.207+0000 7f5b4b535700 1 -- 172.21.15.44:0/1875640861 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5b300a07b0 msgr2=0x7f5b300a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.207+0000 7f5b4b535700 1 --2- 172.21.15.44:0/1875640861 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5b300a07b0 0x7f5b300a2c70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009d70 tx=0x7f5b38009260 comp rx=0 tx=0).stop 2023-09-10T01:21:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.207+0000 7f5b4b535700 1 -- 172.21.15.44:0/1875640861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b44106650 msgr2=0x7f5b44196540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.207+0000 7f5b4b535700 1 --2- 172.21.15.44:0/1875640861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b44106650 0x7f5b44196540 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c007ee0 tx=0x7f5b3c00d3b0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.207+0000 7f5b4b535700 1 -- 172.21.15.44:0/1875640861 shutdown_connections 2023-09-10T01:21:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.207+0000 7f5b4b535700 1 --2- 172.21.15.44:0/1875640861 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5b300a07b0 0x7f5b300a2c70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.207+0000 7f5b4b535700 1 --2- 172.21.15.44:0/1875640861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b44106650 0x7f5b44196540 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.207+0000 7f5b4b535700 1 --2- 172.21.15.44:0/1875640861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b440679f0 0x7f5b4419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.207+0000 7f5b4b535700 1 --2- 172.21.15.44:0/1875640861 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b440683b0 0x7f5b4419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.207+0000 7f5b4b535700 1 -- 172.21.15.44:0/1875640861 >> 172.21.15.44:0/1875640861 conn(0x7f5b44074f30 msgr2=0x7f5b440fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.207+0000 7f5b4b535700 1 -- 172.21.15.44:0/1875640861 shutdown_connections 2023-09-10T01:21:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:02.207+0000 7f5b4b535700 1 -- 172.21.15.44:0/1875640861 wait complete. 2023-09-10T01:21:03.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:03 smithi131 ceph-mon[103713]: pgmap v1304: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 14 MiB/s wr, 98 op/s 2023-09-10T01:21:03.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:03 smithi044 ceph-mon[98676]: pgmap v1304: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 14 MiB/s wr, 98 op/s 2023-09-10T01:21:03.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:03 smithi099 ceph-mon[102692]: pgmap v1304: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 14 MiB/s wr, 98 op/s 2023-09-10T01:21:05.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:05 smithi131 ceph-mon[103713]: pgmap v1305: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 14 MiB/s wr, 98 op/s 2023-09-10T01:21:05.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:05 smithi044 ceph-mon[98676]: pgmap v1305: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 14 MiB/s wr, 98 op/s 2023-09-10T01:21:05.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:05 smithi099 ceph-mon[102692]: pgmap v1305: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 14 MiB/s wr, 98 op/s 2023-09-10T01:21:05.749 INFO:teuthology.orchestra.run.smithi044.stderr:done. 2023-09-10T01:21:05.773 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2023-09-10T01:21:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:starting vacuum...end. 2023-09-10T01:21:07.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:07 smithi131 ceph-mon[103713]: pgmap v1306: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 12 MiB/s wr, 97 op/s 2023-09-10T01:21:07.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:07 smithi044 ceph-mon[98676]: pgmap v1306: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 12 MiB/s wr, 97 op/s 2023-09-10T01:21:07.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:07 smithi099 ceph-mon[102692]: pgmap v1306: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 12 MiB/s wr, 97 op/s 2023-09-10T01:21:09.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:09 smithi131 ceph-mon[103713]: pgmap v1307: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 12 MiB/s wr, 97 op/s 2023-09-10T01:21:09.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:09 smithi044 ceph-mon[98676]: pgmap v1307: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 12 MiB/s wr, 97 op/s 2023-09-10T01:21:09.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:09 smithi099 ceph-mon[102692]: pgmap v1307: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 12 MiB/s wr, 97 op/s 2023-09-10T01:21:10.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:10 smithi131 ceph-mon[103713]: osdmap e170: 12 total, 12 up, 12 in 2023-09-10T01:21:10.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:10 smithi131 ceph-mon[103713]: pgmap v1309: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 11 MiB/s wr, 87 op/s 2023-09-10T01:21:10.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:10 smithi044 ceph-mon[98676]: osdmap e170: 12 total, 12 up, 12 in 2023-09-10T01:21:10.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:10 smithi044 ceph-mon[98676]: pgmap v1309: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 11 MiB/s wr, 87 op/s 2023-09-10T01:21:10.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:10 smithi099 ceph-mon[102692]: osdmap e170: 12 total, 12 up, 12 in 2023-09-10T01:21:10.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:10 smithi099 ceph-mon[102692]: pgmap v1309: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 11 MiB/s wr, 87 op/s 2023-09-10T01:21:11.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 5.0 s, 197.8 tps, lat 457.434 ms stddev 461.071 2023-09-10T01:21:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:13 smithi131 ceph-mon[103713]: pgmap v1310: 129 pgs: 14 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 79 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 7.7 MiB/s wr, 131 op/s 2023-09-10T01:21:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:13 smithi044 ceph-mon[98676]: pgmap v1310: 129 pgs: 14 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 79 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 7.7 MiB/s wr, 131 op/s 2023-09-10T01:21:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:13 smithi099 ceph-mon[102692]: pgmap v1310: 129 pgs: 14 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 79 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 7.7 MiB/s wr, 131 op/s 2023-09-10T01:21:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:15 smithi044 ceph-mon[98676]: pgmap v1311: 129 pgs: 14 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 79 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 7.7 MiB/s wr, 131 op/s 2023-09-10T01:21:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:15 smithi099 ceph-mon[102692]: pgmap v1311: 129 pgs: 14 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 79 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 7.7 MiB/s wr, 131 op/s 2023-09-10T01:21:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.366+0000 7f3ffe10f700 1 -- 172.21.15.44:0/478731931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff8100120 msgr2=0x7f3ff810c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.366+0000 7f3ffe10f700 1 --2- 172.21.15.44:0/478731931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff8100120 0x7f3ff810c330 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000b0d0 tx=0x7f3ff000b3e0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.366+0000 7f3ffe10f700 1 -- 172.21.15.44:0/478731931 shutdown_connections 2023-09-10T01:21:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.366+0000 7f3ffe10f700 1 --2- 172.21.15.44:0/478731931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff8100120 0x7f3ff810c330 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.366+0000 7f3ffe10f700 1 --2- 172.21.15.44:0/478731931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff81055b0 0x7f3ff8105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.366+0000 7f3ffe10f700 1 --2- 172.21.15.44:0/478731931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff80ff760 0x7f3ff80ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.366+0000 7f3ffe10f700 1 -- 172.21.15.44:0/478731931 >> 172.21.15.44:0/478731931 conn(0x7f3ff8074f30 msgr2=0x7f3ff8075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.366+0000 7f3ffe10f700 1 -- 172.21.15.44:0/478731931 shutdown_connections 2023-09-10T01:21:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.366+0000 7f3ffe10f700 1 -- 172.21.15.44:0/478731931 wait complete. 2023-09-10T01:21:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.367+0000 7f3ffe10f700 1 Processor -- start 2023-09-10T01:21:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.367+0000 7f3ffe10f700 1 -- start start 2023-09-10T01:21:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.367+0000 7f3ffe10f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80ff760 0x7f3ff8197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.367+0000 7f3ffe10f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff8100120 0x7f3ff8197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.367+0000 7f3ffe10f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff81055b0 0x7f3ff81a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.367+0000 7f3ffe10f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff81a2af0 con 0x7f3ff8100120 2023-09-10T01:21:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.367+0000 7f3ffe10f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff81a2c60 con 0x7f3ff81055b0 2023-09-10T01:21:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.368+0000 7f3ffe10f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff81a2e20 con 0x7f3ff80ff760 2023-09-10T01:21:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.368+0000 7f3ff6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff8100120 0x7f3ff8197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.368+0000 7f3ff77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80ff760 0x7f3ff8197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.368+0000 7f3ff6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff8100120 0x7f3ff8197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58658/0 (socket says 172.21.15.44:58658) 2023-09-10T01:21:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.368+0000 7f3ff77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80ff760 0x7f3ff8197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37308/0 (socket says 172.21.15.44:37308) 2023-09-10T01:21:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.368+0000 7f3ff6ffd700 1 -- 172.21.15.44:0/312321929 learned_addr learned my addr 172.21.15.44:0/312321929 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.368+0000 7f3ff7fff700 1 --2- 172.21.15.44:0/312321929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff81055b0 0x7f3ff81a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.368+0000 7f3ff77fe700 1 -- 172.21.15.44:0/312321929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff81055b0 msgr2=0x7f3ff81a2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.368+0000 7f3ff77fe700 1 --2- 172.21.15.44:0/312321929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff81055b0 0x7f3ff81a2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.368+0000 7f3ff77fe700 1 -- 172.21.15.44:0/312321929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff8100120 msgr2=0x7f3ff8197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.368+0000 7f3ff77fe700 1 --2- 172.21.15.44:0/312321929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff8100120 0x7f3ff8197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.368+0000 7f3ff77fe700 1 -- 172.21.15.44:0/312321929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff0009d00 con 0x7f3ff80ff760 2023-09-10T01:21:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.368+0000 7f3ff6ffd700 1 --2- 172.21.15.44:0/312321929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff8100120 0x7f3ff8197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:21:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.368+0000 7f3ff77fe700 1 --2- 172.21.15.44:0/312321929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80ff760 0x7f3ff8197680 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800e9f0 tx=0x7f3fe800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.369+0000 7f3ff4ff9700 1 -- 172.21.15.44:0/312321929 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe800c9c0 con 0x7f3ff80ff760 2023-09-10T01:21:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.369+0000 7f3ff4ff9700 1 -- 172.21.15.44:0/312321929 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fe8010430 con 0x7f3ff80ff760 2023-09-10T01:21:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.369+0000 7f3ffe10f700 1 -- 172.21.15.44:0/312321929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff81a3100 con 0x7f3ff80ff760 2023-09-10T01:21:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.369+0000 7f3ff4ff9700 1 -- 172.21.15.44:0/312321929 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8018680 con 0x7f3ff80ff760 2023-09-10T01:21:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.369+0000 7f3ffe10f700 1 -- 172.21.15.44:0/312321929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff81a3610 con 0x7f3ff80ff760 2023-09-10T01:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.372+0000 7f3ff4ff9700 1 -- 172.21.15.44:0/312321929 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f3fe80105a0 con 0x7f3ff80ff760 2023-09-10T01:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.372+0000 7f3ff4ff9700 1 --2- 172.21.15.44:0/312321929 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3fe00a07b0 0x7f3fe00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.373+0000 7f3ff4ff9700 1 -- 172.21.15.44:0/312321929 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f3fe8014070 con 0x7f3ff80ff760 2023-09-10T01:21:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.373+0000 7f400a2d5b80 1 Processor -- start 2023-09-10T01:21:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.373+0000 7f400a2d5b80 1 -- start start 2023-09-10T01:21:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.373+0000 7f400a2d5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288fc990 0x5652288fcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.373+0000 7f3ff6ffd700 1 --2- 172.21.15.44:0/312321929 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3fe00a07b0 0x7f3fe00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.373+0000 7f400a2d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288ef300 0x5652288ef780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.373+0000 7f400a2d5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288efcc0 0x5652289046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.373+0000 7f400a2d5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652288486d0 con 0x5652288efcc0 2023-09-10T01:21:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.373+0000 7f400a2d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x565228845ff0 con 0x5652288ef300 2023-09-10T01:21:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.373+0000 7f400a2d5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652288418e0 con 0x5652288fc990 2023-09-10T01:21:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.373+0000 7f3ff77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288fc990 0x5652288fcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.374+0000 7f3ff77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288fc990 0x5652288fcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37314/0 (socket says 172.21.15.44:37314) 2023-09-10T01:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.374+0000 7f3ff77fe700 1 -- 172.21.15.44:0/2802580723 learned_addr learned my addr 172.21.15.44:0/2802580723 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.374+0000 7f3ff6ffd700 1 --2- 172.21.15.44:0/2802580723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288ef300 0x5652288ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.374+0000 7f3ff7fff700 1 --2- 172.21.15.44:0/2802580723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288efcc0 0x5652289046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.374+0000 7f3ff77fe700 1 -- 172.21.15.44:0/2802580723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288ef300 msgr2=0x5652288ef780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.374+0000 7f3ff77fe700 1 --2- 172.21.15.44:0/2802580723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288ef300 0x5652288ef780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.374+0000 7f3ff77fe700 1 -- 172.21.15.44:0/2802580723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288efcc0 msgr2=0x5652289046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.374+0000 7f3ff77fe700 1 --2- 172.21.15.44:0/2802580723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288efcc0 0x5652289046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.374+0000 7f3ff77fe700 1 -- 172.21.15.44:0/2802580723 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56522866dd80 con 0x5652288fc990 2023-09-10T01:21:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.374+0000 7f3ff7fff700 1 --2- 172.21.15.44:0/2802580723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288efcc0 0x5652289046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:21:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.374+0000 7f3ff77fe700 1 --2- 172.21.15.44:0/2802580723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288fc990 0x5652288fcd70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8099290 tx=0x7f3fe8099150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dc8d8d7a305f91d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.374+0000 7f3ff6ffd700 1 --2- 172.21.15.44:0/312321929 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3fe00a07b0 0x7f3fe00a2c70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80684b0 tx=0x7f3fec011da0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.374+0000 7f3fddffb700 1 -- 172.21.15.44:0/2802580723 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8099c90 con 0x5652288fc990 2023-09-10T01:21:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.374+0000 7f3fddffb700 1 -- 172.21.15.44:0/2802580723 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fe8099590 con 0x5652288fc990 2023-09-10T01:21:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.374+0000 7f3fddffb700 1 -- 172.21.15.44:0/2802580723 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80cc1b0 con 0x5652288fc990 2023-09-10T01:21:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.375+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2802580723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288fc990 msgr2=0x5652288fcd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.375+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/2802580723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288fc990 0x5652288fcd70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8099290 tx=0x7f3fe8099150 comp rx=0 tx=0).stop 2023-09-10T01:21:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.375+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2802580723 shutdown_connections 2023-09-10T01:21:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.375+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/2802580723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288efcc0 0x5652289046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.375+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/2802580723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288fc990 0x5652288fcd70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.375+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/2802580723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288ef300 0x5652288ef780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.375+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2802580723 >> 172.21.15.44:0/2802580723 conn(0x565228864030 msgr2=0x5652288ea240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.375+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2802580723 shutdown_connections 2023-09-10T01:21:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.375+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2802580723 wait complete. 2023-09-10T01:21:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.376+0000 7f400a2d5b80 1 -- start start 2023-09-10T01:21:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.376+0000 7f400a2d5b80 1 Processor -- start 2023-09-10T01:21:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.376+0000 7f400a2d5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288ef300 0x565228907670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.376+0000 7f400a2d5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288efcc0 0x565228907bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.376+0000 7f400a2d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288fc990 0x5652289018c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.376+0000 7f400a2d5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652287dfb10 con 0x5652288ef300 2023-09-10T01:21:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.376+0000 7f400a2d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652287df810 con 0x5652288fc990 2023-09-10T01:21:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.376+0000 7f400a2d5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652286c38c0 con 0x5652288efcc0 2023-09-10T01:21:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.376+0000 7f400a2d5b80 10 client.0 _ll_register_callbacks cb 0x565228729a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:21:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.376+0000 7f3ff77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288ef300 0x565228907670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.376+0000 7f3ff6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288efcc0 0x565228907bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.376+0000 7f3ff6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288efcc0 0x565228907bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37322/0 (socket says 172.21.15.44:37322) 2023-09-10T01:21:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.376+0000 7f3ff6ffd700 1 -- 172.21.15.44:0/2437740957 learned_addr learned my addr 172.21.15.44:0/2437740957 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.376+0000 7f3ff7fff700 1 --2- 172.21.15.44:0/2437740957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288fc990 0x5652289018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.377+0000 7f3ff6ffd700 1 -- 172.21.15.44:0/2437740957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288fc990 msgr2=0x5652289018c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.377+0000 7f3ff6ffd700 1 --2- 172.21.15.44:0/2437740957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288fc990 0x5652289018c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.377+0000 7f3ff6ffd700 1 -- 172.21.15.44:0/2437740957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288ef300 msgr2=0x565228907670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.377+0000 7f3ff6ffd700 1 --2- 172.21.15.44:0/2437740957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288ef300 0x565228907670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.377+0000 7f3ff6ffd700 1 -- 172.21.15.44:0/2437740957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff81a3610 con 0x5652288efcc0 2023-09-10T01:21:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.377+0000 7f3ff77fe700 1 --2- 172.21.15.44:0/2437740957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288ef300 0x565228907670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:21:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.377+0000 7f3ff7fff700 1 --2- 172.21.15.44:0/2437740957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288fc990 0x5652289018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:21:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.377+0000 7f3ff6ffd700 1 --2- 172.21.15.44:0/2437740957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288efcc0 0x565228907bb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00fa20 tx=0x7f3fec0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98067abd42b0dac3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.377+0000 7f3fd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:21:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.377+0000 7f3fd6ffd700 1 -- 172.21.15.44:0/2437740957 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec004a30 con 0x5652288efcc0 2023-09-10T01:21:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.377+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2437740957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565228799490 con 0x5652288efcc0 2023-09-10T01:21:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.377+0000 7f3fd6ffd700 1 -- 172.21.15.44:0/2437740957 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(170..170 src has 1..170) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f3fec004b90 con 0x5652288efcc0 2023-09-10T01:21:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.377+0000 7f3fd6ffd700 1 -- 172.21.15.44:0/2437740957 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=133) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fec004d50 con 0x5652288efcc0 2023-09-10T01:21:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.377+0000 7f400a2d5b80 10 client.29533 fetch_fsmap learned FSMap version 133 2023-09-10T01:21:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.377+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2437740957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=133,monmap=4+}) v3 -- 0x565228656c00 con 0x5652288efcc0 2023-09-10T01:21:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.378+0000 7f3fd6ffd700 1 -- 172.21.15.44:0/2437740957 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x565228799490 con 0x5652288efcc0 2023-09-10T01:21:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.378+0000 7f3fd6ffd700 1 -- 172.21.15.44:0/2437740957 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 133) v1 ==== 7335+0+0 (secure 0 0 0) 0x7f3fec0052b0 con 0x5652288efcc0 2023-09-10T01:21:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.378+0000 7f400a2d5b80 10 client.29533 fetch_fsmap finished waiting for FSMap version 133 2023-09-10T01:21:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.378+0000 7f400a2d5b80 10 client.29533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:21:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.378+0000 7f400a2d5b80 20 client.29533 populate_metadata read hostname 'smithi044' 2023-09-10T01:21:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.378+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/2437740957 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x565228911ae0 0x565228913f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.378+0000 7f400a2d5b80 4 client.29533 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:21:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.378+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2437740957 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565228856840 con 0x565228911ae0 2023-09-10T01:21:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.378+0000 7f3ff77fe700 1 --2- 172.21.15.44:0/2437740957 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x565228911ae0 0x565228913f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.379+0000 7f3fd6ffd700 10 client.29533 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.379+0000 7f3ff77fe700 1 --2- 172.21.15.44:0/2437740957 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x565228911ae0 0x565228913f00 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6f54821c6d2b6a1 server_cookie=76b016086a2e79f2 in_seq=0 out_seq=0 2023-09-10T01:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.379+0000 7f3fd6ffd700 1 -- 172.21.15.44:0/2437740957 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x565228856840 con 0x565228911ae0 2023-09-10T01:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.379+0000 7f3fd6ffd700 10 client.29533 handle_command_reply: tid=0 2023-09-10T01:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.379+0000 7f400a2d5b80 1 client.29533 shutdown 2023-09-10T01:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.380+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2437740957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288efcc0 msgr2=0x565228907bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.380+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/2437740957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288efcc0 0x565228907bb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00fa20 tx=0x7f3fec0047c0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.380+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2437740957 shutdown_connections 2023-09-10T01:21:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.380+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/2437740957 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x565228911ae0 0x565228913f00 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.380+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/2437740957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288fc990 0x5652289018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.380+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/2437740957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288ef300 0x565228907670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.380+0000 7f3fd6ffd700 0 client.29533 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.380+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/2437740957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288efcc0 0x565228907bb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.380+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2437740957 >> 172.21.15.44:0/2437740957 conn(0x565228864030 msgr2=0x5652288e97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.380+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2437740957 shutdown_connections 2023-09-10T01:21:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.380+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2437740957 wait complete. 2023-09-10T01:21:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.380+0000 7f400a2d5b80 20 client.29533 trim_cache size 0 max 16384 2023-09-10T01:21:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.402+0000 7f400a2d5b80 1 Processor -- start 2023-09-10T01:21:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.402+0000 7f400a2d5b80 1 -- start start 2023-09-10T01:21:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.403+0000 7f400a2d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288ef300 0x5652288eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.403+0000 7f400a2d5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288efcc0 0x565228907200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.403+0000 7f400a2d5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288fc990 0x565228907740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.403+0000 7f400a2d5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652288fb870 con 0x5652288efcc0 2023-09-10T01:21:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.403+0000 7f400a2d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x565228902ea0 con 0x5652288ef300 2023-09-10T01:21:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.403+0000 7f400a2d5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652288fd5a0 con 0x5652288fc990 2023-09-10T01:21:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.403+0000 7f3ff77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288ef300 0x5652288eeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.403+0000 7f3ff77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288ef300 0x5652288eeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60418/0 (socket says 172.21.15.44:60418) 2023-09-10T01:21:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.403+0000 7f3ff77fe700 1 -- 172.21.15.44:0/2094221067 learned_addr learned my addr 172.21.15.44:0/2094221067 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.403+0000 7f3ff6ffd700 1 --2- 172.21.15.44:0/2094221067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288efcc0 0x565228907200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.403+0000 7f3ff7fff700 1 --2- 172.21.15.44:0/2094221067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288fc990 0x565228907740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.404+0000 7f3ff6ffd700 1 -- 172.21.15.44:0/2094221067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288fc990 msgr2=0x565228907740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.404+0000 7f3ff6ffd700 1 --2- 172.21.15.44:0/2094221067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288fc990 0x565228907740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.404+0000 7f3ff6ffd700 1 -- 172.21.15.44:0/2094221067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288ef300 msgr2=0x5652288eeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.404+0000 7f3ff6ffd700 1 --2- 172.21.15.44:0/2094221067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288ef300 0x5652288eeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.404+0000 7f3ff6ffd700 1 -- 172.21.15.44:0/2094221067 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652286941f0 con 0x5652288efcc0 2023-09-10T01:21:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.404+0000 7f3ff6ffd700 1 --2- 172.21.15.44:0/2094221067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288efcc0 0x565228907200 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f3fec005690 tx=0x7f3fec011840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e767cfc18355d1ef server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.405+0000 7f3fde7fc700 1 -- 172.21.15.44:0/2094221067 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec00e890 con 0x5652288efcc0 2023-09-10T01:21:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.405+0000 7f3fde7fc700 1 -- 172.21.15.44:0/2094221067 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fec00ee80 con 0x5652288efcc0 2023-09-10T01:21:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.405+0000 7f3fde7fc700 1 -- 172.21.15.44:0/2094221067 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec036c40 con 0x5652288efcc0 2023-09-10T01:21:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.406+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2094221067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288efcc0 msgr2=0x565228907200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.406+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/2094221067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288efcc0 0x565228907200 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f3fec005690 tx=0x7f3fec011840 comp rx=0 tx=0).stop 2023-09-10T01:21:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.406+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2094221067 shutdown_connections 2023-09-10T01:21:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.406+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/2094221067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288fc990 0x565228907740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.406+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/2094221067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288ef300 0x5652288eeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.406+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/2094221067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288efcc0 0x565228907200 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.406+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2094221067 >> 172.21.15.44:0/2094221067 conn(0x565228864030 msgr2=0x565228902250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.406+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2094221067 shutdown_connections 2023-09-10T01:21:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.406+0000 7f400a2d5b80 1 -- 172.21.15.44:0/2094221067 wait complete. 2023-09-10T01:21:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.406+0000 7f400a2d5b80 1 -- start start 2023-09-10T01:21:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.406+0000 7f400a2d5b80 1 Processor -- start 2023-09-10T01:21:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f400a2d5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288ef300 0x5652288f0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f400a2d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288efcc0 0x5652288f0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f400a2d5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288fc990 0x5652288f0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f400a2d5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652288f4730 con 0x5652288fc990 2023-09-10T01:21:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f400a2d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652288f4870 con 0x5652288efcc0 2023-09-10T01:21:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f400a2d5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652288f4a30 con 0x5652288ef300 2023-09-10T01:21:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f400a2d5b80 10 client.0 _ll_register_callbacks cb 0x7f3fec011750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:21:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f3ff77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288ef300 0x5652288f0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f3ff6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288efcc0 0x5652288f0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f3ff7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288fc990 0x5652288f0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f3ff6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288efcc0 0x5652288f0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60426/0 (socket says 172.21.15.44:60426) 2023-09-10T01:21:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f3ff77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288ef300 0x5652288f0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37334/0 (socket says 172.21.15.44:37334) 2023-09-10T01:21:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f3ff6ffd700 1 -- 172.21.15.44:0/1150011291 learned_addr learned my addr 172.21.15.44:0/1150011291 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f3ff6ffd700 1 -- 172.21.15.44:0/1150011291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288ef300 msgr2=0x5652288f0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f3ff6ffd700 1 --2- 172.21.15.44:0/1150011291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288ef300 0x5652288f0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f3ff6ffd700 1 -- 172.21.15.44:0/1150011291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288fc990 msgr2=0x5652288f0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f3ff6ffd700 1 --2- 172.21.15.44:0/1150011291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288fc990 0x5652288f0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.407+0000 7f3ff6ffd700 1 -- 172.21.15.44:0/1150011291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565228656c00 con 0x5652288efcc0 2023-09-10T01:21:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.408+0000 7f3ff6ffd700 1 --2- 172.21.15.44:0/1150011291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288efcc0 0x5652288f0a80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00f9f0 tx=0x7f3fec036660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=455ee34d9bbfaaa8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.408+0000 7f3fd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:21:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.408+0000 7f3fd6ffd700 1 -- 172.21.15.44:0/1150011291 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec036cd0 con 0x5652288efcc0 2023-09-10T01:21:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.408+0000 7f400a2d5b80 1 -- 172.21.15.44:0/1150011291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56522883a8c0 con 0x5652288efcc0 2023-09-10T01:21:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.408+0000 7f3fd6ffd700 1 -- 172.21.15.44:0/1150011291 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(170..170 src has 1..170) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f3fec036e30 con 0x5652288efcc0 2023-09-10T01:21:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.408+0000 7f3fd6ffd700 1 -- 172.21.15.44:0/1150011291 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=133) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fec00e690 con 0x5652288efcc0 2023-09-10T01:21:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.408+0000 7f400a2d5b80 10 client.29558 fetch_fsmap learned FSMap version 133 2023-09-10T01:21:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.408+0000 7f400a2d5b80 1 -- 172.21.15.44:0/1150011291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=133,monmap=4+}) v3 -- 0x56522866c250 con 0x5652288efcc0 2023-09-10T01:21:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.409+0000 7f3fd6ffd700 1 -- 172.21.15.44:0/1150011291 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56522883a8c0 con 0x5652288efcc0 2023-09-10T01:21:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.409+0000 7f3fd6ffd700 1 -- 172.21.15.44:0/1150011291 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 133) v1 ==== 7335+0+0 (secure 0 0 0) 0x7f3fec03d070 con 0x5652288efcc0 2023-09-10T01:21:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.409+0000 7f400a2d5b80 10 client.29558 fetch_fsmap finished waiting for FSMap version 133 2023-09-10T01:21:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.409+0000 7f400a2d5b80 10 client.29558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:21:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.409+0000 7f400a2d5b80 20 client.29558 populate_metadata read hostname 'smithi044' 2023-09-10T01:21:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.409+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/1150011291 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56522895de80 0x5652289602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.409+0000 7f400a2d5b80 4 client.29558 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:21:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.409+0000 7f400a2d5b80 1 -- 172.21.15.44:0/1150011291 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5652287fb050 con 0x56522895de80 2023-09-10T01:21:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.409+0000 7f3ff77fe700 1 --2- 172.21.15.44:0/1150011291 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56522895de80 0x5652289602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.410+0000 7f3fd6ffd700 10 client.29558 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.410+0000 7f3ff77fe700 1 --2- 172.21.15.44:0/1150011291 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56522895de80 0x5652289602a0 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94921da4b76a64a3 server_cookie=659d73ddf1cfc8a6 in_seq=0 out_seq=0 2023-09-10T01:21:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.431+0000 7f3fd6ffd700 1 -- 172.21.15.44:0/1150011291 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5652287fb050 con 0x56522895de80 2023-09-10T01:21:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.431+0000 7f3fd6ffd700 10 client.29558 handle_command_reply: tid=0 2023-09-10T01:21:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.431+0000 7f400a2d5b80 1 client.29558 shutdown 2023-09-10T01:21:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.431+0000 7f400a2d5b80 1 -- 172.21.15.44:0/1150011291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288efcc0 msgr2=0x5652288f0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.431+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/1150011291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288efcc0 0x5652288f0a80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00f9f0 tx=0x7f3fec036660 comp rx=0 tx=0).stop 2023-09-10T01:21:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.431+0000 7f400a2d5b80 1 -- 172.21.15.44:0/1150011291 shutdown_connections 2023-09-10T01:21:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.431+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/1150011291 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56522895de80 0x5652289602a0 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.431+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/1150011291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652288fc990 0x5652288f0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.431+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/1150011291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5652288ef300 0x5652288f0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.431+0000 7f3fd6ffd700 0 client.29558 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.431+0000 7f400a2d5b80 1 --2- 172.21.15.44:0/1150011291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652288efcc0 0x5652288f0a80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.431+0000 7f400a2d5b80 1 -- 172.21.15.44:0/1150011291 >> 172.21.15.44:0/1150011291 conn(0x565228864030 msgr2=0x5652288eeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.431+0000 7f400a2d5b80 1 -- 172.21.15.44:0/1150011291 shutdown_connections 2023-09-10T01:21:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.431+0000 7f400a2d5b80 1 -- 172.21.15.44:0/1150011291 wait complete. 2023-09-10T01:21:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.431+0000 7f400a2d5b80 20 client.29558 trim_cache size 0 max 16384 2023-09-10T01:21:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:21:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:21:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:21:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:21:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.432+0000 7f3ffe10f700 1 -- 172.21.15.44:0/312321929 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3fe00a07b0 msgr2=0x7f3fe00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.432+0000 7f3ffe10f700 1 --2- 172.21.15.44:0/312321929 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3fe00a07b0 0x7f3fe00a2c70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80684b0 tx=0x7f3fec011da0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.432+0000 7f3ffe10f700 1 -- 172.21.15.44:0/312321929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80ff760 msgr2=0x7f3ff8197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.432+0000 7f3ffe10f700 1 --2- 172.21.15.44:0/312321929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80ff760 0x7f3ff8197680 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800e9f0 tx=0x7f3fe800ed00 comp rx=0 tx=0).stop 2023-09-10T01:21:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.432+0000 7f3ffe10f700 1 -- 172.21.15.44:0/312321929 shutdown_connections 2023-09-10T01:21:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.432+0000 7f3ffe10f700 1 --2- 172.21.15.44:0/312321929 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3fe00a07b0 0x7f3fe00a2c70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.432+0000 7f3ffe10f700 1 --2- 172.21.15.44:0/312321929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff81055b0 0x7f3ff81a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.432+0000 7f3ffe10f700 1 --2- 172.21.15.44:0/312321929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80ff760 0x7f3ff8197680 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.432+0000 7f3ffe10f700 1 --2- 172.21.15.44:0/312321929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff8100120 0x7f3ff8197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.432+0000 7f3ffe10f700 1 -- 172.21.15.44:0/312321929 >> 172.21.15.44:0/312321929 conn(0x7f3ff8074f30 msgr2=0x7f3ff80fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.432+0000 7f3ffe10f700 1 -- 172.21.15.44:0/312321929 shutdown_connections 2023-09-10T01:21:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:15.432+0000 7f3ffe10f700 1 -- 172.21.15.44:0/312321929 wait complete. 2023-09-10T01:21:15.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:15 smithi131 ceph-mon[103713]: pgmap v1311: 129 pgs: 14 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 79 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 7.7 MiB/s wr, 131 op/s 2023-09-10T01:21:16.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 10.0 s, 839.6 tps, lat 118.852 ms stddev 34.527 2023-09-10T01:21:16.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:16 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:21:16.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:16 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:21:16.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:16 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:21:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:17 smithi099 ceph-mon[102692]: pgmap v1312: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.0 MiB/s wr, 210 op/s 2023-09-10T01:21:17.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:17 smithi131 ceph-mon[103713]: pgmap v1312: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.0 MiB/s wr, 210 op/s 2023-09-10T01:21:17.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:17 smithi044 ceph-mon[98676]: pgmap v1312: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.0 MiB/s wr, 210 op/s 2023-09-10T01:21:18.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:18 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:21:18.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:18 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:21:18.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:18 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:21:19.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:19 smithi099 ceph-mon[102692]: pgmap v1313: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.0 MiB/s wr, 210 op/s 2023-09-10T01:21:19.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:19 smithi131 ceph-mon[103713]: pgmap v1313: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.0 MiB/s wr, 210 op/s 2023-09-10T01:21:19.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:19 smithi044 ceph-mon[98676]: pgmap v1313: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.0 MiB/s wr, 210 op/s 2023-09-10T01:21:21.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 15.0 s, 603.0 tps, lat 152.868 ms stddev 140.078 2023-09-10T01:21:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:21 smithi099 ceph-mon[102692]: pgmap v1314: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 7.6 MiB/s wr, 200 op/s 2023-09-10T01:21:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:21 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:21:21.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:21 smithi099 ceph-mon[102692]: osdmap e171: 12 total, 12 up, 12 in 2023-09-10T01:21:21.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:21 smithi131 ceph-mon[103713]: pgmap v1314: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 7.6 MiB/s wr, 200 op/s 2023-09-10T01:21:21.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:21 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:21:21.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:21 smithi131 ceph-mon[103713]: osdmap e171: 12 total, 12 up, 12 in 2023-09-10T01:21:21.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:21 smithi044 ceph-mon[98676]: pgmap v1314: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 7.6 MiB/s wr, 200 op/s 2023-09-10T01:21:21.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:21 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:21:21.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:21 smithi044 ceph-mon[98676]: osdmap e171: 12 total, 12 up, 12 in 2023-09-10T01:21:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:23 smithi099 ceph-mon[102692]: pgmap v1316: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 274 op/s 2023-09-10T01:21:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:23 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:21:23.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:23 smithi131 ceph-mon[103713]: pgmap v1316: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 274 op/s 2023-09-10T01:21:23.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:23 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:21:23.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:23 smithi044 ceph-mon[98676]: pgmap v1316: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 274 op/s 2023-09-10T01:21:23.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:23 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:21:25.550 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:25 smithi131 ceph-mon[103713]: pgmap v1317: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 274 op/s 2023-09-10T01:21:25.560 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:25 smithi044 ceph-mon[98676]: pgmap v1317: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 274 op/s 2023-09-10T01:21:25.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:25 smithi099 ceph-mon[102692]: pgmap v1317: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 274 op/s 2023-09-10T01:21:26.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 20.0 s, 171.8 tps, lat 577.884 ms stddev 514.853 2023-09-10T01:21:27.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:27 smithi131 ceph-mon[103713]: pgmap v1318: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 8.6 MiB/s wr, 396 op/s 2023-09-10T01:21:27.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:27 smithi044 ceph-mon[98676]: pgmap v1318: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 8.6 MiB/s wr, 396 op/s 2023-09-10T01:21:27.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:27 smithi099 ceph-mon[102692]: pgmap v1318: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 8.6 MiB/s wr, 396 op/s 2023-09-10T01:21:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.616+0000 7f23e743e700 1 -- 172.21.15.44:0/2478264442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e010cc80 msgr2=0x7f23e010d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.616+0000 7f23e743e700 1 --2- 172.21.15.44:0/2478264442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e010cc80 0x7f23e010d060 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009990 tx=0x7f23d0009ca0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.618+0000 7f23e743e700 1 -- 172.21.15.44:0/2478264442 shutdown_connections 2023-09-10T01:21:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.618+0000 7f23e743e700 1 --2- 172.21.15.44:0/2478264442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e01076e0 0x7f23e0113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.618+0000 7f23e743e700 1 --2- 172.21.15.44:0/2478264442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e010cc80 0x7f23e010d060 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.618+0000 7f23e743e700 1 --2- 172.21.15.44:0/2478264442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23e0106d20 0x7f23e01071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.618+0000 7f23e743e700 1 -- 172.21.15.44:0/2478264442 >> 172.21.15.44:0/2478264442 conn(0x7f23e00764a0 msgr2=0x7f23e0105420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.618+0000 7f23e743e700 1 -- 172.21.15.44:0/2478264442 shutdown_connections 2023-09-10T01:21:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.618+0000 7f23e743e700 1 -- 172.21.15.44:0/2478264442 wait complete. 2023-09-10T01:21:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.619+0000 7f23e743e700 1 Processor -- start 2023-09-10T01:21:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.619+0000 7f23e743e700 1 -- start start 2023-09-10T01:21:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.619+0000 7f23e743e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e0106d20 0x7f23e0071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.619+0000 7f23e743e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e01076e0 0x7f23e0072150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.619+0000 7f23e743e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23e010cc80 0x7f23e006aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.619+0000 7f23e743e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e006b240 con 0x7f23e010cc80 2023-09-10T01:21:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.619+0000 7f23e743e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e006b3b0 con 0x7f23e01076e0 2023-09-10T01:21:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.619+0000 7f23e743e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e006b570 con 0x7f23e0106d20 2023-09-10T01:21:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.619+0000 7f23e49d9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e01076e0 0x7f23e0072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.620+0000 7f23e49d9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e01076e0 0x7f23e0072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60438/0 (socket says 172.21.15.44:60438) 2023-09-10T01:21:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.620+0000 7f23e49d9700 1 -- 172.21.15.44:0/1965137100 learned_addr learned my addr 172.21.15.44:0/1965137100 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.620+0000 7f23e51da700 1 --2- 172.21.15.44:0/1965137100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e0106d20 0x7f23e0071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.620+0000 7f23e49d9700 1 -- 172.21.15.44:0/1965137100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e0106d20 msgr2=0x7f23e0071bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.620+0000 7f23e49d9700 1 --2- 172.21.15.44:0/1965137100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e0106d20 0x7f23e0071bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.620+0000 7f23e49d9700 1 -- 172.21.15.44:0/1965137100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23e010cc80 msgr2=0x7f23e006aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.620+0000 7f23e49d9700 1 --2- 172.21.15.44:0/1965137100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23e010cc80 0x7f23e006aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.620+0000 7f23e49d9700 1 -- 172.21.15.44:0/1965137100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d00095f0 con 0x7f23e01076e0 2023-09-10T01:21:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.620+0000 7f23e49d9700 1 --2- 172.21.15.44:0/1965137100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e01076e0 0x7f23e0072150 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00b530 tx=0x7f23dc00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.621+0000 7f23d67fc700 1 -- 172.21.15.44:0/1965137100 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc011830 con 0x7f23e01076e0 2023-09-10T01:21:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.621+0000 7f23d67fc700 1 -- 172.21.15.44:0/1965137100 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23dc011e20 con 0x7f23e01076e0 2023-09-10T01:21:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.621+0000 7f23d67fc700 1 -- 172.21.15.44:0/1965137100 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc00f2e0 con 0x7f23e01076e0 2023-09-10T01:21:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.621+0000 7f23e743e700 1 -- 172.21.15.44:0/1965137100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e006b850 con 0x7f23e01076e0 2023-09-10T01:21:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.621+0000 7f23e743e700 1 -- 172.21.15.44:0/1965137100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e006fa90 con 0x7f23e01076e0 2023-09-10T01:21:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.632+0000 7f23f3604b80 1 Processor -- start 2023-09-10T01:21:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.635+0000 7f23d67fc700 1 -- 172.21.15.44:0/1965137100 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f23dc011990 con 0x7f23e01076e0 2023-09-10T01:21:28.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.640+0000 7f23d67fc700 1 --2- 172.21.15.44:0/1965137100 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f23cc0a08b0 0x7f23cc0a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.641+0000 7f23d67fc700 1 -- 172.21.15.44:0/1965137100 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f23dc0cbd50 con 0x7f23e01076e0 2023-09-10T01:21:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.646+0000 7f23f3604b80 1 -- start start 2023-09-10T01:21:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.646+0000 7f23f3604b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 0x55cfacde7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.646+0000 7f23f3604b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde1300 0x55cfacde1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.646+0000 7f23f3604b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1cc0 0x55cfacdedf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.646+0000 7f23f3604b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfacd3a6d0 con 0x55cfacde71a0 2023-09-10T01:21:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.646+0000 7f23f3604b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfacd37ff0 con 0x55cfacde1cc0 2023-09-10T01:21:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.646+0000 7f23f3604b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfacd338e0 con 0x55cfacde1300 2023-09-10T01:21:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.646+0000 7f23e59db700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1cc0 0x55cfacdedf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.646+0000 7f23e59db700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1cc0 0x55cfacdedf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60444/0 (socket says 172.21.15.44:60444) 2023-09-10T01:21:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.646+0000 7f23e59db700 1 -- 172.21.15.44:0/1695474675 learned_addr learned my addr 172.21.15.44:0/1695474675 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.647+0000 7f23e59db700 1 -- 172.21.15.44:0/1695474675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde1300 msgr2=0x55cfacde1780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:21:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.647+0000 7f23e59db700 1 --2- 172.21.15.44:0/1695474675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde1300 0x55cfacde1780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.647+0000 7f23e59db700 1 -- 172.21.15.44:0/1695474675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 msgr2=0x55cfacde7580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:21:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.647+0000 7f23e59db700 1 --2- 172.21.15.44:0/1695474675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 0x55cfacde7580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.647+0000 7f23e59db700 1 -- 172.21.15.44:0/1695474675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfacb5fd80 con 0x55cfacde1cc0 2023-09-10T01:21:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.647+0000 7f23e59db700 1 --2- 172.21.15.44:0/1695474675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1cc0 0x55cfacdedf30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009cf0 tx=0x7f23d800e130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6501650e691f0fe5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.647+0000 7f23cb7fe700 1 -- 172.21.15.44:0/1695474675 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800f040 con 0x55cfacde1cc0 2023-09-10T01:21:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.647+0000 7f23cb7fe700 1 -- 172.21.15.44:0/1695474675 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23d800ecf0 con 0x55cfacde1cc0 2023-09-10T01:21:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.647+0000 7f23cb7fe700 1 -- 172.21.15.44:0/1695474675 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8006330 con 0x55cfacde1cc0 2023-09-10T01:21:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.647+0000 7f23e59db700 1 --2- 172.21.15.44:0/1965137100 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f23cc0a08b0 0x7f23cc0a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.648+0000 7f23e59db700 1 --2- 172.21.15.44:0/1965137100 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f23cc0a08b0 0x7f23cc0a2d70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f23d8016000 tx=0x7f23d8014f50 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:28.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.654+0000 7f23f3604b80 1 -- 172.21.15.44:0/1695474675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1cc0 msgr2=0x55cfacdedf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.654+0000 7f23f3604b80 1 --2- 172.21.15.44:0/1695474675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1cc0 0x55cfacdedf30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009cf0 tx=0x7f23d800e130 comp rx=0 tx=0).stop 2023-09-10T01:21:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.657+0000 7f23f3604b80 1 -- 172.21.15.44:0/1695474675 shutdown_connections 2023-09-10T01:21:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.657+0000 7f23f3604b80 1 --2- 172.21.15.44:0/1695474675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1cc0 0x55cfacdedf30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.657+0000 7f23f3604b80 1 --2- 172.21.15.44:0/1695474675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 0x55cfacde7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.657+0000 7f23f3604b80 1 --2- 172.21.15.44:0/1695474675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde1300 0x55cfacde1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.657+0000 7f23f3604b80 1 -- 172.21.15.44:0/1695474675 >> 172.21.15.44:0/1695474675 conn(0x55cfacd56030 msgr2=0x55cfacddc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.658+0000 7f23f3604b80 1 -- 172.21.15.44:0/1695474675 shutdown_connections 2023-09-10T01:21:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.658+0000 7f23f3604b80 1 -- 172.21.15.44:0/1695474675 wait complete. 2023-09-10T01:21:28.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.658+0000 7f23f3604b80 1 -- start start 2023-09-10T01:21:28.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.658+0000 7f23f3604b80 1 Processor -- start 2023-09-10T01:21:28.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.659+0000 7f23f3604b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde1300 0x55cfacdede60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.659+0000 7f23f3604b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1cc0 0x55cfacdeb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.659+0000 7f23f3604b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde71a0 0x55cfacdeb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.659+0000 7f23f3604b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfaccd1b10 con 0x55cfacde1300 2023-09-10T01:21:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.659+0000 7f23f3604b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfaccd1810 con 0x55cfacde1cc0 2023-09-10T01:21:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.659+0000 7f23f3604b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfacbb58c0 con 0x55cfacde71a0 2023-09-10T01:21:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.659+0000 7f23f3604b80 10 client.0 _ll_register_callbacks cb 0x55cfacc1ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:21:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.659+0000 7f23e51da700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde1300 0x55cfacdede60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.659+0000 7f23e51da700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde1300 0x55cfacdede60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58706/0 (socket says 172.21.15.44:58706) 2023-09-10T01:21:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.659+0000 7f23e51da700 1 -- 172.21.15.44:0/3838964169 learned_addr learned my addr 172.21.15.44:0/3838964169 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.660+0000 7f23e51da700 1 -- 172.21.15.44:0/3838964169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde71a0 msgr2=0x55cfacdeb680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:21:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.660+0000 7f23e51da700 1 --2- 172.21.15.44:0/3838964169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde71a0 0x55cfacdeb680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.660+0000 7f23e51da700 1 -- 172.21.15.44:0/3838964169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1cc0 msgr2=0x55cfacdeb070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:21:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.660+0000 7f23e51da700 1 --2- 172.21.15.44:0/3838964169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1cc0 0x55cfacdeb070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.660+0000 7f23e51da700 1 -- 172.21.15.44:0/3838964169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d8009970 con 0x55cfacde1300 2023-09-10T01:21:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.660+0000 7f23e51da700 1 --2- 172.21.15.44:0/3838964169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde1300 0x55cfacdede60 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f23d0015000 tx=0x7f23d000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697892d4affee35b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.660+0000 7f23c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:21:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.660+0000 7f23c8ff9700 1 -- 172.21.15.44:0/3838964169 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d001d070 con 0x55cfacde1300 2023-09-10T01:21:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.660+0000 7f23c8ff9700 1 -- 172.21.15.44:0/3838964169 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f23d0005550 con 0x55cfacde1300 2023-09-10T01:21:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.660+0000 7f23f3604b80 1 -- 172.21.15.44:0/3838964169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cfacc8b490 con 0x55cfacde1300 2023-09-10T01:21:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.660+0000 7f23c8ff9700 1 -- 172.21.15.44:0/3838964169 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23d0005710 con 0x55cfacde1300 2023-09-10T01:21:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.660+0000 7f23f3604b80 10 client.19938 fetch_fsmap learned FSMap version 136 2023-09-10T01:21:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.660+0000 7f23f3604b80 1 -- 172.21.15.44:0/3838964169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=4+}) v3 -- 0x55cfacb48c00 con 0x55cfacde1300 2023-09-10T01:21:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.661+0000 7f23c8ff9700 1 -- 172.21.15.44:0/3838964169 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cfacc8b490 con 0x55cfacde1300 2023-09-10T01:21:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.661+0000 7f23c8ff9700 1 -- 172.21.15.44:0/3838964169 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 136) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f23d0003a50 con 0x55cfacde1300 2023-09-10T01:21:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.661+0000 7f23f3604b80 10 client.19938 fetch_fsmap finished waiting for FSMap version 136 2023-09-10T01:21:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.661+0000 7f23f3604b80 10 client.19938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:21:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.661+0000 7f23f3604b80 20 client.19938 populate_metadata read hostname 'smithi044' 2023-09-10T01:21:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.661+0000 7f23f3604b80 1 --2- 172.21.15.44:0/3838964169 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cfacdfb560 0x55cfacdfda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.661+0000 7f23f3604b80 4 client.19938 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:21:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.661+0000 7f23f3604b80 1 -- 172.21.15.44:0/3838964169 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfacd48840 con 0x55cfacdfb560 2023-09-10T01:21:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.662+0000 7f23e49d9700 1 --2- 172.21.15.44:0/3838964169 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cfacdfb560 0x55cfacdfda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.668+0000 7f23c8ff9700 10 client.19938 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.668+0000 7f23e49d9700 1 --2- 172.21.15.44:0/3838964169 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cfacdfb560 0x55cfacdfda20 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bfc170556a83f83 server_cookie=3e95768712dc5d2a in_seq=0 out_seq=0 2023-09-10T01:21:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.672+0000 7f23c8ff9700 1 -- 172.21.15.44:0/3838964169 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cfacd48840 con 0x55cfacdfb560 2023-09-10T01:21:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.672+0000 7f23c8ff9700 10 client.19938 handle_command_reply: tid=0 2023-09-10T01:21:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.672+0000 7f23f3604b80 1 client.19938 shutdown 2023-09-10T01:21:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.676+0000 7f23f3604b80 1 -- 172.21.15.44:0/3838964169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde1300 msgr2=0x55cfacdede60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.676+0000 7f23f3604b80 1 --2- 172.21.15.44:0/3838964169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde1300 0x55cfacdede60 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f23d0015000 tx=0x7f23d000bf90 comp rx=0 tx=0).stop 2023-09-10T01:21:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.678+0000 7f23f3604b80 1 -- 172.21.15.44:0/3838964169 shutdown_connections 2023-09-10T01:21:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.678+0000 7f23f3604b80 1 --2- 172.21.15.44:0/3838964169 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cfacdfb560 0x55cfacdfda20 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.678+0000 7f23f3604b80 1 --2- 172.21.15.44:0/3838964169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde71a0 0x55cfacdeb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.678+0000 7f23f3604b80 1 --2- 172.21.15.44:0/3838964169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde1300 0x55cfacdede60 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.678+0000 7f23f3604b80 1 --2- 172.21.15.44:0/3838964169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1cc0 0x55cfacdeb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.678+0000 7f23f3604b80 1 -- 172.21.15.44:0/3838964169 >> 172.21.15.44:0/3838964169 conn(0x55cfacd56030 msgr2=0x55cfacddb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.678+0000 7f23c8ff9700 0 client.19938 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.681+0000 7f23f3604b80 1 -- 172.21.15.44:0/3838964169 shutdown_connections 2023-09-10T01:21:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.681+0000 7f23f3604b80 1 -- 172.21.15.44:0/3838964169 wait complete. 2023-09-10T01:21:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.682+0000 7f23f3604b80 20 client.19938 trim_cache size 0 max 16384 2023-09-10T01:21:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.713+0000 7f23f3604b80 1 Processor -- start 2023-09-10T01:21:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.713+0000 7f23f3604b80 1 -- start start 2023-09-10T01:21:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.713+0000 7f23f3604b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde1300 0x55cfacde69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.713+0000 7f23f3604b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1cc0 0x55cfacdf2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.713+0000 7f23f3604b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 0x55cfacde0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.713+0000 7f23f3604b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfacdd8fc0 con 0x55cfacde71a0 2023-09-10T01:21:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.713+0000 7f23f3604b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfacdebc80 con 0x55cfacde1cc0 2023-09-10T01:21:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.713+0000 7f23f3604b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfacddd930 con 0x55cfacde1300 2023-09-10T01:21:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.713+0000 7f23e59db700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 0x55cfacde0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.713+0000 7f23e59db700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 0x55cfacde0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58716/0 (socket says 172.21.15.44:58716) 2023-09-10T01:21:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.713+0000 7f23e59db700 1 -- 172.21.15.44:0/964707801 learned_addr learned my addr 172.21.15.44:0/964707801 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.713+0000 7f23e49d9700 1 --2- 172.21.15.44:0/964707801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1cc0 0x55cfacdf2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.714+0000 7f23e59db700 1 -- 172.21.15.44:0/964707801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde1300 msgr2=0x55cfacde69e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:21:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.714+0000 7f23e59db700 1 --2- 172.21.15.44:0/964707801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde1300 0x55cfacde69e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.714+0000 7f23e59db700 1 -- 172.21.15.44:0/964707801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1cc0 msgr2=0x55cfacdf2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.714+0000 7f23e59db700 1 --2- 172.21.15.44:0/964707801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1cc0 0x55cfacdf2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.714+0000 7f23e59db700 1 -- 172.21.15.44:0/964707801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfacb861f0 con 0x55cfacde71a0 2023-09-10T01:21:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.714+0000 7f23e59db700 1 --2- 172.21.15.44:0/964707801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 0x55cfacde0a00 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f23d8000f80 tx=0x7f23d8023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb440722715384c0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.714+0000 7f23cbfff700 1 -- 172.21.15.44:0/964707801 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800e040 con 0x55cfacde71a0 2023-09-10T01:21:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.714+0000 7f23cbfff700 1 -- 172.21.15.44:0/964707801 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23d80069e0 con 0x55cfacde71a0 2023-09-10T01:21:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.714+0000 7f23cbfff700 1 -- 172.21.15.44:0/964707801 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8006330 con 0x55cfacde71a0 2023-09-10T01:21:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.715+0000 7f23f3604b80 1 -- 172.21.15.44:0/964707801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 msgr2=0x55cfacde0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.715+0000 7f23f3604b80 1 --2- 172.21.15.44:0/964707801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 0x55cfacde0a00 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f23d8000f80 tx=0x7f23d8023000 comp rx=0 tx=0).stop 2023-09-10T01:21:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.715+0000 7f23f3604b80 1 -- 172.21.15.44:0/964707801 shutdown_connections 2023-09-10T01:21:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.715+0000 7f23f3604b80 1 --2- 172.21.15.44:0/964707801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 0x55cfacde0a00 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.715+0000 7f23f3604b80 1 --2- 172.21.15.44:0/964707801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde1300 0x55cfacde69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.715+0000 7f23f3604b80 1 --2- 172.21.15.44:0/964707801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1cc0 0x55cfacdf2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.715+0000 7f23f3604b80 1 -- 172.21.15.44:0/964707801 >> 172.21.15.44:0/964707801 conn(0x55cfacd56030 msgr2=0x55cfacde02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.716+0000 7f23f3604b80 1 -- 172.21.15.44:0/964707801 shutdown_connections 2023-09-10T01:21:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.716+0000 7f23f3604b80 1 -- 172.21.15.44:0/964707801 wait complete. 2023-09-10T01:21:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.716+0000 7f23f3604b80 1 -- start start 2023-09-10T01:21:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.716+0000 7f23f3604b80 1 Processor -- start 2023-09-10T01:21:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.727+0000 7f23f3604b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1300 0x55cface435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.727+0000 7f23f3604b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde1cc0 0x55cface43b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.727+0000 7f23f3604b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 0x55cface48450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.727+0000 7f23f3604b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cface48990 con 0x55cfacde71a0 2023-09-10T01:21:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.727+0000 7f23f3604b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cface48b00 con 0x55cfacde1300 2023-09-10T01:21:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.727+0000 7f23f3604b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cface48cc0 con 0x55cfacde1cc0 2023-09-10T01:21:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.727+0000 7f23f3604b80 10 client.0 _ll_register_callbacks cb 0x7f23d000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:21:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.727+0000 7f23e51da700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1300 0x55cface435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.727+0000 7f23e59db700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 0x55cface48450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.727+0000 7f23e51da700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1300 0x55cface435f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60458/0 (socket says 172.21.15.44:60458) 2023-09-10T01:21:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.727+0000 7f23e59db700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 0x55cface48450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58722/0 (socket says 172.21.15.44:58722) 2023-09-10T01:21:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.727+0000 7f23e51da700 1 -- 172.21.15.44:0/1093992048 learned_addr learned my addr 172.21.15.44:0/1093992048 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.727+0000 7f23e49d9700 1 --2- 172.21.15.44:0/1093992048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde1cc0 0x55cface43b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.728+0000 7f23e51da700 1 -- 172.21.15.44:0/1093992048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde1cc0 msgr2=0x55cface43b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.728+0000 7f23e51da700 1 --2- 172.21.15.44:0/1093992048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde1cc0 0x55cface43b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.728+0000 7f23e51da700 1 -- 172.21.15.44:0/1093992048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 msgr2=0x55cface48450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.728+0000 7f23e51da700 1 --2- 172.21.15.44:0/1093992048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 0x55cface48450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.728+0000 7f23e51da700 1 -- 172.21.15.44:0/1093992048 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfacb5fd80 con 0x55cfacde1300 2023-09-10T01:21:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.728+0000 7f23e51da700 1 --2- 172.21.15.44:0/1093992048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1300 0x55cface435f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009ce0 tx=0x7f23d0003b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df5fd3fd1b317565 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.728+0000 7f23c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:21:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.728+0000 7f23c8ff9700 1 -- 172.21.15.44:0/1093992048 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d001d070 con 0x55cfacde1300 2023-09-10T01:21:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.728+0000 7f23c8ff9700 1 -- 172.21.15.44:0/1093992048 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f23d0000e40 con 0x55cfacde1300 2023-09-10T01:21:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.728+0000 7f23f3604b80 1 -- 172.21.15.44:0/1093992048 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23d800ecf0 con 0x55cfacde1300 2023-09-10T01:21:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.729+0000 7f23c8ff9700 1 -- 172.21.15.44:0/1093992048 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23d0009290 con 0x55cfacde1300 2023-09-10T01:21:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.729+0000 7f23f3604b80 10 client.29570 fetch_fsmap learned FSMap version 136 2023-09-10T01:21:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.729+0000 7f23f3604b80 1 -- 172.21.15.44:0/1093992048 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=4+}) v3 -- 0x55cfacb5e250 con 0x55cfacde1300 2023-09-10T01:21:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.729+0000 7f23c8ff9700 1 -- 172.21.15.44:0/1093992048 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23d800ecf0 con 0x55cfacde1300 2023-09-10T01:21:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.729+0000 7f23c8ff9700 1 -- 172.21.15.44:0/1093992048 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 136) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f23d003a070 con 0x55cfacde1300 2023-09-10T01:21:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.729+0000 7f23f3604b80 10 client.29570 fetch_fsmap finished waiting for FSMap version 136 2023-09-10T01:21:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.729+0000 7f23f3604b80 10 client.29570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:21:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.729+0000 7f23f3604b80 20 client.29570 populate_metadata read hostname 'smithi044' 2023-09-10T01:21:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.729+0000 7f23f3604b80 1 --2- 172.21.15.44:0/1093992048 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cface50090 0x55cface524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.729+0000 7f23f3604b80 4 client.29570 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:21:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.729+0000 7f23f3604b80 1 -- 172.21.15.44:0/1093992048 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cfacd2c8c0 con 0x55cface50090 2023-09-10T01:21:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.730+0000 7f23e49d9700 1 --2- 172.21.15.44:0/1093992048 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cface50090 0x55cface524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.730+0000 7f23c8ff9700 10 client.29570 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.730+0000 7f23e49d9700 1 --2- 172.21.15.44:0/1093992048 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cface50090 0x55cface524b0 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd96d0ac7a646d9e server_cookie=d33296c6efcad3a6 in_seq=0 out_seq=0 2023-09-10T01:21:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.745+0000 7f23c8ff9700 1 -- 172.21.15.44:0/1093992048 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cfacd2c8c0 con 0x55cface50090 2023-09-10T01:21:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.745+0000 7f23c8ff9700 10 client.29570 handle_command_reply: tid=0 2023-09-10T01:21:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.745+0000 7f23f3604b80 1 client.29570 shutdown 2023-09-10T01:21:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.746+0000 7f23f3604b80 1 -- 172.21.15.44:0/1093992048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1300 msgr2=0x55cface435f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.746+0000 7f23f3604b80 1 --2- 172.21.15.44:0/1093992048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1300 0x55cface435f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009ce0 tx=0x7f23d0003b00 comp rx=0 tx=0).stop 2023-09-10T01:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.746+0000 7f23f3604b80 1 -- 172.21.15.44:0/1093992048 shutdown_connections 2023-09-10T01:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.746+0000 7f23f3604b80 1 --2- 172.21.15.44:0/1093992048 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cface50090 0x55cface524b0 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.746+0000 7f23f3604b80 1 --2- 172.21.15.44:0/1093992048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfacde71a0 0x55cface48450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.746+0000 7f23f3604b80 1 --2- 172.21.15.44:0/1093992048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfacde1300 0x55cface435f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.746+0000 7f23f3604b80 1 --2- 172.21.15.44:0/1093992048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfacde1cc0 0x55cface43b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.746+0000 7f23f3604b80 1 -- 172.21.15.44:0/1093992048 >> 172.21.15.44:0/1093992048 conn(0x55cfacd56030 msgr2=0x55cfacde6850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.746+0000 7f23c8ff9700 0 client.29570 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.746+0000 7f23f3604b80 1 -- 172.21.15.44:0/1093992048 shutdown_connections 2023-09-10T01:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.746+0000 7f23f3604b80 1 -- 172.21.15.44:0/1093992048 wait complete. 2023-09-10T01:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.746+0000 7f23f3604b80 20 client.29570 trim_cache size 0 max 16384 2023-09-10T01:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:21:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.747+0000 7f23e743e700 1 -- 172.21.15.44:0/1965137100 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f23cc0a08b0 msgr2=0x7f23cc0a2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.747+0000 7f23e743e700 1 --2- 172.21.15.44:0/1965137100 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f23cc0a08b0 0x7f23cc0a2d70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f23d8016000 tx=0x7f23d8014f50 comp rx=0 tx=0).stop 2023-09-10T01:21:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.747+0000 7f23e743e700 1 -- 172.21.15.44:0/1965137100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e01076e0 msgr2=0x7f23e0072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.747+0000 7f23e743e700 1 --2- 172.21.15.44:0/1965137100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e01076e0 0x7f23e0072150 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00b530 tx=0x7f23dc00b840 comp rx=0 tx=0).stop 2023-09-10T01:21:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.754+0000 7f23e743e700 1 -- 172.21.15.44:0/1965137100 shutdown_connections 2023-09-10T01:21:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.754+0000 7f23e743e700 1 --2- 172.21.15.44:0/1965137100 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f23cc0a08b0 0x7f23cc0a2d70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.754+0000 7f23e743e700 1 --2- 172.21.15.44:0/1965137100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23e010cc80 0x7f23e006aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.754+0000 7f23e743e700 1 --2- 172.21.15.44:0/1965137100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e0106d20 0x7f23e0071bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.754+0000 7f23e743e700 1 --2- 172.21.15.44:0/1965137100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e01076e0 0x7f23e0072150 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.754+0000 7f23e743e700 1 -- 172.21.15.44:0/1965137100 >> 172.21.15.44:0/1965137100 conn(0x7f23e00764a0 msgr2=0x7f23e00773f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.754+0000 7f23e743e700 1 -- 172.21.15.44:0/1965137100 shutdown_connections 2023-09-10T01:21:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:28.754+0000 7f23e743e700 1 -- 172.21.15.44:0/1965137100 wait complete. 2023-09-10T01:21:29.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:29 smithi131 ceph-mon[103713]: pgmap v1319: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 8.6 MiB/s wr, 396 op/s 2023-09-10T01:21:29.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:29 smithi044 ceph-mon[98676]: pgmap v1319: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 8.6 MiB/s wr, 396 op/s 2023-09-10T01:21:29.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:29 smithi099 ceph-mon[102692]: pgmap v1319: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 8.6 MiB/s wr, 396 op/s 2023-09-10T01:21:30.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:30 smithi131 ceph-mon[103713]: pgmap v1320: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 8.6 MiB/s wr, 396 op/s 2023-09-10T01:21:30.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:30 smithi044 ceph-mon[98676]: pgmap v1320: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 8.6 MiB/s wr, 396 op/s 2023-09-10T01:21:30.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:30 smithi099 ceph-mon[102692]: pgmap v1320: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 8.6 MiB/s wr, 396 op/s 2023-09-10T01:21:31.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 25.0 s, 186.6 tps, lat 528.522 ms stddev 522.254 2023-09-10T01:21:33.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:33 smithi099 ceph-mon[102692]: pgmap v1321: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 5.8 MiB/s wr, 489 op/s 2023-09-10T01:21:33.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:33 smithi131 ceph-mon[103713]: pgmap v1321: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 5.8 MiB/s wr, 489 op/s 2023-09-10T01:21:33.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:33 smithi044 ceph-mon[98676]: pgmap v1321: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 5.8 MiB/s wr, 489 op/s 2023-09-10T01:21:35.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:35 smithi099 ceph-mon[102692]: pgmap v1322: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 5.1 MiB/s wr, 433 op/s 2023-09-10T01:21:35.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:35 smithi131 ceph-mon[103713]: pgmap v1322: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 5.1 MiB/s wr, 433 op/s 2023-09-10T01:21:35.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:35 smithi044 ceph-mon[98676]: pgmap v1322: 129 pgs: 129 active+clean; 16 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 5.1 MiB/s wr, 433 op/s 2023-09-10T01:21:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 30.0 s, 215.2 tps, lat 499.930 ms stddev 473.633 2023-09-10T01:21:37.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:37 smithi099 ceph-mon[102692]: pgmap v1323: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 7.9 MiB/s wr, 670 op/s 2023-09-10T01:21:37.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:37 smithi131 ceph-mon[103713]: pgmap v1323: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 7.9 MiB/s wr, 670 op/s 2023-09-10T01:21:37.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:37 smithi044 ceph-mon[98676]: pgmap v1323: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 7.9 MiB/s wr, 670 op/s 2023-09-10T01:21:39.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:39 smithi099 ceph-mon[102692]: pgmap v1324: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 5.4 MiB/s wr, 461 op/s 2023-09-10T01:21:39.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:39 smithi131 ceph-mon[103713]: pgmap v1324: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 5.4 MiB/s wr, 461 op/s 2023-09-10T01:21:39.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:39 smithi044 ceph-mon[98676]: pgmap v1324: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 5.4 MiB/s wr, 461 op/s 2023-09-10T01:21:41.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 35.0 s, 473.8 tps, lat 213.779 ms stddev 100.347 2023-09-10T01:21:41.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:41 smithi131 ceph-mon[103713]: pgmap v1325: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 5.4 MiB/s wr, 461 op/s 2023-09-10T01:21:41.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:41 smithi044 ceph-mon[98676]: pgmap v1325: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 5.4 MiB/s wr, 461 op/s 2023-09-10T01:21:41.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:41 smithi099 ceph-mon[102692]: pgmap v1325: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 5.4 MiB/s wr, 461 op/s 2023-09-10T01:21:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.945+0000 7f0face09700 1 -- 172.21.15.44:0/1919662756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa8104e00 msgr2=0x7f0fa81051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.945+0000 7f0face09700 1 --2- 172.21.15.44:0/1919662756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa8104e00 0x7f0fa81051e0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009990 tx=0x7f0f98009ca0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.945+0000 7f0face09700 1 -- 172.21.15.44:0/1919662756 shutdown_connections 2023-09-10T01:21:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.945+0000 7f0face09700 1 --2- 172.21.15.44:0/1919662756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fa80ff7b0 0x7f0fa810bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.945+0000 7f0face09700 1 --2- 172.21.15.44:0/1919662756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa8104e00 0x7f0fa81051e0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.945+0000 7f0face09700 1 --2- 172.21.15.44:0/1919662756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fa80fee10 0x7f0fa80ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.945+0000 7f0face09700 1 -- 172.21.15.44:0/1919662756 >> 172.21.15.44:0/1919662756 conn(0x7f0fa8075920 msgr2=0x7f0fa8075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.945+0000 7f0face09700 1 -- 172.21.15.44:0/1919662756 shutdown_connections 2023-09-10T01:21:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.945+0000 7f0face09700 1 -- 172.21.15.44:0/1919662756 wait complete. 2023-09-10T01:21:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.946+0000 7f0face09700 1 Processor -- start 2023-09-10T01:21:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.946+0000 7f0face09700 1 -- start start 2023-09-10T01:21:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.946+0000 7f0face09700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fa80fee10 0x7f0fa81943a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.946+0000 7f0face09700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa80ff7b0 0x7f0fa81948e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.946+0000 7f0face09700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fa8104e00 0x7f0fa8194e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.946+0000 7f0face09700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa81a9260 con 0x7f0fa80ff7b0 2023-09-10T01:21:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.946+0000 7f0face09700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa819a980 con 0x7f0fa8104e00 2023-09-10T01:21:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.946+0000 7f0face09700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa819ab40 con 0x7f0fa80fee10 2023-09-10T01:21:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.946+0000 7f0fa5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa80ff7b0 0x7f0fa81948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.946+0000 7f0fa659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fa80fee10 0x7f0fa81943a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.946+0000 7f0fa659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fa80fee10 0x7f0fa81943a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37382/0 (socket says 172.21.15.44:37382) 2023-09-10T01:21:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.946+0000 7f0fa5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa80ff7b0 0x7f0fa81948e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58734/0 (socket says 172.21.15.44:58734) 2023-09-10T01:21:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.946+0000 7f0fa659c700 1 -- 172.21.15.44:0/3042364714 learned_addr learned my addr 172.21.15.44:0/3042364714 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.946+0000 7f0fa6d9d700 1 --2- 172.21.15.44:0/3042364714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fa8104e00 0x7f0fa8194e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.946+0000 7f0fa659c700 1 -- 172.21.15.44:0/3042364714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fa8104e00 msgr2=0x7f0fa8194e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.946+0000 7f0fa659c700 1 --2- 172.21.15.44:0/3042364714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fa8104e00 0x7f0fa8194e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.947+0000 7f0fa659c700 1 -- 172.21.15.44:0/3042364714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa80ff7b0 msgr2=0x7f0fa81948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.947+0000 7f0fa659c700 1 --2- 172.21.15.44:0/3042364714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa80ff7b0 0x7f0fa81948e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.947+0000 7f0fa659c700 1 -- 172.21.15.44:0/3042364714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f980095f0 con 0x7f0fa80fee10 2023-09-10T01:21:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.947+0000 7f0fa6d9d700 1 --2- 172.21.15.44:0/3042364714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fa8104e00 0x7f0fa8194e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:21:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.947+0000 7f0fa5d9b700 1 --2- 172.21.15.44:0/3042364714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa80ff7b0 0x7f0fa81948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:21:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.947+0000 7f0fa659c700 1 --2- 172.21.15.44:0/3042364714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fa80fee10 0x7f0fa81943a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0f98000c00 tx=0x7f0f98004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.947+0000 7f0f977fe700 1 -- 172.21.15.44:0/3042364714 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9801d070 con 0x7f0fa80fee10 2023-09-10T01:21:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.947+0000 7f0f977fe700 1 -- 172.21.15.44:0/3042364714 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f98022470 con 0x7f0fa80fee10 2023-09-10T01:21:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.947+0000 7f0face09700 1 -- 172.21.15.44:0/3042364714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa819adc0 con 0x7f0fa80fee10 2023-09-10T01:21:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.947+0000 7f0f977fe700 1 -- 172.21.15.44:0/3042364714 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98017650 con 0x7f0fa80fee10 2023-09-10T01:21:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.947+0000 7f0face09700 1 -- 172.21.15.44:0/3042364714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa819b270 con 0x7f0fa80fee10 2023-09-10T01:21:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.950+0000 7f0f977fe700 1 -- 172.21.15.44:0/3042364714 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f0f980225e0 con 0x7f0fa80fee10 2023-09-10T01:21:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.951+0000 7f0f977fe700 1 --2- 172.21.15.44:0/3042364714 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f900a07b0 0x7f0f900a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.951+0000 7f0f977fe700 1 -- 172.21.15.44:0/3042364714 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f0f980cd990 con 0x7f0fa80fee10 2023-09-10T01:21:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.951+0000 7f0fa5d9b700 1 --2- 172.21.15.44:0/3042364714 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f900a07b0 0x7f0f900a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.952+0000 7f0fb8fcfb80 1 Processor -- start 2023-09-10T01:21:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.952+0000 7f0fb8fcfb80 1 -- start start 2023-09-10T01:21:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.952+0000 7f0fa5d9b700 1 --2- 172.21.15.44:0/3042364714 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f900a07b0 0x7f0f900a2c70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8195da0 tx=0x7f0f9c010040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.952+0000 7f0fb8fcfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f531a0 0x55a380f53580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.952+0000 7f0fb8fcfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f4d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.952+0000 7f0fb8fcfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f4dcc0 0x55a380f59f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.952+0000 7f0fb8fcfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a380ea66d0 con 0x55a380f531a0 2023-09-10T01:21:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.952+0000 7f0fb8fcfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a380ea3ff0 con 0x55a380f4dcc0 2023-09-10T01:21:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.952+0000 7f0fb8fcfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a380e9f8e0 con 0x55a380f4d300 2023-09-10T01:21:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.952+0000 7f0fa5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f4d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.952+0000 7f0fa659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f531a0 0x55a380f53580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.952+0000 7f0fa659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f531a0 0x55a380f53580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58740/0 (socket says 172.21.15.44:58740) 2023-09-10T01:21:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.952+0000 7f0fa659c700 1 -- 172.21.15.44:0/1989132102 learned_addr learned my addr 172.21.15.44:0/1989132102 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.952+0000 7f0fa5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f4d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37392/0 (socket says 172.21.15.44:37392) 2023-09-10T01:21:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.952+0000 7f0fa6d9d700 1 --2- 172.21.15.44:0/1989132102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f4dcc0 0x55a380f59f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.953+0000 7f0fa5d9b700 1 -- 172.21.15.44:0/1989132102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f4dcc0 msgr2=0x55a380f59f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.953+0000 7f0fa5d9b700 1 --2- 172.21.15.44:0/1989132102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f4dcc0 0x55a380f59f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.953+0000 7f0fa5d9b700 1 -- 172.21.15.44:0/1989132102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f531a0 msgr2=0x55a380f53580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.953+0000 7f0fa5d9b700 1 --2- 172.21.15.44:0/1989132102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f531a0 0x55a380f53580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.953+0000 7f0fa5d9b700 1 -- 172.21.15.44:0/1989132102 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a380ccbd80 con 0x55a380f4d300 2023-09-10T01:21:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.953+0000 7f0fa6d9d700 1 --2- 172.21.15.44:0/1989132102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f4dcc0 0x55a380f59f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:21:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.953+0000 7f0fa659c700 1 --2- 172.21.15.44:0/1989132102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f531a0 0x55a380f53580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:21:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.953+0000 7f0fa5d9b700 1 --2- 172.21.15.44:0/1989132102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f4d780 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00af80 tx=0x7f0f9c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6324eb280d031f81 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.953+0000 7f0f94ff9700 1 -- 172.21.15.44:0/1989132102 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c013040 con 0x55a380f4d300 2023-09-10T01:21:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.953+0000 7f0f94ff9700 1 -- 172.21.15.44:0/1989132102 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f9c01b470 con 0x55a380f4d300 2023-09-10T01:21:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.953+0000 7f0f94ff9700 1 -- 172.21.15.44:0/1989132102 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c01c660 con 0x55a380f4d300 2023-09-10T01:21:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.954+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/1989132102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 msgr2=0x55a380f4d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.954+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/1989132102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f4d780 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00af80 tx=0x7f0f9c014000 comp rx=0 tx=0).stop 2023-09-10T01:21:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.954+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/1989132102 shutdown_connections 2023-09-10T01:21:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.954+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/1989132102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f4dcc0 0x55a380f59f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.954+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/1989132102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f531a0 0x55a380f53580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.954+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/1989132102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f4d780 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.954+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/1989132102 >> 172.21.15.44:0/1989132102 conn(0x55a380ec2030 msgr2=0x55a380f48560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.954+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/1989132102 shutdown_connections 2023-09-10T01:21:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.954+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/1989132102 wait complete. 2023-09-10T01:21:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.954+0000 7f0fb8fcfb80 1 -- start start 2023-09-10T01:21:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fb8fcfb80 1 Processor -- start 2023-09-10T01:21:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fb8fcfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f59d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fb8fcfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f4dcc0 0x55a380f57070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fb8fcfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f531a0 0x55a380f57610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fb8fcfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a380e3db10 con 0x55a380f4dcc0 2023-09-10T01:21:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fb8fcfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a380e3d810 con 0x55a380f531a0 2023-09-10T01:21:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fb8fcfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a380d218c0 con 0x55a380f4d300 2023-09-10T01:21:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fb8fcfb80 10 client.0 _ll_register_callbacks cb 0x55a380d87a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:21:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fa5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f4dcc0 0x55a380f57070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fa659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f59d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fa5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f4dcc0 0x55a380f57070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58748/0 (socket says 172.21.15.44:58748) 2023-09-10T01:21:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fa5d9b700 1 -- 172.21.15.44:0/2666571657 learned_addr learned my addr 172.21.15.44:0/2666571657 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fa6d9d700 1 --2- 172.21.15.44:0/2666571657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f531a0 0x55a380f57610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fa5d9b700 1 -- 172.21.15.44:0/2666571657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 msgr2=0x55a380f59d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fa5d9b700 1 --2- 172.21.15.44:0/2666571657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f59d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fa5d9b700 1 -- 172.21.15.44:0/2666571657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f531a0 msgr2=0x55a380f57610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fa5d9b700 1 --2- 172.21.15.44:0/2666571657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f531a0 0x55a380f57610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fa5d9b700 1 -- 172.21.15.44:0/2666571657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa0003a60 con 0x55a380f4dcc0 2023-09-10T01:21:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fa659c700 1 --2- 172.21.15.44:0/2666571657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f59d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:21:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.955+0000 7f0fa6d9d700 1 --2- 172.21.15.44:0/2666571657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f531a0 0x55a380f57610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:21:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.956+0000 7f0fa5d9b700 1 --2- 172.21.15.44:0/2666571657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f4dcc0 0x55a380f57070 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00edb0 tx=0x7f0f9c01bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23754edcc74448e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.956+0000 7f0f8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:21:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.956+0000 7f0f8dffb700 1 -- 172.21.15.44:0/2666571657 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c02b070 con 0x55a380f4dcc0 2023-09-10T01:21:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.956+0000 7f0f8dffb700 1 -- 172.21.15.44:0/2666571657 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f0f9c003b10 con 0x55a380f4dcc0 2023-09-10T01:21:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.956+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/2666571657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a380df7490 con 0x55a380f4dcc0 2023-09-10T01:21:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.956+0000 7f0f8dffb700 1 -- 172.21.15.44:0/2666571657 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f9c013040 con 0x55a380f4dcc0 2023-09-10T01:21:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.956+0000 7f0fb8fcfb80 10 client.19956 fetch_fsmap learned FSMap version 136 2023-09-10T01:21:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.956+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/2666571657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=4+}) v3 -- 0x7f0fa819b270 con 0x55a380f4dcc0 2023-09-10T01:21:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.956+0000 7f0f8dffb700 1 -- 172.21.15.44:0/2666571657 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a380df7490 con 0x55a380f4dcc0 2023-09-10T01:21:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.956+0000 7f0f8dffb700 1 -- 172.21.15.44:0/2666571657 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 136) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f0f9c004810 con 0x55a380f4dcc0 2023-09-10T01:21:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.956+0000 7f0fb8fcfb80 10 client.19956 fetch_fsmap finished waiting for FSMap version 136 2023-09-10T01:21:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.956+0000 7f0fb8fcfb80 10 client.19956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:21:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.956+0000 7f0fb8fcfb80 20 client.19956 populate_metadata read hostname 'smithi044' 2023-09-10T01:21:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.956+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/2666571657 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a380f672e0 0x55a380f69700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.956+0000 7f0fb8fcfb80 4 client.19956 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:21:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.956+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/2666571657 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a380eb4840 con 0x55a380f672e0 2023-09-10T01:21:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.957+0000 7f0fa659c700 1 --2- 172.21.15.44:0/2666571657 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a380f672e0 0x55a380f69700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.957+0000 7f0f8dffb700 10 client.19956 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.957+0000 7f0fa659c700 1 --2- 172.21.15.44:0/2666571657 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a380f672e0 0x55a380f69700 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0b955604c457fcd server_cookie=ee9be384be8058b0 in_seq=0 out_seq=0 2023-09-10T01:21:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.958+0000 7f0f8dffb700 1 -- 172.21.15.44:0/2666571657 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a380eb4840 con 0x55a380f672e0 2023-09-10T01:21:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.958+0000 7f0f8dffb700 10 client.19956 handle_command_reply: tid=0 2023-09-10T01:21:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.958+0000 7f0fb8fcfb80 1 client.19956 shutdown 2023-09-10T01:21:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.958+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/2666571657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f4dcc0 msgr2=0x55a380f57070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.958+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/2666571657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f4dcc0 0x55a380f57070 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00edb0 tx=0x7f0f9c01bf50 comp rx=0 tx=0).stop 2023-09-10T01:21:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.958+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/2666571657 shutdown_connections 2023-09-10T01:21:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.958+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/2666571657 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a380f672e0 0x55a380f69700 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.958+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/2666571657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f531a0 0x55a380f57610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.958+0000 7f0f8dffb700 0 client.19956 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.958+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/2666571657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f59d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.958+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/2666571657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f4dcc0 0x55a380f57070 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.958+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/2666571657 >> 172.21.15.44:0/2666571657 conn(0x55a380ec2030 msgr2=0x55a380f477a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.958+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/2666571657 shutdown_connections 2023-09-10T01:21:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.958+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/2666571657 wait complete. 2023-09-10T01:21:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.959+0000 7f0fb8fcfb80 20 client.19956 trim_cache size 0 max 16384 2023-09-10T01:21:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.978+0000 7f0fb8fcfb80 1 Processor -- start 2023-09-10T01:21:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.978+0000 7f0fb8fcfb80 1 -- start start 2023-09-10T01:21:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.979+0000 7f0fb8fcfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f5e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.979+0000 7f0fb8fcfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f4dcc0 0x55a380f568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.979+0000 7f0fb8fcfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f531a0 0x55a380f56de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.979+0000 7f0fb8fcfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a380f49930 con 0x55a380f4dcc0 2023-09-10T01:21:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.979+0000 7f0fb8fcfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a380f49d10 con 0x55a380f531a0 2023-09-10T01:21:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.979+0000 7f0fb8fcfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a380f5ea60 con 0x55a380f4d300 2023-09-10T01:21:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.979+0000 7f0fa659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f5e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.979+0000 7f0fa6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f531a0 0x55a380f56de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.979+0000 7f0fa5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f4dcc0 0x55a380f568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.979+0000 7f0fa659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f5e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37404/0 (socket says 172.21.15.44:37404) 2023-09-10T01:21:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.979+0000 7f0fa659c700 1 -- 172.21.15.44:0/3739939797 learned_addr learned my addr 172.21.15.44:0/3739939797 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.979+0000 7f0fa6d9d700 1 -- 172.21.15.44:0/3739939797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 msgr2=0x55a380f5e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.979+0000 7f0fa6d9d700 1 --2- 172.21.15.44:0/3739939797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f5e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.979+0000 7f0fa6d9d700 1 -- 172.21.15.44:0/3739939797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f4dcc0 msgr2=0x55a380f568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.979+0000 7f0fa6d9d700 1 --2- 172.21.15.44:0/3739939797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f4dcc0 0x55a380f568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.979+0000 7f0fa6d9d700 1 -- 172.21.15.44:0/3739939797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c004030 con 0x55a380f531a0 2023-09-10T01:21:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.980+0000 7f0fa659c700 1 --2- 172.21.15.44:0/3739939797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f5e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:21:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.980+0000 7f0fa5d9b700 1 --2- 172.21.15.44:0/3739939797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f4dcc0 0x55a380f568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:21:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.980+0000 7f0fa6d9d700 1 --2- 172.21.15.44:0/3739939797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f531a0 0x55a380f56de0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009480 tx=0x7f0fa0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c6f9cced5819fa8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.980+0000 7f0f957fa700 1 -- 172.21.15.44:0/3739939797 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa000e4f0 con 0x55a380f531a0 2023-09-10T01:21:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.980+0000 7f0f957fa700 1 -- 172.21.15.44:0/3739939797 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fa001b070 con 0x55a380f531a0 2023-09-10T01:21:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.980+0000 7f0f957fa700 1 -- 172.21.15.44:0/3739939797 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0017940 con 0x55a380f531a0 2023-09-10T01:21:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.981+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/3739939797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f531a0 msgr2=0x55a380f56de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.981+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/3739939797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f531a0 0x55a380f56de0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009480 tx=0x7f0fa0009920 comp rx=0 tx=0).stop 2023-09-10T01:21:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.981+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/3739939797 shutdown_connections 2023-09-10T01:21:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.981+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/3739939797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f531a0 0x55a380f56de0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.981+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/3739939797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4d300 0x55a380f5e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.981+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/3739939797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f4dcc0 0x55a380f568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.981+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/3739939797 >> 172.21.15.44:0/3739939797 conn(0x55a380ec2030 msgr2=0x55a380f52170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.981+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/3739939797 shutdown_connections 2023-09-10T01:21:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.981+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/3739939797 wait complete. 2023-09-10T01:21:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.981+0000 7f0fb8fcfb80 1 -- start start 2023-09-10T01:21:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.982+0000 7f0fb8fcfb80 1 Processor -- start 2023-09-10T01:21:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.982+0000 7f0fb8fcfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f4d300 0x55a380fb43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.982+0000 7f0fb8fcfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4dcc0 0x55a380fb4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.982+0000 7f0fb8fcfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f531a0 0x55a380fb3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.982+0000 7f0fb8fcfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a380fb35c0 con 0x55a380f531a0 2023-09-10T01:21:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.982+0000 7f0fb8fcfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a380fb3730 con 0x55a380f4d300 2023-09-10T01:21:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.982+0000 7f0fb8fcfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a380fb38f0 con 0x55a380f4dcc0 2023-09-10T01:21:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.982+0000 7f0fb8fcfb80 10 client.0 _ll_register_callbacks cb 0x7f0f98022ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:21:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.982+0000 7f0fa5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4dcc0 0x55a380fb4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.982+0000 7f0fa5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4dcc0 0x55a380fb4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37412/0 (socket says 172.21.15.44:37412) 2023-09-10T01:21:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.982+0000 7f0fa659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f4d300 0x55a380fb43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.982+0000 7f0fa5d9b700 1 -- 172.21.15.44:0/4226531999 learned_addr learned my addr 172.21.15.44:0/4226531999 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.982+0000 7f0fa6d9d700 1 --2- 172.21.15.44:0/4226531999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f531a0 0x55a380fb3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.983+0000 7f0fa5d9b700 1 -- 172.21.15.44:0/4226531999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f4d300 msgr2=0x55a380fb43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.983+0000 7f0fa5d9b700 1 --2- 172.21.15.44:0/4226531999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f4d300 0x55a380fb43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.983+0000 7f0fa5d9b700 1 -- 172.21.15.44:0/4226531999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f531a0 msgr2=0x55a380fb3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.983+0000 7f0fa5d9b700 1 --2- 172.21.15.44:0/4226531999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f531a0 0x55a380fb3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.983+0000 7f0fa5d9b700 1 -- 172.21.15.44:0/4226531999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f980b8730 con 0x55a380f4dcc0 2023-09-10T01:21:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.983+0000 7f0fa5d9b700 1 --2- 172.21.15.44:0/4226531999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4dcc0 0x55a380fb4910 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c004ba0 tx=0x7f0f9c00a3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ea121cb4ee15595 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.983+0000 7f0f8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:21:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.983+0000 7f0f8dffb700 1 -- 172.21.15.44:0/4226531999 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c013030 con 0x55a380f4dcc0 2023-09-10T01:21:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.983+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/4226531999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f9c01b470 con 0x55a380f4dcc0 2023-09-10T01:21:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.983+0000 7f0f8dffb700 1 -- 172.21.15.44:0/4226531999 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f0f9c003d20 con 0x55a380f4dcc0 2023-09-10T01:21:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.983+0000 7f0f8dffb700 1 -- 172.21.15.44:0/4226531999 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f9c02b040 con 0x55a380f4dcc0 2023-09-10T01:21:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.983+0000 7f0fb8fcfb80 10 client.29566 fetch_fsmap learned FSMap version 136 2023-09-10T01:21:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.983+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/4226531999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=4+}) v3 -- 0x7f0f9c021040 con 0x55a380f4dcc0 2023-09-10T01:21:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.984+0000 7f0f8dffb700 1 -- 172.21.15.44:0/4226531999 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f9c01b470 con 0x55a380f4dcc0 2023-09-10T01:21:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.985+0000 7f0f8dffb700 1 -- 172.21.15.44:0/4226531999 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 136) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f0f9c004d90 con 0x55a380f4dcc0 2023-09-10T01:21:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.985+0000 7f0fb8fcfb80 10 client.29566 fetch_fsmap finished waiting for FSMap version 136 2023-09-10T01:21:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.985+0000 7f0fb8fcfb80 10 client.29566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:21:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.985+0000 7f0fb8fcfb80 20 client.29566 populate_metadata read hostname 'smithi044' 2023-09-10T01:21:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.985+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/4226531999 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a380fccb50 0x55a380fcef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.985+0000 7f0fb8fcfb80 4 client.29566 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:21:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.985+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/4226531999 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a380e988c0 con 0x55a380fccb50 2023-09-10T01:21:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.985+0000 7f0fa659c700 1 --2- 172.21.15.44:0/4226531999 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a380fccb50 0x55a380fcef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.986+0000 7f0fa659c700 1 --2- 172.21.15.44:0/4226531999 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a380fccb50 0x55a380fcef40 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa4895b6dd9aa7c2 server_cookie=daf5e7b120a83ee6 in_seq=0 out_seq=0 2023-09-10T01:21:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:41.986+0000 7f0f8dffb700 10 client.29566 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.000+0000 7f0f8dffb700 1 -- 172.21.15.44:0/4226531999 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a380e988c0 con 0x55a380fccb50 2023-09-10T01:21:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.000+0000 7f0f8dffb700 10 client.29566 handle_command_reply: tid=0 2023-09-10T01:21:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.000+0000 7f0fb8fcfb80 1 client.29566 shutdown 2023-09-10T01:21:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.000+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/4226531999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4dcc0 msgr2=0x55a380fb4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.001+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/4226531999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4dcc0 0x55a380fb4910 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c004ba0 tx=0x7f0f9c00a3e0 comp rx=0 tx=0).stop 2023-09-10T01:21:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.001+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/4226531999 shutdown_connections 2023-09-10T01:21:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.001+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/4226531999 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a380fccb50 0x55a380fcef40 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.001+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/4226531999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a380f531a0 0x55a380fb3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.001+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/4226531999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a380f4d300 0x55a380fb43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.001+0000 7f0fb8fcfb80 1 --2- 172.21.15.44:0/4226531999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a380f4dcc0 0x55a380fb4910 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.001+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/4226531999 >> 172.21.15.44:0/4226531999 conn(0x55a380ec2030 msgr2=0x55a380f4c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.001+0000 7f0f8dffb700 0 client.29566 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.001+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/4226531999 shutdown_connections 2023-09-10T01:21:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.001+0000 7f0fb8fcfb80 1 -- 172.21.15.44:0/4226531999 wait complete. 2023-09-10T01:21:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.001+0000 7f0fb8fcfb80 20 client.29566 trim_cache size 0 max 16384 2023-09-10T01:21:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:21:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:21:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:21:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:21:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.002+0000 7f0face09700 1 -- 172.21.15.44:0/3042364714 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f900a07b0 msgr2=0x7f0f900a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.002+0000 7f0face09700 1 --2- 172.21.15.44:0/3042364714 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f900a07b0 0x7f0f900a2c70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8195da0 tx=0x7f0f9c010040 comp rx=0 tx=0).stop 2023-09-10T01:21:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.002+0000 7f0face09700 1 -- 172.21.15.44:0/3042364714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fa80fee10 msgr2=0x7f0fa81943a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.002+0000 7f0face09700 1 --2- 172.21.15.44:0/3042364714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fa80fee10 0x7f0fa81943a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0f98000c00 tx=0x7f0f98004730 comp rx=0 tx=0).stop 2023-09-10T01:21:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.004+0000 7f0face09700 1 -- 172.21.15.44:0/3042364714 shutdown_connections 2023-09-10T01:21:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.004+0000 7f0face09700 1 --2- 172.21.15.44:0/3042364714 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f0f900a07b0 0x7f0f900a2c70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.004+0000 7f0face09700 1 --2- 172.21.15.44:0/3042364714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fa8104e00 0x7f0fa8194e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.004+0000 7f0face09700 1 --2- 172.21.15.44:0/3042364714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fa80fee10 0x7f0fa81943a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.004+0000 7f0face09700 1 --2- 172.21.15.44:0/3042364714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fa80ff7b0 0x7f0fa81948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.004+0000 7f0face09700 1 -- 172.21.15.44:0/3042364714 >> 172.21.15.44:0/3042364714 conn(0x7f0fa8075920 msgr2=0x7f0fa80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.004+0000 7f0face09700 1 -- 172.21.15.44:0/3042364714 shutdown_connections 2023-09-10T01:21:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:42.004+0000 7f0face09700 1 -- 172.21.15.44:0/3042364714 wait complete. 2023-09-10T01:21:43.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:43 smithi131 ceph-mon[103713]: pgmap v1326: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 10 MiB/s wr, 798 op/s 2023-09-10T01:21:43.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:43 smithi044 ceph-mon[98676]: pgmap v1326: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 10 MiB/s wr, 798 op/s 2023-09-10T01:21:43.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:43 smithi099 ceph-mon[102692]: pgmap v1326: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 10 MiB/s wr, 798 op/s 2023-09-10T01:21:45.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:45 smithi131 ceph-mon[103713]: pgmap v1327: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 7.8 MiB/s wr, 574 op/s 2023-09-10T01:21:45.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:45 smithi044 ceph-mon[98676]: pgmap v1327: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 7.8 MiB/s wr, 574 op/s 2023-09-10T01:21:45.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:45 smithi099 ceph-mon[102692]: pgmap v1327: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 7.8 MiB/s wr, 574 op/s 2023-09-10T01:21:46.185 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 40.0 s, 349.2 tps, lat 286.140 ms stddev 195.797 2023-09-10T01:21:47.552 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:47 smithi131 ceph-mon[103713]: pgmap v1328: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 12 MiB/s wr, 870 op/s 2023-09-10T01:21:47.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:47 smithi044 ceph-mon[98676]: pgmap v1328: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 12 MiB/s wr, 870 op/s 2023-09-10T01:21:47.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:47 smithi099 ceph-mon[102692]: pgmap v1328: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 12 MiB/s wr, 870 op/s 2023-09-10T01:21:49.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:49 smithi131 ceph-mon[103713]: pgmap v1329: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 9.0 MiB/s wr, 633 op/s 2023-09-10T01:21:49.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:49 smithi044 ceph-mon[98676]: pgmap v1329: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 9.0 MiB/s wr, 633 op/s 2023-09-10T01:21:49.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:49 smithi099 ceph-mon[102692]: pgmap v1329: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 9.0 MiB/s wr, 633 op/s 2023-09-10T01:21:51.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 45.0 s, 237.4 tps, lat 392.947 ms stddev 332.379 2023-09-10T01:21:51.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:51 smithi131 ceph-mon[103713]: pgmap v1330: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 9.0 MiB/s wr, 632 op/s 2023-09-10T01:21:51.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:51 smithi044 ceph-mon[98676]: pgmap v1330: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 9.0 MiB/s wr, 632 op/s 2023-09-10T01:21:51.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:51 smithi099 ceph-mon[102692]: pgmap v1330: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 9.0 MiB/s wr, 632 op/s 2023-09-10T01:21:53.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:53 smithi131 ceph-mon[103713]: pgmap v1331: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 12 MiB/s wr, 862 op/s 2023-09-10T01:21:53.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:53 smithi044 ceph-mon[98676]: pgmap v1331: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 12 MiB/s wr, 862 op/s 2023-09-10T01:21:53.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:53 smithi099 ceph-mon[102692]: pgmap v1331: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 12 MiB/s wr, 862 op/s 2023-09-10T01:21:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.208+0000 7f4b1e010700 1 -- 172.21.15.44:0/1826661526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b18107fc0 msgr2=0x7f4b18113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.208+0000 7f4b1e010700 1 --2- 172.21.15.44:0/1826661526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b18107fc0 0x7f4b18113e90 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000c340 tx=0x7f4b1000c650 comp rx=0 tx=0).stop 2023-09-10T01:21:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.208+0000 7f4b1e010700 1 -- 172.21.15.44:0/1826661526 shutdown_connections 2023-09-10T01:21:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.208+0000 7f4b1e010700 1 --2- 172.21.15.44:0/1826661526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b18107fc0 0x7f4b18113e90 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.208+0000 7f4b1e010700 1 --2- 172.21.15.44:0/1826661526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b1810c000 0x7f4b1810c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.208+0000 7f4b1e010700 1 --2- 172.21.15.44:0/1826661526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b18107600 0x7f4b18107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.208+0000 7f4b1e010700 1 -- 172.21.15.44:0/1826661526 >> 172.21.15.44:0/1826661526 conn(0x7f4b1806cb00 msgr2=0x7f4b1806cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.211+0000 7f4b1e010700 1 -- 172.21.15.44:0/1826661526 shutdown_connections 2023-09-10T01:21:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.214+0000 7f4b1e010700 1 -- 172.21.15.44:0/1826661526 wait complete. 2023-09-10T01:21:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.214+0000 7f4b1e010700 1 Processor -- start 2023-09-10T01:21:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.215+0000 7f4b1e010700 1 -- start start 2023-09-10T01:21:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.215+0000 7f4b1e010700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b18107600 0x7f4b1811cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.215+0000 7f4b1e010700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b18107fc0 0x7f4b1811d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.215+0000 7f4b1e010700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b1810c000 0x7f4b18117de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.215+0000 7f4b1e010700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b181185b0 con 0x7f4b18107fc0 2023-09-10T01:21:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.215+0000 7f4b1e010700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b18118720 con 0x7f4b18107600 2023-09-10T01:21:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.215+0000 7f4b1e010700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b181188e0 con 0x7f4b1810c000 2023-09-10T01:21:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.219+0000 7f4b177fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b18107600 0x7f4b1811cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.219+0000 7f4b177fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b18107600 0x7f4b1811cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60514/0 (socket says 172.21.15.44:60514) 2023-09-10T01:21:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.219+0000 7f4b177fe700 1 -- 172.21.15.44:0/726811220 learned_addr learned my addr 172.21.15.44:0/726811220 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.220+0000 7f4b177fe700 1 -- 172.21.15.44:0/726811220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b1810c000 msgr2=0x7f4b18117de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T01:21:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.220+0000 7f4b177fe700 1 --2- 172.21.15.44:0/726811220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b1810c000 0x7f4b18117de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.220+0000 7f4b177fe700 1 -- 172.21.15.44:0/726811220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b18107fc0 msgr2=0x7f4b1811d490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T01:21:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.220+0000 7f4b177fe700 1 --2- 172.21.15.44:0/726811220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b18107fc0 0x7f4b1811d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.220+0000 7f4b177fe700 1 -- 172.21.15.44:0/726811220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1000c050 con 0x7f4b18107600 2023-09-10T01:21:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.220+0000 7f4b177fe700 1 --2- 172.21.15.44:0/726811220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b18107600 0x7f4b1811cf30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800b550 tx=0x7f4b0800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.220+0000 7f4b14ff9700 1 -- 172.21.15.44:0/726811220 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b08011830 con 0x7f4b18107600 2023-09-10T01:21:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.221+0000 7f4b14ff9700 1 -- 172.21.15.44:0/726811220 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b08011e20 con 0x7f4b18107600 2023-09-10T01:21:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.221+0000 7f4b1e010700 1 -- 172.21.15.44:0/726811220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b18118bc0 con 0x7f4b18107600 2023-09-10T01:21:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.221+0000 7f4b14ff9700 1 -- 172.21.15.44:0/726811220 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b08010da0 con 0x7f4b18107600 2023-09-10T01:21:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.221+0000 7f4b1e010700 1 -- 172.21.15.44:0/726811220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b18123020 con 0x7f4b18107600 2023-09-10T01:21:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.225+0000 7f4b14ff9700 1 -- 172.21.15.44:0/726811220 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f4b0801a400 con 0x7f4b18107600 2023-09-10T01:21:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.226+0000 7f4b2a1d6b80 1 Processor -- start 2023-09-10T01:21:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.226+0000 7f4b2a1d6b80 1 -- start start 2023-09-10T01:21:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.226+0000 7f4b2a1d6b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c1061a0 0x55867c106580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.226+0000 7f4b2a1d6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100300 0x55867c100780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.226+0000 7f4b2a1d6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c100cc0 0x55867c10cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.226+0000 7f4b2a1d6b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55867c0596d0 con 0x55867c1061a0 2023-09-10T01:21:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.226+0000 7f4b2a1d6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55867c056ff0 con 0x55867c100300 2023-09-10T01:21:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.226+0000 7f4b2a1d6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55867c0528e0 con 0x55867c100cc0 2023-09-10T01:21:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.227+0000 7f4b17fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c1061a0 0x55867c106580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.227+0000 7f4b17fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c1061a0 0x55867c106580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58776/0 (socket says 172.21.15.44:58776) 2023-09-10T01:21:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.227+0000 7f4b17fff700 1 -- 172.21.15.44:0/465293446 learned_addr learned my addr 172.21.15.44:0/465293446 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.227+0000 7f4b14ff9700 1 --2- 172.21.15.44:0/726811220 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4b000a08f0 0x7f4b000a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.227+0000 7f4b17fff700 1 -- 172.21.15.44:0/465293446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c100cc0 msgr2=0x55867c10cf30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:21:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.227+0000 7f4b17fff700 1 --2- 172.21.15.44:0/465293446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c100cc0 0x55867c10cf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.227+0000 7f4b17fff700 1 -- 172.21.15.44:0/465293446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100300 msgr2=0x55867c100780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:21:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.227+0000 7f4b17fff700 1 --2- 172.21.15.44:0/465293446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100300 0x55867c100780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.227+0000 7f4b17fff700 1 -- 172.21.15.44:0/465293446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55867be7ed80 con 0x55867c1061a0 2023-09-10T01:21:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.227+0000 7f4b14ff9700 1 -- 172.21.15.44:0/726811220 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f4b080ccc40 con 0x7f4b18107600 2023-09-10T01:21:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.227+0000 7f4b17fff700 1 --2- 172.21.15.44:0/726811220 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4b000a08f0 0x7f4b000a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.228+0000 7f4b17fff700 1 --2- 172.21.15.44:0/465293446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c1061a0 0x55867c106580 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000a010 tx=0x7f4b10012390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c679c176ad34ebd4 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.228+0000 7f4afdffb700 1 -- 172.21.15.44:0/465293446 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10013bf0 con 0x55867c1061a0 2023-09-10T01:21:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.228+0000 7f4afdffb700 1 -- 172.21.15.44:0/465293446 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b10013dd0 con 0x55867c1061a0 2023-09-10T01:21:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.228+0000 7f4afdffb700 1 -- 172.21.15.44:0/465293446 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1001f9f0 con 0x55867c1061a0 2023-09-10T01:21:55.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.229+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/465293446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c1061a0 msgr2=0x55867c106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:55.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.229+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/465293446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c1061a0 0x55867c106580 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000a010 tx=0x7f4b10012390 comp rx=0 tx=0).stop 2023-09-10T01:21:55.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.229+0000 7f4b17fff700 1 --2- 172.21.15.44:0/726811220 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4b000a08f0 0x7f4b000a2db0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4b10027000 tx=0x7f4b10028720 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:55.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.229+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/465293446 shutdown_connections 2023-09-10T01:21:55.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.229+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/465293446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c100cc0 0x55867c10cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.229+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/465293446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c1061a0 0x55867c106580 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.229+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/465293446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100300 0x55867c100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.229+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/465293446 >> 172.21.15.44:0/465293446 conn(0x55867c075030 msgr2=0x55867c0fb560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:55.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.233+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/465293446 shutdown_connections 2023-09-10T01:21:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.236+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/465293446 wait complete. 2023-09-10T01:21:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.236+0000 7f4b2a1d6b80 1 -- start start 2023-09-10T01:21:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.236+0000 7f4b2a1d6b80 1 Processor -- start 2023-09-10T01:21:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.237+0000 7f4b2a1d6b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c100300 0x55867c10cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.237+0000 7f4b2a1d6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c100cc0 0x55867c109fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.237+0000 7f4b2a1d6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c1061a0 0x55867c10a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.237+0000 7f4b2a1d6b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55867bff0b10 con 0x55867c100300 2023-09-10T01:21:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.237+0000 7f4b2a1d6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55867bff0810 con 0x55867c1061a0 2023-09-10T01:21:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.237+0000 7f4b2a1d6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55867bed48c0 con 0x55867c100cc0 2023-09-10T01:21:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.237+0000 7f4b2a1d6b80 10 client.0 _ll_register_callbacks cb 0x55867bf3aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:21:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.237+0000 7f4b16ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c100cc0 0x55867c109fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.237+0000 7f4b16ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c100cc0 0x55867c109fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37432/0 (socket says 172.21.15.44:37432) 2023-09-10T01:21:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.237+0000 7f4b16ffd700 1 -- 172.21.15.44:0/1864893605 learned_addr learned my addr 172.21.15.44:0/1864893605 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.238+0000 7f4b16ffd700 1 -- 172.21.15.44:0/1864893605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c1061a0 msgr2=0x55867c10a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.238+0000 7f4b16ffd700 1 --2- 172.21.15.44:0/1864893605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c1061a0 0x55867c10a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.238+0000 7f4b16ffd700 1 -- 172.21.15.44:0/1864893605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c100300 msgr2=0x55867c10cd50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:21:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.238+0000 7f4b16ffd700 1 --2- 172.21.15.44:0/1864893605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c100300 0x55867c10cd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.238+0000 7f4b16ffd700 1 -- 172.21.15.44:0/1864893605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b18114720 con 0x55867c100cc0 2023-09-10T01:21:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.241+0000 7f4b16ffd700 1 --2- 172.21.15.44:0/1864893605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c100cc0 0x55867c109fd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009e60 tx=0x7f4b0c00b3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2535216e7177dee8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.241+0000 7f4af6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:21:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.241+0000 7f4af6ffd700 1 -- 172.21.15.44:0/1864893605 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c011420 con 0x55867c100cc0 2023-09-10T01:21:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.241+0000 7f4af6ffd700 1 -- 172.21.15.44:0/1864893605 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f4b0c011580 con 0x55867c100cc0 2023-09-10T01:21:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.241+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/1864893605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55867bfaa490 con 0x55867c100cc0 2023-09-10T01:21:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.241+0000 7f4af6ffd700 1 -- 172.21.15.44:0/1864893605 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b0c011740 con 0x55867c100cc0 2023-09-10T01:21:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.241+0000 7f4b2a1d6b80 10 client.29569 fetch_fsmap learned FSMap version 136 2023-09-10T01:21:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.241+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/1864893605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=4+}) v3 -- 0x55867be67c00 con 0x55867c100cc0 2023-09-10T01:21:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.242+0000 7f4af6ffd700 1 -- 172.21.15.44:0/1864893605 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55867bfaa490 con 0x55867c100cc0 2023-09-10T01:21:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.243+0000 7f4af6ffd700 1 -- 172.21.15.44:0/1864893605 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 136) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f4b0c019830 con 0x55867c100cc0 2023-09-10T01:21:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.243+0000 7f4b2a1d6b80 10 client.29569 fetch_fsmap finished waiting for FSMap version 136 2023-09-10T01:21:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.243+0000 7f4b2a1d6b80 10 client.29569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:21:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.243+0000 7f4b2a1d6b80 20 client.29569 populate_metadata read hostname 'smithi044' 2023-09-10T01:21:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.243+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/1864893605 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55867c11a540 0x55867c11c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.243+0000 7f4b2a1d6b80 4 client.29569 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:21:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.243+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/1864893605 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55867c067840 con 0x55867c11a540 2023-09-10T01:21:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.243+0000 7f4b177fe700 1 --2- 172.21.15.44:0/1864893605 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55867c11a540 0x55867c11c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.244+0000 7f4b177fe700 1 --2- 172.21.15.44:0/1864893605 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55867c11a540 0x55867c11c960 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31dadbd6e7c34796 server_cookie=4b0828b8dd33c40a in_seq=0 out_seq=0 2023-09-10T01:21:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.244+0000 7f4af6ffd700 10 client.29569 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.245+0000 7f4af6ffd700 1 -- 172.21.15.44:0/1864893605 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55867c067840 con 0x55867c11a540 2023-09-10T01:21:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.245+0000 7f4af6ffd700 10 client.29569 handle_command_reply: tid=0 2023-09-10T01:21:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.245+0000 7f4b2a1d6b80 1 client.29569 shutdown 2023-09-10T01:21:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.245+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/1864893605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c100cc0 msgr2=0x55867c109fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.245+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/1864893605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c100cc0 0x55867c109fd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009e60 tx=0x7f4b0c00b3c0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.245+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/1864893605 shutdown_connections 2023-09-10T01:21:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.245+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/1864893605 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55867c11a540 0x55867c11c960 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.245+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/1864893605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c1061a0 0x55867c10a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.245+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/1864893605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c100300 0x55867c10cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.245+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/1864893605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c100cc0 0x55867c109fd0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.245+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/1864893605 >> 172.21.15.44:0/1864893605 conn(0x55867c075030 msgr2=0x55867c0fa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.245+0000 7f4af6ffd700 0 client.29569 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.248+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/1864893605 shutdown_connections 2023-09-10T01:21:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.248+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/1864893605 wait complete. 2023-09-10T01:21:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.248+0000 7f4b2a1d6b80 20 client.29569 trim_cache size 0 max 16384 2023-09-10T01:21:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.267+0000 7f4b2a1d6b80 1 Processor -- start 2023-09-10T01:21:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.267+0000 7f4b2a1d6b80 1 -- start start 2023-09-10T01:21:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.267+0000 7f4b2a1d6b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c100300 0x55867c111590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.267+0000 7f4b2a1d6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100cc0 0x55867c10bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.267+0000 7f4b2a1d6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c1061a0 0x55867c0faf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.267+0000 7f4b2a1d6b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55867c1010a0 con 0x55867c100300 2023-09-10T01:21:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.267+0000 7f4b2a1d6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55867c106580 con 0x55867c100cc0 2023-09-10T01:21:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.267+0000 7f4b2a1d6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55867c101580 con 0x55867c1061a0 2023-09-10T01:21:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.267+0000 7f4b177fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c100300 0x55867c111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.267+0000 7f4b177fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c100300 0x55867c111590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58788/0 (socket says 172.21.15.44:58788) 2023-09-10T01:21:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.267+0000 7f4b16ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100cc0 0x55867c10bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.267+0000 7f4b177fe700 1 -- 172.21.15.44:0/4147654642 learned_addr learned my addr 172.21.15.44:0/4147654642 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.267+0000 7f4b17fff700 1 --2- 172.21.15.44:0/4147654642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c1061a0 0x55867c0faf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.268+0000 7f4b16ffd700 1 -- 172.21.15.44:0/4147654642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c1061a0 msgr2=0x55867c0faf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.268+0000 7f4b16ffd700 1 --2- 172.21.15.44:0/4147654642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c1061a0 0x55867c0faf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.268+0000 7f4b16ffd700 1 -- 172.21.15.44:0/4147654642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c100300 msgr2=0x55867c111590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.268+0000 7f4b16ffd700 1 --2- 172.21.15.44:0/4147654642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c100300 0x55867c111590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.268+0000 7f4b16ffd700 1 -- 172.21.15.44:0/4147654642 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55867bea51f0 con 0x55867c100cc0 2023-09-10T01:21:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.268+0000 7f4b177fe700 1 --2- 172.21.15.44:0/4147654642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c100300 0x55867c111590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:21:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.268+0000 7f4b16ffd700 1 --2- 172.21.15.44:0/4147654642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100cc0 0x55867c10bff0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00b0b0 tx=0x7f4b0c006bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe3de8792c0bd56 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.268+0000 7f4afe7fc700 1 -- 172.21.15.44:0/4147654642 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c009180 con 0x55867c100cc0 2023-09-10T01:21:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.268+0000 7f4afe7fc700 1 -- 172.21.15.44:0/4147654642 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b0c00b7c0 con 0x55867c100cc0 2023-09-10T01:21:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.268+0000 7f4afe7fc700 1 -- 172.21.15.44:0/4147654642 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c011810 con 0x55867c100cc0 2023-09-10T01:21:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.269+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/4147654642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100cc0 msgr2=0x55867c10bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.269+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/4147654642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100cc0 0x55867c10bff0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00b0b0 tx=0x7f4b0c006bf0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.269+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/4147654642 shutdown_connections 2023-09-10T01:21:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.269+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/4147654642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c1061a0 0x55867c0faf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.269+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/4147654642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c100300 0x55867c111590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.269+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/4147654642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100cc0 0x55867c10bff0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.269+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/4147654642 >> 172.21.15.44:0/4147654642 conn(0x55867c075030 msgr2=0x55867c10b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.269+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/4147654642 shutdown_connections 2023-09-10T01:21:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.269+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/4147654642 wait complete. 2023-09-10T01:21:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.270+0000 7f4b2a1d6b80 1 -- start start 2023-09-10T01:21:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.270+0000 7f4b2a1d6b80 1 Processor -- start 2023-09-10T01:21:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.270+0000 7f4b2a1d6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100300 0x55867c166630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.270+0000 7f4b2a1d6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c100cc0 0x55867c166b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.270+0000 7f4b2a1d6b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c1061a0 0x55867c1673c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.270+0000 7f4b2a1d6b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55867c1679a0 con 0x55867c1061a0 2023-09-10T01:21:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.270+0000 7f4b2a1d6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55867c167ae0 con 0x55867c100300 2023-09-10T01:21:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.271+0000 7f4b2a1d6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55867c167ca0 con 0x55867c100cc0 2023-09-10T01:21:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.271+0000 7f4b2a1d6b80 10 client.0 _ll_register_callbacks cb 0x7f4b0c0096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:21:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.271+0000 7f4b177fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100300 0x55867c166630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.271+0000 7f4b177fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100300 0x55867c166630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60534/0 (socket says 172.21.15.44:60534) 2023-09-10T01:21:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.271+0000 7f4b177fe700 1 -- 172.21.15.44:0/117835292 learned_addr learned my addr 172.21.15.44:0/117835292 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:21:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.271+0000 7f4b16ffd700 1 --2- 172.21.15.44:0/117835292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c100cc0 0x55867c166b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.271+0000 7f4b177fe700 1 -- 172.21.15.44:0/117835292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c100cc0 msgr2=0x55867c166b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.271+0000 7f4b177fe700 1 --2- 172.21.15.44:0/117835292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c100cc0 0x55867c166b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.271+0000 7f4b177fe700 1 -- 172.21.15.44:0/117835292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c1061a0 msgr2=0x55867c1673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:21:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.271+0000 7f4b177fe700 1 --2- 172.21.15.44:0/117835292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c1061a0 0x55867c1673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.271+0000 7f4b177fe700 1 -- 172.21.15.44:0/117835292 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b18123020 con 0x55867c100300 2023-09-10T01:21:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.271+0000 7f4b177fe700 1 --2- 172.21.15.44:0/117835292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100300 0x55867c166630 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4b08000c00 tx=0x7f4b08011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a89f5224451e059 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:21:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.271+0000 7f4af6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:21:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.271+0000 7f4af6ffd700 1 -- 172.21.15.44:0/117835292 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b080a1e10 con 0x55867c100300 2023-09-10T01:21:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.271+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/117835292 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b10013dd0 con 0x55867c100300 2023-09-10T01:21:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.271+0000 7f4af6ffd700 1 -- 172.21.15.44:0/117835292 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f4b08018310 con 0x55867c100300 2023-09-10T01:21:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.272+0000 7f4af6ffd700 1 -- 172.21.15.44:0/117835292 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b080184d0 con 0x55867c100300 2023-09-10T01:21:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.272+0000 7f4b2a1d6b80 10 client.29597 fetch_fsmap learned FSMap version 136 2023-09-10T01:21:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.272+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/117835292 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=4+}) v3 -- 0x55867be67c00 con 0x55867c100300 2023-09-10T01:21:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.272+0000 7f4af6ffd700 1 -- 172.21.15.44:0/117835292 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b10013dd0 con 0x55867c100300 2023-09-10T01:21:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.272+0000 7f4af6ffd700 1 -- 172.21.15.44:0/117835292 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 136) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f4b0809a100 con 0x55867c100300 2023-09-10T01:21:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.272+0000 7f4b2a1d6b80 10 client.29597 fetch_fsmap finished waiting for FSMap version 136 2023-09-10T01:21:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.272+0000 7f4b2a1d6b80 10 client.29597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:21:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.272+0000 7f4b2a1d6b80 20 client.29597 populate_metadata read hostname 'smithi044' 2023-09-10T01:21:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.272+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/117835292 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55867c16f520 0x55867c171960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:21:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.272+0000 7f4b2a1d6b80 4 client.29597 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:21:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.272+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/117835292 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55867c04b8c0 con 0x55867c16f520 2023-09-10T01:21:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.273+0000 7f4b16ffd700 1 --2- 172.21.15.44:0/117835292 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55867c16f520 0x55867c171960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:21:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.273+0000 7f4af6ffd700 10 client.29597 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.273+0000 7f4b16ffd700 1 --2- 172.21.15.44:0/117835292 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55867c16f520 0x55867c171960 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8bff676829fa2a2 server_cookie=ba6f40bcd24bb6d in_seq=0 out_seq=0 2023-09-10T01:21:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.325+0000 7f4af6ffd700 1 -- 172.21.15.44:0/117835292 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55867c04b8c0 con 0x55867c16f520 2023-09-10T01:21:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.325+0000 7f4af6ffd700 10 client.29597 handle_command_reply: tid=0 2023-09-10T01:21:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.325+0000 7f4b2a1d6b80 1 client.29597 shutdown 2023-09-10T01:21:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.325+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/117835292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100300 msgr2=0x55867c166630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.325+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/117835292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100300 0x55867c166630 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4b08000c00 tx=0x7f4b08011f90 comp rx=0 tx=0).stop 2023-09-10T01:21:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.325+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/117835292 shutdown_connections 2023-09-10T01:21:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.325+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/117835292 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55867c16f520 0x55867c171960 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.325+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/117835292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55867c1061a0 0x55867c1673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.325+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/117835292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55867c100300 0x55867c166630 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.325+0000 7f4b2a1d6b80 1 --2- 172.21.15.44:0/117835292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55867c100cc0 0x55867c166b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.325+0000 7f4af6ffd700 0 client.29597 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:21:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.325+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/117835292 >> 172.21.15.44:0/117835292 conn(0x55867c075030 msgr2=0x55867c0ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.325+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/117835292 shutdown_connections 2023-09-10T01:21:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.325+0000 7f4b2a1d6b80 1 -- 172.21.15.44:0/117835292 wait complete. 2023-09-10T01:21:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.325+0000 7f4b2a1d6b80 20 client.29597 trim_cache size 0 max 16384 2023-09-10T01:21:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:21:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:21:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:21:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:21:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.326+0000 7f4b1e010700 1 -- 172.21.15.44:0/726811220 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4b000a08f0 msgr2=0x7f4b000a2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.326+0000 7f4b1e010700 1 --2- 172.21.15.44:0/726811220 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4b000a08f0 0x7f4b000a2db0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4b10027000 tx=0x7f4b10028720 comp rx=0 tx=0).stop 2023-09-10T01:21:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.326+0000 7f4b1e010700 1 -- 172.21.15.44:0/726811220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b18107600 msgr2=0x7f4b1811cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:21:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.326+0000 7f4b1e010700 1 --2- 172.21.15.44:0/726811220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b18107600 0x7f4b1811cf30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800b550 tx=0x7f4b0800b910 comp rx=0 tx=0).stop 2023-09-10T01:21:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.326+0000 7f4b1e010700 1 -- 172.21.15.44:0/726811220 shutdown_connections 2023-09-10T01:21:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.326+0000 7f4b1e010700 1 --2- 172.21.15.44:0/726811220 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4b000a08f0 0x7f4b000a2db0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.326+0000 7f4b1e010700 1 --2- 172.21.15.44:0/726811220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b1810c000 0x7f4b18117de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.326+0000 7f4b1e010700 1 --2- 172.21.15.44:0/726811220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b18107600 0x7f4b1811cf30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.326+0000 7f4b1e010700 1 --2- 172.21.15.44:0/726811220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b18107fc0 0x7f4b1811d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:21:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.326+0000 7f4b1e010700 1 -- 172.21.15.44:0/726811220 >> 172.21.15.44:0/726811220 conn(0x7f4b1806cb00 msgr2=0x7f4b1810ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:21:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.326+0000 7f4b1e010700 1 -- 172.21.15.44:0/726811220 shutdown_connections 2023-09-10T01:21:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:21:55.326+0000 7f4b1e010700 1 -- 172.21.15.44:0/726811220 wait complete. 2023-09-10T01:21:55.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:55 smithi131 ceph-mon[103713]: pgmap v1332: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 7.2 MiB/s wr, 525 op/s 2023-09-10T01:21:55.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:55 smithi044 ceph-mon[98676]: pgmap v1332: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 7.2 MiB/s wr, 525 op/s 2023-09-10T01:21:55.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:55 smithi099 ceph-mon[102692]: pgmap v1332: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 7.2 MiB/s wr, 525 op/s 2023-09-10T01:21:56.184 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 50.0 s, 238.2 tps, lat 415.961 ms stddev 492.038 2023-09-10T01:21:57.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:57 smithi131 ceph-mon[103713]: pgmap v1333: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 10 MiB/s wr, 758 op/s 2023-09-10T01:21:57.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:57 smithi044 ceph-mon[98676]: pgmap v1333: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 10 MiB/s wr, 758 op/s 2023-09-10T01:21:57.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:57 smithi099 ceph-mon[102692]: pgmap v1333: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 10 MiB/s wr, 758 op/s 2023-09-10T01:21:58.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:21:58 smithi131 ceph-mon[103713]: pgmap v1334: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 6.5 MiB/s wr, 462 op/s 2023-09-10T01:21:58.566 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:21:58 smithi044 ceph-mon[98676]: pgmap v1334: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 6.5 MiB/s wr, 462 op/s 2023-09-10T01:21:58.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:21:58 smithi099 ceph-mon[102692]: pgmap v1334: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 6.5 MiB/s wr, 462 op/s 2023-09-10T01:22:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:00 smithi131 ceph-mon[103713]: pgmap v1335: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 6.5 MiB/s wr, 462 op/s 2023-09-10T01:22:01.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:00 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:22:01.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:00 smithi044 ceph-mon[98676]: pgmap v1335: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 6.5 MiB/s wr, 462 op/s 2023-09-10T01:22:01.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:00 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:22:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:00 smithi099 ceph-mon[102692]: pgmap v1335: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 6.5 MiB/s wr, 462 op/s 2023-09-10T01:22:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:00 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:22:01.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 55.0 s, 225.0 tps, lat 467.247 ms stddev 443.742 2023-09-10T01:22:03.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:03 smithi099 ceph-mon[102692]: pgmap v1336: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 10 MiB/s wr, 738 op/s 2023-09-10T01:22:03.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:03 smithi131 ceph-mon[103713]: pgmap v1336: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 10 MiB/s wr, 738 op/s 2023-09-10T01:22:03.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:03 smithi044 ceph-mon[98676]: pgmap v1336: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 10 MiB/s wr, 738 op/s 2023-09-10T01:22:04.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:04 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:04.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:04 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:04.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:04 smithi044 ceph-mon[98676]: pgmap v1337: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 6.8 MiB/s wr, 508 op/s 2023-09-10T01:22:04.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:04 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:04 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:04 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:04 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:04 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:04 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:04 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:04 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:04 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T01:22:04.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:04 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:04.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:04 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:04.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:04 smithi099 ceph-mon[102692]: pgmap v1337: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 6.8 MiB/s wr, 508 op/s 2023-09-10T01:22:04.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:04 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:04 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:04 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:04 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:04 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:04 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:04 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:04 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:04 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T01:22:04.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:04 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:04.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:04 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:04.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:04 smithi131 ceph-mon[103713]: pgmap v1337: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 6.8 MiB/s wr, 508 op/s 2023-09-10T01:22:04.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:04 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:04 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:04 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:04 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:04 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:04 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:04 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:04 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:04.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:04 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T01:22:05.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:05 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:22:05.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:05 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:05.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:05 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:05.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:05 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:05.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:05 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:05.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:05 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:05 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:05 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:05 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:05 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:05 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:05 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:05 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:05 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:22:05.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:05 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:05.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:05 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:05.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:05 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:05.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:05 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:05.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:05 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:05 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:05 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:05 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:05 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:05 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:05 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:05 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:05 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:22:05.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:05 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:05.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:05 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:05.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:05 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:05.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:05 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:05.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:05 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:05 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:05 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:05 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:05 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:05 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:05 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:05.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:05 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 60.0 s, 259.6 tps, lat 384.509 ms stddev 202.290 2023-09-10T01:22:06.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T01:22:06.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T01:22:06.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: pgmap v1338: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 9.7 MiB/s wr, 757 op/s 2023-09-10T01:22:06.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T01:22:06.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T01:22:06.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:22:06.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:22:06.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: pgmap v1339: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 7.7 MiB/s wr, 626 op/s 2023-09-10T01:22:06.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: pgmap v1340: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 9.6 MiB/s wr, 782 op/s 2023-09-10T01:22:06.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:06 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:06.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T01:22:06.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T01:22:06.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: pgmap v1338: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 9.7 MiB/s wr, 757 op/s 2023-09-10T01:22:06.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T01:22:06.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T01:22:06.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:22:06.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:22:06.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: pgmap v1339: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 7.7 MiB/s wr, 626 op/s 2023-09-10T01:22:06.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: pgmap v1340: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 9.6 MiB/s wr, 782 op/s 2023-09-10T01:22:06.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:06 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:06.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T01:22:06.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T01:22:06.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: pgmap v1338: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 9.7 MiB/s wr, 757 op/s 2023-09-10T01:22:06.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:22:06.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T01:22:06.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T01:22:06.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:22:06.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:22:06.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: pgmap v1339: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 7.7 MiB/s wr, 626 op/s 2023-09-10T01:22:06.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: pgmap v1340: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 9.6 MiB/s wr, 782 op/s 2023-09-10T01:22:06.862 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:06 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:22:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.520+0000 7f83089bf700 1 -- 172.21.15.44:0/3071843558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f830410f1c0 msgr2=0x7f8304107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.520+0000 7f83089bf700 1 --2- 172.21.15.44:0/3071843558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f830410f1c0 0x7f8304107b40 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f82f801a9a0 tx=0x7f82f801acb0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.520+0000 7f83089bf700 1 -- 172.21.15.44:0/3071843558 shutdown_connections 2023-09-10T01:22:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.520+0000 7f83089bf700 1 --2- 172.21.15.44:0/3071843558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8304108220 0x7f830411bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.520+0000 7f83089bf700 1 --2- 172.21.15.44:0/3071843558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830410ed50 0x7f8304107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.520+0000 7f83089bf700 1 --2- 172.21.15.44:0/3071843558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f830410f1c0 0x7f8304107b40 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.520+0000 7f83089bf700 1 -- 172.21.15.44:0/3071843558 >> 172.21.15.44:0/3071843558 conn(0x7f830406c820 msgr2=0x7f830406cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.534+0000 7f83089bf700 1 -- 172.21.15.44:0/3071843558 shutdown_connections 2023-09-10T01:22:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.534+0000 7f83089bf700 1 -- 172.21.15.44:0/3071843558 wait complete. 2023-09-10T01:22:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.535+0000 7f83089bf700 1 Processor -- start 2023-09-10T01:22:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.535+0000 7f83089bf700 1 -- start start 2023-09-10T01:22:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.536+0000 7f83089bf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830410ed50 0x7f8304115210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.536+0000 7f83089bf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f830410f1c0 0x7f8304115750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.536+0000 7f83089bf700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83041a6ae0 0x7f83041a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.536+0000 7f83089bf700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83041a94a0 con 0x7f830410f1c0 2023-09-10T01:22:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.536+0000 7f83089bf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83041a95e0 con 0x7f830410ed50 2023-09-10T01:22:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.536+0000 7f83089bf700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83041a97a0 con 0x7f83041a6ae0 2023-09-10T01:22:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.536+0000 7f8302d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83041a6ae0 0x7f83041a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.536+0000 7f8302d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83041a6ae0 0x7f83041a8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37458/0 (socket says 172.21.15.44:37458) 2023-09-10T01:22:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.536+0000 7f8302d9d700 1 -- 172.21.15.44:0/88578142 learned_addr learned my addr 172.21.15.44:0/88578142 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.536+0000 7f830259c700 1 --2- 172.21.15.44:0/88578142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830410ed50 0x7f8304115210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.536+0000 7f8301d9b700 1 --2- 172.21.15.44:0/88578142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f830410f1c0 0x7f8304115750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.537+0000 7f8302d9d700 1 -- 172.21.15.44:0/88578142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830410ed50 msgr2=0x7f8304115210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.537+0000 7f8302d9d700 1 --2- 172.21.15.44:0/88578142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830410ed50 0x7f8304115210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.537+0000 7f8302d9d700 1 -- 172.21.15.44:0/88578142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f830410f1c0 msgr2=0x7f8304115750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.537+0000 7f8302d9d700 1 --2- 172.21.15.44:0/88578142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f830410f1c0 0x7f8304115750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.537+0000 7f8302d9d700 1 -- 172.21.15.44:0/88578142 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f801a600 con 0x7f83041a6ae0 2023-09-10T01:22:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.543+0000 7f8302d9d700 1 --2- 172.21.15.44:0/88578142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83041a6ae0 0x7f83041a8ed0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00b700 tx=0x7f82fc00bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.544+0000 7f82f37fe700 1 -- 172.21.15.44:0/88578142 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc010860 con 0x7f83041a6ae0 2023-09-10T01:22:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.544+0000 7f82f37fe700 1 -- 172.21.15.44:0/88578142 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82fc010e50 con 0x7f83041a6ae0 2023-09-10T01:22:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.544+0000 7f82f37fe700 1 -- 172.21.15.44:0/88578142 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc00fe90 con 0x7f83041a6ae0 2023-09-10T01:22:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.544+0000 7f83089bf700 1 -- 172.21.15.44:0/88578142 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83041a9a80 con 0x7f83041a6ae0 2023-09-10T01:22:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.544+0000 7f83089bf700 1 -- 172.21.15.44:0/88578142 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83041a9f40 con 0x7f83041a6ae0 2023-09-10T01:22:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.549+0000 7f82f37fe700 1 -- 172.21.15.44:0/88578142 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f82fc007370 con 0x7f83041a6ae0 2023-09-10T01:22:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.550+0000 7f82f37fe700 1 --2- 172.21.15.44:0/88578142 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f82ec0a09c0 0x7f82ec0a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.550+0000 7f82f37fe700 1 -- 172.21.15.44:0/88578142 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f82fc0ccfd0 con 0x7f83041a6ae0 2023-09-10T01:22:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.552+0000 7f8314b85b80 1 Processor -- start 2023-09-10T01:22:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.552+0000 7f8314b85b80 1 -- start start 2023-09-10T01:22:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.552+0000 7f8314b85b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deed91a0 0x55a7deed9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.552+0000 7f8314b85b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed3300 0x55a7deed3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.552+0000 7f8314b85b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deed3cc0 0x55a7deedff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.552+0000 7f8314b85b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dee2c6d0 con 0x55a7deed3300 2023-09-10T01:22:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.552+0000 7f8314b85b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dee29ff0 con 0x55a7deed91a0 2023-09-10T01:22:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.552+0000 7f8314b85b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dee258e0 con 0x55a7deed3cc0 2023-09-10T01:22:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.553+0000 7f8302d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deed3cc0 0x55a7deedff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.553+0000 7f8302d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deed3cc0 0x55a7deedff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37466/0 (socket says 172.21.15.44:37466) 2023-09-10T01:22:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.553+0000 7f8302d9d700 1 -- 172.21.15.44:0/1018368618 learned_addr learned my addr 172.21.15.44:0/1018368618 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.556+0000 7f8301d9b700 1 --2- 172.21.15.44:0/1018368618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed3300 0x55a7deed3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.556+0000 7f830259c700 1 --2- 172.21.15.44:0/88578142 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f82ec0a09c0 0x7f82ec0a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.556+0000 7f830259c700 1 --2- 172.21.15.44:0/1018368618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deed91a0 0x55a7deed9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.557+0000 7f830259c700 1 -- 172.21.15.44:0/1018368618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deed3cc0 msgr2=0x55a7deedff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.557+0000 7f830259c700 1 --2- 172.21.15.44:0/1018368618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deed3cc0 0x55a7deedff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.557+0000 7f830259c700 1 -- 172.21.15.44:0/1018368618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed3300 msgr2=0x55a7deed3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.557+0000 7f830259c700 1 --2- 172.21.15.44:0/1018368618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed3300 0x55a7deed3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.557+0000 7f830259c700 1 -- 172.21.15.44:0/1018368618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7dec51d80 con 0x55a7deed91a0 2023-09-10T01:22:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.557+0000 7f830259c700 1 --2- 172.21.15.44:0/1018368618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deed91a0 0x55a7deed9580 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f82f400af80 tx=0x7f82f40145d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f214d959a5436ad2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.558+0000 7f82f0ff9700 1 -- 172.21.15.44:0/1018368618 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4017470 con 0x55a7deed91a0 2023-09-10T01:22:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.558+0000 7f82f0ff9700 1 -- 172.21.15.44:0/1018368618 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82f4017a60 con 0x55a7deed91a0 2023-09-10T01:22:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.558+0000 7f82f0ff9700 1 -- 172.21.15.44:0/1018368618 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f401f9a0 con 0x55a7deed91a0 2023-09-10T01:22:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.559+0000 7f8314b85b80 1 -- 172.21.15.44:0/1018368618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deed91a0 msgr2=0x55a7deed9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.559+0000 7f8314b85b80 1 --2- 172.21.15.44:0/1018368618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deed91a0 0x55a7deed9580 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f82f400af80 tx=0x7f82f40145d0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.559+0000 7f8314b85b80 1 -- 172.21.15.44:0/1018368618 shutdown_connections 2023-09-10T01:22:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.559+0000 7f8314b85b80 1 --2- 172.21.15.44:0/1018368618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deed3cc0 0x55a7deedff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.559+0000 7f8314b85b80 1 --2- 172.21.15.44:0/1018368618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deed91a0 0x55a7deed9580 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.559+0000 7f8314b85b80 1 --2- 172.21.15.44:0/1018368618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed3300 0x55a7deed3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.559+0000 7f8314b85b80 1 -- 172.21.15.44:0/1018368618 >> 172.21.15.44:0/1018368618 conn(0x55a7dee48030 msgr2=0x55a7deece560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.560+0000 7f830259c700 1 --2- 172.21.15.44:0/88578142 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f82ec0a09c0 0x7f82ec0a2e80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f82f401c000 tx=0x7f82f401fc40 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.570+0000 7f8314b85b80 1 -- 172.21.15.44:0/1018368618 shutdown_connections 2023-09-10T01:22:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.570+0000 7f8314b85b80 1 -- 172.21.15.44:0/1018368618 wait complete. 2023-09-10T01:22:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.570+0000 7f8314b85b80 1 -- start start 2023-09-10T01:22:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.571+0000 7f8314b85b80 1 Processor -- start 2023-09-10T01:22:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.571+0000 7f8314b85b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deed3cc0 0x55a7deedfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.571+0000 7f8314b85b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed91a0 0x55a7deedd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.571+0000 7f8314b85b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deedda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.571+0000 7f8314b85b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dedc3b10 con 0x55a7deed91a0 2023-09-10T01:22:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.571+0000 7f8314b85b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dedc3810 con 0x55a7deed3cc0 2023-09-10T01:22:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.571+0000 7f8314b85b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7deca78c0 con 0x55a7deedd5a0 2023-09-10T01:22:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.571+0000 7f8314b85b80 10 client.0 _ll_register_callbacks cb 0x55a7ded0da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:22:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.571+0000 7f8302d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deedda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.571+0000 7f8302d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deedda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37476/0 (socket says 172.21.15.44:37476) 2023-09-10T01:22:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.571+0000 7f8302d9d700 1 -- 172.21.15.44:0/4116195340 learned_addr learned my addr 172.21.15.44:0/4116195340 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.572+0000 7f8302d9d700 1 -- 172.21.15.44:0/4116195340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deed3cc0 msgr2=0x55a7deedfd70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:22:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.572+0000 7f8302d9d700 1 --2- 172.21.15.44:0/4116195340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deed3cc0 0x55a7deedfd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.572+0000 7f8302d9d700 1 -- 172.21.15.44:0/4116195340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed91a0 msgr2=0x55a7deedd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.572+0000 7f8302d9d700 1 --2- 172.21.15.44:0/4116195340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed91a0 0x55a7deedd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.572+0000 7f8302d9d700 1 -- 172.21.15.44:0/4116195340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f400e040 con 0x55a7deedd5a0 2023-09-10T01:22:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.574+0000 7f8302d9d700 1 --2- 172.21.15.44:0/4116195340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deedda20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0ca320 tx=0x7f82fc0ca820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60706c23e7d717ed server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.574+0000 7f82e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:22:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.574+0000 7f82e9ffb700 1 -- 172.21.15.44:0/4116195340 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc0b1070 con 0x55a7deedd5a0 2023-09-10T01:22:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.574+0000 7f82e9ffb700 1 -- 172.21.15.44:0/4116195340 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f82fc0a2090 con 0x55a7deedd5a0 2023-09-10T01:22:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.574+0000 7f8314b85b80 1 -- 172.21.15.44:0/4116195340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7ded7d490 con 0x55a7deedd5a0 2023-09-10T01:22:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.576+0000 7f82e9ffb700 1 -- 172.21.15.44:0/4116195340 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82fc099720 con 0x55a7deedd5a0 2023-09-10T01:22:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.576+0000 7f8314b85b80 10 client.29584 fetch_fsmap learned FSMap version 136 2023-09-10T01:22:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.576+0000 7f8314b85b80 1 -- 172.21.15.44:0/4116195340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=4+}) v3 -- 0x55a7dec3ac00 con 0x55a7deedd5a0 2023-09-10T01:22:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.582+0000 7f82e9ffb700 1 -- 172.21.15.44:0/4116195340 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a7ded7d490 con 0x55a7deedd5a0 2023-09-10T01:22:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.582+0000 7f82e9ffb700 1 -- 172.21.15.44:0/4116195340 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 136) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f82fc095e10 con 0x55a7deedd5a0 2023-09-10T01:22:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.582+0000 7f8314b85b80 10 client.29584 fetch_fsmap finished waiting for FSMap version 136 2023-09-10T01:22:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.582+0000 7f8314b85b80 10 client.29584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:22:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.582+0000 7f8314b85b80 20 client.29584 populate_metadata read hostname 'smithi044' 2023-09-10T01:22:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.582+0000 7f8314b85b80 1 --2- 172.21.15.44:0/4116195340 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a7deeed730 0x55a7deeefb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.582+0000 7f8314b85b80 4 client.29584 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:22:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.582+0000 7f8314b85b80 1 -- 172.21.15.44:0/4116195340 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7dee3a840 con 0x55a7deeed730 2023-09-10T01:22:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.586+0000 7f830259c700 1 --2- 172.21.15.44:0/4116195340 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a7deeed730 0x55a7deeefb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.589+0000 7f830259c700 1 --2- 172.21.15.44:0/4116195340 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a7deeed730 0x55a7deeefb50 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af8c193f6f6d8c1d server_cookie=49dedd9f65d600fa in_seq=0 out_seq=0 2023-09-10T01:22:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.590+0000 7f82e9ffb700 10 client.29584 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:22:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.593+0000 7f82e9ffb700 1 -- 172.21.15.44:0/4116195340 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a7dee3a840 con 0x55a7deeed730 2023-09-10T01:22:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.593+0000 7f82e9ffb700 10 client.29584 handle_command_reply: tid=0 2023-09-10T01:22:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.593+0000 7f8314b85b80 1 client.29584 shutdown 2023-09-10T01:22:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.602+0000 7f8314b85b80 1 -- 172.21.15.44:0/4116195340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 msgr2=0x55a7deedda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.602+0000 7f8314b85b80 1 --2- 172.21.15.44:0/4116195340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deedda20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0ca320 tx=0x7f82fc0ca820 comp rx=0 tx=0).stop 2023-09-10T01:22:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.602+0000 7f8314b85b80 1 -- 172.21.15.44:0/4116195340 shutdown_connections 2023-09-10T01:22:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.602+0000 7f8314b85b80 1 --2- 172.21.15.44:0/4116195340 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a7deeed730 0x55a7deeefb50 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.602+0000 7f8314b85b80 1 --2- 172.21.15.44:0/4116195340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deedda20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.602+0000 7f8314b85b80 1 --2- 172.21.15.44:0/4116195340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deed3cc0 0x55a7deedfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.602+0000 7f8314b85b80 1 --2- 172.21.15.44:0/4116195340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed91a0 0x55a7deedd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.602+0000 7f8314b85b80 1 -- 172.21.15.44:0/4116195340 >> 172.21.15.44:0/4116195340 conn(0x55a7dee48030 msgr2=0x55a7deecd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.602+0000 7f82e9ffb700 0 client.29584 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:22:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.609+0000 7f8314b85b80 1 -- 172.21.15.44:0/4116195340 shutdown_connections 2023-09-10T01:22:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.609+0000 7f8314b85b80 1 -- 172.21.15.44:0/4116195340 wait complete. 2023-09-10T01:22:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.609+0000 7f8314b85b80 20 client.29584 trim_cache size 0 max 16384 2023-09-10T01:22:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.639+0000 7f8314b85b80 1 Processor -- start 2023-09-10T01:22:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.643+0000 7f8314b85b80 1 -- start start 2023-09-10T01:22:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.644+0000 7f8314b85b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deed3cc0 0x55a7deee4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.644+0000 7f8314b85b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed91a0 0x55a7deedf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.644+0000 7f8314b85b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deed8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.644+0000 7f8314b85b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7deed4450 con 0x55a7deed91a0 2023-09-10T01:22:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.644+0000 7f8314b85b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7deec88a0 con 0x55a7deed3cc0 2023-09-10T01:22:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.644+0000 7f8314b85b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7deee9550 con 0x55a7deedd5a0 2023-09-10T01:22:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.644+0000 7f8302d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deed8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.644+0000 7f8302d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deed8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37480/0 (socket says 172.21.15.44:37480) 2023-09-10T01:22:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.644+0000 7f8302d9d700 1 -- 172.21.15.44:0/18702836 learned_addr learned my addr 172.21.15.44:0/18702836 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.644+0000 7f830259c700 1 --2- 172.21.15.44:0/18702836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deed3cc0 0x55a7deee4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.644+0000 7f8302d9d700 1 -- 172.21.15.44:0/18702836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deed3cc0 msgr2=0x55a7deee4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.644+0000 7f8302d9d700 1 --2- 172.21.15.44:0/18702836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deed3cc0 0x55a7deee4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.644+0000 7f8302d9d700 1 -- 172.21.15.44:0/18702836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed91a0 msgr2=0x55a7deedf910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:22:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.644+0000 7f8302d9d700 1 --2- 172.21.15.44:0/18702836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed91a0 0x55a7deedf910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.644+0000 7f8302d9d700 1 -- 172.21.15.44:0/18702836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7deef47c0 con 0x55a7deedd5a0 2023-09-10T01:22:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.644+0000 7f8302d9d700 1 --2- 172.21.15.44:0/18702836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deed8070 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f82fc099d20 tx=0x7f82fc09dfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59ee9b7098852483 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.646+0000 7f82f17fa700 1 -- 172.21.15.44:0/18702836 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc0b1070 con 0x55a7deedd5a0 2023-09-10T01:22:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.646+0000 7f82f17fa700 1 -- 172.21.15.44:0/18702836 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82fc098fd0 con 0x55a7deedd5a0 2023-09-10T01:22:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.646+0000 7f82f17fa700 1 -- 172.21.15.44:0/18702836 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc0b2230 con 0x55a7deedd5a0 2023-09-10T01:22:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.647+0000 7f8314b85b80 1 -- 172.21.15.44:0/18702836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 msgr2=0x55a7deed8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.647+0000 7f8314b85b80 1 --2- 172.21.15.44:0/18702836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deed8070 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f82fc099d20 tx=0x7f82fc09dfd0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.647+0000 7f8314b85b80 1 -- 172.21.15.44:0/18702836 shutdown_connections 2023-09-10T01:22:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.647+0000 7f8314b85b80 1 --2- 172.21.15.44:0/18702836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deed8070 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.647+0000 7f8314b85b80 1 --2- 172.21.15.44:0/18702836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deed3cc0 0x55a7deee4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.647+0000 7f8314b85b80 1 --2- 172.21.15.44:0/18702836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed91a0 0x55a7deedf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.647+0000 7f8314b85b80 1 -- 172.21.15.44:0/18702836 >> 172.21.15.44:0/18702836 conn(0x55a7dee48030 msgr2=0x55a7deed7050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:08.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.649+0000 7f8314b85b80 1 -- 172.21.15.44:0/18702836 shutdown_connections 2023-09-10T01:22:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.655+0000 7f8314b85b80 1 -- 172.21.15.44:0/18702836 wait complete. 2023-09-10T01:22:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.656+0000 7f8314b85b80 1 -- start start 2023-09-10T01:22:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.656+0000 7f8314b85b80 1 Processor -- start 2023-09-10T01:22:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.656+0000 7f8314b85b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed3cc0 0x55a7deedb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.656+0000 7f8314b85b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deedbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.656+0000 7f8314b85b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deedc290 0x55a7deedc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.656+0000 7f8314b85b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7deedcc50 con 0x55a7deed3cc0 2023-09-10T01:22:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.656+0000 7f8314b85b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7deedcdc0 con 0x55a7deedc290 2023-09-10T01:22:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.656+0000 7f8314b85b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7deedcf80 con 0x55a7deedd5a0 2023-09-10T01:22:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.656+0000 7f8314b85b80 10 client.0 _ll_register_callbacks cb 0x55a7ded0da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:22:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.657+0000 7f8302d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deedc290 0x55a7deedc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.657+0000 7f8301d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deedbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.657+0000 7f8302d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deedc290 0x55a7deedc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60576/0 (socket says 172.21.15.44:60576) 2023-09-10T01:22:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.657+0000 7f8302d9d700 1 -- 172.21.15.44:0/4102832421 learned_addr learned my addr 172.21.15.44:0/4102832421 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.657+0000 7f8301d9b700 1 -- 172.21.15.44:0/4102832421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deedc290 msgr2=0x55a7deedc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.657+0000 7f8301d9b700 1 --2- 172.21.15.44:0/4102832421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deedc290 0x55a7deedc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.657+0000 7f8301d9b700 1 -- 172.21.15.44:0/4102832421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed3cc0 msgr2=0x55a7deedb810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:22:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.657+0000 7f8301d9b700 1 --2- 172.21.15.44:0/4102832421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed3cc0 0x55a7deedb810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.657+0000 7f8301d9b700 1 -- 172.21.15.44:0/4102832421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7dec3ac00 con 0x55a7deedd5a0 2023-09-10T01:22:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.657+0000 7f8302d9d700 1 --2- 172.21.15.44:0/4102832421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deedc290 0x55a7deedc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:22:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.657+0000 7f8301d9b700 1 --2- 172.21.15.44:0/4102832421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deedbd50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f82f801a8f0 tx=0x7f82f80043d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69f740cf27a45dc6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.657+0000 7f82e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:22:08.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.657+0000 7f82e9ffb700 1 -- 172.21.15.44:0/4102832421 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f801ccc0 con 0x55a7deedd5a0 2023-09-10T01:22:08.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.657+0000 7f82e9ffb700 1 -- 172.21.15.44:0/4102832421 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f82f801ce20 con 0x55a7deedd5a0 2023-09-10T01:22:08.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.657+0000 7f8314b85b80 1 -- 172.21.15.44:0/4102832421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82f4017a60 con 0x55a7deedd5a0 2023-09-10T01:22:08.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.658+0000 7f82e9ffb700 1 -- 172.21.15.44:0/4102832421 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82f80038c0 con 0x55a7deedd5a0 2023-09-10T01:22:08.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.658+0000 7f8314b85b80 10 client.29590 fetch_fsmap learned FSMap version 136 2023-09-10T01:22:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.658+0000 7f8314b85b80 1 -- 172.21.15.44:0/4102832421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=4+}) v3 -- 0x55a7dec781f0 con 0x55a7deedd5a0 2023-09-10T01:22:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.659+0000 7f82e9ffb700 1 -- 172.21.15.44:0/4102832421 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82f803e080 con 0x55a7deedd5a0 2023-09-10T01:22:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.659+0000 7f82e9ffb700 1 -- 172.21.15.44:0/4102832421 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 136) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f82f8028940 con 0x55a7deedd5a0 2023-09-10T01:22:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.659+0000 7f8314b85b80 10 client.29590 fetch_fsmap finished waiting for FSMap version 136 2023-09-10T01:22:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.659+0000 7f8314b85b80 10 client.29590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:22:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.659+0000 7f8314b85b80 20 client.29590 populate_metadata read hostname 'smithi044' 2023-09-10T01:22:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.659+0000 7f8314b85b80 1 --2- 172.21.15.44:0/4102832421 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a7def64620 0x55a7def66a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.659+0000 7f8314b85b80 4 client.29590 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:22:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.659+0000 7f8314b85b80 1 -- 172.21.15.44:0/4102832421 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a7dee1e8c0 con 0x55a7def64620 2023-09-10T01:22:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.661+0000 7f830259c700 1 --2- 172.21.15.44:0/4102832421 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a7def64620 0x55a7def66a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.661+0000 7f830259c700 1 --2- 172.21.15.44:0/4102832421 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a7def64620 0x55a7def66a40 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2fc43da93a7d591 server_cookie=d116067a35b1b439 in_seq=0 out_seq=0 2023-09-10T01:22:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.661+0000 7f82e9ffb700 10 client.29590 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:22:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.744+0000 7f82e9ffb700 1 -- 172.21.15.44:0/4102832421 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a7dee1e8c0 con 0x55a7def64620 2023-09-10T01:22:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.744+0000 7f82e9ffb700 10 client.29590 handle_command_reply: tid=0 2023-09-10T01:22:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.748+0000 7f8314b85b80 1 client.29590 shutdown 2023-09-10T01:22:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.750+0000 7f8314b85b80 1 -- 172.21.15.44:0/4102832421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 msgr2=0x55a7deedbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.750+0000 7f8314b85b80 1 --2- 172.21.15.44:0/4102832421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deedbd50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f82f801a8f0 tx=0x7f82f80043d0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.750+0000 7f8314b85b80 1 -- 172.21.15.44:0/4102832421 shutdown_connections 2023-09-10T01:22:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.750+0000 7f8314b85b80 1 --2- 172.21.15.44:0/4102832421 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a7def64620 0x55a7def66a40 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.750+0000 7f8314b85b80 1 --2- 172.21.15.44:0/4102832421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7deedc290 0x55a7deedc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.750+0000 7f8314b85b80 1 --2- 172.21.15.44:0/4102832421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7deed3cc0 0x55a7deedb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.750+0000 7f8314b85b80 1 --2- 172.21.15.44:0/4102832421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7deedd5a0 0x55a7deedbd50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.750+0000 7f8314b85b80 1 -- 172.21.15.44:0/4102832421 >> 172.21.15.44:0/4102832421 conn(0x55a7dee48030 msgr2=0x55a7deed8070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.750+0000 7f82e9ffb700 0 client.29590 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:22:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.751+0000 7f8314b85b80 1 -- 172.21.15.44:0/4102832421 shutdown_connections 2023-09-10T01:22:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.751+0000 7f8314b85b80 1 -- 172.21.15.44:0/4102832421 wait complete. 2023-09-10T01:22:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.751+0000 7f8314b85b80 20 client.29590 trim_cache size 0 max 16384 2023-09-10T01:22:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:22:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:22:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:22:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:22:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.756+0000 7f83089bf700 1 -- 172.21.15.44:0/88578142 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f82ec0a09c0 msgr2=0x7f82ec0a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.756+0000 7f83089bf700 1 --2- 172.21.15.44:0/88578142 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f82ec0a09c0 0x7f82ec0a2e80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f82f401c000 tx=0x7f82f401fc40 comp rx=0 tx=0).stop 2023-09-10T01:22:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.756+0000 7f83089bf700 1 -- 172.21.15.44:0/88578142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83041a6ae0 msgr2=0x7f83041a8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.756+0000 7f83089bf700 1 --2- 172.21.15.44:0/88578142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83041a6ae0 0x7f83041a8ed0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00b700 tx=0x7f82fc00bac0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.756+0000 7f83089bf700 1 -- 172.21.15.44:0/88578142 shutdown_connections 2023-09-10T01:22:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.756+0000 7f83089bf700 1 --2- 172.21.15.44:0/88578142 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f82ec0a09c0 0x7f82ec0a2e80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.756+0000 7f83089bf700 1 --2- 172.21.15.44:0/88578142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83041a6ae0 0x7f83041a8ed0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.756+0000 7f83089bf700 1 --2- 172.21.15.44:0/88578142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830410ed50 0x7f8304115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.756+0000 7f83089bf700 1 --2- 172.21.15.44:0/88578142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f830410f1c0 0x7f8304115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.756+0000 7f83089bf700 1 -- 172.21.15.44:0/88578142 >> 172.21.15.44:0/88578142 conn(0x7f830406c820 msgr2=0x7f8304070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.757+0000 7f83089bf700 1 -- 172.21.15.44:0/88578142 shutdown_connections 2023-09-10T01:22:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:08.757+0000 7f83089bf700 1 -- 172.21.15.44:0/88578142 wait complete. 2023-09-10T01:22:09.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:08 smithi099 ceph-mon[102692]: pgmap v1341: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 10 MiB/s wr, 823 op/s 2023-09-10T01:22:09.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:08 smithi131 ceph-mon[103713]: pgmap v1341: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 10 MiB/s wr, 823 op/s 2023-09-10T01:22:09.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:08 smithi044 ceph-mon[98676]: pgmap v1341: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 10 MiB/s wr, 823 op/s 2023-09-10T01:22:10.115 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:09 smithi099 ceph-mon[102692]: osdmap e172: 12 total, 12 up, 12 in 2023-09-10T01:22:10.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:09 smithi131 ceph-mon[103713]: osdmap e172: 12 total, 12 up, 12 in 2023-09-10T01:22:10.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:09 smithi044 ceph-mon[98676]: osdmap e172: 12 total, 12 up, 12 in 2023-09-10T01:22:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:10 smithi099 ceph-mon[102692]: pgmap v1343: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 6.6 MiB/s wr, 549 op/s 2023-09-10T01:22:11.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 65.0 s, 98.2 tps, lat 831.127 ms stddev 1227.053 2023-09-10T01:22:11.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:10 smithi131 ceph-mon[103713]: pgmap v1343: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 6.6 MiB/s wr, 549 op/s 2023-09-10T01:22:11.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:10 smithi044 ceph-mon[98676]: pgmap v1343: 129 pgs: 129 active+clean; 16 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 6.6 MiB/s wr, 549 op/s 2023-09-10T01:22:13.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:13 smithi099 ceph-mon[102692]: pgmap v1344: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 6.8 MiB/s wr, 545 op/s 2023-09-10T01:22:13.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:13 smithi131 ceph-mon[103713]: pgmap v1344: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 6.8 MiB/s wr, 545 op/s 2023-09-10T01:22:13.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:13 smithi044 ceph-mon[98676]: pgmap v1344: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 6.8 MiB/s wr, 545 op/s 2023-09-10T01:22:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:15 smithi099 ceph-mon[102692]: pgmap v1345: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 5.1 MiB/s wr, 411 op/s 2023-09-10T01:22:15.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:15 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:22:15.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:15 smithi131 ceph-mon[103713]: pgmap v1345: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 5.1 MiB/s wr, 411 op/s 2023-09-10T01:22:15.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:15 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:22:15.560 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:15 smithi044 ceph-mon[98676]: pgmap v1345: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 5.1 MiB/s wr, 411 op/s 2023-09-10T01:22:15.560 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:15 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:22:16.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 70.0 s, 323.2 tps, lat 375.482 ms stddev 510.948 2023-09-10T01:22:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:17 smithi099 ceph-mon[102692]: pgmap v1346: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 7.3 MiB/s wr, 527 op/s 2023-09-10T01:22:17.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:17 smithi131 ceph-mon[103713]: pgmap v1346: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 7.3 MiB/s wr, 527 op/s 2023-09-10T01:22:17.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:17 smithi044 ceph-mon[98676]: pgmap v1346: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 7.3 MiB/s wr, 527 op/s 2023-09-10T01:22:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:19 smithi099 ceph-mon[102692]: pgmap v1347: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 6.9 MiB/s wr, 518 op/s 2023-09-10T01:22:19.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:19 smithi131 ceph-mon[103713]: pgmap v1347: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 6.9 MiB/s wr, 518 op/s 2023-09-10T01:22:19.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:19 smithi044 ceph-mon[98676]: pgmap v1347: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 6.9 MiB/s wr, 518 op/s 2023-09-10T01:22:20.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:20 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:22:20.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:20 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:22:20.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:20 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:22:21.185 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 75.0 s, 63.0 tps, lat 291.763 ms stddev 105.061 2023-09-10T01:22:21.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:21 smithi131 ceph-mon[103713]: pgmap v1348: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 6.3 MiB/s wr, 474 op/s 2023-09-10T01:22:21.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:21 smithi131 ceph-mon[103713]: osdmap e173: 12 total, 12 up, 12 in 2023-09-10T01:22:21.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:21 smithi044 ceph-mon[98676]: pgmap v1348: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 6.3 MiB/s wr, 474 op/s 2023-09-10T01:22:21.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:21 smithi044 ceph-mon[98676]: osdmap e173: 12 total, 12 up, 12 in 2023-09-10T01:22:21.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:21 smithi099 ceph-mon[102692]: pgmap v1348: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 6.3 MiB/s wr, 474 op/s 2023-09-10T01:22:21.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:21 smithi099 ceph-mon[102692]: osdmap e173: 12 total, 12 up, 12 in 2023-09-10T01:22:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.908+0000 7fafd551f700 1 -- 172.21.15.44:0/1337578478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafd00ff760 msgr2=0x7fafd00ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.908+0000 7fafd551f700 1 --2- 172.21.15.44:0/1337578478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafd00ff760 0x7fafd00ffbe0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fafc40099c0 tx=0x7fafc4009cd0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.908+0000 7fafd551f700 1 -- 172.21.15.44:0/1337578478 shutdown_connections 2023-09-10T01:22:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.908+0000 7fafd551f700 1 --2- 172.21.15.44:0/1337578478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafd0100120 0x7fafd010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.908+0000 7fafd551f700 1 --2- 172.21.15.44:0/1337578478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd01055b0 0x7fafd0105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.908+0000 7fafd551f700 1 --2- 172.21.15.44:0/1337578478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafd00ff760 0x7fafd00ffbe0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.909+0000 7fafd551f700 1 -- 172.21.15.44:0/1337578478 >> 172.21.15.44:0/1337578478 conn(0x7fafd0074f30 msgr2=0x7fafd0075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.909+0000 7fafd551f700 1 -- 172.21.15.44:0/1337578478 shutdown_connections 2023-09-10T01:22:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.909+0000 7fafd551f700 1 -- 172.21.15.44:0/1337578478 wait complete. 2023-09-10T01:22:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.909+0000 7fafd551f700 1 Processor -- start 2023-09-10T01:22:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.909+0000 7fafd551f700 1 -- start start 2023-09-10T01:22:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafd551f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafd00ff760 0x7fafd01a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafd551f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd0100120 0x7fafd01a3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafd551f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafd01055b0 0x7fafd010a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafd551f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd010b050 con 0x7fafd01055b0 2023-09-10T01:22:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafd551f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd010b1c0 con 0x7fafd0100120 2023-09-10T01:22:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafd551f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd010b380 con 0x7fafd00ff760 2023-09-10T01:22:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafcf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafd01055b0 0x7fafd010a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafcf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafd01055b0 0x7fafd010a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58850/0 (socket says 172.21.15.44:58850) 2023-09-10T01:22:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafcf7fe700 1 -- 172.21.15.44:0/637158895 learned_addr learned my addr 172.21.15.44:0/637158895 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafce7fc700 1 --2- 172.21.15.44:0/637158895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd0100120 0x7fafd01a3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafceffd700 1 --2- 172.21.15.44:0/637158895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafd00ff760 0x7fafd01a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafcf7fe700 1 -- 172.21.15.44:0/637158895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafd00ff760 msgr2=0x7fafd01a2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafcf7fe700 1 --2- 172.21.15.44:0/637158895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafd00ff760 0x7fafd01a2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafcf7fe700 1 -- 172.21.15.44:0/637158895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd0100120 msgr2=0x7fafd01a3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafcf7fe700 1 --2- 172.21.15.44:0/637158895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd0100120 0x7fafd01a3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafcf7fe700 1 -- 172.21.15.44:0/637158895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc40095f0 con 0x7fafd01055b0 2023-09-10T01:22:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafcf7fe700 1 --2- 172.21.15.44:0/637158895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafd01055b0 0x7fafd010a9e0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fafc800ef30 tx=0x7fafc800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.910+0000 7fafb7fff700 1 -- 172.21.15.44:0/637158895 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc8018070 con 0x7fafd01055b0 2023-09-10T01:22:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.911+0000 7fafb7fff700 1 -- 172.21.15.44:0/637158895 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafc800f040 con 0x7fafd01055b0 2023-09-10T01:22:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.911+0000 7fafd551f700 1 -- 172.21.15.44:0/637158895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd010b600 con 0x7fafd01055b0 2023-09-10T01:22:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.911+0000 7fafb7fff700 1 -- 172.21.15.44:0/637158895 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc800cbc0 con 0x7fafd01055b0 2023-09-10T01:22:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.911+0000 7fafd551f700 1 -- 172.21.15.44:0/637158895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd01a9bd0 con 0x7fafd01055b0 2023-09-10T01:22:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.913+0000 7fafb7fff700 1 -- 172.21.15.44:0/637158895 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fafc8007500 con 0x7fafd01055b0 2023-09-10T01:22:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.913+0000 7fafb7fff700 1 --2- 172.21.15.44:0/637158895 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fafb80a0820 0x7fafb80a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.914+0000 7fafceffd700 1 --2- 172.21.15.44:0/637158895 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fafb80a0820 0x7fafb80a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.914+0000 7fafb7fff700 1 -- 172.21.15.44:0/637158895 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fafc80cbe90 con 0x7fafd01055b0 2023-09-10T01:22:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.914+0000 7fafceffd700 1 --2- 172.21.15.44:0/637158895 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fafb80a0820 0x7fafb80a2ce0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fafc0007be0 tx=0x7fafc00073d0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.914+0000 7fafe16e5b80 1 Processor -- start 2023-09-10T01:22:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.914+0000 7fafe16e5b80 1 -- start start 2023-09-10T01:22:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafe16e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc057561a0 0x55fc05756580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafe16e5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750300 0x55fc05750780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafe16e5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05750cc0 0x55fc0575cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafe16e5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc056a96d0 con 0x55fc05750300 2023-09-10T01:22:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafe16e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc056a6ff0 con 0x55fc057561a0 2023-09-10T01:22:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafe16e5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc056a28e0 con 0x55fc05750cc0 2023-09-10T01:22:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafce7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750300 0x55fc05750780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafce7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750300 0x55fc05750780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58856/0 (socket says 172.21.15.44:58856) 2023-09-10T01:22:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafce7fc700 1 -- 172.21.15.44:0/199480995 learned_addr learned my addr 172.21.15.44:0/199480995 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafcf7fe700 1 --2- 172.21.15.44:0/199480995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05750cc0 0x55fc0575cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafceffd700 1 --2- 172.21.15.44:0/199480995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc057561a0 0x55fc05756580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafce7fc700 1 -- 172.21.15.44:0/199480995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05750cc0 msgr2=0x55fc0575cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafce7fc700 1 --2- 172.21.15.44:0/199480995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05750cc0 0x55fc0575cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafce7fc700 1 -- 172.21.15.44:0/199480995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc057561a0 msgr2=0x55fc05756580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafce7fc700 1 --2- 172.21.15.44:0/199480995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc057561a0 0x55fc05756580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafce7fc700 1 -- 172.21.15.44:0/199480995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc054ced80 con 0x55fc05750300 2023-09-10T01:22:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafceffd700 1 --2- 172.21.15.44:0/199480995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc057561a0 0x55fc05756580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:22:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafcf7fe700 1 --2- 172.21.15.44:0/199480995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05750cc0 0x55fc0575cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:22:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafce7fc700 1 --2- 172.21.15.44:0/199480995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750300 0x55fc05750780 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fafc400b580 tx=0x7fafc4005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57681cf28ae9d5da server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafb57fa700 1 -- 172.21.15.44:0/199480995 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc400ba90 con 0x55fc05750300 2023-09-10T01:22:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.915+0000 7fafb57fa700 1 -- 172.21.15.44:0/199480995 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafc401e040 con 0x55fc05750300 2023-09-10T01:22:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.916+0000 7fafb57fa700 1 -- 172.21.15.44:0/199480995 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4003ae0 con 0x55fc05750300 2023-09-10T01:22:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.916+0000 7fafe16e5b80 1 -- 172.21.15.44:0/199480995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750300 msgr2=0x55fc05750780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.916+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/199480995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750300 0x55fc05750780 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fafc400b580 tx=0x7fafc4005c70 comp rx=0 tx=0).stop 2023-09-10T01:22:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.916+0000 7fafe16e5b80 1 -- 172.21.15.44:0/199480995 shutdown_connections 2023-09-10T01:22:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.916+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/199480995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05750cc0 0x55fc0575cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.916+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/199480995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc057561a0 0x55fc05756580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.916+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/199480995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750300 0x55fc05750780 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.916+0000 7fafe16e5b80 1 -- 172.21.15.44:0/199480995 >> 172.21.15.44:0/199480995 conn(0x55fc056c5030 msgr2=0x55fc0574b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.916+0000 7fafe16e5b80 1 -- 172.21.15.44:0/199480995 shutdown_connections 2023-09-10T01:22:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.916+0000 7fafe16e5b80 1 -- 172.21.15.44:0/199480995 wait complete. 2023-09-10T01:22:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafe16e5b80 1 -- start start 2023-09-10T01:22:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafe16e5b80 1 Processor -- start 2023-09-10T01:22:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafe16e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc05750300 0x55fc0575cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafe16e5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750cc0 0x55fc0575a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafe16e5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc057561a0 0x55fc0575a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafe16e5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc05640b10 con 0x55fc05750cc0 2023-09-10T01:22:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafe16e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc05640810 con 0x55fc05750300 2023-09-10T01:22:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafe16e5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc055248c0 con 0x55fc057561a0 2023-09-10T01:22:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafe16e5b80 10 client.0 _ll_register_callbacks cb 0x55fc0558aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:22:21.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafce7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750cc0 0x55fc0575a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafce7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750cc0 0x55fc0575a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58862/0 (socket says 172.21.15.44:58862) 2023-09-10T01:22:21.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafce7fc700 1 -- 172.21.15.44:0/4123783441 learned_addr learned my addr 172.21.15.44:0/4123783441 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:21.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafceffd700 1 --2- 172.21.15.44:0/4123783441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc05750300 0x55fc0575cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafcf7fe700 1 --2- 172.21.15.44:0/4123783441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc057561a0 0x55fc0575a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafce7fc700 1 -- 172.21.15.44:0/4123783441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc057561a0 msgr2=0x55fc0575a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafce7fc700 1 --2- 172.21.15.44:0/4123783441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc057561a0 0x55fc0575a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafce7fc700 1 -- 172.21.15.44:0/4123783441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc05750300 msgr2=0x55fc0575cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.917+0000 7fafce7fc700 1 --2- 172.21.15.44:0/4123783441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc05750300 0x55fc0575cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.918+0000 7fafce7fc700 1 -- 172.21.15.44:0/4123783441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd01a9bd0 con 0x55fc05750cc0 2023-09-10T01:22:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.918+0000 7fafce7fc700 1 --2- 172.21.15.44:0/4123783441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750cc0 0x55fc0575a060 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fafc4000c00 tx=0x7fafc40052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675e2d9f12933f3c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.918+0000 7fafb2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:22:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.918+0000 7fafb2ff5700 1 -- 172.21.15.44:0/4123783441 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4003960 con 0x55fc05750cc0 2023-09-10T01:22:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.918+0000 7fafb2ff5700 1 -- 172.21.15.44:0/4123783441 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(173..173 src has 1..173) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fafc40058e0 con 0x55fc05750cc0 2023-09-10T01:22:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.918+0000 7fafe16e5b80 1 -- 172.21.15.44:0/4123783441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc055fa490 con 0x55fc05750cc0 2023-09-10T01:22:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.918+0000 7fafb2ff5700 1 -- 172.21.15.44:0/4123783441 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=138) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafc400bd00 con 0x55fc05750cc0 2023-09-10T01:22:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.918+0000 7fafe16e5b80 10 client.19992 fetch_fsmap learned FSMap version 138 2023-09-10T01:22:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.918+0000 7fafe16e5b80 1 -- 172.21.15.44:0/4123783441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=138,monmap=4+}) v3 -- 0x7fafd010cbc0 con 0x55fc05750cc0 2023-09-10T01:22:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.918+0000 7fafb2ff5700 1 -- 172.21.15.44:0/4123783441 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fc055fa490 con 0x55fc05750cc0 2023-09-10T01:22:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.918+0000 7fafb2ff5700 1 -- 172.21.15.44:0/4123783441 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 138) v1 ==== 7367+0+0 (secure 0 0 0) 0x7fafc402bb60 con 0x55fc05750cc0 2023-09-10T01:22:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.918+0000 7fafe16e5b80 10 client.19992 fetch_fsmap finished waiting for FSMap version 138 2023-09-10T01:22:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.918+0000 7fafe16e5b80 10 client.19992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:22:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.918+0000 7fafe16e5b80 20 client.19992 populate_metadata read hostname 'smithi044' 2023-09-10T01:22:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.918+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/4123783441 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fc05769fd0 0x55fc0576c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.918+0000 7fafe16e5b80 4 client.19992 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:22:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.919+0000 7fafe16e5b80 1 -- 172.21.15.44:0/4123783441 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc056b7840 con 0x55fc05769fd0 2023-09-10T01:22:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.919+0000 7fafceffd700 1 --2- 172.21.15.44:0/4123783441 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fc05769fd0 0x55fc0576c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.919+0000 7fafb2ff5700 10 client.19992 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:22:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.919+0000 7fafceffd700 1 --2- 172.21.15.44:0/4123783441 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fc05769fd0 0x55fc0576c3f0 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74fd2d8fdf819bdf server_cookie=6ace62597bfdf02c in_seq=0 out_seq=0 2023-09-10T01:22:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.920+0000 7fafb2ff5700 1 -- 172.21.15.44:0/4123783441 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fc056b7840 con 0x55fc05769fd0 2023-09-10T01:22:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.920+0000 7fafb2ff5700 10 client.19992 handle_command_reply: tid=0 2023-09-10T01:22:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.920+0000 7fafe16e5b80 1 client.19992 shutdown 2023-09-10T01:22:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.920+0000 7fafe16e5b80 1 -- 172.21.15.44:0/4123783441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750cc0 msgr2=0x55fc0575a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.920+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/4123783441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750cc0 0x55fc0575a060 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fafc4000c00 tx=0x7fafc40052f0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.920+0000 7fafe16e5b80 1 -- 172.21.15.44:0/4123783441 shutdown_connections 2023-09-10T01:22:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.920+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/4123783441 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fc05769fd0 0x55fc0576c3f0 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.920+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/4123783441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc057561a0 0x55fc0575a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.920+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/4123783441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc05750300 0x55fc0575cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.920+0000 7fafb2ff5700 0 client.19992 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:22:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.920+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/4123783441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750cc0 0x55fc0575a060 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.920+0000 7fafe16e5b80 1 -- 172.21.15.44:0/4123783441 >> 172.21.15.44:0/4123783441 conn(0x55fc056c5030 msgr2=0x55fc0574a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.920+0000 7fafe16e5b80 1 -- 172.21.15.44:0/4123783441 shutdown_connections 2023-09-10T01:22:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.920+0000 7fafe16e5b80 1 -- 172.21.15.44:0/4123783441 wait complete. 2023-09-10T01:22:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.920+0000 7fafe16e5b80 20 client.19992 trim_cache size 0 max 16384 2023-09-10T01:22:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.935+0000 7fafe16e5b80 1 Processor -- start 2023-09-10T01:22:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.935+0000 7fafe16e5b80 1 -- start start 2023-09-10T01:22:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.935+0000 7fafe16e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc05750300 0x55fc05761520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.935+0000 7fafe16e5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750cc0 0x55fc05752b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.935+0000 7fafe16e5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc057561a0 0x55fc057530a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.935+0000 7fafe16e5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc055fa490 con 0x55fc05750cc0 2023-09-10T01:22:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.935+0000 7fafe16e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc057458a0 con 0x55fc05750300 2023-09-10T01:22:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.935+0000 7fafe16e5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc05747fc0 con 0x55fc057561a0 2023-09-10T01:22:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.936+0000 7fafce7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750cc0 0x55fc05752b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.936+0000 7fafce7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750cc0 0x55fc05752b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58870/0 (socket says 172.21.15.44:58870) 2023-09-10T01:22:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.936+0000 7fafce7fc700 1 -- 172.21.15.44:0/2765291618 learned_addr learned my addr 172.21.15.44:0/2765291618 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.936+0000 7fafceffd700 1 --2- 172.21.15.44:0/2765291618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc05750300 0x55fc05761520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.936+0000 7fafcf7fe700 1 --2- 172.21.15.44:0/2765291618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc057561a0 0x55fc057530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.936+0000 7fafce7fc700 1 -- 172.21.15.44:0/2765291618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc057561a0 msgr2=0x55fc057530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.936+0000 7fafce7fc700 1 --2- 172.21.15.44:0/2765291618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc057561a0 0x55fc057530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.936+0000 7fafce7fc700 1 -- 172.21.15.44:0/2765291618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc05750300 msgr2=0x55fc05761520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.936+0000 7fafce7fc700 1 --2- 172.21.15.44:0/2765291618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc05750300 0x55fc05761520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.936+0000 7fafce7fc700 1 -- 172.21.15.44:0/2765291618 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc05756850 con 0x55fc05750cc0 2023-09-10T01:22:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.936+0000 7fafce7fc700 1 --2- 172.21.15.44:0/2765291618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750cc0 0x55fc05752b60 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fafc4005850 tx=0x7fafc4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6da5039322bb7f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.936+0000 7fafb5ffb700 1 -- 172.21.15.44:0/2765291618 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4005410 con 0x55fc05750cc0 2023-09-10T01:22:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.936+0000 7fafb5ffb700 1 -- 172.21.15.44:0/2765291618 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafc400bd00 con 0x55fc05750cc0 2023-09-10T01:22:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.936+0000 7fafb5ffb700 1 -- 172.21.15.44:0/2765291618 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4003c50 con 0x55fc05750cc0 2023-09-10T01:22:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 -- 172.21.15.44:0/2765291618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750cc0 msgr2=0x55fc05752b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/2765291618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750cc0 0x55fc05752b60 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fafc4005850 tx=0x7fafc4009cd0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 -- 172.21.15.44:0/2765291618 shutdown_connections 2023-09-10T01:22:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/2765291618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc057561a0 0x55fc057530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/2765291618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc05750300 0x55fc05761520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/2765291618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750cc0 0x55fc05752b60 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 -- 172.21.15.44:0/2765291618 >> 172.21.15.44:0/2765291618 conn(0x55fc056c5030 msgr2=0x55fc05754220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 -- 172.21.15.44:0/2765291618 shutdown_connections 2023-09-10T01:22:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 -- 172.21.15.44:0/2765291618 wait complete. 2023-09-10T01:22:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 -- start start 2023-09-10T01:22:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 Processor -- start 2023-09-10T01:22:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750300 0x55fc05759a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05750cc0 0x55fc05759fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc057561a0 0x55fc05758620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc05758bf0 con 0x55fc05750300 2023-09-10T01:22:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc05758d60 con 0x55fc057561a0 2023-09-10T01:22:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.937+0000 7fafe16e5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc05758f20 con 0x55fc05750cc0 2023-09-10T01:22:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.938+0000 7fafe16e5b80 10 client.0 _ll_register_callbacks cb 0x7fafc8094b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:22:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.938+0000 7fafceffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750300 0x55fc05759a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.938+0000 7fafceffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750300 0x55fc05759a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58874/0 (socket says 172.21.15.44:58874) 2023-09-10T01:22:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.938+0000 7fafceffd700 1 -- 172.21.15.44:0/3507302911 learned_addr learned my addr 172.21.15.44:0/3507302911 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.938+0000 7fafce7fc700 1 --2- 172.21.15.44:0/3507302911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05750cc0 0x55fc05759fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.938+0000 7fafcf7fe700 1 --2- 172.21.15.44:0/3507302911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc057561a0 0x55fc05758620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.938+0000 7fafceffd700 1 -- 172.21.15.44:0/3507302911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05750cc0 msgr2=0x55fc05759fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.938+0000 7fafceffd700 1 --2- 172.21.15.44:0/3507302911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05750cc0 0x55fc05759fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.938+0000 7fafceffd700 1 -- 172.21.15.44:0/3507302911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc057561a0 msgr2=0x55fc05758620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.938+0000 7fafceffd700 1 --2- 172.21.15.44:0/3507302911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc057561a0 0x55fc05758620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.938+0000 7fafceffd700 1 -- 172.21.15.44:0/3507302911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd010cbc0 con 0x55fc05750300 2023-09-10T01:22:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.938+0000 7fafceffd700 1 --2- 172.21.15.44:0/3507302911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750300 0x55fc05759a90 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fafc0010bf0 tx=0x7fafc00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2228b14c327dca4e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.938+0000 7fafb2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:22:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.938+0000 7fafb2ff5700 1 -- 172.21.15.44:0/3507302911 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc00045b0 con 0x55fc05750300 2023-09-10T01:22:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.938+0000 7fafb2ff5700 1 -- 172.21.15.44:0/3507302911 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(173..173 src has 1..173) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fafc0017040 con 0x55fc05750300 2023-09-10T01:22:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.938+0000 7fafe16e5b80 1 -- 172.21.15.44:0/3507302911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafc401e040 con 0x55fc05750300 2023-09-10T01:22:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.939+0000 7fafb2ff5700 1 -- 172.21.15.44:0/3507302911 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=138) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafc00274d0 con 0x55fc05750300 2023-09-10T01:22:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.939+0000 7fafe16e5b80 10 client.19998 fetch_fsmap learned FSMap version 138 2023-09-10T01:22:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.939+0000 7fafe16e5b80 1 -- 172.21.15.44:0/3507302911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=138,monmap=4+}) v3 -- 0x55fc054b7c00 con 0x55fc05750300 2023-09-10T01:22:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.939+0000 7fafb2ff5700 1 -- 172.21.15.44:0/3507302911 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafc401e040 con 0x55fc05750300 2023-09-10T01:22:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.939+0000 7fafb2ff5700 1 -- 172.21.15.44:0/3507302911 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 138) v1 ==== 7367+0+0 (secure 0 0 0) 0x7fafc0003990 con 0x55fc05750300 2023-09-10T01:22:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.939+0000 7fafe16e5b80 10 client.19998 fetch_fsmap finished waiting for FSMap version 138 2023-09-10T01:22:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.939+0000 7fafe16e5b80 10 client.19998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:22:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.939+0000 7fafe16e5b80 20 client.19998 populate_metadata read hostname 'smithi044' 2023-09-10T01:22:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.939+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/3507302911 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fc057beed0 0x55fc057c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.939+0000 7fafe16e5b80 4 client.19998 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:22:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.939+0000 7fafe16e5b80 1 -- 172.21.15.44:0/3507302911 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fc0569b8c0 con 0x55fc057beed0 2023-09-10T01:22:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.939+0000 7fafce7fc700 1 --2- 172.21.15.44:0/3507302911 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fc057beed0 0x55fc057c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.939+0000 7fafb2ff5700 10 client.19998 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:22:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.939+0000 7fafce7fc700 1 --2- 172.21.15.44:0/3507302911 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fc057beed0 0x55fc057c12f0 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac80dfa0cfb2c5ad server_cookie=537f3d09bf051f23 in_seq=0 out_seq=0 2023-09-10T01:22:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.960+0000 7fafb2ff5700 1 -- 172.21.15.44:0/3507302911 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fc0569b8c0 con 0x55fc057beed0 2023-09-10T01:22:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.960+0000 7fafb2ff5700 10 client.19998 handle_command_reply: tid=0 2023-09-10T01:22:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.960+0000 7fafe16e5b80 1 client.19998 shutdown 2023-09-10T01:22:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.960+0000 7fafe16e5b80 1 -- 172.21.15.44:0/3507302911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750300 msgr2=0x55fc05759a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.960+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/3507302911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750300 0x55fc05759a90 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fafc0010bf0 tx=0x7fafc00047c0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.960+0000 7fafe16e5b80 1 -- 172.21.15.44:0/3507302911 shutdown_connections 2023-09-10T01:22:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.960+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/3507302911 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fc057beed0 0x55fc057c12f0 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.960+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/3507302911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc057561a0 0x55fc05758620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.960+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/3507302911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05750300 0x55fc05759a90 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.960+0000 7fafb2ff5700 0 client.19998 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:22:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.960+0000 7fafe16e5b80 1 --2- 172.21.15.44:0/3507302911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05750cc0 0x55fc05759fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.960+0000 7fafe16e5b80 1 -- 172.21.15.44:0/3507302911 >> 172.21.15.44:0/3507302911 conn(0x55fc056c5030 msgr2=0x55fc057551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.961+0000 7fafe16e5b80 1 -- 172.21.15.44:0/3507302911 shutdown_connections 2023-09-10T01:22:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.961+0000 7fafe16e5b80 1 -- 172.21.15.44:0/3507302911 wait complete. 2023-09-10T01:22:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.961+0000 7fafe16e5b80 20 client.19998 trim_cache size 0 max 16384 2023-09-10T01:22:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:22:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:22:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:22:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:22:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.961+0000 7fafd551f700 1 -- 172.21.15.44:0/637158895 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fafb80a0820 msgr2=0x7fafb80a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.961+0000 7fafd551f700 1 --2- 172.21.15.44:0/637158895 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fafb80a0820 0x7fafb80a2ce0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fafc0007be0 tx=0x7fafc00073d0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.961+0000 7fafd551f700 1 -- 172.21.15.44:0/637158895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafd01055b0 msgr2=0x7fafd010a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.961+0000 7fafd551f700 1 --2- 172.21.15.44:0/637158895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafd01055b0 0x7fafd010a9e0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fafc800ef30 tx=0x7fafc800c3a0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.962+0000 7fafd551f700 1 -- 172.21.15.44:0/637158895 shutdown_connections 2023-09-10T01:22:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.962+0000 7fafd551f700 1 --2- 172.21.15.44:0/637158895 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fafb80a0820 0x7fafb80a2ce0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.962+0000 7fafd551f700 1 --2- 172.21.15.44:0/637158895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafd01055b0 0x7fafd010a9e0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.962+0000 7fafd551f700 1 --2- 172.21.15.44:0/637158895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafd00ff760 0x7fafd01a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.962+0000 7fafd551f700 1 --2- 172.21.15.44:0/637158895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd0100120 0x7fafd01a3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.962+0000 7fafd551f700 1 -- 172.21.15.44:0/637158895 >> 172.21.15.44:0/637158895 conn(0x7fafd0074f30 msgr2=0x7fafd0110180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.965+0000 7fafd551f700 1 -- 172.21.15.44:0/637158895 shutdown_connections 2023-09-10T01:22:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:21.965+0000 7fafd551f700 1 -- 172.21.15.44:0/637158895 wait complete. 2023-09-10T01:22:23.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:23 smithi131 ceph-mon[103713]: pgmap v1350: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 5.3 MiB/s wr, 372 op/s 2023-09-10T01:22:23.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:23 smithi044 ceph-mon[98676]: pgmap v1350: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 5.3 MiB/s wr, 372 op/s 2023-09-10T01:22:23.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:23 smithi099 ceph-mon[102692]: pgmap v1350: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 5.3 MiB/s wr, 372 op/s 2023-09-10T01:22:25.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:25 smithi131 ceph-mon[103713]: pgmap v1351: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 5.3 MiB/s wr, 372 op/s 2023-09-10T01:22:25.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:25 smithi044 ceph-mon[98676]: pgmap v1351: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 5.3 MiB/s wr, 372 op/s 2023-09-10T01:22:25.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:25 smithi099 ceph-mon[102692]: pgmap v1351: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 5.3 MiB/s wr, 372 op/s 2023-09-10T01:22:26.183 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 80.0 s, 207.1 tps, lat 847.164 ms stddev 1454.786 2023-09-10T01:22:27.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:27 smithi131 ceph-mon[103713]: pgmap v1352: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 4.5 MiB/s wr, 364 op/s 2023-09-10T01:22:27.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:27 smithi044 ceph-mon[98676]: pgmap v1352: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 4.5 MiB/s wr, 364 op/s 2023-09-10T01:22:27.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:27 smithi099 ceph-mon[102692]: pgmap v1352: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 4.5 MiB/s wr, 364 op/s 2023-09-10T01:22:29.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:29 smithi131 ceph-mon[103713]: pgmap v1353: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 4.5 MiB/s wr, 362 op/s 2023-09-10T01:22:29.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:29 smithi044 ceph-mon[98676]: pgmap v1353: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 4.5 MiB/s wr, 362 op/s 2023-09-10T01:22:29.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:29 smithi099 ceph-mon[102692]: pgmap v1353: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 4.5 MiB/s wr, 362 op/s 2023-09-10T01:22:31.184 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 85.0 s, 311.1 tps, lat 336.220 ms stddev 193.473 2023-09-10T01:22:31.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:31 smithi131 ceph-mon[103713]: pgmap v1354: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 4.5 MiB/s wr, 361 op/s 2023-09-10T01:22:31.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:31 smithi044 ceph-mon[98676]: pgmap v1354: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 4.5 MiB/s wr, 361 op/s 2023-09-10T01:22:31.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:31 smithi099 ceph-mon[102692]: pgmap v1354: 129 pgs: 129 active+clean; 16 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 4.5 MiB/s wr, 361 op/s 2023-09-10T01:22:33.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:33 smithi131 ceph-mon[103713]: pgmap v1355: 129 pgs: 129 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 6.9 MiB/s wr, 488 op/s 2023-09-10T01:22:33.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:33 smithi044 ceph-mon[98676]: pgmap v1355: 129 pgs: 129 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 6.9 MiB/s wr, 488 op/s 2023-09-10T01:22:33.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:33 smithi099 ceph-mon[102692]: pgmap v1355: 129 pgs: 129 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 6.9 MiB/s wr, 488 op/s 2023-09-10T01:22:35.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.146+0000 7fb1e88bd700 1 -- 172.21.15.44:0/667011709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e41055b0 msgr2=0x7fb1e4105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:35.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.146+0000 7fb1e88bd700 1 --2- 172.21.15.44:0/667011709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e41055b0 0x7fb1e4105990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009990 tx=0x7fb1d4009ca0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.146+0000 7fb1e88bd700 1 -- 172.21.15.44:0/667011709 shutdown_connections 2023-09-10T01:22:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.146+0000 7fb1e88bd700 1 --2- 172.21.15.44:0/667011709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1e4100120 0x7fb1e410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.146+0000 7fb1e88bd700 1 --2- 172.21.15.44:0/667011709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e41055b0 0x7fb1e4105990 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.146+0000 7fb1e88bd700 1 --2- 172.21.15.44:0/667011709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1e40ff760 0x7fb1e40ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.146+0000 7fb1e88bd700 1 -- 172.21.15.44:0/667011709 >> 172.21.15.44:0/667011709 conn(0x7fb1e4074f30 msgr2=0x7fb1e4075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.146+0000 7fb1e88bd700 1 -- 172.21.15.44:0/667011709 shutdown_connections 2023-09-10T01:22:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.146+0000 7fb1e88bd700 1 -- 172.21.15.44:0/667011709 wait complete. 2023-09-10T01:22:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.147+0000 7fb1e88bd700 1 Processor -- start 2023-09-10T01:22:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.147+0000 7fb1e88bd700 1 -- start start 2023-09-10T01:22:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.148+0000 7fb1e88bd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e40ff760 0x7fb1e410bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.148+0000 7fb1e88bd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1e4100120 0x7fb1e4107cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.148+0000 7fb1e88bd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1e41055b0 0x7fb1e4108230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.148+0000 7fb1e88bd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e4108800 con 0x7fb1e41055b0 2023-09-10T01:22:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.148+0000 7fb1e88bd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e4108940 con 0x7fb1e40ff760 2023-09-10T01:22:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.148+0000 7fb1e88bd700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e4108b00 con 0x7fb1e4100120 2023-09-10T01:22:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.148+0000 7fb1dbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1e4100120 0x7fb1e4107cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.148+0000 7fb1dbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1e4100120 0x7fb1e4107cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37542/0 (socket says 172.21.15.44:37542) 2023-09-10T01:22:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.148+0000 7fb1dbfff700 1 -- 172.21.15.44:0/42925578 learned_addr learned my addr 172.21.15.44:0/42925578 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.148+0000 7fb1dbfff700 1 -- 172.21.15.44:0/42925578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e40ff760 msgr2=0x7fb1e410bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.148+0000 7fb1dbfff700 1 --2- 172.21.15.44:0/42925578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e40ff760 0x7fb1e410bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.148+0000 7fb1dbfff700 1 -- 172.21.15.44:0/42925578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1e41055b0 msgr2=0x7fb1e4108230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.148+0000 7fb1dbfff700 1 --2- 172.21.15.44:0/42925578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1e41055b0 0x7fb1e4108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.148+0000 7fb1dbfff700 1 -- 172.21.15.44:0/42925578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d40095f0 con 0x7fb1e4100120 2023-09-10T01:22:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.149+0000 7fb1dbfff700 1 --2- 172.21.15.44:0/42925578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1e4100120 0x7fb1e4107cf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00b530 tx=0x7fb1cc00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.149+0000 7fb1db7fe700 1 -- 172.21.15.44:0/42925578 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc011830 con 0x7fb1e4100120 2023-09-10T01:22:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.149+0000 7fb1db7fe700 1 -- 172.21.15.44:0/42925578 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb1cc011e20 con 0x7fb1e4100120 2023-09-10T01:22:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.149+0000 7fb1db7fe700 1 -- 172.21.15.44:0/42925578 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc010dd0 con 0x7fb1e4100120 2023-09-10T01:22:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.149+0000 7fb1e88bd700 1 -- 172.21.15.44:0/42925578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e406a140 con 0x7fb1e4100120 2023-09-10T01:22:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.149+0000 7fb1e88bd700 1 -- 172.21.15.44:0/42925578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e406a6d0 con 0x7fb1e4100120 2023-09-10T01:22:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.152+0000 7fb1db7fe700 1 -- 172.21.15.44:0/42925578 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fb1cc011990 con 0x7fb1e4100120 2023-09-10T01:22:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.153+0000 7fb1db7fe700 1 --2- 172.21.15.44:0/42925578 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb1d00a0830 0x7fb1d00a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.153+0000 7fb1db7fe700 1 -- 172.21.15.44:0/42925578 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb1cc0cbdb0 con 0x7fb1e4100120 2023-09-10T01:22:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.155+0000 7fb1f4a83b80 1 Processor -- start 2023-09-10T01:22:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.155+0000 7fb1f4a83b80 1 -- start start 2023-09-10T01:22:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.156+0000 7fb1f4a83b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22cb1a0 0x55f3d22cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.156+0000 7fb1f4a83b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22c5300 0x55f3d22c5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.156+0000 7fb1f4a83b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c5cc0 0x55f3d22d1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.156+0000 7fb1f4a83b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d221e6d0 con 0x55f3d22c5cc0 2023-09-10T01:22:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.156+0000 7fb1f4a83b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d221bff0 con 0x55f3d22c5300 2023-09-10T01:22:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.156+0000 7fb1f4a83b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d22178e0 con 0x55f3d22cb1a0 2023-09-10T01:22:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.156+0000 7fb1e259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22cb1a0 0x55f3d22cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.156+0000 7fb1e259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22cb1a0 0x55f3d22cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37546/0 (socket says 172.21.15.44:37546) 2023-09-10T01:22:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.156+0000 7fb1e259c700 1 -- 172.21.15.44:0/641763520 learned_addr learned my addr 172.21.15.44:0/641763520 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.156+0000 7fb1e2d9d700 1 --2- 172.21.15.44:0/641763520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c5cc0 0x55f3d22d1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.156+0000 7fb1e259c700 1 -- 172.21.15.44:0/641763520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22c5300 msgr2=0x55f3d22c5780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:22:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.156+0000 7fb1e259c700 1 --2- 172.21.15.44:0/641763520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22c5300 0x55f3d22c5780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.156+0000 7fb1e259c700 1 -- 172.21.15.44:0/641763520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c5cc0 msgr2=0x55f3d22d1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.156+0000 7fb1e259c700 1 --2- 172.21.15.44:0/641763520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c5cc0 0x55f3d22d1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.156+0000 7fb1e259c700 1 -- 172.21.15.44:0/641763520 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3d2043d80 con 0x55f3d22cb1a0 2023-09-10T01:22:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.156+0000 7fb1e2d9d700 1 --2- 172.21.15.44:0/641763520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c5cc0 0x55f3d22d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:22:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.157+0000 7fb1e259c700 1 --2- 172.21.15.44:0/641763520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22cb1a0 0x55f3d22cb580 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4006010 tx=0x7fb1d4005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a067de3604be8f56 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.157+0000 7fb1d8ff9700 1 -- 172.21.15.44:0/641763520 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400bb30 con 0x55f3d22cb1a0 2023-09-10T01:22:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.157+0000 7fb1e259c700 1 --2- 172.21.15.44:0/42925578 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb1d00a0830 0x7fb1d00a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.157+0000 7fb1d8ff9700 1 -- 172.21.15.44:0/641763520 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb1d400bc90 con 0x55f3d22cb1a0 2023-09-10T01:22:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.157+0000 7fb1d8ff9700 1 -- 172.21.15.44:0/641763520 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4024450 con 0x55f3d22cb1a0 2023-09-10T01:22:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.158+0000 7fb1e259c700 1 --2- 172.21.15.44:0/42925578 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb1d00a0830 0x7fb1d00a2cf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4023000 tx=0x7fb1d40247a0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.164+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/641763520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22cb1a0 msgr2=0x55f3d22cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.164+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/641763520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22cb1a0 0x55f3d22cb580 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4006010 tx=0x7fb1d4005ea0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.164+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/641763520 shutdown_connections 2023-09-10T01:22:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.164+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/641763520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c5cc0 0x55f3d22d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.164+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/641763520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22cb1a0 0x55f3d22cb580 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.164+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/641763520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22c5300 0x55f3d22c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.164+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/641763520 >> 172.21.15.44:0/641763520 conn(0x55f3d223a030 msgr2=0x55f3d22c0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.165+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/641763520 shutdown_connections 2023-09-10T01:22:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.165+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/641763520 wait complete. 2023-09-10T01:22:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.165+0000 7fb1f4a83b80 1 -- start start 2023-09-10T01:22:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.165+0000 7fb1f4a83b80 1 Processor -- start 2023-09-10T01:22:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.166+0000 7fb1f4a83b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22c5300 0x55f3d22d1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.166+0000 7fb1f4a83b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c5cc0 0x55f3d22cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.166+0000 7fb1f4a83b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22cb1a0 0x55f3d22cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.166+0000 7fb1f4a83b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d21b5b10 con 0x55f3d22c5cc0 2023-09-10T01:22:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.166+0000 7fb1f4a83b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d21b5810 con 0x55f3d22c5300 2023-09-10T01:22:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.166+0000 7fb1f4a83b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d20998c0 con 0x55f3d22cb1a0 2023-09-10T01:22:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.166+0000 7fb1f4a83b80 10 client.0 _ll_register_callbacks cb 0x55f3d20ffa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:22:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.166+0000 7fb1e259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22c5300 0x55f3d22d1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.166+0000 7fb1e259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22c5300 0x55f3d22d1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60644/0 (socket says 172.21.15.44:60644) 2023-09-10T01:22:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.166+0000 7fb1e259c700 1 -- 172.21.15.44:0/1905414319 learned_addr learned my addr 172.21.15.44:0/1905414319 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.166+0000 7fb1dbfff700 1 --2- 172.21.15.44:0/1905414319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c5cc0 0x55f3d22cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.167+0000 7fb1e259c700 1 -- 172.21.15.44:0/1905414319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22cb1a0 msgr2=0x55f3d22cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.167+0000 7fb1e259c700 1 --2- 172.21.15.44:0/1905414319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22cb1a0 0x55f3d22cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.167+0000 7fb1e259c700 1 -- 172.21.15.44:0/1905414319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c5cc0 msgr2=0x55f3d22cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.167+0000 7fb1e259c700 1 --2- 172.21.15.44:0/1905414319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c5cc0 0x55f3d22cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.167+0000 7fb1e259c700 1 -- 172.21.15.44:0/1905414319 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1dc00e140 con 0x55f3d22c5300 2023-09-10T01:22:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.167+0000 7fb1e259c700 1 --2- 172.21.15.44:0/1905414319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22c5300 0x55f3d22d1e40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400b650 tx=0x7fb1d4024670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0a5572ecb6b5509 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.168+0000 7fb1c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:22:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.168+0000 7fb1c9ffb700 1 -- 172.21.15.44:0/1905414319 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4028040 con 0x55f3d22c5300 2023-09-10T01:22:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.168+0000 7fb1c9ffb700 1 -- 172.21.15.44:0/1905414319 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(173..173 src has 1..173) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb1d4036070 con 0x55f3d22c5300 2023-09-10T01:22:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.168+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/1905414319 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3d216f490 con 0x55f3d22c5300 2023-09-10T01:22:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.169+0000 7fb1c9ffb700 1 -- 172.21.15.44:0/1905414319 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=138) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1d4024450 con 0x55f3d22c5300 2023-09-10T01:22:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.169+0000 7fb1f4a83b80 10 client.29633 fetch_fsmap learned FSMap version 138 2023-09-10T01:22:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.169+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/1905414319 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=138,monmap=4+}) v3 -- 0x7fb1e410cbc0 con 0x55f3d22c5300 2023-09-10T01:22:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.169+0000 7fb1c9ffb700 1 -- 172.21.15.44:0/1905414319 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f3d216f490 con 0x55f3d22c5300 2023-09-10T01:22:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.170+0000 7fb1c9ffb700 1 -- 172.21.15.44:0/1905414319 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 138) v1 ==== 7367+0+0 (secure 0 0 0) 0x7fb1d4030480 con 0x55f3d22c5300 2023-09-10T01:22:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.170+0000 7fb1f4a83b80 10 client.29633 fetch_fsmap finished waiting for FSMap version 138 2023-09-10T01:22:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.170+0000 7fb1f4a83b80 10 client.29633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:22:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.170+0000 7fb1f4a83b80 20 client.29633 populate_metadata read hostname 'smithi044' 2023-09-10T01:22:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.170+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/1905414319 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f3d22dee40 0x55f3d22e1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.170+0000 7fb1f4a83b80 4 client.29633 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:22:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.170+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/1905414319 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3d222c840 con 0x55f3d22dee40 2023-09-10T01:22:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.170+0000 7fb1dbfff700 1 --2- 172.21.15.44:0/1905414319 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f3d22dee40 0x55f3d22e1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.171+0000 7fb1dbfff700 1 --2- 172.21.15.44:0/1905414319 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f3d22dee40 0x55f3d22e1230 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=761a7b09b0a13be server_cookie=268a0d9e48e07480 in_seq=0 out_seq=0 2023-09-10T01:22:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.171+0000 7fb1c9ffb700 10 client.29633 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:22:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.174+0000 7fb1c9ffb700 1 -- 172.21.15.44:0/1905414319 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f3d222c840 con 0x55f3d22dee40 2023-09-10T01:22:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.174+0000 7fb1c9ffb700 10 client.29633 handle_command_reply: tid=0 2023-09-10T01:22:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.174+0000 7fb1f4a83b80 1 client.29633 shutdown 2023-09-10T01:22:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.174+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/1905414319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22c5300 msgr2=0x55f3d22d1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.174+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/1905414319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22c5300 0x55f3d22d1e40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400b650 tx=0x7fb1d4024670 comp rx=0 tx=0).stop 2023-09-10T01:22:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.174+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/1905414319 shutdown_connections 2023-09-10T01:22:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.174+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/1905414319 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f3d22dee40 0x55f3d22e1230 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.174+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/1905414319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22cb1a0 0x55f3d22cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.174+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/1905414319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22c5300 0x55f3d22d1e40 secure :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400b650 tx=0x7fb1d4024670 comp rx=0 tx=0).stop 2023-09-10T01:22:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.174+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/1905414319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c5cc0 0x55f3d22cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.174+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/1905414319 >> 172.21.15.44:0/1905414319 conn(0x55f3d223a030 msgr2=0x55f3d22bf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.175+0000 7fb1c9ffb700 0 client.29633 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:22:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.176+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/1905414319 shutdown_connections 2023-09-10T01:22:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.176+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/1905414319 wait complete. 2023-09-10T01:22:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.176+0000 7fb1f4a83b80 20 client.29633 trim_cache size 0 max 16384 2023-09-10T01:22:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.203+0000 7fb1f4a83b80 1 Processor -- start 2023-09-10T01:22:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.203+0000 7fb1f4a83b80 1 -- start start 2023-09-10T01:22:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.204+0000 7fb1f4a83b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22c5cc0 0x55f3d22c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.204+0000 7fb1f4a83b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22cb1a0 0x55f3d22bff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.204+0000 7fb1f4a83b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c0460 0x55f3d22d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.204+0000 7fb1f4a83b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d22c6290 con 0x55f3d22c0460 2023-09-10T01:22:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.204+0000 7fb1f4a83b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d22c6430 con 0x55f3d22cb1a0 2023-09-10T01:22:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.204+0000 7fb1f4a83b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d22bcff0 con 0x55f3d22c5cc0 2023-09-10T01:22:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.204+0000 7fb1e259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22c5cc0 0x55f3d22c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.204+0000 7fb1e2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c0460 0x55f3d22d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.204+0000 7fb1e2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c0460 0x55f3d22d19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58910/0 (socket says 172.21.15.44:58910) 2023-09-10T01:22:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.204+0000 7fb1e2d9d700 1 -- 172.21.15.44:0/4072692609 learned_addr learned my addr 172.21.15.44:0/4072692609 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.204+0000 7fb1dbfff700 1 --2- 172.21.15.44:0/4072692609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22cb1a0 0x55f3d22bff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.204+0000 7fb1e2d9d700 1 -- 172.21.15.44:0/4072692609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22c5cc0 msgr2=0x55f3d22c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.204+0000 7fb1e2d9d700 1 --2- 172.21.15.44:0/4072692609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22c5cc0 0x55f3d22c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.204+0000 7fb1e2d9d700 1 -- 172.21.15.44:0/4072692609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22cb1a0 msgr2=0x55f3d22bff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.204+0000 7fb1e2d9d700 1 --2- 172.21.15.44:0/4072692609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22cb1a0 0x55f3d22bff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.204+0000 7fb1e2d9d700 1 -- 172.21.15.44:0/4072692609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3d202cc00 con 0x55f3d22c0460 2023-09-10T01:22:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.205+0000 7fb1e2d9d700 1 --2- 172.21.15.44:0/4072692609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c0460 0x55f3d22d19e0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc003d50 tx=0x7fb1dc004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899f60c5569920a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.205+0000 7fb1d97fa700 1 -- 172.21.15.44:0/4072692609 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc015450 con 0x55f3d22c0460 2023-09-10T01:22:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.205+0000 7fb1d97fa700 1 -- 172.21.15.44:0/4072692609 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb1dc00f040 con 0x55f3d22c0460 2023-09-10T01:22:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.205+0000 7fb1d97fa700 1 -- 172.21.15.44:0/4072692609 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc007840 con 0x55f3d22c0460 2023-09-10T01:22:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.213+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/4072692609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c0460 msgr2=0x55f3d22d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.213+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/4072692609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c0460 0x55f3d22d19e0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc003d50 tx=0x7fb1dc004320 comp rx=0 tx=0).stop 2023-09-10T01:22:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.214+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/4072692609 shutdown_connections 2023-09-10T01:22:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.214+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/4072692609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22c0460 0x55f3d22d19e0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.214+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/4072692609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22c5cc0 0x55f3d22c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.214+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/4072692609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22cb1a0 0x55f3d22bff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.214+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/4072692609 >> 172.21.15.44:0/4072692609 conn(0x55f3d223a030 msgr2=0x55f3d22c42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.222+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/4072692609 shutdown_connections 2023-09-10T01:22:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.222+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/4072692609 wait complete. 2023-09-10T01:22:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.223+0000 7fb1f4a83b80 1 -- start start 2023-09-10T01:22:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.223+0000 7fb1f4a83b80 1 Processor -- start 2023-09-10T01:22:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.223+0000 7fb1f4a83b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22c5cc0 0x55f3d22d4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.223+0000 7fb1f4a83b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22cb1a0 0x55f3d22d4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.223+0000 7fb1f4a83b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22d5490 0x55f3d2332c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.223+0000 7fb1f4a83b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d22d6880 con 0x55f3d22cb1a0 2023-09-10T01:22:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.223+0000 7fb1f4a83b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d22d69c0 con 0x55f3d22d5490 2023-09-10T01:22:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.223+0000 7fb1f4a83b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d23331d0 con 0x55f3d22c5cc0 2023-09-10T01:22:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.223+0000 7fb1f4a83b80 10 client.0 _ll_register_callbacks cb 0x7fb1dc009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:22:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.224+0000 7fb1e259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22c5cc0 0x55f3d22d4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.224+0000 7fb1e259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22c5cc0 0x55f3d22d4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37564/0 (socket says 172.21.15.44:37564) 2023-09-10T01:22:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.224+0000 7fb1e259c700 1 -- 172.21.15.44:0/958424040 learned_addr learned my addr 172.21.15.44:0/958424040 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.224+0000 7fb1dbfff700 1 --2- 172.21.15.44:0/958424040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22cb1a0 0x55f3d22d4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.224+0000 7fb1e259c700 1 -- 172.21.15.44:0/958424040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22d5490 msgr2=0x55f3d2332c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.224+0000 7fb1e259c700 1 --2- 172.21.15.44:0/958424040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22d5490 0x55f3d2332c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.224+0000 7fb1e259c700 1 -- 172.21.15.44:0/958424040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22cb1a0 msgr2=0x55f3d22d4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.224+0000 7fb1e259c700 1 --2- 172.21.15.44:0/958424040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22cb1a0 0x55f3d22d4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.224+0000 7fb1e259c700 1 -- 172.21.15.44:0/958424040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1dc003a00 con 0x55f3d22c5cc0 2023-09-10T01:22:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.224+0000 7fb1e259c700 1 --2- 172.21.15.44:0/958424040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22c5cc0 0x55f3d22d4a10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4024d40 tx=0x7fb1d4030730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c07de96212db969 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.227+0000 7fb1c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:22:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.227+0000 7fb1c9ffb700 1 -- 172.21.15.44:0/958424040 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4009290 con 0x55f3d22c5cc0 2023-09-10T01:22:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.227+0000 7fb1c9ffb700 1 -- 172.21.15.44:0/958424040 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(173..173 src has 1..173) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb1d4036040 con 0x55f3d22c5cc0 2023-09-10T01:22:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.227+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/958424040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1d400bc90 con 0x55f3d22c5cc0 2023-09-10T01:22:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.230+0000 7fb1c9ffb700 1 -- 172.21.15.44:0/958424040 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=138) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1d4024450 con 0x55f3d22c5cc0 2023-09-10T01:22:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.230+0000 7fb1f4a83b80 10 client.29623 fetch_fsmap learned FSMap version 138 2023-09-10T01:22:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.230+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/958424040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=138,monmap=4+}) v3 -- 0x55f3d206a1f0 con 0x55f3d22c5cc0 2023-09-10T01:22:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.230+0000 7fb1c9ffb700 1 -- 172.21.15.44:0/958424040 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb1d400bc90 con 0x55f3d22c5cc0 2023-09-10T01:22:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.236+0000 7fb1c9ffb700 1 -- 172.21.15.44:0/958424040 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 138) v1 ==== 7367+0+0 (secure 0 0 0) 0x7fb1d4030ac0 con 0x55f3d22c5cc0 2023-09-10T01:22:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.236+0000 7fb1f4a83b80 10 client.29623 fetch_fsmap finished waiting for FSMap version 138 2023-09-10T01:22:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.236+0000 7fb1f4a83b80 10 client.29623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:22:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.236+0000 7fb1f4a83b80 20 client.29623 populate_metadata read hostname 'smithi044' 2023-09-10T01:22:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.236+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/958424040 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f3d2334460 0x55f3d2336870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.236+0000 7fb1f4a83b80 4 client.29623 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:22:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.236+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/958424040 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f3d22108c0 con 0x55f3d2334460 2023-09-10T01:22:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.237+0000 7fb1dbfff700 1 --2- 172.21.15.44:0/958424040 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f3d2334460 0x55f3d2336870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.237+0000 7fb1c9ffb700 10 client.29623 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:22:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.237+0000 7fb1dbfff700 1 --2- 172.21.15.44:0/958424040 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f3d2334460 0x55f3d2336870 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8db7e20c10129f1 server_cookie=e5f4376170f2a4b2 in_seq=0 out_seq=0 2023-09-10T01:22:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.268+0000 7fb1c9ffb700 1 -- 172.21.15.44:0/958424040 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f3d22108c0 con 0x55f3d2334460 2023-09-10T01:22:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.268+0000 7fb1c9ffb700 10 client.29623 handle_command_reply: tid=0 2023-09-10T01:22:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.268+0000 7fb1f4a83b80 1 client.29623 shutdown 2023-09-10T01:22:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.269+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/958424040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22c5cc0 msgr2=0x55f3d22d4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.269+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/958424040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22c5cc0 0x55f3d22d4a10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4024d40 tx=0x7fb1d4030730 comp rx=0 tx=0).stop 2023-09-10T01:22:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.273+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/958424040 shutdown_connections 2023-09-10T01:22:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.273+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/958424040 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f3d2334460 0x55f3d2336870 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.273+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/958424040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3d22d5490 0x55f3d2332c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.273+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/958424040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f3d22c5cc0 0x55f3d22d4a10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.273+0000 7fb1f4a83b80 1 --2- 172.21.15.44:0/958424040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3d22cb1a0 0x55f3d22d4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.273+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/958424040 >> 172.21.15.44:0/958424040 conn(0x55f3d223a030 msgr2=0x55f3d22c42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.273+0000 7fb1c9ffb700 0 client.29623 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:22:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.274+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/958424040 shutdown_connections 2023-09-10T01:22:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.276+0000 7fb1f4a83b80 1 -- 172.21.15.44:0/958424040 wait complete. 2023-09-10T01:22:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.282+0000 7fb1f4a83b80 20 client.29623 trim_cache size 0 max 16384 2023-09-10T01:22:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:22:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:22:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:22:35.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:22:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.286+0000 7fb1e88bd700 1 -- 172.21.15.44:0/42925578 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb1d00a0830 msgr2=0x7fb1d00a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.286+0000 7fb1e88bd700 1 --2- 172.21.15.44:0/42925578 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb1d00a0830 0x7fb1d00a2cf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4023000 tx=0x7fb1d40247a0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.287+0000 7fb1e88bd700 1 -- 172.21.15.44:0/42925578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1e4100120 msgr2=0x7fb1e4107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.287+0000 7fb1e88bd700 1 --2- 172.21.15.44:0/42925578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1e4100120 0x7fb1e4107cf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00b530 tx=0x7fb1cc00b8f0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.287+0000 7fb1e88bd700 1 -- 172.21.15.44:0/42925578 shutdown_connections 2023-09-10T01:22:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.287+0000 7fb1e88bd700 1 --2- 172.21.15.44:0/42925578 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb1d00a0830 0x7fb1d00a2cf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.287+0000 7fb1e88bd700 1 --2- 172.21.15.44:0/42925578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1e41055b0 0x7fb1e4108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.287+0000 7fb1e88bd700 1 --2- 172.21.15.44:0/42925578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e40ff760 0x7fb1e410bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.287+0000 7fb1e88bd700 1 --2- 172.21.15.44:0/42925578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1e4100120 0x7fb1e4107cf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.287+0000 7fb1e88bd700 1 -- 172.21.15.44:0/42925578 >> 172.21.15.44:0/42925578 conn(0x7fb1e4074f30 msgr2=0x7fb1e4100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.289+0000 7fb1e88bd700 1 -- 172.21.15.44:0/42925578 shutdown_connections 2023-09-10T01:22:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:35.291+0000 7fb1e88bd700 1 -- 172.21.15.44:0/42925578 wait complete. 2023-09-10T01:22:35.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:35 smithi131 ceph-mon[103713]: pgmap v1356: 129 pgs: 129 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 6.1 MiB/s wr, 433 op/s 2023-09-10T01:22:35.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:35 smithi044 ceph-mon[98676]: pgmap v1356: 129 pgs: 129 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 6.1 MiB/s wr, 433 op/s 2023-09-10T01:22:35.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:35 smithi099 ceph-mon[102692]: pgmap v1356: 129 pgs: 129 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 6.1 MiB/s wr, 433 op/s 2023-09-10T01:22:36.185 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 90.0 s, 260.2 tps, lat 382.055 ms stddev 230.631 2023-09-10T01:22:37.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:37 smithi131 ceph-mon[103713]: pgmap v1357: 129 pgs: 129 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 9.1 MiB/s wr, 671 op/s 2023-09-10T01:22:37.560 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:37 smithi044 ceph-mon[98676]: pgmap v1357: 129 pgs: 129 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 9.1 MiB/s wr, 671 op/s 2023-09-10T01:22:37.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:37 smithi099 ceph-mon[102692]: pgmap v1357: 129 pgs: 129 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 9.1 MiB/s wr, 671 op/s 2023-09-10T01:22:38.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:38 smithi131 ceph-mon[103713]: pgmap v1358: 129 pgs: 129 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 7.7 MiB/s wr, 541 op/s 2023-09-10T01:22:38.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:38 smithi044 ceph-mon[98676]: pgmap v1358: 129 pgs: 129 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 7.7 MiB/s wr, 541 op/s 2023-09-10T01:22:38.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:38 smithi099 ceph-mon[102692]: pgmap v1358: 129 pgs: 129 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 7.7 MiB/s wr, 541 op/s 2023-09-10T01:22:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:40 smithi099 ceph-mon[102692]: pgmap v1359: 129 pgs: 129 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 7.6 MiB/s wr, 523 op/s 2023-09-10T01:22:41.186 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 95.0 s, 382.1 tps, lat 256.150 ms stddev 295.598 2023-09-10T01:22:41.299 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:40 smithi131 ceph-mon[103713]: pgmap v1359: 129 pgs: 129 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 7.6 MiB/s wr, 523 op/s 2023-09-10T01:22:41.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:40 smithi044 ceph-mon[98676]: pgmap v1359: 129 pgs: 129 active+clean; 17 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 7.6 MiB/s wr, 523 op/s 2023-09-10T01:22:43.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:43 smithi131 ceph-mon[103713]: pgmap v1360: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 12 MiB/s wr, 855 op/s 2023-09-10T01:22:43.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:43 smithi044 ceph-mon[98676]: pgmap v1360: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 12 MiB/s wr, 855 op/s 2023-09-10T01:22:43.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:43 smithi099 ceph-mon[102692]: pgmap v1360: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 12 MiB/s wr, 855 op/s 2023-09-10T01:22:45.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:45 smithi131 ceph-mon[103713]: pgmap v1361: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 7.9 MiB/s wr, 598 op/s 2023-09-10T01:22:45.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:45 smithi044 ceph-mon[98676]: pgmap v1361: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 7.9 MiB/s wr, 598 op/s 2023-09-10T01:22:45.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:45 smithi099 ceph-mon[102692]: pgmap v1361: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 7.9 MiB/s wr, 598 op/s 2023-09-10T01:22:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 100.0 s, 230.2 tps, lat 383.421 ms stddev 597.487 2023-09-10T01:22:47.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:47 smithi131 ceph-mon[103713]: pgmap v1362: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 13 MiB/s wr, 874 op/s 2023-09-10T01:22:47.560 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:47 smithi044 ceph-mon[98676]: pgmap v1362: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 13 MiB/s wr, 874 op/s 2023-09-10T01:22:47.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:47 smithi099 ceph-mon[102692]: pgmap v1362: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 13 MiB/s wr, 874 op/s 2023-09-10T01:22:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.509+0000 7f6aa6e48700 1 -- 172.21.15.44:0/836429410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa01054b0 msgr2=0x7f6aa00679f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.509+0000 7f6aa6e48700 1 --2- 172.21.15.44:0/836429410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa01054b0 0x7f6aa00679f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009990 tx=0x7f6a94009ca0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.513+0000 7f6aa6e48700 1 -- 172.21.15.44:0/836429410 shutdown_connections 2023-09-10T01:22:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.513+0000 7f6aa6e48700 1 --2- 172.21.15.44:0/836429410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aa0068610 0x7f6aa0103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.513+0000 7f6aa6e48700 1 --2- 172.21.15.44:0/836429410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa01054b0 0x7f6aa00679f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.513+0000 7f6aa6e48700 1 --2- 172.21.15.44:0/836429410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6aa0105920 0x7f6aa0067f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.513+0000 7f6aa6e48700 1 -- 172.21.15.44:0/836429410 >> 172.21.15.44:0/836429410 conn(0x7f6aa0074f30 msgr2=0x7f6aa0075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.514+0000 7f6aa6e48700 1 -- 172.21.15.44:0/836429410 shutdown_connections 2023-09-10T01:22:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.514+0000 7f6aa6e48700 1 -- 172.21.15.44:0/836429410 wait complete. 2023-09-10T01:22:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.514+0000 7f6aa6e48700 1 Processor -- start 2023-09-10T01:22:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.515+0000 7f6aa6e48700 1 -- start start 2023-09-10T01:22:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.515+0000 7f6aa6e48700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa01054b0 0x7f6aa01a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.515+0000 7f6aa6e48700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6aa0105920 0x7f6aa01a3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.515+0000 7f6aa6e48700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aa019cb40 0x7f6aa019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.515+0000 7f6aa6e48700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa019d5c0 con 0x7f6aa019cb40 2023-09-10T01:22:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.515+0000 7f6aa6e48700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa019d700 con 0x7f6aa01054b0 2023-09-10T01:22:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.515+0000 7f6aa6e48700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa019d8c0 con 0x7f6aa0105920 2023-09-10T01:22:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.515+0000 7f6aa53e5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aa019cb40 0x7f6aa019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.515+0000 7f6aa53e5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aa019cb40 0x7f6aa019cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58932/0 (socket says 172.21.15.44:58932) 2023-09-10T01:22:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.515+0000 7f6aa53e5700 1 -- 172.21.15.44:0/2176651543 learned_addr learned my addr 172.21.15.44:0/2176651543 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.515+0000 7f6aa53e5700 1 -- 172.21.15.44:0/2176651543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6aa0105920 msgr2=0x7f6aa01a3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.515+0000 7f6a9ffff700 1 --2- 172.21.15.44:0/2176651543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6aa0105920 0x7f6aa01a3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.515+0000 7f6aa53e5700 1 --2- 172.21.15.44:0/2176651543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6aa0105920 0x7f6aa01a3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.515+0000 7f6aa53e5700 1 -- 172.21.15.44:0/2176651543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa01054b0 msgr2=0x7f6aa01a2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.516+0000 7f6aa53e5700 1 --2- 172.21.15.44:0/2176651543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa01054b0 0x7f6aa01a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.516+0000 7f6aa53e5700 1 -- 172.21.15.44:0/2176651543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a940095f0 con 0x7f6aa019cb40 2023-09-10T01:22:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.516+0000 7f6a9ffff700 1 --2- 172.21.15.44:0/2176651543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6aa0105920 0x7f6aa01a3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:22:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.516+0000 7f6aa53e5700 1 --2- 172.21.15.44:0/2176651543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aa019cb40 0x7f6aa019cff0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800ef30 tx=0x7f6a9800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.516+0000 7f6a9dffb700 1 -- 172.21.15.44:0/2176651543 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98018070 con 0x7f6aa019cb40 2023-09-10T01:22:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.516+0000 7f6aa6e48700 1 -- 172.21.15.44:0/2176651543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa01a9b50 con 0x7f6aa019cb40 2023-09-10T01:22:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.516+0000 7f6a9dffb700 1 -- 172.21.15.44:0/2176651543 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a9800f040 con 0x7f6aa019cb40 2023-09-10T01:22:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.516+0000 7f6a9dffb700 1 -- 172.21.15.44:0/2176651543 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98003d00 con 0x7f6aa019cb40 2023-09-10T01:22:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.516+0000 7f6aa6e48700 1 -- 172.21.15.44:0/2176651543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa01aa060 con 0x7f6aa019cb40 2023-09-10T01:22:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.522+0000 7f6ab300eb80 1 Processor -- start 2023-09-10T01:22:48.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.522+0000 7f6ab300eb80 1 -- start start 2023-09-10T01:22:48.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.522+0000 7f6ab300eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 0x55e26e5ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.522+0000 7f6ab300eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e26e5c4300 0x55e26e5c4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.522+0000 7f6ab300eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 0x55e26e5d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.522+0000 7f6ab300eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26e51d6d0 con 0x55e26e5c4300 2023-09-10T01:22:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.522+0000 7f6ab300eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26e51aff0 con 0x55e26e5c4cc0 2023-09-10T01:22:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.522+0000 7f6ab300eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26e5168e0 con 0x55e26e5ca1a0 2023-09-10T01:22:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.523+0000 7f6a9ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 0x55e26e5d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.523+0000 7f6a9ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 0x55e26e5d0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60676/0 (socket says 172.21.15.44:60676) 2023-09-10T01:22:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.523+0000 7f6a9ffff700 1 -- 172.21.15.44:0/1256272117 learned_addr learned my addr 172.21.15.44:0/1256272117 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.523+0000 7f6a9ffff700 1 --2- 172.21.15.44:0/1256272117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 0x55e26e5ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.523+0000 7f6a9ffff700 1 -- 172.21.15.44:0/1256272117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 msgr2=0x55e26e5ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.523+0000 7f6a9ffff700 1 --2- 172.21.15.44:0/1256272117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 0x55e26e5ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.523+0000 7f6a9ffff700 1 -- 172.21.15.44:0/1256272117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e26e5c4300 msgr2=0x55e26e5c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.523+0000 7f6a9ffff700 1 --2- 172.21.15.44:0/1256272117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e26e5c4300 0x55e26e5c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.523+0000 7f6a9ffff700 1 -- 172.21.15.44:0/1256272117 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e26e342d80 con 0x55e26e5c4cc0 2023-09-10T01:22:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.524+0000 7f6a9ffff700 1 --2- 172.21.15.44:0/1256272117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 0x55e26e5d0f30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c015640 tx=0x7f6a8c00c250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=927c1c265a4a386e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.524+0000 7f6a8affd700 1 -- 172.21.15.44:0/1256272117 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c015e60 con 0x55e26e5c4cc0 2023-09-10T01:22:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.524+0000 7f6a8affd700 1 -- 172.21.15.44:0/1256272117 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a8c0178a0 con 0x55e26e5c4cc0 2023-09-10T01:22:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.524+0000 7f6a8affd700 1 -- 172.21.15.44:0/1256272117 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c0166d0 con 0x55e26e5c4cc0 2023-09-10T01:22:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.525+0000 7f6ab300eb80 1 -- 172.21.15.44:0/1256272117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 msgr2=0x55e26e5d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.525+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/1256272117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 0x55e26e5d0f30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c015640 tx=0x7f6a8c00c250 comp rx=0 tx=0).stop 2023-09-10T01:22:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.528+0000 7f6a9dffb700 1 -- 172.21.15.44:0/2176651543 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f6a9800caa0 con 0x7f6aa019cb40 2023-09-10T01:22:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.528+0000 7f6ab300eb80 1 -- 172.21.15.44:0/1256272117 shutdown_connections 2023-09-10T01:22:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.528+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/1256272117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 0x55e26e5d0f30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.528+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/1256272117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 0x55e26e5ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.528+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/1256272117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e26e5c4300 0x55e26e5c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.528+0000 7f6ab300eb80 1 -- 172.21.15.44:0/1256272117 >> 172.21.15.44:0/1256272117 conn(0x55e26e539030 msgr2=0x55e26e5bf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.528+0000 7f6a9dffb700 1 --2- 172.21.15.44:0/2176651543 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6a900a07b0 0x7f6a900a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.529+0000 7f6a9dffb700 1 -- 172.21.15.44:0/2176651543 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f6a980cc0f0 con 0x7f6aa019cb40 2023-09-10T01:22:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.529+0000 7f6aa4be4700 1 --2- 172.21.15.44:0/2176651543 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6a900a07b0 0x7f6a900a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.529+0000 7f6aa4be4700 1 --2- 172.21.15.44:0/2176651543 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6a900a07b0 0x7f6a900a2c70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009350 tx=0x7f6a9400b260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.530+0000 7f6ab300eb80 1 -- 172.21.15.44:0/1256272117 shutdown_connections 2023-09-10T01:22:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.530+0000 7f6ab300eb80 1 -- 172.21.15.44:0/1256272117 wait complete. 2023-09-10T01:22:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.531+0000 7f6ab300eb80 1 -- start start 2023-09-10T01:22:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.531+0000 7f6ab300eb80 1 Processor -- start 2023-09-10T01:22:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.531+0000 7f6ab300eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4300 0x55e26e5d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.531+0000 7f6ab300eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5c4cc0 0x55e26e5ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.531+0000 7f6ab300eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e26e5ca1a0 0x55e26e5ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.531+0000 7f6ab300eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26e4b4b10 con 0x55e26e5ca1a0 2023-09-10T01:22:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.531+0000 7f6ab300eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26e4b4810 con 0x55e26e5c4300 2023-09-10T01:22:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.531+0000 7f6ab300eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26e3988c0 con 0x55e26e5c4cc0 2023-09-10T01:22:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.531+0000 7f6ab300eb80 10 client.0 _ll_register_callbacks cb 0x55e26e3fea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:22:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.532+0000 7f6a9ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5c4cc0 0x55e26e5ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.532+0000 7f6a9ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5c4cc0 0x55e26e5ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37594/0 (socket says 172.21.15.44:37594) 2023-09-10T01:22:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.532+0000 7f6a9ffff700 1 -- 172.21.15.44:0/326303631 learned_addr learned my addr 172.21.15.44:0/326303631 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.532+0000 7f6a9ffff700 1 -- 172.21.15.44:0/326303631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4300 msgr2=0x55e26e5d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.532+0000 7f6a9ffff700 1 --2- 172.21.15.44:0/326303631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4300 0x55e26e5d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.532+0000 7f6a9ffff700 1 -- 172.21.15.44:0/326303631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e26e5ca1a0 msgr2=0x55e26e5ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.532+0000 7f6a9ffff700 1 --2- 172.21.15.44:0/326303631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e26e5ca1a0 0x55e26e5ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.532+0000 7f6a9ffff700 1 -- 172.21.15.44:0/326303631 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8c015320 con 0x55e26e5c4cc0 2023-09-10T01:22:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.532+0000 7f6a9ffff700 1 --2- 172.21.15.44:0/326303631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5c4cc0 0x55e26e5ce190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c006cc0 tx=0x7f6a8c017ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2524ec11a179daf5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.532+0000 7f6a7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:22:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.533+0000 7f6a7ffff700 1 -- 172.21.15.44:0/326303631 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c02b400 con 0x55e26e5c4cc0 2023-09-10T01:22:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.533+0000 7f6a7ffff700 1 -- 172.21.15.44:0/326303631 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(173..173 src has 1..173) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f6a8c000e40 con 0x55e26e5c4cc0 2023-09-10T01:22:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.535+0000 7f6ab300eb80 1 -- 172.21.15.44:0/326303631 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e26e46e490 con 0x55e26e5c4cc0 2023-09-10T01:22:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.535+0000 7f6a7ffff700 1 -- 172.21.15.44:0/326303631 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=138) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a8c02b560 con 0x55e26e5c4cc0 2023-09-10T01:22:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.535+0000 7f6ab300eb80 10 client.29629 fetch_fsmap learned FSMap version 138 2023-09-10T01:22:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.535+0000 7f6ab300eb80 1 -- 172.21.15.44:0/326303631 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=138,monmap=4+}) v3 -- 0x55e26e32bc00 con 0x55e26e5c4cc0 2023-09-10T01:22:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.536+0000 7f6a7ffff700 1 -- 172.21.15.44:0/326303631 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e26e46e490 con 0x55e26e5c4cc0 2023-09-10T01:22:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.536+0000 7f6a7ffff700 1 -- 172.21.15.44:0/326303631 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 138) v1 ==== 7367+0+0 (secure 0 0 0) 0x7f6a8c029030 con 0x55e26e5c4cc0 2023-09-10T01:22:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.536+0000 7f6ab300eb80 10 client.29629 fetch_fsmap finished waiting for FSMap version 138 2023-09-10T01:22:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.536+0000 7f6ab300eb80 10 client.29629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:22:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.536+0000 7f6ab300eb80 20 client.29629 populate_metadata read hostname 'smithi044' 2023-09-10T01:22:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.536+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/326303631 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e26e5ef340 0x55e26e5f1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.536+0000 7f6ab300eb80 4 client.29629 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:22:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.536+0000 7f6ab300eb80 1 -- 172.21.15.44:0/326303631 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e26e52b840 con 0x55e26e5ef340 2023-09-10T01:22:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.538+0000 7f6aa4be4700 1 --2- 172.21.15.44:0/326303631 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e26e5ef340 0x55e26e5f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.539+0000 7f6aa4be4700 1 --2- 172.21.15.44:0/326303631 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e26e5ef340 0x55e26e5f1760 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91651d9545606e66 server_cookie=e99e4d8389707448 in_seq=0 out_seq=0 2023-09-10T01:22:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.539+0000 7f6a7ffff700 10 client.29629 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:22:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.541+0000 7f6a7ffff700 1 -- 172.21.15.44:0/326303631 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e26e52b840 con 0x55e26e5ef340 2023-09-10T01:22:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.541+0000 7f6a7ffff700 10 client.29629 handle_command_reply: tid=0 2023-09-10T01:22:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.541+0000 7f6ab300eb80 1 client.29629 shutdown 2023-09-10T01:22:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.545+0000 7f6ab300eb80 1 -- 172.21.15.44:0/326303631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5c4cc0 msgr2=0x55e26e5ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.545+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/326303631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5c4cc0 0x55e26e5ce190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c006cc0 tx=0x7f6a8c017ea0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.551+0000 7f6ab300eb80 1 -- 172.21.15.44:0/326303631 shutdown_connections 2023-09-10T01:22:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.551+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/326303631 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e26e5ef340 0x55e26e5f1760 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.551+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/326303631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e26e5ca1a0 0x55e26e5ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.551+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/326303631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4300 0x55e26e5d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.551+0000 7f6a7ffff700 0 client.29629 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:22:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.551+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/326303631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5c4cc0 0x55e26e5ce190 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.551+0000 7f6ab300eb80 1 -- 172.21.15.44:0/326303631 >> 172.21.15.44:0/326303631 conn(0x55e26e539030 msgr2=0x55e26e5be7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.551+0000 7f6ab300eb80 1 -- 172.21.15.44:0/326303631 shutdown_connections 2023-09-10T01:22:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.551+0000 7f6ab300eb80 1 -- 172.21.15.44:0/326303631 wait complete. 2023-09-10T01:22:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.552+0000 7f6ab300eb80 20 client.29629 trim_cache size 0 max 16384 2023-09-10T01:22:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.580+0000 7f6ab300eb80 1 Processor -- start 2023-09-10T01:22:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.580+0000 7f6ab300eb80 1 -- start start 2023-09-10T01:22:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.580+0000 7f6ab300eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e26e5c4300 0x55e26e5cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.580+0000 7f6ab300eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 0x55e26e5c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.580+0000 7f6ab300eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 0x55e26e5eb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.580+0000 7f6ab300eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26e5c9e30 con 0x55e26e5c4300 2023-09-10T01:22:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.580+0000 7f6ab300eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26e5cc740 con 0x55e26e5c4cc0 2023-09-10T01:22:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.580+0000 7f6ab300eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26e5c8e60 con 0x55e26e5ca1a0 2023-09-10T01:22:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.581+0000 7f6aa53e5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 0x55e26e5eb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.581+0000 7f6aa53e5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 0x55e26e5eb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37600/0 (socket says 172.21.15.44:37600) 2023-09-10T01:22:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.581+0000 7f6aa53e5700 1 -- 172.21.15.44:0/3177328576 learned_addr learned my addr 172.21.15.44:0/3177328576 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.582+0000 7f6aa53e5700 1 -- 172.21.15.44:0/3177328576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 msgr2=0x55e26e5c46e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:22:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.582+0000 7f6aa53e5700 1 --2- 172.21.15.44:0/3177328576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 0x55e26e5c46e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.582+0000 7f6aa53e5700 1 -- 172.21.15.44:0/3177328576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e26e5c4300 msgr2=0x55e26e5cbe20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:22:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.582+0000 7f6aa53e5700 1 --2- 172.21.15.44:0/3177328576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e26e5c4300 0x55e26e5cbe20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.582+0000 7f6aa53e5700 1 -- 172.21.15.44:0/3177328576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e26e3691f0 con 0x55e26e5ca1a0 2023-09-10T01:22:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.582+0000 7f6aa53e5700 1 --2- 172.21.15.44:0/3177328576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 0x55e26e5eb6e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800ef00 tx=0x7f6a98098b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82ac6417fd0f8885 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.584+0000 7f6a8b7fe700 1 -- 172.21.15.44:0/3177328576 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98099230 con 0x55e26e5ca1a0 2023-09-10T01:22:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.584+0000 7f6a8b7fe700 1 -- 172.21.15.44:0/3177328576 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a9809d060 con 0x55e26e5ca1a0 2023-09-10T01:22:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.584+0000 7f6a8b7fe700 1 -- 172.21.15.44:0/3177328576 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9801b580 con 0x55e26e5ca1a0 2023-09-10T01:22:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.585+0000 7f6ab300eb80 1 -- 172.21.15.44:0/3177328576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 msgr2=0x55e26e5eb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.585+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/3177328576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 0x55e26e5eb6e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800ef00 tx=0x7f6a98098b90 comp rx=0 tx=0).stop 2023-09-10T01:22:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.591+0000 7f6ab300eb80 1 -- 172.21.15.44:0/3177328576 shutdown_connections 2023-09-10T01:22:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.591+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/3177328576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 0x55e26e5eb6e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.591+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/3177328576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e26e5c4300 0x55e26e5cbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.591+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/3177328576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 0x55e26e5c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.591+0000 7f6ab300eb80 1 -- 172.21.15.44:0/3177328576 >> 172.21.15.44:0/3177328576 conn(0x55e26e539030 msgr2=0x55e26e5f6100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.596+0000 7f6ab300eb80 1 -- 172.21.15.44:0/3177328576 shutdown_connections 2023-09-10T01:22:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.600+0000 7f6ab300eb80 1 -- 172.21.15.44:0/3177328576 wait complete. 2023-09-10T01:22:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.600+0000 7f6ab300eb80 1 -- start start 2023-09-10T01:22:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.600+0000 7f6ab300eb80 1 Processor -- start 2023-09-10T01:22:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.604+0000 7f6ab300eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e26e5c4300 0x55e26e5e4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.604+0000 7f6ab300eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 0x55e26e5e4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.604+0000 7f6ab300eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 0x55e26e6421b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.604+0000 7f6ab300eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26e6426f0 con 0x55e26e5c4300 2023-09-10T01:22:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.604+0000 7f6ab300eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26e642830 con 0x55e26e5c4cc0 2023-09-10T01:22:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.604+0000 7f6ab300eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26e642970 con 0x55e26e5ca1a0 2023-09-10T01:22:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.604+0000 7f6ab300eb80 10 client.0 _ll_register_callbacks cb 0x7f6a8c00cc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:22:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.604+0000 7f6aa53e5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 0x55e26e6421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.604+0000 7f6aa53e5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 0x55e26e6421b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37602/0 (socket says 172.21.15.44:37602) 2023-09-10T01:22:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.604+0000 7f6aa53e5700 1 -- 172.21.15.44:0/2836186491 learned_addr learned my addr 172.21.15.44:0/2836186491 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:22:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.607+0000 7f6a9ffff700 1 --2- 172.21.15.44:0/2836186491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 0x55e26e5e4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.607+0000 7f6aa53e5700 1 -- 172.21.15.44:0/2836186491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 msgr2=0x55e26e5e4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:22:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.607+0000 7f6aa53e5700 1 --2- 172.21.15.44:0/2836186491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 0x55e26e5e4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.607+0000 7f6aa53e5700 1 -- 172.21.15.44:0/2836186491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e26e5c4300 msgr2=0x55e26e5e4680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:22:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.607+0000 7f6aa53e5700 1 --2- 172.21.15.44:0/2836186491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e26e5c4300 0x55e26e5e4680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.607+0000 7f6aa53e5700 1 -- 172.21.15.44:0/2836186491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa01aa060 con 0x55e26e5ca1a0 2023-09-10T01:22:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.607+0000 7f6a9ffff700 1 --2- 172.21.15.44:0/2836186491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 0x55e26e5e4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:22:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.607+0000 7f6aa53e5700 1 --2- 172.21.15.44:0/2836186491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 0x55e26e6421b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6a98004d70 tx=0x7f6a98009420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b2f55cd94989bae server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:22:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.608+0000 7f6a7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:22:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.616+0000 7f6a7ffff700 1 -- 172.21.15.44:0/2836186491 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9801b320 con 0x55e26e5ca1a0 2023-09-10T01:22:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.616+0000 7f6a7ffff700 1 -- 172.21.15.44:0/2836186491 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(173..173 src has 1..173) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f6a9801b480 con 0x55e26e5ca1a0 2023-09-10T01:22:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.616+0000 7f6ab300eb80 1 -- 172.21.15.44:0/2836186491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a8c0178a0 con 0x55e26e5ca1a0 2023-09-10T01:22:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.616+0000 7f6a7ffff700 1 -- 172.21.15.44:0/2836186491 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=138) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a98099490 con 0x55e26e5ca1a0 2023-09-10T01:22:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.616+0000 7f6ab300eb80 10 client.29635 fetch_fsmap learned FSMap version 138 2023-09-10T01:22:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.616+0000 7f6ab300eb80 1 -- 172.21.15.44:0/2836186491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=138,monmap=4+}) v3 -- 0x55e26e341250 con 0x55e26e5ca1a0 2023-09-10T01:22:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.617+0000 7f6a7ffff700 1 -- 172.21.15.44:0/2836186491 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a8c0178a0 con 0x55e26e5ca1a0 2023-09-10T01:22:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.617+0000 7f6a7ffff700 1 -- 172.21.15.44:0/2836186491 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 138) v1 ==== 7367+0+0 (secure 0 0 0) 0x7f6a980a0d80 con 0x55e26e5ca1a0 2023-09-10T01:22:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.617+0000 7f6ab300eb80 10 client.29635 fetch_fsmap finished waiting for FSMap version 138 2023-09-10T01:22:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.617+0000 7f6ab300eb80 10 client.29635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:22:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.617+0000 7f6ab300eb80 20 client.29635 populate_metadata read hostname 'smithi044' 2023-09-10T01:22:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.617+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/2836186491 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e26e643b90 0x55e26e645fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:22:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.617+0000 7f6ab300eb80 4 client.29635 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:22:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.617+0000 7f6ab300eb80 1 -- 172.21.15.44:0/2836186491 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e26e50f8c0 con 0x55e26e643b90 2023-09-10T01:22:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.617+0000 7f6aa4be4700 1 --2- 172.21.15.44:0/2836186491 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e26e643b90 0x55e26e645fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:22:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.618+0000 7f6aa4be4700 1 --2- 172.21.15.44:0/2836186491 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e26e643b90 0x55e26e645fb0 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4d45bc4505b8f83 server_cookie=20bcbb138a49b304 in_seq=0 out_seq=0 2023-09-10T01:22:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.618+0000 7f6a7ffff700 10 client.29635 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:22:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.696+0000 7f6a7ffff700 1 -- 172.21.15.44:0/2836186491 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e26e50f8c0 con 0x55e26e643b90 2023-09-10T01:22:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.697+0000 7f6a7ffff700 10 client.29635 handle_command_reply: tid=0 2023-09-10T01:22:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.697+0000 7f6ab300eb80 1 client.29635 shutdown 2023-09-10T01:22:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.700+0000 7f6ab300eb80 1 -- 172.21.15.44:0/2836186491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 msgr2=0x55e26e6421b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.700+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/2836186491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 0x55e26e6421b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6a98004d70 tx=0x7f6a98009420 comp rx=0 tx=0).stop 2023-09-10T01:22:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.700+0000 7f6ab300eb80 1 -- 172.21.15.44:0/2836186491 shutdown_connections 2023-09-10T01:22:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.700+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/2836186491 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e26e643b90 0x55e26e645fb0 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:22:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:22:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:22:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:22:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.700+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/2836186491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e26e5ca1a0 0x55e26e6421b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.700+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/2836186491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e26e5c4300 0x55e26e5e4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.700+0000 7f6ab300eb80 1 --2- 172.21.15.44:0/2836186491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e26e5c4cc0 0x55e26e5e4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.700+0000 7f6ab300eb80 1 -- 172.21.15.44:0/2836186491 >> 172.21.15.44:0/2836186491 conn(0x55e26e539030 msgr2=0x55e26e5eb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.701+0000 7f6a7ffff700 0 client.29635 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:22:48.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.701+0000 7f6ab300eb80 1 -- 172.21.15.44:0/2836186491 shutdown_connections 2023-09-10T01:22:48.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.701+0000 7f6ab300eb80 1 -- 172.21.15.44:0/2836186491 wait complete. 2023-09-10T01:22:48.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.701+0000 7f6ab300eb80 20 client.29635 trim_cache size 0 max 16384 2023-09-10T01:22:48.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.702+0000 7f6aa6e48700 1 -- 172.21.15.44:0/2176651543 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6a900a07b0 msgr2=0x7f6a900a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:48.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.702+0000 7f6aa6e48700 1 --2- 172.21.15.44:0/2176651543 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6a900a07b0 0x7f6a900a2c70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009350 tx=0x7f6a9400b260 comp rx=0 tx=0).stop 2023-09-10T01:22:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.702+0000 7f6aa6e48700 1 -- 172.21.15.44:0/2176651543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aa019cb40 msgr2=0x7f6aa019cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:22:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.702+0000 7f6aa6e48700 1 --2- 172.21.15.44:0/2176651543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aa019cb40 0x7f6aa019cff0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800ef30 tx=0x7f6a9800c450 comp rx=0 tx=0).stop 2023-09-10T01:22:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.702+0000 7f6aa6e48700 1 -- 172.21.15.44:0/2176651543 shutdown_connections 2023-09-10T01:22:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.702+0000 7f6aa6e48700 1 --2- 172.21.15.44:0/2176651543 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6a900a07b0 0x7f6a900a2c70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.702+0000 7f6aa6e48700 1 --2- 172.21.15.44:0/2176651543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aa019cb40 0x7f6aa019cff0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.702+0000 7f6aa6e48700 1 --2- 172.21.15.44:0/2176651543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa01054b0 0x7f6aa01a2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.702+0000 7f6aa6e48700 1 --2- 172.21.15.44:0/2176651543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6aa0105920 0x7f6aa01a3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:22:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.702+0000 7f6aa6e48700 1 -- 172.21.15.44:0/2176651543 >> 172.21.15.44:0/2176651543 conn(0x7f6aa0074f30 msgr2=0x7f6aa00fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:22:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.707+0000 7f6aa6e48700 1 -- 172.21.15.44:0/2176651543 shutdown_connections 2023-09-10T01:22:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:22:48.709+0000 7f6aa6e48700 1 -- 172.21.15.44:0/2176651543 wait complete. 2023-09-10T01:22:49.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:49 smithi131 ceph-mon[103713]: pgmap v1363: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 10 MiB/s wr, 660 op/s 2023-09-10T01:22:49.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:49 smithi131 ceph-mon[103713]: scrub summary: idle+waiting paths [/] 2023-09-10T01:22:49.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:49 smithi044 ceph-mon[98676]: pgmap v1363: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 10 MiB/s wr, 660 op/s 2023-09-10T01:22:49.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:49 smithi044 ceph-mon[98676]: scrub summary: idle+waiting paths [/] 2023-09-10T01:22:49.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:49 smithi099 ceph-mon[102692]: pgmap v1363: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 10 MiB/s wr, 660 op/s 2023-09-10T01:22:49.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:49 smithi099 ceph-mon[102692]: scrub summary: idle+waiting paths [/] 2023-09-10T01:22:50.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:50 smithi131 ceph-mon[103713]: scrub summary: active paths [/] 2023-09-10T01:22:50.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:50 smithi131 ceph-mon[103713]: scrub summary: idle+waiting paths [/] 2023-09-10T01:22:50.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:50 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:22:50.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:50 smithi044 ceph-mon[98676]: scrub summary: active paths [/] 2023-09-10T01:22:50.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:50 smithi044 ceph-mon[98676]: scrub summary: idle+waiting paths [/] 2023-09-10T01:22:50.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:50 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:22:50.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:50 smithi099 ceph-mon[102692]: scrub summary: active paths [/] 2023-09-10T01:22:50.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:50 smithi099 ceph-mon[102692]: scrub summary: idle+waiting paths [/] 2023-09-10T01:22:50.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:50 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:22:51.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 105.0 s, 166.8 tps, lat 582.672 ms stddev 650.610 2023-09-10T01:22:51.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:51 smithi131 ceph-mon[103713]: pgmap v1364: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 9.6 MiB/s wr, 631 op/s 2023-09-10T01:22:51.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:51 smithi044 ceph-mon[98676]: pgmap v1364: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 9.6 MiB/s wr, 631 op/s 2023-09-10T01:22:51.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:51 smithi099 ceph-mon[102692]: pgmap v1364: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 9.6 MiB/s wr, 631 op/s 2023-09-10T01:22:53.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:53 smithi131 ceph-mon[103713]: pgmap v1365: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 14 MiB/s wr, 838 op/s 2023-09-10T01:22:53.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:53 smithi044 ceph-mon[98676]: pgmap v1365: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 14 MiB/s wr, 838 op/s 2023-09-10T01:22:53.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:53 smithi099 ceph-mon[102692]: pgmap v1365: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 14 MiB/s wr, 838 op/s 2023-09-10T01:22:55.199 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:55 smithi099 ceph-mon[102692]: pgmap v1366: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 9.2 MiB/s wr, 506 op/s 2023-09-10T01:22:55.199 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:55 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:22:55.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:55 smithi131 ceph-mon[103713]: pgmap v1366: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 9.2 MiB/s wr, 506 op/s 2023-09-10T01:22:55.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:55 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:22:55.565 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:55 smithi044 ceph-mon[98676]: pgmap v1366: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 9.2 MiB/s wr, 506 op/s 2023-09-10T01:22:55.565 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:55 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:22:56.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 110.0 s, 188.6 tps, lat 621.661 ms stddev 1156.008 2023-09-10T01:22:57.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:57 smithi131 ceph-mon[103713]: pgmap v1367: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 11 MiB/s wr, 642 op/s 2023-09-10T01:22:57.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:57 smithi044 ceph-mon[98676]: pgmap v1367: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 11 MiB/s wr, 642 op/s 2023-09-10T01:22:57.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:57 smithi099 ceph-mon[102692]: pgmap v1367: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 11 MiB/s wr, 642 op/s 2023-09-10T01:22:58.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:22:58 smithi131 ceph-mon[103713]: pgmap v1368: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 6.4 MiB/s wr, 380 op/s 2023-09-10T01:22:58.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:22:58 smithi044 ceph-mon[98676]: pgmap v1368: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 6.4 MiB/s wr, 380 op/s 2023-09-10T01:22:58.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:22:58 smithi099 ceph-mon[102692]: pgmap v1368: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 6.4 MiB/s wr, 380 op/s 2023-09-10T01:23:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:00 smithi099 ceph-mon[102692]: pgmap v1369: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 5.9 MiB/s wr, 356 op/s 2023-09-10T01:23:01.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 115.0 s, 131.4 tps, lat 308.082 ms stddev 205.346 2023-09-10T01:23:01.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:00 smithi044 ceph-mon[98676]: pgmap v1369: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 5.9 MiB/s wr, 356 op/s 2023-09-10T01:23:01.308 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:00 smithi131 ceph-mon[103713]: pgmap v1369: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 5.9 MiB/s wr, 356 op/s 2023-09-10T01:23:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.940+0000 7f441c8ef700 1 -- 172.21.15.44:0/990114702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4418108fa0 msgr2=0x7f4418109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.940+0000 7f441c8ef700 1 --2- 172.21.15.44:0/990114702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4418108fa0 0x7f4418109380 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f44080098f0 tx=0x7f4408009c00 comp rx=0 tx=0).stop 2023-09-10T01:23:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.940+0000 7f441c8ef700 1 -- 172.21.15.44:0/990114702 shutdown_connections 2023-09-10T01:23:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.940+0000 7f441c8ef700 1 --2- 172.21.15.44:0/990114702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4418112550 0x7f4418114940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.940+0000 7f441c8ef700 1 --2- 172.21.15.44:0/990114702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4418108fa0 0x7f4418109380 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.940+0000 7f441c8ef700 1 --2- 172.21.15.44:0/990114702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44181098c0 0x7f4418112010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.940+0000 7f441c8ef700 1 -- 172.21.15.44:0/990114702 >> 172.21.15.44:0/990114702 conn(0x7f441806cb00 msgr2=0x7f441806cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.940+0000 7f441c8ef700 1 -- 172.21.15.44:0/990114702 shutdown_connections 2023-09-10T01:23:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.940+0000 7f441c8ef700 1 -- 172.21.15.44:0/990114702 wait complete. 2023-09-10T01:23:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.941+0000 7f441c8ef700 1 Processor -- start 2023-09-10T01:23:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.941+0000 7f441c8ef700 1 -- start start 2023-09-10T01:23:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.941+0000 7f441c8ef700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4418108fa0 0x7f441819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.941+0000 7f441c8ef700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44181098c0 0x7f441819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.941+0000 7f441c8ef700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4418112550 0x7f44181a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.941+0000 7f441c8ef700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44181a3fa0 con 0x7f4418108fa0 2023-09-10T01:23:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.941+0000 7f441c8ef700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44181a4110 con 0x7f44181098c0 2023-09-10T01:23:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.941+0000 7f441c8ef700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44181a42d0 con 0x7f4418112550 2023-09-10T01:23:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.942+0000 7f4416d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4418112550 0x7f44181a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.942+0000 7f4416d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4418112550 0x7f44181a3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37616/0 (socket says 172.21.15.44:37616) 2023-09-10T01:23:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.942+0000 7f4416d9d700 1 -- 172.21.15.44:0/3943370113 learned_addr learned my addr 172.21.15.44:0/3943370113 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.942+0000 7f4415d9b700 1 --2- 172.21.15.44:0/3943370113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44181098c0 0x7f441819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.942+0000 7f441659c700 1 --2- 172.21.15.44:0/3943370113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4418108fa0 0x7f441819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.942+0000 7f4416d9d700 1 -- 172.21.15.44:0/3943370113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44181098c0 msgr2=0x7f441819f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.942+0000 7f4416d9d700 1 --2- 172.21.15.44:0/3943370113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44181098c0 0x7f441819f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.942+0000 7f4416d9d700 1 -- 172.21.15.44:0/3943370113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4418108fa0 msgr2=0x7f441819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.942+0000 7f4416d9d700 1 --2- 172.21.15.44:0/3943370113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4418108fa0 0x7f441819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.942+0000 7f4416d9d700 1 -- 172.21.15.44:0/3943370113 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4408009520 con 0x7f4418112550 2023-09-10T01:23:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.943+0000 7f4416d9d700 1 --2- 172.21.15.44:0/3943370113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4418112550 0x7f44181a3960 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f441000ef90 tx=0x7f441000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.943+0000 7f44077fe700 1 -- 172.21.15.44:0/3943370113 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410018070 con 0x7f4418112550 2023-09-10T01:23:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.943+0000 7f44077fe700 1 -- 172.21.15.44:0/3943370113 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f441000f040 con 0x7f4418112550 2023-09-10T01:23:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.943+0000 7f44077fe700 1 -- 172.21.15.44:0/3943370113 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441000cc40 con 0x7f4418112550 2023-09-10T01:23:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.943+0000 7f441c8ef700 1 -- 172.21.15.44:0/3943370113 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44181a4550 con 0x7f4418112550 2023-09-10T01:23:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.943+0000 7f441c8ef700 1 -- 172.21.15.44:0/3943370113 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44181155b0 con 0x7f4418112550 2023-09-10T01:23:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.950+0000 7f44077fe700 1 -- 172.21.15.44:0/3943370113 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f4410004000 con 0x7f4418112550 2023-09-10T01:23:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.951+0000 7f44077fe700 1 --2- 172.21.15.44:0/3943370113 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44000a0830 0x7f44000a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.951+0000 7f44077fe700 1 -- 172.21.15.44:0/3943370113 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f44100ccd10 con 0x7f4418112550 2023-09-10T01:23:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.952+0000 7f441659c700 1 --2- 172.21.15.44:0/3943370113 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44000a0830 0x7f44000a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.952+0000 7f441659c700 1 --2- 172.21.15.44:0/3943370113 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44000a0830 0x7f44000a2cf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4408009fd0 tx=0x7f4408019040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.953+0000 7f4428ab5b80 1 Processor -- start 2023-09-10T01:23:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.953+0000 7f4428ab5b80 1 -- start start 2023-09-10T01:23:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.953+0000 7f4428ab5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c771a0 0x55eb79c77580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.953+0000 7f4428ab5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 0x55eb79c71780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.953+0000 7f4428ab5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c71cc0 0x55eb79c7df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.953+0000 7f4428ab5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb79bca6d0 con 0x55eb79c771a0 2023-09-10T01:23:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.953+0000 7f4428ab5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb79bc7ff0 con 0x55eb79c71cc0 2023-09-10T01:23:01.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.953+0000 7f4428ab5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb79bc38e0 con 0x55eb79c71300 2023-09-10T01:23:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.953+0000 7f4415d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 0x55eb79c71780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.953+0000 7f4415d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 0x55eb79c71780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37624/0 (socket says 172.21.15.44:37624) 2023-09-10T01:23:01.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.953+0000 7f4415d9b700 1 -- 172.21.15.44:0/1177768689 learned_addr learned my addr 172.21.15.44:0/1177768689 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:01.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.954+0000 7f4416d9d700 1 --2- 172.21.15.44:0/1177768689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c71cc0 0x55eb79c7df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:01.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.954+0000 7f4415d9b700 1 -- 172.21.15.44:0/1177768689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c71cc0 msgr2=0x55eb79c7df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:01.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.954+0000 7f4415d9b700 1 --2- 172.21.15.44:0/1177768689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c71cc0 0x55eb79c7df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.954+0000 7f4415d9b700 1 -- 172.21.15.44:0/1177768689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c771a0 msgr2=0x55eb79c77580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:01.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.954+0000 7f4415d9b700 1 --2- 172.21.15.44:0/1177768689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c771a0 0x55eb79c77580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.954+0000 7f4415d9b700 1 -- 172.21.15.44:0/1177768689 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb799efd80 con 0x55eb79c71300 2023-09-10T01:23:01.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.954+0000 7f4415d9b700 1 --2- 172.21.15.44:0/1177768689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 0x55eb79c71780 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f440c010ca0 tx=0x7f440c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53f091905e0861f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:01.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.954+0000 7f4404ff9700 1 -- 172.21.15.44:0/1177768689 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c015050 con 0x55eb79c71300 2023-09-10T01:23:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.955+0000 7f4404ff9700 1 -- 172.21.15.44:0/1177768689 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f440c016040 con 0x55eb79c71300 2023-09-10T01:23:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.955+0000 7f4404ff9700 1 -- 172.21.15.44:0/1177768689 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c01a400 con 0x55eb79c71300 2023-09-10T01:23:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.955+0000 7f4428ab5b80 1 -- 172.21.15.44:0/1177768689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 msgr2=0x55eb79c71780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.955+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/1177768689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 0x55eb79c71780 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f440c010ca0 tx=0x7f440c009310 comp rx=0 tx=0).stop 2023-09-10T01:23:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.956+0000 7f4428ab5b80 1 -- 172.21.15.44:0/1177768689 shutdown_connections 2023-09-10T01:23:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.956+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/1177768689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c71cc0 0x55eb79c7df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.956+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/1177768689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c771a0 0x55eb79c77580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.956+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/1177768689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 0x55eb79c71780 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.956+0000 7f4428ab5b80 1 -- 172.21.15.44:0/1177768689 >> 172.21.15.44:0/1177768689 conn(0x55eb79be6030 msgr2=0x55eb79c6c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.956+0000 7f4428ab5b80 1 -- 172.21.15.44:0/1177768689 shutdown_connections 2023-09-10T01:23:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.956+0000 7f4428ab5b80 1 -- 172.21.15.44:0/1177768689 wait complete. 2023-09-10T01:23:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.957+0000 7f4428ab5b80 1 -- start start 2023-09-10T01:23:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.957+0000 7f4428ab5b80 1 Processor -- start 2023-09-10T01:23:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.957+0000 7f4428ab5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 0x55eb79c7dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.957+0000 7f4428ab5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71cc0 0x55eb79c7b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.957+0000 7f4428ab5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79c7b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.957+0000 7f4428ab5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb79b61b10 con 0x55eb79c71cc0 2023-09-10T01:23:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.957+0000 7f4428ab5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb79b61810 con 0x55eb79c771a0 2023-09-10T01:23:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.957+0000 7f4428ab5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb79a458c0 con 0x55eb79c71300 2023-09-10T01:23:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.958+0000 7f4428ab5b80 10 client.0 _ll_register_callbacks cb 0x55eb79aaba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:23:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.958+0000 7f4416d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79c7b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.958+0000 7f4416d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79c7b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60720/0 (socket says 172.21.15.44:60720) 2023-09-10T01:23:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.958+0000 7f4416d9d700 1 -- 172.21.15.44:0/454957816 learned_addr learned my addr 172.21.15.44:0/454957816 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.958+0000 7f441659c700 1 --2- 172.21.15.44:0/454957816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 0x55eb79c7dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.958+0000 7f4415d9b700 1 --2- 172.21.15.44:0/454957816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71cc0 0x55eb79c7b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.958+0000 7f4416d9d700 1 -- 172.21.15.44:0/454957816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 msgr2=0x55eb79c7dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.958+0000 7f4416d9d700 1 --2- 172.21.15.44:0/454957816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 0x55eb79c7dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.958+0000 7f4416d9d700 1 -- 172.21.15.44:0/454957816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71cc0 msgr2=0x55eb79c7b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.958+0000 7f4416d9d700 1 --2- 172.21.15.44:0/454957816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71cc0 0x55eb79c7b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.958+0000 7f4416d9d700 1 -- 172.21.15.44:0/454957816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440c010950 con 0x55eb79c771a0 2023-09-10T01:23:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.958+0000 7f4415d9b700 1 --2- 172.21.15.44:0/454957816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71cc0 0x55eb79c7b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:23:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.958+0000 7f4416d9d700 1 --2- 172.21.15.44:0/454957816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79c7b5e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4410099de0 tx=0x7f4410005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9170b162b55b0eb3 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.959+0000 7f43fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:23:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.959+0000 7f43fdffb700 1 -- 172.21.15.44:0/454957816 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441009dc80 con 0x55eb79c771a0 2023-09-10T01:23:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.959+0000 7f4428ab5b80 1 -- 172.21.15.44:0/454957816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb79b1b490 con 0x55eb79c771a0 2023-09-10T01:23:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.959+0000 7f43fdffb700 1 -- 172.21.15.44:0/454957816 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(173..173 src has 1..173) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f44100a1e30 con 0x55eb79c771a0 2023-09-10T01:23:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.959+0000 7f43fdffb700 1 -- 172.21.15.44:0/454957816 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44100b2200 con 0x55eb79c771a0 2023-09-10T01:23:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.959+0000 7f4428ab5b80 10 client.29654 fetch_fsmap learned FSMap version 140 2023-09-10T01:23:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.959+0000 7f4428ab5b80 1 -- 172.21.15.44:0/454957816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=4+}) v3 -- 0x55eb799d8c00 con 0x55eb79c771a0 2023-09-10T01:23:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.960+0000 7f43fdffb700 1 -- 172.21.15.44:0/454957816 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55eb79b1b490 con 0x55eb79c771a0 2023-09-10T01:23:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.960+0000 7f43fdffb700 1 -- 172.21.15.44:0/454957816 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 140) v1 ==== 7375+0+0 (secure 0 0 0) 0x7f44100a19a0 con 0x55eb79c771a0 2023-09-10T01:23:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.960+0000 7f4428ab5b80 10 client.29654 fetch_fsmap finished waiting for FSMap version 140 2023-09-10T01:23:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.960+0000 7f4428ab5b80 10 client.29654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:23:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.960+0000 7f4428ab5b80 20 client.29654 populate_metadata read hostname 'smithi044' 2023-09-10T01:23:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.960+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/454957816 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55eb79c80db0 0x55eb79c831d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.960+0000 7f4428ab5b80 4 client.29654 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:23:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.960+0000 7f4428ab5b80 1 -- 172.21.15.44:0/454957816 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb79bd8840 con 0x55eb79c80db0 2023-09-10T01:23:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.960+0000 7f441659c700 1 --2- 172.21.15.44:0/454957816 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55eb79c80db0 0x55eb79c831d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:01.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.961+0000 7f441659c700 1 --2- 172.21.15.44:0/454957816 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55eb79c80db0 0x55eb79c831d0 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df7789cac4142b64 server_cookie=121060b81a3a715e in_seq=0 out_seq=0 2023-09-10T01:23:01.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.961+0000 7f43fdffb700 10 client.29654 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:01.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.961+0000 7f43fdffb700 1 -- 172.21.15.44:0/454957816 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55eb79bd8840 con 0x55eb79c80db0 2023-09-10T01:23:01.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.961+0000 7f43fdffb700 10 client.29654 handle_command_reply: tid=0 2023-09-10T01:23:01.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.962+0000 7f4428ab5b80 1 client.29654 shutdown 2023-09-10T01:23:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.966+0000 7f4428ab5b80 1 -- 172.21.15.44:0/454957816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 msgr2=0x55eb79c7b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.966+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/454957816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79c7b5e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4410099de0 tx=0x7f4410005be0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.966+0000 7f4428ab5b80 1 -- 172.21.15.44:0/454957816 shutdown_connections 2023-09-10T01:23:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.966+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/454957816 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55eb79c80db0 0x55eb79c831d0 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.966+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/454957816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79c7b5e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.966+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/454957816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 0x55eb79c7dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.966+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/454957816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71cc0 0x55eb79c7b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.966+0000 7f43fdffb700 0 client.29654 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.966+0000 7f4428ab5b80 1 -- 172.21.15.44:0/454957816 >> 172.21.15.44:0/454957816 conn(0x55eb79be6030 msgr2=0x55eb79c6b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.966+0000 7f4428ab5b80 1 -- 172.21.15.44:0/454957816 shutdown_connections 2023-09-10T01:23:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.966+0000 7f4428ab5b80 1 -- 172.21.15.44:0/454957816 wait complete. 2023-09-10T01:23:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.966+0000 7f4428ab5b80 20 client.29654 trim_cache size 0 max 16384 2023-09-10T01:23:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4428ab5b80 1 Processor -- start 2023-09-10T01:23:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4428ab5b80 1 -- start start 2023-09-10T01:23:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4428ab5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 0x55eb79c87b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4428ab5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71cc0 0x55eb79c7d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4428ab5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79c7a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4428ab5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb79c725c0 con 0x55eb79c71cc0 2023-09-10T01:23:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4428ab5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb79c63f50 con 0x55eb79c771a0 2023-09-10T01:23:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4428ab5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb79c720a0 con 0x55eb79c71300 2023-09-10T01:23:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4415d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71cc0 0x55eb79c7d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f441659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 0x55eb79c87b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4415d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71cc0 0x55eb79c7d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58988/0 (socket says 172.21.15.44:58988) 2023-09-10T01:23:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4415d9b700 1 -- 172.21.15.44:0/2655635753 learned_addr learned my addr 172.21.15.44:0/2655635753 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4416d9d700 1 --2- 172.21.15.44:0/2655635753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79c7a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4415d9b700 1 -- 172.21.15.44:0/2655635753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 msgr2=0x55eb79c87b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4415d9b700 1 --2- 172.21.15.44:0/2655635753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 0x55eb79c87b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4415d9b700 1 -- 172.21.15.44:0/2655635753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 msgr2=0x55eb79c7a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4415d9b700 1 --2- 172.21.15.44:0/2655635753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79c7a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.991+0000 7f4415d9b700 1 -- 172.21.15.44:0/2655635753 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb79a161f0 con 0x55eb79c71cc0 2023-09-10T01:23:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.992+0000 7f4416d9d700 1 --2- 172.21.15.44:0/2655635753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79c7a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:23:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.992+0000 7f441659c700 1 --2- 172.21.15.44:0/2655635753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 0x55eb79c87b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:23:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.992+0000 7f4415d9b700 1 --2- 172.21.15.44:0/2655635753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71cc0 0x55eb79c7d910 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x55eb79c6c060 tx=0x7f440c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b89a0a56309cd6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.992+0000 7f44057fa700 1 -- 172.21.15.44:0/2655635753 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c010270 con 0x55eb79c71cc0 2023-09-10T01:23:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.992+0000 7f44057fa700 1 -- 172.21.15.44:0/2655635753 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f440c009e60 con 0x55eb79c71cc0 2023-09-10T01:23:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.992+0000 7f44057fa700 1 -- 172.21.15.44:0/2655635753 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c003a20 con 0x55eb79c71cc0 2023-09-10T01:23:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.993+0000 7f4428ab5b80 1 -- 172.21.15.44:0/2655635753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71cc0 msgr2=0x55eb79c7d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.993+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/2655635753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71cc0 0x55eb79c7d910 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x55eb79c6c060 tx=0x7f440c005c00 comp rx=0 tx=0).stop 2023-09-10T01:23:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.993+0000 7f4428ab5b80 1 -- 172.21.15.44:0/2655635753 shutdown_connections 2023-09-10T01:23:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.993+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/2655635753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79c7a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.993+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/2655635753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71300 0x55eb79c87b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.993+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/2655635753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71cc0 0x55eb79c7d910 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.993+0000 7f4428ab5b80 1 -- 172.21.15.44:0/2655635753 >> 172.21.15.44:0/2655635753 conn(0x55eb79be6030 msgr2=0x55eb79c761a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.993+0000 7f4428ab5b80 1 -- 172.21.15.44:0/2655635753 shutdown_connections 2023-09-10T01:23:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.993+0000 7f4428ab5b80 1 -- 172.21.15.44:0/2655635753 wait complete. 2023-09-10T01:23:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.993+0000 7f4428ab5b80 1 -- start start 2023-09-10T01:23:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.994+0000 7f4428ab5b80 1 Processor -- start 2023-09-10T01:23:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.994+0000 7f4428ab5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71300 0x55eb79c8ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.994+0000 7f4428ab5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71cc0 0x55eb79c8f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.994+0000 7f4428ab5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79ce94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.994+0000 7f4428ab5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb79ce9a30 con 0x55eb79c71300 2023-09-10T01:23:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.994+0000 7f4428ab5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb79ce9ba0 con 0x55eb79c771a0 2023-09-10T01:23:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.994+0000 7f4428ab5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb79ce9d60 con 0x55eb79c71cc0 2023-09-10T01:23:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.994+0000 7f4428ab5b80 10 client.0 _ll_register_callbacks cb 0x7f441000ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:23:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.994+0000 7f4416d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79ce94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.994+0000 7f4415d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71cc0 0x55eb79c8f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.994+0000 7f4415d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71cc0 0x55eb79c8f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37644/0 (socket says 172.21.15.44:37644) 2023-09-10T01:23:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.994+0000 7f4416d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79ce94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60736/0 (socket says 172.21.15.44:60736) 2023-09-10T01:23:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.994+0000 7f4415d9b700 1 -- 172.21.15.44:0/573386804 learned_addr learned my addr 172.21.15.44:0/573386804 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.995+0000 7f4415d9b700 1 -- 172.21.15.44:0/573386804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 msgr2=0x55eb79ce94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.995+0000 7f4415d9b700 1 --2- 172.21.15.44:0/573386804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79ce94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.995+0000 7f4415d9b700 1 -- 172.21.15.44:0/573386804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71300 msgr2=0x55eb79c8ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.995+0000 7f4415d9b700 1 --2- 172.21.15.44:0/573386804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71300 0x55eb79c8ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.995+0000 7f4415d9b700 1 -- 172.21.15.44:0/573386804 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4418107e90 con 0x55eb79c71cc0 2023-09-10T01:23:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.995+0000 7f4416d9d700 1 --2- 172.21.15.44:0/573386804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79ce94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:23:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.995+0000 7f4415d9b700 1 --2- 172.21.15.44:0/573386804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71cc0 0x55eb79c8f260 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f440c005fe0 tx=0x7f440c0050c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b48e2f4c73509802 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.995+0000 7f43fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:23:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.995+0000 7f43fdffb700 1 -- 172.21.15.44:0/573386804 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c015040 con 0x55eb79c71cc0 2023-09-10T01:23:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.995+0000 7f4428ab5b80 1 -- 172.21.15.44:0/573386804 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f440c016040 con 0x55eb79c71cc0 2023-09-10T01:23:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.998+0000 7f43fdffb700 1 -- 172.21.15.44:0/573386804 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(173..173 src has 1..173) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f440c003680 con 0x55eb79c71cc0 2023-09-10T01:23:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.998+0000 7f43fdffb700 1 -- 172.21.15.44:0/573386804 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f440c003840 con 0x55eb79c71cc0 2023-09-10T01:23:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.998+0000 7f4428ab5b80 10 client.29653 fetch_fsmap learned FSMap version 140 2023-09-10T01:23:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.998+0000 7f4428ab5b80 1 -- 172.21.15.44:0/573386804 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=4+}) v3 -- 0x55eb799d8c00 con 0x55eb79c71cc0 2023-09-10T01:23:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.998+0000 7f43fdffb700 1 -- 172.21.15.44:0/573386804 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f440c016040 con 0x55eb79c71cc0 2023-09-10T01:23:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.998+0000 7f43fdffb700 1 -- 172.21.15.44:0/573386804 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 140) v1 ==== 7375+0+0 (secure 0 0 0) 0x7f440c023b50 con 0x55eb79c71cc0 2023-09-10T01:23:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.998+0000 7f4428ab5b80 10 client.29653 fetch_fsmap finished waiting for FSMap version 140 2023-09-10T01:23:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.998+0000 7f4428ab5b80 10 client.29653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:23:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.998+0000 7f4428ab5b80 20 client.29653 populate_metadata read hostname 'smithi044' 2023-09-10T01:23:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.998+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/573386804 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55eb79cf1080 0x55eb79cf34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.998+0000 7f4428ab5b80 4 client.29653 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:23:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.998+0000 7f4428ab5b80 1 -- 172.21.15.44:0/573386804 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eb79bbc8c0 con 0x55eb79cf1080 2023-09-10T01:23:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.999+0000 7f441659c700 1 --2- 172.21.15.44:0/573386804 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55eb79cf1080 0x55eb79cf34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.999+0000 7f43fdffb700 10 client.29653 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:01.999+0000 7f441659c700 1 --2- 172.21.15.44:0/573386804 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55eb79cf1080 0x55eb79cf34a0 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8ce834bb8e4b33 server_cookie=b267a6b89c495f05 in_seq=0 out_seq=0 2023-09-10T01:23:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.032+0000 7f43fdffb700 1 -- 172.21.15.44:0/573386804 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eb79bbc8c0 con 0x55eb79cf1080 2023-09-10T01:23:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.032+0000 7f43fdffb700 10 client.29653 handle_command_reply: tid=0 2023-09-10T01:23:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.032+0000 7f4428ab5b80 1 client.29653 shutdown 2023-09-10T01:23:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.032+0000 7f4428ab5b80 1 -- 172.21.15.44:0/573386804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71cc0 msgr2=0x55eb79c8f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.032+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/573386804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71cc0 0x55eb79c8f260 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f440c005fe0 tx=0x7f440c0050c0 comp rx=0 tx=0).stop 2023-09-10T01:23:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.034+0000 7f4428ab5b80 1 -- 172.21.15.44:0/573386804 shutdown_connections 2023-09-10T01:23:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.034+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/573386804 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55eb79cf1080 0x55eb79cf34a0 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.034+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/573386804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb79c771a0 0x55eb79ce94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.034+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/573386804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb79c71300 0x55eb79c8ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.034+0000 7f4428ab5b80 1 --2- 172.21.15.44:0/573386804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55eb79c71cc0 0x55eb79c8f260 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.034+0000 7f4428ab5b80 1 -- 172.21.15.44:0/573386804 >> 172.21.15.44:0/573386804 conn(0x55eb79be6030 msgr2=0x55eb79c702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.034+0000 7f43fdffb700 0 client.29653 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.034+0000 7f4428ab5b80 1 -- 172.21.15.44:0/573386804 shutdown_connections 2023-09-10T01:23:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.034+0000 7f4428ab5b80 1 -- 172.21.15.44:0/573386804 wait complete. 2023-09-10T01:23:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.035+0000 7f4428ab5b80 20 client.29653 trim_cache size 0 max 16384 2023-09-10T01:23:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:23:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:23:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:23:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:23:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.035+0000 7f441c8ef700 1 -- 172.21.15.44:0/3943370113 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44000a0830 msgr2=0x7f44000a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.035+0000 7f441c8ef700 1 --2- 172.21.15.44:0/3943370113 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44000a0830 0x7f44000a2cf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4408009fd0 tx=0x7f4408019040 comp rx=0 tx=0).stop 2023-09-10T01:23:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.035+0000 7f441c8ef700 1 -- 172.21.15.44:0/3943370113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4418112550 msgr2=0x7f44181a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.035+0000 7f441c8ef700 1 --2- 172.21.15.44:0/3943370113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4418112550 0x7f44181a3960 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f441000ef90 tx=0x7f441000c3a0 comp rx=0 tx=0).stop 2023-09-10T01:23:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.035+0000 7f441c8ef700 1 -- 172.21.15.44:0/3943370113 shutdown_connections 2023-09-10T01:23:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.035+0000 7f441c8ef700 1 --2- 172.21.15.44:0/3943370113 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f44000a0830 0x7f44000a2cf0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.035+0000 7f441c8ef700 1 --2- 172.21.15.44:0/3943370113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4418112550 0x7f44181a3960 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.035+0000 7f441c8ef700 1 --2- 172.21.15.44:0/3943370113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4418108fa0 0x7f441819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.035+0000 7f441c8ef700 1 --2- 172.21.15.44:0/3943370113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44181098c0 0x7f441819f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.035+0000 7f441c8ef700 1 -- 172.21.15.44:0/3943370113 >> 172.21.15.44:0/3943370113 conn(0x7f441806cb00 msgr2=0x7f441806f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.035+0000 7f441c8ef700 1 -- 172.21.15.44:0/3943370113 shutdown_connections 2023-09-10T01:23:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:02.035+0000 7f441c8ef700 1 -- 172.21.15.44:0/3943370113 wait complete. 2023-09-10T01:23:03.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:03 smithi131 ceph-mon[103713]: pgmap v1370: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 8.8 MiB/s wr, 630 op/s 2023-09-10T01:23:03.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:03 smithi044 ceph-mon[98676]: pgmap v1370: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 8.8 MiB/s wr, 630 op/s 2023-09-10T01:23:03.612 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:03 smithi099 ceph-mon[102692]: pgmap v1370: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 8.8 MiB/s wr, 630 op/s 2023-09-10T01:23:04.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:04 smithi131 ceph-mon[103713]: scrub summary: idle 2023-09-10T01:23:04.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:04 smithi131 ceph-mon[103713]: scrub summary: idle 2023-09-10T01:23:04.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:04 smithi131 ceph-mon[103713]: scrub summary: idle 2023-09-10T01:23:04.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:04 smithi131 ceph-mon[103713]: scrub summary: idle 2023-09-10T01:23:04.564 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:04 smithi044 ceph-mon[98676]: scrub summary: idle 2023-09-10T01:23:04.565 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:04 smithi044 ceph-mon[98676]: scrub summary: idle 2023-09-10T01:23:04.565 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:04 smithi044 ceph-mon[98676]: scrub summary: idle 2023-09-10T01:23:04.565 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:04 smithi044 ceph-mon[98676]: scrub summary: idle 2023-09-10T01:23:04.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:04 smithi099 ceph-mon[102692]: scrub summary: idle 2023-09-10T01:23:04.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:04 smithi099 ceph-mon[102692]: scrub summary: idle 2023-09-10T01:23:04.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:04 smithi099 ceph-mon[102692]: scrub summary: idle 2023-09-10T01:23:04.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:04 smithi099 ceph-mon[102692]: scrub summary: idle 2023-09-10T01:23:05.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:05 smithi131 ceph-mon[103713]: scrub summary: idle 2023-09-10T01:23:05.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:05 smithi131 ceph-mon[103713]: pgmap v1371: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 4.9 MiB/s wr, 424 op/s 2023-09-10T01:23:05.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:05 smithi131 ceph-mon[103713]: osdmap e174: 12 total, 12 up, 12 in 2023-09-10T01:23:05.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:05 smithi044 ceph-mon[98676]: scrub summary: idle 2023-09-10T01:23:05.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:05 smithi044 ceph-mon[98676]: pgmap v1371: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 4.9 MiB/s wr, 424 op/s 2023-09-10T01:23:05.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:05 smithi044 ceph-mon[98676]: osdmap e174: 12 total, 12 up, 12 in 2023-09-10T01:23:05.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:05 smithi099 ceph-mon[102692]: scrub summary: idle 2023-09-10T01:23:05.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:05 smithi099 ceph-mon[102692]: pgmap v1371: 129 pgs: 129 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 4.9 MiB/s wr, 424 op/s 2023-09-10T01:23:05.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:05 smithi099 ceph-mon[102692]: osdmap e174: 12 total, 12 up, 12 in 2023-09-10T01:23:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 120.0 s, 254.1 tps, lat 617.143 ms stddev 991.453 2023-09-10T01:23:07.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:07 smithi131 ceph-mon[103713]: pgmap v1373: 129 pgs: 18 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 71 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 6.8 MiB/s wr, 598 op/s 2023-09-10T01:23:07.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:07 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:23:07.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:07 smithi044 ceph-mon[98676]: pgmap v1373: 129 pgs: 18 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 71 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 6.8 MiB/s wr, 598 op/s 2023-09-10T01:23:07.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:07 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:23:07.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:07 smithi099 ceph-mon[102692]: pgmap v1373: 129 pgs: 18 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 71 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 6.8 MiB/s wr, 598 op/s 2023-09-10T01:23:07.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:07 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:23:08.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:08 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:23:08.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:08 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:23:08.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:08 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:23:08.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:08 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:23:08.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:08 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:23:08.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:08 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:23:08.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:08 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:23:08.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:08 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:23:08.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:08 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:23:09.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:09 smithi131 ceph-mon[103713]: pgmap v1374: 129 pgs: 20 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 62 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 7.4 MiB/s wr, 628 op/s 2023-09-10T01:23:09.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:09 smithi044 ceph-mon[98676]: pgmap v1374: 129 pgs: 20 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 62 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 7.4 MiB/s wr, 628 op/s 2023-09-10T01:23:09.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:09 smithi099 ceph-mon[102692]: pgmap v1374: 129 pgs: 20 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 62 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 7.4 MiB/s wr, 628 op/s 2023-09-10T01:23:11.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 125.0 s, 230.7 tps, lat 402.003 ms stddev 531.855 2023-09-10T01:23:11.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:11 smithi131 ceph-mon[103713]: pgmap v1375: 129 pgs: 20 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 62 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 7.4 MiB/s wr, 628 op/s 2023-09-10T01:23:11.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:11 smithi044 ceph-mon[98676]: pgmap v1375: 129 pgs: 20 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 62 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 7.4 MiB/s wr, 628 op/s 2023-09-10T01:23:11.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:11 smithi099 ceph-mon[102692]: pgmap v1375: 129 pgs: 20 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 62 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 7.4 MiB/s wr, 628 op/s 2023-09-10T01:23:13.552 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:13 smithi131 ceph-mon[103713]: pgmap v1376: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 6.9 MiB/s wr, 528 op/s 2023-09-10T01:23:13.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:13 smithi044 ceph-mon[98676]: pgmap v1376: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 6.9 MiB/s wr, 528 op/s 2023-09-10T01:23:13.611 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:13 smithi099 ceph-mon[102692]: pgmap v1376: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 6.9 MiB/s wr, 528 op/s 2023-09-10T01:23:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.236+0000 7fa2b1f4f700 1 -- 172.21.15.44:0/310248024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2ac10ed50 msgr2=0x7fa2ac107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.236+0000 7fa2b1f4f700 1 --2- 172.21.15.44:0/310248024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2ac10ed50 0x7fa2ac107600 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009990 tx=0x7fa2a0009ca0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.237+0000 7fa2b1f4f700 1 -- 172.21.15.44:0/310248024 shutdown_connections 2023-09-10T01:23:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.237+0000 7fa2b1f4f700 1 --2- 172.21.15.44:0/310248024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac119de0 0x7fa2ac11c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.237+0000 7fa2b1f4f700 1 --2- 172.21.15.44:0/310248024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2ac10ed50 0x7fa2ac107600 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.237+0000 7fa2b1f4f700 1 --2- 172.21.15.44:0/310248024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2ac10f1c0 0x7fa2ac107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.237+0000 7fa2b1f4f700 1 -- 172.21.15.44:0/310248024 >> 172.21.15.44:0/310248024 conn(0x7fa2ac06c6c0 msgr2=0x7fa2ac06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.241+0000 7fa2b1f4f700 1 -- 172.21.15.44:0/310248024 shutdown_connections 2023-09-10T01:23:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.241+0000 7fa2b1f4f700 1 -- 172.21.15.44:0/310248024 wait complete. 2023-09-10T01:23:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.249+0000 7fa2b1f4f700 1 Processor -- start 2023-09-10T01:23:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.249+0000 7fa2b1f4f700 1 -- start start 2023-09-10T01:23:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.250+0000 7fa2b1f4f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac10ed50 0x7fa2ac1ab550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.250+0000 7fa2b1f4f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2ac10f1c0 0x7fa2ac1aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.250+0000 7fa2b1f4f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2ac119de0 0x7fa2ac1a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.250+0000 7fa2b1f4f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac1a5a20 con 0x7fa2ac119de0 2023-09-10T01:23:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.250+0000 7fa2b1f4f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac1a5b90 con 0x7fa2ac10ed50 2023-09-10T01:23:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.250+0000 7fa2b1f4f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac1a5d50 con 0x7fa2ac10f1c0 2023-09-10T01:23:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.250+0000 7fa2ab7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac10ed50 0x7fa2ac1ab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.250+0000 7fa2ab7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac10ed50 0x7fa2ac1ab550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60746/0 (socket says 172.21.15.44:60746) 2023-09-10T01:23:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.250+0000 7fa2ab7fe700 1 -- 172.21.15.44:0/3390531335 learned_addr learned my addr 172.21.15.44:0/3390531335 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.250+0000 7fa2abfff700 1 --2- 172.21.15.44:0/3390531335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2ac119de0 0x7fa2ac1a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.250+0000 7fa2aaffd700 1 --2- 172.21.15.44:0/3390531335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2ac10f1c0 0x7fa2ac1aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.251+0000 7fa2abfff700 1 -- 172.21.15.44:0/3390531335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2ac10f1c0 msgr2=0x7fa2ac1aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.251+0000 7fa2abfff700 1 --2- 172.21.15.44:0/3390531335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2ac10f1c0 0x7fa2ac1aba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.251+0000 7fa2abfff700 1 -- 172.21.15.44:0/3390531335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac10ed50 msgr2=0x7fa2ac1ab550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.251+0000 7fa2abfff700 1 --2- 172.21.15.44:0/3390531335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac10ed50 0x7fa2ac1ab550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.251+0000 7fa2abfff700 1 -- 172.21.15.44:0/3390531335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a00095f0 con 0x7fa2ac119de0 2023-09-10T01:23:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.251+0000 7fa2ab7fe700 1 --2- 172.21.15.44:0/3390531335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac10ed50 0x7fa2ac1ab550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:23:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.251+0000 7fa2abfff700 1 --2- 172.21.15.44:0/3390531335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2ac119de0 0x7fa2ac1a53b0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400bad0 tx=0x7fa2a400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.252+0000 7fa2a8ff9700 1 -- 172.21.15.44:0/3390531335 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400ca30 con 0x7fa2ac119de0 2023-09-10T01:23:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.252+0000 7fa2a8ff9700 1 -- 172.21.15.44:0/3390531335 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2a400cb90 con 0x7fa2ac119de0 2023-09-10T01:23:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.252+0000 7fa2a8ff9700 1 -- 172.21.15.44:0/3390531335 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40125a0 con 0x7fa2ac119de0 2023-09-10T01:23:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.260+0000 7fa2b1f4f700 1 -- 172.21.15.44:0/3390531335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ac1a6030 con 0x7fa2ac119de0 2023-09-10T01:23:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.260+0000 7fa2b1f4f700 1 -- 172.21.15.44:0/3390531335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac1b2540 con 0x7fa2ac119de0 2023-09-10T01:23:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.267+0000 7fa2a8ff9700 1 -- 172.21.15.44:0/3390531335 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa2a400cd00 con 0x7fa2ac119de0 2023-09-10T01:23:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.268+0000 7fa2a8ff9700 1 --2- 172.21.15.44:0/3390531335 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa2940a0940 0x7fa2940a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.268+0000 7fa2a8ff9700 1 -- 172.21.15.44:0/3390531335 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fa2a40cb870 con 0x7fa2ac119de0 2023-09-10T01:23:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.268+0000 7fa2ab7fe700 1 --2- 172.21.15.44:0/3390531335 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa2940a0940 0x7fa2940a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.268+0000 7fa2ab7fe700 1 --2- 172.21.15.44:0/3390531335 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa2940a0940 0x7fa2940a2e00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000b5c0 tx=0x7fa2a0005fb0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.269+0000 7fa2be115b80 1 Processor -- start 2023-09-10T01:23:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.269+0000 7fa2be115b80 1 -- start start 2023-09-10T01:23:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.270+0000 7fa2be115b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a41071a0 0x5596a4107580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.270+0000 7fa2be115b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a4101300 0x5596a4101780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.270+0000 7fa2be115b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a4101cc0 0x5596a410df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.270+0000 7fa2be115b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a405a6d0 con 0x5596a41071a0 2023-09-10T01:23:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.270+0000 7fa2be115b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a4057ff0 con 0x5596a4101300 2023-09-10T01:23:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.270+0000 7fa2be115b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a40538e0 con 0x5596a4101cc0 2023-09-10T01:23:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.270+0000 7fa2ab7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a41071a0 0x5596a4107580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.270+0000 7fa2ab7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a41071a0 0x5596a4107580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59014/0 (socket says 172.21.15.44:59014) 2023-09-10T01:23:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.270+0000 7fa2ab7fe700 1 -- 172.21.15.44:0/2224681912 learned_addr learned my addr 172.21.15.44:0/2224681912 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.270+0000 7fa2aaffd700 1 --2- 172.21.15.44:0/2224681912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a4101300 0x5596a4101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.270+0000 7fa2abfff700 1 --2- 172.21.15.44:0/2224681912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a4101cc0 0x5596a410df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.270+0000 7fa2ab7fe700 1 -- 172.21.15.44:0/2224681912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a4101cc0 msgr2=0x5596a410df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.270+0000 7fa2ab7fe700 1 --2- 172.21.15.44:0/2224681912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a4101cc0 0x5596a410df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.270+0000 7fa2ab7fe700 1 -- 172.21.15.44:0/2224681912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a4101300 msgr2=0x5596a4101780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.270+0000 7fa2ab7fe700 1 --2- 172.21.15.44:0/2224681912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a4101300 0x5596a4101780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.270+0000 7fa2ab7fe700 1 -- 172.21.15.44:0/2224681912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596a3e7fd80 con 0x5596a41071a0 2023-09-10T01:23:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.271+0000 7fa2ab7fe700 1 --2- 172.21.15.44:0/2224681912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a41071a0 0x5596a4107580 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0005170 tx=0x7fa2a0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9237aca6c173becd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.275+0000 7fa291ffb700 1 -- 172.21.15.44:0/2224681912 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a002a070 con 0x5596a41071a0 2023-09-10T01:23:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.275+0000 7fa291ffb700 1 -- 172.21.15.44:0/2224681912 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2a001e040 con 0x5596a41071a0 2023-09-10T01:23:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.275+0000 7fa291ffb700 1 -- 172.21.15.44:0/2224681912 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0004920 con 0x5596a41071a0 2023-09-10T01:23:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.276+0000 7fa2be115b80 1 -- 172.21.15.44:0/2224681912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a41071a0 msgr2=0x5596a4107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.276+0000 7fa2be115b80 1 --2- 172.21.15.44:0/2224681912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a41071a0 0x5596a4107580 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0005170 tx=0x7fa2a0020000 comp rx=0 tx=0).stop 2023-09-10T01:23:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.277+0000 7fa2be115b80 1 -- 172.21.15.44:0/2224681912 shutdown_connections 2023-09-10T01:23:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.277+0000 7fa2be115b80 1 --2- 172.21.15.44:0/2224681912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a4101cc0 0x5596a410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.277+0000 7fa2be115b80 1 --2- 172.21.15.44:0/2224681912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a41071a0 0x5596a4107580 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.277+0000 7fa2be115b80 1 --2- 172.21.15.44:0/2224681912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a4101300 0x5596a4101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.277+0000 7fa2be115b80 1 -- 172.21.15.44:0/2224681912 >> 172.21.15.44:0/2224681912 conn(0x5596a4076030 msgr2=0x5596a40fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.277+0000 7fa2be115b80 1 -- 172.21.15.44:0/2224681912 shutdown_connections 2023-09-10T01:23:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.277+0000 7fa2be115b80 1 -- 172.21.15.44:0/2224681912 wait complete. 2023-09-10T01:23:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.277+0000 7fa2be115b80 1 -- start start 2023-09-10T01:23:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.278+0000 7fa2be115b80 1 Processor -- start 2023-09-10T01:23:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.278+0000 7fa2be115b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a4101300 0x5596a410df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.278+0000 7fa2be115b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101cc0 0x5596a410b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.278+0000 7fa2be115b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a41071a0 0x5596a410b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.278+0000 7fa2be115b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a3ff1b10 con 0x5596a4101cc0 2023-09-10T01:23:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.278+0000 7fa2be115b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a3ff1810 con 0x5596a4101300 2023-09-10T01:23:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.278+0000 7fa2be115b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a3ed58c0 con 0x5596a41071a0 2023-09-10T01:23:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.278+0000 7fa2be115b80 10 client.0 _ll_register_callbacks cb 0x5596a3f3ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:23:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.278+0000 7fa2aaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101cc0 0x5596a410b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.278+0000 7fa2aaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101cc0 0x5596a410b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59022/0 (socket says 172.21.15.44:59022) 2023-09-10T01:23:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.278+0000 7fa2aaffd700 1 -- 172.21.15.44:0/1355935332 learned_addr learned my addr 172.21.15.44:0/1355935332 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.278+0000 7fa2abfff700 1 --2- 172.21.15.44:0/1355935332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a41071a0 0x5596a410b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.278+0000 7fa2aaffd700 1 -- 172.21.15.44:0/1355935332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a41071a0 msgr2=0x5596a410b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.278+0000 7fa2aaffd700 1 --2- 172.21.15.44:0/1355935332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a41071a0 0x5596a410b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.279+0000 7fa2aaffd700 1 -- 172.21.15.44:0/1355935332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a4101300 msgr2=0x5596a410df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.279+0000 7fa2aaffd700 1 --2- 172.21.15.44:0/1355935332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a4101300 0x5596a410df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.279+0000 7fa2aaffd700 1 -- 172.21.15.44:0/1355935332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac11cb00 con 0x5596a4101cc0 2023-09-10T01:23:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.279+0000 7fa2aaffd700 1 --2- 172.21.15.44:0/1355935332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101cc0 0x5596a410b1a0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fa29c010500 tx=0x7fa29c010810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ef2e3d63d04650 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.279+0000 7fa28affd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:23:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.279+0000 7fa28affd700 1 -- 172.21.15.44:0/1355935332 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c010ee0 con 0x5596a4101cc0 2023-09-10T01:23:15.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.279+0000 7fa28affd700 1 -- 172.21.15.44:0/1355935332 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(174..174 src has 1..174) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fa29c014640 con 0x5596a4101cc0 2023-09-10T01:23:15.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.279+0000 7fa2be115b80 1 -- 172.21.15.44:0/1355935332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596a3fab490 con 0x5596a4101cc0 2023-09-10T01:23:15.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.279+0000 7fa28affd700 1 -- 172.21.15.44:0/1355935332 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa29c014840 con 0x5596a4101cc0 2023-09-10T01:23:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.279+0000 7fa2be115b80 10 client.20037 fetch_fsmap learned FSMap version 140 2023-09-10T01:23:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.279+0000 7fa2be115b80 1 -- 172.21.15.44:0/1355935332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=4+}) v3 -- 0x5596a3e68c00 con 0x5596a4101cc0 2023-09-10T01:23:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.279+0000 7fa28affd700 1 -- 172.21.15.44:0/1355935332 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5596a3fab490 con 0x5596a4101cc0 2023-09-10T01:23:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.280+0000 7fa28affd700 1 -- 172.21.15.44:0/1355935332 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 140) v1 ==== 7375+0+0 (secure 0 0 0) 0x7fa29c0139d0 con 0x5596a4101cc0 2023-09-10T01:23:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.280+0000 7fa2be115b80 10 client.20037 fetch_fsmap finished waiting for FSMap version 140 2023-09-10T01:23:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.280+0000 7fa2be115b80 10 client.20037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:23:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.280+0000 7fa2be115b80 20 client.20037 populate_metadata read hostname 'smithi044' 2023-09-10T01:23:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.280+0000 7fa2be115b80 1 --2- 172.21.15.44:0/1355935332 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5596a411b580 0x5596a411d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.280+0000 7fa2be115b80 4 client.20037 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:23:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.280+0000 7fa2be115b80 1 -- 172.21.15.44:0/1355935332 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596a4068840 con 0x5596a411b580 2023-09-10T01:23:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.281+0000 7fa2ab7fe700 1 --2- 172.21.15.44:0/1355935332 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5596a411b580 0x5596a411d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.281+0000 7fa28affd700 10 client.20037 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.281+0000 7fa2ab7fe700 1 --2- 172.21.15.44:0/1355935332 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5596a411b580 0x5596a411d9a0 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6590fd0f8d69c568 server_cookie=c57039ea8017184c in_seq=0 out_seq=0 2023-09-10T01:23:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.282+0000 7fa28affd700 1 -- 172.21.15.44:0/1355935332 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5596a4068840 con 0x5596a411b580 2023-09-10T01:23:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.282+0000 7fa28affd700 10 client.20037 handle_command_reply: tid=0 2023-09-10T01:23:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.282+0000 7fa2be115b80 1 client.20037 shutdown 2023-09-10T01:23:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.282+0000 7fa2be115b80 1 -- 172.21.15.44:0/1355935332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101cc0 msgr2=0x5596a410b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.282+0000 7fa2be115b80 1 --2- 172.21.15.44:0/1355935332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101cc0 0x5596a410b1a0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fa29c010500 tx=0x7fa29c010810 comp rx=0 tx=0).stop 2023-09-10T01:23:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.282+0000 7fa2be115b80 1 -- 172.21.15.44:0/1355935332 shutdown_connections 2023-09-10T01:23:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.282+0000 7fa2be115b80 1 --2- 172.21.15.44:0/1355935332 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5596a411b580 0x5596a411d9a0 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.282+0000 7fa2be115b80 1 --2- 172.21.15.44:0/1355935332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a41071a0 0x5596a410b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.282+0000 7fa2be115b80 1 --2- 172.21.15.44:0/1355935332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a4101300 0x5596a410df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.282+0000 7fa28affd700 0 client.20037 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.282+0000 7fa2be115b80 1 --2- 172.21.15.44:0/1355935332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101cc0 0x5596a410b1a0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.282+0000 7fa2be115b80 1 -- 172.21.15.44:0/1355935332 >> 172.21.15.44:0/1355935332 conn(0x5596a4076030 msgr2=0x5596a40fb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.282+0000 7fa2be115b80 1 -- 172.21.15.44:0/1355935332 shutdown_connections 2023-09-10T01:23:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.282+0000 7fa2be115b80 1 -- 172.21.15.44:0/1355935332 wait complete. 2023-09-10T01:23:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.282+0000 7fa2be115b80 20 client.20037 trim_cache size 0 max 16384 2023-09-10T01:23:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.308+0000 7fa2be115b80 1 Processor -- start 2023-09-10T01:23:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2be115b80 1 -- start start 2023-09-10T01:23:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2be115b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101300 0x5596a4110a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2be115b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a4101cc0 0x5596a410a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2be115b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a41071a0 0x5596a41051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2be115b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a4106a20 con 0x5596a4101300 2023-09-10T01:23:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2be115b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a4110fb0 con 0x5596a4101cc0 2023-09-10T01:23:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2be115b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a410c180 con 0x5596a41071a0 2023-09-10T01:23:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2ab7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101300 0x5596a4110a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2ab7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101300 0x5596a4110a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59028/0 (socket says 172.21.15.44:59028) 2023-09-10T01:23:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2ab7fe700 1 -- 172.21.15.44:0/591323166 learned_addr learned my addr 172.21.15.44:0/591323166 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2aaffd700 1 --2- 172.21.15.44:0/591323166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a4101cc0 0x5596a410a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2abfff700 1 --2- 172.21.15.44:0/591323166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a41071a0 0x5596a41051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2aaffd700 1 -- 172.21.15.44:0/591323166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a41071a0 msgr2=0x5596a41051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2aaffd700 1 --2- 172.21.15.44:0/591323166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a41071a0 0x5596a41051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2aaffd700 1 -- 172.21.15.44:0/591323166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101300 msgr2=0x5596a4110a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2aaffd700 1 --2- 172.21.15.44:0/591323166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101300 0x5596a4110a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.309+0000 7fa2aaffd700 1 -- 172.21.15.44:0/591323166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596a3ea61f0 con 0x5596a4101cc0 2023-09-10T01:23:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.310+0000 7fa2aaffd700 1 --2- 172.21.15.44:0/591323166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a4101cc0 0x5596a410a710 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa29c013d60 tx=0x7fa29c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb87c2c32deb2490 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.310+0000 7fa2927fc700 1 -- 172.21.15.44:0/591323166 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c006d00 con 0x5596a4101cc0 2023-09-10T01:23:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.311+0000 7fa2927fc700 1 -- 172.21.15.44:0/591323166 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa29c010c10 con 0x5596a4101cc0 2023-09-10T01:23:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.311+0000 7fa2927fc700 1 -- 172.21.15.44:0/591323166 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c0135e0 con 0x5596a4101cc0 2023-09-10T01:23:15.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.311+0000 7fa2be115b80 1 -- 172.21.15.44:0/591323166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a4101cc0 msgr2=0x5596a410a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:15.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.311+0000 7fa2be115b80 1 --2- 172.21.15.44:0/591323166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a4101cc0 0x5596a410a710 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa29c013d60 tx=0x7fa29c004750 comp rx=0 tx=0).stop 2023-09-10T01:23:15.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.311+0000 7fa2be115b80 1 -- 172.21.15.44:0/591323166 shutdown_connections 2023-09-10T01:23:15.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.311+0000 7fa2be115b80 1 --2- 172.21.15.44:0/591323166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a41071a0 0x5596a41051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.311+0000 7fa2be115b80 1 --2- 172.21.15.44:0/591323166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101300 0x5596a4110a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.311+0000 7fa2be115b80 1 --2- 172.21.15.44:0/591323166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a4101cc0 0x5596a410a710 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.311+0000 7fa2be115b80 1 -- 172.21.15.44:0/591323166 >> 172.21.15.44:0/591323166 conn(0x5596a4076030 msgr2=0x5596a4122340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.312+0000 7fa2be115b80 1 -- 172.21.15.44:0/591323166 shutdown_connections 2023-09-10T01:23:15.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.315+0000 7fa2be115b80 1 -- 172.21.15.44:0/591323166 wait complete. 2023-09-10T01:23:15.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.315+0000 7fa2be115b80 1 -- start start 2023-09-10T01:23:15.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.315+0000 7fa2be115b80 1 Processor -- start 2023-09-10T01:23:15.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.316+0000 7fa2be115b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a4101300 0x5596a4179c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.316+0000 7fa2be115b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101cc0 0x5596a417a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.316+0000 7fa2be115b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a41071a0 0x5596a4174390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.316+0000 7fa2be115b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a41749a0 con 0x5596a4101cc0 2023-09-10T01:23:15.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.316+0000 7fa2be115b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a4174b10 con 0x5596a41071a0 2023-09-10T01:23:15.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.316+0000 7fa2be115b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a4174cd0 con 0x5596a4101300 2023-09-10T01:23:15.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.316+0000 7fa2be115b80 10 client.0 _ll_register_callbacks cb 0x5596a3f3ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:23:15.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.316+0000 7fa2abfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a41071a0 0x5596a4174390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:15.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.316+0000 7fa2abfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a41071a0 0x5596a4174390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60778/0 (socket says 172.21.15.44:60778) 2023-09-10T01:23:15.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.316+0000 7fa2abfff700 1 -- 172.21.15.44:0/3984884559 learned_addr learned my addr 172.21.15.44:0/3984884559 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:15.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.316+0000 7fa2abfff700 1 -- 172.21.15.44:0/3984884559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a4101300 msgr2=0x5596a4179c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:23:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.316+0000 7fa2abfff700 1 --2- 172.21.15.44:0/3984884559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a4101300 0x5596a4179c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.317+0000 7fa2abfff700 1 -- 172.21.15.44:0/3984884559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101cc0 msgr2=0x5596a417a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.317+0000 7fa2abfff700 1 --2- 172.21.15.44:0/3984884559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101cc0 0x5596a417a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.317+0000 7fa2abfff700 1 -- 172.21.15.44:0/3984884559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596a3e68c00 con 0x5596a41071a0 2023-09-10T01:23:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.317+0000 7fa2abfff700 1 --2- 172.21.15.44:0/3984884559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a41071a0 0x5596a4174390 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400bf00 tx=0x7fa2a400c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52d7c1ee5a9c7604 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.317+0000 7fa28affd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:23:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.317+0000 7fa28affd700 1 -- 172.21.15.44:0/3984884559 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40b4070 con 0x5596a41071a0 2023-09-10T01:23:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.317+0000 7fa28affd700 1 -- 172.21.15.44:0/3984884559 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(174..174 src has 1..174) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fa2a4098e50 con 0x5596a41071a0 2023-09-10T01:23:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.317+0000 7fa2be115b80 1 -- 172.21.15.44:0/3984884559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2a001e040 con 0x5596a41071a0 2023-09-10T01:23:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.317+0000 7fa28affd700 1 -- 172.21.15.44:0/3984884559 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2a4097ed0 con 0x5596a41071a0 2023-09-10T01:23:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.317+0000 7fa2be115b80 10 client.29672 fetch_fsmap learned FSMap version 140 2023-09-10T01:23:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.317+0000 7fa2be115b80 1 -- 172.21.15.44:0/3984884559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=4+}) v3 -- 0x5596a3e7e250 con 0x5596a41071a0 2023-09-10T01:23:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.317+0000 7fa28affd700 1 -- 172.21.15.44:0/3984884559 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2a001e040 con 0x5596a41071a0 2023-09-10T01:23:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.318+0000 7fa28affd700 1 -- 172.21.15.44:0/3984884559 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 140) v1 ==== 7375+0+0 (secure 0 0 0) 0x7fa2a40947d0 con 0x5596a41071a0 2023-09-10T01:23:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.318+0000 7fa2be115b80 10 client.29672 fetch_fsmap finished waiting for FSMap version 140 2023-09-10T01:23:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.318+0000 7fa2be115b80 10 client.29672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:23:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.318+0000 7fa2be115b80 20 client.29672 populate_metadata read hostname 'smithi044' 2023-09-10T01:23:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.318+0000 7fa2be115b80 1 --2- 172.21.15.44:0/3984884559 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5596a411b580 0x5596a41a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.318+0000 7fa2be115b80 4 client.29672 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:23:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.318+0000 7fa2be115b80 1 -- 172.21.15.44:0/3984884559 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5596a404c8c0 con 0x5596a411b580 2023-09-10T01:23:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.318+0000 7fa2ab7fe700 1 --2- 172.21.15.44:0/3984884559 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5596a411b580 0x5596a41a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.318+0000 7fa28affd700 10 client.29672 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.318+0000 7fa2ab7fe700 1 --2- 172.21.15.44:0/3984884559 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5596a411b580 0x5596a41a5b70 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f68b187971bf0ea6 server_cookie=127202882c40a7e2 in_seq=0 out_seq=0 2023-09-10T01:23:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.363+0000 7fa28affd700 1 -- 172.21.15.44:0/3984884559 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5596a404c8c0 con 0x5596a411b580 2023-09-10T01:23:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.363+0000 7fa28affd700 10 client.29672 handle_command_reply: tid=0 2023-09-10T01:23:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.363+0000 7fa2be115b80 1 client.29672 shutdown 2023-09-10T01:23:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.363+0000 7fa2be115b80 1 -- 172.21.15.44:0/3984884559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a41071a0 msgr2=0x5596a4174390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.363+0000 7fa2be115b80 1 --2- 172.21.15.44:0/3984884559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a41071a0 0x5596a4174390 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400bf00 tx=0x7fa2a400c4a0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.363+0000 7fa2be115b80 1 -- 172.21.15.44:0/3984884559 shutdown_connections 2023-09-10T01:23:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.363+0000 7fa2be115b80 1 --2- 172.21.15.44:0/3984884559 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5596a411b580 0x5596a41a5b70 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.363+0000 7fa2be115b80 1 --2- 172.21.15.44:0/3984884559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596a41071a0 0x5596a4174390 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.363+0000 7fa28affd700 0 client.29672 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.363+0000 7fa2be115b80 1 --2- 172.21.15.44:0/3984884559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596a4101300 0x5596a4179c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.363+0000 7fa2be115b80 1 --2- 172.21.15.44:0/3984884559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596a4101cc0 0x5596a417a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.363+0000 7fa2be115b80 1 -- 172.21.15.44:0/3984884559 >> 172.21.15.44:0/3984884559 conn(0x5596a4076030 msgr2=0x5596a41051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.363+0000 7fa2be115b80 1 -- 172.21.15.44:0/3984884559 shutdown_connections 2023-09-10T01:23:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.363+0000 7fa2be115b80 1 -- 172.21.15.44:0/3984884559 wait complete. 2023-09-10T01:23:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.364+0000 7fa2be115b80 20 client.29672 trim_cache size 0 max 16384 2023-09-10T01:23:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:23:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:23:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:23:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:23:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.367+0000 7fa2b1f4f700 1 -- 172.21.15.44:0/3390531335 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa2940a0940 msgr2=0x7fa2940a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.367+0000 7fa2b1f4f700 1 --2- 172.21.15.44:0/3390531335 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa2940a0940 0x7fa2940a2e00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000b5c0 tx=0x7fa2a0005fb0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.367+0000 7fa2b1f4f700 1 -- 172.21.15.44:0/3390531335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2ac119de0 msgr2=0x7fa2ac1a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.367+0000 7fa2b1f4f700 1 --2- 172.21.15.44:0/3390531335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2ac119de0 0x7fa2ac1a53b0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400bad0 tx=0x7fa2a400bde0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.367+0000 7fa2b1f4f700 1 -- 172.21.15.44:0/3390531335 shutdown_connections 2023-09-10T01:23:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.367+0000 7fa2b1f4f700 1 --2- 172.21.15.44:0/3390531335 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa2940a0940 0x7fa2940a2e00 secure :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000b5c0 tx=0x7fa2a0005fb0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.367+0000 7fa2b1f4f700 1 --2- 172.21.15.44:0/3390531335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2ac119de0 0x7fa2ac1a53b0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.367+0000 7fa2b1f4f700 1 --2- 172.21.15.44:0/3390531335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac10ed50 0x7fa2ac1ab550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.367+0000 7fa2b1f4f700 1 --2- 172.21.15.44:0/3390531335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2ac10f1c0 0x7fa2ac1aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.367+0000 7fa2b1f4f700 1 -- 172.21.15.44:0/3390531335 >> 172.21.15.44:0/3390531335 conn(0x7fa2ac06c6c0 msgr2=0x7fa2ac10c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.367+0000 7fa2b1f4f700 1 -- 172.21.15.44:0/3390531335 shutdown_connections 2023-09-10T01:23:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:15.368+0000 7fa2b1f4f700 1 -- 172.21.15.44:0/3390531335 wait complete. 2023-09-10T01:23:15.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:15 smithi131 ceph-mon[103713]: pgmap v1377: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 6.9 MiB/s wr, 528 op/s 2023-09-10T01:23:15.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:15 smithi044 ceph-mon[98676]: pgmap v1377: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 6.9 MiB/s wr, 528 op/s 2023-09-10T01:23:15.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:15 smithi099 ceph-mon[102692]: pgmap v1377: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 17 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 6.9 MiB/s wr, 528 op/s 2023-09-10T01:23:16.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 130.0 s, 256.4 tps, lat 420.086 ms stddev 373.957 2023-09-10T01:23:17.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:17 smithi131 ceph-mon[103713]: pgmap v1378: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 8.1 MiB/s wr, 626 op/s 2023-09-10T01:23:17.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:17 smithi131 ceph-mon[103713]: osdmap e175: 12 total, 12 up, 12 in 2023-09-10T01:23:17.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:17 smithi044 ceph-mon[98676]: pgmap v1378: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 8.1 MiB/s wr, 626 op/s 2023-09-10T01:23:17.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:17 smithi044 ceph-mon[98676]: osdmap e175: 12 total, 12 up, 12 in 2023-09-10T01:23:17.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:17 smithi099 ceph-mon[102692]: pgmap v1378: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 8.1 MiB/s wr, 626 op/s 2023-09-10T01:23:17.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:17 smithi099 ceph-mon[102692]: osdmap e175: 12 total, 12 up, 12 in 2023-09-10T01:23:19.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:19 smithi131 ceph-mon[103713]: pgmap v1380: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 7.2 MiB/s wr, 562 op/s 2023-09-10T01:23:19.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:19 smithi044 ceph-mon[98676]: pgmap v1380: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 7.2 MiB/s wr, 562 op/s 2023-09-10T01:23:19.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:19 smithi099 ceph-mon[102692]: pgmap v1380: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 7.2 MiB/s wr, 562 op/s 2023-09-10T01:23:21.188 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 135.0 s, 315.0 tps, lat 320.501 ms stddev 146.439 2023-09-10T01:23:21.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:21 smithi131 ceph-mon[103713]: pgmap v1381: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 7.2 MiB/s wr, 562 op/s 2023-09-10T01:23:21.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:21 smithi131 ceph-mon[103713]: osdmap e176: 12 total, 12 up, 12 in 2023-09-10T01:23:21.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:21 smithi044 ceph-mon[98676]: pgmap v1381: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 7.2 MiB/s wr, 562 op/s 2023-09-10T01:23:21.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:21 smithi044 ceph-mon[98676]: osdmap e176: 12 total, 12 up, 12 in 2023-09-10T01:23:21.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:21 smithi099 ceph-mon[102692]: pgmap v1381: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 7.2 MiB/s wr, 562 op/s 2023-09-10T01:23:21.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:21 smithi099 ceph-mon[102692]: osdmap e176: 12 total, 12 up, 12 in 2023-09-10T01:23:23.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:23 smithi131 ceph-mon[103713]: pgmap v1383: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 886 op/s 2023-09-10T01:23:23.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:23 smithi044 ceph-mon[98676]: pgmap v1383: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 886 op/s 2023-09-10T01:23:23.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:23 smithi099 ceph-mon[102692]: pgmap v1383: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 886 op/s 2023-09-10T01:23:25.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:25 smithi131 ceph-mon[103713]: pgmap v1384: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 8.0 MiB/s wr, 644 op/s 2023-09-10T01:23:25.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:25 smithi044 ceph-mon[98676]: pgmap v1384: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 8.0 MiB/s wr, 644 op/s 2023-09-10T01:23:25.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:25 smithi099 ceph-mon[102692]: pgmap v1384: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 8.0 MiB/s wr, 644 op/s 2023-09-10T01:23:26.184 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 140.0 s, 119.1 tps, lat 205.947 ms stddev 142.809 2023-09-10T01:23:27.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:27 smithi131 ceph-mon[103713]: pgmap v1385: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 7.9 MiB/s wr, 627 op/s 2023-09-10T01:23:27.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:27 smithi044 ceph-mon[98676]: pgmap v1385: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 7.9 MiB/s wr, 627 op/s 2023-09-10T01:23:27.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:27 smithi099 ceph-mon[102692]: pgmap v1385: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 7.9 MiB/s wr, 627 op/s 2023-09-10T01:23:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.539+0000 7f92e82ce700 1 -- 172.21.15.44:0/2926068401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e01055b0 msgr2=0x7f92e0105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.539+0000 7f92e82ce700 1 --2- 172.21.15.44:0/2926068401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e01055b0 0x7f92e0105990 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0099c0 tx=0x7f92dc009cd0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.539+0000 7f92e82ce700 1 -- 172.21.15.44:0/2926068401 shutdown_connections 2023-09-10T01:23:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.539+0000 7f92e82ce700 1 --2- 172.21.15.44:0/2926068401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92e0100120 0x7f92e010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.539+0000 7f92e82ce700 1 --2- 172.21.15.44:0/2926068401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e01055b0 0x7f92e0105990 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.539+0000 7f92e82ce700 1 --2- 172.21.15.44:0/2926068401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e00ff760 0x7f92e00ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.539+0000 7f92e82ce700 1 -- 172.21.15.44:0/2926068401 >> 172.21.15.44:0/2926068401 conn(0x7f92e0074f30 msgr2=0x7f92e0075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.539+0000 7f92e82ce700 1 -- 172.21.15.44:0/2926068401 shutdown_connections 2023-09-10T01:23:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.539+0000 7f92e82ce700 1 -- 172.21.15.44:0/2926068401 wait complete. 2023-09-10T01:23:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.539+0000 7f92e82ce700 1 Processor -- start 2023-09-10T01:23:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.539+0000 7f92e82ce700 1 -- start start 2023-09-10T01:23:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e82ce700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92e00ff760 0x7f92e0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e82ce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e0100120 0x7f92e0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e82ce700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e01055b0 0x7f92e019d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e82ce700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e019d910 con 0x7f92e0100120 2023-09-10T01:23:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e82ce700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e019da80 con 0x7f92e01055b0 2023-09-10T01:23:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e82ce700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e019dc40 con 0x7f92e00ff760 2023-09-10T01:23:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e5869700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e0100120 0x7f92e0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e606a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92e00ff760 0x7f92e0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e5869700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e0100120 0x7f92e0198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59050/0 (socket says 172.21.15.44:59050) 2023-09-10T01:23:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e5869700 1 -- 172.21.15.44:0/3512198173 learned_addr learned my addr 172.21.15.44:0/3512198173 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e686b700 1 --2- 172.21.15.44:0/3512198173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e01055b0 0x7f92e019d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e5869700 1 -- 172.21.15.44:0/3512198173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92e00ff760 msgr2=0x7f92e0198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e5869700 1 --2- 172.21.15.44:0/3512198173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92e00ff760 0x7f92e0198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e5869700 1 -- 172.21.15.44:0/3512198173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e01055b0 msgr2=0x7f92e019d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e5869700 1 --2- 172.21.15.44:0/3512198173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e01055b0 0x7f92e019d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e5869700 1 -- 172.21.15.44:0/3512198173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc0095f0 con 0x7f92e0100120 2023-09-10T01:23:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e606a700 1 --2- 172.21.15.44:0/3512198173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92e00ff760 0x7f92e0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:23:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92e5869700 1 --2- 172.21.15.44:0/3512198173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e0100120 0x7f92e0198ca0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f92d000d980 tx=0x7f92d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.540+0000 7f92d77fe700 1 -- 172.21.15.44:0/3512198173 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d0004220 con 0x7f92e0100120 2023-09-10T01:23:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.541+0000 7f92d77fe700 1 -- 172.21.15.44:0/3512198173 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92d0007640 con 0x7f92e0100120 2023-09-10T01:23:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.541+0000 7f92e82ce700 1 -- 172.21.15.44:0/3512198173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e019df20 con 0x7f92e0100120 2023-09-10T01:23:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.541+0000 7f92d77fe700 1 -- 172.21.15.44:0/3512198173 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d0005660 con 0x7f92e0100120 2023-09-10T01:23:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.541+0000 7f92e82ce700 1 -- 172.21.15.44:0/3512198173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e01a57d0 con 0x7f92e0100120 2023-09-10T01:23:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.543+0000 7f92d77fe700 1 -- 172.21.15.44:0/3512198173 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f92d0004750 con 0x7f92e0100120 2023-09-10T01:23:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.544+0000 7f92d77fe700 1 --2- 172.21.15.44:0/3512198173 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f92cc0a07a0 0x7f92cc0a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.544+0000 7f92d77fe700 1 -- 172.21.15.44:0/3512198173 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f92d00cc180 con 0x7f92e0100120 2023-09-10T01:23:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.544+0000 7f92e606a700 1 --2- 172.21.15.44:0/3512198173 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f92cc0a07a0 0x7f92cc0a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92f4494b80 1 Processor -- start 2023-09-10T01:23:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92f4494b80 1 -- start start 2023-09-10T01:23:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92e606a700 1 --2- 172.21.15.44:0/3512198173 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f92cc0a07a0 0x7f92cc0a2c60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00b580 tx=0x7f92dc009f10 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92f4494b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d751a0 0x55de99d75580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92f4494b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6f300 0x55de99d6f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92f4494b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d6fcc0 0x55de99d7bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92f4494b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de99cc86d0 con 0x55de99d751a0 2023-09-10T01:23:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92f4494b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de99cc5ff0 con 0x55de99d6fcc0 2023-09-10T01:23:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92f4494b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de99cc18e0 con 0x55de99d6f300 2023-09-10T01:23:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92e606a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d751a0 0x55de99d75580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92e606a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d751a0 0x55de99d75580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59056/0 (socket says 172.21.15.44:59056) 2023-09-10T01:23:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92e606a700 1 -- 172.21.15.44:0/391880208 learned_addr learned my addr 172.21.15.44:0/391880208 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92e5869700 1 --2- 172.21.15.44:0/391880208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6f300 0x55de99d6f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92e606a700 1 -- 172.21.15.44:0/391880208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6f300 msgr2=0x55de99d6f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92e686b700 1 --2- 172.21.15.44:0/391880208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d6fcc0 0x55de99d7bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92e606a700 1 --2- 172.21.15.44:0/391880208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6f300 0x55de99d6f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92e606a700 1 -- 172.21.15.44:0/391880208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d6fcc0 msgr2=0x55de99d7bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92e606a700 1 --2- 172.21.15.44:0/391880208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d6fcc0 0x55de99d7bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92e606a700 1 -- 172.21.15.44:0/391880208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de99aedd80 con 0x55de99d751a0 2023-09-10T01:23:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92e5869700 1 --2- 172.21.15.44:0/391880208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6f300 0x55de99d6f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:23:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.545+0000 7f92e606a700 1 --2- 172.21.15.44:0/391880208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d751a0 0x55de99d75580 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0099c0 tx=0x7f92dc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a37856f5d8b4a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.546+0000 7f92d4d89700 1 -- 172.21.15.44:0/391880208 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc02b070 con 0x55de99d751a0 2023-09-10T01:23:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.546+0000 7f92d4d89700 1 -- 172.21.15.44:0/391880208 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92dc01e040 con 0x55de99d751a0 2023-09-10T01:23:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.546+0000 7f92d4d89700 1 -- 172.21.15.44:0/391880208 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc004710 con 0x55de99d751a0 2023-09-10T01:23:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.546+0000 7f92f4494b80 1 -- 172.21.15.44:0/391880208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d751a0 msgr2=0x55de99d75580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.546+0000 7f92f4494b80 1 --2- 172.21.15.44:0/391880208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d751a0 0x55de99d75580 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0099c0 tx=0x7f92dc020000 comp rx=0 tx=0).stop 2023-09-10T01:23:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.546+0000 7f92f4494b80 1 -- 172.21.15.44:0/391880208 shutdown_connections 2023-09-10T01:23:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.546+0000 7f92f4494b80 1 --2- 172.21.15.44:0/391880208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d6fcc0 0x55de99d7bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.546+0000 7f92f4494b80 1 --2- 172.21.15.44:0/391880208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d751a0 0x55de99d75580 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.546+0000 7f92f4494b80 1 --2- 172.21.15.44:0/391880208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6f300 0x55de99d6f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.546+0000 7f92f4494b80 1 -- 172.21.15.44:0/391880208 >> 172.21.15.44:0/391880208 conn(0x55de99ce4030 msgr2=0x55de99d6a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.547+0000 7f92f4494b80 1 -- 172.21.15.44:0/391880208 shutdown_connections 2023-09-10T01:23:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.547+0000 7f92f4494b80 1 -- 172.21.15.44:0/391880208 wait complete. 2023-09-10T01:23:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.547+0000 7f92f4494b80 1 -- start start 2023-09-10T01:23:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.547+0000 7f92f4494b80 1 Processor -- start 2023-09-10T01:23:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.547+0000 7f92f4494b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6f300 0x55de99d7be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.547+0000 7f92f4494b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d6fcc0 0x55de99d790b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.547+0000 7f92f4494b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d751a0 0x55de99d79730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.547+0000 7f92f4494b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de99c5fb10 con 0x55de99d6fcc0 2023-09-10T01:23:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.547+0000 7f92f4494b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de99c5f810 con 0x55de99d751a0 2023-09-10T01:23:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.547+0000 7f92f4494b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de99b438c0 con 0x55de99d6f300 2023-09-10T01:23:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.547+0000 7f92f4494b80 10 client.0 _ll_register_callbacks cb 0x55de99ba9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:23:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.547+0000 7f92e5869700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d6fcc0 0x55de99d790b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.547+0000 7f92e5869700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d6fcc0 0x55de99d790b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59064/0 (socket says 172.21.15.44:59064) 2023-09-10T01:23:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.547+0000 7f92e5869700 1 -- 172.21.15.44:0/2682684091 learned_addr learned my addr 172.21.15.44:0/2682684091 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.547+0000 7f92e686b700 1 --2- 172.21.15.44:0/2682684091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d751a0 0x55de99d79730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.548+0000 7f92e606a700 1 --2- 172.21.15.44:0/2682684091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6f300 0x55de99d7be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.548+0000 7f92e5869700 1 -- 172.21.15.44:0/2682684091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6f300 msgr2=0x55de99d7be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.548+0000 7f92e5869700 1 --2- 172.21.15.44:0/2682684091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6f300 0x55de99d7be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.548+0000 7f92e5869700 1 -- 172.21.15.44:0/2682684091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d751a0 msgr2=0x55de99d79730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.548+0000 7f92e5869700 1 --2- 172.21.15.44:0/2682684091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d751a0 0x55de99d79730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.548+0000 7f92e5869700 1 -- 172.21.15.44:0/2682684091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e010cbc0 con 0x55de99d6fcc0 2023-09-10T01:23:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.548+0000 7f92e5869700 1 --2- 172.21.15.44:0/2682684091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d6fcc0 0x55de99d790b0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f92d0098f40 tx=0x7f92d0004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b91309fa3db00b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.548+0000 7f92c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:23:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.548+0000 7f92c9ffb700 1 -- 172.21.15.44:0/2682684091 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d0099240 con 0x55de99d6fcc0 2023-09-10T01:23:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.548+0000 7f92c9ffb700 1 -- 172.21.15.44:0/2682684091 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(176..176 src has 1..176) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f92d009d070 con 0x55de99d6fcc0 2023-09-10T01:23:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.548+0000 7f92f4494b80 1 -- 172.21.15.44:0/2682684091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de99c19490 con 0x55de99d6fcc0 2023-09-10T01:23:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.548+0000 7f92c9ffb700 1 -- 172.21.15.44:0/2682684091 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92d00a12b0 con 0x55de99d6fcc0 2023-09-10T01:23:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.548+0000 7f92f4494b80 10 client.20052 fetch_fsmap learned FSMap version 140 2023-09-10T01:23:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.548+0000 7f92f4494b80 1 -- 172.21.15.44:0/2682684091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=4+}) v3 -- 0x55de99ad6c00 con 0x55de99d6fcc0 2023-09-10T01:23:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.548+0000 7f92c9ffb700 1 -- 172.21.15.44:0/2682684091 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92d00b7030 con 0x55de99d6fcc0 2023-09-10T01:23:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.549+0000 7f92c9ffb700 1 -- 172.21.15.44:0/2682684091 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 140) v1 ==== 7375+0+0 (secure 0 0 0) 0x7f92d00a0eb0 con 0x55de99d6fcc0 2023-09-10T01:23:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.549+0000 7f92f4494b80 10 client.20052 fetch_fsmap finished waiting for FSMap version 140 2023-09-10T01:23:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.549+0000 7f92f4494b80 10 client.20052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:23:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.549+0000 7f92f4494b80 20 client.20052 populate_metadata read hostname 'smithi044' 2023-09-10T01:23:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.549+0000 7f92f4494b80 1 --2- 172.21.15.44:0/2682684091 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55de99d89480 0x55de99d8b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.549+0000 7f92f4494b80 4 client.20052 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:23:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.549+0000 7f92f4494b80 1 -- 172.21.15.44:0/2682684091 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de99cd6840 con 0x55de99d89480 2023-09-10T01:23:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.549+0000 7f92e606a700 1 --2- 172.21.15.44:0/2682684091 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55de99d89480 0x55de99d8b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.549+0000 7f92c9ffb700 10 client.20052 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.549+0000 7f92e606a700 1 --2- 172.21.15.44:0/2682684091 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55de99d89480 0x55de99d8b8a0 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=215bd0e4a06ff511 server_cookie=89db557e9628cb04 in_seq=0 out_seq=0 2023-09-10T01:23:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.550+0000 7f92c9ffb700 1 -- 172.21.15.44:0/2682684091 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55de99cd6840 con 0x55de99d89480 2023-09-10T01:23:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.550+0000 7f92c9ffb700 10 client.20052 handle_command_reply: tid=0 2023-09-10T01:23:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.550+0000 7f92f4494b80 1 client.20052 shutdown 2023-09-10T01:23:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.550+0000 7f92f4494b80 1 -- 172.21.15.44:0/2682684091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d6fcc0 msgr2=0x55de99d790b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.550+0000 7f92f4494b80 1 --2- 172.21.15.44:0/2682684091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d6fcc0 0x55de99d790b0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f92d0098f40 tx=0x7f92d0004380 comp rx=0 tx=0).stop 2023-09-10T01:23:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.550+0000 7f92f4494b80 1 -- 172.21.15.44:0/2682684091 shutdown_connections 2023-09-10T01:23:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.550+0000 7f92f4494b80 1 --2- 172.21.15.44:0/2682684091 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55de99d89480 0x55de99d8b8a0 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.550+0000 7f92f4494b80 1 --2- 172.21.15.44:0/2682684091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d751a0 0x55de99d79730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.550+0000 7f92f4494b80 1 --2- 172.21.15.44:0/2682684091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6f300 0x55de99d7be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.550+0000 7f92f4494b80 1 --2- 172.21.15.44:0/2682684091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d6fcc0 0x55de99d790b0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.550+0000 7f92c9ffb700 0 client.20052 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.550+0000 7f92f4494b80 1 -- 172.21.15.44:0/2682684091 >> 172.21.15.44:0/2682684091 conn(0x55de99ce4030 msgr2=0x55de99d697a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.550+0000 7f92f4494b80 1 -- 172.21.15.44:0/2682684091 shutdown_connections 2023-09-10T01:23:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.550+0000 7f92f4494b80 1 -- 172.21.15.44:0/2682684091 wait complete. 2023-09-10T01:23:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.550+0000 7f92f4494b80 20 client.20052 trim_cache size 0 max 16384 2023-09-10T01:23:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.565+0000 7f92f4494b80 1 Processor -- start 2023-09-10T01:23:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.565+0000 7f92f4494b80 1 -- start start 2023-09-10T01:23:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.565+0000 7f92f4494b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d6f300 0x55de99d80460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.565+0000 7f92f4494b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6fcc0 0x55de99d69f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.565+0000 7f92f4494b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d751a0 0x55de99d6a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.565+0000 7f92f4494b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de99d7a220 con 0x55de99d6f300 2023-09-10T01:23:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.565+0000 7f92f4494b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de99cd6840 con 0x55de99d751a0 2023-09-10T01:23:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.565+0000 7f92f4494b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de99d86960 con 0x55de99d6fcc0 2023-09-10T01:23:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.565+0000 7f92e606a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d6f300 0x55de99d80460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.565+0000 7f92e606a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d6f300 0x55de99d80460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59070/0 (socket says 172.21.15.44:59070) 2023-09-10T01:23:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.565+0000 7f92e606a700 1 -- 172.21.15.44:0/3041554534 learned_addr learned my addr 172.21.15.44:0/3041554534 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.566+0000 7f92e5869700 1 --2- 172.21.15.44:0/3041554534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6fcc0 0x55de99d69f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.566+0000 7f92e686b700 1 --2- 172.21.15.44:0/3041554534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d751a0 0x55de99d6a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.566+0000 7f92e606a700 1 -- 172.21.15.44:0/3041554534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6fcc0 msgr2=0x55de99d69f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.566+0000 7f92e606a700 1 --2- 172.21.15.44:0/3041554534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6fcc0 0x55de99d69f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.566+0000 7f92e606a700 1 -- 172.21.15.44:0/3041554534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d751a0 msgr2=0x55de99d6a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.566+0000 7f92e606a700 1 --2- 172.21.15.44:0/3041554534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d751a0 0x55de99d6a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.566+0000 7f92e606a700 1 -- 172.21.15.44:0/3041554534 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de99b141f0 con 0x55de99d6f300 2023-09-10T01:23:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.566+0000 7f92e606a700 1 --2- 172.21.15.44:0/3041554534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d6f300 0x55de99d80460 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f92dc000c00 tx=0x7f92dc028040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464d814dfb939942 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.566+0000 7f92d558a700 1 -- 172.21.15.44:0/3041554534 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc02b040 con 0x55de99d6f300 2023-09-10T01:23:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.566+0000 7f92d558a700 1 -- 172.21.15.44:0/3041554534 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92dc004150 con 0x55de99d6f300 2023-09-10T01:23:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.566+0000 7f92d558a700 1 -- 172.21.15.44:0/3041554534 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc038bf0 con 0x55de99d6f300 2023-09-10T01:23:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.567+0000 7f92f4494b80 1 -- 172.21.15.44:0/3041554534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d6f300 msgr2=0x55de99d80460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.567+0000 7f92f4494b80 1 --2- 172.21.15.44:0/3041554534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d6f300 0x55de99d80460 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f92dc000c00 tx=0x7f92dc028040 comp rx=0 tx=0).stop 2023-09-10T01:23:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.567+0000 7f92f4494b80 1 -- 172.21.15.44:0/3041554534 shutdown_connections 2023-09-10T01:23:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.567+0000 7f92f4494b80 1 --2- 172.21.15.44:0/3041554534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d751a0 0x55de99d6a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.567+0000 7f92f4494b80 1 --2- 172.21.15.44:0/3041554534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d6f300 0x55de99d80460 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.567+0000 7f92f4494b80 1 --2- 172.21.15.44:0/3041554534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6fcc0 0x55de99d69f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.567+0000 7f92f4494b80 1 -- 172.21.15.44:0/3041554534 >> 172.21.15.44:0/3041554534 conn(0x55de99ce4030 msgr2=0x55de99d741d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.567+0000 7f92f4494b80 1 -- 172.21.15.44:0/3041554534 shutdown_connections 2023-09-10T01:23:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.567+0000 7f92f4494b80 1 -- 172.21.15.44:0/3041554534 wait complete. 2023-09-10T01:23:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.567+0000 7f92f4494b80 1 -- start start 2023-09-10T01:23:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.567+0000 7f92f4494b80 1 Processor -- start 2023-09-10T01:23:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92f4494b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d6f300 0x55de99dd6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92f4494b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6fcc0 0x55de99dd6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92f4494b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d751a0 0x55de99dd52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92f4494b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de99dd5820 con 0x55de99d751a0 2023-09-10T01:23:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92f4494b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de99dd5990 con 0x55de99d6f300 2023-09-10T01:23:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92f4494b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de99dd5b50 con 0x55de99d6fcc0 2023-09-10T01:23:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92f4494b80 10 client.0 _ll_register_callbacks cb 0x7f92d0098ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:23:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92e686b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d751a0 0x55de99dd52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92e5869700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6fcc0 0x55de99dd6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92e606a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d6f300 0x55de99dd6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92e686b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d751a0 0x55de99dd52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59080/0 (socket says 172.21.15.44:59080) 2023-09-10T01:23:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92e686b700 1 -- 172.21.15.44:0/650159944 learned_addr learned my addr 172.21.15.44:0/650159944 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92e686b700 1 -- 172.21.15.44:0/650159944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6fcc0 msgr2=0x55de99dd6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92e686b700 1 --2- 172.21.15.44:0/650159944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6fcc0 0x55de99dd6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92e686b700 1 -- 172.21.15.44:0/650159944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d6f300 msgr2=0x55de99dd6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92e686b700 1 --2- 172.21.15.44:0/650159944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d6f300 0x55de99dd6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92e686b700 1 -- 172.21.15.44:0/650159944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de99aedd80 con 0x55de99d751a0 2023-09-10T01:23:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92e5869700 1 --2- 172.21.15.44:0/650159944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6fcc0 0x55de99dd6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:23:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92e606a700 1 --2- 172.21.15.44:0/650159944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d6f300 0x55de99dd6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:23:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92e686b700 1 --2- 172.21.15.44:0/650159944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d751a0 0x55de99dd52b0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f92d8004070 tx=0x7f92d80040a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e4068bd94703be server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.568+0000 7f92c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:23:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.569+0000 7f92c9ffb700 1 -- 172.21.15.44:0/650159944 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d80048b0 con 0x55de99d751a0 2023-09-10T01:23:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.569+0000 7f92c9ffb700 1 -- 172.21.15.44:0/650159944 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(176..176 src has 1..176) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f92d8017410 con 0x55de99d751a0 2023-09-10T01:23:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.569+0000 7f92f4494b80 1 -- 172.21.15.44:0/650159944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92dc01e040 con 0x55de99d751a0 2023-09-10T01:23:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.569+0000 7f92c9ffb700 1 -- 172.21.15.44:0/650159944 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92d80175d0 con 0x55de99d751a0 2023-09-10T01:23:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.569+0000 7f92f4494b80 10 client.20058 fetch_fsmap learned FSMap version 140 2023-09-10T01:23:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.569+0000 7f92f4494b80 1 -- 172.21.15.44:0/650159944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=4+}) v3 -- 0x55de99aec250 con 0x55de99d751a0 2023-09-10T01:23:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.569+0000 7f92c9ffb700 1 -- 172.21.15.44:0/650159944 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92dc01e040 con 0x55de99d751a0 2023-09-10T01:23:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.569+0000 7f92c9ffb700 1 -- 172.21.15.44:0/650159944 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 140) v1 ==== 7375+0+0 (secure 0 0 0) 0x7f92d801d020 con 0x55de99d751a0 2023-09-10T01:23:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.569+0000 7f92f4494b80 10 client.20058 fetch_fsmap finished waiting for FSMap version 140 2023-09-10T01:23:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.569+0000 7f92f4494b80 10 client.20058 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:23:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.569+0000 7f92f4494b80 20 client.20058 populate_metadata read hostname 'smithi044' 2023-09-10T01:23:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.569+0000 7f92f4494b80 1 --2- 172.21.15.44:0/650159944 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55de99dde6a0 0x55de99de0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.569+0000 7f92f4494b80 4 client.20058 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:23:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.569+0000 7f92f4494b80 1 -- 172.21.15.44:0/650159944 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55de99cba8c0 con 0x55de99dde6a0 2023-09-10T01:23:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.570+0000 7f92e606a700 1 --2- 172.21.15.44:0/650159944 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55de99dde6a0 0x55de99de0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.570+0000 7f92e606a700 1 --2- 172.21.15.44:0/650159944 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55de99dde6a0 0x55de99de0ac0 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3efde95623cdbc2 server_cookie=8077250c9a686a6e in_seq=0 out_seq=0 2023-09-10T01:23:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.570+0000 7f92c9ffb700 10 client.20058 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.596+0000 7f92c9ffb700 1 -- 172.21.15.44:0/650159944 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55de99cba8c0 con 0x55de99dde6a0 2023-09-10T01:23:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.596+0000 7f92c9ffb700 10 client.20058 handle_command_reply: tid=0 2023-09-10T01:23:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.596+0000 7f92f4494b80 1 client.20058 shutdown 2023-09-10T01:23:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.597+0000 7f92f4494b80 1 -- 172.21.15.44:0/650159944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d751a0 msgr2=0x55de99dd52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.597+0000 7f92f4494b80 1 --2- 172.21.15.44:0/650159944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d751a0 0x55de99dd52b0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f92d8004070 tx=0x7f92d80040a0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.597+0000 7f92f4494b80 1 -- 172.21.15.44:0/650159944 shutdown_connections 2023-09-10T01:23:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.597+0000 7f92f4494b80 1 --2- 172.21.15.44:0/650159944 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55de99dde6a0 0x55de99de0ac0 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.597+0000 7f92f4494b80 1 --2- 172.21.15.44:0/650159944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de99d751a0 0x55de99dd52b0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.597+0000 7f92f4494b80 1 --2- 172.21.15.44:0/650159944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de99d6f300 0x55de99dd6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.597+0000 7f92f4494b80 1 --2- 172.21.15.44:0/650159944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de99d6fcc0 0x55de99dd6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.597+0000 7f92f4494b80 1 -- 172.21.15.44:0/650159944 >> 172.21.15.44:0/650159944 conn(0x55de99ce4030 msgr2=0x55de99d7a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.597+0000 7f92c9ffb700 0 client.20058 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.598+0000 7f92f4494b80 1 -- 172.21.15.44:0/650159944 shutdown_connections 2023-09-10T01:23:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.598+0000 7f92f4494b80 1 -- 172.21.15.44:0/650159944 wait complete. 2023-09-10T01:23:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.602+0000 7f92f4494b80 20 client.20058 trim_cache size 0 max 16384 2023-09-10T01:23:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:23:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:23:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:23:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:23:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.603+0000 7f92e82ce700 1 -- 172.21.15.44:0/3512198173 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f92cc0a07a0 msgr2=0x7f92cc0a2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.603+0000 7f92e82ce700 1 --2- 172.21.15.44:0/3512198173 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f92cc0a07a0 0x7f92cc0a2c60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00b580 tx=0x7f92dc009f10 comp rx=0 tx=0).stop 2023-09-10T01:23:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.604+0000 7f92e82ce700 1 -- 172.21.15.44:0/3512198173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e0100120 msgr2=0x7f92e0198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.604+0000 7f92e82ce700 1 --2- 172.21.15.44:0/3512198173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e0100120 0x7f92e0198ca0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f92d000d980 tx=0x7f92d000dc90 comp rx=0 tx=0).stop 2023-09-10T01:23:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.605+0000 7f92e82ce700 1 -- 172.21.15.44:0/3512198173 shutdown_connections 2023-09-10T01:23:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.605+0000 7f92e82ce700 1 --2- 172.21.15.44:0/3512198173 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f92cc0a07a0 0x7f92cc0a2c60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.605+0000 7f92e82ce700 1 --2- 172.21.15.44:0/3512198173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e01055b0 0x7f92e019d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.605+0000 7f92e82ce700 1 --2- 172.21.15.44:0/3512198173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92e00ff760 0x7f92e0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.605+0000 7f92e82ce700 1 --2- 172.21.15.44:0/3512198173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e0100120 0x7f92e0198ca0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.605+0000 7f92e82ce700 1 -- 172.21.15.44:0/3512198173 >> 172.21.15.44:0/3512198173 conn(0x7f92e0074f30 msgr2=0x7f92e00fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.605+0000 7f92e82ce700 1 -- 172.21.15.44:0/3512198173 shutdown_connections 2023-09-10T01:23:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:28.605+0000 7f92e82ce700 1 -- 172.21.15.44:0/3512198173 wait complete. 2023-09-10T01:23:29.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:29 smithi131 ceph-mon[103713]: pgmap v1386: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 7.5 MiB/s wr, 597 op/s 2023-09-10T01:23:29.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:29 smithi044 ceph-mon[98676]: pgmap v1386: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 7.5 MiB/s wr, 597 op/s 2023-09-10T01:23:29.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:29 smithi099 ceph-mon[102692]: pgmap v1386: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 7.5 MiB/s wr, 597 op/s 2023-09-10T01:23:30.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:30 smithi131 ceph-mon[103713]: pgmap v1387: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 7.5 MiB/s wr, 597 op/s 2023-09-10T01:23:30.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:30 smithi044 ceph-mon[98676]: pgmap v1387: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 7.5 MiB/s wr, 597 op/s 2023-09-10T01:23:30.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:30 smithi099 ceph-mon[102692]: pgmap v1387: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 7.5 MiB/s wr, 597 op/s 2023-09-10T01:23:31.182 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 145.0 s, 197.3 tps, lat 892.851 ms stddev 1926.044 2023-09-10T01:23:31.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:31 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:23:31.563 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:31 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:23:31.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:31 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:23:32.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:32 smithi131 ceph-mon[103713]: pgmap v1388: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 5.8 MiB/s wr, 415 op/s 2023-09-10T01:23:32.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:32 smithi044 ceph-mon[98676]: pgmap v1388: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 5.8 MiB/s wr, 415 op/s 2023-09-10T01:23:32.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:32 smithi099 ceph-mon[102692]: pgmap v1388: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 5.8 MiB/s wr, 415 op/s 2023-09-10T01:23:34.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:34 smithi131 ceph-mon[103713]: pgmap v1389: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 4.3 MiB/s wr, 321 op/s 2023-09-10T01:23:34.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:34 smithi044 ceph-mon[98676]: pgmap v1389: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 4.3 MiB/s wr, 321 op/s 2023-09-10T01:23:34.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:34 smithi099 ceph-mon[102692]: pgmap v1389: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 4.3 MiB/s wr, 321 op/s 2023-09-10T01:23:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 150.0 s, 272.6 tps, lat 371.230 ms stddev 124.310 2023-09-10T01:23:37.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:37 smithi131 ceph-mon[103713]: pgmap v1390: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 6.0 MiB/s wr, 501 op/s 2023-09-10T01:23:37.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:37 smithi044 ceph-mon[98676]: pgmap v1390: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 6.0 MiB/s wr, 501 op/s 2023-09-10T01:23:37.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:37 smithi099 ceph-mon[102692]: pgmap v1390: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 6.0 MiB/s wr, 501 op/s 2023-09-10T01:23:39.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:39 smithi131 ceph-mon[103713]: pgmap v1391: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 5.5 MiB/s wr, 475 op/s 2023-09-10T01:23:39.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:39 smithi044 ceph-mon[98676]: pgmap v1391: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 5.5 MiB/s wr, 475 op/s 2023-09-10T01:23:39.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:39 smithi099 ceph-mon[102692]: pgmap v1391: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 5.5 MiB/s wr, 475 op/s 2023-09-10T01:23:41.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 155.0 s, 267.2 tps, lat 367.818 ms stddev 120.897 2023-09-10T01:23:41.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:41 smithi131 ceph-mon[103713]: pgmap v1392: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 5.2 MiB/s wr, 439 op/s 2023-09-10T01:23:41.565 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:41 smithi044 ceph-mon[98676]: pgmap v1392: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 5.2 MiB/s wr, 439 op/s 2023-09-10T01:23:41.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:41 smithi099 ceph-mon[102692]: pgmap v1392: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 5.2 MiB/s wr, 439 op/s 2023-09-10T01:23:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.799+0000 7f785796b700 1 -- 172.21.15.44:0/1679971021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7850104970 msgr2=0x7f7850104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.799+0000 7f785796b700 1 --2- 172.21.15.44:0/1679971021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7850104970 0x7f7850104e20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f784c01a9a0 tx=0x7f784c01acb0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.801+0000 7f785796b700 1 -- 172.21.15.44:0/1679971021 shutdown_connections 2023-09-10T01:23:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.801+0000 7f785796b700 1 --2- 172.21.15.44:0/1679971021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7850104970 0x7f7850104e20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.801+0000 7f785796b700 1 --2- 172.21.15.44:0/1679971021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78500ff570 0x7f78500ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.801+0000 7f785796b700 1 --2- 172.21.15.44:0/1679971021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78500ffe90 0x7f7850104350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.801+0000 7f785796b700 1 -- 172.21.15.44:0/1679971021 >> 172.21.15.44:0/1679971021 conn(0x7f78500faf40 msgr2=0x7f78500fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.813+0000 7f785796b700 1 -- 172.21.15.44:0/1679971021 shutdown_connections 2023-09-10T01:23:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.813+0000 7f785796b700 1 -- 172.21.15.44:0/1679971021 wait complete. 2023-09-10T01:23:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.814+0000 7f785796b700 1 Processor -- start 2023-09-10T01:23:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.814+0000 7f785796b700 1 -- start start 2023-09-10T01:23:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.814+0000 7f785796b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78500ff570 0x7f7850070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.815+0000 7f785796b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78500ffe90 0x7f7850071130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.815+0000 7f785796b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7850104970 0x7f7850113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.815+0000 7f785796b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7850113ff0 con 0x7f7850104970 2023-09-10T01:23:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.815+0000 7f785796b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785010c870 con 0x7f78500ff570 2023-09-10T01:23:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.815+0000 7f785796b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785010ca30 con 0x7f78500ffe90 2023-09-10T01:23:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.815+0000 7f7855f08700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7850104970 0x7f7850113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.815+0000 7f7855f08700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7850104970 0x7f7850113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59090/0 (socket says 172.21.15.44:59090) 2023-09-10T01:23:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.815+0000 7f7855f08700 1 -- 172.21.15.44:0/144723224 learned_addr learned my addr 172.21.15.44:0/144723224 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.816+0000 7f7854f06700 1 --2- 172.21.15.44:0/144723224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78500ffe90 0x7f7850071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.816+0000 7f7855707700 1 --2- 172.21.15.44:0/144723224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78500ff570 0x7f7850070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.816+0000 7f7855f08700 1 -- 172.21.15.44:0/144723224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78500ffe90 msgr2=0x7f7850071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.816+0000 7f7855f08700 1 --2- 172.21.15.44:0/144723224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78500ffe90 0x7f7850071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.816+0000 7f7855f08700 1 -- 172.21.15.44:0/144723224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78500ff570 msgr2=0x7f7850070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.816+0000 7f7855f08700 1 --2- 172.21.15.44:0/144723224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78500ff570 0x7f7850070bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.816+0000 7f7855f08700 1 -- 172.21.15.44:0/144723224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784c01a600 con 0x7f7850104970 2023-09-10T01:23:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.816+0000 7f7854f06700 1 --2- 172.21.15.44:0/144723224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78500ffe90 0x7f7850071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:23:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.816+0000 7f7855f08700 1 --2- 172.21.15.44:0/144723224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7850104970 0x7f7850113960 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f784c0289c0 tx=0x7f784c0289f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.817+0000 7f78467fc700 1 -- 172.21.15.44:0/144723224 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c01ce00 con 0x7f7850104970 2023-09-10T01:23:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.817+0000 7f785796b700 1 -- 172.21.15.44:0/144723224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785010ccb0 con 0x7f7850104970 2023-09-10T01:23:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.817+0000 7f78467fc700 1 -- 172.21.15.44:0/144723224 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f784c028e50 con 0x7f7850104970 2023-09-10T01:23:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.817+0000 7f78467fc700 1 -- 172.21.15.44:0/144723224 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c004ca0 con 0x7f7850104970 2023-09-10T01:23:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.817+0000 7f785796b700 1 -- 172.21.15.44:0/144723224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785010d1b0 con 0x7f7850104970 2023-09-10T01:23:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.825+0000 7f78467fc700 1 -- 172.21.15.44:0/144723224 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f784c005030 con 0x7f7850104970 2023-09-10T01:23:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.826+0000 7f78467fc700 1 --2- 172.21.15.44:0/144723224 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f783c0a07f0 0x7f783c0a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.826+0000 7f78467fc700 1 -- 172.21.15.44:0/144723224 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f784c0ddb80 con 0x7f7850104970 2023-09-10T01:23:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.827+0000 7f7855707700 1 --2- 172.21.15.44:0/144723224 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f783c0a07f0 0x7f783c0a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.827+0000 7f7855707700 1 --2- 172.21.15.44:0/144723224 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f783c0a07f0 0x7f783c0a2cb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f784800f540 tx=0x7f784800b040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.828+0000 7f7863b31b80 1 Processor -- start 2023-09-10T01:23:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.828+0000 7f7863b31b80 1 -- start start 2023-09-10T01:23:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.829+0000 7f7863b31b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3ef1a0 0x55cb3b3ef580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.829+0000 7f7863b31b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 0x55cb3b3e9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.829+0000 7f7863b31b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3e9cc0 0x55cb3b3f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.829+0000 7f7863b31b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b3426d0 con 0x55cb3b3e9300 2023-09-10T01:23:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.829+0000 7f7863b31b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b33fff0 con 0x55cb3b3e9cc0 2023-09-10T01:23:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.829+0000 7f7863b31b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b33b8e0 con 0x55cb3b3ef1a0 2023-09-10T01:23:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.829+0000 7f7854f06700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 0x55cb3b3e9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.829+0000 7f7854f06700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 0x55cb3b3e9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59098/0 (socket says 172.21.15.44:59098) 2023-09-10T01:23:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.829+0000 7f7854f06700 1 -- 172.21.15.44:0/1196395575 learned_addr learned my addr 172.21.15.44:0/1196395575 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.829+0000 7f7855707700 1 --2- 172.21.15.44:0/1196395575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3ef1a0 0x55cb3b3ef580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.831+0000 7f7854f06700 1 -- 172.21.15.44:0/1196395575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3ef1a0 msgr2=0x55cb3b3ef580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.831+0000 7f7854f06700 1 --2- 172.21.15.44:0/1196395575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3ef1a0 0x55cb3b3ef580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.831+0000 7f7854f06700 1 -- 172.21.15.44:0/1196395575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3e9cc0 msgr2=0x55cb3b3f5f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:23:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.831+0000 7f7854f06700 1 --2- 172.21.15.44:0/1196395575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3e9cc0 0x55cb3b3f5f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.831+0000 7f7854f06700 1 -- 172.21.15.44:0/1196395575 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb3b167d80 con 0x55cb3b3e9300 2023-09-10T01:23:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.831+0000 7f7854f06700 1 --2- 172.21.15.44:0/1196395575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 0x55cb3b3e9780 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f7840009c20 tx=0x7f7840009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef9142c09551963 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.831+0000 7f783b7fe700 1 -- 172.21.15.44:0/1196395575 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840004230 con 0x55cb3b3e9300 2023-09-10T01:23:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.831+0000 7f783b7fe700 1 -- 172.21.15.44:0/1196395575 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f784000fda0 con 0x55cb3b3e9300 2023-09-10T01:23:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.832+0000 7f783b7fe700 1 -- 172.21.15.44:0/1196395575 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840003d40 con 0x55cb3b3e9300 2023-09-10T01:23:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.832+0000 7f7863b31b80 1 -- 172.21.15.44:0/1196395575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 msgr2=0x55cb3b3e9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.832+0000 7f7863b31b80 1 --2- 172.21.15.44:0/1196395575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 0x55cb3b3e9780 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f7840009c20 tx=0x7f7840009f30 comp rx=0 tx=0).stop 2023-09-10T01:23:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.834+0000 7f7863b31b80 1 -- 172.21.15.44:0/1196395575 shutdown_connections 2023-09-10T01:23:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.834+0000 7f7863b31b80 1 --2- 172.21.15.44:0/1196395575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3e9cc0 0x55cb3b3f5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.834+0000 7f7863b31b80 1 --2- 172.21.15.44:0/1196395575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3ef1a0 0x55cb3b3ef580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.834+0000 7f7863b31b80 1 --2- 172.21.15.44:0/1196395575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 0x55cb3b3e9780 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.834+0000 7f7863b31b80 1 -- 172.21.15.44:0/1196395575 >> 172.21.15.44:0/1196395575 conn(0x55cb3b35e030 msgr2=0x55cb3b3e4560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.834+0000 7f7863b31b80 1 -- 172.21.15.44:0/1196395575 shutdown_connections 2023-09-10T01:23:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.834+0000 7f7863b31b80 1 -- 172.21.15.44:0/1196395575 wait complete. 2023-09-10T01:23:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.834+0000 7f7863b31b80 1 -- start start 2023-09-10T01:23:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.835+0000 7f7863b31b80 1 Processor -- start 2023-09-10T01:23:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.835+0000 7f7863b31b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 0x55cb3b3f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.835+0000 7f7863b31b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3e9cc0 0x55cb3b3f3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.835+0000 7f7863b31b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 0x55cb3b3ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.835+0000 7f7863b31b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b2d9b10 con 0x55cb3b3e9300 2023-09-10T01:23:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.835+0000 7f7863b31b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b2d9810 con 0x55cb3b3f3c70 2023-09-10T01:23:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.835+0000 7f7863b31b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b1bd8c0 con 0x55cb3b3e9cc0 2023-09-10T01:23:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.835+0000 7f7863b31b80 10 client.0 _ll_register_callbacks cb 0x55cb3b223a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:23:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.835+0000 7f7855707700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 0x55cb3b3f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.835+0000 7f7855707700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 0x55cb3b3f31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59104/0 (socket says 172.21.15.44:59104) 2023-09-10T01:23:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.835+0000 7f7855707700 1 -- 172.21.15.44:0/2930314076 learned_addr learned my addr 172.21.15.44:0/2930314076 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.835+0000 7f7855f08700 1 --2- 172.21.15.44:0/2930314076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 0x55cb3b3ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.835+0000 7f7854f06700 1 --2- 172.21.15.44:0/2930314076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3e9cc0 0x55cb3b3f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.836+0000 7f7855707700 1 -- 172.21.15.44:0/2930314076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3e9cc0 msgr2=0x55cb3b3f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.836+0000 7f7855707700 1 --2- 172.21.15.44:0/2930314076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3e9cc0 0x55cb3b3f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.836+0000 7f7855707700 1 -- 172.21.15.44:0/2930314076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 msgr2=0x55cb3b3ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.836+0000 7f7855707700 1 --2- 172.21.15.44:0/2930314076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 0x55cb3b3ff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.836+0000 7f7855707700 1 -- 172.21.15.44:0/2930314076 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7840009850 con 0x55cb3b3e9300 2023-09-10T01:23:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.836+0000 7f7855707700 1 --2- 172.21.15.44:0/2930314076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 0x55cb3b3f31f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7848004560 tx=0x7f7848004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a0c59ca599e53a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.836+0000 7f7838ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:23:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.836+0000 7f7838ff9700 1 -- 172.21.15.44:0/2930314076 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784800e040 con 0x55cb3b3e9300 2023-09-10T01:23:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.836+0000 7f7838ff9700 1 -- 172.21.15.44:0/2930314076 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(176..176 src has 1..176) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f7848015c70 con 0x55cb3b3e9300 2023-09-10T01:23:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.837+0000 7f7863b31b80 1 -- 172.21.15.44:0/2930314076 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb3b293490 con 0x55cb3b3e9300 2023-09-10T01:23:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.837+0000 7f7838ff9700 1 -- 172.21.15.44:0/2930314076 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=141) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7848004920 con 0x55cb3b3e9300 2023-09-10T01:23:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.837+0000 7f7863b31b80 10 client.20070 fetch_fsmap learned FSMap version 141 2023-09-10T01:23:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.837+0000 7f7863b31b80 1 -- 172.21.15.44:0/2930314076 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=141,monmap=4+}) v3 -- 0x55cb3b150c00 con 0x55cb3b3e9300 2023-09-10T01:23:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.837+0000 7f7838ff9700 1 -- 172.21.15.44:0/2930314076 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cb3b293490 con 0x55cb3b3e9300 2023-09-10T01:23:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.838+0000 7f7838ff9700 1 -- 172.21.15.44:0/2930314076 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 141) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f7848017440 con 0x55cb3b3e9300 2023-09-10T01:23:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.838+0000 7f7863b31b80 10 client.20070 fetch_fsmap finished waiting for FSMap version 141 2023-09-10T01:23:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.838+0000 7f7863b31b80 10 client.20070 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:23:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.838+0000 7f7863b31b80 20 client.20070 populate_metadata read hostname 'smithi044' 2023-09-10T01:23:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.838+0000 7f7863b31b80 1 --2- 172.21.15.44:0/2930314076 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb3b403a40 0x55cb3b405e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.838+0000 7f7863b31b80 4 client.20070 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:23:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.838+0000 7f7863b31b80 1 -- 172.21.15.44:0/2930314076 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb3b350840 con 0x55cb3b403a40 2023-09-10T01:23:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.838+0000 7f7854f06700 1 --2- 172.21.15.44:0/2930314076 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb3b403a40 0x55cb3b405e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.838+0000 7f7838ff9700 10 client.20070 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.838+0000 7f7854f06700 1 --2- 172.21.15.44:0/2930314076 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb3b403a40 0x55cb3b405e60 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad3880a3fc2bda47 server_cookie=5a714f3e204f46f0 in_seq=0 out_seq=0 2023-09-10T01:23:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.839+0000 7f7838ff9700 1 -- 172.21.15.44:0/2930314076 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cb3b350840 con 0x55cb3b403a40 2023-09-10T01:23:41.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.839+0000 7f7838ff9700 10 client.20070 handle_command_reply: tid=0 2023-09-10T01:23:41.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.839+0000 7f7863b31b80 1 client.20070 shutdown 2023-09-10T01:23:41.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.839+0000 7f7863b31b80 1 -- 172.21.15.44:0/2930314076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 msgr2=0x55cb3b3f31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:41.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.839+0000 7f7863b31b80 1 --2- 172.21.15.44:0/2930314076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 0x55cb3b3f31f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7848004560 tx=0x7f7848004ad0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.839+0000 7f7863b31b80 1 -- 172.21.15.44:0/2930314076 shutdown_connections 2023-09-10T01:23:41.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.839+0000 7f7863b31b80 1 --2- 172.21.15.44:0/2930314076 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb3b403a40 0x55cb3b405e60 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.839+0000 7f7863b31b80 1 --2- 172.21.15.44:0/2930314076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 0x55cb3b3ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.839+0000 7f7863b31b80 1 --2- 172.21.15.44:0/2930314076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 0x55cb3b3f31f0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.839+0000 7f7863b31b80 1 --2- 172.21.15.44:0/2930314076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3e9cc0 0x55cb3b3f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.839+0000 7f7863b31b80 1 -- 172.21.15.44:0/2930314076 >> 172.21.15.44:0/2930314076 conn(0x55cb3b35e030 msgr2=0x55cb3b3e37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.839+0000 7f7838ff9700 0 client.20070 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.840+0000 7f7863b31b80 1 -- 172.21.15.44:0/2930314076 shutdown_connections 2023-09-10T01:23:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.840+0000 7f7863b31b80 1 -- 172.21.15.44:0/2930314076 wait complete. 2023-09-10T01:23:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.840+0000 7f7863b31b80 20 client.20070 trim_cache size 0 max 16384 2023-09-10T01:23:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.870+0000 7f7863b31b80 1 Processor -- start 2023-09-10T01:23:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.871+0000 7f7863b31b80 1 -- start start 2023-09-10T01:23:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.871+0000 7f7863b31b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3e9300 0x55cb3b3f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.872+0000 7f7863b31b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9cc0 0x55cb3b3fa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.872+0000 7f7863b31b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 0x55cb3b3fec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.872+0000 7f7863b31b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b3faa60 con 0x55cb3b3e9cc0 2023-09-10T01:23:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.872+0000 7f7863b31b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b3f1770 con 0x55cb3b3f3c70 2023-09-10T01:23:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.872+0000 7f7863b31b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b3ede30 con 0x55cb3b3e9300 2023-09-10T01:23:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.872+0000 7f7855f08700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 0x55cb3b3fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.872+0000 7f7855f08700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 0x55cb3b3fec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60854/0 (socket says 172.21.15.44:60854) 2023-09-10T01:23:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.872+0000 7f7855f08700 1 -- 172.21.15.44:0/1065661880 learned_addr learned my addr 172.21.15.44:0/1065661880 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.872+0000 7f7855f08700 1 -- 172.21.15.44:0/1065661880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3e9300 msgr2=0x55cb3b3f0d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:23:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.872+0000 7f7855f08700 1 --2- 172.21.15.44:0/1065661880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3e9300 0x55cb3b3f0d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.872+0000 7f7855f08700 1 -- 172.21.15.44:0/1065661880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9cc0 msgr2=0x55cb3b3fa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.872+0000 7f7855f08700 1 --2- 172.21.15.44:0/1065661880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9cc0 0x55cb3b3fa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.872+0000 7f7855f08700 1 -- 172.21.15.44:0/1065661880 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb3b18e1f0 con 0x55cb3b3f3c70 2023-09-10T01:23:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.873+0000 7f7855f08700 1 --2- 172.21.15.44:0/1065661880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 0x55cb3b3fec80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f784c0daf40 tx=0x7f784c0a7130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85879860fc2e038d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.879+0000 7f783bfff700 1 -- 172.21.15.44:0/1065661880 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c0aa630 con 0x55cb3b3f3c70 2023-09-10T01:23:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.879+0000 7f783bfff700 1 -- 172.21.15.44:0/1065661880 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f784c0c8e90 con 0x55cb3b3f3c70 2023-09-10T01:23:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.879+0000 7f783bfff700 1 -- 172.21.15.44:0/1065661880 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c0a73e0 con 0x55cb3b3f3c70 2023-09-10T01:23:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.883+0000 7f7863b31b80 1 -- 172.21.15.44:0/1065661880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 msgr2=0x55cb3b3fec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.883+0000 7f7863b31b80 1 --2- 172.21.15.44:0/1065661880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 0x55cb3b3fec80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f784c0daf40 tx=0x7f784c0a7130 comp rx=0 tx=0).stop 2023-09-10T01:23:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.883+0000 7f7863b31b80 1 -- 172.21.15.44:0/1065661880 shutdown_connections 2023-09-10T01:23:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.883+0000 7f7863b31b80 1 --2- 172.21.15.44:0/1065661880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 0x55cb3b3fec80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.883+0000 7f7863b31b80 1 --2- 172.21.15.44:0/1065661880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3e9300 0x55cb3b3f0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.883+0000 7f7863b31b80 1 --2- 172.21.15.44:0/1065661880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9cc0 0x55cb3b3fa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.883+0000 7f7863b31b80 1 -- 172.21.15.44:0/1065661880 >> 172.21.15.44:0/1065661880 conn(0x55cb3b35e030 msgr2=0x55cb3b3e4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.897+0000 7f7863b31b80 1 -- 172.21.15.44:0/1065661880 shutdown_connections 2023-09-10T01:23:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.902+0000 7f7863b31b80 1 -- 172.21.15.44:0/1065661880 wait complete. 2023-09-10T01:23:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.903+0000 7f7863b31b80 1 -- start start 2023-09-10T01:23:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.903+0000 7f7863b31b80 1 Processor -- start 2023-09-10T01:23:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.914+0000 7f7863b31b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 0x55cb3b44fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.914+0000 7f7863b31b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3e9cc0 0x55cb3b44fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.914+0000 7f7863b31b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 0x55cb3b3f6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.914+0000 7f7863b31b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b3f6ad0 con 0x55cb3b3e9300 2023-09-10T01:23:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.914+0000 7f7863b31b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b3f6c40 con 0x55cb3b3f3c70 2023-09-10T01:23:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.914+0000 7f7863b31b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b3f6e00 con 0x55cb3b3e9cc0 2023-09-10T01:23:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.914+0000 7f7863b31b80 10 client.0 _ll_register_callbacks cb 0x7f7848003ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:23:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.915+0000 7f7855f08700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 0x55cb3b3f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.915+0000 7f7855f08700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 0x55cb3b3f6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60858/0 (socket says 172.21.15.44:60858) 2023-09-10T01:23:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.915+0000 7f7855f08700 1 -- 172.21.15.44:0/1454177554 learned_addr learned my addr 172.21.15.44:0/1454177554 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.915+0000 7f7855f08700 1 -- 172.21.15.44:0/1454177554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3e9cc0 msgr2=0x55cb3b44fff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:23:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.915+0000 7f7855f08700 1 --2- 172.21.15.44:0/1454177554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3e9cc0 0x55cb3b44fff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.915+0000 7f7855f08700 1 -- 172.21.15.44:0/1454177554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 msgr2=0x55cb3b44fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.915+0000 7f7855f08700 1 --2- 172.21.15.44:0/1454177554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 0x55cb3b44fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.915+0000 7f7855f08700 1 -- 172.21.15.44:0/1454177554 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785010d1b0 con 0x55cb3b3f3c70 2023-09-10T01:23:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.916+0000 7f7855f08700 1 --2- 172.21.15.44:0/1454177554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 0x55cb3b3f6500 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f784c028990 tx=0x7f784c0aad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b74c736b9b2c28c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.916+0000 7f7838ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:23:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.916+0000 7f7838ff9700 1 -- 172.21.15.44:0/1454177554 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c0b2cb0 con 0x55cb3b3f3c70 2023-09-10T01:23:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.916+0000 7f7838ff9700 1 -- 172.21.15.44:0/1454177554 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(176..176 src has 1..176) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f784c0aaf30 con 0x55cb3b3f3c70 2023-09-10T01:23:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.916+0000 7f7863b31b80 1 -- 172.21.15.44:0/1454177554 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f784000fda0 con 0x55cb3b3f3c70 2023-09-10T01:23:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.927+0000 7f7838ff9700 1 -- 172.21.15.44:0/1454177554 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=141) v2 ==== 24+0+0 (secure 0 0 0) 0x7f784c0ab0f0 con 0x55cb3b3f3c70 2023-09-10T01:23:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.927+0000 7f7863b31b80 10 client.29702 fetch_fsmap learned FSMap version 141 2023-09-10T01:23:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.927+0000 7f7863b31b80 1 -- 172.21.15.44:0/1454177554 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=141,monmap=4+}) v3 -- 0x55cb3b166250 con 0x55cb3b3f3c70 2023-09-10T01:23:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.927+0000 7f7838ff9700 1 -- 172.21.15.44:0/1454177554 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f784000fda0 con 0x55cb3b3f3c70 2023-09-10T01:23:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.927+0000 7f7838ff9700 1 -- 172.21.15.44:0/1454177554 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 141) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f784c0b2820 con 0x55cb3b3f3c70 2023-09-10T01:23:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.927+0000 7f7863b31b80 10 client.29702 fetch_fsmap finished waiting for FSMap version 141 2023-09-10T01:23:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.927+0000 7f7863b31b80 10 client.29702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:23:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.927+0000 7f7863b31b80 20 client.29702 populate_metadata read hostname 'smithi044' 2023-09-10T01:23:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.927+0000 7f7863b31b80 1 --2- 172.21.15.44:0/1454177554 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb3b458690 0x55cb3b45aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.927+0000 7f7863b31b80 4 client.29702 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:23:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.927+0000 7f7863b31b80 1 -- 172.21.15.44:0/1454177554 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cb3b3348c0 con 0x55cb3b458690 2023-09-10T01:23:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.937+0000 7f7855707700 1 --2- 172.21.15.44:0/1454177554 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb3b458690 0x55cb3b45aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.938+0000 7f7855707700 1 --2- 172.21.15.44:0/1454177554 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb3b458690 0x55cb3b45aab0 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b81c114d94fecf0 server_cookie=a6bdc7394fd079b3 in_seq=0 out_seq=0 2023-09-10T01:23:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.938+0000 7f7838ff9700 10 client.29702 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.977+0000 7f7838ff9700 1 -- 172.21.15.44:0/1454177554 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cb3b3348c0 con 0x55cb3b458690 2023-09-10T01:23:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.977+0000 7f7838ff9700 10 client.29702 handle_command_reply: tid=0 2023-09-10T01:23:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.977+0000 7f7863b31b80 1 client.29702 shutdown 2023-09-10T01:23:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.977+0000 7f7863b31b80 1 -- 172.21.15.44:0/1454177554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 msgr2=0x55cb3b3f6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.977+0000 7f7863b31b80 1 --2- 172.21.15.44:0/1454177554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 0x55cb3b3f6500 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f784c028990 tx=0x7f784c0aad80 comp rx=0 tx=0).stop 2023-09-10T01:23:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.977+0000 7f7863b31b80 1 -- 172.21.15.44:0/1454177554 shutdown_connections 2023-09-10T01:23:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.977+0000 7f7863b31b80 1 --2- 172.21.15.44:0/1454177554 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55cb3b458690 0x55cb3b45aab0 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.977+0000 7f7863b31b80 1 --2- 172.21.15.44:0/1454177554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb3b3f3c70 0x55cb3b3f6500 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.977+0000 7f7863b31b80 1 --2- 172.21.15.44:0/1454177554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb3b3e9300 0x55cb3b44fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.977+0000 7f7863b31b80 1 --2- 172.21.15.44:0/1454177554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3b3e9cc0 0x55cb3b44fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.977+0000 7f7863b31b80 1 -- 172.21.15.44:0/1454177554 >> 172.21.15.44:0/1454177554 conn(0x55cb3b35e030 msgr2=0x55cb3b3fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.977+0000 7f7838ff9700 0 client.29702 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.977+0000 7f7863b31b80 1 -- 172.21.15.44:0/1454177554 shutdown_connections 2023-09-10T01:23:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.977+0000 7f7863b31b80 1 -- 172.21.15.44:0/1454177554 wait complete. 2023-09-10T01:23:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.977+0000 7f7863b31b80 20 client.29702 trim_cache size 0 max 16384 2023-09-10T01:23:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:23:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:23:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:23:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:23:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.978+0000 7f785796b700 1 -- 172.21.15.44:0/144723224 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f783c0a07f0 msgr2=0x7f783c0a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.978+0000 7f785796b700 1 --2- 172.21.15.44:0/144723224 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f783c0a07f0 0x7f783c0a2cb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f784800f540 tx=0x7f784800b040 comp rx=0 tx=0).stop 2023-09-10T01:23:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.978+0000 7f785796b700 1 -- 172.21.15.44:0/144723224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7850104970 msgr2=0x7f7850113960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.978+0000 7f785796b700 1 --2- 172.21.15.44:0/144723224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7850104970 0x7f7850113960 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f784c0289c0 tx=0x7f784c0289f0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.978+0000 7f785796b700 1 -- 172.21.15.44:0/144723224 shutdown_connections 2023-09-10T01:23:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.978+0000 7f785796b700 1 --2- 172.21.15.44:0/144723224 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f783c0a07f0 0x7f783c0a2cb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.978+0000 7f785796b700 1 --2- 172.21.15.44:0/144723224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7850104970 0x7f7850113960 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.978+0000 7f785796b700 1 --2- 172.21.15.44:0/144723224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78500ff570 0x7f7850070bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.978+0000 7f785796b700 1 --2- 172.21.15.44:0/144723224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78500ffe90 0x7f7850071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.978+0000 7f785796b700 1 -- 172.21.15.44:0/144723224 >> 172.21.15.44:0/144723224 conn(0x7f78500faf40 msgr2=0x7f78501023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.978+0000 7f785796b700 1 -- 172.21.15.44:0/144723224 shutdown_connections 2023-09-10T01:23:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:41.978+0000 7f785796b700 1 -- 172.21.15.44:0/144723224 wait complete. 2023-09-10T01:23:43.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:43 smithi131 ceph-mon[103713]: pgmap v1393: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 8.7 MiB/s wr, 743 op/s 2023-09-10T01:23:43.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:43 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:23:43.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:43 smithi044 ceph-mon[98676]: pgmap v1393: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 8.7 MiB/s wr, 743 op/s 2023-09-10T01:23:43.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:43 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:23:43.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:43 smithi099 ceph-mon[102692]: pgmap v1393: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 8.7 MiB/s wr, 743 op/s 2023-09-10T01:23:43.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:43 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:23:45.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:45 smithi131 ceph-mon[103713]: pgmap v1394: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 6.7 MiB/s wr, 603 op/s 2023-09-10T01:23:45.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:45 smithi044 ceph-mon[98676]: pgmap v1394: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 6.7 MiB/s wr, 603 op/s 2023-09-10T01:23:45.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:45 smithi099 ceph-mon[102692]: pgmap v1394: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 6.7 MiB/s wr, 603 op/s 2023-09-10T01:23:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 160.0 s, 301.8 tps, lat 333.447 ms stddev 107.777 2023-09-10T01:23:46.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:46 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:23:46.560 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:46 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:23:46.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:46 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:23:47.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:47 smithi131 ceph-mon[103713]: pgmap v1395: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 9.6 MiB/s wr, 824 op/s 2023-09-10T01:23:47.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:47 smithi044 ceph-mon[98676]: pgmap v1395: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 9.6 MiB/s wr, 824 op/s 2023-09-10T01:23:47.611 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:47 smithi099 ceph-mon[102692]: pgmap v1395: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 9.6 MiB/s wr, 824 op/s 2023-09-10T01:23:49.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:49 smithi131 ceph-mon[103713]: pgmap v1396: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 8.7 MiB/s wr, 719 op/s 2023-09-10T01:23:49.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:49 smithi044 ceph-mon[98676]: pgmap v1396: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 8.7 MiB/s wr, 719 op/s 2023-09-10T01:23:49.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:49 smithi099 ceph-mon[102692]: pgmap v1396: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 8.7 MiB/s wr, 719 op/s 2023-09-10T01:23:51.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 165.0 s, 276.0 tps, lat 367.578 ms stddev 126.196 2023-09-10T01:23:51.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:51 smithi131 ceph-mon[103713]: pgmap v1397: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 7.2 MiB/s wr, 599 op/s 2023-09-10T01:23:51.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:51 smithi044 ceph-mon[98676]: pgmap v1397: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 7.2 MiB/s wr, 599 op/s 2023-09-10T01:23:51.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:51 smithi099 ceph-mon[102692]: pgmap v1397: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 7.2 MiB/s wr, 599 op/s 2023-09-10T01:23:53.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:53 smithi131 ceph-mon[103713]: pgmap v1398: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 893 op/s 2023-09-10T01:23:53.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:53 smithi044 ceph-mon[98676]: pgmap v1398: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 893 op/s 2023-09-10T01:23:53.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:53 smithi099 ceph-mon[102692]: pgmap v1398: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 893 op/s 2023-09-10T01:23:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.292+0000 7fda816f8700 1 -- 172.21.15.44:0/3515358960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda7c10ed50 msgr2=0x7fda7c107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.292+0000 7fda816f8700 1 --2- 172.21.15.44:0/3515358960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda7c10ed50 0x7fda7c107600 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009990 tx=0x7fda6c009ca0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.292+0000 7fda816f8700 1 -- 172.21.15.44:0/3515358960 shutdown_connections 2023-09-10T01:23:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.292+0000 7fda816f8700 1 --2- 172.21.15.44:0/3515358960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda7c119de0 0x7fda7c11c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.292+0000 7fda816f8700 1 --2- 172.21.15.44:0/3515358960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda7c10ed50 0x7fda7c107600 secure :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009990 tx=0x7fda6c009ca0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.292+0000 7fda816f8700 1 --2- 172.21.15.44:0/3515358960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda7c10f1c0 0x7fda7c107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.292+0000 7fda816f8700 1 -- 172.21.15.44:0/3515358960 >> 172.21.15.44:0/3515358960 conn(0x7fda7c06c6c0 msgr2=0x7fda7c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.298+0000 7fda816f8700 1 -- 172.21.15.44:0/3515358960 shutdown_connections 2023-09-10T01:23:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.298+0000 7fda816f8700 1 -- 172.21.15.44:0/3515358960 wait complete. 2023-09-10T01:23:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.299+0000 7fda816f8700 1 Processor -- start 2023-09-10T01:23:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.299+0000 7fda816f8700 1 -- start start 2023-09-10T01:23:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.299+0000 7fda816f8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda7c10f1c0 0x7fda7c1ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.300+0000 7fda816f8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda7c119de0 0x7fda7c1abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.300+0000 7fda816f8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda7c1a5500 0x7fda7c1a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.300+0000 7fda816f8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c1a5f80 con 0x7fda7c1a5500 2023-09-10T01:23:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.300+0000 7fda816f8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c1a60c0 con 0x7fda7c119de0 2023-09-10T01:23:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.300+0000 7fda816f8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c1a6280 con 0x7fda7c10f1c0 2023-09-10T01:23:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.308+0000 7fda7a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda7c119de0 0x7fda7c1abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.308+0000 7fda7a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda7c119de0 0x7fda7c1abbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60874/0 (socket says 172.21.15.44:60874) 2023-09-10T01:23:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.308+0000 7fda7a7fc700 1 -- 172.21.15.44:0/1196479859 learned_addr learned my addr 172.21.15.44:0/1196479859 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.309+0000 7fda7affd700 1 --2- 172.21.15.44:0/1196479859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda7c10f1c0 0x7fda7c1ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.315+0000 7fda7b7fe700 1 --2- 172.21.15.44:0/1196479859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda7c1a5500 0x7fda7c1a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.330+0000 7fda7a7fc700 1 -- 172.21.15.44:0/1196479859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda7c10f1c0 msgr2=0x7fda7c1ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.330+0000 7fda7a7fc700 1 --2- 172.21.15.44:0/1196479859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda7c10f1c0 0x7fda7c1ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.330+0000 7fda7a7fc700 1 -- 172.21.15.44:0/1196479859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda7c1a5500 msgr2=0x7fda7c1a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.330+0000 7fda7a7fc700 1 --2- 172.21.15.44:0/1196479859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda7c1a5500 0x7fda7c1a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.330+0000 7fda7a7fc700 1 -- 172.21.15.44:0/1196479859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c0095f0 con 0x7fda7c119de0 2023-09-10T01:23:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.344+0000 7fda7a7fc700 1 --2- 172.21.15.44:0/1196479859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda7c119de0 0x7fda7c1abbe0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fda7000e970 tx=0x7fda7000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.355+0000 7fda63fff700 1 -- 172.21.15.44:0/1196479859 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70009790 con 0x7fda7c119de0 2023-09-10T01:23:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.355+0000 7fda63fff700 1 -- 172.21.15.44:0/1196479859 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda7000eec0 con 0x7fda7c119de0 2023-09-10T01:23:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.355+0000 7fda63fff700 1 -- 172.21.15.44:0/1196479859 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda700054b0 con 0x7fda7c119de0 2023-09-10T01:23:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.357+0000 7fda816f8700 1 -- 172.21.15.44:0/1196479859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7c10f620 con 0x7fda7c119de0 2023-09-10T01:23:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.357+0000 7fda816f8700 1 -- 172.21.15.44:0/1196479859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7c10fae0 con 0x7fda7c119de0 2023-09-10T01:23:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.361+0000 7fda8d8beb80 1 Processor -- start 2023-09-10T01:23:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.362+0000 7fda8d8beb80 1 -- start start 2023-09-10T01:23:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.362+0000 7fda8d8beb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2471971a0 0x55d247197580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.362+0000 7fda8d8beb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d247191300 0x55d247191780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.362+0000 7fda8d8beb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191cc0 0x55d24719df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.362+0000 7fda8d8beb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2470ea6d0 con 0x55d247191cc0 2023-09-10T01:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.362+0000 7fda8d8beb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2470e7ff0 con 0x55d247191300 2023-09-10T01:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.362+0000 7fda8d8beb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2470e38e0 con 0x55d2471971a0 2023-09-10T01:23:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.362+0000 7fda7affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d247191300 0x55d247191780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.362+0000 7fda7affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d247191300 0x55d247191780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60878/0 (socket says 172.21.15.44:60878) 2023-09-10T01:23:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.362+0000 7fda7affd700 1 -- 172.21.15.44:0/3436370789 learned_addr learned my addr 172.21.15.44:0/3436370789 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.362+0000 7fda7b7fe700 1 --2- 172.21.15.44:0/3436370789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2471971a0 0x55d247197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.363+0000 7fda7affd700 1 -- 172.21.15.44:0/3436370789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2471971a0 msgr2=0x55d247197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.363+0000 7fda7affd700 1 --2- 172.21.15.44:0/3436370789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2471971a0 0x55d247197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.363+0000 7fda7affd700 1 -- 172.21.15.44:0/3436370789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191cc0 msgr2=0x55d24719df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.363+0000 7fda7affd700 1 --2- 172.21.15.44:0/3436370789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191cc0 0x55d24719df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.363+0000 7fda7affd700 1 -- 172.21.15.44:0/3436370789 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d246f0fd80 con 0x55d247191300 2023-09-10T01:23:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.364+0000 7fda7affd700 1 --2- 172.21.15.44:0/3436370789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d247191300 0x55d247191780 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fda6c006010 tx=0x7fda6c004070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e737ab5c52cb5be6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.364+0000 7fda617fa700 1 -- 172.21.15.44:0/3436370789 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c01d070 con 0x55d247191300 2023-09-10T01:23:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.365+0000 7fda617fa700 1 -- 172.21.15.44:0/3436370789 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda6c01e040 con 0x55d247191300 2023-09-10T01:23:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.365+0000 7fda617fa700 1 -- 172.21.15.44:0/3436370789 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c00b6b0 con 0x55d247191300 2023-09-10T01:23:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.366+0000 7fda8d8beb80 1 -- 172.21.15.44:0/3436370789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d247191300 msgr2=0x55d247191780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.366+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/3436370789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d247191300 0x55d247191780 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fda6c006010 tx=0x7fda6c004070 comp rx=0 tx=0).stop 2023-09-10T01:23:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.366+0000 7fda8d8beb80 1 -- 172.21.15.44:0/3436370789 shutdown_connections 2023-09-10T01:23:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.366+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/3436370789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191cc0 0x55d24719df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.366+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/3436370789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2471971a0 0x55d247197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.366+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/3436370789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d247191300 0x55d247191780 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.366+0000 7fda8d8beb80 1 -- 172.21.15.44:0/3436370789 >> 172.21.15.44:0/3436370789 conn(0x55d247106030 msgr2=0x55d24718c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.367+0000 7fda8d8beb80 1 -- 172.21.15.44:0/3436370789 shutdown_connections 2023-09-10T01:23:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.369+0000 7fda63fff700 1 -- 172.21.15.44:0/1196479859 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fda70005610 con 0x7fda7c119de0 2023-09-10T01:23:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.370+0000 7fda63fff700 1 --2- 172.21.15.44:0/1196479859 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda640a0810 0x7fda640a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.370+0000 7fda63fff700 1 -- 172.21.15.44:0/1196479859 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fda70014070 con 0x7fda7c119de0 2023-09-10T01:23:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.370+0000 7fda8d8beb80 1 -- 172.21.15.44:0/3436370789 wait complete. 2023-09-10T01:23:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.371+0000 7fda8d8beb80 1 -- start start 2023-09-10T01:23:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.371+0000 7fda8d8beb80 1 Processor -- start 2023-09-10T01:23:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.371+0000 7fda7affd700 1 --2- 172.21.15.44:0/1196479859 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda640a0810 0x7fda640a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.372+0000 7fda8d8beb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 0x55d24719df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.372+0000 7fda8d8beb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2471971a0 0x55d24719b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.372+0000 7fda8d8beb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24719b850 0x55d2471a75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.372+0000 7fda8d8beb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d247081b10 con 0x55d247191300 2023-09-10T01:23:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.372+0000 7fda8d8beb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d247081810 con 0x55d2471971a0 2023-09-10T01:23:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.372+0000 7fda8d8beb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d246f658c0 con 0x55d24719b850 2023-09-10T01:23:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.372+0000 7fda8d8beb80 10 client.0 _ll_register_callbacks cb 0x55d246fcba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:23:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.372+0000 7fda7b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24719b850 0x55d2471a75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.373+0000 7fda7b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24719b850 0x55d2471a75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37796/0 (socket says 172.21.15.44:37796) 2023-09-10T01:23:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.373+0000 7fda7b7fe700 1 -- 172.21.15.44:0/341671021 learned_addr learned my addr 172.21.15.44:0/341671021 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.373+0000 7fda7a7fc700 1 --2- 172.21.15.44:0/341671021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2471971a0 0x55d24719b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.373+0000 7fda7b7fe700 1 -- 172.21.15.44:0/341671021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2471971a0 msgr2=0x55d24719b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.373+0000 7fda7b7fe700 1 --2- 172.21.15.44:0/341671021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2471971a0 0x55d24719b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.373+0000 7fda7b7fe700 1 -- 172.21.15.44:0/341671021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 msgr2=0x55d24719df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:23:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.373+0000 7fda7b7fe700 1 --2- 172.21.15.44:0/341671021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 0x55d24719df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.373+0000 7fda7b7fe700 1 -- 172.21.15.44:0/341671021 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7c11cb00 con 0x55d24719b850 2023-09-10T01:23:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.373+0000 7fda7a7fc700 1 --2- 172.21.15.44:0/341671021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2471971a0 0x55d24719b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:23:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.373+0000 7fda7b7fe700 1 --2- 172.21.15.44:0/341671021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24719b850 0x55d2471a75a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fda74008f20 tx=0x7fda740114a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8de4ad4888c88d8c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.373+0000 7fda7affd700 1 --2- 172.21.15.44:0/1196479859 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda640a0810 0x7fda640a2cd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fda6c005f20 tx=0x7fda6c009ca0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.382+0000 7fda5eff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:23:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.382+0000 7fda5eff5700 1 -- 172.21.15.44:0/341671021 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74011ec0 con 0x55d24719b850 2023-09-10T01:23:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.382+0000 7fda5eff5700 1 -- 172.21.15.44:0/341671021 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(176..176 src has 1..176) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fda74016440 con 0x55d24719b850 2023-09-10T01:23:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.384+0000 7fda8d8beb80 1 -- 172.21.15.44:0/341671021 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d24703b490 con 0x55d24719b850 2023-09-10T01:23:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.394+0000 7fda5eff5700 1 -- 172.21.15.44:0/341671021 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=144) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda74016600 con 0x55d24719b850 2023-09-10T01:23:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.394+0000 7fda8d8beb80 10 client.29701 fetch_fsmap learned FSMap version 144 2023-09-10T01:23:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.394+0000 7fda8d8beb80 1 -- 172.21.15.44:0/341671021 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=144,monmap=4+}) v3 -- 0x55d246ef8c00 con 0x55d24719b850 2023-09-10T01:23:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.400+0000 7fda5eff5700 1 -- 172.21.15.44:0/341671021 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d24703b490 con 0x55d24719b850 2023-09-10T01:23:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.400+0000 7fda5eff5700 1 -- 172.21.15.44:0/341671021 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 144) v1 ==== 7367+0+0 (secure 0 0 0) 0x7fda74015680 con 0x55d24719b850 2023-09-10T01:23:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.400+0000 7fda8d8beb80 10 client.29701 fetch_fsmap finished waiting for FSMap version 144 2023-09-10T01:23:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.401+0000 7fda8d8beb80 10 client.29701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:23:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.401+0000 7fda8d8beb80 20 client.29701 populate_metadata read hostname 'smithi044' 2023-09-10T01:23:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.401+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/341671021 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d2471aba10 0x55d2471ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.401+0000 7fda8d8beb80 4 client.29701 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:23:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.401+0000 7fda8d8beb80 1 -- 172.21.15.44:0/341671021 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2470f8840 con 0x55d2471aba10 2023-09-10T01:23:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.401+0000 7fda7affd700 1 --2- 172.21.15.44:0/341671021 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d2471aba10 0x55d2471ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.433+0000 7fda7affd700 1 --2- 172.21.15.44:0/341671021 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d2471aba10 0x55d2471ade30 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26bdd200d555a0ae server_cookie=e35378996cdae28a in_seq=0 out_seq=0 2023-09-10T01:23:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.433+0000 7fda5eff5700 10 client.29701 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.443+0000 7fda5eff5700 1 -- 172.21.15.44:0/341671021 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d2470f8840 con 0x55d2471aba10 2023-09-10T01:23:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.443+0000 7fda5eff5700 10 client.29701 handle_command_reply: tid=0 2023-09-10T01:23:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.444+0000 7fda8d8beb80 1 client.29701 shutdown 2023-09-10T01:23:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.453+0000 7fda8d8beb80 1 -- 172.21.15.44:0/341671021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24719b850 msgr2=0x55d2471a75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.453+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/341671021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24719b850 0x55d2471a75a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fda74008f20 tx=0x7fda740114a0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.474+0000 7fda8d8beb80 1 -- 172.21.15.44:0/341671021 shutdown_connections 2023-09-10T01:23:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.474+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/341671021 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d2471aba10 0x55d2471ade30 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.474+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/341671021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24719b850 0x55d2471a75a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.474+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/341671021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 0x55d24719df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.474+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/341671021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2471971a0 0x55d24719b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.474+0000 7fda8d8beb80 1 -- 172.21.15.44:0/341671021 >> 172.21.15.44:0/341671021 conn(0x55d247106030 msgr2=0x55d24718b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.476+0000 7fda5eff5700 0 client.29701 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.476+0000 7fda8d8beb80 1 -- 172.21.15.44:0/341671021 shutdown_connections 2023-09-10T01:23:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.476+0000 7fda8d8beb80 1 -- 172.21.15.44:0/341671021 wait complete. 2023-09-10T01:23:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.476+0000 7fda8d8beb80 20 client.29701 trim_cache size 0 max 16384 2023-09-10T01:23:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.504+0000 7fda8d8beb80 1 Processor -- start 2023-09-10T01:23:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.504+0000 7fda8d8beb80 1 -- start start 2023-09-10T01:23:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.505+0000 7fda8d8beb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 0x55d2471a2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.505+0000 7fda8d8beb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2471971a0 0x55d247198b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.505+0000 7fda8d8beb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24719b850 0x55d24719dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.505+0000 7fda8d8beb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d247196b40 con 0x55d247191300 2023-09-10T01:23:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.505+0000 7fda8d8beb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d247192100 con 0x55d2471971a0 2023-09-10T01:23:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.505+0000 7fda8d8beb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d247194dd0 con 0x55d24719b850 2023-09-10T01:23:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.505+0000 7fda7affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 0x55d2471a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.505+0000 7fda7affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 0x55d2471a2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59154/0 (socket says 172.21.15.44:59154) 2023-09-10T01:23:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.505+0000 7fda7affd700 1 -- 172.21.15.44:0/2425334432 learned_addr learned my addr 172.21.15.44:0/2425334432 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.506+0000 7fda7affd700 1 -- 172.21.15.44:0/2425334432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24719b850 msgr2=0x55d24719dad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:23:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.506+0000 7fda7affd700 1 --2- 172.21.15.44:0/2425334432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24719b850 0x55d24719dad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.506+0000 7fda7affd700 1 -- 172.21.15.44:0/2425334432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2471971a0 msgr2=0x55d247198b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:23:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.506+0000 7fda7affd700 1 --2- 172.21.15.44:0/2425334432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2471971a0 0x55d247198b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.506+0000 7fda7affd700 1 -- 172.21.15.44:0/2425334432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d246f361f0 con 0x55d247191300 2023-09-10T01:23:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.506+0000 7fda7affd700 1 --2- 172.21.15.44:0/2425334432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 0x55d2471a2520 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fda6c004360 tx=0x7fda6c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5f139a068709e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.507+0000 7fda61ffb700 1 -- 172.21.15.44:0/2425334432 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c035040 con 0x55d247191300 2023-09-10T01:23:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.507+0000 7fda61ffb700 1 -- 172.21.15.44:0/2425334432 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda6c0056c0 con 0x55d247191300 2023-09-10T01:23:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.507+0000 7fda61ffb700 1 -- 172.21.15.44:0/2425334432 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c039730 con 0x55d247191300 2023-09-10T01:23:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.508+0000 7fda8d8beb80 1 -- 172.21.15.44:0/2425334432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 msgr2=0x55d2471a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.508+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/2425334432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 0x55d2471a2520 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fda6c004360 tx=0x7fda6c02c000 comp rx=0 tx=0).stop 2023-09-10T01:23:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.518+0000 7fda8d8beb80 1 -- 172.21.15.44:0/2425334432 shutdown_connections 2023-09-10T01:23:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.518+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/2425334432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24719b850 0x55d24719dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.518+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/2425334432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 0x55d2471a2520 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.518+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/2425334432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2471971a0 0x55d247198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.518+0000 7fda8d8beb80 1 -- 172.21.15.44:0/2425334432 >> 172.21.15.44:0/2425334432 conn(0x55d247106030 msgr2=0x55d247195290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.529+0000 7fda8d8beb80 1 -- 172.21.15.44:0/2425334432 shutdown_connections 2023-09-10T01:23:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.529+0000 7fda8d8beb80 1 -- 172.21.15.44:0/2425334432 wait complete. 2023-09-10T01:23:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.529+0000 7fda8d8beb80 1 -- start start 2023-09-10T01:23:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.529+0000 7fda8d8beb80 1 Processor -- start 2023-09-10T01:23:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.530+0000 7fda8d8beb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 0x55d2471a2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.530+0000 7fda8d8beb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2471971a0 0x55d247199730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.530+0000 7fda8d8beb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24719b850 0x55d247199c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.530+0000 7fda8d8beb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24719a340 con 0x55d247191300 2023-09-10T01:23:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.530+0000 7fda8d8beb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24719a4b0 con 0x55d2471971a0 2023-09-10T01:23:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.530+0000 7fda8d8beb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24719a670 con 0x55d24719b850 2023-09-10T01:23:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.530+0000 7fda8d8beb80 10 client.0 _ll_register_callbacks cb 0x7fda74008790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:23:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.530+0000 7fda7affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 0x55d2471a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.530+0000 7fda7affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 0x55d2471a2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59160/0 (socket says 172.21.15.44:59160) 2023-09-10T01:23:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.530+0000 7fda7affd700 1 -- 172.21.15.44:0/3554626486 learned_addr learned my addr 172.21.15.44:0/3554626486 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:23:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.530+0000 7fda7affd700 1 -- 172.21.15.44:0/3554626486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24719b850 msgr2=0x55d247199c70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:23:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.530+0000 7fda7affd700 1 --2- 172.21.15.44:0/3554626486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24719b850 0x55d247199c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.530+0000 7fda7affd700 1 -- 172.21.15.44:0/3554626486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2471971a0 msgr2=0x55d247199730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:23:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.530+0000 7fda7affd700 1 --2- 172.21.15.44:0/3554626486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2471971a0 0x55d247199730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.530+0000 7fda7affd700 1 -- 172.21.15.44:0/3554626486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d246f0fd80 con 0x55d247191300 2023-09-10T01:23:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.531+0000 7fda7affd700 1 --2- 172.21.15.44:0/3554626486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 0x55d2471a2530 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0098e0 tx=0x7fda6c005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a188ee9fc3d66d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:23:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.531+0000 7fda5eff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:23:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.531+0000 7fda5eff5700 1 -- 172.21.15.44:0/3554626486 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c035070 con 0x55d247191300 2023-09-10T01:23:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.532+0000 7fda5eff5700 1 -- 172.21.15.44:0/3554626486 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(176..176 src has 1..176) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fda6c000e40 con 0x55d247191300 2023-09-10T01:23:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.532+0000 7fda8d8beb80 1 -- 172.21.15.44:0/3554626486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda6c01e040 con 0x55d247191300 2023-09-10T01:23:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.532+0000 7fda5eff5700 1 -- 172.21.15.44:0/3554626486 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=144) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda6c039730 con 0x55d247191300 2023-09-10T01:23:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.532+0000 7fda8d8beb80 10 client.20082 fetch_fsmap learned FSMap version 144 2023-09-10T01:23:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.532+0000 7fda8d8beb80 1 -- 172.21.15.44:0/3554626486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=144,monmap=4+}) v3 -- 0x55d246f0e250 con 0x55d247191300 2023-09-10T01:23:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.532+0000 7fda5eff5700 1 -- 172.21.15.44:0/3554626486 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda6c01e040 con 0x55d247191300 2023-09-10T01:23:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.534+0000 7fda5eff5700 1 -- 172.21.15.44:0/3554626486 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 144) v1 ==== 7367+0+0 (secure 0 0 0) 0x7fda6c04a070 con 0x55d247191300 2023-09-10T01:23:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.534+0000 7fda8d8beb80 10 client.20082 fetch_fsmap finished waiting for FSMap version 144 2023-09-10T01:23:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.534+0000 7fda8d8beb80 10 client.20082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:23:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.534+0000 7fda8d8beb80 20 client.20082 populate_metadata read hostname 'smithi044' 2023-09-10T01:23:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.534+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/3554626486 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d2471ffa10 0x55d247201e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:23:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.534+0000 7fda8d8beb80 4 client.20082 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:23:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.534+0000 7fda8d8beb80 1 -- 172.21.15.44:0/3554626486 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d2470dc8c0 con 0x55d2471ffa10 2023-09-10T01:23:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.538+0000 7fda7a7fc700 1 --2- 172.21.15.44:0/3554626486 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d2471ffa10 0x55d247201e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:23:55.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:55 smithi044 ceph-mon[98676]: pgmap v1399: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 7.1 MiB/s wr, 588 op/s 2023-09-10T01:23:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.558+0000 7fda7a7fc700 1 --2- 172.21.15.44:0/3554626486 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d2471ffa10 0x55d247201e60 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db169949e4fc9e9c server_cookie=1bde0b21ade1fc11 in_seq=0 out_seq=0 2023-09-10T01:23:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.558+0000 7fda5eff5700 10 client.20082 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:55.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:55 smithi099 ceph-mon[102692]: pgmap v1399: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 7.1 MiB/s wr, 588 op/s 2023-09-10T01:23:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.671+0000 7fda5eff5700 1 -- 172.21.15.44:0/3554626486 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d2470dc8c0 con 0x55d2471ffa10 2023-09-10T01:23:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.671+0000 7fda5eff5700 10 client.20082 handle_command_reply: tid=0 2023-09-10T01:23:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.676+0000 7fda8d8beb80 1 client.20082 shutdown 2023-09-10T01:23:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.678+0000 7fda8d8beb80 1 -- 172.21.15.44:0/3554626486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 msgr2=0x55d2471a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.678+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/3554626486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 0x55d2471a2530 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0098e0 tx=0x7fda6c005590 comp rx=0 tx=0).stop 2023-09-10T01:23:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.678+0000 7fda8d8beb80 1 -- 172.21.15.44:0/3554626486 shutdown_connections 2023-09-10T01:23:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.678+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/3554626486 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d2471ffa10 0x55d247201e60 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.678+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/3554626486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24719b850 0x55d247199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.678+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/3554626486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d247191300 0x55d2471a2530 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.678+0000 7fda8d8beb80 1 --2- 172.21.15.44:0/3554626486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d2471971a0 0x55d247199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.678+0000 7fda8d8beb80 1 -- 172.21.15.44:0/3554626486 >> 172.21.15.44:0/3554626486 conn(0x55d247106030 msgr2=0x55d24718c090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.678+0000 7fda5eff5700 0 client.20082 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:23:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.678+0000 7fda8d8beb80 1 -- 172.21.15.44:0/3554626486 shutdown_connections 2023-09-10T01:23:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.679+0000 7fda8d8beb80 1 -- 172.21.15.44:0/3554626486 wait complete. 2023-09-10T01:23:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.679+0000 7fda8d8beb80 20 client.20082 trim_cache size 0 max 16384 2023-09-10T01:23:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:23:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:23:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:23:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:23:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.684+0000 7fda816f8700 1 -- 172.21.15.44:0/1196479859 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda640a0810 msgr2=0x7fda640a2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.684+0000 7fda816f8700 1 --2- 172.21.15.44:0/1196479859 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda640a0810 0x7fda640a2cd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fda6c005f20 tx=0x7fda6c009ca0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.684+0000 7fda816f8700 1 -- 172.21.15.44:0/1196479859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda7c119de0 msgr2=0x7fda7c1abbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:23:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.684+0000 7fda816f8700 1 --2- 172.21.15.44:0/1196479859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda7c119de0 0x7fda7c1abbe0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fda7000e970 tx=0x7fda7000ec80 comp rx=0 tx=0).stop 2023-09-10T01:23:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.688+0000 7fda816f8700 1 -- 172.21.15.44:0/1196479859 shutdown_connections 2023-09-10T01:23:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.688+0000 7fda816f8700 1 --2- 172.21.15.44:0/1196479859 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fda640a0810 0x7fda640a2cd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.688+0000 7fda816f8700 1 --2- 172.21.15.44:0/1196479859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda7c1a5500 0x7fda7c1a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.688+0000 7fda816f8700 1 --2- 172.21.15.44:0/1196479859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda7c10f1c0 0x7fda7c1ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.688+0000 7fda816f8700 1 --2- 172.21.15.44:0/1196479859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda7c119de0 0x7fda7c1abbe0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:23:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.688+0000 7fda816f8700 1 -- 172.21.15.44:0/1196479859 >> 172.21.15.44:0/1196479859 conn(0x7fda7c06c6c0 msgr2=0x7fda7c10c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:23:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.691+0000 7fda816f8700 1 -- 172.21.15.44:0/1196479859 shutdown_connections 2023-09-10T01:23:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:23:55.699+0000 7fda816f8700 1 -- 172.21.15.44:0/1196479859 wait complete. 2023-09-10T01:23:55.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:55 smithi131 ceph-mon[103713]: pgmap v1399: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 7.1 MiB/s wr, 588 op/s 2023-09-10T01:23:56.182 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 170.0 s, 233.8 tps, lat 427.435 ms stddev 238.346 2023-09-10T01:23:56.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:56 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:23:56.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:56 smithi131 ceph-mon[103713]: pgmap v1400: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 842 op/s 2023-09-10T01:23:56.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:56 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:23:56.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:56 smithi044 ceph-mon[98676]: pgmap v1400: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 842 op/s 2023-09-10T01:23:56.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:56 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:23:56.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:56 smithi099 ceph-mon[102692]: pgmap v1400: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 842 op/s 2023-09-10T01:23:58.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:58 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:23:58.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:58 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:23:58.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:58 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:23:59.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:23:59 smithi131 ceph-mon[103713]: pgmap v1401: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 8.2 MiB/s wr, 720 op/s 2023-09-10T01:23:59.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:23:59 smithi044 ceph-mon[98676]: pgmap v1401: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 8.2 MiB/s wr, 720 op/s 2023-09-10T01:23:59.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:23:59 smithi099 ceph-mon[102692]: pgmap v1401: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 8.2 MiB/s wr, 720 op/s 2023-09-10T01:24:01.183 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 175.0 s, 294.4 tps, lat 329.580 ms stddev 116.376 2023-09-10T01:24:01.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:01 smithi131 ceph-mon[103713]: pgmap v1402: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 7.4 MiB/s wr, 646 op/s 2023-09-10T01:24:01.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:01 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:24:01.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:01 smithi044 ceph-mon[98676]: pgmap v1402: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 7.4 MiB/s wr, 646 op/s 2023-09-10T01:24:01.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:01 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:24:01.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:01 smithi099 ceph-mon[102692]: pgmap v1402: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 7.4 MiB/s wr, 646 op/s 2023-09-10T01:24:01.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:01 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:24:03.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:03 smithi131 ceph-mon[103713]: pgmap v1403: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 941 op/s 2023-09-10T01:24:03.567 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:03 smithi044 ceph-mon[98676]: pgmap v1403: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 941 op/s 2023-09-10T01:24:03.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:03 smithi099 ceph-mon[102692]: pgmap v1403: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 941 op/s 2023-09-10T01:24:05.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:05 smithi131 ceph-mon[103713]: pgmap v1404: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 8.1 MiB/s wr, 690 op/s 2023-09-10T01:24:05.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:05 smithi044 ceph-mon[98676]: pgmap v1404: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 8.1 MiB/s wr, 690 op/s 2023-09-10T01:24:05.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:05 smithi099 ceph-mon[102692]: pgmap v1404: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 8.1 MiB/s wr, 690 op/s 2023-09-10T01:24:06.190 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 180.0 s, 342.6 tps, lat 296.914 ms stddev 220.217 2023-09-10T01:24:07.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:07 smithi131 ceph-mon[103713]: pgmap v1405: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 903 op/s 2023-09-10T01:24:07.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:07 smithi044 ceph-mon[98676]: pgmap v1405: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 903 op/s 2023-09-10T01:24:07.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:07 smithi099 ceph-mon[102692]: pgmap v1405: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 903 op/s 2023-09-10T01:24:08.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:08 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:24:08.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:08 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:24:08.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:08 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:24:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.203+0000 7f4e7abb9700 1 -- 172.21.15.44:0/2713163371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e74108fa0 msgr2=0x7f4e74109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.203+0000 7f4e7abb9700 1 --2- 172.21.15.44:0/2713163371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e74108fa0 0x7f4e74109380 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00b0a0 tx=0x7f4e6c00b3b0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.208+0000 7f4e7abb9700 1 -- 172.21.15.44:0/2713163371 shutdown_connections 2023-09-10T01:24:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.208+0000 7f4e7abb9700 1 --2- 172.21.15.44:0/2713163371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e74112550 0x7f4e74114940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.208+0000 7f4e7abb9700 1 --2- 172.21.15.44:0/2713163371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e74108fa0 0x7f4e74109380 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.208+0000 7f4e7abb9700 1 --2- 172.21.15.44:0/2713163371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e741098c0 0x7f4e74112010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.208+0000 7f4e7abb9700 1 -- 172.21.15.44:0/2713163371 >> 172.21.15.44:0/2713163371 conn(0x7f4e7406c6c0 msgr2=0x7f4e7406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.222+0000 7f4e7abb9700 1 -- 172.21.15.44:0/2713163371 shutdown_connections 2023-09-10T01:24:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.222+0000 7f4e7abb9700 1 -- 172.21.15.44:0/2713163371 wait complete. 2023-09-10T01:24:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.222+0000 7f4e7abb9700 1 Processor -- start 2023-09-10T01:24:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.223+0000 7f4e7abb9700 1 -- start start 2023-09-10T01:24:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.223+0000 7f4e7abb9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e74108fa0 0x7f4e74111820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.223+0000 7f4e7abb9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e741098c0 0x7f4e74111d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.223+0000 7f4e7abb9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e74112550 0x7f4e7410b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.223+0000 7f4e7abb9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7410bdc0 con 0x7f4e74112550 2023-09-10T01:24:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.223+0000 7f4e7abb9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7410bf30 con 0x7f4e741098c0 2023-09-10T01:24:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.223+0000 7f4e7abb9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7410c0f0 con 0x7f4e74108fa0 2023-09-10T01:24:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.225+0000 7f4e79156700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e74112550 0x7f4e7410b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.225+0000 7f4e79156700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e74112550 0x7f4e7410b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59186/0 (socket says 172.21.15.44:59186) 2023-09-10T01:24:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.225+0000 7f4e79156700 1 -- 172.21.15.44:0/1671037852 learned_addr learned my addr 172.21.15.44:0/1671037852 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.225+0000 7f4e78955700 1 --2- 172.21.15.44:0/1671037852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e74108fa0 0x7f4e74111820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.225+0000 7f4e73fff700 1 --2- 172.21.15.44:0/1671037852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e741098c0 0x7f4e74111d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.225+0000 7f4e79156700 1 -- 172.21.15.44:0/1671037852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e74108fa0 msgr2=0x7f4e74111820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.225+0000 7f4e79156700 1 --2- 172.21.15.44:0/1671037852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e74108fa0 0x7f4e74111820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.225+0000 7f4e79156700 1 -- 172.21.15.44:0/1671037852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e741098c0 msgr2=0x7f4e74111d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.225+0000 7f4e79156700 1 --2- 172.21.15.44:0/1671037852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e741098c0 0x7f4e74111d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.225+0000 7f4e79156700 1 -- 172.21.15.44:0/1671037852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c009d00 con 0x7f4e74112550 2023-09-10T01:24:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.226+0000 7f4e79156700 1 --2- 172.21.15.44:0/1671037852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e74112550 0x7f4e7410b5f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400d980 tx=0x7f4e6400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.226+0000 7f4e71ffb700 1 -- 172.21.15.44:0/1671037852 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64009940 con 0x7f4e74112550 2023-09-10T01:24:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.227+0000 7f4e71ffb700 1 -- 172.21.15.44:0/1671037852 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e64004830 con 0x7f4e74112550 2023-09-10T01:24:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.227+0000 7f4e71ffb700 1 -- 172.21.15.44:0/1671037852 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64005410 con 0x7f4e74112550 2023-09-10T01:24:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.229+0000 7f4e7abb9700 1 -- 172.21.15.44:0/1671037852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7410c3d0 con 0x7f4e74112550 2023-09-10T01:24:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.230+0000 7f4e7abb9700 1 -- 172.21.15.44:0/1671037852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e74078aa0 con 0x7f4e74112550 2023-09-10T01:24:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.234+0000 7f4e86d7fb80 1 Processor -- start 2023-09-10T01:24:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.235+0000 7f4e86d7fb80 1 -- start start 2023-09-10T01:24:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.235+0000 7f4e86d7fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642295851a0 0x564229585580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.235+0000 7f4e86d7fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56422957f300 0x56422957f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.235+0000 7f4e86d7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56422957fcc0 0x56422958bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.235+0000 7f4e86d7fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642294d86d0 con 0x5642295851a0 2023-09-10T01:24:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.235+0000 7f4e86d7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642294d5ff0 con 0x56422957fcc0 2023-09-10T01:24:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.235+0000 7f4e86d7fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642294d18e0 con 0x56422957f300 2023-09-10T01:24:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.236+0000 7f4e78955700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56422957f300 0x56422957f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.236+0000 7f4e78955700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56422957f300 0x56422957f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37846/0 (socket says 172.21.15.44:37846) 2023-09-10T01:24:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.236+0000 7f4e78955700 1 -- 172.21.15.44:0/777646767 learned_addr learned my addr 172.21.15.44:0/777646767 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.236+0000 7f4e73fff700 1 --2- 172.21.15.44:0/777646767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56422957fcc0 0x56422958bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.236+0000 7f4e73fff700 1 --2- 172.21.15.44:0/777646767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642295851a0 0x564229585580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.239+0000 7f4e73fff700 1 -- 172.21.15.44:0/777646767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56422957f300 msgr2=0x56422957f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.239+0000 7f4e73fff700 1 --2- 172.21.15.44:0/777646767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56422957f300 0x56422957f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.239+0000 7f4e73fff700 1 -- 172.21.15.44:0/777646767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642295851a0 msgr2=0x564229585580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.239+0000 7f4e73fff700 1 --2- 172.21.15.44:0/777646767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642295851a0 0x564229585580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.239+0000 7f4e73fff700 1 -- 172.21.15.44:0/777646767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642292fdd80 con 0x56422957fcc0 2023-09-10T01:24:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.240+0000 7f4e73fff700 1 --2- 172.21.15.44:0/777646767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56422957fcc0 0x56422958bf30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4e7410cff0 tx=0x7f4e6800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a6ae5ccd680be32 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.244+0000 7f4e71ffb700 1 -- 172.21.15.44:0/1671037852 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f4e640049a0 con 0x7f4e74112550 2023-09-10T01:24:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.244+0000 7f4e5affd700 1 -- 172.21.15.44:0/777646767 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68014310 con 0x56422957fcc0 2023-09-10T01:24:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.244+0000 7f4e5affd700 1 -- 172.21.15.44:0/777646767 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e6801a070 con 0x56422957fcc0 2023-09-10T01:24:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.244+0000 7f4e5affd700 1 -- 172.21.15.44:0/777646767 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68003cb0 con 0x56422957fcc0 2023-09-10T01:24:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.245+0000 7f4e71ffb700 1 --2- 172.21.15.44:0/1671037852 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e5c0a09c0 0x7f4e5c0a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.245+0000 7f4e71ffb700 1 -- 172.21.15.44:0/1671037852 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f4e640cbf30 con 0x7f4e74112550 2023-09-10T01:24:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.245+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/777646767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56422957fcc0 msgr2=0x56422958bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.245+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/777646767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56422957fcc0 0x56422958bf30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4e7410cff0 tx=0x7f4e6800c2d0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.245+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/777646767 shutdown_connections 2023-09-10T01:24:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.245+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/777646767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56422957fcc0 0x56422958bf30 secure :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4e7410cff0 tx=0x7f4e6800c2d0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.245+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/777646767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642295851a0 0x564229585580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.245+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/777646767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56422957f300 0x56422957f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.245+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/777646767 >> 172.21.15.44:0/777646767 conn(0x5642294f4030 msgr2=0x56422957a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.246+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/777646767 shutdown_connections 2023-09-10T01:24:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.246+0000 7f4e78955700 1 --2- 172.21.15.44:0/1671037852 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e5c0a09c0 0x7f4e5c0a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.246+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/777646767 wait complete. 2023-09-10T01:24:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.246+0000 7f4e86d7fb80 1 -- start start 2023-09-10T01:24:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.247+0000 7f4e86d7fb80 1 Processor -- start 2023-09-10T01:24:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.247+0000 7f4e86d7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56422957f300 0x564229589320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.247+0000 7f4e86d7fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642295851a0 0x564229589860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.247+0000 7f4e86d7fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564229589da0 0x5642295955a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.247+0000 7f4e86d7fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56422946fb10 con 0x564229589da0 2023-09-10T01:24:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.247+0000 7f4e86d7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56422946f810 con 0x56422957f300 2023-09-10T01:24:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.247+0000 7f4e86d7fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642293538c0 con 0x5642295851a0 2023-09-10T01:24:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.247+0000 7f4e86d7fb80 10 client.0 _ll_register_callbacks cb 0x5642293b9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:24:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.247+0000 7f4e73fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642295851a0 0x564229589860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.247+0000 7f4e73fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642295851a0 0x564229589860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37850/0 (socket says 172.21.15.44:37850) 2023-09-10T01:24:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.247+0000 7f4e73fff700 1 -- 172.21.15.44:0/2956886808 learned_addr learned my addr 172.21.15.44:0/2956886808 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.248+0000 7f4e78955700 1 --2- 172.21.15.44:0/2956886808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56422957f300 0x564229589320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.248+0000 7f4e73fff700 1 -- 172.21.15.44:0/2956886808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56422957f300 msgr2=0x564229589320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.248+0000 7f4e73fff700 1 --2- 172.21.15.44:0/2956886808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56422957f300 0x564229589320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.248+0000 7f4e73fff700 1 -- 172.21.15.44:0/2956886808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564229589da0 msgr2=0x5642295955a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.248+0000 7f4e73fff700 1 --2- 172.21.15.44:0/2956886808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564229589da0 0x5642295955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.248+0000 7f4e73fff700 1 -- 172.21.15.44:0/2956886808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e680146c0 con 0x5642295851a0 2023-09-10T01:24:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.248+0000 7f4e73fff700 1 --2- 172.21.15.44:0/2956886808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642295851a0 0x564229589860 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4e68005950 tx=0x7f4e6800e780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3de9bf1a825c8ff7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.248+0000 7f4e78955700 1 --2- 172.21.15.44:0/1671037852 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e5c0a09c0 0x7f4e5c0a2e80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c000f80 tx=0x7f4e6c009330 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.248+0000 7f4e53fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:24:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.249+0000 7f4e53fff700 1 -- 172.21.15.44:0/2956886808 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68004440 con 0x5642295851a0 2023-09-10T01:24:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.249+0000 7f4e53fff700 1 -- 172.21.15.44:0/2956886808 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(176..176 src has 1..176) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f4e68003890 con 0x5642295851a0 2023-09-10T01:24:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.249+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/2956886808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564229429490 con 0x5642295851a0 2023-09-10T01:24:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.249+0000 7f4e53fff700 1 -- 172.21.15.44:0/2956886808 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e68003a90 con 0x5642295851a0 2023-09-10T01:24:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.249+0000 7f4e86d7fb80 10 client.29713 fetch_fsmap learned FSMap version 146 2023-09-10T01:24:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.249+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/2956886808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=4+}) v3 -- 0x5642292e6c00 con 0x5642295851a0 2023-09-10T01:24:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.249+0000 7f4e53fff700 1 -- 172.21.15.44:0/2956886808 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564229429490 con 0x5642295851a0 2023-09-10T01:24:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.250+0000 7f4e53fff700 1 -- 172.21.15.44:0/2956886808 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 146) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f4e6800c620 con 0x5642295851a0 2023-09-10T01:24:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.250+0000 7f4e86d7fb80 10 client.29713 fetch_fsmap finished waiting for FSMap version 146 2023-09-10T01:24:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.250+0000 7f4e86d7fb80 10 client.29713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:24:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.251+0000 7f4e86d7fb80 20 client.29713 populate_metadata read hostname 'smithi044' 2023-09-10T01:24:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.251+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/2956886808 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564229599940 0x56422959bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.251+0000 7f4e86d7fb80 4 client.29713 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:24:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.251+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/2956886808 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642294e6840 con 0x564229599940 2023-09-10T01:24:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.255+0000 7f4e78955700 1 --2- 172.21.15.44:0/2956886808 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564229599940 0x56422959bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.256+0000 7f4e78955700 1 --2- 172.21.15.44:0/2956886808 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564229599940 0x56422959bd60 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=219d106116c904d1 server_cookie=96aa3adca9563f6 in_seq=0 out_seq=0 2023-09-10T01:24:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.256+0000 7f4e53fff700 10 client.29713 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:24:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.257+0000 7f4e53fff700 1 -- 172.21.15.44:0/2956886808 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5642294e6840 con 0x564229599940 2023-09-10T01:24:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.257+0000 7f4e53fff700 10 client.29713 handle_command_reply: tid=0 2023-09-10T01:24:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.257+0000 7f4e86d7fb80 1 client.29713 shutdown 2023-09-10T01:24:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.260+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/2956886808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642295851a0 msgr2=0x564229589860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.260+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/2956886808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642295851a0 0x564229589860 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4e68005950 tx=0x7f4e6800e780 comp rx=0 tx=0).stop 2023-09-10T01:24:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.270+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/2956886808 shutdown_connections 2023-09-10T01:24:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.270+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/2956886808 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564229599940 0x56422959bd60 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.270+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/2956886808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564229589da0 0x5642295955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.270+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/2956886808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56422957f300 0x564229589320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.270+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/2956886808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642295851a0 0x564229589860 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.270+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/2956886808 >> 172.21.15.44:0/2956886808 conn(0x5642294f4030 msgr2=0x56422957e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.270+0000 7f4e53fff700 0 client.29713 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:24:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.270+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/2956886808 shutdown_connections 2023-09-10T01:24:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.270+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/2956886808 wait complete. 2023-09-10T01:24:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.271+0000 7f4e86d7fb80 20 client.29713 trim_cache size 0 max 16384 2023-09-10T01:24:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.300+0000 7f4e86d7fb80 1 Processor -- start 2023-09-10T01:24:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.300+0000 7f4e86d7fb80 1 -- start start 2023-09-10T01:24:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.300+0000 7f4e86d7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56422957f300 0x564229590530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.300+0000 7f4e86d7fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642295851a0 0x564229586bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.300+0000 7f4e86d7fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564229589da0 0x56422957a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.300+0000 7f4e86d7fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56422957bd40 con 0x5642295851a0 2023-09-10T01:24:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.300+0000 7f4e86d7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56422958c270 con 0x56422957f300 2023-09-10T01:24:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.300+0000 7f4e86d7fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56422957b930 con 0x564229589da0 2023-09-10T01:24:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.301+0000 7f4e73fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642295851a0 0x564229586bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.301+0000 7f4e73fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642295851a0 0x564229586bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59208/0 (socket says 172.21.15.44:59208) 2023-09-10T01:24:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.301+0000 7f4e73fff700 1 -- 172.21.15.44:0/1104604245 learned_addr learned my addr 172.21.15.44:0/1104604245 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.301+0000 7f4e79156700 1 --2- 172.21.15.44:0/1104604245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564229589da0 0x56422957a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.301+0000 7f4e73fff700 1 -- 172.21.15.44:0/1104604245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564229589da0 msgr2=0x56422957a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.301+0000 7f4e73fff700 1 --2- 172.21.15.44:0/1104604245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564229589da0 0x56422957a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.301+0000 7f4e73fff700 1 -- 172.21.15.44:0/1104604245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56422957f300 msgr2=0x564229590530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:24:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.301+0000 7f4e73fff700 1 --2- 172.21.15.44:0/1104604245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56422957f300 0x564229590530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.301+0000 7f4e73fff700 1 -- 172.21.15.44:0/1104604245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642293241f0 con 0x5642295851a0 2023-09-10T01:24:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.301+0000 7f4e73fff700 1 --2- 172.21.15.44:0/1104604245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642295851a0 0x564229586bc0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f4e68017040 tx=0x7f4e6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ea34d7c4d29a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.302+0000 7f4e5b7fe700 1 -- 172.21.15.44:0/1104604245 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6800e3a0 con 0x5642295851a0 2023-09-10T01:24:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.302+0000 7f4e5b7fe700 1 -- 172.21.15.44:0/1104604245 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e680144f0 con 0x5642295851a0 2023-09-10T01:24:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.302+0000 7f4e5b7fe700 1 -- 172.21.15.44:0/1104604245 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68003ba0 con 0x5642295851a0 2023-09-10T01:24:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.303+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/1104604245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642295851a0 msgr2=0x564229586bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.303+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/1104604245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642295851a0 0x564229586bc0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f4e68017040 tx=0x7f4e6800c2d0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.307+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/1104604245 shutdown_connections 2023-09-10T01:24:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.307+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/1104604245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564229589da0 0x56422957a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.307+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/1104604245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56422957f300 0x564229590530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.307+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/1104604245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642295851a0 0x564229586bc0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.307+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/1104604245 >> 172.21.15.44:0/1104604245 conn(0x5642294f4030 msgr2=0x564229583340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.319+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/1104604245 shutdown_connections 2023-09-10T01:24:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.320+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/1104604245 wait complete. 2023-09-10T01:24:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.320+0000 7f4e86d7fb80 1 -- start start 2023-09-10T01:24:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.320+0000 7f4e86d7fb80 1 Processor -- start 2023-09-10T01:24:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.322+0000 7f4e86d7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642295851a0 0x56422958e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.322+0000 7f4e86d7fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564229589da0 0x56422958ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.322+0000 7f4e86d7fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642295ea720 0x5642295ecb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.322+0000 7f4e86d7fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56422958f370 con 0x564229589da0 2023-09-10T01:24:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.322+0000 7f4e86d7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642295ed050 con 0x5642295851a0 2023-09-10T01:24:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.322+0000 7f4e86d7fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642295ed1e0 con 0x5642295ea720 2023-09-10T01:24:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.322+0000 7f4e86d7fb80 10 client.0 _ll_register_callbacks cb 0x7f4e680148e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:24:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.322+0000 7f4e78955700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642295851a0 0x56422958e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.322+0000 7f4e78955700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642295851a0 0x56422958e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60952/0 (socket says 172.21.15.44:60952) 2023-09-10T01:24:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.322+0000 7f4e78955700 1 -- 172.21.15.44:0/4000229269 learned_addr learned my addr 172.21.15.44:0/4000229269 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.323+0000 7f4e78955700 1 -- 172.21.15.44:0/4000229269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642295ea720 msgr2=0x5642295ecb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.323+0000 7f4e78955700 1 --2- 172.21.15.44:0/4000229269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642295ea720 0x5642295ecb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.323+0000 7f4e78955700 1 -- 172.21.15.44:0/4000229269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564229589da0 msgr2=0x56422958ec40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:24:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.323+0000 7f4e78955700 1 --2- 172.21.15.44:0/4000229269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564229589da0 0x56422958ec40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.323+0000 7f4e78955700 1 -- 172.21.15.44:0/4000229269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642292e6c00 con 0x5642295851a0 2023-09-10T01:24:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.323+0000 7f4e78955700 1 --2- 172.21.15.44:0/4000229269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642295851a0 0x56422958e700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00b830 tx=0x7f4e6c00b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=645068f324f45e62 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.324+0000 7f4e53fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:24:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.324+0000 7f4e53fff700 1 -- 172.21.15.44:0/4000229269 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c025070 con 0x5642295851a0 2023-09-10T01:24:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.325+0000 7f4e53fff700 1 -- 172.21.15.44:0/4000229269 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(176..176 src has 1..176) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f4e6c0079d0 con 0x5642295851a0 2023-09-10T01:24:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.325+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/4000229269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e6801a070 con 0x5642295851a0 2023-09-10T01:24:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.327+0000 7f4e53fff700 1 -- 172.21.15.44:0/4000229269 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e6c00e040 con 0x5642295851a0 2023-09-10T01:24:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.327+0000 7f4e86d7fb80 10 client.29729 fetch_fsmap learned FSMap version 146 2023-09-10T01:24:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.327+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/4000229269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=4+}) v3 -- 0x5642292fc250 con 0x5642295851a0 2023-09-10T01:24:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.327+0000 7f4e53fff700 1 -- 172.21.15.44:0/4000229269 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e6801a070 con 0x5642295851a0 2023-09-10T01:24:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.327+0000 7f4e53fff700 1 -- 172.21.15.44:0/4000229269 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 146) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f4e6c02b410 con 0x5642295851a0 2023-09-10T01:24:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.327+0000 7f4e86d7fb80 10 client.29729 fetch_fsmap finished waiting for FSMap version 146 2023-09-10T01:24:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.327+0000 7f4e86d7fb80 10 client.29729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:24:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.327+0000 7f4e86d7fb80 20 client.29729 populate_metadata read hostname 'smithi044' 2023-09-10T01:24:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.327+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/4000229269 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5642295eea80 0x5642295f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.327+0000 7f4e86d7fb80 4 client.29729 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:24:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.327+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/4000229269 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5642294ca8c0 con 0x5642295eea80 2023-09-10T01:24:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.328+0000 7f4e73fff700 1 --2- 172.21.15.44:0/4000229269 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5642295eea80 0x5642295f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.328+0000 7f4e53fff700 10 client.29729 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:24:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.328+0000 7f4e73fff700 1 --2- 172.21.15.44:0/4000229269 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5642295eea80 0x5642295f0ee0 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eb4891622584048 server_cookie=1f9c0727fac3eba3 in_seq=0 out_seq=0 2023-09-10T01:24:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.390+0000 7f4e53fff700 1 -- 172.21.15.44:0/4000229269 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5642294ca8c0 con 0x5642295eea80 2023-09-10T01:24:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.390+0000 7f4e53fff700 10 client.29729 handle_command_reply: tid=0 2023-09-10T01:24:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.390+0000 7f4e86d7fb80 1 client.29729 shutdown 2023-09-10T01:24:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.390+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/4000229269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642295851a0 msgr2=0x56422958e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.390+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/4000229269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642295851a0 0x56422958e700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00b830 tx=0x7f4e6c00b3b0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.394+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/4000229269 shutdown_connections 2023-09-10T01:24:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.394+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/4000229269 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5642295eea80 0x5642295f0ee0 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.394+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/4000229269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642295ea720 0x5642295ecb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.394+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/4000229269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642295851a0 0x56422958e700 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.394+0000 7f4e86d7fb80 1 --2- 172.21.15.44:0/4000229269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564229589da0 0x56422958ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.394+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/4000229269 >> 172.21.15.44:0/4000229269 conn(0x5642294f4030 msgr2=0x56422957a090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.394+0000 7f4e53fff700 0 client.29729 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:24:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.398+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/4000229269 shutdown_connections 2023-09-10T01:24:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.399+0000 7f4e86d7fb80 1 -- 172.21.15.44:0/4000229269 wait complete. 2023-09-10T01:24:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.403+0000 7f4e86d7fb80 20 client.29729 trim_cache size 0 max 16384 2023-09-10T01:24:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:24:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:24:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:24:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.406+0000 7f4e7abb9700 1 -- 172.21.15.44:0/1671037852 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e5c0a09c0 msgr2=0x7f4e5c0a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.406+0000 7f4e7abb9700 1 --2- 172.21.15.44:0/1671037852 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e5c0a09c0 0x7f4e5c0a2e80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c000f80 tx=0x7f4e6c009330 comp rx=0 tx=0).stop 2023-09-10T01:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.406+0000 7f4e7abb9700 1 -- 172.21.15.44:0/1671037852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e74112550 msgr2=0x7f4e7410b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.406+0000 7f4e7abb9700 1 --2- 172.21.15.44:0/1671037852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e74112550 0x7f4e7410b5f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400d980 tx=0x7f4e6400dd40 comp rx=0 tx=0).stop 2023-09-10T01:24:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.410+0000 7f4e7abb9700 1 -- 172.21.15.44:0/1671037852 shutdown_connections 2023-09-10T01:24:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.410+0000 7f4e7abb9700 1 --2- 172.21.15.44:0/1671037852 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e5c0a09c0 0x7f4e5c0a2e80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.410+0000 7f4e7abb9700 1 --2- 172.21.15.44:0/1671037852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e74112550 0x7f4e7410b5f0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.410+0000 7f4e7abb9700 1 --2- 172.21.15.44:0/1671037852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e74108fa0 0x7f4e74111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.410+0000 7f4e7abb9700 1 --2- 172.21.15.44:0/1671037852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e741098c0 0x7f4e74111d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.410+0000 7f4e7abb9700 1 -- 172.21.15.44:0/1671037852 >> 172.21.15.44:0/1671037852 conn(0x7f4e7406c6c0 msgr2=0x7f4e740704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.412+0000 7f4e7abb9700 1 -- 172.21.15.44:0/1671037852 shutdown_connections 2023-09-10T01:24:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:09.413+0000 7f4e7abb9700 1 -- 172.21.15.44:0/1671037852 wait complete. 2023-09-10T01:24:09.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:09 smithi131 ceph-mon[103713]: pgmap v1406: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.4 MiB/s wr, 726 op/s 2023-09-10T01:24:09.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:09 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:09.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:09 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:09.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:09 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:09.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:09 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:09.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:09 smithi044 ceph-mon[98676]: pgmap v1406: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.4 MiB/s wr, 726 op/s 2023-09-10T01:24:09.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:09 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:09.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:09 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:09.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:09 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:09.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:09 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:09.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:09 smithi099 ceph-mon[102692]: pgmap v1406: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.4 MiB/s wr, 726 op/s 2023-09-10T01:24:09.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:09 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:09.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:09 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:09.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:09 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:09.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:09 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:10.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:10 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:10.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:10 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:10.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:10 smithi099 ceph-mon[102692]: pgmap v1407: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 7.6 MiB/s wr, 628 op/s 2023-09-10T01:24:10.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:10 smithi099 ceph-mon[102692]: osdmap e177: 12 total, 12 up, 12 in 2023-09-10T01:24:10.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:10 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:10.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:10 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:10.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:10 smithi131 ceph-mon[103713]: pgmap v1407: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 7.6 MiB/s wr, 628 op/s 2023-09-10T01:24:10.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:10 smithi131 ceph-mon[103713]: osdmap e177: 12 total, 12 up, 12 in 2023-09-10T01:24:10.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:10 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:10.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:10 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:10.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:10 smithi044 ceph-mon[98676]: pgmap v1407: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 7.6 MiB/s wr, 628 op/s 2023-09-10T01:24:10.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:10 smithi044 ceph-mon[98676]: osdmap e177: 12 total, 12 up, 12 in 2023-09-10T01:24:11.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 185.0 s, 239.3 tps, lat 386.081 ms stddev 213.366 2023-09-10T01:24:11.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:11 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:24:11.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:11 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:24:11.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:11 smithi131 ceph-mon[103713]: pgmap v1409: 129 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 81 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.6 MiB/s wr, 671 op/s 2023-09-10T01:24:11.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:11 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:11.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:11 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:24:11.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:11 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:24:11.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:11 smithi044 ceph-mon[98676]: pgmap v1409: 129 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 81 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.6 MiB/s wr, 671 op/s 2023-09-10T01:24:11.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:11 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:11.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:11 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:24:11.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:11 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:24:11.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:11 smithi099 ceph-mon[102692]: pgmap v1409: 129 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 81 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.6 MiB/s wr, 671 op/s 2023-09-10T01:24:11.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:11 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:24:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:13 smithi131 ceph-mon[103713]: pgmap v1410: 129 pgs: 20 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 62 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.6 MiB/s wr, 695 op/s 2023-09-10T01:24:14.063 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:13 smithi044 ceph-mon[98676]: pgmap v1410: 129 pgs: 20 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 62 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.6 MiB/s wr, 695 op/s 2023-09-10T01:24:14.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:13 smithi099 ceph-mon[102692]: pgmap v1410: 129 pgs: 20 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 62 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.6 MiB/s wr, 695 op/s 2023-09-10T01:24:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:15 smithi131 ceph-mon[103713]: pgmap v1411: 129 pgs: 20 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 62 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 5.1 MiB/s wr, 417 op/s 2023-09-10T01:24:16.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:15 smithi044 ceph-mon[98676]: pgmap v1411: 129 pgs: 20 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 62 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 5.1 MiB/s wr, 417 op/s 2023-09-10T01:24:16.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:15 smithi099 ceph-mon[102692]: pgmap v1411: 129 pgs: 20 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 62 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 5.1 MiB/s wr, 417 op/s 2023-09-10T01:24:16.182 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 190.0 s, 162.2 tps, lat 667.691 ms stddev 414.969 2023-09-10T01:24:18.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:17 smithi131 ceph-mon[103713]: pgmap v1412: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 6.3 MiB/s wr, 527 op/s 2023-09-10T01:24:18.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:17 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:24:18.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:17 smithi044 ceph-mon[98676]: pgmap v1412: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 6.3 MiB/s wr, 527 op/s 2023-09-10T01:24:18.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:17 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:24:18.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:17 smithi099 ceph-mon[102692]: pgmap v1412: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 6.3 MiB/s wr, 527 op/s 2023-09-10T01:24:18.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:17 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:24:20.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:19 smithi131 ceph-mon[103713]: pgmap v1413: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 6.7 MiB/s wr, 558 op/s 2023-09-10T01:24:20.103 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:19 smithi044 ceph-mon[98676]: pgmap v1413: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 6.7 MiB/s wr, 558 op/s 2023-09-10T01:24:20.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:19 smithi099 ceph-mon[102692]: pgmap v1413: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 6.7 MiB/s wr, 558 op/s 2023-09-10T01:24:21.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:20 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:24:21.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:20 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:24:21.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 195.0 s, 293.9 tps, lat 343.859 ms stddev 172.018 2023-09-10T01:24:21.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:20 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:24:22.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:22 smithi131 ceph-mon[103713]: pgmap v1414: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.9 MiB/s wr, 713 op/s 2023-09-10T01:24:22.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:22 smithi131 ceph-mon[103713]: osdmap e178: 12 total, 12 up, 12 in 2023-09-10T01:24:22.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:22 smithi044 ceph-mon[98676]: pgmap v1414: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.9 MiB/s wr, 713 op/s 2023-09-10T01:24:22.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:22 smithi044 ceph-mon[98676]: osdmap e178: 12 total, 12 up, 12 in 2023-09-10T01:24:22.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:22 smithi099 ceph-mon[102692]: pgmap v1414: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.9 MiB/s wr, 713 op/s 2023-09-10T01:24:22.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:22 smithi099 ceph-mon[102692]: osdmap e178: 12 total, 12 up, 12 in 2023-09-10T01:24:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.638+0000 7f6485417700 1 -- 172.21.15.44:0/2709663288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64801087b0 msgr2=0x7f6480108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.638+0000 7f6485417700 1 --2- 172.21.15.44:0/2709663288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64801087b0 0x7f6480108b90 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f6470009990 tx=0x7f6470009ca0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.638+0000 7f6485417700 1 -- 172.21.15.44:0/2709663288 shutdown_connections 2023-09-10T01:24:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.638+0000 7f6485417700 1 --2- 172.21.15.44:0/2709663288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6480103160 0x7f648010f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.638+0000 7f6485417700 1 --2- 172.21.15.44:0/2709663288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64801087b0 0x7f6480108b90 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.638+0000 7f6485417700 1 --2- 172.21.15.44:0/2709663288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64801027a0 0x7f6480102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.638+0000 7f6485417700 1 -- 172.21.15.44:0/2709663288 >> 172.21.15.44:0/2709663288 conn(0x7f64800fe270 msgr2=0x7f6480100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.638+0000 7f6485417700 1 -- 172.21.15.44:0/2709663288 shutdown_connections 2023-09-10T01:24:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.638+0000 7f6485417700 1 -- 172.21.15.44:0/2709663288 wait complete. 2023-09-10T01:24:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.639+0000 7f6485417700 1 Processor -- start 2023-09-10T01:24:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.639+0000 7f6485417700 1 -- start start 2023-09-10T01:24:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.639+0000 7f6485417700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64801027a0 0x7f6480198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.639+0000 7f6485417700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6480103160 0x7f6480198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.639+0000 7f6485417700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64801087b0 0x7f648019d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.639+0000 7f6485417700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648019d900 con 0x7f64801027a0 2023-09-10T01:24:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.639+0000 7f6485417700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648019da70 con 0x7f6480103160 2023-09-10T01:24:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.639+0000 7f6485417700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648019dc30 con 0x7f64801087b0 2023-09-10T01:24:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.639+0000 7f647effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64801027a0 0x7f6480198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.639+0000 7f647effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64801027a0 0x7f6480198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59224/0 (socket says 172.21.15.44:59224) 2023-09-10T01:24:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.639+0000 7f647f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64801087b0 0x7f648019d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.639+0000 7f647effd700 1 -- 172.21.15.44:0/554227892 learned_addr learned my addr 172.21.15.44:0/554227892 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.639+0000 7f647e7fc700 1 --2- 172.21.15.44:0/554227892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6480103160 0x7f6480198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.640+0000 7f647effd700 1 -- 172.21.15.44:0/554227892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64801087b0 msgr2=0x7f648019d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.640+0000 7f647effd700 1 --2- 172.21.15.44:0/554227892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64801087b0 0x7f648019d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.640+0000 7f647effd700 1 -- 172.21.15.44:0/554227892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6480103160 msgr2=0x7f6480198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.640+0000 7f647effd700 1 --2- 172.21.15.44:0/554227892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6480103160 0x7f6480198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.640+0000 7f647effd700 1 -- 172.21.15.44:0/554227892 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6474009520 con 0x7f64801027a0 2023-09-10T01:24:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.640+0000 7f647f7fe700 1 --2- 172.21.15.44:0/554227892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64801087b0 0x7f648019d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:24:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.640+0000 7f647effd700 1 --2- 172.21.15.44:0/554227892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64801027a0 0x7f6480198750 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f6470005910 tx=0x7f64700047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.640+0000 7f6467fff700 1 -- 172.21.15.44:0/554227892 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647001d070 con 0x7f64801027a0 2023-09-10T01:24:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.640+0000 7f6467fff700 1 -- 172.21.15.44:0/554227892 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6470021410 con 0x7f64801027a0 2023-09-10T01:24:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.640+0000 7f6485417700 1 -- 172.21.15.44:0/554227892 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64700095f0 con 0x7f64801027a0 2023-09-10T01:24:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.640+0000 7f6467fff700 1 -- 172.21.15.44:0/554227892 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6470017650 con 0x7f64801027a0 2023-09-10T01:24:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.640+0000 7f6485417700 1 -- 172.21.15.44:0/554227892 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64801a55e0 con 0x7f64801027a0 2023-09-10T01:24:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.642+0000 7f6467fff700 1 -- 172.21.15.44:0/554227892 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f647000fb00 con 0x7f64801027a0 2023-09-10T01:24:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.643+0000 7f6467fff700 1 --2- 172.21.15.44:0/554227892 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f64680a07b0 0x7f64680a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.643+0000 7f6467fff700 1 -- 172.21.15.44:0/554227892 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f64700cd7d0 con 0x7f64801027a0 2023-09-10T01:24:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.643+0000 7f647e7fc700 1 --2- 172.21.15.44:0/554227892 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f64680a07b0 0x7f64680a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.644+0000 7f647e7fc700 1 --2- 172.21.15.44:0/554227892 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f64680a07b0 0x7f64680a2c70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6480199f70 tx=0x7f6474009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.644+0000 7f64915ddb80 1 Processor -- start 2023-09-10T01:24:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.644+0000 7f64915ddb80 1 -- start start 2023-09-10T01:24:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.644+0000 7f64915ddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb104591a0 0x55fb10459580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.644+0000 7f64915ddb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb10453300 0x55fb10453780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.644+0000 7f64915ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453cc0 0x55fb1045ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.644+0000 7f64915ddb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb103ac6d0 con 0x55fb10453300 2023-09-10T01:24:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.644+0000 7f64915ddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb103a9ff0 con 0x55fb104591a0 2023-09-10T01:24:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.644+0000 7f64915ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb103a58e0 con 0x55fb10453cc0 2023-09-10T01:24:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.644+0000 7f647effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb104591a0 0x55fb10459580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.645+0000 7f647effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb104591a0 0x55fb10459580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60972/0 (socket says 172.21.15.44:60972) 2023-09-10T01:24:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.645+0000 7f647effd700 1 -- 172.21.15.44:0/3470123714 learned_addr learned my addr 172.21.15.44:0/3470123714 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.645+0000 7f647f7fe700 1 --2- 172.21.15.44:0/3470123714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453cc0 0x55fb1045ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.645+0000 7f647effd700 1 -- 172.21.15.44:0/3470123714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453cc0 msgr2=0x55fb1045ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.645+0000 7f647effd700 1 --2- 172.21.15.44:0/3470123714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453cc0 0x55fb1045ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.645+0000 7f647effd700 1 -- 172.21.15.44:0/3470123714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb10453300 msgr2=0x55fb10453780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.645+0000 7f647effd700 1 --2- 172.21.15.44:0/3470123714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb10453300 0x55fb10453780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.645+0000 7f647effd700 1 -- 172.21.15.44:0/3470123714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb101d1d80 con 0x55fb104591a0 2023-09-10T01:24:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.645+0000 7f647effd700 1 --2- 172.21.15.44:0/3470123714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb104591a0 0x55fb10459580 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6470096e10 tx=0x7f647009a800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aeb724c038db44c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.645+0000 7f64657fa700 1 -- 172.21.15.44:0/3470123714 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647009b010 con 0x55fb104591a0 2023-09-10T01:24:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.645+0000 7f64657fa700 1 -- 172.21.15.44:0/3470123714 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f647009a910 con 0x55fb104591a0 2023-09-10T01:24:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.645+0000 7f64657fa700 1 -- 172.21.15.44:0/3470123714 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64700217f0 con 0x55fb104591a0 2023-09-10T01:24:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.646+0000 7f64915ddb80 1 -- 172.21.15.44:0/3470123714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb104591a0 msgr2=0x55fb10459580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.646+0000 7f64915ddb80 1 --2- 172.21.15.44:0/3470123714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb104591a0 0x55fb10459580 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6470096e10 tx=0x7f647009a800 comp rx=0 tx=0).stop 2023-09-10T01:24:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.646+0000 7f64915ddb80 1 -- 172.21.15.44:0/3470123714 shutdown_connections 2023-09-10T01:24:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.646+0000 7f64915ddb80 1 --2- 172.21.15.44:0/3470123714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453cc0 0x55fb1045ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.646+0000 7f64915ddb80 1 --2- 172.21.15.44:0/3470123714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb104591a0 0x55fb10459580 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.646+0000 7f64915ddb80 1 --2- 172.21.15.44:0/3470123714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb10453300 0x55fb10453780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.646+0000 7f64915ddb80 1 -- 172.21.15.44:0/3470123714 >> 172.21.15.44:0/3470123714 conn(0x55fb103c8030 msgr2=0x55fb1044e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.646+0000 7f64915ddb80 1 -- 172.21.15.44:0/3470123714 shutdown_connections 2023-09-10T01:24:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.646+0000 7f64915ddb80 1 -- 172.21.15.44:0/3470123714 wait complete. 2023-09-10T01:24:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.647+0000 7f64915ddb80 1 -- start start 2023-09-10T01:24:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.647+0000 7f64915ddb80 1 Processor -- start 2023-09-10T01:24:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.647+0000 7f64915ddb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb10453300 0x55fb1046b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.647+0000 7f64915ddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453cc0 0x55fb1046b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.647+0000 7f64915ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb104591a0 0x55fb10465650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.647+0000 7f64915ddb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb10343b10 con 0x55fb10453300 2023-09-10T01:24:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.647+0000 7f64915ddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb10343810 con 0x55fb10453cc0 2023-09-10T01:24:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.647+0000 7f64915ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb102278c0 con 0x55fb104591a0 2023-09-10T01:24:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.647+0000 7f64915ddb80 10 client.0 _ll_register_callbacks cb 0x55fb1028da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:24:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.648+0000 7f647e7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453cc0 0x55fb1046b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.648+0000 7f647f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb104591a0 0x55fb10465650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.648+0000 7f647e7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453cc0 0x55fb1046b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60980/0 (socket says 172.21.15.44:60980) 2023-09-10T01:24:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.648+0000 7f647f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb104591a0 0x55fb10465650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37892/0 (socket says 172.21.15.44:37892) 2023-09-10T01:24:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.648+0000 7f647e7fc700 1 -- 172.21.15.44:0/2121573598 learned_addr learned my addr 172.21.15.44:0/2121573598 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.648+0000 7f647effd700 1 --2- 172.21.15.44:0/2121573598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb10453300 0x55fb1046b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.648+0000 7f647e7fc700 1 -- 172.21.15.44:0/2121573598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb104591a0 msgr2=0x55fb10465650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.648+0000 7f647e7fc700 1 --2- 172.21.15.44:0/2121573598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb104591a0 0x55fb10465650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.648+0000 7f647e7fc700 1 -- 172.21.15.44:0/2121573598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb10453300 msgr2=0x55fb1046b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.648+0000 7f647e7fc700 1 --2- 172.21.15.44:0/2121573598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb10453300 0x55fb1046b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.648+0000 7f647e7fc700 1 -- 172.21.15.44:0/2121573598 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64801a55e0 con 0x55fb10453cc0 2023-09-10T01:24:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.648+0000 7f647f7fe700 1 --2- 172.21.15.44:0/2121573598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb104591a0 0x55fb10465650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:24:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.648+0000 7f647e7fc700 1 --2- 172.21.15.44:0/2121573598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453cc0 0x55fb1046b940 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6474009b60 tx=0x7f6474004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dced7be8e3117f75 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.649+0000 7f6462ff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:24:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.649+0000 7f6462ff5700 1 -- 172.21.15.44:0/2121573598 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6474027660 con 0x55fb10453cc0 2023-09-10T01:24:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.650+0000 7f6462ff5700 1 -- 172.21.15.44:0/2121573598 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f6474018070 con 0x55fb10453cc0 2023-09-10T01:24:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.650+0000 7f64915ddb80 1 -- 172.21.15.44:0/2121573598 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb102fd490 con 0x55fb10453cc0 2023-09-10T01:24:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.650+0000 7f6462ff5700 1 -- 172.21.15.44:0/2121573598 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6474012e90 con 0x55fb10453cc0 2023-09-10T01:24:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.650+0000 7f64915ddb80 10 client.29741 fetch_fsmap learned FSMap version 148 2023-09-10T01:24:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.650+0000 7f64915ddb80 1 -- 172.21.15.44:0/2121573598 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=148,monmap=4+}) v3 -- 0x55fb101bac00 con 0x55fb10453cc0 2023-09-10T01:24:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.650+0000 7f6462ff5700 1 -- 172.21.15.44:0/2121573598 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fb102fd490 con 0x55fb10453cc0 2023-09-10T01:24:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.650+0000 7f6462ff5700 1 -- 172.21.15.44:0/2121573598 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 148) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f64740318d0 con 0x55fb10453cc0 2023-09-10T01:24:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.650+0000 7f64915ddb80 10 client.29741 fetch_fsmap finished waiting for FSMap version 148 2023-09-10T01:24:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.650+0000 7f64915ddb80 10 client.29741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:24:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.650+0000 7f64915ddb80 20 client.29741 populate_metadata read hostname 'smithi044' 2023-09-10T01:24:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.650+0000 7f64915ddb80 1 --2- 172.21.15.44:0/2121573598 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fb10475a20 0x55fb10477e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.650+0000 7f64915ddb80 4 client.29741 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:24:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.650+0000 7f64915ddb80 1 -- 172.21.15.44:0/2121573598 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb103ba840 con 0x55fb10475a20 2023-09-10T01:24:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.651+0000 7f647effd700 1 --2- 172.21.15.44:0/2121573598 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fb10475a20 0x55fb10477e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.651+0000 7f6462ff5700 10 client.29741 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:24:22.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.651+0000 7f647effd700 1 --2- 172.21.15.44:0/2121573598 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fb10475a20 0x55fb10477e40 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef4acffbf411da6f server_cookie=85e7286ffe855545 in_seq=0 out_seq=0 2023-09-10T01:24:22.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.652+0000 7f6462ff5700 1 -- 172.21.15.44:0/2121573598 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fb103ba840 con 0x55fb10475a20 2023-09-10T01:24:22.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.652+0000 7f6462ff5700 10 client.29741 handle_command_reply: tid=0 2023-09-10T01:24:22.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.652+0000 7f64915ddb80 1 client.29741 shutdown 2023-09-10T01:24:22.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.652+0000 7f64915ddb80 1 -- 172.21.15.44:0/2121573598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453cc0 msgr2=0x55fb1046b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.652+0000 7f64915ddb80 1 --2- 172.21.15.44:0/2121573598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453cc0 0x55fb1046b940 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6474009b60 tx=0x7f6474004750 comp rx=0 tx=0).stop 2023-09-10T01:24:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.652+0000 7f64915ddb80 1 -- 172.21.15.44:0/2121573598 shutdown_connections 2023-09-10T01:24:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.652+0000 7f64915ddb80 1 --2- 172.21.15.44:0/2121573598 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fb10475a20 0x55fb10477e40 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.652+0000 7f64915ddb80 1 --2- 172.21.15.44:0/2121573598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb104591a0 0x55fb10465650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.652+0000 7f64915ddb80 1 --2- 172.21.15.44:0/2121573598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb10453300 0x55fb1046b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.652+0000 7f64915ddb80 1 --2- 172.21.15.44:0/2121573598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453cc0 0x55fb1046b940 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.652+0000 7f64915ddb80 1 -- 172.21.15.44:0/2121573598 >> 172.21.15.44:0/2121573598 conn(0x55fb103c8030 msgr2=0x55fb1044d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.652+0000 7f6462ff5700 0 client.29741 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:24:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.652+0000 7f64915ddb80 1 -- 172.21.15.44:0/2121573598 shutdown_connections 2023-09-10T01:24:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.652+0000 7f64915ddb80 1 -- 172.21.15.44:0/2121573598 wait complete. 2023-09-10T01:24:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.652+0000 7f64915ddb80 20 client.29741 trim_cache size 0 max 16384 2023-09-10T01:24:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.680+0000 7f64915ddb80 1 Processor -- start 2023-09-10T01:24:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.680+0000 7f64915ddb80 1 -- start start 2023-09-10T01:24:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.680+0000 7f64915ddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453300 0x55fb10458640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.681+0000 7f64915ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453cc0 0x55fb1044df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.681+0000 7f64915ddb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb104591a0 0x55fb1044e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.681+0000 7f64915ddb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1044afc0 con 0x55fb104591a0 2023-09-10T01:24:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.681+0000 7f64915ddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb10472f40 con 0x55fb10453300 2023-09-10T01:24:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.681+0000 7f64915ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb10454580 con 0x55fb10453cc0 2023-09-10T01:24:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.681+0000 7f647effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453300 0x55fb10458640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.681+0000 7f647effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453300 0x55fb10458640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60986/0 (socket says 172.21.15.44:60986) 2023-09-10T01:24:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.681+0000 7f647effd700 1 -- 172.21.15.44:0/3666874702 learned_addr learned my addr 172.21.15.44:0/3666874702 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.681+0000 7f647e7fc700 1 --2- 172.21.15.44:0/3666874702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453cc0 0x55fb1044df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.681+0000 7f647effd700 1 -- 172.21.15.44:0/3666874702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453cc0 msgr2=0x55fb1044df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.681+0000 7f647effd700 1 --2- 172.21.15.44:0/3666874702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453cc0 0x55fb1044df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.681+0000 7f647effd700 1 -- 172.21.15.44:0/3666874702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb104591a0 msgr2=0x55fb1044e460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:24:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.681+0000 7f647effd700 1 --2- 172.21.15.44:0/3666874702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb104591a0 0x55fb1044e460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.681+0000 7f647effd700 1 -- 172.21.15.44:0/3666874702 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb104658e0 con 0x55fb10453300 2023-09-10T01:24:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.681+0000 7f647effd700 1 --2- 172.21.15.44:0/3666874702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453300 0x55fb10458640 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6470096e10 tx=0x7f6470005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25461455a2ab0fc server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.683+0000 7f6465ffb700 1 -- 172.21.15.44:0/3666874702 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647009a450 con 0x55fb10453300 2023-09-10T01:24:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.683+0000 7f6465ffb700 1 -- 172.21.15.44:0/3666874702 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f64700a28f0 con 0x55fb10453300 2023-09-10T01:24:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.683+0000 7f6465ffb700 1 -- 172.21.15.44:0/3666874702 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64700b87a0 con 0x55fb10453300 2023-09-10T01:24:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.684+0000 7f64915ddb80 1 -- 172.21.15.44:0/3666874702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453300 msgr2=0x55fb10458640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.684+0000 7f64915ddb80 1 --2- 172.21.15.44:0/3666874702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453300 0x55fb10458640 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6470096e10 tx=0x7f6470005ad0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.684+0000 7f64915ddb80 1 -- 172.21.15.44:0/3666874702 shutdown_connections 2023-09-10T01:24:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.684+0000 7f64915ddb80 1 --2- 172.21.15.44:0/3666874702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb104591a0 0x55fb1044e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.684+0000 7f64915ddb80 1 --2- 172.21.15.44:0/3666874702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453300 0x55fb10458640 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.684+0000 7f64915ddb80 1 --2- 172.21.15.44:0/3666874702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453cc0 0x55fb1044df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.684+0000 7f64915ddb80 1 -- 172.21.15.44:0/3666874702 >> 172.21.15.44:0/3666874702 conn(0x55fb103c8030 msgr2=0x55fb10457f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.684+0000 7f64915ddb80 1 -- 172.21.15.44:0/3666874702 shutdown_connections 2023-09-10T01:24:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.684+0000 7f64915ddb80 1 -- 172.21.15.44:0/3666874702 wait complete. 2023-09-10T01:24:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.684+0000 7f64915ddb80 1 -- start start 2023-09-10T01:24:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.685+0000 7f64915ddb80 1 Processor -- start 2023-09-10T01:24:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.685+0000 7f64915ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453300 0x55fb1045e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.685+0000 7f64915ddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453cc0 0x55fb1045ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.685+0000 7f64915ddb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb104591a0 0x55fb10470e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.685+0000 7f64915ddb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb10471530 con 0x55fb104591a0 2023-09-10T01:24:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.685+0000 7f64915ddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb104716a0 con 0x55fb10453cc0 2023-09-10T01:24:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.685+0000 7f64915ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb10471860 con 0x55fb10453300 2023-09-10T01:24:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.685+0000 7f64915ddb80 10 client.0 _ll_register_callbacks cb 0x7f6474012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:24:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.685+0000 7f647f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb104591a0 0x55fb10470e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.685+0000 7f647effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453300 0x55fb1045e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.685+0000 7f647effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453300 0x55fb1045e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37900/0 (socket says 172.21.15.44:37900) 2023-09-10T01:24:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.685+0000 7f647effd700 1 -- 172.21.15.44:0/1210839975 learned_addr learned my addr 172.21.15.44:0/1210839975 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.685+0000 7f647f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb104591a0 0x55fb10470e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59252/0 (socket says 172.21.15.44:59252) 2023-09-10T01:24:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.685+0000 7f647effd700 1 -- 172.21.15.44:0/1210839975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453cc0 msgr2=0x55fb1045ee30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:24:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.685+0000 7f647effd700 1 --2- 172.21.15.44:0/1210839975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453cc0 0x55fb1045ee30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.685+0000 7f647effd700 1 -- 172.21.15.44:0/1210839975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb104591a0 msgr2=0x55fb10470e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.686+0000 7f647effd700 1 --2- 172.21.15.44:0/1210839975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb104591a0 0x55fb10470e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.686+0000 7f647effd700 1 -- 172.21.15.44:0/1210839975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb101d1d80 con 0x55fb10453300 2023-09-10T01:24:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.686+0000 7f647f7fe700 1 --2- 172.21.15.44:0/1210839975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb104591a0 0x55fb10470e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:24:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.686+0000 7f647effd700 1 --2- 172.21.15.44:0/1210839975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453300 0x55fb1045e8f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6470098c60 tx=0x7f6470096640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24e450f2ff6c6862 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.686+0000 7f6462ff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:24:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.686+0000 7f6462ff5700 1 -- 172.21.15.44:0/1210839975 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6470021640 con 0x55fb10453300 2023-09-10T01:24:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.686+0000 7f6462ff5700 1 -- 172.21.15.44:0/1210839975 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f6470098cb0 con 0x55fb10453300 2023-09-10T01:24:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.686+0000 7f64915ddb80 1 -- 172.21.15.44:0/1210839975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f647009a910 con 0x55fb10453300 2023-09-10T01:24:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.696+0000 7f6462ff5700 1 -- 172.21.15.44:0/1210839975 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64700217f0 con 0x55fb10453300 2023-09-10T01:24:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.696+0000 7f64915ddb80 10 client.29734 fetch_fsmap learned FSMap version 148 2023-09-10T01:24:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.696+0000 7f64915ddb80 1 -- 172.21.15.44:0/1210839975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=148,monmap=4+}) v3 -- 0x55fb101f81f0 con 0x55fb10453300 2023-09-10T01:24:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.696+0000 7f6462ff5700 1 -- 172.21.15.44:0/1210839975 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f64700a6080 con 0x55fb10453300 2023-09-10T01:24:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.696+0000 7f6462ff5700 1 -- 172.21.15.44:0/1210839975 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 148) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f64700a2460 con 0x55fb10453300 2023-09-10T01:24:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.696+0000 7f64915ddb80 10 client.29734 fetch_fsmap finished waiting for FSMap version 148 2023-09-10T01:24:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.696+0000 7f64915ddb80 10 client.29734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:24:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.696+0000 7f64915ddb80 20 client.29734 populate_metadata read hostname 'smithi044' 2023-09-10T01:24:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.696+0000 7f64915ddb80 1 --2- 172.21.15.44:0/1210839975 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fb104652f0 0x55fb104d47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.696+0000 7f64915ddb80 4 client.29734 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:24:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.696+0000 7f64915ddb80 1 -- 172.21.15.44:0/1210839975 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fb1039e8c0 con 0x55fb104652f0 2023-09-10T01:24:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.696+0000 7f647e7fc700 1 --2- 172.21.15.44:0/1210839975 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fb104652f0 0x55fb104d47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.697+0000 7f6462ff5700 10 client.29734 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:24:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.697+0000 7f647e7fc700 1 --2- 172.21.15.44:0/1210839975 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fb104652f0 0x55fb104d47e0 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b50545c6863d85c0 server_cookie=2fdce78ae057c393 in_seq=0 out_seq=0 2023-09-10T01:24:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.780+0000 7f6462ff5700 1 -- 172.21.15.44:0/1210839975 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f6474012e90 con 0x55fb104652f0 2023-09-10T01:24:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.780+0000 7f6462ff5700 10 client.29734 handle_command_reply: tid=0 2023-09-10T01:24:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.780+0000 7f64915ddb80 1 client.29734 shutdown 2023-09-10T01:24:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.791+0000 7f64915ddb80 1 -- 172.21.15.44:0/1210839975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453300 msgr2=0x55fb1045e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.791+0000 7f64915ddb80 1 --2- 172.21.15.44:0/1210839975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453300 0x55fb1045e8f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6470098c60 tx=0x7f6470096640 comp rx=0 tx=0).stop 2023-09-10T01:24:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.803+0000 7f64915ddb80 1 -- 172.21.15.44:0/1210839975 shutdown_connections 2023-09-10T01:24:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.803+0000 7f64915ddb80 1 --2- 172.21.15.44:0/1210839975 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fb104652f0 0x55fb104d47e0 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.803+0000 7f64915ddb80 1 --2- 172.21.15.44:0/1210839975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb104591a0 0x55fb10470e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.803+0000 7f64915ddb80 1 --2- 172.21.15.44:0/1210839975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb10453300 0x55fb1045e8f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.803+0000 7f64915ddb80 1 --2- 172.21.15.44:0/1210839975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fb10453cc0 0x55fb1045ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.803+0000 7f64915ddb80 1 -- 172.21.15.44:0/1210839975 >> 172.21.15.44:0/1210839975 conn(0x55fb103c8030 msgr2=0x55fb104522f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.807+0000 7f6462ff5700 0 client.29734 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:24:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.814+0000 7f64915ddb80 1 -- 172.21.15.44:0/1210839975 shutdown_connections 2023-09-10T01:24:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.814+0000 7f64915ddb80 1 -- 172.21.15.44:0/1210839975 wait complete. 2023-09-10T01:24:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.839+0000 7f64915ddb80 20 client.29734 trim_cache size 0 max 16384 2023-09-10T01:24:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:24:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:24:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:24:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:24:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.851+0000 7f6485417700 1 -- 172.21.15.44:0/554227892 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f64680a07b0 msgr2=0x7f64680a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.851+0000 7f6485417700 1 --2- 172.21.15.44:0/554227892 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f64680a07b0 0x7f64680a2c70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6480199f70 tx=0x7f6474009260 comp rx=0 tx=0).stop 2023-09-10T01:24:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.852+0000 7f6485417700 1 -- 172.21.15.44:0/554227892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64801027a0 msgr2=0x7f6480198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.852+0000 7f6485417700 1 --2- 172.21.15.44:0/554227892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64801027a0 0x7f6480198750 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f6470005910 tx=0x7f64700047b0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.860+0000 7f6485417700 1 -- 172.21.15.44:0/554227892 shutdown_connections 2023-09-10T01:24:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.860+0000 7f6485417700 1 --2- 172.21.15.44:0/554227892 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f64680a07b0 0x7f64680a2c70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.860+0000 7f6485417700 1 --2- 172.21.15.44:0/554227892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64801087b0 0x7f648019d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.860+0000 7f6485417700 1 --2- 172.21.15.44:0/554227892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64801027a0 0x7f6480198750 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.860+0000 7f6485417700 1 --2- 172.21.15.44:0/554227892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6480103160 0x7f6480198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.860+0000 7f6485417700 1 -- 172.21.15.44:0/554227892 >> 172.21.15.44:0/554227892 conn(0x7f64800fe270 msgr2=0x7f64800fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.861+0000 7f6485417700 1 -- 172.21.15.44:0/554227892 shutdown_connections 2023-09-10T01:24:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:22.861+0000 7f6485417700 1 -- 172.21.15.44:0/554227892 wait complete. 2023-09-10T01:24:24.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:24 smithi131 ceph-mon[103713]: pgmap v1416: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.9 MiB/s wr, 602 op/s 2023-09-10T01:24:24.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:24 smithi044 ceph-mon[98676]: pgmap v1416: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.9 MiB/s wr, 602 op/s 2023-09-10T01:24:24.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:24 smithi099 ceph-mon[102692]: pgmap v1416: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.9 MiB/s wr, 602 op/s 2023-09-10T01:24:26.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 200.0 s, 193.0 tps, lat 382.253 ms stddev 136.346 2023-09-10T01:24:26.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:26 smithi131 ceph-mon[103713]: pgmap v1417: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.9 MiB/s wr, 602 op/s 2023-09-10T01:24:26.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:26 smithi044 ceph-mon[98676]: pgmap v1417: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.9 MiB/s wr, 602 op/s 2023-09-10T01:24:26.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:26 smithi099 ceph-mon[102692]: pgmap v1417: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.9 MiB/s wr, 602 op/s 2023-09-10T01:24:28.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:28 smithi131 ceph-mon[103713]: pgmap v1418: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.2 MiB/s wr, 705 op/s 2023-09-10T01:24:28.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:28 smithi044 ceph-mon[98676]: pgmap v1418: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.2 MiB/s wr, 705 op/s 2023-09-10T01:24:28.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:28 smithi099 ceph-mon[102692]: pgmap v1418: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.2 MiB/s wr, 705 op/s 2023-09-10T01:24:30.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:30 smithi131 ceph-mon[103713]: pgmap v1419: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.1 MiB/s wr, 699 op/s 2023-09-10T01:24:30.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:30 smithi044 ceph-mon[98676]: pgmap v1419: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.1 MiB/s wr, 699 op/s 2023-09-10T01:24:30.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:30 smithi099 ceph-mon[102692]: pgmap v1419: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.1 MiB/s wr, 699 op/s 2023-09-10T01:24:31.186 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 205.0 s, 265.2 tps, lat 473.209 ms stddev 544.572 2023-09-10T01:24:32.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:32 smithi131 ceph-mon[103713]: pgmap v1420: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.1 MiB/s wr, 606 op/s 2023-09-10T01:24:32.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:32 smithi044 ceph-mon[98676]: pgmap v1420: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.1 MiB/s wr, 606 op/s 2023-09-10T01:24:32.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:32 smithi099 ceph-mon[102692]: pgmap v1420: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.1 MiB/s wr, 606 op/s 2023-09-10T01:24:34.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:34 smithi131 ceph-mon[103713]: pgmap v1421: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 6.0 MiB/s wr, 553 op/s 2023-09-10T01:24:34.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:34 smithi044 ceph-mon[98676]: pgmap v1421: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 6.0 MiB/s wr, 553 op/s 2023-09-10T01:24:34.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:34 smithi099 ceph-mon[102692]: pgmap v1421: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 6.0 MiB/s wr, 553 op/s 2023-09-10T01:24:36.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.069+0000 7f348cdf1700 1 -- 172.21.15.44:0/2854018446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348810f1c0 msgr2=0x7f3488107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.069+0000 7f348cdf1700 1 --2- 172.21.15.44:0/2854018446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348810f1c0 0x7f3488107b40 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f34780098f0 tx=0x7f3478009c00 comp rx=0 tx=0).stop 2023-09-10T01:24:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.069+0000 7f348cdf1700 1 -- 172.21.15.44:0/2854018446 shutdown_connections 2023-09-10T01:24:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.069+0000 7f348cdf1700 1 --2- 172.21.15.44:0/2854018446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3488119de0 0x7f348811c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.069+0000 7f348cdf1700 1 --2- 172.21.15.44:0/2854018446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f348810ed50 0x7f3488107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.069+0000 7f348cdf1700 1 --2- 172.21.15.44:0/2854018446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348810f1c0 0x7f3488107b40 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.069+0000 7f348cdf1700 1 -- 172.21.15.44:0/2854018446 >> 172.21.15.44:0/2854018446 conn(0x7f348806c6c0 msgr2=0x7f348806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.070+0000 7f348cdf1700 1 -- 172.21.15.44:0/2854018446 shutdown_connections 2023-09-10T01:24:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.070+0000 7f348cdf1700 1 -- 172.21.15.44:0/2854018446 wait complete. 2023-09-10T01:24:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.071+0000 7f348cdf1700 1 Processor -- start 2023-09-10T01:24:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.071+0000 7f348cdf1700 1 -- start start 2023-09-10T01:24:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.072+0000 7f348cdf1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348810ed50 0x7f3488117100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.072+0000 7f348cdf1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f348810f1c0 0x7f3488117680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.072+0000 7f348cdf1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3488119de0 0x7f348810ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.072+0000 7f348cdf1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34881106f0 con 0x7f348810ed50 2023-09-10T01:24:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.072+0000 7f348cdf1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3488110860 con 0x7f348810f1c0 2023-09-10T01:24:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.072+0000 7f348cdf1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3488110a20 con 0x7f3488119de0 2023-09-10T01:24:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.072+0000 7f3485d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f348810f1c0 0x7f3488117680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.072+0000 7f3485d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f348810f1c0 0x7f3488117680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:32776/0 (socket says 172.21.15.44:32776) 2023-09-10T01:24:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.072+0000 7f3485d9b700 1 -- 172.21.15.44:0/3998450343 learned_addr learned my addr 172.21.15.44:0/3998450343 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.073+0000 7f348659c700 1 --2- 172.21.15.44:0/3998450343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348810ed50 0x7f3488117100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.073+0000 7f348659c700 1 -- 172.21.15.44:0/3998450343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3488119de0 msgr2=0x7f348810ff20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-10T01:24:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.073+0000 7f348659c700 1 --2- 172.21.15.44:0/3998450343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3488119de0 0x7f348810ff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.073+0000 7f348659c700 1 -- 172.21.15.44:0/3998450343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f348810f1c0 msgr2=0x7f3488117680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.073+0000 7f348659c700 1 --2- 172.21.15.44:0/3998450343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f348810f1c0 0x7f3488117680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.073+0000 7f348659c700 1 -- 172.21.15.44:0/3998450343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3480009c40 con 0x7f348810ed50 2023-09-10T01:24:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.074+0000 7f348659c700 1 --2- 172.21.15.44:0/3998450343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348810ed50 0x7f3488117100 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f347c00d950 tx=0x7f347c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.074+0000 7f34777fe700 1 -- 172.21.15.44:0/3998450343 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c009b40 con 0x7f348810ed50 2023-09-10T01:24:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.075+0000 7f34777fe700 1 -- 172.21.15.44:0/3998450343 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f347c00be10 con 0x7f348810ed50 2023-09-10T01:24:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.075+0000 7f34777fe700 1 -- 172.21.15.44:0/3998450343 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c010e90 con 0x7f348810ed50 2023-09-10T01:24:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.079+0000 7f348cdf1700 1 -- 172.21.15.44:0/3998450343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3478009520 con 0x7f348810ed50 2023-09-10T01:24:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.079+0000 7f348cdf1700 1 -- 172.21.15.44:0/3998450343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3488114e70 con 0x7f348810ed50 2023-09-10T01:24:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.087+0000 7f34777fe700 1 -- 172.21.15.44:0/3998450343 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f347c010460 con 0x7f348810ed50 2023-09-10T01:24:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.088+0000 7f34777fe700 1 --2- 172.21.15.44:0/3998450343 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f34700a0830 0x7f34700a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.088+0000 7f34777fe700 1 -- 172.21.15.44:0/3998450343 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f347c0cc050 con 0x7f348810ed50 2023-09-10T01:24:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.092+0000 7f3498fb7b80 1 Processor -- start 2023-09-10T01:24:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.092+0000 7f3498fb7b80 1 -- start start 2023-09-10T01:24:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.092+0000 7f3498fb7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f91a0 0x5613742f9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.092+0000 7f3498fb7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f3300 0x5613742f3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.092+0000 7f3498fb7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f3cc0 0x5613742fff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.092+0000 7f3498fb7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137424c6d0 con 0x5613742f3300 2023-09-10T01:24:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.092+0000 7f3498fb7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561374249ff0 con 0x5613742f3cc0 2023-09-10T01:24:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.092+0000 7f3498fb7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613742458e0 con 0x5613742f91a0 2023-09-10T01:24:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.092+0000 7f348659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f91a0 0x5613742f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.092+0000 7f348659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f91a0 0x5613742f9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37920/0 (socket says 172.21.15.44:37920) 2023-09-10T01:24:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.092+0000 7f348659c700 1 -- 172.21.15.44:0/1676405570 learned_addr learned my addr 172.21.15.44:0/1676405570 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.093+0000 7f3486d9d700 1 --2- 172.21.15.44:0/1676405570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f3cc0 0x5613742fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.093+0000 7f348659c700 1 -- 172.21.15.44:0/1676405570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f3cc0 msgr2=0x5613742fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.093+0000 7f348659c700 1 --2- 172.21.15.44:0/1676405570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f3cc0 0x5613742fff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.093+0000 7f348659c700 1 -- 172.21.15.44:0/1676405570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f3300 msgr2=0x5613742f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.093+0000 7f348659c700 1 --2- 172.21.15.44:0/1676405570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f3300 0x5613742f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.093+0000 7f348659c700 1 -- 172.21.15.44:0/1676405570 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561374071d80 con 0x5613742f91a0 2023-09-10T01:24:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.093+0000 7f348659c700 1 --2- 172.21.15.44:0/1676405570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f91a0 0x5613742f9580 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f347c0c99a0 tx=0x7f347c009d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8627697ef262d0dd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.093+0000 7f3474ff9700 1 -- 172.21.15.44:0/1676405570 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c0b0070 con 0x5613742f91a0 2023-09-10T01:24:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.094+0000 7f3474ff9700 1 -- 172.21.15.44:0/1676405570 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f347c099100 con 0x5613742f91a0 2023-09-10T01:24:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.094+0000 7f3474ff9700 1 -- 172.21.15.44:0/1676405570 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c0a0f40 con 0x5613742f91a0 2023-09-10T01:24:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.095+0000 7f3498fb7b80 1 -- 172.21.15.44:0/1676405570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f91a0 msgr2=0x5613742f9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.095+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/1676405570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f91a0 0x5613742f9580 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f347c0c99a0 tx=0x7f347c009d20 comp rx=0 tx=0).stop 2023-09-10T01:24:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.095+0000 7f3498fb7b80 1 -- 172.21.15.44:0/1676405570 shutdown_connections 2023-09-10T01:24:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.095+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/1676405570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f3cc0 0x5613742fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.095+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/1676405570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f91a0 0x5613742f9580 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.095+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/1676405570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f3300 0x5613742f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.095+0000 7f3498fb7b80 1 -- 172.21.15.44:0/1676405570 >> 172.21.15.44:0/1676405570 conn(0x561374268030 msgr2=0x5613742ee560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.095+0000 7f3498fb7b80 1 -- 172.21.15.44:0/1676405570 shutdown_connections 2023-09-10T01:24:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.095+0000 7f3485d9b700 1 --2- 172.21.15.44:0/3998450343 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f34700a0830 0x7f34700a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.095+0000 7f3498fb7b80 1 -- 172.21.15.44:0/1676405570 wait complete. 2023-09-10T01:24:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.096+0000 7f3498fb7b80 1 -- start start 2023-09-10T01:24:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.096+0000 7f3498fb7b80 1 Processor -- start 2023-09-10T01:24:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.096+0000 7f3498fb7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f3300 0x561374302d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.096+0000 7f3498fb7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f3cc0 0x5613743032d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.096+0000 7f3498fb7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f91a0 0x5613742fd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.097+0000 7f3498fb7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613741e3b10 con 0x5613742f3cc0 2023-09-10T01:24:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.097+0000 7f3498fb7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613741e3810 con 0x5613742f91a0 2023-09-10T01:24:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.097+0000 7f3498fb7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613740c78c0 con 0x5613742f3300 2023-09-10T01:24:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.097+0000 7f3498fb7b80 10 client.0 _ll_register_callbacks cb 0x56137412da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:24:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.097+0000 7f3486d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f91a0 0x5613742fd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.097+0000 7f348659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f3300 0x561374302d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.097+0000 7f3486d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f91a0 0x5613742fd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:32788/0 (socket says 172.21.15.44:32788) 2023-09-10T01:24:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.097+0000 7f3486d9d700 1 -- 172.21.15.44:0/4259749624 learned_addr learned my addr 172.21.15.44:0/4259749624 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.097+0000 7f348659c700 1 -- 172.21.15.44:0/4259749624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f91a0 msgr2=0x5613742fd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.097+0000 7f348659c700 1 --2- 172.21.15.44:0/4259749624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f91a0 0x5613742fd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.097+0000 7f348659c700 1 -- 172.21.15.44:0/4259749624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f3cc0 msgr2=0x5613743032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.097+0000 7f348659c700 1 --2- 172.21.15.44:0/4259749624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f3cc0 0x5613743032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.097+0000 7f348659c700 1 -- 172.21.15.44:0/4259749624 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3488114e70 con 0x5613742f3300 2023-09-10T01:24:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.098+0000 7f348659c700 1 --2- 172.21.15.44:0/4259749624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f3300 0x561374302d90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f347c005950 tx=0x7f347c0a1170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb125d26f8fdb5f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.098+0000 7f3485d9b700 1 --2- 172.21.15.44:0/3998450343 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f34700a0830 0x7f34700a2cf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x5613742ee060 tx=0x7f3478009f50 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.102+0000 7f346dffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:24:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.102+0000 7f346dffb700 1 -- 172.21.15.44:0/4259749624 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c0b0040 con 0x5613742f3300 2023-09-10T01:24:36.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.102+0000 7f346dffb700 1 -- 172.21.15.44:0/4259749624 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f347c099560 con 0x5613742f3300 2023-09-10T01:24:36.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.102+0000 7f3498fb7b80 1 -- 172.21.15.44:0/4259749624 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56137419d490 con 0x5613742f3300 2023-09-10T01:24:36.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.105+0000 7f346dffb700 1 -- 172.21.15.44:0/4259749624 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f347c0985f0 con 0x5613742f3300 2023-09-10T01:24:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.105+0000 7f3498fb7b80 10 client.29743 fetch_fsmap learned FSMap version 148 2023-09-10T01:24:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.105+0000 7f3498fb7b80 1 -- 172.21.15.44:0/4259749624 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=148,monmap=4+}) v3 -- 0x56137405ac00 con 0x5613742f3300 2023-09-10T01:24:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.110+0000 7f346dffb700 1 -- 172.21.15.44:0/4259749624 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56137419d490 con 0x5613742f3300 2023-09-10T01:24:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.110+0000 7f346dffb700 1 -- 172.21.15.44:0/4259749624 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 148) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f347c01bc40 con 0x5613742f3300 2023-09-10T01:24:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.111+0000 7f3498fb7b80 10 client.29743 fetch_fsmap finished waiting for FSMap version 148 2023-09-10T01:24:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.111+0000 7f3498fb7b80 10 client.29743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:24:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.111+0000 7f3498fb7b80 20 client.29743 populate_metadata read hostname 'smithi044' 2023-09-10T01:24:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.111+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/4259749624 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56137430cdb0 0x56137430f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.111+0000 7f3498fb7b80 4 client.29743 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:24:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.111+0000 7f3498fb7b80 1 -- 172.21.15.44:0/4259749624 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56137425a840 con 0x56137430cdb0 2023-09-10T01:24:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.111+0000 7f3485d9b700 1 --2- 172.21.15.44:0/4259749624 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56137430cdb0 0x56137430f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.112+0000 7f3485d9b700 1 --2- 172.21.15.44:0/4259749624 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56137430cdb0 0x56137430f1d0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=480ca6bb4cfcbd8 server_cookie=e710b1c355cdea84 in_seq=0 out_seq=0 2023-09-10T01:24:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.112+0000 7f346dffb700 10 client.29743 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:24:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.112+0000 7f346dffb700 1 -- 172.21.15.44:0/4259749624 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56137425a840 con 0x56137430cdb0 2023-09-10T01:24:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.112+0000 7f346dffb700 10 client.29743 handle_command_reply: tid=0 2023-09-10T01:24:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.112+0000 7f3498fb7b80 1 client.29743 shutdown 2023-09-10T01:24:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.112+0000 7f3498fb7b80 1 -- 172.21.15.44:0/4259749624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f3300 msgr2=0x561374302d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.112+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/4259749624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f3300 0x561374302d90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f347c005950 tx=0x7f347c0a1170 comp rx=0 tx=0).stop 2023-09-10T01:24:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.112+0000 7f3498fb7b80 1 -- 172.21.15.44:0/4259749624 shutdown_connections 2023-09-10T01:24:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.112+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/4259749624 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56137430cdb0 0x56137430f1d0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.112+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/4259749624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f91a0 0x5613742fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.112+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/4259749624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f3300 0x561374302d90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.112+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/4259749624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f3cc0 0x5613743032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.112+0000 7f346dffb700 0 client.29743 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:24:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.112+0000 7f3498fb7b80 1 -- 172.21.15.44:0/4259749624 >> 172.21.15.44:0/4259749624 conn(0x561374268030 msgr2=0x5613742ed7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.113+0000 7f3498fb7b80 1 -- 172.21.15.44:0/4259749624 shutdown_connections 2023-09-10T01:24:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.113+0000 7f3498fb7b80 1 -- 172.21.15.44:0/4259749624 wait complete. 2023-09-10T01:24:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.113+0000 7f3498fb7b80 20 client.29743 trim_cache size 0 max 16384 2023-09-10T01:24:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.135+0000 7f3498fb7b80 1 Processor -- start 2023-09-10T01:24:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.135+0000 7f3498fb7b80 1 -- start start 2023-09-10T01:24:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3498fb7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f3300 0x5613742fc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3498fb7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f3cc0 0x5613742fce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3498fb7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f91a0 0x561374302920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3498fb7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613742fe610 con 0x5613742f3300 2023-09-10T01:24:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3498fb7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613742efd10 con 0x5613742f91a0 2023-09-10T01:24:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3498fb7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613742e5eb0 con 0x5613742f3cc0 2023-09-10T01:24:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3486d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f91a0 0x561374302920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3485d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f3cc0 0x5613742fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3486d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f91a0 0x561374302920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:32796/0 (socket says 172.21.15.44:32796) 2023-09-10T01:24:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3485d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f3cc0 0x5613742fce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37936/0 (socket says 172.21.15.44:37936) 2023-09-10T01:24:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3486d9d700 1 -- 172.21.15.44:0/3963987847 learned_addr learned my addr 172.21.15.44:0/3963987847 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3486d9d700 1 -- 172.21.15.44:0/3963987847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f3cc0 msgr2=0x5613742fce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3486d9d700 1 --2- 172.21.15.44:0/3963987847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f3cc0 0x5613742fce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3486d9d700 1 -- 172.21.15.44:0/3963987847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f3300 msgr2=0x5613742fc8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:24:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3486d9d700 1 --2- 172.21.15.44:0/3963987847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f3300 0x5613742fc8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3486d9d700 1 -- 172.21.15.44:0/3963987847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613740981f0 con 0x5613742f91a0 2023-09-10T01:24:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.136+0000 7f3485d9b700 1 --2- 172.21.15.44:0/3963987847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f3cc0 0x5613742fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:24:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.137+0000 7f3486d9d700 1 --2- 172.21.15.44:0/3963987847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f91a0 0x561374302920 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3480034730 tx=0x7f348000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f037f55192b96699 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.137+0000 7f34757fa700 1 -- 172.21.15.44:0/3963987847 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480010040 con 0x5613742f91a0 2023-09-10T01:24:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.137+0000 7f34757fa700 1 -- 172.21.15.44:0/3963987847 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3480003e90 con 0x5613742f91a0 2023-09-10T01:24:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.137+0000 7f34757fa700 1 -- 172.21.15.44:0/3963987847 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480004810 con 0x5613742f91a0 2023-09-10T01:24:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.138+0000 7f3498fb7b80 1 -- 172.21.15.44:0/3963987847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f91a0 msgr2=0x561374302920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.138+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/3963987847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f91a0 0x561374302920 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3480034730 tx=0x7f348000b040 comp rx=0 tx=0).stop 2023-09-10T01:24:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.138+0000 7f3498fb7b80 1 -- 172.21.15.44:0/3963987847 shutdown_connections 2023-09-10T01:24:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.138+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/3963987847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f91a0 0x561374302920 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.138+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/3963987847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f3300 0x5613742fc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.138+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/3963987847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5613742f3cc0 0x5613742fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.138+0000 7f3498fb7b80 1 -- 172.21.15.44:0/3963987847 >> 172.21.15.44:0/3963987847 conn(0x561374268030 msgr2=0x5613742f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.139+0000 7f3498fb7b80 1 -- 172.21.15.44:0/3963987847 shutdown_connections 2023-09-10T01:24:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.140+0000 7f3498fb7b80 1 -- 172.21.15.44:0/3963987847 wait complete. 2023-09-10T01:24:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.140+0000 7f3498fb7b80 1 -- start start 2023-09-10T01:24:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.140+0000 7f3498fb7b80 1 Processor -- start 2023-09-10T01:24:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3498fb7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f3cc0 0x56137435a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3498fb7b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f91a0 0x56137435a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3498fb7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561374354b70 0x561374354ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3498fb7b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561374355560 con 0x5613742f91a0 2023-09-10T01:24:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3498fb7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613743556d0 con 0x5613742f3cc0 2023-09-10T01:24:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3498fb7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561374355890 con 0x561374354b70 2023-09-10T01:24:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3498fb7b80 10 client.0 _ll_register_callbacks cb 0x56137412da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:24:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3486d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561374354b70 0x561374354ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3485d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f91a0 0x56137435a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3486d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561374354b70 0x561374354ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37940/0 (socket says 172.21.15.44:37940) 2023-09-10T01:24:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3486d9d700 1 -- 172.21.15.44:0/2094980348 learned_addr learned my addr 172.21.15.44:0/2094980348 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f348659c700 1 --2- 172.21.15.44:0/2094980348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f3cc0 0x56137435a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3486d9d700 1 -- 172.21.15.44:0/2094980348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f3cc0 msgr2=0x56137435a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3486d9d700 1 --2- 172.21.15.44:0/2094980348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f3cc0 0x56137435a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3486d9d700 1 -- 172.21.15.44:0/2094980348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f91a0 msgr2=0x56137435a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3486d9d700 1 --2- 172.21.15.44:0/2094980348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f91a0 0x56137435a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3486d9d700 1 -- 172.21.15.44:0/2094980348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348811cb00 con 0x561374354b70 2023-09-10T01:24:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f348659c700 1 --2- 172.21.15.44:0/2094980348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f3cc0 0x56137435a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:24:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.141+0000 7f3485d9b700 1 --2- 172.21.15.44:0/2094980348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f91a0 0x56137435a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:24:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.142+0000 7f3486d9d700 1 --2- 172.21.15.44:0/2094980348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561374354b70 0x561374354ff0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3480003c30 tx=0x7f3480003c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=630beb043ab6a8ff server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.142+0000 7f346dffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:24:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.142+0000 7f346dffb700 1 -- 172.21.15.44:0/2094980348 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480010030 con 0x561374354b70 2023-09-10T01:24:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.142+0000 7f346dffb700 1 -- 172.21.15.44:0/2094980348 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f34800095e0 con 0x561374354b70 2023-09-10T01:24:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.142+0000 7f3498fb7b80 1 -- 172.21.15.44:0/2094980348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f347c099100 con 0x561374354b70 2023-09-10T01:24:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.142+0000 7f346dffb700 1 -- 172.21.15.44:0/2094980348 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3480004410 con 0x561374354b70 2023-09-10T01:24:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.142+0000 7f3498fb7b80 10 client.29749 fetch_fsmap learned FSMap version 148 2023-09-10T01:24:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.142+0000 7f3498fb7b80 1 -- 172.21.15.44:0/2094980348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=148,monmap=4+}) v3 -- 0x7f34800343e0 con 0x561374354b70 2023-09-10T01:24:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.142+0000 7f346dffb700 1 -- 172.21.15.44:0/2094980348 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f347c099100 con 0x561374354b70 2023-09-10T01:24:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.143+0000 7f346dffb700 1 -- 172.21.15.44:0/2094980348 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 148) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f34800079f0 con 0x561374354b70 2023-09-10T01:24:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.143+0000 7f3498fb7b80 10 client.29749 fetch_fsmap finished waiting for FSMap version 148 2023-09-10T01:24:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.143+0000 7f3498fb7b80 10 client.29749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:24:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.143+0000 7f3498fb7b80 20 client.29749 populate_metadata read hostname 'smithi044' 2023-09-10T01:24:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.143+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/2094980348 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561374362640 0x561374364a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.143+0000 7f3498fb7b80 4 client.29749 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:24:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.143+0000 7f3498fb7b80 1 -- 172.21.15.44:0/2094980348 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56137423e8c0 con 0x561374362640 2023-09-10T01:24:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.143+0000 7f348659c700 1 --2- 172.21.15.44:0/2094980348 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561374362640 0x561374364a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.143+0000 7f346dffb700 10 client.29749 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:24:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.143+0000 7f348659c700 1 --2- 172.21.15.44:0/2094980348 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561374362640 0x561374364a60 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4db52623a8e28711 server_cookie=4985002e4f8ae94e in_seq=0 out_seq=0 2023-09-10T01:24:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.171+0000 7f346dffb700 1 -- 172.21.15.44:0/2094980348 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56137423e8c0 con 0x561374362640 2023-09-10T01:24:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.171+0000 7f346dffb700 10 client.29749 handle_command_reply: tid=0 2023-09-10T01:24:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.171+0000 7f3498fb7b80 1 client.29749 shutdown 2023-09-10T01:24:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.171+0000 7f3498fb7b80 1 -- 172.21.15.44:0/2094980348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561374354b70 msgr2=0x561374354ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.171+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/2094980348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561374354b70 0x561374354ff0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3480003c30 tx=0x7f3480003c60 comp rx=0 tx=0).stop 2023-09-10T01:24:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.171+0000 7f3498fb7b80 1 -- 172.21.15.44:0/2094980348 shutdown_connections 2023-09-10T01:24:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.171+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/2094980348 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x561374362640 0x561374364a60 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.171+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/2094980348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561374354b70 0x561374354ff0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.171+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/2094980348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5613742f3cc0 0x56137435a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.171+0000 7f346dffb700 0 client.29749 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:24:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.171+0000 7f3498fb7b80 1 --2- 172.21.15.44:0/2094980348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613742f91a0 0x56137435a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.171+0000 7f3498fb7b80 1 -- 172.21.15.44:0/2094980348 >> 172.21.15.44:0/2094980348 conn(0x561374268030 msgr2=0x5613742ecf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.172+0000 7f3498fb7b80 1 -- 172.21.15.44:0/2094980348 shutdown_connections 2023-09-10T01:24:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:24:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:24:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:24:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:24:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.180+0000 7f3498fb7b80 1 -- 172.21.15.44:0/2094980348 wait complete. 2023-09-10T01:24:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.181+0000 7f3498fb7b80 20 client.29749 trim_cache size 0 max 16384 2023-09-10T01:24:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 210.0 s, 335.4 tps, lat 294.528 ms stddev 318.053 2023-09-10T01:24:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.193+0000 7f348cdf1700 1 -- 172.21.15.44:0/3998450343 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f34700a0830 msgr2=0x7f34700a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.193+0000 7f348cdf1700 1 --2- 172.21.15.44:0/3998450343 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f34700a0830 0x7f34700a2cf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x5613742ee060 tx=0x7f3478009f50 comp rx=0 tx=0).stop 2023-09-10T01:24:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.193+0000 7f348cdf1700 1 -- 172.21.15.44:0/3998450343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348810ed50 msgr2=0x7f3488117100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.193+0000 7f348cdf1700 1 --2- 172.21.15.44:0/3998450343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348810ed50 0x7f3488117100 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f347c00d950 tx=0x7f347c00dd10 comp rx=0 tx=0).stop 2023-09-10T01:24:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.194+0000 7f348cdf1700 1 -- 172.21.15.44:0/3998450343 shutdown_connections 2023-09-10T01:24:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.194+0000 7f348cdf1700 1 --2- 172.21.15.44:0/3998450343 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f34700a0830 0x7f34700a2cf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.194+0000 7f348cdf1700 1 --2- 172.21.15.44:0/3998450343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3488119de0 0x7f348810ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.194+0000 7f348cdf1700 1 --2- 172.21.15.44:0/3998450343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348810ed50 0x7f3488117100 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.194+0000 7f348cdf1700 1 --2- 172.21.15.44:0/3998450343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f348810f1c0 0x7f3488117680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.194+0000 7f348cdf1700 1 -- 172.21.15.44:0/3998450343 >> 172.21.15.44:0/3998450343 conn(0x7f348806c6c0 msgr2=0x7f348810c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.199+0000 7f348cdf1700 1 -- 172.21.15.44:0/3998450343 shutdown_connections 2023-09-10T01:24:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:36.199+0000 7f348cdf1700 1 -- 172.21.15.44:0/3998450343 wait complete. 2023-09-10T01:24:36.556 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:36 smithi131 ceph-mon[103713]: pgmap v1422: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 5.6 MiB/s wr, 498 op/s 2023-09-10T01:24:36.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:36 smithi044 ceph-mon[98676]: pgmap v1422: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 5.6 MiB/s wr, 498 op/s 2023-09-10T01:24:36.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:36 smithi099 ceph-mon[102692]: pgmap v1422: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 5.6 MiB/s wr, 498 op/s 2023-09-10T01:24:38.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:38 smithi131 ceph-mon[103713]: pgmap v1423: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.4 MiB/s wr, 803 op/s 2023-09-10T01:24:38.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:38 smithi044 ceph-mon[98676]: pgmap v1423: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.4 MiB/s wr, 803 op/s 2023-09-10T01:24:38.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:38 smithi099 ceph-mon[102692]: pgmap v1423: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.4 MiB/s wr, 803 op/s 2023-09-10T01:24:40.562 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:40 smithi044 ceph-mon[98676]: pgmap v1424: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.7 MiB/s wr, 585 op/s 2023-09-10T01:24:40.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:40 smithi099 ceph-mon[102692]: pgmap v1424: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.7 MiB/s wr, 585 op/s 2023-09-10T01:24:40.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:40 smithi131 ceph-mon[103713]: pgmap v1424: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.7 MiB/s wr, 585 op/s 2023-09-10T01:24:41.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 215.0 s, 340.6 tps, lat 267.152 ms stddev 355.196 2023-09-10T01:24:42.558 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:42 smithi044 ceph-mon[98676]: pgmap v1425: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.7 MiB/s wr, 829 op/s 2023-09-10T01:24:42.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:42 smithi099 ceph-mon[102692]: pgmap v1425: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.7 MiB/s wr, 829 op/s 2023-09-10T01:24:42.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:42 smithi131 ceph-mon[103713]: pgmap v1425: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.7 MiB/s wr, 829 op/s 2023-09-10T01:24:43.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:43 smithi131 ceph-mon[103713]: pgmap v1426: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.2 MiB/s wr, 760 op/s 2023-09-10T01:24:43.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:43 smithi044 ceph-mon[98676]: pgmap v1426: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.2 MiB/s wr, 760 op/s 2023-09-10T01:24:43.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:43 smithi099 ceph-mon[102692]: pgmap v1426: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.2 MiB/s wr, 760 op/s 2023-09-10T01:24:46.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:45 smithi131 ceph-mon[103713]: pgmap v1427: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.1 MiB/s wr, 686 op/s 2023-09-10T01:24:46.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:45 smithi044 ceph-mon[98676]: pgmap v1427: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.1 MiB/s wr, 686 op/s 2023-09-10T01:24:46.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:45 smithi099 ceph-mon[102692]: pgmap v1427: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.1 MiB/s wr, 686 op/s 2023-09-10T01:24:46.183 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 220.0 s, 285.7 tps, lat 378.197 ms stddev 332.227 2023-09-10T01:24:48.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:48 smithi131 ceph-mon[103713]: pgmap v1428: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 976 op/s 2023-09-10T01:24:48.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:48 smithi044 ceph-mon[98676]: pgmap v1428: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 976 op/s 2023-09-10T01:24:48.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:48 smithi099 ceph-mon[102692]: pgmap v1428: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 976 op/s 2023-09-10T01:24:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.406+0000 7f24e1e36700 1 -- 172.21.15.44:0/2885005906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc100f50 msgr2=0x7f24dc10f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.406+0000 7f24e1e36700 1 --2- 172.21.15.44:0/2885005906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc100f50 0x7f24dc10f810 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009990 tx=0x7f24cc009ca0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.406+0000 7f24e1e36700 1 -- 172.21.15.44:0/2885005906 shutdown_connections 2023-09-10T01:24:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.406+0000 7f24e1e36700 1 --2- 172.21.15.44:0/2885005906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc100f50 0x7f24dc10f810 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.406+0000 7f24e1e36700 1 --2- 172.21.15.44:0/2885005906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc106650 0x7f24dc106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.406+0000 7f24e1e36700 1 --2- 172.21.15.44:0/2885005906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24dc100590 0x7f24dc100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.406+0000 7f24e1e36700 1 -- 172.21.15.44:0/2885005906 >> 172.21.15.44:0/2885005906 conn(0x7f24dc0fc060 msgr2=0x7f24dc0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.417+0000 7f24e1e36700 1 -- 172.21.15.44:0/2885005906 shutdown_connections 2023-09-10T01:24:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.420+0000 7f24e1e36700 1 -- 172.21.15.44:0/2885005906 wait complete. 2023-09-10T01:24:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.421+0000 7f24e1e36700 1 Processor -- start 2023-09-10T01:24:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.421+0000 7f24e1e36700 1 -- start start 2023-09-10T01:24:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.421+0000 7f24e1e36700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc100590 0x7f24dc06a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.421+0000 7f24e1e36700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc100f50 0x7f24dc06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.421+0000 7f24e1e36700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24dc06e600 0x7f24dc06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.421+0000 7f24e1e36700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc06eff0 con 0x7f24dc100590 2023-09-10T01:24:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.421+0000 7f24e1e36700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc06f160 con 0x7f24dc06e600 2023-09-10T01:24:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.421+0000 7f24e1e36700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc06f320 con 0x7f24dc100f50 2023-09-10T01:24:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.421+0000 7f24daffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc100f50 0x7f24dc06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.421+0000 7f24daffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc100f50 0x7f24dc06a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37954/0 (socket says 172.21.15.44:37954) 2023-09-10T01:24:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.421+0000 7f24daffd700 1 -- 172.21.15.44:0/560082934 learned_addr learned my addr 172.21.15.44:0/560082934 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.422+0000 7f24daffd700 1 -- 172.21.15.44:0/560082934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24dc06e600 msgr2=0x7f24dc06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.422+0000 7f24daffd700 1 --2- 172.21.15.44:0/560082934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24dc06e600 0x7f24dc06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.422+0000 7f24daffd700 1 -- 172.21.15.44:0/560082934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc100590 msgr2=0x7f24dc06a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.422+0000 7f24daffd700 1 --2- 172.21.15.44:0/560082934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc100590 0x7f24dc06a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.422+0000 7f24daffd700 1 -- 172.21.15.44:0/560082934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc0095f0 con 0x7f24dc100f50 2023-09-10T01:24:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.422+0000 7f24daffd700 1 --2- 172.21.15.44:0/560082934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc100f50 0x7f24dc06a740 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f24d000e9a0 tx=0x7f24d000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.422+0000 7f24d8ff9700 1 -- 172.21.15.44:0/560082934 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0009790 con 0x7f24dc100f50 2023-09-10T01:24:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.422+0000 7f24e1e36700 1 -- 172.21.15.44:0/560082934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24dc1a13d0 con 0x7f24dc100f50 2023-09-10T01:24:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.422+0000 7f24d8ff9700 1 -- 172.21.15.44:0/560082934 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f24d000cbd0 con 0x7f24dc100f50 2023-09-10T01:24:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.422+0000 7f24d8ff9700 1 -- 172.21.15.44:0/560082934 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0018680 con 0x7f24dc100f50 2023-09-10T01:24:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.422+0000 7f24e1e36700 1 -- 172.21.15.44:0/560082934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24dc1a1930 con 0x7f24dc100f50 2023-09-10T01:24:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.427+0000 7f24edffcb80 1 Processor -- start 2023-09-10T01:24:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.428+0000 7f24edffcb80 1 -- start start 2023-09-10T01:24:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.428+0000 7f24edffcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7505621a0 0x55d750562580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.428+0000 7f24edffcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055c300 0x55d75055c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.428+0000 7f24edffcb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d75055ccc0 0x55d750568f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.428+0000 7f24edffcb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7504b56d0 con 0x55d75055ccc0 2023-09-10T01:24:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.428+0000 7f24edffcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7504b2ff0 con 0x55d7505621a0 2023-09-10T01:24:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.428+0000 7f24edffcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7504ae8e0 con 0x55d75055c300 2023-09-10T01:24:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.430+0000 7f24d8ff9700 1 -- 172.21.15.44:0/560082934 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f24d0010430 con 0x7f24dc100f50 2023-09-10T01:24:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.431+0000 7f24d8ff9700 1 --2- 172.21.15.44:0/560082934 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f24c40a08e0 0x7f24c40a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.431+0000 7f24d8ff9700 1 -- 172.21.15.44:0/560082934 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f24d0014070 con 0x7f24dc100f50 2023-09-10T01:24:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.432+0000 7f24db7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055c300 0x55d75055c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.432+0000 7f24db7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055c300 0x55d75055c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37958/0 (socket says 172.21.15.44:37958) 2023-09-10T01:24:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.432+0000 7f24db7fe700 1 -- 172.21.15.44:0/1793877469 learned_addr learned my addr 172.21.15.44:0/1793877469 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.432+0000 7f24db7fe700 1 -- 172.21.15.44:0/1793877469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7505621a0 msgr2=0x55d750562580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.432+0000 7f24db7fe700 1 --2- 172.21.15.44:0/1793877469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7505621a0 0x55d750562580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.432+0000 7f24db7fe700 1 -- 172.21.15.44:0/1793877469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d75055ccc0 msgr2=0x55d750568f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.432+0000 7f24db7fe700 1 --2- 172.21.15.44:0/1793877469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d75055ccc0 0x55d750568f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.433+0000 7f24db7fe700 1 -- 172.21.15.44:0/1793877469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7502dad80 con 0x55d75055c300 2023-09-10T01:24:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.433+0000 7f24db7fe700 1 --2- 172.21.15.44:0/1793877469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055c300 0x55d75055c780 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009da0 tx=0x7f24d400b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=635a610577c78a3f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.434+0000 7f24c1ffb700 1 -- 172.21.15.44:0/1793877469 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4013410 con 0x55d75055c300 2023-09-10T01:24:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.434+0000 7f24c1ffb700 1 -- 172.21.15.44:0/1793877469 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f24d4013a00 con 0x55d75055c300 2023-09-10T01:24:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.434+0000 7f24c1ffb700 1 -- 172.21.15.44:0/1793877469 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4012950 con 0x55d75055c300 2023-09-10T01:24:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.435+0000 7f24edffcb80 1 -- 172.21.15.44:0/1793877469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055c300 msgr2=0x55d75055c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.435+0000 7f24edffcb80 1 --2- 172.21.15.44:0/1793877469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055c300 0x55d75055c780 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009da0 tx=0x7f24d400b730 comp rx=0 tx=0).stop 2023-09-10T01:24:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.436+0000 7f24edffcb80 1 -- 172.21.15.44:0/1793877469 shutdown_connections 2023-09-10T01:24:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.436+0000 7f24edffcb80 1 --2- 172.21.15.44:0/1793877469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d75055ccc0 0x55d750568f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.436+0000 7f24edffcb80 1 --2- 172.21.15.44:0/1793877469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7505621a0 0x55d750562580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.436+0000 7f24edffcb80 1 --2- 172.21.15.44:0/1793877469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055c300 0x55d75055c780 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.436+0000 7f24edffcb80 1 -- 172.21.15.44:0/1793877469 >> 172.21.15.44:0/1793877469 conn(0x55d7504d1030 msgr2=0x55d750557560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.445+0000 7f24dbfff700 1 --2- 172.21.15.44:0/560082934 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f24c40a08e0 0x7f24c40a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.446+0000 7f24dbfff700 1 --2- 172.21.15.44:0/560082934 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f24c40a08e0 0x7f24c40a2da0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00b2e0 tx=0x7f24cc005fb0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.448+0000 7f24edffcb80 1 -- 172.21.15.44:0/1793877469 shutdown_connections 2023-09-10T01:24:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.452+0000 7f24edffcb80 1 -- 172.21.15.44:0/1793877469 wait complete. 2023-09-10T01:24:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.452+0000 7f24edffcb80 1 -- start start 2023-09-10T01:24:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.453+0000 7f24edffcb80 1 Processor -- start 2023-09-10T01:24:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.454+0000 7f24edffcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d75055c300 0x55d750568de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.454+0000 7f24edffcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055ccc0 0x55d750566060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.454+0000 7f24edffcb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7505621a0 0x55d750566690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.454+0000 7f24edffcb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d75044cb10 con 0x55d7505621a0 2023-09-10T01:24:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.454+0000 7f24edffcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d75044c810 con 0x55d75055c300 2023-09-10T01:24:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.454+0000 7f24edffcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7503308c0 con 0x55d75055ccc0 2023-09-10T01:24:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.454+0000 7f24edffcb80 10 client.0 _ll_register_callbacks cb 0x55d750396a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:24:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.455+0000 7f24dbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7505621a0 0x55d750566690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.455+0000 7f24dbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7505621a0 0x55d750566690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59318/0 (socket says 172.21.15.44:59318) 2023-09-10T01:24:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.455+0000 7f24dbfff700 1 -- 172.21.15.44:0/780999477 learned_addr learned my addr 172.21.15.44:0/780999477 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.457+0000 7f24dbfff700 1 -- 172.21.15.44:0/780999477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055ccc0 msgr2=0x55d750566060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:24:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.457+0000 7f24dbfff700 1 --2- 172.21.15.44:0/780999477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055ccc0 0x55d750566060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.457+0000 7f24dbfff700 1 -- 172.21.15.44:0/780999477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d75055c300 msgr2=0x55d750568de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:24:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.457+0000 7f24dbfff700 1 --2- 172.21.15.44:0/780999477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d75055c300 0x55d750568de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.458+0000 7f24dbfff700 1 -- 172.21.15.44:0/780999477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d40099d0 con 0x55d7505621a0 2023-09-10T01:24:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.458+0000 7f24dbfff700 1 --2- 172.21.15.44:0/780999477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7505621a0 0x55d750566690 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00f6e0 tx=0x7f24cc00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd5cc015b35957e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.458+0000 7f24bf7f6700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:24:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.458+0000 7f24bf7f6700 1 -- 172.21.15.44:0/780999477 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc02a070 con 0x55d7505621a0 2023-09-10T01:24:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.458+0000 7f24bf7f6700 1 -- 172.21.15.44:0/780999477 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f24cc018610 con 0x55d7505621a0 2023-09-10T01:24:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.458+0000 7f24edffcb80 1 -- 172.21.15.44:0/780999477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d750406490 con 0x55d7505621a0 2023-09-10T01:24:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.458+0000 7f24bf7f6700 1 -- 172.21.15.44:0/780999477 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24cc01e070 con 0x55d7505621a0 2023-09-10T01:24:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.458+0000 7f24edffcb80 10 client.20121 fetch_fsmap learned FSMap version 148 2023-09-10T01:24:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.458+0000 7f24edffcb80 1 -- 172.21.15.44:0/780999477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=148,monmap=4+}) v3 -- 0x55d7502c3c00 con 0x55d7505621a0 2023-09-10T01:24:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.459+0000 7f24bf7f6700 1 -- 172.21.15.44:0/780999477 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f24cc038020 con 0x55d7505621a0 2023-09-10T01:24:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.459+0000 7f24bf7f6700 1 -- 172.21.15.44:0/780999477 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 148) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f24cc02f640 con 0x55d7505621a0 2023-09-10T01:24:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.459+0000 7f24edffcb80 10 client.20121 fetch_fsmap finished waiting for FSMap version 148 2023-09-10T01:24:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.459+0000 7f24edffcb80 10 client.20121 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:24:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.459+0000 7f24edffcb80 20 client.20121 populate_metadata read hostname 'smithi044' 2023-09-10T01:24:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.459+0000 7f24edffcb80 1 --2- 172.21.15.44:0/780999477 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d75056c5e0 0x55d75056eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.459+0000 7f24edffcb80 4 client.20121 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:24:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.459+0000 7f24edffcb80 1 -- 172.21.15.44:0/780999477 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7504c3840 con 0x55d75056c5e0 2023-09-10T01:24:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.461+0000 7f24db7fe700 1 --2- 172.21.15.44:0/780999477 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d75056c5e0 0x55d75056eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.461+0000 7f24db7fe700 1 --2- 172.21.15.44:0/780999477 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d75056c5e0 0x55d75056eaa0 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d897f9e860d3a55e server_cookie=7baa841c00e7103 in_seq=0 out_seq=0 2023-09-10T01:24:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.461+0000 7f24bf7f6700 10 client.20121 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:24:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.463+0000 7f24bf7f6700 1 -- 172.21.15.44:0/780999477 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d7504c3840 con 0x55d75056c5e0 2023-09-10T01:24:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.463+0000 7f24bf7f6700 10 client.20121 handle_command_reply: tid=0 2023-09-10T01:24:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.463+0000 7f24edffcb80 1 client.20121 shutdown 2023-09-10T01:24:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.464+0000 7f24edffcb80 1 -- 172.21.15.44:0/780999477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7505621a0 msgr2=0x55d750566690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.464+0000 7f24edffcb80 1 --2- 172.21.15.44:0/780999477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7505621a0 0x55d750566690 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00f6e0 tx=0x7f24cc00b5b0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.464+0000 7f24edffcb80 1 -- 172.21.15.44:0/780999477 shutdown_connections 2023-09-10T01:24:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.464+0000 7f24edffcb80 1 --2- 172.21.15.44:0/780999477 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d75056c5e0 0x55d75056eaa0 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.464+0000 7f24edffcb80 1 --2- 172.21.15.44:0/780999477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7505621a0 0x55d750566690 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.464+0000 7f24bf7f6700 0 client.20121 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:24:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.464+0000 7f24edffcb80 1 --2- 172.21.15.44:0/780999477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d75055c300 0x55d750568de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.464+0000 7f24edffcb80 1 --2- 172.21.15.44:0/780999477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055ccc0 0x55d750566060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.464+0000 7f24edffcb80 1 -- 172.21.15.44:0/780999477 >> 172.21.15.44:0/780999477 conn(0x55d7504d1030 msgr2=0x55d7505567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.464+0000 7f24edffcb80 1 -- 172.21.15.44:0/780999477 shutdown_connections 2023-09-10T01:24:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.464+0000 7f24edffcb80 1 -- 172.21.15.44:0/780999477 wait complete. 2023-09-10T01:24:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.466+0000 7f24edffcb80 20 client.20121 trim_cache size 0 max 16384 2023-09-10T01:24:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.497+0000 7f24edffcb80 1 Processor -- start 2023-09-10T01:24:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.497+0000 7f24edffcb80 1 -- start start 2023-09-10T01:24:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.497+0000 7f24edffcb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d75055c300 0x55d75057e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.497+0000 7f24edffcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d75055ccc0 0x55d750565880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.497+0000 7f24edffcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7505621a0 0x55d750568980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.497+0000 7f24edffcb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d750558320 con 0x55d75055c300 2023-09-10T01:24:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.497+0000 7f24edffcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d75057eab0 con 0x55d75055ccc0 2023-09-10T01:24:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.497+0000 7f24edffcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d750564620 con 0x55d7505621a0 2023-09-10T01:24:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.498+0000 7f24db7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d75055c300 0x55d75057e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.498+0000 7f24db7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d75055c300 0x55d75057e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59322/0 (socket says 172.21.15.44:59322) 2023-09-10T01:24:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.498+0000 7f24daffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d75055ccc0 0x55d750565880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.498+0000 7f24db7fe700 1 -- 172.21.15.44:0/4121247706 learned_addr learned my addr 172.21.15.44:0/4121247706 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.498+0000 7f24dbfff700 1 --2- 172.21.15.44:0/4121247706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7505621a0 0x55d750568980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.498+0000 7f24db7fe700 1 -- 172.21.15.44:0/4121247706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7505621a0 msgr2=0x55d750568980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.498+0000 7f24db7fe700 1 --2- 172.21.15.44:0/4121247706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7505621a0 0x55d750568980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.498+0000 7f24db7fe700 1 -- 172.21.15.44:0/4121247706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d75055ccc0 msgr2=0x55d750565880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.498+0000 7f24db7fe700 1 --2- 172.21.15.44:0/4121247706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d75055ccc0 0x55d750565880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.498+0000 7f24db7fe700 1 -- 172.21.15.44:0/4121247706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7503011f0 con 0x55d75055c300 2023-09-10T01:24:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.498+0000 7f24daffd700 1 --2- 172.21.15.44:0/4121247706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d75055ccc0 0x55d750565880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:24:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.498+0000 7f24dbfff700 1 --2- 172.21.15.44:0/4121247706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7505621a0 0x55d750568980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:24:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.498+0000 7f24db7fe700 1 --2- 172.21.15.44:0/4121247706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d75055c300 0x55d75057e570 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f24d4012520 tx=0x7f24d400d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c107fde2e3440ab server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.499+0000 7f24c27fc700 1 -- 172.21.15.44:0/4121247706 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4020ac0 con 0x55d75055c300 2023-09-10T01:24:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.499+0000 7f24c27fc700 1 -- 172.21.15.44:0/4121247706 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f24d4024070 con 0x55d75055c300 2023-09-10T01:24:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.499+0000 7f24c27fc700 1 -- 172.21.15.44:0/4121247706 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4013790 con 0x55d75055c300 2023-09-10T01:24:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.500+0000 7f24edffcb80 1 -- 172.21.15.44:0/4121247706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d75055c300 msgr2=0x55d75057e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.500+0000 7f24edffcb80 1 --2- 172.21.15.44:0/4121247706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d75055c300 0x55d75057e570 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f24d4012520 tx=0x7f24d400d040 comp rx=0 tx=0).stop 2023-09-10T01:24:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.504+0000 7f24edffcb80 1 -- 172.21.15.44:0/4121247706 shutdown_connections 2023-09-10T01:24:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.504+0000 7f24edffcb80 1 --2- 172.21.15.44:0/4121247706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7505621a0 0x55d750568980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.504+0000 7f24edffcb80 1 --2- 172.21.15.44:0/4121247706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d75055c300 0x55d75057e570 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.504+0000 7f24edffcb80 1 --2- 172.21.15.44:0/4121247706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d75055ccc0 0x55d750565880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.504+0000 7f24edffcb80 1 -- 172.21.15.44:0/4121247706 >> 172.21.15.44:0/4121247706 conn(0x55d7504d1030 msgr2=0x55d750560150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.504+0000 7f24edffcb80 1 -- 172.21.15.44:0/4121247706 shutdown_connections 2023-09-10T01:24:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.504+0000 7f24edffcb80 1 -- 172.21.15.44:0/4121247706 wait complete. 2023-09-10T01:24:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.504+0000 7f24edffcb80 1 -- start start 2023-09-10T01:24:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.504+0000 7f24edffcb80 1 Processor -- start 2023-09-10T01:24:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.505+0000 7f24edffcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d75055c300 0x55d7505a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.505+0000 7f24edffcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055ccc0 0x55d7505a4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.505+0000 7f24edffcb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7505621a0 0x55d7505a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.505+0000 7f24edffcb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7505a51a0 con 0x55d7505621a0 2023-09-10T01:24:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.505+0000 7f24edffcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7505a52e0 con 0x55d75055c300 2023-09-10T01:24:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.505+0000 7f24edffcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7505a54a0 con 0x55d75055ccc0 2023-09-10T01:24:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.505+0000 7f24edffcb80 10 client.0 _ll_register_callbacks cb 0x7f24cc00bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:24:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.505+0000 7f24daffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055ccc0 0x55d7505a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.505+0000 7f24daffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055ccc0 0x55d7505a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:37978/0 (socket says 172.21.15.44:37978) 2023-09-10T01:24:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.505+0000 7f24daffd700 1 -- 172.21.15.44:0/1487838941 learned_addr learned my addr 172.21.15.44:0/1487838941 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:24:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.505+0000 7f24dbfff700 1 --2- 172.21.15.44:0/1487838941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7505621a0 0x55d7505a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.505+0000 7f24daffd700 1 -- 172.21.15.44:0/1487838941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d75055c300 msgr2=0x55d7505a5dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:24:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.505+0000 7f24daffd700 1 --2- 172.21.15.44:0/1487838941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d75055c300 0x55d7505a5dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.505+0000 7f24daffd700 1 -- 172.21.15.44:0/1487838941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7505621a0 msgr2=0x55d7505a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:24:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.505+0000 7f24daffd700 1 --2- 172.21.15.44:0/1487838941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7505621a0 0x55d7505a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.505+0000 7f24daffd700 1 -- 172.21.15.44:0/1487838941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24dc1a1930 con 0x55d75055ccc0 2023-09-10T01:24:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.506+0000 7f24daffd700 1 --2- 172.21.15.44:0/1487838941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055ccc0 0x55d7505a4640 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f24d000f010 tx=0x7f24d00c9e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e4e10e87539a682 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:24:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.506+0000 7f24bf7f6700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:24:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.506+0000 7f24bf7f6700 1 -- 172.21.15.44:0/1487838941 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0099600 con 0x55d75055ccc0 2023-09-10T01:24:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.506+0000 7f24bf7f6700 1 -- 172.21.15.44:0/1487838941 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f24d009d430 con 0x55d75055ccc0 2023-09-10T01:24:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.506+0000 7f24edffcb80 1 -- 172.21.15.44:0/1487838941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24d4013a00 con 0x55d75055ccc0 2023-09-10T01:24:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.506+0000 7f24bf7f6700 1 -- 172.21.15.44:0/1487838941 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24d00a1670 con 0x55d75055ccc0 2023-09-10T01:24:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.506+0000 7f24edffcb80 10 client.29761 fetch_fsmap learned FSMap version 148 2023-09-10T01:24:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.506+0000 7f24edffcb80 1 -- 172.21.15.44:0/1487838941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=148,monmap=4+}) v3 -- 0x55d7502dad80 con 0x55d75055ccc0 2023-09-10T01:24:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.506+0000 7f24bf7f6700 1 -- 172.21.15.44:0/1487838941 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f24d4013a00 con 0x55d75055ccc0 2023-09-10T01:24:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.507+0000 7f24bf7f6700 1 -- 172.21.15.44:0/1487838941 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 148) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f24d0099860 con 0x55d75055ccc0 2023-09-10T01:24:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.507+0000 7f24edffcb80 10 client.29761 fetch_fsmap finished waiting for FSMap version 148 2023-09-10T01:24:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.507+0000 7f24edffcb80 10 client.29761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:24:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.507+0000 7f24edffcb80 20 client.29761 populate_metadata read hostname 'smithi044' 2023-09-10T01:24:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.507+0000 7f24edffcb80 1 --2- 172.21.15.44:0/1487838941 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d75060fd40 0x55d750612160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:24:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.507+0000 7f24edffcb80 4 client.29761 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:24:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.507+0000 7f24edffcb80 1 -- 172.21.15.44:0/1487838941 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d7504a78c0 con 0x55d75060fd40 2023-09-10T01:24:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.510+0000 7f24db7fe700 1 --2- 172.21.15.44:0/1487838941 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d75060fd40 0x55d750612160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:24:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.511+0000 7f24bf7f6700 10 client.29761 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:24:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.511+0000 7f24db7fe700 1 --2- 172.21.15.44:0/1487838941 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d75060fd40 0x55d750612160 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=599ad30320bd2c06 server_cookie=434a0d6dd84835c5 in_seq=0 out_seq=0 2023-09-10T01:24:49.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.573+0000 7f24bf7f6700 1 -- 172.21.15.44:0/1487838941 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d7504a78c0 con 0x55d75060fd40 2023-09-10T01:24:49.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.573+0000 7f24bf7f6700 10 client.29761 handle_command_reply: tid=0 2023-09-10T01:24:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.578+0000 7f24edffcb80 1 client.29761 shutdown 2023-09-10T01:24:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.581+0000 7f24edffcb80 1 -- 172.21.15.44:0/1487838941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055ccc0 msgr2=0x55d7505a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.581+0000 7f24edffcb80 1 --2- 172.21.15.44:0/1487838941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055ccc0 0x55d7505a4640 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f24d000f010 tx=0x7f24d00c9e00 comp rx=0 tx=0).stop 2023-09-10T01:24:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.585+0000 7f24edffcb80 1 -- 172.21.15.44:0/1487838941 shutdown_connections 2023-09-10T01:24:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.585+0000 7f24edffcb80 1 --2- 172.21.15.44:0/1487838941 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d75060fd40 0x55d750612160 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.585+0000 7f24edffcb80 1 --2- 172.21.15.44:0/1487838941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7505621a0 0x55d7505a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.585+0000 7f24edffcb80 1 --2- 172.21.15.44:0/1487838941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d75055c300 0x55d7505a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.585+0000 7f24edffcb80 1 --2- 172.21.15.44:0/1487838941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75055ccc0 0x55d7505a4640 secure :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f24d000f010 tx=0x7f24d00c9e00 comp rx=0 tx=0).stop 2023-09-10T01:24:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.585+0000 7f24edffcb80 1 -- 172.21.15.44:0/1487838941 >> 172.21.15.44:0/1487838941 conn(0x55d7504d1030 msgr2=0x55d750557090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.589+0000 7f24bf7f6700 0 client.29761 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:24:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.593+0000 7f24edffcb80 1 -- 172.21.15.44:0/1487838941 shutdown_connections 2023-09-10T01:24:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.597+0000 7f24edffcb80 1 -- 172.21.15.44:0/1487838941 wait complete. 2023-09-10T01:24:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:24:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:24:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:24:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:24:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.598+0000 7f24edffcb80 20 client.29761 trim_cache size 0 max 16384 2023-09-10T01:24:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.603+0000 7f24e1e36700 1 -- 172.21.15.44:0/560082934 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f24c40a08e0 msgr2=0x7f24c40a2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.603+0000 7f24e1e36700 1 --2- 172.21.15.44:0/560082934 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f24c40a08e0 0x7f24c40a2da0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00b2e0 tx=0x7f24cc005fb0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.603+0000 7f24e1e36700 1 -- 172.21.15.44:0/560082934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc100f50 msgr2=0x7f24dc06a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:24:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.603+0000 7f24e1e36700 1 --2- 172.21.15.44:0/560082934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc100f50 0x7f24dc06a740 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f24d000e9a0 tx=0x7f24d000ed60 comp rx=0 tx=0).stop 2023-09-10T01:24:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.603+0000 7f24e1e36700 1 -- 172.21.15.44:0/560082934 shutdown_connections 2023-09-10T01:24:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.603+0000 7f24e1e36700 1 --2- 172.21.15.44:0/560082934 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f24c40a08e0 0x7f24c40a2da0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.603+0000 7f24e1e36700 1 --2- 172.21.15.44:0/560082934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24dc06e600 0x7f24dc06eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.603+0000 7f24e1e36700 1 --2- 172.21.15.44:0/560082934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24dc100590 0x7f24dc06a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.603+0000 7f24e1e36700 1 --2- 172.21.15.44:0/560082934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc100f50 0x7f24dc06a740 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:24:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.603+0000 7f24e1e36700 1 -- 172.21.15.44:0/560082934 >> 172.21.15.44:0/560082934 conn(0x7f24dc0fc060 msgr2=0x7f24dc073fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:24:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.603+0000 7f24e1e36700 1 -- 172.21.15.44:0/560082934 shutdown_connections 2023-09-10T01:24:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:24:49.603+0000 7f24e1e36700 1 -- 172.21.15.44:0/560082934 wait complete. 2023-09-10T01:24:50.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:50 smithi131 ceph-mon[103713]: pgmap v1429: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.9 MiB/s wr, 706 op/s 2023-09-10T01:24:50.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:50 smithi044 ceph-mon[98676]: pgmap v1429: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.9 MiB/s wr, 706 op/s 2023-09-10T01:24:50.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:50 smithi099 ceph-mon[102692]: pgmap v1429: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.9 MiB/s wr, 706 op/s 2023-09-10T01:24:51.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 225.0 s, 280.7 tps, lat 362.033 ms stddev 443.424 2023-09-10T01:24:52.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:52 smithi131 ceph-mon[103713]: pgmap v1430: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 916 op/s 2023-09-10T01:24:52.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:52 smithi044 ceph-mon[98676]: pgmap v1430: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 916 op/s 2023-09-10T01:24:52.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:52 smithi099 ceph-mon[102692]: pgmap v1430: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 916 op/s 2023-09-10T01:24:54.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:54 smithi044 ceph-mon[98676]: pgmap v1431: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.2 MiB/s wr, 734 op/s 2023-09-10T01:24:54.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:54 smithi099 ceph-mon[102692]: pgmap v1431: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.2 MiB/s wr, 734 op/s 2023-09-10T01:24:54.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:54 smithi131 ceph-mon[103713]: pgmap v1431: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.2 MiB/s wr, 734 op/s 2023-09-10T01:24:55.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:55 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:24:55.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:55 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:24:55.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:55 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:24:56.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 230.0 s, 228.4 tps, lat 439.669 ms stddev 378.570 2023-09-10T01:24:56.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:56 smithi099 ceph-mon[102692]: pgmap v1432: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.2 MiB/s wr, 646 op/s 2023-09-10T01:24:56.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:56 smithi131 ceph-mon[103713]: pgmap v1432: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.2 MiB/s wr, 646 op/s 2023-09-10T01:24:56.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:56 smithi044 ceph-mon[98676]: pgmap v1432: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.2 MiB/s wr, 646 op/s 2023-09-10T01:24:57.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:57 smithi131 ceph-mon[103713]: pgmap v1433: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.7 MiB/s wr, 899 op/s 2023-09-10T01:24:57.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:57 smithi044 ceph-mon[98676]: pgmap v1433: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.7 MiB/s wr, 899 op/s 2023-09-10T01:24:57.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:57 smithi099 ceph-mon[102692]: pgmap v1433: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.7 MiB/s wr, 899 op/s 2023-09-10T01:25:00.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:24:59 smithi131 ceph-mon[103713]: pgmap v1434: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.8 MiB/s wr, 639 op/s 2023-09-10T01:25:00.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:24:59 smithi044 ceph-mon[98676]: pgmap v1434: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.8 MiB/s wr, 639 op/s 2023-09-10T01:25:00.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:24:59 smithi099 ceph-mon[102692]: pgmap v1434: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.8 MiB/s wr, 639 op/s 2023-09-10T01:25:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 235.0 s, 159.6 tps, lat 379.632 ms stddev 138.428 2023-09-10T01:25:02.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:02 smithi131 ceph-mon[103713]: pgmap v1435: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.0 MiB/s wr, 803 op/s 2023-09-10T01:25:02.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:02 smithi044 ceph-mon[98676]: pgmap v1435: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.0 MiB/s wr, 803 op/s 2023-09-10T01:25:02.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:02 smithi099 ceph-mon[102692]: pgmap v1435: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.0 MiB/s wr, 803 op/s 2023-09-10T01:25:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.765+0000 7f0972bbf700 1 -- 172.21.15.44:0/2569776600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0ff760 msgr2=0x7f096c0ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.765+0000 7f0972bbf700 1 --2- 172.21.15.44:0/2569776600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0ff760 0x7f096c0ffbe0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f095c0099c0 tx=0x7f095c009cd0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.765+0000 7f0972bbf700 1 -- 172.21.15.44:0/2569776600 shutdown_connections 2023-09-10T01:25:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.765+0000 7f0972bbf700 1 --2- 172.21.15.44:0/2569776600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f096c100120 0x7f096c10c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.765+0000 7f0972bbf700 1 --2- 172.21.15.44:0/2569776600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f096c1055b0 0x7f096c105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.765+0000 7f0972bbf700 1 --2- 172.21.15.44:0/2569776600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0ff760 0x7f096c0ffbe0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.765+0000 7f0972bbf700 1 -- 172.21.15.44:0/2569776600 >> 172.21.15.44:0/2569776600 conn(0x7f096c074f30 msgr2=0x7f096c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.765+0000 7f0972bbf700 1 -- 172.21.15.44:0/2569776600 shutdown_connections 2023-09-10T01:25:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.765+0000 7f0972bbf700 1 -- 172.21.15.44:0/2569776600 wait complete. 2023-09-10T01:25:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f0972bbf700 1 Processor -- start 2023-09-10T01:25:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f0972bbf700 1 -- start start 2023-09-10T01:25:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f0972bbf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0ff760 0x7f096c195450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f0972bbf700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f096c100120 0x7f096c195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f0972bbf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f096c1055b0 0x7f096c1a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f0972bbf700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c1960e0 con 0x7f096c0ff760 2023-09-10T01:25:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f0972bbf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c1a4e30 con 0x7f096c1055b0 2023-09-10T01:25:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f0972bbf700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c1a4fc0 con 0x7f096c100120 2023-09-10T01:25:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f097095b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0ff760 0x7f096c195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f097095b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0ff760 0x7f096c195450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59340/0 (socket says 172.21.15.44:59340) 2023-09-10T01:25:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f097095b700 1 -- 172.21.15.44:0/2745826479 learned_addr learned my addr 172.21.15.44:0/2745826479 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f096bfff700 1 --2- 172.21.15.44:0/2745826479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f096c100120 0x7f096c195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f097115c700 1 --2- 172.21.15.44:0/2745826479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f096c1055b0 0x7f096c1a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f097095b700 1 -- 172.21.15.44:0/2745826479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f096c100120 msgr2=0x7f096c195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f097095b700 1 --2- 172.21.15.44:0/2745826479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f096c100120 0x7f096c195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f097095b700 1 -- 172.21.15.44:0/2745826479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f096c1055b0 msgr2=0x7f096c1a48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f097095b700 1 --2- 172.21.15.44:0/2745826479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f096c1055b0 0x7f096c1a48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.766+0000 7f097095b700 1 -- 172.21.15.44:0/2745826479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c0095f0 con 0x7f096c0ff760 2023-09-10T01:25:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.767+0000 7f097095b700 1 --2- 172.21.15.44:0/2745826479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0ff760 0x7f096c195450 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f096000c9b0 tx=0x7f096000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.767+0000 7f0969ffb700 1 -- 172.21.15.44:0/2745826479 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960007b00 con 0x7f096c0ff760 2023-09-10T01:25:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.767+0000 7f0969ffb700 1 -- 172.21.15.44:0/2745826479 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f096000ce50 con 0x7f096c0ff760 2023-09-10T01:25:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.767+0000 7f0972bbf700 1 -- 172.21.15.44:0/2745826479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096c1a52a0 con 0x7f096c0ff760 2023-09-10T01:25:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.767+0000 7f0969ffb700 1 -- 172.21.15.44:0/2745826479 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960018420 con 0x7f096c0ff760 2023-09-10T01:25:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.767+0000 7f0972bbf700 1 -- 172.21.15.44:0/2745826479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c1a5780 con 0x7f096c0ff760 2023-09-10T01:25:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.769+0000 7f0969ffb700 1 -- 172.21.15.44:0/2745826479 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f096000f450 con 0x7f096c0ff760 2023-09-10T01:25:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.770+0000 7f0969ffb700 1 --2- 172.21.15.44:0/2745826479 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f09540a07b0 0x7f09540a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.770+0000 7f0969ffb700 1 -- 172.21.15.44:0/2745826479 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f09600cc2b0 con 0x7f096c0ff760 2023-09-10T01:25:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.770+0000 7f096bfff700 1 --2- 172.21.15.44:0/2745826479 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f09540a07b0 0x7f09540a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.770+0000 7f096bfff700 1 --2- 172.21.15.44:0/2745826479 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f09540a07b0 0x7f09540a2c70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f095c00b580 tx=0x7f095c009f10 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.770+0000 7f097ed85b80 1 Processor -- start 2023-09-10T01:25:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.770+0000 7f097ed85b80 1 -- start start 2023-09-10T01:25:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f097ed85b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3fb1a0 0x5623bf3fb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f097ed85b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3f5300 0x5623bf3f5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f097ed85b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5cc0 0x5623bf401f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f097ed85b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bf34e6d0 con 0x5623bf3fb1a0 2023-09-10T01:25:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f097ed85b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bf34bff0 con 0x5623bf3f5300 2023-09-10T01:25:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f097ed85b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bf3478e0 con 0x5623bf3f5cc0 2023-09-10T01:25:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f097095b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3fb1a0 0x5623bf3fb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f097095b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3fb1a0 0x5623bf3fb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59348/0 (socket says 172.21.15.44:59348) 2023-09-10T01:25:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f097095b700 1 -- 172.21.15.44:0/524751147 learned_addr learned my addr 172.21.15.44:0/524751147 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f096bfff700 1 --2- 172.21.15.44:0/524751147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3f5300 0x5623bf3f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f097115c700 1 --2- 172.21.15.44:0/524751147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5cc0 0x5623bf401f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f097095b700 1 -- 172.21.15.44:0/524751147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5cc0 msgr2=0x5623bf401f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f097095b700 1 --2- 172.21.15.44:0/524751147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5cc0 0x5623bf401f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f097095b700 1 -- 172.21.15.44:0/524751147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3f5300 msgr2=0x5623bf3f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f097095b700 1 --2- 172.21.15.44:0/524751147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3f5300 0x5623bf3f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f097095b700 1 -- 172.21.15.44:0/524751147 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623bf173d80 con 0x5623bf3fb1a0 2023-09-10T01:25:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f097095b700 1 --2- 172.21.15.44:0/524751147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3fb1a0 0x5623bf3fb580 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f0960095860 tx=0x7f0960098ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d9977397672b01 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f0952ffd700 1 -- 172.21.15.44:0/524751147 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096000f700 con 0x5623bf3fb1a0 2023-09-10T01:25:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f0952ffd700 1 -- 172.21.15.44:0/524751147 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0960099a60 con 0x5623bf3fb1a0 2023-09-10T01:25:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.771+0000 7f0952ffd700 1 -- 172.21.15.44:0/524751147 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09600a13d0 con 0x5623bf3fb1a0 2023-09-10T01:25:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.772+0000 7f097ed85b80 1 -- 172.21.15.44:0/524751147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3fb1a0 msgr2=0x5623bf3fb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.772+0000 7f097ed85b80 1 --2- 172.21.15.44:0/524751147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3fb1a0 0x5623bf3fb580 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f0960095860 tx=0x7f0960098ea0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.772+0000 7f097ed85b80 1 -- 172.21.15.44:0/524751147 shutdown_connections 2023-09-10T01:25:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.772+0000 7f097ed85b80 1 --2- 172.21.15.44:0/524751147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5cc0 0x5623bf401f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.772+0000 7f097ed85b80 1 --2- 172.21.15.44:0/524751147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3fb1a0 0x5623bf3fb580 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.772+0000 7f097ed85b80 1 --2- 172.21.15.44:0/524751147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3f5300 0x5623bf3f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.772+0000 7f097ed85b80 1 -- 172.21.15.44:0/524751147 >> 172.21.15.44:0/524751147 conn(0x5623bf36a030 msgr2=0x5623bf3f0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.772+0000 7f097ed85b80 1 -- 172.21.15.44:0/524751147 shutdown_connections 2023-09-10T01:25:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.772+0000 7f097ed85b80 1 -- 172.21.15.44:0/524751147 wait complete. 2023-09-10T01:25:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097ed85b80 1 -- start start 2023-09-10T01:25:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097ed85b80 1 Processor -- start 2023-09-10T01:25:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097ed85b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3f5300 0x5623bf401e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097ed85b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5cc0 0x5623bf3ff120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097ed85b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3fb1a0 0x5623bf3ff730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097ed85b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bf2e5b10 con 0x5623bf3fb1a0 2023-09-10T01:25:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097ed85b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bf2e5810 con 0x5623bf3f5300 2023-09-10T01:25:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097ed85b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bf1c98c0 con 0x5623bf3f5cc0 2023-09-10T01:25:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097ed85b80 10 client.0 _ll_register_callbacks cb 0x5623bf22fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:25:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097115c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3fb1a0 0x5623bf3ff730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097095b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3f5300 0x5623bf401e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f096bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5cc0 0x5623bf3ff120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097095b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3f5300 0x5623bf401e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:32862/0 (socket says 172.21.15.44:32862) 2023-09-10T01:25:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097095b700 1 -- 172.21.15.44:0/2125874808 learned_addr learned my addr 172.21.15.44:0/2125874808 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f096bfff700 1 --2- 172.21.15.44:0/2125874808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5cc0 0x5623bf3ff120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38006/0 (socket says 172.21.15.44:38006) 2023-09-10T01:25:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097095b700 1 -- 172.21.15.44:0/2125874808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5cc0 msgr2=0x5623bf3ff120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097095b700 1 --2- 172.21.15.44:0/2125874808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5cc0 0x5623bf3ff120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097095b700 1 -- 172.21.15.44:0/2125874808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3fb1a0 msgr2=0x5623bf3ff730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.773+0000 7f097095b700 1 --2- 172.21.15.44:0/2125874808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3fb1a0 0x5623bf3ff730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f097095b700 1 -- 172.21.15.44:0/2125874808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c1a5780 con 0x5623bf3f5300 2023-09-10T01:25:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f097095b700 1 --2- 172.21.15.44:0/2125874808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3f5300 0x5623bf401e00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f09600992a0 tx=0x7f0960004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69ec74f59e35245c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f094bfff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:25:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f094bfff700 1 -- 172.21.15.44:0/2125874808 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096009d190 con 0x5623bf3f5300 2023-09-10T01:25:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f094bfff700 1 -- 172.21.15.44:0/2125874808 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f0960099360 con 0x5623bf3f5300 2023-09-10T01:25:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f097ed85b80 1 -- 172.21.15.44:0/2125874808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623bf29f490 con 0x5623bf3f5300 2023-09-10T01:25:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f094bfff700 1 -- 172.21.15.44:0/2125874808 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=149) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0960065120 con 0x5623bf3f5300 2023-09-10T01:25:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f097ed85b80 10 client.29777 fetch_fsmap learned FSMap version 149 2023-09-10T01:25:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f097ed85b80 1 -- 172.21.15.44:0/2125874808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=149,monmap=4+}) v3 -- 0x5623bf15cc00 con 0x5623bf3f5300 2023-09-10T01:25:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f094bfff700 1 -- 172.21.15.44:0/2125874808 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5623bf29f490 con 0x5623bf3f5300 2023-09-10T01:25:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f094bfff700 1 -- 172.21.15.44:0/2125874808 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 149) v1 ==== 7367+0+0 (secure 0 0 0) 0x7f0960098850 con 0x5623bf3f5300 2023-09-10T01:25:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f097ed85b80 10 client.29777 fetch_fsmap finished waiting for FSMap version 149 2023-09-10T01:25:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f097ed85b80 10 client.29777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:25:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f097ed85b80 20 client.29777 populate_metadata read hostname 'smithi044' 2023-09-10T01:25:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f097ed85b80 1 --2- 172.21.15.44:0/2125874808 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623bf40eff0 0x5623bf411410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f097ed85b80 4 client.29777 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:25:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.774+0000 7f097ed85b80 1 -- 172.21.15.44:0/2125874808 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623bf35c840 con 0x5623bf40eff0 2023-09-10T01:25:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.775+0000 7f096bfff700 1 --2- 172.21.15.44:0/2125874808 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623bf40eff0 0x5623bf411410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.775+0000 7f096bfff700 1 --2- 172.21.15.44:0/2125874808 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623bf40eff0 0x5623bf411410 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a1f85a96e842502 server_cookie=a858e64f278fb9b5 in_seq=0 out_seq=0 2023-09-10T01:25:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.775+0000 7f094bfff700 10 client.29777 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.776+0000 7f094bfff700 1 -- 172.21.15.44:0/2125874808 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5623bf35c840 con 0x5623bf40eff0 2023-09-10T01:25:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.776+0000 7f094bfff700 10 client.29777 handle_command_reply: tid=0 2023-09-10T01:25:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.776+0000 7f097ed85b80 1 client.29777 shutdown 2023-09-10T01:25:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.776+0000 7f097ed85b80 1 -- 172.21.15.44:0/2125874808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3f5300 msgr2=0x5623bf401e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.776+0000 7f097ed85b80 1 --2- 172.21.15.44:0/2125874808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3f5300 0x5623bf401e00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f09600992a0 tx=0x7f0960004750 comp rx=0 tx=0).stop 2023-09-10T01:25:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.776+0000 7f097ed85b80 1 -- 172.21.15.44:0/2125874808 shutdown_connections 2023-09-10T01:25:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.776+0000 7f097ed85b80 1 --2- 172.21.15.44:0/2125874808 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623bf40eff0 0x5623bf411410 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.776+0000 7f097ed85b80 1 --2- 172.21.15.44:0/2125874808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3fb1a0 0x5623bf3ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.776+0000 7f094bfff700 0 client.29777 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.776+0000 7f097ed85b80 1 --2- 172.21.15.44:0/2125874808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3f5300 0x5623bf401e00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.776+0000 7f097ed85b80 1 --2- 172.21.15.44:0/2125874808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5cc0 0x5623bf3ff120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.776+0000 7f097ed85b80 1 -- 172.21.15.44:0/2125874808 >> 172.21.15.44:0/2125874808 conn(0x5623bf36a030 msgr2=0x5623bf3ef7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.776+0000 7f097ed85b80 1 -- 172.21.15.44:0/2125874808 shutdown_connections 2023-09-10T01:25:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.776+0000 7f097ed85b80 1 -- 172.21.15.44:0/2125874808 wait complete. 2023-09-10T01:25:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.776+0000 7f097ed85b80 20 client.29777 trim_cache size 0 max 16384 2023-09-10T01:25:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097ed85b80 1 Processor -- start 2023-09-10T01:25:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097ed85b80 1 -- start start 2023-09-10T01:25:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097ed85b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5300 0x5623bf406530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097ed85b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3f5cc0 0x5623bf3f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097ed85b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3fb1a0 0x5623bf3eff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097ed85b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bf3ecfc0 con 0x5623bf3f5cc0 2023-09-10T01:25:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097ed85b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bf400700 con 0x5623bf3fb1a0 2023-09-10T01:25:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097ed85b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bf406a70 con 0x5623bf3f5300 2023-09-10T01:25:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f096bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3f5cc0 0x5623bf3f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f096bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3f5cc0 0x5623bf3f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59364/0 (socket says 172.21.15.44:59364) 2023-09-10T01:25:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f096bfff700 1 -- 172.21.15.44:0/3723594398 learned_addr learned my addr 172.21.15.44:0/3723594398 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097095b700 1 --2- 172.21.15.44:0/3723594398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5300 0x5623bf406530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097115c700 1 --2- 172.21.15.44:0/3723594398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3fb1a0 0x5623bf3eff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f096bfff700 1 -- 172.21.15.44:0/3723594398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5300 msgr2=0x5623bf406530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f096bfff700 1 --2- 172.21.15.44:0/3723594398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5300 0x5623bf406530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f096bfff700 1 -- 172.21.15.44:0/3723594398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3fb1a0 msgr2=0x5623bf3eff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f096bfff700 1 --2- 172.21.15.44:0/3723594398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3fb1a0 0x5623bf3eff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f096bfff700 1 -- 172.21.15.44:0/3723594398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623bf19a1f0 con 0x5623bf3f5cc0 2023-09-10T01:25:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f096bfff700 1 --2- 172.21.15.44:0/3723594398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3f5cc0 0x5623bf3f4a00 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f095c0098e0 tx=0x7f095c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3a55d22cc89a84 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f09537fe700 1 -- 172.21.15.44:0/3723594398 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c005230 con 0x5623bf3f5cc0 2023-09-10T01:25:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f09537fe700 1 -- 172.21.15.44:0/3723594398 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f095c01e040 con 0x5623bf3f5cc0 2023-09-10T01:25:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f09537fe700 1 -- 172.21.15.44:0/3723594398 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c004760 con 0x5623bf3f5cc0 2023-09-10T01:25:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097ed85b80 1 -- 172.21.15.44:0/3723594398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3f5cc0 msgr2=0x5623bf3f4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097ed85b80 1 --2- 172.21.15.44:0/3723594398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3f5cc0 0x5623bf3f4a00 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f095c0098e0 tx=0x7f095c020000 comp rx=0 tx=0).stop 2023-09-10T01:25:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097ed85b80 1 -- 172.21.15.44:0/3723594398 shutdown_connections 2023-09-10T01:25:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097ed85b80 1 --2- 172.21.15.44:0/3723594398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3fb1a0 0x5623bf3eff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097ed85b80 1 --2- 172.21.15.44:0/3723594398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5300 0x5623bf406530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097ed85b80 1 --2- 172.21.15.44:0/3723594398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3f5cc0 0x5623bf3f4a00 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.792+0000 7f097ed85b80 1 -- 172.21.15.44:0/3723594398 >> 172.21.15.44:0/3723594398 conn(0x5623bf36a030 msgr2=0x5623bf3f42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.793+0000 7f097ed85b80 1 -- 172.21.15.44:0/3723594398 shutdown_connections 2023-09-10T01:25:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.793+0000 7f097ed85b80 1 -- 172.21.15.44:0/3723594398 wait complete. 2023-09-10T01:25:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.793+0000 7f097ed85b80 1 -- start start 2023-09-10T01:25:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.793+0000 7f097ed85b80 1 Processor -- start 2023-09-10T01:25:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.793+0000 7f097ed85b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5300 0x5623bf4060f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.793+0000 7f097ed85b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3f5cc0 0x5623bf404510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.793+0000 7f097ed85b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3fb1a0 0x5623bf404a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.793+0000 7f097ed85b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bf404f90 con 0x5623bf3f5cc0 2023-09-10T01:25:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.793+0000 7f097ed85b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bf405100 con 0x5623bf3fb1a0 2023-09-10T01:25:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.793+0000 7f097ed85b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bf4052c0 con 0x5623bf3f5300 2023-09-10T01:25:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.793+0000 7f097ed85b80 10 client.0 _ll_register_callbacks cb 0x7f095c005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:25:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.793+0000 7f096bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3f5cc0 0x5623bf404510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.793+0000 7f096bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3f5cc0 0x5623bf404510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59370/0 (socket says 172.21.15.44:59370) 2023-09-10T01:25:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.793+0000 7f097095b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5300 0x5623bf4060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.793+0000 7f096bfff700 1 -- 172.21.15.44:0/1625960643 learned_addr learned my addr 172.21.15.44:0/1625960643 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.793+0000 7f097115c700 1 --2- 172.21.15.44:0/1625960643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3fb1a0 0x5623bf404a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f097095b700 1 -- 172.21.15.44:0/1625960643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3fb1a0 msgr2=0x5623bf404a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f097095b700 1 --2- 172.21.15.44:0/1625960643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3fb1a0 0x5623bf404a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f097095b700 1 -- 172.21.15.44:0/1625960643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3f5cc0 msgr2=0x5623bf404510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f097095b700 1 --2- 172.21.15.44:0/1625960643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3f5cc0 0x5623bf404510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f097095b700 1 -- 172.21.15.44:0/1625960643 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c10cbc0 con 0x5623bf3f5300 2023-09-10T01:25:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f096bfff700 1 --2- 172.21.15.44:0/1625960643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3f5cc0 0x5623bf404510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:25:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f097115c700 1 --2- 172.21.15.44:0/1625960643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3fb1a0 0x5623bf404a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:25:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f097095b700 1 --2- 172.21.15.44:0/1625960643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5300 0x5623bf4060f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0960095860 tx=0x7f09600a11f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81b8417d48c835bc server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f094bfff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:25:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f094bfff700 1 -- 172.21.15.44:0/1625960643 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960065120 con 0x5623bf3f5300 2023-09-10T01:25:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f094bfff700 1 -- 172.21.15.44:0/1625960643 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f09600cc0d0 con 0x5623bf3f5300 2023-09-10T01:25:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f097ed85b80 1 -- 172.21.15.44:0/1625960643 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0960099a60 con 0x5623bf3f5300 2023-09-10T01:25:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f094bfff700 1 -- 172.21.15.44:0/1625960643 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=149) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0960099130 con 0x5623bf3f5300 2023-09-10T01:25:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f097ed85b80 10 client.29779 fetch_fsmap learned FSMap version 149 2023-09-10T01:25:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f097ed85b80 1 -- 172.21.15.44:0/1625960643 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=149,monmap=4+}) v3 -- 0x5623bf172250 con 0x5623bf3f5300 2023-09-10T01:25:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f094bfff700 1 -- 172.21.15.44:0/1625960643 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0960099a60 con 0x5623bf3f5300 2023-09-10T01:25:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.794+0000 7f094bfff700 1 -- 172.21.15.44:0/1625960643 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 149) v1 ==== 7367+0+0 (secure 0 0 0) 0x7f09600aed50 con 0x5623bf3f5300 2023-09-10T01:25:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.795+0000 7f097ed85b80 10 client.29779 fetch_fsmap finished waiting for FSMap version 149 2023-09-10T01:25:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.795+0000 7f097ed85b80 10 client.29779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:25:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.795+0000 7f097ed85b80 20 client.29779 populate_metadata read hostname 'smithi044' 2023-09-10T01:25:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.795+0000 7f097ed85b80 1 --2- 172.21.15.44:0/1625960643 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623bf463d40 0x5623bf466130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.795+0000 7f097ed85b80 4 client.29779 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:25:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.795+0000 7f097ed85b80 1 -- 172.21.15.44:0/1625960643 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5623bf3408c0 con 0x5623bf463d40 2023-09-10T01:25:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.795+0000 7f096bfff700 1 --2- 172.21.15.44:0/1625960643 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623bf463d40 0x5623bf466130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.795+0000 7f094bfff700 10 client.29779 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:02.795+0000 7f096bfff700 1 --2- 172.21.15.44:0/1625960643 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623bf463d40 0x5623bf466130 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff6fe60a194a143 server_cookie=f2a43f812ffd2950 in_seq=0 out_seq=0 2023-09-10T01:25:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.995+0000 7f094bfff700 1 -- 172.21.15.44:0/1625960643 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5623bf3408c0 con 0x5623bf463d40 2023-09-10T01:25:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.995+0000 7f094bfff700 10 client.29779 handle_command_reply: tid=0 2023-09-10T01:25:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.995+0000 7f097ed85b80 1 client.29779 shutdown 2023-09-10T01:25:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.995+0000 7f097ed85b80 1 -- 172.21.15.44:0/1625960643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5300 msgr2=0x5623bf4060f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.995+0000 7f097ed85b80 1 --2- 172.21.15.44:0/1625960643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5300 0x5623bf4060f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0960095860 tx=0x7f09600a11f0 comp rx=0 tx=0).stop 2023-09-10T01:25:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.995+0000 7f097ed85b80 1 -- 172.21.15.44:0/1625960643 shutdown_connections 2023-09-10T01:25:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.995+0000 7f097ed85b80 1 --2- 172.21.15.44:0/1625960643 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5623bf463d40 0x5623bf466130 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.995+0000 7f097ed85b80 1 --2- 172.21.15.44:0/1625960643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623bf3fb1a0 0x5623bf404a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.995+0000 7f097ed85b80 1 --2- 172.21.15.44:0/1625960643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623bf3f5300 0x5623bf4060f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.995+0000 7f097ed85b80 1 --2- 172.21.15.44:0/1625960643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623bf3f5cc0 0x5623bf404510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.995+0000 7f094bfff700 0 client.29779 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.995+0000 7f097ed85b80 1 -- 172.21.15.44:0/1625960643 >> 172.21.15.44:0/1625960643 conn(0x5623bf36a030 msgr2=0x5623bf3f4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.995+0000 7f097ed85b80 1 -- 172.21.15.44:0/1625960643 shutdown_connections 2023-09-10T01:25:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.995+0000 7f097ed85b80 1 -- 172.21.15.44:0/1625960643 wait complete. 2023-09-10T01:25:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.995+0000 7f097ed85b80 20 client.29779 trim_cache size 0 max 16384 2023-09-10T01:25:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:25:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:25:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:25:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:25:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.996+0000 7f0972bbf700 1 -- 172.21.15.44:0/2745826479 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f09540a07b0 msgr2=0x7f09540a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.996+0000 7f0972bbf700 1 --2- 172.21.15.44:0/2745826479 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f09540a07b0 0x7f09540a2c70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f095c00b580 tx=0x7f095c009f10 comp rx=0 tx=0).stop 2023-09-10T01:25:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.996+0000 7f0972bbf700 1 -- 172.21.15.44:0/2745826479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0ff760 msgr2=0x7f096c195450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.996+0000 7f0972bbf700 1 --2- 172.21.15.44:0/2745826479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0ff760 0x7f096c195450 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f096000c9b0 tx=0x7f096000ccc0 comp rx=0 tx=0).stop 2023-09-10T01:25:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.996+0000 7f0972bbf700 1 -- 172.21.15.44:0/2745826479 shutdown_connections 2023-09-10T01:25:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.996+0000 7f0972bbf700 1 --2- 172.21.15.44:0/2745826479 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f09540a07b0 0x7f09540a2c70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.996+0000 7f0972bbf700 1 --2- 172.21.15.44:0/2745826479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f096c1055b0 0x7f096c1a48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.996+0000 7f0972bbf700 1 --2- 172.21.15.44:0/2745826479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0ff760 0x7f096c195450 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.996+0000 7f0972bbf700 1 --2- 172.21.15.44:0/2745826479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f096c100120 0x7f096c195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.996+0000 7f0972bbf700 1 -- 172.21.15.44:0/2745826479 >> 172.21.15.44:0/2745826479 conn(0x7f096c074f30 msgr2=0x7f096c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.996+0000 7f0972bbf700 1 -- 172.21.15.44:0/2745826479 shutdown_connections 2023-09-10T01:25:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:03.996+0000 7f0972bbf700 1 -- 172.21.15.44:0/2745826479 wait complete. 2023-09-10T01:25:04.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:04 smithi131 ceph-mon[103713]: pgmap v1436: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.6 MiB/s wr, 627 op/s 2023-09-10T01:25:04.562 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:04 smithi044 ceph-mon[98676]: pgmap v1436: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.6 MiB/s wr, 627 op/s 2023-09-10T01:25:04.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:04 smithi099 ceph-mon[102692]: pgmap v1436: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.6 MiB/s wr, 627 op/s 2023-09-10T01:25:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 240.0 s, 143.0 tps, lat 986.336 ms stddev 2044.181 2023-09-10T01:25:06.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:06 smithi131 ceph-mon[103713]: pgmap v1437: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.8 MiB/s wr, 545 op/s 2023-09-10T01:25:06.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:06 smithi044 ceph-mon[98676]: pgmap v1437: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.8 MiB/s wr, 545 op/s 2023-09-10T01:25:06.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:06 smithi099 ceph-mon[102692]: pgmap v1437: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.8 MiB/s wr, 545 op/s 2023-09-10T01:25:08.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:08 smithi131 ceph-mon[103713]: pgmap v1438: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.5 MiB/s wr, 738 op/s 2023-09-10T01:25:08.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:08 smithi044 ceph-mon[98676]: pgmap v1438: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.5 MiB/s wr, 738 op/s 2023-09-10T01:25:08.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:08 smithi099 ceph-mon[102692]: pgmap v1438: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.5 MiB/s wr, 738 op/s 2023-09-10T01:25:10.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:10 smithi099 ceph-mon[102692]: pgmap v1439: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 872 KiB/s rd, 5.1 MiB/s wr, 507 op/s 2023-09-10T01:25:10.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:10 smithi099 ceph-mon[102692]: osdmap e179: 12 total, 12 up, 12 in 2023-09-10T01:25:10.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:10 smithi131 ceph-mon[103713]: pgmap v1439: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 872 KiB/s rd, 5.1 MiB/s wr, 507 op/s 2023-09-10T01:25:10.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:10 smithi131 ceph-mon[103713]: osdmap e179: 12 total, 12 up, 12 in 2023-09-10T01:25:10.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:10 smithi044 ceph-mon[98676]: pgmap v1439: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 872 KiB/s rd, 5.1 MiB/s wr, 507 op/s 2023-09-10T01:25:10.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:10 smithi044 ceph-mon[98676]: osdmap e179: 12 total, 12 up, 12 in 2023-09-10T01:25:11.185 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 245.0 s, 224.0 tps, lat 365.420 ms stddev 284.830 2023-09-10T01:25:11.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:11 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:25:11.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:11 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:25:11.813 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:11 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:25:12.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:12 smithi099 ceph-mon[102692]: pgmap v1441: 129 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 81 active+clean; 18 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 521 op/s 2023-09-10T01:25:12.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:12 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:25:12.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:12 smithi131 ceph-mon[103713]: pgmap v1441: 129 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 81 active+clean; 18 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 521 op/s 2023-09-10T01:25:12.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:12 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:25:12.812 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:12 smithi044 ceph-mon[98676]: pgmap v1441: 129 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 81 active+clean; 18 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 521 op/s 2023-09-10T01:25:12.812 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:12 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:25:13.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:13 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:25:13.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:13 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:25:13.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:13 smithi131 ceph-mon[103713]: pgmap v1442: 129 pgs: 20 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 63 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.6 MiB/s wr, 514 op/s 2023-09-10T01:25:13.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:13 smithi131 ceph-mon[103713]: pgmap v1443: 129 pgs: 20 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 63 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 647 op/s 2023-09-10T01:25:13.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:13 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:25:13.815 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:13 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:25:13.815 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:13 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:25:13.815 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:13 smithi044 ceph-mon[98676]: pgmap v1442: 129 pgs: 20 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 63 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.6 MiB/s wr, 514 op/s 2023-09-10T01:25:13.815 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:13 smithi044 ceph-mon[98676]: pgmap v1443: 129 pgs: 20 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 63 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 647 op/s 2023-09-10T01:25:13.816 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:13 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:25:13.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:13 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:25:13.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:13 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:25:13.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:13 smithi099 ceph-mon[102692]: pgmap v1442: 129 pgs: 20 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 63 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.6 MiB/s wr, 514 op/s 2023-09-10T01:25:13.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:13 smithi099 ceph-mon[102692]: pgmap v1443: 129 pgs: 20 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 63 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 647 op/s 2023-09-10T01:25:13.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:13 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:25:15.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:15 smithi044 ceph-mon[98676]: pgmap v1444: 129 pgs: 20 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 63 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 350 op/s 2023-09-10T01:25:15.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:15 smithi099 ceph-mon[102692]: pgmap v1444: 129 pgs: 20 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 63 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 350 op/s 2023-09-10T01:25:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:15 smithi131 ceph-mon[103713]: pgmap v1444: 129 pgs: 20 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 63 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 350 op/s 2023-09-10T01:25:16.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 250.0 s, 132.1 tps, lat 789.677 ms stddev 589.662 2023-09-10T01:25:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.207+0000 7f9bf73b1700 1 -- 172.21.15.44:0/3502100868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf0108220 msgr2=0x7f9bf010b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.207+0000 7f9bf73b1700 1 --2- 172.21.15.44:0/3502100868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf0108220 0x7f9bf010b830 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009d40 tx=0x7f9be000b540 comp rx=0 tx=0).stop 2023-09-10T01:25:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.207+0000 7f9bf73b1700 1 -- 172.21.15.44:0/3502100868 shutdown_connections 2023-09-10T01:25:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.207+0000 7f9bf73b1700 1 --2- 172.21.15.44:0/3502100868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf010bf10 0x7f9bf00788f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.207+0000 7f9bf73b1700 1 --2- 172.21.15.44:0/3502100868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf0107870 0x7f9bf0107c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.207+0000 7f9bf73b1700 1 --2- 172.21.15.44:0/3502100868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf0108220 0x7f9bf010b830 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.208+0000 7f9bf73b1700 1 -- 172.21.15.44:0/3502100868 >> 172.21.15.44:0/3502100868 conn(0x7f9bf006c6c0 msgr2=0x7f9bf006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.208+0000 7f9bf73b1700 1 -- 172.21.15.44:0/3502100868 shutdown_connections 2023-09-10T01:25:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.208+0000 7f9bf73b1700 1 -- 172.21.15.44:0/3502100868 wait complete. 2023-09-10T01:25:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.208+0000 7f9bf73b1700 1 Processor -- start 2023-09-10T01:25:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.209+0000 7f9bf73b1700 1 -- start start 2023-09-10T01:25:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.209+0000 7f9bf73b1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf0107870 0x7f9bf0078510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.209+0000 7f9bf73b1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf010bf10 0x7f9bf0074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.209+0000 7f9bf73b1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf0074840 0x7f9bf0074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.209+0000 7f9bf73b1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf007db40 con 0x7f9bf010bf10 2023-09-10T01:25:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.209+0000 7f9bf73b1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf007dcb0 con 0x7f9bf0107870 2023-09-10T01:25:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.209+0000 7f9bf73b1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf007de70 con 0x7f9bf0074840 2023-09-10T01:25:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.209+0000 7f9bf494c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf010bf10 0x7f9bf0074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.209+0000 7f9bf494c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf010bf10 0x7f9bf0074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59382/0 (socket says 172.21.15.44:59382) 2023-09-10T01:25:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.209+0000 7f9bf494c700 1 -- 172.21.15.44:0/3655498738 learned_addr learned my addr 172.21.15.44:0/3655498738 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.210+0000 7f9bf514d700 1 --2- 172.21.15.44:0/3655498738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf0107870 0x7f9bf0078510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.210+0000 7f9bf494c700 1 -- 172.21.15.44:0/3655498738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf0074840 msgr2=0x7f9bf0074cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.210+0000 7f9bf594e700 1 --2- 172.21.15.44:0/3655498738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf0074840 0x7f9bf0074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.210+0000 7f9bf494c700 1 --2- 172.21.15.44:0/3655498738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf0074840 0x7f9bf0074cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.210+0000 7f9bf494c700 1 -- 172.21.15.44:0/3655498738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf0107870 msgr2=0x7f9bf0078510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.210+0000 7f9bf494c700 1 --2- 172.21.15.44:0/3655498738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf0107870 0x7f9bf0078510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.210+0000 7f9bf494c700 1 -- 172.21.15.44:0/3655498738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be0009520 con 0x7f9bf010bf10 2023-09-10T01:25:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.210+0000 7f9bf514d700 1 --2- 172.21.15.44:0/3655498738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf0107870 0x7f9bf0078510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:25:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.210+0000 7f9bf594e700 1 --2- 172.21.15.44:0/3655498738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf0074840 0x7f9bf0074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:25:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.210+0000 7f9bf494c700 1 --2- 172.21.15.44:0/3655498738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf010bf10 0x7f9bf0074300 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009230 tx=0x7f9be00053a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.210+0000 7f9be67fc700 1 -- 172.21.15.44:0/3655498738 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be000bee0 con 0x7f9bf010bf10 2023-09-10T01:25:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.210+0000 7f9bf73b1700 1 -- 172.21.15.44:0/3655498738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf007e0f0 con 0x7f9bf010bf10 2023-09-10T01:25:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.210+0000 7f9bf73b1700 1 -- 172.21.15.44:0/3655498738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf007e5f0 con 0x7f9bf010bf10 2023-09-10T01:25:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.211+0000 7f9be67fc700 1 -- 172.21.15.44:0/3655498738 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9be0004b60 con 0x7f9bf010bf10 2023-09-10T01:25:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.211+0000 7f9be67fc700 1 -- 172.21.15.44:0/3655498738 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be000fb40 con 0x7f9bf010bf10 2023-09-10T01:25:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.214+0000 7f9be67fc700 1 -- 172.21.15.44:0/3655498738 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f9be0028070 con 0x7f9bf010bf10 2023-09-10T01:25:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.216+0000 7f9c03577b80 1 Processor -- start 2023-09-10T01:25:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.217+0000 7f9c03577b80 1 -- start start 2023-09-10T01:25:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.217+0000 7f9c03577b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a6641a0 0x55c27a664580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.217+0000 7f9c03577b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a65e300 0x55c27a65e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.217+0000 7f9c03577b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a65ecc0 0x55c27a66af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.217+0000 7f9c03577b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27a5b76d0 con 0x55c27a65e300 2023-09-10T01:25:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.217+0000 7f9c03577b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27a5b4ff0 con 0x55c27a6641a0 2023-09-10T01:25:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.217+0000 7f9c03577b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27a5b08e0 con 0x55c27a65ecc0 2023-09-10T01:25:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.217+0000 7f9be67fc700 1 --2- 172.21.15.44:0/3655498738 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9bdc0a08b0 0x7f9bdc0a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.218+0000 7f9be67fc700 1 -- 172.21.15.44:0/3655498738 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f9be00d5f00 con 0x7f9bf010bf10 2023-09-10T01:25:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.219+0000 7f9bf594e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a6641a0 0x55c27a664580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.219+0000 7f9bf594e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a6641a0 0x55c27a664580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:32898/0 (socket says 172.21.15.44:32898) 2023-09-10T01:25:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.219+0000 7f9bf594e700 1 -- 172.21.15.44:0/4125588687 learned_addr learned my addr 172.21.15.44:0/4125588687 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.219+0000 7f9bf594e700 1 --2- 172.21.15.44:0/3655498738 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9bdc0a08b0 0x7f9bdc0a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.220+0000 7f9bf594e700 1 -- 172.21.15.44:0/4125588687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a65ecc0 msgr2=0x55c27a66af30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:25:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.220+0000 7f9bf594e700 1 --2- 172.21.15.44:0/4125588687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a65ecc0 0x55c27a66af30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.220+0000 7f9bf594e700 1 -- 172.21.15.44:0/4125588687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a65e300 msgr2=0x55c27a65e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.220+0000 7f9bf594e700 1 --2- 172.21.15.44:0/4125588687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a65e300 0x55c27a65e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.220+0000 7f9bf594e700 1 -- 172.21.15.44:0/4125588687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c27a3dcd80 con 0x55c27a6641a0 2023-09-10T01:25:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.220+0000 7f9bf594e700 1 --2- 172.21.15.44:0/4125588687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a6641a0 0x55c27a664580 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9be8011b40 tx=0x7f9be8011e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff6df7921b7d5ffe server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.221+0000 7f9bdb7fe700 1 -- 172.21.15.44:0/4125588687 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be800e050 con 0x55c27a6641a0 2023-09-10T01:25:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.221+0000 7f9bdb7fe700 1 -- 172.21.15.44:0/4125588687 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9be801d070 con 0x55c27a6641a0 2023-09-10T01:25:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.221+0000 7f9bdb7fe700 1 -- 172.21.15.44:0/4125588687 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8007d00 con 0x55c27a6641a0 2023-09-10T01:25:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.222+0000 7f9bf594e700 1 --2- 172.21.15.44:0/3655498738 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9bdc0a08b0 0x7f9bdc0a2d70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9be8000f80 tx=0x7f9be80197f0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.226+0000 7f9c03577b80 1 -- 172.21.15.44:0/4125588687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a6641a0 msgr2=0x55c27a664580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.226+0000 7f9c03577b80 1 --2- 172.21.15.44:0/4125588687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a6641a0 0x55c27a664580 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9be8011b40 tx=0x7f9be8011e50 comp rx=0 tx=0).stop 2023-09-10T01:25:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.227+0000 7f9c03577b80 1 -- 172.21.15.44:0/4125588687 shutdown_connections 2023-09-10T01:25:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.227+0000 7f9c03577b80 1 --2- 172.21.15.44:0/4125588687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a65ecc0 0x55c27a66af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.227+0000 7f9c03577b80 1 --2- 172.21.15.44:0/4125588687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a6641a0 0x55c27a664580 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.227+0000 7f9c03577b80 1 --2- 172.21.15.44:0/4125588687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a65e300 0x55c27a65e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.227+0000 7f9c03577b80 1 -- 172.21.15.44:0/4125588687 >> 172.21.15.44:0/4125588687 conn(0x55c27a5d3030 msgr2=0x55c27a659560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.227+0000 7f9c03577b80 1 -- 172.21.15.44:0/4125588687 shutdown_connections 2023-09-10T01:25:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.227+0000 7f9c03577b80 1 -- 172.21.15.44:0/4125588687 wait complete. 2023-09-10T01:25:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.227+0000 7f9c03577b80 1 -- start start 2023-09-10T01:25:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.227+0000 7f9c03577b80 1 Processor -- start 2023-09-10T01:25:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.228+0000 7f9c03577b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a65e300 0x55c27a66ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.228+0000 7f9c03577b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a65ecc0 0x55c27a667ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.228+0000 7f9c03577b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a6641a0 0x55c27a668570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.228+0000 7f9c03577b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27a54eb10 con 0x55c27a6641a0 2023-09-10T01:25:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.228+0000 7f9c03577b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27a54e810 con 0x55c27a65e300 2023-09-10T01:25:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.228+0000 7f9c03577b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27a4328c0 con 0x55c27a65ecc0 2023-09-10T01:25:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.228+0000 7f9c03577b80 10 client.0 _ll_register_callbacks cb 0x55c27a498a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:25:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.228+0000 7f9bf594e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a6641a0 0x55c27a668570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.228+0000 7f9bf594e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a6641a0 0x55c27a668570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59396/0 (socket says 172.21.15.44:59396) 2023-09-10T01:25:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.228+0000 7f9bf594e700 1 -- 172.21.15.44:0/3807833291 learned_addr learned my addr 172.21.15.44:0/3807833291 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.228+0000 7f9bf494c700 1 --2- 172.21.15.44:0/3807833291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a65ecc0 0x55c27a667ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.229+0000 7f9bf594e700 1 -- 172.21.15.44:0/3807833291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a65ecc0 msgr2=0x55c27a667ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.229+0000 7f9bf594e700 1 --2- 172.21.15.44:0/3807833291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a65ecc0 0x55c27a667ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.229+0000 7f9bf594e700 1 -- 172.21.15.44:0/3807833291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a65e300 msgr2=0x55c27a66ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.229+0000 7f9bf594e700 1 --2- 172.21.15.44:0/3807833291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a65e300 0x55c27a66ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.229+0000 7f9bf594e700 1 -- 172.21.15.44:0/3807833291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be8011770 con 0x55c27a6641a0 2023-09-10T01:25:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.229+0000 7f9bf494c700 1 --2- 172.21.15.44:0/3807833291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a65ecc0 0x55c27a667ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:25:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.229+0000 7f9bf594e700 1 --2- 172.21.15.44:0/3807833291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a6641a0 0x55c27a668570 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009990 tx=0x7f9be80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401162616a5f5b5a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.229+0000 7f9bd8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:25:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.229+0000 7f9bd8ff9700 1 -- 172.21.15.44:0/3807833291 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8012040 con 0x55c27a6641a0 2023-09-10T01:25:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.229+0000 7f9bd8ff9700 1 -- 172.21.15.44:0/3807833291 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(179..179 src has 1..179) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f9be8004b40 con 0x55c27a6641a0 2023-09-10T01:25:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.230+0000 7f9c03577b80 1 -- 172.21.15.44:0/3807833291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c27a508490 con 0x55c27a6641a0 2023-09-10T01:25:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.230+0000 7f9bd8ff9700 1 -- 172.21.15.44:0/3807833291 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=150) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9be800e020 con 0x55c27a6641a0 2023-09-10T01:25:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.230+0000 7f9c03577b80 10 client.20154 fetch_fsmap learned FSMap version 150 2023-09-10T01:25:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.230+0000 7f9c03577b80 1 -- 172.21.15.44:0/3807833291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=150,monmap=4+}) v3 -- 0x55c27a3c5c00 con 0x55c27a6641a0 2023-09-10T01:25:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.239+0000 7f9bd8ff9700 1 -- 172.21.15.44:0/3807833291 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c27a508490 con 0x55c27a6641a0 2023-09-10T01:25:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.239+0000 7f9bd8ff9700 1 -- 172.21.15.44:0/3807833291 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 150) v1 ==== 7363+0+0 (secure 0 0 0) 0x7f9be8016890 con 0x55c27a6641a0 2023-09-10T01:25:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.241+0000 7f9c03577b80 10 client.20154 fetch_fsmap finished waiting for FSMap version 150 2023-09-10T01:25:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.241+0000 7f9c03577b80 10 client.20154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:25:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.241+0000 7f9c03577b80 20 client.20154 populate_metadata read hostname 'smithi044' 2023-09-10T01:25:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.241+0000 7f9c03577b80 1 --2- 172.21.15.44:0/3807833291 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c27a689550 0x55c27a68b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.241+0000 7f9c03577b80 4 client.20154 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:25:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.241+0000 7f9c03577b80 1 -- 172.21.15.44:0/3807833291 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c27a5c5840 con 0x55c27a689550 2023-09-10T01:25:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.246+0000 7f9bf514d700 1 --2- 172.21.15.44:0/3807833291 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c27a689550 0x55c27a68b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.246+0000 7f9bf514d700 1 --2- 172.21.15.44:0/3807833291 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c27a689550 0x55c27a68b970 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97972bfd2d37d6d1 server_cookie=18651522c7095fcb in_seq=0 out_seq=0 2023-09-10T01:25:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.246+0000 7f9bd8ff9700 10 client.20154 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.247+0000 7f9bd8ff9700 1 -- 172.21.15.44:0/3807833291 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c27a5c5840 con 0x55c27a689550 2023-09-10T01:25:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.247+0000 7f9bd8ff9700 10 client.20154 handle_command_reply: tid=0 2023-09-10T01:25:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.247+0000 7f9c03577b80 1 client.20154 shutdown 2023-09-10T01:25:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.247+0000 7f9c03577b80 1 -- 172.21.15.44:0/3807833291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a6641a0 msgr2=0x55c27a668570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.247+0000 7f9c03577b80 1 --2- 172.21.15.44:0/3807833291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a6641a0 0x55c27a668570 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009990 tx=0x7f9be80099e0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.252+0000 7f9c03577b80 1 -- 172.21.15.44:0/3807833291 shutdown_connections 2023-09-10T01:25:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.252+0000 7f9c03577b80 1 --2- 172.21.15.44:0/3807833291 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c27a689550 0x55c27a68b970 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.253+0000 7f9c03577b80 1 --2- 172.21.15.44:0/3807833291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a6641a0 0x55c27a668570 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.253+0000 7f9c03577b80 1 --2- 172.21.15.44:0/3807833291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a65e300 0x55c27a66ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.253+0000 7f9c03577b80 1 --2- 172.21.15.44:0/3807833291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a65ecc0 0x55c27a667ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.253+0000 7f9c03577b80 1 -- 172.21.15.44:0/3807833291 >> 172.21.15.44:0/3807833291 conn(0x55c27a5d3030 msgr2=0x55c27a6587a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.253+0000 7f9bd8ff9700 0 client.20154 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.259+0000 7f9c03577b80 1 -- 172.21.15.44:0/3807833291 shutdown_connections 2023-09-10T01:25:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.259+0000 7f9c03577b80 1 -- 172.21.15.44:0/3807833291 wait complete. 2023-09-10T01:25:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.260+0000 7f9c03577b80 20 client.20154 trim_cache size 0 max 16384 2023-09-10T01:25:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.285+0000 7f9c03577b80 1 Processor -- start 2023-09-10T01:25:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.285+0000 7f9c03577b80 1 -- start start 2023-09-10T01:25:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.285+0000 7f9c03577b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a65e300 0x55c27a66f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.285+0000 7f9c03577b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a65ecc0 0x55c27a65da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.285+0000 7f9c03577b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a6641a0 0x55c27a658f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.285+0000 7f9c03577b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27a686a50 con 0x55c27a65e300 2023-09-10T01:25:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.285+0000 7f9c03577b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27a664580 con 0x55c27a65ecc0 2023-09-10T01:25:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.285+0000 7f9c03577b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27a65f580 con 0x55c27a6641a0 2023-09-10T01:25:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.296+0000 7f9bf594e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a6641a0 0x55c27a658f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.296+0000 7f9bf594e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a6641a0 0x55c27a658f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38052/0 (socket says 172.21.15.44:38052) 2023-09-10T01:25:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.296+0000 7f9bf594e700 1 -- 172.21.15.44:0/3481808789 learned_addr learned my addr 172.21.15.44:0/3481808789 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.296+0000 7f9bf494c700 1 --2- 172.21.15.44:0/3481808789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a65ecc0 0x55c27a65da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.296+0000 7f9bf514d700 1 --2- 172.21.15.44:0/3481808789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a65e300 0x55c27a66f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.297+0000 7f9bf514d700 1 -- 172.21.15.44:0/3481808789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a6641a0 msgr2=0x55c27a658f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.297+0000 7f9bf514d700 1 --2- 172.21.15.44:0/3481808789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a6641a0 0x55c27a658f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.297+0000 7f9bf514d700 1 -- 172.21.15.44:0/3481808789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a65ecc0 msgr2=0x55c27a65da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.297+0000 7f9bf514d700 1 --2- 172.21.15.44:0/3481808789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a65ecc0 0x55c27a65da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.297+0000 7f9bf514d700 1 -- 172.21.15.44:0/3481808789 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c27a6905a0 con 0x55c27a65e300 2023-09-10T01:25:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.297+0000 7f9bf594e700 1 --2- 172.21.15.44:0/3481808789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a6641a0 0x55c27a658f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:25:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.297+0000 7f9bf514d700 1 --2- 172.21.15.44:0/3481808789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a65e300 0x55c27a66f530 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f9be8011e90 tx=0x7f9bec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eca6a979b0f0996 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.297+0000 7f9bdbfff700 1 -- 172.21.15.44:0/3481808789 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0195d0 con 0x55c27a65e300 2023-09-10T01:25:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.297+0000 7f9bdbfff700 1 -- 172.21.15.44:0/3481808789 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9bec018040 con 0x55c27a65e300 2023-09-10T01:25:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.297+0000 7f9bdbfff700 1 -- 172.21.15.44:0/3481808789 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0127b0 con 0x55c27a65e300 2023-09-10T01:25:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.298+0000 7f9c03577b80 1 -- 172.21.15.44:0/3481808789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a65e300 msgr2=0x55c27a66f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.298+0000 7f9c03577b80 1 --2- 172.21.15.44:0/3481808789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a65e300 0x55c27a66f530 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f9be8011e90 tx=0x7f9bec009750 comp rx=0 tx=0).stop 2023-09-10T01:25:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.304+0000 7f9c03577b80 1 -- 172.21.15.44:0/3481808789 shutdown_connections 2023-09-10T01:25:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.304+0000 7f9c03577b80 1 --2- 172.21.15.44:0/3481808789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a6641a0 0x55c27a658f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.304+0000 7f9c03577b80 1 --2- 172.21.15.44:0/3481808789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a65e300 0x55c27a66f530 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.304+0000 7f9c03577b80 1 --2- 172.21.15.44:0/3481808789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a65ecc0 0x55c27a65da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.304+0000 7f9c03577b80 1 -- 172.21.15.44:0/3481808789 >> 172.21.15.44:0/3481808789 conn(0x55c27a5d3030 msgr2=0x55c27a65d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.308+0000 7f9c03577b80 1 -- 172.21.15.44:0/3481808789 shutdown_connections 2023-09-10T01:25:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.310+0000 7f9c03577b80 1 -- 172.21.15.44:0/3481808789 wait complete. 2023-09-10T01:25:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.310+0000 7f9c03577b80 1 -- start start 2023-09-10T01:25:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.310+0000 7f9c03577b80 1 Processor -- start 2023-09-10T01:25:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.320+0000 7f9c03577b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a65e300 0x55c27a6d1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.320+0000 7f9c03577b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a65ecc0 0x55c27a6d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.320+0000 7f9c03577b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a6641a0 0x55c27a6dc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.320+0000 7f9c03577b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27a6dc8d0 con 0x55c27a65ecc0 2023-09-10T01:25:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.320+0000 7f9c03577b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27a6dca10 con 0x55c27a6641a0 2023-09-10T01:25:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.320+0000 7f9c03577b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27a6dcbd0 con 0x55c27a65e300 2023-09-10T01:25:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.320+0000 7f9c03577b80 10 client.0 _ll_register_callbacks cb 0x55c27a498a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:25:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.320+0000 7f9bf514d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a65e300 0x55c27a6d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.321+0000 7f9bf514d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a65e300 0x55c27a6d1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38058/0 (socket says 172.21.15.44:38058) 2023-09-10T01:25:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.321+0000 7f9bf514d700 1 -- 172.21.15.44:0/246184725 learned_addr learned my addr 172.21.15.44:0/246184725 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.323+0000 7f9bf594e700 1 --2- 172.21.15.44:0/246184725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a6641a0 0x55c27a6dc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.323+0000 7f9bf594e700 1 -- 172.21.15.44:0/246184725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a65e300 msgr2=0x55c27a6d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.323+0000 7f9bf594e700 1 --2- 172.21.15.44:0/246184725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a65e300 0x55c27a6d1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.323+0000 7f9bf594e700 1 -- 172.21.15.44:0/246184725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a65ecc0 msgr2=0x55c27a6d1a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:25:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.323+0000 7f9bf594e700 1 --2- 172.21.15.44:0/246184725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a65ecc0 0x55c27a6d1a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.323+0000 7f9bf594e700 1 -- 172.21.15.44:0/246184725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c27a3c5c00 con 0x55c27a6641a0 2023-09-10T01:25:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.325+0000 7f9bf594e700 1 --2- 172.21.15.44:0/246184725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a6641a0 0x55c27a6dc390 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009750 tx=0x7f9be8011320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a0bf812e7a2a53 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.325+0000 7f9bd8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:25:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.325+0000 7f9bd8ff9700 1 -- 172.21.15.44:0/246184725 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be800e030 con 0x55c27a6641a0 2023-09-10T01:25:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.325+0000 7f9bd8ff9700 1 -- 172.21.15.44:0/246184725 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(179..179 src has 1..179) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f9be80077d0 con 0x55c27a6641a0 2023-09-10T01:25:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.325+0000 7f9c03577b80 1 -- 172.21.15.44:0/246184725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9be801d070 con 0x55c27a6641a0 2023-09-10T01:25:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.325+0000 7f9bd8ff9700 1 -- 172.21.15.44:0/246184725 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=150) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9be8007990 con 0x55c27a6641a0 2023-09-10T01:25:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.325+0000 7f9c03577b80 10 client.29795 fetch_fsmap learned FSMap version 150 2023-09-10T01:25:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.325+0000 7f9c03577b80 1 -- 172.21.15.44:0/246184725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=150,monmap=4+}) v3 -- 0x7f9bec009990 con 0x55c27a6641a0 2023-09-10T01:25:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.326+0000 7f9bd8ff9700 1 -- 172.21.15.44:0/246184725 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9be801d070 con 0x55c27a6641a0 2023-09-10T01:25:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.326+0000 7f9bd8ff9700 1 -- 172.21.15.44:0/246184725 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 150) v1 ==== 7363+0+0 (secure 0 0 0) 0x7f9be802ecb0 con 0x55c27a6641a0 2023-09-10T01:25:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.326+0000 7f9c03577b80 10 client.29795 fetch_fsmap finished waiting for FSMap version 150 2023-09-10T01:25:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.326+0000 7f9c03577b80 10 client.29795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:25:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.326+0000 7f9c03577b80 20 client.29795 populate_metadata read hostname 'smithi044' 2023-09-10T01:25:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.326+0000 7f9c03577b80 1 --2- 172.21.15.44:0/246184725 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c27a6de030 0x55c27a6e0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.326+0000 7f9c03577b80 4 client.29795 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:25:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.326+0000 7f9c03577b80 1 -- 172.21.15.44:0/246184725 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c27a5a98c0 con 0x55c27a6de030 2023-09-10T01:25:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.327+0000 7f9bf514d700 1 --2- 172.21.15.44:0/246184725 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c27a6de030 0x55c27a6e0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.327+0000 7f9bd8ff9700 10 client.29795 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.327+0000 7f9bf514d700 1 --2- 172.21.15.44:0/246184725 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c27a6de030 0x55c27a6e0450 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f9145c796b4fcda server_cookie=16b21666b0f187b5 in_seq=0 out_seq=0 2023-09-10T01:25:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.387+0000 7f9bd8ff9700 1 -- 172.21.15.44:0/246184725 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c27a5a98c0 con 0x55c27a6de030 2023-09-10T01:25:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.387+0000 7f9bd8ff9700 10 client.29795 handle_command_reply: tid=0 2023-09-10T01:25:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.387+0000 7f9c03577b80 1 client.29795 shutdown 2023-09-10T01:25:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.387+0000 7f9c03577b80 1 -- 172.21.15.44:0/246184725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a6641a0 msgr2=0x55c27a6dc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.387+0000 7f9c03577b80 1 --2- 172.21.15.44:0/246184725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a6641a0 0x55c27a6dc390 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009750 tx=0x7f9be8011320 comp rx=0 tx=0).stop 2023-09-10T01:25:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.391+0000 7f9c03577b80 1 -- 172.21.15.44:0/246184725 shutdown_connections 2023-09-10T01:25:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.391+0000 7f9c03577b80 1 --2- 172.21.15.44:0/246184725 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c27a6de030 0x55c27a6e0450 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.391+0000 7f9c03577b80 1 --2- 172.21.15.44:0/246184725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c27a6641a0 0x55c27a6dc390 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.391+0000 7f9c03577b80 1 --2- 172.21.15.44:0/246184725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c27a65e300 0x55c27a6d1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.391+0000 7f9c03577b80 1 --2- 172.21.15.44:0/246184725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c27a65ecc0 0x55c27a6d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.391+0000 7f9c03577b80 1 -- 172.21.15.44:0/246184725 >> 172.21.15.44:0/246184725 conn(0x55c27a5d3030 msgr2=0x55c27a65db80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.391+0000 7f9bd8ff9700 0 client.29795 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.396+0000 7f9c03577b80 1 -- 172.21.15.44:0/246184725 shutdown_connections 2023-09-10T01:25:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.399+0000 7f9c03577b80 1 -- 172.21.15.44:0/246184725 wait complete. 2023-09-10T01:25:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.399+0000 7f9c03577b80 20 client.29795 trim_cache size 0 max 16384 2023-09-10T01:25:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:25:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:25:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:25:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:25:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.400+0000 7f9bf73b1700 1 -- 172.21.15.44:0/3655498738 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9bdc0a08b0 msgr2=0x7f9bdc0a2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.400+0000 7f9bf73b1700 1 --2- 172.21.15.44:0/3655498738 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9bdc0a08b0 0x7f9bdc0a2d70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9be8000f80 tx=0x7f9be80197f0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.400+0000 7f9bf73b1700 1 -- 172.21.15.44:0/3655498738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf010bf10 msgr2=0x7f9bf0074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.400+0000 7f9bf73b1700 1 --2- 172.21.15.44:0/3655498738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf010bf10 0x7f9bf0074300 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009230 tx=0x7f9be00053a0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.406+0000 7f9bf73b1700 1 -- 172.21.15.44:0/3655498738 shutdown_connections 2023-09-10T01:25:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.406+0000 7f9bf73b1700 1 --2- 172.21.15.44:0/3655498738 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9bdc0a08b0 0x7f9bdc0a2d70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.406+0000 7f9bf73b1700 1 --2- 172.21.15.44:0/3655498738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf0074840 0x7f9bf0074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.406+0000 7f9bf73b1700 1 --2- 172.21.15.44:0/3655498738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf0107870 0x7f9bf0078510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.406+0000 7f9bf73b1700 1 --2- 172.21.15.44:0/3655498738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf010bf10 0x7f9bf0074300 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.406+0000 7f9bf73b1700 1 -- 172.21.15.44:0/3655498738 >> 172.21.15.44:0/3655498738 conn(0x7f9bf006c6c0 msgr2=0x7f9bf0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.413+0000 7f9bf73b1700 1 -- 172.21.15.44:0/3655498738 shutdown_connections 2023-09-10T01:25:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:17.413+0000 7f9bf73b1700 1 -- 172.21.15.44:0/3655498738 wait complete. 2023-09-10T01:25:18.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:18 smithi131 ceph-mon[103713]: pgmap v1445: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.8 MiB/s wr, 488 op/s 2023-09-10T01:25:18.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:18 smithi044 ceph-mon[98676]: pgmap v1445: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.8 MiB/s wr, 488 op/s 2023-09-10T01:25:18.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:18 smithi099 ceph-mon[102692]: pgmap v1445: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.8 MiB/s wr, 488 op/s 2023-09-10T01:25:20.547 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:20 smithi131 ceph-mon[103713]: pgmap v1446: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 471 op/s 2023-09-10T01:25:20.565 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:20 smithi044 ceph-mon[98676]: pgmap v1446: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 471 op/s 2023-09-10T01:25:20.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:20 smithi099 ceph-mon[102692]: pgmap v1446: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 471 op/s 2023-09-10T01:25:21.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 255.0 s, 215.4 tps, lat 521.977 ms stddev 992.750 2023-09-10T01:25:22.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:22 smithi131 ceph-mon[103713]: pgmap v1447: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.8 MiB/s wr, 459 op/s 2023-09-10T01:25:22.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:22 smithi131 ceph-mon[103713]: osdmap e180: 12 total, 12 up, 12 in 2023-09-10T01:25:22.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:22 smithi044 ceph-mon[98676]: pgmap v1447: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.8 MiB/s wr, 459 op/s 2023-09-10T01:25:22.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:22 smithi044 ceph-mon[98676]: osdmap e180: 12 total, 12 up, 12 in 2023-09-10T01:25:22.612 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:22 smithi099 ceph-mon[102692]: pgmap v1447: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.8 MiB/s wr, 459 op/s 2023-09-10T01:25:22.612 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:22 smithi099 ceph-mon[102692]: osdmap e180: 12 total, 12 up, 12 in 2023-09-10T01:25:24.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:24 smithi131 ceph-mon[103713]: pgmap v1449: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.5 MiB/s wr, 533 op/s 2023-09-10T01:25:24.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:24 smithi044 ceph-mon[98676]: pgmap v1449: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.5 MiB/s wr, 533 op/s 2023-09-10T01:25:24.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:24 smithi099 ceph-mon[102692]: pgmap v1449: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.5 MiB/s wr, 533 op/s 2023-09-10T01:25:26.182 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 260.0 s, 263.9 tps, lat 368.192 ms stddev 139.970 2023-09-10T01:25:26.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:26 smithi044 ceph-mon[98676]: pgmap v1450: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.5 MiB/s wr, 533 op/s 2023-09-10T01:25:26.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:26 smithi099 ceph-mon[102692]: pgmap v1450: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.5 MiB/s wr, 533 op/s 2023-09-10T01:25:26.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:26 smithi131 ceph-mon[103713]: pgmap v1450: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.5 MiB/s wr, 533 op/s 2023-09-10T01:25:27.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:27 smithi044 ceph-mon[98676]: osdmap e181: 12 total, 12 up, 12 in 2023-09-10T01:25:27.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:27 smithi099 ceph-mon[102692]: osdmap e181: 12 total, 12 up, 12 in 2023-09-10T01:25:27.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:27 smithi131 ceph-mon[103713]: osdmap e181: 12 total, 12 up, 12 in 2023-09-10T01:25:28.610 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:28 smithi099 ceph-mon[102692]: pgmap v1452: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.9 MiB/s wr, 795 op/s 2023-09-10T01:25:28.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:28 smithi131 ceph-mon[103713]: pgmap v1452: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.9 MiB/s wr, 795 op/s 2023-09-10T01:25:28.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:28 smithi044 ceph-mon[98676]: pgmap v1452: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.9 MiB/s wr, 795 op/s 2023-09-10T01:25:30.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:30 smithi099 ceph-mon[102692]: pgmap v1453: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.1 MiB/s wr, 642 op/s 2023-09-10T01:25:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.623+0000 7fe6260b7700 1 -- 172.21.15.44:0/3376087099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe62010aec0 msgr2=0x7fe62010d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.623+0000 7fe6260b7700 1 --2- 172.21.15.44:0/3376087099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe62010aec0 0x7fe62010d2b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe618009f90 tx=0x7fe61800b330 comp rx=0 tx=0).stop 2023-09-10T01:25:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.623+0000 7fe6260b7700 1 -- 172.21.15.44:0/3376087099 shutdown_connections 2023-09-10T01:25:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.623+0000 7fe6260b7700 1 --2- 172.21.15.44:0/3376087099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe62010aec0 0x7fe62010d2b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.623+0000 7fe6260b7700 1 --2- 172.21.15.44:0/3376087099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6201044c0 0x7fe6201048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.623+0000 7fe6260b7700 1 --2- 172.21.15.44:0/3376087099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe620108590 0x7fe62010a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.623+0000 7fe6260b7700 1 -- 172.21.15.44:0/3376087099 >> 172.21.15.44:0/3376087099 conn(0x7fe6200769d0 msgr2=0x7fe6200fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.623+0000 7fe6260b7700 1 -- 172.21.15.44:0/3376087099 shutdown_connections 2023-09-10T01:25:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.623+0000 7fe6260b7700 1 -- 172.21.15.44:0/3376087099 wait complete. 2023-09-10T01:25:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.624+0000 7fe6260b7700 1 Processor -- start 2023-09-10T01:25:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.624+0000 7fe6260b7700 1 -- start start 2023-09-10T01:25:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.624+0000 7fe6260b7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6201044c0 0x7fe62010d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.624+0000 7fe6260b7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe620108590 0x7fe62010de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.624+0000 7fe6260b7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe62010aec0 0x7fe6201a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.624+0000 7fe6260b7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62010e6f0 con 0x7fe620108590 2023-09-10T01:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.624+0000 7fe6260b7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe620198630 con 0x7fe62010aec0 2023-09-10T01:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.624+0000 7fe6260b7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6201987f0 con 0x7fe6201044c0 2023-09-10T01:25:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.625+0000 7fe61effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe620108590 0x7fe62010de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.625+0000 7fe61effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe620108590 0x7fe62010de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59422/0 (socket says 172.21.15.44:59422) 2023-09-10T01:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.625+0000 7fe61effd700 1 -- 172.21.15.44:0/2000540099 learned_addr learned my addr 172.21.15.44:0/2000540099 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.625+0000 7fe61ffff700 1 --2- 172.21.15.44:0/2000540099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe62010aec0 0x7fe6201a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.625+0000 7fe61effd700 1 -- 172.21.15.44:0/2000540099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6201044c0 msgr2=0x7fe62010d950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-10T01:25:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.625+0000 7fe61effd700 1 --2- 172.21.15.44:0/2000540099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6201044c0 0x7fe62010d950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.625+0000 7fe61effd700 1 -- 172.21.15.44:0/2000540099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe62010aec0 msgr2=0x7fe6201a8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.625+0000 7fe61effd700 1 --2- 172.21.15.44:0/2000540099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe62010aec0 0x7fe6201a8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.625+0000 7fe61effd700 1 -- 172.21.15.44:0/2000540099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6140095f0 con 0x7fe620108590 2023-09-10T01:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.625+0000 7fe61ffff700 1 --2- 172.21.15.44:0/2000540099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe62010aec0 0x7fe6201a8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:25:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.626+0000 7fe61effd700 1 --2- 172.21.15.44:0/2000540099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe620108590 0x7fe62010de90 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe61000e970 tx=0x7fe61000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.626+0000 7fe61cff9700 1 -- 172.21.15.44:0/2000540099 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61000cbd0 con 0x7fe620108590 2023-09-10T01:25:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.626+0000 7fe61cff9700 1 -- 172.21.15.44:0/2000540099 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe61000eec0 con 0x7fe620108590 2023-09-10T01:25:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.626+0000 7fe6260b7700 1 -- 172.21.15.44:0/2000540099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe618009c40 con 0x7fe620108590 2023-09-10T01:25:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.626+0000 7fe61cff9700 1 -- 172.21.15.44:0/2000540099 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610005570 con 0x7fe620108590 2023-09-10T01:25:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.626+0000 7fe6260b7700 1 -- 172.21.15.44:0/2000540099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe620198ea0 con 0x7fe620108590 2023-09-10T01:25:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.631+0000 7fe61cff9700 1 -- 172.21.15.44:0/2000540099 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fe610004750 con 0x7fe620108590 2023-09-10T01:25:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.631+0000 7fe63227db80 1 Processor -- start 2023-09-10T01:25:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.631+0000 7fe63227db80 1 -- start start 2023-09-10T01:25:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.631+0000 7fe63227db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 0x5599e426c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.631+0000 7fe63227db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266300 0x5599e4266780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.631+0000 7fe61cff9700 1 --2- 172.21.15.44:0/2000540099 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe6080a0830 0x7fe6080a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.631+0000 7fe63227db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266cc0 0x5599e4272f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.631+0000 7fe63227db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e41bf6d0 con 0x5599e426c1a0 2023-09-10T01:25:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.631+0000 7fe63227db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e41bcff0 con 0x5599e4266cc0 2023-09-10T01:25:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.631+0000 7fe63227db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e41b88e0 con 0x5599e4266300 2023-09-10T01:25:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.632+0000 7fe61ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 0x5599e426c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.632+0000 7fe61cff9700 1 -- 172.21.15.44:0/2000540099 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fe610014070 con 0x7fe620108590 2023-09-10T01:25:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.632+0000 7fe61ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 0x5599e426c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59426/0 (socket says 172.21.15.44:59426) 2023-09-10T01:25:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.632+0000 7fe61ffff700 1 -- 172.21.15.44:0/3889378526 learned_addr learned my addr 172.21.15.44:0/3889378526 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.632+0000 7fe61ffff700 1 --2- 172.21.15.44:0/3889378526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266cc0 0x5599e4272f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.632+0000 7fe61f7fe700 1 --2- 172.21.15.44:0/2000540099 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe6080a0830 0x7fe6080a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.632+0000 7fe61ffff700 1 -- 172.21.15.44:0/3889378526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266300 msgr2=0x5599e4266780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.632+0000 7fe61ffff700 1 --2- 172.21.15.44:0/3889378526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266300 0x5599e4266780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.632+0000 7fe61ffff700 1 -- 172.21.15.44:0/3889378526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 msgr2=0x5599e426c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.632+0000 7fe61ffff700 1 --2- 172.21.15.44:0/3889378526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 0x5599e426c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.632+0000 7fe61ffff700 1 -- 172.21.15.44:0/3889378526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599e3fe4d80 con 0x5599e4266cc0 2023-09-10T01:25:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.632+0000 7fe61ffff700 1 --2- 172.21.15.44:0/3889378526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266cc0 0x5599e4272f30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe618008df0 tx=0x7fe61800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f037d3840188a7c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.633+0000 7fe605ffb700 1 -- 172.21.15.44:0/3889378526 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61800e040 con 0x5599e4266cc0 2023-09-10T01:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.633+0000 7fe605ffb700 1 -- 172.21.15.44:0/3889378526 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe618003a00 con 0x5599e4266cc0 2023-09-10T01:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.633+0000 7fe605ffb700 1 -- 172.21.15.44:0/3889378526 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61800bd00 con 0x5599e4266cc0 2023-09-10T01:25:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.633+0000 7fe63227db80 1 -- 172.21.15.44:0/3889378526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266cc0 msgr2=0x5599e4272f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.633+0000 7fe63227db80 1 --2- 172.21.15.44:0/3889378526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266cc0 0x5599e4272f30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe618008df0 tx=0x7fe61800b330 comp rx=0 tx=0).stop 2023-09-10T01:25:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.633+0000 7fe61f7fe700 1 --2- 172.21.15.44:0/2000540099 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe6080a0830 0x7fe6080a2cf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe614005fd0 tx=0x7fe614009310 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.634+0000 7fe63227db80 1 -- 172.21.15.44:0/3889378526 shutdown_connections 2023-09-10T01:25:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.634+0000 7fe63227db80 1 --2- 172.21.15.44:0/3889378526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266cc0 0x5599e4272f30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.634+0000 7fe63227db80 1 --2- 172.21.15.44:0/3889378526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 0x5599e426c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.634+0000 7fe63227db80 1 --2- 172.21.15.44:0/3889378526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266300 0x5599e4266780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.634+0000 7fe63227db80 1 -- 172.21.15.44:0/3889378526 >> 172.21.15.44:0/3889378526 conn(0x5599e41db030 msgr2=0x5599e4261560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.634+0000 7fe63227db80 1 -- 172.21.15.44:0/3889378526 shutdown_connections 2023-09-10T01:25:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.634+0000 7fe63227db80 1 -- 172.21.15.44:0/3889378526 wait complete. 2023-09-10T01:25:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.634+0000 7fe63227db80 1 -- start start 2023-09-10T01:25:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.634+0000 7fe63227db80 1 Processor -- start 2023-09-10T01:25:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe63227db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266300 0x5599e4270200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe63227db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e4266cc0 0x5599e4270740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe63227db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e426c1a0 0x5599e4270c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe63227db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e4156b10 con 0x5599e4266cc0 2023-09-10T01:25:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe63227db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e4156810 con 0x5599e426c1a0 2023-09-10T01:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe63227db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e403a8c0 con 0x5599e4266300 2023-09-10T01:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe63227db80 10 client.0 _ll_register_callbacks cb 0x5599e40a0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe61f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266300 0x5599e4270200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe61effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e4266cc0 0x5599e4270740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe61f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266300 0x5599e4270200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38084/0 (socket says 172.21.15.44:38084) 2023-09-10T01:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe61f7fe700 1 -- 172.21.15.44:0/4185301891 learned_addr learned my addr 172.21.15.44:0/4185301891 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe61ffff700 1 --2- 172.21.15.44:0/4185301891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e426c1a0 0x5599e4270c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe61f7fe700 1 -- 172.21.15.44:0/4185301891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e426c1a0 msgr2=0x5599e4270c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe61f7fe700 1 --2- 172.21.15.44:0/4185301891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e426c1a0 0x5599e4270c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe61f7fe700 1 -- 172.21.15.44:0/4185301891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e4266cc0 msgr2=0x5599e4270740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe61f7fe700 1 --2- 172.21.15.44:0/4185301891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e4266cc0 0x5599e4270740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe61f7fe700 1 -- 172.21.15.44:0/4185301891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe620106870 con 0x5599e4266300 2023-09-10T01:25:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe61f7fe700 1 --2- 172.21.15.44:0/4185301891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266300 0x5599e4270200 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe6140106e0 tx=0x7fe6140047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a4be341cf3b883d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.635+0000 7fe5feffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:25:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.636+0000 7fe5feffd700 1 -- 172.21.15.44:0/4185301891 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61401c830 con 0x5599e4266300 2023-09-10T01:25:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.636+0000 7fe5feffd700 1 -- 172.21.15.44:0/4185301891 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(181..181 src has 1..181) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fe61401c990 con 0x5599e4266300 2023-09-10T01:25:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.636+0000 7fe63227db80 1 -- 172.21.15.44:0/4185301891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599e4110490 con 0x5599e4266300 2023-09-10T01:25:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.636+0000 7fe5feffd700 1 -- 172.21.15.44:0/4185301891 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=150) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe61401cb50 con 0x5599e4266300 2023-09-10T01:25:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.636+0000 7fe63227db80 10 client.29797 fetch_fsmap learned FSMap version 150 2023-09-10T01:25:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.636+0000 7fe63227db80 1 -- 172.21.15.44:0/4185301891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=150,monmap=4+}) v3 -- 0x5599e3fcdc00 con 0x5599e4266300 2023-09-10T01:25:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.636+0000 7fe5feffd700 1 -- 172.21.15.44:0/4185301891 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5599e4110490 con 0x5599e4266300 2023-09-10T01:25:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.636+0000 7fe5feffd700 1 -- 172.21.15.44:0/4185301891 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 150) v1 ==== 7363+0+0 (secure 0 0 0) 0x7fe614018070 con 0x5599e4266300 2023-09-10T01:25:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.636+0000 7fe63227db80 10 client.29797 fetch_fsmap finished waiting for FSMap version 150 2023-09-10T01:25:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.636+0000 7fe63227db80 10 client.29797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:25:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.636+0000 7fe63227db80 20 client.29797 populate_metadata read hostname 'smithi044' 2023-09-10T01:25:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.636+0000 7fe63227db80 1 --2- 172.21.15.44:0/4185301891 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5599e4280590 0x5599e42829b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.636+0000 7fe63227db80 4 client.29797 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:25:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.636+0000 7fe63227db80 1 -- 172.21.15.44:0/4185301891 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599e41cd840 con 0x5599e4280590 2023-09-10T01:25:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.637+0000 7fe61effd700 1 --2- 172.21.15.44:0/4185301891 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5599e4280590 0x5599e42829b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.637+0000 7fe5feffd700 10 client.29797 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.637+0000 7fe61effd700 1 --2- 172.21.15.44:0/4185301891 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5599e4280590 0x5599e42829b0 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22c58ee2a539dbf0 server_cookie=c92529c4a920ff1d in_seq=0 out_seq=0 2023-09-10T01:25:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.638+0000 7fe5feffd700 1 -- 172.21.15.44:0/4185301891 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5599e41cd840 con 0x5599e4280590 2023-09-10T01:25:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.638+0000 7fe5feffd700 10 client.29797 handle_command_reply: tid=0 2023-09-10T01:25:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.638+0000 7fe63227db80 1 client.29797 shutdown 2023-09-10T01:25:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.638+0000 7fe63227db80 1 -- 172.21.15.44:0/4185301891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266300 msgr2=0x5599e4270200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.638+0000 7fe63227db80 1 --2- 172.21.15.44:0/4185301891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266300 0x5599e4270200 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe6140106e0 tx=0x7fe6140047c0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.638+0000 7fe63227db80 1 -- 172.21.15.44:0/4185301891 shutdown_connections 2023-09-10T01:25:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.638+0000 7fe63227db80 1 --2- 172.21.15.44:0/4185301891 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5599e4280590 0x5599e42829b0 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.638+0000 7fe63227db80 1 --2- 172.21.15.44:0/4185301891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e426c1a0 0x5599e4270c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.638+0000 7fe5feffd700 0 client.29797 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.638+0000 7fe63227db80 1 --2- 172.21.15.44:0/4185301891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266300 0x5599e4270200 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.638+0000 7fe63227db80 1 --2- 172.21.15.44:0/4185301891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e4266cc0 0x5599e4270740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.638+0000 7fe63227db80 1 -- 172.21.15.44:0/4185301891 >> 172.21.15.44:0/4185301891 conn(0x5599e41db030 msgr2=0x5599e42607a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.638+0000 7fe63227db80 1 -- 172.21.15.44:0/4185301891 shutdown_connections 2023-09-10T01:25:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.638+0000 7fe63227db80 1 -- 172.21.15.44:0/4185301891 wait complete. 2023-09-10T01:25:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.638+0000 7fe63227db80 20 client.29797 trim_cache size 0 max 16384 2023-09-10T01:25:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.672+0000 7fe63227db80 1 Processor -- start 2023-09-10T01:25:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.674+0000 7fe63227db80 1 -- start start 2023-09-10T01:25:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.674+0000 7fe63227db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266300 0x5599e426de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.674+0000 7fe63227db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266cc0 0x5599e4270e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.674+0000 7fe63227db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 0x5599e4277360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.674+0000 7fe63227db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e427db50 con 0x5599e426c1a0 2023-09-10T01:25:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.674+0000 7fe63227db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e4262930 con 0x5599e4266cc0 2023-09-10T01:25:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.674+0000 7fe63227db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e4269db0 con 0x5599e4266300 2023-09-10T01:25:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.674+0000 7fe61effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266cc0 0x5599e4270e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.674+0000 7fe61effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266cc0 0x5599e4270e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:32950/0 (socket says 172.21.15.44:32950) 2023-09-10T01:25:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.674+0000 7fe61effd700 1 -- 172.21.15.44:0/3887193553 learned_addr learned my addr 172.21.15.44:0/3887193553 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.675+0000 7fe61ffff700 1 --2- 172.21.15.44:0/3887193553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 0x5599e4277360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.675+0000 7fe61effd700 1 -- 172.21.15.44:0/3887193553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266300 msgr2=0x5599e426de20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:25:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.675+0000 7fe61effd700 1 --2- 172.21.15.44:0/3887193553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266300 0x5599e426de20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.675+0000 7fe61effd700 1 -- 172.21.15.44:0/3887193553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 msgr2=0x5599e4277360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.675+0000 7fe61effd700 1 --2- 172.21.15.44:0/3887193553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 0x5599e4277360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.675+0000 7fe61effd700 1 -- 172.21.15.44:0/3887193553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599e400b1f0 con 0x5599e4266cc0 2023-09-10T01:25:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.676+0000 7fe61effd700 1 --2- 172.21.15.44:0/3887193553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266cc0 0x5599e4270e80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe6100995f0 tx=0x7fe610084040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4826395fea6a065b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.676+0000 7fe6067fc700 1 -- 172.21.15.44:0/3887193553 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610099660 con 0x5599e4266cc0 2023-09-10T01:25:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.676+0000 7fe6067fc700 1 -- 172.21.15.44:0/3887193553 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe61009d490 con 0x5599e4266cc0 2023-09-10T01:25:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.676+0000 7fe6067fc700 1 -- 172.21.15.44:0/3887193553 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610095d70 con 0x5599e4266cc0 2023-09-10T01:25:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.677+0000 7fe63227db80 1 -- 172.21.15.44:0/3887193553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266cc0 msgr2=0x5599e4270e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.677+0000 7fe63227db80 1 --2- 172.21.15.44:0/3887193553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266cc0 0x5599e4270e80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe6100995f0 tx=0x7fe610084040 comp rx=0 tx=0).stop 2023-09-10T01:25:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.678+0000 7fe63227db80 1 -- 172.21.15.44:0/3887193553 shutdown_connections 2023-09-10T01:25:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.678+0000 7fe63227db80 1 --2- 172.21.15.44:0/3887193553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 0x5599e4277360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.678+0000 7fe63227db80 1 --2- 172.21.15.44:0/3887193553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266300 0x5599e426de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.678+0000 7fe63227db80 1 --2- 172.21.15.44:0/3887193553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266cc0 0x5599e4270e80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.678+0000 7fe63227db80 1 -- 172.21.15.44:0/3887193553 >> 172.21.15.44:0/3887193553 conn(0x5599e41db030 msgr2=0x5599e4270770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.679+0000 7fe63227db80 1 -- 172.21.15.44:0/3887193553 shutdown_connections 2023-09-10T01:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.679+0000 7fe63227db80 1 -- 172.21.15.44:0/3887193553 wait complete. 2023-09-10T01:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.679+0000 7fe63227db80 1 -- start start 2023-09-10T01:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.679+0000 7fe63227db80 1 Processor -- start 2023-09-10T01:25:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.680+0000 7fe63227db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266300 0x5599e42cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.680+0000 7fe63227db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266cc0 0x5599e42cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.680+0000 7fe63227db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 0x5599e42c8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.680+0000 7fe63227db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e42ce220 con 0x5599e426c1a0 2023-09-10T01:25:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.680+0000 7fe63227db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e42c88a0 con 0x5599e4266300 2023-09-10T01:25:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.680+0000 7fe63227db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e42c8a60 con 0x5599e4266cc0 2023-09-10T01:25:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.680+0000 7fe63227db80 10 client.0 _ll_register_callbacks cb 0x7fe61400eef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:25:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.682+0000 7fe61ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 0x5599e42c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.682+0000 7fe61ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 0x5599e42c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59446/0 (socket says 172.21.15.44:59446) 2023-09-10T01:25:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.682+0000 7fe61ffff700 1 -- 172.21.15.44:0/1535324401 learned_addr learned my addr 172.21.15.44:0/1535324401 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.682+0000 7fe61effd700 1 --2- 172.21.15.44:0/1535324401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266cc0 0x5599e42cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.682+0000 7fe61effd700 1 -- 172.21.15.44:0/1535324401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266300 msgr2=0x5599e42cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.682+0000 7fe61effd700 1 --2- 172.21.15.44:0/1535324401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266300 0x5599e42cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.682+0000 7fe61effd700 1 -- 172.21.15.44:0/1535324401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 msgr2=0x5599e42c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.682+0000 7fe61effd700 1 --2- 172.21.15.44:0/1535324401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 0x5599e42c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.682+0000 7fe61effd700 1 -- 172.21.15.44:0/1535324401 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe620198ea0 con 0x5599e4266cc0 2023-09-10T01:25:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.691+0000 7fe61effd700 1 --2- 172.21.15.44:0/1535324401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266cc0 0x5599e42cdab0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe610005060 tx=0x7fe61000ce80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f9629cdf8cce199 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.691+0000 7fe5feffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:25:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.694+0000 7fe5feffd700 1 -- 172.21.15.44:0/1535324401 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6100a1750 con 0x5599e4266cc0 2023-09-10T01:25:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.694+0000 7fe5feffd700 1 -- 172.21.15.44:0/1535324401 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(181..181 src has 1..181) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fe610097a00 con 0x5599e4266cc0 2023-09-10T01:25:30.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.696+0000 7fe63227db80 1 -- 172.21.15.44:0/1535324401 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe618003a00 con 0x5599e4266cc0 2023-09-10T01:25:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.701+0000 7fe5feffd700 1 -- 172.21.15.44:0/1535324401 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=150) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe61009d150 con 0x5599e4266cc0 2023-09-10T01:25:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.704+0000 7fe63227db80 10 client.29800 fetch_fsmap learned FSMap version 150 2023-09-10T01:25:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.704+0000 7fe63227db80 1 -- 172.21.15.44:0/1535324401 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=150,monmap=4+}) v3 -- 0x5599e3fe3250 con 0x5599e4266cc0 2023-09-10T01:25:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.704+0000 7fe5feffd700 1 -- 172.21.15.44:0/1535324401 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe610080020 con 0x5599e4266cc0 2023-09-10T01:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.704+0000 7fe5feffd700 1 -- 172.21.15.44:0/1535324401 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 150) v1 ==== 7363+0+0 (secure 0 0 0) 0x7fe610004a00 con 0x5599e4266cc0 2023-09-10T01:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.704+0000 7fe63227db80 10 client.29800 fetch_fsmap finished waiting for FSMap version 150 2023-09-10T01:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.704+0000 7fe63227db80 10 client.29800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.704+0000 7fe63227db80 20 client.29800 populate_metadata read hostname 'smithi044' 2023-09-10T01:25:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.704+0000 7fe63227db80 1 --2- 172.21.15.44:0/1535324401 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5599e42e5e00 0x5599e42e8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.704+0000 7fe63227db80 4 client.29800 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:25:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.704+0000 7fe63227db80 1 -- 172.21.15.44:0/1535324401 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5599e41b18c0 con 0x5599e42e5e00 2023-09-10T01:25:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.705+0000 7fe61f7fe700 1 --2- 172.21.15.44:0/1535324401 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5599e42e5e00 0x5599e42e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.705+0000 7fe61f7fe700 1 --2- 172.21.15.44:0/1535324401 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5599e42e5e00 0x5599e42e8220 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4690577e42ac2ef server_cookie=ff9acfde9b230bc9 in_seq=0 out_seq=0 2023-09-10T01:25:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.710+0000 7fe5feffd700 10 client.29800 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.727+0000 7fe5feffd700 1 -- 172.21.15.44:0/1535324401 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5599e41b18c0 con 0x5599e42e5e00 2023-09-10T01:25:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.727+0000 7fe5feffd700 10 client.29800 handle_command_reply: tid=0 2023-09-10T01:25:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.728+0000 7fe63227db80 1 client.29800 shutdown 2023-09-10T01:25:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.729+0000 7fe63227db80 1 -- 172.21.15.44:0/1535324401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266cc0 msgr2=0x5599e42cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.729+0000 7fe63227db80 1 --2- 172.21.15.44:0/1535324401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266cc0 0x5599e42cdab0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe610005060 tx=0x7fe61000ce80 comp rx=0 tx=0).stop 2023-09-10T01:25:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.729+0000 7fe63227db80 1 -- 172.21.15.44:0/1535324401 shutdown_connections 2023-09-10T01:25:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.729+0000 7fe63227db80 1 --2- 172.21.15.44:0/1535324401 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5599e42e5e00 0x5599e42e8220 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.729+0000 7fe63227db80 1 --2- 172.21.15.44:0/1535324401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599e426c1a0 0x5599e42c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.729+0000 7fe63227db80 1 --2- 172.21.15.44:0/1535324401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e4266300 0x5599e42cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.729+0000 7fe63227db80 1 --2- 172.21.15.44:0/1535324401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599e4266cc0 0x5599e42cdab0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.729+0000 7fe63227db80 1 -- 172.21.15.44:0/1535324401 >> 172.21.15.44:0/1535324401 conn(0x5599e41db030 msgr2=0x5599e42652f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.729+0000 7fe5feffd700 0 client.29800 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.729+0000 7fe63227db80 1 -- 172.21.15.44:0/1535324401 shutdown_connections 2023-09-10T01:25:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.729+0000 7fe63227db80 1 -- 172.21.15.44:0/1535324401 wait complete. 2023-09-10T01:25:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.730+0000 7fe63227db80 20 client.29800 trim_cache size 0 max 16384 2023-09-10T01:25:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:25:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:25:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:25:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:25:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.730+0000 7fe6260b7700 1 -- 172.21.15.44:0/2000540099 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe6080a0830 msgr2=0x7fe6080a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.730+0000 7fe6260b7700 1 --2- 172.21.15.44:0/2000540099 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe6080a0830 0x7fe6080a2cf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe614005fd0 tx=0x7fe614009310 comp rx=0 tx=0).stop 2023-09-10T01:25:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.730+0000 7fe6260b7700 1 -- 172.21.15.44:0/2000540099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe620108590 msgr2=0x7fe62010de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.730+0000 7fe6260b7700 1 --2- 172.21.15.44:0/2000540099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe620108590 0x7fe62010de90 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe61000e970 tx=0x7fe61000ec80 comp rx=0 tx=0).stop 2023-09-10T01:25:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.730+0000 7fe6260b7700 1 -- 172.21.15.44:0/2000540099 shutdown_connections 2023-09-10T01:25:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.730+0000 7fe6260b7700 1 --2- 172.21.15.44:0/2000540099 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe6080a0830 0x7fe6080a2cf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.730+0000 7fe6260b7700 1 --2- 172.21.15.44:0/2000540099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe62010aec0 0x7fe6201a8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.730+0000 7fe6260b7700 1 --2- 172.21.15.44:0/2000540099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6201044c0 0x7fe62010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.730+0000 7fe6260b7700 1 --2- 172.21.15.44:0/2000540099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe620108590 0x7fe62010de90 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.730+0000 7fe6260b7700 1 -- 172.21.15.44:0/2000540099 >> 172.21.15.44:0/2000540099 conn(0x7fe6200769d0 msgr2=0x7fe620109480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.730+0000 7fe6260b7700 1 -- 172.21.15.44:0/2000540099 shutdown_connections 2023-09-10T01:25:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:30.731+0000 7fe6260b7700 1 -- 172.21.15.44:0/2000540099 wait complete. 2023-09-10T01:25:30.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:30 smithi131 ceph-mon[103713]: pgmap v1453: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.1 MiB/s wr, 642 op/s 2023-09-10T01:25:30.813 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:30 smithi044 ceph-mon[98676]: pgmap v1453: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.1 MiB/s wr, 642 op/s 2023-09-10T01:25:31.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 265.0 s, 302.9 tps, lat 342.806 ms stddev 160.552 2023-09-10T01:25:32.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:32 smithi099 ceph-mon[102692]: pgmap v1454: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.0 MiB/s wr, 747 op/s 2023-09-10T01:25:32.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:32 smithi131 ceph-mon[103713]: pgmap v1454: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.0 MiB/s wr, 747 op/s 2023-09-10T01:25:32.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:32 smithi044 ceph-mon[98676]: pgmap v1454: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.0 MiB/s wr, 747 op/s 2023-09-10T01:25:34.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:34 smithi099 ceph-mon[102692]: pgmap v1455: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.0 MiB/s wr, 774 op/s 2023-09-10T01:25:34.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:34 smithi131 ceph-mon[103713]: pgmap v1455: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.0 MiB/s wr, 774 op/s 2023-09-10T01:25:34.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:34 smithi044 ceph-mon[98676]: pgmap v1455: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.0 MiB/s wr, 774 op/s 2023-09-10T01:25:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 270.0 s, 270.2 tps, lat 359.354 ms stddev 118.243 2023-09-10T01:25:36.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:36 smithi131 ceph-mon[103713]: pgmap v1456: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.0 MiB/s wr, 774 op/s 2023-09-10T01:25:36.837 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:36 smithi044 ceph-mon[98676]: pgmap v1456: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.0 MiB/s wr, 774 op/s 2023-09-10T01:25:36.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:36 smithi099 ceph-mon[102692]: pgmap v1456: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.0 MiB/s wr, 774 op/s 2023-09-10T01:25:37.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:37 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:25:37.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:37 smithi131 ceph-mon[103713]: pgmap v1457: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 776 op/s 2023-09-10T01:25:37.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:37 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:25:37.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:37 smithi044 ceph-mon[98676]: pgmap v1457: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 776 op/s 2023-09-10T01:25:37.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:37 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:25:37.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:37 smithi099 ceph-mon[102692]: pgmap v1457: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 776 op/s 2023-09-10T01:25:39.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:39 smithi131 ceph-mon[103713]: pgmap v1458: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.9 MiB/s wr, 727 op/s 2023-09-10T01:25:39.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:39 smithi044 ceph-mon[98676]: pgmap v1458: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.9 MiB/s wr, 727 op/s 2023-09-10T01:25:39.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:39 smithi099 ceph-mon[102692]: pgmap v1458: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.9 MiB/s wr, 727 op/s 2023-09-10T01:25:41.185 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 275.0 s, 284.2 tps, lat 353.991 ms stddev 141.158 2023-09-10T01:25:42.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:42 smithi131 ceph-mon[103713]: pgmap v1459: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 954 op/s 2023-09-10T01:25:42.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:42 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:25:42.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:42 smithi044 ceph-mon[98676]: pgmap v1459: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 954 op/s 2023-09-10T01:25:42.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:42 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:25:42.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:42 smithi099 ceph-mon[102692]: pgmap v1459: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 954 op/s 2023-09-10T01:25:42.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:42 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:25:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.937+0000 7f002fe09700 1 -- 172.21.15.44:0/627953621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0028113750 msgr2=0x7f0028115b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.937+0000 7f002fe09700 1 --2- 172.21.15.44:0/627953621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0028113750 0x7f0028115b40 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f0020009990 tx=0x7f0020009ca0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.937+0000 7f002fe09700 1 -- 172.21.15.44:0/627953621 shutdown_connections 2023-09-10T01:25:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.937+0000 7f002fe09700 1 --2- 172.21.15.44:0/627953621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0028113750 0x7f0028115b40 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.937+0000 7f002fe09700 1 --2- 172.21.15.44:0/627953621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028107600 0x7f00281079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.937+0000 7f002fe09700 1 --2- 172.21.15.44:0/627953621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0028107f20 0x7f0028113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.937+0000 7f002fe09700 1 -- 172.21.15.44:0/627953621 >> 172.21.15.44:0/627953621 conn(0x7f002806c6c0 msgr2=0x7f002806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.940+0000 7f002fe09700 1 -- 172.21.15.44:0/627953621 shutdown_connections 2023-09-10T01:25:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.940+0000 7f002fe09700 1 -- 172.21.15.44:0/627953621 wait complete. 2023-09-10T01:25:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.941+0000 7f002fe09700 1 Processor -- start 2023-09-10T01:25:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.941+0000 7f002fe09700 1 -- start start 2023-09-10T01:25:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.941+0000 7f002fe09700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0028107600 0x7f002810c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.941+0000 7f002fe09700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028107f20 0x7f002810c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.941+0000 7f002fe09700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0028113750 0x7f00281111b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.941+0000 7f002fe09700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002810d2c0 con 0x7f0028113750 2023-09-10T01:25:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.941+0000 7f002fe09700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00281116f0 con 0x7f0028107f20 2023-09-10T01:25:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.941+0000 7f002fe09700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0028111860 con 0x7f0028107600 2023-09-10T01:25:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.942+0000 7f002d3a4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028107f20 0x7f002810c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.942+0000 7f002dba5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0028107600 0x7f002810c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.942+0000 7f002d3a4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028107f20 0x7f002810c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:32998/0 (socket says 172.21.15.44:32998) 2023-09-10T01:25:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.942+0000 7f002dba5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0028107600 0x7f002810c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38138/0 (socket says 172.21.15.44:38138) 2023-09-10T01:25:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.942+0000 7f002d3a4700 1 -- 172.21.15.44:0/636457453 learned_addr learned my addr 172.21.15.44:0/636457453 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.942+0000 7f002d3a4700 1 -- 172.21.15.44:0/636457453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0028107600 msgr2=0x7f002810c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.942+0000 7f002d3a4700 1 --2- 172.21.15.44:0/636457453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0028107600 0x7f002810c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.942+0000 7f002d3a4700 1 -- 172.21.15.44:0/636457453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0028113750 msgr2=0x7f00281111b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.942+0000 7f002d3a4700 1 --2- 172.21.15.44:0/636457453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0028113750 0x7f00281111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.942+0000 7f002d3a4700 1 -- 172.21.15.44:0/636457453 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0018009c40 con 0x7f0028107f20 2023-09-10T01:25:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.942+0000 7f002d3a4700 1 --2- 172.21.15.44:0/636457453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028107f20 0x7f002810c9b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f001c00e9a0 tx=0x7f001c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.943+0000 7f0026ffd700 1 -- 172.21.15.44:0/636457453 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c009790 con 0x7f0028107f20 2023-09-10T01:25:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.943+0000 7f0026ffd700 1 -- 172.21.15.44:0/636457453 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f001c010430 con 0x7f0028107f20 2023-09-10T01:25:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.943+0000 7f0026ffd700 1 -- 172.21.15.44:0/636457453 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c018680 con 0x7f0028107f20 2023-09-10T01:25:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.943+0000 7f002fe09700 1 -- 172.21.15.44:0/636457453 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00200095f0 con 0x7f0028107f20 2023-09-10T01:25:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.943+0000 7f002fe09700 1 -- 172.21.15.44:0/636457453 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0028111e70 con 0x7f0028107f20 2023-09-10T01:25:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.947+0000 7f0026ffd700 1 -- 172.21.15.44:0/636457453 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f001c00cb50 con 0x7f0028107f20 2023-09-10T01:25:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.949+0000 7f003bfcfb80 1 Processor -- start 2023-09-10T01:25:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.949+0000 7f003bfcfb80 1 -- start start 2023-09-10T01:25:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.949+0000 7f003bfcfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c06e1a0 0x55a87c06e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.949+0000 7f003bfcfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c068300 0x55a87c068780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.949+0000 7f003bfcfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068cc0 0x55a87c074f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.949+0000 7f003bfcfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87bfc16d0 con 0x55a87c068cc0 2023-09-10T01:25:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.949+0000 7f003bfcfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87bfbeff0 con 0x55a87c068300 2023-09-10T01:25:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.949+0000 7f003bfcfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87bfba8e0 con 0x55a87c06e1a0 2023-09-10T01:25:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.949+0000 7f0026ffd700 1 --2- 172.21.15.44:0/636457453 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f00100a0860 0x7f00100a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.949+0000 7f002e3a6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c06e1a0 0x55a87c06e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.949+0000 7f002e3a6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c06e1a0 0x55a87c06e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38144/0 (socket says 172.21.15.44:38144) 2023-09-10T01:25:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.949+0000 7f002e3a6700 1 -- 172.21.15.44:0/1257638001 learned_addr learned my addr 172.21.15.44:0/1257638001 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.949+0000 7f002d3a4700 1 --2- 172.21.15.44:0/1257638001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068cc0 0x55a87c074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.949+0000 7f0026ffd700 1 -- 172.21.15.44:0/636457453 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f001c014070 con 0x7f0028107f20 2023-09-10T01:25:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.950+0000 7f002e3a6700 1 -- 172.21.15.44:0/1257638001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c068300 msgr2=0x55a87c068780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.950+0000 7f002e3a6700 1 --2- 172.21.15.44:0/1257638001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c068300 0x55a87c068780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.950+0000 7f002e3a6700 1 -- 172.21.15.44:0/1257638001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068cc0 msgr2=0x55a87c074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.950+0000 7f002e3a6700 1 --2- 172.21.15.44:0/1257638001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068cc0 0x55a87c074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.950+0000 7f002e3a6700 1 -- 172.21.15.44:0/1257638001 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a87bde6d80 con 0x55a87c06e1a0 2023-09-10T01:25:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.950+0000 7f002e3a6700 1 --2- 172.21.15.44:0/1257638001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c06e1a0 0x55a87c06e580 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0020005930 tx=0x7f0020004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14e8fcadc4a66b66 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.953+0000 7f002e3a6700 1 --2- 172.21.15.44:0/636457453 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f00100a0860 0x7f00100a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.953+0000 7f000ffff700 1 -- 172.21.15.44:0/1257638001 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002000bb80 con 0x55a87c06e1a0 2023-09-10T01:25:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.953+0000 7f000ffff700 1 -- 172.21.15.44:0/1257638001 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f002000bce0 con 0x55a87c06e1a0 2023-09-10T01:25:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.953+0000 7f000ffff700 1 -- 172.21.15.44:0/1257638001 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020017650 con 0x55a87c06e1a0 2023-09-10T01:25:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.954+0000 7f003bfcfb80 1 -- 172.21.15.44:0/1257638001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c06e1a0 msgr2=0x55a87c06e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.954+0000 7f002e3a6700 1 --2- 172.21.15.44:0/636457453 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f00100a0860 0x7f00100a2d20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0020011040 tx=0x7f0020010f70 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.954+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/1257638001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c06e1a0 0x55a87c06e580 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0020005930 tx=0x7f0020004570 comp rx=0 tx=0).stop 2023-09-10T01:25:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.954+0000 7f003bfcfb80 1 -- 172.21.15.44:0/1257638001 shutdown_connections 2023-09-10T01:25:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.954+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/1257638001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068cc0 0x55a87c074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.954+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/1257638001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c06e1a0 0x55a87c06e580 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.954+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/1257638001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c068300 0x55a87c068780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.955+0000 7f003bfcfb80 1 -- 172.21.15.44:0/1257638001 >> 172.21.15.44:0/1257638001 conn(0x55a87bfdd030 msgr2=0x55a87c063560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.955+0000 7f003bfcfb80 1 -- 172.21.15.44:0/1257638001 shutdown_connections 2023-09-10T01:25:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.955+0000 7f003bfcfb80 1 -- 172.21.15.44:0/1257638001 wait complete. 2023-09-10T01:25:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.955+0000 7f003bfcfb80 1 -- start start 2023-09-10T01:25:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.955+0000 7f003bfcfb80 1 Processor -- start 2023-09-10T01:25:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.956+0000 7f003bfcfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c068300 0x55a87c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.956+0000 7f003bfcfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068cc0 0x55a87c0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.956+0000 7f003bfcfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 0x55a87c072730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.956+0000 7f003bfcfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87bf58b10 con 0x55a87c068cc0 2023-09-10T01:25:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.956+0000 7f003bfcfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87bf58810 con 0x55a87c06e1a0 2023-09-10T01:25:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.956+0000 7f003bfcfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87be3c8c0 con 0x55a87c068300 2023-09-10T01:25:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.956+0000 7f003bfcfb80 10 client.0 _ll_register_callbacks cb 0x55a87bea2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:25:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.956+0000 7f002e3a6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 0x55a87c072730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.956+0000 7f002e3a6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 0x55a87c072730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33010/0 (socket says 172.21.15.44:33010) 2023-09-10T01:25:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.956+0000 7f002e3a6700 1 -- 172.21.15.44:0/3519004107 learned_addr learned my addr 172.21.15.44:0/3519004107 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.956+0000 7f002dba5700 1 --2- 172.21.15.44:0/3519004107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c068300 0x55a87c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.957+0000 7f002e3a6700 1 -- 172.21.15.44:0/3519004107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c068300 msgr2=0x55a87c074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.957+0000 7f002e3a6700 1 --2- 172.21.15.44:0/3519004107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c068300 0x55a87c074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.957+0000 7f002e3a6700 1 -- 172.21.15.44:0/3519004107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068cc0 msgr2=0x55a87c0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.957+0000 7f002e3a6700 1 --2- 172.21.15.44:0/3519004107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068cc0 0x55a87c0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.957+0000 7f002e3a6700 1 -- 172.21.15.44:0/3519004107 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0028116540 con 0x55a87c06e1a0 2023-09-10T01:25:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.957+0000 7f002e3a6700 1 --2- 172.21.15.44:0/3519004107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 0x55a87c072730 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0020009990 tx=0x7f002002d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3dd173d40d04807 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.957+0000 7f000d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:25:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.957+0000 7f000d7fa700 1 -- 172.21.15.44:0/3519004107 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020010af0 con 0x55a87c06e1a0 2023-09-10T01:25:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.958+0000 7f000d7fa700 1 -- 172.21.15.44:0/3519004107 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(181..181 src has 1..181) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f002000be50 con 0x55a87c06e1a0 2023-09-10T01:25:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.958+0000 7f003bfcfb80 1 -- 172.21.15.44:0/3519004107 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a87bf12490 con 0x55a87c06e1a0 2023-09-10T01:25:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.958+0000 7f000d7fa700 1 -- 172.21.15.44:0/3519004107 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=152) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0020024960 con 0x55a87c06e1a0 2023-09-10T01:25:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.963+0000 7f003bfcfb80 10 client.29819 fetch_fsmap learned FSMap version 152 2023-09-10T01:25:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.963+0000 7f003bfcfb80 1 -- 172.21.15.44:0/3519004107 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=152,monmap=4+}) v3 -- 0x55a87bdcfc00 con 0x55a87c06e1a0 2023-09-10T01:25:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.968+0000 7f000d7fa700 1 -- 172.21.15.44:0/3519004107 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a87bf12490 con 0x55a87c06e1a0 2023-09-10T01:25:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.968+0000 7f000d7fa700 1 -- 172.21.15.44:0/3519004107 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 152) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f00200107b0 con 0x55a87c06e1a0 2023-09-10T01:25:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.970+0000 7f003bfcfb80 10 client.29819 fetch_fsmap finished waiting for FSMap version 152 2023-09-10T01:25:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.970+0000 7f003bfcfb80 10 client.29819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:25:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.970+0000 7f003bfcfb80 20 client.29819 populate_metadata read hostname 'smithi044' 2023-09-10T01:25:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.970+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/3519004107 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a87c081f30 0x55a87c084350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.970+0000 7f003bfcfb80 4 client.29819 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:25:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.970+0000 7f003bfcfb80 1 -- 172.21.15.44:0/3519004107 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a87bfcf840 con 0x55a87c081f30 2023-09-10T01:25:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.970+0000 7f002dba5700 1 --2- 172.21.15.44:0/3519004107 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a87c081f30 0x55a87c084350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.971+0000 7f002dba5700 1 --2- 172.21.15.44:0/3519004107 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a87c081f30 0x55a87c084350 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44744554c91794bd server_cookie=fb6223146255756b in_seq=0 out_seq=0 2023-09-10T01:25:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.974+0000 7f000d7fa700 10 client.29819 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.974+0000 7f000d7fa700 1 -- 172.21.15.44:0/3519004107 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a87bfcf840 con 0x55a87c081f30 2023-09-10T01:25:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.974+0000 7f000d7fa700 10 client.29819 handle_command_reply: tid=0 2023-09-10T01:25:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.974+0000 7f003bfcfb80 1 client.29819 shutdown 2023-09-10T01:25:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.975+0000 7f003bfcfb80 1 -- 172.21.15.44:0/3519004107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 msgr2=0x55a87c072730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.975+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/3519004107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 0x55a87c072730 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0020009990 tx=0x7f002002d000 comp rx=0 tx=0).stop 2023-09-10T01:25:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.977+0000 7f003bfcfb80 1 -- 172.21.15.44:0/3519004107 shutdown_connections 2023-09-10T01:25:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.977+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/3519004107 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a87c081f30 0x55a87c084350 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.977+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/3519004107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 0x55a87c072730 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.977+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/3519004107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c068300 0x55a87c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.977+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/3519004107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068cc0 0x55a87c0720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.977+0000 7f003bfcfb80 1 -- 172.21.15.44:0/3519004107 >> 172.21.15.44:0/3519004107 conn(0x55a87bfdd030 msgr2=0x55a87c0627a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.977+0000 7f000d7fa700 0 client.29819 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:43.989+0000 7f003bfcfb80 1 -- 172.21.15.44:0/3519004107 shutdown_connections 2023-09-10T01:25:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.001+0000 7f003bfcfb80 1 -- 172.21.15.44:0/3519004107 wait complete. 2023-09-10T01:25:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.012+0000 7f003bfcfb80 20 client.29819 trim_cache size 0 max 16384 2023-09-10T01:25:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.038+0000 7f003bfcfb80 1 Processor -- start 2023-09-10T01:25:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.038+0000 7f003bfcfb80 1 -- start start 2023-09-10T01:25:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.038+0000 7f003bfcfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068300 0x55a87c079460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.038+0000 7f003bfcfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c068cc0 0x55a87c0749d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.038+0000 7f003bfcfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 0x55a87c061f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.038+0000 7f003bfcfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87bf124c0 con 0x55a87c068300 2023-09-10T01:25:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.038+0000 7f003bfcfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87c05d8d0 con 0x55a87c06e1a0 2023-09-10T01:25:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.038+0000 7f003bfcfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87c072ac0 con 0x55a87c068cc0 2023-09-10T01:25:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.038+0000 7f002d3a4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c068cc0 0x55a87c0749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.038+0000 7f002d3a4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c068cc0 0x55a87c0749d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38160/0 (socket says 172.21.15.44:38160) 2023-09-10T01:25:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.038+0000 7f002e3a6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 0x55a87c061f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.038+0000 7f002d3a4700 1 -- 172.21.15.44:0/2116763953 learned_addr learned my addr 172.21.15.44:0/2116763953 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.039+0000 7f002d3a4700 1 -- 172.21.15.44:0/2116763953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 msgr2=0x55a87c061f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.039+0000 7f002d3a4700 1 --2- 172.21.15.44:0/2116763953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 0x55a87c061f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.039+0000 7f002d3a4700 1 -- 172.21.15.44:0/2116763953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068300 msgr2=0x55a87c079460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:25:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.039+0000 7f002d3a4700 1 --2- 172.21.15.44:0/2116763953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068300 0x55a87c079460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.039+0000 7f002d3a4700 1 -- 172.21.15.44:0/2116763953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a87be0d1f0 con 0x55a87c068cc0 2023-09-10T01:25:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.039+0000 7f002d3a4700 1 --2- 172.21.15.44:0/2116763953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c068cc0 0x55a87c0749d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f001c0c9850 tx=0x7f001c01c620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eba9778d572dbb45 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.043+0000 7f0024ff9700 1 -- 172.21.15.44:0/2116763953 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c099090 con 0x55a87c068cc0 2023-09-10T01:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.043+0000 7f0024ff9700 1 -- 172.21.15.44:0/2116763953 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f001c098eb0 con 0x55a87c068cc0 2023-09-10T01:25:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.045+0000 7f003bfcfb80 1 -- 172.21.15.44:0/2116763953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c068cc0 msgr2=0x55a87c0749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.045+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/2116763953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c068cc0 0x55a87c0749d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f001c0c9850 tx=0x7f001c01c620 comp rx=0 tx=0).stop 2023-09-10T01:25:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.048+0000 7f003bfcfb80 1 -- 172.21.15.44:0/2116763953 shutdown_connections 2023-09-10T01:25:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.048+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/2116763953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 0x55a87c061f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.048+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/2116763953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068300 0x55a87c079460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.048+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/2116763953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c068cc0 0x55a87c0749d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.048+0000 7f003bfcfb80 1 -- 172.21.15.44:0/2116763953 >> 172.21.15.44:0/2116763953 conn(0x55a87bfdd030 msgr2=0x55a87c06d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.045+0000 7f0024ff9700 1 -- 172.21.15.44:0/2116763953 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c09d380 con 0x55a87c068cc0 2023-09-10T01:25:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.049+0000 7f003bfcfb80 1 -- 172.21.15.44:0/2116763953 shutdown_connections 2023-09-10T01:25:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.049+0000 7f003bfcfb80 1 -- 172.21.15.44:0/2116763953 wait complete. 2023-09-10T01:25:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.049+0000 7f003bfcfb80 1 -- start start 2023-09-10T01:25:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.050+0000 7f003bfcfb80 1 Processor -- start 2023-09-10T01:25:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.050+0000 7f003bfcfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068300 0x55a87c0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.050+0000 7f003bfcfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 0x55a87c0ca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.050+0000 7f003bfcfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c0d2d80 0x55a87c0d5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.050+0000 7f003bfcfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87c0cab90 con 0x55a87c068300 2023-09-10T01:25:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.050+0000 7f003bfcfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87c0d56b0 con 0x55a87c06e1a0 2023-09-10T01:25:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.050+0000 7f003bfcfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87c0d5870 con 0x55a87c0d2d80 2023-09-10T01:25:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.050+0000 7f003bfcfb80 10 client.0 _ll_register_callbacks cb 0x7f001c010a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:25:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.050+0000 7f002d3a4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 0x55a87c0ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.050+0000 7f002d3a4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 0x55a87c0ca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33026/0 (socket says 172.21.15.44:33026) 2023-09-10T01:25:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.050+0000 7f002d3a4700 1 -- 172.21.15.44:0/967615774 learned_addr learned my addr 172.21.15.44:0/967615774 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.051+0000 7f002d3a4700 1 -- 172.21.15.44:0/967615774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c0d2d80 msgr2=0x55a87c0d5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.051+0000 7f002d3a4700 1 --2- 172.21.15.44:0/967615774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c0d2d80 0x55a87c0d5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.051+0000 7f002d3a4700 1 -- 172.21.15.44:0/967615774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068300 msgr2=0x55a87c0c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.051+0000 7f002d3a4700 1 --2- 172.21.15.44:0/967615774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068300 0x55a87c0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.051+0000 7f002d3a4700 1 -- 172.21.15.44:0/967615774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c095040 con 0x55a87c06e1a0 2023-09-10T01:25:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.051+0000 7f002d3a4700 1 --2- 172.21.15.44:0/967615774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 0x55a87c0ca440 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f001c0c98a0 tx=0x7f001c099090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffd35abf9a8c540d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.051+0000 7f000d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:25:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.051+0000 7f000d7fa700 1 -- 172.21.15.44:0/967615774 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c01d210 con 0x55a87c06e1a0 2023-09-10T01:25:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.051+0000 7f000d7fa700 1 -- 172.21.15.44:0/967615774 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(181..181 src has 1..181) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f001c099c50 con 0x55a87c06e1a0 2023-09-10T01:25:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.051+0000 7f003bfcfb80 1 -- 172.21.15.44:0/967615774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f002000bce0 con 0x55a87c06e1a0 2023-09-10T01:25:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.051+0000 7f000d7fa700 1 -- 172.21.15.44:0/967615774 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=152) v2 ==== 24+0+0 (secure 0 0 0) 0x7f001c0a15c0 con 0x55a87c06e1a0 2023-09-10T01:25:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.051+0000 7f003bfcfb80 10 client.29825 fetch_fsmap learned FSMap version 152 2023-09-10T01:25:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.051+0000 7f003bfcfb80 1 -- 172.21.15.44:0/967615774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=152,monmap=4+}) v3 -- 0x55a87bde5250 con 0x55a87c06e1a0 2023-09-10T01:25:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.052+0000 7f000d7fa700 1 -- 172.21.15.44:0/967615774 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f002000bce0 con 0x55a87c06e1a0 2023-09-10T01:25:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.052+0000 7f000d7fa700 1 -- 172.21.15.44:0/967615774 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 152) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f001c0cc130 con 0x55a87c06e1a0 2023-09-10T01:25:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.052+0000 7f003bfcfb80 10 client.29825 fetch_fsmap finished waiting for FSMap version 152 2023-09-10T01:25:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.052+0000 7f003bfcfb80 10 client.29825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:25:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.052+0000 7f003bfcfb80 20 client.29825 populate_metadata read hostname 'smithi044' 2023-09-10T01:25:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.052+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/967615774 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a87c0d6e40 0x55a87c0d9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.052+0000 7f003bfcfb80 4 client.29825 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:25:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.052+0000 7f003bfcfb80 1 -- 172.21.15.44:0/967615774 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a87bfb38c0 con 0x55a87c0d6e40 2023-09-10T01:25:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.052+0000 7f002dba5700 1 --2- 172.21.15.44:0/967615774 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a87c0d6e40 0x55a87c0d9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.054+0000 7f000d7fa700 10 client.29825 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.054+0000 7f002dba5700 1 --2- 172.21.15.44:0/967615774 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a87c0d6e40 0x55a87c0d9260 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b3d2f872f39521 server_cookie=34d52fd05d183d43 in_seq=0 out_seq=0 2023-09-10T01:25:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.095+0000 7f000d7fa700 1 -- 172.21.15.44:0/967615774 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a87bfb38c0 con 0x55a87c0d6e40 2023-09-10T01:25:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.095+0000 7f000d7fa700 10 client.29825 handle_command_reply: tid=0 2023-09-10T01:25:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.097+0000 7f003bfcfb80 1 client.29825 shutdown 2023-09-10T01:25:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.105+0000 7f003bfcfb80 1 -- 172.21.15.44:0/967615774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 msgr2=0x55a87c0ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.105+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/967615774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 0x55a87c0ca440 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f001c0c98a0 tx=0x7f001c099090 comp rx=0 tx=0).stop 2023-09-10T01:25:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.105+0000 7f003bfcfb80 1 -- 172.21.15.44:0/967615774 shutdown_connections 2023-09-10T01:25:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.105+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/967615774 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a87c0d6e40 0x55a87c0d9260 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.105+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/967615774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a87c0d2d80 0x55a87c0d5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.105+0000 7f000d7fa700 0 client.29825 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.105+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/967615774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a87c068300 0x55a87c0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.105+0000 7f003bfcfb80 1 --2- 172.21.15.44:0/967615774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a87c06e1a0 0x55a87c0ca440 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.105+0000 7f003bfcfb80 1 -- 172.21.15.44:0/967615774 >> 172.21.15.44:0/967615774 conn(0x55a87bfdd030 msgr2=0x55a87c061f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.117+0000 7f003bfcfb80 1 -- 172.21.15.44:0/967615774 shutdown_connections 2023-09-10T01:25:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.117+0000 7f003bfcfb80 1 -- 172.21.15.44:0/967615774 wait complete. 2023-09-10T01:25:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.130+0000 7f003bfcfb80 20 client.29825 trim_cache size 0 max 16384 2023-09-10T01:25:44.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:25:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:25:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:25:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:25:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.139+0000 7f002fe09700 1 -- 172.21.15.44:0/636457453 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f00100a0860 msgr2=0x7f00100a2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.139+0000 7f002fe09700 1 --2- 172.21.15.44:0/636457453 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f00100a0860 0x7f00100a2d20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0020011040 tx=0x7f0020010f70 comp rx=0 tx=0).stop 2023-09-10T01:25:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.139+0000 7f002fe09700 1 -- 172.21.15.44:0/636457453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028107f20 msgr2=0x7f002810c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.139+0000 7f002fe09700 1 --2- 172.21.15.44:0/636457453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028107f20 0x7f002810c9b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f001c00e9a0 tx=0x7f001c00ecb0 comp rx=0 tx=0).stop 2023-09-10T01:25:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.140+0000 7f002fe09700 1 -- 172.21.15.44:0/636457453 shutdown_connections 2023-09-10T01:25:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.140+0000 7f002fe09700 1 --2- 172.21.15.44:0/636457453 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f00100a0860 0x7f00100a2d20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.140+0000 7f002fe09700 1 --2- 172.21.15.44:0/636457453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0028113750 0x7f00281111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.140+0000 7f002fe09700 1 --2- 172.21.15.44:0/636457453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0028107600 0x7f002810c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.140+0000 7f002fe09700 1 --2- 172.21.15.44:0/636457453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028107f20 0x7f002810c9b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.140+0000 7f002fe09700 1 -- 172.21.15.44:0/636457453 >> 172.21.15.44:0/636457453 conn(0x7f002806c6c0 msgr2=0x7f002806efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.140+0000 7f002fe09700 1 -- 172.21.15.44:0/636457453 shutdown_connections 2023-09-10T01:25:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:44.140+0000 7f002fe09700 1 -- 172.21.15.44:0/636457453 wait complete. 2023-09-10T01:25:44.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:44 smithi131 ceph-mon[103713]: pgmap v1460: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.0 MiB/s wr, 752 op/s 2023-09-10T01:25:44.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:44 smithi044 ceph-mon[98676]: pgmap v1460: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.0 MiB/s wr, 752 op/s 2023-09-10T01:25:44.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:44 smithi099 ceph-mon[102692]: pgmap v1460: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.0 MiB/s wr, 752 op/s 2023-09-10T01:25:45.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:45 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:25:45.612 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:45 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:25:45.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:45 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:25:46.183 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 280.0 s, 265.5 tps, lat 376.066 ms stddev 134.694 2023-09-10T01:25:46.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:46 smithi099 ceph-mon[102692]: pgmap v1461: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.7 MiB/s wr, 617 op/s 2023-09-10T01:25:46.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:46 smithi131 ceph-mon[103713]: pgmap v1461: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.7 MiB/s wr, 617 op/s 2023-09-10T01:25:46.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:46 smithi044 ceph-mon[98676]: pgmap v1461: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.7 MiB/s wr, 617 op/s 2023-09-10T01:25:48.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:48 smithi099 ceph-mon[102692]: pgmap v1462: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 939 op/s 2023-09-10T01:25:48.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:48 smithi131 ceph-mon[103713]: pgmap v1462: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 939 op/s 2023-09-10T01:25:48.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:48 smithi044 ceph-mon[98676]: pgmap v1462: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 939 op/s 2023-09-10T01:25:50.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:50 smithi131 ceph-mon[103713]: pgmap v1463: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.4 MiB/s wr, 684 op/s 2023-09-10T01:25:50.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:50 smithi044 ceph-mon[98676]: pgmap v1463: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.4 MiB/s wr, 684 op/s 2023-09-10T01:25:50.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:50 smithi099 ceph-mon[102692]: pgmap v1463: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.4 MiB/s wr, 684 op/s 2023-09-10T01:25:51.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 285.0 s, 174.3 tps, lat 567.070 ms stddev 374.888 2023-09-10T01:25:52.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:52 smithi131 ceph-mon[103713]: pgmap v1464: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 973 op/s 2023-09-10T01:25:52.561 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:52 smithi044 ceph-mon[98676]: pgmap v1464: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 973 op/s 2023-09-10T01:25:52.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:52 smithi099 ceph-mon[102692]: pgmap v1464: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 973 op/s 2023-09-10T01:25:54.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:54 smithi131 ceph-mon[103713]: pgmap v1465: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.7 MiB/s wr, 677 op/s 2023-09-10T01:25:54.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:54 smithi044 ceph-mon[98676]: pgmap v1465: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.7 MiB/s wr, 677 op/s 2023-09-10T01:25:54.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:54 smithi099 ceph-mon[102692]: pgmap v1465: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.7 MiB/s wr, 677 op/s 2023-09-10T01:25:56.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 290.0 s, 329.4 tps, lat 308.898 ms stddev 114.649 2023-09-10T01:25:56.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:56 smithi131 ceph-mon[103713]: pgmap v1466: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.7 MiB/s wr, 677 op/s 2023-09-10T01:25:56.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:56 smithi044 ceph-mon[98676]: pgmap v1466: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.7 MiB/s wr, 677 op/s 2023-09-10T01:25:56.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:56 smithi099 ceph-mon[102692]: pgmap v1466: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.7 MiB/s wr, 677 op/s 2023-09-10T01:25:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.355+0000 7f419ffa4700 1 -- 172.21.15.44:0/703777264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4198119de0 msgr2=0x7f419811c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:57.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.355+0000 7f419ffa4700 1 --2- 172.21.15.44:0/703777264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4198119de0 0x7f419811c1d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4190009f90 tx=0x7f419000b330 comp rx=0 tx=0).stop 2023-09-10T01:25:57.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.355+0000 7f419ffa4700 1 -- 172.21.15.44:0/703777264 shutdown_connections 2023-09-10T01:25:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.355+0000 7f419ffa4700 1 --2- 172.21.15.44:0/703777264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4198119de0 0x7f419811c1d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.355+0000 7f419ffa4700 1 --2- 172.21.15.44:0/703777264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f419810ed50 0x7f4198107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.355+0000 7f419ffa4700 1 --2- 172.21.15.44:0/703777264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419810f1c0 0x7f4198107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.355+0000 7f419ffa4700 1 -- 172.21.15.44:0/703777264 >> 172.21.15.44:0/703777264 conn(0x7f419806c6c0 msgr2=0x7f419806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:57.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.355+0000 7f419ffa4700 1 -- 172.21.15.44:0/703777264 shutdown_connections 2023-09-10T01:25:57.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.355+0000 7f419ffa4700 1 -- 172.21.15.44:0/703777264 wait complete. 2023-09-10T01:25:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.356+0000 7f419ffa4700 1 Processor -- start 2023-09-10T01:25:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.356+0000 7f419ffa4700 1 -- start start 2023-09-10T01:25:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.356+0000 7f419ffa4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f419810ed50 0x7f41981a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.356+0000 7f419ffa4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f419810f1c0 0x7f41981a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.356+0000 7f419ffa4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4198119de0 0x7f41981a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.356+0000 7f419ffa4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41981aa630 con 0x7f419810f1c0 2023-09-10T01:25:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.356+0000 7f419ffa4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41981aa7a0 con 0x7f4198119de0 2023-09-10T01:25:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.356+0000 7f419ffa4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41981aa960 con 0x7f419810ed50 2023-09-10T01:25:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.357+0000 7f419d53f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f419810f1c0 0x7f41981a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.357+0000 7f419d53f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f419810f1c0 0x7f41981a5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59528/0 (socket says 172.21.15.44:59528) 2023-09-10T01:25:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.357+0000 7f419d53f700 1 -- 172.21.15.44:0/3259474444 learned_addr learned my addr 172.21.15.44:0/3259474444 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.357+0000 7f419e541700 1 --2- 172.21.15.44:0/3259474444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4198119de0 0x7f41981a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.357+0000 7f419e541700 1 -- 172.21.15.44:0/3259474444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f419810ed50 msgr2=0x7f41981a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.357+0000 7f419dd40700 1 --2- 172.21.15.44:0/3259474444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f419810ed50 0x7f41981a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.357+0000 7f419e541700 1 --2- 172.21.15.44:0/3259474444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f419810ed50 0x7f41981a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.357+0000 7f419e541700 1 -- 172.21.15.44:0/3259474444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f419810f1c0 msgr2=0x7f41981a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.357+0000 7f419e541700 1 --2- 172.21.15.44:0/3259474444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f419810f1c0 0x7f41981a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.357+0000 7f419e541700 1 -- 172.21.15.44:0/3259474444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4188009520 con 0x7f4198119de0 2023-09-10T01:25:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.357+0000 7f419d53f700 1 --2- 172.21.15.44:0/3259474444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f419810f1c0 0x7f41981a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:25:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.357+0000 7f419dd40700 1 --2- 172.21.15.44:0/3259474444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f419810ed50 0x7f41981a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:25:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.357+0000 7f419e541700 1 --2- 172.21.15.44:0/3259474444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4198119de0 0x7f41981a9ff0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f419000d010 tx=0x7f4190007730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.358+0000 7f418effd700 1 -- 172.21.15.44:0/3259474444 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419000e040 con 0x7f4198119de0 2023-09-10T01:25:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.358+0000 7f418effd700 1 -- 172.21.15.44:0/3259474444 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4190004e60 con 0x7f4198119de0 2023-09-10T01:25:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.358+0000 7f419ffa4700 1 -- 172.21.15.44:0/3259474444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4190009c40 con 0x7f4198119de0 2023-09-10T01:25:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.358+0000 7f419ffa4700 1 -- 172.21.15.44:0/3259474444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41981b22a0 con 0x7f4198119de0 2023-09-10T01:25:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.358+0000 7f418effd700 1 -- 172.21.15.44:0/3259474444 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419001b6b0 con 0x7f4198119de0 2023-09-10T01:25:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.361+0000 7f418effd700 1 -- 172.21.15.44:0/3259474444 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f419001b810 con 0x7f4198119de0 2023-09-10T01:25:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.361+0000 7f418effd700 1 --2- 172.21.15.44:0/3259474444 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f41840a0830 0x7f41840a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.361+0000 7f418effd700 1 -- 172.21.15.44:0/3259474444 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f41900ce500 con 0x7f4198119de0 2023-09-10T01:25:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.362+0000 7f41ac16ab80 1 Processor -- start 2023-09-10T01:25:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.362+0000 7f41ac16ab80 1 -- start start 2023-09-10T01:25:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.362+0000 7f419dd40700 1 --2- 172.21.15.44:0/3259474444 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f41840a0830 0x7f41840a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.362+0000 7f41ac16ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f799a1a0 0x5567f799a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.362+0000 7f41ac16ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f7994300 0x5567f7994780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.362+0000 7f41ac16ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994cc0 0x5567f79a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.362+0000 7f41ac16ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f78ed6d0 con 0x5567f7994300 2023-09-10T01:25:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.362+0000 7f41ac16ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f78eaff0 con 0x5567f799a1a0 2023-09-10T01:25:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.362+0000 7f41ac16ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f78e68e0 con 0x5567f7994cc0 2023-09-10T01:25:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.363+0000 7f419dd40700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f799a1a0 0x5567f799a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.363+0000 7f419e541700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994cc0 0x5567f79a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.363+0000 7f419dd40700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f799a1a0 0x5567f799a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33046/0 (socket says 172.21.15.44:33046) 2023-09-10T01:25:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.363+0000 7f419dd40700 1 -- 172.21.15.44:0/489715984 learned_addr learned my addr 172.21.15.44:0/489715984 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.363+0000 7f419dd40700 1 --2- 172.21.15.44:0/3259474444 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f41840a0830 0x7f41840a2cf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4194005b40 tx=0x7f4194012670 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.363+0000 7f419dd40700 1 -- 172.21.15.44:0/489715984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994cc0 msgr2=0x5567f79a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.363+0000 7f419dd40700 1 --2- 172.21.15.44:0/489715984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994cc0 0x5567f79a0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.363+0000 7f419dd40700 1 -- 172.21.15.44:0/489715984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f7994300 msgr2=0x5567f7994780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.363+0000 7f419dd40700 1 --2- 172.21.15.44:0/489715984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f7994300 0x5567f7994780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.363+0000 7f419dd40700 1 -- 172.21.15.44:0/489715984 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567f7712d80 con 0x5567f799a1a0 2023-09-10T01:25:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.363+0000 7f419dd40700 1 --2- 172.21.15.44:0/489715984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f799a1a0 0x5567f799a580 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4194013500 tx=0x7f4194019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8879f222da29fd01 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.363+0000 7f4183fff700 1 -- 172.21.15.44:0/489715984 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419401d450 con 0x5567f799a1a0 2023-09-10T01:25:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.364+0000 7f4183fff700 1 -- 172.21.15.44:0/489715984 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f419401da40 con 0x5567f799a1a0 2023-09-10T01:25:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.364+0000 7f4183fff700 1 -- 172.21.15.44:0/489715984 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419400f960 con 0x5567f799a1a0 2023-09-10T01:25:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.364+0000 7f41ac16ab80 1 -- 172.21.15.44:0/489715984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f799a1a0 msgr2=0x5567f799a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.364+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/489715984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f799a1a0 0x5567f799a580 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4194013500 tx=0x7f4194019000 comp rx=0 tx=0).stop 2023-09-10T01:25:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.365+0000 7f41ac16ab80 1 -- 172.21.15.44:0/489715984 shutdown_connections 2023-09-10T01:25:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.365+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/489715984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994cc0 0x5567f79a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.365+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/489715984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f799a1a0 0x5567f799a580 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.365+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/489715984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f7994300 0x5567f7994780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.365+0000 7f41ac16ab80 1 -- 172.21.15.44:0/489715984 >> 172.21.15.44:0/489715984 conn(0x5567f7909030 msgr2=0x5567f798f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.365+0000 7f41ac16ab80 1 -- 172.21.15.44:0/489715984 shutdown_connections 2023-09-10T01:25:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.365+0000 7f41ac16ab80 1 -- 172.21.15.44:0/489715984 wait complete. 2023-09-10T01:25:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.365+0000 7f41ac16ab80 1 -- start start 2023-09-10T01:25:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.365+0000 7f41ac16ab80 1 Processor -- start 2023-09-10T01:25:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.366+0000 7f41ac16ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994300 0x5567f79a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.366+0000 7f41ac16ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f7994cc0 0x5567f799df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.366+0000 7f41ac16ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f799a1a0 0x5567f799e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.366+0000 7f41ac16ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f7884b10 con 0x5567f7994cc0 2023-09-10T01:25:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.366+0000 7f41ac16ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f7884810 con 0x5567f799a1a0 2023-09-10T01:25:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.366+0000 7f41ac16ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f77688c0 con 0x5567f7994300 2023-09-10T01:25:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.366+0000 7f41ac16ab80 10 client.0 _ll_register_callbacks cb 0x5567f77cea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:25:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.366+0000 7f419e541700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f799a1a0 0x5567f799e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.366+0000 7f419e541700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f799a1a0 0x5567f799e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33050/0 (socket says 172.21.15.44:33050) 2023-09-10T01:25:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.366+0000 7f419e541700 1 -- 172.21.15.44:0/2782930996 learned_addr learned my addr 172.21.15.44:0/2782930996 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.366+0000 7f419e541700 1 -- 172.21.15.44:0/2782930996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994300 msgr2=0x5567f79a0d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:25:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.366+0000 7f419e541700 1 --2- 172.21.15.44:0/2782930996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994300 0x5567f79a0d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.366+0000 7f419e541700 1 -- 172.21.15.44:0/2782930996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f7994cc0 msgr2=0x5567f799df90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:25:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.366+0000 7f419e541700 1 --2- 172.21.15.44:0/2782930996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f7994cc0 0x5567f799df90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.366+0000 7f419e541700 1 -- 172.21.15.44:0/2782930996 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4194012db0 con 0x5567f799a1a0 2023-09-10T01:25:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.367+0000 7f419e541700 1 --2- 172.21.15.44:0/2782930996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f799a1a0 0x5567f799e4d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f41900980c0 tx=0x7f419009b0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77bf1ba968677e92 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.367+0000 7f41817fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:25:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.367+0000 7f41817fa700 1 -- 172.21.15.44:0/2782930996 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41900862f0 con 0x5567f799a1a0 2023-09-10T01:25:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.367+0000 7f41817fa700 1 -- 172.21.15.44:0/2782930996 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(181..181 src has 1..181) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f419009f480 con 0x5567f799a1a0 2023-09-10T01:25:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.367+0000 7f41ac16ab80 1 -- 172.21.15.44:0/2782930996 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567f783e490 con 0x5567f799a1a0 2023-09-10T01:25:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.367+0000 7f41817fa700 1 -- 172.21.15.44:0/2782930996 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=153) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41900a3630 con 0x5567f799a1a0 2023-09-10T01:25:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.367+0000 7f41ac16ab80 10 client.29837 fetch_fsmap learned FSMap version 153 2023-09-10T01:25:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.367+0000 7f41ac16ab80 1 -- 172.21.15.44:0/2782930996 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=153,monmap=4+}) v3 -- 0x5567f76fbc00 con 0x5567f799a1a0 2023-09-10T01:25:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.367+0000 7f41817fa700 1 -- 172.21.15.44:0/2782930996 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5567f783e490 con 0x5567f799a1a0 2023-09-10T01:25:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.367+0000 7f41817fa700 1 -- 172.21.15.44:0/2782930996 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 153) v1 ==== 7367+0+0 (secure 0 0 0) 0x7f419009ab40 con 0x5567f799a1a0 2023-09-10T01:25:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.367+0000 7f41ac16ab80 10 client.29837 fetch_fsmap finished waiting for FSMap version 153 2023-09-10T01:25:57.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.367+0000 7f41ac16ab80 10 client.29837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:25:57.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.367+0000 7f41ac16ab80 20 client.29837 populate_metadata read hostname 'smithi044' 2023-09-10T01:25:57.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.367+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/2782930996 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5567f79d0280 0x5567f79d2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.367+0000 7f41ac16ab80 4 client.29837 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:25:57.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.367+0000 7f41ac16ab80 1 -- 172.21.15.44:0/2782930996 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567f78fb840 con 0x5567f79d0280 2023-09-10T01:25:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.370+0000 7f419dd40700 1 --2- 172.21.15.44:0/2782930996 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5567f79d0280 0x5567f79d2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.370+0000 7f41817fa700 10 client.29837 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.370+0000 7f419dd40700 1 --2- 172.21.15.44:0/2782930996 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5567f79d0280 0x5567f79d2720 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f401ec4104775a37 server_cookie=ab847e8fe11302c0 in_seq=0 out_seq=0 2023-09-10T01:25:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.371+0000 7f41817fa700 1 -- 172.21.15.44:0/2782930996 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5567f78fb840 con 0x5567f79d0280 2023-09-10T01:25:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.371+0000 7f41817fa700 10 client.29837 handle_command_reply: tid=0 2023-09-10T01:25:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.371+0000 7f41ac16ab80 1 client.29837 shutdown 2023-09-10T01:25:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.371+0000 7f41ac16ab80 1 -- 172.21.15.44:0/2782930996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f799a1a0 msgr2=0x5567f799e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.371+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/2782930996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f799a1a0 0x5567f799e4d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f41900980c0 tx=0x7f419009b0e0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.371+0000 7f41ac16ab80 1 -- 172.21.15.44:0/2782930996 shutdown_connections 2023-09-10T01:25:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.371+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/2782930996 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5567f79d0280 0x5567f79d2720 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.371+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/2782930996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f799a1a0 0x5567f799e4d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.371+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/2782930996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994300 0x5567f79a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.371+0000 7f41817fa700 0 client.29837 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.371+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/2782930996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f7994cc0 0x5567f799df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.371+0000 7f41ac16ab80 1 -- 172.21.15.44:0/2782930996 >> 172.21.15.44:0/2782930996 conn(0x5567f7909030 msgr2=0x5567f798e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.372+0000 7f41ac16ab80 1 -- 172.21.15.44:0/2782930996 shutdown_connections 2023-09-10T01:25:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.372+0000 7f41ac16ab80 1 -- 172.21.15.44:0/2782930996 wait complete. 2023-09-10T01:25:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.372+0000 7f41ac16ab80 20 client.29837 trim_cache size 0 max 16384 2023-09-10T01:25:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.396+0000 7f41ac16ab80 1 Processor -- start 2023-09-10T01:25:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.396+0000 7f41ac16ab80 1 -- start start 2023-09-10T01:25:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.396+0000 7f41ac16ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994300 0x5567f79a5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.396+0000 7f41ac16ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f7994cc0 0x5567f7993a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.396+0000 7f41ac16ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f799a1a0 0x5567f79a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.396+0000 7f41ac16ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f799ead0 con 0x5567f799a1a0 2023-09-10T01:25:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.396+0000 7f41ac16ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f7990b80 con 0x5567f7994cc0 2023-09-10T01:25:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.396+0000 7f41ac16ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f798fad0 con 0x5567f7994300 2023-09-10T01:25:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.397+0000 7f419e541700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f799a1a0 0x5567f79a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.397+0000 7f419e541700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f799a1a0 0x5567f79a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59548/0 (socket says 172.21.15.44:59548) 2023-09-10T01:25:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.397+0000 7f419e541700 1 -- 172.21.15.44:0/693790772 learned_addr learned my addr 172.21.15.44:0/693790772 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.397+0000 7f419dd40700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994300 0x5567f79a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.397+0000 7f419e541700 1 -- 172.21.15.44:0/693790772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994300 msgr2=0x5567f79a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.397+0000 7f419e541700 1 --2- 172.21.15.44:0/693790772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994300 0x5567f79a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.397+0000 7f419e541700 1 -- 172.21.15.44:0/693790772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f7994cc0 msgr2=0x5567f7993a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:25:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.397+0000 7f419e541700 1 --2- 172.21.15.44:0/693790772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f7994cc0 0x5567f7993a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.397+0000 7f419e541700 1 -- 172.21.15.44:0/693790772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567f77391f0 con 0x5567f799a1a0 2023-09-10T01:25:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.397+0000 7f419dd40700 1 --2- 172.21.15.44:0/693790772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994300 0x5567f79a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:25:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.397+0000 7f419e541700 1 --2- 172.21.15.44:0/693790772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f799a1a0 0x5567f79a08b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f419009b350 tx=0x7f4190097350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bd2c7b8e74c14b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.397+0000 7f418cff9700 1 -- 172.21.15.44:0/693790772 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190097990 con 0x5567f799a1a0 2023-09-10T01:25:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.397+0000 7f418cff9700 1 -- 172.21.15.44:0/693790772 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41900862f0 con 0x5567f799a1a0 2023-09-10T01:25:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.397+0000 7f418cff9700 1 -- 172.21.15.44:0/693790772 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41900ce170 con 0x5567f799a1a0 2023-09-10T01:25:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.398+0000 7f41ac16ab80 1 -- 172.21.15.44:0/693790772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f799a1a0 msgr2=0x5567f79a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.398+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/693790772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f799a1a0 0x5567f79a08b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f419009b350 tx=0x7f4190097350 comp rx=0 tx=0).stop 2023-09-10T01:25:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.399+0000 7f41ac16ab80 1 -- 172.21.15.44:0/693790772 shutdown_connections 2023-09-10T01:25:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.399+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/693790772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f799a1a0 0x5567f79a08b0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.399+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/693790772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994300 0x5567f79a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.399+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/693790772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f7994cc0 0x5567f7993a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.399+0000 7f41ac16ab80 1 -- 172.21.15.44:0/693790772 >> 172.21.15.44:0/693790772 conn(0x5567f7909030 msgr2=0x5567f79932f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.404+0000 7f41ac16ab80 1 -- 172.21.15.44:0/693790772 shutdown_connections 2023-09-10T01:25:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.404+0000 7f41ac16ab80 1 -- 172.21.15.44:0/693790772 wait complete. 2023-09-10T01:25:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.404+0000 7f41ac16ab80 1 -- start start 2023-09-10T01:25:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.404+0000 7f41ac16ab80 1 Processor -- start 2023-09-10T01:25:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.405+0000 7f41ac16ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994300 0x5567f7a182f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.405+0000 7f41ac16ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f799a1a0 0x5567f7a18830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.405+0000 7f41ac16ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f7a1c4e0 0x5567f7a1c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.405+0000 7f41ac16ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f7a1cea0 con 0x5567f799a1a0 2023-09-10T01:25:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.405+0000 7f41ac16ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f7a1d010 con 0x5567f7a1c4e0 2023-09-10T01:25:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.405+0000 7f41ac16ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f7a1d150 con 0x5567f7994300 2023-09-10T01:25:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.405+0000 7f41ac16ab80 10 client.0 _ll_register_callbacks cb 0x7f419009b510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:25:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.405+0000 7f419e541700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f7a1c4e0 0x5567f7a1c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.405+0000 7f419e541700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f7a1c4e0 0x5567f7a1c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33060/0 (socket says 172.21.15.44:33060) 2023-09-10T01:25:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.405+0000 7f419e541700 1 -- 172.21.15.44:0/2795573158 learned_addr learned my addr 172.21.15.44:0/2795573158 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:25:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.406+0000 7f419d53f700 1 --2- 172.21.15.44:0/2795573158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f799a1a0 0x5567f7a18830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.406+0000 7f419dd40700 1 --2- 172.21.15.44:0/2795573158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994300 0x5567f7a182f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.406+0000 7f419e541700 1 -- 172.21.15.44:0/2795573158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994300 msgr2=0x5567f7a182f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.406+0000 7f419e541700 1 --2- 172.21.15.44:0/2795573158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994300 0x5567f7a182f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.406+0000 7f419e541700 1 -- 172.21.15.44:0/2795573158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f799a1a0 msgr2=0x5567f7a18830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:25:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.406+0000 7f419e541700 1 --2- 172.21.15.44:0/2795573158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f799a1a0 0x5567f7a18830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.406+0000 7f419e541700 1 -- 172.21.15.44:0/2795573158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567f7712d80 con 0x5567f7a1c4e0 2023-09-10T01:25:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.406+0000 7f419d53f700 1 --2- 172.21.15.44:0/2795573158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f799a1a0 0x5567f7a18830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:25:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.406+0000 7f419e541700 1 --2- 172.21.15.44:0/2795573158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f7a1c4e0 0x5567f7a1c960 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4190009f90 tx=0x7f4190009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=541661a64271ca7b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:25:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.406+0000 7f41817fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:25:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.407+0000 7f41817fa700 1 -- 172.21.15.44:0/2795573158 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41900ce370 con 0x5567f7a1c4e0 2023-09-10T01:25:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.407+0000 7f41817fa700 1 -- 172.21.15.44:0/2795573158 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(181..181 src has 1..181) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f419009f480 con 0x5567f7a1c4e0 2023-09-10T01:25:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.407+0000 7f41ac16ab80 1 -- 172.21.15.44:0/2795573158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f419401da40 con 0x5567f7a1c4e0 2023-09-10T01:25:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.407+0000 7f41817fa700 1 -- 172.21.15.44:0/2795573158 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=153) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41900a31a0 con 0x5567f7a1c4e0 2023-09-10T01:25:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.407+0000 7f41ac16ab80 10 client.29840 fetch_fsmap learned FSMap version 153 2023-09-10T01:25:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.407+0000 7f41ac16ab80 1 -- 172.21.15.44:0/2795573158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=153,monmap=4+}) v3 -- 0x5567f76fbc00 con 0x5567f7a1c4e0 2023-09-10T01:25:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.409+0000 7f41817fa700 1 -- 172.21.15.44:0/2795573158 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f419401da40 con 0x5567f7a1c4e0 2023-09-10T01:25:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.409+0000 7f41817fa700 1 -- 172.21.15.44:0/2795573158 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 153) v1 ==== 7367+0+0 (secure 0 0 0) 0x7f419001ac30 con 0x5567f7a1c4e0 2023-09-10T01:25:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.409+0000 7f41ac16ab80 10 client.29840 fetch_fsmap finished waiting for FSMap version 153 2023-09-10T01:25:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.409+0000 7f41ac16ab80 10 client.29840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:25:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.409+0000 7f41ac16ab80 20 client.29840 populate_metadata read hostname 'smithi044' 2023-09-10T01:25:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.409+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/2795573158 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5567f7a36600 0x5567f7a389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:25:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.409+0000 7f41ac16ab80 4 client.29840 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:25:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.409+0000 7f41ac16ab80 1 -- 172.21.15.44:0/2795573158 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5567f78df8c0 con 0x5567f7a36600 2023-09-10T01:25:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.409+0000 7f419dd40700 1 --2- 172.21.15.44:0/2795573158 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5567f7a36600 0x5567f7a389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:25:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.410+0000 7f41817fa700 10 client.29840 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.410+0000 7f419dd40700 1 --2- 172.21.15.44:0/2795573158 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5567f7a36600 0x5567f7a389f0 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aed51afb9670a26 server_cookie=a5c03968c6f3c8ea in_seq=0 out_seq=0 2023-09-10T01:25:57.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.460+0000 7f41817fa700 1 -- 172.21.15.44:0/2795573158 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5567f78df8c0 con 0x5567f7a36600 2023-09-10T01:25:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.460+0000 7f41817fa700 10 client.29840 handle_command_reply: tid=0 2023-09-10T01:25:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.460+0000 7f41ac16ab80 1 client.29840 shutdown 2023-09-10T01:25:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.463+0000 7f41ac16ab80 1 -- 172.21.15.44:0/2795573158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f7a1c4e0 msgr2=0x5567f7a1c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.463+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/2795573158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f7a1c4e0 0x5567f7a1c960 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4190009f90 tx=0x7f4190009550 comp rx=0 tx=0).stop 2023-09-10T01:25:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.467+0000 7f41ac16ab80 1 -- 172.21.15.44:0/2795573158 shutdown_connections 2023-09-10T01:25:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.467+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/2795573158 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5567f7a36600 0x5567f7a389f0 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.467+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/2795573158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567f7a1c4e0 0x5567f7a1c960 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.467+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/2795573158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f7994300 0x5567f7a182f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.467+0000 7f41ac16ab80 1 --2- 172.21.15.44:0/2795573158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567f799a1a0 0x5567f7a18830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.467+0000 7f41ac16ab80 1 -- 172.21.15.44:0/2795573158 >> 172.21.15.44:0/2795573158 conn(0x5567f7909030 msgr2=0x5567f7993b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.468+0000 7f41817fa700 0 client.29840 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:25:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.468+0000 7f41ac16ab80 1 -- 172.21.15.44:0/2795573158 shutdown_connections 2023-09-10T01:25:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.468+0000 7f41ac16ab80 1 -- 172.21.15.44:0/2795573158 wait complete. 2023-09-10T01:25:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.482+0000 7f41ac16ab80 20 client.29840 trim_cache size 0 max 16384 2023-09-10T01:25:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:25:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:25:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:25:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:25:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.496+0000 7f419ffa4700 1 -- 172.21.15.44:0/3259474444 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f41840a0830 msgr2=0x7f41840a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.496+0000 7f419ffa4700 1 --2- 172.21.15.44:0/3259474444 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f41840a0830 0x7f41840a2cf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4194005b40 tx=0x7f4194012670 comp rx=0 tx=0).stop 2023-09-10T01:25:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.496+0000 7f419ffa4700 1 -- 172.21.15.44:0/3259474444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4198119de0 msgr2=0x7f41981a9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:25:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.496+0000 7f419ffa4700 1 --2- 172.21.15.44:0/3259474444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4198119de0 0x7f41981a9ff0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f419000d010 tx=0x7f4190007730 comp rx=0 tx=0).stop 2023-09-10T01:25:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.510+0000 7f419ffa4700 1 -- 172.21.15.44:0/3259474444 shutdown_connections 2023-09-10T01:25:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.510+0000 7f419ffa4700 1 --2- 172.21.15.44:0/3259474444 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f41840a0830 0x7f41840a2cf0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.510+0000 7f419ffa4700 1 --2- 172.21.15.44:0/3259474444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4198119de0 0x7f41981a9ff0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.510+0000 7f419ffa4700 1 --2- 172.21.15.44:0/3259474444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f419810ed50 0x7f41981a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.510+0000 7f419ffa4700 1 --2- 172.21.15.44:0/3259474444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f419810f1c0 0x7f41981a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:25:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.510+0000 7f419ffa4700 1 -- 172.21.15.44:0/3259474444 >> 172.21.15.44:0/3259474444 conn(0x7f419806c6c0 msgr2=0x7f4198117dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:25:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.517+0000 7f419ffa4700 1 -- 172.21.15.44:0/3259474444 shutdown_connections 2023-09-10T01:25:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:25:57.522+0000 7f419ffa4700 1 -- 172.21.15.44:0/3259474444 wait complete. 2023-09-10T01:25:58.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:25:58 smithi131 ceph-mon[103713]: pgmap v1467: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2023-09-10T01:25:58.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:25:58 smithi044 ceph-mon[98676]: pgmap v1467: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2023-09-10T01:25:58.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:25:58 smithi099 ceph-mon[102692]: pgmap v1467: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2023-09-10T01:26:00.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:00 smithi044 ceph-mon[98676]: pgmap v1468: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.9 MiB/s wr, 699 op/s 2023-09-10T01:26:00.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:00 smithi099 ceph-mon[102692]: pgmap v1468: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.9 MiB/s wr, 699 op/s 2023-09-10T01:26:00.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:00 smithi131 ceph-mon[103713]: pgmap v1468: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.9 MiB/s wr, 699 op/s 2023-09-10T01:26:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 295.0 s, 294.9 tps, lat 323.593 ms stddev 104.724 2023-09-10T01:26:02.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:02 smithi099 ceph-mon[102692]: pgmap v1469: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 986 op/s 2023-09-10T01:26:02.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:02 smithi131 ceph-mon[103713]: pgmap v1469: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 986 op/s 2023-09-10T01:26:02.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:02 smithi044 ceph-mon[98676]: pgmap v1469: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 986 op/s 2023-09-10T01:26:04.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:04 smithi131 ceph-mon[103713]: pgmap v1470: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.1 MiB/s wr, 666 op/s 2023-09-10T01:26:04.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:04 smithi044 ceph-mon[98676]: pgmap v1470: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.1 MiB/s wr, 666 op/s 2023-09-10T01:26:04.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:04 smithi099 ceph-mon[102692]: pgmap v1470: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.1 MiB/s wr, 666 op/s 2023-09-10T01:26:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 300.0 s, 301.9 tps, lat 350.825 ms stddev 224.256 2023-09-10T01:26:06.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:06 smithi131 ceph-mon[103713]: pgmap v1471: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.1 MiB/s wr, 666 op/s 2023-09-10T01:26:06.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:06 smithi044 ceph-mon[98676]: pgmap v1471: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.1 MiB/s wr, 666 op/s 2023-09-10T01:26:06.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:06 smithi099 ceph-mon[102692]: pgmap v1471: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.1 MiB/s wr, 666 op/s 2023-09-10T01:26:08.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:08 smithi131 ceph-mon[103713]: pgmap v1472: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.00k op/s 2023-09-10T01:26:08.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:08 smithi044 ceph-mon[98676]: pgmap v1472: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.00k op/s 2023-09-10T01:26:08.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:08 smithi099 ceph-mon[102692]: pgmap v1472: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.00k op/s 2023-09-10T01:26:09.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:09 smithi131 ceph-mon[103713]: pgmap v1473: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 663 op/s 2023-09-10T01:26:09.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:09 smithi044 ceph-mon[98676]: pgmap v1473: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 663 op/s 2023-09-10T01:26:09.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:09 smithi099 ceph-mon[102692]: pgmap v1473: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 663 op/s 2023-09-10T01:26:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.767+0000 7fb5f762c700 1 -- 172.21.15.44:0/3067516623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5f010ed50 msgr2=0x7fb5f0107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.767+0000 7fb5f762c700 1 --2- 172.21.15.44:0/3067516623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5f010ed50 0x7fb5f0107600 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec0098f0 tx=0x7fb5ec009c00 comp rx=0 tx=0).stop 2023-09-10T01:26:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.768+0000 7fb5f762c700 1 -- 172.21.15.44:0/3067516623 shutdown_connections 2023-09-10T01:26:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.768+0000 7fb5f762c700 1 --2- 172.21.15.44:0/3067516623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f0119de0 0x7fb5f011c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.768+0000 7fb5f762c700 1 --2- 172.21.15.44:0/3067516623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5f010ed50 0x7fb5f0107600 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.768+0000 7fb5f762c700 1 --2- 172.21.15.44:0/3067516623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f010f1c0 0x7fb5f0107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.768+0000 7fb5f762c700 1 -- 172.21.15.44:0/3067516623 >> 172.21.15.44:0/3067516623 conn(0x7fb5f006c6c0 msgr2=0x7fb5f006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.777+0000 7fb5f762c700 1 -- 172.21.15.44:0/3067516623 shutdown_connections 2023-09-10T01:26:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.782+0000 7fb5f762c700 1 -- 172.21.15.44:0/3067516623 wait complete. 2023-09-10T01:26:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.783+0000 7fb5f762c700 1 Processor -- start 2023-09-10T01:26:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.783+0000 7fb5f762c700 1 -- start start 2023-09-10T01:26:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.783+0000 7fb5f762c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f010ed50 0x7fb5f0119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.783+0000 7fb5f762c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5f010f1c0 0x7fb5f0119790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.783+0000 7fb5f762c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f0119de0 0x7fb5f0112130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.783+0000 7fb5f762c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f0112900 con 0x7fb5f010ed50 2023-09-10T01:26:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.783+0000 7fb5f762c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f0112a70 con 0x7fb5f0119de0 2023-09-10T01:26:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.783+0000 7fb5f762c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f0112c30 con 0x7fb5f010f1c0 2023-09-10T01:26:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.785+0000 7fb5f4bc7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5f010f1c0 0x7fb5f0119790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.785+0000 7fb5f4bc7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5f010f1c0 0x7fb5f0119790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38218/0 (socket says 172.21.15.44:38218) 2023-09-10T01:26:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.785+0000 7fb5f4bc7700 1 -- 172.21.15.44:0/2085846622 learned_addr learned my addr 172.21.15.44:0/2085846622 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.785+0000 7fb5f53c8700 1 --2- 172.21.15.44:0/2085846622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f010ed50 0x7fb5f0119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.785+0000 7fb5f4bc7700 1 -- 172.21.15.44:0/2085846622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f0119de0 msgr2=0x7fb5f0112130 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-10T01:26:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.785+0000 7fb5f4bc7700 1 --2- 172.21.15.44:0/2085846622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f0119de0 0x7fb5f0112130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.785+0000 7fb5f4bc7700 1 -- 172.21.15.44:0/2085846622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f010ed50 msgr2=0x7fb5f0119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.785+0000 7fb5f4bc7700 1 --2- 172.21.15.44:0/2085846622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f010ed50 0x7fb5f0119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.785+0000 7fb5f4bc7700 1 -- 172.21.15.44:0/2085846622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec009520 con 0x7fb5f010f1c0 2023-09-10T01:26:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.788+0000 7fb5f4bc7700 1 --2- 172.21.15.44:0/2085846622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5f010f1c0 0x7fb5f0119790 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800ada0 tx=0x7fb5e800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.788+0000 7fb5e67fc700 1 -- 172.21.15.44:0/2085846622 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800f070 con 0x7fb5f010f1c0 2023-09-10T01:26:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.788+0000 7fb5e67fc700 1 -- 172.21.15.44:0/2085846622 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5e800ce10 con 0x7fb5f010f1c0 2023-09-10T01:26:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.788+0000 7fb5e67fc700 1 -- 172.21.15.44:0/2085846622 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8005460 con 0x7fb5f010f1c0 2023-09-10T01:26:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.788+0000 7fb5f762c700 1 -- 172.21.15.44:0/2085846622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f0116da0 con 0x7fb5f010f1c0 2023-09-10T01:26:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.788+0000 7fb5f762c700 1 -- 172.21.15.44:0/2085846622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f0117260 con 0x7fb5f010f1c0 2023-09-10T01:26:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.793+0000 7fb6037f2b80 1 Processor -- start 2023-09-10T01:26:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.793+0000 7fb6037f2b80 1 -- start start 2023-09-10T01:26:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.793+0000 7fb6037f2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04c4401a0 0x55f04c440580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.793+0000 7fb6037f2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c43a300 0x55f04c43a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.793+0000 7fb6037f2b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c446f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.793+0000 7fb6037f2b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04c3936d0 con 0x55f04c43acc0 2023-09-10T01:26:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.793+0000 7fb6037f2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04c390ff0 con 0x55f04c4401a0 2023-09-10T01:26:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.793+0000 7fb6037f2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04c38c8e0 con 0x55f04c43a300 2023-09-10T01:26:10.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:10 smithi131 ceph-mon[103713]: osdmap e182: 12 total, 12 up, 12 in 2023-09-10T01:26:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.796+0000 7fb5e67fc700 1 -- 172.21.15.44:0/2085846622 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fb5e8004ad0 con 0x7fb5f010f1c0 2023-09-10T01:26:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.797+0000 7fb5e67fc700 1 --2- 172.21.15.44:0/2085846622 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5dc0a0940 0x7fb5dc0a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.797+0000 7fb5e67fc700 1 -- 172.21.15.44:0/2085846622 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fb5e80cbad0 con 0x7fb5f010f1c0 2023-09-10T01:26:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.798+0000 7fb5f4bc7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c446f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.798+0000 7fb5f4bc7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c446f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59574/0 (socket says 172.21.15.44:59574) 2023-09-10T01:26:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.798+0000 7fb5f4bc7700 1 -- 172.21.15.44:0/2270993586 learned_addr learned my addr 172.21.15.44:0/2270993586 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.798+0000 7fb5f4bc7700 1 -- 172.21.15.44:0/2270993586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c43a300 msgr2=0x55f04c43a780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:26:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.798+0000 7fb5f4bc7700 1 --2- 172.21.15.44:0/2270993586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c43a300 0x55f04c43a780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.798+0000 7fb5f4bc7700 1 -- 172.21.15.44:0/2270993586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04c4401a0 msgr2=0x55f04c440580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:26:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.798+0000 7fb5f4bc7700 1 --2- 172.21.15.44:0/2270993586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04c4401a0 0x55f04c440580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.798+0000 7fb5f4bc7700 1 -- 172.21.15.44:0/2270993586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f04c1b8d80 con 0x55f04c43acc0 2023-09-10T01:26:10.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:10 smithi044 ceph-mon[98676]: osdmap e182: 12 total, 12 up, 12 in 2023-09-10T01:26:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.799+0000 7fb5f4bc7700 1 --2- 172.21.15.44:0/2270993586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c446f30 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8000f80 tx=0x7fb5e80987a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb758a1935a879e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.799+0000 7fb5db7fe700 1 -- 172.21.15.44:0/2270993586 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e809c990 con 0x55f04c43acc0 2023-09-10T01:26:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.799+0000 7fb5db7fe700 1 -- 172.21.15.44:0/2270993586 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5e80a0b40 con 0x55f04c43acc0 2023-09-10T01:26:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.799+0000 7fb5db7fe700 1 -- 172.21.15.44:0/2270993586 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8099020 con 0x55f04c43acc0 2023-09-10T01:26:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.800+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2270993586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 msgr2=0x55f04c446f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.800+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/2270993586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c446f30 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8000f80 tx=0x7fb5e80987a0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.801+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2270993586 shutdown_connections 2023-09-10T01:26:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.801+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/2270993586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c446f30 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.801+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/2270993586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04c4401a0 0x55f04c440580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.801+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/2270993586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c43a300 0x55f04c43a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.801+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2270993586 >> 172.21.15.44:0/2270993586 conn(0x55f04c3af030 msgr2=0x55f04c435560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.804+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2270993586 shutdown_connections 2023-09-10T01:26:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.807+0000 7fb5f5bc9700 1 --2- 172.21.15.44:0/2085846622 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5dc0a0940 0x7fb5dc0a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.810+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2270993586 wait complete. 2023-09-10T01:26:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.811+0000 7fb6037f2b80 1 -- start start 2023-09-10T01:26:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.811+0000 7fb6037f2b80 1 Processor -- start 2023-09-10T01:26:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.811+0000 7fb6037f2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04c43acc0 0x55f04c446de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.811+0000 7fb6037f2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c4401a0 0x55f04c4440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.811+0000 7fb6037f2b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c444660 0x55f04c444ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.811+0000 7fb6037f2b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04c32ab10 con 0x55f04c444660 2023-09-10T01:26:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.811+0000 7fb6037f2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04c32a810 con 0x55f04c43acc0 2023-09-10T01:26:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.811+0000 7fb6037f2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04c20e8c0 con 0x55f04c4401a0 2023-09-10T01:26:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.812+0000 7fb6037f2b80 10 client.0 _ll_register_callbacks cb 0x55f04c274a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:26:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.812+0000 7fb5f4bc7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c4401a0 0x55f04c4440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.812+0000 7fb5f4bc7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c4401a0 0x55f04c4440d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38228/0 (socket says 172.21.15.44:38228) 2023-09-10T01:26:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.812+0000 7fb5f4bc7700 1 -- 172.21.15.44:0/2116491022 learned_addr learned my addr 172.21.15.44:0/2116491022 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.812+0000 7fb5f5bc9700 1 --2- 172.21.15.44:0/2116491022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c444660 0x55f04c444ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.812+0000 7fb5f5bc9700 1 --2- 172.21.15.44:0/2085846622 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5dc0a0940 0x7fb5dc0a2e00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0005950 tx=0x7fb5e00058e0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.813+0000 7fb5f5bc9700 1 -- 172.21.15.44:0/2116491022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c4401a0 msgr2=0x55f04c4440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.813+0000 7fb5f5bc9700 1 --2- 172.21.15.44:0/2116491022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c4401a0 0x55f04c4440d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.813+0000 7fb5f5bc9700 1 -- 172.21.15.44:0/2116491022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04c43acc0 msgr2=0x55f04c446de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:26:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.813+0000 7fb5f5bc9700 1 --2- 172.21.15.44:0/2116491022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04c43acc0 0x55f04c446de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.813+0000 7fb5f5bc9700 1 -- 172.21.15.44:0/2116491022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f0117260 con 0x55f04c444660 2023-09-10T01:26:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.814+0000 7fb5f5bc9700 1 --2- 172.21.15.44:0/2116491022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c444660 0x55f04c444ae0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0069970 tx=0x7fb5e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82722bbe2199c24 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.814+0000 7fb5d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:26:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.815+0000 7fb5d8ff9700 1 -- 172.21.15.44:0/2116491022 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0010af0 con 0x55f04c444660 2023-09-10T01:26:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.815+0000 7fb5d8ff9700 1 -- 172.21.15.44:0/2116491022 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(182..182 src has 1..182) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fb5e0010c50 con 0x55f04c444660 2023-09-10T01:26:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.815+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2116491022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f04c2e4490 con 0x55f04c444660 2023-09-10T01:26:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.815+0000 7fb5d8ff9700 1 -- 172.21.15.44:0/2116491022 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=153) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5e0010e10 con 0x55f04c444660 2023-09-10T01:26:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.815+0000 7fb6037f2b80 10 client.20205 fetch_fsmap learned FSMap version 153 2023-09-10T01:26:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.815+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2116491022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=153,monmap=4+}) v3 -- 0x55f04c1a1c00 con 0x55f04c444660 2023-09-10T01:26:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.817+0000 7fb5d8ff9700 1 -- 172.21.15.44:0/2116491022 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f04c2e4490 con 0x55f04c444660 2023-09-10T01:26:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.817+0000 7fb5d8ff9700 1 -- 172.21.15.44:0/2116491022 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 153) v1 ==== 7367+0+0 (secure 0 0 0) 0x7fb5e0026490 con 0x55f04c444660 2023-09-10T01:26:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.817+0000 7fb6037f2b80 10 client.20205 fetch_fsmap finished waiting for FSMap version 153 2023-09-10T01:26:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.817+0000 7fb6037f2b80 10 client.20205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:26:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.817+0000 7fb6037f2b80 20 client.20205 populate_metadata read hostname 'smithi044' 2023-09-10T01:26:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.817+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/2116491022 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f04c476940 0x55f04c478d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.817+0000 7fb6037f2b80 4 client.20205 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:26:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.817+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2116491022 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f04c3a1840 con 0x55f04c476940 2023-09-10T01:26:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.820+0000 7fb5f53c8700 1 --2- 172.21.15.44:0/2116491022 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f04c476940 0x55f04c478d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.820+0000 7fb5f53c8700 1 --2- 172.21.15.44:0/2116491022 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f04c476940 0x55f04c478d60 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc03c3c88649a3a0 server_cookie=fcbcb84aa0df7c10 in_seq=0 out_seq=0 2023-09-10T01:26:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.820+0000 7fb5d8ff9700 10 client.20205 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:26:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.821+0000 7fb5d8ff9700 1 -- 172.21.15.44:0/2116491022 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f04c3a1840 con 0x55f04c476940 2023-09-10T01:26:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.821+0000 7fb5d8ff9700 10 client.20205 handle_command_reply: tid=0 2023-09-10T01:26:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.822+0000 7fb6037f2b80 1 client.20205 shutdown 2023-09-10T01:26:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.822+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2116491022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c444660 msgr2=0x55f04c444ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.822+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/2116491022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c444660 0x55f04c444ae0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0069970 tx=0x7fb5e0004750 comp rx=0 tx=0).stop 2023-09-10T01:26:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.828+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2116491022 shutdown_connections 2023-09-10T01:26:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.828+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/2116491022 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f04c476940 0x55f04c478d60 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.828+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/2116491022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c444660 0x55f04c444ae0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.828+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/2116491022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04c43acc0 0x55f04c446de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.828+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/2116491022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c4401a0 0x55f04c4440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.828+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2116491022 >> 172.21.15.44:0/2116491022 conn(0x55f04c3af030 msgr2=0x55f04c4347a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.828+0000 7fb5d8ff9700 0 client.20205 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:26:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.847+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2116491022 shutdown_connections 2023-09-10T01:26:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.852+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2116491022 wait complete. 2023-09-10T01:26:10.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:10 smithi099 ceph-mon[102692]: osdmap e182: 12 total, 12 up, 12 in 2023-09-10T01:26:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.862+0000 7fb6037f2b80 20 client.20205 trim_cache size 0 max 16384 2023-09-10T01:26:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.891+0000 7fb6037f2b80 1 Processor -- start 2023-09-10T01:26:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.891+0000 7fb6037f2b80 1 -- start start 2023-09-10T01:26:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.891+0000 7fb6037f2b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c44b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.891+0000 7fb6037f2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04c4401a0 0x55f04c441b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.891+0000 7fb6037f2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c444660 0x55f04c43e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.891+0000 7fb6037f2b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04c2e4490 con 0x55f04c43acc0 2023-09-10T01:26:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.891+0000 7fb6037f2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04c42cf50 con 0x55f04c4401a0 2023-09-10T01:26:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.891+0000 7fb6037f2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04c42f8a0 con 0x55f04c444660 2023-09-10T01:26:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.892+0000 7fb5f53c8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c44b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.892+0000 7fb5f53c8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c44b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59584/0 (socket says 172.21.15.44:59584) 2023-09-10T01:26:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.892+0000 7fb5f53c8700 1 -- 172.21.15.44:0/2340745404 learned_addr learned my addr 172.21.15.44:0/2340745404 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.892+0000 7fb5f53c8700 1 -- 172.21.15.44:0/2340745404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c444660 msgr2=0x55f04c43e190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:26:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.892+0000 7fb5f53c8700 1 --2- 172.21.15.44:0/2340745404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c444660 0x55f04c43e190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.892+0000 7fb5f53c8700 1 -- 172.21.15.44:0/2340745404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04c4401a0 msgr2=0x55f04c441b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:26:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.892+0000 7fb5f53c8700 1 --2- 172.21.15.44:0/2340745404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04c4401a0 0x55f04c441b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.892+0000 7fb5f53c8700 1 -- 172.21.15.44:0/2340745404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f04c47d990 con 0x55f04c43acc0 2023-09-10T01:26:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.892+0000 7fb5f53c8700 1 --2- 172.21.15.44:0/2340745404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c44b520 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec005950 tx=0x7fb5ec009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ab9295ed9e9019 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.897+0000 7fb5dbfff700 1 -- 172.21.15.44:0/2340745404 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec0223c0 con 0x55f04c43acc0 2023-09-10T01:26:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.897+0000 7fb5dbfff700 1 -- 172.21.15.44:0/2340745404 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5ec01d040 con 0x55f04c43acc0 2023-09-10T01:26:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.897+0000 7fb5dbfff700 1 -- 172.21.15.44:0/2340745404 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec022de0 con 0x55f04c43acc0 2023-09-10T01:26:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.898+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2340745404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 msgr2=0x55f04c44b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.898+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/2340745404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c44b520 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec005950 tx=0x7fb5ec009260 comp rx=0 tx=0).stop 2023-09-10T01:26:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.903+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2340745404 shutdown_connections 2023-09-10T01:26:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.903+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/2340745404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c444660 0x55f04c43e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.903+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/2340745404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c44b520 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.903+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/2340745404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04c4401a0 0x55f04c441b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.903+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2340745404 >> 172.21.15.44:0/2340745404 conn(0x55f04c3af030 msgr2=0x55f04c44fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.903+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2340745404 shutdown_connections 2023-09-10T01:26:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.903+0000 7fb6037f2b80 1 -- 172.21.15.44:0/2340745404 wait complete. 2023-09-10T01:26:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.904+0000 7fb6037f2b80 1 -- start start 2023-09-10T01:26:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.904+0000 7fb6037f2b80 1 Processor -- start 2023-09-10T01:26:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.904+0000 7fb6037f2b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c43cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.904+0000 7fb6037f2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c4401a0 0x55f04c43d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.904+0000 7fb6037f2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04c444660 0x55f04c43bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.904+0000 7fb6037f2b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04c44b8b0 con 0x55f04c43acc0 2023-09-10T01:26:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.904+0000 7fb6037f2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04c44b9f0 con 0x55f04c444660 2023-09-10T01:26:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.904+0000 7fb6037f2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04c43c090 con 0x55f04c4401a0 2023-09-10T01:26:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.904+0000 7fb6037f2b80 10 client.0 _ll_register_callbacks cb 0x7fb5e000d740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:26:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.905+0000 7fb5f4bc7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c4401a0 0x55f04c43d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.905+0000 7fb5f4bc7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c4401a0 0x55f04c43d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38238/0 (socket says 172.21.15.44:38238) 2023-09-10T01:26:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.905+0000 7fb5f4bc7700 1 -- 172.21.15.44:0/1309771749 learned_addr learned my addr 172.21.15.44:0/1309771749 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.905+0000 7fb5f53c8700 1 --2- 172.21.15.44:0/1309771749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c43cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.905+0000 7fb5f53c8700 1 -- 172.21.15.44:0/1309771749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c4401a0 msgr2=0x55f04c43d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.905+0000 7fb5f53c8700 1 --2- 172.21.15.44:0/1309771749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c4401a0 0x55f04c43d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.905+0000 7fb5f53c8700 1 -- 172.21.15.44:0/1309771749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04c444660 msgr2=0x55f04c43bb50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:26:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.905+0000 7fb5f53c8700 1 --2- 172.21.15.44:0/1309771749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04c444660 0x55f04c43bb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.905+0000 7fb5f53c8700 1 -- 172.21.15.44:0/1309771749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f011cb00 con 0x55f04c43acc0 2023-09-10T01:26:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.905+0000 7fb5f53c8700 1 --2- 172.21.15.44:0/1309771749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c43cc40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00b2e0 tx=0x7fb5ec022470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f5a9863ac59a2a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.911+0000 7fb5d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:26:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.911+0000 7fb5d8ff9700 1 -- 172.21.15.44:0/1309771749 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec00f870 con 0x55f04c43acc0 2023-09-10T01:26:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.911+0000 7fb5d8ff9700 1 -- 172.21.15.44:0/1309771749 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(182..182 src has 1..182) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fb5ec000e40 con 0x55f04c43acc0 2023-09-10T01:26:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.911+0000 7fb6037f2b80 1 -- 172.21.15.44:0/1309771749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5e80a0b40 con 0x55f04c43acc0 2023-09-10T01:26:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.916+0000 7fb5d8ff9700 1 -- 172.21.15.44:0/1309771749 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=153) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5ec00f9d0 con 0x55f04c43acc0 2023-09-10T01:26:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.916+0000 7fb6037f2b80 10 client.20211 fetch_fsmap learned FSMap version 153 2023-09-10T01:26:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.916+0000 7fb6037f2b80 1 -- 172.21.15.44:0/1309771749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=153,monmap=4+}) v3 -- 0x55f04c1df1f0 con 0x55f04c43acc0 2023-09-10T01:26:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.916+0000 7fb5d8ff9700 1 -- 172.21.15.44:0/1309771749 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5e80a0b40 con 0x55f04c43acc0 2023-09-10T01:26:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.916+0000 7fb5d8ff9700 1 -- 172.21.15.44:0/1309771749 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 153) v1 ==== 7367+0+0 (secure 0 0 0) 0x7fb5ec015020 con 0x55f04c43acc0 2023-09-10T01:26:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.916+0000 7fb6037f2b80 10 client.20211 fetch_fsmap finished waiting for FSMap version 153 2023-09-10T01:26:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.916+0000 7fb6037f2b80 10 client.20211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:26:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.916+0000 7fb6037f2b80 20 client.20211 populate_metadata read hostname 'smithi044' 2023-09-10T01:26:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.916+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/1309771749 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f04c4cae30 0x55f04c4cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.916+0000 7fb6037f2b80 4 client.20211 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:26:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.916+0000 7fb6037f2b80 1 -- 172.21.15.44:0/1309771749 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f04c3858c0 con 0x55f04c4cae30 2023-09-10T01:26:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.921+0000 7fb5f4bc7700 1 --2- 172.21.15.44:0/1309771749 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f04c4cae30 0x55f04c4cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.929+0000 7fb5f4bc7700 1 --2- 172.21.15.44:0/1309771749 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f04c4cae30 0x55f04c4cd250 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91b732fcde0e9525 server_cookie=f852078c4196e4f7 in_seq=0 out_seq=0 2023-09-10T01:26:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:10.929+0000 7fb5d8ff9700 10 client.20211 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:26:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.086+0000 7fb5d8ff9700 1 -- 172.21.15.44:0/1309771749 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f04c3858c0 con 0x55f04c4cae30 2023-09-10T01:26:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.086+0000 7fb5d8ff9700 10 client.20211 handle_command_reply: tid=0 2023-09-10T01:26:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.086+0000 7fb6037f2b80 1 client.20211 shutdown 2023-09-10T01:26:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.098+0000 7fb6037f2b80 1 -- 172.21.15.44:0/1309771749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 msgr2=0x55f04c43cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.098+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/1309771749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c43cc40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00b2e0 tx=0x7fb5ec022470 comp rx=0 tx=0).stop 2023-09-10T01:26:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.105+0000 7fb6037f2b80 1 -- 172.21.15.44:0/1309771749 shutdown_connections 2023-09-10T01:26:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.105+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/1309771749 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f04c4cae30 0x55f04c4cd250 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.105+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/1309771749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04c444660 0x55f04c43bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.105+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/1309771749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f04c43acc0 0x55f04c43cc40 secure :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00b2e0 tx=0x7fb5ec022470 comp rx=0 tx=0).stop 2023-09-10T01:26:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.105+0000 7fb6037f2b80 1 --2- 172.21.15.44:0/1309771749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f04c4401a0 0x55f04c43d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.105+0000 7fb6037f2b80 1 -- 172.21.15.44:0/1309771749 >> 172.21.15.44:0/1309771749 conn(0x55f04c3af030 msgr2=0x55f04c43e190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.113+0000 7fb5d8ff9700 0 client.20211 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:26:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.116+0000 7fb6037f2b80 1 -- 172.21.15.44:0/1309771749 shutdown_connections 2023-09-10T01:26:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.119+0000 7fb6037f2b80 1 -- 172.21.15.44:0/1309771749 wait complete. 2023-09-10T01:26:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:26:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:26:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:26:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:26:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.126+0000 7fb6037f2b80 20 client.20211 trim_cache size 0 max 16384 2023-09-10T01:26:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.127+0000 7fb5f762c700 1 -- 172.21.15.44:0/2085846622 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5dc0a0940 msgr2=0x7fb5dc0a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.127+0000 7fb5f762c700 1 --2- 172.21.15.44:0/2085846622 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5dc0a0940 0x7fb5dc0a2e00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0005950 tx=0x7fb5e00058e0 comp rx=0 tx=0).stop 2023-09-10T01:26:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.127+0000 7fb5f762c700 1 -- 172.21.15.44:0/2085846622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5f010f1c0 msgr2=0x7fb5f0119790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.127+0000 7fb5f762c700 1 --2- 172.21.15.44:0/2085846622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5f010f1c0 0x7fb5f0119790 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800ada0 tx=0x7fb5e800c330 comp rx=0 tx=0).stop 2023-09-10T01:26:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.127+0000 7fb5f762c700 1 -- 172.21.15.44:0/2085846622 shutdown_connections 2023-09-10T01:26:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.127+0000 7fb5f762c700 1 --2- 172.21.15.44:0/2085846622 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb5dc0a0940 0x7fb5dc0a2e00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.127+0000 7fb5f762c700 1 --2- 172.21.15.44:0/2085846622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f0119de0 0x7fb5f0112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.127+0000 7fb5f762c700 1 --2- 172.21.15.44:0/2085846622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f010ed50 0x7fb5f0119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.127+0000 7fb5f762c700 1 --2- 172.21.15.44:0/2085846622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5f010f1c0 0x7fb5f0119790 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.127+0000 7fb5f762c700 1 -- 172.21.15.44:0/2085846622 >> 172.21.15.44:0/2085846622 conn(0x7fb5f006c6c0 msgr2=0x7fb5f011b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:11.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.138+0000 7fb5f762c700 1 -- 172.21.15.44:0/2085846622 shutdown_connections 2023-09-10T01:26:11.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:11.138+0000 7fb5f762c700 1 -- 172.21.15.44:0/2085846622 wait complete. 2023-09-10T01:26:11.182 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 305.0 s, 256.5 tps, lat 292.626 ms stddev 328.165 2023-09-10T01:26:12.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:12 smithi131 ceph-mon[103713]: pgmap v1475: 129 pgs: 18 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 67 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.6 MiB/s wr, 785 op/s 2023-09-10T01:26:12.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:12 smithi044 ceph-mon[98676]: pgmap v1475: 129 pgs: 18 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 67 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.6 MiB/s wr, 785 op/s 2023-09-10T01:26:12.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:12 smithi099 ceph-mon[102692]: pgmap v1475: 129 pgs: 18 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 67 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.6 MiB/s wr, 785 op/s 2023-09-10T01:26:13.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:13 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:26:13.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:13 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:26:13.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:13 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:26:14.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:14 smithi099 ceph-mon[102692]: pgmap v1476: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.2 MiB/s wr, 781 op/s 2023-09-10T01:26:14.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:14 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:26:14.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:14 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:26:14.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:14 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:26:14.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:14 smithi131 ceph-mon[103713]: pgmap v1476: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.2 MiB/s wr, 781 op/s 2023-09-10T01:26:14.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:14 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:26:14.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:14 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:26:14.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:14 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:26:14.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:14 smithi044 ceph-mon[98676]: pgmap v1476: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.2 MiB/s wr, 781 op/s 2023-09-10T01:26:14.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:14 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:26:14.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:14 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:26:14.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:14 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:26:15.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:15 smithi131 ceph-mon[103713]: pgmap v1477: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 808 op/s 2023-09-10T01:26:15.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:15 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:26:15.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:15 smithi044 ceph-mon[98676]: pgmap v1477: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 808 op/s 2023-09-10T01:26:15.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:15 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:26:15.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:15 smithi099 ceph-mon[102692]: pgmap v1477: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 18 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 808 op/s 2023-09-10T01:26:15.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:15 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:26:16.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 310.0 s, 98.6 tps, lat 1075.523 ms stddev 1775.966 2023-09-10T01:26:17.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:17 smithi131 ceph-mon[103713]: pgmap v1478: 129 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 120 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 573 op/s 2023-09-10T01:26:17.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:17 smithi044 ceph-mon[98676]: pgmap v1478: 129 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 120 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 573 op/s 2023-09-10T01:26:17.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:17 smithi099 ceph-mon[102692]: pgmap v1478: 129 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 120 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 573 op/s 2023-09-10T01:26:19.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:19 smithi131 ceph-mon[103713]: pgmap v1479: 129 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 120 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 573 op/s 2023-09-10T01:26:19.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:19 smithi044 ceph-mon[98676]: pgmap v1479: 129 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 120 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 573 op/s 2023-09-10T01:26:19.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:19 smithi099 ceph-mon[102692]: pgmap v1479: 129 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 120 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 573 op/s 2023-09-10T01:26:20.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:20 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:26:20.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:20 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:26:20.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:20 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:26:21.185 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 315.0 s, 79.9 tps, lat 256.254 ms stddev 678.963 2023-09-10T01:26:21.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:21 smithi131 ceph-mon[103713]: pgmap v1480: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.9 MiB/s wr, 563 op/s 2023-09-10T01:26:21.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:21 smithi131 ceph-mon[103713]: osdmap e183: 12 total, 12 up, 12 in 2023-09-10T01:26:21.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:21 smithi044 ceph-mon[98676]: pgmap v1480: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.9 MiB/s wr, 563 op/s 2023-09-10T01:26:21.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:21 smithi044 ceph-mon[98676]: osdmap e183: 12 total, 12 up, 12 in 2023-09-10T01:26:21.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:21 smithi099 ceph-mon[102692]: pgmap v1480: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.9 MiB/s wr, 563 op/s 2023-09-10T01:26:21.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:21 smithi099 ceph-mon[102692]: osdmap e183: 12 total, 12 up, 12 in 2023-09-10T01:26:23.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:23 smithi131 ceph-mon[103713]: pgmap v1482: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 914 KiB/s rd, 3.5 MiB/s wr, 350 op/s 2023-09-10T01:26:23.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:23 smithi044 ceph-mon[98676]: pgmap v1482: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 914 KiB/s rd, 3.5 MiB/s wr, 350 op/s 2023-09-10T01:26:23.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:23 smithi099 ceph-mon[102692]: pgmap v1482: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 914 KiB/s rd, 3.5 MiB/s wr, 350 op/s 2023-09-10T01:26:24.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.321+0000 7f701cb28700 1 -- 172.21.15.44:0/967066651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70181054b0 msgr2=0x7f70180679f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:24.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.321+0000 7f701cb28700 1 --2- 172.21.15.44:0/967066651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70181054b0 0x7f70180679f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f700c009990 tx=0x7f700c009ca0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.321+0000 7f701cb28700 1 -- 172.21.15.44:0/967066651 shutdown_connections 2023-09-10T01:26:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.321+0000 7f701cb28700 1 --2- 172.21.15.44:0/967066651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018068610 0x7f7018103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.322+0000 7f701cb28700 1 --2- 172.21.15.44:0/967066651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70181054b0 0x7f70180679f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.322+0000 7f701cb28700 1 --2- 172.21.15.44:0/967066651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7018105920 0x7f7018067f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.322+0000 7f701cb28700 1 -- 172.21.15.44:0/967066651 >> 172.21.15.44:0/967066651 conn(0x7f7018074f30 msgr2=0x7f7018075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.323+0000 7f701cb28700 1 -- 172.21.15.44:0/967066651 shutdown_connections 2023-09-10T01:26:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.323+0000 7f701cb28700 1 -- 172.21.15.44:0/967066651 wait complete. 2023-09-10T01:26:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.324+0000 7f701cb28700 1 Processor -- start 2023-09-10T01:26:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.324+0000 7f701cb28700 1 -- start start 2023-09-10T01:26:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.324+0000 7f701cb28700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70181054b0 0x7f7018109e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.324+0000 7f701cb28700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7018105920 0x7f701810a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.324+0000 7f701cb28700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018070ad0 0x7f7018070f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.324+0000 7f701cb28700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7018071690 con 0x7f70181054b0 2023-09-10T01:26:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.324+0000 7f701cb28700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7018071800 con 0x7f7018105920 2023-09-10T01:26:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.324+0000 7f701cb28700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7018111bb0 con 0x7f7018070ad0 2023-09-10T01:26:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.324+0000 7f7016d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018070ad0 0x7f7018070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.324+0000 7f701659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70181054b0 0x7f7018109e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.324+0000 7f7016d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018070ad0 0x7f7018070f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38252/0 (socket says 172.21.15.44:38252) 2023-09-10T01:26:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.324+0000 7f7016d9d700 1 -- 172.21.15.44:0/1888271259 learned_addr learned my addr 172.21.15.44:0/1888271259 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.325+0000 7f7016d9d700 1 -- 172.21.15.44:0/1888271259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7018105920 msgr2=0x7f701810a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.325+0000 7f7016d9d700 1 --2- 172.21.15.44:0/1888271259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7018105920 0x7f701810a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.325+0000 7f7016d9d700 1 -- 172.21.15.44:0/1888271259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70181054b0 msgr2=0x7f7018109e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.325+0000 7f7016d9d700 1 --2- 172.21.15.44:0/1888271259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70181054b0 0x7f7018109e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.325+0000 7f7016d9d700 1 -- 172.21.15.44:0/1888271259 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c0095f0 con 0x7f7018070ad0 2023-09-10T01:26:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.325+0000 7f7016d9d700 1 --2- 172.21.15.44:0/1888271259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018070ad0 0x7f7018070f80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f701000ada0 tx=0x7f701000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.325+0000 7f70077fe700 1 -- 172.21.15.44:0/1888271259 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701000f070 con 0x7f7018070ad0 2023-09-10T01:26:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.325+0000 7f701cb28700 1 -- 172.21.15.44:0/1888271259 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7018111e90 con 0x7f7018070ad0 2023-09-10T01:26:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.325+0000 7f701cb28700 1 -- 172.21.15.44:0/1888271259 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70181123a0 con 0x7f7018070ad0 2023-09-10T01:26:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.325+0000 7f70077fe700 1 -- 172.21.15.44:0/1888271259 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7010007b30 con 0x7f7018070ad0 2023-09-10T01:26:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.325+0000 7f70077fe700 1 -- 172.21.15.44:0/1888271259 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010005280 con 0x7f7018070ad0 2023-09-10T01:26:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.330+0000 7f7028ceeb80 1 Processor -- start 2023-09-10T01:26:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.334+0000 7f70077fe700 1 -- 172.21.15.44:0/1888271259 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f701000c980 con 0x7f7018070ad0 2023-09-10T01:26:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.334+0000 7f7028ceeb80 1 -- start start 2023-09-10T01:26:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.334+0000 7f7028ceeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb061a0 0x5639eeb06580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.334+0000 7f7028ceeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb00300 0x5639eeb00780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.334+0000 7f7028ceeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb00cc0 0x5639eeb0cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.334+0000 7f7028ceeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639eea596d0 con 0x5639eeb00300 2023-09-10T01:26:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.334+0000 7f7028ceeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639eea56ff0 con 0x5639eeb00cc0 2023-09-10T01:26:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.334+0000 7f7028ceeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639eea528e0 con 0x5639eeb061a0 2023-09-10T01:26:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.334+0000 7f7015d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb061a0 0x5639eeb06580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.334+0000 7f7015d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb061a0 0x5639eeb06580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38256/0 (socket says 172.21.15.44:38256) 2023-09-10T01:26:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.334+0000 7f7015d9b700 1 -- 172.21.15.44:0/4236211013 learned_addr learned my addr 172.21.15.44:0/4236211013 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.334+0000 7f7015d9b700 1 --2- 172.21.15.44:0/4236211013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb00cc0 0x5639eeb0cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.335+0000 7f70077fe700 1 --2- 172.21.15.44:0/1888271259 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f70000a0870 0x7f70000a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.335+0000 7f7015d9b700 1 -- 172.21.15.44:0/4236211013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb00cc0 msgr2=0x5639eeb0cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.335+0000 7f7015d9b700 1 --2- 172.21.15.44:0/4236211013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb00cc0 0x5639eeb0cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.335+0000 7f7015d9b700 1 -- 172.21.15.44:0/4236211013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb00300 msgr2=0x5639eeb00780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.335+0000 7f7015d9b700 1 --2- 172.21.15.44:0/4236211013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb00300 0x5639eeb00780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.335+0000 7f7015d9b700 1 -- 172.21.15.44:0/4236211013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639ee87ed80 con 0x5639eeb061a0 2023-09-10T01:26:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.335+0000 7f70077fe700 1 -- 172.21.15.44:0/1888271259 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f70100cb920 con 0x7f7018070ad0 2023-09-10T01:26:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.335+0000 7f7015d9b700 1 --2- 172.21.15.44:0/4236211013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb061a0 0x5639eeb06580 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7008015430 tx=0x7f7008009e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e3e425139ef5125 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.335+0000 7f7016d9d700 1 --2- 172.21.15.44:0/1888271259 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f70000a0870 0x7f70000a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.335+0000 7f7004ff9700 1 -- 172.21.15.44:0/4236211013 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008015e60 con 0x5639eeb061a0 2023-09-10T01:26:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.335+0000 7f7004ff9700 1 -- 172.21.15.44:0/4236211013 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7008001710 con 0x5639eeb061a0 2023-09-10T01:26:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.335+0000 7f7004ff9700 1 -- 172.21.15.44:0/4236211013 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008009290 con 0x5639eeb061a0 2023-09-10T01:26:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.336+0000 7f7016d9d700 1 --2- 172.21.15.44:0/1888271259 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f70000a0870 0x7f70000a2d30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f701000c730 tx=0x7f7010094620 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.336+0000 7f7028ceeb80 1 -- 172.21.15.44:0/4236211013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb061a0 msgr2=0x5639eeb06580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.336+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/4236211013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb061a0 0x5639eeb06580 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7008015430 tx=0x7f7008009e90 comp rx=0 tx=0).stop 2023-09-10T01:26:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.337+0000 7f7028ceeb80 1 -- 172.21.15.44:0/4236211013 shutdown_connections 2023-09-10T01:26:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.337+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/4236211013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb00cc0 0x5639eeb0cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.337+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/4236211013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb061a0 0x5639eeb06580 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.337+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/4236211013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb00300 0x5639eeb00780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.337+0000 7f7028ceeb80 1 -- 172.21.15.44:0/4236211013 >> 172.21.15.44:0/4236211013 conn(0x5639eea75030 msgr2=0x5639eeafb560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.337+0000 7f7028ceeb80 1 -- 172.21.15.44:0/4236211013 shutdown_connections 2023-09-10T01:26:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.337+0000 7f7028ceeb80 1 -- 172.21.15.44:0/4236211013 wait complete. 2023-09-10T01:26:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.338+0000 7f7028ceeb80 1 -- start start 2023-09-10T01:26:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.338+0000 7f7028ceeb80 1 Processor -- start 2023-09-10T01:26:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.338+0000 7f7028ceeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb00300 0x5639eeb0ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.338+0000 7f7028ceeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00cc0 0x5639eeb0a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.338+0000 7f7028ceeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb061a0 0x5639eeb0a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.338+0000 7f7028ceeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639ee9f0b10 con 0x5639eeb00300 2023-09-10T01:26:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.338+0000 7f7028ceeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639ee9f0810 con 0x5639eeb061a0 2023-09-10T01:26:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.338+0000 7f7028ceeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639ee8d48c0 con 0x5639eeb00cc0 2023-09-10T01:26:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.338+0000 7f7028ceeb80 10 client.0 _ll_register_callbacks cb 0x5639ee93aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:26:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.338+0000 7f7015d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb061a0 0x5639eeb0a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.338+0000 7f7015d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb061a0 0x5639eeb0a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33124/0 (socket says 172.21.15.44:33124) 2023-09-10T01:26:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.338+0000 7f7015d9b700 1 -- 172.21.15.44:0/3344599785 learned_addr learned my addr 172.21.15.44:0/3344599785 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:24.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.339+0000 7f7015d9b700 1 --2- 172.21.15.44:0/3344599785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb00300 0x5639eeb0ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:24.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.340+0000 7f701659c700 1 --2- 172.21.15.44:0/3344599785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00cc0 0x5639eeb0a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:24.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.340+0000 7f7015d9b700 1 -- 172.21.15.44:0/3344599785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00cc0 msgr2=0x5639eeb0a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:24.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.340+0000 7f7015d9b700 1 --2- 172.21.15.44:0/3344599785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00cc0 0x5639eeb0a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.340+0000 7f7015d9b700 1 -- 172.21.15.44:0/3344599785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb00300 msgr2=0x5639eeb0ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:24.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.340+0000 7f7015d9b700 1 --2- 172.21.15.44:0/3344599785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb00300 0x5639eeb0ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.340+0000 7f7015d9b700 1 -- 172.21.15.44:0/3344599785 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639ee867c00 con 0x5639eeb061a0 2023-09-10T01:26:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.340+0000 7f701659c700 1 --2- 172.21.15.44:0/3344599785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00cc0 0x5639eeb0a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:26:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.340+0000 7f7015d9b700 1 --2- 172.21.15.44:0/3344599785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb061a0 0x5639eeb0a730 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7008007110 tx=0x7f7008007490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10d942e98c0b26ee server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.340+0000 7f6ffdffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:26:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.340+0000 7f6ffdffb700 1 -- 172.21.15.44:0/3344599785 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008015e60 con 0x5639eeb061a0 2023-09-10T01:26:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.340+0000 7f6ffdffb700 1 -- 172.21.15.44:0/3344599785 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(183..183 src has 1..183) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f7008008a10 con 0x5639eeb061a0 2023-09-10T01:26:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.340+0000 7f7028ceeb80 1 -- 172.21.15.44:0/3344599785 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639ee9aa490 con 0x5639eeb061a0 2023-09-10T01:26:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.341+0000 7f6ffdffb700 1 -- 172.21.15.44:0/3344599785 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=155) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7008008bd0 con 0x5639eeb061a0 2023-09-10T01:26:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.341+0000 7f7028ceeb80 10 client.29849 fetch_fsmap learned FSMap version 155 2023-09-10T01:26:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.341+0000 7f7028ceeb80 1 -- 172.21.15.44:0/3344599785 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=155,monmap=4+}) v3 -- 0x5639ee8a51f0 con 0x5639eeb061a0 2023-09-10T01:26:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.341+0000 7f6ffdffb700 1 -- 172.21.15.44:0/3344599785 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5639ee9aa490 con 0x5639eeb061a0 2023-09-10T01:26:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.341+0000 7f6ffdffb700 1 -- 172.21.15.44:0/3344599785 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 155) v1 ==== 7383+0+0 (secure 0 0 0) 0x7f7008009100 con 0x5639eeb061a0 2023-09-10T01:26:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.341+0000 7f7028ceeb80 10 client.29849 fetch_fsmap finished waiting for FSMap version 155 2023-09-10T01:26:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.341+0000 7f7028ceeb80 10 client.29849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:26:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.341+0000 7f7028ceeb80 20 client.29849 populate_metadata read hostname 'smithi044' 2023-09-10T01:26:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.341+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/3344599785 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5639eeb1a2c0 0x5639eeb1c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.341+0000 7f7028ceeb80 4 client.29849 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:26:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.341+0000 7f7028ceeb80 1 -- 172.21.15.44:0/3344599785 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639eea67840 con 0x5639eeb1a2c0 2023-09-10T01:26:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.341+0000 7f701659c700 1 --2- 172.21.15.44:0/3344599785 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5639eeb1a2c0 0x5639eeb1c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.342+0000 7f6ffdffb700 10 client.29849 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:26:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.342+0000 7f701659c700 1 --2- 172.21.15.44:0/3344599785 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5639eeb1a2c0 0x5639eeb1c6e0 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ce124f5eeffe691 server_cookie=ff9e5d036d6385d9 in_seq=0 out_seq=0 2023-09-10T01:26:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.343+0000 7f6ffdffb700 1 -- 172.21.15.44:0/3344599785 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5639eea67840 con 0x5639eeb1a2c0 2023-09-10T01:26:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.343+0000 7f6ffdffb700 10 client.29849 handle_command_reply: tid=0 2023-09-10T01:26:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.343+0000 7f7028ceeb80 1 client.29849 shutdown 2023-09-10T01:26:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.343+0000 7f7028ceeb80 1 -- 172.21.15.44:0/3344599785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb061a0 msgr2=0x5639eeb0a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.343+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/3344599785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb061a0 0x5639eeb0a730 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7008007110 tx=0x7f7008007490 comp rx=0 tx=0).stop 2023-09-10T01:26:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.343+0000 7f7028ceeb80 1 -- 172.21.15.44:0/3344599785 shutdown_connections 2023-09-10T01:26:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.343+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/3344599785 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5639eeb1a2c0 0x5639eeb1c6e0 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.343+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/3344599785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb061a0 0x5639eeb0a730 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.343+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/3344599785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb00300 0x5639eeb0ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.343+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/3344599785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00cc0 0x5639eeb0a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.343+0000 7f7028ceeb80 1 -- 172.21.15.44:0/3344599785 >> 172.21.15.44:0/3344599785 conn(0x5639eea75030 msgr2=0x5639eeafa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.343+0000 7f6ffdffb700 0 client.29849 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:26:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.343+0000 7f7028ceeb80 1 -- 172.21.15.44:0/3344599785 shutdown_connections 2023-09-10T01:26:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.344+0000 7f7028ceeb80 1 -- 172.21.15.44:0/3344599785 wait complete. 2023-09-10T01:26:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.344+0000 7f7028ceeb80 20 client.29849 trim_cache size 0 max 16384 2023-09-10T01:26:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.368+0000 7f7028ceeb80 1 Processor -- start 2023-09-10T01:26:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.368+0000 7f7028ceeb80 1 -- start start 2023-09-10T01:26:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.368+0000 7f7028ceeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb00300 0x5639eeb11530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.368+0000 7f7028ceeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00cc0 0x5639eeb050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.368+0000 7f7028ceeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb061a0 0x5639eeb055e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.368+0000 7f7028ceeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639eea67870 con 0x5639eeb00300 2023-09-10T01:26:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.368+0000 7f7028ceeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639ee9aa4c0 con 0x5639eeb061a0 2023-09-10T01:26:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.368+0000 7f7028ceeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639eeb069b0 con 0x5639eeb00cc0 2023-09-10T01:26:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.369+0000 7f7015d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb00300 0x5639eeb11530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.369+0000 7f7015d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb00300 0x5639eeb11530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59622/0 (socket says 172.21.15.44:59622) 2023-09-10T01:26:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.369+0000 7f7015d9b700 1 -- 172.21.15.44:0/181712592 learned_addr learned my addr 172.21.15.44:0/181712592 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.369+0000 7f701659c700 1 --2- 172.21.15.44:0/181712592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00cc0 0x5639eeb050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.369+0000 7f7015d9b700 1 --2- 172.21.15.44:0/181712592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb061a0 0x5639eeb055e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.369+0000 7f7015d9b700 1 -- 172.21.15.44:0/181712592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00cc0 msgr2=0x5639eeb050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.369+0000 7f7015d9b700 1 --2- 172.21.15.44:0/181712592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00cc0 0x5639eeb050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.369+0000 7f7015d9b700 1 -- 172.21.15.44:0/181712592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb061a0 msgr2=0x5639eeb055e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.369+0000 7f7015d9b700 1 --2- 172.21.15.44:0/181712592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb061a0 0x5639eeb055e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.369+0000 7f7015d9b700 1 -- 172.21.15.44:0/181712592 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639ee87d250 con 0x5639eeb00300 2023-09-10T01:26:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.369+0000 7f7015d9b700 1 --2- 172.21.15.44:0/181712592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb00300 0x5639eeb11530 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f7008000c00 tx=0x7f70080153b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e3f869a406760a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.369+0000 7f70057fa700 1 -- 172.21.15.44:0/181712592 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080072a0 con 0x5639eeb00300 2023-09-10T01:26:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.369+0000 7f70057fa700 1 -- 172.21.15.44:0/181712592 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7008015c60 con 0x5639eeb00300 2023-09-10T01:26:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.369+0000 7f70057fa700 1 -- 172.21.15.44:0/181712592 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008008d90 con 0x5639eeb00300 2023-09-10T01:26:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.370+0000 7f7028ceeb80 1 -- 172.21.15.44:0/181712592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb00300 msgr2=0x5639eeb11530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.370+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/181712592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb00300 0x5639eeb11530 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f7008000c00 tx=0x7f70080153b0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.370+0000 7f7028ceeb80 1 -- 172.21.15.44:0/181712592 shutdown_connections 2023-09-10T01:26:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.370+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/181712592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb061a0 0x5639eeb055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.370+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/181712592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb00300 0x5639eeb11530 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.370+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/181712592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00cc0 0x5639eeb050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.370+0000 7f7028ceeb80 1 -- 172.21.15.44:0/181712592 >> 172.21.15.44:0/181712592 conn(0x5639eea75030 msgr2=0x5639eeb04150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.370+0000 7f7028ceeb80 1 -- 172.21.15.44:0/181712592 shutdown_connections 2023-09-10T01:26:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.370+0000 7f7028ceeb80 1 -- 172.21.15.44:0/181712592 wait complete. 2023-09-10T01:26:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.371+0000 7f7028ceeb80 1 -- start start 2023-09-10T01:26:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.371+0000 7f7028ceeb80 1 Processor -- start 2023-09-10T01:26:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.371+0000 7f7028ceeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00300 0x5639eeb62270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.371+0000 7f7028ceeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb00cc0 0x5639eeb627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.371+0000 7f7028ceeb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb061a0 0x5639eeb6d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.371+0000 7f7028ceeb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639eeb62ec0 con 0x5639eeb061a0 2023-09-10T01:26:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.371+0000 7f7028ceeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639eeb6d640 con 0x5639eeb00cc0 2023-09-10T01:26:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.371+0000 7f7028ceeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639eeb6d780 con 0x5639eeb00300 2023-09-10T01:26:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.371+0000 7f7028ceeb80 10 client.0 _ll_register_callbacks cb 0x7f70080019c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:26:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.371+0000 7f7015d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00300 0x5639eeb62270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.371+0000 7f7015d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00300 0x5639eeb62270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38278/0 (socket says 172.21.15.44:38278) 2023-09-10T01:26:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.371+0000 7f701659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb00cc0 0x5639eeb627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.371+0000 7f7015d9b700 1 -- 172.21.15.44:0/3105779379 learned_addr learned my addr 172.21.15.44:0/3105779379 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.371+0000 7f7015d9b700 1 --2- 172.21.15.44:0/3105779379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb061a0 0x5639eeb6d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.372+0000 7f7015d9b700 1 -- 172.21.15.44:0/3105779379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb00cc0 msgr2=0x5639eeb627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.372+0000 7f7015d9b700 1 --2- 172.21.15.44:0/3105779379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb00cc0 0x5639eeb627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.372+0000 7f7015d9b700 1 -- 172.21.15.44:0/3105779379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb061a0 msgr2=0x5639eeb6d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.372+0000 7f7015d9b700 1 --2- 172.21.15.44:0/3105779379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb061a0 0x5639eeb6d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.372+0000 7f7015d9b700 1 -- 172.21.15.44:0/3105779379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639ee8a51f0 con 0x5639eeb00300 2023-09-10T01:26:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.372+0000 7f7015d9b700 1 --2- 172.21.15.44:0/3105779379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00300 0x5639eeb62270 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f70080010c0 tx=0x7f70080153b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2c029a9e0e07ba0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.372+0000 7f6ffdffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:26:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.372+0000 7f6ffdffb700 1 -- 172.21.15.44:0/3105779379 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008008a10 con 0x5639eeb00300 2023-09-10T01:26:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.372+0000 7f7028ceeb80 1 -- 172.21.15.44:0/3105779379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7008001710 con 0x5639eeb00300 2023-09-10T01:26:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.372+0000 7f6ffdffb700 1 -- 172.21.15.44:0/3105779379 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(183..183 src has 1..183) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f7008008b70 con 0x5639eeb00300 2023-09-10T01:26:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.372+0000 7f6ffdffb700 1 -- 172.21.15.44:0/3105779379 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=155) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7008008d30 con 0x5639eeb00300 2023-09-10T01:26:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.372+0000 7f7028ceeb80 10 client.29845 fetch_fsmap learned FSMap version 155 2023-09-10T01:26:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.372+0000 7f7028ceeb80 1 -- 172.21.15.44:0/3105779379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=155,monmap=4+}) v3 -- 0x5639ee94ba20 con 0x5639eeb00300 2023-09-10T01:26:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.373+0000 7f6ffdffb700 1 -- 172.21.15.44:0/3105779379 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7008001710 con 0x5639eeb00300 2023-09-10T01:26:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.374+0000 7f6ffdffb700 1 -- 172.21.15.44:0/3105779379 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 155) v1 ==== 7383+0+0 (secure 0 0 0) 0x7f7008004a80 con 0x5639eeb00300 2023-09-10T01:26:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.374+0000 7f7028ceeb80 10 client.29845 fetch_fsmap finished waiting for FSMap version 155 2023-09-10T01:26:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.374+0000 7f7028ceeb80 10 client.29845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:26:24.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.374+0000 7f7028ceeb80 20 client.29845 populate_metadata read hostname 'smithi044' 2023-09-10T01:26:24.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.374+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/3105779379 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5639eeb6e9b0 0x5639eeb70da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:24.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.374+0000 7f7028ceeb80 4 client.29845 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:26:24.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.374+0000 7f7028ceeb80 1 -- 172.21.15.44:0/3105779379 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5639eea4b8c0 con 0x5639eeb6e9b0 2023-09-10T01:26:24.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.374+0000 7f701659c700 1 --2- 172.21.15.44:0/3105779379 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5639eeb6e9b0 0x5639eeb70da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:24.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.374+0000 7f6ffdffb700 10 client.29845 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:26:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.374+0000 7f701659c700 1 --2- 172.21.15.44:0/3105779379 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5639eeb6e9b0 0x5639eeb70da0 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1057898d84b17dd1 server_cookie=bdd51811eaecfb1d in_seq=0 out_seq=0 2023-09-10T01:26:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.408+0000 7f6ffdffb700 1 -- 172.21.15.44:0/3105779379 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5639eea4b8c0 con 0x5639eeb6e9b0 2023-09-10T01:26:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.408+0000 7f6ffdffb700 10 client.29845 handle_command_reply: tid=0 2023-09-10T01:26:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.412+0000 7f7028ceeb80 1 client.29845 shutdown 2023-09-10T01:26:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.412+0000 7f7028ceeb80 1 -- 172.21.15.44:0/3105779379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00300 msgr2=0x5639eeb62270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.412+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/3105779379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00300 0x5639eeb62270 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f70080010c0 tx=0x7f70080153b0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.415+0000 7f7028ceeb80 1 -- 172.21.15.44:0/3105779379 shutdown_connections 2023-09-10T01:26:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.415+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/3105779379 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5639eeb6e9b0 0x5639eeb70da0 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.415+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/3105779379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639eeb061a0 0x5639eeb6d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.415+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/3105779379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5639eeb00300 0x5639eeb62270 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.415+0000 7f7028ceeb80 1 --2- 172.21.15.44:0/3105779379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639eeb00cc0 0x5639eeb627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.415+0000 7f7028ceeb80 1 -- 172.21.15.44:0/3105779379 >> 172.21.15.44:0/3105779379 conn(0x5639eea75030 msgr2=0x5639eeb050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.415+0000 7f6ffdffb700 0 client.29845 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:26:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.415+0000 7f7028ceeb80 1 -- 172.21.15.44:0/3105779379 shutdown_connections 2023-09-10T01:26:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.415+0000 7f7028ceeb80 1 -- 172.21.15.44:0/3105779379 wait complete. 2023-09-10T01:26:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.416+0000 7f7028ceeb80 20 client.29845 trim_cache size 0 max 16384 2023-09-10T01:26:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:26:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:26:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:26:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:26:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.416+0000 7f701cb28700 1 -- 172.21.15.44:0/1888271259 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f70000a0870 msgr2=0x7f70000a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.416+0000 7f701cb28700 1 --2- 172.21.15.44:0/1888271259 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f70000a0870 0x7f70000a2d30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f701000c730 tx=0x7f7010094620 comp rx=0 tx=0).stop 2023-09-10T01:26:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.416+0000 7f701cb28700 1 -- 172.21.15.44:0/1888271259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018070ad0 msgr2=0x7f7018070f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.416+0000 7f701cb28700 1 --2- 172.21.15.44:0/1888271259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018070ad0 0x7f7018070f80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f701000ada0 tx=0x7f701000c330 comp rx=0 tx=0).stop 2023-09-10T01:26:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.416+0000 7f701cb28700 1 -- 172.21.15.44:0/1888271259 shutdown_connections 2023-09-10T01:26:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.416+0000 7f701cb28700 1 --2- 172.21.15.44:0/1888271259 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f70000a0870 0x7f70000a2d30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.416+0000 7f701cb28700 1 --2- 172.21.15.44:0/1888271259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018070ad0 0x7f7018070f80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.416+0000 7f701cb28700 1 --2- 172.21.15.44:0/1888271259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70181054b0 0x7f7018109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.416+0000 7f701cb28700 1 --2- 172.21.15.44:0/1888271259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7018105920 0x7f701810a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.416+0000 7f701cb28700 1 -- 172.21.15.44:0/1888271259 >> 172.21.15.44:0/1888271259 conn(0x7f7018074f30 msgr2=0x7f7018101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.417+0000 7f701cb28700 1 -- 172.21.15.44:0/1888271259 shutdown_connections 2023-09-10T01:26:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:24.417+0000 7f701cb28700 1 -- 172.21.15.44:0/1888271259 wait complete. 2023-09-10T01:26:25.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:25 smithi131 ceph-mon[103713]: pgmap v1483: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 884 KiB/s rd, 3.4 MiB/s wr, 338 op/s 2023-09-10T01:26:25.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:25 smithi044 ceph-mon[98676]: pgmap v1483: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 884 KiB/s rd, 3.4 MiB/s wr, 338 op/s 2023-09-10T01:26:25.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:25 smithi099 ceph-mon[102692]: pgmap v1483: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 884 KiB/s rd, 3.4 MiB/s wr, 338 op/s 2023-09-10T01:26:26.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 320.0 s, 139.1 tps, lat 1423.880 ms stddev 2765.653 2023-09-10T01:26:27.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:27 smithi044 ceph-mon[98676]: pgmap v1484: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 287 op/s 2023-09-10T01:26:27.557 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:27 smithi044 ceph-mon[98676]: osdmap e184: 12 total, 12 up, 12 in 2023-09-10T01:26:27.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:27 smithi099 ceph-mon[102692]: pgmap v1484: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 287 op/s 2023-09-10T01:26:27.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:27 smithi099 ceph-mon[102692]: osdmap e184: 12 total, 12 up, 12 in 2023-09-10T01:26:27.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:27 smithi131 ceph-mon[103713]: pgmap v1484: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 287 op/s 2023-09-10T01:26:27.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:27 smithi131 ceph-mon[103713]: osdmap e184: 12 total, 12 up, 12 in 2023-09-10T01:26:29.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:29 smithi044 ceph-mon[98676]: pgmap v1486: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 296 op/s 2023-09-10T01:26:29.608 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:29 smithi099 ceph-mon[102692]: pgmap v1486: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 296 op/s 2023-09-10T01:26:29.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:29 smithi131 ceph-mon[103713]: pgmap v1486: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 296 op/s 2023-09-10T01:26:31.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 325.0 s, 261.8 tps, lat 375.802 ms stddev 194.639 2023-09-10T01:26:31.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:31 smithi099 ceph-mon[102692]: pgmap v1487: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 961 KiB/s rd, 2.5 MiB/s wr, 277 op/s 2023-09-10T01:26:31.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:31 smithi131 ceph-mon[103713]: pgmap v1487: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 961 KiB/s rd, 2.5 MiB/s wr, 277 op/s 2023-09-10T01:26:31.819 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:31 smithi044 ceph-mon[98676]: pgmap v1487: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 961 KiB/s rd, 2.5 MiB/s wr, 277 op/s 2023-09-10T01:26:33.609 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:33 smithi099 ceph-mon[102692]: pgmap v1488: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 542 op/s 2023-09-10T01:26:33.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:33 smithi131 ceph-mon[103713]: pgmap v1488: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 542 op/s 2023-09-10T01:26:33.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:33 smithi044 ceph-mon[98676]: pgmap v1488: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 542 op/s 2023-09-10T01:26:35.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:35 smithi131 ceph-mon[103713]: pgmap v1489: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 542 op/s 2023-09-10T01:26:35.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:35 smithi044 ceph-mon[98676]: pgmap v1489: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 542 op/s 2023-09-10T01:26:35.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:35 smithi099 ceph-mon[102692]: pgmap v1489: 129 pgs: 129 active+clean; 18 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 542 op/s 2023-09-10T01:26:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 330.0 s, 246.0 tps, lat 394.529 ms stddev 146.903 2023-09-10T01:26:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.621+0000 7f625e1a8700 1 -- 172.21.15.44:0/1631050480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6258106650 msgr2=0x7f6258106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.621+0000 7f625e1a8700 1 --2- 172.21.15.44:0/1631050480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6258106650 0x7f6258106a30 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f62400099c0 tx=0x7f6240009cd0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.621+0000 7f625e1a8700 1 -- 172.21.15.44:0/1631050480 shutdown_connections 2023-09-10T01:26:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.621+0000 7f625e1a8700 1 --2- 172.21.15.44:0/1631050480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62580683b0 0x7f625810d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.621+0000 7f625e1a8700 1 --2- 172.21.15.44:0/1631050480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6258106650 0x7f6258106a30 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.621+0000 7f625e1a8700 1 --2- 172.21.15.44:0/1631050480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62580679f0 0x7f6258067e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.621+0000 7f625e1a8700 1 -- 172.21.15.44:0/1631050480 >> 172.21.15.44:0/1631050480 conn(0x7f6258074f30 msgr2=0x7f6258075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.626+0000 7f625e1a8700 1 -- 172.21.15.44:0/1631050480 shutdown_connections 2023-09-10T01:26:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.626+0000 7f625e1a8700 1 -- 172.21.15.44:0/1631050480 wait complete. 2023-09-10T01:26:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.627+0000 7f625e1a8700 1 Processor -- start 2023-09-10T01:26:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.627+0000 7f625e1a8700 1 -- start start 2023-09-10T01:26:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.627+0000 7f625e1a8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62580683b0 0x7f6258199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.627+0000 7f625e1a8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6258106650 0x7f62581999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.627+0000 7f625e1a8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f625810c160 0x7f625810c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.627+0000 7f625e1a8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625810cd20 con 0x7f625810c160 2023-09-10T01:26:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.627+0000 7f625e1a8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625810ce90 con 0x7f62580683b0 2023-09-10T01:26:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.627+0000 7f625e1a8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625819d1d0 con 0x7f6258106650 2023-09-10T01:26:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.628+0000 7f6256ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6258106650 0x7f62581999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.628+0000 7f6256ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6258106650 0x7f62581999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38294/0 (socket says 172.21.15.44:38294) 2023-09-10T01:26:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.628+0000 7f6256ffd700 1 -- 172.21.15.44:0/407909435 learned_addr learned my addr 172.21.15.44:0/407909435 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.628+0000 7f62577fe700 1 --2- 172.21.15.44:0/407909435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62580683b0 0x7f6258199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.628+0000 7f6256ffd700 1 -- 172.21.15.44:0/407909435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62580683b0 msgr2=0x7f6258199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.628+0000 7f6256ffd700 1 --2- 172.21.15.44:0/407909435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62580683b0 0x7f6258199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.628+0000 7f6256ffd700 1 -- 172.21.15.44:0/407909435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f625810c160 msgr2=0x7f625810c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.628+0000 7f6256ffd700 1 --2- 172.21.15.44:0/407909435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f625810c160 0x7f625810c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.628+0000 7f6256ffd700 1 -- 172.21.15.44:0/407909435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62400095f0 con 0x7f6258106650 2023-09-10T01:26:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.630+0000 7f6256ffd700 1 --2- 172.21.15.44:0/407909435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6258106650 0x7f62581999c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f62480094f0 tx=0x7f624800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.630+0000 7f6254ff9700 1 -- 172.21.15.44:0/407909435 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624800ddd0 con 0x7f6258106650 2023-09-10T01:26:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.630+0000 7f6254ff9700 1 -- 172.21.15.44:0/407909435 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f624800f040 con 0x7f6258106650 2023-09-10T01:26:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.630+0000 7f6254ff9700 1 -- 172.21.15.44:0/407909435 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62480125f0 con 0x7f6258106650 2023-09-10T01:26:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.634+0000 7f625e1a8700 1 -- 172.21.15.44:0/407909435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625819d4b0 con 0x7f6258106650 2023-09-10T01:26:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.634+0000 7f625e1a8700 1 -- 172.21.15.44:0/407909435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625819d940 con 0x7f6258106650 2023-09-10T01:26:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.639+0000 7f626a36eb80 1 Processor -- start 2023-09-10T01:26:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.642+0000 7f626a36eb80 1 -- start start 2023-09-10T01:26:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.642+0000 7f626a36eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 0x5640fbf57580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.642+0000 7f626a36eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf51300 0x5640fbf51780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.642+0000 7f626a36eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51cc0 0x5640fbf5df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.642+0000 7f626a36eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640fbeaa6d0 con 0x5640fbf51cc0 2023-09-10T01:26:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.642+0000 7f626a36eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640fbea7ff0 con 0x5640fbf51300 2023-09-10T01:26:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.642+0000 7f626a36eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640fbea38e0 con 0x5640fbf571a0 2023-09-10T01:26:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.646+0000 7f6257fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 0x5640fbf57580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.646+0000 7f6257fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 0x5640fbf57580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38298/0 (socket says 172.21.15.44:38298) 2023-09-10T01:26:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.646+0000 7f6257fff700 1 -- 172.21.15.44:0/4190925293 learned_addr learned my addr 172.21.15.44:0/4190925293 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.646+0000 7f6256ffd700 1 --2- 172.21.15.44:0/4190925293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51cc0 0x5640fbf5df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.647+0000 7f6257fff700 1 -- 172.21.15.44:0/4190925293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf51300 msgr2=0x5640fbf51780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:26:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.647+0000 7f6257fff700 1 --2- 172.21.15.44:0/4190925293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf51300 0x5640fbf51780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.647+0000 7f6257fff700 1 -- 172.21.15.44:0/4190925293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51cc0 msgr2=0x5640fbf5df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.647+0000 7f6257fff700 1 --2- 172.21.15.44:0/4190925293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51cc0 0x5640fbf5df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.647+0000 7f6257fff700 1 -- 172.21.15.44:0/4190925293 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640fbccfd80 con 0x5640fbf571a0 2023-09-10T01:26:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.647+0000 7f6257fff700 1 --2- 172.21.15.44:0/4190925293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 0x5640fbf57580 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6250009cd0 tx=0x7f625000e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=535887b26508ed30 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.647+0000 7f624dffb700 1 -- 172.21.15.44:0/4190925293 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625000ec00 con 0x5640fbf571a0 2023-09-10T01:26:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.648+0000 7f6254ff9700 1 -- 172.21.15.44:0/407909435 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f6248015400 con 0x7f6258106650 2023-09-10T01:26:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.649+0000 7f6254ff9700 1 --2- 172.21.15.44:0/407909435 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f62440a0900 0x7f62440a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.649+0000 7f6254ff9700 1 -- 172.21.15.44:0/407909435 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 9496+0+0 (secure 0 0 0) 0x7f62480d16f0 con 0x7f6258106650 2023-09-10T01:26:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.649+0000 7f624dffb700 1 -- 172.21.15.44:0/4190925293 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f625000ed60 con 0x5640fbf571a0 2023-09-10T01:26:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.649+0000 7f624dffb700 1 -- 172.21.15.44:0/4190925293 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6250005680 con 0x5640fbf571a0 2023-09-10T01:26:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.650+0000 7f626a36eb80 1 -- 172.21.15.44:0/4190925293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 msgr2=0x5640fbf57580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.650+0000 7f626a36eb80 1 --2- 172.21.15.44:0/4190925293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 0x5640fbf57580 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6250009cd0 tx=0x7f625000e350 comp rx=0 tx=0).stop 2023-09-10T01:26:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.651+0000 7f62577fe700 1 --2- 172.21.15.44:0/407909435 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f62440a0900 0x7f62440a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.653+0000 7f62577fe700 1 --2- 172.21.15.44:0/407909435 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f62440a0900 0x7f62440a2dc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6240009990 tx=0x7f6240005fb0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.670+0000 7f626a36eb80 1 -- 172.21.15.44:0/4190925293 shutdown_connections 2023-09-10T01:26:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.670+0000 7f626a36eb80 1 --2- 172.21.15.44:0/4190925293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51cc0 0x5640fbf5df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.670+0000 7f626a36eb80 1 --2- 172.21.15.44:0/4190925293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 0x5640fbf57580 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.670+0000 7f626a36eb80 1 --2- 172.21.15.44:0/4190925293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf51300 0x5640fbf51780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.670+0000 7f626a36eb80 1 -- 172.21.15.44:0/4190925293 >> 172.21.15.44:0/4190925293 conn(0x5640fbec6030 msgr2=0x5640fbf4c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.673+0000 7f626a36eb80 1 -- 172.21.15.44:0/4190925293 shutdown_connections 2023-09-10T01:26:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.673+0000 7f626a36eb80 1 -- 172.21.15.44:0/4190925293 wait complete. 2023-09-10T01:26:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.674+0000 7f626a36eb80 1 -- start start 2023-09-10T01:26:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.674+0000 7f626a36eb80 1 Processor -- start 2023-09-10T01:26:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.674+0000 7f626a36eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf51300 0x5640fbf5dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.674+0000 7f626a36eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51cc0 0x5640fbf5afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.674+0000 7f626a36eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf571a0 0x5640fbf5b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.674+0000 7f626a36eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640fbe41b10 con 0x5640fbf51cc0 2023-09-10T01:26:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.674+0000 7f626a36eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640fbe41810 con 0x5640fbf571a0 2023-09-10T01:26:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.674+0000 7f626a36eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640fbd258c0 con 0x5640fbf51300 2023-09-10T01:26:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.675+0000 7f626a36eb80 10 client.0 _ll_register_callbacks cb 0x5640fbd8ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:26:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.675+0000 7f6257fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf571a0 0x5640fbf5b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.675+0000 7f62577fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf51300 0x5640fbf5dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.675+0000 7f62577fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf51300 0x5640fbf5dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38306/0 (socket says 172.21.15.44:38306) 2023-09-10T01:26:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.675+0000 7f62577fe700 1 -- 172.21.15.44:0/50126087 learned_addr learned my addr 172.21.15.44:0/50126087 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.678+0000 7f6256ffd700 1 --2- 172.21.15.44:0/50126087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51cc0 0x5640fbf5afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.681+0000 7f62577fe700 1 -- 172.21.15.44:0/50126087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf571a0 msgr2=0x5640fbf5b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.681+0000 7f62577fe700 1 --2- 172.21.15.44:0/50126087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf571a0 0x5640fbf5b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.682+0000 7f62577fe700 1 -- 172.21.15.44:0/50126087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51cc0 msgr2=0x5640fbf5afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.682+0000 7f62577fe700 1 --2- 172.21.15.44:0/50126087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51cc0 0x5640fbf5afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.682+0000 7f62577fe700 1 -- 172.21.15.44:0/50126087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6250009980 con 0x5640fbf51300 2023-09-10T01:26:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.682+0000 7f6256ffd700 1 --2- 172.21.15.44:0/50126087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51cc0 0x5640fbf5afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:26:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.686+0000 7f62577fe700 1 --2- 172.21.15.44:0/50126087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf51300 0x5640fbf5dd90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6240015000 tx=0x7f6240009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79914003f7c05219 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.686+0000 7f6236ffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:26:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.686+0000 7f6236ffd700 1 -- 172.21.15.44:0/50126087 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240003e90 con 0x5640fbf51300 2023-09-10T01:26:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.686+0000 7f6236ffd700 1 -- 172.21.15.44:0/50126087 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 9496+0+0 (secure 0 0 0) 0x7f6240023460 con 0x5640fbf51300 2023-09-10T01:26:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.686+0000 7f626a36eb80 1 -- 172.21.15.44:0/50126087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640fbdfb490 con 0x5640fbf51300 2023-09-10T01:26:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.688+0000 7f6236ffd700 1 -- 172.21.15.44:0/50126087 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=155) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6240023620 con 0x5640fbf51300 2023-09-10T01:26:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.692+0000 7f626a36eb80 10 client.29857 fetch_fsmap learned FSMap version 155 2023-09-10T01:26:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.692+0000 7f626a36eb80 1 -- 172.21.15.44:0/50126087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=155,monmap=4+}) v3 -- 0x5640fbcb8c00 con 0x5640fbf51300 2023-09-10T01:26:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.693+0000 7f6236ffd700 1 -- 172.21.15.44:0/50126087 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5640fbdfb490 con 0x5640fbf51300 2023-09-10T01:26:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.693+0000 7f6236ffd700 1 -- 172.21.15.44:0/50126087 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 155) v1 ==== 7383+0+0 (secure 0 0 0) 0x7f62400107e0 con 0x5640fbf51300 2023-09-10T01:26:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.701+0000 7f626a36eb80 10 client.29857 fetch_fsmap finished waiting for FSMap version 155 2023-09-10T01:26:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.701+0000 7f626a36eb80 10 client.29857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:26:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.701+0000 7f626a36eb80 20 client.29857 populate_metadata read hostname 'smithi044' 2023-09-10T01:26:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.701+0000 7f626a36eb80 1 --2- 172.21.15.44:0/50126087 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5640fbf7c390 0x5640fbf7e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.701+0000 7f626a36eb80 4 client.29857 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:26:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.701+0000 7f626a36eb80 1 -- 172.21.15.44:0/50126087 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640fbeb8840 con 0x5640fbf7c390 2023-09-10T01:26:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.701+0000 7f6256ffd700 1 --2- 172.21.15.44:0/50126087 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5640fbf7c390 0x5640fbf7e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.702+0000 7f6256ffd700 1 --2- 172.21.15.44:0/50126087 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5640fbf7c390 0x5640fbf7e7b0 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1a0f569961fa692 server_cookie=5cfa6bb5858268ef in_seq=0 out_seq=0 2023-09-10T01:26:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.703+0000 7f6236ffd700 10 client.29857 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:26:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.703+0000 7f6236ffd700 1 -- 172.21.15.44:0/50126087 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5640fbeb8840 con 0x5640fbf7c390 2023-09-10T01:26:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.703+0000 7f6236ffd700 10 client.29857 handle_command_reply: tid=0 2023-09-10T01:26:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.709+0000 7f626a36eb80 1 client.29857 shutdown 2023-09-10T01:26:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.714+0000 7f626a36eb80 1 -- 172.21.15.44:0/50126087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf51300 msgr2=0x5640fbf5dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.714+0000 7f626a36eb80 1 --2- 172.21.15.44:0/50126087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf51300 0x5640fbf5dd90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6240015000 tx=0x7f6240009cd0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.715+0000 7f626a36eb80 1 -- 172.21.15.44:0/50126087 shutdown_connections 2023-09-10T01:26:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.715+0000 7f626a36eb80 1 --2- 172.21.15.44:0/50126087 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5640fbf7c390 0x5640fbf7e7b0 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.715+0000 7f626a36eb80 1 --2- 172.21.15.44:0/50126087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf571a0 0x5640fbf5b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.715+0000 7f626a36eb80 1 --2- 172.21.15.44:0/50126087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf51300 0x5640fbf5dd90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.715+0000 7f626a36eb80 1 --2- 172.21.15.44:0/50126087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51cc0 0x5640fbf5afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.715+0000 7f626a36eb80 1 -- 172.21.15.44:0/50126087 >> 172.21.15.44:0/50126087 conn(0x5640fbec6030 msgr2=0x5640fbf4b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.715+0000 7f6236ffd700 0 client.29857 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:26:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.715+0000 7f626a36eb80 1 -- 172.21.15.44:0/50126087 shutdown_connections 2023-09-10T01:26:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.715+0000 7f626a36eb80 1 -- 172.21.15.44:0/50126087 wait complete. 2023-09-10T01:26:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.715+0000 7f626a36eb80 20 client.29857 trim_cache size 0 max 16384 2023-09-10T01:26:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.743+0000 7f626a36eb80 1 Processor -- start 2023-09-10T01:26:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.743+0000 7f626a36eb80 1 -- start start 2023-09-10T01:26:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.744+0000 7f626a36eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51300 0x5640fbf566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.744+0000 7f626a36eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf51cc0 0x5640fbf62520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.744+0000 7f626a36eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 0x5640fbf5d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.744+0000 7f626a36eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640fbf48fc0 con 0x5640fbf51300 2023-09-10T01:26:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.744+0000 7f626a36eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640fbf5b850 con 0x5640fbf51cc0 2023-09-10T01:26:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.744+0000 7f626a36eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640fbf57910 con 0x5640fbf571a0 2023-09-10T01:26:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.744+0000 7f62577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51300 0x5640fbf566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.744+0000 7f62577fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51300 0x5640fbf566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59660/0 (socket says 172.21.15.44:59660) 2023-09-10T01:26:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.744+0000 7f62577fe700 1 -- 172.21.15.44:0/1254945076 learned_addr learned my addr 172.21.15.44:0/1254945076 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.744+0000 7f6257fff700 1 --2- 172.21.15.44:0/1254945076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 0x5640fbf5d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.744+0000 7f6257fff700 1 -- 172.21.15.44:0/1254945076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf51cc0 msgr2=0x5640fbf62520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.744+0000 7f6257fff700 1 --2- 172.21.15.44:0/1254945076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf51cc0 0x5640fbf62520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.744+0000 7f6257fff700 1 -- 172.21.15.44:0/1254945076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51300 msgr2=0x5640fbf566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.744+0000 7f6257fff700 1 --2- 172.21.15.44:0/1254945076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51300 0x5640fbf566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.744+0000 7f6257fff700 1 -- 172.21.15.44:0/1254945076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640fbcf61f0 con 0x5640fbf571a0 2023-09-10T01:26:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.745+0000 7f6257fff700 1 --2- 172.21.15.44:0/1254945076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 0x5640fbf5d8c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6250009ca0 tx=0x7f6250004700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ea71ba6bf7588e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.745+0000 7f624e7fc700 1 -- 172.21.15.44:0/1254945076 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625000eed0 con 0x5640fbf571a0 2023-09-10T01:26:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.745+0000 7f624e7fc700 1 -- 172.21.15.44:0/1254945076 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6250004cc0 con 0x5640fbf571a0 2023-09-10T01:26:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.745+0000 7f624e7fc700 1 -- 172.21.15.44:0/1254945076 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62500053b0 con 0x5640fbf571a0 2023-09-10T01:26:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.746+0000 7f626a36eb80 1 -- 172.21.15.44:0/1254945076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 msgr2=0x5640fbf5d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.746+0000 7f626a36eb80 1 --2- 172.21.15.44:0/1254945076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 0x5640fbf5d8c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6250009ca0 tx=0x7f6250004700 comp rx=0 tx=0).stop 2023-09-10T01:26:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.746+0000 7f626a36eb80 1 -- 172.21.15.44:0/1254945076 shutdown_connections 2023-09-10T01:26:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.746+0000 7f626a36eb80 1 --2- 172.21.15.44:0/1254945076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 0x5640fbf5d8c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.746+0000 7f626a36eb80 1 --2- 172.21.15.44:0/1254945076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51300 0x5640fbf566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.746+0000 7f626a36eb80 1 --2- 172.21.15.44:0/1254945076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf51cc0 0x5640fbf62520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.746+0000 7f626a36eb80 1 -- 172.21.15.44:0/1254945076 >> 172.21.15.44:0/1254945076 conn(0x5640fbec6030 msgr2=0x5640fbf55f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.748+0000 7f626a36eb80 1 -- 172.21.15.44:0/1254945076 shutdown_connections 2023-09-10T01:26:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.748+0000 7f626a36eb80 1 -- 172.21.15.44:0/1254945076 wait complete. 2023-09-10T01:26:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.748+0000 7f626a36eb80 1 -- start start 2023-09-10T01:26:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.748+0000 7f626a36eb80 1 Processor -- start 2023-09-10T01:26:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.749+0000 7f626a36eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51300 0x5640fbf602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.749+0000 7f626a36eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf51cc0 0x5640fbf60810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.749+0000 7f626a36eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 0x5640fbfcf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.749+0000 7f626a36eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640fbfcf700 con 0x5640fbf51300 2023-09-10T01:26:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.749+0000 7f626a36eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640fbfcf840 con 0x5640fbf51cc0 2023-09-10T01:26:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.749+0000 7f626a36eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640fbfcf980 con 0x5640fbf571a0 2023-09-10T01:26:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.749+0000 7f626a36eb80 10 client.0 _ll_register_callbacks cb 0x7f624000be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:26:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.749+0000 7f6256ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf51cc0 0x5640fbf60810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.749+0000 7f6256ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf51cc0 0x5640fbf60810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33174/0 (socket says 172.21.15.44:33174) 2023-09-10T01:26:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.749+0000 7f6256ffd700 1 -- 172.21.15.44:0/298509745 learned_addr learned my addr 172.21.15.44:0/298509745 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.750+0000 7f62577fe700 1 --2- 172.21.15.44:0/298509745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51300 0x5640fbf602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.750+0000 7f6256ffd700 1 -- 172.21.15.44:0/298509745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 msgr2=0x5640fbfcf1c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:26:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.750+0000 7f6256ffd700 1 --2- 172.21.15.44:0/298509745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 0x5640fbfcf1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.750+0000 7f6256ffd700 1 -- 172.21.15.44:0/298509745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51300 msgr2=0x5640fbf602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.750+0000 7f6256ffd700 1 --2- 172.21.15.44:0/298509745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51300 0x5640fbf602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.750+0000 7f6256ffd700 1 -- 172.21.15.44:0/298509745 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625001c040 con 0x5640fbf51cc0 2023-09-10T01:26:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.756+0000 7f6256ffd700 1 --2- 172.21.15.44:0/298509745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf51cc0 0x5640fbf60810 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f62480cf070 tx=0x7f6248009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cc18f977c6fc018 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.756+0000 7f6236ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:26:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.757+0000 7f6236ffd700 1 -- 172.21.15.44:0/298509745 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248009e20 con 0x5640fbf51cc0 2023-09-10T01:26:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.757+0000 7f6236ffd700 1 -- 172.21.15.44:0/298509745 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 9496+0+0 (secure 0 0 0) 0x7f624809eed0 con 0x5640fbf51cc0 2023-09-10T01:26:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.757+0000 7f626a36eb80 1 -- 172.21.15.44:0/298509745 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f625000ed60 con 0x5640fbf51cc0 2023-09-10T01:26:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.758+0000 7f6236ffd700 1 -- 172.21.15.44:0/298509745 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=155) v2 ==== 24+0+0 (secure 0 0 0) 0x7f624809e7d0 con 0x5640fbf51cc0 2023-09-10T01:26:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.758+0000 7f626a36eb80 10 client.29864 fetch_fsmap learned FSMap version 155 2023-09-10T01:26:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.758+0000 7f626a36eb80 1 -- 172.21.15.44:0/298509745 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=155,monmap=4+}) v3 -- 0x5640fbcce250 con 0x5640fbf51cc0 2023-09-10T01:26:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.759+0000 7f6236ffd700 1 -- 172.21.15.44:0/298509745 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6248081020 con 0x5640fbf51cc0 2023-09-10T01:26:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.759+0000 7f6236ffd700 1 -- 172.21.15.44:0/298509745 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 155) v1 ==== 7383+0+0 (secure 0 0 0) 0x7f624809eb30 con 0x5640fbf51cc0 2023-09-10T01:26:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.759+0000 7f626a36eb80 10 client.29864 fetch_fsmap finished waiting for FSMap version 155 2023-09-10T01:26:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.759+0000 7f626a36eb80 10 client.29864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:26:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.759+0000 7f626a36eb80 20 client.29864 populate_metadata read hostname 'smithi044' 2023-09-10T01:26:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.759+0000 7f626a36eb80 1 --2- 172.21.15.44:0/298509745 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5640fbfd12a0 0x5640fbfd36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.759+0000 7f626a36eb80 4 client.29864 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:26:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.759+0000 7f626a36eb80 1 -- 172.21.15.44:0/298509745 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5640fbe9c8c0 con 0x5640fbfd12a0 2023-09-10T01:26:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.761+0000 7f62577fe700 1 --2- 172.21.15.44:0/298509745 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5640fbfd12a0 0x5640fbfd36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.763+0000 7f6236ffd700 10 client.29864 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:26:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.763+0000 7f62577fe700 1 --2- 172.21.15.44:0/298509745 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5640fbfd12a0 0x5640fbfd36c0 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac4d6ac36c112f2 server_cookie=671254e6acc03019 in_seq=0 out_seq=0 2023-09-10T01:26:37.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.786+0000 7f6236ffd700 1 -- 172.21.15.44:0/298509745 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5640fbe9c8c0 con 0x5640fbfd12a0 2023-09-10T01:26:37.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.786+0000 7f6236ffd700 10 client.29864 handle_command_reply: tid=0 2023-09-10T01:26:37.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.786+0000 7f626a36eb80 1 client.29864 shutdown 2023-09-10T01:26:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.789+0000 7f626a36eb80 1 -- 172.21.15.44:0/298509745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf51cc0 msgr2=0x5640fbf60810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.789+0000 7f626a36eb80 1 --2- 172.21.15.44:0/298509745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf51cc0 0x5640fbf60810 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f62480cf070 tx=0x7f6248009650 comp rx=0 tx=0).stop 2023-09-10T01:26:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.789+0000 7f626a36eb80 1 -- 172.21.15.44:0/298509745 shutdown_connections 2023-09-10T01:26:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.789+0000 7f626a36eb80 1 --2- 172.21.15.44:0/298509745 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5640fbfd12a0 0x5640fbfd36c0 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.789+0000 7f626a36eb80 1 --2- 172.21.15.44:0/298509745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5640fbf571a0 0x5640fbfcf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.789+0000 7f626a36eb80 1 --2- 172.21.15.44:0/298509745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fbf51300 0x5640fbf602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.789+0000 7f626a36eb80 1 --2- 172.21.15.44:0/298509745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640fbf51cc0 0x5640fbf60810 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.789+0000 7f626a36eb80 1 -- 172.21.15.44:0/298509745 >> 172.21.15.44:0/298509745 conn(0x5640fbec6030 msgr2=0x5640fbf502f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.791+0000 7f6236ffd700 0 client.29864 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:26:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.791+0000 7f626a36eb80 1 -- 172.21.15.44:0/298509745 shutdown_connections 2023-09-10T01:26:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.791+0000 7f626a36eb80 1 -- 172.21.15.44:0/298509745 wait complete. 2023-09-10T01:26:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.792+0000 7f626a36eb80 20 client.29864 trim_cache size 0 max 16384 2023-09-10T01:26:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:26:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:26:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:26:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:26:37.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:37 smithi131 ceph-mon[103713]: pgmap v1490: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.0 MiB/s wr, 818 op/s 2023-09-10T01:26:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.796+0000 7f625e1a8700 1 -- 172.21.15.44:0/407909435 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f62440a0900 msgr2=0x7f62440a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.796+0000 7f625e1a8700 1 --2- 172.21.15.44:0/407909435 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f62440a0900 0x7f62440a2dc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6240009990 tx=0x7f6240005fb0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.796+0000 7f625e1a8700 1 -- 172.21.15.44:0/407909435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6258106650 msgr2=0x7f62581999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.796+0000 7f625e1a8700 1 --2- 172.21.15.44:0/407909435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6258106650 0x7f62581999c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f62480094f0 tx=0x7f624800d3b0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.797+0000 7f625e1a8700 1 -- 172.21.15.44:0/407909435 shutdown_connections 2023-09-10T01:26:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.797+0000 7f625e1a8700 1 --2- 172.21.15.44:0/407909435 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f62440a0900 0x7f62440a2dc0 secure :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6240009990 tx=0x7f6240005fb0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.797+0000 7f625e1a8700 1 --2- 172.21.15.44:0/407909435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f625810c160 0x7f625810c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.797+0000 7f625e1a8700 1 --2- 172.21.15.44:0/407909435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62580683b0 0x7f6258199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.797+0000 7f625e1a8700 1 --2- 172.21.15.44:0/407909435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6258106650 0x7f62581999c0 secure :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f62480094f0 tx=0x7f624800d3b0 comp rx=0 tx=0).stop 2023-09-10T01:26:37.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.797+0000 7f625e1a8700 1 -- 172.21.15.44:0/407909435 >> 172.21.15.44:0/407909435 conn(0x7f6258074f30 msgr2=0x7f62580fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.800+0000 7f625e1a8700 1 -- 172.21.15.44:0/407909435 shutdown_connections 2023-09-10T01:26:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:37.800+0000 7f625e1a8700 1 -- 172.21.15.44:0/407909435 wait complete. 2023-09-10T01:26:37.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:37 smithi044 ceph-mon[98676]: pgmap v1490: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.0 MiB/s wr, 818 op/s 2023-09-10T01:26:37.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:37 smithi099 ceph-mon[102692]: pgmap v1490: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.0 MiB/s wr, 818 op/s 2023-09-10T01:26:39.801 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:39 smithi131 ceph-mon[103713]: pgmap v1491: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 688 op/s 2023-09-10T01:26:39.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:39 smithi044 ceph-mon[98676]: pgmap v1491: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 688 op/s 2023-09-10T01:26:39.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:39 smithi099 ceph-mon[102692]: pgmap v1491: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 688 op/s 2023-09-10T01:26:41.187 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 335.0 s, 256.4 tps, lat 395.580 ms stddev 176.336 2023-09-10T01:26:41.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:41 smithi131 ceph-mon[103713]: pgmap v1492: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 682 op/s 2023-09-10T01:26:41.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:41 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:26:41.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:41 smithi044 ceph-mon[98676]: pgmap v1492: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 682 op/s 2023-09-10T01:26:41.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:41 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:26:41.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:41 smithi099 ceph-mon[102692]: pgmap v1492: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 682 op/s 2023-09-10T01:26:41.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:41 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:26:43.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:43 smithi131 ceph-mon[103713]: pgmap v1493: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.5 MiB/s wr, 942 op/s 2023-09-10T01:26:43.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:43 smithi044 ceph-mon[98676]: pgmap v1493: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.5 MiB/s wr, 942 op/s 2023-09-10T01:26:43.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:43 smithi099 ceph-mon[102692]: pgmap v1493: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.5 MiB/s wr, 942 op/s 2023-09-10T01:26:45.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:45 smithi131 ceph-mon[103713]: pgmap v1494: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.8 MiB/s wr, 695 op/s 2023-09-10T01:26:45.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:45 smithi044 ceph-mon[98676]: pgmap v1494: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.8 MiB/s wr, 695 op/s 2023-09-10T01:26:45.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:45 smithi099 ceph-mon[102692]: pgmap v1494: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.8 MiB/s wr, 695 op/s 2023-09-10T01:26:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 340.0 s, 231.8 tps, lat 445.579 ms stddev 614.575 2023-09-10T01:26:47.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:47 smithi131 ceph-mon[103713]: pgmap v1495: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.5 MiB/s wr, 952 op/s 2023-09-10T01:26:47.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:47 smithi044 ceph-mon[98676]: pgmap v1495: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.5 MiB/s wr, 952 op/s 2023-09-10T01:26:47.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:47 smithi099 ceph-mon[102692]: pgmap v1495: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.5 MiB/s wr, 952 op/s 2023-09-10T01:26:48.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:48 smithi131 ceph-mon[103713]: pgmap v1496: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 615 op/s 2023-09-10T01:26:48.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:48 smithi044 ceph-mon[98676]: pgmap v1496: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 615 op/s 2023-09-10T01:26:48.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:48 smithi099 ceph-mon[102692]: pgmap v1496: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 615 op/s 2023-09-10T01:26:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.017+0000 7f104d341700 1 -- 172.21.15.44:0/2650755575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10480ff760 msgr2=0x7f10480ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.017+0000 7f104d341700 1 --2- 172.21.15.44:0/2650755575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10480ff760 0x7f10480ffbe0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f104000d330 tx=0x7f104000d640 comp rx=0 tx=0).stop 2023-09-10T01:26:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.025+0000 7f104d341700 1 -- 172.21.15.44:0/2650755575 shutdown_connections 2023-09-10T01:26:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.025+0000 7f104d341700 1 --2- 172.21.15.44:0/2650755575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1048100120 0x7f104810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.025+0000 7f104d341700 1 --2- 172.21.15.44:0/2650755575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10481055b0 0x7f1048105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.025+0000 7f104d341700 1 --2- 172.21.15.44:0/2650755575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10480ff760 0x7f10480ffbe0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.025+0000 7f104d341700 1 -- 172.21.15.44:0/2650755575 >> 172.21.15.44:0/2650755575 conn(0x7f1048074f30 msgr2=0x7f1048075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.030+0000 7f104d341700 1 -- 172.21.15.44:0/2650755575 shutdown_connections 2023-09-10T01:26:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.030+0000 7f104d341700 1 -- 172.21.15.44:0/2650755575 wait complete. 2023-09-10T01:26:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.031+0000 7f104d341700 1 Processor -- start 2023-09-10T01:26:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.031+0000 7f104d341700 1 -- start start 2023-09-10T01:26:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.032+0000 7f104d341700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10480ff760 0x7f1048194370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.032+0000 7f104d341700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1048100120 0x7f10481948b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.032+0000 7f104d341700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10481055b0 0x7f1048198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.032+0000 7f104d341700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1048199560 con 0x7f10481055b0 2023-09-10T01:26:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.032+0000 7f104d341700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10481996d0 con 0x7f10480ff760 2023-09-10T01:26:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.032+0000 7f104d341700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1048199890 con 0x7f1048100120 2023-09-10T01:26:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.032+0000 7f10467fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1048100120 0x7f10481948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.032+0000 7f10467fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1048100120 0x7f10481948b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38328/0 (socket says 172.21.15.44:38328) 2023-09-10T01:26:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.032+0000 7f10467fc700 1 -- 172.21.15.44:0/2236259768 learned_addr learned my addr 172.21.15.44:0/2236259768 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.032+0000 7f10477fe700 1 --2- 172.21.15.44:0/2236259768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10481055b0 0x7f1048198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.032+0000 7f1046ffd700 1 --2- 172.21.15.44:0/2236259768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10480ff760 0x7f1048194370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.033+0000 7f10467fc700 1 -- 172.21.15.44:0/2236259768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10480ff760 msgr2=0x7f1048194370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.033+0000 7f10467fc700 1 --2- 172.21.15.44:0/2236259768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10480ff760 0x7f1048194370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.033+0000 7f10467fc700 1 -- 172.21.15.44:0/2236259768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10481055b0 msgr2=0x7f1048198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.033+0000 7f10467fc700 1 --2- 172.21.15.44:0/2236259768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10481055b0 0x7f1048198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.033+0000 7f10467fc700 1 -- 172.21.15.44:0/2236259768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1040009d00 con 0x7f1048100120 2023-09-10T01:26:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.033+0000 7f10467fc700 1 --2- 172.21.15.44:0/2236259768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1048100120 0x7f10481948b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f104000dac0 tx=0x7f1040006070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.034+0000 7f102ffff700 1 -- 172.21.15.44:0/2236259768 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10400075c0 con 0x7f1048100120 2023-09-10T01:26:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.034+0000 7f102ffff700 1 -- 172.21.15.44:0/2236259768 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1040005040 con 0x7f1048100120 2023-09-10T01:26:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.034+0000 7f102ffff700 1 -- 172.21.15.44:0/2236259768 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104001f660 con 0x7f1048100120 2023-09-10T01:26:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.034+0000 7f104d341700 1 -- 172.21.15.44:0/2236259768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1048199b10 con 0x7f1048100120 2023-09-10T01:26:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.034+0000 7f104d341700 1 -- 172.21.15.44:0/2236259768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10481a13b0 con 0x7f1048100120 2023-09-10T01:26:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.037+0000 7f102ffff700 1 -- 172.21.15.44:0/2236259768 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f10400051b0 con 0x7f1048100120 2023-09-10T01:26:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.038+0000 7f102ffff700 1 --2- 172.21.15.44:0/2236259768 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f10300a0940 0x7f10300a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.038+0000 7f102ffff700 1 -- 172.21.15.44:0/2236259768 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 9496+0+0 (secure 0 0 0) 0x7f10400d5a00 con 0x7f1048100120 2023-09-10T01:26:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.038+0000 7f1046ffd700 1 --2- 172.21.15.44:0/2236259768 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f10300a0940 0x7f10300a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.039+0000 7f1046ffd700 1 --2- 172.21.15.44:0/2236259768 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f10300a0940 0x7f10300a2e00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f103c005950 tx=0x7f103c00b410 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.043+0000 7f1059507b80 1 Processor -- start 2023-09-10T01:26:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.043+0000 7f1059507b80 1 -- start start 2023-09-10T01:26:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.043+0000 7f1059507b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da081a0 0x55f40da08580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.043+0000 7f1059507b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da02300 0x55f40da02780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.043+0000 7f1059507b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da02cc0 0x55f40da0ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.043+0000 7f1059507b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f40d95b6d0 con 0x55f40da02cc0 2023-09-10T01:26:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.043+0000 7f1059507b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f40d958ff0 con 0x55f40da081a0 2023-09-10T01:26:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.043+0000 7f1059507b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f40d9548e0 con 0x55f40da02300 2023-09-10T01:26:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.045+0000 7f10467fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da02300 0x55f40da02780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.045+0000 7f10477fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da02cc0 0x55f40da0ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.045+0000 7f10477fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da02cc0 0x55f40da0ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59688/0 (socket says 172.21.15.44:59688) 2023-09-10T01:26:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.045+0000 7f10477fe700 1 -- 172.21.15.44:0/82517003 learned_addr learned my addr 172.21.15.44:0/82517003 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.045+0000 7f10477fe700 1 -- 172.21.15.44:0/82517003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da02300 msgr2=0x55f40da02780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.045+0000 7f10477fe700 1 --2- 172.21.15.44:0/82517003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da02300 0x55f40da02780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.045+0000 7f10477fe700 1 -- 172.21.15.44:0/82517003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da081a0 msgr2=0x55f40da08580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:26:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.045+0000 7f10477fe700 1 --2- 172.21.15.44:0/82517003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da081a0 0x55f40da08580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.045+0000 7f10477fe700 1 -- 172.21.15.44:0/82517003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f40d780d80 con 0x55f40da02cc0 2023-09-10T01:26:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.046+0000 7f10477fe700 1 --2- 172.21.15.44:0/82517003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da02cc0 0x55f40da0ef30 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f103800bc30 tx=0x7f103800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba19943f624424f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.046+0000 7f102d7fa700 1 -- 172.21.15.44:0/82517003 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1038012070 con 0x55f40da02cc0 2023-09-10T01:26:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.046+0000 7f102d7fa700 1 -- 172.21.15.44:0/82517003 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1038004830 con 0x55f40da02cc0 2023-09-10T01:26:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.046+0000 7f102d7fa700 1 -- 172.21.15.44:0/82517003 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10380054f0 con 0x55f40da02cc0 2023-09-10T01:26:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.047+0000 7f1059507b80 1 -- 172.21.15.44:0/82517003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da02cc0 msgr2=0x55f40da0ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.047+0000 7f1059507b80 1 --2- 172.21.15.44:0/82517003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da02cc0 0x55f40da0ef30 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f103800bc30 tx=0x7f103800bf40 comp rx=0 tx=0).stop 2023-09-10T01:26:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.047+0000 7f1059507b80 1 -- 172.21.15.44:0/82517003 shutdown_connections 2023-09-10T01:26:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.047+0000 7f1059507b80 1 --2- 172.21.15.44:0/82517003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da02cc0 0x55f40da0ef30 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.047+0000 7f1059507b80 1 --2- 172.21.15.44:0/82517003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da081a0 0x55f40da08580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.047+0000 7f1059507b80 1 --2- 172.21.15.44:0/82517003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da02300 0x55f40da02780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.047+0000 7f1059507b80 1 -- 172.21.15.44:0/82517003 >> 172.21.15.44:0/82517003 conn(0x55f40d977030 msgr2=0x55f40d9fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.048+0000 7f1059507b80 1 -- 172.21.15.44:0/82517003 shutdown_connections 2023-09-10T01:26:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.049+0000 7f1059507b80 1 -- 172.21.15.44:0/82517003 wait complete. 2023-09-10T01:26:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.049+0000 7f1059507b80 1 -- start start 2023-09-10T01:26:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.049+0000 7f1059507b80 1 Processor -- start 2023-09-10T01:26:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.049+0000 7f1059507b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da02300 0x55f40da0edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.049+0000 7f1059507b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da02cc0 0x55f40da0c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f1059507b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da081a0 0x55f40da0c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f1059507b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f40d8f2b10 con 0x55f40da02cc0 2023-09-10T01:26:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f1059507b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f40d8f2810 con 0x55f40da02300 2023-09-10T01:26:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f1059507b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f40d7d68c0 con 0x55f40da081a0 2023-09-10T01:26:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f1059507b80 10 client.0 _ll_register_callbacks cb 0x55f40d83ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:26:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f1046ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da02300 0x55f40da0edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f1046ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da02300 0x55f40da0edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33200/0 (socket says 172.21.15.44:33200) 2023-09-10T01:26:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f1046ffd700 1 -- 172.21.15.44:0/273101683 learned_addr learned my addr 172.21.15.44:0/273101683 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f10477fe700 1 --2- 172.21.15.44:0/273101683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da081a0 0x55f40da0c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f10467fc700 1 --2- 172.21.15.44:0/273101683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da02cc0 0x55f40da0c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f1046ffd700 1 -- 172.21.15.44:0/273101683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da081a0 msgr2=0x55f40da0c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f1046ffd700 1 --2- 172.21.15.44:0/273101683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da081a0 0x55f40da0c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f1046ffd700 1 -- 172.21.15.44:0/273101683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da02cc0 msgr2=0x55f40da0c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f1046ffd700 1 --2- 172.21.15.44:0/273101683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da02cc0 0x55f40da0c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f1046ffd700 1 -- 172.21.15.44:0/273101683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f103800b8e0 con 0x55f40da02300 2023-09-10T01:26:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f1046ffd700 1 --2- 172.21.15.44:0/273101683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da02300 0x55f40da0edb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f103c0116c0 tx=0x7f103c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8abaa5439c87a9db server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.050+0000 7f102aff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:26:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.051+0000 7f102aff5700 1 -- 172.21.15.44:0/273101683 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c00be80 con 0x55f40da02300 2023-09-10T01:26:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.051+0000 7f102aff5700 1 -- 172.21.15.44:0/273101683 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 9496+0+0 (secure 0 0 0) 0x7f103c017070 con 0x55f40da02300 2023-09-10T01:26:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.051+0000 7f1059507b80 1 -- 172.21.15.44:0/273101683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f40d8ac490 con 0x55f40da02300 2023-09-10T01:26:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.051+0000 7f102aff5700 1 -- 172.21.15.44:0/273101683 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=156) v2 ==== 24+0+0 (secure 0 0 0) 0x7f103c010e60 con 0x55f40da02300 2023-09-10T01:26:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.051+0000 7f1059507b80 10 client.29873 fetch_fsmap learned FSMap version 156 2023-09-10T01:26:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.051+0000 7f1059507b80 1 -- 172.21.15.44:0/273101683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=156,monmap=4+}) v3 -- 0x55f40d769c00 con 0x55f40da02300 2023-09-10T01:26:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.051+0000 7f102aff5700 1 -- 172.21.15.44:0/273101683 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f40d8ac490 con 0x55f40da02300 2023-09-10T01:26:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.053+0000 7f102aff5700 1 -- 172.21.15.44:0/273101683 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 156) v1 ==== 7379+0+0 (secure 0 0 0) 0x7f103c024020 con 0x55f40da02300 2023-09-10T01:26:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.053+0000 7f1059507b80 10 client.29873 fetch_fsmap finished waiting for FSMap version 156 2023-09-10T01:26:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.053+0000 7f1059507b80 10 client.29873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:26:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.053+0000 7f1059507b80 20 client.29873 populate_metadata read hostname 'smithi044' 2023-09-10T01:26:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.053+0000 7f1059507b80 1 --2- 172.21.15.44:0/273101683 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f40da2d410 0x55f40da2f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.053+0000 7f1059507b80 4 client.29873 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:26:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.053+0000 7f1059507b80 1 -- 172.21.15.44:0/273101683 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f40d969840 con 0x55f40da2d410 2023-09-10T01:26:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.053+0000 7f10467fc700 1 --2- 172.21.15.44:0/273101683 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f40da2d410 0x55f40da2f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.053+0000 7f102aff5700 10 client.29873 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:26:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.053+0000 7f10467fc700 1 --2- 172.21.15.44:0/273101683 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f40da2d410 0x55f40da2f830 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f3131bcb0ae8678 server_cookie=10072c944de33285 in_seq=0 out_seq=0 2023-09-10T01:26:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.054+0000 7f102aff5700 1 -- 172.21.15.44:0/273101683 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f40d969840 con 0x55f40da2d410 2023-09-10T01:26:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.054+0000 7f102aff5700 10 client.29873 handle_command_reply: tid=0 2023-09-10T01:26:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.054+0000 7f1059507b80 1 client.29873 shutdown 2023-09-10T01:26:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.054+0000 7f1059507b80 1 -- 172.21.15.44:0/273101683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da02300 msgr2=0x55f40da0edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.054+0000 7f1059507b80 1 --2- 172.21.15.44:0/273101683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da02300 0x55f40da0edb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f103c0116c0 tx=0x7f103c0047c0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.054+0000 7f1059507b80 1 -- 172.21.15.44:0/273101683 shutdown_connections 2023-09-10T01:26:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.054+0000 7f1059507b80 1 --2- 172.21.15.44:0/273101683 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f40da2d410 0x55f40da2f830 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.054+0000 7f1059507b80 1 --2- 172.21.15.44:0/273101683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da081a0 0x55f40da0c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.054+0000 7f1059507b80 1 --2- 172.21.15.44:0/273101683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da02300 0x55f40da0edb0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.054+0000 7f102aff5700 0 client.29873 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:26:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.054+0000 7f1059507b80 1 --2- 172.21.15.44:0/273101683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da02cc0 0x55f40da0c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.055+0000 7f1059507b80 1 -- 172.21.15.44:0/273101683 >> 172.21.15.44:0/273101683 conn(0x55f40d977030 msgr2=0x55f40d9fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.055+0000 7f1059507b80 1 -- 172.21.15.44:0/273101683 shutdown_connections 2023-09-10T01:26:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.055+0000 7f1059507b80 1 -- 172.21.15.44:0/273101683 wait complete. 2023-09-10T01:26:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.055+0000 7f1059507b80 20 client.29873 trim_cache size 0 max 16384 2023-09-10T01:26:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.097+0000 7f1059507b80 1 Processor -- start 2023-09-10T01:26:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.103+0000 7f1059507b80 1 -- start start 2023-09-10T01:26:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.103+0000 7f1059507b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da02300 0x55f40da0e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.103+0000 7f1059507b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da02cc0 0x55f40da0edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.103+0000 7f1059507b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da081a0 0x55f40d9fcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.103+0000 7f1059507b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f40d9fec80 con 0x55f40da081a0 2023-09-10T01:26:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.103+0000 7f1059507b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f40d9fe320 con 0x55f40da02300 2023-09-10T01:26:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.103+0000 7f1059507b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f40da2a9e0 con 0x55f40da02cc0 2023-09-10T01:26:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.104+0000 7f10477fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da081a0 0x55f40d9fcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.104+0000 7f10467fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da02cc0 0x55f40da0edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.104+0000 7f10477fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da081a0 0x55f40d9fcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59700/0 (socket says 172.21.15.44:59700) 2023-09-10T01:26:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.104+0000 7f10477fe700 1 -- 172.21.15.44:0/3000451738 learned_addr learned my addr 172.21.15.44:0/3000451738 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.104+0000 7f10467fc700 1 -- 172.21.15.44:0/3000451738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da02300 msgr2=0x55f40da0e870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:26:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.104+0000 7f10467fc700 1 --2- 172.21.15.44:0/3000451738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da02300 0x55f40da0e870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.104+0000 7f10467fc700 1 -- 172.21.15.44:0/3000451738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da081a0 msgr2=0x55f40d9fcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.104+0000 7f10467fc700 1 --2- 172.21.15.44:0/3000451738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da081a0 0x55f40d9fcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.104+0000 7f10467fc700 1 -- 172.21.15.44:0/3000451738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f40d7a71f0 con 0x55f40da02cc0 2023-09-10T01:26:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.104+0000 7f10467fc700 1 --2- 172.21.15.44:0/3000451738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da02cc0 0x55f40da0edb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f10400d2d80 tx=0x7f10400099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1630b188d4ecc504 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.105+0000 7f102dffb700 1 -- 172.21.15.44:0/3000451738 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10400a69a0 con 0x55f40da02cc0 2023-09-10T01:26:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.105+0000 7f102dffb700 1 -- 172.21.15.44:0/3000451738 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10400aab50 con 0x55f40da02cc0 2023-09-10T01:26:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.105+0000 7f102dffb700 1 -- 172.21.15.44:0/3000451738 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10400054b0 con 0x55f40da02cc0 2023-09-10T01:26:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.105+0000 7f1059507b80 1 -- 172.21.15.44:0/3000451738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da02cc0 msgr2=0x55f40da0edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.105+0000 7f1059507b80 1 --2- 172.21.15.44:0/3000451738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da02cc0 0x55f40da0edb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f10400d2d80 tx=0x7f10400099e0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.105+0000 7f1059507b80 1 -- 172.21.15.44:0/3000451738 shutdown_connections 2023-09-10T01:26:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.105+0000 7f1059507b80 1 --2- 172.21.15.44:0/3000451738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da081a0 0x55f40d9fcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.105+0000 7f1059507b80 1 --2- 172.21.15.44:0/3000451738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da02300 0x55f40da0e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.105+0000 7f1059507b80 1 --2- 172.21.15.44:0/3000451738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da02cc0 0x55f40da0edb0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.105+0000 7f1059507b80 1 -- 172.21.15.44:0/3000451738 >> 172.21.15.44:0/3000451738 conn(0x55f40d977030 msgr2=0x55f40da01b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.117+0000 7f1059507b80 1 -- 172.21.15.44:0/3000451738 shutdown_connections 2023-09-10T01:26:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.118+0000 7f1059507b80 1 -- 172.21.15.44:0/3000451738 wait complete. 2023-09-10T01:26:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.118+0000 7f1059507b80 1 -- start start 2023-09-10T01:26:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.119+0000 7f1059507b80 1 Processor -- start 2023-09-10T01:26:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.124+0000 7f1059507b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da02cc0 0x55f40da246c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.124+0000 7f1059507b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da081a0 0x55f40da22970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.124+0000 7f1059507b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da23010 0x55f40da19400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.124+0000 7f1059507b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f40da23620 con 0x55f40da02cc0 2023-09-10T01:26:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.124+0000 7f1059507b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f40da23790 con 0x55f40da081a0 2023-09-10T01:26:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.124+0000 7f1059507b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f40da19940 con 0x55f40da23010 2023-09-10T01:26:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.124+0000 7f1059507b80 10 client.0 _ll_register_callbacks cb 0x7f103c010ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:26:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.125+0000 7f10467fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da081a0 0x55f40da22970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.125+0000 7f10477fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da23010 0x55f40da19400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.125+0000 7f10477fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da23010 0x55f40da19400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38360/0 (socket says 172.21.15.44:38360) 2023-09-10T01:26:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.125+0000 7f10477fe700 1 -- 172.21.15.44:0/2576370784 learned_addr learned my addr 172.21.15.44:0/2576370784 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:26:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.125+0000 7f10477fe700 1 -- 172.21.15.44:0/2576370784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da081a0 msgr2=0x55f40da22970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.125+0000 7f10477fe700 1 --2- 172.21.15.44:0/2576370784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da081a0 0x55f40da22970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.125+0000 7f10477fe700 1 -- 172.21.15.44:0/2576370784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da02cc0 msgr2=0x55f40da246c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:26:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.125+0000 7f10477fe700 1 --2- 172.21.15.44:0/2576370784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da02cc0 0x55f40da246c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.125+0000 7f10477fe700 1 -- 172.21.15.44:0/2576370784 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10481a13b0 con 0x55f40da23010 2023-09-10T01:26:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.132+0000 7f10477fe700 1 --2- 172.21.15.44:0/2576370784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da23010 0x55f40da19400 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1038018000 tx=0x7f1038004bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e313e0234f746e2d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:26:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.132+0000 7f102aff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:26:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.132+0000 7f102aff5700 1 -- 172.21.15.44:0/2576370784 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1038012040 con 0x55f40da23010 2023-09-10T01:26:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.132+0000 7f102aff5700 1 -- 172.21.15.44:0/2576370784 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 9496+0+0 (secure 0 0 0) 0x7f10380053e0 con 0x55f40da23010 2023-09-10T01:26:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.135+0000 7f1059507b80 1 -- 172.21.15.44:0/2576370784 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1038004830 con 0x55f40da23010 2023-09-10T01:26:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.139+0000 7f102aff5700 1 -- 172.21.15.44:0/2576370784 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=156) v2 ==== 24+0+0 (secure 0 0 0) 0x7f103800b2c0 con 0x55f40da23010 2023-09-10T01:26:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.139+0000 7f1059507b80 10 client.29878 fetch_fsmap learned FSMap version 156 2023-09-10T01:26:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.139+0000 7f1059507b80 1 -- 172.21.15.44:0/2576370784 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=156,monmap=4+}) v3 -- 0x55f40d77f250 con 0x55f40da23010 2023-09-10T01:26:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.150+0000 7f102aff5700 1 -- 172.21.15.44:0/2576370784 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1038004830 con 0x55f40da23010 2023-09-10T01:26:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.150+0000 7f102aff5700 1 -- 172.21.15.44:0/2576370784 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 156) v1 ==== 7379+0+0 (secure 0 0 0) 0x7f103800bc60 con 0x55f40da23010 2023-09-10T01:26:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.150+0000 7f1059507b80 10 client.29878 fetch_fsmap finished waiting for FSMap version 156 2023-09-10T01:26:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.150+0000 7f1059507b80 10 client.29878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:26:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.150+0000 7f1059507b80 20 client.29878 populate_metadata read hostname 'smithi044' 2023-09-10T01:26:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.150+0000 7f1059507b80 1 --2- 172.21.15.44:0/2576370784 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f40da1af60 0x55f40da1d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:26:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.150+0000 7f1059507b80 4 client.29878 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:26:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.150+0000 7f1059507b80 1 -- 172.21.15.44:0/2576370784 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f40d94d8c0 con 0x55f40da1af60 2023-09-10T01:26:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.152+0000 7f1046ffd700 1 --2- 172.21.15.44:0/2576370784 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f40da1af60 0x55f40da1d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:26:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.154+0000 7f1046ffd700 1 --2- 172.21.15.44:0/2576370784 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f40da1af60 0x55f40da1d380 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dd5db502f0d0e08 server_cookie=72a8479fc14d521b in_seq=0 out_seq=0 2023-09-10T01:26:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.154+0000 7f102aff5700 10 client.29878 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:26:51.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 345.0 s, 256.2 tps, lat 376.299 ms stddev 122.682 2023-09-10T01:26:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.182+0000 7f102aff5700 1 -- 172.21.15.44:0/2576370784 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f40d94d8c0 con 0x55f40da1af60 2023-09-10T01:26:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.182+0000 7f102aff5700 10 client.29878 handle_command_reply: tid=0 2023-09-10T01:26:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.182+0000 7f1059507b80 1 client.29878 shutdown 2023-09-10T01:26:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.184+0000 7f1059507b80 1 -- 172.21.15.44:0/2576370784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da23010 msgr2=0x55f40da19400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.184+0000 7f1059507b80 1 --2- 172.21.15.44:0/2576370784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da23010 0x55f40da19400 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1038018000 tx=0x7f1038004bd0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.184+0000 7f1059507b80 1 -- 172.21.15.44:0/2576370784 shutdown_connections 2023-09-10T01:26:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.184+0000 7f1059507b80 1 --2- 172.21.15.44:0/2576370784 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f40da1af60 0x55f40da1d380 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.184+0000 7f1059507b80 1 --2- 172.21.15.44:0/2576370784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f40da23010 0x55f40da19400 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.184+0000 7f1059507b80 1 --2- 172.21.15.44:0/2576370784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f40da02cc0 0x55f40da246c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.184+0000 7f1059507b80 1 --2- 172.21.15.44:0/2576370784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40da081a0 0x55f40da22970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.184+0000 7f1059507b80 1 -- 172.21.15.44:0/2576370784 >> 172.21.15.44:0/2576370784 conn(0x55f40d977030 msgr2=0x55f40da0a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.184+0000 7f102aff5700 0 client.29878 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:26:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.186+0000 7f1059507b80 1 -- 172.21.15.44:0/2576370784 shutdown_connections 2023-09-10T01:26:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.186+0000 7f1059507b80 1 -- 172.21.15.44:0/2576370784 wait complete. 2023-09-10T01:26:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.186+0000 7f1059507b80 20 client.29878 trim_cache size 0 max 16384 2023-09-10T01:26:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:26:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:26:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:26:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:26:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.187+0000 7f104d341700 1 -- 172.21.15.44:0/2236259768 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f10300a0940 msgr2=0x7f10300a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.187+0000 7f104d341700 1 --2- 172.21.15.44:0/2236259768 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f10300a0940 0x7f10300a2e00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f103c005950 tx=0x7f103c00b410 comp rx=0 tx=0).stop 2023-09-10T01:26:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.187+0000 7f104d341700 1 -- 172.21.15.44:0/2236259768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1048100120 msgr2=0x7f10481948b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:26:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.187+0000 7f104d341700 1 --2- 172.21.15.44:0/2236259768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1048100120 0x7f10481948b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f104000dac0 tx=0x7f1040006070 comp rx=0 tx=0).stop 2023-09-10T01:26:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.187+0000 7f104d341700 1 -- 172.21.15.44:0/2236259768 shutdown_connections 2023-09-10T01:26:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.187+0000 7f104d341700 1 --2- 172.21.15.44:0/2236259768 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f10300a0940 0x7f10300a2e00 secure :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f103c005950 tx=0x7f103c00b410 comp rx=0 tx=0).stop 2023-09-10T01:26:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.187+0000 7f104d341700 1 --2- 172.21.15.44:0/2236259768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10481055b0 0x7f1048198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.187+0000 7f104d341700 1 --2- 172.21.15.44:0/2236259768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10480ff760 0x7f1048194370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:26:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.187+0000 7f104d341700 1 --2- 172.21.15.44:0/2236259768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1048100120 0x7f10481948b0 secure :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f104000dac0 tx=0x7f1040006070 comp rx=0 tx=0).stop 2023-09-10T01:26:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.187+0000 7f104d341700 1 -- 172.21.15.44:0/2236259768 >> 172.21.15.44:0/2236259768 conn(0x7f1048074f30 msgr2=0x7f1048103000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:26:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.192+0000 7f104d341700 1 -- 172.21.15.44:0/2236259768 shutdown_connections 2023-09-10T01:26:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:26:51.192+0000 7f104d341700 1 -- 172.21.15.44:0/2236259768 wait complete. 2023-09-10T01:26:51.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:51 smithi131 ceph-mon[103713]: pgmap v1497: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 615 op/s 2023-09-10T01:26:51.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:51 smithi044 ceph-mon[98676]: pgmap v1497: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 615 op/s 2023-09-10T01:26:51.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:51 smithi099 ceph-mon[102692]: pgmap v1497: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 615 op/s 2023-09-10T01:26:53.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:53 smithi099 ceph-mon[102692]: pgmap v1498: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 9.7 MiB/s wr, 967 op/s 2023-09-10T01:26:53.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:53 smithi131 ceph-mon[103713]: pgmap v1498: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 9.7 MiB/s wr, 967 op/s 2023-09-10T01:26:53.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:53 smithi044 ceph-mon[98676]: pgmap v1498: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 9.7 MiB/s wr, 967 op/s 2023-09-10T01:26:55.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:55 smithi099 ceph-mon[102692]: pgmap v1499: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 608 op/s 2023-09-10T01:26:55.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:55 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:26:55.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:55 smithi131 ceph-mon[103713]: pgmap v1499: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 608 op/s 2023-09-10T01:26:55.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:55 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:26:55.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:55 smithi044 ceph-mon[98676]: pgmap v1499: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 608 op/s 2023-09-10T01:26:55.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:55 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:26:56.184 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 350.0 s, 204.0 tps, lat 506.468 ms stddev 512.470 2023-09-10T01:26:57.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:57 smithi131 ceph-mon[103713]: pgmap v1500: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.7 MiB/s wr, 881 op/s 2023-09-10T01:26:57.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:57 smithi044 ceph-mon[98676]: pgmap v1500: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.7 MiB/s wr, 881 op/s 2023-09-10T01:26:57.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:57 smithi099 ceph-mon[102692]: pgmap v1500: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.7 MiB/s wr, 881 op/s 2023-09-10T01:26:59.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:26:59 smithi131 ceph-mon[103713]: pgmap v1501: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 625 op/s 2023-09-10T01:26:59.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:26:59 smithi044 ceph-mon[98676]: pgmap v1501: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 625 op/s 2023-09-10T01:26:59.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:26:59 smithi099 ceph-mon[102692]: pgmap v1501: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 625 op/s 2023-09-10T01:27:01.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 355.0 s, 258.8 tps, lat 361.718 ms stddev 114.646 2023-09-10T01:27:01.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:01 smithi131 ceph-mon[103713]: pgmap v1502: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 625 op/s 2023-09-10T01:27:01.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:01 smithi044 ceph-mon[98676]: pgmap v1502: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 625 op/s 2023-09-10T01:27:01.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:01 smithi099 ceph-mon[102692]: pgmap v1502: 129 pgs: 129 active+clean; 18 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 625 op/s 2023-09-10T01:27:03.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:03 smithi131 ceph-mon[103713]: pgmap v1503: 129 pgs: 129 active+clean; 18 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.2 MiB/s wr, 969 op/s 2023-09-10T01:27:03.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:03 smithi044 ceph-mon[98676]: pgmap v1503: 129 pgs: 129 active+clean; 18 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.2 MiB/s wr, 969 op/s 2023-09-10T01:27:03.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:03 smithi099 ceph-mon[102692]: pgmap v1503: 129 pgs: 129 active+clean; 18 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.2 MiB/s wr, 969 op/s 2023-09-10T01:27:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.402+0000 7fd7655da700 1 -- 172.21.15.44:0/773876486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd76010f1c0 msgr2=0x7fd760107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.402+0000 7fd7655da700 1 --2- 172.21.15.44:0/773876486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd76010f1c0 0x7fd760107b40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd758009f90 tx=0x7fd75800b330 comp rx=0 tx=0).stop 2023-09-10T01:27:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.405+0000 7fd7655da700 1 -- 172.21.15.44:0/773876486 shutdown_connections 2023-09-10T01:27:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.405+0000 7fd7655da700 1 --2- 172.21.15.44:0/773876486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd760119de0 0x7fd76011c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.405+0000 7fd7655da700 1 --2- 172.21.15.44:0/773876486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd76010ed50 0x7fd760107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.405+0000 7fd7655da700 1 --2- 172.21.15.44:0/773876486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd76010f1c0 0x7fd760107b40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.405+0000 7fd7655da700 1 -- 172.21.15.44:0/773876486 >> 172.21.15.44:0/773876486 conn(0x7fd76006c6c0 msgr2=0x7fd76006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.407+0000 7fd7655da700 1 -- 172.21.15.44:0/773876486 shutdown_connections 2023-09-10T01:27:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.407+0000 7fd7655da700 1 -- 172.21.15.44:0/773876486 wait complete. 2023-09-10T01:27:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.407+0000 7fd7655da700 1 Processor -- start 2023-09-10T01:27:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.408+0000 7fd7655da700 1 -- start start 2023-09-10T01:27:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.408+0000 7fd7655da700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76010ed50 0x7fd760114f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.408+0000 7fd7655da700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd76010f1c0 0x7fd7601154d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.408+0000 7fd7655da700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd760119de0 0x7fd7601a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.408+0000 7fd7655da700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7601a3070 con 0x7fd760119de0 2023-09-10T01:27:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.408+0000 7fd7655da700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7601a31e0 con 0x7fd76010ed50 2023-09-10T01:27:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.408+0000 7fd7655da700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7601a33a0 con 0x7fd76010f1c0 2023-09-10T01:27:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.411+0000 7fd75effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76010ed50 0x7fd760114f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.411+0000 7fd75effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76010ed50 0x7fd760114f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33226/0 (socket says 172.21.15.44:33226) 2023-09-10T01:27:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.411+0000 7fd75effd700 1 -- 172.21.15.44:0/4267344009 learned_addr learned my addr 172.21.15.44:0/4267344009 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.411+0000 7fd75f7fe700 1 --2- 172.21.15.44:0/4267344009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd760119de0 0x7fd7601a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.411+0000 7fd75e7fc700 1 --2- 172.21.15.44:0/4267344009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd76010f1c0 0x7fd7601154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.411+0000 7fd75effd700 1 -- 172.21.15.44:0/4267344009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd76010f1c0 msgr2=0x7fd7601154d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.411+0000 7fd75effd700 1 --2- 172.21.15.44:0/4267344009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd76010f1c0 0x7fd7601154d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.411+0000 7fd75effd700 1 -- 172.21.15.44:0/4267344009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd760119de0 msgr2=0x7fd7601a2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.411+0000 7fd75effd700 1 --2- 172.21.15.44:0/4267344009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd760119de0 0x7fd7601a2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.411+0000 7fd75effd700 1 -- 172.21.15.44:0/4267344009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd758009c40 con 0x7fd76010ed50 2023-09-10T01:27:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.411+0000 7fd75f7fe700 1 --2- 172.21.15.44:0/4267344009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd760119de0 0x7fd7601a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:27:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.411+0000 7fd75effd700 1 --2- 172.21.15.44:0/4267344009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76010ed50 0x7fd760114f90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd75000b550 tx=0x7fd75000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.412+0000 7fd747fff700 1 -- 172.21.15.44:0/4267344009 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd750011830 con 0x7fd76010ed50 2023-09-10T01:27:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.412+0000 7fd747fff700 1 -- 172.21.15.44:0/4267344009 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd750011e20 con 0x7fd76010ed50 2023-09-10T01:27:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.412+0000 7fd747fff700 1 -- 172.21.15.44:0/4267344009 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd750010e10 con 0x7fd76010ed50 2023-09-10T01:27:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.412+0000 7fd7655da700 1 -- 172.21.15.44:0/4267344009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7601a3680 con 0x7fd76010ed50 2023-09-10T01:27:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.412+0000 7fd7655da700 1 -- 172.21.15.44:0/4267344009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7601a9d00 con 0x7fd76010ed50 2023-09-10T01:27:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.415+0000 7fd747fff700 1 -- 172.21.15.44:0/4267344009 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fd750011990 con 0x7fd76010ed50 2023-09-10T01:27:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.416+0000 7fd747fff700 1 --2- 172.21.15.44:0/4267344009 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7480a0870 0x7fd7480a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.416+0000 7fd747fff700 1 -- 172.21.15.44:0/4267344009 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 9496+0+0 (secure 0 0 0) 0x7fd7500cbaf0 con 0x7fd76010ed50 2023-09-10T01:27:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.416+0000 7fd75e7fc700 1 --2- 172.21.15.44:0/4267344009 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7480a0870 0x7fd7480a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.417+0000 7fd75e7fc700 1 --2- 172.21.15.44:0/4267344009 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7480a0870 0x7fd7480a2d30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd75800b770 tx=0x7fd75800baa0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.422+0000 7fd7717a0b80 1 Processor -- start 2023-09-10T01:27:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.422+0000 7fd7717a0b80 1 -- start start 2023-09-10T01:27:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.423+0000 7fd7717a0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb04d1a0 0x555cdb04d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.423+0000 7fd7717a0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb047300 0x555cdb047780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.423+0000 7fd7717a0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb047cc0 0x555cdb053f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.423+0000 7fd7717a0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdafa06d0 con 0x555cdb047cc0 2023-09-10T01:27:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.423+0000 7fd7717a0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdaf9dff0 con 0x555cdb04d1a0 2023-09-10T01:27:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.423+0000 7fd7717a0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdaf998e0 con 0x555cdb047300 2023-09-10T01:27:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.423+0000 7fd75f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb047cc0 0x555cdb053f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.423+0000 7fd75effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb04d1a0 0x555cdb04d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.423+0000 7fd75f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb047cc0 0x555cdb053f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59728/0 (socket says 172.21.15.44:59728) 2023-09-10T01:27:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.423+0000 7fd75f7fe700 1 -- 172.21.15.44:0/43663451 learned_addr learned my addr 172.21.15.44:0/43663451 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.428+0000 7fd75effd700 1 -- 172.21.15.44:0/43663451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb047300 msgr2=0x555cdb047780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.428+0000 7fd75effd700 1 --2- 172.21.15.44:0/43663451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb047300 0x555cdb047780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.428+0000 7fd75effd700 1 -- 172.21.15.44:0/43663451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb047cc0 msgr2=0x555cdb053f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.428+0000 7fd75effd700 1 --2- 172.21.15.44:0/43663451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb047cc0 0x555cdb053f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.428+0000 7fd75effd700 1 -- 172.21.15.44:0/43663451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cdadc5d80 con 0x555cdb04d1a0 2023-09-10T01:27:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.428+0000 7fd75effd700 1 --2- 172.21.15.44:0/43663451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb04d1a0 0x555cdb04d580 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd7500c8f80 tx=0x7fd750095130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e16e978f4189b5b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.432+0000 7fd7457fa700 1 -- 172.21.15.44:0/43663451 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd750011c40 con 0x555cdb04d1a0 2023-09-10T01:27:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.432+0000 7fd7457fa700 1 -- 172.21.15.44:0/43663451 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7500993e0 con 0x555cdb04d1a0 2023-09-10T01:27:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.432+0000 7fd7457fa700 1 -- 172.21.15.44:0/43663451 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7500981d0 con 0x555cdb04d1a0 2023-09-10T01:27:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.436+0000 7fd7717a0b80 1 -- 172.21.15.44:0/43663451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb04d1a0 msgr2=0x555cdb04d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.436+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/43663451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb04d1a0 0x555cdb04d580 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd7500c8f80 tx=0x7fd750095130 comp rx=0 tx=0).stop 2023-09-10T01:27:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.453+0000 7fd7717a0b80 1 -- 172.21.15.44:0/43663451 shutdown_connections 2023-09-10T01:27:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.453+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/43663451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb047cc0 0x555cdb053f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.453+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/43663451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb04d1a0 0x555cdb04d580 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.453+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/43663451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb047300 0x555cdb047780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.453+0000 7fd7717a0b80 1 -- 172.21.15.44:0/43663451 >> 172.21.15.44:0/43663451 conn(0x555cdafbc030 msgr2=0x555cdb042560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.459+0000 7fd7717a0b80 1 -- 172.21.15.44:0/43663451 shutdown_connections 2023-09-10T01:27:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.463+0000 7fd7717a0b80 1 -- 172.21.15.44:0/43663451 wait complete. 2023-09-10T01:27:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.463+0000 7fd7717a0b80 1 -- start start 2023-09-10T01:27:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.463+0000 7fd7717a0b80 1 Processor -- start 2023-09-10T01:27:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.464+0000 7fd7717a0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb047300 0x555cdb053e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.464+0000 7fd7717a0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb047cc0 0x555cdb0511b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.464+0000 7fd7717a0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb04d1a0 0x555cdb051850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.464+0000 7fd7717a0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdaf37b10 con 0x555cdb04d1a0 2023-09-10T01:27:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.464+0000 7fd7717a0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdaf37810 con 0x555cdb047cc0 2023-09-10T01:27:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.464+0000 7fd7717a0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdae1b8c0 con 0x555cdb047300 2023-09-10T01:27:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.464+0000 7fd7717a0b80 10 client.0 _ll_register_callbacks cb 0x555cdae81a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:27:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.465+0000 7fd75effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb047300 0x555cdb053e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.465+0000 7fd75effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb047300 0x555cdb053e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38382/0 (socket says 172.21.15.44:38382) 2023-09-10T01:27:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.465+0000 7fd75effd700 1 -- 172.21.15.44:0/3224050352 learned_addr learned my addr 172.21.15.44:0/3224050352 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.466+0000 7fd75effd700 1 -- 172.21.15.44:0/3224050352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb047cc0 msgr2=0x555cdb0511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.466+0000 7fd75effd700 1 --2- 172.21.15.44:0/3224050352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb047cc0 0x555cdb0511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.466+0000 7fd75effd700 1 -- 172.21.15.44:0/3224050352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb04d1a0 msgr2=0x555cdb051850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.466+0000 7fd75effd700 1 --2- 172.21.15.44:0/3224050352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb04d1a0 0x555cdb051850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.466+0000 7fd75effd700 1 -- 172.21.15.44:0/3224050352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7601a9d00 con 0x555cdb047300 2023-09-10T01:27:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.467+0000 7fd75effd700 1 --2- 172.21.15.44:0/3224050352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb047300 0x555cdb053e90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd75000b520 tx=0x7fd750011cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22eecd234c00dc3f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.467+0000 7fd742ff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:27:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.480+0000 7fd742ff5700 1 -- 172.21.15.44:0/3224050352 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7500983d0 con 0x555cdb047300 2023-09-10T01:27:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.480+0000 7fd7717a0b80 1 -- 172.21.15.44:0/3224050352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555cdaef1490 con 0x555cdb047300 2023-09-10T01:27:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.480+0000 7fd742ff5700 1 -- 172.21.15.44:0/3224050352 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(185..185 src has 1..185) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fd75009cb10 con 0x555cdb047300 2023-09-10T01:27:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.481+0000 7fd742ff5700 1 -- 172.21.15.44:0/3224050352 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=158) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd750098f40 con 0x555cdb047300 2023-09-10T01:27:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.481+0000 7fd7717a0b80 10 client.29884 fetch_fsmap learned FSMap version 158 2023-09-10T01:27:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.481+0000 7fd7717a0b80 1 -- 172.21.15.44:0/3224050352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=158,monmap=4+}) v3 -- 0x555cdadaec00 con 0x555cdb047300 2023-09-10T01:27:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.481+0000 7fd742ff5700 1 -- 172.21.15.44:0/3224050352 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555cdaef1490 con 0x555cdb047300 2023-09-10T01:27:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.481+0000 7fd742ff5700 1 -- 172.21.15.44:0/3224050352 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 158) v1 ==== 7371+0+0 (secure 0 0 0) 0x7fd750077bb0 con 0x555cdb047300 2023-09-10T01:27:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.481+0000 7fd7717a0b80 10 client.29884 fetch_fsmap finished waiting for FSMap version 158 2023-09-10T01:27:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.481+0000 7fd7717a0b80 10 client.29884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:27:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.481+0000 7fd7717a0b80 20 client.29884 populate_metadata read hostname 'smithi044' 2023-09-10T01:27:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.481+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/3224050352 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555cdb061190 0x555cdb063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.482+0000 7fd7717a0b80 4 client.29884 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:27:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.482+0000 7fd7717a0b80 1 -- 172.21.15.44:0/3224050352 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555cdafae840 con 0x555cdb061190 2023-09-10T01:27:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.483+0000 7fd75e7fc700 1 --2- 172.21.15.44:0/3224050352 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555cdb061190 0x555cdb063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.483+0000 7fd742ff5700 10 client.29884 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.483+0000 7fd75e7fc700 1 --2- 172.21.15.44:0/3224050352 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555cdb061190 0x555cdb063580 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d292f9841449436 server_cookie=fc9223dc4a9c0873 in_seq=0 out_seq=0 2023-09-10T01:27:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.490+0000 7fd742ff5700 1 -- 172.21.15.44:0/3224050352 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555cdafae840 con 0x555cdb061190 2023-09-10T01:27:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.490+0000 7fd742ff5700 10 client.29884 handle_command_reply: tid=0 2023-09-10T01:27:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.490+0000 7fd7717a0b80 1 client.29884 shutdown 2023-09-10T01:27:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.501+0000 7fd7717a0b80 1 -- 172.21.15.44:0/3224050352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb047300 msgr2=0x555cdb053e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.501+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/3224050352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb047300 0x555cdb053e90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd75000b520 tx=0x7fd750011cf0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.515+0000 7fd7717a0b80 1 -- 172.21.15.44:0/3224050352 shutdown_connections 2023-09-10T01:27:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.515+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/3224050352 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555cdb061190 0x555cdb063580 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.515+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/3224050352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb04d1a0 0x555cdb051850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.515+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/3224050352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb047300 0x555cdb053e90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.515+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/3224050352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb047cc0 0x555cdb0511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.515+0000 7fd7717a0b80 1 -- 172.21.15.44:0/3224050352 >> 172.21.15.44:0/3224050352 conn(0x555cdafbc030 msgr2=0x555cdb0417a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.517+0000 7fd742ff5700 0 client.29884 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.517+0000 7fd7717a0b80 1 -- 172.21.15.44:0/3224050352 shutdown_connections 2023-09-10T01:27:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.518+0000 7fd7717a0b80 1 -- 172.21.15.44:0/3224050352 wait complete. 2023-09-10T01:27:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.518+0000 7fd7717a0b80 20 client.29884 trim_cache size 0 max 16384 2023-09-10T01:27:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.546+0000 7fd7717a0b80 1 Processor -- start 2023-09-10T01:27:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.546+0000 7fd7717a0b80 1 -- start start 2023-09-10T01:27:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.546+0000 7fd7717a0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb047300 0x555cdb04ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.546+0000 7fd7717a0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb047cc0 0x555cdb05cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.546+0000 7fd7717a0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb04d1a0 0x555cdb0539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.546+0000 7fd7717a0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdb04f7a0 con 0x555cdb04d1a0 2023-09-10T01:27:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.546+0000 7fd7717a0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdb04bff0 con 0x555cdb047300 2023-09-10T01:27:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.546+0000 7fd7717a0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdb04d7d0 con 0x555cdb047cc0 2023-09-10T01:27:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.547+0000 7fd75f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb04d1a0 0x555cdb0539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.547+0000 7fd75f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb04d1a0 0x555cdb0539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59744/0 (socket says 172.21.15.44:59744) 2023-09-10T01:27:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.547+0000 7fd75f7fe700 1 -- 172.21.15.44:0/1693282153 learned_addr learned my addr 172.21.15.44:0/1693282153 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.547+0000 7fd75f7fe700 1 -- 172.21.15.44:0/1693282153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb047cc0 msgr2=0x555cdb05cc80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:27:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.547+0000 7fd75f7fe700 1 --2- 172.21.15.44:0/1693282153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb047cc0 0x555cdb05cc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.547+0000 7fd75f7fe700 1 -- 172.21.15.44:0/1693282153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb047300 msgr2=0x555cdb04ee60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:27:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.547+0000 7fd75f7fe700 1 --2- 172.21.15.44:0/1693282153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb047300 0x555cdb04ee60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.547+0000 7fd75f7fe700 1 -- 172.21.15.44:0/1693282153 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cdadec1f0 con 0x555cdb04d1a0 2023-09-10T01:27:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.548+0000 7fd75f7fe700 1 --2- 172.21.15.44:0/1693282153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb04d1a0 0x555cdb0539c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fd754042b80 tx=0x7fd754042e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc70d7977717f01 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.564+0000 7fd745ffb700 1 -- 172.21.15.44:0/1693282153 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754044c20 con 0x555cdb04d1a0 2023-09-10T01:27:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.564+0000 7fd745ffb700 1 -- 172.21.15.44:0/1693282153 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd754044d80 con 0x555cdb04d1a0 2023-09-10T01:27:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.564+0000 7fd745ffb700 1 -- 172.21.15.44:0/1693282153 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75404b5f0 con 0x555cdb04d1a0 2023-09-10T01:27:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.565+0000 7fd7717a0b80 1 -- 172.21.15.44:0/1693282153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb04d1a0 msgr2=0x555cdb0539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.565+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/1693282153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb04d1a0 0x555cdb0539c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fd754042b80 tx=0x7fd754042e90 comp rx=0 tx=0).stop 2023-09-10T01:27:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.573+0000 7fd7717a0b80 1 -- 172.21.15.44:0/1693282153 shutdown_connections 2023-09-10T01:27:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.573+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/1693282153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb04d1a0 0x555cdb0539c0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.573+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/1693282153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb047300 0x555cdb04ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.573+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/1693282153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb047cc0 0x555cdb05cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.573+0000 7fd7717a0b80 1 -- 172.21.15.44:0/1693282153 >> 172.21.15.44:0/1693282153 conn(0x555cdafbc030 msgr2=0x555cdb042150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.585+0000 7fd7717a0b80 1 -- 172.21.15.44:0/1693282153 shutdown_connections 2023-09-10T01:27:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.603+0000 7fd7717a0b80 1 -- 172.21.15.44:0/1693282153 wait complete. 2023-09-10T01:27:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.603+0000 7fd7717a0b80 1 -- start start 2023-09-10T01:27:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.603+0000 7fd7717a0b80 1 Processor -- start 2023-09-10T01:27:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.604+0000 7fd7717a0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb047300 0x555cdb0ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.604+0000 7fd7717a0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb047cc0 0x555cdb0ae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.604+0000 7fd7717a0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb04d1a0 0x555cdb056510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.604+0000 7fd7717a0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdb056a50 con 0x555cdb047300 2023-09-10T01:27:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.604+0000 7fd7717a0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdb056bc0 con 0x555cdb047cc0 2023-09-10T01:27:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.604+0000 7fd7717a0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdb056d80 con 0x555cdb04d1a0 2023-09-10T01:27:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.604+0000 7fd7717a0b80 10 client.0 _ll_register_callbacks cb 0x555cdae81a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:27:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.605+0000 7fd75effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb047300 0x555cdb0ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.605+0000 7fd75effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb047300 0x555cdb0ae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59748/0 (socket says 172.21.15.44:59748) 2023-09-10T01:27:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.605+0000 7fd75effd700 1 -- 172.21.15.44:0/3259555023 learned_addr learned my addr 172.21.15.44:0/3259555023 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.605+0000 7fd75effd700 1 -- 172.21.15.44:0/3259555023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb04d1a0 msgr2=0x555cdb056510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.605+0000 7fd75effd700 1 --2- 172.21.15.44:0/3259555023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb04d1a0 0x555cdb056510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.605+0000 7fd75effd700 1 -- 172.21.15.44:0/3259555023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb047cc0 msgr2=0x555cdb0ae660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:27:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.605+0000 7fd75effd700 1 --2- 172.21.15.44:0/3259555023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb047cc0 0x555cdb0ae660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.605+0000 7fd75effd700 1 -- 172.21.15.44:0/3259555023 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7540427e0 con 0x555cdb047300 2023-09-10T01:27:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.606+0000 7fd75effd700 1 --2- 172.21.15.44:0/3259555023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb047300 0x555cdb0ae120 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fd7500c93f0 tx=0x7fd7500b8520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f10c17fcddd32fd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.612+0000 7fd742ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:27:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.612+0000 7fd742ff5700 1 -- 172.21.15.44:0/3259555023 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd750099140 con 0x555cdb047300 2023-09-10T01:27:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.612+0000 7fd742ff5700 1 -- 172.21.15.44:0/3259555023 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(185..185 src has 1..185) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fd7500092a0 con 0x555cdb047300 2023-09-10T01:27:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.612+0000 7fd7717a0b80 1 -- 172.21.15.44:0/3259555023 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7500993e0 con 0x555cdb047300 2023-09-10T01:27:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.614+0000 7fd742ff5700 1 -- 172.21.15.44:0/3259555023 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=158) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd750098640 con 0x555cdb047300 2023-09-10T01:27:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.618+0000 7fd7717a0b80 10 client.20265 fetch_fsmap learned FSMap version 158 2023-09-10T01:27:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.618+0000 7fd7717a0b80 1 -- 172.21.15.44:0/3259555023 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=158,monmap=4+}) v3 -- 0x555cdadc4250 con 0x555cdb047300 2023-09-10T01:27:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.618+0000 7fd742ff5700 1 -- 172.21.15.44:0/3259555023 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7500993e0 con 0x555cdb047300 2023-09-10T01:27:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.618+0000 7fd742ff5700 1 -- 172.21.15.44:0/3259555023 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 158) v1 ==== 7371+0+0 (secure 0 0 0) 0x7fd7500981d0 con 0x555cdb047300 2023-09-10T01:27:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.619+0000 7fd7717a0b80 10 client.20265 fetch_fsmap finished waiting for FSMap version 158 2023-09-10T01:27:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.619+0000 7fd7717a0b80 10 client.20265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:27:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.619+0000 7fd7717a0b80 20 client.20265 populate_metadata read hostname 'smithi044' 2023-09-10T01:27:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.619+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/3259555023 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555cdb0b5af0 0x555cdb0b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.619+0000 7fd7717a0b80 4 client.20265 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:27:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.619+0000 7fd7717a0b80 1 -- 172.21.15.44:0/3259555023 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555cdaf928c0 con 0x555cdb0b5af0 2023-09-10T01:27:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.619+0000 7fd75e7fc700 1 --2- 172.21.15.44:0/3259555023 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555cdb0b5af0 0x555cdb0b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.619+0000 7fd742ff5700 10 client.20265 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.619+0000 7fd75e7fc700 1 --2- 172.21.15.44:0/3259555023 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555cdb0b5af0 0x555cdb0b7ee0 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=912824b98069748 server_cookie=ad624b776b0ab615 in_seq=0 out_seq=0 2023-09-10T01:27:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.950+0000 7fd742ff5700 1 -- 172.21.15.44:0/3259555023 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555cdaf928c0 con 0x555cdb0b5af0 2023-09-10T01:27:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.950+0000 7fd742ff5700 10 client.20265 handle_command_reply: tid=0 2023-09-10T01:27:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.950+0000 7fd7717a0b80 1 client.20265 shutdown 2023-09-10T01:27:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.953+0000 7fd7717a0b80 1 -- 172.21.15.44:0/3259555023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb047300 msgr2=0x555cdb0ae120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.953+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/3259555023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb047300 0x555cdb0ae120 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fd7500c93f0 tx=0x7fd7500b8520 comp rx=0 tx=0).stop 2023-09-10T01:27:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.954+0000 7fd7717a0b80 1 -- 172.21.15.44:0/3259555023 shutdown_connections 2023-09-10T01:27:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.954+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/3259555023 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555cdb0b5af0 0x555cdb0b7ee0 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.954+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/3259555023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555cdb04d1a0 0x555cdb056510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.954+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/3259555023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555cdb047300 0x555cdb0ae120 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.954+0000 7fd7717a0b80 1 --2- 172.21.15.44:0/3259555023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cdb047cc0 0x555cdb0ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.954+0000 7fd7717a0b80 1 -- 172.21.15.44:0/3259555023 >> 172.21.15.44:0/3259555023 conn(0x555cdafbc030 msgr2=0x555cdb04b480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.954+0000 7fd742ff5700 0 client.20265 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.954+0000 7fd7717a0b80 1 -- 172.21.15.44:0/3259555023 shutdown_connections 2023-09-10T01:27:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.954+0000 7fd7717a0b80 1 -- 172.21.15.44:0/3259555023 wait complete. 2023-09-10T01:27:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:27:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:27:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:27:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:27:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.954+0000 7fd7717a0b80 20 client.20265 trim_cache size 0 max 16384 2023-09-10T01:27:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.958+0000 7fd7655da700 1 -- 172.21.15.44:0/4267344009 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7480a0870 msgr2=0x7fd7480a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.958+0000 7fd7655da700 1 --2- 172.21.15.44:0/4267344009 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7480a0870 0x7fd7480a2d30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd75800b770 tx=0x7fd75800baa0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.958+0000 7fd7655da700 1 -- 172.21.15.44:0/4267344009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76010ed50 msgr2=0x7fd760114f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.958+0000 7fd7655da700 1 --2- 172.21.15.44:0/4267344009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76010ed50 0x7fd760114f90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd75000b550 tx=0x7fd75000b910 comp rx=0 tx=0).stop 2023-09-10T01:27:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.958+0000 7fd7655da700 1 -- 172.21.15.44:0/4267344009 shutdown_connections 2023-09-10T01:27:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.958+0000 7fd7655da700 1 --2- 172.21.15.44:0/4267344009 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd7480a0870 0x7fd7480a2d30 secure :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd75800b770 tx=0x7fd75800baa0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.958+0000 7fd7655da700 1 --2- 172.21.15.44:0/4267344009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd760119de0 0x7fd7601a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.958+0000 7fd7655da700 1 --2- 172.21.15.44:0/4267344009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76010ed50 0x7fd760114f90 secure :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd75000b550 tx=0x7fd75000b910 comp rx=0 tx=0).stop 2023-09-10T01:27:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.958+0000 7fd7655da700 1 --2- 172.21.15.44:0/4267344009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd76010f1c0 0x7fd7601154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.958+0000 7fd7655da700 1 -- 172.21.15.44:0/4267344009 >> 172.21.15.44:0/4267344009 conn(0x7fd76006c6c0 msgr2=0x7fd760070fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.961+0000 7fd7655da700 1 -- 172.21.15.44:0/4267344009 shutdown_connections 2023-09-10T01:27:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:04.961+0000 7fd7655da700 1 -- 172.21.15.44:0/4267344009 wait complete. 2023-09-10T01:27:05.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:05 smithi131 ceph-mon[103713]: pgmap v1504: 129 pgs: 129 active+clean; 18 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.8 MiB/s wr, 617 op/s 2023-09-10T01:27:05.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:05 smithi131 ceph-mon[103713]: osdmap e185: 12 total, 12 up, 12 in 2023-09-10T01:27:05.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:05 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:27:05.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:05 smithi044 ceph-mon[98676]: pgmap v1504: 129 pgs: 129 active+clean; 18 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.8 MiB/s wr, 617 op/s 2023-09-10T01:27:05.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:05 smithi044 ceph-mon[98676]: osdmap e185: 12 total, 12 up, 12 in 2023-09-10T01:27:05.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:05 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:27:05.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:05 smithi099 ceph-mon[102692]: pgmap v1504: 129 pgs: 129 active+clean; 18 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.8 MiB/s wr, 617 op/s 2023-09-10T01:27:05.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:05 smithi099 ceph-mon[102692]: osdmap e185: 12 total, 12 up, 12 in 2023-09-10T01:27:05.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:05 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:27:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 360.0 s, 178.4 tps, lat 580.847 ms stddev 610.022 2023-09-10T01:27:07.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:07 smithi099 ceph-mon[102692]: pgmap v1506: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 18 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 736 op/s 2023-09-10T01:27:07.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:07 smithi131 ceph-mon[103713]: pgmap v1506: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 18 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 736 op/s 2023-09-10T01:27:07.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:07 smithi044 ceph-mon[98676]: pgmap v1506: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 18 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 736 op/s 2023-09-10T01:27:09.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:09 smithi099 ceph-mon[102692]: pgmap v1507: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 18 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 736 op/s 2023-09-10T01:27:09.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:09 smithi131 ceph-mon[103713]: pgmap v1507: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 18 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 736 op/s 2023-09-10T01:27:09.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:09 smithi044 ceph-mon[98676]: pgmap v1507: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 18 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 736 op/s 2023-09-10T01:27:11.188 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 365.0 s, 81.1 tps, lat 1221.037 ms stddev 890.194 2023-09-10T01:27:11.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:11 smithi131 ceph-mon[103713]: pgmap v1508: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 18 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 736 op/s 2023-09-10T01:27:11.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:11 smithi044 ceph-mon[98676]: pgmap v1508: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 18 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 736 op/s 2023-09-10T01:27:11.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:11 smithi099 ceph-mon[102692]: pgmap v1508: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 18 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 736 op/s 2023-09-10T01:27:13.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:13 smithi131 ceph-mon[103713]: pgmap v1509: 129 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 116 active+clean; 17 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 462 op/s 2023-09-10T01:27:13.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:13 smithi044 ceph-mon[98676]: pgmap v1509: 129 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 116 active+clean; 17 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 462 op/s 2023-09-10T01:27:13.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:13 smithi099 ceph-mon[102692]: pgmap v1509: 129 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 116 active+clean; 17 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 462 op/s 2023-09-10T01:27:15.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:15 smithi131 ceph-mon[103713]: pgmap v1510: 129 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 116 active+clean; 17 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 461 op/s 2023-09-10T01:27:15.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:15 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:27:15.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:15 smithi044 ceph-mon[98676]: pgmap v1510: 129 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 116 active+clean; 17 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 461 op/s 2023-09-10T01:27:15.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:15 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:27:15.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:15 smithi099 ceph-mon[102692]: pgmap v1510: 129 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 116 active+clean; 17 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 461 op/s 2023-09-10T01:27:15.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:15 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:27:16.183 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 370.0 s, 243.5 tps, lat 426.514 ms stddev 204.428 2023-09-10T01:27:16.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:16 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:27:16.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:16 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:27:16.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:16 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:27:16.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:16 smithi131 ceph-mon[103713]: osdmap e186: 12 total, 12 up, 12 in 2023-09-10T01:27:16.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:16 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:27:16.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:16 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:27:16.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:16 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:27:16.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:16 smithi044 ceph-mon[98676]: osdmap e186: 12 total, 12 up, 12 in 2023-09-10T01:27:16.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:16 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:27:16.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:16 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:27:16.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:16 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:27:16.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:16 smithi099 ceph-mon[102692]: osdmap e186: 12 total, 12 up, 12 in 2023-09-10T01:27:17.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:17 smithi131 ceph-mon[103713]: pgmap v1511: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.8 MiB/s wr, 708 op/s 2023-09-10T01:27:17.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:17 smithi131 ceph-mon[103713]: pgmap v1512: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 497 op/s 2023-09-10T01:27:17.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:17 smithi044 ceph-mon[98676]: pgmap v1511: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.8 MiB/s wr, 708 op/s 2023-09-10T01:27:17.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:17 smithi044 ceph-mon[98676]: pgmap v1512: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 497 op/s 2023-09-10T01:27:17.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:17 smithi099 ceph-mon[102692]: pgmap v1511: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.8 MiB/s wr, 708 op/s 2023-09-10T01:27:17.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:17 smithi099 ceph-mon[102692]: pgmap v1512: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 497 op/s 2023-09-10T01:27:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.189+0000 7f122c7c8700 1 -- 172.21.15.44:0/3537380716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122410f1c0 msgr2=0x7f1224107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.189+0000 7f122c7c8700 1 --2- 172.21.15.44:0/3537380716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122410f1c0 0x7f1224107b40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f121401a9a0 tx=0x7f121401acb0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.190+0000 7f122c7c8700 1 -- 172.21.15.44:0/3537380716 shutdown_connections 2023-09-10T01:27:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.190+0000 7f122c7c8700 1 --2- 172.21.15.44:0/3537380716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224119de0 0x7f122411c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.190+0000 7f122c7c8700 1 --2- 172.21.15.44:0/3537380716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f122410ed50 0x7f1224107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.190+0000 7f122c7c8700 1 --2- 172.21.15.44:0/3537380716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122410f1c0 0x7f1224107b40 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.190+0000 7f122c7c8700 1 -- 172.21.15.44:0/3537380716 >> 172.21.15.44:0/3537380716 conn(0x7f122406c6c0 msgr2=0x7f122406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.195+0000 7f122c7c8700 1 -- 172.21.15.44:0/3537380716 shutdown_connections 2023-09-10T01:27:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.195+0000 7f122c7c8700 1 -- 172.21.15.44:0/3537380716 wait complete. 2023-09-10T01:27:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.196+0000 7f122c7c8700 1 Processor -- start 2023-09-10T01:27:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.196+0000 7f122c7c8700 1 -- start start 2023-09-10T01:27:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.197+0000 7f122c7c8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f122410ed50 0x7f1224115070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.197+0000 7f122c7c8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122410f1c0 0x7f12241155d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.197+0000 7f122c7c8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1224119de0 0x7f12241c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.197+0000 7f122c7c8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12241c5150 con 0x7f1224119de0 2023-09-10T01:27:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.197+0000 7f122c7c8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12241c52c0 con 0x7f122410ed50 2023-09-10T01:27:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.197+0000 7f122c7c8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12241c5480 con 0x7f122410f1c0 2023-09-10T01:27:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.197+0000 7f1229d63700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122410f1c0 0x7f12241155d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.197+0000 7f122a564700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f122410ed50 0x7f1224115070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.197+0000 7f1229d63700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122410f1c0 0x7f12241155d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38410/0 (socket says 172.21.15.44:38410) 2023-09-10T01:27:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.197+0000 7f1229d63700 1 -- 172.21.15.44:0/2851898634 learned_addr learned my addr 172.21.15.44:0/2851898634 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.197+0000 7f122a564700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f122410ed50 0x7f1224115070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33266/0 (socket says 172.21.15.44:33266) 2023-09-10T01:27:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.197+0000 7f1229d63700 1 -- 172.21.15.44:0/2851898634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f122410ed50 msgr2=0x7f1224115070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.197+0000 7f1229d63700 1 --2- 172.21.15.44:0/2851898634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f122410ed50 0x7f1224115070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.197+0000 7f1229d63700 1 -- 172.21.15.44:0/2851898634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1224119de0 msgr2=0x7f12241c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.197+0000 7f1229d63700 1 --2- 172.21.15.44:0/2851898634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1224119de0 0x7f12241c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.197+0000 7f1229d63700 1 -- 172.21.15.44:0/2851898634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121401a600 con 0x7f122410f1c0 2023-09-10T01:27:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.198+0000 7f1229d63700 1 --2- 172.21.15.44:0/2851898634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122410f1c0 0x7f12241155d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f121401a8f0 tx=0x7f121401ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.198+0000 7f121b7fe700 1 -- 172.21.15.44:0/2851898634 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214003cd0 con 0x7f122410f1c0 2023-09-10T01:27:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.198+0000 7f122c7c8700 1 -- 172.21.15.44:0/2851898634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12241c5700 con 0x7f122410f1c0 2023-09-10T01:27:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.198+0000 7f122c7c8700 1 -- 172.21.15.44:0/2851898634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12241cbd80 con 0x7f122410f1c0 2023-09-10T01:27:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.199+0000 7f121b7fe700 1 -- 172.21.15.44:0/2851898634 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1214003e30 con 0x7f122410f1c0 2023-09-10T01:27:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.199+0000 7f121b7fe700 1 -- 172.21.15.44:0/2851898634 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121400e640 con 0x7f122410f1c0 2023-09-10T01:27:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.203+0000 7f123898eb80 1 Processor -- start 2023-09-10T01:27:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.203+0000 7f123898eb80 1 -- start start 2023-09-10T01:27:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.203+0000 7f123898eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 0x5584d4320580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.203+0000 7f123898eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d431a300 0x5584d431a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.204+0000 7f123898eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431acc0 0x5584d4326f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.204+0000 7f123898eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d42736d0 con 0x5584d43201a0 2023-09-10T01:27:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.204+0000 7f123898eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d4270ff0 con 0x5584d431a300 2023-09-10T01:27:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.204+0000 7f123898eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d426c8e0 con 0x5584d431acc0 2023-09-10T01:27:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.204+0000 7f122a564700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d431a300 0x5584d431a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.204+0000 7f122ad65700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 0x5584d4320580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.204+0000 7f122a564700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d431a300 0x5584d431a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33274/0 (socket says 172.21.15.44:33274) 2023-09-10T01:27:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.204+0000 7f122a564700 1 -- 172.21.15.44:0/101348019 learned_addr learned my addr 172.21.15.44:0/101348019 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.204+0000 7f122ad65700 1 -- 172.21.15.44:0/101348019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431acc0 msgr2=0x5584d4326f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.204+0000 7f122ad65700 1 --2- 172.21.15.44:0/101348019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431acc0 0x5584d4326f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.204+0000 7f122ad65700 1 -- 172.21.15.44:0/101348019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d431a300 msgr2=0x5584d431a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.204+0000 7f122ad65700 1 --2- 172.21.15.44:0/101348019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d431a300 0x5584d431a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.204+0000 7f122ad65700 1 -- 172.21.15.44:0/101348019 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584d4098d80 con 0x5584d43201a0 2023-09-10T01:27:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.204+0000 7f122a564700 1 --2- 172.21.15.44:0/101348019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d431a300 0x5584d431a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:27:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.205+0000 7f122ad65700 1 --2- 172.21.15.44:0/101348019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 0x5584d4320580 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f121c009cb0 tx=0x7f121c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ff59b174b8d69e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.205+0000 7f1218ff9700 1 -- 172.21.15.44:0/101348019 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c00bd90 con 0x5584d43201a0 2023-09-10T01:27:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.205+0000 7f1218ff9700 1 -- 172.21.15.44:0/101348019 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f121c0138a0 con 0x5584d43201a0 2023-09-10T01:27:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.205+0000 7f1218ff9700 1 -- 172.21.15.44:0/101348019 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c0127a0 con 0x5584d43201a0 2023-09-10T01:27:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.206+0000 7f123898eb80 1 -- 172.21.15.44:0/101348019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 msgr2=0x5584d4320580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.206+0000 7f123898eb80 1 --2- 172.21.15.44:0/101348019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 0x5584d4320580 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f121c009cb0 tx=0x7f121c00b410 comp rx=0 tx=0).stop 2023-09-10T01:27:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.206+0000 7f123898eb80 1 -- 172.21.15.44:0/101348019 shutdown_connections 2023-09-10T01:27:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.206+0000 7f123898eb80 1 --2- 172.21.15.44:0/101348019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431acc0 0x5584d4326f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.206+0000 7f123898eb80 1 --2- 172.21.15.44:0/101348019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 0x5584d4320580 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.206+0000 7f123898eb80 1 --2- 172.21.15.44:0/101348019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d431a300 0x5584d431a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.206+0000 7f123898eb80 1 -- 172.21.15.44:0/101348019 >> 172.21.15.44:0/101348019 conn(0x5584d428f030 msgr2=0x5584d4315560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.206+0000 7f123898eb80 1 -- 172.21.15.44:0/101348019 shutdown_connections 2023-09-10T01:27:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.206+0000 7f123898eb80 1 -- 172.21.15.44:0/101348019 wait complete. 2023-09-10T01:27:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.206+0000 7f123898eb80 1 -- start start 2023-09-10T01:27:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.206+0000 7f123898eb80 1 Processor -- start 2023-09-10T01:27:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.207+0000 7f123898eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d431a300 0x5584d4326e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.207+0000 7f123898eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431acc0 0x5584d4324110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.207+0000 7f123898eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d43201a0 0x5584d43247b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.207+0000 7f123898eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d420ab10 con 0x5584d431a300 2023-09-10T01:27:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.207+0000 7f123898eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d420a810 con 0x5584d43201a0 2023-09-10T01:27:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.207+0000 7f123898eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d40ee8c0 con 0x5584d431acc0 2023-09-10T01:27:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.207+0000 7f123898eb80 10 client.0 _ll_register_callbacks cb 0x5584d4154a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:27:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.207+0000 7f122ad65700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d431a300 0x5584d4326e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.207+0000 7f122ad65700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d431a300 0x5584d4326e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59772/0 (socket says 172.21.15.44:59772) 2023-09-10T01:27:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.207+0000 7f122ad65700 1 -- 172.21.15.44:0/1706643969 learned_addr learned my addr 172.21.15.44:0/1706643969 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.207+0000 7f1229d63700 1 --2- 172.21.15.44:0/1706643969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d43201a0 0x5584d43247b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.207+0000 7f122ad65700 1 -- 172.21.15.44:0/1706643969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431acc0 msgr2=0x5584d4324110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.207+0000 7f122ad65700 1 --2- 172.21.15.44:0/1706643969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431acc0 0x5584d4324110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.207+0000 7f122ad65700 1 -- 172.21.15.44:0/1706643969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d43201a0 msgr2=0x5584d43247b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.207+0000 7f122ad65700 1 --2- 172.21.15.44:0/1706643969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d43201a0 0x5584d43247b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.208+0000 7f122ad65700 1 -- 172.21.15.44:0/1706643969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f121c009910 con 0x5584d431a300 2023-09-10T01:27:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.209+0000 7f122ad65700 1 --2- 172.21.15.44:0/1706643969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d431a300 0x5584d4326e90 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f121c016040 tx=0x7f121c013cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5550c1ecfcfed8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.209+0000 7f120dffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:27:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.209+0000 7f121b7fe700 1 -- 172.21.15.44:0/2851898634 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f121400f480 con 0x7f122410f1c0 2023-09-10T01:27:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.209+0000 7f120dffb700 1 -- 172.21.15.44:0/1706643969 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c003aa0 con 0x5584d431a300 2023-09-10T01:27:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.210+0000 7f120dffb700 1 -- 172.21.15.44:0/1706643969 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 9552+0+0 (secure 0 0 0) 0x7f121c003c00 con 0x5584d431a300 2023-09-10T01:27:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.210+0000 7f123898eb80 1 -- 172.21.15.44:0/1706643969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584d41c4490 con 0x5584d431a300 2023-09-10T01:27:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.210+0000 7f120dffb700 1 -- 172.21.15.44:0/1706643969 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=158) v2 ==== 24+0+0 (secure 0 0 0) 0x7f121c003dc0 con 0x5584d431a300 2023-09-10T01:27:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.211+0000 7f121b7fe700 1 --2- 172.21.15.44:0/2851898634 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f12100a0700 0x7f12100a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.211+0000 7f121b7fe700 1 -- 172.21.15.44:0/2851898634 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 9552+0+0 (secure 0 0 0) 0x7f12140cdf00 con 0x7f122410f1c0 2023-09-10T01:27:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.211+0000 7f123898eb80 10 client.20271 fetch_fsmap learned FSMap version 158 2023-09-10T01:27:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.211+0000 7f123898eb80 1 -- 172.21.15.44:0/1706643969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=158,monmap=4+}) v3 -- 0x5584d4081c00 con 0x5584d431a300 2023-09-10T01:27:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.215+0000 7f122a564700 1 --2- 172.21.15.44:0/2851898634 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f12100a0700 0x7f12100a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.216+0000 7f122a564700 1 --2- 172.21.15.44:0/2851898634 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f12100a0700 0x7f12100a2bc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1220006ea0 tx=0x7f1220006e30 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.217+0000 7f120dffb700 1 -- 172.21.15.44:0/1706643969 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f121c02b030 con 0x5584d431a300 2023-09-10T01:27:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.218+0000 7f120dffb700 1 -- 172.21.15.44:0/1706643969 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 158) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f121c008030 con 0x5584d431a300 2023-09-10T01:27:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.218+0000 7f123898eb80 10 client.20271 fetch_fsmap finished waiting for FSMap version 158 2023-09-10T01:27:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.218+0000 7f123898eb80 10 client.20271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:27:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.218+0000 7f123898eb80 20 client.20271 populate_metadata read hostname 'smithi044' 2023-09-10T01:27:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.218+0000 7f123898eb80 1 --2- 172.21.15.44:0/1706643969 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5584d432b460 0x5584d432d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.218+0000 7f123898eb80 4 client.20271 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:27:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.218+0000 7f123898eb80 1 -- 172.21.15.44:0/1706643969 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584d4281840 con 0x5584d432b460 2023-09-10T01:27:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.219+0000 7f1229d63700 1 --2- 172.21.15.44:0/1706643969 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5584d432b460 0x5584d432d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.219+0000 7f1229d63700 1 --2- 172.21.15.44:0/1706643969 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5584d432b460 0x5584d432d880 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=535eed46acdd05fe server_cookie=cb53faf0bfb2fc12 in_seq=0 out_seq=0 2023-09-10T01:27:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.219+0000 7f120dffb700 10 client.20271 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.220+0000 7f120dffb700 1 -- 172.21.15.44:0/1706643969 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5584d4281840 con 0x5584d432b460 2023-09-10T01:27:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.220+0000 7f120dffb700 10 client.20271 handle_command_reply: tid=0 2023-09-10T01:27:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.220+0000 7f123898eb80 1 client.20271 shutdown 2023-09-10T01:27:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.221+0000 7f123898eb80 1 -- 172.21.15.44:0/1706643969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d431a300 msgr2=0x5584d4326e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.221+0000 7f123898eb80 1 --2- 172.21.15.44:0/1706643969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d431a300 0x5584d4326e90 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f121c016040 tx=0x7f121c013cf0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.221+0000 7f123898eb80 1 -- 172.21.15.44:0/1706643969 shutdown_connections 2023-09-10T01:27:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.221+0000 7f123898eb80 1 --2- 172.21.15.44:0/1706643969 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5584d432b460 0x5584d432d880 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.221+0000 7f123898eb80 1 --2- 172.21.15.44:0/1706643969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d43201a0 0x5584d43247b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.221+0000 7f123898eb80 1 --2- 172.21.15.44:0/1706643969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d431a300 0x5584d4326e90 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.221+0000 7f123898eb80 1 --2- 172.21.15.44:0/1706643969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431acc0 0x5584d4324110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.221+0000 7f123898eb80 1 -- 172.21.15.44:0/1706643969 >> 172.21.15.44:0/1706643969 conn(0x5584d428f030 msgr2=0x5584d43147a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.221+0000 7f120dffb700 0 client.20271 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.223+0000 7f123898eb80 1 -- 172.21.15.44:0/1706643969 shutdown_connections 2023-09-10T01:27:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.223+0000 7f123898eb80 1 -- 172.21.15.44:0/1706643969 wait complete. 2023-09-10T01:27:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.224+0000 7f123898eb80 20 client.20271 trim_cache size 0 max 16384 2023-09-10T01:27:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.254+0000 7f123898eb80 1 Processor -- start 2023-09-10T01:27:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.254+0000 7f123898eb80 1 -- start start 2023-09-10T01:27:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.260+0000 7f123898eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431a300 0x5584d431a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.260+0000 7f123898eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d431acc0 0x5584d4326a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.260+0000 7f123898eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 0x5584d4315090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.260+0000 7f123898eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d4324d90 con 0x5584d43201a0 2023-09-10T01:27:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.260+0000 7f123898eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d431ed00 con 0x5584d431acc0 2023-09-10T01:27:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.260+0000 7f123898eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d4320580 con 0x5584d431a300 2023-09-10T01:27:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.260+0000 7f122a564700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431a300 0x5584d431a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.260+0000 7f122a564700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431a300 0x5584d431a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38430/0 (socket says 172.21.15.44:38430) 2023-09-10T01:27:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.260+0000 7f122a564700 1 -- 172.21.15.44:0/1196673793 learned_addr learned my addr 172.21.15.44:0/1196673793 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.261+0000 7f122a564700 1 -- 172.21.15.44:0/1196673793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d431acc0 msgr2=0x5584d4326a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.261+0000 7f122a564700 1 --2- 172.21.15.44:0/1196673793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d431acc0 0x5584d4326a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.261+0000 7f122a564700 1 -- 172.21.15.44:0/1196673793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 msgr2=0x5584d4315090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.261+0000 7f122a564700 1 --2- 172.21.15.44:0/1196673793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 0x5584d4315090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.261+0000 7f122a564700 1 -- 172.21.15.44:0/1196673793 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584d40bf1f0 con 0x5584d431a300 2023-09-10T01:27:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.261+0000 7f122a564700 1 --2- 172.21.15.44:0/1196673793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431a300 0x5584d431a6e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f12200073f0 tx=0x7f1220012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cab9e15eb811e140 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.264+0000 7f12197fa700 1 -- 172.21.15.44:0/1196673793 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220014660 con 0x5584d431a300 2023-09-10T01:27:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.265+0000 7f12197fa700 1 -- 172.21.15.44:0/1196673793 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1220019070 con 0x5584d431a300 2023-09-10T01:27:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.265+0000 7f12197fa700 1 -- 172.21.15.44:0/1196673793 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220020b60 con 0x5584d431a300 2023-09-10T01:27:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.265+0000 7f123898eb80 1 -- 172.21.15.44:0/1196673793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431a300 msgr2=0x5584d431a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.265+0000 7f123898eb80 1 --2- 172.21.15.44:0/1196673793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431a300 0x5584d431a6e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f12200073f0 tx=0x7f1220012000 comp rx=0 tx=0).stop 2023-09-10T01:27:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.266+0000 7f123898eb80 1 -- 172.21.15.44:0/1196673793 shutdown_connections 2023-09-10T01:27:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.266+0000 7f123898eb80 1 --2- 172.21.15.44:0/1196673793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 0x5584d4315090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.266+0000 7f123898eb80 1 --2- 172.21.15.44:0/1196673793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431a300 0x5584d431a6e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.266+0000 7f123898eb80 1 --2- 172.21.15.44:0/1196673793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d431acc0 0x5584d4326a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.266+0000 7f123898eb80 1 -- 172.21.15.44:0/1196673793 >> 172.21.15.44:0/1196673793 conn(0x5584d428f030 msgr2=0x5584d431e190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.266+0000 7f123898eb80 1 -- 172.21.15.44:0/1196673793 shutdown_connections 2023-09-10T01:27:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.266+0000 7f123898eb80 1 -- 172.21.15.44:0/1196673793 wait complete. 2023-09-10T01:27:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.266+0000 7f123898eb80 1 -- start start 2023-09-10T01:27:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.266+0000 7f123898eb80 1 Processor -- start 2023-09-10T01:27:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.267+0000 7f123898eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431a300 0x5584d4339200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.267+0000 7f123898eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d431acc0 0x5584d4339740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.267+0000 7f123898eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 0x5584d43cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.267+0000 7f123898eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d43cbd40 con 0x5584d43201a0 2023-09-10T01:27:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.267+0000 7f123898eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d43cbe80 con 0x5584d431acc0 2023-09-10T01:27:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.267+0000 7f123898eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d43cbfc0 con 0x5584d431a300 2023-09-10T01:27:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.267+0000 7f123898eb80 10 client.0 _ll_register_callbacks cb 0x7f121c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:27:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.267+0000 7f122ad65700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 0x5584d43cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.267+0000 7f122ad65700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 0x5584d43cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59790/0 (socket says 172.21.15.44:59790) 2023-09-10T01:27:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.267+0000 7f122ad65700 1 -- 172.21.15.44:0/2933600696 learned_addr learned my addr 172.21.15.44:0/2933600696 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.267+0000 7f122a564700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431a300 0x5584d4339200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.268+0000 7f122ad65700 1 -- 172.21.15.44:0/2933600696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431a300 msgr2=0x5584d4339200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.268+0000 7f122ad65700 1 --2- 172.21.15.44:0/2933600696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431a300 0x5584d4339200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.268+0000 7f122ad65700 1 -- 172.21.15.44:0/2933600696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d431acc0 msgr2=0x5584d4339740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.268+0000 7f122ad65700 1 --2- 172.21.15.44:0/2933600696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d431acc0 0x5584d4339740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.268+0000 7f122ad65700 1 -- 172.21.15.44:0/2933600696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122000e040 con 0x5584d43201a0 2023-09-10T01:27:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.268+0000 7f122ad65700 1 --2- 172.21.15.44:0/2933600696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 0x5584d43cb800 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f121c009630 tx=0x7f121c003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa002747383a652 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.268+0000 7f120dffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:27:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.268+0000 7f120dffb700 1 -- 172.21.15.44:0/2933600696 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c009170 con 0x5584d43201a0 2023-09-10T01:27:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.269+0000 7f120dffb700 1 -- 172.21.15.44:0/2933600696 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 9552+0+0 (secure 0 0 0) 0x7f121c003ed0 con 0x5584d43201a0 2023-09-10T01:27:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.269+0000 7f123898eb80 1 -- 172.21.15.44:0/2933600696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f121c0138a0 con 0x5584d43201a0 2023-09-10T01:27:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.269+0000 7f120dffb700 1 -- 172.21.15.44:0/2933600696 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=158) v2 ==== 24+0+0 (secure 0 0 0) 0x7f121c004090 con 0x5584d43201a0 2023-09-10T01:27:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.275+0000 7f123898eb80 10 client.20274 fetch_fsmap learned FSMap version 158 2023-09-10T01:27:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.275+0000 7f123898eb80 1 -- 172.21.15.44:0/2933600696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=158,monmap=4+}) v3 -- 0x5584d4097250 con 0x5584d43201a0 2023-09-10T01:27:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.284+0000 7f120dffb700 1 -- 172.21.15.44:0/2933600696 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f121c0138a0 con 0x5584d43201a0 2023-09-10T01:27:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.284+0000 7f120dffb700 1 -- 172.21.15.44:0/2933600696 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 158) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f121c01c6a0 con 0x5584d43201a0 2023-09-10T01:27:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.284+0000 7f123898eb80 10 client.20274 fetch_fsmap finished waiting for FSMap version 158 2023-09-10T01:27:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.284+0000 7f123898eb80 10 client.20274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:27:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.284+0000 7f123898eb80 20 client.20274 populate_metadata read hostname 'smithi044' 2023-09-10T01:27:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.284+0000 7f123898eb80 1 --2- 172.21.15.44:0/2933600696 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5584d43cd420 0x5584d43cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.284+0000 7f123898eb80 4 client.20274 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:27:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.284+0000 7f123898eb80 1 -- 172.21.15.44:0/2933600696 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5584d42658c0 con 0x5584d43cd420 2023-09-10T01:27:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.292+0000 7f122a564700 1 --2- 172.21.15.44:0/2933600696 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5584d43cd420 0x5584d43cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.294+0000 7f122a564700 1 --2- 172.21.15.44:0/2933600696 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5584d43cd420 0x5584d43cf840 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41df553fd32f6d05 server_cookie=d5edecee9707f45e in_seq=0 out_seq=0 2023-09-10T01:27:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.294+0000 7f120dffb700 10 client.20274 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.383+0000 7f120dffb700 1 -- 172.21.15.44:0/2933600696 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5584d42658c0 con 0x5584d43cd420 2023-09-10T01:27:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.383+0000 7f120dffb700 10 client.20274 handle_command_reply: tid=0 2023-09-10T01:27:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.383+0000 7f123898eb80 1 client.20274 shutdown 2023-09-10T01:27:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.387+0000 7f123898eb80 1 -- 172.21.15.44:0/2933600696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 msgr2=0x5584d43cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.387+0000 7f123898eb80 1 --2- 172.21.15.44:0/2933600696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 0x5584d43cb800 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f121c009630 tx=0x7f121c003aa0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.395+0000 7f123898eb80 1 -- 172.21.15.44:0/2933600696 shutdown_connections 2023-09-10T01:27:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.395+0000 7f123898eb80 1 --2- 172.21.15.44:0/2933600696 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5584d43cd420 0x5584d43cf840 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.395+0000 7f123898eb80 1 --2- 172.21.15.44:0/2933600696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584d43201a0 0x5584d43cb800 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.395+0000 7f123898eb80 1 --2- 172.21.15.44:0/2933600696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5584d431a300 0x5584d4339200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.395+0000 7f123898eb80 1 --2- 172.21.15.44:0/2933600696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584d431acc0 0x5584d4339740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.395+0000 7f123898eb80 1 -- 172.21.15.44:0/2933600696 >> 172.21.15.44:0/2933600696 conn(0x5584d428f030 msgr2=0x5584d4315060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.402+0000 7f120dffb700 0 client.20274 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.402+0000 7f123898eb80 1 -- 172.21.15.44:0/2933600696 shutdown_connections 2023-09-10T01:27:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.402+0000 7f123898eb80 1 -- 172.21.15.44:0/2933600696 wait complete. 2023-09-10T01:27:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.408+0000 7f123898eb80 20 client.20274 trim_cache size 0 max 16384 2023-09-10T01:27:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:27:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:27:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:27:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:27:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.410+0000 7f122c7c8700 1 -- 172.21.15.44:0/2851898634 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f12100a0700 msgr2=0x7f12100a2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.410+0000 7f122c7c8700 1 --2- 172.21.15.44:0/2851898634 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f12100a0700 0x7f12100a2bc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1220006ea0 tx=0x7f1220006e30 comp rx=0 tx=0).stop 2023-09-10T01:27:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.410+0000 7f122c7c8700 1 -- 172.21.15.44:0/2851898634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122410f1c0 msgr2=0x7f12241155d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.410+0000 7f122c7c8700 1 --2- 172.21.15.44:0/2851898634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122410f1c0 0x7f12241155d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f121401a8f0 tx=0x7f121401ccd0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.410+0000 7f122c7c8700 1 -- 172.21.15.44:0/2851898634 shutdown_connections 2023-09-10T01:27:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.410+0000 7f122c7c8700 1 --2- 172.21.15.44:0/2851898634 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f12100a0700 0x7f12100a2bc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.410+0000 7f122c7c8700 1 --2- 172.21.15.44:0/2851898634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1224119de0 0x7f12241c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.410+0000 7f122c7c8700 1 --2- 172.21.15.44:0/2851898634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f122410ed50 0x7f1224115070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.410+0000 7f122c7c8700 1 --2- 172.21.15.44:0/2851898634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122410f1c0 0x7f12241155d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.410+0000 7f122c7c8700 1 -- 172.21.15.44:0/2851898634 >> 172.21.15.44:0/2851898634 conn(0x7f122406c6c0 msgr2=0x7f122411b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.411+0000 7f122c7c8700 1 -- 172.21.15.44:0/2851898634 shutdown_connections 2023-09-10T01:27:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:18.411+0000 7f122c7c8700 1 -- 172.21.15.44:0/2851898634 wait complete. 2023-09-10T01:27:19.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:19 smithi131 ceph-mon[103713]: pgmap v1514: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.1 MiB/s wr, 629 op/s 2023-09-10T01:27:19.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:19 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:27:19.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:19 smithi044 ceph-mon[98676]: pgmap v1514: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.1 MiB/s wr, 629 op/s 2023-09-10T01:27:19.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:19 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:27:19.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:19 smithi099 ceph-mon[102692]: pgmap v1514: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.1 MiB/s wr, 629 op/s 2023-09-10T01:27:19.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:19 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:27:21.188 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 375.0 s, 259.9 tps, lat 375.348 ms stddev 197.847 2023-09-10T01:27:21.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:21 smithi131 ceph-mon[103713]: pgmap v1515: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 444 op/s 2023-09-10T01:27:21.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:21 smithi131 ceph-mon[103713]: osdmap e187: 12 total, 12 up, 12 in 2023-09-10T01:27:21.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:21 smithi044 ceph-mon[98676]: pgmap v1515: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 444 op/s 2023-09-10T01:27:21.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:21 smithi044 ceph-mon[98676]: osdmap e187: 12 total, 12 up, 12 in 2023-09-10T01:27:21.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:21 smithi099 ceph-mon[102692]: pgmap v1515: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 444 op/s 2023-09-10T01:27:21.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:21 smithi099 ceph-mon[102692]: osdmap e187: 12 total, 12 up, 12 in 2023-09-10T01:27:23.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:23 smithi131 ceph-mon[103713]: pgmap v1517: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.6 MiB/s wr, 693 op/s 2023-09-10T01:27:23.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:23 smithi044 ceph-mon[98676]: pgmap v1517: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.6 MiB/s wr, 693 op/s 2023-09-10T01:27:23.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:23 smithi099 ceph-mon[102692]: pgmap v1517: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.6 MiB/s wr, 693 op/s 2023-09-10T01:27:24.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:24 smithi131 ceph-mon[103713]: pgmap v1518: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 520 op/s 2023-09-10T01:27:24.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:24 smithi044 ceph-mon[98676]: pgmap v1518: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 520 op/s 2023-09-10T01:27:24.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:24 smithi099 ceph-mon[102692]: pgmap v1518: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 520 op/s 2023-09-10T01:27:26.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 380.0 s, 233.1 tps, lat 398.317 ms stddev 157.686 2023-09-10T01:27:27.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:27 smithi099 ceph-mon[102692]: pgmap v1519: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.0 MiB/s wr, 845 op/s 2023-09-10T01:27:27.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:27 smithi099 ceph-mon[102692]: osdmap e188: 12 total, 12 up, 12 in 2023-09-10T01:27:27.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:27 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:27:27.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:27 smithi131 ceph-mon[103713]: pgmap v1519: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.0 MiB/s wr, 845 op/s 2023-09-10T01:27:27.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:27 smithi131 ceph-mon[103713]: osdmap e188: 12 total, 12 up, 12 in 2023-09-10T01:27:27.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:27 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:27:27.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:27 smithi044 ceph-mon[98676]: pgmap v1519: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.0 MiB/s wr, 845 op/s 2023-09-10T01:27:27.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:27 smithi044 ceph-mon[98676]: osdmap e188: 12 total, 12 up, 12 in 2023-09-10T01:27:27.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:27 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:27:29.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:29 smithi099 ceph-mon[102692]: pgmap v1521: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.4 MiB/s wr, 992 op/s 2023-09-10T01:27:29.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:29 smithi131 ceph-mon[103713]: pgmap v1521: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.4 MiB/s wr, 992 op/s 2023-09-10T01:27:29.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:29 smithi044 ceph-mon[98676]: pgmap v1521: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.4 MiB/s wr, 992 op/s 2023-09-10T01:27:31.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 385.0 s, 232.8 tps, lat 468.256 ms stddev 1020.487 2023-09-10T01:27:31.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:31 smithi131 ceph-mon[103713]: pgmap v1522: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 450 op/s 2023-09-10T01:27:31.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:31 smithi044 ceph-mon[98676]: pgmap v1522: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 450 op/s 2023-09-10T01:27:31.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:31 smithi099 ceph-mon[102692]: pgmap v1522: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 450 op/s 2023-09-10T01:27:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.920+0000 7f5e00336700 1 -- 172.21.15.44:0/4187448336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80ffc30 msgr2=0x7f5df8100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.920+0000 7f5e00336700 1 --2- 172.21.15.44:0/4187448336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80ffc30 0x7f5df8100090 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5de80099c0 tx=0x7f5de8009cd0 comp rx=0 tx=0).stop 2023-09-10T01:27:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.920+0000 7f5e00336700 1 -- 172.21.15.44:0/4187448336 shutdown_connections 2023-09-10T01:27:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.920+0000 7f5e00336700 1 --2- 172.21.15.44:0/4187448336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5df81005d0 0x7f5df810c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.920+0000 7f5e00336700 1 --2- 172.21.15.44:0/4187448336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5df8105600 0x7f5df81059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.920+0000 7f5e00336700 1 --2- 172.21.15.44:0/4187448336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80ffc30 0x7f5df8100090 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.920+0000 7f5e00336700 1 -- 172.21.15.44:0/4187448336 >> 172.21.15.44:0/4187448336 conn(0x7f5df80769d0 msgr2=0x7f5df80fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.924+0000 7f5e00336700 1 -- 172.21.15.44:0/4187448336 shutdown_connections 2023-09-10T01:27:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.934+0000 7f5e00336700 1 -- 172.21.15.44:0/4187448336 wait complete. 2023-09-10T01:27:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.935+0000 7f5e00336700 1 Processor -- start 2023-09-10T01:27:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.935+0000 7f5e00336700 1 -- start start 2023-09-10T01:27:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.935+0000 7f5e00336700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80ffc30 0x7f5df8107130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.935+0000 7f5e00336700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5df81005d0 0x7f5df8107670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.935+0000 7f5e00336700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5df8107bb0 0x7f5df8071060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.935+0000 7f5e00336700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df810c610 con 0x7f5df81005d0 2023-09-10T01:27:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.935+0000 7f5e00336700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80715a0 con 0x7f5df8107bb0 2023-09-10T01:27:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.935+0000 7f5e00336700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df8071760 con 0x7f5df80ffc30 2023-09-10T01:27:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.935+0000 7f5dfd8d1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5df81005d0 0x7f5df8107670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.935+0000 7f5dfd8d1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5df81005d0 0x7f5df8107670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59802/0 (socket says 172.21.15.44:59802) 2023-09-10T01:27:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.935+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/688425616 learned_addr learned my addr 172.21.15.44:0/688425616 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.935+0000 7f5dfe8d3700 1 --2- 172.21.15.44:0/688425616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5df8107bb0 0x7f5df8071060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.936+0000 7f5dfe0d2700 1 --2- 172.21.15.44:0/688425616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80ffc30 0x7f5df8107130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.936+0000 7f5dfe8d3700 1 -- 172.21.15.44:0/688425616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80ffc30 msgr2=0x7f5df8107130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.936+0000 7f5dfe8d3700 1 --2- 172.21.15.44:0/688425616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80ffc30 0x7f5df8107130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.936+0000 7f5dfe8d3700 1 -- 172.21.15.44:0/688425616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5df81005d0 msgr2=0x7f5df8107670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.936+0000 7f5dfe8d3700 1 --2- 172.21.15.44:0/688425616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5df81005d0 0x7f5df8107670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.936+0000 7f5dfe8d3700 1 -- 172.21.15.44:0/688425616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de80095f0 con 0x7f5df8107bb0 2023-09-10T01:27:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.936+0000 7f5dfe8d3700 1 --2- 172.21.15.44:0/688425616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5df8107bb0 0x7f5df8071060 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5df000be20 tx=0x7f5df000bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.936+0000 7f5def7fe700 1 -- 172.21.15.44:0/688425616 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df000cd10 con 0x7f5df8107bb0 2023-09-10T01:27:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.937+0000 7f5def7fe700 1 -- 172.21.15.44:0/688425616 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5df000ce70 con 0x7f5df8107bb0 2023-09-10T01:27:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.937+0000 7f5e00336700 1 -- 172.21.15.44:0/688425616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df8071a40 con 0x7f5df8107bb0 2023-09-10T01:27:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.937+0000 7f5def7fe700 1 -- 172.21.15.44:0/688425616 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0012720 con 0x7f5df8107bb0 2023-09-10T01:27:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.937+0000 7f5e00336700 1 -- 172.21.15.44:0/688425616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df8071f50 con 0x7f5df8107bb0 2023-09-10T01:27:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.942+0000 7f5e0c4fcb80 1 Processor -- start 2023-09-10T01:27:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.944+0000 7f5e0c4fcb80 1 -- start start 2023-09-10T01:27:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.944+0000 7f5e0c4fcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab453f1a0 0x55aab453f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.944+0000 7f5e0c4fcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab4539300 0x55aab4539780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.944+0000 7f5e0c4fcb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab4539cc0 0x55aab4545f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.944+0000 7f5e0c4fcb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab44926d0 con 0x55aab4539cc0 2023-09-10T01:27:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.944+0000 7f5e0c4fcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab448fff0 con 0x55aab453f1a0 2023-09-10T01:27:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.944+0000 7f5e0c4fcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab448b8e0 con 0x55aab4539300 2023-09-10T01:27:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.945+0000 7f5def7fe700 1 -- 172.21.15.44:0/688425616 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f5df0014440 con 0x7f5df8107bb0 2023-09-10T01:27:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.951+0000 7f5def7fe700 1 --2- 172.21.15.44:0/688425616 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5de40a08f0 0x7f5de40a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.952+0000 7f5def7fe700 1 -- 172.21.15.44:0/688425616 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 9480+0+0 (secure 0 0 0) 0x7f5df0006e40 con 0x7f5df8107bb0 2023-09-10T01:27:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.952+0000 7f5dfe8d3700 1 --2- 172.21.15.44:0/688425616 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5de40a08f0 0x7f5de40a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.953+0000 7f5dfe8d3700 1 --2- 172.21.15.44:0/688425616 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5de40a08f0 0x7f5de40a2db0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5df0095830 tx=0x7f5df0004ad0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.954+0000 7f5dfe0d2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab4539300 0x55aab4539780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.954+0000 7f5dfd8d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab453f1a0 0x55aab453f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.954+0000 7f5dfd8d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab453f1a0 0x55aab453f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33314/0 (socket says 172.21.15.44:33314) 2023-09-10T01:27:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.954+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/943415900 learned_addr learned my addr 172.21.15.44:0/943415900 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.958+0000 7f5dfd8d1700 1 --2- 172.21.15.44:0/943415900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab4539cc0 0x55aab4545f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.958+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/943415900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab4539300 msgr2=0x55aab4539780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.958+0000 7f5dfd8d1700 1 --2- 172.21.15.44:0/943415900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab4539300 0x55aab4539780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.958+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/943415900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab4539cc0 msgr2=0x55aab4545f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.958+0000 7f5dfd8d1700 1 --2- 172.21.15.44:0/943415900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab4539cc0 0x55aab4545f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.958+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/943415900 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aab42b7d80 con 0x55aab453f1a0 2023-09-10T01:27:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.958+0000 7f5dfe0d2700 1 --2- 172.21.15.44:0/943415900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab4539300 0x55aab4539780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:27:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.961+0000 7f5dfd8d1700 1 --2- 172.21.15.44:0/943415900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab453f1a0 0x55aab453f580 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5de80036c0 tx=0x7f5de801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9cb4010568c141a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.962+0000 7f5decd89700 1 -- 172.21.15.44:0/943415900 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8005050 con 0x55aab453f1a0 2023-09-10T01:27:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.962+0000 7f5decd89700 1 -- 172.21.15.44:0/943415900 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5de801e070 con 0x55aab453f1a0 2023-09-10T01:27:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.962+0000 7f5decd89700 1 -- 172.21.15.44:0/943415900 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8004860 con 0x55aab453f1a0 2023-09-10T01:27:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.969+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/943415900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab453f1a0 msgr2=0x55aab453f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.969+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/943415900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab453f1a0 0x55aab453f580 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5de80036c0 tx=0x7f5de801a040 comp rx=0 tx=0).stop 2023-09-10T01:27:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.970+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/943415900 shutdown_connections 2023-09-10T01:27:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.970+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/943415900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab4539cc0 0x55aab4545f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.970+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/943415900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab453f1a0 0x55aab453f580 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.970+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/943415900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab4539300 0x55aab4539780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.970+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/943415900 >> 172.21.15.44:0/943415900 conn(0x55aab44ae030 msgr2=0x55aab4534560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.975+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/943415900 shutdown_connections 2023-09-10T01:27:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.975+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/943415900 wait complete. 2023-09-10T01:27:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.975+0000 7f5e0c4fcb80 1 -- start start 2023-09-10T01:27:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.975+0000 7f5e0c4fcb80 1 Processor -- start 2023-09-10T01:27:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5e0c4fcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539300 0x55aab4548ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5e0c4fcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab4539cc0 0x55aab4549400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5e0c4fcb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab453f1a0 0x55aab4543110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5e0c4fcb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab4429b10 con 0x55aab453f1a0 2023-09-10T01:27:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5e0c4fcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab4429810 con 0x55aab4539300 2023-09-10T01:27:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5e0c4fcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab430d8c0 con 0x55aab4539cc0 2023-09-10T01:27:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5e0c4fcb80 10 client.0 _ll_register_callbacks cb 0x55aab4373a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:27:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5dfd8d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539300 0x55aab4548ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5dfd8d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539300 0x55aab4548ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33322/0 (socket says 172.21.15.44:33322) 2023-09-10T01:27:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/2242713026 learned_addr learned my addr 172.21.15.44:0/2242713026 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5dfd8d1700 1 --2- 172.21.15.44:0/2242713026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab453f1a0 0x55aab4543110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5dfe0d2700 1 --2- 172.21.15.44:0/2242713026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab4539cc0 0x55aab4549400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/2242713026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab4539cc0 msgr2=0x55aab4549400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5dfd8d1700 1 --2- 172.21.15.44:0/2242713026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab4539cc0 0x55aab4549400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/2242713026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab453f1a0 msgr2=0x55aab4543110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5dfd8d1700 1 --2- 172.21.15.44:0/2242713026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab453f1a0 0x55aab4543110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.976+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/2242713026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df400e040 con 0x55aab4539300 2023-09-10T01:27:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.979+0000 7f5dfd8d1700 1 --2- 172.21.15.44:0/2242713026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539300 0x55aab4548ec0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5de800bf30 tx=0x7f5de8005540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf891eba885fe81 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.984+0000 7f5de1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:27:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.984+0000 7f5de1ffb700 1 -- 172.21.15.44:0/2242713026 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8003c10 con 0x55aab4539300 2023-09-10T01:27:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.984+0000 7f5de1ffb700 1 -- 172.21.15.44:0/2242713026 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 9480+0+0 (secure 0 0 0) 0x7f5de8003d70 con 0x55aab4539300 2023-09-10T01:27:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.984+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/2242713026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aab43e3490 con 0x55aab4539300 2023-09-10T01:27:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.997+0000 7f5de1ffb700 1 -- 172.21.15.44:0/2242713026 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=160) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5de8003f30 con 0x55aab4539300 2023-09-10T01:27:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.997+0000 7f5e0c4fcb80 10 client.29903 fetch_fsmap learned FSMap version 160 2023-09-10T01:27:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:31.997+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/2242713026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=160,monmap=4+}) v3 -- 0x7f5df810cca0 con 0x55aab4539300 2023-09-10T01:27:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.000+0000 7f5de1ffb700 1 -- 172.21.15.44:0/2242713026 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aab43e3490 con 0x55aab4539300 2023-09-10T01:27:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.001+0000 7f5de1ffb700 1 -- 172.21.15.44:0/2242713026 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 160) v1 ==== 7363+0+0 (secure 0 0 0) 0x7f5de800fd20 con 0x55aab4539300 2023-09-10T01:27:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.001+0000 7f5e0c4fcb80 10 client.29903 fetch_fsmap finished waiting for FSMap version 160 2023-09-10T01:27:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.001+0000 7f5e0c4fcb80 10 client.29903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:27:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.001+0000 7f5e0c4fcb80 20 client.29903 populate_metadata read hostname 'smithi044' 2023-09-10T01:27:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.001+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/2242713026 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aab4563e50 0x55aab4566240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.001+0000 7f5e0c4fcb80 4 client.29903 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:27:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.001+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/2242713026 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aab44a0840 con 0x55aab4563e50 2023-09-10T01:27:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.002+0000 7f5dfe0d2700 1 --2- 172.21.15.44:0/2242713026 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aab4563e50 0x55aab4566240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.003+0000 7f5dfe0d2700 1 --2- 172.21.15.44:0/2242713026 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aab4563e50 0x55aab4566240 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14eb2ef26504914 server_cookie=71c961cdf1446c82 in_seq=0 out_seq=0 2023-09-10T01:27:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.006+0000 7f5de1ffb700 10 client.29903 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.010+0000 7f5de1ffb700 1 -- 172.21.15.44:0/2242713026 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aab44a0840 con 0x55aab4563e50 2023-09-10T01:27:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.010+0000 7f5de1ffb700 10 client.29903 handle_command_reply: tid=0 2023-09-10T01:27:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.013+0000 7f5e0c4fcb80 1 client.29903 shutdown 2023-09-10T01:27:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.023+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/2242713026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539300 msgr2=0x55aab4548ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.023+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/2242713026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539300 0x55aab4548ec0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5de800bf30 tx=0x7f5de8005540 comp rx=0 tx=0).stop 2023-09-10T01:27:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.027+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/2242713026 shutdown_connections 2023-09-10T01:27:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.027+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/2242713026 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aab4563e50 0x55aab4566240 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.027+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/2242713026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab453f1a0 0x55aab4543110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.027+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/2242713026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539300 0x55aab4548ec0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.027+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/2242713026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab4539cc0 0x55aab4549400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.027+0000 7f5de1ffb700 0 client.29903 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.027+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/2242713026 >> 172.21.15.44:0/2242713026 conn(0x55aab44ae030 msgr2=0x55aab45337a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.038+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/2242713026 shutdown_connections 2023-09-10T01:27:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.045+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/2242713026 wait complete. 2023-09-10T01:27:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.045+0000 7f5e0c4fcb80 20 client.29903 trim_cache size 0 max 16384 2023-09-10T01:27:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.074+0000 7f5e0c4fcb80 1 Processor -- start 2023-09-10T01:27:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.074+0000 7f5e0c4fcb80 1 -- start start 2023-09-10T01:27:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.074+0000 7f5e0c4fcb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab4539300 0x55aab45430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.074+0000 7f5e0c4fcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539cc0 0x55aab4548a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.074+0000 7f5e0c4fcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab453f1a0 0x55aab4548f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.074+0000 7f5e0c4fcb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab44a0840 con 0x55aab4539300 2023-09-10T01:27:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.074+0000 7f5e0c4fcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab43e3490 con 0x55aab4539cc0 2023-09-10T01:27:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.074+0000 7f5e0c4fcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab452bf50 con 0x55aab453f1a0 2023-09-10T01:27:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.075+0000 7f5dfe0d2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539cc0 0x55aab4548a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.075+0000 7f5dfd8d1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab453f1a0 0x55aab4548f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.075+0000 7f5dfd8d1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab453f1a0 0x55aab4548f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38474/0 (socket says 172.21.15.44:38474) 2023-09-10T01:27:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.075+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/3206857979 learned_addr learned my addr 172.21.15.44:0/3206857979 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.075+0000 7f5dfe0d2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539cc0 0x55aab4548a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33334/0 (socket says 172.21.15.44:33334) 2023-09-10T01:27:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.075+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/3206857979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539cc0 msgr2=0x55aab4548a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.075+0000 7f5dfd8d1700 1 --2- 172.21.15.44:0/3206857979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539cc0 0x55aab4548a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.075+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/3206857979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab4539300 msgr2=0x55aab45430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.075+0000 7f5dfd8d1700 1 --2- 172.21.15.44:0/3206857979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab4539300 0x55aab45430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.075+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/3206857979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aab42a0c00 con 0x55aab453f1a0 2023-09-10T01:27:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.076+0000 7f5dfe0d2700 1 --2- 172.21.15.44:0/3206857979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539cc0 0x55aab4548a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:27:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.076+0000 7f5dfd8d1700 1 --2- 172.21.15.44:0/3206857979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab453f1a0 0x55aab4548f90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5de80167e0 tx=0x7f5de8003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c49bfc622740747d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.079+0000 7f5ded58a700 1 -- 172.21.15.44:0/3206857979 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de801b030 con 0x55aab453f1a0 2023-09-10T01:27:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.079+0000 7f5ded58a700 1 -- 172.21.15.44:0/3206857979 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5de8017700 con 0x55aab453f1a0 2023-09-10T01:27:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.079+0000 7f5ded58a700 1 -- 172.21.15.44:0/3206857979 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8016890 con 0x55aab453f1a0 2023-09-10T01:27:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.080+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/3206857979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab453f1a0 msgr2=0x55aab4548f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.080+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/3206857979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab453f1a0 0x55aab4548f90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5de80167e0 tx=0x7f5de8003680 comp rx=0 tx=0).stop 2023-09-10T01:27:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.091+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/3206857979 shutdown_connections 2023-09-10T01:27:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.091+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/3206857979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab453f1a0 0x55aab4548f90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.091+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/3206857979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab4539300 0x55aab45430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.091+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/3206857979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539cc0 0x55aab4548a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.091+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/3206857979 >> 172.21.15.44:0/3206857979 conn(0x55aab44ae030 msgr2=0x55aab45429b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.094+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/3206857979 shutdown_connections 2023-09-10T01:27:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.097+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/3206857979 wait complete. 2023-09-10T01:27:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.097+0000 7f5e0c4fcb80 1 -- start start 2023-09-10T01:27:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.097+0000 7f5e0c4fcb80 1 Processor -- start 2023-09-10T01:27:32.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.097+0000 7f5e0c4fcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539300 0x55aab45abda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:32.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.097+0000 7f5e0c4fcb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab4539cc0 0x55aab45ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:32.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.097+0000 7f5e0c4fcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab453f1a0 0x55aab45b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.098+0000 7f5e0c4fcb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab45ac9f0 con 0x55aab4539cc0 2023-09-10T01:27:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.098+0000 7f5e0c4fcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab45b7170 con 0x55aab4539300 2023-09-10T01:27:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.098+0000 7f5e0c4fcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab45b72e0 con 0x55aab453f1a0 2023-09-10T01:27:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.098+0000 7f5e0c4fcb80 10 client.0 _ll_register_callbacks cb 0x7f5de800baf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:27:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.098+0000 7f5dfd8d1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab453f1a0 0x55aab45b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.098+0000 7f5dfd8d1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab453f1a0 0x55aab45b6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38480/0 (socket says 172.21.15.44:38480) 2023-09-10T01:27:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.098+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/358137664 learned_addr learned my addr 172.21.15.44:0/358137664 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.098+0000 7f5dfe0d2700 1 --2- 172.21.15.44:0/358137664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab4539cc0 0x55aab45ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.098+0000 7f5dfd8d1700 1 --2- 172.21.15.44:0/358137664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539300 0x55aab45abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.098+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/358137664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539300 msgr2=0x55aab45abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.098+0000 7f5dfd8d1700 1 --2- 172.21.15.44:0/358137664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539300 0x55aab45abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.098+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/358137664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab4539cc0 msgr2=0x55aab45ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.098+0000 7f5dfd8d1700 1 --2- 172.21.15.44:0/358137664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab4539cc0 0x55aab45ac2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.098+0000 7f5dfd8d1700 1 -- 172.21.15.44:0/358137664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df400b610 con 0x55aab453f1a0 2023-09-10T01:27:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.098+0000 7f5dfd8d1700 1 --2- 172.21.15.44:0/358137664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab453f1a0 0x55aab45b6c30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009350 tx=0x7f5de8029040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77d86929626065ac server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.100+0000 7f5de1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:27:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.100+0000 7f5de1ffb700 1 -- 172.21.15.44:0/358137664 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8005520 con 0x55aab453f1a0 2023-09-10T01:27:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.100+0000 7f5de1ffb700 1 -- 172.21.15.44:0/358137664 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 9480+0+0 (secure 0 0 0) 0x7f5de801c070 con 0x55aab453f1a0 2023-09-10T01:27:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.101+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/358137664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5de801e070 con 0x55aab453f1a0 2023-09-10T01:27:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.101+0000 7f5de1ffb700 1 -- 172.21.15.44:0/358137664 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=160) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5de80163a0 con 0x55aab453f1a0 2023-09-10T01:27:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.101+0000 7f5e0c4fcb80 10 client.29914 fetch_fsmap learned FSMap version 160 2023-09-10T01:27:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.101+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/358137664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=160,monmap=4+}) v3 -- 0x7f5df810cca0 con 0x55aab453f1a0 2023-09-10T01:27:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.101+0000 7f5de1ffb700 1 -- 172.21.15.44:0/358137664 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5de801e070 con 0x55aab453f1a0 2023-09-10T01:27:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.101+0000 7f5de1ffb700 1 -- 172.21.15.44:0/358137664 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 160) v1 ==== 7363+0+0 (secure 0 0 0) 0x7f5de8016b30 con 0x55aab453f1a0 2023-09-10T01:27:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.102+0000 7f5e0c4fcb80 10 client.29914 fetch_fsmap finished waiting for FSMap version 160 2023-09-10T01:27:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.102+0000 7f5e0c4fcb80 10 client.29914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:27:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.102+0000 7f5e0c4fcb80 20 client.29914 populate_metadata read hostname 'smithi044' 2023-09-10T01:27:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.102+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/358137664 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aab45b8540 0x55aab45ba930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.102+0000 7f5e0c4fcb80 4 client.29914 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:27:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.102+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/358137664 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aab44848c0 con 0x55aab45b8540 2023-09-10T01:27:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.102+0000 7f5dfe0d2700 1 --2- 172.21.15.44:0/358137664 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aab45b8540 0x55aab45ba930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.103+0000 7f5dfe0d2700 1 --2- 172.21.15.44:0/358137664 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aab45b8540 0x55aab45ba930 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59aca947c68a2965 server_cookie=b371d0a59cadc598 in_seq=0 out_seq=0 2023-09-10T01:27:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.103+0000 7f5de1ffb700 10 client.29914 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.141+0000 7f5de1ffb700 1 -- 172.21.15.44:0/358137664 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aab44848c0 con 0x55aab45b8540 2023-09-10T01:27:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.141+0000 7f5de1ffb700 10 client.29914 handle_command_reply: tid=0 2023-09-10T01:27:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.142+0000 7f5e0c4fcb80 1 client.29914 shutdown 2023-09-10T01:27:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.150+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/358137664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab453f1a0 msgr2=0x55aab45b6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.150+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/358137664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab453f1a0 0x55aab45b6c30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009350 tx=0x7f5de8029040 comp rx=0 tx=0).stop 2023-09-10T01:27:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.151+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/358137664 shutdown_connections 2023-09-10T01:27:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.151+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/358137664 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aab45b8540 0x55aab45ba930 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.151+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/358137664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aab453f1a0 0x55aab45b6c30 secure :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009350 tx=0x7f5de8029040 comp rx=0 tx=0).stop 2023-09-10T01:27:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.151+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/358137664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab4539300 0x55aab45abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.151+0000 7f5e0c4fcb80 1 --2- 172.21.15.44:0/358137664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aab4539cc0 0x55aab45ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.151+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/358137664 >> 172.21.15.44:0/358137664 conn(0x55aab44ae030 msgr2=0x55aab45418c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.153+0000 7f5de1ffb700 0 client.29914 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.154+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/358137664 shutdown_connections 2023-09-10T01:27:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.157+0000 7f5e0c4fcb80 1 -- 172.21.15.44:0/358137664 wait complete. 2023-09-10T01:27:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.157+0000 7f5e0c4fcb80 20 client.29914 trim_cache size 0 max 16384 2023-09-10T01:27:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:27:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:27:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:27:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:27:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.157+0000 7f5e00336700 1 -- 172.21.15.44:0/688425616 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5de40a08f0 msgr2=0x7f5de40a2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.157+0000 7f5e00336700 1 --2- 172.21.15.44:0/688425616 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5de40a08f0 0x7f5de40a2db0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5df0095830 tx=0x7f5df0004ad0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.157+0000 7f5e00336700 1 -- 172.21.15.44:0/688425616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5df8107bb0 msgr2=0x7f5df8071060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.157+0000 7f5e00336700 1 --2- 172.21.15.44:0/688425616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5df8107bb0 0x7f5df8071060 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5df000be20 tx=0x7f5df000bf00 comp rx=0 tx=0).stop 2023-09-10T01:27:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.158+0000 7f5e00336700 1 -- 172.21.15.44:0/688425616 shutdown_connections 2023-09-10T01:27:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.158+0000 7f5e00336700 1 --2- 172.21.15.44:0/688425616 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5de40a08f0 0x7f5de40a2db0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.158+0000 7f5e00336700 1 --2- 172.21.15.44:0/688425616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5df8107bb0 0x7f5df8071060 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.158+0000 7f5e00336700 1 --2- 172.21.15.44:0/688425616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80ffc30 0x7f5df8107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.158+0000 7f5e00336700 1 --2- 172.21.15.44:0/688425616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5df81005d0 0x7f5df8107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.158+0000 7f5e00336700 1 -- 172.21.15.44:0/688425616 >> 172.21.15.44:0/688425616 conn(0x7f5df80769d0 msgr2=0x7f5df80fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.159+0000 7f5e00336700 1 -- 172.21.15.44:0/688425616 shutdown_connections 2023-09-10T01:27:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:32.159+0000 7f5e00336700 1 -- 172.21.15.44:0/688425616 wait complete. 2023-09-10T01:27:33.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:33 smithi131 ceph-mon[103713]: pgmap v1523: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.4 MiB/s wr, 779 op/s 2023-09-10T01:27:33.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:33 smithi044 ceph-mon[98676]: pgmap v1523: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.4 MiB/s wr, 779 op/s 2023-09-10T01:27:33.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:33 smithi099 ceph-mon[102692]: pgmap v1523: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.4 MiB/s wr, 779 op/s 2023-09-10T01:27:35.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:35 smithi131 ceph-mon[103713]: pgmap v1524: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.4 MiB/s wr, 779 op/s 2023-09-10T01:27:35.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:35 smithi044 ceph-mon[98676]: pgmap v1524: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.4 MiB/s wr, 779 op/s 2023-09-10T01:27:35.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:35 smithi099 ceph-mon[102692]: pgmap v1524: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.4 MiB/s wr, 779 op/s 2023-09-10T01:27:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 390.0 s, 307.4 tps, lat 320.571 ms stddev 140.280 2023-09-10T01:27:37.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:37 smithi131 ceph-mon[103713]: pgmap v1525: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.4 MiB/s wr, 767 op/s 2023-09-10T01:27:37.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:37 smithi044 ceph-mon[98676]: pgmap v1525: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.4 MiB/s wr, 767 op/s 2023-09-10T01:27:37.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:37 smithi099 ceph-mon[102692]: pgmap v1525: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.4 MiB/s wr, 767 op/s 2023-09-10T01:27:39.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:39 smithi131 ceph-mon[103713]: pgmap v1526: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.0 MiB/s wr, 713 op/s 2023-09-10T01:27:39.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:39 smithi044 ceph-mon[98676]: pgmap v1526: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.0 MiB/s wr, 713 op/s 2023-09-10T01:27:39.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:39 smithi099 ceph-mon[102692]: pgmap v1526: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.0 MiB/s wr, 713 op/s 2023-09-10T01:27:41.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 395.0 s, 308.0 tps, lat 320.452 ms stddev 441.400 2023-09-10T01:27:41.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:41 smithi131 ceph-mon[103713]: pgmap v1527: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 676 op/s 2023-09-10T01:27:41.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:41 smithi044 ceph-mon[98676]: pgmap v1527: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 676 op/s 2023-09-10T01:27:41.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:41 smithi099 ceph-mon[102692]: pgmap v1527: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 676 op/s 2023-09-10T01:27:43.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:43 smithi131 ceph-mon[103713]: pgmap v1528: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 10 MiB/s wr, 1.06k op/s 2023-09-10T01:27:43.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:43 smithi044 ceph-mon[98676]: pgmap v1528: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 10 MiB/s wr, 1.06k op/s 2023-09-10T01:27:43.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:43 smithi099 ceph-mon[102692]: pgmap v1528: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 10 MiB/s wr, 1.06k op/s 2023-09-10T01:27:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.376+0000 7f55d540f700 1 -- 172.21.15.44:0/4217123649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d0106d20 msgr2=0x7f55d01071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.376+0000 7f55d540f700 1 --2- 172.21.15.44:0/4217123649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d0106d20 0x7f55d01071a0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f55c801c0d0 tx=0x7f55c801c3e0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.376+0000 7f55d540f700 1 -- 172.21.15.44:0/4217123649 shutdown_connections 2023-09-10T01:27:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.376+0000 7f55d540f700 1 --2- 172.21.15.44:0/4217123649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55d01076e0 0x7f55d0113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.376+0000 7f55d540f700 1 --2- 172.21.15.44:0/4217123649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d010cc80 0x7f55d010d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.376+0000 7f55d540f700 1 --2- 172.21.15.44:0/4217123649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d0106d20 0x7f55d01071a0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.376+0000 7f55d540f700 1 -- 172.21.15.44:0/4217123649 >> 172.21.15.44:0/4217123649 conn(0x7f55d00764a0 msgr2=0x7f55d0105420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.376+0000 7f55d540f700 1 -- 172.21.15.44:0/4217123649 shutdown_connections 2023-09-10T01:27:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.376+0000 7f55d540f700 1 -- 172.21.15.44:0/4217123649 wait complete. 2023-09-10T01:27:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.377+0000 7f55d540f700 1 Processor -- start 2023-09-10T01:27:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.377+0000 7f55d540f700 1 -- start start 2023-09-10T01:27:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.377+0000 7f55d540f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55d0106d20 0x7f55d010f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.377+0000 7f55d540f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d01076e0 0x7f55d010f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.377+0000 7f55d540f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d010cc80 0x7f55d0080f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.377+0000 7f55d540f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d0078e20 con 0x7f55d01076e0 2023-09-10T01:27:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.377+0000 7f55d540f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d0078f60 con 0x7f55d0106d20 2023-09-10T01:27:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.377+0000 7f55d540f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d0079120 con 0x7f55d010cc80 2023-09-10T01:27:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.377+0000 7f55ce7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d01076e0 0x7f55d010f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.377+0000 7f55ce7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d01076e0 0x7f55d010f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59844/0 (socket says 172.21.15.44:59844) 2023-09-10T01:27:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.377+0000 7f55ce7fc700 1 -- 172.21.15.44:0/2587415825 learned_addr learned my addr 172.21.15.44:0/2587415825 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.377+0000 7f55ceffd700 1 --2- 172.21.15.44:0/2587415825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55d0106d20 0x7f55d010f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.377+0000 7f55cf7fe700 1 --2- 172.21.15.44:0/2587415825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d010cc80 0x7f55d0080f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.378+0000 7f55ce7fc700 1 -- 172.21.15.44:0/2587415825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d010cc80 msgr2=0x7f55d0080f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.378+0000 7f55ce7fc700 1 --2- 172.21.15.44:0/2587415825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d010cc80 0x7f55d0080f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.378+0000 7f55ce7fc700 1 -- 172.21.15.44:0/2587415825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55d0106d20 msgr2=0x7f55d010f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.378+0000 7f55ce7fc700 1 --2- 172.21.15.44:0/2587415825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55d0106d20 0x7f55d010f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.378+0000 7f55ce7fc700 1 -- 172.21.15.44:0/2587415825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c801ad10 con 0x7f55d01076e0 2023-09-10T01:27:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.378+0000 7f55ce7fc700 1 --2- 172.21.15.44:0/2587415825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d01076e0 0x7f55d010f9c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f55c801a300 tx=0x7f55c801a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.380+0000 7f55b7fff700 1 -- 172.21.15.44:0/2587415825 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c801f040 con 0x7f55d01076e0 2023-09-10T01:27:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.380+0000 7f55d540f700 1 -- 172.21.15.44:0/2587415825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d00793a0 con 0x7f55d01076e0 2023-09-10T01:27:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.380+0000 7f55d540f700 1 -- 172.21.15.44:0/2587415825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d00798b0 con 0x7f55d01076e0 2023-09-10T01:27:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.380+0000 7f55b7fff700 1 -- 172.21.15.44:0/2587415825 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55c8003e10 con 0x7f55d01076e0 2023-09-10T01:27:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.381+0000 7f55b7fff700 1 -- 172.21.15.44:0/2587415825 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8005330 con 0x7f55d01076e0 2023-09-10T01:27:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.384+0000 7f55b7fff700 1 -- 172.21.15.44:0/2587415825 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f55c8005490 con 0x7f55d01076e0 2023-09-10T01:27:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.385+0000 7f55b7fff700 1 --2- 172.21.15.44:0/2587415825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55b80a0940 0x7f55b80a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.385+0000 7f55b7fff700 1 -- 172.21.15.44:0/2587415825 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 9480+0+0 (secure 0 0 0) 0x7f55c80de420 con 0x7f55d01076e0 2023-09-10T01:27:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.385+0000 7f55ceffd700 1 --2- 172.21.15.44:0/2587415825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55b80a0940 0x7f55b80a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.389+0000 7f55e15d5b80 1 Processor -- start 2023-09-10T01:27:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.391+0000 7f55e15d5b80 1 -- start start 2023-09-10T01:27:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.391+0000 7f55e15d5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5622955261a0 0x562295526580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.391+0000 7f55e15d5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562295520300 0x562295520780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.391+0000 7f55e15d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562295520cc0 0x56229552cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.391+0000 7f55e15d5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622954796d0 con 0x562295520300 2023-09-10T01:27:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.391+0000 7f55e15d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562295476ff0 con 0x562295520cc0 2023-09-10T01:27:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.391+0000 7f55e15d5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622954728e0 con 0x5622955261a0 2023-09-10T01:27:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.391+0000 7f55ce7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562295520300 0x562295520780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.391+0000 7f55ce7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562295520300 0x562295520780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59850/0 (socket says 172.21.15.44:59850) 2023-09-10T01:27:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.391+0000 7f55ce7fc700 1 -- 172.21.15.44:0/3442416632 learned_addr learned my addr 172.21.15.44:0/3442416632 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.391+0000 7f55cf7fe700 1 --2- 172.21.15.44:0/3442416632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562295520cc0 0x56229552cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.391+0000 7f55ceffd700 1 --2- 172.21.15.44:0/2587415825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55b80a0940 0x7f55b80a2e00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f55d0108050 tx=0x7f55c404d1a0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.392+0000 7f55ceffd700 1 --2- 172.21.15.44:0/3442416632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5622955261a0 0x562295526580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.392+0000 7f55ce7fc700 1 -- 172.21.15.44:0/3442416632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5622955261a0 msgr2=0x562295526580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.392+0000 7f55ce7fc700 1 --2- 172.21.15.44:0/3442416632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5622955261a0 0x562295526580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.392+0000 7f55ce7fc700 1 -- 172.21.15.44:0/3442416632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562295520cc0 msgr2=0x56229552cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.392+0000 7f55ce7fc700 1 --2- 172.21.15.44:0/3442416632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562295520cc0 0x56229552cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.392+0000 7f55ce7fc700 1 -- 172.21.15.44:0/3442416632 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56229529ed80 con 0x562295520300 2023-09-10T01:27:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.392+0000 7f55ce7fc700 1 --2- 172.21.15.44:0/3442416632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562295520300 0x562295520780 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f55c80a8010 tx=0x7f55c80ab440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93caf173123a0762 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.393+0000 7f55b57fa700 1 -- 172.21.15.44:0/3442416632 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c80ab5a0 con 0x562295520300 2023-09-10T01:27:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.393+0000 7f55b57fa700 1 -- 172.21.15.44:0/3442416632 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55c80af3d0 con 0x562295520300 2023-09-10T01:27:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.393+0000 7f55b57fa700 1 -- 172.21.15.44:0/3442416632 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c80a9a60 con 0x562295520300 2023-09-10T01:27:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.394+0000 7f55e15d5b80 1 -- 172.21.15.44:0/3442416632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562295520300 msgr2=0x562295520780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.394+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/3442416632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562295520300 0x562295520780 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f55c80a8010 tx=0x7f55c80ab440 comp rx=0 tx=0).stop 2023-09-10T01:27:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.400+0000 7f55e15d5b80 1 -- 172.21.15.44:0/3442416632 shutdown_connections 2023-09-10T01:27:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.400+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/3442416632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562295520cc0 0x56229552cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.400+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/3442416632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5622955261a0 0x562295526580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.400+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/3442416632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562295520300 0x562295520780 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.400+0000 7f55e15d5b80 1 -- 172.21.15.44:0/3442416632 >> 172.21.15.44:0/3442416632 conn(0x562295495030 msgr2=0x56229551b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.400+0000 7f55e15d5b80 1 -- 172.21.15.44:0/3442416632 shutdown_connections 2023-09-10T01:27:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.400+0000 7f55e15d5b80 1 -- 172.21.15.44:0/3442416632 wait complete. 2023-09-10T01:27:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.401+0000 7f55e15d5b80 1 -- start start 2023-09-10T01:27:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.401+0000 7f55e15d5b80 1 Processor -- start 2023-09-10T01:27:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.401+0000 7f55e15d5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562295520300 0x56229552fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.401+0000 7f55e15d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562295520cc0 0x5622955300e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.401+0000 7f55e15d5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622955261a0 0x562295529e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.401+0000 7f55e15d5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562295410b10 con 0x5622955261a0 2023-09-10T01:27:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.401+0000 7f55e15d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562295410810 con 0x562295520cc0 2023-09-10T01:27:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.401+0000 7f55e15d5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622952f48c0 con 0x562295520300 2023-09-10T01:27:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.401+0000 7f55e15d5b80 10 client.0 _ll_register_callbacks cb 0x56229535aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:27:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.402+0000 7f55cf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622955261a0 0x562295529e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.402+0000 7f55cf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622955261a0 0x562295529e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59858/0 (socket says 172.21.15.44:59858) 2023-09-10T01:27:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.402+0000 7f55cf7fe700 1 -- 172.21.15.44:0/2758737529 learned_addr learned my addr 172.21.15.44:0/2758737529 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.402+0000 7f55ceffd700 1 --2- 172.21.15.44:0/2758737529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562295520300 0x56229552fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.402+0000 7f55cf7fe700 1 -- 172.21.15.44:0/2758737529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562295520300 msgr2=0x56229552fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.402+0000 7f55cf7fe700 1 --2- 172.21.15.44:0/2758737529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562295520300 0x56229552fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.402+0000 7f55cf7fe700 1 -- 172.21.15.44:0/2758737529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562295520cc0 msgr2=0x5622955300e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.402+0000 7f55cf7fe700 1 --2- 172.21.15.44:0/2758737529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562295520cc0 0x5622955300e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.402+0000 7f55cf7fe700 1 -- 172.21.15.44:0/2758737529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d00798b0 con 0x5622955261a0 2023-09-10T01:27:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.403+0000 7f55cf7fe700 1 --2- 172.21.15.44:0/2758737529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622955261a0 0x562295529e30 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f55c0010800 tx=0x7f55c0010b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e916a2177580a285 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.403+0000 7f55b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:27:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.403+0000 7f55b2ff5700 1 -- 172.21.15.44:0/2758737529 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0015070 con 0x5622955261a0 2023-09-10T01:27:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.403+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2758737529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622953ca490 con 0x5622955261a0 2023-09-10T01:27:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.403+0000 7f55b2ff5700 1 -- 172.21.15.44:0/2758737529 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 9480+0+0 (secure 0 0 0) 0x7f55c000f830 con 0x5622955261a0 2023-09-10T01:27:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.403+0000 7f55b2ff5700 1 -- 172.21.15.44:0/2758737529 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=160) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55c000f9f0 con 0x5622955261a0 2023-09-10T01:27:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.403+0000 7f55e15d5b80 10 client.20298 fetch_fsmap learned FSMap version 160 2023-09-10T01:27:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.403+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2758737529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=160,monmap=4+}) v3 -- 0x562295287c00 con 0x5622955261a0 2023-09-10T01:27:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.404+0000 7f55b2ff5700 1 -- 172.21.15.44:0/2758737529 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5622953ca490 con 0x5622955261a0 2023-09-10T01:27:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.404+0000 7f55b2ff5700 1 -- 172.21.15.44:0/2758737529 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 160) v1 ==== 7363+0+0 (secure 0 0 0) 0x7f55c0019c00 con 0x5622955261a0 2023-09-10T01:27:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.404+0000 7f55e15d5b80 10 client.20298 fetch_fsmap finished waiting for FSMap version 160 2023-09-10T01:27:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.404+0000 7f55e15d5b80 10 client.20298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:27:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.404+0000 7f55e15d5b80 20 client.20298 populate_metadata read hostname 'smithi044' 2023-09-10T01:27:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.404+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/2758737529 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56229553a1a0 0x56229553c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.404+0000 7f55e15d5b80 4 client.20298 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:27:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.404+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2758737529 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562295487840 con 0x56229553a1a0 2023-09-10T01:27:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.404+0000 7f55ceffd700 1 --2- 172.21.15.44:0/2758737529 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56229553a1a0 0x56229553c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.411+0000 7f55ceffd700 1 --2- 172.21.15.44:0/2758737529 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56229553a1a0 0x56229553c5c0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1833da3307a58638 server_cookie=546d1d343d7dabe0 in_seq=0 out_seq=0 2023-09-10T01:27:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.411+0000 7f55b2ff5700 10 client.20298 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.415+0000 7f55b2ff5700 1 -- 172.21.15.44:0/2758737529 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562295487840 con 0x56229553a1a0 2023-09-10T01:27:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.415+0000 7f55b2ff5700 10 client.20298 handle_command_reply: tid=0 2023-09-10T01:27:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.415+0000 7f55e15d5b80 1 client.20298 shutdown 2023-09-10T01:27:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.415+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2758737529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622955261a0 msgr2=0x562295529e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.415+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/2758737529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622955261a0 0x562295529e30 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f55c0010800 tx=0x7f55c0010b10 comp rx=0 tx=0).stop 2023-09-10T01:27:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.420+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2758737529 shutdown_connections 2023-09-10T01:27:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.420+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/2758737529 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56229553a1a0 0x56229553c5c0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.420+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/2758737529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622955261a0 0x562295529e30 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.420+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/2758737529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562295520300 0x56229552fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.420+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/2758737529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562295520cc0 0x5622955300e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.420+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2758737529 >> 172.21.15.44:0/2758737529 conn(0x562295495030 msgr2=0x56229551a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.420+0000 7f55b2ff5700 0 client.20298 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.432+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2758737529 shutdown_connections 2023-09-10T01:27:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.433+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2758737529 wait complete. 2023-09-10T01:27:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.443+0000 7f55e15d5b80 20 client.20298 trim_cache size 0 max 16384 2023-09-10T01:27:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.472+0000 7f55e15d5b80 1 Processor -- start 2023-09-10T01:27:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.472+0000 7f55e15d5b80 1 -- start start 2023-09-10T01:27:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.472+0000 7f55e15d5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562295520300 0x562295529de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.472+0000 7f55e15d5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562295520cc0 0x56229552f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.473+0000 7f55e15d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622955261a0 0x56229552fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.473+0000 7f55e15d5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56229552a320 con 0x562295520cc0 2023-09-10T01:27:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.473+0000 7f55e15d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562295517fc0 con 0x5622955261a0 2023-09-10T01:27:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.473+0000 7f55e15d5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622955377a0 con 0x562295520300 2023-09-10T01:27:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.479+0000 7f55cf7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622955261a0 0x56229552fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.479+0000 7f55cf7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622955261a0 0x56229552fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33374/0 (socket says 172.21.15.44:33374) 2023-09-10T01:27:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.479+0000 7f55cf7fe700 1 -- 172.21.15.44:0/2399802224 learned_addr learned my addr 172.21.15.44:0/2399802224 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.479+0000 7f55cf7fe700 1 -- 172.21.15.44:0/2399802224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562295520300 msgr2=0x562295529de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.479+0000 7f55cf7fe700 1 --2- 172.21.15.44:0/2399802224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562295520300 0x562295529de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.479+0000 7f55cf7fe700 1 -- 172.21.15.44:0/2399802224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562295520cc0 msgr2=0x56229552f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.479+0000 7f55cf7fe700 1 --2- 172.21.15.44:0/2399802224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562295520cc0 0x56229552f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.479+0000 7f55cf7fe700 1 -- 172.21.15.44:0/2399802224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622952c51f0 con 0x5622955261a0 2023-09-10T01:27:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.480+0000 7f55cf7fe700 1 --2- 172.21.15.44:0/2399802224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622955261a0 0x56229552fcb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f55c0005850 tx=0x7f55c000a360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b72c7948b4efbd2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.480+0000 7f55b5ffb700 1 -- 172.21.15.44:0/2399802224 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0015040 con 0x5622955261a0 2023-09-10T01:27:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.480+0000 7f55b5ffb700 1 -- 172.21.15.44:0/2399802224 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55c0018df0 con 0x5622955261a0 2023-09-10T01:27:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.480+0000 7f55b5ffb700 1 -- 172.21.15.44:0/2399802224 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c000f830 con 0x5622955261a0 2023-09-10T01:27:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.481+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2399802224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622955261a0 msgr2=0x56229552fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.481+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/2399802224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622955261a0 0x56229552fcb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f55c0005850 tx=0x7f55c000a360 comp rx=0 tx=0).stop 2023-09-10T01:27:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.481+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2399802224 shutdown_connections 2023-09-10T01:27:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.481+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/2399802224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622955261a0 0x56229552fcb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.481+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/2399802224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562295520300 0x562295529de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.481+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/2399802224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562295520cc0 0x56229552f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.481+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2399802224 >> 172.21.15.44:0/2399802224 conn(0x562295495030 msgr2=0x5622955296d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.492+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2399802224 shutdown_connections 2023-09-10T01:27:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.497+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2399802224 wait complete. 2023-09-10T01:27:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.498+0000 7f55e15d5b80 1 -- start start 2023-09-10T01:27:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.498+0000 7f55e15d5b80 1 Processor -- start 2023-09-10T01:27:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.498+0000 7f55e15d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562295520300 0x562295587ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.499+0000 7f55e15d5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622955261a0 0x562295588010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.499+0000 7f55e15d5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562295581f60 0x5622955823e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.499+0000 7f55e15d5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562295582920 con 0x5622955261a0 2023-09-10T01:27:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.499+0000 7f55e15d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562295582a90 con 0x562295520300 2023-09-10T01:27:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.499+0000 7f55e15d5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562295582c50 con 0x562295581f60 2023-09-10T01:27:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.499+0000 7f55e15d5b80 10 client.0 _ll_register_callbacks cb 0x7f55c000ac90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:27:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.499+0000 7f55ceffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562295520300 0x562295587ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.499+0000 7f55ceffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562295520300 0x562295587ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33378/0 (socket says 172.21.15.44:33378) 2023-09-10T01:27:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.499+0000 7f55ceffd700 1 -- 172.21.15.44:0/2916752570 learned_addr learned my addr 172.21.15.44:0/2916752570 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.499+0000 7f55cf7fe700 1 --2- 172.21.15.44:0/2916752570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562295581f60 0x5622955823e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.499+0000 7f55ce7fc700 1 --2- 172.21.15.44:0/2916752570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622955261a0 0x562295588010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.499+0000 7f55ceffd700 1 -- 172.21.15.44:0/2916752570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562295581f60 msgr2=0x5622955823e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.499+0000 7f55ceffd700 1 --2- 172.21.15.44:0/2916752570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562295581f60 0x5622955823e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.499+0000 7f55ceffd700 1 -- 172.21.15.44:0/2916752570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622955261a0 msgr2=0x562295588010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.499+0000 7f55ceffd700 1 --2- 172.21.15.44:0/2916752570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622955261a0 0x562295588010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.499+0000 7f55ceffd700 1 -- 172.21.15.44:0/2916752570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562295287c00 con 0x562295520300 2023-09-10T01:27:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.500+0000 7f55ceffd700 1 --2- 172.21.15.44:0/2916752570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562295520300 0x562295587ad0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x56229553cea0 tx=0x7f55c4004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94d3f6befc8b746a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.504+0000 7f55b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:27:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.504+0000 7f55b2ff5700 1 -- 172.21.15.44:0/2916752570 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c406c520 con 0x562295520300 2023-09-10T01:27:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.504+0000 7f55b2ff5700 1 -- 172.21.15.44:0/2916752570 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 9480+0+0 (secure 0 0 0) 0x7f55c4069070 con 0x562295520300 2023-09-10T01:27:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.504+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2916752570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55c80af3d0 con 0x562295520300 2023-09-10T01:27:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.504+0000 7f55b2ff5700 1 -- 172.21.15.44:0/2916752570 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=160) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55c4005e90 con 0x562295520300 2023-09-10T01:27:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.504+0000 7f55e15d5b80 10 client.29921 fetch_fsmap learned FSMap version 160 2023-09-10T01:27:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.504+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2916752570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=160,monmap=4+}) v3 -- 0x56229529d250 con 0x562295520300 2023-09-10T01:27:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.509+0000 7f55b2ff5700 1 -- 172.21.15.44:0/2916752570 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55c80af3d0 con 0x562295520300 2023-09-10T01:27:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.509+0000 7f55b2ff5700 1 -- 172.21.15.44:0/2916752570 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 160) v1 ==== 7363+0+0 (secure 0 0 0) 0x7f55c40064c0 con 0x562295520300 2023-09-10T01:27:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.509+0000 7f55e15d5b80 10 client.29921 fetch_fsmap finished waiting for FSMap version 160 2023-09-10T01:27:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.509+0000 7f55e15d5b80 10 client.29921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:27:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.509+0000 7f55e15d5b80 20 client.29921 populate_metadata read hostname 'smithi044' 2023-09-10T01:27:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.509+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/2916752570 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56229558fde0 0x562295592200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.509+0000 7f55e15d5b80 4 client.29921 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:27:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.509+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2916752570 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56229546b8c0 con 0x56229558fde0 2023-09-10T01:27:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.511+0000 7f55ce7fc700 1 --2- 172.21.15.44:0/2916752570 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56229558fde0 0x562295592200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.534+0000 7f55b2ff5700 10 client.29921 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.534+0000 7f55ce7fc700 1 --2- 172.21.15.44:0/2916752570 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56229558fde0 0x562295592200 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca4143dbdcd06bc server_cookie=71dedd19f591f490 in_seq=0 out_seq=0 2023-09-10T01:27:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.569+0000 7f55b2ff5700 1 -- 172.21.15.44:0/2916752570 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56229546b8c0 con 0x56229558fde0 2023-09-10T01:27:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.569+0000 7f55b2ff5700 10 client.29921 handle_command_reply: tid=0 2023-09-10T01:27:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.569+0000 7f55e15d5b80 1 client.29921 shutdown 2023-09-10T01:27:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.577+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2916752570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562295520300 msgr2=0x562295587ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.578+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/2916752570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562295520300 0x562295587ad0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x56229553cea0 tx=0x7f55c4004860 comp rx=0 tx=0).stop 2023-09-10T01:27:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.581+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2916752570 shutdown_connections 2023-09-10T01:27:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.581+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/2916752570 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56229558fde0 0x562295592200 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.581+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/2916752570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562295581f60 0x5622955823e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.581+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/2916752570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562295520300 0x562295587ad0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.581+0000 7f55e15d5b80 1 --2- 172.21.15.44:0/2916752570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622955261a0 0x562295588010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.581+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2916752570 >> 172.21.15.44:0/2916752570 conn(0x562295495030 msgr2=0x562295528600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.581+0000 7f55b2ff5700 0 client.29921 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.581+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2916752570 shutdown_connections 2023-09-10T01:27:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.581+0000 7f55e15d5b80 1 -- 172.21.15.44:0/2916752570 wait complete. 2023-09-10T01:27:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.581+0000 7f55e15d5b80 20 client.29921 trim_cache size 0 max 16384 2023-09-10T01:27:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:27:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:27:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:27:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:27:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.582+0000 7f55d540f700 1 -- 172.21.15.44:0/2587415825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55b80a0940 msgr2=0x7f55b80a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.582+0000 7f55d540f700 1 --2- 172.21.15.44:0/2587415825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55b80a0940 0x7f55b80a2e00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f55d0108050 tx=0x7f55c404d1a0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.582+0000 7f55d540f700 1 -- 172.21.15.44:0/2587415825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d01076e0 msgr2=0x7f55d010f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.582+0000 7f55d540f700 1 --2- 172.21.15.44:0/2587415825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d01076e0 0x7f55d010f9c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f55c801a300 tx=0x7f55c801a330 comp rx=0 tx=0).stop 2023-09-10T01:27:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.582+0000 7f55d540f700 1 -- 172.21.15.44:0/2587415825 shutdown_connections 2023-09-10T01:27:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.582+0000 7f55d540f700 1 --2- 172.21.15.44:0/2587415825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f55b80a0940 0x7f55b80a2e00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.582+0000 7f55d540f700 1 --2- 172.21.15.44:0/2587415825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d010cc80 0x7f55d0080f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.582+0000 7f55d540f700 1 --2- 172.21.15.44:0/2587415825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55d0106d20 0x7f55d010f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.582+0000 7f55d540f700 1 --2- 172.21.15.44:0/2587415825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d01076e0 0x7f55d010f9c0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.582+0000 7f55d540f700 1 -- 172.21.15.44:0/2587415825 >> 172.21.15.44:0/2587415825 conn(0x7f55d00764a0 msgr2=0x7f55d0077410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.582+0000 7f55d540f700 1 -- 172.21.15.44:0/2587415825 shutdown_connections 2023-09-10T01:27:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:45.582+0000 7f55d540f700 1 -- 172.21.15.44:0/2587415825 wait complete. 2023-09-10T01:27:45.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:45 smithi131 ceph-mon[103713]: pgmap v1529: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.2 MiB/s wr, 725 op/s 2023-09-10T01:27:45.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:45 smithi044 ceph-mon[98676]: pgmap v1529: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.2 MiB/s wr, 725 op/s 2023-09-10T01:27:45.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:45 smithi099 ceph-mon[102692]: pgmap v1529: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.2 MiB/s wr, 725 op/s 2023-09-10T01:27:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 400.0 s, 268.6 tps, lat 386.331 ms stddev 157.673 2023-09-10T01:27:47.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:47 smithi099 ceph-mon[102692]: pgmap v1530: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-09-10T01:27:47.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:47 smithi131 ceph-mon[103713]: pgmap v1530: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-09-10T01:27:47.816 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:47 smithi044 ceph-mon[98676]: pgmap v1530: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-09-10T01:27:49.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:49 smithi131 ceph-mon[103713]: pgmap v1531: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.8 MiB/s wr, 759 op/s 2023-09-10T01:27:49.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:49 smithi044 ceph-mon[98676]: pgmap v1531: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.8 MiB/s wr, 759 op/s 2023-09-10T01:27:49.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:49 smithi099 ceph-mon[102692]: pgmap v1531: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.8 MiB/s wr, 759 op/s 2023-09-10T01:27:51.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 405.0 s, 235.8 tps, lat 400.981 ms stddev 147.056 2023-09-10T01:27:51.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:51 smithi131 ceph-mon[103713]: pgmap v1532: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.3 MiB/s wr, 722 op/s 2023-09-10T01:27:51.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:51 smithi044 ceph-mon[98676]: pgmap v1532: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.3 MiB/s wr, 722 op/s 2023-09-10T01:27:51.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:51 smithi099 ceph-mon[102692]: pgmap v1532: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.3 MiB/s wr, 722 op/s 2023-09-10T01:27:53.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:53 smithi131 ceph-mon[103713]: pgmap v1533: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 11 MiB/s wr, 1.05k op/s 2023-09-10T01:27:53.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:53 smithi044 ceph-mon[98676]: pgmap v1533: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 11 MiB/s wr, 1.05k op/s 2023-09-10T01:27:53.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:53 smithi099 ceph-mon[102692]: pgmap v1533: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 11 MiB/s wr, 1.05k op/s 2023-09-10T01:27:55.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:55 smithi131 ceph-mon[103713]: pgmap v1534: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 667 op/s 2023-09-10T01:27:55.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:55 smithi044 ceph-mon[98676]: pgmap v1534: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 667 op/s 2023-09-10T01:27:55.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:55 smithi099 ceph-mon[102692]: pgmap v1534: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 667 op/s 2023-09-10T01:27:56.196 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 410.0 s, 211.0 tps, lat 501.534 ms stddev 570.873 2023-09-10T01:27:57.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:57 smithi131 ceph-mon[103713]: pgmap v1535: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-09-10T01:27:57.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:57 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:27:57.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:57 smithi044 ceph-mon[98676]: pgmap v1535: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-09-10T01:27:57.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:57 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:27:57.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:57 smithi099 ceph-mon[102692]: pgmap v1535: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-09-10T01:27:57.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:57 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:27:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.946+0000 7fbcec464700 1 -- 172.21.15.44:0/3421336602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbce4108fa0 msgr2=0x7fbce4109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.946+0000 7fbcec464700 1 --2- 172.21.15.44:0/3421336602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbce4108fa0 0x7fbce4109380 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00b0a0 tx=0x7fbcdc00b3b0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.948+0000 7fbcec464700 1 -- 172.21.15.44:0/3421336602 shutdown_connections 2023-09-10T01:27:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.948+0000 7fbcec464700 1 --2- 172.21.15.44:0/3421336602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbce4112550 0x7fbce4114940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.948+0000 7fbcec464700 1 --2- 172.21.15.44:0/3421336602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbce4108fa0 0x7fbce4109380 secure :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00b0a0 tx=0x7fbcdc00b3b0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.948+0000 7fbcec464700 1 --2- 172.21.15.44:0/3421336602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbce41098c0 0x7fbce4112010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.948+0000 7fbcec464700 1 -- 172.21.15.44:0/3421336602 >> 172.21.15.44:0/3421336602 conn(0x7fbce406c6c0 msgr2=0x7fbce406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.951+0000 7fbcec464700 1 -- 172.21.15.44:0/3421336602 shutdown_connections 2023-09-10T01:27:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.951+0000 7fbcec464700 1 -- 172.21.15.44:0/3421336602 wait complete. 2023-09-10T01:27:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.952+0000 7fbcec464700 1 Processor -- start 2023-09-10T01:27:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.952+0000 7fbcec464700 1 -- start start 2023-09-10T01:27:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.952+0000 7fbcec464700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbce41098c0 0x7fbce41ab790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.952+0000 7fbcec464700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbce4112550 0x7fbce41abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.952+0000 7fbcec464700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbce41a5a50 0x7fbce41a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.953+0000 7fbcec464700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce41a6560 con 0x7fbce41098c0 2023-09-10T01:27:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.953+0000 7fbcec464700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce41a66d0 con 0x7fbce4112550 2023-09-10T01:27:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.953+0000 7fbcec464700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce41a6890 con 0x7fbce41a5a50 2023-09-10T01:27:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.953+0000 7fbce99ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbce4112550 0x7fbce41abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.953+0000 7fbce99ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbce4112550 0x7fbce41abcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33392/0 (socket says 172.21.15.44:33392) 2023-09-10T01:27:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.953+0000 7fbce99ff700 1 -- 172.21.15.44:0/387001509 learned_addr learned my addr 172.21.15.44:0/387001509 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.953+0000 7fbceaa01700 1 --2- 172.21.15.44:0/387001509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbce41a5a50 0x7fbce41a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.953+0000 7fbce99ff700 1 -- 172.21.15.44:0/387001509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbce41a5a50 msgr2=0x7fbce41a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.953+0000 7fbce99ff700 1 --2- 172.21.15.44:0/387001509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbce41a5a50 0x7fbce41a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.953+0000 7fbce99ff700 1 -- 172.21.15.44:0/387001509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbce41098c0 msgr2=0x7fbce41ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.953+0000 7fbce99ff700 1 --2- 172.21.15.44:0/387001509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbce41098c0 0x7fbce41ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.953+0000 7fbce99ff700 1 -- 172.21.15.44:0/387001509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcdc009d00 con 0x7fbce4112550 2023-09-10T01:27:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.954+0000 7fbce99ff700 1 --2- 172.21.15.44:0/387001509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbce4112550 0x7fbce41abcd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400b560 tx=0x7fbcd400b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.954+0000 7fbcdb7fe700 1 -- 172.21.15.44:0/387001509 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4011830 con 0x7fbce4112550 2023-09-10T01:27:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.954+0000 7fbcec464700 1 -- 172.21.15.44:0/387001509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce41b26c0 con 0x7fbce4112550 2023-09-10T01:27:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.954+0000 7fbcec464700 1 -- 172.21.15.44:0/387001509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce41b2c20 con 0x7fbce4112550 2023-09-10T01:27:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.954+0000 7fbcdb7fe700 1 -- 172.21.15.44:0/387001509 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcd4011e20 con 0x7fbce4112550 2023-09-10T01:27:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.954+0000 7fbcdb7fe700 1 -- 172.21.15.44:0/387001509 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4010e10 con 0x7fbce4112550 2023-09-10T01:27:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.958+0000 7fbcf862ab80 1 Processor -- start 2023-09-10T01:27:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.958+0000 7fbcf862ab80 1 -- start start 2023-09-10T01:27:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.959+0000 7fbcf862ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a6201b0 0x560f6a620590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.959+0000 7fbcf862ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a61a310 0x560f6a61a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.959+0000 7fbcf862ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a626f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.959+0000 7fbcf862ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6a5626d0 con 0x560f6a6201b0 2023-09-10T01:27:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.959+0000 7fbcf862ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6a55fff0 con 0x560f6a61acd0 2023-09-10T01:27:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.959+0000 7fbcf862ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6a55b8e0 con 0x560f6a61a310 2023-09-10T01:27:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.959+0000 7fbce99ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a626f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.959+0000 7fbce99ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a626f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33396/0 (socket says 172.21.15.44:33396) 2023-09-10T01:27:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.959+0000 7fbce99ff700 1 -- 172.21.15.44:0/639881562 learned_addr learned my addr 172.21.15.44:0/639881562 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.959+0000 7fbce99ff700 1 -- 172.21.15.44:0/639881562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a61a310 msgr2=0x560f6a61a790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:27:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.959+0000 7fbce99ff700 1 --2- 172.21.15.44:0/639881562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a61a310 0x560f6a61a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.959+0000 7fbce99ff700 1 -- 172.21.15.44:0/639881562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a6201b0 msgr2=0x560f6a620590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.959+0000 7fbce99ff700 1 --2- 172.21.15.44:0/639881562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a6201b0 0x560f6a620590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.959+0000 7fbce99ff700 1 -- 172.21.15.44:0/639881562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f6a387d80 con 0x560f6a61acd0 2023-09-10T01:27:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.960+0000 7fbce99ff700 1 --2- 172.21.15.44:0/639881562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a626f40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40c9070 tx=0x7fbcd40f63f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=408e220864813720 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.960+0000 7fbcdb7fe700 1 -- 172.21.15.44:0/387001509 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fbcd4011990 con 0x7fbce4112550 2023-09-10T01:27:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.961+0000 7fbcdb7fe700 1 --2- 172.21.15.44:0/387001509 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbcd00a08b0 0x7fbcd00a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.961+0000 7fbceaa01700 1 --2- 172.21.15.44:0/387001509 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbcd00a08b0 0x7fbcd00a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.961+0000 7fbcdb7fe700 1 -- 172.21.15.44:0/387001509 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 9480+0+0 (secure 0 0 0) 0x7fbcd40cbcf0 con 0x7fbce4112550 2023-09-10T01:27:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.962+0000 7fbceaa01700 1 --2- 172.21.15.44:0/387001509 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbcd00a08b0 0x7fbcd00a2d70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009300 tx=0x7fbce0009290 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.963+0000 7fbcd8d89700 1 -- 172.21.15.44:0/639881562 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40f4470 con 0x560f6a61acd0 2023-09-10T01:27:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.963+0000 7fbcd8d89700 1 -- 172.21.15.44:0/639881562 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcd40f4a60 con 0x560f6a61acd0 2023-09-10T01:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.963+0000 7fbcd8d89700 1 -- 172.21.15.44:0/639881562 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40fe9c0 con 0x560f6a61acd0 2023-09-10T01:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.964+0000 7fbcf862ab80 1 -- 172.21.15.44:0/639881562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 msgr2=0x560f6a626f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.964+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/639881562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a626f40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40c9070 tx=0x7fbcd40f63f0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.964+0000 7fbcf862ab80 1 -- 172.21.15.44:0/639881562 shutdown_connections 2023-09-10T01:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.964+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/639881562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a626f40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.964+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/639881562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a6201b0 0x560f6a620590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.964+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/639881562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a61a310 0x560f6a61a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.964+0000 7fbcf862ab80 1 -- 172.21.15.44:0/639881562 >> 172.21.15.44:0/639881562 conn(0x560f6a57e030 msgr2=0x560f6a604560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.966+0000 7fbcf862ab80 1 -- 172.21.15.44:0/639881562 shutdown_connections 2023-09-10T01:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.966+0000 7fbcf862ab80 1 -- 172.21.15.44:0/639881562 wait complete. 2023-09-10T01:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.966+0000 7fbcf862ab80 1 -- start start 2023-09-10T01:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.966+0000 7fbcf862ab80 1 Processor -- start 2023-09-10T01:27:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.967+0000 7fbcf862ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a61a310 0x560f6a6113a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.967+0000 7fbcf862ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a6118e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.967+0000 7fbcf862ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a6201b0 0x560f6a60c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.967+0000 7fbcf862ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6a4f9b10 con 0x560f6a6201b0 2023-09-10T01:27:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.967+0000 7fbcf862ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6a4f9810 con 0x560f6a61acd0 2023-09-10T01:27:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.967+0000 7fbcf862ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6a3dd8c0 con 0x560f6a61a310 2023-09-10T01:27:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.967+0000 7fbcf862ab80 10 client.0 _ll_register_callbacks cb 0x560f6a443a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:27:58.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.967+0000 7fbce99ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a6118e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:58.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.967+0000 7fbce99ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a6118e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33404/0 (socket says 172.21.15.44:33404) 2023-09-10T01:27:58.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.967+0000 7fbce99ff700 1 -- 172.21.15.44:0/141875251 learned_addr learned my addr 172.21.15.44:0/141875251 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.967+0000 7fbce99ff700 1 -- 172.21.15.44:0/141875251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a61a310 msgr2=0x560f6a6113a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:27:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.967+0000 7fbce99ff700 1 --2- 172.21.15.44:0/141875251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a61a310 0x560f6a6113a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.967+0000 7fbce99ff700 1 -- 172.21.15.44:0/141875251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a6201b0 msgr2=0x560f6a60c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.967+0000 7fbce99ff700 1 --2- 172.21.15.44:0/141875251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a6201b0 0x560f6a60c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.967+0000 7fbce99ff700 1 -- 172.21.15.44:0/141875251 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce41b2c20 con 0x560f6a61acd0 2023-09-10T01:27:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.968+0000 7fbce99ff700 1 --2- 172.21.15.44:0/141875251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a6118e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40048a0 tx=0x7fbcd4010850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea65a83bc67cd644 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.968+0000 7fbccdffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:27:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.968+0000 7fbccdffb700 1 -- 172.21.15.44:0/141875251 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4099ea0 con 0x560f6a61acd0 2023-09-10T01:27:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.968+0000 7fbcf862ab80 1 -- 172.21.15.44:0/141875251 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f6a4b3490 con 0x560f6a61acd0 2023-09-10T01:27:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.968+0000 7fbccdffb700 1 -- 172.21.15.44:0/141875251 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 9480+0+0 (secure 0 0 0) 0x7fbcd4011c40 con 0x560f6a61acd0 2023-09-10T01:27:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.968+0000 7fbccdffb700 1 -- 172.21.15.44:0/141875251 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcd40951d0 con 0x560f6a61acd0 2023-09-10T01:27:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.968+0000 7fbcf862ab80 10 client.29933 fetch_fsmap learned FSMap version 161 2023-09-10T01:27:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.968+0000 7fbcf862ab80 1 -- 172.21.15.44:0/141875251 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=161,monmap=4+}) v3 -- 0x560f6a370c00 con 0x560f6a61acd0 2023-09-10T01:27:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.969+0000 7fbccdffb700 1 -- 172.21.15.44:0/141875251 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560f6a4b3490 con 0x560f6a61acd0 2023-09-10T01:27:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.969+0000 7fbccdffb700 1 -- 172.21.15.44:0/141875251 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 161) v1 ==== 7359+0+0 (secure 0 0 0) 0x7fbcd409bd20 con 0x560f6a61acd0 2023-09-10T01:27:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.969+0000 7fbcf862ab80 10 client.29933 fetch_fsmap finished waiting for FSMap version 161 2023-09-10T01:27:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.969+0000 7fbcf862ab80 10 client.29933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:27:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.969+0000 7fbcf862ab80 20 client.29933 populate_metadata read hostname 'smithi044' 2023-09-10T01:27:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.969+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/141875251 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560f6a6231d0 0x560f6a6255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.969+0000 7fbcf862ab80 4 client.29933 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:27:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.969+0000 7fbcf862ab80 1 -- 172.21.15.44:0/141875251 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f6a570840 con 0x560f6a6231d0 2023-09-10T01:27:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.974+0000 7fbcea200700 1 --2- 172.21.15.44:0/141875251 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560f6a6231d0 0x560f6a6255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.974+0000 7fbcea200700 1 --2- 172.21.15.44:0/141875251 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560f6a6231d0 0x560f6a6255c0 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45bda6cc352de91a server_cookie=97e29f5e5f11d428 in_seq=0 out_seq=0 2023-09-10T01:27:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.979+0000 7fbccdffb700 10 client.29933 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.979+0000 7fbccdffb700 1 -- 172.21.15.44:0/141875251 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560f6a570840 con 0x560f6a6231d0 2023-09-10T01:27:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.979+0000 7fbccdffb700 10 client.29933 handle_command_reply: tid=0 2023-09-10T01:27:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.979+0000 7fbcf862ab80 1 client.29933 shutdown 2023-09-10T01:27:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.984+0000 7fbcf862ab80 1 -- 172.21.15.44:0/141875251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 msgr2=0x560f6a6118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.984+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/141875251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a6118e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40048a0 tx=0x7fbcd4010850 comp rx=0 tx=0).stop 2023-09-10T01:27:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.986+0000 7fbcf862ab80 1 -- 172.21.15.44:0/141875251 shutdown_connections 2023-09-10T01:27:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.986+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/141875251 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560f6a6231d0 0x560f6a6255c0 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.986+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/141875251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a6201b0 0x560f6a60c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.986+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/141875251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a61a310 0x560f6a6113a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.986+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/141875251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a6118e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.986+0000 7fbcf862ab80 1 -- 172.21.15.44:0/141875251 >> 172.21.15.44:0/141875251 conn(0x560f6a57e030 msgr2=0x560f6a6037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.986+0000 7fbccdffb700 0 client.29933 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.986+0000 7fbcf862ab80 1 -- 172.21.15.44:0/141875251 shutdown_connections 2023-09-10T01:27:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.986+0000 7fbcf862ab80 1 -- 172.21.15.44:0/141875251 wait complete. 2023-09-10T01:27:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:58.987+0000 7fbcf862ab80 20 client.29933 trim_cache size 0 max 16384 2023-09-10T01:27:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.015+0000 7fbcf862ab80 1 Processor -- start 2023-09-10T01:27:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.015+0000 7fbcf862ab80 1 -- start start 2023-09-10T01:27:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.015+0000 7fbcf862ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a57e030 0x560f6a60c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.015+0000 7fbcf862ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61a310 0x560f6a618790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.015+0000 7fbcf862ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a61acd0 0x560f6a618cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.015+0000 7fbcf862ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6a60cce0 con 0x560f6a61acd0 2023-09-10T01:27:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.015+0000 7fbcf862ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6a600ff0 con 0x560f6a61a310 2023-09-10T01:27:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.015+0000 7fbcf862ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6a605930 con 0x560f6a57e030 2023-09-10T01:27:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.015+0000 7fbcea200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a57e030 0x560f6a60c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.015+0000 7fbcea200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a57e030 0x560f6a60c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38552/0 (socket says 172.21.15.44:38552) 2023-09-10T01:27:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.015+0000 7fbcea200700 1 -- 172.21.15.44:0/3731302154 learned_addr learned my addr 172.21.15.44:0/3731302154 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.016+0000 7fbcea200700 1 -- 172.21.15.44:0/3731302154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61a310 msgr2=0x560f6a618790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:27:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.016+0000 7fbcea200700 1 --2- 172.21.15.44:0/3731302154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61a310 0x560f6a618790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.016+0000 7fbcea200700 1 -- 172.21.15.44:0/3731302154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a61acd0 msgr2=0x560f6a618cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.016+0000 7fbcea200700 1 --2- 172.21.15.44:0/3731302154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a61acd0 0x560f6a618cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.016+0000 7fbcea200700 1 -- 172.21.15.44:0/3731302154 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f6a3ae1f0 con 0x560f6a57e030 2023-09-10T01:27:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.021+0000 7fbcea200700 1 --2- 172.21.15.44:0/3731302154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a57e030 0x560f6a60c670 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc009fd0 tx=0x7fbcdc006070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6a6f52f3bbcf6c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.026+0000 7fbcd958a700 1 -- 172.21.15.44:0/3731302154 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc00e040 con 0x560f6a57e030 2023-09-10T01:27:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.026+0000 7fbcd958a700 1 -- 172.21.15.44:0/3731302154 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcdc006160 con 0x560f6a57e030 2023-09-10T01:27:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.026+0000 7fbcd958a700 1 -- 172.21.15.44:0/3731302154 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc003a00 con 0x560f6a57e030 2023-09-10T01:27:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.027+0000 7fbcf862ab80 1 -- 172.21.15.44:0/3731302154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a57e030 msgr2=0x560f6a60c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.027+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/3731302154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a57e030 0x560f6a60c670 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc009fd0 tx=0x7fbcdc006070 comp rx=0 tx=0).stop 2023-09-10T01:27:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.027+0000 7fbcf862ab80 1 -- 172.21.15.44:0/3731302154 shutdown_connections 2023-09-10T01:27:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.027+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/3731302154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a61acd0 0x560f6a618cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.027+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/3731302154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a57e030 0x560f6a60c670 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.027+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/3731302154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61a310 0x560f6a618790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.028+0000 7fbcf862ab80 1 -- 172.21.15.44:0/3731302154 >> 172.21.15.44:0/3731302154 conn(0x560f6a61e880 msgr2=0x560f6a60bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.028+0000 7fbcf862ab80 1 -- 172.21.15.44:0/3731302154 shutdown_connections 2023-09-10T01:27:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.028+0000 7fbcf862ab80 1 -- 172.21.15.44:0/3731302154 wait complete. 2023-09-10T01:27:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.028+0000 7fbcf862ab80 1 -- start start 2023-09-10T01:27:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.029+0000 7fbcf862ab80 1 Processor -- start 2023-09-10T01:27:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.029+0000 7fbcf862ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a57e030 0x560f6a615e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.029+0000 7fbcf862ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a61a310 0x560f6a614170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.029+0000 7fbcf862ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a6146b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.029+0000 7fbcf862ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6a614bf0 con 0x560f6a61a310 2023-09-10T01:27:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.029+0000 7fbcf862ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6a614d30 con 0x560f6a61acd0 2023-09-10T01:27:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.029+0000 7fbcf862ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6a614ef0 con 0x560f6a57e030 2023-09-10T01:27:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.029+0000 7fbcf862ab80 10 client.0 _ll_register_callbacks cb 0x560f6a443a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:27:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.029+0000 7fbcea200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a57e030 0x560f6a615e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.029+0000 7fbcea200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a57e030 0x560f6a615e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38558/0 (socket says 172.21.15.44:38558) 2023-09-10T01:27:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.030+0000 7fbcea200700 1 -- 172.21.15.44:0/2128058026 learned_addr learned my addr 172.21.15.44:0/2128058026 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:27:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.030+0000 7fbce99ff700 1 --2- 172.21.15.44:0/2128058026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a61a310 0x560f6a614170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.030+0000 7fbceaa01700 1 --2- 172.21.15.44:0/2128058026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a6146b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.030+0000 7fbceaa01700 1 -- 172.21.15.44:0/2128058026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a57e030 msgr2=0x560f6a615e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.030+0000 7fbceaa01700 1 --2- 172.21.15.44:0/2128058026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a57e030 0x560f6a615e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.030+0000 7fbceaa01700 1 -- 172.21.15.44:0/2128058026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a61a310 msgr2=0x560f6a614170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:27:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.030+0000 7fbceaa01700 1 --2- 172.21.15.44:0/2128058026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a61a310 0x560f6a614170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.030+0000 7fbceaa01700 1 -- 172.21.15.44:0/2128058026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce41153d0 con 0x560f6a61acd0 2023-09-10T01:27:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.033+0000 7fbceaa01700 1 --2- 172.21.15.44:0/2128058026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a6146b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009fd0 tx=0x7fbce0015480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53988c62b9f099f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:27:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.034+0000 7fbccdffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:27:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.034+0000 7fbccdffb700 1 -- 172.21.15.44:0/2128058026 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0015a60 con 0x560f6a61acd0 2023-09-10T01:27:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.034+0000 7fbccdffb700 1 -- 172.21.15.44:0/2128058026 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 9480+0+0 (secure 0 0 0) 0x7fbce0015bc0 con 0x560f6a61acd0 2023-09-10T01:27:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.034+0000 7fbcf862ab80 1 -- 172.21.15.44:0/2128058026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcd40f4a60 con 0x560f6a61acd0 2023-09-10T01:27:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.035+0000 7fbccdffb700 1 -- 172.21.15.44:0/2128058026 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbce0015d80 con 0x560f6a61acd0 2023-09-10T01:27:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.035+0000 7fbcf862ab80 10 client.29936 fetch_fsmap learned FSMap version 161 2023-09-10T01:27:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.035+0000 7fbcf862ab80 1 -- 172.21.15.44:0/2128058026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=161,monmap=4+}) v3 -- 0x560f6a386250 con 0x560f6a61acd0 2023-09-10T01:27:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.035+0000 7fbccdffb700 1 -- 172.21.15.44:0/2128058026 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcd40f4a60 con 0x560f6a61acd0 2023-09-10T01:27:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.035+0000 7fbccdffb700 1 -- 172.21.15.44:0/2128058026 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 161) v1 ==== 7359+0+0 (secure 0 0 0) 0x7fbce001e070 con 0x560f6a61acd0 2023-09-10T01:27:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.035+0000 7fbcf862ab80 10 client.29936 fetch_fsmap finished waiting for FSMap version 161 2023-09-10T01:27:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.035+0000 7fbcf862ab80 10 client.29936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:27:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.035+0000 7fbcf862ab80 20 client.29936 populate_metadata read hostname 'smithi044' 2023-09-10T01:27:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.035+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/2128058026 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560f6a6201b0 0x560f6a67a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:27:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.035+0000 7fbcf862ab80 4 client.29936 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:27:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.035+0000 7fbcf862ab80 1 -- 172.21.15.44:0/2128058026 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560f6a5548c0 con 0x560f6a6201b0 2023-09-10T01:27:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.035+0000 7fbcea200700 1 --2- 172.21.15.44:0/2128058026 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560f6a6201b0 0x560f6a67a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:27:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.039+0000 7fbccdffb700 10 client.29936 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.039+0000 7fbcea200700 1 --2- 172.21.15.44:0/2128058026 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560f6a6201b0 0x560f6a67a4a0 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6870b8e15f32890f server_cookie=2400dcb55931138c in_seq=0 out_seq=0 2023-09-10T01:27:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.072+0000 7fbccdffb700 1 -- 172.21.15.44:0/2128058026 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560f6a5548c0 con 0x560f6a6201b0 2023-09-10T01:27:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.072+0000 7fbccdffb700 10 client.29936 handle_command_reply: tid=0 2023-09-10T01:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.072+0000 7fbcf862ab80 1 client.29936 shutdown 2023-09-10T01:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.072+0000 7fbcf862ab80 1 -- 172.21.15.44:0/2128058026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 msgr2=0x560f6a6146b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.072+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/2128058026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a6146b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009fd0 tx=0x7fbce0015480 comp rx=0 tx=0).stop 2023-09-10T01:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.072+0000 7fbcf862ab80 1 -- 172.21.15.44:0/2128058026 shutdown_connections 2023-09-10T01:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.072+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/2128058026 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560f6a6201b0 0x560f6a67a4a0 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.072+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/2128058026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f6a61acd0 0x560f6a6146b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.072+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/2128058026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f6a57e030 0x560f6a615e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.072+0000 7fbcf862ab80 1 --2- 172.21.15.44:0/2128058026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f6a61a310 0x560f6a614170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.072+0000 7fbcf862ab80 1 -- 172.21.15.44:0/2128058026 >> 172.21.15.44:0/2128058026 conn(0x560f6a61e880 msgr2=0x560f6a60bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.072+0000 7fbccdffb700 0 client.29936 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.072+0000 7fbcf862ab80 1 -- 172.21.15.44:0/2128058026 shutdown_connections 2023-09-10T01:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.072+0000 7fbcf862ab80 1 -- 172.21.15.44:0/2128058026 wait complete. 2023-09-10T01:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.073+0000 7fbcf862ab80 20 client.29936 trim_cache size 0 max 16384 2023-09-10T01:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:27:59.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:27:59.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:27:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.073+0000 7fbcec464700 1 -- 172.21.15.44:0/387001509 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbcd00a08b0 msgr2=0x7fbcd00a2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.073+0000 7fbcec464700 1 --2- 172.21.15.44:0/387001509 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbcd00a08b0 0x7fbcd00a2d70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009300 tx=0x7fbce0009290 comp rx=0 tx=0).stop 2023-09-10T01:27:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.073+0000 7fbcec464700 1 -- 172.21.15.44:0/387001509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbce4112550 msgr2=0x7fbce41abcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:27:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.073+0000 7fbcec464700 1 --2- 172.21.15.44:0/387001509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbce4112550 0x7fbce41abcd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400b560 tx=0x7fbcd400b870 comp rx=0 tx=0).stop 2023-09-10T01:27:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.073+0000 7fbcec464700 1 -- 172.21.15.44:0/387001509 shutdown_connections 2023-09-10T01:27:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.073+0000 7fbcec464700 1 --2- 172.21.15.44:0/387001509 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbcd00a08b0 0x7fbcd00a2d70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.073+0000 7fbcec464700 1 --2- 172.21.15.44:0/387001509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbce41a5a50 0x7fbce41a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.073+0000 7fbcec464700 1 --2- 172.21.15.44:0/387001509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbce41098c0 0x7fbce41ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.073+0000 7fbcec464700 1 --2- 172.21.15.44:0/387001509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbce4112550 0x7fbce41abcd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:27:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.073+0000 7fbcec464700 1 -- 172.21.15.44:0/387001509 >> 172.21.15.44:0/387001509 conn(0x7fbce406c6c0 msgr2=0x7fbce4070fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:27:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.073+0000 7fbcec464700 1 -- 172.21.15.44:0/387001509 shutdown_connections 2023-09-10T01:27:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:27:59.073+0000 7fbcec464700 1 -- 172.21.15.44:0/387001509 wait complete. 2023-09-10T01:27:59.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:27:59 smithi131 ceph-mon[103713]: pgmap v1536: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.1 MiB/s wr, 747 op/s 2023-09-10T01:27:59.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:27:59 smithi044 ceph-mon[98676]: pgmap v1536: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.1 MiB/s wr, 747 op/s 2023-09-10T01:27:59.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:27:59 smithi099 ceph-mon[102692]: pgmap v1536: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.1 MiB/s wr, 747 op/s 2023-09-10T01:28:01.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 415.0 s, 281.5 tps, lat 357.531 ms stddev 190.504 2023-09-10T01:28:01.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:01 smithi131 ceph-mon[103713]: pgmap v1537: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.7 MiB/s wr, 712 op/s 2023-09-10T01:28:01.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:01 smithi044 ceph-mon[98676]: pgmap v1537: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.7 MiB/s wr, 712 op/s 2023-09-10T01:28:01.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:01 smithi099 ceph-mon[102692]: pgmap v1537: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.7 MiB/s wr, 712 op/s 2023-09-10T01:28:03.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:03 smithi099 ceph-mon[102692]: pgmap v1538: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.09k op/s 2023-09-10T01:28:04.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:03 smithi131 ceph-mon[103713]: pgmap v1538: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.09k op/s 2023-09-10T01:28:04.069 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:03 smithi044 ceph-mon[98676]: pgmap v1538: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.09k op/s 2023-09-10T01:28:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:04 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:28:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:04 smithi131 ceph-mon[103713]: pgmap v1539: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.2 MiB/s wr, 765 op/s 2023-09-10T01:28:05.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:04 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:28:05.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:04 smithi044 ceph-mon[98676]: pgmap v1539: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.2 MiB/s wr, 765 op/s 2023-09-10T01:28:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:04 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:28:05.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:04 smithi099 ceph-mon[102692]: pgmap v1539: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.2 MiB/s wr, 765 op/s 2023-09-10T01:28:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 420.0 s, 303.8 tps, lat 323.768 ms stddev 186.808 2023-09-10T01:28:07.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:07 smithi131 ceph-mon[103713]: pgmap v1540: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.3 MiB/s wr, 996 op/s 2023-09-10T01:28:07.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:07 smithi044 ceph-mon[98676]: pgmap v1540: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.3 MiB/s wr, 996 op/s 2023-09-10T01:28:07.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:07 smithi099 ceph-mon[102692]: pgmap v1540: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.3 MiB/s wr, 996 op/s 2023-09-10T01:28:09.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:09 smithi131 ceph-mon[103713]: pgmap v1541: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 716 op/s 2023-09-10T01:28:09.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:09 smithi044 ceph-mon[98676]: pgmap v1541: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 716 op/s 2023-09-10T01:28:09.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:09 smithi099 ceph-mon[102692]: pgmap v1541: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 716 op/s 2023-09-10T01:28:10.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:10 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:28:10.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:10 smithi131 ceph-mon[103713]: osdmap e189: 12 total, 12 up, 12 in 2023-09-10T01:28:10.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:10 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:28:10.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:10 smithi044 ceph-mon[98676]: osdmap e189: 12 total, 12 up, 12 in 2023-09-10T01:28:10.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:10 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:28:10.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:10 smithi099 ceph-mon[102692]: osdmap e189: 12 total, 12 up, 12 in 2023-09-10T01:28:11.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 425.0 s, 247.2 tps, lat 367.877 ms stddev 258.312 2023-09-10T01:28:11.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:11 smithi131 ceph-mon[103713]: pgmap v1543: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.0 MiB/s wr, 813 op/s 2023-09-10T01:28:11.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:11 smithi044 ceph-mon[98676]: pgmap v1543: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.0 MiB/s wr, 813 op/s 2023-09-10T01:28:11.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:11 smithi099 ceph-mon[102692]: pgmap v1543: 129 pgs: 129 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.0 MiB/s wr, 813 op/s 2023-09-10T01:28:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.329+0000 7fed7442b700 1 -- 172.21.15.44:0/3569298802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed6c10ed50 msgr2=0x7fed6c107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.329+0000 7fed7442b700 1 --2- 172.21.15.44:0/3569298802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed6c10ed50 0x7fed6c107600 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fed68009990 tx=0x7fed68009ca0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.335+0000 7fed7442b700 1 -- 172.21.15.44:0/3569298802 shutdown_connections 2023-09-10T01:28:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.335+0000 7fed7442b700 1 --2- 172.21.15.44:0/3569298802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed6c119de0 0x7fed6c11c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.335+0000 7fed7442b700 1 --2- 172.21.15.44:0/3569298802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed6c10ed50 0x7fed6c107600 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.335+0000 7fed7442b700 1 --2- 172.21.15.44:0/3569298802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c10f1c0 0x7fed6c107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.335+0000 7fed7442b700 1 -- 172.21.15.44:0/3569298802 >> 172.21.15.44:0/3569298802 conn(0x7fed6c06c6c0 msgr2=0x7fed6c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.337+0000 7fed7442b700 1 -- 172.21.15.44:0/3569298802 shutdown_connections 2023-09-10T01:28:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.337+0000 7fed7442b700 1 -- 172.21.15.44:0/3569298802 wait complete. 2023-09-10T01:28:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.338+0000 7fed7442b700 1 Processor -- start 2023-09-10T01:28:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.338+0000 7fed7442b700 1 -- start start 2023-09-10T01:28:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.338+0000 7fed7442b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed6c10ed50 0x7fed6c117270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.338+0000 7fed7442b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c10f1c0 0x7fed6c1177f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.338+0000 7fed7442b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed6c119de0 0x7fed6c110200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.338+0000 7fed7442b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c1109d0 con 0x7fed6c119de0 2023-09-10T01:28:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.338+0000 7fed7442b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c110b40 con 0x7fed6c10ed50 2023-09-10T01:28:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.338+0000 7fed7442b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c110d00 con 0x7fed6c10f1c0 2023-09-10T01:28:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.339+0000 7fed721c7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed6c10ed50 0x7fed6c117270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.339+0000 7fed721c7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed6c10ed50 0x7fed6c117270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33428/0 (socket says 172.21.15.44:33428) 2023-09-10T01:28:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.339+0000 7fed721c7700 1 -- 172.21.15.44:0/586552572 learned_addr learned my addr 172.21.15.44:0/586552572 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.339+0000 7fed719c6700 1 --2- 172.21.15.44:0/586552572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c10f1c0 0x7fed6c1177f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.339+0000 7fed729c8700 1 --2- 172.21.15.44:0/586552572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed6c119de0 0x7fed6c110200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.339+0000 7fed721c7700 1 -- 172.21.15.44:0/586552572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c10f1c0 msgr2=0x7fed6c1177f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.339+0000 7fed721c7700 1 --2- 172.21.15.44:0/586552572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c10f1c0 0x7fed6c1177f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.339+0000 7fed721c7700 1 -- 172.21.15.44:0/586552572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed6c119de0 msgr2=0x7fed6c110200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.339+0000 7fed721c7700 1 --2- 172.21.15.44:0/586552572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed6c119de0 0x7fed6c110200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.339+0000 7fed721c7700 1 -- 172.21.15.44:0/586552572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed680095f0 con 0x7fed6c10ed50 2023-09-10T01:28:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.339+0000 7fed729c8700 1 --2- 172.21.15.44:0/586552572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed6c119de0 0x7fed6c110200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:28:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.339+0000 7fed721c7700 1 --2- 172.21.15.44:0/586552572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed6c10ed50 0x7fed6c117270 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fed68005740 tx=0x7fed68005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.344+0000 7fed637fe700 1 -- 172.21.15.44:0/586552572 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6801d070 con 0x7fed6c10ed50 2023-09-10T01:28:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.344+0000 7fed637fe700 1 -- 172.21.15.44:0/586552572 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed68004080 con 0x7fed6c10ed50 2023-09-10T01:28:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.344+0000 7fed637fe700 1 -- 172.21.15.44:0/586552572 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68022ca0 con 0x7fed6c10ed50 2023-09-10T01:28:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.349+0000 7fed7442b700 1 -- 172.21.15.44:0/586552572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6c114e00 con 0x7fed6c10ed50 2023-09-10T01:28:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.349+0000 7fed7442b700 1 -- 172.21.15.44:0/586552572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c115230 con 0x7fed6c10ed50 2023-09-10T01:28:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.359+0000 7fed805f1b80 1 Processor -- start 2023-09-10T01:28:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.359+0000 7fed805f1b80 1 -- start start 2023-09-10T01:28:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.359+0000 7fed805f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c61161a0 0x55a8c6116580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.359+0000 7fed805f1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110300 0x55a8c6110780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.359+0000 7fed805f1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c6110cc0 0x55a8c611cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.360+0000 7fed805f1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c60696d0 con 0x55a8c6110cc0 2023-09-10T01:28:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.360+0000 7fed805f1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c6066ff0 con 0x55a8c6110300 2023-09-10T01:28:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.360+0000 7fed805f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c60628e0 con 0x55a8c61161a0 2023-09-10T01:28:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.361+0000 7fed637fe700 1 -- 172.21.15.44:0/586552572 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fed68022470 con 0x7fed6c10ed50 2023-09-10T01:28:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.362+0000 7fed637fe700 1 --2- 172.21.15.44:0/586552572 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fed580a09c0 0x7fed580a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.362+0000 7fed637fe700 1 -- 172.21.15.44:0/586552572 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fed680cd5d0 con 0x7fed6c10ed50 2023-09-10T01:28:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.362+0000 7fed721c7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110300 0x55a8c6110780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.362+0000 7fed721c7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110300 0x55a8c6110780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33436/0 (socket says 172.21.15.44:33436) 2023-09-10T01:28:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.362+0000 7fed721c7700 1 -- 172.21.15.44:0/3812415977 learned_addr learned my addr 172.21.15.44:0/3812415977 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.363+0000 7fed719c6700 1 --2- 172.21.15.44:0/3812415977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c6110cc0 0x55a8c611cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.364+0000 7fed719c6700 1 -- 172.21.15.44:0/3812415977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c61161a0 msgr2=0x55a8c6116580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.364+0000 7fed719c6700 1 --2- 172.21.15.44:0/3812415977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c61161a0 0x55a8c6116580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.364+0000 7fed719c6700 1 -- 172.21.15.44:0/3812415977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110300 msgr2=0x55a8c6110780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.364+0000 7fed719c6700 1 --2- 172.21.15.44:0/3812415977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110300 0x55a8c6110780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.364+0000 7fed719c6700 1 -- 172.21.15.44:0/3812415977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8c5e8ed80 con 0x55a8c6110cc0 2023-09-10T01:28:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.364+0000 7fed719c6700 1 --2- 172.21.15.44:0/3812415977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c6110cc0 0x55a8c611cf30 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fed6c06de40 tx=0x7fed640096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a05a5a3c998217 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.365+0000 7fed60d89700 1 -- 172.21.15.44:0/3812415977 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64010040 con 0x55a8c6110cc0 2023-09-10T01:28:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.365+0000 7fed60d89700 1 -- 172.21.15.44:0/3812415977 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed64014ab0 con 0x55a8c6110cc0 2023-09-10T01:28:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.365+0000 7fed60d89700 1 -- 172.21.15.44:0/3812415977 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64013970 con 0x55a8c6110cc0 2023-09-10T01:28:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.366+0000 7fed729c8700 1 --2- 172.21.15.44:0/586552572 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fed580a09c0 0x7fed580a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.366+0000 7fed805f1b80 1 -- 172.21.15.44:0/3812415977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c6110cc0 msgr2=0x55a8c611cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.366+0000 7fed805f1b80 1 --2- 172.21.15.44:0/3812415977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c6110cc0 0x55a8c611cf30 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fed6c06de40 tx=0x7fed640096d0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.366+0000 7fed729c8700 1 --2- 172.21.15.44:0/586552572 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fed580a09c0 0x7fed580a2e80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fed6c069970 tx=0x7fed5c0090c0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.368+0000 7fed805f1b80 1 -- 172.21.15.44:0/3812415977 shutdown_connections 2023-09-10T01:28:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.368+0000 7fed805f1b80 1 --2- 172.21.15.44:0/3812415977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c6110cc0 0x55a8c611cf30 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.368+0000 7fed805f1b80 1 --2- 172.21.15.44:0/3812415977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c61161a0 0x55a8c6116580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.368+0000 7fed805f1b80 1 --2- 172.21.15.44:0/3812415977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110300 0x55a8c6110780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.368+0000 7fed805f1b80 1 -- 172.21.15.44:0/3812415977 >> 172.21.15.44:0/3812415977 conn(0x55a8c6085030 msgr2=0x55a8c610b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.372+0000 7fed805f1b80 1 -- 172.21.15.44:0/3812415977 shutdown_connections 2023-09-10T01:28:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.372+0000 7fed805f1b80 1 -- 172.21.15.44:0/3812415977 wait complete. 2023-09-10T01:28:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.372+0000 7fed805f1b80 1 -- start start 2023-09-10T01:28:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.372+0000 7fed805f1b80 1 Processor -- start 2023-09-10T01:28:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.373+0000 7fed805f1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110300 0x55a8c611cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.373+0000 7fed805f1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c6110cc0 0x55a8c611a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.373+0000 7fed805f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c61161a0 0x55a8c611a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.373+0000 7fed805f1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c6000b10 con 0x55a8c6110cc0 2023-09-10T01:28:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.373+0000 7fed805f1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c6000810 con 0x55a8c6110300 2023-09-10T01:28:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.373+0000 7fed805f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5ee48c0 con 0x55a8c61161a0 2023-09-10T01:28:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.373+0000 7fed805f1b80 10 client.0 _ll_register_callbacks cb 0x55a8c5f4aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:28:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.373+0000 7fed729c8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c61161a0 0x55a8c611a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.373+0000 7fed729c8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c61161a0 0x55a8c611a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38588/0 (socket says 172.21.15.44:38588) 2023-09-10T01:28:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.373+0000 7fed729c8700 1 -- 172.21.15.44:0/710642810 learned_addr learned my addr 172.21.15.44:0/710642810 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.374+0000 7fed729c8700 1 -- 172.21.15.44:0/710642810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110300 msgr2=0x55a8c611cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.374+0000 7fed729c8700 1 --2- 172.21.15.44:0/710642810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110300 0x55a8c611cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.374+0000 7fed729c8700 1 -- 172.21.15.44:0/710642810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c6110cc0 msgr2=0x55a8c611a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.374+0000 7fed729c8700 1 --2- 172.21.15.44:0/710642810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c6110cc0 0x55a8c611a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.374+0000 7fed729c8700 1 -- 172.21.15.44:0/710642810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed64009c40 con 0x55a8c61161a0 2023-09-10T01:28:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.374+0000 7fed729c8700 1 --2- 172.21.15.44:0/710642810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c61161a0 0x55a8c611a6b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00dbd0 tx=0x7fed5c006de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c0d1b5070c5cdd9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.374+0000 7fed55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:28:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.375+0000 7fed55ffb700 1 -- 172.21.15.44:0/710642810 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c014b30 con 0x55a8c61161a0 2023-09-10T01:28:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.375+0000 7fed55ffb700 1 -- 172.21.15.44:0/710642810 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(189..189 src has 1..189) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fed5c014c90 con 0x55a8c61161a0 2023-09-10T01:28:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.375+0000 7fed805f1b80 1 -- 172.21.15.44:0/710642810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8c5fba490 con 0x55a8c61161a0 2023-09-10T01:28:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.375+0000 7fed55ffb700 1 -- 172.21.15.44:0/710642810 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=163) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed5c014e50 con 0x55a8c61161a0 2023-09-10T01:28:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.378+0000 7fed805f1b80 10 client.29941 fetch_fsmap learned FSMap version 163 2023-09-10T01:28:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.378+0000 7fed805f1b80 1 -- 172.21.15.44:0/710642810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=163,monmap=4+}) v3 -- 0x55a8c5e77c00 con 0x55a8c61161a0 2023-09-10T01:28:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.384+0000 7fed55ffb700 1 -- 172.21.15.44:0/710642810 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a8c5fba490 con 0x55a8c61161a0 2023-09-10T01:28:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.385+0000 7fed55ffb700 1 -- 172.21.15.44:0/710642810 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 163) v1 ==== 7367+0+0 (secure 0 0 0) 0x7fed5c027070 con 0x55a8c61161a0 2023-09-10T01:28:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.385+0000 7fed805f1b80 10 client.29941 fetch_fsmap finished waiting for FSMap version 163 2023-09-10T01:28:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.385+0000 7fed805f1b80 10 client.29941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:28:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.385+0000 7fed805f1b80 20 client.29941 populate_metadata read hostname 'smithi044' 2023-09-10T01:28:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.385+0000 7fed805f1b80 1 --2- 172.21.15.44:0/710642810 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a8c612a450 0x55a8c612c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.385+0000 7fed805f1b80 4 client.29941 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:28:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.385+0000 7fed805f1b80 1 -- 172.21.15.44:0/710642810 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8c6077840 con 0x55a8c612a450 2023-09-10T01:28:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.385+0000 7fed721c7700 1 --2- 172.21.15.44:0/710642810 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a8c612a450 0x55a8c612c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.387+0000 7fed721c7700 1 --2- 172.21.15.44:0/710642810 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a8c612a450 0x55a8c612c8b0 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=917cb7dae0249715 server_cookie=103a9f9eb0a0919d in_seq=0 out_seq=0 2023-09-10T01:28:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.391+0000 7fed55ffb700 10 client.29941 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:28:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.401+0000 7fed55ffb700 1 -- 172.21.15.44:0/710642810 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a8c6077840 con 0x55a8c612a450 2023-09-10T01:28:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.401+0000 7fed55ffb700 10 client.29941 handle_command_reply: tid=0 2023-09-10T01:28:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.401+0000 7fed805f1b80 1 client.29941 shutdown 2023-09-10T01:28:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.417+0000 7fed805f1b80 1 -- 172.21.15.44:0/710642810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c61161a0 msgr2=0x55a8c611a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.417+0000 7fed805f1b80 1 --2- 172.21.15.44:0/710642810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c61161a0 0x55a8c611a6b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00dbd0 tx=0x7fed5c006de0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.424+0000 7fed805f1b80 1 -- 172.21.15.44:0/710642810 shutdown_connections 2023-09-10T01:28:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.424+0000 7fed805f1b80 1 --2- 172.21.15.44:0/710642810 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a8c612a450 0x55a8c612c8b0 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.424+0000 7fed805f1b80 1 --2- 172.21.15.44:0/710642810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c61161a0 0x55a8c611a6b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.424+0000 7fed805f1b80 1 --2- 172.21.15.44:0/710642810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110300 0x55a8c611cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.424+0000 7fed805f1b80 1 --2- 172.21.15.44:0/710642810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c6110cc0 0x55a8c611a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.424+0000 7fed805f1b80 1 -- 172.21.15.44:0/710642810 >> 172.21.15.44:0/710642810 conn(0x55a8c6085030 msgr2=0x55a8c610a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.425+0000 7fed55ffb700 0 client.29941 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:28:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.432+0000 7fed805f1b80 1 -- 172.21.15.44:0/710642810 shutdown_connections 2023-09-10T01:28:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.436+0000 7fed805f1b80 1 -- 172.21.15.44:0/710642810 wait complete. 2023-09-10T01:28:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.441+0000 7fed805f1b80 20 client.29941 trim_cache size 0 max 16384 2023-09-10T01:28:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.494+0000 7fed805f1b80 1 Processor -- start 2023-09-10T01:28:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.494+0000 7fed805f1b80 1 -- start start 2023-09-10T01:28:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.495+0000 7fed805f1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110300 0x55a8c6115140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.495+0000 7fed805f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c6110cc0 0x55a8c6115680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.495+0000 7fed805f1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c61161a0 0x55a8c6112b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.495+0000 7fed805f1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c6115e20 con 0x55a8c61161a0 2023-09-10T01:28:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.495+0000 7fed805f1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c6102e00 con 0x55a8c6110300 2023-09-10T01:28:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.495+0000 7fed805f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c61110a0 con 0x55a8c6110cc0 2023-09-10T01:28:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.502+0000 7fed729c8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c61161a0 0x55a8c6112b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.502+0000 7fed729c8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c61161a0 0x55a8c6112b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59942/0 (socket says 172.21.15.44:59942) 2023-09-10T01:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.502+0000 7fed729c8700 1 -- 172.21.15.44:0/3993418855 learned_addr learned my addr 172.21.15.44:0/3993418855 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.502+0000 7fed729c8700 1 -- 172.21.15.44:0/3993418855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c6110cc0 msgr2=0x55a8c6115680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.503+0000 7fed729c8700 1 --2- 172.21.15.44:0/3993418855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c6110cc0 0x55a8c6115680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.503+0000 7fed729c8700 1 -- 172.21.15.44:0/3993418855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110300 msgr2=0x55a8c6115140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.503+0000 7fed729c8700 1 --2- 172.21.15.44:0/3993418855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110300 0x55a8c6115140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.503+0000 7fed729c8700 1 -- 172.21.15.44:0/3993418855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8c6131520 con 0x55a8c61161a0 2023-09-10T01:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.503+0000 7fed729c8700 1 --2- 172.21.15.44:0/3993418855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c61161a0 0x55a8c6112b60 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fed5c028580 tx=0x7fed5c0147d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7902d91f7023f65 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.510+0000 7fed6158a700 1 -- 172.21.15.44:0/3993418855 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c028770 con 0x55a8c61161a0 2023-09-10T01:28:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.510+0000 7fed6158a700 1 -- 172.21.15.44:0/3993418855 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed5c0288d0 con 0x55a8c61161a0 2023-09-10T01:28:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.510+0000 7fed6158a700 1 -- 172.21.15.44:0/3993418855 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c004040 con 0x55a8c61161a0 2023-09-10T01:28:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.511+0000 7fed805f1b80 1 -- 172.21.15.44:0/3993418855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c61161a0 msgr2=0x55a8c6112b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.511+0000 7fed805f1b80 1 --2- 172.21.15.44:0/3993418855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c61161a0 0x55a8c6112b60 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fed5c028580 tx=0x7fed5c0147d0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.516+0000 7fed805f1b80 1 -- 172.21.15.44:0/3993418855 shutdown_connections 2023-09-10T01:28:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.516+0000 7fed805f1b80 1 --2- 172.21.15.44:0/3993418855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c61161a0 0x55a8c6112b60 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.516+0000 7fed805f1b80 1 --2- 172.21.15.44:0/3993418855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110300 0x55a8c6115140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.516+0000 7fed805f1b80 1 --2- 172.21.15.44:0/3993418855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c6110cc0 0x55a8c6115680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.516+0000 7fed805f1b80 1 -- 172.21.15.44:0/3993418855 >> 172.21.15.44:0/3993418855 conn(0x55a8c6085030 msgr2=0x55a8c612c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.516+0000 7fed805f1b80 1 -- 172.21.15.44:0/3993418855 shutdown_connections 2023-09-10T01:28:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.516+0000 7fed805f1b80 1 -- 172.21.15.44:0/3993418855 wait complete. 2023-09-10T01:28:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.516+0000 7fed805f1b80 1 -- start start 2023-09-10T01:28:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.517+0000 7fed805f1b80 1 Processor -- start 2023-09-10T01:28:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.517+0000 7fed805f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c6110300 0x55a8c611f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.517+0000 7fed805f1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110cc0 0x55a8c611fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.517+0000 7fed805f1b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c61161a0 0x55a8c617d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.517+0000 7fed805f1b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c617d840 con 0x55a8c61161a0 2023-09-10T01:28:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.517+0000 7fed805f1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c617d9b0 con 0x55a8c6110cc0 2023-09-10T01:28:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.517+0000 7fed805f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c617db70 con 0x55a8c6110300 2023-09-10T01:28:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.517+0000 7fed805f1b80 10 client.0 _ll_register_callbacks cb 0x55a8c5f4aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:28:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.517+0000 7fed721c7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c6110300 0x55a8c611f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.518+0000 7fed721c7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c6110300 0x55a8c611f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38596/0 (socket says 172.21.15.44:38596) 2023-09-10T01:28:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.518+0000 7fed721c7700 1 -- 172.21.15.44:0/553870064 learned_addr learned my addr 172.21.15.44:0/553870064 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.518+0000 7fed719c6700 1 --2- 172.21.15.44:0/553870064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110cc0 0x55a8c611fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.518+0000 7fed719c6700 1 -- 172.21.15.44:0/553870064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c6110300 msgr2=0x55a8c611f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.518+0000 7fed719c6700 1 --2- 172.21.15.44:0/553870064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c6110300 0x55a8c611f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.518+0000 7fed719c6700 1 -- 172.21.15.44:0/553870064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c61161a0 msgr2=0x55a8c617d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.518+0000 7fed719c6700 1 --2- 172.21.15.44:0/553870064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c61161a0 0x55a8c617d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.518+0000 7fed719c6700 1 -- 172.21.15.44:0/553870064 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8c5e77c00 con 0x55a8c6110cc0 2023-09-10T01:28:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.518+0000 7fed719c6700 1 --2- 172.21.15.44:0/553870064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110cc0 0x55a8c611fc40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x55a8c6108000 tx=0x7fed64014700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6a2e4b9a624144c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.527+0000 7fed55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:28:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.527+0000 7fed55ffb700 1 -- 172.21.15.44:0/553870064 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64010050 con 0x55a8c6110cc0 2023-09-10T01:28:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.527+0000 7fed55ffb700 1 -- 172.21.15.44:0/553870064 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(189..189 src has 1..189) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fed64004030 con 0x55a8c6110cc0 2023-09-10T01:28:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.531+0000 7fed805f1b80 1 -- 172.21.15.44:0/553870064 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed64014ab0 con 0x55a8c6110cc0 2023-09-10T01:28:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.538+0000 7fed55ffb700 1 -- 172.21.15.44:0/553870064 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=163) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed64009400 con 0x55a8c6110cc0 2023-09-10T01:28:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.540+0000 7fed805f1b80 10 client.29945 fetch_fsmap learned FSMap version 163 2023-09-10T01:28:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.540+0000 7fed805f1b80 1 -- 172.21.15.44:0/553870064 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=163,monmap=4+}) v3 -- 0x55a8c5eb51f0 con 0x55a8c6110cc0 2023-09-10T01:28:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.540+0000 7fed55ffb700 1 -- 172.21.15.44:0/553870064 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed64014ab0 con 0x55a8c6110cc0 2023-09-10T01:28:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.540+0000 7fed55ffb700 1 -- 172.21.15.44:0/553870064 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 163) v1 ==== 7367+0+0 (secure 0 0 0) 0x7fed64023020 con 0x55a8c6110cc0 2023-09-10T01:28:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.540+0000 7fed805f1b80 10 client.29945 fetch_fsmap finished waiting for FSMap version 163 2023-09-10T01:28:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.540+0000 7fed805f1b80 10 client.29945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:28:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.540+0000 7fed805f1b80 20 client.29945 populate_metadata read hostname 'smithi044' 2023-09-10T01:28:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.540+0000 7fed805f1b80 1 --2- 172.21.15.44:0/553870064 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a8c617f180 0x55a8c61815a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.540+0000 7fed805f1b80 4 client.29945 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:28:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.540+0000 7fed805f1b80 1 -- 172.21.15.44:0/553870064 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a8c605b8c0 con 0x55a8c617f180 2023-09-10T01:28:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.540+0000 7fed721c7700 1 --2- 172.21.15.44:0/553870064 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a8c617f180 0x55a8c61815a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.541+0000 7fed55ffb700 10 client.29945 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:28:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.541+0000 7fed721c7700 1 --2- 172.21.15.44:0/553870064 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a8c617f180 0x55a8c61815a0 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a315318bfc7daaf1 server_cookie=ebdc6331747ac4e1 in_seq=0 out_seq=0 2023-09-10T01:28:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.632+0000 7fed55ffb700 1 -- 172.21.15.44:0/553870064 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a8c605b8c0 con 0x55a8c617f180 2023-09-10T01:28:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.633+0000 7fed55ffb700 10 client.29945 handle_command_reply: tid=0 2023-09-10T01:28:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.633+0000 7fed805f1b80 1 client.29945 shutdown 2023-09-10T01:28:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.639+0000 7fed805f1b80 1 -- 172.21.15.44:0/553870064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110cc0 msgr2=0x55a8c611fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.639+0000 7fed805f1b80 1 --2- 172.21.15.44:0/553870064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110cc0 0x55a8c611fc40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x55a8c6108000 tx=0x7fed64014700 comp rx=0 tx=0).stop 2023-09-10T01:28:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.648+0000 7fed805f1b80 1 -- 172.21.15.44:0/553870064 shutdown_connections 2023-09-10T01:28:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.648+0000 7fed805f1b80 1 --2- 172.21.15.44:0/553870064 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a8c617f180 0x55a8c61815a0 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.648+0000 7fed805f1b80 1 --2- 172.21.15.44:0/553870064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8c61161a0 0x55a8c617d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.648+0000 7fed805f1b80 1 --2- 172.21.15.44:0/553870064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a8c6110300 0x55a8c611f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.648+0000 7fed805f1b80 1 --2- 172.21.15.44:0/553870064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8c6110cc0 0x55a8c611fc40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.648+0000 7fed805f1b80 1 -- 172.21.15.44:0/553870064 >> 172.21.15.44:0/553870064 conn(0x55a8c6085030 msgr2=0x55a8c6115140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.648+0000 7fed55ffb700 0 client.29945 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:28:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.648+0000 7fed805f1b80 1 -- 172.21.15.44:0/553870064 shutdown_connections 2023-09-10T01:28:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.648+0000 7fed805f1b80 1 -- 172.21.15.44:0/553870064 wait complete. 2023-09-10T01:28:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.656+0000 7fed805f1b80 20 client.29945 trim_cache size 0 max 16384 2023-09-10T01:28:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:28:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:28:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:28:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:28:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.675+0000 7fed7442b700 1 -- 172.21.15.44:0/586552572 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fed580a09c0 msgr2=0x7fed580a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.675+0000 7fed7442b700 1 --2- 172.21.15.44:0/586552572 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fed580a09c0 0x7fed580a2e80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fed6c069970 tx=0x7fed5c0090c0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.675+0000 7fed7442b700 1 -- 172.21.15.44:0/586552572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed6c10ed50 msgr2=0x7fed6c117270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.675+0000 7fed7442b700 1 --2- 172.21.15.44:0/586552572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed6c10ed50 0x7fed6c117270 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fed68005740 tx=0x7fed68005770 comp rx=0 tx=0).stop 2023-09-10T01:28:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.683+0000 7fed7442b700 1 -- 172.21.15.44:0/586552572 shutdown_connections 2023-09-10T01:28:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.683+0000 7fed7442b700 1 --2- 172.21.15.44:0/586552572 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fed580a09c0 0x7fed580a2e80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.683+0000 7fed7442b700 1 --2- 172.21.15.44:0/586552572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed6c119de0 0x7fed6c110200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.683+0000 7fed7442b700 1 --2- 172.21.15.44:0/586552572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed6c10ed50 0x7fed6c117270 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.683+0000 7fed7442b700 1 --2- 172.21.15.44:0/586552572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c10f1c0 0x7fed6c1177f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.683+0000 7fed7442b700 1 -- 172.21.15.44:0/586552572 >> 172.21.15.44:0/586552572 conn(0x7fed6c06c6c0 msgr2=0x7fed6c10c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.687+0000 7fed7442b700 1 -- 172.21.15.44:0/586552572 shutdown_connections 2023-09-10T01:28:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:12.687+0000 7fed7442b700 1 -- 172.21.15.44:0/586552572 wait complete. 2023-09-10T01:28:13.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:13 smithi131 ceph-mon[103713]: pgmap v1544: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 714 op/s 2023-09-10T01:28:13.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:13 smithi044 ceph-mon[98676]: pgmap v1544: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 714 op/s 2023-09-10T01:28:13.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:13 smithi099 ceph-mon[102692]: pgmap v1544: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 714 op/s 2023-09-10T01:28:15.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:15 smithi131 ceph-mon[103713]: pgmap v1545: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 714 op/s 2023-09-10T01:28:15.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:15 smithi044 ceph-mon[98676]: pgmap v1545: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 714 op/s 2023-09-10T01:28:15.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:15 smithi099 ceph-mon[102692]: pgmap v1545: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 714 op/s 2023-09-10T01:28:16.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 430.0 s, 104.4 tps, lat 808.601 ms stddev 1228.090 2023-09-10T01:28:16.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:16 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:28:16.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:16 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:28:16.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:16 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:28:17.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:17 smithi131 ceph-mon[103713]: pgmap v1546: 129 pgs: 8 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 98 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.3 MiB/s wr, 574 op/s 2023-09-10T01:28:17.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:17 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:28:17.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:17 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:28:17.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:17 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:28:17.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:17 smithi044 ceph-mon[98676]: pgmap v1546: 129 pgs: 8 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 98 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.3 MiB/s wr, 574 op/s 2023-09-10T01:28:17.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:17 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:28:17.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:17 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:28:17.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:17 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:28:17.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:17 smithi099 ceph-mon[102692]: pgmap v1546: 129 pgs: 8 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 98 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.3 MiB/s wr, 574 op/s 2023-09-10T01:28:17.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:17 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:28:17.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:17 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:28:17.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:17 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:28:18.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:18 smithi131 ceph-mon[103713]: pgmap v1547: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 580 op/s 2023-09-10T01:28:18.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:18 smithi131 ceph-mon[103713]: pgmap v1548: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.1 MiB/s wr, 703 op/s 2023-09-10T01:28:18.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:18 smithi044 ceph-mon[98676]: pgmap v1547: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 580 op/s 2023-09-10T01:28:18.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:18 smithi044 ceph-mon[98676]: pgmap v1548: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.1 MiB/s wr, 703 op/s 2023-09-10T01:28:18.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:18 smithi099 ceph-mon[102692]: pgmap v1547: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 580 op/s 2023-09-10T01:28:18.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:18 smithi099 ceph-mon[102692]: pgmap v1548: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.1 MiB/s wr, 703 op/s 2023-09-10T01:28:20.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:20 smithi044 ceph-mon[98676]: pgmap v1549: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 580 op/s 2023-09-10T01:28:20.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:20 smithi099 ceph-mon[102692]: pgmap v1549: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 580 op/s 2023-09-10T01:28:21.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:20 smithi131 ceph-mon[103713]: pgmap v1549: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 580 op/s 2023-09-10T01:28:21.184 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 435.0 s, 308.1 tps, lat 382.520 ms stddev 675.954 2023-09-10T01:28:22.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:22 smithi099 ceph-mon[102692]: pgmap v1550: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.9 MiB/s wr, 598 op/s 2023-09-10T01:28:23.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:22 smithi131 ceph-mon[103713]: pgmap v1550: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.9 MiB/s wr, 598 op/s 2023-09-10T01:28:23.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:22 smithi044 ceph-mon[98676]: pgmap v1550: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.9 MiB/s wr, 598 op/s 2023-09-10T01:28:23.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:23 smithi099 ceph-mon[102692]: pgmap v1551: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.9 MiB/s wr, 598 op/s 2023-09-10T01:28:24.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:23 smithi131 ceph-mon[103713]: pgmap v1551: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.9 MiB/s wr, 598 op/s 2023-09-10T01:28:24.063 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:23 smithi044 ceph-mon[98676]: pgmap v1551: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.9 MiB/s wr, 598 op/s 2023-09-10T01:28:25.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:25 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:28:25.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:25 smithi099 ceph-mon[102692]: pgmap v1552: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 452 op/s 2023-09-10T01:28:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.923+0000 7f7c9358d700 1 -- 172.21.15.44:0/1359578995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c8c0ff760 msgr2=0x7f7c8c0ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.923+0000 7f7c9358d700 1 --2- 172.21.15.44:0/1359578995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c8c0ff760 0x7f7c8c0ffbe0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009990 tx=0x7f7c88009ca0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.923+0000 7f7c9358d700 1 -- 172.21.15.44:0/1359578995 shutdown_connections 2023-09-10T01:28:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.923+0000 7f7c9358d700 1 --2- 172.21.15.44:0/1359578995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c8c100120 0x7f7c8c10c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.923+0000 7f7c9358d700 1 --2- 172.21.15.44:0/1359578995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c8c1055b0 0x7f7c8c105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.923+0000 7f7c9358d700 1 --2- 172.21.15.44:0/1359578995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c8c0ff760 0x7f7c8c0ffbe0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.923+0000 7f7c9358d700 1 -- 172.21.15.44:0/1359578995 >> 172.21.15.44:0/1359578995 conn(0x7f7c8c074f30 msgr2=0x7f7c8c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.924+0000 7f7c9358d700 1 -- 172.21.15.44:0/1359578995 shutdown_connections 2023-09-10T01:28:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.924+0000 7f7c9358d700 1 -- 172.21.15.44:0/1359578995 wait complete. 2023-09-10T01:28:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.924+0000 7f7c9358d700 1 Processor -- start 2023-09-10T01:28:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.924+0000 7f7c9358d700 1 -- start start 2023-09-10T01:28:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.925+0000 7f7c9358d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c8c0ff760 0x7f7c8c195480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.925+0000 7f7c9358d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c8c100120 0x7f7c8c1959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.925+0000 7f7c9358d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c8c1055b0 0x7f7c8c198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.925+0000 7f7c9358d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c199580 con 0x7f7c8c1055b0 2023-09-10T01:28:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.925+0000 7f7c9358d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c1996f0 con 0x7f7c8c0ff760 2023-09-10T01:28:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.925+0000 7f7c9358d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c1998b0 con 0x7f7c8c100120 2023-09-10T01:28:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.925+0000 7f7c90b28700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c8c100120 0x7f7c8c1959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.925+0000 7f7c90b28700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c8c100120 0x7f7c8c1959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38610/0 (socket says 172.21.15.44:38610) 2023-09-10T01:28:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.925+0000 7f7c90b28700 1 -- 172.21.15.44:0/3889026640 learned_addr learned my addr 172.21.15.44:0/3889026640 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.925+0000 7f7c91b2a700 1 --2- 172.21.15.44:0/3889026640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c8c1055b0 0x7f7c8c198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.926+0000 7f7c90b28700 1 -- 172.21.15.44:0/3889026640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c8c0ff760 msgr2=0x7f7c8c195480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.926+0000 7f7c90b28700 1 --2- 172.21.15.44:0/3889026640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c8c0ff760 0x7f7c8c195480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.926+0000 7f7c90b28700 1 -- 172.21.15.44:0/3889026640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c8c1055b0 msgr2=0x7f7c8c198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.926+0000 7f7c90b28700 1 --2- 172.21.15.44:0/3889026640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c8c1055b0 0x7f7c8c198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.926+0000 7f7c90b28700 1 -- 172.21.15.44:0/3889026640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c880095f0 con 0x7f7c8c100120 2023-09-10T01:28:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.926+0000 7f7c91b2a700 1 --2- 172.21.15.44:0/3889026640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c8c1055b0 0x7f7c8c198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:28:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.926+0000 7f7c90b28700 1 --2- 172.21.15.44:0/3889026640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c8c100120 0x7f7c8c1959c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7c88006010 tx=0x7f7c88004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.926+0000 7f7c7e7fc700 1 -- 172.21.15.44:0/3889026640 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8801f070 con 0x7f7c8c100120 2023-09-10T01:28:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.926+0000 7f7c7e7fc700 1 -- 172.21.15.44:0/3889026640 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c88005030 con 0x7f7c8c100120 2023-09-10T01:28:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.926+0000 7f7c7e7fc700 1 -- 172.21.15.44:0/3889026640 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c88004ad0 con 0x7f7c8c100120 2023-09-10T01:28:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.926+0000 7f7c9358d700 1 -- 172.21.15.44:0/3889026640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8c199b30 con 0x7f7c8c100120 2023-09-10T01:28:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.926+0000 7f7c9358d700 1 -- 172.21.15.44:0/3889026640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8c0679f0 con 0x7f7c8c100120 2023-09-10T01:28:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.932+0000 7f7c7e7fc700 1 -- 172.21.15.44:0/3889026640 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f7c88023070 con 0x7f7c8c100120 2023-09-10T01:28:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.932+0000 7f7c9f753b80 1 Processor -- start 2023-09-10T01:28:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.932+0000 7f7c9f753b80 1 -- start start 2023-09-10T01:28:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.932+0000 7f7c9f753b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b1d270 0x555f07b19300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.932+0000 7f7c9f753b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07b19840 0x555f07b19cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.932+0000 7f7c9f753b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b2bf60 0x555f07b2e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.932+0000 7f7c7e7fc700 1 --2- 172.21.15.44:0/3889026640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7c780a0830 0x7f7c780a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.932+0000 7f7c9f753b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f07a726d0 con 0x555f07b2bf60 2023-09-10T01:28:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.932+0000 7f7c9f753b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f07a6fff0 con 0x555f07b19840 2023-09-10T01:28:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.932+0000 7f7c9f753b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f07a6b8e0 con 0x555f07b1d270 2023-09-10T01:28:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.933+0000 7f7c91b2a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b1d270 0x555f07b19300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.933+0000 7f7c91b2a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b1d270 0x555f07b19300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38616/0 (socket says 172.21.15.44:38616) 2023-09-10T01:28:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.933+0000 7f7c91b2a700 1 -- 172.21.15.44:0/331800822 learned_addr learned my addr 172.21.15.44:0/331800822 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.933+0000 7f7c7e7fc700 1 -- 172.21.15.44:0/3889026640 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f7c880cf680 con 0x7f7c8c100120 2023-09-10T01:28:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.933+0000 7f7c91329700 1 --2- 172.21.15.44:0/331800822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07b19840 0x555f07b19cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.934+0000 7f7c90b28700 1 --2- 172.21.15.44:0/331800822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b2bf60 0x555f07b2e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.934+0000 7f7c91329700 1 -- 172.21.15.44:0/331800822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b1d270 msgr2=0x555f07b19300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.934+0000 7f7c91329700 1 --2- 172.21.15.44:0/331800822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b1d270 0x555f07b19300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.934+0000 7f7c91329700 1 -- 172.21.15.44:0/331800822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b2bf60 msgr2=0x555f07b2e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.934+0000 7f7c91329700 1 --2- 172.21.15.44:0/331800822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b2bf60 0x555f07b2e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.934+0000 7f7c91329700 1 -- 172.21.15.44:0/331800822 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f07897d80 con 0x555f07b19840 2023-09-10T01:28:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.934+0000 7f7c91329700 1 --2- 172.21.15.44:0/331800822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07b19840 0x555f07b19cc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7c800097f0 tx=0x7f7c80009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39959ff0b6194d08 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:25.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.937+0000 7f7c91b2a700 1 --2- 172.21.15.44:0/3889026640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7c780a0830 0x7f7c780a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.942+0000 7f7c777fe700 1 -- 172.21.15.44:0/331800822 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c80011830 con 0x555f07b19840 2023-09-10T01:28:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.942+0000 7f7c777fe700 1 -- 172.21.15.44:0/331800822 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c80011e20 con 0x555f07b19840 2023-09-10T01:28:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.942+0000 7f7c777fe700 1 -- 172.21.15.44:0/331800822 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8000be30 con 0x555f07b19840 2023-09-10T01:28:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.942+0000 7f7c91b2a700 1 --2- 172.21.15.44:0/3889026640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7c780a0830 0x7f7c780a2cf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009550 tx=0x7f7c8400b040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.945+0000 7f7c9f753b80 1 -- 172.21.15.44:0/331800822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07b19840 msgr2=0x555f07b19cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.945+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/331800822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07b19840 0x555f07b19cc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7c800097f0 tx=0x7f7c80009b00 comp rx=0 tx=0).stop 2023-09-10T01:28:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.949+0000 7f7c9f753b80 1 -- 172.21.15.44:0/331800822 shutdown_connections 2023-09-10T01:28:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.949+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/331800822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b2bf60 0x555f07b2e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.949+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/331800822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b1d270 0x555f07b19300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.949+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/331800822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07b19840 0x555f07b19cc0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.949+0000 7f7c9f753b80 1 -- 172.21.15.44:0/331800822 >> 172.21.15.44:0/331800822 conn(0x555f07a8e030 msgr2=0x555f07b14560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.955+0000 7f7c9f753b80 1 -- 172.21.15.44:0/331800822 shutdown_connections 2023-09-10T01:28:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.962+0000 7f7c9f753b80 1 -- 172.21.15.44:0/331800822 wait complete. 2023-09-10T01:28:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.962+0000 7f7c9f753b80 1 -- start start 2023-09-10T01:28:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.962+0000 7f7c9f753b80 1 Processor -- start 2023-09-10T01:28:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.963+0000 7f7c9f753b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b19840 0x555f07b25510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.963+0000 7f7c9f753b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07b1d270 0x555f07b25a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.963+0000 7f7c9f753b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b2bf60 0x555f07b2b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.963+0000 7f7c9f753b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f07a09b10 con 0x555f07b19840 2023-09-10T01:28:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.963+0000 7f7c9f753b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f07a09810 con 0x555f07b1d270 2023-09-10T01:28:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.963+0000 7f7c9f753b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f078ed8c0 con 0x555f07b2bf60 2023-09-10T01:28:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.963+0000 7f7c9f753b80 10 client.0 _ll_register_callbacks cb 0x555f07953a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:28:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.963+0000 7f7c91b2a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b2bf60 0x555f07b2b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.963+0000 7f7c91b2a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b2bf60 0x555f07b2b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38626/0 (socket says 172.21.15.44:38626) 2023-09-10T01:28:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.963+0000 7f7c91b2a700 1 -- 172.21.15.44:0/1075194011 learned_addr learned my addr 172.21.15.44:0/1075194011 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.964+0000 7f7c91b2a700 1 -- 172.21.15.44:0/1075194011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07b1d270 msgr2=0x555f07b25a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:28:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.964+0000 7f7c91b2a700 1 --2- 172.21.15.44:0/1075194011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07b1d270 0x555f07b25a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.964+0000 7f7c91b2a700 1 -- 172.21.15.44:0/1075194011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b19840 msgr2=0x555f07b25510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.964+0000 7f7c91b2a700 1 --2- 172.21.15.44:0/1075194011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b19840 0x555f07b25510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.964+0000 7f7c91b2a700 1 -- 172.21.15.44:0/1075194011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c80009420 con 0x555f07b2bf60 2023-09-10T01:28:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.968+0000 7f7c91b2a700 1 --2- 172.21.15.44:0/1075194011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b2bf60 0x555f07b2b280 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7c84007c50 tx=0x7f7c84004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa0c64ac7f1c1afb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.968+0000 7f7c74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:28:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.969+0000 7f7c74ff9700 1 -- 172.21.15.44:0/1075194011 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84018c10 con 0x555f07b2bf60 2023-09-10T01:28:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.969+0000 7f7c74ff9700 1 -- 172.21.15.44:0/1075194011 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(189..189 src has 1..189) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f7c84018d70 con 0x555f07b2bf60 2023-09-10T01:28:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.969+0000 7f7c9f753b80 1 -- 172.21.15.44:0/1075194011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f079c3490 con 0x555f07b2bf60 2023-09-10T01:28:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.974+0000 7f7c74ff9700 1 -- 172.21.15.44:0/1075194011 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=164) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c84004bb0 con 0x555f07b2bf60 2023-09-10T01:28:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.974+0000 7f7c9f753b80 10 client.29956 fetch_fsmap learned FSMap version 164 2023-09-10T01:28:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.974+0000 7f7c9f753b80 1 -- 172.21.15.44:0/1075194011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=164,monmap=4+}) v3 -- 0x555f07880c00 con 0x555f07b2bf60 2023-09-10T01:28:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.974+0000 7f7c74ff9700 1 -- 172.21.15.44:0/1075194011 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555f079c3490 con 0x555f07b2bf60 2023-09-10T01:28:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.974+0000 7f7c74ff9700 1 -- 172.21.15.44:0/1075194011 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 164) v1 ==== 7363+0+0 (secure 0 0 0) 0x7f7c840043e0 con 0x555f07b2bf60 2023-09-10T01:28:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.974+0000 7f7c9f753b80 10 client.29956 fetch_fsmap finished waiting for FSMap version 164 2023-09-10T01:28:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.974+0000 7f7c9f753b80 10 client.29956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:28:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.974+0000 7f7c9f753b80 20 client.29956 populate_metadata read hostname 'smithi044' 2023-09-10T01:28:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.974+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/1075194011 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f07b44480 0x555f07b468a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.974+0000 7f7c9f753b80 4 client.29956 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:28:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.974+0000 7f7c9f753b80 1 -- 172.21.15.44:0/1075194011 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f07a80840 con 0x555f07b44480 2023-09-10T01:28:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.975+0000 7f7c91329700 1 --2- 172.21.15.44:0/1075194011 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f07b44480 0x555f07b468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.976+0000 7f7c74ff9700 10 client.29956 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:28:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.976+0000 7f7c91329700 1 --2- 172.21.15.44:0/1075194011 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f07b44480 0x555f07b468a0 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e366b3c3c9b0bc67 server_cookie=68066423d718cfb5 in_seq=0 out_seq=0 2023-09-10T01:28:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.981+0000 7f7c74ff9700 1 -- 172.21.15.44:0/1075194011 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555f07a80840 con 0x555f07b44480 2023-09-10T01:28:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.981+0000 7f7c74ff9700 10 client.29956 handle_command_reply: tid=0 2023-09-10T01:28:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.981+0000 7f7c9f753b80 1 client.29956 shutdown 2023-09-10T01:28:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.982+0000 7f7c9f753b80 1 -- 172.21.15.44:0/1075194011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b2bf60 msgr2=0x555f07b2b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.982+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/1075194011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b2bf60 0x555f07b2b280 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7c84007c50 tx=0x7f7c84004ad0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.982+0000 7f7c9f753b80 1 -- 172.21.15.44:0/1075194011 shutdown_connections 2023-09-10T01:28:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.982+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/1075194011 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f07b44480 0x555f07b468a0 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.982+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/1075194011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b2bf60 0x555f07b2b280 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.982+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/1075194011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b19840 0x555f07b25510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.982+0000 7f7c74ff9700 0 client.29956 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:28:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.982+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/1075194011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07b1d270 0x555f07b25a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.982+0000 7f7c9f753b80 1 -- 172.21.15.44:0/1075194011 >> 172.21.15.44:0/1075194011 conn(0x555f07a8e030 msgr2=0x555f07b137a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.982+0000 7f7c9f753b80 1 -- 172.21.15.44:0/1075194011 shutdown_connections 2023-09-10T01:28:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.983+0000 7f7c9f753b80 1 -- 172.21.15.44:0/1075194011 wait complete. 2023-09-10T01:28:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:25.983+0000 7f7c9f753b80 20 client.29956 trim_cache size 0 max 16384 2023-09-10T01:28:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.008+0000 7f7c9f753b80 1 Processor -- start 2023-09-10T01:28:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.008+0000 7f7c9f753b80 1 -- start start 2023-09-10T01:28:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.008+0000 7f7c9f753b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07a8e030 0x555f07b1e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.008+0000 7f7c9f753b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b19840 0x555f07b2b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.009+0000 7f7c9f753b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b1d270 0x555f07b13f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.009+0000 7f7c9f753b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f07b1e890 con 0x555f07b1d270 2023-09-10T01:28:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.009+0000 7f7c9f753b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f07b217f0 con 0x555f07a8e030 2023-09-10T01:28:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.009+0000 7f7c9f753b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f07b21dc0 con 0x555f07b19840 2023-09-10T01:28:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.009+0000 7f7c91b2a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b1d270 0x555f07b13f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.009+0000 7f7c91b2a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b1d270 0x555f07b13f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59984/0 (socket says 172.21.15.44:59984) 2023-09-10T01:28:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.009+0000 7f7c91b2a700 1 -- 172.21.15.44:0/1224480503 learned_addr learned my addr 172.21.15.44:0/1224480503 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:26.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.009+0000 7f7c91b2a700 1 -- 172.21.15.44:0/1224480503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b19840 msgr2=0x555f07b2b080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:28:26.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.009+0000 7f7c91b2a700 1 --2- 172.21.15.44:0/1224480503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b19840 0x555f07b2b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.009+0000 7f7c91b2a700 1 -- 172.21.15.44:0/1224480503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07a8e030 msgr2=0x555f07b1e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.009+0000 7f7c91b2a700 1 --2- 172.21.15.44:0/1224480503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07a8e030 0x555f07b1e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.009+0000 7f7c91b2a700 1 -- 172.21.15.44:0/1224480503 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f078be1f0 con 0x555f07b1d270 2023-09-10T01:28:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.009+0000 7f7c91b2a700 1 --2- 172.21.15.44:0/1224480503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b1d270 0x555f07b13f20 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009a00 tx=0x7f7c84006080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d07646fccad740 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.010+0000 7f7c77fff700 1 -- 172.21.15.44:0/1224480503 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84004bb0 con 0x555f07b1d270 2023-09-10T01:28:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.010+0000 7f7c77fff700 1 -- 172.21.15.44:0/1224480503 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c8400f040 con 0x555f07b1d270 2023-09-10T01:28:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.010+0000 7f7c77fff700 1 -- 172.21.15.44:0/1224480503 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84036ae0 con 0x555f07b1d270 2023-09-10T01:28:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.011+0000 7f7c9f753b80 1 -- 172.21.15.44:0/1224480503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b1d270 msgr2=0x555f07b13f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.011+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/1224480503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b1d270 0x555f07b13f20 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009a00 tx=0x7f7c84006080 comp rx=0 tx=0).stop 2023-09-10T01:28:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.011+0000 7f7c9f753b80 1 -- 172.21.15.44:0/1224480503 shutdown_connections 2023-09-10T01:28:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.011+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/1224480503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b1d270 0x555f07b13f20 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.011+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/1224480503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07a8e030 0x555f07b1e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.011+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/1224480503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b19840 0x555f07b2b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.011+0000 7f7c9f753b80 1 -- 172.21.15.44:0/1224480503 >> 172.21.15.44:0/1224480503 conn(0x555f07b17fb0 msgr2=0x555f07b20770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.011+0000 7f7c9f753b80 1 -- 172.21.15.44:0/1224480503 shutdown_connections 2023-09-10T01:28:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.011+0000 7f7c9f753b80 1 -- 172.21.15.44:0/1224480503 wait complete. 2023-09-10T01:28:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.011+0000 7f7c9f753b80 1 -- start start 2023-09-10T01:28:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.011+0000 7f7c9f753b80 1 Processor -- start 2023-09-10T01:28:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.012+0000 7f7c9f753b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07a8e030 0x555f07b2ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.012+0000 7f7c9f753b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b19840 0x555f07b28fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.012+0000 7f7c9f753b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b1d270 0x555f07b29500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.012+0000 7f7c9f753b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f07b29a40 con 0x555f07b19840 2023-09-10T01:28:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.012+0000 7f7c9f753b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f07b29bb0 con 0x555f07a8e030 2023-09-10T01:28:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.012+0000 7f7c9f753b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f07b29d70 con 0x555f07b1d270 2023-09-10T01:28:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.012+0000 7f7c9f753b80 10 client.0 _ll_register_callbacks cb 0x555f07953a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:28:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.012+0000 7f7c90b28700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b19840 0x555f07b28fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.012+0000 7f7c90b28700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b19840 0x555f07b28fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59988/0 (socket says 172.21.15.44:59988) 2023-09-10T01:28:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.012+0000 7f7c90b28700 1 -- 172.21.15.44:0/3177504231 learned_addr learned my addr 172.21.15.44:0/3177504231 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.012+0000 7f7c91b2a700 1 --2- 172.21.15.44:0/3177504231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b1d270 0x555f07b29500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.012+0000 7f7c90b28700 1 -- 172.21.15.44:0/3177504231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b1d270 msgr2=0x555f07b29500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.012+0000 7f7c90b28700 1 --2- 172.21.15.44:0/3177504231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b1d270 0x555f07b29500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.012+0000 7f7c90b28700 1 -- 172.21.15.44:0/3177504231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07a8e030 msgr2=0x555f07b2ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.012+0000 7f7c90b28700 1 --2- 172.21.15.44:0/3177504231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07a8e030 0x555f07b2ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.012+0000 7f7c90b28700 1 -- 172.21.15.44:0/3177504231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f07880c00 con 0x555f07b19840 2023-09-10T01:28:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.013+0000 7f7c90b28700 1 --2- 172.21.15.44:0/3177504231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b19840 0x555f07b28fc0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f7c88005fe0 tx=0x7f7c8809c0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da02b78ae8b89f4b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.013+0000 7f7c74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:28:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.013+0000 7f7c74ff9700 1 -- 172.21.15.44:0/3177504231 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c880b4ce0 con 0x555f07b19840 2023-09-10T01:28:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.013+0000 7f7c9f753b80 1 -- 172.21.15.44:0/3177504231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c80011e20 con 0x555f07b19840 2023-09-10T01:28:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.013+0000 7f7c74ff9700 1 -- 172.21.15.44:0/3177504231 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(189..189 src has 1..189) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f7c8809ce70 con 0x555f07b19840 2023-09-10T01:28:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.013+0000 7f7c74ff9700 1 -- 172.21.15.44:0/3177504231 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=164) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c8809c770 con 0x555f07b19840 2023-09-10T01:28:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.013+0000 7f7c9f753b80 10 client.20328 fetch_fsmap learned FSMap version 164 2023-09-10T01:28:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.013+0000 7f7c9f753b80 1 -- 172.21.15.44:0/3177504231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=164,monmap=4+}) v3 -- 0x555f07896250 con 0x555f07b19840 2023-09-10T01:28:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.015+0000 7f7c74ff9700 1 -- 172.21.15.44:0/3177504231 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c80011e20 con 0x555f07b19840 2023-09-10T01:28:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.015+0000 7f7c74ff9700 1 -- 172.21.15.44:0/3177504231 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 164) v1 ==== 7363+0+0 (secure 0 0 0) 0x7f7c8800e3e0 con 0x555f07b19840 2023-09-10T01:28:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.015+0000 7f7c9f753b80 10 client.20328 fetch_fsmap finished waiting for FSMap version 164 2023-09-10T01:28:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.015+0000 7f7c9f753b80 10 client.20328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:28:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.015+0000 7f7c9f753b80 20 client.20328 populate_metadata read hostname 'smithi044' 2023-09-10T01:28:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.015+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/3177504231 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f07b2bf60 0x555f07b9b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.015+0000 7f7c9f753b80 4 client.20328 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:28:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.015+0000 7f7c9f753b80 1 -- 172.21.15.44:0/3177504231 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555f07a648c0 con 0x555f07b2bf60 2023-09-10T01:28:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.015+0000 7f7c91329700 1 --2- 172.21.15.44:0/3177504231 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f07b2bf60 0x555f07b9b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.015+0000 7f7c74ff9700 10 client.20328 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:28:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.016+0000 7f7c91329700 1 --2- 172.21.15.44:0/3177504231 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f07b2bf60 0x555f07b9b5b0 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=535bc81f0ff11210 server_cookie=1e787cee76c28959 in_seq=0 out_seq=0 2023-09-10T01:28:26.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:25 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:28:26.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:25 smithi131 ceph-mon[103713]: pgmap v1552: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 452 op/s 2023-09-10T01:28:26.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:25 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:28:26.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:25 smithi044 ceph-mon[98676]: pgmap v1552: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 452 op/s 2023-09-10T01:28:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.065+0000 7f7c74ff9700 1 -- 172.21.15.44:0/3177504231 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555f07a648c0 con 0x555f07b2bf60 2023-09-10T01:28:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.065+0000 7f7c74ff9700 10 client.20328 handle_command_reply: tid=0 2023-09-10T01:28:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.065+0000 7f7c9f753b80 1 client.20328 shutdown 2023-09-10T01:28:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.069+0000 7f7c9f753b80 1 -- 172.21.15.44:0/3177504231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b19840 msgr2=0x555f07b28fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.069+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/3177504231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b19840 0x555f07b28fc0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f7c88005fe0 tx=0x7f7c8809c0d0 comp rx=0 tx=0).stop 2023-09-10T01:28:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.073+0000 7f7c9f753b80 1 -- 172.21.15.44:0/3177504231 shutdown_connections 2023-09-10T01:28:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.073+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/3177504231 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x555f07b2bf60 0x555f07b9b5b0 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.073+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/3177504231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f07b1d270 0x555f07b29500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.073+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/3177504231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f07a8e030 0x555f07b2ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.073+0000 7f7c9f753b80 1 --2- 172.21.15.44:0/3177504231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f07b19840 0x555f07b28fc0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.073+0000 7f7c9f753b80 1 -- 172.21.15.44:0/3177504231 >> 172.21.15.44:0/3177504231 conn(0x555f07b17fb0 msgr2=0x555f07b1f020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.074+0000 7f7c74ff9700 0 client.20328 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:28:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.075+0000 7f7c9f753b80 1 -- 172.21.15.44:0/3177504231 shutdown_connections 2023-09-10T01:28:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.075+0000 7f7c9f753b80 1 -- 172.21.15.44:0/3177504231 wait complete. 2023-09-10T01:28:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.076+0000 7f7c9f753b80 20 client.20328 trim_cache size 0 max 16384 2023-09-10T01:28:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:28:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:28:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:28:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:28:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.078+0000 7f7c9358d700 1 -- 172.21.15.44:0/3889026640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7c780a0830 msgr2=0x7f7c780a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.078+0000 7f7c9358d700 1 --2- 172.21.15.44:0/3889026640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7c780a0830 0x7f7c780a2cf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009550 tx=0x7f7c8400b040 comp rx=0 tx=0).stop 2023-09-10T01:28:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.078+0000 7f7c9358d700 1 -- 172.21.15.44:0/3889026640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c8c100120 msgr2=0x7f7c8c1959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.078+0000 7f7c9358d700 1 --2- 172.21.15.44:0/3889026640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c8c100120 0x7f7c8c1959c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7c88006010 tx=0x7f7c88004730 comp rx=0 tx=0).stop 2023-09-10T01:28:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.078+0000 7f7c9358d700 1 -- 172.21.15.44:0/3889026640 shutdown_connections 2023-09-10T01:28:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.078+0000 7f7c9358d700 1 --2- 172.21.15.44:0/3889026640 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7c780a0830 0x7f7c780a2cf0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.078+0000 7f7c9358d700 1 --2- 172.21.15.44:0/3889026640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c8c1055b0 0x7f7c8c198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.078+0000 7f7c9358d700 1 --2- 172.21.15.44:0/3889026640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c8c0ff760 0x7f7c8c195480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.078+0000 7f7c9358d700 1 --2- 172.21.15.44:0/3889026640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c8c100120 0x7f7c8c1959c0 secure :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7c88006010 tx=0x7f7c88004730 comp rx=0 tx=0).stop 2023-09-10T01:28:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.078+0000 7f7c9358d700 1 -- 172.21.15.44:0/3889026640 >> 172.21.15.44:0/3889026640 conn(0x7f7c8c074f30 msgr2=0x7f7c8c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.092+0000 7f7c9358d700 1 -- 172.21.15.44:0/3889026640 shutdown_connections 2023-09-10T01:28:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:26.092+0000 7f7c9358d700 1 -- 172.21.15.44:0/3889026640 wait complete. 2023-09-10T01:28:26.184 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 440.0 s, 263.8 tps, lat 377.950 ms stddev 554.459 2023-09-10T01:28:27.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:27 smithi131 ceph-mon[103713]: osdmap e190: 12 total, 12 up, 12 in 2023-09-10T01:28:27.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:27 smithi044 ceph-mon[98676]: osdmap e190: 12 total, 12 up, 12 in 2023-09-10T01:28:27.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:27 smithi099 ceph-mon[102692]: osdmap e190: 12 total, 12 up, 12 in 2023-09-10T01:28:28.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:28 smithi131 ceph-mon[103713]: pgmap v1554: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.8 MiB/s wr, 800 op/s 2023-09-10T01:28:28.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:28 smithi044 ceph-mon[98676]: pgmap v1554: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.8 MiB/s wr, 800 op/s 2023-09-10T01:28:28.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:28 smithi099 ceph-mon[102692]: pgmap v1554: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.8 MiB/s wr, 800 op/s 2023-09-10T01:28:30.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:30 smithi131 ceph-mon[103713]: pgmap v1555: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.8 MiB/s wr, 800 op/s 2023-09-10T01:28:30.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:30 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:28:30.812 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:30 smithi044 ceph-mon[98676]: pgmap v1555: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.8 MiB/s wr, 800 op/s 2023-09-10T01:28:30.812 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:30 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:28:30.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:30 smithi099 ceph-mon[102692]: pgmap v1555: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.8 MiB/s wr, 800 op/s 2023-09-10T01:28:30.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:30 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:28:31.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 445.0 s, 266.5 tps, lat 374.334 ms stddev 443.388 2023-09-10T01:28:32.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:32 smithi131 ceph-mon[103713]: pgmap v1556: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.9 MiB/s wr, 840 op/s 2023-09-10T01:28:32.816 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:32 smithi044 ceph-mon[98676]: pgmap v1556: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.9 MiB/s wr, 840 op/s 2023-09-10T01:28:32.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:32 smithi099 ceph-mon[102692]: pgmap v1556: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.9 MiB/s wr, 840 op/s 2023-09-10T01:28:34.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:34 smithi131 ceph-mon[103713]: pgmap v1557: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.9 MiB/s wr, 841 op/s 2023-09-10T01:28:34.812 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:34 smithi044 ceph-mon[98676]: pgmap v1557: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.9 MiB/s wr, 841 op/s 2023-09-10T01:28:34.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:34 smithi099 ceph-mon[102692]: pgmap v1557: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.9 MiB/s wr, 841 op/s 2023-09-10T01:28:36.183 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 450.0 s, 258.7 tps, lat 389.128 ms stddev 444.555 2023-09-10T01:28:36.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:36 smithi131 ceph-mon[103713]: pgmap v1558: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.9 MiB/s wr, 841 op/s 2023-09-10T01:28:36.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:36 smithi044 ceph-mon[98676]: pgmap v1558: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.9 MiB/s wr, 841 op/s 2023-09-10T01:28:36.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:36 smithi099 ceph-mon[102692]: pgmap v1558: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.9 MiB/s wr, 841 op/s 2023-09-10T01:28:38.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:38 smithi131 ceph-mon[103713]: pgmap v1559: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.2 MiB/s wr, 764 op/s 2023-09-10T01:28:38.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:38 smithi044 ceph-mon[98676]: pgmap v1559: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.2 MiB/s wr, 764 op/s 2023-09-10T01:28:38.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:38 smithi099 ceph-mon[102692]: pgmap v1559: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.2 MiB/s wr, 764 op/s 2023-09-10T01:28:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.331+0000 7f7197bb8700 1 -- 172.21.15.44:0/575044828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7190119de0 msgr2=0x7f719011c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.331+0000 7f7197bb8700 1 --2- 172.21.15.44:0/575044828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7190119de0 0x7f719011c1d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f718001a9a0 tx=0x7f718001acb0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.336+0000 7f7197bb8700 1 -- 172.21.15.44:0/575044828 shutdown_connections 2023-09-10T01:28:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.336+0000 7f7197bb8700 1 --2- 172.21.15.44:0/575044828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7190119de0 0x7f719011c1d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.336+0000 7f7197bb8700 1 --2- 172.21.15.44:0/575044828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f719010ed50 0x7f7190107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.336+0000 7f7197bb8700 1 --2- 172.21.15.44:0/575044828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719010f1c0 0x7f7190107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.336+0000 7f7197bb8700 1 -- 172.21.15.44:0/575044828 >> 172.21.15.44:0/575044828 conn(0x7f719006c6c0 msgr2=0x7f719006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.336+0000 7f7197bb8700 1 -- 172.21.15.44:0/575044828 shutdown_connections 2023-09-10T01:28:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.336+0000 7f7197bb8700 1 -- 172.21.15.44:0/575044828 wait complete. 2023-09-10T01:28:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.337+0000 7f7197bb8700 1 Processor -- start 2023-09-10T01:28:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.337+0000 7f7197bb8700 1 -- start start 2023-09-10T01:28:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.337+0000 7f7197bb8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719010ed50 0x7f7190115030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.337+0000 7f7197bb8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f719010f1c0 0x7f7190115570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.337+0000 7f7197bb8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7190119de0 0x7f71901c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.337+0000 7f7197bb8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71901c50c0 con 0x7f719010ed50 2023-09-10T01:28:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.337+0000 7f7197bb8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71901c5230 con 0x7f719010f1c0 2023-09-10T01:28:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.337+0000 7f7197bb8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71901c53f0 con 0x7f7190119de0 2023-09-10T01:28:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.337+0000 7f7196155700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7190119de0 0x7f71901c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.337+0000 7f7196155700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7190119de0 0x7f71901c4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38652/0 (socket says 172.21.15.44:38652) 2023-09-10T01:28:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.337+0000 7f7196155700 1 -- 172.21.15.44:0/8365600 learned_addr learned my addr 172.21.15.44:0/8365600 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.338+0000 7f7195954700 1 --2- 172.21.15.44:0/8365600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719010ed50 0x7f7190115030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.338+0000 7f7196155700 1 -- 172.21.15.44:0/8365600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f719010f1c0 msgr2=0x7f7190115570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T01:28:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.338+0000 7f7196155700 1 --2- 172.21.15.44:0/8365600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f719010f1c0 0x7f7190115570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.338+0000 7f7196155700 1 -- 172.21.15.44:0/8365600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719010ed50 msgr2=0x7f7190115030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.338+0000 7f7196155700 1 --2- 172.21.15.44:0/8365600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719010ed50 0x7f7190115030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.338+0000 7f7196155700 1 -- 172.21.15.44:0/8365600 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718001a600 con 0x7f7190119de0 2023-09-10T01:28:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.338+0000 7f7195954700 1 --2- 172.21.15.44:0/8365600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719010ed50 0x7f7190115030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:28:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.338+0000 7f7196155700 1 --2- 172.21.15.44:0/8365600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7190119de0 0x7f71901c4a80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7180005b40 tx=0x7f7180005490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.338+0000 7f7186ffd700 1 -- 172.21.15.44:0/8365600 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7180003cd0 con 0x7f7190119de0 2023-09-10T01:28:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.338+0000 7f7197bb8700 1 -- 172.21.15.44:0/8365600 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71901c5670 con 0x7f7190119de0 2023-09-10T01:28:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.338+0000 7f7197bb8700 1 -- 172.21.15.44:0/8365600 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71901cbd10 con 0x7f7190119de0 2023-09-10T01:28:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.338+0000 7f7186ffd700 1 -- 172.21.15.44:0/8365600 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71800056b0 con 0x7f7190119de0 2023-09-10T01:28:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.339+0000 7f7186ffd700 1 -- 172.21.15.44:0/8365600 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718000fea0 con 0x7f7190119de0 2023-09-10T01:28:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.341+0000 7f7186ffd700 1 -- 172.21.15.44:0/8365600 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f718000f480 con 0x7f7190119de0 2023-09-10T01:28:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.341+0000 7f7186ffd700 1 --2- 172.21.15.44:0/8365600 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f717c0a0830 0x7f717c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.341+0000 7f7186ffd700 1 -- 172.21.15.44:0/8365600 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f71800cdb50 con 0x7f7190119de0 2023-09-10T01:28:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.342+0000 7f7195954700 1 --2- 172.21.15.44:0/8365600 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f717c0a0830 0x7f717c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.342+0000 7f7195954700 1 --2- 172.21.15.44:0/8365600 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f717c0a0830 0x7f717c0a2cf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f718c007eb0 tx=0x7f718c0073c0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.347+0000 7f71a3d7eb80 1 Processor -- start 2023-09-10T01:28:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.359+0000 7f71a3d7eb80 1 -- start start 2023-09-10T01:28:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.359+0000 7f71a3d7eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb9481a0 0x558fcb948580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.359+0000 7f71a3d7eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb942300 0x558fcb942780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.359+0000 7f71a3d7eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb942cc0 0x558fcb94ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.359+0000 7f71a3d7eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb89b6d0 con 0x558fcb9481a0 2023-09-10T01:28:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.359+0000 7f71a3d7eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb898ff0 con 0x558fcb942cc0 2023-09-10T01:28:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.359+0000 7f71a3d7eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb8948e0 con 0x558fcb942300 2023-09-10T01:28:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.359+0000 7f7195954700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb9481a0 0x558fcb948580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.359+0000 7f7195954700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb9481a0 0x558fcb948580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60006/0 (socket says 172.21.15.44:60006) 2023-09-10T01:28:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.359+0000 7f7195954700 1 -- 172.21.15.44:0/4216653740 learned_addr learned my addr 172.21.15.44:0/4216653740 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.359+0000 7f7195153700 1 --2- 172.21.15.44:0/4216653740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb942300 0x558fcb942780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.359+0000 7f7196155700 1 --2- 172.21.15.44:0/4216653740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb942cc0 0x558fcb94ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.360+0000 7f7195954700 1 -- 172.21.15.44:0/4216653740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb942300 msgr2=0x558fcb942780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.360+0000 7f7195954700 1 --2- 172.21.15.44:0/4216653740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb942300 0x558fcb942780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.360+0000 7f7195954700 1 -- 172.21.15.44:0/4216653740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb942cc0 msgr2=0x558fcb94ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.360+0000 7f7195954700 1 --2- 172.21.15.44:0/4216653740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb942cc0 0x558fcb94ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.360+0000 7f7195954700 1 -- 172.21.15.44:0/4216653740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fcb6c0d80 con 0x558fcb9481a0 2023-09-10T01:28:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.360+0000 7f7195954700 1 --2- 172.21.15.44:0/4216653740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb9481a0 0x558fcb948580 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f718c00f440 tx=0x7f718c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7279e605f7f34d67 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.361+0000 7f717bfff700 1 -- 172.21.15.44:0/4216653740 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c020070 con 0x558fcb9481a0 2023-09-10T01:28:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.361+0000 7f717bfff700 1 -- 172.21.15.44:0/4216653740 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f718c00fbe0 con 0x558fcb9481a0 2023-09-10T01:28:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.361+0000 7f717bfff700 1 -- 172.21.15.44:0/4216653740 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c004750 con 0x558fcb9481a0 2023-09-10T01:28:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.364+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/4216653740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb9481a0 msgr2=0x558fcb948580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.364+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/4216653740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb9481a0 0x558fcb948580 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f718c00f440 tx=0x7f718c00f000 comp rx=0 tx=0).stop 2023-09-10T01:28:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.364+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/4216653740 shutdown_connections 2023-09-10T01:28:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.364+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/4216653740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb942cc0 0x558fcb94ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.364+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/4216653740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb9481a0 0x558fcb948580 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.364+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/4216653740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb942300 0x558fcb942780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.364+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/4216653740 >> 172.21.15.44:0/4216653740 conn(0x558fcb8b7030 msgr2=0x558fcb93d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.364+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/4216653740 shutdown_connections 2023-09-10T01:28:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.364+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/4216653740 wait complete. 2023-09-10T01:28:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.365+0000 7f71a3d7eb80 1 -- start start 2023-09-10T01:28:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.365+0000 7f71a3d7eb80 1 Processor -- start 2023-09-10T01:28:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.366+0000 7f71a3d7eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb942300 0x558fcb94edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.366+0000 7f71a3d7eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb942cc0 0x558fcb94c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.366+0000 7f71a3d7eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb9481a0 0x558fcb94c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.366+0000 7f71a3d7eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb832b10 con 0x558fcb942cc0 2023-09-10T01:28:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.366+0000 7f71a3d7eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb832810 con 0x558fcb9481a0 2023-09-10T01:28:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.366+0000 7f71a3d7eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb7168c0 con 0x558fcb942300 2023-09-10T01:28:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.366+0000 7f71a3d7eb80 10 client.0 _ll_register_callbacks cb 0x558fcb77ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:28:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.366+0000 7f7195954700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb942300 0x558fcb94edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.366+0000 7f7195954700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb942300 0x558fcb94edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38662/0 (socket says 172.21.15.44:38662) 2023-09-10T01:28:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.366+0000 7f7195954700 1 -- 172.21.15.44:0/1043533877 learned_addr learned my addr 172.21.15.44:0/1043533877 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.366+0000 7f7195153700 1 --2- 172.21.15.44:0/1043533877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb942cc0 0x558fcb94c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.366+0000 7f7195954700 1 -- 172.21.15.44:0/1043533877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb9481a0 msgr2=0x558fcb94c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.366+0000 7f7195954700 1 --2- 172.21.15.44:0/1043533877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb9481a0 0x558fcb94c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.366+0000 7f7195954700 1 -- 172.21.15.44:0/1043533877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb942cc0 msgr2=0x558fcb94c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.366+0000 7f7195954700 1 --2- 172.21.15.44:0/1043533877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb942cc0 0x558fcb94c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.366+0000 7f7195954700 1 -- 172.21.15.44:0/1043533877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718c00cd10 con 0x558fcb942300 2023-09-10T01:28:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.367+0000 7f7195954700 1 --2- 172.21.15.44:0/1043533877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb942300 0x558fcb94edd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f718c00c930 tx=0x7f718c007d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2862bf563474fd95 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.367+0000 7f71797fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:28:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.367+0000 7f71797fa700 1 -- 172.21.15.44:0/1043533877 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c020070 con 0x558fcb942300 2023-09-10T01:28:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.367+0000 7f71797fa700 1 -- 172.21.15.44:0/1043533877 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f718c004560 con 0x558fcb942300 2023-09-10T01:28:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.368+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/1043533877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fcb7ec490 con 0x558fcb942300 2023-09-10T01:28:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.368+0000 7f71797fa700 1 -- 172.21.15.44:0/1043533877 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7f718c00fe10 con 0x558fcb942300 2023-09-10T01:28:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.368+0000 7f71a3d7eb80 10 client.29971 fetch_fsmap learned FSMap version 165 2023-09-10T01:28:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.368+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/1043533877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=165,monmap=4+}) v3 -- 0x558fcb6a9c00 con 0x558fcb942300 2023-09-10T01:28:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.369+0000 7f71797fa700 1 -- 172.21.15.44:0/1043533877 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558fcb7ec490 con 0x558fcb942300 2023-09-10T01:28:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.369+0000 7f71797fa700 1 -- 172.21.15.44:0/1043533877 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 165) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f718c026c50 con 0x558fcb942300 2023-09-10T01:28:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.369+0000 7f71a3d7eb80 10 client.29971 fetch_fsmap finished waiting for FSMap version 165 2023-09-10T01:28:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.369+0000 7f71a3d7eb80 10 client.29971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:28:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.369+0000 7f71a3d7eb80 20 client.29971 populate_metadata read hostname 'smithi044' 2023-09-10T01:28:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.369+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/1043533877 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558fcb95c440 0x558fcb95e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.369+0000 7f71a3d7eb80 4 client.29971 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:28:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.369+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/1043533877 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fcb8a9840 con 0x558fcb95c440 2023-09-10T01:28:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.369+0000 7f7195153700 1 --2- 172.21.15.44:0/1043533877 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558fcb95c440 0x558fcb95e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.369+0000 7f71797fa700 10 client.29971 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:28:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.370+0000 7f7195153700 1 --2- 172.21.15.44:0/1043533877 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558fcb95c440 0x558fcb95e860 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14afe3597c2edb3e server_cookie=8df191ae25628370 in_seq=0 out_seq=0 2023-09-10T01:28:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.370+0000 7f71797fa700 1 -- 172.21.15.44:0/1043533877 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558fcb8a9840 con 0x558fcb95c440 2023-09-10T01:28:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.370+0000 7f71797fa700 10 client.29971 handle_command_reply: tid=0 2023-09-10T01:28:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.370+0000 7f71a3d7eb80 1 client.29971 shutdown 2023-09-10T01:28:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.370+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/1043533877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb942300 msgr2=0x558fcb94edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.370+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/1043533877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb942300 0x558fcb94edd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f718c00c930 tx=0x7f718c007d10 comp rx=0 tx=0).stop 2023-09-10T01:28:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.370+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/1043533877 shutdown_connections 2023-09-10T01:28:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.370+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/1043533877 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558fcb95c440 0x558fcb95e860 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.370+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/1043533877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb9481a0 0x558fcb94c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.370+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/1043533877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb942300 0x558fcb94edd0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.370+0000 7f71797fa700 0 client.29971 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:28:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.370+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/1043533877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb942cc0 0x558fcb94c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.370+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/1043533877 >> 172.21.15.44:0/1043533877 conn(0x558fcb8b7030 msgr2=0x558fcb93c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.371+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/1043533877 shutdown_connections 2023-09-10T01:28:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.371+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/1043533877 wait complete. 2023-09-10T01:28:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.372+0000 7f71a3d7eb80 20 client.29971 trim_cache size 0 max 16384 2023-09-10T01:28:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.411+0000 7f71a3d7eb80 1 Processor -- start 2023-09-10T01:28:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.413+0000 7f71a3d7eb80 1 -- start start 2023-09-10T01:28:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.413+0000 7f71a3d7eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb942300 0x558fcb949d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.413+0000 7f71a3d7eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb942cc0 0x558fcb94e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.413+0000 7f71a3d7eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb9481a0 0x558fcb9470b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.413+0000 7f71a3d7eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb93e310 con 0x558fcb942cc0 2023-09-10T01:28:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.413+0000 7f71a3d7eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb94a640 con 0x558fcb942300 2023-09-10T01:28:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.413+0000 7f71a3d7eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb946ca0 con 0x558fcb9481a0 2023-09-10T01:28:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.425+0000 7f7196155700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb9481a0 0x558fcb9470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.425+0000 7f7196155700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb9481a0 0x558fcb9470b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38670/0 (socket says 172.21.15.44:38670) 2023-09-10T01:28:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.425+0000 7f7196155700 1 -- 172.21.15.44:0/1136336647 learned_addr learned my addr 172.21.15.44:0/1136336647 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.425+0000 7f7196155700 1 -- 172.21.15.44:0/1136336647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb942300 msgr2=0x558fcb949d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.425+0000 7f7196155700 1 --2- 172.21.15.44:0/1136336647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb942300 0x558fcb949d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.425+0000 7f7196155700 1 -- 172.21.15.44:0/1136336647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb942cc0 msgr2=0x558fcb94e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.425+0000 7f7196155700 1 --2- 172.21.15.44:0/1136336647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb942cc0 0x558fcb94e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.425+0000 7f7196155700 1 -- 172.21.15.44:0/1136336647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fcb6e71f0 con 0x558fcb9481a0 2023-09-10T01:28:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.427+0000 7f7196155700 1 --2- 172.21.15.44:0/1136336647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb9481a0 0x558fcb9470b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7180097220 tx=0x7f718009a750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e6f7ee61a2c6c11 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.434+0000 7f7184d89700 1 -- 172.21.15.44:0/1136336647 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71800a2d00 con 0x558fcb9481a0 2023-09-10T01:28:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.434+0000 7f7184d89700 1 -- 172.21.15.44:0/1136336647 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71800990c0 con 0x558fcb9481a0 2023-09-10T01:28:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.434+0000 7f7184d89700 1 -- 172.21.15.44:0/1136336647 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718009a470 con 0x558fcb9481a0 2023-09-10T01:28:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.435+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/1136336647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb9481a0 msgr2=0x558fcb9470b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.435+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/1136336647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb9481a0 0x558fcb9470b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7180097220 tx=0x7f718009a750 comp rx=0 tx=0).stop 2023-09-10T01:28:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.457+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/1136336647 shutdown_connections 2023-09-10T01:28:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.457+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/1136336647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb9481a0 0x558fcb9470b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.457+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/1136336647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb942300 0x558fcb949d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.457+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/1136336647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb942cc0 0x558fcb94e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.457+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/1136336647 >> 172.21.15.44:0/1136336647 conn(0x558fcb8b7030 msgr2=0x558fcb946190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.472+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/1136336647 shutdown_connections 2023-09-10T01:28:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.474+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/1136336647 wait complete. 2023-09-10T01:28:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.474+0000 7f71a3d7eb80 1 -- start start 2023-09-10T01:28:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.474+0000 7f71a3d7eb80 1 Processor -- start 2023-09-10T01:28:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.475+0000 7f71a3d7eb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb942300 0x558fcb951510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.475+0000 7f71a3d7eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb942cc0 0x558fcb951a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.475+0000 7f71a3d7eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb9481a0 0x558fcb9af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.475+0000 7f71a3d7eb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb9521c0 con 0x558fcb942300 2023-09-10T01:28:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.475+0000 7f71a3d7eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb952330 con 0x558fcb9481a0 2023-09-10T01:28:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.475+0000 7f71a3d7eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb9af7b0 con 0x558fcb942cc0 2023-09-10T01:28:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.475+0000 7f71a3d7eb80 10 client.0 _ll_register_callbacks cb 0x7f7188006f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:28:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.475+0000 7f7196155700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb9481a0 0x558fcb9af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.475+0000 7f7196155700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb9481a0 0x558fcb9af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33536/0 (socket says 172.21.15.44:33536) 2023-09-10T01:28:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.475+0000 7f7196155700 1 -- 172.21.15.44:0/272435911 learned_addr learned my addr 172.21.15.44:0/272435911 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.476+0000 7f7196155700 1 -- 172.21.15.44:0/272435911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb942cc0 msgr2=0x558fcb951a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.476+0000 7f7196155700 1 --2- 172.21.15.44:0/272435911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb942cc0 0x558fcb951a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.476+0000 7f7196155700 1 -- 172.21.15.44:0/272435911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb942300 msgr2=0x558fcb951510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:28:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.476+0000 7f7196155700 1 --2- 172.21.15.44:0/272435911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb942300 0x558fcb951510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.476+0000 7f7196155700 1 -- 172.21.15.44:0/272435911 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71901cbd10 con 0x558fcb9481a0 2023-09-10T01:28:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.476+0000 7f7196155700 1 --2- 172.21.15.44:0/272435911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb9481a0 0x558fcb9af270 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f718001c5d0 tx=0x7f718009a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a52b91c50a52f85 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.476+0000 7f71797fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:28:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.476+0000 7f71797fa700 1 -- 172.21.15.44:0/272435911 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71800a2d00 con 0x558fcb9481a0 2023-09-10T01:28:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.476+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/272435911 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f718c00fbe0 con 0x558fcb9481a0 2023-09-10T01:28:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.476+0000 7f71797fa700 1 -- 172.21.15.44:0/272435911 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f718009b190 con 0x558fcb9481a0 2023-09-10T01:28:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.477+0000 7f71797fa700 1 -- 172.21.15.44:0/272435911 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71800a2870 con 0x558fcb9481a0 2023-09-10T01:28:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.478+0000 7f71a3d7eb80 10 client.29957 fetch_fsmap learned FSMap version 165 2023-09-10T01:28:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.478+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/272435911 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=165,monmap=4+}) v3 -- 0x558fcb6bf250 con 0x558fcb9481a0 2023-09-10T01:28:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.478+0000 7f71797fa700 1 -- 172.21.15.44:0/272435911 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f718c00fbe0 con 0x558fcb9481a0 2023-09-10T01:28:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.478+0000 7f71797fa700 1 -- 172.21.15.44:0/272435911 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 165) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f718007bd20 con 0x558fcb9481a0 2023-09-10T01:28:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.478+0000 7f71a3d7eb80 10 client.29957 fetch_fsmap finished waiting for FSMap version 165 2023-09-10T01:28:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.478+0000 7f71a3d7eb80 10 client.29957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:28:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.478+0000 7f71a3d7eb80 20 client.29957 populate_metadata read hostname 'smithi044' 2023-09-10T01:28:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.478+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/272435911 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558fcb9b0ba0 0x558fcb9b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.478+0000 7f71a3d7eb80 4 client.29957 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:28:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.478+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/272435911 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558fcb88d8c0 con 0x558fcb9b0ba0 2023-09-10T01:28:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.484+0000 7f7195954700 1 --2- 172.21.15.44:0/272435911 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558fcb9b0ba0 0x558fcb9b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.485+0000 7f7195954700 1 --2- 172.21.15.44:0/272435911 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558fcb9b0ba0 0x558fcb9b3000 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19f7a7f63c8a0d0b server_cookie=40cb1dc4aaf3715c in_seq=0 out_seq=0 2023-09-10T01:28:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.485+0000 7f71797fa700 10 client.29957 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:28:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.556+0000 7f71797fa700 1 -- 172.21.15.44:0/272435911 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558fcb88d8c0 con 0x558fcb9b0ba0 2023-09-10T01:28:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.556+0000 7f71797fa700 10 client.29957 handle_command_reply: tid=0 2023-09-10T01:28:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.556+0000 7f71a3d7eb80 1 client.29957 shutdown 2023-09-10T01:28:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.556+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/272435911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb9481a0 msgr2=0x558fcb9af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.556+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/272435911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb9481a0 0x558fcb9af270 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f718001c5d0 tx=0x7f718009a4d0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.558+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/272435911 shutdown_connections 2023-09-10T01:28:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.558+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/272435911 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558fcb9b0ba0 0x558fcb9b3000 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.558+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/272435911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558fcb9481a0 0x558fcb9af270 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.558+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/272435911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558fcb942300 0x558fcb951510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.558+0000 7f71797fa700 0 client.29957 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:28:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.558+0000 7f71a3d7eb80 1 --2- 172.21.15.44:0/272435911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558fcb942cc0 0x558fcb951a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.558+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/272435911 >> 172.21.15.44:0/272435911 conn(0x558fcb8b7030 msgr2=0x558fcb9470b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.558+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/272435911 shutdown_connections 2023-09-10T01:28:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.558+0000 7f71a3d7eb80 1 -- 172.21.15.44:0/272435911 wait complete. 2023-09-10T01:28:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.559+0000 7f71a3d7eb80 20 client.29957 trim_cache size 0 max 16384 2023-09-10T01:28:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:28:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:28:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:28:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:28:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.559+0000 7f7197bb8700 1 -- 172.21.15.44:0/8365600 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f717c0a0830 msgr2=0x7f717c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.559+0000 7f7197bb8700 1 --2- 172.21.15.44:0/8365600 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f717c0a0830 0x7f717c0a2cf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f718c007eb0 tx=0x7f718c0073c0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.559+0000 7f7197bb8700 1 -- 172.21.15.44:0/8365600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7190119de0 msgr2=0x7f71901c4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.559+0000 7f7197bb8700 1 --2- 172.21.15.44:0/8365600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7190119de0 0x7f71901c4a80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7180005b40 tx=0x7f7180005490 comp rx=0 tx=0).stop 2023-09-10T01:28:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.560+0000 7f7197bb8700 1 -- 172.21.15.44:0/8365600 shutdown_connections 2023-09-10T01:28:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.560+0000 7f7197bb8700 1 --2- 172.21.15.44:0/8365600 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f717c0a0830 0x7f717c0a2cf0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.560+0000 7f7197bb8700 1 --2- 172.21.15.44:0/8365600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7190119de0 0x7f71901c4a80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.560+0000 7f7197bb8700 1 --2- 172.21.15.44:0/8365600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f719010ed50 0x7f7190115030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.560+0000 7f7197bb8700 1 --2- 172.21.15.44:0/8365600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f719010f1c0 0x7f7190115570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.560+0000 7f7197bb8700 1 -- 172.21.15.44:0/8365600 >> 172.21.15.44:0/8365600 conn(0x7f719006c6c0 msgr2=0x7f7190070fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.562+0000 7f7197bb8700 1 -- 172.21.15.44:0/8365600 shutdown_connections 2023-09-10T01:28:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:39.563+0000 7f7197bb8700 1 -- 172.21.15.44:0/8365600 wait complete. 2023-09-10T01:28:40.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:40 smithi131 ceph-mon[103713]: pgmap v1560: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 689 op/s 2023-09-10T01:28:40.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:40 smithi044 ceph-mon[98676]: pgmap v1560: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 689 op/s 2023-09-10T01:28:40.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:40 smithi099 ceph-mon[102692]: pgmap v1560: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 689 op/s 2023-09-10T01:28:41.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 455.0 s, 253.7 tps, lat 427.644 ms stddev 402.302 2023-09-10T01:28:41.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:41 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:28:41.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:41 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:28:42.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:41 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:28:42.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:42 smithi099 ceph-mon[102692]: pgmap v1561: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.7 MiB/s wr, 1.03k op/s 2023-09-10T01:28:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:42 smithi131 ceph-mon[103713]: pgmap v1561: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.7 MiB/s wr, 1.03k op/s 2023-09-10T01:28:43.065 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:42 smithi044 ceph-mon[98676]: pgmap v1561: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.7 MiB/s wr, 1.03k op/s 2023-09-10T01:28:44.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:44 smithi099 ceph-mon[102692]: pgmap v1562: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.6 MiB/s wr, 688 op/s 2023-09-10T01:28:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:44 smithi131 ceph-mon[103713]: pgmap v1562: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.6 MiB/s wr, 688 op/s 2023-09-10T01:28:45.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:44 smithi044 ceph-mon[98676]: pgmap v1562: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.6 MiB/s wr, 688 op/s 2023-09-10T01:28:46.183 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 460.0 s, 255.6 tps, lat 373.089 ms stddev 155.507 2023-09-10T01:28:46.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:46 smithi099 ceph-mon[102692]: pgmap v1563: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.6 MiB/s wr, 688 op/s 2023-09-10T01:28:47.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:46 smithi131 ceph-mon[103713]: pgmap v1563: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.6 MiB/s wr, 688 op/s 2023-09-10T01:28:47.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:46 smithi044 ceph-mon[98676]: pgmap v1563: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.6 MiB/s wr, 688 op/s 2023-09-10T01:28:48.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:48 smithi099 ceph-mon[102692]: pgmap v1564: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 10 MiB/s wr, 1.04k op/s 2023-09-10T01:28:49.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:48 smithi131 ceph-mon[103713]: pgmap v1564: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 10 MiB/s wr, 1.04k op/s 2023-09-10T01:28:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:48 smithi044 ceph-mon[98676]: pgmap v1564: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 10 MiB/s wr, 1.04k op/s 2023-09-10T01:28:50.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:50 smithi099 ceph-mon[102692]: pgmap v1565: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.8 MiB/s wr, 700 op/s 2023-09-10T01:28:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:50 smithi131 ceph-mon[103713]: pgmap v1565: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.8 MiB/s wr, 700 op/s 2023-09-10T01:28:51.070 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:50 smithi044 ceph-mon[98676]: pgmap v1565: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.8 MiB/s wr, 700 op/s 2023-09-10T01:28:51.182 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 465.0 s, 274.6 tps, lat 371.669 ms stddev 144.927 2023-09-10T01:28:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.760+0000 7fa220a36700 1 -- 172.21.15.44:0/4031477096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa21c1055b0 msgr2=0x7fa21c105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.760+0000 7fa220a36700 1 --2- 172.21.15.44:0/4031477096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa21c1055b0 0x7fa21c105990 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0099c0 tx=0x7fa20c009cd0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.760+0000 7fa220a36700 1 -- 172.21.15.44:0/4031477096 shutdown_connections 2023-09-10T01:28:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.760+0000 7fa220a36700 1 --2- 172.21.15.44:0/4031477096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa21c100050 0x7fa21c10c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.760+0000 7fa220a36700 1 --2- 172.21.15.44:0/4031477096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa21c1055b0 0x7fa21c105990 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.760+0000 7fa220a36700 1 --2- 172.21.15.44:0/4031477096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa21c0ff690 0x7fa21c0ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.760+0000 7fa220a36700 1 -- 172.21.15.44:0/4031477096 >> 172.21.15.44:0/4031477096 conn(0x7fa21c077430 msgr2=0x7fa21c077840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.760+0000 7fa220a36700 1 -- 172.21.15.44:0/4031477096 shutdown_connections 2023-09-10T01:28:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.760+0000 7fa220a36700 1 -- 172.21.15.44:0/4031477096 wait complete. 2023-09-10T01:28:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.761+0000 7fa220a36700 1 Processor -- start 2023-09-10T01:28:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.761+0000 7fa220a36700 1 -- start start 2023-09-10T01:28:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.761+0000 7fa220a36700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa21c0ff690 0x7fa21c19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.761+0000 7fa220a36700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa21c100050 0x7fa21c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.761+0000 7fa220a36700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa21c1055b0 0x7fa21c1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.761+0000 7fa220a36700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c196a50 con 0x7fa21c0ff690 2023-09-10T01:28:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.761+0000 7fa220a36700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c196bc0 con 0x7fa21c1055b0 2023-09-10T01:28:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.761+0000 7fa220a36700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c196d80 con 0x7fa21c100050 2023-09-10T01:28:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.761+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa21c0ff690 0x7fa21c19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.761+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa21c0ff690 0x7fa21c19c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60038/0 (socket says 172.21.15.44:60038) 2023-09-10T01:28:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.761+0000 7fa21a59c700 1 -- 172.21.15.44:0/3818571534 learned_addr learned my addr 172.21.15.44:0/3818571534 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.761+0000 7fa21ad9d700 1 --2- 172.21.15.44:0/3818571534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa21c1055b0 0x7fa21c1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.761+0000 7fa213fff700 1 --2- 172.21.15.44:0/3818571534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa21c100050 0x7fa21c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.762+0000 7fa21a59c700 1 -- 172.21.15.44:0/3818571534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa21c100050 msgr2=0x7fa21c19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.762+0000 7fa21a59c700 1 --2- 172.21.15.44:0/3818571534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa21c100050 0x7fa21c19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.762+0000 7fa21a59c700 1 -- 172.21.15.44:0/3818571534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa21c1055b0 msgr2=0x7fa21c1963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.762+0000 7fa21a59c700 1 --2- 172.21.15.44:0/3818571534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa21c1055b0 0x7fa21c1963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.762+0000 7fa21a59c700 1 -- 172.21.15.44:0/3818571534 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c0095f0 con 0x7fa21c0ff690 2023-09-10T01:28:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.762+0000 7fa21ad9d700 1 --2- 172.21.15.44:0/3818571534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa21c1055b0 0x7fa21c1963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:28:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.762+0000 7fa21a59c700 1 --2- 172.21.15.44:0/3818571534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa21c0ff690 0x7fa21c19c580 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fa20c000c00 tx=0x7fa20c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.762+0000 7fa2137fe700 1 -- 172.21.15.44:0/3818571534 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c01d070 con 0x7fa21c0ff690 2023-09-10T01:28:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.762+0000 7fa2137fe700 1 -- 172.21.15.44:0/3818571534 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa20c00fe70 con 0x7fa21c0ff690 2023-09-10T01:28:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.762+0000 7fa220a36700 1 -- 172.21.15.44:0/3818571534 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c197000 con 0x7fa21c0ff690 2023-09-10T01:28:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.762+0000 7fa2137fe700 1 -- 172.21.15.44:0/3818571534 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0173f0 con 0x7fa21c0ff690 2023-09-10T01:28:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.762+0000 7fa220a36700 1 -- 172.21.15.44:0/3818571534 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c1a3570 con 0x7fa21c0ff690 2023-09-10T01:28:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.764+0000 7fa2137fe700 1 -- 172.21.15.44:0/3818571534 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa20c00f9e0 con 0x7fa21c0ff690 2023-09-10T01:28:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.765+0000 7fa2137fe700 1 --2- 172.21.15.44:0/3818571534 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa2080a07b0 0x7fa2080a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.765+0000 7fa2137fe700 1 -- 172.21.15.44:0/3818571534 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa20c0cd440 con 0x7fa21c0ff690 2023-09-10T01:28:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.765+0000 7fa213fff700 1 --2- 172.21.15.44:0/3818571534 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa2080a07b0 0x7fa2080a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.765+0000 7fa213fff700 1 --2- 172.21.15.44:0/3818571534 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa2080a07b0 0x7fa2080a2c70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa21c077ca0 tx=0x7fa204009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa22cbfcb80 1 Processor -- start 2023-09-10T01:28:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa22cbfcb80 1 -- start start 2023-09-10T01:28:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa22cbfcb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd1281a0 0x55d7bd128580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa22cbfcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 0x55d7bd122780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa22cbfcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd122cc0 0x55d7bd12ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa22cbfcb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bd07b6d0 con 0x55d7bd1281a0 2023-09-10T01:28:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa22cbfcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bd078ff0 con 0x55d7bd122300 2023-09-10T01:28:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa22cbfcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bd0748e0 con 0x55d7bd122cc0 2023-09-10T01:28:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd1281a0 0x55d7bd128580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd1281a0 0x55d7bd128580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60046/0 (socket says 172.21.15.44:60046) 2023-09-10T01:28:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa21a59c700 1 -- 172.21.15.44:0/135284222 learned_addr learned my addr 172.21.15.44:0/135284222 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa21ad9d700 1 --2- 172.21.15.44:0/135284222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd122cc0 0x55d7bd12ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa213fff700 1 --2- 172.21.15.44:0/135284222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 0x55d7bd122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa21a59c700 1 -- 172.21.15.44:0/135284222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd122cc0 msgr2=0x55d7bd12ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa21a59c700 1 --2- 172.21.15.44:0/135284222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd122cc0 0x55d7bd12ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa21a59c700 1 -- 172.21.15.44:0/135284222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 msgr2=0x55d7bd122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa21a59c700 1 --2- 172.21.15.44:0/135284222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 0x55d7bd122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.766+0000 7fa21a59c700 1 -- 172.21.15.44:0/135284222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7bcea0d80 con 0x55d7bd1281a0 2023-09-10T01:28:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.767+0000 7fa21a59c700 1 --2- 172.21.15.44:0/135284222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd1281a0 0x55d7bd128580 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fa20c096a80 tx=0x7fa20c099ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f988b17f15a83c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.767+0000 7fa210ff9700 1 -- 172.21.15.44:0/135284222 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c00fc90 con 0x55d7bd1281a0 2023-09-10T01:28:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.767+0000 7fa210ff9700 1 -- 172.21.15.44:0/135284222 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa20c09ac80 con 0x55d7bd1281a0 2023-09-10T01:28:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.767+0000 7fa210ff9700 1 -- 172.21.15.44:0/135284222 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0850f0 con 0x55d7bd1281a0 2023-09-10T01:28:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.767+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/135284222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd1281a0 msgr2=0x55d7bd128580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.767+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/135284222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd1281a0 0x55d7bd128580 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fa20c096a80 tx=0x7fa20c099ee0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/135284222 shutdown_connections 2023-09-10T01:28:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/135284222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd122cc0 0x55d7bd12ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/135284222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd1281a0 0x55d7bd128580 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/135284222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 0x55d7bd122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/135284222 >> 172.21.15.44:0/135284222 conn(0x55d7bd097030 msgr2=0x55d7bd11d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:52.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/135284222 shutdown_connections 2023-09-10T01:28:52.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/135284222 wait complete. 2023-09-10T01:28:52.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa22cbfcb80 1 -- start start 2023-09-10T01:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa22cbfcb80 1 Processor -- start 2023-09-10T01:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa22cbfcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 0x55d7bd131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa22cbfcb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd122cc0 0x55d7bd131fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa22cbfcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd1281a0 0x55d7bd12bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa22cbfcb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bd012b10 con 0x55d7bd122cc0 2023-09-10T01:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa22cbfcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bd012810 con 0x55d7bd122300 2023-09-10T01:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa22cbfcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bcef68c0 con 0x55d7bd1281a0 2023-09-10T01:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa22cbfcb80 10 client.0 _ll_register_callbacks cb 0x55d7bcf5ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa213fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd122cc0 0x55d7bd131fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa213fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd122cc0 0x55d7bd131fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60054/0 (socket says 172.21.15.44:60054) 2023-09-10T01:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa213fff700 1 -- 172.21.15.44:0/1688726524 learned_addr learned my addr 172.21.15.44:0/1688726524 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:52.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.768+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 0x55d7bd131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.769+0000 7fa21ad9d700 1 --2- 172.21.15.44:0/1688726524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd1281a0 0x55d7bd12bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.769+0000 7fa213fff700 1 -- 172.21.15.44:0/1688726524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd1281a0 msgr2=0x55d7bd12bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:52.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.769+0000 7fa213fff700 1 --2- 172.21.15.44:0/1688726524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd1281a0 0x55d7bd12bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.786 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.769+0000 7fa213fff700 1 -- 172.21.15.44:0/1688726524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 msgr2=0x55d7bd131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.769+0000 7fa213fff700 1 --2- 172.21.15.44:0/1688726524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 0x55d7bd131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.769+0000 7fa213fff700 1 -- 172.21.15.44:0/1688726524 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c1a3570 con 0x55d7bd122cc0 2023-09-10T01:28:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.769+0000 7fa21a59c700 1 --2- 172.21.15.44:0/1688726524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 0x55d7bd131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:28:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.769+0000 7fa213fff700 1 --2- 172.21.15.44:0/1688726524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd122cc0 0x55d7bd131fd0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fa204004580 tx=0x7fa2040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65682be99e96de42 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.769+0000 7fa201ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:28:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.769+0000 7fa201ffb700 1 -- 172.21.15.44:0/1688726524 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204022070 con 0x55d7bd122cc0 2023-09-10T01:28:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.769+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/1688726524 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7bcfcc490 con 0x55d7bd122cc0 2023-09-10T01:28:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.769+0000 7fa201ffb700 1 -- 172.21.15.44:0/1688726524 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa20401d410 con 0x55d7bd122cc0 2023-09-10T01:28:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.769+0000 7fa201ffb700 1 -- 172.21.15.44:0/1688726524 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=166) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2040045b0 con 0x55d7bd122cc0 2023-09-10T01:28:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.769+0000 7fa22cbfcb80 10 client.20346 fetch_fsmap learned FSMap version 166 2023-09-10T01:28:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.769+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/1688726524 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=166,monmap=4+}) v3 -- 0x55d7bce89c00 con 0x55d7bd122cc0 2023-09-10T01:28:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.769+0000 7fa201ffb700 1 -- 172.21.15.44:0/1688726524 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d7bcfcc490 con 0x55d7bd122cc0 2023-09-10T01:28:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.770+0000 7fa201ffb700 1 -- 172.21.15.44:0/1688726524 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 166) v1 ==== 7359+0+0 (secure 0 0 0) 0x7fa2040275b0 con 0x55d7bd122cc0 2023-09-10T01:28:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.770+0000 7fa22cbfcb80 10 client.20346 fetch_fsmap finished waiting for FSMap version 166 2023-09-10T01:28:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.770+0000 7fa22cbfcb80 10 client.20346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:28:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.770+0000 7fa22cbfcb80 20 client.20346 populate_metadata read hostname 'smithi044' 2023-09-10T01:28:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.770+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/1688726524 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d7bd13c0b0 0x55d7bd13e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.770+0000 7fa22cbfcb80 4 client.20346 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:28:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.770+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/1688726524 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7bd089840 con 0x55d7bd13c0b0 2023-09-10T01:28:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.770+0000 7fa21a59c700 1 --2- 172.21.15.44:0/1688726524 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d7bd13c0b0 0x55d7bd13e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.770+0000 7fa201ffb700 10 client.20346 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:28:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.770+0000 7fa21a59c700 1 --2- 172.21.15.44:0/1688726524 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d7bd13c0b0 0x55d7bd13e4d0 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df5317d4133a167e server_cookie=4e086b785d6f10f9 in_seq=0 out_seq=0 2023-09-10T01:28:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.771+0000 7fa201ffb700 1 -- 172.21.15.44:0/1688726524 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d7bd089840 con 0x55d7bd13c0b0 2023-09-10T01:28:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.771+0000 7fa201ffb700 10 client.20346 handle_command_reply: tid=0 2023-09-10T01:28:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.771+0000 7fa22cbfcb80 1 client.20346 shutdown 2023-09-10T01:28:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.771+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/1688726524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd122cc0 msgr2=0x55d7bd131fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.771+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/1688726524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd122cc0 0x55d7bd131fd0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fa204004580 tx=0x7fa2040047c0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.771+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/1688726524 shutdown_connections 2023-09-10T01:28:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.771+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/1688726524 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d7bd13c0b0 0x55d7bd13e4d0 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.771+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/1688726524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd1281a0 0x55d7bd12bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.771+0000 7fa201ffb700 0 client.20346 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.771+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/1688726524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 0x55d7bd131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.771+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/1688726524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd122cc0 0x55d7bd131fd0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.771+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/1688726524 >> 172.21.15.44:0/1688726524 conn(0x55d7bd097030 msgr2=0x55d7bd11c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.771+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/1688726524 shutdown_connections 2023-09-10T01:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.771+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/1688726524 wait complete. 2023-09-10T01:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.771+0000 7fa22cbfcb80 20 client.20346 trim_cache size 0 max 16384 2023-09-10T01:28:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa22cbfcb80 1 Processor -- start 2023-09-10T01:28:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa22cbfcb80 1 -- start start 2023-09-10T01:28:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa22cbfcb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd122300 0x55d7bd12bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa22cbfcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd122cc0 0x55d7bd11cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa22cbfcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd1281a0 0x55d7bd11d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa22cbfcb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bd1269a0 con 0x55d7bd122300 2023-09-10T01:28:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa22cbfcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bd119fc0 con 0x55d7bd1281a0 2023-09-10T01:28:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa22cbfcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bd139610 con 0x55d7bd122cc0 2023-09-10T01:28:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd122300 0x55d7bd12bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd122300 0x55d7bd12bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60060/0 (socket says 172.21.15.44:60060) 2023-09-10T01:28:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa21a59c700 1 -- 172.21.15.44:0/4197600332 learned_addr learned my addr 172.21.15.44:0/4197600332 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa21ad9d700 1 --2- 172.21.15.44:0/4197600332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd1281a0 0x55d7bd11d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa213fff700 1 --2- 172.21.15.44:0/4197600332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd122cc0 0x55d7bd11cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa21a59c700 1 -- 172.21.15.44:0/4197600332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd122cc0 msgr2=0x55d7bd11cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa21a59c700 1 --2- 172.21.15.44:0/4197600332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd122cc0 0x55d7bd11cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa21a59c700 1 -- 172.21.15.44:0/4197600332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd1281a0 msgr2=0x55d7bd11d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa21a59c700 1 --2- 172.21.15.44:0/4197600332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd1281a0 0x55d7bd11d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa21a59c700 1 -- 172.21.15.44:0/4197600332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7bcec71f0 con 0x55d7bd122300 2023-09-10T01:28:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa21ad9d700 1 --2- 172.21.15.44:0/4197600332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd1281a0 0x55d7bd11d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:28:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.786+0000 7fa21a59c700 1 --2- 172.21.15.44:0/4197600332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd122300 0x55d7bd12bc40 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0ca840 tx=0x7fa20c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be733f6606c497ad server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.787+0000 7fa2117fa700 1 -- 172.21.15.44:0/4197600332 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c09a580 con 0x55d7bd122300 2023-09-10T01:28:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.787+0000 7fa2117fa700 1 -- 172.21.15.44:0/4197600332 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa20c09e3b0 con 0x55d7bd122300 2023-09-10T01:28:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.787+0000 7fa2117fa700 1 -- 172.21.15.44:0/4197600332 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c022770 con 0x55d7bd122300 2023-09-10T01:28:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.787+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/4197600332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd122300 msgr2=0x55d7bd12bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.787+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/4197600332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd122300 0x55d7bd12bc40 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0ca840 tx=0x7fa20c009310 comp rx=0 tx=0).stop 2023-09-10T01:28:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.787+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/4197600332 shutdown_connections 2023-09-10T01:28:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.787+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/4197600332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd1281a0 0x55d7bd11d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.787+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/4197600332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd122300 0x55d7bd12bc40 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.787+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/4197600332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd122cc0 0x55d7bd11cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.787+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/4197600332 >> 172.21.15.44:0/4197600332 conn(0x55d7bd097030 msgr2=0x55d7bd121b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.787+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/4197600332 shutdown_connections 2023-09-10T01:28:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.787+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/4197600332 wait complete. 2023-09-10T01:28:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.787+0000 7fa22cbfcb80 1 -- start start 2023-09-10T01:28:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.787+0000 7fa22cbfcb80 1 Processor -- start 2023-09-10T01:28:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa22cbfcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 0x55d7bd188260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa22cbfcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd122cc0 0x55d7bd1887a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa22cbfcb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd1281a0 0x55d7bd188ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa22cbfcb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bd1933d0 con 0x55d7bd1281a0 2023-09-10T01:28:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa22cbfcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bd193510 con 0x55d7bd122300 2023-09-10T01:28:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa22cbfcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bd1936d0 con 0x55d7bd122cc0 2023-09-10T01:28:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa22cbfcb80 10 client.0 _ll_register_callbacks cb 0x7fa2040043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:28:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 0x55d7bd188260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa21ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd1281a0 0x55d7bd188ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 0x55d7bd188260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33574/0 (socket says 172.21.15.44:33574) 2023-09-10T01:28:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa21a59c700 1 -- 172.21.15.44:0/2613816792 learned_addr learned my addr 172.21.15.44:0/2613816792 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:28:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa21ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd1281a0 0x55d7bd188ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60070/0 (socket says 172.21.15.44:60070) 2023-09-10T01:28:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa213fff700 1 --2- 172.21.15.44:0/2613816792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd122cc0 0x55d7bd1887a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa21a59c700 1 -- 172.21.15.44:0/2613816792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd122cc0 msgr2=0x55d7bd1887a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa21a59c700 1 --2- 172.21.15.44:0/2613816792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd122cc0 0x55d7bd1887a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa21a59c700 1 -- 172.21.15.44:0/2613816792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd1281a0 msgr2=0x55d7bd188ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:28:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa21a59c700 1 --2- 172.21.15.44:0/2613816792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd1281a0 0x55d7bd188ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa21a59c700 1 -- 172.21.15.44:0/2613816792 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa214007500 con 0x55d7bd122300 2023-09-10T01:28:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa21ad9d700 1 --2- 172.21.15.44:0/2613816792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd1281a0 0x55d7bd188ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:28:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa21a59c700 1 --2- 172.21.15.44:0/2613816792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 0x55d7bd188260 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0988d0 tx=0x7fa20c00fd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa3ca4f8ddf6edcb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:28:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.788+0000 7fa201ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:28:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.789+0000 7fa201ffb700 1 -- 172.21.15.44:0/2613816792 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0a2560 con 0x55d7bd122300 2023-09-10T01:28:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.789+0000 7fa201ffb700 1 -- 172.21.15.44:0/2613816792 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fa20c096040 con 0x55d7bd122300 2023-09-10T01:28:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.789+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/2613816792 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa20c09ac80 con 0x55d7bd122300 2023-09-10T01:28:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.789+0000 7fa201ffb700 1 -- 172.21.15.44:0/2613816792 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=166) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa20c09a580 con 0x55d7bd122300 2023-09-10T01:28:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.789+0000 7fa22cbfcb80 10 client.29975 fetch_fsmap learned FSMap version 166 2023-09-10T01:28:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.789+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/2613816792 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=166,monmap=4+}) v3 -- 0x55d7bcea0d80 con 0x55d7bd122300 2023-09-10T01:28:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.789+0000 7fa201ffb700 1 -- 172.21.15.44:0/2613816792 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa20c09ac80 con 0x55d7bd122300 2023-09-10T01:28:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.789+0000 7fa201ffb700 1 -- 172.21.15.44:0/2613816792 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 166) v1 ==== 7359+0+0 (secure 0 0 0) 0x7fa20c0b8300 con 0x55d7bd122300 2023-09-10T01:28:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.789+0000 7fa22cbfcb80 10 client.29975 fetch_fsmap finished waiting for FSMap version 166 2023-09-10T01:28:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.789+0000 7fa22cbfcb80 10 client.29975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:28:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.789+0000 7fa22cbfcb80 20 client.29975 populate_metadata read hostname 'smithi044' 2023-09-10T01:28:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.789+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/2613816792 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d7bd194980 0x55d7bd196d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:28:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.789+0000 7fa22cbfcb80 4 client.29975 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:28:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.789+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/2613816792 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d7bd06d8c0 con 0x55d7bd194980 2023-09-10T01:28:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.789+0000 7fa213fff700 1 --2- 172.21.15.44:0/2613816792 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d7bd194980 0x55d7bd196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:28:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.790+0000 7fa201ffb700 10 client.29975 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:28:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.790+0000 7fa213fff700 1 --2- 172.21.15.44:0/2613816792 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d7bd194980 0x55d7bd196d70 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c8580308f308013 server_cookie=cce132d5761f54dc in_seq=0 out_seq=0 2023-09-10T01:28:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.820+0000 7fa201ffb700 1 -- 172.21.15.44:0/2613816792 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d7bd06d8c0 con 0x55d7bd194980 2023-09-10T01:28:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.820+0000 7fa201ffb700 10 client.29975 handle_command_reply: tid=0 2023-09-10T01:28:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.820+0000 7fa22cbfcb80 1 client.29975 shutdown 2023-09-10T01:28:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.820+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/2613816792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 msgr2=0x55d7bd188260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.820+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/2613816792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 0x55d7bd188260 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0988d0 tx=0x7fa20c00fd90 comp rx=0 tx=0).stop 2023-09-10T01:28:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.820+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/2613816792 shutdown_connections 2023-09-10T01:28:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.820+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/2613816792 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55d7bd194980 0x55d7bd196d70 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.820+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/2613816792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7bd1281a0 0x55d7bd188ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.820+0000 7fa201ffb700 0 client.29975 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:28:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.820+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/2613816792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7bd122300 0x55d7bd188260 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.820+0000 7fa22cbfcb80 1 --2- 172.21.15.44:0/2613816792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7bd122cc0 0x55d7bd1887a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.820+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/2613816792 >> 172.21.15.44:0/2613816792 conn(0x55d7bd097030 msgr2=0x55d7bd121b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.821+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/2613816792 shutdown_connections 2023-09-10T01:28:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.821+0000 7fa22cbfcb80 1 -- 172.21.15.44:0/2613816792 wait complete. 2023-09-10T01:28:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.821+0000 7fa22cbfcb80 20 client.29975 trim_cache size 0 max 16384 2023-09-10T01:28:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:28:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:28:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:28:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:28:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.821+0000 7fa220a36700 1 -- 172.21.15.44:0/3818571534 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa2080a07b0 msgr2=0x7fa2080a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.821+0000 7fa220a36700 1 --2- 172.21.15.44:0/3818571534 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa2080a07b0 0x7fa2080a2c70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa21c077ca0 tx=0x7fa204009260 comp rx=0 tx=0).stop 2023-09-10T01:28:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.821+0000 7fa220a36700 1 -- 172.21.15.44:0/3818571534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa21c0ff690 msgr2=0x7fa21c19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:28:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.821+0000 7fa220a36700 1 --2- 172.21.15.44:0/3818571534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa21c0ff690 0x7fa21c19c580 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fa20c000c00 tx=0x7fa20c005f10 comp rx=0 tx=0).stop 2023-09-10T01:28:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.821+0000 7fa220a36700 1 -- 172.21.15.44:0/3818571534 shutdown_connections 2023-09-10T01:28:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.821+0000 7fa220a36700 1 --2- 172.21.15.44:0/3818571534 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa2080a07b0 0x7fa2080a2c70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.821+0000 7fa220a36700 1 --2- 172.21.15.44:0/3818571534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa21c1055b0 0x7fa21c1963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.821+0000 7fa220a36700 1 --2- 172.21.15.44:0/3818571534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa21c0ff690 0x7fa21c19c580 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.821+0000 7fa220a36700 1 --2- 172.21.15.44:0/3818571534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa21c100050 0x7fa21c19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:28:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.821+0000 7fa220a36700 1 -- 172.21.15.44:0/3818571534 >> 172.21.15.44:0/3818571534 conn(0x7fa21c077430 msgr2=0x7fa21c068550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:28:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.821+0000 7fa220a36700 1 -- 172.21.15.44:0/3818571534 shutdown_connections 2023-09-10T01:28:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:28:52.821+0000 7fa220a36700 1 -- 172.21.15.44:0/3818571534 wait complete. 2023-09-10T01:28:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:52 smithi131 ceph-mon[103713]: pgmap v1566: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 10 MiB/s wr, 1.05k op/s 2023-09-10T01:28:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:52 smithi044 ceph-mon[98676]: pgmap v1566: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 10 MiB/s wr, 1.05k op/s 2023-09-10T01:28:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:52 smithi099 ceph-mon[102692]: pgmap v1566: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 10 MiB/s wr, 1.05k op/s 2023-09-10T01:28:54.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:53 smithi131 ceph-mon[103713]: pgmap v1567: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.0 MiB/s wr, 707 op/s 2023-09-10T01:28:54.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:53 smithi044 ceph-mon[98676]: pgmap v1567: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.0 MiB/s wr, 707 op/s 2023-09-10T01:28:54.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:53 smithi099 ceph-mon[102692]: pgmap v1567: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.0 MiB/s wr, 707 op/s 2023-09-10T01:28:56.183 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 470.0 s, 74.0 tps, lat 712.887 ms stddev 622.619 2023-09-10T01:28:56.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:56 smithi099 ceph-mon[102692]: pgmap v1568: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.0 MiB/s wr, 707 op/s 2023-09-10T01:28:56.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:56 smithi131 ceph-mon[103713]: pgmap v1568: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.0 MiB/s wr, 707 op/s 2023-09-10T01:28:56.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:56 smithi044 ceph-mon[98676]: pgmap v1568: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.0 MiB/s wr, 707 op/s 2023-09-10T01:28:58.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:28:58 smithi131 ceph-mon[103713]: pgmap v1569: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.5 MiB/s wr, 903 op/s 2023-09-10T01:28:58.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:28:58 smithi044 ceph-mon[98676]: pgmap v1569: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.5 MiB/s wr, 903 op/s 2023-09-10T01:28:58.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:28:58 smithi099 ceph-mon[102692]: pgmap v1569: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.5 MiB/s wr, 903 op/s 2023-09-10T01:29:00.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:00 smithi131 ceph-mon[103713]: pgmap v1570: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 547 op/s 2023-09-10T01:29:00.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:00 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:00.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:00 smithi044 ceph-mon[98676]: pgmap v1570: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 547 op/s 2023-09-10T01:29:00.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:00 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:00.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:00 smithi099 ceph-mon[102692]: pgmap v1570: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 547 op/s 2023-09-10T01:29:00.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:00 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:01.189 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 475.0 s, 63.0 tps, lat 1223.993 ms stddev 1525.273 2023-09-10T01:29:02.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:02 smithi131 ceph-mon[103713]: pgmap v1571: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 682 op/s 2023-09-10T01:29:02.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:02 smithi044 ceph-mon[98676]: pgmap v1571: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 682 op/s 2023-09-10T01:29:02.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:02 smithi099 ceph-mon[102692]: pgmap v1571: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 682 op/s 2023-09-10T01:29:04.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:04 smithi131 ceph-mon[103713]: pgmap v1572: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.8 MiB/s wr, 347 op/s 2023-09-10T01:29:04.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:04 smithi044 ceph-mon[98676]: pgmap v1572: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.8 MiB/s wr, 347 op/s 2023-09-10T01:29:04.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:04 smithi099 ceph-mon[102692]: pgmap v1572: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.8 MiB/s wr, 347 op/s 2023-09-10T01:29:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.948+0000 7f2aef8b6700 1 -- 172.21.15.44:0/4244744844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae8100120 msgr2=0x7f2ae810c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.948+0000 7f2aef8b6700 1 --2- 172.21.15.44:0/4244744844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae8100120 0x7f2ae810c330 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000b0d0 tx=0x7f2ae000b3e0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.948+0000 7f2aef8b6700 1 -- 172.21.15.44:0/4244744844 shutdown_connections 2023-09-10T01:29:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.948+0000 7f2aef8b6700 1 --2- 172.21.15.44:0/4244744844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae8100120 0x7f2ae810c330 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.948+0000 7f2aef8b6700 1 --2- 172.21.15.44:0/4244744844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae81055b0 0x7f2ae8105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.948+0000 7f2aef8b6700 1 --2- 172.21.15.44:0/4244744844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80ff760 0x7f2ae80ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.948+0000 7f2aef8b6700 1 -- 172.21.15.44:0/4244744844 >> 172.21.15.44:0/4244744844 conn(0x7f2ae8074f30 msgr2=0x7f2ae8075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.948+0000 7f2aef8b6700 1 -- 172.21.15.44:0/4244744844 shutdown_connections 2023-09-10T01:29:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.949+0000 7f2aef8b6700 1 -- 172.21.15.44:0/4244744844 wait complete. 2023-09-10T01:29:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.949+0000 7f2aef8b6700 1 Processor -- start 2023-09-10T01:29:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.949+0000 7f2aef8b6700 1 -- start start 2023-09-10T01:29:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.949+0000 7f2aef8b6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80ff760 0x7f2ae819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.949+0000 7f2aef8b6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae8100120 0x7f2ae819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.949+0000 7f2aef8b6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae81055b0 0x7f2ae81964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.949+0000 7f2aef8b6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae8196b40 con 0x7f2ae81055b0 2023-09-10T01:29:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.949+0000 7f2aef8b6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae8196cb0 con 0x7f2ae8100120 2023-09-10T01:29:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.949+0000 7f2aef8b6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae8196e70 con 0x7f2ae80ff760 2023-09-10T01:29:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.950+0000 7f2aed652700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80ff760 0x7f2ae819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.950+0000 7f2aed652700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80ff760 0x7f2ae819c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38730/0 (socket says 172.21.15.44:38730) 2023-09-10T01:29:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.950+0000 7f2aed652700 1 -- 172.21.15.44:0/1567140856 learned_addr learned my addr 172.21.15.44:0/1567140856 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.950+0000 7f2aede53700 1 --2- 172.21.15.44:0/1567140856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae81055b0 0x7f2ae81964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.950+0000 7f2aece51700 1 --2- 172.21.15.44:0/1567140856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae8100120 0x7f2ae819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.950+0000 7f2aed652700 1 -- 172.21.15.44:0/1567140856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae8100120 msgr2=0x7f2ae819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.950+0000 7f2aed652700 1 --2- 172.21.15.44:0/1567140856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae8100120 0x7f2ae819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.950+0000 7f2aed652700 1 -- 172.21.15.44:0/1567140856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae81055b0 msgr2=0x7f2ae81964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.950+0000 7f2aed652700 1 --2- 172.21.15.44:0/1567140856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae81055b0 0x7f2ae81964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.950+0000 7f2aed652700 1 -- 172.21.15.44:0/1567140856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae0009d00 con 0x7f2ae80ff760 2023-09-10T01:29:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.950+0000 7f2aede53700 1 --2- 172.21.15.44:0/1567140856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae81055b0 0x7f2ae81964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:29:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.950+0000 7f2aed652700 1 --2- 172.21.15.44:0/1567140856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80ff760 0x7f2ae819c670 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400d950 tx=0x7f2ae400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.950+0000 7f2ade7fc700 1 -- 172.21.15.44:0/1567140856 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae400f890 con 0x7f2ae80ff760 2023-09-10T01:29:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.951+0000 7f2aef8b6700 1 -- 172.21.15.44:0/1567140856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae8197150 con 0x7f2ae80ff760 2023-09-10T01:29:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.951+0000 7f2ade7fc700 1 -- 172.21.15.44:0/1567140856 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ae400fe80 con 0x7f2ae80ff760 2023-09-10T01:29:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.951+0000 7f2ade7fc700 1 -- 172.21.15.44:0/1567140856 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae400e3b0 con 0x7f2ae80ff760 2023-09-10T01:29:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.951+0000 7f2aef8b6700 1 -- 172.21.15.44:0/1567140856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae81a3660 con 0x7f2ae80ff760 2023-09-10T01:29:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.953+0000 7f2ade7fc700 1 -- 172.21.15.44:0/1567140856 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f2ae4010460 con 0x7f2ae80ff760 2023-09-10T01:29:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.954+0000 7f2ade7fc700 1 --2- 172.21.15.44:0/1567140856 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2ad40a07b0 0x7f2ad40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.954+0000 7f2ade7fc700 1 -- 172.21.15.44:0/1567140856 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2ae40cc140 con 0x7f2ae80ff760 2023-09-10T01:29:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.954+0000 7f2afba7cb80 1 Processor -- start 2023-09-10T01:29:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.954+0000 7f2afba7cb80 1 -- start start 2023-09-10T01:29:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.954+0000 7f2afba7cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.954+0000 7f2afba7cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f415300 0x55594f415780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.954+0000 7f2afba7cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415cc0 0x55594f421f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.954+0000 7f2afba7cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594f36e6d0 con 0x55594f41b1a0 2023-09-10T01:29:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.954+0000 7f2afba7cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594f36bff0 con 0x55594f415cc0 2023-09-10T01:29:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.954+0000 7f2afba7cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594f3678e0 con 0x55594f415300 2023-09-10T01:29:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2aed652700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2aed652700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60088/0 (socket says 172.21.15.44:60088) 2023-09-10T01:29:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2aed652700 1 -- 172.21.15.44:0/1954268242 learned_addr learned my addr 172.21.15.44:0/1954268242 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2aece51700 1 --2- 172.21.15.44:0/1567140856 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2ad40a07b0 0x7f2ad40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2aede53700 1 --2- 172.21.15.44:0/1954268242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415cc0 0x55594f421f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2aece51700 1 --2- 172.21.15.44:0/1954268242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f415300 0x55594f415780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2aed652700 1 -- 172.21.15.44:0/1954268242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f415300 msgr2=0x55594f415780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2aed652700 1 --2- 172.21.15.44:0/1954268242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f415300 0x55594f415780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2aed652700 1 -- 172.21.15.44:0/1954268242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415cc0 msgr2=0x55594f421f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2aed652700 1 --2- 172.21.15.44:0/1954268242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415cc0 0x55594f421f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2aed652700 1 -- 172.21.15.44:0/1954268242 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55594f193d80 con 0x55594f41b1a0 2023-09-10T01:29:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2aece51700 1 --2- 172.21.15.44:0/1954268242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f415300 0x55594f415780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:29:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2aed652700 1 --2- 172.21.15.44:0/1954268242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41b580 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40a0d30 tx=0x7f2ae4010760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c8868360ce9e01 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2ad37fe700 1 -- 172.21.15.44:0/1954268242 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae409d020 con 0x55594f41b1a0 2023-09-10T01:29:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2aece51700 1 --2- 172.21.15.44:0/1567140856 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2ad40a07b0 0x7f2ad40a2c70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55594f410060 tx=0x7f2ad80099b0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2ad37fe700 1 -- 172.21.15.44:0/1954268242 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ae40a1260 con 0x55594f41b1a0 2023-09-10T01:29:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.955+0000 7f2ad37fe700 1 -- 172.21.15.44:0/1954268242 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40996b0 con 0x55594f41b1a0 2023-09-10T01:29:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.956+0000 7f2afba7cb80 1 -- 172.21.15.44:0/1954268242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 msgr2=0x55594f41b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.956+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/1954268242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41b580 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40a0d30 tx=0x7f2ae4010760 comp rx=0 tx=0).stop 2023-09-10T01:29:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.956+0000 7f2afba7cb80 1 -- 172.21.15.44:0/1954268242 shutdown_connections 2023-09-10T01:29:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.956+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/1954268242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415cc0 0x55594f421f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.956+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/1954268242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41b580 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.956+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/1954268242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f415300 0x55594f415780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.956+0000 7f2afba7cb80 1 -- 172.21.15.44:0/1954268242 >> 172.21.15.44:0/1954268242 conn(0x55594f38a030 msgr2=0x55594f410560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.956+0000 7f2afba7cb80 1 -- 172.21.15.44:0/1954268242 shutdown_connections 2023-09-10T01:29:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.956+0000 7f2afba7cb80 1 -- 172.21.15.44:0/1954268242 wait complete. 2023-09-10T01:29:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.956+0000 7f2afba7cb80 1 -- start start 2023-09-10T01:29:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.956+0000 7f2afba7cb80 1 Processor -- start 2023-09-10T01:29:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2afba7cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415300 0x55594f421e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2afba7cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f415cc0 0x55594f41f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2afba7cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2afba7cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594f305b10 con 0x55594f41b1a0 2023-09-10T01:29:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2afba7cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594f305810 con 0x55594f415300 2023-09-10T01:29:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2afba7cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594f1e98c0 con 0x55594f415cc0 2023-09-10T01:29:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2afba7cb80 10 client.0 _ll_register_callbacks cb 0x55594f24fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:29:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2aede53700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2aece51700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f415cc0 0x55594f41f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2aede53700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60098/0 (socket says 172.21.15.44:60098) 2023-09-10T01:29:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2aede53700 1 -- 172.21.15.44:0/2878213541 learned_addr learned my addr 172.21.15.44:0/2878213541 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2aed652700 1 --2- 172.21.15.44:0/2878213541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415300 0x55594f421e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2aede53700 1 -- 172.21.15.44:0/2878213541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f415cc0 msgr2=0x55594f41f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2aede53700 1 --2- 172.21.15.44:0/2878213541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f415cc0 0x55594f41f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2aede53700 1 -- 172.21.15.44:0/2878213541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415300 msgr2=0x55594f421e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2aede53700 1 --2- 172.21.15.44:0/2878213541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415300 0x55594f421e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2aede53700 1 -- 172.21.15.44:0/2878213541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae81a3660 con 0x55594f41b1a0 2023-09-10T01:29:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2aede53700 1 --2- 172.21.15.44:0/2878213541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41f690 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0008f10 tx=0x7f2ae0008f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b872fb4427d95fa server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.957+0000 7f2ad0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:29:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.958+0000 7f2ad0ff9700 1 -- 172.21.15.44:0/2878213541 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae000e040 con 0x55594f41b1a0 2023-09-10T01:29:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.958+0000 7f2ad0ff9700 1 -- 172.21.15.44:0/2878213541 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2ae0003a00 con 0x55594f41b1a0 2023-09-10T01:29:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.958+0000 7f2afba7cb80 1 -- 172.21.15.44:0/2878213541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55594f2bf490 con 0x55594f41b1a0 2023-09-10T01:29:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.958+0000 7f2ad0ff9700 1 -- 172.21.15.44:0/2878213541 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=167) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ae0003bc0 con 0x55594f41b1a0 2023-09-10T01:29:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.958+0000 7f2afba7cb80 10 client.20364 fetch_fsmap learned FSMap version 167 2023-09-10T01:29:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.958+0000 7f2afba7cb80 1 -- 172.21.15.44:0/2878213541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=167,monmap=4+}) v3 -- 0x55594f17cc00 con 0x55594f41b1a0 2023-09-10T01:29:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.958+0000 7f2ad0ff9700 1 -- 172.21.15.44:0/2878213541 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55594f2bf490 con 0x55594f41b1a0 2023-09-10T01:29:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.958+0000 7f2ad0ff9700 1 -- 172.21.15.44:0/2878213541 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 167) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f2ae000b100 con 0x55594f41b1a0 2023-09-10T01:29:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.958+0000 7f2afba7cb80 10 client.20364 fetch_fsmap finished waiting for FSMap version 167 2023-09-10T01:29:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.958+0000 7f2afba7cb80 10 client.20364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:29:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.958+0000 7f2afba7cb80 20 client.20364 populate_metadata read hostname 'smithi044' 2023-09-10T01:29:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.958+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/2878213541 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55594f42f430 0x55594f431850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.958+0000 7f2afba7cb80 4 client.20364 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:29:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.958+0000 7f2afba7cb80 1 -- 172.21.15.44:0/2878213541 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55594f37c840 con 0x55594f42f430 2023-09-10T01:29:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.958+0000 7f2aed652700 1 --2- 172.21.15.44:0/2878213541 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55594f42f430 0x55594f431850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.959+0000 7f2aed652700 1 --2- 172.21.15.44:0/2878213541 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55594f42f430 0x55594f431850 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=704770b403546b72 server_cookie=de0f4df585549dbb in_seq=0 out_seq=0 2023-09-10T01:29:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.959+0000 7f2ad0ff9700 10 client.20364 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.959+0000 7f2ad0ff9700 1 -- 172.21.15.44:0/2878213541 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55594f37c840 con 0x55594f42f430 2023-09-10T01:29:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.959+0000 7f2ad0ff9700 10 client.20364 handle_command_reply: tid=0 2023-09-10T01:29:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.959+0000 7f2afba7cb80 1 client.20364 shutdown 2023-09-10T01:29:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.960+0000 7f2afba7cb80 1 -- 172.21.15.44:0/2878213541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 msgr2=0x55594f41f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.960+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/2878213541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41f690 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0008f10 tx=0x7f2ae0008f40 comp rx=0 tx=0).stop 2023-09-10T01:29:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.960+0000 7f2afba7cb80 1 -- 172.21.15.44:0/2878213541 shutdown_connections 2023-09-10T01:29:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.960+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/2878213541 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55594f42f430 0x55594f431850 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.960+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/2878213541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41f690 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.960+0000 7f2ad0ff9700 0 client.20364 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.960+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/2878213541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415300 0x55594f421e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.960+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/2878213541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f415cc0 0x55594f41f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.960+0000 7f2afba7cb80 1 -- 172.21.15.44:0/2878213541 >> 172.21.15.44:0/2878213541 conn(0x55594f38a030 msgr2=0x55594f40f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.960+0000 7f2afba7cb80 1 -- 172.21.15.44:0/2878213541 shutdown_connections 2023-09-10T01:29:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.960+0000 7f2afba7cb80 1 -- 172.21.15.44:0/2878213541 wait complete. 2023-09-10T01:29:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.960+0000 7f2afba7cb80 20 client.20364 trim_cache size 0 max 16384 2023-09-10T01:29:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.974+0000 7f2afba7cb80 1 Processor -- start 2023-09-10T01:29:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.974+0000 7f2afba7cb80 1 -- start start 2023-09-10T01:29:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2afba7cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415300 0x55594f4219a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2afba7cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f415cc0 0x55594f41f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2afba7cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2afba7cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594f411380 con 0x55594f41b1a0 2023-09-10T01:29:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2afba7cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594f40cfc0 con 0x55594f415300 2023-09-10T01:29:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2afba7cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594f42c9e0 con 0x55594f415cc0 2023-09-10T01:29:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2aede53700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2aede53700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60106/0 (socket says 172.21.15.44:60106) 2023-09-10T01:29:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2aede53700 1 -- 172.21.15.44:0/3707837783 learned_addr learned my addr 172.21.15.44:0/3707837783 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2aece51700 1 --2- 172.21.15.44:0/3707837783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f415cc0 0x55594f41f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2aed652700 1 --2- 172.21.15.44:0/3707837783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415300 0x55594f4219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2aede53700 1 -- 172.21.15.44:0/3707837783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f415cc0 msgr2=0x55594f41f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2aede53700 1 --2- 172.21.15.44:0/3707837783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f415cc0 0x55594f41f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2aede53700 1 -- 172.21.15.44:0/3707837783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415300 msgr2=0x55594f4219a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2aede53700 1 --2- 172.21.15.44:0/3707837783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415300 0x55594f4219a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2aede53700 1 -- 172.21.15.44:0/3707837783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55594f1ba1f0 con 0x55594f41b1a0 2023-09-10T01:29:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2aed652700 1 --2- 172.21.15.44:0/3707837783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415300 0x55594f4219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:29:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2aede53700 1 --2- 172.21.15.44:0/3707837783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41f580 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0009850 tx=0x7f2ae0005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14034e937a2a8751 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.975+0000 7f2ad3fff700 1 -- 172.21.15.44:0/3707837783 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae000e070 con 0x55594f41b1a0 2023-09-10T01:29:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.976+0000 7f2ad3fff700 1 -- 172.21.15.44:0/3707837783 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ae0003e90 con 0x55594f41b1a0 2023-09-10T01:29:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.976+0000 7f2ad3fff700 1 -- 172.21.15.44:0/3707837783 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae000b1e0 con 0x55594f41b1a0 2023-09-10T01:29:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.976+0000 7f2afba7cb80 1 -- 172.21.15.44:0/3707837783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 msgr2=0x55594f41f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.976+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/3707837783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41f580 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0009850 tx=0x7f2ae0005b40 comp rx=0 tx=0).stop 2023-09-10T01:29:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.976+0000 7f2afba7cb80 1 -- 172.21.15.44:0/3707837783 shutdown_connections 2023-09-10T01:29:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.976+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/3707837783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f41b1a0 0x55594f41f580 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.976+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/3707837783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415300 0x55594f4219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.976+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/3707837783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f415cc0 0x55594f41f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.976+0000 7f2afba7cb80 1 -- 172.21.15.44:0/3707837783 >> 172.21.15.44:0/3707837783 conn(0x55594f38a030 msgr2=0x55594f41a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.976+0000 7f2afba7cb80 1 -- 172.21.15.44:0/3707837783 shutdown_connections 2023-09-10T01:29:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.976+0000 7f2afba7cb80 1 -- 172.21.15.44:0/3707837783 wait complete. 2023-09-10T01:29:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.976+0000 7f2afba7cb80 1 -- start start 2023-09-10T01:29:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2afba7cb80 1 Processor -- start 2023-09-10T01:29:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2afba7cb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f415300 0x55594f424510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2afba7cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415cc0 0x55594f424a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2afba7cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f41b1a0 0x55594f424f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2afba7cb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594f482260 con 0x55594f415300 2023-09-10T01:29:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2afba7cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594f4823a0 con 0x55594f415cc0 2023-09-10T01:29:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2afba7cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594f4824e0 con 0x55594f41b1a0 2023-09-10T01:29:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2afba7cb80 10 client.0 _ll_register_callbacks cb 0x7f2ae0009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:29:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2aed652700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f415300 0x55594f424510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2aed652700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f415300 0x55594f424510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60108/0 (socket says 172.21.15.44:60108) 2023-09-10T01:29:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2aece51700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415cc0 0x55594f424a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2aed652700 1 -- 172.21.15.44:0/2918374775 learned_addr learned my addr 172.21.15.44:0/2918374775 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2aede53700 1 --2- 172.21.15.44:0/2918374775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f41b1a0 0x55594f424f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2aed652700 1 -- 172.21.15.44:0/2918374775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f41b1a0 msgr2=0x55594f424f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2aed652700 1 --2- 172.21.15.44:0/2918374775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f41b1a0 0x55594f424f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2aed652700 1 -- 172.21.15.44:0/2918374775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415cc0 msgr2=0x55594f424a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2aed652700 1 --2- 172.21.15.44:0/2918374775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415cc0 0x55594f424a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2aed652700 1 -- 172.21.15.44:0/2918374775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55594f17cc00 con 0x55594f415300 2023-09-10T01:29:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2aede53700 1 --2- 172.21.15.44:0/2918374775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f41b1a0 0x55594f424f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:29:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.977+0000 7f2aece51700 1 --2- 172.21.15.44:0/2918374775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415cc0 0x55594f424a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:29:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2aed652700 1 --2- 172.21.15.44:0/2918374775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f415300 0x55594f424510 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400b2f0 tx=0x7f2ae4010880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a81b7a36a9aec7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2ad0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:29:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2ad0ff9700 1 -- 172.21.15.44:0/2918374775 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4097690 con 0x55594f415300 2023-09-10T01:29:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2ad0ff9700 1 -- 172.21.15.44:0/2918374775 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2ae40c9df0 con 0x55594f415300 2023-09-10T01:29:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2afba7cb80 1 -- 172.21.15.44:0/2918374775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ae40a1260 con 0x55594f415300 2023-09-10T01:29:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2ad0ff9700 1 -- 172.21.15.44:0/2918374775 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=167) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ae40b71b0 con 0x55594f415300 2023-09-10T01:29:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2afba7cb80 10 client.20370 fetch_fsmap learned FSMap version 167 2023-09-10T01:29:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2afba7cb80 1 -- 172.21.15.44:0/2918374775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=167,monmap=4+}) v3 -- 0x55594f192250 con 0x55594f415300 2023-09-10T01:29:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2ad0ff9700 1 -- 172.21.15.44:0/2918374775 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ae40a1260 con 0x55594f415300 2023-09-10T01:29:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2ad0ff9700 1 -- 172.21.15.44:0/2918374775 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 167) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f2ae40a0dd0 con 0x55594f415300 2023-09-10T01:29:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2afba7cb80 10 client.20370 fetch_fsmap finished waiting for FSMap version 167 2023-09-10T01:29:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2afba7cb80 10 client.20370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:29:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2afba7cb80 20 client.20370 populate_metadata read hostname 'smithi044' 2023-09-10T01:29:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/2918374775 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55594f42f430 0x55594f4858d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2afba7cb80 4 client.20370 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:29:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2afba7cb80 1 -- 172.21.15.44:0/2918374775 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55594f3608c0 con 0x55594f42f430 2023-09-10T01:29:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.978+0000 7f2aece51700 1 --2- 172.21.15.44:0/2918374775 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55594f42f430 0x55594f4858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.979+0000 7f2ad0ff9700 10 client.20370 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:05.979+0000 7f2aece51700 1 --2- 172.21.15.44:0/2918374775 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55594f42f430 0x55594f4858d0 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=654765dc4fc4b5b9 server_cookie=e1c48c5356a2c2b5 in_seq=0 out_seq=0 2023-09-10T01:29:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.002+0000 7f2ad0ff9700 1 -- 172.21.15.44:0/2918374775 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55594f3608c0 con 0x55594f42f430 2023-09-10T01:29:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2ad0ff9700 10 client.20370 handle_command_reply: tid=0 2023-09-10T01:29:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2afba7cb80 1 client.20370 shutdown 2023-09-10T01:29:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2afba7cb80 1 -- 172.21.15.44:0/2918374775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f415300 msgr2=0x55594f424510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/2918374775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f415300 0x55594f424510 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400b2f0 tx=0x7f2ae4010880 comp rx=0 tx=0).stop 2023-09-10T01:29:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2afba7cb80 1 -- 172.21.15.44:0/2918374775 shutdown_connections 2023-09-10T01:29:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/2918374775 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55594f42f430 0x55594f4858d0 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/2918374775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55594f41b1a0 0x55594f424f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2ad0ff9700 0 client.20370 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/2918374775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55594f415300 0x55594f424510 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2afba7cb80 1 --2- 172.21.15.44:0/2918374775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55594f415cc0 0x55594f424a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2afba7cb80 1 -- 172.21.15.44:0/2918374775 >> 172.21.15.44:0/2918374775 conn(0x55594f38a030 msgr2=0x55594f410090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2afba7cb80 1 -- 172.21.15.44:0/2918374775 shutdown_connections 2023-09-10T01:29:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2afba7cb80 1 -- 172.21.15.44:0/2918374775 wait complete. 2023-09-10T01:29:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2afba7cb80 20 client.20370 trim_cache size 0 max 16384 2023-09-10T01:29:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:29:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:29:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:29:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:29:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2aef8b6700 1 -- 172.21.15.44:0/1567140856 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2ad40a07b0 msgr2=0x7f2ad40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2aef8b6700 1 --2- 172.21.15.44:0/1567140856 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2ad40a07b0 0x7f2ad40a2c70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55594f410060 tx=0x7f2ad80099b0 comp rx=0 tx=0).stop 2023-09-10T01:29:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2aef8b6700 1 -- 172.21.15.44:0/1567140856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80ff760 msgr2=0x7f2ae819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.003+0000 7f2aef8b6700 1 --2- 172.21.15.44:0/1567140856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80ff760 0x7f2ae819c670 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400d950 tx=0x7f2ae400dc60 comp rx=0 tx=0).stop 2023-09-10T01:29:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.004+0000 7f2aef8b6700 1 -- 172.21.15.44:0/1567140856 shutdown_connections 2023-09-10T01:29:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.004+0000 7f2aef8b6700 1 --2- 172.21.15.44:0/1567140856 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2ad40a07b0 0x7f2ad40a2c70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.004+0000 7f2aef8b6700 1 --2- 172.21.15.44:0/1567140856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae81055b0 0x7f2ae81964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.004+0000 7f2aef8b6700 1 --2- 172.21.15.44:0/1567140856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80ff760 0x7f2ae819c670 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.004+0000 7f2aef8b6700 1 --2- 172.21.15.44:0/1567140856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae8100120 0x7f2ae819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.004+0000 7f2aef8b6700 1 -- 172.21.15.44:0/1567140856 >> 172.21.15.44:0/1567140856 conn(0x7f2ae8074f30 msgr2=0x7f2ae8067b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.004+0000 7f2aef8b6700 1 -- 172.21.15.44:0/1567140856 shutdown_connections 2023-09-10T01:29:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:06.004+0000 7f2aef8b6700 1 -- 172.21.15.44:0/1567140856 wait complete. 2023-09-10T01:29:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 480.0 s, 27.4 tps, lat 5856.343 ms stddev 2013.982 2023-09-10T01:29:06.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:06 smithi131 ceph-mon[103713]: pgmap v1573: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.8 MiB/s wr, 347 op/s 2023-09-10T01:29:06.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:06 smithi044 ceph-mon[98676]: pgmap v1573: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.8 MiB/s wr, 347 op/s 2023-09-10T01:29:06.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:06 smithi099 ceph-mon[102692]: pgmap v1573: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.8 MiB/s wr, 347 op/s 2023-09-10T01:29:08.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:08 smithi131 ceph-mon[103713]: pgmap v1574: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 4.0 MiB/s wr, 468 op/s 2023-09-10T01:29:08.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:08 smithi044 ceph-mon[98676]: pgmap v1574: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 4.0 MiB/s wr, 468 op/s 2023-09-10T01:29:08.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:08 smithi099 ceph-mon[102692]: pgmap v1574: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 4.0 MiB/s wr, 468 op/s 2023-09-10T01:29:09.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:09 smithi131 ceph-mon[103713]: osdmap e191: 12 total, 12 up, 12 in 2023-09-10T01:29:09.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:09 smithi044 ceph-mon[98676]: osdmap e191: 12 total, 12 up, 12 in 2023-09-10T01:29:09.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:09 smithi099 ceph-mon[102692]: osdmap e191: 12 total, 12 up, 12 in 2023-09-10T01:29:10.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:10 smithi099 ceph-mon[102692]: pgmap v1576: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 3.0 MiB/s wr, 327 op/s 2023-09-10T01:29:10.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:10 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:10 smithi131 ceph-mon[103713]: pgmap v1576: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 3.0 MiB/s wr, 327 op/s 2023-09-10T01:29:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:10 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:11.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:10 smithi044 ceph-mon[98676]: pgmap v1576: 129 pgs: 129 active+clean; 18 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 3.0 MiB/s wr, 327 op/s 2023-09-10T01:29:11.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:10 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:11.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 485.0 s, 279.4 tps, lat 341.489 ms stddev 634.049 2023-09-10T01:29:12.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:12 smithi099 ceph-mon[102692]: pgmap v1577: 129 pgs: 16 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 71 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 355 op/s 2023-09-10T01:29:13.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:12 smithi131 ceph-mon[103713]: pgmap v1577: 129 pgs: 16 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 71 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 355 op/s 2023-09-10T01:29:13.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:12 smithi044 ceph-mon[98676]: pgmap v1577: 129 pgs: 16 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 71 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 355 op/s 2023-09-10T01:29:14.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:14 smithi099 ceph-mon[102692]: pgmap v1578: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 376 op/s 2023-09-10T01:29:14.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:14 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:14 smithi131 ceph-mon[103713]: pgmap v1578: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 376 op/s 2023-09-10T01:29:15.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:14 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:15.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:14 smithi044 ceph-mon[98676]: pgmap v1578: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 376 op/s 2023-09-10T01:29:15.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:14 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:15 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:16.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:15 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:16.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:15 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:16.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 490.0 s, 111.4 tps, lat 1019.539 ms stddev 615.531 2023-09-10T01:29:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:16 smithi131 ceph-mon[103713]: pgmap v1579: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 376 op/s 2023-09-10T01:29:17.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:16 smithi044 ceph-mon[98676]: pgmap v1579: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 376 op/s 2023-09-10T01:29:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:16 smithi099 ceph-mon[102692]: pgmap v1579: 129 pgs: 20 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 59 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 376 op/s 2023-09-10T01:29:18.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:17 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:29:18.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:17 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:29:18.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:17 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:29:19.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:18 smithi131 ceph-mon[103713]: pgmap v1580: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 368 op/s 2023-09-10T01:29:19.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:18 smithi044 ceph-mon[98676]: pgmap v1580: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 368 op/s 2023-09-10T01:29:19.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:18 smithi099 ceph-mon[102692]: pgmap v1580: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 368 op/s 2023-09-10T01:29:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.245+0000 7f5b3f795700 1 -- 172.21.15.44:0/402250744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b3810f1c0 msgr2=0x7f5b38107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.245+0000 7f5b3f795700 1 --2- 172.21.15.44:0/402250744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b3810f1c0 0x7f5b38107b40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5b3001c0d0 tx=0x7f5b3001c3e0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.246+0000 7f5b3f795700 1 -- 172.21.15.44:0/402250744 shutdown_connections 2023-09-10T01:29:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.246+0000 7f5b3f795700 1 --2- 172.21.15.44:0/402250744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b38119de0 0x7f5b3811c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.246+0000 7f5b3f795700 1 --2- 172.21.15.44:0/402250744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b3810ed50 0x7f5b38107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.246+0000 7f5b3f795700 1 --2- 172.21.15.44:0/402250744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b3810f1c0 0x7f5b38107b40 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.246+0000 7f5b3f795700 1 -- 172.21.15.44:0/402250744 >> 172.21.15.44:0/402250744 conn(0x7f5b3806c6c0 msgr2=0x7f5b3806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.246+0000 7f5b3f795700 1 -- 172.21.15.44:0/402250744 shutdown_connections 2023-09-10T01:29:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.246+0000 7f5b3f795700 1 -- 172.21.15.44:0/402250744 wait complete. 2023-09-10T01:29:19.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.249+0000 7f5b3f795700 1 Processor -- start 2023-09-10T01:29:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.257+0000 7f5b3f795700 1 -- start start 2023-09-10T01:29:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.257+0000 7f5b3f795700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b3810ed50 0x7f5b38119070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.257+0000 7f5b3f795700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b3810f1c0 0x7f5b381195b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.257+0000 7f5b3f795700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b38119de0 0x7f5b38114f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.257+0000 7f5b3f795700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b381156b0 con 0x7f5b3810f1c0 2023-09-10T01:29:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.257+0000 7f5b3f795700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b38115820 con 0x7f5b38119de0 2023-09-10T01:29:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.257+0000 7f5b3f795700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b381159e0 con 0x7f5b3810ed50 2023-09-10T01:29:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.257+0000 7f5b3d531700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b3810ed50 0x7f5b38119070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.257+0000 7f5b3dd32700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b38119de0 0x7f5b38114f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.257+0000 7f5b3dd32700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b38119de0 0x7f5b38114f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33632/0 (socket says 172.21.15.44:33632) 2023-09-10T01:29:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.257+0000 7f5b3dd32700 1 -- 172.21.15.44:0/1576413149 learned_addr learned my addr 172.21.15.44:0/1576413149 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.257+0000 7f5b3d531700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b3810ed50 0x7f5b38119070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38772/0 (socket says 172.21.15.44:38772) 2023-09-10T01:29:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.258+0000 7f5b3dd32700 1 -- 172.21.15.44:0/1576413149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b3810ed50 msgr2=0x7f5b38119070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.258+0000 7f5b3dd32700 1 --2- 172.21.15.44:0/1576413149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b3810ed50 0x7f5b38119070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.258+0000 7f5b3dd32700 1 -- 172.21.15.44:0/1576413149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b3810f1c0 msgr2=0x7f5b381195b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T01:29:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.258+0000 7f5b3dd32700 1 --2- 172.21.15.44:0/1576413149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b3810f1c0 0x7f5b381195b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.258+0000 7f5b3dd32700 1 -- 172.21.15.44:0/1576413149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3001ad10 con 0x7f5b38119de0 2023-09-10T01:29:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.258+0000 7f5b3dd32700 1 --2- 172.21.15.44:0/1576413149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b38119de0 0x7f5b38114f10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400e970 tx=0x7f5b3400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.259+0000 7f5b2e7fc700 1 -- 172.21.15.44:0/1576413149 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400cbd0 con 0x7f5b38119de0 2023-09-10T01:29:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.259+0000 7f5b2e7fc700 1 -- 172.21.15.44:0/1576413149 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b3400ee60 con 0x7f5b38119de0 2023-09-10T01:29:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.259+0000 7f5b2e7fc700 1 -- 172.21.15.44:0/1576413149 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b340054c0 con 0x7f5b38119de0 2023-09-10T01:29:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.259+0000 7f5b3f795700 1 -- 172.21.15.44:0/1576413149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b38115cc0 con 0x7f5b38119de0 2023-09-10T01:29:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.259+0000 7f5b3f795700 1 -- 172.21.15.44:0/1576413149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b381a9cf0 con 0x7f5b38119de0 2023-09-10T01:29:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.265+0000 7f5b4b95bb80 1 Processor -- start 2023-09-10T01:29:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.265+0000 7f5b4b95bb80 1 -- start start 2023-09-10T01:29:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.266+0000 7f5b4b95bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 0x562ac110c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.266+0000 7f5b4b95bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106300 0x562ac1106780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.266+0000 7f5b4b95bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac1106cc0 0x562ac1112f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.266+0000 7f5b4b95bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac105f6d0 con 0x562ac1106cc0 2023-09-10T01:29:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.266+0000 7f5b4b95bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac105cff0 con 0x562ac110c1a0 2023-09-10T01:29:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.266+0000 7f5b4b95bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac10588e0 con 0x562ac1106300 2023-09-10T01:29:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.273+0000 7f5b3d531700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106300 0x562ac1106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.275+0000 7f5b3d531700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106300 0x562ac1106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38776/0 (socket says 172.21.15.44:38776) 2023-09-10T01:29:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.275+0000 7f5b3d531700 1 -- 172.21.15.44:0/1538344731 learned_addr learned my addr 172.21.15.44:0/1538344731 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.275+0000 7f5b2e7fc700 1 -- 172.21.15.44:0/1576413149 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f5b34005620 con 0x7f5b38119de0 2023-09-10T01:29:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.275+0000 7f5b3cd30700 1 --2- 172.21.15.44:0/1538344731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 0x562ac110c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.275+0000 7f5b3cd30700 1 --2- 172.21.15.44:0/1538344731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac1106cc0 0x562ac1112f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.275+0000 7f5b3cd30700 1 -- 172.21.15.44:0/1538344731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106300 msgr2=0x562ac1106780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.275+0000 7f5b3cd30700 1 --2- 172.21.15.44:0/1538344731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106300 0x562ac1106780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.275+0000 7f5b3cd30700 1 -- 172.21.15.44:0/1538344731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac1106cc0 msgr2=0x562ac1112f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.275+0000 7f5b3cd30700 1 --2- 172.21.15.44:0/1538344731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac1106cc0 0x562ac1112f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.275+0000 7f5b3cd30700 1 -- 172.21.15.44:0/1538344731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ac0e84d80 con 0x562ac110c1a0 2023-09-10T01:29:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.276+0000 7f5b3cd30700 1 --2- 172.21.15.44:0/1538344731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 0x562ac110c580 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5b38116910 tx=0x7f5b3001a7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f57853e9880e19 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.276+0000 7f5b2e7fc700 1 --2- 172.21.15.44:0/1576413149 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5b240c2990 0x7f5b240c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.276+0000 7f5b2e7fc700 1 -- 172.21.15.44:0/1576413149 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f5b34014070 con 0x7f5b38119de0 2023-09-10T01:29:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.276+0000 7f5b3d531700 1 --2- 172.21.15.44:0/1576413149 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5b240c2990 0x7f5b240c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.277+0000 7f5b237fe700 1 -- 172.21.15.44:0/1538344731 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3001f030 con 0x562ac110c1a0 2023-09-10T01:29:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.277+0000 7f5b237fe700 1 -- 172.21.15.44:0/1538344731 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b30006040 con 0x562ac110c1a0 2023-09-10T01:29:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.277+0000 7f5b237fe700 1 -- 172.21.15.44:0/1538344731 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b30003e20 con 0x562ac110c1a0 2023-09-10T01:29:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.284+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/1538344731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 msgr2=0x562ac110c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.284+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/1538344731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 0x562ac110c580 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5b38116910 tx=0x7f5b3001a7d0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.285+0000 7f5b3d531700 1 --2- 172.21.15.44:0/1576413149 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5b240c2990 0x7f5b240c4e50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x562ac1101060 tx=0x7f5b28009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.293+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/1538344731 shutdown_connections 2023-09-10T01:29:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.293+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/1538344731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac1106cc0 0x562ac1112f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.293+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/1538344731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 0x562ac110c580 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.293+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/1538344731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106300 0x562ac1106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.293+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/1538344731 >> 172.21.15.44:0/1538344731 conn(0x562ac107b030 msgr2=0x562ac1101560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.299+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/1538344731 shutdown_connections 2023-09-10T01:29:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.299+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/1538344731 wait complete. 2023-09-10T01:29:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.300+0000 7f5b4b95bb80 1 -- start start 2023-09-10T01:29:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.300+0000 7f5b4b95bb80 1 Processor -- start 2023-09-10T01:29:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.303+0000 7f5b4b95bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106300 0x562ac1112df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.303+0000 7f5b4b95bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac1106cc0 0x562ac1110110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.303+0000 7f5b4b95bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 0x562ac1110740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.303+0000 7f5b4b95bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac0ff6b10 con 0x562ac1106cc0 2023-09-10T01:29:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.303+0000 7f5b4b95bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac0ff6810 con 0x562ac110c1a0 2023-09-10T01:29:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.303+0000 7f5b4b95bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac0eda8c0 con 0x562ac1106300 2023-09-10T01:29:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.303+0000 7f5b4b95bb80 10 client.0 _ll_register_callbacks cb 0x562ac0f40a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:29:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.303+0000 7f5b3dd32700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 0x562ac1110740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.303+0000 7f5b3dd32700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 0x562ac1110740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33646/0 (socket says 172.21.15.44:33646) 2023-09-10T01:29:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.303+0000 7f5b3dd32700 1 -- 172.21.15.44:0/1771780247 learned_addr learned my addr 172.21.15.44:0/1771780247 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.304+0000 7f5b3cd30700 1 --2- 172.21.15.44:0/1771780247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac1106cc0 0x562ac1110110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.304+0000 7f5b3cd30700 1 -- 172.21.15.44:0/1771780247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106300 msgr2=0x562ac1112df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.304+0000 7f5b3cd30700 1 --2- 172.21.15.44:0/1771780247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106300 0x562ac1112df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.304+0000 7f5b3cd30700 1 -- 172.21.15.44:0/1771780247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 msgr2=0x562ac1110740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.304+0000 7f5b3cd30700 1 --2- 172.21.15.44:0/1771780247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 0x562ac1110740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.304+0000 7f5b3cd30700 1 -- 172.21.15.44:0/1771780247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b30010040 con 0x562ac1106cc0 2023-09-10T01:29:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.305+0000 7f5b3cd30700 1 --2- 172.21.15.44:0/1771780247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac1106cc0 0x562ac1110110 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f5b3001a860 tx=0x7f5b3001ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc8ec0e80e04b98 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.305+0000 7f5b20ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:29:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.305+0000 7f5b20ff9700 1 -- 172.21.15.44:0/1771780247 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3001f040 con 0x562ac1106cc0 2023-09-10T01:29:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.305+0000 7f5b20ff9700 1 -- 172.21.15.44:0/1771780247 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(191..191 src has 1..191) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f5b30003c10 con 0x562ac1106cc0 2023-09-10T01:29:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.305+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/1771780247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ac0fb0490 con 0x562ac1106cc0 2023-09-10T01:29:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.305+0000 7f5b20ff9700 1 -- 172.21.15.44:0/1771780247 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=170) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b30003dd0 con 0x562ac1106cc0 2023-09-10T01:29:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.305+0000 7f5b4b95bb80 10 client.20379 fetch_fsmap learned FSMap version 170 2023-09-10T01:29:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.305+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/1771780247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=170,monmap=4+}) v3 -- 0x562ac0e6dc00 con 0x562ac1106cc0 2023-09-10T01:29:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.305+0000 7f5b20ff9700 1 -- 172.21.15.44:0/1771780247 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562ac0fb0490 con 0x562ac1106cc0 2023-09-10T01:29:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.310+0000 7f5b20ff9700 1 -- 172.21.15.44:0/1771780247 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 170) v1 ==== 7375+0+0 (secure 0 0 0) 0x7f5b3002a020 con 0x562ac1106cc0 2023-09-10T01:29:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.310+0000 7f5b4b95bb80 10 client.20379 fetch_fsmap finished waiting for FSMap version 170 2023-09-10T01:29:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.310+0000 7f5b4b95bb80 10 client.20379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:29:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.310+0000 7f5b4b95bb80 20 client.20379 populate_metadata read hostname 'smithi044' 2023-09-10T01:29:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.310+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/1771780247 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ac1120250 0x562ac1122670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.310+0000 7f5b4b95bb80 4 client.20379 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:29:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.310+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/1771780247 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ac106d840 con 0x562ac1120250 2023-09-10T01:29:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.311+0000 7f5b3d531700 1 --2- 172.21.15.44:0/1771780247 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ac1120250 0x562ac1122670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.311+0000 7f5b20ff9700 10 client.20379 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.311+0000 7f5b3d531700 1 --2- 172.21.15.44:0/1771780247 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ac1120250 0x562ac1122670 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=625b11430829f380 server_cookie=1bc14e26f0deea65 in_seq=0 out_seq=0 2023-09-10T01:29:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.312+0000 7f5b20ff9700 1 -- 172.21.15.44:0/1771780247 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562ac106d840 con 0x562ac1120250 2023-09-10T01:29:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.312+0000 7f5b20ff9700 10 client.20379 handle_command_reply: tid=0 2023-09-10T01:29:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.312+0000 7f5b4b95bb80 1 client.20379 shutdown 2023-09-10T01:29:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.312+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/1771780247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac1106cc0 msgr2=0x562ac1110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.312+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/1771780247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac1106cc0 0x562ac1110110 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f5b3001a860 tx=0x7f5b3001ccb0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.312+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/1771780247 shutdown_connections 2023-09-10T01:29:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.312+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/1771780247 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ac1120250 0x562ac1122670 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.312+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/1771780247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 0x562ac1110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.312+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/1771780247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106300 0x562ac1112df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.312+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/1771780247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac1106cc0 0x562ac1110110 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.312+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/1771780247 >> 172.21.15.44:0/1771780247 conn(0x562ac107b030 msgr2=0x562ac11007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.312+0000 7f5b20ff9700 0 client.20379 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.312+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/1771780247 shutdown_connections 2023-09-10T01:29:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.312+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/1771780247 wait complete. 2023-09-10T01:29:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.316+0000 7f5b4b95bb80 20 client.20379 trim_cache size 0 max 16384 2023-09-10T01:29:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b4b95bb80 1 Processor -- start 2023-09-10T01:29:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b4b95bb80 1 -- start start 2023-09-10T01:29:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b4b95bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106300 0x562ac1112920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b4b95bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac1106cc0 0x562ac1101090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b4b95bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac110c1a0 0x562ac11015d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b4b95bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac10fdfc0 con 0x562ac110c1a0 2023-09-10T01:29:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b4b95bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac11116f0 con 0x562ac1106cc0 2023-09-10T01:29:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b4b95bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac1102d10 con 0x562ac1106300 2023-09-10T01:29:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b3dd32700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac110c1a0 0x562ac11015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b3dd32700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac110c1a0 0x562ac11015d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60146/0 (socket says 172.21.15.44:60146) 2023-09-10T01:29:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b3dd32700 1 -- 172.21.15.44:0/127025207 learned_addr learned my addr 172.21.15.44:0/127025207 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b3cd30700 1 --2- 172.21.15.44:0/127025207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac1106cc0 0x562ac1101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b3d531700 1 --2- 172.21.15.44:0/127025207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106300 0x562ac1112920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b3dd32700 1 -- 172.21.15.44:0/127025207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106300 msgr2=0x562ac1112920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b3dd32700 1 --2- 172.21.15.44:0/127025207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106300 0x562ac1112920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b3dd32700 1 -- 172.21.15.44:0/127025207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac1106cc0 msgr2=0x562ac1101090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b3dd32700 1 --2- 172.21.15.44:0/127025207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac1106cc0 0x562ac1101090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b3dd32700 1 -- 172.21.15.44:0/127025207 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ac0eab1f0 con 0x562ac110c1a0 2023-09-10T01:29:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.343+0000 7f5b3cd30700 1 --2- 172.21.15.44:0/127025207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac1106cc0 0x562ac1101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:29:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.344+0000 7f5b3dd32700 1 --2- 172.21.15.44:0/127025207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac110c1a0 0x562ac11015d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f5b34099120 tx=0x7f5b34098c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f611e9624d6e51 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.344+0000 7f5b23fff700 1 -- 172.21.15.44:0/127025207 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3409cfc0 con 0x562ac110c1a0 2023-09-10T01:29:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.344+0000 7f5b23fff700 1 -- 172.21.15.44:0/127025207 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b340a1200 con 0x562ac110c1a0 2023-09-10T01:29:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.344+0000 7f5b23fff700 1 -- 172.21.15.44:0/127025207 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34098680 con 0x562ac110c1a0 2023-09-10T01:29:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.345+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/127025207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac110c1a0 msgr2=0x562ac11015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.345+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/127025207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac110c1a0 0x562ac11015d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f5b34099120 tx=0x7f5b34098c20 comp rx=0 tx=0).stop 2023-09-10T01:29:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.345+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/127025207 shutdown_connections 2023-09-10T01:29:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.345+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/127025207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac110c1a0 0x562ac11015d0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.345+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/127025207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106300 0x562ac1112920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.345+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/127025207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac1106cc0 0x562ac1101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.345+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/127025207 >> 172.21.15.44:0/127025207 conn(0x562ac107b030 msgr2=0x562ac110a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.345+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/127025207 shutdown_connections 2023-09-10T01:29:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.345+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/127025207 wait complete. 2023-09-10T01:29:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.345+0000 7f5b4b95bb80 1 -- start start 2023-09-10T01:29:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.345+0000 7f5b4b95bb80 1 Processor -- start 2023-09-10T01:29:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.345+0000 7f5b4b95bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac1106300 0x562ac1116e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.345+0000 7f5b4b95bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106cc0 0x562ac1117390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.345+0000 7f5b4b95bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 0x562ac1115510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.346+0000 7f5b4b95bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac1115b90 con 0x562ac1106300 2023-09-10T01:29:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.346+0000 7f5b4b95bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac1115d00 con 0x562ac110c1a0 2023-09-10T01:29:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.346+0000 7f5b4b95bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac1115ec0 con 0x562ac1106cc0 2023-09-10T01:29:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.346+0000 7f5b4b95bb80 10 client.0 _ll_register_callbacks cb 0x7f5b28005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:29:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.346+0000 7f5b3cd30700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106cc0 0x562ac1117390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.346+0000 7f5b3dd32700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 0x562ac1115510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.346+0000 7f5b3cd30700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106cc0 0x562ac1117390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38802/0 (socket says 172.21.15.44:38802) 2023-09-10T01:29:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.346+0000 7f5b3cd30700 1 -- 172.21.15.44:0/2042074728 learned_addr learned my addr 172.21.15.44:0/2042074728 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.346+0000 7f5b3cd30700 1 -- 172.21.15.44:0/2042074728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 msgr2=0x562ac1115510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.346+0000 7f5b3cd30700 1 --2- 172.21.15.44:0/2042074728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 0x562ac1115510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.346+0000 7f5b3cd30700 1 -- 172.21.15.44:0/2042074728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac1106300 msgr2=0x562ac1116e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.346+0000 7f5b3cd30700 1 --2- 172.21.15.44:0/2042074728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac1106300 0x562ac1116e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.346+0000 7f5b3cd30700 1 -- 172.21.15.44:0/2042074728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ac0e6dc00 con 0x562ac1106cc0 2023-09-10T01:29:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.346+0000 7f5b3dd32700 1 --2- 172.21.15.44:0/2042074728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 0x562ac1115510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:29:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.346+0000 7f5b3cd30700 1 --2- 172.21.15.44:0/2042074728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106cc0 0x562ac1117390 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5b300062a0 tx=0x7f5b30018380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=523a8253359b7fef server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.347+0000 7f5b20ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:29:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.347+0000 7f5b20ff9700 1 -- 172.21.15.44:0/2042074728 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3001f070 con 0x562ac1106cc0 2023-09-10T01:29:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.347+0000 7f5b20ff9700 1 -- 172.21.15.44:0/2042074728 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(191..191 src has 1..191) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f5b300050d0 con 0x562ac1106cc0 2023-09-10T01:29:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.347+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/2042074728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b30006040 con 0x562ac1106cc0 2023-09-10T01:29:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.347+0000 7f5b20ff9700 1 -- 172.21.15.44:0/2042074728 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=170) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b30005290 con 0x562ac1106cc0 2023-09-10T01:29:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.347+0000 7f5b4b95bb80 10 client.30022 fetch_fsmap learned FSMap version 170 2023-09-10T01:29:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.347+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/2042074728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=170,monmap=4+}) v3 -- 0x7f5b381a9cf0 con 0x562ac1106cc0 2023-09-10T01:29:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.347+0000 7f5b20ff9700 1 -- 172.21.15.44:0/2042074728 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b30006040 con 0x562ac1106cc0 2023-09-10T01:29:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.347+0000 7f5b20ff9700 1 -- 172.21.15.44:0/2042074728 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 170) v1 ==== 7375+0+0 (secure 0 0 0) 0x7f5b3001c100 con 0x562ac1106cc0 2023-09-10T01:29:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.347+0000 7f5b4b95bb80 10 client.30022 fetch_fsmap finished waiting for FSMap version 170 2023-09-10T01:29:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.347+0000 7f5b4b95bb80 10 client.30022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:29:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.347+0000 7f5b4b95bb80 20 client.30022 populate_metadata read hostname 'smithi044' 2023-09-10T01:29:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.347+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/2042074728 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ac1174d30 0x562ac1177150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.347+0000 7f5b4b95bb80 4 client.30022 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:29:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.347+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/2042074728 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562ac10518c0 con 0x562ac1174d30 2023-09-10T01:29:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.347+0000 7f5b3d531700 1 --2- 172.21.15.44:0/2042074728 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ac1174d30 0x562ac1177150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.348+0000 7f5b20ff9700 10 client.30022 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.348+0000 7f5b3d531700 1 --2- 172.21.15.44:0/2042074728 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ac1174d30 0x562ac1177150 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b04dd738cbaddb9 server_cookie=8960e5da4ba35779 in_seq=0 out_seq=0 2023-09-10T01:29:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.391+0000 7f5b20ff9700 1 -- 172.21.15.44:0/2042074728 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562ac10518c0 con 0x562ac1174d30 2023-09-10T01:29:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.391+0000 7f5b20ff9700 10 client.30022 handle_command_reply: tid=0 2023-09-10T01:29:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.391+0000 7f5b4b95bb80 1 client.30022 shutdown 2023-09-10T01:29:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.391+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/2042074728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106cc0 msgr2=0x562ac1117390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.391+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/2042074728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106cc0 0x562ac1117390 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5b300062a0 tx=0x7f5b30018380 comp rx=0 tx=0).stop 2023-09-10T01:29:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.391+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/2042074728 shutdown_connections 2023-09-10T01:29:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.391+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/2042074728 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ac1174d30 0x562ac1177150 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.391+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/2042074728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ac110c1a0 0x562ac1115510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.391+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/2042074728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ac1106300 0x562ac1116e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.391+0000 7f5b4b95bb80 1 --2- 172.21.15.44:0/2042074728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ac1106cc0 0x562ac1117390 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.391+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/2042074728 >> 172.21.15.44:0/2042074728 conn(0x562ac107b030 msgr2=0x562ac1101090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.391+0000 7f5b20ff9700 0 client.30022 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.391+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/2042074728 shutdown_connections 2023-09-10T01:29:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.392+0000 7f5b4b95bb80 1 -- 172.21.15.44:0/2042074728 wait complete. 2023-09-10T01:29:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.392+0000 7f5b4b95bb80 20 client.30022 trim_cache size 0 max 16384 2023-09-10T01:29:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:29:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:29:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:29:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:29:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.392+0000 7f5b3f795700 1 -- 172.21.15.44:0/1576413149 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5b240c2990 msgr2=0x7f5b240c4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.392+0000 7f5b3f795700 1 --2- 172.21.15.44:0/1576413149 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5b240c2990 0x7f5b240c4e50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x562ac1101060 tx=0x7f5b28009260 comp rx=0 tx=0).stop 2023-09-10T01:29:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.392+0000 7f5b3f795700 1 -- 172.21.15.44:0/1576413149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b38119de0 msgr2=0x7f5b38114f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.392+0000 7f5b3f795700 1 --2- 172.21.15.44:0/1576413149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b38119de0 0x7f5b38114f10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400e970 tx=0x7f5b3400ec80 comp rx=0 tx=0).stop 2023-09-10T01:29:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.392+0000 7f5b3f795700 1 -- 172.21.15.44:0/1576413149 shutdown_connections 2023-09-10T01:29:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.392+0000 7f5b3f795700 1 --2- 172.21.15.44:0/1576413149 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f5b240c2990 0x7f5b240c4e50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.392+0000 7f5b3f795700 1 --2- 172.21.15.44:0/1576413149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b38119de0 0x7f5b38114f10 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.392+0000 7f5b3f795700 1 --2- 172.21.15.44:0/1576413149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b3810ed50 0x7f5b38119070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.392+0000 7f5b3f795700 1 --2- 172.21.15.44:0/1576413149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b3810f1c0 0x7f5b381195b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.392+0000 7f5b3f795700 1 -- 172.21.15.44:0/1576413149 >> 172.21.15.44:0/1576413149 conn(0x7f5b3806c6c0 msgr2=0x7f5b3811b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.392+0000 7f5b3f795700 1 -- 172.21.15.44:0/1576413149 shutdown_connections 2023-09-10T01:29:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:19.392+0000 7f5b3f795700 1 -- 172.21.15.44:0/1576413149 wait complete. 2023-09-10T01:29:20.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:19 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:29:20.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:19 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:29:20.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:19 smithi131 ceph-mon[103713]: pgmap v1581: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2023-09-10T01:29:20.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:19 smithi131 ceph-mon[103713]: pgmap v1582: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 388 op/s 2023-09-10T01:29:20.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:19 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:29:20.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:19 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:29:20.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:19 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:29:20.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:19 smithi044 ceph-mon[98676]: pgmap v1581: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2023-09-10T01:29:20.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:19 smithi044 ceph-mon[98676]: pgmap v1582: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 388 op/s 2023-09-10T01:29:20.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:19 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:29:20.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:19 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:29:20.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:19 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:29:20.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:19 smithi099 ceph-mon[102692]: pgmap v1581: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2023-09-10T01:29:20.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:19 smithi099 ceph-mon[102692]: pgmap v1582: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 388 op/s 2023-09-10T01:29:20.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:19 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:29:21.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 495.0 s, 186.8 tps, lat 534.632 ms stddev 315.927 2023-09-10T01:29:22.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:22 smithi131 ceph-mon[103713]: pgmap v1583: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 372 op/s 2023-09-10T01:29:22.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:22 smithi131 ceph-mon[103713]: osdmap e192: 12 total, 12 up, 12 in 2023-09-10T01:29:22.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:22 smithi044 ceph-mon[98676]: pgmap v1583: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 372 op/s 2023-09-10T01:29:22.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:22 smithi044 ceph-mon[98676]: osdmap e192: 12 total, 12 up, 12 in 2023-09-10T01:29:22.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:22 smithi099 ceph-mon[102692]: pgmap v1583: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 372 op/s 2023-09-10T01:29:22.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:22 smithi099 ceph-mon[102692]: osdmap e192: 12 total, 12 up, 12 in 2023-09-10T01:29:24.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:24 smithi131 ceph-mon[103713]: pgmap v1585: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 583 op/s 2023-09-10T01:29:24.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:24 smithi044 ceph-mon[98676]: pgmap v1585: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 583 op/s 2023-09-10T01:29:24.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:24 smithi099 ceph-mon[102692]: pgmap v1585: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 583 op/s 2023-09-10T01:29:26.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 500.0 s, 230.2 tps, lat 436.780 ms stddev 156.548 2023-09-10T01:29:26.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:26 smithi131 ceph-mon[103713]: pgmap v1586: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 399 op/s 2023-09-10T01:29:26.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:26 smithi044 ceph-mon[98676]: pgmap v1586: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 399 op/s 2023-09-10T01:29:26.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:26 smithi099 ceph-mon[102692]: pgmap v1586: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 399 op/s 2023-09-10T01:29:28.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:28 smithi131 ceph-mon[103713]: pgmap v1587: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.5 MiB/s wr, 838 op/s 2023-09-10T01:29:28.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:28 smithi044 ceph-mon[98676]: pgmap v1587: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.5 MiB/s wr, 838 op/s 2023-09-10T01:29:28.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:28 smithi099 ceph-mon[102692]: pgmap v1587: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.5 MiB/s wr, 838 op/s 2023-09-10T01:29:30.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:30 smithi131 ceph-mon[103713]: pgmap v1588: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 670 op/s 2023-09-10T01:29:30.809 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:30 smithi044 ceph-mon[98676]: pgmap v1588: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 670 op/s 2023-09-10T01:29:30.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:30 smithi099 ceph-mon[102692]: pgmap v1588: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 670 op/s 2023-09-10T01:29:31.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 505.0 s, 232.2 tps, lat 418.421 ms stddev 150.196 2023-09-10T01:29:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.576+0000 7fdd67dd1700 1 -- 172.21.15.44:0/1744773400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd60111070 msgr2=0x7fdd60111450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.576+0000 7fdd67dd1700 1 --2- 172.21.15.44:0/1744773400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd60111070 0x7fdd60111450 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009990 tx=0x7fdd5c009ca0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.576+0000 7fdd67dd1700 1 -- 172.21.15.44:0/1744773400 shutdown_connections 2023-09-10T01:29:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.576+0000 7fdd67dd1700 1 --2- 172.21.15.44:0/1744773400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd6010b970 0x7fdd60117e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.576+0000 7fdd67dd1700 1 --2- 172.21.15.44:0/1744773400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd60111070 0x7fdd60111450 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.576+0000 7fdd67dd1700 1 --2- 172.21.15.44:0/1744773400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd6010afb0 0x7fdd6010b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.576+0000 7fdd67dd1700 1 -- 172.21.15.44:0/1744773400 >> 172.21.15.44:0/1744773400 conn(0x7fdd6006b3c0 msgr2=0x7fdd60109610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.576+0000 7fdd67dd1700 1 -- 172.21.15.44:0/1744773400 shutdown_connections 2023-09-10T01:29:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.576+0000 7fdd67dd1700 1 -- 172.21.15.44:0/1744773400 wait complete. 2023-09-10T01:29:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.577+0000 7fdd67dd1700 1 Processor -- start 2023-09-10T01:29:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.577+0000 7fdd67dd1700 1 -- start start 2023-09-10T01:29:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.577+0000 7fdd67dd1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd6010afb0 0x7fdd601a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.577+0000 7fdd67dd1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd6010b970 0x7fdd601a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.577+0000 7fdd67dd1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd60111070 0x7fdd6019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.578+0000 7fdd67dd1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6019dd40 con 0x7fdd6010afb0 2023-09-10T01:29:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.578+0000 7fdd67dd1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6019deb0 con 0x7fdd60111070 2023-09-10T01:29:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.578+0000 7fdd67dd1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6019e070 con 0x7fdd6010b970 2023-09-10T01:29:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.578+0000 7fdd6536c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd6010b970 0x7fdd601a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.578+0000 7fdd6636e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd60111070 0x7fdd6019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.578+0000 7fdd6536c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd6010b970 0x7fdd601a31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38814/0 (socket says 172.21.15.44:38814) 2023-09-10T01:29:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.578+0000 7fdd6636e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd60111070 0x7fdd6019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33674/0 (socket says 172.21.15.44:33674) 2023-09-10T01:29:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.578+0000 7fdd6536c700 1 -- 172.21.15.44:0/3335284652 learned_addr learned my addr 172.21.15.44:0/3335284652 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.578+0000 7fdd6636e700 1 -- 172.21.15.44:0/3335284652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd6010b970 msgr2=0x7fdd601a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.578+0000 7fdd6636e700 1 --2- 172.21.15.44:0/3335284652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd6010b970 0x7fdd601a31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.578+0000 7fdd6636e700 1 -- 172.21.15.44:0/3335284652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd6010afb0 msgr2=0x7fdd601a2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.578+0000 7fdd6636e700 1 --2- 172.21.15.44:0/3335284652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd6010afb0 0x7fdd601a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.578+0000 7fdd6636e700 1 -- 172.21.15.44:0/3335284652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c0095f0 con 0x7fdd60111070 2023-09-10T01:29:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.578+0000 7fdd6536c700 1 --2- 172.21.15.44:0/3335284652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd6010b970 0x7fdd601a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:29:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.578+0000 7fdd6636e700 1 --2- 172.21.15.44:0/3335284652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd60111070 0x7fdd6019d660 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800efc0 tx=0x7fdd5800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.579+0000 7fdd56ffd700 1 -- 172.21.15.44:0/3335284652 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580092e0 con 0x7fdd60111070 2023-09-10T01:29:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.579+0000 7fdd56ffd700 1 -- 172.21.15.44:0/3335284652 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd5800f040 con 0x7fdd60111070 2023-09-10T01:29:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.579+0000 7fdd56ffd700 1 -- 172.21.15.44:0/3335284652 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800cdb0 con 0x7fdd60111070 2023-09-10T01:29:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.579+0000 7fdd67dd1700 1 -- 172.21.15.44:0/3335284652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6019e350 con 0x7fdd60111070 2023-09-10T01:29:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.579+0000 7fdd67dd1700 1 -- 172.21.15.44:0/3335284652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd601a7c00 con 0x7fdd60111070 2023-09-10T01:29:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.583+0000 7fdd56ffd700 1 -- 172.21.15.44:0/3335284652 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fdd58003a00 con 0x7fdd60111070 2023-09-10T01:29:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.585+0000 7fdd56ffd700 1 --2- 172.21.15.44:0/3335284652 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdd4c0a0830 0x7fdd4c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.585+0000 7fdd56ffd700 1 -- 172.21.15.44:0/3335284652 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fdd580eded0 con 0x7fdd60111070 2023-09-10T01:29:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.586+0000 7fdd73f97b80 1 Processor -- start 2023-09-10T01:29:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.586+0000 7fdd73f97b80 1 -- start start 2023-09-10T01:29:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.587+0000 7fdd73f97b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c40580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.587+0000 7fdd73f97b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3a300 0x5646b4c3a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.587+0000 7fdd73f97b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3acc0 0x5646b4c46f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.587+0000 7fdd73f97b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b4b936d0 con 0x5646b4c3a300 2023-09-10T01:29:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.587+0000 7fdd73f97b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b4b90ff0 con 0x5646b4c401a0 2023-09-10T01:29:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.587+0000 7fdd73f97b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b4b8c8e0 con 0x5646b4c3acc0 2023-09-10T01:29:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.587+0000 7fdd65b6d700 1 --2- 172.21.15.44:0/3335284652 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdd4c0a0830 0x7fdd4c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.588+0000 7fdd65b6d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c40580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.588+0000 7fdd65b6d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c40580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33678/0 (socket says 172.21.15.44:33678) 2023-09-10T01:29:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.588+0000 7fdd65b6d700 1 -- 172.21.15.44:0/937710600 learned_addr learned my addr 172.21.15.44:0/937710600 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.588+0000 7fdd65b6d700 1 -- 172.21.15.44:0/937710600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3acc0 msgr2=0x5646b4c46f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:29:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.588+0000 7fdd65b6d700 1 --2- 172.21.15.44:0/937710600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3acc0 0x5646b4c46f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.588+0000 7fdd65b6d700 1 -- 172.21.15.44:0/937710600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3a300 msgr2=0x5646b4c3a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.588+0000 7fdd65b6d700 1 --2- 172.21.15.44:0/937710600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3a300 0x5646b4c3a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.588+0000 7fdd65b6d700 1 -- 172.21.15.44:0/937710600 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646b49b8d80 con 0x5646b4c401a0 2023-09-10T01:29:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.595+0000 7fdd65b6d700 1 --2- 172.21.15.44:0/937710600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c40580 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00b7f0 tx=0x7fdd5c00b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a35fef11511382af server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.597+0000 7fdd4bfff700 1 -- 172.21.15.44:0/937710600 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c00fd00 con 0x5646b4c401a0 2023-09-10T01:29:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.597+0000 7fdd4bfff700 1 -- 172.21.15.44:0/937710600 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd5c00fe60 con 0x5646b4c401a0 2023-09-10T01:29:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.597+0000 7fdd4bfff700 1 -- 172.21.15.44:0/937710600 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c004b70 con 0x5646b4c401a0 2023-09-10T01:29:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.598+0000 7fdd73f97b80 1 -- 172.21.15.44:0/937710600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 msgr2=0x5646b4c40580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.598+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/937710600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c40580 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00b7f0 tx=0x7fdd5c00b820 comp rx=0 tx=0).stop 2023-09-10T01:29:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.598+0000 7fdd65b6d700 1 --2- 172.21.15.44:0/3335284652 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdd4c0a0830 0x7fdd4c0a2cf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00bc20 tx=0x7fdd5c01c000 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.604+0000 7fdd73f97b80 1 -- 172.21.15.44:0/937710600 shutdown_connections 2023-09-10T01:29:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.604+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/937710600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3acc0 0x5646b4c46f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.604+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/937710600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c40580 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.604+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/937710600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3a300 0x5646b4c3a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.604+0000 7fdd73f97b80 1 -- 172.21.15.44:0/937710600 >> 172.21.15.44:0/937710600 conn(0x5646b4baf030 msgr2=0x5646b4c35560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.610+0000 7fdd73f97b80 1 -- 172.21.15.44:0/937710600 shutdown_connections 2023-09-10T01:29:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.610+0000 7fdd73f97b80 1 -- 172.21.15.44:0/937710600 wait complete. 2023-09-10T01:29:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.611+0000 7fdd73f97b80 1 -- start start 2023-09-10T01:29:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.611+0000 7fdd73f97b80 1 Processor -- start 2023-09-10T01:29:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.611+0000 7fdd73f97b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3a300 0x5646b4c46e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.611+0000 7fdd73f97b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3acc0 0x5646b4c44100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.611+0000 7fdd73f97b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c447a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.611+0000 7fdd73f97b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b4b2ab10 con 0x5646b4c3acc0 2023-09-10T01:29:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.611+0000 7fdd73f97b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b4b2a810 con 0x5646b4c401a0 2023-09-10T01:29:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.611+0000 7fdd73f97b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b4a0e8c0 con 0x5646b4c3a300 2023-09-10T01:29:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.612+0000 7fdd73f97b80 10 client.0 _ll_register_callbacks cb 0x5646b4a74a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:29:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.612+0000 7fdd65b6d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3a300 0x5646b4c46e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.612+0000 7fdd6636e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.612+0000 7fdd6636e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c447a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33688/0 (socket says 172.21.15.44:33688) 2023-09-10T01:29:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.612+0000 7fdd65b6d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3a300 0x5646b4c46e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38828/0 (socket says 172.21.15.44:38828) 2023-09-10T01:29:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.612+0000 7fdd6636e700 1 -- 172.21.15.44:0/2618028831 learned_addr learned my addr 172.21.15.44:0/2618028831 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.612+0000 7fdd6536c700 1 --2- 172.21.15.44:0/2618028831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3acc0 0x5646b4c44100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.612+0000 7fdd6636e700 1 -- 172.21.15.44:0/2618028831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3a300 msgr2=0x5646b4c46e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.612+0000 7fdd6636e700 1 --2- 172.21.15.44:0/2618028831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3a300 0x5646b4c46e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.612+0000 7fdd6636e700 1 -- 172.21.15.44:0/2618028831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3acc0 msgr2=0x5646b4c44100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.612+0000 7fdd6636e700 1 --2- 172.21.15.44:0/2618028831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3acc0 0x5646b4c44100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.612+0000 7fdd6636e700 1 -- 172.21.15.44:0/2618028831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd60118700 con 0x5646b4c401a0 2023-09-10T01:29:32.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.612+0000 7fdd6536c700 1 --2- 172.21.15.44:0/2618028831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3acc0 0x5646b4c44100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:29:32.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.613+0000 7fdd6636e700 1 --2- 172.21.15.44:0/2618028831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c447a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdd580c2a60 tx=0x7fdd580ba900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfad16731a88bbc1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.616+0000 7fdd497fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:29:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.616+0000 7fdd497fa700 1 -- 172.21.15.44:0/2618028831 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5803b310 con 0x5646b4c401a0 2023-09-10T01:29:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.616+0000 7fdd497fa700 1 -- 172.21.15.44:0/2618028831 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fdd580bb620 con 0x5646b4c401a0 2023-09-10T01:29:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.616+0000 7fdd73f97b80 1 -- 172.21.15.44:0/2618028831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646b4ae4490 con 0x5646b4c401a0 2023-09-10T01:29:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.618+0000 7fdd497fa700 1 -- 172.21.15.44:0/2618028831 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=170) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd580baf20 con 0x5646b4c401a0 2023-09-10T01:29:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.618+0000 7fdd73f97b80 10 client.30023 fetch_fsmap learned FSMap version 170 2023-09-10T01:29:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.618+0000 7fdd73f97b80 1 -- 172.21.15.44:0/2618028831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=170,monmap=4+}) v3 -- 0x5646b49a1c00 con 0x5646b4c401a0 2023-09-10T01:29:32.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.626+0000 7fdd497fa700 1 -- 172.21.15.44:0/2618028831 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5646b4ae4490 con 0x5646b4c401a0 2023-09-10T01:29:32.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.626+0000 7fdd497fa700 1 -- 172.21.15.44:0/2618028831 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 170) v1 ==== 7375+0+0 (secure 0 0 0) 0x7fdd580ba520 con 0x5646b4c401a0 2023-09-10T01:29:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.627+0000 7fdd73f97b80 10 client.30023 fetch_fsmap finished waiting for FSMap version 170 2023-09-10T01:29:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.627+0000 7fdd73f97b80 10 client.30023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:29:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.627+0000 7fdd73f97b80 20 client.30023 populate_metadata read hostname 'smithi044' 2023-09-10T01:29:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.628+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/2618028831 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5646b4c544d0 0x5646b4c568f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.628+0000 7fdd73f97b80 4 client.30023 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:29:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.628+0000 7fdd73f97b80 1 -- 172.21.15.44:0/2618028831 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646b4ba1840 con 0x5646b4c544d0 2023-09-10T01:29:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.634+0000 7fdd65b6d700 1 --2- 172.21.15.44:0/2618028831 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5646b4c544d0 0x5646b4c568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.634+0000 7fdd65b6d700 1 --2- 172.21.15.44:0/2618028831 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5646b4c544d0 0x5646b4c568f0 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1348bedaccb191bd server_cookie=56b7f93135b3bc5c in_seq=0 out_seq=0 2023-09-10T01:29:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.634+0000 7fdd497fa700 10 client.30023 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.637+0000 7fdd497fa700 1 -- 172.21.15.44:0/2618028831 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5646b4ba1840 con 0x5646b4c544d0 2023-09-10T01:29:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.637+0000 7fdd497fa700 10 client.30023 handle_command_reply: tid=0 2023-09-10T01:29:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.637+0000 7fdd73f97b80 1 client.30023 shutdown 2023-09-10T01:29:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.642+0000 7fdd73f97b80 1 -- 172.21.15.44:0/2618028831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 msgr2=0x5646b4c447a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:32.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.642+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/2618028831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c447a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdd580c2a60 tx=0x7fdd580ba900 comp rx=0 tx=0).stop 2023-09-10T01:29:32.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.644+0000 7fdd73f97b80 1 -- 172.21.15.44:0/2618028831 shutdown_connections 2023-09-10T01:29:32.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.644+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/2618028831 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5646b4c544d0 0x5646b4c568f0 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.644+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/2618028831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c447a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.644+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/2618028831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3a300 0x5646b4c46e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.644+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/2618028831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3acc0 0x5646b4c44100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.644+0000 7fdd73f97b80 1 -- 172.21.15.44:0/2618028831 >> 172.21.15.44:0/2618028831 conn(0x5646b4baf030 msgr2=0x5646b4c347a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.644+0000 7fdd497fa700 0 client.30023 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.645+0000 7fdd73f97b80 1 -- 172.21.15.44:0/2618028831 shutdown_connections 2023-09-10T01:29:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.656+0000 7fdd73f97b80 1 -- 172.21.15.44:0/2618028831 wait complete. 2023-09-10T01:29:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.661+0000 7fdd73f97b80 20 client.30023 trim_cache size 0 max 16384 2023-09-10T01:29:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.688+0000 7fdd73f97b80 1 Processor -- start 2023-09-10T01:29:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.688+0000 7fdd73f97b80 1 -- start start 2023-09-10T01:29:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.688+0000 7fdd73f97b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3a300 0x5646b4c46a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.688+0000 7fdd73f97b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3acc0 0x5646b4c35090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.688+0000 7fdd73f97b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c355d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.688+0000 7fdd73f97b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b4c450b0 con 0x5646b4c3a300 2023-09-10T01:29:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.688+0000 7fdd73f97b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b4c51a50 con 0x5646b4c401a0 2023-09-10T01:29:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.688+0000 7fdd73f97b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b4c456e0 con 0x5646b4c3acc0 2023-09-10T01:29:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.689+0000 7fdd6536c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3acc0 0x5646b4c35090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.689+0000 7fdd6536c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3acc0 0x5646b4c35090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38836/0 (socket says 172.21.15.44:38836) 2023-09-10T01:29:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.689+0000 7fdd6536c700 1 -- 172.21.15.44:0/2006243938 learned_addr learned my addr 172.21.15.44:0/2006243938 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:32.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.689+0000 7fdd6536c700 1 -- 172.21.15.44:0/2006243938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 msgr2=0x5646b4c355d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:32.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.689+0000 7fdd6536c700 1 --2- 172.21.15.44:0/2006243938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.689+0000 7fdd6536c700 1 -- 172.21.15.44:0/2006243938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3a300 msgr2=0x5646b4c46a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:32.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.689+0000 7fdd6536c700 1 --2- 172.21.15.44:0/2006243938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3a300 0x5646b4c46a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.689+0000 7fdd6536c700 1 -- 172.21.15.44:0/2006243938 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646b49df1f0 con 0x5646b4c3acc0 2023-09-10T01:29:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.694+0000 7fdd6536c700 1 --2- 172.21.15.44:0/2006243938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3acc0 0x5646b4c35090 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdd50005950 tx=0x7fdd500118b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9c3088b9fc33a45 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.694+0000 7fdd54ff9700 1 -- 172.21.15.44:0/2006243938 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50004440 con 0x5646b4c3acc0 2023-09-10T01:29:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.694+0000 7fdd54ff9700 1 -- 172.21.15.44:0/2006243938 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd500045a0 con 0x5646b4c3acc0 2023-09-10T01:29:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.694+0000 7fdd54ff9700 1 -- 172.21.15.44:0/2006243938 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50010a00 con 0x5646b4c3acc0 2023-09-10T01:29:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.695+0000 7fdd73f97b80 1 -- 172.21.15.44:0/2006243938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3acc0 msgr2=0x5646b4c35090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.695+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/2006243938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3acc0 0x5646b4c35090 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdd50005950 tx=0x7fdd500118b0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.695+0000 7fdd73f97b80 1 -- 172.21.15.44:0/2006243938 shutdown_connections 2023-09-10T01:29:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.695+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/2006243938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c355d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.695+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/2006243938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3a300 0x5646b4c46a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.695+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/2006243938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3acc0 0x5646b4c35090 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.695+0000 7fdd73f97b80 1 -- 172.21.15.44:0/2006243938 >> 172.21.15.44:0/2006243938 conn(0x5646b4baf030 msgr2=0x5646b4c3e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.695+0000 7fdd73f97b80 1 -- 172.21.15.44:0/2006243938 shutdown_connections 2023-09-10T01:29:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.695+0000 7fdd73f97b80 1 -- 172.21.15.44:0/2006243938 wait complete. 2023-09-10T01:29:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.696+0000 7fdd73f97b80 1 -- start start 2023-09-10T01:29:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.696+0000 7fdd73f97b80 1 Processor -- start 2023-09-10T01:29:32.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.696+0000 7fdd73f97b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3a300 0x5646b4c4b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.696+0000 7fdd73f97b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3acc0 0x5646b4c495c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.696+0000 7fdd73f97b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c49b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.696+0000 7fdd73f97b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b4c4a040 con 0x5646b4c3acc0 2023-09-10T01:29:32.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.696+0000 7fdd73f97b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b4c4a180 con 0x5646b4c401a0 2023-09-10T01:29:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.696+0000 7fdd73f97b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b4c4a340 con 0x5646b4c3a300 2023-09-10T01:29:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.696+0000 7fdd73f97b80 10 client.0 _ll_register_callbacks cb 0x7fdd580bacf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:29:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.696+0000 7fdd65b6d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3a300 0x5646b4c4b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.697+0000 7fdd65b6d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3a300 0x5646b4c4b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38840/0 (socket says 172.21.15.44:38840) 2023-09-10T01:29:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.697+0000 7fdd65b6d700 1 -- 172.21.15.44:0/3475206315 learned_addr learned my addr 172.21.15.44:0/3475206315 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:32.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.697+0000 7fdd6636e700 1 --2- 172.21.15.44:0/3475206315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c49b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:32.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.697+0000 7fdd65b6d700 1 -- 172.21.15.44:0/3475206315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 msgr2=0x5646b4c49b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:32.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.697+0000 7fdd65b6d700 1 --2- 172.21.15.44:0/3475206315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c49b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.697+0000 7fdd65b6d700 1 -- 172.21.15.44:0/3475206315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3acc0 msgr2=0x5646b4c495c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:29:32.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.697+0000 7fdd65b6d700 1 --2- 172.21.15.44:0/3475206315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3acc0 0x5646b4c495c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.697+0000 7fdd65b6d700 1 -- 172.21.15.44:0/3475206315 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5000ae30 con 0x5646b4c3a300 2023-09-10T01:29:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.697+0000 7fdd65b6d700 1 --2- 172.21.15.44:0/3475206315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3a300 0x5646b4c4b2d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00b7c0 tx=0x7fdd5c00bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2b9f894f3f930a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.697+0000 7fdd497fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:29:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.697+0000 7fdd497fa700 1 -- 172.21.15.44:0/3475206315 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c037040 con 0x5646b4c3a300 2023-09-10T01:29:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.697+0000 7fdd73f97b80 1 -- 172.21.15.44:0/3475206315 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd5c00fe60 con 0x5646b4c3a300 2023-09-10T01:29:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.697+0000 7fdd497fa700 1 -- 172.21.15.44:0/3475206315 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fdd5c03c900 con 0x5646b4c3a300 2023-09-10T01:29:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.698+0000 7fdd497fa700 1 -- 172.21.15.44:0/3475206315 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=170) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd5c00fce0 con 0x5646b4c3a300 2023-09-10T01:29:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.698+0000 7fdd73f97b80 10 client.30037 fetch_fsmap learned FSMap version 170 2023-09-10T01:29:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.698+0000 7fdd73f97b80 1 -- 172.21.15.44:0/3475206315 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=170,monmap=4+}) v3 -- 0x5646b49b7250 con 0x5646b4c3a300 2023-09-10T01:29:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.698+0000 7fdd497fa700 1 -- 172.21.15.44:0/3475206315 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd5c00fe60 con 0x5646b4c3a300 2023-09-10T01:29:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.698+0000 7fdd497fa700 1 -- 172.21.15.44:0/3475206315 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 170) v1 ==== 7375+0+0 (secure 0 0 0) 0x7fdd5c02e070 con 0x5646b4c3a300 2023-09-10T01:29:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.699+0000 7fdd73f97b80 10 client.30037 fetch_fsmap finished waiting for FSMap version 170 2023-09-10T01:29:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.699+0000 7fdd73f97b80 10 client.30037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:29:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.699+0000 7fdd73f97b80 20 client.30037 populate_metadata read hostname 'smithi044' 2023-09-10T01:29:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.699+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/3475206315 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5646b4ca94e0 0x5646b4cab900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.699+0000 7fdd73f97b80 4 client.30037 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:29:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.699+0000 7fdd73f97b80 1 -- 172.21.15.44:0/3475206315 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5646b4b858c0 con 0x5646b4ca94e0 2023-09-10T01:29:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.699+0000 7fdd6536c700 1 --2- 172.21.15.44:0/3475206315 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5646b4ca94e0 0x5646b4cab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.699+0000 7fdd6536c700 1 --2- 172.21.15.44:0/3475206315 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5646b4ca94e0 0x5646b4cab900 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7d2a42cecea4c59 server_cookie=92b7c1aa713a9cd3 in_seq=0 out_seq=0 2023-09-10T01:29:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.699+0000 7fdd497fa700 10 client.30037 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.745+0000 7fdd497fa700 1 -- 172.21.15.44:0/3475206315 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5646b4b858c0 con 0x5646b4ca94e0 2023-09-10T01:29:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.745+0000 7fdd497fa700 10 client.30037 handle_command_reply: tid=0 2023-09-10T01:29:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.748+0000 7fdd73f97b80 1 client.30037 shutdown 2023-09-10T01:29:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.750+0000 7fdd73f97b80 1 -- 172.21.15.44:0/3475206315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3a300 msgr2=0x5646b4c4b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.750+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/3475206315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3a300 0x5646b4c4b2d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00b7c0 tx=0x7fdd5c00bcd0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.750+0000 7fdd73f97b80 1 -- 172.21.15.44:0/3475206315 shutdown_connections 2023-09-10T01:29:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.750+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/3475206315 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5646b4ca94e0 0x5646b4cab900 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.750+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/3475206315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646b4c401a0 0x5646b4c49b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.750+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/3475206315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5646b4c3a300 0x5646b4c4b2d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.750+0000 7fdd73f97b80 1 --2- 172.21.15.44:0/3475206315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5646b4c3acc0 0x5646b4c495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.750+0000 7fdd73f97b80 1 -- 172.21.15.44:0/3475206315 >> 172.21.15.44:0/3475206315 conn(0x5646b4baf030 msgr2=0x5646b4c3f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.750+0000 7fdd497fa700 0 client.30037 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.755+0000 7fdd73f97b80 1 -- 172.21.15.44:0/3475206315 shutdown_connections 2023-09-10T01:29:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.755+0000 7fdd73f97b80 1 -- 172.21.15.44:0/3475206315 wait complete. 2023-09-10T01:29:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.756+0000 7fdd73f97b80 20 client.30037 trim_cache size 0 max 16384 2023-09-10T01:29:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:29:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:29:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:29:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:29:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.756+0000 7fdd67dd1700 1 -- 172.21.15.44:0/3335284652 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdd4c0a0830 msgr2=0x7fdd4c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.756+0000 7fdd67dd1700 1 --2- 172.21.15.44:0/3335284652 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdd4c0a0830 0x7fdd4c0a2cf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00bc20 tx=0x7fdd5c01c000 comp rx=0 tx=0).stop 2023-09-10T01:29:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.756+0000 7fdd67dd1700 1 -- 172.21.15.44:0/3335284652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd60111070 msgr2=0x7fdd6019d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.756+0000 7fdd67dd1700 1 --2- 172.21.15.44:0/3335284652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd60111070 0x7fdd6019d660 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800efc0 tx=0x7fdd5800c3a0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.757+0000 7fdd67dd1700 1 -- 172.21.15.44:0/3335284652 shutdown_connections 2023-09-10T01:29:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.757+0000 7fdd67dd1700 1 --2- 172.21.15.44:0/3335284652 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdd4c0a0830 0x7fdd4c0a2cf0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.757+0000 7fdd67dd1700 1 --2- 172.21.15.44:0/3335284652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd60111070 0x7fdd6019d660 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.757+0000 7fdd67dd1700 1 --2- 172.21.15.44:0/3335284652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd6010afb0 0x7fdd601a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.757+0000 7fdd67dd1700 1 --2- 172.21.15.44:0/3335284652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd6010b970 0x7fdd601a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.757+0000 7fdd67dd1700 1 -- 172.21.15.44:0/3335284652 >> 172.21.15.44:0/3335284652 conn(0x7fdd6006b3c0 msgr2=0x7fdd60109840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.761+0000 7fdd67dd1700 1 -- 172.21.15.44:0/3335284652 shutdown_connections 2023-09-10T01:29:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:32.761+0000 7fdd67dd1700 1 -- 172.21.15.44:0/3335284652 wait complete. 2023-09-10T01:29:32.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:32 smithi131 ceph-mon[103713]: pgmap v1589: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 696 op/s 2023-09-10T01:29:32.812 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:32 smithi044 ceph-mon[98676]: pgmap v1589: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 696 op/s 2023-09-10T01:29:32.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:32 smithi099 ceph-mon[102692]: pgmap v1589: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 696 op/s 2023-09-10T01:29:34.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:34 smithi131 ceph-mon[103713]: pgmap v1590: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.3 MiB/s wr, 689 op/s 2023-09-10T01:29:34.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:34 smithi044 ceph-mon[98676]: pgmap v1590: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.3 MiB/s wr, 689 op/s 2023-09-10T01:29:34.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:34 smithi099 ceph-mon[102692]: pgmap v1590: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.3 MiB/s wr, 689 op/s 2023-09-10T01:29:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 510.0 s, 235.8 tps, lat 366.779 ms stddev 255.597 2023-09-10T01:29:36.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:36 smithi131 ceph-mon[103713]: pgmap v1591: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.8 MiB/s wr, 625 op/s 2023-09-10T01:29:36.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:36 smithi044 ceph-mon[98676]: pgmap v1591: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.8 MiB/s wr, 625 op/s 2023-09-10T01:29:36.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:36 smithi099 ceph-mon[102692]: pgmap v1591: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.8 MiB/s wr, 625 op/s 2023-09-10T01:29:38.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:38 smithi131 ceph-mon[103713]: pgmap v1592: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.9 MiB/s wr, 966 op/s 2023-09-10T01:29:38.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:38 smithi044 ceph-mon[98676]: pgmap v1592: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.9 MiB/s wr, 966 op/s 2023-09-10T01:29:38.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:38 smithi099 ceph-mon[102692]: pgmap v1592: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.9 MiB/s wr, 966 op/s 2023-09-10T01:29:40.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:40 smithi131 ceph-mon[103713]: pgmap v1593: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 657 op/s 2023-09-10T01:29:40.819 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:40 smithi044 ceph-mon[98676]: pgmap v1593: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 657 op/s 2023-09-10T01:29:40.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:40 smithi099 ceph-mon[102692]: pgmap v1593: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 657 op/s 2023-09-10T01:29:41.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 515.0 s, 147.6 tps, lat 783.138 ms stddev 1312.205 2023-09-10T01:29:42.812 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:42 smithi044 ceph-mon[98676]: pgmap v1594: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.8 MiB/s wr, 728 op/s 2023-09-10T01:29:42.862 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:42 smithi099 ceph-mon[102692]: pgmap v1594: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.8 MiB/s wr, 728 op/s 2023-09-10T01:29:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:42 smithi131 ceph-mon[103713]: pgmap v1594: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.8 MiB/s wr, 728 op/s 2023-09-10T01:29:43.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:43 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:44.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:43 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:44.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:43 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:44.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:44 smithi099 ceph-mon[102692]: pgmap v1595: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 636 op/s 2023-09-10T01:29:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:44 smithi131 ceph-mon[103713]: pgmap v1595: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 636 op/s 2023-09-10T01:29:45.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:44 smithi044 ceph-mon[98676]: pgmap v1595: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 636 op/s 2023-09-10T01:29:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:45.998+0000 7ffb1b505700 1 -- 172.21.15.44:0/2699425761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb14105560 msgr2=0x7ffb14105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:45.998+0000 7ffb1b505700 1 --2- 172.21.15.44:0/2699425761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb14105560 0x7ffb14105940 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009f90 tx=0x7ffb0400b330 comp rx=0 tx=0).stop 2023-09-10T01:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.002+0000 7ffb1b505700 1 -- 172.21.15.44:0/2699425761 shutdown_connections 2023-09-10T01:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.002+0000 7ffb1b505700 1 --2- 172.21.15.44:0/2699425761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb140ffe60 0x7ffb1410c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.002+0000 7ffb1b505700 1 --2- 172.21.15.44:0/2699425761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb14105560 0x7ffb14105940 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.002+0000 7ffb1b505700 1 --2- 172.21.15.44:0/2699425761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb140ff4a0 0x7ffb140ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.002+0000 7ffb1b505700 1 -- 172.21.15.44:0/2699425761 >> 172.21.15.44:0/2699425761 conn(0x7ffb140faf30 msgr2=0x7ffb140fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.004+0000 7ffb1b505700 1 -- 172.21.15.44:0/2699425761 shutdown_connections 2023-09-10T01:29:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.010+0000 7ffb1b505700 1 -- 172.21.15.44:0/2699425761 wait complete. 2023-09-10T01:29:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.011+0000 7ffb1b505700 1 Processor -- start 2023-09-10T01:29:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.011+0000 7ffb1b505700 1 -- start start 2023-09-10T01:29:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.011+0000 7ffb1b505700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb140ff4a0 0x7ffb1419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.011+0000 7ffb1b505700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb140ffe60 0x7ffb141945b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.011+0000 7ffb1b505700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb14194af0 0x7ffb14194fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.011+0000 7ffb1b505700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb14199340 con 0x7ffb14194af0 2023-09-10T01:29:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.011+0000 7ffb1b505700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb14199480 con 0x7ffb140ff4a0 2023-09-10T01:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.011+0000 7ffb1b505700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb14199640 con 0x7ffb140ffe60 2023-09-10T01:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.012+0000 7ffb19aa2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb14194af0 0x7ffb14194fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.012+0000 7ffb18aa0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb140ffe60 0x7ffb141945b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.012+0000 7ffb19aa2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb14194af0 0x7ffb14194fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60200/0 (socket says 172.21.15.44:60200) 2023-09-10T01:29:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.012+0000 7ffb192a1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb140ff4a0 0x7ffb1419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.012+0000 7ffb19aa2700 1 -- 172.21.15.44:0/3945217776 learned_addr learned my addr 172.21.15.44:0/3945217776 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.012+0000 7ffb192a1700 1 -- 172.21.15.44:0/3945217776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb140ffe60 msgr2=0x7ffb141945b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.012+0000 7ffb192a1700 1 --2- 172.21.15.44:0/3945217776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb140ffe60 0x7ffb141945b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.012+0000 7ffb192a1700 1 -- 172.21.15.44:0/3945217776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb14194af0 msgr2=0x7ffb14194fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.012+0000 7ffb192a1700 1 --2- 172.21.15.44:0/3945217776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb14194af0 0x7ffb14194fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.012+0000 7ffb192a1700 1 -- 172.21.15.44:0/3945217776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb04009c40 con 0x7ffb140ff4a0 2023-09-10T01:29:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.016+0000 7ffb192a1700 1 --2- 172.21.15.44:0/3945217776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb140ff4a0 0x7ffb1419b7b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ffb04008000 tx=0x7ffb04007d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.016+0000 7ffb127fc700 1 -- 172.21.15.44:0/3945217776 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0400e040 con 0x7ffb140ff4a0 2023-09-10T01:29:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.017+0000 7ffb127fc700 1 -- 172.21.15.44:0/3945217776 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb04003e10 con 0x7ffb140ff4a0 2023-09-10T01:29:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.017+0000 7ffb127fc700 1 -- 172.21.15.44:0/3945217776 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0401d6b0 con 0x7ffb140ff4a0 2023-09-10T01:29:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.020+0000 7ffb1b505700 1 -- 172.21.15.44:0/3945217776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb141998c0 con 0x7ffb140ff4a0 2023-09-10T01:29:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.020+0000 7ffb1b505700 1 -- 172.21.15.44:0/3945217776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb14199d50 con 0x7ffb140ff4a0 2023-09-10T01:29:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.025+0000 7ffb276cbb80 1 Processor -- start 2023-09-10T01:29:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.029+0000 7ffb127fc700 1 -- 172.21.15.44:0/3945217776 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ffb0401b460 con 0x7ffb140ff4a0 2023-09-10T01:29:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.029+0000 7ffb127fc700 1 --2- 172.21.15.44:0/3945217776 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffafc0a07e0 0x7ffafc0a2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.030+0000 7ffb127fc700 1 -- 172.21.15.44:0/3945217776 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ffb040cd2c0 con 0x7ffb140ff4a0 2023-09-10T01:29:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.030+0000 7ffb276cbb80 1 -- start start 2023-09-10T01:29:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.030+0000 7ffb276cbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c471a0 0x55af73c47580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.030+0000 7ffb276cbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c41300 0x55af73c41780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.030+0000 7ffb276cbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c41cc0 0x55af73c4df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.030+0000 7ffb276cbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af73b9a6d0 con 0x55af73c41300 2023-09-10T01:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.030+0000 7ffb276cbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af73b97ff0 con 0x55af73c471a0 2023-09-10T01:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.030+0000 7ffb276cbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af73b938e0 con 0x55af73c41cc0 2023-09-10T01:29:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.031+0000 7ffb192a1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c471a0 0x55af73c47580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.031+0000 7ffb192a1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c471a0 0x55af73c47580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33716/0 (socket says 172.21.15.44:33716) 2023-09-10T01:29:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.031+0000 7ffb192a1700 1 -- 172.21.15.44:0/69990905 learned_addr learned my addr 172.21.15.44:0/69990905 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.031+0000 7ffb19aa2700 1 --2- 172.21.15.44:0/69990905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c41cc0 0x55af73c4df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.031+0000 7ffb18aa0700 1 --2- 172.21.15.44:0/69990905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c41300 0x55af73c41780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.031+0000 7ffb192a1700 1 -- 172.21.15.44:0/69990905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c41cc0 msgr2=0x55af73c4df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.031+0000 7ffb192a1700 1 --2- 172.21.15.44:0/69990905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c41cc0 0x55af73c4df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.031+0000 7ffb192a1700 1 -- 172.21.15.44:0/69990905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c41300 msgr2=0x55af73c41780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.031+0000 7ffb192a1700 1 --2- 172.21.15.44:0/69990905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c41300 0x55af73c41780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.031+0000 7ffb192a1700 1 -- 172.21.15.44:0/69990905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af739bfd80 con 0x55af73c471a0 2023-09-10T01:29:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.032+0000 7ffb192a1700 1 --2- 172.21.15.44:0/69990905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c471a0 0x55af73c47580 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400b6c0 tx=0x7ffb04099f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=349455471a84b4d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.034+0000 7ffafb7fe700 1 -- 172.21.15.44:0/69990905 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04009420 con 0x55af73c471a0 2023-09-10T01:29:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.034+0000 7ffafb7fe700 1 -- 172.21.15.44:0/69990905 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb0409a3a0 con 0x55af73c471a0 2023-09-10T01:29:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.034+0000 7ffafb7fe700 1 -- 172.21.15.44:0/69990905 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0401b710 con 0x55af73c471a0 2023-09-10T01:29:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.035+0000 7ffb276cbb80 1 -- 172.21.15.44:0/69990905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c471a0 msgr2=0x55af73c47580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.035+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/69990905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c471a0 0x55af73c47580 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400b6c0 tx=0x7ffb04099f60 comp rx=0 tx=0).stop 2023-09-10T01:29:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.036+0000 7ffb19aa2700 1 --2- 172.21.15.44:0/3945217776 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffafc0a07e0 0x7ffafc0a2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.036+0000 7ffb276cbb80 1 -- 172.21.15.44:0/69990905 shutdown_connections 2023-09-10T01:29:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.036+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/69990905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c41cc0 0x55af73c4df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.036+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/69990905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c471a0 0x55af73c47580 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.036+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/69990905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c41300 0x55af73c41780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.036+0000 7ffb276cbb80 1 -- 172.21.15.44:0/69990905 >> 172.21.15.44:0/69990905 conn(0x55af73bb6030 msgr2=0x55af73c3c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.036+0000 7ffb276cbb80 1 -- 172.21.15.44:0/69990905 shutdown_connections 2023-09-10T01:29:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.036+0000 7ffb276cbb80 1 -- 172.21.15.44:0/69990905 wait complete. 2023-09-10T01:29:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.036+0000 7ffb19aa2700 1 --2- 172.21.15.44:0/3945217776 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffafc0a07e0 0x7ffafc0a2ca0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009790 tx=0x7ffb0c0047c0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.036+0000 7ffb276cbb80 1 -- start start 2023-09-10T01:29:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.037+0000 7ffb276cbb80 1 Processor -- start 2023-09-10T01:29:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.037+0000 7ffb276cbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c41300 0x55af73c50d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.037+0000 7ffb276cbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c41cc0 0x55af73c51250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.037+0000 7ffb276cbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c471a0 0x55af73c4af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.037+0000 7ffb276cbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af73b31b10 con 0x55af73c471a0 2023-09-10T01:29:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.037+0000 7ffb276cbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af73b31810 con 0x55af73c41cc0 2023-09-10T01:29:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.037+0000 7ffb276cbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af73a158c0 con 0x55af73c41300 2023-09-10T01:29:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.037+0000 7ffb276cbb80 10 client.0 _ll_register_callbacks cb 0x55af73a7ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:29:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.037+0000 7ffb19aa2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c471a0 0x55af73c4af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.037+0000 7ffb192a1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c41300 0x55af73c50d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.037+0000 7ffb19aa2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c471a0 0x55af73c4af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60216/0 (socket says 172.21.15.44:60216) 2023-09-10T01:29:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.037+0000 7ffb19aa2700 1 -- 172.21.15.44:0/1743834716 learned_addr learned my addr 172.21.15.44:0/1743834716 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.038+0000 7ffb18aa0700 1 --2- 172.21.15.44:0/1743834716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c41cc0 0x55af73c51250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.038+0000 7ffb19aa2700 1 -- 172.21.15.44:0/1743834716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c41300 msgr2=0x55af73c50d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.038+0000 7ffb19aa2700 1 --2- 172.21.15.44:0/1743834716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c41300 0x55af73c50d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.038+0000 7ffb19aa2700 1 -- 172.21.15.44:0/1743834716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c41cc0 msgr2=0x55af73c51250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.038+0000 7ffb19aa2700 1 --2- 172.21.15.44:0/1743834716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c41cc0 0x55af73c51250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.038+0000 7ffb19aa2700 1 -- 172.21.15.44:0/1743834716 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb14199d50 con 0x55af73c471a0 2023-09-10T01:29:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.038+0000 7ffb19aa2700 1 --2- 172.21.15.44:0/1743834716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c471a0 0x55af73c4af60 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c01e8b0 tx=0x7ffb0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803d56bef520dda6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.038+0000 7ffaf8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:29:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.043+0000 7ffaf8ff9700 1 -- 172.21.15.44:0/1743834716 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c023070 con 0x55af73c471a0 2023-09-10T01:29:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.044+0000 7ffaf8ff9700 1 -- 172.21.15.44:0/1743834716 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ffb0c00f650 con 0x55af73c471a0 2023-09-10T01:29:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.048+0000 7ffb276cbb80 1 -- 172.21.15.44:0/1743834716 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af73aeb490 con 0x55af73c471a0 2023-09-10T01:29:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.049+0000 7ffaf8ff9700 1 -- 172.21.15.44:0/1743834716 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=171) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb0c018070 con 0x55af73c471a0 2023-09-10T01:29:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.049+0000 7ffb276cbb80 10 client.20394 fetch_fsmap learned FSMap version 171 2023-09-10T01:29:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.049+0000 7ffb276cbb80 1 -- 172.21.15.44:0/1743834716 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=171,monmap=4+}) v3 -- 0x55af739a8c00 con 0x55af73c471a0 2023-09-10T01:29:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.050+0000 7ffaf8ff9700 1 -- 172.21.15.44:0/1743834716 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55af73aeb490 con 0x55af73c471a0 2023-09-10T01:29:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.050+0000 7ffaf8ff9700 1 -- 172.21.15.44:0/1743834716 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 171) v1 ==== 7371+0+0 (secure 0 0 0) 0x7ffb0c02f070 con 0x55af73c471a0 2023-09-10T01:29:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.050+0000 7ffb276cbb80 10 client.20394 fetch_fsmap finished waiting for FSMap version 171 2023-09-10T01:29:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.050+0000 7ffb276cbb80 10 client.20394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:29:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.050+0000 7ffb276cbb80 20 client.20394 populate_metadata read hostname 'smithi044' 2023-09-10T01:29:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.050+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/1743834716 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af73c5b2c0 0x55af73c5d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.050+0000 7ffb276cbb80 4 client.20394 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:29:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.050+0000 7ffb276cbb80 1 -- 172.21.15.44:0/1743834716 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af73ba8840 con 0x55af73c5b2c0 2023-09-10T01:29:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.050+0000 7ffb192a1700 1 --2- 172.21.15.44:0/1743834716 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af73c5b2c0 0x55af73c5d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.051+0000 7ffaf8ff9700 10 client.20394 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.051+0000 7ffb192a1700 1 --2- 172.21.15.44:0/1743834716 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af73c5b2c0 0x55af73c5d6e0 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c593b5cab2bc8730 server_cookie=b6226e92448bb5dd in_seq=0 out_seq=0 2023-09-10T01:29:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.051+0000 7ffaf8ff9700 1 -- 172.21.15.44:0/1743834716 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55af73ba8840 con 0x55af73c5b2c0 2023-09-10T01:29:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.051+0000 7ffaf8ff9700 10 client.20394 handle_command_reply: tid=0 2023-09-10T01:29:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.051+0000 7ffb276cbb80 1 client.20394 shutdown 2023-09-10T01:29:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.054+0000 7ffb276cbb80 1 -- 172.21.15.44:0/1743834716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c471a0 msgr2=0x55af73c4af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.054+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/1743834716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c471a0 0x55af73c4af60 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c01e8b0 tx=0x7ffb0c004750 comp rx=0 tx=0).stop 2023-09-10T01:29:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.054+0000 7ffb276cbb80 1 -- 172.21.15.44:0/1743834716 shutdown_connections 2023-09-10T01:29:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.054+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/1743834716 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af73c5b2c0 0x55af73c5d6e0 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.054+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/1743834716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c471a0 0x55af73c4af60 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.054+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/1743834716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c41300 0x55af73c50d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.054+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/1743834716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c41cc0 0x55af73c51250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.054+0000 7ffaf8ff9700 0 client.20394 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.054+0000 7ffb276cbb80 1 -- 172.21.15.44:0/1743834716 >> 172.21.15.44:0/1743834716 conn(0x55af73bb6030 msgr2=0x55af73c3b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.054+0000 7ffb276cbb80 1 -- 172.21.15.44:0/1743834716 shutdown_connections 2023-09-10T01:29:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.054+0000 7ffb276cbb80 1 -- 172.21.15.44:0/1743834716 wait complete. 2023-09-10T01:29:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.058+0000 7ffb276cbb80 20 client.20394 trim_cache size 0 max 16384 2023-09-10T01:29:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.091+0000 7ffb276cbb80 1 Processor -- start 2023-09-10T01:29:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.091+0000 7ffb276cbb80 1 -- start start 2023-09-10T01:29:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.091+0000 7ffb276cbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c41300 0x55af73c416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.091+0000 7ffb276cbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c41cc0 0x55af73c4a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.091+0000 7ffb276cbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c471a0 0x55af73c50b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.091+0000 7ffb276cbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af73c39000 con 0x55af73c41cc0 2023-09-10T01:29:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.091+0000 7ffb276cbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af73c4bf40 con 0x55af73c41300 2023-09-10T01:29:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.091+0000 7ffb276cbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af73c4c3d0 con 0x55af73c471a0 2023-09-10T01:29:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.097+0000 7ffb18aa0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c41cc0 0x55af73c4a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.097+0000 7ffb18aa0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c41cc0 0x55af73c4a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60222/0 (socket says 172.21.15.44:60222) 2023-09-10T01:29:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.097+0000 7ffb18aa0700 1 -- 172.21.15.44:0/3325277450 learned_addr learned my addr 172.21.15.44:0/3325277450 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.097+0000 7ffb19aa2700 1 --2- 172.21.15.44:0/3325277450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c471a0 0x55af73c50b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.098+0000 7ffb18aa0700 1 -- 172.21.15.44:0/3325277450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c471a0 msgr2=0x55af73c50b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.098+0000 7ffb18aa0700 1 --2- 172.21.15.44:0/3325277450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c471a0 0x55af73c50b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.098+0000 7ffb18aa0700 1 -- 172.21.15.44:0/3325277450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c41300 msgr2=0x55af73c416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.098+0000 7ffb18aa0700 1 --2- 172.21.15.44:0/3325277450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c41300 0x55af73c416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.098+0000 7ffb18aa0700 1 -- 172.21.15.44:0/3325277450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af739e61f0 con 0x55af73c41cc0 2023-09-10T01:29:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.098+0000 7ffb18aa0700 1 --2- 172.21.15.44:0/3325277450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c41cc0 0x55af73c4a720 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800b860 tx=0x7ffb0800bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ebf88c8a681514 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.099+0000 7ffafbfff700 1 -- 172.21.15.44:0/3325277450 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08012590 con 0x55af73c41cc0 2023-09-10T01:29:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.099+0000 7ffafbfff700 1 -- 172.21.15.44:0/3325277450 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb08012b80 con 0x55af73c41cc0 2023-09-10T01:29:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.099+0000 7ffafbfff700 1 -- 172.21.15.44:0/3325277450 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08015a70 con 0x55af73c41cc0 2023-09-10T01:29:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.100+0000 7ffb276cbb80 1 -- 172.21.15.44:0/3325277450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c41cc0 msgr2=0x55af73c4a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.100+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/3325277450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c41cc0 0x55af73c4a720 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800b860 tx=0x7ffb0800bb70 comp rx=0 tx=0).stop 2023-09-10T01:29:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.102+0000 7ffb276cbb80 1 -- 172.21.15.44:0/3325277450 shutdown_connections 2023-09-10T01:29:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.102+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/3325277450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c471a0 0x55af73c50b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.102+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/3325277450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c41300 0x55af73c416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.102+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/3325277450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c41cc0 0x55af73c4a720 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.102+0000 7ffb276cbb80 1 -- 172.21.15.44:0/3325277450 >> 172.21.15.44:0/3325277450 conn(0x55af73bb6030 msgr2=0x55af73c46070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.102+0000 7ffb276cbb80 1 -- 172.21.15.44:0/3325277450 shutdown_connections 2023-09-10T01:29:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.102+0000 7ffb276cbb80 1 -- 172.21.15.44:0/3325277450 wait complete. 2023-09-10T01:29:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.103+0000 7ffb276cbb80 1 -- start start 2023-09-10T01:29:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.103+0000 7ffb276cbb80 1 Processor -- start 2023-09-10T01:29:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.103+0000 7ffb276cbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c41300 0x55af73c57660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.103+0000 7ffb276cbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c41cc0 0x55af73c55990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.103+0000 7ffb276cbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c471a0 0x55af73c55ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.103+0000 7ffb276cbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af73c565a0 con 0x55af73c471a0 2023-09-10T01:29:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.103+0000 7ffb276cbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af73c56710 con 0x55af73c41cc0 2023-09-10T01:29:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.103+0000 7ffb276cbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af73caf100 con 0x55af73c41300 2023-09-10T01:29:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.103+0000 7ffb276cbb80 10 client.0 _ll_register_callbacks cb 0x55af73a7ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:29:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.104+0000 7ffb18aa0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c41cc0 0x55af73c55990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.104+0000 7ffb18aa0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c41cc0 0x55af73c55990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33736/0 (socket says 172.21.15.44:33736) 2023-09-10T01:29:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.104+0000 7ffb18aa0700 1 -- 172.21.15.44:0/3663373542 learned_addr learned my addr 172.21.15.44:0/3663373542 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.104+0000 7ffb19aa2700 1 --2- 172.21.15.44:0/3663373542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c471a0 0x55af73c55ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.104+0000 7ffb192a1700 1 --2- 172.21.15.44:0/3663373542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c41300 0x55af73c57660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.104+0000 7ffb18aa0700 1 -- 172.21.15.44:0/3663373542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c41300 msgr2=0x55af73c57660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.104+0000 7ffb18aa0700 1 --2- 172.21.15.44:0/3663373542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c41300 0x55af73c57660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.104+0000 7ffb18aa0700 1 -- 172.21.15.44:0/3663373542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c471a0 msgr2=0x55af73c55ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.104+0000 7ffb18aa0700 1 --2- 172.21.15.44:0/3663373542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c471a0 0x55af73c55ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.104+0000 7ffb18aa0700 1 -- 172.21.15.44:0/3663373542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0800e040 con 0x55af73c41cc0 2023-09-10T01:29:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.105+0000 7ffb18aa0700 1 --2- 172.21.15.44:0/3663373542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c41cc0 0x55af73c55990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800b410 tx=0x7ffb08005d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5327b63edbbe471b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.105+0000 7ffaf8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:29:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.105+0000 7ffaf8ff9700 1 -- 172.21.15.44:0/3663373542 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08015670 con 0x55af73c41cc0 2023-09-10T01:29:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.105+0000 7ffaf8ff9700 1 -- 172.21.15.44:0/3663373542 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ffb080127a0 con 0x55af73c41cc0 2023-09-10T01:29:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.105+0000 7ffb276cbb80 1 -- 172.21.15.44:0/3663373542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb0409a3a0 con 0x55af73c41cc0 2023-09-10T01:29:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.105+0000 7ffaf8ff9700 1 -- 172.21.15.44:0/3663373542 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=171) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb08012960 con 0x55af73c41cc0 2023-09-10T01:29:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.111+0000 7ffb276cbb80 10 client.30041 fetch_fsmap learned FSMap version 171 2023-09-10T01:29:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.111+0000 7ffb276cbb80 1 -- 172.21.15.44:0/3663373542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=171,monmap=4+}) v3 -- 0x55af739be250 con 0x55af73c41cc0 2023-09-10T01:29:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.113+0000 7ffaf8ff9700 1 -- 172.21.15.44:0/3663373542 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb0409a3a0 con 0x55af73c41cc0 2023-09-10T01:29:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.114+0000 7ffaf8ff9700 1 -- 172.21.15.44:0/3663373542 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 171) v1 ==== 7371+0+0 (secure 0 0 0) 0x7ffb08006580 con 0x55af73c41cc0 2023-09-10T01:29:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.114+0000 7ffb276cbb80 10 client.30041 fetch_fsmap finished waiting for FSMap version 171 2023-09-10T01:29:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.114+0000 7ffb276cbb80 10 client.30041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:29:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.114+0000 7ffb276cbb80 20 client.30041 populate_metadata read hostname 'smithi044' 2023-09-10T01:29:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.114+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/3663373542 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af73cb0830 0x55af73cb2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.114+0000 7ffb276cbb80 4 client.30041 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:29:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.114+0000 7ffb276cbb80 1 -- 172.21.15.44:0/3663373542 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55af73b8c8c0 con 0x55af73cb0830 2023-09-10T01:29:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.114+0000 7ffb192a1700 1 --2- 172.21.15.44:0/3663373542 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af73cb0830 0x55af73cb2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.114+0000 7ffaf8ff9700 10 client.30041 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.114+0000 7ffb192a1700 1 --2- 172.21.15.44:0/3663373542 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af73cb0830 0x55af73cb2c50 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14fb533aac3d973b server_cookie=a06146f523709385 in_seq=0 out_seq=0 2023-09-10T01:29:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.176+0000 7ffaf8ff9700 1 -- 172.21.15.44:0/3663373542 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55af73b8c8c0 con 0x55af73cb0830 2023-09-10T01:29:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.176+0000 7ffaf8ff9700 10 client.30041 handle_command_reply: tid=0 2023-09-10T01:29:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.180+0000 7ffb276cbb80 1 client.30041 shutdown 2023-09-10T01:29:46.187 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 520.0 s, 285.5 tps, lat 348.480 ms stddev 124.878 2023-09-10T01:29:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.188+0000 7ffb276cbb80 1 -- 172.21.15.44:0/3663373542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c41cc0 msgr2=0x55af73c55990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.188+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/3663373542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c41cc0 0x55af73c55990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800b410 tx=0x7ffb08005d00 comp rx=0 tx=0).stop 2023-09-10T01:29:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.188+0000 7ffb276cbb80 1 -- 172.21.15.44:0/3663373542 shutdown_connections 2023-09-10T01:29:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.188+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/3663373542 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af73cb0830 0x55af73cb2c50 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.188+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/3663373542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af73c471a0 0x55af73c55ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.188+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/3663373542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af73c41300 0x55af73c57660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.188+0000 7ffb276cbb80 1 --2- 172.21.15.44:0/3663373542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af73c41cc0 0x55af73c55990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.188+0000 7ffb276cbb80 1 -- 172.21.15.44:0/3663373542 >> 172.21.15.44:0/3663373542 conn(0x55af73bb6030 msgr2=0x55af73c46070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.190+0000 7ffaf8ff9700 0 client.30041 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.191+0000 7ffb276cbb80 1 -- 172.21.15.44:0/3663373542 shutdown_connections 2023-09-10T01:29:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.193+0000 7ffb276cbb80 1 -- 172.21.15.44:0/3663373542 wait complete. 2023-09-10T01:29:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.194+0000 7ffb276cbb80 20 client.30041 trim_cache size 0 max 16384 2023-09-10T01:29:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:29:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:29:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:29:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:29:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.195+0000 7ffb1b505700 1 -- 172.21.15.44:0/3945217776 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffafc0a07e0 msgr2=0x7ffafc0a2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.195+0000 7ffb1b505700 1 --2- 172.21.15.44:0/3945217776 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffafc0a07e0 0x7ffafc0a2ca0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009790 tx=0x7ffb0c0047c0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.195+0000 7ffb1b505700 1 -- 172.21.15.44:0/3945217776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb140ff4a0 msgr2=0x7ffb1419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.195+0000 7ffb1b505700 1 --2- 172.21.15.44:0/3945217776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb140ff4a0 0x7ffb1419b7b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ffb04008000 tx=0x7ffb04007d00 comp rx=0 tx=0).stop 2023-09-10T01:29:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.195+0000 7ffb1b505700 1 -- 172.21.15.44:0/3945217776 shutdown_connections 2023-09-10T01:29:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.195+0000 7ffb1b505700 1 --2- 172.21.15.44:0/3945217776 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffafc0a07e0 0x7ffafc0a2ca0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.195+0000 7ffb1b505700 1 --2- 172.21.15.44:0/3945217776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb14194af0 0x7ffb14194fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.195+0000 7ffb1b505700 1 --2- 172.21.15.44:0/3945217776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb140ff4a0 0x7ffb1419b7b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.195+0000 7ffb1b505700 1 --2- 172.21.15.44:0/3945217776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb140ffe60 0x7ffb141945b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.195+0000 7ffb1b505700 1 -- 172.21.15.44:0/3945217776 >> 172.21.15.44:0/3945217776 conn(0x7ffb140faf30 msgr2=0x7ffb140fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.195+0000 7ffb1b505700 1 -- 172.21.15.44:0/3945217776 shutdown_connections 2023-09-10T01:29:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:46.200+0000 7ffb1b505700 1 -- 172.21.15.44:0/3945217776 wait complete. 2023-09-10T01:29:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:46 smithi131 ceph-mon[103713]: pgmap v1596: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 486 op/s 2023-09-10T01:29:47.061 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:46 smithi044 ceph-mon[98676]: pgmap v1596: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 486 op/s 2023-09-10T01:29:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:46 smithi099 ceph-mon[102692]: pgmap v1596: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 486 op/s 2023-09-10T01:29:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:48 smithi131 ceph-mon[103713]: pgmap v1597: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.9 MiB/s wr, 846 op/s 2023-09-10T01:29:49.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:48 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:49.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:48 smithi044 ceph-mon[98676]: pgmap v1597: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.9 MiB/s wr, 846 op/s 2023-09-10T01:29:49.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:48 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:48 smithi099 ceph-mon[102692]: pgmap v1597: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.9 MiB/s wr, 846 op/s 2023-09-10T01:29:49.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:48 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:29:51.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:50 smithi131 ceph-mon[103713]: pgmap v1598: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 505 op/s 2023-09-10T01:29:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:50 smithi044 ceph-mon[98676]: pgmap v1598: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 505 op/s 2023-09-10T01:29:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:50 smithi099 ceph-mon[102692]: pgmap v1598: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 505 op/s 2023-09-10T01:29:51.187 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 525.0 s, 263.7 tps, lat 384.139 ms stddev 167.449 2023-09-10T01:29:53.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:52 smithi131 ceph-mon[103713]: pgmap v1599: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.6 MiB/s wr, 684 op/s 2023-09-10T01:29:53.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:52 smithi044 ceph-mon[98676]: pgmap v1599: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.6 MiB/s wr, 684 op/s 2023-09-10T01:29:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:52 smithi099 ceph-mon[102692]: pgmap v1599: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.6 MiB/s wr, 684 op/s 2023-09-10T01:29:54.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:53 smithi131 ceph-mon[103713]: pgmap v1600: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.2 MiB/s wr, 763 op/s 2023-09-10T01:29:54.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:53 smithi044 ceph-mon[98676]: pgmap v1600: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.2 MiB/s wr, 763 op/s 2023-09-10T01:29:54.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:53 smithi099 ceph-mon[102692]: pgmap v1600: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.2 MiB/s wr, 763 op/s 2023-09-10T01:29:56.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:55 smithi099 ceph-mon[102692]: pgmap v1601: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.6 MiB/s wr, 688 op/s 2023-09-10T01:29:56.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 530.0 s, 267.0 tps, lat 325.012 ms stddev 172.947 2023-09-10T01:29:56.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:55 smithi131 ceph-mon[103713]: pgmap v1601: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.6 MiB/s wr, 688 op/s 2023-09-10T01:29:56.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:55 smithi044 ceph-mon[98676]: pgmap v1601: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.6 MiB/s wr, 688 op/s 2023-09-10T01:29:58.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:29:58 smithi131 ceph-mon[103713]: pgmap v1602: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.9 MiB/s wr, 1.06k op/s 2023-09-10T01:29:58.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:29:58 smithi044 ceph-mon[98676]: pgmap v1602: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.9 MiB/s wr, 1.06k op/s 2023-09-10T01:29:58.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:29:58 smithi099 ceph-mon[102692]: pgmap v1602: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.9 MiB/s wr, 1.06k op/s 2023-09-10T01:29:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.427+0000 7f2972951700 1 -- 172.21.15.44:0/1603332413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f296c119de0 msgr2=0x7f296c11c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.427+0000 7f2972951700 1 --2- 172.21.15.44:0/1603332413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f296c119de0 0x7f296c11c1d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2960009990 tx=0x7f2960009ca0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.427+0000 7f2972951700 1 -- 172.21.15.44:0/1603332413 shutdown_connections 2023-09-10T01:29:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.427+0000 7f2972951700 1 --2- 172.21.15.44:0/1603332413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f296c119de0 0x7f296c11c1d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.427+0000 7f2972951700 1 --2- 172.21.15.44:0/1603332413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f296c10ed50 0x7f296c107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.427+0000 7f2972951700 1 --2- 172.21.15.44:0/1603332413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f296c10f1c0 0x7f296c107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.427+0000 7f2972951700 1 -- 172.21.15.44:0/1603332413 >> 172.21.15.44:0/1603332413 conn(0x7f296c06c6c0 msgr2=0x7f296c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.428+0000 7f2972951700 1 -- 172.21.15.44:0/1603332413 shutdown_connections 2023-09-10T01:29:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.428+0000 7f2972951700 1 -- 172.21.15.44:0/1603332413 wait complete. 2023-09-10T01:29:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.429+0000 7f2972951700 1 Processor -- start 2023-09-10T01:29:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.429+0000 7f2972951700 1 -- start start 2023-09-10T01:29:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.429+0000 7f2972951700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f296c10ed50 0x7f296c1190b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.429+0000 7f2972951700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f296c10f1c0 0x7f296c1195f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.429+0000 7f2972951700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f296c119de0 0x7f296c112e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.429+0000 7f2972951700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c113540 con 0x7f296c119de0 2023-09-10T01:29:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.429+0000 7f2972951700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c1136b0 con 0x7f296c10f1c0 2023-09-10T01:29:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.429+0000 7f2972951700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c113870 con 0x7f296c10ed50 2023-09-10T01:29:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.430+0000 7f296bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f296c10ed50 0x7f296c1190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.430+0000 7f296bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f296c10ed50 0x7f296c1190b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38892/0 (socket says 172.21.15.44:38892) 2023-09-10T01:29:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.430+0000 7f296bfff700 1 -- 172.21.15.44:0/4245517517 learned_addr learned my addr 172.21.15.44:0/4245517517 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.430+0000 7f296b7fe700 1 --2- 172.21.15.44:0/4245517517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f296c10f1c0 0x7f296c1195f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.430+0000 7f296b7fe700 1 -- 172.21.15.44:0/4245517517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f296c10ed50 msgr2=0x7f296c1190b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.430+0000 7f296b7fe700 1 --2- 172.21.15.44:0/4245517517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f296c10ed50 0x7f296c1190b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.430+0000 7f296b7fe700 1 -- 172.21.15.44:0/4245517517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f296c119de0 msgr2=0x7f296c112e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.430+0000 7f296b7fe700 1 --2- 172.21.15.44:0/4245517517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f296c119de0 0x7f296c112e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.430+0000 7f296b7fe700 1 -- 172.21.15.44:0/4245517517 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29600095f0 con 0x7f296c10f1c0 2023-09-10T01:29:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.430+0000 7f296b7fe700 1 --2- 172.21.15.44:0/4245517517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f296c10f1c0 0x7f296c1195f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f296c06de40 tx=0x7f295c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.433+0000 7f29697fa700 1 -- 172.21.15.44:0/4245517517 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c009930 con 0x7f296c10f1c0 2023-09-10T01:29:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.433+0000 7f29697fa700 1 -- 172.21.15.44:0/4245517517 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f295c010460 con 0x7f296c10f1c0 2023-09-10T01:29:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.433+0000 7f29697fa700 1 -- 172.21.15.44:0/4245517517 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c00b5d0 con 0x7f296c10f1c0 2023-09-10T01:29:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.433+0000 7f2972951700 1 -- 172.21.15.44:0/4245517517 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296c113b50 con 0x7f296c10f1c0 2023-09-10T01:29:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.433+0000 7f2972951700 1 -- 172.21.15.44:0/4245517517 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296c1bab70 con 0x7f296c10f1c0 2023-09-10T01:29:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.438+0000 7f297eb17b80 1 Processor -- start 2023-09-10T01:29:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.439+0000 7f297eb17b80 1 -- start start 2023-09-10T01:29:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.439+0000 7f297eb17b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de07471a0 0x558de0747580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.439+0000 7f297eb17b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 0x558de0741780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.439+0000 7f297eb17b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de0741cc0 0x558de074df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.439+0000 7f297eb17b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558de069a6d0 con 0x558de07471a0 2023-09-10T01:29:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.439+0000 7f297eb17b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558de0697ff0 con 0x558de0741300 2023-09-10T01:29:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.439+0000 7f297eb17b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558de06938e0 con 0x558de0741cc0 2023-09-10T01:29:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.440+0000 7f29697fa700 1 -- 172.21.15.44:0/4245517517 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f295c0105d0 con 0x7f296c10f1c0 2023-09-10T01:29:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.440+0000 7f296b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de0741cc0 0x558de074df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.440+0000 7f2970eee700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de07471a0 0x558de0747580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.440+0000 7f2970eee700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de07471a0 0x558de0747580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60250/0 (socket says 172.21.15.44:60250) 2023-09-10T01:29:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.440+0000 7f2970eee700 1 -- 172.21.15.44:0/3707315404 learned_addr learned my addr 172.21.15.44:0/3707315404 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.441+0000 7f2970eee700 1 -- 172.21.15.44:0/3707315404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de0741cc0 msgr2=0x558de074df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.441+0000 7f2970eee700 1 --2- 172.21.15.44:0/3707315404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de0741cc0 0x558de074df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.441+0000 7f2970eee700 1 -- 172.21.15.44:0/3707315404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 msgr2=0x558de0741780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:29:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.441+0000 7f2970eee700 1 --2- 172.21.15.44:0/3707315404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 0x558de0741780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.441+0000 7f2970eee700 1 -- 172.21.15.44:0/3707315404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558de04bfd80 con 0x558de07471a0 2023-09-10T01:29:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.441+0000 7f29697fa700 1 --2- 172.21.15.44:0/4245517517 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29540a0940 0x7f29540a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.441+0000 7f2970eee700 1 --2- 172.21.15.44:0/3707315404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de07471a0 0x558de0747580 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f2960005b30 tx=0x7f2960009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c120c1dd96d464e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.441+0000 7f29697fa700 1 -- 172.21.15.44:0/4245517517 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f295c0cbdc0 con 0x7f296c10f1c0 2023-09-10T01:29:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.445+0000 7f296bfff700 1 --2- 172.21.15.44:0/4245517517 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29540a0940 0x7f29540a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.446+0000 7f296bfff700 1 --2- 172.21.15.44:0/4245517517 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29540a0940 0x7f29540a2e00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x558de073c060 tx=0x7f296400b040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.458+0000 7f29527fc700 1 -- 172.21.15.44:0/3707315404 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296001d070 con 0x558de07471a0 2023-09-10T01:29:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.458+0000 7f29527fc700 1 -- 172.21.15.44:0/3707315404 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f29600054e0 con 0x558de07471a0 2023-09-10T01:29:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.458+0000 7f29527fc700 1 -- 172.21.15.44:0/3707315404 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960004500 con 0x558de07471a0 2023-09-10T01:29:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.462+0000 7f297eb17b80 1 -- 172.21.15.44:0/3707315404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de07471a0 msgr2=0x558de0747580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.462+0000 7f297eb17b80 1 --2- 172.21.15.44:0/3707315404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de07471a0 0x558de0747580 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f2960005b30 tx=0x7f2960009510 comp rx=0 tx=0).stop 2023-09-10T01:29:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.463+0000 7f297eb17b80 1 -- 172.21.15.44:0/3707315404 shutdown_connections 2023-09-10T01:29:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.463+0000 7f297eb17b80 1 --2- 172.21.15.44:0/3707315404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de0741cc0 0x558de074df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.463+0000 7f297eb17b80 1 --2- 172.21.15.44:0/3707315404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de07471a0 0x558de0747580 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.463+0000 7f297eb17b80 1 --2- 172.21.15.44:0/3707315404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 0x558de0741780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.463+0000 7f297eb17b80 1 -- 172.21.15.44:0/3707315404 >> 172.21.15.44:0/3707315404 conn(0x558de06b6030 msgr2=0x558de073c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.464+0000 7f297eb17b80 1 -- 172.21.15.44:0/3707315404 shutdown_connections 2023-09-10T01:29:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.466+0000 7f297eb17b80 1 -- 172.21.15.44:0/3707315404 wait complete. 2023-09-10T01:29:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.466+0000 7f297eb17b80 1 -- start start 2023-09-10T01:29:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.466+0000 7f297eb17b80 1 Processor -- start 2023-09-10T01:29:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.467+0000 7f297eb17b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de0741300 0x558de074b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.467+0000 7f297eb17b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741cc0 0x558de074b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.467+0000 7f297eb17b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de07471a0 0x558de074bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.467+0000 7f297eb17b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558de0631b10 con 0x558de0741300 2023-09-10T01:29:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.467+0000 7f297eb17b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558de0631810 con 0x558de0741cc0 2023-09-10T01:29:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.467+0000 7f297eb17b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558de05158c0 con 0x558de07471a0 2023-09-10T01:29:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.467+0000 7f297eb17b80 10 client.0 _ll_register_callbacks cb 0x558de057ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:29:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.467+0000 7f2970eee700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de07471a0 0x558de074bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.467+0000 7f2970eee700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de07471a0 0x558de074bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38906/0 (socket says 172.21.15.44:38906) 2023-09-10T01:29:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.467+0000 7f2970eee700 1 -- 172.21.15.44:0/4260500061 learned_addr learned my addr 172.21.15.44:0/4260500061 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.467+0000 7f296bfff700 1 --2- 172.21.15.44:0/4260500061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de0741300 0x558de074b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.468+0000 7f2970eee700 1 -- 172.21.15.44:0/4260500061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741cc0 msgr2=0x558de074b700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:29:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.468+0000 7f2970eee700 1 --2- 172.21.15.44:0/4260500061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741cc0 0x558de074b700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.468+0000 7f2970eee700 1 -- 172.21.15.44:0/4260500061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de0741300 msgr2=0x558de074b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.468+0000 7f2970eee700 1 --2- 172.21.15.44:0/4260500061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de0741300 0x558de074b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.468+0000 7f2970eee700 1 -- 172.21.15.44:0/4260500061 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296c11cb00 con 0x558de07471a0 2023-09-10T01:29:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.468+0000 7f2970eee700 1 --2- 172.21.15.44:0/4260500061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de07471a0 0x558de074bc40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2960005480 tx=0x7f2960005690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6374efb696795c84 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.468+0000 7f294b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:29:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.479+0000 7f294b7fe700 1 -- 172.21.15.44:0/4260500061 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296001d070 con 0x558de07471a0 2023-09-10T01:29:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.479+0000 7f294b7fe700 1 -- 172.21.15.44:0/4260500061 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2960004500 con 0x558de07471a0 2023-09-10T01:29:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.479+0000 7f297eb17b80 1 -- 172.21.15.44:0/4260500061 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558de05eb490 con 0x558de07471a0 2023-09-10T01:29:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.479+0000 7f294b7fe700 1 -- 172.21.15.44:0/4260500061 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=172) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29600046c0 con 0x558de07471a0 2023-09-10T01:29:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.479+0000 7f297eb17b80 10 client.30064 fetch_fsmap learned FSMap version 172 2023-09-10T01:29:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.479+0000 7f297eb17b80 1 -- 172.21.15.44:0/4260500061 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=172,monmap=4+}) v3 -- 0x558de04a8c00 con 0x558de07471a0 2023-09-10T01:29:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.480+0000 7f294b7fe700 1 -- 172.21.15.44:0/4260500061 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558de05eb490 con 0x558de07471a0 2023-09-10T01:29:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.480+0000 7f294b7fe700 1 -- 172.21.15.44:0/4260500061 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 172) v1 ==== 7363+0+0 (secure 0 0 0) 0x7f2960003c70 con 0x558de07471a0 2023-09-10T01:29:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.480+0000 7f297eb17b80 10 client.30064 fetch_fsmap finished waiting for FSMap version 172 2023-09-10T01:29:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.480+0000 7f297eb17b80 10 client.30064 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:29:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.480+0000 7f297eb17b80 20 client.30064 populate_metadata read hostname 'smithi044' 2023-09-10T01:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.480+0000 7f297eb17b80 1 --2- 172.21.15.44:0/4260500061 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558de075afc0 0x558de075d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.480+0000 7f297eb17b80 4 client.30064 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.480+0000 7f297eb17b80 1 -- 172.21.15.44:0/4260500061 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558de06a8840 con 0x558de075afc0 2023-09-10T01:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.481+0000 7f296bfff700 1 --2- 172.21.15.44:0/4260500061 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558de075afc0 0x558de075d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.492+0000 7f296bfff700 1 --2- 172.21.15.44:0/4260500061 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558de075afc0 0x558de075d3e0 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=459db16ed1622372 server_cookie=a3d0251add2ac36 in_seq=0 out_seq=0 2023-09-10T01:29:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.492+0000 7f294b7fe700 10 client.30064 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.495+0000 7f294b7fe700 1 -- 172.21.15.44:0/4260500061 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558de06a8840 con 0x558de075afc0 2023-09-10T01:29:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.495+0000 7f294b7fe700 10 client.30064 handle_command_reply: tid=0 2023-09-10T01:29:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.500+0000 7f297eb17b80 1 client.30064 shutdown 2023-09-10T01:29:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.501+0000 7f297eb17b80 1 -- 172.21.15.44:0/4260500061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de07471a0 msgr2=0x558de074bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.501+0000 7f297eb17b80 1 --2- 172.21.15.44:0/4260500061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de07471a0 0x558de074bc40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2960005480 tx=0x7f2960005690 comp rx=0 tx=0).stop 2023-09-10T01:29:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.508+0000 7f297eb17b80 1 -- 172.21.15.44:0/4260500061 shutdown_connections 2023-09-10T01:29:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.508+0000 7f297eb17b80 1 --2- 172.21.15.44:0/4260500061 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558de075afc0 0x558de075d3e0 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.508+0000 7f297eb17b80 1 --2- 172.21.15.44:0/4260500061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de07471a0 0x558de074bc40 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.508+0000 7f297eb17b80 1 --2- 172.21.15.44:0/4260500061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de0741300 0x558de074b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.508+0000 7f297eb17b80 1 --2- 172.21.15.44:0/4260500061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741cc0 0x558de074b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.508+0000 7f297eb17b80 1 -- 172.21.15.44:0/4260500061 >> 172.21.15.44:0/4260500061 conn(0x558de06b6030 msgr2=0x558de073b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.508+0000 7f294b7fe700 0 client.30064 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.509+0000 7f297eb17b80 1 -- 172.21.15.44:0/4260500061 shutdown_connections 2023-09-10T01:29:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.515+0000 7f297eb17b80 1 -- 172.21.15.44:0/4260500061 wait complete. 2023-09-10T01:29:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.515+0000 7f297eb17b80 20 client.30064 trim_cache size 0 max 16384 2023-09-10T01:29:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.540+0000 7f297eb17b80 1 Processor -- start 2023-09-10T01:29:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.541+0000 7f297eb17b80 1 -- start start 2023-09-10T01:29:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.541+0000 7f297eb17b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 0x558de0748df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.541+0000 7f297eb17b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de0741cc0 0x558de0752420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.541+0000 7f297eb17b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de07471a0 0x558de07452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.541+0000 7f297eb17b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558de06a8840 con 0x558de07471a0 2023-09-10T01:29:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.541+0000 7f297eb17b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558de05eb490 con 0x558de0741300 2023-09-10T01:29:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.541+0000 7f297eb17b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558de07368a0 con 0x558de0741cc0 2023-09-10T01:29:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.541+0000 7f2970eee700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de07471a0 0x558de07452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.541+0000 7f2970eee700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de07471a0 0x558de07452d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60262/0 (socket says 172.21.15.44:60262) 2023-09-10T01:29:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.541+0000 7f2970eee700 1 -- 172.21.15.44:0/2227043090 learned_addr learned my addr 172.21.15.44:0/2227043090 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.541+0000 7f296b7fe700 1 --2- 172.21.15.44:0/2227043090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de0741cc0 0x558de0752420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.541+0000 7f296bfff700 1 --2- 172.21.15.44:0/2227043090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 0x558de0748df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.542+0000 7f296bfff700 1 -- 172.21.15.44:0/2227043090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de0741cc0 msgr2=0x558de0752420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.542+0000 7f296bfff700 1 --2- 172.21.15.44:0/2227043090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de0741cc0 0x558de0752420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.542+0000 7f296bfff700 1 -- 172.21.15.44:0/2227043090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de07471a0 msgr2=0x558de07452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.542+0000 7f296bfff700 1 --2- 172.21.15.44:0/2227043090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de07471a0 0x558de07452d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.542+0000 7f296bfff700 1 -- 172.21.15.44:0/2227043090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558de04e61f0 con 0x558de0741300 2023-09-10T01:29:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.542+0000 7f2970eee700 1 --2- 172.21.15.44:0/2227043090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de07471a0 0x558de07452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:29:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.542+0000 7f296b7fe700 1 --2- 172.21.15.44:0/2227043090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de0741cc0 0x558de0752420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:29:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.542+0000 7f296bfff700 1 --2- 172.21.15.44:0/2227043090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 0x558de0748df0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x558de075dcc0 tx=0x7f2964011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=722da612dd2f77db server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.542+0000 7f2952ffd700 1 -- 172.21.15.44:0/2227043090 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964022cf0 con 0x558de0741300 2023-09-10T01:29:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.543+0000 7f2952ffd700 1 -- 172.21.15.44:0/2227043090 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f296402a070 con 0x558de0741300 2023-09-10T01:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.543+0000 7f2952ffd700 1 -- 172.21.15.44:0/2227043090 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964015650 con 0x558de0741300 2023-09-10T01:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.543+0000 7f297eb17b80 1 -- 172.21.15.44:0/2227043090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 msgr2=0x558de0748df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.544+0000 7f297eb17b80 1 --2- 172.21.15.44:0/2227043090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 0x558de0748df0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x558de075dcc0 tx=0x7f2964011000 comp rx=0 tx=0).stop 2023-09-10T01:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.544+0000 7f297eb17b80 1 -- 172.21.15.44:0/2227043090 shutdown_connections 2023-09-10T01:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.544+0000 7f297eb17b80 1 --2- 172.21.15.44:0/2227043090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de07471a0 0x558de07452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.544+0000 7f297eb17b80 1 --2- 172.21.15.44:0/2227043090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 0x558de0748df0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.544+0000 7f297eb17b80 1 --2- 172.21.15.44:0/2227043090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de0741cc0 0x558de0752420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.544+0000 7f297eb17b80 1 -- 172.21.15.44:0/2227043090 >> 172.21.15.44:0/2227043090 conn(0x558de06b6030 msgr2=0x558de0746250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.546+0000 7f297eb17b80 1 -- 172.21.15.44:0/2227043090 shutdown_connections 2023-09-10T01:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.546+0000 7f297eb17b80 1 -- 172.21.15.44:0/2227043090 wait complete. 2023-09-10T01:29:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.546+0000 7f297eb17b80 1 -- start start 2023-09-10T01:29:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.546+0000 7f297eb17b80 1 Processor -- start 2023-09-10T01:29:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.547+0000 7f297eb17b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 0x558de07a8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.547+0000 7f297eb17b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de0741cc0 0x558de07a87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.547+0000 7f297eb17b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de07471a0 0x558de07a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.547+0000 7f297eb17b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558de07a3360 con 0x558de0741cc0 2023-09-10T01:29:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.547+0000 7f297eb17b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558de07a34d0 con 0x558de0741300 2023-09-10T01:29:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.547+0000 7f297eb17b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558de07a3690 con 0x558de07471a0 2023-09-10T01:29:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.547+0000 7f297eb17b80 10 client.0 _ll_register_callbacks cb 0x7f295c0cf7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:29:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.547+0000 7f2970eee700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de07471a0 0x558de07a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.547+0000 7f296bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 0x558de07a8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.547+0000 7f2970eee700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de07471a0 0x558de07a2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38922/0 (socket says 172.21.15.44:38922) 2023-09-10T01:29:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.547+0000 7f296bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 0x558de07a8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33778/0 (socket says 172.21.15.44:33778) 2023-09-10T01:29:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.547+0000 7f2970eee700 1 -- 172.21.15.44:0/108365764 learned_addr learned my addr 172.21.15.44:0/108365764 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:29:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.547+0000 7f296b7fe700 1 --2- 172.21.15.44:0/108365764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de0741cc0 0x558de07a87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.547+0000 7f296bfff700 1 -- 172.21.15.44:0/108365764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de07471a0 msgr2=0x558de07a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.548+0000 7f296bfff700 1 --2- 172.21.15.44:0/108365764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de07471a0 0x558de07a2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.548+0000 7f296bfff700 1 -- 172.21.15.44:0/108365764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de0741cc0 msgr2=0x558de07a87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:29:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.548+0000 7f296bfff700 1 --2- 172.21.15.44:0/108365764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de0741cc0 0x558de07a87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.548+0000 7f296bfff700 1 -- 172.21.15.44:0/108365764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296c1bab70 con 0x558de0741300 2023-09-10T01:29:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.548+0000 7f296b7fe700 1 --2- 172.21.15.44:0/108365764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de0741cc0 0x558de07a87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:29:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.548+0000 7f296bfff700 1 --2- 172.21.15.44:0/108365764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 0x558de07a8260 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2964009e60 tx=0x7f2964022af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa992967b078aa99 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:29:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.548+0000 7f294b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:29:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.557+0000 7f294b7fe700 1 -- 172.21.15.44:0/108365764 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964027070 con 0x558de0741300 2023-09-10T01:29:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.557+0000 7f294b7fe700 1 -- 172.21.15.44:0/108365764 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f296402c070 con 0x558de0741300 2023-09-10T01:29:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.557+0000 7f297eb17b80 1 -- 172.21.15.44:0/108365764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29600054e0 con 0x558de0741300 2023-09-10T01:29:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.563+0000 7f294b7fe700 1 -- 172.21.15.44:0/108365764 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=172) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2964022e90 con 0x558de0741300 2023-09-10T01:29:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.563+0000 7f297eb17b80 10 client.30053 fetch_fsmap learned FSMap version 172 2023-09-10T01:29:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.563+0000 7f297eb17b80 1 -- 172.21.15.44:0/108365764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=172,monmap=4+}) v3 -- 0x558de04a8c00 con 0x558de0741300 2023-09-10T01:29:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.566+0000 7f294b7fe700 1 -- 172.21.15.44:0/108365764 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f29600054e0 con 0x558de0741300 2023-09-10T01:29:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.566+0000 7f294b7fe700 1 -- 172.21.15.44:0/108365764 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 172) v1 ==== 7363+0+0 (secure 0 0 0) 0x7f296403c8b0 con 0x558de0741300 2023-09-10T01:29:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.566+0000 7f297eb17b80 10 client.30053 fetch_fsmap finished waiting for FSMap version 172 2023-09-10T01:29:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.566+0000 7f297eb17b80 10 client.30053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:29:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.566+0000 7f297eb17b80 20 client.30053 populate_metadata read hostname 'smithi044' 2023-09-10T01:29:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.566+0000 7f297eb17b80 1 --2- 172.21.15.44:0/108365764 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558de07b0020 0x558de07b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:29:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.566+0000 7f297eb17b80 4 client.30053 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:29:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.566+0000 7f297eb17b80 1 -- 172.21.15.44:0/108365764 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558de068c8c0 con 0x558de07b0020 2023-09-10T01:29:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.566+0000 7f296b7fe700 1 --2- 172.21.15.44:0/108365764 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558de07b0020 0x558de07b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:29:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.573+0000 7f296b7fe700 1 --2- 172.21.15.44:0/108365764 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558de07b0020 0x558de07b2440 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51e1004e4e339cc7 server_cookie=1ba0824b1f12aa51 in_seq=0 out_seq=0 2023-09-10T01:29:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.573+0000 7f294b7fe700 10 client.30053 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.603+0000 7f294b7fe700 1 -- 172.21.15.44:0/108365764 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558de068c8c0 con 0x558de07b0020 2023-09-10T01:29:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.603+0000 7f294b7fe700 10 client.30053 handle_command_reply: tid=0 2023-09-10T01:29:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.603+0000 7f297eb17b80 1 client.30053 shutdown 2023-09-10T01:29:59.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.609+0000 7f297eb17b80 1 -- 172.21.15.44:0/108365764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 msgr2=0x558de07a8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.609+0000 7f297eb17b80 1 --2- 172.21.15.44:0/108365764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 0x558de07a8260 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2964009e60 tx=0x7f2964022af0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.609+0000 7f297eb17b80 1 -- 172.21.15.44:0/108365764 shutdown_connections 2023-09-10T01:29:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.609+0000 7f297eb17b80 1 --2- 172.21.15.44:0/108365764 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x558de07b0020 0x558de07b2440 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.609+0000 7f297eb17b80 1 --2- 172.21.15.44:0/108365764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558de07471a0 0x558de07a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.609+0000 7f297eb17b80 1 --2- 172.21.15.44:0/108365764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558de0741300 0x558de07a8260 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.609+0000 7f297eb17b80 1 --2- 172.21.15.44:0/108365764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558de0741cc0 0x558de07a87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.609+0000 7f297eb17b80 1 -- 172.21.15.44:0/108365764 >> 172.21.15.44:0/108365764 conn(0x558de06b6030 msgr2=0x558de0746250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.611+0000 7f294b7fe700 0 client.30053 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:29:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.611+0000 7f297eb17b80 1 -- 172.21.15.44:0/108365764 shutdown_connections 2023-09-10T01:29:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.611+0000 7f297eb17b80 1 -- 172.21.15.44:0/108365764 wait complete. 2023-09-10T01:29:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.611+0000 7f297eb17b80 20 client.30053 trim_cache size 0 max 16384 2023-09-10T01:29:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:29:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:29:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:29:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:29:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.612+0000 7f2972951700 1 -- 172.21.15.44:0/4245517517 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29540a0940 msgr2=0x7f29540a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.612+0000 7f2972951700 1 --2- 172.21.15.44:0/4245517517 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29540a0940 0x7f29540a2e00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x558de073c060 tx=0x7f296400b040 comp rx=0 tx=0).stop 2023-09-10T01:29:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.612+0000 7f2972951700 1 -- 172.21.15.44:0/4245517517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f296c10f1c0 msgr2=0x7f296c1195f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:29:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.612+0000 7f2972951700 1 --2- 172.21.15.44:0/4245517517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f296c10f1c0 0x7f296c1195f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f296c06de40 tx=0x7f295c00dc30 comp rx=0 tx=0).stop 2023-09-10T01:29:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.615+0000 7f2972951700 1 -- 172.21.15.44:0/4245517517 shutdown_connections 2023-09-10T01:29:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.615+0000 7f2972951700 1 --2- 172.21.15.44:0/4245517517 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f29540a0940 0x7f29540a2e00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.615+0000 7f2972951700 1 --2- 172.21.15.44:0/4245517517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f296c119de0 0x7f296c112e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.615+0000 7f2972951700 1 --2- 172.21.15.44:0/4245517517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f296c10ed50 0x7f296c1190b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.615+0000 7f2972951700 1 --2- 172.21.15.44:0/4245517517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f296c10f1c0 0x7f296c1195f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:29:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.615+0000 7f2972951700 1 -- 172.21.15.44:0/4245517517 >> 172.21.15.44:0/4245517517 conn(0x7f296c06c6c0 msgr2=0x7f296c070fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:29:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.618+0000 7f2972951700 1 -- 172.21.15.44:0/4245517517 shutdown_connections 2023-09-10T01:29:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:29:59.618+0000 7f2972951700 1 -- 172.21.15.44:0/4245517517 wait complete. 2023-09-10T01:30:00.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:00 smithi131 ceph-mon[103713]: pgmap v1603: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 698 op/s 2023-09-10T01:30:00.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:00 smithi131 ceph-mon[103713]: overall HEALTH_WARN 1 failed cephadm daemon(s) 2023-09-10T01:30:00.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:00 smithi044 ceph-mon[98676]: pgmap v1603: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 698 op/s 2023-09-10T01:30:00.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:00 smithi044 ceph-mon[98676]: overall HEALTH_WARN 1 failed cephadm daemon(s) 2023-09-10T01:30:00.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:00 smithi099 ceph-mon[102692]: pgmap v1603: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 698 op/s 2023-09-10T01:30:00.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:00 smithi099 ceph-mon[102692]: overall HEALTH_WARN 1 failed cephadm daemon(s) 2023-09-10T01:30:01.185 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 535.0 s, 219.4 tps, lat 337.195 ms stddev 460.833 2023-09-10T01:30:02.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:02 smithi131 ceph-mon[103713]: pgmap v1604: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.3 MiB/s wr, 885 op/s 2023-09-10T01:30:02.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:02 smithi044 ceph-mon[98676]: pgmap v1604: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.3 MiB/s wr, 885 op/s 2023-09-10T01:30:02.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:02 smithi099 ceph-mon[102692]: pgmap v1604: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.3 MiB/s wr, 885 op/s 2023-09-10T01:30:03.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:03 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:30:03.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:03 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:30:03.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:03 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:30:04.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:04 smithi044 ceph-mon[98676]: pgmap v1605: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.0 MiB/s wr, 874 op/s 2023-09-10T01:30:04.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:04 smithi099 ceph-mon[102692]: pgmap v1605: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.0 MiB/s wr, 874 op/s 2023-09-10T01:30:05.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:04 smithi131 ceph-mon[103713]: pgmap v1605: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.0 MiB/s wr, 874 op/s 2023-09-10T01:30:06.182 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 540.0 s, 108.5 tps, lat 249.300 ms stddev 363.658 2023-09-10T01:30:06.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:06 smithi099 ceph-mon[102692]: pgmap v1606: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.6 MiB/s wr, 725 op/s 2023-09-10T01:30:07.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:06 smithi131 ceph-mon[103713]: pgmap v1606: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.6 MiB/s wr, 725 op/s 2023-09-10T01:30:07.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:06 smithi044 ceph-mon[98676]: pgmap v1606: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.6 MiB/s wr, 725 op/s 2023-09-10T01:30:09.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:08 smithi131 ceph-mon[103713]: pgmap v1607: 129 pgs: 129 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 982 op/s 2023-09-10T01:30:09.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:08 smithi044 ceph-mon[98676]: pgmap v1607: 129 pgs: 129 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 982 op/s 2023-09-10T01:30:09.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:08 smithi099 ceph-mon[102692]: pgmap v1607: 129 pgs: 129 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 982 op/s 2023-09-10T01:30:10.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:09 smithi131 ceph-mon[103713]: osdmap e193: 12 total, 12 up, 12 in 2023-09-10T01:30:10.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:09 smithi044 ceph-mon[98676]: osdmap e193: 12 total, 12 up, 12 in 2023-09-10T01:30:10.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:09 smithi099 ceph-mon[102692]: osdmap e193: 12 total, 12 up, 12 in 2023-09-10T01:30:11.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:10 smithi131 ceph-mon[103713]: pgmap v1609: 129 pgs: 129 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.3 MiB/s wr, 770 op/s 2023-09-10T01:30:11.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:10 smithi044 ceph-mon[98676]: pgmap v1609: 129 pgs: 129 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.3 MiB/s wr, 770 op/s 2023-09-10T01:30:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:10 smithi099 ceph-mon[102692]: pgmap v1609: 129 pgs: 129 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.3 MiB/s wr, 770 op/s 2023-09-10T01:30:11.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 545.0 s, 103.0 tps, lat 1896.820 ms stddev 3000.851 2023-09-10T01:30:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.855+0000 7fa65720f700 1 -- 172.21.15.44:0/202462967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa65010f1c0 msgr2=0x7fa650107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.855+0000 7fa65720f700 1 --2- 172.21.15.44:0/202462967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa65010f1c0 0x7fa650107b40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa6400098f0 tx=0x7fa640009c00 comp rx=0 tx=0).stop 2023-09-10T01:30:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.863+0000 7fa65720f700 1 -- 172.21.15.44:0/202462967 shutdown_connections 2023-09-10T01:30:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.863+0000 7fa65720f700 1 --2- 172.21.15.44:0/202462967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa650119de0 0x7fa65011c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.863+0000 7fa65720f700 1 --2- 172.21.15.44:0/202462967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa65010ed50 0x7fa650107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.863+0000 7fa65720f700 1 --2- 172.21.15.44:0/202462967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa65010f1c0 0x7fa650107b40 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.863+0000 7fa65720f700 1 -- 172.21.15.44:0/202462967 >> 172.21.15.44:0/202462967 conn(0x7fa65006c6c0 msgr2=0x7fa65006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.873+0000 7fa65720f700 1 -- 172.21.15.44:0/202462967 shutdown_connections 2023-09-10T01:30:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.877+0000 7fa65720f700 1 -- 172.21.15.44:0/202462967 wait complete. 2023-09-10T01:30:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.877+0000 7fa65720f700 1 Processor -- start 2023-09-10T01:30:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.878+0000 7fa65720f700 1 -- start start 2023-09-10T01:30:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.878+0000 7fa65720f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa65010ed50 0x7fa6501a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.878+0000 7fa65720f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65010f1c0 0x7fa6501a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.878+0000 7fa65720f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa650119de0 0x7fa6501a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.878+0000 7fa65720f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6501aa620 con 0x7fa65010ed50 2023-09-10T01:30:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.878+0000 7fa65720f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6501aa790 con 0x7fa65010f1c0 2023-09-10T01:30:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.878+0000 7fa65720f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6501aa950 con 0x7fa650119de0 2023-09-10T01:30:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.882+0000 7fa6557ac700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa650119de0 0x7fa6501a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.882+0000 7fa64ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65010f1c0 0x7fa6501a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.883+0000 7fa6557ac700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa650119de0 0x7fa6501a9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38930/0 (socket says 172.21.15.44:38930) 2023-09-10T01:30:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.883+0000 7fa6557ac700 1 -- 172.21.15.44:0/159115541 learned_addr learned my addr 172.21.15.44:0/159115541 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.883+0000 7fa6557ac700 1 -- 172.21.15.44:0/159115541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65010f1c0 msgr2=0x7fa6501a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.883+0000 7fa6557ac700 1 --2- 172.21.15.44:0/159115541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65010f1c0 0x7fa6501a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.883+0000 7fa6557ac700 1 -- 172.21.15.44:0/159115541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa65010ed50 msgr2=0x7fa6501a5430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T01:30:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.883+0000 7fa6557ac700 1 --2- 172.21.15.44:0/159115541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa65010ed50 0x7fa6501a5430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.883+0000 7fa6557ac700 1 -- 172.21.15.44:0/159115541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa640009520 con 0x7fa650119de0 2023-09-10T01:30:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.883+0000 7fa64ffff700 1 --2- 172.21.15.44:0/159115541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65010f1c0 0x7fa6501a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:30:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.883+0000 7fa6557ac700 1 --2- 172.21.15.44:0/159115541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa650119de0 0x7fa6501a9fe0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa64400b450 tx=0x7fa64400b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.884+0000 7fa64dffb700 1 -- 172.21.15.44:0/159115541 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644011830 con 0x7fa650119de0 2023-09-10T01:30:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.884+0000 7fa64dffb700 1 -- 172.21.15.44:0/159115541 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa644011e20 con 0x7fa650119de0 2023-09-10T01:30:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.884+0000 7fa64dffb700 1 -- 172.21.15.44:0/159115541 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644010d40 con 0x7fa650119de0 2023-09-10T01:30:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.884+0000 7fa65720f700 1 -- 172.21.15.44:0/159115541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6501aac30 con 0x7fa650119de0 2023-09-10T01:30:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.884+0000 7fa65720f700 1 -- 172.21.15.44:0/159115541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6501b2470 con 0x7fa650119de0 2023-09-10T01:30:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.890+0000 7fa64dffb700 1 -- 172.21.15.44:0/159115541 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa644011990 con 0x7fa650119de0 2023-09-10T01:30:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.891+0000 7fa64dffb700 1 --2- 172.21.15.44:0/159115541 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa6380a08b0 0x7fa6380a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.892+0000 7fa64dffb700 1 -- 172.21.15.44:0/159115541 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fa6440cbdc0 con 0x7fa650119de0 2023-09-10T01:30:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.894+0000 7fa6633d5b80 1 Processor -- start 2023-09-10T01:30:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.894+0000 7fa654fab700 1 --2- 172.21.15.44:0/159115541 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa6380a08b0 0x7fa6380a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.894+0000 7fa6633d5b80 1 -- start start 2023-09-10T01:30:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.894+0000 7fa6633d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 0x559ce7183580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.894+0000 7fa6633d5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce717d300 0x559ce717d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.894+0000 7fa6633d5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717dcc0 0x559ce7189f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.894+0000 7fa6633d5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce70d66d0 con 0x559ce717d300 2023-09-10T01:30:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.894+0000 7fa6633d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce70d3ff0 con 0x559ce71831a0 2023-09-10T01:30:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.894+0000 7fa6633d5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce70cf8e0 con 0x559ce717dcc0 2023-09-10T01:30:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.895+0000 7fa6557ac700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717dcc0 0x559ce7189f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.895+0000 7fa6557ac700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717dcc0 0x559ce7189f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38936/0 (socket says 172.21.15.44:38936) 2023-09-10T01:30:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.895+0000 7fa6557ac700 1 -- 172.21.15.44:0/2415069567 learned_addr learned my addr 172.21.15.44:0/2415069567 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.895+0000 7fa654fab700 1 --2- 172.21.15.44:0/159115541 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa6380a08b0 0x7fa6380a2d70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa6480060b0 tx=0x7fa648006040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.895+0000 7fa654fab700 1 --2- 172.21.15.44:0/2415069567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 0x559ce7183580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.899+0000 7fa64ffff700 1 --2- 172.21.15.44:0/2415069567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce717d300 0x559ce717d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.901+0000 7fa6557ac700 1 -- 172.21.15.44:0/2415069567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 msgr2=0x559ce7183580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.901+0000 7fa6557ac700 1 --2- 172.21.15.44:0/2415069567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 0x559ce7183580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.901+0000 7fa6557ac700 1 -- 172.21.15.44:0/2415069567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce717d300 msgr2=0x559ce717d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.901+0000 7fa6557ac700 1 --2- 172.21.15.44:0/2415069567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce717d300 0x559ce717d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.901+0000 7fa6557ac700 1 -- 172.21.15.44:0/2415069567 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ce6efbd80 con 0x559ce717dcc0 2023-09-10T01:30:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.910+0000 7fa6557ac700 1 --2- 172.21.15.44:0/2415069567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717dcc0 0x559ce7189f30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa6440a0a10 tx=0x7fa644098ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e09bf322ba98328 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.914+0000 7fa636ffd700 1 -- 172.21.15.44:0/2415069567 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644010480 con 0x559ce717dcc0 2023-09-10T01:30:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.914+0000 7fa636ffd700 1 -- 172.21.15.44:0/2415069567 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa644011c40 con 0x559ce717dcc0 2023-09-10T01:30:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.914+0000 7fa636ffd700 1 -- 172.21.15.44:0/2415069567 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6440a0f40 con 0x559ce717dcc0 2023-09-10T01:30:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.915+0000 7fa6633d5b80 1 -- 172.21.15.44:0/2415069567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717dcc0 msgr2=0x559ce7189f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.915+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/2415069567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717dcc0 0x559ce7189f30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa6440a0a10 tx=0x7fa644098ca0 comp rx=0 tx=0).stop 2023-09-10T01:30:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.928+0000 7fa6633d5b80 1 -- 172.21.15.44:0/2415069567 shutdown_connections 2023-09-10T01:30:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.928+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/2415069567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717dcc0 0x559ce7189f30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.928+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/2415069567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 0x559ce7183580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.928+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/2415069567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce717d300 0x559ce717d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.928+0000 7fa6633d5b80 1 -- 172.21.15.44:0/2415069567 >> 172.21.15.44:0/2415069567 conn(0x559ce70f2030 msgr2=0x559ce7178560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.935+0000 7fa6633d5b80 1 -- 172.21.15.44:0/2415069567 shutdown_connections 2023-09-10T01:30:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.940+0000 7fa6633d5b80 1 -- 172.21.15.44:0/2415069567 wait complete. 2023-09-10T01:30:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.940+0000 7fa6633d5b80 1 -- start start 2023-09-10T01:30:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.940+0000 7fa6633d5b80 1 Processor -- start 2023-09-10T01:30:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.951+0000 7fa6633d5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717d300 0x559ce7189e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.951+0000 7fa6633d5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce717dcc0 0x559ce7187170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.951+0000 7fa6633d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 0x559ce7187810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.951+0000 7fa6633d5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce706db10 con 0x559ce717dcc0 2023-09-10T01:30:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.951+0000 7fa6633d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce706d810 con 0x559ce71831a0 2023-09-10T01:30:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.951+0000 7fa6633d5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce6f518c0 con 0x559ce717d300 2023-09-10T01:30:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.952+0000 7fa6633d5b80 10 client.0 _ll_register_callbacks cb 0x559ce6fb7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:30:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.952+0000 7fa654fab700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717d300 0x559ce7189e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.952+0000 7fa654fab700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717d300 0x559ce7189e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38942/0 (socket says 172.21.15.44:38942) 2023-09-10T01:30:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.952+0000 7fa654fab700 1 -- 172.21.15.44:0/4262316628 learned_addr learned my addr 172.21.15.44:0/4262316628 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.952+0000 7fa6557ac700 1 --2- 172.21.15.44:0/4262316628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 0x559ce7187810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.952+0000 7fa6557ac700 1 -- 172.21.15.44:0/4262316628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717d300 msgr2=0x559ce7189e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.952+0000 7fa6557ac700 1 --2- 172.21.15.44:0/4262316628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717d300 0x559ce7189e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.952+0000 7fa6557ac700 1 -- 172.21.15.44:0/4262316628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce717dcc0 msgr2=0x559ce7187170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.952+0000 7fa6557ac700 1 --2- 172.21.15.44:0/4262316628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce717dcc0 0x559ce7187170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.952+0000 7fa6557ac700 1 -- 172.21.15.44:0/4262316628 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6501b2470 con 0x559ce71831a0 2023-09-10T01:30:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.954+0000 7fa6557ac700 1 --2- 172.21.15.44:0/4262316628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 0x559ce7187810 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa6440c9660 tx=0x7fa6440c9110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cf2e432c8fc7f80 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.957+0000 7fa62ffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:30:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.957+0000 7fa62ffff700 1 -- 172.21.15.44:0/4262316628 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644098ed0 con 0x559ce71831a0 2023-09-10T01:30:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.957+0000 7fa62ffff700 1 -- 172.21.15.44:0/4262316628 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(193..193 src has 1..193) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fa6440995d0 con 0x559ce71831a0 2023-09-10T01:30:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.957+0000 7fa6633d5b80 1 -- 172.21.15.44:0/4262316628 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ce7027490 con 0x559ce71831a0 2023-09-10T01:30:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.971+0000 7fa62ffff700 1 -- 172.21.15.44:0/4262316628 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=173) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6440a0f40 con 0x559ce71831a0 2023-09-10T01:30:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.971+0000 7fa6633d5b80 10 client.30062 fetch_fsmap learned FSMap version 173 2023-09-10T01:30:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.971+0000 7fa6633d5b80 1 -- 172.21.15.44:0/4262316628 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=173,monmap=4+}) v3 -- 0x559ce6ee4c00 con 0x559ce71831a0 2023-09-10T01:30:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.984+0000 7fa62ffff700 1 -- 172.21.15.44:0/4262316628 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559ce7027490 con 0x559ce71831a0 2023-09-10T01:30:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.984+0000 7fa62ffff700 1 -- 172.21.15.44:0/4262316628 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 173) v1 ==== 7359+0+0 (secure 0 0 0) 0x7fa6440983c0 con 0x559ce71831a0 2023-09-10T01:30:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.984+0000 7fa6633d5b80 10 client.30062 fetch_fsmap finished waiting for FSMap version 173 2023-09-10T01:30:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.984+0000 7fa6633d5b80 10 client.30062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:30:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.984+0000 7fa6633d5b80 20 client.30062 populate_metadata read hostname 'smithi044' 2023-09-10T01:30:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.984+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/4262316628 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559ce71a80e0 0x559ce71aa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.984+0000 7fa6633d5b80 4 client.30062 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:30:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.984+0000 7fa6633d5b80 1 -- 172.21.15.44:0/4262316628 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ce70e4840 con 0x559ce71a80e0 2023-09-10T01:30:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.991+0000 7fa654fab700 1 --2- 172.21.15.44:0/4262316628 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559ce71a80e0 0x559ce71aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:12.997+0000 7fa654fab700 1 --2- 172.21.15.44:0/4262316628 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559ce71a80e0 0x559ce71aa500 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39a3b6416f96f1db server_cookie=7bcd88ea767b5a85 in_seq=0 out_seq=0 2023-09-10T01:30:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.006+0000 7fa62ffff700 10 client.30062 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:30:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.006+0000 7fa62ffff700 1 -- 172.21.15.44:0/4262316628 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559ce70e4840 con 0x559ce71a80e0 2023-09-10T01:30:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.006+0000 7fa62ffff700 10 client.30062 handle_command_reply: tid=0 2023-09-10T01:30:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.006+0000 7fa6633d5b80 1 client.30062 shutdown 2023-09-10T01:30:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.016+0000 7fa6633d5b80 1 -- 172.21.15.44:0/4262316628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 msgr2=0x559ce7187810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.016+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/4262316628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 0x559ce7187810 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa6440c9660 tx=0x7fa6440c9110 comp rx=0 tx=0).stop 2023-09-10T01:30:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.038+0000 7fa6633d5b80 1 -- 172.21.15.44:0/4262316628 shutdown_connections 2023-09-10T01:30:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.038+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/4262316628 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559ce71a80e0 0x559ce71aa500 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.038+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/4262316628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 0x559ce7187810 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.038+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/4262316628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717d300 0x559ce7189e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.038+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/4262316628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce717dcc0 0x559ce7187170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.038+0000 7fa6633d5b80 1 -- 172.21.15.44:0/4262316628 >> 172.21.15.44:0/4262316628 conn(0x559ce70f2030 msgr2=0x559ce71777a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.038+0000 7fa62ffff700 0 client.30062 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:30:13.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:12 smithi131 ceph-mon[103713]: pgmap v1610: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 651 op/s 2023-09-10T01:30:13.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:12 smithi044 ceph-mon[98676]: pgmap v1610: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 651 op/s 2023-09-10T01:30:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.055+0000 7fa6633d5b80 1 -- 172.21.15.44:0/4262316628 shutdown_connections 2023-09-10T01:30:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.059+0000 7fa6633d5b80 1 -- 172.21.15.44:0/4262316628 wait complete. 2023-09-10T01:30:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.065+0000 7fa6633d5b80 20 client.30062 trim_cache size 0 max 16384 2023-09-10T01:30:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.096+0000 7fa6633d5b80 1 Processor -- start 2023-09-10T01:30:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.096+0000 7fa6633d5b80 1 -- start start 2023-09-10T01:30:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.096+0000 7fa6633d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce717d300 0x559ce7184df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.096+0000 7fa6633d5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717dcc0 0x559ce7181230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.096+0000 7fa6633d5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce71831a0 0x559ce7181770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.096+0000 7fa6633d5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce70e4840 con 0x559ce71831a0 2023-09-10T01:30:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.096+0000 7fa6633d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce7027490 con 0x559ce717d300 2023-09-10T01:30:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.096+0000 7fa6633d5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce716fe00 con 0x559ce717dcc0 2023-09-10T01:30:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.103+0000 7fa6557ac700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce71831a0 0x559ce7181770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.103+0000 7fa6557ac700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce71831a0 0x559ce7181770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60300/0 (socket says 172.21.15.44:60300) 2023-09-10T01:30:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.103+0000 7fa6557ac700 1 -- 172.21.15.44:0/754275207 learned_addr learned my addr 172.21.15.44:0/754275207 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:13.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:12 smithi099 ceph-mon[102692]: pgmap v1610: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 651 op/s 2023-09-10T01:30:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.106+0000 7fa654fab700 1 --2- 172.21.15.44:0/754275207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce717d300 0x559ce7184df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.107+0000 7fa64ffff700 1 --2- 172.21.15.44:0/754275207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717dcc0 0x559ce7181230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.111+0000 7fa64ffff700 1 -- 172.21.15.44:0/754275207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce717d300 msgr2=0x559ce7184df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.111+0000 7fa64ffff700 1 --2- 172.21.15.44:0/754275207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce717d300 0x559ce7184df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.111+0000 7fa64ffff700 1 -- 172.21.15.44:0/754275207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce71831a0 msgr2=0x559ce7181770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.111+0000 7fa64ffff700 1 --2- 172.21.15.44:0/754275207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce71831a0 0x559ce7181770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.111+0000 7fa64ffff700 1 -- 172.21.15.44:0/754275207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ce6f221f0 con 0x559ce717dcc0 2023-09-10T01:30:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.112+0000 7fa64ffff700 1 --2- 172.21.15.44:0/754275207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717dcc0 0x559ce7181230 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa640000c00 tx=0x7fa640005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9069671cae3dc369 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.112+0000 7fa6377fe700 1 -- 172.21.15.44:0/754275207 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64001c050 con 0x559ce717dcc0 2023-09-10T01:30:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.112+0000 7fa6377fe700 1 -- 172.21.15.44:0/754275207 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa64001d040 con 0x559ce717dcc0 2023-09-10T01:30:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.112+0000 7fa6377fe700 1 -- 172.21.15.44:0/754275207 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64000bc80 con 0x559ce717dcc0 2023-09-10T01:30:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.114+0000 7fa6633d5b80 1 -- 172.21.15.44:0/754275207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717dcc0 msgr2=0x559ce7181230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.114+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/754275207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717dcc0 0x559ce7181230 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa640000c00 tx=0x7fa640005df0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.136+0000 7fa6633d5b80 1 -- 172.21.15.44:0/754275207 shutdown_connections 2023-09-10T01:30:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.136+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/754275207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce71831a0 0x559ce7181770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.136+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/754275207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce717d300 0x559ce7184df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.136+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/754275207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717dcc0 0x559ce7181230 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.136+0000 7fa6633d5b80 1 -- 172.21.15.44:0/754275207 >> 172.21.15.44:0/754275207 conn(0x559ce70f2030 msgr2=0x559ce71821d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.137+0000 7fa6633d5b80 1 -- 172.21.15.44:0/754275207 shutdown_connections 2023-09-10T01:30:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.142+0000 7fa6633d5b80 1 -- 172.21.15.44:0/754275207 wait complete. 2023-09-10T01:30:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.143+0000 7fa6633d5b80 1 -- start start 2023-09-10T01:30:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.143+0000 7fa6633d5b80 1 Processor -- start 2023-09-10T01:30:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.143+0000 7fa6633d5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717d300 0x559ce719d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.143+0000 7fa6633d5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce717dcc0 0x559ce719dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.143+0000 7fa6633d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 0x559ce71f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.143+0000 7fa6633d5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce71f5490 con 0x559ce717dcc0 2023-09-10T01:30:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.143+0000 7fa6633d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce71f55d0 con 0x559ce71831a0 2023-09-10T01:30:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.143+0000 7fa6633d5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce71f5790 con 0x559ce717d300 2023-09-10T01:30:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.143+0000 7fa6633d5b80 10 client.0 _ll_register_callbacks cb 0x559ce6fb7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:30:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.144+0000 7fa6557ac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 0x559ce71f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.144+0000 7fa6557ac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 0x559ce71f4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33814/0 (socket says 172.21.15.44:33814) 2023-09-10T01:30:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.144+0000 7fa6557ac700 1 -- 172.21.15.44:0/18280417 learned_addr learned my addr 172.21.15.44:0/18280417 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.144+0000 7fa654fab700 1 --2- 172.21.15.44:0/18280417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717d300 0x559ce719d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.145+0000 7fa654fab700 1 -- 172.21.15.44:0/18280417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 msgr2=0x559ce71f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.145+0000 7fa654fab700 1 --2- 172.21.15.44:0/18280417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 0x559ce71f4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.145+0000 7fa654fab700 1 -- 172.21.15.44:0/18280417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce717dcc0 msgr2=0x559ce719dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.145+0000 7fa654fab700 1 --2- 172.21.15.44:0/18280417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce717dcc0 0x559ce719dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.145+0000 7fa654fab700 1 -- 172.21.15.44:0/18280417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65011cb00 con 0x559ce717d300 2023-09-10T01:30:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.146+0000 7fa654fab700 1 --2- 172.21.15.44:0/18280417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717d300 0x559ce719d680 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa648009fd0 tx=0x7fa648004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27bd8c1a7b7cbb39 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:13.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.149+0000 7fa62ffff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:30:13.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.149+0000 7fa62ffff700 1 -- 172.21.15.44:0/18280417 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6480264f0 con 0x559ce717d300 2023-09-10T01:30:13.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.149+0000 7fa62ffff700 1 -- 172.21.15.44:0/18280417 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(193..193 src has 1..193) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fa648015870 con 0x559ce717d300 2023-09-10T01:30:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.149+0000 7fa6633d5b80 1 -- 172.21.15.44:0/18280417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa644011c40 con 0x559ce717d300 2023-09-10T01:30:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.154+0000 7fa62ffff700 1 -- 172.21.15.44:0/18280417 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=173) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa648026e50 con 0x559ce717d300 2023-09-10T01:30:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.164+0000 7fa6633d5b80 10 client.30088 fetch_fsmap learned FSMap version 173 2023-09-10T01:30:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.164+0000 7fa6633d5b80 1 -- 172.21.15.44:0/18280417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=173,monmap=4+}) v3 -- 0x559ce6efa250 con 0x559ce717d300 2023-09-10T01:30:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.166+0000 7fa62ffff700 1 -- 172.21.15.44:0/18280417 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa644011c40 con 0x559ce717d300 2023-09-10T01:30:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.166+0000 7fa62ffff700 1 -- 172.21.15.44:0/18280417 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 173) v1 ==== 7359+0+0 (secure 0 0 0) 0x7fa64802fa00 con 0x559ce717d300 2023-09-10T01:30:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.166+0000 7fa6633d5b80 10 client.30088 fetch_fsmap finished waiting for FSMap version 173 2023-09-10T01:30:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.166+0000 7fa6633d5b80 10 client.30088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:30:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.166+0000 7fa6633d5b80 20 client.30088 populate_metadata read hostname 'smithi044' 2023-09-10T01:30:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.166+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/18280417 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559ce718b550 0x559ce718d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.166+0000 7fa6633d5b80 4 client.30088 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:30:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.166+0000 7fa6633d5b80 1 -- 172.21.15.44:0/18280417 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559ce70c88c0 con 0x559ce718b550 2023-09-10T01:30:13.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.168+0000 7fa64ffff700 1 --2- 172.21.15.44:0/18280417 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559ce718b550 0x559ce718d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:13.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.173+0000 7fa64ffff700 1 --2- 172.21.15.44:0/18280417 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559ce718b550 0x559ce718d970 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fa031279d781cd8 server_cookie=f6933a3d1d5fdbb1 in_seq=0 out_seq=0 2023-09-10T01:30:13.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.173+0000 7fa62ffff700 10 client.30088 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:30:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.305+0000 7fa62ffff700 1 -- 172.21.15.44:0/18280417 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559ce70c88c0 con 0x559ce718b550 2023-09-10T01:30:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.305+0000 7fa62ffff700 10 client.30088 handle_command_reply: tid=0 2023-09-10T01:30:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.307+0000 7fa6633d5b80 1 client.30088 shutdown 2023-09-10T01:30:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.314+0000 7fa6633d5b80 1 -- 172.21.15.44:0/18280417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717d300 msgr2=0x559ce719d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.314+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/18280417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717d300 0x559ce719d680 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa648009fd0 tx=0x7fa648004ad0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.335+0000 7fa6633d5b80 1 -- 172.21.15.44:0/18280417 shutdown_connections 2023-09-10T01:30:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.335+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/18280417 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x559ce718b550 0x559ce718d970 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.335+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/18280417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ce71831a0 0x559ce71f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.335+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/18280417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ce717d300 0x559ce719d680 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.335+0000 7fa6633d5b80 1 --2- 172.21.15.44:0/18280417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559ce717dcc0 0x559ce719dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.335+0000 7fa6633d5b80 1 -- 172.21.15.44:0/18280417 >> 172.21.15.44:0/18280417 conn(0x559ce70f2030 msgr2=0x559ce71821d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.336+0000 7fa62ffff700 0 client.30088 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:30:13.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.338+0000 7fa6633d5b80 1 -- 172.21.15.44:0/18280417 shutdown_connections 2023-09-10T01:30:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.339+0000 7fa6633d5b80 1 -- 172.21.15.44:0/18280417 wait complete. 2023-09-10T01:30:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:30:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:30:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:30:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:30:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.349+0000 7fa6633d5b80 20 client.30088 trim_cache size 0 max 16384 2023-09-10T01:30:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.360+0000 7fa65720f700 1 -- 172.21.15.44:0/159115541 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa6380a08b0 msgr2=0x7fa6380a2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.360+0000 7fa65720f700 1 --2- 172.21.15.44:0/159115541 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa6380a08b0 0x7fa6380a2d70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa6480060b0 tx=0x7fa648006040 comp rx=0 tx=0).stop 2023-09-10T01:30:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.360+0000 7fa65720f700 1 -- 172.21.15.44:0/159115541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa650119de0 msgr2=0x7fa6501a9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.360+0000 7fa65720f700 1 --2- 172.21.15.44:0/159115541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa650119de0 0x7fa6501a9fe0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa64400b450 tx=0x7fa64400b760 comp rx=0 tx=0).stop 2023-09-10T01:30:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.360+0000 7fa65720f700 1 -- 172.21.15.44:0/159115541 shutdown_connections 2023-09-10T01:30:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.360+0000 7fa65720f700 1 --2- 172.21.15.44:0/159115541 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fa6380a08b0 0x7fa6380a2d70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.360+0000 7fa65720f700 1 --2- 172.21.15.44:0/159115541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa650119de0 0x7fa6501a9fe0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.360+0000 7fa65720f700 1 --2- 172.21.15.44:0/159115541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa65010ed50 0x7fa6501a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.360+0000 7fa65720f700 1 --2- 172.21.15.44:0/159115541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65010f1c0 0x7fa6501a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.360+0000 7fa65720f700 1 -- 172.21.15.44:0/159115541 >> 172.21.15.44:0/159115541 conn(0x7fa65006c6c0 msgr2=0x7fa650117dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.368+0000 7fa65720f700 1 -- 172.21.15.44:0/159115541 shutdown_connections 2023-09-10T01:30:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:13.368+0000 7fa65720f700 1 -- 172.21.15.44:0/159115541 wait complete. 2023-09-10T01:30:15.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:14 smithi131 ceph-mon[103713]: pgmap v1611: 129 pgs: 19 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 61 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 541 op/s 2023-09-10T01:30:15.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:14 smithi044 ceph-mon[98676]: pgmap v1611: 129 pgs: 19 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 61 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 541 op/s 2023-09-10T01:30:15.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:14 smithi099 ceph-mon[102692]: pgmap v1611: 129 pgs: 19 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 61 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 541 op/s 2023-09-10T01:30:16.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 550.0 s, 60.4 tps, lat 1839.032 ms stddev 905.816 2023-09-10T01:30:17.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:16 smithi131 ceph-mon[103713]: pgmap v1612: 129 pgs: 19 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 61 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 541 op/s 2023-09-10T01:30:17.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:16 smithi044 ceph-mon[98676]: pgmap v1612: 129 pgs: 19 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 61 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 541 op/s 2023-09-10T01:30:17.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:16 smithi099 ceph-mon[102692]: pgmap v1612: 129 pgs: 19 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 61 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 541 op/s 2023-09-10T01:30:18.050 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:17 smithi131 ceph-mon[103713]: pgmap v1613: 129 pgs: 8 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 105 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 382 op/s 2023-09-10T01:30:18.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:17 smithi044 ceph-mon[98676]: pgmap v1613: 129 pgs: 8 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 105 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 382 op/s 2023-09-10T01:30:18.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:17 smithi099 ceph-mon[102692]: pgmap v1613: 129 pgs: 8 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 105 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 382 op/s 2023-09-10T01:30:20.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:19 smithi131 ceph-mon[103713]: pgmap v1614: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 383 op/s 2023-09-10T01:30:20.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:19 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:30:20.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:19 smithi044 ceph-mon[98676]: pgmap v1614: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 383 op/s 2023-09-10T01:30:20.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:19 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:30:20.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:19 smithi099 ceph-mon[102692]: pgmap v1614: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 383 op/s 2023-09-10T01:30:20.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:19 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:30:21.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 555.0 s, 342.8 tps, lat 307.382 ms stddev 181.998 2023-09-10T01:30:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:20 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:30:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:20 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:30:21.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:20 smithi131 ceph-mon[103713]: pgmap v1615: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 3.9 MiB/s wr, 336 op/s 2023-09-10T01:30:21.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:20 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:30:21.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:20 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:30:21.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:20 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:30:21.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:20 smithi044 ceph-mon[98676]: pgmap v1615: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 3.9 MiB/s wr, 336 op/s 2023-09-10T01:30:21.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:20 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:30:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:20 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:30:21.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:20 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:30:21.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:20 smithi099 ceph-mon[102692]: pgmap v1615: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 3.9 MiB/s wr, 336 op/s 2023-09-10T01:30:21.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:20 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:30:22.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:22 smithi131 ceph-mon[103713]: osdmap e194: 12 total, 12 up, 12 in 2023-09-10T01:30:22.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:22 smithi044 ceph-mon[98676]: osdmap e194: 12 total, 12 up, 12 in 2023-09-10T01:30:22.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:22 smithi099 ceph-mon[102692]: osdmap e194: 12 total, 12 up, 12 in 2023-09-10T01:30:23.802 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:23 smithi131 ceph-mon[103713]: pgmap v1617: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 571 op/s 2023-09-10T01:30:23.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:23 smithi044 ceph-mon[98676]: pgmap v1617: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 571 op/s 2023-09-10T01:30:23.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:23 smithi099 ceph-mon[102692]: pgmap v1617: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 571 op/s 2023-09-10T01:30:25.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:25 smithi131 ceph-mon[103713]: pgmap v1618: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 571 op/s 2023-09-10T01:30:25.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:25 smithi044 ceph-mon[98676]: pgmap v1618: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 571 op/s 2023-09-10T01:30:25.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:25 smithi099 ceph-mon[102692]: pgmap v1618: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 571 op/s 2023-09-10T01:30:26.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 560.0 s, 332.2 tps, lat 310.975 ms stddev 106.985 2023-09-10T01:30:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.601+0000 7fd2c7513700 1 -- 172.21.15.44:0/1733008970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2c0107fc0 msgr2=0x7fd2c0108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.601+0000 7fd2c7513700 1 --2- 172.21.15.44:0/1733008970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2c0107fc0 0x7fd2c0108450 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009990 tx=0x7fd2b0009ca0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.606+0000 7fd2c7513700 1 -- 172.21.15.44:0/1733008970 shutdown_connections 2023-09-10T01:30:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.606+0000 7fd2c7513700 1 --2- 172.21.15.44:0/1733008970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2c0107fc0 0x7fd2c0108450 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.606+0000 7fd2c7513700 1 --2- 172.21.15.44:0/1733008970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2c010d4a0 0x7fd2c010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.606+0000 7fd2c7513700 1 --2- 172.21.15.44:0/1733008970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2c0107600 0x7fd2c0107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.606+0000 7fd2c7513700 1 -- 172.21.15.44:0/1733008970 >> 172.21.15.44:0/1733008970 conn(0x7fd2c006c6c0 msgr2=0x7fd2c006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.621+0000 7fd2c7513700 1 -- 172.21.15.44:0/1733008970 shutdown_connections 2023-09-10T01:30:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.628+0000 7fd2c7513700 1 -- 172.21.15.44:0/1733008970 wait complete. 2023-09-10T01:30:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.629+0000 7fd2c7513700 1 Processor -- start 2023-09-10T01:30:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.638+0000 7fd2c7513700 1 -- start start 2023-09-10T01:30:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.638+0000 7fd2c7513700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2c0107600 0x7fd2c011cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.638+0000 7fd2c7513700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2c0107fc0 0x7fd2c011d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.638+0000 7fd2c7513700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2c010d4a0 0x7fd2c0117db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.638+0000 7fd2c7513700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c0118580 con 0x7fd2c0107fc0 2023-09-10T01:30:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.638+0000 7fd2c7513700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c01186f0 con 0x7fd2c010d4a0 2023-09-10T01:30:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.638+0000 7fd2c7513700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c01188b0 con 0x7fd2c0107600 2023-09-10T01:30:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.639+0000 7fd2c4aae700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2c0107fc0 0x7fd2c011d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.639+0000 7fd2c4aae700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2c0107fc0 0x7fd2c011d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60320/0 (socket says 172.21.15.44:60320) 2023-09-10T01:30:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.639+0000 7fd2c4aae700 1 -- 172.21.15.44:0/3934349238 learned_addr learned my addr 172.21.15.44:0/3934349238 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.641+0000 7fd2c4aae700 1 -- 172.21.15.44:0/3934349238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2c0107600 msgr2=0x7fd2c011cf00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T01:30:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.641+0000 7fd2c4aae700 1 --2- 172.21.15.44:0/3934349238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2c0107600 0x7fd2c011cf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.641+0000 7fd2c4aae700 1 -- 172.21.15.44:0/3934349238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2c010d4a0 msgr2=0x7fd2c0117db0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-10T01:30:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.641+0000 7fd2c4aae700 1 --2- 172.21.15.44:0/3934349238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2c010d4a0 0x7fd2c0117db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.641+0000 7fd2c4aae700 1 -- 172.21.15.44:0/3934349238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b00095f0 con 0x7fd2c0107fc0 2023-09-10T01:30:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.641+0000 7fd2c4aae700 1 --2- 172.21.15.44:0/3934349238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2c0107fc0 0x7fd2c011d460 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00b530 tx=0x7fd2bc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.641+0000 7fd2b67fc700 1 -- 172.21.15.44:0/3934349238 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc011830 con 0x7fd2c0107fc0 2023-09-10T01:30:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.642+0000 7fd2b67fc700 1 -- 172.21.15.44:0/3934349238 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2bc011e20 con 0x7fd2c0107fc0 2023-09-10T01:30:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.642+0000 7fd2c7513700 1 -- 172.21.15.44:0/3934349238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c0118b90 con 0x7fd2c0107fc0 2023-09-10T01:30:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.642+0000 7fd2b67fc700 1 -- 172.21.15.44:0/3934349238 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc010e30 con 0x7fd2c0107fc0 2023-09-10T01:30:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.642+0000 7fd2c7513700 1 -- 172.21.15.44:0/3934349238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c0122ff0 con 0x7fd2c0107fc0 2023-09-10T01:30:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.645+0000 7fd2b67fc700 1 -- 172.21.15.44:0/3934349238 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fd2bc01a400 con 0x7fd2c0107fc0 2023-09-10T01:30:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.646+0000 7fd2b67fc700 1 --2- 172.21.15.44:0/3934349238 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd2ac0c2880 0x7fd2ac0c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.646+0000 7fd2b67fc700 1 -- 172.21.15.44:0/3934349238 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fd2bc0ccc40 con 0x7fd2c0107fc0 2023-09-10T01:30:26.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.646+0000 7fd2c52af700 1 --2- 172.21.15.44:0/3934349238 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd2ac0c2880 0x7fd2ac0c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.648+0000 7fd2d36d9b80 1 Processor -- start 2023-09-10T01:30:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.649+0000 7fd2d36d9b80 1 -- start start 2023-09-10T01:30:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.649+0000 7fd2c52af700 1 --2- 172.21.15.44:0/3934349238 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd2ac0c2880 0x7fd2ac0c4d40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800ab90 tx=0x7fd2b8009230 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.649+0000 7fd2d36d9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb591a0 0x55624cb59580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.649+0000 7fd2d36d9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb53300 0x55624cb53780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.649+0000 7fd2d36d9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb53cc0 0x55624cb5ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.649+0000 7fd2d36d9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624caac6d0 con 0x55624cb53300 2023-09-10T01:30:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.649+0000 7fd2d36d9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624caa9ff0 con 0x55624cb53cc0 2023-09-10T01:30:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.649+0000 7fd2d36d9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624caa58e0 con 0x55624cb591a0 2023-09-10T01:30:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.652+0000 7fd2c52af700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb591a0 0x55624cb59580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.652+0000 7fd2c52af700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb591a0 0x55624cb59580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38976/0 (socket says 172.21.15.44:38976) 2023-09-10T01:30:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.652+0000 7fd2c52af700 1 -- 172.21.15.44:0/3961217747 learned_addr learned my addr 172.21.15.44:0/3961217747 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.652+0000 7fd2c52af700 1 -- 172.21.15.44:0/3961217747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb53cc0 msgr2=0x55624cb5ff30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:30:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.652+0000 7fd2c52af700 1 --2- 172.21.15.44:0/3961217747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb53cc0 0x55624cb5ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.652+0000 7fd2c52af700 1 -- 172.21.15.44:0/3961217747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb53300 msgr2=0x55624cb53780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.652+0000 7fd2c52af700 1 --2- 172.21.15.44:0/3961217747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb53300 0x55624cb53780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.652+0000 7fd2c52af700 1 -- 172.21.15.44:0/3961217747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55624c8d1d80 con 0x55624cb591a0 2023-09-10T01:30:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.662+0000 7fd2c52af700 1 --2- 172.21.15.44:0/3961217747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb591a0 0x55624cb59580 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800cde0 tx=0x7fd2b800f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=584020ce53548850 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:26.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.670+0000 7fd2ab7fe700 1 -- 172.21.15.44:0/3961217747 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8019420 con 0x55624cb591a0 2023-09-10T01:30:26.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.670+0000 7fd2ab7fe700 1 -- 172.21.15.44:0/3961217747 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2b8019a10 con 0x55624cb591a0 2023-09-10T01:30:26.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.670+0000 7fd2ab7fe700 1 -- 172.21.15.44:0/3961217747 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b80218e0 con 0x55624cb591a0 2023-09-10T01:30:26.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.671+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/3961217747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb591a0 msgr2=0x55624cb59580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:26.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.671+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/3961217747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb591a0 0x55624cb59580 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800cde0 tx=0x7fd2b800f000 comp rx=0 tx=0).stop 2023-09-10T01:30:26.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.671+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/3961217747 shutdown_connections 2023-09-10T01:30:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.671+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/3961217747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb53cc0 0x55624cb5ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.671+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/3961217747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb591a0 0x55624cb59580 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.671+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/3961217747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb53300 0x55624cb53780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.671+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/3961217747 >> 172.21.15.44:0/3961217747 conn(0x55624cac8030 msgr2=0x55624cb4e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.682+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/3961217747 shutdown_connections 2023-09-10T01:30:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.683+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/3961217747 wait complete. 2023-09-10T01:30:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.683+0000 7fd2d36d9b80 1 -- start start 2023-09-10T01:30:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.683+0000 7fd2d36d9b80 1 Processor -- start 2023-09-10T01:30:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.685+0000 7fd2d36d9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb53300 0x55624cb5fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.685+0000 7fd2d36d9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb53cc0 0x55624cb5d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.685+0000 7fd2d36d9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb591a0 0x55624cb5d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.685+0000 7fd2d36d9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624ca43b10 con 0x55624cb53300 2023-09-10T01:30:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.685+0000 7fd2d36d9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624ca43810 con 0x55624cb591a0 2023-09-10T01:30:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.685+0000 7fd2d36d9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624c9278c0 con 0x55624cb53cc0 2023-09-10T01:30:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.685+0000 7fd2d36d9b80 10 client.0 _ll_register_callbacks cb 0x55624c98da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:30:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.686+0000 7fd2c5ab0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb591a0 0x55624cb5d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.686+0000 7fd2c52af700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb53300 0x55624cb5fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.686+0000 7fd2c5ab0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb591a0 0x55624cb5d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33838/0 (socket says 172.21.15.44:33838) 2023-09-10T01:30:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.686+0000 7fd2c5ab0700 1 -- 172.21.15.44:0/3591749687 learned_addr learned my addr 172.21.15.44:0/3591749687 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.686+0000 7fd2c52af700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb53300 0x55624cb5fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60332/0 (socket says 172.21.15.44:60332) 2023-09-10T01:30:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.686+0000 7fd2c5ab0700 1 -- 172.21.15.44:0/3591749687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb53cc0 msgr2=0x55624cb5d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.686+0000 7fd2c5ab0700 1 --2- 172.21.15.44:0/3591749687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb53cc0 0x55624cb5d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.686+0000 7fd2c5ab0700 1 -- 172.21.15.44:0/3591749687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb53300 msgr2=0x55624cb5fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.686+0000 7fd2c5ab0700 1 --2- 172.21.15.44:0/3591749687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb53300 0x55624cb5fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.686+0000 7fd2c5ab0700 1 -- 172.21.15.44:0/3591749687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b8012070 con 0x55624cb591a0 2023-09-10T01:30:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.686+0000 7fd2c5ab0700 1 --2- 172.21.15.44:0/3591749687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb591a0 0x55624cb5d7a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0006010 tx=0x7fd2b000b9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2527376cceb33445 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.688+0000 7fd2a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.688+0000 7fd2a8ff9700 1 -- 172.21.15.44:0/3591749687 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b001d070 con 0x55624cb591a0 2023-09-10T01:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.688+0000 7fd2a8ff9700 1 -- 172.21.15.44:0/3591749687 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(195..195 src has 1..195) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fd2b000bdc0 con 0x55624cb591a0 2023-09-10T01:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.688+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/3591749687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55624c9fd490 con 0x55624cb591a0 2023-09-10T01:30:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.690+0000 7fd2a8ff9700 1 -- 172.21.15.44:0/3591749687 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=173) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2b0003850 con 0x55624cb591a0 2023-09-10T01:30:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.696+0000 7fd2d36d9b80 10 client.30071 fetch_fsmap learned FSMap version 173 2023-09-10T01:30:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.696+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/3591749687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=173,monmap=4+}) v3 -- 0x55624c8bac00 con 0x55624cb591a0 2023-09-10T01:30:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.716+0000 7fd2a8ff9700 1 -- 172.21.15.44:0/3591749687 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55624c9fd490 con 0x55624cb591a0 2023-09-10T01:30:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.716+0000 7fd2a8ff9700 1 -- 172.21.15.44:0/3591749687 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 173) v1 ==== 7359+0+0 (secure 0 0 0) 0x7fd2b0029070 con 0x55624cb591a0 2023-09-10T01:30:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.716+0000 7fd2d36d9b80 10 client.30071 fetch_fsmap finished waiting for FSMap version 173 2023-09-10T01:30:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.716+0000 7fd2d36d9b80 10 client.30071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:30:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.716+0000 7fd2d36d9b80 20 client.30071 populate_metadata read hostname 'smithi044' 2023-09-10T01:30:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.716+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/3591749687 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55624cb6d5a0 0x55624cb6f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.716+0000 7fd2d36d9b80 4 client.30071 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:30:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.716+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/3591749687 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55624caba840 con 0x55624cb6d5a0 2023-09-10T01:30:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.718+0000 7fd2c52af700 1 --2- 172.21.15.44:0/3591749687 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55624cb6d5a0 0x55624cb6f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.719+0000 7fd2c52af700 1 --2- 172.21.15.44:0/3591749687 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55624cb6d5a0 0x55624cb6f9c0 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3440f0e8d60d9740 server_cookie=49819c141256abc7 in_seq=0 out_seq=0 2023-09-10T01:30:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.719+0000 7fd2a8ff9700 10 client.30071 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:30:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.720+0000 7fd2a8ff9700 1 -- 172.21.15.44:0/3591749687 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55624caba840 con 0x55624cb6d5a0 2023-09-10T01:30:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.720+0000 7fd2a8ff9700 10 client.30071 handle_command_reply: tid=0 2023-09-10T01:30:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.720+0000 7fd2d36d9b80 1 client.30071 shutdown 2023-09-10T01:30:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.726+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/3591749687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb591a0 msgr2=0x55624cb5d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.726+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/3591749687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb591a0 0x55624cb5d7a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0006010 tx=0x7fd2b000b9e0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.728+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/3591749687 shutdown_connections 2023-09-10T01:30:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.728+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/3591749687 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55624cb6d5a0 0x55624cb6f9c0 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.728+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/3591749687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb591a0 0x55624cb5d7a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.728+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/3591749687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb53300 0x55624cb5fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.728+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/3591749687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb53cc0 0x55624cb5d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.728+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/3591749687 >> 172.21.15.44:0/3591749687 conn(0x55624cac8030 msgr2=0x55624cb4d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.731+0000 7fd2a8ff9700 0 client.30071 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:30:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.742+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/3591749687 shutdown_connections 2023-09-10T01:30:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.752+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/3591749687 wait complete. 2023-09-10T01:30:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.752+0000 7fd2d36d9b80 20 client.30071 trim_cache size 0 max 16384 2023-09-10T01:30:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.780+0000 7fd2d36d9b80 1 Processor -- start 2023-09-10T01:30:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.780+0000 7fd2d36d9b80 1 -- start start 2023-09-10T01:30:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.781+0000 7fd2d36d9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb53300 0x55624cb5ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.781+0000 7fd2d36d9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb53cc0 0x55624cb5fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.781+0000 7fd2d36d9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb591a0 0x55624cb571b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.781+0000 7fd2d36d9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624cb6ab90 con 0x55624cb53cc0 2023-09-10T01:30:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.781+0000 7fd2d36d9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624cb4afc0 con 0x55624cb591a0 2023-09-10T01:30:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.781+0000 7fd2d36d9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624cb5b700 con 0x55624cb53300 2023-09-10T01:30:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.781+0000 7fd2c52af700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb53300 0x55624cb5ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.781+0000 7fd2c52af700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb53300 0x55624cb5ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:38988/0 (socket says 172.21.15.44:38988) 2023-09-10T01:30:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.781+0000 7fd2c52af700 1 -- 172.21.15.44:0/1003139993 learned_addr learned my addr 172.21.15.44:0/1003139993 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.781+0000 7fd2c5ab0700 1 --2- 172.21.15.44:0/1003139993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb591a0 0x55624cb571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.781+0000 7fd2c52af700 1 -- 172.21.15.44:0/1003139993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb591a0 msgr2=0x55624cb571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.781+0000 7fd2c52af700 1 --2- 172.21.15.44:0/1003139993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb591a0 0x55624cb571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.781+0000 7fd2c52af700 1 -- 172.21.15.44:0/1003139993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb53cc0 msgr2=0x55624cb5fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.782+0000 7fd2c52af700 1 --2- 172.21.15.44:0/1003139993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb53cc0 0x55624cb5fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.782+0000 7fd2c52af700 1 -- 172.21.15.44:0/1003139993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55624c8f81f0 con 0x55624cb53300 2023-09-10T01:30:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.787+0000 7fd2c52af700 1 --2- 172.21.15.44:0/1003139993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb53300 0x55624cb5ad80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800a970 tx=0x7fd2b80214a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68d07bdb7ba6d3b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:26.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.792+0000 7fd2abfff700 1 -- 172.21.15.44:0/1003139993 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8027040 con 0x55624cb53300 2023-09-10T01:30:26.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.792+0000 7fd2abfff700 1 -- 172.21.15.44:0/1003139993 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2b8019420 con 0x55624cb53300 2023-09-10T01:30:26.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.792+0000 7fd2abfff700 1 -- 172.21.15.44:0/1003139993 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b802cbd0 con 0x55624cb53300 2023-09-10T01:30:26.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.793+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/1003139993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb53300 msgr2=0x55624cb5ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:26.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.793+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/1003139993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb53300 0x55624cb5ad80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800a970 tx=0x7fd2b80214a0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.793+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/1003139993 shutdown_connections 2023-09-10T01:30:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.793+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/1003139993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb591a0 0x55624cb571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.793+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/1003139993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb53300 0x55624cb5ad80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.793+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/1003139993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb53cc0 0x55624cb5fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.793+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/1003139993 >> 172.21.15.44:0/1003139993 conn(0x55624cac8030 msgr2=0x55624cb58130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.793+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/1003139993 shutdown_connections 2023-09-10T01:30:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.793+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/1003139993 wait complete. 2023-09-10T01:30:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.793+0000 7fd2d36d9b80 1 -- start start 2023-09-10T01:30:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.794+0000 7fd2d36d9b80 1 Processor -- start 2023-09-10T01:30:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.794+0000 7fd2d36d9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb53300 0x55624cb62700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.794+0000 7fd2d36d9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb53cc0 0x55624cb62c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.794+0000 7fd2d36d9b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb591a0 0x55624cbc03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.794+0000 7fd2d36d9b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624cbc0910 con 0x55624cb591a0 2023-09-10T01:30:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.794+0000 7fd2d36d9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624cbc0a50 con 0x55624cb53cc0 2023-09-10T01:30:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.794+0000 7fd2d36d9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624cbc0b90 con 0x55624cb53300 2023-09-10T01:30:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.794+0000 7fd2d36d9b80 10 client.0 _ll_register_callbacks cb 0x7fd2b0009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:30:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.795+0000 7fd2c52af700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb53300 0x55624cb62700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.795+0000 7fd2c4aae700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb53cc0 0x55624cb62c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.795+0000 7fd2c4aae700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb53cc0 0x55624cb62c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33852/0 (socket says 172.21.15.44:33852) 2023-09-10T01:30:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.795+0000 7fd2c4aae700 1 -- 172.21.15.44:0/2327564002 learned_addr learned my addr 172.21.15.44:0/2327564002 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.795+0000 7fd2c5ab0700 1 --2- 172.21.15.44:0/2327564002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb591a0 0x55624cbc03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.795+0000 7fd2c4aae700 1 -- 172.21.15.44:0/2327564002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb53300 msgr2=0x55624cb62700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.795+0000 7fd2c4aae700 1 --2- 172.21.15.44:0/2327564002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb53300 0x55624cb62700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.795+0000 7fd2c4aae700 1 -- 172.21.15.44:0/2327564002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb591a0 msgr2=0x55624cbc03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.795+0000 7fd2c4aae700 1 --2- 172.21.15.44:0/2327564002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb591a0 0x55624cbc03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.795+0000 7fd2c4aae700 1 -- 172.21.15.44:0/2327564002 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55624c8d1d80 con 0x55624cb53cc0 2023-09-10T01:30:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.795+0000 7fd2c4aae700 1 --2- 172.21.15.44:0/2327564002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb53cc0 0x55624cb62c40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0ca160 tx=0x7fd2bc099860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7999d82f601a9135 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.796+0000 7fd2a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:30:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.796+0000 7fd2a8ff9700 1 -- 172.21.15.44:0/2327564002 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc09a270 con 0x55624cb53cc0 2023-09-10T01:30:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.796+0000 7fd2a8ff9700 1 -- 172.21.15.44:0/2327564002 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(195..195 src has 1..195) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fd2bc018520 con 0x55624cb53cc0 2023-09-10T01:30:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.797+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/2327564002 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2b8019a10 con 0x55624cb53cc0 2023-09-10T01:30:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.797+0000 7fd2a8ff9700 1 -- 172.21.15.44:0/2327564002 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=173) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2bc09dc40 con 0x55624cb53cc0 2023-09-10T01:30:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.797+0000 7fd2d36d9b80 10 client.30077 fetch_fsmap learned FSMap version 173 2023-09-10T01:30:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.797+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/2327564002 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=173,monmap=4+}) v3 -- 0x55624c8d0250 con 0x55624cb53cc0 2023-09-10T01:30:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.798+0000 7fd2a8ff9700 1 -- 172.21.15.44:0/2327564002 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2b8019a10 con 0x55624cb53cc0 2023-09-10T01:30:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.798+0000 7fd2a8ff9700 1 -- 172.21.15.44:0/2327564002 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 173) v1 ==== 7359+0+0 (secure 0 0 0) 0x7fd2bc0a19f0 con 0x55624cb53cc0 2023-09-10T01:30:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.798+0000 7fd2d36d9b80 10 client.30077 fetch_fsmap finished waiting for FSMap version 173 2023-09-10T01:30:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.798+0000 7fd2d36d9b80 10 client.30077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:30:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.798+0000 7fd2d36d9b80 20 client.30077 populate_metadata read hostname 'smithi044' 2023-09-10T01:30:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.798+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/2327564002 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55624cbc2290 0x55624cbc46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.798+0000 7fd2d36d9b80 4 client.30077 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:30:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.798+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/2327564002 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55624ca9e8c0 con 0x55624cbc2290 2023-09-10T01:30:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.799+0000 7fd2c52af700 1 --2- 172.21.15.44:0/2327564002 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55624cbc2290 0x55624cbc46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.805+0000 7fd2c52af700 1 --2- 172.21.15.44:0/2327564002 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55624cbc2290 0x55624cbc46b0 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f283096e27b7ceb0 server_cookie=99c46512f28059c3 in_seq=0 out_seq=0 2023-09-10T01:30:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.807+0000 7fd2a8ff9700 10 client.30077 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:30:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.895+0000 7fd2a8ff9700 1 -- 172.21.15.44:0/2327564002 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55624ca9e8c0 con 0x55624cbc2290 2023-09-10T01:30:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.895+0000 7fd2a8ff9700 10 client.30077 handle_command_reply: tid=0 2023-09-10T01:30:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.895+0000 7fd2d36d9b80 1 client.30077 shutdown 2023-09-10T01:30:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.896+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/2327564002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb53cc0 msgr2=0x55624cb62c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.896+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/2327564002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb53cc0 0x55624cb62c40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0ca160 tx=0x7fd2bc099860 comp rx=0 tx=0).stop 2023-09-10T01:30:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.897+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/2327564002 shutdown_connections 2023-09-10T01:30:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.897+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/2327564002 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55624cbc2290 0x55624cbc46b0 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.897+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/2327564002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55624cb591a0 0x55624cbc03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.897+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/2327564002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55624cb53300 0x55624cb62700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.897+0000 7fd2d36d9b80 1 --2- 172.21.15.44:0/2327564002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55624cb53cc0 0x55624cb62c40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.897+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/2327564002 >> 172.21.15.44:0/2327564002 conn(0x55624cac8030 msgr2=0x55624cb571b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.898+0000 7fd2a8ff9700 0 client.30077 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:30:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.902+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/2327564002 shutdown_connections 2023-09-10T01:30:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.912+0000 7fd2d36d9b80 1 -- 172.21.15.44:0/2327564002 wait complete. 2023-09-10T01:30:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.926+0000 7fd2d36d9b80 20 client.30077 trim_cache size 0 max 16384 2023-09-10T01:30:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:30:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:30:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:30:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:30:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.937+0000 7fd2c7513700 1 -- 172.21.15.44:0/3934349238 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd2ac0c2880 msgr2=0x7fd2ac0c4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.937+0000 7fd2c7513700 1 --2- 172.21.15.44:0/3934349238 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd2ac0c2880 0x7fd2ac0c4d40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800ab90 tx=0x7fd2b8009230 comp rx=0 tx=0).stop 2023-09-10T01:30:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.937+0000 7fd2c7513700 1 -- 172.21.15.44:0/3934349238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2c0107fc0 msgr2=0x7fd2c011d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.937+0000 7fd2c7513700 1 --2- 172.21.15.44:0/3934349238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2c0107fc0 0x7fd2c011d460 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00b530 tx=0x7fd2bc00b8f0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.937+0000 7fd2c7513700 1 -- 172.21.15.44:0/3934349238 shutdown_connections 2023-09-10T01:30:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.937+0000 7fd2c7513700 1 --2- 172.21.15.44:0/3934349238 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd2ac0c2880 0x7fd2ac0c4d40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.937+0000 7fd2c7513700 1 --2- 172.21.15.44:0/3934349238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2c010d4a0 0x7fd2c0117db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.937+0000 7fd2c7513700 1 --2- 172.21.15.44:0/3934349238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2c0107600 0x7fd2c011cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.937+0000 7fd2c7513700 1 --2- 172.21.15.44:0/3934349238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2c0107fc0 0x7fd2c011d460 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.937+0000 7fd2c7513700 1 -- 172.21.15.44:0/3934349238 >> 172.21.15.44:0/3934349238 conn(0x7fd2c006c6c0 msgr2=0x7fd2c010b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.938+0000 7fd2c7513700 1 -- 172.21.15.44:0/3934349238 shutdown_connections 2023-09-10T01:30:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:26.938+0000 7fd2c7513700 1 -- 172.21.15.44:0/3934349238 wait complete. 2023-09-10T01:30:27.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:27 smithi131 ceph-mon[103713]: pgmap v1619: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.9 MiB/s wr, 780 op/s 2023-09-10T01:30:27.798 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:27 smithi131 ceph-mon[103713]: osdmap e195: 12 total, 12 up, 12 in 2023-09-10T01:30:27.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:27 smithi044 ceph-mon[98676]: pgmap v1619: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.9 MiB/s wr, 780 op/s 2023-09-10T01:30:27.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:27 smithi044 ceph-mon[98676]: osdmap e195: 12 total, 12 up, 12 in 2023-09-10T01:30:27.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:27 smithi099 ceph-mon[102692]: pgmap v1619: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.9 MiB/s wr, 780 op/s 2023-09-10T01:30:27.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:27 smithi099 ceph-mon[102692]: osdmap e195: 12 total, 12 up, 12 in 2023-09-10T01:30:29.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:29 smithi131 ceph-mon[103713]: pgmap v1621: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.4 MiB/s wr, 982 op/s 2023-09-10T01:30:29.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:29 smithi044 ceph-mon[98676]: pgmap v1621: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.4 MiB/s wr, 982 op/s 2023-09-10T01:30:29.864 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:29 smithi099 ceph-mon[102692]: pgmap v1621: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.4 MiB/s wr, 982 op/s 2023-09-10T01:30:31.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 565.0 s, 286.2 tps, lat 335.326 ms stddev 158.642 2023-09-10T01:30:31.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:31 smithi131 ceph-mon[103713]: pgmap v1622: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 577 op/s 2023-09-10T01:30:31.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:31 smithi044 ceph-mon[98676]: pgmap v1622: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 577 op/s 2023-09-10T01:30:31.862 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:31 smithi099 ceph-mon[102692]: pgmap v1622: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 577 op/s 2023-09-10T01:30:33.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:33 smithi131 ceph-mon[103713]: pgmap v1623: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.5 MiB/s wr, 833 op/s 2023-09-10T01:30:33.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:33 smithi044 ceph-mon[98676]: pgmap v1623: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.5 MiB/s wr, 833 op/s 2023-09-10T01:30:33.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:33 smithi099 ceph-mon[102692]: pgmap v1623: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.5 MiB/s wr, 833 op/s 2023-09-10T01:30:35.858 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:35 smithi099 ceph-mon[102692]: pgmap v1624: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.5 MiB/s wr, 834 op/s 2023-09-10T01:30:36.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:35 smithi131 ceph-mon[103713]: pgmap v1624: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.5 MiB/s wr, 834 op/s 2023-09-10T01:30:36.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:35 smithi044 ceph-mon[98676]: pgmap v1624: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.5 MiB/s wr, 834 op/s 2023-09-10T01:30:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 570.0 s, 272.4 tps, lat 371.217 ms stddev 118.029 2023-09-10T01:30:37.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:37 smithi099 ceph-mon[102692]: pgmap v1625: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.7 MiB/s wr, 819 op/s 2023-09-10T01:30:38.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:37 smithi131 ceph-mon[103713]: pgmap v1625: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.7 MiB/s wr, 819 op/s 2023-09-10T01:30:38.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:37 smithi044 ceph-mon[98676]: pgmap v1625: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.7 MiB/s wr, 819 op/s 2023-09-10T01:30:39.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:39 smithi099 ceph-mon[102692]: pgmap v1626: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 752 op/s 2023-09-10T01:30:40.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:39 smithi131 ceph-mon[103713]: pgmap v1626: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 752 op/s 2023-09-10T01:30:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:39 smithi044 ceph-mon[98676]: pgmap v1626: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 752 op/s 2023-09-10T01:30:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.192+0000 7f6def36d700 1 -- 172.21.15.44:0/3355435899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6de8119de0 msgr2=0x7f6de811c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.192+0000 7f6def36d700 1 --2- 172.21.15.44:0/3355435899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6de8119de0 0x7f6de811c1d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6de401a9a0 tx=0x7f6de401acb0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.192+0000 7f6def36d700 1 -- 172.21.15.44:0/3355435899 shutdown_connections 2023-09-10T01:30:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.192+0000 7f6def36d700 1 --2- 172.21.15.44:0/3355435899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6de8119de0 0x7f6de811c1d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.192+0000 7f6def36d700 1 --2- 172.21.15.44:0/3355435899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de810ed50 0x7f6de8107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.192+0000 7f6def36d700 1 --2- 172.21.15.44:0/3355435899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6de810f1c0 0x7f6de8107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.192+0000 7f6def36d700 1 -- 172.21.15.44:0/3355435899 >> 172.21.15.44:0/3355435899 conn(0x7f6de806c6c0 msgr2=0x7f6de806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.200+0000 7f6def36d700 1 -- 172.21.15.44:0/3355435899 shutdown_connections 2023-09-10T01:30:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.209+0000 7f6def36d700 1 -- 172.21.15.44:0/3355435899 wait complete. 2023-09-10T01:30:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.210+0000 7f6def36d700 1 Processor -- start 2023-09-10T01:30:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.210+0000 7f6def36d700 1 -- start start 2023-09-10T01:30:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.210+0000 7f6def36d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6de810ed50 0x7f6de81152c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.210+0000 7f6def36d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6de8119de0 0x7f6de8115800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.210+0000 7f6def36d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de81c3a70 0x7f6de81c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.210+0000 7f6def36d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de81c4560 con 0x7f6de81c3a70 2023-09-10T01:30:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.210+0000 7f6def36d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de81c46d0 con 0x7f6de8119de0 2023-09-10T01:30:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.210+0000 7f6def36d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de81c4890 con 0x7f6de810ed50 2023-09-10T01:30:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.211+0000 7f6ded90a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de81c3a70 0x7f6de81c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.211+0000 7f6ded90a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de81c3a70 0x7f6de81c3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60362/0 (socket says 172.21.15.44:60362) 2023-09-10T01:30:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.211+0000 7f6ded90a700 1 -- 172.21.15.44:0/848460039 learned_addr learned my addr 172.21.15.44:0/848460039 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.212+0000 7f6ded90a700 1 -- 172.21.15.44:0/848460039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6de810ed50 msgr2=0x7f6de81152c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T01:30:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.212+0000 7f6ded90a700 1 --2- 172.21.15.44:0/848460039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6de810ed50 0x7f6de81152c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.212+0000 7f6ded90a700 1 -- 172.21.15.44:0/848460039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6de8119de0 msgr2=0x7f6de8115800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-10T01:30:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.212+0000 7f6ded90a700 1 --2- 172.21.15.44:0/848460039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6de8119de0 0x7f6de8115800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.212+0000 7f6ded90a700 1 -- 172.21.15.44:0/848460039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de401a600 con 0x7f6de81c3a70 2023-09-10T01:30:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.215+0000 7f6ded90a700 1 --2- 172.21.15.44:0/848460039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de81c3a70 0x7f6de81c3ef0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f6de401cbc0 tx=0x7f6de401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.215+0000 7f6dde7fc700 1 -- 172.21.15.44:0/848460039 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4003d20 con 0x7f6de81c3a70 2023-09-10T01:30:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.216+0000 7f6dde7fc700 1 -- 172.21.15.44:0/848460039 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6de4003e80 con 0x7f6de81c3a70 2023-09-10T01:30:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.216+0000 7f6def36d700 1 -- 172.21.15.44:0/848460039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de8078920 con 0x7f6de81c3a70 2023-09-10T01:30:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.216+0000 7f6def36d700 1 -- 172.21.15.44:0/848460039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de8078da0 con 0x7f6de81c3a70 2023-09-10T01:30:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.216+0000 7f6dde7fc700 1 -- 172.21.15.44:0/848460039 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de401f6e0 con 0x7f6de81c3a70 2023-09-10T01:30:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.221+0000 7f6dfb533b80 1 Processor -- start 2023-09-10T01:30:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.227+0000 7f6dde7fc700 1 -- 172.21.15.44:0/848460039 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f6de400f480 con 0x7f6de81c3a70 2023-09-10T01:30:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.228+0000 7f6dde7fc700 1 --2- 172.21.15.44:0/848460039 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6dd40a0980 0x7f6dd40a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.228+0000 7f6dde7fc700 1 -- 172.21.15.44:0/848460039 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f6de40cdb40 con 0x7f6de81c3a70 2023-09-10T01:30:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.242+0000 7f6dfb533b80 1 -- start start 2023-09-10T01:30:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.245+0000 7f6dec908700 1 --2- 172.21.15.44:0/848460039 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6dd40a0980 0x7f6dd40a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.245+0000 7f6dfb533b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316f41a0 0x563a316f4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.245+0000 7f6dfb533b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316ee300 0x563a316ee780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.245+0000 7f6dfb533b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316eecc0 0x563a316faf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.245+0000 7f6dfb533b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a316476d0 con 0x563a316f41a0 2023-09-10T01:30:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.245+0000 7f6dfb533b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a31644ff0 con 0x563a316eecc0 2023-09-10T01:30:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.245+0000 7f6dfb533b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a316408e0 con 0x563a316ee300 2023-09-10T01:30:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.245+0000 7f6dec908700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316ee300 0x563a316ee780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.245+0000 7f6dec908700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316ee300 0x563a316ee780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39018/0 (socket says 172.21.15.44:39018) 2023-09-10T01:30:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.245+0000 7f6dec908700 1 -- 172.21.15.44:0/624431918 learned_addr learned my addr 172.21.15.44:0/624431918 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.246+0000 7f6dec908700 1 -- 172.21.15.44:0/624431918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316eecc0 msgr2=0x563a316faf30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:30:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.246+0000 7f6dec908700 1 --2- 172.21.15.44:0/624431918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316eecc0 0x563a316faf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.246+0000 7f6dec908700 1 -- 172.21.15.44:0/624431918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316f41a0 msgr2=0x563a316f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.246+0000 7f6dec908700 1 --2- 172.21.15.44:0/624431918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316f41a0 0x563a316f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.246+0000 7f6dec908700 1 -- 172.21.15.44:0/624431918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a3146cd80 con 0x563a316ee300 2023-09-10T01:30:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.246+0000 7f6dec908700 1 --2- 172.21.15.44:0/624431918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316ee300 0x563a316ee780 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6de000f4f0 tx=0x7f6de000f800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33ab05e5f3751ec8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.246+0000 7f6dd37fe700 1 -- 172.21.15.44:0/624431918 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de000c830 con 0x563a316ee300 2023-09-10T01:30:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.246+0000 7f6dd37fe700 1 -- 172.21.15.44:0/624431918 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6de000ce20 con 0x563a316ee300 2023-09-10T01:30:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.246+0000 7f6dd37fe700 1 -- 172.21.15.44:0/624431918 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de0017e30 con 0x563a316ee300 2023-09-10T01:30:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.247+0000 7f6dfb533b80 1 -- 172.21.15.44:0/624431918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316ee300 msgr2=0x563a316ee780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.247+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/624431918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316ee300 0x563a316ee780 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6de000f4f0 tx=0x7f6de000f800 comp rx=0 tx=0).stop 2023-09-10T01:30:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.247+0000 7f6dec908700 1 --2- 172.21.15.44:0/848460039 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6dd40a0980 0x7f6dd40a2e40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6de0017fd0 tx=0x7f6de000b420 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.257+0000 7f6dfb533b80 1 -- 172.21.15.44:0/624431918 shutdown_connections 2023-09-10T01:30:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.257+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/624431918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316eecc0 0x563a316faf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.257+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/624431918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316f41a0 0x563a316f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.257+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/624431918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316ee300 0x563a316ee780 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.257+0000 7f6dfb533b80 1 -- 172.21.15.44:0/624431918 >> 172.21.15.44:0/624431918 conn(0x563a31663030 msgr2=0x563a316e9560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.268+0000 7f6dfb533b80 1 -- 172.21.15.44:0/624431918 shutdown_connections 2023-09-10T01:30:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.272+0000 7f6dfb533b80 1 -- 172.21.15.44:0/624431918 wait complete. 2023-09-10T01:30:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.272+0000 7f6dfb533b80 1 -- start start 2023-09-10T01:30:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.272+0000 7f6dfb533b80 1 Processor -- start 2023-09-10T01:30:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.273+0000 7f6dfb533b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316ee300 0x563a316fadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.273+0000 7f6dfb533b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316eecc0 0x563a316f8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.273+0000 7f6dfb533b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316f41a0 0x563a316f86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.273+0000 7f6dfb533b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a315deb10 con 0x563a316eecc0 2023-09-10T01:30:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.273+0000 7f6dfb533b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a315de810 con 0x563a316f41a0 2023-09-10T01:30:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.273+0000 7f6dfb533b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a314c28c0 con 0x563a316ee300 2023-09-10T01:30:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.273+0000 7f6dfb533b80 10 client.0 _ll_register_callbacks cb 0x563a31528a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:30:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.282+0000 7f6ded109700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316ee300 0x563a316fadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.282+0000 7f6ded109700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316ee300 0x563a316fadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39022/0 (socket says 172.21.15.44:39022) 2023-09-10T01:30:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.282+0000 7f6ded109700 1 -- 172.21.15.44:0/3937727479 learned_addr learned my addr 172.21.15.44:0/3937727479 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.282+0000 7f6ded90a700 1 --2- 172.21.15.44:0/3937727479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316f41a0 0x563a316f86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.282+0000 7f6dec908700 1 --2- 172.21.15.44:0/3937727479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316eecc0 0x563a316f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.283+0000 7f6ded109700 1 -- 172.21.15.44:0/3937727479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316f41a0 msgr2=0x563a316f86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.283+0000 7f6ded109700 1 --2- 172.21.15.44:0/3937727479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316f41a0 0x563a316f86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.283+0000 7f6ded109700 1 -- 172.21.15.44:0/3937727479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316eecc0 msgr2=0x563a316f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.283+0000 7f6ded109700 1 --2- 172.21.15.44:0/3937727479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316eecc0 0x563a316f8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.283+0000 7f6ded109700 1 -- 172.21.15.44:0/3937727479 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de000acc0 con 0x563a316ee300 2023-09-10T01:30:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.283+0000 7f6dec908700 1 --2- 172.21.15.44:0/3937727479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316eecc0 0x563a316f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:30:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.283+0000 7f6ded109700 1 --2- 172.21.15.44:0/3937727479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316ee300 0x563a316fadf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009e90 tx=0x7f6dd800b3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8828f47450f09e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.283+0000 7f6dd0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:30:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.283+0000 7f6dd0ff9700 1 -- 172.21.15.44:0/3937727479 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8011420 con 0x563a316ee300 2023-09-10T01:30:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.283+0000 7f6dd0ff9700 1 -- 172.21.15.44:0/3937727479 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(195..195 src has 1..195) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f6dd8011580 con 0x563a316ee300 2023-09-10T01:30:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.283+0000 7f6dfb533b80 1 -- 172.21.15.44:0/3937727479 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a31598490 con 0x563a316ee300 2023-09-10T01:30:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.283+0000 7f6dd0ff9700 1 -- 172.21.15.44:0/3937727479 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=173) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dd8011740 con 0x563a316ee300 2023-09-10T01:30:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.283+0000 7f6dfb533b80 10 client.30106 fetch_fsmap learned FSMap version 173 2023-09-10T01:30:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.283+0000 7f6dfb533b80 1 -- 172.21.15.44:0/3937727479 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=173,monmap=4+}) v3 -- 0x563a31455c00 con 0x563a316ee300 2023-09-10T01:30:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.284+0000 7f6dd0ff9700 1 -- 172.21.15.44:0/3937727479 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563a31598490 con 0x563a316ee300 2023-09-10T01:30:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.285+0000 7f6dd0ff9700 1 -- 172.21.15.44:0/3937727479 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 173) v1 ==== 7359+0+0 (secure 0 0 0) 0x7f6dd80118a0 con 0x563a316ee300 2023-09-10T01:30:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.285+0000 7f6dfb533b80 10 client.30106 fetch_fsmap finished waiting for FSMap version 173 2023-09-10T01:30:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.285+0000 7f6dfb533b80 10 client.30106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:30:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.285+0000 7f6dfb533b80 20 client.30106 populate_metadata read hostname 'smithi044' 2023-09-10T01:30:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.285+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/3937727479 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a317085b0 0x563a3170a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.285+0000 7f6dfb533b80 4 client.30106 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:30:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.285+0000 7f6dfb533b80 1 -- 172.21.15.44:0/3937727479 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a31655840 con 0x563a317085b0 2023-09-10T01:30:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.285+0000 7f6dec908700 1 --2- 172.21.15.44:0/3937727479 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a317085b0 0x563a3170a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.286+0000 7f6dec908700 1 --2- 172.21.15.44:0/3937727479 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a317085b0 0x563a3170a9d0 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=331a9d82e44c4896 server_cookie=1f38fa422e88c067 in_seq=0 out_seq=0 2023-09-10T01:30:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.286+0000 7f6dd0ff9700 10 client.30106 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:30:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.286+0000 7f6dd0ff9700 1 -- 172.21.15.44:0/3937727479 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563a31655840 con 0x563a317085b0 2023-09-10T01:30:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.286+0000 7f6dd0ff9700 10 client.30106 handle_command_reply: tid=0 2023-09-10T01:30:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.286+0000 7f6dfb533b80 1 client.30106 shutdown 2023-09-10T01:30:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.287+0000 7f6dfb533b80 1 -- 172.21.15.44:0/3937727479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316ee300 msgr2=0x563a316fadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.287+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/3937727479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316ee300 0x563a316fadf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009e90 tx=0x7f6dd800b3c0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.288+0000 7f6dfb533b80 1 -- 172.21.15.44:0/3937727479 shutdown_connections 2023-09-10T01:30:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.288+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/3937727479 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a317085b0 0x563a3170a9d0 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.288+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/3937727479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316f41a0 0x563a316f86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.288+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/3937727479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316ee300 0x563a316fadf0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.288+0000 7f6dd0ff9700 0 client.30106 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:30:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.288+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/3937727479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316eecc0 0x563a316f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.288+0000 7f6dfb533b80 1 -- 172.21.15.44:0/3937727479 >> 172.21.15.44:0/3937727479 conn(0x563a31663030 msgr2=0x563a316e87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.288+0000 7f6dfb533b80 1 -- 172.21.15.44:0/3937727479 shutdown_connections 2023-09-10T01:30:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.288+0000 7f6dfb533b80 1 -- 172.21.15.44:0/3937727479 wait complete. 2023-09-10T01:30:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.290+0000 7f6dfb533b80 20 client.30106 trim_cache size 0 max 16384 2023-09-10T01:30:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.322+0000 7f6dfb533b80 1 Processor -- start 2023-09-10T01:30:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.322+0000 7f6dfb533b80 1 -- start start 2023-09-10T01:30:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.322+0000 7f6dfb533b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316ee300 0x563a316ff520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.322+0000 7f6dfb533b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316eecc0 0x563a316fa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.322+0000 7f6dfb533b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316f41a0 0x563a316f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.322+0000 7f6dfb533b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a316f4b90 con 0x563a316ee300 2023-09-10T01:30:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.322+0000 7f6dfb533b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a316f9460 con 0x563a316f41a0 2023-09-10T01:30:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.322+0000 7f6dfb533b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a316ea960 con 0x563a316eecc0 2023-09-10T01:30:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.322+0000 7f6ded109700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316ee300 0x563a316ff520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.323+0000 7f6ded109700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316ee300 0x563a316ff520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60380/0 (socket says 172.21.15.44:60380) 2023-09-10T01:30:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.323+0000 7f6ded109700 1 -- 172.21.15.44:0/3143077786 learned_addr learned my addr 172.21.15.44:0/3143077786 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.324+0000 7f6ded109700 1 -- 172.21.15.44:0/3143077786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316eecc0 msgr2=0x563a316fa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.324+0000 7f6ded109700 1 --2- 172.21.15.44:0/3143077786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316eecc0 0x563a316fa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.324+0000 7f6ded109700 1 -- 172.21.15.44:0/3143077786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316f41a0 msgr2=0x563a316f0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.324+0000 7f6ded109700 1 --2- 172.21.15.44:0/3143077786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316f41a0 0x563a316f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.324+0000 7f6ded109700 1 -- 172.21.15.44:0/3143077786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a314931f0 con 0x563a316ee300 2023-09-10T01:30:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.325+0000 7f6ded109700 1 --2- 172.21.15.44:0/3143077786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316ee300 0x563a316ff520 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009e60 tx=0x7f6dd8006c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438cd6db98a9b9d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.337+0000 7f6dd3fff700 1 -- 172.21.15.44:0/3143077786 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800bd20 con 0x563a316ee300 2023-09-10T01:30:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.337+0000 7f6dd3fff700 1 -- 172.21.15.44:0/3143077786 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6dd800be80 con 0x563a316ee300 2023-09-10T01:30:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.337+0000 7f6dd3fff700 1 -- 172.21.15.44:0/3143077786 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8024a80 con 0x563a316ee300 2023-09-10T01:30:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.338+0000 7f6dfb533b80 1 -- 172.21.15.44:0/3143077786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316ee300 msgr2=0x563a316ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.338+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/3143077786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316ee300 0x563a316ff520 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009e60 tx=0x7f6dd8006c70 comp rx=0 tx=0).stop 2023-09-10T01:30:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.343+0000 7f6dfb533b80 1 -- 172.21.15.44:0/3143077786 shutdown_connections 2023-09-10T01:30:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.343+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/3143077786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316f41a0 0x563a316f0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.343+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/3143077786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316ee300 0x563a316ff520 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.343+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/3143077786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316eecc0 0x563a316fa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.343+0000 7f6dfb533b80 1 -- 172.21.15.44:0/3143077786 >> 172.21.15.44:0/3143077786 conn(0x563a31663030 msgr2=0x563a316f3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.346+0000 7f6dfb533b80 1 -- 172.21.15.44:0/3143077786 shutdown_connections 2023-09-10T01:30:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.346+0000 7f6dfb533b80 1 -- 172.21.15.44:0/3143077786 wait complete. 2023-09-10T01:30:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.346+0000 7f6dfb533b80 1 -- start start 2023-09-10T01:30:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.346+0000 7f6dfb533b80 1 Processor -- start 2023-09-10T01:30:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.351+0000 7f6dfb533b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316ee300 0x563a316f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.351+0000 7f6dfb533b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316eecc0 0x563a316f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.351+0000 7f6dfb533b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316f41a0 0x563a316f6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.351+0000 7f6dfb533b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a316f6f90 con 0x563a316eecc0 2023-09-10T01:30:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.351+0000 7f6dfb533b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a316f7100 con 0x563a316ee300 2023-09-10T01:30:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.351+0000 7f6dfb533b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a316f72c0 con 0x563a316f41a0 2023-09-10T01:30:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.351+0000 7f6dfb533b80 10 client.0 _ll_register_callbacks cb 0x7f6dd80096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:30:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.352+0000 7f6dec908700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316eecc0 0x563a316f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.352+0000 7f6dec908700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316eecc0 0x563a316f7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60388/0 (socket says 172.21.15.44:60388) 2023-09-10T01:30:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.352+0000 7f6dec908700 1 -- 172.21.15.44:0/1949412074 learned_addr learned my addr 172.21.15.44:0/1949412074 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.353+0000 7f6ded90a700 1 --2- 172.21.15.44:0/1949412074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316f41a0 0x563a316f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.354+0000 7f6ded109700 1 --2- 172.21.15.44:0/1949412074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316ee300 0x563a316f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.354+0000 7f6ded90a700 1 -- 172.21.15.44:0/1949412074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316ee300 msgr2=0x563a316f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.354+0000 7f6ded90a700 1 --2- 172.21.15.44:0/1949412074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316ee300 0x563a316f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.354+0000 7f6ded90a700 1 -- 172.21.15.44:0/1949412074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316eecc0 msgr2=0x563a316f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.354+0000 7f6ded90a700 1 --2- 172.21.15.44:0/1949412074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316eecc0 0x563a316f7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.354+0000 7f6ded90a700 1 -- 172.21.15.44:0/1949412074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a31455c00 con 0x563a316f41a0 2023-09-10T01:30:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.355+0000 7f6ded90a700 1 --2- 172.21.15.44:0/1949412074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316f41a0 0x563a316f6870 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6de40caed0 tx=0x7f6de409b3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=256956e7b7f017e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.355+0000 7f6dd0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:30:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.355+0000 7f6dd0ff9700 1 -- 172.21.15.44:0/1949412074 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de409eaf0 con 0x563a316f41a0 2023-09-10T01:30:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.355+0000 7f6dd0ff9700 1 -- 172.21.15.44:0/1949412074 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(195..195 src has 1..195) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f6de40a2ca0 con 0x563a316f41a0 2023-09-10T01:30:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.355+0000 7f6dfb533b80 1 -- 172.21.15.44:0/1949412074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6de000ce20 con 0x563a316f41a0 2023-09-10T01:30:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.355+0000 7f6dd0ff9700 1 -- 172.21.15.44:0/1949412074 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=173) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6de409a1b0 con 0x563a316f41a0 2023-09-10T01:30:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.355+0000 7f6dfb533b80 10 client.30109 fetch_fsmap learned FSMap version 173 2023-09-10T01:30:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.355+0000 7f6dfb533b80 1 -- 172.21.15.44:0/1949412074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=173,monmap=4+}) v3 -- 0x563a3146b250 con 0x563a316f41a0 2023-09-10T01:30:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.355+0000 7f6dd0ff9700 1 -- 172.21.15.44:0/1949412074 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6de000ce20 con 0x563a316f41a0 2023-09-10T01:30:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.358+0000 7f6dd0ff9700 1 -- 172.21.15.44:0/1949412074 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 173) v1 ==== 7359+0+0 (secure 0 0 0) 0x7f6de40a2810 con 0x563a316f41a0 2023-09-10T01:30:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.358+0000 7f6dfb533b80 10 client.30109 fetch_fsmap finished waiting for FSMap version 173 2023-09-10T01:30:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.358+0000 7f6dfb533b80 10 client.30109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:30:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.358+0000 7f6dfb533b80 20 client.30109 populate_metadata read hostname 'smithi044' 2023-09-10T01:30:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.358+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/1949412074 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a3175d3d0 0x563a3175f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.358+0000 7f6dfb533b80 4 client.30109 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:30:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.358+0000 7f6dfb533b80 1 -- 172.21.15.44:0/1949412074 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563a316398c0 con 0x563a3175d3d0 2023-09-10T01:30:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.362+0000 7f6ded109700 1 --2- 172.21.15.44:0/1949412074 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a3175d3d0 0x563a3175f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.362+0000 7f6dd0ff9700 10 client.30109 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:30:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.362+0000 7f6ded109700 1 --2- 172.21.15.44:0/1949412074 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a3175d3d0 0x563a3175f7f0 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d58001dc37f7caba server_cookie=a9ab594fba3e3ab5 in_seq=0 out_seq=0 2023-09-10T01:30:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.393+0000 7f6dd0ff9700 1 -- 172.21.15.44:0/1949412074 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563a316398c0 con 0x563a3175d3d0 2023-09-10T01:30:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.393+0000 7f6dd0ff9700 10 client.30109 handle_command_reply: tid=0 2023-09-10T01:30:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.393+0000 7f6dfb533b80 1 client.30109 shutdown 2023-09-10T01:30:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.399+0000 7f6dfb533b80 1 -- 172.21.15.44:0/1949412074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316f41a0 msgr2=0x563a316f6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.399+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/1949412074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316f41a0 0x563a316f6870 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6de40caed0 tx=0x7f6de409b3c0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.399+0000 7f6dfb533b80 1 -- 172.21.15.44:0/1949412074 shutdown_connections 2023-09-10T01:30:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.399+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/1949412074 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563a3175d3d0 0x563a3175f7f0 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.399+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/1949412074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a316f41a0 0x563a316f6870 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.399+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/1949412074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563a316ee300 0x563a316f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.399+0000 7f6dfb533b80 1 --2- 172.21.15.44:0/1949412074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563a316eecc0 0x563a316f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.399+0000 7f6dfb533b80 1 -- 172.21.15.44:0/1949412074 >> 172.21.15.44:0/1949412074 conn(0x563a31663030 msgr2=0x563a316e9090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.399+0000 7f6dd0ff9700 0 client.30109 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:30:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.400+0000 7f6dfb533b80 1 -- 172.21.15.44:0/1949412074 shutdown_connections 2023-09-10T01:30:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:30:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:30:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:30:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:30:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.401+0000 7f6dfb533b80 1 -- 172.21.15.44:0/1949412074 wait complete. 2023-09-10T01:30:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.401+0000 7f6dfb533b80 20 client.30109 trim_cache size 0 max 16384 2023-09-10T01:30:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.405+0000 7f6def36d700 1 -- 172.21.15.44:0/848460039 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6dd40a0980 msgr2=0x7f6dd40a2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.405+0000 7f6def36d700 1 --2- 172.21.15.44:0/848460039 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6dd40a0980 0x7f6dd40a2e40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6de0017fd0 tx=0x7f6de000b420 comp rx=0 tx=0).stop 2023-09-10T01:30:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.405+0000 7f6def36d700 1 -- 172.21.15.44:0/848460039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de81c3a70 msgr2=0x7f6de81c3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.405+0000 7f6def36d700 1 --2- 172.21.15.44:0/848460039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de81c3a70 0x7f6de81c3ef0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f6de401cbc0 tx=0x7f6de401cca0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.406+0000 7f6def36d700 1 -- 172.21.15.44:0/848460039 shutdown_connections 2023-09-10T01:30:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.406+0000 7f6def36d700 1 --2- 172.21.15.44:0/848460039 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6dd40a0980 0x7f6dd40a2e40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.406+0000 7f6def36d700 1 --2- 172.21.15.44:0/848460039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de81c3a70 0x7f6de81c3ef0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.406+0000 7f6def36d700 1 --2- 172.21.15.44:0/848460039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6de810ed50 0x7f6de81152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.406+0000 7f6def36d700 1 --2- 172.21.15.44:0/848460039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6de8119de0 0x7f6de8115800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.406+0000 7f6def36d700 1 -- 172.21.15.44:0/848460039 >> 172.21.15.44:0/848460039 conn(0x7f6de806c6c0 msgr2=0x7f6de811b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.406+0000 7f6def36d700 1 -- 172.21.15.44:0/848460039 shutdown_connections 2023-09-10T01:30:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:40.406+0000 7f6def36d700 1 -- 172.21.15.44:0/848460039 wait complete. 2023-09-10T01:30:41.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 575.0 s, 290.6 tps, lat 348.615 ms stddev 129.791 2023-09-10T01:30:42.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:41 smithi131 ceph-mon[103713]: pgmap v1627: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.1 MiB/s wr, 680 op/s 2023-09-10T01:30:42.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:41 smithi044 ceph-mon[98676]: pgmap v1627: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.1 MiB/s wr, 680 op/s 2023-09-10T01:30:42.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:41 smithi099 ceph-mon[102692]: pgmap v1627: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.1 MiB/s wr, 680 op/s 2023-09-10T01:30:44.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:43 smithi131 ceph-mon[103713]: pgmap v1628: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-09-10T01:30:44.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:43 smithi044 ceph-mon[98676]: pgmap v1628: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-09-10T01:30:44.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:43 smithi099 ceph-mon[102692]: pgmap v1628: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-09-10T01:30:46.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:45 smithi131 ceph-mon[103713]: pgmap v1629: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 691 op/s 2023-09-10T01:30:46.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:45 smithi044 ceph-mon[98676]: pgmap v1629: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 691 op/s 2023-09-10T01:30:46.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:45 smithi099 ceph-mon[102692]: pgmap v1629: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 691 op/s 2023-09-10T01:30:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 580.0 s, 291.8 tps, lat 328.439 ms stddev 390.008 2023-09-10T01:30:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:46 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:30:47.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:46 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:30:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:46 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:30:48.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:47 smithi131 ceph-mon[103713]: pgmap v1630: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.5 MiB/s wr, 974 op/s 2023-09-10T01:30:48.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:47 smithi044 ceph-mon[98676]: pgmap v1630: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.5 MiB/s wr, 974 op/s 2023-09-10T01:30:48.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:47 smithi099 ceph-mon[102692]: pgmap v1630: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.5 MiB/s wr, 974 op/s 2023-09-10T01:30:50.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:49 smithi131 ceph-mon[103713]: pgmap v1631: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.1 MiB/s wr, 760 op/s 2023-09-10T01:30:50.061 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:49 smithi044 ceph-mon[98676]: pgmap v1631: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.1 MiB/s wr, 760 op/s 2023-09-10T01:30:50.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:49 smithi099 ceph-mon[102692]: pgmap v1631: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.1 MiB/s wr, 760 op/s 2023-09-10T01:30:51.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 585.0 s, 272.6 tps, lat 357.242 ms stddev 299.075 2023-09-10T01:30:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:51 smithi131 ceph-mon[103713]: pgmap v1632: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.8 MiB/s wr, 694 op/s 2023-09-10T01:30:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:51 smithi044 ceph-mon[98676]: pgmap v1632: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.8 MiB/s wr, 694 op/s 2023-09-10T01:30:52.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:51 smithi099 ceph-mon[102692]: pgmap v1632: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.8 MiB/s wr, 694 op/s 2023-09-10T01:30:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.615+0000 7f69cff95700 1 -- 172.21.15.44:0/1092320736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c8107870 msgr2=0x7f69c8107c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.615+0000 7f69cff95700 1 --2- 172.21.15.44:0/1092320736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c8107870 0x7f69c8107c50 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009990 tx=0x7f69c4009ca0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.615+0000 7f69cff95700 1 -- 172.21.15.44:0/1092320736 shutdown_connections 2023-09-10T01:30:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.615+0000 7f69cff95700 1 --2- 172.21.15.44:0/1092320736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69c810bf10 0x7f69c80788f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.615+0000 7f69cff95700 1 --2- 172.21.15.44:0/1092320736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c8107870 0x7f69c8107c50 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.615+0000 7f69cff95700 1 --2- 172.21.15.44:0/1092320736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8108220 0x7f69c810b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.615+0000 7f69cff95700 1 -- 172.21.15.44:0/1092320736 >> 172.21.15.44:0/1092320736 conn(0x7f69c806c6c0 msgr2=0x7f69c806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.616+0000 7f69cff95700 1 -- 172.21.15.44:0/1092320736 shutdown_connections 2023-09-10T01:30:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.616+0000 7f69cff95700 1 -- 172.21.15.44:0/1092320736 wait complete. 2023-09-10T01:30:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.617+0000 7f69cff95700 1 Processor -- start 2023-09-10T01:30:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.617+0000 7f69cff95700 1 -- start start 2023-09-10T01:30:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.617+0000 7f69cff95700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c8107870 0x7f69c8082ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.617+0000 7f69cff95700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69c810bf10 0x7f69c80768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.617+0000 7f69cff95700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8076e20 0x7f69c80772d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.617+0000 7f69cff95700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c81c2fa0 con 0x7f69c8107870 2023-09-10T01:30:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.617+0000 7f69cff95700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c81c30e0 con 0x7f69c810bf10 2023-09-10T01:30:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.617+0000 7f69cff95700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c81c3220 con 0x7f69c8076e20 2023-09-10T01:30:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.618+0000 7f69ce532700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8076e20 0x7f69c80772d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.618+0000 7f69cdd31700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c8107870 0x7f69c8082ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.618+0000 7f69cdd31700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c8107870 0x7f69c8082ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60400/0 (socket says 172.21.15.44:60400) 2023-09-10T01:30:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.618+0000 7f69cdd31700 1 -- 172.21.15.44:0/2344713451 learned_addr learned my addr 172.21.15.44:0/2344713451 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.618+0000 7f69cd530700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69c810bf10 0x7f69c80768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.618+0000 7f69ce532700 1 -- 172.21.15.44:0/2344713451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69c810bf10 msgr2=0x7f69c80768e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.618+0000 7f69ce532700 1 --2- 172.21.15.44:0/2344713451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69c810bf10 0x7f69c80768e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.618+0000 7f69ce532700 1 -- 172.21.15.44:0/2344713451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c8107870 msgr2=0x7f69c8082ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.618+0000 7f69ce532700 1 --2- 172.21.15.44:0/2344713451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c8107870 0x7f69c8082ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.618+0000 7f69ce532700 1 -- 172.21.15.44:0/2344713451 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c40095f0 con 0x7f69c8076e20 2023-09-10T01:30:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.618+0000 7f69cd530700 1 --2- 172.21.15.44:0/2344713451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69c810bf10 0x7f69c80768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:30:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.619+0000 7f69ce532700 1 --2- 172.21.15.44:0/2344713451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8076e20 0x7f69c80772d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f69c0007f10 tx=0x7f69c000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.620+0000 7f69beffd700 1 -- 172.21.15.44:0/2344713451 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0004db0 con 0x7f69c8076e20 2023-09-10T01:30:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.620+0000 7f69cff95700 1 -- 172.21.15.44:0/2344713451 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c81c3450 con 0x7f69c8076e20 2023-09-10T01:30:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.620+0000 7f69beffd700 1 -- 172.21.15.44:0/2344713451 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f69c000f040 con 0x7f69c8076e20 2023-09-10T01:30:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.620+0000 7f69beffd700 1 -- 172.21.15.44:0/2344713451 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0005960 con 0x7f69c8076e20 2023-09-10T01:30:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.624+0000 7f69cff95700 1 -- 172.21.15.44:0/2344713451 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c81c3900 con 0x7f69c8076e20 2023-09-10T01:30:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.630+0000 7f69dc15bb80 1 Processor -- start 2023-09-10T01:30:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.633+0000 7f69beffd700 1 -- 172.21.15.44:0/2344713451 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f69c000db20 con 0x7f69c8076e20 2023-09-10T01:30:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.634+0000 7f69beffd700 1 --2- 172.21.15.44:0/2344713451 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f69b40a0960 0x7f69b40a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.634+0000 7f69beffd700 1 -- 172.21.15.44:0/2344713451 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f69c00cbc20 con 0x7f69c8076e20 2023-09-10T01:30:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.637+0000 7f69cd530700 1 --2- 172.21.15.44:0/2344713451 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f69b40a0960 0x7f69b40a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.638+0000 7f69cd530700 1 --2- 172.21.15.44:0/2344713451 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f69b40a0960 0x7f69b40a2e20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f69c8078290 tx=0x7f69b8009190 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.642+0000 7f69dc15bb80 1 -- start start 2023-09-10T01:30:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.642+0000 7f69dc15bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b430c1a0 0x55c2b430c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.642+0000 7f69dc15bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b4306300 0x55c2b4306780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.642+0000 7f69dc15bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306cc0 0x55c2b4312f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.642+0000 7f69dc15bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b425f6d0 con 0x55c2b430c1a0 2023-09-10T01:30:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.642+0000 7f69dc15bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b425cff0 con 0x55c2b4306300 2023-09-10T01:30:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.642+0000 7f69dc15bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b42588e0 con 0x55c2b4306cc0 2023-09-10T01:30:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.642+0000 7f69cdd31700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b430c1a0 0x55c2b430c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.642+0000 7f69cdd31700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b430c1a0 0x55c2b430c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60408/0 (socket says 172.21.15.44:60408) 2023-09-10T01:30:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.642+0000 7f69cdd31700 1 -- 172.21.15.44:0/1331079373 learned_addr learned my addr 172.21.15.44:0/1331079373 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.646+0000 7f69cd530700 1 --2- 172.21.15.44:0/1331079373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b4306300 0x55c2b4306780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.647+0000 7f69ce532700 1 --2- 172.21.15.44:0/1331079373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306cc0 0x55c2b4312f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.647+0000 7f69cdd31700 1 -- 172.21.15.44:0/1331079373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306cc0 msgr2=0x55c2b4312f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.647+0000 7f69cdd31700 1 --2- 172.21.15.44:0/1331079373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306cc0 0x55c2b4312f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.647+0000 7f69cdd31700 1 -- 172.21.15.44:0/1331079373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b4306300 msgr2=0x55c2b4306780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.647+0000 7f69cdd31700 1 --2- 172.21.15.44:0/1331079373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b4306300 0x55c2b4306780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.647+0000 7f69cdd31700 1 -- 172.21.15.44:0/1331079373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2b4084d80 con 0x55c2b430c1a0 2023-09-10T01:30:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.647+0000 7f69ce532700 1 --2- 172.21.15.44:0/1331079373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306cc0 0x55c2b4312f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:30:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.647+0000 7f69cd530700 1 --2- 172.21.15.44:0/1331079373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b4306300 0x55c2b4306780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:30:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.648+0000 7f69cdd31700 1 --2- 172.21.15.44:0/1331079373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b430c1a0 0x55c2b430c580 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009fc0 tx=0x7f69c400b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f627be267dce7942 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.657+0000 7f69b3fff700 1 -- 172.21.15.44:0/1331079373 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c401d070 con 0x55c2b430c1a0 2023-09-10T01:30:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.657+0000 7f69b3fff700 1 -- 172.21.15.44:0/1331079373 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f69c401e040 con 0x55c2b430c1a0 2023-09-10T01:30:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.657+0000 7f69b3fff700 1 -- 172.21.15.44:0/1331079373 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c4004760 con 0x55c2b430c1a0 2023-09-10T01:30:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.658+0000 7f69dc15bb80 1 -- 172.21.15.44:0/1331079373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b430c1a0 msgr2=0x55c2b430c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.658+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/1331079373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b430c1a0 0x55c2b430c580 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009fc0 tx=0x7f69c400b7d0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.670+0000 7f69dc15bb80 1 -- 172.21.15.44:0/1331079373 shutdown_connections 2023-09-10T01:30:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.670+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/1331079373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306cc0 0x55c2b4312f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.670+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/1331079373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b430c1a0 0x55c2b430c580 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.670+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/1331079373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b4306300 0x55c2b4306780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.670+0000 7f69dc15bb80 1 -- 172.21.15.44:0/1331079373 >> 172.21.15.44:0/1331079373 conn(0x55c2b427b030 msgr2=0x55c2b4301560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.680+0000 7f69dc15bb80 1 -- 172.21.15.44:0/1331079373 shutdown_connections 2023-09-10T01:30:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.681+0000 7f69dc15bb80 1 -- 172.21.15.44:0/1331079373 wait complete. 2023-09-10T01:30:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.682+0000 7f69dc15bb80 1 -- start start 2023-09-10T01:30:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.682+0000 7f69dc15bb80 1 Processor -- start 2023-09-10T01:30:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.692+0000 7f69dc15bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b4306300 0x55c2b4312d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.692+0000 7f69dc15bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b4306cc0 0x55c2b4310040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.692+0000 7f69dc15bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b430c1a0 0x55c2b4310580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.692+0000 7f69dc15bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b41f6b10 con 0x55c2b4306cc0 2023-09-10T01:30:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.692+0000 7f69dc15bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b41f6810 con 0x55c2b4306300 2023-09-10T01:30:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.692+0000 7f69dc15bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b40da8c0 con 0x55c2b430c1a0 2023-09-10T01:30:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.692+0000 7f69dc15bb80 10 client.0 _ll_register_callbacks cb 0x55c2b4140a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:30:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.692+0000 7f69cdd31700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b4306300 0x55c2b4312d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.692+0000 7f69cdd31700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b4306300 0x55c2b4312d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33922/0 (socket says 172.21.15.44:33922) 2023-09-10T01:30:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.692+0000 7f69cdd31700 1 -- 172.21.15.44:0/3703982388 learned_addr learned my addr 172.21.15.44:0/3703982388 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.693+0000 7f69cd530700 1 --2- 172.21.15.44:0/3703982388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b4306cc0 0x55c2b4310040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.693+0000 7f69cd530700 1 -- 172.21.15.44:0/3703982388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b430c1a0 msgr2=0x55c2b4310580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:30:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.693+0000 7f69cd530700 1 --2- 172.21.15.44:0/3703982388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b430c1a0 0x55c2b4310580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.693+0000 7f69cd530700 1 -- 172.21.15.44:0/3703982388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b4306300 msgr2=0x55c2b4312d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.693+0000 7f69cd530700 1 --2- 172.21.15.44:0/3703982388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b4306300 0x55c2b4312d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.693+0000 7f69cd530700 1 -- 172.21.15.44:0/3703982388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b800ea20 con 0x55c2b4306cc0 2023-09-10T01:30:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.693+0000 7f69cd530700 1 --2- 172.21.15.44:0/3703982388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b4306cc0 0x55c2b4310040 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f69c810d4a0 tx=0x7f69b8004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7a583d0287851a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.693+0000 7f69b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:30:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.694+0000 7f69b17fa700 1 -- 172.21.15.44:0/3703982388 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8013660 con 0x55c2b4306cc0 2023-09-10T01:30:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.694+0000 7f69b17fa700 1 -- 172.21.15.44:0/3703982388 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(195..195 src has 1..195) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f69b80137c0 con 0x55c2b4306cc0 2023-09-10T01:30:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.694+0000 7f69dc15bb80 1 -- 172.21.15.44:0/3703982388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2b41b0490 con 0x55c2b4306cc0 2023-09-10T01:30:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.694+0000 7f69b17fa700 1 -- 172.21.15.44:0/3703982388 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69b8005a20 con 0x55c2b4306cc0 2023-09-10T01:30:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.694+0000 7f69dc15bb80 10 client.20454 fetch_fsmap learned FSMap version 174 2023-09-10T01:30:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.694+0000 7f69dc15bb80 1 -- 172.21.15.44:0/3703982388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=4+}) v3 -- 0x7f69c80791b0 con 0x55c2b4306cc0 2023-09-10T01:30:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.695+0000 7f69b17fa700 1 -- 172.21.15.44:0/3703982388 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c2b41b0490 con 0x55c2b4306cc0 2023-09-10T01:30:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.695+0000 7f69b17fa700 1 -- 172.21.15.44:0/3703982388 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 174) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f69b80061c0 con 0x55c2b4306cc0 2023-09-10T01:30:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.696+0000 7f69dc15bb80 10 client.20454 fetch_fsmap finished waiting for FSMap version 174 2023-09-10T01:30:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.696+0000 7f69dc15bb80 10 client.20454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:30:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.696+0000 7f69dc15bb80 20 client.20454 populate_metadata read hostname 'smithi044' 2023-09-10T01:30:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.696+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/3703982388 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c2b4331370 0x55c2b4333830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.696+0000 7f69dc15bb80 4 client.20454 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:30:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.696+0000 7f69dc15bb80 1 -- 172.21.15.44:0/3703982388 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2b426d840 con 0x55c2b4331370 2023-09-10T01:30:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.697+0000 7f69cdd31700 1 --2- 172.21.15.44:0/3703982388 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c2b4331370 0x55c2b4333830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.704+0000 7f69b17fa700 10 client.20454 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:30:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.704+0000 7f69cdd31700 1 --2- 172.21.15.44:0/3703982388 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c2b4331370 0x55c2b4333830 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e8cb249f3fb562 server_cookie=1d274cde09b231ba in_seq=0 out_seq=0 2023-09-10T01:30:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.710+0000 7f69b17fa700 1 -- 172.21.15.44:0/3703982388 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c2b426d840 con 0x55c2b4331370 2023-09-10T01:30:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.710+0000 7f69b17fa700 10 client.20454 handle_command_reply: tid=0 2023-09-10T01:30:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.712+0000 7f69dc15bb80 1 client.20454 shutdown 2023-09-10T01:30:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.717+0000 7f69dc15bb80 1 -- 172.21.15.44:0/3703982388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b4306cc0 msgr2=0x55c2b4310040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.717+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/3703982388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b4306cc0 0x55c2b4310040 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f69c810d4a0 tx=0x7f69b8004800 comp rx=0 tx=0).stop 2023-09-10T01:30:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.717+0000 7f69dc15bb80 1 -- 172.21.15.44:0/3703982388 shutdown_connections 2023-09-10T01:30:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.717+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/3703982388 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c2b4331370 0x55c2b4333830 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.717+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/3703982388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b430c1a0 0x55c2b4310580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.717+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/3703982388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b4306300 0x55c2b4312d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.717+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/3703982388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b4306cc0 0x55c2b4310040 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.717+0000 7f69dc15bb80 1 -- 172.21.15.44:0/3703982388 >> 172.21.15.44:0/3703982388 conn(0x55c2b427b030 msgr2=0x55c2b43007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.717+0000 7f69b17fa700 0 client.20454 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:30:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.720+0000 7f69dc15bb80 1 -- 172.21.15.44:0/3703982388 shutdown_connections 2023-09-10T01:30:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.720+0000 7f69dc15bb80 1 -- 172.21.15.44:0/3703982388 wait complete. 2023-09-10T01:30:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.721+0000 7f69dc15bb80 20 client.20454 trim_cache size 0 max 16384 2023-09-10T01:30:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.747+0000 7f69dc15bb80 1 Processor -- start 2023-09-10T01:30:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.747+0000 7f69dc15bb80 1 -- start start 2023-09-10T01:30:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.747+0000 7f69dc15bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306300 0x55c2b430f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.747+0000 7f69dc15bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b4306cc0 0x55c2b4312850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.747+0000 7f69dc15bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b430c1a0 0x55c2b4300f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.747+0000 7f69dc15bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b42fdfc0 con 0x55c2b430c1a0 2023-09-10T01:30:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.747+0000 7f69dc15bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b4311620 con 0x55c2b4306cc0 2023-09-10T01:30:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.747+0000 7f69dc15bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b430e620 con 0x55c2b4306300 2023-09-10T01:30:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.748+0000 7f69ce532700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b430c1a0 0x55c2b4300f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.748+0000 7f69cdd31700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306300 0x55c2b430f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.748+0000 7f69ce532700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b430c1a0 0x55c2b4300f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60426/0 (socket says 172.21.15.44:60426) 2023-09-10T01:30:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.748+0000 7f69ce532700 1 -- 172.21.15.44:0/763191514 learned_addr learned my addr 172.21.15.44:0/763191514 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.748+0000 7f69ce532700 1 -- 172.21.15.44:0/763191514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306300 msgr2=0x55c2b430f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.748+0000 7f69ce532700 1 --2- 172.21.15.44:0/763191514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306300 0x55c2b430f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.748+0000 7f69ce532700 1 -- 172.21.15.44:0/763191514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b4306cc0 msgr2=0x55c2b4312850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.748+0000 7f69ce532700 1 --2- 172.21.15.44:0/763191514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b4306cc0 0x55c2b4312850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.748+0000 7f69ce532700 1 -- 172.21.15.44:0/763191514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2b406dc00 con 0x55c2b430c1a0 2023-09-10T01:30:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.748+0000 7f69ce532700 1 --2- 172.21.15.44:0/763191514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b430c1a0 0x55c2b4300f20 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f69c0007ee0 tx=0x7f69c00950d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6177364a2fc42e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.751+0000 7f69bcff9700 1 -- 172.21.15.44:0/763191514 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0005ac0 con 0x55c2b430c1a0 2023-09-10T01:30:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.751+0000 7f69bcff9700 1 -- 172.21.15.44:0/763191514 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f69c0099380 con 0x55c2b430c1a0 2023-09-10T01:30:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.751+0000 7f69bcff9700 1 -- 172.21.15.44:0/763191514 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c00c91a0 con 0x55c2b430c1a0 2023-09-10T01:30:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.752+0000 7f69dc15bb80 1 -- 172.21.15.44:0/763191514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b430c1a0 msgr2=0x55c2b4300f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.752+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/763191514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b430c1a0 0x55c2b4300f20 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f69c0007ee0 tx=0x7f69c00950d0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.752+0000 7f69dc15bb80 1 -- 172.21.15.44:0/763191514 shutdown_connections 2023-09-10T01:30:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.752+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/763191514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b430c1a0 0x55c2b4300f20 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.752+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/763191514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306300 0x55c2b430f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.752+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/763191514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b4306cc0 0x55c2b4312850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.752+0000 7f69dc15bb80 1 -- 172.21.15.44:0/763191514 >> 172.21.15.44:0/763191514 conn(0x55c2b427b030 msgr2=0x55c2b4305b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.755+0000 7f69dc15bb80 1 -- 172.21.15.44:0/763191514 shutdown_connections 2023-09-10T01:30:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.769+0000 7f69dc15bb80 1 -- 172.21.15.44:0/763191514 wait complete. 2023-09-10T01:30:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.770+0000 7f69dc15bb80 1 -- start start 2023-09-10T01:30:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.770+0000 7f69dc15bb80 1 Processor -- start 2023-09-10T01:30:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.772+0000 7f69dc15bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b4306300 0x55c2b4328530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.772+0000 7f69dc15bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306cc0 0x55c2b4326970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.772+0000 7f69dc15bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b430c1a0 0x55c2b4326eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.772+0000 7f69dc15bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b43275c0 con 0x55c2b4306300 2023-09-10T01:30:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.772+0000 7f69dc15bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b4327730 con 0x55c2b430c1a0 2023-09-10T01:30:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.772+0000 7f69dc15bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b4385290 con 0x55c2b4306cc0 2023-09-10T01:30:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.772+0000 7f69dc15bb80 10 client.0 _ll_register_callbacks cb 0x7f69b8015510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:30:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.772+0000 7f69cd530700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306cc0 0x55c2b4326970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.772+0000 7f69cd530700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306cc0 0x55c2b4326970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39080/0 (socket says 172.21.15.44:39080) 2023-09-10T01:30:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.772+0000 7f69cd530700 1 -- 172.21.15.44:0/1214765460 learned_addr learned my addr 172.21.15.44:0/1214765460 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:30:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.772+0000 7f69ce532700 1 --2- 172.21.15.44:0/1214765460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b430c1a0 0x55c2b4326eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.773+0000 7f69cdd31700 1 --2- 172.21.15.44:0/1214765460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b4306300 0x55c2b4328530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.773+0000 7f69cd530700 1 -- 172.21.15.44:0/1214765460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b430c1a0 msgr2=0x55c2b4326eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.773+0000 7f69cd530700 1 --2- 172.21.15.44:0/1214765460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b430c1a0 0x55c2b4326eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.773+0000 7f69cd530700 1 -- 172.21.15.44:0/1214765460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b4306300 msgr2=0x55c2b4328530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:30:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.773+0000 7f69cd530700 1 --2- 172.21.15.44:0/1214765460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b4306300 0x55c2b4328530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.773+0000 7f69cd530700 1 -- 172.21.15.44:0/1214765460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c81c3900 con 0x55c2b4306cc0 2023-09-10T01:30:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.773+0000 7f69ce532700 1 --2- 172.21.15.44:0/1214765460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b430c1a0 0x55c2b4326eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:30:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.773+0000 7f69cd530700 1 --2- 172.21.15.44:0/1214765460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306cc0 0x55c2b4326970 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f69b8021440 tx=0x7f69b8013890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af1e409a99d07cfa server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:30:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.773+0000 7f69b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:30:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.773+0000 7f69b17fa700 1 -- 172.21.15.44:0/1214765460 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8004030 con 0x55c2b4306cc0 2023-09-10T01:30:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.773+0000 7f69dc15bb80 1 -- 172.21.15.44:0/1214765460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69c401e040 con 0x55c2b4306cc0 2023-09-10T01:30:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.773+0000 7f69b17fa700 1 -- 172.21.15.44:0/1214765460 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(195..195 src has 1..195) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f69b8004190 con 0x55c2b4306cc0 2023-09-10T01:30:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.773+0000 7f69b17fa700 1 -- 172.21.15.44:0/1214765460 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69b8004350 con 0x55c2b4306cc0 2023-09-10T01:30:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.773+0000 7f69dc15bb80 10 client.30118 fetch_fsmap learned FSMap version 174 2023-09-10T01:30:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.773+0000 7f69dc15bb80 1 -- 172.21.15.44:0/1214765460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=4+}) v3 -- 0x55c2b40ab1f0 con 0x55c2b4306cc0 2023-09-10T01:30:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.774+0000 7f69b17fa700 1 -- 172.21.15.44:0/1214765460 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f69c401e040 con 0x55c2b4306cc0 2023-09-10T01:30:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.775+0000 7f69b17fa700 1 -- 172.21.15.44:0/1214765460 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 174) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f69b8005a20 con 0x55c2b4306cc0 2023-09-10T01:30:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.775+0000 7f69dc15bb80 10 client.30118 fetch_fsmap finished waiting for FSMap version 174 2023-09-10T01:30:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.775+0000 7f69dc15bb80 10 client.30118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:30:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.775+0000 7f69dc15bb80 20 client.30118 populate_metadata read hostname 'smithi044' 2023-09-10T01:30:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.775+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/1214765460 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c2b4386990 0x55c2b4388db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:30:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.775+0000 7f69dc15bb80 4 client.30118 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:30:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.775+0000 7f69dc15bb80 1 -- 172.21.15.44:0/1214765460 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c2b42518c0 con 0x55c2b4386990 2023-09-10T01:30:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.775+0000 7f69cdd31700 1 --2- 172.21.15.44:0/1214765460 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c2b4386990 0x55c2b4388db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:30:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.775+0000 7f69b17fa700 10 client.30118 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:30:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.775+0000 7f69cdd31700 1 --2- 172.21.15.44:0/1214765460 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c2b4386990 0x55c2b4388db0 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb4b379a0135a401 server_cookie=170a75cf034f541f in_seq=0 out_seq=0 2023-09-10T01:30:53.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.802+0000 7f69b17fa700 1 -- 172.21.15.44:0/1214765460 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c2b42518c0 con 0x55c2b4386990 2023-09-10T01:30:53.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.802+0000 7f69b17fa700 10 client.30118 handle_command_reply: tid=0 2023-09-10T01:30:53.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.802+0000 7f69dc15bb80 1 client.30118 shutdown 2023-09-10T01:30:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.802+0000 7f69dc15bb80 1 -- 172.21.15.44:0/1214765460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306cc0 msgr2=0x55c2b4326970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.802+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/1214765460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306cc0 0x55c2b4326970 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f69b8021440 tx=0x7f69b8013890 comp rx=0 tx=0).stop 2023-09-10T01:30:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.805+0000 7f69dc15bb80 1 -- 172.21.15.44:0/1214765460 shutdown_connections 2023-09-10T01:30:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.805+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/1214765460 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c2b4386990 0x55c2b4388db0 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.805+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/1214765460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2b430c1a0 0x55c2b4326eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.805+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/1214765460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2b4306300 0x55c2b4328530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.805+0000 7f69dc15bb80 1 --2- 172.21.15.44:0/1214765460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2b4306cc0 0x55c2b4326970 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.805+0000 7f69dc15bb80 1 -- 172.21.15.44:0/1214765460 >> 172.21.15.44:0/1214765460 conn(0x55c2b427b030 msgr2=0x55c2b432cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.805+0000 7f69b17fa700 0 client.30118 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:30:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.806+0000 7f69dc15bb80 1 -- 172.21.15.44:0/1214765460 shutdown_connections 2023-09-10T01:30:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.806+0000 7f69dc15bb80 1 -- 172.21.15.44:0/1214765460 wait complete. 2023-09-10T01:30:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.806+0000 7f69dc15bb80 20 client.30118 trim_cache size 0 max 16384 2023-09-10T01:30:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:30:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:30:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:30:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:30:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.807+0000 7f69cff95700 1 -- 172.21.15.44:0/2344713451 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f69b40a0960 msgr2=0x7f69b40a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.807+0000 7f69cff95700 1 --2- 172.21.15.44:0/2344713451 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f69b40a0960 0x7f69b40a2e20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f69c8078290 tx=0x7f69b8009190 comp rx=0 tx=0).stop 2023-09-10T01:30:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.807+0000 7f69cff95700 1 -- 172.21.15.44:0/2344713451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8076e20 msgr2=0x7f69c80772d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:30:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.807+0000 7f69cff95700 1 --2- 172.21.15.44:0/2344713451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8076e20 0x7f69c80772d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f69c0007f10 tx=0x7f69c000d3b0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.807+0000 7f69cff95700 1 -- 172.21.15.44:0/2344713451 shutdown_connections 2023-09-10T01:30:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.807+0000 7f69cff95700 1 --2- 172.21.15.44:0/2344713451 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f69b40a0960 0x7f69b40a2e20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.807+0000 7f69cff95700 1 --2- 172.21.15.44:0/2344713451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69c8076e20 0x7f69c80772d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.807+0000 7f69cff95700 1 --2- 172.21.15.44:0/2344713451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c8107870 0x7f69c8082ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.807+0000 7f69cff95700 1 --2- 172.21.15.44:0/2344713451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69c810bf10 0x7f69c80768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:30:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.807+0000 7f69cff95700 1 -- 172.21.15.44:0/2344713451 >> 172.21.15.44:0/2344713451 conn(0x7f69c806c6c0 msgr2=0x7f69c806ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:30:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.814+0000 7f69cff95700 1 -- 172.21.15.44:0/2344713451 shutdown_connections 2023-09-10T01:30:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:30:53.814+0000 7f69cff95700 1 -- 172.21.15.44:0/2344713451 wait complete. 2023-09-10T01:30:54.051 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:53 smithi131 ceph-mon[103713]: pgmap v1633: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.8 MiB/s wr, 1.03k op/s 2023-09-10T01:30:54.071 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:53 smithi044 ceph-mon[98676]: pgmap v1633: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.8 MiB/s wr, 1.03k op/s 2023-09-10T01:30:54.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:53 smithi099 ceph-mon[102692]: pgmap v1633: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.8 MiB/s wr, 1.03k op/s 2023-09-10T01:30:56.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:55 smithi131 ceph-mon[103713]: pgmap v1634: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 679 op/s 2023-09-10T01:30:56.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:55 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:30:56.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:55 smithi044 ceph-mon[98676]: pgmap v1634: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 679 op/s 2023-09-10T01:30:56.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:55 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:30:56.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:55 smithi099 ceph-mon[102692]: pgmap v1634: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 679 op/s 2023-09-10T01:30:56.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:55 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:30:56.183 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 590.0 s, 296.1 tps, lat 327.089 ms stddev 531.389 2023-09-10T01:30:57.797 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:57 smithi131 ceph-mon[103713]: pgmap v1635: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.1 MiB/s wr, 974 op/s 2023-09-10T01:30:57.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:57 smithi044 ceph-mon[98676]: pgmap v1635: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.1 MiB/s wr, 974 op/s 2023-09-10T01:30:57.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:57 smithi099 ceph-mon[102692]: pgmap v1635: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.1 MiB/s wr, 974 op/s 2023-09-10T01:30:59.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:30:59 smithi131 ceph-mon[103713]: pgmap v1636: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.6 MiB/s wr, 771 op/s 2023-09-10T01:30:59.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:30:59 smithi044 ceph-mon[98676]: pgmap v1636: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.6 MiB/s wr, 771 op/s 2023-09-10T01:30:59.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:30:59 smithi099 ceph-mon[102692]: pgmap v1636: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.6 MiB/s wr, 771 op/s 2023-09-10T01:31:01.183 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 595.0 s, 60.6 tps, lat 461.630 ms stddev 650.772 2023-09-10T01:31:01.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:01 smithi099 ceph-mon[102692]: pgmap v1637: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.9 MiB/s wr, 705 op/s 2023-09-10T01:31:02.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:01 smithi131 ceph-mon[103713]: pgmap v1637: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.9 MiB/s wr, 705 op/s 2023-09-10T01:31:02.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:01 smithi044 ceph-mon[98676]: pgmap v1637: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.9 MiB/s wr, 705 op/s 2023-09-10T01:31:03.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:03 smithi099 ceph-mon[102692]: pgmap v1638: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.3 MiB/s wr, 914 op/s 2023-09-10T01:31:04.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:03 smithi131 ceph-mon[103713]: pgmap v1638: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.3 MiB/s wr, 914 op/s 2023-09-10T01:31:04.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:03 smithi044 ceph-mon[98676]: pgmap v1638: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.3 MiB/s wr, 914 op/s 2023-09-10T01:31:05.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:05 smithi099 ceph-mon[102692]: pgmap v1639: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 582 op/s 2023-09-10T01:31:05.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:05 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:06.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:05 smithi131 ceph-mon[103713]: pgmap v1639: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 582 op/s 2023-09-10T01:31:06.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:05 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:06.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:05 smithi044 ceph-mon[98676]: pgmap v1639: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 582 op/s 2023-09-10T01:31:06.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:05 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:06.182 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 600.0 s, 58.6 tps, lat 2349.367 ms stddev 2363.901 2023-09-10T01:31:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.041+0000 7fb26640a700 1 -- 172.21.15.44:0/887418588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2600729a0 msgr2=0x7fb260072d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.041+0000 7fb26640a700 1 --2- 172.21.15.44:0/887418588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2600729a0 0x7fb260072d80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb2540098f0 tx=0x7fb254009c00 comp rx=0 tx=0).stop 2023-09-10T01:31:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.042+0000 7fb26640a700 1 -- 172.21.15.44:0/887418588 shutdown_connections 2023-09-10T01:31:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.042+0000 7fb26640a700 1 --2- 172.21.15.44:0/887418588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2601115d0 0x7fb2601139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.042+0000 7fb26640a700 1 --2- 172.21.15.44:0/887418588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2600729a0 0x7fb260072d80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.042+0000 7fb26640a700 1 --2- 172.21.15.44:0/887418588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2600732c0 0x7fb260111090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.042+0000 7fb26640a700 1 -- 172.21.15.44:0/887418588 >> 172.21.15.44:0/887418588 conn(0x7fb2600fc0d0 msgr2=0x7fb2600fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.042+0000 7fb26640a700 1 -- 172.21.15.44:0/887418588 shutdown_connections 2023-09-10T01:31:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.042+0000 7fb26640a700 1 -- 172.21.15.44:0/887418588 wait complete. 2023-09-10T01:31:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.043+0000 7fb26640a700 1 Processor -- start 2023-09-10T01:31:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.043+0000 7fb26640a700 1 -- start start 2023-09-10T01:31:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.044+0000 7fb26640a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2600729a0 0x7fb26010c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.044+0000 7fb26640a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2600732c0 0x7fb26010cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.044+0000 7fb26640a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2601115d0 0x7fb26019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.044+0000 7fb26640a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26019a9e0 con 0x7fb2601115d0 2023-09-10T01:31:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.044+0000 7fb26640a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26019ab50 con 0x7fb2600732c0 2023-09-10T01:31:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.044+0000 7fb26640a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26019ad10 con 0x7fb2600729a0 2023-09-10T01:31:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.044+0000 7fb2649a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2601115d0 0x7fb26019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.044+0000 7fb2649a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2601115d0 0x7fb26019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60446/0 (socket says 172.21.15.44:60446) 2023-09-10T01:31:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.044+0000 7fb2649a7700 1 -- 172.21.15.44:0/1280739505 learned_addr learned my addr 172.21.15.44:0/1280739505 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.044+0000 7fb25f7fe700 1 --2- 172.21.15.44:0/1280739505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2600732c0 0x7fb26010cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.046+0000 7fb25f7fe700 1 -- 172.21.15.44:0/1280739505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2600729a0 msgr2=0x7fb26010c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.046+0000 7fb25f7fe700 1 --2- 172.21.15.44:0/1280739505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2600729a0 0x7fb26010c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.046+0000 7fb25f7fe700 1 -- 172.21.15.44:0/1280739505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2601115d0 msgr2=0x7fb26019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.046+0000 7fb25f7fe700 1 --2- 172.21.15.44:0/1280739505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2601115d0 0x7fb26019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.046+0000 7fb25f7fe700 1 -- 172.21.15.44:0/1280739505 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb254009520 con 0x7fb2600732c0 2023-09-10T01:31:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.046+0000 7fb25f7fe700 1 --2- 172.21.15.44:0/1280739505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2600732c0 0x7fb26010cd90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb25000e9f0 tx=0x7fb25000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.046+0000 7fb25d7fa700 1 -- 172.21.15.44:0/1280739505 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25000eee0 con 0x7fb2600732c0 2023-09-10T01:31:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.046+0000 7fb25d7fa700 1 -- 172.21.15.44:0/1280739505 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb25000cbd0 con 0x7fb2600732c0 2023-09-10T01:31:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.046+0000 7fb25d7fa700 1 -- 172.21.15.44:0/1280739505 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250018680 con 0x7fb2600732c0 2023-09-10T01:31:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.047+0000 7fb26640a700 1 -- 172.21.15.44:0/1280739505 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26019aff0 con 0x7fb2600732c0 2023-09-10T01:31:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.047+0000 7fb26640a700 1 -- 172.21.15.44:0/1280739505 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26006a200 con 0x7fb2600732c0 2023-09-10T01:31:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.052+0000 7fb2725d0b80 1 Processor -- start 2023-09-10T01:31:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.052+0000 7fb2725d0b80 1 -- start start 2023-09-10T01:31:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.053+0000 7fb2725d0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635cee1a0 0x55f635cee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.053+0000 7fb2725d0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8300 0x55f635ce8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.053+0000 7fb2725d0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635ce8cc0 0x55f635cf4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.053+0000 7fb2725d0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f635c416d0 con 0x55f635ce8cc0 2023-09-10T01:31:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.053+0000 7fb2725d0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f635c3eff0 con 0x55f635cee1a0 2023-09-10T01:31:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.053+0000 7fb2725d0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f635c3a8e0 con 0x55f635ce8300 2023-09-10T01:31:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.053+0000 7fb25ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8300 0x55f635ce8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.053+0000 7fb25ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8300 0x55f635ce8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39100/0 (socket says 172.21.15.44:39100) 2023-09-10T01:31:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.053+0000 7fb25ffff700 1 -- 172.21.15.44:0/3992982410 learned_addr learned my addr 172.21.15.44:0/3992982410 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.053+0000 7fb2649a7700 1 --2- 172.21.15.44:0/3992982410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635cee1a0 0x55f635cee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.053+0000 7fb25ffff700 1 -- 172.21.15.44:0/3992982410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635cee1a0 msgr2=0x55f635cee580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.053+0000 7fb25ffff700 1 --2- 172.21.15.44:0/3992982410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635cee1a0 0x55f635cee580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.053+0000 7fb25ffff700 1 -- 172.21.15.44:0/3992982410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635ce8cc0 msgr2=0x55f635cf4f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:31:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.053+0000 7fb25ffff700 1 --2- 172.21.15.44:0/3992982410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635ce8cc0 0x55f635cf4f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.053+0000 7fb25ffff700 1 -- 172.21.15.44:0/3992982410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f635a66d80 con 0x55f635ce8300 2023-09-10T01:31:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.054+0000 7fb25ffff700 1 --2- 172.21.15.44:0/3992982410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8300 0x55f635ce8780 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb254009810 tx=0x7fb25400bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b246c425bed9fce server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.054+0000 7fb25d7fa700 1 -- 172.21.15.44:0/1280739505 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fb25000cd40 con 0x7fb2600732c0 2023-09-10T01:31:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.054+0000 7fb25d7fa700 1 --2- 172.21.15.44:0/1280739505 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb2480a08f0 0x7fb2480a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.055+0000 7fb25d7fa700 1 -- 172.21.15.44:0/1280739505 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fb250014070 con 0x7fb2600732c0 2023-09-10T01:31:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.055+0000 7fb2467fc700 1 -- 172.21.15.44:0/3992982410 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25401c070 con 0x55f635ce8300 2023-09-10T01:31:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.055+0000 7fb2467fc700 1 -- 172.21.15.44:0/3992982410 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb25400f8f0 con 0x55f635ce8300 2023-09-10T01:31:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.055+0000 7fb2467fc700 1 -- 172.21.15.44:0/3992982410 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400bbd0 con 0x55f635ce8300 2023-09-10T01:31:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.055+0000 7fb2649a7700 1 --2- 172.21.15.44:0/1280739505 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb2480a08f0 0x7fb2480a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.056+0000 7fb2725d0b80 1 -- 172.21.15.44:0/3992982410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8300 msgr2=0x55f635ce8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.056+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/3992982410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8300 0x55f635ce8780 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb254009810 tx=0x7fb25400bf00 comp rx=0 tx=0).stop 2023-09-10T01:31:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.056+0000 7fb2649a7700 1 --2- 172.21.15.44:0/1280739505 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb2480a08f0 0x7fb2480a2db0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb2580062a0 tx=0x7fb25801c040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.056+0000 7fb2725d0b80 1 -- 172.21.15.44:0/3992982410 shutdown_connections 2023-09-10T01:31:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.056+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/3992982410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635ce8cc0 0x55f635cf4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.056+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/3992982410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635cee1a0 0x55f635cee580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.056+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/3992982410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8300 0x55f635ce8780 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.056+0000 7fb2725d0b80 1 -- 172.21.15.44:0/3992982410 >> 172.21.15.44:0/3992982410 conn(0x55f635c5d030 msgr2=0x55f635ce3560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.056+0000 7fb2725d0b80 1 -- 172.21.15.44:0/3992982410 shutdown_connections 2023-09-10T01:31:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.056+0000 7fb2725d0b80 1 -- 172.21.15.44:0/3992982410 wait complete. 2023-09-10T01:31:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.056+0000 7fb2725d0b80 1 -- start start 2023-09-10T01:31:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.057+0000 7fb2725d0b80 1 Processor -- start 2023-09-10T01:31:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.057+0000 7fb2725d0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635ce8300 0x55f635cf7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.057+0000 7fb2725d0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8cc0 0x55f635cf8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.057+0000 7fb2725d0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635cee1a0 0x55f635cf2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.057+0000 7fb2725d0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f635bd8b10 con 0x55f635ce8300 2023-09-10T01:31:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.057+0000 7fb2725d0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f635bd8810 con 0x55f635cee1a0 2023-09-10T01:31:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.057+0000 7fb2725d0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f635abc8c0 con 0x55f635ce8cc0 2023-09-10T01:31:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.057+0000 7fb2725d0b80 10 client.0 _ll_register_callbacks cb 0x55f635b22a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:31:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.057+0000 7fb25ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635ce8300 0x55f635cf7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.057+0000 7fb25ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635ce8300 0x55f635cf7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60454/0 (socket says 172.21.15.44:60454) 2023-09-10T01:31:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.057+0000 7fb25ffff700 1 -- 172.21.15.44:0/2347350882 learned_addr learned my addr 172.21.15.44:0/2347350882 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.057+0000 7fb25f7fe700 1 --2- 172.21.15.44:0/2347350882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8cc0 0x55f635cf8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.057+0000 7fb2649a7700 1 --2- 172.21.15.44:0/2347350882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635cee1a0 0x55f635cf2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.058+0000 7fb25f7fe700 1 -- 172.21.15.44:0/2347350882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635cee1a0 msgr2=0x55f635cf2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.058+0000 7fb25f7fe700 1 --2- 172.21.15.44:0/2347350882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635cee1a0 0x55f635cf2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.058+0000 7fb25f7fe700 1 -- 172.21.15.44:0/2347350882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635ce8300 msgr2=0x55f635cf7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.058+0000 7fb25f7fe700 1 --2- 172.21.15.44:0/2347350882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635ce8300 0x55f635cf7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.058+0000 7fb25f7fe700 1 -- 172.21.15.44:0/2347350882 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb260114370 con 0x55f635ce8cc0 2023-09-10T01:31:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.058+0000 7fb25f7fe700 1 --2- 172.21.15.44:0/2347350882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8cc0 0x55f635cf8300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb2500c9b10 tx=0x7fb250099ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=791d0df8608df7cf server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.058+0000 7fb23f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:31:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.058+0000 7fb23f7fe700 1 -- 172.21.15.44:0/2347350882 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250099cd0 con 0x55f635ce8cc0 2023-09-10T01:31:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.058+0000 7fb23f7fe700 1 -- 172.21.15.44:0/2347350882 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(195..195 src has 1..195) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fb2500ae020 con 0x55f635ce8cc0 2023-09-10T01:31:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.058+0000 7fb2725d0b80 1 -- 172.21.15.44:0/2347350882 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f635c4f840 con 0x55f635ce8cc0 2023-09-10T01:31:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.058+0000 7fb23f7fe700 1 -- 172.21.15.44:0/2347350882 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=176) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2500a18e0 con 0x55f635ce8cc0 2023-09-10T01:31:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.058+0000 7fb2725d0b80 10 client.30127 fetch_fsmap learned FSMap version 176 2023-09-10T01:31:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.058+0000 7fb2725d0b80 1 -- 172.21.15.44:0/2347350882 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=176,monmap=4+}) v3 -- 0x55f635a4fc00 con 0x55f635ce8cc0 2023-09-10T01:31:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.059+0000 7fb23f7fe700 1 -- 172.21.15.44:0/2347350882 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f635c4f840 con 0x55f635ce8cc0 2023-09-10T01:31:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.059+0000 7fb23f7fe700 1 -- 172.21.15.44:0/2347350882 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 176) v1 ==== 7355+0+0 (secure 0 0 0) 0x7fb2500cc410 con 0x55f635ce8cc0 2023-09-10T01:31:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.059+0000 7fb2725d0b80 10 client.30127 fetch_fsmap finished waiting for FSMap version 176 2023-09-10T01:31:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.059+0000 7fb2725d0b80 10 client.30127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:31:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.059+0000 7fb2725d0b80 20 client.30127 populate_metadata read hostname 'smithi044' 2023-09-10T01:31:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.059+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/2347350882 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f635d022e0 0x55f635d04700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.059+0000 7fb2725d0b80 4 client.30127 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:31:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.059+0000 7fb2725d0b80 1 -- 172.21.15.44:0/2347350882 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f635c338c0 con 0x55f635d022e0 2023-09-10T01:31:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.059+0000 7fb25ffff700 1 --2- 172.21.15.44:0/2347350882 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f635d022e0 0x55f635d04700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.060+0000 7fb25ffff700 1 --2- 172.21.15.44:0/2347350882 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f635d022e0 0x55f635d04700 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37712e3df161d417 server_cookie=27c2af4f9dfd0790 in_seq=0 out_seq=0 2023-09-10T01:31:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.060+0000 7fb23f7fe700 10 client.30127 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:31:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.066+0000 7fb23f7fe700 1 -- 172.21.15.44:0/2347350882 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f635c338c0 con 0x55f635d022e0 2023-09-10T01:31:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.066+0000 7fb23f7fe700 10 client.30127 handle_command_reply: tid=0 2023-09-10T01:31:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.066+0000 7fb2725d0b80 1 client.30127 shutdown 2023-09-10T01:31:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.066+0000 7fb2725d0b80 1 -- 172.21.15.44:0/2347350882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8cc0 msgr2=0x55f635cf8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.066+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/2347350882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8cc0 0x55f635cf8300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb2500c9b10 tx=0x7fb250099ff0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.066+0000 7fb2725d0b80 1 -- 172.21.15.44:0/2347350882 shutdown_connections 2023-09-10T01:31:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.066+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/2347350882 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f635d022e0 0x55f635d04700 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.066+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/2347350882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635cee1a0 0x55f635cf2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.066+0000 7fb23f7fe700 0 client.30127 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:31:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.066+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/2347350882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635ce8300 0x55f635cf7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.066+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/2347350882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8cc0 0x55f635cf8300 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.066+0000 7fb2725d0b80 1 -- 172.21.15.44:0/2347350882 >> 172.21.15.44:0/2347350882 conn(0x55f635c5d030 msgr2=0x55f635ce27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.073+0000 7fb2725d0b80 1 -- 172.21.15.44:0/2347350882 shutdown_connections 2023-09-10T01:31:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.075+0000 7fb2725d0b80 1 -- 172.21.15.44:0/2347350882 wait complete. 2023-09-10T01:31:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.077+0000 7fb2725d0b80 20 client.30127 trim_cache size 0 max 16384 2023-09-10T01:31:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.105+0000 7fb2725d0b80 1 Processor -- start 2023-09-10T01:31:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.107+0000 7fb2725d0b80 1 -- start start 2023-09-10T01:31:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.108+0000 7fb2725d0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635ce8300 0x55f635cf2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.108+0000 7fb2725d0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8cc0 0x55f635ce7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.108+0000 7fb2725d0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635cee1a0 0x55f635ceab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.108+0000 7fb2725d0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f635cf05d0 con 0x55f635cee1a0 2023-09-10T01:31:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.108+0000 7fb2725d0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f635cecca0 con 0x55f635ce8300 2023-09-10T01:31:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.108+0000 7fb2725d0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f635ceca40 con 0x55f635ce8cc0 2023-09-10T01:31:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.110+0000 7fb2649a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635cee1a0 0x55f635ceab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.110+0000 7fb2649a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635cee1a0 0x55f635ceab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60462/0 (socket says 172.21.15.44:60462) 2023-09-10T01:31:07.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.110+0000 7fb2649a7700 1 -- 172.21.15.44:0/2644818217 learned_addr learned my addr 172.21.15.44:0/2644818217 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:07.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.110+0000 7fb2649a7700 1 -- 172.21.15.44:0/2644818217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8cc0 msgr2=0x55f635ce7a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:31:07.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.110+0000 7fb2649a7700 1 --2- 172.21.15.44:0/2644818217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8cc0 0x55f635ce7a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.110+0000 7fb2649a7700 1 -- 172.21.15.44:0/2644818217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635ce8300 msgr2=0x55f635cf2180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:31:07.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.110+0000 7fb2649a7700 1 --2- 172.21.15.44:0/2644818217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635ce8300 0x55f635cf2180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.110+0000 7fb2649a7700 1 -- 172.21.15.44:0/2644818217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f635a8d1f0 con 0x55f635cee1a0 2023-09-10T01:31:07.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.110+0000 7fb2649a7700 1 --2- 172.21.15.44:0/2644818217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635cee1a0 0x55f635ceab60 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fb25801a4a0 tx=0x7fb258028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea48cf84fb5b37b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:07.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.111+0000 7fb246ffd700 1 -- 172.21.15.44:0/2644818217 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25801f040 con 0x55f635cee1a0 2023-09-10T01:31:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.111+0000 7fb246ffd700 1 -- 172.21.15.44:0/2644818217 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb258020040 con 0x55f635cee1a0 2023-09-10T01:31:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.111+0000 7fb246ffd700 1 -- 172.21.15.44:0/2644818217 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258033d80 con 0x55f635cee1a0 2023-09-10T01:31:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.112+0000 7fb2725d0b80 1 -- 172.21.15.44:0/2644818217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635cee1a0 msgr2=0x55f635ceab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.112+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/2644818217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635cee1a0 0x55f635ceab60 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fb25801a4a0 tx=0x7fb258028000 comp rx=0 tx=0).stop 2023-09-10T01:31:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.124+0000 7fb2725d0b80 1 -- 172.21.15.44:0/2644818217 shutdown_connections 2023-09-10T01:31:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.124+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/2644818217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635cee1a0 0x55f635ceab60 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.124+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/2644818217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635ce8300 0x55f635cf2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.124+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/2644818217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8cc0 0x55f635ce7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.124+0000 7fb2725d0b80 1 -- 172.21.15.44:0/2644818217 >> 172.21.15.44:0/2644818217 conn(0x55f635c5d030 msgr2=0x55f635ce72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.130+0000 7fb2725d0b80 1 -- 172.21.15.44:0/2644818217 shutdown_connections 2023-09-10T01:31:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.131+0000 7fb2725d0b80 1 -- 172.21.15.44:0/2644818217 wait complete. 2023-09-10T01:31:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.132+0000 7fb2725d0b80 1 -- start start 2023-09-10T01:31:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.132+0000 7fb2725d0b80 1 Processor -- start 2023-09-10T01:31:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.132+0000 7fb2725d0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635ce8300 0x55f635cf6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.132+0000 7fb2725d0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8cc0 0x55f635cf7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.132+0000 7fb2725d0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635cee1a0 0x55f635d560e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.132+0000 7fb2725d0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f635d56620 con 0x55f635cee1a0 2023-09-10T01:31:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.132+0000 7fb2725d0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f635d56760 con 0x55f635ce8300 2023-09-10T01:31:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.132+0000 7fb2725d0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f635d568a0 con 0x55f635ce8cc0 2023-09-10T01:31:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.132+0000 7fb2725d0b80 10 client.0 _ll_register_callbacks cb 0x7fb2500994a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:31:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.133+0000 7fb25ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635ce8300 0x55f635cf6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.133+0000 7fb25ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635ce8300 0x55f635cf6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33974/0 (socket says 172.21.15.44:33974) 2023-09-10T01:31:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.133+0000 7fb25ffff700 1 -- 172.21.15.44:0/3308637070 learned_addr learned my addr 172.21.15.44:0/3308637070 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.133+0000 7fb25f7fe700 1 --2- 172.21.15.44:0/3308637070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8cc0 0x55f635cf7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.133+0000 7fb25ffff700 1 -- 172.21.15.44:0/3308637070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8cc0 msgr2=0x55f635cf7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.133+0000 7fb25ffff700 1 --2- 172.21.15.44:0/3308637070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8cc0 0x55f635cf7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.133+0000 7fb25ffff700 1 -- 172.21.15.44:0/3308637070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635cee1a0 msgr2=0x55f635d560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.133+0000 7fb25ffff700 1 --2- 172.21.15.44:0/3308637070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635cee1a0 0x55f635d560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.133+0000 7fb25ffff700 1 -- 172.21.15.44:0/3308637070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25801a180 con 0x55f635ce8300 2023-09-10T01:31:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.134+0000 7fb25ffff700 1 --2- 172.21.15.44:0/3308637070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635ce8300 0x55f635cf6dc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb254009810 tx=0x7fb254005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f968ca10b8f5bd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.135+0000 7fb23f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:31:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.135+0000 7fb23f7fe700 1 -- 172.21.15.44:0/3308637070 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb254031030 con 0x55f635ce8300 2023-09-10T01:31:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.135+0000 7fb23f7fe700 1 -- 172.21.15.44:0/3308637070 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(195..195 src has 1..195) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fb254032070 con 0x55f635ce8300 2023-09-10T01:31:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.135+0000 7fb2725d0b80 1 -- 172.21.15.44:0/3308637070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f635b92490 con 0x55f635ce8300 2023-09-10T01:31:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.135+0000 7fb23f7fe700 1 -- 172.21.15.44:0/3308637070 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=176) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb25401c040 con 0x55f635ce8300 2023-09-10T01:31:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.138+0000 7fb2725d0b80 10 client.30116 fetch_fsmap learned FSMap version 176 2023-09-10T01:31:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.138+0000 7fb2725d0b80 1 -- 172.21.15.44:0/3308637070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=176,monmap=4+}) v3 -- 0x55f635a65250 con 0x55f635ce8300 2023-09-10T01:31:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.138+0000 7fb23f7fe700 1 -- 172.21.15.44:0/3308637070 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f635b92490 con 0x55f635ce8300 2023-09-10T01:31:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.138+0000 7fb23f7fe700 1 -- 172.21.15.44:0/3308637070 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 176) v1 ==== 7355+0+0 (secure 0 0 0) 0x7fb254029070 con 0x55f635ce8300 2023-09-10T01:31:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.139+0000 7fb2725d0b80 10 client.30116 fetch_fsmap finished waiting for FSMap version 176 2023-09-10T01:31:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.139+0000 7fb2725d0b80 10 client.30116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:31:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.139+0000 7fb2725d0b80 20 client.30116 populate_metadata read hostname 'smithi044' 2023-09-10T01:31:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.139+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/3308637070 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f635d57c00 0x55f635d5a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.139+0000 7fb2725d0b80 4 client.30116 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:31:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.139+0000 7fb2725d0b80 1 -- 172.21.15.44:0/3308637070 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f635bf4050 con 0x55f635d57c00 2023-09-10T01:31:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.139+0000 7fb25f7fe700 1 --2- 172.21.15.44:0/3308637070 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f635d57c00 0x55f635d5a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.140+0000 7fb23f7fe700 10 client.30116 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:31:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.140+0000 7fb25f7fe700 1 --2- 172.21.15.44:0/3308637070 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f635d57c00 0x55f635d5a020 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf018180fbb74367 server_cookie=a26742bbc901f079 in_seq=0 out_seq=0 2023-09-10T01:31:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.181+0000 7fb23f7fe700 1 -- 172.21.15.44:0/3308637070 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f635bf4050 con 0x55f635d57c00 2023-09-10T01:31:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.181+0000 7fb23f7fe700 10 client.30116 handle_command_reply: tid=0 2023-09-10T01:31:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.181+0000 7fb2725d0b80 1 client.30116 shutdown 2023-09-10T01:31:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.183+0000 7fb2725d0b80 1 -- 172.21.15.44:0/3308637070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635ce8300 msgr2=0x55f635cf6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.183+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/3308637070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635ce8300 0x55f635cf6dc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb254009810 tx=0x7fb254005b90 comp rx=0 tx=0).stop 2023-09-10T01:31:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.183+0000 7fb2725d0b80 1 -- 172.21.15.44:0/3308637070 shutdown_connections 2023-09-10T01:31:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.183+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/3308637070 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f635d57c00 0x55f635d5a020 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.183+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/3308637070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f635cee1a0 0x55f635d560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.183+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/3308637070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f635ce8300 0x55f635cf6dc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.183+0000 7fb2725d0b80 1 --2- 172.21.15.44:0/3308637070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f635ce8cc0 0x55f635cf7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.183+0000 7fb2725d0b80 1 -- 172.21.15.44:0/3308637070 >> 172.21.15.44:0/3308637070 conn(0x55f635c5d030 msgr2=0x55f635ce7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.183+0000 7fb23f7fe700 0 client.30116 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:31:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.183+0000 7fb2725d0b80 1 -- 172.21.15.44:0/3308637070 shutdown_connections 2023-09-10T01:31:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.183+0000 7fb2725d0b80 1 -- 172.21.15.44:0/3308637070 wait complete. 2023-09-10T01:31:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.183+0000 7fb2725d0b80 20 client.30116 trim_cache size 0 max 16384 2023-09-10T01:31:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:31:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:31:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:31:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:31:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.184+0000 7fb26640a700 1 -- 172.21.15.44:0/1280739505 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb2480a08f0 msgr2=0x7fb2480a2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.184+0000 7fb26640a700 1 --2- 172.21.15.44:0/1280739505 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb2480a08f0 0x7fb2480a2db0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb2580062a0 tx=0x7fb25801c040 comp rx=0 tx=0).stop 2023-09-10T01:31:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.184+0000 7fb26640a700 1 -- 172.21.15.44:0/1280739505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2600732c0 msgr2=0x7fb26010cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.184+0000 7fb26640a700 1 --2- 172.21.15.44:0/1280739505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2600732c0 0x7fb26010cd90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb25000e9f0 tx=0x7fb25000ed00 comp rx=0 tx=0).stop 2023-09-10T01:31:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.184+0000 7fb26640a700 1 -- 172.21.15.44:0/1280739505 shutdown_connections 2023-09-10T01:31:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.184+0000 7fb26640a700 1 --2- 172.21.15.44:0/1280739505 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb2480a08f0 0x7fb2480a2db0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.184+0000 7fb26640a700 1 --2- 172.21.15.44:0/1280739505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2601115d0 0x7fb26019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.184+0000 7fb26640a700 1 --2- 172.21.15.44:0/1280739505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2600729a0 0x7fb26010c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.184+0000 7fb26640a700 1 --2- 172.21.15.44:0/1280739505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2600732c0 0x7fb26010cd90 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.184+0000 7fb26640a700 1 -- 172.21.15.44:0/1280739505 >> 172.21.15.44:0/1280739505 conn(0x7fb2600fc0d0 msgr2=0x7fb2600fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.188+0000 7fb26640a700 1 -- 172.21.15.44:0/1280739505 shutdown_connections 2023-09-10T01:31:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:07.189+0000 7fb26640a700 1 -- 172.21.15.44:0/1280739505 wait complete. 2023-09-10T01:31:07.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:07 smithi099 ceph-mon[102692]: pgmap v1640: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 686 op/s 2023-09-10T01:31:08.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:07 smithi131 ceph-mon[103713]: pgmap v1640: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 686 op/s 2023-09-10T01:31:08.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:07 smithi044 ceph-mon[98676]: pgmap v1640: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 686 op/s 2023-09-10T01:31:10.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:09 smithi131 ceph-mon[103713]: pgmap v1641: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 418 op/s 2023-09-10T01:31:10.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:09 smithi131 ceph-mon[103713]: osdmap e196: 12 total, 12 up, 12 in 2023-09-10T01:31:10.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:09 smithi044 ceph-mon[98676]: pgmap v1641: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 418 op/s 2023-09-10T01:31:10.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:09 smithi044 ceph-mon[98676]: osdmap e196: 12 total, 12 up, 12 in 2023-09-10T01:31:10.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:09 smithi099 ceph-mon[102692]: pgmap v1641: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 418 op/s 2023-09-10T01:31:10.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:09 smithi099 ceph-mon[102692]: osdmap e196: 12 total, 12 up, 12 in 2023-09-10T01:31:11.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 605.0 s, 193.0 tps, lat 650.576 ms stddev 779.986 2023-09-10T01:31:12.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:11 smithi131 ceph-mon[103713]: pgmap v1643: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 407 op/s 2023-09-10T01:31:12.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:11 smithi044 ceph-mon[98676]: pgmap v1643: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 407 op/s 2023-09-10T01:31:12.115 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:11 smithi099 ceph-mon[102692]: pgmap v1643: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 407 op/s 2023-09-10T01:31:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:13 smithi131 ceph-mon[103713]: pgmap v1644: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 332 op/s 2023-09-10T01:31:14.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:13 smithi044 ceph-mon[98676]: pgmap v1644: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 332 op/s 2023-09-10T01:31:14.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:13 smithi099 ceph-mon[102692]: pgmap v1644: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 332 op/s 2023-09-10T01:31:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:15 smithi131 ceph-mon[103713]: pgmap v1645: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 350 op/s 2023-09-10T01:31:16.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:15 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:16.067 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:15 smithi044 ceph-mon[98676]: pgmap v1645: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 350 op/s 2023-09-10T01:31:16.068 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:15 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:16.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:15 smithi099 ceph-mon[102692]: pgmap v1645: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 350 op/s 2023-09-10T01:31:16.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:15 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:16.182 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 610.0 s, 114.8 tps, lat 971.709 ms stddev 586.859 2023-09-10T01:31:18.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:17 smithi131 ceph-mon[103713]: pgmap v1646: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 340 op/s 2023-09-10T01:31:18.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:17 smithi044 ceph-mon[98676]: pgmap v1646: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 340 op/s 2023-09-10T01:31:18.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:17 smithi099 ceph-mon[102692]: pgmap v1646: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 340 op/s 2023-09-10T01:31:20.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:19 smithi131 ceph-mon[103713]: pgmap v1647: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2023-09-10T01:31:20.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:19 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:20.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:19 smithi044 ceph-mon[98676]: pgmap v1647: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2023-09-10T01:31:20.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:19 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:20.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:19 smithi099 ceph-mon[102692]: pgmap v1647: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2023-09-10T01:31:20.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:19 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.403+0000 7f803a16e700 1 -- 172.21.15.44:0/2329376482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8034106f90 msgr2=0x7f8034107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.403+0000 7f803a16e700 1 --2- 172.21.15.44:0/2329376482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8034106f90 0x7f8034107370 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8024009990 tx=0x7f8024009ca0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.403+0000 7f803a16e700 1 -- 172.21.15.44:0/2329376482 shutdown_connections 2023-09-10T01:31:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.403+0000 7f803a16e700 1 --2- 172.21.15.44:0/2329376482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803410bf60 0x7f8034113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.403+0000 7f803a16e700 1 --2- 172.21.15.44:0/2329376482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8034106f90 0x7f8034107370 secure :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8024009990 tx=0x7f8024009ca0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.403+0000 7f803a16e700 1 --2- 172.21.15.44:0/2329376482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8034107940 0x7f803410b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.403+0000 7f803a16e700 1 -- 172.21.15.44:0/2329376482 >> 172.21.15.44:0/2329376482 conn(0x7f80340764a0 msgr2=0x7f8034105420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.404+0000 7f803a16e700 1 -- 172.21.15.44:0/2329376482 shutdown_connections 2023-09-10T01:31:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.404+0000 7f803a16e700 1 -- 172.21.15.44:0/2329376482 wait complete. 2023-09-10T01:31:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.405+0000 7f803a16e700 1 Processor -- start 2023-09-10T01:31:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.405+0000 7f803a16e700 1 -- start start 2023-09-10T01:31:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.405+0000 7f803a16e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f803410bf60 0x7f803419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.405+0000 7f803a16e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f803419d480 0x7f80341a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.405+0000 7f803a16e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80341a2030 0x7f80341a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.405+0000 7f803a16e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803419db70 con 0x7f803410bf60 2023-09-10T01:31:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.405+0000 7f803a16e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803406a020 con 0x7f80341a2030 2023-09-10T01:31:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.405+0000 7f803a16e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803406a1e0 con 0x7f803419d480 2023-09-10T01:31:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.405+0000 7f80337fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f803410bf60 0x7f803419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.405+0000 7f80337fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f803410bf60 0x7f803419cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60480/0 (socket says 172.21.15.44:60480) 2023-09-10T01:31:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.405+0000 7f80337fe700 1 -- 172.21.15.44:0/3471769194 learned_addr learned my addr 172.21.15.44:0/3471769194 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.405+0000 7f8032ffd700 1 --2- 172.21.15.44:0/3471769194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f803419d480 0x7f80341a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.406+0000 7f8033fff700 1 --2- 172.21.15.44:0/3471769194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80341a2030 0x7f80341a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.406+0000 7f80337fe700 1 -- 172.21.15.44:0/3471769194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f803419d480 msgr2=0x7f80341a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.406+0000 7f80337fe700 1 --2- 172.21.15.44:0/3471769194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f803419d480 0x7f80341a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.406+0000 7f80337fe700 1 -- 172.21.15.44:0/3471769194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80341a2030 msgr2=0x7f80341a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.406+0000 7f80337fe700 1 --2- 172.21.15.44:0/3471769194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80341a2030 0x7f80341a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.406+0000 7f80337fe700 1 -- 172.21.15.44:0/3471769194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80240095f0 con 0x7f803410bf60 2023-09-10T01:31:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.406+0000 7f8032ffd700 1 --2- 172.21.15.44:0/3471769194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f803419d480 0x7f80341a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:31:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.406+0000 7f8033fff700 1 --2- 172.21.15.44:0/3471769194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80341a2030 0x7f80341a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:31:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.406+0000 7f80337fe700 1 --2- 172.21.15.44:0/3471769194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f803410bf60 0x7f803419cf40 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f80240098e0 tx=0x7f8024005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.407+0000 7f8030ff9700 1 -- 172.21.15.44:0/3471769194 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802401d070 con 0x7f803410bf60 2023-09-10T01:31:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.407+0000 7f803a16e700 1 -- 172.21.15.44:0/3471769194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803406a460 con 0x7f803410bf60 2023-09-10T01:31:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.407+0000 7f803a16e700 1 -- 172.21.15.44:0/3471769194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803406a860 con 0x7f803410bf60 2023-09-10T01:31:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.408+0000 7f8030ff9700 1 -- 172.21.15.44:0/3471769194 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8024004c70 con 0x7f803410bf60 2023-09-10T01:31:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.408+0000 7f8030ff9700 1 -- 172.21.15.44:0/3471769194 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024003a30 con 0x7f803410bf60 2023-09-10T01:31:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.411+0000 7f8030ff9700 1 -- 172.21.15.44:0/3471769194 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f80240047e0 con 0x7f803410bf60 2023-09-10T01:31:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.411+0000 7f8030ff9700 1 --2- 172.21.15.44:0/3471769194 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f801c0a0870 0x7f801c0a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.412+0000 7f8030ff9700 1 -- 172.21.15.44:0/3471769194 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f80240cceb0 con 0x7f803410bf60 2023-09-10T01:31:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.412+0000 7f8032ffd700 1 --2- 172.21.15.44:0/3471769194 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f801c0a0870 0x7f801c0a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.412+0000 7f8046334b80 1 Processor -- start 2023-09-10T01:31:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.412+0000 7f8046334b80 1 -- start start 2023-09-10T01:31:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.412+0000 7f8032ffd700 1 --2- 172.21.15.44:0/3471769194 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f801c0a0870 0x7f801c0a2d30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8034069cb0 tx=0x7f8028009190 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.412+0000 7f8046334b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9ea590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.412+0000 7f8046334b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4310 0x55bdbd9e4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.412+0000 7f8046334b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9e4cd0 0x55bdbd9f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.412+0000 7f8046334b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbd92c6d0 con 0x55bdbd9e4cd0 2023-09-10T01:31:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.412+0000 7f8046334b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbd929ff0 con 0x55bdbd9ea1b0 2023-09-10T01:31:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.412+0000 7f8046334b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbd9258e0 con 0x55bdbd9e4310 2023-09-10T01:31:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.413+0000 7f80337fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9ea590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.413+0000 7f80337fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9ea590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33996/0 (socket says 172.21.15.44:33996) 2023-09-10T01:31:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.413+0000 7f80337fe700 1 -- 172.21.15.44:0/3640123687 learned_addr learned my addr 172.21.15.44:0/3640123687 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.413+0000 7f8032ffd700 1 --2- 172.21.15.44:0/3640123687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4310 0x55bdbd9e4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.413+0000 7f8033fff700 1 --2- 172.21.15.44:0/3640123687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9e4cd0 0x55bdbd9f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.413+0000 7f80337fe700 1 -- 172.21.15.44:0/3640123687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4310 msgr2=0x55bdbd9e4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.413+0000 7f80337fe700 1 --2- 172.21.15.44:0/3640123687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4310 0x55bdbd9e4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.413+0000 7f80337fe700 1 -- 172.21.15.44:0/3640123687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9e4cd0 msgr2=0x55bdbd9f0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.413+0000 7f80337fe700 1 --2- 172.21.15.44:0/3640123687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9e4cd0 0x55bdbd9f0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.413+0000 7f80337fe700 1 -- 172.21.15.44:0/3640123687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdbd751d80 con 0x55bdbd9ea1b0 2023-09-10T01:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.413+0000 7f80337fe700 1 --2- 172.21.15.44:0/3640123687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9ea590 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8024096a10 tx=0x7f8024003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ae77b027add9c89 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.413+0000 7f8019ffb700 1 -- 172.21.15.44:0/3640123687 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802409a6a0 con 0x55bdbd9ea1b0 2023-09-10T01:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.414+0000 7f8019ffb700 1 -- 172.21.15.44:0/3640123687 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8024004de0 con 0x55bdbd9ea1b0 2023-09-10T01:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.414+0000 7f8019ffb700 1 -- 172.21.15.44:0/3640123687 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024020610 con 0x55bdbd9ea1b0 2023-09-10T01:31:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.414+0000 7f8046334b80 1 -- 172.21.15.44:0/3640123687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9ea1b0 msgr2=0x55bdbd9ea590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.414+0000 7f8046334b80 1 --2- 172.21.15.44:0/3640123687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9ea590 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8024096a10 tx=0x7f8024003680 comp rx=0 tx=0).stop 2023-09-10T01:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.415+0000 7f8046334b80 1 -- 172.21.15.44:0/3640123687 shutdown_connections 2023-09-10T01:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.415+0000 7f8046334b80 1 --2- 172.21.15.44:0/3640123687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9e4cd0 0x55bdbd9f0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.415+0000 7f8046334b80 1 --2- 172.21.15.44:0/3640123687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9ea590 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.415+0000 7f8046334b80 1 --2- 172.21.15.44:0/3640123687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4310 0x55bdbd9e4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.415+0000 7f8046334b80 1 -- 172.21.15.44:0/3640123687 >> 172.21.15.44:0/3640123687 conn(0x55bdbd948030 msgr2=0x55bdbd9ce560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.415+0000 7f8046334b80 1 -- 172.21.15.44:0/3640123687 shutdown_connections 2023-09-10T01:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.415+0000 7f8046334b80 1 -- 172.21.15.44:0/3640123687 wait complete. 2023-09-10T01:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.415+0000 7f8046334b80 1 -- start start 2023-09-10T01:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.415+0000 7f8046334b80 1 Processor -- start 2023-09-10T01:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.415+0000 7f8046334b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9e4310 0x55bdbd9f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.415+0000 7f8046334b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 0x55bdbd9f4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.415+0000 7f8046334b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9edd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.415+0000 7f8046334b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbd8c3b10 con 0x55bdbd9ea1b0 2023-09-10T01:31:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.415+0000 7f8046334b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbd8c3810 con 0x55bdbd9e4310 2023-09-10T01:31:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.415+0000 7f8046334b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbd7a78c0 con 0x55bdbd9e4cd0 2023-09-10T01:31:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f8046334b80 10 client.0 _ll_register_callbacks cb 0x55bdbd80da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:31:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f8032ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 0x55bdbd9f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f8032ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 0x55bdbd9f4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39146/0 (socket says 172.21.15.44:39146) 2023-09-10T01:31:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f80337fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9e4310 0x55bdbd9f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f8032ffd700 1 -- 172.21.15.44:0/72474868 learned_addr learned my addr 172.21.15.44:0/72474868 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f8033fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f8032ffd700 1 -- 172.21.15.44:0/72474868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9e4310 msgr2=0x55bdbd9f3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f8032ffd700 1 --2- 172.21.15.44:0/72474868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9e4310 0x55bdbd9f3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f8032ffd700 1 -- 172.21.15.44:0/72474868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9ea1b0 msgr2=0x55bdbd9edd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f8032ffd700 1 --2- 172.21.15.44:0/72474868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9edd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f8032ffd700 1 -- 172.21.15.44:0/72474868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80240b9040 con 0x55bdbd9e4cd0 2023-09-10T01:31:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f80337fe700 1 --2- 172.21.15.44:0/72474868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9e4310 0x55bdbd9f3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:31:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f8033fff700 1 --2- 172.21.15.44:0/72474868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9edd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:31:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f8032ffd700 1 --2- 172.21.15.44:0/72474868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 0x55bdbd9f4040 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8028015800 tx=0x7f8028004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91b7993e127f66ef server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f8012ffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:31:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f8012ffd700 1 -- 172.21.15.44:0/72474868 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028013450 con 0x55bdbd9e4cd0 2023-09-10T01:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f8012ffd700 1 -- 172.21.15.44:0/72474868 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f80280135b0 con 0x55bdbd9e4cd0 2023-09-10T01:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.416+0000 7f8046334b80 1 -- 172.21.15.44:0/72474868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdbd87d490 con 0x55bdbd9e4cd0 2023-09-10T01:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.417+0000 7f8012ffd700 1 -- 172.21.15.44:0/72474868 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=178) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8028005a50 con 0x55bdbd9e4cd0 2023-09-10T01:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.417+0000 7f8046334b80 10 client.30142 fetch_fsmap learned FSMap version 178 2023-09-10T01:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.417+0000 7f8046334b80 1 -- 172.21.15.44:0/72474868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=178,monmap=4+}) v3 -- 0x55bdbd73ac00 con 0x55bdbd9e4cd0 2023-09-10T01:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.417+0000 7f8012ffd700 1 -- 172.21.15.44:0/72474868 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bdbd87d490 con 0x55bdbd9e4cd0 2023-09-10T01:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.417+0000 7f8012ffd700 1 -- 172.21.15.44:0/72474868 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 178) v1 ==== 7339+0+0 (secure 0 0 0) 0x7f8028006020 con 0x55bdbd9e4cd0 2023-09-10T01:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.417+0000 7f8046334b80 10 client.30142 fetch_fsmap finished waiting for FSMap version 178 2023-09-10T01:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.417+0000 7f8046334b80 10 client.30142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.417+0000 7f8046334b80 20 client.30142 populate_metadata read hostname 'smithi044' 2023-09-10T01:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.417+0000 7f8046334b80 1 --2- 172.21.15.44:0/72474868 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bdbd9fe160 0x55bdbda00580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.417+0000 7f8046334b80 4 client.30142 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.417+0000 7f8046334b80 1 -- 172.21.15.44:0/72474868 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdbd93a840 con 0x55bdbd9fe160 2023-09-10T01:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.417+0000 7f80337fe700 1 --2- 172.21.15.44:0/72474868 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bdbd9fe160 0x55bdbda00580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.418+0000 7f8012ffd700 10 client.30142 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.418+0000 7f80337fe700 1 --2- 172.21.15.44:0/72474868 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bdbd9fe160 0x55bdbda00580 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ca195bc16552d6c server_cookie=8bfb3b4f05faecf7 in_seq=0 out_seq=0 2023-09-10T01:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.418+0000 7f8012ffd700 1 -- 172.21.15.44:0/72474868 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bdbd93a840 con 0x55bdbd9fe160 2023-09-10T01:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.418+0000 7f8012ffd700 10 client.30142 handle_command_reply: tid=0 2023-09-10T01:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.418+0000 7f8046334b80 1 client.30142 shutdown 2023-09-10T01:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.419+0000 7f8046334b80 1 -- 172.21.15.44:0/72474868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 msgr2=0x55bdbd9f4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.419+0000 7f8046334b80 1 --2- 172.21.15.44:0/72474868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 0x55bdbd9f4040 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8028015800 tx=0x7f8028004860 comp rx=0 tx=0).stop 2023-09-10T01:31:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.419+0000 7f8046334b80 1 -- 172.21.15.44:0/72474868 shutdown_connections 2023-09-10T01:31:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.419+0000 7f8046334b80 1 --2- 172.21.15.44:0/72474868 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bdbd9fe160 0x55bdbda00580 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.419+0000 7f8046334b80 1 --2- 172.21.15.44:0/72474868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9edd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.419+0000 7f8012ffd700 0 client.30142 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:31:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.419+0000 7f8046334b80 1 --2- 172.21.15.44:0/72474868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9e4310 0x55bdbd9f3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.419+0000 7f8046334b80 1 --2- 172.21.15.44:0/72474868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 0x55bdbd9f4040 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.419+0000 7f8046334b80 1 -- 172.21.15.44:0/72474868 >> 172.21.15.44:0/72474868 conn(0x55bdbd948030 msgr2=0x55bdbd9cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.419+0000 7f8046334b80 1 -- 172.21.15.44:0/72474868 shutdown_connections 2023-09-10T01:31:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.419+0000 7f8046334b80 1 -- 172.21.15.44:0/72474868 wait complete. 2023-09-10T01:31:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.419+0000 7f8046334b80 20 client.30142 trim_cache size 0 max 16384 2023-09-10T01:31:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.434+0000 7f8046334b80 1 Processor -- start 2023-09-10T01:31:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.434+0000 7f8046334b80 1 -- start start 2023-09-10T01:31:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.434+0000 7f8046334b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9e4310 0x55bdbd9edcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.434+0000 7f8046334b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 0x55bdbd9cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.434+0000 7f8046334b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.434+0000 7f8046334b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbd9ef180 con 0x55bdbd9e4310 2023-09-10T01:31:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.434+0000 7f8046334b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbd9cf930 con 0x55bdbd9ea1b0 2023-09-10T01:31:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.434+0000 7f8046334b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbd9cfca0 con 0x55bdbd9e4cd0 2023-09-10T01:31:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.434+0000 7f80337fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9e4310 0x55bdbd9edcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.434+0000 7f8032ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 0x55bdbd9cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.434+0000 7f80337fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9e4310 0x55bdbd9edcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60502/0 (socket says 172.21.15.44:60502) 2023-09-10T01:31:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.434+0000 7f8032ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 0x55bdbd9cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39154/0 (socket says 172.21.15.44:39154) 2023-09-10T01:31:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.434+0000 7f8033fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.434+0000 7f80337fe700 1 -- 172.21.15.44:0/2642644377 learned_addr learned my addr 172.21.15.44:0/2642644377 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.435+0000 7f80337fe700 1 -- 172.21.15.44:0/2642644377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 msgr2=0x55bdbd9cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.435+0000 7f80337fe700 1 --2- 172.21.15.44:0/2642644377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 0x55bdbd9cdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.435+0000 7f80337fe700 1 -- 172.21.15.44:0/2642644377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9ea1b0 msgr2=0x55bdbd9ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.435+0000 7f80337fe700 1 --2- 172.21.15.44:0/2642644377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.435+0000 7f80337fe700 1 -- 172.21.15.44:0/2642644377 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdbd7781f0 con 0x55bdbd9e4310 2023-09-10T01:31:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.435+0000 7f8033fff700 1 --2- 172.21.15.44:0/2642644377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:31:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.435+0000 7f8032ffd700 1 --2- 172.21.15.44:0/2642644377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 0x55bdbd9cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:31:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.435+0000 7f80337fe700 1 --2- 172.21.15.44:0/2642644377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9e4310 0x55bdbd9edcb0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f8024000c00 tx=0x7f8024009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff3aa400c90b82c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.435+0000 7f801a7fc700 1 -- 172.21.15.44:0/2642644377 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80240a1f80 con 0x55bdbd9e4310 2023-09-10T01:31:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.435+0000 7f801a7fc700 1 -- 172.21.15.44:0/2642644377 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f802409ddd0 con 0x55bdbd9e4310 2023-09-10T01:31:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.435+0000 7f801a7fc700 1 -- 172.21.15.44:0/2642644377 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024098460 con 0x55bdbd9e4310 2023-09-10T01:31:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.436+0000 7f8046334b80 1 -- 172.21.15.44:0/2642644377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9e4310 msgr2=0x55bdbd9edcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.436+0000 7f8046334b80 1 --2- 172.21.15.44:0/2642644377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9e4310 0x55bdbd9edcb0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f8024000c00 tx=0x7f8024009ca0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.436+0000 7f8046334b80 1 -- 172.21.15.44:0/2642644377 shutdown_connections 2023-09-10T01:31:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.436+0000 7f8046334b80 1 --2- 172.21.15.44:0/2642644377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.436+0000 7f8046334b80 1 --2- 172.21.15.44:0/2642644377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9e4310 0x55bdbd9edcb0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.436+0000 7f8046334b80 1 --2- 172.21.15.44:0/2642644377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 0x55bdbd9cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.436+0000 7f8046334b80 1 -- 172.21.15.44:0/2642644377 >> 172.21.15.44:0/2642644377 conn(0x55bdbd948030 msgr2=0x55bdbd9e9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.436+0000 7f8046334b80 1 -- 172.21.15.44:0/2642644377 shutdown_connections 2023-09-10T01:31:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.436+0000 7f8046334b80 1 -- 172.21.15.44:0/2642644377 wait complete. 2023-09-10T01:31:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.436+0000 7f8046334b80 1 -- start start 2023-09-10T01:31:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.436+0000 7f8046334b80 1 Processor -- start 2023-09-10T01:31:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.436+0000 7f8046334b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9e4310 0x55bdbda4aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.436+0000 7f8046334b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 0x55bdbda4afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8046334b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9f3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8046334b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbd9f3a70 con 0x55bdbd9ea1b0 2023-09-10T01:31:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8046334b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbd9f3be0 con 0x55bdbd9e4310 2023-09-10T01:31:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8046334b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbd9f3da0 con 0x55bdbd9e4cd0 2023-09-10T01:31:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8046334b80 10 client.0 _ll_register_callbacks cb 0x55bdbd80da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:31:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8033fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9f3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8033fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9f3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60512/0 (socket says 172.21.15.44:60512) 2023-09-10T01:31:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f80337fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9e4310 0x55bdbda4aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8033fff700 1 -- 172.21.15.44:0/4096446543 learned_addr learned my addr 172.21.15.44:0/4096446543 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8032ffd700 1 --2- 172.21.15.44:0/4096446543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 0x55bdbda4afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8033fff700 1 -- 172.21.15.44:0/4096446543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 msgr2=0x55bdbda4afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8033fff700 1 --2- 172.21.15.44:0/4096446543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 0x55bdbda4afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8033fff700 1 -- 172.21.15.44:0/4096446543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9e4310 msgr2=0x55bdbda4aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8033fff700 1 --2- 172.21.15.44:0/4096446543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9e4310 0x55bdbda4aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8033fff700 1 -- 172.21.15.44:0/4096446543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdbd73ac00 con 0x55bdbd9ea1b0 2023-09-10T01:31:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f80337fe700 1 --2- 172.21.15.44:0/4096446543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9e4310 0x55bdbda4aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:31:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8033fff700 1 --2- 172.21.15.44:0/4096446543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9f3490 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f802c017660 tx=0x7f802c017690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6cb3a0d55b1744 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8012ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:31:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8012ffd700 1 -- 172.21.15.44:0/4096446543 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c018420 con 0x55bdbd9ea1b0 2023-09-10T01:31:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8012ffd700 1 -- 172.21.15.44:0/4096446543 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f802c018580 con 0x55bdbd9ea1b0 2023-09-10T01:31:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.437+0000 7f8046334b80 1 -- 172.21.15.44:0/4096446543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8024004de0 con 0x55bdbd9ea1b0 2023-09-10T01:31:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.438+0000 7f8012ffd700 1 -- 172.21.15.44:0/4096446543 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=178) v2 ==== 24+0+0 (secure 0 0 0) 0x7f802c018740 con 0x55bdbd9ea1b0 2023-09-10T01:31:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.438+0000 7f8046334b80 10 client.20487 fetch_fsmap learned FSMap version 178 2023-09-10T01:31:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.438+0000 7f8046334b80 1 -- 172.21.15.44:0/4096446543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=178,monmap=4+}) v3 -- 0x7f802c007500 con 0x55bdbd9ea1b0 2023-09-10T01:31:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.438+0000 7f8012ffd700 1 -- 172.21.15.44:0/4096446543 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8024004de0 con 0x55bdbd9ea1b0 2023-09-10T01:31:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.438+0000 7f8012ffd700 1 -- 172.21.15.44:0/4096446543 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 178) v1 ==== 7339+0+0 (secure 0 0 0) 0x7f802c020740 con 0x55bdbd9ea1b0 2023-09-10T01:31:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.438+0000 7f8046334b80 10 client.20487 fetch_fsmap finished waiting for FSMap version 178 2023-09-10T01:31:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.438+0000 7f8046334b80 10 client.20487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:31:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.438+0000 7f8046334b80 20 client.20487 populate_metadata read hostname 'smithi044' 2023-09-10T01:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.438+0000 7f8046334b80 1 --2- 172.21.15.44:0/4096446543 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bdbda52a80 0x55bdbda54ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.438+0000 7f8046334b80 4 client.20487 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.438+0000 7f8046334b80 1 -- 172.21.15.44:0/4096446543 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bdbd91e8c0 con 0x55bdbda52a80 2023-09-10T01:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.438+0000 7f80337fe700 1 --2- 172.21.15.44:0/4096446543 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bdbda52a80 0x55bdbda54ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.439+0000 7f8012ffd700 10 client.20487 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.439+0000 7f80337fe700 1 --2- 172.21.15.44:0/4096446543 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bdbda52a80 0x55bdbda54ea0 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4979a6867d624858 server_cookie=358f81a16ad1d76c in_seq=0 out_seq=0 2023-09-10T01:31:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.505+0000 7f8012ffd700 1 -- 172.21.15.44:0/4096446543 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bdbd91e8c0 con 0x55bdbda52a80 2023-09-10T01:31:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.505+0000 7f8012ffd700 10 client.20487 handle_command_reply: tid=0 2023-09-10T01:31:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.508+0000 7f8046334b80 1 client.20487 shutdown 2023-09-10T01:31:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.527+0000 7f8046334b80 1 -- 172.21.15.44:0/4096446543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9ea1b0 msgr2=0x55bdbd9f3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.527+0000 7f8046334b80 1 --2- 172.21.15.44:0/4096446543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9f3490 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f802c017660 tx=0x7f802c017690 comp rx=0 tx=0).stop 2023-09-10T01:31:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.528+0000 7f8046334b80 1 -- 172.21.15.44:0/4096446543 shutdown_connections 2023-09-10T01:31:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.528+0000 7f8046334b80 1 --2- 172.21.15.44:0/4096446543 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55bdbda52a80 0x55bdbda54ea0 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.528+0000 7f8046334b80 1 --2- 172.21.15.44:0/4096446543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bdbd9ea1b0 0x55bdbd9f3490 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.528+0000 7f8046334b80 1 --2- 172.21.15.44:0/4096446543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdbd9e4310 0x55bdbda4aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.528+0000 7f8046334b80 1 --2- 172.21.15.44:0/4096446543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdbd9e4cd0 0x55bdbda4afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.528+0000 7f8046334b80 1 -- 172.21.15.44:0/4096446543 >> 172.21.15.44:0/4096446543 conn(0x55bdbd948030 msgr2=0x55bdbd9e9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.528+0000 7f8012ffd700 0 client.20487 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:31:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.528+0000 7f8046334b80 1 -- 172.21.15.44:0/4096446543 shutdown_connections 2023-09-10T01:31:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.529+0000 7f8046334b80 1 -- 172.21.15.44:0/4096446543 wait complete. 2023-09-10T01:31:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.529+0000 7f8046334b80 20 client.20487 trim_cache size 0 max 16384 2023-09-10T01:31:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:31:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:31:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:31:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:31:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.529+0000 7f803a16e700 1 -- 172.21.15.44:0/3471769194 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f801c0a0870 msgr2=0x7f801c0a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.529+0000 7f803a16e700 1 --2- 172.21.15.44:0/3471769194 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f801c0a0870 0x7f801c0a2d30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8034069cb0 tx=0x7f8028009190 comp rx=0 tx=0).stop 2023-09-10T01:31:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.529+0000 7f803a16e700 1 -- 172.21.15.44:0/3471769194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f803410bf60 msgr2=0x7f803419cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.529+0000 7f803a16e700 1 --2- 172.21.15.44:0/3471769194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f803410bf60 0x7f803419cf40 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f80240098e0 tx=0x7f8024005b00 comp rx=0 tx=0).stop 2023-09-10T01:31:20.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.535+0000 7f803a16e700 1 -- 172.21.15.44:0/3471769194 shutdown_connections 2023-09-10T01:31:20.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.535+0000 7f803a16e700 1 --2- 172.21.15.44:0/3471769194 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f801c0a0870 0x7f801c0a2d30 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.535+0000 7f803a16e700 1 --2- 172.21.15.44:0/3471769194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80341a2030 0x7f80341a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.535+0000 7f803a16e700 1 --2- 172.21.15.44:0/3471769194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f803410bf60 0x7f803419cf40 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.535+0000 7f803a16e700 1 --2- 172.21.15.44:0/3471769194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f803419d480 0x7f80341a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.535+0000 7f803a16e700 1 -- 172.21.15.44:0/3471769194 >> 172.21.15.44:0/3471769194 conn(0x7f80340764a0 msgr2=0x7f8034076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.541+0000 7f803a16e700 1 -- 172.21.15.44:0/3471769194 shutdown_connections 2023-09-10T01:31:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:20.549+0000 7f803a16e700 1 -- 172.21.15.44:0/3471769194 wait complete. 2023-09-10T01:31:21.182 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 615.0 s, 198.0 tps, lat 526.803 ms stddev 429.330 2023-09-10T01:31:22.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:21 smithi131 ceph-mon[103713]: pgmap v1648: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 304 op/s 2023-09-10T01:31:22.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:21 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:31:22.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:21 smithi131 ceph-mon[103713]: osdmap e197: 12 total, 12 up, 12 in 2023-09-10T01:31:22.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:21 smithi044 ceph-mon[98676]: pgmap v1648: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 304 op/s 2023-09-10T01:31:22.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:21 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:31:22.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:21 smithi044 ceph-mon[98676]: osdmap e197: 12 total, 12 up, 12 in 2023-09-10T01:31:22.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:21 smithi099 ceph-mon[102692]: pgmap v1648: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 304 op/s 2023-09-10T01:31:22.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:21 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:31:22.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:21 smithi099 ceph-mon[102692]: osdmap e197: 12 total, 12 up, 12 in 2023-09-10T01:31:23.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:22 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:23.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:22 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:31:23.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:22 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:31:23.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:22 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:31:23.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:22 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:23.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:22 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:31:23.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:22 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:31:23.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:22 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:31:23.312 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:22 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:23.312 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:22 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:31:23.312 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:22 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:31:23.312 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:22 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:31:24.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:23 smithi099 ceph-mon[102692]: pgmap v1650: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 455 op/s 2023-09-10T01:31:24.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:23 smithi099 ceph-mon[102692]: pgmap v1651: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 542 op/s 2023-09-10T01:31:24.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:23 smithi131 ceph-mon[103713]: pgmap v1650: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 455 op/s 2023-09-10T01:31:24.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:23 smithi131 ceph-mon[103713]: pgmap v1651: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 542 op/s 2023-09-10T01:31:24.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:23 smithi044 ceph-mon[98676]: pgmap v1650: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 455 op/s 2023-09-10T01:31:24.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:23 smithi044 ceph-mon[98676]: pgmap v1651: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 542 op/s 2023-09-10T01:31:25.299 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:24 smithi131 ceph-mon[103713]: pgmap v1652: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 441 op/s 2023-09-10T01:31:25.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:24 smithi044 ceph-mon[98676]: pgmap v1652: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 441 op/s 2023-09-10T01:31:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:24 smithi099 ceph-mon[102692]: pgmap v1652: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 441 op/s 2023-09-10T01:31:26.182 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 620.0 s, 243.4 tps, lat 402.154 ms stddev 416.027 2023-09-10T01:31:26.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:25 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:26.310 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:25 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:26.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:25 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:27.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:27 smithi131 ceph-mon[103713]: osdmap e198: 12 total, 12 up, 12 in 2023-09-10T01:31:27.795 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:27 smithi131 ceph-mon[103713]: pgmap v1654: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.3 MiB/s wr, 958 op/s 2023-09-10T01:31:27.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:27 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:27.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:27 smithi044 ceph-mon[98676]: osdmap e198: 12 total, 12 up, 12 in 2023-09-10T01:31:27.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:27 smithi044 ceph-mon[98676]: pgmap v1654: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.3 MiB/s wr, 958 op/s 2023-09-10T01:31:27.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:27 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:27.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:27 smithi099 ceph-mon[102692]: osdmap e198: 12 total, 12 up, 12 in 2023-09-10T01:31:27.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:27 smithi099 ceph-mon[102692]: pgmap v1654: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.3 MiB/s wr, 958 op/s 2023-09-10T01:31:27.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:27 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:29.805 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:29 smithi044 ceph-mon[98676]: pgmap v1655: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.8 MiB/s wr, 669 op/s 2023-09-10T01:31:29.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:29 smithi099 ceph-mon[102692]: pgmap v1655: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.8 MiB/s wr, 669 op/s 2023-09-10T01:31:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:29 smithi131 ceph-mon[103713]: pgmap v1655: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.8 MiB/s wr, 669 op/s 2023-09-10T01:31:31.184 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 625.0 s, 186.5 tps, lat 560.982 ms stddev 493.428 2023-09-10T01:31:31.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:31 smithi099 ceph-mon[102692]: pgmap v1656: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 529 op/s 2023-09-10T01:31:32.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:31 smithi131 ceph-mon[103713]: pgmap v1656: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 529 op/s 2023-09-10T01:31:32.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:31 smithi044 ceph-mon[98676]: pgmap v1656: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 529 op/s 2023-09-10T01:31:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.812+0000 7fd8359b9700 1 -- 172.21.15.44:0/594138662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83010f1c0 msgr2=0x7fd830107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.812+0000 7fd8359b9700 1 --2- 172.21.15.44:0/594138662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83010f1c0 0x7fd830107b40 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd8240098f0 tx=0x7fd824009c00 comp rx=0 tx=0).stop 2023-09-10T01:31:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.820+0000 7fd8359b9700 1 -- 172.21.15.44:0/594138662 shutdown_connections 2023-09-10T01:31:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.820+0000 7fd8359b9700 1 --2- 172.21.15.44:0/594138662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd830119de0 0x7fd83011c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.820+0000 7fd8359b9700 1 --2- 172.21.15.44:0/594138662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd83010ed50 0x7fd830107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.820+0000 7fd8359b9700 1 --2- 172.21.15.44:0/594138662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83010f1c0 0x7fd830107b40 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.820+0000 7fd8359b9700 1 -- 172.21.15.44:0/594138662 >> 172.21.15.44:0/594138662 conn(0x7fd83006c6c0 msgr2=0x7fd83006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.822+0000 7fd8359b9700 1 -- 172.21.15.44:0/594138662 shutdown_connections 2023-09-10T01:31:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.822+0000 7fd8359b9700 1 -- 172.21.15.44:0/594138662 wait complete. 2023-09-10T01:31:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.823+0000 7fd8359b9700 1 Processor -- start 2023-09-10T01:31:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.823+0000 7fd8359b9700 1 -- start start 2023-09-10T01:31:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.824+0000 7fd8359b9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd83010ed50 0x7fd830118140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.824+0000 7fd8359b9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd83010f1c0 0x7fd8301186c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.824+0000 7fd8359b9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd830119de0 0x7fd830110fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.824+0000 7fd8359b9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd830111770 con 0x7fd830119de0 2023-09-10T01:31:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.824+0000 7fd8359b9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8301118e0 con 0x7fd83010f1c0 2023-09-10T01:31:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.824+0000 7fd8359b9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd830111aa0 con 0x7fd83010ed50 2023-09-10T01:31:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.824+0000 7fd82effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd83010ed50 0x7fd830118140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.824+0000 7fd82effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd83010ed50 0x7fd830118140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39172/0 (socket says 172.21.15.44:39172) 2023-09-10T01:31:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.824+0000 7fd82effd700 1 -- 172.21.15.44:0/611890419 learned_addr learned my addr 172.21.15.44:0/611890419 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.824+0000 7fd82e7fc700 1 --2- 172.21.15.44:0/611890419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd83010f1c0 0x7fd8301186c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.824+0000 7fd82f7fe700 1 --2- 172.21.15.44:0/611890419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd830119de0 0x7fd830110fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.824+0000 7fd82effd700 1 -- 172.21.15.44:0/611890419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd83010f1c0 msgr2=0x7fd8301186c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.824+0000 7fd82effd700 1 --2- 172.21.15.44:0/611890419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd83010f1c0 0x7fd8301186c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.825+0000 7fd82effd700 1 -- 172.21.15.44:0/611890419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd830119de0 msgr2=0x7fd830110fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.825+0000 7fd82effd700 1 --2- 172.21.15.44:0/611890419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd830119de0 0x7fd830110fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.825+0000 7fd82effd700 1 -- 172.21.15.44:0/611890419 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd824009520 con 0x7fd83010ed50 2023-09-10T01:31:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.825+0000 7fd82f7fe700 1 --2- 172.21.15.44:0/611890419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd830119de0 0x7fd830110fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:31:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.825+0000 7fd82effd700 1 --2- 172.21.15.44:0/611890419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd83010ed50 0x7fd830118140 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd82000d980 tx=0x7fd82000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.825+0000 7fd8349b7700 1 -- 172.21.15.44:0/611890419 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820009930 con 0x7fd83010ed50 2023-09-10T01:31:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.825+0000 7fd8349b7700 1 -- 172.21.15.44:0/611890419 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd820010460 con 0x7fd83010ed50 2023-09-10T01:31:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.825+0000 7fd8349b7700 1 -- 172.21.15.44:0/611890419 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82000f620 con 0x7fd83010ed50 2023-09-10T01:31:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.827+0000 7fd8359b9700 1 -- 172.21.15.44:0/611890419 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd830111d80 con 0x7fd83010ed50 2023-09-10T01:31:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.827+0000 7fd8359b9700 1 -- 172.21.15.44:0/611890419 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd830115fd0 con 0x7fd83010ed50 2023-09-10T01:31:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.837+0000 7fd8349b7700 1 -- 172.21.15.44:0/611890419 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fd8200105d0 con 0x7fd83010ed50 2023-09-10T01:31:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.838+0000 7fd8349b7700 1 --2- 172.21.15.44:0/611890419 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd8180a08f0 0x7fd8180a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.838+0000 7fd8349b7700 1 -- 172.21.15.44:0/611890419 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fd8200cc2f0 con 0x7fd83010ed50 2023-09-10T01:31:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.839+0000 7fd841b7fb80 1 Processor -- start 2023-09-10T01:31:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.844+0000 7fd841b7fb80 1 -- start start 2023-09-10T01:31:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.844+0000 7fd841b7fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1dfe1a0 0x557ae1dfe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.844+0000 7fd841b7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 0x557ae1df8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.844+0000 7fd841b7fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e04f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.844+0000 7fd841b7fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae1d516d0 con 0x557ae1df8cc0 2023-09-10T01:31:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.844+0000 7fd841b7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae1d4eff0 con 0x557ae1df8300 2023-09-10T01:31:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.844+0000 7fd841b7fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae1d4a8e0 con 0x557ae1dfe1a0 2023-09-10T01:31:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.844+0000 7fd82e7fc700 1 --2- 172.21.15.44:0/611890419 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd8180a08f0 0x7fd8180a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.845+0000 7fd82e7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 0x557ae1df8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.845+0000 7fd82e7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 0x557ae1df8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34042/0 (socket says 172.21.15.44:34042) 2023-09-10T01:31:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.845+0000 7fd82e7fc700 1 -- 172.21.15.44:0/4184242152 learned_addr learned my addr 172.21.15.44:0/4184242152 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.846+0000 7fd82f7fe700 1 --2- 172.21.15.44:0/4184242152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e04f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.846+0000 7fd82e7fc700 1 --2- 172.21.15.44:0/611890419 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd8180a08f0 0x7fd8180a2db0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd824000c00 tx=0x7fd824010ec0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.846+0000 7fd82e7fc700 1 -- 172.21.15.44:0/4184242152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1dfe1a0 msgr2=0x557ae1dfe580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.846+0000 7fd82e7fc700 1 --2- 172.21.15.44:0/4184242152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1dfe1a0 0x557ae1dfe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.846+0000 7fd82e7fc700 1 -- 172.21.15.44:0/4184242152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 msgr2=0x557ae1e04f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.846+0000 7fd82e7fc700 1 --2- 172.21.15.44:0/4184242152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e04f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.846+0000 7fd82e7fc700 1 -- 172.21.15.44:0/4184242152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ae1b76d80 con 0x557ae1df8300 2023-09-10T01:31:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.846+0000 7fd82e7fc700 1 --2- 172.21.15.44:0/4184242152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 0x557ae1df8780 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd824009230 tx=0x7fd82402a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4291ea06e8f04442 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.853+0000 7fd81dffb700 1 -- 172.21.15.44:0/4184242152 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82401c030 con 0x557ae1df8300 2023-09-10T01:31:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.853+0000 7fd81dffb700 1 -- 172.21.15.44:0/4184242152 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8240054c0 con 0x557ae1df8300 2023-09-10T01:31:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.853+0000 7fd81dffb700 1 -- 172.21.15.44:0/4184242152 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824004a10 con 0x557ae1df8300 2023-09-10T01:31:33.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:33 smithi099 ceph-mon[102692]: pgmap v1657: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 672 op/s 2023-09-10T01:31:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.854+0000 7fd841b7fb80 1 -- 172.21.15.44:0/4184242152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 msgr2=0x557ae1df8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.854+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/4184242152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 0x557ae1df8780 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd824009230 tx=0x7fd82402a000 comp rx=0 tx=0).stop 2023-09-10T01:31:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.856+0000 7fd841b7fb80 1 -- 172.21.15.44:0/4184242152 shutdown_connections 2023-09-10T01:31:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.856+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/4184242152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e04f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.856+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/4184242152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1dfe1a0 0x557ae1dfe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.856+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/4184242152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 0x557ae1df8780 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.856+0000 7fd841b7fb80 1 -- 172.21.15.44:0/4184242152 >> 172.21.15.44:0/4184242152 conn(0x557ae1d6d030 msgr2=0x557ae1df3560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.863+0000 7fd841b7fb80 1 -- 172.21.15.44:0/4184242152 shutdown_connections 2023-09-10T01:31:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.863+0000 7fd841b7fb80 1 -- 172.21.15.44:0/4184242152 wait complete. 2023-09-10T01:31:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.863+0000 7fd841b7fb80 1 -- start start 2023-09-10T01:31:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.864+0000 7fd841b7fb80 1 Processor -- start 2023-09-10T01:31:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.864+0000 7fd841b7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 0x557ae1e07e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.864+0000 7fd841b7fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e08350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.864+0000 7fd841b7fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1dfe1a0 0x557ae1e02060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.864+0000 7fd841b7fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae1ce8b10 con 0x557ae1df8cc0 2023-09-10T01:31:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.864+0000 7fd841b7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae1ce8810 con 0x557ae1df8300 2023-09-10T01:31:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.864+0000 7fd841b7fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae1bcc8c0 con 0x557ae1dfe1a0 2023-09-10T01:31:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.864+0000 7fd841b7fb80 10 client.0 _ll_register_callbacks cb 0x557ae1c32a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:31:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.865+0000 7fd82effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 0x557ae1e07e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.865+0000 7fd82effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 0x557ae1e07e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34048/0 (socket says 172.21.15.44:34048) 2023-09-10T01:31:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.865+0000 7fd82effd700 1 -- 172.21.15.44:0/266610618 learned_addr learned my addr 172.21.15.44:0/266610618 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.865+0000 7fd82f7fe700 1 --2- 172.21.15.44:0/266610618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1dfe1a0 0x557ae1e02060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.865+0000 7fd82effd700 1 -- 172.21.15.44:0/266610618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1dfe1a0 msgr2=0x557ae1e02060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.865+0000 7fd82effd700 1 --2- 172.21.15.44:0/266610618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1dfe1a0 0x557ae1e02060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.865+0000 7fd82effd700 1 -- 172.21.15.44:0/266610618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 msgr2=0x557ae1e08350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.865+0000 7fd82effd700 1 --2- 172.21.15.44:0/266610618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e08350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.865+0000 7fd82effd700 1 -- 172.21.15.44:0/266610618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82401f040 con 0x557ae1df8300 2023-09-10T01:31:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.865+0000 7fd82f7fe700 1 --2- 172.21.15.44:0/266610618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1dfe1a0 0x557ae1e02060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:31:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.865+0000 7fd82effd700 1 --2- 172.21.15.44:0/266610618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 0x557ae1e07e10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd8200a0e90 tx=0x7fd820099b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=954dee4c097a2aa9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.865+0000 7fd812ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:31:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.866+0000 7fd812ffd700 1 -- 172.21.15.44:0/266610618 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8200a13c0 con 0x557ae1df8300 2023-09-10T01:31:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.866+0000 7fd812ffd700 1 -- 172.21.15.44:0/266610618 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fd8200b16a0 con 0x557ae1df8300 2023-09-10T01:31:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.866+0000 7fd841b7fb80 1 -- 172.21.15.44:0/266610618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ae1ca2490 con 0x557ae1df8300 2023-09-10T01:31:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.866+0000 7fd812ffd700 1 -- 172.21.15.44:0/266610618 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=182) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8200995b0 con 0x557ae1df8300 2023-09-10T01:31:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.866+0000 7fd841b7fb80 10 client.30143 fetch_fsmap learned FSMap version 182 2023-09-10T01:31:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.866+0000 7fd841b7fb80 1 -- 172.21.15.44:0/266610618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=182,monmap=4+}) v3 -- 0x557ae1b5fc00 con 0x557ae1df8300 2023-09-10T01:31:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.866+0000 7fd812ffd700 1 -- 172.21.15.44:0/266610618 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557ae1ca2490 con 0x557ae1df8300 2023-09-10T01:31:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.867+0000 7fd812ffd700 1 -- 172.21.15.44:0/266610618 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 182) v1 ==== 7343+0+0 (secure 0 0 0) 0x7fd820098840 con 0x557ae1df8300 2023-09-10T01:31:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.867+0000 7fd841b7fb80 10 client.30143 fetch_fsmap finished waiting for FSMap version 182 2023-09-10T01:31:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.867+0000 7fd841b7fb80 10 client.30143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:31:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.867+0000 7fd841b7fb80 20 client.30143 populate_metadata read hostname 'smithi044' 2023-09-10T01:31:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.867+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/266610618 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557ae1e12390 0x557ae1e147b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.867+0000 7fd841b7fb80 4 client.30143 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:31:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.867+0000 7fd841b7fb80 1 -- 172.21.15.44:0/266610618 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ae1d5f840 con 0x557ae1e12390 2023-09-10T01:31:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.868+0000 7fd82e7fc700 1 --2- 172.21.15.44:0/266610618 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557ae1e12390 0x557ae1e147b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.868+0000 7fd812ffd700 10 client.30143 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:31:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.868+0000 7fd82e7fc700 1 --2- 172.21.15.44:0/266610618 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557ae1e12390 0x557ae1e147b0 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e54d8d3577b0ef4f server_cookie=5f2e8899d35bb2bf in_seq=0 out_seq=0 2023-09-10T01:31:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.869+0000 7fd812ffd700 1 -- 172.21.15.44:0/266610618 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557ae1d5f840 con 0x557ae1e12390 2023-09-10T01:31:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.869+0000 7fd812ffd700 10 client.30143 handle_command_reply: tid=0 2023-09-10T01:31:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.869+0000 7fd841b7fb80 1 client.30143 shutdown 2023-09-10T01:31:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.870+0000 7fd841b7fb80 1 -- 172.21.15.44:0/266610618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 msgr2=0x557ae1e07e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.870+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/266610618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 0x557ae1e07e10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd8200a0e90 tx=0x7fd820099b80 comp rx=0 tx=0).stop 2023-09-10T01:31:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.871+0000 7fd841b7fb80 1 -- 172.21.15.44:0/266610618 shutdown_connections 2023-09-10T01:31:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.871+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/266610618 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557ae1e12390 0x557ae1e147b0 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.871+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/266610618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1dfe1a0 0x557ae1e02060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.871+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/266610618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 0x557ae1e07e10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.871+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/266610618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e08350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.871+0000 7fd841b7fb80 1 -- 172.21.15.44:0/266610618 >> 172.21.15.44:0/266610618 conn(0x557ae1d6d030 msgr2=0x557ae1df27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.871+0000 7fd812ffd700 0 client.30143 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.871+0000 7fd841b7fb80 1 -- 172.21.15.44:0/266610618 shutdown_connections 2023-09-10T01:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.871+0000 7fd841b7fb80 1 -- 172.21.15.44:0/266610618 wait complete. 2023-09-10T01:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.871+0000 7fd841b7fb80 20 client.30143 trim_cache size 0 max 16384 2023-09-10T01:31:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.900+0000 7fd841b7fb80 1 Processor -- start 2023-09-10T01:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.900+0000 7fd841b7fb80 1 -- start start 2023-09-10T01:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.900+0000 7fd841b7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 0x557ae1e01900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.900+0000 7fd841b7fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e01e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.900+0000 7fd841b7fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1dfe1a0 0x557ae1e079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.901+0000 7fd841b7fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae1ca2490 con 0x557ae1df8cc0 2023-09-10T01:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.901+0000 7fd841b7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae1deffc0 con 0x557ae1df8300 2023-09-10T01:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.901+0000 7fd841b7fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae1e03640 con 0x557ae1dfe1a0 2023-09-10T01:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.901+0000 7fd82e7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e01e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.901+0000 7fd82e7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e01e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60544/0 (socket says 172.21.15.44:60544) 2023-09-10T01:31:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.901+0000 7fd82e7fc700 1 -- 172.21.15.44:0/2338478243 learned_addr learned my addr 172.21.15.44:0/2338478243 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.901+0000 7fd82f7fe700 1 --2- 172.21.15.44:0/2338478243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1dfe1a0 0x557ae1e079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.901+0000 7fd82e7fc700 1 -- 172.21.15.44:0/2338478243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1dfe1a0 msgr2=0x557ae1e079a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.901+0000 7fd82e7fc700 1 --2- 172.21.15.44:0/2338478243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1dfe1a0 0x557ae1e079a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.901+0000 7fd82e7fc700 1 -- 172.21.15.44:0/2338478243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 msgr2=0x557ae1e01900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:31:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.901+0000 7fd82e7fc700 1 --2- 172.21.15.44:0/2338478243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 0x557ae1e01900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.901+0000 7fd82e7fc700 1 -- 172.21.15.44:0/2338478243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ae1b9d1f0 con 0x557ae1df8cc0 2023-09-10T01:31:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.902+0000 7fd82e7fc700 1 --2- 172.21.15.44:0/2338478243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e01e40 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fd824009230 tx=0x7fd82402b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6471d28b47a5ac74 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.902+0000 7fd81e7fc700 1 -- 172.21.15.44:0/2338478243 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82401c040 con 0x557ae1df8cc0 2023-09-10T01:31:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.902+0000 7fd81e7fc700 1 -- 172.21.15.44:0/2338478243 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd824005900 con 0x557ae1df8cc0 2023-09-10T01:31:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.902+0000 7fd81e7fc700 1 -- 172.21.15.44:0/2338478243 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824030410 con 0x557ae1df8cc0 2023-09-10T01:31:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.904+0000 7fd841b7fb80 1 -- 172.21.15.44:0/2338478243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 msgr2=0x557ae1e01e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.904+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/2338478243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e01e40 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fd824009230 tx=0x7fd82402b000 comp rx=0 tx=0).stop 2023-09-10T01:31:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.907+0000 7fd841b7fb80 1 -- 172.21.15.44:0/2338478243 shutdown_connections 2023-09-10T01:31:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.907+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/2338478243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1dfe1a0 0x557ae1e079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.907+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/2338478243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1df8300 0x557ae1e01900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.907+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/2338478243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e01e40 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.907+0000 7fd841b7fb80 1 -- 172.21.15.44:0/2338478243 >> 172.21.15.44:0/2338478243 conn(0x557ae1d6d030 msgr2=0x557ae1df3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.907+0000 7fd841b7fb80 1 -- 172.21.15.44:0/2338478243 shutdown_connections 2023-09-10T01:31:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.907+0000 7fd841b7fb80 1 -- 172.21.15.44:0/2338478243 wait complete. 2023-09-10T01:31:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.908+0000 7fd841b7fb80 1 -- start start 2023-09-10T01:31:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.908+0000 7fd841b7fb80 1 Processor -- start 2023-09-10T01:31:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.908+0000 7fd841b7fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1df8300 0x557ae1e5ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.908+0000 7fd841b7fb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e5f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.908+0000 7fd841b7fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1dfe1a0 0x557ae1e07790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.908+0000 7fd841b7fb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae1e07d90 con 0x557ae1df8cc0 2023-09-10T01:31:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.908+0000 7fd841b7fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae1e07f00 con 0x557ae1dfe1a0 2023-09-10T01:31:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.908+0000 7fd841b7fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae1e080c0 con 0x557ae1df8300 2023-09-10T01:31:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.908+0000 7fd841b7fb80 10 client.0 _ll_register_callbacks cb 0x557ae1c32a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:31:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.908+0000 7fd82e7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e5f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.909+0000 7fd82e7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e5f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60550/0 (socket says 172.21.15.44:60550) 2023-09-10T01:31:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.909+0000 7fd82e7fc700 1 -- 172.21.15.44:0/4110305989 learned_addr learned my addr 172.21.15.44:0/4110305989 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.909+0000 7fd82e7fc700 1 -- 172.21.15.44:0/4110305989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1df8300 msgr2=0x557ae1e5ecc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:31:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.909+0000 7fd82e7fc700 1 --2- 172.21.15.44:0/4110305989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1df8300 0x557ae1e5ecc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.909+0000 7fd82e7fc700 1 -- 172.21.15.44:0/4110305989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1dfe1a0 msgr2=0x557ae1e07790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:31:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.909+0000 7fd82e7fc700 1 --2- 172.21.15.44:0/4110305989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1dfe1a0 0x557ae1e07790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.909+0000 7fd82e7fc700 1 -- 172.21.15.44:0/4110305989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd824034040 con 0x557ae1df8cc0 2023-09-10T01:31:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.909+0000 7fd82e7fc700 1 --2- 172.21.15.44:0/4110305989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e5f200 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fd82400fad0 tx=0x7fd82403b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1996d3f5956d04 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.910+0000 7fd812ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:31:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.910+0000 7fd812ffd700 1 -- 172.21.15.44:0/4110305989 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82401c030 con 0x557ae1df8cc0 2023-09-10T01:31:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.910+0000 7fd812ffd700 1 -- 172.21.15.44:0/4110305989 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fd82403bcf0 con 0x557ae1df8cc0 2023-09-10T01:31:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.910+0000 7fd841b7fb80 1 -- 172.21.15.44:0/4110305989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8240054c0 con 0x557ae1df8cc0 2023-09-10T01:31:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.910+0000 7fd812ffd700 1 -- 172.21.15.44:0/4110305989 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=182) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd82400b630 con 0x557ae1df8cc0 2023-09-10T01:31:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.914+0000 7fd841b7fb80 10 client.20499 fetch_fsmap learned FSMap version 182 2023-09-10T01:31:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.914+0000 7fd841b7fb80 1 -- 172.21.15.44:0/4110305989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=182,monmap=4+}) v3 -- 0x557ae1b75250 con 0x557ae1df8cc0 2023-09-10T01:31:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.914+0000 7fd812ffd700 1 -- 172.21.15.44:0/4110305989 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8240054c0 con 0x557ae1df8cc0 2023-09-10T01:31:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.914+0000 7fd812ffd700 1 -- 172.21.15.44:0/4110305989 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 182) v1 ==== 7343+0+0 (secure 0 0 0) 0x7fd824044a70 con 0x557ae1df8cc0 2023-09-10T01:31:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.914+0000 7fd841b7fb80 10 client.20499 fetch_fsmap finished waiting for FSMap version 182 2023-09-10T01:31:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.914+0000 7fd841b7fb80 10 client.20499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:31:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.914+0000 7fd841b7fb80 20 client.20499 populate_metadata read hostname 'smithi044' 2023-09-10T01:31:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.914+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/4110305989 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557ae1e665f0 0x557ae1e689e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.914+0000 7fd841b7fb80 4 client.20499 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:31:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.914+0000 7fd841b7fb80 1 -- 172.21.15.44:0/4110305989 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557ae1d438c0 con 0x557ae1e665f0 2023-09-10T01:31:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.915+0000 7fd82effd700 1 --2- 172.21.15.44:0/4110305989 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557ae1e665f0 0x557ae1e689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.915+0000 7fd812ffd700 10 client.20499 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:31:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:33.915+0000 7fd82effd700 1 --2- 172.21.15.44:0/4110305989 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557ae1e665f0 0x557ae1e689e0 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1b0bd558dfce2cb server_cookie=a1aa17bbc55281b7 in_seq=0 out_seq=0 2023-09-10T01:31:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.005+0000 7fd812ffd700 1 -- 172.21.15.44:0/4110305989 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557ae1d438c0 con 0x557ae1e665f0 2023-09-10T01:31:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.005+0000 7fd812ffd700 10 client.20499 handle_command_reply: tid=0 2023-09-10T01:31:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.005+0000 7fd841b7fb80 1 client.20499 shutdown 2023-09-10T01:31:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.013+0000 7fd841b7fb80 1 -- 172.21.15.44:0/4110305989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 msgr2=0x557ae1e5f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.013+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/4110305989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e5f200 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fd82400fad0 tx=0x7fd82403b620 comp rx=0 tx=0).stop 2023-09-10T01:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.013+0000 7fd841b7fb80 1 -- 172.21.15.44:0/4110305989 shutdown_connections 2023-09-10T01:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.013+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/4110305989 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557ae1e665f0 0x557ae1e689e0 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.013+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/4110305989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ae1dfe1a0 0x557ae1e07790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.013+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/4110305989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ae1df8300 0x557ae1e5ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.013+0000 7fd841b7fb80 1 --2- 172.21.15.44:0/4110305989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae1df8cc0 0x557ae1e5f200 secure :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fd82400fad0 tx=0x7fd82403b620 comp rx=0 tx=0).stop 2023-09-10T01:31:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.013+0000 7fd841b7fb80 1 -- 172.21.15.44:0/4110305989 >> 172.21.15.44:0/4110305989 conn(0x557ae1d6d030 msgr2=0x557ae1dfd090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.016+0000 7fd812ffd700 0 client.20499 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:31:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.016+0000 7fd841b7fb80 1 -- 172.21.15.44:0/4110305989 shutdown_connections 2023-09-10T01:31:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.016+0000 7fd841b7fb80 1 -- 172.21.15.44:0/4110305989 wait complete. 2023-09-10T01:31:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:31:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:31:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:31:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:31:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.020+0000 7fd841b7fb80 20 client.20499 trim_cache size 0 max 16384 2023-09-10T01:31:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.028+0000 7fd8359b9700 1 -- 172.21.15.44:0/611890419 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd8180a08f0 msgr2=0x7fd8180a2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.028+0000 7fd8359b9700 1 --2- 172.21.15.44:0/611890419 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd8180a08f0 0x7fd8180a2db0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd824000c00 tx=0x7fd824010ec0 comp rx=0 tx=0).stop 2023-09-10T01:31:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.028+0000 7fd8359b9700 1 -- 172.21.15.44:0/611890419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd83010ed50 msgr2=0x7fd830118140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.028+0000 7fd8359b9700 1 --2- 172.21.15.44:0/611890419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd83010ed50 0x7fd830118140 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd82000d980 tx=0x7fd82000dc90 comp rx=0 tx=0).stop 2023-09-10T01:31:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.031+0000 7fd8359b9700 1 -- 172.21.15.44:0/611890419 shutdown_connections 2023-09-10T01:31:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.031+0000 7fd8359b9700 1 --2- 172.21.15.44:0/611890419 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd8180a08f0 0x7fd8180a2db0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.031+0000 7fd8359b9700 1 --2- 172.21.15.44:0/611890419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd830119de0 0x7fd830110fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.031+0000 7fd8359b9700 1 --2- 172.21.15.44:0/611890419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd83010ed50 0x7fd830118140 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.031+0000 7fd8359b9700 1 --2- 172.21.15.44:0/611890419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd83010f1c0 0x7fd8301186c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.031+0000 7fd8359b9700 1 -- 172.21.15.44:0/611890419 >> 172.21.15.44:0/611890419 conn(0x7fd83006c6c0 msgr2=0x7fd83006efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.033+0000 7fd8359b9700 1 -- 172.21.15.44:0/611890419 shutdown_connections 2023-09-10T01:31:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:34.033+0000 7fd8359b9700 1 -- 172.21.15.44:0/611890419 wait complete. 2023-09-10T01:31:34.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:33 smithi131 ceph-mon[103713]: pgmap v1657: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 672 op/s 2023-09-10T01:31:34.073 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:33 smithi044 ceph-mon[98676]: pgmap v1657: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 672 op/s 2023-09-10T01:31:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:34 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:35.062 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:34 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:34 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:36.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:35 smithi131 ceph-mon[103713]: pgmap v1658: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 670 op/s 2023-09-10T01:31:36.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:35 smithi044 ceph-mon[98676]: pgmap v1658: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 670 op/s 2023-09-10T01:31:36.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:35 smithi099 ceph-mon[102692]: pgmap v1658: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 670 op/s 2023-09-10T01:31:36.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 630.0 s, 273.2 tps, lat 323.157 ms stddev 318.806 2023-09-10T01:31:37.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:37 smithi131 ceph-mon[103713]: pgmap v1659: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.3 MiB/s wr, 709 op/s 2023-09-10T01:31:37.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:37 smithi044 ceph-mon[98676]: pgmap v1659: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.3 MiB/s wr, 709 op/s 2023-09-10T01:31:37.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:37 smithi099 ceph-mon[102692]: pgmap v1659: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.3 MiB/s wr, 709 op/s 2023-09-10T01:31:38.811 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:38 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:38.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:38 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:38 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:39.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:39 smithi099 ceph-mon[102692]: pgmap v1660: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.5 MiB/s wr, 679 op/s 2023-09-10T01:31:40.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:39 smithi131 ceph-mon[103713]: pgmap v1660: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.5 MiB/s wr, 679 op/s 2023-09-10T01:31:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:39 smithi044 ceph-mon[98676]: pgmap v1660: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.5 MiB/s wr, 679 op/s 2023-09-10T01:31:41.184 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 635.0 s, 270.2 tps, lat 398.264 ms stddev 593.511 2023-09-10T01:31:41.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:41 smithi099 ceph-mon[102692]: pgmap v1661: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.5 MiB/s wr, 613 op/s 2023-09-10T01:31:42.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:41 smithi131 ceph-mon[103713]: pgmap v1661: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.5 MiB/s wr, 613 op/s 2023-09-10T01:31:42.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:41 smithi044 ceph-mon[98676]: pgmap v1661: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.5 MiB/s wr, 613 op/s 2023-09-10T01:31:43.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:43 smithi099 ceph-mon[102692]: pgmap v1662: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.5 MiB/s wr, 895 op/s 2023-09-10T01:31:43.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:43 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:44.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:43 smithi131 ceph-mon[103713]: pgmap v1662: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.5 MiB/s wr, 895 op/s 2023-09-10T01:31:44.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:43 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:44.063 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:43 smithi044 ceph-mon[98676]: pgmap v1662: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.5 MiB/s wr, 895 op/s 2023-09-10T01:31:44.064 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:43 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:46.051 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:45 smithi131 ceph-mon[103713]: pgmap v1663: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.8 MiB/s wr, 721 op/s 2023-09-10T01:31:46.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:45 smithi044 ceph-mon[98676]: pgmap v1663: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.8 MiB/s wr, 721 op/s 2023-09-10T01:31:46.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:45 smithi099 ceph-mon[102692]: pgmap v1663: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.8 MiB/s wr, 721 op/s 2023-09-10T01:31:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 640.0 s, 293.0 tps, lat 354.514 ms stddev 385.056 2023-09-10T01:31:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.295+0000 7f4b794be700 1 -- 172.21.15.44:0/1100628136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b7410f1c0 msgr2=0x7f4b74107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.295+0000 7f4b794be700 1 --2- 172.21.15.44:0/1100628136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b7410f1c0 0x7f4b74107b40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00b0a0 tx=0x7f4b6c00b3b0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.296+0000 7f4b794be700 1 -- 172.21.15.44:0/1100628136 shutdown_connections 2023-09-10T01:31:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.296+0000 7f4b794be700 1 --2- 172.21.15.44:0/1100628136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b74119de0 0x7f4b7411c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.296+0000 7f4b794be700 1 --2- 172.21.15.44:0/1100628136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b7410ed50 0x7f4b74107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.296+0000 7f4b794be700 1 --2- 172.21.15.44:0/1100628136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b7410f1c0 0x7f4b74107b40 secure :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00b0a0 tx=0x7f4b6c00b3b0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.296+0000 7f4b794be700 1 -- 172.21.15.44:0/1100628136 >> 172.21.15.44:0/1100628136 conn(0x7f4b7406c6c0 msgr2=0x7f4b7406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.301+0000 7f4b794be700 1 -- 172.21.15.44:0/1100628136 shutdown_connections 2023-09-10T01:31:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.303+0000 7f4b794be700 1 -- 172.21.15.44:0/1100628136 wait complete. 2023-09-10T01:31:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.304+0000 7f4b794be700 1 Processor -- start 2023-09-10T01:31:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.304+0000 7f4b794be700 1 -- start start 2023-09-10T01:31:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.305+0000 7f4b794be700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b7410ed50 0x7f4b740785e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.305+0000 7f4b794be700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b74119de0 0x7f4b74078b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.305+0000 7f4b794be700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b74073470 0x7f4b74073920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.305+0000 7f4b794be700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b74074030 con 0x7f4b74119de0 2023-09-10T01:31:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.305+0000 7f4b794be700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740741a0 con 0x7f4b7410ed50 2023-09-10T01:31:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.305+0000 7f4b794be700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b74080370 con 0x7f4b74073470 2023-09-10T01:31:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.305+0000 7f4b737fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b74073470 0x7f4b74073920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.305+0000 7f4b737fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b74073470 0x7f4b74073920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39212/0 (socket says 172.21.15.44:39212) 2023-09-10T01:31:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.305+0000 7f4b737fe700 1 -- 172.21.15.44:0/3738177055 learned_addr learned my addr 172.21.15.44:0/3738177055 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.305+0000 7f4b72ffd700 1 --2- 172.21.15.44:0/3738177055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b7410ed50 0x7f4b740785e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.305+0000 7f4b727fc700 1 --2- 172.21.15.44:0/3738177055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b74119de0 0x7f4b74078b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.305+0000 7f4b737fe700 1 -- 172.21.15.44:0/3738177055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b7410ed50 msgr2=0x7f4b740785e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.305+0000 7f4b737fe700 1 --2- 172.21.15.44:0/3738177055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b7410ed50 0x7f4b740785e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.306+0000 7f4b737fe700 1 -- 172.21.15.44:0/3738177055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b74119de0 msgr2=0x7f4b74078b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.306+0000 7f4b737fe700 1 --2- 172.21.15.44:0/3738177055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b74119de0 0x7f4b74078b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.306+0000 7f4b737fe700 1 -- 172.21.15.44:0/3738177055 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6c009d00 con 0x7f4b74073470 2023-09-10T01:31:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.306+0000 7f4b737fe700 1 --2- 172.21.15.44:0/3738177055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b74073470 0x7f4b74073920 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800b480 tx=0x7f4b6800b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.306+0000 7f4b5bfff700 1 -- 172.21.15.44:0/3738177055 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68011830 con 0x7f4b74073470 2023-09-10T01:31:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.306+0000 7f4b794be700 1 -- 172.21.15.44:0/3738177055 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b74080650 con 0x7f4b74073470 2023-09-10T01:31:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.306+0000 7f4b794be700 1 -- 172.21.15.44:0/3738177055 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b74080b30 con 0x7f4b74073470 2023-09-10T01:31:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.306+0000 7f4b5bfff700 1 -- 172.21.15.44:0/3738177055 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b6800be90 con 0x7f4b74073470 2023-09-10T01:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.306+0000 7f4b5bfff700 1 -- 172.21.15.44:0/3738177055 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68010cd0 con 0x7f4b74073470 2023-09-10T01:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.310+0000 7f4b5bfff700 1 -- 172.21.15.44:0/3738177055 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f4b68010410 con 0x7f4b74073470 2023-09-10T01:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.311+0000 7f4b5bfff700 1 --2- 172.21.15.44:0/3738177055 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4b5c0a0870 0x7f4b5c0a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.311+0000 7f4b85684b80 1 Processor -- start 2023-09-10T01:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.311+0000 7f4b5bfff700 1 -- 172.21.15.44:0/3738177055 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f4b680cbab0 con 0x7f4b74073470 2023-09-10T01:31:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.312+0000 7f4b72ffd700 1 --2- 172.21.15.44:0/3738177055 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4b5c0a0870 0x7f4b5c0a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.312+0000 7f4b72ffd700 1 --2- 172.21.15.44:0/3738177055 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4b5c0a0870 0x7f4b5c0a2d30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4b64006fd0 tx=0x7f4b64009190 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.315+0000 7f4b85684b80 1 -- start start 2023-09-10T01:31:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.315+0000 7f4b85684b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cfc1a0 0x557f78cfc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.315+0000 7f4b85684b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cf6300 0x557f78cf6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.315+0000 7f4b85684b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cf6cc0 0x557f78d02f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.315+0000 7f4b85684b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f78c4f6d0 con 0x557f78cf6cc0 2023-09-10T01:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.315+0000 7f4b85684b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f78c4cff0 con 0x557f78cfc1a0 2023-09-10T01:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.315+0000 7f4b85684b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f78c488e0 con 0x557f78cf6300 2023-09-10T01:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.315+0000 7f4b72ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cfc1a0 0x557f78cfc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.315+0000 7f4b727fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cf6300 0x557f78cf6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.315+0000 7f4b727fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cf6300 0x557f78cf6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39218/0 (socket says 172.21.15.44:39218) 2023-09-10T01:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.315+0000 7f4b72ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cfc1a0 0x557f78cfc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34074/0 (socket says 172.21.15.44:34074) 2023-09-10T01:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.315+0000 7f4b727fc700 1 -- 172.21.15.44:0/1219133306 learned_addr learned my addr 172.21.15.44:0/1219133306 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.316+0000 7f4b727fc700 1 -- 172.21.15.44:0/1219133306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cfc1a0 msgr2=0x557f78cfc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.316+0000 7f4b727fc700 1 --2- 172.21.15.44:0/1219133306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cfc1a0 0x557f78cfc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.316+0000 7f4b727fc700 1 -- 172.21.15.44:0/1219133306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cf6cc0 msgr2=0x557f78d02f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:31:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.316+0000 7f4b727fc700 1 --2- 172.21.15.44:0/1219133306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cf6cc0 0x557f78d02f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.316+0000 7f4b727fc700 1 -- 172.21.15.44:0/1219133306 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f78a74d80 con 0x557f78cf6300 2023-09-10T01:31:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.316+0000 7f4b72ffd700 1 --2- 172.21.15.44:0/1219133306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cfc1a0 0x557f78cfc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:31:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.316+0000 7f4b727fc700 1 --2- 172.21.15.44:0/1219133306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cf6300 0x557f78cf6780 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009040 tx=0x7f4b6c007ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44145fd99af14dcd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.320+0000 7f4b597fa700 1 -- 172.21.15.44:0/1219133306 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c00e040 con 0x557f78cf6300 2023-09-10T01:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.320+0000 7f4b597fa700 1 -- 172.21.15.44:0/1219133306 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b6c003e90 con 0x557f78cf6300 2023-09-10T01:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.320+0000 7f4b597fa700 1 -- 172.21.15.44:0/1219133306 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0079d0 con 0x557f78cf6300 2023-09-10T01:31:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.321+0000 7f4b85684b80 1 -- 172.21.15.44:0/1219133306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cf6300 msgr2=0x557f78cf6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.321+0000 7f4b85684b80 1 --2- 172.21.15.44:0/1219133306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cf6300 0x557f78cf6780 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009040 tx=0x7f4b6c007ed0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.321+0000 7f4b85684b80 1 -- 172.21.15.44:0/1219133306 shutdown_connections 2023-09-10T01:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.321+0000 7f4b85684b80 1 --2- 172.21.15.44:0/1219133306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cf6cc0 0x557f78d02f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.321+0000 7f4b85684b80 1 --2- 172.21.15.44:0/1219133306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cfc1a0 0x557f78cfc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.321+0000 7f4b85684b80 1 --2- 172.21.15.44:0/1219133306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cf6300 0x557f78cf6780 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.321+0000 7f4b85684b80 1 -- 172.21.15.44:0/1219133306 >> 172.21.15.44:0/1219133306 conn(0x557f78c6b030 msgr2=0x557f78cf1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.324+0000 7f4b85684b80 1 -- 172.21.15.44:0/1219133306 shutdown_connections 2023-09-10T01:31:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.324+0000 7f4b85684b80 1 -- 172.21.15.44:0/1219133306 wait complete. 2023-09-10T01:31:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.324+0000 7f4b85684b80 1 -- start start 2023-09-10T01:31:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.325+0000 7f4b85684b80 1 Processor -- start 2023-09-10T01:31:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.325+0000 7f4b85684b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cf6300 0x557f78d02db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.325+0000 7f4b85684b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cf6cc0 0x557f78d000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.325+0000 7f4b85684b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cfc1a0 0x557f78d00660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.325+0000 7f4b85684b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f78be6b10 con 0x557f78cf6300 2023-09-10T01:31:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.325+0000 7f4b85684b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f78be6810 con 0x557f78cf6cc0 2023-09-10T01:31:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.325+0000 7f4b85684b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f78aca8c0 con 0x557f78cfc1a0 2023-09-10T01:31:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.325+0000 7f4b85684b80 10 client.0 _ll_register_callbacks cb 0x557f78b30a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:31:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.325+0000 7f4b737fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cfc1a0 0x557f78d00660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.325+0000 7f4b737fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cfc1a0 0x557f78d00660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39220/0 (socket says 172.21.15.44:39220) 2023-09-10T01:31:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.325+0000 7f4b737fe700 1 -- 172.21.15.44:0/1408656604 learned_addr learned my addr 172.21.15.44:0/1408656604 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.326+0000 7f4b72ffd700 1 --2- 172.21.15.44:0/1408656604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cf6300 0x557f78d02db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.326+0000 7f4b737fe700 1 -- 172.21.15.44:0/1408656604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cf6cc0 msgr2=0x557f78d000a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:31:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.326+0000 7f4b737fe700 1 --2- 172.21.15.44:0/1408656604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cf6cc0 0x557f78d000a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.326+0000 7f4b737fe700 1 -- 172.21.15.44:0/1408656604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cf6300 msgr2=0x557f78d02db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.326+0000 7f4b737fe700 1 --2- 172.21.15.44:0/1408656604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cf6300 0x557f78d02db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.326+0000 7f4b737fe700 1 -- 172.21.15.44:0/1408656604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6400e6c0 con 0x557f78cfc1a0 2023-09-10T01:31:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.326+0000 7f4b737fe700 1 --2- 172.21.15.44:0/1408656604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cfc1a0 0x557f78d00660 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4b680c9410 tx=0x7f4b68098710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c89d394ef1c95c1b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.326+0000 7f4b56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:31:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.326+0000 7f4b56ff5700 1 -- 172.21.15.44:0/1408656604 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b680990e0 con 0x557f78cfc1a0 2023-09-10T01:31:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.326+0000 7f4b56ff5700 1 -- 172.21.15.44:0/1408656604 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f4b6809cab0 con 0x557f78cfc1a0 2023-09-10T01:31:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.326+0000 7f4b85684b80 1 -- 172.21.15.44:0/1408656604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f78ba0490 con 0x557f78cfc1a0 2023-09-10T01:31:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.326+0000 7f4b56ff5700 1 -- 172.21.15.44:0/1408656604 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=184) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b680a0cf0 con 0x557f78cfc1a0 2023-09-10T01:31:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.326+0000 7f4b85684b80 10 client.30169 fetch_fsmap learned FSMap version 184 2023-09-10T01:31:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.326+0000 7f4b85684b80 1 -- 172.21.15.44:0/1408656604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=184,monmap=4+}) v3 -- 0x7f4b7411cb00 con 0x557f78cfc1a0 2023-09-10T01:31:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.327+0000 7f4b56ff5700 1 -- 172.21.15.44:0/1408656604 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557f78ba0490 con 0x557f78cfc1a0 2023-09-10T01:31:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.327+0000 7f4b56ff5700 1 -- 172.21.15.44:0/1408656604 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 184) v1 ==== 7359+0+0 (secure 0 0 0) 0x7f4b680b80b0 con 0x557f78cfc1a0 2023-09-10T01:31:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.327+0000 7f4b85684b80 10 client.30169 fetch_fsmap finished waiting for FSMap version 184 2023-09-10T01:31:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.327+0000 7f4b85684b80 10 client.30169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:31:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.327+0000 7f4b85684b80 20 client.30169 populate_metadata read hostname 'smithi044' 2023-09-10T01:31:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.327+0000 7f4b85684b80 1 --2- 172.21.15.44:0/1408656604 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557f78d06370 0x557f78d08790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.327+0000 7f4b85684b80 4 client.30169 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:31:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.327+0000 7f4b85684b80 1 -- 172.21.15.44:0/1408656604 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f78c5d840 con 0x557f78d06370 2023-09-10T01:31:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.327+0000 7f4b72ffd700 1 --2- 172.21.15.44:0/1408656604 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557f78d06370 0x557f78d08790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.328+0000 7f4b56ff5700 10 client.30169 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:31:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.328+0000 7f4b72ffd700 1 --2- 172.21.15.44:0/1408656604 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557f78d06370 0x557f78d08790 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc2228909ca55f35 server_cookie=952ed89c1864d5a9 in_seq=0 out_seq=0 2023-09-10T01:31:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.329+0000 7f4b56ff5700 1 -- 172.21.15.44:0/1408656604 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557f78c5d840 con 0x557f78d06370 2023-09-10T01:31:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.329+0000 7f4b56ff5700 10 client.30169 handle_command_reply: tid=0 2023-09-10T01:31:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.329+0000 7f4b85684b80 1 client.30169 shutdown 2023-09-10T01:31:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.329+0000 7f4b85684b80 1 -- 172.21.15.44:0/1408656604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cfc1a0 msgr2=0x557f78d00660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.329+0000 7f4b85684b80 1 --2- 172.21.15.44:0/1408656604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cfc1a0 0x557f78d00660 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4b680c9410 tx=0x7f4b68098710 comp rx=0 tx=0).stop 2023-09-10T01:31:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.332+0000 7f4b85684b80 1 -- 172.21.15.44:0/1408656604 shutdown_connections 2023-09-10T01:31:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.332+0000 7f4b85684b80 1 --2- 172.21.15.44:0/1408656604 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557f78d06370 0x557f78d08790 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.333+0000 7f4b85684b80 1 --2- 172.21.15.44:0/1408656604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cfc1a0 0x557f78d00660 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.333+0000 7f4b85684b80 1 --2- 172.21.15.44:0/1408656604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cf6300 0x557f78d02db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.333+0000 7f4b85684b80 1 --2- 172.21.15.44:0/1408656604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cf6cc0 0x557f78d000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.333+0000 7f4b85684b80 1 -- 172.21.15.44:0/1408656604 >> 172.21.15.44:0/1408656604 conn(0x557f78c6b030 msgr2=0x557f78cf07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.333+0000 7f4b56ff5700 0 client.30169 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:31:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.333+0000 7f4b85684b80 1 -- 172.21.15.44:0/1408656604 shutdown_connections 2023-09-10T01:31:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.333+0000 7f4b85684b80 1 -- 172.21.15.44:0/1408656604 wait complete. 2023-09-10T01:31:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.333+0000 7f4b85684b80 20 client.30169 trim_cache size 0 max 16384 2023-09-10T01:31:47.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.361+0000 7f4b85684b80 1 Processor -- start 2023-09-10T01:31:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.361+0000 7f4b85684b80 1 -- start start 2023-09-10T01:31:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.361+0000 7f4b85684b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cf6300 0x557f78cfdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.361+0000 7f4b85684b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cf6cc0 0x557f78cf8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.362+0000 7f4b85684b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cfc1a0 0x557f78cf90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.362+0000 7f4b85684b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f78d00c60 con 0x557f78cfc1a0 2023-09-10T01:31:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.362+0000 7f4b85684b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f78cf7580 con 0x557f78cf6300 2023-09-10T01:31:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.362+0000 7f4b85684b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f78ce8f10 con 0x557f78cf6cc0 2023-09-10T01:31:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.362+0000 7f4b737fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cfc1a0 0x557f78cf90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.362+0000 7f4b737fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cfc1a0 0x557f78cf90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60578/0 (socket says 172.21.15.44:60578) 2023-09-10T01:31:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.362+0000 7f4b737fe700 1 -- 172.21.15.44:0/3995963199 learned_addr learned my addr 172.21.15.44:0/3995963199 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.363+0000 7f4b737fe700 1 -- 172.21.15.44:0/3995963199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cf6cc0 msgr2=0x557f78cf8b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:31:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.363+0000 7f4b737fe700 1 --2- 172.21.15.44:0/3995963199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cf6cc0 0x557f78cf8b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.363+0000 7f4b737fe700 1 -- 172.21.15.44:0/3995963199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cf6300 msgr2=0x557f78cfdd20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:31:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.363+0000 7f4b737fe700 1 --2- 172.21.15.44:0/3995963199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cf6300 0x557f78cfdd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.363+0000 7f4b737fe700 1 -- 172.21.15.44:0/3995963199 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f78a5dc00 con 0x557f78cfc1a0 2023-09-10T01:31:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.372+0000 7f4b737fe700 1 --2- 172.21.15.44:0/3995963199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cfc1a0 0x557f78cf90a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f4b680c8fd0 tx=0x7f4b68097230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5770ca2600a4ec server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.378+0000 7f4b59ffb700 1 -- 172.21.15.44:0/3995963199 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b680a0cf0 con 0x557f78cfc1a0 2023-09-10T01:31:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.378+0000 7f4b59ffb700 1 -- 172.21.15.44:0/3995963199 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b68098c80 con 0x557f78cfc1a0 2023-09-10T01:31:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.378+0000 7f4b59ffb700 1 -- 172.21.15.44:0/3995963199 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68094c00 con 0x557f78cfc1a0 2023-09-10T01:31:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.379+0000 7f4b85684b80 1 -- 172.21.15.44:0/3995963199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cfc1a0 msgr2=0x557f78cf90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.379+0000 7f4b85684b80 1 --2- 172.21.15.44:0/3995963199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cfc1a0 0x557f78cf90a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f4b680c8fd0 tx=0x7f4b68097230 comp rx=0 tx=0).stop 2023-09-10T01:31:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.379+0000 7f4b85684b80 1 -- 172.21.15.44:0/3995963199 shutdown_connections 2023-09-10T01:31:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.379+0000 7f4b85684b80 1 --2- 172.21.15.44:0/3995963199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cfc1a0 0x557f78cf90a0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.379+0000 7f4b85684b80 1 --2- 172.21.15.44:0/3995963199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cf6300 0x557f78cfdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.379+0000 7f4b85684b80 1 --2- 172.21.15.44:0/3995963199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cf6cc0 0x557f78cf8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.379+0000 7f4b85684b80 1 -- 172.21.15.44:0/3995963199 >> 172.21.15.44:0/3995963199 conn(0x557f78c6b030 msgr2=0x557f78cfa220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.384+0000 7f4b85684b80 1 -- 172.21.15.44:0/3995963199 shutdown_connections 2023-09-10T01:31:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.386+0000 7f4b85684b80 1 -- 172.21.15.44:0/3995963199 wait complete. 2023-09-10T01:31:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.387+0000 7f4b85684b80 1 -- start start 2023-09-10T01:31:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.387+0000 7f4b85684b80 1 Processor -- start 2023-09-10T01:31:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.387+0000 7f4b85684b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cf6300 0x557f78d7fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.387+0000 7f4b85684b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cf6cc0 0x557f78d7ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.387+0000 7f4b85684b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cfc1a0 0x557f78d16520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.387+0000 7f4b85684b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f78d16af0 con 0x557f78cf6cc0 2023-09-10T01:31:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.387+0000 7f4b85684b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f78d16c60 con 0x557f78cf6300 2023-09-10T01:31:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.387+0000 7f4b85684b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f78d16e20 con 0x557f78cfc1a0 2023-09-10T01:31:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.387+0000 7f4b85684b80 10 client.0 _ll_register_callbacks cb 0x557f78b30a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:31:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.388+0000 7f4b72ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cf6300 0x557f78d7fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.388+0000 7f4b72ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cf6300 0x557f78d7fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34088/0 (socket says 172.21.15.44:34088) 2023-09-10T01:31:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.388+0000 7f4b72ffd700 1 -- 172.21.15.44:0/366475444 learned_addr learned my addr 172.21.15.44:0/366475444 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:31:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.388+0000 7f4b737fe700 1 --2- 172.21.15.44:0/366475444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cfc1a0 0x557f78d16520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.388+0000 7f4b72ffd700 1 -- 172.21.15.44:0/366475444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cfc1a0 msgr2=0x557f78d16520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:47.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.388+0000 7f4b72ffd700 1 --2- 172.21.15.44:0/366475444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cfc1a0 0x557f78d16520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.388+0000 7f4b72ffd700 1 -- 172.21.15.44:0/366475444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cf6cc0 msgr2=0x557f78d7ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:31:47.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.388+0000 7f4b72ffd700 1 --2- 172.21.15.44:0/366475444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cf6cc0 0x557f78d7ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.388+0000 7f4b72ffd700 1 -- 172.21.15.44:0/366475444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7411cb00 con 0x557f78cf6300 2023-09-10T01:31:47.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.388+0000 7f4b72ffd700 1 --2- 172.21.15.44:0/366475444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cf6300 0x557f78d7fa00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009530 tx=0x7f4b64004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cf6b3ec1d62673a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:31:47.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.391+0000 7f4b56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:31:47.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.391+0000 7f4b56ff5700 1 -- 172.21.15.44:0/366475444 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64027070 con 0x557f78cf6300 2023-09-10T01:31:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.391+0000 7f4b56ff5700 1 -- 172.21.15.44:0/366475444 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f4b64028460 con 0x557f78cf6300 2023-09-10T01:31:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.395+0000 7f4b85684b80 1 -- 172.21.15.44:0/366475444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b6c003e90 con 0x557f78cf6300 2023-09-10T01:31:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.395+0000 7f4b56ff5700 1 -- 172.21.15.44:0/366475444 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=184) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b6400eec0 con 0x557f78cf6300 2023-09-10T01:31:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.395+0000 7f4b85684b80 10 client.30152 fetch_fsmap learned FSMap version 184 2023-09-10T01:31:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.395+0000 7f4b85684b80 1 -- 172.21.15.44:0/366475444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=184,monmap=4+}) v3 -- 0x557f78a9b1f0 con 0x557f78cf6300 2023-09-10T01:31:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.396+0000 7f4b56ff5700 1 -- 172.21.15.44:0/366475444 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b6c003e90 con 0x557f78cf6300 2023-09-10T01:31:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.396+0000 7f4b56ff5700 1 -- 172.21.15.44:0/366475444 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 184) v1 ==== 7359+0+0 (secure 0 0 0) 0x7f4b64015b80 con 0x557f78cf6300 2023-09-10T01:31:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.396+0000 7f4b85684b80 10 client.30152 fetch_fsmap finished waiting for FSMap version 184 2023-09-10T01:31:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.396+0000 7f4b85684b80 10 client.30152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:31:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.396+0000 7f4b85684b80 20 client.30152 populate_metadata read hostname 'smithi044' 2023-09-10T01:31:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.396+0000 7f4b85684b80 1 --2- 172.21.15.44:0/366475444 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557f78d87880 0x557f78d89ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:31:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.396+0000 7f4b85684b80 4 client.30152 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:31:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.396+0000 7f4b85684b80 1 -- 172.21.15.44:0/366475444 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557f78c418c0 con 0x557f78d87880 2023-09-10T01:31:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.398+0000 7f4b727fc700 1 --2- 172.21.15.44:0/366475444 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557f78d87880 0x557f78d89ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:31:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.400+0000 7f4b56ff5700 10 client.30152 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:31:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.400+0000 7f4b727fc700 1 --2- 172.21.15.44:0/366475444 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557f78d87880 0x557f78d89ca0 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f063400c469456b server_cookie=c8fe7e1354178926 in_seq=0 out_seq=0 2023-09-10T01:31:47.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.467+0000 7f4b56ff5700 1 -- 172.21.15.44:0/366475444 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557f78c418c0 con 0x557f78d87880 2023-09-10T01:31:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.467+0000 7f4b56ff5700 10 client.30152 handle_command_reply: tid=0 2023-09-10T01:31:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.467+0000 7f4b85684b80 1 client.30152 shutdown 2023-09-10T01:31:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.471+0000 7f4b85684b80 1 -- 172.21.15.44:0/366475444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cf6300 msgr2=0x557f78d7fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.471+0000 7f4b85684b80 1 --2- 172.21.15.44:0/366475444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cf6300 0x557f78d7fa00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009530 tx=0x7f4b64004790 comp rx=0 tx=0).stop 2023-09-10T01:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.475+0000 7f4b85684b80 1 -- 172.21.15.44:0/366475444 shutdown_connections 2023-09-10T01:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.475+0000 7f4b85684b80 1 --2- 172.21.15.44:0/366475444 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x557f78d87880 0x557f78d89ca0 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.475+0000 7f4b85684b80 1 --2- 172.21.15.44:0/366475444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f78cfc1a0 0x557f78d16520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.475+0000 7f4b85684b80 1 --2- 172.21.15.44:0/366475444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f78cf6300 0x557f78d7fa00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.475+0000 7f4b85684b80 1 --2- 172.21.15.44:0/366475444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f78cf6cc0 0x557f78d7ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.475+0000 7f4b85684b80 1 -- 172.21.15.44:0/366475444 >> 172.21.15.44:0/366475444 conn(0x557f78c6b030 msgr2=0x557f78cfb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.475+0000 7f4b56ff5700 0 client.30152 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:31:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.482+0000 7f4b85684b80 1 -- 172.21.15.44:0/366475444 shutdown_connections 2023-09-10T01:31:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.488+0000 7f4b85684b80 1 -- 172.21.15.44:0/366475444 wait complete. 2023-09-10T01:31:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.492+0000 7f4b85684b80 20 client.30152 trim_cache size 0 max 16384 2023-09-10T01:31:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:31:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:31:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:31:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:31:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.493+0000 7f4b794be700 1 -- 172.21.15.44:0/3738177055 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4b5c0a0870 msgr2=0x7f4b5c0a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.493+0000 7f4b794be700 1 --2- 172.21.15.44:0/3738177055 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4b5c0a0870 0x7f4b5c0a2d30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4b64006fd0 tx=0x7f4b64009190 comp rx=0 tx=0).stop 2023-09-10T01:31:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.493+0000 7f4b794be700 1 -- 172.21.15.44:0/3738177055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b74073470 msgr2=0x7f4b74073920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:31:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.493+0000 7f4b794be700 1 --2- 172.21.15.44:0/3738177055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b74073470 0x7f4b74073920 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800b480 tx=0x7f4b6800b790 comp rx=0 tx=0).stop 2023-09-10T01:31:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.493+0000 7f4b794be700 1 -- 172.21.15.44:0/3738177055 shutdown_connections 2023-09-10T01:31:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.493+0000 7f4b794be700 1 --2- 172.21.15.44:0/3738177055 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4b5c0a0870 0x7f4b5c0a2d30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.493+0000 7f4b794be700 1 --2- 172.21.15.44:0/3738177055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b74073470 0x7f4b74073920 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.493+0000 7f4b794be700 1 --2- 172.21.15.44:0/3738177055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b7410ed50 0x7f4b740785e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.493+0000 7f4b794be700 1 --2- 172.21.15.44:0/3738177055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b74119de0 0x7f4b74078b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:31:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.493+0000 7f4b794be700 1 -- 172.21.15.44:0/3738177055 >> 172.21.15.44:0/3738177055 conn(0x7f4b7406c6c0 msgr2=0x7f4b74117dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:31:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.494+0000 7f4b794be700 1 -- 172.21.15.44:0/3738177055 shutdown_connections 2023-09-10T01:31:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:31:47.494+0000 7f4b794be700 1 -- 172.21.15.44:0/3738177055 wait complete. 2023-09-10T01:31:47.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:47 smithi099 ceph-mon[102692]: pgmap v1664: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.3 MiB/s wr, 976 op/s 2023-09-10T01:31:48.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:47 smithi131 ceph-mon[103713]: pgmap v1664: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.3 MiB/s wr, 976 op/s 2023-09-10T01:31:48.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:47 smithi044 ceph-mon[98676]: pgmap v1664: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.3 MiB/s wr, 976 op/s 2023-09-10T01:31:49.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:49 smithi099 ceph-mon[102692]: pgmap v1665: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.5 MiB/s wr, 813 op/s 2023-09-10T01:31:50.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:49 smithi131 ceph-mon[103713]: pgmap v1665: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.5 MiB/s wr, 813 op/s 2023-09-10T01:31:50.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:49 smithi044 ceph-mon[98676]: pgmap v1665: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.5 MiB/s wr, 813 op/s 2023-09-10T01:31:51.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 645.0 s, 279.6 tps, lat 360.026 ms stddev 150.521 2023-09-10T01:31:51.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:51 smithi099 ceph-mon[102692]: pgmap v1666: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.2 MiB/s wr, 744 op/s 2023-09-10T01:31:52.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:51 smithi131 ceph-mon[103713]: pgmap v1666: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.2 MiB/s wr, 744 op/s 2023-09-10T01:31:52.065 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:51 smithi044 ceph-mon[98676]: pgmap v1666: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.2 MiB/s wr, 744 op/s 2023-09-10T01:31:53.859 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:53 smithi099 ceph-mon[102692]: pgmap v1667: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.8 MiB/s wr, 1.04k op/s 2023-09-10T01:31:54.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:53 smithi131 ceph-mon[103713]: pgmap v1667: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.8 MiB/s wr, 1.04k op/s 2023-09-10T01:31:54.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:53 smithi044 ceph-mon[98676]: pgmap v1667: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.8 MiB/s wr, 1.04k op/s 2023-09-10T01:31:55.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:54 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:55.061 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:54 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:54 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:31:56.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:55 smithi131 ceph-mon[103713]: pgmap v1668: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.4 MiB/s wr, 781 op/s 2023-09-10T01:31:56.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:55 smithi044 ceph-mon[98676]: pgmap v1668: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.4 MiB/s wr, 781 op/s 2023-09-10T01:31:56.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:55 smithi099 ceph-mon[102692]: pgmap v1668: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.4 MiB/s wr, 781 op/s 2023-09-10T01:31:56.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 650.0 s, 275.4 tps, lat 344.431 ms stddev 218.111 2023-09-10T01:31:57.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:57 smithi099 ceph-mon[102692]: pgmap v1669: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 1.01k op/s 2023-09-10T01:31:58.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:57 smithi131 ceph-mon[103713]: pgmap v1669: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 1.01k op/s 2023-09-10T01:31:58.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:57 smithi044 ceph-mon[98676]: pgmap v1669: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 1.01k op/s 2023-09-10T01:32:00.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:59 smithi131 ceph-mon[103713]: pgmap v1670: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 821 op/s 2023-09-10T01:32:00.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:31:59 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:32:00.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:59 smithi044 ceph-mon[98676]: pgmap v1670: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 821 op/s 2023-09-10T01:32:00.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:31:59 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:32:00.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:59 smithi099 ceph-mon[102692]: pgmap v1670: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 821 op/s 2023-09-10T01:32:00.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:31:59 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:32:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.767+0000 7f07c6dd5700 1 -- 172.21.15.44:0/3039105513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07c0119de0 msgr2=0x7f07c011c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.767+0000 7f07c6dd5700 1 --2- 172.21.15.44:0/3039105513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07c0119de0 0x7f07c011c1d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f07b40098f0 tx=0x7f07b4009c00 comp rx=0 tx=0).stop 2023-09-10T01:32:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.774+0000 7f07c6dd5700 1 -- 172.21.15.44:0/3039105513 shutdown_connections 2023-09-10T01:32:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.774+0000 7f07c6dd5700 1 --2- 172.21.15.44:0/3039105513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07c0119de0 0x7f07c011c1d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.774+0000 7f07c6dd5700 1 --2- 172.21.15.44:0/3039105513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c010ed50 0x7f07c0107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.774+0000 7f07c6dd5700 1 --2- 172.21.15.44:0/3039105513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07c010f1c0 0x7f07c0107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.774+0000 7f07c6dd5700 1 -- 172.21.15.44:0/3039105513 >> 172.21.15.44:0/3039105513 conn(0x7f07c006c6c0 msgr2=0x7f07c006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.792+0000 7f07c6dd5700 1 -- 172.21.15.44:0/3039105513 shutdown_connections 2023-09-10T01:32:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.803+0000 7f07c6dd5700 1 -- 172.21.15.44:0/3039105513 wait complete. 2023-09-10T01:32:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.804+0000 7f07c6dd5700 1 Processor -- start 2023-09-10T01:32:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.804+0000 7f07c6dd5700 1 -- start start 2023-09-10T01:32:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.805+0000 7f07c6dd5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07c010ed50 0x7f07c01194b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.805+0000 7f07c6dd5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07c010f1c0 0x7f07c0113280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.805+0000 7f07c6dd5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c0119de0 0x7f07c01137c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.805+0000 7f07c6dd5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c0113dd0 con 0x7f07c010f1c0 2023-09-10T01:32:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.805+0000 7f07c6dd5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c0113f40 con 0x7f07c0119de0 2023-09-10T01:32:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.805+0000 7f07c6dd5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c0114100 con 0x7f07c010ed50 2023-09-10T01:32:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.809+0000 7f07c5372700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c0119de0 0x7f07c01137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.809+0000 7f07c5372700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c0119de0 0x7f07c01137c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34104/0 (socket says 172.21.15.44:34104) 2023-09-10T01:32:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.809+0000 7f07c5372700 1 -- 172.21.15.44:0/3868551796 learned_addr learned my addr 172.21.15.44:0/3868551796 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.809+0000 7f07bffff700 1 --2- 172.21.15.44:0/3868551796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07c010f1c0 0x7f07c0113280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.809+0000 7f07c4b71700 1 --2- 172.21.15.44:0/3868551796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07c010ed50 0x7f07c01194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.809+0000 7f07bffff700 1 -- 172.21.15.44:0/3868551796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07c010ed50 msgr2=0x7f07c01194b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.809+0000 7f07bffff700 1 --2- 172.21.15.44:0/3868551796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07c010ed50 0x7f07c01194b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.809+0000 7f07bffff700 1 -- 172.21.15.44:0/3868551796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c0119de0 msgr2=0x7f07c01137c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.809+0000 7f07bffff700 1 --2- 172.21.15.44:0/3868551796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c0119de0 0x7f07c01137c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.809+0000 7f07bffff700 1 -- 172.21.15.44:0/3868551796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b4009520 con 0x7f07c010f1c0 2023-09-10T01:32:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.817+0000 7f07bffff700 1 --2- 172.21.15.44:0/3868551796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07c010f1c0 0x7f07c0113280 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f07b800eee0 tx=0x7f07b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.817+0000 7f07bdffb700 1 -- 172.21.15.44:0/3868551796 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b800cad0 con 0x7f07c010f1c0 2023-09-10T01:32:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.817+0000 7f07bdffb700 1 -- 172.21.15.44:0/3868551796 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07b800f040 con 0x7f07c010f1c0 2023-09-10T01:32:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.817+0000 7f07bdffb700 1 -- 172.21.15.44:0/3868551796 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8014480 con 0x7f07c010f1c0 2023-09-10T01:32:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.817+0000 7f07c6dd5700 1 -- 172.21.15.44:0/3868551796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c01ba5d0 con 0x7f07c010f1c0 2023-09-10T01:32:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.817+0000 7f07c6dd5700 1 -- 172.21.15.44:0/3868551796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c01baa90 con 0x7f07c010f1c0 2023-09-10T01:32:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.822+0000 7f07d2f9bb80 1 Processor -- start 2023-09-10T01:32:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.823+0000 7f07bdffb700 1 -- 172.21.15.44:0/3868551796 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f07b80075c0 con 0x7f07c010f1c0 2023-09-10T01:32:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.823+0000 7f07d2f9bb80 1 -- start start 2023-09-10T01:32:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.823+0000 7f07d2f9bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637374871a0 0x563737487580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.823+0000 7f07d2f9bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481300 0x563737481780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.823+0000 7f07d2f9bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563737481cc0 0x56373748df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.823+0000 7f07d2f9bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637373da6d0 con 0x563737481cc0 2023-09-10T01:32:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.823+0000 7f07d2f9bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637373d7ff0 con 0x563737481300 2023-09-10T01:32:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.823+0000 7f07d2f9bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637373d38e0 con 0x5637374871a0 2023-09-10T01:32:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.824+0000 7f07c5372700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637374871a0 0x563737487580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.824+0000 7f07c5372700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637374871a0 0x563737487580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39250/0 (socket says 172.21.15.44:39250) 2023-09-10T01:32:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.824+0000 7f07c5372700 1 -- 172.21.15.44:0/402021670 learned_addr learned my addr 172.21.15.44:0/402021670 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.824+0000 7f07bdffb700 1 --2- 172.21.15.44:0/3868551796 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f07a80a0900 0x7f07a80a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.825+0000 7f07c5372700 1 -- 172.21.15.44:0/402021670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481300 msgr2=0x563737481780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:32:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.825+0000 7f07c5372700 1 --2- 172.21.15.44:0/402021670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481300 0x563737481780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.825+0000 7f07c5372700 1 -- 172.21.15.44:0/402021670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563737481cc0 msgr2=0x56373748df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.825+0000 7f07c5372700 1 --2- 172.21.15.44:0/402021670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563737481cc0 0x56373748df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.825+0000 7f07c5372700 1 -- 172.21.15.44:0/402021670 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637371ffd80 con 0x5637374871a0 2023-09-10T01:32:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.825+0000 7f07bdffb700 1 -- 172.21.15.44:0/3868551796 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f07b80cc650 con 0x7f07c010f1c0 2023-09-10T01:32:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.825+0000 7f07c5372700 1 --2- 172.21.15.44:0/402021670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637374871a0 0x563737487580 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f07b4005d50 tx=0x7f07b4005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=887bd38a510d1b6e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.829+0000 7f07c5372700 1 --2- 172.21.15.44:0/3868551796 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f07a80a0900 0x7f07a80a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.829+0000 7f07a6ffd700 1 -- 172.21.15.44:0/402021670 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b400f810 con 0x5637374871a0 2023-09-10T01:32:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.829+0000 7f07a6ffd700 1 -- 172.21.15.44:0/402021670 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07b400f970 con 0x5637374871a0 2023-09-10T01:32:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.829+0000 7f07a6ffd700 1 -- 172.21.15.44:0/402021670 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b40046d0 con 0x5637374871a0 2023-09-10T01:32:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.830+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/402021670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637374871a0 msgr2=0x563737487580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.830+0000 7f07c5372700 1 --2- 172.21.15.44:0/3868551796 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f07a80a0900 0x7f07a80a2dc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f07b4026000 tx=0x7f07b4004980 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.830+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/402021670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637374871a0 0x563737487580 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f07b4005d50 tx=0x7f07b4005d80 comp rx=0 tx=0).stop 2023-09-10T01:32:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.831+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/402021670 shutdown_connections 2023-09-10T01:32:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.831+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/402021670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563737481cc0 0x56373748df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.831+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/402021670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637374871a0 0x563737487580 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.831+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/402021670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481300 0x563737481780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.831+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/402021670 >> 172.21.15.44:0/402021670 conn(0x5637373f6030 msgr2=0x56373747c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.837+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/402021670 shutdown_connections 2023-09-10T01:32:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.837+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/402021670 wait complete. 2023-09-10T01:32:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.837+0000 7f07d2f9bb80 1 -- start start 2023-09-10T01:32:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.838+0000 7f07d2f9bb80 1 Processor -- start 2023-09-10T01:32:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.838+0000 7f07d2f9bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563737481300 0x56373748b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.838+0000 7f07d2f9bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481cc0 0x56373748b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.838+0000 7f07d2f9bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637374871a0 0x56373748bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.838+0000 7f07d2f9bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563737371b10 con 0x5637374871a0 2023-09-10T01:32:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.838+0000 7f07d2f9bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563737371810 con 0x563737481cc0 2023-09-10T01:32:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.838+0000 7f07d2f9bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637372558c0 con 0x563737481300 2023-09-10T01:32:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.838+0000 7f07d2f9bb80 10 client.0 _ll_register_callbacks cb 0x5637372bba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:32:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.839+0000 7f07bffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481cc0 0x56373748b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.839+0000 7f07bffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481cc0 0x56373748b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34114/0 (socket says 172.21.15.44:34114) 2023-09-10T01:32:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.839+0000 7f07bffff700 1 -- 172.21.15.44:0/478715684 learned_addr learned my addr 172.21.15.44:0/478715684 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.839+0000 7f07c4b71700 1 --2- 172.21.15.44:0/478715684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563737481300 0x56373748b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.844+0000 7f07c5372700 1 --2- 172.21.15.44:0/478715684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637374871a0 0x56373748bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.844+0000 7f07bffff700 1 -- 172.21.15.44:0/478715684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563737481300 msgr2=0x56373748b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.844+0000 7f07bffff700 1 --2- 172.21.15.44:0/478715684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563737481300 0x56373748b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.844+0000 7f07bffff700 1 -- 172.21.15.44:0/478715684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637374871a0 msgr2=0x56373748bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.844+0000 7f07bffff700 1 --2- 172.21.15.44:0/478715684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637374871a0 0x56373748bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.844+0000 7f07bffff700 1 -- 172.21.15.44:0/478715684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c011cb00 con 0x563737481cc0 2023-09-10T01:32:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.852+0000 7f07bffff700 1 --2- 172.21.15.44:0/478715684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481cc0 0x56373748b690 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f07b8095c10 tx=0x7f07b80992d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9226d1991c7c42af server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.852+0000 7f079ffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:32:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.852+0000 7f079ffff700 1 -- 172.21.15.44:0/478715684 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b800ccc0 con 0x563737481cc0 2023-09-10T01:32:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.852+0000 7f079ffff700 1 -- 172.21.15.44:0/478715684 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f07b8099e10 con 0x563737481cc0 2023-09-10T01:32:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.852+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/478715684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56373732b490 con 0x563737481cc0 2023-09-10T01:32:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.853+0000 7f079ffff700 1 -- 172.21.15.44:0/478715684 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=186) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07b8099710 con 0x563737481cc0 2023-09-10T01:32:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.853+0000 7f07d2f9bb80 10 client.30161 fetch_fsmap learned FSMap version 186 2023-09-10T01:32:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.853+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/478715684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=186,monmap=4+}) v3 -- 0x5637371e8c00 con 0x563737481cc0 2023-09-10T01:32:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.853+0000 7f079ffff700 1 -- 172.21.15.44:0/478715684 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07b80b5020 con 0x563737481cc0 2023-09-10T01:32:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.853+0000 7f079ffff700 1 -- 172.21.15.44:0/478715684 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 186) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f07b80cc240 con 0x563737481cc0 2023-09-10T01:32:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.853+0000 7f07d2f9bb80 10 client.30161 fetch_fsmap finished waiting for FSMap version 186 2023-09-10T01:32:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.853+0000 7f07d2f9bb80 10 client.30161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:32:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.853+0000 7f07d2f9bb80 20 client.30161 populate_metadata read hostname 'smithi044' 2023-09-10T01:32:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.853+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/478715684 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5637374bd3c0 0x5637374bf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.853+0000 7f07d2f9bb80 4 client.30161 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:32:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.853+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/478715684 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637373e8840 con 0x5637374bd3c0 2023-09-10T01:32:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.854+0000 7f07c4b71700 1 --2- 172.21.15.44:0/478715684 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5637374bd3c0 0x5637374bf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.855+0000 7f07c4b71700 1 --2- 172.21.15.44:0/478715684 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5637374bd3c0 0x5637374bf7e0 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=118cc3959ac65d5e server_cookie=97784c2957c1a769 in_seq=0 out_seq=0 2023-09-10T01:32:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.855+0000 7f079ffff700 10 client.30161 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.856+0000 7f079ffff700 1 -- 172.21.15.44:0/478715684 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5637373e8840 con 0x5637374bd3c0 2023-09-10T01:32:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.856+0000 7f079ffff700 10 client.30161 handle_command_reply: tid=0 2023-09-10T01:32:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.856+0000 7f07d2f9bb80 1 client.30161 shutdown 2023-09-10T01:32:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.859+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/478715684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481cc0 msgr2=0x56373748b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.859+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/478715684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481cc0 0x56373748b690 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f07b8095c10 tx=0x7f07b80992d0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.869+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/478715684 shutdown_connections 2023-09-10T01:32:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.869+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/478715684 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5637374bd3c0 0x5637374bf7e0 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.869+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/478715684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637374871a0 0x56373748bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.869+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/478715684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563737481300 0x56373748b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.869+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/478715684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481cc0 0x56373748b690 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.869+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/478715684 >> 172.21.15.44:0/478715684 conn(0x5637373f6030 msgr2=0x56373747b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.872+0000 7f079ffff700 0 client.30161 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.894+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/478715684 shutdown_connections 2023-09-10T01:32:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.901+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/478715684 wait complete. 2023-09-10T01:32:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.901+0000 7f07d2f9bb80 20 client.30161 trim_cache size 0 max 16384 2023-09-10T01:32:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.938+0000 7f07d2f9bb80 1 Processor -- start 2023-09-10T01:32:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.938+0000 7f07d2f9bb80 1 -- start start 2023-09-10T01:32:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.938+0000 7f07d2f9bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481300 0x563737497a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.938+0000 7f07d2f9bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563737481cc0 0x56373748a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.938+0000 7f07d2f9bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637374871a0 0x563737492470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.938+0000 7f07d2f9bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637374820a0 con 0x5637374871a0 2023-09-10T01:32:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.938+0000 7f07d2f9bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637374baaa0 con 0x563737481300 2023-09-10T01:32:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.938+0000 7f07d2f9bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563737478fc0 con 0x563737481cc0 2023-09-10T01:32:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.943+0000 7f07bffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563737481cc0 0x56373748a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.943+0000 7f07c4b71700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481300 0x563737497a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.943+0000 7f07bffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563737481cc0 0x56373748a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39266/0 (socket says 172.21.15.44:39266) 2023-09-10T01:32:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.943+0000 7f07bffff700 1 -- 172.21.15.44:0/198819386 learned_addr learned my addr 172.21.15.44:0/198819386 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.944+0000 7f07bffff700 1 -- 172.21.15.44:0/198819386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481300 msgr2=0x563737497a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.944+0000 7f07bffff700 1 --2- 172.21.15.44:0/198819386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481300 0x563737497a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.944+0000 7f07bffff700 1 -- 172.21.15.44:0/198819386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637374871a0 msgr2=0x563737492470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.944+0000 7f07bffff700 1 --2- 172.21.15.44:0/198819386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637374871a0 0x563737492470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.944+0000 7f07bffff700 1 -- 172.21.15.44:0/198819386 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56373748acc0 con 0x563737481cc0 2023-09-10T01:32:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.948+0000 7f07bffff700 1 --2- 172.21.15.44:0/198819386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563737481cc0 0x56373748a400 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f07b8096180 tx=0x7f07b8097cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6150db9581bea5c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.960+0000 7f07a77fe700 1 -- 172.21.15.44:0/198819386 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b800ccc0 con 0x563737481cc0 2023-09-10T01:32:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.960+0000 7f07a77fe700 1 -- 172.21.15.44:0/198819386 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07b80b5070 con 0x563737481cc0 2023-09-10T01:32:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.960+0000 7f07a77fe700 1 -- 172.21.15.44:0/198819386 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b809d540 con 0x563737481cc0 2023-09-10T01:32:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.966+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/198819386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563737481cc0 msgr2=0x56373748a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.966+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/198819386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563737481cc0 0x56373748a400 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f07b8096180 tx=0x7f07b8097cc0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.970+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/198819386 shutdown_connections 2023-09-10T01:32:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.970+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/198819386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637374871a0 0x563737492470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.970+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/198819386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481300 0x563737497a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.970+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/198819386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563737481cc0 0x56373748a400 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.970+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/198819386 >> 172.21.15.44:0/198819386 conn(0x5637373f6030 msgr2=0x563737486170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.982+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/198819386 shutdown_connections 2023-09-10T01:32:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.985+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/198819386 wait complete. 2023-09-10T01:32:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.985+0000 7f07d2f9bb80 1 -- start start 2023-09-10T01:32:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.985+0000 7f07d2f9bb80 1 Processor -- start 2023-09-10T01:32:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.986+0000 7f07d2f9bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481300 0x5637374903e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.986+0000 7f07d2f9bb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563737481cc0 0x563737490920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.986+0000 7f07d2f9bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637374871a0 0x56373750a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.986+0000 7f07d2f9bb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56373750a6c0 con 0x563737481cc0 2023-09-10T01:32:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.986+0000 7f07d2f9bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56373750a830 con 0x563737481300 2023-09-10T01:32:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.986+0000 7f07d2f9bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56373750a9f0 con 0x5637374871a0 2023-09-10T01:32:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.986+0000 7f07d2f9bb80 10 client.0 _ll_register_callbacks cb 0x5637372bba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:32:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.994+0000 7f07bffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563737481cc0 0x563737490920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.994+0000 7f07bffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563737481cc0 0x563737490920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60624/0 (socket says 172.21.15.44:60624) 2023-09-10T01:32:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.994+0000 7f07bffff700 1 -- 172.21.15.44:0/3138640464 learned_addr learned my addr 172.21.15.44:0/3138640464 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.995+0000 7f07c4b71700 1 --2- 172.21.15.44:0/3138640464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481300 0x5637374903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.995+0000 7f07c5372700 1 --2- 172.21.15.44:0/3138640464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637374871a0 0x56373750a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.998+0000 7f07bffff700 1 -- 172.21.15.44:0/3138640464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637374871a0 msgr2=0x56373750a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.998+0000 7f07bffff700 1 --2- 172.21.15.44:0/3138640464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637374871a0 0x56373750a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.998+0000 7f07bffff700 1 -- 172.21.15.44:0/3138640464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481300 msgr2=0x5637374903e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.998+0000 7f07bffff700 1 --2- 172.21.15.44:0/3138640464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481300 0x5637374903e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:00.998+0000 7f07bffff700 1 -- 172.21.15.44:0/3138640464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637371e8c00 con 0x563737481cc0 2023-09-10T01:32:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.024+0000 7f07bffff700 1 --2- 172.21.15.44:0/3138640464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563737481cc0 0x563737490920 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f07b80a1250 tx=0x7f07b8007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d09e4e5b4ffa4fd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.025+0000 7f079ffff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:32:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.025+0000 7f079ffff700 1 -- 172.21.15.44:0/3138640464 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b800ccc0 con 0x563737481cc0 2023-09-10T01:32:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.025+0000 7f079ffff700 1 -- 172.21.15.44:0/3138640464 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f07b80b6030 con 0x563737481cc0 2023-09-10T01:32:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.025+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/3138640464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07b400f970 con 0x563737481cc0 2023-09-10T01:32:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.025+0000 7f079ffff700 1 -- 172.21.15.44:0/3138640464 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=186) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07b8099e10 con 0x563737481cc0 2023-09-10T01:32:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.025+0000 7f07d2f9bb80 10 client.20520 fetch_fsmap learned FSMap version 186 2023-09-10T01:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.025+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/3138640464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=186,monmap=4+}) v3 -- 0x5637372261f0 con 0x563737481cc0 2023-09-10T01:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.025+0000 7f079ffff700 1 -- 172.21.15.44:0/3138640464 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07b400f970 con 0x563737481cc0 2023-09-10T01:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.026+0000 7f079ffff700 1 -- 172.21.15.44:0/3138640464 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 186) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f07b80cc240 con 0x563737481cc0 2023-09-10T01:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.026+0000 7f07d2f9bb80 10 client.20520 fetch_fsmap finished waiting for FSMap version 186 2023-09-10T01:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.026+0000 7f07d2f9bb80 10 client.20520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:32:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.026+0000 7f07d2f9bb80 20 client.20520 populate_metadata read hostname 'smithi044' 2023-09-10T01:32:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.026+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/3138640464 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563737511ca0 0x563737514090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.026+0000 7f07d2f9bb80 4 client.20520 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:32:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.026+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/3138640464 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5637373cc8c0 con 0x563737511ca0 2023-09-10T01:32:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.027+0000 7f07c4b71700 1 --2- 172.21.15.44:0/3138640464 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563737511ca0 0x563737514090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.028+0000 7f07c4b71700 1 --2- 172.21.15.44:0/3138640464 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563737511ca0 0x563737514090 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70da56085927848 server_cookie=fec561c7261b31a3 in_seq=0 out_seq=0 2023-09-10T01:32:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.032+0000 7f079ffff700 10 client.20520 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:01.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 655.0 s, 227.6 tps, lat 449.667 ms stddev 361.096 2023-09-10T01:32:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.269+0000 7f079ffff700 1 -- 172.21.15.44:0/3138640464 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5637373cc8c0 con 0x563737511ca0 2023-09-10T01:32:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.269+0000 7f079ffff700 10 client.20520 handle_command_reply: tid=0 2023-09-10T01:32:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.269+0000 7f07d2f9bb80 1 client.20520 shutdown 2023-09-10T01:32:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.269+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/3138640464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563737481cc0 msgr2=0x563737490920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.269+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/3138640464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563737481cc0 0x563737490920 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f07b80a1250 tx=0x7f07b8007a80 comp rx=0 tx=0).stop 2023-09-10T01:32:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.269+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/3138640464 shutdown_connections 2023-09-10T01:32:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.269+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/3138640464 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x563737511ca0 0x563737514090 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.269+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/3138640464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5637374871a0 0x56373750a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.269+0000 7f079ffff700 0 client.20520 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.269+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/3138640464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563737481300 0x5637374903e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.269+0000 7f07d2f9bb80 1 --2- 172.21.15.44:0/3138640464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563737481cc0 0x563737490920 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.269+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/3138640464 >> 172.21.15.44:0/3138640464 conn(0x5637373f6030 msgr2=0x5637374851f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.269+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/3138640464 shutdown_connections 2023-09-10T01:32:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.269+0000 7f07d2f9bb80 1 -- 172.21.15.44:0/3138640464 wait complete. 2023-09-10T01:32:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.269+0000 7f07d2f9bb80 20 client.20520 trim_cache size 0 max 16384 2023-09-10T01:32:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:32:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:32:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:32:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:32:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.270+0000 7f07c6dd5700 1 -- 172.21.15.44:0/3868551796 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f07a80a0900 msgr2=0x7f07a80a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.270+0000 7f07c6dd5700 1 --2- 172.21.15.44:0/3868551796 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f07a80a0900 0x7f07a80a2dc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f07b4026000 tx=0x7f07b4004980 comp rx=0 tx=0).stop 2023-09-10T01:32:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.270+0000 7f07c6dd5700 1 -- 172.21.15.44:0/3868551796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07c010f1c0 msgr2=0x7f07c0113280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.270+0000 7f07c6dd5700 1 --2- 172.21.15.44:0/3868551796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07c010f1c0 0x7f07c0113280 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f07b800eee0 tx=0x7f07b800c3a0 comp rx=0 tx=0).stop 2023-09-10T01:32:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.270+0000 7f07c6dd5700 1 -- 172.21.15.44:0/3868551796 shutdown_connections 2023-09-10T01:32:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.270+0000 7f07c6dd5700 1 --2- 172.21.15.44:0/3868551796 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f07a80a0900 0x7f07a80a2dc0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.270+0000 7f07c6dd5700 1 --2- 172.21.15.44:0/3868551796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c0119de0 0x7f07c01137c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.270+0000 7f07c6dd5700 1 --2- 172.21.15.44:0/3868551796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07c010ed50 0x7f07c01194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.270+0000 7f07c6dd5700 1 --2- 172.21.15.44:0/3868551796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07c010f1c0 0x7f07c0113280 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.270+0000 7f07c6dd5700 1 -- 172.21.15.44:0/3868551796 >> 172.21.15.44:0/3868551796 conn(0x7f07c006c6c0 msgr2=0x7f07c011b240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.270+0000 7f07c6dd5700 1 -- 172.21.15.44:0/3868551796 shutdown_connections 2023-09-10T01:32:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:01.270+0000 7f07c6dd5700 1 -- 172.21.15.44:0/3868551796 wait complete. 2023-09-10T01:32:02.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:01 smithi131 ceph-mon[103713]: pgmap v1671: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.4 MiB/s wr, 732 op/s 2023-09-10T01:32:02.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:01 smithi044 ceph-mon[98676]: pgmap v1671: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.4 MiB/s wr, 732 op/s 2023-09-10T01:32:02.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:01 smithi099 ceph-mon[102692]: pgmap v1671: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.4 MiB/s wr, 732 op/s 2023-09-10T01:32:04.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:03 smithi131 ceph-mon[103713]: pgmap v1672: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.2 MiB/s wr, 1.04k op/s 2023-09-10T01:32:04.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:03 smithi044 ceph-mon[98676]: pgmap v1672: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.2 MiB/s wr, 1.04k op/s 2023-09-10T01:32:04.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:03 smithi099 ceph-mon[102692]: pgmap v1672: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.2 MiB/s wr, 1.04k op/s 2023-09-10T01:32:06.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:05 smithi131 ceph-mon[103713]: pgmap v1673: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 770 op/s 2023-09-10T01:32:06.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:05 smithi044 ceph-mon[98676]: pgmap v1673: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 770 op/s 2023-09-10T01:32:06.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:05 smithi099 ceph-mon[102692]: pgmap v1673: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 770 op/s 2023-09-10T01:32:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 660.0 s, 242.8 tps, lat 356.895 ms stddev 434.408 2023-09-10T01:32:07.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:07 smithi099 ceph-mon[102692]: pgmap v1674: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.2 MiB/s wr, 912 op/s 2023-09-10T01:32:08.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:07 smithi131 ceph-mon[103713]: pgmap v1674: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.2 MiB/s wr, 912 op/s 2023-09-10T01:32:08.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:07 smithi044 ceph-mon[98676]: pgmap v1674: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.2 MiB/s wr, 912 op/s 2023-09-10T01:32:10.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:09 smithi131 ceph-mon[103713]: pgmap v1675: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.6 MiB/s wr, 711 op/s 2023-09-10T01:32:10.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:09 smithi131 ceph-mon[103713]: osdmap e199: 12 total, 12 up, 12 in 2023-09-10T01:32:10.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:09 smithi044 ceph-mon[98676]: pgmap v1675: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.6 MiB/s wr, 711 op/s 2023-09-10T01:32:10.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:09 smithi044 ceph-mon[98676]: osdmap e199: 12 total, 12 up, 12 in 2023-09-10T01:32:10.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:09 smithi099 ceph-mon[102692]: pgmap v1675: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.6 MiB/s wr, 711 op/s 2023-09-10T01:32:10.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:09 smithi099 ceph-mon[102692]: osdmap e199: 12 total, 12 up, 12 in 2023-09-10T01:32:11.188 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 665.0 s, 84.3 tps, lat 893.838 ms stddev 1108.038 2023-09-10T01:32:12.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:11 smithi044 ceph-mon[98676]: pgmap v1677: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 732 op/s 2023-09-10T01:32:12.056 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:11 smithi131 ceph-mon[103713]: pgmap v1677: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 732 op/s 2023-09-10T01:32:12.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:11 smithi099 ceph-mon[102692]: pgmap v1677: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 732 op/s 2023-09-10T01:32:14.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:13 smithi131 ceph-mon[103713]: pgmap v1678: 129 pgs: 17 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 66 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 521 op/s 2023-09-10T01:32:14.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:13 smithi044 ceph-mon[98676]: pgmap v1678: 129 pgs: 17 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 66 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 521 op/s 2023-09-10T01:32:14.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:13 smithi099 ceph-mon[102692]: pgmap v1678: 129 pgs: 17 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 66 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 521 op/s 2023-09-10T01:32:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.490+0000 7fccd503c700 1 -- 172.21.15.44:0/2480453856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccd0119de0 msgr2=0x7fccd011c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.490+0000 7fccd503c700 1 --2- 172.21.15.44:0/2480453856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccd0119de0 0x7fccd011c1d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fccc801c0d0 tx=0x7fccc801c3e0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.504+0000 7fccd503c700 1 -- 172.21.15.44:0/2480453856 shutdown_connections 2023-09-10T01:32:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.504+0000 7fccd503c700 1 --2- 172.21.15.44:0/2480453856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccd0119de0 0x7fccd011c1d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.504+0000 7fccd503c700 1 --2- 172.21.15.44:0/2480453856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd010ed50 0x7fccd0107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.504+0000 7fccd503c700 1 --2- 172.21.15.44:0/2480453856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccd010f1c0 0x7fccd0107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.504+0000 7fccd503c700 1 -- 172.21.15.44:0/2480453856 >> 172.21.15.44:0/2480453856 conn(0x7fccd006c6c0 msgr2=0x7fccd006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.509+0000 7fccd503c700 1 -- 172.21.15.44:0/2480453856 shutdown_connections 2023-09-10T01:32:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.518+0000 7fccd503c700 1 -- 172.21.15.44:0/2480453856 wait complete. 2023-09-10T01:32:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.519+0000 7fccd503c700 1 Processor -- start 2023-09-10T01:32:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.526+0000 7fccd503c700 1 -- start start 2023-09-10T01:32:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.526+0000 7fccd503c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccd010ed50 0x7fccd0115060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.526+0000 7fccd503c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd010f1c0 0x7fccd01155a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.526+0000 7fccd503c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccd0119de0 0x7fccd01a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.526+0000 7fccd503c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd01a30b0 con 0x7fccd010ed50 2023-09-10T01:32:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.526+0000 7fccd503c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd01a3220 con 0x7fccd0119de0 2023-09-10T01:32:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.526+0000 7fccd503c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd01a33e0 con 0x7fccd010f1c0 2023-09-10T01:32:14.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.532+0000 7fccce59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd010f1c0 0x7fccd01155a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:14.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.532+0000 7fccce59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd010f1c0 0x7fccd01155a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39286/0 (socket says 172.21.15.44:39286) 2023-09-10T01:32:14.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.532+0000 7fccce59c700 1 -- 172.21.15.44:0/1591573170 learned_addr learned my addr 172.21.15.44:0/1591573170 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:14.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.533+0000 7fccced9d700 1 --2- 172.21.15.44:0/1591573170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccd010ed50 0x7fccd0115060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:14.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.533+0000 7fccced9d700 1 -- 172.21.15.44:0/1591573170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd010f1c0 msgr2=0x7fccd01155a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:14.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.533+0000 7fccced9d700 1 --2- 172.21.15.44:0/1591573170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd010f1c0 0x7fccd01155a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.533+0000 7fccced9d700 1 -- 172.21.15.44:0/1591573170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccd0119de0 msgr2=0x7fccd01a2a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T01:32:14.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.533+0000 7fccced9d700 1 --2- 172.21.15.44:0/1591573170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccd0119de0 0x7fccd01a2a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.533+0000 7fccced9d700 1 -- 172.21.15.44:0/1591573170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc801ad10 con 0x7fccd010ed50 2023-09-10T01:32:14.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.533+0000 7fccced9d700 1 --2- 172.21.15.44:0/1591573170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccd010ed50 0x7fccd0115060 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fccc4006fd0 tx=0x7fccc400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:14.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.534+0000 7fccb7fff700 1 -- 172.21.15.44:0/1591573170 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4011830 con 0x7fccd010ed50 2023-09-10T01:32:14.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.534+0000 7fccb7fff700 1 -- 172.21.15.44:0/1591573170 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fccc4011e20 con 0x7fccd010ed50 2023-09-10T01:32:14.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.534+0000 7fccb7fff700 1 -- 172.21.15.44:0/1591573170 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4010d40 con 0x7fccd010ed50 2023-09-10T01:32:14.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.534+0000 7fccd503c700 1 -- 172.21.15.44:0/1591573170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd01a36c0 con 0x7fccd010ed50 2023-09-10T01:32:14.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.534+0000 7fccd503c700 1 -- 172.21.15.44:0/1591573170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd01a9d40 con 0x7fccd010ed50 2023-09-10T01:32:14.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.537+0000 7fccb7fff700 1 -- 172.21.15.44:0/1591573170 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fccc4011990 con 0x7fccd010ed50 2023-09-10T01:32:14.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.538+0000 7fccb7fff700 1 --2- 172.21.15.44:0/1591573170 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fccb80a0870 0x7fccb80a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.538+0000 7fccb7fff700 1 -- 172.21.15.44:0/1591573170 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fccc40cb910 con 0x7fccd010ed50 2023-09-10T01:32:14.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.539+0000 7fcce1202b80 1 Processor -- start 2023-09-10T01:32:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.539+0000 7fcce1202b80 1 -- start start 2023-09-10T01:32:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.539+0000 7fcce1202b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d2f1a0 0x55aa90d2f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.539+0000 7fcce1202b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa90d29300 0x55aa90d29780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.539+0000 7fcce1202b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d29cc0 0x55aa90d35f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.539+0000 7fcce1202b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa90c826d0 con 0x55aa90d29cc0 2023-09-10T01:32:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.539+0000 7fcce1202b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa90c7fff0 con 0x55aa90d2f1a0 2023-09-10T01:32:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.539+0000 7fcce1202b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa90c7b8e0 con 0x55aa90d29300 2023-09-10T01:32:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.540+0000 7fcccf59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d29cc0 0x55aa90d35f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.540+0000 7fcccf59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d29cc0 0x55aa90d35f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60640/0 (socket says 172.21.15.44:60640) 2023-09-10T01:32:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.540+0000 7fcccf59e700 1 -- 172.21.15.44:0/3938580291 learned_addr learned my addr 172.21.15.44:0/3938580291 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.540+0000 7fcccf59e700 1 -- 172.21.15.44:0/3938580291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa90d29300 msgr2=0x55aa90d29780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:32:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.540+0000 7fcccf59e700 1 --2- 172.21.15.44:0/3938580291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa90d29300 0x55aa90d29780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.540+0000 7fcccf59e700 1 -- 172.21.15.44:0/3938580291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d2f1a0 msgr2=0x55aa90d2f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.540+0000 7fcccf59e700 1 --2- 172.21.15.44:0/3938580291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d2f1a0 0x55aa90d2f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.540+0000 7fcccf59e700 1 -- 172.21.15.44:0/3938580291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa90aa7d80 con 0x55aa90d29cc0 2023-09-10T01:32:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.541+0000 7fcccf59e700 1 --2- 172.21.15.44:0/3938580291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d29cc0 0x55aa90d35f30 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fccc8019ca0 tx=0x7fccc8019cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b54c68228173b72 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.544+0000 7fccb57fa700 1 -- 172.21.15.44:0/3938580291 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc801f040 con 0x55aa90d29cc0 2023-09-10T01:32:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.544+0000 7fccb57fa700 1 -- 172.21.15.44:0/3938580291 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fccc801a2d0 con 0x55aa90d29cc0 2023-09-10T01:32:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.544+0000 7fccb57fa700 1 -- 172.21.15.44:0/3938580291 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc802c6c0 con 0x55aa90d29cc0 2023-09-10T01:32:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.548+0000 7fcce1202b80 1 -- 172.21.15.44:0/3938580291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d29cc0 msgr2=0x55aa90d35f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.548+0000 7fcce1202b80 1 --2- 172.21.15.44:0/3938580291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d29cc0 0x55aa90d35f30 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fccc8019ca0 tx=0x7fccc8019cd0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.548+0000 7fcce1202b80 1 -- 172.21.15.44:0/3938580291 shutdown_connections 2023-09-10T01:32:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.548+0000 7fcce1202b80 1 --2- 172.21.15.44:0/3938580291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d29cc0 0x55aa90d35f30 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.548+0000 7fcce1202b80 1 --2- 172.21.15.44:0/3938580291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d2f1a0 0x55aa90d2f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.548+0000 7fcce1202b80 1 --2- 172.21.15.44:0/3938580291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa90d29300 0x55aa90d29780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.548+0000 7fcce1202b80 1 -- 172.21.15.44:0/3938580291 >> 172.21.15.44:0/3938580291 conn(0x55aa90c9e030 msgr2=0x55aa90d24560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.554+0000 7fccce59c700 1 --2- 172.21.15.44:0/1591573170 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fccb80a0870 0x7fccb80a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.559+0000 7fcce1202b80 1 -- 172.21.15.44:0/3938580291 shutdown_connections 2023-09-10T01:32:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.560+0000 7fcce1202b80 1 -- 172.21.15.44:0/3938580291 wait complete. 2023-09-10T01:32:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.560+0000 7fcce1202b80 1 -- start start 2023-09-10T01:32:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.560+0000 7fcce1202b80 1 Processor -- start 2023-09-10T01:32:14.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.561+0000 7fcce1202b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 0x55aa90d33290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.561+0000 7fcce1202b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa90d29cc0 0x55aa90d337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.561+0000 7fcce1202b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d33d10 0x55aa90d3f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.561+0000 7fcce1202b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa90c19b10 con 0x55aa90d33d10 2023-09-10T01:32:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.561+0000 7fcce1202b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa90c19810 con 0x55aa90d29300 2023-09-10T01:32:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.561+0000 7fcce1202b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa90afd8c0 con 0x55aa90d29cc0 2023-09-10T01:32:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.561+0000 7fcce1202b80 10 client.0 _ll_register_callbacks cb 0x55aa90b63a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:32:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.562+0000 7fccced9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 0x55aa90d33290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.562+0000 7fccced9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 0x55aa90d33290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34156/0 (socket says 172.21.15.44:34156) 2023-09-10T01:32:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.562+0000 7fccced9d700 1 -- 172.21.15.44:0/2611360542 learned_addr learned my addr 172.21.15.44:0/2611360542 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.562+0000 7fccced9d700 1 -- 172.21.15.44:0/2611360542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa90d29cc0 msgr2=0x55aa90d337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.562+0000 7fccced9d700 1 --2- 172.21.15.44:0/2611360542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa90d29cc0 0x55aa90d337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.562+0000 7fccced9d700 1 -- 172.21.15.44:0/2611360542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d33d10 msgr2=0x55aa90d3f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.562+0000 7fccced9d700 1 --2- 172.21.15.44:0/2611360542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d33d10 0x55aa90d3f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.562+0000 7fccced9d700 1 -- 172.21.15.44:0/2611360542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd011cb00 con 0x55aa90d29300 2023-09-10T01:32:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.562+0000 7fccced9d700 1 --2- 172.21.15.44:0/2611360542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 0x55aa90d33290 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fccc40c9210 tx=0x7fccc4098670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c97e4b2a5efbd8f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.563+0000 7fccb2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:32:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.563+0000 7fccb2ff5700 1 -- 172.21.15.44:0/2611360542 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4010480 con 0x55aa90d29300 2023-09-10T01:32:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.563+0000 7fccb2ff5700 1 -- 172.21.15.44:0/2611360542 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(199..199 src has 1..199) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fccc4011c40 con 0x55aa90d29300 2023-09-10T01:32:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.563+0000 7fcce1202b80 1 -- 172.21.15.44:0/2611360542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa90bd3490 con 0x55aa90d29300 2023-09-10T01:32:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.563+0000 7fccb2ff5700 1 -- 172.21.15.44:0/2611360542 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=186) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccc4099180 con 0x55aa90d29300 2023-09-10T01:32:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.563+0000 7fcce1202b80 10 client.30173 fetch_fsmap learned FSMap version 186 2023-09-10T01:32:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.563+0000 7fcce1202b80 1 -- 172.21.15.44:0/2611360542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=186,monmap=4+}) v3 -- 0x55aa90a90c00 con 0x55aa90d29300 2023-09-10T01:32:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.563+0000 7fccb2ff5700 1 -- 172.21.15.44:0/2611360542 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aa90bd3490 con 0x55aa90d29300 2023-09-10T01:32:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.563+0000 7fccb2ff5700 1 -- 172.21.15.44:0/2611360542 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 186) v1 ==== 7371+0+0 (secure 0 0 0) 0x7fccc4098ce0 con 0x55aa90d29300 2023-09-10T01:32:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.563+0000 7fcce1202b80 10 client.30173 fetch_fsmap finished waiting for FSMap version 186 2023-09-10T01:32:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.563+0000 7fcce1202b80 10 client.30173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:32:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.563+0000 7fcce1202b80 20 client.30173 populate_metadata read hostname 'smithi044' 2023-09-10T01:32:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.563+0000 7fcce1202b80 1 --2- 172.21.15.44:0/2611360542 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aa90d439f0 0x55aa90d45e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.564+0000 7fcce1202b80 4 client.30173 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:32:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.564+0000 7fcce1202b80 1 -- 172.21.15.44:0/2611360542 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa90c90840 con 0x55aa90d439f0 2023-09-10T01:32:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.564+0000 7fccce59c700 1 --2- 172.21.15.44:0/2611360542 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aa90d439f0 0x55aa90d45e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.579+0000 7fccce59c700 1 --2- 172.21.15.44:0/2611360542 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aa90d439f0 0x55aa90d45e10 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f8dfbdf26f31c7 server_cookie=1d12425a3f3bdd22 in_seq=0 out_seq=0 2023-09-10T01:32:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.581+0000 7fccb2ff5700 10 client.30173 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.583+0000 7fccb2ff5700 1 -- 172.21.15.44:0/2611360542 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aa90c90840 con 0x55aa90d439f0 2023-09-10T01:32:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.583+0000 7fccb2ff5700 10 client.30173 handle_command_reply: tid=0 2023-09-10T01:32:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.583+0000 7fcce1202b80 1 client.30173 shutdown 2023-09-10T01:32:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.585+0000 7fcce1202b80 1 -- 172.21.15.44:0/2611360542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 msgr2=0x55aa90d33290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.585+0000 7fcce1202b80 1 --2- 172.21.15.44:0/2611360542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 0x55aa90d33290 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fccc40c9210 tx=0x7fccc4098670 comp rx=0 tx=0).stop 2023-09-10T01:32:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.586+0000 7fccce59c700 1 --2- 172.21.15.44:0/1591573170 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fccb80a0870 0x7fccb80a2d30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009fd0 tx=0x7fccc00099b0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.593+0000 7fcce1202b80 1 -- 172.21.15.44:0/2611360542 shutdown_connections 2023-09-10T01:32:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.593+0000 7fcce1202b80 1 --2- 172.21.15.44:0/2611360542 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aa90d439f0 0x55aa90d45e10 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.593+0000 7fcce1202b80 1 --2- 172.21.15.44:0/2611360542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d33d10 0x55aa90d3f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.593+0000 7fcce1202b80 1 --2- 172.21.15.44:0/2611360542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 0x55aa90d33290 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.593+0000 7fcce1202b80 1 --2- 172.21.15.44:0/2611360542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa90d29cc0 0x55aa90d337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.593+0000 7fcce1202b80 1 -- 172.21.15.44:0/2611360542 >> 172.21.15.44:0/2611360542 conn(0x55aa90c9e030 msgr2=0x55aa90d237a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.593+0000 7fccb2ff5700 0 client.30173 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.602+0000 7fcce1202b80 1 -- 172.21.15.44:0/2611360542 shutdown_connections 2023-09-10T01:32:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.611+0000 7fcce1202b80 1 -- 172.21.15.44:0/2611360542 wait complete. 2023-09-10T01:32:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.620+0000 7fcce1202b80 20 client.30173 trim_cache size 0 max 16384 2023-09-10T01:32:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.648+0000 7fcce1202b80 1 Processor -- start 2023-09-10T01:32:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.648+0000 7fcce1202b80 1 -- start start 2023-09-10T01:32:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.649+0000 7fcce1202b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 0x55aa90d30e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.649+0000 7fcce1202b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d29cc0 0x55aa90d3a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.649+0000 7fcce1202b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa90d33d10 0x55aa90d3f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.649+0000 7fcce1202b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa90d20fc0 con 0x55aa90d29cc0 2023-09-10T01:32:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.649+0000 7fcce1202b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa90d34870 con 0x55aa90d29300 2023-09-10T01:32:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.649+0000 7fcce1202b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa90d25d90 con 0x55aa90d33d10 2023-09-10T01:32:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.649+0000 7fccced9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 0x55aa90d30e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.649+0000 7fccced9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 0x55aa90d30e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34162/0 (socket says 172.21.15.44:34162) 2023-09-10T01:32:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.649+0000 7fccced9d700 1 -- 172.21.15.44:0/696233739 learned_addr learned my addr 172.21.15.44:0/696233739 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.650+0000 7fccced9d700 1 -- 172.21.15.44:0/696233739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa90d33d10 msgr2=0x55aa90d3f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.650+0000 7fccced9d700 1 --2- 172.21.15.44:0/696233739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa90d33d10 0x55aa90d3f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.650+0000 7fccced9d700 1 -- 172.21.15.44:0/696233739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d29cc0 msgr2=0x55aa90d3a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.650+0000 7fccced9d700 1 --2- 172.21.15.44:0/696233739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d29cc0 0x55aa90d3a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.650+0000 7fccced9d700 1 -- 172.21.15.44:0/696233739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa90ace1f0 con 0x55aa90d29300 2023-09-10T01:32:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.650+0000 7fccced9d700 1 --2- 172.21.15.44:0/696233739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 0x55aa90d30e60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fccc40c8dd0 tx=0x7fccc4097030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db3a2acc22ba75fa server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.659+0000 7fccb5ffb700 1 -- 172.21.15.44:0/696233739 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc40b4070 con 0x55aa90d29300 2023-09-10T01:32:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.659+0000 7fccb5ffb700 1 -- 172.21.15.44:0/696233739 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fccc4097f70 con 0x55aa90d29300 2023-09-10T01:32:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.659+0000 7fccb5ffb700 1 -- 172.21.15.44:0/696233739 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4099180 con 0x55aa90d29300 2023-09-10T01:32:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.660+0000 7fcce1202b80 1 -- 172.21.15.44:0/696233739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 msgr2=0x55aa90d30e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.660+0000 7fcce1202b80 1 --2- 172.21.15.44:0/696233739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 0x55aa90d30e60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fccc40c8dd0 tx=0x7fccc4097030 comp rx=0 tx=0).stop 2023-09-10T01:32:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.667+0000 7fcce1202b80 1 -- 172.21.15.44:0/696233739 shutdown_connections 2023-09-10T01:32:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.667+0000 7fcce1202b80 1 --2- 172.21.15.44:0/696233739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa90d33d10 0x55aa90d3f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.667+0000 7fcce1202b80 1 --2- 172.21.15.44:0/696233739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 0x55aa90d30e60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.667+0000 7fcce1202b80 1 --2- 172.21.15.44:0/696233739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d29cc0 0x55aa90d3a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.667+0000 7fcce1202b80 1 -- 172.21.15.44:0/696233739 >> 172.21.15.44:0/696233739 conn(0x55aa90c9e030 msgr2=0x55aa90d3ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.675+0000 7fcce1202b80 1 -- 172.21.15.44:0/696233739 shutdown_connections 2023-09-10T01:32:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.681+0000 7fcce1202b80 1 -- 172.21.15.44:0/696233739 wait complete. 2023-09-10T01:32:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.682+0000 7fcce1202b80 1 -- start start 2023-09-10T01:32:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.682+0000 7fcce1202b80 1 Processor -- start 2023-09-10T01:32:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.682+0000 7fcce1202b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 0x55aa90d8fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.683+0000 7fcce1202b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa90d29cc0 0x55aa90d90170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.683+0000 7fcce1202b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d33d10 0x55aa90d8b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.683+0000 7fcce1202b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa90d8bd90 con 0x55aa90d33d10 2023-09-10T01:32:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.683+0000 7fcce1202b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa90d8bf00 con 0x55aa90d29300 2023-09-10T01:32:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.683+0000 7fcce1202b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa90d8c0c0 con 0x55aa90d29cc0 2023-09-10T01:32:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.683+0000 7fcce1202b80 10 client.0 _ll_register_callbacks cb 0x7fccc4094dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:32:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.683+0000 7fcccf59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d33d10 0x55aa90d8b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.683+0000 7fcccf59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d33d10 0x55aa90d8b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60658/0 (socket says 172.21.15.44:60658) 2023-09-10T01:32:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.683+0000 7fcccf59e700 1 -- 172.21.15.44:0/266490810 learned_addr learned my addr 172.21.15.44:0/266490810 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.684+0000 7fcccf59e700 1 -- 172.21.15.44:0/266490810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa90d29cc0 msgr2=0x55aa90d90170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:32:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.684+0000 7fcccf59e700 1 --2- 172.21.15.44:0/266490810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa90d29cc0 0x55aa90d90170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.684+0000 7fcccf59e700 1 -- 172.21.15.44:0/266490810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 msgr2=0x55aa90d8fc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:32:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.684+0000 7fcccf59e700 1 --2- 172.21.15.44:0/266490810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 0x55aa90d8fc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.684+0000 7fcccf59e700 1 -- 172.21.15.44:0/266490810 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa90a90c00 con 0x55aa90d33d10 2023-09-10T01:32:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.684+0000 7fcccf59e700 1 --2- 172.21.15.44:0/266490810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d33d10 0x55aa90d8b790 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fccc801e010 tx=0x7fccc80079e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72de1344d80f9f04 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.684+0000 7fccb2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:32:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.684+0000 7fccb2ff5700 1 -- 172.21.15.44:0/266490810 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc801c910 con 0x55aa90d33d10 2023-09-10T01:32:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.684+0000 7fccb2ff5700 1 -- 172.21.15.44:0/266490810 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(199..199 src has 1..199) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fccc8003a00 con 0x55aa90d33d10 2023-09-10T01:32:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.684+0000 7fcce1202b80 1 -- 172.21.15.44:0/266490810 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa90d24720 con 0x55aa90d33d10 2023-09-10T01:32:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.685+0000 7fccb2ff5700 1 -- 172.21.15.44:0/266490810 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=186) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccc801ab90 con 0x55aa90d33d10 2023-09-10T01:32:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.685+0000 7fcce1202b80 10 client.20529 fetch_fsmap learned FSMap version 186 2023-09-10T01:32:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.685+0000 7fcce1202b80 1 -- 172.21.15.44:0/266490810 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=186,monmap=4+}) v3 -- 0x55aa90aa6250 con 0x55aa90d33d10 2023-09-10T01:32:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.685+0000 7fccb2ff5700 1 -- 172.21.15.44:0/266490810 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aa90d24720 con 0x55aa90d33d10 2023-09-10T01:32:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.685+0000 7fccb2ff5700 1 -- 172.21.15.44:0/266490810 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 186) v1 ==== 7371+0+0 (secure 0 0 0) 0x7fccc8033070 con 0x55aa90d33d10 2023-09-10T01:32:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.685+0000 7fcce1202b80 10 client.20529 fetch_fsmap finished waiting for FSMap version 186 2023-09-10T01:32:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.685+0000 7fcce1202b80 10 client.20529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:32:14.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.685+0000 7fcce1202b80 20 client.20529 populate_metadata read hostname 'smithi044' 2023-09-10T01:32:14.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.685+0000 7fcce1202b80 1 --2- 172.21.15.44:0/266490810 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aa90d98790 0x55aa90d9abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:14.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.685+0000 7fcce1202b80 4 client.20529 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:32:14.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.685+0000 7fcce1202b80 1 -- 172.21.15.44:0/266490810 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fccc801a2d0 con 0x55aa90d98790 2023-09-10T01:32:14.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.687+0000 7fccced9d700 1 --2- 172.21.15.44:0/266490810 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aa90d98790 0x55aa90d9abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.687+0000 7fccced9d700 1 --2- 172.21.15.44:0/266490810 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aa90d98790 0x55aa90d9abf0 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a7bc2e33c828625 server_cookie=ce13cc3db33fe626 in_seq=0 out_seq=0 2023-09-10T01:32:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.687+0000 7fccb2ff5700 10 client.20529 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.830+0000 7fccb2ff5700 1 -- 172.21.15.44:0/266490810 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fccc801a2d0 con 0x55aa90d98790 2023-09-10T01:32:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.830+0000 7fccb2ff5700 10 client.20529 handle_command_reply: tid=0 2023-09-10T01:32:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.830+0000 7fcce1202b80 1 client.20529 shutdown 2023-09-10T01:32:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.832+0000 7fcce1202b80 1 -- 172.21.15.44:0/266490810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d33d10 msgr2=0x55aa90d8b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.832+0000 7fcce1202b80 1 --2- 172.21.15.44:0/266490810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d33d10 0x55aa90d8b790 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fccc801e010 tx=0x7fccc80079e0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.832+0000 7fcce1202b80 1 -- 172.21.15.44:0/266490810 shutdown_connections 2023-09-10T01:32:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.832+0000 7fcce1202b80 1 --2- 172.21.15.44:0/266490810 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55aa90d98790 0x55aa90d9abf0 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.832+0000 7fcce1202b80 1 --2- 172.21.15.44:0/266490810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa90d33d10 0x55aa90d8b790 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.832+0000 7fcce1202b80 1 --2- 172.21.15.44:0/266490810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa90d29300 0x55aa90d8fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.832+0000 7fcce1202b80 1 --2- 172.21.15.44:0/266490810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa90d29cc0 0x55aa90d90170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.832+0000 7fcce1202b80 1 -- 172.21.15.44:0/266490810 >> 172.21.15.44:0/266490810 conn(0x55aa90c9e030 msgr2=0x55aa90d3f460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.832+0000 7fccb2ff5700 0 client.20529 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.843+0000 7fcce1202b80 1 -- 172.21.15.44:0/266490810 shutdown_connections 2023-09-10T01:32:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.843+0000 7fcce1202b80 1 -- 172.21.15.44:0/266490810 wait complete. 2023-09-10T01:32:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.852+0000 7fcce1202b80 20 client.20529 trim_cache size 0 max 16384 2023-09-10T01:32:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:32:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:32:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:32:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:32:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.867+0000 7fccd503c700 1 -- 172.21.15.44:0/1591573170 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fccb80a0870 msgr2=0x7fccb80a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.867+0000 7fccd503c700 1 --2- 172.21.15.44:0/1591573170 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fccb80a0870 0x7fccb80a2d30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009fd0 tx=0x7fccc00099b0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.867+0000 7fccd503c700 1 -- 172.21.15.44:0/1591573170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccd010ed50 msgr2=0x7fccd0115060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.867+0000 7fccd503c700 1 --2- 172.21.15.44:0/1591573170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccd010ed50 0x7fccd0115060 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fccc4006fd0 tx=0x7fccc400b730 comp rx=0 tx=0).stop 2023-09-10T01:32:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.872+0000 7fccd503c700 1 -- 172.21.15.44:0/1591573170 shutdown_connections 2023-09-10T01:32:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.872+0000 7fccd503c700 1 --2- 172.21.15.44:0/1591573170 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fccb80a0870 0x7fccb80a2d30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.872+0000 7fccd503c700 1 --2- 172.21.15.44:0/1591573170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccd0119de0 0x7fccd01a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.872+0000 7fccd503c700 1 --2- 172.21.15.44:0/1591573170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccd010ed50 0x7fccd0115060 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.872+0000 7fccd503c700 1 --2- 172.21.15.44:0/1591573170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd010f1c0 0x7fccd01155a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.872+0000 7fccd503c700 1 -- 172.21.15.44:0/1591573170 >> 172.21.15.44:0/1591573170 conn(0x7fccd006c6c0 msgr2=0x7fccd0070fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.872+0000 7fccd503c700 1 -- 172.21.15.44:0/1591573170 shutdown_connections 2023-09-10T01:32:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:14.872+0000 7fccd503c700 1 -- 172.21.15.44:0/1591573170 wait complete. 2023-09-10T01:32:16.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:15 smithi131 ceph-mon[103713]: pgmap v1679: 129 pgs: 19 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 61 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 482 op/s 2023-09-10T01:32:16.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:15 smithi044 ceph-mon[98676]: pgmap v1679: 129 pgs: 19 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 61 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 482 op/s 2023-09-10T01:32:16.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:15 smithi099 ceph-mon[102692]: pgmap v1679: 129 pgs: 19 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 61 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 482 op/s 2023-09-10T01:32:16.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 670.0 s, 81.9 tps, lat 1136.266 ms stddev 2134.453 2023-09-10T01:32:17.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:17 smithi099 ceph-mon[102692]: pgmap v1680: 129 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 97 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 429 op/s 2023-09-10T01:32:18.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:17 smithi131 ceph-mon[103713]: pgmap v1680: 129 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 97 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 429 op/s 2023-09-10T01:32:18.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:17 smithi044 ceph-mon[98676]: pgmap v1680: 129 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 97 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 429 op/s 2023-09-10T01:32:19.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:19 smithi099 ceph-mon[102692]: pgmap v1681: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 410 op/s 2023-09-10T01:32:20.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:19 smithi131 ceph-mon[103713]: pgmap v1681: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 410 op/s 2023-09-10T01:32:20.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:19 smithi044 ceph-mon[98676]: pgmap v1681: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 410 op/s 2023-09-10T01:32:21.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 675.0 s, 129.0 tps, lat 685.867 ms stddev 1433.891 2023-09-10T01:32:22.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:21 smithi131 ceph-mon[103713]: pgmap v1682: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 2.8 MiB/s wr, 343 op/s 2023-09-10T01:32:22.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:21 smithi131 ceph-mon[103713]: osdmap e200: 12 total, 12 up, 12 in 2023-09-10T01:32:22.061 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:21 smithi044 ceph-mon[98676]: pgmap v1682: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 2.8 MiB/s wr, 343 op/s 2023-09-10T01:32:22.061 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:21 smithi044 ceph-mon[98676]: osdmap e200: 12 total, 12 up, 12 in 2023-09-10T01:32:22.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:21 smithi099 ceph-mon[102692]: pgmap v1682: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 2.8 MiB/s wr, 343 op/s 2023-09-10T01:32:22.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:21 smithi099 ceph-mon[102692]: osdmap e200: 12 total, 12 up, 12 in 2023-09-10T01:32:24.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:23 smithi131 ceph-mon[103713]: pgmap v1684: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 502 op/s 2023-09-10T01:32:24.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:23 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:32:24.061 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:23 smithi044 ceph-mon[98676]: pgmap v1684: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 502 op/s 2023-09-10T01:32:24.061 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:23 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:32:24.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:23 smithi099 ceph-mon[102692]: pgmap v1684: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 502 op/s 2023-09-10T01:32:24.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:23 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:32:26.051 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:25 smithi131 ceph-mon[103713]: pgmap v1685: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 520 op/s 2023-09-10T01:32:26.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:25 smithi044 ceph-mon[98676]: pgmap v1685: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 520 op/s 2023-09-10T01:32:26.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:25 smithi099 ceph-mon[102692]: pgmap v1685: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 520 op/s 2023-09-10T01:32:26.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 680.0 s, 221.6 tps, lat 701.845 ms stddev 1034.459 2023-09-10T01:32:27.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:27 smithi099 ceph-mon[102692]: osdmap e201: 12 total, 12 up, 12 in 2023-09-10T01:32:27.856 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:27 smithi099 ceph-mon[102692]: pgmap v1686: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 638 op/s 2023-09-10T01:32:27.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:27 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:27.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:27 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:27.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:27 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:27.857 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:27 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:27 smithi131 ceph-mon[103713]: osdmap e201: 12 total, 12 up, 12 in 2023-09-10T01:32:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:27 smithi131 ceph-mon[103713]: pgmap v1686: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 638 op/s 2023-09-10T01:32:28.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:27 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:28.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:27 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:28.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:27 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:28.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:27 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:28.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:27 smithi044 ceph-mon[98676]: osdmap e201: 12 total, 12 up, 12 in 2023-09-10T01:32:28.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:27 smithi044 ceph-mon[98676]: pgmap v1686: 129 pgs: 129 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 638 op/s 2023-09-10T01:32:28.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:27 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:28.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:27 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:28.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:27 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:28.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:27 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.097+0000 7f6af79f5700 1 -- 172.21.15.44:0/4090772003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6af0107940 msgr2=0x7f6af010b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.097+0000 7f6af79f5700 1 --2- 172.21.15.44:0/4090772003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6af0107940 0x7f6af010b830 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009990 tx=0x7f6ae0009ca0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.103+0000 7f6af79f5700 1 -- 172.21.15.44:0/4090772003 shutdown_connections 2023-09-10T01:32:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.103+0000 7f6af79f5700 1 --2- 172.21.15.44:0/4090772003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6af010bf60 0x7f6af0113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.103+0000 7f6af79f5700 1 --2- 172.21.15.44:0/4090772003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6af0106f90 0x7f6af0107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.103+0000 7f6af79f5700 1 --2- 172.21.15.44:0/4090772003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6af0107940 0x7f6af010b830 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.103+0000 7f6af79f5700 1 -- 172.21.15.44:0/4090772003 >> 172.21.15.44:0/4090772003 conn(0x7f6af00764a0 msgr2=0x7f6af0105420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.107+0000 7f6af79f5700 1 -- 172.21.15.44:0/4090772003 shutdown_connections 2023-09-10T01:32:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.107+0000 7f6af79f5700 1 -- 172.21.15.44:0/4090772003 wait complete. 2023-09-10T01:32:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.108+0000 7f6af79f5700 1 Processor -- start 2023-09-10T01:32:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.108+0000 7f6af79f5700 1 -- start start 2023-09-10T01:32:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.108+0000 7f6af79f5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6af0106f90 0x7f6af019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.108+0000 7f6af79f5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6af010bf60 0x7f6af019d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.108+0000 7f6af79f5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6af01a18f0 0x7f6af01a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.108+0000 7f6af79f5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af01a2370 con 0x7f6af01a18f0 2023-09-10T01:32:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.108+0000 7f6af79f5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af01a24e0 con 0x7f6af0106f90 2023-09-10T01:32:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.108+0000 7f6af79f5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af01a26a0 con 0x7f6af010bf60 2023-09-10T01:32:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.108+0000 7f6af4f90700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6af010bf60 0x7f6af019d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.108+0000 7f6af4f90700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6af010bf60 0x7f6af019d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39320/0 (socket says 172.21.15.44:39320) 2023-09-10T01:32:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.108+0000 7f6af4f90700 1 -- 172.21.15.44:0/185586537 learned_addr learned my addr 172.21.15.44:0/185586537 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.109+0000 7f6af4f90700 1 -- 172.21.15.44:0/185586537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6af0106f90 msgr2=0x7f6af019cd40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-10T01:32:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.109+0000 7f6af4f90700 1 --2- 172.21.15.44:0/185586537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6af0106f90 0x7f6af019cd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.109+0000 7f6af4f90700 1 -- 172.21.15.44:0/185586537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6af01a18f0 msgr2=0x7f6af01a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.109+0000 7f6af4f90700 1 --2- 172.21.15.44:0/185586537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6af01a18f0 0x7f6af01a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.109+0000 7f6af4f90700 1 -- 172.21.15.44:0/185586537 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae00095f0 con 0x7f6af010bf60 2023-09-10T01:32:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.109+0000 7f6af4f90700 1 --2- 172.21.15.44:0/185586537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6af010bf60 0x7f6af019d280 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0005910 tx=0x7f6ae00057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.109+0000 7f6ae67fc700 1 -- 172.21.15.44:0/185586537 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae001d070 con 0x7f6af010bf60 2023-09-10T01:32:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.109+0000 7f6af79f5700 1 -- 172.21.15.44:0/185586537 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af01a9ba0 con 0x7f6af010bf60 2023-09-10T01:32:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.109+0000 7f6ae67fc700 1 -- 172.21.15.44:0/185586537 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ae0022470 con 0x7f6af010bf60 2023-09-10T01:32:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.109+0000 7f6ae67fc700 1 -- 172.21.15.44:0/185586537 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0017650 con 0x7f6af010bf60 2023-09-10T01:32:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.109+0000 7f6af79f5700 1 -- 172.21.15.44:0/185586537 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af01a9fc0 con 0x7f6af010bf60 2023-09-10T01:32:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.114+0000 7f6b03bbbb80 1 Processor -- start 2023-09-10T01:32:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.114+0000 7f6b03bbbb80 1 -- start start 2023-09-10T01:32:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.126+0000 7f6ae67fc700 1 -- 172.21.15.44:0/185586537 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f6ae00225e0 con 0x7f6af010bf60 2023-09-10T01:32:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.136+0000 7f6b03bbbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c4d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.136+0000 7f6b03bbbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c47310 0x55b8a4c47790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.136+0000 7f6b03bbbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c53f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.136+0000 7f6b03bbbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a4b8f6d0 con 0x55b8a4c47310 2023-09-10T01:32:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.136+0000 7f6b03bbbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a4b8cff0 con 0x55b8a4c47cd0 2023-09-10T01:32:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.136+0000 7f6b03bbbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a4b888e0 con 0x55b8a4c4d1b0 2023-09-10T01:32:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.137+0000 7f6af4f90700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c53f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.137+0000 7f6ae67fc700 1 --2- 172.21.15.44:0/185586537 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6adc0a07b0 0x7f6adc0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.137+0000 7f6af4f90700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c53f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34184/0 (socket says 172.21.15.44:34184) 2023-09-10T01:32:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.137+0000 7f6af4f90700 1 -- 172.21.15.44:0/2141117545 learned_addr learned my addr 172.21.15.44:0/2141117545 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.139+0000 7f6af5f92700 1 --2- 172.21.15.44:0/2141117545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c4d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.142+0000 7f6af5791700 1 --2- 172.21.15.44:0/2141117545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c47310 0x55b8a4c47790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.143+0000 7f6af5f92700 1 --2- 172.21.15.44:0/185586537 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6adc0a07b0 0x7f6adc0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.143+0000 7f6ae67fc700 1 -- 172.21.15.44:0/185586537 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f6ae00cd6f0 con 0x7f6af010bf60 2023-09-10T01:32:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.143+0000 7f6af5f92700 1 -- 172.21.15.44:0/2141117545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c47cd0 msgr2=0x55b8a4c53f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.143+0000 7f6af5f92700 1 --2- 172.21.15.44:0/2141117545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c53f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.144+0000 7f6af5f92700 1 -- 172.21.15.44:0/2141117545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c47310 msgr2=0x55b8a4c47790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.144+0000 7f6af5f92700 1 --2- 172.21.15.44:0/2141117545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c47310 0x55b8a4c47790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.144+0000 7f6af5f92700 1 -- 172.21.15.44:0/2141117545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8a49b4d80 con 0x55b8a4c4d1b0 2023-09-10T01:32:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.144+0000 7f6af5f92700 1 --2- 172.21.15.44:0/2141117545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c4d590 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8008720 tx=0x7f6ae8008a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fa4deb663a11c8c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.146+0000 7f6adb7fe700 1 -- 172.21.15.44:0/2141117545 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8003e30 con 0x55b8a4c4d1b0 2023-09-10T01:32:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.146+0000 7f6adb7fe700 1 -- 172.21.15.44:0/2141117545 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ae80044c0 con 0x55b8a4c4d1b0 2023-09-10T01:32:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.146+0000 7f6adb7fe700 1 -- 172.21.15.44:0/2141117545 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8017a80 con 0x55b8a4c4d1b0 2023-09-10T01:32:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.147+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/2141117545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c4d1b0 msgr2=0x55b8a4c4d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.147+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/2141117545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c4d590 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8008720 tx=0x7f6ae8008a30 comp rx=0 tx=0).stop 2023-09-10T01:32:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.147+0000 7f6af5f92700 1 --2- 172.21.15.44:0/185586537 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6adc0a07b0 0x7f6adc0a2c70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6ae801d000 tx=0x7f6ae8017be0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.147+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/2141117545 shutdown_connections 2023-09-10T01:32:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.147+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/2141117545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c53f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.147+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/2141117545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c4d590 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.147+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/2141117545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c47310 0x55b8a4c47790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.147+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/2141117545 >> 172.21.15.44:0/2141117545 conn(0x55b8a4bab030 msgr2=0x55b8a4c31560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.153+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/2141117545 shutdown_connections 2023-09-10T01:32:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.155+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/2141117545 wait complete. 2023-09-10T01:32:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.155+0000 7f6b03bbbb80 1 -- start start 2023-09-10T01:32:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.155+0000 7f6b03bbbb80 1 Processor -- start 2023-09-10T01:32:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.156+0000 7f6b03bbbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c47310 0x55b8a4c53ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.156+0000 7f6b03bbbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c510d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.156+0000 7f6b03bbbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c51810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.156+0000 7f6b03bbbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a4b26b10 con 0x55b8a4c47310 2023-09-10T01:32:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.156+0000 7f6b03bbbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a4b26810 con 0x55b8a4c4d1b0 2023-09-10T01:32:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.156+0000 7f6b03bbbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a4a0a8c0 con 0x55b8a4c47cd0 2023-09-10T01:32:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.156+0000 7f6b03bbbb80 10 client.0 _ll_register_callbacks cb 0x55b8a4a70a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:32:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.156+0000 7f6af5f92700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c51810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.156+0000 7f6af5f92700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c51810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34192/0 (socket says 172.21.15.44:34192) 2023-09-10T01:32:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.156+0000 7f6af5f92700 1 -- 172.21.15.44:0/1865495530 learned_addr learned my addr 172.21.15.44:0/1865495530 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.156+0000 7f6af4f90700 1 --2- 172.21.15.44:0/1865495530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.156+0000 7f6af5f92700 1 -- 172.21.15.44:0/1865495530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c47cd0 msgr2=0x55b8a4c510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.156+0000 7f6af5f92700 1 --2- 172.21.15.44:0/1865495530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.156+0000 7f6af5f92700 1 -- 172.21.15.44:0/1865495530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c47310 msgr2=0x55b8a4c53ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:32:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.156+0000 7f6af5f92700 1 --2- 172.21.15.44:0/1865495530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c47310 0x55b8a4c53ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.156+0000 7f6af5f92700 1 -- 172.21.15.44:0/1865495530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae8011040 con 0x55b8a4c4d1b0 2023-09-10T01:32:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.161+0000 7f6af5f92700 1 --2- 172.21.15.44:0/1865495530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c51810 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009fb0 tx=0x7f6ae8009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=963a71b355161e07 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.164+0000 7f6ad8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:32:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.165+0000 7f6ad8ff9700 1 -- 172.21.15.44:0/1865495530 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8017d70 con 0x55b8a4c4d1b0 2023-09-10T01:32:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.165+0000 7f6ad8ff9700 1 -- 172.21.15.44:0/1865495530 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(201..201 src has 1..201) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f6ae8017410 con 0x55b8a4c4d1b0 2023-09-10T01:32:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.165+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/1865495530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8a4ae0490 con 0x55b8a4c4d1b0 2023-09-10T01:32:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.165+0000 7f6ad8ff9700 1 -- 172.21.15.44:0/1865495530 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=186) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ae8004630 con 0x55b8a4c4d1b0 2023-09-10T01:32:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.169+0000 7f6b03bbbb80 10 client.30182 fetch_fsmap learned FSMap version 186 2023-09-10T01:32:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.169+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/1865495530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=186,monmap=4+}) v3 -- 0x55b8a499dc00 con 0x55b8a4c4d1b0 2023-09-10T01:32:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.169+0000 7f6ad8ff9700 1 -- 172.21.15.44:0/1865495530 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b8a4ae0490 con 0x55b8a4c4d1b0 2023-09-10T01:32:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.170+0000 7f6ad8ff9700 1 -- 172.21.15.44:0/1865495530 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 186) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f6ae8032020 con 0x55b8a4c4d1b0 2023-09-10T01:32:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.170+0000 7f6b03bbbb80 10 client.30182 fetch_fsmap finished waiting for FSMap version 186 2023-09-10T01:32:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.170+0000 7f6b03bbbb80 10 client.30182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:32:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.170+0000 7f6b03bbbb80 20 client.30182 populate_metadata read hostname 'smithi044' 2023-09-10T01:32:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.170+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/1865495530 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8a4c61400 0x55b8a4c63820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.170+0000 7f6b03bbbb80 4 client.30182 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:32:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.170+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/1865495530 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8a4b9d840 con 0x55b8a4c61400 2023-09-10T01:32:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.171+0000 7f6af5791700 1 --2- 172.21.15.44:0/1865495530 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8a4c61400 0x55b8a4c63820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.171+0000 7f6ad8ff9700 10 client.30182 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.171+0000 7f6af5791700 1 --2- 172.21.15.44:0/1865495530 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8a4c61400 0x55b8a4c63820 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8d04ec14487fc5 server_cookie=7d9652cb39d3da64 in_seq=0 out_seq=0 2023-09-10T01:32:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.173+0000 7f6ad8ff9700 1 -- 172.21.15.44:0/1865495530 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b8a4b9d840 con 0x55b8a4c61400 2023-09-10T01:32:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.173+0000 7f6ad8ff9700 10 client.30182 handle_command_reply: tid=0 2023-09-10T01:32:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.173+0000 7f6b03bbbb80 1 client.30182 shutdown 2023-09-10T01:32:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.174+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/1865495530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c4d1b0 msgr2=0x55b8a4c51810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.174+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/1865495530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c51810 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009fb0 tx=0x7f6ae8009e60 comp rx=0 tx=0).stop 2023-09-10T01:32:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.184+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/1865495530 shutdown_connections 2023-09-10T01:32:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.184+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/1865495530 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8a4c61400 0x55b8a4c63820 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.184+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/1865495530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c51810 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.184+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/1865495530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c47310 0x55b8a4c53ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.184+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/1865495530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.184+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/1865495530 >> 172.21.15.44:0/1865495530 conn(0x55b8a4bab030 msgr2=0x55b8a4c307a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.184+0000 7f6ad8ff9700 0 client.30182 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.185+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/1865495530 shutdown_connections 2023-09-10T01:32:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.185+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/1865495530 wait complete. 2023-09-10T01:32:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.195+0000 7f6b03bbbb80 20 client.30182 trim_cache size 0 max 16384 2023-09-10T01:32:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.227+0000 7f6b03bbbb80 1 Processor -- start 2023-09-10T01:32:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.228+0000 7f6b03bbbb80 1 -- start start 2023-09-10T01:32:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.228+0000 7f6b03bbbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c47310 0x55b8a4c5d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.228+0000 7f6b03bbbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.228+0000 7f6b03bbbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c50640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.228+0000 7f6b03bbbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a4c2dfc0 con 0x55b8a4c47310 2023-09-10T01:32:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.228+0000 7f6b03bbbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a4c4d590 con 0x55b8a4c47cd0 2023-09-10T01:32:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.228+0000 7f6b03bbbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a4c4bcd0 con 0x55b8a4c4d1b0 2023-09-10T01:32:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.229+0000 7f6af5791700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c47310 0x55b8a4c5d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.229+0000 7f6af5791700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c47310 0x55b8a4c5d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60692/0 (socket says 172.21.15.44:60692) 2023-09-10T01:32:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.229+0000 7f6af5791700 1 -- 172.21.15.44:0/3050650347 learned_addr learned my addr 172.21.15.44:0/3050650347 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.229+0000 7f6af5f92700 1 --2- 172.21.15.44:0/3050650347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c50640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.229+0000 7f6af5791700 1 -- 172.21.15.44:0/3050650347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c4d1b0 msgr2=0x55b8a4c50640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.229+0000 7f6af5791700 1 --2- 172.21.15.44:0/3050650347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c50640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.229+0000 7f6af5791700 1 -- 172.21.15.44:0/3050650347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c47cd0 msgr2=0x55b8a4c539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.229+0000 7f6af5791700 1 --2- 172.21.15.44:0/3050650347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.229+0000 7f6af5791700 1 -- 172.21.15.44:0/3050650347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8a49db1f0 con 0x55b8a4c47310 2023-09-10T01:32:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.230+0000 7f6af5791700 1 --2- 172.21.15.44:0/3050650347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c47310 0x55b8a4c5d710 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x55b8a4c64120 tx=0x7f6aec016b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8b1cf05a09f9d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.237+0000 7f6adbfff700 1 -- 172.21.15.44:0/3050650347 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec0161a0 con 0x55b8a4c47310 2023-09-10T01:32:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.237+0000 7f6adbfff700 1 -- 172.21.15.44:0/3050650347 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6aec01a070 con 0x55b8a4c47310 2023-09-10T01:32:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.237+0000 7f6adbfff700 1 -- 172.21.15.44:0/3050650347 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec01f7b0 con 0x55b8a4c47310 2023-09-10T01:32:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.238+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/3050650347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c47310 msgr2=0x55b8a4c5d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.238+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/3050650347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c47310 0x55b8a4c5d710 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x55b8a4c64120 tx=0x7f6aec016b30 comp rx=0 tx=0).stop 2023-09-10T01:32:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.238+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/3050650347 shutdown_connections 2023-09-10T01:32:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.238+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/3050650347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c50640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.238+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/3050650347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c47310 0x55b8a4c5d710 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.238+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/3050650347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.238+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/3050650347 >> 172.21.15.44:0/3050650347 conn(0x55b8a4bab030 msgr2=0x55b8a4c681a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.239+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/3050650347 shutdown_connections 2023-09-10T01:32:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.240+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/3050650347 wait complete. 2023-09-10T01:32:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.240+0000 7f6b03bbbb80 1 -- start start 2023-09-10T01:32:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.240+0000 7f6b03bbbb80 1 Processor -- start 2023-09-10T01:32:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.241+0000 7f6b03bbbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c47310 0x55b8a4c582a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.241+0000 7f6b03bbbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.241+0000 7f6b03bbbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c56b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.241+0000 7f6b03bbbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a4c57040 con 0x55b8a4c4d1b0 2023-09-10T01:32:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.241+0000 7f6b03bbbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a4c571b0 con 0x55b8a4c47cd0 2023-09-10T01:32:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.241+0000 7f6b03bbbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a4c572f0 con 0x55b8a4c47310 2023-09-10T01:32:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.241+0000 7f6b03bbbb80 10 client.0 _ll_register_callbacks cb 0x7f6ae80085a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:32:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.242+0000 7f6af5791700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c47310 0x55b8a4c582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.242+0000 7f6af5791700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c47310 0x55b8a4c582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39346/0 (socket says 172.21.15.44:39346) 2023-09-10T01:32:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.242+0000 7f6af5791700 1 -- 172.21.15.44:0/2608480812 learned_addr learned my addr 172.21.15.44:0/2608480812 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.244+0000 7f6af4f90700 1 --2- 172.21.15.44:0/2608480812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.246+0000 7f6af4f90700 1 -- 172.21.15.44:0/2608480812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c47310 msgr2=0x55b8a4c582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.246+0000 7f6af4f90700 1 --2- 172.21.15.44:0/2608480812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c47310 0x55b8a4c582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.246+0000 7f6af4f90700 1 -- 172.21.15.44:0/2608480812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c4d1b0 msgr2=0x55b8a4c56b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.246+0000 7f6af4f90700 1 --2- 172.21.15.44:0/2608480812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c56b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.246+0000 7f6af4f90700 1 -- 172.21.15.44:0/2608480812 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec00e040 con 0x55b8a4c47cd0 2023-09-10T01:32:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.247+0000 7f6af4f90700 1 --2- 172.21.15.44:0/2608480812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c565c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00cb050 tx=0x7f6ae00cd8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=878ec8093fa4c431 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.247+0000 7f6ad8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:32:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.247+0000 7f6ad8ff9700 1 -- 172.21.15.44:0/2608480812 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae009a380 con 0x55b8a4c47cd0 2023-09-10T01:32:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.247+0000 7f6ad8ff9700 1 -- 172.21.15.44:0/2608480812 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(201..201 src has 1..201) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f6ae009af40 con 0x55b8a4c47cd0 2023-09-10T01:32:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.247+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/2608480812 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ae80044c0 con 0x55b8a4c47cd0 2023-09-10T01:32:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.247+0000 7f6ad8ff9700 1 -- 172.21.15.44:0/2608480812 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=186) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ae009a1a0 con 0x55b8a4c47cd0 2023-09-10T01:32:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.247+0000 7f6b03bbbb80 10 client.30185 fetch_fsmap learned FSMap version 186 2023-09-10T01:32:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.247+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/2608480812 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=186,monmap=4+}) v3 -- 0x55b8a49b3250 con 0x55b8a4c47cd0 2023-09-10T01:32:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.248+0000 7f6ad8ff9700 1 -- 172.21.15.44:0/2608480812 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ae80044c0 con 0x55b8a4c47cd0 2023-09-10T01:32:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.248+0000 7f6ad8ff9700 1 -- 172.21.15.44:0/2608480812 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 186) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f6ae00a2390 con 0x55b8a4c47cd0 2023-09-10T01:32:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.248+0000 7f6b03bbbb80 10 client.30185 fetch_fsmap finished waiting for FSMap version 186 2023-09-10T01:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.248+0000 7f6b03bbbb80 10 client.30185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.248+0000 7f6b03bbbb80 20 client.30185 populate_metadata read hostname 'smithi044' 2023-09-10T01:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.248+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/2608480812 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8a4cc7360 0x55b8a4cc97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.248+0000 7f6b03bbbb80 4 client.30185 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:32:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.248+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/2608480812 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b8a4b818c0 con 0x55b8a4cc7360 2023-09-10T01:32:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.252+0000 7f6af5791700 1 --2- 172.21.15.44:0/2608480812 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8a4cc7360 0x55b8a4cc97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.254+0000 7f6af5791700 1 --2- 172.21.15.44:0/2608480812 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8a4cc7360 0x55b8a4cc97a0 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcacb8ab6c70f964 server_cookie=c4358d341ebfc6c8 in_seq=0 out_seq=0 2023-09-10T01:32:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.254+0000 7f6ad8ff9700 10 client.30185 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.283+0000 7f6ad8ff9700 1 -- 172.21.15.44:0/2608480812 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b8a4b818c0 con 0x55b8a4cc7360 2023-09-10T01:32:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.283+0000 7f6ad8ff9700 10 client.30185 handle_command_reply: tid=0 2023-09-10T01:32:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.284+0000 7f6b03bbbb80 1 client.30185 shutdown 2023-09-10T01:32:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.286+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/2608480812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c47cd0 msgr2=0x55b8a4c565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.286+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/2608480812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c565c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00cb050 tx=0x7f6ae00cd8b0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.286+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/2608480812 shutdown_connections 2023-09-10T01:32:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.286+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/2608480812 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55b8a4cc7360 0x55b8a4cc97a0 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.286+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/2608480812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b8a4c4d1b0 0x55b8a4c56b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.286+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/2608480812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b8a4c47310 0x55b8a4c582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.286+0000 7f6b03bbbb80 1 --2- 172.21.15.44:0/2608480812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8a4c47cd0 0x55b8a4c565c0 secure :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00cb050 tx=0x7f6ae00cd8b0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.286+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/2608480812 >> 172.21.15.44:0/2608480812 conn(0x55b8a4bab030 msgr2=0x55b8a4c5d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.286+0000 7f6ad8ff9700 0 client.30185 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.288+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/2608480812 shutdown_connections 2023-09-10T01:32:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.293+0000 7f6b03bbbb80 1 -- 172.21.15.44:0/2608480812 wait complete. 2023-09-10T01:32:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.295+0000 7f6b03bbbb80 20 client.30185 trim_cache size 0 max 16384 2023-09-10T01:32:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:32:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:32:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:32:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:32:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.297+0000 7f6af79f5700 1 -- 172.21.15.44:0/185586537 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6adc0a07b0 msgr2=0x7f6adc0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.297+0000 7f6af79f5700 1 --2- 172.21.15.44:0/185586537 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6adc0a07b0 0x7f6adc0a2c70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6ae801d000 tx=0x7f6ae8017be0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.297+0000 7f6af79f5700 1 -- 172.21.15.44:0/185586537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6af010bf60 msgr2=0x7f6af019d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.297+0000 7f6af79f5700 1 --2- 172.21.15.44:0/185586537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6af010bf60 0x7f6af019d280 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0005910 tx=0x7f6ae00057f0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.297+0000 7f6af79f5700 1 -- 172.21.15.44:0/185586537 shutdown_connections 2023-09-10T01:32:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.297+0000 7f6af79f5700 1 --2- 172.21.15.44:0/185586537 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f6adc0a07b0 0x7f6adc0a2c70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.297+0000 7f6af79f5700 1 --2- 172.21.15.44:0/185586537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6af01a18f0 0x7f6af01a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.297+0000 7f6af79f5700 1 --2- 172.21.15.44:0/185586537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6af0106f90 0x7f6af019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.297+0000 7f6af79f5700 1 --2- 172.21.15.44:0/185586537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6af010bf60 0x7f6af019d280 secure :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0005910 tx=0x7f6ae00057f0 comp rx=0 tx=0).stop 2023-09-10T01:32:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.297+0000 7f6af79f5700 1 -- 172.21.15.44:0/185586537 >> 172.21.15.44:0/185586537 conn(0x7f6af00764a0 msgr2=0x7f6af010a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.298+0000 7f6af79f5700 1 -- 172.21.15.44:0/185586537 shutdown_connections 2023-09-10T01:32:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:28.298+0000 7f6af79f5700 1 -- 172.21.15.44:0/185586537 wait complete. 2023-09-10T01:32:29.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T01:32:29.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T01:32:29.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.050 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.050 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:28 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T01:32:29.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T01:32:29.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.061 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.061 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:28 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.114 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.114 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.114 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.115 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.115 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.115 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.115 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.115 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.116 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi131 to 196.2M 2023-09-10T01:32:29.116 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi131 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-10T01:32:29.116 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.116 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.117 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.117 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.117 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.117 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.117 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:29.118 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:28 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:29 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T01:32:30.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:29 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:32:30.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:29 smithi131 ceph-mon[103713]: pgmap v1688: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.5 MiB/s wr, 859 op/s 2023-09-10T01:32:30.061 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:29 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T01:32:30.062 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:29 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:32:30.062 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:29 smithi044 ceph-mon[98676]: pgmap v1688: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.5 MiB/s wr, 859 op/s 2023-09-10T01:32:30.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:29 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-09-10T01:32:30.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:29 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:32:30.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:29 smithi099 ceph-mon[102692]: pgmap v1688: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.5 MiB/s wr, 859 op/s 2023-09-10T01:32:31.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:30 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:31.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:30 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:31.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:30 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:30 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:30 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:30 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:30 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:30 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:30 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:30 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:30 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:32:31.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:30 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:32:31.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:30 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:31.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 685.0 s, 266.6 tps, lat 389.948 ms stddev 165.683 2023-09-10T01:32:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:30 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:30 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:31.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:30 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:30 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:30 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:30 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:30 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:30 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:30 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:30 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:30 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:32:31.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:30 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:32:31.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:30 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:31.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:30 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:31.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:30 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:31.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:30 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:30 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:30 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:30 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:30 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:30 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:30 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:30 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:32:31.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:30 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:32:31.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:30 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:32:31.310 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:30 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:32:32.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:31 smithi131 ceph-mon[103713]: pgmap v1689: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 524 op/s 2023-09-10T01:32:32.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:31 smithi131 ceph-mon[103713]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T01:32:32.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:31 smithi131 ceph-mon[103713]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T01:32:32.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:31 smithi131 ceph-mon[103713]: pgmap v1690: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 569 op/s 2023-09-10T01:32:32.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:31 smithi044 ceph-mon[98676]: pgmap v1689: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 524 op/s 2023-09-10T01:32:32.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:31 smithi044 ceph-mon[98676]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T01:32:32.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:31 smithi044 ceph-mon[98676]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T01:32:32.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:31 smithi044 ceph-mon[98676]: pgmap v1690: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 569 op/s 2023-09-10T01:32:32.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:31 smithi099 ceph-mon[102692]: pgmap v1689: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 524 op/s 2023-09-10T01:32:32.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:31 smithi099 ceph-mon[102692]: Adjusting osd_memory_target on smithi044 to 197.2M 2023-09-10T01:32:32.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:31 smithi099 ceph-mon[102692]: Unable to set osd_memory_target on smithi044 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-10T01:32:32.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:31 smithi099 ceph-mon[102692]: pgmap v1690: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 569 op/s 2023-09-10T01:32:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:32 smithi131 ceph-mon[103713]: pgmap v1691: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 873 op/s 2023-09-10T01:32:33.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:32 smithi044 ceph-mon[98676]: pgmap v1691: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 873 op/s 2023-09-10T01:32:33.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:32 smithi099 ceph-mon[102692]: pgmap v1691: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 873 op/s 2023-09-10T01:32:35.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:34 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:32:35.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:34 smithi131 ceph-mon[103713]: pgmap v1692: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 586 op/s 2023-09-10T01:32:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:34 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:32:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:34 smithi044 ceph-mon[98676]: pgmap v1692: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 586 op/s 2023-09-10T01:32:35.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:34 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:32:35.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:34 smithi099 ceph-mon[102692]: pgmap v1692: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 586 op/s 2023-09-10T01:32:36.184 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 690.0 s, 279.2 tps, lat 343.499 ms stddev 143.273 2023-09-10T01:32:38.051 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:37 smithi131 ceph-mon[103713]: pgmap v1693: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 779 op/s 2023-09-10T01:32:38.062 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:37 smithi044 ceph-mon[98676]: pgmap v1693: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 779 op/s 2023-09-10T01:32:38.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:37 smithi099 ceph-mon[102692]: pgmap v1693: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 779 op/s 2023-09-10T01:32:40.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:39 smithi099 ceph-mon[102692]: pgmap v1694: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.3 MiB/s wr, 782 op/s 2023-09-10T01:32:40.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:39 smithi131 ceph-mon[103713]: pgmap v1694: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.3 MiB/s wr, 782 op/s 2023-09-10T01:32:40.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:39 smithi044 ceph-mon[98676]: pgmap v1694: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.3 MiB/s wr, 782 op/s 2023-09-10T01:32:41.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 695.0 s, 316.2 tps, lat 325.187 ms stddev 127.800 2023-09-10T01:32:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.544+0000 7ffa9754c700 1 -- 172.21.15.44:0/1788783044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa90108fa0 msgr2=0x7ffa90109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.544+0000 7ffa9754c700 1 --2- 172.21.15.44:0/1788783044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa90108fa0 0x7ffa90109380 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7ffa80009990 tx=0x7ffa80009ca0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.545+0000 7ffa9754c700 1 -- 172.21.15.44:0/1788783044 shutdown_connections 2023-09-10T01:32:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.545+0000 7ffa9754c700 1 --2- 172.21.15.44:0/1788783044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa90112550 0x7ffa90114940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.545+0000 7ffa9754c700 1 --2- 172.21.15.44:0/1788783044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa90108fa0 0x7ffa90109380 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.545+0000 7ffa9754c700 1 --2- 172.21.15.44:0/1788783044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa901098c0 0x7ffa90112010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.545+0000 7ffa9754c700 1 -- 172.21.15.44:0/1788783044 >> 172.21.15.44:0/1788783044 conn(0x7ffa9006c6c0 msgr2=0x7ffa9006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.545+0000 7ffa9754c700 1 -- 172.21.15.44:0/1788783044 shutdown_connections 2023-09-10T01:32:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.545+0000 7ffa9754c700 1 -- 172.21.15.44:0/1788783044 wait complete. 2023-09-10T01:32:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.546+0000 7ffa9754c700 1 Processor -- start 2023-09-10T01:32:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.546+0000 7ffa9754c700 1 -- start start 2023-09-10T01:32:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.546+0000 7ffa9754c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa90108fa0 0x7ffa90111eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.546+0000 7ffa9754c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa901098c0 0x7ffa9010dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.546+0000 7ffa9754c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90112550 0x7ffa9010e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.546+0000 7ffa9754c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9010ea10 con 0x7ffa90108fa0 2023-09-10T01:32:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.546+0000 7ffa9754c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9011c520 con 0x7ffa90112550 2023-09-10T01:32:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.546+0000 7ffa9754c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9011c6e0 con 0x7ffa901098c0 2023-09-10T01:32:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.546+0000 7ffa952e8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa90108fa0 0x7ffa90111eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.546+0000 7ffa952e8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa90108fa0 0x7ffa90111eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60714/0 (socket says 172.21.15.44:60714) 2023-09-10T01:32:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.546+0000 7ffa952e8700 1 -- 172.21.15.44:0/3838458778 learned_addr learned my addr 172.21.15.44:0/3838458778 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.547+0000 7ffa95ae9700 1 --2- 172.21.15.44:0/3838458778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90112550 0x7ffa9010e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.547+0000 7ffa94ae7700 1 --2- 172.21.15.44:0/3838458778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa901098c0 0x7ffa9010dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.547+0000 7ffa95ae9700 1 -- 172.21.15.44:0/3838458778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa901098c0 msgr2=0x7ffa9010dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.547+0000 7ffa95ae9700 1 --2- 172.21.15.44:0/3838458778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa901098c0 0x7ffa9010dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.547+0000 7ffa95ae9700 1 -- 172.21.15.44:0/3838458778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa90108fa0 msgr2=0x7ffa90111eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.547+0000 7ffa95ae9700 1 --2- 172.21.15.44:0/3838458778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa90108fa0 0x7ffa90111eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.547+0000 7ffa95ae9700 1 -- 172.21.15.44:0/3838458778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa800095f0 con 0x7ffa90112550 2023-09-10T01:32:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.547+0000 7ffa95ae9700 1 --2- 172.21.15.44:0/3838458778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90112550 0x7ffa9010e240 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800efc0 tx=0x7ffa8800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.547+0000 7ffa867fc700 1 -- 172.21.15.44:0/3838458778 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88018070 con 0x7ffa90112550 2023-09-10T01:32:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.548+0000 7ffa867fc700 1 -- 172.21.15.44:0/3838458778 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffa8800f040 con 0x7ffa90112550 2023-09-10T01:32:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.548+0000 7ffa867fc700 1 -- 172.21.15.44:0/3838458778 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8800cbc0 con 0x7ffa90112550 2023-09-10T01:32:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.548+0000 7ffa9754c700 1 -- 172.21.15.44:0/3838458778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9011c960 con 0x7ffa90112550 2023-09-10T01:32:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.548+0000 7ffa9754c700 1 -- 172.21.15.44:0/3838458778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9011ce20 con 0x7ffa90112550 2023-09-10T01:32:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.550+0000 7ffa867fc700 1 -- 172.21.15.44:0/3838458778 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ffa8800cd40 con 0x7ffa90112550 2023-09-10T01:32:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.551+0000 7ffa867fc700 1 --2- 172.21.15.44:0/3838458778 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffa7c0a0900 0x7ffa7c0a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.551+0000 7ffa867fc700 1 -- 172.21.15.44:0/3838458778 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 9524+0+0 (secure 0 0 0) 0x7ffa880cbcb0 con 0x7ffa90112550 2023-09-10T01:32:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.552+0000 7ffaa3712b80 1 Processor -- start 2023-09-10T01:32:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.552+0000 7ffaa3712b80 1 -- start start 2023-09-10T01:32:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.553+0000 7ffaa3712b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede4c1a0 0x55f2ede4c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.553+0000 7ffaa3712b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46300 0x55f2ede46780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.553+0000 7ffaa3712b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede46cc0 0x55f2ede52f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.553+0000 7ffaa3712b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2edd9f6d0 con 0x55f2ede46300 2023-09-10T01:32:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.553+0000 7ffaa3712b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2edd9cff0 con 0x55f2ede46cc0 2023-09-10T01:32:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.553+0000 7ffaa3712b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2edd988e0 con 0x55f2ede4c1a0 2023-09-10T01:32:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.553+0000 7ffa95ae9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede46cc0 0x55f2ede52f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.553+0000 7ffa95ae9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede46cc0 0x55f2ede52f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34226/0 (socket says 172.21.15.44:34226) 2023-09-10T01:32:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.553+0000 7ffa95ae9700 1 -- 172.21.15.44:0/1528293923 learned_addr learned my addr 172.21.15.44:0/1528293923 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.554+0000 7ffa95ae9700 1 -- 172.21.15.44:0/1528293923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede4c1a0 msgr2=0x55f2ede4c580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:32:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.554+0000 7ffa95ae9700 1 --2- 172.21.15.44:0/1528293923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede4c1a0 0x55f2ede4c580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.554+0000 7ffa95ae9700 1 -- 172.21.15.44:0/1528293923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46300 msgr2=0x55f2ede46780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.554+0000 7ffa95ae9700 1 --2- 172.21.15.44:0/1528293923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46300 0x55f2ede46780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.554+0000 7ffa95ae9700 1 -- 172.21.15.44:0/1528293923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2edbc4d80 con 0x55f2ede46cc0 2023-09-10T01:32:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.555+0000 7ffa95ae9700 1 --2- 172.21.15.44:0/1528293923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede46cc0 0x55f2ede52f30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ffa880090a0 tx=0x7ffa880987e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51d3cb06081e1c44 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.555+0000 7ffa7b7fe700 1 -- 172.21.15.44:0/1528293923 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88098e00 con 0x55f2ede46cc0 2023-09-10T01:32:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.555+0000 7ffa7b7fe700 1 -- 172.21.15.44:0/1528293923 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffa8809cc30 con 0x55f2ede46cc0 2023-09-10T01:32:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.555+0000 7ffa7b7fe700 1 -- 172.21.15.44:0/1528293923 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88099060 con 0x55f2ede46cc0 2023-09-10T01:32:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.556+0000 7ffaa3712b80 1 -- 172.21.15.44:0/1528293923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede46cc0 msgr2=0x55f2ede52f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.556+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/1528293923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede46cc0 0x55f2ede52f30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ffa880090a0 tx=0x7ffa880987e0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.561+0000 7ffa952e8700 1 --2- 172.21.15.44:0/3838458778 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffa7c0a0900 0x7ffa7c0a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.561+0000 7ffaa3712b80 1 -- 172.21.15.44:0/1528293923 shutdown_connections 2023-09-10T01:32:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.561+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/1528293923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede46cc0 0x55f2ede52f30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.561+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/1528293923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede4c1a0 0x55f2ede4c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.561+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/1528293923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46300 0x55f2ede46780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.561+0000 7ffaa3712b80 1 -- 172.21.15.44:0/1528293923 >> 172.21.15.44:0/1528293923 conn(0x55f2eddbb030 msgr2=0x55f2ede41560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.561+0000 7ffaa3712b80 1 -- 172.21.15.44:0/1528293923 shutdown_connections 2023-09-10T01:32:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.561+0000 7ffaa3712b80 1 -- 172.21.15.44:0/1528293923 wait complete. 2023-09-10T01:32:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.561+0000 7ffaa3712b80 1 -- start start 2023-09-10T01:32:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.562+0000 7ffaa3712b80 1 Processor -- start 2023-09-10T01:32:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.562+0000 7ffaa3712b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede46300 0x55f2ede52ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.562+0000 7ffaa3712b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46cc0 0x55f2ede50170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.562+0000 7ffaa3712b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede4c1a0 0x55f2ede508b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.562+0000 7ffaa3712b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2edd36b10 con 0x55f2ede46cc0 2023-09-10T01:32:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.562+0000 7ffaa3712b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2edd36810 con 0x55f2ede4c1a0 2023-09-10T01:32:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.562+0000 7ffaa3712b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2edc1a8c0 con 0x55f2ede46300 2023-09-10T01:32:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.562+0000 7ffaa3712b80 10 client.0 _ll_register_callbacks cb 0x55f2edc80a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:32:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.562+0000 7ffa94ae7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46cc0 0x55f2ede50170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.562+0000 7ffa94ae7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46cc0 0x55f2ede50170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60722/0 (socket says 172.21.15.44:60722) 2023-09-10T01:32:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.562+0000 7ffa94ae7700 1 -- 172.21.15.44:0/3132934697 learned_addr learned my addr 172.21.15.44:0/3132934697 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.563+0000 7ffa95ae9700 1 --2- 172.21.15.44:0/3132934697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede4c1a0 0x55f2ede508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.563+0000 7ffa94ae7700 1 -- 172.21.15.44:0/3132934697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede46300 msgr2=0x55f2ede52ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:32:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.563+0000 7ffa94ae7700 1 --2- 172.21.15.44:0/3132934697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede46300 0x55f2ede52ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.563+0000 7ffa94ae7700 1 -- 172.21.15.44:0/3132934697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede4c1a0 msgr2=0x55f2ede508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.563+0000 7ffa94ae7700 1 --2- 172.21.15.44:0/3132934697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede4c1a0 0x55f2ede508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.563+0000 7ffa94ae7700 1 -- 172.21.15.44:0/3132934697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9011ce20 con 0x55f2ede46cc0 2023-09-10T01:32:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.563+0000 7ffa95ae9700 1 --2- 172.21.15.44:0/3132934697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede4c1a0 0x55f2ede508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:32:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.563+0000 7ffa94ae7700 1 --2- 172.21.15.44:0/3132934697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46cc0 0x55f2ede50170 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0116c0 tx=0x7ffa8c011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4dfd8f234f3acf server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.563+0000 7ffa78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:32:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.563+0000 7ffa78ff9700 1 -- 172.21.15.44:0/3132934697 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c004a30 con 0x55f2ede46cc0 2023-09-10T01:32:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.563+0000 7ffa78ff9700 1 -- 172.21.15.44:0/3132934697 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(201..201 src has 1..201) v4 ==== 9524+0+0 (secure 0 0 0) 0x7ffa8c004b90 con 0x55f2ede46cc0 2023-09-10T01:32:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.566+0000 7ffaa3712b80 1 -- 172.21.15.44:0/3132934697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2edcf0490 con 0x55f2ede46cc0 2023-09-10T01:32:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.566+0000 7ffa78ff9700 1 -- 172.21.15.44:0/3132934697 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa8c004d50 con 0x55f2ede46cc0 2023-09-10T01:32:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.566+0000 7ffaa3712b80 10 client.20547 fetch_fsmap learned FSMap version 187 2023-09-10T01:32:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.566+0000 7ffaa3712b80 1 -- 172.21.15.44:0/3132934697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=187,monmap=4+}) v3 -- 0x55f2edbadc00 con 0x55f2ede46cc0 2023-09-10T01:32:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.566+0000 7ffa78ff9700 1 -- 172.21.15.44:0/3132934697 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f2edcf0490 con 0x55f2ede46cc0 2023-09-10T01:32:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.567+0000 7ffa78ff9700 1 -- 172.21.15.44:0/3132934697 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 187) v1 ==== 7375+0+0 (secure 0 0 0) 0x7ffa8c005230 con 0x55f2ede46cc0 2023-09-10T01:32:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.567+0000 7ffaa3712b80 10 client.20547 fetch_fsmap finished waiting for FSMap version 187 2023-09-10T01:32:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.567+0000 7ffaa3712b80 10 client.20547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:32:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.567+0000 7ffaa3712b80 20 client.20547 populate_metadata read hostname 'smithi044' 2023-09-10T01:32:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.567+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/3132934697 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2ede605c0 0x55f2ede629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.567+0000 7ffaa3712b80 4 client.20547 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:32:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.567+0000 7ffaa3712b80 1 -- 172.21.15.44:0/3132934697 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2eddad840 con 0x55f2ede605c0 2023-09-10T01:32:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.568+0000 7ffa952e8700 1 --2- 172.21.15.44:0/3132934697 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2ede605c0 0x55f2ede629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.568+0000 7ffa952e8700 1 --2- 172.21.15.44:0/3132934697 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2ede605c0 0x55f2ede629e0 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=650dd83d4ba96888 server_cookie=7447b8bdd0466071 in_seq=0 out_seq=0 2023-09-10T01:32:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.568+0000 7ffa78ff9700 10 client.20547 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.569+0000 7ffa78ff9700 1 -- 172.21.15.44:0/3132934697 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f2eddad840 con 0x55f2ede605c0 2023-09-10T01:32:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.569+0000 7ffa78ff9700 10 client.20547 handle_command_reply: tid=0 2023-09-10T01:32:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.569+0000 7ffaa3712b80 1 client.20547 shutdown 2023-09-10T01:32:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.569+0000 7ffaa3712b80 1 -- 172.21.15.44:0/3132934697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46cc0 msgr2=0x55f2ede50170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.569+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/3132934697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46cc0 0x55f2ede50170 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0116c0 tx=0x7ffa8c011a80 comp rx=0 tx=0).stop 2023-09-10T01:32:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.569+0000 7ffaa3712b80 1 -- 172.21.15.44:0/3132934697 shutdown_connections 2023-09-10T01:32:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.569+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/3132934697 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2ede605c0 0x55f2ede629e0 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.569+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/3132934697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede4c1a0 0x55f2ede508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.569+0000 7ffa78ff9700 0 client.20547 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.569+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/3132934697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede46300 0x55f2ede52ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.569+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/3132934697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46cc0 0x55f2ede50170 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.569+0000 7ffaa3712b80 1 -- 172.21.15.44:0/3132934697 >> 172.21.15.44:0/3132934697 conn(0x55f2eddbb030 msgr2=0x55f2ede407a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.569+0000 7ffaa3712b80 1 -- 172.21.15.44:0/3132934697 shutdown_connections 2023-09-10T01:32:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.569+0000 7ffaa3712b80 1 -- 172.21.15.44:0/3132934697 wait complete. 2023-09-10T01:32:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.569+0000 7ffaa3712b80 20 client.20547 trim_cache size 0 max 16384 2023-09-10T01:32:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.571+0000 7ffa952e8700 1 --2- 172.21.15.44:0/3838458778 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffa7c0a0900 0x7ffa7c0a2dc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55f2ede609a0 tx=0x7ffa8c009db0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.595+0000 7ffaa3712b80 1 Processor -- start 2023-09-10T01:32:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.595+0000 7ffaa3712b80 1 -- start start 2023-09-10T01:32:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.595+0000 7ffaa3712b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede46300 0x55f2ede4f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.595+0000 7ffaa3712b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46cc0 0x55f2ede52980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.595+0000 7ffaa3712b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede4c1a0 0x55f2ede52ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.595+0000 7ffaa3712b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ede3dfc0 con 0x55f2ede46cc0 2023-09-10T01:32:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.595+0000 7ffaa3712b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ede42b80 con 0x55f2ede4c1a0 2023-09-10T01:32:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.595+0000 7ffaa3712b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ede5daf0 con 0x55f2ede46300 2023-09-10T01:32:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.596+0000 7ffa94ae7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46cc0 0x55f2ede52980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.596+0000 7ffa94ae7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46cc0 0x55f2ede52980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60728/0 (socket says 172.21.15.44:60728) 2023-09-10T01:32:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.596+0000 7ffa94ae7700 1 -- 172.21.15.44:0/1259779579 learned_addr learned my addr 172.21.15.44:0/1259779579 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.596+0000 7ffa94ae7700 1 -- 172.21.15.44:0/1259779579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede46300 msgr2=0x55f2ede4f400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:32:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.596+0000 7ffa94ae7700 1 --2- 172.21.15.44:0/1259779579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede46300 0x55f2ede4f400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.596+0000 7ffa94ae7700 1 -- 172.21.15.44:0/1259779579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede4c1a0 msgr2=0x55f2ede52ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:32:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.596+0000 7ffa94ae7700 1 --2- 172.21.15.44:0/1259779579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede4c1a0 0x55f2ede52ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.596+0000 7ffa94ae7700 1 -- 172.21.15.44:0/1259779579 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2edbeb1f0 con 0x55f2ede46cc0 2023-09-10T01:32:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.596+0000 7ffa94ae7700 1 --2- 172.21.15.44:0/1259779579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46cc0 0x55f2ede52980 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c001eb0 tx=0x7ffa8c005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf31fa0377657488 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.597+0000 7ffa7bfff700 1 -- 172.21.15.44:0/1259779579 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c00fa10 con 0x55f2ede46cc0 2023-09-10T01:32:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.597+0000 7ffa7bfff700 1 -- 172.21.15.44:0/1259779579 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffa8c019430 con 0x55f2ede46cc0 2023-09-10T01:32:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.597+0000 7ffa7bfff700 1 -- 172.21.15.44:0/1259779579 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c04d680 con 0x55f2ede46cc0 2023-09-10T01:32:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.598+0000 7ffaa3712b80 1 -- 172.21.15.44:0/1259779579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46cc0 msgr2=0x55f2ede52980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.598+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/1259779579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46cc0 0x55f2ede52980 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c001eb0 tx=0x7ffa8c005ce0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.598+0000 7ffaa3712b80 1 -- 172.21.15.44:0/1259779579 shutdown_connections 2023-09-10T01:32:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.598+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/1259779579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede4c1a0 0x55f2ede52ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.598+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/1259779579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede46300 0x55f2ede4f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.598+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/1259779579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46cc0 0x55f2ede52980 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.598+0000 7ffaa3712b80 1 -- 172.21.15.44:0/1259779579 >> 172.21.15.44:0/1259779579 conn(0x55f2eddbb030 msgr2=0x55f2ede4a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.599+0000 7ffaa3712b80 1 -- 172.21.15.44:0/1259779579 shutdown_connections 2023-09-10T01:32:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.599+0000 7ffaa3712b80 1 -- 172.21.15.44:0/1259779579 wait complete. 2023-09-10T01:32:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.599+0000 7ffaa3712b80 1 -- start start 2023-09-10T01:32:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.599+0000 7ffaa3712b80 1 Processor -- start 2023-09-10T01:32:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.599+0000 7ffaa3712b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46300 0x55f2ede56ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.599+0000 7ffaa3712b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede46cc0 0x55f2ede57410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.599+0000 7ffaa3712b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede4c1a0 0x55f2ede55510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.599+0000 7ffaa3712b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ede55ae0 con 0x55f2ede46300 2023-09-10T01:32:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.599+0000 7ffaa3712b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ede55c50 con 0x55f2ede46cc0 2023-09-10T01:32:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.599+0000 7ffaa3712b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ede55e10 con 0x55f2ede4c1a0 2023-09-10T01:32:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.600+0000 7ffaa3712b80 10 client.0 _ll_register_callbacks cb 0x55f2edc80a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:32:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.600+0000 7ffa94ae7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede46cc0 0x55f2ede57410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.600+0000 7ffa95ae9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede4c1a0 0x55f2ede55510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.600+0000 7ffa95ae9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede4c1a0 0x55f2ede55510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39388/0 (socket says 172.21.15.44:39388) 2023-09-10T01:32:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.600+0000 7ffa95ae9700 1 -- 172.21.15.44:0/4047084159 learned_addr learned my addr 172.21.15.44:0/4047084159 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.600+0000 7ffa952e8700 1 --2- 172.21.15.44:0/4047084159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46300 0x55f2ede56ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.600+0000 7ffa95ae9700 1 -- 172.21.15.44:0/4047084159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede46cc0 msgr2=0x55f2ede57410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.600+0000 7ffa95ae9700 1 --2- 172.21.15.44:0/4047084159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede46cc0 0x55f2ede57410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.600+0000 7ffa95ae9700 1 -- 172.21.15.44:0/4047084159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46300 msgr2=0x55f2ede56ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.600+0000 7ffa95ae9700 1 --2- 172.21.15.44:0/4047084159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46300 0x55f2ede56ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.600+0000 7ffa95ae9700 1 -- 172.21.15.44:0/4047084159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2edbadc00 con 0x55f2ede4c1a0 2023-09-10T01:32:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.601+0000 7ffa95ae9700 1 --2- 172.21.15.44:0/4047084159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede4c1a0 0x55f2ede55510 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800ef90 tx=0x7ffa8809cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc76246e6ecde1f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.601+0000 7ffa78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:32:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.601+0000 7ffa78ff9700 1 -- 172.21.15.44:0/4047084159 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa880984f0 con 0x55f2ede4c1a0 2023-09-10T01:32:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.601+0000 7ffa78ff9700 1 -- 172.21.15.44:0/4047084159 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(201..201 src has 1..201) v4 ==== 9524+0+0 (secure 0 0 0) 0x7ffa880a0de0 con 0x55f2ede4c1a0 2023-09-10T01:32:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.601+0000 7ffaa3712b80 1 -- 172.21.15.44:0/4047084159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa8809cc30 con 0x55f2ede4c1a0 2023-09-10T01:32:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.601+0000 7ffa78ff9700 1 -- 172.21.15.44:0/4047084159 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa88098e80 con 0x55f2ede4c1a0 2023-09-10T01:32:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.601+0000 7ffaa3712b80 10 client.30217 fetch_fsmap learned FSMap version 187 2023-09-10T01:32:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.601+0000 7ffaa3712b80 1 -- 172.21.15.44:0/4047084159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=187,monmap=4+}) v3 -- 0x55f2edbc3250 con 0x55f2ede4c1a0 2023-09-10T01:32:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.601+0000 7ffa78ff9700 1 -- 172.21.15.44:0/4047084159 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffa8809cc30 con 0x55f2ede4c1a0 2023-09-10T01:32:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.602+0000 7ffa78ff9700 1 -- 172.21.15.44:0/4047084159 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 187) v1 ==== 7375+0+0 (secure 0 0 0) 0x7ffa88097d60 con 0x55f2ede4c1a0 2023-09-10T01:32:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.603+0000 7ffaa3712b80 10 client.30217 fetch_fsmap finished waiting for FSMap version 187 2023-09-10T01:32:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.603+0000 7ffaa3712b80 10 client.30217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:32:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.603+0000 7ffaa3712b80 20 client.30217 populate_metadata read hostname 'smithi044' 2023-09-10T01:32:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.603+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/4047084159 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2edeb51b0 0x55f2edeb75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.603+0000 7ffaa3712b80 4 client.30217 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:32:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.603+0000 7ffaa3712b80 1 -- 172.21.15.44:0/4047084159 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f2edd918c0 con 0x55f2edeb51b0 2023-09-10T01:32:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.609+0000 7ffa952e8700 1 --2- 172.21.15.44:0/4047084159 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2edeb51b0 0x55f2edeb75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.629+0000 7ffa952e8700 1 --2- 172.21.15.44:0/4047084159 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2edeb51b0 0x55f2edeb75d0 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6947542feee9af6 server_cookie=8e49318c9585959b in_seq=0 out_seq=0 2023-09-10T01:32:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.629+0000 7ffa78ff9700 10 client.30217 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.674+0000 7ffa78ff9700 1 -- 172.21.15.44:0/4047084159 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f2edd918c0 con 0x55f2edeb51b0 2023-09-10T01:32:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.674+0000 7ffa78ff9700 10 client.30217 handle_command_reply: tid=0 2023-09-10T01:32:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.678+0000 7ffaa3712b80 1 client.30217 shutdown 2023-09-10T01:32:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.679+0000 7ffaa3712b80 1 -- 172.21.15.44:0/4047084159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede4c1a0 msgr2=0x55f2ede55510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.679+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/4047084159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede4c1a0 0x55f2ede55510 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800ef90 tx=0x7ffa8809cda0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.679+0000 7ffaa3712b80 1 -- 172.21.15.44:0/4047084159 shutdown_connections 2023-09-10T01:32:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.679+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/4047084159 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2edeb51b0 0x55f2edeb75d0 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.679+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/4047084159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2ede4c1a0 0x55f2ede55510 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.679+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/4047084159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2ede46300 0x55f2ede56ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.679+0000 7ffaa3712b80 1 --2- 172.21.15.44:0/4047084159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2ede46cc0 0x55f2ede57410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.679+0000 7ffaa3712b80 1 -- 172.21.15.44:0/4047084159 >> 172.21.15.44:0/4047084159 conn(0x55f2eddbb030 msgr2=0x55f2ede4fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.679+0000 7ffa78ff9700 0 client.30217 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.679+0000 7ffaa3712b80 1 -- 172.21.15.44:0/4047084159 shutdown_connections 2023-09-10T01:32:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.679+0000 7ffaa3712b80 1 -- 172.21.15.44:0/4047084159 wait complete. 2023-09-10T01:32:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.679+0000 7ffaa3712b80 20 client.30217 trim_cache size 0 max 16384 2023-09-10T01:32:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:32:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:32:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:32:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:32:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.680+0000 7ffa9754c700 1 -- 172.21.15.44:0/3838458778 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffa7c0a0900 msgr2=0x7ffa7c0a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.680+0000 7ffa9754c700 1 --2- 172.21.15.44:0/3838458778 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffa7c0a0900 0x7ffa7c0a2dc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55f2ede609a0 tx=0x7ffa8c009db0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.680+0000 7ffa9754c700 1 -- 172.21.15.44:0/3838458778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90112550 msgr2=0x7ffa9010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.680+0000 7ffa9754c700 1 --2- 172.21.15.44:0/3838458778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90112550 0x7ffa9010e240 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800efc0 tx=0x7ffa8800c3a0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.680+0000 7ffa9754c700 1 -- 172.21.15.44:0/3838458778 shutdown_connections 2023-09-10T01:32:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.680+0000 7ffa9754c700 1 --2- 172.21.15.44:0/3838458778 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffa7c0a0900 0x7ffa7c0a2dc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.680+0000 7ffa9754c700 1 --2- 172.21.15.44:0/3838458778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90112550 0x7ffa9010e240 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.680+0000 7ffa9754c700 1 --2- 172.21.15.44:0/3838458778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa90108fa0 0x7ffa90111eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.680+0000 7ffa9754c700 1 --2- 172.21.15.44:0/3838458778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa901098c0 0x7ffa9010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.680+0000 7ffa9754c700 1 -- 172.21.15.44:0/3838458778 >> 172.21.15.44:0/3838458778 conn(0x7ffa9006c6c0 msgr2=0x7ffa90070490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.680+0000 7ffa9754c700 1 -- 172.21.15.44:0/3838458778 shutdown_connections 2023-09-10T01:32:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:41.680+0000 7ffa9754c700 1 -- 172.21.15.44:0/3838458778 wait complete. 2023-09-10T01:32:42.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:41 smithi131 ceph-mon[103713]: pgmap v1695: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.7 MiB/s wr, 830 op/s 2023-09-10T01:32:42.310 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:41 smithi044 ceph-mon[98676]: pgmap v1695: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.7 MiB/s wr, 830 op/s 2023-09-10T01:32:42.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:41 smithi099 ceph-mon[102692]: pgmap v1695: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.7 MiB/s wr, 830 op/s 2023-09-10T01:32:44.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:43 smithi131 ceph-mon[103713]: pgmap v1696: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.4 MiB/s wr, 949 op/s 2023-09-10T01:32:44.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:43 smithi044 ceph-mon[98676]: pgmap v1696: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.4 MiB/s wr, 949 op/s 2023-09-10T01:32:44.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:43 smithi099 ceph-mon[102692]: pgmap v1696: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.4 MiB/s wr, 949 op/s 2023-09-10T01:32:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:44 smithi131 ceph-mon[103713]: pgmap v1697: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 748 op/s 2023-09-10T01:32:45.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:44 smithi044 ceph-mon[98676]: pgmap v1697: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 748 op/s 2023-09-10T01:32:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:44 smithi099 ceph-mon[102692]: pgmap v1697: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 748 op/s 2023-09-10T01:32:46.190 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 700.0 s, 303.9 tps, lat 333.031 ms stddev 127.624 2023-09-10T01:32:48.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:47 smithi131 ceph-mon[103713]: pgmap v1698: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.7 MiB/s wr, 978 op/s 2023-09-10T01:32:48.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:47 smithi044 ceph-mon[98676]: pgmap v1698: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.7 MiB/s wr, 978 op/s 2023-09-10T01:32:48.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:47 smithi099 ceph-mon[102692]: pgmap v1698: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.7 MiB/s wr, 978 op/s 2023-09-10T01:32:50.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:49 smithi131 ceph-mon[103713]: pgmap v1699: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.8 MiB/s wr, 807 op/s 2023-09-10T01:32:50.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:49 smithi044 ceph-mon[98676]: pgmap v1699: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.8 MiB/s wr, 807 op/s 2023-09-10T01:32:50.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:49 smithi099 ceph-mon[102692]: pgmap v1699: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.8 MiB/s wr, 807 op/s 2023-09-10T01:32:51.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 705.0 s, 316.5 tps, lat 292.696 ms stddev 242.044 2023-09-10T01:32:52.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:51 smithi131 ceph-mon[103713]: pgmap v1700: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.9 MiB/s wr, 768 op/s 2023-09-10T01:32:52.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:51 smithi044 ceph-mon[98676]: pgmap v1700: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.9 MiB/s wr, 768 op/s 2023-09-10T01:32:52.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:51 smithi099 ceph-mon[102692]: pgmap v1700: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.9 MiB/s wr, 768 op/s 2023-09-10T01:32:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:52 smithi131 ceph-mon[103713]: pgmap v1701: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.9 MiB/s wr, 1.02k op/s 2023-09-10T01:32:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:52 smithi044 ceph-mon[98676]: pgmap v1701: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.9 MiB/s wr, 1.02k op/s 2023-09-10T01:32:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:52 smithi099 ceph-mon[102692]: pgmap v1701: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.9 MiB/s wr, 1.02k op/s 2023-09-10T01:32:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.887+0000 7fe973832700 1 -- 172.21.15.44:0/543524705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe96c119de0 msgr2=0x7fe96c11c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.887+0000 7fe973832700 1 --2- 172.21.15.44:0/543524705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe96c119de0 0x7fe96c11c1d0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009990 tx=0x7fe95c009ca0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.887+0000 7fe973832700 1 -- 172.21.15.44:0/543524705 shutdown_connections 2023-09-10T01:32:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.887+0000 7fe973832700 1 --2- 172.21.15.44:0/543524705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe96c119de0 0x7fe96c11c1d0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.887+0000 7fe973832700 1 --2- 172.21.15.44:0/543524705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe96c10ed50 0x7fe96c107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.887+0000 7fe973832700 1 --2- 172.21.15.44:0/543524705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96c10f1c0 0x7fe96c107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.887+0000 7fe973832700 1 -- 172.21.15.44:0/543524705 >> 172.21.15.44:0/543524705 conn(0x7fe96c06c6c0 msgr2=0x7fe96c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.889+0000 7fe973832700 1 -- 172.21.15.44:0/543524705 shutdown_connections 2023-09-10T01:32:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.893+0000 7fe973832700 1 -- 172.21.15.44:0/543524705 wait complete. 2023-09-10T01:32:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.894+0000 7fe973832700 1 Processor -- start 2023-09-10T01:32:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.894+0000 7fe973832700 1 -- start start 2023-09-10T01:32:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.895+0000 7fe973832700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe96c10ed50 0x7fe96c077a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.895+0000 7fe973832700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe96c10f1c0 0x7fe96c077fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.895+0000 7fe973832700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96c119de0 0x7fe96c072920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.895+0000 7fe973832700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c0730f0 con 0x7fe96c10ed50 2023-09-10T01:32:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.895+0000 7fe973832700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c073260 con 0x7fe96c119de0 2023-09-10T01:32:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.895+0000 7fe973832700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c073420 con 0x7fe96c10f1c0 2023-09-10T01:32:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.895+0000 7fe971dcf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96c119de0 0x7fe96c072920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.895+0000 7fe971dcf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96c119de0 0x7fe96c072920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34258/0 (socket says 172.21.15.44:34258) 2023-09-10T01:32:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.895+0000 7fe971dcf700 1 -- 172.21.15.44:0/4046456319 learned_addr learned my addr 172.21.15.44:0/4046456319 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.895+0000 7fe971dcf700 1 -- 172.21.15.44:0/4046456319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe96c10f1c0 msgr2=0x7fe96c077fd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-10T01:32:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.895+0000 7fe971dcf700 1 --2- 172.21.15.44:0/4046456319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe96c10f1c0 0x7fe96c077fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.895+0000 7fe971dcf700 1 -- 172.21.15.44:0/4046456319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe96c10ed50 msgr2=0x7fe96c077a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T01:32:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.895+0000 7fe971dcf700 1 --2- 172.21.15.44:0/4046456319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe96c10ed50 0x7fe96c077a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.896+0000 7fe971dcf700 1 -- 172.21.15.44:0/4046456319 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c0095f0 con 0x7fe96c119de0 2023-09-10T01:32:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.896+0000 7fe971dcf700 1 --2- 172.21.15.44:0/4046456319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96c119de0 0x7fe96c072920 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe95c006010 tx=0x7fe95c0038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.905+0000 7fe9627fc700 1 -- 172.21.15.44:0/4046456319 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c01d070 con 0x7fe96c119de0 2023-09-10T01:32:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.905+0000 7fe9627fc700 1 -- 172.21.15.44:0/4046456319 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe95c0040a0 con 0x7fe96c119de0 2023-09-10T01:32:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.905+0000 7fe9627fc700 1 -- 172.21.15.44:0/4046456319 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c022ce0 con 0x7fe96c119de0 2023-09-10T01:32:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.908+0000 7fe973832700 1 -- 172.21.15.44:0/4046456319 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96c0736a0 con 0x7fe96c119de0 2023-09-10T01:32:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.908+0000 7fe973832700 1 -- 172.21.15.44:0/4046456319 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96c07f9e0 con 0x7fe96c119de0 2023-09-10T01:32:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.917+0000 7fe97f9f8b80 1 Processor -- start 2023-09-10T01:32:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.917+0000 7fe97f9f8b80 1 -- start start 2023-09-10T01:32:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.917+0000 7fe97f9f8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8d9360 0x55652c8d9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.917+0000 7fe97f9f8b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8d9c80 0x55652c8e45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.918+0000 7fe97f9f8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8e4b30 0x55652c8e6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.918+0000 7fe97f9f8b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55652c8336d0 con 0x55652c8d9c80 2023-09-10T01:32:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.918+0000 7fe97f9f8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55652c830ff0 con 0x55652c8e4b30 2023-09-10T01:32:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.918+0000 7fe97f9f8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55652c82c8e0 con 0x55652c8d9360 2023-09-10T01:32:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.921+0000 7fe9715ce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8d9c80 0x55652c8e45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.921+0000 7fe9715ce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8d9c80 0x55652c8e45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60756/0 (socket says 172.21.15.44:60756) 2023-09-10T01:32:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.921+0000 7fe9715ce700 1 -- 172.21.15.44:0/763913286 learned_addr learned my addr 172.21.15.44:0/763913286 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.926+0000 7fe9627fc700 1 -- 172.21.15.44:0/4046456319 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fe95c003c10 con 0x7fe96c119de0 2023-09-10T01:32:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.926+0000 7fe9627fc700 1 --2- 172.21.15.44:0/4046456319 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe9580a09c0 0x7fe9580a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.927+0000 7fe9627fc700 1 -- 172.21.15.44:0/4046456319 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fe95c022e40 con 0x7fe96c119de0 2023-09-10T01:32:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.928+0000 7fe9715ce700 1 -- 172.21.15.44:0/763913286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8d9360 msgr2=0x55652c8d9740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:32:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.928+0000 7fe9715ce700 1 --2- 172.21.15.44:0/763913286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8d9360 0x55652c8d9740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.928+0000 7fe9715ce700 1 -- 172.21.15.44:0/763913286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8e4b30 msgr2=0x55652c8e6f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:32:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.928+0000 7fe9715ce700 1 --2- 172.21.15.44:0/763913286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8e4b30 0x55652c8e6f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.928+0000 7fe9715ce700 1 -- 172.21.15.44:0/763913286 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55652c658d80 con 0x55652c8d9c80 2023-09-10T01:32:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.929+0000 7fe9715ce700 1 --2- 172.21.15.44:0/763913286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8d9c80 0x55652c8e45f0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fe964009da0 tx=0x7fe96400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99dfad33f082b59 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.929+0000 7fe973832700 1 -- 172.21.15.44:0/763913286 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96400e070 con 0x55652c8d9c80 2023-09-10T01:32:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.929+0000 7fe973832700 1 -- 172.21.15.44:0/763913286 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe96400be50 con 0x55652c8d9c80 2023-09-10T01:32:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.929+0000 7fe973832700 1 -- 172.21.15.44:0/763913286 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964012e00 con 0x55652c8d9c80 2023-09-10T01:32:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.930+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/763913286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8d9c80 msgr2=0x55652c8e45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.930+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/763913286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8d9c80 0x55652c8e45f0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fe964009da0 tx=0x7fe96400b330 comp rx=0 tx=0).stop 2023-09-10T01:32:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.931+0000 7fe971dcf700 1 --2- 172.21.15.44:0/4046456319 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe9580a09c0 0x7fe9580a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.931+0000 7fe971dcf700 1 --2- 172.21.15.44:0/4046456319 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe9580a09c0 0x7fe9580a2e80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0ca930 tx=0x7fe95c00bfb0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.934+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/763913286 shutdown_connections 2023-09-10T01:32:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.934+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/763913286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8e4b30 0x55652c8e6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.934+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/763913286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8d9360 0x55652c8d9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.934+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/763913286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8d9c80 0x55652c8e45f0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.934+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/763913286 >> 172.21.15.44:0/763913286 conn(0x55652c84f030 msgr2=0x55652c8d57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.940+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/763913286 shutdown_connections 2023-09-10T01:32:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.940+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/763913286 wait complete. 2023-09-10T01:32:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.940+0000 7fe97f9f8b80 1 -- start start 2023-09-10T01:32:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.940+0000 7fe97f9f8b80 1 Processor -- start 2023-09-10T01:32:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.941+0000 7fe97f9f8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8d9360 0x55652c8e4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.941+0000 7fe97f9f8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 0x55652c8e1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.941+0000 7fe97f9f8b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8e4b30 0x55652c8e1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.941+0000 7fe97f9f8b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55652c7cab10 con 0x55652c8e4b30 2023-09-10T01:32:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.941+0000 7fe97f9f8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55652c7ca810 con 0x55652c8d9c80 2023-09-10T01:32:54.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.941+0000 7fe97f9f8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55652c6ae8c0 con 0x55652c8d9360 2023-09-10T01:32:54.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.941+0000 7fe97f9f8b80 10 client.0 _ll_register_callbacks cb 0x55652c714a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:32:54.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.941+0000 7fe9715ce700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 0x55652c8e1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:54.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.941+0000 7fe9715ce700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 0x55652c8e1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34268/0 (socket says 172.21.15.44:34268) 2023-09-10T01:32:54.945 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.941+0000 7fe9715ce700 1 -- 172.21.15.44:0/1807088698 learned_addr learned my addr 172.21.15.44:0/1807088698 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.941+0000 7fe970dcd700 1 --2- 172.21.15.44:0/1807088698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8d9360 0x55652c8e4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.943+0000 7fe9715ce700 1 -- 172.21.15.44:0/1807088698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8d9360 msgr2=0x55652c8e4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.943+0000 7fe9715ce700 1 --2- 172.21.15.44:0/1807088698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8d9360 0x55652c8e4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.943+0000 7fe9715ce700 1 -- 172.21.15.44:0/1807088698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8e4b30 msgr2=0x55652c8e1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.943+0000 7fe9715ce700 1 --2- 172.21.15.44:0/1807088698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8e4b30 0x55652c8e1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.943+0000 7fe9715ce700 1 -- 172.21.15.44:0/1807088698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe964009a50 con 0x55652c8d9c80 2023-09-10T01:32:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.944+0000 7fe9715ce700 1 --2- 172.21.15.44:0/1807088698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 0x55652c8e1710 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe964007070 tx=0x7fe9640124e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2af68978139cd40e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.944+0000 7fe954ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:32:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.944+0000 7fe954ff9700 1 -- 172.21.15.44:0/1807088698 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96400e040 con 0x55652c8d9c80 2023-09-10T01:32:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.944+0000 7fe954ff9700 1 -- 172.21.15.44:0/1807088698 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(201..201 src has 1..201) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fe96401b830 con 0x55652c8d9c80 2023-09-10T01:32:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.944+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/1807088698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55652c784490 con 0x55652c8d9c80 2023-09-10T01:32:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.944+0000 7fe954ff9700 1 -- 172.21.15.44:0/1807088698 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=188) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe964012840 con 0x55652c8d9c80 2023-09-10T01:32:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.944+0000 7fe97f9f8b80 10 client.30203 fetch_fsmap learned FSMap version 188 2023-09-10T01:32:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.944+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/1807088698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=188,monmap=4+}) v3 -- 0x55652c641c00 con 0x55652c8d9c80 2023-09-10T01:32:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.945+0000 7fe954ff9700 1 -- 172.21.15.44:0/1807088698 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55652c784490 con 0x55652c8d9c80 2023-09-10T01:32:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.946+0000 7fe954ff9700 1 -- 172.21.15.44:0/1807088698 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 188) v1 ==== 7371+0+0 (secure 0 0 0) 0x7fe96401ba30 con 0x55652c8d9c80 2023-09-10T01:32:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.946+0000 7fe97f9f8b80 10 client.30203 fetch_fsmap finished waiting for FSMap version 188 2023-09-10T01:32:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.946+0000 7fe97f9f8b80 10 client.30203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:32:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.946+0000 7fe97f9f8b80 20 client.30203 populate_metadata read hostname 'smithi044' 2023-09-10T01:32:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.946+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/1807088698 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55652c8f43f0 0x55652c8f6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.946+0000 7fe97f9f8b80 4 client.30203 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:32:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.946+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/1807088698 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55652c841840 con 0x55652c8f43f0 2023-09-10T01:32:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.946+0000 7fe970dcd700 1 --2- 172.21.15.44:0/1807088698 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55652c8f43f0 0x55652c8f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.947+0000 7fe970dcd700 1 --2- 172.21.15.44:0/1807088698 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55652c8f43f0 0x55652c8f6810 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73a617b36c4d747 server_cookie=22cad50459a34624 in_seq=0 out_seq=0 2023-09-10T01:32:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.947+0000 7fe954ff9700 10 client.30203 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.947+0000 7fe954ff9700 1 -- 172.21.15.44:0/1807088698 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55652c841840 con 0x55652c8f43f0 2023-09-10T01:32:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.947+0000 7fe954ff9700 10 client.30203 handle_command_reply: tid=0 2023-09-10T01:32:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.948+0000 7fe97f9f8b80 1 client.30203 shutdown 2023-09-10T01:32:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.948+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/1807088698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 msgr2=0x55652c8e1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.948+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/1807088698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 0x55652c8e1710 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe964007070 tx=0x7fe9640124e0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.948+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/1807088698 shutdown_connections 2023-09-10T01:32:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.948+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/1807088698 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55652c8f43f0 0x55652c8f6810 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.948+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/1807088698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8e4b30 0x55652c8e1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.948+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/1807088698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8d9360 0x55652c8e4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.948+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/1807088698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 0x55652c8e1710 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.948+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/1807088698 >> 172.21.15.44:0/1807088698 conn(0x55652c84f030 msgr2=0x55652c8d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.948+0000 7fe954ff9700 0 client.30203 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.957+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/1807088698 shutdown_connections 2023-09-10T01:32:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.958+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/1807088698 wait complete. 2023-09-10T01:32:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.958+0000 7fe97f9f8b80 20 client.30203 trim_cache size 0 max 16384 2023-09-10T01:32:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.981+0000 7fe97f9f8b80 1 Processor -- start 2023-09-10T01:32:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.981+0000 7fe97f9f8b80 1 -- start start 2023-09-10T01:32:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.981+0000 7fe97f9f8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8d9360 0x55652c8e3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.981+0000 7fe97f9f8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 0x55652c8eb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.981+0000 7fe97f9f8b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8e4b30 0x55652c8de980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.981+0000 7fe97f9f8b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55652c8da450 con 0x55652c8e4b30 2023-09-10T01:32:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.981+0000 7fe97f9f8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55652c8d6210 con 0x55652c8d9c80 2023-09-10T01:32:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.981+0000 7fe97f9f8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55652c8cfa20 con 0x55652c8d9360 2023-09-10T01:32:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.981+0000 7fe970dcd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8e4b30 0x55652c8de980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.981+0000 7fe970dcd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8e4b30 0x55652c8de980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60772/0 (socket says 172.21.15.44:60772) 2023-09-10T01:32:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.981+0000 7fe9715ce700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 0x55652c8eb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.981+0000 7fe970dcd700 1 -- 172.21.15.44:0/663182865 learned_addr learned my addr 172.21.15.44:0/663182865 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.981+0000 7fe970dcd700 1 --2- 172.21.15.44:0/663182865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8d9360 0x55652c8e3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.982+0000 7fe9715ce700 1 -- 172.21.15.44:0/663182865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8d9360 msgr2=0x55652c8e3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.982+0000 7fe9715ce700 1 --2- 172.21.15.44:0/663182865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8d9360 0x55652c8e3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.982+0000 7fe9715ce700 1 -- 172.21.15.44:0/663182865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8e4b30 msgr2=0x55652c8de980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.982+0000 7fe9715ce700 1 --2- 172.21.15.44:0/663182865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8e4b30 0x55652c8de980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.982+0000 7fe9715ce700 1 -- 172.21.15.44:0/663182865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55652c8e7040 con 0x55652c8d9c80 2023-09-10T01:32:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.982+0000 7fe970dcd700 1 --2- 172.21.15.44:0/663182865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8e4b30 0x55652c8de980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:32:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.982+0000 7fe9715ce700 1 --2- 172.21.15.44:0/663182865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 0x55652c8eb510 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe96401bde0 tx=0x7fe96400bd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7feaab940fa461e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.982+0000 7fe9577fe700 1 -- 172.21.15.44:0/663182865 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96400e050 con 0x55652c8d9c80 2023-09-10T01:32:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.982+0000 7fe9577fe700 1 -- 172.21.15.44:0/663182865 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe96401f070 con 0x55652c8d9c80 2023-09-10T01:32:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.982+0000 7fe9577fe700 1 -- 172.21.15.44:0/663182865 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96401a940 con 0x55652c8d9c80 2023-09-10T01:32:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.983+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/663182865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 msgr2=0x55652c8eb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.983+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/663182865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 0x55652c8eb510 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe96401bde0 tx=0x7fe96400bd10 comp rx=0 tx=0).stop 2023-09-10T01:32:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.983+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/663182865 shutdown_connections 2023-09-10T01:32:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.983+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/663182865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8e4b30 0x55652c8de980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.983+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/663182865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8d9360 0x55652c8e3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.983+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/663182865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 0x55652c8eb510 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.983+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/663182865 >> 172.21.15.44:0/663182865 conn(0x55652c84f030 msgr2=0x55652c8d3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.983+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/663182865 shutdown_connections 2023-09-10T01:32:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.983+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/663182865 wait complete. 2023-09-10T01:32:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.983+0000 7fe97f9f8b80 1 -- start start 2023-09-10T01:32:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.983+0000 7fe97f9f8b80 1 Processor -- start 2023-09-10T01:32:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe97f9f8b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8d9360 0x55652c8e4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe97f9f8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 0x55652c8e5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe97f9f8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8e4b30 0x55652c8e5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe97f9f8b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55652c8e9660 con 0x55652c8d9360 2023-09-10T01:32:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe97f9f8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55652c8e97d0 con 0x55652c8d9c80 2023-09-10T01:32:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe97f9f8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55652c8e9990 con 0x55652c8e4b30 2023-09-10T01:32:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe97f9f8b80 10 client.0 _ll_register_callbacks cb 0x7fe964009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:32:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe970dcd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8d9360 0x55652c8e4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe970dcd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8d9360 0x55652c8e4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60774/0 (socket says 172.21.15.44:60774) 2023-09-10T01:32:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe970dcd700 1 -- 172.21.15.44:0/1871187502 learned_addr learned my addr 172.21.15.44:0/1871187502 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:32:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe970dcd700 1 --2- 172.21.15.44:0/1871187502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8e4b30 0x55652c8e5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe9715ce700 1 --2- 172.21.15.44:0/1871187502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 0x55652c8e5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe970dcd700 1 -- 172.21.15.44:0/1871187502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8e4b30 msgr2=0x55652c8e5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe970dcd700 1 --2- 172.21.15.44:0/1871187502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8e4b30 0x55652c8e5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe970dcd700 1 -- 172.21.15.44:0/1871187502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 msgr2=0x55652c8e5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:32:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe970dcd700 1 --2- 172.21.15.44:0/1871187502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 0x55652c8e5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe970dcd700 1 -- 172.21.15.44:0/1871187502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe968018910 con 0x55652c8d9360 2023-09-10T01:32:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe970dcd700 1 --2- 172.21.15.44:0/1871187502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8d9360 0x55652c8e4f10 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fe96c074300 tx=0x7fe9680068f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c48328b5ebdc68d server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:32:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.984+0000 7fe954ff9700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:32:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.985+0000 7fe954ff9700 1 -- 172.21.15.44:0/1871187502 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96801d070 con 0x55652c8d9360 2023-09-10T01:32:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.985+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/1871187502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe96400be50 con 0x55652c8d9360 2023-09-10T01:32:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.985+0000 7fe954ff9700 1 -- 172.21.15.44:0/1871187502 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(201..201 src has 1..201) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fe968006040 con 0x55652c8d9360 2023-09-10T01:32:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.985+0000 7fe954ff9700 1 -- 172.21.15.44:0/1871187502 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=188) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe968011040 con 0x55652c8d9360 2023-09-10T01:32:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.985+0000 7fe97f9f8b80 10 client.20565 fetch_fsmap learned FSMap version 188 2023-09-10T01:32:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.985+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/1871187502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=188,monmap=4+}) v3 -- 0x55652c641c00 con 0x55652c8d9360 2023-09-10T01:32:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.985+0000 7fe954ff9700 1 -- 172.21.15.44:0/1871187502 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe96400be50 con 0x55652c8d9360 2023-09-10T01:32:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.985+0000 7fe954ff9700 1 -- 172.21.15.44:0/1871187502 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 188) v1 ==== 7371+0+0 (secure 0 0 0) 0x7fe968025900 con 0x55652c8d9360 2023-09-10T01:32:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.985+0000 7fe97f9f8b80 10 client.20565 fetch_fsmap finished waiting for FSMap version 188 2023-09-10T01:32:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.985+0000 7fe97f9f8b80 10 client.20565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:32:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.985+0000 7fe97f9f8b80 20 client.20565 populate_metadata read hostname 'smithi044' 2023-09-10T01:32:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.985+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/1871187502 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55652c959eb0 0x55652c95c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:32:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.985+0000 7fe97f9f8b80 4 client.20565 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:32:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.985+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/1871187502 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55652c8258c0 con 0x55652c959eb0 2023-09-10T01:32:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.986+0000 7fe9715ce700 1 --2- 172.21.15.44:0/1871187502 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55652c959eb0 0x55652c95c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:32:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.986+0000 7fe954ff9700 10 client.20565 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:54.986+0000 7fe9715ce700 1 --2- 172.21.15.44:0/1871187502 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55652c959eb0 0x55652c95c2d0 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb00d31d5a48e238 server_cookie=85eaf9f473d6b316 in_seq=0 out_seq=0 2023-09-10T01:32:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.014+0000 7fe954ff9700 1 -- 172.21.15.44:0/1871187502 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55652c8258c0 con 0x55652c959eb0 2023-09-10T01:32:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.014+0000 7fe954ff9700 10 client.20565 handle_command_reply: tid=0 2023-09-10T01:32:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.016+0000 7fe97f9f8b80 1 client.20565 shutdown 2023-09-10T01:32:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.020+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/1871187502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8d9360 msgr2=0x55652c8e4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.020+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/1871187502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8d9360 0x55652c8e4f10 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fe96c074300 tx=0x7fe9680068f0 comp rx=0 tx=0).stop 2023-09-10T01:32:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.020+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/1871187502 shutdown_connections 2023-09-10T01:32:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.020+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/1871187502 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55652c959eb0 0x55652c95c2d0 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.020+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/1871187502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55652c8e4b30 0x55652c8e5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.020+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/1871187502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55652c8d9360 0x55652c8e4f10 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.020+0000 7fe97f9f8b80 1 --2- 172.21.15.44:0/1871187502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55652c8d9c80 0x55652c8e5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.020+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/1871187502 >> 172.21.15.44:0/1871187502 conn(0x55652c84f030 msgr2=0x55652c8db390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.020+0000 7fe954ff9700 0 client.20565 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:32:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.027+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/1871187502 shutdown_connections 2023-09-10T01:32:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.027+0000 7fe97f9f8b80 1 -- 172.21.15.44:0/1871187502 wait complete. 2023-09-10T01:32:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.037+0000 7fe97f9f8b80 20 client.20565 trim_cache size 0 max 16384 2023-09-10T01:32:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:32:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:32:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:32:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:32:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.038+0000 7fe957fff700 1 -- 172.21.15.44:0/4046456319 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe9580a09c0 msgr2=0x7fe9580a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.038+0000 7fe957fff700 1 --2- 172.21.15.44:0/4046456319 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe9580a09c0 0x7fe9580a2e80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0ca930 tx=0x7fe95c00bfb0 comp rx=0 tx=0).stop 2023-09-10T01:32:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.038+0000 7fe957fff700 1 -- 172.21.15.44:0/4046456319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96c119de0 msgr2=0x7fe96c072920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:32:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.038+0000 7fe957fff700 1 --2- 172.21.15.44:0/4046456319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96c119de0 0x7fe96c072920 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe95c006010 tx=0x7fe95c0038b0 comp rx=0 tx=0).stop 2023-09-10T01:32:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.038+0000 7fe957fff700 1 -- 172.21.15.44:0/4046456319 shutdown_connections 2023-09-10T01:32:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.038+0000 7fe957fff700 1 --2- 172.21.15.44:0/4046456319 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe9580a09c0 0x7fe9580a2e80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.038+0000 7fe957fff700 1 --2- 172.21.15.44:0/4046456319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96c119de0 0x7fe96c072920 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.038+0000 7fe957fff700 1 --2- 172.21.15.44:0/4046456319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe96c10ed50 0x7fe96c077a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.038+0000 7fe957fff700 1 --2- 172.21.15.44:0/4046456319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe96c10f1c0 0x7fe96c077fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:32:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.038+0000 7fe957fff700 1 -- 172.21.15.44:0/4046456319 >> 172.21.15.44:0/4046456319 conn(0x7fe96c06c6c0 msgr2=0x7fe96c117dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:32:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.041+0000 7fe957fff700 1 -- 172.21.15.44:0/4046456319 shutdown_connections 2023-09-10T01:32:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:32:55.051+0000 7fe957fff700 1 -- 172.21.15.44:0/4046456319 wait complete. 2023-09-10T01:32:56.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:55 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:32:56.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:55 smithi131 ceph-mon[103713]: pgmap v1702: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 768 op/s 2023-09-10T01:32:56.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:55 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:32:56.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:55 smithi044 ceph-mon[98676]: pgmap v1702: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 768 op/s 2023-09-10T01:32:56.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:55 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:32:56.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:55 smithi099 ceph-mon[102692]: pgmap v1702: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 768 op/s 2023-09-10T01:32:56.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 710.0 s, 255.0 tps, lat 406.716 ms stddev 378.076 2023-09-10T01:32:58.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:57 smithi131 ceph-mon[103713]: pgmap v1703: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 986 op/s 2023-09-10T01:32:58.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:57 smithi044 ceph-mon[98676]: pgmap v1703: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 986 op/s 2023-09-10T01:32:58.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:57 smithi099 ceph-mon[102692]: pgmap v1703: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 986 op/s 2023-09-10T01:33:00.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:32:59 smithi131 ceph-mon[103713]: pgmap v1704: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 805 op/s 2023-09-10T01:33:00.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:32:59 smithi044 ceph-mon[98676]: pgmap v1704: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 805 op/s 2023-09-10T01:33:00.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:32:59 smithi099 ceph-mon[102692]: pgmap v1704: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 805 op/s 2023-09-10T01:33:01.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 715.0 s, 218.6 tps, lat 404.936 ms stddev 148.914 2023-09-10T01:33:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:00 smithi131 ceph-mon[103713]: pgmap v1705: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.7 MiB/s wr, 747 op/s 2023-09-10T01:33:01.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:00 smithi044 ceph-mon[98676]: pgmap v1705: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.7 MiB/s wr, 747 op/s 2023-09-10T01:33:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:00 smithi099 ceph-mon[102692]: pgmap v1705: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.7 MiB/s wr, 747 op/s 2023-09-10T01:33:04.301 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:03 smithi131 ceph-mon[103713]: pgmap v1706: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.9 MiB/s wr, 999 op/s 2023-09-10T01:33:04.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:03 smithi044 ceph-mon[98676]: pgmap v1706: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.9 MiB/s wr, 999 op/s 2023-09-10T01:33:04.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:03 smithi099 ceph-mon[102692]: pgmap v1706: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.9 MiB/s wr, 999 op/s 2023-09-10T01:33:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 720.0 s, 266.4 tps, lat 364.395 ms stddev 667.715 2023-09-10T01:33:06.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:05 smithi131 ceph-mon[103713]: pgmap v1707: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.8 MiB/s wr, 756 op/s 2023-09-10T01:33:06.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:05 smithi044 ceph-mon[98676]: pgmap v1707: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.8 MiB/s wr, 756 op/s 2023-09-10T01:33:06.364 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:05 smithi099 ceph-mon[102692]: pgmap v1707: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.8 MiB/s wr, 756 op/s 2023-09-10T01:33:08.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:07 smithi131 ceph-mon[103713]: pgmap v1708: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 908 op/s 2023-09-10T01:33:08.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:07 smithi044 ceph-mon[98676]: pgmap v1708: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 908 op/s 2023-09-10T01:33:08.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:07 smithi099 ceph-mon[102692]: pgmap v1708: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 908 op/s 2023-09-10T01:33:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.275+0000 7fb39e41f700 1 -- 172.21.15.44:0/3991537678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb398119de0 msgr2=0x7fb39811c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.275+0000 7fb39e41f700 1 --2- 172.21.15.44:0/3991537678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb398119de0 0x7fb39811c1d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb39001c0d0 tx=0x7fb39001c3e0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.275+0000 7fb39e41f700 1 -- 172.21.15.44:0/3991537678 shutdown_connections 2023-09-10T01:33:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.275+0000 7fb39e41f700 1 --2- 172.21.15.44:0/3991537678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb398119de0 0x7fb39811c1d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.275+0000 7fb39e41f700 1 --2- 172.21.15.44:0/3991537678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39810ed50 0x7fb398107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.275+0000 7fb39e41f700 1 --2- 172.21.15.44:0/3991537678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb39810f1c0 0x7fb398107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.275+0000 7fb39e41f700 1 -- 172.21.15.44:0/3991537678 >> 172.21.15.44:0/3991537678 conn(0x7fb39806c6c0 msgr2=0x7fb39806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.302+0000 7fb39e41f700 1 -- 172.21.15.44:0/3991537678 shutdown_connections 2023-09-10T01:33:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.311+0000 7fb39e41f700 1 -- 172.21.15.44:0/3991537678 wait complete. 2023-09-10T01:33:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.311+0000 7fb39e41f700 1 Processor -- start 2023-09-10T01:33:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.312+0000 7fb39e41f700 1 -- start start 2023-09-10T01:33:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.312+0000 7fb39e41f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39810ed50 0x7fb398114f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.312+0000 7fb39e41f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb39810f1c0 0x7fb398115470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.312+0000 7fb39e41f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb398119de0 0x7fb3981a8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.312+0000 7fb39e41f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3981a9070 con 0x7fb398119de0 2023-09-10T01:33:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.312+0000 7fb39e41f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3981a91b0 con 0x7fb39810ed50 2023-09-10T01:33:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.312+0000 7fb39e41f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3981a9370 con 0x7fb39810f1c0 2023-09-10T01:33:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.313+0000 7fb397fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39810ed50 0x7fb398114f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.313+0000 7fb397fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39810ed50 0x7fb398114f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34298/0 (socket says 172.21.15.44:34298) 2023-09-10T01:33:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.313+0000 7fb397fff700 1 -- 172.21.15.44:0/1991031410 learned_addr learned my addr 172.21.15.44:0/1991031410 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.316+0000 7fb3977fe700 1 --2- 172.21.15.44:0/1991031410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb39810f1c0 0x7fb398115470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.316+0000 7fb3977fe700 1 -- 172.21.15.44:0/1991031410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39810ed50 msgr2=0x7fb398114f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.316+0000 7fb3977fe700 1 --2- 172.21.15.44:0/1991031410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39810ed50 0x7fb398114f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.316+0000 7fb3977fe700 1 -- 172.21.15.44:0/1991031410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb398119de0 msgr2=0x7fb3981a8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.316+0000 7fb3977fe700 1 --2- 172.21.15.44:0/1991031410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb398119de0 0x7fb3981a8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.316+0000 7fb3977fe700 1 -- 172.21.15.44:0/1991031410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39001ad10 con 0x7fb39810f1c0 2023-09-10T01:33:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.316+0000 7fb3977fe700 1 --2- 172.21.15.44:0/1991031410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb39810f1c0 0x7fb398115470 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00b470 tx=0x7fb38c00b780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.324+0000 7fb3957fa700 1 -- 172.21.15.44:0/1991031410 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c011830 con 0x7fb39810f1c0 2023-09-10T01:33:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.324+0000 7fb3957fa700 1 -- 172.21.15.44:0/1991031410 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb38c011e20 con 0x7fb39810f1c0 2023-09-10T01:33:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.324+0000 7fb3957fa700 1 -- 172.21.15.44:0/1991031410 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c010ce0 con 0x7fb39810f1c0 2023-09-10T01:33:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.330+0000 7fb39e41f700 1 -- 172.21.15.44:0/1991031410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3981a9650 con 0x7fb39810f1c0 2023-09-10T01:33:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.330+0000 7fb39e41f700 1 -- 172.21.15.44:0/1991031410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3981a9b60 con 0x7fb39810f1c0 2023-09-10T01:33:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.335+0000 7fb3957fa700 1 -- 172.21.15.44:0/1991031410 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fb38c011990 con 0x7fb39810f1c0 2023-09-10T01:33:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.336+0000 7fb3957fa700 1 --2- 172.21.15.44:0/1991031410 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb3800a0830 0x7fb3800a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.336+0000 7fb3957fa700 1 -- 172.21.15.44:0/1991031410 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fb38c0cba00 con 0x7fb39810f1c0 2023-09-10T01:33:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.338+0000 7fb3aa5e5b80 1 Processor -- start 2023-09-10T01:33:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.340+0000 7fb397fff700 1 --2- 172.21.15.44:0/1991031410 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb3800a0830 0x7fb3800a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.341+0000 7fb3aa5e5b80 1 -- start start 2023-09-10T01:33:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.341+0000 7fb3aa5e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2e21a0 0x55af3d2e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.341+0000 7fb3aa5e5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2dc300 0x55af3d2dc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.341+0000 7fb3aa5e5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 0x55af3d2e8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.341+0000 7fb3aa5e5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3d2356d0 con 0x55af3d2dc300 2023-09-10T01:33:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.341+0000 7fb3aa5e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3d232ff0 con 0x55af3d2e21a0 2023-09-10T01:33:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.341+0000 7fb3aa5e5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3d22e8e0 con 0x55af3d2dccc0 2023-09-10T01:33:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.342+0000 7fb39c9bc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 0x55af3d2e8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.342+0000 7fb39c9bc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 0x55af3d2e8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39446/0 (socket says 172.21.15.44:39446) 2023-09-10T01:33:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.342+0000 7fb39c9bc700 1 -- 172.21.15.44:0/1673212417 learned_addr learned my addr 172.21.15.44:0/1673212417 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.347+0000 7fb397fff700 1 --2- 172.21.15.44:0/1673212417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2e21a0 0x55af3d2e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.351+0000 7fb3977fe700 1 --2- 172.21.15.44:0/1673212417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2dc300 0x55af3d2dc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.351+0000 7fb397fff700 1 -- 172.21.15.44:0/1673212417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 msgr2=0x55af3d2e8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.351+0000 7fb397fff700 1 --2- 172.21.15.44:0/1673212417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 0x55af3d2e8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.351+0000 7fb397fff700 1 -- 172.21.15.44:0/1673212417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2dc300 msgr2=0x55af3d2dc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.351+0000 7fb397fff700 1 --2- 172.21.15.44:0/1673212417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2dc300 0x55af3d2dc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.351+0000 7fb397fff700 1 -- 172.21.15.44:0/1673212417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af3d05ad80 con 0x55af3d2e21a0 2023-09-10T01:33:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.351+0000 7fb3977fe700 1 --2- 172.21.15.44:0/1673212417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2dc300 0x55af3d2dc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:33:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.351+0000 7fb397fff700 1 --2- 172.21.15.44:0/1991031410 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb3800a0830 0x7fb3800a2cf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb388009f80 tx=0x7fb388015690 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.358+0000 7fb397fff700 1 --2- 172.21.15.44:0/1673212417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2e21a0 0x55af3d2e2580 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb3880095d0 tx=0x7fb38801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecfb48a416d2b0a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.362+0000 7fb37e7fc700 1 -- 172.21.15.44:0/1673212417 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388018c50 con 0x55af3d2e21a0 2023-09-10T01:33:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.362+0000 7fb37e7fc700 1 -- 172.21.15.44:0/1673212417 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb388018db0 con 0x55af3d2e21a0 2023-09-10T01:33:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.362+0000 7fb37e7fc700 1 -- 172.21.15.44:0/1673212417 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388012680 con 0x55af3d2e21a0 2023-09-10T01:33:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.363+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/1673212417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2e21a0 msgr2=0x55af3d2e2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.363+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/1673212417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2e21a0 0x55af3d2e2580 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb3880095d0 tx=0x7fb38801d000 comp rx=0 tx=0).stop 2023-09-10T01:33:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.363+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/1673212417 shutdown_connections 2023-09-10T01:33:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.363+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/1673212417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 0x55af3d2e8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.363+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/1673212417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2e21a0 0x55af3d2e2580 secure :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb3880095d0 tx=0x7fb38801d000 comp rx=0 tx=0).stop 2023-09-10T01:33:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.363+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/1673212417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2dc300 0x55af3d2dc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.363+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/1673212417 >> 172.21.15.44:0/1673212417 conn(0x55af3d251030 msgr2=0x55af3d2d7560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.364+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/1673212417 shutdown_connections 2023-09-10T01:33:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.364+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/1673212417 wait complete. 2023-09-10T01:33:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.365+0000 7fb3aa5e5b80 1 -- start start 2023-09-10T01:33:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.365+0000 7fb3aa5e5b80 1 Processor -- start 2023-09-10T01:33:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.365+0000 7fb3aa5e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2dc300 0x55af3d2ec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.365+0000 7fb3aa5e5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2dccc0 0x55af3d2ec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.365+0000 7fb3aa5e5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2e6370 0x55af3d2e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.365+0000 7fb3aa5e5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3d1ccb10 con 0x55af3d2dccc0 2023-09-10T01:33:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.365+0000 7fb3aa5e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3d1cc810 con 0x55af3d2dc300 2023-09-10T01:33:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.365+0000 7fb3aa5e5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3d0b08c0 con 0x55af3d2e6370 2023-09-10T01:33:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.366+0000 7fb3aa5e5b80 10 client.0 _ll_register_callbacks cb 0x55af3d116a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:33:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.366+0000 7fb39c9bc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2e6370 0x55af3d2e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.366+0000 7fb39c9bc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2e6370 0x55af3d2e67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39452/0 (socket says 172.21.15.44:39452) 2023-09-10T01:33:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.366+0000 7fb39c9bc700 1 -- 172.21.15.44:0/2392158189 learned_addr learned my addr 172.21.15.44:0/2392158189 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.366+0000 7fb3977fe700 1 --2- 172.21.15.44:0/2392158189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2dccc0 0x55af3d2ec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.379+0000 7fb39c9bc700 1 -- 172.21.15.44:0/2392158189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2dc300 msgr2=0x55af3d2ec070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:33:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.379+0000 7fb39c9bc700 1 --2- 172.21.15.44:0/2392158189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2dc300 0x55af3d2ec070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.379+0000 7fb39c9bc700 1 -- 172.21.15.44:0/2392158189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2dccc0 msgr2=0x55af3d2ec5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.379+0000 7fb39c9bc700 1 --2- 172.21.15.44:0/2392158189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2dccc0 0x55af3d2ec5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.379+0000 7fb39c9bc700 1 -- 172.21.15.44:0/2392158189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb388015d40 con 0x55af3d2e6370 2023-09-10T01:33:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.394+0000 7fb39c9bc700 1 --2- 172.21.15.44:0/2392158189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2e6370 0x55af3d2e67f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb390003de0 tx=0x7fb390003e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a82967094bb41d48 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.396+0000 7fb3777fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:33:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.401+0000 7fb3777fe700 1 -- 172.21.15.44:0/2392158189 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39001f040 con 0x55af3d2e6370 2023-09-10T01:33:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.401+0000 7fb3777fe700 1 -- 172.21.15.44:0/2392158189 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(201..201 src has 1..201) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fb390018980 con 0x55af3d2e6370 2023-09-10T01:33:08.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.404+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/2392158189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af3d186490 con 0x55af3d2e6370 2023-09-10T01:33:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.410+0000 7fb3777fe700 1 -- 172.21.15.44:0/2392158189 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=188) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3900050d0 con 0x55af3d2e6370 2023-09-10T01:33:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.416+0000 7fb3aa5e5b80 10 client.30238 fetch_fsmap learned FSMap version 188 2023-09-10T01:33:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.416+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/2392158189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=188,monmap=4+}) v3 -- 0x55af3d043c00 con 0x55af3d2e6370 2023-09-10T01:33:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.428+0000 7fb3777fe700 1 -- 172.21.15.44:0/2392158189 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb39003b030 con 0x55af3d2e6370 2023-09-10T01:33:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.428+0000 7fb3777fe700 1 -- 172.21.15.44:0/2392158189 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 188) v1 ==== 7371+0+0 (secure 0 0 0) 0x7fb390035060 con 0x55af3d2e6370 2023-09-10T01:33:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.429+0000 7fb3aa5e5b80 10 client.30238 fetch_fsmap finished waiting for FSMap version 188 2023-09-10T01:33:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.429+0000 7fb3aa5e5b80 10 client.30238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:33:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.429+0000 7fb3aa5e5b80 20 client.30238 populate_metadata read hostname 'smithi044' 2023-09-10T01:33:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.429+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/2392158189 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af3d2f69d0 0x55af3d2f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.429+0000 7fb3aa5e5b80 4 client.30238 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:33:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.429+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/2392158189 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af3d243840 con 0x55af3d2f69d0 2023-09-10T01:33:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.433+0000 7fb397fff700 1 --2- 172.21.15.44:0/2392158189 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af3d2f69d0 0x55af3d2f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.460+0000 7fb397fff700 1 --2- 172.21.15.44:0/2392158189 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af3d2f69d0 0x55af3d2f8df0 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a53f1b1a8f1f1bda server_cookie=7998e97765ad6804 in_seq=0 out_seq=0 2023-09-10T01:33:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.464+0000 7fb3777fe700 10 client.30238 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:33:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.465+0000 7fb3777fe700 1 -- 172.21.15.44:0/2392158189 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55af3d243840 con 0x55af3d2f69d0 2023-09-10T01:33:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.465+0000 7fb3777fe700 10 client.30238 handle_command_reply: tid=0 2023-09-10T01:33:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.466+0000 7fb3aa5e5b80 1 client.30238 shutdown 2023-09-10T01:33:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.469+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/2392158189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2e6370 msgr2=0x55af3d2e67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.469+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/2392158189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2e6370 0x55af3d2e67f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb390003de0 tx=0x7fb390003e10 comp rx=0 tx=0).stop 2023-09-10T01:33:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.474+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/2392158189 shutdown_connections 2023-09-10T01:33:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.474+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/2392158189 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af3d2f69d0 0x55af3d2f8df0 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.474+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/2392158189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2e6370 0x55af3d2e67f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.474+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/2392158189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2dc300 0x55af3d2ec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.474+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/2392158189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2dccc0 0x55af3d2ec5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.474+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/2392158189 >> 172.21.15.44:0/2392158189 conn(0x55af3d251030 msgr2=0x55af3d2d67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.474+0000 7fb3777fe700 0 client.30238 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:33:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.481+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/2392158189 shutdown_connections 2023-09-10T01:33:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.485+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/2392158189 wait complete. 2023-09-10T01:33:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.497+0000 7fb3aa5e5b80 20 client.30238 trim_cache size 0 max 16384 2023-09-10T01:33:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.529+0000 7fb3aa5e5b80 1 Processor -- start 2023-09-10T01:33:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.538+0000 7fb3aa5e5b80 1 -- start start 2023-09-10T01:33:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.538+0000 7fb3aa5e5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2dc300 0x55af3d2e3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.538+0000 7fb3aa5e5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 0x55af3d2e0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.538+0000 7fb3aa5e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2e6370 0x55af3d2e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.538+0000 7fb3aa5e5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3d2f3fe0 con 0x55af3d2dc300 2023-09-10T01:33:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.538+0000 7fb3aa5e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3d2d3fc0 con 0x55af3d2e6370 2023-09-10T01:33:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.538+0000 7fb3aa5e5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3d2d7a90 con 0x55af3d2dccc0 2023-09-10T01:33:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.538+0000 7fb3977fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 0x55af3d2e0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.538+0000 7fb3977fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 0x55af3d2e0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39458/0 (socket says 172.21.15.44:39458) 2023-09-10T01:33:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.538+0000 7fb3977fe700 1 -- 172.21.15.44:0/206136123 learned_addr learned my addr 172.21.15.44:0/206136123 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.545+0000 7fb39c9bc700 1 --2- 172.21.15.44:0/206136123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2e6370 0x55af3d2e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.545+0000 7fb3977fe700 1 -- 172.21.15.44:0/206136123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2e6370 msgr2=0x55af3d2e0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.545+0000 7fb3977fe700 1 --2- 172.21.15.44:0/206136123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2e6370 0x55af3d2e0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.545+0000 7fb3977fe700 1 -- 172.21.15.44:0/206136123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2dc300 msgr2=0x55af3d2e3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.545+0000 7fb3977fe700 1 --2- 172.21.15.44:0/206136123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2dc300 0x55af3d2e3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.545+0000 7fb3977fe700 1 -- 172.21.15.44:0/206136123 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af3d2fda20 con 0x55af3d2dccc0 2023-09-10T01:33:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.559+0000 7fb3977fe700 1 --2- 172.21.15.44:0/206136123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 0x55af3d2e0630 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0955e0 tx=0x7fb38c011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30ca29fc0deccc30 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.577+0000 7fb37effd700 1 -- 172.21.15.44:0/206136123 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c098b70 con 0x55af3d2dccc0 2023-09-10T01:33:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.577+0000 7fb37effd700 1 -- 172.21.15.44:0/206136123 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb38c09c9a0 con 0x55af3d2dccc0 2023-09-10T01:33:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.577+0000 7fb37effd700 1 -- 172.21.15.44:0/206136123 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c096ff0 con 0x55af3d2dccc0 2023-09-10T01:33:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.581+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/206136123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 msgr2=0x55af3d2e0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.581+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/206136123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 0x55af3d2e0630 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0955e0 tx=0x7fb38c011f90 comp rx=0 tx=0).stop 2023-09-10T01:33:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.594+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/206136123 shutdown_connections 2023-09-10T01:33:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.594+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/206136123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2e6370 0x55af3d2e0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.594+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/206136123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2dc300 0x55af3d2e3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.594+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/206136123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 0x55af3d2e0630 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.594+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/206136123 >> 172.21.15.44:0/206136123 conn(0x55af3d251030 msgr2=0x55af3d2dfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.603+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/206136123 shutdown_connections 2023-09-10T01:33:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.604+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/206136123 wait complete. 2023-09-10T01:33:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.604+0000 7fb3aa5e5b80 1 -- start start 2023-09-10T01:33:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.604+0000 7fb3aa5e5b80 1 Processor -- start 2023-09-10T01:33:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.605+0000 7fb3aa5e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2dc300 0x55af3d2e9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.605+0000 7fb3aa5e5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 0x55af3d2ea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.605+0000 7fb3aa5e5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2e6370 0x55af3d349800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.605+0000 7fb3aa5e5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3d2eabd0 con 0x55af3d2e6370 2023-09-10T01:33:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.605+0000 7fb3aa5e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3d349d40 con 0x55af3d2dc300 2023-09-10T01:33:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.605+0000 7fb3aa5e5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3d349e80 con 0x55af3d2dccc0 2023-09-10T01:33:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.605+0000 7fb3aa5e5b80 10 client.0 _ll_register_callbacks cb 0x7fb39001a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:33:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.607+0000 7fb39c9bc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2e6370 0x55af3d349800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.607+0000 7fb39c9bc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2e6370 0x55af3d349800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60818/0 (socket says 172.21.15.44:60818) 2023-09-10T01:33:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.607+0000 7fb39c9bc700 1 -- 172.21.15.44:0/4036935719 learned_addr learned my addr 172.21.15.44:0/4036935719 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.611+0000 7fb397fff700 1 --2- 172.21.15.44:0/4036935719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2dc300 0x55af3d2e9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.613+0000 7fb3977fe700 1 --2- 172.21.15.44:0/4036935719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 0x55af3d2ea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.613+0000 7fb39c9bc700 1 -- 172.21.15.44:0/4036935719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 msgr2=0x55af3d2ea2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.613+0000 7fb39c9bc700 1 --2- 172.21.15.44:0/4036935719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 0x55af3d2ea2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.613+0000 7fb39c9bc700 1 -- 172.21.15.44:0/4036935719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2dc300 msgr2=0x55af3d2e9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.613+0000 7fb39c9bc700 1 --2- 172.21.15.44:0/4036935719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2dc300 0x55af3d2e9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.613+0000 7fb39c9bc700 1 -- 172.21.15.44:0/4036935719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3981a9b60 con 0x55af3d2e6370 2023-09-10T01:33:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.627+0000 7fb39c9bc700 1 --2- 172.21.15.44:0/4036935719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2e6370 0x55af3d349800 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fb39002c830 tx=0x7fb390006110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258ad3d264865a82 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.627+0000 7fb3777fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:33:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.643+0000 7fb3777fe700 1 -- 172.21.15.44:0/4036935719 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39001f070 con 0x55af3d2e6370 2023-09-10T01:33:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.643+0000 7fb3777fe700 1 -- 172.21.15.44:0/4036935719 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(201..201 src has 1..201) v4 ==== 9524+0+0 (secure 0 0 0) 0x7fb390005150 con 0x55af3d2e6370 2023-09-10T01:33:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.644+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/4036935719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb388018db0 con 0x55af3d2e6370 2023-09-10T01:33:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.651+0000 7fb3777fe700 1 -- 172.21.15.44:0/4036935719 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=188) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb39002ce70 con 0x55af3d2e6370 2023-09-10T01:33:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.651+0000 7fb3aa5e5b80 10 client.20574 fetch_fsmap learned FSMap version 188 2023-09-10T01:33:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.651+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/4036935719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=188,monmap=4+}) v3 -- 0x55af3d0811f0 con 0x55af3d2e6370 2023-09-10T01:33:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.663+0000 7fb3777fe700 1 -- 172.21.15.44:0/4036935719 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb388018db0 con 0x55af3d2e6370 2023-09-10T01:33:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.663+0000 7fb3777fe700 1 -- 172.21.15.44:0/4036935719 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 188) v1 ==== 7371+0+0 (secure 0 0 0) 0x7fb390035070 con 0x55af3d2e6370 2023-09-10T01:33:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.663+0000 7fb3aa5e5b80 10 client.20574 fetch_fsmap finished waiting for FSMap version 188 2023-09-10T01:33:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.663+0000 7fb3aa5e5b80 10 client.20574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:33:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.663+0000 7fb3aa5e5b80 20 client.20574 populate_metadata read hostname 'smithi044' 2023-09-10T01:33:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.663+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/4036935719 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af3d34b280 0x55af3d34d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.663+0000 7fb3aa5e5b80 4 client.20574 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:33:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.663+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/4036935719 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55af3d2278c0 con 0x55af3d34b280 2023-09-10T01:33:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.668+0000 7fb397fff700 1 --2- 172.21.15.44:0/4036935719 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af3d34b280 0x55af3d34d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.695+0000 7fb3777fe700 10 client.20574 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:33:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:08.695+0000 7fb397fff700 1 --2- 172.21.15.44:0/4036935719 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af3d34b280 0x55af3d34d6a0 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8606e8fc473ae39 server_cookie=67af3cd7a4cb8f64 in_seq=0 out_seq=0 2023-09-10T01:33:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.000+0000 7fb3777fe700 1 -- 172.21.15.44:0/4036935719 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55af3d2278c0 con 0x55af3d34b280 2023-09-10T01:33:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.000+0000 7fb3777fe700 10 client.20574 handle_command_reply: tid=0 2023-09-10T01:33:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.000+0000 7fb3aa5e5b80 1 client.20574 shutdown 2023-09-10T01:33:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.006+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/4036935719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2e6370 msgr2=0x55af3d349800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.006+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/4036935719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2e6370 0x55af3d349800 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fb39002c830 tx=0x7fb390006110 comp rx=0 tx=0).stop 2023-09-10T01:33:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.012+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/4036935719 shutdown_connections 2023-09-10T01:33:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.012+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/4036935719 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af3d34b280 0x55af3d34d6a0 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.012+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/4036935719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af3d2e6370 0x55af3d349800 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.012+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/4036935719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af3d2dc300 0x55af3d2e9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.012+0000 7fb3aa5e5b80 1 --2- 172.21.15.44:0/4036935719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af3d2dccc0 0x55af3d2ea2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.012+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/4036935719 >> 172.21.15.44:0/4036935719 conn(0x55af3d251030 msgr2=0x55af3d2dfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.012+0000 7fb3777fe700 0 client.20574 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:33:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.020+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/4036935719 shutdown_connections 2023-09-10T01:33:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.020+0000 7fb3aa5e5b80 1 -- 172.21.15.44:0/4036935719 wait complete. 2023-09-10T01:33:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.020+0000 7fb3aa5e5b80 20 client.20574 trim_cache size 0 max 16384 2023-09-10T01:33:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:33:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:33:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:33:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:33:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.026+0000 7fb39e41f700 1 -- 172.21.15.44:0/1991031410 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb3800a0830 msgr2=0x7fb3800a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.026+0000 7fb39e41f700 1 --2- 172.21.15.44:0/1991031410 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb3800a0830 0x7fb3800a2cf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb388009f80 tx=0x7fb388015690 comp rx=0 tx=0).stop 2023-09-10T01:33:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.026+0000 7fb39e41f700 1 -- 172.21.15.44:0/1991031410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb39810f1c0 msgr2=0x7fb398115470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.026+0000 7fb39e41f700 1 --2- 172.21.15.44:0/1991031410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb39810f1c0 0x7fb398115470 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00b470 tx=0x7fb38c00b780 comp rx=0 tx=0).stop 2023-09-10T01:33:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.026+0000 7fb39e41f700 1 -- 172.21.15.44:0/1991031410 shutdown_connections 2023-09-10T01:33:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.026+0000 7fb39e41f700 1 --2- 172.21.15.44:0/1991031410 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb3800a0830 0x7fb3800a2cf0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.026+0000 7fb39e41f700 1 --2- 172.21.15.44:0/1991031410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb398119de0 0x7fb3981a8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.026+0000 7fb39e41f700 1 --2- 172.21.15.44:0/1991031410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39810ed50 0x7fb398114f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.026+0000 7fb39e41f700 1 --2- 172.21.15.44:0/1991031410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb39810f1c0 0x7fb398115470 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.026+0000 7fb39e41f700 1 -- 172.21.15.44:0/1991031410 >> 172.21.15.44:0/1991031410 conn(0x7fb39806c6c0 msgr2=0x7fb39811b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.026+0000 7fb39e41f700 1 -- 172.21.15.44:0/1991031410 shutdown_connections 2023-09-10T01:33:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:09.026+0000 7fb39e41f700 1 -- 172.21.15.44:0/1991031410 wait complete. 2023-09-10T01:33:10.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:09 smithi131 ceph-mon[103713]: osdmap e202: 12 total, 12 up, 12 in 2023-09-10T01:33:10.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:09 smithi131 ceph-mon[103713]: pgmap v1710: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.8 MiB/s wr, 761 op/s 2023-09-10T01:33:10.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:09 smithi044 ceph-mon[98676]: osdmap e202: 12 total, 12 up, 12 in 2023-09-10T01:33:10.059 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:09 smithi044 ceph-mon[98676]: pgmap v1710: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.8 MiB/s wr, 761 op/s 2023-09-10T01:33:10.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:09 smithi099 ceph-mon[102692]: osdmap e202: 12 total, 12 up, 12 in 2023-09-10T01:33:10.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:09 smithi099 ceph-mon[102692]: pgmap v1710: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.8 MiB/s wr, 761 op/s 2023-09-10T01:33:11.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 725.0 s, 93.8 tps, lat 754.629 ms stddev 996.237 2023-09-10T01:33:12.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:11 smithi131 ceph-mon[103713]: pgmap v1711: 129 pgs: 3 active+clean+snaptrim, 126 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.8 MiB/s wr, 746 op/s 2023-09-10T01:33:12.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:11 smithi044 ceph-mon[98676]: pgmap v1711: 129 pgs: 3 active+clean+snaptrim, 126 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.8 MiB/s wr, 746 op/s 2023-09-10T01:33:12.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:11 smithi099 ceph-mon[102692]: pgmap v1711: 129 pgs: 3 active+clean+snaptrim, 126 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.8 MiB/s wr, 746 op/s 2023-09-10T01:33:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:12 smithi131 ceph-mon[103713]: pgmap v1712: 129 pgs: 49 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 62 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 561 op/s 2023-09-10T01:33:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:12 smithi044 ceph-mon[98676]: pgmap v1712: 129 pgs: 49 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 62 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 561 op/s 2023-09-10T01:33:13.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:12 smithi099 ceph-mon[102692]: pgmap v1712: 129 pgs: 49 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 62 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 561 op/s 2023-09-10T01:33:16.184 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 730.0 s, 77.9 tps, lat 1420.527 ms stddev 2280.707 2023-09-10T01:33:16.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:15 smithi131 ceph-mon[103713]: pgmap v1713: 129 pgs: 52 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 57 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 523 op/s 2023-09-10T01:33:16.312 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:15 smithi044 ceph-mon[98676]: pgmap v1713: 129 pgs: 52 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 57 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 523 op/s 2023-09-10T01:33:16.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:15 smithi099 ceph-mon[102692]: pgmap v1713: 129 pgs: 52 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 57 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 523 op/s 2023-09-10T01:33:18.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:17 smithi131 ceph-mon[103713]: pgmap v1714: 129 pgs: 22 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 97 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 358 op/s 2023-09-10T01:33:18.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:17 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:33:18.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:17 smithi044 ceph-mon[98676]: pgmap v1714: 129 pgs: 22 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 97 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 358 op/s 2023-09-10T01:33:18.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:17 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:33:18.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:17 smithi099 ceph-mon[102692]: pgmap v1714: 129 pgs: 22 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 97 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 358 op/s 2023-09-10T01:33:18.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:17 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:33:20.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:19 smithi131 ceph-mon[103713]: pgmap v1715: 129 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 120 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 311 op/s 2023-09-10T01:33:20.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:19 smithi044 ceph-mon[98676]: pgmap v1715: 129 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 120 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 311 op/s 2023-09-10T01:33:20.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:19 smithi099 ceph-mon[102692]: pgmap v1715: 129 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 120 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 311 op/s 2023-09-10T01:33:21.183 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 735.0 s, 212.5 tps, lat 645.964 ms stddev 1147.598 2023-09-10T01:33:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.250+0000 7f478ccbb700 1 -- 172.21.15.44:0/693551816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4788101740 msgr2=0x7f4788101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.250+0000 7f478ccbb700 1 --2- 172.21.15.44:0/693551816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4788101740 0x7f4788101b20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f477c009990 tx=0x7f477c009ca0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.250+0000 7f478ccbb700 1 -- 172.21.15.44:0/693551816 shutdown_connections 2023-09-10T01:33:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.250+0000 7f478ccbb700 1 --2- 172.21.15.44:0/693551816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f478810ab60 0x7f478810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.250+0000 7f478ccbb700 1 --2- 172.21.15.44:0/693551816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4788101740 0x7f4788101b20 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.250+0000 7f478ccbb700 1 --2- 172.21.15.44:0/693551816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4788102060 0x7f478810a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.250+0000 7f478ccbb700 1 -- 172.21.15.44:0/693551816 >> 172.21.15.44:0/693551816 conn(0x7f47880faf40 msgr2=0x7f47880fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.257+0000 7f478ccbb700 1 -- 172.21.15.44:0/693551816 shutdown_connections 2023-09-10T01:33:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.263+0000 7f478ccbb700 1 -- 172.21.15.44:0/693551816 wait complete. 2023-09-10T01:33:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.263+0000 7f478ccbb700 1 Processor -- start 2023-09-10T01:33:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.264+0000 7f478ccbb700 1 -- start start 2023-09-10T01:33:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.264+0000 7f478ccbb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4788101740 0x7f4788109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.264+0000 7f478ccbb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4788102060 0x7f478810a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.264+0000 7f478ccbb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478810ab60 0x7f4788103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.264+0000 7f478ccbb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47881042b0 con 0x7f4788102060 2023-09-10T01:33:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.264+0000 7f478ccbb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4788104420 con 0x7f4788101740 2023-09-10T01:33:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.264+0000 7f478ccbb700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47881045e0 con 0x7f478810ab60 2023-09-10T01:33:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.264+0000 7f478659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4788101740 0x7f4788109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.264+0000 7f478659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4788101740 0x7f4788109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34336/0 (socket says 172.21.15.44:34336) 2023-09-10T01:33:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.264+0000 7f478659c700 1 -- 172.21.15.44:0/1785798404 learned_addr learned my addr 172.21.15.44:0/1785798404 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.264+0000 7f4786d9d700 1 --2- 172.21.15.44:0/1785798404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478810ab60 0x7f4788103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.265+0000 7f478659c700 1 -- 172.21.15.44:0/1785798404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478810ab60 msgr2=0x7f4788103b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.265+0000 7f478659c700 1 --2- 172.21.15.44:0/1785798404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478810ab60 0x7f4788103b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.265+0000 7f478659c700 1 -- 172.21.15.44:0/1785798404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4788102060 msgr2=0x7f478810a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.265+0000 7f478659c700 1 --2- 172.21.15.44:0/1785798404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4788102060 0x7f478810a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.265+0000 7f478659c700 1 -- 172.21.15.44:0/1785798404 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c0095f0 con 0x7f4788101740 2023-09-10T01:33:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.265+0000 7f478659c700 1 --2- 172.21.15.44:0/1785798404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4788101740 0x7f4788109d60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f477c005950 tx=0x7f477c009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.276+0000 7f47777fe700 1 -- 172.21.15.44:0/1785798404 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c01d070 con 0x7f4788101740 2023-09-10T01:33:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.276+0000 7f47777fe700 1 -- 172.21.15.44:0/1785798404 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f477c0040b0 con 0x7f4788101740 2023-09-10T01:33:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.276+0000 7f47777fe700 1 -- 172.21.15.44:0/1785798404 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c005290 con 0x7f4788101740 2023-09-10T01:33:22.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.276+0000 7f478ccbb700 1 -- 172.21.15.44:0/1785798404 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4788104860 con 0x7f4788101740 2023-09-10T01:33:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.276+0000 7f478ccbb700 1 -- 172.21.15.44:0/1785798404 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47880ff4a0 con 0x7f4788101740 2023-09-10T01:33:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.281+0000 7f4798e81b80 1 Processor -- start 2023-09-10T01:33:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.289+0000 7f47777fe700 1 -- 172.21.15.44:0/1785798404 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f477c0048e0 con 0x7f4788101740 2023-09-10T01:33:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.290+0000 7f4798e81b80 1 -- start start 2023-09-10T01:33:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.291+0000 7f4798e81b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cb31a0 0x562ed6cb3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.291+0000 7f4798e81b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cad300 0x562ed6cad780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.291+0000 7f4798e81b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cadcc0 0x562ed6cb9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.291+0000 7f4798e81b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed6c066d0 con 0x562ed6cadcc0 2023-09-10T01:33:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.291+0000 7f4798e81b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed6c03ff0 con 0x562ed6cb31a0 2023-09-10T01:33:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.291+0000 7f4798e81b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed6bff8e0 con 0x562ed6cad300 2023-09-10T01:33:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.291+0000 7f4785d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cadcc0 0x562ed6cb9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.291+0000 7f4785d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cadcc0 0x562ed6cb9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60834/0 (socket says 172.21.15.44:60834) 2023-09-10T01:33:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.291+0000 7f4785d9b700 1 -- 172.21.15.44:0/948432682 learned_addr learned my addr 172.21.15.44:0/948432682 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.291+0000 7f47777fe700 1 --2- 172.21.15.44:0/1785798404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f47700a09c0 0x7f47700a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.291+0000 7f47777fe700 1 -- 172.21.15.44:0/1785798404 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f477c0cd0e0 con 0x7f4788101740 2023-09-10T01:33:22.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:21 smithi131 ceph-mon[103713]: pgmap v1716: 129 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 120 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 289 op/s 2023-09-10T01:33:22.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:21 smithi131 ceph-mon[103713]: osdmap e203: 12 total, 12 up, 12 in 2023-09-10T01:33:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.294+0000 7f478659c700 1 --2- 172.21.15.44:0/948432682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cad300 0x562ed6cad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.294+0000 7f478659c700 1 -- 172.21.15.44:0/948432682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cb31a0 msgr2=0x562ed6cb3580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:33:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.294+0000 7f478659c700 1 --2- 172.21.15.44:0/948432682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cb31a0 0x562ed6cb3580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.294+0000 7f478659c700 1 -- 172.21.15.44:0/948432682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cadcc0 msgr2=0x562ed6cb9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.294+0000 7f478659c700 1 --2- 172.21.15.44:0/948432682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cadcc0 0x562ed6cb9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.294+0000 7f478659c700 1 -- 172.21.15.44:0/948432682 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ed6a2bd80 con 0x562ed6cad300 2023-09-10T01:33:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.294+0000 7f478659c700 1 --2- 172.21.15.44:0/948432682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cad300 0x562ed6cad780 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f477c0ca3d0 tx=0x7f477c099f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6558dfd1c66de10 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.295+0000 7f4774ff9700 1 -- 172.21.15.44:0/948432682 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c0b1070 con 0x562ed6cad300 2023-09-10T01:33:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.295+0000 7f4774ff9700 1 -- 172.21.15.44:0/948432682 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f477c0a00f0 con 0x562ed6cad300 2023-09-10T01:33:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.295+0000 7f4774ff9700 1 -- 172.21.15.44:0/948432682 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c098560 con 0x562ed6cad300 2023-09-10T01:33:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.296+0000 7f4798e81b80 1 -- 172.21.15.44:0/948432682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cad300 msgr2=0x562ed6cad780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.296+0000 7f4798e81b80 1 --2- 172.21.15.44:0/948432682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cad300 0x562ed6cad780 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f477c0ca3d0 tx=0x7f477c099f90 comp rx=0 tx=0).stop 2023-09-10T01:33:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.297+0000 7f4798e81b80 1 -- 172.21.15.44:0/948432682 shutdown_connections 2023-09-10T01:33:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.297+0000 7f4798e81b80 1 --2- 172.21.15.44:0/948432682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cadcc0 0x562ed6cb9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.297+0000 7f4798e81b80 1 --2- 172.21.15.44:0/948432682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cb31a0 0x562ed6cb3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.297+0000 7f4798e81b80 1 --2- 172.21.15.44:0/948432682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cad300 0x562ed6cad780 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.297+0000 7f4798e81b80 1 -- 172.21.15.44:0/948432682 >> 172.21.15.44:0/948432682 conn(0x562ed6c22030 msgr2=0x562ed6ca8560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.297+0000 7f4798e81b80 1 -- 172.21.15.44:0/948432682 shutdown_connections 2023-09-10T01:33:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.298+0000 7f4798e81b80 1 -- 172.21.15.44:0/948432682 wait complete. 2023-09-10T01:33:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.298+0000 7f4798e81b80 1 -- start start 2023-09-10T01:33:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.298+0000 7f4798e81b80 1 Processor -- start 2023-09-10T01:33:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.298+0000 7f4798e81b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cad300 0x562ed6cbcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.298+0000 7f4798e81b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cadcc0 0x562ed6cbd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.299+0000 7f4798e81b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cb31a0 0x562ed6cb6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.299+0000 7f4798e81b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed6b9db10 con 0x562ed6cad300 2023-09-10T01:33:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.299+0000 7f4798e81b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed6b9d810 con 0x562ed6cb31a0 2023-09-10T01:33:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.299+0000 7f4798e81b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed6a818c0 con 0x562ed6cadcc0 2023-09-10T01:33:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.299+0000 7f4798e81b80 10 client.0 _ll_register_callbacks cb 0x562ed6ae7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:33:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.299+0000 7f4786d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cb31a0 0x562ed6cb6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.299+0000 7f4786d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cb31a0 0x562ed6cb6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34348/0 (socket says 172.21.15.44:34348) 2023-09-10T01:33:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.299+0000 7f4786d9d700 1 -- 172.21.15.44:0/2090945213 learned_addr learned my addr 172.21.15.44:0/2090945213 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.299+0000 7f4786d9d700 1 -- 172.21.15.44:0/2090945213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cadcc0 msgr2=0x562ed6cbd120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:33:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.299+0000 7f4786d9d700 1 --2- 172.21.15.44:0/2090945213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cadcc0 0x562ed6cbd120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.299+0000 7f4786d9d700 1 -- 172.21.15.44:0/2090945213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cad300 msgr2=0x562ed6cbcbe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:33:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.299+0000 7f4786d9d700 1 --2- 172.21.15.44:0/2090945213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cad300 0x562ed6cbcbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.299+0000 7f4786d9d700 1 -- 172.21.15.44:0/2090945213 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47880ff4a0 con 0x562ed6cb31a0 2023-09-10T01:33:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.299+0000 7f4786d9d700 1 --2- 172.21.15.44:0/2090945213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cb31a0 0x562ed6cb6ea0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f478000a9c0 tx=0x7f478000ad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2441758e7a148f2e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.301+0000 7f476dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:33:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.309+0000 7f4786d9d700 1 --2- 172.21.15.44:0/1785798404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f47700a09c0 0x7f47700a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.309+0000 7f476dffb700 1 -- 172.21.15.44:0/2090945213 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478000f860 con 0x562ed6cb31a0 2023-09-10T01:33:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.309+0000 7f476dffb700 1 -- 172.21.15.44:0/2090945213 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(203..203 src has 1..203) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f4780006e40 con 0x562ed6cb31a0 2023-09-10T01:33:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.309+0000 7f4798e81b80 1 -- 172.21.15.44:0/2090945213 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ed6b57490 con 0x562ed6cb31a0 2023-09-10T01:33:22.314 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:21 smithi044 ceph-mon[98676]: pgmap v1716: 129 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 120 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 289 op/s 2023-09-10T01:33:22.315 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:21 smithi044 ceph-mon[98676]: osdmap e203: 12 total, 12 up, 12 in 2023-09-10T01:33:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.312+0000 7f476dffb700 1 -- 172.21.15.44:0/2090945213 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f478000fce0 con 0x562ed6cb31a0 2023-09-10T01:33:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.317+0000 7f4798e81b80 10 client.30230 fetch_fsmap learned FSMap version 189 2023-09-10T01:33:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.317+0000 7f4798e81b80 1 -- 172.21.15.44:0/2090945213 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x562ed6a14c00 con 0x562ed6cb31a0 2023-09-10T01:33:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.319+0000 7f476dffb700 1 -- 172.21.15.44:0/2090945213 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562ed6b57490 con 0x562ed6cb31a0 2023-09-10T01:33:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.319+0000 7f476dffb700 1 -- 172.21.15.44:0/2090945213 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 189) v1 ==== 7363+0+0 (secure 0 0 0) 0x7f4780027d60 con 0x562ed6cb31a0 2023-09-10T01:33:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.319+0000 7f4786d9d700 1 --2- 172.21.15.44:0/1785798404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f47700a09c0 0x7f47700a2e80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4780021000 tx=0x7f4780022000 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.319+0000 7f4798e81b80 10 client.30230 fetch_fsmap finished waiting for FSMap version 189 2023-09-10T01:33:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.319+0000 7f4798e81b80 10 client.30230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:33:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.319+0000 7f4798e81b80 20 client.30230 populate_metadata read hostname 'smithi044' 2023-09-10T01:33:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.319+0000 7f4798e81b80 1 --2- 172.21.15.44:0/2090945213 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ed6cc7280 0x562ed6cc96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.319+0000 7f4798e81b80 4 client.30230 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:33:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.319+0000 7f4798e81b80 1 -- 172.21.15.44:0/2090945213 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ed6c14840 con 0x562ed6cc7280 2023-09-10T01:33:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.319+0000 7f478659c700 1 --2- 172.21.15.44:0/2090945213 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ed6cc7280 0x562ed6cc96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.320+0000 7f476dffb700 10 client.30230 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:33:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.320+0000 7f478659c700 1 --2- 172.21.15.44:0/2090945213 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ed6cc7280 0x562ed6cc96a0 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33397c57393f6fa3 server_cookie=ae9b42f4ec277a1d in_seq=0 out_seq=0 2023-09-10T01:33:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.322+0000 7f476dffb700 1 -- 172.21.15.44:0/2090945213 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562ed6c14840 con 0x562ed6cc7280 2023-09-10T01:33:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.322+0000 7f476dffb700 10 client.30230 handle_command_reply: tid=0 2023-09-10T01:33:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.323+0000 7f4798e81b80 1 client.30230 shutdown 2023-09-10T01:33:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.323+0000 7f4798e81b80 1 -- 172.21.15.44:0/2090945213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cb31a0 msgr2=0x562ed6cb6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.323+0000 7f4798e81b80 1 --2- 172.21.15.44:0/2090945213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cb31a0 0x562ed6cb6ea0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f478000a9c0 tx=0x7f478000ad80 comp rx=0 tx=0).stop 2023-09-10T01:33:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.323+0000 7f4798e81b80 1 -- 172.21.15.44:0/2090945213 shutdown_connections 2023-09-10T01:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.323+0000 7f4798e81b80 1 --2- 172.21.15.44:0/2090945213 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ed6cc7280 0x562ed6cc96a0 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.323+0000 7f4798e81b80 1 --2- 172.21.15.44:0/2090945213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cb31a0 0x562ed6cb6ea0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.323+0000 7f4798e81b80 1 --2- 172.21.15.44:0/2090945213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cad300 0x562ed6cbcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.323+0000 7f4798e81b80 1 --2- 172.21.15.44:0/2090945213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cadcc0 0x562ed6cbd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.323+0000 7f4798e81b80 1 -- 172.21.15.44:0/2090945213 >> 172.21.15.44:0/2090945213 conn(0x562ed6c22030 msgr2=0x562ed6ca77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.323+0000 7f476dffb700 0 client.30230 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:33:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.323+0000 7f4798e81b80 1 -- 172.21.15.44:0/2090945213 shutdown_connections 2023-09-10T01:33:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.323+0000 7f4798e81b80 1 -- 172.21.15.44:0/2090945213 wait complete. 2023-09-10T01:33:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.324+0000 7f4798e81b80 20 client.30230 trim_cache size 0 max 16384 2023-09-10T01:33:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.350+0000 7f4798e81b80 1 Processor -- start 2023-09-10T01:33:22.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:21 smithi099 ceph-mon[102692]: pgmap v1716: 129 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 120 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 289 op/s 2023-09-10T01:33:22.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:21 smithi099 ceph-mon[102692]: osdmap e203: 12 total, 12 up, 12 in 2023-09-10T01:33:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.357+0000 7f4798e81b80 1 -- start start 2023-09-10T01:33:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.357+0000 7f4798e81b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cad300 0x562ed6cb26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.357+0000 7f4798e81b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cadcc0 0x562ed6cb6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.357+0000 7f4798e81b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cb31a0 0x562ed6cb6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.357+0000 7f4798e81b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed6cae580 con 0x562ed6cad300 2023-09-10T01:33:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.357+0000 7f4798e81b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed6cb81f0 con 0x562ed6cadcc0 2023-09-10T01:33:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.357+0000 7f4798e81b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed6cb2db0 con 0x562ed6cb31a0 2023-09-10T01:33:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.357+0000 7f4786d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cb31a0 0x562ed6cb6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.357+0000 7f4786d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cb31a0 0x562ed6cb6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39494/0 (socket says 172.21.15.44:39494) 2023-09-10T01:33:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.357+0000 7f4786d9d700 1 -- 172.21.15.44:0/821492751 learned_addr learned my addr 172.21.15.44:0/821492751 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.358+0000 7f4786d9d700 1 -- 172.21.15.44:0/821492751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cadcc0 msgr2=0x562ed6cb6740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:33:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.358+0000 7f4786d9d700 1 --2- 172.21.15.44:0/821492751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cadcc0 0x562ed6cb6740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.358+0000 7f4786d9d700 1 -- 172.21.15.44:0/821492751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cad300 msgr2=0x562ed6cb26b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:33:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.358+0000 7f4786d9d700 1 --2- 172.21.15.44:0/821492751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cad300 0x562ed6cb26b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.358+0000 7f4786d9d700 1 -- 172.21.15.44:0/821492751 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ed6a521f0 con 0x562ed6cb31a0 2023-09-10T01:33:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.358+0000 7f4786d9d700 1 --2- 172.21.15.44:0/821492751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cb31a0 0x562ed6cb6c80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4780004a50 tx=0x7f4780004a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3d7ea84b96f433c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.358+0000 7f47757fa700 1 -- 172.21.15.44:0/821492751 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780006e40 con 0x562ed6cb31a0 2023-09-10T01:33:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.364+0000 7f47757fa700 1 -- 172.21.15.44:0/821492751 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4780009390 con 0x562ed6cb31a0 2023-09-10T01:33:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.364+0000 7f47757fa700 1 -- 172.21.15.44:0/821492751 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780023c60 con 0x562ed6cb31a0 2023-09-10T01:33:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.365+0000 7f4798e81b80 1 -- 172.21.15.44:0/821492751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cb31a0 msgr2=0x562ed6cb6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.365+0000 7f4798e81b80 1 --2- 172.21.15.44:0/821492751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cb31a0 0x562ed6cb6c80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4780004a50 tx=0x7f4780004a80 comp rx=0 tx=0).stop 2023-09-10T01:33:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.366+0000 7f4798e81b80 1 -- 172.21.15.44:0/821492751 shutdown_connections 2023-09-10T01:33:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.366+0000 7f4798e81b80 1 --2- 172.21.15.44:0/821492751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cb31a0 0x562ed6cb6c80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.366+0000 7f4798e81b80 1 --2- 172.21.15.44:0/821492751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cad300 0x562ed6cb26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.366+0000 7f4798e81b80 1 --2- 172.21.15.44:0/821492751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cadcc0 0x562ed6cb6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.366+0000 7f4798e81b80 1 -- 172.21.15.44:0/821492751 >> 172.21.15.44:0/821492751 conn(0x562ed6c22030 msgr2=0x562ed6cb1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.366+0000 7f4798e81b80 1 -- 172.21.15.44:0/821492751 shutdown_connections 2023-09-10T01:33:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.367+0000 7f4798e81b80 1 -- 172.21.15.44:0/821492751 wait complete. 2023-09-10T01:33:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.367+0000 7f4798e81b80 1 -- start start 2023-09-10T01:33:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.367+0000 7f4798e81b80 1 Processor -- start 2023-09-10T01:33:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.367+0000 7f4798e81b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cad300 0x562ed6d20010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.367+0000 7f4798e81b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cadcc0 0x562ed6d20550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.367+0000 7f4798e81b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cb31a0 0x562ed6d20a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.367+0000 7f4798e81b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed6d2b210 con 0x562ed6cb31a0 2023-09-10T01:33:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.367+0000 7f4798e81b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed6d2b380 con 0x562ed6cad300 2023-09-10T01:33:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.367+0000 7f4798e81b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed6d2b540 con 0x562ed6cadcc0 2023-09-10T01:33:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.368+0000 7f4798e81b80 10 client.0 _ll_register_callbacks cb 0x7f4780009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:33:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.368+0000 7f478659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cad300 0x562ed6d20010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.368+0000 7f478659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cad300 0x562ed6d20010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34354/0 (socket says 172.21.15.44:34354) 2023-09-10T01:33:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.368+0000 7f478659c700 1 -- 172.21.15.44:0/476236055 learned_addr learned my addr 172.21.15.44:0/476236055 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.368+0000 7f478659c700 1 -- 172.21.15.44:0/476236055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cadcc0 msgr2=0x562ed6d20550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:33:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.368+0000 7f478659c700 1 --2- 172.21.15.44:0/476236055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cadcc0 0x562ed6d20550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.368+0000 7f478659c700 1 -- 172.21.15.44:0/476236055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cb31a0 msgr2=0x562ed6d20a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.368+0000 7f478659c700 1 --2- 172.21.15.44:0/476236055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cb31a0 0x562ed6d20a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.368+0000 7f478659c700 1 -- 172.21.15.44:0/476236055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ed6a14c00 con 0x562ed6cad300 2023-09-10T01:33:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.368+0000 7f478659c700 1 --2- 172.21.15.44:0/476236055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cad300 0x562ed6d20010 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f477c096670 tx=0x7f477c00eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c16d3964cdf8b78 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.368+0000 7f476dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:33:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.368+0000 7f476dffb700 1 -- 172.21.15.44:0/476236055 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c0b1040 con 0x562ed6cad300 2023-09-10T01:33:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.369+0000 7f476dffb700 1 -- 172.21.15.44:0/476236055 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(203..203 src has 1..203) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f477c0a21a0 con 0x562ed6cad300 2023-09-10T01:33:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.369+0000 7f4798e81b80 1 -- 172.21.15.44:0/476236055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f477c0a00f0 con 0x562ed6cad300 2023-09-10T01:33:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.369+0000 7f476dffb700 1 -- 172.21.15.44:0/476236055 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f477c09a1c0 con 0x562ed6cad300 2023-09-10T01:33:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.369+0000 7f4798e81b80 10 client.30233 fetch_fsmap learned FSMap version 189 2023-09-10T01:33:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.369+0000 7f4798e81b80 1 -- 172.21.15.44:0/476236055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x562ed6a2a250 con 0x562ed6cad300 2023-09-10T01:33:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.369+0000 7f476dffb700 1 -- 172.21.15.44:0/476236055 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f477c0a00f0 con 0x562ed6cad300 2023-09-10T01:33:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.369+0000 7f476dffb700 1 -- 172.21.15.44:0/476236055 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 189) v1 ==== 7363+0+0 (secure 0 0 0) 0x7f477c0997c0 con 0x562ed6cad300 2023-09-10T01:33:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.369+0000 7f4798e81b80 10 client.30233 fetch_fsmap finished waiting for FSMap version 189 2023-09-10T01:33:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.369+0000 7f4798e81b80 10 client.30233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:33:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.369+0000 7f4798e81b80 20 client.30233 populate_metadata read hostname 'smithi044' 2023-09-10T01:33:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.370+0000 7f4798e81b80 1 --2- 172.21.15.44:0/476236055 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ed6d0f410 0x562ed6d11890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.370+0000 7f4798e81b80 4 client.30233 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:33:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.370+0000 7f4798e81b80 1 -- 172.21.15.44:0/476236055 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562ed6bf88c0 con 0x562ed6d0f410 2023-09-10T01:33:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.370+0000 7f4785d9b700 1 --2- 172.21.15.44:0/476236055 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ed6d0f410 0x562ed6d11890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.371+0000 7f4785d9b700 1 --2- 172.21.15.44:0/476236055 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ed6d0f410 0x562ed6d11890 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff13a46848d2d77d server_cookie=d68911082759b282 in_seq=0 out_seq=0 2023-09-10T01:33:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.371+0000 7f476dffb700 10 client.30233 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:33:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.387+0000 7f476dffb700 1 -- 172.21.15.44:0/476236055 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562ed6bf88c0 con 0x562ed6d0f410 2023-09-10T01:33:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.387+0000 7f476dffb700 10 client.30233 handle_command_reply: tid=0 2023-09-10T01:33:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.387+0000 7f4798e81b80 1 client.30233 shutdown 2023-09-10T01:33:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.387+0000 7f4798e81b80 1 -- 172.21.15.44:0/476236055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cad300 msgr2=0x562ed6d20010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.387+0000 7f4798e81b80 1 --2- 172.21.15.44:0/476236055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cad300 0x562ed6d20010 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f477c096670 tx=0x7f477c00eec0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.389+0000 7f4798e81b80 1 -- 172.21.15.44:0/476236055 shutdown_connections 2023-09-10T01:33:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.389+0000 7f4798e81b80 1 --2- 172.21.15.44:0/476236055 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562ed6d0f410 0x562ed6d11890 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.389+0000 7f4798e81b80 1 --2- 172.21.15.44:0/476236055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ed6cb31a0 0x562ed6d20a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.389+0000 7f4798e81b80 1 --2- 172.21.15.44:0/476236055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed6cad300 0x562ed6d20010 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.389+0000 7f4798e81b80 1 --2- 172.21.15.44:0/476236055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ed6cadcc0 0x562ed6d20550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.389+0000 7f4798e81b80 1 -- 172.21.15.44:0/476236055 >> 172.21.15.44:0/476236055 conn(0x562ed6c22030 msgr2=0x562ed6cb6740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.389+0000 7f476dffb700 0 client.30233 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:33:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.393+0000 7f4798e81b80 1 -- 172.21.15.44:0/476236055 shutdown_connections 2023-09-10T01:33:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.393+0000 7f4798e81b80 1 -- 172.21.15.44:0/476236055 wait complete. 2023-09-10T01:33:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.393+0000 7f4798e81b80 20 client.30233 trim_cache size 0 max 16384 2023-09-10T01:33:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:33:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:33:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:33:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:33:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.394+0000 7f478ccbb700 1 -- 172.21.15.44:0/1785798404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f47700a09c0 msgr2=0x7f47700a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.394+0000 7f478ccbb700 1 --2- 172.21.15.44:0/1785798404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f47700a09c0 0x7f47700a2e80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4780021000 tx=0x7f4780022000 comp rx=0 tx=0).stop 2023-09-10T01:33:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.394+0000 7f478ccbb700 1 -- 172.21.15.44:0/1785798404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4788101740 msgr2=0x7f4788109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.394+0000 7f478ccbb700 1 --2- 172.21.15.44:0/1785798404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4788101740 0x7f4788109d60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f477c005950 tx=0x7f477c009ee0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.394+0000 7f478ccbb700 1 -- 172.21.15.44:0/1785798404 shutdown_connections 2023-09-10T01:33:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.394+0000 7f478ccbb700 1 --2- 172.21.15.44:0/1785798404 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f47700a09c0 0x7f47700a2e80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.394+0000 7f478ccbb700 1 --2- 172.21.15.44:0/1785798404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478810ab60 0x7f4788103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.394+0000 7f478ccbb700 1 --2- 172.21.15.44:0/1785798404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4788101740 0x7f4788109d60 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.394+0000 7f478ccbb700 1 --2- 172.21.15.44:0/1785798404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4788102060 0x7f478810a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.394+0000 7f478ccbb700 1 -- 172.21.15.44:0/1785798404 >> 172.21.15.44:0/1785798404 conn(0x7f47880faf40 msgr2=0x7f478810cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.394+0000 7f478ccbb700 1 -- 172.21.15.44:0/1785798404 shutdown_connections 2023-09-10T01:33:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:22.394+0000 7f478ccbb700 1 -- 172.21.15.44:0/1785798404 wait complete. 2023-09-10T01:33:23.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:22 smithi131 ceph-mon[103713]: pgmap v1718: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 424 op/s 2023-09-10T01:33:23.310 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:22 smithi044 ceph-mon[98676]: pgmap v1718: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 424 op/s 2023-09-10T01:33:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:22 smithi099 ceph-mon[102692]: pgmap v1718: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 424 op/s 2023-09-10T01:33:26.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 740.0 s, 241.7 tps, lat 395.804 ms stddev 197.303 2023-09-10T01:33:26.300 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:25 smithi131 ceph-mon[103713]: pgmap v1719: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 445 op/s 2023-09-10T01:33:26.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:25 smithi044 ceph-mon[98676]: pgmap v1719: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 445 op/s 2023-09-10T01:33:26.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:25 smithi099 ceph-mon[102692]: pgmap v1719: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 445 op/s 2023-09-10T01:33:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:27 smithi131 ceph-mon[103713]: osdmap e204: 12 total, 12 up, 12 in 2023-09-10T01:33:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:27 smithi131 ceph-mon[103713]: pgmap v1721: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 796 op/s 2023-09-10T01:33:28.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:27 smithi044 ceph-mon[98676]: osdmap e204: 12 total, 12 up, 12 in 2023-09-10T01:33:28.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:27 smithi044 ceph-mon[98676]: pgmap v1721: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 796 op/s 2023-09-10T01:33:28.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:27 smithi099 ceph-mon[102692]: osdmap e204: 12 total, 12 up, 12 in 2023-09-10T01:33:28.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:27 smithi099 ceph-mon[102692]: pgmap v1721: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 796 op/s 2023-09-10T01:33:30.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:29 smithi131 ceph-mon[103713]: pgmap v1722: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.8 MiB/s wr, 891 op/s 2023-09-10T01:33:30.312 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:29 smithi044 ceph-mon[98676]: pgmap v1722: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.8 MiB/s wr, 891 op/s 2023-09-10T01:33:30.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:29 smithi099 ceph-mon[102692]: pgmap v1722: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.8 MiB/s wr, 891 op/s 2023-09-10T01:33:31.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 745.0 s, 247.8 tps, lat 415.185 ms stddev 208.229 2023-09-10T01:33:32.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:31 smithi131 ceph-mon[103713]: pgmap v1723: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.6 MiB/s wr, 578 op/s 2023-09-10T01:33:32.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:31 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:33:32.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:31 smithi044 ceph-mon[98676]: pgmap v1723: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.6 MiB/s wr, 578 op/s 2023-09-10T01:33:32.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:31 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:33:32.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:31 smithi099 ceph-mon[102692]: pgmap v1723: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.6 MiB/s wr, 578 op/s 2023-09-10T01:33:32.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:31 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:33:34.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:33 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:33:34.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:33 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:33:34.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:33 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:33:34.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:33 smithi131 ceph-mon[103713]: pgmap v1724: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.7 MiB/s wr, 749 op/s 2023-09-10T01:33:34.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:33 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:33:34.061 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:33 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:33:34.061 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:33 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:33:34.062 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:33 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:33:34.062 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:33 smithi044 ceph-mon[98676]: pgmap v1724: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.7 MiB/s wr, 749 op/s 2023-09-10T01:33:34.062 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:33 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:33:34.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:33 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:33:34.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:33 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:33:34.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:33 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:33:34.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:33 smithi099 ceph-mon[102692]: pgmap v1724: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.7 MiB/s wr, 749 op/s 2023-09-10T01:33:34.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:33 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:33:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.590+0000 7fd39ce09700 1 -- 172.21.15.44:0/3797933622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd398100120 msgr2=0x7fd39810c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.590+0000 7fd39ce09700 1 --2- 172.21.15.44:0/3797933622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd398100120 0x7fd39810c330 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd39000b0d0 tx=0x7fd39000b3e0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.594+0000 7fd39ce09700 1 -- 172.21.15.44:0/3797933622 shutdown_connections 2023-09-10T01:33:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.594+0000 7fd39ce09700 1 --2- 172.21.15.44:0/3797933622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd398100120 0x7fd39810c330 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.594+0000 7fd39ce09700 1 --2- 172.21.15.44:0/3797933622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3981055b0 0x7fd398105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.594+0000 7fd39ce09700 1 --2- 172.21.15.44:0/3797933622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3980ff760 0x7fd3980ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.595+0000 7fd39ce09700 1 -- 172.21.15.44:0/3797933622 >> 172.21.15.44:0/3797933622 conn(0x7fd398074f30 msgr2=0x7fd398075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.595+0000 7fd39ce09700 1 -- 172.21.15.44:0/3797933622 shutdown_connections 2023-09-10T01:33:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.595+0000 7fd39ce09700 1 -- 172.21.15.44:0/3797933622 wait complete. 2023-09-10T01:33:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.596+0000 7fd39ce09700 1 Processor -- start 2023-09-10T01:33:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.596+0000 7fd39ce09700 1 -- start start 2023-09-10T01:33:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.596+0000 7fd39ce09700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3980ff760 0x7fd39810c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.596+0000 7fd39ce09700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd398100120 0x7fd398106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.596+0000 7fd39ce09700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3981055b0 0x7fd3981075b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.596+0000 7fd39ce09700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd398107d80 con 0x7fd3980ff760 2023-09-10T01:33:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.596+0000 7fd39ce09700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3981a0af0 con 0x7fd398100120 2023-09-10T01:33:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.596+0000 7fd39ce09700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3981a0cb0 con 0x7fd3981055b0 2023-09-10T01:33:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.596+0000 7fd39659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3980ff760 0x7fd39810c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.596+0000 7fd396d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3981055b0 0x7fd3981075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.596+0000 7fd39659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3980ff760 0x7fd39810c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60858/0 (socket says 172.21.15.44:60858) 2023-09-10T01:33:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.596+0000 7fd39659c700 1 -- 172.21.15.44:0/3849535599 learned_addr learned my addr 172.21.15.44:0/3849535599 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.596+0000 7fd396d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3981055b0 0x7fd3981075b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39508/0 (socket says 172.21.15.44:39508) 2023-09-10T01:33:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.596+0000 7fd38dd9b700 1 --2- 172.21.15.44:0/3849535599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd398100120 0x7fd398106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.597+0000 7fd396d9d700 1 -- 172.21.15.44:0/3849535599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd398100120 msgr2=0x7fd398106fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.597+0000 7fd396d9d700 1 --2- 172.21.15.44:0/3849535599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd398100120 0x7fd398106fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.597+0000 7fd396d9d700 1 -- 172.21.15.44:0/3849535599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3980ff760 msgr2=0x7fd39810c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.597+0000 7fd396d9d700 1 --2- 172.21.15.44:0/3849535599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3980ff760 0x7fd39810c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.597+0000 7fd396d9d700 1 -- 172.21.15.44:0/3849535599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd390009d00 con 0x7fd3981055b0 2023-09-10T01:33:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.597+0000 7fd396d9d700 1 --2- 172.21.15.44:0/3849535599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3981055b0 0x7fd3981075b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd390009fd0 tx=0x7fd39000cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.597+0000 7fd38ffff700 1 -- 172.21.15.44:0/3849535599 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39000e040 con 0x7fd3981055b0 2023-09-10T01:33:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.597+0000 7fd39ce09700 1 -- 172.21.15.44:0/3849535599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3981a0f30 con 0x7fd3981055b0 2023-09-10T01:33:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.597+0000 7fd38ffff700 1 -- 172.21.15.44:0/3849535599 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd3900094a0 con 0x7fd3981055b0 2023-09-10T01:33:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.597+0000 7fd38ffff700 1 -- 172.21.15.44:0/3849535599 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39001b6b0 con 0x7fd3981055b0 2023-09-10T01:33:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.597+0000 7fd39ce09700 1 -- 172.21.15.44:0/3849535599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3981a13f0 con 0x7fd3981055b0 2023-09-10T01:33:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.600+0000 7fd38ffff700 1 -- 172.21.15.44:0/3849535599 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fd39001d480 con 0x7fd3981055b0 2023-09-10T01:33:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.601+0000 7fd38ffff700 1 --2- 172.21.15.44:0/3849535599 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd3840a07b0 0x7fd3840a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.601+0000 7fd38ffff700 1 -- 172.21.15.44:0/3849535599 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 9496+0+0 (secure 0 0 0) 0x7fd3900cdbf0 con 0x7fd3981055b0 2023-09-10T01:33:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.602+0000 7fd39659c700 1 --2- 172.21.15.44:0/3849535599 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd3840a07b0 0x7fd3840a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.603+0000 7fd3a8fcfb80 1 Processor -- start 2023-09-10T01:33:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.603+0000 7fd3a8fcfb80 1 -- start start 2023-09-10T01:33:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.603+0000 7fd3a8fcfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008c51a0 0x55af008c5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.603+0000 7fd3a8fcfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008bf300 0x55af008bf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.603+0000 7fd3a8fcfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008bfcc0 0x55af008cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.603+0000 7fd3a8fcfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af008186d0 con 0x55af008bf300 2023-09-10T01:33:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.603+0000 7fd3a8fcfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af00815ff0 con 0x55af008bfcc0 2023-09-10T01:33:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.603+0000 7fd3a8fcfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af008118e0 con 0x55af008c51a0 2023-09-10T01:33:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.603+0000 7fd38dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008bf300 0x55af008bf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.603+0000 7fd38dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008bf300 0x55af008bf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60866/0 (socket says 172.21.15.44:60866) 2023-09-10T01:33:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.603+0000 7fd39659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008c51a0 0x55af008c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.603+0000 7fd38dd9b700 1 -- 172.21.15.44:0/261174122 learned_addr learned my addr 172.21.15.44:0/261174122 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.603+0000 7fd396d9d700 1 --2- 172.21.15.44:0/261174122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008bfcc0 0x55af008cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.604+0000 7fd38dd9b700 1 -- 172.21.15.44:0/261174122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008c51a0 msgr2=0x55af008c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.604+0000 7fd38dd9b700 1 --2- 172.21.15.44:0/261174122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008c51a0 0x55af008c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.604+0000 7fd38dd9b700 1 -- 172.21.15.44:0/261174122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008bfcc0 msgr2=0x55af008cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.604+0000 7fd38dd9b700 1 --2- 172.21.15.44:0/261174122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008bfcc0 0x55af008cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.604+0000 7fd38dd9b700 1 -- 172.21.15.44:0/261174122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af0063dd80 con 0x55af008bf300 2023-09-10T01:33:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.604+0000 7fd39659c700 1 --2- 172.21.15.44:0/261174122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008c51a0 0x55af008c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:33:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.604+0000 7fd38dd9b700 1 --2- 172.21.15.44:0/261174122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008bf300 0x55af008bf780 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fd38800bcf0 tx=0x7fd3880103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8415e313e188f12a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.604+0000 7fd38cd99700 1 -- 172.21.15.44:0/261174122 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388010d10 con 0x55af008bf300 2023-09-10T01:33:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.604+0000 7fd38cd99700 1 -- 172.21.15.44:0/261174122 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd388015070 con 0x55af008bf300 2023-09-10T01:33:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.604+0000 7fd38cd99700 1 -- 172.21.15.44:0/261174122 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388005020 con 0x55af008bf300 2023-09-10T01:33:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.605+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/261174122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008bf300 msgr2=0x55af008bf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:35.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.605+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/261174122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008bf300 0x55af008bf780 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fd38800bcf0 tx=0x7fd3880103d0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.605+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/261174122 shutdown_connections 2023-09-10T01:33:35.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.605+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/261174122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008bfcc0 0x55af008cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.605+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/261174122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008c51a0 0x55af008c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.605+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/261174122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008bf300 0x55af008bf780 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.605+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/261174122 >> 172.21.15.44:0/261174122 conn(0x55af00834030 msgr2=0x55af008ba560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.605+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/261174122 shutdown_connections 2023-09-10T01:33:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.605+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/261174122 wait complete. 2023-09-10T01:33:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.606+0000 7fd3a8fcfb80 1 -- start start 2023-09-10T01:33:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.606+0000 7fd3a8fcfb80 1 Processor -- start 2023-09-10T01:33:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.607+0000 7fd3a8fcfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008bf300 0x55af008cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.607+0000 7fd3a8fcfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008bfcc0 0x55af008c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.607+0000 7fd3a8fcfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 0x55af008c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.607+0000 7fd3a8fcfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af007afb10 con 0x55af008c51a0 2023-09-10T01:33:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.607+0000 7fd3a8fcfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af007af810 con 0x55af008bf300 2023-09-10T01:33:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.607+0000 7fd3a8fcfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af006938c0 con 0x55af008bfcc0 2023-09-10T01:33:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.607+0000 7fd3a8fcfb80 10 client.0 _ll_register_callbacks cb 0x55af006f9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:33:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.607+0000 7fd39659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008bf300 0x55af008cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.607+0000 7fd39659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008bf300 0x55af008cbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34378/0 (socket says 172.21.15.44:34378) 2023-09-10T01:33:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.607+0000 7fd39659c700 1 -- 172.21.15.44:0/196718681 learned_addr learned my addr 172.21.15.44:0/196718681 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.608+0000 7fd39659c700 1 -- 172.21.15.44:0/196718681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008bfcc0 msgr2=0x55af008c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.608+0000 7fd38dd9b700 1 --2- 172.21.15.44:0/196718681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008bfcc0 0x55af008c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.608+0000 7fd39659c700 1 --2- 172.21.15.44:0/196718681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008bfcc0 0x55af008c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.608+0000 7fd39659c700 1 -- 172.21.15.44:0/196718681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 msgr2=0x55af008c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.608+0000 7fd39659c700 1 --2- 172.21.15.44:0/196718681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 0x55af008c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.608+0000 7fd39659c700 1 -- 172.21.15.44:0/196718681 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd380017070 con 0x55af008bf300 2023-09-10T01:33:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.608+0000 7fd39659c700 1 --2- 172.21.15.44:0/3849535599 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd3840a07b0 0x7fd3840a2c70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd380009d20 tx=0x7fd380009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.608+0000 7fd38dd9b700 1 --2- 172.21.15.44:0/196718681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008bfcc0 0x55af008c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:33:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.608+0000 7fd39659c700 1 --2- 172.21.15.44:0/196718681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008bf300 0x55af008cbd40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd38001dbf0 tx=0x7fd3800047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9304294ba094fd0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.608+0000 7fd37dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:33:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.608+0000 7fd37dffb700 1 -- 172.21.15.44:0/196718681 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380022070 con 0x55af008bf300 2023-09-10T01:33:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.608+0000 7fd37dffb700 1 -- 172.21.15.44:0/196718681 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(204..204 src has 1..204) v4 ==== 9496+0+0 (secure 0 0 0) 0x7fd38001ca90 con 0x55af008bf300 2023-09-10T01:33:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.608+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/196718681 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af00769490 con 0x55af008bf300 2023-09-10T01:33:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.608+0000 7fd37dffb700 1 -- 172.21.15.44:0/196718681 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=191) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd38001cc50 con 0x55af008bf300 2023-09-10T01:33:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.608+0000 7fd3a8fcfb80 10 client.30245 fetch_fsmap learned FSMap version 191 2023-09-10T01:33:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.608+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/196718681 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=191,monmap=4+}) v3 -- 0x7fd38800b920 con 0x55af008bf300 2023-09-10T01:33:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.609+0000 7fd37dffb700 1 -- 172.21.15.44:0/196718681 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55af00769490 con 0x55af008bf300 2023-09-10T01:33:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.609+0000 7fd37dffb700 1 -- 172.21.15.44:0/196718681 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 191) v1 ==== 7363+0+0 (secure 0 0 0) 0x7fd38000e3b0 con 0x55af008bf300 2023-09-10T01:33:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.609+0000 7fd3a8fcfb80 10 client.30245 fetch_fsmap finished waiting for FSMap version 191 2023-09-10T01:33:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.609+0000 7fd3a8fcfb80 10 client.30245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:33:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.609+0000 7fd3a8fcfb80 20 client.30245 populate_metadata read hostname 'smithi044' 2023-09-10T01:33:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.609+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/196718681 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af008d9520 0x55af008db960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.609+0000 7fd3a8fcfb80 4 client.30245 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:33:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.609+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/196718681 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af00826840 con 0x55af008d9520 2023-09-10T01:33:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.609+0000 7fd38dd9b700 1 --2- 172.21.15.44:0/196718681 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af008d9520 0x55af008db960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.610+0000 7fd37dffb700 10 client.30245 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:33:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.610+0000 7fd38dd9b700 1 --2- 172.21.15.44:0/196718681 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af008d9520 0x55af008db960 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=845fe674d6a88e80 server_cookie=b8acf46fe316c2ff in_seq=0 out_seq=0 2023-09-10T01:33:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.610+0000 7fd37dffb700 1 -- 172.21.15.44:0/196718681 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55af00826840 con 0x55af008d9520 2023-09-10T01:33:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.610+0000 7fd37dffb700 10 client.30245 handle_command_reply: tid=0 2023-09-10T01:33:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.610+0000 7fd3a8fcfb80 1 client.30245 shutdown 2023-09-10T01:33:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.610+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/196718681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008bf300 msgr2=0x55af008cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.610+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/196718681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008bf300 0x55af008cbd40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd38001dbf0 tx=0x7fd3800047c0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.610+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/196718681 shutdown_connections 2023-09-10T01:33:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.610+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/196718681 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af008d9520 0x55af008db960 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.610+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/196718681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 0x55af008c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.610+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/196718681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008bf300 0x55af008cbd40 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.610+0000 7fd37dffb700 0 client.30245 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:33:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.610+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/196718681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008bfcc0 0x55af008c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.610+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/196718681 >> 172.21.15.44:0/196718681 conn(0x55af00834030 msgr2=0x55af008b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.610+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/196718681 shutdown_connections 2023-09-10T01:33:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.611+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/196718681 wait complete. 2023-09-10T01:33:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.611+0000 7fd3a8fcfb80 20 client.30245 trim_cache size 0 max 16384 2023-09-10T01:33:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.639+0000 7fd3a8fcfb80 1 Processor -- start 2023-09-10T01:33:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.639+0000 7fd3a8fcfb80 1 -- start start 2023-09-10T01:33:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.639+0000 7fd3a8fcfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008bf300 0x55af008d0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.639+0000 7fd3a8fcfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008bfcc0 0x55af008cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.639+0000 7fd3a8fcfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 0x55af008c1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.639+0000 7fd3a8fcfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af008b6fc0 con 0x55af008c51a0 2023-09-10T01:33:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.639+0000 7fd3a8fcfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af008bbd10 con 0x55af008bfcc0 2023-09-10T01:33:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.639+0000 7fd3a8fcfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af008d5c80 con 0x55af008bf300 2023-09-10T01:33:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.639+0000 7fd396d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 0x55af008c1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.639+0000 7fd396d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 0x55af008c1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60880/0 (socket says 172.21.15.44:60880) 2023-09-10T01:33:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.639+0000 7fd396d9d700 1 -- 172.21.15.44:0/1729057258 learned_addr learned my addr 172.21.15.44:0/1729057258 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.639+0000 7fd396d9d700 1 -- 172.21.15.44:0/1729057258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008bf300 msgr2=0x55af008d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.639+0000 7fd396d9d700 1 --2- 172.21.15.44:0/1729057258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008bf300 0x55af008d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.639+0000 7fd396d9d700 1 -- 172.21.15.44:0/1729057258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008bfcc0 msgr2=0x55af008cb8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:33:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.639+0000 7fd396d9d700 1 --2- 172.21.15.44:0/1729057258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008bfcc0 0x55af008cb8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.639+0000 7fd396d9d700 1 -- 172.21.15.44:0/1729057258 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af00626c00 con 0x55af008c51a0 2023-09-10T01:33:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.640+0000 7fd396d9d700 1 --2- 172.21.15.44:0/1729057258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 0x55af008c1b60 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fd3900cb020 tx=0x7fd39000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7508e96ae406fa39 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.640+0000 7fd38d59a700 1 -- 172.21.15.44:0/1729057258 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39009a6c0 con 0x55af008c51a0 2023-09-10T01:33:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.640+0000 7fd38d59a700 1 -- 172.21.15.44:0/1729057258 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd39009b460 con 0x55af008c51a0 2023-09-10T01:33:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.640+0000 7fd38d59a700 1 -- 172.21.15.44:0/1729057258 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39001d730 con 0x55af008c51a0 2023-09-10T01:33:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.641+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/1729057258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 msgr2=0x55af008c1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.641+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/1729057258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 0x55af008c1b60 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fd3900cb020 tx=0x7fd39000b3e0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.641+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/1729057258 shutdown_connections 2023-09-10T01:33:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.641+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/1729057258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 0x55af008c1b60 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.641+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/1729057258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008bf300 0x55af008d0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.641+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/1729057258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008bfcc0 0x55af008cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.641+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/1729057258 >> 172.21.15.44:0/1729057258 conn(0x55af00834030 msgr2=0x55af008ba090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.648+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/1729057258 shutdown_connections 2023-09-10T01:33:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.651+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/1729057258 wait complete. 2023-09-10T01:33:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.651+0000 7fd3a8fcfb80 1 -- start start 2023-09-10T01:33:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.651+0000 7fd3a8fcfb80 1 Processor -- start 2023-09-10T01:33:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.655+0000 7fd3a8fcfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008bf300 0x55af008c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.655+0000 7fd3a8fcfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 0x55af008c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.655+0000 7fd3a8fcfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008c7d10 0x55af008c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.655+0000 7fd3a8fcfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af008c9240 con 0x55af008c51a0 2023-09-10T01:33:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.655+0000 7fd3a8fcfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af008c93b0 con 0x55af008c7d10 2023-09-10T01:33:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.655+0000 7fd3a8fcfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0092d3b0 con 0x55af008bf300 2023-09-10T01:33:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.655+0000 7fd3a8fcfb80 10 client.0 _ll_register_callbacks cb 0x7fd38001d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:33:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.660+0000 7fd38dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 0x55af008c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.660+0000 7fd396d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008c7d10 0x55af008c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.660+0000 7fd38dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 0x55af008c77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60886/0 (socket says 172.21.15.44:60886) 2023-09-10T01:33:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.660+0000 7fd38dd9b700 1 -- 172.21.15.44:0/3450116859 learned_addr learned my addr 172.21.15.44:0/3450116859 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.660+0000 7fd39659c700 1 --2- 172.21.15.44:0/3450116859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008bf300 0x55af008c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.664+0000 7fd396d9d700 1 -- 172.21.15.44:0/3450116859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008bf300 msgr2=0x55af008c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.664+0000 7fd396d9d700 1 --2- 172.21.15.44:0/3450116859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008bf300 0x55af008c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.664+0000 7fd396d9d700 1 -- 172.21.15.44:0/3450116859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 msgr2=0x55af008c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.664+0000 7fd396d9d700 1 --2- 172.21.15.44:0/3450116859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 0x55af008c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.664+0000 7fd396d9d700 1 -- 172.21.15.44:0/3450116859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3981a13f0 con 0x55af008c7d10 2023-09-10T01:33:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.664+0000 7fd396d9d700 1 --2- 172.21.15.44:0/3450116859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008c7d10 0x55af008c8190 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd3900cb020 tx=0x7fd39009a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56735c0d0574a208 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.665+0000 7fd37dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:33:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.665+0000 7fd37dffb700 1 -- 172.21.15.44:0/3450116859 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39009eb90 con 0x55af008c7d10 2023-09-10T01:33:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.665+0000 7fd37dffb700 1 -- 172.21.15.44:0/3450116859 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(204..204 src has 1..204) v4 ==== 9496+0+0 (secure 0 0 0) 0x7fd3900a2d40 con 0x55af008c7d10 2023-09-10T01:33:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.665+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/3450116859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd388015070 con 0x55af008c7d10 2023-09-10T01:33:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.668+0000 7fd37dffb700 1 -- 172.21.15.44:0/3450116859 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=191) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd39001ce80 con 0x55af008c7d10 2023-09-10T01:33:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.668+0000 7fd3a8fcfb80 10 client.30248 fetch_fsmap learned FSMap version 191 2023-09-10T01:33:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.668+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/3450116859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=191,monmap=4+}) v3 -- 0x55af006641f0 con 0x55af008c7d10 2023-09-10T01:33:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.668+0000 7fd37dffb700 1 -- 172.21.15.44:0/3450116859 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd388015070 con 0x55af008c7d10 2023-09-10T01:33:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.668+0000 7fd37dffb700 1 -- 172.21.15.44:0/3450116859 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 191) v1 ==== 7363+0+0 (secure 0 0 0) 0x7fd3900a28b0 con 0x55af008c7d10 2023-09-10T01:33:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.668+0000 7fd3a8fcfb80 10 client.30248 fetch_fsmap finished waiting for FSMap version 191 2023-09-10T01:33:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.668+0000 7fd3a8fcfb80 10 client.30248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:33:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.668+0000 7fd3a8fcfb80 20 client.30248 populate_metadata read hostname 'smithi044' 2023-09-10T01:33:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.668+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/3450116859 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af0092e690 0x55af00930ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.668+0000 7fd3a8fcfb80 4 client.30248 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:33:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.668+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/3450116859 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55af0080a8c0 con 0x55af0092e690 2023-09-10T01:33:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.669+0000 7fd39659c700 1 --2- 172.21.15.44:0/3450116859 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af0092e690 0x55af00930ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.669+0000 7fd37dffb700 10 client.30248 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:33:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.669+0000 7fd39659c700 1 --2- 172.21.15.44:0/3450116859 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af0092e690 0x55af00930ab0 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d873310b03683cb1 server_cookie=a72aabb9a83848c8 in_seq=0 out_seq=0 2023-09-10T01:33:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.722+0000 7fd37dffb700 1 -- 172.21.15.44:0/3450116859 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55af0080a8c0 con 0x55af0092e690 2023-09-10T01:33:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.722+0000 7fd37dffb700 10 client.30248 handle_command_reply: tid=0 2023-09-10T01:33:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.722+0000 7fd3a8fcfb80 1 client.30248 shutdown 2023-09-10T01:33:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.723+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/3450116859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008c7d10 msgr2=0x55af008c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.723+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/3450116859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008c7d10 0x55af008c8190 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd3900cb020 tx=0x7fd39009a6c0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.723+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/3450116859 shutdown_connections 2023-09-10T01:33:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.723+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/3450116859 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55af0092e690 0x55af00930ab0 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.723+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/3450116859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af008c7d10 0x55af008c8190 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.723+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/3450116859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55af008bf300 0x55af008c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.723+0000 7fd3a8fcfb80 1 --2- 172.21.15.44:0/3450116859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af008c51a0 0x55af008c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.723+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/3450116859 >> 172.21.15.44:0/3450116859 conn(0x55af00834030 msgr2=0x55af008c4140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.723+0000 7fd37dffb700 0 client.30248 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:33:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.725+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/3450116859 shutdown_connections 2023-09-10T01:33:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.725+0000 7fd3a8fcfb80 1 -- 172.21.15.44:0/3450116859 wait complete. 2023-09-10T01:33:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.726+0000 7fd3a8fcfb80 20 client.30248 trim_cache size 0 max 16384 2023-09-10T01:33:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:33:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:33:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:33:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:33:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.726+0000 7fd39ce09700 1 -- 172.21.15.44:0/3849535599 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd3840a07b0 msgr2=0x7fd3840a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.726+0000 7fd39ce09700 1 --2- 172.21.15.44:0/3849535599 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd3840a07b0 0x7fd3840a2c70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd380009d20 tx=0x7fd380009260 comp rx=0 tx=0).stop 2023-09-10T01:33:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.726+0000 7fd39ce09700 1 -- 172.21.15.44:0/3849535599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3981055b0 msgr2=0x7fd3981075b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.726+0000 7fd39ce09700 1 --2- 172.21.15.44:0/3849535599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3981055b0 0x7fd3981075b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd390009fd0 tx=0x7fd39000cce0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.727+0000 7fd39ce09700 1 -- 172.21.15.44:0/3849535599 shutdown_connections 2023-09-10T01:33:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.727+0000 7fd39ce09700 1 --2- 172.21.15.44:0/3849535599 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd3840a07b0 0x7fd3840a2c70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.727+0000 7fd39ce09700 1 --2- 172.21.15.44:0/3849535599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3981055b0 0x7fd3981075b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.727+0000 7fd39ce09700 1 --2- 172.21.15.44:0/3849535599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3980ff760 0x7fd39810c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.727+0000 7fd39ce09700 1 --2- 172.21.15.44:0/3849535599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd398100120 0x7fd398106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.727+0000 7fd39ce09700 1 -- 172.21.15.44:0/3849535599 >> 172.21.15.44:0/3849535599 conn(0x7fd398074f30 msgr2=0x7fd3980fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:35.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.732+0000 7fd39ce09700 1 -- 172.21.15.44:0/3849535599 shutdown_connections 2023-09-10T01:33:35.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:35.733+0000 7fd39ce09700 1 -- 172.21.15.44:0/3849535599 wait complete. 2023-09-10T01:33:36.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:35 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:33:36.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:35 smithi131 ceph-mon[103713]: pgmap v1725: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.8 MiB/s wr, 789 op/s 2023-09-10T01:33:36.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:35 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:33:36.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:35 smithi044 ceph-mon[98676]: pgmap v1725: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.8 MiB/s wr, 789 op/s 2023-09-10T01:33:36.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:35 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:33:36.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:35 smithi099 ceph-mon[102692]: pgmap v1725: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.8 MiB/s wr, 789 op/s 2023-09-10T01:33:36.184 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 750.0 s, 292.0 tps, lat 298.241 ms stddev 409.661 2023-09-10T01:33:38.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:37 smithi131 ceph-mon[103713]: pgmap v1726: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.7 MiB/s wr, 819 op/s 2023-09-10T01:33:38.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:37 smithi044 ceph-mon[98676]: pgmap v1726: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.7 MiB/s wr, 819 op/s 2023-09-10T01:33:38.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:37 smithi099 ceph-mon[102692]: pgmap v1726: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.7 MiB/s wr, 819 op/s 2023-09-10T01:33:40.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:39 smithi099 ceph-mon[102692]: pgmap v1727: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.0 MiB/s wr, 778 op/s 2023-09-10T01:33:40.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:39 smithi131 ceph-mon[103713]: pgmap v1727: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.0 MiB/s wr, 778 op/s 2023-09-10T01:33:40.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:39 smithi044 ceph-mon[98676]: pgmap v1727: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.0 MiB/s wr, 778 op/s 2023-09-10T01:33:41.182 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 755.0 s, 68.2 tps, lat 932.804 ms stddev 1058.763 2023-09-10T01:33:42.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:41 smithi099 ceph-mon[102692]: pgmap v1728: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.2 MiB/s wr, 703 op/s 2023-09-10T01:33:42.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:41 smithi131 ceph-mon[103713]: pgmap v1728: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.2 MiB/s wr, 703 op/s 2023-09-10T01:33:42.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:41 smithi044 ceph-mon[98676]: pgmap v1728: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.2 MiB/s wr, 703 op/s 2023-09-10T01:33:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:42 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:33:43.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:42 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:33:43.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:42 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:33:44.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:43 smithi131 ceph-mon[103713]: pgmap v1729: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 793 op/s 2023-09-10T01:33:44.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:43 smithi044 ceph-mon[98676]: pgmap v1729: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 793 op/s 2023-09-10T01:33:44.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:43 smithi099 ceph-mon[102692]: pgmap v1729: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 793 op/s 2023-09-10T01:33:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 760.0 s, 276.1 tps, lat 533.805 ms stddev 864.835 2023-09-10T01:33:46.299 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:45 smithi131 ceph-mon[103713]: pgmap v1730: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 600 op/s 2023-09-10T01:33:46.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:45 smithi044 ceph-mon[98676]: pgmap v1730: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 600 op/s 2023-09-10T01:33:46.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:45 smithi099 ceph-mon[102692]: pgmap v1730: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 600 op/s 2023-09-10T01:33:48.046 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:47 smithi131 ceph-mon[103713]: pgmap v1731: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 762 op/s 2023-09-10T01:33:48.060 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:47 smithi044 ceph-mon[98676]: pgmap v1731: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 762 op/s 2023-09-10T01:33:48.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:47 smithi099 ceph-mon[102692]: pgmap v1731: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 762 op/s 2023-09-10T01:33:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.949+0000 7fc6ba3d7700 1 -- 172.21.15.44:0/2997944415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6b4119de0 msgr2=0x7fc6b411c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.949+0000 7fc6ba3d7700 1 --2- 172.21.15.44:0/2997944415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6b4119de0 0x7fc6b411c1d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009f90 tx=0x7fc6ac00b330 comp rx=0 tx=0).stop 2023-09-10T01:33:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.950+0000 7fc6ba3d7700 1 -- 172.21.15.44:0/2997944415 shutdown_connections 2023-09-10T01:33:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.950+0000 7fc6ba3d7700 1 --2- 172.21.15.44:0/2997944415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6b4119de0 0x7fc6b411c1d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.950+0000 7fc6ba3d7700 1 --2- 172.21.15.44:0/2997944415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b410ed50 0x7fc6b4107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.950+0000 7fc6ba3d7700 1 --2- 172.21.15.44:0/2997944415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6b410f1c0 0x7fc6b4107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.950+0000 7fc6ba3d7700 1 -- 172.21.15.44:0/2997944415 >> 172.21.15.44:0/2997944415 conn(0x7fc6b406c6c0 msgr2=0x7fc6b406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.952+0000 7fc6ba3d7700 1 -- 172.21.15.44:0/2997944415 shutdown_connections 2023-09-10T01:33:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.955+0000 7fc6ba3d7700 1 -- 172.21.15.44:0/2997944415 wait complete. 2023-09-10T01:33:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.956+0000 7fc6ba3d7700 1 Processor -- start 2023-09-10T01:33:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.959+0000 7fc6ba3d7700 1 -- start start 2023-09-10T01:33:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.960+0000 7fc6ba3d7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6b410ed50 0x7fc6b4118290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.960+0000 7fc6ba3d7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6b410f1c0 0x7fc6b4118810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.960+0000 7fc6ba3d7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b4119de0 0x7fc6b41112c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.960+0000 7fc6ba3d7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b4111a00 con 0x7fc6b4119de0 2023-09-10T01:33:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.960+0000 7fc6ba3d7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b4111b70 con 0x7fc6b410ed50 2023-09-10T01:33:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.960+0000 7fc6ba3d7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b4111d30 con 0x7fc6b410f1c0 2023-09-10T01:33:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.961+0000 7fc6b37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6b410f1c0 0x7fc6b4118810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.961+0000 7fc6b37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6b410f1c0 0x7fc6b4118810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39546/0 (socket says 172.21.15.44:39546) 2023-09-10T01:33:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.961+0000 7fc6b37fe700 1 -- 172.21.15.44:0/231208317 learned_addr learned my addr 172.21.15.44:0/231208317 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.962+0000 7fc6b37fe700 1 -- 172.21.15.44:0/231208317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6b410ed50 msgr2=0x7fc6b4118290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T01:33:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.962+0000 7fc6b37fe700 1 --2- 172.21.15.44:0/231208317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6b410ed50 0x7fc6b4118290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.962+0000 7fc6b37fe700 1 -- 172.21.15.44:0/231208317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b4119de0 msgr2=0x7fc6b41112c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T01:33:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.962+0000 7fc6b37fe700 1 --2- 172.21.15.44:0/231208317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b4119de0 0x7fc6b41112c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.962+0000 7fc6b37fe700 1 -- 172.21.15.44:0/231208317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac009c40 con 0x7fc6b410f1c0 2023-09-10T01:33:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.962+0000 7fc6b37fe700 1 --2- 172.21.15.44:0/231208317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6b410f1c0 0x7fc6b4118810 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400b550 tx=0x7fc6a400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.962+0000 7fc6b17fa700 1 -- 172.21.15.44:0/231208317 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4011830 con 0x7fc6b410f1c0 2023-09-10T01:33:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.962+0000 7fc6ba3d7700 1 -- 172.21.15.44:0/231208317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b4115e20 con 0x7fc6b410f1c0 2023-09-10T01:33:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.962+0000 7fc6ba3d7700 1 -- 172.21.15.44:0/231208317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b4116380 con 0x7fc6b410f1c0 2023-09-10T01:33:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.967+0000 7fc6c659db80 1 Processor -- start 2023-09-10T01:33:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.967+0000 7fc6b17fa700 1 -- 172.21.15.44:0/231208317 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc6a4011e20 con 0x7fc6b410f1c0 2023-09-10T01:33:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.967+0000 7fc6b17fa700 1 -- 172.21.15.44:0/231208317 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4010e10 con 0x7fc6b410f1c0 2023-09-10T01:33:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.967+0000 7fc6b17fa700 1 -- 172.21.15.44:0/231208317 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc6a401a420 con 0x7fc6b410f1c0 2023-09-10T01:33:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.968+0000 7fc6b17fa700 1 --2- 172.21.15.44:0/231208317 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc69c0a0930 0x7fc69c0a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.969+0000 7fc6b17fa700 1 -- 172.21.15.44:0/231208317 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 9496+0+0 (secure 0 0 0) 0x7fc6a40cccc0 con 0x7fc6b410f1c0 2023-09-10T01:33:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.974+0000 7fc6c659db80 1 -- start start 2023-09-10T01:33:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.974+0000 7fc6c659db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5f51a0 0x56470d5f5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.974+0000 7fc6c659db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5ef300 0x56470d5ef780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.974+0000 7fc6c659db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5efcc0 0x56470d5fbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.974+0000 7fc6c659db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470d5486d0 con 0x56470d5ef300 2023-09-10T01:33:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.974+0000 7fc6c659db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470d545ff0 con 0x56470d5efcc0 2023-09-10T01:33:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.974+0000 7fc6c659db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470d5418e0 con 0x56470d5f51a0 2023-09-10T01:33:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.974+0000 7fc6b3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5f51a0 0x56470d5f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.974+0000 7fc6b3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5f51a0 0x56470d5f5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39550/0 (socket says 172.21.15.44:39550) 2023-09-10T01:33:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.974+0000 7fc6b3fff700 1 -- 172.21.15.44:0/2042793874 learned_addr learned my addr 172.21.15.44:0/2042793874 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.975+0000 7fc6b37fe700 1 --2- 172.21.15.44:0/2042793874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5ef300 0x56470d5ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.975+0000 7fc6b3fff700 1 -- 172.21.15.44:0/2042793874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5efcc0 msgr2=0x56470d5fbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.975+0000 7fc6b3fff700 1 --2- 172.21.15.44:0/2042793874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5efcc0 0x56470d5fbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.975+0000 7fc6b3fff700 1 -- 172.21.15.44:0/2042793874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5ef300 msgr2=0x56470d5ef780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.975+0000 7fc6b3fff700 1 --2- 172.21.15.44:0/2042793874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5ef300 0x56470d5ef780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.975+0000 7fc6b3fff700 1 -- 172.21.15.44:0/2042793874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56470d36dd80 con 0x56470d5f51a0 2023-09-10T01:33:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.975+0000 7fc6b3fff700 1 --2- 172.21.15.44:0/2042793874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5f51a0 0x56470d5f5580 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009b50 tx=0x7fc6a8009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c33bd09b0b0cf68e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.975+0000 7fc69a7fc700 1 -- 172.21.15.44:0/2042793874 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8004a30 con 0x56470d5f51a0 2023-09-10T01:33:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.976+0000 7fc69a7fc700 1 -- 172.21.15.44:0/2042793874 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc6a800ecf0 con 0x56470d5f51a0 2023-09-10T01:33:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.976+0000 7fc69a7fc700 1 -- 172.21.15.44:0/2042793874 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8005530 con 0x56470d5f51a0 2023-09-10T01:33:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.977+0000 7fc6c659db80 1 -- 172.21.15.44:0/2042793874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5f51a0 msgr2=0x56470d5f5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.977+0000 7fc6c659db80 1 --2- 172.21.15.44:0/2042793874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5f51a0 0x56470d5f5580 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009b50 tx=0x7fc6a8009e60 comp rx=0 tx=0).stop 2023-09-10T01:33:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.977+0000 7fc6c659db80 1 -- 172.21.15.44:0/2042793874 shutdown_connections 2023-09-10T01:33:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.977+0000 7fc6c659db80 1 --2- 172.21.15.44:0/2042793874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5efcc0 0x56470d5fbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.977+0000 7fc6c659db80 1 --2- 172.21.15.44:0/2042793874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5f51a0 0x56470d5f5580 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.977+0000 7fc6c659db80 1 --2- 172.21.15.44:0/2042793874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5ef300 0x56470d5ef780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.977+0000 7fc6c659db80 1 -- 172.21.15.44:0/2042793874 >> 172.21.15.44:0/2042793874 conn(0x56470d564030 msgr2=0x56470d5ea560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.977+0000 7fc6c659db80 1 -- 172.21.15.44:0/2042793874 shutdown_connections 2023-09-10T01:33:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.977+0000 7fc6c659db80 1 -- 172.21.15.44:0/2042793874 wait complete. 2023-09-10T01:33:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.977+0000 7fc6b8974700 1 --2- 172.21.15.44:0/231208317 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc69c0a0930 0x7fc69c0a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.978+0000 7fc6c659db80 1 -- start start 2023-09-10T01:33:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.978+0000 7fc6c659db80 1 Processor -- start 2023-09-10T01:33:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.978+0000 7fc6c659db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5ef300 0x56470d5fbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.978+0000 7fc6c659db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5efcc0 0x56470d5f9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.978+0000 7fc6c659db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5f51a0 0x56470d5f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.978+0000 7fc6c659db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470d4dfb10 con 0x56470d5f51a0 2023-09-10T01:33:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.978+0000 7fc6c659db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470d4df810 con 0x56470d5ef300 2023-09-10T01:33:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.978+0000 7fc6c659db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470d3c38c0 con 0x56470d5efcc0 2023-09-10T01:33:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.979+0000 7fc6c659db80 10 client.0 _ll_register_callbacks cb 0x56470d429a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:33:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.979+0000 7fc6b8974700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5f51a0 0x56470d5f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.979+0000 7fc6b8974700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5f51a0 0x56470d5f95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60906/0 (socket says 172.21.15.44:60906) 2023-09-10T01:33:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.979+0000 7fc6b8974700 1 -- 172.21.15.44:0/16600644 learned_addr learned my addr 172.21.15.44:0/16600644 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.979+0000 7fc6b37fe700 1 --2- 172.21.15.44:0/16600644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5efcc0 0x56470d5f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.979+0000 7fc6b8974700 1 --2- 172.21.15.44:0/231208317 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc69c0a0930 0x7fc69c0a2df0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac006070 tx=0x7fc6ac006000 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.980+0000 7fc6b8974700 1 -- 172.21.15.44:0/16600644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5efcc0 msgr2=0x56470d5f9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.980+0000 7fc6b8974700 1 --2- 172.21.15.44:0/16600644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5efcc0 0x56470d5f9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.980+0000 7fc6b8974700 1 -- 172.21.15.44:0/16600644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5ef300 msgr2=0x56470d5fbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.980+0000 7fc6b8974700 1 --2- 172.21.15.44:0/16600644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5ef300 0x56470d5fbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.980+0000 7fc6b8974700 1 -- 172.21.15.44:0/16600644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a8009800 con 0x56470d5f51a0 2023-09-10T01:33:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.980+0000 7fc6b8974700 1 --2- 172.21.15.44:0/16600644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5f51a0 0x56470d5f95b0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac007800 tx=0x7fc6ac009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49516224d6c05dda server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.980+0000 7fc6937fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:33:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.980+0000 7fc6937fe700 1 -- 172.21.15.44:0/16600644 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac00be00 con 0x56470d5f51a0 2023-09-10T01:33:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.980+0000 7fc6937fe700 1 -- 172.21.15.44:0/16600644 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(204..204 src has 1..204) v4 ==== 9496+0+0 (secure 0 0 0) 0x7fc6ac003e10 con 0x56470d5f51a0 2023-09-10T01:33:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.980+0000 7fc6c659db80 1 -- 172.21.15.44:0/16600644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56470d499490 con 0x56470d5f51a0 2023-09-10T01:33:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.981+0000 7fc6937fe700 1 -- 172.21.15.44:0/16600644 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=192) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6ac0050b0 con 0x56470d5f51a0 2023-09-10T01:33:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.986+0000 7fc6c659db80 10 client.20601 fetch_fsmap learned FSMap version 192 2023-09-10T01:33:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.986+0000 7fc6c659db80 1 -- 172.21.15.44:0/16600644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=192,monmap=4+}) v3 -- 0x56470d356c00 con 0x56470d5f51a0 2023-09-10T01:33:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.990+0000 7fc6937fe700 1 -- 172.21.15.44:0/16600644 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56470d499490 con 0x56470d5f51a0 2023-09-10T01:33:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.990+0000 7fc6937fe700 1 -- 172.21.15.44:0/16600644 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 192) v1 ==== 7359+0+0 (secure 0 0 0) 0x7fc6ac01b6d0 con 0x56470d5f51a0 2023-09-10T01:33:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.990+0000 7fc6c659db80 10 client.20601 fetch_fsmap finished waiting for FSMap version 192 2023-09-10T01:33:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.990+0000 7fc6c659db80 10 client.20601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:33:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.990+0000 7fc6c659db80 20 client.20601 populate_metadata read hostname 'smithi044' 2023-09-10T01:33:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.990+0000 7fc6c659db80 1 --2- 172.21.15.44:0/16600644 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56470d5feec0 0x56470d601300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.990+0000 7fc6c659db80 4 client.20601 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:33:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.990+0000 7fc6c659db80 1 -- 172.21.15.44:0/16600644 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56470d556840 con 0x56470d5feec0 2023-09-10T01:33:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.991+0000 7fc6b3fff700 1 --2- 172.21.15.44:0/16600644 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56470d5feec0 0x56470d601300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.991+0000 7fc6937fe700 10 client.20601 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:33:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.991+0000 7fc6b3fff700 1 --2- 172.21.15.44:0/16600644 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56470d5feec0 0x56470d601300 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c45b3dd17486db server_cookie=3155472d6613918e in_seq=0 out_seq=0 2023-09-10T01:33:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.993+0000 7fc6937fe700 1 -- 172.21.15.44:0/16600644 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56470d556840 con 0x56470d5feec0 2023-09-10T01:33:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.993+0000 7fc6937fe700 10 client.20601 handle_command_reply: tid=0 2023-09-10T01:33:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.994+0000 7fc6c659db80 1 client.20601 shutdown 2023-09-10T01:33:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.994+0000 7fc6c659db80 1 -- 172.21.15.44:0/16600644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5f51a0 msgr2=0x56470d5f95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:48.994+0000 7fc6c659db80 1 --2- 172.21.15.44:0/16600644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5f51a0 0x56470d5f95b0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac007800 tx=0x7fc6ac009920 comp rx=0 tx=0).stop 2023-09-10T01:33:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.001+0000 7fc6c659db80 1 -- 172.21.15.44:0/16600644 shutdown_connections 2023-09-10T01:33:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.001+0000 7fc6c659db80 1 --2- 172.21.15.44:0/16600644 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56470d5feec0 0x56470d601300 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.001+0000 7fc6c659db80 1 --2- 172.21.15.44:0/16600644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5f51a0 0x56470d5f95b0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.001+0000 7fc6c659db80 1 --2- 172.21.15.44:0/16600644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5ef300 0x56470d5fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.001+0000 7fc6c659db80 1 --2- 172.21.15.44:0/16600644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5efcc0 0x56470d5f9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.001+0000 7fc6c659db80 1 -- 172.21.15.44:0/16600644 >> 172.21.15.44:0/16600644 conn(0x56470d564030 msgr2=0x56470d5e97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.001+0000 7fc6937fe700 0 client.20601 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:33:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.002+0000 7fc6c659db80 1 -- 172.21.15.44:0/16600644 shutdown_connections 2023-09-10T01:33:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.002+0000 7fc6c659db80 1 -- 172.21.15.44:0/16600644 wait complete. 2023-09-10T01:33:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.006+0000 7fc6c659db80 20 client.20601 trim_cache size 0 max 16384 2023-09-10T01:33:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.031+0000 7fc6c659db80 1 Processor -- start 2023-09-10T01:33:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.031+0000 7fc6c659db80 1 -- start start 2023-09-10T01:33:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6c659db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5ef300 0x56470d60eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6c659db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5efcc0 0x56470d5f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6c659db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5f51a0 0x56470d5f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6c659db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470d5f00a0 con 0x56470d5f51a0 2023-09-10T01:33:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6c659db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470d5f5580 con 0x56470d5ef300 2023-09-10T01:33:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6c659db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470d5f0580 con 0x56470d5efcc0 2023-09-10T01:33:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6b37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5efcc0 0x56470d5f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6b8974700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5f51a0 0x56470d5f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6b37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5efcc0 0x56470d5f30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39566/0 (socket says 172.21.15.44:39566) 2023-09-10T01:33:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6b37fe700 1 -- 172.21.15.44:0/435208446 learned_addr learned my addr 172.21.15.44:0/435208446 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6b3fff700 1 --2- 172.21.15.44:0/435208446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5ef300 0x56470d60eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6b8974700 1 -- 172.21.15.44:0/435208446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5efcc0 msgr2=0x56470d5f30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6b8974700 1 --2- 172.21.15.44:0/435208446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5efcc0 0x56470d5f30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6b8974700 1 -- 172.21.15.44:0/435208446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5ef300 msgr2=0x56470d60eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6b8974700 1 --2- 172.21.15.44:0/435208446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5ef300 0x56470d60eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6b8974700 1 -- 172.21.15.44:0/435208446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56470d3941f0 con 0x56470d5f51a0 2023-09-10T01:33:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6b37fe700 1 --2- 172.21.15.44:0/435208446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5efcc0 0x56470d5f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:33:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6b3fff700 1 --2- 172.21.15.44:0/435208446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5ef300 0x56470d60eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:33:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.032+0000 7fc6b8974700 1 --2- 172.21.15.44:0/435208446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5f51a0 0x56470d5f35f0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac007800 tx=0x7fc6ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48691bf5b7b20656 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.033+0000 7fc69affd700 1 -- 172.21.15.44:0/435208446 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac01bea0 con 0x56470d5f51a0 2023-09-10T01:33:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.033+0000 7fc69affd700 1 -- 172.21.15.44:0/435208446 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc6ac00f020 con 0x56470d5f51a0 2023-09-10T01:33:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.033+0000 7fc69affd700 1 -- 172.21.15.44:0/435208446 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0059a0 con 0x56470d5f51a0 2023-09-10T01:33:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.033+0000 7fc6c659db80 1 -- 172.21.15.44:0/435208446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5f51a0 msgr2=0x56470d5f35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.033+0000 7fc6c659db80 1 --2- 172.21.15.44:0/435208446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5f51a0 0x56470d5f35f0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac007800 tx=0x7fc6ac00b330 comp rx=0 tx=0).stop 2023-09-10T01:33:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.033+0000 7fc6c659db80 1 -- 172.21.15.44:0/435208446 shutdown_connections 2023-09-10T01:33:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.033+0000 7fc6c659db80 1 --2- 172.21.15.44:0/435208446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5f51a0 0x56470d5f35f0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.033+0000 7fc6c659db80 1 --2- 172.21.15.44:0/435208446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5ef300 0x56470d60eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.033+0000 7fc6c659db80 1 --2- 172.21.15.44:0/435208446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5efcc0 0x56470d5f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.033+0000 7fc6c659db80 1 -- 172.21.15.44:0/435208446 >> 172.21.15.44:0/435208446 conn(0x56470d564030 msgr2=0x56470d600ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.035+0000 7fc6c659db80 1 -- 172.21.15.44:0/435208446 shutdown_connections 2023-09-10T01:33:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.035+0000 7fc6c659db80 1 -- 172.21.15.44:0/435208446 wait complete. 2023-09-10T01:33:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.035+0000 7fc6c659db80 1 -- start start 2023-09-10T01:33:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.035+0000 7fc6c659db80 1 Processor -- start 2023-09-10T01:33:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.035+0000 7fc6c659db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5ef300 0x56470d650d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.035+0000 7fc6c659db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5efcc0 0x56470d651260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.035+0000 7fc6c659db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5f51a0 0x56470d65bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.035+0000 7fc6c659db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470d651950 con 0x56470d5efcc0 2023-09-10T01:33:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.035+0000 7fc6c659db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470d65c0f0 con 0x56470d5ef300 2023-09-10T01:33:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.035+0000 7fc6c659db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470d65c230 con 0x56470d5f51a0 2023-09-10T01:33:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.036+0000 7fc6c659db80 10 client.0 _ll_register_callbacks cb 0x7fc6ac0073e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:33:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.036+0000 7fc6b8974700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5f51a0 0x56470d65bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.036+0000 7fc6b37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5efcc0 0x56470d651260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.036+0000 7fc6b8974700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5f51a0 0x56470d65bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39572/0 (socket says 172.21.15.44:39572) 2023-09-10T01:33:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.036+0000 7fc6b8974700 1 -- 172.21.15.44:0/609462537 learned_addr learned my addr 172.21.15.44:0/609462537 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:33:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.036+0000 7fc6b37fe700 1 -- 172.21.15.44:0/609462537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5f51a0 msgr2=0x56470d65bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:33:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.036+0000 7fc6b37fe700 1 --2- 172.21.15.44:0/609462537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5f51a0 0x56470d65bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.036+0000 7fc6b37fe700 1 -- 172.21.15.44:0/609462537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5ef300 msgr2=0x56470d650d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:33:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.036+0000 7fc6b37fe700 1 --2- 172.21.15.44:0/609462537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5ef300 0x56470d650d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.036+0000 7fc6b37fe700 1 -- 172.21.15.44:0/609462537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a801c040 con 0x56470d5efcc0 2023-09-10T01:33:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.036+0000 7fc6b37fe700 1 --2- 172.21.15.44:0/609462537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5efcc0 0x56470d651260 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4099bd0 tx=0x7fc6a4099fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63717d51d1f09c4e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:33:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.036+0000 7fc6937fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:33:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.037+0000 7fc6937fe700 1 -- 172.21.15.44:0/609462537 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a409a330 con 0x56470d5efcc0 2023-09-10T01:33:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.037+0000 7fc6937fe700 1 -- 172.21.15.44:0/609462537 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(204..204 src has 1..204) v4 ==== 9496+0+0 (secure 0 0 0) 0x7fc6a40ca4f0 con 0x56470d5efcc0 2023-09-10T01:33:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.037+0000 7fc6c659db80 1 -- 172.21.15.44:0/609462537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6a800ecf0 con 0x56470d5efcc0 2023-09-10T01:33:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.037+0000 7fc6937fe700 1 -- 172.21.15.44:0/609462537 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=192) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6a40b2200 con 0x56470d5efcc0 2023-09-10T01:33:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.037+0000 7fc6c659db80 10 client.20607 fetch_fsmap learned FSMap version 192 2023-09-10T01:33:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.037+0000 7fc6c659db80 1 -- 172.21.15.44:0/609462537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=192,monmap=4+}) v3 -- 0x7fc6b4116380 con 0x56470d5efcc0 2023-09-10T01:33:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.037+0000 7fc6937fe700 1 -- 172.21.15.44:0/609462537 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc6a800ecf0 con 0x56470d5efcc0 2023-09-10T01:33:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.037+0000 7fc6937fe700 1 -- 172.21.15.44:0/609462537 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 192) v1 ==== 7359+0+0 (secure 0 0 0) 0x7fc6a40a1ab0 con 0x56470d5efcc0 2023-09-10T01:33:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.037+0000 7fc6c659db80 10 client.20607 fetch_fsmap finished waiting for FSMap version 192 2023-09-10T01:33:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.037+0000 7fc6c659db80 10 client.20607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:33:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.037+0000 7fc6c659db80 20 client.20607 populate_metadata read hostname 'smithi044' 2023-09-10T01:33:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.037+0000 7fc6c659db80 1 --2- 172.21.15.44:0/609462537 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56470d65db00 0x56470d65ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:33:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.037+0000 7fc6c659db80 4 client.20607 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:33:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.037+0000 7fc6c659db80 1 -- 172.21.15.44:0/609462537 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56470d53a8c0 con 0x56470d65db00 2023-09-10T01:33:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.038+0000 7fc6b3fff700 1 --2- 172.21.15.44:0/609462537 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56470d65db00 0x56470d65ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:33:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.038+0000 7fc6937fe700 10 client.20607 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:33:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.038+0000 7fc6b3fff700 1 --2- 172.21.15.44:0/609462537 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56470d65db00 0x56470d65ff20 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88023bdff54b45ce server_cookie=d4a0d450c3263d02 in_seq=0 out_seq=0 2023-09-10T01:33:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.070+0000 7fc6937fe700 1 -- 172.21.15.44:0/609462537 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56470d53a8c0 con 0x56470d65db00 2023-09-10T01:33:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.070+0000 7fc6937fe700 10 client.20607 handle_command_reply: tid=0 2023-09-10T01:33:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.070+0000 7fc6c659db80 1 client.20607 shutdown 2023-09-10T01:33:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.080+0000 7fc6c659db80 1 -- 172.21.15.44:0/609462537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5efcc0 msgr2=0x56470d651260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.080+0000 7fc6c659db80 1 --2- 172.21.15.44:0/609462537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5efcc0 0x56470d651260 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4099bd0 tx=0x7fc6a4099fc0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.085+0000 7fc6c659db80 1 -- 172.21.15.44:0/609462537 shutdown_connections 2023-09-10T01:33:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.085+0000 7fc6c659db80 1 --2- 172.21.15.44:0/609462537 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x56470d65db00 0x56470d65ff20 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.085+0000 7fc6c659db80 1 --2- 172.21.15.44:0/609462537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56470d5f51a0 0x56470d65bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.085+0000 7fc6c659db80 1 --2- 172.21.15.44:0/609462537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56470d5ef300 0x56470d650d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.085+0000 7fc6c659db80 1 --2- 172.21.15.44:0/609462537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56470d5efcc0 0x56470d651260 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.085+0000 7fc6c659db80 1 -- 172.21.15.44:0/609462537 >> 172.21.15.44:0/609462537 conn(0x56470d564030 msgr2=0x56470d5f30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.085+0000 7fc6937fe700 0 client.20607 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:33:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.102+0000 7fc6c659db80 1 -- 172.21.15.44:0/609462537 shutdown_connections 2023-09-10T01:33:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.102+0000 7fc6c659db80 1 -- 172.21.15.44:0/609462537 wait complete. 2023-09-10T01:33:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.113+0000 7fc6c659db80 20 client.20607 trim_cache size 0 max 16384 2023-09-10T01:33:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:33:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:33:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:33:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:33:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.121+0000 7fc6ba3d7700 1 -- 172.21.15.44:0/231208317 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc69c0a0930 msgr2=0x7fc69c0a2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.121+0000 7fc6ba3d7700 1 --2- 172.21.15.44:0/231208317 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc69c0a0930 0x7fc69c0a2df0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac006070 tx=0x7fc6ac006000 comp rx=0 tx=0).stop 2023-09-10T01:33:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.121+0000 7fc6ba3d7700 1 -- 172.21.15.44:0/231208317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6b410f1c0 msgr2=0x7fc6b4118810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:33:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.121+0000 7fc6ba3d7700 1 --2- 172.21.15.44:0/231208317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6b410f1c0 0x7fc6b4118810 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400b550 tx=0x7fc6a400b910 comp rx=0 tx=0).stop 2023-09-10T01:33:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.130+0000 7fc6ba3d7700 1 -- 172.21.15.44:0/231208317 shutdown_connections 2023-09-10T01:33:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.130+0000 7fc6ba3d7700 1 --2- 172.21.15.44:0/231208317 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc69c0a0930 0x7fc69c0a2df0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.130+0000 7fc6ba3d7700 1 --2- 172.21.15.44:0/231208317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6b4119de0 0x7fc6b41112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.130+0000 7fc6ba3d7700 1 --2- 172.21.15.44:0/231208317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6b410ed50 0x7fc6b4118290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.130+0000 7fc6ba3d7700 1 --2- 172.21.15.44:0/231208317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6b410f1c0 0x7fc6b4118810 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:33:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.130+0000 7fc6ba3d7700 1 -- 172.21.15.44:0/231208317 >> 172.21.15.44:0/231208317 conn(0x7fc6b406c6c0 msgr2=0x7fc6b406efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:33:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.131+0000 7fc6ba3d7700 1 -- 172.21.15.44:0/231208317 shutdown_connections 2023-09-10T01:33:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:33:49.131+0000 7fc6ba3d7700 1 -- 172.21.15.44:0/231208317 wait complete. 2023-09-10T01:33:50.300 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:49 smithi131 ceph-mon[103713]: pgmap v1732: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 572 op/s 2023-09-10T01:33:50.310 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:49 smithi044 ceph-mon[98676]: pgmap v1732: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 572 op/s 2023-09-10T01:33:50.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:49 smithi099 ceph-mon[102692]: pgmap v1732: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 572 op/s 2023-09-10T01:33:51.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 765.0 s, 215.0 tps, lat 468.811 ms stddev 206.582 2023-09-10T01:33:52.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:51 smithi131 ceph-mon[103713]: pgmap v1733: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 513 op/s 2023-09-10T01:33:52.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:51 smithi044 ceph-mon[98676]: pgmap v1733: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 513 op/s 2023-09-10T01:33:52.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:51 smithi099 ceph-mon[102692]: pgmap v1733: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 513 op/s 2023-09-10T01:33:54.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:53 smithi131 ceph-mon[103713]: pgmap v1734: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 737 op/s 2023-09-10T01:33:54.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:53 smithi044 ceph-mon[98676]: pgmap v1734: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 737 op/s 2023-09-10T01:33:54.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:53 smithi099 ceph-mon[102692]: pgmap v1734: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 737 op/s 2023-09-10T01:33:56.186 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 770.0 s, 340.0 tps, lat 295.945 ms stddev 173.988 2023-09-10T01:33:56.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:55 smithi131 ceph-mon[103713]: pgmap v1735: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 679 op/s 2023-09-10T01:33:56.314 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:55 smithi044 ceph-mon[98676]: pgmap v1735: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 679 op/s 2023-09-10T01:33:56.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:55 smithi099 ceph-mon[102692]: pgmap v1735: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 679 op/s 2023-09-10T01:33:58.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:57 smithi131 ceph-mon[103713]: pgmap v1736: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.7 MiB/s wr, 931 op/s 2023-09-10T01:33:58.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:57 smithi044 ceph-mon[98676]: pgmap v1736: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.7 MiB/s wr, 931 op/s 2023-09-10T01:33:58.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:57 smithi099 ceph-mon[102692]: pgmap v1736: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.7 MiB/s wr, 931 op/s 2023-09-10T01:34:00.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:33:59 smithi131 ceph-mon[103713]: pgmap v1737: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.4 MiB/s wr, 809 op/s 2023-09-10T01:34:00.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:33:59 smithi044 ceph-mon[98676]: pgmap v1737: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.4 MiB/s wr, 809 op/s 2023-09-10T01:34:00.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:33:59 smithi099 ceph-mon[102692]: pgmap v1737: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.4 MiB/s wr, 809 op/s 2023-09-10T01:34:01.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 775.0 s, 240.9 tps, lat 351.292 ms stddev 259.406 2023-09-10T01:34:02.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:01 smithi131 ceph-mon[103713]: pgmap v1738: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 733 op/s 2023-09-10T01:34:02.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:01 smithi044 ceph-mon[98676]: pgmap v1738: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 733 op/s 2023-09-10T01:34:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.318+0000 7fdc3072c700 1 -- 172.21.15.44:0/2668941405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc280ff4b0 msgr2=0x7fdc280ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.318+0000 7fdc3072c700 1 --2- 172.21.15.44:0/2668941405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc280ff4b0 0x7fdc280ff930 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800ba80 tx=0x7fdc1800bd90 comp rx=0 tx=0).stop 2023-09-10T01:34:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.325+0000 7fdc3072c700 1 -- 172.21.15.44:0/2668941405 shutdown_connections 2023-09-10T01:34:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.325+0000 7fdc3072c700 1 --2- 172.21.15.44:0/2668941405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc280ffe70 0x7fdc2810c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.325+0000 7fdc3072c700 1 --2- 172.21.15.44:0/2668941405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc28104410 0x7fdc281047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.325+0000 7fdc3072c700 1 --2- 172.21.15.44:0/2668941405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc280ff4b0 0x7fdc280ff930 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.325+0000 7fdc3072c700 1 -- 172.21.15.44:0/2668941405 >> 172.21.15.44:0/2668941405 conn(0x7fdc280765b0 msgr2=0x7fdc280769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.325+0000 7fdc3072c700 1 -- 172.21.15.44:0/2668941405 shutdown_connections 2023-09-10T01:34:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.325+0000 7fdc3072c700 1 -- 172.21.15.44:0/2668941405 wait complete. 2023-09-10T01:34:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.326+0000 7fdc3072c700 1 Processor -- start 2023-09-10T01:34:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.326+0000 7fdc3072c700 1 -- start start 2023-09-10T01:34:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.327+0000 7fdc3072c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc280ff4b0 0x7fdc28072020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.327+0000 7fdc3072c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc280ffe70 0x7fdc2806ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.327+0000 7fdc3072c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc28104410 0x7fdc2806f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.327+0000 7fdc3072c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2806fa40 con 0x7fdc280ff4b0 2023-09-10T01:34:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.327+0000 7fdc3072c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2806fbb0 con 0x7fdc28104410 2023-09-10T01:34:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.327+0000 7fdc3072c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc281ad8e0 con 0x7fdc280ffe70 2023-09-10T01:34:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.327+0000 7fdc2dcc7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc280ffe70 0x7fdc2806ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.327+0000 7fdc2dcc7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc280ffe70 0x7fdc2806ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39582/0 (socket says 172.21.15.44:39582) 2023-09-10T01:34:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.327+0000 7fdc2dcc7700 1 -- 172.21.15.44:0/2099967788 learned_addr learned my addr 172.21.15.44:0/2099967788 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.328+0000 7fdc2dcc7700 1 -- 172.21.15.44:0/2099967788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc28104410 msgr2=0x7fdc2806f360 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-10T01:34:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.328+0000 7fdc2dcc7700 1 --2- 172.21.15.44:0/2099967788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc28104410 0x7fdc2806f360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.328+0000 7fdc2dcc7700 1 -- 172.21.15.44:0/2099967788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc280ff4b0 msgr2=0x7fdc28072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.328+0000 7fdc2dcc7700 1 --2- 172.21.15.44:0/2099967788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc280ff4b0 0x7fdc28072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.328+0000 7fdc2dcc7700 1 -- 172.21.15.44:0/2099967788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc24009520 con 0x7fdc280ffe70 2023-09-10T01:34:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.328+0000 7fdc2dcc7700 1 --2- 172.21.15.44:0/2099967788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc280ffe70 0x7fdc2806ee20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800daf0 tx=0x7fdc1800dbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.328+0000 7fdc1f7fe700 1 -- 172.21.15.44:0/2099967788 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18010040 con 0x7fdc280ffe70 2023-09-10T01:34:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.328+0000 7fdc1f7fe700 1 -- 172.21.15.44:0/2099967788 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc180056d0 con 0x7fdc280ffe70 2023-09-10T01:34:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.328+0000 7fdc1f7fe700 1 -- 172.21.15.44:0/2099967788 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1801f440 con 0x7fdc280ffe70 2023-09-10T01:34:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.329+0000 7fdc3072c700 1 -- 172.21.15.44:0/2099967788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1800b730 con 0x7fdc280ffe70 2023-09-10T01:34:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.329+0000 7fdc3072c700 1 -- 172.21.15.44:0/2099967788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc281adde0 con 0x7fdc280ffe70 2023-09-10T01:34:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.332+0000 7fdc1f7fe700 1 -- 172.21.15.44:0/2099967788 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fdc1801f5a0 con 0x7fdc280ffe70 2023-09-10T01:34:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.332+0000 7fdc1f7fe700 1 --2- 172.21.15.44:0/2099967788 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdc140a0830 0x7fdc140a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.333+0000 7fdc1f7fe700 1 -- 172.21.15.44:0/2099967788 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 9496+0+0 (secure 0 0 0) 0x7fdc180ce860 con 0x7fdc280ffe70 2023-09-10T01:34:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.333+0000 7fdc2e4c8700 1 --2- 172.21.15.44:0/2099967788 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdc140a0830 0x7fdc140a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.335+0000 7fdc2e4c8700 1 --2- 172.21.15.44:0/2099967788 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdc140a0830 0x7fdc140a2cf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009d10 tx=0x7fdc240097b0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.335+0000 7fdc3c8f2b80 1 Processor -- start 2023-09-10T01:34:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.335+0000 7fdc3c8f2b80 1 -- start start 2023-09-10T01:34:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.335+0000 7fdc3c8f2b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f3931a0 0x55fe6f393580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.335+0000 7fdc3c8f2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f38d300 0x55fe6f38d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.335+0000 7fdc3c8f2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38dcc0 0x55fe6f399f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.335+0000 7fdc3c8f2b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6f2e66d0 con 0x55fe6f3931a0 2023-09-10T01:34:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.335+0000 7fdc3c8f2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6f2e3ff0 con 0x55fe6f38dcc0 2023-09-10T01:34:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.335+0000 7fdc3c8f2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6f2df8e0 con 0x55fe6f38d300 2023-09-10T01:34:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.335+0000 7fdc2dcc7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f38d300 0x55fe6f38d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.336+0000 7fdc2dcc7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f38d300 0x55fe6f38d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39592/0 (socket says 172.21.15.44:39592) 2023-09-10T01:34:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.336+0000 7fdc2dcc7700 1 -- 172.21.15.44:0/2174317123 learned_addr learned my addr 172.21.15.44:0/2174317123 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.336+0000 7fdc2ecc9700 1 --2- 172.21.15.44:0/2174317123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38dcc0 0x55fe6f399f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.336+0000 7fdc2dcc7700 1 -- 172.21.15.44:0/2174317123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38dcc0 msgr2=0x55fe6f399f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.336+0000 7fdc2dcc7700 1 --2- 172.21.15.44:0/2174317123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38dcc0 0x55fe6f399f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.336+0000 7fdc2dcc7700 1 -- 172.21.15.44:0/2174317123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f3931a0 msgr2=0x55fe6f393580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.336+0000 7fdc2dcc7700 1 --2- 172.21.15.44:0/2174317123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f3931a0 0x55fe6f393580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.336+0000 7fdc2dcc7700 1 -- 172.21.15.44:0/2174317123 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe6f10bd80 con 0x55fe6f38d300 2023-09-10T01:34:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.336+0000 7fdc2dcc7700 1 --2- 172.21.15.44:0/2174317123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f38d300 0x55fe6f38d780 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdc180cbc40 tx=0x7fdc1809b5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15a0aa23f781f23e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.336+0000 7fdc1cff9700 1 -- 172.21.15.44:0/2174317123 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1809b9d0 con 0x55fe6f38d300 2023-09-10T01:34:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.336+0000 7fdc1cff9700 1 -- 172.21.15.44:0/2174317123 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc1809f800 con 0x55fe6f38d300 2023-09-10T01:34:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.336+0000 7fdc1cff9700 1 -- 172.21.15.44:0/2174317123 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc180ce450 con 0x55fe6f38d300 2023-09-10T01:34:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.337+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/2174317123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f38d300 msgr2=0x55fe6f38d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.337+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/2174317123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f38d300 0x55fe6f38d780 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdc180cbc40 tx=0x7fdc1809b5e0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.338+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/2174317123 shutdown_connections 2023-09-10T01:34:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.338+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/2174317123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38dcc0 0x55fe6f399f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.338+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/2174317123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f3931a0 0x55fe6f393580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.338+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/2174317123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f38d300 0x55fe6f38d780 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.338+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/2174317123 >> 172.21.15.44:0/2174317123 conn(0x55fe6f302030 msgr2=0x55fe6f388560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.338+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/2174317123 shutdown_connections 2023-09-10T01:34:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.338+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/2174317123 wait complete. 2023-09-10T01:34:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.338+0000 7fdc3c8f2b80 1 -- start start 2023-09-10T01:34:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.338+0000 7fdc3c8f2b80 1 Processor -- start 2023-09-10T01:34:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.339+0000 7fdc3c8f2b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38d300 0x55fe6f3a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.339+0000 7fdc3c8f2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.339+0000 7fdc3c8f2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f3931a0 0x55fe6f39f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.339+0000 7fdc3c8f2b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6f27db10 con 0x55fe6f38d300 2023-09-10T01:34:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.339+0000 7fdc3c8f2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6f27d810 con 0x55fe6f3931a0 2023-09-10T01:34:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.339+0000 7fdc3c8f2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6f1618c0 con 0x55fe6f38dcc0 2023-09-10T01:34:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.339+0000 7fdc3c8f2b80 10 client.0 _ll_register_callbacks cb 0x55fe6f1c7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:34:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.340+0000 7fdc2dcc7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.340+0000 7fdc2dcc7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3a58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39598/0 (socket says 172.21.15.44:39598) 2023-09-10T01:34:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.340+0000 7fdc2dcc7700 1 -- 172.21.15.44:0/3650976006 learned_addr learned my addr 172.21.15.44:0/3650976006 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.340+0000 7fdc2e4c8700 1 --2- 172.21.15.44:0/3650976006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38d300 0x55fe6f3a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.340+0000 7fdc2ecc9700 1 --2- 172.21.15.44:0/3650976006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f3931a0 0x55fe6f39f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.340+0000 7fdc2dcc7700 1 -- 172.21.15.44:0/3650976006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f3931a0 msgr2=0x55fe6f39f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.340+0000 7fdc2dcc7700 1 --2- 172.21.15.44:0/3650976006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f3931a0 0x55fe6f39f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.340+0000 7fdc2dcc7700 1 -- 172.21.15.44:0/3650976006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38d300 msgr2=0x55fe6f3a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.340+0000 7fdc2dcc7700 1 --2- 172.21.15.44:0/3650976006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38d300 0x55fe6f3a53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.340+0000 7fdc2dcc7700 1 -- 172.21.15.44:0/3650976006 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc281adde0 con 0x55fe6f38dcc0 2023-09-10T01:34:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.340+0000 7fdc2ecc9700 1 --2- 172.21.15.44:0/3650976006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f3931a0 0x55fe6f39f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:34:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.341+0000 7fdc2dcc7700 1 --2- 172.21.15.44:0/3650976006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3a58f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdc180a3480 tx=0x7fdc1809c240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5ff4f35c1584f7c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.341+0000 7fdc11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:34:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.341+0000 7fdc11ffb700 1 -- 172.21.15.44:0/3650976006 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc180a39b0 con 0x55fe6f38dcc0 2023-09-10T01:34:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.341+0000 7fdc11ffb700 1 -- 172.21.15.44:0/3650976006 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(204..204 src has 1..204) v4 ==== 9496+0+0 (secure 0 0 0) 0x7fdc18026e30 con 0x55fe6f38dcc0 2023-09-10T01:34:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.341+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/3650976006 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe6f237490 con 0x55fe6f38dcc0 2023-09-10T01:34:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.341+0000 7fdc11ffb700 1 -- 172.21.15.44:0/3650976006 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=192) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc180ce660 con 0x55fe6f38dcc0 2023-09-10T01:34:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.344+0000 7fdc3c8f2b80 10 client.30298 fetch_fsmap learned FSMap version 192 2023-09-10T01:34:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.344+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/3650976006 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=192,monmap=4+}) v3 -- 0x55fe6f0f4c00 con 0x55fe6f38dcc0 2023-09-10T01:34:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.346+0000 7fdc11ffb700 1 -- 172.21.15.44:0/3650976006 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc18080080 con 0x55fe6f38dcc0 2023-09-10T01:34:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.346+0000 7fdc11ffb700 1 -- 172.21.15.44:0/3650976006 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 192) v1 ==== 7359+0+0 (secure 0 0 0) 0x7fdc180a3520 con 0x55fe6f38dcc0 2023-09-10T01:34:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.346+0000 7fdc3c8f2b80 10 client.30298 fetch_fsmap finished waiting for FSMap version 192 2023-09-10T01:34:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.346+0000 7fdc3c8f2b80 10 client.30298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:34:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.346+0000 7fdc3c8f2b80 20 client.30298 populate_metadata read hostname 'smithi044' 2023-09-10T01:34:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.346+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/3650976006 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe6f3af380 0x55fe6f3b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.346+0000 7fdc3c8f2b80 4 client.30298 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:34:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.346+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/3650976006 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe6f2f4840 con 0x55fe6f3af380 2023-09-10T01:34:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.346+0000 7fdc2e4c8700 1 --2- 172.21.15.44:0/3650976006 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe6f3af380 0x55fe6f3b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.347+0000 7fdc11ffb700 10 client.30298 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.347+0000 7fdc2e4c8700 1 --2- 172.21.15.44:0/3650976006 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe6f3af380 0x55fe6f3b17a0 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68f11e2a3823ea07 server_cookie=7521813480cef711 in_seq=0 out_seq=0 2023-09-10T01:34:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.347+0000 7fdc11ffb700 1 -- 172.21.15.44:0/3650976006 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fe6f2f4840 con 0x55fe6f3af380 2023-09-10T01:34:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.347+0000 7fdc11ffb700 10 client.30298 handle_command_reply: tid=0 2023-09-10T01:34:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.347+0000 7fdc3c8f2b80 1 client.30298 shutdown 2023-09-10T01:34:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.347+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/3650976006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f38dcc0 msgr2=0x55fe6f3a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.348+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/3650976006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3a58f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdc180a3480 tx=0x7fdc1809c240 comp rx=0 tx=0).stop 2023-09-10T01:34:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.348+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/3650976006 shutdown_connections 2023-09-10T01:34:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.348+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/3650976006 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe6f3af380 0x55fe6f3b17a0 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.348+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/3650976006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f3931a0 0x55fe6f39f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.348+0000 7fdc11ffb700 0 client.30298 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.348+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/3650976006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38d300 0x55fe6f3a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.348+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/3650976006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3a58f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.348+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/3650976006 >> 172.21.15.44:0/3650976006 conn(0x55fe6f302030 msgr2=0x55fe6f3877a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.348+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/3650976006 shutdown_connections 2023-09-10T01:34:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.348+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/3650976006 wait complete. 2023-09-10T01:34:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.348+0000 7fdc3c8f2b80 20 client.30298 trim_cache size 0 max 16384 2023-09-10T01:34:02.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:01 smithi099 ceph-mon[102692]: pgmap v1738: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 733 op/s 2023-09-10T01:34:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.372+0000 7fdc3c8f2b80 1 Processor -- start 2023-09-10T01:34:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc3c8f2b80 1 -- start start 2023-09-10T01:34:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc3c8f2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38d300 0x55fe6f38cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc3c8f2b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc3c8f2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f3931a0 0x55fe6f3a5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc3c8f2b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6f3828a0 con 0x55fe6f38dcc0 2023-09-10T01:34:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc3c8f2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6f38e0a0 con 0x55fe6f38d300 2023-09-10T01:34:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc3c8f2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6f37ff30 con 0x55fe6f3931a0 2023-09-10T01:34:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc2dcc7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc2dcc7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3a4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60954/0 (socket says 172.21.15.44:60954) 2023-09-10T01:34:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc2e4c8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38d300 0x55fe6f38cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc2dcc7700 1 -- 172.21.15.44:0/2160123163 learned_addr learned my addr 172.21.15.44:0/2160123163 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc2ecc9700 1 --2- 172.21.15.44:0/2160123163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f3931a0 0x55fe6f3a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc2e4c8700 1 -- 172.21.15.44:0/2160123163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f3931a0 msgr2=0x55fe6f3a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc2e4c8700 1 --2- 172.21.15.44:0/2160123163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f3931a0 0x55fe6f3a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc2e4c8700 1 -- 172.21.15.44:0/2160123163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38dcc0 msgr2=0x55fe6f3a4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc2e4c8700 1 --2- 172.21.15.44:0/2160123163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3a4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc2e4c8700 1 -- 172.21.15.44:0/2160123163 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe6f3a0560 con 0x55fe6f38d300 2023-09-10T01:34:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.373+0000 7fdc2dcc7700 1 --2- 172.21.15.44:0/2160123163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:34:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.374+0000 7fdc2e4c8700 1 --2- 172.21.15.44:0/2160123163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38d300 0x55fe6f38cb80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55fe6f3b2040 tx=0x7fdc24011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d157c877eef8c1e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.374+0000 7fdc1d7fa700 1 -- 172.21.15.44:0/2160123163 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24025520 con 0x55fe6f38d300 2023-09-10T01:34:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.374+0000 7fdc1d7fa700 1 -- 172.21.15.44:0/2160123163 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc240044b0 con 0x55fe6f38d300 2023-09-10T01:34:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.374+0000 7fdc1d7fa700 1 -- 172.21.15.44:0/2160123163 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24004a90 con 0x55fe6f38d300 2023-09-10T01:34:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.378+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/2160123163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38d300 msgr2=0x55fe6f38cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.378+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/2160123163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38d300 0x55fe6f38cb80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55fe6f3b2040 tx=0x7fdc24011000 comp rx=0 tx=0).stop 2023-09-10T01:34:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.379+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/2160123163 shutdown_connections 2023-09-10T01:34:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.379+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/2160123163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f3931a0 0x55fe6f3a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.379+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/2160123163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38d300 0x55fe6f38cb80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.379+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/2160123163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.379+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/2160123163 >> 172.21.15.44:0/2160123163 conn(0x55fe6f302030 msgr2=0x55fe6f392490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.381+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/2160123163 shutdown_connections 2023-09-10T01:34:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.386+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/2160123163 wait complete. 2023-09-10T01:34:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.387+0000 7fdc3c8f2b80 1 -- start start 2023-09-10T01:34:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.387+0000 7fdc3c8f2b80 1 Processor -- start 2023-09-10T01:34:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.387+0000 7fdc3c8f2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38d300 0x55fe6f3fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.387+0000 7fdc3c8f2b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3fce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.387+0000 7fdc3c8f2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f3931a0 0x55fe6f3f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.387+0000 7fdc3c8f2b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6f3f7260 con 0x55fe6f38dcc0 2023-09-10T01:34:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.387+0000 7fdc3c8f2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6f3f73d0 con 0x55fe6f38d300 2023-09-10T01:34:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.387+0000 7fdc3c8f2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6f3f7590 con 0x55fe6f3931a0 2023-09-10T01:34:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.387+0000 7fdc3c8f2b80 10 client.0 _ll_register_callbacks cb 0x55fe6f1c7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:34:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.387+0000 7fdc2e4c8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38d300 0x55fe6f3fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.387+0000 7fdc2e4c8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38d300 0x55fe6f3fc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34466/0 (socket says 172.21.15.44:34466) 2023-09-10T01:34:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.387+0000 7fdc2e4c8700 1 -- 172.21.15.44:0/3056395799 learned_addr learned my addr 172.21.15.44:0/3056395799 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.388+0000 7fdc2dcc7700 1 --2- 172.21.15.44:0/3056395799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.388+0000 7fdc2e4c8700 1 -- 172.21.15.44:0/3056395799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f3931a0 msgr2=0x55fe6f3f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.388+0000 7fdc2e4c8700 1 --2- 172.21.15.44:0/3056395799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f3931a0 0x55fe6f3f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.388+0000 7fdc2e4c8700 1 -- 172.21.15.44:0/3056395799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38dcc0 msgr2=0x55fe6f3fce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.388+0000 7fdc2e4c8700 1 --2- 172.21.15.44:0/3056395799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3fce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.388+0000 7fdc2e4c8700 1 -- 172.21.15.44:0/3056395799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc281adde0 con 0x55fe6f38d300 2023-09-10T01:34:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.388+0000 7fdc2dcc7700 1 --2- 172.21.15.44:0/3056395799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:34:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.388+0000 7fdc2e4c8700 1 --2- 172.21.15.44:0/3056395799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38d300 0x55fe6f3fc8e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400f010 tx=0x7fdc24005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25491d7f313b2d41 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.388+0000 7fdc11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:34:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.389+0000 7fdc11ffb700 1 -- 172.21.15.44:0/3056395799 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24025a80 con 0x55fe6f38d300 2023-09-10T01:34:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.389+0000 7fdc11ffb700 1 -- 172.21.15.44:0/3056395799 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(204..204 src has 1..204) v4 ==== 9496+0+0 (secure 0 0 0) 0x7fdc24025e40 con 0x55fe6f38d300 2023-09-10T01:34:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.389+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/3056395799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc1809f800 con 0x55fe6f38d300 2023-09-10T01:34:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.389+0000 7fdc11ffb700 1 -- 172.21.15.44:0/3056395799 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=192) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc24025be0 con 0x55fe6f38d300 2023-09-10T01:34:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.389+0000 7fdc3c8f2b80 10 client.30263 fetch_fsmap learned FSMap version 192 2023-09-10T01:34:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.389+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/3056395799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=192,monmap=4+}) v3 -- 0x55fe6f1321f0 con 0x55fe6f38d300 2023-09-10T01:34:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.390+0000 7fdc11ffb700 1 -- 172.21.15.44:0/3056395799 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc1809f800 con 0x55fe6f38d300 2023-09-10T01:34:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.390+0000 7fdc11ffb700 1 -- 172.21.15.44:0/3056395799 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 192) v1 ==== 7359+0+0 (secure 0 0 0) 0x7fdc240107d0 con 0x55fe6f38d300 2023-09-10T01:34:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.390+0000 7fdc3c8f2b80 10 client.30263 fetch_fsmap finished waiting for FSMap version 192 2023-09-10T01:34:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.390+0000 7fdc3c8f2b80 10 client.30263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:34:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.390+0000 7fdc3c8f2b80 20 client.30263 populate_metadata read hostname 'smithi044' 2023-09-10T01:34:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.390+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/3056395799 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe6f404810 0x55fe6f406c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.390+0000 7fdc3c8f2b80 4 client.30263 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:34:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.390+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/3056395799 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fe6f2d88c0 con 0x55fe6f404810 2023-09-10T01:34:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.397+0000 7fdc2dcc7700 1 --2- 172.21.15.44:0/3056395799 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe6f404810 0x55fe6f406c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.398+0000 7fdc2dcc7700 1 --2- 172.21.15.44:0/3056395799 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe6f404810 0x55fe6f406c90 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b422e1aa6b9240c0 server_cookie=a309b096162c81c9 in_seq=0 out_seq=0 2023-09-10T01:34:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.398+0000 7fdc11ffb700 10 client.30263 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.421+0000 7fdc11ffb700 1 -- 172.21.15.44:0/3056395799 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fe6f2d88c0 con 0x55fe6f404810 2023-09-10T01:34:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.421+0000 7fdc11ffb700 10 client.30263 handle_command_reply: tid=0 2023-09-10T01:34:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.421+0000 7fdc3c8f2b80 1 client.30263 shutdown 2023-09-10T01:34:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.421+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/3056395799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38d300 msgr2=0x55fe6f3fc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.421+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/3056395799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38d300 0x55fe6f3fc8e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400f010 tx=0x7fdc24005e50 comp rx=0 tx=0).stop 2023-09-10T01:34:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.421+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/3056395799 shutdown_connections 2023-09-10T01:34:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.421+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/3056395799 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55fe6f404810 0x55fe6f406c90 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.421+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/3056395799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe6f3931a0 0x55fe6f3f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.421+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/3056395799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe6f38d300 0x55fe6f3fc8e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.421+0000 7fdc11ffb700 0 client.30263 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.421+0000 7fdc3c8f2b80 1 --2- 172.21.15.44:0/3056395799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe6f38dcc0 0x55fe6f3fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.421+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/3056395799 >> 172.21.15.44:0/3056395799 conn(0x55fe6f302030 msgr2=0x55fe6f38cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.421+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/3056395799 shutdown_connections 2023-09-10T01:34:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.421+0000 7fdc3c8f2b80 1 -- 172.21.15.44:0/3056395799 wait complete. 2023-09-10T01:34:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.421+0000 7fdc3c8f2b80 20 client.30263 trim_cache size 0 max 16384 2023-09-10T01:34:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:34:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:34:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:34:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:34:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.422+0000 7fdc3072c700 1 -- 172.21.15.44:0/2099967788 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdc140a0830 msgr2=0x7fdc140a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.422+0000 7fdc3072c700 1 --2- 172.21.15.44:0/2099967788 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdc140a0830 0x7fdc140a2cf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009d10 tx=0x7fdc240097b0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.422+0000 7fdc3072c700 1 -- 172.21.15.44:0/2099967788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc280ffe70 msgr2=0x7fdc2806ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.422+0000 7fdc3072c700 1 --2- 172.21.15.44:0/2099967788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc280ffe70 0x7fdc2806ee20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800daf0 tx=0x7fdc1800dbd0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.422+0000 7fdc3072c700 1 -- 172.21.15.44:0/2099967788 shutdown_connections 2023-09-10T01:34:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.422+0000 7fdc3072c700 1 --2- 172.21.15.44:0/2099967788 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdc140a0830 0x7fdc140a2cf0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.422+0000 7fdc3072c700 1 --2- 172.21.15.44:0/2099967788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc28104410 0x7fdc2806f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.422+0000 7fdc3072c700 1 --2- 172.21.15.44:0/2099967788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc280ff4b0 0x7fdc28072020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.422+0000 7fdc3072c700 1 --2- 172.21.15.44:0/2099967788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc280ffe70 0x7fdc2806ee20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.422+0000 7fdc3072c700 1 -- 172.21.15.44:0/2099967788 >> 172.21.15.44:0/2099967788 conn(0x7fdc280765b0 msgr2=0x7fdc280fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.422+0000 7fdc3072c700 1 -- 172.21.15.44:0/2099967788 shutdown_connections 2023-09-10T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:02.422+0000 7fdc3072c700 1 -- 172.21.15.44:0/2099967788 wait complete. 2023-09-10T01:34:04.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:03 smithi131 ceph-mon[103713]: pgmap v1739: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.2 MiB/s wr, 1.00k op/s 2023-09-10T01:34:04.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:03 smithi044 ceph-mon[98676]: pgmap v1739: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.2 MiB/s wr, 1.00k op/s 2023-09-10T01:34:04.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:03 smithi099 ceph-mon[102692]: pgmap v1739: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.2 MiB/s wr, 1.00k op/s 2023-09-10T01:34:05.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:05 smithi131 ceph-mon[103713]: osdmap e205: 12 total, 12 up, 12 in 2023-09-10T01:34:05.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:05 smithi131 ceph-mon[103713]: pgmap v1741: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.9 MiB/s wr, 937 op/s 2023-09-10T01:34:05.314 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:05 smithi044 ceph-mon[98676]: osdmap e205: 12 total, 12 up, 12 in 2023-09-10T01:34:05.315 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:05 smithi044 ceph-mon[98676]: pgmap v1741: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.9 MiB/s wr, 937 op/s 2023-09-10T01:34:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:05 smithi099 ceph-mon[102692]: osdmap e205: 12 total, 12 up, 12 in 2023-09-10T01:34:05.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:05 smithi099 ceph-mon[102692]: pgmap v1741: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.9 MiB/s wr, 937 op/s 2023-09-10T01:34:06.187 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 780.0 s, 185.2 tps, lat 506.445 ms stddev 521.557 2023-09-10T01:34:08.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:07 smithi131 ceph-mon[103713]: pgmap v1742: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.3 MiB/s wr, 740 op/s 2023-09-10T01:34:08.058 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:07 smithi044 ceph-mon[98676]: pgmap v1742: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.3 MiB/s wr, 740 op/s 2023-09-10T01:34:08.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:07 smithi099 ceph-mon[102692]: pgmap v1742: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.3 MiB/s wr, 740 op/s 2023-09-10T01:34:10.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:09 smithi131 ceph-mon[103713]: pgmap v1743: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 638 op/s 2023-09-10T01:34:10.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:09 smithi044 ceph-mon[98676]: pgmap v1743: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 638 op/s 2023-09-10T01:34:10.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:09 smithi099 ceph-mon[102692]: pgmap v1743: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 638 op/s 2023-09-10T01:34:11.188 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 785.0 s, 72.4 tps, lat 1487.455 ms stddev 1317.265 2023-09-10T01:34:12.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:11 smithi131 ceph-mon[103713]: pgmap v1744: 129 pgs: 19 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 63 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 634 op/s 2023-09-10T01:34:12.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:11 smithi044 ceph-mon[98676]: pgmap v1744: 129 pgs: 19 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 63 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 634 op/s 2023-09-10T01:34:12.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:11 smithi099 ceph-mon[102692]: pgmap v1744: 129 pgs: 19 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 63 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 634 op/s 2023-09-10T01:34:14.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:13 smithi131 ceph-mon[103713]: pgmap v1745: 129 pgs: 9 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 107 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 371 op/s 2023-09-10T01:34:14.311 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:13 smithi044 ceph-mon[98676]: pgmap v1745: 129 pgs: 9 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 107 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 371 op/s 2023-09-10T01:34:14.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:13 smithi099 ceph-mon[102692]: pgmap v1745: 129 pgs: 9 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 107 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 371 op/s 2023-09-10T01:34:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.630+0000 7fe77beea700 1 -- 172.21.15.44:0/2498643515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe774119de0 msgr2=0x7fe77411c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.630+0000 7fe77beea700 1 --2- 172.21.15.44:0/2498643515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe774119de0 0x7fe77411c1d0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fe76401a9a0 tx=0x7fe76401acb0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.630+0000 7fe77beea700 1 -- 172.21.15.44:0/2498643515 shutdown_connections 2023-09-10T01:34:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.630+0000 7fe77beea700 1 --2- 172.21.15.44:0/2498643515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe774119de0 0x7fe77411c1d0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.630+0000 7fe77beea700 1 --2- 172.21.15.44:0/2498643515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe77410ed50 0x7fe774107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.630+0000 7fe77beea700 1 --2- 172.21.15.44:0/2498643515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe77410f1c0 0x7fe774107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.630+0000 7fe77beea700 1 -- 172.21.15.44:0/2498643515 >> 172.21.15.44:0/2498643515 conn(0x7fe77406c6c0 msgr2=0x7fe77406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.637+0000 7fe77beea700 1 -- 172.21.15.44:0/2498643515 shutdown_connections 2023-09-10T01:34:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.641+0000 7fe77beea700 1 -- 172.21.15.44:0/2498643515 wait complete. 2023-09-10T01:34:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.641+0000 7fe77beea700 1 Processor -- start 2023-09-10T01:34:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.642+0000 7fe77beea700 1 -- start start 2023-09-10T01:34:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.642+0000 7fe77beea700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe77410ed50 0x7fe774115040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.642+0000 7fe77beea700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe77410f1c0 0x7fe774115580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.642+0000 7fe77beea700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe774119de0 0x7fe7741c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.642+0000 7fe77beea700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7741c5110 con 0x7fe774119de0 2023-09-10T01:34:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.642+0000 7fe77beea700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7741c5280 con 0x7fe77410f1c0 2023-09-10T01:34:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.642+0000 7fe77beea700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7741c5440 con 0x7fe77410ed50 2023-09-10T01:34:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.643+0000 7fe77a487700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe774119de0 0x7fe7741c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.643+0000 7fe77a487700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe774119de0 0x7fe7741c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60972/0 (socket says 172.21.15.44:60972) 2023-09-10T01:34:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.643+0000 7fe77a487700 1 -- 172.21.15.44:0/3583019359 learned_addr learned my addr 172.21.15.44:0/3583019359 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.645+0000 7fe779485700 1 --2- 172.21.15.44:0/3583019359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe77410f1c0 0x7fe774115580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.646+0000 7fe77a487700 1 -- 172.21.15.44:0/3583019359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe77410ed50 msgr2=0x7fe774115040 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-10T01:34:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.646+0000 7fe77a487700 1 --2- 172.21.15.44:0/3583019359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe77410ed50 0x7fe774115040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.646+0000 7fe77a487700 1 -- 172.21.15.44:0/3583019359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe77410f1c0 msgr2=0x7fe774115580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.646+0000 7fe77a487700 1 --2- 172.21.15.44:0/3583019359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe77410f1c0 0x7fe774115580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.646+0000 7fe77a487700 1 -- 172.21.15.44:0/3583019359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76401a600 con 0x7fe774119de0 2023-09-10T01:34:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.646+0000 7fe77a487700 1 --2- 172.21.15.44:0/3583019359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe774119de0 0x7fe7741c4af0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fe7640053f0 tx=0x7fe7640054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.648+0000 7fe76affd700 1 -- 172.21.15.44:0/3583019359 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764005720 con 0x7fe774119de0 2023-09-10T01:34:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.648+0000 7fe76affd700 1 -- 172.21.15.44:0/3583019359 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe76400f480 con 0x7fe774119de0 2023-09-10T01:34:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.648+0000 7fe76affd700 1 -- 172.21.15.44:0/3583019359 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76401f650 con 0x7fe774119de0 2023-09-10T01:34:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.648+0000 7fe77beea700 1 -- 172.21.15.44:0/3583019359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7741c56c0 con 0x7fe774119de0 2023-09-10T01:34:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.648+0000 7fe77beea700 1 -- 172.21.15.44:0/3583019359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7741c5940 con 0x7fe774119de0 2023-09-10T01:34:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.656+0000 7fe7880b0b80 1 Processor -- start 2023-09-10T01:34:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.657+0000 7fe7880b0b80 1 -- start start 2023-09-10T01:34:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.657+0000 7fe7880b0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e293b1a0 0x55c2e293b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.657+0000 7fe7880b0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935300 0x55c2e2935780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.657+0000 7fe7880b0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e2935cc0 0x55c2e2941f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.657+0000 7fe7880b0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e288e6d0 con 0x55c2e293b1a0 2023-09-10T01:34:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.657+0000 7fe7880b0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e288bff0 con 0x55c2e2935300 2023-09-10T01:34:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.657+0000 7fe7880b0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e28878e0 con 0x55c2e2935cc0 2023-09-10T01:34:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.658+0000 7fe779485700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e2935cc0 0x55c2e2941f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.658+0000 7fe779485700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e2935cc0 0x55c2e2941f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39630/0 (socket says 172.21.15.44:39630) 2023-09-10T01:34:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.658+0000 7fe779485700 1 -- 172.21.15.44:0/3461205082 learned_addr learned my addr 172.21.15.44:0/3461205082 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.658+0000 7fe779485700 1 --2- 172.21.15.44:0/3461205082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e293b1a0 0x55c2e293b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.658+0000 7fe779485700 1 -- 172.21.15.44:0/3461205082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935300 msgr2=0x55c2e2935780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:34:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.658+0000 7fe779485700 1 --2- 172.21.15.44:0/3461205082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935300 0x55c2e2935780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.658+0000 7fe779485700 1 -- 172.21.15.44:0/3461205082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e293b1a0 msgr2=0x55c2e293b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.658+0000 7fe779485700 1 --2- 172.21.15.44:0/3461205082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e293b1a0 0x55c2e293b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.658+0000 7fe779485700 1 -- 172.21.15.44:0/3461205082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2e26b3d80 con 0x55c2e2935cc0 2023-09-10T01:34:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.659+0000 7fe779485700 1 --2- 172.21.15.44:0/3461205082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e2935cc0 0x55c2e2941f30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe770016550 tx=0x7fe77000a4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51c5ba6c518ec357 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.662+0000 7fe76affd700 1 -- 172.21.15.44:0/3583019359 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fe764003b90 con 0x7fe774119de0 2023-09-10T01:34:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.662+0000 7fe76affd700 1 --2- 172.21.15.44:0/3583019359 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe7600a0980 0x7fe7600a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.663+0000 7fe76affd700 1 -- 172.21.15.44:0/3583019359 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fe7640cd9b0 con 0x7fe774119de0 2023-09-10T01:34:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.663+0000 7fe779c86700 1 --2- 172.21.15.44:0/3583019359 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe7600a0980 0x7fe7600a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.664+0000 7fe779c86700 1 --2- 172.21.15.44:0/3583019359 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe7600a0980 0x7fe7600a2e40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55c2e2930060 tx=0x7fe76c009250 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.668+0000 7fe75ffff700 1 -- 172.21.15.44:0/3461205082 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770016c30 con 0x55c2e2935cc0 2023-09-10T01:34:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.668+0000 7fe75ffff700 1 -- 172.21.15.44:0/3461205082 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe77000c030 con 0x55c2e2935cc0 2023-09-10T01:34:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.668+0000 7fe75ffff700 1 -- 172.21.15.44:0/3461205082 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700143f0 con 0x55c2e2935cc0 2023-09-10T01:34:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.669+0000 7fe7880b0b80 1 -- 172.21.15.44:0/3461205082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e2935cc0 msgr2=0x55c2e2941f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.669+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/3461205082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e2935cc0 0x55c2e2941f30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe770016550 tx=0x7fe77000a4d0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.669+0000 7fe7880b0b80 1 -- 172.21.15.44:0/3461205082 shutdown_connections 2023-09-10T01:34:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.669+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/3461205082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e2935cc0 0x55c2e2941f30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.669+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/3461205082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e293b1a0 0x55c2e293b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.669+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/3461205082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935300 0x55c2e2935780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.669+0000 7fe7880b0b80 1 -- 172.21.15.44:0/3461205082 >> 172.21.15.44:0/3461205082 conn(0x55c2e28aa030 msgr2=0x55c2e2930560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.672+0000 7fe7880b0b80 1 -- 172.21.15.44:0/3461205082 shutdown_connections 2023-09-10T01:34:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.672+0000 7fe7880b0b80 1 -- 172.21.15.44:0/3461205082 wait complete. 2023-09-10T01:34:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.672+0000 7fe7880b0b80 1 -- start start 2023-09-10T01:34:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.673+0000 7fe7880b0b80 1 Processor -- start 2023-09-10T01:34:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.673+0000 7fe7880b0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935300 0x55c2e2941e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.673+0000 7fe7880b0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e2935cc0 0x55c2e293f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.673+0000 7fe7880b0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e293b1a0 0x55c2e293f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.673+0000 7fe7880b0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e2825b10 con 0x55c2e2935cc0 2023-09-10T01:34:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.673+0000 7fe7880b0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e2825810 con 0x55c2e2935300 2023-09-10T01:34:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.673+0000 7fe7880b0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e27098c0 con 0x55c2e293b1a0 2023-09-10T01:34:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.673+0000 7fe7880b0b80 10 client.0 _ll_register_callbacks cb 0x55c2e276fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:34:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.674+0000 7fe77a487700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e293b1a0 0x55c2e293f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.674+0000 7fe77a487700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e293b1a0 0x55c2e293f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39634/0 (socket says 172.21.15.44:39634) 2023-09-10T01:34:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.674+0000 7fe77a487700 1 -- 172.21.15.44:0/3509126735 learned_addr learned my addr 172.21.15.44:0/3509126735 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.674+0000 7fe779485700 1 --2- 172.21.15.44:0/3509126735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e2935cc0 0x55c2e293f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.674+0000 7fe779c86700 1 --2- 172.21.15.44:0/3509126735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935300 0x55c2e2941e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.674+0000 7fe779485700 1 -- 172.21.15.44:0/3509126735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e293b1a0 msgr2=0x55c2e293f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.674+0000 7fe779485700 1 --2- 172.21.15.44:0/3509126735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e293b1a0 0x55c2e293f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.674+0000 7fe779485700 1 -- 172.21.15.44:0/3509126735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935300 msgr2=0x55c2e2941e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.674+0000 7fe779485700 1 --2- 172.21.15.44:0/3509126735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935300 0x55c2e2941e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.674+0000 7fe779485700 1 -- 172.21.15.44:0/3509126735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77000e040 con 0x55c2e2935cc0 2023-09-10T01:34:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.674+0000 7fe779c86700 1 --2- 172.21.15.44:0/3509126735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935300 0x55c2e2941e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:34:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.675+0000 7fe779485700 1 --2- 172.21.15.44:0/3509126735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e2935cc0 0x55c2e293f110 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fe770006b00 tx=0x7fe770018640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ba61232f964f57 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.675+0000 7fe75d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:34:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.675+0000 7fe75d7fa700 1 -- 172.21.15.44:0/3509126735 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770018ac0 con 0x55c2e2935cc0 2023-09-10T01:34:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.675+0000 7fe75d7fa700 1 -- 172.21.15.44:0/3509126735 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(205..205 src has 1..205) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fe770000e40 con 0x55c2e2935cc0 2023-09-10T01:34:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.678+0000 7fe7880b0b80 1 -- 172.21.15.44:0/3509126735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2e27df490 con 0x55c2e2935cc0 2023-09-10T01:34:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.684+0000 7fe75d7fa700 1 -- 172.21.15.44:0/3509126735 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=192) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe770018c20 con 0x55c2e2935cc0 2023-09-10T01:34:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.686+0000 7fe7880b0b80 10 client.20631 fetch_fsmap learned FSMap version 192 2023-09-10T01:34:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.686+0000 7fe7880b0b80 1 -- 172.21.15.44:0/3509126735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=192,monmap=4+}) v3 -- 0x55c2e269cc00 con 0x55c2e2935cc0 2023-09-10T01:34:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.690+0000 7fe75d7fa700 1 -- 172.21.15.44:0/3509126735 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe77002e030 con 0x55c2e2935cc0 2023-09-10T01:34:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.690+0000 7fe75d7fa700 1 -- 172.21.15.44:0/3509126735 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 192) v1 ==== 7359+0+0 (secure 0 0 0) 0x7fe770006590 con 0x55c2e2935cc0 2023-09-10T01:34:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.690+0000 7fe7880b0b80 10 client.20631 fetch_fsmap finished waiting for FSMap version 192 2023-09-10T01:34:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.690+0000 7fe7880b0b80 10 client.20631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:34:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.690+0000 7fe7880b0b80 20 client.20631 populate_metadata read hostname 'smithi044' 2023-09-10T01:34:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.690+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/3509126735 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c2e294f440 0x55c2e2951860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.690+0000 7fe7880b0b80 4 client.20631 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:34:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.690+0000 7fe7880b0b80 1 -- 172.21.15.44:0/3509126735 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2e289c840 con 0x55c2e294f440 2023-09-10T01:34:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.691+0000 7fe779c86700 1 --2- 172.21.15.44:0/3509126735 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c2e294f440 0x55c2e2951860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.691+0000 7fe75d7fa700 10 client.20631 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.691+0000 7fe779c86700 1 --2- 172.21.15.44:0/3509126735 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c2e294f440 0x55c2e2951860 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=829b47193646a9bd server_cookie=2f2563014278d4f2 in_seq=0 out_seq=0 2023-09-10T01:34:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.698+0000 7fe75d7fa700 1 -- 172.21.15.44:0/3509126735 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c2e289c840 con 0x55c2e294f440 2023-09-10T01:34:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.698+0000 7fe75d7fa700 10 client.20631 handle_command_reply: tid=0 2023-09-10T01:34:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.698+0000 7fe7880b0b80 1 client.20631 shutdown 2023-09-10T01:34:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.702+0000 7fe7880b0b80 1 -- 172.21.15.44:0/3509126735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e2935cc0 msgr2=0x55c2e293f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.702+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/3509126735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e2935cc0 0x55c2e293f110 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fe770006b00 tx=0x7fe770018640 comp rx=0 tx=0).stop 2023-09-10T01:34:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.702+0000 7fe7880b0b80 1 -- 172.21.15.44:0/3509126735 shutdown_connections 2023-09-10T01:34:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.702+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/3509126735 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c2e294f440 0x55c2e2951860 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.702+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/3509126735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e293b1a0 0x55c2e293f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.702+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/3509126735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935300 0x55c2e2941e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.702+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/3509126735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e2935cc0 0x55c2e293f110 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.702+0000 7fe75d7fa700 0 client.20631 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.702+0000 7fe7880b0b80 1 -- 172.21.15.44:0/3509126735 >> 172.21.15.44:0/3509126735 conn(0x55c2e28aa030 msgr2=0x55c2e292f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.702+0000 7fe7880b0b80 1 -- 172.21.15.44:0/3509126735 shutdown_connections 2023-09-10T01:34:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.703+0000 7fe7880b0b80 1 -- 172.21.15.44:0/3509126735 wait complete. 2023-09-10T01:34:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.709+0000 7fe7880b0b80 20 client.20631 trim_cache size 0 max 16384 2023-09-10T01:34:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.746+0000 7fe7880b0b80 1 Processor -- start 2023-09-10T01:34:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.746+0000 7fe7880b0b80 1 -- start start 2023-09-10T01:34:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.746+0000 7fe7880b0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e2935300 0x55c2e293cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.746+0000 7fe7880b0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935cc0 0x55c2e293a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.747+0000 7fe7880b0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e293b1a0 0x55c2e293a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.747+0000 7fe7880b0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e293ac40 con 0x55c2e2935300 2023-09-10T01:34:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.747+0000 7fe7880b0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e294bda0 con 0x55c2e2935cc0 2023-09-10T01:34:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.747+0000 7fe7880b0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e2940460 con 0x55c2e293b1a0 2023-09-10T01:34:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.747+0000 7fe77a487700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e293b1a0 0x55c2e293a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.747+0000 7fe77a487700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e293b1a0 0x55c2e293a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39646/0 (socket says 172.21.15.44:39646) 2023-09-10T01:34:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.747+0000 7fe77a487700 1 -- 172.21.15.44:0/334183072 learned_addr learned my addr 172.21.15.44:0/334183072 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.747+0000 7fe779c86700 1 --2- 172.21.15.44:0/334183072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e2935300 0x55c2e293cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.747+0000 7fe77a487700 1 -- 172.21.15.44:0/334183072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935cc0 msgr2=0x55c2e293a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.747+0000 7fe77a487700 1 --2- 172.21.15.44:0/334183072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935cc0 0x55c2e293a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.747+0000 7fe77a487700 1 -- 172.21.15.44:0/334183072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e2935300 msgr2=0x55c2e293cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.747+0000 7fe77a487700 1 --2- 172.21.15.44:0/334183072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e2935300 0x55c2e293cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.747+0000 7fe77a487700 1 -- 172.21.15.44:0/334183072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2e26da1f0 con 0x55c2e293b1a0 2023-09-10T01:34:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.749+0000 7fe77a487700 1 --2- 172.21.15.44:0/334183072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e293b1a0 0x55c2e293a640 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe76401cf50 tx=0x7fe76401a320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a47e5e2a36b96c0f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.750+0000 7fe768ff9700 1 -- 172.21.15.44:0/334183072 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76409b1d0 con 0x55c2e293b1a0 2023-09-10T01:34:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.750+0000 7fe768ff9700 1 -- 172.21.15.44:0/334183072 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe76409aad0 con 0x55c2e293b1a0 2023-09-10T01:34:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.750+0000 7fe768ff9700 1 -- 172.21.15.44:0/334183072 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764003eb0 con 0x55c2e293b1a0 2023-09-10T01:34:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.751+0000 7fe7880b0b80 1 -- 172.21.15.44:0/334183072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e293b1a0 msgr2=0x55c2e293a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:15.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.751+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/334183072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e293b1a0 0x55c2e293a640 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe76401cf50 tx=0x7fe76401a320 comp rx=0 tx=0).stop 2023-09-10T01:34:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.766+0000 7fe7880b0b80 1 -- 172.21.15.44:0/334183072 shutdown_connections 2023-09-10T01:34:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.766+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/334183072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e293b1a0 0x55c2e293a640 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.766+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/334183072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e2935300 0x55c2e293cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.766+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/334183072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935cc0 0x55c2e293a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.766+0000 7fe7880b0b80 1 -- 172.21.15.44:0/334183072 >> 172.21.15.44:0/334183072 conn(0x55c2e28aa030 msgr2=0x55c2e29391f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.769+0000 7fe7880b0b80 1 -- 172.21.15.44:0/334183072 shutdown_connections 2023-09-10T01:34:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.769+0000 7fe7880b0b80 1 -- 172.21.15.44:0/334183072 wait complete. 2023-09-10T01:34:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.769+0000 7fe7880b0b80 1 -- start start 2023-09-10T01:34:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.769+0000 7fe7880b0b80 1 Processor -- start 2023-09-10T01:34:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.770+0000 7fe7880b0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e2935300 0x55c2e2945f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.770+0000 7fe7880b0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935cc0 0x55c2e29464b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.770+0000 7fe7880b0b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e293b1a0 0x55c2e2944510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.770+0000 7fe7880b0b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e2944c20 con 0x55c2e293b1a0 2023-09-10T01:34:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.770+0000 7fe7880b0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e2944d90 con 0x55c2e2935cc0 2023-09-10T01:34:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.770+0000 7fe7880b0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e2944f50 con 0x55c2e2935300 2023-09-10T01:34:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.770+0000 7fe7880b0b80 10 client.0 _ll_register_callbacks cb 0x55c2e276fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:34:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.770+0000 7fe779c86700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e2935300 0x55c2e2945f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.770+0000 7fe779c86700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e2935300 0x55c2e2945f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39648/0 (socket says 172.21.15.44:39648) 2023-09-10T01:34:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.770+0000 7fe779c86700 1 -- 172.21.15.44:0/2126520101 learned_addr learned my addr 172.21.15.44:0/2126520101 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.770+0000 7fe779485700 1 --2- 172.21.15.44:0/2126520101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935cc0 0x55c2e29464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.770+0000 7fe779c86700 1 -- 172.21.15.44:0/2126520101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935cc0 msgr2=0x55c2e29464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.771+0000 7fe779c86700 1 --2- 172.21.15.44:0/2126520101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935cc0 0x55c2e29464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.771+0000 7fe779c86700 1 -- 172.21.15.44:0/2126520101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e293b1a0 msgr2=0x55c2e2944510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.771+0000 7fe779c86700 1 --2- 172.21.15.44:0/2126520101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e293b1a0 0x55c2e2944510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.771+0000 7fe779c86700 1 -- 172.21.15.44:0/2126520101 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7741c5940 con 0x55c2e2935300 2023-09-10T01:34:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.771+0000 7fe779c86700 1 --2- 172.21.15.44:0/2126520101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e2935300 0x55c2e2945f70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00afd0 tx=0x7fe76c00f070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c77cebd096ddf14 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.771+0000 7fe75d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:34:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.771+0000 7fe75d7fa700 1 -- 172.21.15.44:0/2126520101 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c022e30 con 0x55c2e2935300 2023-09-10T01:34:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.771+0000 7fe75d7fa700 1 -- 172.21.15.44:0/2126520101 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(205..205 src has 1..205) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fe76c013070 con 0x55c2e2935300 2023-09-10T01:34:15.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.772+0000 7fe7880b0b80 1 -- 172.21.15.44:0/2126520101 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe77000c030 con 0x55c2e2935300 2023-09-10T01:34:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.775+0000 7fe75d7fa700 1 -- 172.21.15.44:0/2126520101 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=192) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe76c01b650 con 0x55c2e2935300 2023-09-10T01:34:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.792+0000 7fe7880b0b80 10 client.30313 fetch_fsmap learned FSMap version 192 2023-09-10T01:34:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.792+0000 7fe7880b0b80 1 -- 172.21.15.44:0/2126520101 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=192,monmap=4+}) v3 -- 0x55c2e26b2250 con 0x55c2e2935300 2023-09-10T01:34:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.792+0000 7fe75d7fa700 1 -- 172.21.15.44:0/2126520101 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe77000c030 con 0x55c2e2935300 2023-09-10T01:34:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.800+0000 7fe75d7fa700 1 -- 172.21.15.44:0/2126520101 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 192) v1 ==== 7359+0+0 (secure 0 0 0) 0x7fe76c01b7b0 con 0x55c2e2935300 2023-09-10T01:34:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.800+0000 7fe7880b0b80 10 client.30313 fetch_fsmap finished waiting for FSMap version 192 2023-09-10T01:34:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.800+0000 7fe7880b0b80 10 client.30313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:34:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.800+0000 7fe7880b0b80 20 client.30313 populate_metadata read hostname 'smithi044' 2023-09-10T01:34:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.800+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/2126520101 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c2e29c6500 0x55c2e29c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.800+0000 7fe7880b0b80 4 client.30313 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:34:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.800+0000 7fe7880b0b80 1 -- 172.21.15.44:0/2126520101 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c2e28808c0 con 0x55c2e29c6500 2023-09-10T01:34:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.817+0000 7fe779485700 1 --2- 172.21.15.44:0/2126520101 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c2e29c6500 0x55c2e29c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.824+0000 7fe779485700 1 --2- 172.21.15.44:0/2126520101 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c2e29c6500 0x55c2e29c8920 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba7cd628c9afaf55 server_cookie=3f43f60f909014d2 in_seq=0 out_seq=0 2023-09-10T01:34:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.830+0000 7fe75d7fa700 10 client.30313 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.881+0000 7fe75d7fa700 1 -- 172.21.15.44:0/2126520101 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c2e28808c0 con 0x55c2e29c6500 2023-09-10T01:34:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.881+0000 7fe75d7fa700 10 client.30313 handle_command_reply: tid=0 2023-09-10T01:34:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.881+0000 7fe7880b0b80 1 client.30313 shutdown 2023-09-10T01:34:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.884+0000 7fe7880b0b80 1 -- 172.21.15.44:0/2126520101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e2935300 msgr2=0x55c2e2945f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.884+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/2126520101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e2935300 0x55c2e2945f70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00afd0 tx=0x7fe76c00f070 comp rx=0 tx=0).stop 2023-09-10T01:34:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.884+0000 7fe7880b0b80 1 -- 172.21.15.44:0/2126520101 shutdown_connections 2023-09-10T01:34:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.884+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/2126520101 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c2e29c6500 0x55c2e29c8920 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.884+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/2126520101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2e293b1a0 0x55c2e2944510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.884+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/2126520101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e2935300 0x55c2e2945f70 secure :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00afd0 tx=0x7fe76c00f070 comp rx=0 tx=0).stop 2023-09-10T01:34:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.884+0000 7fe7880b0b80 1 --2- 172.21.15.44:0/2126520101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c2e2935cc0 0x55c2e29464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.884+0000 7fe7880b0b80 1 -- 172.21.15.44:0/2126520101 >> 172.21.15.44:0/2126520101 conn(0x55c2e28aa030 msgr2=0x55c2e293a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.884+0000 7fe75d7fa700 0 client.30313 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.884+0000 7fe7880b0b80 1 -- 172.21.15.44:0/2126520101 shutdown_connections 2023-09-10T01:34:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.884+0000 7fe7880b0b80 1 -- 172.21.15.44:0/2126520101 wait complete. 2023-09-10T01:34:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.885+0000 7fe7880b0b80 20 client.30313 trim_cache size 0 max 16384 2023-09-10T01:34:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:34:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:34:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:34:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:34:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.895+0000 7fe77beea700 1 -- 172.21.15.44:0/3583019359 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe7600a0980 msgr2=0x7fe7600a2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.895+0000 7fe77beea700 1 --2- 172.21.15.44:0/3583019359 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe7600a0980 0x7fe7600a2e40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55c2e2930060 tx=0x7fe76c009250 comp rx=0 tx=0).stop 2023-09-10T01:34:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.895+0000 7fe77beea700 1 -- 172.21.15.44:0/3583019359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe774119de0 msgr2=0x7fe7741c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.895+0000 7fe77beea700 1 --2- 172.21.15.44:0/3583019359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe774119de0 0x7fe7741c4af0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fe7640053f0 tx=0x7fe7640054d0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.895+0000 7fe77beea700 1 -- 172.21.15.44:0/3583019359 shutdown_connections 2023-09-10T01:34:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.895+0000 7fe77beea700 1 --2- 172.21.15.44:0/3583019359 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fe7600a0980 0x7fe7600a2e40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.895+0000 7fe77beea700 1 --2- 172.21.15.44:0/3583019359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe774119de0 0x7fe7741c4af0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.895+0000 7fe77beea700 1 --2- 172.21.15.44:0/3583019359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe77410ed50 0x7fe774115040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.895+0000 7fe77beea700 1 --2- 172.21.15.44:0/3583019359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe77410f1c0 0x7fe774115580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.895+0000 7fe77beea700 1 -- 172.21.15.44:0/3583019359 >> 172.21.15.44:0/3583019359 conn(0x7fe77406c6c0 msgr2=0x7fe77411b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.895+0000 7fe77beea700 1 -- 172.21.15.44:0/3583019359 shutdown_connections 2023-09-10T01:34:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:15.895+0000 7fe77beea700 1 -- 172.21.15.44:0/3583019359 wait complete. 2023-09-10T01:34:16.183 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 790.0 s, 143.5 tps, lat 714.369 ms stddev 1027.655 2023-09-10T01:34:16.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:15 smithi131 ceph-mon[103713]: pgmap v1746: 129 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 116 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2023-09-10T01:34:16.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:15 smithi044 ceph-mon[98676]: pgmap v1746: 129 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 116 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2023-09-10T01:34:16.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:15 smithi099 ceph-mon[102692]: pgmap v1746: 129 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 116 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2023-09-10T01:34:18.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:17 smithi131 ceph-mon[103713]: osdmap e206: 12 total, 12 up, 12 in 2023-09-10T01:34:18.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:17 smithi131 ceph-mon[103713]: pgmap v1748: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2023-09-10T01:34:18.061 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:17 smithi044 ceph-mon[98676]: osdmap e206: 12 total, 12 up, 12 in 2023-09-10T01:34:18.061 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:17 smithi044 ceph-mon[98676]: pgmap v1748: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2023-09-10T01:34:18.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:17 smithi099 ceph-mon[102692]: osdmap e206: 12 total, 12 up, 12 in 2023-09-10T01:34:18.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:17 smithi099 ceph-mon[102692]: pgmap v1748: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2023-09-10T01:34:20.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:19 smithi131 ceph-mon[103713]: pgmap v1749: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 367 op/s 2023-09-10T01:34:20.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:19 smithi044 ceph-mon[98676]: pgmap v1749: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 367 op/s 2023-09-10T01:34:20.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:19 smithi099 ceph-mon[102692]: pgmap v1749: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 367 op/s 2023-09-10T01:34:21.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 795.0 s, 231.9 tps, lat 466.098 ms stddev 614.778 2023-09-10T01:34:22.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:21 smithi131 ceph-mon[103713]: pgmap v1750: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2023-09-10T01:34:22.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:21 smithi131 ceph-mon[103713]: osdmap e207: 12 total, 12 up, 12 in 2023-09-10T01:34:22.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:21 smithi044 ceph-mon[98676]: pgmap v1750: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2023-09-10T01:34:22.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:21 smithi044 ceph-mon[98676]: osdmap e207: 12 total, 12 up, 12 in 2023-09-10T01:34:22.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:21 smithi099 ceph-mon[102692]: pgmap v1750: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2023-09-10T01:34:22.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:21 smithi099 ceph-mon[102692]: osdmap e207: 12 total, 12 up, 12 in 2023-09-10T01:34:24.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:23 smithi131 ceph-mon[103713]: pgmap v1752: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.1 MiB/s wr, 708 op/s 2023-09-10T01:34:24.312 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:23 smithi044 ceph-mon[98676]: pgmap v1752: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.1 MiB/s wr, 708 op/s 2023-09-10T01:34:24.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:23 smithi099 ceph-mon[102692]: pgmap v1752: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.1 MiB/s wr, 708 op/s 2023-09-10T01:34:26.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 800.0 s, 231.8 tps, lat 440.055 ms stddev 343.271 2023-09-10T01:34:26.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:25 smithi131 ceph-mon[103713]: pgmap v1753: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 600 op/s 2023-09-10T01:34:26.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:25 smithi044 ceph-mon[98676]: pgmap v1753: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 600 op/s 2023-09-10T01:34:26.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:25 smithi099 ceph-mon[102692]: pgmap v1753: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 600 op/s 2023-09-10T01:34:28.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:27 smithi131 ceph-mon[103713]: pgmap v1754: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 747 op/s 2023-09-10T01:34:28.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:27 smithi044 ceph-mon[98676]: pgmap v1754: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 747 op/s 2023-09-10T01:34:28.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:27 smithi099 ceph-mon[102692]: pgmap v1754: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 747 op/s 2023-09-10T01:34:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.110+0000 7f4ea1677700 1 -- 172.21.15.44:0/276491858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c10f1c0 msgr2=0x7f4e9c107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.110+0000 7f4ea1677700 1 --2- 172.21.15.44:0/276491858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c10f1c0 0x7f4e9c107b40 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f4e84009990 tx=0x7f4e84009ca0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.111+0000 7f4ea1677700 1 -- 172.21.15.44:0/276491858 shutdown_connections 2023-09-10T01:34:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.111+0000 7f4ea1677700 1 --2- 172.21.15.44:0/276491858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e9c119de0 0x7f4e9c11c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.111+0000 7f4ea1677700 1 --2- 172.21.15.44:0/276491858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e9c10ed50 0x7f4e9c107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.111+0000 7f4ea1677700 1 --2- 172.21.15.44:0/276491858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c10f1c0 0x7f4e9c107b40 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.111+0000 7f4ea1677700 1 -- 172.21.15.44:0/276491858 >> 172.21.15.44:0/276491858 conn(0x7f4e9c06c6c0 msgr2=0x7f4e9c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.113+0000 7f4ea1677700 1 -- 172.21.15.44:0/276491858 shutdown_connections 2023-09-10T01:34:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.117+0000 7f4ea1677700 1 -- 172.21.15.44:0/276491858 wait complete. 2023-09-10T01:34:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.118+0000 7f4ea1677700 1 Processor -- start 2023-09-10T01:34:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.118+0000 7f4ea1677700 1 -- start start 2023-09-10T01:34:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.119+0000 7f4ea1677700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e9c10f1c0 0x7f4e9c1ab600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.119+0000 7f4ea1677700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e9c119de0 0x7f4e9c1abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.119+0000 7f4ea1677700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c1a5460 0x7f4e9c1a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.119+0000 7f4ea1677700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c1a5ee0 con 0x7f4e9c1a5460 2023-09-10T01:34:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.119+0000 7f4ea1677700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c1a6020 con 0x7f4e9c119de0 2023-09-10T01:34:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.119+0000 7f4ea1677700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c1a61e0 con 0x7f4e9c10f1c0 2023-09-10T01:34:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.120+0000 7f4e927fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e9c119de0 0x7f4e9c1abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.120+0000 7f4e9b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c1a5460 0x7f4e9c1a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.120+0000 7f4e927fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e9c119de0 0x7f4e9c1abb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34526/0 (socket says 172.21.15.44:34526) 2023-09-10T01:34:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.120+0000 7f4e927fc700 1 -- 172.21.15.44:0/1343302967 learned_addr learned my addr 172.21.15.44:0/1343302967 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.120+0000 7f4e9affd700 1 --2- 172.21.15.44:0/1343302967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e9c10f1c0 0x7f4e9c1ab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.120+0000 7f4e9b7fe700 1 -- 172.21.15.44:0/1343302967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e9c10f1c0 msgr2=0x7f4e9c1ab600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.120+0000 7f4e9b7fe700 1 --2- 172.21.15.44:0/1343302967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e9c10f1c0 0x7f4e9c1ab600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.120+0000 7f4e9b7fe700 1 -- 172.21.15.44:0/1343302967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e9c119de0 msgr2=0x7f4e9c1abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.120+0000 7f4e9b7fe700 1 --2- 172.21.15.44:0/1343302967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e9c119de0 0x7f4e9c1abb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.120+0000 7f4e9b7fe700 1 -- 172.21.15.44:0/1343302967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e840095f0 con 0x7f4e9c1a5460 2023-09-10T01:34:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.121+0000 7f4e9b7fe700 1 --2- 172.21.15.44:0/1343302967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c1a5460 0x7f4e9c1a5910 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400b3c0 tx=0x7f4e9400b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.128+0000 7f4e98ff9700 1 -- 172.21.15.44:0/1343302967 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94011830 con 0x7f4e9c1a5460 2023-09-10T01:34:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.128+0000 7f4ea1677700 1 -- 172.21.15.44:0/1343302967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c1b2470 con 0x7f4e9c1a5460 2023-09-10T01:34:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.128+0000 7f4e98ff9700 1 -- 172.21.15.44:0/1343302967 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e9400be80 con 0x7f4e9c1a5460 2023-09-10T01:34:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.128+0000 7f4e98ff9700 1 -- 172.21.15.44:0/1343302967 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94010ca0 con 0x7f4e9c1a5460 2023-09-10T01:34:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.128+0000 7f4ea1677700 1 -- 172.21.15.44:0/1343302967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c1b2a00 con 0x7f4e9c1a5460 2023-09-10T01:34:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.133+0000 7f4ead83db80 1 Processor -- start 2023-09-10T01:34:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.141+0000 7f4e98ff9700 1 -- 172.21.15.44:0/1343302967 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f4e94010380 con 0x7f4e9c1a5460 2023-09-10T01:34:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.142+0000 7f4e98ff9700 1 --2- 172.21.15.44:0/1343302967 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e880a0940 0x7f4e880a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.142+0000 7f4e98ff9700 1 -- 172.21.15.44:0/1343302967 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f4e940cbaf0 con 0x7f4e9c1a5460 2023-09-10T01:34:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.142+0000 7f4e927fc700 1 --2- 172.21.15.44:0/1343302967 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e880a0940 0x7f4e880a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.143+0000 7f4e927fc700 1 --2- 172.21.15.44:0/1343302967 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e880a0940 0x7f4e880a2e00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c119590 tx=0x7f4e840058e0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.145+0000 7f4ead83db80 1 -- start start 2023-09-10T01:34:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.146+0000 7f4ead83db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 0x55f2902e7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.146+0000 7f4ead83db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1300 0x55f2902e1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.146+0000 7f4ead83db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e1cc0 0x55f2902edf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.146+0000 7f4ead83db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f29023a6d0 con 0x55f2902e71a0 2023-09-10T01:34:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.146+0000 7f4ead83db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f290237ff0 con 0x55f2902e1300 2023-09-10T01:34:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.146+0000 7f4ead83db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2902338e0 con 0x55f2902e1cc0 2023-09-10T01:34:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.152+0000 7f4e927fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1300 0x55f2902e1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.152+0000 7f4e927fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1300 0x55f2902e1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34534/0 (socket says 172.21.15.44:34534) 2023-09-10T01:34:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.152+0000 7f4e927fc700 1 -- 172.21.15.44:0/1071540932 learned_addr learned my addr 172.21.15.44:0/1071540932 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.152+0000 7f4e9b7fe700 1 --2- 172.21.15.44:0/1071540932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e1cc0 0x55f2902edf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.153+0000 7f4e927fc700 1 -- 172.21.15.44:0/1071540932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e1cc0 msgr2=0x55f2902edf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.153+0000 7f4e927fc700 1 --2- 172.21.15.44:0/1071540932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e1cc0 0x55f2902edf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.153+0000 7f4e927fc700 1 -- 172.21.15.44:0/1071540932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 msgr2=0x55f2902e7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.153+0000 7f4e927fc700 1 --2- 172.21.15.44:0/1071540932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 0x55f2902e7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.153+0000 7f4e927fc700 1 -- 172.21.15.44:0/1071540932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f29005fd80 con 0x55f2902e1300 2023-09-10T01:34:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.153+0000 7f4e927fc700 1 --2- 172.21.15.44:0/1071540932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1300 0x55f2902e1780 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4e840054c0 tx=0x7f4e84020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a269dd651eebffb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.160+0000 7f4e917fa700 1 -- 172.21.15.44:0/1071540932 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8400be00 con 0x55f2902e1300 2023-09-10T01:34:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.160+0000 7f4e917fa700 1 -- 172.21.15.44:0/1071540932 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e8401e040 con 0x55f2902e1300 2023-09-10T01:34:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.160+0000 7f4e917fa700 1 -- 172.21.15.44:0/1071540932 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8400f860 con 0x55f2902e1300 2023-09-10T01:34:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.161+0000 7f4ead83db80 1 -- 172.21.15.44:0/1071540932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1300 msgr2=0x55f2902e1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.161+0000 7f4ead83db80 1 --2- 172.21.15.44:0/1071540932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1300 0x55f2902e1780 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4e840054c0 tx=0x7f4e84020000 comp rx=0 tx=0).stop 2023-09-10T01:34:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.167+0000 7f4ead83db80 1 -- 172.21.15.44:0/1071540932 shutdown_connections 2023-09-10T01:34:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.167+0000 7f4ead83db80 1 --2- 172.21.15.44:0/1071540932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e1cc0 0x55f2902edf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.167+0000 7f4ead83db80 1 --2- 172.21.15.44:0/1071540932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 0x55f2902e7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.167+0000 7f4ead83db80 1 --2- 172.21.15.44:0/1071540932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1300 0x55f2902e1780 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.167+0000 7f4ead83db80 1 -- 172.21.15.44:0/1071540932 >> 172.21.15.44:0/1071540932 conn(0x55f290256030 msgr2=0x55f2902dc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.172+0000 7f4ead83db80 1 -- 172.21.15.44:0/1071540932 shutdown_connections 2023-09-10T01:34:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.180+0000 7f4ead83db80 1 -- 172.21.15.44:0/1071540932 wait complete. 2023-09-10T01:34:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.180+0000 7f4ead83db80 1 -- start start 2023-09-10T01:34:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.180+0000 7f4ead83db80 1 Processor -- start 2023-09-10T01:34:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.181+0000 7f4ead83db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1300 0x55f2902edea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.181+0000 7f4ead83db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e1cc0 0x55f2902eb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.181+0000 7f4ead83db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 0x55f2902eb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.181+0000 7f4ead83db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2901d1b10 con 0x55f2902e71a0 2023-09-10T01:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.181+0000 7f4ead83db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2901d1810 con 0x55f2902e1300 2023-09-10T01:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.181+0000 7f4ead83db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2900b58c0 con 0x55f2902e1cc0 2023-09-10T01:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.181+0000 7f4ead83db80 10 client.0 _ll_register_callbacks cb 0x55f29011ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.181+0000 7f4e9b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 0x55f2902eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.181+0000 7f4e9b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 0x55f2902eb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32800/0 (socket says 172.21.15.44:32800) 2023-09-10T01:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.181+0000 7f4e9b7fe700 1 -- 172.21.15.44:0/3463833079 learned_addr learned my addr 172.21.15.44:0/3463833079 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.182+0000 7f4e9affd700 1 --2- 172.21.15.44:0/3463833079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1300 0x55f2902edea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.182+0000 7f4e9b7fe700 1 -- 172.21.15.44:0/3463833079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e1cc0 msgr2=0x55f2902eb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.182+0000 7f4e9b7fe700 1 --2- 172.21.15.44:0/3463833079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e1cc0 0x55f2902eb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.182+0000 7f4e9b7fe700 1 -- 172.21.15.44:0/3463833079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1300 msgr2=0x55f2902edea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.182+0000 7f4e9b7fe700 1 --2- 172.21.15.44:0/3463833079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1300 0x55f2902edea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.182+0000 7f4e9b7fe700 1 -- 172.21.15.44:0/3463833079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c11cb00 con 0x55f2902e71a0 2023-09-10T01:34:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.182+0000 7f4e9b7fe700 1 --2- 172.21.15.44:0/3463833079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 0x55f2902eb860 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400bad0 tx=0x7f4e94099410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7739858965f7b8c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.182+0000 7f4e827fc700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:34:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.182+0000 7f4e827fc700 1 -- 172.21.15.44:0/3463833079 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e940a0cd0 con 0x55f2902e71a0 2023-09-10T01:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.182+0000 7f4e827fc700 1 -- 172.21.15.44:0/3463833079 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(207..207 src has 1..207) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f4e94098150 con 0x55f2902e71a0 2023-09-10T01:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.182+0000 7f4ead83db80 1 -- 172.21.15.44:0/3463833079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f29018b490 con 0x55f2902e71a0 2023-09-10T01:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.183+0000 7f4e827fc700 1 -- 172.21.15.44:0/3463833079 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=193) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e94098310 con 0x55f2902e71a0 2023-09-10T01:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.183+0000 7f4ead83db80 10 client.20643 fetch_fsmap learned FSMap version 193 2023-09-10T01:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.183+0000 7f4ead83db80 1 -- 172.21.15.44:0/3463833079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=193,monmap=4+}) v3 -- 0x55f290048c00 con 0x55f2902e71a0 2023-09-10T01:34:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.183+0000 7f4e827fc700 1 -- 172.21.15.44:0/3463833079 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e940a4020 con 0x55f2902e71a0 2023-09-10T01:34:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.183+0000 7f4e827fc700 1 -- 172.21.15.44:0/3463833079 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 193) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f4e940b3030 con 0x55f2902e71a0 2023-09-10T01:34:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.183+0000 7f4ead83db80 10 client.20643 fetch_fsmap finished waiting for FSMap version 193 2023-09-10T01:34:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.183+0000 7f4ead83db80 10 client.20643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:34:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.183+0000 7f4ead83db80 20 client.20643 populate_metadata read hostname 'smithi044' 2023-09-10T01:34:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.183+0000 7f4ead83db80 1 --2- 172.21.15.44:0/3463833079 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2902fb5d0 0x55f2902fd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.183+0000 7f4ead83db80 4 client.20643 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:34:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.183+0000 7f4ead83db80 1 -- 172.21.15.44:0/3463833079 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f290248840 con 0x55f2902fb5d0 2023-09-10T01:34:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.194+0000 7f4e9affd700 1 --2- 172.21.15.44:0/3463833079 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2902fb5d0 0x55f2902fd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.200+0000 7f4e827fc700 10 client.20643 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.200+0000 7f4e9affd700 1 --2- 172.21.15.44:0/3463833079 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2902fb5d0 0x55f2902fd9f0 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d418ef28119c942 server_cookie=345ec911a44acc93 in_seq=0 out_seq=0 2023-09-10T01:34:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.201+0000 7f4e827fc700 1 -- 172.21.15.44:0/3463833079 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f290248840 con 0x55f2902fb5d0 2023-09-10T01:34:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.201+0000 7f4e827fc700 10 client.20643 handle_command_reply: tid=0 2023-09-10T01:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.201+0000 7f4ead83db80 1 client.20643 shutdown 2023-09-10T01:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.207+0000 7f4ead83db80 1 -- 172.21.15.44:0/3463833079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 msgr2=0x55f2902eb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.208+0000 7f4ead83db80 1 --2- 172.21.15.44:0/3463833079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 0x55f2902eb860 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400bad0 tx=0x7f4e94099410 comp rx=0 tx=0).stop 2023-09-10T01:34:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.212+0000 7f4ead83db80 1 -- 172.21.15.44:0/3463833079 shutdown_connections 2023-09-10T01:34:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.212+0000 7f4ead83db80 1 --2- 172.21.15.44:0/3463833079 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f2902fb5d0 0x55f2902fd9f0 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.212+0000 7f4ead83db80 1 --2- 172.21.15.44:0/3463833079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 0x55f2902eb860 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.212+0000 7f4ead83db80 1 --2- 172.21.15.44:0/3463833079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1300 0x55f2902edea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.212+0000 7f4ead83db80 1 --2- 172.21.15.44:0/3463833079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e1cc0 0x55f2902eb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.212+0000 7f4ead83db80 1 -- 172.21.15.44:0/3463833079 >> 172.21.15.44:0/3463833079 conn(0x55f290256030 msgr2=0x55f2902db7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.215+0000 7f4e827fc700 0 client.20643 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.215+0000 7f4ead83db80 1 -- 172.21.15.44:0/3463833079 shutdown_connections 2023-09-10T01:34:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.221+0000 7f4ead83db80 1 -- 172.21.15.44:0/3463833079 wait complete. 2023-09-10T01:34:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.234+0000 7f4ead83db80 20 client.20643 trim_cache size 0 max 16384 2023-09-10T01:34:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.274+0000 7f4ead83db80 1 Processor -- start 2023-09-10T01:34:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.274+0000 7f4ead83db80 1 -- start start 2023-09-10T01:34:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.275+0000 7f4ead83db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e1300 0x55f2902e8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.275+0000 7f4ead83db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1cc0 0x55f2902ea3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.275+0000 7f4ead83db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 0x55f2902ed9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.275+0000 7f4ead83db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2902ebeb0 con 0x55f2902e71a0 2023-09-10T01:34:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.275+0000 7f4ead83db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2902dd930 con 0x55f2902e1cc0 2023-09-10T01:34:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.275+0000 7f4ead83db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2902ddb40 con 0x55f2902e1300 2023-09-10T01:34:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.275+0000 7f4e9b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 0x55f2902ed9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.275+0000 7f4e9b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 0x55f2902ed9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32810/0 (socket says 172.21.15.44:32810) 2023-09-10T01:34:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.275+0000 7f4e9b7fe700 1 -- 172.21.15.44:0/855496779 learned_addr learned my addr 172.21.15.44:0/855496779 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.275+0000 7f4e9affd700 1 --2- 172.21.15.44:0/855496779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e1300 0x55f2902e8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.275+0000 7f4e927fc700 1 --2- 172.21.15.44:0/855496779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1cc0 0x55f2902ea3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.276+0000 7f4e9b7fe700 1 -- 172.21.15.44:0/855496779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e1300 msgr2=0x55f2902e8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.276+0000 7f4e9b7fe700 1 --2- 172.21.15.44:0/855496779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e1300 0x55f2902e8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.276+0000 7f4e9b7fe700 1 -- 172.21.15.44:0/855496779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1cc0 msgr2=0x55f2902ea3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.276+0000 7f4e9b7fe700 1 --2- 172.21.15.44:0/855496779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1cc0 0x55f2902ea3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.276+0000 7f4e9b7fe700 1 -- 172.21.15.44:0/855496779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2900861f0 con 0x55f2902e71a0 2023-09-10T01:34:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.276+0000 7f4e9b7fe700 1 --2- 172.21.15.44:0/855496779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 0x55f2902ed9d0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f4e94095160 tx=0x7f4e94098b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff068d34125fdd5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:29.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.276+0000 7f4e91ffb700 1 -- 172.21.15.44:0/855496779 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e940b80b0 con 0x55f2902e71a0 2023-09-10T01:34:29.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.276+0000 7f4e91ffb700 1 -- 172.21.15.44:0/855496779 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e940a4070 con 0x55f2902e71a0 2023-09-10T01:34:29.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.276+0000 7f4e91ffb700 1 -- 172.21.15.44:0/855496779 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e940a0840 con 0x55f2902e71a0 2023-09-10T01:34:29.282 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.277+0000 7f4ead83db80 1 -- 172.21.15.44:0/855496779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 msgr2=0x55f2902ed9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.277+0000 7f4ead83db80 1 --2- 172.21.15.44:0/855496779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 0x55f2902ed9d0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f4e94095160 tx=0x7f4e94098b30 comp rx=0 tx=0).stop 2023-09-10T01:34:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.277+0000 7f4ead83db80 1 -- 172.21.15.44:0/855496779 shutdown_connections 2023-09-10T01:34:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.277+0000 7f4ead83db80 1 --2- 172.21.15.44:0/855496779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2902e71a0 0x55f2902ed9d0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.277+0000 7f4ead83db80 1 --2- 172.21.15.44:0/855496779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e1300 0x55f2902e8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.277+0000 7f4ead83db80 1 --2- 172.21.15.44:0/855496779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902e1cc0 0x55f2902ea3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.277+0000 7f4ead83db80 1 -- 172.21.15.44:0/855496779 >> 172.21.15.44:0/855496779 conn(0x55f290256030 msgr2=0x55f2902e6150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.280+0000 7f4ead83db80 1 -- 172.21.15.44:0/855496779 shutdown_connections 2023-09-10T01:34:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.288+0000 7f4ead83db80 1 -- 172.21.15.44:0/855496779 wait complete. 2023-09-10T01:34:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.288+0000 7f4ead83db80 1 -- start start 2023-09-10T01:34:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.289+0000 7f4ead83db80 1 Processor -- start 2023-09-10T01:34:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.289+0000 7f4ead83db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e71a0 0x55f2902f0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.289+0000 7f4ead83db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902f0c40 0x55f2903483d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.289+0000 7f4ead83db80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f290348910 0x55f290348d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.289+0000 7f4ead83db80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2902f11c0 con 0x55f290348910 2023-09-10T01:34:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.289+0000 7f4ead83db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2902f1330 con 0x55f2902f0c40 2023-09-10T01:34:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.289+0000 7f4ead83db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f29034f480 con 0x55f2902e71a0 2023-09-10T01:34:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.289+0000 7f4ead83db80 10 client.0 _ll_register_callbacks cb 0x7f4e940985c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:34:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.296+0000 7f4e9b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f290348910 0x55f290348d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.296+0000 7f4e9b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f290348910 0x55f290348d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32814/0 (socket says 172.21.15.44:32814) 2023-09-10T01:34:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.296+0000 7f4e9b7fe700 1 -- 172.21.15.44:0/46062649 learned_addr learned my addr 172.21.15.44:0/46062649 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.297+0000 7f4e9b7fe700 1 -- 172.21.15.44:0/46062649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e71a0 msgr2=0x55f2902f0700 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:34:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.297+0000 7f4e9b7fe700 1 --2- 172.21.15.44:0/46062649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e71a0 0x55f2902f0700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.297+0000 7f4e9b7fe700 1 -- 172.21.15.44:0/46062649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902f0c40 msgr2=0x55f2903483d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:34:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.297+0000 7f4e9b7fe700 1 --2- 172.21.15.44:0/46062649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902f0c40 0x55f2903483d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.297+0000 7f4e9b7fe700 1 -- 172.21.15.44:0/46062649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f290048c00 con 0x55f290348910 2023-09-10T01:34:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.297+0000 7f4e9b7fe700 1 --2- 172.21.15.44:0/46062649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f290348910 0x55f290348d90 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f4e940048a0 tx=0x7f4e94010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8135db17e8b0ab5f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.297+0000 7f4e827fc700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:34:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.297+0000 7f4e827fc700 1 -- 172.21.15.44:0/46062649 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94097ea0 con 0x55f290348910 2023-09-10T01:34:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.297+0000 7f4e827fc700 1 -- 172.21.15.44:0/46062649 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(207..207 src has 1..207) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f4e940a5080 con 0x55f290348910 2023-09-10T01:34:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.297+0000 7f4ead83db80 1 -- 172.21.15.44:0/46062649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e8401e040 con 0x55f290348910 2023-09-10T01:34:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.298+0000 7f4e827fc700 1 -- 172.21.15.44:0/46062649 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=193) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e94098c60 con 0x55f290348910 2023-09-10T01:34:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.298+0000 7f4ead83db80 10 client.20649 fetch_fsmap learned FSMap version 193 2023-09-10T01:34:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.298+0000 7f4ead83db80 1 -- 172.21.15.44:0/46062649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=193,monmap=4+}) v3 -- 0x55f29005e250 con 0x55f290348910 2023-09-10T01:34:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.298+0000 7f4e827fc700 1 -- 172.21.15.44:0/46062649 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e8401e040 con 0x55f290348910 2023-09-10T01:34:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.299+0000 7f4e827fc700 1 -- 172.21.15.44:0/46062649 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 193) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f4e940b3070 con 0x55f290348910 2023-09-10T01:34:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.299+0000 7f4ead83db80 10 client.20649 fetch_fsmap finished waiting for FSMap version 193 2023-09-10T01:34:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.299+0000 7f4ead83db80 10 client.20649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:34:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.299+0000 7f4ead83db80 20 client.20649 populate_metadata read hostname 'smithi044' 2023-09-10T01:34:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.299+0000 7f4ead83db80 1 --2- 172.21.15.44:0/46062649 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f290383760 0x55f290385c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.299+0000 7f4ead83db80 4 client.20649 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:34:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.299+0000 7f4ead83db80 1 -- 172.21.15.44:0/46062649 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f29022c8c0 con 0x55f290383760 2023-09-10T01:34:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.302+0000 7f4e9affd700 1 --2- 172.21.15.44:0/46062649 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f290383760 0x55f290385c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.302+0000 7f4e9affd700 1 --2- 172.21.15.44:0/46062649 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f290383760 0x55f290385c20 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e30c842a09436d2c server_cookie=273c9bb75b345371 in_seq=0 out_seq=0 2023-09-10T01:34:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.302+0000 7f4e827fc700 10 client.20649 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.334+0000 7f4e827fc700 1 -- 172.21.15.44:0/46062649 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f29022c8c0 con 0x55f290383760 2023-09-10T01:34:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.334+0000 7f4e827fc700 10 client.20649 handle_command_reply: tid=0 2023-09-10T01:34:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.334+0000 7f4ead83db80 1 client.20649 shutdown 2023-09-10T01:34:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.334+0000 7f4ead83db80 1 -- 172.21.15.44:0/46062649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f290348910 msgr2=0x55f290348d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.334+0000 7f4ead83db80 1 --2- 172.21.15.44:0/46062649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f290348910 0x55f290348d90 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f4e940048a0 tx=0x7f4e94010eb0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.335+0000 7f4ead83db80 1 -- 172.21.15.44:0/46062649 shutdown_connections 2023-09-10T01:34:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.335+0000 7f4ead83db80 1 --2- 172.21.15.44:0/46062649 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55f290383760 0x55f290385c20 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.335+0000 7f4ead83db80 1 --2- 172.21.15.44:0/46062649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f290348910 0x55f290348d90 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.335+0000 7f4e827fc700 0 client.20649 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.335+0000 7f4ead83db80 1 --2- 172.21.15.44:0/46062649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2902e71a0 0x55f2902f0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.335+0000 7f4ead83db80 1 --2- 172.21.15.44:0/46062649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2902f0c40 0x55f2903483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.335+0000 7f4ead83db80 1 -- 172.21.15.44:0/46062649 >> 172.21.15.44:0/46062649 conn(0x55f290256030 msgr2=0x55f2902e51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.335+0000 7f4ead83db80 1 -- 172.21.15.44:0/46062649 shutdown_connections 2023-09-10T01:34:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.335+0000 7f4ead83db80 1 -- 172.21.15.44:0/46062649 wait complete. 2023-09-10T01:34:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.338+0000 7f4ead83db80 20 client.20649 trim_cache size 0 max 16384 2023-09-10T01:34:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:34:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:34:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:34:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:34:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.345+0000 7f4ea1677700 1 -- 172.21.15.44:0/1343302967 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e880a0940 msgr2=0x7f4e880a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.345+0000 7f4ea1677700 1 --2- 172.21.15.44:0/1343302967 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e880a0940 0x7f4e880a2e00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c119590 tx=0x7f4e840058e0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.345+0000 7f4ea1677700 1 -- 172.21.15.44:0/1343302967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c1a5460 msgr2=0x7f4e9c1a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.345+0000 7f4ea1677700 1 --2- 172.21.15.44:0/1343302967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c1a5460 0x7f4e9c1a5910 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400b3c0 tx=0x7f4e9400b6d0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.345+0000 7f4ea1677700 1 -- 172.21.15.44:0/1343302967 shutdown_connections 2023-09-10T01:34:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.345+0000 7f4ea1677700 1 --2- 172.21.15.44:0/1343302967 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f4e880a0940 0x7f4e880a2e00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.345+0000 7f4ea1677700 1 --2- 172.21.15.44:0/1343302967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c1a5460 0x7f4e9c1a5910 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.345+0000 7f4ea1677700 1 --2- 172.21.15.44:0/1343302967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e9c10f1c0 0x7f4e9c1ab600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.345+0000 7f4ea1677700 1 --2- 172.21.15.44:0/1343302967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e9c119de0 0x7f4e9c1abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.345+0000 7f4ea1677700 1 -- 172.21.15.44:0/1343302967 >> 172.21.15.44:0/1343302967 conn(0x7f4e9c06c6c0 msgr2=0x7f4e9c10c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.345+0000 7f4ea1677700 1 -- 172.21.15.44:0/1343302967 shutdown_connections 2023-09-10T01:34:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:29.346+0000 7f4ea1677700 1 -- 172.21.15.44:0/1343302967 wait complete. 2023-09-10T01:34:30.048 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:29 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:34:30.049 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:29 smithi131 ceph-mon[103713]: pgmap v1755: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.9 MiB/s wr, 773 op/s 2023-09-10T01:34:30.061 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:29 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:34:30.062 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:29 smithi044 ceph-mon[98676]: pgmap v1755: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.9 MiB/s wr, 773 op/s 2023-09-10T01:34:30.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:29 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:34:30.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:29 smithi099 ceph-mon[102692]: pgmap v1755: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.9 MiB/s wr, 773 op/s 2023-09-10T01:34:31.191 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 805.0 s, 243.8 tps, lat 411.653 ms stddev 155.032 2023-09-10T01:34:32.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:31 smithi131 ceph-mon[103713]: pgmap v1756: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.9 MiB/s wr, 772 op/s 2023-09-10T01:34:32.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:31 smithi044 ceph-mon[98676]: pgmap v1756: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.9 MiB/s wr, 772 op/s 2023-09-10T01:34:32.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:31 smithi099 ceph-mon[102692]: pgmap v1756: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.9 MiB/s wr, 772 op/s 2023-09-10T01:34:34.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:33 smithi131 ceph-mon[103713]: pgmap v1757: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 760 op/s 2023-09-10T01:34:34.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:33 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:34:34.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:33 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:34.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:33 smithi044 ceph-mon[98676]: pgmap v1757: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 760 op/s 2023-09-10T01:34:34.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:33 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:34:34.310 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:33 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:34.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:33 smithi099 ceph-mon[102692]: pgmap v1757: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 760 op/s 2023-09-10T01:34:34.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:33 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:34:34.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:33 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:35.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:34 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:35.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:34 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:35.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:34 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:35.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:34 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:35.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:34 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:35.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:34 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:35.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:34 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:35.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:34 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:35.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:34 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:35.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:34 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:35.361 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:34 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:35.361 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:34 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:35.361 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:34 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:35.361 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:34 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:35.361 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:34 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:36.185 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 810.0 s, 151.0 tps, lat 643.028 ms stddev 451.422 2023-09-10T01:34:36.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:35 smithi131 ceph-mon[103713]: pgmap v1758: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.8 MiB/s wr, 733 op/s 2023-09-10T01:34:36.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:35 smithi044 ceph-mon[98676]: pgmap v1758: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.8 MiB/s wr, 733 op/s 2023-09-10T01:34:36.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:35 smithi099 ceph-mon[102692]: pgmap v1758: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.8 MiB/s wr, 733 op/s 2023-09-10T01:34:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:36 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:34:37.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:36 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:34:37.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:36 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:37.312 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:36 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:34:37.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:36 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:34:37.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:36 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:37.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:36 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:34:37.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:36 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:34:37.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:36 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:34:38.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:37 smithi131 ceph-mon[103713]: pgmap v1759: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.5 MiB/s wr, 751 op/s 2023-09-10T01:34:38.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:37 smithi044 ceph-mon[98676]: pgmap v1759: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.5 MiB/s wr, 751 op/s 2023-09-10T01:34:38.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:37 smithi099 ceph-mon[102692]: pgmap v1759: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.5 MiB/s wr, 751 op/s 2023-09-10T01:34:40.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:39 smithi131 ceph-mon[103713]: pgmap v1760: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.4 MiB/s wr, 788 op/s 2023-09-10T01:34:40.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:39 smithi044 ceph-mon[98676]: pgmap v1760: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.4 MiB/s wr, 788 op/s 2023-09-10T01:34:40.358 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:39 smithi099 ceph-mon[102692]: pgmap v1760: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.4 MiB/s wr, 788 op/s 2023-09-10T01:34:41.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 815.0 s, 306.3 tps, lat 342.543 ms stddev 179.879 2023-09-10T01:34:42.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:41 smithi131 ceph-mon[103713]: pgmap v1761: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 781 op/s 2023-09-10T01:34:42.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:41 smithi044 ceph-mon[98676]: pgmap v1761: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 781 op/s 2023-09-10T01:34:42.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:41 smithi099 ceph-mon[102692]: pgmap v1761: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 781 op/s 2023-09-10T01:34:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.552+0000 7f156d3aa700 1 -- 172.21.15.44:0/919274137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1568100f70 msgr2=0x7f156810d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.552+0000 7f156d3aa700 1 --2- 172.21.15.44:0/919274137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1568100f70 0x7f156810d430 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f156000b0a0 tx=0x7f156000b3b0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.557+0000 7f156d3aa700 1 -- 172.21.15.44:0/919274137 shutdown_connections 2023-09-10T01:34:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.557+0000 7f156d3aa700 1 --2- 172.21.15.44:0/919274137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1568100f70 0x7f156810d430 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.557+0000 7f156d3aa700 1 --2- 172.21.15.44:0/919274137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1568106670 0x7f1568106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.557+0000 7f156d3aa700 1 --2- 172.21.15.44:0/919274137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15681005b0 0x7f1568100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.557+0000 7f156d3aa700 1 -- 172.21.15.44:0/919274137 >> 172.21.15.44:0/919274137 conn(0x7f15680fc060 msgr2=0x7f15680fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.558+0000 7f156d3aa700 1 -- 172.21.15.44:0/919274137 shutdown_connections 2023-09-10T01:34:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.558+0000 7f156d3aa700 1 -- 172.21.15.44:0/919274137 wait complete. 2023-09-10T01:34:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.559+0000 7f156d3aa700 1 Processor -- start 2023-09-10T01:34:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.560+0000 7f156d3aa700 1 -- start start 2023-09-10T01:34:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.560+0000 7f156d3aa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15681005b0 0x7f156819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.560+0000 7f156d3aa700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1568100f70 0x7f156819d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.560+0000 7f156d3aa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1568106670 0x7f15681a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.560+0000 7f156d3aa700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15681a0dc0 con 0x7f15681005b0 2023-09-10T01:34:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.560+0000 7f156d3aa700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15681a0f30 con 0x7f1568106670 2023-09-10T01:34:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.560+0000 7f156d3aa700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15681a10f0 con 0x7f1568100f70 2023-09-10T01:34:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.560+0000 7f15667fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1568100f70 0x7f156819d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.560+0000 7f15667fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1568100f70 0x7f156819d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39708/0 (socket says 172.21.15.44:39708) 2023-09-10T01:34:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.560+0000 7f15667fc700 1 -- 172.21.15.44:0/2543069168 learned_addr learned my addr 172.21.15.44:0/2543069168 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.561+0000 7f15667fc700 1 -- 172.21.15.44:0/2543069168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1568106670 msgr2=0x7f15681a07c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-10T01:34:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.561+0000 7f15667fc700 1 --2- 172.21.15.44:0/2543069168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1568106670 0x7f15681a07c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.561+0000 7f15667fc700 1 -- 172.21.15.44:0/2543069168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15681005b0 msgr2=0x7f156819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.561+0000 7f15667fc700 1 --2- 172.21.15.44:0/2543069168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15681005b0 0x7f156819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.561+0000 7f15667fc700 1 -- 172.21.15.44:0/2543069168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1560009d00 con 0x7f1568100f70 2023-09-10T01:34:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.562+0000 7f15667fc700 1 --2- 172.21.15.44:0/2543069168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1568100f70 0x7f156819d200 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f155800b530 tx=0x7f155800b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.563+0000 7f154ffff700 1 -- 172.21.15.44:0/2543069168 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558011830 con 0x7f1568100f70 2023-09-10T01:34:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.563+0000 7f154ffff700 1 -- 172.21.15.44:0/2543069168 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1558011e20 con 0x7f1568100f70 2023-09-10T01:34:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.563+0000 7f154ffff700 1 -- 172.21.15.44:0/2543069168 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558010dd0 con 0x7f1568100f70 2023-09-10T01:34:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.563+0000 7f156d3aa700 1 -- 172.21.15.44:0/2543069168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15681a13d0 con 0x7f1568100f70 2023-09-10T01:34:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.563+0000 7f156d3aa700 1 -- 172.21.15.44:0/2543069168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15681a9d30 con 0x7f1568100f70 2023-09-10T01:34:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.567+0000 7f1579570b80 1 Processor -- start 2023-09-10T01:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.567+0000 7f1579570b80 1 -- start start 2023-09-10T01:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.567+0000 7f1579570b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f00371a0 0x5641f0037580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.567+0000 7f1579570b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f0031300 0x5641f0031780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.567+0000 7f1579570b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031cc0 0x5641f003df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.567+0000 7f1579570b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641eff8a6d0 con 0x5641f0031cc0 2023-09-10T01:34:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.567+0000 7f1579570b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641eff87ff0 con 0x5641f0031300 2023-09-10T01:34:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.567+0000 7f1579570b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641eff838e0 con 0x5641f00371a0 2023-09-10T01:34:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.569+0000 7f15677fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f00371a0 0x5641f0037580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.569+0000 7f15677fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f00371a0 0x5641f0037580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39714/0 (socket says 172.21.15.44:39714) 2023-09-10T01:34:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.569+0000 7f15677fe700 1 -- 172.21.15.44:0/1852573499 learned_addr learned my addr 172.21.15.44:0/1852573499 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.569+0000 7f15677fe700 1 -- 172.21.15.44:0/1852573499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f0031300 msgr2=0x5641f0031780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.569+0000 7f15677fe700 1 --2- 172.21.15.44:0/1852573499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f0031300 0x5641f0031780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.569+0000 7f15677fe700 1 -- 172.21.15.44:0/1852573499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031cc0 msgr2=0x5641f003df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.569+0000 7f15677fe700 1 --2- 172.21.15.44:0/1852573499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031cc0 0x5641f003df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.569+0000 7f15677fe700 1 -- 172.21.15.44:0/1852573499 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641efdafd80 con 0x5641f00371a0 2023-09-10T01:34:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.570+0000 7f15677fe700 1 --2- 172.21.15.44:0/1852573499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f00371a0 0x5641f0037580 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f156000b070 tx=0x7f1560005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5966026c8abb9b29 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.571+0000 7f154ffff700 1 -- 172.21.15.44:0/2543069168 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1558011990 con 0x7f1568100f70 2023-09-10T01:34:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.572+0000 7f154ffff700 1 --2- 172.21.15.44:0/2543069168 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f15500a08e0 0x7f15500a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.572+0000 7f154ffff700 1 -- 172.21.15.44:0/2543069168 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f15580cde70 con 0x7f1568100f70 2023-09-10T01:34:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.572+0000 7f1566ffd700 1 --2- 172.21.15.44:0/2543069168 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f15500a08e0 0x7f15500a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.573+0000 7f1566ffd700 1 --2- 172.21.15.44:0/2543069168 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f15500a08e0 0x7f15500a2da0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x5641f002c060 tx=0x7f155c006d90 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.581+0000 7f154d7fa700 1 -- 172.21.15.44:0/1852573499 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156000e040 con 0x5641f00371a0 2023-09-10T01:34:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.581+0000 7f154d7fa700 1 -- 172.21.15.44:0/1852573499 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1560003e90 con 0x5641f00371a0 2023-09-10T01:34:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.581+0000 7f154d7fa700 1 -- 172.21.15.44:0/1852573499 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15600053e0 con 0x5641f00371a0 2023-09-10T01:34:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.582+0000 7f1579570b80 1 -- 172.21.15.44:0/1852573499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f00371a0 msgr2=0x5641f0037580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.582+0000 7f1579570b80 1 --2- 172.21.15.44:0/1852573499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f00371a0 0x5641f0037580 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f156000b070 tx=0x7f1560005ad0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.582+0000 7f1579570b80 1 -- 172.21.15.44:0/1852573499 shutdown_connections 2023-09-10T01:34:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.582+0000 7f1579570b80 1 --2- 172.21.15.44:0/1852573499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031cc0 0x5641f003df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.582+0000 7f1579570b80 1 --2- 172.21.15.44:0/1852573499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f00371a0 0x5641f0037580 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.582+0000 7f1579570b80 1 --2- 172.21.15.44:0/1852573499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f0031300 0x5641f0031780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.582+0000 7f1579570b80 1 -- 172.21.15.44:0/1852573499 >> 172.21.15.44:0/1852573499 conn(0x5641effa6030 msgr2=0x5641f002c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.585+0000 7f1579570b80 1 -- 172.21.15.44:0/1852573499 shutdown_connections 2023-09-10T01:34:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.586+0000 7f1579570b80 1 -- 172.21.15.44:0/1852573499 wait complete. 2023-09-10T01:34:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.586+0000 7f1579570b80 1 -- start start 2023-09-10T01:34:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.586+0000 7f1579570b80 1 Processor -- start 2023-09-10T01:34:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.587+0000 7f1579570b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f0031300 0x5641f003dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.587+0000 7f1579570b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031cc0 0x5641f003b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.587+0000 7f1579570b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f00371a0 0x5641f003b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.587+0000 7f1579570b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641eff21b10 con 0x5641f0031cc0 2023-09-10T01:34:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.587+0000 7f1579570b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641eff21810 con 0x5641f00371a0 2023-09-10T01:34:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.587+0000 7f1579570b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641efe058c0 con 0x5641f0031300 2023-09-10T01:34:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.587+0000 7f1579570b80 10 client.0 _ll_register_callbacks cb 0x5641efe6ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:34:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.587+0000 7f1566ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f0031300 0x5641f003dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.587+0000 7f1566ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f0031300 0x5641f003dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39722/0 (socket says 172.21.15.44:39722) 2023-09-10T01:34:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.587+0000 7f1566ffd700 1 -- 172.21.15.44:0/50252320 learned_addr learned my addr 172.21.15.44:0/50252320 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.587+0000 7f15667fc700 1 --2- 172.21.15.44:0/50252320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031cc0 0x5641f003b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.587+0000 7f15677fe700 1 --2- 172.21.15.44:0/50252320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f00371a0 0x5641f003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.588+0000 7f15677fe700 1 -- 172.21.15.44:0/50252320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f0031300 msgr2=0x5641f003dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.588+0000 7f15677fe700 1 --2- 172.21.15.44:0/50252320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f0031300 0x5641f003dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.588+0000 7f15677fe700 1 -- 172.21.15.44:0/50252320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031cc0 msgr2=0x5641f003b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.588+0000 7f15677fe700 1 --2- 172.21.15.44:0/50252320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031cc0 0x5641f003b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.588+0000 7f15677fe700 1 -- 172.21.15.44:0/50252320 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156810dd30 con 0x5641f00371a0 2023-09-10T01:34:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.590+0000 7f15677fe700 1 --2- 172.21.15.44:0/50252320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f00371a0 0x5641f003b610 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f156000b0a0 tx=0x7f15600094e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dee196e9dd89e72 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.591+0000 7f154aff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:34:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.591+0000 7f154aff5700 1 -- 172.21.15.44:0/50252320 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156000e040 con 0x5641f00371a0 2023-09-10T01:34:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.591+0000 7f154aff5700 1 -- 172.21.15.44:0/50252320 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(207..207 src has 1..207) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f15600052d0 con 0x5641f00371a0 2023-09-10T01:34:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.591+0000 7f1579570b80 1 -- 172.21.15.44:0/50252320 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641efedb490 con 0x5641f00371a0 2023-09-10T01:34:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.591+0000 7f154aff5700 1 -- 172.21.15.44:0/50252320 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=193) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1560005490 con 0x5641f00371a0 2023-09-10T01:34:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.591+0000 7f1579570b80 10 client.30287 fetch_fsmap learned FSMap version 193 2023-09-10T01:34:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.591+0000 7f1579570b80 1 -- 172.21.15.44:0/50252320 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=193,monmap=4+}) v3 -- 0x5641efd98c00 con 0x5641f00371a0 2023-09-10T01:34:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.591+0000 7f154aff5700 1 -- 172.21.15.44:0/50252320 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5641efedb490 con 0x5641f00371a0 2023-09-10T01:34:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.591+0000 7f154aff5700 1 -- 172.21.15.44:0/50252320 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 193) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f1560019070 con 0x5641f00371a0 2023-09-10T01:34:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.592+0000 7f1579570b80 10 client.30287 fetch_fsmap finished waiting for FSMap version 193 2023-09-10T01:34:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.592+0000 7f1579570b80 10 client.30287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:34:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.592+0000 7f1579570b80 20 client.30287 populate_metadata read hostname 'smithi044' 2023-09-10T01:34:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.592+0000 7f1579570b80 1 --2- 172.21.15.44:0/50252320 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5641f005c0a0 0x5641f005e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.592+0000 7f1579570b80 4 client.30287 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:34:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.592+0000 7f1579570b80 1 -- 172.21.15.44:0/50252320 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641eff98840 con 0x5641f005c0a0 2023-09-10T01:34:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.592+0000 7f1566ffd700 1 --2- 172.21.15.44:0/50252320 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5641f005c0a0 0x5641f005e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.592+0000 7f154aff5700 10 client.30287 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.592+0000 7f1566ffd700 1 --2- 172.21.15.44:0/50252320 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5641f005c0a0 0x5641f005e4c0 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70d03c56ab137077 server_cookie=a92d615f9ff4bed4 in_seq=0 out_seq=0 2023-09-10T01:34:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.593+0000 7f154aff5700 1 -- 172.21.15.44:0/50252320 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5641eff98840 con 0x5641f005c0a0 2023-09-10T01:34:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.593+0000 7f154aff5700 10 client.30287 handle_command_reply: tid=0 2023-09-10T01:34:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.593+0000 7f1579570b80 1 client.30287 shutdown 2023-09-10T01:34:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.593+0000 7f1579570b80 1 -- 172.21.15.44:0/50252320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f00371a0 msgr2=0x5641f003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.593+0000 7f1579570b80 1 --2- 172.21.15.44:0/50252320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f00371a0 0x5641f003b610 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f156000b0a0 tx=0x7f15600094e0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.593+0000 7f1579570b80 1 -- 172.21.15.44:0/50252320 shutdown_connections 2023-09-10T01:34:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.593+0000 7f1579570b80 1 --2- 172.21.15.44:0/50252320 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5641f005c0a0 0x5641f005e4c0 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.593+0000 7f1579570b80 1 --2- 172.21.15.44:0/50252320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f00371a0 0x5641f003b610 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.593+0000 7f1579570b80 1 --2- 172.21.15.44:0/50252320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f0031300 0x5641f003dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.593+0000 7f154aff5700 0 client.30287 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.593+0000 7f1579570b80 1 --2- 172.21.15.44:0/50252320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031cc0 0x5641f003b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.593+0000 7f1579570b80 1 -- 172.21.15.44:0/50252320 >> 172.21.15.44:0/50252320 conn(0x5641effa6030 msgr2=0x5641f002b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.593+0000 7f1579570b80 1 -- 172.21.15.44:0/50252320 shutdown_connections 2023-09-10T01:34:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.593+0000 7f1579570b80 1 -- 172.21.15.44:0/50252320 wait complete. 2023-09-10T01:34:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.593+0000 7f1579570b80 20 client.30287 trim_cache size 0 max 16384 2023-09-10T01:34:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.621+0000 7f1579570b80 1 Processor -- start 2023-09-10T01:34:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.621+0000 7f1579570b80 1 -- start start 2023-09-10T01:34:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.621+0000 7f1579570b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031300 0x5641f003a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.621+0000 7f1579570b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f0031cc0 0x5641f003d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.621+0000 7f1579570b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f00371a0 0x5641f002c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.621+0000 7f1579570b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641f0028fc0 con 0x5641f0031300 2023-09-10T01:34:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.621+0000 7f1579570b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641f0032410 con 0x5641f0031cc0 2023-09-10T01:34:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.621+0000 7f1579570b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641f0039640 con 0x5641f00371a0 2023-09-10T01:34:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.622+0000 7f15667fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f0031cc0 0x5641f003d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.622+0000 7f15677fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f00371a0 0x5641f002c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.622+0000 7f15677fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f00371a0 0x5641f002c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39730/0 (socket says 172.21.15.44:39730) 2023-09-10T01:34:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.622+0000 7f15677fe700 1 -- 172.21.15.44:0/1908242576 learned_addr learned my addr 172.21.15.44:0/1908242576 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.622+0000 7f15677fe700 1 -- 172.21.15.44:0/1908242576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f0031cc0 msgr2=0x5641f003d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.622+0000 7f15677fe700 1 --2- 172.21.15.44:0/1908242576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f0031cc0 0x5641f003d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.622+0000 7f15677fe700 1 -- 172.21.15.44:0/1908242576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031300 msgr2=0x5641f003a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.622+0000 7f15677fe700 1 --2- 172.21.15.44:0/1908242576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031300 0x5641f003a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.622+0000 7f15677fe700 1 -- 172.21.15.44:0/1908242576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641efdd61f0 con 0x5641f00371a0 2023-09-10T01:34:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.622+0000 7f15677fe700 1 --2- 172.21.15.44:0/1908242576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f00371a0 0x5641f002c090 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f156001dc60 tx=0x7f15600099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b06612e2392ddd75 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.622+0000 7f154dffb700 1 -- 172.21.15.44:0/1908242576 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156000e070 con 0x5641f00371a0 2023-09-10T01:34:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.622+0000 7f154dffb700 1 -- 172.21.15.44:0/1908242576 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1560003a00 con 0x5641f00371a0 2023-09-10T01:34:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.622+0000 7f154dffb700 1 -- 172.21.15.44:0/1908242576 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156001bb70 con 0x5641f00371a0 2023-09-10T01:34:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.623+0000 7f1579570b80 1 -- 172.21.15.44:0/1908242576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f00371a0 msgr2=0x5641f002c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.623+0000 7f1579570b80 1 --2- 172.21.15.44:0/1908242576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f00371a0 0x5641f002c090 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f156001dc60 tx=0x7f15600099e0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.625+0000 7f1579570b80 1 -- 172.21.15.44:0/1908242576 shutdown_connections 2023-09-10T01:34:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.625+0000 7f1579570b80 1 --2- 172.21.15.44:0/1908242576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f00371a0 0x5641f002c090 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.625+0000 7f1579570b80 1 --2- 172.21.15.44:0/1908242576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031300 0x5641f003a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.625+0000 7f1579570b80 1 --2- 172.21.15.44:0/1908242576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f0031cc0 0x5641f003d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.625+0000 7f1579570b80 1 -- 172.21.15.44:0/1908242576 >> 172.21.15.44:0/1908242576 conn(0x5641effa6030 msgr2=0x5641f0036080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.625+0000 7f1579570b80 1 -- 172.21.15.44:0/1908242576 shutdown_connections 2023-09-10T01:34:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.625+0000 7f1579570b80 1 -- 172.21.15.44:0/1908242576 wait complete. 2023-09-10T01:34:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.625+0000 7f1579570b80 1 -- start start 2023-09-10T01:34:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.625+0000 7f1579570b80 1 Processor -- start 2023-09-10T01:34:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.626+0000 7f1579570b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031300 0x5641f00ab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.626+0000 7f1579570b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f0031cc0 0x5641f00ab810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.626+0000 7f1579570b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f00371a0 0x5641f0051320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.626+0000 7f1579570b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641f0051a30 con 0x5641f0031300 2023-09-10T01:34:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.626+0000 7f1579570b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641f0051ba0 con 0x5641f00371a0 2023-09-10T01:34:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.626+0000 7f1579570b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641f0051d60 con 0x5641f0031cc0 2023-09-10T01:34:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.626+0000 7f1579570b80 10 client.0 _ll_register_callbacks cb 0x7f1560007a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:34:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.635+0000 7f1566ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031300 0x5641f00ab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.635+0000 7f1566ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031300 0x5641f00ab2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32852/0 (socket says 172.21.15.44:32852) 2023-09-10T01:34:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.635+0000 7f1566ffd700 1 -- 172.21.15.44:0/4098916990 learned_addr learned my addr 172.21.15.44:0/4098916990 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.635+0000 7f15667fc700 1 --2- 172.21.15.44:0/4098916990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f0031cc0 0x5641f00ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.635+0000 7f15677fe700 1 --2- 172.21.15.44:0/4098916990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f00371a0 0x5641f0051320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.635+0000 7f15667fc700 1 -- 172.21.15.44:0/4098916990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f00371a0 msgr2=0x5641f0051320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.635+0000 7f15667fc700 1 --2- 172.21.15.44:0/4098916990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f00371a0 0x5641f0051320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.635+0000 7f15667fc700 1 -- 172.21.15.44:0/4098916990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031300 msgr2=0x5641f00ab2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.635+0000 7f15667fc700 1 --2- 172.21.15.44:0/4098916990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031300 0x5641f00ab2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.635+0000 7f15667fc700 1 -- 172.21.15.44:0/4098916990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641efd98c00 con 0x5641f0031cc0 2023-09-10T01:34:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.637+0000 7f15667fc700 1 --2- 172.21.15.44:0/4098916990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f0031cc0 0x5641f00ab810 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f15580cb2e0 tx=0x7f155809ac00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f9fa98e17ecb9d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.637+0000 7f154aff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:34:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.637+0000 7f154aff5700 1 -- 172.21.15.44:0/4098916990 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580b2070 con 0x5641f0031cc0 2023-09-10T01:34:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.637+0000 7f154aff5700 1 -- 172.21.15.44:0/4098916990 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(207..207 src has 1..207) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f1558011c40 con 0x5641f0031cc0 2023-09-10T01:34:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.637+0000 7f1579570b80 1 -- 172.21.15.44:0/4098916990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641eff7c8c0 con 0x5641f0031cc0 2023-09-10T01:34:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.637+0000 7f154aff5700 1 -- 172.21.15.44:0/4098916990 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=193) v2 ==== 24+0+0 (secure 0 0 0) 0x7f155809b740 con 0x5641f0031cc0 2023-09-10T01:34:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.637+0000 7f1579570b80 10 client.30340 fetch_fsmap learned FSMap version 193 2023-09-10T01:34:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.637+0000 7f1579570b80 1 -- 172.21.15.44:0/4098916990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=193,monmap=4+}) v3 -- 0x5641efdae250 con 0x5641f0031cc0 2023-09-10T01:34:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.637+0000 7f154aff5700 1 -- 172.21.15.44:0/4098916990 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5641eff7c8c0 con 0x5641f0031cc0 2023-09-10T01:34:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.638+0000 7f154aff5700 1 -- 172.21.15.44:0/4098916990 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 193) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f15580b3260 con 0x5641f0031cc0 2023-09-10T01:34:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.638+0000 7f1579570b80 10 client.30340 fetch_fsmap finished waiting for FSMap version 193 2023-09-10T01:34:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.638+0000 7f1579570b80 10 client.30340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:34:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.638+0000 7f1579570b80 20 client.30340 populate_metadata read hostname 'smithi044' 2023-09-10T01:34:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.638+0000 7f1579570b80 1 --2- 172.21.15.44:0/4098916990 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5641f00c2040 0x5641f00c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.638+0000 7f1579570b80 4 client.30340 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:34:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.638+0000 7f1579570b80 1 -- 172.21.15.44:0/4098916990 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5641eff3d050 con 0x5641f00c2040 2023-09-10T01:34:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.638+0000 7f1566ffd700 1 --2- 172.21.15.44:0/4098916990 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5641f00c2040 0x5641f00c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.638+0000 7f154aff5700 10 client.30340 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.638+0000 7f1566ffd700 1 --2- 172.21.15.44:0/4098916990 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5641f00c2040 0x5641f00c4460 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1ec02ba93a1f349 server_cookie=12dbb3b760792e8a in_seq=0 out_seq=0 2023-09-10T01:34:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.714+0000 7f154aff5700 1 -- 172.21.15.44:0/4098916990 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5641eff3d050 con 0x5641f00c2040 2023-09-10T01:34:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.714+0000 7f154aff5700 10 client.30340 handle_command_reply: tid=0 2023-09-10T01:34:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.714+0000 7f1579570b80 1 client.30340 shutdown 2023-09-10T01:34:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.715+0000 7f1579570b80 1 -- 172.21.15.44:0/4098916990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f0031cc0 msgr2=0x5641f00ab810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.715+0000 7f1579570b80 1 --2- 172.21.15.44:0/4098916990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f0031cc0 0x5641f00ab810 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f15580cb2e0 tx=0x7f155809ac00 comp rx=0 tx=0).stop 2023-09-10T01:34:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.715+0000 7f1579570b80 1 -- 172.21.15.44:0/4098916990 shutdown_connections 2023-09-10T01:34:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.715+0000 7f1579570b80 1 --2- 172.21.15.44:0/4098916990 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5641f00c2040 0x5641f00c4460 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.715+0000 7f1579570b80 1 --2- 172.21.15.44:0/4098916990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f00371a0 0x5641f0051320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.715+0000 7f1579570b80 1 --2- 172.21.15.44:0/4098916990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641f0031300 0x5641f00ab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.715+0000 7f1579570b80 1 --2- 172.21.15.44:0/4098916990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641f0031cc0 0x5641f00ab810 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.715+0000 7f1579570b80 1 -- 172.21.15.44:0/4098916990 >> 172.21.15.44:0/4098916990 conn(0x5641effa6030 msgr2=0x5641f00302f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.715+0000 7f154aff5700 0 client.30340 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.716+0000 7f1579570b80 1 -- 172.21.15.44:0/4098916990 shutdown_connections 2023-09-10T01:34:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.716+0000 7f1579570b80 1 -- 172.21.15.44:0/4098916990 wait complete. 2023-09-10T01:34:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.716+0000 7f1579570b80 20 client.30340 trim_cache size 0 max 16384 2023-09-10T01:34:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:34:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:34:42.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:34:42.722 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:34:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.720+0000 7f156d3aa700 1 -- 172.21.15.44:0/2543069168 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f15500a08e0 msgr2=0x7f15500a2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.720+0000 7f156d3aa700 1 --2- 172.21.15.44:0/2543069168 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f15500a08e0 0x7f15500a2da0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x5641f002c060 tx=0x7f155c006d90 comp rx=0 tx=0).stop 2023-09-10T01:34:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.720+0000 7f156d3aa700 1 -- 172.21.15.44:0/2543069168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1568100f70 msgr2=0x7f156819d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.720+0000 7f156d3aa700 1 --2- 172.21.15.44:0/2543069168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1568100f70 0x7f156819d200 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f155800b530 tx=0x7f155800b8f0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.720+0000 7f156d3aa700 1 -- 172.21.15.44:0/2543069168 shutdown_connections 2023-09-10T01:34:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.720+0000 7f156d3aa700 1 --2- 172.21.15.44:0/2543069168 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f15500a08e0 0x7f15500a2da0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.720+0000 7f156d3aa700 1 --2- 172.21.15.44:0/2543069168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1568106670 0x7f15681a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.720+0000 7f156d3aa700 1 --2- 172.21.15.44:0/2543069168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15681005b0 0x7f156819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.720+0000 7f156d3aa700 1 --2- 172.21.15.44:0/2543069168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1568100f70 0x7f156819d200 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.720+0000 7f156d3aa700 1 -- 172.21.15.44:0/2543069168 >> 172.21.15.44:0/2543069168 conn(0x7f15680fc060 msgr2=0x7f15680fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.721+0000 7f156d3aa700 1 -- 172.21.15.44:0/2543069168 shutdown_connections 2023-09-10T01:34:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:42.721+0000 7f156d3aa700 1 -- 172.21.15.44:0/2543069168 wait complete. 2023-09-10T01:34:44.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:43 smithi131 ceph-mon[103713]: pgmap v1762: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.4 MiB/s wr, 1.04k op/s 2023-09-10T01:34:44.311 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:43 smithi044 ceph-mon[98676]: pgmap v1762: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.4 MiB/s wr, 1.04k op/s 2023-09-10T01:34:44.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:43 smithi099 ceph-mon[102692]: pgmap v1762: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.4 MiB/s wr, 1.04k op/s 2023-09-10T01:34:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 820.0 s, 309.6 tps, lat 318.910 ms stddev 140.506 2023-09-10T01:34:46.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:45 smithi131 ceph-mon[103713]: pgmap v1763: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.2 MiB/s wr, 825 op/s 2023-09-10T01:34:46.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:45 smithi044 ceph-mon[98676]: pgmap v1763: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.2 MiB/s wr, 825 op/s 2023-09-10T01:34:46.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:45 smithi099 ceph-mon[102692]: pgmap v1763: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.2 MiB/s wr, 825 op/s 2023-09-10T01:34:48.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:47 smithi131 ceph-mon[103713]: pgmap v1764: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.5 MiB/s wr, 831 op/s 2023-09-10T01:34:48.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:47 smithi044 ceph-mon[98676]: pgmap v1764: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.5 MiB/s wr, 831 op/s 2023-09-10T01:34:48.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:47 smithi099 ceph-mon[102692]: pgmap v1764: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.5 MiB/s wr, 831 op/s 2023-09-10T01:34:50.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:49 smithi131 ceph-mon[103713]: pgmap v1765: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.3 MiB/s wr, 993 op/s 2023-09-10T01:34:50.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:49 smithi044 ceph-mon[98676]: pgmap v1765: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.3 MiB/s wr, 993 op/s 2023-09-10T01:34:50.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:49 smithi099 ceph-mon[102692]: pgmap v1765: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.3 MiB/s wr, 993 op/s 2023-09-10T01:34:51.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 825.0 s, 293.8 tps, lat 330.103 ms stddev 107.468 2023-09-10T01:34:52.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:51 smithi131 ceph-mon[103713]: pgmap v1766: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 811 op/s 2023-09-10T01:34:52.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:51 smithi044 ceph-mon[98676]: pgmap v1766: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 811 op/s 2023-09-10T01:34:52.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:51 smithi099 ceph-mon[102692]: pgmap v1766: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 811 op/s 2023-09-10T01:34:54.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:53 smithi131 ceph-mon[103713]: pgmap v1767: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.1 MiB/s wr, 1k op/s 2023-09-10T01:34:54.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:53 smithi044 ceph-mon[98676]: pgmap v1767: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.1 MiB/s wr, 1k op/s 2023-09-10T01:34:54.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:53 smithi099 ceph-mon[102692]: pgmap v1767: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.1 MiB/s wr, 1k op/s 2023-09-10T01:34:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.974+0000 7f9c06550700 1 -- 172.21.15.44:0/769510035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c0010bf10 msgr2=0x7f9c000788f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.974+0000 7f9c06550700 1 --2- 172.21.15.44:0/769510035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c0010bf10 0x7f9c000788f0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80080b0 tx=0x7f9bf800d650 comp rx=0 tx=0).stop 2023-09-10T01:34:55.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.979+0000 7f9c06550700 1 -- 172.21.15.44:0/769510035 shutdown_connections 2023-09-10T01:34:55.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.979+0000 7f9c06550700 1 --2- 172.21.15.44:0/769510035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c0010bf10 0x7f9c000788f0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:55.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.979+0000 7f9c06550700 1 --2- 172.21.15.44:0/769510035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c00107870 0x7f9c00107c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:55.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.979+0000 7f9c06550700 1 --2- 172.21.15.44:0/769510035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c00108220 0x7f9c0010b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.979+0000 7f9c06550700 1 -- 172.21.15.44:0/769510035 >> 172.21.15.44:0/769510035 conn(0x7f9c0006c6c0 msgr2=0x7f9c0006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.979+0000 7f9c06550700 1 -- 172.21.15.44:0/769510035 shutdown_connections 2023-09-10T01:34:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.979+0000 7f9c06550700 1 -- 172.21.15.44:0/769510035 wait complete. 2023-09-10T01:34:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.980+0000 7f9c06550700 1 Processor -- start 2023-09-10T01:34:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.980+0000 7f9c06550700 1 -- start start 2023-09-10T01:34:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.980+0000 7f9c06550700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c00107870 0x7f9c00082dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.980+0000 7f9c06550700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c0010bf10 0x7f9c0007cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.980+0000 7f9c06550700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c00083420 0x7f9c0007d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.980+0000 7f9c06550700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0007d900 con 0x7f9c00083420 2023-09-10T01:34:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.980+0000 7f9c06550700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c001ba790 con 0x7f9c00107870 2023-09-10T01:34:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.980+0000 7f9c06550700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c001ba8d0 con 0x7f9c0010bf10 2023-09-10T01:34:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.980+0000 7f9bfffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c00107870 0x7f9c00082dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.981+0000 7f9bff7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c0010bf10 0x7f9c0007cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.981+0000 7f9bff7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c0010bf10 0x7f9c0007cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39750/0 (socket says 172.21.15.44:39750) 2023-09-10T01:34:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.981+0000 7f9bff7fe700 1 -- 172.21.15.44:0/2947277584 learned_addr learned my addr 172.21.15.44:0/2947277584 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.981+0000 7f9c04aed700 1 --2- 172.21.15.44:0/2947277584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c00083420 0x7f9c0007d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.981+0000 7f9bff7fe700 1 -- 172.21.15.44:0/2947277584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c00107870 msgr2=0x7f9c00082dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.981+0000 7f9bff7fe700 1 --2- 172.21.15.44:0/2947277584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c00107870 0x7f9c00082dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.981+0000 7f9bff7fe700 1 -- 172.21.15.44:0/2947277584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c00083420 msgr2=0x7f9c0007d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.981+0000 7f9bff7fe700 1 --2- 172.21.15.44:0/2947277584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c00083420 0x7f9c0007d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.981+0000 7f9bff7fe700 1 -- 172.21.15.44:0/2947277584 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf8007ce0 con 0x7f9c0010bf10 2023-09-10T01:34:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.981+0000 7f9bff7fe700 1 --2- 172.21.15.44:0/2947277584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c0010bf10 0x7f9c0007cb50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400b500 tx=0x7f9bf400b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.982+0000 7f9bfd7fa700 1 -- 172.21.15.44:0/2947277584 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4011830 con 0x7f9c0010bf10 2023-09-10T01:34:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.982+0000 7f9c06550700 1 -- 172.21.15.44:0/2947277584 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c001bab60 con 0x7f9c0010bf10 2023-09-10T01:34:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.982+0000 7f9c06550700 1 -- 172.21.15.44:0/2947277584 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c001bb0c0 con 0x7f9c0010bf10 2023-09-10T01:34:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.982+0000 7f9bfd7fa700 1 -- 172.21.15.44:0/2947277584 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9bf4011e20 con 0x7f9c0010bf10 2023-09-10T01:34:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.982+0000 7f9bfd7fa700 1 -- 172.21.15.44:0/2947277584 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4010db0 con 0x7f9c0010bf10 2023-09-10T01:34:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.987+0000 7f9c12716b80 1 Processor -- start 2023-09-10T01:34:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.987+0000 7f9c12716b80 1 -- start start 2023-09-10T01:34:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.987+0000 7f9c12716b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2ab01a0 0x5589d2ab0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.987+0000 7f9c12716b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaa300 0x5589d2aaa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.987+0000 7f9c12716b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2aaacc0 0x5589d2ab6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.987+0000 7f9c12716b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d2a036d0 con 0x5589d2aaacc0 2023-09-10T01:34:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.987+0000 7f9c12716b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d2a00ff0 con 0x5589d2aaa300 2023-09-10T01:34:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.987+0000 7f9c12716b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d29fc8e0 con 0x5589d2ab01a0 2023-09-10T01:34:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.987+0000 7f9c04aed700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2ab01a0 0x5589d2ab0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.987+0000 7f9c04aed700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2ab01a0 0x5589d2ab0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39754/0 (socket says 172.21.15.44:39754) 2023-09-10T01:34:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.987+0000 7f9c04aed700 1 -- 172.21.15.44:0/1483501494 learned_addr learned my addr 172.21.15.44:0/1483501494 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.988+0000 7f9c04aed700 1 -- 172.21.15.44:0/1483501494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaa300 msgr2=0x5589d2aaa780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.988+0000 7f9c04aed700 1 --2- 172.21.15.44:0/1483501494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaa300 0x5589d2aaa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.988+0000 7f9c04aed700 1 -- 172.21.15.44:0/1483501494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2aaacc0 msgr2=0x5589d2ab6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.988+0000 7f9c04aed700 1 --2- 172.21.15.44:0/1483501494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2aaacc0 0x5589d2ab6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.988+0000 7f9c04aed700 1 -- 172.21.15.44:0/1483501494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589d2828d80 con 0x5589d2ab01a0 2023-09-10T01:34:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.988+0000 7f9c04aed700 1 --2- 172.21.15.44:0/1483501494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2ab01a0 0x5589d2ab0580 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8017000 tx=0x7f9bf800f590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6da0f0138b6118d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.990+0000 7f9be67fc700 1 -- 172.21.15.44:0/1483501494 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf800f760 con 0x5589d2ab01a0 2023-09-10T01:34:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.990+0000 7f9be67fc700 1 -- 172.21.15.44:0/1483501494 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9bf8003a00 con 0x5589d2ab01a0 2023-09-10T01:34:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.990+0000 7f9be67fc700 1 -- 172.21.15.44:0/1483501494 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8018480 con 0x5589d2ab01a0 2023-09-10T01:34:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.995+0000 7f9bfd7fa700 1 -- 172.21.15.44:0/2947277584 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f9bf4011990 con 0x7f9c0010bf10 2023-09-10T01:34:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.996+0000 7f9bfd7fa700 1 --2- 172.21.15.44:0/2947277584 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9be80a0900 0x7f9be80a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.996+0000 7f9bfd7fa700 1 -- 172.21.15.44:0/2947277584 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f9bf40cddf0 con 0x7f9c0010bf10 2023-09-10T01:34:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.996+0000 7f9bfffff700 1 --2- 172.21.15.44:0/2947277584 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9be80a0900 0x7f9be80a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.997+0000 7f9c12716b80 1 -- 172.21.15.44:0/1483501494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2ab01a0 msgr2=0x5589d2ab0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.997+0000 7f9c12716b80 1 --2- 172.21.15.44:0/1483501494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2ab01a0 0x5589d2ab0580 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8017000 tx=0x7f9bf800f590 comp rx=0 tx=0).stop 2023-09-10T01:34:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.997+0000 7f9bfffff700 1 --2- 172.21.15.44:0/2947277584 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9be80a0900 0x7f9be80a2dc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x5589d2aa5060 tx=0x7f9bf0008040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.998+0000 7f9c12716b80 1 -- 172.21.15.44:0/1483501494 shutdown_connections 2023-09-10T01:34:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.998+0000 7f9c12716b80 1 --2- 172.21.15.44:0/1483501494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2aaacc0 0x5589d2ab6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.998+0000 7f9c12716b80 1 --2- 172.21.15.44:0/1483501494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2ab01a0 0x5589d2ab0580 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.998+0000 7f9c12716b80 1 --2- 172.21.15.44:0/1483501494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaa300 0x5589d2aaa780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.998+0000 7f9c12716b80 1 -- 172.21.15.44:0/1483501494 >> 172.21.15.44:0/1483501494 conn(0x5589d2a1f030 msgr2=0x5589d2aa5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.998+0000 7f9c12716b80 1 -- 172.21.15.44:0/1483501494 shutdown_connections 2023-09-10T01:34:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.998+0000 7f9c12716b80 1 -- 172.21.15.44:0/1483501494 wait complete. 2023-09-10T01:34:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.999+0000 7f9c12716b80 1 -- start start 2023-09-10T01:34:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.999+0000 7f9c12716b80 1 Processor -- start 2023-09-10T01:34:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.999+0000 7f9c12716b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaa300 0x5589d2ab6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.999+0000 7f9c12716b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2aaacc0 0x5589d2ab4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.999+0000 7f9c12716b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2ab01a0 0x5589d2ab45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.999+0000 7f9c12716b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d299ab10 con 0x5589d2ab01a0 2023-09-10T01:34:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.999+0000 7f9c12716b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d299a810 con 0x5589d2aaa300 2023-09-10T01:34:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.999+0000 7f9c12716b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d287e8c0 con 0x5589d2aaacc0 2023-09-10T01:34:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:55.999+0000 7f9c12716b80 10 client.0 _ll_register_callbacks cb 0x5589d28e4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:34:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.001+0000 7f9bff7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2aaacc0 0x5589d2ab4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.001+0000 7f9bff7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2aaacc0 0x5589d2ab4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39764/0 (socket says 172.21.15.44:39764) 2023-09-10T01:34:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.001+0000 7f9bff7fe700 1 -- 172.21.15.44:0/3805789092 learned_addr learned my addr 172.21.15.44:0/3805789092 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.005+0000 7f9bfffff700 1 --2- 172.21.15.44:0/3805789092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaa300 0x5589d2ab6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.006+0000 7f9bfffff700 1 -- 172.21.15.44:0/3805789092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2aaacc0 msgr2=0x5589d2ab4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.006+0000 7f9bfffff700 1 --2- 172.21.15.44:0/3805789092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2aaacc0 0x5589d2ab4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.006+0000 7f9bfffff700 1 -- 172.21.15.44:0/3805789092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2ab01a0 msgr2=0x5589d2ab45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.006+0000 7f9bfffff700 1 --2- 172.21.15.44:0/3805789092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2ab01a0 0x5589d2ab45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.006+0000 7f9bfffff700 1 -- 172.21.15.44:0/3805789092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c000791b0 con 0x5589d2aaa300 2023-09-10T01:34:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.007+0000 7f9bfffff700 1 --2- 172.21.15.44:0/3805789092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaa300 0x5589d2ab6df0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000d440 tx=0x7f9bf0004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6d17b878ac6d462 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.007+0000 7f9bdf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:34:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.007+0000 7f9bdf7fe700 1 -- 172.21.15.44:0/3805789092 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00149a0 con 0x5589d2aaa300 2023-09-10T01:34:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.007+0000 7f9bdf7fe700 1 -- 172.21.15.44:0/3805789092 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(207..207 src has 1..207) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f9bf0014b00 con 0x5589d2aaa300 2023-09-10T01:34:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.011+0000 7f9c12716b80 1 -- 172.21.15.44:0/3805789092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589d2954490 con 0x5589d2aaa300 2023-09-10T01:34:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.013+0000 7f9bdf7fe700 1 -- 172.21.15.44:0/3805789092 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=193) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bf0014cc0 con 0x5589d2aaa300 2023-09-10T01:34:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.013+0000 7f9c12716b80 10 client.30296 fetch_fsmap learned FSMap version 193 2023-09-10T01:34:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.013+0000 7f9c12716b80 1 -- 172.21.15.44:0/3805789092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=193,monmap=4+}) v3 -- 0x5589d2811c00 con 0x5589d2aaa300 2023-09-10T01:34:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.014+0000 7f9bdf7fe700 1 -- 172.21.15.44:0/3805789092 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5589d2954490 con 0x5589d2aaa300 2023-09-10T01:34:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.014+0000 7f9bdf7fe700 1 -- 172.21.15.44:0/3805789092 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 193) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f9bf0027070 con 0x5589d2aaa300 2023-09-10T01:34:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.014+0000 7f9c12716b80 10 client.30296 fetch_fsmap finished waiting for FSMap version 193 2023-09-10T01:34:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.014+0000 7f9c12716b80 10 client.30296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:34:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.014+0000 7f9c12716b80 20 client.30296 populate_metadata read hostname 'smithi044' 2023-09-10T01:34:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.014+0000 7f9c12716b80 1 --2- 172.21.15.44:0/3805789092 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5589d2ab9e90 0x5589d2abc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.014+0000 7f9c12716b80 4 client.30296 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:34:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.014+0000 7f9c12716b80 1 -- 172.21.15.44:0/3805789092 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589d2a11840 con 0x5589d2ab9e90 2023-09-10T01:34:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.014+0000 7f9bff7fe700 1 --2- 172.21.15.44:0/3805789092 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5589d2ab9e90 0x5589d2abc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.015+0000 7f9bff7fe700 1 --2- 172.21.15.44:0/3805789092 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5589d2ab9e90 0x5589d2abc2b0 crc :-1 s=READY pgs=608 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57d55dab129cf51f server_cookie=1fc2cd535d6176de in_seq=0 out_seq=0 2023-09-10T01:34:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.015+0000 7f9bdf7fe700 10 client.30296 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.020+0000 7f9bdf7fe700 1 -- 172.21.15.44:0/3805789092 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5589d2a11840 con 0x5589d2ab9e90 2023-09-10T01:34:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.020+0000 7f9bdf7fe700 10 client.30296 handle_command_reply: tid=0 2023-09-10T01:34:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.020+0000 7f9c12716b80 1 client.30296 shutdown 2023-09-10T01:34:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.020+0000 7f9c12716b80 1 -- 172.21.15.44:0/3805789092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaa300 msgr2=0x5589d2ab6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.020+0000 7f9c12716b80 1 --2- 172.21.15.44:0/3805789092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaa300 0x5589d2ab6df0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000d440 tx=0x7f9bf0004770 comp rx=0 tx=0).stop 2023-09-10T01:34:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.026+0000 7f9c12716b80 1 -- 172.21.15.44:0/3805789092 shutdown_connections 2023-09-10T01:34:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.026+0000 7f9c12716b80 1 --2- 172.21.15.44:0/3805789092 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5589d2ab9e90 0x5589d2abc2b0 crc :-1 s=READY pgs=608 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.026+0000 7f9c12716b80 1 --2- 172.21.15.44:0/3805789092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2ab01a0 0x5589d2ab45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.026+0000 7f9c12716b80 1 --2- 172.21.15.44:0/3805789092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaa300 0x5589d2ab6df0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.026+0000 7f9c12716b80 1 --2- 172.21.15.44:0/3805789092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2aaacc0 0x5589d2ab4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.026+0000 7f9c12716b80 1 -- 172.21.15.44:0/3805789092 >> 172.21.15.44:0/3805789092 conn(0x5589d2a1f030 msgr2=0x5589d2aa47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.026+0000 7f9bdf7fe700 0 client.30296 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.026+0000 7f9c12716b80 1 -- 172.21.15.44:0/3805789092 shutdown_connections 2023-09-10T01:34:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.026+0000 7f9c12716b80 1 -- 172.21.15.44:0/3805789092 wait complete. 2023-09-10T01:34:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.027+0000 7f9c12716b80 20 client.30296 trim_cache size 0 max 16384 2023-09-10T01:34:56.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:55 smithi131 ceph-mon[103713]: pgmap v1768: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.9 MiB/s wr, 783 op/s 2023-09-10T01:34:56.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:55 smithi044 ceph-mon[98676]: pgmap v1768: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.9 MiB/s wr, 783 op/s 2023-09-10T01:34:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.055+0000 7f9c12716b80 1 Processor -- start 2023-09-10T01:34:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.059+0000 7f9c12716b80 1 -- start start 2023-09-10T01:34:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.059+0000 7f9c12716b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaa300 0x5589d2ac9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.059+0000 7f9c12716b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2aaacc0 0x5589d2ab6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.059+0000 7f9c12716b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2ab01a0 0x5589d2aa5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.059+0000 7f9c12716b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d2ab4ba0 con 0x5589d2ab01a0 2023-09-10T01:34:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.059+0000 7f9c12716b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d2aab1b0 con 0x5589d2aaa300 2023-09-10T01:34:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.059+0000 7f9c12716b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d2ac9fd0 con 0x5589d2aaacc0 2023-09-10T01:34:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.062+0000 7f9bff7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2aaacc0 0x5589d2ab6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.062+0000 7f9bff7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2aaacc0 0x5589d2ab6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39774/0 (socket says 172.21.15.44:39774) 2023-09-10T01:34:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.062+0000 7f9bff7fe700 1 -- 172.21.15.44:0/416466089 learned_addr learned my addr 172.21.15.44:0/416466089 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.067+0000 7f9c04aed700 1 --2- 172.21.15.44:0/416466089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2ab01a0 0x5589d2aa5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.067+0000 7f9bfffff700 1 --2- 172.21.15.44:0/416466089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaa300 0x5589d2ac9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.068+0000 7f9c04aed700 1 -- 172.21.15.44:0/416466089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2aaacc0 msgr2=0x5589d2ab6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.068+0000 7f9c04aed700 1 --2- 172.21.15.44:0/416466089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2aaacc0 0x5589d2ab6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.068+0000 7f9c04aed700 1 -- 172.21.15.44:0/416466089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaa300 msgr2=0x5589d2ac9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.068+0000 7f9c04aed700 1 --2- 172.21.15.44:0/416466089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaa300 0x5589d2ac9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.068+0000 7f9c04aed700 1 -- 172.21.15.44:0/416466089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589d284f1f0 con 0x5589d2ab01a0 2023-09-10T01:34:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.068+0000 7f9c04aed700 1 --2- 172.21.15.44:0/416466089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2ab01a0 0x5589d2aa5090 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8007fd0 tx=0x7f9bf8007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb8a7b2a13287d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.071+0000 7f9be6ffd700 1 -- 172.21.15.44:0/416466089 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8003d70 con 0x5589d2ab01a0 2023-09-10T01:34:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.071+0000 7f9be6ffd700 1 -- 172.21.15.44:0/416466089 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9bf801e040 con 0x5589d2ab01a0 2023-09-10T01:34:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.071+0000 7f9be6ffd700 1 -- 172.21.15.44:0/416466089 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8005070 con 0x5589d2ab01a0 2023-09-10T01:34:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.072+0000 7f9c12716b80 1 -- 172.21.15.44:0/416466089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2ab01a0 msgr2=0x5589d2aa5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.072+0000 7f9c12716b80 1 --2- 172.21.15.44:0/416466089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2ab01a0 0x5589d2aa5090 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8007fd0 tx=0x7f9bf8007690 comp rx=0 tx=0).stop 2023-09-10T01:34:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.073+0000 7f9c12716b80 1 -- 172.21.15.44:0/416466089 shutdown_connections 2023-09-10T01:34:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.073+0000 7f9c12716b80 1 --2- 172.21.15.44:0/416466089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2ab01a0 0x5589d2aa5090 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.073+0000 7f9c12716b80 1 --2- 172.21.15.44:0/416466089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaa300 0x5589d2ac9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.073+0000 7f9c12716b80 1 --2- 172.21.15.44:0/416466089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2aaacc0 0x5589d2ab6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.073+0000 7f9c12716b80 1 -- 172.21.15.44:0/416466089 >> 172.21.15.44:0/416466089 conn(0x5589d2a1f030 msgr2=0x5589d2aae0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.073+0000 7f9c12716b80 1 -- 172.21.15.44:0/416466089 shutdown_connections 2023-09-10T01:34:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.076+0000 7f9c12716b80 1 -- 172.21.15.44:0/416466089 wait complete. 2023-09-10T01:34:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.076+0000 7f9c12716b80 1 -- start start 2023-09-10T01:34:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.076+0000 7f9c12716b80 1 Processor -- start 2023-09-10T01:34:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.079+0000 7f9c12716b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2aaa300 0x5589d2ab5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.079+0000 7f9c12716b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaacc0 0x5589d2ab6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.079+0000 7f9c12716b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2ab01a0 0x5589d2b16c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.079+0000 7f9c12716b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d2ab6bd0 con 0x5589d2aaa300 2023-09-10T01:34:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.079+0000 7f9c12716b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d2b17160 con 0x5589d2aaacc0 2023-09-10T01:34:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.079+0000 7f9c12716b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d2b172f0 con 0x5589d2ab01a0 2023-09-10T01:34:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.079+0000 7f9c12716b80 10 client.0 _ll_register_callbacks cb 0x7f9bf0009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:34:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.079+0000 7f9bfffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2aaa300 0x5589d2ab5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.079+0000 7f9bfffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2aaa300 0x5589d2ab5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32896/0 (socket says 172.21.15.44:32896) 2023-09-10T01:34:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.079+0000 7f9bfffff700 1 -- 172.21.15.44:0/3305581202 learned_addr learned my addr 172.21.15.44:0/3305581202 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:34:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.080+0000 7f9c04aed700 1 --2- 172.21.15.44:0/3305581202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2ab01a0 0x5589d2b16c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.080+0000 7f9bfffff700 1 -- 172.21.15.44:0/3305581202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2ab01a0 msgr2=0x5589d2b16c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:34:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.080+0000 7f9bfffff700 1 --2- 172.21.15.44:0/3305581202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2ab01a0 0x5589d2b16c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.080+0000 7f9bfffff700 1 -- 172.21.15.44:0/3305581202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaacc0 msgr2=0x5589d2ab6490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:34:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.080+0000 7f9bfffff700 1 --2- 172.21.15.44:0/3305581202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaacc0 0x5589d2ab6490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.080+0000 7f9bfffff700 1 -- 172.21.15.44:0/3305581202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589d2828d80 con 0x5589d2aaa300 2023-09-10T01:34:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.080+0000 7f9bfffff700 1 --2- 172.21.15.44:0/3305581202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2aaa300 0x5589d2ab5f50 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00284a0 tx=0x7f9bf0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c629ed48bc8e0c36 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:34:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.081+0000 7f9bdf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:34:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.081+0000 7f9bdf7fe700 1 -- 172.21.15.44:0/3305581202 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0005e80 con 0x5589d2aaa300 2023-09-10T01:34:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.081+0000 7f9bdf7fe700 1 -- 172.21.15.44:0/3305581202 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(207..207 src has 1..207) v4 ==== 9524+0+0 (secure 0 0 0) 0x7f9bf0028c50 con 0x5589d2aaa300 2023-09-10T01:34:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.081+0000 7f9c12716b80 1 -- 172.21.15.44:0/3305581202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bf8003a00 con 0x5589d2aaa300 2023-09-10T01:34:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.082+0000 7f9bdf7fe700 1 -- 172.21.15.44:0/3305581202 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=193) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bf0028e10 con 0x5589d2aaa300 2023-09-10T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.082+0000 7f9c12716b80 10 client.20667 fetch_fsmap learned FSMap version 193 2023-09-10T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.082+0000 7f9c12716b80 1 -- 172.21.15.44:0/3305581202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=193,monmap=4+}) v3 -- 0x5589d2827250 con 0x5589d2aaa300 2023-09-10T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.082+0000 7f9bdf7fe700 1 -- 172.21.15.44:0/3305581202 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9bf8003a00 con 0x5589d2aaa300 2023-09-10T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.082+0000 7f9bdf7fe700 1 -- 172.21.15.44:0/3305581202 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 193) v1 ==== 7355+0+0 (secure 0 0 0) 0x7f9bf000dc80 con 0x5589d2aaa300 2023-09-10T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.082+0000 7f9c12716b80 10 client.20667 fetch_fsmap finished waiting for FSMap version 193 2023-09-10T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.082+0000 7f9c12716b80 10 client.20667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.082+0000 7f9c12716b80 20 client.20667 populate_metadata read hostname 'smithi044' 2023-09-10T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.082+0000 7f9c12716b80 1 --2- 172.21.15.44:0/3305581202 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5589d2b18a50 0x5589d2b1aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.082+0000 7f9c12716b80 4 client.20667 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.082+0000 7f9c12716b80 1 -- 172.21.15.44:0/3305581202 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5589d29f58c0 con 0x5589d2b18a50 2023-09-10T01:34:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.087+0000 7f9bff7fe700 1 --2- 172.21.15.44:0/3305581202 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5589d2b18a50 0x5589d2b1aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:34:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.088+0000 7f9bff7fe700 1 --2- 172.21.15.44:0/3305581202 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5589d2b18a50 0x5589d2b1aeb0 crc :-1 s=READY pgs=609 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7d5b387dea7fb9b server_cookie=ed2029910094fc48 in_seq=0 out_seq=0 2023-09-10T01:34:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.088+0000 7f9bdf7fe700 10 client.20667 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.099+0000 7f9bdf7fe700 1 -- 172.21.15.44:0/3305581202 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5589d29f58c0 con 0x5589d2b18a50 2023-09-10T01:34:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.099+0000 7f9bdf7fe700 10 client.20667 handle_command_reply: tid=0 2023-09-10T01:34:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.099+0000 7f9c12716b80 1 client.20667 shutdown 2023-09-10T01:34:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.099+0000 7f9c12716b80 1 -- 172.21.15.44:0/3305581202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2aaa300 msgr2=0x5589d2ab5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.099+0000 7f9c12716b80 1 --2- 172.21.15.44:0/3305581202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2aaa300 0x5589d2ab5f50 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00284a0 tx=0x7f9bf0005c90 comp rx=0 tx=0).stop 2023-09-10T01:34:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.099+0000 7f9c12716b80 1 -- 172.21.15.44:0/3305581202 shutdown_connections 2023-09-10T01:34:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.099+0000 7f9c12716b80 1 --2- 172.21.15.44:0/3305581202 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5589d2b18a50 0x5589d2b1aeb0 crc :-1 s=READY pgs=609 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.099+0000 7f9c12716b80 1 --2- 172.21.15.44:0/3305581202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5589d2ab01a0 0x5589d2b16c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.099+0000 7f9c12716b80 1 --2- 172.21.15.44:0/3305581202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5589d2aaa300 0x5589d2ab5f50 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.099+0000 7f9c12716b80 1 --2- 172.21.15.44:0/3305581202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589d2aaacc0 0x5589d2ab6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.099+0000 7f9c12716b80 1 -- 172.21.15.44:0/3305581202 >> 172.21.15.44:0/3305581202 conn(0x5589d2a1f030 msgr2=0x5589d2aa5090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.100+0000 7f9bdf7fe700 0 client.20667 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:34:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.100+0000 7f9c12716b80 1 -- 172.21.15.44:0/3305581202 shutdown_connections 2023-09-10T01:34:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.101+0000 7f9c12716b80 1 -- 172.21.15.44:0/3305581202 wait complete. 2023-09-10T01:34:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.101+0000 7f9c12716b80 20 client.20667 trim_cache size 0 max 16384 2023-09-10T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:34:56.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:55 smithi099 ceph-mon[102692]: pgmap v1768: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.9 MiB/s wr, 783 op/s 2023-09-10T01:34:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.107+0000 7f9c06550700 1 -- 172.21.15.44:0/2947277584 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9be80a0900 msgr2=0x7f9be80a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.107+0000 7f9c06550700 1 --2- 172.21.15.44:0/2947277584 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9be80a0900 0x7f9be80a2dc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x5589d2aa5060 tx=0x7f9bf0008040 comp rx=0 tx=0).stop 2023-09-10T01:34:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.107+0000 7f9c06550700 1 -- 172.21.15.44:0/2947277584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c0010bf10 msgr2=0x7f9c0007cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:34:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.107+0000 7f9c06550700 1 --2- 172.21.15.44:0/2947277584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c0010bf10 0x7f9c0007cb50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400b500 tx=0x7f9bf400b8c0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.109+0000 7f9c06550700 1 -- 172.21.15.44:0/2947277584 shutdown_connections 2023-09-10T01:34:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.109+0000 7f9c06550700 1 --2- 172.21.15.44:0/2947277584 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9be80a0900 0x7f9be80a2dc0 secure :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0x5589d2aa5060 tx=0x7f9bf0008040 comp rx=0 tx=0).stop 2023-09-10T01:34:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.109+0000 7f9c06550700 1 --2- 172.21.15.44:0/2947277584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c00083420 0x7f9c0007d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.109+0000 7f9c06550700 1 --2- 172.21.15.44:0/2947277584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c00107870 0x7f9c00082dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.109+0000 7f9c06550700 1 --2- 172.21.15.44:0/2947277584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c0010bf10 0x7f9c0007cb50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:34:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.109+0000 7f9c06550700 1 -- 172.21.15.44:0/2947277584 >> 172.21.15.44:0/2947277584 conn(0x7f9c0006c6c0 msgr2=0x7f9c00070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:34:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.112+0000 7f9c06550700 1 -- 172.21.15.44:0/2947277584 shutdown_connections 2023-09-10T01:34:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:34:56.112+0000 7f9c06550700 1 -- 172.21.15.44:0/2947277584 wait complete. 2023-09-10T01:34:56.187 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 830.0 s, 258.3 tps, lat 395.644 ms stddev 128.446 2023-09-10T01:34:58.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:57 smithi131 ceph-mon[103713]: pgmap v1769: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.3 MiB/s wr, 789 op/s 2023-09-10T01:34:58.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:57 smithi044 ceph-mon[98676]: pgmap v1769: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.3 MiB/s wr, 789 op/s 2023-09-10T01:34:58.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:57 smithi099 ceph-mon[102692]: pgmap v1769: 129 pgs: 129 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.3 MiB/s wr, 789 op/s 2023-09-10T01:35:00.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:34:59 smithi131 ceph-mon[103713]: pgmap v1770: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.9 MiB/s wr, 972 op/s 2023-09-10T01:35:00.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:34:59 smithi044 ceph-mon[98676]: pgmap v1770: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.9 MiB/s wr, 972 op/s 2023-09-10T01:35:00.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:34:59 smithi099 ceph-mon[102692]: pgmap v1770: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.9 MiB/s wr, 972 op/s 2023-09-10T01:35:01.190 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 835.0 s, 285.8 tps, lat 336.427 ms stddev 140.102 2023-09-10T01:35:02.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:01 smithi131 ceph-mon[103713]: pgmap v1771: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.3 MiB/s wr, 782 op/s 2023-09-10T01:35:02.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:01 smithi044 ceph-mon[98676]: pgmap v1771: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.3 MiB/s wr, 782 op/s 2023-09-10T01:35:02.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:01 smithi099 ceph-mon[102692]: pgmap v1771: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.3 MiB/s wr, 782 op/s 2023-09-10T01:35:04.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:03 smithi131 ceph-mon[103713]: pgmap v1772: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.3 MiB/s wr, 1.01k op/s 2023-09-10T01:35:04.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:03 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:35:04.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:03 smithi044 ceph-mon[98676]: pgmap v1772: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.3 MiB/s wr, 1.01k op/s 2023-09-10T01:35:04.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:03 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:35:04.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:03 smithi099 ceph-mon[102692]: pgmap v1772: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.3 MiB/s wr, 1.01k op/s 2023-09-10T01:35:04.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:03 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:35:05.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:04 smithi131 ceph-mon[103713]: osdmap e208: 12 total, 12 up, 12 in 2023-09-10T01:35:05.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:04 smithi044 ceph-mon[98676]: osdmap e208: 12 total, 12 up, 12 in 2023-09-10T01:35:05.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:04 smithi099 ceph-mon[102692]: osdmap e208: 12 total, 12 up, 12 in 2023-09-10T01:35:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 840.0 s, 206.1 tps, lat 288.596 ms stddev 517.787 2023-09-10T01:35:06.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:05 smithi131 ceph-mon[103713]: pgmap v1774: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 914 op/s 2023-09-10T01:35:06.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:05 smithi044 ceph-mon[98676]: pgmap v1774: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 914 op/s 2023-09-10T01:35:06.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:05 smithi099 ceph-mon[102692]: pgmap v1774: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 914 op/s 2023-09-10T01:35:08.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:07 smithi131 ceph-mon[103713]: pgmap v1775: 129 pgs: 5 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 122 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 914 op/s 2023-09-10T01:35:08.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:07 smithi044 ceph-mon[98676]: pgmap v1775: 129 pgs: 5 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 122 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 914 op/s 2023-09-10T01:35:08.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:07 smithi099 ceph-mon[102692]: pgmap v1775: 129 pgs: 5 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 122 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 914 op/s 2023-09-10T01:35:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.341+0000 7f10613de700 1 -- 172.21.15.44:0/3667356696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f105c10ed50 msgr2=0x7f105c107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.341+0000 7f10613de700 1 --2- 172.21.15.44:0/3667356696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f105c10ed50 0x7f105c107600 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1054009f90 tx=0x7f105400b330 comp rx=0 tx=0).stop 2023-09-10T01:35:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.342+0000 7f10613de700 1 -- 172.21.15.44:0/3667356696 shutdown_connections 2023-09-10T01:35:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.342+0000 7f10613de700 1 --2- 172.21.15.44:0/3667356696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f105c119de0 0x7f105c11c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.342+0000 7f10613de700 1 --2- 172.21.15.44:0/3667356696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f105c10ed50 0x7f105c107600 secure :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1054009f90 tx=0x7f105400b330 comp rx=0 tx=0).stop 2023-09-10T01:35:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.342+0000 7f10613de700 1 --2- 172.21.15.44:0/3667356696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f105c10f1c0 0x7f105c107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.342+0000 7f10613de700 1 -- 172.21.15.44:0/3667356696 >> 172.21.15.44:0/3667356696 conn(0x7f105c06c6c0 msgr2=0x7f105c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.352+0000 7f10613de700 1 -- 172.21.15.44:0/3667356696 shutdown_connections 2023-09-10T01:35:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.354+0000 7f10613de700 1 -- 172.21.15.44:0/3667356696 wait complete. 2023-09-10T01:35:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.355+0000 7f10613de700 1 Processor -- start 2023-09-10T01:35:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.360+0000 7f10613de700 1 -- start start 2023-09-10T01:35:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.361+0000 7f10613de700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f105c10f1c0 0x7f105c119500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.361+0000 7f10613de700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f105c119de0 0x7f105c112300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.361+0000 7f10613de700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f105c112840 0x7f105c112cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.361+0000 7f10613de700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c117090 con 0x7f105c10f1c0 2023-09-10T01:35:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.361+0000 7f10613de700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c1171d0 con 0x7f105c119de0 2023-09-10T01:35:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.361+0000 7f10613de700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c117390 con 0x7f105c112840 2023-09-10T01:35:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.361+0000 7f105b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f105c112840 0x7f105c112cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.361+0000 7f105b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f105c112840 0x7f105c112cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39790/0 (socket says 172.21.15.44:39790) 2023-09-10T01:35:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.361+0000 7f105b7fe700 1 -- 172.21.15.44:0/1804637777 learned_addr learned my addr 172.21.15.44:0/1804637777 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.362+0000 7f105b7fe700 1 -- 172.21.15.44:0/1804637777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f105c119de0 msgr2=0x7f105c112300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-10T01:35:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.362+0000 7f105b7fe700 1 --2- 172.21.15.44:0/1804637777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f105c119de0 0x7f105c112300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.362+0000 7f105b7fe700 1 -- 172.21.15.44:0/1804637777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f105c10f1c0 msgr2=0x7f105c119500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.362+0000 7f105b7fe700 1 --2- 172.21.15.44:0/1804637777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f105c10f1c0 0x7f105c119500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.362+0000 7f105b7fe700 1 -- 172.21.15.44:0/1804637777 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1054009c40 con 0x7f105c112840 2023-09-10T01:35:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.362+0000 7f105b7fe700 1 --2- 172.21.15.44:0/1804637777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f105c112840 0x7f105c112cf0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f105000e970 tx=0x7f105000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.363+0000 7f1043fff700 1 -- 172.21.15.44:0/1804637777 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105000cbd0 con 0x7f105c112840 2023-09-10T01:35:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.363+0000 7f1043fff700 1 -- 172.21.15.44:0/1804637777 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f105000cd30 con 0x7f105c112840 2023-09-10T01:35:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.363+0000 7f1043fff700 1 -- 172.21.15.44:0/1804637777 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050005520 con 0x7f105c112840 2023-09-10T01:35:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.363+0000 7f10613de700 1 -- 172.21.15.44:0/1804637777 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105c117670 con 0x7f105c112840 2023-09-10T01:35:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.363+0000 7f10613de700 1 -- 172.21.15.44:0/1804637777 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c117b30 con 0x7f105c112840 2023-09-10T01:35:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.367+0000 7f106d5a4b80 1 Processor -- start 2023-09-10T01:35:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.369+0000 7f106d5a4b80 1 -- start start 2023-09-10T01:35:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.369+0000 7f106d5a4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 0x55943c263580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.369+0000 7f106d5a4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25d300 0x55943c25d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.369+0000 7f106d5a4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25dcc0 0x55943c269f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.369+0000 7f106d5a4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943c1b66d0 con 0x55943c2631a0 2023-09-10T01:35:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.369+0000 7f106d5a4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943c1b3ff0 con 0x55943c25d300 2023-09-10T01:35:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.369+0000 7f106d5a4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943c1af8e0 con 0x55943c25dcc0 2023-09-10T01:35:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.369+0000 7f105affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25d300 0x55943c25d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.369+0000 7f105affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25d300 0x55943c25d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34650/0 (socket says 172.21.15.44:34650) 2023-09-10T01:35:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.369+0000 7f105affd700 1 -- 172.21.15.44:0/1058294901 learned_addr learned my addr 172.21.15.44:0/1058294901 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.370+0000 7f105affd700 1 -- 172.21.15.44:0/1058294901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25dcc0 msgr2=0x55943c269f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:35:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.370+0000 7f105affd700 1 --2- 172.21.15.44:0/1058294901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25dcc0 0x55943c269f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.370+0000 7f105affd700 1 -- 172.21.15.44:0/1058294901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 msgr2=0x55943c263580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.370+0000 7f105affd700 1 --2- 172.21.15.44:0/1058294901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 0x55943c263580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.371+0000 7f105affd700 1 -- 172.21.15.44:0/1058294901 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55943bfdbd80 con 0x55943c25d300 2023-09-10T01:35:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.371+0000 7f1043fff700 1 -- 172.21.15.44:0/1804637777 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1050021440 con 0x7f105c112840 2023-09-10T01:35:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.371+0000 7f105affd700 1 --2- 172.21.15.44:0/1058294901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25d300 0x55943c25d780 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1054015000 tx=0x7f10540092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83f114a90144c6bc server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.372+0000 7f1043fff700 1 --2- 172.21.15.44:0/1804637777 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f10440a07b0 0x7f10440a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.372+0000 7f105a7fc700 1 --2- 172.21.15.44:0/1804637777 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f10440a07b0 0x7f10440a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.372+0000 7f1043fff700 1 -- 172.21.15.44:0/1804637777 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f1050014070 con 0x7f105c112840 2023-09-10T01:35:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.372+0000 7f105a7fc700 1 --2- 172.21.15.44:0/1804637777 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f10440a07b0 0x7f10440a2c70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f104c005b40 tx=0x7f104c005ab0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.377+0000 7f10417fa700 1 -- 172.21.15.44:0/1058294901 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054021070 con 0x55943c25d300 2023-09-10T01:35:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.377+0000 7f10417fa700 1 -- 172.21.15.44:0/1058294901 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1054009ac0 con 0x55943c25d300 2023-09-10T01:35:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.377+0000 7f10417fa700 1 -- 172.21.15.44:0/1058294901 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105400e040 con 0x55943c25d300 2023-09-10T01:35:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.383+0000 7f106d5a4b80 1 -- 172.21.15.44:0/1058294901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25d300 msgr2=0x55943c25d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.383+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/1058294901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25d300 0x55943c25d780 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1054015000 tx=0x7f10540092e0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.384+0000 7f106d5a4b80 1 -- 172.21.15.44:0/1058294901 shutdown_connections 2023-09-10T01:35:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.384+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/1058294901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25dcc0 0x55943c269f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.384+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/1058294901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 0x55943c263580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.384+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/1058294901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25d300 0x55943c25d780 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.384+0000 7f106d5a4b80 1 -- 172.21.15.44:0/1058294901 >> 172.21.15.44:0/1058294901 conn(0x55943c1d2030 msgr2=0x55943c258560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.386+0000 7f106d5a4b80 1 -- 172.21.15.44:0/1058294901 shutdown_connections 2023-09-10T01:35:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.387+0000 7f106d5a4b80 1 -- 172.21.15.44:0/1058294901 wait complete. 2023-09-10T01:35:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.387+0000 7f106d5a4b80 1 -- start start 2023-09-10T01:35:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.387+0000 7f106d5a4b80 1 Processor -- start 2023-09-10T01:35:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.388+0000 7f106d5a4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25d300 0x55943c269dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.388+0000 7f106d5a4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25dcc0 0x55943c267040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.388+0000 7f106d5a4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 0x55943c267620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.388+0000 7f106d5a4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943c14db10 con 0x55943c2631a0 2023-09-10T01:35:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.388+0000 7f106d5a4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943c14d810 con 0x55943c25dcc0 2023-09-10T01:35:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.388+0000 7f106d5a4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943c0318c0 con 0x55943c25d300 2023-09-10T01:35:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.388+0000 7f106d5a4b80 10 client.0 _ll_register_callbacks cb 0x55943c097a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:35:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.389+0000 7f105b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 0x55943c267620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.389+0000 7f105b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 0x55943c267620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32922/0 (socket says 172.21.15.44:32922) 2023-09-10T01:35:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.389+0000 7f105b7fe700 1 -- 172.21.15.44:0/817439281 learned_addr learned my addr 172.21.15.44:0/817439281 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.389+0000 7f105a7fc700 1 --2- 172.21.15.44:0/817439281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25dcc0 0x55943c267040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.389+0000 7f105b7fe700 1 -- 172.21.15.44:0/817439281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25d300 msgr2=0x55943c269dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.389+0000 7f105b7fe700 1 --2- 172.21.15.44:0/817439281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25d300 0x55943c269dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.389+0000 7f105b7fe700 1 -- 172.21.15.44:0/817439281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25dcc0 msgr2=0x55943c267040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.389+0000 7f105b7fe700 1 --2- 172.21.15.44:0/817439281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25dcc0 0x55943c267040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.389+0000 7f105b7fe700 1 -- 172.21.15.44:0/817439281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c11cb00 con 0x55943c2631a0 2023-09-10T01:35:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.389+0000 7f105a7fc700 1 --2- 172.21.15.44:0/817439281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25dcc0 0x55943c267040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:35:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.389+0000 7f105b7fe700 1 --2- 172.21.15.44:0/817439281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 0x55943c267620 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f10500a20f0 tx=0x7f1050099f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5591d55e5c7f202 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.389+0000 7f103eff5700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:35:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.389+0000 7f103eff5700 1 -- 172.21.15.44:0/817439281 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105009a5b0 con 0x55943c2631a0 2023-09-10T01:35:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.389+0000 7f106d5a4b80 1 -- 172.21.15.44:0/817439281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55943c1c4840 con 0x55943c2631a0 2023-09-10T01:35:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.390+0000 7f103eff5700 1 -- 172.21.15.44:0/817439281 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(208..208 src has 1..208) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f105009e3e0 con 0x55943c2631a0 2023-09-10T01:35:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.390+0000 7f103eff5700 1 -- 172.21.15.44:0/817439281 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=194) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10500a2620 con 0x55943c2631a0 2023-09-10T01:35:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.390+0000 7f106d5a4b80 10 client.20673 fetch_fsmap learned FSMap version 194 2023-09-10T01:35:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.390+0000 7f106d5a4b80 1 -- 172.21.15.44:0/817439281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=194,monmap=4+}) v3 -- 0x55943bfc4c00 con 0x55943c2631a0 2023-09-10T01:35:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.390+0000 7f103eff5700 1 -- 172.21.15.44:0/817439281 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1050064030 con 0x55943c2631a0 2023-09-10T01:35:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.390+0000 7f103eff5700 1 -- 172.21.15.44:0/817439281 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 194) v1 ==== 7359+0+0 (secure 0 0 0) 0x7f1050099aa0 con 0x55943c2631a0 2023-09-10T01:35:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.390+0000 7f106d5a4b80 10 client.20673 fetch_fsmap finished waiting for FSMap version 194 2023-09-10T01:35:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.390+0000 7f106d5a4b80 10 client.20673 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:35:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.390+0000 7f106d5a4b80 20 client.20673 populate_metadata read hostname 'smithi044' 2023-09-10T01:35:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.390+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/817439281 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55943c277360 0x55943c2797e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.390+0000 7f106d5a4b80 4 client.20673 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:35:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.390+0000 7f106d5a4b80 1 -- 172.21.15.44:0/817439281 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55943c1a88c0 con 0x55943c277360 2023-09-10T01:35:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.393+0000 7f105affd700 1 --2- 172.21.15.44:0/817439281 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55943c277360 0x55943c2797e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.394+0000 7f105affd700 1 --2- 172.21.15.44:0/817439281 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55943c277360 0x55943c2797e0 crc :-1 s=READY pgs=610 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a383a1829fb52c23 server_cookie=79f641f1c4efab4d in_seq=0 out_seq=0 2023-09-10T01:35:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.394+0000 7f103eff5700 10 client.20673 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:35:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.394+0000 7f103eff5700 1 -- 172.21.15.44:0/817439281 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55943c1a88c0 con 0x55943c277360 2023-09-10T01:35:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.394+0000 7f103eff5700 10 client.20673 handle_command_reply: tid=0 2023-09-10T01:35:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.394+0000 7f106d5a4b80 1 client.20673 shutdown 2023-09-10T01:35:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.399+0000 7f106d5a4b80 1 -- 172.21.15.44:0/817439281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 msgr2=0x55943c267620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.399+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/817439281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 0x55943c267620 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f10500a20f0 tx=0x7f1050099f90 comp rx=0 tx=0).stop 2023-09-10T01:35:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.400+0000 7f106d5a4b80 1 -- 172.21.15.44:0/817439281 shutdown_connections 2023-09-10T01:35:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.400+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/817439281 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55943c277360 0x55943c2797e0 crc :-1 s=READY pgs=610 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.400+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/817439281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 0x55943c267620 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.400+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/817439281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25d300 0x55943c269dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.400+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/817439281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25dcc0 0x55943c267040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.400+0000 7f103eff5700 0 client.20673 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:35:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.400+0000 7f106d5a4b80 1 -- 172.21.15.44:0/817439281 >> 172.21.15.44:0/817439281 conn(0x55943c1d2030 msgr2=0x55943c2577a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.403+0000 7f106d5a4b80 1 -- 172.21.15.44:0/817439281 shutdown_connections 2023-09-10T01:35:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.405+0000 7f106d5a4b80 1 -- 172.21.15.44:0/817439281 wait complete. 2023-09-10T01:35:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.406+0000 7f106d5a4b80 20 client.20673 trim_cache size 0 max 16384 2023-09-10T01:35:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.431+0000 7f106d5a4b80 1 Processor -- start 2023-09-10T01:35:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.431+0000 7f106d5a4b80 1 -- start start 2023-09-10T01:35:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.431+0000 7f106d5a4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25d300 0x55943c26e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.431+0000 7f106d5a4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25dcc0 0x55943c269960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.431+0000 7f106d5a4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 0x55943c262080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.431+0000 7f106d5a4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943c254fc0 con 0x55943c2631a0 2023-09-10T01:35:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.431+0000 7f106d5a4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943c25e400 con 0x55943c25dcc0 2023-09-10T01:35:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.431+0000 7f106d5a4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943c26ea70 con 0x55943c25d300 2023-09-10T01:35:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.432+0000 7f105a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25dcc0 0x55943c269960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.432+0000 7f105a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25dcc0 0x55943c269960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34666/0 (socket says 172.21.15.44:34666) 2023-09-10T01:35:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.432+0000 7f105a7fc700 1 -- 172.21.15.44:0/1634770580 learned_addr learned my addr 172.21.15.44:0/1634770580 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.432+0000 7f105b7fe700 1 --2- 172.21.15.44:0/1634770580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 0x55943c262080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.432+0000 7f105a7fc700 1 -- 172.21.15.44:0/1634770580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25d300 msgr2=0x55943c26e530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:35:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.432+0000 7f105a7fc700 1 --2- 172.21.15.44:0/1634770580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25d300 0x55943c26e530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.432+0000 7f105a7fc700 1 -- 172.21.15.44:0/1634770580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 msgr2=0x55943c262080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.432+0000 7f105a7fc700 1 --2- 172.21.15.44:0/1634770580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 0x55943c262080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.432+0000 7f105a7fc700 1 -- 172.21.15.44:0/1634770580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55943c0021f0 con 0x55943c25dcc0 2023-09-10T01:35:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.432+0000 7f105b7fe700 1 --2- 172.21.15.44:0/1634770580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 0x55943c262080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:35:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.432+0000 7f105a7fc700 1 --2- 172.21.15.44:0/1634770580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25dcc0 0x55943c269960 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f105c06de40 tx=0x7f104c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8517ae1d96f3c117 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.440+0000 7f1041ffb700 1 -- 172.21.15.44:0/1634770580 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c00e760 con 0x55943c25dcc0 2023-09-10T01:35:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.440+0000 7f1041ffb700 1 -- 172.21.15.44:0/1634770580 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f104c00ed50 con 0x55943c25dcc0 2023-09-10T01:35:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.440+0000 7f1041ffb700 1 -- 172.21.15.44:0/1634770580 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c027c40 con 0x55943c25dcc0 2023-09-10T01:35:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.443+0000 7f106d5a4b80 1 -- 172.21.15.44:0/1634770580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25dcc0 msgr2=0x55943c269960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.443+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/1634770580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25dcc0 0x55943c269960 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f105c06de40 tx=0x7f104c011000 comp rx=0 tx=0).stop 2023-09-10T01:35:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.443+0000 7f106d5a4b80 1 -- 172.21.15.44:0/1634770580 shutdown_connections 2023-09-10T01:35:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.443+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/1634770580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c2631a0 0x55943c262080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.443+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/1634770580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25d300 0x55943c26e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.443+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/1634770580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c25dcc0 0x55943c269960 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.443+0000 7f106d5a4b80 1 -- 172.21.15.44:0/1634770580 >> 172.21.15.44:0/1634770580 conn(0x55943c1d2030 msgr2=0x55943c2736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.444+0000 7f106d5a4b80 1 -- 172.21.15.44:0/1634770580 shutdown_connections 2023-09-10T01:35:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.444+0000 7f106d5a4b80 1 -- 172.21.15.44:0/1634770580 wait complete. 2023-09-10T01:35:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.444+0000 7f106d5a4b80 1 -- start start 2023-09-10T01:35:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.444+0000 7f106d5a4b80 1 Processor -- start 2023-09-10T01:35:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.445+0000 7f106d5a4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c25d300 0x55943c26c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.445+0000 7f106d5a4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25dcc0 0x55943c26c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.445+0000 7f106d5a4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c2631a0 0x55943c2c4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.445+0000 7f106d5a4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943c2c47e0 con 0x55943c25d300 2023-09-10T01:35:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.445+0000 7f106d5a4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943c2c4950 con 0x55943c2631a0 2023-09-10T01:35:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.445+0000 7f106d5a4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943c2c4b10 con 0x55943c25dcc0 2023-09-10T01:35:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.445+0000 7f106d5a4b80 10 client.0 _ll_register_callbacks cb 0x7f105009a380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:35:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.445+0000 7f105b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c2631a0 0x55943c2c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.445+0000 7f105b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c2631a0 0x55943c2c4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34670/0 (socket says 172.21.15.44:34670) 2023-09-10T01:35:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.445+0000 7f105b7fe700 1 -- 172.21.15.44:0/2104359857 learned_addr learned my addr 172.21.15.44:0/2104359857 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.446+0000 7f105a7fc700 1 --2- 172.21.15.44:0/2104359857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25dcc0 0x55943c26c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.446+0000 7f105b7fe700 1 -- 172.21.15.44:0/2104359857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25dcc0 msgr2=0x55943c26c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.446+0000 7f105b7fe700 1 --2- 172.21.15.44:0/2104359857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25dcc0 0x55943c26c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.446+0000 7f105b7fe700 1 -- 172.21.15.44:0/2104359857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c25d300 msgr2=0x55943c26c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.446+0000 7f105b7fe700 1 --2- 172.21.15.44:0/2104359857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c25d300 0x55943c26c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.446+0000 7f105b7fe700 1 -- 172.21.15.44:0/2104359857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104c005230 con 0x55943c2631a0 2023-09-10T01:35:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.446+0000 7f105b7fe700 1 --2- 172.21.15.44:0/2104359857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c2631a0 0x55943c2c4270 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f10500cc710 tx=0x7f105000cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeea7e1c0e548f4c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.446+0000 7f103eff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:35:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.446+0000 7f103eff5700 1 -- 172.21.15.44:0/2104359857 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105009acb0 con 0x55943c2631a0 2023-09-10T01:35:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.446+0000 7f106d5a4b80 1 -- 172.21.15.44:0/2104359857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55943c107490 con 0x55943c2631a0 2023-09-10T01:35:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.446+0000 7f103eff5700 1 -- 172.21.15.44:0/2104359857 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(208..208 src has 1..208) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f105009e3e0 con 0x55943c2631a0 2023-09-10T01:35:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.447+0000 7f103eff5700 1 -- 172.21.15.44:0/2104359857 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=194) v2 ==== 24+0+0 (secure 0 0 0) 0x7f105009a5b0 con 0x55943c2631a0 2023-09-10T01:35:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.455+0000 7f106d5a4b80 10 client.30314 fetch_fsmap learned FSMap version 194 2023-09-10T01:35:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.455+0000 7f106d5a4b80 1 -- 172.21.15.44:0/2104359857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=194,monmap=4+}) v3 -- 0x55943bfda250 con 0x55943c2631a0 2023-09-10T01:35:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.465+0000 7f103eff5700 1 -- 172.21.15.44:0/2104359857 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55943c107490 con 0x55943c2631a0 2023-09-10T01:35:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.465+0000 7f103eff5700 1 -- 172.21.15.44:0/2104359857 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 194) v1 ==== 7359+0+0 (secure 0 0 0) 0x7f10500963a0 con 0x55943c2631a0 2023-09-10T01:35:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.465+0000 7f106d5a4b80 10 client.30314 fetch_fsmap finished waiting for FSMap version 194 2023-09-10T01:35:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.465+0000 7f106d5a4b80 10 client.30314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:35:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.465+0000 7f106d5a4b80 20 client.30314 populate_metadata read hostname 'smithi044' 2023-09-10T01:35:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.465+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/2104359857 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55943c2cbde0 0x55943c2ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.465+0000 7f106d5a4b80 4 client.30314 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:35:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.465+0000 7f106d5a4b80 1 -- 172.21.15.44:0/2104359857 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55943c169050 con 0x55943c2cbde0 2023-09-10T01:35:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.473+0000 7f105affd700 1 --2- 172.21.15.44:0/2104359857 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55943c2cbde0 0x55943c2ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.491+0000 7f103eff5700 10 client.30314 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:35:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.491+0000 7f105affd700 1 --2- 172.21.15.44:0/2104359857 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55943c2cbde0 0x55943c2ce1d0 crc :-1 s=READY pgs=611 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dcd2526374fe143 server_cookie=47fd6279f8a435b5 in_seq=0 out_seq=0 2023-09-10T01:35:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.753+0000 7f103eff5700 1 -- 172.21.15.44:0/2104359857 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55943c169050 con 0x55943c2cbde0 2023-09-10T01:35:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.753+0000 7f103eff5700 10 client.30314 handle_command_reply: tid=0 2023-09-10T01:35:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.759+0000 7f106d5a4b80 1 client.30314 shutdown 2023-09-10T01:35:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.765+0000 7f106d5a4b80 1 -- 172.21.15.44:0/2104359857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c2631a0 msgr2=0x55943c2c4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.765+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/2104359857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c2631a0 0x55943c2c4270 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f10500cc710 tx=0x7f105000cea0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.774+0000 7f106d5a4b80 1 -- 172.21.15.44:0/2104359857 shutdown_connections 2023-09-10T01:35:09.778 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.774+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/2104359857 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55943c2cbde0 0x55943c2ce1d0 crc :-1 s=READY pgs=611 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.774+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/2104359857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55943c2631a0 0x55943c2c4270 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.774+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/2104359857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943c25d300 0x55943c26c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.774+0000 7f106d5a4b80 1 --2- 172.21.15.44:0/2104359857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55943c25dcc0 0x55943c26c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.774+0000 7f106d5a4b80 1 -- 172.21.15.44:0/2104359857 >> 172.21.15.44:0/2104359857 conn(0x55943c1d2030 msgr2=0x55943c262080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:09.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.777+0000 7f103eff5700 0 client.30314 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:35:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.783+0000 7f106d5a4b80 1 -- 172.21.15.44:0/2104359857 shutdown_connections 2023-09-10T01:35:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.783+0000 7f106d5a4b80 1 -- 172.21.15.44:0/2104359857 wait complete. 2023-09-10T01:35:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.791+0000 7f106d5a4b80 20 client.30314 trim_cache size 0 max 16384 2023-09-10T01:35:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:35:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:35:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:35:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:35:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.806+0000 7f10613de700 1 -- 172.21.15.44:0/1804637777 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f10440a07b0 msgr2=0x7f10440a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.806+0000 7f10613de700 1 --2- 172.21.15.44:0/1804637777 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f10440a07b0 0x7f10440a2c70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f104c005b40 tx=0x7f104c005ab0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.806+0000 7f10613de700 1 -- 172.21.15.44:0/1804637777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f105c112840 msgr2=0x7f105c112cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.806+0000 7f10613de700 1 --2- 172.21.15.44:0/1804637777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f105c112840 0x7f105c112cf0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f105000e970 tx=0x7f105000ed30 comp rx=0 tx=0).stop 2023-09-10T01:35:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.816+0000 7f10613de700 1 -- 172.21.15.44:0/1804637777 shutdown_connections 2023-09-10T01:35:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.816+0000 7f10613de700 1 --2- 172.21.15.44:0/1804637777 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f10440a07b0 0x7f10440a2c70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.816+0000 7f10613de700 1 --2- 172.21.15.44:0/1804637777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f105c112840 0x7f105c112cf0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.816+0000 7f10613de700 1 --2- 172.21.15.44:0/1804637777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f105c10f1c0 0x7f105c119500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.816+0000 7f10613de700 1 --2- 172.21.15.44:0/1804637777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f105c119de0 0x7f105c112300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.816+0000 7f10613de700 1 -- 172.21.15.44:0/1804637777 >> 172.21.15.44:0/1804637777 conn(0x7f105c06c6c0 msgr2=0x7f105c11b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.830+0000 7f10613de700 1 -- 172.21.15.44:0/1804637777 shutdown_connections 2023-09-10T01:35:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:09.832+0000 7f10613de700 1 -- 172.21.15.44:0/1804637777 wait complete. 2023-09-10T01:35:10.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:09 smithi131 ceph-mon[103713]: pgmap v1776: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 767 op/s 2023-09-10T01:35:10.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:09 smithi044 ceph-mon[98676]: pgmap v1776: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 767 op/s 2023-09-10T01:35:10.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:09 smithi099 ceph-mon[102692]: pgmap v1776: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 767 op/s 2023-09-10T01:35:11.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 845.0 s, 65.2 tps, lat 1461.530 ms stddev 2379.710 2023-09-10T01:35:12.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:12 smithi131 ceph-mon[103713]: pgmap v1777: 129 pgs: 20 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 57 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.1 MiB/s wr, 707 op/s 2023-09-10T01:35:12.313 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:12 smithi044 ceph-mon[98676]: pgmap v1777: 129 pgs: 20 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 57 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.1 MiB/s wr, 707 op/s 2023-09-10T01:35:12.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:12 smithi099 ceph-mon[102692]: pgmap v1777: 129 pgs: 20 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 57 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.1 MiB/s wr, 707 op/s 2023-09-10T01:35:14.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:14 smithi131 ceph-mon[103713]: pgmap v1778: 129 pgs: 11 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 86 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2023-09-10T01:35:14.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:14 smithi044 ceph-mon[98676]: pgmap v1778: 129 pgs: 11 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 86 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2023-09-10T01:35:14.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:14 smithi099 ceph-mon[102692]: pgmap v1778: 129 pgs: 11 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 86 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2023-09-10T01:35:16.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 850.0 s, 170.6 tps, lat 801.998 ms stddev 1602.477 2023-09-10T01:35:16.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:16 smithi131 ceph-mon[103713]: pgmap v1779: 129 pgs: 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 115 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 371 op/s 2023-09-10T01:35:16.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:16 smithi044 ceph-mon[98676]: pgmap v1779: 129 pgs: 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 115 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 371 op/s 2023-09-10T01:35:16.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:16 smithi099 ceph-mon[102692]: pgmap v1779: 129 pgs: 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 115 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 371 op/s 2023-09-10T01:35:18.052 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:17 smithi131 ceph-mon[103713]: pgmap v1780: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 359 op/s 2023-09-10T01:35:18.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:17 smithi044 ceph-mon[98676]: pgmap v1780: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 359 op/s 2023-09-10T01:35:18.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:17 smithi099 ceph-mon[102692]: pgmap v1780: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 359 op/s 2023-09-10T01:35:20.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:19 smithi131 ceph-mon[103713]: pgmap v1781: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 439 op/s 2023-09-10T01:35:20.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:19 smithi044 ceph-mon[98676]: pgmap v1781: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 439 op/s 2023-09-10T01:35:20.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:19 smithi099 ceph-mon[102692]: pgmap v1781: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 439 op/s 2023-09-10T01:35:21.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 855.0 s, 206.8 tps, lat 498.052 ms stddev 702.404 2023-09-10T01:35:22.051 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:21 smithi131 ceph-mon[103713]: pgmap v1782: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 360 op/s 2023-09-10T01:35:22.051 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:21 smithi131 ceph-mon[103713]: osdmap e209: 12 total, 12 up, 12 in 2023-09-10T01:35:22.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:21 smithi044 ceph-mon[98676]: pgmap v1782: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 360 op/s 2023-09-10T01:35:22.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:21 smithi044 ceph-mon[98676]: osdmap e209: 12 total, 12 up, 12 in 2023-09-10T01:35:22.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:21 smithi099 ceph-mon[102692]: pgmap v1782: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 360 op/s 2023-09-10T01:35:22.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:21 smithi099 ceph-mon[102692]: osdmap e209: 12 total, 12 up, 12 in 2023-09-10T01:35:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.067+0000 7f66aff19700 1 -- 172.21.15.44:0/3779600064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66a8119de0 msgr2=0x7f66a811c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.067+0000 7f66aff19700 1 --2- 172.21.15.44:0/3779600064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66a8119de0 0x7f66a811c1d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009f90 tx=0x7f66a000b330 comp rx=0 tx=0).stop 2023-09-10T01:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.068+0000 7f66aff19700 1 -- 172.21.15.44:0/3779600064 shutdown_connections 2023-09-10T01:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.068+0000 7f66aff19700 1 --2- 172.21.15.44:0/3779600064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66a8119de0 0x7f66a811c1d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.068+0000 7f66aff19700 1 --2- 172.21.15.44:0/3779600064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66a810ed50 0x7f66a8107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.068+0000 7f66aff19700 1 --2- 172.21.15.44:0/3779600064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66a810f1c0 0x7f66a8107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.068+0000 7f66aff19700 1 -- 172.21.15.44:0/3779600064 >> 172.21.15.44:0/3779600064 conn(0x7f66a806c6c0 msgr2=0x7f66a806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.068+0000 7f66aff19700 1 -- 172.21.15.44:0/3779600064 shutdown_connections 2023-09-10T01:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.068+0000 7f66aff19700 1 -- 172.21.15.44:0/3779600064 wait complete. 2023-09-10T01:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.069+0000 7f66aff19700 1 Processor -- start 2023-09-10T01:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.069+0000 7f66aff19700 1 -- start start 2023-09-10T01:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.069+0000 7f66aff19700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66a810ed50 0x7f66a81a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.069+0000 7f66aff19700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66a810f1c0 0x7f66a81a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.069+0000 7f66aff19700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66a8119de0 0x7f66a81a9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.069+0000 7f66aff19700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a81a9600 con 0x7f66a8119de0 2023-09-10T01:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.070+0000 7f66aff19700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a81a9770 con 0x7f66a810ed50 2023-09-10T01:35:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.070+0000 7f66aff19700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a81a9930 con 0x7f66a810f1c0 2023-09-10T01:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.070+0000 7f66ae4b6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66a8119de0 0x7f66a81a9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.070+0000 7f66ae4b6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66a8119de0 0x7f66a81a9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32946/0 (socket says 172.21.15.44:32946) 2023-09-10T01:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.070+0000 7f66ae4b6700 1 -- 172.21.15.44:0/80322445 learned_addr learned my addr 172.21.15.44:0/80322445 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.070+0000 7f66ae4b6700 1 -- 172.21.15.44:0/80322445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66a810f1c0 msgr2=0x7f66a81a5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.070+0000 7f66ae4b6700 1 --2- 172.21.15.44:0/80322445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66a810f1c0 0x7f66a81a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.070+0000 7f66ae4b6700 1 -- 172.21.15.44:0/80322445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66a810ed50 msgr2=0x7f66a81a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.070+0000 7f66ae4b6700 1 --2- 172.21.15.44:0/80322445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66a810ed50 0x7f66a81a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.070+0000 7f66ae4b6700 1 -- 172.21.15.44:0/80322445 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a0009c40 con 0x7f66a8119de0 2023-09-10T01:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.071+0000 7f66ae4b6700 1 --2- 172.21.15.44:0/80322445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66a8119de0 0x7f66a81a9000 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f66a000bbc0 tx=0x7f66a000bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.071+0000 7f669effd700 1 -- 172.21.15.44:0/80322445 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a000e040 con 0x7f66a8119de0 2023-09-10T01:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.071+0000 7f669effd700 1 -- 172.21.15.44:0/80322445 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66a0004e30 con 0x7f66a8119de0 2023-09-10T01:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.071+0000 7f66aff19700 1 -- 172.21.15.44:0/80322445 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a81a9bb0 con 0x7f66a8119de0 2023-09-10T01:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.071+0000 7f669effd700 1 -- 172.21.15.44:0/80322445 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a001b680 con 0x7f66a8119de0 2023-09-10T01:35:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.071+0000 7f66aff19700 1 -- 172.21.15.44:0/80322445 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a810f7a0 con 0x7f66a8119de0 2023-09-10T01:35:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.074+0000 7f669effd700 1 -- 172.21.15.44:0/80322445 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f66a0005040 con 0x7f66a8119de0 2023-09-10T01:35:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.075+0000 7f669effd700 1 --2- 172.21.15.44:0/80322445 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f66940a0830 0x7f66940a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.075+0000 7f669effd700 1 -- 172.21.15.44:0/80322445 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f66a00cd390 con 0x7f66a8119de0 2023-09-10T01:35:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.076+0000 7f66bc0dfb80 1 Processor -- start 2023-09-10T01:35:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.076+0000 7f66bc0dfb80 1 -- start start 2023-09-10T01:35:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.076+0000 7f66bc0dfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c905031a0 0x560c90503580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.076+0000 7f66bc0dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c904fd300 0x560c904fd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.076+0000 7f66bc0dfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fdcc0 0x560c90509f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.076+0000 7f66bc0dfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c904566d0 con 0x560c905031a0 2023-09-10T01:35:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.076+0000 7f66bc0dfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c90453ff0 con 0x560c904fdcc0 2023-09-10T01:35:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.076+0000 7f66bc0dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9044f8e0 con 0x560c904fd300 2023-09-10T01:35:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.077+0000 7f66ae4b6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fdcc0 0x560c90509f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.077+0000 7f66ae4b6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fdcc0 0x560c90509f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34694/0 (socket says 172.21.15.44:34694) 2023-09-10T01:35:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.077+0000 7f66ae4b6700 1 -- 172.21.15.44:0/2143918301 learned_addr learned my addr 172.21.15.44:0/2143918301 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.082+0000 7f66ae4b6700 1 -- 172.21.15.44:0/2143918301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c904fd300 msgr2=0x560c904fd780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:35:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.082+0000 7f66ae4b6700 1 --2- 172.21.15.44:0/2143918301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c904fd300 0x560c904fd780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.082+0000 7f66ae4b6700 1 -- 172.21.15.44:0/2143918301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c905031a0 msgr2=0x560c90503580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.082+0000 7f66ae4b6700 1 --2- 172.21.15.44:0/2143918301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c905031a0 0x560c90503580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.082+0000 7f66ae4b6700 1 -- 172.21.15.44:0/2143918301 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c9027bd80 con 0x560c904fdcc0 2023-09-10T01:35:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.085+0000 7f66adcb5700 1 --2- 172.21.15.44:0/80322445 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f66940a0830 0x7f66940a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.085+0000 7f66ae4b6700 1 --2- 172.21.15.44:0/2143918301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fdcc0 0x560c90509f30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f66a009a460 tx=0x7f66a009a090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4bb3dca8ec1429c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.087+0000 7f66adcb5700 1 --2- 172.21.15.44:0/80322445 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f66940a0830 0x7f66940a2cf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f66a40095a0 tx=0x7f66a4009510 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.095+0000 7f6693fff700 1 -- 172.21.15.44:0/2143918301 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a009abd0 con 0x560c904fdcc0 2023-09-10T01:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.095+0000 7f6693fff700 1 -- 172.21.15.44:0/2143918301 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66a009a4d0 con 0x560c904fdcc0 2023-09-10T01:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.095+0000 7f6693fff700 1 -- 172.21.15.44:0/2143918301 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a00052f0 con 0x560c904fdcc0 2023-09-10T01:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.096+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/2143918301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fdcc0 msgr2=0x560c90509f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.096+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/2143918301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fdcc0 0x560c90509f30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f66a009a460 tx=0x7f66a009a090 comp rx=0 tx=0).stop 2023-09-10T01:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.096+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/2143918301 shutdown_connections 2023-09-10T01:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.096+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/2143918301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fdcc0 0x560c90509f30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.096+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/2143918301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c905031a0 0x560c90503580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.096+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/2143918301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c904fd300 0x560c904fd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.096+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/2143918301 >> 172.21.15.44:0/2143918301 conn(0x560c90472030 msgr2=0x560c904f8560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.096+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/2143918301 shutdown_connections 2023-09-10T01:35:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.096+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/2143918301 wait complete. 2023-09-10T01:35:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.096+0000 7f66bc0dfb80 1 -- start start 2023-09-10T01:35:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.097+0000 7f66bc0dfb80 1 Processor -- start 2023-09-10T01:35:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.097+0000 7f66bc0dfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c9050cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.097+0000 7f66bc0dfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c904fdcc0 0x560c9050d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.097+0000 7f66bc0dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c905031a0 0x560c90506eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.097+0000 7f66bc0dfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c903edb10 con 0x560c904fdcc0 2023-09-10T01:35:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.097+0000 7f66bc0dfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c903ed810 con 0x560c904fd300 2023-09-10T01:35:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.097+0000 7f66bc0dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c902d18c0 con 0x560c905031a0 2023-09-10T01:35:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.097+0000 7f66bc0dfb80 10 client.0 _ll_register_callbacks cb 0x560c90337a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:35:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.097+0000 7f66adcb5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c9050cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.097+0000 7f66adcb5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c9050cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34696/0 (socket says 172.21.15.44:34696) 2023-09-10T01:35:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.097+0000 7f66adcb5700 1 -- 172.21.15.44:0/162613021 learned_addr learned my addr 172.21.15.44:0/162613021 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.097+0000 7f66ad4b4700 1 --2- 172.21.15.44:0/162613021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c904fdcc0 0x560c9050d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.097+0000 7f66ae4b6700 1 --2- 172.21.15.44:0/162613021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c905031a0 0x560c90506eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.098+0000 7f66adcb5700 1 -- 172.21.15.44:0/162613021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c905031a0 msgr2=0x560c90506eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.098+0000 7f66adcb5700 1 --2- 172.21.15.44:0/162613021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c905031a0 0x560c90506eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.098+0000 7f66adcb5700 1 -- 172.21.15.44:0/162613021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c904fdcc0 msgr2=0x560c9050d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.098+0000 7f66adcb5700 1 --2- 172.21.15.44:0/162613021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c904fdcc0 0x560c9050d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.098+0000 7f66adcb5700 1 -- 172.21.15.44:0/162613021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a810f7a0 con 0x560c904fd300 2023-09-10T01:35:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.098+0000 7f66ae4b6700 1 --2- 172.21.15.44:0/162613021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c905031a0 0x560c90506eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:35:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.098+0000 7f66ad4b4700 1 --2- 172.21.15.44:0/162613021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c904fdcc0 0x560c9050d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:35:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.098+0000 7f66adcb5700 1 --2- 172.21.15.44:0/162613021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c9050cc60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009350 tx=0x7f66a4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e775c0ffa9d047e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.098+0000 7f66917fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:35:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.102+0000 7f66917fa700 1 -- 172.21.15.44:0/162613021 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4025460 con 0x560c904fd300 2023-09-10T01:35:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.102+0000 7f66917fa700 1 -- 172.21.15.44:0/162613021 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(209..209 src has 1..209) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f66a40255c0 con 0x560c904fd300 2023-09-10T01:35:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.102+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/162613021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c903a7490 con 0x560c904fd300 2023-09-10T01:35:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.102+0000 7f66917fa700 1 -- 172.21.15.44:0/162613021 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=194) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66a4025780 con 0x560c904fd300 2023-09-10T01:35:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.102+0000 7f66bc0dfb80 10 client.30323 fetch_fsmap learned FSMap version 194 2023-09-10T01:35:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.102+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/162613021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=194,monmap=4+}) v3 -- 0x560c90264c00 con 0x560c904fd300 2023-09-10T01:35:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.102+0000 7f66917fa700 1 -- 172.21.15.44:0/162613021 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560c903a7490 con 0x560c904fd300 2023-09-10T01:35:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.103+0000 7f66917fa700 1 -- 172.21.15.44:0/162613021 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 194) v1 ==== 7359+0+0 (secure 0 0 0) 0x7f66a4018070 con 0x560c904fd300 2023-09-10T01:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.103+0000 7f66bc0dfb80 10 client.30323 fetch_fsmap finished waiting for FSMap version 194 2023-09-10T01:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.103+0000 7f66bc0dfb80 10 client.30323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.103+0000 7f66bc0dfb80 20 client.30323 populate_metadata read hostname 'smithi044' 2023-09-10T01:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.103+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/162613021 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c905171f0 0x560c90519610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.103+0000 7f66bc0dfb80 4 client.30323 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.103+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/162613021 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c90464840 con 0x560c905171f0 2023-09-10T01:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.106+0000 7f66ad4b4700 1 --2- 172.21.15.44:0/162613021 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c905171f0 0x560c90519610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.106+0000 7f66917fa700 10 client.30323 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.106+0000 7f66ad4b4700 1 --2- 172.21.15.44:0/162613021 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c905171f0 0x560c90519610 crc :-1 s=READY pgs=612 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9428e3bfc0992bf server_cookie=20fd227191ccae72 in_seq=0 out_seq=0 2023-09-10T01:35:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.111+0000 7f66917fa700 1 -- 172.21.15.44:0/162613021 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560c90464840 con 0x560c905171f0 2023-09-10T01:35:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.111+0000 7f66917fa700 10 client.30323 handle_command_reply: tid=0 2023-09-10T01:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.111+0000 7f66bc0dfb80 1 client.30323 shutdown 2023-09-10T01:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.111+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/162613021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 msgr2=0x560c9050cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.111+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/162613021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c9050cc60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009350 tx=0x7f66a4004750 comp rx=0 tx=0).stop 2023-09-10T01:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.111+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/162613021 shutdown_connections 2023-09-10T01:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.111+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/162613021 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c905171f0 0x560c90519610 crc :-1 s=READY pgs=612 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.111+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/162613021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c905031a0 0x560c90506eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.111+0000 7f66917fa700 0 client.30323 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:35:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.111+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/162613021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c9050cc60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.111+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/162613021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c904fdcc0 0x560c9050d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.111+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/162613021 >> 172.21.15.44:0/162613021 conn(0x560c90472030 msgr2=0x560c904f77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.111+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/162613021 shutdown_connections 2023-09-10T01:35:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.111+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/162613021 wait complete. 2023-09-10T01:35:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.111+0000 7f66bc0dfb80 20 client.30323 trim_cache size 0 max 16384 2023-09-10T01:35:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.133+0000 7f66bc0dfb80 1 Processor -- start 2023-09-10T01:35:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.133+0000 7f66bc0dfb80 1 -- start start 2023-09-10T01:35:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.133+0000 7f66bc0dfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c904f8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.133+0000 7f66bc0dfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c904fdcc0 0x560c90506e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.133+0000 7f66bc0dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c905031a0 0x560c9050c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.133+0000 7f66bc0dfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c90508030 con 0x560c904fdcc0 2023-09-10T01:35:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.133+0000 7f66bc0dfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c90464840 con 0x560c904fd300 2023-09-10T01:35:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.133+0000 7f66bc0dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c903a7490 con 0x560c905031a0 2023-09-10T01:35:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.134+0000 7f66adcb5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c904f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.134+0000 7f66adcb5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c904f8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34704/0 (socket says 172.21.15.44:34704) 2023-09-10T01:35:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.134+0000 7f66adcb5700 1 -- 172.21.15.44:0/650566260 learned_addr learned my addr 172.21.15.44:0/650566260 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.134+0000 7f66ad4b4700 1 --2- 172.21.15.44:0/650566260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c904fdcc0 0x560c90506e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.134+0000 7f66ae4b6700 1 --2- 172.21.15.44:0/650566260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c905031a0 0x560c9050c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.134+0000 7f66adcb5700 1 -- 172.21.15.44:0/650566260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c905031a0 msgr2=0x560c9050c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.134+0000 7f66adcb5700 1 --2- 172.21.15.44:0/650566260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c905031a0 0x560c9050c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.134+0000 7f66adcb5700 1 -- 172.21.15.44:0/650566260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c904fdcc0 msgr2=0x560c90506e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.134+0000 7f66adcb5700 1 --2- 172.21.15.44:0/650566260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c904fdcc0 0x560c90506e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.134+0000 7f66adcb5700 1 -- 172.21.15.44:0/650566260 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c902a21f0 con 0x560c904fd300 2023-09-10T01:35:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.134+0000 7f66adcb5700 1 --2- 172.21.15.44:0/650566260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c904f8710 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f66a401cb40 tx=0x7f66a401cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f38f861ed92b67a server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.134+0000 7f669cff9700 1 -- 172.21.15.44:0/650566260 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4010db0 con 0x560c904fd300 2023-09-10T01:35:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.134+0000 7f669cff9700 1 -- 172.21.15.44:0/650566260 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66a401c8b0 con 0x560c904fd300 2023-09-10T01:35:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.134+0000 7f669cff9700 1 -- 172.21.15.44:0/650566260 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a400fd70 con 0x560c904fd300 2023-09-10T01:35:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.135+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/650566260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 msgr2=0x560c904f8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.135+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/650566260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c904f8710 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f66a401cb40 tx=0x7f66a401cd70 comp rx=0 tx=0).stop 2023-09-10T01:35:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.135+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/650566260 shutdown_connections 2023-09-10T01:35:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.135+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/650566260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c905031a0 0x560c9050c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.135+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/650566260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c904f8710 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.135+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/650566260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c904fdcc0 0x560c90506e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.135+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/650566260 >> 172.21.15.44:0/650566260 conn(0x560c90472030 msgr2=0x560c90506750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.135+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/650566260 shutdown_connections 2023-09-10T01:35:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.136+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/650566260 wait complete. 2023-09-10T01:35:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.136+0000 7f66bc0dfb80 1 -- start start 2023-09-10T01:35:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.136+0000 7f66bc0dfb80 1 Processor -- start 2023-09-10T01:35:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.136+0000 7f66bc0dfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c9055f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.136+0000 7f66bc0dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c904fdcc0 0x560c9055f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.136+0000 7f66bc0dfb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c905031a0 0x560c90564170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.136+0000 7f66bc0dfb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c905646b0 con 0x560c905031a0 2023-09-10T01:35:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.136+0000 7f66bc0dfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c90564820 con 0x560c904fd300 2023-09-10T01:35:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.136+0000 7f66bc0dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c905649e0 con 0x560c904fdcc0 2023-09-10T01:35:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.136+0000 7f66bc0dfb80 10 client.0 _ll_register_callbacks cb 0x7f66a400ede0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:35:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.137+0000 7f66ae4b6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c905031a0 0x560c90564170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.137+0000 7f66ae4b6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c905031a0 0x560c90564170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32970/0 (socket says 172.21.15.44:32970) 2023-09-10T01:35:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.137+0000 7f66ae4b6700 1 -- 172.21.15.44:0/1971482869 learned_addr learned my addr 172.21.15.44:0/1971482869 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.137+0000 7f66adcb5700 1 --2- 172.21.15.44:0/1971482869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c9055f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.137+0000 7f66ae4b6700 1 -- 172.21.15.44:0/1971482869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c904fdcc0 msgr2=0x560c9055f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.137+0000 7f66ae4b6700 1 --2- 172.21.15.44:0/1971482869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c904fdcc0 0x560c9055f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.137+0000 7f66ae4b6700 1 -- 172.21.15.44:0/1971482869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 msgr2=0x560c9055f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.137+0000 7f66ae4b6700 1 --2- 172.21.15.44:0/1971482869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c9055f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.137+0000 7f66ae4b6700 1 -- 172.21.15.44:0/1971482869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a402a040 con 0x560c905031a0 2023-09-10T01:35:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.137+0000 7f66adcb5700 1 --2- 172.21.15.44:0/1971482869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c9055f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:35:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.137+0000 7f66ae4b6700 1 --2- 172.21.15.44:0/1971482869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c905031a0 0x560c90564170 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f66a0096f40 tx=0x7f66a0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aafc55d68cfab9e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.137+0000 7f66917fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:35:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.138+0000 7f66917fa700 1 -- 172.21.15.44:0/1971482869 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a00a24b0 con 0x560c905031a0 2023-09-10T01:35:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.138+0000 7f66917fa700 1 -- 172.21.15.44:0/1971482869 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(209..209 src has 1..209) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f66a009e300 con 0x560c905031a0 2023-09-10T01:35:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.138+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/1971482869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66a009a4d0 con 0x560c905031a0 2023-09-10T01:35:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.138+0000 7f66917fa700 1 -- 172.21.15.44:0/1971482869 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=194) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66a009a730 con 0x560c905031a0 2023-09-10T01:35:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.138+0000 7f66bc0dfb80 10 client.20688 fetch_fsmap learned FSMap version 194 2023-09-10T01:35:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.138+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/1971482869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=194,monmap=4+}) v3 -- 0x560c9027a250 con 0x560c905031a0 2023-09-10T01:35:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.138+0000 7f66917fa700 1 -- 172.21.15.44:0/1971482869 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66a009a4d0 con 0x560c905031a0 2023-09-10T01:35:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.139+0000 7f66917fa700 1 -- 172.21.15.44:0/1971482869 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 194) v1 ==== 7359+0+0 (secure 0 0 0) 0x7f66a00a2020 con 0x560c905031a0 2023-09-10T01:35:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.139+0000 7f66bc0dfb80 10 client.20688 fetch_fsmap finished waiting for FSMap version 194 2023-09-10T01:35:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.139+0000 7f66bc0dfb80 10 client.20688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:35:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.139+0000 7f66bc0dfb80 20 client.20688 populate_metadata read hostname 'smithi044' 2023-09-10T01:35:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.139+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/1971482869 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c9056be10 0x560c9056e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.139+0000 7f66bc0dfb80 4 client.20688 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:35:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.139+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/1971482869 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c904488c0 con 0x560c9056be10 2023-09-10T01:35:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.139+0000 7f66adcb5700 1 --2- 172.21.15.44:0/1971482869 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c9056be10 0x560c9056e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.139+0000 7f66adcb5700 1 --2- 172.21.15.44:0/1971482869 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c9056be10 0x560c9056e230 crc :-1 s=READY pgs=613 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c30346650c0fe4a server_cookie=b27dd1527a7e7da0 in_seq=0 out_seq=0 2023-09-10T01:35:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.139+0000 7f66917fa700 10 client.20688 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:35:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.229+0000 7f66917fa700 1 -- 172.21.15.44:0/1971482869 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c904488c0 con 0x560c9056be10 2023-09-10T01:35:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.229+0000 7f66917fa700 10 client.20688 handle_command_reply: tid=0 2023-09-10T01:35:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.229+0000 7f66bc0dfb80 1 client.20688 shutdown 2023-09-10T01:35:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.229+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/1971482869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c905031a0 msgr2=0x560c90564170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.229+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/1971482869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c905031a0 0x560c90564170 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f66a0096f40 tx=0x7f66a0009920 comp rx=0 tx=0).stop 2023-09-10T01:35:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.229+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/1971482869 shutdown_connections 2023-09-10T01:35:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.229+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/1971482869 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x560c9056be10 0x560c9056e230 crc :-1 s=READY pgs=613 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.229+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/1971482869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560c905031a0 0x560c90564170 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.229+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/1971482869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c904fd300 0x560c9055f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.229+0000 7f66bc0dfb80 1 --2- 172.21.15.44:0/1971482869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c904fdcc0 0x560c9055f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.229+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/1971482869 >> 172.21.15.44:0/1971482869 conn(0x560c90472030 msgr2=0x560c904fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.229+0000 7f66917fa700 0 client.20688 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:35:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.229+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/1971482869 shutdown_connections 2023-09-10T01:35:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.230+0000 7f66bc0dfb80 1 -- 172.21.15.44:0/1971482869 wait complete. 2023-09-10T01:35:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.230+0000 7f66bc0dfb80 20 client.20688 trim_cache size 0 max 16384 2023-09-10T01:35:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:35:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:35:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:35:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:35:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.234+0000 7f66aff19700 1 -- 172.21.15.44:0/80322445 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f66940a0830 msgr2=0x7f66940a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.234+0000 7f66aff19700 1 --2- 172.21.15.44:0/80322445 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f66940a0830 0x7f66940a2cf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f66a40095a0 tx=0x7f66a4009510 comp rx=0 tx=0).stop 2023-09-10T01:35:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.234+0000 7f66aff19700 1 -- 172.21.15.44:0/80322445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66a8119de0 msgr2=0x7f66a81a9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.234+0000 7f66aff19700 1 --2- 172.21.15.44:0/80322445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66a8119de0 0x7f66a81a9000 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f66a000bbc0 tx=0x7f66a000bca0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.234+0000 7f66aff19700 1 -- 172.21.15.44:0/80322445 shutdown_connections 2023-09-10T01:35:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.234+0000 7f66aff19700 1 --2- 172.21.15.44:0/80322445 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f66940a0830 0x7f66940a2cf0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.234+0000 7f66aff19700 1 --2- 172.21.15.44:0/80322445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66a8119de0 0x7f66a81a9000 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.234+0000 7f66aff19700 1 --2- 172.21.15.44:0/80322445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66a810ed50 0x7f66a81a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.234+0000 7f66aff19700 1 --2- 172.21.15.44:0/80322445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66a810f1c0 0x7f66a81a5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.234+0000 7f66aff19700 1 -- 172.21.15.44:0/80322445 >> 172.21.15.44:0/80322445 conn(0x7f66a806c6c0 msgr2=0x7f66a8117dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.234+0000 7f66aff19700 1 -- 172.21.15.44:0/80322445 shutdown_connections 2023-09-10T01:35:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:23.234+0000 7f66aff19700 1 -- 172.21.15.44:0/80322445 wait complete. 2023-09-10T01:35:24.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:23 smithi044 ceph-mon[98676]: pgmap v1784: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 564 op/s 2023-09-10T01:35:24.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:23 smithi099 ceph-mon[102692]: pgmap v1784: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 564 op/s 2023-09-10T01:35:24.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:23 smithi131 ceph-mon[103713]: pgmap v1784: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 564 op/s 2023-09-10T01:35:26.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:25 smithi099 ceph-mon[102692]: pgmap v1785: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 613 op/s 2023-09-10T01:35:26.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 860.0 s, 268.0 tps, lat 359.904 ms stddev 760.629 2023-09-10T01:35:26.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:25 smithi131 ceph-mon[103713]: pgmap v1785: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 613 op/s 2023-09-10T01:35:26.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:25 smithi044 ceph-mon[98676]: pgmap v1785: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 613 op/s 2023-09-10T01:35:28.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:27 smithi099 ceph-mon[102692]: pgmap v1786: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 646 op/s 2023-09-10T01:35:28.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:27 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:35:28.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:27 smithi131 ceph-mon[103713]: pgmap v1786: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 646 op/s 2023-09-10T01:35:28.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:27 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:35:28.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:27 smithi044 ceph-mon[98676]: pgmap v1786: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 646 op/s 2023-09-10T01:35:28.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:27 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:35:30.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:29 smithi131 ceph-mon[103713]: pgmap v1787: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 754 op/s 2023-09-10T01:35:30.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:29 smithi044 ceph-mon[98676]: pgmap v1787: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 754 op/s 2023-09-10T01:35:30.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:29 smithi099 ceph-mon[102692]: pgmap v1787: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 754 op/s 2023-09-10T01:35:31.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 865.0 s, 250.6 tps, lat 398.432 ms stddev 806.118 2023-09-10T01:35:32.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:31 smithi131 ceph-mon[103713]: pgmap v1788: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 773 op/s 2023-09-10T01:35:32.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:31 smithi044 ceph-mon[98676]: pgmap v1788: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 773 op/s 2023-09-10T01:35:32.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:31 smithi099 ceph-mon[102692]: pgmap v1788: 129 pgs: 129 active+clean; 17 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 773 op/s 2023-09-10T01:35:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:32 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:35:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:32 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:35:33.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:32 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:35:34.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:33 smithi131 ceph-mon[103713]: pgmap v1789: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.5 MiB/s wr, 801 op/s 2023-09-10T01:35:34.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:33 smithi044 ceph-mon[98676]: pgmap v1789: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.5 MiB/s wr, 801 op/s 2023-09-10T01:35:34.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:33 smithi099 ceph-mon[102692]: pgmap v1789: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.5 MiB/s wr, 801 op/s 2023-09-10T01:35:36.183 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 870.0 s, 157.9 tps, lat 717.540 ms stddev 1263.966 2023-09-10T01:35:36.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:35 smithi131 ceph-mon[103713]: pgmap v1790: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 745 op/s 2023-09-10T01:35:36.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:35 smithi044 ceph-mon[98676]: pgmap v1790: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 745 op/s 2023-09-10T01:35:36.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:35 smithi099 ceph-mon[102692]: pgmap v1790: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 745 op/s 2023-09-10T01:35:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.450+0000 7feadbcb4700 1 -- 172.21.15.44:0/1717515602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fead4106d20 msgr2=0x7fead41071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.450+0000 7feadbcb4700 1 --2- 172.21.15.44:0/1717515602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fead4106d20 0x7fead41071a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fead0009990 tx=0x7fead0009ca0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.468+0000 7feadbcb4700 1 -- 172.21.15.44:0/1717515602 shutdown_connections 2023-09-10T01:35:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.468+0000 7feadbcb4700 1 --2- 172.21.15.44:0/1717515602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead41076e0 0x7fead4113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.469+0000 7feadbcb4700 1 --2- 172.21.15.44:0/1717515602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fead410cc80 0x7fead410d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.469+0000 7feadbcb4700 1 --2- 172.21.15.44:0/1717515602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fead4106d20 0x7fead41071a0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.469+0000 7feadbcb4700 1 -- 172.21.15.44:0/1717515602 >> 172.21.15.44:0/1717515602 conn(0x7fead40764a0 msgr2=0x7fead4105420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.471+0000 7feadbcb4700 1 -- 172.21.15.44:0/1717515602 shutdown_connections 2023-09-10T01:35:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.471+0000 7feadbcb4700 1 -- 172.21.15.44:0/1717515602 wait complete. 2023-09-10T01:35:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.472+0000 7feadbcb4700 1 Processor -- start 2023-09-10T01:35:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.480+0000 7feadbcb4700 1 -- start start 2023-09-10T01:35:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.480+0000 7feadbcb4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead4106d20 0x7fead41a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.480+0000 7feadbcb4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fead41076e0 0x7fead41a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.480+0000 7feadbcb4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fead410cc80 0x7fead419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.480+0000 7feadbcb4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead419d1e0 con 0x7fead41076e0 2023-09-10T01:35:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.480+0000 7feadbcb4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead419d320 con 0x7fead4106d20 2023-09-10T01:35:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.480+0000 7feadbcb4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead419d4e0 con 0x7fead410cc80 2023-09-10T01:35:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.481+0000 7feada251700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fead410cc80 0x7fead419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.481+0000 7feada251700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fead410cc80 0x7fead419ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39866/0 (socket says 172.21.15.44:39866) 2023-09-10T01:35:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.481+0000 7feada251700 1 -- 172.21.15.44:0/3248171910 learned_addr learned my addr 172.21.15.44:0/3248171910 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.481+0000 7fead9a50700 1 --2- 172.21.15.44:0/3248171910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead4106d20 0x7fead41a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.487+0000 7feada251700 1 -- 172.21.15.44:0/3248171910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead4106d20 msgr2=0x7fead41a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.487+0000 7feada251700 1 --2- 172.21.15.44:0/3248171910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead4106d20 0x7fead41a2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.487+0000 7feada251700 1 -- 172.21.15.44:0/3248171910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fead41076e0 msgr2=0x7fead41a30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.487+0000 7feada251700 1 --2- 172.21.15.44:0/3248171910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fead41076e0 0x7fead41a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.487+0000 7feada251700 1 -- 172.21.15.44:0/3248171910 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead00095f0 con 0x7fead410cc80 2023-09-10T01:35:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.488+0000 7feada251700 1 --2- 172.21.15.44:0/3248171910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fead410cc80 0x7fead419ca10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7feacc007ee0 tx=0x7feacc00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.488+0000 7feacaffd700 1 -- 172.21.15.44:0/3248171910 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc017070 con 0x7fead410cc80 2023-09-10T01:35:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.489+0000 7feadbcb4700 1 -- 172.21.15.44:0/3248171910 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead419d7c0 con 0x7fead410cc80 2023-09-10T01:35:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.489+0000 7feadbcb4700 1 -- 172.21.15.44:0/3248171910 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead4079080 con 0x7fead410cc80 2023-09-10T01:35:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.489+0000 7feacaffd700 1 -- 172.21.15.44:0/3248171910 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feacc00f040 con 0x7fead410cc80 2023-09-10T01:35:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.489+0000 7feacaffd700 1 -- 172.21.15.44:0/3248171910 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc005710 con 0x7fead410cc80 2023-09-10T01:35:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.495+0000 7feacaffd700 1 -- 172.21.15.44:0/3248171910 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7feacc005870 con 0x7fead410cc80 2023-09-10T01:35:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.495+0000 7feacaffd700 1 --2- 172.21.15.44:0/3248171910 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feac00a08f0 0x7feac00a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.496+0000 7feacaffd700 1 -- 172.21.15.44:0/3248171910 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 9540+0+0 (secure 0 0 0) 0x7feacc0cbb60 con 0x7fead410cc80 2023-09-10T01:35:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.506+0000 7fead9a50700 1 --2- 172.21.15.44:0/3248171910 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feac00a08f0 0x7feac00a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.507+0000 7feae7e7ab80 1 Processor -- start 2023-09-10T01:35:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.508+0000 7feae7e7ab80 1 -- start start 2023-09-10T01:35:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.508+0000 7fead9a50700 1 --2- 172.21.15.44:0/3248171910 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feac00a08f0 0x7feac00a2db0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7feac4005fd0 tx=0x7feac4005dd0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.508+0000 7feae7e7ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 0x564762d8f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.508+0000 7feae7e7ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d89300 0x564762d89780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.508+0000 7feae7e7ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 0x564762d95f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.508+0000 7feae7e7ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564762ce26d0 con 0x564762d89300 2023-09-10T01:35:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.508+0000 7feae7e7ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564762cdfff0 con 0x564762d89cc0 2023-09-10T01:35:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.508+0000 7feae7e7ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564762cdb8e0 con 0x564762d8f1a0 2023-09-10T01:35:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.509+0000 7fead9a50700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 0x564762d8f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.509+0000 7fead9a50700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 0x564762d8f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39874/0 (socket says 172.21.15.44:39874) 2023-09-10T01:35:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.509+0000 7fead9a50700 1 -- 172.21.15.44:0/1428990719 learned_addr learned my addr 172.21.15.44:0/1428990719 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.510+0000 7feada251700 1 --2- 172.21.15.44:0/1428990719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 0x564762d95f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.510+0000 7fead9a50700 1 -- 172.21.15.44:0/1428990719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 msgr2=0x564762d95f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.510+0000 7fead9a50700 1 --2- 172.21.15.44:0/1428990719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 0x564762d95f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.510+0000 7fead9a50700 1 -- 172.21.15.44:0/1428990719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d89300 msgr2=0x564762d89780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.510+0000 7fead9a50700 1 --2- 172.21.15.44:0/1428990719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d89300 0x564762d89780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.510+0000 7fead9a50700 1 -- 172.21.15.44:0/1428990719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564762b07d80 con 0x564762d8f1a0 2023-09-10T01:35:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.510+0000 7fead9a50700 1 --2- 172.21.15.44:0/1428990719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 0x564762d8f580 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7feac400ec60 tx=0x7feac4011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2edd5f94d3e79371 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.518+0000 7feabffff700 1 -- 172.21.15.44:0/1428990719 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4010d90 con 0x564762d8f1a0 2023-09-10T01:35:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.518+0000 7feabffff700 1 -- 172.21.15.44:0/1428990719 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feac40044b0 con 0x564762d8f1a0 2023-09-10T01:35:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.518+0000 7feabffff700 1 -- 172.21.15.44:0/1428990719 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4003dd0 con 0x564762d8f1a0 2023-09-10T01:35:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.519+0000 7feae7e7ab80 1 -- 172.21.15.44:0/1428990719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 msgr2=0x564762d8f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.519+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/1428990719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 0x564762d8f580 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7feac400ec60 tx=0x7feac4011000 comp rx=0 tx=0).stop 2023-09-10T01:35:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.519+0000 7feae7e7ab80 1 -- 172.21.15.44:0/1428990719 shutdown_connections 2023-09-10T01:35:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.519+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/1428990719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 0x564762d95f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.520+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/1428990719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 0x564762d8f580 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.520+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/1428990719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d89300 0x564762d89780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.520+0000 7feae7e7ab80 1 -- 172.21.15.44:0/1428990719 >> 172.21.15.44:0/1428990719 conn(0x564762cfe030 msgr2=0x564762d84560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.520+0000 7feae7e7ab80 1 -- 172.21.15.44:0/1428990719 shutdown_connections 2023-09-10T01:35:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.520+0000 7feae7e7ab80 1 -- 172.21.15.44:0/1428990719 wait complete. 2023-09-10T01:35:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.521+0000 7feae7e7ab80 1 -- start start 2023-09-10T01:35:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.521+0000 7feae7e7ab80 1 Processor -- start 2023-09-10T01:35:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.521+0000 7feae7e7ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d89300 0x564762d95eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.521+0000 7feae7e7ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 0x564762d93130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.521+0000 7feae7e7ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 0x564762d93870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.521+0000 7feae7e7ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564762c79b10 con 0x564762d89300 2023-09-10T01:35:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.521+0000 7feae7e7ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564762c79810 con 0x564762d89cc0 2023-09-10T01:35:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.521+0000 7feae7e7ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564762b5d8c0 con 0x564762d8f1a0 2023-09-10T01:35:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.522+0000 7feae7e7ab80 10 client.0 _ll_register_callbacks cb 0x564762bc3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:35:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.522+0000 7feada251700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 0x564762d93870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.522+0000 7feada251700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 0x564762d93870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39882/0 (socket says 172.21.15.44:39882) 2023-09-10T01:35:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.522+0000 7feada251700 1 -- 172.21.15.44:0/236384828 learned_addr learned my addr 172.21.15.44:0/236384828 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.522+0000 7fead924f700 1 --2- 172.21.15.44:0/236384828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 0x564762d93130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.522+0000 7feada251700 1 -- 172.21.15.44:0/236384828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 msgr2=0x564762d93130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.522+0000 7feada251700 1 --2- 172.21.15.44:0/236384828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 0x564762d93130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.522+0000 7feada251700 1 -- 172.21.15.44:0/236384828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d89300 msgr2=0x564762d95eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.522+0000 7feada251700 1 --2- 172.21.15.44:0/236384828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d89300 0x564762d95eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.522+0000 7feada251700 1 -- 172.21.15.44:0/236384828 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac400e910 con 0x564762d8f1a0 2023-09-10T01:35:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.522+0000 7fead924f700 1 --2- 172.21.15.44:0/236384828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 0x564762d93130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:35:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.523+0000 7feada251700 1 --2- 172.21.15.44:0/236384828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 0x564762d93870 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7feacc0950b0 tx=0x7feacc004c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0e6f095efe4344 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.523+0000 7feabd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:35:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.523+0000 7feabd7fa700 1 -- 172.21.15.44:0/236384828 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc0b82c0 con 0x564762d8f1a0 2023-09-10T01:35:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.523+0000 7feae7e7ab80 1 -- 172.21.15.44:0/236384828 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564762c33490 con 0x564762d8f1a0 2023-09-10T01:35:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.523+0000 7feabd7fa700 1 -- 172.21.15.44:0/236384828 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(209..209 src has 1..209) v4 ==== 9540+0+0 (secure 0 0 0) 0x7feacc09c9e0 con 0x564762d8f1a0 2023-09-10T01:35:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.523+0000 7feabd7fa700 1 -- 172.21.15.44:0/236384828 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=196) v2 ==== 24+0+0 (secure 0 0 0) 0x7feacc0a0c20 con 0x564762d8f1a0 2023-09-10T01:35:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.524+0000 7feae7e7ab80 10 client.30379 fetch_fsmap learned FSMap version 196 2023-09-10T01:35:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.524+0000 7feae7e7ab80 1 -- 172.21.15.44:0/236384828 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=196,monmap=4+}) v3 -- 0x564762af0c00 con 0x564762d8f1a0 2023-09-10T01:35:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.532+0000 7feabd7fa700 1 -- 172.21.15.44:0/236384828 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564762c33490 con 0x564762d8f1a0 2023-09-10T01:35:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.532+0000 7feabd7fa700 1 -- 172.21.15.44:0/236384828 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 196) v1 ==== 7371+0+0 (secure 0 0 0) 0x7feacc098e10 con 0x564762d8f1a0 2023-09-10T01:35:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.532+0000 7feae7e7ab80 10 client.30379 fetch_fsmap finished waiting for FSMap version 196 2023-09-10T01:35:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.532+0000 7feae7e7ab80 10 client.30379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:35:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.532+0000 7feae7e7ab80 20 client.30379 populate_metadata read hostname 'smithi044' 2023-09-10T01:35:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.532+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/236384828 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564762da3560 0x564762da5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.532+0000 7feae7e7ab80 4 client.30379 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:35:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.532+0000 7feae7e7ab80 1 -- 172.21.15.44:0/236384828 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564762cf0840 con 0x564762da3560 2023-09-10T01:35:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.542+0000 7fead9a50700 1 --2- 172.21.15.44:0/236384828 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564762da3560 0x564762da5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.562+0000 7fead9a50700 1 --2- 172.21.15.44:0/236384828 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564762da3560 0x564762da5980 crc :-1 s=READY pgs=614 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d82159b38aa4860 server_cookie=31008e3bf2f286e8 in_seq=0 out_seq=0 2023-09-10T01:35:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.562+0000 7feabd7fa700 10 client.30379 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:35:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.563+0000 7feabd7fa700 1 -- 172.21.15.44:0/236384828 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564762cf0840 con 0x564762da3560 2023-09-10T01:35:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.563+0000 7feabd7fa700 10 client.30379 handle_command_reply: tid=0 2023-09-10T01:35:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.563+0000 7feae7e7ab80 1 client.30379 shutdown 2023-09-10T01:35:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.564+0000 7feae7e7ab80 1 -- 172.21.15.44:0/236384828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 msgr2=0x564762d93870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.564+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/236384828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 0x564762d93870 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7feacc0950b0 tx=0x7feacc004c10 comp rx=0 tx=0).stop 2023-09-10T01:35:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.564+0000 7feae7e7ab80 1 -- 172.21.15.44:0/236384828 shutdown_connections 2023-09-10T01:35:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.564+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/236384828 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564762da3560 0x564762da5980 crc :-1 s=READY pgs=614 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.564+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/236384828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 0x564762d93870 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.564+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/236384828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d89300 0x564762d95eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.564+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/236384828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 0x564762d93130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.564+0000 7feae7e7ab80 1 -- 172.21.15.44:0/236384828 >> 172.21.15.44:0/236384828 conn(0x564762cfe030 msgr2=0x564762d837a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.564+0000 7feabd7fa700 0 client.30379 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:35:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.564+0000 7feae7e7ab80 1 -- 172.21.15.44:0/236384828 shutdown_connections 2023-09-10T01:35:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.564+0000 7feae7e7ab80 1 -- 172.21.15.44:0/236384828 wait complete. 2023-09-10T01:35:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.564+0000 7feae7e7ab80 20 client.30379 trim_cache size 0 max 16384 2023-09-10T01:35:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.602+0000 7feae7e7ab80 1 Processor -- start 2023-09-10T01:35:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.613+0000 7feae7e7ab80 1 -- start start 2023-09-10T01:35:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.613+0000 7feae7e7ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d89300 0x564762d90d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.613+0000 7feae7e7ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 0x564762d959e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.613+0000 7feae7e7ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 0x564762d84090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.613+0000 7feae7e7ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564762d94110 con 0x564762d89300 2023-09-10T01:35:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.613+0000 7feae7e7ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564762d8e9d0 con 0x564762d89cc0 2023-09-10T01:35:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.613+0000 7feae7e7ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564762d8dd30 con 0x564762d8f1a0 2023-09-10T01:35:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.613+0000 7fead9a50700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d89300 0x564762d90d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.613+0000 7fead924f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 0x564762d959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.613+0000 7fead9a50700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d89300 0x564762d90d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33006/0 (socket says 172.21.15.44:33006) 2023-09-10T01:35:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.613+0000 7fead9a50700 1 -- 172.21.15.44:0/3841585566 learned_addr learned my addr 172.21.15.44:0/3841585566 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.614+0000 7fead924f700 1 -- 172.21.15.44:0/3841585566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 msgr2=0x564762d84090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-10T01:35:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.614+0000 7fead924f700 1 --2- 172.21.15.44:0/3841585566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 0x564762d84090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.614+0000 7fead924f700 1 -- 172.21.15.44:0/3841585566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d89300 msgr2=0x564762d90d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.614+0000 7fead924f700 1 --2- 172.21.15.44:0/3841585566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d89300 0x564762d90d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.614+0000 7fead924f700 1 -- 172.21.15.44:0/3841585566 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564762b2e1f0 con 0x564762d89cc0 2023-09-10T01:35:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.614+0000 7fead924f700 1 --2- 172.21.15.44:0/3841585566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 0x564762d959e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x564762d84060 tx=0x7fead0009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9497716e730a105 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.614+0000 7feac8d89700 1 -- 172.21.15.44:0/3841585566 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead000b9d0 con 0x564762d89cc0 2023-09-10T01:35:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.621+0000 7feac8d89700 1 -- 172.21.15.44:0/3841585566 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fead001e040 con 0x564762d89cc0 2023-09-10T01:35:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.621+0000 7feac8d89700 1 -- 172.21.15.44:0/3841585566 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead000f460 con 0x564762d89cc0 2023-09-10T01:35:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.622+0000 7feae7e7ab80 1 -- 172.21.15.44:0/3841585566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 msgr2=0x564762d959e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.622+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/3841585566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 0x564762d959e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x564762d84060 tx=0x7fead0009ee0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.622+0000 7feae7e7ab80 1 -- 172.21.15.44:0/3841585566 shutdown_connections 2023-09-10T01:35:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.622+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/3841585566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d8f1a0 0x564762d84090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.622+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/3841585566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d89300 0x564762d90d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.622+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/3841585566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89cc0 0x564762d959e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.622+0000 7feae7e7ab80 1 -- 172.21.15.44:0/3841585566 >> 172.21.15.44:0/3841585566 conn(0x564762cfe030 msgr2=0x564762d8d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.623+0000 7feae7e7ab80 1 -- 172.21.15.44:0/3841585566 shutdown_connections 2023-09-10T01:35:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.623+0000 7feae7e7ab80 1 -- 172.21.15.44:0/3841585566 wait complete. 2023-09-10T01:35:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.623+0000 7feae7e7ab80 1 -- start start 2023-09-10T01:35:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.623+0000 7feae7e7ab80 1 Processor -- start 2023-09-10T01:35:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.624+0000 7feae7e7ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89300 0x564762d98700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.624+0000 7feae7e7ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d89cc0 0x564762d98c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.624+0000 7feae7e7ab80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d8f1a0 0x564762df0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.624+0000 7feae7e7ab80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564762df08a0 con 0x564762d8f1a0 2023-09-10T01:35:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.624+0000 7feae7e7ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564762df09e0 con 0x564762d89300 2023-09-10T01:35:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.624+0000 7feae7e7ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564762df0ba0 con 0x564762d89cc0 2023-09-10T01:35:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.624+0000 7feae7e7ab80 10 client.0 _ll_register_callbacks cb 0x7feacc098510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:35:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.624+0000 7fead9a50700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89300 0x564762d98700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.624+0000 7feada251700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d8f1a0 0x564762df0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.624+0000 7feada251700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d8f1a0 0x564762df0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33016/0 (socket says 172.21.15.44:33016) 2023-09-10T01:35:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.624+0000 7feada251700 1 -- 172.21.15.44:0/676023127 learned_addr learned my addr 172.21.15.44:0/676023127 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.625+0000 7fead924f700 1 --2- 172.21.15.44:0/676023127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d89cc0 0x564762d98c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.625+0000 7feada251700 1 -- 172.21.15.44:0/676023127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d89cc0 msgr2=0x564762d98c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.625+0000 7feada251700 1 --2- 172.21.15.44:0/676023127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d89cc0 0x564762d98c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.625+0000 7feada251700 1 -- 172.21.15.44:0/676023127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89300 msgr2=0x564762d98700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.625+0000 7feada251700 1 --2- 172.21.15.44:0/676023127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89300 0x564762d98700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.625+0000 7feada251700 1 -- 172.21.15.44:0/676023127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead4114370 con 0x564762d8f1a0 2023-09-10T01:35:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.625+0000 7fead924f700 1 --2- 172.21.15.44:0/676023127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d89cc0 0x564762d98c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:35:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.626+0000 7feada251700 1 --2- 172.21.15.44:0/676023127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d8f1a0 0x564762df0360 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7feacc098860 tx=0x7feacc094ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab5115d803c99d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.626+0000 7feabd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:35:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.626+0000 7feabd7fa700 1 -- 172.21.15.44:0/676023127 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc0b82c0 con 0x564762d8f1a0 2023-09-10T01:35:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.626+0000 7feabd7fa700 1 -- 172.21.15.44:0/676023127 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(209..209 src has 1..209) v4 ==== 9540+0+0 (secure 0 0 0) 0x7feacc00dde0 con 0x564762d8f1a0 2023-09-10T01:35:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.627+0000 7feae7e7ab80 1 -- 172.21.15.44:0/676023127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feac40044b0 con 0x564762d8f1a0 2023-09-10T01:35:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.627+0000 7feabd7fa700 1 -- 172.21.15.44:0/676023127 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=196) v2 ==== 24+0+0 (secure 0 0 0) 0x7feacc0a0e70 con 0x564762d8f1a0 2023-09-10T01:35:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.627+0000 7feae7e7ab80 10 client.20697 fetch_fsmap learned FSMap version 196 2023-09-10T01:35:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.627+0000 7feae7e7ab80 1 -- 172.21.15.44:0/676023127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=196,monmap=4+}) v3 -- 0x564762b06250 con 0x564762d8f1a0 2023-09-10T01:35:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.628+0000 7feabd7fa700 1 -- 172.21.15.44:0/676023127 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feacc0a4020 con 0x564762d8f1a0 2023-09-10T01:35:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.628+0000 7feabd7fa700 1 -- 172.21.15.44:0/676023127 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 196) v1 ==== 7371+0+0 (secure 0 0 0) 0x7feacc099bc0 con 0x564762d8f1a0 2023-09-10T01:35:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.628+0000 7feae7e7ab80 10 client.20697 fetch_fsmap finished waiting for FSMap version 196 2023-09-10T01:35:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.628+0000 7feae7e7ab80 10 client.20697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:35:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.628+0000 7feae7e7ab80 20 client.20697 populate_metadata read hostname 'smithi044' 2023-09-10T01:35:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.628+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/676023127 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564762df2050 0x564762df2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.629+0000 7feae7e7ab80 4 client.20697 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:35:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.629+0000 7feae7e7ab80 1 -- 172.21.15.44:0/676023127 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564762cd48c0 con 0x564762df2050 2023-09-10T01:35:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.629+0000 7fead9a50700 1 --2- 172.21.15.44:0/676023127 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564762df2050 0x564762df2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.630+0000 7fead9a50700 1 --2- 172.21.15.44:0/676023127 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564762df2050 0x564762df2460 crc :-1 s=READY pgs=615 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67b4540941229838 server_cookie=fbe2a00dc76f917 in_seq=0 out_seq=0 2023-09-10T01:35:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.630+0000 7feabd7fa700 10 client.20697 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:35:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.675+0000 7feabd7fa700 1 -- 172.21.15.44:0/676023127 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564762cd48c0 con 0x564762df2050 2023-09-10T01:35:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.675+0000 7feabd7fa700 10 client.20697 handle_command_reply: tid=0 2023-09-10T01:35:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.675+0000 7feae7e7ab80 1 client.20697 shutdown 2023-09-10T01:35:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.680+0000 7feae7e7ab80 1 -- 172.21.15.44:0/676023127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d8f1a0 msgr2=0x564762df0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.680+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/676023127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d8f1a0 0x564762df0360 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7feacc098860 tx=0x7feacc094ff0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.680+0000 7feae7e7ab80 1 -- 172.21.15.44:0/676023127 shutdown_connections 2023-09-10T01:35:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.680+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/676023127 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x564762df2050 0x564762df2460 crc :-1 s=READY pgs=615 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.680+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/676023127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564762d8f1a0 0x564762df0360 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.680+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/676023127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564762d89300 0x564762d98700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.680+0000 7feae7e7ab80 1 --2- 172.21.15.44:0/676023127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564762d89cc0 0x564762d98c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.680+0000 7feae7e7ab80 1 -- 172.21.15.44:0/676023127 >> 172.21.15.44:0/676023127 conn(0x564762cfe030 msgr2=0x564762d84090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.680+0000 7feabd7fa700 0 client.20697 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:35:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.686+0000 7feae7e7ab80 1 -- 172.21.15.44:0/676023127 shutdown_connections 2023-09-10T01:35:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.686+0000 7feae7e7ab80 1 -- 172.21.15.44:0/676023127 wait complete. 2023-09-10T01:35:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.693+0000 7feae7e7ab80 20 client.20697 trim_cache size 0 max 16384 2023-09-10T01:35:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:35:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:35:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:35:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:35:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.693+0000 7feadbcb4700 1 -- 172.21.15.44:0/3248171910 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feac00a08f0 msgr2=0x7feac00a2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.693+0000 7feadbcb4700 1 --2- 172.21.15.44:0/3248171910 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feac00a08f0 0x7feac00a2db0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7feac4005fd0 tx=0x7feac4005dd0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.693+0000 7feadbcb4700 1 -- 172.21.15.44:0/3248171910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fead410cc80 msgr2=0x7fead419ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.693+0000 7feadbcb4700 1 --2- 172.21.15.44:0/3248171910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fead410cc80 0x7fead419ca10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7feacc007ee0 tx=0x7feacc00d3b0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.696+0000 7feadbcb4700 1 -- 172.21.15.44:0/3248171910 shutdown_connections 2023-09-10T01:35:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.696+0000 7feadbcb4700 1 --2- 172.21.15.44:0/3248171910 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feac00a08f0 0x7feac00a2db0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.696+0000 7feadbcb4700 1 --2- 172.21.15.44:0/3248171910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fead410cc80 0x7fead419ca10 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.696+0000 7feadbcb4700 1 --2- 172.21.15.44:0/3248171910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead4106d20 0x7fead41a2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.696+0000 7feadbcb4700 1 --2- 172.21.15.44:0/3248171910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fead41076e0 0x7fead41a30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.696+0000 7feadbcb4700 1 -- 172.21.15.44:0/3248171910 >> 172.21.15.44:0/3248171910 conn(0x7fead40764a0 msgr2=0x7fead40773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.710+0000 7feadbcb4700 1 -- 172.21.15.44:0/3248171910 shutdown_connections 2023-09-10T01:35:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:36.710+0000 7feadbcb4700 1 -- 172.21.15.44:0/3248171910 wait complete. 2023-09-10T01:35:37.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:36 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:35:37.330 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:36 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:35:37.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:36 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:35:38.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:37 smithi131 ceph-mon[103713]: pgmap v1791: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.9 MiB/s wr, 721 op/s 2023-09-10T01:35:38.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:37 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:35:38.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:37 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:35:38.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:37 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:35:38.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:37 smithi044 ceph-mon[98676]: pgmap v1791: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.9 MiB/s wr, 721 op/s 2023-09-10T01:35:38.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:37 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:35:38.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:37 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:35:38.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:37 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:35:38.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:37 smithi099 ceph-mon[102692]: pgmap v1791: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.9 MiB/s wr, 721 op/s 2023-09-10T01:35:38.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:37 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:35:38.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:37 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:35:38.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:37 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:35:39.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:38 smithi131 ceph-mon[103713]: pgmap v1792: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.6 MiB/s wr, 850 op/s 2023-09-10T01:35:39.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:38 smithi044 ceph-mon[98676]: pgmap v1792: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.6 MiB/s wr, 850 op/s 2023-09-10T01:35:39.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:38 smithi099 ceph-mon[102692]: pgmap v1792: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.6 MiB/s wr, 850 op/s 2023-09-10T01:35:41.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 875.0 s, 263.1 tps, lat 308.894 ms stddev 355.116 2023-09-10T01:35:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:40 smithi131 ceph-mon[103713]: pgmap v1793: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 622 op/s 2023-09-10T01:35:41.310 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:40 smithi044 ceph-mon[98676]: pgmap v1793: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 622 op/s 2023-09-10T01:35:41.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:40 smithi099 ceph-mon[102692]: pgmap v1793: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 622 op/s 2023-09-10T01:35:43.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:43 smithi131 ceph-mon[103713]: pgmap v1794: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.6 MiB/s wr, 794 op/s 2023-09-10T01:35:43.316 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:43 smithi044 ceph-mon[98676]: pgmap v1794: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.6 MiB/s wr, 794 op/s 2023-09-10T01:35:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:43 smithi099 ceph-mon[102692]: pgmap v1794: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.6 MiB/s wr, 794 op/s 2023-09-10T01:35:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:45 smithi131 ceph-mon[103713]: pgmap v1795: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.4 MiB/s wr, 678 op/s 2023-09-10T01:35:45.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:45 smithi044 ceph-mon[98676]: pgmap v1795: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.4 MiB/s wr, 678 op/s 2023-09-10T01:35:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:45 smithi099 ceph-mon[102692]: pgmap v1795: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.4 MiB/s wr, 678 op/s 2023-09-10T01:35:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 880.0 s, 165.2 tps, lat 719.126 ms stddev 1666.811 2023-09-10T01:35:47.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:47 smithi131 ceph-mon[103713]: pgmap v1796: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 652 op/s 2023-09-10T01:35:47.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:47 smithi044 ceph-mon[98676]: pgmap v1796: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 652 op/s 2023-09-10T01:35:47.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:47 smithi099 ceph-mon[102692]: pgmap v1796: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 652 op/s 2023-09-10T01:35:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:49 smithi131 ceph-mon[103713]: pgmap v1797: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 859 op/s 2023-09-10T01:35:49.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:49 smithi044 ceph-mon[98676]: pgmap v1797: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 859 op/s 2023-09-10T01:35:49.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:49 smithi099 ceph-mon[102692]: pgmap v1797: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 859 op/s 2023-09-10T01:35:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.827+0000 7f8d3bb29700 1 -- 172.21.15.44:0/370495662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d34100120 msgr2=0x7f8d3410c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.827+0000 7f8d3bb29700 1 --2- 172.21.15.44:0/370495662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d34100120 0x7f8d3410c330 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00b0d0 tx=0x7f8d2c00b3e0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.827+0000 7f8d3bb29700 1 -- 172.21.15.44:0/370495662 shutdown_connections 2023-09-10T01:35:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.827+0000 7f8d3bb29700 1 --2- 172.21.15.44:0/370495662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d34100120 0x7f8d3410c330 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.827+0000 7f8d3bb29700 1 --2- 172.21.15.44:0/370495662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d341055b0 0x7f8d34105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.827+0000 7f8d3bb29700 1 --2- 172.21.15.44:0/370495662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d340ff760 0x7f8d340ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.827+0000 7f8d3bb29700 1 -- 172.21.15.44:0/370495662 >> 172.21.15.44:0/370495662 conn(0x7f8d34074f30 msgr2=0x7f8d34075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.827+0000 7f8d3bb29700 1 -- 172.21.15.44:0/370495662 shutdown_connections 2023-09-10T01:35:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.828+0000 7f8d3bb29700 1 -- 172.21.15.44:0/370495662 wait complete. 2023-09-10T01:35:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.828+0000 7f8d3bb29700 1 Processor -- start 2023-09-10T01:35:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.828+0000 7f8d3bb29700 1 -- start start 2023-09-10T01:35:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d3bb29700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d340ff760 0x7f8d3419b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d3bb29700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d34100120 0x7f8d3419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d3bb29700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d341055b0 0x7f8d341954a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d3bb29700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d34195b50 con 0x7f8d34100120 2023-09-10T01:35:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d3bb29700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d34195cc0 con 0x7f8d340ff760 2023-09-10T01:35:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d3bb29700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d34195e80 con 0x7f8d341055b0 2023-09-10T01:35:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d390c4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d34100120 0x7f8d3419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d390c4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d34100120 0x7f8d3419bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33028/0 (socket says 172.21.15.44:33028) 2023-09-10T01:35:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d3a0c6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d341055b0 0x7f8d341954a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d390c4700 1 -- 172.21.15.44:0/3748891391 learned_addr learned my addr 172.21.15.44:0/3748891391 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d398c5700 1 --2- 172.21.15.44:0/3748891391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d340ff760 0x7f8d3419b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d390c4700 1 -- 172.21.15.44:0/3748891391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d341055b0 msgr2=0x7f8d341954a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d390c4700 1 --2- 172.21.15.44:0/3748891391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d341055b0 0x7f8d341954a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d390c4700 1 -- 172.21.15.44:0/3748891391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d340ff760 msgr2=0x7f8d3419b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d390c4700 1 --2- 172.21.15.44:0/3748891391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d340ff760 0x7f8d3419b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d390c4700 1 -- 172.21.15.44:0/3748891391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c009d00 con 0x7f8d34100120 2023-09-10T01:35:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d398c5700 1 --2- 172.21.15.44:0/3748891391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d340ff760 0x7f8d3419b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:35:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d390c4700 1 --2- 172.21.15.44:0/3748891391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d34100120 0x7f8d3419bb60 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400d950 tx=0x7f8d2400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d2affd700 1 -- 172.21.15.44:0/3748891391 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24009940 con 0x7f8d34100120 2023-09-10T01:35:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d2affd700 1 -- 172.21.15.44:0/3748891391 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d24010460 con 0x7f8d34100120 2023-09-10T01:35:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.829+0000 7f8d3bb29700 1 -- 172.21.15.44:0/3748891391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d34196160 con 0x7f8d34100120 2023-09-10T01:35:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.830+0000 7f8d2affd700 1 -- 172.21.15.44:0/3748891391 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400f620 con 0x7f8d34100120 2023-09-10T01:35:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.830+0000 7f8d3bb29700 1 -- 172.21.15.44:0/3748891391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d341a2650 con 0x7f8d34100120 2023-09-10T01:35:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.832+0000 7f8d2affd700 1 -- 172.21.15.44:0/3748891391 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f8d240105d0 con 0x7f8d34100120 2023-09-10T01:35:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.832+0000 7f8d2affd700 1 --2- 172.21.15.44:0/3748891391 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8d200a07b0 0x7f8d200a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.832+0000 7f8d2affd700 1 -- 172.21.15.44:0/3748891391 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8d240cc1f0 con 0x7f8d34100120 2023-09-10T01:35:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.832+0000 7f8d398c5700 1 --2- 172.21.15.44:0/3748891391 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8d200a07b0 0x7f8d200a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.833+0000 7f8d398c5700 1 --2- 172.21.15.44:0/3748891391 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8d200a07b0 0x7f8d200a2c70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000bb90 tx=0x7f8d30005d80 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.833+0000 7f8d47cefb80 1 Processor -- start 2023-09-10T01:35:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.833+0000 7f8d47cefb80 1 -- start start 2023-09-10T01:35:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.833+0000 7f8d47cefb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d91a0 0x5591d86d9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.833+0000 7f8d47cefb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3300 0x5591d86d3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.833+0000 7f8d47cefb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d3cc0 0x5591d86dff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.833+0000 7f8d47cefb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591d862c6d0 con 0x5591d86d91a0 2023-09-10T01:35:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.833+0000 7f8d47cefb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591d8629ff0 con 0x5591d86d3cc0 2023-09-10T01:35:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.833+0000 7f8d47cefb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591d86258e0 con 0x5591d86d3300 2023-09-10T01:35:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.833+0000 7f8d398c5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d91a0 0x5591d86d9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.833+0000 7f8d398c5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d91a0 0x5591d86d9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33034/0 (socket says 172.21.15.44:33034) 2023-09-10T01:35:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.833+0000 7f8d398c5700 1 -- 172.21.15.44:0/1345932041 learned_addr learned my addr 172.21.15.44:0/1345932041 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.834+0000 7f8d390c4700 1 --2- 172.21.15.44:0/1345932041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3300 0x5591d86d3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.834+0000 7f8d3a0c6700 1 --2- 172.21.15.44:0/1345932041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d3cc0 0x5591d86dff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.834+0000 7f8d398c5700 1 -- 172.21.15.44:0/1345932041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3300 msgr2=0x5591d86d3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.834+0000 7f8d398c5700 1 --2- 172.21.15.44:0/1345932041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3300 0x5591d86d3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.834+0000 7f8d398c5700 1 -- 172.21.15.44:0/1345932041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d3cc0 msgr2=0x5591d86dff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.834+0000 7f8d398c5700 1 --2- 172.21.15.44:0/1345932041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d3cc0 0x5591d86dff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.834+0000 7f8d398c5700 1 -- 172.21.15.44:0/1345932041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591d8451d80 con 0x5591d86d91a0 2023-09-10T01:35:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.834+0000 7f8d398c5700 1 --2- 172.21.15.44:0/1345932041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d91a0 0x5591d86d9580 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000b970 tx=0x7f8d30012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67f80ebdfd78524 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.834+0000 7f8d1ffff700 1 -- 172.21.15.44:0/1345932041 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30011ad0 con 0x5591d86d91a0 2023-09-10T01:35:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.834+0000 7f8d1ffff700 1 -- 172.21.15.44:0/1345932041 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d30011c30 con 0x5591d86d91a0 2023-09-10T01:35:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.834+0000 7f8d1ffff700 1 -- 172.21.15.44:0/1345932041 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3001d610 con 0x5591d86d91a0 2023-09-10T01:35:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.835+0000 7f8d47cefb80 1 -- 172.21.15.44:0/1345932041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d91a0 msgr2=0x5591d86d9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.835+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/1345932041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d91a0 0x5591d86d9580 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000b970 tx=0x7f8d30012000 comp rx=0 tx=0).stop 2023-09-10T01:35:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.835+0000 7f8d47cefb80 1 -- 172.21.15.44:0/1345932041 shutdown_connections 2023-09-10T01:35:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.835+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/1345932041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d3cc0 0x5591d86dff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.835+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/1345932041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d91a0 0x5591d86d9580 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.835+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/1345932041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3300 0x5591d86d3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.835+0000 7f8d47cefb80 1 -- 172.21.15.44:0/1345932041 >> 172.21.15.44:0/1345932041 conn(0x5591d8648030 msgr2=0x5591d86ce560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.835+0000 7f8d47cefb80 1 -- 172.21.15.44:0/1345932041 shutdown_connections 2023-09-10T01:35:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.835+0000 7f8d47cefb80 1 -- 172.21.15.44:0/1345932041 wait complete. 2023-09-10T01:35:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.835+0000 7f8d47cefb80 1 -- start start 2023-09-10T01:35:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.835+0000 7f8d47cefb80 1 Processor -- start 2023-09-10T01:35:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.835+0000 7f8d47cefb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d3300 0x5591d86de900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d47cefb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3cc0 0x5591d86df1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d47cefb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d91a0 0x5591d86df720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d47cefb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591d85c3b10 con 0x5591d86d3300 2023-09-10T01:35:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d47cefb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591d85c3810 con 0x5591d86d91a0 2023-09-10T01:35:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d47cefb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591d84a78c0 con 0x5591d86d3cc0 2023-09-10T01:35:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d47cefb80 10 client.0 _ll_register_callbacks cb 0x5591d850da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:35:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d398c5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d3300 0x5591d86de900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d398c5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d3300 0x5591d86de900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33040/0 (socket says 172.21.15.44:33040) 2023-09-10T01:35:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d398c5700 1 -- 172.21.15.44:0/3681639655 learned_addr learned my addr 172.21.15.44:0/3681639655 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d390c4700 1 --2- 172.21.15.44:0/3681639655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3cc0 0x5591d86df1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d3a0c6700 1 --2- 172.21.15.44:0/3681639655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d91a0 0x5591d86df720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d398c5700 1 -- 172.21.15.44:0/3681639655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3cc0 msgr2=0x5591d86df1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d398c5700 1 --2- 172.21.15.44:0/3681639655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3cc0 0x5591d86df1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d398c5700 1 -- 172.21.15.44:0/3681639655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d91a0 msgr2=0x5591d86df720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d398c5700 1 --2- 172.21.15.44:0/3681639655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d91a0 0x5591d86df720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d398c5700 1 -- 172.21.15.44:0/3681639655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3000f9b0 con 0x5591d86d3300 2023-09-10T01:35:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d390c4700 1 --2- 172.21.15.44:0/3681639655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3cc0 0x5591d86df1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:35:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d398c5700 1 --2- 172.21.15.44:0/3681639655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d3300 0x5591d86de900 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000ba20 tx=0x7f8d3001ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb88326ae72e217b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:35:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d1d7fa700 1 -- 172.21.15.44:0/3681639655 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3002d070 con 0x5591d86d3300 2023-09-10T01:35:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d1d7fa700 1 -- 172.21.15.44:0/3681639655 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(209..209 src has 1..209) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8d3001e660 con 0x5591d86d3300 2023-09-10T01:35:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.836+0000 7f8d47cefb80 1 -- 172.21.15.44:0/3681639655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591d857d490 con 0x5591d86d3300 2023-09-10T01:35:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.837+0000 7f8d1d7fa700 1 -- 172.21.15.44:0/3681639655 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=196) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d30011a90 con 0x5591d86d3300 2023-09-10T01:35:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.837+0000 7f8d47cefb80 10 client.20709 fetch_fsmap learned FSMap version 196 2023-09-10T01:35:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.837+0000 7f8d47cefb80 1 -- 172.21.15.44:0/3681639655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=196,monmap=4+}) v3 -- 0x5591d843ac00 con 0x5591d86d3300 2023-09-10T01:35:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.837+0000 7f8d1d7fa700 1 -- 172.21.15.44:0/3681639655 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5591d857d490 con 0x5591d86d3300 2023-09-10T01:35:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.837+0000 7f8d1d7fa700 1 -- 172.21.15.44:0/3681639655 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 196) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f8d30027450 con 0x5591d86d3300 2023-09-10T01:35:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.837+0000 7f8d47cefb80 10 client.20709 fetch_fsmap finished waiting for FSMap version 196 2023-09-10T01:35:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.837+0000 7f8d47cefb80 10 client.20709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:35:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.837+0000 7f8d47cefb80 20 client.20709 populate_metadata read hostname 'smithi044' 2023-09-10T01:35:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.837+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/3681639655 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5591d86f16b0 0x5591d86f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.837+0000 7f8d47cefb80 4 client.20709 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:35:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.837+0000 7f8d47cefb80 1 -- 172.21.15.44:0/3681639655 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591d863a840 con 0x5591d86f16b0 2023-09-10T01:35:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.837+0000 7f8d390c4700 1 --2- 172.21.15.44:0/3681639655 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5591d86f16b0 0x5591d86f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.837+0000 7f8d1d7fa700 10 client.20709 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:35:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.837+0000 7f8d390c4700 1 --2- 172.21.15.44:0/3681639655 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5591d86f16b0 0x5591d86f3ad0 crc :-1 s=READY pgs=616 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49db41460bc90297 server_cookie=bf6253e3235c7bc4 in_seq=0 out_seq=0 2023-09-10T01:35:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.838+0000 7f8d1d7fa700 1 -- 172.21.15.44:0/3681639655 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5591d863a840 con 0x5591d86f16b0 2023-09-10T01:35:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.838+0000 7f8d1d7fa700 10 client.20709 handle_command_reply: tid=0 2023-09-10T01:35:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.838+0000 7f8d47cefb80 1 client.20709 shutdown 2023-09-10T01:35:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.838+0000 7f8d47cefb80 1 -- 172.21.15.44:0/3681639655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d3300 msgr2=0x5591d86de900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.838+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/3681639655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d3300 0x5591d86de900 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000ba20 tx=0x7f8d3001ead0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.838+0000 7f8d47cefb80 1 -- 172.21.15.44:0/3681639655 shutdown_connections 2023-09-10T01:35:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.838+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/3681639655 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5591d86f16b0 0x5591d86f3ad0 crc :-1 s=READY pgs=616 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.838+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/3681639655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d91a0 0x5591d86df720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.838+0000 7f8d1d7fa700 0 client.20709 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:35:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.838+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/3681639655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d3300 0x5591d86de900 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.838+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/3681639655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3cc0 0x5591d86df1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.838+0000 7f8d47cefb80 1 -- 172.21.15.44:0/3681639655 >> 172.21.15.44:0/3681639655 conn(0x5591d8648030 msgr2=0x5591d86cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.838+0000 7f8d47cefb80 1 -- 172.21.15.44:0/3681639655 shutdown_connections 2023-09-10T01:35:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.838+0000 7f8d47cefb80 1 -- 172.21.15.44:0/3681639655 wait complete. 2023-09-10T01:35:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.838+0000 7f8d47cefb80 20 client.20709 trim_cache size 0 max 16384 2023-09-10T01:35:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.853+0000 7f8d47cefb80 1 Processor -- start 2023-09-10T01:35:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.853+0000 7f8d47cefb80 1 -- start start 2023-09-10T01:35:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.853+0000 7f8d47cefb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3300 0x5591d86df6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.853+0000 7f8d47cefb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d3cc0 0x5591d86dc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.853+0000 7f8d47cefb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d91a0 0x5591d86d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.853+0000 7f8d47cefb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591d86ee120 con 0x5591d86d3cc0 2023-09-10T01:35:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.853+0000 7f8d47cefb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591d86dfd00 con 0x5591d86d91a0 2023-09-10T01:35:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.853+0000 7f8d47cefb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591d86dd200 con 0x5591d86d3300 2023-09-10T01:35:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.853+0000 7f8d390c4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d3cc0 0x5591d86dc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.853+0000 7f8d390c4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d3cc0 0x5591d86dc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33050/0 (socket says 172.21.15.44:33050) 2023-09-10T01:35:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.853+0000 7f8d390c4700 1 -- 172.21.15.44:0/3125598052 learned_addr learned my addr 172.21.15.44:0/3125598052 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.854+0000 7f8d3a0c6700 1 --2- 172.21.15.44:0/3125598052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d91a0 0x5591d86d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.854+0000 7f8d390c4700 1 -- 172.21.15.44:0/3125598052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3300 msgr2=0x5591d86df6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.854+0000 7f8d390c4700 1 --2- 172.21.15.44:0/3125598052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3300 0x5591d86df6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.854+0000 7f8d390c4700 1 -- 172.21.15.44:0/3125598052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d91a0 msgr2=0x5591d86d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.854+0000 7f8d390c4700 1 --2- 172.21.15.44:0/3125598052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d91a0 0x5591d86d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.854+0000 7f8d390c4700 1 -- 172.21.15.44:0/3125598052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591d84781f0 con 0x5591d86d3cc0 2023-09-10T01:35:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.854+0000 7f8d390c4700 1 --2- 172.21.15.44:0/3125598052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d3cc0 0x5591d86dc7b0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f8d24010880 tx=0x7f8d240cb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20be3b0acf34156b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.854+0000 7f8d28d89700 1 -- 172.21.15.44:0/3125598052 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d240b5070 con 0x5591d86d3cc0 2023-09-10T01:35:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.854+0000 7f8d28d89700 1 -- 172.21.15.44:0/3125598052 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d240b72c0 con 0x5591d86d3cc0 2023-09-10T01:35:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.854+0000 7f8d28d89700 1 -- 172.21.15.44:0/3125598052 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24097470 con 0x5591d86d3cc0 2023-09-10T01:35:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 -- 172.21.15.44:0/3125598052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d3cc0 msgr2=0x5591d86dc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/3125598052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d3cc0 0x5591d86dc7b0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f8d24010880 tx=0x7f8d240cb040 comp rx=0 tx=0).stop 2023-09-10T01:35:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 -- 172.21.15.44:0/3125598052 shutdown_connections 2023-09-10T01:35:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/3125598052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d91a0 0x5591d86d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/3125598052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3300 0x5591d86df6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/3125598052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d3cc0 0x5591d86dc7b0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 -- 172.21.15.44:0/3125598052 >> 172.21.15.44:0/3125598052 conn(0x5591d8648030 msgr2=0x5591d86d22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 -- 172.21.15.44:0/3125598052 shutdown_connections 2023-09-10T01:35:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 -- 172.21.15.44:0/3125598052 wait complete. 2023-09-10T01:35:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 -- start start 2023-09-10T01:35:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 Processor -- start 2023-09-10T01:35:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3300 0x5591d86e86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d3cc0 0x5591d86e6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d91a0 0x5591d86e6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591d86e8fb0 con 0x5591d86d91a0 2023-09-10T01:35:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591d86e74b0 con 0x5591d86d3cc0 2023-09-10T01:35:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591d86e7670 con 0x5591d86d3300 2023-09-10T01:35:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d47cefb80 10 client.0 _ll_register_callbacks cb 0x7f8d2400bf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:35:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d3a0c6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d91a0 0x5591d86e6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d3a0c6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d91a0 0x5591d86e6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33058/0 (socket says 172.21.15.44:33058) 2023-09-10T01:35:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.855+0000 7f8d3a0c6700 1 -- 172.21.15.44:0/1192183515 learned_addr learned my addr 172.21.15.44:0/1192183515 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:35:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d398c5700 1 --2- 172.21.15.44:0/1192183515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3300 0x5591d86e86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d390c4700 1 --2- 172.21.15.44:0/1192183515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d3cc0 0x5591d86e6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d3a0c6700 1 -- 172.21.15.44:0/1192183515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3300 msgr2=0x5591d86e86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d3a0c6700 1 --2- 172.21.15.44:0/1192183515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3300 0x5591d86e86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d3a0c6700 1 -- 172.21.15.44:0/1192183515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d3cc0 msgr2=0x5591d86e6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:35:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d3a0c6700 1 --2- 172.21.15.44:0/1192183515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d3cc0 0x5591d86e6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d3a0c6700 1 -- 172.21.15.44:0/1192183515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d341a2650 con 0x5591d86d91a0 2023-09-10T01:35:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d390c4700 1 --2- 172.21.15.44:0/1192183515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d3cc0 0x5591d86e6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:35:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d3a0c6700 1 --2- 172.21.15.44:0/1192183515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d91a0 0x5591d86e6f40 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00d010 tx=0x7f8d2c007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e899a4805b9acdb9 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:35:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:35:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d1d7fa700 1 -- 172.21.15.44:0/1192183515 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c021070 con 0x5591d86d91a0 2023-09-10T01:35:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d1d7fa700 1 -- 172.21.15.44:0/1192183515 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(209..209 src has 1..209) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f8d2c003c10 con 0x5591d86d91a0 2023-09-10T01:35:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d47cefb80 1 -- 172.21.15.44:0/1192183515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d30011c30 con 0x5591d86d91a0 2023-09-10T01:35:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d1d7fa700 1 -- 172.21.15.44:0/1192183515 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=196) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d2c00e040 con 0x5591d86d91a0 2023-09-10T01:35:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d47cefb80 10 client.20715 fetch_fsmap learned FSMap version 196 2023-09-10T01:35:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d47cefb80 1 -- 172.21.15.44:0/1192183515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=196,monmap=4+}) v3 -- 0x5591d8450250 con 0x5591d86d91a0 2023-09-10T01:35:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.856+0000 7f8d1d7fa700 1 -- 172.21.15.44:0/1192183515 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d30011c30 con 0x5591d86d91a0 2023-09-10T01:35:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.857+0000 7f8d1d7fa700 1 -- 172.21.15.44:0/1192183515 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 196) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f8d2c00b100 con 0x5591d86d91a0 2023-09-10T01:35:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.857+0000 7f8d47cefb80 10 client.20715 fetch_fsmap finished waiting for FSMap version 196 2023-09-10T01:35:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.857+0000 7f8d47cefb80 10 client.20715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:35:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.857+0000 7f8d47cefb80 20 client.20715 populate_metadata read hostname 'smithi044' 2023-09-10T01:35:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.857+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/1192183515 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5591d87469e0 0x5591d8748e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:35:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.857+0000 7f8d47cefb80 4 client.20715 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:35:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.857+0000 7f8d47cefb80 1 -- 172.21.15.44:0/1192183515 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5591d861e8c0 con 0x5591d87469e0 2023-09-10T01:35:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.857+0000 7f8d398c5700 1 --2- 172.21.15.44:0/1192183515 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5591d87469e0 0x5591d8748e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:35:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.857+0000 7f8d1d7fa700 10 client.20715 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:35:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:49.857+0000 7f8d398c5700 1 --2- 172.21.15.44:0/1192183515 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5591d87469e0 0x5591d8748e00 crc :-1 s=READY pgs=617 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=736193dcc8e0937b server_cookie=e5d18927a7c499ca in_seq=0 out_seq=0 2023-09-10T01:35:51.182 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 885.0 s, 145.6 tps, lat 350.244 ms stddev 133.327 2023-09-10T01:35:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.209+0000 7f8d1d7fa700 1 -- 172.21.15.44:0/1192183515 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5591d861e8c0 con 0x5591d87469e0 2023-09-10T01:35:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.209+0000 7f8d1d7fa700 10 client.20715 handle_command_reply: tid=0 2023-09-10T01:35:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.209+0000 7f8d47cefb80 1 client.20715 shutdown 2023-09-10T01:35:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.209+0000 7f8d47cefb80 1 -- 172.21.15.44:0/1192183515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d91a0 msgr2=0x5591d86e6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.209+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/1192183515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d91a0 0x5591d86e6f40 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00d010 tx=0x7f8d2c007830 comp rx=0 tx=0).stop 2023-09-10T01:35:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.209+0000 7f8d47cefb80 1 -- 172.21.15.44:0/1192183515 shutdown_connections 2023-09-10T01:35:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.209+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/1192183515 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x5591d87469e0 0x5591d8748e00 crc :-1 s=READY pgs=617 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.209+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/1192183515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591d86d91a0 0x5591d86e6f40 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.210+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/1192183515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591d86d3300 0x5591d86e86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.210+0000 7f8d47cefb80 1 --2- 172.21.15.44:0/1192183515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591d86d3cc0 0x5591d86e6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.210+0000 7f8d47cefb80 1 -- 172.21.15.44:0/1192183515 >> 172.21.15.44:0/1192183515 conn(0x5591d8648030 msgr2=0x5591d86d2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.210+0000 7f8d1d7fa700 0 client.20715 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:35:51.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.218+0000 7f8d47cefb80 1 -- 172.21.15.44:0/1192183515 shutdown_connections 2023-09-10T01:35:51.220 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.218+0000 7f8d47cefb80 1 -- 172.21.15.44:0/1192183515 wait complete. 2023-09-10T01:35:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.221+0000 7f8d47cefb80 20 client.20715 trim_cache size 0 max 16384 2023-09-10T01:35:51.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:35:51.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:35:51.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:35:51.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:35:51.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.223+0000 7f8d3bb29700 1 -- 172.21.15.44:0/3748891391 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8d200a07b0 msgr2=0x7f8d200a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:51.224 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.223+0000 7f8d3bb29700 1 --2- 172.21.15.44:0/3748891391 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8d200a07b0 0x7f8d200a2c70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000bb90 tx=0x7f8d30005d80 comp rx=0 tx=0).stop 2023-09-10T01:35:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.223+0000 7f8d3bb29700 1 -- 172.21.15.44:0/3748891391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d34100120 msgr2=0x7f8d3419bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:35:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.223+0000 7f8d3bb29700 1 --2- 172.21.15.44:0/3748891391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d34100120 0x7f8d3419bb60 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400d950 tx=0x7f8d2400dc60 comp rx=0 tx=0).stop 2023-09-10T01:35:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.225+0000 7f8d3bb29700 1 -- 172.21.15.44:0/3748891391 shutdown_connections 2023-09-10T01:35:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.225+0000 7f8d3bb29700 1 --2- 172.21.15.44:0/3748891391 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f8d200a07b0 0x7f8d200a2c70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.225+0000 7f8d3bb29700 1 --2- 172.21.15.44:0/3748891391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d341055b0 0x7f8d341954a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.225+0000 7f8d3bb29700 1 --2- 172.21.15.44:0/3748891391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d340ff760 0x7f8d3419b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.225+0000 7f8d3bb29700 1 --2- 172.21.15.44:0/3748891391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d34100120 0x7f8d3419bb60 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:35:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.225+0000 7f8d3bb29700 1 -- 172.21.15.44:0/3748891391 >> 172.21.15.44:0/3748891391 conn(0x7f8d34074f30 msgr2=0x7f8d340fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:35:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.231+0000 7f8d3bb29700 1 -- 172.21.15.44:0/3748891391 shutdown_connections 2023-09-10T01:35:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:35:51.234+0000 7f8d3bb29700 1 -- 172.21.15.44:0/3748891391 wait complete. 2023-09-10T01:35:51.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:51 smithi044 ceph-mon[98676]: pgmap v1798: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 687 op/s 2023-09-10T01:35:51.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:51 smithi099 ceph-mon[102692]: pgmap v1798: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 687 op/s 2023-09-10T01:35:51.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:51 smithi131 ceph-mon[103713]: pgmap v1798: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 687 op/s 2023-09-10T01:35:53.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:53 smithi099 ceph-mon[102692]: pgmap v1799: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.5 MiB/s wr, 821 op/s 2023-09-10T01:35:53.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:53 smithi131 ceph-mon[103713]: pgmap v1799: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.5 MiB/s wr, 821 op/s 2023-09-10T01:35:53.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:53 smithi044 ceph-mon[98676]: pgmap v1799: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.5 MiB/s wr, 821 op/s 2023-09-10T01:35:55.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:55 smithi099 ceph-mon[102692]: pgmap v1800: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.0 MiB/s wr, 664 op/s 2023-09-10T01:35:55.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:55 smithi131 ceph-mon[103713]: pgmap v1800: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.0 MiB/s wr, 664 op/s 2023-09-10T01:35:55.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:55 smithi044 ceph-mon[98676]: pgmap v1800: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.0 MiB/s wr, 664 op/s 2023-09-10T01:35:56.185 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 890.0 s, 18.8 tps, lat 2845.328 ms stddev 198.250 2023-09-10T01:35:57.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:57 smithi099 ceph-mon[102692]: pgmap v1801: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 551 op/s 2023-09-10T01:35:57.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:57 smithi131 ceph-mon[103713]: pgmap v1801: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 551 op/s 2023-09-10T01:35:57.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:57 smithi044 ceph-mon[98676]: pgmap v1801: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 551 op/s 2023-09-10T01:35:59.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:35:59 smithi099 ceph-mon[102692]: pgmap v1802: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 621 op/s 2023-09-10T01:35:59.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:35:59 smithi131 ceph-mon[103713]: pgmap v1802: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 621 op/s 2023-09-10T01:35:59.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:35:59 smithi044 ceph-mon[98676]: pgmap v1802: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 621 op/s 2023-09-10T01:36:01.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 895.0 s, 180.1 tps, lat 1069.383 ms stddev 1761.020 2023-09-10T01:36:01.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:01 smithi099 ceph-mon[102692]: pgmap v1803: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 394 op/s 2023-09-10T01:36:01.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:01 smithi131 ceph-mon[103713]: pgmap v1803: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 394 op/s 2023-09-10T01:36:01.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:01 smithi044 ceph-mon[98676]: pgmap v1803: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 394 op/s 2023-09-10T01:36:03.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:03 smithi131 ceph-mon[103713]: pgmap v1804: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 443 op/s 2023-09-10T01:36:03.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:03 smithi044 ceph-mon[98676]: pgmap v1804: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 443 op/s 2023-09-10T01:36:03.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:03 smithi099 ceph-mon[102692]: pgmap v1804: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 443 op/s 2023-09-10T01:36:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.370+0000 7feb951fe700 1 -- 172.21.15.44:0/3596667184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb90100120 msgr2=0x7feb9010c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.370+0000 7feb951fe700 1 --2- 172.21.15.44:0/3596667184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb90100120 0x7feb9010c330 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7feb8800b0d0 tx=0x7feb8800b3e0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.370+0000 7feb951fe700 1 -- 172.21.15.44:0/3596667184 shutdown_connections 2023-09-10T01:36:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.370+0000 7feb951fe700 1 --2- 172.21.15.44:0/3596667184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb90100120 0x7feb9010c330 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.370+0000 7feb951fe700 1 --2- 172.21.15.44:0/3596667184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb901055b0 0x7feb90105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.370+0000 7feb951fe700 1 --2- 172.21.15.44:0/3596667184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb900ff760 0x7feb900ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.370+0000 7feb951fe700 1 -- 172.21.15.44:0/3596667184 >> 172.21.15.44:0/3596667184 conn(0x7feb90074f30 msgr2=0x7feb90075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.370+0000 7feb951fe700 1 -- 172.21.15.44:0/3596667184 shutdown_connections 2023-09-10T01:36:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.370+0000 7feb951fe700 1 -- 172.21.15.44:0/3596667184 wait complete. 2023-09-10T01:36:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb951fe700 1 Processor -- start 2023-09-10T01:36:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb951fe700 1 -- start start 2023-09-10T01:36:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb951fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb900ff760 0x7feb9019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb951fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb90100120 0x7feb9019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb951fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb901055b0 0x7feb901986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb951fe700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb90198d60 con 0x7feb900ff760 2023-09-10T01:36:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb951fe700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb90198ed0 con 0x7feb90100120 2023-09-10T01:36:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb951fe700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb90199090 con 0x7feb901055b0 2023-09-10T01:36:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb8ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb900ff760 0x7feb9019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb8ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb900ff760 0x7feb9019e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33068/0 (socket says 172.21.15.44:33068) 2023-09-10T01:36:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb8ed9d700 1 -- 172.21.15.44:0/2277498806 learned_addr learned my addr 172.21.15.44:0/2277498806 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb8e59c700 1 --2- 172.21.15.44:0/2277498806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb90100120 0x7feb9019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb8f59e700 1 --2- 172.21.15.44:0/2277498806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb901055b0 0x7feb901986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb8ed9d700 1 -- 172.21.15.44:0/2277498806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb901055b0 msgr2=0x7feb901986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb8ed9d700 1 --2- 172.21.15.44:0/2277498806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb901055b0 0x7feb901986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb8ed9d700 1 -- 172.21.15.44:0/2277498806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb90100120 msgr2=0x7feb9019edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb8ed9d700 1 --2- 172.21.15.44:0/2277498806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb90100120 0x7feb9019edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.371+0000 7feb8ed9d700 1 -- 172.21.15.44:0/2277498806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb88009d00 con 0x7feb900ff760 2023-09-10T01:36:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.372+0000 7feb8ed9d700 1 --2- 172.21.15.44:0/2277498806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb900ff760 0x7feb9019e890 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7feb8000e9a0 tx=0x7feb8000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.372+0000 7feb87fff700 1 -- 172.21.15.44:0/2277498806 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb80009790 con 0x7feb900ff760 2023-09-10T01:36:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.372+0000 7feb87fff700 1 -- 172.21.15.44:0/2277498806 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb80010430 con 0x7feb900ff760 2023-09-10T01:36:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.372+0000 7feb951fe700 1 -- 172.21.15.44:0/2277498806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb90199370 con 0x7feb900ff760 2023-09-10T01:36:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.372+0000 7feb87fff700 1 -- 172.21.15.44:0/2277498806 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8000cea0 con 0x7feb900ff760 2023-09-10T01:36:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.372+0000 7feb951fe700 1 -- 172.21.15.44:0/2277498806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb901a5880 con 0x7feb900ff760 2023-09-10T01:36:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.374+0000 7feb87fff700 1 -- 172.21.15.44:0/2277498806 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7feb800105a0 con 0x7feb900ff760 2023-09-10T01:36:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.375+0000 7feb87fff700 1 --2- 172.21.15.44:0/2277498806 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb7c0a0830 0x7feb7c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.375+0000 7feb87fff700 1 -- 172.21.15.44:0/2277498806 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 9540+0+0 (secure 0 0 0) 0x7feb80014070 con 0x7feb900ff760 2023-09-10T01:36:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.375+0000 7feb8e59c700 1 --2- 172.21.15.44:0/2277498806 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb7c0a0830 0x7feb7c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.375+0000 7feb8e59c700 1 --2- 172.21.15.44:0/2277498806 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb7c0a0830 0x7feb7c0a2cf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7feb900753a0 tx=0x7feb78009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.376+0000 7feba13c4b80 1 Processor -- start 2023-09-10T01:36:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.376+0000 7feba13c4b80 1 -- start start 2023-09-10T01:36:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.376+0000 7feba13c4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f09d1a0 0x55634f09d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.376+0000 7feba13c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f097300 0x55634f097780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.376+0000 7feba13c4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097cc0 0x55634f0a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.376+0000 7feba13c4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634eff06d0 con 0x55634f097cc0 2023-09-10T01:36:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.376+0000 7feba13c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634efedff0 con 0x55634f097300 2023-09-10T01:36:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.376+0000 7feba13c4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634efe98e0 con 0x55634f09d1a0 2023-09-10T01:36:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.376+0000 7feb8f59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097cc0 0x55634f0a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.376+0000 7feb8f59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097cc0 0x55634f0a3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33080/0 (socket says 172.21.15.44:33080) 2023-09-10T01:36:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.377+0000 7feb8f59e700 1 -- 172.21.15.44:0/2123045241 learned_addr learned my addr 172.21.15.44:0/2123045241 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.377+0000 7feb8e59c700 1 --2- 172.21.15.44:0/2123045241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f097300 0x55634f097780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.377+0000 7feb8ed9d700 1 --2- 172.21.15.44:0/2123045241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f09d1a0 0x55634f09d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.377+0000 7feb8e59c700 1 -- 172.21.15.44:0/2123045241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f09d1a0 msgr2=0x55634f09d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.377+0000 7feb8e59c700 1 --2- 172.21.15.44:0/2123045241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f09d1a0 0x55634f09d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.377+0000 7feb8e59c700 1 -- 172.21.15.44:0/2123045241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097cc0 msgr2=0x55634f0a3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.377+0000 7feb8e59c700 1 --2- 172.21.15.44:0/2123045241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097cc0 0x55634f0a3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.377+0000 7feb8e59c700 1 -- 172.21.15.44:0/2123045241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55634ee15d80 con 0x55634f097300 2023-09-10T01:36:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.377+0000 7feb8f59e700 1 --2- 172.21.15.44:0/2123045241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097cc0 0x55634f0a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:36:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.377+0000 7feb8e59c700 1 --2- 172.21.15.44:0/2123045241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f097300 0x55634f097780 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7feb78011c90 tx=0x7feb78014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f39f0f1c50b9bfc5 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.377+0000 7feb857fa700 1 -- 172.21.15.44:0/2123045241 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb78004230 con 0x55634f097300 2023-09-10T01:36:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.377+0000 7feb857fa700 1 -- 172.21.15.44:0/2123045241 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb78021070 con 0x55634f097300 2023-09-10T01:36:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.377+0000 7feb857fa700 1 -- 172.21.15.44:0/2123045241 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb78003b00 con 0x55634f097300 2023-09-10T01:36:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.378+0000 7feba13c4b80 1 -- 172.21.15.44:0/2123045241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f097300 msgr2=0x55634f097780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.378+0000 7feba13c4b80 1 --2- 172.21.15.44:0/2123045241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f097300 0x55634f097780 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7feb78011c90 tx=0x7feb78014000 comp rx=0 tx=0).stop 2023-09-10T01:36:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.378+0000 7feba13c4b80 1 -- 172.21.15.44:0/2123045241 shutdown_connections 2023-09-10T01:36:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.378+0000 7feba13c4b80 1 --2- 172.21.15.44:0/2123045241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097cc0 0x55634f0a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.378+0000 7feba13c4b80 1 --2- 172.21.15.44:0/2123045241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f09d1a0 0x55634f09d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.378+0000 7feba13c4b80 1 --2- 172.21.15.44:0/2123045241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f097300 0x55634f097780 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.378+0000 7feba13c4b80 1 -- 172.21.15.44:0/2123045241 >> 172.21.15.44:0/2123045241 conn(0x55634f00c030 msgr2=0x55634f092560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.378+0000 7feba13c4b80 1 -- 172.21.15.44:0/2123045241 shutdown_connections 2023-09-10T01:36:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.378+0000 7feba13c4b80 1 -- 172.21.15.44:0/2123045241 wait complete. 2023-09-10T01:36:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.378+0000 7feba13c4b80 1 -- start start 2023-09-10T01:36:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.378+0000 7feba13c4b80 1 Processor -- start 2023-09-10T01:36:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feba13c4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097300 0x55634f0a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feba13c4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f097cc0 0x55634f0a1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feba13c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f09d1a0 0x55634f0a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feba13c4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634ef87b10 con 0x55634f097300 2023-09-10T01:36:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feba13c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634ef87810 con 0x55634f09d1a0 2023-09-10T01:36:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feba13c4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634ee6b8c0 con 0x55634f097cc0 2023-09-10T01:36:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feba13c4b80 10 client.0 _ll_register_callbacks cb 0x55634eed1a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:36:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feb8ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097300 0x55634f0a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feb8ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097300 0x55634f0a3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33082/0 (socket says 172.21.15.44:33082) 2023-09-10T01:36:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feb8ed9d700 1 -- 172.21.15.44:0/1727400949 learned_addr learned my addr 172.21.15.44:0/1727400949 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feb8f59e700 1 --2- 172.21.15.44:0/1727400949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f09d1a0 0x55634f0a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feb8e59c700 1 --2- 172.21.15.44:0/1727400949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f097cc0 0x55634f0a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feb8ed9d700 1 -- 172.21.15.44:0/1727400949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f097cc0 msgr2=0x55634f0a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feb8ed9d700 1 --2- 172.21.15.44:0/1727400949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f097cc0 0x55634f0a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feb8ed9d700 1 -- 172.21.15.44:0/1727400949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f09d1a0 msgr2=0x55634f0a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feb8ed9d700 1 --2- 172.21.15.44:0/1727400949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f09d1a0 0x55634f0a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feb8ed9d700 1 -- 172.21.15.44:0/1727400949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb9010cbc0 con 0x55634f097300 2023-09-10T01:36:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feb8ed9d700 1 --2- 172.21.15.44:0/1727400949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097300 0x55634f0a3e10 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7feb80095fe0 tx=0x7feb80098fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54e1bf560e57ca1 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.379+0000 7feb767fc700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.380+0000 7feb767fc700 1 -- 172.21.15.44:0/1727400949 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb80099570 con 0x55634f097300 2023-09-10T01:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.380+0000 7feb767fc700 1 -- 172.21.15.44:0/1727400949 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(209..209 src has 1..209) v4 ==== 9540+0+0 (secure 0 0 0) 0x7feb8009d3a0 con 0x55634f097300 2023-09-10T01:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.380+0000 7feba13c4b80 1 -- 172.21.15.44:0/1727400949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55634ef41490 con 0x55634f097300 2023-09-10T01:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.380+0000 7feb767fc700 1 -- 172.21.15.44:0/1727400949 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=196) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb800a15e0 con 0x55634f097300 2023-09-10T01:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.380+0000 7feba13c4b80 10 client.20727 fetch_fsmap learned FSMap version 196 2023-09-10T01:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.380+0000 7feba13c4b80 1 -- 172.21.15.44:0/1727400949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=196,monmap=4+}) v3 -- 0x7feb78017070 con 0x55634f097300 2023-09-10T01:36:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.380+0000 7feb767fc700 1 -- 172.21.15.44:0/1727400949 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55634ef41490 con 0x55634f097300 2023-09-10T01:36:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.380+0000 7feb767fc700 1 -- 172.21.15.44:0/1727400949 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 196) v1 ==== 7371+0+0 (secure 0 0 0) 0x7feb800a1150 con 0x55634f097300 2023-09-10T01:36:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.380+0000 7feba13c4b80 10 client.20727 fetch_fsmap finished waiting for FSMap version 196 2023-09-10T01:36:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.380+0000 7feba13c4b80 10 client.20727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:36:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.380+0000 7feba13c4b80 20 client.20727 populate_metadata read hostname 'smithi044' 2023-09-10T01:36:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.380+0000 7feba13c4b80 1 --2- 172.21.15.44:0/1727400949 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55634f0b1450 0x55634f0b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.380+0000 7feba13c4b80 4 client.20727 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:36:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.380+0000 7feba13c4b80 1 -- 172.21.15.44:0/1727400949 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55634effe840 con 0x55634f0b1450 2023-09-10T01:36:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.380+0000 7feb8e59c700 1 --2- 172.21.15.44:0/1727400949 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55634f0b1450 0x55634f0b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.381+0000 7feb767fc700 10 client.20727 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.381+0000 7feb8e59c700 1 --2- 172.21.15.44:0/1727400949 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55634f0b1450 0x55634f0b3870 crc :-1 s=READY pgs=618 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29618d8de1a342cb server_cookie=f39971e9efa977a6 in_seq=0 out_seq=0 2023-09-10T01:36:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.381+0000 7feb767fc700 1 -- 172.21.15.44:0/1727400949 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55634effe840 con 0x55634f0b1450 2023-09-10T01:36:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.381+0000 7feb767fc700 10 client.20727 handle_command_reply: tid=0 2023-09-10T01:36:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.381+0000 7feba13c4b80 1 client.20727 shutdown 2023-09-10T01:36:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.381+0000 7feba13c4b80 1 -- 172.21.15.44:0/1727400949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097300 msgr2=0x55634f0a3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.381+0000 7feba13c4b80 1 --2- 172.21.15.44:0/1727400949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097300 0x55634f0a3e10 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7feb80095fe0 tx=0x7feb80098fc0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.382+0000 7feba13c4b80 1 -- 172.21.15.44:0/1727400949 shutdown_connections 2023-09-10T01:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.382+0000 7feba13c4b80 1 --2- 172.21.15.44:0/1727400949 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55634f0b1450 0x55634f0b3870 crc :-1 s=READY pgs=618 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.382+0000 7feba13c4b80 1 --2- 172.21.15.44:0/1727400949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f09d1a0 0x55634f0a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.382+0000 7feba13c4b80 1 --2- 172.21.15.44:0/1727400949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097300 0x55634f0a3e10 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.382+0000 7feb767fc700 0 client.20727 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.382+0000 7feba13c4b80 1 --2- 172.21.15.44:0/1727400949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f097cc0 0x55634f0a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.382+0000 7feba13c4b80 1 -- 172.21.15.44:0/1727400949 >> 172.21.15.44:0/1727400949 conn(0x55634f00c030 msgr2=0x55634f0917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.382+0000 7feba13c4b80 1 -- 172.21.15.44:0/1727400949 shutdown_connections 2023-09-10T01:36:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.382+0000 7feba13c4b80 1 -- 172.21.15.44:0/1727400949 wait complete. 2023-09-10T01:36:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.382+0000 7feba13c4b80 20 client.20727 trim_cache size 0 max 16384 2023-09-10T01:36:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.397+0000 7feba13c4b80 1 Processor -- start 2023-09-10T01:36:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.397+0000 7feba13c4b80 1 -- start start 2023-09-10T01:36:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.397+0000 7feba13c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f097300 0x55634f0a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.397+0000 7feba13c4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f097cc0 0x55634f09c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.397+0000 7feba13c4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f09d1a0 0x55634f09c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.397+0000 7feba13c4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634ef41490 con 0x55634f09d1a0 2023-09-10T01:36:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.397+0000 7feba13c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634f09cc20 con 0x55634f097300 2023-09-10T01:36:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.397+0000 7feba13c4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634f08c8a0 con 0x55634f097cc0 2023-09-10T01:36:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.397+0000 7feb8f59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f09d1a0 0x55634f09c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.397+0000 7feb8f59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f09d1a0 0x55634f09c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33094/0 (socket says 172.21.15.44:33094) 2023-09-10T01:36:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.397+0000 7feb8f59e700 1 -- 172.21.15.44:0/3524986627 learned_addr learned my addr 172.21.15.44:0/3524986627 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.397+0000 7feb8ed9d700 1 --2- 172.21.15.44:0/3524986627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f097300 0x55634f0a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.397+0000 7feb8e59c700 1 --2- 172.21.15.44:0/3524986627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f097cc0 0x55634f09c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.398+0000 7feb8f59e700 1 -- 172.21.15.44:0/3524986627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f097cc0 msgr2=0x55634f09c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.398+0000 7feb8f59e700 1 --2- 172.21.15.44:0/3524986627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f097cc0 0x55634f09c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.398+0000 7feb8f59e700 1 -- 172.21.15.44:0/3524986627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f097300 msgr2=0x55634f0a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.398+0000 7feb8f59e700 1 --2- 172.21.15.44:0/3524986627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f097300 0x55634f0a39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.398+0000 7feb8f59e700 1 -- 172.21.15.44:0/3524986627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55634edfec00 con 0x55634f09d1a0 2023-09-10T01:36:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.398+0000 7feb8e59c700 1 --2- 172.21.15.44:0/3524986627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f097cc0 0x55634f09c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:36:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.398+0000 7feb8f59e700 1 --2- 172.21.15.44:0/3524986627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f09d1a0 0x55634f09c610 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7feb88007c40 tx=0x7feb8800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f404fb9796330cbb server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.398+0000 7feb85ffb700 1 -- 172.21.15.44:0/3524986627 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8800e040 con 0x55634f09d1a0 2023-09-10T01:36:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.398+0000 7feb85ffb700 1 -- 172.21.15.44:0/3524986627 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb8800bca0 con 0x55634f09d1a0 2023-09-10T01:36:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.398+0000 7feb85ffb700 1 -- 172.21.15.44:0/3524986627 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88005310 con 0x55634f09d1a0 2023-09-10T01:36:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.399+0000 7feba13c4b80 1 -- 172.21.15.44:0/3524986627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f09d1a0 msgr2=0x55634f09c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.399+0000 7feba13c4b80 1 --2- 172.21.15.44:0/3524986627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f09d1a0 0x55634f09c610 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7feb88007c40 tx=0x7feb8800b040 comp rx=0 tx=0).stop 2023-09-10T01:36:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.399+0000 7feba13c4b80 1 -- 172.21.15.44:0/3524986627 shutdown_connections 2023-09-10T01:36:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.399+0000 7feba13c4b80 1 --2- 172.21.15.44:0/3524986627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f09d1a0 0x55634f09c610 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.399+0000 7feba13c4b80 1 --2- 172.21.15.44:0/3524986627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f097300 0x55634f0a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.399+0000 7feba13c4b80 1 --2- 172.21.15.44:0/3524986627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f097cc0 0x55634f09c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.399+0000 7feba13c4b80 1 -- 172.21.15.44:0/3524986627 >> 172.21.15.44:0/3524986627 conn(0x55634f00c030 msgr2=0x55634f09b150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.399+0000 7feba13c4b80 1 -- 172.21.15.44:0/3524986627 shutdown_connections 2023-09-10T01:36:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.399+0000 7feba13c4b80 1 -- 172.21.15.44:0/3524986627 wait complete. 2023-09-10T01:36:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.399+0000 7feba13c4b80 1 -- start start 2023-09-10T01:36:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.399+0000 7feba13c4b80 1 Processor -- start 2023-09-10T01:36:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feba13c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f097300 0x55634f0a7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feba13c4b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097cc0 0x55634f0a8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feba13c4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f09d1a0 0x55634f104240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feba13c4b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634f104780 con 0x55634f097cc0 2023-09-10T01:36:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feba13c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634f1048c0 con 0x55634f097300 2023-09-10T01:36:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feba13c4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634f104a00 con 0x55634f09d1a0 2023-09-10T01:36:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feba13c4b80 10 client.0 _ll_register_callbacks cb 0x7feb800990b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:36:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feb8e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097cc0 0x55634f0a8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feb8e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097cc0 0x55634f0a8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33098/0 (socket says 172.21.15.44:33098) 2023-09-10T01:36:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feb8e59c700 1 -- 172.21.15.44:0/3636631670 learned_addr learned my addr 172.21.15.44:0/3636631670 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feb8ed9d700 1 --2- 172.21.15.44:0/3636631670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f097300 0x55634f0a7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feb8f59e700 1 --2- 172.21.15.44:0/3636631670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f09d1a0 0x55634f104240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feb8e59c700 1 -- 172.21.15.44:0/3636631670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f09d1a0 msgr2=0x55634f104240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feb8e59c700 1 --2- 172.21.15.44:0/3636631670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f09d1a0 0x55634f104240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feb8e59c700 1 -- 172.21.15.44:0/3636631670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f097300 msgr2=0x55634f0a7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feb8e59c700 1 --2- 172.21.15.44:0/3636631670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f097300 0x55634f0a7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feb8e59c700 1 -- 172.21.15.44:0/3636631670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb88009420 con 0x55634f097cc0 2023-09-10T01:36:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feb8e59c700 1 --2- 172.21.15.44:0/3636631670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097cc0 0x55634f0a8010 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7feb78011c90 tx=0x7feb78005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52bebf5b87492ac server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feb767fc700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:36:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feb767fc700 1 -- 172.21.15.44:0/3636631670 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7802b040 con 0x55634f097cc0 2023-09-10T01:36:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feba13c4b80 1 -- 172.21.15.44:0/3636631670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb78021070 con 0x55634f097cc0 2023-09-10T01:36:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.400+0000 7feb767fc700 1 -- 172.21.15.44:0/3636631670 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(209..209 src has 1..209) v4 ==== 9540+0+0 (secure 0 0 0) 0x7feb7800ea00 con 0x55634f097cc0 2023-09-10T01:36:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.401+0000 7feb767fc700 1 -- 172.21.15.44:0/3636631670 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=196) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb7800ed60 con 0x55634f097cc0 2023-09-10T01:36:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.401+0000 7feba13c4b80 10 client.20733 fetch_fsmap learned FSMap version 196 2023-09-10T01:36:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.401+0000 7feba13c4b80 1 -- 172.21.15.44:0/3636631670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=196,monmap=4+}) v3 -- 0x55634ee3c1f0 con 0x55634f097cc0 2023-09-10T01:36:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.401+0000 7feb767fc700 1 -- 172.21.15.44:0/3636631670 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb78021070 con 0x55634f097cc0 2023-09-10T01:36:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.401+0000 7feb767fc700 1 -- 172.21.15.44:0/3636631670 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 196) v1 ==== 7371+0+0 (secure 0 0 0) 0x7feb780046c0 con 0x55634f097cc0 2023-09-10T01:36:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.401+0000 7feba13c4b80 10 client.20733 fetch_fsmap finished waiting for FSMap version 196 2023-09-10T01:36:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.401+0000 7feba13c4b80 10 client.20733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:36:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.401+0000 7feba13c4b80 20 client.20733 populate_metadata read hostname 'smithi044' 2023-09-10T01:36:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.401+0000 7feba13c4b80 1 --2- 172.21.15.44:0/3636631670 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55634f1061c0 0x55634f1085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.401+0000 7feba13c4b80 4 client.20733 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:36:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.401+0000 7feba13c4b80 1 -- 172.21.15.44:0/3636631670 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55634efe28c0 con 0x55634f1061c0 2023-09-10T01:36:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.401+0000 7feb8ed9d700 1 --2- 172.21.15.44:0/3636631670 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55634f1061c0 0x55634f1085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.402+0000 7feb8ed9d700 1 --2- 172.21.15.44:0/3636631670 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55634f1061c0 0x55634f1085e0 crc :-1 s=READY pgs=619 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96040d396ff2a48b server_cookie=cfe3a1c613987af7 in_seq=0 out_seq=0 2023-09-10T01:36:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.402+0000 7feb767fc700 10 client.20733 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.422+0000 7feb767fc700 1 -- 172.21.15.44:0/3636631670 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55634efe28c0 con 0x55634f1061c0 2023-09-10T01:36:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.422+0000 7feb767fc700 10 client.20733 handle_command_reply: tid=0 2023-09-10T01:36:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.422+0000 7feba13c4b80 1 client.20733 shutdown 2023-09-10T01:36:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feba13c4b80 1 -- 172.21.15.44:0/3636631670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097cc0 msgr2=0x55634f0a8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feba13c4b80 1 --2- 172.21.15.44:0/3636631670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097cc0 0x55634f0a8010 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7feb78011c90 tx=0x7feb78005420 comp rx=0 tx=0).stop 2023-09-10T01:36:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feba13c4b80 1 -- 172.21.15.44:0/3636631670 shutdown_connections 2023-09-10T01:36:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feba13c4b80 1 --2- 172.21.15.44:0/3636631670 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55634f1061c0 0x55634f1085e0 crc :-1 s=READY pgs=619 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feba13c4b80 1 --2- 172.21.15.44:0/3636631670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55634f09d1a0 0x55634f104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feb767fc700 0 client.20733 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feba13c4b80 1 --2- 172.21.15.44:0/3636631670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55634f097300 0x55634f0a7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feba13c4b80 1 --2- 172.21.15.44:0/3636631670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55634f097cc0 0x55634f0a8010 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feba13c4b80 1 -- 172.21.15.44:0/3636631670 >> 172.21.15.44:0/3636631670 conn(0x55634f00c030 msgr2=0x55634f09b150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feba13c4b80 1 -- 172.21.15.44:0/3636631670 shutdown_connections 2023-09-10T01:36:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feba13c4b80 1 -- 172.21.15.44:0/3636631670 wait complete. 2023-09-10T01:36:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feba13c4b80 20 client.20733 trim_cache size 0 max 16384 2023-09-10T01:36:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:36:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:36:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:36:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:36:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feb951fe700 1 -- 172.21.15.44:0/2277498806 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb7c0a0830 msgr2=0x7feb7c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feb951fe700 1 --2- 172.21.15.44:0/2277498806 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb7c0a0830 0x7feb7c0a2cf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7feb900753a0 tx=0x7feb78009260 comp rx=0 tx=0).stop 2023-09-10T01:36:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feb951fe700 1 -- 172.21.15.44:0/2277498806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb900ff760 msgr2=0x7feb9019e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feb951fe700 1 --2- 172.21.15.44:0/2277498806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb900ff760 0x7feb9019e890 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7feb8000e9a0 tx=0x7feb8000ecb0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feb951fe700 1 -- 172.21.15.44:0/2277498806 shutdown_connections 2023-09-10T01:36:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feb951fe700 1 --2- 172.21.15.44:0/2277498806 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7feb7c0a0830 0x7feb7c0a2cf0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feb951fe700 1 --2- 172.21.15.44:0/2277498806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb901055b0 0x7feb901986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feb951fe700 1 --2- 172.21.15.44:0/2277498806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb900ff760 0x7feb9019e890 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feb951fe700 1 --2- 172.21.15.44:0/2277498806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb90100120 0x7feb9019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feb951fe700 1 -- 172.21.15.44:0/2277498806 >> 172.21.15.44:0/2277498806 conn(0x7feb90074f30 msgr2=0x7feb900679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feb951fe700 1 -- 172.21.15.44:0/2277498806 shutdown_connections 2023-09-10T01:36:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:04.423+0000 7feb951fe700 1 -- 172.21.15.44:0/2277498806 wait complete. 2023-09-10T01:36:04.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:04 smithi131 ceph-mon[103713]: pgmap v1805: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 347 op/s 2023-09-10T01:36:04.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:04 smithi044 ceph-mon[98676]: pgmap v1805: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 347 op/s 2023-09-10T01:36:04.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:04 smithi099 ceph-mon[102692]: pgmap v1805: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 347 op/s 2023-09-10T01:36:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:progress: 900.0 s, 49.8 tps, lat 1591.316 ms stddev 1524.553 2023-09-10T01:36:07.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:06 smithi131 ceph-mon[103713]: pgmap v1806: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1021 KiB/s rd, 2.4 MiB/s wr, 302 op/s 2023-09-10T01:36:07.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:06 smithi044 ceph-mon[98676]: pgmap v1806: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1021 KiB/s rd, 2.4 MiB/s wr, 302 op/s 2023-09-10T01:36:07.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:06 smithi099 ceph-mon[102692]: pgmap v1806: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 1021 KiB/s rd, 2.4 MiB/s wr, 302 op/s 2023-09-10T01:36:09.047 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:08 smithi131 ceph-mon[103713]: pgmap v1807: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 397 op/s 2023-09-10T01:36:09.061 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:08 smithi044 ceph-mon[98676]: pgmap v1807: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 397 op/s 2023-09-10T01:36:09.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:08 smithi099 ceph-mon[102692]: pgmap v1807: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 397 op/s 2023-09-10T01:36:09.546 INFO:teuthology.orchestra.run.smithi044.stdout:transaction type: 2023-09-10T01:36:09.546 INFO:teuthology.orchestra.run.smithi044.stdout:scaling factor: 500 2023-09-10T01:36:09.547 INFO:teuthology.orchestra.run.smithi044.stdout:query mode: simple 2023-09-10T01:36:09.547 INFO:teuthology.orchestra.run.smithi044.stdout:number of clients: 100 2023-09-10T01:36:09.547 INFO:teuthology.orchestra.run.smithi044.stdout:number of threads: 4 2023-09-10T01:36:09.547 INFO:teuthology.orchestra.run.smithi044.stdout:duration: 900 s 2023-09-10T01:36:09.547 INFO:teuthology.orchestra.run.smithi044.stdout:number of transactions actually processed: 207478 2023-09-10T01:36:09.547 INFO:teuthology.orchestra.run.smithi044.stdout:latency average = 434.766 ms 2023-09-10T01:36:09.548 INFO:teuthology.orchestra.run.smithi044.stdout:latency stddev = 664.540 ms 2023-09-10T01:36:09.548 INFO:teuthology.orchestra.run.smithi044.stdout:tps = 229.675055 (including connections establishing) 2023-09-10T01:36:09.548 INFO:teuthology.orchestra.run.smithi044.stdout:tps = 229.679346 (excluding connections establishing) 2023-09-10T01:36:09.557 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2023-09-10T01:36:09.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:09 smithi044 ceph-mon[98676]: osdmap e210: 12 total, 12 up, 12 in 2023-09-10T01:36:10.050 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:09 smithi131 ceph-mon[103713]: osdmap e210: 12 total, 12 up, 12 in 2023-09-10T01:36:10.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:09 smithi099 ceph-mon[102692]: osdmap e210: 12 total, 12 up, 12 in 2023-09-10T01:36:11.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:10 smithi099 ceph-mon[102692]: pgmap v1809: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 343 op/s 2023-09-10T01:36:11.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:10 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:10 smithi131 ceph-mon[103713]: pgmap v1809: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 343 op/s 2023-09-10T01:36:11.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:10 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:11.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:10 smithi044 ceph-mon[98676]: pgmap v1809: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 343 op/s 2023-09-10T01:36:11.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:10 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:13.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:12 smithi131 ceph-mon[103713]: pgmap v1810: 129 pgs: 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 93 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 279 op/s 2023-09-10T01:36:13.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:12 smithi044 ceph-mon[98676]: pgmap v1810: 129 pgs: 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 93 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 279 op/s 2023-09-10T01:36:13.362 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:12 smithi099 ceph-mon[102692]: pgmap v1810: 129 pgs: 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 93 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 279 op/s 2023-09-10T01:36:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:14 smithi131 ceph-mon[103713]: pgmap v1811: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 1.9 MiB/s wr, 220 op/s 2023-09-10T01:36:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:14 smithi044 ceph-mon[98676]: pgmap v1811: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 1.9 MiB/s wr, 220 op/s 2023-09-10T01:36:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:14 smithi099 ceph-mon[102692]: pgmap v1811: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 1.9 MiB/s wr, 220 op/s 2023-09-10T01:36:16.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:15 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:16.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:15 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:16.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:15 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:17.298 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:16 smithi131 ceph-mon[103713]: pgmap v1812: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.8 MiB/s wr, 188 op/s 2023-09-10T01:36:17.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:16 smithi044 ceph-mon[98676]: pgmap v1812: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.8 MiB/s wr, 188 op/s 2023-09-10T01:36:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:16 smithi099 ceph-mon[102692]: pgmap v1812: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.8 MiB/s wr, 188 op/s 2023-09-10T01:36:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.556+0000 7f30eaf9c700 1 -- 172.21.15.44:0/498239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30e41055b0 msgr2=0x7f30e4105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.556+0000 7f30eaf9c700 1 --2- 172.21.15.44:0/498239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30e41055b0 0x7f30e4105990 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f30d80099c0 tx=0x7f30d8009cd0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.556+0000 7f30eaf9c700 1 -- 172.21.15.44:0/498239 shutdown_connections 2023-09-10T01:36:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.556+0000 7f30eaf9c700 1 --2- 172.21.15.44:0/498239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30e4100120 0x7f30e410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.556+0000 7f30eaf9c700 1 --2- 172.21.15.44:0/498239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30e41055b0 0x7f30e4105990 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.556+0000 7f30eaf9c700 1 --2- 172.21.15.44:0/498239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30e40ff760 0x7f30e40ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.556+0000 7f30eaf9c700 1 -- 172.21.15.44:0/498239 >> 172.21.15.44:0/498239 conn(0x7f30e4074f30 msgr2=0x7f30e4075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.556+0000 7f30eaf9c700 1 -- 172.21.15.44:0/498239 shutdown_connections 2023-09-10T01:36:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.556+0000 7f30eaf9c700 1 -- 172.21.15.44:0/498239 wait complete. 2023-09-10T01:36:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.557+0000 7f30eaf9c700 1 Processor -- start 2023-09-10T01:36:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.557+0000 7f30eaf9c700 1 -- start start 2023-09-10T01:36:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.557+0000 7f30eaf9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30e40ff760 0x7f30e41975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.557+0000 7f30eaf9c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30e4100120 0x7f30e4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.557+0000 7f30eaf9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30e41055b0 0x7f30e41a2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.557+0000 7f30eaf9c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e41a2980 con 0x7f30e4100120 2023-09-10T01:36:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.557+0000 7f30eaf9c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e41a2ac0 con 0x7f30e41055b0 2023-09-10T01:36:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.557+0000 7f30eaf9c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e41a2c80 con 0x7f30e40ff760 2023-09-10T01:36:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.557+0000 7f30e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30e4100120 0x7f30e4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.557+0000 7f30e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30e4100120 0x7f30e4197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33110/0 (socket says 172.21.15.44:33110) 2023-09-10T01:36:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.557+0000 7f30e3fff700 1 -- 172.21.15.44:0/514702941 learned_addr learned my addr 172.21.15.44:0/514702941 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.557+0000 7f30e8d38700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30e40ff760 0x7f30e41975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.557+0000 7f30e9539700 1 --2- 172.21.15.44:0/514702941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30e41055b0 0x7f30e41a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.558+0000 7f30e3fff700 1 -- 172.21.15.44:0/514702941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30e40ff760 msgr2=0x7f30e41975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.558+0000 7f30e3fff700 1 --2- 172.21.15.44:0/514702941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30e40ff760 0x7f30e41975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.558+0000 7f30e3fff700 1 -- 172.21.15.44:0/514702941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30e41055b0 msgr2=0x7f30e41a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.558+0000 7f30e3fff700 1 --2- 172.21.15.44:0/514702941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30e41055b0 0x7f30e41a2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.558+0000 7f30e3fff700 1 -- 172.21.15.44:0/514702941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d80095f0 con 0x7f30e4100120 2023-09-10T01:36:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.558+0000 7f30e3fff700 1 --2- 172.21.15.44:0/514702941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30e4100120 0x7f30e4197ae0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f30d000e9a0 tx=0x7f30d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.558+0000 7f30e1ffb700 1 -- 172.21.15.44:0/514702941 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0009790 con 0x7f30e4100120 2023-09-10T01:36:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.558+0000 7f30eaf9c700 1 -- 172.21.15.44:0/514702941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e41a2f60 con 0x7f30e4100120 2023-09-10T01:36:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.558+0000 7f30e1ffb700 1 -- 172.21.15.44:0/514702941 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30d0010430 con 0x7f30e4100120 2023-09-10T01:36:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.558+0000 7f30e1ffb700 1 -- 172.21.15.44:0/514702941 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d000cea0 con 0x7f30e4100120 2023-09-10T01:36:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.558+0000 7f30eaf9c700 1 -- 172.21.15.44:0/514702941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e41a33e0 con 0x7f30e4100120 2023-09-10T01:36:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.560+0000 7f30e1ffb700 1 -- 172.21.15.44:0/514702941 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f30d00105a0 con 0x7f30e4100120 2023-09-10T01:36:17.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.561+0000 7f30e1ffb700 1 --2- 172.21.15.44:0/514702941 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f30d40a07b0 0x7f30d40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.561+0000 7f30e8d38700 1 --2- 172.21.15.44:0/514702941 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f30d40a07b0 0x7f30d40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.561+0000 7f30e1ffb700 1 -- 172.21.15.44:0/514702941 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f30d0014070 con 0x7f30e4100120 2023-09-10T01:36:17.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.561+0000 7f30e8d38700 1 --2- 172.21.15.44:0/514702941 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f30d40a07b0 0x7f30d40a2c70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f30d800b580 tx=0x7f30d8005c00 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:17.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.562+0000 7f30f7162b80 1 Processor -- start 2023-09-10T01:36:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.562+0000 7f30f7162b80 1 -- start start 2023-09-10T01:36:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.562+0000 7f30f7162b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a7621a0 0x55a32a762580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.562+0000 7f30f7162b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75c300 0x55a32a75c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.562+0000 7f30f7162b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a75ccc0 0x55a32a768f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.562+0000 7f30f7162b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32a6b56d0 con 0x55a32a75ccc0 2023-09-10T01:36:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.562+0000 7f30f7162b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32a6b2ff0 con 0x55a32a75c300 2023-09-10T01:36:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.562+0000 7f30f7162b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32a6ae8e0 con 0x55a32a7621a0 2023-09-10T01:36:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.563+0000 7f30e9539700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a75ccc0 0x55a32a768f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.563+0000 7f30e8d38700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a7621a0 0x55a32a762580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.563+0000 7f30e8d38700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a7621a0 0x55a32a762580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:39998/0 (socket says 172.21.15.44:39998) 2023-09-10T01:36:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.563+0000 7f30e9539700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a75ccc0 0x55a32a768f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33120/0 (socket says 172.21.15.44:33120) 2023-09-10T01:36:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.563+0000 7f30e8d38700 1 -- 172.21.15.44:0/2651932197 learned_addr learned my addr 172.21.15.44:0/2651932197 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.563+0000 7f30e3fff700 1 --2- 172.21.15.44:0/2651932197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75c300 0x55a32a75c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.563+0000 7f30e9539700 1 -- 172.21.15.44:0/2651932197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a7621a0 msgr2=0x55a32a762580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.563+0000 7f30e9539700 1 --2- 172.21.15.44:0/2651932197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a7621a0 0x55a32a762580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.563+0000 7f30e9539700 1 -- 172.21.15.44:0/2651932197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75c300 msgr2=0x55a32a75c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.563+0000 7f30e9539700 1 --2- 172.21.15.44:0/2651932197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75c300 0x55a32a75c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.563+0000 7f30e9539700 1 -- 172.21.15.44:0/2651932197 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a32a4dad80 con 0x55a32a75ccc0 2023-09-10T01:36:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.563+0000 7f30e8d38700 1 --2- 172.21.15.44:0/2651932197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a7621a0 0x55a32a762580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:36:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.563+0000 7f30e9539700 1 --2- 172.21.15.44:0/2651932197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a75ccc0 0x55a32a768f30 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f30dc011580 tx=0x7f30dc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0f618ca55f7459 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.563+0000 7f30ceffd700 1 -- 172.21.15.44:0/2651932197 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00e040 con 0x55a32a75ccc0 2023-09-10T01:36:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.563+0000 7f30ceffd700 1 -- 172.21.15.44:0/2651932197 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30dc00f040 con 0x55a32a75ccc0 2023-09-10T01:36:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.563+0000 7f30ceffd700 1 -- 172.21.15.44:0/2651932197 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc004270 con 0x55a32a75ccc0 2023-09-10T01:36:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.564+0000 7f30f7162b80 1 -- 172.21.15.44:0/2651932197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a75ccc0 msgr2=0x55a32a768f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.564+0000 7f30f7162b80 1 --2- 172.21.15.44:0/2651932197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a75ccc0 0x55a32a768f30 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f30dc011580 tx=0x7f30dc009920 comp rx=0 tx=0).stop 2023-09-10T01:36:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.564+0000 7f30f7162b80 1 -- 172.21.15.44:0/2651932197 shutdown_connections 2023-09-10T01:36:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.564+0000 7f30f7162b80 1 --2- 172.21.15.44:0/2651932197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a75ccc0 0x55a32a768f30 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.564+0000 7f30f7162b80 1 --2- 172.21.15.44:0/2651932197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a7621a0 0x55a32a762580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.564+0000 7f30f7162b80 1 --2- 172.21.15.44:0/2651932197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75c300 0x55a32a75c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.564+0000 7f30f7162b80 1 -- 172.21.15.44:0/2651932197 >> 172.21.15.44:0/2651932197 conn(0x55a32a6d1030 msgr2=0x55a32a757560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.564+0000 7f30f7162b80 1 -- 172.21.15.44:0/2651932197 shutdown_connections 2023-09-10T01:36:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.564+0000 7f30f7162b80 1 -- 172.21.15.44:0/2651932197 wait complete. 2023-09-10T01:36:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.564+0000 7f30f7162b80 1 -- start start 2023-09-10T01:36:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.564+0000 7f30f7162b80 1 Processor -- start 2023-09-10T01:36:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30f7162b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a75c300 0x55a32a768da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30f7162b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75ccc0 0x55a32a766050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30f7162b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a7621a0 0x55a32a7665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30f7162b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32a64cb10 con 0x55a32a7621a0 2023-09-10T01:36:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30f7162b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32a64c810 con 0x55a32a75ccc0 2023-09-10T01:36:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30f7162b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32a5308c0 con 0x55a32a75c300 2023-09-10T01:36:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30f7162b80 10 client.0 _ll_register_callbacks cb 0x55a32a596a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:36:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30e9539700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a7621a0 0x55a32a7665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30e8d38700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a75c300 0x55a32a768da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30e9539700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a7621a0 0x55a32a7665b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33126/0 (socket says 172.21.15.44:33126) 2023-09-10T01:36:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30e9539700 1 -- 172.21.15.44:0/406767149 learned_addr learned my addr 172.21.15.44:0/406767149 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30e3fff700 1 --2- 172.21.15.44:0/406767149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75ccc0 0x55a32a766050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30e9539700 1 -- 172.21.15.44:0/406767149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a75c300 msgr2=0x55a32a768da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30e9539700 1 --2- 172.21.15.44:0/406767149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a75c300 0x55a32a768da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30e9539700 1 -- 172.21.15.44:0/406767149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75ccc0 msgr2=0x55a32a766050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30e9539700 1 --2- 172.21.15.44:0/406767149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75ccc0 0x55a32a766050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30e9539700 1 -- 172.21.15.44:0/406767149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e410cbc0 con 0x55a32a7621a0 2023-09-10T01:36:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30e9539700 1 --2- 172.21.15.44:0/406767149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a7621a0 0x55a32a7665b0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f30dc003a80 tx=0x7f30dc003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755bc3a761a7058b server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:36:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30c3fff700 1 -- 172.21.15.44:0/406767149 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00e040 con 0x55a32a7621a0 2023-09-10T01:36:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30c3fff700 1 -- 172.21.15.44:0/406767149 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(210..210 src has 1..210) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f30dc004240 con 0x55a32a7621a0 2023-09-10T01:36:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.565+0000 7f30f7162b80 1 -- 172.21.15.44:0/406767149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a32a606490 con 0x55a32a7621a0 2023-09-10T01:36:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.566+0000 7f30c3fff700 1 -- 172.21.15.44:0/406767149 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=198) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30dc004400 con 0x55a32a7621a0 2023-09-10T01:36:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.566+0000 7f30f7162b80 10 client.20745 fetch_fsmap learned FSMap version 198 2023-09-10T01:36:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.566+0000 7f30f7162b80 1 -- 172.21.15.44:0/406767149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=198,monmap=4+}) v3 -- 0x7f30dc011230 con 0x55a32a7621a0 2023-09-10T01:36:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.566+0000 7f30c3fff700 1 -- 172.21.15.44:0/406767149 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a32a606490 con 0x55a32a7621a0 2023-09-10T01:36:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.566+0000 7f30c3fff700 1 -- 172.21.15.44:0/406767149 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 198) v1 ==== 7379+0+0 (secure 0 0 0) 0x7f30dc01b020 con 0x55a32a7621a0 2023-09-10T01:36:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.566+0000 7f30f7162b80 10 client.20745 fetch_fsmap finished waiting for FSMap version 198 2023-09-10T01:36:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.566+0000 7f30f7162b80 10 client.20745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:36:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.566+0000 7f30f7162b80 20 client.20745 populate_metadata read hostname 'smithi044' 2023-09-10T01:36:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.566+0000 7f30f7162b80 1 --2- 172.21.15.44:0/406767149 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a32a7763c0 0x55a32a7787e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.566+0000 7f30f7162b80 4 client.20745 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:36:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.566+0000 7f30f7162b80 1 -- 172.21.15.44:0/406767149 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a32a6c3840 con 0x55a32a7763c0 2023-09-10T01:36:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.566+0000 7f30e8d38700 1 --2- 172.21.15.44:0/406767149 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a32a7763c0 0x55a32a7787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.567+0000 7f30c3fff700 10 client.20745 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.567+0000 7f30e8d38700 1 --2- 172.21.15.44:0/406767149 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a32a7763c0 0x55a32a7787e0 crc :-1 s=READY pgs=620 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0cf03022a643ad7 server_cookie=4dda92955a32362c in_seq=0 out_seq=0 2023-09-10T01:36:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.567+0000 7f30c3fff700 1 -- 172.21.15.44:0/406767149 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a32a6c3840 con 0x55a32a7763c0 2023-09-10T01:36:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.567+0000 7f30c3fff700 10 client.20745 handle_command_reply: tid=0 2023-09-10T01:36:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.567+0000 7f30f7162b80 1 client.20745 shutdown 2023-09-10T01:36:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.568+0000 7f30f7162b80 1 -- 172.21.15.44:0/406767149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a7621a0 msgr2=0x55a32a7665b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.568+0000 7f30f7162b80 1 --2- 172.21.15.44:0/406767149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a7621a0 0x55a32a7665b0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f30dc003a80 tx=0x7f30dc003ab0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.568+0000 7f30f7162b80 1 -- 172.21.15.44:0/406767149 shutdown_connections 2023-09-10T01:36:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.568+0000 7f30f7162b80 1 --2- 172.21.15.44:0/406767149 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a32a7763c0 0x55a32a7787e0 crc :-1 s=READY pgs=620 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.568+0000 7f30f7162b80 1 --2- 172.21.15.44:0/406767149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a7621a0 0x55a32a7665b0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.568+0000 7f30f7162b80 1 --2- 172.21.15.44:0/406767149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a75c300 0x55a32a768da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.568+0000 7f30f7162b80 1 --2- 172.21.15.44:0/406767149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75ccc0 0x55a32a766050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.568+0000 7f30f7162b80 1 -- 172.21.15.44:0/406767149 >> 172.21.15.44:0/406767149 conn(0x55a32a6d1030 msgr2=0x55a32a7567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.568+0000 7f30c3fff700 0 client.20745 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.568+0000 7f30f7162b80 1 -- 172.21.15.44:0/406767149 shutdown_connections 2023-09-10T01:36:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.568+0000 7f30f7162b80 1 -- 172.21.15.44:0/406767149 wait complete. 2023-09-10T01:36:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.568+0000 7f30f7162b80 20 client.20745 trim_cache size 0 max 16384 2023-09-10T01:36:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.582+0000 7f30f7162b80 1 Processor -- start 2023-09-10T01:36:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.582+0000 7f30f7162b80 1 -- start start 2023-09-10T01:36:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30f7162b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a75c300 0x55a32a761a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30f7162b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75ccc0 0x55a32a764fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30f7162b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a7621a0 0x55a32a768860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30f7162b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32a7739d0 con 0x55a32a7621a0 2023-09-10T01:36:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30f7162b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32a7581c0 con 0x55a32a75ccc0 2023-09-10T01:36:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30f7162b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32a758d10 con 0x55a32a75c300 2023-09-10T01:36:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30e9539700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a7621a0 0x55a32a768860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30e9539700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a7621a0 0x55a32a768860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33134/0 (socket says 172.21.15.44:33134) 2023-09-10T01:36:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30e9539700 1 -- 172.21.15.44:0/214803300 learned_addr learned my addr 172.21.15.44:0/214803300 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30e3fff700 1 --2- 172.21.15.44:0/214803300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75ccc0 0x55a32a764fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30e8d38700 1 --2- 172.21.15.44:0/214803300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a75c300 0x55a32a761a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30e9539700 1 -- 172.21.15.44:0/214803300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a75c300 msgr2=0x55a32a761a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30e9539700 1 --2- 172.21.15.44:0/214803300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a75c300 0x55a32a761a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30e9539700 1 -- 172.21.15.44:0/214803300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75ccc0 msgr2=0x55a32a764fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30e9539700 1 --2- 172.21.15.44:0/214803300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75ccc0 0x55a32a764fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30e9539700 1 -- 172.21.15.44:0/214803300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a32a765c10 con 0x55a32a7621a0 2023-09-10T01:36:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30e8d38700 1 --2- 172.21.15.44:0/214803300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a75c300 0x55a32a761a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:36:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30e9539700 1 --2- 172.21.15.44:0/214803300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a7621a0 0x55a32a768860 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f30dc011580 tx=0x7f30dc006020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfdfc818fa1a04c server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30cf7fe700 1 -- 172.21.15.44:0/214803300 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc007700 con 0x55a32a7621a0 2023-09-10T01:36:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30cf7fe700 1 -- 172.21.15.44:0/214803300 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30dc009ac0 con 0x55a32a7621a0 2023-09-10T01:36:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.583+0000 7f30cf7fe700 1 -- 172.21.15.44:0/214803300 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc027de0 con 0x55a32a7621a0 2023-09-10T01:36:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.584+0000 7f30f7162b80 1 -- 172.21.15.44:0/214803300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a7621a0 msgr2=0x55a32a768860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.584+0000 7f30f7162b80 1 --2- 172.21.15.44:0/214803300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a7621a0 0x55a32a768860 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f30dc011580 tx=0x7f30dc006020 comp rx=0 tx=0).stop 2023-09-10T01:36:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.584+0000 7f30f7162b80 1 -- 172.21.15.44:0/214803300 shutdown_connections 2023-09-10T01:36:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.584+0000 7f30f7162b80 1 --2- 172.21.15.44:0/214803300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a7621a0 0x55a32a768860 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.584+0000 7f30f7162b80 1 --2- 172.21.15.44:0/214803300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a75c300 0x55a32a761a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.584+0000 7f30f7162b80 1 --2- 172.21.15.44:0/214803300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75ccc0 0x55a32a764fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.584+0000 7f30f7162b80 1 -- 172.21.15.44:0/214803300 >> 172.21.15.44:0/214803300 conn(0x55a32a6d1030 msgr2=0x55a32a764890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:17.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.584+0000 7f30f7162b80 1 -- 172.21.15.44:0/214803300 shutdown_connections 2023-09-10T01:36:17.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.584+0000 7f30f7162b80 1 -- 172.21.15.44:0/214803300 wait complete. 2023-09-10T01:36:17.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.584+0000 7f30f7162b80 1 -- start start 2023-09-10T01:36:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.584+0000 7f30f7162b80 1 Processor -- start 2023-09-10T01:36:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30f7162b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75c300 0x55a32a76b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30f7162b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a75ccc0 0x55a32a76ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30f7162b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a7621a0 0x55a32a76bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30f7162b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32a7c9300 con 0x55a32a75ccc0 2023-09-10T01:36:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30f7162b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32a7c9470 con 0x55a32a75c300 2023-09-10T01:36:17.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30f7162b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32a7c9630 con 0x55a32a7621a0 2023-09-10T01:36:17.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30f7162b80 10 client.0 _ll_register_callbacks cb 0x7f30d0095240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:36:17.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a75ccc0 0x55a32a76ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a75ccc0 0x55a32a76ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33138/0 (socket says 172.21.15.44:33138) 2023-09-10T01:36:17.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30e3fff700 1 -- 172.21.15.44:0/1942631934 learned_addr learned my addr 172.21.15.44:0/1942631934 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30e8d38700 1 --2- 172.21.15.44:0/1942631934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75c300 0x55a32a76b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30e9539700 1 --2- 172.21.15.44:0/1942631934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a7621a0 0x55a32a76bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30e3fff700 1 -- 172.21.15.44:0/1942631934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a7621a0 msgr2=0x55a32a76bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30e3fff700 1 --2- 172.21.15.44:0/1942631934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a7621a0 0x55a32a76bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30e3fff700 1 -- 172.21.15.44:0/1942631934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75c300 msgr2=0x55a32a76b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30e3fff700 1 --2- 172.21.15.44:0/1942631934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75c300 0x55a32a76b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30e3fff700 1 -- 172.21.15.44:0/1942631934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30dc011230 con 0x55a32a75ccc0 2023-09-10T01:36:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30e9539700 1 --2- 172.21.15.44:0/1942631934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a7621a0 0x55a32a76bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:36:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30e3fff700 1 --2- 172.21.15.44:0/1942631934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a75ccc0 0x55a32a76ba50 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f30d00992c0 tx=0x7f30d0095900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9c5803523caf77 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.585+0000 7f30c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:36:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.586+0000 7f30c3fff700 1 -- 172.21.15.44:0/1942631934 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d00995c0 con 0x55a32a75ccc0 2023-09-10T01:36:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.586+0000 7f30c3fff700 1 -- 172.21.15.44:0/1942631934 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(210..210 src has 1..210) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f30d009d3f0 con 0x55a32a75ccc0 2023-09-10T01:36:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.586+0000 7f30f7162b80 1 -- 172.21.15.44:0/1942631934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a32a6a78c0 con 0x55a32a75ccc0 2023-09-10T01:36:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.586+0000 7f30c3fff700 1 -- 172.21.15.44:0/1942631934 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=198) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30d00a1630 con 0x55a32a75ccc0 2023-09-10T01:36:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.586+0000 7f30f7162b80 10 client.20751 fetch_fsmap learned FSMap version 198 2023-09-10T01:36:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.586+0000 7f30f7162b80 1 -- 172.21.15.44:0/1942631934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=198,monmap=4+}) v3 -- 0x55a32a4c3c00 con 0x55a32a75ccc0 2023-09-10T01:36:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.586+0000 7f30c3fff700 1 -- 172.21.15.44:0/1942631934 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30d0063080 con 0x55a32a75ccc0 2023-09-10T01:36:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.586+0000 7f30c3fff700 1 -- 172.21.15.44:0/1942631934 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 198) v1 ==== 7379+0+0 (secure 0 0 0) 0x7f30d00a1230 con 0x55a32a75ccc0 2023-09-10T01:36:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.586+0000 7f30f7162b80 10 client.20751 fetch_fsmap finished waiting for FSMap version 198 2023-09-10T01:36:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.586+0000 7f30f7162b80 10 client.20751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:36:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.586+0000 7f30f7162b80 20 client.20751 populate_metadata read hostname 'smithi044' 2023-09-10T01:36:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.586+0000 7f30f7162b80 1 --2- 172.21.15.44:0/1942631934 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a32a7cae40 0x55a32a7cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.586+0000 7f30f7162b80 4 client.20751 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:36:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.586+0000 7f30f7162b80 1 -- 172.21.15.44:0/1942631934 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a32a668050 con 0x55a32a7cae40 2023-09-10T01:36:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.586+0000 7f30e8d38700 1 --2- 172.21.15.44:0/1942631934 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a32a7cae40 0x55a32a7cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.587+0000 7f30c3fff700 10 client.20751 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.587+0000 7f30e8d38700 1 --2- 172.21.15.44:0/1942631934 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a32a7cae40 0x55a32a7cd260 crc :-1 s=READY pgs=621 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a757634e179690 server_cookie=ade431f7898ad79a in_seq=0 out_seq=0 2023-09-10T01:36:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.609+0000 7f30c3fff700 1 -- 172.21.15.44:0/1942631934 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a32a668050 con 0x55a32a7cae40 2023-09-10T01:36:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.609+0000 7f30c3fff700 10 client.20751 handle_command_reply: tid=0 2023-09-10T01:36:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.609+0000 7f30f7162b80 1 client.20751 shutdown 2023-09-10T01:36:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.609+0000 7f30f7162b80 1 -- 172.21.15.44:0/1942631934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a75ccc0 msgr2=0x55a32a76ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.609+0000 7f30f7162b80 1 --2- 172.21.15.44:0/1942631934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a75ccc0 0x55a32a76ba50 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f30d00992c0 tx=0x7f30d0095900 comp rx=0 tx=0).stop 2023-09-10T01:36:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.609+0000 7f30f7162b80 1 -- 172.21.15.44:0/1942631934 shutdown_connections 2023-09-10T01:36:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.609+0000 7f30f7162b80 1 --2- 172.21.15.44:0/1942631934 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55a32a7cae40 0x55a32a7cd260 crc :-1 s=READY pgs=621 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.609+0000 7f30f7162b80 1 --2- 172.21.15.44:0/1942631934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a32a7621a0 0x55a32a76bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.609+0000 7f30c3fff700 0 client.20751 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.609+0000 7f30f7162b80 1 --2- 172.21.15.44:0/1942631934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a32a75c300 0x55a32a76b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.609+0000 7f30f7162b80 1 --2- 172.21.15.44:0/1942631934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a32a75ccc0 0x55a32a76ba50 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.609+0000 7f30f7162b80 1 -- 172.21.15.44:0/1942631934 >> 172.21.15.44:0/1942631934 conn(0x55a32a6d1030 msgr2=0x55a32a761870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.609+0000 7f30f7162b80 1 -- 172.21.15.44:0/1942631934 shutdown_connections 2023-09-10T01:36:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.609+0000 7f30f7162b80 1 -- 172.21.15.44:0/1942631934 wait complete. 2023-09-10T01:36:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.609+0000 7f30f7162b80 20 client.20751 trim_cache size 0 max 16384 2023-09-10T01:36:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:36:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:36:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:36:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:36:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.610+0000 7f30eaf9c700 1 -- 172.21.15.44:0/514702941 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f30d40a07b0 msgr2=0x7f30d40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.610+0000 7f30eaf9c700 1 --2- 172.21.15.44:0/514702941 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f30d40a07b0 0x7f30d40a2c70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f30d800b580 tx=0x7f30d8005c00 comp rx=0 tx=0).stop 2023-09-10T01:36:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.610+0000 7f30eaf9c700 1 -- 172.21.15.44:0/514702941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30e4100120 msgr2=0x7f30e4197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.610+0000 7f30eaf9c700 1 --2- 172.21.15.44:0/514702941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30e4100120 0x7f30e4197ae0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f30d000e9a0 tx=0x7f30d000ecb0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.610+0000 7f30eaf9c700 1 -- 172.21.15.44:0/514702941 shutdown_connections 2023-09-10T01:36:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.610+0000 7f30eaf9c700 1 --2- 172.21.15.44:0/514702941 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f30d40a07b0 0x7f30d40a2c70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.610+0000 7f30eaf9c700 1 --2- 172.21.15.44:0/514702941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30e41055b0 0x7f30e41a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.610+0000 7f30eaf9c700 1 --2- 172.21.15.44:0/514702941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30e40ff760 0x7f30e41975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.610+0000 7f30eaf9c700 1 --2- 172.21.15.44:0/514702941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30e4100120 0x7f30e4197ae0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.610+0000 7f30eaf9c700 1 -- 172.21.15.44:0/514702941 >> 172.21.15.44:0/514702941 conn(0x7f30e4074f30 msgr2=0x7f30e40fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.610+0000 7f30eaf9c700 1 -- 172.21.15.44:0/514702941 shutdown_connections 2023-09-10T01:36:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:17.610+0000 7f30eaf9c700 1 -- 172.21.15.44:0/514702941 wait complete. 2023-09-10T01:36:19.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:18 smithi131 ceph-mon[103713]: pgmap v1813: 129 pgs: 11 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 97 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 837 KiB/s wr, 89 op/s 2023-09-10T01:36:19.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:18 smithi044 ceph-mon[98676]: pgmap v1813: 129 pgs: 11 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 97 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 837 KiB/s wr, 89 op/s 2023-09-10T01:36:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:18 smithi099 ceph-mon[102692]: pgmap v1813: 129 pgs: 11 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 97 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 837 KiB/s wr, 89 op/s 2023-09-10T01:36:21.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:20 smithi131 ceph-mon[103713]: pgmap v1814: 129 pgs: 7 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 112 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 884 KiB/s wr, 86 op/s 2023-09-10T01:36:21.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:20 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:21.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:20 smithi044 ceph-mon[98676]: pgmap v1814: 129 pgs: 7 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 112 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 884 KiB/s wr, 86 op/s 2023-09-10T01:36:21.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:20 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:20 smithi099 ceph-mon[102692]: pgmap v1814: 129 pgs: 7 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 112 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 884 KiB/s wr, 86 op/s 2023-09-10T01:36:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:20 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:22 smithi131 ceph-mon[103713]: pgmap v1815: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.7 MiB/s wr, 203 op/s 2023-09-10T01:36:23.310 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:22 smithi044 ceph-mon[98676]: pgmap v1815: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.7 MiB/s wr, 203 op/s 2023-09-10T01:36:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:22 smithi099 ceph-mon[102692]: pgmap v1815: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.7 MiB/s wr, 203 op/s 2023-09-10T01:36:25.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:25 smithi131 ceph-mon[103713]: pgmap v1816: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 2.1 MiB/s wr, 242 op/s 2023-09-10T01:36:25.308 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:25 smithi044 ceph-mon[98676]: pgmap v1816: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 2.1 MiB/s wr, 242 op/s 2023-09-10T01:36:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:25 smithi099 ceph-mon[102692]: pgmap v1816: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 2.1 MiB/s wr, 242 op/s 2023-09-10T01:36:27.363 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:27 smithi099 ceph-mon[102692]: pgmap v1817: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 2.1 MiB/s wr, 253 op/s 2023-09-10T01:36:27.364 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:27 smithi099 ceph-mon[102692]: osdmap e211: 12 total, 12 up, 12 in 2023-09-10T01:36:27.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:27 smithi131 ceph-mon[103713]: pgmap v1817: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 2.1 MiB/s wr, 253 op/s 2023-09-10T01:36:27.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:27 smithi131 ceph-mon[103713]: osdmap e211: 12 total, 12 up, 12 in 2023-09-10T01:36:27.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:27 smithi044 ceph-mon[98676]: pgmap v1817: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 2.1 MiB/s wr, 253 op/s 2023-09-10T01:36:27.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:27 smithi044 ceph-mon[98676]: osdmap e211: 12 total, 12 up, 12 in 2023-09-10T01:36:29.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:29 smithi099 ceph-mon[102692]: pgmap v1819: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 389 KiB/s rd, 4.9 MiB/s wr, 561 op/s 2023-09-10T01:36:29.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:29 smithi131 ceph-mon[103713]: pgmap v1819: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 389 KiB/s rd, 4.9 MiB/s wr, 561 op/s 2023-09-10T01:36:29.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:29 smithi044 ceph-mon[98676]: pgmap v1819: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 389 KiB/s rd, 4.9 MiB/s wr, 561 op/s 2023-09-10T01:36:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.797+0000 7f76c2b1f700 1 -- 172.21.15.44:0/586275876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76bc104410 msgr2=0x7f76bc1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.797+0000 7f76c2b1f700 1 --2- 172.21.15.44:0/586275876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76bc104410 0x7f76bc1047f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f76ac01a9a0 tx=0x7f76ac01acb0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.798+0000 7f76c2b1f700 1 -- 172.21.15.44:0/586275876 shutdown_connections 2023-09-10T01:36:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.798+0000 7f76c2b1f700 1 --2- 172.21.15.44:0/586275876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76bc0ffe70 0x7f76bc10c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.798+0000 7f76c2b1f700 1 --2- 172.21.15.44:0/586275876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76bc104410 0x7f76bc1047f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.798+0000 7f76c2b1f700 1 --2- 172.21.15.44:0/586275876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76bc0ff4b0 0x7f76bc0ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.798+0000 7f76c2b1f700 1 -- 172.21.15.44:0/586275876 >> 172.21.15.44:0/586275876 conn(0x7f76bc0765b0 msgr2=0x7f76bc0769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.798+0000 7f76c2b1f700 1 -- 172.21.15.44:0/586275876 shutdown_connections 2023-09-10T01:36:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.798+0000 7f76c2b1f700 1 -- 172.21.15.44:0/586275876 wait complete. 2023-09-10T01:36:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.799+0000 7f76c2b1f700 1 Processor -- start 2023-09-10T01:36:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.799+0000 7f76c2b1f700 1 -- start start 2023-09-10T01:36:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.799+0000 7f76c2b1f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76bc0ff4b0 0x7f76bc06ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.799+0000 7f76c2b1f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76bc0ffe70 0x7f76bc06cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.799+0000 7f76c2b1f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76bc104410 0x7f76bc191b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.799+0000 7f76c2b1f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc1921a0 con 0x7f76bc104410 2023-09-10T01:36:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.799+0000 7f76c2b1f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc192310 con 0x7f76bc0ff4b0 2023-09-10T01:36:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.799+0000 7f76c2b1f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc1924d0 con 0x7f76bc0ffe70 2023-09-10T01:36:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.799+0000 7f76c08bb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76bc0ff4b0 0x7f76bc06ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.799+0000 7f76c08bb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76bc0ff4b0 0x7f76bc06ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34890/0 (socket says 172.21.15.44:34890) 2023-09-10T01:36:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.799+0000 7f76c08bb700 1 -- 172.21.15.44:0/1436547825 learned_addr learned my addr 172.21.15.44:0/1436547825 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.800+0000 7f76bbfff700 1 --2- 172.21.15.44:0/1436547825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76bc0ffe70 0x7f76bc06cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.800+0000 7f76c08bb700 1 -- 172.21.15.44:0/1436547825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76bc0ffe70 msgr2=0x7f76bc06cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.800+0000 7f76c08bb700 1 --2- 172.21.15.44:0/1436547825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76bc0ffe70 0x7f76bc06cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.800+0000 7f76c08bb700 1 -- 172.21.15.44:0/1436547825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76bc104410 msgr2=0x7f76bc191b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.800+0000 7f76c08bb700 1 --2- 172.21.15.44:0/1436547825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76bc104410 0x7f76bc191b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.800+0000 7f76c08bb700 1 -- 172.21.15.44:0/1436547825 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac01a600 con 0x7f76bc0ff4b0 2023-09-10T01:36:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.800+0000 7f76c08bb700 1 --2- 172.21.15.44:0/1436547825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76bc0ff4b0 0x7f76bc06ca30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0058e0 tx=0x7f76ac005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.809+0000 7f76b9ffb700 1 -- 172.21.15.44:0/1436547825 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac02d070 con 0x7f76bc0ff4b0 2023-09-10T01:36:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.809+0000 7f76b9ffb700 1 -- 172.21.15.44:0/1436547825 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76ac031410 con 0x7f76bc0ff4b0 2023-09-10T01:36:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.809+0000 7f76c2b1f700 1 -- 172.21.15.44:0/1436547825 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76bc192750 con 0x7f76bc0ff4b0 2023-09-10T01:36:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.809+0000 7f76b9ffb700 1 -- 172.21.15.44:0/1436547825 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac0283f0 con 0x7f76bc0ff4b0 2023-09-10T01:36:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.809+0000 7f76c2b1f700 1 -- 172.21.15.44:0/1436547825 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76bc198de0 con 0x7f76bc0ff4b0 2023-09-10T01:36:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.812+0000 7f76b9ffb700 1 -- 172.21.15.44:0/1436547825 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f76ac031580 con 0x7f76bc0ff4b0 2023-09-10T01:36:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.813+0000 7f76b9ffb700 1 --2- 172.21.15.44:0/1436547825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76a40a0860 0x7f76a40a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.813+0000 7f76b9ffb700 1 -- 172.21.15.44:0/1436547825 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f76ac0ddca0 con 0x7f76bc0ff4b0 2023-09-10T01:36:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.814+0000 7f76cece5b80 1 Processor -- start 2023-09-10T01:36:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.814+0000 7f76bbfff700 1 --2- 172.21.15.44:0/1436547825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76a40a0860 0x7f76a40a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.814+0000 7f76cece5b80 1 -- start start 2023-09-10T01:36:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.814+0000 7f76cece5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a4b1a0 0x562499a4b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.814+0000 7f76cece5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a45300 0x562499a45780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.814+0000 7f76cece5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a45cc0 0x562499a51f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.814+0000 7f76cece5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249999e6d0 con 0x562499a4b1a0 2023-09-10T01:36:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.814+0000 7f76cece5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249999bff0 con 0x562499a45300 2023-09-10T01:36:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.814+0000 7f76cece5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624999978e0 con 0x562499a45cc0 2023-09-10T01:36:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.815+0000 7f76bbfff700 1 --2- 172.21.15.44:0/1436547825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76a40a0860 0x7f76a40a2d20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f76b0006fd0 tx=0x7f76b0008040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.815+0000 7f76c10bc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a45cc0 0x562499a51f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.815+0000 7f76c10bc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a45cc0 0x562499a51f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:40040/0 (socket says 172.21.15.44:40040) 2023-09-10T01:36:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.815+0000 7f76bbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a45300 0x562499a45780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.815+0000 7f76c10bc700 1 -- 172.21.15.44:0/669965736 learned_addr learned my addr 172.21.15.44:0/669965736 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.815+0000 7f76bbfff700 1 -- 172.21.15.44:0/669965736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a45cc0 msgr2=0x562499a51f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.815+0000 7f76bbfff700 1 --2- 172.21.15.44:0/669965736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a45cc0 0x562499a51f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.815+0000 7f76bbfff700 1 -- 172.21.15.44:0/669965736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a4b1a0 msgr2=0x562499a4b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.815+0000 7f76bbfff700 1 --2- 172.21.15.44:0/669965736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a4b1a0 0x562499a4b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.815+0000 7f76bbfff700 1 -- 172.21.15.44:0/669965736 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624997c3d80 con 0x562499a45300 2023-09-10T01:36:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.815+0000 7f76bbfff700 1 --2- 172.21.15.44:0/669965736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a45300 0x562499a45780 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f76bc105940 tx=0x7f76b0010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=658080d599142e93 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.826+0000 7f76a2ffd700 1 -- 172.21.15.44:0/669965736 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b0014430 con 0x562499a45300 2023-09-10T01:36:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.826+0000 7f76a2ffd700 1 -- 172.21.15.44:0/669965736 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76b0014a20 con 0x562499a45300 2023-09-10T01:36:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.826+0000 7f76a2ffd700 1 -- 172.21.15.44:0/669965736 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b0030960 con 0x562499a45300 2023-09-10T01:36:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.827+0000 7f76cece5b80 1 -- 172.21.15.44:0/669965736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a45300 msgr2=0x562499a45780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.827+0000 7f76cece5b80 1 --2- 172.21.15.44:0/669965736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a45300 0x562499a45780 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f76bc105940 tx=0x7f76b0010000 comp rx=0 tx=0).stop 2023-09-10T01:36:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.828+0000 7f76cece5b80 1 -- 172.21.15.44:0/669965736 shutdown_connections 2023-09-10T01:36:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.828+0000 7f76cece5b80 1 --2- 172.21.15.44:0/669965736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a45cc0 0x562499a51f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.828+0000 7f76cece5b80 1 --2- 172.21.15.44:0/669965736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a4b1a0 0x562499a4b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.828+0000 7f76cece5b80 1 --2- 172.21.15.44:0/669965736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a45300 0x562499a45780 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.828+0000 7f76cece5b80 1 -- 172.21.15.44:0/669965736 >> 172.21.15.44:0/669965736 conn(0x5624999ba030 msgr2=0x562499a40560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.828+0000 7f76cece5b80 1 -- 172.21.15.44:0/669965736 shutdown_connections 2023-09-10T01:36:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.828+0000 7f76cece5b80 1 -- 172.21.15.44:0/669965736 wait complete. 2023-09-10T01:36:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.828+0000 7f76cece5b80 1 -- start start 2023-09-10T01:36:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.828+0000 7f76cece5b80 1 Processor -- start 2023-09-10T01:36:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.829+0000 7f76cece5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a45300 0x562499a51f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.829+0000 7f76cece5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a45cc0 0x562499a4f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.829+0000 7f76cece5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a4b1a0 0x562499a4f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.829+0000 7f76cece5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562499935b10 con 0x562499a45cc0 2023-09-10T01:36:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.829+0000 7f76cece5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562499935810 con 0x562499a45300 2023-09-10T01:36:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.829+0000 7f76cece5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624998198c0 con 0x562499a4b1a0 2023-09-10T01:36:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.829+0000 7f76cece5b80 10 client.0 _ll_register_callbacks cb 0x56249987fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:36:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.829+0000 7f76c10bc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a4b1a0 0x562499a4f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.829+0000 7f76c10bc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a4b1a0 0x562499a4f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:40048/0 (socket says 172.21.15.44:40048) 2023-09-10T01:36:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.829+0000 7f76c10bc700 1 -- 172.21.15.44:0/20325229 learned_addr learned my addr 172.21.15.44:0/20325229 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.829+0000 7f76bbfff700 1 --2- 172.21.15.44:0/20325229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a45cc0 0x562499a4f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.829+0000 7f76c10bc700 1 -- 172.21.15.44:0/20325229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a45300 msgr2=0x562499a51f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.829+0000 7f76c10bc700 1 --2- 172.21.15.44:0/20325229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a45300 0x562499a51f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.829+0000 7f76c10bc700 1 -- 172.21.15.44:0/20325229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a45cc0 msgr2=0x562499a4f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.829+0000 7f76c10bc700 1 --2- 172.21.15.44:0/20325229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a45cc0 0x562499a4f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.829+0000 7f76c10bc700 1 -- 172.21.15.44:0/20325229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b000e040 con 0x562499a4b1a0 2023-09-10T01:36:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.830+0000 7f76c10bc700 1 --2- 172.21.15.44:0/20325229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a4b1a0 0x562499a4f880 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f76b400e810 tx=0x7f76b400eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8043e1314fdd7a46 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.830+0000 7f769bfff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-10T01:36:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.840+0000 7f769bfff700 1 -- 172.21.15.44:0/20325229 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4009a40 con 0x562499a4b1a0 2023-09-10T01:36:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.840+0000 7f76cece5b80 1 -- 172.21.15.44:0/20325229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624998ef490 con 0x562499a4b1a0 2023-09-10T01:36:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.840+0000 7f769bfff700 1 -- 172.21.15.44:0/20325229 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(211..211 src has 1..211) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f76b4014890 con 0x562499a4b1a0 2023-09-10T01:36:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.841+0000 7f769bfff700 1 -- 172.21.15.44:0/20325229 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=200) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76b4014a50 con 0x562499a4b1a0 2023-09-10T01:36:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.842+0000 7f76cece5b80 10 client.30436 fetch_fsmap learned FSMap version 200 2023-09-10T01:36:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.842+0000 7f76cece5b80 1 -- 172.21.15.44:0/20325229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=200,monmap=4+}) v3 -- 0x5624997acc00 con 0x562499a4b1a0 2023-09-10T01:36:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.842+0000 7f769bfff700 1 -- 172.21.15.44:0/20325229 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5624998ef490 con 0x562499a4b1a0 2023-09-10T01:36:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.842+0000 7f769bfff700 1 -- 172.21.15.44:0/20325229 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 200) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f76b401c3a0 con 0x562499a4b1a0 2023-09-10T01:36:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.842+0000 7f76cece5b80 10 client.30436 fetch_fsmap finished waiting for FSMap version 200 2023-09-10T01:36:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.842+0000 7f76cece5b80 10 client.30436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:36:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.842+0000 7f76cece5b80 20 client.30436 populate_metadata read hostname 'smithi044' 2023-09-10T01:36:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.842+0000 7f76cece5b80 1 --2- 172.21.15.44:0/20325229 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562499a5f5e0 0x562499a61a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.842+0000 7f76cece5b80 4 client.30436 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:36:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.842+0000 7f76cece5b80 1 -- 172.21.15.44:0/20325229 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624999ac840 con 0x562499a5f5e0 2023-09-10T01:36:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.842+0000 7f76c08bb700 1 --2- 172.21.15.44:0/20325229 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562499a5f5e0 0x562499a61a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.845+0000 7f769bfff700 10 client.30436 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.845+0000 7f76c08bb700 1 --2- 172.21.15.44:0/20325229 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562499a5f5e0 0x562499a61a00 crc :-1 s=READY pgs=622 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4880643aa890ff8 server_cookie=a9646f67dc0d2702 in_seq=0 out_seq=0 2023-09-10T01:36:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.846+0000 7f769bfff700 1 -- 172.21.15.44:0/20325229 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5624999ac840 con 0x562499a5f5e0 2023-09-10T01:36:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.846+0000 7f769bfff700 10 client.30436 handle_command_reply: tid=0 2023-09-10T01:36:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.846+0000 7f76cece5b80 1 client.30436 shutdown 2023-09-10T01:36:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.846+0000 7f76cece5b80 1 -- 172.21.15.44:0/20325229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a4b1a0 msgr2=0x562499a4f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.846+0000 7f76cece5b80 1 --2- 172.21.15.44:0/20325229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a4b1a0 0x562499a4f880 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f76b400e810 tx=0x7f76b400eb20 comp rx=0 tx=0).stop 2023-09-10T01:36:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.846+0000 7f76cece5b80 1 -- 172.21.15.44:0/20325229 shutdown_connections 2023-09-10T01:36:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.846+0000 7f76cece5b80 1 --2- 172.21.15.44:0/20325229 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562499a5f5e0 0x562499a61a00 crc :-1 s=READY pgs=622 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.846+0000 7f76cece5b80 1 --2- 172.21.15.44:0/20325229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a4b1a0 0x562499a4f880 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.846+0000 7f76cece5b80 1 --2- 172.21.15.44:0/20325229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a45300 0x562499a51f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.846+0000 7f76cece5b80 1 --2- 172.21.15.44:0/20325229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a45cc0 0x562499a4f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.846+0000 7f76cece5b80 1 -- 172.21.15.44:0/20325229 >> 172.21.15.44:0/20325229 conn(0x5624999ba030 msgr2=0x562499a3f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.846+0000 7f769bfff700 0 client.30436 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.847+0000 7f76cece5b80 1 -- 172.21.15.44:0/20325229 shutdown_connections 2023-09-10T01:36:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.847+0000 7f76cece5b80 1 -- 172.21.15.44:0/20325229 wait complete. 2023-09-10T01:36:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.847+0000 7f76cece5b80 20 client.30436 trim_cache size 0 max 16384 2023-09-10T01:36:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.870+0000 7f76cece5b80 1 Processor -- start 2023-09-10T01:36:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.870+0000 7f76cece5b80 1 -- start start 2023-09-10T01:36:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.870+0000 7f76cece5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a45300 0x562499a4cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.870+0000 7f76cece5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a45cc0 0x562499a51a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.870+0000 7f76cece5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a4b1a0 0x562499a491f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.870+0000 7f76cece5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562499a3cfc0 con 0x562499a45cc0 2023-09-10T01:36:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.870+0000 7f76cece5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562499a41930 con 0x562499a45300 2023-09-10T01:36:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.870+0000 7f76cece5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562499a41b40 con 0x562499a4b1a0 2023-09-10T01:36:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.870+0000 7f76c10bc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a4b1a0 0x562499a491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.870+0000 7f76c08bb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a45300 0x562499a4cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.870+0000 7f76c10bc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a4b1a0 0x562499a491f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.44:40058/0 (socket says 172.21.15.44:40058) 2023-09-10T01:36:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.870+0000 7f76c10bc700 1 -- 172.21.15.44:0/969862707 learned_addr learned my addr 172.21.15.44:0/969862707 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.871+0000 7f76c10bc700 1 -- 172.21.15.44:0/969862707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a45300 msgr2=0x562499a4cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.871+0000 7f76c10bc700 1 --2- 172.21.15.44:0/969862707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a45300 0x562499a4cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.871+0000 7f76c10bc700 1 -- 172.21.15.44:0/969862707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a45cc0 msgr2=0x562499a51a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.871+0000 7f76c10bc700 1 --2- 172.21.15.44:0/969862707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a45cc0 0x562499a51a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.871+0000 7f76c10bc700 1 -- 172.21.15.44:0/969862707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624997ea1f0 con 0x562499a4b1a0 2023-09-10T01:36:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.871+0000 7f76c10bc700 1 --2- 172.21.15.44:0/969862707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a4b1a0 0x562499a491f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f76b4007030 tx=0x7f76b400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=144de95a3cef1a24 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.871+0000 7f76a37fe700 1 -- 172.21.15.44:0/969862707 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b401cd50 con 0x562499a4b1a0 2023-09-10T01:36:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.871+0000 7f76a37fe700 1 -- 172.21.15.44:0/969862707 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76b400f040 con 0x562499a4b1a0 2023-09-10T01:36:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.871+0000 7f76a37fe700 1 -- 172.21.15.44:0/969862707 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4014e10 con 0x562499a4b1a0 2023-09-10T01:36:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.872+0000 7f76cece5b80 1 -- 172.21.15.44:0/969862707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a4b1a0 msgr2=0x562499a491f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.872+0000 7f76cece5b80 1 --2- 172.21.15.44:0/969862707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a4b1a0 0x562499a491f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f76b4007030 tx=0x7f76b400ef90 comp rx=0 tx=0).stop 2023-09-10T01:36:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.872+0000 7f76cece5b80 1 -- 172.21.15.44:0/969862707 shutdown_connections 2023-09-10T01:36:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.872+0000 7f76cece5b80 1 --2- 172.21.15.44:0/969862707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a4b1a0 0x562499a491f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.872+0000 7f76cece5b80 1 --2- 172.21.15.44:0/969862707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a45300 0x562499a4cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.872+0000 7f76cece5b80 1 --2- 172.21.15.44:0/969862707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a45cc0 0x562499a51a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.872+0000 7f76cece5b80 1 -- 172.21.15.44:0/969862707 >> 172.21.15.44:0/969862707 conn(0x5624999ba030 msgr2=0x562499a66320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.872+0000 7f76cece5b80 1 -- 172.21.15.44:0/969862707 shutdown_connections 2023-09-10T01:36:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.873+0000 7f76cece5b80 1 -- 172.21.15.44:0/969862707 wait complete. 2023-09-10T01:36:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.873+0000 7f76cece5b80 1 -- start start 2023-09-10T01:36:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.873+0000 7f76cece5b80 1 Processor -- start 2023-09-10T01:36:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.873+0000 7f76cece5b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a45300 0x562499a55ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.873+0000 7f76cece5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a45cc0 0x562499a56410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.874+0000 7f76cece5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a4b1a0 0x562499a54510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.874+0000 7f76cece5b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562499a54ad0 con 0x562499a45300 2023-09-10T01:36:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.874+0000 7f76cece5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562499a54c40 con 0x562499a4b1a0 2023-09-10T01:36:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.874+0000 7f76cece5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562499a54e00 con 0x562499a45cc0 2023-09-10T01:36:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.874+0000 7f76cece5b80 10 client.0 _ll_register_callbacks cb 0x7f76b400e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:36:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.874+0000 7f76c10bc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a4b1a0 0x562499a54510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.874+0000 7f76c10bc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a4b1a0 0x562499a54510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34920/0 (socket says 172.21.15.44:34920) 2023-09-10T01:36:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.874+0000 7f76c10bc700 1 -- 172.21.15.44:0/325352426 learned_addr learned my addr 172.21.15.44:0/325352426 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.874+0000 7f76c08bb700 1 --2- 172.21.15.44:0/325352426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a45300 0x562499a55ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.874+0000 7f76c10bc700 1 -- 172.21.15.44:0/325352426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a45cc0 msgr2=0x562499a56410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-10T01:36:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.874+0000 7f76c10bc700 1 --2- 172.21.15.44:0/325352426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a45cc0 0x562499a56410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.874+0000 7f76c10bc700 1 -- 172.21.15.44:0/325352426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a45300 msgr2=0x562499a55ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.874+0000 7f76c10bc700 1 --2- 172.21.15.44:0/325352426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a45300 0x562499a55ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.874+0000 7f76c10bc700 1 -- 172.21.15.44:0/325352426 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624997acc00 con 0x562499a4b1a0 2023-09-10T01:36:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.875+0000 7f76c10bc700 1 --2- 172.21.15.44:0/325352426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a4b1a0 0x562499a54510 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f76b4002230 tx=0x7f76b401ca70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a629ee79fd2a14e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.881+0000 7f769bfff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-09-10T01:36:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.881+0000 7f769bfff700 1 -- 172.21.15.44:0/325352426 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4014690 con 0x562499a4b1a0 2023-09-10T01:36:30.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.881+0000 7f769bfff700 1 -- 172.21.15.44:0/325352426 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(211..211 src has 1..211) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f76b4005e40 con 0x562499a4b1a0 2023-09-10T01:36:30.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.881+0000 7f76cece5b80 1 -- 172.21.15.44:0/325352426 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76b0014a20 con 0x562499a4b1a0 2023-09-10T01:36:30.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.881+0000 7f769bfff700 1 -- 172.21.15.44:0/325352426 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=200) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76b40147f0 con 0x562499a4b1a0 2023-09-10T01:36:30.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.881+0000 7f76cece5b80 10 client.30410 fetch_fsmap learned FSMap version 200 2023-09-10T01:36:30.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.881+0000 7f76cece5b80 1 -- 172.21.15.44:0/325352426 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=200,monmap=4+}) v3 -- 0x5624997c2250 con 0x562499a4b1a0 2023-09-10T01:36:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.881+0000 7f769bfff700 1 -- 172.21.15.44:0/325352426 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76b0014a20 con 0x562499a4b1a0 2023-09-10T01:36:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.882+0000 7f769bfff700 1 -- 172.21.15.44:0/325352426 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 200) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f76b402d030 con 0x562499a4b1a0 2023-09-10T01:36:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.882+0000 7f76cece5b80 10 client.30410 fetch_fsmap finished waiting for FSMap version 200 2023-09-10T01:36:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.882+0000 7f76cece5b80 10 client.30410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:36:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.882+0000 7f76cece5b80 20 client.30410 populate_metadata read hostname 'smithi044' 2023-09-10T01:36:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.882+0000 7f76cece5b80 1 --2- 172.21.15.44:0/325352426 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562499ab4150 0x562499ab6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.882+0000 7f76cece5b80 4 client.30410 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:36:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.882+0000 7f76cece5b80 1 -- 172.21.15.44:0/325352426 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5624999908c0 con 0x562499ab4150 2023-09-10T01:36:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.882+0000 7f76c08bb700 1 --2- 172.21.15.44:0/325352426 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562499ab4150 0x562499ab6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.882+0000 7f769bfff700 10 client.30410 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:30.882+0000 7f76c08bb700 1 --2- 172.21.15.44:0/325352426 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562499ab4150 0x562499ab6570 crc :-1 s=READY pgs=623 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b363406624a794ae server_cookie=2e91ef5523a860a3 in_seq=0 out_seq=0 2023-09-10T01:36:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.010+0000 7f769bfff700 1 -- 172.21.15.44:0/325352426 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5624999908c0 con 0x562499ab4150 2023-09-10T01:36:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.010+0000 7f769bfff700 10 client.30410 handle_command_reply: tid=0 2023-09-10T01:36:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.010+0000 7f76cece5b80 1 client.30410 shutdown 2023-09-10T01:36:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.017+0000 7f76cece5b80 1 -- 172.21.15.44:0/325352426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a4b1a0 msgr2=0x562499a54510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.017+0000 7f76cece5b80 1 --2- 172.21.15.44:0/325352426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a4b1a0 0x562499a54510 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f76b4002230 tx=0x7f76b401ca70 comp rx=0 tx=0).stop 2023-09-10T01:36:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.017+0000 7f76cece5b80 1 -- 172.21.15.44:0/325352426 shutdown_connections 2023-09-10T01:36:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.017+0000 7f76cece5b80 1 --2- 172.21.15.44:0/325352426 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x562499ab4150 0x562499ab6570 crc :-1 s=READY pgs=623 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.017+0000 7f76cece5b80 1 --2- 172.21.15.44:0/325352426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562499a4b1a0 0x562499a54510 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.017+0000 7f76cece5b80 1 --2- 172.21.15.44:0/325352426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562499a45300 0x562499a55ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.017+0000 7f76cece5b80 1 --2- 172.21.15.44:0/325352426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562499a45cc0 0x562499a56410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.017+0000 7f76cece5b80 1 -- 172.21.15.44:0/325352426 >> 172.21.15.44:0/325352426 conn(0x5624999ba030 msgr2=0x562499a491f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.019+0000 7f769bfff700 0 client.30410 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.019+0000 7f76cece5b80 1 -- 172.21.15.44:0/325352426 shutdown_connections 2023-09-10T01:36:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.019+0000 7f76cece5b80 1 -- 172.21.15.44:0/325352426 wait complete. 2023-09-10T01:36:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.020+0000 7f76cece5b80 20 client.30410 trim_cache size 0 max 16384 2023-09-10T01:36:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:36:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:36:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:36:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:36:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.029+0000 7f76c2b1f700 1 -- 172.21.15.44:0/1436547825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76a40a0860 msgr2=0x7f76a40a2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.029+0000 7f76c2b1f700 1 --2- 172.21.15.44:0/1436547825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76a40a0860 0x7f76a40a2d20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f76b0006fd0 tx=0x7f76b0008040 comp rx=0 tx=0).stop 2023-09-10T01:36:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.029+0000 7f76c2b1f700 1 -- 172.21.15.44:0/1436547825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76bc0ff4b0 msgr2=0x7f76bc06ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.029+0000 7f76c2b1f700 1 --2- 172.21.15.44:0/1436547825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76bc0ff4b0 0x7f76bc06ca30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0058e0 tx=0x7f76ac005750 comp rx=0 tx=0).stop 2023-09-10T01:36:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.031+0000 7f76c2b1f700 1 -- 172.21.15.44:0/1436547825 shutdown_connections 2023-09-10T01:36:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.031+0000 7f76c2b1f700 1 --2- 172.21.15.44:0/1436547825 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f76a40a0860 0x7f76a40a2d20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.031+0000 7f76c2b1f700 1 --2- 172.21.15.44:0/1436547825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76bc104410 0x7f76bc191b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.031+0000 7f76c2b1f700 1 --2- 172.21.15.44:0/1436547825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76bc0ff4b0 0x7f76bc06ca30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.031+0000 7f76c2b1f700 1 --2- 172.21.15.44:0/1436547825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76bc0ffe70 0x7f76bc06cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.031+0000 7f76c2b1f700 1 -- 172.21.15.44:0/1436547825 >> 172.21.15.44:0/1436547825 conn(0x7f76bc0765b0 msgr2=0x7f76bc0fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.031+0000 7f76c2b1f700 1 -- 172.21.15.44:0/1436547825 shutdown_connections 2023-09-10T01:36:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:31.035+0000 7f76c2b1f700 1 -- 172.21.15.44:0/1436547825 wait complete. 2023-09-10T01:36:31.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:31 smithi131 ceph-mon[103713]: pgmap v1820: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 314 KiB/s rd, 5.4 MiB/s wr, 625 op/s 2023-09-10T01:36:31.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:31 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:31.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:31 smithi044 ceph-mon[98676]: pgmap v1820: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 314 KiB/s rd, 5.4 MiB/s wr, 625 op/s 2023-09-10T01:36:31.555 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:31 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:31.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:31 smithi099 ceph-mon[102692]: pgmap v1820: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 314 KiB/s rd, 5.4 MiB/s wr, 625 op/s 2023-09-10T01:36:31.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:31 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:32 smithi131 ceph-mon[103713]: pgmap v1821: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 184 KiB/s rd, 7.0 MiB/s wr, 795 op/s 2023-09-10T01:36:33.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:32 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:32 smithi044 ceph-mon[98676]: pgmap v1821: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 184 KiB/s rd, 7.0 MiB/s wr, 795 op/s 2023-09-10T01:36:33.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:32 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:32 smithi099 ceph-mon[102692]: pgmap v1821: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 184 KiB/s rd, 7.0 MiB/s wr, 795 op/s 2023-09-10T01:36:33.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:32 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:35.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:34 smithi131 ceph-mon[103713]: pgmap v1822: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 184 KiB/s rd, 7.5 MiB/s wr, 863 op/s 2023-09-10T01:36:35.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:34 smithi044 ceph-mon[98676]: pgmap v1822: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 184 KiB/s rd, 7.5 MiB/s wr, 863 op/s 2023-09-10T01:36:35.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:34 smithi099 ceph-mon[102692]: pgmap v1822: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 184 KiB/s rd, 7.5 MiB/s wr, 863 op/s 2023-09-10T01:36:37.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:36 smithi131 ceph-mon[103713]: pgmap v1823: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 224 KiB/s rd, 7.6 MiB/s wr, 872 op/s 2023-09-10T01:36:37.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:36 smithi044 ceph-mon[98676]: pgmap v1823: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 224 KiB/s rd, 7.6 MiB/s wr, 872 op/s 2023-09-10T01:36:37.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:36 smithi099 ceph-mon[102692]: pgmap v1823: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 224 KiB/s rd, 7.6 MiB/s wr, 872 op/s 2023-09-10T01:36:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:38 smithi131 ceph-mon[103713]: pgmap v1824: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 228 KiB/s rd, 6.2 MiB/s wr, 675 op/s 2023-09-10T01:36:39.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:38 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:36:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:38 smithi044 ceph-mon[98676]: pgmap v1824: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 228 KiB/s rd, 6.2 MiB/s wr, 675 op/s 2023-09-10T01:36:39.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:38 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:36:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:38 smithi099 ceph-mon[102692]: pgmap v1824: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 228 KiB/s rd, 6.2 MiB/s wr, 675 op/s 2023-09-10T01:36:39.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:38 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:36:40.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:36:40.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:36:40.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:39 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:36:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:36:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:36:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:39 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:36:40.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:36:40.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:36:40.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:39 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:36:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:40 smithi131 ceph-mon[103713]: pgmap v1825: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 266 KiB/s rd, 5.3 MiB/s wr, 605 op/s 2023-09-10T01:36:41.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:40 smithi131 ceph-mon[103713]: pgmap v1826: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 299 KiB/s rd, 5.8 MiB/s wr, 657 op/s 2023-09-10T01:36:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:40 smithi044 ceph-mon[98676]: pgmap v1825: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 266 KiB/s rd, 5.3 MiB/s wr, 605 op/s 2023-09-10T01:36:41.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:40 smithi044 ceph-mon[98676]: pgmap v1826: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 299 KiB/s rd, 5.8 MiB/s wr, 657 op/s 2023-09-10T01:36:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:40 smithi099 ceph-mon[102692]: pgmap v1825: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 266 KiB/s rd, 5.3 MiB/s wr, 605 op/s 2023-09-10T01:36:41.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:40 smithi099 ceph-mon[102692]: pgmap v1826: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 299 KiB/s rd, 5.8 MiB/s wr, 657 op/s 2023-09-10T01:36:43.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:42 smithi131 ceph-mon[103713]: pgmap v1827: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 301 KiB/s rd, 2.9 MiB/s wr, 329 op/s 2023-09-10T01:36:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:42 smithi044 ceph-mon[98676]: pgmap v1827: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 301 KiB/s rd, 2.9 MiB/s wr, 329 op/s 2023-09-10T01:36:43.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:42 smithi099 ceph-mon[102692]: pgmap v1827: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 301 KiB/s rd, 2.9 MiB/s wr, 329 op/s 2023-09-10T01:36:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.169+0000 7f08f8a63700 1 -- 172.21.15.44:0/4204639885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08f4068f30 msgr2=0x7f08f410d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.169+0000 7f08f8a63700 1 --2- 172.21.15.44:0/4204639885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08f4068f30 0x7f08f410d450 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00b0d0 tx=0x7f08ec00b3e0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.169+0000 7f08f8a63700 1 -- 172.21.15.44:0/4204639885 shutdown_connections 2023-09-10T01:36:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.169+0000 7f08f8a63700 1 --2- 172.21.15.44:0/4204639885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08f4068f30 0x7f08f410d450 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.169+0000 7f08f8a63700 1 --2- 172.21.15.44:0/4204639885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08f41066d0 0x7f08f4106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.169+0000 7f08f8a63700 1 --2- 172.21.15.44:0/4204639885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08f4068570 0x7f08f40689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.169+0000 7f08f8a63700 1 -- 172.21.15.44:0/4204639885 >> 172.21.15.44:0/4204639885 conn(0x7f08f40773d0 msgr2=0x7f08f40777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.169+0000 7f08f8a63700 1 -- 172.21.15.44:0/4204639885 shutdown_connections 2023-09-10T01:36:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.169+0000 7f08f8a63700 1 -- 172.21.15.44:0/4204639885 wait complete. 2023-09-10T01:36:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f8a63700 1 Processor -- start 2023-09-10T01:36:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f8a63700 1 -- start start 2023-09-10T01:36:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f8a63700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08f4068570 0x7f08f40ff940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f8a63700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08f4068f30 0x7f08f40ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f8a63700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08f41066d0 0x7f08f41a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f8a63700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f41a4e20 con 0x7f08f4068f30 2023-09-10T01:36:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f8a63700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f41a4f90 con 0x7f08f41066d0 2023-09-10T01:36:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f8a63700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f41a5150 con 0x7f08f4068570 2023-09-10T01:36:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08f4068f30 0x7f08f40ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08f4068f30 0x7f08f40ffe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33192/0 (socket says 172.21.15.44:33192) 2023-09-10T01:36:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f1d9b700 1 -- 172.21.15.44:0/2459750719 learned_addr learned my addr 172.21.15.44:0/2459750719 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f2d9d700 1 --2- 172.21.15.44:0/2459750719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08f41066d0 0x7f08f41a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f259c700 1 --2- 172.21.15.44:0/2459750719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08f4068570 0x7f08f40ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f1d9b700 1 -- 172.21.15.44:0/2459750719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08f4068570 msgr2=0x7f08f40ff940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f1d9b700 1 --2- 172.21.15.44:0/2459750719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08f4068570 0x7f08f40ff940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f1d9b700 1 -- 172.21.15.44:0/2459750719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08f41066d0 msgr2=0x7f08f41a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f1d9b700 1 --2- 172.21.15.44:0/2459750719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08f41066d0 0x7f08f41a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.170+0000 7f08f1d9b700 1 -- 172.21.15.44:0/2459750719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec009d00 con 0x7f08f4068f30 2023-09-10T01:36:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.171+0000 7f08f1d9b700 1 --2- 172.21.15.44:0/2459750719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08f4068f30 0x7f08f40ffe80 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f08e400d950 tx=0x7f08e400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.171+0000 7f08eb7fe700 1 -- 172.21.15.44:0/2459750719 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4009940 con 0x7f08f4068f30 2023-09-10T01:36:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.171+0000 7f08eb7fe700 1 -- 172.21.15.44:0/2459750719 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08e4010460 con 0x7f08f4068f30 2023-09-10T01:36:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.171+0000 7f08f8a63700 1 -- 172.21.15.44:0/2459750719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f41a5430 con 0x7f08f4068f30 2023-09-10T01:36:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.171+0000 7f08eb7fe700 1 -- 172.21.15.44:0/2459750719 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e400f620 con 0x7f08f4068f30 2023-09-10T01:36:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.171+0000 7f08f8a63700 1 -- 172.21.15.44:0/2459750719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f41a5940 con 0x7f08f4068f30 2023-09-10T01:36:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.173+0000 7f08eb7fe700 1 -- 172.21.15.44:0/2459750719 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f08e40105d0 con 0x7f08f4068f30 2023-09-10T01:36:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.174+0000 7f08eb7fe700 1 --2- 172.21.15.44:0/2459750719 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f08e00a07b0 0x7f08e00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.174+0000 7f08eb7fe700 1 -- 172.21.15.44:0/2459750719 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f08e40cc250 con 0x7f08f4068f30 2023-09-10T01:36:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.174+0000 7f08f259c700 1 --2- 172.21.15.44:0/2459750719 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f08e00a07b0 0x7f08e00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.174+0000 7f08f259c700 1 --2- 172.21.15.44:0/2459750719 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f08e00a07b0 0x7f08e00a2c70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f08dc005950 tx=0x7f08dc0058e0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f0904c29b80 1 Processor -- start 2023-09-10T01:36:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f0904c29b80 1 -- start start 2023-09-10T01:36:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f0904c29b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a4b1a0 0x55c694a4b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f0904c29b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45300 0x55c694a45780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f0904c29b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a45cc0 0x55c694a51f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f0904c29b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69499e6d0 con 0x55c694a4b1a0 2023-09-10T01:36:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f0904c29b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69499bff0 con 0x55c694a45300 2023-09-10T01:36:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f0904c29b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6949978e0 con 0x55c694a45cc0 2023-09-10T01:36:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f08f259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a4b1a0 0x55c694a4b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f08f259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a4b1a0 0x55c694a4b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33198/0 (socket says 172.21.15.44:33198) 2023-09-10T01:36:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f08f259c700 1 -- 172.21.15.44:0/268845072 learned_addr learned my addr 172.21.15.44:0/268845072 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f08f2d9d700 1 --2- 172.21.15.44:0/268845072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a45cc0 0x55c694a51f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f08f1d9b700 1 --2- 172.21.15.44:0/268845072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45300 0x55c694a45780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f08f259c700 1 -- 172.21.15.44:0/268845072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a45cc0 msgr2=0x55c694a51f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f08f259c700 1 --2- 172.21.15.44:0/268845072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a45cc0 0x55c694a51f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f08f259c700 1 -- 172.21.15.44:0/268845072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45300 msgr2=0x55c694a45780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f08f259c700 1 --2- 172.21.15.44:0/268845072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45300 0x55c694a45780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f08f259c700 1 -- 172.21.15.44:0/268845072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6947c3d80 con 0x55c694a4b1a0 2023-09-10T01:36:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f08f2d9d700 1 --2- 172.21.15.44:0/268845072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a45cc0 0x55c694a51f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:36:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f08f1d9b700 1 --2- 172.21.15.44:0/268845072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45300 0x55c694a45780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:36:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.175+0000 7f08f259c700 1 --2- 172.21.15.44:0/268845072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a4b1a0 0x55c694a4b580 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00fcd0 tx=0x7f08dc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ac4964fb5d814 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.176+0000 7f08e8ff9700 1 -- 172.21.15.44:0/268845072 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc011c40 con 0x55c694a4b1a0 2023-09-10T01:36:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.176+0000 7f08e8ff9700 1 -- 172.21.15.44:0/268845072 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08dc011da0 con 0x55c694a4b1a0 2023-09-10T01:36:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.176+0000 7f08e8ff9700 1 -- 172.21.15.44:0/268845072 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc01d610 con 0x55c694a4b1a0 2023-09-10T01:36:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.176+0000 7f0904c29b80 1 -- 172.21.15.44:0/268845072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a4b1a0 msgr2=0x55c694a4b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.176+0000 7f0904c29b80 1 --2- 172.21.15.44:0/268845072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a4b1a0 0x55c694a4b580 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00fcd0 tx=0x7f08dc012000 comp rx=0 tx=0).stop 2023-09-10T01:36:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.176+0000 7f0904c29b80 1 -- 172.21.15.44:0/268845072 shutdown_connections 2023-09-10T01:36:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.176+0000 7f0904c29b80 1 --2- 172.21.15.44:0/268845072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a45cc0 0x55c694a51f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.176+0000 7f0904c29b80 1 --2- 172.21.15.44:0/268845072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a4b1a0 0x55c694a4b580 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.176+0000 7f0904c29b80 1 --2- 172.21.15.44:0/268845072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45300 0x55c694a45780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.176+0000 7f0904c29b80 1 -- 172.21.15.44:0/268845072 >> 172.21.15.44:0/268845072 conn(0x55c6949ba030 msgr2=0x55c694a40560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.177+0000 7f0904c29b80 1 -- 172.21.15.44:0/268845072 shutdown_connections 2023-09-10T01:36:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.177+0000 7f0904c29b80 1 -- 172.21.15.44:0/268845072 wait complete. 2023-09-10T01:36:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.177+0000 7f0904c29b80 1 -- start start 2023-09-10T01:36:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.177+0000 7f0904c29b80 1 Processor -- start 2023-09-10T01:36:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.177+0000 7f0904c29b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45300 0x55c694a51d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.177+0000 7f0904c29b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a45cc0 0x55c694a4f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.177+0000 7f0904c29b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a4b1a0 0x55c694a4f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.177+0000 7f0904c29b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c694935b10 con 0x55c694a45cc0 2023-09-10T01:36:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.177+0000 7f0904c29b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c694935810 con 0x55c694a45300 2023-09-10T01:36:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.177+0000 7f0904c29b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6948198c0 con 0x55c694a4b1a0 2023-09-10T01:36:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f0904c29b80 10 client.0 _ll_register_callbacks cb 0x55c69487fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:36:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f08f1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a45cc0 0x55c694a4f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f08f1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a45cc0 0x55c694a4f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33206/0 (socket says 172.21.15.44:33206) 2023-09-10T01:36:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f08f1d9b700 1 -- 172.21.15.44:0/3587655760 learned_addr learned my addr 172.21.15.44:0/3587655760 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f08f259c700 1 --2- 172.21.15.44:0/3587655760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45300 0x55c694a51d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f08f2d9d700 1 --2- 172.21.15.44:0/3587655760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a4b1a0 0x55c694a4f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f08f1d9b700 1 -- 172.21.15.44:0/3587655760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a4b1a0 msgr2=0x55c694a4f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f08f1d9b700 1 --2- 172.21.15.44:0/3587655760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a4b1a0 0x55c694a4f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f08f1d9b700 1 -- 172.21.15.44:0/3587655760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45300 msgr2=0x55c694a51d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f08f1d9b700 1 --2- 172.21.15.44:0/3587655760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45300 0x55c694a51d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f08f1d9b700 1 -- 172.21.15.44:0/3587655760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc00f980 con 0x55c694a45cc0 2023-09-10T01:36:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f08f2d9d700 1 --2- 172.21.15.44:0/3587655760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a4b1a0 0x55c694a4f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:36:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f08f1d9b700 1 --2- 172.21.15.44:0/3587655760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a45cc0 0x55c694a4f030 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f08e4099230 tx=0x7f08e4098cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6deba6ebdc8abd37 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f08d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:36:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f08d9ffb700 1 -- 172.21.15.44:0/3587655760 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e409d0d0 con 0x55c694a45cc0 2023-09-10T01:36:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f08d9ffb700 1 -- 172.21.15.44:0/3587655760 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(211..211 src has 1..211) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f08e40a1310 con 0x55c694a45cc0 2023-09-10T01:36:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f0904c29b80 1 -- 172.21.15.44:0/3587655760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6948ef490 con 0x55c694a45cc0 2023-09-10T01:36:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f08d9ffb700 1 -- 172.21.15.44:0/3587655760 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08e4098790 con 0x55c694a45cc0 2023-09-10T01:36:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f0904c29b80 10 client.20769 fetch_fsmap learned FSMap version 201 2023-09-10T01:36:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.178+0000 7f0904c29b80 1 -- 172.21.15.44:0/3587655760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=201,monmap=4+}) v3 -- 0x55c6947acc00 con 0x55c694a45cc0 2023-09-10T01:36:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.179+0000 7f08d9ffb700 1 -- 172.21.15.44:0/3587655760 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08e40b7030 con 0x55c694a45cc0 2023-09-10T01:36:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.179+0000 7f08d9ffb700 1 -- 172.21.15.44:0/3587655760 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 201) v1 ==== 7375+0+0 (secure 0 0 0) 0x7f08e409b070 con 0x55c694a45cc0 2023-09-10T01:36:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.179+0000 7f0904c29b80 10 client.20769 fetch_fsmap finished waiting for FSMap version 201 2023-09-10T01:36:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.179+0000 7f0904c29b80 10 client.20769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:36:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.179+0000 7f0904c29b80 20 client.20769 populate_metadata read hostname 'smithi044' 2023-09-10T01:36:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.179+0000 7f0904c29b80 1 --2- 172.21.15.44:0/3587655760 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c694a5f3a0 0x55c694a617c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.179+0000 7f0904c29b80 4 client.20769 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:36:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.179+0000 7f0904c29b80 1 -- 172.21.15.44:0/3587655760 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6949ac840 con 0x55c694a5f3a0 2023-09-10T01:36:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.179+0000 7f08f259c700 1 --2- 172.21.15.44:0/3587655760 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c694a5f3a0 0x55c694a617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.180+0000 7f08f259c700 1 --2- 172.21.15.44:0/3587655760 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c694a5f3a0 0x55c694a617c0 crc :-1 s=READY pgs=624 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9327dbb9f02e1e server_cookie=802beee987f6db03 in_seq=0 out_seq=0 2023-09-10T01:36:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.180+0000 7f08d9ffb700 10 client.20769 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.180+0000 7f08d9ffb700 1 -- 172.21.15.44:0/3587655760 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c6949ac840 con 0x55c694a5f3a0 2023-09-10T01:36:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.180+0000 7f08d9ffb700 10 client.20769 handle_command_reply: tid=0 2023-09-10T01:36:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.180+0000 7f0904c29b80 1 client.20769 shutdown 2023-09-10T01:36:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.180+0000 7f0904c29b80 1 -- 172.21.15.44:0/3587655760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a45cc0 msgr2=0x55c694a4f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.180+0000 7f0904c29b80 1 --2- 172.21.15.44:0/3587655760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a45cc0 0x55c694a4f030 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f08e4099230 tx=0x7f08e4098cf0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.180+0000 7f0904c29b80 1 -- 172.21.15.44:0/3587655760 shutdown_connections 2023-09-10T01:36:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.180+0000 7f0904c29b80 1 --2- 172.21.15.44:0/3587655760 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c694a5f3a0 0x55c694a617c0 crc :-1 s=READY pgs=624 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.180+0000 7f0904c29b80 1 --2- 172.21.15.44:0/3587655760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a4b1a0 0x55c694a4f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.180+0000 7f0904c29b80 1 --2- 172.21.15.44:0/3587655760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45300 0x55c694a51d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.180+0000 7f08d9ffb700 0 client.20769 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.180+0000 7f0904c29b80 1 --2- 172.21.15.44:0/3587655760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a45cc0 0x55c694a4f030 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.180+0000 7f0904c29b80 1 -- 172.21.15.44:0/3587655760 >> 172.21.15.44:0/3587655760 conn(0x55c6949ba030 msgr2=0x55c694a3f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.180+0000 7f0904c29b80 1 -- 172.21.15.44:0/3587655760 shutdown_connections 2023-09-10T01:36:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.180+0000 7f0904c29b80 1 -- 172.21.15.44:0/3587655760 wait complete. 2023-09-10T01:36:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.181+0000 7f0904c29b80 20 client.20769 trim_cache size 0 max 16384 2023-09-10T01:36:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.195+0000 7f0904c29b80 1 Processor -- start 2023-09-10T01:36:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.195+0000 7f0904c29b80 1 -- start start 2023-09-10T01:36:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f0904c29b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a45300 0x55c694a56530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f0904c29b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45cc0 0x55c694a44a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f0904c29b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a4b1a0 0x55c694a51840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f0904c29b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c694a50610 con 0x55c694a4b1a0 2023-09-10T01:36:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f0904c29b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c694a56a70 con 0x55c694a45cc0 2023-09-10T01:36:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f0904c29b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c694a4adb0 con 0x55c694a45300 2023-09-10T01:36:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f08f2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a4b1a0 0x55c694a51840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f08f2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a4b1a0 0x55c694a51840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33216/0 (socket says 172.21.15.44:33216) 2023-09-10T01:36:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f08f2d9d700 1 -- 172.21.15.44:0/2535355254 learned_addr learned my addr 172.21.15.44:0/2535355254 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f08f259c700 1 --2- 172.21.15.44:0/2535355254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a45300 0x55c694a56530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f08f1d9b700 1 --2- 172.21.15.44:0/2535355254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45cc0 0x55c694a44a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f08f2d9d700 1 -- 172.21.15.44:0/2535355254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a45300 msgr2=0x55c694a56530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f08f2d9d700 1 --2- 172.21.15.44:0/2535355254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a45300 0x55c694a56530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f08f2d9d700 1 -- 172.21.15.44:0/2535355254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45cc0 msgr2=0x55c694a44a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f08f2d9d700 1 --2- 172.21.15.44:0/2535355254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45cc0 0x55c694a44a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f08f2d9d700 1 -- 172.21.15.44:0/2535355254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6947ea1f0 con 0x55c694a4b1a0 2023-09-10T01:36:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f08f1d9b700 1 --2- 172.21.15.44:0/2535355254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45cc0 0x55c694a44a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:36:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.196+0000 7f08f2d9d700 1 --2- 172.21.15.44:0/2535355254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a4b1a0 0x55c694a51840 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0060a0 tx=0x7f08ec0060d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b9016b7537c3cd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.197+0000 7f08e97fa700 1 -- 172.21.15.44:0/2535355254 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec00e040 con 0x55c694a4b1a0 2023-09-10T01:36:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.197+0000 7f08e97fa700 1 -- 172.21.15.44:0/2535355254 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08ec020070 con 0x55c694a4b1a0 2023-09-10T01:36:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.197+0000 7f08e97fa700 1 -- 172.21.15.44:0/2535355254 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec005320 con 0x55c694a4b1a0 2023-09-10T01:36:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.197+0000 7f0904c29b80 1 -- 172.21.15.44:0/2535355254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a4b1a0 msgr2=0x55c694a51840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.197+0000 7f0904c29b80 1 --2- 172.21.15.44:0/2535355254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a4b1a0 0x55c694a51840 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0060a0 tx=0x7f08ec0060d0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.197+0000 7f0904c29b80 1 -- 172.21.15.44:0/2535355254 shutdown_connections 2023-09-10T01:36:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.197+0000 7f0904c29b80 1 --2- 172.21.15.44:0/2535355254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a4b1a0 0x55c694a51840 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.197+0000 7f0904c29b80 1 --2- 172.21.15.44:0/2535355254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a45300 0x55c694a56530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.197+0000 7f0904c29b80 1 --2- 172.21.15.44:0/2535355254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45cc0 0x55c694a44a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.197+0000 7f0904c29b80 1 -- 172.21.15.44:0/2535355254 >> 172.21.15.44:0/2535355254 conn(0x55c6949ba030 msgr2=0x55c694a442f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.197+0000 7f0904c29b80 1 -- 172.21.15.44:0/2535355254 shutdown_connections 2023-09-10T01:36:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.197+0000 7f0904c29b80 1 -- 172.21.15.44:0/2535355254 wait complete. 2023-09-10T01:36:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f0904c29b80 1 -- start start 2023-09-10T01:36:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f0904c29b80 1 Processor -- start 2023-09-10T01:36:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f0904c29b80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a45300 0x55c694aa7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f0904c29b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45cc0 0x55c694aa78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f0904c29b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a4b1a0 0x55c694aac220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f0904c29b80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c694aac7b0 con 0x55c694a45300 2023-09-10T01:36:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f0904c29b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c694aac920 con 0x55c694a45cc0 2023-09-10T01:36:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f0904c29b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c694aacae0 con 0x55c694a4b1a0 2023-09-10T01:36:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f0904c29b80 10 client.0 _ll_register_callbacks cb 0x7f08e4099160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:36:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f08f259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a45300 0x55c694aa7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f08f259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a45300 0x55c694aa7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33218/0 (socket says 172.21.15.44:33218) 2023-09-10T01:36:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f08f259c700 1 -- 172.21.15.44:0/570976737 learned_addr learned my addr 172.21.15.44:0/570976737 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f08f2d9d700 1 --2- 172.21.15.44:0/570976737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a4b1a0 0x55c694aac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f08f1d9b700 1 --2- 172.21.15.44:0/570976737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45cc0 0x55c694aa78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f08f259c700 1 -- 172.21.15.44:0/570976737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a4b1a0 msgr2=0x55c694aac220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f08f259c700 1 --2- 172.21.15.44:0/570976737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a4b1a0 0x55c694aac220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f08f259c700 1 -- 172.21.15.44:0/570976737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45cc0 msgr2=0x55c694aa78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f08f259c700 1 --2- 172.21.15.44:0/570976737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45cc0 0x55c694aa78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f08f259c700 1 -- 172.21.15.44:0/570976737 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6947acc00 con 0x55c694a45300 2023-09-10T01:36:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.198+0000 7f08f1d9b700 1 --2- 172.21.15.44:0/570976737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45cc0 0x55c694aa78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:36:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f08f259c700 1 --2- 172.21.15.44:0/570976737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a45300 0x55c694aa7350 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00fc70 tx=0x7f08dc005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df543685a641ca4e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f08d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:36:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f08d9ffb700 1 -- 172.21.15.44:0/570976737 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc02d040 con 0x55c694a45300 2023-09-10T01:36:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f08d9ffb700 1 -- 172.21.15.44:0/570976737 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(211..211 src has 1..211) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f08dc036070 con 0x55c694a45300 2023-09-10T01:36:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f0904c29b80 1 -- 172.21.15.44:0/570976737 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08dc011da0 con 0x55c694a45300 2023-09-10T01:36:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f08d9ffb700 1 -- 172.21.15.44:0/570976737 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08dc036e70 con 0x55c694a45300 2023-09-10T01:36:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f0904c29b80 10 client.20775 fetch_fsmap learned FSMap version 201 2023-09-10T01:36:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f0904c29b80 1 -- 172.21.15.44:0/570976737 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=201,monmap=4+}) v3 -- 0x7f08f410dce0 con 0x55c694a45300 2023-09-10T01:36:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f08d9ffb700 1 -- 172.21.15.44:0/570976737 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08dc011da0 con 0x55c694a45300 2023-09-10T01:36:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f08d9ffb700 1 -- 172.21.15.44:0/570976737 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 201) v1 ==== 7375+0+0 (secure 0 0 0) 0x7f08dc01e950 con 0x55c694a45300 2023-09-10T01:36:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f0904c29b80 10 client.20775 fetch_fsmap finished waiting for FSMap version 201 2023-09-10T01:36:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f0904c29b80 10 client.20775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:36:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f0904c29b80 20 client.20775 populate_metadata read hostname 'smithi044' 2023-09-10T01:36:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f0904c29b80 1 --2- 172.21.15.44:0/570976737 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c694ab4320 0x55c694ab6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f0904c29b80 4 client.20775 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:36:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f0904c29b80 1 -- 172.21.15.44:0/570976737 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c6949908c0 con 0x55c694ab4320 2023-09-10T01:36:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.199+0000 7f08f1d9b700 1 --2- 172.21.15.44:0/570976737 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c694ab4320 0x55c694ab6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.200+0000 7f08d9ffb700 10 client.20775 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.200+0000 7f08f1d9b700 1 --2- 172.21.15.44:0/570976737 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c694ab4320 0x55c694ab6740 crc :-1 s=READY pgs=625 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e9f2c218db20cf server_cookie=f1c0ded393ca6dc0 in_seq=0 out_seq=0 2023-09-10T01:36:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.706+0000 7f08d9ffb700 1 -- 172.21.15.44:0/570976737 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c6949908c0 con 0x55c694ab4320 2023-09-10T01:36:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.706+0000 7f08d9ffb700 10 client.20775 handle_command_reply: tid=0 2023-09-10T01:36:44.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.706+0000 7f0904c29b80 1 client.20775 shutdown 2023-09-10T01:36:44.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.707+0000 7f0904c29b80 1 -- 172.21.15.44:0/570976737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a45300 msgr2=0x55c694aa7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:44.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.707+0000 7f0904c29b80 1 --2- 172.21.15.44:0/570976737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a45300 0x55c694aa7350 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00fc70 tx=0x7f08dc005720 comp rx=0 tx=0).stop 2023-09-10T01:36:44.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.707+0000 7f0904c29b80 1 -- 172.21.15.44:0/570976737 shutdown_connections 2023-09-10T01:36:44.709 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.707+0000 7f0904c29b80 1 --2- 172.21.15.44:0/570976737 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55c694ab4320 0x55c694ab6740 crc :-1 s=READY pgs=625 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.707+0000 7f0904c29b80 1 --2- 172.21.15.44:0/570976737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c694a4b1a0 0x55c694aac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.707+0000 7f0904c29b80 1 --2- 172.21.15.44:0/570976737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c694a45300 0x55c694aa7350 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.707+0000 7f0904c29b80 1 --2- 172.21.15.44:0/570976737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c694a45cc0 0x55c694aa78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.707+0000 7f08d9ffb700 0 client.20775 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.707+0000 7f0904c29b80 1 -- 172.21.15.44:0/570976737 >> 172.21.15.44:0/570976737 conn(0x55c6949ba030 msgr2=0x55c694a44b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.707+0000 7f0904c29b80 1 -- 172.21.15.44:0/570976737 shutdown_connections 2023-09-10T01:36:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.707+0000 7f0904c29b80 1 -- 172.21.15.44:0/570976737 wait complete. 2023-09-10T01:36:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.707+0000 7f0904c29b80 20 client.20775 trim_cache size 0 max 16384 2023-09-10T01:36:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:36:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:36:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:36:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:36:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.708+0000 7f08f8a63700 1 -- 172.21.15.44:0/2459750719 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f08e00a07b0 msgr2=0x7f08e00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.708+0000 7f08f8a63700 1 --2- 172.21.15.44:0/2459750719 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f08e00a07b0 0x7f08e00a2c70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f08dc005950 tx=0x7f08dc0058e0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.708+0000 7f08f8a63700 1 -- 172.21.15.44:0/2459750719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08f4068f30 msgr2=0x7f08f40ffe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.708+0000 7f08f8a63700 1 --2- 172.21.15.44:0/2459750719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08f4068f30 0x7f08f40ffe80 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f08e400d950 tx=0x7f08e400dc60 comp rx=0 tx=0).stop 2023-09-10T01:36:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.708+0000 7f08f8a63700 1 -- 172.21.15.44:0/2459750719 shutdown_connections 2023-09-10T01:36:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.708+0000 7f08f8a63700 1 --2- 172.21.15.44:0/2459750719 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f08e00a07b0 0x7f08e00a2c70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.708+0000 7f08f8a63700 1 --2- 172.21.15.44:0/2459750719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08f41066d0 0x7f08f41a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.708+0000 7f08f8a63700 1 --2- 172.21.15.44:0/2459750719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08f4068570 0x7f08f40ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.708+0000 7f08f8a63700 1 --2- 172.21.15.44:0/2459750719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08f4068f30 0x7f08f40ffe80 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.708+0000 7f08f8a63700 1 -- 172.21.15.44:0/2459750719 >> 172.21.15.44:0/2459750719 conn(0x7f08f40773d0 msgr2=0x7f08f40fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.708+0000 7f08f8a63700 1 -- 172.21.15.44:0/2459750719 shutdown_connections 2023-09-10T01:36:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:44.708+0000 7f08f8a63700 1 -- 172.21.15.44:0/2459750719 wait complete. 2023-09-10T01:36:45.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:44 smithi131 ceph-mon[103713]: pgmap v1828: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 274 KiB/s rd, 1.7 MiB/s wr, 184 op/s 2023-09-10T01:36:45.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:44 smithi044 ceph-mon[98676]: pgmap v1828: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 274 KiB/s rd, 1.7 MiB/s wr, 184 op/s 2023-09-10T01:36:45.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:44 smithi099 ceph-mon[102692]: pgmap v1828: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 274 KiB/s rd, 1.7 MiB/s wr, 184 op/s 2023-09-10T01:36:47.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:46 smithi131 ceph-mon[103713]: pgmap v1829: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 350 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2023-09-10T01:36:47.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:46 smithi044 ceph-mon[98676]: pgmap v1829: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 350 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2023-09-10T01:36:47.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:46 smithi099 ceph-mon[102692]: pgmap v1829: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 350 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2023-09-10T01:36:49.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:48 smithi044 ceph-mon[98676]: pgmap v1830: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 401 KiB/s rd, 413 KiB/s wr, 36 op/s 2023-09-10T01:36:49.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:48 smithi099 ceph-mon[102692]: pgmap v1830: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 401 KiB/s rd, 413 KiB/s wr, 36 op/s 2023-09-10T01:36:49.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:48 smithi131 ceph-mon[103713]: pgmap v1830: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 401 KiB/s rd, 413 KiB/s wr, 36 op/s 2023-09-10T01:36:51.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:50 smithi044 ceph-mon[98676]: pgmap v1831: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 326 KiB/s rd, 336 KiB/s wr, 34 op/s 2023-09-10T01:36:51.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:50 smithi099 ceph-mon[102692]: pgmap v1831: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 326 KiB/s rd, 336 KiB/s wr, 34 op/s 2023-09-10T01:36:51.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:50 smithi131 ceph-mon[103713]: pgmap v1831: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 326 KiB/s rd, 336 KiB/s wr, 34 op/s 2023-09-10T01:36:53.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:52 smithi099 ceph-mon[102692]: pgmap v1832: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 276 KiB/s rd, 283 KiB/s wr, 31 op/s 2023-09-10T01:36:53.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:52 smithi131 ceph-mon[103713]: pgmap v1832: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 276 KiB/s rd, 283 KiB/s wr, 31 op/s 2023-09-10T01:36:53.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:52 smithi044 ceph-mon[98676]: pgmap v1832: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 276 KiB/s rd, 283 KiB/s wr, 31 op/s 2023-09-10T01:36:54.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:53 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:54.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:53 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:54.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:53 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:36:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:54 smithi099 ceph-mon[102692]: pgmap v1833: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 307 KiB/s rd, 313 KiB/s wr, 34 op/s 2023-09-10T01:36:55.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:54 smithi131 ceph-mon[103713]: pgmap v1833: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 307 KiB/s rd, 313 KiB/s wr, 34 op/s 2023-09-10T01:36:55.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:54 smithi044 ceph-mon[98676]: pgmap v1833: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 307 KiB/s rd, 313 KiB/s wr, 34 op/s 2023-09-10T01:36:57.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:56 smithi099 ceph-mon[102692]: pgmap v1834: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 273 KiB/s rd, 278 KiB/s wr, 31 op/s 2023-09-10T01:36:57.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:56 smithi131 ceph-mon[103713]: pgmap v1834: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 273 KiB/s rd, 278 KiB/s wr, 31 op/s 2023-09-10T01:36:57.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:56 smithi044 ceph-mon[98676]: pgmap v1834: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 273 KiB/s rd, 278 KiB/s wr, 31 op/s 2023-09-10T01:36:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.834+0000 7f7af9d05700 1 -- 172.21.15.44:0/1793815482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7af40ffe70 msgr2=0x7f7af410c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.834+0000 7f7af9d05700 1 --2- 172.21.15.44:0/1793815482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7af40ffe70 0x7f7af410c310 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00b0a0 tx=0x7f7aec00b3b0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.834+0000 7f7af9d05700 1 -- 172.21.15.44:0/1793815482 shutdown_connections 2023-09-10T01:36:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.834+0000 7f7af9d05700 1 --2- 172.21.15.44:0/1793815482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7af40ffe70 0x7f7af410c310 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.834+0000 7f7af9d05700 1 --2- 172.21.15.44:0/1793815482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7af4104420 0x7f7af4104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.834+0000 7f7af9d05700 1 --2- 172.21.15.44:0/1793815482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af40ff4b0 0x7f7af40ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.834+0000 7f7af9d05700 1 -- 172.21.15.44:0/1793815482 >> 172.21.15.44:0/1793815482 conn(0x7f7af4076600 msgr2=0x7f7af4076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.834+0000 7f7af9d05700 1 -- 172.21.15.44:0/1793815482 shutdown_connections 2023-09-10T01:36:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.834+0000 7f7af9d05700 1 -- 172.21.15.44:0/1793815482 wait complete. 2023-09-10T01:36:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.835+0000 7f7af9d05700 1 Processor -- start 2023-09-10T01:36:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.835+0000 7f7af9d05700 1 -- start start 2023-09-10T01:36:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.835+0000 7f7af9d05700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af40ff4b0 0x7f7af419b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.835+0000 7f7af9d05700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7af40ffe70 0x7f7af419bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.835+0000 7f7af9d05700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7af4104420 0x7f7af4195450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.835+0000 7f7af9d05700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af4195ac0 con 0x7f7af4104420 2023-09-10T01:36:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.835+0000 7f7af9d05700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af4195c30 con 0x7f7af40ffe70 2023-09-10T01:36:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.835+0000 7f7af9d05700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af4195df0 con 0x7f7af40ff4b0 2023-09-10T01:36:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.835+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7af4104420 0x7f7af4195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.835+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7af4104420 0x7f7af4195450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33236/0 (socket says 172.21.15.44:33236) 2023-09-10T01:36:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.835+0000 7f7af3fff700 1 -- 172.21.15.44:0/3383822933 learned_addr learned my addr 172.21.15.44:0/3383822933 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.836+0000 7f7af37fe700 1 --2- 172.21.15.44:0/3383822933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af40ff4b0 0x7f7af419b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.836+0000 7f7af2ffd700 1 --2- 172.21.15.44:0/3383822933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7af40ffe70 0x7f7af419bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.836+0000 7f7af3fff700 1 -- 172.21.15.44:0/3383822933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af40ff4b0 msgr2=0x7f7af419b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.836+0000 7f7af3fff700 1 --2- 172.21.15.44:0/3383822933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af40ff4b0 0x7f7af419b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.836+0000 7f7af3fff700 1 -- 172.21.15.44:0/3383822933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7af40ffe70 msgr2=0x7f7af419bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.836+0000 7f7af3fff700 1 --2- 172.21.15.44:0/3383822933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7af40ffe70 0x7f7af419bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.836+0000 7f7af3fff700 1 -- 172.21.15.44:0/3383822933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec009d00 con 0x7f7af4104420 2023-09-10T01:36:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.836+0000 7f7af2ffd700 1 --2- 172.21.15.44:0/3383822933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7af40ffe70 0x7f7af419bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:36:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.836+0000 7f7af3fff700 1 --2- 172.21.15.44:0/3383822933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7af4104420 0x7f7af4195450 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00b830 tx=0x7f7aec0094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.836+0000 7f7af0ff9700 1 -- 172.21.15.44:0/3383822933 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec00e040 con 0x7f7af4104420 2023-09-10T01:36:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.836+0000 7f7af0ff9700 1 -- 172.21.15.44:0/3383822933 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7aec003e90 con 0x7f7af4104420 2023-09-10T01:36:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.836+0000 7f7af9d05700 1 -- 172.21.15.44:0/3383822933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af4196070 con 0x7f7af4104420 2023-09-10T01:36:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.836+0000 7f7af0ff9700 1 -- 172.21.15.44:0/3383822933 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec01a720 con 0x7f7af4104420 2023-09-10T01:36:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.836+0000 7f7af9d05700 1 -- 172.21.15.44:0/3383822933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af4068b00 con 0x7f7af4104420 2023-09-10T01:36:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.838+0000 7f7af0ff9700 1 -- 172.21.15.44:0/3383822933 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f7aec003a00 con 0x7f7af4104420 2023-09-10T01:36:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.839+0000 7f7af0ff9700 1 --2- 172.21.15.44:0/3383822933 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7adc0a07b0 0x7f7adc0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.839+0000 7f7af0ff9700 1 -- 172.21.15.44:0/3383822933 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f7aec0cf690 con 0x7f7af4104420 2023-09-10T01:36:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.839+0000 7f7af37fe700 1 --2- 172.21.15.44:0/3383822933 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7adc0a07b0 0x7f7adc0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.840+0000 7f7af37fe700 1 --2- 172.21.15.44:0/3383822933 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7adc0a07b0 0x7f7adc0a2c70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4005950 tx=0x7f7ae400a300 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.840+0000 7f7b05ecbb80 1 Processor -- start 2023-09-10T01:36:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.840+0000 7f7b05ecbb80 1 -- start start 2023-09-10T01:36:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.840+0000 7f7b05ecbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b17221a0 0x55e3b1722580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.840+0000 7f7b05ecbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b171c300 0x55e3b171c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.840+0000 7f7b05ecbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b171ccc0 0x55e3b1728f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.840+0000 7f7b05ecbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b16756d0 con 0x55e3b17221a0 2023-09-10T01:36:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.840+0000 7f7b05ecbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b1672ff0 con 0x55e3b171ccc0 2023-09-10T01:36:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.840+0000 7f7b05ecbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b166e8e0 con 0x55e3b171c300 2023-09-10T01:36:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.840+0000 7f7af37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b17221a0 0x55e3b1722580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.840+0000 7f7af37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b17221a0 0x55e3b1722580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33240/0 (socket says 172.21.15.44:33240) 2023-09-10T01:36:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.840+0000 7f7af37fe700 1 -- 172.21.15.44:0/2456956755 learned_addr learned my addr 172.21.15.44:0/2456956755 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.840+0000 7f7af2ffd700 1 --2- 172.21.15.44:0/2456956755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b171c300 0x55e3b171c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.840+0000 7f7af3fff700 1 --2- 172.21.15.44:0/2456956755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b171ccc0 0x55e3b1728f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.841+0000 7f7af37fe700 1 -- 172.21.15.44:0/2456956755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b171c300 msgr2=0x55e3b171c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.841+0000 7f7af37fe700 1 --2- 172.21.15.44:0/2456956755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b171c300 0x55e3b171c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.841+0000 7f7af37fe700 1 -- 172.21.15.44:0/2456956755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b171ccc0 msgr2=0x55e3b1728f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.841+0000 7f7af37fe700 1 --2- 172.21.15.44:0/2456956755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b171ccc0 0x55e3b1728f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.841+0000 7f7af37fe700 1 -- 172.21.15.44:0/2456956755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3b149ad80 con 0x55e3b17221a0 2023-09-10T01:36:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.841+0000 7f7af3fff700 1 --2- 172.21.15.44:0/2456956755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b171ccc0 0x55e3b1728f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:36:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.841+0000 7f7af37fe700 1 --2- 172.21.15.44:0/2456956755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b17221a0 0x55e3b1722580 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400fd10 tx=0x7f7ae4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0053ca98fa8e09 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.841+0000 7f7ad9ffb700 1 -- 172.21.15.44:0/2456956755 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae40107e0 con 0x55e3b17221a0 2023-09-10T01:36:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.841+0000 7f7ad9ffb700 1 -- 172.21.15.44:0/2456956755 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ae4020070 con 0x55e3b17221a0 2023-09-10T01:36:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.841+0000 7f7ad9ffb700 1 -- 172.21.15.44:0/2456956755 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4025c60 con 0x55e3b17221a0 2023-09-10T01:36:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.842+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/2456956755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b17221a0 msgr2=0x55e3b1722580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.842+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/2456956755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b17221a0 0x55e3b1722580 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400fd10 tx=0x7f7ae4012000 comp rx=0 tx=0).stop 2023-09-10T01:36:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.842+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/2456956755 shutdown_connections 2023-09-10T01:36:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.842+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/2456956755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b171ccc0 0x55e3b1728f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.842+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/2456956755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b17221a0 0x55e3b1722580 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.842+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/2456956755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b171c300 0x55e3b171c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.842+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/2456956755 >> 172.21.15.44:0/2456956755 conn(0x55e3b1691030 msgr2=0x55e3b1717560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.842+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/2456956755 shutdown_connections 2023-09-10T01:36:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.842+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/2456956755 wait complete. 2023-09-10T01:36:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.842+0000 7f7b05ecbb80 1 -- start start 2023-09-10T01:36:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.842+0000 7f7b05ecbb80 1 Processor -- start 2023-09-10T01:36:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.842+0000 7f7b05ecbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b171c300 0x55e3b1728df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.842+0000 7f7b05ecbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b171ccc0 0x55e3b1726070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.842+0000 7f7b05ecbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b17221a0 0x55e3b17266a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7b05ecbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b160cb10 con 0x55e3b171ccc0 2023-09-10T01:36:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7b05ecbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b160c810 con 0x55e3b171c300 2023-09-10T01:36:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7b05ecbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b14f08c0 con 0x55e3b17221a0 2023-09-10T01:36:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7b05ecbb80 10 client.0 _ll_register_callbacks cb 0x55e3b1556a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:36:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7af2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b171ccc0 0x55e3b1726070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7af2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b171ccc0 0x55e3b1726070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33248/0 (socket says 172.21.15.44:33248) 2023-09-10T01:36:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7af2ffd700 1 -- 172.21.15.44:0/1938241025 learned_addr learned my addr 172.21.15.44:0/1938241025 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7af3fff700 1 --2- 172.21.15.44:0/1938241025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b17221a0 0x55e3b17266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7af37fe700 1 --2- 172.21.15.44:0/1938241025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b171c300 0x55e3b1728df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7af2ffd700 1 -- 172.21.15.44:0/1938241025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b17221a0 msgr2=0x55e3b17266a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7af2ffd700 1 --2- 172.21.15.44:0/1938241025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b17221a0 0x55e3b17266a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7af2ffd700 1 -- 172.21.15.44:0/1938241025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b171c300 msgr2=0x55e3b1728df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7af2ffd700 1 --2- 172.21.15.44:0/1938241025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b171c300 0x55e3b1728df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7af2ffd700 1 -- 172.21.15.44:0/1938241025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae400f9f0 con 0x55e3b171ccc0 2023-09-10T01:36:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7af2ffd700 1 --2- 172.21.15.44:0/1938241025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b171ccc0 0x55e3b1726070 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800bad0 tx=0x7f7ae800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b008499ca0131e7e server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7ad2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:36:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7ad2ffd700 1 -- 172.21.15.44:0/1938241025 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae801b640 con 0x55e3b171ccc0 2023-09-10T01:36:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7ad2ffd700 1 -- 172.21.15.44:0/1938241025 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(211..211 src has 1..211) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f7ae8003e40 con 0x55e3b171ccc0 2023-09-10T01:36:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.843+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/1938241025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3b15c6490 con 0x55e3b171ccc0 2023-09-10T01:36:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.844+0000 7f7ad2ffd700 1 -- 172.21.15.44:0/1938241025 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ae801b7a0 con 0x55e3b171ccc0 2023-09-10T01:36:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.844+0000 7f7b05ecbb80 10 client.20787 fetch_fsmap learned FSMap version 202 2023-09-10T01:36:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.844+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/1938241025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=202,monmap=4+}) v3 -- 0x55e3b1483c00 con 0x55e3b171ccc0 2023-09-10T01:36:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.844+0000 7f7ad2ffd700 1 -- 172.21.15.44:0/1938241025 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e3b15c6490 con 0x55e3b171ccc0 2023-09-10T01:36:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.844+0000 7f7ad2ffd700 1 -- 172.21.15.44:0/1938241025 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 202) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f7ae8011970 con 0x55e3b171ccc0 2023-09-10T01:36:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.844+0000 7f7b05ecbb80 10 client.20787 fetch_fsmap finished waiting for FSMap version 202 2023-09-10T01:36:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.844+0000 7f7b05ecbb80 10 client.20787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:36:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.844+0000 7f7b05ecbb80 20 client.20787 populate_metadata read hostname 'smithi044' 2023-09-10T01:36:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.844+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/1938241025 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e3b1736530 0x55e3b1738950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.844+0000 7f7b05ecbb80 4 client.20787 mds_command: new command op to 24605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-10T01:36:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.844+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/1938241025 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3b1683840 con 0x55e3b1736530 2023-09-10T01:36:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.844+0000 7f7af37fe700 1 --2- 172.21.15.44:0/1938241025 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e3b1736530 0x55e3b1738950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.845+0000 7f7af37fe700 1 --2- 172.21.15.44:0/1938241025 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e3b1736530 0x55e3b1738950 crc :-1 s=READY pgs=626 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=accc144c6163c20d server_cookie=26014147a0c51f73 in_seq=0 out_seq=0 2023-09-10T01:36:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.845+0000 7f7ad2ffd700 10 client.20787 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.845+0000 7f7ad2ffd700 1 -- 172.21.15.44:0/1938241025 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e3b1683840 con 0x55e3b1736530 2023-09-10T01:36:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.845+0000 7f7ad2ffd700 10 client.20787 handle_command_reply: tid=0 2023-09-10T01:36:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.845+0000 7f7b05ecbb80 1 client.20787 shutdown 2023-09-10T01:36:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.846+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/1938241025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b171ccc0 msgr2=0x55e3b1726070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.846+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/1938241025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b171ccc0 0x55e3b1726070 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800bad0 tx=0x7f7ae800be90 comp rx=0 tx=0).stop 2023-09-10T01:36:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.846+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/1938241025 shutdown_connections 2023-09-10T01:36:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.846+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/1938241025 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e3b1736530 0x55e3b1738950 crc :-1 s=READY pgs=626 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.846+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/1938241025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b17221a0 0x55e3b17266a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.846+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/1938241025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b171c300 0x55e3b1728df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.846+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/1938241025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b171ccc0 0x55e3b1726070 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.846+0000 7f7ad2ffd700 0 client.20787 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.846+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/1938241025 >> 172.21.15.44:0/1938241025 conn(0x55e3b1691030 msgr2=0x55e3b17167a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.846+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/1938241025 shutdown_connections 2023-09-10T01:36:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.846+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/1938241025 wait complete. 2023-09-10T01:36:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.846+0000 7f7b05ecbb80 20 client.20787 trim_cache size 0 max 16384 2023-09-10T01:36:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.860+0000 7f7b05ecbb80 1 Processor -- start 2023-09-10T01:36:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7b05ecbb80 1 -- start start 2023-09-10T01:36:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7b05ecbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b171c300 0x55e3b1716f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7b05ecbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b171ccc0 0x55e3b1717460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7b05ecbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b17221a0 0x55e3b1725870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7b05ecbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b15c6490 con 0x55e3b17221a0 2023-09-10T01:36:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7b05ecbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b171fdb0 con 0x55e3b171ccc0 2023-09-10T01:36:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7b05ecbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b1721e00 con 0x55e3b171c300 2023-09-10T01:36:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b17221a0 0x55e3b1725870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b17221a0 0x55e3b1725870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33258/0 (socket says 172.21.15.44:33258) 2023-09-10T01:36:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7af3fff700 1 -- 172.21.15.44:0/3978592759 learned_addr learned my addr 172.21.15.44:0/3978592759 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7af2ffd700 1 --2- 172.21.15.44:0/3978592759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b171ccc0 0x55e3b1717460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7af37fe700 1 --2- 172.21.15.44:0/3978592759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b171c300 0x55e3b1716f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7af3fff700 1 -- 172.21.15.44:0/3978592759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b171c300 msgr2=0x55e3b1716f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7af3fff700 1 --2- 172.21.15.44:0/3978592759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b171c300 0x55e3b1716f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7af3fff700 1 -- 172.21.15.44:0/3978592759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b171ccc0 msgr2=0x55e3b1717460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7af3fff700 1 --2- 172.21.15.44:0/3978592759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b171ccc0 0x55e3b1717460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7af3fff700 1 -- 172.21.15.44:0/3978592759 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3b14c11f0 con 0x55e3b17221a0 2023-09-10T01:36:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7af3fff700 1 --2- 172.21.15.44:0/3978592759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b17221a0 0x55e3b1725870 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f7aec09c4d0 tx=0x7f7aec009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce67d28ecade0cd server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7ada7fc700 1 -- 172.21.15.44:0/3978592759 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec0ce040 con 0x55e3b17221a0 2023-09-10T01:36:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.861+0000 7f7ada7fc700 1 -- 172.21.15.44:0/3978592759 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7aec003cb0 con 0x55e3b17221a0 2023-09-10T01:36:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.862+0000 7f7ada7fc700 1 -- 172.21.15.44:0/3978592759 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec004270 con 0x55e3b17221a0 2023-09-10T01:36:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.862+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/3978592759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b17221a0 msgr2=0x55e3b1725870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.862+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/3978592759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b17221a0 0x55e3b1725870 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f7aec09c4d0 tx=0x7f7aec009040 comp rx=0 tx=0).stop 2023-09-10T01:36:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.862+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/3978592759 shutdown_connections 2023-09-10T01:36:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.862+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/3978592759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b17221a0 0x55e3b1725870 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.862+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/3978592759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b171c300 0x55e3b1716f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.862+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/3978592759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b171ccc0 0x55e3b1717460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.862+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/3978592759 >> 172.21.15.44:0/3978592759 conn(0x55e3b1691030 msgr2=0x55e3b17211d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.862+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/3978592759 shutdown_connections 2023-09-10T01:36:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.862+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/3978592759 wait complete. 2023-09-10T01:36:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.862+0000 7f7b05ecbb80 1 -- start start 2023-09-10T01:36:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7b05ecbb80 1 Processor -- start 2023-09-10T01:36:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7b05ecbb80 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b171c300 0x55e3b172b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7b05ecbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b171ccc0 0x55e3b172bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7b05ecbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b17221a0 0x55e3b1783350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7b05ecbb80 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b1783890 con 0x55e3b171c300 2023-09-10T01:36:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7b05ecbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b1783a00 con 0x55e3b17221a0 2023-09-10T01:36:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7b05ecbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b1783bc0 con 0x55e3b171ccc0 2023-09-10T01:36:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7b05ecbb80 10 client.0 _ll_register_callbacks cb 0x7f7ae8004380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-10T01:36:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7af37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b171c300 0x55e3b172b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7af37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b171c300 0x55e3b172b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33260/0 (socket says 172.21.15.44:33260) 2023-09-10T01:36:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7af37fe700 1 -- 172.21.15.44:0/3129605304 learned_addr learned my addr 172.21.15.44:0/3129605304 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:36:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7af2ffd700 1 --2- 172.21.15.44:0/3129605304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b171ccc0 0x55e3b172bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7af3fff700 1 --2- 172.21.15.44:0/3129605304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b17221a0 0x55e3b1783350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7af37fe700 1 -- 172.21.15.44:0/3129605304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b171ccc0 msgr2=0x55e3b172bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7af37fe700 1 --2- 172.21.15.44:0/3129605304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b171ccc0 0x55e3b172bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7af37fe700 1 -- 172.21.15.44:0/3129605304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b17221a0 msgr2=0x55e3b1783350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-10T01:36:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7af37fe700 1 --2- 172.21.15.44:0/3129605304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b17221a0 0x55e3b1783350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7af37fe700 1 -- 172.21.15.44:0/3129605304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af4068b00 con 0x55e3b171c300 2023-09-10T01:36:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7af2ffd700 1 --2- 172.21.15.44:0/3129605304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b171ccc0 0x55e3b172bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:36:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7af3fff700 1 --2- 172.21.15.44:0/3129605304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b17221a0 0x55e3b1783350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:36:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7af37fe700 1 --2- 172.21.15.44:0/3129605304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b171c300 0x55e3b172b700 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4010430 tx=0x7f7ae4026c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3675513ec67d745f server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:36:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.863+0000 7f7ad2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2023-09-10T01:36:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.864+0000 7f7ad2ffd700 1 -- 172.21.15.44:0/3129605304 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae402b050 con 0x55e3b171c300 2023-09-10T01:36:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.864+0000 7f7ad2ffd700 1 -- 172.21.15.44:0/3129605304 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(211..211 src has 1..211) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f7ae4005450 con 0x55e3b171c300 2023-09-10T01:36:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.864+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/3129605304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ae4020070 con 0x55e3b171c300 2023-09-10T01:36:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.864+0000 7f7ad2ffd700 1 -- 172.21.15.44:0/3129605304 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ae4005610 con 0x55e3b171c300 2023-09-10T01:36:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.864+0000 7f7b05ecbb80 10 client.20793 fetch_fsmap learned FSMap version 202 2023-09-10T01:36:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.864+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/3129605304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=202,monmap=4+}) v3 -- 0x55e3b1499250 con 0x55e3b171c300 2023-09-10T01:36:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.864+0000 7f7ad2ffd700 1 -- 172.21.15.44:0/3129605304 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ae4020070 con 0x55e3b171c300 2023-09-10T01:36:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.864+0000 7f7ad2ffd700 1 -- 172.21.15.44:0/3129605304 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 202) v1 ==== 7371+0+0 (secure 0 0 0) 0x7f7ae4032370 con 0x55e3b171c300 2023-09-10T01:36:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.864+0000 7f7b05ecbb80 10 client.20793 fetch_fsmap finished waiting for FSMap version 202 2023-09-10T01:36:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.864+0000 7f7b05ecbb80 10 client.20793 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-10T01:36:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.864+0000 7f7b05ecbb80 20 client.20793 populate_metadata read hostname 'smithi044' 2023-09-10T01:36:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.864+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/3129605304 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e3b178b400 0x55e3b178d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:36:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.864+0000 7f7b05ecbb80 4 client.20793 mds_command: new command op to 24605 tid=0[{"prefix": "flush journal"}] 2023-09-10T01:36:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.864+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/3129605304 --> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e3b16678c0 con 0x55e3b178b400 2023-09-10T01:36:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.865+0000 7f7af2ffd700 1 --2- 172.21.15.44:0/3129605304 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e3b178b400 0x55e3b178d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:36:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.865+0000 7f7ad2ffd700 10 client.20793 ms_handle_connect on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.865+0000 7f7af2ffd700 1 --2- 172.21.15.44:0/3129605304 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e3b178b400 0x55e3b178d820 crc :-1 s=READY pgs=627 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c37a6fbda8959e47 server_cookie=c67806824a09c3a5 in_seq=0 out_seq=0 2023-09-10T01:36:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.898+0000 7f7ad2ffd700 1 -- 172.21.15.44:0/3129605304 <== mds.0 v2:172.21.15.99:6838/2622966260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e3b16678c0 con 0x55e3b178b400 2023-09-10T01:36:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.898+0000 7f7ad2ffd700 10 client.20793 handle_command_reply: tid=0 2023-09-10T01:36:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.898+0000 7f7b05ecbb80 1 client.20793 shutdown 2023-09-10T01:36:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.898+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/3129605304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b171c300 msgr2=0x55e3b172b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.898+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/3129605304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b171c300 0x55e3b172b700 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4010430 tx=0x7f7ae4026c80 comp rx=0 tx=0).stop 2023-09-10T01:36:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.898+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/3129605304 shutdown_connections 2023-09-10T01:36:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.898+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/3129605304 >> [v2:172.21.15.99:6838/2622966260,v1:172.21.15.99:6839/2622966260] conn(0x55e3b178b400 0x55e3b178d820 crc :-1 s=READY pgs=627 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.898+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/3129605304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b17221a0 0x55e3b1783350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.898+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/3129605304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b171c300 0x55e3b172b700 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.898+0000 7f7ad2ffd700 0 client.20793 ms_handle_reset on v2:172.21.15.99:6838/2622966260 2023-09-10T01:36:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.898+0000 7f7b05ecbb80 1 --2- 172.21.15.44:0/3129605304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e3b171ccc0 0x55e3b172bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.898+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/3129605304 >> 172.21.15.44:0/3129605304 conn(0x55e3b1691030 msgr2=0x55e3b1727980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.898+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/3129605304 shutdown_connections 2023-09-10T01:36:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.899+0000 7f7b05ecbb80 1 -- 172.21.15.44:0/3129605304 wait complete. 2023-09-10T01:36:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.899+0000 7f7b05ecbb80 20 client.20793 trim_cache size 0 max 16384 2023-09-10T01:36:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2023-09-10T01:36:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2023-09-10T01:36:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2023-09-10T01:36:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2023-09-10T01:36:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.899+0000 7f7af9d05700 1 -- 172.21.15.44:0/3383822933 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7adc0a07b0 msgr2=0x7f7adc0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.899+0000 7f7af9d05700 1 --2- 172.21.15.44:0/3383822933 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7adc0a07b0 0x7f7adc0a2c70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4005950 tx=0x7f7ae400a300 comp rx=0 tx=0).stop 2023-09-10T01:36:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.899+0000 7f7af9d05700 1 -- 172.21.15.44:0/3383822933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7af4104420 msgr2=0x7f7af4195450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:36:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.899+0000 7f7af9d05700 1 --2- 172.21.15.44:0/3383822933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7af4104420 0x7f7af4195450 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00b830 tx=0x7f7aec0094e0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.899+0000 7f7af9d05700 1 -- 172.21.15.44:0/3383822933 shutdown_connections 2023-09-10T01:36:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.899+0000 7f7af9d05700 1 --2- 172.21.15.44:0/3383822933 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f7adc0a07b0 0x7f7adc0a2c70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.899+0000 7f7af9d05700 1 --2- 172.21.15.44:0/3383822933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7af4104420 0x7f7af4195450 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.899+0000 7f7af9d05700 1 --2- 172.21.15.44:0/3383822933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af40ff4b0 0x7f7af419b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.899+0000 7f7af9d05700 1 --2- 172.21.15.44:0/3383822933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7af40ffe70 0x7f7af419bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:36:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.899+0000 7f7af9d05700 1 -- 172.21.15.44:0/3383822933 >> 172.21.15.44:0/3383822933 conn(0x7f7af4076600 msgr2=0x7f7af40fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:36:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.899+0000 7f7af9d05700 1 -- 172.21.15.44:0/3383822933 shutdown_connections 2023-09-10T01:36:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2023-09-10T01:36:57.899+0000 7f7af9d05700 1 -- 172.21.15.44:0/3383822933 wait complete. 2023-09-10T01:36:59.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:58 smithi099 ceph-mon[102692]: pgmap v1835: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 270 KiB/s rd, 276 KiB/s wr, 39 op/s 2023-09-10T01:36:59.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:58 smithi131 ceph-mon[103713]: pgmap v1835: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 270 KiB/s rd, 276 KiB/s wr, 39 op/s 2023-09-10T01:36:59.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:58 smithi044 ceph-mon[98676]: pgmap v1835: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 270 KiB/s rd, 276 KiB/s wr, 39 op/s 2023-09-10T01:36:59.998 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-09-10T01:37:00.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:36:59 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:00.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:36:59 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:00.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:36:59 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:00.431 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/: 2023-09-10T01:37:00.431 INFO:teuthology.orchestra.run.smithi044.stdout:total 2 2023-09-10T01:37:00.431 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 8977398149 Sep 10 00:40 . 2023-09-10T01:37:00.431 INFO:teuthology.orchestra.run.smithi044.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 8977398149 Sep 10 00:40 .. 2023-09-10T01:37:00.432 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 data 2023-09-10T01:37:00.432 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:00.432 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data: 2023-09-10T01:37:00.432 INFO:teuthology.orchestra.run.smithi044.stdout:total 41 2023-09-10T01:37:00.432 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 . 2023-09-10T01:37:00.432 INFO:teuthology.orchestra.run.smithi044.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 8977398149 Sep 10 00:40 .. 2023-09-10T01:37:00.433 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7902896269 Sep 10 01:18 base 2023-09-10T01:37:00.433 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Sep 10 00:40 current_logfiles 2023-09-10T01:37:00.433 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 587404 Sep 10 00:41 global 2023-09-10T01:37:00.433 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 627 Sep 10 00:40 log 2023-09-10T01:37:00.433 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 pg_commit_ts 2023-09-10T01:37:00.434 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 pg_dynshmem 2023-09-10T01:37:00.434 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Sep 10 00:40 pg_hba.conf 2023-09-10T01:37:00.434 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 10 00:40 pg_ident.conf 2023-09-10T01:37:00.434 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 10 01:36 pg_logical 2023-09-10T01:37:00.434 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 pg_multixact 2023-09-10T01:37:00.435 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 pg_notify 2023-09-10T01:37:00.435 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 pg_replslot 2023-09-10T01:37:00.435 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 pg_serial 2023-09-10T01:37:00.435 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 pg_snapshots 2023-09-10T01:37:00.435 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 11771 Sep 10 01:36 pg_stat 2023-09-10T01:37:00.435 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 11771 Sep 10 01:36 pg_stat_tmp 2023-09-10T01:37:00.437 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 01:36 pg_subtrans 2023-09-10T01:37:00.437 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 pg_tblspc 2023-09-10T01:37:00.437 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 pg_twophase 2023-09-10T01:37:00.437 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 00:40 PG_VERSION 2023-09-10T01:37:00.438 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 10 01:36 pg_wal 2023-09-10T01:37:00.439 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 00:40 pg_xact 2023-09-10T01:37:00.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 10 00:40 postgresql.auto.conf 2023-09-10T01:37:00.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Sep 10 00:40 postgresql.conf 2023-09-10T01:37:00.439 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Sep 10 00:40 postmaster.opts 2023-09-10T01:37:00.439 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:00.440 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/base: 2023-09-10T01:37:00.440 INFO:teuthology.orchestra.run.smithi044.stdout:total 3 2023-09-10T01:37:00.440 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7902896269 Sep 10 01:18 . 2023-09-10T01:37:00.440 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:00.440 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7365123 Sep 10 00:40 1 2023-09-10T01:37:00.441 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7365123 Sep 10 00:40 12346 2023-09-10T01:37:00.441 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7888166023 Sep 10 01:22 12347 2023-09-10T01:37:00.441 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 01:21 pgsql_tmp 2023-09-10T01:37:00.441 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:00.441 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/base/1: 2023-09-10T01:37:00.442 INFO:teuthology.orchestra.run.smithi044.stdout:total 7194 2023-09-10T01:37:00.442 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7365123 Sep 10 00:40 . 2023-09-10T01:37:00.442 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7902896269 Sep 10 01:18 .. 2023-09-10T01:37:00.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 112 2023-09-10T01:37:00.442 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 113 2023-09-10T01:37:00.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 00:40 12182 2023-09-10T01:37:00.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12182_fsm 2023-09-10T01:37:00.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12182_vm 2023-09-10T01:37:00.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12184 2023-09-10T01:37:00.443 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12186 2023-09-10T01:37:00.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12187 2023-09-10T01:37:00.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12187_fsm 2023-09-10T01:37:00.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12187_vm 2023-09-10T01:37:00.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12189 2023-09-10T01:37:00.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12191 2023-09-10T01:37:00.444 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12192 2023-09-10T01:37:00.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12192_fsm 2023-09-10T01:37:00.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12192_vm 2023-09-10T01:37:00.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12194 2023-09-10T01:37:00.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12196 2023-09-10T01:37:00.445 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12197 2023-09-10T01:37:00.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12197_fsm 2023-09-10T01:37:00.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12197_vm 2023-09-10T01:37:00.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12199 2023-09-10T01:37:00.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12201 2023-09-10T01:37:00.446 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12202 2023-09-10T01:37:00.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12202_fsm 2023-09-10T01:37:00.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12202_vm 2023-09-10T01:37:00.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12204 2023-09-10T01:37:00.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12206 2023-09-10T01:37:00.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12207 2023-09-10T01:37:00.447 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12207_fsm 2023-09-10T01:37:00.448 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12207_vm 2023-09-10T01:37:00.448 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12209 2023-09-10T01:37:00.448 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12211 2023-09-10T01:37:00.448 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12212 2023-09-10T01:37:00.448 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12214 2023-09-10T01:37:00.449 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12216 2023-09-10T01:37:00.449 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 10 00:40 1247 2023-09-10T01:37:00.449 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1247_fsm 2023-09-10T01:37:00.449 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1247_vm 2023-09-10T01:37:00.449 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 10 00:40 1249 2023-09-10T01:37:00.449 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1249_fsm 2023-09-10T01:37:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1249_vm 2023-09-10T01:37:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 10 00:40 1255 2023-09-10T01:37:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1255_fsm 2023-09-10T01:37:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1255_vm 2023-09-10T01:37:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 00:40 1259 2023-09-10T01:37:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1259_fsm 2023-09-10T01:37:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1259_vm 2023-09-10T01:37:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1417 2023-09-10T01:37:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1417_vm 2023-09-10T01:37:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1418 2023-09-10T01:37:00.452 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1418_vm 2023-09-10T01:37:00.452 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 174 2023-09-10T01:37:00.452 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 175 2023-09-10T01:37:00.452 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2187 2023-09-10T01:37:00.452 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2224 2023-09-10T01:37:00.453 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2224_vm 2023-09-10T01:37:00.453 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2328 2023-09-10T01:37:00.453 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2328_vm 2023-09-10T01:37:00.453 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2336 2023-09-10T01:37:00.453 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2336_vm 2023-09-10T01:37:00.453 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2337 2023-09-10T01:37:00.454 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2600 2023-09-10T01:37:00.454 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2600_fsm 2023-09-10T01:37:00.454 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2600_vm 2023-09-10T01:37:00.454 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2601 2023-09-10T01:37:00.454 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2601_fsm 2023-09-10T01:37:00.455 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2601_vm 2023-09-10T01:37:00.455 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2602 2023-09-10T01:37:00.455 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2602_fsm 2023-09-10T01:37:00.455 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2602_vm 2023-09-10T01:37:00.456 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2603 2023-09-10T01:37:00.456 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2603_fsm 2023-09-10T01:37:00.456 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2603_vm 2023-09-10T01:37:00.456 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2604 2023-09-10T01:37:00.456 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2604_vm 2023-09-10T01:37:00.457 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2605 2023-09-10T01:37:00.457 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2605_fsm 2023-09-10T01:37:00.457 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2605_vm 2023-09-10T01:37:00.457 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2606 2023-09-10T01:37:00.457 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2606_fsm 2023-09-10T01:37:00.457 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2606_vm 2023-09-10T01:37:00.458 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2607 2023-09-10T01:37:00.458 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2607_fsm 2023-09-10T01:37:00.458 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2607_vm 2023-09-10T01:37:00.458 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 10 00:40 2608 2023-09-10T01:37:00.458 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2608_fsm 2023-09-10T01:37:00.459 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2608_vm 2023-09-10T01:37:00.459 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 10 00:40 2609 2023-09-10T01:37:00.459 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2609_fsm 2023-09-10T01:37:00.459 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2609_vm 2023-09-10T01:37:00.459 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2610 2023-09-10T01:37:00.460 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2610_fsm 2023-09-10T01:37:00.460 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2610_vm 2023-09-10T01:37:00.460 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2611 2023-09-10T01:37:00.460 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2611_vm 2023-09-10T01:37:00.460 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2612 2023-09-10T01:37:00.461 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2612_fsm 2023-09-10T01:37:00.461 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2612_vm 2023-09-10T01:37:00.461 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2613 2023-09-10T01:37:00.461 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2613_vm 2023-09-10T01:37:00.461 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2615 2023-09-10T01:37:00.461 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2615_fsm 2023-09-10T01:37:00.462 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2615_vm 2023-09-10T01:37:00.462 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2616 2023-09-10T01:37:00.462 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2616_fsm 2023-09-10T01:37:00.462 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2616_vm 2023-09-10T01:37:00.462 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 00:40 2617 2023-09-10T01:37:00.463 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2617_fsm 2023-09-10T01:37:00.463 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2617_vm 2023-09-10T01:37:00.463 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 00:40 2618 2023-09-10T01:37:00.463 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2618_fsm 2023-09-10T01:37:00.463 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2618_vm 2023-09-10T01:37:00.464 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 10 00:40 2619 2023-09-10T01:37:00.464 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2619_fsm 2023-09-10T01:37:00.464 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2619_vm 2023-09-10T01:37:00.464 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2620 2023-09-10T01:37:00.464 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2620_vm 2023-09-10T01:37:00.464 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2650 2023-09-10T01:37:00.465 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2651 2023-09-10T01:37:00.465 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2652 2023-09-10T01:37:00.465 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2653 2023-09-10T01:37:00.465 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2654 2023-09-10T01:37:00.465 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2655 2023-09-10T01:37:00.465 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2656 2023-09-10T01:37:00.466 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2657 2023-09-10T01:37:00.466 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 00:40 2658 2023-09-10T01:37:00.466 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 00:40 2659 2023-09-10T01:37:00.466 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2660 2023-09-10T01:37:00.466 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2661 2023-09-10T01:37:00.466 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2662 2023-09-10T01:37:00.467 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2663 2023-09-10T01:37:00.467 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2664 2023-09-10T01:37:00.467 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2665 2023-09-10T01:37:00.467 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2666 2023-09-10T01:37:00.467 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2667 2023-09-10T01:37:00.468 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2668 2023-09-10T01:37:00.468 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2669 2023-09-10T01:37:00.468 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2670 2023-09-10T01:37:00.468 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Sep 10 00:40 2673 2023-09-10T01:37:00.468 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 10 00:40 2674 2023-09-10T01:37:00.468 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 10 00:40 2675 2023-09-10T01:37:00.469 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2678 2023-09-10T01:37:00.469 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2679 2023-09-10T01:37:00.469 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2680 2023-09-10T01:37:00.469 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2681 2023-09-10T01:37:00.469 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2682 2023-09-10T01:37:00.470 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2683 2023-09-10T01:37:00.470 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2684 2023-09-10T01:37:00.470 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2685 2023-09-10T01:37:00.470 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2686 2023-09-10T01:37:00.470 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2687 2023-09-10T01:37:00.471 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2688 2023-09-10T01:37:00.471 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2689 2023-09-10T01:37:00.471 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 00:40 2690 2023-09-10T01:37:00.471 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 10 00:40 2691 2023-09-10T01:37:00.471 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2692 2023-09-10T01:37:00.471 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2693 2023-09-10T01:37:00.472 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2696 2023-09-10T01:37:00.472 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2699 2023-09-10T01:37:00.472 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2701 2023-09-10T01:37:00.472 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2702 2023-09-10T01:37:00.472 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2703 2023-09-10T01:37:00.472 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2704 2023-09-10T01:37:00.473 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2753 2023-09-10T01:37:00.473 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2753_fsm 2023-09-10T01:37:00.473 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2753_vm 2023-09-10T01:37:00.473 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2754 2023-09-10T01:37:00.473 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2755 2023-09-10T01:37:00.473 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2756 2023-09-10T01:37:00.474 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2757 2023-09-10T01:37:00.474 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2830 2023-09-10T01:37:00.474 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2830_vm 2023-09-10T01:37:00.474 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2831 2023-09-10T01:37:00.474 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2832 2023-09-10T01:37:00.474 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2832_vm 2023-09-10T01:37:00.475 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2833 2023-09-10T01:37:00.475 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2834 2023-09-10T01:37:00.475 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2834_vm 2023-09-10T01:37:00.475 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2835 2023-09-10T01:37:00.475 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2836 2023-09-10T01:37:00.475 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2836_vm 2023-09-10T01:37:00.476 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2837 2023-09-10T01:37:00.476 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 10 00:40 2838 2023-09-10T01:37:00.476 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2838_fsm 2023-09-10T01:37:00.476 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2838_vm 2023-09-10T01:37:00.477 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2839 2023-09-10T01:37:00.477 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2840 2023-09-10T01:37:00.477 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2840_fsm 2023-09-10T01:37:00.477 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2840_vm 2023-09-10T01:37:00.477 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2841 2023-09-10T01:37:00.477 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2995 2023-09-10T01:37:00.478 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2995_vm 2023-09-10T01:37:00.478 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2996 2023-09-10T01:37:00.478 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3079 2023-09-10T01:37:00.478 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3079_fsm 2023-09-10T01:37:00.478 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3079_vm 2023-09-10T01:37:00.478 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3080 2023-09-10T01:37:00.479 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3081 2023-09-10T01:37:00.479 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3085 2023-09-10T01:37:00.479 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3118 2023-09-10T01:37:00.479 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3118_vm 2023-09-10T01:37:00.479 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3119 2023-09-10T01:37:00.479 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3164 2023-09-10T01:37:00.480 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3256 2023-09-10T01:37:00.480 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3256_vm 2023-09-10T01:37:00.480 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3257 2023-09-10T01:37:00.480 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3258 2023-09-10T01:37:00.480 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3350 2023-09-10T01:37:00.480 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3350_vm 2023-09-10T01:37:00.481 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3351 2023-09-10T01:37:00.481 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3379 2023-09-10T01:37:00.481 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3380 2023-09-10T01:37:00.481 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3381 2023-09-10T01:37:00.481 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3381_vm 2023-09-10T01:37:00.482 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3394 2023-09-10T01:37:00.482 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3394_fsm 2023-09-10T01:37:00.482 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3394_vm 2023-09-10T01:37:00.482 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3395 2023-09-10T01:37:00.482 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3439 2023-09-10T01:37:00.483 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3439_vm 2023-09-10T01:37:00.483 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3440 2023-09-10T01:37:00.483 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 3455 2023-09-10T01:37:00.483 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3456 2023-09-10T01:37:00.483 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3456_fsm 2023-09-10T01:37:00.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3456_vm 2023-09-10T01:37:00.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3466 2023-09-10T01:37:00.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3466_vm 2023-09-10T01:37:00.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3467 2023-09-10T01:37:00.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3468 2023-09-10T01:37:00.484 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3501 2023-09-10T01:37:00.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3501_vm 2023-09-10T01:37:00.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3502 2023-09-10T01:37:00.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3503 2023-09-10T01:37:00.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3534 2023-09-10T01:37:00.485 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3541 2023-09-10T01:37:00.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3541_fsm 2023-09-10T01:37:00.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3541_vm 2023-09-10T01:37:00.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3542 2023-09-10T01:37:00.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3574 2023-09-10T01:37:00.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3575 2023-09-10T01:37:00.486 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3576 2023-09-10T01:37:00.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3576_vm 2023-09-10T01:37:00.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3596 2023-09-10T01:37:00.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3596_vm 2023-09-10T01:37:00.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3597 2023-09-10T01:37:00.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3598 2023-09-10T01:37:00.487 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3598_vm 2023-09-10T01:37:00.488 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3599 2023-09-10T01:37:00.488 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3600 2023-09-10T01:37:00.488 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3600_fsm 2023-09-10T01:37:00.488 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3600_vm 2023-09-10T01:37:00.488 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3601 2023-09-10T01:37:00.489 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3601_fsm 2023-09-10T01:37:00.489 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3601_vm 2023-09-10T01:37:00.489 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3602 2023-09-10T01:37:00.489 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3602_fsm 2023-09-10T01:37:00.489 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3602_vm 2023-09-10T01:37:00.490 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3603 2023-09-10T01:37:00.490 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3603_fsm 2023-09-10T01:37:00.490 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3603_vm 2023-09-10T01:37:00.490 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3604 2023-09-10T01:37:00.490 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3605 2023-09-10T01:37:00.491 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3606 2023-09-10T01:37:00.491 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3607 2023-09-10T01:37:00.491 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3608 2023-09-10T01:37:00.491 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 3609 2023-09-10T01:37:00.491 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3712 2023-09-10T01:37:00.492 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3764 2023-09-10T01:37:00.492 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3764_fsm 2023-09-10T01:37:00.492 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3764_vm 2023-09-10T01:37:00.492 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3766 2023-09-10T01:37:00.492 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3767 2023-09-10T01:37:00.493 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3997 2023-09-10T01:37:00.493 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 5002 2023-09-10T01:37:00.493 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 548 2023-09-10T01:37:00.493 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 549 2023-09-10T01:37:00.493 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6102 2023-09-10T01:37:00.493 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6102_vm 2023-09-10T01:37:00.494 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6104 2023-09-10T01:37:00.494 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6104_vm 2023-09-10T01:37:00.494 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6106 2023-09-10T01:37:00.494 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6106_vm 2023-09-10T01:37:00.494 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6110 2023-09-10T01:37:00.495 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6111 2023-09-10T01:37:00.495 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6112 2023-09-10T01:37:00.495 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6113 2023-09-10T01:37:00.495 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6117 2023-09-10T01:37:00.495 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 826 2023-09-10T01:37:00.496 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 826_vm 2023-09-10T01:37:00.730 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 827 2023-09-10T01:37:00.730 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 828 2023-09-10T01:37:00.730 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 00:40 pg_filenode.map 2023-09-10T01:37:00.731 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 00:40 PG_VERSION 2023-09-10T01:37:00.731 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:00.731 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-09-10T01:37:00.731 INFO:teuthology.orchestra.run.smithi044.stdout:total 7194 2023-09-10T01:37:00.731 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7365123 Sep 10 00:40 . 2023-09-10T01:37:00.732 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7902896269 Sep 10 01:18 .. 2023-09-10T01:37:00.732 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 112 2023-09-10T01:37:00.732 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 113 2023-09-10T01:37:00.732 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 00:40 12182 2023-09-10T01:37:00.732 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12182_fsm 2023-09-10T01:37:00.733 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12182_vm 2023-09-10T01:37:00.733 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12184 2023-09-10T01:37:00.733 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12186 2023-09-10T01:37:00.733 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12187 2023-09-10T01:37:00.733 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12187_fsm 2023-09-10T01:37:00.733 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12187_vm 2023-09-10T01:37:00.734 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12189 2023-09-10T01:37:00.734 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12191 2023-09-10T01:37:00.734 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12192 2023-09-10T01:37:00.734 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12192_fsm 2023-09-10T01:37:00.734 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12192_vm 2023-09-10T01:37:00.735 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12194 2023-09-10T01:37:00.735 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12196 2023-09-10T01:37:00.735 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12197 2023-09-10T01:37:00.735 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12197_fsm 2023-09-10T01:37:00.735 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12197_vm 2023-09-10T01:37:00.736 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12199 2023-09-10T01:37:00.736 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12201 2023-09-10T01:37:00.736 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12202 2023-09-10T01:37:00.736 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12202_fsm 2023-09-10T01:37:00.736 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12202_vm 2023-09-10T01:37:00.736 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12204 2023-09-10T01:37:00.737 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12206 2023-09-10T01:37:00.737 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12207 2023-09-10T01:37:00.737 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12207_fsm 2023-09-10T01:37:00.737 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12207_vm 2023-09-10T01:37:00.737 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12209 2023-09-10T01:37:00.738 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12211 2023-09-10T01:37:00.738 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12212 2023-09-10T01:37:00.738 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12214 2023-09-10T01:37:00.738 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12216 2023-09-10T01:37:00.738 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 10 00:40 1247 2023-09-10T01:37:00.739 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1247_fsm 2023-09-10T01:37:00.739 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1247_vm 2023-09-10T01:37:00.739 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 10 00:40 1249 2023-09-10T01:37:00.739 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1249_fsm 2023-09-10T01:37:00.739 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1249_vm 2023-09-10T01:37:00.739 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 10 00:40 1255 2023-09-10T01:37:00.740 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1255_fsm 2023-09-10T01:37:00.740 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1255_vm 2023-09-10T01:37:00.740 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 00:40 1259 2023-09-10T01:37:00.740 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1259_fsm 2023-09-10T01:37:00.740 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1259_vm 2023-09-10T01:37:00.741 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1417 2023-09-10T01:37:00.741 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1417_vm 2023-09-10T01:37:00.741 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1418 2023-09-10T01:37:00.741 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1418_vm 2023-09-10T01:37:00.741 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 174 2023-09-10T01:37:00.741 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 175 2023-09-10T01:37:00.742 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2187 2023-09-10T01:37:00.742 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2224 2023-09-10T01:37:00.742 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2224_vm 2023-09-10T01:37:00.742 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2328 2023-09-10T01:37:00.742 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2328_vm 2023-09-10T01:37:00.743 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2336 2023-09-10T01:37:00.743 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2336_vm 2023-09-10T01:37:00.743 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2337 2023-09-10T01:37:00.743 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2600 2023-09-10T01:37:00.743 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2600_fsm 2023-09-10T01:37:00.743 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2600_vm 2023-09-10T01:37:00.744 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2601 2023-09-10T01:37:00.744 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2601_fsm 2023-09-10T01:37:00.744 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2601_vm 2023-09-10T01:37:00.744 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2602 2023-09-10T01:37:00.744 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2602_fsm 2023-09-10T01:37:00.745 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2602_vm 2023-09-10T01:37:00.745 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2603 2023-09-10T01:37:00.745 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2603_fsm 2023-09-10T01:37:00.745 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2603_vm 2023-09-10T01:37:00.745 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2604 2023-09-10T01:37:00.745 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2604_vm 2023-09-10T01:37:00.746 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2605 2023-09-10T01:37:00.746 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2605_fsm 2023-09-10T01:37:00.746 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2605_vm 2023-09-10T01:37:00.746 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2606 2023-09-10T01:37:00.746 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2606_fsm 2023-09-10T01:37:00.747 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2606_vm 2023-09-10T01:37:00.747 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2607 2023-09-10T01:37:00.747 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2607_fsm 2023-09-10T01:37:00.747 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2607_vm 2023-09-10T01:37:00.747 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 10 00:40 2608 2023-09-10T01:37:00.747 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2608_fsm 2023-09-10T01:37:00.748 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2608_vm 2023-09-10T01:37:00.748 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 10 00:40 2609 2023-09-10T01:37:00.748 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2609_fsm 2023-09-10T01:37:00.748 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2609_vm 2023-09-10T01:37:00.748 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2610 2023-09-10T01:37:00.749 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2610_fsm 2023-09-10T01:37:00.749 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2610_vm 2023-09-10T01:37:00.749 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2611 2023-09-10T01:37:00.749 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2611_vm 2023-09-10T01:37:00.749 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2612 2023-09-10T01:37:00.749 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2612_fsm 2023-09-10T01:37:00.750 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2612_vm 2023-09-10T01:37:00.750 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2613 2023-09-10T01:37:00.750 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2613_vm 2023-09-10T01:37:00.750 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2615 2023-09-10T01:37:00.750 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2615_fsm 2023-09-10T01:37:00.750 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2615_vm 2023-09-10T01:37:00.751 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2616 2023-09-10T01:37:00.751 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2616_fsm 2023-09-10T01:37:00.751 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2616_vm 2023-09-10T01:37:00.751 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 00:40 2617 2023-09-10T01:37:00.751 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2617_fsm 2023-09-10T01:37:00.752 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2617_vm 2023-09-10T01:37:00.752 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 00:40 2618 2023-09-10T01:37:00.752 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2618_fsm 2023-09-10T01:37:00.752 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2618_vm 2023-09-10T01:37:00.752 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 10 00:40 2619 2023-09-10T01:37:00.753 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2619_fsm 2023-09-10T01:37:00.753 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2619_vm 2023-09-10T01:37:00.753 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2620 2023-09-10T01:37:00.753 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2620_vm 2023-09-10T01:37:00.753 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2650 2023-09-10T01:37:00.753 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2651 2023-09-10T01:37:00.754 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2652 2023-09-10T01:37:00.754 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2653 2023-09-10T01:37:00.754 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2654 2023-09-10T01:37:00.754 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2655 2023-09-10T01:37:00.754 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2656 2023-09-10T01:37:00.755 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2657 2023-09-10T01:37:00.755 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 00:40 2658 2023-09-10T01:37:00.755 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 00:40 2659 2023-09-10T01:37:00.755 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2660 2023-09-10T01:37:00.755 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2661 2023-09-10T01:37:00.756 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2662 2023-09-10T01:37:00.756 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2663 2023-09-10T01:37:00.756 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2664 2023-09-10T01:37:00.756 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2665 2023-09-10T01:37:00.756 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2666 2023-09-10T01:37:00.756 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2667 2023-09-10T01:37:00.757 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2668 2023-09-10T01:37:00.757 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2669 2023-09-10T01:37:00.757 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2670 2023-09-10T01:37:00.757 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Sep 10 00:40 2673 2023-09-10T01:37:00.757 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 10 00:40 2674 2023-09-10T01:37:00.757 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 10 00:40 2675 2023-09-10T01:37:00.758 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2678 2023-09-10T01:37:00.758 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2679 2023-09-10T01:37:00.758 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2680 2023-09-10T01:37:00.758 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2681 2023-09-10T01:37:00.758 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2682 2023-09-10T01:37:00.759 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2683 2023-09-10T01:37:00.759 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2684 2023-09-10T01:37:00.759 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2685 2023-09-10T01:37:00.759 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2686 2023-09-10T01:37:00.759 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2687 2023-09-10T01:37:00.759 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2688 2023-09-10T01:37:00.760 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2689 2023-09-10T01:37:00.760 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 00:40 2690 2023-09-10T01:37:00.760 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 10 00:40 2691 2023-09-10T01:37:00.760 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2692 2023-09-10T01:37:00.760 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2693 2023-09-10T01:37:00.761 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2696 2023-09-10T01:37:00.761 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2699 2023-09-10T01:37:00.761 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2701 2023-09-10T01:37:00.761 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2702 2023-09-10T01:37:00.761 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2703 2023-09-10T01:37:00.762 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2704 2023-09-10T01:37:00.762 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2753 2023-09-10T01:37:00.762 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2753_fsm 2023-09-10T01:37:00.762 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2753_vm 2023-09-10T01:37:00.762 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2754 2023-09-10T01:37:00.763 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2755 2023-09-10T01:37:00.763 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2756 2023-09-10T01:37:00.763 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2757 2023-09-10T01:37:00.763 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2830 2023-09-10T01:37:00.763 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2830_vm 2023-09-10T01:37:00.763 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2831 2023-09-10T01:37:00.764 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2832 2023-09-10T01:37:00.764 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2832_vm 2023-09-10T01:37:00.764 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2833 2023-09-10T01:37:00.764 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2834 2023-09-10T01:37:00.764 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2834_vm 2023-09-10T01:37:00.764 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2835 2023-09-10T01:37:00.765 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2836 2023-09-10T01:37:00.765 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2836_vm 2023-09-10T01:37:00.765 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2837 2023-09-10T01:37:00.765 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 10 00:40 2838 2023-09-10T01:37:00.765 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2838_fsm 2023-09-10T01:37:00.765 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2838_vm 2023-09-10T01:37:00.766 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2839 2023-09-10T01:37:00.766 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2840 2023-09-10T01:37:00.766 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2840_fsm 2023-09-10T01:37:00.766 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2840_vm 2023-09-10T01:37:00.766 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2841 2023-09-10T01:37:00.767 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2995 2023-09-10T01:37:00.767 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2995_vm 2023-09-10T01:37:00.767 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2996 2023-09-10T01:37:00.767 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3079 2023-09-10T01:37:00.767 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3079_fsm 2023-09-10T01:37:00.767 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3079_vm 2023-09-10T01:37:00.768 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3080 2023-09-10T01:37:00.768 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3081 2023-09-10T01:37:00.768 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3085 2023-09-10T01:37:00.768 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3118 2023-09-10T01:37:00.768 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3118_vm 2023-09-10T01:37:00.769 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3119 2023-09-10T01:37:00.769 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3164 2023-09-10T01:37:00.769 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3256 2023-09-10T01:37:00.769 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3256_vm 2023-09-10T01:37:00.769 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3257 2023-09-10T01:37:00.769 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3258 2023-09-10T01:37:00.770 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3350 2023-09-10T01:37:00.770 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3350_vm 2023-09-10T01:37:00.770 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3351 2023-09-10T01:37:00.770 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3379 2023-09-10T01:37:00.770 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3380 2023-09-10T01:37:00.771 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3381 2023-09-10T01:37:00.771 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3381_vm 2023-09-10T01:37:00.771 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3394 2023-09-10T01:37:00.771 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3394_fsm 2023-09-10T01:37:00.771 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3394_vm 2023-09-10T01:37:00.772 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3395 2023-09-10T01:37:00.772 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3439 2023-09-10T01:37:00.772 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3439_vm 2023-09-10T01:37:00.772 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3440 2023-09-10T01:37:00.772 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 3455 2023-09-10T01:37:00.772 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3456 2023-09-10T01:37:00.773 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3456_fsm 2023-09-10T01:37:00.773 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3456_vm 2023-09-10T01:37:00.773 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3466 2023-09-10T01:37:00.773 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3466_vm 2023-09-10T01:37:00.773 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3467 2023-09-10T01:37:00.774 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3468 2023-09-10T01:37:00.774 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3501 2023-09-10T01:37:00.774 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3501_vm 2023-09-10T01:37:00.774 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3502 2023-09-10T01:37:00.774 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3503 2023-09-10T01:37:00.774 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3534 2023-09-10T01:37:00.775 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3541 2023-09-10T01:37:00.775 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3541_fsm 2023-09-10T01:37:00.775 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3541_vm 2023-09-10T01:37:00.775 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3542 2023-09-10T01:37:00.775 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3574 2023-09-10T01:37:00.776 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3575 2023-09-10T01:37:00.776 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3576 2023-09-10T01:37:00.776 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3576_vm 2023-09-10T01:37:00.776 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3596 2023-09-10T01:37:00.776 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3596_vm 2023-09-10T01:37:00.776 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3597 2023-09-10T01:37:00.777 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3598 2023-09-10T01:37:00.777 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3598_vm 2023-09-10T01:37:00.777 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3599 2023-09-10T01:37:00.777 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3600 2023-09-10T01:37:00.777 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3600_fsm 2023-09-10T01:37:00.778 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3600_vm 2023-09-10T01:37:00.778 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3601 2023-09-10T01:37:00.778 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3601_fsm 2023-09-10T01:37:00.778 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3601_vm 2023-09-10T01:37:00.778 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3602 2023-09-10T01:37:00.779 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3602_fsm 2023-09-10T01:37:00.779 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3602_vm 2023-09-10T01:37:00.779 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3603 2023-09-10T01:37:00.779 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3603_fsm 2023-09-10T01:37:00.779 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3603_vm 2023-09-10T01:37:00.779 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3604 2023-09-10T01:37:00.780 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3605 2023-09-10T01:37:00.780 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3606 2023-09-10T01:37:00.780 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3607 2023-09-10T01:37:00.780 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3608 2023-09-10T01:37:00.780 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 3609 2023-09-10T01:37:00.781 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3712 2023-09-10T01:37:00.781 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3764 2023-09-10T01:37:00.781 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3764_fsm 2023-09-10T01:37:00.781 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3764_vm 2023-09-10T01:37:00.781 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3766 2023-09-10T01:37:00.781 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3767 2023-09-10T01:37:00.782 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3997 2023-09-10T01:37:00.782 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 5002 2023-09-10T01:37:00.782 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 548 2023-09-10T01:37:00.782 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 549 2023-09-10T01:37:00.782 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6102 2023-09-10T01:37:00.783 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6102_vm 2023-09-10T01:37:00.783 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6104 2023-09-10T01:37:00.783 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6104_vm 2023-09-10T01:37:00.783 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6106 2023-09-10T01:37:00.783 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6106_vm 2023-09-10T01:37:01.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:00 smithi099 ceph-mon[102692]: pgmap v1836: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 142 KiB/s rd, 140 KiB/s wr, 25 op/s 2023-09-10T01:37:01.156 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6110 2023-09-10T01:37:01.156 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6111 2023-09-10T01:37:01.156 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6112 2023-09-10T01:37:01.156 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6113 2023-09-10T01:37:01.156 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6117 2023-09-10T01:37:01.157 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 826 2023-09-10T01:37:01.157 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 826_vm 2023-09-10T01:37:01.157 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 827 2023-09-10T01:37:01.157 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 828 2023-09-10T01:37:01.157 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 00:40 pg_filenode.map 2023-09-10T01:37:01.157 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 00:40 PG_VERSION 2023-09-10T01:37:01.158 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.158 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-09-10T01:37:01.158 INFO:teuthology.orchestra.run.smithi044.stdout:total 7703289 2023-09-10T01:37:01.158 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7888166023 Sep 10 01:22 . 2023-09-10T01:37:01.158 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7902896269 Sep 10 01:18 .. 2023-09-10T01:37:01.159 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 112 2023-09-10T01:37:01.159 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 113 2023-09-10T01:37:01.159 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 00:40 12182 2023-09-10T01:37:01.159 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12182_fsm 2023-09-10T01:37:01.159 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12182_vm 2023-09-10T01:37:01.160 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12184 2023-09-10T01:37:01.160 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12186 2023-09-10T01:37:01.160 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12187 2023-09-10T01:37:01.160 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12187_fsm 2023-09-10T01:37:01.160 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12187_vm 2023-09-10T01:37:01.161 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12189 2023-09-10T01:37:01.161 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12191 2023-09-10T01:37:01.161 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12192 2023-09-10T01:37:01.161 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12192_fsm 2023-09-10T01:37:01.161 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12192_vm 2023-09-10T01:37:01.162 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12194 2023-09-10T01:37:01.162 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12196 2023-09-10T01:37:01.162 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12197 2023-09-10T01:37:01.162 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12197_fsm 2023-09-10T01:37:01.162 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12197_vm 2023-09-10T01:37:01.163 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12199 2023-09-10T01:37:01.163 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12201 2023-09-10T01:37:01.163 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12202 2023-09-10T01:37:01.163 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12202_fsm 2023-09-10T01:37:01.163 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12202_vm 2023-09-10T01:37:01.164 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12204 2023-09-10T01:37:01.164 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12206 2023-09-10T01:37:01.164 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12207 2023-09-10T01:37:01.164 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 12207_fsm 2023-09-10T01:37:01.164 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12207_vm 2023-09-10T01:37:01.165 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12209 2023-09-10T01:37:01.165 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12211 2023-09-10T01:37:01.165 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12212 2023-09-10T01:37:01.165 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 12214 2023-09-10T01:37:01.165 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 12216 2023-09-10T01:37:01.165 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 10 00:41 1247 2023-09-10T01:37:01.166 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:41 1247_fsm 2023-09-10T01:37:01.166 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:41 1247_vm 2023-09-10T01:37:01.166 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 10 01:21 1249 2023-09-10T01:37:01.166 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:41 1249_fsm 2023-09-10T01:37:01.166 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:41 1249_vm 2023-09-10T01:37:01.167 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 10 00:40 1255 2023-09-10T01:37:01.167 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1255_fsm 2023-09-10T01:37:01.167 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1255_vm 2023-09-10T01:37:01.167 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 01:36 1259 2023-09-10T01:37:01.167 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1259_fsm 2023-09-10T01:37:01.168 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:41 1259_vm 2023-09-10T01:37:01.168 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1417 2023-09-10T01:37:01.168 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1417_vm 2023-09-10T01:37:01.168 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1418 2023-09-10T01:37:01.168 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 1418_vm 2023-09-10T01:37:01.169 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 581632 Sep 10 01:36 16387 2023-09-10T01:37:01.169 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 01:36 16387_fsm 2023-09-10T01:37:01.169 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 01:36 16387_vm 2023-09-10T01:37:01.169 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 10 01:36 16393 2023-09-10T01:37:01.169 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 01:36 16393_fsm 2023-09-10T01:37:01.170 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 01:35 16393_vm 2023-09-10T01:37:01.170 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 10 01:36 16396 2023-09-10T01:37:01.170 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 10 01:36 16396.1 2023-09-10T01:37:01.170 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 10 01:36 16396.2 2023-09-10T01:37:01.171 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 10 01:36 16396.3 2023-09-10T01:37:01.171 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 10 01:36 16396.4 2023-09-10T01:37:01.171 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 10 01:36 16396.5 2023-09-10T01:37:01.171 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 300916736 Sep 10 01:36 16396.6 2023-09-10T01:37:01.171 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1679360 Sep 10 01:36 16396_fsm 2023-09-10T01:37:01.172 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 212992 Sep 10 01:36 16396_vm 2023-09-10T01:37:01.172 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 01:36 16397 2023-09-10T01:37:01.172 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 237568 Sep 10 01:36 16399 2023-09-10T01:37:01.172 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 10 01:36 16401 2023-09-10T01:37:01.173 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49373184 Sep 10 01:36 16401.1 2023-09-10T01:37:01.173 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 11042816 Sep 10 01:36 16403 2023-09-10T01:37:01.173 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 01:36 16403_fsm 2023-09-10T01:37:01.173 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 174 2023-09-10T01:37:01.173 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 175 2023-09-10T01:37:01.173 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2187 2023-09-10T01:37:01.174 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2224 2023-09-10T01:37:01.174 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2224_vm 2023-09-10T01:37:01.174 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2328 2023-09-10T01:37:01.174 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2328_vm 2023-09-10T01:37:01.174 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2336 2023-09-10T01:37:01.175 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2336_vm 2023-09-10T01:37:01.175 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2337 2023-09-10T01:37:01.175 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2600 2023-09-10T01:37:01.175 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2600_fsm 2023-09-10T01:37:01.175 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2600_vm 2023-09-10T01:37:01.175 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2601 2023-09-10T01:37:01.176 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2601_fsm 2023-09-10T01:37:01.176 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2601_vm 2023-09-10T01:37:01.176 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2602 2023-09-10T01:37:01.176 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2602_fsm 2023-09-10T01:37:01.176 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2602_vm 2023-09-10T01:37:01.177 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2603 2023-09-10T01:37:01.177 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2603_fsm 2023-09-10T01:37:01.177 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2603_vm 2023-09-10T01:37:01.177 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2604 2023-09-10T01:37:01.177 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2604_vm 2023-09-10T01:37:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2605 2023-09-10T01:37:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2605_fsm 2023-09-10T01:37:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2605_vm 2023-09-10T01:37:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 01:19 2606 2023-09-10T01:37:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2606_fsm 2023-09-10T01:37:01.179 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 01:19 2606_vm 2023-09-10T01:37:01.179 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2607 2023-09-10T01:37:01.179 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2607_fsm 2023-09-10T01:37:01.179 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2607_vm 2023-09-10T01:37:01.179 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 Sep 10 01:19 2608 2023-09-10T01:37:01.179 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 01:19 2608_fsm 2023-09-10T01:37:01.180 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 01:19 2608_vm 2023-09-10T01:37:01.180 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 10 00:40 2609 2023-09-10T01:37:01.180 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2609_fsm 2023-09-10T01:37:01.180 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2609_vm 2023-09-10T01:37:01.180 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 01:21 2610 2023-09-10T01:37:01.181 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 01:19 2610_fsm 2023-09-10T01:37:01.181 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 01:19 2610_vm 2023-09-10T01:37:01.181 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2611 2023-09-10T01:37:01.181 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2611_vm 2023-09-10T01:37:01.181 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2612 2023-09-10T01:37:01.182 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2612_fsm 2023-09-10T01:37:01.182 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2612_vm 2023-09-10T01:37:01.182 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2613 2023-09-10T01:37:01.182 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2613_vm 2023-09-10T01:37:01.182 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2615 2023-09-10T01:37:01.182 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2615_fsm 2023-09-10T01:37:01.183 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2615_vm 2023-09-10T01:37:01.183 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2616 2023-09-10T01:37:01.183 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2616_fsm 2023-09-10T01:37:01.183 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2616_vm 2023-09-10T01:37:01.183 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 00:40 2617 2023-09-10T01:37:01.184 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2617_fsm 2023-09-10T01:37:01.184 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2617_vm 2023-09-10T01:37:01.184 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 00:40 2618 2023-09-10T01:37:01.184 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2618_fsm 2023-09-10T01:37:01.184 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2618_vm 2023-09-10T01:37:01.185 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Sep 10 01:36 2619 2023-09-10T01:37:01.185 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 01:29 2619_fsm 2023-09-10T01:37:01.185 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:58 2619_vm 2023-09-10T01:37:01.185 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2620 2023-09-10T01:37:01.185 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2620_vm 2023-09-10T01:37:01.185 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2650 2023-09-10T01:37:01.186 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2651 2023-09-10T01:37:01.186 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2652 2023-09-10T01:37:01.186 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2653 2023-09-10T01:37:01.186 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2654 2023-09-10T01:37:01.186 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2655 2023-09-10T01:37:01.187 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2656 2023-09-10T01:37:01.187 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2657 2023-09-10T01:37:01.187 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 10 01:19 2658 2023-09-10T01:37:01.187 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 01:19 2659 2023-09-10T01:37:01.187 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2660 2023-09-10T01:37:01.187 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2661 2023-09-10T01:37:01.188 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 01:21 2662 2023-09-10T01:37:01.188 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 01:21 2663 2023-09-10T01:37:01.188 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 01:19 2664 2023-09-10T01:37:01.188 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 01:19 2665 2023-09-10T01:37:01.188 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 01:19 2666 2023-09-10T01:37:01.189 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 01:19 2667 2023-09-10T01:37:01.189 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2668 2023-09-10T01:37:01.189 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2669 2023-09-10T01:37:01.189 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2670 2023-09-10T01:37:01.189 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Sep 10 01:19 2673 2023-09-10T01:37:01.189 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 10 01:19 2674 2023-09-10T01:37:01.190 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 10 00:40 2675 2023-09-10T01:37:01.190 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 01:19 2678 2023-09-10T01:37:01.190 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 01:19 2679 2023-09-10T01:37:01.190 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2680 2023-09-10T01:37:01.190 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2681 2023-09-10T01:37:01.191 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2682 2023-09-10T01:37:01.191 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2683 2023-09-10T01:37:01.191 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2684 2023-09-10T01:37:01.191 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2685 2023-09-10T01:37:01.191 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2686 2023-09-10T01:37:01.192 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2687 2023-09-10T01:37:01.192 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:40 2688 2023-09-10T01:37:01.192 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 00:40 2689 2023-09-10T01:37:01.192 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 00:40 2690 2023-09-10T01:37:01.192 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 10 00:40 2691 2023-09-10T01:37:01.192 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2692 2023-09-10T01:37:01.193 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2693 2023-09-10T01:37:01.193 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 01:29 2696 2023-09-10T01:37:01.193 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2699 2023-09-10T01:37:01.193 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2701 2023-09-10T01:37:01.193 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2702 2023-09-10T01:37:01.194 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:41 2703 2023-09-10T01:37:01.194 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 00:41 2704 2023-09-10T01:37:01.194 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2753 2023-09-10T01:37:01.194 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2753_fsm 2023-09-10T01:37:01.194 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2753_vm 2023-09-10T01:37:01.195 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2754 2023-09-10T01:37:01.195 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2755 2023-09-10T01:37:01.195 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2756 2023-09-10T01:37:01.195 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 2757 2023-09-10T01:37:01.195 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2830 2023-09-10T01:37:01.195 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2830_vm 2023-09-10T01:37:01.196 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2831 2023-09-10T01:37:01.196 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2832 2023-09-10T01:37:01.196 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2832_vm 2023-09-10T01:37:01.196 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2833 2023-09-10T01:37:01.196 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2834 2023-09-10T01:37:01.197 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2834_vm 2023-09-10T01:37:01.197 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2835 2023-09-10T01:37:01.197 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2836 2023-09-10T01:37:01.197 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2836_vm 2023-09-10T01:37:01.197 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2837 2023-09-10T01:37:01.197 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 10 00:40 2838 2023-09-10T01:37:01.198 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2838_fsm 2023-09-10T01:37:01.198 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2838_vm 2023-09-10T01:37:01.198 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2839 2023-09-10T01:37:01.198 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2840 2023-09-10T01:37:01.198 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2840_fsm 2023-09-10T01:37:01.199 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2840_vm 2023-09-10T01:37:01.199 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2841 2023-09-10T01:37:01.199 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2995 2023-09-10T01:37:01.199 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2995_vm 2023-09-10T01:37:01.199 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2996 2023-09-10T01:37:01.200 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3079 2023-09-10T01:37:01.200 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3079_fsm 2023-09-10T01:37:01.200 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3079_vm 2023-09-10T01:37:01.200 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3080 2023-09-10T01:37:01.200 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3081 2023-09-10T01:37:01.200 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3085 2023-09-10T01:37:01.201 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3118 2023-09-10T01:37:01.201 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3118_vm 2023-09-10T01:37:01.201 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3119 2023-09-10T01:37:01.201 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3164 2023-09-10T01:37:01.201 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3256 2023-09-10T01:37:01.201 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3256_vm 2023-09-10T01:37:01.202 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3257 2023-09-10T01:37:01.202 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3258 2023-09-10T01:37:01.202 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3350 2023-09-10T01:37:01.202 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3350_vm 2023-09-10T01:37:01.202 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3351 2023-09-10T01:37:01.202 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3379 2023-09-10T01:37:01.203 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3380 2023-09-10T01:37:01.203 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3381 2023-09-10T01:37:01.203 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3381_vm 2023-09-10T01:37:01.203 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3394 2023-09-10T01:37:01.203 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3394_fsm 2023-09-10T01:37:01.203 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3394_vm 2023-09-10T01:37:01.204 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3395 2023-09-10T01:37:01.204 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3439 2023-09-10T01:37:01.204 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3439_vm 2023-09-10T01:37:01.204 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3440 2023-09-10T01:37:01.204 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 01:21 3455 2023-09-10T01:37:01.205 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3456 2023-09-10T01:37:01.205 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3456_fsm 2023-09-10T01:37:01.205 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3456_vm 2023-09-10T01:37:01.205 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3466 2023-09-10T01:37:01.205 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3466_vm 2023-09-10T01:37:01.205 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3467 2023-09-10T01:37:01.206 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3468 2023-09-10T01:37:01.206 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3501 2023-09-10T01:37:01.206 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3501_vm 2023-09-10T01:37:01.206 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3502 2023-09-10T01:37:01.206 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3503 2023-09-10T01:37:01.206 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3534 2023-09-10T01:37:01.207 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3541 2023-09-10T01:37:01.207 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3541_fsm 2023-09-10T01:37:01.207 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3541_vm 2023-09-10T01:37:01.207 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3542 2023-09-10T01:37:01.207 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3574 2023-09-10T01:37:01.207 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3575 2023-09-10T01:37:01.208 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3576 2023-09-10T01:37:01.208 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3576_vm 2023-09-10T01:37:01.208 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3596 2023-09-10T01:37:01.208 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3596_vm 2023-09-10T01:37:01.208 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3597 2023-09-10T01:37:01.208 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3598 2023-09-10T01:37:01.209 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3598_vm 2023-09-10T01:37:01.209 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3599 2023-09-10T01:37:01.209 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3600 2023-09-10T01:37:01.209 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3600_fsm 2023-09-10T01:37:01.209 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3600_vm 2023-09-10T01:37:01.209 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3601 2023-09-10T01:37:01.210 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3601_fsm 2023-09-10T01:37:01.210 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3601_vm 2023-09-10T01:37:01.210 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3602 2023-09-10T01:37:01.211 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3602_fsm 2023-09-10T01:37:01.211 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3602_vm 2023-09-10T01:37:01.211 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3603 2023-09-10T01:37:01.211 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3603_fsm 2023-09-10T01:37:01.211 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3603_vm 2023-09-10T01:37:01.211 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3604 2023-09-10T01:37:01.212 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3605 2023-09-10T01:37:01.212 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3606 2023-09-10T01:37:01.212 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3607 2023-09-10T01:37:01.212 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3608 2023-09-10T01:37:01.212 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 00:40 3609 2023-09-10T01:37:01.213 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3712 2023-09-10T01:37:01.213 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3764 2023-09-10T01:37:01.213 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 3764_fsm 2023-09-10T01:37:01.213 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3764_vm 2023-09-10T01:37:01.213 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3766 2023-09-10T01:37:01.213 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 3767 2023-09-10T01:37:01.214 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3997 2023-09-10T01:37:01.214 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 5002 2023-09-10T01:37:01.214 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 548 2023-09-10T01:37:01.214 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 549 2023-09-10T01:37:01.214 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6102 2023-09-10T01:37:01.215 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6102_vm 2023-09-10T01:37:01.215 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6104 2023-09-10T01:37:01.215 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6104_vm 2023-09-10T01:37:01.215 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6106 2023-09-10T01:37:01.215 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6106_vm 2023-09-10T01:37:01.215 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6110 2023-09-10T01:37:01.216 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6111 2023-09-10T01:37:01.216 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6112 2023-09-10T01:37:01.216 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6113 2023-09-10T01:37:01.216 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6117 2023-09-10T01:37:01.216 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 826 2023-09-10T01:37:01.216 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 826_vm 2023-09-10T01:37:01.217 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 827 2023-09-10T01:37:01.217 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 828 2023-09-10T01:37:01.217 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 00:40 pg_filenode.map 2023-09-10T01:37:01.217 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 Sep 10 00:41 pg_internal.init 2023-09-10T01:37:01.217 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 00:40 PG_VERSION 2023-09-10T01:37:01.218 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.218 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2023-09-10T01:37:01.218 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T01:37:01.218 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 01:21 . 2023-09-10T01:37:01.219 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7902896269 Sep 10 01:18 .. 2023-09-10T01:37:01.219 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.219 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/global: 2023-09-10T01:37:01.219 INFO:teuthology.orchestra.run.smithi044.stdout:total 575 2023-09-10T01:37:01.219 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 587404 Sep 10 00:41 . 2023-09-10T01:37:01.220 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.220 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1136 2023-09-10T01:37:01.220 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1136_fsm 2023-09-10T01:37:01.220 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1136_vm 2023-09-10T01:37:01.220 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 1137 2023-09-10T01:37:01.221 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1213 2023-09-10T01:37:01.221 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1213_fsm 2023-09-10T01:37:01.221 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1213_vm 2023-09-10T01:37:01.221 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1214 2023-09-10T01:37:01.221 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1214_fsm 2023-09-10T01:37:01.221 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1214_vm 2023-09-10T01:37:01.222 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 1232 2023-09-10T01:37:01.222 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 1233 2023-09-10T01:37:01.222 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1260 2023-09-10T01:37:01.222 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1260_fsm 2023-09-10T01:37:01.222 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1260_vm 2023-09-10T01:37:01.223 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1261 2023-09-10T01:37:01.223 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1261_fsm 2023-09-10T01:37:01.223 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1261_vm 2023-09-10T01:37:01.223 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1262 2023-09-10T01:37:01.223 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 1262_fsm 2023-09-10T01:37:01.224 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 1262_vm 2023-09-10T01:37:01.224 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2396 2023-09-10T01:37:01.224 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 00:40 2396_fsm 2023-09-10T01:37:01.224 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2396_vm 2023-09-10T01:37:01.224 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2397 2023-09-10T01:37:01.224 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2671 2023-09-10T01:37:01.225 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2672 2023-09-10T01:37:01.225 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2676 2023-09-10T01:37:01.225 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2677 2023-09-10T01:37:01.225 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2694 2023-09-10T01:37:01.225 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2695 2023-09-10T01:37:01.226 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2697 2023-09-10T01:37:01.226 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 2698 2023-09-10T01:37:01.226 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2846 2023-09-10T01:37:01.226 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2846_vm 2023-09-10T01:37:01.226 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2847 2023-09-10T01:37:01.226 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2964 2023-09-10T01:37:01.246 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2964_vm 2023-09-10T01:37:01.247 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2965 2023-09-10T01:37:01.247 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2966 2023-09-10T01:37:01.247 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 2966_vm 2023-09-10T01:37:01.247 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 2967 2023-09-10T01:37:01.247 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3592 2023-09-10T01:37:01.247 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 3592_vm 2023-09-10T01:37:01.248 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 3593 2023-09-10T01:37:01.248 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 4060 2023-09-10T01:37:01.248 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 4060_vm 2023-09-10T01:37:01.248 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 4061 2023-09-10T01:37:01.248 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6000 2023-09-10T01:37:01.249 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6000_vm 2023-09-10T01:37:01.249 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6001 2023-09-10T01:37:01.249 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6002 2023-09-10T01:37:01.249 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6100 2023-09-10T01:37:01.249 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 6100_vm 2023-09-10T01:37:01.250 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6114 2023-09-10T01:37:01.250 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 6115 2023-09-10T01:37:01.250 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 01:36 pg_control 2023-09-10T01:37:01.250 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 00:40 pg_filenode.map 2023-09-10T01:37:01.250 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 Sep 10 00:41 pg_internal.init 2023-09-10T01:37:01.250 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.251 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/log: 2023-09-10T01:37:01.251 INFO:teuthology.orchestra.run.smithi044.stdout:total 2 2023-09-10T01:37:01.251 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 699 Sep 10 00:40 . 2023-09-10T01:37:01.251 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.251 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 699 Sep 10 01:36 postgresql-Sun.log 2023-09-10T01:37:01.252 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.252 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-09-10T01:37:01.252 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T01:37:01.252 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T01:37:01.252 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.253 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.253 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-09-10T01:37:01.253 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T01:37:01.253 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T01:37:01.253 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.253 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.254 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-09-10T01:37:01.254 INFO:teuthology.orchestra.run.smithi044.stdout:total 3 2023-09-10T01:37:01.254 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 10 01:36 . 2023-09-10T01:37:01.254 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.254 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 mappings 2023-09-10T01:37:01.255 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 10 01:36 replorigin_checkpoint 2023-09-10T01:37:01.255 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 snapshots 2023-09-10T01:37:01.255 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.255 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-09-10T01:37:01.255 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T01:37:01.255 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T01:37:01.256 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 10 01:36 .. 2023-09-10T01:37:01.256 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.256 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-09-10T01:37:01.256 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T01:37:01.256 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T01:37:01.257 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 10 01:36 .. 2023-09-10T01:37:01.257 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.257 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-09-10T01:37:01.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:00 smithi131 ceph-mon[103713]: pgmap v1836: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 142 KiB/s rd, 140 KiB/s wr, 25 op/s 2023-09-10T01:37:01.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:00 smithi044 ceph-mon[98676]: pgmap v1836: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 142 KiB/s rd, 140 KiB/s wr, 25 op/s 2023-09-10T01:37:01.371 INFO:teuthology.orchestra.run.smithi044.stdout:total 2 2023-09-10T01:37:01.372 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 . 2023-09-10T01:37:01.372 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.372 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 members 2023-09-10T01:37:01.372 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 offsets 2023-09-10T01:37:01.373 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.373 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-09-10T01:37:01.373 INFO:teuthology.orchestra.run.smithi044.stdout:total 9 2023-09-10T01:37:01.373 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 . 2023-09-10T01:37:01.373 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 .. 2023-09-10T01:37:01.374 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 0000 2023-09-10T01:37:01.374 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.374 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-09-10T01:37:01.374 INFO:teuthology.orchestra.run.smithi044.stdout:total 9 2023-09-10T01:37:01.374 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 . 2023-09-10T01:37:01.375 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 00:40 .. 2023-09-10T01:37:01.375 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:41 0000 2023-09-10T01:37:01.375 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.375 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-09-10T01:37:01.375 INFO:teuthology.orchestra.run.smithi044.stdout:total 9 2023-09-10T01:37:01.375 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 . 2023-09-10T01:37:01.376 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.376 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 00:40 0000 2023-09-10T01:37:01.376 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.376 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-09-10T01:37:01.376 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T01:37:01.377 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T01:37:01.377 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.377 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.377 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-09-10T01:37:01.377 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T01:37:01.378 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T01:37:01.378 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.378 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.378 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-09-10T01:37:01.378 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T01:37:01.379 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T01:37:01.379 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.379 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.379 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-09-10T01:37:01.379 INFO:teuthology.orchestra.run.smithi044.stdout:total 14 2023-09-10T01:37:01.380 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 11771 Sep 10 01:36 . 2023-09-10T01:37:01.380 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.380 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 Sep 10 01:36 db_0.stat 2023-09-10T01:37:01.380 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 Sep 10 01:36 db_12347.stat 2023-09-10T01:37:01.380 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 Sep 10 01:36 global.stat 2023-09-10T01:37:01.380 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.381 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-09-10T01:37:01.381 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T01:37:01.381 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 11771 Sep 10 01:36 . 2023-09-10T01:37:01.381 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.381 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.382 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-09-10T01:37:01.382 INFO:teuthology.orchestra.run.smithi044.stdout:total 49 2023-09-10T01:37:01.382 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 01:36 . 2023-09-10T01:37:01.382 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.382 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 01:36 0003 2023-09-10T01:37:01.383 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.383 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-09-10T01:37:01.383 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T01:37:01.383 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T01:37:01.383 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.384 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.384 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-09-10T01:37:01.384 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T01:37:01.384 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T01:37:01.384 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.385 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.385 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-09-10T01:37:01.385 INFO:teuthology.orchestra.run.smithi044.stdout:total 1048578 2023-09-10T01:37:01.385 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 10 01:36 . 2023-09-10T01:37:01.385 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:34 000000010000000200000025 2023-09-10T01:37:01.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:34 000000010000000200000026 2023-09-10T01:37:01.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:34 000000010000000200000027 2023-09-10T01:37:01.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:34 000000010000000200000028 2023-09-10T01:37:01.386 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:34 000000010000000200000029 2023-09-10T01:37:01.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:34 00000001000000020000002A 2023-09-10T01:37:01.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:34 00000001000000020000002B 2023-09-10T01:37:01.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:34 00000001000000020000002C 2023-09-10T01:37:01.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:35 00000001000000020000002D 2023-09-10T01:37:01.387 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:35 00000001000000020000002E 2023-09-10T01:37:01.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:35 00000001000000020000002F 2023-09-10T01:37:01.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:35 000000010000000200000030 2023-09-10T01:37:01.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:35 000000010000000200000031 2023-09-10T01:37:01.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:35 000000010000000200000032 2023-09-10T01:37:01.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:35 000000010000000200000033 2023-09-10T01:37:01.388 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:35 000000010000000200000034 2023-09-10T01:37:01.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:36 000000010000000200000035 2023-09-10T01:37:01.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:36 000000010000000200000036 2023-09-10T01:37:01.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:31 000000010000000200000037 2023-09-10T01:37:01.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:30 000000010000000200000038 2023-09-10T01:37:01.389 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:31 000000010000000200000039 2023-09-10T01:37:01.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:30 00000001000000020000003A 2023-09-10T01:37:01.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:29 00000001000000020000003B 2023-09-10T01:37:01.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:31 00000001000000020000003C 2023-09-10T01:37:01.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:31 00000001000000020000003D 2023-09-10T01:37:01.390 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:30 00000001000000020000003E 2023-09-10T01:37:01.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:31 00000001000000020000003F 2023-09-10T01:37:01.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:30 000000010000000200000040 2023-09-10T01:37:01.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:30 000000010000000200000041 2023-09-10T01:37:01.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:30 000000010000000200000042 2023-09-10T01:37:01.391 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:31 000000010000000200000043 2023-09-10T01:37:01.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:30 000000010000000200000044 2023-09-10T01:37:01.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:31 000000010000000200000045 2023-09-10T01:37:01.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:31 000000010000000200000046 2023-09-10T01:37:01.392 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:30 000000010000000200000047 2023-09-10T01:37:01.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:31 000000010000000200000048 2023-09-10T01:37:01.393 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:31 000000010000000200000049 2023-09-10T01:37:01.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:31 00000001000000020000004A 2023-09-10T01:37:01.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:30 00000001000000020000004B 2023-09-10T01:37:01.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:34 00000001000000020000004C 2023-09-10T01:37:01.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:33 00000001000000020000004D 2023-09-10T01:37:01.394 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:33 00000001000000020000004E 2023-09-10T01:37:01.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:32 00000001000000020000004F 2023-09-10T01:37:01.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:33 000000010000000200000050 2023-09-10T01:37:01.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:32 000000010000000200000051 2023-09-10T01:37:01.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:32 000000010000000200000052 2023-09-10T01:37:01.395 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:33 000000010000000200000053 2023-09-10T01:37:01.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:33 000000010000000200000054 2023-09-10T01:37:01.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:32 000000010000000200000055 2023-09-10T01:37:01.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:34 000000010000000200000056 2023-09-10T01:37:01.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:33 000000010000000200000057 2023-09-10T01:37:01.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:34 000000010000000200000058 2023-09-10T01:37:01.396 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:32 000000010000000200000059 2023-09-10T01:37:01.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:33 00000001000000020000005A 2023-09-10T01:37:01.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:33 00000001000000020000005B 2023-09-10T01:37:01.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:32 00000001000000020000005C 2023-09-10T01:37:01.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:32 00000001000000020000005D 2023-09-10T01:37:01.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:32 00000001000000020000005E 2023-09-10T01:37:01.397 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:32 00000001000000020000005F 2023-09-10T01:37:01.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:33 000000010000000200000060 2023-09-10T01:37:01.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:33 000000010000000200000061 2023-09-10T01:37:01.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:33 000000010000000200000062 2023-09-10T01:37:01.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:34 000000010000000200000063 2023-09-10T01:37:01.398 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 01:32 000000010000000200000064 2023-09-10T01:37:01.399 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 archive_status 2023-09-10T01:37:01.399 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.399 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-09-10T01:37:01.399 INFO:teuthology.orchestra.run.smithi044.stdout:total 1 2023-09-10T01:37:01.399 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 00:40 . 2023-09-10T01:37:01.400 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Sep 10 01:36 .. 2023-09-10T01:37:01.400 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:01.400 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-09-10T01:37:01.400 INFO:teuthology.orchestra.run.smithi044.stdout:total 57 2023-09-10T01:37:01.400 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 00:40 . 2023-09-10T01:37:01.400 INFO:teuthology.orchestra.run.smithi044.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8977409812 Sep 10 01:36 .. 2023-09-10T01:37:01.401 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 01:36 0000 2023-09-10T01:37:01.401 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-09-10T01:37:01.412 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi044.front.sepia.ceph.com 2023-09-10T01:37:01.420 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-09-10T01:37:01.447 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-09-10T01:37:01.448 INFO:teuthology.task.exec:Executing custom commands... 2023-09-10T01:37:01.448 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2023-09-10T01:37:01.449 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-09-10T01:37:01.498 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-09-10T01:37:01.581 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-09-10T01:37:01.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.747+0000 7f2d29350700 1 -- 172.21.15.44:0/1553386534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d240ff4d0 msgr2=0x7f2d240ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:01.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.747+0000 7f2d29350700 1 --2- 172.21.15.44:0/1553386534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d240ff4d0 0x7f2d240ff950 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0099c0 tx=0x7f2d0c009cd0 comp rx=0 tx=0).stop 2023-09-10T01:37:01.751 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.747+0000 7f2d29350700 1 -- 172.21.15.44:0/1553386534 shutdown_connections 2023-09-10T01:37:01.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.747+0000 7f2d29350700 1 --2- 172.21.15.44:0/1553386534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d240ffe90 0x7f2d2410c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:01.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.747+0000 7f2d29350700 1 --2- 172.21.15.44:0/1553386534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d24105590 0x7f2d24105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:01.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.747+0000 7f2d29350700 1 --2- 172.21.15.44:0/1553386534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d240ff4d0 0x7f2d240ff950 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:01.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.747+0000 7f2d29350700 1 -- 172.21.15.44:0/1553386534 >> 172.21.15.44:0/1553386534 conn(0x7f2d240faf60 msgr2=0x7f2d240fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:01.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.747+0000 7f2d29350700 1 -- 172.21.15.44:0/1553386534 shutdown_connections 2023-09-10T01:37:01.752 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.747+0000 7f2d29350700 1 -- 172.21.15.44:0/1553386534 wait complete. 2023-09-10T01:37:01.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d29350700 1 Processor -- start 2023-09-10T01:37:01.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d29350700 1 -- start start 2023-09-10T01:37:01.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d29350700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d240ff4d0 0x7f2d24196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:01.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d29350700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d240ffe90 0x7f2d24196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:01.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d29350700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d24105590 0x7f2d2419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:01.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d29350700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2419a610 con 0x7f2d240ffe90 2023-09-10T01:37:01.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d29350700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2419a780 con 0x7f2d24105590 2023-09-10T01:37:01.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d29350700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2419a940 con 0x7f2d240ff4d0 2023-09-10T01:37:01.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d227fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d240ffe90 0x7f2d24196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:01.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d227fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d240ffe90 0x7f2d24196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33278/0 (socket says 172.21.15.44:33278) 2023-09-10T01:37:01.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d227fc700 1 -- 172.21.15.44:0/439884956 learned_addr learned my addr 172.21.15.44:0/439884956 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:01.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d22ffd700 1 --2- 172.21.15.44:0/439884956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d240ff4d0 0x7f2d24196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:01.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d237fe700 1 --2- 172.21.15.44:0/439884956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d24105590 0x7f2d2419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:01.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d227fc700 1 -- 172.21.15.44:0/439884956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d240ff4d0 msgr2=0x7f2d24196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:01.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d227fc700 1 --2- 172.21.15.44:0/439884956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d240ff4d0 0x7f2d24196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:01.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d227fc700 1 -- 172.21.15.44:0/439884956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d24105590 msgr2=0x7f2d2419a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:01.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d227fc700 1 --2- 172.21.15.44:0/439884956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d24105590 0x7f2d2419a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:01.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d227fc700 1 -- 172.21.15.44:0/439884956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d14009520 con 0x7f2d240ffe90 2023-09-10T01:37:01.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.748+0000 7f2d227fc700 1 --2- 172.21.15.44:0/439884956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d240ffe90 0x7f2d24196ac0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c005910 tx=0x7f2d0c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:01.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.749+0000 7f2d1bfff700 1 -- 172.21.15.44:0/439884956 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c01d070 con 0x7f2d240ffe90 2023-09-10T01:37:01.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.749+0000 7f2d1bfff700 1 -- 172.21.15.44:0/439884956 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d0c005050 con 0x7f2d240ffe90 2023-09-10T01:37:01.756 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.749+0000 7f2d29350700 1 -- 172.21.15.44:0/439884956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c0095f0 con 0x7f2d240ffe90 2023-09-10T01:37:01.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.749+0000 7f2d1bfff700 1 -- 172.21.15.44:0/439884956 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c004950 con 0x7f2d240ffe90 2023-09-10T01:37:01.757 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.749+0000 7f2d29350700 1 -- 172.21.15.44:0/439884956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2419ae50 con 0x7f2d240ffe90 2023-09-10T01:37:01.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.750+0000 7f2d29350700 1 -- 172.21.15.44:0/439884956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d240febf0 con 0x7f2d240ffe90 2023-09-10T01:37:01.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.751+0000 7f2d1bfff700 1 -- 172.21.15.44:0/439884956 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f2d0c0051c0 con 0x7f2d240ffe90 2023-09-10T01:37:01.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.752+0000 7f2d1bfff700 1 --2- 172.21.15.44:0/439884956 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2d100a07b0 0x7f2d100a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:01.758 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.752+0000 7f2d22ffd700 1 --2- 172.21.15.44:0/439884956 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2d100a07b0 0x7f2d100a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:01.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.752+0000 7f2d1bfff700 1 -- 172.21.15.44:0/439884956 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f2d0c0cd450 con 0x7f2d240ffe90 2023-09-10T01:37:01.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.752+0000 7f2d22ffd700 1 --2- 172.21.15.44:0/439884956 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2d100a07b0 0x7f2d100a2c70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009df0 tx=0x7f2d14009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:01.759 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.755+0000 7f2d1bfff700 1 -- 172.21.15.44:0/439884956 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2d0c0968d0 con 0x7f2d240ffe90 2023-09-10T01:37:01.986 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.984+0000 7f2d29350700 1 -- 172.21.15.44:0/439884956 --> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d2410f360 con 0x7f2d100a07b0 2023-09-10T01:37:01.995 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.994+0000 7f2d1bfff700 1 -- 172.21.15.44:0/439884956 <== mgr.15825 v2:172.21.15.44:6800/605958391 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f2d2410f360 con 0x7f2d100a07b0 2023-09-10T01:37:01.996 INFO:teuthology.orchestra.run.smithi044.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-09-10T00:00:00", "created": "2023-09-10T00:40:39", "first": "2023-09-10T00:41:00", "last": "2023-09-10T01:37:00", "last_pruned": "2023-09-10T01:37:01", "created_count": 57, "pruned_count": 50, "active": true} 2023-09-10T01:37:01.997 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.996+0000 7f2d29350700 1 -- 172.21.15.44:0/439884956 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2d100a07b0 msgr2=0x7f2d100a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:01.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.996+0000 7f2d29350700 1 --2- 172.21.15.44:0/439884956 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2d100a07b0 0x7f2d100a2c70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009df0 tx=0x7f2d14009260 comp rx=0 tx=0).stop 2023-09-10T01:37:01.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.996+0000 7f2d29350700 1 -- 172.21.15.44:0/439884956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d240ffe90 msgr2=0x7f2d24196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:01.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.996+0000 7f2d29350700 1 --2- 172.21.15.44:0/439884956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d240ffe90 0x7f2d24196ac0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c005910 tx=0x7f2d0c00f690 comp rx=0 tx=0).stop 2023-09-10T01:37:01.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.996+0000 7f2d29350700 1 -- 172.21.15.44:0/439884956 shutdown_connections 2023-09-10T01:37:01.998 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.996+0000 7f2d29350700 1 --2- 172.21.15.44:0/439884956 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f2d100a07b0 0x7f2d100a2c70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.996+0000 7f2d29350700 1 --2- 172.21.15.44:0/439884956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d24105590 0x7f2d2419a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.996+0000 7f2d29350700 1 --2- 172.21.15.44:0/439884956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d240ff4d0 0x7f2d24196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.996+0000 7f2d29350700 1 --2- 172.21.15.44:0/439884956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d240ffe90 0x7f2d24196ac0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.996+0000 7f2d29350700 1 -- 172.21.15.44:0/439884956 >> 172.21.15.44:0/439884956 conn(0x7f2d240faf60 msgr2=0x7f2d240fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.996+0000 7f2d29350700 1 -- 172.21.15.44:0/439884956 shutdown_connections 2023-09-10T01:37:02.000 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:01.996+0000 7f2d29350700 1 -- 172.21.15.44:0/439884956 wait complete. 2023-09-10T01:37:02.010 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-09-10T01:37:02.140 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.136+0000 7f069ce20700 1 -- 172.21.15.44:0/1901010629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06981055b0 msgr2=0x7f0698105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.136+0000 7f069ce20700 1 --2- 172.21.15.44:0/1901010629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06981055b0 0x7f0698105990 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f06880099c0 tx=0x7f0688009cd0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.136+0000 7f069ce20700 1 -- 172.21.15.44:0/1901010629 shutdown_connections 2023-09-10T01:37:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.136+0000 7f069ce20700 1 --2- 172.21.15.44:0/1901010629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0698100120 0x7f069810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.136+0000 7f069ce20700 1 --2- 172.21.15.44:0/1901010629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06981055b0 0x7f0698105990 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.136+0000 7f069ce20700 1 --2- 172.21.15.44:0/1901010629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06980ff760 0x7f06980ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.136+0000 7f069ce20700 1 -- 172.21.15.44:0/1901010629 >> 172.21.15.44:0/1901010629 conn(0x7f0698074f30 msgr2=0x7f0698075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.136+0000 7f069ce20700 1 -- 172.21.15.44:0/1901010629 shutdown_connections 2023-09-10T01:37:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.136+0000 7f069ce20700 1 -- 172.21.15.44:0/1901010629 wait complete. 2023-09-10T01:37:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.137+0000 7f069ce20700 1 Processor -- start 2023-09-10T01:37:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.137+0000 7f069ce20700 1 -- start start 2023-09-10T01:37:02.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.137+0000 7f069ce20700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06980ff760 0x7f069819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:02.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.137+0000 7f069ce20700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0698100120 0x7f069819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:02.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.137+0000 7f069ce20700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06981055b0 0x7f06981964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:02.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.137+0000 7f069ce20700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0698196ba0 con 0x7f0698100120 2023-09-10T01:37:02.143 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.137+0000 7f069ce20700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0698196d10 con 0x7f06981055b0 2023-09-10T01:37:02.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.137+0000 7f069ce20700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0698196ed0 con 0x7f06980ff760 2023-09-10T01:37:02.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.137+0000 7f068ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0698100120 0x7f069819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:02.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.137+0000 7f068ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0698100120 0x7f069819cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33292/0 (socket says 172.21.15.44:33292) 2023-09-10T01:37:02.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.137+0000 7f068ffff700 1 -- 172.21.15.44:0/3194699633 learned_addr learned my addr 172.21.15.44:0/3194699633 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:02.144 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.137+0000 7f069659c700 1 --2- 172.21.15.44:0/3194699633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06980ff760 0x7f069819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.138+0000 7f0696d9d700 1 --2- 172.21.15.44:0/3194699633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06981055b0 0x7f06981964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.138+0000 7f068ffff700 1 -- 172.21.15.44:0/3194699633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06980ff760 msgr2=0x7f069819c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.138+0000 7f068ffff700 1 --2- 172.21.15.44:0/3194699633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06980ff760 0x7f069819c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.138+0000 7f068ffff700 1 -- 172.21.15.44:0/3194699633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06981055b0 msgr2=0x7f06981964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.138+0000 7f068ffff700 1 --2- 172.21.15.44:0/3194699633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06981055b0 0x7f06981964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.138+0000 7f068ffff700 1 -- 172.21.15.44:0/3194699633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06880095f0 con 0x7f0698100120 2023-09-10T01:37:02.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.138+0000 7f068ffff700 1 --2- 172.21.15.44:0/3194699633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0698100120 0x7f069819cbb0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f068000d980 tx=0x7f068000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:02.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.138+0000 7f068f7fe700 1 -- 172.21.15.44:0/3194699633 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068000bb60 con 0x7f0698100120 2023-09-10T01:37:02.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.138+0000 7f068f7fe700 1 -- 172.21.15.44:0/3194699633 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0680007640 con 0x7f0698100120 2023-09-10T01:37:02.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.138+0000 7f069ce20700 1 -- 172.21.15.44:0/3194699633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06981971b0 con 0x7f0698100120 2023-09-10T01:37:02.146 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.138+0000 7f068f7fe700 1 -- 172.21.15.44:0/3194699633 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680004d60 con 0x7f0698100120 2023-09-10T01:37:02.147 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.138+0000 7f069ce20700 1 -- 172.21.15.44:0/3194699633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06981a36a0 con 0x7f0698100120 2023-09-10T01:37:02.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.140+0000 7f069ce20700 1 -- 172.21.15.44:0/3194699633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0698108a70 con 0x7f0698100120 2023-09-10T01:37:02.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.140+0000 7f068f7fe700 1 -- 172.21.15.44:0/3194699633 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f0680004020 con 0x7f0698100120 2023-09-10T01:37:02.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.141+0000 7f068f7fe700 1 --2- 172.21.15.44:0/3194699633 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f06840a07b0 0x7f06840a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:02.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.141+0000 7f068f7fe700 1 -- 172.21.15.44:0/3194699633 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f06800cc250 con 0x7f0698100120 2023-09-10T01:37:02.148 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.141+0000 7f069659c700 1 --2- 172.21.15.44:0/3194699633 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f06840a07b0 0x7f06840a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:02.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.141+0000 7f069659c700 1 --2- 172.21.15.44:0/3194699633 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f06840a07b0 0x7f06840a2c70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0688000c00 tx=0x7f068801a040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:02.149 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.146+0000 7f068f7fe700 1 -- 172.21.15.44:0/3194699633 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0680095640 con 0x7f0698100120 2023-09-10T01:37:02.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.379+0000 7f069ce20700 1 -- 172.21.15.44:0/3194699633 --> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0698197c30 con 0x7f06840a07b0 2023-09-10T01:37:02.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.388+0000 7f068f7fe700 1 -- 172.21.15.44:0/3194699633 <== mgr.15825 v2:172.21.15.44:6800/605958391 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f0698197c30 con 0x7f06840a07b0 2023-09-10T01:37:02.390 INFO:teuthology.orchestra.run.smithi044.stdout:/ 1M 6M3h 2023-09-10T01:37:02.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.390+0000 7f069ce20700 1 -- 172.21.15.44:0/3194699633 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f06840a07b0 msgr2=0x7f06840a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:02.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.390+0000 7f069ce20700 1 --2- 172.21.15.44:0/3194699633 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f06840a07b0 0x7f06840a2c70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0688000c00 tx=0x7f068801a040 comp rx=0 tx=0).stop 2023-09-10T01:37:02.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.390+0000 7f069ce20700 1 -- 172.21.15.44:0/3194699633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0698100120 msgr2=0x7f069819cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:02.392 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.390+0000 7f069ce20700 1 --2- 172.21.15.44:0/3194699633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0698100120 0x7f069819cbb0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f068000d980 tx=0x7f068000dd40 comp rx=0 tx=0).stop 2023-09-10T01:37:02.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.390+0000 7f069ce20700 1 -- 172.21.15.44:0/3194699633 shutdown_connections 2023-09-10T01:37:02.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.390+0000 7f069ce20700 1 --2- 172.21.15.44:0/3194699633 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f06840a07b0 0x7f06840a2c70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.390+0000 7f069ce20700 1 --2- 172.21.15.44:0/3194699633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06981055b0 0x7f06981964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.390+0000 7f069ce20700 1 --2- 172.21.15.44:0/3194699633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06980ff760 0x7f069819c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.393 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.390+0000 7f069ce20700 1 --2- 172.21.15.44:0/3194699633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0698100120 0x7f069819cbb0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.390+0000 7f069ce20700 1 -- 172.21.15.44:0/3194699633 >> 172.21.15.44:0/3194699633 conn(0x7f0698074f30 msgr2=0x7f0698067b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:02.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.390+0000 7f069ce20700 1 -- 172.21.15.44:0/3194699633 shutdown_connections 2023-09-10T01:37:02.394 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.390+0000 7f069ce20700 1 -- 172.21.15.44:0/3194699633 wait complete. 2023-09-10T01:37:02.404 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-09-10T01:37:02.415 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-09-10T01:37:02.415 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-09-10T01:37:02.415 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-09-10T01:37:02.425 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T01:37:02.570 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.568+0000 7f375318b700 1 -- 172.21.15.44:0/788069347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374c10ab40 msgr2=0x7f374c10cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:02.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.568+0000 7f375318b700 1 --2- 172.21.15.44:0/788069347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374c10ab40 0x7f374c10cfe0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f374400b0a0 tx=0x7f374400b3b0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.568+0000 7f375318b700 1 -- 172.21.15.44:0/788069347 shutdown_connections 2023-09-10T01:37:02.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.568+0000 7f375318b700 1 --2- 172.21.15.44:0/788069347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374c10ab40 0x7f374c10cfe0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.571 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.568+0000 7f375318b700 1 --2- 172.21.15.44:0/788069347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f374c101770 0x7f374c101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.568+0000 7f375318b700 1 --2- 172.21.15.44:0/788069347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f374c102090 0x7f374c10a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.568+0000 7f375318b700 1 -- 172.21.15.44:0/788069347 >> 172.21.15.44:0/788069347 conn(0x7f374c0faf10 msgr2=0x7f374c0fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:02.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.568+0000 7f375318b700 1 -- 172.21.15.44:0/788069347 shutdown_connections 2023-09-10T01:37:02.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.568+0000 7f375318b700 1 -- 172.21.15.44:0/788069347 wait complete. 2023-09-10T01:37:02.572 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.569+0000 7f375318b700 1 Processor -- start 2023-09-10T01:37:02.573 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.569+0000 7f375318b700 1 -- start start 2023-09-10T01:37:02.575 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.569+0000 7f375318b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f374c101770 0x7f374c196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:02.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.569+0000 7f375318b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374c102090 0x7f374c196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:02.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.569+0000 7f375318b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f374c10ab40 0x7f374c19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:02.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.569+0000 7f375318b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c19b500 con 0x7f374c102090 2023-09-10T01:37:02.576 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.569+0000 7f375318b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c19b670 con 0x7f374c10ab40 2023-09-10T01:37:02.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.569+0000 7f375318b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c19b830 con 0x7f374c101770 2023-09-10T01:37:02.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f374bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374c102090 0x7f374c196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:02.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f374bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374c102090 0x7f374c196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33304/0 (socket says 172.21.15.44:33304) 2023-09-10T01:37:02.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f374bfff700 1 -- 172.21.15.44:0/3355936679 learned_addr learned my addr 172.21.15.44:0/3355936679 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:02.577 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f3750f27700 1 --2- 172.21.15.44:0/3355936679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f374c101770 0x7f374c196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:02.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f3751728700 1 --2- 172.21.15.44:0/3355936679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f374c10ab40 0x7f374c19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:02.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f374bfff700 1 -- 172.21.15.44:0/3355936679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f374c101770 msgr2=0x7f374c196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:02.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f374bfff700 1 --2- 172.21.15.44:0/3355936679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f374c101770 0x7f374c196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f374bfff700 1 -- 172.21.15.44:0/3355936679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f374c10ab40 msgr2=0x7f374c19aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:02.578 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f374bfff700 1 --2- 172.21.15.44:0/3355936679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f374c10ab40 0x7f374c19aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f374bfff700 1 -- 172.21.15.44:0/3355936679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3744009d00 con 0x7f374c102090 2023-09-10T01:37:02.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f3751728700 1 --2- 172.21.15.44:0/3355936679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f374c10ab40 0x7f374c19aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:37:02.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f374bfff700 1 --2- 172.21.15.44:0/3355936679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374c102090 0x7f374c196a60 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f373800c9b0 tx=0x7f373800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:02.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f3749ffb700 1 -- 172.21.15.44:0/3355936679 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3738007b00 con 0x7f374c102090 2023-09-10T01:37:02.579 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f3749ffb700 1 -- 172.21.15.44:0/3355936679 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3738007c60 con 0x7f374c102090 2023-09-10T01:37:02.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f375318b700 1 -- 172.21.15.44:0/3355936679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374c19bb10 con 0x7f374c102090 2023-09-10T01:37:02.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f3749ffb700 1 -- 172.21.15.44:0/3355936679 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3738018660 con 0x7f374c102090 2023-09-10T01:37:02.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.570+0000 7f375318b700 1 -- 172.21.15.44:0/3355936679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374c1a3350 con 0x7f374c102090 2023-09-10T01:37:02.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.571+0000 7f375318b700 1 -- 172.21.15.44:0/3355936679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f374c1053d0 con 0x7f374c102090 2023-09-10T01:37:02.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.572+0000 7f3749ffb700 1 -- 172.21.15.44:0/3355936679 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f373800f450 con 0x7f374c102090 2023-09-10T01:37:02.580 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.573+0000 7f3749ffb700 1 --2- 172.21.15.44:0/3355936679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f373c0a07f0 0x7f373c0a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:02.581 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.573+0000 7f3749ffb700 1 -- 172.21.15.44:0/3355936679 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f37380cc5d0 con 0x7f374c102090 2023-09-10T01:37:02.581 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.573+0000 7f3750f27700 1 --2- 172.21.15.44:0/3355936679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f373c0a07f0 0x7f373c0a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:02.581 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.573+0000 7f3750f27700 1 --2- 172.21.15.44:0/3355936679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f373c0a07f0 0x7f373c0a2cb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f374000ac50 tx=0x7f374000a380 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:02.582 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.576+0000 7f3749ffb700 1 -- 172.21.15.44:0/3355936679 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3738095a50 con 0x7f374c102090 2023-09-10T01:37:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.782+0000 7f375318b700 1 -- 172.21.15.44:0/3355936679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f374c0673f0 con 0x7f374c102090 2023-09-10T01:37:02.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.783+0000 7f3749ffb700 1 -- 172.21.15.44:0/3355936679 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+25247 (secure 0 0 0) 0x7f373800fe60 con 0x7f374c102090 2023-09-10T01:37:02.786 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:02.786 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":211,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T01:36:26.754698+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":106,"snap_epoch":210,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":106,"snap_epoch":210,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/902389643":"2023-09-11T00:40:38.970014+0000","172.21.15.44:0/3642136197":"2023-09-11T00:40:38.970014+0000","172.21.15.44:0/3766354728":"2023-09-11T00:40:38.970014+0000","172.21.15.44:6801/1488156576":"2023-09-11T00:40:38.970014+0000","172.21.15.44:6800/1488156576":"2023-09-11T00:40:38.970014+0000","172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1968699888":"2023-09-11T00:40:38.970014+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":93,"length":1},{"begin":106,"length":1}]},{"pool":4,"snaps":[{"begin":93,"length":1},{"begin":106,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T01:37:02.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.786+0000 7f375318b700 1 -- 172.21.15.44:0/3355936679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f373c0a07f0 msgr2=0x7f373c0a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:02.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.786+0000 7f375318b700 1 --2- 172.21.15.44:0/3355936679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f373c0a07f0 0x7f373c0a2cb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f374000ac50 tx=0x7f374000a380 comp rx=0 tx=0).stop 2023-09-10T01:37:02.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.786+0000 7f375318b700 1 -- 172.21.15.44:0/3355936679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374c102090 msgr2=0x7f374c196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:02.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.786+0000 7f375318b700 1 --2- 172.21.15.44:0/3355936679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374c102090 0x7f374c196a60 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f373800c9b0 tx=0x7f373800ccc0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.788 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.786+0000 7f375318b700 1 -- 172.21.15.44:0/3355936679 shutdown_connections 2023-09-10T01:37:02.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.786+0000 7f375318b700 1 --2- 172.21.15.44:0/3355936679 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f373c0a07f0 0x7f373c0a2cb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.786+0000 7f375318b700 1 --2- 172.21.15.44:0/3355936679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f374c10ab40 0x7f374c19aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.786+0000 7f375318b700 1 --2- 172.21.15.44:0/3355936679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f374c101770 0x7f374c196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.786+0000 7f375318b700 1 --2- 172.21.15.44:0/3355936679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374c102090 0x7f374c196a60 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.789 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.786+0000 7f375318b700 1 -- 172.21.15.44:0/3355936679 >> 172.21.15.44:0/3355936679 conn(0x7f374c0faf10 msgr2=0x7f374c102510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:02.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.786+0000 7f375318b700 1 -- 172.21.15.44:0/3355936679 shutdown_connections 2023-09-10T01:37:02.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.786+0000 7f375318b700 1 -- 172.21.15.44:0/3355936679 wait complete. 2023-09-10T01:37:02.801 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-10T00:34:44.456741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-10T00:38:37.194891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-10T00:38:38.573268+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '210', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 106, 'snap_epoch': 210, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-10T00:38:42.198896+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '210', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 106, 'snap_epoch': 210, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-10T01:37:02.801 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-10T01:37:02.933 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.930+0000 7f9e61a54700 1 -- 172.21.15.44:0/4234356201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e5c0ff440 msgr2=0x7f9e5c0ff8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:02.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.930+0000 7f9e61a54700 1 --2- 172.21.15.44:0/4234356201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e5c0ff440 0x7f9e5c0ff8c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009990 tx=0x7f9e50009ca0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.930+0000 7f9e61a54700 1 -- 172.21.15.44:0/4234356201 shutdown_connections 2023-09-10T01:37:02.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.930+0000 7f9e61a54700 1 --2- 172.21.15.44:0/4234356201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5c0ffe00 0x7f9e5c10c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.930+0000 7f9e61a54700 1 --2- 172.21.15.44:0/4234356201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e5c1054e0 0x7f9e5c1058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.934 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.930+0000 7f9e61a54700 1 --2- 172.21.15.44:0/4234356201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e5c0ff440 0x7f9e5c0ff8c0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.930+0000 7f9e61a54700 1 -- 172.21.15.44:0/4234356201 >> 172.21.15.44:0/4234356201 conn(0x7f9e5c0faf50 msgr2=0x7f9e5c0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.930+0000 7f9e61a54700 1 -- 172.21.15.44:0/4234356201 shutdown_connections 2023-09-10T01:37:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.930+0000 7f9e61a54700 1 -- 172.21.15.44:0/4234356201 wait complete. 2023-09-10T01:37:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.930+0000 7f9e61a54700 1 Processor -- start 2023-09-10T01:37:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e61a54700 1 -- start start 2023-09-10T01:37:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e61a54700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e5c0ff440 0x7f9e5c10cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e61a54700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e5c0ffe00 0x7f9e5c10d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e61a54700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5c1054e0 0x7f9e5c1a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e61a54700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c10da50 con 0x7f9e5c0ffe00 2023-09-10T01:37:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e61a54700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c1a4cd0 con 0x7f9e5c1054e0 2023-09-10T01:37:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e61a54700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c1a4e10 con 0x7f9e5c0ff440 2023-09-10T01:37:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e5a7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e5c0ffe00 0x7f9e5c10d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e5a7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e5c0ffe00 0x7f9e5c10d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33318/0 (socket says 172.21.15.44:33318) 2023-09-10T01:37:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e5a7fc700 1 -- 172.21.15.44:0/2745183999 learned_addr learned my addr 172.21.15.44:0/2745183999 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e5b7fe700 1 --2- 172.21.15.44:0/2745183999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5c1054e0 0x7f9e5c1a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e5affd700 1 --2- 172.21.15.44:0/2745183999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e5c0ff440 0x7f9e5c10cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e5a7fc700 1 -- 172.21.15.44:0/2745183999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e5c0ff440 msgr2=0x7f9e5c10cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e5a7fc700 1 --2- 172.21.15.44:0/2745183999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e5c0ff440 0x7f9e5c10cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e5a7fc700 1 -- 172.21.15.44:0/2745183999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5c1054e0 msgr2=0x7f9e5c1a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e5a7fc700 1 --2- 172.21.15.44:0/2745183999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5c1054e0 0x7f9e5c1a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e5a7fc700 1 -- 172.21.15.44:0/2745183999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e500095f0 con 0x7f9e5c0ffe00 2023-09-10T01:37:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.931+0000 7f9e5a7fc700 1 --2- 172.21.15.44:0/2745183999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e5c0ffe00 0x7f9e5c10d220 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f9e50006010 tx=0x7f9e50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.932+0000 7f9e60a52700 1 -- 172.21.15.44:0/2745183999 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5001d070 con 0x7f9e5c0ffe00 2023-09-10T01:37:02.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.932+0000 7f9e60a52700 1 -- 172.21.15.44:0/2745183999 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e50004000 con 0x7f9e5c0ffe00 2023-09-10T01:37:02.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.932+0000 7f9e61a54700 1 -- 172.21.15.44:0/2745183999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5c1a5010 con 0x7f9e5c0ffe00 2023-09-10T01:37:02.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.932+0000 7f9e60a52700 1 -- 172.21.15.44:0/2745183999 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50022ac0 con 0x7f9e5c0ffe00 2023-09-10T01:37:02.939 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.932+0000 7f9e61a54700 1 -- 172.21.15.44:0/2745183999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5c1a5430 con 0x7f9e5c0ffe00 2023-09-10T01:37:02.940 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.932+0000 7f9e61a54700 1 -- 172.21.15.44:0/2745183999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5c067380 con 0x7f9e5c0ffe00 2023-09-10T01:37:02.940 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.934+0000 7f9e60a52700 1 -- 172.21.15.44:0/2745183999 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f9e50003b70 con 0x7f9e5c0ffe00 2023-09-10T01:37:02.940 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.934+0000 7f9e60a52700 1 --2- 172.21.15.44:0/2745183999 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9e440a07b0 0x7f9e440a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:02.941 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.935+0000 7f9e5affd700 1 --2- 172.21.15.44:0/2745183999 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9e440a07b0 0x7f9e440a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:02.941 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.935+0000 7f9e60a52700 1 -- 172.21.15.44:0/2745183999 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f9e500ce1d0 con 0x7f9e5c0ffe00 2023-09-10T01:37:02.941 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.935+0000 7f9e5affd700 1 --2- 172.21.15.44:0/2745183999 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9e440a07b0 0x7f9e440a2c70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c005fd0 tx=0x7f9e4c005db0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:02.941 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:02.938+0000 7f9e60a52700 1 -- 172.21.15.44:0/2745183999 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9e50097650 con 0x7f9e5c0ffe00 2023-09-10T01:37:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:02 smithi044 ceph-mon[98676]: pgmap v1837: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 181 KiB/s rd, 171 KiB/s wr, 32 op/s 2023-09-10T01:37:03.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:02 smithi044 ceph-mon[98676]: from='client.20799 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T01:37:03.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:02 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3355936679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T01:37:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:02 smithi099 ceph-mon[102692]: pgmap v1837: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 181 KiB/s rd, 171 KiB/s wr, 32 op/s 2023-09-10T01:37:03.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:02 smithi099 ceph-mon[102692]: from='client.20799 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T01:37:03.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:02 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3355936679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T01:37:03.142 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.140+0000 7f9e61a54700 1 -- 172.21.15.44:0/2745183999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9e5c110600 con 0x7f9e5c0ffe00 2023-09-10T01:37:03.151 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.150+0000 7f9e60a52700 1 -- 172.21.15.44:0/2745183999 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v211) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9e5009ac70 con 0x7f9e5c0ffe00 2023-09-10T01:37:03.152 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2023-09-10T01:37:03.154 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.152+0000 7f9e61a54700 1 -- 172.21.15.44:0/2745183999 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9e440a07b0 msgr2=0x7f9e440a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:03.154 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.152+0000 7f9e61a54700 1 --2- 172.21.15.44:0/2745183999 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9e440a07b0 0x7f9e440a2c70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c005fd0 tx=0x7f9e4c005db0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.154 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.152+0000 7f9e61a54700 1 -- 172.21.15.44:0/2745183999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e5c0ffe00 msgr2=0x7f9e5c10d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:03.154 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.152+0000 7f9e61a54700 1 --2- 172.21.15.44:0/2745183999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e5c0ffe00 0x7f9e5c10d220 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f9e50006010 tx=0x7f9e50004730 comp rx=0 tx=0).stop 2023-09-10T01:37:03.155 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.152+0000 7f9e61a54700 1 -- 172.21.15.44:0/2745183999 shutdown_connections 2023-09-10T01:37:03.155 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.152+0000 7f9e61a54700 1 --2- 172.21.15.44:0/2745183999 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9e440a07b0 0x7f9e440a2c70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.155 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.152+0000 7f9e61a54700 1 --2- 172.21.15.44:0/2745183999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5c1054e0 0x7f9e5c1a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.155 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.152+0000 7f9e61a54700 1 --2- 172.21.15.44:0/2745183999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e5c0ff440 0x7f9e5c10cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.155 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.152+0000 7f9e61a54700 1 --2- 172.21.15.44:0/2745183999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e5c0ffe00 0x7f9e5c10d220 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.156 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.152+0000 7f9e61a54700 1 -- 172.21.15.44:0/2745183999 >> 172.21.15.44:0/2745183999 conn(0x7f9e5c0faf50 msgr2=0x7f9e5c0fb690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:03.156 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.152+0000 7f9e61a54700 1 -- 172.21.15.44:0/2745183999 shutdown_connections 2023-09-10T01:37:03.156 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.152+0000 7f9e61a54700 1 -- 172.21.15.44:0/2745183999 wait complete. 2023-09-10T01:37:03.166 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-10T01:37:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:02 smithi131 ceph-mon[103713]: pgmap v1837: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 181 KiB/s rd, 171 KiB/s wr, 32 op/s 2023-09-10T01:37:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:02 smithi131 ceph-mon[103713]: from='client.20799 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T01:37:03.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:02 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3355936679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T01:37:03.298 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.294+0000 7f97f0046700 1 -- 172.21.15.44:0/2023709244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97e8101710 msgr2=0x7f97e8101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:03.298 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.294+0000 7f97f0046700 1 --2- 172.21.15.44:0/2023709244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97e8101710 0x7f97e8101af0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009990 tx=0x7f97e4009ca0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.294+0000 7f97f0046700 1 -- 172.21.15.44:0/2023709244 shutdown_connections 2023-09-10T01:37:03.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.294+0000 7f97f0046700 1 --2- 172.21.15.44:0/2023709244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97e810ab30 0x7f97e810cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.294+0000 7f97f0046700 1 --2- 172.21.15.44:0/2023709244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97e8101710 0x7f97e8101af0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.294+0000 7f97f0046700 1 --2- 172.21.15.44:0/2023709244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97e8102030 0x7f97e810a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.299 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.294+0000 7f97f0046700 1 -- 172.21.15.44:0/2023709244 >> 172.21.15.44:0/2023709244 conn(0x7f97e80faf50 msgr2=0x7f97e80fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:03.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.294+0000 7f97f0046700 1 -- 172.21.15.44:0/2023709244 shutdown_connections 2023-09-10T01:37:03.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.294+0000 7f97f0046700 1 -- 172.21.15.44:0/2023709244 wait complete. 2023-09-10T01:37:03.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.295+0000 7f97f0046700 1 Processor -- start 2023-09-10T01:37:03.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.295+0000 7f97f0046700 1 -- start start 2023-09-10T01:37:03.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.295+0000 7f97f0046700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97e8101710 0x7f97e80ff360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:03.300 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.295+0000 7f97f0046700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97e8102030 0x7f97e80ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:03.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.295+0000 7f97f0046700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97e810ab30 0x7f97e81a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:03.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.295+0000 7f97f0046700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e81000d0 con 0x7f97e8101710 2023-09-10T01:37:03.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.295+0000 7f97f0046700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e81a4d00 con 0x7f97e810ab30 2023-09-10T01:37:03.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.295+0000 7f97f0046700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e81a4e40 con 0x7f97e8102030 2023-09-10T01:37:03.301 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.295+0000 7f97edde2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97e8101710 0x7f97e80ff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:03.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.295+0000 7f97edde2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97e8101710 0x7f97e80ff360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33330/0 (socket says 172.21.15.44:33330) 2023-09-10T01:37:03.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.295+0000 7f97edde2700 1 -- 172.21.15.44:0/2211969514 learned_addr learned my addr 172.21.15.44:0/2211969514 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:03.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.296+0000 7f97ee5e3700 1 --2- 172.21.15.44:0/2211969514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97e810ab30 0x7f97e81a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:03.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.296+0000 7f97ed5e1700 1 --2- 172.21.15.44:0/2211969514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97e8102030 0x7f97e80ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:03.302 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.296+0000 7f97edde2700 1 -- 172.21.15.44:0/2211969514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97e8102030 msgr2=0x7f97e80ff8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:03.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.296+0000 7f97edde2700 1 --2- 172.21.15.44:0/2211969514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97e8102030 0x7f97e80ff8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.296+0000 7f97edde2700 1 -- 172.21.15.44:0/2211969514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97e810ab30 msgr2=0x7f97e81a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:03.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.296+0000 7f97edde2700 1 --2- 172.21.15.44:0/2211969514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97e810ab30 0x7f97e81a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.303 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.296+0000 7f97edde2700 1 -- 172.21.15.44:0/2211969514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e40095f0 con 0x7f97e8101710 2023-09-10T01:37:03.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.296+0000 7f97ed5e1700 1 --2- 172.21.15.44:0/2211969514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97e8102030 0x7f97e80ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:37:03.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.296+0000 7f97edde2700 1 --2- 172.21.15.44:0/2211969514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97e8101710 0x7f97e80ff360 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f97e4005910 tx=0x7f97e4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:03.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.296+0000 7f97deffd700 1 -- 172.21.15.44:0/2211969514 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e401d070 con 0x7f97e8101710 2023-09-10T01:37:03.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.296+0000 7f97deffd700 1 -- 172.21.15.44:0/2211969514 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f97e4003f40 con 0x7f97e8101710 2023-09-10T01:37:03.304 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.296+0000 7f97deffd700 1 -- 172.21.15.44:0/2211969514 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e400fc60 con 0x7f97e8101710 2023-09-10T01:37:03.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.296+0000 7f97f0046700 1 -- 172.21.15.44:0/2211969514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e81a5040 con 0x7f97e8101710 2023-09-10T01:37:03.305 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.296+0000 7f97f0046700 1 -- 172.21.15.44:0/2211969514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e81a5470 con 0x7f97e8101710 2023-09-10T01:37:03.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.297+0000 7f97f0046700 1 -- 172.21.15.44:0/2211969514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e8105c60 con 0x7f97e8101710 2023-09-10T01:37:03.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.298+0000 7f97deffd700 1 -- 172.21.15.44:0/2211969514 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f97e40040b0 con 0x7f97e8101710 2023-09-10T01:37:03.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.299+0000 7f97deffd700 1 --2- 172.21.15.44:0/2211969514 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f97d40a07b0 0x7f97d40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:03.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.299+0000 7f97ed5e1700 1 --2- 172.21.15.44:0/2211969514 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f97d40a07b0 0x7f97d40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:03.306 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.299+0000 7f97deffd700 1 -- 172.21.15.44:0/2211969514 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f97e40cd400 con 0x7f97e8101710 2023-09-10T01:37:03.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.299+0000 7f97ed5e1700 1 --2- 172.21.15.44:0/2211969514 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f97d40a07b0 0x7f97d40a2c70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f97e8100b50 tx=0x7f97d80092c0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:03.307 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.303+0000 7f97deffd700 1 -- 172.21.15.44:0/2211969514 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f97e4096820 con 0x7f97e8101710 2023-09-10T01:37:03.510 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.508+0000 7f97f0046700 1 -- 172.21.15.44:0/2211969514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97e8100940 con 0x7f97e8101710 2023-09-10T01:37:03.510 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.508+0000 7f97deffd700 1 -- 172.21.15.44:0/2211969514 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v211) v1 ==== 104+0+11 (secure 0 0 0) 0x7f97e4099e40 con 0x7f97e8101710 2023-09-10T01:37:03.511 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T01:37:03.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.510+0000 7f97f0046700 1 -- 172.21.15.44:0/2211969514 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f97d40a07b0 msgr2=0x7f97d40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:03.512 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.510+0000 7f97f0046700 1 --2- 172.21.15.44:0/2211969514 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f97d40a07b0 0x7f97d40a2c70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f97e8100b50 tx=0x7f97d80092c0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.510+0000 7f97f0046700 1 -- 172.21.15.44:0/2211969514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97e8101710 msgr2=0x7f97e80ff360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:03.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.510+0000 7f97f0046700 1 --2- 172.21.15.44:0/2211969514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97e8101710 0x7f97e80ff360 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f97e4005910 tx=0x7f97e4009580 comp rx=0 tx=0).stop 2023-09-10T01:37:03.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.510+0000 7f97f0046700 1 -- 172.21.15.44:0/2211969514 shutdown_connections 2023-09-10T01:37:03.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.510+0000 7f97f0046700 1 --2- 172.21.15.44:0/2211969514 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f97d40a07b0 0x7f97d40a2c70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.510+0000 7f97f0046700 1 --2- 172.21.15.44:0/2211969514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97e810ab30 0x7f97e81a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.510+0000 7f97f0046700 1 --2- 172.21.15.44:0/2211969514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97e8101710 0x7f97e80ff360 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.510+0000 7f97f0046700 1 --2- 172.21.15.44:0/2211969514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97e8102030 0x7f97e80ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.510+0000 7f97f0046700 1 -- 172.21.15.44:0/2211969514 >> 172.21.15.44:0/2211969514 conn(0x7f97e80faf50 msgr2=0x7f97e80fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:03.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.510+0000 7f97f0046700 1 -- 172.21.15.44:0/2211969514 shutdown_connections 2023-09-10T01:37:03.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.510+0000 7f97f0046700 1 -- 172.21.15.44:0/2211969514 wait complete. 2023-09-10T01:37:03.524 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-10T01:37:03.655 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.652+0000 7fbf44eff700 1 -- 172.21.15.44:0/1165240303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf40101770 msgr2=0x7fbf40101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:03.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.652+0000 7fbf44eff700 1 --2- 172.21.15.44:0/1165240303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf40101770 0x7fbf40101b50 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009990 tx=0x7fbf34009ca0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.652+0000 7fbf44eff700 1 -- 172.21.15.44:0/1165240303 shutdown_connections 2023-09-10T01:37:03.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.652+0000 7fbf44eff700 1 --2- 172.21.15.44:0/1165240303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf4010ab40 0x7fbf4010cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.656 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.652+0000 7fbf44eff700 1 --2- 172.21.15.44:0/1165240303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf40101770 0x7fbf40101b50 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.652+0000 7fbf44eff700 1 --2- 172.21.15.44:0/1165240303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf40102090 0x7fbf4010a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.652+0000 7fbf44eff700 1 -- 172.21.15.44:0/1165240303 >> 172.21.15.44:0/1165240303 conn(0x7fbf400faf50 msgr2=0x7fbf400fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:03.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.652+0000 7fbf44eff700 1 -- 172.21.15.44:0/1165240303 shutdown_connections 2023-09-10T01:37:03.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.652+0000 7fbf44eff700 1 -- 172.21.15.44:0/1165240303 wait complete. 2023-09-10T01:37:03.657 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.652+0000 7fbf44eff700 1 Processor -- start 2023-09-10T01:37:03.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf44eff700 1 -- start start 2023-09-10T01:37:03.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf44eff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf40101770 0x7fbf401965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:03.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf44eff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf40102090 0x7fbf40196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:03.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf44eff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf4010ab40 0x7fbf4019af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:03.658 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf44eff700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4019b590 con 0x7fbf40102090 2023-09-10T01:37:03.659 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf44eff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4019b700 con 0x7fbf4010ab40 2023-09-10T01:37:03.659 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf44eff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4019b8c0 con 0x7fbf40101770 2023-09-10T01:37:03.659 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf3dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf40102090 0x7fbf40196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:03.659 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf3dd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf40102090 0x7fbf40196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33346/0 (socket says 172.21.15.44:33346) 2023-09-10T01:37:03.659 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf3dd9b700 1 -- 172.21.15.44:0/2205250790 learned_addr learned my addr 172.21.15.44:0/2205250790 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:03.659 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf3e59c700 1 --2- 172.21.15.44:0/2205250790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf40101770 0x7fbf401965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:03.660 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf3ed9d700 1 --2- 172.21.15.44:0/2205250790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf4010ab40 0x7fbf4019af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:03.660 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf3dd9b700 1 -- 172.21.15.44:0/2205250790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf40101770 msgr2=0x7fbf401965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:03.660 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf3dd9b700 1 --2- 172.21.15.44:0/2205250790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf40101770 0x7fbf401965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.660 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf3dd9b700 1 -- 172.21.15.44:0/2205250790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf4010ab40 msgr2=0x7fbf4019af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:03.660 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf3dd9b700 1 --2- 172.21.15.44:0/2205250790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf4010ab40 0x7fbf4019af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf3dd9b700 1 -- 172.21.15.44:0/2205250790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf340095f0 con 0x7fbf40102090 2023-09-10T01:37:03.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf3dd9b700 1 --2- 172.21.15.44:0/2205250790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf40102090 0x7fbf40196af0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000d950 tx=0x7fbf3000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:03.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.653+0000 7fbf2f7fe700 1 -- 172.21.15.44:0/2205250790 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30009b40 con 0x7fbf40102090 2023-09-10T01:37:03.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.654+0000 7fbf44eff700 1 -- 172.21.15.44:0/2205250790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4019bba0 con 0x7fbf40102090 2023-09-10T01:37:03.661 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.654+0000 7fbf2f7fe700 1 -- 172.21.15.44:0/2205250790 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf30009ca0 con 0x7fbf40102090 2023-09-10T01:37:03.662 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.654+0000 7fbf2f7fe700 1 -- 172.21.15.44:0/2205250790 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3000f620 con 0x7fbf40102090 2023-09-10T01:37:03.662 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.654+0000 7fbf44eff700 1 -- 172.21.15.44:0/2205250790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf401a33e0 con 0x7fbf40102090 2023-09-10T01:37:03.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.655+0000 7fbf44eff700 1 -- 172.21.15.44:0/2205250790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf40105cc0 con 0x7fbf40102090 2023-09-10T01:37:03.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.656+0000 7fbf2f7fe700 1 -- 172.21.15.44:0/2205250790 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fbf30010460 con 0x7fbf40102090 2023-09-10T01:37:03.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.656+0000 7fbf2f7fe700 1 --2- 172.21.15.44:0/2205250790 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbf280a07b0 0x7fbf280a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:03.663 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.656+0000 7fbf3e59c700 1 --2- 172.21.15.44:0/2205250790 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbf280a07b0 0x7fbf280a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:03.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.656+0000 7fbf2f7fe700 1 -- 172.21.15.44:0/2205250790 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fbf300cc750 con 0x7fbf40102090 2023-09-10T01:37:03.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.657+0000 7fbf3e59c700 1 --2- 172.21.15.44:0/2205250790 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbf280a07b0 0x7fbf280a2c70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400b5c0 tx=0x7fbf34005fb0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:03.664 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.660+0000 7fbf2f7fe700 1 -- 172.21.15.44:0/2205250790 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbf30095b40 con 0x7fbf40102090 2023-09-10T01:37:03.884 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.882+0000 7fbf44eff700 1 -- 172.21.15.44:0/2205250790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbf40110330 con 0x7fbf40102090 2023-09-10T01:37:03.893 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.892+0000 7fbf2f7fe700 1 -- 172.21.15.44:0/2205250790 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v212) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbf30099160 con 0x7fbf40102090 2023-09-10T01:37:03.895 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2023-09-10T01:37:03.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.898+0000 7fbf44eff700 1 -- 172.21.15.44:0/2205250790 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbf280a07b0 msgr2=0x7fbf280a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:03.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.898+0000 7fbf44eff700 1 --2- 172.21.15.44:0/2205250790 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbf280a07b0 0x7fbf280a2c70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400b5c0 tx=0x7fbf34005fb0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.935 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.898+0000 7fbf44eff700 1 -- 172.21.15.44:0/2205250790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf40102090 msgr2=0x7fbf40196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:03.936 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.898+0000 7fbf44eff700 1 --2- 172.21.15.44:0/2205250790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf40102090 0x7fbf40196af0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000d950 tx=0x7fbf3000dd10 comp rx=0 tx=0).stop 2023-09-10T01:37:03.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.911+0000 7fbf44eff700 1 -- 172.21.15.44:0/2205250790 shutdown_connections 2023-09-10T01:37:03.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.911+0000 7fbf44eff700 1 --2- 172.21.15.44:0/2205250790 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fbf280a07b0 0x7fbf280a2c70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.937 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.911+0000 7fbf44eff700 1 --2- 172.21.15.44:0/2205250790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf4010ab40 0x7fbf4019af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.911+0000 7fbf44eff700 1 --2- 172.21.15.44:0/2205250790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf40101770 0x7fbf401965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.911+0000 7fbf44eff700 1 --2- 172.21.15.44:0/2205250790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf40102090 0x7fbf40196af0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:03.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.911+0000 7fbf44eff700 1 -- 172.21.15.44:0/2205250790 >> 172.21.15.44:0/2205250790 conn(0x7fbf400faf50 msgr2=0x7fbf400fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:03.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.927+0000 7fbf44eff700 1 -- 172.21.15.44:0/2205250790 shutdown_connections 2023-09-10T01:37:03.940 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:03.933+0000 7fbf44eff700 1 -- 172.21.15.44:0/2205250790 wait complete. 2023-09-10T01:37:03.985 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-10T01:37:04.105 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:03 smithi044 ceph-mon[98676]: from='client.20805 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T01:37:04.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:03 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2745183999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T01:37:04.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:03 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2211969514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T01:37:04.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:03 smithi099 ceph-mon[102692]: from='client.20805 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T01:37:04.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:03 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2745183999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T01:37:04.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:03 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2211969514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T01:37:04.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:03 smithi131 ceph-mon[103713]: from='client.20805 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-10T01:37:04.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:03 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2745183999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T01:37:04.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:03 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2211969514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T01:37:04.345 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.338+0000 7fd5cb6c5700 1 -- 172.21.15.44:0/3345807890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c410c180 msgr2=0x7fd5c4078880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:04.346 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.338+0000 7fd5cb6c5700 1 --2- 172.21.15.44:0/3345807890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c410c180 0x7fd5c4078880 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400b970 tx=0x7fd5b400bc80 comp rx=0 tx=0).stop 2023-09-10T01:37:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.347+0000 7fd5cb6c5700 1 -- 172.21.15.44:0/3345807890 shutdown_connections 2023-09-10T01:37:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.347+0000 7fd5cb6c5700 1 --2- 172.21.15.44:0/3345807890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c410c180 0x7fd5c4078880 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.347+0000 7fd5cb6c5700 1 --2- 172.21.15.44:0/3345807890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c410cb10 0x7fd5c410cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.347+0000 7fd5cb6c5700 1 --2- 172.21.15.44:0/3345807890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5c410b7c0 0x7fd5c410bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.347+0000 7fd5cb6c5700 1 -- 172.21.15.44:0/3345807890 >> 172.21.15.44:0/3345807890 conn(0x7fd5c406c8c0 msgr2=0x7fd5c406ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:04.355 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.351+0000 7fd5cb6c5700 1 -- 172.21.15.44:0/3345807890 shutdown_connections 2023-09-10T01:37:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.359+0000 7fd5cb6c5700 1 -- 172.21.15.44:0/3345807890 wait complete. 2023-09-10T01:37:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.359+0000 7fd5cb6c5700 1 Processor -- start 2023-09-10T01:37:04.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.369+0000 7fd5cb6c5700 1 -- start start 2023-09-10T01:37:04.371 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.370+0000 7fd5cb6c5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5c410b7c0 0x7fd5c407cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:04.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.370+0000 7fd5cb6c5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c410c180 0x7fd5c407d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:04.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.370+0000 7fd5cb6c5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c410cb10 0x7fd5c4082820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:04.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.370+0000 7fd5cb6c5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c4082ff0 con 0x7fd5c410c180 2023-09-10T01:37:04.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.370+0000 7fd5cb6c5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c4083160 con 0x7fd5c410cb10 2023-09-10T01:37:04.372 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.370+0000 7fd5cb6c5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c4083320 con 0x7fd5c410b7c0 2023-09-10T01:37:04.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.376+0000 7fd5c8c60700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c410c180 0x7fd5c407d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:04.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.376+0000 7fd5c8c60700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c410c180 0x7fd5c407d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33362/0 (socket says 172.21.15.44:33362) 2023-09-10T01:37:04.378 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.376+0000 7fd5c8c60700 1 -- 172.21.15.44:0/1584345370 learned_addr learned my addr 172.21.15.44:0/1584345370 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:04.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.376+0000 7fd5c9461700 1 --2- 172.21.15.44:0/1584345370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5c410b7c0 0x7fd5c407cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:04.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.376+0000 7fd5c8c60700 1 -- 172.21.15.44:0/1584345370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5c410b7c0 msgr2=0x7fd5c407cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:04.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.376+0000 7fd5c8c60700 1 --2- 172.21.15.44:0/1584345370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5c410b7c0 0x7fd5c407cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:04.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.376+0000 7fd5c8c60700 1 -- 172.21.15.44:0/1584345370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c410cb10 msgr2=0x7fd5c4082820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:04.379 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.376+0000 7fd5c8c60700 1 --2- 172.21.15.44:0/1584345370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c410cb10 0x7fd5c4082820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:04.380 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.377+0000 7fd5c8c60700 1 -- 172.21.15.44:0/1584345370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b400b5a0 con 0x7fd5c410c180 2023-09-10T01:37:04.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.377+0000 7fd5c8c60700 1 --2- 172.21.15.44:0/1584345370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c410c180 0x7fd5c407d4f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00078a0 tx=0x7fd5c0007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:04.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.377+0000 7fd5ba7fc700 1 -- 172.21.15.44:0/1584345370 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c000f040 con 0x7fd5c410c180 2023-09-10T01:37:04.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.377+0000 7fd5ba7fc700 1 -- 172.21.15.44:0/1584345370 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5c000de50 con 0x7fd5c410c180 2023-09-10T01:37:04.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.377+0000 7fd5ba7fc700 1 -- 172.21.15.44:0/1584345370 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0013e30 con 0x7fd5c410c180 2023-09-10T01:37:04.381 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.377+0000 7fd5cb6c5700 1 -- 172.21.15.44:0/1584345370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c4083600 con 0x7fd5c410c180 2023-09-10T01:37:04.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.377+0000 7fd5cb6c5700 1 -- 172.21.15.44:0/1584345370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c41c31b0 con 0x7fd5c410c180 2023-09-10T01:37:04.382 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.379+0000 7fd5cb6c5700 1 -- 172.21.15.44:0/1584345370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c4073d70 con 0x7fd5c410c180 2023-09-10T01:37:04.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.382+0000 7fd5ba7fc700 1 -- 172.21.15.44:0/1584345370 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fd5c000d9c0 con 0x7fd5c410c180 2023-09-10T01:37:04.384 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.383+0000 7fd5ba7fc700 1 --2- 172.21.15.44:0/1584345370 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd5b00a08b0 0x7fd5b00a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:04.385 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.383+0000 7fd5ba7fc700 1 -- 172.21.15.44:0/1584345370 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 9628+0+0 (secure 0 0 0) 0x7fd5c00cbc30 con 0x7fd5c410c180 2023-09-10T01:37:04.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.385+0000 7fd5c9461700 1 --2- 172.21.15.44:0/1584345370 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd5b00a08b0 0x7fd5b00a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:04.388 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.386+0000 7fd5c9461700 1 --2- 172.21.15.44:0/1584345370 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd5b00a08b0 0x7fd5b00a2d70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc009bd0 tx=0x7fd5bc009040 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.398+0000 7fd5ba7fc700 1 -- 172.21.15.44:0/1584345370 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd5c0094f70 con 0x7fd5c410c180 2023-09-10T01:37:04.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.783+0000 7fd5cb6c5700 1 -- 172.21.15.44:0/1584345370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd5c4067380 con 0x7fd5c410c180 2023-09-10T01:37:04.786 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.785+0000 7fd5ba7fc700 1 -- 172.21.15.44:0/1584345370 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v212) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd5c0017020 con 0x7fd5c410c180 2023-09-10T01:37:04.791 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T01:37:04.813 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.804+0000 7fd5cb6c5700 1 -- 172.21.15.44:0/1584345370 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd5b00a08b0 msgr2=0x7fd5b00a2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:04.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.804+0000 7fd5cb6c5700 1 --2- 172.21.15.44:0/1584345370 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd5b00a08b0 0x7fd5b00a2d70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc009bd0 tx=0x7fd5bc009040 comp rx=0 tx=0).stop 2023-09-10T01:37:04.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.804+0000 7fd5cb6c5700 1 -- 172.21.15.44:0/1584345370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c410c180 msgr2=0x7fd5c407d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:04.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.804+0000 7fd5cb6c5700 1 --2- 172.21.15.44:0/1584345370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c410c180 0x7fd5c407d4f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00078a0 tx=0x7fd5c0007c60 comp rx=0 tx=0).stop 2023-09-10T01:37:04.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.808+0000 7fd5cb6c5700 1 -- 172.21.15.44:0/1584345370 shutdown_connections 2023-09-10T01:37:04.814 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.808+0000 7fd5cb6c5700 1 --2- 172.21.15.44:0/1584345370 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd5b00a08b0 0x7fd5b00a2d70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:04.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.808+0000 7fd5cb6c5700 1 --2- 172.21.15.44:0/1584345370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c410cb10 0x7fd5c4082820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:04.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.808+0000 7fd5cb6c5700 1 --2- 172.21.15.44:0/1584345370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5c410b7c0 0x7fd5c407cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:04.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.808+0000 7fd5cb6c5700 1 --2- 172.21.15.44:0/1584345370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c410c180 0x7fd5c407d4f0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:04.815 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.808+0000 7fd5cb6c5700 1 -- 172.21.15.44:0/1584345370 >> 172.21.15.44:0/1584345370 conn(0x7fd5c406c8c0 msgr2=0x7fd5c4109590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:04.819 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.818+0000 7fd5cb6c5700 1 -- 172.21.15.44:0/1584345370 shutdown_connections 2023-09-10T01:37:04.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:04.821+0000 7fd5cb6c5700 1 -- 172.21.15.44:0/1584345370 wait complete. 2023-09-10T01:37:04.865 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-09-10T01:37:05.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.177+0000 7f46aeb86700 1 -- 172.21.15.44:0/3230099286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a810f160 msgr2=0x7f46a8107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:05.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.177+0000 7f46aeb86700 1 --2- 172.21.15.44:0/3230099286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a810f160 0x7f46a8107b40 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f46980098f0 tx=0x7f4698009c00 comp rx=0 tx=0).stop 2023-09-10T01:37:05.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.186+0000 7f46aeb86700 1 -- 172.21.15.44:0/3230099286 shutdown_connections 2023-09-10T01:37:05.199 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.186+0000 7f46aeb86700 1 --2- 172.21.15.44:0/3230099286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46a81081b0 0x7f46a811be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:05.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.186+0000 7f46aeb86700 1 --2- 172.21.15.44:0/3230099286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46a810ed80 0x7f46a8107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:05.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.186+0000 7f46aeb86700 1 --2- 172.21.15.44:0/3230099286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a810f160 0x7f46a8107b40 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:05.200 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.186+0000 7f46aeb86700 1 -- 172.21.15.44:0/3230099286 >> 172.21.15.44:0/3230099286 conn(0x7f46a806b640 msgr2=0x7f46a806ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:05.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.198+0000 7f46aeb86700 1 -- 172.21.15.44:0/3230099286 shutdown_connections 2023-09-10T01:37:05.205 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.204+0000 7f46aeb86700 1 -- 172.21.15.44:0/3230099286 wait complete. 2023-09-10T01:37:05.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.205+0000 7f46aeb86700 1 Processor -- start 2023-09-10T01:37:05.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.209+0000 7f46aeb86700 1 -- start start 2023-09-10T01:37:05.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.209+0000 7f46aeb86700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a81081b0 0x7f46a8117600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:05.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.209+0000 7f46aeb86700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46a810ed80 0x7f46a8117b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:05.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.209+0000 7f46aeb86700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46a810f160 0x7f46a81a8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:05.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.209+0000 7f46aeb86700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a81a8fb0 con 0x7f46a81081b0 2023-09-10T01:37:05.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.209+0000 7f46aeb86700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a81a90f0 con 0x7f46a810f160 2023-09-10T01:37:05.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.209+0000 7f46aeb86700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a81a9280 con 0x7f46a810ed80 2023-09-10T01:37:05.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.212+0000 7f46ac922700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a81081b0 0x7f46a8117600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:05.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.212+0000 7f46ac922700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a81081b0 0x7f46a8117600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33374/0 (socket says 172.21.15.44:33374) 2023-09-10T01:37:05.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.212+0000 7f46ac922700 1 -- 172.21.15.44:0/303873822 learned_addr learned my addr 172.21.15.44:0/303873822 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:05.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.212+0000 7f46ac922700 1 -- 172.21.15.44:0/303873822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46a810ed80 msgr2=0x7f46a8117b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:05.215 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.212+0000 7f46ac922700 1 --2- 172.21.15.44:0/303873822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46a810ed80 0x7f46a8117b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:05.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.212+0000 7f46ac922700 1 -- 172.21.15.44:0/303873822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46a810f160 msgr2=0x7f46a81a8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:05.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.212+0000 7f46ac922700 1 --2- 172.21.15.44:0/303873822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46a810f160 0x7f46a81a8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:05.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.213+0000 7f46ac922700 1 -- 172.21.15.44:0/303873822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4698009520 con 0x7f46a81081b0 2023-09-10T01:37:05.216 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.213+0000 7f46ac922700 1 --2- 172.21.15.44:0/303873822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a81081b0 0x7f46a8117600 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f46a000ba50 tx=0x7f46a000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.215+0000 7f46a5ffb700 1 -- 172.21.15.44:0/303873822 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a000cab0 con 0x7f46a81081b0 2023-09-10T01:37:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.215+0000 7f46a5ffb700 1 -- 172.21.15.44:0/303873822 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46a000cc10 con 0x7f46a81081b0 2023-09-10T01:37:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.215+0000 7f46a5ffb700 1 -- 172.21.15.44:0/303873822 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a00075b0 con 0x7f46a81081b0 2023-09-10T01:37:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.215+0000 7f46aeb86700 1 -- 172.21.15.44:0/303873822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a81a9560 con 0x7f46a81081b0 2023-09-10T01:37:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.216+0000 7f46aeb86700 1 -- 172.21.15.44:0/303873822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a81a9a20 con 0x7f46a81081b0 2023-09-10T01:37:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.218+0000 7f46aeb86700 1 -- 172.21.15.44:0/303873822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4694005320 con 0x7f46a81081b0 2023-09-10T01:37:05.232 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.230+0000 7f46a5ffb700 1 -- 172.21.15.44:0/303873822 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f46a0014440 con 0x7f46a81081b0 2023-09-10T01:37:05.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.230+0000 7f46a5ffb700 1 --2- 172.21.15.44:0/303873822 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f46900a0990 0x7f46900a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:05.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.231+0000 7f46a5ffb700 1 -- 172.21.15.44:0/303873822 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 9628+0+0 (secure 0 0 0) 0x7f46a00cbd50 con 0x7f46a81081b0 2023-09-10T01:37:05.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.231+0000 7f46a7fff700 1 --2- 172.21.15.44:0/303873822 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f46900a0990 0x7f46900a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:05.233 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.231+0000 7f46a5ffb700 1 -- 172.21.15.44:0/303873822 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f46a00cc210 con 0x7f46a81081b0 2023-09-10T01:37:05.234 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.231+0000 7f46a7fff700 1 --2- 172.21.15.44:0/303873822 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f46900a0990 0x7f46900a2e50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4698009810 tx=0x7f46980058e0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:05.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:04 smithi131 ceph-mon[103713]: pgmap v1838: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 249 KiB/s rd, 256 KiB/s wr, 42 op/s 2023-09-10T01:37:05.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:04 smithi131 ceph-mon[103713]: osdmap e212: 12 total, 12 up, 12 in 2023-09-10T01:37:05.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:04 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2205250790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T01:37:05.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:04 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1584345370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T01:37:05.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:04 smithi044 ceph-mon[98676]: pgmap v1838: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 249 KiB/s rd, 256 KiB/s wr, 42 op/s 2023-09-10T01:37:05.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:04 smithi044 ceph-mon[98676]: osdmap e212: 12 total, 12 up, 12 in 2023-09-10T01:37:05.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:04 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2205250790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T01:37:05.307 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:04 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1584345370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T01:37:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:04 smithi099 ceph-mon[102692]: pgmap v1838: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 249 KiB/s rd, 256 KiB/s wr, 42 op/s 2023-09-10T01:37:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:04 smithi099 ceph-mon[102692]: osdmap e212: 12 total, 12 up, 12 in 2023-09-10T01:37:05.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:04 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2205250790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T01:37:05.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:04 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1584345370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T01:37:05.747 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.746+0000 7f46aeb86700 1 -- 172.21.15.44:0/303873822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f4694005190 con 0x7f46a81081b0 2023-09-10T01:37:05.780 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.778+0000 7f46a5ffb700 1 -- 172.21.15.44:0/303873822 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+730421 (secure 0 0 0) 0x7f46a0019070 con 0x7f46a81081b0 2023-09-10T01:37:05.786 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:05.786 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T01:37:05.786 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 26, 2023-09-10T01:37:05.786 INFO:teuthology.orchestra.run.smithi044.stdout: "active_gid": 15825, 2023-09-10T01:37:05.786 INFO:teuthology.orchestra.run.smithi044.stdout: "active_name": "x", 2023-09-10T01:37:05.787 INFO:teuthology.orchestra.run.smithi044.stdout: "active_addrs": { 2023-09-10T01:37:05.787 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2023-09-10T01:37:05.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:05.787 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2023-09-10T01:37:05.787 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6800", 2023-09-10T01:37:05.788 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 605958391 2023-09-10T01:37:05.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:05.788 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2023-09-10T01:37:05.788 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6801", 2023-09-10T01:37:05.789 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 605958391 2023-09-10T01:37:05.789 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:05.789 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2023-09-10T01:37:05.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.789 INFO:teuthology.orchestra.run.smithi044.stdout: "active_addr": "172.21.15.44:6801/605958391", 2023-09-10T01:37:05.790 INFO:teuthology.orchestra.run.smithi044.stdout: "active_change": "2023-09-10T00:40:38.970265+0000", 2023-09-10T01:37:05.790 INFO:teuthology.orchestra.run.smithi044.stdout: "active_mgr_features": 4540138322906710015, 2023-09-10T01:37:05.790 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2023-09-10T01:37:05.790 INFO:teuthology.orchestra.run.smithi044.stdout: "standbys": [ 2023-09-10T01:37:05.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:05.791 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 25429, 2023-09-10T01:37:05.791 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "z", 2023-09-10T01:37:05.791 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_features": 4540138322906710015, 2023-09-10T01:37:05.791 INFO:teuthology.orchestra.run.smithi044.stdout: "available_modules": [ 2023-09-10T01:37:05.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:05.791 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "alerts", 2023-09-10T01:37:05.792 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:05.792 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:05.792 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:05.792 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2023-09-10T01:37:05.792 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2023-09-10T01:37:05.793 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:05.793 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.793 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.793 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2023-09-10T01:37:05.793 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.793 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.794 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.794 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How frequently to reexamine health status", 2023-09-10T01:37:05.794 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.794 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.794 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.795 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:05.795 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:05.795 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.795 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.796 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.796 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:05.796 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.796 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.796 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:05.796 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:05.797 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:05.797 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:05.797 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:05.797 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:05.797 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:05.798 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:05.798 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:05.798 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.798 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.798 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.799 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:05.799 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:05.799 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.799 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.800 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.800 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:05.800 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.800 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.800 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.800 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:05.801 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.801 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.801 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.801 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:05.802 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:05.802 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.802 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.802 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.802 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:05.803 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.803 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.803 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:05.803 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:05.803 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:05.804 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:05.804 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:05.804 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:05.804 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:05.804 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:05.805 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:05.805 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.805 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.805 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.805 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:05.806 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:05.806 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.806 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.806 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.806 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:05.807 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.807 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.807 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.807 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:05.807 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.807 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.808 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.808 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.808 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_destination": { 2023-09-10T01:37:05.808 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_destination", 2023-09-10T01:37:05.808 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.809 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.809 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.809 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:05.809 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.809 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.810 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.810 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email address to send alerts to", 2023-09-10T01:37:05.810 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.810 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.810 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.811 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_from_name": { 2023-09-10T01:37:05.811 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_from_name", 2023-09-10T01:37:05.811 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.811 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.811 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.812 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "Ceph", 2023-09-10T01:37:05.812 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.812 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.812 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.812 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email From: name", 2023-09-10T01:37:05.812 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.813 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.813 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.813 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_host": { 2023-09-10T01:37:05.813 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_host", 2023-09-10T01:37:05.814 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.814 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.814 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.814 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:05.814 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.815 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.815 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.815 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP server", 2023-09-10T01:37:05.815 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.815 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.815 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.816 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_password": { 2023-09-10T01:37:05.816 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_password", 2023-09-10T01:37:05.816 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.816 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.817 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.817 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:05.817 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.817 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.817 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.817 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Password to authenticate with", 2023-09-10T01:37:05.818 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.818 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.818 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.818 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_port": { 2023-09-10T01:37:05.819 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_port", 2023-09-10T01:37:05.819 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:05.819 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.819 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.819 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "465", 2023-09-10T01:37:05.819 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.820 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.820 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.820 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP port", 2023-09-10T01:37:05.820 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.820 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.820 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.821 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_sender": { 2023-09-10T01:37:05.821 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_sender", 2023-09-10T01:37:05.821 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.821 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.822 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.822 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:05.822 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.822 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.822 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.822 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP envelope sender", 2023-09-10T01:37:05.823 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.823 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.823 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.823 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_ssl": { 2023-09-10T01:37:05.824 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_ssl", 2023-09-10T01:37:05.824 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.824 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.824 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.824 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:05.825 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.825 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.825 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.825 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-10T01:37:05.825 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.826 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.826 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.826 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_user": { 2023-09-10T01:37:05.826 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_user", 2023-09-10T01:37:05.826 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.827 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.827 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.827 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:05.827 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.827 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.828 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.828 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "User to authenticate as", 2023-09-10T01:37:05.828 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.828 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.828 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.829 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:05.829 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:05.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:05.829 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "balancer", 2023-09-10T01:37:05.829 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:05.830 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:05.830 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:05.830 INFO:teuthology.orchestra.run.smithi044.stdout: "active": { 2023-09-10T01:37:05.830 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "active", 2023-09-10T01:37:05.830 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.831 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.831 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.831 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:05.831 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.831 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.832 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.832 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically balance PGs across cluster", 2023-09-10T01:37:05.832 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.832 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.832 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.832 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.833 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_time": { 2023-09-10T01:37:05.833 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_time", 2023-09-10T01:37:05.833 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.833 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.833 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.834 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0000", 2023-09-10T01:37:05.834 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.834 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.834 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.834 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "beginning time of day to automatically balance", 2023-09-10T01:37:05.835 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-10T01:37:05.835 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.835 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.835 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_weekday": { 2023-09-10T01:37:05.835 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_weekday", 2023-09-10T01:37:05.836 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2023-09-10T01:37:05.836 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.836 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.836 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2023-09-10T01:37:05.836 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2023-09-10T01:37:05.837 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2023-09-10T01:37:05.837 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.837 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-10T01:37:05.837 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-10T01:37:05.837 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.837 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.838 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_max_iterations": { 2023-09-10T01:37:05.838 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_max_iterations", 2023-09-10T01:37:05.838 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2023-09-10T01:37:05.838 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.839 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.839 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "25", 2023-09-10T01:37:05.839 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2023-09-10T01:37:05.839 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "250", 2023-09-10T01:37:05.839 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.839 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-10T01:37:05.840 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.840 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.840 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.840 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_metrics": { 2023-09-10T01:37:05.840 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_metrics", 2023-09-10T01:37:05.841 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.841 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.841 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.841 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "pgs,objects,bytes", 2023-09-10T01:37:05.841 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.841 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.842 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.842 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-10T01:37:05.842 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-10T01:37:05.842 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.843 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.843 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_step": { 2023-09-10T01:37:05.844 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_step", 2023-09-10T01:37:05.844 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:05.844 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.844 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.844 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.5", 2023-09-10T01:37:05.845 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0.001", 2023-09-10T01:37:05.845 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "0.999", 2023-09-10T01:37:05.845 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.845 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "aggressiveness of optimization", 2023-09-10T01:37:05.845 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-10T01:37:05.845 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.846 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.846 INFO:teuthology.orchestra.run.smithi044.stdout: "end_time": { 2023-09-10T01:37:05.846 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_time", 2023-09-10T01:37:05.846 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.847 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.847 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.847 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2359", 2023-09-10T01:37:05.847 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.847 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.847 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.848 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ending time of day to automatically balance", 2023-09-10T01:37:05.848 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-10T01:37:05.848 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.848 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.849 INFO:teuthology.orchestra.run.smithi044.stdout: "end_weekday": { 2023-09-10T01:37:05.849 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_weekday", 2023-09-10T01:37:05.849 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2023-09-10T01:37:05.849 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.849 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.849 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2023-09-10T01:37:05.850 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2023-09-10T01:37:05.850 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2023-09-10T01:37:05.850 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.850 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-10T01:37:05.850 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-10T01:37:05.851 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.851 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.851 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.851 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:05.851 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:05.852 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.852 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.852 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.852 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:05.852 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.852 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.853 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:05.853 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:05.853 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:05.853 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:05.853 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:05.853 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:05.854 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:05.854 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:05.854 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:05.854 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.854 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.855 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.855 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:05.855 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:05.855 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.856 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.856 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.856 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:05.856 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.856 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.857 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.857 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:05.857 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.857 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.857 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.858 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:05.858 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:05.858 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.858 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.858 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.859 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:05.859 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.859 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.859 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:05.859 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:05.860 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:05.860 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:05.860 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:05.860 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:05.860 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:05.860 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:05.861 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:05.861 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.861 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.861 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.861 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:05.862 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:05.862 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.862 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.862 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.862 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:05.863 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.863 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.863 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.863 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:05.863 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.863 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.864 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.864 INFO:teuthology.orchestra.run.smithi044.stdout: "min_score": { 2023-09-10T01:37:05.864 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_score", 2023-09-10T01:37:05.864 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:05.865 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.865 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.865 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2023-09-10T01:37:05.865 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.865 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.865 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.866 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-10T01:37:05.866 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.866 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.866 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.867 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2023-09-10T01:37:05.867 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2023-09-10T01:37:05.867 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.867 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.867 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.867 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "upmap", 2023-09-10T01:37:05.868 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.868 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.868 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:05.868 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-compat", 2023-09-10T01:37:05.868 INFO:teuthology.orchestra.run.smithi044.stdout: "none", 2023-09-10T01:37:05.868 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap" 2023-09-10T01:37:05.869 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:05.869 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Balancer mode", 2023-09-10T01:37:05.869 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.869 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.869 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.870 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_ids": { 2023-09-10T01:37:05.870 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_ids", 2023-09-10T01:37:05.870 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.870 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.870 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.871 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:05.871 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.871 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.871 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.871 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-10T01:37:05.872 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.872 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.872 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.872 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2023-09-10T01:37:05.872 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2023-09-10T01:37:05.873 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:05.873 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.873 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.873 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2023-09-10T01:37:05.873 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.874 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.874 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.874 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-10T01:37:05.874 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.874 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.874 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.875 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_deviation": { 2023-09-10T01:37:05.875 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_deviation", 2023-09-10T01:37:05.875 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:05.875 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.876 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.876 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2023-09-10T01:37:05.876 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2023-09-10T01:37:05.876 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.876 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.876 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-10T01:37:05.877 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-10T01:37:05.877 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.877 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.877 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_optimizations": { 2023-09-10T01:37:05.877 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_optimizations", 2023-09-10T01:37:05.878 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2023-09-10T01:37:05.878 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.878 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.878 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:05.878 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.879 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.879 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.879 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-10T01:37:05.879 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.879 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.879 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.880 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:05.880 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:05.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:05.880 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm", 2023-09-10T01:37:05.880 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:05.881 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:05.881 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:05.881 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_down_multiplier": { 2023-09-10T01:37:05.881 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_down_multiplier", 2023-09-10T01:37:05.881 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:05.882 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.882 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.882 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2023-09-10T01:37:05.882 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.882 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.882 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.883 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-10T01:37:05.883 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.883 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.883 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.883 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_refresh_rate": { 2023-09-10T01:37:05.884 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_refresh_rate", 2023-09-10T01:37:05.884 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:05.884 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.884 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.884 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "20", 2023-09-10T01:37:05.884 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.885 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.885 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.885 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-10T01:37:05.885 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.885 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.886 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.886 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_starting_port": { 2023-09-10T01:37:05.886 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_starting_port", 2023-09-10T01:37:05.886 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:05.886 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.887 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.887 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4721", 2023-09-10T01:37:05.887 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.887 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.887 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.888 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-10T01:37:05.888 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.888 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.888 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.889 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_ptrace": { 2023-09-10T01:37:05.889 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_ptrace", 2023-09-10T01:37:05.889 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.889 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.889 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.889 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:05.890 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.890 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.890 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.890 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-10T01:37:05.890 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-10T01:37:05.891 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.891 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.891 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_interval": { 2023-09-10T01:37:05.891 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_interval", 2023-09-10T01:37:05.892 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:05.892 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.892 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.892 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2023-09-10T01:37:05.892 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.892 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.893 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.893 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-10T01:37:05.893 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.893 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.893 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.894 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_memory_target_ratio": { 2023-09-10T01:37:05.894 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_memory_target_ratio", 2023-09-10T01:37:05.894 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:05.894 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.895 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.895 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.7", 2023-09-10T01:37:05.895 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.895 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.895 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.895 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-10T01:37:05.896 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.896 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.896 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.896 INFO:teuthology.orchestra.run.smithi044.stdout: "cgroups_split": { 2023-09-10T01:37:05.896 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cgroups_split", 2023-09-10T01:37:05.897 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.897 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.897 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.897 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:05.897 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.897 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.898 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.898 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-10T01:37:05.898 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.898 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.898 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.899 INFO:teuthology.orchestra.run.smithi044.stdout: "config_checks_enabled": { 2023-09-10T01:37:05.899 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_checks_enabled", 2023-09-10T01:37:05.899 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.899 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.899 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.900 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:05.900 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.900 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.900 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.900 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-10T01:37:05.901 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.901 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.901 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.901 INFO:teuthology.orchestra.run.smithi044.stdout: "config_dashboard": { 2023-09-10T01:37:05.901 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_dashboard", 2023-09-10T01:37:05.902 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.902 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.902 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.902 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:05.902 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.903 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.903 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.903 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-10T01:37:05.903 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.903 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.904 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.904 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_alertmanager": { 2023-09-10T01:37:05.904 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_alertmanager", 2023-09-10T01:37:05.904 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.904 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.905 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.905 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-10T01:37:05.905 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.905 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.905 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.905 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2023-09-10T01:37:05.906 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.906 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.906 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.906 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_base": { 2023-09-10T01:37:05.906 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_base", 2023-09-10T01:37:05.907 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.907 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.907 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.907 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-10T01:37:05.907 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.907 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.908 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.908 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Container image name, without the tag", 2023-09-10T01:37:05.908 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.908 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.908 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.909 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_elasticsearch": { 2023-09-10T01:37:05.909 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_elasticsearch", 2023-09-10T01:37:05.909 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.909 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.909 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.910 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-10T01:37:05.910 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.910 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.910 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.910 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "elasticsearch container image", 2023-09-10T01:37:05.910 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.911 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.911 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.911 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_grafana": { 2023-09-10T01:37:05.911 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_grafana", 2023-09-10T01:37:05.912 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.912 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.912 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.912 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-10T01:37:05.912 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.912 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.913 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.913 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2023-09-10T01:37:05.913 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.913 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.913 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.914 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_haproxy": { 2023-09-10T01:37:05.914 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_haproxy", 2023-09-10T01:37:05.914 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.914 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.915 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.915 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-10T01:37:05.915 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.915 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.915 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.915 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "HAproxy container image", 2023-09-10T01:37:05.916 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.916 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.916 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.916 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_agent": { 2023-09-10T01:37:05.916 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_agent", 2023-09-10T01:37:05.916 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.917 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.917 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.917 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-10T01:37:05.917 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.917 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.918 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.918 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger agent container image", 2023-09-10T01:37:05.918 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.918 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.918 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.919 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_collector": { 2023-09-10T01:37:05.919 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_collector", 2023-09-10T01:37:05.919 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.919 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.919 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.920 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-10T01:37:05.920 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.920 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.920 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.920 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger collector container image", 2023-09-10T01:37:05.920 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.921 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.921 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.921 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_query": { 2023-09-10T01:37:05.921 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_query", 2023-09-10T01:37:05.922 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.922 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.922 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.922 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-10T01:37:05.922 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.922 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.923 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.923 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger query container image", 2023-09-10T01:37:05.923 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.923 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.923 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.924 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_keepalived": { 2023-09-10T01:37:05.924 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_keepalived", 2023-09-10T01:37:05.924 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.924 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.925 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.925 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-10T01:37:05.925 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.925 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.925 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.925 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Keepalived container image", 2023-09-10T01:37:05.926 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.926 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.926 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.926 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_loki": { 2023-09-10T01:37:05.927 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_loki", 2023-09-10T01:37:05.927 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.927 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.927 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.927 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-10T01:37:05.927 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.928 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.928 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.928 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Loki container image", 2023-09-10T01:37:05.928 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.928 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.929 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.929 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_node_exporter": { 2023-09-10T01:37:05.929 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_node_exporter", 2023-09-10T01:37:05.929 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.930 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.930 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.930 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-10T01:37:05.930 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.930 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.930 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.931 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2023-09-10T01:37:05.931 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.931 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.931 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.931 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_nvmeof": { 2023-09-10T01:37:05.932 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_nvmeof", 2023-09-10T01:37:05.932 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.932 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.932 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.932 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-10T01:37:05.933 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.933 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.933 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.933 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Nvme-of container image", 2023-09-10T01:37:05.933 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.934 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.934 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.934 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_prometheus": { 2023-09-10T01:37:05.934 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_prometheus", 2023-09-10T01:37:05.934 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.935 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.935 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.935 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-10T01:37:05.935 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.935 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.935 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.936 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2023-09-10T01:37:05.936 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.936 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.936 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.937 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_promtail": { 2023-09-10T01:37:05.937 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_promtail", 2023-09-10T01:37:05.937 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.937 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.937 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.937 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-10T01:37:05.938 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.938 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.938 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.938 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Promtail container image", 2023-09-10T01:37:05.938 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.939 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.939 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.939 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_snmp_gateway": { 2023-09-10T01:37:05.939 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_snmp_gateway", 2023-09-10T01:37:05.939 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.940 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.940 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.940 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-10T01:37:05.940 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.940 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.940 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.941 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SNMP Gateway container image", 2023-09-10T01:37:05.941 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.941 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.941 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.942 INFO:teuthology.orchestra.run.smithi044.stdout: "container_init": { 2023-09-10T01:37:05.942 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_init", 2023-09-10T01:37:05.942 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.942 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.942 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.942 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:05.943 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.943 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.943 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.943 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Run podman/docker with `--init`", 2023-09-10T01:37:05.943 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.943 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.944 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.944 INFO:teuthology.orchestra.run.smithi044.stdout: "daemon_cache_timeout": { 2023-09-10T01:37:05.944 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "daemon_cache_timeout", 2023-09-10T01:37:05.944 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:05.944 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.945 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.945 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2023-09-10T01:37:05.945 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.945 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.945 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.945 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-10T01:37:05.946 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.946 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.946 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.946 INFO:teuthology.orchestra.run.smithi044.stdout: "default_cephadm_command_timeout": { 2023-09-10T01:37:05.947 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_cephadm_command_timeout", 2023-09-10T01:37:05.947 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:05.947 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.947 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.947 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "900", 2023-09-10T01:37:05.947 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.948 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.948 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.948 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-10T01:37:05.948 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.948 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.948 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.949 INFO:teuthology.orchestra.run.smithi044.stdout: "default_registry": { 2023-09-10T01:37:05.949 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_registry", 2023-09-10T01:37:05.949 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.949 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.950 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.950 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io", 2023-09-10T01:37:05.950 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.950 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.950 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.950 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-10T01:37:05.951 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.951 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.951 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.951 INFO:teuthology.orchestra.run.smithi044.stdout: "device_cache_timeout": { 2023-09-10T01:37:05.952 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_cache_timeout", 2023-09-10T01:37:05.952 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:05.952 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.952 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.952 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1800", 2023-09-10T01:37:05.953 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.953 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.953 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.953 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache device inventory", 2023-09-10T01:37:05.953 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.953 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.954 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.954 INFO:teuthology.orchestra.run.smithi044.stdout: "device_enhanced_scan": { 2023-09-10T01:37:05.954 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_enhanced_scan", 2023-09-10T01:37:05.954 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.955 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.955 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.955 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:05.955 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.955 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.955 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.956 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-10T01:37:05.956 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.956 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.956 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.956 INFO:teuthology.orchestra.run.smithi044.stdout: "facts_cache_timeout": { 2023-09-10T01:37:05.957 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "facts_cache_timeout", 2023-09-10T01:37:05.957 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:05.957 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.957 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.957 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2023-09-10T01:37:05.957 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.958 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.958 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.958 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache host facts data", 2023-09-10T01:37:05.958 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.958 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.959 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.959 INFO:teuthology.orchestra.run.smithi044.stdout: "host_check_interval": { 2023-09-10T01:37:05.959 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "host_check_interval", 2023-09-10T01:37:05.959 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:05.959 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.960 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.960 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2023-09-10T01:37:05.960 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.960 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.960 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.961 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to perform a host check", 2023-09-10T01:37:05.961 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.961 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.961 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.961 INFO:teuthology.orchestra.run.smithi044.stdout: "inventory_list_all": { 2023-09-10T01:37:05.962 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "inventory_list_all", 2023-09-10T01:37:05.962 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.962 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.962 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.962 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:05.963 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.963 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.963 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.963 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-10T01:37:05.963 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.964 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.964 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.964 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:05.964 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:05.965 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.965 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.965 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.965 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:05.965 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.965 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.966 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:05.966 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:05.966 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:05.966 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:05.966 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:05.966 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:05.967 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:05.967 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:05.967 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:05.967 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.967 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.967 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "log_refresh_metadata": { 2023-09-10T01:37:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_refresh_metadata", 2023-09-10T01:37:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-10T01:37:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-10T01:37:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:05.975 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:05.975 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:05.975 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:05.975 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:05.975 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:05.975 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-10T01:37:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-10T01:37:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-10T01:37:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-10T01:37:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-10T01:37:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "*", 2023-09-10T01:37:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-10T01:37:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: "max_count_per_host": { 2023-09-10T01:37:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_count_per_host", 2023-09-10T01:37:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.987 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.987 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of daemons per service per host", 2023-09-10T01:37:05.987 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.987 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.987 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd_draining_count": { 2023-09-10T01:37:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_osd_draining_count", 2023-09-10T01:37:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-10T01:37:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: "migration_current": { 2023-09-10T01:37:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "migration_current", 2023-09-10T01:37:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "internal - do not modify", 2023-09-10T01:37:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2023-09-10T01:37:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2023-09-10T01:37:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "root", 2023-09-10T01:37:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm-package", 2023-09-10T01:37:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: "root" 2023-09-10T01:37:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "mode for remote execution of cephadm", 2023-09-10T01:37:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "prometheus_alerts_path": { 2023-09-10T01:37:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus_alerts_path", 2023-09-10T01:37:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-10T01:37:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-10T01:37:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_insecure": { 2023-09-10T01:37:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_insecure", 2023-09-10T01:37:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-10T01:37:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_password": { 2023-09-10T01:37:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_password", 2023-09-10T01:37:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-10T01:37:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_url": { 2023-09-10T01:37:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_url", 2023-09-10T01:37:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-10T01:37:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_username": { 2023-09-10T01:37:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_username", 2023-09-10T01:37:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-10T01:37:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: "secure_monitoring_stack": { 2023-09-10T01:37:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "secure_monitoring_stack", 2023-09-10T01:37:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-10T01:37:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: "service_discovery_port": { 2023-09-10T01:37:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "service_discovery_port", 2023-09-10T01:37:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8765", 2023-09-10T01:37:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "cephadm service discovery port", 2023-09-10T01:37:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: "ssh_config_file": { 2023-09-10T01:37:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssh_config_file", 2023-09-10T01:37:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-10T01:37:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: "use_agent": { 2023-09-10T01:37:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_agent", 2023-09-10T01:37:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-10T01:37:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: "use_repo_digest": { 2023-09-10T01:37:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_repo_digest", 2023-09-10T01:37:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-10T01:37:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_failed_host_check": { 2023-09-10T01:37:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_failed_host_check", 2023-09-10T01:37:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if the host check fails", 2023-09-10T01:37:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_daemons": { 2023-09-10T01:37:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_daemons", 2023-09-10T01:37:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-10T01:37:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_hosts": { 2023-09-10T01:37:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_hosts", 2023-09-10T01:37:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-10T01:37:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crash", 2023-09-10T01:37:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: "retain_interval": { 2023-09-10T01:37:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retain_interval", 2023-09-10T01:37:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "31536000", 2023-09-10T01:37:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-10T01:37:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_recent_interval": { 2023-09-10T01:37:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_recent_interval", 2023-09-10T01:37:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1209600", 2023-09-10T01:37:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-10T01:37:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dashboard", 2023-09-10T01:37:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-10T01:37:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-10T01:37:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_HOST": { 2023-09-10T01:37:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-10T01:37:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.054 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.054 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.054 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.054 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.054 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-10T01:37:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-10T01:37:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.056 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.056 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.056 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.056 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.056 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_ENABLED": { 2023-09-10T01:37:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_ENABLED", 2023-09-10T01:37:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-10T01:37:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-10T01:37:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: "ENABLE_BROWSABLE_API": { 2023-09-10T01:37:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-10T01:37:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.063 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.063 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.063 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.063 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.063 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.064 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.064 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.064 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.064 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.064 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.065 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-10T01:37:06.065 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-10T01:37:06.065 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.065 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.065 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.066 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.066 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.066 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.066 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.066 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.066 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.067 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.067 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.067 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-10T01:37:06.067 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-10T01:37:06.068 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.068 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.068 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.068 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.068 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.068 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.069 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.069 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.069 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.069 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.069 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.070 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-10T01:37:06.070 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-10T01:37:06.070 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.070 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.070 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.071 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.071 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.071 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.071 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.071 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.071 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.072 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.072 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.072 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-10T01:37:06.072 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-10T01:37:06.073 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.073 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.073 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.073 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.073 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.073 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.074 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.074 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.074 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.074 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.074 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.075 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-10T01:37:06.075 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-10T01:37:06.075 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.075 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.075 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.076 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.076 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.076 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.076 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.076 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.077 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.077 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.077 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.077 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-10T01:37:06.077 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-10T01:37:06.078 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.078 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.078 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.078 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.078 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.079 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.079 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.079 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.079 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.079 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.080 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.080 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-10T01:37:06.080 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-10T01:37:06.080 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.081 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.081 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.081 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.081 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.081 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.082 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.082 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.082 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.082 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.082 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.083 INFO:teuthology.orchestra.run.smithi044.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-10T01:37:06.083 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-10T01:37:06.083 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.083 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.083 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.084 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.084 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.084 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.084 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.084 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.085 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.085 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.085 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.085 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_PASSWORD": { 2023-09-10T01:37:06.085 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-10T01:37:06.086 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.086 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.086 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.086 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2023-09-10T01:37:06.086 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.087 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.087 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.087 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.087 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.087 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.088 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.088 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-10T01:37:06.088 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-10T01:37:06.088 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.088 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.089 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.089 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.089 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.089 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.089 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.090 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.090 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.090 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.090 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.090 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_URL": { 2023-09-10T01:37:06.091 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_URL", 2023-09-10T01:37:06.091 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.091 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.091 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.091 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.092 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.092 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.092 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.092 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.092 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.093 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.093 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.093 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_USERNAME": { 2023-09-10T01:37:06.093 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-10T01:37:06.093 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.094 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.094 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.094 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2023-09-10T01:37:06.094 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.094 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.095 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.095 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.095 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.095 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.095 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.096 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-10T01:37:06.096 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-10T01:37:06.096 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.096 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.096 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.096 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.097 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.097 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.097 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.097 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.097 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.098 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.098 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.098 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-10T01:37:06.098 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-10T01:37:06.098 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.099 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.099 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.099 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.099 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.099 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.100 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.100 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.100 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.100 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.100 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.101 INFO:teuthology.orchestra.run.smithi044.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-10T01:37:06.101 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-10T01:37:06.101 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.101 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.101 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.102 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.102 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.102 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.102 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.102 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.102 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.103 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.103 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.103 INFO:teuthology.orchestra.run.smithi044.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-10T01:37:06.103 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-10T01:37:06.103 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.104 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.104 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.104 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.104 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.104 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.105 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.105 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.105 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.105 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.105 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.106 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_HOST": { 2023-09-10T01:37:06.106 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-10T01:37:06.106 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.106 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.106 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.107 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.107 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.107 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.107 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.107 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.108 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.108 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.108 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.108 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-10T01:37:06.108 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-10T01:37:06.109 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.109 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.109 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.109 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.109 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.109 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.110 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.110 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.110 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.110 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.110 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.111 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-10T01:37:06.111 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-10T01:37:06.111 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.111 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.111 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.112 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.112 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.112 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.112 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.112 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.112 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.113 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.113 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.113 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-10T01:37:06.113 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-10T01:37:06.113 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.114 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.114 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.114 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.114 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.114 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.114 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.115 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.115 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.115 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.115 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.115 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-10T01:37:06.116 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-10T01:37:06.116 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.116 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.116 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.116 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.117 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.117 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.117 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.117 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.117 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.117 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.117 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.118 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-10T01:37:06.118 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-10T01:37:06.118 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.118 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.119 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.119 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.119 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.119 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.119 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.119 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.119 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.120 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.120 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.120 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-10T01:37:06.120 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-10T01:37:06.121 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.121 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.121 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.121 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.121 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.121 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.122 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.122 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.122 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.122 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.122 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.123 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-10T01:37:06.123 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-10T01:37:06.123 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.123 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.123 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.124 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.124 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.124 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.124 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.124 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.125 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.125 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.125 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.125 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-10T01:37:06.125 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-10T01:37:06.126 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.126 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.126 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.126 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.126 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.127 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.127 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.127 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.127 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.127 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.127 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.128 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_ENABLED": { 2023-09-10T01:37:06.128 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-10T01:37:06.128 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.128 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.128 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.129 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.129 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.129 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.129 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.129 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.129 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.130 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.130 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.130 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-10T01:37:06.130 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-10T01:37:06.131 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.131 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.131 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.131 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-10T01:37:06.131 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.131 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.132 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.132 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.132 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.132 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.132 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.133 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-10T01:37:06.133 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-10T01:37:06.133 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.133 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.133 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.133 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:06.134 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.134 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.134 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.134 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.134 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.135 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.135 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.135 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-10T01:37:06.135 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-10T01:37:06.135 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.136 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.136 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.136 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8", 2023-09-10T01:37:06.136 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.136 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.137 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.137 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.137 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.137 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.137 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.137 INFO:teuthology.orchestra.run.smithi044.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-10T01:37:06.138 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-10T01:37:06.138 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.138 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.138 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.138 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "45", 2023-09-10T01:37:06.139 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.139 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.139 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.139 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.139 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.139 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.140 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.140 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ACCESS_KEY": { 2023-09-10T01:37:06.140 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-10T01:37:06.140 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.140 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.141 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.141 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.141 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.141 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.141 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.142 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.142 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.142 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.142 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.143 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-10T01:37:06.143 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-10T01:37:06.143 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.143 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.143 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.143 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2023-09-10T01:37:06.144 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.144 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.144 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.144 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.144 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.144 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.145 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.145 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SECRET_KEY": { 2023-09-10T01:37:06.145 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-10T01:37:06.145 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.146 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.146 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.146 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.146 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.146 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.146 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.147 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.147 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.147 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.147 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.147 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SSL_VERIFY": { 2023-09-10T01:37:06.148 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-10T01:37:06.148 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.148 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.148 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.148 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.148 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.149 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.149 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.149 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.149 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.149 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.150 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.150 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-10T01:37:06.150 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-10T01:37:06.150 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.150 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.151 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.151 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2023-09-10T01:37:06.151 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.151 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.151 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.152 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.152 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.152 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.152 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.152 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-10T01:37:06.153 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-10T01:37:06.153 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.153 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.153 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.153 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:06.153 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.154 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.154 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.154 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.154 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.154 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.154 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.155 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-10T01:37:06.155 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-10T01:37:06.155 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.155 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.155 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.156 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2023-09-10T01:37:06.156 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.156 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.156 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.156 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.157 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.157 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.157 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.157 INFO:teuthology.orchestra.run.smithi044.stdout: "cross_origin_url": { 2023-09-10T01:37:06.157 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cross_origin_url", 2023-09-10T01:37:06.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.158 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.158 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.158 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.158 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.158 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.159 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.159 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.159 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.159 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.159 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.160 INFO:teuthology.orchestra.run.smithi044.stdout: "crt_file": { 2023-09-10T01:37:06.160 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crt_file", 2023-09-10T01:37:06.160 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.160 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.161 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.161 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.161 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.161 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.161 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.161 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.162 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.162 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.162 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.162 INFO:teuthology.orchestra.run.smithi044.stdout: "debug": { 2023-09-10T01:37:06.163 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "debug", 2023-09-10T01:37:06.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.163 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.163 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.163 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.163 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.164 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.164 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.164 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable/disable debug options", 2023-09-10T01:37:06.164 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.164 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.165 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.165 INFO:teuthology.orchestra.run.smithi044.stdout: "jwt_token_ttl": { 2023-09-10T01:37:06.165 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "jwt_token_ttl", 2023-09-10T01:37:06.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.165 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.166 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.166 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "28800", 2023-09-10T01:37:06.166 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.166 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.166 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.166 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.167 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.167 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.167 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.167 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2023-09-10T01:37:06.168 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2023-09-10T01:37:06.168 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.168 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.168 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.168 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.168 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.169 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.169 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.169 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.169 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.169 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.169 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.170 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.170 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.170 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.170 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.170 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.171 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.171 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.171 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.171 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.171 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.172 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.172 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.172 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.172 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.172 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.172 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.173 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.173 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.173 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.173 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.173 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.174 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.174 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.174 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.174 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.174 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.174 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.175 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.175 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.175 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.175 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.175 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.176 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.176 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.176 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.176 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.176 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.177 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.177 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.177 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.177 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.177 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.177 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.178 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.178 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.178 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.178 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.178 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.178 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.179 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.179 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.179 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.179 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.179 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.180 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.180 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.180 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.180 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.180 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.180 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.181 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.181 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.181 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.181 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.181 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.182 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.182 INFO:teuthology.orchestra.run.smithi044.stdout: "motd": { 2023-09-10T01:37:06.182 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "motd", 2023-09-10T01:37:06.182 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.182 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.183 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.183 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.183 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.183 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.183 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.184 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "The message of the day", 2023-09-10T01:37:06.184 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.184 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.184 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.184 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect_resolve_ip_addr": { 2023-09-10T01:37:06.184 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "redirect_resolve_ip_addr", 2023-09-10T01:37:06.185 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.185 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.185 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.185 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.185 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.185 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.186 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.186 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.186 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.186 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.186 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.187 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2023-09-10T01:37:06.187 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2023-09-10T01:37:06.187 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.187 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.187 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.187 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2023-09-10T01:37:06.188 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.188 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.188 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.188 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.188 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.188 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.189 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.189 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2023-09-10T01:37:06.189 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2023-09-10T01:37:06.189 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.189 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.190 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.190 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8080", 2023-09-10T01:37:06.190 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.190 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.190 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.191 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.191 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.191 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.191 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.191 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2023-09-10T01:37:06.192 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2023-09-10T01:37:06.192 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.192 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.192 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.192 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.192 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.193 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.193 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.193 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.193 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.193 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.193 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.194 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl_server_port": { 2023-09-10T01:37:06.194 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl_server_port", 2023-09-10T01:37:06.194 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.194 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.195 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.195 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8443", 2023-09-10T01:37:06.195 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.195 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.195 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.196 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.196 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.196 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.196 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.196 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2023-09-10T01:37:06.197 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2023-09-10T01:37:06.197 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.197 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.197 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.197 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "redirect", 2023-09-10T01:37:06.197 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.198 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.198 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.198 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.198 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect" 2023-09-10T01:37:06.198 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2023-09-10T01:37:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2023-09-10T01:37:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2023-09-10T01:37:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2023-09-10T01:37:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2023-09-10T01:37:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: "url_prefix": { 2023-09-10T01:37:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url_prefix", 2023-09-10T01:37:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.204 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.204 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.204 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.204 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.204 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "devicehealth", 2023-09-10T01:37:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_monitoring": { 2023-09-10T01:37:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_monitoring", 2023-09-10T01:37:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "monitor device health metrics", 2023-09-10T01:37:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.209 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.209 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.209 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.209 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.209 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.209 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.210 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.210 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.210 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.210 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.210 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.210 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.211 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.211 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.211 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.211 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.211 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.212 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.212 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.212 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.212 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.212 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.213 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.213 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.213 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.213 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.213 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.214 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.214 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.214 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.214 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.214 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.215 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.215 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.215 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.215 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.215 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.215 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.216 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.216 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.216 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.216 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.216 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.216 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.217 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.217 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.217 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.217 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.217 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.217 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.218 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.218 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.218 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.218 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.219 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.219 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.219 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.219 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.219 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.219 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: "mark_out_threshold": { 2023-09-10T01:37:06.221 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mark_out_threshold", 2023-09-10T01:37:06.221 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:06.221 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.221 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.221 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2419200", 2023-09-10T01:37:06.221 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.222 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.222 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.222 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-10T01:37:06.222 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.222 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.222 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": { 2023-09-10T01:37:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_name", 2023-09-10T01:37:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "device_health_metrics", 2023-09-10T01:37:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-10T01:37:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: "retention_period": { 2023-09-10T01:37:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retention_period", 2023-09-10T01:37:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15552000", 2023-09-10T01:37:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain device health metrics", 2023-09-10T01:37:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_frequency": { 2023-09-10T01:37:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_frequency", 2023-09-10T01:37:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2023-09-10T01:37:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-10T01:37:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.230 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.230 INFO:teuthology.orchestra.run.smithi044.stdout: "self_heal": { 2023-09-10T01:37:06.230 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "self_heal", 2023-09-10T01:37:06.230 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.230 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.231 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.231 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.231 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.231 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.231 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.231 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-10T01:37:06.232 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.232 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.232 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.232 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2023-09-10T01:37:06.232 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2023-09-10T01:37:06.233 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:06.233 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.233 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.233 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2023-09-10T01:37:06.233 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.233 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.233 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.234 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and check device health", 2023-09-10T01:37:06.234 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.234 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.234 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.235 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_threshold": { 2023-09-10T01:37:06.235 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_threshold", 2023-09-10T01:37:06.235 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:06.235 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.235 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.235 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7257600", 2023-09-10T01:37:06.236 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.236 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.236 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.236 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-10T01:37:06.236 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.237 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.237 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.237 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.237 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.238 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "diskprediction_local", 2023-09-10T01:37:06.238 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.238 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.238 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.238 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.239 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.239 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.239 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.239 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.239 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.239 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.240 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.240 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.240 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.240 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.240 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.240 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.241 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.241 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.241 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.241 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.241 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.242 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.242 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.242 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.242 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.242 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.243 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.243 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.243 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.243 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.243 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.243 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.244 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.244 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.244 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.244 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.244 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.245 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.245 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.245 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.245 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.245 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.246 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.246 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.246 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.246 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.246 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.247 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.247 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.247 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.247 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.247 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.248 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.248 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.248 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.248 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.248 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.249 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.249 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.249 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.249 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.249 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.249 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.250 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.250 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.250 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.250 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.250 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.251 INFO:teuthology.orchestra.run.smithi044.stdout: "predict_interval": { 2023-09-10T01:37:06.251 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predict_interval", 2023-09-10T01:37:06.251 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.251 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.251 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.252 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2023-09-10T01:37:06.252 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.252 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.252 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.252 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.252 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.253 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.253 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.253 INFO:teuthology.orchestra.run.smithi044.stdout: "predictor_model": { 2023-09-10T01:37:06.253 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predictor_model", 2023-09-10T01:37:06.254 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.254 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.254 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.254 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "prophetstor", 2023-09-10T01:37:06.254 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.255 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.255 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.255 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.255 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.255 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.255 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.256 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2023-09-10T01:37:06.256 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2023-09-10T01:37:06.256 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.256 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.256 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.257 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2023-09-10T01:37:06.257 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.257 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.257 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.257 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.257 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.258 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.258 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.258 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.258 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.259 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "influx", 2023-09-10T01:37:06.259 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": false, 2023-09-10T01:37:06.259 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "influxdb python module not found", 2023-09-10T01:37:06.259 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.259 INFO:teuthology.orchestra.run.smithi044.stdout: "batch_size": { 2023-09-10T01:37:06.260 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "batch_size", 2023-09-10T01:37:06.260 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.260 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.260 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.260 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5000", 2023-09-10T01:37:06.261 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.261 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.261 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.261 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-10T01:37:06.261 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.261 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.262 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.262 INFO:teuthology.orchestra.run.smithi044.stdout: "database": { 2023-09-10T01:37:06.262 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "database", 2023-09-10T01:37:06.262 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.263 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.263 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.263 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "ceph", 2023-09-10T01:37:06.263 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.263 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.263 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.263 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-10T01:37:06.264 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.264 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.264 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.264 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": { 2023-09-10T01:37:06.264 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hostname", 2023-09-10T01:37:06.265 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.265 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.265 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.265 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.265 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.266 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.266 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.266 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server hostname", 2023-09-10T01:37:06.266 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.266 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.266 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.267 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2023-09-10T01:37:06.267 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2023-09-10T01:37:06.267 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:06.267 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.267 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.268 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "30", 2023-09-10T01:37:06.268 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "5", 2023-09-10T01:37:06.268 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.268 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.268 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-10T01:37:06.268 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.269 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.269 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.269 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.269 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.269 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.270 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.270 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.270 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.270 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.270 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.270 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.271 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.271 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.271 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.271 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.271 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.272 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.272 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.272 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.272 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.272 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.272 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.273 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.273 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.273 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.273 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.274 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.274 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.274 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.274 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.274 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.274 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.275 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.275 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.275 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.275 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.276 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.276 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.276 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.276 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.276 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.277 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.277 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.277 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.277 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.277 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.277 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.278 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.278 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.278 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.278 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.278 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.279 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.279 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.279 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.279 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.279 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.280 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.280 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.280 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.280 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.280 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.280 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.281 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.281 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.281 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.281 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.281 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.282 INFO:teuthology.orchestra.run.smithi044.stdout: "password": { 2023-09-10T01:37:06.282 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "password", 2023-09-10T01:37:06.282 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.282 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.282 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.283 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.283 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.283 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.283 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.283 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "password of InfluxDB server user", 2023-09-10T01:37:06.284 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.284 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.284 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.284 INFO:teuthology.orchestra.run.smithi044.stdout: "port": { 2023-09-10T01:37:06.285 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "port", 2023-09-10T01:37:06.285 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.285 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.285 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.285 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8086", 2023-09-10T01:37:06.285 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.286 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.286 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.286 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server port", 2023-09-10T01:37:06.286 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.286 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.287 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.287 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2023-09-10T01:37:06.287 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2023-09-10T01:37:06.287 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.287 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.288 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.288 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "false", 2023-09-10T01:37:06.288 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.288 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.288 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.289 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-10T01:37:06.289 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.289 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.289 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.289 INFO:teuthology.orchestra.run.smithi044.stdout: "threads": { 2023-09-10T01:37:06.290 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threads", 2023-09-10T01:37:06.290 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.290 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.290 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.290 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2023-09-10T01:37:06.291 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2023-09-10T01:37:06.291 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "32", 2023-09-10T01:37:06.291 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.291 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-10T01:37:06.291 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.292 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.292 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.292 INFO:teuthology.orchestra.run.smithi044.stdout: "username": { 2023-09-10T01:37:06.292 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "username", 2023-09-10T01:37:06.293 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.293 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.293 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.293 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.293 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.293 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.294 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.294 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "username of InfluxDB server user", 2023-09-10T01:37:06.294 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.294 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.294 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.295 INFO:teuthology.orchestra.run.smithi044.stdout: "verify_ssl": { 2023-09-10T01:37:06.295 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "verify_ssl", 2023-09-10T01:37:06.295 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.295 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.295 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.296 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "true", 2023-09-10T01:37:06.296 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.296 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.296 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.296 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-10T01:37:06.297 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.297 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.297 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.297 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.297 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.298 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "insights", 2023-09-10T01:37:06.298 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.298 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.298 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.299 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.299 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.299 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.299 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.299 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.299 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.300 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.300 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.300 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.300 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.300 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.301 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.301 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.301 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.301 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.301 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.301 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.302 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.302 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.302 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.302 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.302 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.303 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.303 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.303 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.303 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.303 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.304 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.304 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.304 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.304 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.304 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.304 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.305 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.305 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.305 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.305 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.305 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.306 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.306 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.306 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.306 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.306 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.307 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.307 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.307 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.307 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.307 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.307 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.308 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.308 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.308 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.308 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.308 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.309 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.309 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.309 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.309 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.309 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.310 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.310 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.310 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.310 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.310 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.311 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.311 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.311 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.311 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.312 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "iostat", 2023-09-10T01:37:06.312 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.312 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.312 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.312 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.313 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.313 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.313 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.313 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.313 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.314 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.314 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.314 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.314 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.314 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.315 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.315 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.315 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.315 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.315 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.315 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.316 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.316 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.316 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.316 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.316 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.317 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.317 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.317 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.317 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.317 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.317 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.318 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.318 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.318 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.318 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.318 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.319 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.319 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.319 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.319 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.319 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.320 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.320 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.320 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.320 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.320 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.321 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.321 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.321 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.321 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.321 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.321 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.322 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.322 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.322 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.322 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.323 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.323 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.323 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.323 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.323 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.323 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.324 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.324 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.324 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.324 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.324 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.325 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.325 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.325 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.325 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.326 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "k8sevents", 2023-09-10T01:37:06.326 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.326 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.326 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.326 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_event_retention_days": { 2023-09-10T01:37:06.327 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ceph_event_retention_days", 2023-09-10T01:37:06.327 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.327 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.327 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.327 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7", 2023-09-10T01:37:06.327 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.328 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.328 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.328 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-10T01:37:06.328 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.328 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.328 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.329 INFO:teuthology.orchestra.run.smithi044.stdout: "config_check_secs": { 2023-09-10T01:37:06.329 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_check_secs", 2023-09-10T01:37:06.329 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.329 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.330 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.330 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:06.330 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "10", 2023-09-10T01:37:06.330 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.330 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.330 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-10T01:37:06.331 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.331 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.331 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.331 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.332 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.332 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.332 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.332 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.332 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.332 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.333 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.333 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.333 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.333 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.333 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.334 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.334 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.334 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.334 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.334 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.334 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.335 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.335 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.335 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.335 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.336 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.336 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.336 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.336 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.336 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.336 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.337 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.337 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.337 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.337 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.337 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.338 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.338 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.338 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.338 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.339 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.339 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.339 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.339 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.339 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.339 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.340 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.340 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.340 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.340 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.340 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.341 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.341 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.341 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.341 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.341 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.342 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.342 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.342 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.342 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.342 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.343 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.343 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.343 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.343 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.343 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.343 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.344 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.344 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.344 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.344 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.345 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "localpool", 2023-09-10T01:37:06.345 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.345 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.345 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.345 INFO:teuthology.orchestra.run.smithi044.stdout: "failure_domain": { 2023-09-10T01:37:06.346 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "failure_domain", 2023-09-10T01:37:06.346 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.346 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.346 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.346 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "host", 2023-09-10T01:37:06.347 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.347 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.347 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.347 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "failure domain for any created local pool", 2023-09-10T01:37:06.347 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-10T01:37:06.347 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.348 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.348 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.348 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.348 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.349 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.349 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.349 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.349 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.349 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.349 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.350 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.350 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.350 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.350 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.350 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.351 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.351 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.351 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.351 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.351 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.351 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.352 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.352 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.352 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.352 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.352 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.353 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.353 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.353 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.353 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.353 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.353 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.354 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.354 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.354 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.354 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.355 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.355 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.355 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.355 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.355 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.356 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.356 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.356 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.356 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.356 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.357 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.357 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.357 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.357 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.357 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.357 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.358 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.358 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.358 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.358 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.359 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.359 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.359 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.359 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.359 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.359 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.360 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.360 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.360 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.360 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.360 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.361 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": { 2023-09-10T01:37:06.361 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_size", 2023-09-10T01:37:06.361 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.361 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.361 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.362 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.362 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.362 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.362 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.362 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default min_size for any created local pool", 2023-09-10T01:37:06.363 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-10T01:37:06.363 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.363 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.363 INFO:teuthology.orchestra.run.smithi044.stdout: "num_rep": { 2023-09-10T01:37:06.364 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "num_rep", 2023-09-10T01:37:06.364 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.364 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.364 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.364 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3", 2023-09-10T01:37:06.364 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.365 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.365 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.365 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default replica count for any created local pool", 2023-09-10T01:37:06.365 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.365 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.366 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.366 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": { 2023-09-10T01:37:06.366 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_num", 2023-09-10T01:37:06.366 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.367 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.367 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.367 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "128", 2023-09-10T01:37:06.367 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.367 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.368 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.368 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default pg_num for any created local pool", 2023-09-10T01:37:06.368 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.368 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.368 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.369 INFO:teuthology.orchestra.run.smithi044.stdout: "prefix": { 2023-09-10T01:37:06.369 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prefix", 2023-09-10T01:37:06.369 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.369 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.369 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.369 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.370 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.370 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.370 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.370 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name prefix for any created local pool", 2023-09-10T01:37:06.370 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.370 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.371 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.371 INFO:teuthology.orchestra.run.smithi044.stdout: "subtree": { 2023-09-10T01:37:06.371 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subtree", 2023-09-10T01:37:06.371 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.372 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.372 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.372 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "rack", 2023-09-10T01:37:06.372 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.372 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.372 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.373 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-10T01:37:06.373 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-10T01:37:06.373 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.373 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.373 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.373 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.374 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mds_autoscaler", 2023-09-10T01:37:06.374 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.374 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.375 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.375 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.375 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.375 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.375 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.375 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.376 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.376 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.376 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.376 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.376 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.377 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.377 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.377 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.377 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.377 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.377 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.378 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.378 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.378 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.378 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.379 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.379 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.379 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.379 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.379 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.379 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.380 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.380 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.380 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.380 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.380 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.381 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.381 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.381 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.381 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.381 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.382 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.382 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.382 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.382 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.382 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.383 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.383 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.383 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.383 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.383 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.383 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.384 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.384 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.384 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.384 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.384 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.384 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.385 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.385 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.385 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.385 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.385 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.386 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.386 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.386 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.386 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.386 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.386 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.387 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.387 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.388 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirroring", 2023-09-10T01:37:06.388 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.388 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.388 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.388 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.388 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.389 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.389 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.389 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.389 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.389 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.390 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.390 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.390 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.390 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.390 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.391 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.391 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.391 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.391 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.391 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.392 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.392 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.392 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.392 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.392 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.393 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.393 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.393 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.393 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.393 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.393 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.394 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.394 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.394 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.394 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.394 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.395 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.395 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.395 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.395 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.395 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.396 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.396 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.396 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.396 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.396 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.397 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.397 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.397 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.397 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.397 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.397 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.398 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.398 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.398 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.398 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.399 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.399 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.399 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.399 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.399 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.400 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.400 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.400 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.400 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.400 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.401 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.401 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.401 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.401 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.401 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.402 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "nfs", 2023-09-10T01:37:06.402 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.402 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.402 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.403 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.403 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.403 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.403 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.403 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.403 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.404 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.404 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.404 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.404 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.404 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.404 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.405 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.405 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.405 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.405 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.405 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.405 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.406 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.406 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.406 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.406 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.406 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.407 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.407 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.407 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.407 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.407 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.408 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.408 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.408 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.408 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.408 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.409 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.409 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.409 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.409 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.409 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.409 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.410 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.410 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.410 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.410 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.410 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.411 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.411 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.411 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.411 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.411 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.412 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.412 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.412 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.412 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.412 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.413 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.413 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.413 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.413 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.413 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.413 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.414 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.414 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.414 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.414 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.414 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.415 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.415 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.415 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.416 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2023-09-10T01:37:06.416 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.416 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.416 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.416 INFO:teuthology.orchestra.run.smithi044.stdout: "fail_fs": { 2023-09-10T01:37:06.416 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "fail_fs", 2023-09-10T01:37:06.417 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.417 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.417 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.417 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.417 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.418 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.418 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.418 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-10T01:37:06.418 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.418 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.419 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.419 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.419 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.419 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.419 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.420 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.420 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.420 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.420 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.420 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.420 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.421 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.421 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.421 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.421 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.421 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.422 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.422 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.422 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.422 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.422 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.423 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.423 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.423 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.423 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.423 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.424 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.424 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.424 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.424 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.424 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.424 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.425 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.425 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.425 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.425 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.426 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.426 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.426 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.426 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.426 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.426 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.427 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.427 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.427 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.427 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.427 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.427 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.428 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.428 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.428 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.428 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.428 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.429 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.429 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.429 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.429 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.429 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.430 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.430 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.430 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.430 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.430 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.431 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.431 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.431 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.431 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.432 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator": { 2023-09-10T01:37:06.432 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2023-09-10T01:37:06.432 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.432 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.432 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.432 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.433 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.433 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.433 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.433 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm", 2023-09-10T01:37:06.433 INFO:teuthology.orchestra.run.smithi044.stdout: "rook", 2023-09-10T01:37:06.434 INFO:teuthology.orchestra.run.smithi044.stdout: "test_orchestrator" 2023-09-10T01:37:06.434 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.434 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Orchestrator backend", 2023-09-10T01:37:06.434 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.434 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.434 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.435 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.435 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.435 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_perf_query", 2023-09-10T01:37:06.436 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.436 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.436 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.436 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.436 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.436 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.437 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.437 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.437 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.437 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.437 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.438 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.438 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.438 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.438 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.438 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.438 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.439 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.439 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.439 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.439 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.439 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.439 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.440 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.440 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.440 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.440 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.440 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.441 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.441 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.441 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.441 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.441 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.441 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.441 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.442 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.442 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.442 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.442 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.443 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.443 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.443 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.443 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.443 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.443 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.444 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.444 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.444 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.444 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.444 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.444 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.445 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.445 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.445 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.445 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.445 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.446 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.446 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.446 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.446 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.446 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.447 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.447 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.447 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.447 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.447 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.447 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.448 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.448 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.448 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.448 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.449 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_support", 2023-09-10T01:37:06.449 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.449 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.449 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.449 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.449 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.449 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.450 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.450 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.450 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.450 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.450 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.451 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.451 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.451 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.451 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.451 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.452 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.452 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.452 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.452 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.452 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.452 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.453 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.453 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.453 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.453 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.453 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.454 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.454 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.454 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.454 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.454 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.455 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.455 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.455 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.455 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.455 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.456 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.456 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.456 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.456 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.456 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.456 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.457 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.457 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.457 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.457 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.457 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.457 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.458 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.458 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.458 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.458 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.458 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.458 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.459 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.459 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.459 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.459 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.459 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.460 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.460 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.460 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.460 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.460 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.460 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.461 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.461 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.461 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.461 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.461 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.462 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_autoscaler", 2023-09-10T01:37:06.462 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.462 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.462 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.462 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.463 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.463 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.463 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.463 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.463 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.463 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.464 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.464 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.464 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.464 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.464 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.465 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.465 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.465 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.465 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.465 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.465 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.466 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.466 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.466 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.466 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.466 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.467 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.467 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.467 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.467 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.467 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.467 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.468 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.468 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.468 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.468 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.468 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.469 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.469 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.469 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.469 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.469 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.469 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.470 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.470 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.470 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.470 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.470 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.470 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.471 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.471 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.471 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.471 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.471 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.471 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.472 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.472 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.472 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.472 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.473 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.473 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.473 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.473 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.473 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.473 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.474 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.474 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.474 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.474 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.475 INFO:teuthology.orchestra.run.smithi044.stdout: "noautoscale": { 2023-09-10T01:37:06.475 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "noautoscale", 2023-09-10T01:37:06.475 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.475 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.475 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.475 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.476 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.476 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.476 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.476 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "global autoscale flag", 2023-09-10T01:37:06.476 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-10T01:37:06.476 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.477 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.477 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2023-09-10T01:37:06.477 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2023-09-10T01:37:06.477 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:06.478 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.478 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.478 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2023-09-10T01:37:06.478 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.478 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.478 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.479 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.479 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.479 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.479 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.479 INFO:teuthology.orchestra.run.smithi044.stdout: "threshold": { 2023-09-10T01:37:06.480 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threshold", 2023-09-10T01:37:06.480 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:06.480 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.480 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.480 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2023-09-10T01:37:06.480 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1.0", 2023-09-10T01:37:06.481 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.481 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.481 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "scaling threshold", 2023-09-10T01:37:06.481 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-10T01:37:06.481 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.481 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.482 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.482 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.482 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "progress", 2023-09-10T01:37:06.483 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.483 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.483 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.483 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_pg_recovery_event": { 2023-09-10T01:37:06.483 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_pg_recovery_event", 2023-09-10T01:37:06.484 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.484 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.484 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.484 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.484 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.485 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.485 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.485 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-10T01:37:06.485 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.485 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.485 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.486 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2023-09-10T01:37:06.486 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2023-09-10T01:37:06.486 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.486 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.487 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.487 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.487 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.487 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.487 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.487 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.488 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.488 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.488 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.488 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.489 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.489 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.489 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.489 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.489 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.490 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.490 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.490 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.490 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.490 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.490 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.491 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.491 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.491 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.491 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.491 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.492 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.492 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.492 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.492 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.492 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.493 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.493 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.493 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.493 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.493 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.493 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.494 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.494 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.494 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.494 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.494 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.495 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.495 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.495 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.495 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.495 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.496 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.496 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.496 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.496 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.496 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.496 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.497 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.497 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.497 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.497 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.497 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.497 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.498 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.498 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.498 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.498 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.498 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.499 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.499 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.499 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.499 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.499 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.499 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.500 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.500 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.500 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.500 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.500 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.501 INFO:teuthology.orchestra.run.smithi044.stdout: "max_completed_events": { 2023-09-10T01:37:06.501 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_completed_events", 2023-09-10T01:37:06.501 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.501 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.501 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.502 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "50", 2023-09-10T01:37:06.502 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.502 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.502 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.502 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "number of past completed events to remember", 2023-09-10T01:37:06.502 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.503 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.503 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.503 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2023-09-10T01:37:06.503 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2023-09-10T01:37:06.503 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:06.504 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.504 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.504 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2023-09-10T01:37:06.504 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.504 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.505 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.505 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long the module is going to sleep", 2023-09-10T01:37:06.505 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.505 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.505 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.505 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.506 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.506 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus", 2023-09-10T01:37:06.506 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.507 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.507 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.507 INFO:teuthology.orchestra.run.smithi044.stdout: "cache": { 2023-09-10T01:37:06.507 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cache", 2023-09-10T01:37:06.507 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.507 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.508 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.508 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.508 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.508 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.508 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.508 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.509 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.509 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.509 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.509 INFO:teuthology.orchestra.run.smithi044.stdout: "exclude_perf_counters": { 2023-09-10T01:37:06.510 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "exclude_perf_counters", 2023-09-10T01:37:06.510 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.510 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.510 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.510 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.510 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.511 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.511 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.511 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-10T01:37:06.511 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-10T01:37:06.511 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.512 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.512 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.512 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.512 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.512 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.513 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.513 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.513 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.513 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.513 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.513 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.514 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.514 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.514 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.514 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.514 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.514 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.515 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.515 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.515 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.515 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.515 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.516 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.516 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.516 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.516 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.516 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.516 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.517 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.517 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.517 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.517 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.517 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.517 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.518 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.518 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.518 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.518 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.519 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.519 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.519 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.519 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.519 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.519 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.520 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.520 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.520 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.520 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.520 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.520 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.521 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.521 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.521 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.521 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.522 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.522 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.522 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.522 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.522 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.522 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.523 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.523 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.523 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.523 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.523 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.523 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.524 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.524 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools": { 2023-09-10T01:37:06.524 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools", 2023-09-10T01:37:06.524 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.524 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.525 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.525 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.525 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.525 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.525 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.525 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.526 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.526 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.526 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.526 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-10T01:37:06.526 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-10T01:37:06.527 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.527 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.527 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.527 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "300", 2023-09-10T01:37:06.527 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.527 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.528 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.528 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.528 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.528 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.528 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.529 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_interval": { 2023-09-10T01:37:06.529 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_interval", 2023-09-10T01:37:06.529 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:06.529 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.529 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.529 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15.0", 2023-09-10T01:37:06.530 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.530 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.530 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.530 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.530 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.531 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.531 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.531 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2023-09-10T01:37:06.531 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2023-09-10T01:37:06.531 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.532 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.532 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.532 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2023-09-10T01:37:06.532 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.532 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.532 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.533 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-10T01:37:06.533 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.533 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.533 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.533 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2023-09-10T01:37:06.534 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2023-09-10T01:37:06.534 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.534 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.534 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.534 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "9283", 2023-09-10T01:37:06.535 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.535 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.535 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.535 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-10T01:37:06.535 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.535 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.536 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.536 INFO:teuthology.orchestra.run.smithi044.stdout: "stale_cache_strategy": { 2023-09-10T01:37:06.536 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stale_cache_strategy", 2023-09-10T01:37:06.536 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.537 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.537 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.537 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "log", 2023-09-10T01:37:06.537 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.537 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.537 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.538 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.538 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.538 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.538 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.538 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2023-09-10T01:37:06.539 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2023-09-10T01:37:06.539 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.539 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.539 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.539 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "default", 2023-09-10T01:37:06.540 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.540 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.540 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.540 INFO:teuthology.orchestra.run.smithi044.stdout: "default", 2023-09-10T01:37:06.540 INFO:teuthology.orchestra.run.smithi044.stdout: "error" 2023-09-10T01:37:06.540 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.541 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.541 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.541 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.541 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.542 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2023-09-10T01:37:06.542 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2023-09-10T01:37:06.542 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.542 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.542 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.543 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2023-09-10T01:37:06.543 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2023-09-10T01:37:06.543 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2023-09-10T01:37:06.543 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.543 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.543 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.544 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.544 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.544 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.544 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.545 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_support", 2023-09-10T01:37:06.545 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.545 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.545 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.545 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.546 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.546 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.546 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.546 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.546 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.546 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.547 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.547 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.547 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.547 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.547 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.548 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.548 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.548 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.548 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.548 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.549 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.549 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.549 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.549 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.550 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.550 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.550 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.550 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.550 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.550 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.551 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.551 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.551 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.551 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.551 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.552 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.552 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.552 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.552 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.553 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.553 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.553 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.553 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.553 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.553 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.554 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.554 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.554 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.554 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.554 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.554 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.555 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.555 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.555 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.555 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.555 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.556 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.556 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.556 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.556 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.556 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.557 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.557 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.557 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.557 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.557 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.558 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.558 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.558 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.558 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_snap_create": { 2023-09-10T01:37:06.559 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_snap_create", 2023-09-10T01:37:06.559 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.559 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.559 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.559 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:06.560 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.560 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.560 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.560 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.560 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.560 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.561 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.561 INFO:teuthology.orchestra.run.smithi044.stdout: "mirror_snapshot_schedule": { 2023-09-10T01:37:06.561 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirror_snapshot_schedule", 2023-09-10T01:37:06.561 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.562 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.562 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.562 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.562 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.562 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.563 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.563 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.563 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.563 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.563 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.564 INFO:teuthology.orchestra.run.smithi044.stdout: "trash_purge_schedule": { 2023-09-10T01:37:06.564 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "trash_purge_schedule", 2023-09-10T01:37:06.564 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.564 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.564 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.564 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.565 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.565 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.565 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.565 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.565 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.566 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.566 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.566 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.566 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.567 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "restful", 2023-09-10T01:37:06.567 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.567 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.567 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.567 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_auth": { 2023-09-10T01:37:06.568 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_auth", 2023-09-10T01:37:06.568 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.568 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.568 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.568 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.568 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.569 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.569 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.569 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.569 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.569 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.570 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.570 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2023-09-10T01:37:06.570 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2023-09-10T01:37:06.570 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.571 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.571 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.571 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.571 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.571 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.571 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.572 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.572 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.572 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.572 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.573 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.573 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.573 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.573 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.573 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.573 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.574 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.574 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.574 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.574 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.574 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.575 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.575 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.575 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.575 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.575 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.575 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.576 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.576 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.576 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.576 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.577 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.577 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.577 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.577 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.577 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.577 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.578 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.578 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.578 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.578 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.578 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.579 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.579 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.579 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.579 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.579 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.580 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.580 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.580 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.580 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.580 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.580 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.581 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.581 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.581 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.581 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.581 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.582 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.582 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.582 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.582 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.582 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.583 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.583 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.583 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.583 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.583 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.584 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.584 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.584 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.584 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.584 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.584 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.585 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.585 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.585 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2023-09-10T01:37:06.586 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2023-09-10T01:37:06.586 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.586 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.586 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.586 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.587 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.587 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.587 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.587 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.587 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.588 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.588 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.588 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2023-09-10T01:37:06.588 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2023-09-10T01:37:06.588 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.589 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.589 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.589 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.589 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.589 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.589 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.590 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.590 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.590 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.590 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.590 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.591 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.591 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rgw", 2023-09-10T01:37:06.591 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.591 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.592 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.592 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.592 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.592 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.592 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.593 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.593 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.593 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.593 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.593 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.593 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.594 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.594 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.594 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.594 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.594 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.595 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.595 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.595 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.595 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.595 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.596 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.596 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.596 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.596 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.596 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.597 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.597 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.597 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.597 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.597 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.597 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.598 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.598 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.598 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.598 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.599 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.599 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.599 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.599 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.599 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.599 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.600 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.600 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.600 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.600 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.600 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.601 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.601 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.601 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.601 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.601 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.601 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.602 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.602 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.602 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.602 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.603 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.603 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.603 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.603 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.603 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.603 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.604 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.604 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.604 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.604 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.604 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.604 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.605 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rook", 2023-09-10T01:37:06.605 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.605 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.606 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.606 INFO:teuthology.orchestra.run.smithi044.stdout: "drive_group_interval": { 2023-09-10T01:37:06.606 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "drive_group_interval", 2023-09-10T01:37:06.606 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:06.606 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.606 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.607 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "300.0", 2023-09-10T01:37:06.607 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.607 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.607 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.607 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-10T01:37:06.607 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.608 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.608 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.608 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.608 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.609 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.609 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.609 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.609 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.609 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.609 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.610 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.610 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.610 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.610 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.610 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.610 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.611 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.611 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.611 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.611 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.611 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.612 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.612 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.612 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.612 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.612 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.613 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.613 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.613 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.613 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.613 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.614 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.614 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.614 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.614 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.614 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.615 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.615 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.615 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.615 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.615 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.616 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.616 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.616 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.616 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.616 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.617 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.617 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.617 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.617 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.617 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.617 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.618 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.618 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.618 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.618 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.619 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.619 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.619 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.619 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.619 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.619 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.620 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.620 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.620 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.620 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.620 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.621 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.621 INFO:teuthology.orchestra.run.smithi044.stdout: "storage_class": { 2023-09-10T01:37:06.621 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "storage_class", 2023-09-10T01:37:06.621 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.621 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.622 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.622 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "local", 2023-09-10T01:37:06.622 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.622 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.622 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.623 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-10T01:37:06.623 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.623 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.623 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.623 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.623 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.624 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "selftest", 2023-09-10T01:37:06.624 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.624 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.625 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.625 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.625 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.625 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.625 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.625 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.626 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.626 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.626 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.626 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.626 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.626 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.627 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.627 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.627 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.627 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.627 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.628 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.628 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.628 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.628 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.629 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.629 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.629 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.629 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.629 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.629 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.630 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.630 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.630 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.630 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.630 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.631 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.631 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.631 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.631 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.631 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.632 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.632 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.632 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.632 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.632 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.633 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.633 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.633 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.633 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.633 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.633 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.634 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.634 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.634 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.634 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.634 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.635 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.635 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.635 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.635 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.635 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.636 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.636 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.636 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.636 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.636 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.637 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.637 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.637 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.637 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.637 INFO:teuthology.orchestra.run.smithi044.stdout: "roption1": { 2023-09-10T01:37:06.638 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption1", 2023-09-10T01:37:06.638 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.638 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.638 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.638 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.639 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.639 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.639 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.639 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.639 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.639 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.640 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.640 INFO:teuthology.orchestra.run.smithi044.stdout: "roption2": { 2023-09-10T01:37:06.640 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption2", 2023-09-10T01:37:06.640 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.641 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.641 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.641 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "xyz", 2023-09-10T01:37:06.641 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.641 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.641 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.642 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.642 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.642 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.642 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.643 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption1": { 2023-09-10T01:37:06.643 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption1", 2023-09-10T01:37:06.643 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.643 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.643 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.643 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.644 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.644 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.644 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.644 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.644 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.644 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.645 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.645 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption2": { 2023-09-10T01:37:06.645 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption2", 2023-09-10T01:37:06.645 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.646 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.646 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.646 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.646 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.646 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.646 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.647 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.647 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.647 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.647 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.648 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption3": { 2023-09-10T01:37:06.648 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption3", 2023-09-10T01:37:06.648 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:06.648 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.648 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.649 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.649 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.649 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.649 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.649 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.649 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.650 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.650 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.650 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption4": { 2023-09-10T01:37:06.650 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption4", 2023-09-10T01:37:06.651 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.651 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.651 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.651 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.651 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.651 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.652 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.652 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.652 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.652 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.652 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.653 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption5": { 2023-09-10T01:37:06.653 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption5", 2023-09-10T01:37:06.653 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.653 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.653 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.654 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.654 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.654 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.654 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.654 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.655 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.655 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.655 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.655 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption6": { 2023-09-10T01:37:06.655 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption6", 2023-09-10T01:37:06.656 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.656 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.656 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.656 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.656 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.657 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.657 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.657 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.657 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.657 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.657 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.658 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption7": { 2023-09-10T01:37:06.658 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption7", 2023-09-10T01:37:06.658 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.658 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.659 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.659 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.659 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2023-09-10T01:37:06.659 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "42", 2023-09-10T01:37:06.659 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.659 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.660 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.660 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.660 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.660 INFO:teuthology.orchestra.run.smithi044.stdout: "testkey": { 2023-09-10T01:37:06.661 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testkey", 2023-09-10T01:37:06.661 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.661 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.661 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.661 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.662 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.662 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.662 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.662 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.662 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.663 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.663 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.663 INFO:teuthology.orchestra.run.smithi044.stdout: "testlkey": { 2023-09-10T01:37:06.663 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testlkey", 2023-09-10T01:37:06.663 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.664 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.664 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.664 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.664 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.664 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.665 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.665 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.665 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.665 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.665 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.666 INFO:teuthology.orchestra.run.smithi044.stdout: "testnewline": { 2023-09-10T01:37:06.666 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testnewline", 2023-09-10T01:37:06.666 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.666 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.666 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.667 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.667 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.667 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.667 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.667 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.668 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.668 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.668 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.668 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.668 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.669 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2023-09-10T01:37:06.669 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.669 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.669 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.670 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_m_granularity": { 2023-09-10T01:37:06.670 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_m_granularity", 2023-09-10T01:37:06.670 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.670 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.670 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.671 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.671 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.671 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.671 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.671 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow minute scheduled snapshots", 2023-09-10T01:37:06.672 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.672 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.672 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.672 INFO:teuthology.orchestra.run.smithi044.stdout: "dump_on_update": { 2023-09-10T01:37:06.672 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dump_on_update", 2023-09-10T01:37:06.673 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.673 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.673 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.673 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.673 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.673 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.674 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.674 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "dump database to debug log on update", 2023-09-10T01:37:06.674 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.674 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.674 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.675 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.675 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.675 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.675 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.676 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.676 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.676 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.676 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.676 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.677 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.677 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.677 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.677 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.677 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.677 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.678 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.678 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.678 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.678 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.678 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.679 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.679 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.679 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.679 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.679 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.680 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.680 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.680 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.680 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.680 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.681 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.681 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.681 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.681 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.682 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.682 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.682 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.682 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.682 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.682 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.683 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.683 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.683 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.683 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.683 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.684 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.684 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.684 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.684 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.684 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.685 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.685 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.685 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.685 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.685 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.686 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.686 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.686 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.686 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.686 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.687 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.687 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.687 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.687 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.687 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.687 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.688 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.688 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.688 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stats", 2023-09-10T01:37:06.689 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.689 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.689 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.689 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.689 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.690 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.690 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.690 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.690 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.690 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.690 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.691 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.691 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.691 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.691 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.691 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.692 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.692 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.692 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.692 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.692 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.693 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.693 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.693 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.693 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.693 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.694 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.694 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.694 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.694 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.694 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.695 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.695 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.695 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.695 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.695 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.696 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.696 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.696 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.696 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.696 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.697 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.697 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.697 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.697 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.697 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.697 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.698 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.698 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.698 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.698 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.698 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.698 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.699 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.699 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.699 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.699 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.699 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.700 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.700 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.700 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.700 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.700 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.701 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.701 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.701 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.701 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.701 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.702 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.702 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.702 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.702 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "status", 2023-09-10T01:37:06.703 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.703 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.703 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.703 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.703 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.703 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.704 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.704 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.704 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.704 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.704 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.704 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.705 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.705 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.705 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.705 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.705 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.706 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.706 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.706 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.706 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.706 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.706 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.707 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.707 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.707 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.707 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.707 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.708 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.708 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.708 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.708 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.708 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.708 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.709 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.709 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.709 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.709 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.709 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.710 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.710 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.710 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.710 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.710 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.711 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.711 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.711 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.711 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.711 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.711 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.712 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.712 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.712 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.712 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.712 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.713 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.713 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.713 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.713 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.713 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.714 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.714 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.714 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.714 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.714 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.714 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.715 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.715 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.715 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.715 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.715 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.716 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telegraf", 2023-09-10T01:37:06.716 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.716 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.716 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.717 INFO:teuthology.orchestra.run.smithi044.stdout: "address": { 2023-09-10T01:37:06.717 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "address", 2023-09-10T01:37:06.717 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.717 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.717 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.717 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-10T01:37:06.718 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.718 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.718 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.718 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.718 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.719 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.719 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.719 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2023-09-10T01:37:06.719 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2023-09-10T01:37:06.719 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:06.720 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.720 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.720 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15", 2023-09-10T01:37:06.720 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.720 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.720 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.721 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.721 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.721 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.721 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.722 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.722 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.722 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.722 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.722 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.722 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.723 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.723 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.723 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.723 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.723 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.723 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.724 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.724 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.724 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.724 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.724 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.725 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.725 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.725 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.725 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.725 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.726 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.726 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.726 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.726 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.726 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.726 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.727 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.727 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.727 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.727 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.727 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.728 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.728 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.728 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.728 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.728 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.729 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.729 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.729 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.729 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.729 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.730 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.730 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.730 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.730 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.730 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.730 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.731 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.731 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.731 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.731 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.731 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.732 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.732 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.732 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.732 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.732 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.733 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.733 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.733 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.733 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.733 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.733 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.734 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.734 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.734 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.735 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telemetry", 2023-09-10T01:37:06.735 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.735 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.735 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.735 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_basic": { 2023-09-10T01:37:06.735 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_basic", 2023-09-10T01:37:06.736 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.736 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.736 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.736 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.736 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.736 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.736 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.737 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-10T01:37:06.737 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.737 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.737 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.738 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_crash": { 2023-09-10T01:37:06.738 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_crash", 2023-09-10T01:37:06.738 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.738 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.738 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.738 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.739 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.739 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.739 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.739 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-10T01:37:06.739 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.739 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.740 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.740 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_device": { 2023-09-10T01:37:06.740 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_device", 2023-09-10T01:37:06.740 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.740 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.741 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.741 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.741 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.741 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.741 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.741 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-10T01:37:06.742 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.742 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.742 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.742 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_ident": { 2023-09-10T01:37:06.743 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_ident", 2023-09-10T01:37:06.743 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.743 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.743 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.743 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.743 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.744 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.744 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.744 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-10T01:37:06.744 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.744 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.744 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.745 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_perf": { 2023-09-10T01:37:06.745 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_perf", 2023-09-10T01:37:06.745 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.745 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.745 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.746 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.746 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.746 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.746 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.746 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-10T01:37:06.747 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.747 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.747 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.749 INFO:teuthology.orchestra.run.smithi044.stdout: "contact": { 2023-09-10T01:37:06.749 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "contact", 2023-09-10T01:37:06.749 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.749 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.749 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.749 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.750 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.750 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.750 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.750 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.750 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.751 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.751 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.751 INFO:teuthology.orchestra.run.smithi044.stdout: "description": { 2023-09-10T01:37:06.751 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "description", 2023-09-10T01:37:06.751 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.752 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.752 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.752 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.752 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.752 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.753 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.753 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.753 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.753 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.753 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.754 INFO:teuthology.orchestra.run.smithi044.stdout: "device_url": { 2023-09-10T01:37:06.754 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_url", 2023-09-10T01:37:06.754 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.754 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.755 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.755 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-10T01:37:06.755 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.755 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.755 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.755 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.756 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.756 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.756 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.756 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2023-09-10T01:37:06.756 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2023-09-10T01:37:06.757 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.757 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.757 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.757 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.757 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.757 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.758 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.758 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.758 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.758 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.758 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.759 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2023-09-10T01:37:06.759 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2023-09-10T01:37:06.759 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.759 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.759 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.760 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "24", 2023-09-10T01:37:06.760 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "8", 2023-09-10T01:37:06.760 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.760 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.760 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.760 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.761 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.761 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.761 INFO:teuthology.orchestra.run.smithi044.stdout: "last_opt_revision": { 2023-09-10T01:37:06.761 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "last_opt_revision", 2023-09-10T01:37:06.762 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.762 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.762 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.762 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1", 2023-09-10T01:37:06.762 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.762 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.763 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.763 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.763 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.763 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.763 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.764 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard": { 2023-09-10T01:37:06.764 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard", 2023-09-10T01:37:06.764 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.764 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.764 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.764 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.765 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.765 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.765 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.765 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.765 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.765 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.766 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.766 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard_description": { 2023-09-10T01:37:06.766 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard_description", 2023-09-10T01:37:06.766 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.766 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.767 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.767 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.767 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.767 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.767 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.767 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.768 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.768 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.768 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.768 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.768 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.768 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.769 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.769 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.769 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.769 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.769 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.770 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.770 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.770 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.770 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.770 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.770 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.771 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.771 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.771 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.771 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.771 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.771 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.772 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.772 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.772 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.772 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.772 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.773 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.773 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.773 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.773 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.773 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.773 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.774 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.774 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.774 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.774 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.774 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.775 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.775 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.775 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.775 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.775 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.776 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.776 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.776 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.776 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.776 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.776 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.776 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.777 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.777 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.777 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.777 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.777 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.778 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.778 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.778 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.778 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.778 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.779 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.779 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.779 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.779 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.779 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.779 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.780 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.780 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.780 INFO:teuthology.orchestra.run.smithi044.stdout: "organization": { 2023-09-10T01:37:06.780 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "organization", 2023-09-10T01:37:06.780 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.781 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.781 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.781 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.781 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.781 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.781 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.782 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.782 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.782 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.782 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.782 INFO:teuthology.orchestra.run.smithi044.stdout: "proxy": { 2023-09-10T01:37:06.783 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "proxy", 2023-09-10T01:37:06.783 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.783 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.783 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.783 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.784 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.784 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.784 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.784 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.784 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.784 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.785 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.785 INFO:teuthology.orchestra.run.smithi044.stdout: "url": { 2023-09-10T01:37:06.785 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url", 2023-09-10T01:37:06.785 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.786 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.786 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.786 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-10T01:37:06.786 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.786 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.786 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.787 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.787 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.787 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.787 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.787 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.788 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.788 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "test_orchestrator", 2023-09-10T01:37:06.788 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.788 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.789 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.789 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.789 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.789 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.789 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.790 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.790 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.790 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.790 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.790 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.791 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.791 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.791 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.791 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.791 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.791 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.792 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.792 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.792 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.792 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.792 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.793 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.793 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.793 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.793 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.793 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.794 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.794 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.794 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.794 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.794 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.795 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.795 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.795 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.795 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.795 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.796 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.796 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.796 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.796 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.796 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.797 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.797 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.797 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.797 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.797 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.797 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.798 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.798 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.798 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.798 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.798 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.798 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.799 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.799 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.799 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.799 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.799 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.799 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.800 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.800 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.800 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.800 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.800 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.800 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.801 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.801 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.801 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.801 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.802 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "volumes", 2023-09-10T01:37:06.802 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.802 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.802 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.802 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.802 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.803 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.803 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.803 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.803 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.803 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.803 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.804 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.804 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.804 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.804 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.804 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.804 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.805 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.805 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.805 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.805 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.805 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.805 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.806 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.806 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.806 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.806 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.806 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.806 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.807 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.807 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.807 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.807 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.807 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.808 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.808 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.808 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.808 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.808 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.808 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.808 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.809 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.809 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.809 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.809 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.809 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.809 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.810 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.810 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.810 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.810 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.810 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.811 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.811 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.811 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.811 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.811 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.811 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.812 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.812 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.812 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.812 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.812 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.813 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.813 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.813 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.813 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.813 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.813 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.814 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.814 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.814 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_clones": { 2023-09-10T01:37:06.814 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_clones", 2023-09-10T01:37:06.815 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.815 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.815 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.815 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4", 2023-09-10T01:37:06.815 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.815 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.815 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.816 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-10T01:37:06.816 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.816 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.816 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.816 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_delay": { 2023-09-10T01:37:06.817 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_delay", 2023-09-10T01:37:06.817 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.817 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.817 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.817 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2023-09-10T01:37:06.818 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.818 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.818 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.818 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-10T01:37:06.818 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.819 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.819 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.819 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.819 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.819 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix", 2023-09-10T01:37:06.820 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.820 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.820 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.820 INFO:teuthology.orchestra.run.smithi044.stdout: "discovery_interval": { 2023-09-10T01:37:06.820 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "discovery_interval", 2023-09-10T01:37:06.820 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2023-09-10T01:37:06.821 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.821 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.821 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "100", 2023-09-10T01:37:06.821 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.821 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.822 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.822 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.822 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.822 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.822 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.823 INFO:teuthology.orchestra.run.smithi044.stdout: "identifier": { 2023-09-10T01:37:06.823 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "identifier", 2023-09-10T01:37:06.823 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.823 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.823 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.823 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.824 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.824 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.824 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.824 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.824 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.824 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.825 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.825 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2023-09-10T01:37:06.825 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2023-09-10T01:37:06.825 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:06.826 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.826 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.826 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2023-09-10T01:37:06.826 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.826 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.827 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.827 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.827 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.827 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.827 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.828 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.828 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.828 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.828 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.828 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.829 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.829 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.829 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.829 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.829 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.829 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.830 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.830 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.830 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.830 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.830 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.831 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.831 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.831 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.831 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.831 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.832 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.832 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.832 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.832 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.832 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.833 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.833 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.833 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.833 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.833 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.833 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.834 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.834 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.834 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.834 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.834 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.835 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.835 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.835 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.835 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.835 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.836 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.836 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.836 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.836 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.836 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.836 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.837 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.837 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.837 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.837 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.837 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.837 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.838 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.838 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.838 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.838 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.838 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.839 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.839 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.839 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.839 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.839 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.839 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.840 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.840 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.840 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_host": { 2023-09-10T01:37:06.840 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_host", 2023-09-10T01:37:06.841 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.841 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.841 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.841 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.841 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.841 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.842 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.842 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.842 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.842 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.843 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.843 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_port": { 2023-09-10T01:37:06.844 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_port", 2023-09-10T01:37:06.844 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.844 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.844 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.844 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10051", 2023-09-10T01:37:06.845 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.845 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.845 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.845 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.845 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.846 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.846 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.846 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_sender": { 2023-09-10T01:37:06.847 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_sender", 2023-09-10T01:37:06.847 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.847 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.847 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.847 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-10T01:37:06.848 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.848 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.848 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.848 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.849 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.849 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.849 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.849 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.849 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.850 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.850 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2023-09-10T01:37:06.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.850 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.850 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 25505, 2023-09-10T01:37:06.851 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "y", 2023-09-10T01:37:06.851 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_features": 4540138322906710015, 2023-09-10T01:37:06.851 INFO:teuthology.orchestra.run.smithi044.stdout: "available_modules": [ 2023-09-10T01:37:06.851 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.851 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "alerts", 2023-09-10T01:37:06.852 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.852 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.852 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.852 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2023-09-10T01:37:06.852 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2023-09-10T01:37:06.853 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:06.853 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.853 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.853 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2023-09-10T01:37:06.853 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.854 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.854 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.854 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How frequently to reexamine health status", 2023-09-10T01:37:06.854 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.854 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.855 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.855 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.855 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.855 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.856 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.856 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.856 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.856 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.856 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.857 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.857 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.857 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.857 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.857 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.858 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.858 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.858 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.858 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.859 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.859 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.859 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.859 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.860 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.860 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.860 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.860 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.861 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.861 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.861 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.861 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.861 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.862 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.862 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.862 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.863 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.863 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.863 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.863 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.863 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.863 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.864 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.864 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.864 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.864 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.864 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.865 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.865 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.865 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.865 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.865 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.866 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.866 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.866 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.866 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.867 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.867 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.867 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.867 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.867 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.868 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.868 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.868 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.868 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.868 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.869 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.869 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.869 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.869 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_destination": { 2023-09-10T01:37:06.870 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_destination", 2023-09-10T01:37:06.870 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.870 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.870 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.870 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.871 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.871 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.871 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.871 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email address to send alerts to", 2023-09-10T01:37:06.872 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.872 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.872 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.872 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_from_name": { 2023-09-10T01:37:06.873 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_from_name", 2023-09-10T01:37:06.873 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.873 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.873 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.873 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "Ceph", 2023-09-10T01:37:06.874 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.874 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.874 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.874 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email From: name", 2023-09-10T01:37:06.874 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.875 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.875 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.875 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_host": { 2023-09-10T01:37:06.875 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_host", 2023-09-10T01:37:06.876 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.876 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.876 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.876 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.877 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.877 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.877 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.877 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP server", 2023-09-10T01:37:06.878 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.878 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.878 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.878 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_password": { 2023-09-10T01:37:06.879 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_password", 2023-09-10T01:37:06.879 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.879 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.879 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.879 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.880 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.880 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.880 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.880 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Password to authenticate with", 2023-09-10T01:37:06.880 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.880 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.881 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.881 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_port": { 2023-09-10T01:37:06.881 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_port", 2023-09-10T01:37:06.882 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.882 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.882 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.882 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "465", 2023-09-10T01:37:06.882 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.883 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.883 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.883 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP port", 2023-09-10T01:37:06.883 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.883 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.884 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.884 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_sender": { 2023-09-10T01:37:06.884 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_sender", 2023-09-10T01:37:06.884 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.885 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.885 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.885 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.885 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.886 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.886 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.886 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP envelope sender", 2023-09-10T01:37:06.886 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.886 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.887 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.887 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_ssl": { 2023-09-10T01:37:06.887 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_ssl", 2023-09-10T01:37:06.887 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.888 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.888 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.888 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.888 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.889 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.889 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.889 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-10T01:37:06.889 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.889 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.890 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.890 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_user": { 2023-09-10T01:37:06.890 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_user", 2023-09-10T01:37:06.890 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.891 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.891 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.891 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.891 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.891 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.892 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.892 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "User to authenticate as", 2023-09-10T01:37:06.892 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.892 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.893 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.893 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.893 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.894 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "balancer", 2023-09-10T01:37:06.894 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.894 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.894 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.894 INFO:teuthology.orchestra.run.smithi044.stdout: "active": { 2023-09-10T01:37:06.895 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "active", 2023-09-10T01:37:06.895 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.895 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.895 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.896 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.896 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.896 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.896 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.896 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically balance PGs across cluster", 2023-09-10T01:37:06.897 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.897 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.897 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.897 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_time": { 2023-09-10T01:37:06.898 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_time", 2023-09-10T01:37:06.898 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.898 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.898 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.899 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0000", 2023-09-10T01:37:06.899 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.899 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.899 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.899 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "beginning time of day to automatically balance", 2023-09-10T01:37:06.900 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-10T01:37:06.900 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.900 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.900 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_weekday": { 2023-09-10T01:37:06.901 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_weekday", 2023-09-10T01:37:06.901 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2023-09-10T01:37:06.901 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.901 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.901 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2023-09-10T01:37:06.902 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2023-09-10T01:37:06.902 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2023-09-10T01:37:06.902 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.902 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-10T01:37:06.903 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-10T01:37:06.903 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.903 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.903 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_max_iterations": { 2023-09-10T01:37:06.904 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_max_iterations", 2023-09-10T01:37:06.904 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2023-09-10T01:37:06.904 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.904 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.905 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "25", 2023-09-10T01:37:06.905 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2023-09-10T01:37:06.905 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "250", 2023-09-10T01:37:06.905 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.905 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-10T01:37:06.906 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.906 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.906 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.907 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_metrics": { 2023-09-10T01:37:06.907 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_metrics", 2023-09-10T01:37:06.907 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.907 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.907 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.908 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "pgs,objects,bytes", 2023-09-10T01:37:06.908 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.908 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.908 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.909 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-10T01:37:06.909 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-10T01:37:06.909 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.909 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.910 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_step": { 2023-09-10T01:37:06.910 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_step", 2023-09-10T01:37:06.910 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:06.910 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.910 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.911 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.5", 2023-09-10T01:37:06.911 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0.001", 2023-09-10T01:37:06.911 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "0.999", 2023-09-10T01:37:06.911 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.911 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "aggressiveness of optimization", 2023-09-10T01:37:06.912 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-10T01:37:06.912 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.912 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.912 INFO:teuthology.orchestra.run.smithi044.stdout: "end_time": { 2023-09-10T01:37:06.913 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_time", 2023-09-10T01:37:06.913 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.913 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.913 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.913 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2359", 2023-09-10T01:37:06.914 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.914 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.914 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.914 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ending time of day to automatically balance", 2023-09-10T01:37:06.915 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-10T01:37:06.915 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.915 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.915 INFO:teuthology.orchestra.run.smithi044.stdout: "end_weekday": { 2023-09-10T01:37:06.916 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_weekday", 2023-09-10T01:37:06.916 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2023-09-10T01:37:06.916 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.916 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.916 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2023-09-10T01:37:06.917 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2023-09-10T01:37:06.917 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2023-09-10T01:37:06.917 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.917 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-10T01:37:06.917 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-10T01:37:06.918 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.918 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.918 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:06.918 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:06.919 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.919 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.919 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.919 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.920 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.920 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.920 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.920 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.920 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.921 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.921 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.921 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.921 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.921 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.922 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.922 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.922 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.922 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.923 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:06.923 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:06.923 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.923 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.924 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.924 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.924 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.924 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.924 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.924 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.925 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.925 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.925 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.925 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:06.926 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:06.926 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.926 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.926 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.927 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:06.927 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.927 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.927 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.927 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:06.928 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:06.928 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:06.928 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:06.928 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:06.928 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:06.929 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.929 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.929 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.929 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.929 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.930 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:06.930 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:06.930 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.931 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.931 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.931 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.931 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.931 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.932 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.932 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:06.932 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.932 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.932 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.933 INFO:teuthology.orchestra.run.smithi044.stdout: "min_score": { 2023-09-10T01:37:06.933 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_score", 2023-09-10T01:37:06.933 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:06.933 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.934 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.934 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2023-09-10T01:37:06.934 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.934 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.934 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.935 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-10T01:37:06.935 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.935 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.935 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.936 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2023-09-10T01:37:06.936 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2023-09-10T01:37:06.936 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.936 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.936 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.937 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "upmap", 2023-09-10T01:37:06.937 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.937 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.937 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:06.937 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-compat", 2023-09-10T01:37:06.938 INFO:teuthology.orchestra.run.smithi044.stdout: "none", 2023-09-10T01:37:06.938 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap" 2023-09-10T01:37:06.938 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:06.938 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Balancer mode", 2023-09-10T01:37:06.938 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.939 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.939 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.939 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_ids": { 2023-09-10T01:37:06.940 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_ids", 2023-09-10T01:37:06.940 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.940 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.940 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.941 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:06.941 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.941 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.941 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.941 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-10T01:37:06.942 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.942 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.942 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.943 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2023-09-10T01:37:06.943 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2023-09-10T01:37:06.943 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:06.943 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.943 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.944 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2023-09-10T01:37:06.944 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.944 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.944 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.944 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-10T01:37:06.945 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.945 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.945 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.946 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_deviation": { 2023-09-10T01:37:06.946 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_deviation", 2023-09-10T01:37:06.946 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.946 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.947 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.947 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2023-09-10T01:37:06.947 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2023-09-10T01:37:06.947 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.947 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.948 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-10T01:37:06.948 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-10T01:37:06.948 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.948 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.949 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_optimizations": { 2023-09-10T01:37:06.949 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_optimizations", 2023-09-10T01:37:06.949 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2023-09-10T01:37:06.949 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.949 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.950 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:06.950 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.950 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.950 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.950 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-10T01:37:06.951 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.951 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.951 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.951 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.951 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:06.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:06.952 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm", 2023-09-10T01:37:06.952 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:06.952 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:06.953 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:06.953 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_down_multiplier": { 2023-09-10T01:37:06.953 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_down_multiplier", 2023-09-10T01:37:06.953 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:06.953 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.953 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.954 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2023-09-10T01:37:06.954 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.954 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.954 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.954 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-10T01:37:06.955 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.955 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.955 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.955 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_refresh_rate": { 2023-09-10T01:37:06.956 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_refresh_rate", 2023-09-10T01:37:06.956 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:06.956 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.956 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.956 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "20", 2023-09-10T01:37:06.956 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.957 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.957 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.957 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-10T01:37:06.957 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.958 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.958 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.958 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_starting_port": { 2023-09-10T01:37:06.958 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_starting_port", 2023-09-10T01:37:06.959 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:06.959 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.959 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.959 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4721", 2023-09-10T01:37:06.959 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.960 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.960 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.960 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-10T01:37:06.960 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.960 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.961 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.961 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_ptrace": { 2023-09-10T01:37:06.961 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_ptrace", 2023-09-10T01:37:06.961 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.962 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.962 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.962 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.962 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.963 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.963 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.963 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-10T01:37:06.963 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-10T01:37:06.964 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.964 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.964 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_interval": { 2023-09-10T01:37:06.964 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_interval", 2023-09-10T01:37:06.965 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:06.965 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.965 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.965 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2023-09-10T01:37:06.965 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.966 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.966 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.966 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-10T01:37:06.966 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.967 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.967 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.967 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_memory_target_ratio": { 2023-09-10T01:37:06.967 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_memory_target_ratio", 2023-09-10T01:37:06.968 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:06.968 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.968 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.968 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.7", 2023-09-10T01:37:06.968 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.969 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.969 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.969 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-10T01:37:06.969 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.969 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.970 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.970 INFO:teuthology.orchestra.run.smithi044.stdout: "cgroups_split": { 2023-09-10T01:37:06.970 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cgroups_split", 2023-09-10T01:37:06.971 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.971 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.971 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.971 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.971 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.972 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.972 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.972 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-10T01:37:06.972 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.973 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.973 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.973 INFO:teuthology.orchestra.run.smithi044.stdout: "config_checks_enabled": { 2023-09-10T01:37:06.973 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_checks_enabled", 2023-09-10T01:37:06.974 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.974 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.974 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.974 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:06.974 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.975 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.975 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.975 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-10T01:37:06.975 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.975 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.976 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.976 INFO:teuthology.orchestra.run.smithi044.stdout: "config_dashboard": { 2023-09-10T01:37:06.976 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_dashboard", 2023-09-10T01:37:06.976 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:06.977 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.977 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.977 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:06.977 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.977 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.978 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.978 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-10T01:37:06.978 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.978 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.979 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.979 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_alertmanager": { 2023-09-10T01:37:06.979 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_alertmanager", 2023-09-10T01:37:06.979 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.980 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.980 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.980 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-10T01:37:06.980 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.980 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.981 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.981 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2023-09-10T01:37:06.981 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.981 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.982 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.982 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_base": { 2023-09-10T01:37:06.982 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_base", 2023-09-10T01:37:06.982 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.983 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.983 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:06.983 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-10T01:37:06.983 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.984 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.984 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.984 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Container image name, without the tag", 2023-09-10T01:37:06.984 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.984 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.984 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.985 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_elasticsearch": { 2023-09-10T01:37:06.985 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_elasticsearch", 2023-09-10T01:37:06.985 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.985 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.986 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.986 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-10T01:37:06.986 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.986 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.987 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.987 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "elasticsearch container image", 2023-09-10T01:37:06.987 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.987 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.987 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.988 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_grafana": { 2023-09-10T01:37:06.988 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_grafana", 2023-09-10T01:37:06.988 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.988 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.988 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.989 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-10T01:37:06.989 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.989 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.989 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.989 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2023-09-10T01:37:06.990 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.990 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.990 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.990 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_haproxy": { 2023-09-10T01:37:06.991 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_haproxy", 2023-09-10T01:37:06.991 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.991 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.991 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.991 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-10T01:37:06.992 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.992 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.992 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.992 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "HAproxy container image", 2023-09-10T01:37:06.992 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.993 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.993 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.993 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_agent": { 2023-09-10T01:37:06.993 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_agent", 2023-09-10T01:37:06.993 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.994 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.994 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.994 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-10T01:37:06.994 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.994 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.995 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.995 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger agent container image", 2023-09-10T01:37:06.995 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.995 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.995 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.996 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_collector": { 2023-09-10T01:37:06.996 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_collector", 2023-09-10T01:37:06.996 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.996 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:06.997 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:06.997 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-10T01:37:06.997 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:06.997 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:06.997 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:06.998 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger collector container image", 2023-09-10T01:37:06.998 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:06.998 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:06.998 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:06.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:06.999 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_query": { 2023-09-10T01:37:06.999 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_query", 2023-09-10T01:37:06.999 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:06.999 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.000 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.000 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-10T01:37:07.000 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.000 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.000 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.001 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger query container image", 2023-09-10T01:37:07.001 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.001 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.001 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.002 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_keepalived": { 2023-09-10T01:37:07.002 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_keepalived", 2023-09-10T01:37:07.002 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.002 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.003 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.003 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-10T01:37:07.003 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.003 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.003 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.004 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Keepalived container image", 2023-09-10T01:37:07.004 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.004 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.004 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.005 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_loki": { 2023-09-10T01:37:07.005 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_loki", 2023-09-10T01:37:07.005 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.005 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.005 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.005 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-10T01:37:07.006 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.006 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.006 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.006 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Loki container image", 2023-09-10T01:37:07.007 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.007 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.007 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.007 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_node_exporter": { 2023-09-10T01:37:07.007 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_node_exporter", 2023-09-10T01:37:07.008 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.008 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.008 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.008 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-10T01:37:07.009 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.009 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.009 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.009 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2023-09-10T01:37:07.009 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.010 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.010 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.010 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_nvmeof": { 2023-09-10T01:37:07.010 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_nvmeof", 2023-09-10T01:37:07.011 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.011 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.011 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.011 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-10T01:37:07.011 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.012 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.012 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.012 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Nvme-of container image", 2023-09-10T01:37:07.012 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.012 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.013 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.013 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_prometheus": { 2023-09-10T01:37:07.013 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_prometheus", 2023-09-10T01:37:07.013 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.014 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.014 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.014 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-10T01:37:07.014 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.015 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.015 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.015 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2023-09-10T01:37:07.015 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.015 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.016 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.016 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_promtail": { 2023-09-10T01:37:07.016 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_promtail", 2023-09-10T01:37:07.016 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.017 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.017 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.017 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-10T01:37:07.017 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.017 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.018 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.018 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Promtail container image", 2023-09-10T01:37:07.018 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.018 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.018 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.019 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_snmp_gateway": { 2023-09-10T01:37:07.019 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_snmp_gateway", 2023-09-10T01:37:07.019 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.019 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.020 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.020 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-10T01:37:07.020 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.020 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.020 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SNMP Gateway container image", 2023-09-10T01:37:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "container_init": { 2023-09-10T01:37:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_init", 2023-09-10T01:37:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Run podman/docker with `--init`", 2023-09-10T01:37:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "daemon_cache_timeout": { 2023-09-10T01:37:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "daemon_cache_timeout", 2023-09-10T01:37:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2023-09-10T01:37:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-10T01:37:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: "default_cephadm_command_timeout": { 2023-09-10T01:37:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_cephadm_command_timeout", 2023-09-10T01:37:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "900", 2023-09-10T01:37:07.029 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.029 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.029 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.029 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-10T01:37:07.029 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "default_registry": { 2023-09-10T01:37:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_registry", 2023-09-10T01:37:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io", 2023-09-10T01:37:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-10T01:37:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "device_cache_timeout": { 2023-09-10T01:37:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_cache_timeout", 2023-09-10T01:37:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1800", 2023-09-10T01:37:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache device inventory", 2023-09-10T01:37:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "device_enhanced_scan": { 2023-09-10T01:37:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_enhanced_scan", 2023-09-10T01:37:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-10T01:37:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "facts_cache_timeout": { 2023-09-10T01:37:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "facts_cache_timeout", 2023-09-10T01:37:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2023-09-10T01:37:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache host facts data", 2023-09-10T01:37:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "host_check_interval": { 2023-09-10T01:37:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "host_check_interval", 2023-09-10T01:37:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2023-09-10T01:37:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to perform a host check", 2023-09-10T01:37:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "inventory_list_all": { 2023-09-10T01:37:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "inventory_list_all", 2023-09-10T01:37:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-10T01:37:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "log_refresh_metadata": { 2023-09-10T01:37:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_refresh_metadata", 2023-09-10T01:37:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-10T01:37:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-10T01:37:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-10T01:37:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-10T01:37:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.066 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.066 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.066 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.066 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-10T01:37:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-10T01:37:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-10T01:37:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "*", 2023-09-10T01:37:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-10T01:37:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.071 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.071 INFO:teuthology.orchestra.run.smithi044.stdout: "max_count_per_host": { 2023-09-10T01:37:07.071 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_count_per_host", 2023-09-10T01:37:07.071 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.072 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.072 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.072 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:07.072 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.072 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.073 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.073 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of daemons per service per host", 2023-09-10T01:37:07.073 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.073 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.073 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.074 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd_draining_count": { 2023-09-10T01:37:07.074 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_osd_draining_count", 2023-09-10T01:37:07.074 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.075 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.075 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.075 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:07.075 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.075 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.076 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.076 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-10T01:37:07.076 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.076 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.076 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.077 INFO:teuthology.orchestra.run.smithi044.stdout: "migration_current": { 2023-09-10T01:37:07.077 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "migration_current", 2023-09-10T01:37:07.077 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.077 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.078 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.078 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.078 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.078 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.078 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.079 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "internal - do not modify", 2023-09-10T01:37:07.079 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.079 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.079 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.080 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2023-09-10T01:37:07.080 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2023-09-10T01:37:07.080 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.080 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.081 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.081 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "root", 2023-09-10T01:37:07.081 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.081 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.081 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.082 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm-package", 2023-09-10T01:37:07.082 INFO:teuthology.orchestra.run.smithi044.stdout: "root" 2023-09-10T01:37:07.082 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.082 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "mode for remote execution of cephadm", 2023-09-10T01:37:07.083 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.083 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.083 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.083 INFO:teuthology.orchestra.run.smithi044.stdout: "prometheus_alerts_path": { 2023-09-10T01:37:07.084 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus_alerts_path", 2023-09-10T01:37:07.084 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.084 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.084 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.084 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-10T01:37:07.085 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.085 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.085 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.085 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-10T01:37:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_insecure": { 2023-09-10T01:37:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_insecure", 2023-09-10T01:37:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-10T01:37:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_password": { 2023-09-10T01:37:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_password", 2023-09-10T01:37:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-10T01:37:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_url": { 2023-09-10T01:37:07.093 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_url", 2023-09-10T01:37:07.093 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.093 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.093 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.093 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-10T01:37:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_username": { 2023-09-10T01:37:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_username", 2023-09-10T01:37:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-10T01:37:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: "secure_monitoring_stack": { 2023-09-10T01:37:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "secure_monitoring_stack", 2023-09-10T01:37:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.100 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.100 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.100 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-10T01:37:07.100 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.100 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.101 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.101 INFO:teuthology.orchestra.run.smithi044.stdout: "service_discovery_port": { 2023-09-10T01:37:07.101 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "service_discovery_port", 2023-09-10T01:37:07.101 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.102 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.102 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.102 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8765", 2023-09-10T01:37:07.102 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.103 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.103 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.103 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "cephadm service discovery port", 2023-09-10T01:37:07.103 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.103 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.104 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.104 INFO:teuthology.orchestra.run.smithi044.stdout: "ssh_config_file": { 2023-09-10T01:37:07.104 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssh_config_file", 2023-09-10T01:37:07.104 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.105 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.105 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.105 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.105 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.105 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.106 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.106 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-10T01:37:07.106 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.106 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.107 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.107 INFO:teuthology.orchestra.run.smithi044.stdout: "use_agent": { 2023-09-10T01:37:07.107 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_agent", 2023-09-10T01:37:07.107 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.108 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.108 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.108 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.108 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.108 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.109 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.109 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-10T01:37:07.109 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.109 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.109 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.110 INFO:teuthology.orchestra.run.smithi044.stdout: "use_repo_digest": { 2023-09-10T01:37:07.110 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_repo_digest", 2023-09-10T01:37:07.110 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.111 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.111 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.111 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.111 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.111 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.112 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.112 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-10T01:37:07.112 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.112 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.112 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.113 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_failed_host_check": { 2023-09-10T01:37:07.113 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_failed_host_check", 2023-09-10T01:37:07.113 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.114 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.114 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.114 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.114 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.114 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.115 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.115 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if the host check fails", 2023-09-10T01:37:07.115 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.115 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.115 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.116 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_daemons": { 2023-09-10T01:37:07.116 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_daemons", 2023-09-10T01:37:07.116 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.116 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.117 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.117 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.117 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.117 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.117 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.118 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-10T01:37:07.118 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.118 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.118 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_hosts": { 2023-09-10T01:37:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_hosts", 2023-09-10T01:37:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-10T01:37:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crash", 2023-09-10T01:37:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: "retain_interval": { 2023-09-10T01:37:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retain_interval", 2023-09-10T01:37:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "31536000", 2023-09-10T01:37:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-10T01:37:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_recent_interval": { 2023-09-10T01:37:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_recent_interval", 2023-09-10T01:37:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1209600", 2023-09-10T01:37:07.142 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.142 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.142 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.142 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-10T01:37:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dashboard", 2023-09-10T01:37:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-10T01:37:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-10T01:37:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_HOST": { 2023-09-10T01:37:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-10T01:37:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-10T01:37:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-10T01:37:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_ENABLED": { 2023-09-10T01:37:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_ENABLED", 2023-09-10T01:37:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-10T01:37:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-10T01:37:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: "ENABLE_BROWSABLE_API": { 2023-09-10T01:37:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-10T01:37:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-10T01:37:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-10T01:37:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-10T01:37:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-10T01:37:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-10T01:37:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-10T01:37:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-10T01:37:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-10T01:37:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-10T01:37:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-10T01:37:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.177 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.177 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.177 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.177 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.177 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.178 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.178 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.178 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-10T01:37:07.178 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-10T01:37:07.179 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.179 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.179 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.179 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.179 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.180 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.180 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.180 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.180 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.180 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.181 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.181 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-10T01:37:07.181 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-10T01:37:07.181 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.182 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.182 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.182 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.182 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.182 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.183 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.183 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.183 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.183 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.183 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.184 INFO:teuthology.orchestra.run.smithi044.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-10T01:37:07.184 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-10T01:37:07.184 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.184 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.185 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.185 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.185 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.185 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.185 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.186 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.186 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.186 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.186 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.187 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_PASSWORD": { 2023-09-10T01:37:07.187 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-10T01:37:07.187 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.187 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.188 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.188 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2023-09-10T01:37:07.188 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.188 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.188 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.189 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.189 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.189 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.189 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.190 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-10T01:37:07.190 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-10T01:37:07.190 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.190 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.191 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.191 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.191 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.191 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.191 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.192 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.192 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.192 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.192 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.193 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_URL": { 2023-09-10T01:37:07.193 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_URL", 2023-09-10T01:37:07.193 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.193 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.194 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.194 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.194 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.194 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.194 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.196 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_USERNAME": { 2023-09-10T01:37:07.196 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-10T01:37:07.196 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.196 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.196 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.197 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2023-09-10T01:37:07.197 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.197 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.197 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.197 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.198 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.198 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.198 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.199 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-10T01:37:07.199 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-10T01:37:07.199 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.199 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.199 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.200 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.200 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.200 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.200 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.201 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.201 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.201 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.201 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.201 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-10T01:37:07.202 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-10T01:37:07.202 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.202 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.202 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.202 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.203 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.203 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.203 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.203 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.204 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.205 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.205 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.205 INFO:teuthology.orchestra.run.smithi044.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-10T01:37:07.205 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-10T01:37:07.206 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.206 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.206 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.206 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.206 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.207 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.207 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.207 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.207 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.207 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-10T01:37:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-10T01:37:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_HOST": { 2023-09-10T01:37:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-10T01:37:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-10T01:37:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-10T01:37:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-10T01:37:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-10T01:37:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.220 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-10T01:37:07.220 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-10T01:37:07.220 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.220 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.221 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.221 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.221 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.221 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.221 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.222 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.222 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.222 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.222 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.223 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-10T01:37:07.223 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-10T01:37:07.223 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.223 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.224 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.224 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.224 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.224 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.224 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.225 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.225 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.225 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.225 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.226 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-10T01:37:07.226 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-10T01:37:07.226 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.226 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.227 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.227 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.227 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.227 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.227 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.228 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.228 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.228 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.228 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.229 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-10T01:37:07.229 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-10T01:37:07.229 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.229 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.230 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.230 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.230 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.230 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.230 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.231 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.231 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.231 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.231 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.232 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-10T01:37:07.232 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-10T01:37:07.232 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.232 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.232 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.233 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.233 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.233 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.233 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.233 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.234 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.234 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.234 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.235 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-10T01:37:07.235 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-10T01:37:07.235 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.235 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.235 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.235 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.236 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.236 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.236 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.236 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.237 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.237 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.237 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.237 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_ENABLED": { 2023-09-10T01:37:07.238 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-10T01:37:07.238 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.238 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.238 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.239 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.239 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.239 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.239 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.239 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.240 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.240 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.240 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.240 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-10T01:37:07.241 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-10T01:37:07.241 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.241 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.241 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.241 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-10T01:37:07.242 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.242 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.242 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.242 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.243 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.243 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.243 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.243 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-10T01:37:07.244 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-10T01:37:07.244 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.244 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.244 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.244 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.245 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.246 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-10T01:37:07.247 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-10T01:37:07.247 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.247 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.247 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.247 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8", 2023-09-10T01:37:07.248 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.248 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.248 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.248 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.248 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.249 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.249 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.249 INFO:teuthology.orchestra.run.smithi044.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-10T01:37:07.249 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-10T01:37:07.250 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.250 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.250 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.250 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "45", 2023-09-10T01:37:07.250 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.251 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.251 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.251 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.251 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.252 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.252 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.252 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ACCESS_KEY": { 2023-09-10T01:37:07.252 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-10T01:37:07.253 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.253 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.253 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.253 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.253 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.254 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.254 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.254 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.254 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.255 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.255 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.255 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-10T01:37:07.255 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-10T01:37:07.255 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.256 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.256 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.256 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2023-09-10T01:37:07.256 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.256 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.257 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.257 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.257 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.257 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.257 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.258 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SECRET_KEY": { 2023-09-10T01:37:07.258 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-10T01:37:07.258 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.259 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.259 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.259 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.259 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.259 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.260 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.260 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.260 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.260 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.261 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.261 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SSL_VERIFY": { 2023-09-10T01:37:07.261 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-10T01:37:07.261 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.262 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.262 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.262 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.262 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.263 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.263 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.263 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.263 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.263 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.264 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.264 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-10T01:37:07.264 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-10T01:37:07.265 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.265 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.265 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.265 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2023-09-10T01:37:07.265 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.266 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.266 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.266 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.266 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.267 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.267 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.267 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-10T01:37:07.268 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-10T01:37:07.268 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.268 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.268 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.268 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:07.269 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.269 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.269 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.269 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.270 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.270 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.270 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.270 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-10T01:37:07.271 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-10T01:37:07.271 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.271 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.271 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.271 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2023-09-10T01:37:07.272 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.272 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.272 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.272 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.272 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.272 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.273 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.273 INFO:teuthology.orchestra.run.smithi044.stdout: "cross_origin_url": { 2023-09-10T01:37:07.273 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cross_origin_url", 2023-09-10T01:37:07.273 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.274 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.274 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.274 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.274 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.274 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.275 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.275 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.275 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.275 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.275 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.276 INFO:teuthology.orchestra.run.smithi044.stdout: "crt_file": { 2023-09-10T01:37:07.276 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crt_file", 2023-09-10T01:37:07.276 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.276 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.276 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.277 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.277 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.277 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.277 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.277 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.278 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.278 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.278 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.278 INFO:teuthology.orchestra.run.smithi044.stdout: "debug": { 2023-09-10T01:37:07.279 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "debug", 2023-09-10T01:37:07.279 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.279 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.279 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.279 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.280 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.280 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.280 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.280 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable/disable debug options", 2023-09-10T01:37:07.281 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.281 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.281 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.281 INFO:teuthology.orchestra.run.smithi044.stdout: "jwt_token_ttl": { 2023-09-10T01:37:07.282 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "jwt_token_ttl", 2023-09-10T01:37:07.282 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.282 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.282 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.283 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "28800", 2023-09-10T01:37:07.283 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.283 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.283 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.283 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.284 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.284 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.284 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.284 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2023-09-10T01:37:07.285 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2023-09-10T01:37:07.285 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.285 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.285 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.285 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.286 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.286 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.286 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.286 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.286 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.287 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.287 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.287 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.287 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.288 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.288 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.288 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.288 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.289 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.289 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.289 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.289 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.289 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.290 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.290 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.290 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.290 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.290 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.291 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.291 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.291 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.291 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.292 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.292 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.292 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.292 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.292 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.293 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.293 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.293 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.293 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.294 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.294 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.294 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.294 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.295 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.295 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.295 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.295 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.295 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.296 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.296 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.296 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.296 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.296 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.297 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.297 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.297 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.297 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.297 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.298 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.298 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.298 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.298 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.298 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.299 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.299 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.299 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.299 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.300 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.300 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.300 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.300 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.300 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.301 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.301 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.301 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.301 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.302 INFO:teuthology.orchestra.run.smithi044.stdout: "motd": { 2023-09-10T01:37:07.302 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "motd", 2023-09-10T01:37:07.302 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.302 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.303 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.303 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.303 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.303 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.303 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.304 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "The message of the day", 2023-09-10T01:37:07.304 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.304 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.304 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.305 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect_resolve_ip_addr": { 2023-09-10T01:37:07.305 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "redirect_resolve_ip_addr", 2023-09-10T01:37:07.305 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.305 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.305 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.306 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.306 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.306 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.306 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.306 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.307 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.307 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.307 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.307 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2023-09-10T01:37:07.308 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2023-09-10T01:37:07.308 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.308 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.308 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.308 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2023-09-10T01:37:07.309 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.309 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.309 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.309 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.309 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.310 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.310 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.310 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2023-09-10T01:37:07.311 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2023-09-10T01:37:07.311 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.311 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.311 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.311 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8080", 2023-09-10T01:37:07.312 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.312 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.312 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.312 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.312 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.313 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.313 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.313 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2023-09-10T01:37:07.313 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2023-09-10T01:37:07.313 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.314 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.314 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.314 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.314 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.315 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.315 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.315 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.315 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.315 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.316 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.316 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl_server_port": { 2023-09-10T01:37:07.316 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl_server_port", 2023-09-10T01:37:07.316 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.317 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.317 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.317 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8443", 2023-09-10T01:37:07.317 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.317 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.318 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.318 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.318 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.318 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.318 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.319 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2023-09-10T01:37:07.319 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2023-09-10T01:37:07.319 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.319 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.320 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.320 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "redirect", 2023-09-10T01:37:07.320 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.320 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.320 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.320 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.321 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect" 2023-09-10T01:37:07.321 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.321 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.321 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.321 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.322 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.322 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2023-09-10T01:37:07.322 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2023-09-10T01:37:07.323 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.323 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.323 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.323 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2023-09-10T01:37:07.323 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2023-09-10T01:37:07.324 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2023-09-10T01:37:07.324 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.324 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.324 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.324 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.324 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.325 INFO:teuthology.orchestra.run.smithi044.stdout: "url_prefix": { 2023-09-10T01:37:07.325 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url_prefix", 2023-09-10T01:37:07.325 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.325 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.326 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.326 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.326 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.326 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.326 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.327 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.327 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.327 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.327 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.327 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.327 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.328 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "devicehealth", 2023-09-10T01:37:07.328 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.328 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.329 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.329 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_monitoring": { 2023-09-10T01:37:07.329 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_monitoring", 2023-09-10T01:37:07.329 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.330 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.330 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.330 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.330 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.330 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.331 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.331 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "monitor device health metrics", 2023-09-10T01:37:07.331 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.331 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.331 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.332 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.332 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.332 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.332 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.333 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.333 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.333 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.333 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.333 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.334 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.334 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.334 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.334 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.335 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.335 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.335 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.335 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.335 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.336 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.336 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.336 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.336 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.337 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.337 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.337 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.337 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.337 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.338 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.338 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.338 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.338 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.338 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.339 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.339 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.339 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.339 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.340 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.340 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.340 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.340 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.340 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.341 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.341 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.341 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.341 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.342 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.342 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.342 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.342 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.342 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.343 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.343 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.343 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.343 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.344 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.344 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.344 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.344 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.344 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.344 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.345 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.345 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.345 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.345 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.345 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.346 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.346 INFO:teuthology.orchestra.run.smithi044.stdout: "mark_out_threshold": { 2023-09-10T01:37:07.346 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mark_out_threshold", 2023-09-10T01:37:07.346 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:07.347 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.347 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.347 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2419200", 2023-09-10T01:37:07.347 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.347 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.348 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.348 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-10T01:37:07.348 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.348 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.348 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.349 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": { 2023-09-10T01:37:07.349 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_name", 2023-09-10T01:37:07.349 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.349 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.350 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.350 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "device_health_metrics", 2023-09-10T01:37:07.350 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.350 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.350 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.351 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-10T01:37:07.351 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.351 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.351 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.352 INFO:teuthology.orchestra.run.smithi044.stdout: "retention_period": { 2023-09-10T01:37:07.352 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retention_period", 2023-09-10T01:37:07.352 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:07.352 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.353 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.353 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15552000", 2023-09-10T01:37:07.353 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.353 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.353 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.354 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain device health metrics", 2023-09-10T01:37:07.354 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.354 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.354 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.355 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_frequency": { 2023-09-10T01:37:07.355 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_frequency", 2023-09-10T01:37:07.355 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:07.355 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.356 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.356 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2023-09-10T01:37:07.356 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.356 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.356 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.357 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-10T01:37:07.357 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.357 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.357 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.358 INFO:teuthology.orchestra.run.smithi044.stdout: "self_heal": { 2023-09-10T01:37:07.358 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "self_heal", 2023-09-10T01:37:07.358 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.358 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.358 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.359 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.359 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.359 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.359 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.359 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-10T01:37:07.360 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.360 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.360 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.360 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2023-09-10T01:37:07.361 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2023-09-10T01:37:07.361 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:07.361 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.361 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.362 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2023-09-10T01:37:07.362 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.362 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.362 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.362 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and check device health", 2023-09-10T01:37:07.363 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.363 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.363 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.363 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_threshold": { 2023-09-10T01:37:07.364 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_threshold", 2023-09-10T01:37:07.364 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:07.364 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.364 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.364 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7257600", 2023-09-10T01:37:07.365 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.365 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.365 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.365 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-10T01:37:07.365 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.366 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.366 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.366 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.366 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.367 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "diskprediction_local", 2023-09-10T01:37:07.367 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.367 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.367 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.368 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.368 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.368 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.368 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.368 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.369 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.369 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.369 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.369 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.369 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.370 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.370 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.370 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.370 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.370 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.371 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.371 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.371 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.371 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.371 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.372 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.372 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.372 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.372 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.373 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.373 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.373 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.373 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.373 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.374 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.374 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.374 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.374 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.375 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.375 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.375 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.375 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.375 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.375 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.376 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.376 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.376 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.376 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.376 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.377 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.377 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.377 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.377 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.377 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.378 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.378 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.378 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.378 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.379 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.379 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.379 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.379 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.379 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.380 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.380 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.380 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.380 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.380 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.381 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.381 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.381 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.381 INFO:teuthology.orchestra.run.smithi044.stdout: "predict_interval": { 2023-09-10T01:37:07.382 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predict_interval", 2023-09-10T01:37:07.382 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.382 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.382 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.383 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2023-09-10T01:37:07.383 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.383 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.383 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.383 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.383 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.384 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.384 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.384 INFO:teuthology.orchestra.run.smithi044.stdout: "predictor_model": { 2023-09-10T01:37:07.384 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predictor_model", 2023-09-10T01:37:07.385 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.385 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.385 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.385 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "prophetstor", 2023-09-10T01:37:07.385 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.385 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.386 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.386 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.386 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.386 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.386 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.387 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2023-09-10T01:37:07.387 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2023-09-10T01:37:07.387 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.387 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.388 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.388 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2023-09-10T01:37:07.388 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.388 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.388 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.388 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.389 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.389 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.389 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.389 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.389 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.390 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "influx", 2023-09-10T01:37:07.390 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": false, 2023-09-10T01:37:07.390 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "influxdb python module not found", 2023-09-10T01:37:07.391 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.391 INFO:teuthology.orchestra.run.smithi044.stdout: "batch_size": { 2023-09-10T01:37:07.391 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "batch_size", 2023-09-10T01:37:07.391 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.391 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.392 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.392 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5000", 2023-09-10T01:37:07.392 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.392 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.392 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.392 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-10T01:37:07.393 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.393 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.393 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.393 INFO:teuthology.orchestra.run.smithi044.stdout: "database": { 2023-09-10T01:37:07.394 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "database", 2023-09-10T01:37:07.394 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.394 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.394 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.395 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "ceph", 2023-09-10T01:37:07.395 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.395 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.395 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.395 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-10T01:37:07.396 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.396 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.396 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.396 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": { 2023-09-10T01:37:07.396 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hostname", 2023-09-10T01:37:07.397 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.397 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.397 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.397 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.397 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.398 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.398 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.398 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server hostname", 2023-09-10T01:37:07.398 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.398 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.399 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.399 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2023-09-10T01:37:07.399 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2023-09-10T01:37:07.399 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:07.400 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.400 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.400 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "30", 2023-09-10T01:37:07.400 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "5", 2023-09-10T01:37:07.400 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.400 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.401 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-10T01:37:07.401 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.401 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.401 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.402 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.402 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.402 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.402 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.403 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.403 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.403 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.403 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.403 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.404 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.404 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.404 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.404 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.404 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.405 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.405 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.405 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.405 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.405 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.406 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.406 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.406 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.406 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.409 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.409 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.409 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.409 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.410 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.410 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.410 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.410 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.411 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.411 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.411 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.411 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.412 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.412 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.412 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.412 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.413 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.413 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.413 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.413 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.413 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.414 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.414 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.414 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.414 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.414 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.415 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.415 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.415 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.415 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.416 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.416 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.416 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.416 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.416 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.417 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.417 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.417 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.417 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.418 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.418 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.418 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.418 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.419 INFO:teuthology.orchestra.run.smithi044.stdout: "password": { 2023-09-10T01:37:07.419 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "password", 2023-09-10T01:37:07.419 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.419 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.419 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.420 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.420 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.420 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.420 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.420 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "password of InfluxDB server user", 2023-09-10T01:37:07.421 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.421 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.421 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.421 INFO:teuthology.orchestra.run.smithi044.stdout: "port": { 2023-09-10T01:37:07.422 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "port", 2023-09-10T01:37:07.422 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.422 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.422 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.423 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8086", 2023-09-10T01:37:07.423 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.423 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.423 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.423 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server port", 2023-09-10T01:37:07.424 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.424 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.424 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.424 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2023-09-10T01:37:07.425 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2023-09-10T01:37:07.425 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.425 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.425 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.425 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "false", 2023-09-10T01:37:07.426 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.426 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.426 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.426 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-10T01:37:07.427 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.427 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.427 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.427 INFO:teuthology.orchestra.run.smithi044.stdout: "threads": { 2023-09-10T01:37:07.427 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threads", 2023-09-10T01:37:07.428 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.428 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.428 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.428 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2023-09-10T01:37:07.429 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2023-09-10T01:37:07.429 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "32", 2023-09-10T01:37:07.429 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.429 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-10T01:37:07.429 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.429 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.430 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.430 INFO:teuthology.orchestra.run.smithi044.stdout: "username": { 2023-09-10T01:37:07.430 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "username", 2023-09-10T01:37:07.430 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.431 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.431 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.431 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.431 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.431 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.432 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.432 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "username of InfluxDB server user", 2023-09-10T01:37:07.432 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.432 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.432 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.433 INFO:teuthology.orchestra.run.smithi044.stdout: "verify_ssl": { 2023-09-10T01:37:07.433 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "verify_ssl", 2023-09-10T01:37:07.433 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.433 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.433 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.434 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "true", 2023-09-10T01:37:07.434 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.434 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.434 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.435 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-10T01:37:07.435 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.435 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.435 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.435 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.436 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.436 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "insights", 2023-09-10T01:37:07.436 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.437 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.437 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.437 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.437 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.437 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.438 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.438 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.438 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.438 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.438 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.439 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.439 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.439 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.439 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.439 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.440 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.440 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.440 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.440 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.441 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.441 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.441 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.441 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.442 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.442 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.442 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.442 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.442 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.443 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.443 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.443 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.443 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.443 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.443 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.444 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.444 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.444 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.444 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.445 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.445 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.445 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.445 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.446 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.446 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.446 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.446 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.446 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.447 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.447 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.447 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.447 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.447 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.448 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.448 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.448 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.448 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.449 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.449 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.449 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.449 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.449 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.450 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.450 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.450 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.450 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.450 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.451 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.451 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.451 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.451 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.452 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "iostat", 2023-09-10T01:37:07.452 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.452 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.453 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.453 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.453 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.453 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.453 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.454 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.454 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.454 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.454 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.454 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.455 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.455 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.455 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.455 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.456 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.456 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.456 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.456 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.456 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.457 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.457 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.457 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.457 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.458 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.458 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.458 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.458 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.458 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.459 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.459 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.459 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.459 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.459 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.460 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.460 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.460 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.460 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.461 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.461 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.461 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.461 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.461 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.462 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.462 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.462 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.462 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.463 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.463 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.463 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.463 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.463 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.464 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.464 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.464 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.464 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.465 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.465 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.465 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.465 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.465 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.466 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.466 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.466 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.466 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.466 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.467 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.467 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.467 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.467 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.468 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "k8sevents", 2023-09-10T01:37:07.468 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.468 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.468 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.469 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_event_retention_days": { 2023-09-10T01:37:07.469 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ceph_event_retention_days", 2023-09-10T01:37:07.469 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.469 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.469 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.470 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7", 2023-09-10T01:37:07.470 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.470 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.470 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.470 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-10T01:37:07.471 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.471 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.471 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.471 INFO:teuthology.orchestra.run.smithi044.stdout: "config_check_secs": { 2023-09-10T01:37:07.472 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_check_secs", 2023-09-10T01:37:07.472 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.472 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.472 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.472 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:07.473 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "10", 2023-09-10T01:37:07.473 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.473 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.473 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-10T01:37:07.473 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.474 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.474 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.474 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.474 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.475 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.475 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.475 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.475 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.476 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.476 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.476 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.476 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.476 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.477 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.477 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.477 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.477 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.477 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.478 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.478 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.478 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.478 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.479 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.479 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.479 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.479 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.479 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.480 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.480 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.480 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.480 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.480 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.481 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.481 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.481 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.481 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.482 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.482 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.482 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.482 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.483 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.483 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.483 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.483 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.483 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.484 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.484 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.484 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.484 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.484 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.485 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.485 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.485 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.485 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.485 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.486 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.486 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.486 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.486 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.487 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.487 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.487 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.487 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.487 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.488 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.488 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.488 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.488 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.488 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.489 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.489 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "localpool", 2023-09-10T01:37:07.489 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.490 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.490 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.490 INFO:teuthology.orchestra.run.smithi044.stdout: "failure_domain": { 2023-09-10T01:37:07.490 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "failure_domain", 2023-09-10T01:37:07.491 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.491 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.491 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.491 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "host", 2023-09-10T01:37:07.491 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.492 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.492 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.492 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "failure domain for any created local pool", 2023-09-10T01:37:07.492 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-10T01:37:07.493 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.493 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.493 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.493 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.494 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.494 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.494 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.494 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.494 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.495 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.495 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.495 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.495 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.495 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.496 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.496 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.496 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.496 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.496 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.497 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.497 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.497 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.497 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.498 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.498 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.498 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.498 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.498 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.499 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.499 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.499 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.499 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.499 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.500 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.500 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.500 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.500 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.501 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.501 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.501 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.501 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.501 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.502 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.502 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.502 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.502 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.503 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.503 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.503 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.503 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.503 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.504 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.504 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.504 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.504 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.505 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.505 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.505 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.505 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.505 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.505 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.506 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.506 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.506 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.506 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.507 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.507 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.507 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.507 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": { 2023-09-10T01:37:07.507 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_size", 2023-09-10T01:37:07.508 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.508 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.508 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.508 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.509 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.509 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.509 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.509 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default min_size for any created local pool", 2023-09-10T01:37:07.509 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-10T01:37:07.510 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.510 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.510 INFO:teuthology.orchestra.run.smithi044.stdout: "num_rep": { 2023-09-10T01:37:07.510 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "num_rep", 2023-09-10T01:37:07.511 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.511 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.511 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.511 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3", 2023-09-10T01:37:07.511 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.512 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.512 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.512 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default replica count for any created local pool", 2023-09-10T01:37:07.512 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.512 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.513 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.513 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": { 2023-09-10T01:37:07.513 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_num", 2023-09-10T01:37:07.513 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.514 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.514 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.514 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "128", 2023-09-10T01:37:07.514 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.514 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.515 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.515 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default pg_num for any created local pool", 2023-09-10T01:37:07.515 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.515 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.515 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.516 INFO:teuthology.orchestra.run.smithi044.stdout: "prefix": { 2023-09-10T01:37:07.516 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prefix", 2023-09-10T01:37:07.516 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.517 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.517 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.517 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.517 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.517 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.518 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.518 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name prefix for any created local pool", 2023-09-10T01:37:07.518 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.518 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.519 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.519 INFO:teuthology.orchestra.run.smithi044.stdout: "subtree": { 2023-09-10T01:37:07.519 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subtree", 2023-09-10T01:37:07.519 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.520 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.520 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.520 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "rack", 2023-09-10T01:37:07.520 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.520 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.521 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.521 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-10T01:37:07.521 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-10T01:37:07.521 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.521 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.522 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.522 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.523 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mds_autoscaler", 2023-09-10T01:37:07.523 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.523 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.523 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.523 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.524 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.524 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.524 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.524 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.524 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.525 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.525 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.525 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.525 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.525 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.526 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.526 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.526 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.526 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.526 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.527 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.527 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.527 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.527 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.528 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.528 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.528 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.528 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.528 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.529 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.529 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.529 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.529 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.530 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.530 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.530 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.530 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.531 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.531 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.531 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.531 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.531 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.532 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.532 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.532 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.532 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.533 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.533 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.533 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.533 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.533 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.534 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.534 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.534 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.534 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.534 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.535 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.535 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.535 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.535 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.536 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.536 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.536 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.536 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.536 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.537 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.537 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.537 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.537 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.537 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.538 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.538 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.538 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirroring", 2023-09-10T01:37:07.539 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.539 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.539 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.539 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.539 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.540 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.540 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.540 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.540 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.540 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.541 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.541 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.541 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.541 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.542 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.542 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.542 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.542 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.542 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.543 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.543 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.543 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.543 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.544 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.544 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.544 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.544 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.544 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.545 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.545 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.545 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.545 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.545 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.546 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.546 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.546 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.546 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.547 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.547 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.547 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.547 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.547 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.548 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.548 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.548 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.548 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.548 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.549 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.549 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.549 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.549 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.550 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.550 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.550 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.550 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.551 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.551 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.551 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.551 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.552 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.552 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.552 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.552 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.553 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.553 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.553 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.553 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.553 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.554 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.554 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.554 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.555 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "nfs", 2023-09-10T01:37:07.555 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.555 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.555 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.556 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.556 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.556 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.556 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.556 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.557 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.557 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.557 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.557 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.557 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.558 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.558 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.558 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.558 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.558 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.559 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.559 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.559 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.559 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.559 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.560 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.560 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.560 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.561 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.561 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.561 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.561 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.561 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.562 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.562 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.562 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.562 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.562 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.563 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.563 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.563 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.564 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.564 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.564 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.564 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.564 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.565 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.565 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.565 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.565 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.565 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.566 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.566 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.566 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.566 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.567 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.567 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.567 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.567 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.568 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.568 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.568 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.568 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.568 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.569 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.569 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.569 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.569 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.569 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.570 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.570 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.570 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.570 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.571 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2023-09-10T01:37:07.571 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.571 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.572 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.572 INFO:teuthology.orchestra.run.smithi044.stdout: "fail_fs": { 2023-09-10T01:37:07.572 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "fail_fs", 2023-09-10T01:37:07.572 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.572 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.573 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.573 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.573 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.573 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.574 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.574 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-10T01:37:07.574 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.574 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.574 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.575 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.575 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.575 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.575 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.576 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.576 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.576 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.576 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.576 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.576 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.577 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.577 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.577 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.577 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.577 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.577 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.578 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.578 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.578 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.578 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.579 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.579 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.579 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.579 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.580 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.580 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.580 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.580 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.580 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.581 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.581 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.581 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.581 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.582 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.582 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.582 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.582 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.583 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.583 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.583 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.583 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.584 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.584 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.584 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.584 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.584 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.585 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.585 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.585 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.585 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.585 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.586 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.586 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.586 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.586 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.587 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.587 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.587 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.587 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.587 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.588 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.588 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.588 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.588 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.588 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.589 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.589 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator": { 2023-09-10T01:37:07.589 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2023-09-10T01:37:07.589 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.590 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.590 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.590 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.590 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.591 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.591 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.591 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm", 2023-09-10T01:37:07.591 INFO:teuthology.orchestra.run.smithi044.stdout: "rook", 2023-09-10T01:37:07.592 INFO:teuthology.orchestra.run.smithi044.stdout: "test_orchestrator" 2023-09-10T01:37:07.592 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.592 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Orchestrator backend", 2023-09-10T01:37:07.592 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.592 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.593 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.593 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.593 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.594 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_perf_query", 2023-09-10T01:37:07.594 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.594 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.594 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.595 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.595 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.595 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.595 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.595 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.595 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.596 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.596 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.596 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.596 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.597 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.597 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.597 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.597 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.597 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.598 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.598 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.598 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.598 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.598 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.599 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.599 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.599 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.599 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.600 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.600 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.600 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.600 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.600 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.601 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.601 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.601 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.601 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.602 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.602 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.602 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.602 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.603 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.603 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.603 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.603 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.603 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.604 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.604 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.604 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.604 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.604 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.605 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.605 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.605 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.605 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.605 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.606 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.606 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.606 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.607 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.607 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.607 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.607 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.607 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.608 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.608 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.608 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.608 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.608 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.609 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.609 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.609 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.610 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_support", 2023-09-10T01:37:07.610 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.610 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.610 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.611 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.611 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.611 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.611 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.611 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.612 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.612 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.612 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.612 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.612 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.613 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.613 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.613 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.613 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.613 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.614 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.614 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.614 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.614 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.615 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.615 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.616 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.617 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.617 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.617 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.617 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.618 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.618 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.618 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.618 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.618 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.619 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.619 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.619 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.619 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.620 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.620 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.620 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.620 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.620 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.621 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.621 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.621 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.621 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.622 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.622 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.622 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.622 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.622 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.623 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.623 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.623 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.623 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.624 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.624 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.624 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.624 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.624 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.625 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.625 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.625 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.625 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.625 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.626 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.626 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.626 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.626 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.626 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.627 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_autoscaler", 2023-09-10T01:37:07.627 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.627 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.628 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.628 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.628 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.628 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.628 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.629 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.629 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.629 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.629 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.629 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.630 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.630 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.630 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.630 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.631 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.631 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.631 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.631 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.631 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.632 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.632 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.632 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.632 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.633 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.633 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.633 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.633 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.633 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.635 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:05 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:07.635 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:05 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/303873822' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-10T01:37:07.636 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:06 smithi099 ceph-mon[102692]: pgmap v1840: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 258 KiB/s rd, 270 KiB/s wr, 44 op/s 2023-09-10T01:37:07.636 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:05 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:07.636 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:05 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/303873822' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-10T01:37:07.636 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:06 smithi131 ceph-mon[103713]: pgmap v1840: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 258 KiB/s rd, 270 KiB/s wr, 44 op/s 2023-09-10T01:37:07.639 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.640 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.640 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.640 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.640 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.641 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.641 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.641 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.641 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.642 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.642 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.642 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.642 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.643 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.643 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.643 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.643 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.643 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.644 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.644 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.644 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.644 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.644 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.645 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.645 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.645 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.645 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.646 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.646 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.646 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.646 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.646 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.647 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.647 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.647 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.647 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.647 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.648 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.648 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.648 INFO:teuthology.orchestra.run.smithi044.stdout: "noautoscale": { 2023-09-10T01:37:07.648 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "noautoscale", 2023-09-10T01:37:07.649 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.649 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.649 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.649 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.649 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.650 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.650 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.650 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "global autoscale flag", 2023-09-10T01:37:07.650 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-10T01:37:07.650 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.651 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.651 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2023-09-10T01:37:07.651 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2023-09-10T01:37:07.652 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:07.652 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.652 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.652 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2023-09-10T01:37:07.652 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.653 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.653 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.653 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.653 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.653 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.654 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.654 INFO:teuthology.orchestra.run.smithi044.stdout: "threshold": { 2023-09-10T01:37:07.654 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threshold", 2023-09-10T01:37:07.654 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:07.655 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.655 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.655 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2023-09-10T01:37:07.655 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1.0", 2023-09-10T01:37:07.656 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.656 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.656 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "scaling threshold", 2023-09-10T01:37:07.656 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-10T01:37:07.656 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.657 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.657 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.657 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.658 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "progress", 2023-09-10T01:37:07.658 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.658 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.658 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.658 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_pg_recovery_event": { 2023-09-10T01:37:07.659 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_pg_recovery_event", 2023-09-10T01:37:07.659 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.659 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.659 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.660 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.660 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.660 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.660 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.660 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-10T01:37:07.660 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.661 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.661 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.661 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2023-09-10T01:37:07.662 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2023-09-10T01:37:07.662 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.662 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.662 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.662 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.663 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.663 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.663 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.663 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.663 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.664 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.664 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.664 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.664 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.665 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.665 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.665 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.665 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.666 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.666 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.666 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.666 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.667 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.667 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.667 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.667 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.667 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.668 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.668 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.668 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.668 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.668 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.669 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.669 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.669 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.669 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.669 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.670 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.670 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.670 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.670 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.671 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.671 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.671 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.671 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.672 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.672 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.672 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.672 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.673 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.673 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.673 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.673 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.673 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.674 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.674 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.674 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.674 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.674 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.675 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.675 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.675 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.675 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.675 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.675 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.676 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.676 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.676 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.676 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.677 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.677 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.677 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.677 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.677 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.678 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.678 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.678 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.678 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.679 INFO:teuthology.orchestra.run.smithi044.stdout: "max_completed_events": { 2023-09-10T01:37:07.679 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_completed_events", 2023-09-10T01:37:07.679 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.679 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.680 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.680 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "50", 2023-09-10T01:37:07.680 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.680 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.680 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.681 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "number of past completed events to remember", 2023-09-10T01:37:07.681 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.681 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.681 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.682 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2023-09-10T01:37:07.682 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2023-09-10T01:37:07.682 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:07.682 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.682 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.683 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2023-09-10T01:37:07.683 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.683 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.683 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.684 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long the module is going to sleep", 2023-09-10T01:37:07.684 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.684 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.684 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.684 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.685 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.685 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus", 2023-09-10T01:37:07.685 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.686 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.686 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.686 INFO:teuthology.orchestra.run.smithi044.stdout: "cache": { 2023-09-10T01:37:07.686 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cache", 2023-09-10T01:37:07.686 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.687 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.687 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.687 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.687 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.687 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.688 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.688 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.688 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.688 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.689 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.689 INFO:teuthology.orchestra.run.smithi044.stdout: "exclude_perf_counters": { 2023-09-10T01:37:07.689 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "exclude_perf_counters", 2023-09-10T01:37:07.689 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.689 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.690 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.690 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.690 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.690 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.691 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.691 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-10T01:37:07.691 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-10T01:37:07.691 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.692 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.692 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.692 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.692 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.692 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.693 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.693 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.693 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.693 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.693 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.695 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.695 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.695 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.695 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.695 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.696 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.696 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.696 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.696 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.697 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.697 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.697 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.697 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.697 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.698 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.698 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.698 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.698 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.698 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.699 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.699 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.699 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.699 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.700 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.700 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.700 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.700 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.701 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.701 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.701 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.701 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.701 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.702 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.702 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.702 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.702 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.702 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.703 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.703 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.703 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.703 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.704 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.704 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.704 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.704 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.705 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.705 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.705 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.705 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.705 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.706 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.706 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.706 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools": { 2023-09-10T01:37:07.706 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools", 2023-09-10T01:37:07.707 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.707 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.707 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.707 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.707 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.708 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.708 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.708 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.709 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.709 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.709 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.710 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-10T01:37:07.710 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-10T01:37:07.710 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.710 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.711 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.711 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "300", 2023-09-10T01:37:07.711 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.711 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.711 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.712 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.712 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.712 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.712 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.713 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_interval": { 2023-09-10T01:37:07.713 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_interval", 2023-09-10T01:37:07.713 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:07.714 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.714 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.714 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15.0", 2023-09-10T01:37:07.714 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.715 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.715 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.715 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.715 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.716 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.716 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.716 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2023-09-10T01:37:07.716 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2023-09-10T01:37:07.717 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.717 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.717 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.717 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2023-09-10T01:37:07.718 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.718 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.718 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.719 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-10T01:37:07.719 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.719 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.719 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.720 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2023-09-10T01:37:07.720 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2023-09-10T01:37:07.720 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.720 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.721 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.721 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "9283", 2023-09-10T01:37:07.721 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.721 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.722 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.722 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-10T01:37:07.722 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.722 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.723 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.723 INFO:teuthology.orchestra.run.smithi044.stdout: "stale_cache_strategy": { 2023-09-10T01:37:07.723 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stale_cache_strategy", 2023-09-10T01:37:07.724 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.724 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.724 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.724 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "log", 2023-09-10T01:37:07.725 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.725 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.725 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.725 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.726 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.726 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.726 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.727 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2023-09-10T01:37:07.727 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2023-09-10T01:37:07.727 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.727 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.727 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.728 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "default", 2023-09-10T01:37:07.728 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.728 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.728 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.728 INFO:teuthology.orchestra.run.smithi044.stdout: "default", 2023-09-10T01:37:07.729 INFO:teuthology.orchestra.run.smithi044.stdout: "error" 2023-09-10T01:37:07.729 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.729 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.729 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.729 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.730 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.730 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2023-09-10T01:37:07.730 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2023-09-10T01:37:07.731 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.731 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.731 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.731 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2023-09-10T01:37:07.731 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2023-09-10T01:37:07.732 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2023-09-10T01:37:07.732 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.732 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.732 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.732 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.733 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.733 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.733 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.734 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_support", 2023-09-10T01:37:07.734 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.734 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.734 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.734 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.735 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.735 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.735 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.735 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.735 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.736 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.736 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.736 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.736 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.736 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.737 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.737 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.737 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.737 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.737 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.738 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.738 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.738 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.738 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.739 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.739 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.739 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.739 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.740 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.740 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.740 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.740 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.740 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.740 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.741 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.741 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.741 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.742 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.742 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.742 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.742 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.742 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.743 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.743 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.743 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.743 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.745 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.745 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.745 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.745 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.746 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.746 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.746 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.746 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.749 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_snap_create": { 2023-09-10T01:37:07.749 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_snap_create", 2023-09-10T01:37:07.749 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.749 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.751 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.751 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.751 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.751 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.752 INFO:teuthology.orchestra.run.smithi044.stdout: "mirror_snapshot_schedule": { 2023-09-10T01:37:07.752 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirror_snapshot_schedule", 2023-09-10T01:37:07.752 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.752 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.752 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.754 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.754 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.754 INFO:teuthology.orchestra.run.smithi044.stdout: "trash_purge_schedule": { 2023-09-10T01:37:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "trash_purge_schedule", 2023-09-10T01:37:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "restful", 2023-09-10T01:37:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_auth": { 2023-09-10T01:37:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_auth", 2023-09-10T01:37:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.761 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.761 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.761 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.762 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2023-09-10T01:37:07.762 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2023-09-10T01:37:07.762 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.762 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.762 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.763 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.763 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.763 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.763 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.769 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.769 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.769 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.769 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.773 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.773 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.773 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.773 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.774 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.774 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.774 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.774 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.774 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.775 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.775 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.775 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.775 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.775 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.777 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.777 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.777 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.777 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.777 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.778 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.778 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.778 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.778 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.778 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.779 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.779 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.779 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.779 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2023-09-10T01:37:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2023-09-10T01:37:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.781 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.781 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.781 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.781 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.781 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.782 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.782 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.782 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.782 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2023-09-10T01:37:07.783 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2023-09-10T01:37:07.783 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.783 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.783 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.783 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.784 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.784 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.784 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.784 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rgw", 2023-09-10T01:37:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.793 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.793 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.793 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.793 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.793 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.794 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.794 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.794 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.794 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.800 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.800 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.800 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.800 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.800 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rook", 2023-09-10T01:37:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: "drive_group_interval": { 2023-09-10T01:37:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "drive_group_interval", 2023-09-10T01:37:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.804 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "300.0", 2023-09-10T01:37:07.804 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.804 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.804 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.804 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-10T01:37:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.813 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.813 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.813 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.813 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.813 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.814 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.814 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.814 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.814 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.815 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.815 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.815 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.815 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.815 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.815 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.816 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.816 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.816 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.816 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.816 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.817 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.817 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.817 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.817 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.818 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.818 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.818 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.818 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.819 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.819 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.819 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.819 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.819 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.820 INFO:teuthology.orchestra.run.smithi044.stdout: "storage_class": { 2023-09-10T01:37:07.820 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "storage_class", 2023-09-10T01:37:07.820 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.820 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.821 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.821 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "local", 2023-09-10T01:37:07.821 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.821 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.821 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.822 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-10T01:37:07.822 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.822 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.822 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.823 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.823 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.823 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.823 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "selftest", 2023-09-10T01:37:07.824 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.824 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.824 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.824 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.824 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.825 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.825 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.825 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.825 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.826 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.826 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.826 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.826 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.826 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.827 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.827 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.827 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.827 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.828 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.828 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.828 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.828 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.828 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.829 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.829 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.829 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.829 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.830 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.830 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.830 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.830 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.831 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.831 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.831 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.831 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.831 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.832 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.832 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.832 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.832 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.832 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.833 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.833 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.833 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.833 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.833 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.834 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.834 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.834 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.834 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.834 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.835 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.835 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.835 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.835 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.835 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.836 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.836 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.836 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.836 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.837 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.837 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.837 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.837 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.837 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.838 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.838 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.838 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.838 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.839 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.839 INFO:teuthology.orchestra.run.smithi044.stdout: "roption1": { 2023-09-10T01:37:07.839 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption1", 2023-09-10T01:37:07.839 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.840 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.840 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.840 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.840 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.841 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.841 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.841 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.841 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.841 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.842 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.842 INFO:teuthology.orchestra.run.smithi044.stdout: "roption2": { 2023-09-10T01:37:07.842 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption2", 2023-09-10T01:37:07.842 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.843 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.843 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.843 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "xyz", 2023-09-10T01:37:07.843 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.844 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.844 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.844 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.844 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.844 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.845 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.845 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption1": { 2023-09-10T01:37:07.845 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption1", 2023-09-10T01:37:07.845 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.846 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.846 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.846 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.846 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.846 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.847 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.847 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.847 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.847 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.847 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.848 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption2": { 2023-09-10T01:37:07.848 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption2", 2023-09-10T01:37:07.848 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.848 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.849 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.849 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.849 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.849 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.849 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.850 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.850 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.850 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.850 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.851 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption3": { 2023-09-10T01:37:07.851 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption3", 2023-09-10T01:37:07.851 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:07.851 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.851 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.852 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.852 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.852 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.852 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.852 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.853 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.853 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.853 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.853 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption4": { 2023-09-10T01:37:07.854 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption4", 2023-09-10T01:37:07.854 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.854 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.854 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.854 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.855 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.855 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.855 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.855 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.855 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.856 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.856 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.856 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption5": { 2023-09-10T01:37:07.856 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption5", 2023-09-10T01:37:07.857 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.857 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.857 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.857 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.857 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.858 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.858 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.858 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.858 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.859 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.859 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.859 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption6": { 2023-09-10T01:37:07.859 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption6", 2023-09-10T01:37:07.860 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.860 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.860 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.860 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.860 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.860 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.861 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.861 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.861 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.861 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.861 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.862 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption7": { 2023-09-10T01:37:07.862 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption7", 2023-09-10T01:37:07.862 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.863 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.863 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.863 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.863 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2023-09-10T01:37:07.864 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "42", 2023-09-10T01:37:07.864 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.864 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.864 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.864 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.865 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.865 INFO:teuthology.orchestra.run.smithi044.stdout: "testkey": { 2023-09-10T01:37:07.865 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testkey", 2023-09-10T01:37:07.865 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.866 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.866 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.866 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.866 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.867 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.867 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.867 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.867 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.867 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.868 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.868 INFO:teuthology.orchestra.run.smithi044.stdout: "testlkey": { 2023-09-10T01:37:07.868 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testlkey", 2023-09-10T01:37:07.868 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.869 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.869 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.869 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.869 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.869 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.870 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.870 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.870 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.870 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.870 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.871 INFO:teuthology.orchestra.run.smithi044.stdout: "testnewline": { 2023-09-10T01:37:07.871 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testnewline", 2023-09-10T01:37:07.871 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.872 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.872 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.872 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.872 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.872 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.873 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.873 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.873 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.873 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.873 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.874 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.874 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.874 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2023-09-10T01:37:07.875 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.875 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.875 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.875 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_m_granularity": { 2023-09-10T01:37:07.875 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_m_granularity", 2023-09-10T01:37:07.876 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.876 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.876 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.876 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.877 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.877 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.877 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.877 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow minute scheduled snapshots", 2023-09-10T01:37:07.877 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.878 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.878 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.878 INFO:teuthology.orchestra.run.smithi044.stdout: "dump_on_update": { 2023-09-10T01:37:07.878 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dump_on_update", 2023-09-10T01:37:07.879 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.879 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.879 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.879 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.879 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.880 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.880 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.880 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "dump database to debug log on update", 2023-09-10T01:37:07.880 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.881 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.881 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.881 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.881 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.882 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.882 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.882 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.882 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.882 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.883 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.883 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.883 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.883 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.883 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.883 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.884 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.884 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.884 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.884 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.884 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.885 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.885 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.885 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.885 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.886 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.886 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.886 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.886 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.886 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.887 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.887 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.887 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.887 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.887 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.888 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.888 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.888 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.888 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.889 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.889 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.889 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.889 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.889 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.890 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.890 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.890 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.890 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.890 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.891 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.891 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.891 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.891 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.891 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.892 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.892 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.892 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.892 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.893 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.893 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.893 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.893 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.893 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.894 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.894 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.894 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.894 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.894 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.895 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.895 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.895 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.896 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stats", 2023-09-10T01:37:07.896 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.896 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.896 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.896 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.897 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.897 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.897 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.897 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.897 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.898 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.898 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.898 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.898 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.898 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.899 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.899 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.899 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.899 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.900 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.900 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.900 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.900 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.900 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.901 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.901 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.901 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.901 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.902 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.902 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.902 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.902 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.902 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.903 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.903 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.903 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.903 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.904 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.904 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.904 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.904 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.904 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.905 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.905 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.905 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.905 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.905 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.906 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.906 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.906 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.906 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.907 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.907 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.907 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.907 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.907 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.908 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.908 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.908 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.908 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.909 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.909 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.909 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.909 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.909 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.910 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.910 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.910 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.910 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.910 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.911 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.911 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.911 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "status", 2023-09-10T01:37:07.912 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.912 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.912 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.912 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.912 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.913 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.913 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.913 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.913 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.913 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.914 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.914 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.914 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.914 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.915 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.915 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.915 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.915 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.915 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.916 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.916 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.916 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.916 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.917 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.917 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.917 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.917 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.917 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.918 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.918 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.918 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.918 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.918 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.919 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.919 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.919 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.919 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.920 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.920 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.920 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.920 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.921 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.921 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.921 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.921 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.921 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.922 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.922 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.922 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.922 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.923 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.923 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.923 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.923 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.923 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.924 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.924 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.924 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.924 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.925 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.925 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.925 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.925 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.925 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.926 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.926 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.926 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.926 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.926 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.927 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.927 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.927 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telegraf", 2023-09-10T01:37:07.927 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.928 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.928 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.928 INFO:teuthology.orchestra.run.smithi044.stdout: "address": { 2023-09-10T01:37:07.928 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "address", 2023-09-10T01:37:07.928 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.929 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.929 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.929 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-10T01:37:07.929 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.929 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.929 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.930 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.930 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.930 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.930 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.931 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2023-09-10T01:37:07.931 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2023-09-10T01:37:07.931 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:07.931 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.932 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.932 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15", 2023-09-10T01:37:07.932 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.932 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.932 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.932 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.933 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.933 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.933 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.934 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.934 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.934 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.934 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.934 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.935 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.935 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.935 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.935 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.935 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.936 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.936 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.936 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.936 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.936 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.937 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.937 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.937 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.937 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.937 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.938 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.938 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.938 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.938 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.939 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.939 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.939 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.939 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.939 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.939 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.940 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.940 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.940 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.940 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.941 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.941 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.941 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.941 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.941 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.942 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.942 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.942 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.942 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.943 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.943 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.943 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.943 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.943 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.943 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.944 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.944 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.944 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.944 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.945 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.945 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.945 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.945 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.946 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.946 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.946 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.946 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.946 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.947 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.947 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.947 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.947 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.947 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.948 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:07.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:07.948 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telemetry", 2023-09-10T01:37:07.948 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:07.949 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:07.949 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:07.949 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_basic": { 2023-09-10T01:37:07.949 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_basic", 2023-09-10T01:37:07.949 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.950 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.950 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.950 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.950 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.951 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.951 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.951 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-10T01:37:07.951 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.951 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.952 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.952 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_crash": { 2023-09-10T01:37:07.952 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_crash", 2023-09-10T01:37:07.952 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.953 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.953 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.953 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.953 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.953 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.954 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.954 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-10T01:37:07.954 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.954 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.955 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.955 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_device": { 2023-09-10T01:37:07.955 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_device", 2023-09-10T01:37:07.955 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.956 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.956 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.956 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:07.956 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.956 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.956 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.957 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-10T01:37:07.957 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.957 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.957 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.958 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_ident": { 2023-09-10T01:37:07.958 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_ident", 2023-09-10T01:37:07.958 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.958 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.959 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.959 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.959 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.959 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.959 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.960 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-10T01:37:07.960 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.960 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.960 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.960 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_perf": { 2023-09-10T01:37:07.961 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_perf", 2023-09-10T01:37:07.961 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.961 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.961 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.961 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.962 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.962 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.962 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.962 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-10T01:37:07.962 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.963 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.963 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.963 INFO:teuthology.orchestra.run.smithi044.stdout: "contact": { 2023-09-10T01:37:07.964 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "contact", 2023-09-10T01:37:07.964 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.964 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.964 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.964 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.965 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.965 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.965 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.965 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.966 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.966 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.966 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.967 INFO:teuthology.orchestra.run.smithi044.stdout: "description": { 2023-09-10T01:37:07.967 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "description", 2023-09-10T01:37:07.967 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.967 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.967 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.967 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.968 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.968 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.968 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.968 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.969 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.969 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.969 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.969 INFO:teuthology.orchestra.run.smithi044.stdout: "device_url": { 2023-09-10T01:37:07.970 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_url", 2023-09-10T01:37:07.970 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.970 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.970 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.970 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-10T01:37:07.971 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.971 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.971 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.971 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.971 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.972 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.972 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.972 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2023-09-10T01:37:07.972 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2023-09-10T01:37:07.973 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.973 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.973 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.973 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.974 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.974 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.974 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.974 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.974 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.975 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.975 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.975 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2023-09-10T01:37:07.975 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2023-09-10T01:37:07.976 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.976 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.976 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.976 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "24", 2023-09-10T01:37:07.976 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "8", 2023-09-10T01:37:07.977 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.977 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.977 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.977 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.977 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.978 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.978 INFO:teuthology.orchestra.run.smithi044.stdout: "last_opt_revision": { 2023-09-10T01:37:07.978 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "last_opt_revision", 2023-09-10T01:37:07.978 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:07.979 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.979 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.979 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1", 2023-09-10T01:37:07.979 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.979 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.980 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.980 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.980 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.980 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.981 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.981 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard": { 2023-09-10T01:37:07.981 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard", 2023-09-10T01:37:07.981 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.981 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.982 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.982 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.982 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.982 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.982 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.983 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.983 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.983 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.983 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.984 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard_description": { 2023-09-10T01:37:07.984 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard_description", 2023-09-10T01:37:07.984 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.984 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.985 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:07.985 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.985 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.985 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.985 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.986 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.986 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.986 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.986 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.987 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:07.987 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:07.987 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.987 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.988 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.988 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:07.988 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.988 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.988 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.989 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.989 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.989 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.989 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.989 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.990 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.990 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.990 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.990 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.991 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.991 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.991 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:07.991 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:07.992 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.992 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.992 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.992 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:07.992 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.993 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.993 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:07.993 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.993 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.993 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.994 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.994 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:07.994 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:07.995 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:07.995 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.995 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.995 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:07.995 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:07.996 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:07.996 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:07.996 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:07.996 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:07.996 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:07.997 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:07.997 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:07.997 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:07.997 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:07.997 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:07.998 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:07.998 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:07.998 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:07.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:07.998 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:07.999 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:07.999 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:07.999 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:07.999 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:07.999 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.000 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.000 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.000 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.000 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.001 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.001 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.001 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.001 INFO:teuthology.orchestra.run.smithi044.stdout: "organization": { 2023-09-10T01:37:08.002 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "organization", 2023-09-10T01:37:08.002 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.002 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.002 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.003 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.003 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.003 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.003 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.003 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.004 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.004 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.004 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.004 INFO:teuthology.orchestra.run.smithi044.stdout: "proxy": { 2023-09-10T01:37:08.005 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "proxy", 2023-09-10T01:37:08.005 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.005 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.005 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.005 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.006 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.006 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.006 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.006 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.006 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.007 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.007 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.007 INFO:teuthology.orchestra.run.smithi044.stdout: "url": { 2023-09-10T01:37:08.007 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url", 2023-09-10T01:37:08.008 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.008 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.008 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.008 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-10T01:37:08.009 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.009 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.009 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.009 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.009 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.010 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.010 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.010 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.010 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.011 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "test_orchestrator", 2023-09-10T01:37:08.011 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.011 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.012 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.012 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.012 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.012 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.012 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.013 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.013 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.013 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.013 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.013 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.013 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.014 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.014 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.014 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.014 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.015 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.015 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.015 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.015 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.015 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.016 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.016 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.016 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.016 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.017 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.017 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.017 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.017 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.017 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.018 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.018 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.018 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.018 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.018 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.019 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.019 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.019 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.020 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.020 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.020 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.020 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.020 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.021 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.021 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.021 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.021 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.021 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.022 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.022 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.022 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.022 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.022 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.023 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.023 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.023 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.023 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.024 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.024 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.024 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.024 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.024 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.025 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.025 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.025 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.025 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.026 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.026 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.026 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.026 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.027 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "volumes", 2023-09-10T01:37:08.027 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.027 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.027 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.028 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.028 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.028 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.028 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.028 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.029 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.029 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.029 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.029 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.029 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.030 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.030 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.030 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.030 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.030 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.031 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.031 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.031 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.033 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.033 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.033 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.034 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.034 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.034 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.034 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.034 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.035 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.035 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.035 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.035 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.035 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.035 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.036 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.036 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.037 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.037 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.037 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.037 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.037 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.037 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.038 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.038 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.038 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.038 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.038 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.039 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.039 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.039 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.039 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.039 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.039 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.040 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.040 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.040 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.041 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.041 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.041 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.041 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.041 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.041 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.042 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.042 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.042 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.042 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.042 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.043 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.043 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_clones": { 2023-09-10T01:37:08.043 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_clones", 2023-09-10T01:37:08.043 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.043 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.044 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.044 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4", 2023-09-10T01:37:08.044 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.044 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.044 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.045 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-10T01:37:08.045 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.045 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.045 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.046 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_delay": { 2023-09-10T01:37:08.046 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_delay", 2023-09-10T01:37:08.046 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.046 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.046 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.047 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2023-09-10T01:37:08.047 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.047 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.047 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.047 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-10T01:37:08.047 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.048 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.048 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.049 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix", 2023-09-10T01:37:08.049 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.049 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.049 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.049 INFO:teuthology.orchestra.run.smithi044.stdout: "discovery_interval": { 2023-09-10T01:37:08.050 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "discovery_interval", 2023-09-10T01:37:08.050 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2023-09-10T01:37:08.050 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.050 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.050 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "100", 2023-09-10T01:37:08.051 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.051 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.051 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.051 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.051 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.052 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.052 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.052 INFO:teuthology.orchestra.run.smithi044.stdout: "identifier": { 2023-09-10T01:37:08.052 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "identifier", 2023-09-10T01:37:08.053 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.053 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.053 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.053 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.053 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.053 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.054 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.054 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.054 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.054 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.054 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.055 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2023-09-10T01:37:08.055 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2023-09-10T01:37:08.055 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.055 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.056 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.056 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2023-09-10T01:37:08.056 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.056 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.056 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.056 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.057 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.057 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.057 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.057 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.058 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.058 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.058 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.058 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.058 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.059 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.059 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.059 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.059 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.059 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.060 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.060 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.060 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.060 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.060 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.061 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.061 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.061 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.061 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.062 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.062 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.062 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.062 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.062 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.063 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.063 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.063 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.063 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.063 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.063 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.064 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.064 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.064 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.064 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.065 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.065 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.065 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.065 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.065 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.065 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.066 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.066 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.066 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.066 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.066 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.067 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.067 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.067 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.067 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.067 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.068 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.068 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.069 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.069 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.070 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.070 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.070 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.071 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.071 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.071 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.072 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.072 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.072 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.073 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.073 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.074 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_host": { 2023-09-10T01:37:08.074 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_host", 2023-09-10T01:37:08.074 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.075 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.075 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.075 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.075 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.076 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.076 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.076 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.076 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.077 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.077 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.077 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_port": { 2023-09-10T01:37:08.078 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_port", 2023-09-10T01:37:08.078 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.078 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.078 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.079 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10051", 2023-09-10T01:37:08.079 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.079 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.079 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.079 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.079 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.080 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.080 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.080 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_sender": { 2023-09-10T01:37:08.080 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_sender", 2023-09-10T01:37:08.081 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.081 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.081 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.081 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-10T01:37:08.081 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.082 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.082 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.082 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.082 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.082 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.083 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.083 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.083 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.083 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.083 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2023-09-10T01:37:08.084 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.084 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.084 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2023-09-10T01:37:08.084 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm", 2023-09-10T01:37:08.084 INFO:teuthology.orchestra.run.smithi044.stdout: "dashboard", 2023-09-10T01:37:08.085 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2023-09-10T01:37:08.085 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2023-09-10T01:37:08.085 INFO:teuthology.orchestra.run.smithi044.stdout: "restful", 2023-09-10T01:37:08.085 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_schedule" 2023-09-10T01:37:08.085 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.085 INFO:teuthology.orchestra.run.smithi044.stdout: "available_modules": [ 2023-09-10T01:37:08.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.086 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "alerts", 2023-09-10T01:37:08.086 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.086 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.086 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.087 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2023-09-10T01:37:08.087 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2023-09-10T01:37:08.087 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.087 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.087 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.088 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2023-09-10T01:37:08.088 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.088 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.088 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.088 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How frequently to reexamine health status", 2023-09-10T01:37:08.089 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.089 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.089 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.089 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.090 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.090 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.090 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.090 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.091 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.091 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.091 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.091 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.091 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.092 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.092 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.092 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.093 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.093 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.093 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.093 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.093 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.093 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.094 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.094 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.094 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.094 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.095 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.095 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.095 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.095 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.096 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.096 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.096 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.096 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.096 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.097 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.097 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.097 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.097 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.097 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.098 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.098 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.098 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.098 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.098 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.099 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.099 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.099 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.099 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.099 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.100 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.100 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.100 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.100 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.100 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.101 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.101 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.101 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.101 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.102 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.102 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.102 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.102 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.102 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.103 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.103 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.103 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.103 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.103 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.104 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_destination": { 2023-09-10T01:37:08.104 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_destination", 2023-09-10T01:37:08.104 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.104 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.104 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.105 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.105 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.105 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.105 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.105 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email address to send alerts to", 2023-09-10T01:37:08.105 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.106 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.106 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.106 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_from_name": { 2023-09-10T01:37:08.106 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_from_name", 2023-09-10T01:37:08.107 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.107 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.107 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.107 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "Ceph", 2023-09-10T01:37:08.107 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.108 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.108 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.108 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email From: name", 2023-09-10T01:37:08.108 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.108 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.108 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.109 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_host": { 2023-09-10T01:37:08.109 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_host", 2023-09-10T01:37:08.109 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.109 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.110 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.110 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.110 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.110 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.110 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.111 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP server", 2023-09-10T01:37:08.111 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.111 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.111 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.112 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_password": { 2023-09-10T01:37:08.112 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_password", 2023-09-10T01:37:08.112 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.112 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.112 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.113 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.113 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.113 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.113 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.113 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Password to authenticate with", 2023-09-10T01:37:08.113 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.114 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.114 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.114 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_port": { 2023-09-10T01:37:08.115 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_port", 2023-09-10T01:37:08.115 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.115 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.115 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.115 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "465", 2023-09-10T01:37:08.115 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.116 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.116 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.116 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP port", 2023-09-10T01:37:08.117 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.117 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.117 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.118 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_sender": { 2023-09-10T01:37:08.118 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_sender", 2023-09-10T01:37:08.119 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.119 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.119 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.120 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.120 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.120 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.121 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.121 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP envelope sender", 2023-09-10T01:37:08.121 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.122 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.122 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.123 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_ssl": { 2023-09-10T01:37:08.123 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_ssl", 2023-09-10T01:37:08.124 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.124 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.124 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.125 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.125 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.125 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.126 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.126 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-10T01:37:08.126 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.126 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.127 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.127 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_user": { 2023-09-10T01:37:08.127 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_user", 2023-09-10T01:37:08.127 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.128 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.128 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.128 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.128 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.128 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.128 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.129 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "User to authenticate as", 2023-09-10T01:37:08.129 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.129 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.129 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.129 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.130 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.130 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "balancer", 2023-09-10T01:37:08.130 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.131 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.131 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.131 INFO:teuthology.orchestra.run.smithi044.stdout: "active": { 2023-09-10T01:37:08.131 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "active", 2023-09-10T01:37:08.131 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.132 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.132 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.132 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.132 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.132 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.133 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.133 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically balance PGs across cluster", 2023-09-10T01:37:08.133 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.133 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.133 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.134 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_time": { 2023-09-10T01:37:08.134 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_time", 2023-09-10T01:37:08.134 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.134 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.134 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.135 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0000", 2023-09-10T01:37:08.135 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.135 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.135 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.135 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "beginning time of day to automatically balance", 2023-09-10T01:37:08.135 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-10T01:37:08.136 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.136 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.136 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_weekday": { 2023-09-10T01:37:08.136 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_weekday", 2023-09-10T01:37:08.137 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2023-09-10T01:37:08.137 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.137 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.137 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2023-09-10T01:37:08.138 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2023-09-10T01:37:08.138 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2023-09-10T01:37:08.138 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.138 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-10T01:37:08.138 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-10T01:37:08.139 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.139 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.139 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_max_iterations": { 2023-09-10T01:37:08.139 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_max_iterations", 2023-09-10T01:37:08.140 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2023-09-10T01:37:08.140 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.140 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.140 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "25", 2023-09-10T01:37:08.140 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2023-09-10T01:37:08.140 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "250", 2023-09-10T01:37:08.141 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.141 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-10T01:37:08.141 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.141 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.142 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.142 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_metrics": { 2023-09-10T01:37:08.142 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_metrics", 2023-09-10T01:37:08.142 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.143 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.143 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.143 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "pgs,objects,bytes", 2023-09-10T01:37:08.143 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.143 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.143 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.144 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-10T01:37:08.144 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-10T01:37:08.144 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.144 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.145 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_step": { 2023-09-10T01:37:08.145 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_step", 2023-09-10T01:37:08.145 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:08.145 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.145 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.146 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.5", 2023-09-10T01:37:08.146 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0.001", 2023-09-10T01:37:08.146 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "0.999", 2023-09-10T01:37:08.146 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.146 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "aggressiveness of optimization", 2023-09-10T01:37:08.147 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-10T01:37:08.147 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.147 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.147 INFO:teuthology.orchestra.run.smithi044.stdout: "end_time": { 2023-09-10T01:37:08.148 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_time", 2023-09-10T01:37:08.148 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.148 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.148 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.148 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2359", 2023-09-10T01:37:08.148 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.149 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.149 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.149 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ending time of day to automatically balance", 2023-09-10T01:37:08.149 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-10T01:37:08.149 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.150 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.150 INFO:teuthology.orchestra.run.smithi044.stdout: "end_weekday": { 2023-09-10T01:37:08.150 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_weekday", 2023-09-10T01:37:08.150 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2023-09-10T01:37:08.151 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.151 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.151 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2023-09-10T01:37:08.151 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2023-09-10T01:37:08.151 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2023-09-10T01:37:08.152 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.152 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-10T01:37:08.152 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-10T01:37:08.152 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.152 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.153 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.153 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.153 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.153 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.153 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.154 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.154 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.154 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.154 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.154 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.155 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.155 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.155 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.155 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.155 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.155 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.156 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.156 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.156 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.156 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.157 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.157 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.157 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.157 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.157 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.158 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.158 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.158 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.158 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.158 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.159 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.159 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.159 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.159 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.159 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.160 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.160 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.160 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.160 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.160 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.161 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.161 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.161 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.161 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.161 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.162 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.162 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.162 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.162 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.162 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.163 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.163 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.163 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.163 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.164 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.164 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.164 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.164 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.164 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.165 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.165 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.165 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.165 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.165 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.166 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.166 INFO:teuthology.orchestra.run.smithi044.stdout: "min_score": { 2023-09-10T01:37:08.166 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_score", 2023-09-10T01:37:08.166 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:08.167 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.167 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.167 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2023-09-10T01:37:08.167 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.167 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.167 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.168 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-10T01:37:08.168 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.168 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.168 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.169 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2023-09-10T01:37:08.169 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2023-09-10T01:37:08.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.169 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.169 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.169 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "upmap", 2023-09-10T01:37:08.170 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.170 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.170 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.170 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-compat", 2023-09-10T01:37:08.170 INFO:teuthology.orchestra.run.smithi044.stdout: "none", 2023-09-10T01:37:08.171 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap" 2023-09-10T01:37:08.171 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.171 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Balancer mode", 2023-09-10T01:37:08.171 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.171 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.172 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.172 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_ids": { 2023-09-10T01:37:08.172 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_ids", 2023-09-10T01:37:08.172 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.173 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.173 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.173 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.173 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.174 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.174 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.174 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-10T01:37:08.174 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.174 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.174 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.175 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2023-09-10T01:37:08.175 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2023-09-10T01:37:08.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.175 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.176 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.176 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2023-09-10T01:37:08.176 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.177 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.177 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.177 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-10T01:37:08.178 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.178 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.178 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.179 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_deviation": { 2023-09-10T01:37:08.179 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_deviation", 2023-09-10T01:37:08.179 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.180 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.180 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.180 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2023-09-10T01:37:08.181 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2023-09-10T01:37:08.181 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.182 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.182 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-10T01:37:08.182 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-10T01:37:08.183 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.183 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.184 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_optimizations": { 2023-09-10T01:37:08.184 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_optimizations", 2023-09-10T01:37:08.184 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2023-09-10T01:37:08.185 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.185 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.185 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:08.186 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.186 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.186 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.187 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-10T01:37:08.187 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.187 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.187 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.188 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.188 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.189 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm", 2023-09-10T01:37:08.189 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.189 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.189 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.189 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_down_multiplier": { 2023-09-10T01:37:08.190 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_down_multiplier", 2023-09-10T01:37:08.190 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:08.190 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.190 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.191 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2023-09-10T01:37:08.191 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.191 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.191 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.191 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-10T01:37:08.192 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.192 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.192 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.192 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_refresh_rate": { 2023-09-10T01:37:08.192 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_refresh_rate", 2023-09-10T01:37:08.193 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.193 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.193 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.193 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "20", 2023-09-10T01:37:08.193 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.194 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.194 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.194 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-10T01:37:08.194 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.194 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.195 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.195 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_starting_port": { 2023-09-10T01:37:08.195 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_starting_port", 2023-09-10T01:37:08.195 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.196 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.196 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.196 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4721", 2023-09-10T01:37:08.196 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.196 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.197 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.197 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-10T01:37:08.197 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.197 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.197 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.198 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_ptrace": { 2023-09-10T01:37:08.198 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_ptrace", 2023-09-10T01:37:08.198 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.198 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.199 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.199 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.199 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.199 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.199 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.200 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-10T01:37:08.200 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-10T01:37:08.200 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.200 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.201 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_interval": { 2023-09-10T01:37:08.201 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_interval", 2023-09-10T01:37:08.201 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.201 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.201 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.201 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2023-09-10T01:37:08.202 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.202 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.202 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.202 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-10T01:37:08.203 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.203 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.203 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.203 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_memory_target_ratio": { 2023-09-10T01:37:08.204 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_memory_target_ratio", 2023-09-10T01:37:08.204 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:08.204 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.204 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.204 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.7", 2023-09-10T01:37:08.205 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.205 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.205 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.205 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-10T01:37:08.205 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.206 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.206 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.206 INFO:teuthology.orchestra.run.smithi044.stdout: "cgroups_split": { 2023-09-10T01:37:08.206 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cgroups_split", 2023-09-10T01:37:08.207 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.207 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.207 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.207 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.207 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.207 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.208 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.208 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-10T01:37:08.208 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.208 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.208 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.209 INFO:teuthology.orchestra.run.smithi044.stdout: "config_checks_enabled": { 2023-09-10T01:37:08.209 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_checks_enabled", 2023-09-10T01:37:08.209 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.209 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.210 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.210 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.210 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.210 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.210 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.211 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-10T01:37:08.211 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.211 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.211 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.211 INFO:teuthology.orchestra.run.smithi044.stdout: "config_dashboard": { 2023-09-10T01:37:08.212 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_dashboard", 2023-09-10T01:37:08.212 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.212 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.212 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.212 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.213 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.213 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.213 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.213 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-10T01:37:08.214 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.214 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.214 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.215 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_alertmanager": { 2023-09-10T01:37:08.215 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_alertmanager", 2023-09-10T01:37:08.215 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.216 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.216 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.216 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-10T01:37:08.217 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.217 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.217 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.218 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2023-09-10T01:37:08.218 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.218 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.219 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.219 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_base": { 2023-09-10T01:37:08.220 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_base", 2023-09-10T01:37:08.220 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.220 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.221 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.221 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-10T01:37:08.221 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.222 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.222 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.222 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Container image name, without the tag", 2023-09-10T01:37:08.223 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.223 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.223 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.224 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_elasticsearch": { 2023-09-10T01:37:08.224 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_elasticsearch", 2023-09-10T01:37:08.224 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.224 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.225 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.225 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-10T01:37:08.225 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.225 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.225 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.225 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "elasticsearch container image", 2023-09-10T01:37:08.226 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.226 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.226 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.226 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_grafana": { 2023-09-10T01:37:08.227 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_grafana", 2023-09-10T01:37:08.227 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.227 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.227 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.227 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-10T01:37:08.227 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.228 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.228 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.228 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2023-09-10T01:37:08.228 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.229 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.229 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.229 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_haproxy": { 2023-09-10T01:37:08.229 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_haproxy", 2023-09-10T01:37:08.230 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.230 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.230 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.230 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-10T01:37:08.231 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.231 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.231 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.231 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "HAproxy container image", 2023-09-10T01:37:08.231 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.231 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.232 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.232 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_agent": { 2023-09-10T01:37:08.232 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_agent", 2023-09-10T01:37:08.232 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.232 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.233 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.233 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-10T01:37:08.233 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.233 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.233 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.234 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger agent container image", 2023-09-10T01:37:08.234 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.234 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.234 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.235 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_collector": { 2023-09-10T01:37:08.235 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_collector", 2023-09-10T01:37:08.235 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.235 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.235 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.235 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-10T01:37:08.236 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.236 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.236 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.236 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger collector container image", 2023-09-10T01:37:08.236 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.236 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.237 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.237 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_query": { 2023-09-10T01:37:08.237 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_query", 2023-09-10T01:37:08.237 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.237 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.238 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.238 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-10T01:37:08.238 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.238 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.239 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.239 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger query container image", 2023-09-10T01:37:08.239 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.239 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.239 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.240 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_keepalived": { 2023-09-10T01:37:08.240 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_keepalived", 2023-09-10T01:37:08.240 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.240 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.240 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.241 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-10T01:37:08.241 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.241 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.241 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.241 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Keepalived container image", 2023-09-10T01:37:08.242 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.242 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.242 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.242 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_loki": { 2023-09-10T01:37:08.243 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_loki", 2023-09-10T01:37:08.243 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.243 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.243 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.243 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-10T01:37:08.243 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.244 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.244 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.244 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Loki container image", 2023-09-10T01:37:08.244 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.244 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.245 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.245 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_node_exporter": { 2023-09-10T01:37:08.245 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_node_exporter", 2023-09-10T01:37:08.245 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.245 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.246 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.246 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-10T01:37:08.246 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.246 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.246 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.247 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2023-09-10T01:37:08.247 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.247 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.247 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.247 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_nvmeof": { 2023-09-10T01:37:08.248 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_nvmeof", 2023-09-10T01:37:08.248 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.248 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.248 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.248 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-10T01:37:08.248 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.249 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.249 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.249 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Nvme-of container image", 2023-09-10T01:37:08.249 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.249 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.249 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.250 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_prometheus": { 2023-09-10T01:37:08.250 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_prometheus", 2023-09-10T01:37:08.250 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.250 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.251 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.251 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-10T01:37:08.251 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.251 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.251 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.252 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2023-09-10T01:37:08.252 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.252 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.252 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.252 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_promtail": { 2023-09-10T01:37:08.253 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_promtail", 2023-09-10T01:37:08.253 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.253 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.253 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.253 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-10T01:37:08.254 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.254 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.254 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.254 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Promtail container image", 2023-09-10T01:37:08.254 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.255 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.255 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.255 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_snmp_gateway": { 2023-09-10T01:37:08.255 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_snmp_gateway", 2023-09-10T01:37:08.255 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.256 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.256 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.256 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-10T01:37:08.256 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.256 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.257 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.257 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SNMP Gateway container image", 2023-09-10T01:37:08.257 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.257 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.257 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.258 INFO:teuthology.orchestra.run.smithi044.stdout: "container_init": { 2023-09-10T01:37:08.258 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_init", 2023-09-10T01:37:08.258 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.258 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.258 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.259 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.259 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.259 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.260 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.260 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Run podman/docker with `--init`", 2023-09-10T01:37:08.260 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.261 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.261 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.261 INFO:teuthology.orchestra.run.smithi044.stdout: "daemon_cache_timeout": { 2023-09-10T01:37:08.261 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "daemon_cache_timeout", 2023-09-10T01:37:08.262 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.262 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.262 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.262 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2023-09-10T01:37:08.262 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.263 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.263 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.263 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-10T01:37:08.263 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.263 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.264 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.264 INFO:teuthology.orchestra.run.smithi044.stdout: "default_cephadm_command_timeout": { 2023-09-10T01:37:08.264 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_cephadm_command_timeout", 2023-09-10T01:37:08.264 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.265 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.265 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.265 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "900", 2023-09-10T01:37:08.265 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.265 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.265 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.266 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-10T01:37:08.266 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.266 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.266 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.267 INFO:teuthology.orchestra.run.smithi044.stdout: "default_registry": { 2023-09-10T01:37:08.267 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_registry", 2023-09-10T01:37:08.267 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.267 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.267 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.268 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io", 2023-09-10T01:37:08.268 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.268 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.268 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.269 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-10T01:37:08.269 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.269 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.269 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.270 INFO:teuthology.orchestra.run.smithi044.stdout: "device_cache_timeout": { 2023-09-10T01:37:08.270 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_cache_timeout", 2023-09-10T01:37:08.270 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.270 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.271 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.271 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1800", 2023-09-10T01:37:08.271 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.271 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.271 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.272 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache device inventory", 2023-09-10T01:37:08.272 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.272 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.272 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.273 INFO:teuthology.orchestra.run.smithi044.stdout: "device_enhanced_scan": { 2023-09-10T01:37:08.273 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_enhanced_scan", 2023-09-10T01:37:08.273 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.273 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.273 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.274 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.274 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.274 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.274 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.275 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-10T01:37:08.275 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.275 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.275 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.276 INFO:teuthology.orchestra.run.smithi044.stdout: "facts_cache_timeout": { 2023-09-10T01:37:08.276 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "facts_cache_timeout", 2023-09-10T01:37:08.276 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.276 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.276 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.277 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2023-09-10T01:37:08.277 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.277 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.277 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.278 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache host facts data", 2023-09-10T01:37:08.278 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.278 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.278 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.279 INFO:teuthology.orchestra.run.smithi044.stdout: "host_check_interval": { 2023-09-10T01:37:08.279 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "host_check_interval", 2023-09-10T01:37:08.279 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.279 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.280 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.280 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2023-09-10T01:37:08.280 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.280 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.281 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.281 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to perform a host check", 2023-09-10T01:37:08.281 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.281 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.281 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.282 INFO:teuthology.orchestra.run.smithi044.stdout: "inventory_list_all": { 2023-09-10T01:37:08.282 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "inventory_list_all", 2023-09-10T01:37:08.282 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.283 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.283 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.283 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.283 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.284 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.284 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.284 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-10T01:37:08.284 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.284 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.285 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.285 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.285 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.285 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.286 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.286 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.286 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.286 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.287 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.287 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.287 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.287 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.287 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.288 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.288 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.288 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.288 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.288 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.289 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.289 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.289 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.290 INFO:teuthology.orchestra.run.smithi044.stdout: "log_refresh_metadata": { 2023-09-10T01:37:08.290 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_refresh_metadata", 2023-09-10T01:37:08.290 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.290 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.290 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.291 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.291 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.291 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.291 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.292 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-10T01:37:08.292 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.292 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.292 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.293 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.293 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.293 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.293 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.294 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.294 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.294 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.294 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.294 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.294 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-10T01:37:08.295 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.295 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.295 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.295 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.296 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.296 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.296 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.296 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.296 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.297 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.297 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.297 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.297 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.297 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.298 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.298 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.298 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.298 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.298 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.299 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.299 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.299 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.299 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.300 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.300 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.300 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.300 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.300 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.300 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.301 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.301 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.301 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.301 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.301 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.302 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.302 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.302 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-10T01:37:08.302 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-10T01:37:08.303 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.303 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.303 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.303 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.303 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.304 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.304 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.304 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-10T01:37:08.304 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.304 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.305 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.305 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-10T01:37:08.305 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-10T01:37:08.305 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.305 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.306 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.306 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "*", 2023-09-10T01:37:08.306 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.306 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.306 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.307 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-10T01:37:08.307 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.307 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.307 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.308 INFO:teuthology.orchestra.run.smithi044.stdout: "max_count_per_host": { 2023-09-10T01:37:08.308 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_count_per_host", 2023-09-10T01:37:08.308 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.308 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.308 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.308 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:08.309 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.309 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.309 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.309 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of daemons per service per host", 2023-09-10T01:37:08.309 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.309 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.310 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.310 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd_draining_count": { 2023-09-10T01:37:08.310 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_osd_draining_count", 2023-09-10T01:37:08.310 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.311 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.311 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.311 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:08.311 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.311 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.312 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.312 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-10T01:37:08.312 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.312 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.312 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.313 INFO:teuthology.orchestra.run.smithi044.stdout: "migration_current": { 2023-09-10T01:37:08.313 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "migration_current", 2023-09-10T01:37:08.313 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.313 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.313 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.314 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.314 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.314 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.314 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.314 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "internal - do not modify", 2023-09-10T01:37:08.315 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.315 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.315 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.315 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2023-09-10T01:37:08.316 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2023-09-10T01:37:08.316 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.316 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.316 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.316 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "root", 2023-09-10T01:37:08.317 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.317 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.317 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.318 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm-package", 2023-09-10T01:37:08.318 INFO:teuthology.orchestra.run.smithi044.stdout: "root" 2023-09-10T01:37:08.318 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.318 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "mode for remote execution of cephadm", 2023-09-10T01:37:08.318 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.319 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.319 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.319 INFO:teuthology.orchestra.run.smithi044.stdout: "prometheus_alerts_path": { 2023-09-10T01:37:08.319 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus_alerts_path", 2023-09-10T01:37:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-10T01:37:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-10T01:37:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_insecure": { 2023-09-10T01:37:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_insecure", 2023-09-10T01:37:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-10T01:37:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_password": { 2023-09-10T01:37:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_password", 2023-09-10T01:37:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-10T01:37:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_url": { 2023-09-10T01:37:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_url", 2023-09-10T01:37:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.328 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.328 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.328 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.328 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.328 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.328 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-10T01:37:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_username": { 2023-09-10T01:37:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_username", 2023-09-10T01:37:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.331 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.331 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.331 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.331 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.331 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-10T01:37:08.331 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.332 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.332 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.332 INFO:teuthology.orchestra.run.smithi044.stdout: "secure_monitoring_stack": { 2023-09-10T01:37:08.332 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "secure_monitoring_stack", 2023-09-10T01:37:08.333 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.333 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.333 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.333 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.333 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.333 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.334 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.334 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-10T01:37:08.334 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.334 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.335 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.335 INFO:teuthology.orchestra.run.smithi044.stdout: "service_discovery_port": { 2023-09-10T01:37:08.335 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "service_discovery_port", 2023-09-10T01:37:08.335 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.335 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.336 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.336 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8765", 2023-09-10T01:37:08.336 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.336 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.336 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.337 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "cephadm service discovery port", 2023-09-10T01:37:08.337 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.337 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.337 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.337 INFO:teuthology.orchestra.run.smithi044.stdout: "ssh_config_file": { 2023-09-10T01:37:08.338 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssh_config_file", 2023-09-10T01:37:08.338 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.338 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.338 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.338 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.339 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.339 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.339 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.339 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-10T01:37:08.339 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.340 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.340 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.340 INFO:teuthology.orchestra.run.smithi044.stdout: "use_agent": { 2023-09-10T01:37:08.340 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_agent", 2023-09-10T01:37:08.341 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.341 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.341 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.341 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.341 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.341 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.342 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.342 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-10T01:37:08.342 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.342 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.342 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.343 INFO:teuthology.orchestra.run.smithi044.stdout: "use_repo_digest": { 2023-09-10T01:37:08.343 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_repo_digest", 2023-09-10T01:37:08.343 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.343 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.344 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.344 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.344 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.344 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.344 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.345 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-10T01:37:08.345 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.345 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.345 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.346 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_failed_host_check": { 2023-09-10T01:37:08.346 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_failed_host_check", 2023-09-10T01:37:08.346 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.346 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.346 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.347 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.347 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.347 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.347 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.347 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if the host check fails", 2023-09-10T01:37:08.348 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.348 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.348 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.348 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_daemons": { 2023-09-10T01:37:08.348 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_daemons", 2023-09-10T01:37:08.349 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.349 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.349 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.349 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.349 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.350 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.350 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.350 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-10T01:37:08.350 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.350 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.351 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.351 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_hosts": { 2023-09-10T01:37:08.351 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_hosts", 2023-09-10T01:37:08.351 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.351 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.352 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.352 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.352 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.352 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.352 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.353 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-10T01:37:08.353 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.353 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.353 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.353 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.353 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.354 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crash", 2023-09-10T01:37:08.354 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.354 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.355 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.355 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.355 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.355 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.355 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.356 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.356 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.356 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.356 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.356 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.356 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.357 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.357 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.357 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.357 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.357 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.358 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.358 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.358 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.358 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.358 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.359 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.359 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.359 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.359 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.359 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.360 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.360 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.360 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.360 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.360 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.361 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.361 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.361 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.361 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.362 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.362 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.362 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.362 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.362 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.363 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.363 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.363 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.363 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.364 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.364 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.364 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.364 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.364 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.365 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.365 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.365 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.365 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.365 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.366 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.366 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.366 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.366 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.366 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.367 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.367 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.367 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.367 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.367 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.368 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.368 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.368 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.368 INFO:teuthology.orchestra.run.smithi044.stdout: "retain_interval": { 2023-09-10T01:37:08.368 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retain_interval", 2023-09-10T01:37:08.369 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.369 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.369 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.369 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "31536000", 2023-09-10T01:37:08.369 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.370 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.370 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.370 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-10T01:37:08.370 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.370 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.371 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.371 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_recent_interval": { 2023-09-10T01:37:08.371 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_recent_interval", 2023-09-10T01:37:08.371 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.372 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.372 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.372 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1209600", 2023-09-10T01:37:08.372 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.372 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.373 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.373 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-10T01:37:08.373 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.373 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.373 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.373 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.374 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.374 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dashboard", 2023-09-10T01:37:08.375 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.375 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.375 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.375 INFO:teuthology.orchestra.run.smithi044.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-10T01:37:08.375 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-10T01:37:08.375 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.376 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.376 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.376 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:08.376 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.376 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.377 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.377 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.377 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.377 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.377 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.378 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_HOST": { 2023-09-10T01:37:08.378 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-10T01:37:08.378 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.379 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.379 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.379 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.379 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.379 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.380 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.380 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.380 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.380 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.380 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.381 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-10T01:37:08.381 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-10T01:37:08.381 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.381 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.382 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.382 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.382 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.382 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.382 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.383 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.383 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.383 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.383 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.384 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_ENABLED": { 2023-09-10T01:37:08.384 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_ENABLED", 2023-09-10T01:37:08.384 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.384 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.384 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.384 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.385 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.385 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.385 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.385 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.385 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.386 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.386 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.386 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-10T01:37:08.386 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-10T01:37:08.387 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.387 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.387 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.387 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.387 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.388 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.388 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.388 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.388 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.388 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.388 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.389 INFO:teuthology.orchestra.run.smithi044.stdout: "ENABLE_BROWSABLE_API": { 2023-09-10T01:37:08.389 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-10T01:37:08.389 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.389 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.390 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.390 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.390 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.390 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.390 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.391 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.391 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.391 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.391 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.391 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-10T01:37:08.392 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-10T01:37:08.392 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.392 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.392 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.392 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.393 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.393 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.393 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.393 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.393 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.393 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.394 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.394 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-10T01:37:08.394 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-10T01:37:08.395 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.395 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.395 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.395 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.395 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.396 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.396 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.396 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.396 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.396 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.396 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.397 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-10T01:37:08.397 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-10T01:37:08.397 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.397 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.398 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.398 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.398 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.398 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.398 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.399 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.399 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.399 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.399 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.400 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-10T01:37:08.400 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-10T01:37:08.400 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.400 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.400 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.401 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.401 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.401 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.401 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.401 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.402 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.402 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.402 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.402 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-10T01:37:08.403 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-10T01:37:08.403 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.403 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.403 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.403 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.404 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.404 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.404 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.404 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.404 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.404 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.405 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.405 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-10T01:37:08.405 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-10T01:37:08.405 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.406 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.406 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.406 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.406 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.406 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.407 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.407 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.407 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.407 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.407 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.408 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-10T01:37:08.408 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-10T01:37:08.408 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.408 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.408 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.409 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.409 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.409 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.409 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.409 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.410 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.410 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.410 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.410 INFO:teuthology.orchestra.run.smithi044.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-10T01:37:08.411 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-10T01:37:08.411 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.411 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.411 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.411 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.411 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.412 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.412 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.412 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.412 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.412 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.413 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.413 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_PASSWORD": { 2023-09-10T01:37:08.413 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-10T01:37:08.413 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.414 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.414 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.414 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2023-09-10T01:37:08.414 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.414 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.415 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.415 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.415 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.415 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.415 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.416 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-10T01:37:08.416 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-10T01:37:08.416 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.416 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.416 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.417 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.417 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.417 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.417 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.417 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.417 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.418 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.418 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.418 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_URL": { 2023-09-10T01:37:08.418 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_URL", 2023-09-10T01:37:08.419 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.419 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.419 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.419 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.419 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.420 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.420 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.420 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.420 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.420 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.420 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.421 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_USERNAME": { 2023-09-10T01:37:08.421 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-10T01:37:08.421 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.421 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.422 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.422 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2023-09-10T01:37:08.422 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.422 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.422 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.423 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.423 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.423 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.423 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.423 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-10T01:37:08.424 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-10T01:37:08.424 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.424 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.424 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.424 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.425 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.425 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.425 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.425 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.425 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.426 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.426 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.426 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-10T01:37:08.426 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-10T01:37:08.427 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.427 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.427 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.427 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.427 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.428 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.428 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.428 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.428 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.428 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.428 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.429 INFO:teuthology.orchestra.run.smithi044.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-10T01:37:08.429 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-10T01:37:08.429 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.429 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.430 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.430 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.430 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.430 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.430 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.431 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.431 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.431 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.431 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.432 INFO:teuthology.orchestra.run.smithi044.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-10T01:37:08.432 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-10T01:37:08.432 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.432 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.432 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.432 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.433 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.433 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.433 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.433 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.433 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.433 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.434 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.434 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_HOST": { 2023-09-10T01:37:08.434 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-10T01:37:08.434 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.435 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.435 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.435 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.435 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.435 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.435 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.436 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.436 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.436 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.436 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.436 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-10T01:37:08.437 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-10T01:37:08.437 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.437 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.437 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.437 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.438 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.438 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.438 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.438 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.438 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.439 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.439 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.439 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-10T01:37:08.439 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-10T01:37:08.440 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.440 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.440 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.440 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.440 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.440 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.441 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.441 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.441 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.441 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.441 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.442 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-10T01:37:08.445 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-10T01:37:08.445 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.445 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.445 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.445 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.446 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.446 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.446 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.446 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.446 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.447 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.447 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.447 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-10T01:37:08.447 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-10T01:37:08.448 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.448 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.448 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.448 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.448 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.448 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.449 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.449 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.449 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.449 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.449 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.450 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-10T01:37:08.450 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-10T01:37:08.450 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.450 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.451 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.451 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.451 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.451 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.451 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.452 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.452 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.452 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.452 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.452 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-10T01:37:08.453 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-10T01:37:08.453 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.453 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.453 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.453 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.454 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.454 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.454 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.454 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.454 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.455 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.455 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.455 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-10T01:37:08.455 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-10T01:37:08.455 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.456 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.456 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.456 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.456 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.456 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.457 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.457 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.457 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.457 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.457 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.458 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-10T01:37:08.458 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-10T01:37:08.458 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.458 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.458 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.459 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.459 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.459 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.459 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.459 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.459 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.460 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.460 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.460 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_ENABLED": { 2023-09-10T01:37:08.460 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-10T01:37:08.461 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.461 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.461 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.461 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.461 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.462 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.462 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.462 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.462 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.462 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.463 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.463 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-10T01:37:08.463 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-10T01:37:08.463 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.463 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.464 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.464 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-10T01:37:08.464 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.464 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.464 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.464 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.465 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.465 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.465 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.465 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-10T01:37:08.466 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-10T01:37:08.466 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.466 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.466 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.466 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:08.467 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.467 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.467 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.467 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.467 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.467 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.468 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.468 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-10T01:37:08.468 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-10T01:37:08.468 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.469 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.469 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.469 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8", 2023-09-10T01:37:08.469 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.469 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.469 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.470 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.470 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.470 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.470 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.471 INFO:teuthology.orchestra.run.smithi044.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-10T01:37:08.471 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-10T01:37:08.471 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.471 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.471 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.471 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "45", 2023-09-10T01:37:08.472 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.472 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.472 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.472 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.472 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.473 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.473 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.473 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ACCESS_KEY": { 2023-09-10T01:37:08.473 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-10T01:37:08.473 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.474 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.474 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.474 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.474 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.474 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.475 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.475 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.475 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.475 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.475 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.476 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-10T01:37:08.476 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-10T01:37:08.476 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.476 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.476 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.477 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2023-09-10T01:37:08.477 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.477 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.477 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.477 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.478 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.478 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.478 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.478 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SECRET_KEY": { 2023-09-10T01:37:08.478 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-10T01:37:08.479 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.479 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.479 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.479 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.479 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.479 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.480 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.480 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.480 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.480 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.480 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.481 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SSL_VERIFY": { 2023-09-10T01:37:08.481 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-10T01:37:08.481 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.481 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.481 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.482 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.482 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.482 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.482 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.482 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.482 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.483 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.483 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.483 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-10T01:37:08.483 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-10T01:37:08.484 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.484 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.484 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.484 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2023-09-10T01:37:08.484 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.484 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.485 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.485 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.485 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.485 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.485 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.486 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-10T01:37:08.486 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-10T01:37:08.486 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.486 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.486 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.486 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:08.487 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.487 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.487 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.487 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.487 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.488 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.488 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.488 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-10T01:37:08.488 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-10T01:37:08.488 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.489 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.489 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.489 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2023-09-10T01:37:08.489 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.489 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.489 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.490 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.490 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.490 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.490 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.491 INFO:teuthology.orchestra.run.smithi044.stdout: "cross_origin_url": { 2023-09-10T01:37:08.491 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cross_origin_url", 2023-09-10T01:37:08.491 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.491 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.491 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.491 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.492 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.492 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.492 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.492 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.492 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.493 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.493 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.493 INFO:teuthology.orchestra.run.smithi044.stdout: "crt_file": { 2023-09-10T01:37:08.493 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crt_file", 2023-09-10T01:37:08.493 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.494 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.494 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.494 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.494 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.494 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.495 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.495 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.495 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.495 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.495 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.496 INFO:teuthology.orchestra.run.smithi044.stdout: "debug": { 2023-09-10T01:37:08.496 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "debug", 2023-09-10T01:37:08.496 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.496 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.496 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.497 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.497 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.497 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.497 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.497 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable/disable debug options", 2023-09-10T01:37:08.497 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.498 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.498 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.498 INFO:teuthology.orchestra.run.smithi044.stdout: "jwt_token_ttl": { 2023-09-10T01:37:08.498 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "jwt_token_ttl", 2023-09-10T01:37:08.499 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.499 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.499 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.499 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "28800", 2023-09-10T01:37:08.499 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.499 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.500 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.500 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.500 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.500 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.500 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.501 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2023-09-10T01:37:08.501 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2023-09-10T01:37:08.501 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.501 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.501 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.501 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.502 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.502 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.502 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.502 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.502 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.503 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.503 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.503 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.503 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.503 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.504 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.504 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.504 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.504 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.504 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.504 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.505 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.505 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.505 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.505 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.505 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.505 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.506 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.506 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.506 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.506 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.506 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.507 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.507 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.507 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.507 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.507 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.508 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.508 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.508 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.508 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.508 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.508 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.509 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.509 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.509 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.509 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.509 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.510 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.510 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.510 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.510 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.510 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.511 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.511 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.511 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.511 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.511 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.512 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.512 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.512 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.512 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.512 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.513 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.513 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.513 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.513 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.513 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.514 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.514 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.514 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.514 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.514 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.515 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.515 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.515 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.515 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.515 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.516 INFO:teuthology.orchestra.run.smithi044.stdout: "motd": { 2023-09-10T01:37:08.516 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "motd", 2023-09-10T01:37:08.516 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.516 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.517 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.517 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.517 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.517 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.517 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.517 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "The message of the day", 2023-09-10T01:37:08.518 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.518 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.518 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.518 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect_resolve_ip_addr": { 2023-09-10T01:37:08.519 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "redirect_resolve_ip_addr", 2023-09-10T01:37:08.519 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.519 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.519 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.519 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.520 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.520 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.520 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.520 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.520 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.521 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.521 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.521 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2023-09-10T01:37:08.521 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2023-09-10T01:37:08.522 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.522 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.522 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.522 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2023-09-10T01:37:08.522 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.523 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.523 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.523 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.523 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.523 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.523 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.524 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2023-09-10T01:37:08.524 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2023-09-10T01:37:08.524 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.524 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.525 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.525 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8080", 2023-09-10T01:37:08.525 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.525 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.525 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.525 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.526 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.526 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.526 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.526 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2023-09-10T01:37:08.527 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2023-09-10T01:37:08.527 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.527 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.527 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.527 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.528 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.528 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.528 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.528 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.528 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.529 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.529 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.529 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl_server_port": { 2023-09-10T01:37:08.529 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl_server_port", 2023-09-10T01:37:08.529 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.530 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.530 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.530 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8443", 2023-09-10T01:37:08.530 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.530 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.531 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.531 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.531 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.531 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.531 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.532 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2023-09-10T01:37:08.532 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2023-09-10T01:37:08.532 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.532 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.532 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.533 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "redirect", 2023-09-10T01:37:08.533 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.533 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.533 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.533 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.534 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect" 2023-09-10T01:37:08.534 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.534 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.534 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.534 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.535 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.535 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2023-09-10T01:37:08.535 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2023-09-10T01:37:08.535 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.535 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.536 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.536 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2023-09-10T01:37:08.536 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2023-09-10T01:37:08.536 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2023-09-10T01:37:08.536 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.537 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.537 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.537 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.537 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.537 INFO:teuthology.orchestra.run.smithi044.stdout: "url_prefix": { 2023-09-10T01:37:08.538 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url_prefix", 2023-09-10T01:37:08.538 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.538 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.538 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.538 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.539 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.539 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.539 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.539 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.539 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.540 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.540 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.540 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.540 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.541 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "devicehealth", 2023-09-10T01:37:08.541 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.541 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.541 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.541 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_monitoring": { 2023-09-10T01:37:08.542 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_monitoring", 2023-09-10T01:37:08.542 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.542 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.542 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.542 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.543 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.543 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.543 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.543 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "monitor device health metrics", 2023-09-10T01:37:08.543 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.543 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.544 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.544 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.544 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.544 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.545 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.545 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.545 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.545 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.545 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.545 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.546 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.546 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.546 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.546 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.546 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.547 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.547 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.547 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.547 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.547 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.548 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.548 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.548 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.548 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.549 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.549 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.549 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.549 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.549 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.550 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.550 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.550 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.550 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.550 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.551 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.551 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.551 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.551 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.551 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.552 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.552 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.552 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.552 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.552 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.553 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.553 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.553 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.553 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.553 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.554 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.554 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.554 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.554 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.554 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.555 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.555 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.555 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.555 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.555 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.556 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.556 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.556 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.556 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.556 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.556 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.557 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.557 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.557 INFO:teuthology.orchestra.run.smithi044.stdout: "mark_out_threshold": { 2023-09-10T01:37:08.557 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mark_out_threshold", 2023-09-10T01:37:08.558 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.558 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.558 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.558 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2419200", 2023-09-10T01:37:08.558 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.559 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.559 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.559 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-10T01:37:08.559 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.559 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.560 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.560 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": { 2023-09-10T01:37:08.560 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_name", 2023-09-10T01:37:08.560 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.561 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.561 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.561 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "device_health_metrics", 2023-09-10T01:37:08.561 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.561 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.562 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.562 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-10T01:37:08.562 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.562 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.562 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.563 INFO:teuthology.orchestra.run.smithi044.stdout: "retention_period": { 2023-09-10T01:37:08.563 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retention_period", 2023-09-10T01:37:08.563 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.563 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.563 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.563 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15552000", 2023-09-10T01:37:08.564 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.564 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.564 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.564 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain device health metrics", 2023-09-10T01:37:08.564 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.564 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_frequency": { 2023-09-10T01:37:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_frequency", 2023-09-10T01:37:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.566 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.566 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.566 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2023-09-10T01:37:08.566 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.566 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.567 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.567 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-10T01:37:08.567 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.567 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.567 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.568 INFO:teuthology.orchestra.run.smithi044.stdout: "self_heal": { 2023-09-10T01:37:08.568 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "self_heal", 2023-09-10T01:37:08.568 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.568 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.568 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.569 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.569 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.569 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.569 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.569 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-10T01:37:08.570 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.570 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.570 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.570 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2023-09-10T01:37:08.571 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2023-09-10T01:37:08.571 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.571 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.571 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.571 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2023-09-10T01:37:08.572 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.572 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.572 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.572 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and check device health", 2023-09-10T01:37:08.572 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.573 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.573 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.573 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_threshold": { 2023-09-10T01:37:08.573 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_threshold", 2023-09-10T01:37:08.574 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.574 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.574 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.574 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7257600", 2023-09-10T01:37:08.574 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.575 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.575 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.575 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-10T01:37:08.575 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.575 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.576 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.576 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.576 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.576 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "diskprediction_local", 2023-09-10T01:37:08.577 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.577 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.577 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.577 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.577 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.578 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.578 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.578 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.578 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.578 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.579 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.579 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.579 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.579 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.579 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.579 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.580 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.580 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.580 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.580 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.580 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.581 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.581 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.581 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.581 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.581 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.582 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.582 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.582 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.582 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.582 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.583 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.583 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.583 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.583 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.583 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.584 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.584 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.584 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.584 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.584 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.585 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.585 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.585 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.585 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.585 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.586 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.586 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.586 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.586 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.586 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.586 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.587 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.587 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.587 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.587 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.588 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.588 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.588 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.588 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.588 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.589 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.589 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.589 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.589 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.589 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.590 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.590 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.590 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.590 INFO:teuthology.orchestra.run.smithi044.stdout: "predict_interval": { 2023-09-10T01:37:08.590 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predict_interval", 2023-09-10T01:37:08.591 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.591 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.591 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.591 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2023-09-10T01:37:08.591 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.592 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.592 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.592 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.592 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.592 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.593 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.593 INFO:teuthology.orchestra.run.smithi044.stdout: "predictor_model": { 2023-09-10T01:37:08.593 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predictor_model", 2023-09-10T01:37:08.593 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.594 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.594 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.594 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "prophetstor", 2023-09-10T01:37:08.594 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.594 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.594 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.595 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.595 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.595 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.595 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.596 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2023-09-10T01:37:08.596 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2023-09-10T01:37:08.596 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.596 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.596 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.596 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2023-09-10T01:37:08.597 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.597 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.597 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.597 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.597 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.598 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.598 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.598 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.598 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.599 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "influx", 2023-09-10T01:37:08.599 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": false, 2023-09-10T01:37:08.599 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "influxdb python module not found", 2023-09-10T01:37:08.599 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.599 INFO:teuthology.orchestra.run.smithi044.stdout: "batch_size": { 2023-09-10T01:37:08.600 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "batch_size", 2023-09-10T01:37:08.600 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.600 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.600 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.600 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5000", 2023-09-10T01:37:08.600 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.601 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.601 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.601 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-10T01:37:08.601 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.601 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.602 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.602 INFO:teuthology.orchestra.run.smithi044.stdout: "database": { 2023-09-10T01:37:08.602 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "database", 2023-09-10T01:37:08.602 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.603 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.603 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.603 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "ceph", 2023-09-10T01:37:08.603 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.603 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.604 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.604 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-10T01:37:08.604 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.604 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.604 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.605 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": { 2023-09-10T01:37:08.605 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hostname", 2023-09-10T01:37:08.605 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.605 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.605 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.606 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.606 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.606 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.606 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.606 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server hostname", 2023-09-10T01:37:08.607 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.607 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.607 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.607 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2023-09-10T01:37:08.608 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2023-09-10T01:37:08.608 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.608 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.608 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.608 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "30", 2023-09-10T01:37:08.608 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "5", 2023-09-10T01:37:08.609 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.609 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.609 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-10T01:37:08.609 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.609 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.610 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.610 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.610 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.610 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.611 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.611 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.611 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.611 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.611 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.611 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.612 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.612 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.612 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.612 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.612 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.613 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.613 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.613 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.613 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.613 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.614 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.614 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.614 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.614 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.614 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.615 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.615 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.615 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.615 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.615 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.616 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.616 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.616 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.616 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.616 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.617 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.617 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.617 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.617 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.617 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.618 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.618 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.618 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.618 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.618 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.619 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.619 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.619 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.619 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.619 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.619 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.620 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.620 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.620 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.620 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.620 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.621 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.621 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.621 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.621 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.621 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.622 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.622 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.622 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.622 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.622 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.622 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.623 INFO:teuthology.orchestra.run.smithi044.stdout: "password": { 2023-09-10T01:37:08.623 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "password", 2023-09-10T01:37:08.623 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.623 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.624 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.624 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.624 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.624 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.624 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.624 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "password of InfluxDB server user", 2023-09-10T01:37:08.625 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.625 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.625 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.625 INFO:teuthology.orchestra.run.smithi044.stdout: "port": { 2023-09-10T01:37:08.626 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "port", 2023-09-10T01:37:08.626 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.626 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.626 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.626 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8086", 2023-09-10T01:37:08.627 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.627 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.627 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.627 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server port", 2023-09-10T01:37:08.627 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.627 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.628 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.628 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2023-09-10T01:37:08.628 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2023-09-10T01:37:08.628 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.629 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.629 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.629 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "false", 2023-09-10T01:37:08.629 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.629 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.630 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.630 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-10T01:37:08.630 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.630 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.630 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.631 INFO:teuthology.orchestra.run.smithi044.stdout: "threads": { 2023-09-10T01:37:08.631 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threads", 2023-09-10T01:37:08.631 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.631 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.631 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.632 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2023-09-10T01:37:08.632 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2023-09-10T01:37:08.632 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "32", 2023-09-10T01:37:08.632 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.632 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-10T01:37:08.632 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.633 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.633 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.633 INFO:teuthology.orchestra.run.smithi044.stdout: "username": { 2023-09-10T01:37:08.633 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "username", 2023-09-10T01:37:08.634 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.634 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.634 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.634 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.634 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.635 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.635 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.635 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "username of InfluxDB server user", 2023-09-10T01:37:08.635 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.635 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.635 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.636 INFO:teuthology.orchestra.run.smithi044.stdout: "verify_ssl": { 2023-09-10T01:37:08.636 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "verify_ssl", 2023-09-10T01:37:08.636 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.636 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.637 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.637 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "true", 2023-09-10T01:37:08.637 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.637 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.637 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.637 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-10T01:37:08.638 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.638 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.638 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.638 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.638 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.639 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "insights", 2023-09-10T01:37:08.639 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.639 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.640 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.640 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.640 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.640 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.640 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.641 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.641 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.641 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.641 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.641 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.641 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.642 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.642 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.642 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.642 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.642 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.643 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.643 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.643 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.643 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.643 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.644 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.644 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.644 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.645 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.645 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.645 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.645 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.645 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.645 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.646 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.650 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.650 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.650 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.651 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.651 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.651 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.651 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.651 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.652 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.652 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.652 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.652 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.652 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.653 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.653 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.653 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.653 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.653 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.653 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.654 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.654 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.654 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.654 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.655 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.655 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.655 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.655 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.655 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.656 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.656 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.656 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.656 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.656 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.656 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.657 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.657 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.657 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.657 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.658 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "iostat", 2023-09-10T01:37:08.658 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.658 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.658 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.659 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.659 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.659 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.659 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.659 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.659 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.660 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.660 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.660 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.660 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.660 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.661 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.661 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.661 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.661 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.661 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.661 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.662 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.662 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.662 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.662 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.663 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.663 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.663 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.663 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.663 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.663 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.664 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.664 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.664 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.664 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.664 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.665 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.665 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.665 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.665 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.665 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.666 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.666 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.666 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.666 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.666 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.667 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.667 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.667 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.667 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.667 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.667 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.668 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.668 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.668 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.668 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.668 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.669 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.669 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.669 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.669 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.670 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.670 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.670 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.670 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.670 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.670 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.671 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.671 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.671 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.671 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.671 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.672 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "k8sevents", 2023-09-10T01:37:08.672 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.672 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.673 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.673 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_event_retention_days": { 2023-09-10T01:37:08.673 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ceph_event_retention_days", 2023-09-10T01:37:08.673 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.673 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.673 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.674 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7", 2023-09-10T01:37:08.674 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.674 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.674 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.674 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-10T01:37:08.675 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.675 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.675 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.675 INFO:teuthology.orchestra.run.smithi044.stdout: "config_check_secs": { 2023-09-10T01:37:08.676 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_check_secs", 2023-09-10T01:37:08.676 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.676 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.676 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.676 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:08.676 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "10", 2023-09-10T01:37:08.677 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.677 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.677 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-10T01:37:08.677 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.677 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.678 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.678 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.678 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.678 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.679 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.679 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.679 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.679 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.679 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.680 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.680 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.680 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.680 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.680 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.680 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.681 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.681 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.681 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.681 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.681 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.681 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.682 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.682 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.682 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.682 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.683 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.683 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.683 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.683 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.683 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.684 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.684 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.684 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.684 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.684 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.685 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.685 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.685 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.685 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.685 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.686 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.686 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.686 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.686 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.686 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.687 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.687 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.687 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.687 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.687 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.687 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.688 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.688 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.688 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.688 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.689 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.689 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.689 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.689 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.689 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.689 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.690 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.690 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.690 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.690 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.690 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.691 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.691 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.691 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.692 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "localpool", 2023-09-10T01:37:08.692 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.692 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.692 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.692 INFO:teuthology.orchestra.run.smithi044.stdout: "failure_domain": { 2023-09-10T01:37:08.692 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "failure_domain", 2023-09-10T01:37:08.693 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.693 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.693 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.693 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "host", 2023-09-10T01:37:08.693 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.694 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.694 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.694 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "failure domain for any created local pool", 2023-09-10T01:37:08.694 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-10T01:37:08.694 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.695 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.695 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.695 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.695 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.696 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.696 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.696 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.696 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.696 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.696 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.697 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.697 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.697 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.697 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.697 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.698 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.698 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.698 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.698 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.698 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.698 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.699 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.699 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.699 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.699 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.700 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.700 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.700 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.700 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.700 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.700 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.701 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.701 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.701 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.701 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.702 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.702 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.702 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.702 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.702 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.703 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.703 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.703 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.703 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.703 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.704 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.704 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.704 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.704 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.704 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.704 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.705 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.705 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.705 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.705 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.705 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.706 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.706 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.706 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.706 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.706 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.707 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.707 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.707 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.707 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.707 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.707 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.708 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": { 2023-09-10T01:37:08.708 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_size", 2023-09-10T01:37:08.708 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.708 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.708 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.709 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.709 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.709 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.709 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.709 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default min_size for any created local pool", 2023-09-10T01:37:08.710 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-10T01:37:08.710 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.710 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.710 INFO:teuthology.orchestra.run.smithi044.stdout: "num_rep": { 2023-09-10T01:37:08.711 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "num_rep", 2023-09-10T01:37:08.711 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.711 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.711 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.711 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3", 2023-09-10T01:37:08.711 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.712 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.712 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.712 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default replica count for any created local pool", 2023-09-10T01:37:08.712 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.712 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.713 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.713 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": { 2023-09-10T01:37:08.713 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_num", 2023-09-10T01:37:08.713 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.713 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.714 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.714 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "128", 2023-09-10T01:37:08.714 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.714 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.714 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.715 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default pg_num for any created local pool", 2023-09-10T01:37:08.715 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.715 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.715 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.715 INFO:teuthology.orchestra.run.smithi044.stdout: "prefix": { 2023-09-10T01:37:08.716 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prefix", 2023-09-10T01:37:08.716 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.716 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.716 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.716 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.717 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.717 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.717 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.717 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name prefix for any created local pool", 2023-09-10T01:37:08.717 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.718 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.718 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.718 INFO:teuthology.orchestra.run.smithi044.stdout: "subtree": { 2023-09-10T01:37:08.718 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subtree", 2023-09-10T01:37:08.719 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.719 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.719 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.719 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "rack", 2023-09-10T01:37:08.719 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.719 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.720 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.720 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-10T01:37:08.720 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-10T01:37:08.720 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.720 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.721 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.721 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.721 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mds_autoscaler", 2023-09-10T01:37:08.722 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.722 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.722 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.722 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.722 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.723 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.723 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.723 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.723 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.723 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.723 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.724 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.724 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.724 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.724 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.724 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.724 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.725 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.725 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.725 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.725 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.725 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.726 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.726 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.726 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.726 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.727 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.727 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.727 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.727 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.727 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.728 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.728 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.728 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.728 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.728 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.729 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.729 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.729 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.729 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.729 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.730 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.730 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.730 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.730 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.730 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.731 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.731 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.731 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.731 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.731 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.732 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.732 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.732 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.732 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.732 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.733 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.733 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.733 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.733 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.733 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.734 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.734 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.734 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.734 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.734 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.735 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.735 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.735 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.735 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.735 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.736 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirroring", 2023-09-10T01:37:08.736 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.736 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.736 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.737 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.737 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.737 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.737 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.737 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.737 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.738 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.738 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.738 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.738 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.738 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.739 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.739 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.739 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.739 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.739 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.740 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.740 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.740 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.740 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.741 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.741 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.741 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.741 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.741 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.741 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.742 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.742 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.742 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.742 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.742 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.743 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.743 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.743 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.743 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.744 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.744 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.744 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.744 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.744 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.745 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.745 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.745 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.745 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.745 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.745 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.746 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.746 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.746 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.746 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.746 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.747 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.747 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.747 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.747 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.747 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.748 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.748 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.748 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.748 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.748 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.749 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.749 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.749 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.749 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.749 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.750 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.750 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.750 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "nfs", 2023-09-10T01:37:08.750 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.751 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.751 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.751 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.751 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.751 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.752 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.752 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.752 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.752 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.752 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.753 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.753 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.753 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.753 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.753 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.753 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.754 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.754 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.754 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.754 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.754 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.755 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.755 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.755 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.755 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.755 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.756 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.756 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.756 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.756 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.756 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.757 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.757 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.757 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.757 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.757 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.758 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.758 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.758 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.758 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.758 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.759 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.759 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.759 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.759 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.759 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.760 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.760 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.760 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.760 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.760 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.760 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.761 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.761 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.761 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.761 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.762 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.762 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.762 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.762 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.762 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.763 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.763 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.763 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.763 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.763 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.764 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.764 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.764 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.764 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.765 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2023-09-10T01:37:08.765 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.765 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.765 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.765 INFO:teuthology.orchestra.run.smithi044.stdout: "fail_fs": { 2023-09-10T01:37:08.766 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "fail_fs", 2023-09-10T01:37:08.766 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.766 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.766 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.766 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.767 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.767 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.767 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.767 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-10T01:37:08.767 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.767 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.768 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.768 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.768 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.768 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.768 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.769 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.769 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.769 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.769 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.769 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.770 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.770 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.770 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.770 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.770 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.771 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.771 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.771 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.771 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.771 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.771 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.772 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.772 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.772 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.772 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.772 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.773 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.773 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.773 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.773 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.773 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.774 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.774 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.774 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.774 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.775 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.775 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.775 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.775 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.775 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.775 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.775 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.776 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.776 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.776 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.776 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.776 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.776 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.777 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.777 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.777 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.777 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.777 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.778 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.778 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.778 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.778 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.778 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.779 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.779 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.779 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.779 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.779 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.779 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.780 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.780 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.780 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.780 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator": { 2023-09-10T01:37:08.780 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2023-09-10T01:37:08.781 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.781 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.781 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.781 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.781 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.782 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.782 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.782 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm", 2023-09-10T01:37:08.782 INFO:teuthology.orchestra.run.smithi044.stdout: "rook", 2023-09-10T01:37:08.782 INFO:teuthology.orchestra.run.smithi044.stdout: "test_orchestrator" 2023-09-10T01:37:08.782 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.783 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Orchestrator backend", 2023-09-10T01:37:08.783 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.783 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.783 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.783 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.784 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.784 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_perf_query", 2023-09-10T01:37:08.784 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.784 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.785 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.785 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.785 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.785 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.785 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.786 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.786 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.786 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.786 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.786 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.786 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.787 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.787 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.787 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.787 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.787 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.788 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.788 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.788 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.788 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.788 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.788 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.789 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.789 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.789 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.789 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.789 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.790 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.790 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.790 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.790 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.790 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.790 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.791 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.791 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.791 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.791 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.792 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.792 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.792 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.792 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.792 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.792 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.793 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.793 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.793 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.793 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.793 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.794 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.794 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.794 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.794 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.794 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.794 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.795 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.795 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.795 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.795 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.796 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.796 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.796 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.796 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.796 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.796 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.797 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.797 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.797 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.797 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.797 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.798 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.798 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_support", 2023-09-10T01:37:08.798 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.798 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.799 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.799 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.799 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.799 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.799 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.799 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.800 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.800 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.800 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.800 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.800 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.801 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.801 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.801 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.801 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.801 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.802 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.802 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.802 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.802 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.802 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.803 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.803 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.803 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.803 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.803 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.804 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.804 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.804 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.804 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.805 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.805 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.805 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.805 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.806 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.806 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.806 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.806 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.807 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.807 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.807 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.807 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.808 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.808 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.808 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.808 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.808 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.809 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.809 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.809 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.809 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.809 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.810 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.810 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.811 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.811 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.811 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.811 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.811 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.811 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.812 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.812 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.812 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.812 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.812 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.813 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.813 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.813 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.813 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.814 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.814 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_autoscaler", 2023-09-10T01:37:08.814 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.814 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.814 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.814 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.815 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.815 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.815 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.815 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.815 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.816 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.816 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.816 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.816 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.816 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.817 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.817 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.817 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.817 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.817 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.817 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.817 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.818 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.818 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.818 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.818 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.819 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.819 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.819 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.819 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.819 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.819 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.820 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.820 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.820 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.820 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.820 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.821 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.821 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.821 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.821 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.821 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.822 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.822 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.822 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.822 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.822 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.822 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.823 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.823 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.823 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.823 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.823 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.823 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.824 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.824 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.824 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.824 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.824 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.825 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.825 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.825 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.825 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.825 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.825 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.826 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.826 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.826 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.826 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.826 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.827 INFO:teuthology.orchestra.run.smithi044.stdout: "noautoscale": { 2023-09-10T01:37:08.827 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "noautoscale", 2023-09-10T01:37:08.827 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.827 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.827 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.827 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.828 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.828 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.828 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.828 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "global autoscale flag", 2023-09-10T01:37:08.828 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-10T01:37:08.828 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.829 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.829 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2023-09-10T01:37:08.829 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2023-09-10T01:37:08.829 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.829 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.830 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.830 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2023-09-10T01:37:08.830 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.830 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.830 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.831 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.831 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.831 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.831 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.832 INFO:teuthology.orchestra.run.smithi044.stdout: "threshold": { 2023-09-10T01:37:08.832 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threshold", 2023-09-10T01:37:08.832 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:08.832 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.832 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.833 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2023-09-10T01:37:08.833 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1.0", 2023-09-10T01:37:08.833 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.833 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.833 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "scaling threshold", 2023-09-10T01:37:08.833 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-10T01:37:08.834 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.834 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.834 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.834 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.835 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "progress", 2023-09-10T01:37:08.835 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.835 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.835 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.836 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_pg_recovery_event": { 2023-09-10T01:37:08.836 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_pg_recovery_event", 2023-09-10T01:37:08.836 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.836 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.836 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.837 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.837 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.837 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.837 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.837 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-10T01:37:08.837 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.838 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.838 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.838 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2023-09-10T01:37:08.838 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2023-09-10T01:37:08.839 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.839 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.839 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.839 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.839 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.840 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.840 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.840 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.840 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.840 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.840 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.841 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.841 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.841 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.841 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.842 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.842 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.842 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.842 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.842 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.843 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.843 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.843 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.843 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.843 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.844 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.844 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.844 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.844 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.844 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.844 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.845 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.845 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.845 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.845 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.846 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.846 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.846 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.846 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.846 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.847 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.847 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.847 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.847 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.847 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.848 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.848 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.848 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.848 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.848 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.849 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.849 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.849 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.849 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.849 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.849 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.850 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.850 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.850 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.853 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.853 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.853 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.853 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.853 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.854 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.854 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.854 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.854 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.854 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.855 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.855 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.855 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.855 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.855 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.855 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.856 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.856 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.856 INFO:teuthology.orchestra.run.smithi044.stdout: "max_completed_events": { 2023-09-10T01:37:08.856 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_completed_events", 2023-09-10T01:37:08.856 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.857 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.857 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.857 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "50", 2023-09-10T01:37:08.857 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.857 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.858 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.858 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "number of past completed events to remember", 2023-09-10T01:37:08.858 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.858 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.858 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.859 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2023-09-10T01:37:08.859 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2023-09-10T01:37:08.859 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:08.859 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.859 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.860 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2023-09-10T01:37:08.860 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.860 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.860 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.860 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long the module is going to sleep", 2023-09-10T01:37:08.861 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.861 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.861 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.861 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.861 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.862 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus", 2023-09-10T01:37:08.862 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.862 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.862 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.863 INFO:teuthology.orchestra.run.smithi044.stdout: "cache": { 2023-09-10T01:37:08.863 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cache", 2023-09-10T01:37:08.863 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.863 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.863 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.864 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.864 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.864 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.864 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.864 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.864 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.864 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.865 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.865 INFO:teuthology.orchestra.run.smithi044.stdout: "exclude_perf_counters": { 2023-09-10T01:37:08.865 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "exclude_perf_counters", 2023-09-10T01:37:08.865 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.866 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.866 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.866 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.866 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.866 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.866 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.867 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-10T01:37:08.867 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-10T01:37:08.867 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.867 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.868 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.868 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.868 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.868 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.868 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.869 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.869 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.869 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.869 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.869 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.869 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.870 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.870 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.870 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.870 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.870 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.871 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.871 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.871 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.871 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.871 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.872 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.872 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.872 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.872 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.872 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.872 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.873 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.873 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.873 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.873 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.873 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.874 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.874 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.874 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.874 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.875 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.875 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.875 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.875 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.875 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.875 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.876 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.876 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.876 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.876 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.876 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.877 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.877 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.877 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.877 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.877 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.878 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.878 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.878 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.878 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.879 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.879 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.879 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.879 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.879 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.879 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.880 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.880 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.880 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.880 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.881 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools": { 2023-09-10T01:37:08.881 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools", 2023-09-10T01:37:08.881 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.881 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.881 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.881 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.882 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.882 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.882 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.882 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.882 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.883 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.883 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.883 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-10T01:37:08.883 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-10T01:37:08.883 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.884 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.884 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.884 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "300", 2023-09-10T01:37:08.884 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.884 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.885 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.885 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.885 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.885 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.885 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.886 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_interval": { 2023-09-10T01:37:08.886 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_interval", 2023-09-10T01:37:08.886 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:08.886 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.886 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.887 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15.0", 2023-09-10T01:37:08.887 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.887 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.887 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.887 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.887 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.888 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.888 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.888 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2023-09-10T01:37:08.888 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2023-09-10T01:37:08.889 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.889 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.889 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.889 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2023-09-10T01:37:08.889 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.890 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.890 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.890 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-10T01:37:08.890 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.890 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.891 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.891 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2023-09-10T01:37:08.891 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2023-09-10T01:37:08.891 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.891 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.892 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.892 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "9283", 2023-09-10T01:37:08.892 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.892 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.892 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.893 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-10T01:37:08.893 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.893 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.893 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.894 INFO:teuthology.orchestra.run.smithi044.stdout: "stale_cache_strategy": { 2023-09-10T01:37:08.894 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stale_cache_strategy", 2023-09-10T01:37:08.894 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.894 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.894 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.895 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "log", 2023-09-10T01:37:08.895 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.895 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.895 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.895 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.895 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.896 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.896 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.896 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2023-09-10T01:37:08.896 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2023-09-10T01:37:08.897 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.897 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.897 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.897 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "default", 2023-09-10T01:37:08.897 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.897 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.898 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.898 INFO:teuthology.orchestra.run.smithi044.stdout: "default", 2023-09-10T01:37:08.898 INFO:teuthology.orchestra.run.smithi044.stdout: "error" 2023-09-10T01:37:08.898 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.898 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.899 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.899 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.899 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.899 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2023-09-10T01:37:08.900 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2023-09-10T01:37:08.900 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.900 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.900 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.900 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2023-09-10T01:37:08.901 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2023-09-10T01:37:08.901 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2023-09-10T01:37:08.901 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.901 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.901 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.901 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.902 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.902 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.902 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.903 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_support", 2023-09-10T01:37:08.903 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.903 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.903 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.903 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.903 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.904 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.904 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.904 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.904 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.904 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.905 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.905 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.905 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.905 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.905 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.905 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.906 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.906 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.906 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.906 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.906 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.907 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.907 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.907 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.907 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.908 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.908 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.908 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.908 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.908 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.908 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.909 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.909 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.909 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.909 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.909 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.910 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.910 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.910 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.910 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.910 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.911 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.911 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.911 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.911 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.911 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.912 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.912 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.912 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.912 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.912 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.912 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.913 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.913 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.913 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.913 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.914 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.914 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.914 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.914 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.914 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.915 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.915 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.915 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.915 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.915 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_snap_create": { 2023-09-10T01:37:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_snap_create", 2023-09-10T01:37:08.917 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:08.917 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.917 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.917 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2023-09-10T01:37:08.917 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.918 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.918 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.918 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.918 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.918 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.919 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.919 INFO:teuthology.orchestra.run.smithi044.stdout: "mirror_snapshot_schedule": { 2023-09-10T01:37:08.919 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirror_snapshot_schedule", 2023-09-10T01:37:08.919 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.920 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.920 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.920 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.920 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.920 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.921 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.921 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.921 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.921 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.921 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.922 INFO:teuthology.orchestra.run.smithi044.stdout: "trash_purge_schedule": { 2023-09-10T01:37:08.922 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "trash_purge_schedule", 2023-09-10T01:37:08.922 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.922 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.923 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.923 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.923 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.923 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.923 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.923 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.924 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.924 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.924 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.924 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.924 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.925 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "restful", 2023-09-10T01:37:08.925 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.925 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.925 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.926 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_auth": { 2023-09-10T01:37:08.926 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_auth", 2023-09-10T01:37:08.926 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.926 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.926 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.927 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:08.927 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.927 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.927 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.927 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.928 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.928 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.928 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.928 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2023-09-10T01:37:08.928 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2023-09-10T01:37:08.929 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.929 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.929 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.929 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.929 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.930 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.930 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.930 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.930 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.930 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.930 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.931 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.931 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.931 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.931 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.932 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.932 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.932 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.932 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.932 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.933 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.933 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.933 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.933 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.933 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.934 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.934 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.934 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.934 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.934 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.934 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.935 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.935 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.935 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.935 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.936 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.936 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.936 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.936 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.936 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.936 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.937 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.937 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.937 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.937 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.938 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.938 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.938 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.938 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.938 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.939 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.939 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.939 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.939 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.939 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.939 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.940 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.940 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.940 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.940 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.940 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.941 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.941 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.941 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.941 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.941 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.942 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.942 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.942 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.942 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.942 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.943 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.943 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.943 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.943 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.943 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.944 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.944 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2023-09-10T01:37:08.944 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2023-09-10T01:37:08.944 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.945 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.945 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.945 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.945 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.945 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.945 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.946 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.946 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.946 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.946 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.947 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2023-09-10T01:37:08.947 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2023-09-10T01:37:08.947 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.947 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.947 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.948 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.948 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.948 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.948 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.948 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.949 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.949 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.949 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.949 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.949 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.950 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rgw", 2023-09-10T01:37:08.950 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.950 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.950 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.951 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.951 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.951 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.951 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.951 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.952 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.952 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.952 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.952 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.952 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.952 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.953 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.953 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.953 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.953 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.953 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.954 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.954 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.954 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.954 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.955 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.955 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.955 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.955 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.955 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.956 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.956 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.956 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.956 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.956 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.957 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.957 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.957 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.957 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.957 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.958 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.958 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.958 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.958 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.958 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.959 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.959 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.959 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.959 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.959 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.960 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.960 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.960 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.960 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.960 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.960 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.961 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.961 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.961 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.961 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.962 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.962 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.962 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.962 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.962 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.962 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.963 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.963 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.963 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.963 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.963 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.964 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.964 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.964 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rook", 2023-09-10T01:37:08.964 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.965 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.965 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.965 INFO:teuthology.orchestra.run.smithi044.stdout: "drive_group_interval": { 2023-09-10T01:37:08.965 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "drive_group_interval", 2023-09-10T01:37:08.965 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:08.966 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.966 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.966 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "300.0", 2023-09-10T01:37:08.966 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.966 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.967 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.967 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-10T01:37:08.967 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.967 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.967 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.968 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.968 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.968 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.968 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.969 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.969 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.969 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.969 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.969 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.969 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.970 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.970 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.970 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.970 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.970 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.971 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.971 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.971 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.971 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.971 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.972 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.972 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.972 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.972 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.972 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.973 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.973 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.973 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.973 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.973 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.973 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.974 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.974 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.974 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.974 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.975 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.975 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.975 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.975 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.975 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.975 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.976 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.976 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.976 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.976 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.976 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.976 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.977 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.977 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.977 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.977 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.977 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.978 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.978 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.978 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.978 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.978 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.979 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.979 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.979 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.979 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.979 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.979 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.980 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.980 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.980 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.980 INFO:teuthology.orchestra.run.smithi044.stdout: "storage_class": { 2023-09-10T01:37:08.981 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "storage_class", 2023-09-10T01:37:08.981 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.981 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.981 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.981 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "local", 2023-09-10T01:37:08.982 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.982 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.982 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.982 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-10T01:37:08.982 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.983 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.983 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.983 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.983 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:08.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:08.984 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "selftest", 2023-09-10T01:37:08.984 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:08.984 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:08.984 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:08.984 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:08.985 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:08.985 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.985 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.985 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.985 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.985 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.986 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.986 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.986 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.986 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.986 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.987 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.987 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.987 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.987 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.987 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.987 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.988 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.988 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.988 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:08.988 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:08.988 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.989 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.989 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.989 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.989 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.989 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.989 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.990 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.990 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.990 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.990 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.991 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:08.991 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:08.991 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.991 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.991 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.991 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:08.992 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.992 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.992 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:08.992 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:08.992 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:08.993 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:08.993 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:08.993 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:08.993 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:08.993 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:08.994 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.994 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.994 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.994 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.995 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:08.995 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:08.995 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:08.995 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.995 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:08.995 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:08.996 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.996 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.996 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.996 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.996 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.997 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.997 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:08.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:08.997 INFO:teuthology.orchestra.run.smithi044.stdout: "roption1": { 2023-09-10T01:37:08.997 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption1", 2023-09-10T01:37:08.997 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:08.998 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:08.998 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:08.998 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:08.998 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:08.998 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:08.999 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:08.999 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:08.999 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:08.999 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:08.999 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.000 INFO:teuthology.orchestra.run.smithi044.stdout: "roption2": { 2023-09-10T01:37:09.000 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption2", 2023-09-10T01:37:09.000 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.000 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.000 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.001 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "xyz", 2023-09-10T01:37:09.001 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.001 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.001 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.001 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.002 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.002 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.002 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.002 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption1": { 2023-09-10T01:37:09.003 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption1", 2023-09-10T01:37:09.003 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.003 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.003 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.003 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.003 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.004 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.004 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.004 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.004 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.004 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.005 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.005 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption2": { 2023-09-10T01:37:09.005 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption2", 2023-09-10T01:37:09.005 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:09.006 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.006 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.006 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.006 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.006 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.007 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.007 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.007 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.007 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.007 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.008 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption3": { 2023-09-10T01:37:09.008 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption3", 2023-09-10T01:37:09.008 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2023-09-10T01:37:09.008 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.008 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.008 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.009 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.009 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.009 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.009 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.009 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.010 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.010 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.010 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption4": { 2023-09-10T01:37:09.010 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption4", 2023-09-10T01:37:09.011 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.011 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.011 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.011 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.011 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.012 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.012 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.012 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.012 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.012 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.012 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.013 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption5": { 2023-09-10T01:37:09.013 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption5", 2023-09-10T01:37:09.013 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.013 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.014 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.014 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.014 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.014 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.014 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.014 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.015 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.015 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.015 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.015 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption6": { 2023-09-10T01:37:09.016 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption6", 2023-09-10T01:37:09.016 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.016 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.016 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.016 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:09.016 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.017 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.017 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.017 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.017 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.017 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.018 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.018 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption7": { 2023-09-10T01:37:09.018 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption7", 2023-09-10T01:37:09.018 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:09.019 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.019 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.019 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.019 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2023-09-10T01:37:09.019 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "42", 2023-09-10T01:37:09.019 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.020 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.020 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.020 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.020 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.021 INFO:teuthology.orchestra.run.smithi044.stdout: "testkey": { 2023-09-10T01:37:09.021 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testkey", 2023-09-10T01:37:09.021 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.021 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.021 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.021 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.022 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.022 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.022 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.022 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.022 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.023 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.023 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.023 INFO:teuthology.orchestra.run.smithi044.stdout: "testlkey": { 2023-09-10T01:37:09.023 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testlkey", 2023-09-10T01:37:09.023 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.024 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.024 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.024 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.024 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.024 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.025 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.025 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.025 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.025 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.025 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.026 INFO:teuthology.orchestra.run.smithi044.stdout: "testnewline": { 2023-09-10T01:37:09.026 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testnewline", 2023-09-10T01:37:09.026 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.026 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.026 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.027 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.027 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.027 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.027 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.027 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.027 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.028 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.028 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.028 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.028 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.029 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2023-09-10T01:37:09.029 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:09.029 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:09.029 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:09.029 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_m_granularity": { 2023-09-10T01:37:09.030 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_m_granularity", 2023-09-10T01:37:09.030 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.030 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.030 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.030 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.031 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.031 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.031 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.031 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow minute scheduled snapshots", 2023-09-10T01:37:09.031 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.032 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.032 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.032 INFO:teuthology.orchestra.run.smithi044.stdout: "dump_on_update": { 2023-09-10T01:37:09.032 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dump_on_update", 2023-09-10T01:37:09.032 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.033 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.033 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.033 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.033 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.033 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.034 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.034 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "dump database to debug log on update", 2023-09-10T01:37:09.034 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.034 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.034 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.035 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:09.035 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:09.035 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.035 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.035 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.036 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.036 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.036 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.036 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:09.036 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:09.036 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:09.037 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:09.037 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:09.037 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:09.037 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:09.037 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.037 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.038 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.038 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.038 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.038 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:09.039 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:09.039 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.039 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.039 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.039 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.039 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.040 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.040 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.040 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.040 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.040 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.040 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.041 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:09.041 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:09.041 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.041 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.041 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.042 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:09.042 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.042 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.042 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:09.042 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:09.043 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:09.043 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:09.043 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:09.043 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:09.043 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:09.043 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.044 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.044 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.044 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.044 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.044 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.045 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.046 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.047 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stats", 2023-09-10T01:37:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:09.048 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.049 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:09.050 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.051 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.052 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:09.052 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:09.052 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.052 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.053 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.053 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.053 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.053 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.053 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.053 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.054 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.054 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.054 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.054 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:09.055 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:09.055 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.055 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.057 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.057 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:09.057 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.057 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.058 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:09.058 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:09.058 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:09.058 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:09.058 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:09.059 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:09.059 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:09.059 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.059 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.059 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.059 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.060 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.060 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:09.060 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:09.060 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.061 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.061 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.061 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.061 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.061 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.061 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.062 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.062 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.062 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.062 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.062 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.063 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.063 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "status", 2023-09-10T01:37:09.063 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:09.064 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:09.064 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:09.064 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:09.064 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:09.064 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.065 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.065 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.065 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.065 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.065 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.065 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:09.066 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:09.066 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:09.066 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:09.066 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:09.066 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:09.067 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:09.067 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.067 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.067 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.067 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.068 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.068 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:09.068 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:09.068 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.069 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.069 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.069 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.069 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.069 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.069 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.070 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.070 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.070 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.070 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.071 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:09.071 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:09.071 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.071 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.071 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.072 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:09.072 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.072 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.072 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:09.072 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:09.073 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:09.073 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:09.073 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:09.073 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:09.073 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:09.074 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.074 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.074 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.074 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.074 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.075 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:09.075 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:09.075 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.075 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.075 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.076 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.076 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.076 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.076 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.076 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.077 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.077 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.077 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.077 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.077 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.078 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telegraf", 2023-09-10T01:37:09.078 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:09.078 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:09.078 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:09.079 INFO:teuthology.orchestra.run.smithi044.stdout: "address": { 2023-09-10T01:37:09.079 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "address", 2023-09-10T01:37:09.079 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.079 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.079 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.080 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-10T01:37:09.080 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.080 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.080 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.080 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.080 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.081 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.081 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.081 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2023-09-10T01:37:09.081 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2023-09-10T01:37:09.082 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:09.082 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.082 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.082 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15", 2023-09-10T01:37:09.082 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.083 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.083 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.083 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.083 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.083 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.083 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.084 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:09.084 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:09.084 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.084 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.085 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.085 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.085 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.085 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.085 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:09.085 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:09.086 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:09.086 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:09.086 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:09.086 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:09.086 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:09.087 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.087 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.087 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.087 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.087 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.088 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:09.088 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:09.088 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.088 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.088 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.089 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.089 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.089 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.089 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.089 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.089 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.090 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.090 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.090 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:09.090 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:09.091 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.091 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.091 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.091 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:09.091 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.092 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.092 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:09.092 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:09.092 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:09.092 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:09.093 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:09.093 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:09.093 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:09.093 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.093 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.093 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.094 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.094 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.094 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:09.094 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:09.095 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.095 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.095 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.095 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.095 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.096 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.096 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.096 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.096 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.096 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.096 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.096 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.097 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.097 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telemetry", 2023-09-10T01:37:09.097 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:09.097 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:09.098 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:09.098 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_basic": { 2023-09-10T01:37:09.098 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_basic", 2023-09-10T01:37:09.098 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.098 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.099 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.099 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:09.099 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.099 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.099 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.099 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-10T01:37:09.100 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.100 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.100 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.100 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_crash": { 2023-09-10T01:37:09.100 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_crash", 2023-09-10T01:37:09.101 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.101 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.101 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.101 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:09.101 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.101 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.102 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.102 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-10T01:37:09.102 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.102 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.102 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.103 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_device": { 2023-09-10T01:37:09.103 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_device", 2023-09-10T01:37:09.103 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.103 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.103 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.103 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2023-09-10T01:37:09.104 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.104 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.104 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.104 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-10T01:37:09.104 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.104 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.105 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.105 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_ident": { 2023-09-10T01:37:09.105 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_ident", 2023-09-10T01:37:09.105 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.105 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.106 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.106 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.106 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.106 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.106 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.106 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-10T01:37:09.107 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.107 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.107 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.107 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_perf": { 2023-09-10T01:37:09.107 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_perf", 2023-09-10T01:37:09.108 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.108 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.108 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.108 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.108 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.108 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.109 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.109 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-10T01:37:09.109 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.109 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.109 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.110 INFO:teuthology.orchestra.run.smithi044.stdout: "contact": { 2023-09-10T01:37:09.110 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "contact", 2023-09-10T01:37:09.110 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.110 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.110 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.110 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.111 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.111 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.111 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.111 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.111 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.112 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.112 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.112 INFO:teuthology.orchestra.run.smithi044.stdout: "description": { 2023-09-10T01:37:09.112 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "description", 2023-09-10T01:37:09.112 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.112 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.113 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.113 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.113 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.113 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.113 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.114 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.114 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.114 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.114 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.115 INFO:teuthology.orchestra.run.smithi044.stdout: "device_url": { 2023-09-10T01:37:09.115 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_url", 2023-09-10T01:37:09.115 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.115 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.115 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.115 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-10T01:37:09.115 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.116 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.116 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.116 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.116 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.116 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.116 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.117 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2023-09-10T01:37:09.117 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2023-09-10T01:37:09.117 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.117 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.117 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.118 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.118 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.118 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.118 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.118 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.119 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.119 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.119 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.119 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2023-09-10T01:37:09.119 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2023-09-10T01:37:09.120 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:09.120 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.120 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.120 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "24", 2023-09-10T01:37:09.120 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "8", 2023-09-10T01:37:09.121 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.121 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.121 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.121 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.121 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.121 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.122 INFO:teuthology.orchestra.run.smithi044.stdout: "last_opt_revision": { 2023-09-10T01:37:09.122 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "last_opt_revision", 2023-09-10T01:37:09.122 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:09.122 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.123 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.123 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1", 2023-09-10T01:37:09.123 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.123 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.123 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.124 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.124 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.124 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.124 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.124 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard": { 2023-09-10T01:37:09.125 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard", 2023-09-10T01:37:09.125 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.125 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.125 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.125 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.126 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.126 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.126 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.126 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.126 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.126 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.127 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.127 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard_description": { 2023-09-10T01:37:09.127 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard_description", 2023-09-10T01:37:09.127 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.128 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.128 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.128 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.128 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.128 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.128 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.129 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.129 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.129 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.129 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.129 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:09.130 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:09.130 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.130 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.130 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.130 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.130 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.131 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.131 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:09.131 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:09.131 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:09.131 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:09.131 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:09.132 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:09.132 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:09.132 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.132 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.132 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.132 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.133 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.133 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:09.133 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:09.133 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.133 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.134 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.134 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.134 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.134 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.134 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.135 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.135 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.135 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.135 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.135 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:09.136 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:09.136 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.136 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.136 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.136 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:09.136 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.137 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.137 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:09.137 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:09.137 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:09.137 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:09.137 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:09.138 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:09.138 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:09.138 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.138 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.138 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.139 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.139 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.139 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:09.139 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:09.139 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.140 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.140 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.140 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.140 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.140 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.141 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.141 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.141 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.141 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.141 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.142 INFO:teuthology.orchestra.run.smithi044.stdout: "organization": { 2023-09-10T01:37:09.142 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "organization", 2023-09-10T01:37:09.142 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.142 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.143 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.143 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.143 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.143 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.143 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.143 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.144 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.144 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.144 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.144 INFO:teuthology.orchestra.run.smithi044.stdout: "proxy": { 2023-09-10T01:37:09.145 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "proxy", 2023-09-10T01:37:09.145 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.145 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.145 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.145 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.146 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.146 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.146 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.146 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.146 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.146 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.147 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.147 INFO:teuthology.orchestra.run.smithi044.stdout: "url": { 2023-09-10T01:37:09.147 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url", 2023-09-10T01:37:09.147 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.148 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.148 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.148 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-10T01:37:09.148 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.148 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.149 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.149 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.149 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.149 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.149 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.149 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.150 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.150 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "test_orchestrator", 2023-09-10T01:37:09.150 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:09.151 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:09.151 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:09.151 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:09.151 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:09.151 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.151 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.152 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.152 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.152 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.152 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.152 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:09.153 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:09.153 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:09.153 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:09.153 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:09.153 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:09.153 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:09.154 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.154 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.154 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.154 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.154 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.155 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:09.155 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:09.155 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.155 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.155 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.156 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.156 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.156 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.156 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.156 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.156 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.157 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.157 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.157 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:09.157 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:09.157 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.158 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.158 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.158 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:09.158 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.158 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.159 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:09.159 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:09.159 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:09.159 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:09.159 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:09.160 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:09.160 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:09.160 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.160 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.160 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.160 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.161 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.161 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:09.161 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:09.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.162 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.162 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.162 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.162 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.162 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.163 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.163 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.163 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.163 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.163 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.163 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.164 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.164 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "volumes", 2023-09-10T01:37:09.164 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:09.165 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:09.165 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:09.165 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:09.165 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:09.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.165 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.166 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.166 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.166 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.166 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.166 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:09.166 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:09.167 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:09.167 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:09.167 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:09.167 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:09.167 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:09.168 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.168 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.168 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.168 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.168 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.169 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:09.169 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:09.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.169 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.169 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.170 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.170 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.170 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.170 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.170 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.171 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.171 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.171 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.171 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:09.171 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:09.172 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.172 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.172 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.172 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:09.172 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.173 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.173 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:09.173 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:09.173 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:09.173 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:09.174 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:09.174 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:09.174 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:09.174 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.174 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.174 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.175 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.175 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.175 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:09.175 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:09.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.176 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.176 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.176 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.176 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.176 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.177 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.177 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.177 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.177 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.177 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.178 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_clones": { 2023-09-10T01:37:09.178 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_clones", 2023-09-10T01:37:09.178 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:09.178 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.178 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.179 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4", 2023-09-10T01:37:09.179 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.179 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.179 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.179 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-10T01:37:09.180 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.180 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.180 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.180 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_delay": { 2023-09-10T01:37:09.181 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_delay", 2023-09-10T01:37:09.181 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:09.181 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.181 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.181 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2023-09-10T01:37:09.182 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.182 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.182 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.182 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-10T01:37:09.182 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.183 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.183 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.183 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.183 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.184 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix", 2023-09-10T01:37:09.184 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2023-09-10T01:37:09.184 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2023-09-10T01:37:09.184 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2023-09-10T01:37:09.184 INFO:teuthology.orchestra.run.smithi044.stdout: "discovery_interval": { 2023-09-10T01:37:09.185 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "discovery_interval", 2023-09-10T01:37:09.185 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2023-09-10T01:37:09.185 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.185 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.185 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "100", 2023-09-10T01:37:09.186 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.186 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.186 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.186 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.186 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.187 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.187 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.187 INFO:teuthology.orchestra.run.smithi044.stdout: "identifier": { 2023-09-10T01:37:09.187 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "identifier", 2023-09-10T01:37:09.188 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.188 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.188 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.188 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.188 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.188 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.189 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.189 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.189 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.189 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.189 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.190 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2023-09-10T01:37:09.190 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2023-09-10T01:37:09.190 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2023-09-10T01:37:09.190 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.191 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.191 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2023-09-10T01:37:09.191 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.191 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.191 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.191 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.192 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.192 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.192 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.192 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2023-09-10T01:37:09.193 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2023-09-10T01:37:09.193 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.193 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.193 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.193 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.193 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.194 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.194 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:09.194 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:09.194 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:09.194 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:09.195 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:09.195 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:09.195 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:09.195 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.195 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.196 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.196 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.196 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.196 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2023-09-10T01:37:09.196 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2023-09-10T01:37:09.197 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.197 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.197 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.197 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.197 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.198 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.198 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.198 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.198 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.198 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.198 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.199 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2023-09-10T01:37:09.199 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2023-09-10T01:37:09.199 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.199 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.200 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.200 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2023-09-10T01:37:09.200 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.200 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.200 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2023-09-10T01:37:09.201 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2023-09-10T01:37:09.201 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2023-09-10T01:37:09.201 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2023-09-10T01:37:09.201 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2023-09-10T01:37:09.201 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2023-09-10T01:37:09.201 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2023-09-10T01:37:09.202 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.202 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.202 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.202 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.202 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.203 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2023-09-10T01:37:09.203 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2023-09-10T01:37:09.203 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2023-09-10T01:37:09.203 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.204 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2023-09-10T01:37:09.204 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2023-09-10T01:37:09.204 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.204 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.204 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.205 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.205 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.205 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.205 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.206 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_host": { 2023-09-10T01:37:09.206 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_host", 2023-09-10T01:37:09.206 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.206 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.206 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.207 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2023-09-10T01:37:09.207 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.207 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.207 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.207 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.208 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.208 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.208 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.208 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_port": { 2023-09-10T01:37:09.208 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_port", 2023-09-10T01:37:09.209 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2023-09-10T01:37:09.209 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.209 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.209 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10051", 2023-09-10T01:37:09.209 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.209 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.210 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.210 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.210 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.210 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.210 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.211 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_sender": { 2023-09-10T01:37:09.211 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_sender", 2023-09-10T01:37:09.211 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2023-09-10T01:37:09.211 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2023-09-10T01:37:09.211 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2023-09-10T01:37:09.212 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-10T01:37:09.212 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2023-09-10T01:37:09.212 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2023-09-10T01:37:09.212 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2023-09-10T01:37:09.212 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2023-09-10T01:37:09.213 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2023-09-10T01:37:09.213 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2023-09-10T01:37:09.213 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2023-09-10T01:37:09.213 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.213 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.213 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.214 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.214 INFO:teuthology.orchestra.run.smithi044.stdout: "services": { 2023-09-10T01:37:09.214 INFO:teuthology.orchestra.run.smithi044.stdout: "dashboard": "https://172.21.15.44:8443/" 2023-09-10T01:37:09.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.214 INFO:teuthology.orchestra.run.smithi044.stdout: "always_on_modules": { 2023-09-10T01:37:09.215 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus": [ 2023-09-10T01:37:09.215 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2023-09-10T01:37:09.215 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2023-09-10T01:37:09.215 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2023-09-10T01:37:09.215 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2023-09-10T01:37:09.215 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2023-09-10T01:37:09.216 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2023-09-10T01:37:09.216 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2023-09-10T01:37:09.216 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2023-09-10T01:37:09.216 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2023-09-10T01:37:09.216 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2023-09-10T01:37:09.217 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.217 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific": [ 2023-09-10T01:37:09.217 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2023-09-10T01:37:09.217 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2023-09-10T01:37:09.217 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2023-09-10T01:37:09.217 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2023-09-10T01:37:09.218 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2023-09-10T01:37:09.218 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2023-09-10T01:37:09.218 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2023-09-10T01:37:09.218 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2023-09-10T01:37:09.218 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2023-09-10T01:37:09.219 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2023-09-10T01:37:09.219 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.219 INFO:teuthology.orchestra.run.smithi044.stdout: "quincy": [ 2023-09-10T01:37:09.219 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2023-09-10T01:37:09.219 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2023-09-10T01:37:09.220 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2023-09-10T01:37:09.220 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2023-09-10T01:37:09.220 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2023-09-10T01:37:09.220 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2023-09-10T01:37:09.220 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2023-09-10T01:37:09.221 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2023-09-10T01:37:09.221 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2023-09-10T01:37:09.221 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2023-09-10T01:37:09.221 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2023-09-10T01:37:09.221 INFO:teuthology.orchestra.run.smithi044.stdout: "reef": [ 2023-09-10T01:37:09.222 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2023-09-10T01:37:09.222 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2023-09-10T01:37:09.222 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2023-09-10T01:37:09.222 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2023-09-10T01:37:09.222 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2023-09-10T01:37:09.223 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2023-09-10T01:37:09.223 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2023-09-10T01:37:09.223 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2023-09-10T01:37:09.223 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2023-09-10T01:37:09.223 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2023-09-10T01:37:09.224 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2023-09-10T01:37:09.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.224 INFO:teuthology.orchestra.run.smithi044.stdout: "last_failure_osd_epoch": 92, 2023-09-10T01:37:09.224 INFO:teuthology.orchestra.run.smithi044.stdout: "active_clients": [ 2023-09-10T01:37:09.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.224 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "devicehealth", 2023-09-10T01:37:09.225 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2023-09-10T01:37:09.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.225 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2023-09-10T01:37:09.225 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2023-09-10T01:37:09.225 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3071429327 2023-09-10T01:37:09.226 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.226 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2023-09-10T01:37:09.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.226 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "libcephsqlite", 2023-09-10T01:37:09.227 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2023-09-10T01:37:09.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.227 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2023-09-10T01:37:09.227 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2023-09-10T01:37:09.227 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 316201714 2023-09-10T01:37:09.228 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.228 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2023-09-10T01:37:09.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.228 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_support", 2023-09-10T01:37:09.228 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2023-09-10T01:37:09.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.229 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2023-09-10T01:37:09.229 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2023-09-10T01:37:09.229 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 466568886 2023-09-10T01:37:09.229 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.230 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2023-09-10T01:37:09.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.230 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2023-09-10T01:37:09.230 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2023-09-10T01:37:09.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.231 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2023-09-10T01:37:09.231 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2023-09-10T01:37:09.231 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 710405696 2023-09-10T01:37:09.231 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.232 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2023-09-10T01:37:09.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.232 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2023-09-10T01:37:09.232 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2023-09-10T01:37:09.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.233 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2023-09-10T01:37:09.233 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2023-09-10T01:37:09.233 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 4056256473 2023-09-10T01:37:09.233 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.233 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2023-09-10T01:37:09.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:09.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.234 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "volumes", 2023-09-10T01:37:09.234 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2023-09-10T01:37:09.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2023-09-10T01:37:09.234 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2023-09-10T01:37:09.235 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2023-09-10T01:37:09.235 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2081789656 2023-09-10T01:37:09.235 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.235 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2023-09-10T01:37:09.235 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:09.236 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2023-09-10T01:37:09.236 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T01:37:09.237 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:08 smithi099 ceph-mon[102692]: pgmap v1841: 129 pgs: 13 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 87 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 258 KiB/s rd, 283 KiB/s wr, 56 op/s 2023-09-10T01:37:09.237 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:08 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.814+0000 7f46aeb86700 1 -- 172.21.15.44:0/303873822 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f46900a0990 msgr2=0x7f46900a2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.815+0000 7f46aeb86700 1 --2- 172.21.15.44:0/303873822 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f46900a0990 0x7f46900a2e50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4698009810 tx=0x7f46980058e0 comp rx=0 tx=0).stop 2023-09-10T01:37:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.815+0000 7f46aeb86700 1 -- 172.21.15.44:0/303873822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a81081b0 msgr2=0x7f46a8117600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.815+0000 7f46aeb86700 1 --2- 172.21.15.44:0/303873822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a81081b0 0x7f46a8117600 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f46a000ba50 tx=0x7f46a000be10 comp rx=0 tx=0).stop 2023-09-10T01:37:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.821+0000 7f46aeb86700 1 -- 172.21.15.44:0/303873822 shutdown_connections 2023-09-10T01:37:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.821+0000 7f46aeb86700 1 --2- 172.21.15.44:0/303873822 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f46900a0990 0x7f46900a2e50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.821+0000 7f46aeb86700 1 --2- 172.21.15.44:0/303873822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46a810f160 0x7f46a81a8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:09.239 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.821+0000 7f46aeb86700 1 --2- 172.21.15.44:0/303873822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a81081b0 0x7f46a8117600 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:09.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.821+0000 7f46aeb86700 1 --2- 172.21.15.44:0/303873822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46a810ed80 0x7f46a8117b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:09.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.821+0000 7f46aeb86700 1 -- 172.21.15.44:0/303873822 >> 172.21.15.44:0/303873822 conn(0x7f46a806b640 msgr2=0x7f46a806f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:09.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.830+0000 7f46aeb86700 1 -- 172.21.15.44:0/303873822 shutdown_connections 2023-09-10T01:37:09.240 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:05.831+0000 7f46aeb86700 1 -- 172.21.15.44:0/303873822 wait complete. 2023-09-10T01:37:09.246 DEBUG:tasks.check_counter:Getting stats from a 2023-09-10T01:37:09.247 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.a perf dump 2023-09-10T01:37:09.247 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:05 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:09.248 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:05 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/303873822' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-10T01:37:09.248 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:06 smithi044 ceph-mon[98676]: pgmap v1840: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 258 KiB/s rd, 270 KiB/s wr, 44 op/s 2023-09-10T01:37:09.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:08 smithi131 ceph-mon[103713]: pgmap v1841: 129 pgs: 13 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 87 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 258 KiB/s rd, 283 KiB/s wr, 56 op/s 2023-09-10T01:37:09.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:08 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:09.323 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:08 smithi044 ceph-mon[98676]: pgmap v1841: 129 pgs: 13 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 87 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 258 KiB/s rd, 283 KiB/s wr, 56 op/s 2023-09-10T01:37:09.323 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:08 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T01:37:11.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:10 smithi099 ceph-mon[102692]: pgmap v1842: 129 pgs: 17 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 73 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 256 KiB/s rd, 280 KiB/s wr, 55 op/s 2023-09-10T01:37:11.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:10 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:11.218 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:10 smithi044 ceph-mon[98676]: pgmap v1842: 129 pgs: 17 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 73 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 256 KiB/s rd, 280 KiB/s wr, 55 op/s 2023-09-10T01:37:11.218 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:10 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:10 smithi131 ceph-mon[103713]: pgmap v1842: 129 pgs: 17 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 73 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 256 KiB/s rd, 280 KiB/s wr, 55 op/s 2023-09-10T01:37:11.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:10 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:12.123 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T01:37:12.123 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2023-09-10T01:37:12.123 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 5249, 2023-09-10T01:37:12.123 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 5060, 2023-09-10T01:37:12.124 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 8562536, 2023-09-10T01:37:12.124 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 1829694, 2023-09-10T01:37:12.124 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 6, 2023-09-10T01:37:12.124 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 2, 2023-09-10T01:37:12.124 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.533298176, 2023-09-10T01:37:12.125 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.168624952, 2023-09-10T01:37:12.125 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.171950418, 2023-09-10T01:37:12.125 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.165714880, 2023-09-10T01:37:12.125 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:12.125 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 5059, 2023-09-10T01:37:12.126 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.933991086, 2023-09-10T01:37:12.126 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000184619 2023-09-10T01:37:12.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.126 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:12.126 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.126 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.127 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.127 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 3067888, 2023-09-10T01:37:12.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 585072 2023-09-10T01:37:12.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.129 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2023-09-10T01:37:12.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 3818, 2023-09-10T01:37:12.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 4524, 2023-09-10T01:37:12.130 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 17612028, 2023-09-10T01:37:12.130 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 2669717, 2023-09-10T01:37:12.130 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 10, 2023-09-10T01:37:12.130 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2023-09-10T01:37:12.130 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.348655226, 2023-09-10T01:37:12.131 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.113717207, 2023-09-10T01:37:12.131 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.088973029, 2023-09-10T01:37:12.131 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.114842308, 2023-09-10T01:37:12.131 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:12.131 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 4521, 2023-09-10T01:37:12.132 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.589794489, 2023-09-10T01:37:12.132 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000130456 2023-09-10T01:37:12.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.132 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:12.132 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.132 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.133 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.133 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 480, 2023-09-10T01:37:12.133 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 1483184 2023-09-10T01:37:12.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.134 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2023-09-10T01:37:12.134 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 3528, 2023-09-10T01:37:12.134 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 3527, 2023-09-10T01:37:12.134 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 13475069, 2023-09-10T01:37:12.134 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 1096102, 2023-09-10T01:37:12.134 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 10, 2023-09-10T01:37:12.135 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2023-09-10T01:37:12.135 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.298695820, 2023-09-10T01:37:12.135 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.087935903, 2023-09-10T01:37:12.135 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.078262181, 2023-09-10T01:37:12.135 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.107860813, 2023-09-10T01:37:12.136 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:12.136 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3526, 2023-09-10T01:37:12.136 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.337015533, 2023-09-10T01:37:12.136 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000095580 2023-09-10T01:37:12.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.137 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:12.137 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.137 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.137 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.137 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 2912, 2023-09-10T01:37:12.138 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 448 2023-09-10T01:37:12.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.138 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2023-09-10T01:37:12.138 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2023-09-10T01:37:12.138 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2023-09-10T01:37:12.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.139 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2023-09-10T01:37:12.139 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2023-09-10T01:37:12.139 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2023-09-10T01:37:12.139 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 18862, 2023-09-10T01:37:12.139 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.503161798, 2023-09-10T01:37:12.140 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000026675 2023-09-10T01:37:12.140 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:12.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.140 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2023-09-10T01:37:12.140 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2023-09-10T01:37:12.141 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2023-09-10T01:37:12.141 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.141 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.141 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.141 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:12.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.142 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2023-09-10T01:37:12.142 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 0, 2023-09-10T01:37:12.142 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 0, 2023-09-10T01:37:12.142 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2023-09-10T01:37:12.143 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.143 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.143 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.143 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2023-09-10T01:37:12.143 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 0, 2023-09-10T01:37:12.144 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2023-09-10T01:37:12.144 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2023-09-10T01:37:12.144 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 0, 2023-09-10T01:37:12.144 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2023-09-10T01:37:12.144 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2023-09-10T01:37:12.145 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 1736, 2023-09-10T01:37:12.145 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 778, 2023-09-10T01:37:12.145 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 785, 2023-09-10T01:37:12.145 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 173, 2023-09-10T01:37:12.146 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 624, 2023-09-10T01:37:12.146 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 0, 2023-09-10T01:37:12.146 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 0, 2023-09-10T01:37:12.146 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 0, 2023-09-10T01:37:12.146 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 3, 2023-09-10T01:37:12.147 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 0, 2023-09-10T01:37:12.147 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 0, 2023-09-10T01:37:12.147 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 0, 2023-09-10T01:37:12.147 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 0, 2023-09-10T01:37:12.147 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2023-09-10T01:37:12.147 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2023-09-10T01:37:12.148 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2023-09-10T01:37:12.148 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2023-09-10T01:37:12.148 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2023-09-10T01:37:12.148 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 0, 2023-09-10T01:37:12.148 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 0, 2023-09-10T01:37:12.149 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 0, 2023-09-10T01:37:12.149 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 0, 2023-09-10T01:37:12.149 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2023-09-10T01:37:12.149 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 0, 2023-09-10T01:37:12.149 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2023-09-10T01:37:12.149 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 0, 2023-09-10T01:37:12.150 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 0, 2023-09-10T01:37:12.150 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 0, 2023-09-10T01:37:12.150 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 0, 2023-09-10T01:37:12.150 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2023-09-10T01:37:12.150 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2023-09-10T01:37:12.151 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2023-09-10T01:37:12.151 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 0, 2023-09-10T01:37:12.151 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2023-09-10T01:37:12.151 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 0, 2023-09-10T01:37:12.151 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 0, 2023-09-10T01:37:12.151 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2023-09-10T01:37:12.152 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 0, 2023-09-10T01:37:12.152 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 0, 2023-09-10T01:37:12.152 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2023-09-10T01:37:12.152 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-10T01:37:12.152 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-10T01:37:12.153 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 0, 2023-09-10T01:37:12.153 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 0, 2023-09-10T01:37:12.153 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 0, 2023-09-10T01:37:12.153 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 0 2023-09-10T01:37:12.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.154 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2023-09-10T01:37:12.154 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 0, 2023-09-10T01:37:12.154 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 0, 2023-09-10T01:37:12.154 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 0, 2023-09-10T01:37:12.154 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 0, 2023-09-10T01:37:12.154 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 0, 2023-09-10T01:37:12.155 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 0, 2023-09-10T01:37:12.155 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2023-09-10T01:37:12.155 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2023-09-10T01:37:12.155 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 0, 2023-09-10T01:37:12.155 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 0, 2023-09-10T01:37:12.156 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2023-09-10T01:37:12.156 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2023-09-10T01:37:12.156 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2023-09-10T01:37:12.156 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2023-09-10T01:37:12.156 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2023-09-10T01:37:12.156 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2023-09-10T01:37:12.157 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2023-09-10T01:37:12.157 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2023-09-10T01:37:12.157 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 0, 2023-09-10T01:37:12.157 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2023-09-10T01:37:12.157 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2023-09-10T01:37:12.158 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2023-09-10T01:37:12.158 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2023-09-10T01:37:12.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.158 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2023-09-10T01:37:12.158 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2023-09-10T01:37:12.159 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 0, 2023-09-10T01:37:12.159 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2023-09-10T01:37:12.159 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2023-09-10T01:37:12.159 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 8485, 2023-09-10T01:37:12.160 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2023-09-10T01:37:12.160 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2023-09-10T01:37:12.160 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 0, 2023-09-10T01:37:12.160 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2023-09-10T01:37:12.160 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2023-09-10T01:37:12.161 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 13, 2023-09-10T01:37:12.161 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 2, 2023-09-10T01:37:12.161 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2023-09-10T01:37:12.161 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2023-09-10T01:37:12.161 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 4194304, 2023-09-10T01:37:12.161 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 37575233, 2023-09-10T01:37:12.162 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 37574659, 2023-09-10T01:37:12.162 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2023-09-10T01:37:12.162 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.162 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.162 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.163 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 8485 2023-09-10T01:37:12.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.163 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2023-09-10T01:37:12.163 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 1577, 2023-09-10T01:37:12.164 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 2526, 2023-09-10T01:37:12.164 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 949, 2023-09-10T01:37:12.164 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 71, 2023-09-10T01:37:12.164 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 118, 2023-09-10T01:37:12.164 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 47, 2023-09-10T01:37:12.164 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 1736, 2023-09-10T01:37:12.165 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 2975, 2023-09-10T01:37:12.165 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 1239, 2023-09-10T01:37:12.165 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 0, 2023-09-10T01:37:12.165 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 0, 2023-09-10T01:37:12.165 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 0, 2023-09-10T01:37:12.166 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 55556, 2023-09-10T01:37:12.166 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 207152 2023-09-10T01:37:12.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.166 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2023-09-10T01:37:12.166 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 0, 2023-09-10T01:37:12.166 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 0, 2023-09-10T01:37:12.167 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 0, 2023-09-10T01:37:12.167 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 0, 2023-09-10T01:37:12.167 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 0, 2023-09-10T01:37:12.167 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2023-09-10T01:37:12.167 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.168 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.168 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.168 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2023-09-10T01:37:12.168 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.168 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.169 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.169 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2023-09-10T01:37:12.169 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.169 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.170 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.170 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2023-09-10T01:37:12.170 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.170 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.171 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.171 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2023-09-10T01:37:12.171 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.171 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.171 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.172 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2023-09-10T01:37:12.172 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.172 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.172 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.173 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2023-09-10T01:37:12.173 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.173 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.173 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.174 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2023-09-10T01:37:12.174 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.174 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.174 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.175 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2023-09-10T01:37:12.175 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.175 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.175 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.176 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2023-09-10T01:37:12.176 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.176 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.176 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.176 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2023-09-10T01:37:12.177 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.177 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.177 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.177 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2023-09-10T01:37:12.178 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.178 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.178 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.178 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2023-09-10T01:37:12.178 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.179 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.179 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.179 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2023-09-10T01:37:12.179 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.180 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.180 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.180 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2023-09-10T01:37:12.180 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.180 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.181 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.181 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2023-09-10T01:37:12.181 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.181 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.181 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.182 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2023-09-10T01:37:12.182 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.182 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.182 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.183 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2023-09-10T01:37:12.183 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.183 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.183 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.184 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2023-09-10T01:37:12.184 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.184 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.184 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.184 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2023-09-10T01:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2023-09-10T01:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.186 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.186 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.186 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2023-09-10T01:37:12.186 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.187 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.187 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.187 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2023-09-10T01:37:12.187 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.187 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.188 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.188 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2023-09-10T01:37:12.188 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.188 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.188 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.189 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2023-09-10T01:37:12.189 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.189 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.189 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.190 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2023-09-10T01:37:12.190 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.190 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.190 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.190 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2023-09-10T01:37:12.191 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.191 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.191 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.191 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2023-09-10T01:37:12.192 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.192 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.192 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.192 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2023-09-10T01:37:12.192 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.193 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.193 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.193 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2023-09-10T01:37:12.193 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2023-09-10T01:37:12.193 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2023-09-10T01:37:12.194 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.194 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.194 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.194 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:12.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.195 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2023-09-10T01:37:12.195 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 1, 2023-09-10T01:37:12.195 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 2, 2023-09-10T01:37:12.195 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 1, 2023-09-10T01:37:12.195 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 2, 2023-09-10T01:37:12.195 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2023-09-10T01:37:12.195 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 0, 2023-09-10T01:37:12.196 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 0, 2023-09-10T01:37:12.196 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 6205, 2023-09-10T01:37:12.196 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2023-09-10T01:37:12.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.196 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2023-09-10T01:37:12.196 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2023-09-10T01:37:12.197 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2023-09-10T01:37:12.197 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2023-09-10T01:37:12.197 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2023-09-10T01:37:12.197 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2023-09-10T01:37:12.197 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2023-09-10T01:37:12.198 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-10T01:37:12.198 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2023-09-10T01:37:12.198 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-10T01:37:12.198 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2023-09-10T01:37:12.198 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2023-09-10T01:37:12.198 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2023-09-10T01:37:12.199 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-10T01:37:12.199 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2023-09-10T01:37:12.199 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2023-09-10T01:37:12.199 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2023-09-10T01:37:12.199 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2023-09-10T01:37:12.200 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2023-09-10T01:37:12.200 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-10T01:37:12.200 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2023-09-10T01:37:12.200 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-10T01:37:12.200 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2023-09-10T01:37:12.200 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2023-09-10T01:37:12.201 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2023-09-10T01:37:12.201 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2023-09-10T01:37:12.201 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2023-09-10T01:37:12.201 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-10T01:37:12.201 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2023-09-10T01:37:12.202 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2023-09-10T01:37:12.202 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2023-09-10T01:37:12.202 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2023-09-10T01:37:12.202 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2023-09-10T01:37:12.202 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2023-09-10T01:37:12.203 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2023-09-10T01:37:12.203 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2023-09-10T01:37:12.203 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2023-09-10T01:37:12.203 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 208615, 2023-09-10T01:37:12.203 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 64, 2023-09-10T01:37:12.203 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2023-09-10T01:37:12.204 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2023-09-10T01:37:12.204 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2023-09-10T01:37:12.204 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2023-09-10T01:37:12.204 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2023-09-10T01:37:12.204 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2023-09-10T01:37:12.204 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2023-09-10T01:37:12.204 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2023-09-10T01:37:12.205 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 10136, 2023-09-10T01:37:12.205 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 196, 2023-09-10T01:37:12.205 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2023-09-10T01:37:12.205 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2023-09-10T01:37:12.205 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2023-09-10T01:37:12.205 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2023-09-10T01:37:12.206 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 4704180, 2023-09-10T01:37:12.206 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 10990, 2023-09-10T01:37:12.206 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2023-09-10T01:37:12.206 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2023-09-10T01:37:12.206 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2023-09-10T01:37:12.207 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2023-09-10T01:37:12.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.207 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2023-09-10T01:37:12.207 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2023-09-10T01:37:12.207 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2023-09-10T01:37:12.207 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 11340, 2023-09-10T01:37:12.208 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 21, 2023-09-10T01:37:12.208 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 3, 2023-09-10T01:37:12.208 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 11340, 2023-09-10T01:37:12.208 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2023-09-10T01:37:12.208 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 11340, 2023-09-10T01:37:12.208 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 18.735718323, 2023-09-10T01:37:12.209 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001652179 2023-09-10T01:37:12.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.209 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2023-09-10T01:37:12.209 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2023-09-10T01:37:12.209 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 11340, 2023-09-10T01:37:12.209 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 11341 2023-09-10T01:37:12.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.210 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 11340, 2023-09-10T01:37:12.210 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 11340, 2023-09-10T01:37:12.210 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 0, 2023-09-10T01:37:12.210 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2023-09-10T01:37:12.210 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2023-09-10T01:37:12.211 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 6861, 2023-09-10T01:37:12.211 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 0, 2023-09-10T01:37:12.211 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 4477, 2023-09-10T01:37:12.211 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 0, 2023-09-10T01:37:12.211 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 0, 2023-09-10T01:37:12.212 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2023-09-10T01:37:12.212 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2023-09-10T01:37:12.212 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2023-09-10T01:37:12.212 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2023-09-10T01:37:12.212 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 0, 2023-09-10T01:37:12.212 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2023-09-10T01:37:12.213 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2023-09-10T01:37:12.213 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2023-09-10T01:37:12.213 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 1, 2023-09-10T01:37:12.213 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 0, 2023-09-10T01:37:12.213 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2023-09-10T01:37:12.213 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2023-09-10T01:37:12.214 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2023-09-10T01:37:12.214 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2023-09-10T01:37:12.214 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2023-09-10T01:37:12.214 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2023-09-10T01:37:12.214 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2023-09-10T01:37:12.215 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2023-09-10T01:37:12.215 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2023-09-10T01:37:12.215 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2023-09-10T01:37:12.215 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2023-09-10T01:37:12.215 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2023-09-10T01:37:12.216 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2023-09-10T01:37:12.216 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2023-09-10T01:37:12.216 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2023-09-10T01:37:12.216 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2023-09-10T01:37:12.216 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2023-09-10T01:37:12.217 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2023-09-10T01:37:12.217 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2023-09-10T01:37:12.217 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2023-09-10T01:37:12.217 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2023-09-10T01:37:12.217 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2023-09-10T01:37:12.217 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2023-09-10T01:37:12.218 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2023-09-10T01:37:12.218 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2023-09-10T01:37:12.218 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2023-09-10T01:37:12.218 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 212, 2023-09-10T01:37:12.218 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2023-09-10T01:37:12.218 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 125, 2023-09-10T01:37:12.219 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 7, 2023-09-10T01:37:12.219 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 14, 2023-09-10T01:37:12.219 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 7, 2023-09-10T01:37:12.219 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2023-09-10T01:37:12.219 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 0, 2023-09-10T01:37:12.220 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 2, 2023-09-10T01:37:12.220 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 0 2023-09-10T01:37:12.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.220 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2023-09-10T01:37:12.220 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 0, 2023-09-10T01:37:12.220 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2023-09-10T01:37:12.221 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2023-09-10T01:37:12.221 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2023-09-10T01:37:12.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.221 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2023-09-10T01:37:12.221 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2023-09-10T01:37:12.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.221 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2023-09-10T01:37:12.222 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2023-09-10T01:37:12.222 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 0, 2023-09-10T01:37:12.222 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2023-09-10T01:37:12.222 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 0, 2023-09-10T01:37:12.222 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 0, 2023-09-10T01:37:12.223 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2023-09-10T01:37:12.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.223 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-10T01:37:12.223 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2023-09-10T01:37:12.223 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2023-09-10T01:37:12.224 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2023-09-10T01:37:12.224 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 12591, 2023-09-10T01:37:12.224 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 38450948, 2023-09-10T01:37:12.224 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:12.224 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 12591, 2023-09-10T01:37:12.224 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2023-09-10T01:37:12.225 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2023-09-10T01:37:12.225 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 12591, 2023-09-10T01:37:12.225 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 38450948, 2023-09-10T01:37:12.225 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2023-09-10T01:37:12.225 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.225 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.226 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.226 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:12.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.226 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2023-09-10T01:37:12.226 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2023-09-10T01:37:12.227 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2023-09-10T01:37:12.227 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2023-09-10T01:37:12.227 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2023-09-10T01:37:12.227 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2023-09-10T01:37:12.227 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:12.227 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:12.228 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 11340, 2023-09-10T01:37:12.228 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 33418331, 2023-09-10T01:37:12.228 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 11340, 2023-09-10T01:37:12.228 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 33418331, 2023-09-10T01:37:12.228 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2023-09-10T01:37:12.228 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.229 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.229 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.229 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:12.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.229 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2023-09-10T01:37:12.230 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2023-09-10T01:37:12.230 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2023-09-10T01:37:12.230 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2023-09-10T01:37:12.230 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2023-09-10T01:37:12.230 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2023-09-10T01:37:12.231 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:12.231 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:12.231 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 11340, 2023-09-10T01:37:12.231 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 11340, 2023-09-10T01:37:12.231 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 11340, 2023-09-10T01:37:12.232 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 11340, 2023-09-10T01:37:12.232 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2023-09-10T01:37:12.232 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.232 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.232 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.232 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:12.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.233 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2023-09-10T01:37:12.233 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2023-09-10T01:37:12.233 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2023-09-10T01:37:12.233 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2023-09-10T01:37:12.234 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2023-09-10T01:37:12.234 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2023-09-10T01:37:12.234 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:12.234 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:12.234 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2023-09-10T01:37:12.235 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2023-09-10T01:37:12.235 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2023-09-10T01:37:12.235 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2023-09-10T01:37:12.235 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2023-09-10T01:37:12.235 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.235 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.236 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.236 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:12.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:12.236 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x56205d68a960": { 2023-09-10T01:37:12.236 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2023-09-10T01:37:12.237 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2023-09-10T01:37:12.237 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2023-09-10T01:37:12.237 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2023-09-10T01:37:12.237 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2023-09-10T01:37:12.237 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:12.237 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:12.238 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2023-09-10T01:37:12.238 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2023-09-10T01:37:12.238 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2023-09-10T01:37:12.238 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2023-09-10T01:37:12.238 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2023-09-10T01:37:12.238 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:12.239 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:12.239 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:12.239 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:12.239 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:12.239 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T01:37:12.466 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-09-10T01:37:12.466 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-09-10T01:37:12.466 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2023-09-10T01:37:12.467 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2023-09-10T01:37:12.467 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2023-09-10T01:37:12.467 DEBUG:tasks.check_counter:Getting stats from d 2023-09-10T01:37:12.467 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.d perf dump 2023-09-10T01:37:13.041 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T01:37:13.249 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:12 smithi044 ceph-mon[98676]: pgmap v1843: 129 pgs: 13 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 80 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 244 KiB/s wr, 47 op/s 2023-09-10T01:37:13.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:12 smithi131 ceph-mon[103713]: pgmap v1843: 129 pgs: 13 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 80 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 244 KiB/s wr, 47 op/s 2023-09-10T01:37:13.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:12 smithi099 ceph-mon[102692]: pgmap v1843: 129 pgs: 13 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 80 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 244 KiB/s wr, 47 op/s 2023-09-10T01:37:14.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:13 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:14.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:13 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:14.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:13 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:15.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:14 smithi131 ceph-mon[103713]: pgmap v1844: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 138 KiB/s wr, 34 op/s 2023-09-10T01:37:15.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:14 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:14 smithi044 ceph-mon[98676]: pgmap v1844: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 138 KiB/s wr, 34 op/s 2023-09-10T01:37:15.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:14 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:15.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:14 smithi099 ceph-mon[102692]: pgmap v1844: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 138 KiB/s wr, 34 op/s 2023-09-10T01:37:15.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:14 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:16 smithi131 ceph-mon[103713]: pgmap v1845: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T01:37:17.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:16 smithi131 ceph-mon[103713]: osdmap e213: 12 total, 12 up, 12 in 2023-09-10T01:37:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:16 smithi044 ceph-mon[98676]: pgmap v1845: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T01:37:17.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:16 smithi044 ceph-mon[98676]: osdmap e213: 12 total, 12 up, 12 in 2023-09-10T01:37:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:16 smithi099 ceph-mon[102692]: pgmap v1845: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T01:37:17.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:16 smithi099 ceph-mon[102692]: osdmap e213: 12 total, 12 up, 12 in 2023-09-10T01:37:17.887 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T01:37:17.887 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2023-09-10T01:37:17.888 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 2747, 2023-09-10T01:37:17.888 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 2747, 2023-09-10T01:37:17.888 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 13225671, 2023-09-10T01:37:17.888 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 853531, 2023-09-10T01:37:17.889 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 7, 2023-09-10T01:37:17.889 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2023-09-10T01:37:17.889 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.277838701, 2023-09-10T01:37:17.889 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.066725914, 2023-09-10T01:37:17.889 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.083525053, 2023-09-10T01:37:17.889 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.106470885, 2023-09-10T01:37:17.890 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:17.890 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2746, 2023-09-10T01:37:17.890 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.989393657, 2023-09-10T01:37:17.890 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000360303 2023-09-10T01:37:17.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.891 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:17.891 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.891 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.891 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.892 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 2912, 2023-09-10T01:37:17.892 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 448 2023-09-10T01:37:17.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.892 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2023-09-10T01:37:17.893 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 694, 2023-09-10T01:37:17.893 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 913, 2023-09-10T01:37:17.893 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 4369723, 2023-09-10T01:37:17.893 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 766933, 2023-09-10T01:37:17.893 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 7, 2023-09-10T01:37:17.893 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 5, 2023-09-10T01:37:17.894 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.083941265, 2023-09-10T01:37:17.894 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.027941393, 2023-09-10T01:37:17.894 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.022754881, 2023-09-10T01:37:17.894 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.026249884, 2023-09-10T01:37:17.894 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:17.895 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 911, 2023-09-10T01:37:17.895 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.345413585, 2023-09-10T01:37:17.895 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000379158 2023-09-10T01:37:17.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.895 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:17.896 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.896 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.896 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.896 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-10T01:37:17.896 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 551664 2023-09-10T01:37:17.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.897 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2023-09-10T01:37:17.897 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 738, 2023-09-10T01:37:17.897 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 656, 2023-09-10T01:37:17.897 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1239660, 2023-09-10T01:37:17.898 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 280864, 2023-09-10T01:37:17.898 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2023-09-10T01:37:17.898 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2023-09-10T01:37:17.898 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.082970638, 2023-09-10T01:37:17.898 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.026379115, 2023-09-10T01:37:17.898 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.031348815, 2023-09-10T01:37:17.899 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.022696012, 2023-09-10T01:37:17.899 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:17.899 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 655, 2023-09-10T01:37:17.899 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.208798407, 2023-09-10T01:37:17.899 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000318776 2023-09-10T01:37:17.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.900 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:17.900 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.900 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.900 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 814928, 2023-09-10T01:37:17.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 183200 2023-09-10T01:37:17.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.901 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2023-09-10T01:37:17.901 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2023-09-10T01:37:17.902 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2023-09-10T01:37:17.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.902 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2023-09-10T01:37:17.902 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2023-09-10T01:37:17.902 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2023-09-10T01:37:17.903 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 6202, 2023-09-10T01:37:17.903 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.199062976, 2023-09-10T01:37:17.903 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000032096 2023-09-10T01:37:17.903 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:17.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.904 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2023-09-10T01:37:17.904 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2023-09-10T01:37:17.904 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2023-09-10T01:37:17.904 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.904 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.904 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.905 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:17.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.905 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2023-09-10T01:37:17.905 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 0, 2023-09-10T01:37:17.905 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 0, 2023-09-10T01:37:17.906 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2023-09-10T01:37:17.906 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.906 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.906 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.906 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2023-09-10T01:37:17.907 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 0, 2023-09-10T01:37:17.907 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2023-09-10T01:37:17.907 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2023-09-10T01:37:17.907 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 0, 2023-09-10T01:37:17.907 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2023-09-10T01:37:17.908 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2023-09-10T01:37:17.908 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 366, 2023-09-10T01:37:17.908 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 243, 2023-09-10T01:37:17.908 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 116, 2023-09-10T01:37:17.908 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 7, 2023-09-10T01:37:17.908 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 18, 2023-09-10T01:37:17.909 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 35, 2023-09-10T01:37:17.909 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 0, 2023-09-10T01:37:17.909 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 0, 2023-09-10T01:37:17.909 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 7, 2023-09-10T01:37:17.909 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 0, 2023-09-10T01:37:17.910 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 0, 2023-09-10T01:37:17.910 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 0, 2023-09-10T01:37:17.910 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 0, 2023-09-10T01:37:17.910 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2023-09-10T01:37:17.910 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2023-09-10T01:37:17.910 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2023-09-10T01:37:17.911 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2023-09-10T01:37:17.911 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2023-09-10T01:37:17.911 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 0, 2023-09-10T01:37:17.911 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 0, 2023-09-10T01:37:17.911 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 0, 2023-09-10T01:37:17.912 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 0, 2023-09-10T01:37:17.912 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2023-09-10T01:37:17.912 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 0, 2023-09-10T01:37:17.912 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2023-09-10T01:37:17.912 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 1048, 2023-09-10T01:37:17.912 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 8977410130, 2023-09-10T01:37:17.913 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 7, 2023-09-10T01:37:17.913 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 0, 2023-09-10T01:37:17.913 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2023-09-10T01:37:17.913 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2023-09-10T01:37:17.913 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2023-09-10T01:37:17.914 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 0, 2023-09-10T01:37:17.914 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2023-09-10T01:37:17.914 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 0, 2023-09-10T01:37:17.914 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 0, 2023-09-10T01:37:17.914 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2023-09-10T01:37:17.914 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 0, 2023-09-10T01:37:17.915 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 0, 2023-09-10T01:37:17.915 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2023-09-10T01:37:17.915 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-10T01:37:17.915 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-10T01:37:17.915 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 0, 2023-09-10T01:37:17.916 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 0, 2023-09-10T01:37:17.916 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 0, 2023-09-10T01:37:17.916 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 0 2023-09-10T01:37:17.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.916 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2023-09-10T01:37:17.916 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 0, 2023-09-10T01:37:17.917 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 0, 2023-09-10T01:37:17.917 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 0, 2023-09-10T01:37:17.917 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 0, 2023-09-10T01:37:17.917 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 0, 2023-09-10T01:37:17.917 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 0, 2023-09-10T01:37:17.918 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2023-09-10T01:37:17.918 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2023-09-10T01:37:17.918 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 0, 2023-09-10T01:37:17.918 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 0, 2023-09-10T01:37:17.918 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2023-09-10T01:37:17.919 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2023-09-10T01:37:17.919 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2023-09-10T01:37:17.919 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2023-09-10T01:37:17.919 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2023-09-10T01:37:17.919 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2023-09-10T01:37:17.920 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2023-09-10T01:37:17.920 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2023-09-10T01:37:17.920 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 0, 2023-09-10T01:37:17.920 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2023-09-10T01:37:17.920 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2023-09-10T01:37:17.920 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2023-09-10T01:37:17.921 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2023-09-10T01:37:17.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.921 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2023-09-10T01:37:17.921 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2023-09-10T01:37:17.921 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 0, 2023-09-10T01:37:17.922 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2023-09-10T01:37:17.922 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2023-09-10T01:37:17.922 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 4538, 2023-09-10T01:37:17.922 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2023-09-10T01:37:17.922 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2023-09-10T01:37:17.922 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 0, 2023-09-10T01:37:17.923 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2023-09-10T01:37:17.923 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2023-09-10T01:37:17.923 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 2, 2023-09-10T01:37:17.923 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 80, 2023-09-10T01:37:17.923 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2023-09-10T01:37:17.924 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2023-09-10T01:37:17.924 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 58947428, 2023-09-10T01:37:17.924 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 59066058, 2023-09-10T01:37:17.924 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 59042703, 2023-09-10T01:37:17.924 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2023-09-10T01:37:17.924 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.925 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.925 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.925 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 4538 2023-09-10T01:37:17.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.926 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2023-09-10T01:37:17.926 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 64, 2023-09-10T01:37:17.926 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 273, 2023-09-10T01:37:17.926 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 209, 2023-09-10T01:37:17.926 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 21, 2023-09-10T01:37:17.927 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 47, 2023-09-10T01:37:17.927 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 26, 2023-09-10T01:37:17.927 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 366, 2023-09-10T01:37:17.927 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 709, 2023-09-10T01:37:17.927 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 343, 2023-09-10T01:37:17.928 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 0, 2023-09-10T01:37:17.928 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 0, 2023-09-10T01:37:17.928 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 0, 2023-09-10T01:37:17.928 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 46048, 2023-09-10T01:37:17.929 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 207152 2023-09-10T01:37:17.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.929 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2023-09-10T01:37:17.929 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 0, 2023-09-10T01:37:17.929 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 0, 2023-09-10T01:37:17.929 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 0, 2023-09-10T01:37:17.930 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 0, 2023-09-10T01:37:17.930 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 0, 2023-09-10T01:37:17.930 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2023-09-10T01:37:17.930 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.930 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.931 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.931 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2023-09-10T01:37:17.931 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.931 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.932 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.932 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2023-09-10T01:37:17.932 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.932 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.933 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.933 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2023-09-10T01:37:17.933 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.933 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.934 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.934 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2023-09-10T01:37:17.934 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.934 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.935 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.935 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2023-09-10T01:37:17.935 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.935 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.935 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.936 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2023-09-10T01:37:17.936 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.936 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.936 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.937 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2023-09-10T01:37:17.937 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.937 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.937 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.938 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2023-09-10T01:37:17.938 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.938 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.938 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.938 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2023-09-10T01:37:17.939 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.939 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.939 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.939 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2023-09-10T01:37:17.940 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.940 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.940 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.940 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2023-09-10T01:37:17.941 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.941 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.941 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.941 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2023-09-10T01:37:17.942 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.942 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.942 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.942 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2023-09-10T01:37:17.942 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.943 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.943 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.943 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2023-09-10T01:37:17.943 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.944 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.944 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.944 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2023-09-10T01:37:17.944 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.945 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.945 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.945 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2023-09-10T01:37:17.945 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.946 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.946 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.946 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2023-09-10T01:37:17.946 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.946 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.947 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.947 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2023-09-10T01:37:17.947 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.947 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.948 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.948 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2023-09-10T01:37:17.948 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.948 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.948 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.949 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2023-09-10T01:37:17.949 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.949 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.949 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.950 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2023-09-10T01:37:17.950 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.950 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.950 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.951 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2023-09-10T01:37:17.951 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.951 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.951 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.952 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2023-09-10T01:37:17.952 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.952 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.952 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.953 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2023-09-10T01:37:17.953 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.953 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.953 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.954 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2023-09-10T01:37:17.954 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.954 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.954 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.954 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2023-09-10T01:37:17.955 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.955 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.955 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.955 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2023-09-10T01:37:17.956 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.956 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.956 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.956 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2023-09-10T01:37:17.956 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.957 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.957 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.957 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2023-09-10T01:37:17.957 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2023-09-10T01:37:17.958 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2023-09-10T01:37:17.958 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.958 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.958 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.958 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:17.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.959 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2023-09-10T01:37:17.959 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 2, 2023-09-10T01:37:17.959 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 2, 2023-09-10T01:37:17.959 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 0, 2023-09-10T01:37:17.960 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 0, 2023-09-10T01:37:17.960 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2023-09-10T01:37:17.960 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 0, 2023-09-10T01:37:17.960 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 0, 2023-09-10T01:37:17.960 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 4803, 2023-09-10T01:37:17.960 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2023-09-10T01:37:17.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.961 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2023-09-10T01:37:17.961 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2023-09-10T01:37:17.961 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2023-09-10T01:37:17.961 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2023-09-10T01:37:17.962 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2023-09-10T01:37:17.962 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2023-09-10T01:37:17.962 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2023-09-10T01:37:17.962 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-10T01:37:17.962 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2023-09-10T01:37:17.962 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-10T01:37:17.963 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2023-09-10T01:37:17.963 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2023-09-10T01:37:17.963 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2023-09-10T01:37:17.963 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-10T01:37:17.963 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2023-09-10T01:37:17.964 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2023-09-10T01:37:17.964 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2023-09-10T01:37:17.964 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2023-09-10T01:37:17.964 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2023-09-10T01:37:17.964 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-10T01:37:17.964 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2023-09-10T01:37:17.965 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-10T01:37:17.965 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2023-09-10T01:37:17.965 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2023-09-10T01:37:17.965 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2023-09-10T01:37:17.965 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2023-09-10T01:37:17.966 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2023-09-10T01:37:17.966 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-10T01:37:17.966 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2023-09-10T01:37:17.966 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2023-09-10T01:37:17.966 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2023-09-10T01:37:17.967 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2023-09-10T01:37:17.967 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2023-09-10T01:37:17.967 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2023-09-10T01:37:17.967 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2023-09-10T01:37:17.967 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2023-09-10T01:37:17.968 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2023-09-10T01:37:17.968 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 217253, 2023-09-10T01:37:17.968 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 68, 2023-09-10T01:37:17.968 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2023-09-10T01:37:17.968 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2023-09-10T01:37:17.968 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2023-09-10T01:37:17.969 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2023-09-10T01:37:17.969 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2023-09-10T01:37:17.969 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2023-09-10T01:37:17.969 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2023-09-10T01:37:17.969 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2023-09-10T01:37:17.969 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 10136, 2023-09-10T01:37:17.970 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 196, 2023-09-10T01:37:17.970 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2023-09-10T01:37:17.970 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2023-09-10T01:37:17.970 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2023-09-10T01:37:17.970 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2023-09-10T01:37:17.971 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 558115, 2023-09-10T01:37:17.971 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 19666, 2023-09-10T01:37:17.971 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2023-09-10T01:37:17.971 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2023-09-10T01:37:17.971 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2023-09-10T01:37:17.971 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2023-09-10T01:37:17.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.972 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2023-09-10T01:37:17.972 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2023-09-10T01:37:17.972 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2023-09-10T01:37:17.972 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 3749, 2023-09-10T01:37:17.973 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 26, 2023-09-10T01:37:17.973 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 4, 2023-09-10T01:37:17.973 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 3749, 2023-09-10T01:37:17.973 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2023-09-10T01:37:17.974 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3749, 2023-09-10T01:37:17.974 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 10.222846725, 2023-09-10T01:37:17.974 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.002726819 2023-09-10T01:37:17.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.974 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2023-09-10T01:37:17.974 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2023-09-10T01:37:17.975 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3749, 2023-09-10T01:37:17.975 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 3750 2023-09-10T01:37:17.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.975 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 3749, 2023-09-10T01:37:17.975 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 3749, 2023-09-10T01:37:17.976 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 0, 2023-09-10T01:37:17.976 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2023-09-10T01:37:17.976 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2023-09-10T01:37:17.976 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 2138, 2023-09-10T01:37:17.976 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 0, 2023-09-10T01:37:17.976 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 1608, 2023-09-10T01:37:17.977 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 0, 2023-09-10T01:37:17.977 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 0, 2023-09-10T01:37:17.977 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2023-09-10T01:37:17.977 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2023-09-10T01:37:17.977 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2023-09-10T01:37:17.978 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2023-09-10T01:37:17.978 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 0, 2023-09-10T01:37:17.978 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2023-09-10T01:37:17.978 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2023-09-10T01:37:17.978 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2023-09-10T01:37:17.979 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 2, 2023-09-10T01:37:17.979 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 0, 2023-09-10T01:37:17.979 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2023-09-10T01:37:17.979 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2023-09-10T01:37:17.979 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2023-09-10T01:37:17.979 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2023-09-10T01:37:17.980 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2023-09-10T01:37:17.980 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2023-09-10T01:37:17.980 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2023-09-10T01:37:17.980 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2023-09-10T01:37:17.980 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2023-09-10T01:37:17.981 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2023-09-10T01:37:17.981 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2023-09-10T01:37:17.981 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2023-09-10T01:37:17.981 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2023-09-10T01:37:17.981 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2023-09-10T01:37:17.981 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2023-09-10T01:37:17.982 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2023-09-10T01:37:17.982 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2023-09-10T01:37:17.982 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2023-09-10T01:37:17.982 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2023-09-10T01:37:17.982 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2023-09-10T01:37:17.983 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2023-09-10T01:37:17.983 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2023-09-10T01:37:17.983 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2023-09-10T01:37:17.983 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2023-09-10T01:37:17.983 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2023-09-10T01:37:17.983 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2023-09-10T01:37:17.984 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 213, 2023-09-10T01:37:17.984 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2023-09-10T01:37:17.984 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 46, 2023-09-10T01:37:17.984 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 8, 2023-09-10T01:37:17.984 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 9, 2023-09-10T01:37:17.985 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 1, 2023-09-10T01:37:17.985 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2023-09-10T01:37:17.985 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 0, 2023-09-10T01:37:17.985 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 2, 2023-09-10T01:37:17.985 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 0 2023-09-10T01:37:17.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.986 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2023-09-10T01:37:17.986 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 0, 2023-09-10T01:37:17.986 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2023-09-10T01:37:17.986 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2023-09-10T01:37:17.986 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2023-09-10T01:37:17.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.987 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2023-09-10T01:37:17.987 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2023-09-10T01:37:17.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.987 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2023-09-10T01:37:17.987 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2023-09-10T01:37:17.988 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 0, 2023-09-10T01:37:17.988 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2023-09-10T01:37:17.988 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 0, 2023-09-10T01:37:17.988 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 0, 2023-09-10T01:37:17.988 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2023-09-10T01:37:17.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.989 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-10T01:37:17.989 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2023-09-10T01:37:17.989 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2023-09-10T01:37:17.989 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2023-09-10T01:37:17.989 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 4175, 2023-09-10T01:37:17.990 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 18435283, 2023-09-10T01:37:17.990 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:17.990 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 4175, 2023-09-10T01:37:17.990 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2023-09-10T01:37:17.990 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2023-09-10T01:37:17.991 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 4175, 2023-09-10T01:37:17.991 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 18435283, 2023-09-10T01:37:17.991 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2023-09-10T01:37:17.991 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.991 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.991 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.992 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:17.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.992 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2023-09-10T01:37:17.992 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2023-09-10T01:37:17.992 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2023-09-10T01:37:17.993 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2023-09-10T01:37:17.993 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2023-09-10T01:37:17.993 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2023-09-10T01:37:17.993 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:17.993 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:17.994 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 3749, 2023-09-10T01:37:17.994 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 16974366, 2023-09-10T01:37:17.994 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 3749, 2023-09-10T01:37:17.994 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 16974366, 2023-09-10T01:37:17.994 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2023-09-10T01:37:17.994 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.995 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.995 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.995 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:17.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.995 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2023-09-10T01:37:17.996 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2023-09-10T01:37:17.996 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2023-09-10T01:37:17.996 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2023-09-10T01:37:17.996 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2023-09-10T01:37:17.996 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2023-09-10T01:37:17.997 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:17.997 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:17.997 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 3749, 2023-09-10T01:37:17.997 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 3749, 2023-09-10T01:37:17.997 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 3749, 2023-09-10T01:37:17.997 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 3749, 2023-09-10T01:37:17.998 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2023-09-10T01:37:17.998 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:17.998 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:17.998 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:17.998 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:17.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:17.999 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2023-09-10T01:37:17.999 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2023-09-10T01:37:17.999 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2023-09-10T01:37:17.999 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2023-09-10T01:37:17.999 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2023-09-10T01:37:18.000 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2023-09-10T01:37:18.000 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:18.000 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:18.000 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2023-09-10T01:37:18.000 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2023-09-10T01:37:18.000 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2023-09-10T01:37:18.001 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2023-09-10T01:37:18.001 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2023-09-10T01:37:18.001 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:18.001 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:18.001 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:18.002 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:18.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:18.002 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x5615a01c6960": { 2023-09-10T01:37:18.002 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2023-09-10T01:37:18.002 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2023-09-10T01:37:18.003 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2023-09-10T01:37:18.003 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2023-09-10T01:37:18.003 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2023-09-10T01:37:18.003 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:18.003 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:18.003 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2023-09-10T01:37:18.004 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2023-09-10T01:37:18.004 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2023-09-10T01:37:18.004 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2023-09-10T01:37:18.004 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2023-09-10T01:37:18.004 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:18.005 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:18.005 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:18.005 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:18.005 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:18.005 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T01:37:19.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:18 smithi044 ceph-mon[98676]: pgmap v1847: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-09-10T01:37:19.297 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:18 smithi131 ceph-mon[103713]: pgmap v1847: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-09-10T01:37:19.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:18 smithi099 ceph-mon[102692]: pgmap v1847: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-09-10T01:37:20.149 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2023-09-10T01:37:20.149 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2023-09-10T01:37:20.149 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=7 2023-09-10T01:37:20.150 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2023-09-10T01:37:20.150 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2023-09-10T01:37:20.150 DEBUG:tasks.check_counter:Getting stats from g 2023-09-10T01:37:20.151 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.g perf dump 2023-09-10T01:37:20.737 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T01:37:21.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:20 smithi131 ceph-mon[103713]: pgmap v1848: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-09-10T01:37:21.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:20 smithi044 ceph-mon[98676]: pgmap v1848: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-09-10T01:37:21.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:20 smithi099 ceph-mon[102692]: pgmap v1848: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-09-10T01:37:23.023 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:22 smithi044 ceph-mon[98676]: pgmap v1849: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 KiB/s wr, 12 op/s 2023-09-10T01:37:23.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:22 smithi131 ceph-mon[103713]: pgmap v1849: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 KiB/s wr, 12 op/s 2023-09-10T01:37:23.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:22 smithi099 ceph-mon[102692]: pgmap v1849: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 KiB/s wr, 12 op/s 2023-09-10T01:37:23.997 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T01:37:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2023-09-10T01:37:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 4261, 2023-09-10T01:37:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 4259, 2023-09-10T01:37:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 22492770, 2023-09-10T01:37:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 1323928, 2023-09-10T01:37:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 11, 2023-09-10T01:37:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 4, 2023-09-10T01:37:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.455488329, 2023-09-10T01:37:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.145634500, 2023-09-10T01:37:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.120500930, 2023-09-10T01:37:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.153254114, 2023-09-10T01:37:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 4257, 2023-09-10T01:37:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.768250766, 2023-09-10T01:37:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000180467 2023-09-10T01:37:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-10T01:37:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2023-09-10T01:37:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2023-09-10T01:37:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 6170, 2023-09-10T01:37:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 6879, 2023-09-10T01:37:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 26352278, 2023-09-10T01:37:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 3491811, 2023-09-10T01:37:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 12, 2023-09-10T01:37:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 6, 2023-09-10T01:37:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.441707948, 2023-09-10T01:37:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.132230492, 2023-09-10T01:37:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.114052848, 2023-09-10T01:37:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.162677584, 2023-09-10T01:37:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 6876, 2023-09-10T01:37:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.462958769, 2023-09-10T01:37:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000067329 2023-09-10T01:37:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 640, 2023-09-10T01:37:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 1574128 2023-09-10T01:37:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2023-09-10T01:37:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 2073, 2023-09-10T01:37:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 1880, 2023-09-10T01:37:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 28517075, 2023-09-10T01:37:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 841355, 2023-09-10T01:37:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 10, 2023-09-10T01:37:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 4, 2023-09-10T01:37:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.250341474, 2023-09-10T01:37:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.073030987, 2023-09-10T01:37:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.124450130, 2023-09-10T01:37:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.054577804, 2023-09-10T01:37:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1880, 2023-09-10T01:37:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.447790220, 2023-09-10T01:37:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000238186 2023-09-10T01:37:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 3085408, 2023-09-10T01:37:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 587568 2023-09-10T01:37:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2023-09-10T01:37:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2023-09-10T01:37:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2023-09-10T01:37:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2023-09-10T01:37:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2023-09-10T01:37:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2023-09-10T01:37:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 18842, 2023-09-10T01:37:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.479216967, 2023-09-10T01:37:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000025433 2023-09-10T01:37:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2023-09-10T01:37:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2023-09-10T01:37:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2023-09-10T01:37:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2023-09-10T01:37:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 0, 2023-09-10T01:37:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 0, 2023-09-10T01:37:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2023-09-10T01:37:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2023-09-10T01:37:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 0, 2023-09-10T01:37:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2023-09-10T01:37:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2023-09-10T01:37:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 0, 2023-09-10T01:37:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2023-09-10T01:37:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2023-09-10T01:37:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 241, 2023-09-10T01:37:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 18, 2023-09-10T01:37:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 110, 2023-09-10T01:37:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 113, 2023-09-10T01:37:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 214, 2023-09-10T01:37:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 0, 2023-09-10T01:37:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 0, 2023-09-10T01:37:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 0, 2023-09-10T01:37:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 3, 2023-09-10T01:37:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 0, 2023-09-10T01:37:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 0, 2023-09-10T01:37:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 0, 2023-09-10T01:37:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 0, 2023-09-10T01:37:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2023-09-10T01:37:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2023-09-10T01:37:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2023-09-10T01:37:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2023-09-10T01:37:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2023-09-10T01:37:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 0, 2023-09-10T01:37:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 0, 2023-09-10T01:37:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 0, 2023-09-10T01:37:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 0, 2023-09-10T01:37:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2023-09-10T01:37:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 0, 2023-09-10T01:37:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2023-09-10T01:37:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 0, 2023-09-10T01:37:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 0, 2023-09-10T01:37:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 0, 2023-09-10T01:37:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 0, 2023-09-10T01:37:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2023-09-10T01:37:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2023-09-10T01:37:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2023-09-10T01:37:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 0, 2023-09-10T01:37:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2023-09-10T01:37:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 0, 2023-09-10T01:37:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 0, 2023-09-10T01:37:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2023-09-10T01:37:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 0, 2023-09-10T01:37:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 0, 2023-09-10T01:37:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2023-09-10T01:37:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-10T01:37:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-10T01:37:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 0, 2023-09-10T01:37:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 0, 2023-09-10T01:37:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 0, 2023-09-10T01:37:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 0 2023-09-10T01:37:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2023-09-10T01:37:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 0, 2023-09-10T01:37:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 0, 2023-09-10T01:37:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 0, 2023-09-10T01:37:24.028 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 0, 2023-09-10T01:37:24.028 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 0, 2023-09-10T01:37:24.028 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 0, 2023-09-10T01:37:24.028 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2023-09-10T01:37:24.028 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2023-09-10T01:37:24.028 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 0, 2023-09-10T01:37:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 0, 2023-09-10T01:37:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2023-09-10T01:37:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2023-09-10T01:37:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2023-09-10T01:37:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2023-09-10T01:37:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2023-09-10T01:37:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2023-09-10T01:37:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2023-09-10T01:37:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2023-09-10T01:37:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 0, 2023-09-10T01:37:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2023-09-10T01:37:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2023-09-10T01:37:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2023-09-10T01:37:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2023-09-10T01:37:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2023-09-10T01:37:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2023-09-10T01:37:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 0, 2023-09-10T01:37:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2023-09-10T01:37:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2023-09-10T01:37:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 7646, 2023-09-10T01:37:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2023-09-10T01:37:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2023-09-10T01:37:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 0, 2023-09-10T01:37:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2023-09-10T01:37:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2023-09-10T01:37:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 21, 2023-09-10T01:37:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 2, 2023-09-10T01:37:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2023-09-10T01:37:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2023-09-10T01:37:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 4194304, 2023-09-10T01:37:24.035 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 75099194, 2023-09-10T01:37:24.035 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 75064817, 2023-09-10T01:37:24.035 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2023-09-10T01:37:24.035 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.035 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 7646 2023-09-10T01:37:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2023-09-10T01:37:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 275, 2023-09-10T01:37:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 2099, 2023-09-10T01:37:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 1824, 2023-09-10T01:37:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 22, 2023-09-10T01:37:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 82, 2023-09-10T01:37:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 60, 2023-09-10T01:37:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 471, 2023-09-10T01:37:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 2425, 2023-09-10T01:37:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 1954, 2023-09-10T01:37:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 0, 2023-09-10T01:37:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 0, 2023-09-10T01:37:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 0, 2023-09-10T01:37:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 56516, 2023-09-10T01:37:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 207152 2023-09-10T01:37:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2023-09-10T01:37:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 0, 2023-09-10T01:37:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 0, 2023-09-10T01:37:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 0, 2023-09-10T01:37:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 0, 2023-09-10T01:37:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 0, 2023-09-10T01:37:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2023-09-10T01:37:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2023-09-10T01:37:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2023-09-10T01:37:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2023-09-10T01:37:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2023-09-10T01:37:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2023-09-10T01:37:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2023-09-10T01:37:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2023-09-10T01:37:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2023-09-10T01:37:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2023-09-10T01:37:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2023-09-10T01:37:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2023-09-10T01:37:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2023-09-10T01:37:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2023-09-10T01:37:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2023-09-10T01:37:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2023-09-10T01:37:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2023-09-10T01:37:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2023-09-10T01:37:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2023-09-10T01:37:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2023-09-10T01:37:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.059 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.059 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.059 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2023-09-10T01:37:24.059 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.059 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.060 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.060 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2023-09-10T01:37:24.060 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.060 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.060 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.061 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2023-09-10T01:37:24.061 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.061 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.061 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.062 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2023-09-10T01:37:24.062 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.062 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.062 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.063 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2023-09-10T01:37:24.063 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.063 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.063 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.064 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2023-09-10T01:37:24.064 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.064 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.064 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.065 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2023-09-10T01:37:24.065 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.065 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.065 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.066 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2023-09-10T01:37:24.066 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.066 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.066 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.067 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2023-09-10T01:37:24.067 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.067 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.067 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.068 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2023-09-10T01:37:24.068 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2023-09-10T01:37:24.068 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2023-09-10T01:37:24.068 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.068 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.069 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.069 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:24.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.069 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2023-09-10T01:37:24.069 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 2, 2023-09-10T01:37:24.069 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 2, 2023-09-10T01:37:24.070 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 0, 2023-09-10T01:37:24.070 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 2, 2023-09-10T01:37:24.070 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2023-09-10T01:37:24.070 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 0, 2023-09-10T01:37:24.070 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 0, 2023-09-10T01:37:24.071 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 4806, 2023-09-10T01:37:24.071 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2023-09-10T01:37:24.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.071 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2023-09-10T01:37:24.071 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2023-09-10T01:37:24.072 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2023-09-10T01:37:24.072 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2023-09-10T01:37:24.072 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2023-09-10T01:37:24.072 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2023-09-10T01:37:24.072 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2023-09-10T01:37:24.072 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-10T01:37:24.073 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2023-09-10T01:37:24.073 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-10T01:37:24.073 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2023-09-10T01:37:24.073 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2023-09-10T01:37:24.073 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2023-09-10T01:37:24.074 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-10T01:37:24.074 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2023-09-10T01:37:24.074 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2023-09-10T01:37:24.074 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2023-09-10T01:37:24.074 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2023-09-10T01:37:24.075 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2023-09-10T01:37:24.075 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-10T01:37:24.075 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2023-09-10T01:37:24.075 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-10T01:37:24.075 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2023-09-10T01:37:24.075 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2023-09-10T01:37:24.076 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2023-09-10T01:37:24.076 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2023-09-10T01:37:24.076 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2023-09-10T01:37:24.076 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-10T01:37:24.076 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2023-09-10T01:37:24.077 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2023-09-10T01:37:24.077 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2023-09-10T01:37:24.077 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2023-09-10T01:37:24.077 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2023-09-10T01:37:24.077 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2023-09-10T01:37:24.078 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2023-09-10T01:37:24.078 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2023-09-10T01:37:24.078 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2023-09-10T01:37:24.078 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 232145, 2023-09-10T01:37:24.078 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 77, 2023-09-10T01:37:24.078 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2023-09-10T01:37:24.079 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2023-09-10T01:37:24.079 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2023-09-10T01:37:24.079 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2023-09-10T01:37:24.079 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2023-09-10T01:37:24.079 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2023-09-10T01:37:24.080 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2023-09-10T01:37:24.080 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2023-09-10T01:37:24.080 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 9896, 2023-09-10T01:37:24.080 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 190, 2023-09-10T01:37:24.080 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2023-09-10T01:37:24.080 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2023-09-10T01:37:24.081 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2023-09-10T01:37:24.081 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2023-09-10T01:37:24.081 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 425707, 2023-09-10T01:37:24.081 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 16752, 2023-09-10T01:37:24.081 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2023-09-10T01:37:24.082 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2023-09-10T01:37:24.082 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2023-09-10T01:37:24.082 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2023-09-10T01:37:24.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.082 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2023-09-10T01:37:24.083 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2023-09-10T01:37:24.083 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2023-09-10T01:37:24.083 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 11239, 2023-09-10T01:37:24.083 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 21, 2023-09-10T01:37:24.083 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 3, 2023-09-10T01:37:24.084 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 11239, 2023-09-10T01:37:24.084 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2023-09-10T01:37:24.084 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 11239, 2023-09-10T01:37:24.084 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 40.868386963, 2023-09-10T01:37:24.084 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.003636301 2023-09-10T01:37:24.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.085 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2023-09-10T01:37:24.085 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2023-09-10T01:37:24.085 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 11239, 2023-09-10T01:37:24.085 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 11240 2023-09-10T01:37:24.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.086 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 11239, 2023-09-10T01:37:24.086 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 11239, 2023-09-10T01:37:24.086 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 0, 2023-09-10T01:37:24.086 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2023-09-10T01:37:24.086 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2023-09-10T01:37:24.086 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 6839, 2023-09-10T01:37:24.087 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 0, 2023-09-10T01:37:24.087 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 4398, 2023-09-10T01:37:24.087 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 0, 2023-09-10T01:37:24.087 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 0, 2023-09-10T01:37:24.087 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2023-09-10T01:37:24.088 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2023-09-10T01:37:24.088 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2023-09-10T01:37:24.088 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2023-09-10T01:37:24.088 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 0, 2023-09-10T01:37:24.088 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2023-09-10T01:37:24.088 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2023-09-10T01:37:24.089 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2023-09-10T01:37:24.089 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 1, 2023-09-10T01:37:24.089 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 0, 2023-09-10T01:37:24.089 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2023-09-10T01:37:24.089 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2023-09-10T01:37:24.089 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2023-09-10T01:37:24.090 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2023-09-10T01:37:24.090 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2023-09-10T01:37:24.090 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2023-09-10T01:37:24.090 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2023-09-10T01:37:24.091 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2023-09-10T01:37:24.091 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2023-09-10T01:37:24.091 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2023-09-10T01:37:24.091 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2023-09-10T01:37:24.091 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2023-09-10T01:37:24.092 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2023-09-10T01:37:24.092 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2023-09-10T01:37:24.092 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2023-09-10T01:37:24.092 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2023-09-10T01:37:24.092 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2023-09-10T01:37:24.092 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2023-09-10T01:37:24.093 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2023-09-10T01:37:24.093 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2023-09-10T01:37:24.093 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2023-09-10T01:37:24.093 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2023-09-10T01:37:24.093 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2023-09-10T01:37:24.093 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2023-09-10T01:37:24.094 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2023-09-10T01:37:24.094 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2023-09-10T01:37:24.094 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 213, 2023-09-10T01:37:24.094 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2023-09-10T01:37:24.094 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 126, 2023-09-10T01:37:24.095 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 10, 2023-09-10T01:37:24.095 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 20, 2023-09-10T01:37:24.095 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 10, 2023-09-10T01:37:24.095 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2023-09-10T01:37:24.095 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 0, 2023-09-10T01:37:24.095 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 2, 2023-09-10T01:37:24.096 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 0 2023-09-10T01:37:24.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.096 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2023-09-10T01:37:24.096 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 0, 2023-09-10T01:37:24.096 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2023-09-10T01:37:24.097 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2023-09-10T01:37:24.097 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2023-09-10T01:37:24.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.097 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2023-09-10T01:37:24.097 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2023-09-10T01:37:24.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.098 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2023-09-10T01:37:24.098 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2023-09-10T01:37:24.098 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 0, 2023-09-10T01:37:24.098 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2023-09-10T01:37:24.099 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 0, 2023-09-10T01:37:24.099 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 0, 2023-09-10T01:37:24.099 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2023-09-10T01:37:24.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.099 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-10T01:37:24.099 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2023-09-10T01:37:24.100 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2023-09-10T01:37:24.100 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2023-09-10T01:37:24.100 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 12500, 2023-09-10T01:37:24.100 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 76171293, 2023-09-10T01:37:24.100 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:24.101 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 12500, 2023-09-10T01:37:24.101 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2023-09-10T01:37:24.101 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2023-09-10T01:37:24.101 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 12500, 2023-09-10T01:37:24.101 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 76171293, 2023-09-10T01:37:24.102 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2023-09-10T01:37:24.102 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.102 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.102 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.102 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:24.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.103 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2023-09-10T01:37:24.103 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2023-09-10T01:37:24.103 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2023-09-10T01:37:24.103 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2023-09-10T01:37:24.103 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2023-09-10T01:37:24.104 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2023-09-10T01:37:24.104 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:24.104 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:24.104 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 11239, 2023-09-10T01:37:24.104 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 71135501, 2023-09-10T01:37:24.104 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 11239, 2023-09-10T01:37:24.105 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 71135501, 2023-09-10T01:37:24.105 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2023-09-10T01:37:24.105 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.105 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.105 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.106 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:24.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.106 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2023-09-10T01:37:24.106 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2023-09-10T01:37:24.106 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2023-09-10T01:37:24.107 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2023-09-10T01:37:24.107 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2023-09-10T01:37:24.107 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2023-09-10T01:37:24.107 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:24.107 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:24.107 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 11239, 2023-09-10T01:37:24.108 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 11239, 2023-09-10T01:37:24.108 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 11239, 2023-09-10T01:37:24.108 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 11239, 2023-09-10T01:37:24.108 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2023-09-10T01:37:24.108 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.109 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.109 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.109 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:24.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.109 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2023-09-10T01:37:24.109 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2023-09-10T01:37:24.110 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2023-09-10T01:37:24.110 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2023-09-10T01:37:24.110 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2023-09-10T01:37:24.110 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2023-09-10T01:37:24.110 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:24.111 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:24.111 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2023-09-10T01:37:24.111 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2023-09-10T01:37:24.111 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2023-09-10T01:37:24.111 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2023-09-10T01:37:24.111 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2023-09-10T01:37:24.112 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.112 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.112 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.112 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:24.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:24.113 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x55b0c3340960": { 2023-09-10T01:37:24.113 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2023-09-10T01:37:24.113 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2023-09-10T01:37:24.113 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2023-09-10T01:37:24.113 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2023-09-10T01:37:24.113 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2023-09-10T01:37:24.114 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:24.114 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:24.114 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2023-09-10T01:37:24.114 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2023-09-10T01:37:24.114 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2023-09-10T01:37:24.115 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2023-09-10T01:37:24.115 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2023-09-10T01:37:24.115 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:24.115 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:24.115 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:24.115 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:24.116 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:24.116 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T01:37:24.418 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-09-10T01:37:24.418 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-09-10T01:37:24.418 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2023-09-10T01:37:24.419 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2023-09-10T01:37:24.419 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2023-09-10T01:37:24.419 DEBUG:tasks.check_counter:Getting stats from j 2023-09-10T01:37:24.420 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.j perf dump 2023-09-10T01:37:24.993 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-10T01:37:25.164 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:24 smithi044 ceph-mon[98676]: pgmap v1850: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:37:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:24 smithi131 ceph-mon[103713]: pgmap v1850: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:37:25.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:24 smithi099 ceph-mon[102692]: pgmap v1850: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:37:26.216 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2023-09-10T01:37:26.217 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2023-09-10T01:37:26.217 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 951, 2023-09-10T01:37:26.217 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 938, 2023-09-10T01:37:26.217 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1763840, 2023-09-10T01:37:26.218 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 567680, 2023-09-10T01:37:26.218 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2023-09-10T01:37:26.218 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 0, 2023-09-10T01:37:26.218 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.128971504, 2023-09-10T01:37:26.218 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.049926636, 2023-09-10T01:37:26.218 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.054288916, 2023-09-10T01:37:26.219 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.022975247, 2023-09-10T01:37:26.219 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:26.219 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 938, 2023-09-10T01:37:26.219 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.142301837, 2023-09-10T01:37:26.219 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000151707 2023-09-10T01:37:26.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:26.220 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:26.220 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:26.220 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:26.220 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:26.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:26.221 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1763840, 2023-09-10T01:37:26.221 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 567680 2023-09-10T01:37:26.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:26.221 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2023-09-10T01:37:26.221 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 6, 2023-09-10T01:37:26.222 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 720, 2023-09-10T01:37:26.222 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 960, 2023-09-10T01:37:26.222 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 584992, 2023-09-10T01:37:26.222 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 9, 2023-09-10T01:37:26.222 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2023-09-10T01:37:26.223 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.045036581, 2023-09-10T01:37:26.223 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.034817882, 2023-09-10T01:37:26.223 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.003802219, 2023-09-10T01:37:26.223 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-10T01:37:26.223 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:26.224 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 716, 2023-09-10T01:37:26.224 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.110734649, 2023-09-10T01:37:26.224 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000154657 2023-09-10T01:37:26.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:26.224 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:26.225 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:26.225 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:26.225 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:26.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:26.225 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 960, 2023-09-10T01:37:26.225 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 584992 2023-09-10T01:37:26.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:26.226 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2023-09-10T01:37:26.226 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 0, 2023-09-10T01:37:26.226 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 2, 2023-09-10T01:37:26.226 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 0, 2023-09-10T01:37:26.227 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 0, 2023-09-10T01:37:26.227 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 2, 2023-09-10T01:37:26.227 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 1, 2023-09-10T01:37:26.227 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.000536047, 2023-09-10T01:37:26.227 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.000000000, 2023-09-10T01:37:26.228 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.000419309, 2023-09-10T01:37:26.228 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-10T01:37:26.228 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:26.228 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:26.228 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:26.229 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:26.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:26.229 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:26.229 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:26.229 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:26.229 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:26.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:26.230 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-10T01:37:26.230 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2023-09-10T01:37:26.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:26.230 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2023-09-10T01:37:26.231 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 0, 2023-09-10T01:37:26.231 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2023-09-10T01:37:26.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:26.231 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2023-09-10T01:37:26.231 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2023-09-10T01:37:26.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2023-09-10T01:37:26.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2023-09-10T01:37:26.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2023-09-10T01:37:26.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2023-09-10T01:37:26.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2023-09-10T01:37:26.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-10T01:37:26.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2023-09-10T01:37:26.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-10T01:37:26.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2023-09-10T01:37:26.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2023-09-10T01:37:26.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2023-09-10T01:37:26.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-10T01:37:26.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2023-09-10T01:37:26.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2023-09-10T01:37:26.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2023-09-10T01:37:26.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2023-09-10T01:37:26.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2023-09-10T01:37:26.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-10T01:37:26.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2023-09-10T01:37:26.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-10T01:37:26.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2023-09-10T01:37:26.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2023-09-10T01:37:26.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2023-09-10T01:37:26.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2023-09-10T01:37:26.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2023-09-10T01:37:26.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-10T01:37:26.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2023-09-10T01:37:26.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2023-09-10T01:37:26.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2023-09-10T01:37:26.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2023-09-10T01:37:26.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2023-09-10T01:37:26.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2023-09-10T01:37:26.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2023-09-10T01:37:26.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2023-09-10T01:37:26.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2023-09-10T01:37:26.238 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 148754, 2023-09-10T01:37:26.238 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 31, 2023-09-10T01:37:26.238 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2023-09-10T01:37:26.239 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2023-09-10T01:37:26.239 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2023-09-10T01:37:26.239 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2023-09-10T01:37:26.239 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2023-09-10T01:37:26.239 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2023-09-10T01:37:26.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2023-09-10T01:37:26.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2023-09-10T01:37:26.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 0, 2023-09-10T01:37:26.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 0, 2023-09-10T01:37:26.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2023-09-10T01:37:26.241 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2023-09-10T01:37:26.241 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2023-09-10T01:37:26.241 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2023-09-10T01:37:26.241 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 928, 2023-09-10T01:37:26.241 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 2, 2023-09-10T01:37:26.241 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2023-09-10T01:37:26.242 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2023-09-10T01:37:26.242 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2023-09-10T01:37:26.242 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2023-09-10T01:37:26.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2023-09-10T01:37:26.242 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-10T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2023-09-10T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2023-09-10T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2023-09-10T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 953, 2023-09-10T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1627051, 2023-09-10T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 953, 2023-09-10T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2023-09-10T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2023-09-10T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 953, 2023-09-10T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1627051, 2023-09-10T01:37:26.245 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2023-09-10T01:37:26.245 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2023-09-10T01:37:26.245 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2023-09-10T01:37:26.245 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2023-09-10T01:37:26.245 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:26.245 INFO:teuthology.orchestra.run.smithi044.stdout: } 2023-09-10T01:37:26.246 INFO:teuthology.orchestra.run.smithi044.stdout:} 2023-09-10T01:37:26.555 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-09-10T01:37:26.555 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-09-10T01:37:26.556 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-09-10T01:37:26.556 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-09-10T01:37:26.556 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-09-10T01:37:26.556 DEBUG:tasks.check_counter:Getting stats from b 2023-09-10T01:37:26.557 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.b perf dump 2023-09-10T01:37:27.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:26 smithi099 ceph-mon[102692]: pgmap v1851: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:37:27.275 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-10T01:37:27.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:26 smithi131 ceph-mon[103713]: pgmap v1851: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:37:27.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:26 smithi044 ceph-mon[98676]: pgmap v1851: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:37:28.687 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-09-10T01:37:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2023-09-10T01:37:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 279113, 2023-09-10T01:37:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 284606, 2023-09-10T01:37:28.688 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 56605056, 2023-09-10T01:37:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 81398666, 2023-09-10T01:37:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 251, 2023-09-10T01:37:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 7, 2023-09-10T01:37:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 9.770442472, 2023-09-10T01:37:28.689 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 3.715303544, 2023-09-10T01:37:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 6.520542617, 2023-09-10T01:37:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.358867923, 2023-09-10T01:37:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 284603, 2023-09-10T01:37:28.690 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 9.798525061, 2023-09-10T01:37:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000034428 2023-09-10T01:37:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 548883, 2023-09-10T01:37:28.691 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.062806646, 2023-09-10T01:37:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000114 2023-09-10T01:37:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3192384, 2023-09-10T01:37:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 1400240 2023-09-10T01:37:28.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2023-09-10T01:37:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 263070, 2023-09-10T01:37:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 262231, 2023-09-10T01:37:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 50587078, 2023-09-10T01:37:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 77468428, 2023-09-10T01:37:28.693 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 19, 2023-09-10T01:37:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 7, 2023-09-10T01:37:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 9.965758954, 2023-09-10T01:37:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 4.317441196, 2023-09-10T01:37:28.694 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 6.572013555, 2023-09-10T01:37:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.354490479, 2023-09-10T01:37:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 262228, 2023-09-10T01:37:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 5.412247742, 2023-09-10T01:37:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000020639 2023-09-10T01:37:28.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 514387, 2023-09-10T01:37:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.056371178, 2023-09-10T01:37:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000109 2023-09-10T01:37:28.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1877424, 2023-09-10T01:37:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 789120 2023-09-10T01:37:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2023-09-10T01:37:28.697 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 128606, 2023-09-10T01:37:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 128573, 2023-09-10T01:37:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 34855185, 2023-09-10T01:37:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 57837087, 2023-09-10T01:37:28.698 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 357, 2023-09-10T01:37:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 6, 2023-09-10T01:37:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 5.871614468, 2023-09-10T01:37:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 2.428157213, 2023-09-10T01:37:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 3.877554403, 2023-09-10T01:37:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.254904808, 2023-09-10T01:37:28.699 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 128574, 2023-09-10T01:37:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 7.241320138, 2023-09-10T01:37:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000056320 2023-09-10T01:37:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.700 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 241389, 2023-09-10T01:37:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.040938454, 2023-09-10T01:37:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000169 2023-09-10T01:37:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.701 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 46208, 2023-09-10T01:37:28.702 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 668464 2023-09-10T01:37:28.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.702 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2023-09-10T01:37:28.702 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2023-09-10T01:37:28.702 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2023-09-10T01:37:28.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.703 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2023-09-10T01:37:28.703 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2023-09-10T01:37:28.703 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2023-09-10T01:37:28.703 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 15773, 2023-09-10T01:37:28.703 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 5.915475045, 2023-09-10T01:37:28.704 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000375038 2023-09-10T01:37:28.704 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:28.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.704 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2023-09-10T01:37:28.704 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2023-09-10T01:37:28.705 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2023-09-10T01:37:28.705 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 193, 2023-09-10T01:37:28.705 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.008753227, 2023-09-10T01:37:28.705 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000045353 2023-09-10T01:37:28.705 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:28.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.706 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2023-09-10T01:37:28.706 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 14368, 2023-09-10T01:37:28.706 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 10902, 2023-09-10T01:37:28.706 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2023-09-10T01:37:28.706 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 10902, 2023-09-10T01:37:28.707 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 689.651984134, 2023-09-10T01:37:28.707 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.063259217 2023-09-10T01:37:28.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.707 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2023-09-10T01:37:28.707 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 3466, 2023-09-10T01:37:28.707 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 10, 2023-09-10T01:37:28.708 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 2, 2023-09-10T01:37:28.708 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 1838, 2023-09-10T01:37:28.708 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 4, 2023-09-10T01:37:28.708 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2023-09-10T01:37:28.708 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 200, 2023-09-10T01:37:28.708 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 87, 2023-09-10T01:37:28.709 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 101, 2023-09-10T01:37:28.709 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 12, 2023-09-10T01:37:28.709 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 67, 2023-09-10T01:37:28.709 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 2778, 2023-09-10T01:37:28.709 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 56, 2023-09-10T01:37:28.709 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 56, 2023-09-10T01:37:28.710 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 21, 2023-09-10T01:37:28.710 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 34605, 2023-09-10T01:37:28.710 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 27904, 2023-09-10T01:37:28.710 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 3354, 2023-09-10T01:37:28.710 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 225, 2023-09-10T01:37:28.711 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 2, 2023-09-10T01:37:28.711 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2023-09-10T01:37:28.711 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 2068, 2023-09-10T01:37:28.711 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 389, 2023-09-10T01:37:28.711 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2023-09-10T01:37:28.711 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 81, 2023-09-10T01:37:28.712 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 2340, 2023-09-10T01:37:28.712 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 43, 2023-09-10T01:37:28.712 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 1506, 2023-09-10T01:37:28.712 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2023-09-10T01:37:28.712 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 1061, 2023-09-10T01:37:28.712 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2023-09-10T01:37:28.713 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 1045, 2023-09-10T01:37:28.713 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 8977398359, 2023-09-10T01:37:28.713 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 7, 2023-09-10T01:37:28.713 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 1061, 2023-09-10T01:37:28.713 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2023-09-10T01:37:28.713 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2023-09-10T01:37:28.714 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2023-09-10T01:37:28.714 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 71, 2023-09-10T01:37:28.714 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 3, 2023-09-10T01:37:28.714 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 68, 2023-09-10T01:37:28.714 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 990, 2023-09-10T01:37:28.715 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 414, 2023-09-10T01:37:28.715 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 7891, 2023-09-10T01:37:28.715 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 4559, 2023-09-10T01:37:28.715 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2023-09-10T01:37:28.715 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-10T01:37:28.715 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-10T01:37:28.716 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 13738, 2023-09-10T01:37:28.716 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 3120, 2023-09-10T01:37:28.716 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 138, 2023-09-10T01:37:28.716 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 1371 2023-09-10T01:37:28.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.716 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2023-09-10T01:37:28.717 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2023-09-10T01:37:28.717 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2023-09-10T01:37:28.717 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2023-09-10T01:37:28.717 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 254, 2023-09-10T01:37:28.717 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 232, 2023-09-10T01:37:28.718 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2023-09-10T01:37:28.718 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2023-09-10T01:37:28.718 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2023-09-10T01:37:28.718 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 105, 2023-09-10T01:37:28.719 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 40, 2023-09-10T01:37:28.719 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 351, 2023-09-10T01:37:28.719 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 51, 2023-09-10T01:37:28.719 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2023-09-10T01:37:28.719 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2023-09-10T01:37:28.719 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2023-09-10T01:37:28.720 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2023-09-10T01:37:28.720 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2023-09-10T01:37:28.720 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 3, 2023-09-10T01:37:28.720 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 87, 2023-09-10T01:37:28.720 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2023-09-10T01:37:28.721 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 4, 2023-09-10T01:37:28.721 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2023-09-10T01:37:28.721 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2023-09-10T01:37:28.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.721 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2023-09-10T01:37:28.722 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2023-09-10T01:37:28.722 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 12845, 2023-09-10T01:37:28.722 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 12803, 2023-09-10T01:37:28.722 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 12803, 2023-09-10T01:37:28.722 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 42, 2023-09-10T01:37:28.722 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2023-09-10T01:37:28.723 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 179, 2023-09-10T01:37:28.723 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 270, 2023-09-10T01:37:28.723 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 269, 2023-09-10T01:37:28.723 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 269, 2023-09-10T01:37:28.723 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 1, 2023-09-10T01:37:28.724 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 1, 2023-09-10T01:37:28.724 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2023-09-10T01:37:28.724 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 2, 2023-09-10T01:37:28.724 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 58947428, 2023-09-10T01:37:28.724 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 59090561, 2023-09-10T01:37:28.725 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 0, 2023-09-10T01:37:28.725 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2023-09-10T01:37:28.725 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5685, 2023-09-10T01:37:28.725 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 35.193285249, 2023-09-10T01:37:28.725 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.006190551 2023-09-10T01:37:28.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.726 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 0 2023-09-10T01:37:28.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.726 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2023-09-10T01:37:28.726 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 119, 2023-09-10T01:37:28.726 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 2503, 2023-09-10T01:37:28.727 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 2384, 2023-09-10T01:37:28.727 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 98, 2023-09-10T01:37:28.727 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 195, 2023-09-10T01:37:28.727 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 97, 2023-09-10T01:37:28.727 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 200, 2023-09-10T01:37:28.727 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 3434, 2023-09-10T01:37:28.728 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 3234, 2023-09-10T01:37:28.728 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 56, 2023-09-10T01:37:28.728 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 2084, 2023-09-10T01:37:28.728 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 2028, 2023-09-10T01:37:28.728 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 77612, 2023-09-10T01:37:28.729 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 207152 2023-09-10T01:37:28.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.729 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2023-09-10T01:37:28.729 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 28734, 2023-09-10T01:37:28.729 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 5877, 2023-09-10T01:37:28.729 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 14368, 2023-09-10T01:37:28.730 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 220, 2023-09-10T01:37:28.730 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 11241, 2023-09-10T01:37:28.730 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2023-09-10T01:37:28.730 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1014, 2023-09-10T01:37:28.730 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 5.508992178, 2023-09-10T01:37:28.730 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.005432931 2023-09-10T01:37:28.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.731 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2023-09-10T01:37:28.731 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1355, 2023-09-10T01:37:28.731 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.156621687, 2023-09-10T01:37:28.731 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000853595 2023-09-10T01:37:28.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.732 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2023-09-10T01:37:28.732 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.732 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.732 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.733 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2023-09-10T01:37:28.733 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 56, 2023-09-10T01:37:28.733 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.579167477, 2023-09-10T01:37:28.733 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.028199419 2023-09-10T01:37:28.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.733 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2023-09-10T01:37:28.734 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5469, 2023-09-10T01:37:28.734 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 523.202906999, 2023-09-10T01:37:28.734 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.095667015 2023-09-10T01:37:28.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.734 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2023-09-10T01:37:28.735 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.735 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.735 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.737 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2023-09-10T01:37:28.737 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.737 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.737 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.738 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2023-09-10T01:37:28.738 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.738 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.738 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.739 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2023-09-10T01:37:28.739 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.739 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.739 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.740 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2023-09-10T01:37:28.740 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.740 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.740 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.740 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2023-09-10T01:37:28.741 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 57, 2023-09-10T01:37:28.741 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.011319560, 2023-09-10T01:37:28.741 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000198588 2023-09-10T01:37:28.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.741 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2023-09-10T01:37:28.741 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 34, 2023-09-10T01:37:28.742 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.041356075, 2023-09-10T01:37:28.742 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001216355 2023-09-10T01:37:28.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.742 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2023-09-10T01:37:28.742 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.743 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.743 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.743 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2023-09-10T01:37:28.743 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 57, 2023-09-10T01:37:28.743 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 19.925048993, 2023-09-10T01:37:28.744 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.349562263 2023-09-10T01:37:28.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.744 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2023-09-10T01:37:28.744 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2220, 2023-09-10T01:37:28.744 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 5.118726291, 2023-09-10T01:37:28.744 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002305732 2023-09-10T01:37:28.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.745 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2023-09-10T01:37:28.745 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 74, 2023-09-10T01:37:28.745 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.122045298, 2023-09-10T01:37:28.745 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001649260 2023-09-10T01:37:28.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.746 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2023-09-10T01:37:28.746 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 422, 2023-09-10T01:37:28.746 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 124.582885961, 2023-09-10T01:37:28.746 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.295220108 2023-09-10T01:37:28.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.747 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2023-09-10T01:37:28.747 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.747 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.747 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.748 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2023-09-10T01:37:28.748 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.748 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.748 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.748 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2023-09-10T01:37:28.749 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.749 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.749 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.749 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2023-09-10T01:37:28.749 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 50, 2023-09-10T01:37:28.750 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 7.498172570, 2023-09-10T01:37:28.750 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.149963451 2023-09-10T01:37:28.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.750 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2023-09-10T01:37:28.750 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.751 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.751 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.751 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2023-09-10T01:37:28.751 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2, 2023-09-10T01:37:28.751 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.001672464, 2023-09-10T01:37:28.752 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000836232 2023-09-10T01:37:28.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.752 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2023-09-10T01:37:28.752 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.752 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.752 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.753 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2023-09-10T01:37:28.753 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.753 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.753 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.754 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2023-09-10T01:37:28.754 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.754 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.754 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.755 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2023-09-10T01:37:28.755 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 7, 2023-09-10T01:37:28.755 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.019250491, 2023-09-10T01:37:28.755 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002750070 2023-09-10T01:37:28.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.756 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2023-09-10T01:37:28.756 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.756 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.756 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.757 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2023-09-10T01:37:28.757 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 85, 2023-09-10T01:37:28.757 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.883818090, 2023-09-10T01:37:28.757 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.010397859 2023-09-10T01:37:28.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.757 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2023-09-10T01:37:28.758 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2023-09-10T01:37:28.758 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2023-09-10T01:37:28.758 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.758 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.758 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.759 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:28.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.759 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2023-09-10T01:37:28.759 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 2, 2023-09-10T01:37:28.759 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 6, 2023-09-10T01:37:28.760 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 4, 2023-09-10T01:37:28.760 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 2, 2023-09-10T01:37:28.760 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2023-09-10T01:37:28.760 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 367, 2023-09-10T01:37:28.760 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 183, 2023-09-10T01:37:28.760 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 3407, 2023-09-10T01:37:28.761 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2023-09-10T01:37:28.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.761 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2023-09-10T01:37:28.761 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2023-09-10T01:37:28.761 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2023-09-10T01:37:28.762 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2023-09-10T01:37:28.762 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2023-09-10T01:37:28.762 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2023-09-10T01:37:28.762 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2023-09-10T01:37:28.762 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-10T01:37:28.763 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2023-09-10T01:37:28.763 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-10T01:37:28.763 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2023-09-10T01:37:28.763 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2023-09-10T01:37:28.763 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2023-09-10T01:37:28.763 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-10T01:37:28.764 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2023-09-10T01:37:28.764 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2023-09-10T01:37:28.764 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2023-09-10T01:37:28.764 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2023-09-10T01:37:28.764 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2023-09-10T01:37:28.764 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-10T01:37:28.765 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2023-09-10T01:37:28.765 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-10T01:37:28.765 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2023-09-10T01:37:28.765 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2023-09-10T01:37:28.765 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2023-09-10T01:37:28.766 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2023-09-10T01:37:28.766 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2023-09-10T01:37:28.766 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-10T01:37:28.766 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2023-09-10T01:37:28.766 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2023-09-10T01:37:28.767 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2023-09-10T01:37:28.767 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2023-09-10T01:37:28.767 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2023-09-10T01:37:28.767 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2023-09-10T01:37:28.767 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2023-09-10T01:37:28.767 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2023-09-10T01:37:28.768 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2023-09-10T01:37:28.768 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 718365, 2023-09-10T01:37:28.768 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 219, 2023-09-10T01:37:28.768 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2023-09-10T01:37:28.768 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2023-09-10T01:37:28.768 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2023-09-10T01:37:28.769 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2023-09-10T01:37:28.769 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2023-09-10T01:37:28.769 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2023-09-10T01:37:28.769 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2023-09-10T01:37:28.769 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2023-09-10T01:37:28.770 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 9896, 2023-09-10T01:37:28.770 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 190, 2023-09-10T01:37:28.770 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2023-09-10T01:37:28.770 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2023-09-10T01:37:28.770 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2023-09-10T01:37:28.771 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2023-09-10T01:37:28.771 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 964912, 2023-09-10T01:37:28.771 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 30730, 2023-09-10T01:37:28.771 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2023-09-10T01:37:28.771 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2023-09-10T01:37:28.771 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2023-09-10T01:37:28.772 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2023-09-10T01:37:28.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.772 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2023-09-10T01:37:28.772 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2023-09-10T01:37:28.772 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2023-09-10T01:37:28.772 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 13072, 2023-09-10T01:37:28.773 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 62967374, 2023-09-10T01:37:28.773 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 37, 2023-09-10T01:37:28.773 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 13072, 2023-09-10T01:37:28.773 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2023-09-10T01:37:28.773 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 13072, 2023-09-10T01:37:28.773 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 295.455937810, 2023-09-10T01:37:28.774 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.022602198 2023-09-10T01:37:28.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.774 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2023-09-10T01:37:28.774 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2023-09-10T01:37:28.774 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 13072, 2023-09-10T01:37:28.775 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 20072 2023-09-10T01:37:28.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.775 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 13072, 2023-09-10T01:37:28.775 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 1124, 2023-09-10T01:37:28.775 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 11948, 2023-09-10T01:37:28.776 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2023-09-10T01:37:28.776 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2023-09-10T01:37:28.776 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 1760, 2023-09-10T01:37:28.776 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 2220, 2023-09-10T01:37:28.776 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 28, 2023-09-10T01:37:28.776 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 5710, 2023-09-10T01:37:28.777 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 891, 2023-09-10T01:37:28.777 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2023-09-10T01:37:28.777 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2023-09-10T01:37:28.777 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2023-09-10T01:37:28.777 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 3, 2023-09-10T01:37:28.777 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 619, 2023-09-10T01:37:28.777 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2023-09-10T01:37:28.778 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2023-09-10T01:37:28.778 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2023-09-10T01:37:28.778 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 1072, 2023-09-10T01:37:28.778 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 3386, 2023-09-10T01:37:28.778 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2023-09-10T01:37:28.779 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2023-09-10T01:37:28.779 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2023-09-10T01:37:28.779 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2023-09-10T01:37:28.779 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2023-09-10T01:37:28.779 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2023-09-10T01:37:28.779 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2023-09-10T01:37:28.780 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2023-09-10T01:37:28.780 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2023-09-10T01:37:28.780 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2023-09-10T01:37:28.780 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2023-09-10T01:37:28.780 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2023-09-10T01:37:28.780 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2023-09-10T01:37:28.781 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2023-09-10T01:37:28.781 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2023-09-10T01:37:28.781 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2023-09-10T01:37:28.781 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2023-09-10T01:37:28.781 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2023-09-10T01:37:28.781 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2023-09-10T01:37:28.782 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2023-09-10T01:37:28.782 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2023-09-10T01:37:28.782 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2023-09-10T01:37:28.782 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2023-09-10T01:37:28.782 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2023-09-10T01:37:28.783 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2023-09-10T01:37:28.783 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2023-09-10T01:37:28.783 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 213, 2023-09-10T01:37:28.783 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2023-09-10T01:37:28.783 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 131, 2023-09-10T01:37:28.783 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 12, 2023-09-10T01:37:28.784 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 12, 2023-09-10T01:37:28.784 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2023-09-10T01:37:28.784 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2023-09-10T01:37:28.784 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 4107, 2023-09-10T01:37:28.784 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 44, 2023-09-10T01:37:28.785 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 232 2023-09-10T01:37:28.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.785 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2023-09-10T01:37:28.785 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 1, 2023-09-10T01:37:28.785 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2023-09-10T01:37:28.785 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2023-09-10T01:37:28.786 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2023-09-10T01:37:28.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.786 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2023-09-10T01:37:28.786 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2023-09-10T01:37:28.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.787 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2023-09-10T01:37:28.787 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2023-09-10T01:37:28.787 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 10, 2023-09-10T01:37:28.787 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2023-09-10T01:37:28.787 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 5, 2023-09-10T01:37:28.787 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 40, 2023-09-10T01:37:28.788 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2023-09-10T01:37:28.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.788 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-10T01:37:28.788 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:28.788 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-09-10T01:37:28.788 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:28.789 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 670785, 2023-09-10T01:37:28.789 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 81398019, 2023-09-10T01:37:28.789 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:28.789 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 670785, 2023-09-10T01:37:28.789 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-09-10T01:37:28.789 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-09-10T01:37:28.790 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 670785, 2023-09-10T01:37:28.790 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 81398019, 2023-09-10T01:37:28.790 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:28.790 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.790 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.791 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:28.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.791 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2023-09-10T01:37:28.791 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:28.791 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-09-10T01:37:28.792 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:28.792 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-09-10T01:37:28.792 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-09-10T01:37:28.792 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:28.792 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:28.793 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 13072, 2023-09-10T01:37:28.793 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 62971141, 2023-09-10T01:37:28.793 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 13072, 2023-09-10T01:37:28.793 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 62971141, 2023-09-10T01:37:28.793 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:28.793 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.794 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.794 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.794 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:28.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.794 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2023-09-10T01:37:28.795 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:28.795 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2023-09-10T01:37:28.795 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:28.795 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-09-10T01:37:28.795 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-09-10T01:37:28.796 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:28.796 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:28.796 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 13072, 2023-09-10T01:37:28.796 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 13072, 2023-09-10T01:37:28.796 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 13072, 2023-09-10T01:37:28.796 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 13072, 2023-09-10T01:37:28.797 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:28.797 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.797 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.797 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.797 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:28.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.798 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2023-09-10T01:37:28.798 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:28.798 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2023-09-10T01:37:28.798 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:28.798 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 40, 2023-09-10T01:37:28.799 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 4576, 2023-09-10T01:37:28.799 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:28.799 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 40, 2023-09-10T01:37:28.799 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-09-10T01:37:28.799 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-09-10T01:37:28.799 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 25, 2023-09-10T01:37:28.800 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 4576, 2023-09-10T01:37:28.800 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:28.800 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.800 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.800 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:28.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:28.801 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x55f1e67526e0": { 2023-09-10T01:37:28.801 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:28.801 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2023-09-10T01:37:28.801 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:28.802 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 12845, 2023-09-10T01:37:28.802 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 54896257, 2023-09-10T01:37:28.802 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:28.802 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 12845, 2023-09-10T01:37:28.802 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-09-10T01:37:28.803 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-09-10T01:37:28.803 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 5685, 2023-09-10T01:37:28.803 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 54896257, 2023-09-10T01:37:28.803 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:28.803 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:28.804 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:28.804 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:28.804 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:28.804 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:28.804 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-09-10T01:37:28.978 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:28 smithi099 ceph-mon[102692]: pgmap v1852: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T01:37:29.024 INFO:tasks.check_counter:Daemon mds.b mds.exported=81 2023-09-10T01:37:29.024 INFO:tasks.check_counter:Daemon mds.b mds.imported=43 2023-09-10T01:37:29.025 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2023-09-10T01:37:29.025 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=57 2023-09-10T01:37:29.025 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=50 2023-09-10T01:37:29.025 DEBUG:tasks.check_counter:Getting stats from e 2023-09-10T01:37:29.026 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.e perf dump 2023-09-10T01:37:29.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:28 smithi131 ceph-mon[103713]: pgmap v1852: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T01:37:29.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:28 smithi044 ceph-mon[98676]: pgmap v1852: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T01:37:29.605 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-10T01:37:30.793 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-09-10T01:37:30.794 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2023-09-10T01:37:30.794 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 5279, 2023-09-10T01:37:30.794 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 5281, 2023-09-10T01:37:30.794 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 39090921, 2023-09-10T01:37:30.795 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 1641833, 2023-09-10T01:37:30.795 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 9, 2023-09-10T01:37:30.795 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2023-09-10T01:37:30.795 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.333786985, 2023-09-10T01:37:30.795 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.077919556, 2023-09-10T01:37:30.796 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.098840136, 2023-09-10T01:37:30.796 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.131160843, 2023-09-10T01:37:30.796 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:30.796 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5279, 2023-09-10T01:37:30.796 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.194859297, 2023-09-10T01:37:30.797 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000036912 2023-09-10T01:37:30.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.797 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:30.797 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.797 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.798 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.798 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-10T01:37:30.798 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2023-09-10T01:37:30.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.798 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2023-09-10T01:37:30.799 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1644, 2023-09-10T01:37:30.799 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2355, 2023-09-10T01:37:30.799 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 40966311, 2023-09-10T01:37:30.799 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 2172602, 2023-09-10T01:37:30.799 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 17, 2023-09-10T01:37:30.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 6, 2023-09-10T01:37:30.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.167389247, 2023-09-10T01:37:30.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.047448148, 2023-09-10T01:37:30.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.056240351, 2023-09-10T01:37:30.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.048798105, 2023-09-10T01:37:30.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:30.801 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2353, 2023-09-10T01:37:30.801 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.245684883, 2023-09-10T01:37:30.801 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000104413 2023-09-10T01:37:30.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.801 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:30.802 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.802 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.802 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.802 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3712, 2023-09-10T01:37:30.803 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 1664112 2023-09-10T01:37:30.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.803 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2023-09-10T01:37:30.803 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 5877, 2023-09-10T01:37:30.803 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 5681, 2023-09-10T01:37:30.803 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 42313428, 2023-09-10T01:37:30.804 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 2025194, 2023-09-10T01:37:30.804 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 8, 2023-09-10T01:37:30.804 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2023-09-10T01:37:30.804 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.530139436, 2023-09-10T01:37:30.804 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.146200550, 2023-09-10T01:37:30.805 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.187039657, 2023-09-10T01:37:30.805 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.169714215, 2023-09-10T01:37:30.805 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:30.805 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5680, 2023-09-10T01:37:30.805 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.610467037, 2023-09-10T01:37:30.805 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000107476 2023-09-10T01:37:30.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.806 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:30.806 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.806 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.806 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.807 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3107872, 2023-09-10T01:37:30.807 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 590240 2023-09-10T01:37:30.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.808 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2023-09-10T01:37:30.808 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2023-09-10T01:37:30.808 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2023-09-10T01:37:30.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.808 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2023-09-10T01:37:30.808 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2023-09-10T01:37:30.809 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2023-09-10T01:37:30.809 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 19305, 2023-09-10T01:37:30.809 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.459416422, 2023-09-10T01:37:30.809 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000023797 2023-09-10T01:37:30.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:30.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.810 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2023-09-10T01:37:30.810 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2023-09-10T01:37:30.810 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2023-09-10T01:37:30.810 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.811 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.811 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:30.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.811 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2023-09-10T01:37:30.812 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 0, 2023-09-10T01:37:30.812 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 0, 2023-09-10T01:37:30.812 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2023-09-10T01:37:30.812 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.812 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.812 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.813 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2023-09-10T01:37:30.813 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 0, 2023-09-10T01:37:30.813 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2023-09-10T01:37:30.813 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2023-09-10T01:37:30.813 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 0, 2023-09-10T01:37:30.814 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2023-09-10T01:37:30.814 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2023-09-10T01:37:30.814 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 762, 2023-09-10T01:37:30.814 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 36, 2023-09-10T01:37:30.815 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 212, 2023-09-10T01:37:30.815 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 514, 2023-09-10T01:37:30.815 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 710, 2023-09-10T01:37:30.815 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 11, 2023-09-10T01:37:30.815 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2023-09-10T01:37:30.815 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2023-09-10T01:37:30.816 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 8, 2023-09-10T01:37:30.816 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 0, 2023-09-10T01:37:30.816 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 0, 2023-09-10T01:37:30.816 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2023-09-10T01:37:30.816 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 0, 2023-09-10T01:37:30.816 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2023-09-10T01:37:30.817 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2023-09-10T01:37:30.817 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2023-09-10T01:37:30.817 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2023-09-10T01:37:30.817 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2023-09-10T01:37:30.817 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2023-09-10T01:37:30.818 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2023-09-10T01:37:30.818 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2023-09-10T01:37:30.818 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2023-09-10T01:37:30.818 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2023-09-10T01:37:30.818 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2023-09-10T01:37:30.819 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2023-09-10T01:37:30.819 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 0, 2023-09-10T01:37:30.819 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 0, 2023-09-10T01:37:30.819 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 0, 2023-09-10T01:37:30.820 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2023-09-10T01:37:30.820 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2023-09-10T01:37:30.820 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2023-09-10T01:37:30.820 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2023-09-10T01:37:30.820 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2023-09-10T01:37:30.821 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2023-09-10T01:37:30.821 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2023-09-10T01:37:30.821 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2023-09-10T01:37:30.821 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2023-09-10T01:37:30.821 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 0, 2023-09-10T01:37:30.821 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2023-09-10T01:37:30.822 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2023-09-10T01:37:30.822 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-10T01:37:30.822 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-10T01:37:30.822 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 0, 2023-09-10T01:37:30.823 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2023-09-10T01:37:30.823 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2023-09-10T01:37:30.823 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2023-09-10T01:37:30.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.823 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2023-09-10T01:37:30.824 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2023-09-10T01:37:30.824 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2023-09-10T01:37:30.824 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2023-09-10T01:37:30.824 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 0, 2023-09-10T01:37:30.824 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2023-09-10T01:37:30.825 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2023-09-10T01:37:30.825 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2023-09-10T01:37:30.825 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2023-09-10T01:37:30.825 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2023-09-10T01:37:30.825 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2023-09-10T01:37:30.826 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2023-09-10T01:37:30.826 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2023-09-10T01:37:30.826 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2023-09-10T01:37:30.826 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2023-09-10T01:37:30.826 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2023-09-10T01:37:30.827 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2023-09-10T01:37:30.827 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2023-09-10T01:37:30.827 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2023-09-10T01:37:30.827 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2023-09-10T01:37:30.827 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2023-09-10T01:37:30.827 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2023-09-10T01:37:30.828 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2023-09-10T01:37:30.828 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2023-09-10T01:37:30.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.828 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2023-09-10T01:37:30.828 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2023-09-10T01:37:30.829 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 0, 2023-09-10T01:37:30.829 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2023-09-10T01:37:30.829 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2023-09-10T01:37:30.829 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 22236, 2023-09-10T01:37:30.829 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2023-09-10T01:37:30.829 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2023-09-10T01:37:30.830 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 0, 2023-09-10T01:37:30.830 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2023-09-10T01:37:30.830 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2023-09-10T01:37:30.830 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 34, 2023-09-10T01:37:30.830 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 3, 2023-09-10T01:37:30.831 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2023-09-10T01:37:30.831 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2023-09-10T01:37:30.831 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2023-09-10T01:37:30.831 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 120239536, 2023-09-10T01:37:30.831 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 120238962, 2023-09-10T01:37:30.831 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2023-09-10T01:37:30.832 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.832 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.832 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.832 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 22236 2023-09-10T01:37:30.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.833 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2023-09-10T01:37:30.833 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 152, 2023-09-10T01:37:30.833 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 2597, 2023-09-10T01:37:30.833 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 2445, 2023-09-10T01:37:30.833 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 26, 2023-09-10T01:37:30.834 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 122, 2023-09-10T01:37:30.834 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 96, 2023-09-10T01:37:30.834 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 762, 2023-09-10T01:37:30.834 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 3180, 2023-09-10T01:37:30.834 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 2418, 2023-09-10T01:37:30.835 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2023-09-10T01:37:30.835 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 0, 2023-09-10T01:37:30.835 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 0, 2023-09-10T01:37:30.835 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 55436, 2023-09-10T01:37:30.835 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 207152 2023-09-10T01:37:30.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.836 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2023-09-10T01:37:30.836 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 0, 2023-09-10T01:37:30.836 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2023-09-10T01:37:30.836 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 0, 2023-09-10T01:37:30.837 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 0, 2023-09-10T01:37:30.837 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2023-09-10T01:37:30.837 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2023-09-10T01:37:30.837 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.837 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.837 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.838 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2023-09-10T01:37:30.838 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.838 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.839 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.839 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2023-09-10T01:37:30.839 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.839 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.839 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.840 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2023-09-10T01:37:30.840 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.840 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.840 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.841 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2023-09-10T01:37:30.841 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.841 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.841 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.842 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2023-09-10T01:37:30.842 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.842 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.842 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.842 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2023-09-10T01:37:30.843 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.843 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.843 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.843 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2023-09-10T01:37:30.844 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.844 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.844 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.844 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2023-09-10T01:37:30.844 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.845 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.845 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.845 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2023-09-10T01:37:30.845 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.846 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.846 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.846 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2023-09-10T01:37:30.846 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.846 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.847 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.847 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2023-09-10T01:37:30.847 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.847 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.848 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.848 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2023-09-10T01:37:30.848 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.848 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.848 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.849 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2023-09-10T01:37:30.849 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.849 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.849 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.850 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2023-09-10T01:37:30.850 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.850 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.850 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.851 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2023-09-10T01:37:30.851 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.851 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.851 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.852 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2023-09-10T01:37:30.852 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.852 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.852 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.852 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2023-09-10T01:37:30.853 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.853 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.853 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.853 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2023-09-10T01:37:30.854 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.854 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.854 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.854 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2023-09-10T01:37:30.854 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.855 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.855 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.855 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2023-09-10T01:37:30.855 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.856 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.856 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.856 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2023-09-10T01:37:30.856 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.856 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.857 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.857 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2023-09-10T01:37:30.857 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.857 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.858 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.858 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2023-09-10T01:37:30.858 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.858 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.858 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.859 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2023-09-10T01:37:30.859 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.859 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.859 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.860 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2023-09-10T01:37:30.860 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.860 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.860 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.861 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2023-09-10T01:37:30.861 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.861 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.861 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.861 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2023-09-10T01:37:30.862 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.862 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.862 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.862 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2023-09-10T01:37:30.863 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.863 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.863 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.863 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2023-09-10T01:37:30.864 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2023-09-10T01:37:30.864 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2023-09-10T01:37:30.864 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.864 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.864 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:30.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.865 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2023-09-10T01:37:30.865 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 2, 2023-09-10T01:37:30.865 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 4, 2023-09-10T01:37:30.865 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 2, 2023-09-10T01:37:30.866 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 3, 2023-09-10T01:37:30.866 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2023-09-10T01:37:30.866 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 0, 2023-09-10T01:37:30.866 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 0, 2023-09-10T01:37:30.866 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 5510, 2023-09-10T01:37:30.866 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2023-09-10T01:37:30.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.867 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2023-09-10T01:37:30.867 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2023-09-10T01:37:30.867 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2023-09-10T01:37:30.868 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2023-09-10T01:37:30.868 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2023-09-10T01:37:30.868 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2023-09-10T01:37:30.868 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2023-09-10T01:37:30.868 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-10T01:37:30.869 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2023-09-10T01:37:30.869 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-10T01:37:30.869 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2023-09-10T01:37:30.869 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2023-09-10T01:37:30.869 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2023-09-10T01:37:30.869 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-10T01:37:30.870 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2023-09-10T01:37:30.870 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2023-09-10T01:37:30.870 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2023-09-10T01:37:30.870 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2023-09-10T01:37:30.870 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2023-09-10T01:37:30.871 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-10T01:37:30.871 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2023-09-10T01:37:30.871 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-10T01:37:30.871 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2023-09-10T01:37:30.871 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2023-09-10T01:37:30.871 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2023-09-10T01:37:30.872 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2023-09-10T01:37:30.872 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2023-09-10T01:37:30.872 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-10T01:37:30.872 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2023-09-10T01:37:30.873 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2023-09-10T01:37:30.873 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2023-09-10T01:37:30.873 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2023-09-10T01:37:30.873 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2023-09-10T01:37:30.873 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2023-09-10T01:37:30.874 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2023-09-10T01:37:30.874 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2023-09-10T01:37:30.874 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2023-09-10T01:37:30.874 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 224839, 2023-09-10T01:37:30.874 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 74, 2023-09-10T01:37:30.874 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2023-09-10T01:37:30.875 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2023-09-10T01:37:30.875 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2023-09-10T01:37:30.875 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2023-09-10T01:37:30.875 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2023-09-10T01:37:30.876 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2023-09-10T01:37:30.876 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2023-09-10T01:37:30.876 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2023-09-10T01:37:30.876 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 9896, 2023-09-10T01:37:30.876 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 190, 2023-09-10T01:37:30.876 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2023-09-10T01:37:30.877 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2023-09-10T01:37:30.877 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2023-09-10T01:37:30.877 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2023-09-10T01:37:30.877 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 1139004, 2023-09-10T01:37:30.877 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 25036, 2023-09-10T01:37:30.877 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2023-09-10T01:37:30.878 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2023-09-10T01:37:30.878 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2023-09-10T01:37:30.878 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2023-09-10T01:37:30.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.878 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2023-09-10T01:37:30.879 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2023-09-10T01:37:30.879 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2023-09-10T01:37:30.879 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 11528, 2023-09-10T01:37:30.879 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 21, 2023-09-10T01:37:30.879 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 3, 2023-09-10T01:37:30.880 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 11528, 2023-09-10T01:37:30.880 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2023-09-10T01:37:30.880 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 11528, 2023-09-10T01:37:30.880 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 22.582472876, 2023-09-10T01:37:30.880 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001958923 2023-09-10T01:37:30.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.881 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2023-09-10T01:37:30.881 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2023-09-10T01:37:30.881 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 11528, 2023-09-10T01:37:30.881 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 11529 2023-09-10T01:37:30.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.882 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 11528, 2023-09-10T01:37:30.882 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 11528, 2023-09-10T01:37:30.882 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 0, 2023-09-10T01:37:30.882 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2023-09-10T01:37:30.882 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2023-09-10T01:37:30.883 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 6957, 2023-09-10T01:37:30.883 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 0, 2023-09-10T01:37:30.883 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 4569, 2023-09-10T01:37:30.883 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 0, 2023-09-10T01:37:30.883 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 0, 2023-09-10T01:37:30.884 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2023-09-10T01:37:30.884 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2023-09-10T01:37:30.884 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2023-09-10T01:37:30.884 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2023-09-10T01:37:30.884 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 0, 2023-09-10T01:37:30.884 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2023-09-10T01:37:30.885 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2023-09-10T01:37:30.885 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2023-09-10T01:37:30.885 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 1, 2023-09-10T01:37:30.885 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 0, 2023-09-10T01:37:30.885 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2023-09-10T01:37:30.886 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2023-09-10T01:37:30.886 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2023-09-10T01:37:30.886 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2023-09-10T01:37:30.886 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2023-09-10T01:37:30.886 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2023-09-10T01:37:30.887 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2023-09-10T01:37:30.887 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2023-09-10T01:37:30.887 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2023-09-10T01:37:30.887 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2023-09-10T01:37:30.887 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2023-09-10T01:37:30.888 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2023-09-10T01:37:30.888 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2023-09-10T01:37:30.888 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2023-09-10T01:37:30.888 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2023-09-10T01:37:30.889 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2023-09-10T01:37:30.889 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2023-09-10T01:37:30.889 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2023-09-10T01:37:30.889 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2023-09-10T01:37:30.889 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2023-09-10T01:37:30.890 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2023-09-10T01:37:30.890 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2023-09-10T01:37:30.890 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2023-09-10T01:37:30.890 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2023-09-10T01:37:30.890 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2023-09-10T01:37:30.890 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2023-09-10T01:37:30.891 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 213, 2023-09-10T01:37:30.891 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2023-09-10T01:37:30.891 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 126, 2023-09-10T01:37:30.891 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 9, 2023-09-10T01:37:30.891 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 20, 2023-09-10T01:37:30.892 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 11, 2023-09-10T01:37:30.892 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2023-09-10T01:37:30.892 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 0, 2023-09-10T01:37:30.892 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 2, 2023-09-10T01:37:30.892 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 0 2023-09-10T01:37:30.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.893 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2023-09-10T01:37:30.893 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 0, 2023-09-10T01:37:30.893 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2023-09-10T01:37:30.893 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2023-09-10T01:37:30.893 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2023-09-10T01:37:30.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.894 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2023-09-10T01:37:30.894 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2023-09-10T01:37:30.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.894 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2023-09-10T01:37:30.895 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2023-09-10T01:37:30.895 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2023-09-10T01:37:30.895 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2023-09-10T01:37:30.895 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2023-09-10T01:37:30.895 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2023-09-10T01:37:30.895 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2023-09-10T01:37:30.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.896 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-10T01:37:30.896 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:30.896 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-09-10T01:37:30.896 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:30.897 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 12796, 2023-09-10T01:37:30.897 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 121152626, 2023-09-10T01:37:30.897 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:30.897 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 12796, 2023-09-10T01:37:30.897 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-09-10T01:37:30.897 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-09-10T01:37:30.898 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 12796, 2023-09-10T01:37:30.898 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 121152626, 2023-09-10T01:37:30.898 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:30.898 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.898 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.899 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.899 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:30.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.899 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2023-09-10T01:37:30.899 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:30.899 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-09-10T01:37:30.900 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:30.900 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-09-10T01:37:30.900 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-09-10T01:37:30.900 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:30.900 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:30.900 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 11528, 2023-09-10T01:37:30.901 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 116048132, 2023-09-10T01:37:30.901 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 11528, 2023-09-10T01:37:30.901 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 116048132, 2023-09-10T01:37:30.901 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:30.901 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.902 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.902 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.902 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:30.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.903 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2023-09-10T01:37:30.903 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:30.903 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2023-09-10T01:37:30.903 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:30.903 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-09-10T01:37:30.904 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-09-10T01:37:30.904 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:30.904 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:30.904 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 11528, 2023-09-10T01:37:30.904 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 11528, 2023-09-10T01:37:30.904 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 11528, 2023-09-10T01:37:30.905 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 11528, 2023-09-10T01:37:30.905 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:30.905 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.905 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.905 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.905 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:30.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.906 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2023-09-10T01:37:30.906 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:30.906 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2023-09-10T01:37:30.906 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:30.907 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-09-10T01:37:30.907 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-09-10T01:37:30.907 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:30.907 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:30.907 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-09-10T01:37:30.907 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-09-10T01:37:30.908 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2023-09-10T01:37:30.908 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2023-09-10T01:37:30.908 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:30.908 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.908 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.908 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.909 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:30.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:30.909 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x562a571ac960": { 2023-09-10T01:37:30.909 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:30.909 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2023-09-10T01:37:30.909 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:30.910 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-09-10T01:37:30.910 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-09-10T01:37:30.910 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:30.910 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:30.910 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-09-10T01:37:30.910 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-09-10T01:37:30.911 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2023-09-10T01:37:30.911 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2023-09-10T01:37:30.911 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:30.911 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:30.912 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:30.912 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:30.912 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:30.912 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:30.912 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-09-10T01:37:31.150 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:30 smithi099 ceph-mon[102692]: pgmap v1853: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:37:31.152 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-09-10T01:37:31.152 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-09-10T01:37:31.153 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2023-09-10T01:37:31.153 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-09-10T01:37:31.153 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-09-10T01:37:31.153 DEBUG:tasks.check_counter:Getting stats from h 2023-09-10T01:37:31.154 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.h perf dump 2023-09-10T01:37:31.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:30 smithi131 ceph-mon[103713]: pgmap v1853: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:37:31.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:30 smithi044 ceph-mon[98676]: pgmap v1853: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:37:31.733 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-10T01:37:32.978 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-09-10T01:37:32.979 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2023-09-10T01:37:32.979 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 70439, 2023-09-10T01:37:32.979 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 102777, 2023-09-10T01:37:32.979 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 21957504, 2023-09-10T01:37:32.980 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 101901038, 2023-09-10T01:37:32.980 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 24, 2023-09-10T01:37:32.980 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 5, 2023-09-10T01:37:32.980 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 4.073962647, 2023-09-10T01:37:32.981 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 1.616005095, 2023-09-10T01:37:32.981 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 2.499086919, 2023-09-10T01:37:32.981 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.247425614, 2023-09-10T01:37:32.981 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:32.982 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 102773, 2023-09-10T01:37:32.982 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 4.561017283, 2023-09-10T01:37:32.982 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000044379 2023-09-10T01:37:32.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:32.982 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:32.983 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 149850, 2023-09-10T01:37:32.983 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.024757234, 2023-09-10T01:37:32.983 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000165 2023-09-10T01:37:32.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:32.984 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 4736, 2023-09-10T01:37:32.984 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 2304 2023-09-10T01:37:32.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:32.984 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2023-09-10T01:37:32.984 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 4187134, 2023-09-10T01:37:32.985 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 4188083, 2023-09-10T01:37:32.985 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 667951943, 2023-09-10T01:37:32.985 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 750112970, 2023-09-10T01:37:32.985 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 11, 2023-09-10T01:37:32.985 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 7, 2023-09-10T01:37:32.986 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 109.504165387, 2023-09-10T01:37:32.986 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 42.443563773, 2023-09-10T01:37:32.986 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 74.673855053, 2023-09-10T01:37:32.986 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.218019143, 2023-09-10T01:37:32.986 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:32.987 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4188082, 2023-09-10T01:37:32.987 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 41.260748850, 2023-09-10T01:37:32.987 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000009851 2023-09-10T01:37:32.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:32.987 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:32.988 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 8354623, 2023-09-10T01:37:32.988 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.754422501, 2023-09-10T01:37:32.988 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000090 2023-09-10T01:37:32.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:32.989 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3122368, 2023-09-10T01:37:32.989 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 2078320 2023-09-10T01:37:32.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:32.989 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2023-09-10T01:37:32.989 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 2499765, 2023-09-10T01:37:32.989 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2520471, 2023-09-10T01:37:32.990 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 425443680, 2023-09-10T01:37:32.990 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 433543730, 2023-09-10T01:37:32.990 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 10, 2023-09-10T01:37:32.990 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 6, 2023-09-10T01:37:32.991 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 90.921724679, 2023-09-10T01:37:32.991 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 40.207139878, 2023-09-10T01:37:32.991 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 62.142659835, 2023-09-10T01:37:32.991 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.064404965, 2023-09-10T01:37:32.991 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:32.992 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2520471, 2023-09-10T01:37:32.992 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 26.413838555, 2023-09-10T01:37:32.992 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000010479 2023-09-10T01:37:32.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:32.992 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5012447, 2023-09-10T01:37:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.536030730, 2023-09-10T01:37:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000106 2023-09-10T01:37:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 6192, 2023-09-10T01:37:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 156480 2023-09-10T01:37:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2023-09-10T01:37:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2023-09-10T01:37:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2023-09-10T01:37:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2023-09-10T01:37:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2023-09-10T01:37:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2023-09-10T01:37:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 35242, 2023-09-10T01:37:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 13.092783421, 2023-09-10T01:37:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000371510 2023-09-10T01:37:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2023-09-10T01:37:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2023-09-10T01:37:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2023-09-10T01:37:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1549, 2023-09-10T01:37:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.082874481, 2023-09-10T01:37:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000053501 2023-09-10T01:37:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2023-09-10T01:37:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 23456, 2023-09-10T01:37:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 21048, 2023-09-10T01:37:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2023-09-10T01:37:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 21048, 2023-09-10T01:37:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 650.016710644, 2023-09-10T01:37:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.030882587 2023-09-10T01:37:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2023-09-10T01:37:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 2408, 2023-09-10T01:37:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 7, 2023-09-10T01:37:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2023-09-10T01:37:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 29, 2023-09-10T01:37:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 2, 2023-09-10T01:37:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 2, 2023-09-10T01:37:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 449, 2023-09-10T01:37:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 51, 2023-09-10T01:37:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 238, 2023-09-10T01:37:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 160, 2023-09-10T01:37:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 375, 2023-09-10T01:37:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 2931, 2023-09-10T01:37:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 71, 2023-09-10T01:37:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 71, 2023-09-10T01:37:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 21, 2023-09-10T01:37:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 65883, 2023-09-10T01:37:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 57571, 2023-09-10T01:37:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 2361, 2023-09-10T01:37:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 339, 2023-09-10T01:37:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 2, 2023-09-10T01:37:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2023-09-10T01:37:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 2670, 2023-09-10T01:37:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 565, 2023-09-10T01:37:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2023-09-10T01:37:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 69, 2023-09-10T01:37:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 2309, 2023-09-10T01:37:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 62, 2023-09-10T01:37:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 2257, 2023-09-10T01:37:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2023-09-10T01:37:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 281, 2023-09-10T01:37:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2023-09-10T01:37:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 1038, 2023-09-10T01:37:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 7721252804, 2023-09-10T01:37:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 6, 2023-09-10T01:37:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 281, 2023-09-10T01:37:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2023-09-10T01:37:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2023-09-10T01:37:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2023-09-10T01:37:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 3, 2023-09-10T01:37:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2023-09-10T01:37:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 3, 2023-09-10T01:37:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 278, 2023-09-10T01:37:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 415, 2023-09-10T01:37:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 32942, 2023-09-10T01:37:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 18335, 2023-09-10T01:37:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2023-09-10T01:37:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-10T01:37:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-10T01:37:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 37452, 2023-09-10T01:37:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 15165, 2023-09-10T01:37:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 312, 2023-09-10T01:37:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 1856 2023-09-10T01:37:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2023-09-10T01:37:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 2, 2023-09-10T01:37:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2023-09-10T01:37:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2023-09-10T01:37:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 366, 2023-09-10T01:37:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 237, 2023-09-10T01:37:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2023-09-10T01:37:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2023-09-10T01:37:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2023-09-10T01:37:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 456, 2023-09-10T01:37:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 378, 2023-09-10T01:37:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 488, 2023-09-10T01:37:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 60, 2023-09-10T01:37:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2023-09-10T01:37:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2023-09-10T01:37:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2023-09-10T01:37:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2023-09-10T01:37:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2023-09-10T01:37:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2023-09-10T01:37:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 72, 2023-09-10T01:37:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2023-09-10T01:37:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 4, 2023-09-10T01:37:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2023-09-10T01:37:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2023-09-10T01:37:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2023-09-10T01:37:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2023-09-10T01:37:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 22236, 2023-09-10T01:37:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2023-09-10T01:37:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2023-09-10T01:37:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 22236, 2023-09-10T01:37:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2023-09-10T01:37:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2023-09-10T01:37:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 34, 2023-09-10T01:37:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2023-09-10T01:37:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2023-09-10T01:37:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 34, 2023-09-10T01:37:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 3, 2023-09-10T01:37:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2023-09-10T01:37:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2023-09-10T01:37:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2023-09-10T01:37:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 120239536, 2023-09-10T01:37:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 0, 2023-09-10T01:37:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2023-09-10T01:37:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 17350, 2023-09-10T01:37:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 62.959724793, 2023-09-10T01:37:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.003628802 2023-09-10T01:37:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 0 2023-09-10T01:37:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2023-09-10T01:37:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 196, 2023-09-10T01:37:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 2662, 2023-09-10T01:37:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 2466, 2023-09-10T01:37:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 83, 2023-09-10T01:37:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 180, 2023-09-10T01:37:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 97, 2023-09-10T01:37:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 449, 2023-09-10T01:37:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 3773, 2023-09-10T01:37:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 3324, 2023-09-10T01:37:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 71, 2023-09-10T01:37:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 1792, 2023-09-10T01:37:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 1721, 2023-09-10T01:37:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 69776, 2023-09-10T01:37:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 198960 2023-09-10T01:37:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2023-09-10T01:37:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 55286, 2023-09-10T01:37:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 4199, 2023-09-10T01:37:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 23456, 2023-09-10T01:37:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 193, 2023-09-10T01:37:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 10043, 2023-09-10T01:37:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2023-09-10T01:37:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 255, 2023-09-10T01:37:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.580769390, 2023-09-10T01:37:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002277527 2023-09-10T01:37:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2023-09-10T01:37:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 13442, 2023-09-10T01:37:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 43.580381817, 2023-09-10T01:37:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.003242105 2023-09-10T01:37:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2023-09-10T01:37:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2023-09-10T01:37:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 82, 2023-09-10T01:37:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.642620039, 2023-09-10T01:37:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.020031951 2023-09-10T01:37:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2023-09-10T01:37:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4699, 2023-09-10T01:37:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 135.367932854, 2023-09-10T01:37:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.028807817 2023-09-10T01:37:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2023-09-10T01:37:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2023-09-10T01:37:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2023-09-10T01:37:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2023-09-10T01:37:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2023-09-10T01:37:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2023-09-10T01:37:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2023-09-10T01:37:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2023-09-10T01:37:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2023-09-10T01:37:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2023-09-10T01:37:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1510, 2023-09-10T01:37:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.021441287, 2023-09-10T01:37:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000676451 2023-09-10T01:37:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2023-09-10T01:37:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 52, 2023-09-10T01:37:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.069170965, 2023-09-10T01:37:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001330210 2023-09-10T01:37:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2023-09-10T01:37:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 927, 2023-09-10T01:37:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 467.146837184, 2023-09-10T01:37:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.503934020 2023-09-10T01:37:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2023-09-10T01:37:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2023-09-10T01:37:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2023-09-10T01:37:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2023-09-10T01:37:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2023-09-10T01:37:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2023-09-10T01:37:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2023-09-10T01:37:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2023-09-10T01:37:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2023-09-10T01:37:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2023-09-10T01:37:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2023-09-10T01:37:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2023-09-10T01:37:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 81, 2023-09-10T01:37:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.607557108, 2023-09-10T01:37:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.007500705 2023-09-10T01:37:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2023-09-10T01:37:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2023-09-10T01:37:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2023-09-10T01:37:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2023-09-10T01:37:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 2, 2023-09-10T01:37:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 4, 2023-09-10T01:37:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 2, 2023-09-10T01:37:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 2, 2023-09-10T01:37:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2023-09-10T01:37:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 450, 2023-09-10T01:37:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 225, 2023-09-10T01:37:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 3408, 2023-09-10T01:37:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2023-09-10T01:37:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2023-09-10T01:37:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2023-09-10T01:37:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2023-09-10T01:37:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2023-09-10T01:37:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2023-09-10T01:37:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2023-09-10T01:37:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2023-09-10T01:37:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-10T01:37:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2023-09-10T01:37:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-10T01:37:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2023-09-10T01:37:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2023-09-10T01:37:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2023-09-10T01:37:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-10T01:37:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2023-09-10T01:37:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2023-09-10T01:37:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2023-09-10T01:37:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2023-09-10T01:37:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2023-09-10T01:37:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-10T01:37:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2023-09-10T01:37:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-10T01:37:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2023-09-10T01:37:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2023-09-10T01:37:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2023-09-10T01:37:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2023-09-10T01:37:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2023-09-10T01:37:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-10T01:37:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2023-09-10T01:37:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2023-09-10T01:37:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2023-09-10T01:37:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2023-09-10T01:37:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2023-09-10T01:37:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2023-09-10T01:37:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2023-09-10T01:37:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2023-09-10T01:37:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2023-09-10T01:37:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 703455, 2023-09-10T01:37:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 218, 2023-09-10T01:37:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2023-09-10T01:37:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2023-09-10T01:37:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2023-09-10T01:37:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2023-09-10T01:37:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2023-09-10T01:37:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2023-09-10T01:37:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2023-09-10T01:37:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2023-09-10T01:37:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 9896, 2023-09-10T01:37:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 190, 2023-09-10T01:37:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2023-09-10T01:37:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2023-09-10T01:37:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2023-09-10T01:37:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2023-09-10T01:37:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 1414478, 2023-09-10T01:37:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 47316, 2023-09-10T01:37:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2023-09-10T01:37:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2023-09-10T01:37:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2023-09-10T01:37:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2023-09-10T01:37:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2023-09-10T01:37:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2023-09-10T01:37:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2023-09-10T01:37:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 19580, 2023-09-10T01:37:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 116313120, 2023-09-10T01:37:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 33, 2023-09-10T01:37:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 19580, 2023-09-10T01:37:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2023-09-10T01:37:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 19580, 2023-09-10T01:37:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 85.307010324, 2023-09-10T01:37:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.004356844 2023-09-10T01:37:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2023-09-10T01:37:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2023-09-10T01:37:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 19580, 2023-09-10T01:37:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 20060 2023-09-10T01:37:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 19580, 2023-09-10T01:37:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 514, 2023-09-10T01:37:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 19066, 2023-09-10T01:37:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2023-09-10T01:37:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2023-09-10T01:37:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 0, 2023-09-10T01:37:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 10, 2023-09-10T01:37:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 203, 2023-09-10T01:37:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 17553, 2023-09-10T01:37:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 340, 2023-09-10T01:37:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2023-09-10T01:37:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2023-09-10T01:37:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2023-09-10T01:37:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 2, 2023-09-10T01:37:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 799, 2023-09-10T01:37:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2023-09-10T01:37:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2023-09-10T01:37:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2023-09-10T01:37:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 283, 2023-09-10T01:37:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 20, 2023-09-10T01:37:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2023-09-10T01:37:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2023-09-10T01:37:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2023-09-10T01:37:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2023-09-10T01:37:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2023-09-10T01:37:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2023-09-10T01:37:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2023-09-10T01:37:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2023-09-10T01:37:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2023-09-10T01:37:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2023-09-10T01:37:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2023-09-10T01:37:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2023-09-10T01:37:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2023-09-10T01:37:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2023-09-10T01:37:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2023-09-10T01:37:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2023-09-10T01:37:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2023-09-10T01:37:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2023-09-10T01:37:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2023-09-10T01:37:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2023-09-10T01:37:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2023-09-10T01:37:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2023-09-10T01:37:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2023-09-10T01:37:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2023-09-10T01:37:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2023-09-10T01:37:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2023-09-10T01:37:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 213, 2023-09-10T01:37:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2023-09-10T01:37:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 126, 2023-09-10T01:37:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 12, 2023-09-10T01:37:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 12, 2023-09-10T01:37:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2023-09-10T01:37:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2023-09-10T01:37:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 626, 2023-09-10T01:37:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 37, 2023-09-10T01:37:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 187 2023-09-10T01:37:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2023-09-10T01:37:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 1, 2023-09-10T01:37:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2023-09-10T01:37:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2023-09-10T01:37:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2023-09-10T01:37:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2023-09-10T01:37:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2023-09-10T01:37:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2023-09-10T01:37:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2023-09-10T01:37:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 10, 2023-09-10T01:37:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2023-09-10T01:37:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 5, 2023-09-10T01:37:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 378, 2023-09-10T01:37:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2023-09-10T01:37:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-10T01:37:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-09-10T01:37:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 6757334, 2023-09-10T01:37:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 507123286, 2023-09-10T01:37:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 6757334, 2023-09-10T01:37:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-09-10T01:37:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-09-10T01:37:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 6757334, 2023-09-10T01:37:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 507123286, 2023-09-10T01:37:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.094 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:33.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.094 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2023-09-10T01:37:33.095 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:33.095 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-09-10T01:37:33.095 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:33.095 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-09-10T01:37:33.095 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-09-10T01:37:33.095 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:33.096 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:33.096 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 19580, 2023-09-10T01:37:33.096 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 116354530, 2023-09-10T01:37:33.096 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 19580, 2023-09-10T01:37:33.096 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 116354530, 2023-09-10T01:37:33.097 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:33.097 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.097 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.097 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.097 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:33.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.098 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2023-09-10T01:37:33.098 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:33.098 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2023-09-10T01:37:33.099 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:33.099 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-09-10T01:37:33.099 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-09-10T01:37:33.099 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:33.099 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:33.100 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 19580, 2023-09-10T01:37:33.100 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 19580, 2023-09-10T01:37:33.100 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 19580, 2023-09-10T01:37:33.100 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 19580, 2023-09-10T01:37:33.100 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:33.101 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.101 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.101 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.101 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:33.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.102 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2023-09-10T01:37:33.102 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:33.102 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2023-09-10T01:37:33.102 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:33.102 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 378, 2023-09-10T01:37:33.103 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 42306, 2023-09-10T01:37:33.103 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:33.103 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 378, 2023-09-10T01:37:33.103 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-09-10T01:37:33.103 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-09-10T01:37:33.104 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 203, 2023-09-10T01:37:33.104 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 42306, 2023-09-10T01:37:33.104 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:33.104 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.104 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.105 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:33.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:33.105 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x562da03706e0": { 2023-09-10T01:37:33.105 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:33.106 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2023-09-10T01:37:33.106 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:33.106 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 22236, 2023-09-10T01:37:33.106 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 116045232, 2023-09-10T01:37:33.106 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:33.107 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 22236, 2023-09-10T01:37:33.107 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-09-10T01:37:33.107 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-09-10T01:37:33.107 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 17350, 2023-09-10T01:37:33.107 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 116045232, 2023-09-10T01:37:33.108 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:33.108 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:33.108 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:33.108 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:33.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:33.109 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:33.109 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-09-10T01:37:33.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:32 smithi099 ceph-mon[102692]: pgmap v1854: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:37:33.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:32 smithi131 ceph-mon[103713]: pgmap v1854: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:37:33.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:32 smithi044 ceph-mon[98676]: pgmap v1854: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:37:33.397 INFO:tasks.check_counter:Daemon mds.h mds.exported=69 2023-09-10T01:37:33.398 INFO:tasks.check_counter:Daemon mds.h mds.imported=62 2023-09-10T01:37:33.398 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2023-09-10T01:37:33.398 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-09-10T01:37:33.398 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-09-10T01:37:33.399 DEBUG:tasks.check_counter:Getting stats from k 2023-09-10T01:37:33.399 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.k perf dump 2023-09-10T01:37:33.957 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-10T01:37:34.162 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:33 smithi099 ceph-mon[102692]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:34.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:33 smithi131 ceph-mon[103713]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:34.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:33 smithi044 ceph-mon[98676]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2023-09-10T01:37:35.183 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-09-10T01:37:35.183 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2023-09-10T01:37:35.183 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 2275040, 2023-09-10T01:37:35.184 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2269395, 2023-09-10T01:37:35.184 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 377699347, 2023-09-10T01:37:35.184 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 400017991, 2023-09-10T01:37:35.184 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 17, 2023-09-10T01:37:35.184 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2023-09-10T01:37:35.185 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 76.546365502, 2023-09-10T01:37:35.185 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 34.449349077, 2023-09-10T01:37:35.185 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 52.956611457, 2023-09-10T01:37:35.185 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.092339376, 2023-09-10T01:37:35.185 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:35.186 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2269394, 2023-09-10T01:37:35.186 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 25.187898324, 2023-09-10T01:37:35.186 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000011098 2023-09-10T01:37:35.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.186 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:35.186 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4531278, 2023-09-10T01:37:35.187 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.442063122, 2023-09-10T01:37:35.187 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000097 2023-09-10T01:37:35.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.187 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3116416, 2023-09-10T01:37:35.187 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 602912 2023-09-10T01:37:35.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.188 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2023-09-10T01:37:35.188 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 2112994, 2023-09-10T01:37:35.188 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2114554, 2023-09-10T01:37:35.188 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 339270382, 2023-09-10T01:37:35.189 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 368723236, 2023-09-10T01:37:35.189 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 18, 2023-09-10T01:37:35.189 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 8, 2023-09-10T01:37:35.189 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 58.334904248, 2023-09-10T01:37:35.189 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 23.190806855, 2023-09-10T01:37:35.190 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 40.280592799, 2023-09-10T01:37:35.190 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.056650662, 2023-09-10T01:37:35.190 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:35.190 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2114550, 2023-09-10T01:37:35.190 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 20.686682852, 2023-09-10T01:37:35.190 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000009783 2023-09-10T01:37:35.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.191 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:35.191 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4222074, 2023-09-10T01:37:35.191 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.405647150, 2023-09-10T01:37:35.191 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000096 2023-09-10T01:37:35.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.192 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1712, 2023-09-10T01:37:35.192 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 1673680 2023-09-10T01:37:35.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.192 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2023-09-10T01:37:35.193 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 125172, 2023-09-10T01:37:35.193 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 118801, 2023-09-10T01:37:35.193 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 34843365, 2023-09-10T01:37:35.193 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 35745111, 2023-09-10T01:37:35.193 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 17, 2023-09-10T01:37:35.193 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 6, 2023-09-10T01:37:35.194 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 5.524989099, 2023-09-10T01:37:35.194 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 2.391602191, 2023-09-10T01:37:35.194 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 3.857329953, 2023-09-10T01:37:35.194 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.054214088, 2023-09-10T01:37:35.194 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-09-10T01:37:35.195 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 118800, 2023-09-10T01:37:35.195 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 4.312640656, 2023-09-10T01:37:35.195 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000036301 2023-09-10T01:37:35.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.195 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-09-10T01:37:35.196 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 222787, 2023-09-10T01:37:35.196 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.033889073, 2023-09-10T01:37:35.196 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000152 2023-09-10T01:37:35.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.196 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 2912, 2023-09-10T01:37:35.197 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 448 2023-09-10T01:37:35.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.197 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2023-09-10T01:37:35.197 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2023-09-10T01:37:35.197 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2023-09-10T01:37:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2023-09-10T01:37:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2023-09-10T01:37:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2023-09-10T01:37:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 8614, 2023-09-10T01:37:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 2.865087135, 2023-09-10T01:37:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000332608 2023-09-10T01:37:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2023-09-10T01:37:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2023-09-10T01:37:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2023-09-10T01:37:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 219, 2023-09-10T01:37:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.009600493, 2023-09-10T01:37:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000043837 2023-09-10T01:37:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2023-09-10T01:37:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 9846, 2023-09-10T01:37:35.202 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 8163, 2023-09-10T01:37:35.202 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2023-09-10T01:37:35.202 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 8163, 2023-09-10T01:37:35.202 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 795.181169397, 2023-09-10T01:37:35.202 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.097412859 2023-09-10T01:37:35.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.203 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2023-09-10T01:37:35.203 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 1683, 2023-09-10T01:37:35.203 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 15, 2023-09-10T01:37:35.203 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2023-09-10T01:37:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 11, 2023-09-10T01:37:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2023-09-10T01:37:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2023-09-10T01:37:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 1366, 2023-09-10T01:37:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 140, 2023-09-10T01:37:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 995, 2023-09-10T01:37:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 231, 2023-09-10T01:37:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 1166, 2023-09-10T01:37:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 2143, 2023-09-10T01:37:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 737, 2023-09-10T01:37:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 737, 2023-09-10T01:37:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 20, 2023-09-10T01:37:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 44502, 2023-09-10T01:37:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 36711, 2023-09-10T01:37:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 1623, 2023-09-10T01:37:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 262, 2023-09-10T01:37:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 3, 2023-09-10T01:37:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2023-09-10T01:37:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 3911, 2023-09-10T01:37:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 4362, 2023-09-10T01:37:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2023-09-10T01:37:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 24, 2023-09-10T01:37:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 1541, 2023-09-10T01:37:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 31, 2023-09-10T01:37:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 2427, 2023-09-10T01:37:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2023-09-10T01:37:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 370, 2023-09-10T01:37:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2023-09-10T01:37:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 1038, 2023-09-10T01:37:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 7721252804, 2023-09-10T01:37:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 6, 2023-09-10T01:37:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 370, 2023-09-10T01:37:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2023-09-10T01:37:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2023-09-10T01:37:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2023-09-10T01:37:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 1, 2023-09-10T01:37:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2023-09-10T01:37:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 1, 2023-09-10T01:37:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 369, 2023-09-10T01:37:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 628, 2023-09-10T01:37:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 22177, 2023-09-10T01:37:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 19436, 2023-09-10T01:37:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2023-09-10T01:37:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-10T01:37:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-10T01:37:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 24726, 2023-09-10T01:37:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 2163, 2023-09-10T01:37:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 350, 2023-09-10T01:37:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 553 2023-09-10T01:37:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2023-09-10T01:37:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2023-09-10T01:37:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2023-09-10T01:37:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2023-09-10T01:37:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 296, 2023-09-10T01:37:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 258, 2023-09-10T01:37:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2023-09-10T01:37:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2023-09-10T01:37:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2023-09-10T01:37:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 160, 2023-09-10T01:37:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 42, 2023-09-10T01:37:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 740, 2023-09-10T01:37:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 90, 2023-09-10T01:37:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2023-09-10T01:37:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2023-09-10T01:37:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2023-09-10T01:37:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2023-09-10T01:37:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2023-09-10T01:37:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2023-09-10T01:37:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 28, 2023-09-10T01:37:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2023-09-10T01:37:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2023-09-10T01:37:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2023-09-10T01:37:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2023-09-10T01:37:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2023-09-10T01:37:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 1, 2023-09-10T01:37:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 8491, 2023-09-10T01:37:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2023-09-10T01:37:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2023-09-10T01:37:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 8491, 2023-09-10T01:37:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2023-09-10T01:37:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2023-09-10T01:37:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 14, 2023-09-10T01:37:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2023-09-10T01:37:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2023-09-10T01:37:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 14, 2023-09-10T01:37:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 2, 2023-09-10T01:37:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2023-09-10T01:37:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2023-09-10T01:37:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2023-09-10T01:37:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 37916313, 2023-09-10T01:37:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 0, 2023-09-10T01:37:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2023-09-10T01:37:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3980, 2023-09-10T01:37:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 38.130175049, 2023-09-10T01:37:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.009580445 2023-09-10T01:37:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 0 2023-09-10T01:37:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2023-09-10T01:37:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 1257, 2023-09-10T01:37:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 2558, 2023-09-10T01:37:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 1301, 2023-09-10T01:37:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 98, 2023-09-10T01:37:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 188, 2023-09-10T01:37:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 90, 2023-09-10T01:37:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 1366, 2023-09-10T01:37:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 3559, 2023-09-10T01:37:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 2193, 2023-09-10T01:37:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 737, 2023-09-10T01:37:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 1786, 2023-09-10T01:37:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 1049, 2023-09-10T01:37:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 73308, 2023-09-10T01:37:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 198960 2023-09-10T01:37:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2023-09-10T01:37:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 31826, 2023-09-10T01:37:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 5799, 2023-09-10T01:37:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 9846, 2023-09-10T01:37:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 155, 2023-09-10T01:37:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 11462, 2023-09-10T01:37:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2023-09-10T01:37:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 47, 2023-09-10T01:37:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.099119602, 2023-09-10T01:37:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002108927 2023-09-10T01:37:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2023-09-10T01:37:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 286, 2023-09-10T01:37:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.784222902, 2023-09-10T01:37:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002742038 2023-09-10T01:37:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2023-09-10T01:37:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2023-09-10T01:37:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 128, 2023-09-10T01:37:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 3.340574148, 2023-09-10T01:37:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.026098235 2023-09-10T01:37:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2023-09-10T01:37:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4842, 2023-09-10T01:37:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 154.675510745, 2023-09-10T01:37:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.031944549 2023-09-10T01:37:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2023-09-10T01:37:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2023-09-10T01:37:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2023-09-10T01:37:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2023-09-10T01:37:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2023-09-10T01:37:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2023-09-10T01:37:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2023-09-10T01:37:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2023-09-10T01:37:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2023-09-10T01:37:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2023-09-10T01:37:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1758, 2023-09-10T01:37:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.098463428, 2023-09-10T01:37:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000624836 2023-09-10T01:37:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2023-09-10T01:37:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 100, 2023-09-10T01:37:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.094836926, 2023-09-10T01:37:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000948369 2023-09-10T01:37:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2023-09-10T01:37:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 884, 2023-09-10T01:37:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 634.167879429, 2023-09-10T01:37:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.717384478 2023-09-10T01:37:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2023-09-10T01:37:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2023-09-10T01:37:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2023-09-10T01:37:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2023-09-10T01:37:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2023-09-10T01:37:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2023-09-10T01:37:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2023-09-10T01:37:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2023-09-10T01:37:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2023-09-10T01:37:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2023-09-10T01:37:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2023-09-10T01:37:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2023-09-10T01:37:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 118, 2023-09-10T01:37:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.920562217, 2023-09-10T01:37:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.007801374 2023-09-10T01:37:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2023-09-10T01:37:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2023-09-10T01:37:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2023-09-10T01:37:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2023-09-10T01:37:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 1, 2023-09-10T01:37:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 2, 2023-09-10T01:37:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 1, 2023-09-10T01:37:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 1, 2023-09-10T01:37:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2023-09-10T01:37:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 356, 2023-09-10T01:37:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 356, 2023-09-10T01:37:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 3429, 2023-09-10T01:37:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2023-09-10T01:37:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2023-09-10T01:37:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2023-09-10T01:37:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2023-09-10T01:37:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2023-09-10T01:37:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2023-09-10T01:37:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2023-09-10T01:37:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2023-09-10T01:37:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-10T01:37:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2023-09-10T01:37:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-10T01:37:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2023-09-10T01:37:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2023-09-10T01:37:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2023-09-10T01:37:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-10T01:37:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2023-09-10T01:37:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2023-09-10T01:37:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2023-09-10T01:37:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2023-09-10T01:37:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2023-09-10T01:37:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-10T01:37:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2023-09-10T01:37:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-10T01:37:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2023-09-10T01:37:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2023-09-10T01:37:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2023-09-10T01:37:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2023-09-10T01:37:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2023-09-10T01:37:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-10T01:37:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2023-09-10T01:37:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2023-09-10T01:37:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2023-09-10T01:37:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2023-09-10T01:37:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2023-09-10T01:37:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2023-09-10T01:37:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2023-09-10T01:37:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2023-09-10T01:37:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2023-09-10T01:37:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 631511, 2023-09-10T01:37:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 201, 2023-09-10T01:37:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2023-09-10T01:37:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2023-09-10T01:37:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2023-09-10T01:37:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2023-09-10T01:37:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2023-09-10T01:37:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2023-09-10T01:37:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2023-09-10T01:37:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2023-09-10T01:37:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 9896, 2023-09-10T01:37:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 190, 2023-09-10T01:37:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2023-09-10T01:37:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2023-09-10T01:37:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2023-09-10T01:37:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2023-09-10T01:37:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 4499642, 2023-09-10T01:37:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 46091, 2023-09-10T01:37:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2023-09-10T01:37:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2023-09-10T01:37:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2023-09-10T01:37:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2023-09-10T01:37:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2023-09-10T01:37:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2023-09-10T01:37:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2023-09-10T01:37:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 5142, 2023-09-10T01:37:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 33824997, 2023-09-10T01:37:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 33, 2023-09-10T01:37:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 5142, 2023-09-10T01:37:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2023-09-10T01:37:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5142, 2023-09-10T01:37:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 44.667957482, 2023-09-10T01:37:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.008686883 2023-09-10T01:37:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2023-09-10T01:37:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2023-09-10T01:37:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5142, 2023-09-10T01:37:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 5646 2023-09-10T01:37:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 5142, 2023-09-10T01:37:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 446, 2023-09-10T01:37:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 4696, 2023-09-10T01:37:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2023-09-10T01:37:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2023-09-10T01:37:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 0, 2023-09-10T01:37:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 10, 2023-09-10T01:37:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 30, 2023-09-10T01:37:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 4010, 2023-09-10T01:37:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 233, 2023-09-10T01:37:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2023-09-10T01:37:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2023-09-10T01:37:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2023-09-10T01:37:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2023-09-10T01:37:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 104, 2023-09-10T01:37:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2023-09-10T01:37:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2023-09-10T01:37:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2023-09-10T01:37:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 370, 2023-09-10T01:37:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 20, 2023-09-10T01:37:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2023-09-10T01:37:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2023-09-10T01:37:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2023-09-10T01:37:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2023-09-10T01:37:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2023-09-10T01:37:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2023-09-10T01:37:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2023-09-10T01:37:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2023-09-10T01:37:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2023-09-10T01:37:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2023-09-10T01:37:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2023-09-10T01:37:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2023-09-10T01:37:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2023-09-10T01:37:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2023-09-10T01:37:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2023-09-10T01:37:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2023-09-10T01:37:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2023-09-10T01:37:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2023-09-10T01:37:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2023-09-10T01:37:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2023-09-10T01:37:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2023-09-10T01:37:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2023-09-10T01:37:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2023-09-10T01:37:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2023-09-10T01:37:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2023-09-10T01:37:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2023-09-10T01:37:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 213, 2023-09-10T01:37:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2023-09-10T01:37:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 126, 2023-09-10T01:37:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 12, 2023-09-10T01:37:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 20, 2023-09-10T01:37:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 8, 2023-09-10T01:37:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2023-09-10T01:37:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 598, 2023-09-10T01:37:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 61, 2023-09-10T01:37:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 210 2023-09-10T01:37:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2023-09-10T01:37:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 1, 2023-09-10T01:37:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2023-09-10T01:37:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2023-09-10T01:37:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2023-09-10T01:37:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2023-09-10T01:37:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 1 2023-09-10T01:37:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2023-09-10T01:37:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2023-09-10T01:37:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 10, 2023-09-10T01:37:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2023-09-10T01:37:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 5, 2023-09-10T01:37:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 42, 2023-09-10T01:37:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2023-09-10T01:37:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-10T01:37:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-09-10T01:37:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 4513202, 2023-09-10T01:37:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 345556655, 2023-09-10T01:37:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 4513202, 2023-09-10T01:37:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-09-10T01:37:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-09-10T01:37:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 4513202, 2023-09-10T01:37:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 345556655, 2023-09-10T01:37:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2023-09-10T01:37:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-09-10T01:37:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-09-10T01:37:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-09-10T01:37:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 5142, 2023-09-10T01:37:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 33828932, 2023-09-10T01:37:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 5142, 2023-09-10T01:37:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 33828932, 2023-09-10T01:37:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2023-09-10T01:37:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2023-09-10T01:37:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-09-10T01:37:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-09-10T01:37:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-09-10T01:37:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 5142, 2023-09-10T01:37:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 5142, 2023-09-10T01:37:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 5142, 2023-09-10T01:37:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 5142, 2023-09-10T01:37:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2023-09-10T01:37:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2023-09-10T01:37:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 42, 2023-09-10T01:37:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 5026, 2023-09-10T01:37:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 42, 2023-09-10T01:37:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-09-10T01:37:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-09-10T01:37:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 30, 2023-09-10T01:37:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 5026, 2023-09-10T01:37:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-09-10T01:37:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x5600915306e0": { 2023-09-10T01:37:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-09-10T01:37:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2023-09-10T01:37:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-09-10T01:37:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 8491, 2023-09-10T01:37:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 33722009, 2023-09-10T01:37:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-09-10T01:37:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 8491, 2023-09-10T01:37:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-09-10T01:37:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-09-10T01:37:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3980, 2023-09-10T01:37:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 33722009, 2023-09-10T01:37:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-09-10T01:37:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-09-10T01:37:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-09-10T01:37:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-09-10T01:37:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-09-10T01:37:35.306 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-09-10T01:37:35.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:34 smithi044 ceph-mon[98676]: pgmap v1855: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 13 op/s 2023-09-10T01:37:35.307 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:34 smithi131 ceph-mon[103713]: pgmap v1855: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 13 op/s 2023-09-10T01:37:35.307 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:34 smithi099 ceph-mon[102692]: pgmap v1855: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 13 op/s 2023-09-10T01:37:35.603 INFO:tasks.check_counter:Daemon mds.k mds.exported=24 2023-09-10T01:37:35.604 INFO:tasks.check_counter:Daemon mds.k mds.imported=31 2023-09-10T01:37:35.604 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2023-09-10T01:37:35.604 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2023-09-10T01:37:35.604 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2023-09-10T01:37:35.605 DEBUG:tasks.check_counter:Getting stats from c 2023-09-10T01:37:35.605 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.c perf dump 2023-09-10T01:37:36.299 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-10T01:37:37.161 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:36 smithi131 ceph-mon[103713]: pgmap v1856: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:37:37.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:36 smithi044 ceph-mon[98676]: pgmap v1856: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:37:37.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:36 smithi099 ceph-mon[102692]: pgmap v1856: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:37:37.818 INFO:teuthology.orchestra.run.smithi131.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2023-09-10T01:37:38.167 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-09-10T01:37:38.168 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/ceph_manager.py", line 1838, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi131 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.c perf dump' 2023-09-10T01:37:38.169 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-09-10T01:37:38.180 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-09-10T01:37:38.181 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T01:37:38.181 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-10T01:37:38.219 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T01:37:38.220 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-10T01:37:38.281 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T01:37:38.463 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.461+0000 7fc1ed361700 1 -- 172.21.15.44:0/3091423110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e8100f50 msgr2=0x7fc1e810d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:38.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.461+0000 7fc1ed361700 1 --2- 172.21.15.44:0/3091423110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e8100f50 0x7fc1e810d410 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000b0d0 tx=0x7fc1e000b3e0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.461+0000 7fc1ed361700 1 -- 172.21.15.44:0/3091423110 shutdown_connections 2023-09-10T01:37:38.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.461+0000 7fc1ed361700 1 --2- 172.21.15.44:0/3091423110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e8100f50 0x7fc1e810d410 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.461+0000 7fc1ed361700 1 --2- 172.21.15.44:0/3091423110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1e8106650 0x7fc1e8106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.461+0000 7fc1ed361700 1 --2- 172.21.15.44:0/3091423110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1e8100590 0x7fc1e8100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.461+0000 7fc1ed361700 1 -- 172.21.15.44:0/3091423110 >> 172.21.15.44:0/3091423110 conn(0x7fc1e80fc020 msgr2=0x7fc1e80fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:38.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.461+0000 7fc1ed361700 1 -- 172.21.15.44:0/3091423110 shutdown_connections 2023-09-10T01:37:38.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.461+0000 7fc1ed361700 1 -- 172.21.15.44:0/3091423110 wait complete. 2023-09-10T01:37:38.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.461+0000 7fc1ed361700 1 Processor -- start 2023-09-10T01:37:38.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.461+0000 7fc1ed361700 1 -- start start 2023-09-10T01:37:38.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1ed361700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1e8100590 0x7fc1e81965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:38.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1ed361700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e8100f50 0x7fc1e8196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:38.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1ed361700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1e8106650 0x7fc1e819af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:38.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1ed361700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e819b5b0 con 0x7fc1e8100f50 2023-09-10T01:37:38.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1ed361700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e819b720 con 0x7fc1e8100590 2023-09-10T01:37:38.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1ed361700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e819b8e0 con 0x7fc1e8106650 2023-09-10T01:37:38.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1e67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e8100f50 0x7fc1e8196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:38.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1e67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e8100f50 0x7fc1e8196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33386/0 (socket says 172.21.15.44:33386) 2023-09-10T01:37:38.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1e67fc700 1 -- 172.21.15.44:0/2111054803 learned_addr learned my addr 172.21.15.44:0/2111054803 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:38.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1e6ffd700 1 --2- 172.21.15.44:0/2111054803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1e8100590 0x7fc1e81965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:38.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1e77fe700 1 --2- 172.21.15.44:0/2111054803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1e8106650 0x7fc1e819af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:38.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1e67fc700 1 -- 172.21.15.44:0/2111054803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1e8106650 msgr2=0x7fc1e819af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:38.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1e67fc700 1 --2- 172.21.15.44:0/2111054803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1e8106650 0x7fc1e819af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1e67fc700 1 -- 172.21.15.44:0/2111054803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1e8100590 msgr2=0x7fc1e81965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1e67fc700 1 --2- 172.21.15.44:0/2111054803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1e8100590 0x7fc1e81965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1e67fc700 1 -- 172.21.15.44:0/2111054803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1dc009520 con 0x7fc1e8100f50 2023-09-10T01:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1e67fc700 1 --2- 172.21.15.44:0/2111054803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e8100f50 0x7fc1e8196b10 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00eb30 tx=0x7fc1dc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:38.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1cffff700 1 -- 172.21.15.44:0/2111054803 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc009880 con 0x7fc1e8100f50 2023-09-10T01:37:38.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1ed361700 1 -- 172.21.15.44:0/2111054803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e0009d00 con 0x7fc1e8100f50 2023-09-10T01:37:38.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1cffff700 1 -- 172.21.15.44:0/2111054803 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1dc0044b0 con 0x7fc1e8100f50 2023-09-10T01:37:38.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1cffff700 1 -- 172.21.15.44:0/2111054803 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc003e40 con 0x7fc1e8100f50 2023-09-10T01:37:38.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.462+0000 7fc1ed361700 1 -- 172.21.15.44:0/2111054803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e81a3220 con 0x7fc1e8100f50 2023-09-10T01:37:38.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.463+0000 7fc1ed361700 1 -- 172.21.15.44:0/2111054803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e80673f0 con 0x7fc1e8100f50 2023-09-10T01:37:38.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.465+0000 7fc1cffff700 1 -- 172.21.15.44:0/2111054803 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc1dc003fc0 con 0x7fc1e8100f50 2023-09-10T01:37:38.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.466+0000 7fc1cffff700 1 --2- 172.21.15.44:0/2111054803 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1d00a07f0 0x7fc1d00a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:38.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.466+0000 7fc1cffff700 1 -- 172.21.15.44:0/2111054803 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fc1dc014070 con 0x7fc1e8100f50 2023-09-10T01:37:38.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.466+0000 7fc1e6ffd700 1 --2- 172.21.15.44:0/2111054803 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1d00a07f0 0x7fc1d00a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:38.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.466+0000 7fc1e6ffd700 1 --2- 172.21.15.44:0/2111054803 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1d00a07f0 0x7fc1d00a2cb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8005fd0 tx=0x7fc1d8005f60 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:38.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.469+0000 7fc1cffff700 1 -- 172.21.15.44:0/2111054803 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc1dc095d50 con 0x7fc1e8100f50 2023-09-10T01:37:38.673 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.672+0000 7fc1ed361700 1 -- 172.21.15.44:0/2111054803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1e8197920 con 0x7fc1e8100f50 2023-09-10T01:37:38.674 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.674+0000 7fc1cffff700 1 -- 172.21.15.44:0/2111054803 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v213) v1 ==== 74+0+25247 (secure 0 0 0) 0x7fc1dc099370 con 0x7fc1e8100f50 2023-09-10T01:37:38.675 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:38.675 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":213,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T01:37:16.783072+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":108,"snap_epoch":212,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":108,"snap_epoch":212,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/902389643":"2023-09-11T00:40:38.970014+0000","172.21.15.44:0/3642136197":"2023-09-11T00:40:38.970014+0000","172.21.15.44:0/3766354728":"2023-09-11T00:40:38.970014+0000","172.21.15.44:6801/1488156576":"2023-09-11T00:40:38.970014+0000","172.21.15.44:6800/1488156576":"2023-09-11T00:40:38.970014+0000","172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1968699888":"2023-09-11T00:40:38.970014+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":95,"length":1},{"begin":108,"length":1}]},{"pool":4,"snaps":[{"begin":95,"length":1},{"begin":108,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T01:37:38.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.676+0000 7fc1ed361700 1 -- 172.21.15.44:0/2111054803 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1d00a07f0 msgr2=0x7fc1d00a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:38.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.676+0000 7fc1ed361700 1 --2- 172.21.15.44:0/2111054803 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1d00a07f0 0x7fc1d00a2cb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8005fd0 tx=0x7fc1d8005f60 comp rx=0 tx=0).stop 2023-09-10T01:37:38.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.676+0000 7fc1ed361700 1 -- 172.21.15.44:0/2111054803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e8100f50 msgr2=0x7fc1e8196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:38.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.676+0000 7fc1ed361700 1 --2- 172.21.15.44:0/2111054803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e8100f50 0x7fc1e8196b10 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00eb30 tx=0x7fc1dc00eef0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.677 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.676+0000 7fc1ed361700 1 -- 172.21.15.44:0/2111054803 shutdown_connections 2023-09-10T01:37:38.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.676+0000 7fc1ed361700 1 --2- 172.21.15.44:0/2111054803 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fc1d00a07f0 0x7fc1d00a2cb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.676+0000 7fc1ed361700 1 --2- 172.21.15.44:0/2111054803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1e8106650 0x7fc1e819af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.676+0000 7fc1ed361700 1 --2- 172.21.15.44:0/2111054803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1e8100590 0x7fc1e81965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.676+0000 7fc1ed361700 1 --2- 172.21.15.44:0/2111054803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e8100f50 0x7fc1e8196b10 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.676+0000 7fc1ed361700 1 -- 172.21.15.44:0/2111054803 >> 172.21.15.44:0/2111054803 conn(0x7fc1e80fc020 msgr2=0x7fc1e81042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:38.679 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.676+0000 7fc1ed361700 1 -- 172.21.15.44:0/2111054803 shutdown_connections 2023-09-10T01:37:38.679 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.676+0000 7fc1ed361700 1 -- 172.21.15.44:0/2111054803 wait complete. 2023-09-10T01:37:38.690 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-10T00:34:44.456741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-10T00:38:37.194891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-10T00:38:38.573268+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '212', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 108, 'snap_epoch': 212, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-10T00:38:42.198896+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '212', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 108, 'snap_epoch': 212, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-10T01:37:38.691 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-10T01:37:38.822 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.820+0000 7ffafaa99700 1 -- 172.21.15.44:0/1577698448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40ffe00 msgr2=0x7ffaf410c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:38.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.820+0000 7ffafaa99700 1 --2- 172.21.15.44:0/1577698448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40ffe00 0x7ffaf410c310 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00b0a0 tx=0x7ffaec00b3b0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.820+0000 7ffafaa99700 1 -- 172.21.15.44:0/1577698448 shutdown_connections 2023-09-10T01:37:38.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.820+0000 7ffafaa99700 1 --2- 172.21.15.44:0/1577698448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40ffe00 0x7ffaf410c310 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.820+0000 7ffafaa99700 1 --2- 172.21.15.44:0/1577698448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf41054e0 0x7ffaf41058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.823 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.820+0000 7ffafaa99700 1 --2- 172.21.15.44:0/1577698448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaf40ff440 0x7ffaf40ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.820+0000 7ffafaa99700 1 -- 172.21.15.44:0/1577698448 >> 172.21.15.44:0/1577698448 conn(0x7ffaf40faf50 msgr2=0x7ffaf40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:38.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.820+0000 7ffafaa99700 1 -- 172.21.15.44:0/1577698448 shutdown_connections 2023-09-10T01:37:38.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.820+0000 7ffafaa99700 1 -- 172.21.15.44:0/1577698448 wait complete. 2023-09-10T01:37:38.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.820+0000 7ffafaa99700 1 Processor -- start 2023-09-10T01:37:38.824 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.820+0000 7ffafaa99700 1 -- start start 2023-09-10T01:37:38.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffafaa99700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf40ff440 0x7ffaf4197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:38.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffafaa99700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40ffe00 0x7ffaf41978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:38.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffafaa99700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaf41054e0 0x7ffaf41a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:38.825 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffafaa99700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf41981f0 con 0x7ffaf40ffe00 2023-09-10T01:37:38.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffafaa99700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf41a2930 con 0x7ffaf40ff440 2023-09-10T01:37:38.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffafaa99700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf41a2af0 con 0x7ffaf41054e0 2023-09-10T01:37:38.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffaf3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40ffe00 0x7ffaf41978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:38.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffaf3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40ffe00 0x7ffaf41978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33400/0 (socket says 172.21.15.44:33400) 2023-09-10T01:37:38.826 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffaf3fff700 1 -- 172.21.15.44:0/325742330 learned_addr learned my addr 172.21.15.44:0/325742330 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:38.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffaf8835700 1 --2- 172.21.15.44:0/325742330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf40ff440 0x7ffaf4197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:38.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffaf9036700 1 --2- 172.21.15.44:0/325742330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaf41054e0 0x7ffaf41a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:38.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffaf3fff700 1 -- 172.21.15.44:0/325742330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaf41054e0 msgr2=0x7ffaf41a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:38.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffaf3fff700 1 --2- 172.21.15.44:0/325742330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaf41054e0 0x7ffaf41a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.827 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffaf3fff700 1 -- 172.21.15.44:0/325742330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf40ff440 msgr2=0x7ffaf4197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:38.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffaf3fff700 1 --2- 172.21.15.44:0/325742330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf40ff440 0x7ffaf4197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:38.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffaf3fff700 1 -- 172.21.15.44:0/325742330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec009d00 con 0x7ffaf40ffe00 2023-09-10T01:37:38.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffaf3fff700 1 --2- 172.21.15.44:0/325742330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40ffe00 0x7ffaf41978b0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ffae800e9a0 tx=0x7ffae800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:38.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffaf1ffb700 1 -- 172.21.15.44:0/325742330 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8009790 con 0x7ffaf40ffe00 2023-09-10T01:37:38.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffaf1ffb700 1 -- 172.21.15.44:0/325742330 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffae8010430 con 0x7ffaf40ffe00 2023-09-10T01:37:38.828 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffafaa99700 1 -- 172.21.15.44:0/325742330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf41a2dd0 con 0x7ffaf40ffe00 2023-09-10T01:37:38.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffaf1ffb700 1 -- 172.21.15.44:0/325742330 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae800cea0 con 0x7ffaf40ffe00 2023-09-10T01:37:38.829 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.821+0000 7ffafaa99700 1 -- 172.21.15.44:0/325742330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf41a3260 con 0x7ffaf40ffe00 2023-09-10T01:37:38.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.822+0000 7ffafaa99700 1 -- 172.21.15.44:0/325742330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf4067380 con 0x7ffaf40ffe00 2023-09-10T01:37:38.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.824+0000 7ffaf1ffb700 1 -- 172.21.15.44:0/325742330 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ffae80105a0 con 0x7ffaf40ffe00 2023-09-10T01:37:38.830 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.825+0000 7ffaf1ffb700 1 --2- 172.21.15.44:0/325742330 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffae40a0830 0x7ffae40a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:38.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.825+0000 7ffaf1ffb700 1 -- 172.21.15.44:0/325742330 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 9540+0+0 (secure 0 0 0) 0x7ffae8014070 con 0x7ffaf40ffe00 2023-09-10T01:37:38.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.825+0000 7ffaf8835700 1 --2- 172.21.15.44:0/325742330 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffae40a0830 0x7ffae40a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:38.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.825+0000 7ffaf8835700 1 --2- 172.21.15.44:0/325742330 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffae40a0830 0x7ffae40a2cf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ffae0007b60 tx=0x7ffae00073d0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:38.831 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:38.828+0000 7ffaf1ffb700 1 -- 172.21.15.44:0/325742330 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ffae8095af0 con 0x7ffaf40ffe00 2023-09-10T01:37:39.027 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.026+0000 7ffafaa99700 1 -- 172.21.15.44:0/325742330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffaf4067d70 con 0x7ffaf40ffe00 2023-09-10T01:37:39.028 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.027+0000 7ffaf1ffb700 1 -- 172.21.15.44:0/325742330 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v213) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffae8099110 con 0x7ffaf40ffe00 2023-09-10T01:37:39.028 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2023-09-10T01:37:39.030 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.029+0000 7ffafaa99700 1 -- 172.21.15.44:0/325742330 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffae40a0830 msgr2=0x7ffae40a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.029+0000 7ffafaa99700 1 --2- 172.21.15.44:0/325742330 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffae40a0830 0x7ffae40a2cf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ffae0007b60 tx=0x7ffae00073d0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.029+0000 7ffafaa99700 1 -- 172.21.15.44:0/325742330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40ffe00 msgr2=0x7ffaf41978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:39.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.029+0000 7ffafaa99700 1 --2- 172.21.15.44:0/325742330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40ffe00 0x7ffaf41978b0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ffae800e9a0 tx=0x7ffae800ecb0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.032 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.029+0000 7ffafaa99700 1 -- 172.21.15.44:0/325742330 shutdown_connections 2023-09-10T01:37:39.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.029+0000 7ffafaa99700 1 --2- 172.21.15.44:0/325742330 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7ffae40a0830 0x7ffae40a2cf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.029+0000 7ffafaa99700 1 --2- 172.21.15.44:0/325742330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaf41054e0 0x7ffaf41a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.033 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.029+0000 7ffafaa99700 1 --2- 172.21.15.44:0/325742330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf40ff440 0x7ffaf4197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.029+0000 7ffafaa99700 1 --2- 172.21.15.44:0/325742330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf40ffe00 0x7ffaf41978b0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.029+0000 7ffafaa99700 1 -- 172.21.15.44:0/325742330 >> 172.21.15.44:0/325742330 conn(0x7ffaf40faf50 msgr2=0x7ffaf40fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:39.035 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.029+0000 7ffafaa99700 1 -- 172.21.15.44:0/325742330 shutdown_connections 2023-09-10T01:37:39.036 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.029+0000 7ffafaa99700 1 -- 172.21.15.44:0/325742330 wait complete. 2023-09-10T01:37:39.041 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:38 smithi044 ceph-mon[98676]: pgmap v1857: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 14 op/s 2023-09-10T01:37:39.042 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:38 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2111054803' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T01:37:39.043 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-10T01:37:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.172+0000 7f47ec057700 1 -- 172.21.15.44:0/1810579662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47e410ab30 msgr2=0x7f47e410cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.172+0000 7f47ec057700 1 --2- 172.21.15.44:0/1810579662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47e410ab30 0x7f47e410cff0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00b0a0 tx=0x7f47dc00b3b0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.173+0000 7f47ec057700 1 -- 172.21.15.44:0/1810579662 shutdown_connections 2023-09-10T01:37:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.173+0000 7f47ec057700 1 --2- 172.21.15.44:0/1810579662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47e410ab30 0x7f47e410cff0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.173+0000 7f47ec057700 1 --2- 172.21.15.44:0/1810579662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47e4101710 0x7f47e4101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.173+0000 7f47ec057700 1 --2- 172.21.15.44:0/1810579662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47e4102030 0x7f47e410a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.173+0000 7f47ec057700 1 -- 172.21.15.44:0/1810579662 >> 172.21.15.44:0/1810579662 conn(0x7f47e40faf50 msgr2=0x7f47e40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.173+0000 7f47ec057700 1 -- 172.21.15.44:0/1810579662 shutdown_connections 2023-09-10T01:37:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.173+0000 7f47ec057700 1 -- 172.21.15.44:0/1810579662 wait complete. 2023-09-10T01:37:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.173+0000 7f47ec057700 1 Processor -- start 2023-09-10T01:37:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.173+0000 7f47ec057700 1 -- start start 2023-09-10T01:37:39.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.173+0000 7f47ec057700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47e4101710 0x7f47e419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:39.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.173+0000 7f47ec057700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47e4102030 0x7f47e419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:39.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.173+0000 7f47ec057700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47e410ab30 0x7f47e41965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:39.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.173+0000 7f47ec057700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e4196c10 con 0x7f47e410ab30 2023-09-10T01:37:39.178 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.173+0000 7f47ec057700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e4196d80 con 0x7f47e4102030 2023-09-10T01:37:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.173+0000 7f47ec057700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e4196f40 con 0x7f47e4101710 2023-09-10T01:37:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.174+0000 7f47ea5f4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47e410ab30 0x7f47e41965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.174+0000 7f47ea5f4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47e410ab30 0x7f47e41965a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33416/0 (socket says 172.21.15.44:33416) 2023-09-10T01:37:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.174+0000 7f47ea5f4700 1 -- 172.21.15.44:0/3377701078 learned_addr learned my addr 172.21.15.44:0/3377701078 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.174+0000 7f47e9df3700 1 --2- 172.21.15.44:0/3377701078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47e4101710 0x7f47e419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:39.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.174+0000 7f47e95f2700 1 --2- 172.21.15.44:0/3377701078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47e4102030 0x7f47e419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:39.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.174+0000 7f47ea5f4700 1 -- 172.21.15.44:0/3377701078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47e4101710 msgr2=0x7f47e419c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:39.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.174+0000 7f47ea5f4700 1 --2- 172.21.15.44:0/3377701078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47e4101710 0x7f47e419c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.174+0000 7f47ea5f4700 1 -- 172.21.15.44:0/3377701078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47e4102030 msgr2=0x7f47e419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:39.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.174+0000 7f47ea5f4700 1 --2- 172.21.15.44:0/3377701078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47e4102030 0x7f47e419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.180 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.174+0000 7f47ea5f4700 1 -- 172.21.15.44:0/3377701078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc009d00 con 0x7f47e410ab30 2023-09-10T01:37:39.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.174+0000 7f47ea5f4700 1 --2- 172.21.15.44:0/3377701078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47e410ab30 0x7f47e41965a0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00b780 tx=0x7f47dc00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:39.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.174+0000 7f47daffd700 1 -- 172.21.15.44:0/3377701078 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc00e040 con 0x7f47e410ab30 2023-09-10T01:37:39.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.174+0000 7f47daffd700 1 -- 172.21.15.44:0/3377701078 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f47dc004c50 con 0x7f47e410ab30 2023-09-10T01:37:39.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.174+0000 7f47ec057700 1 -- 172.21.15.44:0/3377701078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e41971c0 con 0x7f47e410ab30 2023-09-10T01:37:39.181 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.174+0000 7f47daffd700 1 -- 172.21.15.44:0/3377701078 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc0052a0 con 0x7f47e410ab30 2023-09-10T01:37:39.182 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.174+0000 7f47ec057700 1 -- 172.21.15.44:0/3377701078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e41a3520 con 0x7f47e410ab30 2023-09-10T01:37:39.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.175+0000 7f47ec057700 1 -- 172.21.15.44:0/3377701078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e4105c60 con 0x7f47e410ab30 2023-09-10T01:37:39.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.176+0000 7f47daffd700 1 -- 172.21.15.44:0/3377701078 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f47dc005400 con 0x7f47e410ab30 2023-09-10T01:37:39.183 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.177+0000 7f47daffd700 1 --2- 172.21.15.44:0/3377701078 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f47d00a0830 0x7f47d00a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:39.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.177+0000 7f47daffd700 1 -- 172.21.15.44:0/3377701078 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f47dc0cdff0 con 0x7f47e410ab30 2023-09-10T01:37:39.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.177+0000 7f47e9df3700 1 --2- 172.21.15.44:0/3377701078 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f47d00a0830 0x7f47d00a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:39.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.178+0000 7f47e9df3700 1 --2- 172.21.15.44:0/3377701078 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f47d00a0830 0x7f47d00a2cf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009bf0 tx=0x7f47e0009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:39.184 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.180+0000 7f47daffd700 1 -- 172.21.15.44:0/3377701078 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f47dc097430 con 0x7f47e410ab30 2023-09-10T01:37:39.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:38 smithi131 ceph-mon[103713]: pgmap v1857: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 14 op/s 2023-09-10T01:37:39.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:38 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2111054803' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T01:37:39.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:38 smithi099 ceph-mon[102692]: pgmap v1857: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 14 op/s 2023-09-10T01:37:39.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:38 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2111054803' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T01:37:39.386 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.385+0000 7f47ec057700 1 -- 172.21.15.44:0/3377701078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f47e41102f0 con 0x7f47e410ab30 2023-09-10T01:37:39.387 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.386+0000 7f47daffd700 1 -- 172.21.15.44:0/3377701078 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v213) v1 ==== 104+0+11 (secure 0 0 0) 0x7f47dc09aa50 con 0x7f47e410ab30 2023-09-10T01:37:39.387 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T01:37:39.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.388+0000 7f47ec057700 1 -- 172.21.15.44:0/3377701078 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f47d00a0830 msgr2=0x7f47d00a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:39.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.388+0000 7f47ec057700 1 --2- 172.21.15.44:0/3377701078 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f47d00a0830 0x7f47d00a2cf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009bf0 tx=0x7f47e0009260 comp rx=0 tx=0).stop 2023-09-10T01:37:39.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.388+0000 7f47ec057700 1 -- 172.21.15.44:0/3377701078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47e410ab30 msgr2=0x7f47e41965a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:39.389 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.388+0000 7f47ec057700 1 --2- 172.21.15.44:0/3377701078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47e410ab30 0x7f47e41965a0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00b780 tx=0x7f47dc00bab0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.388+0000 7f47ec057700 1 -- 172.21.15.44:0/3377701078 shutdown_connections 2023-09-10T01:37:39.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.388+0000 7f47ec057700 1 --2- 172.21.15.44:0/3377701078 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f47d00a0830 0x7f47d00a2cf0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.388+0000 7f47ec057700 1 --2- 172.21.15.44:0/3377701078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47e410ab30 0x7f47e41965a0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.388+0000 7f47ec057700 1 --2- 172.21.15.44:0/3377701078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47e4101710 0x7f47e419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.390 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.388+0000 7f47ec057700 1 --2- 172.21.15.44:0/3377701078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47e4102030 0x7f47e419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.388+0000 7f47ec057700 1 -- 172.21.15.44:0/3377701078 >> 172.21.15.44:0/3377701078 conn(0x7f47e40faf50 msgr2=0x7f47e40fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:39.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.388+0000 7f47ec057700 1 -- 172.21.15.44:0/3377701078 shutdown_connections 2023-09-10T01:37:39.391 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.388+0000 7f47ec057700 1 -- 172.21.15.44:0/3377701078 wait complete. 2023-09-10T01:37:39.401 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-10T01:37:39.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.531+0000 7f423f851700 1 -- 172.21.15.44:0/436125256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380ffe00 msgr2=0x7f423810c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:39.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.531+0000 7f423f851700 1 --2- 172.21.15.44:0/436125256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380ffe00 0x7f423810c310 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f423000b0a0 tx=0x7f423000b3b0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.531+0000 7f423f851700 1 -- 172.21.15.44:0/436125256 shutdown_connections 2023-09-10T01:37:39.534 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.531+0000 7f423f851700 1 --2- 172.21.15.44:0/436125256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380ffe00 0x7f423810c310 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.531+0000 7f423f851700 1 --2- 172.21.15.44:0/436125256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42381054e0 0x7f42381058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.531+0000 7f423f851700 1 --2- 172.21.15.44:0/436125256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42380ff440 0x7f42380ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.531+0000 7f423f851700 1 -- 172.21.15.44:0/436125256 >> 172.21.15.44:0/436125256 conn(0x7f42380faf50 msgr2=0x7f42380fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:39.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.531+0000 7f423f851700 1 -- 172.21.15.44:0/436125256 shutdown_connections 2023-09-10T01:37:39.535 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.531+0000 7f423f851700 1 -- 172.21.15.44:0/436125256 wait complete. 2023-09-10T01:37:39.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423f851700 1 Processor -- start 2023-09-10T01:37:39.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423f851700 1 -- start start 2023-09-10T01:37:39.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423f851700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380ff440 0x7f42381974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:39.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423f851700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42380ffe00 0x7f4238197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:39.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423f851700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42381054e0 0x7f42381a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:39.536 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423f851700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4238198300 con 0x7f42380ff440 2023-09-10T01:37:39.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423f851700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42381a2aa0 con 0x7f42380ffe00 2023-09-10T01:37:39.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423f851700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42381a2be0 con 0x7f42381054e0 2023-09-10T01:37:39.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423d5ed700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380ff440 0x7f42381974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:39.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423d5ed700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380ff440 0x7f42381974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33426/0 (socket says 172.21.15.44:33426) 2023-09-10T01:37:39.537 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423d5ed700 1 -- 172.21.15.44:0/2289347938 learned_addr learned my addr 172.21.15.44:0/2289347938 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:39.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423cdec700 1 --2- 172.21.15.44:0/2289347938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42380ffe00 0x7f4238197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:39.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423d5ed700 1 -- 172.21.15.44:0/2289347938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42381054e0 msgr2=0x7f42381a2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:39.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423d5ed700 1 --2- 172.21.15.44:0/2289347938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42381054e0 0x7f42381a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423d5ed700 1 -- 172.21.15.44:0/2289347938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42380ffe00 msgr2=0x7f4238197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:39.538 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423d5ed700 1 --2- 172.21.15.44:0/2289347938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42380ffe00 0x7f4238197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423d5ed700 1 -- 172.21.15.44:0/2289347938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4230009d00 con 0x7f42380ff440 2023-09-10T01:37:39.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.532+0000 7f423d5ed700 1 --2- 172.21.15.44:0/2289347938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380ff440 0x7f42381974e0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f422800b550 tx=0x7f422800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:39.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.533+0000 7f422e7fc700 1 -- 172.21.15.44:0/2289347938 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228011830 con 0x7f42380ff440 2023-09-10T01:37:39.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.533+0000 7f422e7fc700 1 -- 172.21.15.44:0/2289347938 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4228011e20 con 0x7f42380ff440 2023-09-10T01:37:39.539 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.533+0000 7f423f851700 1 -- 172.21.15.44:0/2289347938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42381a2ec0 con 0x7f42380ff440 2023-09-10T01:37:39.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.533+0000 7f422e7fc700 1 -- 172.21.15.44:0/2289347938 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228010e30 con 0x7f42380ff440 2023-09-10T01:37:39.540 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.533+0000 7f423f851700 1 -- 172.21.15.44:0/2289347938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42381a3340 con 0x7f42380ff440 2023-09-10T01:37:39.541 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.533+0000 7f423f851700 1 -- 172.21.15.44:0/2289347938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4238067380 con 0x7f42380ff440 2023-09-10T01:37:39.541 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.535+0000 7f422e7fc700 1 -- 172.21.15.44:0/2289347938 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f4228011990 con 0x7f42380ff440 2023-09-10T01:37:39.541 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.535+0000 7f422e7fc700 1 --2- 172.21.15.44:0/2289347938 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f42240a07b0 0x7f42240a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:39.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.536+0000 7f423cdec700 1 --2- 172.21.15.44:0/2289347938 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f42240a07b0 0x7f42240a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:39.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.536+0000 7f422e7fc700 1 -- 172.21.15.44:0/2289347938 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f42280cc0a0 con 0x7f42380ff440 2023-09-10T01:37:39.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.536+0000 7f423cdec700 1 --2- 172.21.15.44:0/2289347938 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f42240a07b0 0x7f42240a2c70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4238067fe0 tx=0x7f423400b410 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:39.542 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.539+0000 7f422e7fc700 1 -- 172.21.15.44:0/2289347938 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f42280955a0 con 0x7f42380ff440 2023-09-10T01:37:39.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.748+0000 7f423f851700 1 -- 172.21.15.44:0/2289347938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f423810f450 con 0x7f42380ff440 2023-09-10T01:37:39.749 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.749+0000 7f422e7fc700 1 -- 172.21.15.44:0/2289347938 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v213) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4228098bc0 con 0x7f42380ff440 2023-09-10T01:37:39.750 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2023-09-10T01:37:39.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.752+0000 7f423f851700 1 -- 172.21.15.44:0/2289347938 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f42240a07b0 msgr2=0x7f42240a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:39.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.752+0000 7f423f851700 1 --2- 172.21.15.44:0/2289347938 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f42240a07b0 0x7f42240a2c70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4238067fe0 tx=0x7f423400b410 comp rx=0 tx=0).stop 2023-09-10T01:37:39.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.752+0000 7f423f851700 1 -- 172.21.15.44:0/2289347938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380ff440 msgr2=0x7f42381974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:39.753 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.752+0000 7f423f851700 1 --2- 172.21.15.44:0/2289347938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380ff440 0x7f42381974e0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f422800b550 tx=0x7f422800b860 comp rx=0 tx=0).stop 2023-09-10T01:37:39.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.752+0000 7f423f851700 1 -- 172.21.15.44:0/2289347938 shutdown_connections 2023-09-10T01:37:39.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.752+0000 7f423f851700 1 --2- 172.21.15.44:0/2289347938 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f42240a07b0 0x7f42240a2c70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.752+0000 7f423f851700 1 --2- 172.21.15.44:0/2289347938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42381054e0 0x7f42381a2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.752+0000 7f423f851700 1 --2- 172.21.15.44:0/2289347938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380ff440 0x7f42381974e0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.754 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.752+0000 7f423f851700 1 --2- 172.21.15.44:0/2289347938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42380ffe00 0x7f4238197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.752+0000 7f423f851700 1 -- 172.21.15.44:0/2289347938 >> 172.21.15.44:0/2289347938 conn(0x7f42380faf50 msgr2=0x7f42380fb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:39.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.752+0000 7f423f851700 1 -- 172.21.15.44:0/2289347938 shutdown_connections 2023-09-10T01:37:39.755 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.752+0000 7f423f851700 1 -- 172.21.15.44:0/2289347938 wait complete. 2023-09-10T01:37:39.766 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-10T01:37:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.898+0000 7f3e3f39b700 1 -- 172.21.15.44:0/3359297381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38102030 msgr2=0x7f3e3810a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:39.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.898+0000 7f3e3f39b700 1 --2- 172.21.15.44:0/3359297381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38102030 0x7f3e3810a510 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009990 tx=0x7f3e2c009ca0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.902 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.898+0000 7f3e3f39b700 1 -- 172.21.15.44:0/3359297381 shutdown_connections 2023-09-10T01:37:39.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.898+0000 7f3e3f39b700 1 --2- 172.21.15.44:0/3359297381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e3810ab30 0x7f3e3810cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.898+0000 7f3e3f39b700 1 --2- 172.21.15.44:0/3359297381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e38101710 0x7f3e38101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.898+0000 7f3e3f39b700 1 --2- 172.21.15.44:0/3359297381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38102030 0x7f3e3810a510 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.903 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.898+0000 7f3e3f39b700 1 -- 172.21.15.44:0/3359297381 >> 172.21.15.44:0/3359297381 conn(0x7f3e380faf50 msgr2=0x7f3e380fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:39.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.898+0000 7f3e3f39b700 1 -- 172.21.15.44:0/3359297381 shutdown_connections 2023-09-10T01:37:39.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.898+0000 7f3e3f39b700 1 -- 172.21.15.44:0/3359297381 wait complete. 2023-09-10T01:37:39.904 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3f39b700 1 Processor -- start 2023-09-10T01:37:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3f39b700 1 -- start start 2023-09-10T01:37:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3f39b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38101710 0x7f3e381965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3f39b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e38102030 0x7f3e38196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3f39b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e3810ab30 0x7f3e3819af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3f39b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3819b5a0 con 0x7f3e38101710 2023-09-10T01:37:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3f39b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3819b710 con 0x7f3e38102030 2023-09-10T01:37:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3f39b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3819b8d0 con 0x7f3e3810ab30 2023-09-10T01:37:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3d137700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38101710 0x7f3e381965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3d137700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38101710 0x7f3e381965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33440/0 (socket says 172.21.15.44:33440) 2023-09-10T01:37:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3d137700 1 -- 172.21.15.44:0/2965350125 learned_addr learned my addr 172.21.15.44:0/2965350125 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3c936700 1 --2- 172.21.15.44:0/2965350125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e38102030 0x7f3e38196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3d938700 1 --2- 172.21.15.44:0/2965350125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e3810ab30 0x7f3e3819af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3d137700 1 -- 172.21.15.44:0/2965350125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e3810ab30 msgr2=0x7f3e3819af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3d137700 1 --2- 172.21.15.44:0/2965350125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e3810ab30 0x7f3e3819af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3d137700 1 -- 172.21.15.44:0/2965350125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e38102030 msgr2=0x7f3e38196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3d137700 1 --2- 172.21.15.44:0/2965350125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e38102030 0x7f3e38196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:39.910 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3d137700 1 -- 172.21.15.44:0/2965350125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2c0095f0 con 0x7f3e38101710 2023-09-10T01:37:39.910 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.899+0000 7f3e3d938700 1 --2- 172.21.15.44:0/2965350125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e3810ab30 0x7f3e3819af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:37:39.910 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.900+0000 7f3e3d137700 1 --2- 172.21.15.44:0/2965350125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38101710 0x7f3e381965c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f3e300060b0 tx=0x7f3e3000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.900+0000 7f3e367fc700 1 -- 172.21.15.44:0/2965350125 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000ddd0 con 0x7f3e38101710 2023-09-10T01:37:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.900+0000 7f3e367fc700 1 -- 172.21.15.44:0/2965350125 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e3000f040 con 0x7f3e38101710 2023-09-10T01:37:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.900+0000 7f3e3f39b700 1 -- 172.21.15.44:0/2965350125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3819bbb0 con 0x7f3e38101710 2023-09-10T01:37:39.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.900+0000 7f3e367fc700 1 -- 172.21.15.44:0/2965350125 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e30014600 con 0x7f3e38101710 2023-09-10T01:37:39.912 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.900+0000 7f3e3f39b700 1 -- 172.21.15.44:0/2965350125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e381a33f0 con 0x7f3e38101710 2023-09-10T01:37:39.913 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.901+0000 7f3e3f39b700 1 -- 172.21.15.44:0/2965350125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e38105c00 con 0x7f3e38101710 2023-09-10T01:37:39.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.902+0000 7f3e367fc700 1 -- 172.21.15.44:0/2965350125 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f3e30015400 con 0x7f3e38101710 2023-09-10T01:37:39.914 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.902+0000 7f3e367fc700 1 --2- 172.21.15.44:0/2965350125 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3e200a0830 0x7f3e200a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:39.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.903+0000 7f3e367fc700 1 -- 172.21.15.44:0/2965350125 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f3e300cbd10 con 0x7f3e38101710 2023-09-10T01:37:39.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.903+0000 7f3e3c936700 1 --2- 172.21.15.44:0/2965350125 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3e200a0830 0x7f3e200a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:39.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.903+0000 7f3e3c936700 1 --2- 172.21.15.44:0/2965350125 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3e200a0830 0x7f3e200a2cf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3e38197bd0 tx=0x7f3e2c005fb0 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:39.915 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:39.907+0000 7f3e367fc700 1 -- 172.21.15.44:0/2965350125 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3e30095100 con 0x7f3e38101710 2023-09-10T01:37:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:39 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/325742330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T01:37:40.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:39 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3377701078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T01:37:40.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:39 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:37:40.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:39 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2289347938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T01:37:40.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.132+0000 7f3e3f39b700 1 -- 172.21.15.44:0/2965350125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3e38110390 con 0x7f3e38101710 2023-09-10T01:37:40.133 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.132+0000 7f3e367fc700 1 -- 172.21.15.44:0/2965350125 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v213) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3e3001a020 con 0x7f3e38101710 2023-09-10T01:37:40.134 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T01:37:40.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.135+0000 7f3e3f39b700 1 -- 172.21.15.44:0/2965350125 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3e200a0830 msgr2=0x7f3e200a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:40.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.135+0000 7f3e3f39b700 1 --2- 172.21.15.44:0/2965350125 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3e200a0830 0x7f3e200a2cf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3e38197bd0 tx=0x7f3e2c005fb0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.135+0000 7f3e3f39b700 1 -- 172.21.15.44:0/2965350125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38101710 msgr2=0x7f3e381965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:40.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.135+0000 7f3e3f39b700 1 --2- 172.21.15.44:0/2965350125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38101710 0x7f3e381965c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f3e300060b0 tx=0x7f3e3000d3b0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.136 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.135+0000 7f3e3f39b700 1 -- 172.21.15.44:0/2965350125 shutdown_connections 2023-09-10T01:37:40.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.135+0000 7f3e3f39b700 1 --2- 172.21.15.44:0/2965350125 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f3e200a0830 0x7f3e200a2cf0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.135+0000 7f3e3f39b700 1 --2- 172.21.15.44:0/2965350125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e3810ab30 0x7f3e3819af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.135+0000 7f3e3f39b700 1 --2- 172.21.15.44:0/2965350125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38101710 0x7f3e381965c0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.135+0000 7f3e3f39b700 1 --2- 172.21.15.44:0/2965350125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e38102030 0x7f3e38196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.137 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.135+0000 7f3e3f39b700 1 -- 172.21.15.44:0/2965350125 >> 172.21.15.44:0/2965350125 conn(0x7f3e380faf50 msgr2=0x7f3e380fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:40.138 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.135+0000 7f3e3f39b700 1 -- 172.21.15.44:0/2965350125 shutdown_connections 2023-09-10T01:37:40.138 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.135+0000 7f3e3f39b700 1 -- 172.21.15.44:0/2965350125 wait complete. 2023-09-10T01:37:40.149 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-10T01:37:40.279 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.278+0000 7fdb336f4700 1 -- 172.21.15.44:0/2237148435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb2c0ff310 msgr2=0x7fdb2c0ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:40.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.278+0000 7fdb336f4700 1 --2- 172.21.15.44:0/2237148435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb2c0ff310 0x7fdb2c0ff6f0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009990 tx=0x7fdb28009ca0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.278+0000 7fdb336f4700 1 -- 172.21.15.44:0/2237148435 shutdown_connections 2023-09-10T01:37:40.280 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.278+0000 7fdb336f4700 1 --2- 172.21.15.44:0/2237148435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb2c10cec0 0x7fdb2c10f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.281 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.278+0000 7fdb336f4700 1 --2- 172.21.15.44:0/2237148435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb2c0ff310 0x7fdb2c0ff6f0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.281 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.278+0000 7fdb336f4700 1 --2- 172.21.15.44:0/2237148435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb2c0ffcc0 0x7fdb2c10c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.281 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.278+0000 7fdb336f4700 1 -- 172.21.15.44:0/2237148435 >> 172.21.15.44:0/2237148435 conn(0x7fdb2c0faf50 msgr2=0x7fdb2c0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:40.281 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.278+0000 7fdb336f4700 1 -- 172.21.15.44:0/2237148435 shutdown_connections 2023-09-10T01:37:40.281 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.278+0000 7fdb336f4700 1 -- 172.21.15.44:0/2237148435 wait complete. 2023-09-10T01:37:40.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.279+0000 7fdb336f4700 1 Processor -- start 2023-09-10T01:37:40.282 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.279+0000 7fdb336f4700 1 -- start start 2023-09-10T01:37:40.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.279+0000 7fdb336f4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb2c0ff310 0x7fdb2c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:40.284 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.279+0000 7fdb336f4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb2c0ffcc0 0x7fdb2c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:40.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.279+0000 7fdb336f4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb2c10cec0 0x7fdb2c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:40.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb336f4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c198c60 con 0x7fdb2c10cec0 2023-09-10T01:37:40.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb336f4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c198dd0 con 0x7fdb2c0ffcc0 2023-09-10T01:37:40.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb336f4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c198f90 con 0x7fdb2c0ff310 2023-09-10T01:37:40.285 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb31c91700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb2c10cec0 0x7fdb2c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:40.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb31490700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb2c0ff310 0x7fdb2c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:40.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb30c8f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb2c0ffcc0 0x7fdb2c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:40.286 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb31c91700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb2c10cec0 0x7fdb2c1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33458/0 (socket says 172.21.15.44:33458) 2023-09-10T01:37:40.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb31c91700 1 -- 172.21.15.44:0/1534657974 learned_addr learned my addr 172.21.15.44:0/1534657974 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:40.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb30c8f700 1 -- 172.21.15.44:0/1534657974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb2c0ff310 msgr2=0x7fdb2c19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:40.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb30c8f700 1 --2- 172.21.15.44:0/1534657974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb2c0ff310 0x7fdb2c19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.287 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb30c8f700 1 -- 172.21.15.44:0/1534657974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb2c10cec0 msgr2=0x7fdb2c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:40.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb30c8f700 1 --2- 172.21.15.44:0/1534657974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb2c10cec0 0x7fdb2c1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb30c8f700 1 -- 172.21.15.44:0/1534657974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb280095f0 con 0x7fdb2c0ffcc0 2023-09-10T01:37:40.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb31c91700 1 --2- 172.21.15.44:0/1534657974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb2c10cec0 0x7fdb2c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-10T01:37:40.288 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb30c8f700 1 --2- 172.21.15.44:0/1534657974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb2c0ffcc0 0x7fdb2c19eac0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00e9f0 tx=0x7fdb1c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:40.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb227fc700 1 -- 172.21.15.44:0/1534657974 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c00c9c0 con 0x7fdb2c0ffcc0 2023-09-10T01:37:40.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb227fc700 1 -- 172.21.15.44:0/1534657974 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb1c010430 con 0x7fdb2c0ffcc0 2023-09-10T01:37:40.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb227fc700 1 -- 172.21.15.44:0/1534657974 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c018680 con 0x7fdb2c0ffcc0 2023-09-10T01:37:40.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb336f4700 1 -- 172.21.15.44:0/1534657974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb2c199270 con 0x7fdb2c0ffcc0 2023-09-10T01:37:40.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.280+0000 7fdb336f4700 1 -- 172.21.15.44:0/1534657974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb2c1a5570 con 0x7fdb2c0ffcc0 2023-09-10T01:37:40.289 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.281+0000 7fdb336f4700 1 -- 172.21.15.44:0/1534657974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb2c1080e0 con 0x7fdb2c0ffcc0 2023-09-10T01:37:40.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.283+0000 7fdb227fc700 1 -- 172.21.15.44:0/1534657974 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fdb1c0105a0 con 0x7fdb2c0ffcc0 2023-09-10T01:37:40.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.283+0000 7fdb227fc700 1 --2- 172.21.15.44:0/1534657974 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdb180a07b0 0x7fdb180a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:40.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.283+0000 7fdb227fc700 1 -- 172.21.15.44:0/1534657974 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fdb1c014070 con 0x7fdb2c0ffcc0 2023-09-10T01:37:40.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.283+0000 7fdb31490700 1 --2- 172.21.15.44:0/1534657974 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdb180a07b0 0x7fdb180a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:40.290 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.284+0000 7fdb31490700 1 --2- 172.21.15.44:0/1534657974 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdb180a07b0 0x7fdb180a2c70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdb280059d0 tx=0x7fdb28005960 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:40.291 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.287+0000 7fdb227fc700 1 -- 172.21.15.44:0/1534657974 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fdb1c095bd0 con 0x7fdb2c0ffcc0 2023-09-10T01:37:40.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:39 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/325742330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T01:37:40.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:39 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3377701078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T01:37:40.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:39 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:37:40.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:39 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2289347938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T01:37:40.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:39 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/325742330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T01:37:40.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:39 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3377701078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T01:37:40.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:39 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:37:40.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:39 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2289347938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T01:37:40.508 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.508+0000 7fdb336f4700 1 -- 172.21.15.44:0/1534657974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdb2c067380 con 0x7fdb2c0ffcc0 2023-09-10T01:37:40.509 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.508+0000 7fdb227fc700 1 -- 172.21.15.44:0/1534657974 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v213) v1 ==== 81+0+937 (secure 0 0 0) 0x7fdb1c0991f0 con 0x7fdb2c0ffcc0 2023-09-10T01:37:40.509 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/902389643 2023-09-11T00:40:38.970014+0000 2023-09-10T01:37:40.510 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/3642136197 2023-09-11T00:40:38.970014+0000 2023-09-10T01:37:40.510 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/3766354728 2023-09-11T00:40:38.970014+0000 2023-09-10T01:37:40.510 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/1488156576 2023-09-11T00:40:38.970014+0000 2023-09-10T01:37:40.510 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/1488156576 2023-09-11T00:40:38.970014+0000 2023-09-10T01:37:40.510 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/18127588 2023-09-11T00:32:15.913770+0000 2023-09-10T01:37:40.511 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/18127588 2023-09-11T00:32:15.913770+0000 2023-09-10T01:37:40.511 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1414498756 2023-09-11T00:32:15.913770+0000 2023-09-10T01:37:40.511 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1298831914 2023-09-11T00:32:15.913770+0000 2023-09-10T01:37:40.511 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2415771009 2023-09-11T00:31:49.496807+0000 2023-09-10T01:37:40.511 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/193660691 2023-09-11T00:31:49.496807+0000 2023-09-10T01:37:40.512 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/871815312 2023-09-11T00:31:49.496807+0000 2023-09-10T01:37:40.512 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1968699888 2023-09-11T00:40:38.970014+0000 2023-09-10T01:37:40.512 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1481976351 2023-09-11T00:32:15.913770+0000 2023-09-10T01:37:40.512 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/871815312 2023-09-11T00:31:49.496807+0000 2023-09-10T01:37:40.512 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/625830807 2023-09-11T00:31:49.496807+0000 2023-09-10T01:37:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.511+0000 7fdb336f4700 1 -- 172.21.15.44:0/1534657974 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdb180a07b0 msgr2=0x7fdb180a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.511+0000 7fdb336f4700 1 --2- 172.21.15.44:0/1534657974 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdb180a07b0 0x7fdb180a2c70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdb280059d0 tx=0x7fdb28005960 comp rx=0 tx=0).stop 2023-09-10T01:37:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.511+0000 7fdb336f4700 1 -- 172.21.15.44:0/1534657974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb2c0ffcc0 msgr2=0x7fdb2c19eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.511+0000 7fdb336f4700 1 --2- 172.21.15.44:0/1534657974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb2c0ffcc0 0x7fdb2c19eac0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00e9f0 tx=0x7fdb1c00ed00 comp rx=0 tx=0).stop 2023-09-10T01:37:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.511+0000 7fdb336f4700 1 -- 172.21.15.44:0/1534657974 shutdown_connections 2023-09-10T01:37:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.511+0000 7fdb336f4700 1 --2- 172.21.15.44:0/1534657974 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fdb180a07b0 0x7fdb180a2c70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.511+0000 7fdb336f4700 1 --2- 172.21.15.44:0/1534657974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb2c10cec0 0x7fdb2c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.511+0000 7fdb336f4700 1 --2- 172.21.15.44:0/1534657974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb2c0ff310 0x7fdb2c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.511+0000 7fdb336f4700 1 --2- 172.21.15.44:0/1534657974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb2c0ffcc0 0x7fdb2c19eac0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.511+0000 7fdb336f4700 1 -- 172.21.15.44:0/1534657974 >> 172.21.15.44:0/1534657974 conn(0x7fdb2c0faf50 msgr2=0x7fdb2c10f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.511+0000 7fdb336f4700 1 -- 172.21.15.44:0/1534657974 shutdown_connections 2023-09-10T01:37:40.516 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.511+0000 7fdb336f4700 1 -- 172.21.15.44:0/1534657974 wait complete. 2023-09-10T01:37:40.516 INFO:teuthology.orchestra.run.smithi044.stderr:listed 16 entries 2023-09-10T01:37:40.526 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2023-09-10T01:37:40.526 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-10T01:37:40.547 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-10T01:37:40.720 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.717+0000 7fca8eb4d700 1 -- 172.21.15.44:0/1125281318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca880ff310 msgr2=0x7fca880ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:40.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.717+0000 7fca8eb4d700 1 --2- 172.21.15.44:0/1125281318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca880ff310 0x7fca880ff6f0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fca74009990 tx=0x7fca74009ca0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.717+0000 7fca8eb4d700 1 -- 172.21.15.44:0/1125281318 shutdown_connections 2023-09-10T01:37:40.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.717+0000 7fca8eb4d700 1 --2- 172.21.15.44:0/1125281318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8810cec0 0x7fca8810f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.717+0000 7fca8eb4d700 1 --2- 172.21.15.44:0/1125281318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca880ff310 0x7fca880ff6f0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.721 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.717+0000 7fca8eb4d700 1 --2- 172.21.15.44:0/1125281318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca880ffcc0 0x7fca8810c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.717+0000 7fca8eb4d700 1 -- 172.21.15.44:0/1125281318 >> 172.21.15.44:0/1125281318 conn(0x7fca880faf10 msgr2=0x7fca880fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:40.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.717+0000 7fca8eb4d700 1 -- 172.21.15.44:0/1125281318 shutdown_connections 2023-09-10T01:37:40.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.718+0000 7fca8eb4d700 1 -- 172.21.15.44:0/1125281318 wait complete. 2023-09-10T01:37:40.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.718+0000 7fca8eb4d700 1 Processor -- start 2023-09-10T01:37:40.722 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.718+0000 7fca8eb4d700 1 -- start start 2023-09-10T01:37:40.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.718+0000 7fca8eb4d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca880ff310 0x7fca8819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:40.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.718+0000 7fca8eb4d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca880ffcc0 0x7fca8819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:40.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.718+0000 7fca8eb4d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8810cec0 0x7fca881985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:40.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.718+0000 7fca8eb4d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca88198c20 con 0x7fca880ffcc0 2023-09-10T01:37:40.723 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.718+0000 7fca8eb4d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca88198d90 con 0x7fca880ff310 2023-09-10T01:37:40.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.718+0000 7fca8eb4d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca88198f50 con 0x7fca8810cec0 2023-09-10T01:37:40.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.718+0000 7fca87fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca880ffcc0 0x7fca8819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:40.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.718+0000 7fca87fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca880ffcc0 0x7fca8819ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33470/0 (socket says 172.21.15.44:33470) 2023-09-10T01:37:40.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.718+0000 7fca87fff700 1 -- 172.21.15.44:0/625751875 learned_addr learned my addr 172.21.15.44:0/625751875 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:40.724 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.719+0000 7fca8d0ea700 1 --2- 172.21.15.44:0/625751875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8810cec0 0x7fca881985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:40.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.719+0000 7fca8c8e9700 1 --2- 172.21.15.44:0/625751875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca880ff310 0x7fca8819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:40.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.719+0000 7fca87fff700 1 -- 172.21.15.44:0/625751875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8810cec0 msgr2=0x7fca881985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:40.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.719+0000 7fca87fff700 1 --2- 172.21.15.44:0/625751875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8810cec0 0x7fca881985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.719+0000 7fca87fff700 1 -- 172.21.15.44:0/625751875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca880ff310 msgr2=0x7fca8819e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:40.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.719+0000 7fca87fff700 1 --2- 172.21.15.44:0/625751875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca880ff310 0x7fca8819e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.725 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.719+0000 7fca87fff700 1 -- 172.21.15.44:0/625751875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca740095f0 con 0x7fca880ffcc0 2023-09-10T01:37:40.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.719+0000 7fca8c8e9700 1 --2- 172.21.15.44:0/625751875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca880ff310 0x7fca8819e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:37:40.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.719+0000 7fca87fff700 1 --2- 172.21.15.44:0/625751875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca880ffcc0 0x7fca8819ea80 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00d950 tx=0x7fca7c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:40.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.719+0000 7fca85ffb700 1 -- 172.21.15.44:0/625751875 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c009940 con 0x7fca880ffcc0 2023-09-10T01:37:40.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.719+0000 7fca85ffb700 1 -- 172.21.15.44:0/625751875 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fca7c010460 con 0x7fca880ffcc0 2023-09-10T01:37:40.726 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.719+0000 7fca8eb4d700 1 -- 172.21.15.44:0/625751875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca88199230 con 0x7fca880ffcc0 2023-09-10T01:37:40.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.719+0000 7fca85ffb700 1 -- 172.21.15.44:0/625751875 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c00f620 con 0x7fca880ffcc0 2023-09-10T01:37:40.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.719+0000 7fca8eb4d700 1 -- 172.21.15.44:0/625751875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca881a5530 con 0x7fca880ffcc0 2023-09-10T01:37:40.727 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.720+0000 7fca8eb4d700 1 -- 172.21.15.44:0/625751875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca881080e0 con 0x7fca880ffcc0 2023-09-10T01:37:40.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.721+0000 7fca85ffb700 1 -- 172.21.15.44:0/625751875 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fca7c0105d0 con 0x7fca880ffcc0 2023-09-10T01:37:40.728 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.722+0000 7fca85ffb700 1 --2- 172.21.15.44:0/625751875 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fca780a0870 0x7fca780a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:40.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.722+0000 7fca8c8e9700 1 --2- 172.21.15.44:0/625751875 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fca780a0870 0x7fca780a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:40.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.722+0000 7fca85ffb700 1 -- 172.21.15.44:0/625751875 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fca7c0cc170 con 0x7fca880ffcc0 2023-09-10T01:37:40.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.722+0000 7fca8c8e9700 1 --2- 172.21.15.44:0/625751875 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fca780a0870 0x7fca780a2d30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fca740095c0 tx=0x7fca74009510 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:40.729 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.726+0000 7fca85ffb700 1 -- 172.21.15.44:0/625751875 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fca7c095560 con 0x7fca880ffcc0 2023-09-10T01:37:40.938 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.937+0000 7fca8eb4d700 1 -- 172.21.15.44:0/625751875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca880673f0 con 0x7fca880ffcc0 2023-09-10T01:37:40.941 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.940+0000 7fca85ffb700 1 -- 172.21.15.44:0/625751875 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v213) v1 ==== 74+0+25247 (secure 0 0 0) 0x7fca7c098b80 con 0x7fca880ffcc0 2023-09-10T01:37:40.941 INFO:teuthology.orchestra.run.smithi044.stdout: 2023-09-10T01:37:40.942 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":213,"fsid":"3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da","created":"2023-09-10T00:31:16.073750+0000","modified":"2023-09-10T01:37:16.783072+0000","last_up_change":"2023-09-10T00:37:57.786066+0000","last_in_change":"2023-09-10T00:37:39.470316+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-10T00:34:44.456741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-10T00:38:37.194891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-10T00:38:38.573268+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":108,"snap_epoch":212,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-10T00:38:42.198896+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":108,"snap_epoch":212,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a7d2e1e-5817-489e-af0e-72e9bcccc119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6803","nonce":424109980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6805","nonce":424109980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6809","nonce":424109980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":424109980},{"type":"v1","addr":"172.21.15.44:6807","nonce":424109980}]},"public_addr":"172.21.15.44:6803/424109980","cluster_addr":"172.21.15.44:6805/424109980","heartbeat_back_addr":"172.21.15.44:6809/424109980","heartbeat_front_addr":"172.21.15.44:6807/424109980","state":["exists","up"]},{"osd":1,"uuid":"cf261516-01eb-4f04-b7be-19f32e9e55e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6801","nonce":63099758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6803","nonce":63099758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6807","nonce":63099758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":63099758},{"type":"v1","addr":"172.21.15.99:6805","nonce":63099758}]},"public_addr":"172.21.15.99:6801/63099758","cluster_addr":"172.21.15.99:6803/63099758","heartbeat_back_addr":"172.21.15.99:6807/63099758","heartbeat_front_addr":"172.21.15.99:6805/63099758","state":["exists","up"]},{"osd":2,"uuid":"1d73ba44-ef7a-495b-97cb-550517aaab45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6801","nonce":3245285432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6803","nonce":3245285432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6807","nonce":3245285432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3245285432},{"type":"v1","addr":"172.21.15.131:6805","nonce":3245285432}]},"public_addr":"172.21.15.131:6801/3245285432","cluster_addr":"172.21.15.131:6803/3245285432","heartbeat_back_addr":"172.21.15.131:6807/3245285432","heartbeat_front_addr":"172.21.15.131:6805/3245285432","state":["exists","up"]},{"osd":3,"uuid":"e6fe9647-238c-4f5c-a933-ace40dc5bc0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6811","nonce":2588997584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6813","nonce":2588997584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6817","nonce":2588997584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2588997584},{"type":"v1","addr":"172.21.15.44:6815","nonce":2588997584}]},"public_addr":"172.21.15.44:6811/2588997584","cluster_addr":"172.21.15.44:6813/2588997584","heartbeat_back_addr":"172.21.15.44:6817/2588997584","heartbeat_front_addr":"172.21.15.44:6815/2588997584","state":["exists","up"]},{"osd":4,"uuid":"1ae06a56-c8b7-4881-8ead-68835a9015a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6809","nonce":532684490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6811","nonce":532684490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6815","nonce":532684490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":532684490},{"type":"v1","addr":"172.21.15.99:6813","nonce":532684490}]},"public_addr":"172.21.15.99:6809/532684490","cluster_addr":"172.21.15.99:6811/532684490","heartbeat_back_addr":"172.21.15.99:6815/532684490","heartbeat_front_addr":"172.21.15.99:6813/532684490","state":["exists","up"]},{"osd":5,"uuid":"917c392a-bb6c-408b-80c6-9d81c194b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6809","nonce":979810489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6811","nonce":979810489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6815","nonce":979810489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":979810489},{"type":"v1","addr":"172.21.15.131:6813","nonce":979810489}]},"public_addr":"172.21.15.131:6809/979810489","cluster_addr":"172.21.15.131:6811/979810489","heartbeat_back_addr":"172.21.15.131:6815/979810489","heartbeat_front_addr":"172.21.15.131:6813/979810489","state":["exists","up"]},{"osd":6,"uuid":"4519b9ce-f8c7-4afc-bd85-a9a9d82e8a1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6819","nonce":1858317767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6821","nonce":1858317767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6825","nonce":1858317767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1858317767},{"type":"v1","addr":"172.21.15.44:6823","nonce":1858317767}]},"public_addr":"172.21.15.44:6819/1858317767","cluster_addr":"172.21.15.44:6821/1858317767","heartbeat_back_addr":"172.21.15.44:6825/1858317767","heartbeat_front_addr":"172.21.15.44:6823/1858317767","state":["exists","up"]},{"osd":7,"uuid":"4cfd6428-45b9-4884-973d-6f980996fe99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6817","nonce":107594338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6819","nonce":107594338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6823","nonce":107594338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":107594338},{"type":"v1","addr":"172.21.15.99:6821","nonce":107594338}]},"public_addr":"172.21.15.99:6817/107594338","cluster_addr":"172.21.15.99:6819/107594338","heartbeat_back_addr":"172.21.15.99:6823/107594338","heartbeat_front_addr":"172.21.15.99:6821/107594338","state":["exists","up"]},{"osd":8,"uuid":"81cefab0-1fc4-46ef-a468-6cc1cc5f3d2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6817","nonce":1528398267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6819","nonce":1528398267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6823","nonce":1528398267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1528398267},{"type":"v1","addr":"172.21.15.131:6821","nonce":1528398267}]},"public_addr":"172.21.15.131:6817/1528398267","cluster_addr":"172.21.15.131:6819/1528398267","heartbeat_back_addr":"172.21.15.131:6823/1528398267","heartbeat_front_addr":"172.21.15.131:6821/1528398267","state":["exists","up"]},{"osd":9,"uuid":"fa68741f-6ffa-4525-bfce-4b29c806bc53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6827","nonce":1885092640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6829","nonce":1885092640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6833","nonce":1885092640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1885092640},{"type":"v1","addr":"172.21.15.44:6831","nonce":1885092640}]},"public_addr":"172.21.15.44:6827/1885092640","cluster_addr":"172.21.15.44:6829/1885092640","heartbeat_back_addr":"172.21.15.44:6833/1885092640","heartbeat_front_addr":"172.21.15.44:6831/1885092640","state":["exists","up"]},{"osd":10,"uuid":"aee4c5aa-0fb1-447d-948f-61cd27d5eecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6825","nonce":1454470032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6827","nonce":1454470032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6831","nonce":1454470032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1454470032},{"type":"v1","addr":"172.21.15.99:6829","nonce":1454470032}]},"public_addr":"172.21.15.99:6825/1454470032","cluster_addr":"172.21.15.99:6827/1454470032","heartbeat_back_addr":"172.21.15.99:6831/1454470032","heartbeat_front_addr":"172.21.15.99:6829/1454470032","state":["exists","up"]},{"osd":11,"uuid":"f8d68bdc-aa91-4e82-9965-f74b9efa488a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6825","nonce":1396964632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6827","nonce":1396964632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6831","nonce":1396964632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1396964632},{"type":"v1","addr":"172.21.15.131:6829","nonce":1396964632}]},"public_addr":"172.21.15.131:6825/1396964632","cluster_addr":"172.21.15.131:6827/1396964632","heartbeat_back_addr":"172.21.15.131:6831/1396964632","heartbeat_front_addr":"172.21.15.131:6829/1396964632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:33:59.942076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:20.697256+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:34:42.047513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:04.888406+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:25.539856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:35:47.112857+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:08.446825+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:29.802703+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:36:51.391194+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:12.960496+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:33.626473+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-10T00:37:55.859799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/902389643":"2023-09-11T00:40:38.970014+0000","172.21.15.44:0/3642136197":"2023-09-11T00:40:38.970014+0000","172.21.15.44:0/3766354728":"2023-09-11T00:40:38.970014+0000","172.21.15.44:6801/1488156576":"2023-09-11T00:40:38.970014+0000","172.21.15.44:6800/1488156576":"2023-09-11T00:40:38.970014+0000","172.21.15.44:6801/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6800/18127588":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1414498756":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/1298831914":"2023-09-11T00:32:15.913770+0000","172.21.15.44:0/2415771009":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/193660691":"2023-09-11T00:31:49.496807+0000","172.21.15.44:6800/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/1968699888":"2023-09-11T00:40:38.970014+0000","172.21.15.44:0/1481976351":"2023-09-11T00:32:15.913770+0000","172.21.15.44:6801/871815312":"2023-09-11T00:31:49.496807+0000","172.21.15.44:0/625830807":"2023-09-11T00:31:49.496807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":95,"length":1},{"begin":108,"length":1}]},{"pool":4,"snaps":[{"begin":95,"length":1},{"begin":108,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-10T01:37:40.943 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.943+0000 7fca8eb4d700 1 -- 172.21.15.44:0/625751875 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fca780a0870 msgr2=0x7fca780a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:40.944 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.943+0000 7fca8eb4d700 1 --2- 172.21.15.44:0/625751875 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fca780a0870 0x7fca780a2d30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fca740095c0 tx=0x7fca74009510 comp rx=0 tx=0).stop 2023-09-10T01:37:40.944 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.943+0000 7fca8eb4d700 1 -- 172.21.15.44:0/625751875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca880ffcc0 msgr2=0x7fca8819ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:40.944 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.943+0000 7fca8eb4d700 1 --2- 172.21.15.44:0/625751875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca880ffcc0 0x7fca8819ea80 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00d950 tx=0x7fca7c00dc60 comp rx=0 tx=0).stop 2023-09-10T01:37:40.944 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.943+0000 7fca8eb4d700 1 -- 172.21.15.44:0/625751875 shutdown_connections 2023-09-10T01:37:40.944 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.943+0000 7fca8eb4d700 1 --2- 172.21.15.44:0/625751875 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fca780a0870 0x7fca780a2d30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.945 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.943+0000 7fca8eb4d700 1 --2- 172.21.15.44:0/625751875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8810cec0 0x7fca881985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.945 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.943+0000 7fca8eb4d700 1 --2- 172.21.15.44:0/625751875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca880ff310 0x7fca8819e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.945 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.943+0000 7fca8eb4d700 1 --2- 172.21.15.44:0/625751875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca880ffcc0 0x7fca8819ea80 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:40.945 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.943+0000 7fca8eb4d700 1 -- 172.21.15.44:0/625751875 >> 172.21.15.44:0/625751875 conn(0x7fca880faf10 msgr2=0x7fca8810f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:40.945 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.943+0000 7fca8eb4d700 1 -- 172.21.15.44:0/625751875 shutdown_connections 2023-09-10T01:37:40.945 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:40.943+0000 7fca8eb4d700 1 -- 172.21.15.44:0/625751875 wait complete. 2023-09-10T01:37:40.958 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-10T00:34:44.456741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-10T00:38:37.194891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-10T00:38:38.573268+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '212', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 108, 'snap_epoch': 212, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-10T00:38:42.198896+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '212', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 108, 'snap_epoch': 212, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-10T01:37:40.959 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-10T01:37:41.093 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.090+0000 7fb9b76ff700 1 -- 172.21.15.44:0/4247094537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9b01020a0 msgr2=0x7fb9b010a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:41.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.090+0000 7fb9b76ff700 1 --2- 172.21.15.44:0/4247094537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9b01020a0 0x7fb9b010a580 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009990 tx=0x7fb9a0009ca0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.090+0000 7fb9b76ff700 1 -- 172.21.15.44:0/4247094537 shutdown_connections 2023-09-10T01:37:41.094 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.090+0000 7fb9b76ff700 1 --2- 172.21.15.44:0/4247094537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9b010ab50 0x7fb9b010cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.090+0000 7fb9b76ff700 1 --2- 172.21.15.44:0/4247094537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b0101780 0x7fb9b0101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.090+0000 7fb9b76ff700 1 --2- 172.21.15.44:0/4247094537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9b01020a0 0x7fb9b010a580 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.090+0000 7fb9b76ff700 1 -- 172.21.15.44:0/4247094537 >> 172.21.15.44:0/4247094537 conn(0x7fb9b00faf20 msgr2=0x7fb9b00fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:41.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.090+0000 7fb9b76ff700 1 -- 172.21.15.44:0/4247094537 shutdown_connections 2023-09-10T01:37:41.095 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.090+0000 7fb9b76ff700 1 -- 172.21.15.44:0/4247094537 wait complete. 2023-09-10T01:37:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.091+0000 7fb9b76ff700 1 Processor -- start 2023-09-10T01:37:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.091+0000 7fb9b76ff700 1 -- start start 2023-09-10T01:37:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.091+0000 7fb9b76ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b0101780 0x7fb9b0196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.091+0000 7fb9b76ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9b01020a0 0x7fb9b0196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.091+0000 7fb9b76ff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9b010ab50 0x7fb9b019af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:41.096 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.091+0000 7fb9b76ff700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b019b570 con 0x7fb9b010ab50 2023-09-10T01:37:41.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.091+0000 7fb9b76ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b019b6e0 con 0x7fb9b0101780 2023-09-10T01:37:41.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.091+0000 7fb9b76ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b019b8a0 con 0x7fb9b01020a0 2023-09-10T01:37:41.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.091+0000 7fb9b5c9c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9b010ab50 0x7fb9b019af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:41.097 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.091+0000 7fb9b5c9c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9b010ab50 0x7fb9b019af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33486/0 (socket says 172.21.15.44:33486) 2023-09-10T01:37:41.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.091+0000 7fb9b5c9c700 1 -- 172.21.15.44:0/1623142245 learned_addr learned my addr 172.21.15.44:0/1623142245 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:41.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.091+0000 7fb9b4c9a700 1 --2- 172.21.15.44:0/1623142245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9b01020a0 0x7fb9b0196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:41.098 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.092+0000 7fb9b549b700 1 --2- 172.21.15.44:0/1623142245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b0101780 0x7fb9b0196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:41.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.092+0000 7fb9b5c9c700 1 -- 172.21.15.44:0/1623142245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9b01020a0 msgr2=0x7fb9b0196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:41.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.092+0000 7fb9b5c9c700 1 --2- 172.21.15.44:0/1623142245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9b01020a0 0x7fb9b0196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.092+0000 7fb9b5c9c700 1 -- 172.21.15.44:0/1623142245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b0101780 msgr2=0x7fb9b0196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:41.099 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.092+0000 7fb9b5c9c700 1 --2- 172.21.15.44:0/1623142245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b0101780 0x7fb9b0196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.092+0000 7fb9b5c9c700 1 -- 172.21.15.44:0/1623142245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a00095f0 con 0x7fb9b010ab50 2023-09-10T01:37:41.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.092+0000 7fb9b549b700 1 --2- 172.21.15.44:0/1623142245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b0101780 0x7fb9b0196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-10T01:37:41.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.092+0000 7fb9b5c9c700 1 --2- 172.21.15.44:0/1623142245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9b010ab50 0x7fb9b019af30 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800ef30 tx=0x7fb9a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:41.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.092+0000 7fb9a67fc700 1 -- 172.21.15.44:0/1623142245 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8018070 con 0x7fb9b010ab50 2023-09-10T01:37:41.100 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.092+0000 7fb9a67fc700 1 -- 172.21.15.44:0/1623142245 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9a800f040 con 0x7fb9b010ab50 2023-09-10T01:37:41.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.092+0000 7fb9b76ff700 1 -- 172.21.15.44:0/1623142245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b019bb20 con 0x7fb9b010ab50 2023-09-10T01:37:41.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.092+0000 7fb9a67fc700 1 -- 172.21.15.44:0/1623142245 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a800cc70 con 0x7fb9b010ab50 2023-09-10T01:37:41.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.092+0000 7fb9b76ff700 1 -- 172.21.15.44:0/1623142245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b01a33c0 con 0x7fb9b010ab50 2023-09-10T01:37:41.101 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.093+0000 7fb9b76ff700 1 -- 172.21.15.44:0/1623142245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b0105c70 con 0x7fb9b010ab50 2023-09-10T01:37:41.102 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.094+0000 7fb9a67fc700 1 -- 172.21.15.44:0/1623142245 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fb9a8003a00 con 0x7fb9b010ab50 2023-09-10T01:37:41.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.095+0000 7fb9a67fc700 1 --2- 172.21.15.44:0/1623142245 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb99c0a0830 0x7fb99c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:41.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.095+0000 7fb9a67fc700 1 -- 172.21.15.44:0/1623142245 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fb9a80cc450 con 0x7fb9b010ab50 2023-09-10T01:37:41.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.095+0000 7fb9b549b700 1 --2- 172.21.15.44:0/1623142245 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb99c0a0830 0x7fb99c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:41.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.095+0000 7fb9b549b700 1 --2- 172.21.15.44:0/1623142245 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb99c0a0830 0x7fb99c0a2cf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009cb0 tx=0x7fb9ac009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:41.103 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.099+0000 7fb9a67fc700 1 -- 172.21.15.44:0/1623142245 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb9a80958d0 con 0x7fb9b010ab50 2023-09-10T01:37:41.173 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:40 smithi044 ceph-mon[98676]: pgmap v1858: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-09-10T01:37:41.174 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:40 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2965350125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T01:37:41.174 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:40 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1534657974' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-10T01:37:41.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:40 smithi131 ceph-mon[103713]: pgmap v1858: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-09-10T01:37:41.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:40 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2965350125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T01:37:41.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:40 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1534657974' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-10T01:37:41.310 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.310+0000 7fb9b76ff700 1 -- 172.21.15.44:0/1623142245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb9b0110350 con 0x7fb9b010ab50 2023-09-10T01:37:41.311 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.310+0000 7fb9a67fc700 1 -- 172.21.15.44:0/1623142245 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v213) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb9a8098ef0 con 0x7fb9b010ab50 2023-09-10T01:37:41.311 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2023-09-10T01:37:41.313 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.313+0000 7fb9b76ff700 1 -- 172.21.15.44:0/1623142245 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb99c0a0830 msgr2=0x7fb99c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:41.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.313+0000 7fb9b76ff700 1 --2- 172.21.15.44:0/1623142245 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb99c0a0830 0x7fb99c0a2cf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009cb0 tx=0x7fb9ac009260 comp rx=0 tx=0).stop 2023-09-10T01:37:41.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.313+0000 7fb9b76ff700 1 -- 172.21.15.44:0/1623142245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9b010ab50 msgr2=0x7fb9b019af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:41.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.313+0000 7fb9b76ff700 1 --2- 172.21.15.44:0/1623142245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9b010ab50 0x7fb9b019af30 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800ef30 tx=0x7fb9a800c3a0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.313+0000 7fb9b76ff700 1 -- 172.21.15.44:0/1623142245 shutdown_connections 2023-09-10T01:37:41.314 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.313+0000 7fb9b76ff700 1 --2- 172.21.15.44:0/1623142245 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fb99c0a0830 0x7fb99c0a2cf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.313+0000 7fb9b76ff700 1 --2- 172.21.15.44:0/1623142245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9b010ab50 0x7fb9b019af30 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.313+0000 7fb9b76ff700 1 --2- 172.21.15.44:0/1623142245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b0101780 0x7fb9b0196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.313+0000 7fb9b76ff700 1 --2- 172.21.15.44:0/1623142245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9b01020a0 0x7fb9b0196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.315 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.313+0000 7fb9b76ff700 1 -- 172.21.15.44:0/1623142245 >> 172.21.15.44:0/1623142245 conn(0x7fb9b00faf20 msgr2=0x7fb9b00fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:41.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.313+0000 7fb9b76ff700 1 -- 172.21.15.44:0/1623142245 shutdown_connections 2023-09-10T01:37:41.316 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.313+0000 7fb9b76ff700 1 -- 172.21.15.44:0/1623142245 wait complete. 2023-09-10T01:37:41.327 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-10T01:37:41.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:40 smithi099 ceph-mon[102692]: pgmap v1858: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-09-10T01:37:41.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:40 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2965350125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T01:37:41.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:40 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1534657974' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-10T01:37:41.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.461+0000 7f45bba6e700 1 -- 172.21.15.44:0/3980529720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45b40ffa00 msgr2=0x7f45b4103860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:41.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.461+0000 7f45bba6e700 1 --2- 172.21.15.44:0/3980529720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45b40ffa00 0x7f45b4103860 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009990 tx=0x7f45b0009ca0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.461+0000 7f45bba6e700 1 -- 172.21.15.44:0/3980529720 shutdown_connections 2023-09-10T01:37:41.464 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.461+0000 7f45bba6e700 1 --2- 172.21.15.44:0/3980529720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b4103f90 0x7f45b410bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.461+0000 7f45bba6e700 1 --2- 172.21.15.44:0/3980529720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b40ff050 0x7f45b40ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.461+0000 7f45bba6e700 1 --2- 172.21.15.44:0/3980529720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45b40ffa00 0x7f45b4103860 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.461+0000 7f45bba6e700 1 -- 172.21.15.44:0/3980529720 >> 172.21.15.44:0/3980529720 conn(0x7f45b40747a0 msgr2=0x7f45b4074bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:41.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.461+0000 7f45bba6e700 1 -- 172.21.15.44:0/3980529720 shutdown_connections 2023-09-10T01:37:41.465 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.461+0000 7f45bba6e700 1 -- 172.21.15.44:0/3980529720 wait complete. 2023-09-10T01:37:41.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.462+0000 7f45bba6e700 1 Processor -- start 2023-09-10T01:37:41.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.462+0000 7f45bba6e700 1 -- start start 2023-09-10T01:37:41.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.462+0000 7f45bba6e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b40ff050 0x7f45b4196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:41.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.462+0000 7f45bba6e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b4103f90 0x7f45b4196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:41.466 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.462+0000 7f45bba6e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45b419af20 0x7f45b419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:41.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.462+0000 7f45bba6e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b419b9a0 con 0x7f45b419af20 2023-09-10T01:37:41.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.462+0000 7f45bba6e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b419bb10 con 0x7f45b4103f90 2023-09-10T01:37:41.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.462+0000 7f45bba6e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b419bcd0 con 0x7f45b40ff050 2023-09-10T01:37:41.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.462+0000 7f45ba00b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45b419af20 0x7f45b419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:41.467 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.462+0000 7f45ba00b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45b419af20 0x7f45b419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33500/0 (socket says 172.21.15.44:33500) 2023-09-10T01:37:41.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.462+0000 7f45ba00b700 1 -- 172.21.15.44:0/1213508887 learned_addr learned my addr 172.21.15.44:0/1213508887 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:41.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.462+0000 7f45b9009700 1 --2- 172.21.15.44:0/1213508887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b4103f90 0x7f45b4196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:41.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.462+0000 7f45b980a700 1 --2- 172.21.15.44:0/1213508887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b40ff050 0x7f45b4196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:41.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.463+0000 7f45ba00b700 1 -- 172.21.15.44:0/1213508887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b40ff050 msgr2=0x7f45b4196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:41.468 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.463+0000 7f45ba00b700 1 --2- 172.21.15.44:0/1213508887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b40ff050 0x7f45b4196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.463+0000 7f45ba00b700 1 -- 172.21.15.44:0/1213508887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b4103f90 msgr2=0x7f45b4196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:41.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.463+0000 7f45ba00b700 1 --2- 172.21.15.44:0/1213508887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b4103f90 0x7f45b4196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.463+0000 7f45ba00b700 1 -- 172.21.15.44:0/1213508887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b00095f0 con 0x7f45b419af20 2023-09-10T01:37:41.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.463+0000 7f45ba00b700 1 --2- 172.21.15.44:0/1213508887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45b419af20 0x7f45b419b3a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f45ac007ee0 tx=0x7f45ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:41.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.463+0000 7f45a6ffd700 1 -- 172.21.15.44:0/1213508887 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac017070 con 0x7f45b419af20 2023-09-10T01:37:41.469 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.463+0000 7f45bba6e700 1 -- 172.21.15.44:0/1213508887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b41a3230 con 0x7f45b419af20 2023-09-10T01:37:41.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.463+0000 7f45a6ffd700 1 -- 172.21.15.44:0/1213508887 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45ac00f040 con 0x7f45b419af20 2023-09-10T01:37:41.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.463+0000 7f45a6ffd700 1 -- 172.21.15.44:0/1213508887 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac005570 con 0x7f45b419af20 2023-09-10T01:37:41.470 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.463+0000 7f45bba6e700 1 -- 172.21.15.44:0/1213508887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b41a36c0 con 0x7f45b419af20 2023-09-10T01:37:41.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.464+0000 7f45bba6e700 1 -- 172.21.15.44:0/1213508887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b40677d0 con 0x7f45b419af20 2023-09-10T01:37:41.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.465+0000 7f45a6ffd700 1 -- 172.21.15.44:0/1213508887 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f45ac00db20 con 0x7f45b419af20 2023-09-10T01:37:41.471 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.466+0000 7f45a6ffd700 1 --2- 172.21.15.44:0/1213508887 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45a00a07b0 0x7f45a00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:41.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.466+0000 7f45a6ffd700 1 -- 172.21.15.44:0/1213508887 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f45ac0cc0e0 con 0x7f45b419af20 2023-09-10T01:37:41.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.466+0000 7f45b980a700 1 --2- 172.21.15.44:0/1213508887 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45a00a07b0 0x7f45a00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:41.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.467+0000 7f45b980a700 1 --2- 172.21.15.44:0/1213508887 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45a00a07b0 0x7f45a00a2c70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009c60 tx=0x7f45a8009260 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:41.472 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.469+0000 7f45a6ffd700 1 -- 172.21.15.44:0/1213508887 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f45ac0954d0 con 0x7f45b419af20 2023-09-10T01:37:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.675+0000 7f45bba6e700 1 -- 172.21.15.44:0/1213508887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45b40ffb40 con 0x7f45b419af20 2023-09-10T01:37:41.676 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.675+0000 7f45a6ffd700 1 -- 172.21.15.44:0/1213508887 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v213) v1 ==== 104+0+11 (secure 0 0 0) 0x7f45ac098af0 con 0x7f45b419af20 2023-09-10T01:37:41.676 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T01:37:41.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.677+0000 7f45bba6e700 1 -- 172.21.15.44:0/1213508887 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45a00a07b0 msgr2=0x7f45a00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:41.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.677+0000 7f45bba6e700 1 --2- 172.21.15.44:0/1213508887 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45a00a07b0 0x7f45a00a2c70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009c60 tx=0x7f45a8009260 comp rx=0 tx=0).stop 2023-09-10T01:37:41.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.677+0000 7f45bba6e700 1 -- 172.21.15.44:0/1213508887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45b419af20 msgr2=0x7f45b419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:41.678 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.677+0000 7f45bba6e700 1 --2- 172.21.15.44:0/1213508887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45b419af20 0x7f45b419b3a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f45ac007ee0 tx=0x7f45ac00d3b0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.679 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.678+0000 7f45bba6e700 1 -- 172.21.15.44:0/1213508887 shutdown_connections 2023-09-10T01:37:41.679 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.678+0000 7f45bba6e700 1 --2- 172.21.15.44:0/1213508887 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f45a00a07b0 0x7f45a00a2c70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.680 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.678+0000 7f45bba6e700 1 --2- 172.21.15.44:0/1213508887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45b419af20 0x7f45b419b3a0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.680 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.678+0000 7f45bba6e700 1 --2- 172.21.15.44:0/1213508887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b40ff050 0x7f45b4196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.680 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.678+0000 7f45bba6e700 1 --2- 172.21.15.44:0/1213508887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b4103f90 0x7f45b4196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.680 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.678+0000 7f45bba6e700 1 -- 172.21.15.44:0/1213508887 >> 172.21.15.44:0/1213508887 conn(0x7f45b40747a0 msgr2=0x7f45b40fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:41.680 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.678+0000 7f45bba6e700 1 -- 172.21.15.44:0/1213508887 shutdown_connections 2023-09-10T01:37:41.681 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.678+0000 7f45bba6e700 1 -- 172.21.15.44:0/1213508887 wait complete. 2023-09-10T01:37:41.691 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-10T01:37:41.834 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.834+0000 7fd985a86700 1 -- 172.21.15.44:0/147806344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9800ff440 msgr2=0x7fd9800ff8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:41.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.834+0000 7fd985a86700 1 --2- 172.21.15.44:0/147806344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9800ff440 0x7fd9800ff8c0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fd970009990 tx=0x7fd970009ca0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.834+0000 7fd985a86700 1 -- 172.21.15.44:0/147806344 shutdown_connections 2023-09-10T01:37:41.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.834+0000 7fd985a86700 1 --2- 172.21.15.44:0/147806344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9800ffe00 0x7fd98010c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.835 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.834+0000 7fd985a86700 1 --2- 172.21.15.44:0/147806344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9801054e0 0x7fd9801058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.834+0000 7fd985a86700 1 --2- 172.21.15.44:0/147806344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9800ff440 0x7fd9800ff8c0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.834+0000 7fd985a86700 1 -- 172.21.15.44:0/147806344 >> 172.21.15.44:0/147806344 conn(0x7fd9800faf50 msgr2=0x7fd9800fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:41.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.834+0000 7fd985a86700 1 -- 172.21.15.44:0/147806344 shutdown_connections 2023-09-10T01:37:41.836 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.834+0000 7fd985a86700 1 -- 172.21.15.44:0/147806344 wait complete. 2023-09-10T01:37:41.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd985a86700 1 Processor -- start 2023-09-10T01:37:41.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd985a86700 1 -- start start 2023-09-10T01:37:41.839 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd985a86700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9800ff440 0x7fd98019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:41.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd985a86700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9800ffe00 0x7fd98019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:41.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd985a86700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9801054e0 0x7fd980198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:41.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd985a86700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd980198c00 con 0x7fd9801054e0 2023-09-10T01:37:41.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd985a86700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd980198d70 con 0x7fd9800ff440 2023-09-10T01:37:41.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd985a86700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd980198f30 con 0x7fd9800ffe00 2023-09-10T01:37:41.840 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd97ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9801054e0 0x7fd980198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:41.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd97ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9801054e0 0x7fd980198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33514/0 (socket says 172.21.15.44:33514) 2023-09-10T01:37:41.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd97ffff700 1 -- 172.21.15.44:0/3060131127 learned_addr learned my addr 172.21.15.44:0/3060131127 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:41.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd97f7fe700 1 --2- 172.21.15.44:0/3060131127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9800ff440 0x7fd98019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:41.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd97effd700 1 --2- 172.21.15.44:0/3060131127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9800ffe00 0x7fd98019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:41.841 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd97ffff700 1 -- 172.21.15.44:0/3060131127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9800ffe00 msgr2=0x7fd98019ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:41.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd97ffff700 1 --2- 172.21.15.44:0/3060131127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9800ffe00 0x7fd98019ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd97ffff700 1 -- 172.21.15.44:0/3060131127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9800ff440 msgr2=0x7fd98019e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:41.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd97ffff700 1 --2- 172.21.15.44:0/3060131127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9800ff440 0x7fd98019e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:41.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd97ffff700 1 -- 172.21.15.44:0/3060131127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9700095f0 con 0x7fd9801054e0 2023-09-10T01:37:41.842 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.835+0000 7fd97effd700 1 --2- 172.21.15.44:0/3060131127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9800ffe00 0x7fd98019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-10T01:37:41.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.836+0000 7fd97ffff700 1 --2- 172.21.15.44:0/3060131127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9801054e0 0x7fd980198590 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fd978007ee0 tx=0x7fd97800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:41.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.836+0000 7fd97cff9700 1 -- 172.21.15.44:0/3060131127 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978017070 con 0x7fd9801054e0 2023-09-10T01:37:41.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.836+0000 7fd97cff9700 1 -- 172.21.15.44:0/3060131127 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd97800f040 con 0x7fd9801054e0 2023-09-10T01:37:41.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.836+0000 7fd985a86700 1 -- 172.21.15.44:0/3060131127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd980199210 con 0x7fd9801054e0 2023-09-10T01:37:41.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.836+0000 7fd97cff9700 1 -- 172.21.15.44:0/3060131127 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978005710 con 0x7fd9801054e0 2023-09-10T01:37:41.843 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.836+0000 7fd985a86700 1 -- 172.21.15.44:0/3060131127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9801a5510 con 0x7fd9801054e0 2023-09-10T01:37:41.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.837+0000 7fd985a86700 1 -- 172.21.15.44:0/3060131127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd980102770 con 0x7fd9801054e0 2023-09-10T01:37:41.844 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.838+0000 7fd97cff9700 1 -- 172.21.15.44:0/3060131127 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fd978005870 con 0x7fd9801054e0 2023-09-10T01:37:41.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.839+0000 7fd97cff9700 1 --2- 172.21.15.44:0/3060131127 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd9680a07b0 0x7fd9680a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:41.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.839+0000 7fd97f7fe700 1 --2- 172.21.15.44:0/3060131127 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd9680a07b0 0x7fd9680a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:41.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.839+0000 7fd97cff9700 1 -- 172.21.15.44:0/3060131127 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 9540+0+0 (secure 0 0 0) 0x7fd9780cc260 con 0x7fd9801054e0 2023-09-10T01:37:41.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.840+0000 7fd97f7fe700 1 --2- 172.21.15.44:0/3060131127 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd9680a07b0 0x7fd9680a2c70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd974005950 tx=0x7fd97400a300 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:41.845 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:41.843+0000 7fd97cff9700 1 -- 172.21.15.44:0/3060131127 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd978095650 con 0x7fd9801054e0 2023-09-10T01:37:41.926 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:41 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/625751875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T01:37:41.926 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:41 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1623142245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T01:37:41.926 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:41 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/1213508887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T01:37:42.053 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.053+0000 7fd985a86700 1 -- 172.21.15.44:0/3060131127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd98010f1d0 con 0x7fd9801054e0 2023-09-10T01:37:42.054 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.053+0000 7fd97cff9700 1 -- 172.21.15.44:0/3060131127 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v213) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd978098c70 con 0x7fd9801054e0 2023-09-10T01:37:42.054 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2023-09-10T01:37:42.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.055+0000 7fd985a86700 1 -- 172.21.15.44:0/3060131127 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd9680a07b0 msgr2=0x7fd9680a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:42.056 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.055+0000 7fd985a86700 1 --2- 172.21.15.44:0/3060131127 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd9680a07b0 0x7fd9680a2c70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd974005950 tx=0x7fd97400a300 comp rx=0 tx=0).stop 2023-09-10T01:37:42.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.055+0000 7fd985a86700 1 -- 172.21.15.44:0/3060131127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9801054e0 msgr2=0x7fd980198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:42.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.055+0000 7fd985a86700 1 --2- 172.21.15.44:0/3060131127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9801054e0 0x7fd980198590 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fd978007ee0 tx=0x7fd97800d3b0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.055+0000 7fd985a86700 1 -- 172.21.15.44:0/3060131127 shutdown_connections 2023-09-10T01:37:42.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.055+0000 7fd985a86700 1 --2- 172.21.15.44:0/3060131127 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7fd9680a07b0 0x7fd9680a2c70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.057 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.055+0000 7fd985a86700 1 --2- 172.21.15.44:0/3060131127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9801054e0 0x7fd980198590 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.055+0000 7fd985a86700 1 --2- 172.21.15.44:0/3060131127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9800ff440 0x7fd98019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.055+0000 7fd985a86700 1 --2- 172.21.15.44:0/3060131127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9800ffe00 0x7fd98019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.055+0000 7fd985a86700 1 -- 172.21.15.44:0/3060131127 >> 172.21.15.44:0/3060131127 conn(0x7fd9800faf50 msgr2=0x7fd98010abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.055+0000 7fd985a86700 1 -- 172.21.15.44:0/3060131127 shutdown_connections 2023-09-10T01:37:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.055+0000 7fd985a86700 1 -- 172.21.15.44:0/3060131127 wait complete. 2023-09-10T01:37:42.069 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-10T01:37:42.202 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.202+0000 7f9daa2ff700 1 -- 172.21.15.44:0/2121798719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9da41054f0 msgr2=0x7f9da41058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:42.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.202+0000 7f9daa2ff700 1 --2- 172.21.15.44:0/2121798719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9da41054f0 0x7f9da41058d0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009990 tx=0x7f9d94009ca0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.202+0000 7f9daa2ff700 1 -- 172.21.15.44:0/2121798719 shutdown_connections 2023-09-10T01:37:42.203 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.202+0000 7f9daa2ff700 1 --2- 172.21.15.44:0/2121798719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9da40ffe10 0x7f9da410c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.202+0000 7f9daa2ff700 1 --2- 172.21.15.44:0/2121798719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9da41054f0 0x7f9da41058d0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.202+0000 7f9daa2ff700 1 --2- 172.21.15.44:0/2121798719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9da40ff450 0x7f9da40ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.202+0000 7f9daa2ff700 1 -- 172.21.15.44:0/2121798719 >> 172.21.15.44:0/2121798719 conn(0x7f9da40faf20 msgr2=0x7f9da40fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:42.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.202+0000 7f9daa2ff700 1 -- 172.21.15.44:0/2121798719 shutdown_connections 2023-09-10T01:37:42.204 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.202+0000 7f9daa2ff700 1 -- 172.21.15.44:0/2121798719 wait complete. 2023-09-10T01:37:42.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9daa2ff700 1 Processor -- start 2023-09-10T01:37:42.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9daa2ff700 1 -- start start 2023-09-10T01:37:42.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9daa2ff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9da40ff450 0x7f9da4195400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:42.207 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9daa2ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9da40ffe10 0x7f9da4195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:42.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9daa2ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9da41054f0 0x7f9da4199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:42.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9daa2ff700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da419a3e0 con 0x7f9da40ff450 2023-09-10T01:37:42.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9daa2ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da419a550 con 0x7f9da41054f0 2023-09-10T01:37:42.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9daa2ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da419a710 con 0x7f9da40ffe10 2023-09-10T01:37:42.208 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9da3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9da40ff450 0x7f9da4195400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:42.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9da3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9da40ff450 0x7f9da4195400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33522/0 (socket says 172.21.15.44:33522) 2023-09-10T01:37:42.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9da3fff700 1 -- 172.21.15.44:0/2987891129 learned_addr learned my addr 172.21.15.44:0/2987891129 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:42.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9d9b7fe700 1 --2- 172.21.15.44:0/2987891129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9da40ffe10 0x7f9da4195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:42.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9da889c700 1 --2- 172.21.15.44:0/2987891129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9da41054f0 0x7f9da4199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:42.209 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9da3fff700 1 -- 172.21.15.44:0/2987891129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9da40ffe10 msgr2=0x7f9da4195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:42.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9da3fff700 1 --2- 172.21.15.44:0/2987891129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9da40ffe10 0x7f9da4195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9da3fff700 1 -- 172.21.15.44:0/2987891129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9da41054f0 msgr2=0x7f9da4199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:42.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9da3fff700 1 --2- 172.21.15.44:0/2987891129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9da41054f0 0x7f9da4199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.203+0000 7f9da3fff700 1 -- 172.21.15.44:0/2987891129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d940095f0 con 0x7f9da40ff450 2023-09-10T01:37:42.210 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.204+0000 7f9da3fff700 1 --2- 172.21.15.44:0/2987891129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9da40ff450 0x7f9da4195400 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f9d94006010 tx=0x7f9d94005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:42.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.204+0000 7f9da1ffb700 1 -- 172.21.15.44:0/2987891129 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9401d070 con 0x7f9da40ff450 2023-09-10T01:37:42.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.204+0000 7f9da1ffb700 1 -- 172.21.15.44:0/2987891129 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d94003f80 con 0x7f9da40ff450 2023-09-10T01:37:42.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.204+0000 7f9daa2ff700 1 -- 172.21.15.44:0/2987891129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da419a990 con 0x7f9da40ff450 2023-09-10T01:37:42.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.204+0000 7f9da1ffb700 1 -- 172.21.15.44:0/2987891129 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d94022b90 con 0x7f9da40ff450 2023-09-10T01:37:42.211 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.204+0000 7f9daa2ff700 1 -- 172.21.15.44:0/2987891129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da41a2230 con 0x7f9da40ff450 2023-09-10T01:37:42.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.205+0000 7f9daa2ff700 1 -- 172.21.15.44:0/2987891129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da4066af0 con 0x7f9da40ff450 2023-09-10T01:37:42.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.206+0000 7f9da1ffb700 1 -- 172.21.15.44:0/2987891129 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f9d94003af0 con 0x7f9da40ff450 2023-09-10T01:37:42.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.206+0000 7f9da1ffb700 1 --2- 172.21.15.44:0/2987891129 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9d900a07b0 0x7f9d900a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:42.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.206+0000 7f9da1ffb700 1 -- 172.21.15.44:0/2987891129 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f9d940ce040 con 0x7f9da40ff450 2023-09-10T01:37:42.212 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.206+0000 7f9d9b7fe700 1 --2- 172.21.15.44:0/2987891129 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9d900a07b0 0x7f9d900a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:42.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.207+0000 7f9d9b7fe700 1 --2- 172.21.15.44:0/2987891129 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9d900a07b0 0x7f9d900a2c70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9da4196a10 tx=0x7f9d8c00b410 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:42.213 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.210+0000 7f9da1ffb700 1 -- 172.21.15.44:0/2987891129 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9d940974c0 con 0x7f9da40ff450 2023-09-10T01:37:42.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:41 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/625751875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T01:37:42.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:41 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1623142245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T01:37:42.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:41 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/1213508887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T01:37:42.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:41 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/625751875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-10T01:37:42.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:41 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1623142245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-10T01:37:42.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:41 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/1213508887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-10T01:37:42.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.411+0000 7f9daa2ff700 1 -- 172.21.15.44:0/2987891129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9da410f1e0 con 0x7f9da40ff450 2023-09-10T01:37:42.412 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.411+0000 7f9da1ffb700 1 -- 172.21.15.44:0/2987891129 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v213) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9d9409aae0 con 0x7f9da40ff450 2023-09-10T01:37:42.412 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2023-09-10T01:37:42.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.413+0000 7f9daa2ff700 1 -- 172.21.15.44:0/2987891129 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9d900a07b0 msgr2=0x7f9d900a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:42.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.413+0000 7f9daa2ff700 1 --2- 172.21.15.44:0/2987891129 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9d900a07b0 0x7f9d900a2c70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9da4196a10 tx=0x7f9d8c00b410 comp rx=0 tx=0).stop 2023-09-10T01:37:42.414 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.413+0000 7f9daa2ff700 1 -- 172.21.15.44:0/2987891129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9da40ff450 msgr2=0x7f9da4195400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:42.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.413+0000 7f9daa2ff700 1 --2- 172.21.15.44:0/2987891129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9da40ff450 0x7f9da4195400 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f9d94006010 tx=0x7f9d94005740 comp rx=0 tx=0).stop 2023-09-10T01:37:42.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.413+0000 7f9daa2ff700 1 -- 172.21.15.44:0/2987891129 shutdown_connections 2023-09-10T01:37:42.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.413+0000 7f9daa2ff700 1 --2- 172.21.15.44:0/2987891129 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f9d900a07b0 0x7f9d900a2c70 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.413+0000 7f9daa2ff700 1 --2- 172.21.15.44:0/2987891129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9da41054f0 0x7f9da4199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.415 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.413+0000 7f9daa2ff700 1 --2- 172.21.15.44:0/2987891129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9da40ff450 0x7f9da4195400 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.413+0000 7f9daa2ff700 1 --2- 172.21.15.44:0/2987891129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9da40ffe10 0x7f9da4195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.413+0000 7f9daa2ff700 1 -- 172.21.15.44:0/2987891129 >> 172.21.15.44:0/2987891129 conn(0x7f9da40faf20 msgr2=0x7f9da4102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:42.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.413+0000 7f9daa2ff700 1 -- 172.21.15.44:0/2987891129 shutdown_connections 2023-09-10T01:37:42.416 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.413+0000 7f9daa2ff700 1 -- 172.21.15.44:0/2987891129 wait complete. 2023-09-10T01:37:42.427 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-10T01:37:42.557 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.555+0000 7f0438443700 1 -- 172.21.15.44:0/2199905224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04300ff460 msgr2=0x7f04300ff8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:42.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.555+0000 7f0438443700 1 --2- 172.21.15.44:0/2199905224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04300ff460 0x7f04300ff8e0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f042c009990 tx=0x7f042c009ca0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.555+0000 7f0438443700 1 -- 172.21.15.44:0/2199905224 shutdown_connections 2023-09-10T01:37:42.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.555+0000 7f0438443700 1 --2- 172.21.15.44:0/2199905224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04300ffe20 0x7f043010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.558 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.555+0000 7f0438443700 1 --2- 172.21.15.44:0/2199905224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0430105500 0x7f04301058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.555+0000 7f0438443700 1 --2- 172.21.15.44:0/2199905224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04300ff460 0x7f04300ff8e0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.555+0000 7f0438443700 1 -- 172.21.15.44:0/2199905224 >> 172.21.15.44:0/2199905224 conn(0x7f04300faf50 msgr2=0x7f04300fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:42.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.555+0000 7f0438443700 1 -- 172.21.15.44:0/2199905224 shutdown_connections 2023-09-10T01:37:42.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.555+0000 7f0438443700 1 -- 172.21.15.44:0/2199905224 wait complete. 2023-09-10T01:37:42.559 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f0438443700 1 Processor -- start 2023-09-10T01:37:42.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f0438443700 1 -- start start 2023-09-10T01:37:42.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f0438443700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04300ff460 0x7f0430197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:42.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f0438443700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04300ffe20 0x7f0430197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:42.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f0438443700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0430105500 0x7f04301a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:42.560 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f0438443700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0430198190 con 0x7f04300ffe20 2023-09-10T01:37:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f0438443700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04301a2900 con 0x7f0430105500 2023-09-10T01:37:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f0438443700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04301a2a70 con 0x7f04300ff460 2023-09-10T01:37:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f04359de700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04300ffe20 0x7f0430197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f04359de700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04300ffe20 0x7f0430197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33538/0 (socket says 172.21.15.44:33538) 2023-09-10T01:37:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f04359de700 1 -- 172.21.15.44:0/2245032699 learned_addr learned my addr 172.21.15.44:0/2245032699 (peer_addr_for_me v2:172.21.15.44:0/0) 2023-09-10T01:37:42.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f04361df700 1 --2- 172.21.15.44:0/2245032699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04300ff460 0x7f0430197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:42.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f04369e0700 1 --2- 172.21.15.44:0/2245032699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0430105500 0x7f04301a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:42.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f04359de700 1 -- 172.21.15.44:0/2245032699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04300ff460 msgr2=0x7f0430197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:42.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f04359de700 1 --2- 172.21.15.44:0/2245032699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04300ff460 0x7f0430197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.562 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f04359de700 1 -- 172.21.15.44:0/2245032699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0430105500 msgr2=0x7f04301a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:42.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f04359de700 1 --2- 172.21.15.44:0/2245032699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0430105500 0x7f04301a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f04359de700 1 -- 172.21.15.44:0/2245032699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c0095f0 con 0x7f04300ffe20 2023-09-10T01:37:42.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.556+0000 7f04359de700 1 --2- 172.21.15.44:0/2245032699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04300ffe20 0x7f0430197880 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f042c005770 tx=0x7f042c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:42.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.557+0000 7f04237fe700 1 -- 172.21.15.44:0/2245032699 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c01d070 con 0x7f04300ffe20 2023-09-10T01:37:42.563 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.557+0000 7f04237fe700 1 -- 172.21.15.44:0/2245032699 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f042c021410 con 0x7f04300ffe20 2023-09-10T01:37:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.557+0000 7f0438443700 1 -- 172.21.15.44:0/2245032699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04301a2d50 con 0x7f04300ffe20 2023-09-10T01:37:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.557+0000 7f04237fe700 1 -- 172.21.15.44:0/2245032699 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c0173f0 con 0x7f04300ffe20 2023-09-10T01:37:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.557+0000 7f0438443700 1 -- 172.21.15.44:0/2245032699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04301a3250 con 0x7f04300ffe20 2023-09-10T01:37:42.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.558+0000 7f0438443700 1 -- 172.21.15.44:0/2245032699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0430066af0 con 0x7f04300ffe20 2023-09-10T01:37:42.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.559+0000 7f04237fe700 1 -- 172.21.15.44:0/2245032699 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f042c021780 con 0x7f04300ffe20 2023-09-10T01:37:42.565 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.559+0000 7f04237fe700 1 --2- 172.21.15.44:0/2245032699 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f041c0a0830 0x7f041c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-10T01:37:42.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.559+0000 7f04361df700 1 --2- 172.21.15.44:0/2245032699 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f041c0a0830 0x7f041c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-10T01:37:42.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.559+0000 7f04237fe700 1 -- 172.21.15.44:0/2245032699 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 9540+0+0 (secure 0 0 0) 0x7f042c0cd390 con 0x7f04300ffe20 2023-09-10T01:37:42.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.560+0000 7f04361df700 1 --2- 172.21.15.44:0/2245032699 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f041c0a0830 0x7f041c0a2cf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f042400abd0 tx=0x7f042400a380 comp rx=0 tx=0).ready entity=mgr.15825 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-10T01:37:42.566 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.564+0000 7f04237fe700 1 -- 172.21.15.44:0/2245032699 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f042c096810 con 0x7f04300ffe20 2023-09-10T01:37:42.784 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.784+0000 7f0438443700 1 -- 172.21.15.44:0/2245032699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0430068d60 con 0x7f04300ffe20 2023-09-10T01:37:42.785 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.784+0000 7f04237fe700 1 -- 172.21.15.44:0/2245032699 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v213) v1 ==== 81+0+937 (secure 0 0 0) 0x7f042c099e30 con 0x7f04300ffe20 2023-09-10T01:37:42.785 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/902389643 2023-09-11T00:40:38.970014+0000 2023-09-10T01:37:42.786 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/3642136197 2023-09-11T00:40:38.970014+0000 2023-09-10T01:37:42.786 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/3766354728 2023-09-11T00:40:38.970014+0000 2023-09-10T01:37:42.786 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/1488156576 2023-09-11T00:40:38.970014+0000 2023-09-10T01:37:42.787 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/1488156576 2023-09-11T00:40:38.970014+0000 2023-09-10T01:37:42.787 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/18127588 2023-09-11T00:32:15.913770+0000 2023-09-10T01:37:42.787 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/18127588 2023-09-11T00:32:15.913770+0000 2023-09-10T01:37:42.787 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1414498756 2023-09-11T00:32:15.913770+0000 2023-09-10T01:37:42.788 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1298831914 2023-09-11T00:32:15.913770+0000 2023-09-10T01:37:42.788 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2415771009 2023-09-11T00:31:49.496807+0000 2023-09-10T01:37:42.788 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/193660691 2023-09-11T00:31:49.496807+0000 2023-09-10T01:37:42.788 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/871815312 2023-09-11T00:31:49.496807+0000 2023-09-10T01:37:42.789 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1968699888 2023-09-11T00:40:38.970014+0000 2023-09-10T01:37:42.789 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1481976351 2023-09-11T00:32:15.913770+0000 2023-09-10T01:37:42.789 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/871815312 2023-09-11T00:31:49.496807+0000 2023-09-10T01:37:42.789 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/625830807 2023-09-11T00:31:49.496807+0000 2023-09-10T01:37:42.790 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.787+0000 7f0438443700 1 -- 172.21.15.44:0/2245032699 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f041c0a0830 msgr2=0x7f041c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:42.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.787+0000 7f0438443700 1 --2- 172.21.15.44:0/2245032699 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f041c0a0830 0x7f041c0a2cf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f042400abd0 tx=0x7f042400a380 comp rx=0 tx=0).stop 2023-09-10T01:37:42.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.787+0000 7f0438443700 1 -- 172.21.15.44:0/2245032699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04300ffe20 msgr2=0x7f0430197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-10T01:37:42.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.787+0000 7f0438443700 1 --2- 172.21.15.44:0/2245032699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04300ffe20 0x7f0430197880 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f042c005770 tx=0x7f042c0057a0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.787+0000 7f0438443700 1 -- 172.21.15.44:0/2245032699 shutdown_connections 2023-09-10T01:37:42.791 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.787+0000 7f0438443700 1 --2- 172.21.15.44:0/2245032699 >> [v2:172.21.15.44:6800/605958391,v1:172.21.15.44:6801/605958391] conn(0x7f041c0a0830 0x7f041c0a2cf0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.787+0000 7f0438443700 1 --2- 172.21.15.44:0/2245032699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0430105500 0x7f04301a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.787+0000 7f0438443700 1 --2- 172.21.15.44:0/2245032699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04300ff460 0x7f0430197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.787+0000 7f0438443700 1 --2- 172.21.15.44:0/2245032699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04300ffe20 0x7f0430197880 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-10T01:37:42.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.787+0000 7f0438443700 1 -- 172.21.15.44:0/2245032699 >> 172.21.15.44:0/2245032699 conn(0x7f04300faf50 msgr2=0x7f0430103000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-10T01:37:42.792 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.787+0000 7f0438443700 1 -- 172.21.15.44:0/2245032699 shutdown_connections 2023-09-10T01:37:42.793 INFO:teuthology.orchestra.run.smithi044.stderr:2023-09-10T01:37:42.787+0000 7f0438443700 1 -- 172.21.15.44:0/2245032699 wait complete. 2023-09-10T01:37:42.793 INFO:teuthology.orchestra.run.smithi044.stderr:listed 16 entries 2023-09-10T01:37:42.800 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi044.front.sepia.ceph.com... 2023-09-10T01:37:42.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T01:37:42.801 DEBUG:teuthology.orchestra.run.smithi044:> sudo fusermount -u /tmp/cephfs 2023-09-10T01:37:42.869 INFO:teuthology.orchestra.run:waiting for 300 2023-09-10T01:37:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:42 smithi044 ceph-mon[98676]: pgmap v1859: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-10T01:37:43.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:42 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/3060131127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T01:37:43.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:42 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2987891129' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T01:37:43.056 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:42 smithi044 ceph-mon[98676]: from='client.? 172.21.15.44:0/2245032699' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-10T01:37:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:42 smithi131 ceph-mon[103713]: pgmap v1859: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-10T01:37:43.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:42 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/3060131127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T01:37:43.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:42 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2987891129' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T01:37:43.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:42 smithi131 ceph-mon[103713]: from='client.? 172.21.15.44:0/2245032699' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-10T01:37:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:42 smithi099 ceph-mon[102692]: pgmap v1859: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-10T01:37:43.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:42 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/3060131127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-10T01:37:43.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:42 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2987891129' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-10T01:37:43.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:42 smithi099 ceph-mon[102692]: from='client.? 172.21.15.44:0/2245032699' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-10T01:37:44.199 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:ceph-fuse[170792]: fuse finished with error 0 and tester_r 0 2023-09-10T01:37:45.116 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:44 smithi044 ceph-mon[98676]: pgmap v1860: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-10T01:37:45.116 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:37:45.117 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:44 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:37:45.117 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:37:45.117 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:44 smithi044 ceph-mon[98676]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:37:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:44 smithi131 ceph-mon[103713]: pgmap v1860: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-10T01:37:45.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:37:45.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:44 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:37:45.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:37:45.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:44 smithi131 ceph-mon[103713]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:37:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:44 smithi099 ceph-mon[102692]: pgmap v1860: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-10T01:37:45.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:37:45.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:44 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:37:45.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:37:45.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:44 smithi099 ceph-mon[102692]: from='mgr.15825 172.21.15.44:0/3090180143' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:37:46.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:45 smithi131 ceph-mon[103713]: pgmap v1861: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-09-10T01:37:46.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:45 smithi131 ceph-mon[103713]: pgmap v1862: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2023-09-10T01:37:46.296 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:45 smithi131 ceph-mon[103713]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:37:46.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:45 smithi044 ceph-mon[98676]: pgmap v1861: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-09-10T01:37:46.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:45 smithi044 ceph-mon[98676]: pgmap v1862: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2023-09-10T01:37:46.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:45 smithi044 ceph-mon[98676]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:37:46.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:45 smithi099 ceph-mon[102692]: pgmap v1861: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-09-10T01:37:46.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:45 smithi099 ceph-mon[102692]: pgmap v1862: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2023-09-10T01:37:46.357 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:45 smithi099 ceph-mon[102692]: from='mgr.15825 ' entity='mgr.x' 2023-09-10T01:37:48.055 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:47 smithi044 ceph-mon[98676]: pgmap v1863: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2023-09-10T01:37:48.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:47 smithi099 ceph-mon[102692]: pgmap v1863: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2023-09-10T01:37:48.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:47 smithi131 ceph-mon[103713]: pgmap v1863: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2023-09-10T01:37:48.971 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2023-09-10T01:37:48.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T01:37:48.972 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2023-09-10T01:37:49.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T01:37:49.031 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2023-09-10T01:37:49.071 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--tmp-cephfs (id: 0) 2023-09-10T01:37:49.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T01:37:49.073 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns delete ceph-ns--tmp-cephfs 2023-09-10T01:37:49.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-10T01:37:49.159 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2023-09-10T01:37:49.273 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-09-10T01:37:49.287 INFO:tasks.cephadm:Teardown begin 2023-09-10T01:37:49.288 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/cephadm.py", line 1789, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-09-10T01:37:49.288 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-10T01:37:49.345 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-10T01:37:49.396 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-10T01:37:49.524 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-09-10T01:37:49.525 DEBUG:teuthology.orchestra.run.smithi044:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-10T01:37:49.561 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-10T01:37:49.584 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-10T01:37:49.616 INFO:tasks.cephadm:Stopping all daemons... 2023-09-10T01:37:49.616 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-09-10T01:37:49.617 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mon.a 2023-09-10T01:37:49.986 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:49 smithi044 systemd[1]: Stopping Ceph mon.a for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:37:50.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:49 smithi131 ceph-mon[103713]: pgmap v1864: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2023-09-10T01:37:50.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:49 smithi044 ceph-mon[98676]: pgmap v1864: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2023-09-10T01:37:50.305 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:50 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mon-a[98652]: 2023-09-10T01:37:50.004+0000 7fabd0884700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:37:50.306 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:50 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mon-a[98652]: 2023-09-10T01:37:50.004+0000 7fabd0884700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-09-10T01:37:50.356 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:49 smithi099 ceph-mon[102692]: pgmap v1864: 129 pgs: 129 active+clean; 17 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2023-09-10T01:37:50.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Sep 10 01:37:50 smithi044 bash[239428]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mon-a 2023-09-10T01:37:50.770 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mon.a.service' 2023-09-10T01:37:50.825 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:37:50.825 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-09-10T01:37:50.826 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-09-10T01:37:50.826 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mon.b 2023-09-10T01:37:51.237 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:50 smithi099 systemd[1]: Stopping Ceph mon.b for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:37:51.237 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:51 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mon-b[102668]: 2023-09-10T01:37:51.234+0000 7f6132d23700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:37:51.237 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 10 01:37:51 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mon-b[102668]: 2023-09-10T01:37:51.234+0000 7f6132d23700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-09-10T01:37:51.776 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mon.b.service' 2023-09-10T01:37:51.835 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:37:51.835 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-09-10T01:37:51.836 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-09-10T01:37:51.836 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mon.c 2023-09-10T01:37:52.209 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:51 smithi131 systemd[1]: Stopping Ceph mon.c for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:37:52.473 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:52 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mon-c[103689]: 2023-09-10T01:37:52.206+0000 7ff19b122700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:37:52.473 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 01:37:52 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mon-c[103689]: 2023-09-10T01:37:52.206+0000 7ff19b122700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-09-10T01:37:52.947 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mon.c.service' 2023-09-10T01:37:52.989 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:37:52.990 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-09-10T01:37:52.990 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-09-10T01:37:52.991 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mgr.x 2023-09-10T01:37:53.305 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 01:37:53 smithi044 systemd[1]: Stopping Ceph mgr.x for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:37:53.627 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 01:37:53 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x[98933]: 2023-09-10T01:37:53.345+0000 7f34af2d1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:37:53.904 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Sep 10 01:37:53 smithi044 bash[239579]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-x 2023-09-10T01:37:53.910 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mgr.x.service' 2023-09-10T01:37:53.948 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:37:53.949 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-09-10T01:37:53.949 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-09-10T01:37:53.950 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mgr.y 2023-09-10T01:37:54.356 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 01:37:53 smithi099 systemd[1]: Stopping Ceph mgr.y for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:37:54.661 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 01:37:54 smithi099 bash[157789]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-y 2023-09-10T01:37:54.842 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 01:37:54 smithi099 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-09-10T01:37:54.847 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mgr.y.service' 2023-09-10T01:37:54.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:37:54.885 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-09-10T01:37:54.885 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-09-10T01:37:54.886 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mgr.z 2023-09-10T01:37:55.295 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 01:37:54 smithi131 systemd[1]: Stopping Ceph mgr.z for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:37:56.183 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 01:37:55 smithi131 bash[160908]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mgr-z 2023-09-10T01:37:56.184 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 10 01:37:56 smithi131 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-09-10T01:37:56.190 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mgr.z.service' 2023-09-10T01:37:56.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:37:56.228 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-09-10T01:37:56.228 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-09-10T01:37:56.229 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.0 2023-09-10T01:37:56.555 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 01:37:56 smithi044 systemd[1]: Stopping Ceph osd.0 for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:37:57.055 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 01:37:56 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-0[121843]: 2023-09-10T01:37:56.592+0000 7f7c69a37700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:37:57.055 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 01:37:56 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-0[121843]: 2023-09-10T01:37:56.592+0000 7f7c69a37700 -1 osd.0 213 *** Got signal Terminated *** 2023-09-10T01:37:57.056 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 01:37:56 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-0[121843]: 2023-09-10T01:37:56.592+0000 7f7c69a37700 -1 osd.0 213 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-10T01:38:02.805 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 01:38:02 smithi044 bash[239722]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-0 2023-09-10T01:38:04.089 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.0.service' 2023-09-10T01:38:04.116 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 01:38:04 smithi044 systemd[1]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.0.service: Succeeded. 2023-09-10T01:38:04.116 INFO:journalctl@ceph.osd.0.smithi044.stdout:Sep 10 01:38:04 smithi044 systemd[1]: Stopped Ceph osd.0 for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da. 2023-09-10T01:38:04.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:38:04.132 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-09-10T01:38:04.132 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-09-10T01:38:04.133 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.1 2023-09-10T01:38:04.484 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 01:38:04 smithi099 systemd[1]: Stopping Ceph osd.1 for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:38:04.856 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 01:38:04 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-1[107794]: 2023-09-10T01:38:04.482+0000 7fd01cbd3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:38:04.856 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 01:38:04 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-1[107794]: 2023-09-10T01:38:04.482+0000 7fd01cbd3700 -1 osd.1 213 *** Got signal Terminated *** 2023-09-10T01:38:04.857 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 01:38:04 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-1[107794]: 2023-09-10T01:38:04.482+0000 7fd01cbd3700 -1 osd.1 213 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-10T01:38:10.856 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 01:38:10 smithi099 bash[157938]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-1 2023-09-10T01:38:12.032 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.1.service' 2023-09-10T01:38:12.072 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:38:12.073 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-09-10T01:38:12.073 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-09-10T01:38:12.074 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.2 2023-09-10T01:38:12.545 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 01:38:12 smithi131 systemd[1]: Stopping Ceph osd.2 for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:38:13.045 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 01:38:12 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-2[109574]: 2023-09-10T01:38:12.561+0000 7f459bdd6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:38:13.045 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 01:38:12 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-2[109574]: 2023-09-10T01:38:12.561+0000 7f459bdd6700 -1 osd.2 213 *** Got signal Terminated *** 2023-09-10T01:38:13.045 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 01:38:12 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-2[109574]: 2023-09-10T01:38:12.561+0000 7f459bdd6700 -1 osd.2 213 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-10T01:38:19.045 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 01:38:18 smithi131 bash[161057]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-2 2023-09-10T01:38:20.283 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.2.service' 2023-09-10T01:38:20.323 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:38:20.324 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-09-10T01:38:20.324 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-09-10T01:38:20.325 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.3 2023-09-10T01:38:20.676 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 10 01:38:20 smithi044 systemd[1]: Stopping Ceph osd.3 for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:38:21.055 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 10 01:38:20 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-3[130551]: 2023-09-10T01:38:20.674+0000 7f5bad017700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:38:21.055 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 10 01:38:20 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-3[130551]: 2023-09-10T01:38:20.674+0000 7f5bad017700 -1 osd.3 213 *** Got signal Terminated *** 2023-09-10T01:38:21.056 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 10 01:38:20 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-3[130551]: 2023-09-10T01:38:20.674+0000 7f5bad017700 -1 osd.3 213 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-10T01:38:23.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:23 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:23.552+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:24.056 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:23 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:23.678+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:24.056 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:23 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:23.921+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:24.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:24 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:24.580+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:25.055 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:24 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:24.718+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:25.056 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:24 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:24.968+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:25.295 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:24 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:24.950+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:25.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:25 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:25.055+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:25.910 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:25 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:25.570+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.497818+0000 front 2023-09-10T01:38:01.497810+0000 (oldest deadline 2023-09-10T01:38:24.997840+0000) 2023-09-10T01:38:26.015 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:25 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:25.721+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:26.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:25 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:25.612+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:26.295 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:25 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:25.909+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:26.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:26 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:26.072+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:26.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:26 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:26.014+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:26.673 INFO:journalctl@ceph.osd.3.smithi044.stdout:Sep 10 01:38:26 smithi044 bash[239997]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-3 2023-09-10T01:38:26.953 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:26 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:26.585+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.497818+0000 front 2023-09-10T01:38:01.497810+0000 (oldest deadline 2023-09-10T01:38:24.997840+0000) 2023-09-10T01:38:27.055 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:26 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:26.758+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:27.055 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:26 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:26.995+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:27.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:26 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:26.637+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:27.295 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:26 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:26.952+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:27.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:27 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:27.093+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:27.837 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:27 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:27.772+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:27.844 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.3.service' 2023-09-10T01:38:27.886 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:38:27.886 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-09-10T01:38:27.887 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-09-10T01:38:27.887 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.4 2023-09-10T01:38:27.916 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:27 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:27.641+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:27.983 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:27 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:27.585+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.497818+0000 front 2023-09-10T01:38:01.497810+0000 (oldest deadline 2023-09-10T01:38:24.997840+0000) 2023-09-10T01:38:28.244 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 10 01:38:27 smithi099 systemd[1]: Stopping Ceph osd.4 for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:38:28.295 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:27 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:27.982+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:28.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:28 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:28.076+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:28.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:27 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:27.987+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:28.606 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 10 01:38:28 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-4[115726]: 2023-09-10T01:38:28.243+0000 7f487f827700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:38:28.606 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 10 01:38:28 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-4[115726]: 2023-09-10T01:38:28.243+0000 7f487f827700 -1 osd.4 213 *** Got signal Terminated *** 2023-09-10T01:38:28.606 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 10 01:38:28 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-4[115726]: 2023-09-10T01:38:28.243+0000 7f487f827700 -1 osd.4 213 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-10T01:38:28.941 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:28 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:28.577+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.497818+0000 front 2023-09-10T01:38:01.497810+0000 (oldest deadline 2023-09-10T01:38:24.997840+0000) 2023-09-10T01:38:29.055 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:28 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:28.786+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:29.056 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:29 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:29.013+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:29.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:28 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:28.678+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:29.295 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:28 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:28.939+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:29.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:29 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:29.060+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:29.977 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:29 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:29.622+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.497818+0000 front 2023-09-10T01:38:01.497810+0000 (oldest deadline 2023-09-10T01:38:24.997840+0000) 2023-09-10T01:38:30.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:29 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:29.699+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:30.295 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:29 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:29.976+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:30.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:30 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:30.037+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:30.305 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:29 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:29.808+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:30.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:30 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:30.052+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:30.950 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:30 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:30.618+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.497818+0000 front 2023-09-10T01:38:01.497810+0000 (oldest deadline 2023-09-10T01:38:24.997840+0000) 2023-09-10T01:38:31.055 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:30 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:30.763+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:31.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:30 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:30.676+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:31.295 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:30 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:30.949+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:31.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:31 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:31.076+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:31.555 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:31 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:31.100+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:31.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:31 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:31.178+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:31.921 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:31 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:31.637+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.497818+0000 front 2023-09-10T01:38:01.497810+0000 (oldest deadline 2023-09-10T01:38:24.997840+0000) 2023-09-10T01:38:32.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:31 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:31.681+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:32.118 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:31 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:31.809+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:32.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:32 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:32.125+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:32.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:32 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:32.125+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:32.296 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:31 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:31.920+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:32.296 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:31 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:31.920+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:32.555 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:32 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:32.118+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:32.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:32 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:32.194+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:32.935 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:32 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:32.634+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.497818+0000 front 2023-09-10T01:38:01.497810+0000 (oldest deadline 2023-09-10T01:38:24.997840+0000) 2023-09-10T01:38:32.969 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:32 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:32.635+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:33.055 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:32 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:32.776+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:33.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:33 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:33.147+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:33.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:33 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:33.147+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:33.296 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:32 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:32.935+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:33.296 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:32 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:32.935+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:33.356 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:33 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:33.199+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:33.356 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 10 01:38:32 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-4[115726]: 2023-09-10T01:38:32.968+0000 7f4879e19700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.978817+0000 front 2023-09-10T01:38:08.978845+0000 (oldest deadline 2023-09-10T01:38:32.479025+0000) 2023-09-10T01:38:33.555 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:33 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:33.147+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:33.897 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:33 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:33.587+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.497818+0000 front 2023-09-10T01:38:01.497810+0000 (oldest deadline 2023-09-10T01:38:24.997840+0000) 2023-09-10T01:38:34.055 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:33 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:33.776+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:34.092 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:33 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:33.619+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:34.295 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:33 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:33.897+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:34.295 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:33 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:33.897+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:34.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:34 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:34.114+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:34.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:34 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:34.114+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:34.357 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 10 01:38:34 smithi099 bash[158202]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-4 2023-09-10T01:38:34.357 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:34 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:34.152+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:34.555 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:34 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:34.157+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:34.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:34 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:34.578+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:34.879 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:34 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:34.568+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.497818+0000 front 2023-09-10T01:38:01.497810+0000 (oldest deadline 2023-09-10T01:38:24.997840+0000) 2023-09-10T01:38:34.879 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:34 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:34.568+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.999669+0000 front 2023-09-10T01:38:08.999797+0000 (oldest deadline 2023-09-10T01:38:34.200292+0000) 2023-09-10T01:38:35.055 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:34 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:34.727+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:35.295 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:34 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:34.878+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:35.295 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:34 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:34.878+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:35.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:35 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:35.116+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:35.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:35 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:35.116+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:35.430 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:35 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:35.169+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:35.436 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.4.service' 2023-09-10T01:38:35.516 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:38:35.517 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-09-10T01:38:35.517 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-09-10T01:38:35.518 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.5 2023-09-10T01:38:35.555 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:35 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:35.113+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:35.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:35 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:35.545+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:35.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:35 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:35.545+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:35.879 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:35 smithi131 systemd[1]: Stopping Ceph osd.5 for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:38:35.880 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:35 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:35.610+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.497818+0000 front 2023-09-10T01:38:01.497810+0000 (oldest deadline 2023-09-10T01:38:24.997840+0000) 2023-09-10T01:38:35.880 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:35 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:35.610+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.999669+0000 front 2023-09-10T01:38:08.999797+0000 (oldest deadline 2023-09-10T01:38:34.200292+0000) 2023-09-10T01:38:36.055 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:35 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:35.714+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:36.157 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:35 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:35.878+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:36.158 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:35 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:35.878+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:36.158 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:35 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:35.882+0000 7f1a49e03700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:38:36.158 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:35 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:35.882+0000 7f1a49e03700 -1 osd.5 213 *** Got signal Terminated *** 2023-09-10T01:38:36.158 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:35 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:35.882+0000 7f1a49e03700 -1 osd.5 213 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-10T01:38:36.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:36 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:36.157+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:36.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:36 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:36.157+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:36.555 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:36 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:36.088+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:36.555 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:36 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:36.088+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:36.563 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:36 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:36.203+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:36.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:36 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:36.564+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:36.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:36 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:36.564+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:37.045 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:36 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:36.642+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.497818+0000 front 2023-09-10T01:38:01.497810+0000 (oldest deadline 2023-09-10T01:38:24.997840+0000) 2023-09-10T01:38:37.046 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:36 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:36.642+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.999669+0000 front 2023-09-10T01:38:08.999797+0000 (oldest deadline 2023-09-10T01:38:34.200292+0000) 2023-09-10T01:38:37.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:36 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:36.829+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:37.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:36 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:36.829+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:37.047 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:36 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:36.736+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:37.047 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:36 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:36.736+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:05.959189+0000 front 2023-09-10T01:38:05.959169+0000 (oldest deadline 2023-09-10T01:38:35.959894+0000) 2023-09-10T01:38:37.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:37 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:37.046+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:37.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:37 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:37.047+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:37.540 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:37 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:37.240+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:37.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:37 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:37.111+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:37.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:37 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:37.111+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:37.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:37 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:37.539+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:37.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:37 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:37.539+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:38.045 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:37 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:37.668+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.497818+0000 front 2023-09-10T01:38:01.497810+0000 (oldest deadline 2023-09-10T01:38:24.997840+0000) 2023-09-10T01:38:38.045 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:37 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:37.668+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.999669+0000 front 2023-09-10T01:38:08.999797+0000 (oldest deadline 2023-09-10T01:38:34.200292+0000) 2023-09-10T01:38:38.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:37 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:37.853+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:38.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:37 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:37.853+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:38.055 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:37 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:37.741+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:38.055 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:37 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:37.741+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:05.959189+0000 front 2023-09-10T01:38:05.959169+0000 (oldest deadline 2023-09-10T01:38:35.959894+0000) 2023-09-10T01:38:38.498 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:38 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:38.208+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:38.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:38 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:38.106+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:38.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:38 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:38.106+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:38.555 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:38 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:38.058+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:38.555 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:38 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:38.058+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:38.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:38 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:38.497+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:38.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:38 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:38.497+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:39.045 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:38 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:38.694+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.497818+0000 front 2023-09-10T01:38:01.497810+0000 (oldest deadline 2023-09-10T01:38:24.997840+0000) 2023-09-10T01:38:39.045 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:38 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:38.694+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.999669+0000 front 2023-09-10T01:38:08.999797+0000 (oldest deadline 2023-09-10T01:38:34.200292+0000) 2023-09-10T01:38:39.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:38 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:38.832+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:39.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:38 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:38.832+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:39.055 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:38 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:38.694+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:39.055 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:38 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:38.694+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:05.959189+0000 front 2023-09-10T01:38:05.959169+0000 (oldest deadline 2023-09-10T01:38:35.959894+0000) 2023-09-10T01:38:39.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:39 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:39.148+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:39.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:39 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:39.148+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:39.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:39 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:39.148+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:39.555 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:39 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:39.066+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:39.555 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:39 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:39.066+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:39.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:39 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:39.251+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:39.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:39 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:39.471+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:39.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:39 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:39.472+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:40.023 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:39 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:39.720+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:40.023 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:39 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:39.720+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:05.959189+0000 front 2023-09-10T01:38:05.959169+0000 (oldest deadline 2023-09-10T01:38:35.959894+0000) 2023-09-10T01:38:40.023 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:39 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:39.720+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:15.960653+0000 front 2023-09-10T01:38:15.960685+0000 (oldest deadline 2023-09-10T01:38:39.460610+0000) 2023-09-10T01:38:40.045 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:39 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:39.717+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.497818+0000 front 2023-09-10T01:38:01.497810+0000 (oldest deadline 2023-09-10T01:38:24.997840+0000) 2023-09-10T01:38:40.045 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:39 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:39.717+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.999669+0000 front 2023-09-10T01:38:08.999797+0000 (oldest deadline 2023-09-10T01:38:34.200292+0000) 2023-09-10T01:38:40.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:39 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:39.852+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:40.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:39 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:39.852+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:40.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:40 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:40.022+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:40.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:40 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:40.022+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:40.519 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:40 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:40.268+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:40.520 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:40 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:40.518+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:40.520 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:40 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:40.518+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:40.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:40 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:40.190+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:40.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:40 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:40.191+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:40.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:40 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:40.191+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:41.003 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:40 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:40.679+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:41.003 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:40 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:40.679+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:05.959189+0000 front 2023-09-10T01:38:05.959169+0000 (oldest deadline 2023-09-10T01:38:35.959894+0000) 2023-09-10T01:38:41.003 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:40 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:40.679+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:15.960653+0000 front 2023-09-10T01:38:15.960685+0000 (oldest deadline 2023-09-10T01:38:39.460610+0000) 2023-09-10T01:38:41.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:40 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:40.860+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:41.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:40 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:40.860+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:41.046 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:40 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:40.733+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.497818+0000 front 2023-09-10T01:38:01.497810+0000 (oldest deadline 2023-09-10T01:38:24.997840+0000) 2023-09-10T01:38:41.046 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:40 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:40.733+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.999669+0000 front 2023-09-10T01:38:08.999797+0000 (oldest deadline 2023-09-10T01:38:34.200292+0000) 2023-09-10T01:38:41.046 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:40 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5[118187]: 2023-09-10T01:38:40.733+0000 7f1a43bf4700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.200720+0000 front 2023-09-10T01:38:14.200662+0000 (oldest deadline 2023-09-10T01:38:40.100892+0000) 2023-09-10T01:38:41.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:41 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:41.002+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:41.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:41 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:41.002+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:41.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:41 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:41.150+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:41.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:41 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:41.150+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:41.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:41 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:41.150+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:41.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:41 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:41.291+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:41.607 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:41 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:41.291+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.410511+0000 front 2023-09-10T01:38:16.410561+0000 (oldest deadline 2023-09-10T01:38:40.510760+0000) 2023-09-10T01:38:41.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:41 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:41.504+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:41.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:41 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:41.504+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:41.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:41 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:41.504+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:41.988 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:41 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:41.640+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:41.989 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:41 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:41.640+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:05.959189+0000 front 2023-09-10T01:38:05.959169+0000 (oldest deadline 2023-09-10T01:38:35.959894+0000) 2023-09-10T01:38:41.989 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:41 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:41.640+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:15.960653+0000 front 2023-09-10T01:38:15.960685+0000 (oldest deadline 2023-09-10T01:38:39.460610+0000) 2023-09-10T01:38:42.122 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:41 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:41.833+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:42.123 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:41 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:41.833+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:42.123 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:41 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:41.833+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:42.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:41 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:41.988+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:42.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:41 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:41.988+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:42.545 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 10 01:38:42 smithi131 bash[161331]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-5 2023-09-10T01:38:42.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:42 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:42.136+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:42.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:42 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:42.136+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:42.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:42 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:42.136+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:42.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:42 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:42.318+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:42.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:42 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:42.318+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.410511+0000 front 2023-09-10T01:38:16.410561+0000 (oldest deadline 2023-09-10T01:38:40.510760+0000) 2023-09-10T01:38:42.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:42 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:42.513+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:42.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:42 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:42.513+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:42.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:42 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:42.513+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:42.982 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:42 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:42.623+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:42.982 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:42 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:42.623+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:05.959189+0000 front 2023-09-10T01:38:05.959169+0000 (oldest deadline 2023-09-10T01:38:35.959894+0000) 2023-09-10T01:38:42.983 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:42 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:42.623+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:15.960653+0000 front 2023-09-10T01:38:15.960685+0000 (oldest deadline 2023-09-10T01:38:39.460610+0000) 2023-09-10T01:38:43.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:42 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:42.784+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:43.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:42 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:42.784+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:43.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:42 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:42.784+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:43.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:42 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:42.982+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:43.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:42 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:42.982+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:43.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:42 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:42.982+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:43.516 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:43 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:43.174+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:43.517 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:43 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:43.174+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:43.517 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:43 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:43.174+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:43.524 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.5.service' 2023-09-10T01:38:43.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:38:43.566 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-09-10T01:38:43.567 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-09-10T01:38:43.567 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.6 2023-09-10T01:38:43.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:43 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:43.287+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:43.607 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:43 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:43.287+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.410511+0000 front 2023-09-10T01:38:16.410561+0000 (oldest deadline 2023-09-10T01:38:40.510760+0000) 2023-09-10T01:38:43.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:43 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:43.477+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:43.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:43 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:43.477+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:43.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:43 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:43.477+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:43.923 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:43 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:43.578+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:43.923 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:43 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:43.578+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:05.959189+0000 front 2023-09-10T01:38:05.959169+0000 (oldest deadline 2023-09-10T01:38:35.959894+0000) 2023-09-10T01:38:43.923 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:43 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:43.578+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:15.960653+0000 front 2023-09-10T01:38:15.960685+0000 (oldest deadline 2023-09-10T01:38:39.460610+0000) 2023-09-10T01:38:43.923 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:43 smithi044 systemd[1]: Stopping Ceph osd.6 for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:38:43.924 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:43 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:43.921+0000 7fc96ac1f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:38:43.924 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:43 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:43.921+0000 7fc96ac1f700 -1 osd.6 213 *** Got signal Terminated *** 2023-09-10T01:38:44.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:43 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:43.772+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:44.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:43 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:43.772+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:44.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:43 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:43.772+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:44.305 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:43 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:43.921+0000 7fc96ac1f700 -1 osd.6 213 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-10T01:38:44.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:43 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:43.946+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:44.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:43 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:43.946+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:44.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:43 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:43.946+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:44.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:44 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:44.221+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:44.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:44 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:44.221+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:44.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:44 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:44.221+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:44.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:44 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:44.284+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:44.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:44 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:44.284+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.410511+0000 front 2023-09-10T01:38:16.410561+0000 (oldest deadline 2023-09-10T01:38:40.510760+0000) 2023-09-10T01:38:44.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:44 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:44.448+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:44.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:44 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:44.448+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:44.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:44 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:44.448+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:44.992 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:44 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:44.609+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:44.993 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:44 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:44.609+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:05.959189+0000 front 2023-09-10T01:38:05.959169+0000 (oldest deadline 2023-09-10T01:38:35.959894+0000) 2023-09-10T01:38:44.993 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:44 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:44.609+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:15.960653+0000 front 2023-09-10T01:38:15.960685+0000 (oldest deadline 2023-09-10T01:38:39.460610+0000) 2023-09-10T01:38:45.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:44 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:44.735+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:45.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:44 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:44.735+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:45.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:44 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:44.735+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:45.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:44 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:44.991+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:45.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:44 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:44.991+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:45.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:44 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:44.991+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:45.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:45 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:45.232+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:45.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:45 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:45.232+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:45.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:45 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:45.232+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:45.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:45 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:45.334+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:45.607 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:45 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:45.334+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.410511+0000 front 2023-09-10T01:38:16.410561+0000 (oldest deadline 2023-09-10T01:38:40.510760+0000) 2023-09-10T01:38:45.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:45 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:45.420+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:45.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:45 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:45.420+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:45.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:45 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:45.420+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:45.950 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:45 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:45.593+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:45.951 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:45 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:45.593+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:05.959189+0000 front 2023-09-10T01:38:05.959169+0000 (oldest deadline 2023-09-10T01:38:35.959894+0000) 2023-09-10T01:38:45.951 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:45 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:45.593+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:15.960653+0000 front 2023-09-10T01:38:15.960685+0000 (oldest deadline 2023-09-10T01:38:39.460610+0000) 2023-09-10T01:38:46.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:45 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:45.744+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:46.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:45 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:45.745+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:46.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:45 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:45.745+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:46.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:45 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:45.950+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:46.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:45 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:45.950+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:46.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:45 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:45.950+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:46.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:46 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:46.182+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:46.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:46 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:46.182+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:46.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:46 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:46.182+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:46.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:46 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:46.316+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:46.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:46 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:46.316+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.410511+0000 front 2023-09-10T01:38:16.410561+0000 (oldest deadline 2023-09-10T01:38:40.510760+0000) 2023-09-10T01:38:46.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:46 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:46.462+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:46.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:46 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:46.462+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:46.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:46 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:46.462+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:46.968 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:46 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:46.613+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:46.968 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:46 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:46.613+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:05.959189+0000 front 2023-09-10T01:38:05.959169+0000 (oldest deadline 2023-09-10T01:38:35.959894+0000) 2023-09-10T01:38:46.968 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:46 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:46.613+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:15.960653+0000 front 2023-09-10T01:38:15.960685+0000 (oldest deadline 2023-09-10T01:38:39.460610+0000) 2023-09-10T01:38:47.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:46 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:46.769+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:47.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:46 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:46.769+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:47.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:46 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:46.769+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:47.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:46 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:46.967+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:47.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:46 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:46.967+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:47.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:46 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:46.967+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:47.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:47 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:47.158+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:47.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:47 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:47.158+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:47.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:47 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:47.158+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:47.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:47 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:47.346+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:47.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:47 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:47.346+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.410511+0000 front 2023-09-10T01:38:16.410561+0000 (oldest deadline 2023-09-10T01:38:40.510760+0000) 2023-09-10T01:38:47.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:47 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:47.505+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:47.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:47 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:47.505+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:47.608 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:47 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:47.505+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:47.951 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:47 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:47.635+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:47.952 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:47 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:47.635+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:05.959189+0000 front 2023-09-10T01:38:05.959169+0000 (oldest deadline 2023-09-10T01:38:35.959894+0000) 2023-09-10T01:38:47.952 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:47 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:47.635+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:15.960653+0000 front 2023-09-10T01:38:15.960685+0000 (oldest deadline 2023-09-10T01:38:39.460610+0000) 2023-09-10T01:38:48.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:47 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:47.769+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:48.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:47 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:47.769+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:48.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:47 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:47.769+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:48.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:47 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:47.950+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:48.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:47 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:47.950+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:48.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:47 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:47.950+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:48.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:47 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:47.950+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:38:48.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:48 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:48.148+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:48.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:48 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:48.148+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:48.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:48 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:48.148+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:48.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:48 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:48.310+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:48.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:48 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:48.310+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.410511+0000 front 2023-09-10T01:38:16.410561+0000 (oldest deadline 2023-09-10T01:38:40.510760+0000) 2023-09-10T01:38:48.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:48 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:48.539+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:48.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:48 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:48.539+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:48.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:48 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:48.539+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:48.934 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:48 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:48.633+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.357498+0000 front 2023-09-10T01:38:01.357527+0000 (oldest deadline 2023-09-10T01:38:23.657641+0000) 2023-09-10T01:38:48.935 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:48 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:48.633+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:05.959189+0000 front 2023-09-10T01:38:05.959169+0000 (oldest deadline 2023-09-10T01:38:35.959894+0000) 2023-09-10T01:38:48.935 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:48 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:48.633+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:15.960653+0000 front 2023-09-10T01:38:15.960685+0000 (oldest deadline 2023-09-10T01:38:39.460610+0000) 2023-09-10T01:38:48.935 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:48 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6[138634]: 2023-09-10T01:38:48.633+0000 7fc966213700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.762179+0000 front 2023-09-10T01:38:24.762203+0000 (oldest deadline 2023-09-10T01:38:47.662303+0000) 2023-09-10T01:38:49.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:48 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:48.786+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:49.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:48 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:48.786+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:49.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:48 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:48.786+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:49.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:48 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:48.933+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:49.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:48 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:48.933+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:49.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:48 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:48.933+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:49.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:48 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:48.933+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:38:49.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:49 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:49.151+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:49.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:49 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:49.151+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:49.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:49 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:49.151+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:49.566 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:49 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:49.312+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:49.566 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:49 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:49.312+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.410511+0000 front 2023-09-10T01:38:16.410561+0000 (oldest deadline 2023-09-10T01:38:40.510760+0000) 2023-09-10T01:38:49.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:49 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:49.565+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:49.858 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:49 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:49.565+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:49.858 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:49 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:49.565+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:49.858 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:49 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:49.565+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:38:50.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:49 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:49.777+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:50.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:49 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:49.777+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:50.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:49 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:49.777+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:50.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:49 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:49.777+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.951898+0000 front 2023-09-10T01:38:24.952074+0000 (oldest deadline 2023-09-10T01:38:49.651938+0000) 2023-09-10T01:38:50.055 INFO:journalctl@ceph.osd.6.smithi044.stdout:Sep 10 01:38:49 smithi044 bash[240271]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-6 2023-09-10T01:38:50.056 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:49 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:49.957+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:50.056 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:49 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:49.957+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:50.056 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:49 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:49.957+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:50.056 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:49 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:49.957+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:38:50.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:50 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:50.134+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:50.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:50 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:50.134+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:50.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:50 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:50.134+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:50.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:50 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:50.307+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:50.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:50 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:50.307+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.410511+0000 front 2023-09-10T01:38:16.410561+0000 (oldest deadline 2023-09-10T01:38:40.510760+0000) 2023-09-10T01:38:51.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:50 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:50.781+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:51.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:50 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:50.781+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:51.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:50 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:50.781+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:51.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:50 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:50.781+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.951898+0000 front 2023-09-10T01:38:24.952074+0000 (oldest deadline 2023-09-10T01:38:49.651938+0000) 2023-09-10T01:38:51.082 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:50 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:50.916+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:51.082 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:50 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:50.916+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:51.083 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:50 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:50.916+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:51.083 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:50 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:50.916+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:38:51.090 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.6.service' 2023-09-10T01:38:51.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:50 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:50.612+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:51.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:50 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:50.612+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:51.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:50 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:50.612+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:51.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:50 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:50.612+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:38:51.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:38:51.132 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-09-10T01:38:51.132 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-09-10T01:38:51.133 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.7 2023-09-10T01:38:51.472 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:51 smithi099 systemd[1]: Stopping Ceph osd.7 for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:38:51.473 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:51 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:51.357+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:51.473 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:51 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:51.357+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.410511+0000 front 2023-09-10T01:38:16.410561+0000 (oldest deadline 2023-09-10T01:38:40.510760+0000) 2023-09-10T01:38:51.473 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:51 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:51.357+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.612294+0000 front 2023-09-10T01:38:24.612313+0000 (oldest deadline 2023-09-10T01:38:51.012555+0000) 2023-09-10T01:38:51.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:51 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:51.150+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:51.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:51 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:51.150+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:51.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:51 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:51.150+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:51.856 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:51 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:51.471+0000 7f55f86cf700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:38:51.856 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:51 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:51.471+0000 7f55f86cf700 -1 osd.7 213 *** Got signal Terminated *** 2023-09-10T01:38:51.857 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:51 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:51.471+0000 7f55f86cf700 -1 osd.7 213 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-10T01:38:51.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:51 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:51.597+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:51.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:51 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:51.597+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:51.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:51 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:51.597+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:51.858 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:51 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:51.597+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:38:52.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:51 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:51.734+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:52.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:51 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:51.734+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:52.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:51 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:51.734+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:52.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:51 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:51.734+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.951898+0000 front 2023-09-10T01:38:24.952074+0000 (oldest deadline 2023-09-10T01:38:49.651938+0000) 2023-09-10T01:38:52.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:51 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:51.920+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:52.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:51 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:51.920+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:52.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:51 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:51.920+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:52.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:51 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:51.920+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:38:52.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:52 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:52.141+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:52.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:52 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:52.141+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:52.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:52 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:52.141+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:52.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:52 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:52.343+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:52.606 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:52 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:52.343+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.410511+0000 front 2023-09-10T01:38:16.410561+0000 (oldest deadline 2023-09-10T01:38:40.510760+0000) 2023-09-10T01:38:52.607 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:52 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:52.343+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.612294+0000 front 2023-09-10T01:38:24.612313+0000 (oldest deadline 2023-09-10T01:38:51.012555+0000) 2023-09-10T01:38:53.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:52 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:52.745+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:53.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:52 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:52.745+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:53.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:52 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:52.745+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:53.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:52 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:52.745+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.951898+0000 front 2023-09-10T01:38:24.952074+0000 (oldest deadline 2023-09-10T01:38:49.651938+0000) 2023-09-10T01:38:53.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:52 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:52.641+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:53.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:52 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:52.641+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:53.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:52 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:52.641+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:53.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:52 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:52.641+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:38:53.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:52 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:52.949+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:53.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:52 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:52.949+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:53.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:52 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:52.949+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:53.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:52 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:52.949+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:38:53.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:53 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:53.100+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:53.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:53 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:53.100+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:53.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:53 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:53.100+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:53.675 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:53 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:53.386+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:53.675 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:53 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:53.386+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.410511+0000 front 2023-09-10T01:38:16.410561+0000 (oldest deadline 2023-09-10T01:38:40.510760+0000) 2023-09-10T01:38:53.675 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:53 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:53.386+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.612294+0000 front 2023-09-10T01:38:24.612313+0000 (oldest deadline 2023-09-10T01:38:51.012555+0000) 2023-09-10T01:38:54.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:53 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:53.740+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:54.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:53 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:53.740+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:54.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:53 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:53.740+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:54.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:53 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:53.740+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.951898+0000 front 2023-09-10T01:38:24.952074+0000 (oldest deadline 2023-09-10T01:38:49.651938+0000) 2023-09-10T01:38:54.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:53 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:53.674+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:54.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:53 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:53.674+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:54.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:53 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:53.674+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:54.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:53 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:53.674+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:38:54.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:53 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:53.992+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:54.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:53 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:53.992+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:54.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:53 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:53.992+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:54.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:53 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:53.992+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:38:54.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:54 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:54.091+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:54.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:54 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:54.091+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:54.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:54 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:54.091+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:54.646 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:54 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:54.371+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:54.646 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:54 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:54.371+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.410511+0000 front 2023-09-10T01:38:16.410561+0000 (oldest deadline 2023-09-10T01:38:40.510760+0000) 2023-09-10T01:38:54.646 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:54 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:54.371+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.612294+0000 front 2023-09-10T01:38:24.612313+0000 (oldest deadline 2023-09-10T01:38:51.012555+0000) 2023-09-10T01:38:55.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:54 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:54.704+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:55.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:54 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:54.704+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:55.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:54 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:54.704+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:55.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:54 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:54.704+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.951898+0000 front 2023-09-10T01:38:24.952074+0000 (oldest deadline 2023-09-10T01:38:49.651938+0000) 2023-09-10T01:38:55.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:54 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:54.645+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:55.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:54 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:54.645+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:55.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:54 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:54.645+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:55.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:54 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:54.645+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:38:55.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:54 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:54.645+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:38:55.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:54.987+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:55.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:54.987+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:55.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:54.987+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:55.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:54 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:54.987+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:38:55.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:55 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:55.063+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:55.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:55 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:55.063+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:55.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:55 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:55.063+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:55.638 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:55 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:55.364+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:55.639 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:55 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:55.364+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.410511+0000 front 2023-09-10T01:38:16.410561+0000 (oldest deadline 2023-09-10T01:38:40.510760+0000) 2023-09-10T01:38:55.639 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:55 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:55.364+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.612294+0000 front 2023-09-10T01:38:24.612313+0000 (oldest deadline 2023-09-10T01:38:51.012555+0000) 2023-09-10T01:38:56.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:55 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:55.718+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:56.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:55 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:55.718+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:56.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:55 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:55.718+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:56.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:55 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:55.718+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.951898+0000 front 2023-09-10T01:38:24.952074+0000 (oldest deadline 2023-09-10T01:38:49.651938+0000) 2023-09-10T01:38:56.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:55 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:55.718+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:29.652611+0000 front 2023-09-10T01:38:29.652576+0000 (oldest deadline 2023-09-10T01:38:55.552676+0000) 2023-09-10T01:38:56.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:55 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:55.637+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:56.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:55 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:55.637+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:56.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:55 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:55.637+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:56.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:55 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:55.637+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:38:56.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:55 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:55.637+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:38:56.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:55 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:55.980+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:56.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:55 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:55.980+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:56.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:55 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:55.980+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:56.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:55 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:55.980+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:38:56.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:55 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:55.980+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:38:56.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:56 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:56.061+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:56.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:56 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:56.061+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:56.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:56 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:56.061+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:56.650 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:56 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:56.361+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.408178+0000 front 2023-09-10T01:38:08.408205+0000 (oldest deadline 2023-09-10T01:38:30.608982+0000) 2023-09-10T01:38:56.650 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:56 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:56.361+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.410511+0000 front 2023-09-10T01:38:16.410561+0000 (oldest deadline 2023-09-10T01:38:40.510760+0000) 2023-09-10T01:38:56.650 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:56 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:56.361+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.612294+0000 front 2023-09-10T01:38:24.612313+0000 (oldest deadline 2023-09-10T01:38:51.012555+0000) 2023-09-10T01:38:56.651 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:56 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7[123802]: 2023-09-10T01:38:56.361+0000 7f55f34c2700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:31.012866+0000 front 2023-09-10T01:38:31.012892+0000 (oldest deadline 2023-09-10T01:38:55.713106+0000) 2023-09-10T01:38:57.045 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:56 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:56.714+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:57.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:56 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:56.714+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:57.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:56 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:56.714+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:57.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:56 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:56.714+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.951898+0000 front 2023-09-10T01:38:24.952074+0000 (oldest deadline 2023-09-10T01:38:49.651938+0000) 2023-09-10T01:38:57.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:56 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:56.714+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:29.652611+0000 front 2023-09-10T01:38:29.652576+0000 (oldest deadline 2023-09-10T01:38:55.552676+0000) 2023-09-10T01:38:57.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:56 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:56.649+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:57.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:56 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:56.649+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:57.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:56 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:56.649+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:57.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:56 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:56.649+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:38:57.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:56 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:56.649+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:38:57.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:56 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:56.931+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:57.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:56 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:56.931+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:57.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:56 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:56.931+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:57.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:56 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:56.931+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:38:57.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:56 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:56.931+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:38:57.545 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:57 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:57.055+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:57.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:57 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:57.055+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:57.546 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:57 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:57.055+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:57.603 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 10 01:38:57 smithi099 bash[158483]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-7 2023-09-10T01:38:57.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:57 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:57.603+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:57.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:57 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:57.603+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:57.858 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:57 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:57.603+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:57.858 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:57 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:57.603+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:38:57.858 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:57 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:57.603+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:38:58.042 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:57 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:57.712+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:58.042 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:57 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:57.712+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:58.042 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:57 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:57.712+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:58.042 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:57 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:57.712+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.951898+0000 front 2023-09-10T01:38:24.952074+0000 (oldest deadline 2023-09-10T01:38:49.651938+0000) 2023-09-10T01:38:58.043 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:57 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:57.712+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:29.652611+0000 front 2023-09-10T01:38:29.652576+0000 (oldest deadline 2023-09-10T01:38:55.552676+0000) 2023-09-10T01:38:58.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:58 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:58.041+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:58.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:58 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:58.041+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:58.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:58 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:58.041+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:58.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:58 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:58.041+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:38:58.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:57 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:57.897+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:58.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:57 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:57.897+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:58.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:57 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:57.897+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:58.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:57 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:57.897+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:38:58.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:57 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:57.897+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:38:58.628 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.7.service' 2023-09-10T01:38:58.667 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:38:58.667 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-09-10T01:38:58.668 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-09-10T01:38:58.668 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.8 2023-09-10T01:38:59.002 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:58 smithi131 systemd[1]: Stopping Ceph osd.8 for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:38:59.002 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:58 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:58.761+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:38:59.003 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:58 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:58.761+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:38:59.003 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:58 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:58.761+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:38:59.003 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:58 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:58.761+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.951898+0000 front 2023-09-10T01:38:24.952074+0000 (oldest deadline 2023-09-10T01:38:49.651938+0000) 2023-09-10T01:38:59.003 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:58 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:58.761+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:29.652611+0000 front 2023-09-10T01:38:29.652576+0000 (oldest deadline 2023-09-10T01:38:55.552676+0000) 2023-09-10T01:38:59.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:58 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:58.609+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:38:59.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:58 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:58.609+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:38:59.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:58 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:58.609+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:38:59.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:58 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:58.609+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:38:59.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:58 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:58.609+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:38:59.295 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:59 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:59.037+0000 7ff39bdca700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:38:59.295 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:59 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:59.037+0000 7ff39bdca700 -1 osd.8 213 *** Got signal Terminated *** 2023-09-10T01:38:59.296 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:59 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:59.037+0000 7ff39bdca700 -1 osd.8 213 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-10T01:38:59.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:59 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:59.000+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:38:59.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:59 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:59.000+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:38:59.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:59 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:59.000+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:38:59.297 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:59 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:59.000+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:38:59.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:58 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:58.874+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:38:59.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:58 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:58.874+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:38:59.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:58 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:58.874+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:38:59.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:58 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:58.874+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:38:59.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:58 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:58.874+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:39:00.045 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:59 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:59.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:00.046 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:59 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:59.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:00.046 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:59 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:59.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:00.046 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:38:59 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:38:59.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:00.047 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:59 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:59.757+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:39:00.047 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:59 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:59.757+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:39:00.047 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:59 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:59.757+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:39:00.047 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:59 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:59.757+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.951898+0000 front 2023-09-10T01:38:24.952074+0000 (oldest deadline 2023-09-10T01:38:49.651938+0000) 2023-09-10T01:39:00.048 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:38:59 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:38:59.757+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:29.652611+0000 front 2023-09-10T01:38:29.652576+0000 (oldest deadline 2023-09-10T01:38:55.552676+0000) 2023-09-10T01:39:00.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:59 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:59.621+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:00.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:59 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:59.621+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:00.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:59 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:59.621+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:00.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:59 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:59.621+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:00.108 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:38:59 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:38:59.621+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:00.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:59 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:59.868+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:39:00.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:59 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:59.868+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:39:00.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:59 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:59.868+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:39:00.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:59 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:59.868+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:39:00.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:38:59 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:38:59.868+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:39:01.045 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:00 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:00.953+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:01.046 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:00 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:00.953+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:01.046 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:00 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:00.953+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:01.046 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:00 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:00.953+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:01.046 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:00 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:00.719+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:39:01.047 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:00 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:00.719+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:39:01.047 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:00 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:00.719+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:39:01.047 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:00 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:00.719+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.951898+0000 front 2023-09-10T01:38:24.952074+0000 (oldest deadline 2023-09-10T01:38:49.651938+0000) 2023-09-10T01:39:01.047 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:00 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:00.719+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:29.652611+0000 front 2023-09-10T01:38:29.652576+0000 (oldest deadline 2023-09-10T01:38:55.552676+0000) 2023-09-10T01:39:01.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:00 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:00.658+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:01.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:00 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:00.658+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:01.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:00 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:00.658+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:01.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:00 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:00.658+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:01.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:00 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:00.658+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:01.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:00.834+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:39:01.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:00.834+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:39:01.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:00.834+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:39:01.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:00.834+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:39:01.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:00 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:00.834+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:39:01.994 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:01 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:01.675+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:39:01.994 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:01 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:01.675+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:39:01.994 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:01 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:01.675+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:39:01.994 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:01 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:01.675+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.951898+0000 front 2023-09-10T01:38:24.952074+0000 (oldest deadline 2023-09-10T01:38:49.651938+0000) 2023-09-10T01:39:01.995 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:01 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:01.675+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:29.652611+0000 front 2023-09-10T01:38:29.652576+0000 (oldest deadline 2023-09-10T01:38:55.552676+0000) 2023-09-10T01:39:02.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:01 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:01.695+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:02.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:01 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:01.695+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:02.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:01 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:01.695+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:02.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:01 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:01.695+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:02.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:01 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:01.695+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:02.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:01 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:01.993+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:02.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:01 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:01.993+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:02.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:01 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:01.993+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:02.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:01 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:01.993+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:02.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:01 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:01.822+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:39:02.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:01 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:01.822+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:39:02.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:01 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:01.822+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:39:02.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:01 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:01.822+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:39:02.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:01 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:01.822+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:39:03.002 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:02 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:02.663+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:39:03.003 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:02 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:02.663+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:39:03.003 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:02 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:02.663+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:39:03.003 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:02 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:02.663+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.951898+0000 front 2023-09-10T01:38:24.952074+0000 (oldest deadline 2023-09-10T01:38:49.651938+0000) 2023-09-10T01:39:03.003 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:02 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:02.663+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:29.652611+0000 front 2023-09-10T01:38:29.652576+0000 (oldest deadline 2023-09-10T01:38:55.552676+0000) 2023-09-10T01:39:03.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:02 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:02.691+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:03.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:02 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:02.691+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:03.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:02 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:02.691+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:03.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:02 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:02.691+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:03.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:02 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:02.691+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:03.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:03.002+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:03.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:03.002+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:03.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:03.002+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:03.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:03.002+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:03.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:03.002+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:03.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:02 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:02.831+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:39:03.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:02 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:02.831+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:39:03.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:02 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:02.831+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:39:03.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:02 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:02.831+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:39:03.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:02 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:02.831+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:39:04.043 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:03.655+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:37:58.745591+0000 front 2023-09-10T01:37:58.745567+0000 (oldest deadline 2023-09-10T01:38:24.645821+0000) 2023-09-10T01:39:04.043 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:03.655+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:08.747349+0000 front 2023-09-10T01:38:08.747414+0000 (oldest deadline 2023-09-10T01:38:31.047421+0000) 2023-09-10T01:39:04.043 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:03.655+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.249463+0000 front 2023-09-10T01:38:16.249490+0000 (oldest deadline 2023-09-10T01:38:41.549689+0000) 2023-09-10T01:39:04.043 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:03.655+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:24.951898+0000 front 2023-09-10T01:38:24.952074+0000 (oldest deadline 2023-09-10T01:38:49.651938+0000) 2023-09-10T01:39:04.044 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:03.655+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:29.652611+0000 front 2023-09-10T01:38:29.652576+0000 (oldest deadline 2023-09-10T01:38:55.552676+0000) 2023-09-10T01:39:04.044 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:03 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8[126246]: 2023-09-10T01:39:03.655+0000 7ff3943b8700 -1 osd.8 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.754512+0000 front 2023-09-10T01:38:40.754450+0000 (oldest deadline 2023-09-10T01:39:03.054691+0000) 2023-09-10T01:39:04.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:03 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:03.669+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:04.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:03 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:03.669+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:04.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:03 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:03.669+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:04.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:03 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:03.669+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:04.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:03 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:03.669+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:04.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:04.041+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:04.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:04.041+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:04.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:04.041+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:04.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:04.041+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:04.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:04 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:04.041+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:04.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:03.878+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:39:04.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:03.878+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:39:04.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:03.878+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:39:04.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:03.878+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:39:04.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:03 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:03.878+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:39:05.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:04 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:04.657+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:05.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:04 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:04.657+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:05.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:04 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:04.657+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:05.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:04 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:04.657+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:05.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:04 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:04.657+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:05.295 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 10 01:39:05 smithi131 bash[161598]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-8 2023-09-10T01:39:05.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:05 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:05.036+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:05.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:05 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:05.036+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:05.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:05 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:05.036+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:05.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:05 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:05.036+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:05.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:05 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:05.036+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:05.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:04.835+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:39:05.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:04.835+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:39:05.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:04.835+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:39:05.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:04.835+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:39:05.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:04 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:04.835+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:39:06.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:05 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:05.679+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:06.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:05 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:05.679+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:06.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:05 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:05.679+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:06.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:05 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:05.679+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:06.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:05 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:05.679+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:06.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:05 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:05.824+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:39:06.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:05 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:05.824+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:39:06.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:05 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:05.824+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:39:06.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:05 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:05.824+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:39:06.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:05 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:05.824+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:39:06.500 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:06 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:06.057+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:06.500 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:06 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:06.057+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:06.501 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:06 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:06.057+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:06.501 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:06 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:06.057+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:06.501 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:06 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:06.057+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:06.508 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.8.service' 2023-09-10T01:39:06.549 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:39:06.550 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-09-10T01:39:06.550 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-09-10T01:39:06.551 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.9 2023-09-10T01:39:06.862 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:06 smithi044 systemd[1]: Stopping Ceph osd.9 for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:39:07.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:06 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:06.635+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:07.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:06 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:06.635+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:07.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:06 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:06.635+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:07.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:06 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:06.635+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:07.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:06 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:06.635+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:07.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:06 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:06.635+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:37.381546+0000 front 2023-09-10T01:38:37.381418+0000 (oldest deadline 2023-09-10T01:39:06.181768+0000) 2023-09-10T01:39:07.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:07 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:07.016+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:07.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:07 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:07.016+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:07.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:07 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:07.016+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:07.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:07 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:07.016+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:07.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:07 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:07.016+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:07.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:06 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:06.861+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:39:07.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:06 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:06.861+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:39:07.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:06 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:06.861+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:39:07.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:06 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:06.861+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:39:07.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:06 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:06.861+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:39:07.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:06 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:06.861+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.785264+0000 front 2023-09-10T01:38:40.785165+0000 (oldest deadline 2023-09-10T01:39:06.684992+0000) 2023-09-10T01:39:07.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:06 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:06.910+0000 7f77fb3ff700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:39:07.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:06 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:06.910+0000 7f77fb3ff700 -1 osd.9 213 *** Got signal Terminated *** 2023-09-10T01:39:07.307 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:06 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:06.910+0000 7f77fb3ff700 -1 osd.9 213 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-10T01:39:08.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:07 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:07.613+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:08.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:07 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:07.613+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:08.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:07 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:07.613+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:08.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:07 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:07.613+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:08.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:07 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:07.613+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:08.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:07 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:07.613+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:37.381546+0000 front 2023-09-10T01:38:37.381418+0000 (oldest deadline 2023-09-10T01:39:06.181768+0000) 2023-09-10T01:39:08.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:07 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:07.979+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:08.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:07 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:07.979+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:08.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:07 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:07.979+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:08.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:07 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:07.979+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:08.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:07 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:07.979+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:08.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:07 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:07.819+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:39:08.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:07 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:07.819+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:39:08.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:07 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:07.819+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:39:08.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:07 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:07.819+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:39:08.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:07 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:07.819+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:39:08.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:07 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:07.819+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.785264+0000 front 2023-09-10T01:38:40.785165+0000 (oldest deadline 2023-09-10T01:39:06.684992+0000) 2023-09-10T01:39:08.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:08 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:08.591+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:08.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:08 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:08.591+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:08.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:08 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:08.591+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:08.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:08 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:08.591+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:08.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:08 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:08.591+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:08.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:08 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:08.591+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:37.381546+0000 front 2023-09-10T01:38:37.381418+0000 (oldest deadline 2023-09-10T01:39:06.181768+0000) 2023-09-10T01:39:09.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:08 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:08.958+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:09.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:08 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:08.958+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:09.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:08 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:08.958+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:09.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:08 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:08.958+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:09.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:08 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:08.958+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:09.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:08 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:08.830+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:39:09.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:08 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:08.830+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:39:09.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:08 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:08.830+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:39:09.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:08 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:08.830+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:39:09.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:08 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:08.830+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:39:09.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:08 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:08.830+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.785264+0000 front 2023-09-10T01:38:40.785165+0000 (oldest deadline 2023-09-10T01:39:06.684992+0000) 2023-09-10T01:39:09.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:09 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:09.581+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:09.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:09 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:09.581+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:09.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:09 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:09.581+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:09.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:09 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:09.581+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:09.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:09 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:09.581+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:09.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:09 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:09.581+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:37.381546+0000 front 2023-09-10T01:38:37.381418+0000 (oldest deadline 2023-09-10T01:39:06.181768+0000) 2023-09-10T01:39:10.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:09 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:09.993+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:10.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:09 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:09.993+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:10.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:09 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:09.993+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:10.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:09 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:09.993+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:10.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:09 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:09.993+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:10.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:09 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:09.874+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:39:10.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:09 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:09.874+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:39:10.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:09 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:09.874+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:39:10.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:09 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:09.874+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:39:10.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:09 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:09.874+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:39:10.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:09 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:09.874+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.785264+0000 front 2023-09-10T01:38:40.785165+0000 (oldest deadline 2023-09-10T01:39:06.684992+0000) 2023-09-10T01:39:10.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:10 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:10.535+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:10.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:10 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:10.535+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:10.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:10 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:10.535+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:10.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:10 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:10.535+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:10.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:10 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:10.535+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:10.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:10 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:10.535+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:37.381546+0000 front 2023-09-10T01:38:37.381418+0000 (oldest deadline 2023-09-10T01:39:06.181768+0000) 2023-09-10T01:39:11.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:11 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:11.021+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:11.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:11 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:11.021+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:11.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:11 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:11.021+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:11.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:11 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:11.021+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:11.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:11 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:11.021+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:11.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:10 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:10.846+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:39:11.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:10 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:10.846+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:39:11.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:10 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:10.846+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:39:11.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:10 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:10.846+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:39:11.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:10 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:10.846+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:39:11.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:10 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:10.846+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.785264+0000 front 2023-09-10T01:38:40.785165+0000 (oldest deadline 2023-09-10T01:39:06.684992+0000) 2023-09-10T01:39:11.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:11 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:11.551+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:11.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:11 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:11.551+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:11.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:11 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:11.551+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:11.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:11 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:11.551+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:11.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:11 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:11.551+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:11.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:11 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:11.551+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:37.381546+0000 front 2023-09-10T01:38:37.381418+0000 (oldest deadline 2023-09-10T01:39:06.181768+0000) 2023-09-10T01:39:12.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:12 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:12.009+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:12.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:12 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:12.009+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:12.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:12 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:12.009+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:12.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:12 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:12.009+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:12.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:12 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:12.009+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:12.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:11 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:11.875+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:01.275944+0000 front 2023-09-10T01:38:01.275917+0000 (oldest deadline 2023-09-10T01:38:22.976034+0000) 2023-09-10T01:39:12.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:11 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:11.875+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:04.077373+0000 front 2023-09-10T01:38:04.077232+0000 (oldest deadline 2023-09-10T01:38:35.878128+0000) 2023-09-10T01:39:12.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:11 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:11.875+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.379695+0000 front 2023-09-10T01:38:16.379623+0000 (oldest deadline 2023-09-10T01:38:42.279813+0000) 2023-09-10T01:39:12.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:11 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:11.875+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:22.280308+0000 front 2023-09-10T01:38:22.280369+0000 (oldest deadline 2023-09-10T01:38:46.980576+0000) 2023-09-10T01:39:12.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:11 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:11.875+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:30.382750+0000 front 2023-09-10T01:38:30.382909+0000 (oldest deadline 2023-09-10T01:38:55.082741+0000) 2023-09-10T01:39:12.306 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:11 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9[147214]: 2023-09-10T01:39:11.875+0000 7f77f39ed700 -1 osd.9 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.785264+0000 front 2023-09-10T01:38:40.785165+0000 (oldest deadline 2023-09-10T01:39:06.684992+0000) 2023-09-10T01:39:12.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:12 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:12.564+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:12.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:12 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:12.564+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:12.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:12 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:12.564+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:12.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:12 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:12.564+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:12.858 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:12 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:12.564+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:12.858 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:12 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:12.564+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:37.381546+0000 front 2023-09-10T01:38:37.381418+0000 (oldest deadline 2023-09-10T01:39:06.181768+0000) 2023-09-10T01:39:13.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:12 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:12.978+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:13.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:12 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:12.978+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:13.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:12 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:12.978+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:13.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:12 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:12.978+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:13.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:12 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:12.978+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:13.305 INFO:journalctl@ceph.osd.9.smithi044.stdout:Sep 10 01:39:13 smithi044 bash[240538]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-9 2023-09-10T01:39:13.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:13 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:13.567+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:13.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:13 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:13.567+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:13.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:13 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:13.567+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:13.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:13 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:13.567+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:13.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:13 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:13.567+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:13.858 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:13 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:13.567+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:37.381546+0000 front 2023-09-10T01:38:37.381418+0000 (oldest deadline 2023-09-10T01:39:06.181768+0000) 2023-09-10T01:39:13.858 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:13 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:13.567+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:47.883172+0000 front 2023-09-10T01:38:47.883120+0000 (oldest deadline 2023-09-10T01:39:13.083569+0000) 2023-09-10T01:39:14.265 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.9.service' 2023-09-10T01:39:14.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:13 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:13.980+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:14.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:13 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:13.980+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:14.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:13 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:13.980+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:14.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:13 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:13.980+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:14.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:13 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:13.980+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:14.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:13 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:13.980+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:48.711294+0000 front 2023-09-10T01:38:48.711304+0000 (oldest deadline 2023-09-10T01:39:13.411180+0000) 2023-09-10T01:39:14.306 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:39:14.307 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-09-10T01:39:14.307 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-09-10T01:39:14.307 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.10 2023-09-10T01:39:14.606 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:14 smithi099 systemd[1]: Stopping Ceph osd.10 for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:39:14.606 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:14 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:14.525+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:14.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:14 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:14.525+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:14.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:14 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:14.525+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:14.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:14 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:14.525+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:14.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:14 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:14.525+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:14.607 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:14 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:14.525+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:37.381546+0000 front 2023-09-10T01:38:37.381418+0000 (oldest deadline 2023-09-10T01:39:06.181768+0000) 2023-09-10T01:39:14.608 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:14 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:14.525+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:47.883172+0000 front 2023-09-10T01:38:47.883120+0000 (oldest deadline 2023-09-10T01:39:13.083569+0000) 2023-09-10T01:39:15.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:14 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:14.649+0000 7f40b8d68700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:39:15.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:14 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:14.649+0000 7f40b8d68700 -1 osd.10 213 *** Got signal Terminated *** 2023-09-10T01:39:15.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:14 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:14.649+0000 7f40b8d68700 -1 osd.10 213 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-10T01:39:15.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:14 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:14.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:15.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:14 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:14.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:15.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:14 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:14.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:15.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:14 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:14.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:15.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:14 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:14.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:15.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:14 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:14.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:48.711294+0000 front 2023-09-10T01:38:48.711304+0000 (oldest deadline 2023-09-10T01:39:13.411180+0000) 2023-09-10T01:39:15.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:15 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:15.533+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:15.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:15 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:15.533+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:15.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:15 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:15.533+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:15.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:15 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:15.533+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:15.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:15 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:15.533+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:15.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:15 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:15.533+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:37.381546+0000 front 2023-09-10T01:38:37.381418+0000 (oldest deadline 2023-09-10T01:39:06.181768+0000) 2023-09-10T01:39:15.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:15 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:15.533+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:47.883172+0000 front 2023-09-10T01:38:47.883120+0000 (oldest deadline 2023-09-10T01:39:13.083569+0000) 2023-09-10T01:39:16.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:15 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:15.950+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:16.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:15 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:15.950+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:16.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:15 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:15.950+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:16.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:15 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:15.950+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:16.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:15 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:15.950+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:16.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:15 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:15.950+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:48.711294+0000 front 2023-09-10T01:38:48.711304+0000 (oldest deadline 2023-09-10T01:39:13.411180+0000) 2023-09-10T01:39:16.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:16 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:16.570+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:16.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:16 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:16.570+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:16.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:16 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:16.570+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:16.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:16 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:16.570+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:16.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:16 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:16.570+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:16.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:16 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:16.570+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:37.381546+0000 front 2023-09-10T01:38:37.381418+0000 (oldest deadline 2023-09-10T01:39:06.181768+0000) 2023-09-10T01:39:16.857 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:16 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:16.570+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:47.883172+0000 front 2023-09-10T01:38:47.883120+0000 (oldest deadline 2023-09-10T01:39:13.083569+0000) 2023-09-10T01:39:17.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:16 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:16.935+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:17.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:16 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:16.935+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:17.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:16 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:16.935+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:17.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:16 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:16.935+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:17.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:16 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:16.935+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:17.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:16 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:16.935+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:48.711294+0000 front 2023-09-10T01:38:48.711304+0000 (oldest deadline 2023-09-10T01:39:13.411180+0000) 2023-09-10T01:39:18.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:17 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:17.614+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:18.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:17 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:17.614+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:18.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:17 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:17.614+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:18.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:17 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:17.614+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:18.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:17 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:17.614+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:18.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:17 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:17.614+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:37.381546+0000 front 2023-09-10T01:38:37.381418+0000 (oldest deadline 2023-09-10T01:39:06.181768+0000) 2023-09-10T01:39:18.108 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:17 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:17.614+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:47.883172+0000 front 2023-09-10T01:38:47.883120+0000 (oldest deadline 2023-09-10T01:39:13.083569+0000) 2023-09-10T01:39:18.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:17 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:17.914+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:18.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:17 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:17.914+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:18.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:17 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:17.914+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:18.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:17 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:17.914+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:18.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:17 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:17.914+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:18.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:17 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:17.914+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:48.711294+0000 front 2023-09-10T01:38:48.711304+0000 (oldest deadline 2023-09-10T01:39:13.411180+0000) 2023-09-10T01:39:18.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:17 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:17.914+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-09-10T01:38:53.411788+0000 front 2023-09-10T01:38:53.411753+0000 (oldest deadline 2023-09-10T01:39:17.511793+0000) 2023-09-10T01:39:19.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:18 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:18.616+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.874229+0000 front 2023-09-10T01:38:00.874147+0000 (oldest deadline 2023-09-10T01:38:23.174176+0000) 2023-09-10T01:39:19.106 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:18 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:18.616+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.776029+0000 front 2023-09-10T01:38:07.776054+0000 (oldest deadline 2023-09-10T01:38:34.776779+0000) 2023-09-10T01:39:19.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:18 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:18.616+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:14.777240+0000 front 2023-09-10T01:38:14.777394+0000 (oldest deadline 2023-09-10T01:38:40.677240+0000) 2023-09-10T01:39:19.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:18 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:18.616+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2023-09-10T01:38:25.179637+0000 front 2023-09-10T01:38:25.179421+0000 (oldest deadline 2023-09-10T01:38:48.679417+0000) 2023-09-10T01:39:19.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:18 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:18.616+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.679748+0000 front 2023-09-10T01:38:28.679719+0000 (oldest deadline 2023-09-10T01:38:53.979865+0000) 2023-09-10T01:39:19.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:18 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:18.616+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:37.381546+0000 front 2023-09-10T01:38:37.381418+0000 (oldest deadline 2023-09-10T01:39:06.181768+0000) 2023-09-10T01:39:19.107 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:18 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10[131814]: 2023-09-10T01:39:18.616+0000 7f40b1b57700 -1 osd.10 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:47.883172+0000 front 2023-09-10T01:38:47.883120+0000 (oldest deadline 2023-09-10T01:39:13.083569+0000) 2023-09-10T01:39:19.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:18 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:18.915+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:19.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:18 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:18.915+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:19.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:18 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:18.915+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:19.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:18 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:18.915+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:19.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:18 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:18.915+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:19.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:18 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:18.915+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:48.711294+0000 front 2023-09-10T01:38:48.711304+0000 (oldest deadline 2023-09-10T01:39:13.411180+0000) 2023-09-10T01:39:19.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:18 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:18.915+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-09-10T01:38:53.411788+0000 front 2023-09-10T01:38:53.411753+0000 (oldest deadline 2023-09-10T01:39:17.511793+0000) 2023-09-10T01:39:20.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:19 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:19.880+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:20.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:19 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:19.880+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:20.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:19 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:19.880+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:20.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:19 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:19.880+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:20.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:19 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:19.880+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:20.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:19 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:19.880+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:48.711294+0000 front 2023-09-10T01:38:48.711304+0000 (oldest deadline 2023-09-10T01:39:13.411180+0000) 2023-09-10T01:39:20.297 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:19 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:19.880+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-09-10T01:38:53.411788+0000 front 2023-09-10T01:38:53.411753+0000 (oldest deadline 2023-09-10T01:39:17.511793+0000) 2023-09-10T01:39:20.856 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 10 01:39:20 smithi099 bash[158747]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-10 2023-09-10T01:39:21.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:20 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:20.851+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:21.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:20 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:20.851+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:21.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:20 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:20.851+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:21.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:20 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:20.851+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:21.297 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:20 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:20.851+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:21.297 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:20 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:20.851+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:48.711294+0000 front 2023-09-10T01:38:48.711304+0000 (oldest deadline 2023-09-10T01:39:13.411180+0000) 2023-09-10T01:39:21.298 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:20 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:20.851+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-09-10T01:38:53.411788+0000 front 2023-09-10T01:38:53.411753+0000 (oldest deadline 2023-09-10T01:39:17.511793+0000) 2023-09-10T01:39:21.907 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.10.service' 2023-09-10T01:39:21.947 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:39:21.947 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-09-10T01:39:21.948 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-09-10T01:39:21.948 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.11 2023-09-10T01:39:22.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:21 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:21.843+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:22.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:21 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:21.843+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:22.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:21 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:21.843+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:22.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:21 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:21.843+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:22.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:21 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:21.843+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:22.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:21 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:21.843+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:48.711294+0000 front 2023-09-10T01:38:48.711304+0000 (oldest deadline 2023-09-10T01:39:13.411180+0000) 2023-09-10T01:39:22.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:21 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:21.843+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-09-10T01:38:53.411788+0000 front 2023-09-10T01:38:53.411753+0000 (oldest deadline 2023-09-10T01:39:17.511793+0000) 2023-09-10T01:39:22.297 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:21 smithi131 systemd[1]: Stopping Ceph osd.11 for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:39:22.795 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:22 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:22.309+0000 7faed2c70700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:39:22.795 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:22 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:22.309+0000 7faed2c70700 -1 osd.11 213 *** Got signal Terminated *** 2023-09-10T01:39:22.796 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:22 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:22.309+0000 7faed2c70700 -1 osd.11 213 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-10T01:39:23.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:22 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:22.890+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:23.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:22 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:22.890+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:23.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:22 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:22.890+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:23.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:22 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:22.890+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:23.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:22 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:22.890+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:23.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:22 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:22.890+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:48.711294+0000 front 2023-09-10T01:38:48.711304+0000 (oldest deadline 2023-09-10T01:39:13.411180+0000) 2023-09-10T01:39:23.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:22 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:22.890+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-09-10T01:38:53.411788+0000 front 2023-09-10T01:38:53.411753+0000 (oldest deadline 2023-09-10T01:39:17.511793+0000) 2023-09-10T01:39:24.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:23 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:23.896+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:24.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:23 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:23.896+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:24.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:23 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:23.896+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:24.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:23 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:23.896+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:24.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:23 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:23.896+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:24.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:23 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:23.896+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:48.711294+0000 front 2023-09-10T01:38:48.711304+0000 (oldest deadline 2023-09-10T01:39:13.411180+0000) 2023-09-10T01:39:24.297 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:23 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:23.896+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-09-10T01:38:53.411788+0000 front 2023-09-10T01:38:53.411753+0000 (oldest deadline 2023-09-10T01:39:17.511793+0000) 2023-09-10T01:39:25.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:24 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:24.920+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:25.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:24 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:24.920+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:25.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:24 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:24.920+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:25.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:24 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:24.920+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:25.297 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:24 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:24.920+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:25.297 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:24 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:24.920+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:48.711294+0000 front 2023-09-10T01:38:48.711304+0000 (oldest deadline 2023-09-10T01:39:13.411180+0000) 2023-09-10T01:39:25.297 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:24 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:24.920+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-09-10T01:38:53.411788+0000 front 2023-09-10T01:38:53.411753+0000 (oldest deadline 2023-09-10T01:39:17.511793+0000) 2023-09-10T01:39:26.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:25 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:25.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:26.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:25 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:25.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:26.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:25 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:25.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:26.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:25 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:25.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:26.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:25 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:25.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:26.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:25 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:25.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:48.711294+0000 front 2023-09-10T01:38:48.711304+0000 (oldest deadline 2023-09-10T01:39:13.411180+0000) 2023-09-10T01:39:26.297 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:25 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:25.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-09-10T01:38:53.411788+0000 front 2023-09-10T01:38:53.411753+0000 (oldest deadline 2023-09-10T01:39:17.511793+0000) 2023-09-10T01:39:27.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:26 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:26.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2023-09-10T01:38:00.901395+0000 front 2023-09-10T01:38:00.901334+0000 (oldest deadline 2023-09-10T01:38:24.902335+0000) 2023-09-10T01:39:27.295 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:26 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:26.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-09-10T01:38:07.203771+0000 front 2023-09-10T01:38:07.203693+0000 (oldest deadline 2023-09-10T01:38:31.303842+0000) 2023-09-10T01:39:27.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:26 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:26.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2023-09-10T01:38:16.605095+0000 front 2023-09-10T01:38:16.605123+0000 (oldest deadline 2023-09-10T01:38:38.905283+0000) 2023-09-10T01:39:27.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:26 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:26.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-09-10T01:38:28.307208+0000 front 2023-09-10T01:38:28.307232+0000 (oldest deadline 2023-09-10T01:38:57.108224+0000) 2023-09-10T01:39:27.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:26 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:26.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2023-09-10T01:38:40.009298+0000 front 2023-09-10T01:38:40.009249+0000 (oldest deadline 2023-09-10T01:39:02.909407+0000) 2023-09-10T01:39:27.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:26 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:26.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2023-09-10T01:38:48.711294+0000 front 2023-09-10T01:38:48.711304+0000 (oldest deadline 2023-09-10T01:39:13.411180+0000) 2023-09-10T01:39:27.296 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:26 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11[135057]: 2023-09-10T01:39:26.952+0000 7faecea65700 -1 osd.11 213 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-09-10T01:38:53.411788+0000 front 2023-09-10T01:38:53.411753+0000 (oldest deadline 2023-09-10T01:39:17.511793+0000) 2023-09-10T01:39:28.288 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 10 01:39:28 smithi131 bash[161872]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-osd-11 2023-09-10T01:39:29.549 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@osd.11.service' 2023-09-10T01:39:29.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:39:29.589 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-09-10T01:39:29.589 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-09-10T01:39:29.590 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.a 2023-09-10T01:39:29.942 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 10 01:39:29 smithi044 systemd[1]: Stopping Ceph mds.a for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:39:30.305 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 10 01:39:29 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-a[155089]: 2023-09-10T01:39:29.940+0000 7fe21b13e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:39:30.305 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 10 01:39:29 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-a[155089]: 2023-09-10T01:39:29.940+0000 7fe21b13e700 -1 mds.a *** got signal Terminated *** 2023-09-10T01:39:32.055 INFO:journalctl@ceph.mds.a.smithi044.stdout:Sep 10 01:39:31 smithi044 bash[240810]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-a 2023-09-10T01:39:32.123 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.a.service' 2023-09-10T01:39:32.162 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:39:32.162 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-09-10T01:39:32.163 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-09-10T01:39:32.163 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d 2023-09-10T01:39:32.541 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:39:32 smithi044 systemd[1]: Stopping Ceph mds.d for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:39:32.805 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:39:32 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[198064]: 2023-09-10T01:39:32.539+0000 7fd570bd1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:39:32.805 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:39:32 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d[198064]: 2023-09-10T01:39:32.539+0000 7fd570bd1700 -1 mds.d *** got signal Terminated *** 2023-09-10T01:39:37.055 INFO:journalctl@ceph.mds.d.smithi044.stdout:Sep 10 01:39:36 smithi044 bash[240960]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-d 2023-09-10T01:39:37.172 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.d.service' 2023-09-10T01:39:37.210 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:39:37.211 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-09-10T01:39:37.211 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-09-10T01:39:37.212 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.g 2023-09-10T01:39:37.555 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 10 01:39:37 smithi044 systemd[1]: Stopping Ceph mds.g for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:39:38.055 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 10 01:39:37 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-g[154705]: 2023-09-10T01:39:37.597+0000 7f458330c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:39:38.056 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 10 01:39:37 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-g[154705]: 2023-09-10T01:39:37.597+0000 7f458330c700 -1 mds.g *** got signal Terminated *** 2023-09-10T01:39:41.805 INFO:journalctl@ceph.mds.g.smithi044.stdout:Sep 10 01:39:41 smithi044 bash[241106]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-g 2023-09-10T01:39:41.861 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.g.service' 2023-09-10T01:39:41.897 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:39:41.898 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-09-10T01:39:41.898 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-09-10T01:39:41.899 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.j 2023-09-10T01:39:42.282 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 10 01:39:41 smithi044 systemd[1]: Stopping Ceph mds.j for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:39:42.555 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 10 01:39:42 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-j[153740]: 2023-09-10T01:39:42.281+0000 7f82acc43700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:39:42.555 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 10 01:39:42 smithi044 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-j[153740]: 2023-09-10T01:39:42.281+0000 7f82acc43700 -1 mds.j *** got signal Terminated *** 2023-09-10T01:39:46.305 INFO:journalctl@ceph.mds.j.smithi044.stdout:Sep 10 01:39:46 smithi044 bash[241247]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-j 2023-09-10T01:39:46.435 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.j.service' 2023-09-10T01:39:46.470 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:39:46.471 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-09-10T01:39:46.471 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-09-10T01:39:46.472 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.b 2023-09-10T01:39:46.818 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 10 01:39:46 smithi099 systemd[1]: Stopping Ceph mds.b for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:39:47.106 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 10 01:39:46 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-b[138846]: 2023-09-10T01:39:46.817+0000 7fe447b57700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:39:47.106 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 10 01:39:46 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-b[138846]: 2023-09-10T01:39:46.817+0000 7fe447b57700 -1 mds.b *** got signal Terminated *** 2023-09-10T01:39:51.106 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 10 01:39:50 smithi099 bash[159014]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-b 2023-09-10T01:39:51.192 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.b.service' 2023-09-10T01:39:51.229 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:39:51.230 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-09-10T01:39:51.231 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-09-10T01:39:51.231 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.e 2023-09-10T01:39:51.606 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 10 01:39:51 smithi099 systemd[1]: Stopping Ceph mds.e for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:39:52.106 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 10 01:39:51 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-e[138162]: 2023-09-10T01:39:51.608+0000 7f7334ad0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:39:52.106 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 10 01:39:51 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-e[138162]: 2023-09-10T01:39:51.608+0000 7f7334ad0700 -1 mds.e *** got signal Terminated *** 2023-09-10T01:39:54.960 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.e.service' 2023-09-10T01:39:54.994 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:39:54.994 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-09-10T01:39:54.995 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-09-10T01:39:54.995 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.h 2023-09-10T01:39:55.356 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 10 01:39:55 smithi099 systemd[1]: Stopping Ceph mds.h for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:39:55.856 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 10 01:39:55 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-h[138497]: 2023-09-10T01:39:55.365+0000 7f04af0a7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:39:55.856 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 10 01:39:55 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-h[138497]: 2023-09-10T01:39:55.365+0000 7f04af0a7700 -1 mds.h *** got signal Terminated *** 2023-09-10T01:40:01.154 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 10 01:40:01 smithi099 bash[159292]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-h 2023-09-10T01:40:01.340 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.h.service' 2023-09-10T01:40:01.376 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:40:01.377 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-09-10T01:40:01.377 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-09-10T01:40:01.378 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.k 2023-09-10T01:40:01.608 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 10 01:40:01 smithi099 systemd[1]: Stopping Ceph mds.k for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:40:02.106 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 10 01:40:01 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-k[137821]: 2023-09-10T01:40:01.748+0000 7f7907324700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:40:02.106 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 10 01:40:01 smithi099 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-k[137821]: 2023-09-10T01:40:01.748+0000 7f7907324700 -1 mds.k *** got signal Terminated *** 2023-09-10T01:40:06.432 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 10 01:40:06 smithi099 bash[159436]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-k 2023-09-10T01:40:06.642 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.k.service' 2023-09-10T01:40:06.675 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:40:06.676 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-09-10T01:40:06.676 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-09-10T01:40:06.677 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c 2023-09-10T01:40:06.721 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.c.service' 2023-09-10T01:40:06.799 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:40:06.799 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-09-10T01:40:06.800 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-09-10T01:40:06.800 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.f 2023-09-10T01:40:07.045 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 10 01:40:06 smithi131 systemd[1]: Stopping Ceph mds.f for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:40:07.545 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 10 01:40:07 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-f[140782]: 2023-09-10T01:40:07.185+0000 7f741ddf9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:40:07.545 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 10 01:40:07 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-f[140782]: 2023-09-10T01:40:07.185+0000 7f741ddf9700 -1 mds.f *** got signal Terminated *** 2023-09-10T01:40:08.841 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 10 01:40:08 smithi131 bash[162182]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-f 2023-09-10T01:40:09.031 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.f.service' 2023-09-10T01:40:09.067 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:40:09.068 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-09-10T01:40:09.068 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-09-10T01:40:09.069 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.i 2023-09-10T01:40:09.295 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 10 01:40:09 smithi131 systemd[1]: Stopping Ceph mds.i for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:40:09.795 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 10 01:40:09 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-i[141462]: 2023-09-10T01:40:09.451+0000 7f3b916b7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:40:09.795 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 10 01:40:09 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-i[141462]: 2023-09-10T01:40:09.451+0000 7f3b916b7700 -1 mds.i *** got signal Terminated *** 2023-09-10T01:40:12.512 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 10 01:40:12 smithi131 bash[162322]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-i 2023-09-10T01:40:12.701 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.i.service' 2023-09-10T01:40:12.736 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:40:12.737 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-09-10T01:40:12.737 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-09-10T01:40:12.737 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.l 2023-09-10T01:40:13.045 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 10 01:40:12 smithi131 systemd[1]: Stopping Ceph mds.l for 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da... 2023-09-10T01:40:13.545 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 10 01:40:13 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-l[140438]: 2023-09-10T01:40:13.116+0000 7f397b491700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-10T01:40:13.545 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 10 01:40:13 smithi131 ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-l[140438]: 2023-09-10T01:40:13.116+0000 7f397b491700 -1 mds.l *** got signal Terminated *** 2023-09-10T01:40:19.295 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 10 01:40:19 smithi131 bash[162462]: ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da-mds-l 2023-09-10T01:40:19.503 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da@mds.l.service' 2023-09-10T01:40:19.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-10T01:40:19.539 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-09-10T01:40:19.540 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da --force --keep-logs 2023-09-10T01:40:19.855 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T01:40:21.747 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da --force --keep-logs 2023-09-10T01:40:22.047 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T01:40:23.647 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da --force --keep-logs 2023-09-10T01:40:23.953 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T01:40:25.648 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-10T01:40:25.687 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-10T01:40:25.724 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-10T01:40:25.753 INFO:tasks.cephadm:Archiving crash dumps... 2023-09-10T01:40:25.756 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/crash to /home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526/remote/smithi044/crash 2023-09-10T01:40:25.757 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/crash -- . 2023-09-10T01:40:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:tar: /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/crash: Cannot open: No such file or directory 2023-09-10T01:40:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:tar: Error is not recoverable: exiting now 2023-09-10T01:40:25.814 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/crash to /home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526/remote/smithi099/crash 2023-09-10T01:40:25.815 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/crash -- . 2023-09-10T01:40:25.871 INFO:teuthology.orchestra.run.smithi099.stderr:tar: /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/crash: Cannot open: No such file or directory 2023-09-10T01:40:25.871 INFO:teuthology.orchestra.run.smithi099.stderr:tar: Error is not recoverable: exiting now 2023-09-10T01:40:25.873 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/crash to /home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526/remote/smithi131/crash 2023-09-10T01:40:25.874 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/crash -- . 2023-09-10T01:40:25.919 INFO:teuthology.orchestra.run.smithi131.stderr:tar: /var/lib/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/crash: Cannot open: No such file or directory 2023-09-10T01:40:25.919 INFO:teuthology.orchestra.run.smithi131.stderr:tar: Error is not recoverable: exiting now 2023-09-10T01:40:25.920 INFO:tasks.cephadm:Checking cluster log for badness... 2023-09-10T01:40:25.920 DEBUG:teuthology.orchestra.run.smithi044:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-09-10T01:40:25.957 INFO:teuthology.orchestra.run.smithi044.stderr:grep: /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph.log: No such file or directory 2023-09-10T01:40:25.962 INFO:tasks.cephadm:Compressing logs... 2023-09-10T01:40:25.963 DEBUG:teuthology.orchestra.run.smithi044:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-10T01:40:26.005 DEBUG:teuthology.orchestra.run.smithi099:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-10T01:40:26.007 DEBUG:teuthology.orchestra.run.smithi131:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-10T01:40:26.053 INFO:teuthology.orchestra.run.smithi044.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-10T01:40:26.053 INFO:teuthology.orchestra.run.smithi044.stderr:: No such file or directory 2023-09-10T01:40:26.054 INFO:teuthology.orchestra.run.smithi099.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-10T01:40:26.054 INFO:teuthology.orchestra.run.smithi099.stderr:: No such file or directory 2023-09-10T01:40:26.058 INFO:teuthology.orchestra.run.smithi131.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-10T01:40:26.058 INFO:teuthology.orchestra.run.smithi131.stderr:: No such file or directory 2023-09-10T01:40:26.061 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.10.log 2023-09-10T01:40:26.061 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.h.log 2023-09-10T01:40:26.061 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.e.log 2023-09-10T01:40:26.064 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.4.log 2023-09-10T01:40:26.064 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.1.log 2023-09-10T01:40:26.064 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.k.log 2023-09-10T01:40:26.064 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-volume.log 2023-09-10T01:40:26.065 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.7.log 2023-09-10T01:40:26.065 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mon.b.log 2023-09-10T01:40:26.065 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.b.log 2023-09-10T01:40:26.065 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mgr.y.log 2023-09-10T01:40:26.066 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.0.log 2023-09-10T01:40:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.g.log 2023-09-10T01:40:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.a.log 2023-09-10T01:40:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.6.log 2023-09-10T01:40:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.3.log 2023-09-10T01:40:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.d.log 2023-09-10T01:40:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mgr.x.log 2023-09-10T01:40:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.9.log 2023-09-10T01:40:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-volume.log 2023-09-10T01:40:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.j.log 2023-09-10T01:40:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mon.a.log 2023-09-10T01:40:26.071 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-09-10T01:40:26.071 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2023-09-10T01:40:26.075 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.b.log: /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mgr.y.log: 93.3% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mgr.y.log.gz 2023-09-10T01:40:26.078 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.5.log 2023-09-10T01:40:26.078 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.2.log 2023-09-10T01:40:26.078 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.11.log 2023-09-10T01:40:26.078 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mgr.z.log 2023-09-10T01:40:26.079 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.8.log 2023-09-10T01:40:26.079 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.f.log 2023-09-10T01:40:26.079 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.c.log 2023-09-10T01:40:26.079 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.i.log 2023-09-10T01:40:26.079 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mon.c.log 2023-09-10T01:40:26.079 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-volume.log 2023-09-10T01:40:26.080 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.i.log: /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.l.log 2023-09-10T01:40:26.121 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.log: /var/log/ceph/ceph-client.0.log: 94.6% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.j.log.gz 2023-09-10T01:40:26.159 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-volume.log: /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.l.log: 93.4% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mgr.z.log.gz 2023-09-10T01:40:26.311 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-volume.log.gz 2023-09-10T01:40:26.369 INFO:teuthology.orchestra.run.smithi044.stderr: 92.8% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-10T01:40:26.474 INFO:teuthology.orchestra.run.smithi044.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-09-10T01:40:26.502 INFO:teuthology.orchestra.run.smithi099.stderr: 91.5% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-10T01:40:26.595 INFO:teuthology.orchestra.run.smithi044.stderr: 91.1% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-volume.log.gz 2023-09-10T01:40:26.741 INFO:teuthology.orchestra.run.smithi131.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-10T01:40:26.976 INFO:teuthology.orchestra.run.smithi099.stderr: 91.0% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-volume.log.gz 2023-09-10T01:40:29.299 INFO:teuthology.orchestra.run.smithi131.stderr: 94.0% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.c.log.gz 2023-09-10T01:40:29.372 INFO:teuthology.orchestra.run.smithi044.stderr: 89.9% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mgr.x.log.gz 2023-09-10T01:40:29.427 INFO:teuthology.orchestra.run.smithi044.stderr: 92.8% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.d.log.gz 2023-09-10T01:40:29.441 INFO:teuthology.orchestra.run.smithi044.stderr: 94.0% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.g.log.gz 2023-09-10T01:40:30.019 INFO:teuthology.orchestra.run.smithi044.stderr: 94.0% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.a.log.gz 2023-09-10T01:40:31.432 INFO:teuthology.orchestra.run.smithi131.stderr: 94.1% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.f.log.gz 2023-09-10T01:40:32.480 INFO:teuthology.orchestra.run.smithi099.stderr: 91.8% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mon.b.log.gz 2023-09-10T01:40:34.265 INFO:teuthology.orchestra.run.smithi099.stderr: 95.5% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.e.log.gz 2023-09-10T01:40:38.081 INFO:teuthology.orchestra.run.smithi131.stderr: 91.7% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mon.c.log.gz 2023-09-10T01:40:40.104 INFO:teuthology.orchestra.run.smithi044.stderr: 90.3% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mon.a.log.gz 2023-09-10T01:42:46.344 INFO:teuthology.orchestra.run.smithi099.stderr: 95.8% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.b.log.gz 2023-09-10T01:48:39.386 INFO:teuthology.orchestra.run.smithi099.stderr: 91.5% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.10.log.gz 2023-09-10T01:50:31.038 INFO:teuthology.orchestra.run.smithi131.stderr: 91.4% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.8.log.gz 2023-09-10T01:50:43.360 INFO:teuthology.orchestra.run.smithi044.stderr: 91.4% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.0.log.gz 2023-09-10T01:50:45.082 INFO:teuthology.orchestra.run.smithi099.stderr: 97.5% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.k.log.gz 2023-09-10T01:50:56.540 INFO:teuthology.orchestra.run.smithi099.stderr: 91.4% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.1.log.gz 2023-09-10T01:50:58.258 INFO:teuthology.orchestra.run.smithi099.stderr: 97.4% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.h.log.gz 2023-09-10T01:51:26.759 INFO:teuthology.orchestra.run.smithi131.stderr: 91.4% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.11.log.gz 2023-09-10T01:51:28.866 INFO:teuthology.orchestra.run.smithi044.stderr: 91.3% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.3.log.gz 2023-09-10T01:51:52.769 INFO:teuthology.orchestra.run.smithi099.stderr: 91.5% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.7.log.gz 2023-09-10T01:51:55.392 INFO:teuthology.orchestra.run.smithi044.stderr: 91.4% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.6.log.gz 2023-09-10T01:52:09.230 INFO:teuthology.orchestra.run.smithi044.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2023-09-10T01:52:49.262 INFO:teuthology.orchestra.run.smithi099.stderr: 91.3% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.4.log.gz 2023-09-10T01:52:49.275 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-09-10T01:52:49.275 INFO:teuthology.orchestra.run.smithi099.stderr:real 12m23.252s 2023-09-10T01:52:49.276 INFO:teuthology.orchestra.run.smithi099.stderr:user 14m25.082s 2023-09-10T01:52:49.276 INFO:teuthology.orchestra.run.smithi099.stderr:sys 0m35.132s 2023-09-10T01:53:05.086 INFO:teuthology.orchestra.run.smithi131.stderr: 91.3% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.2.log.gz 2023-09-10T01:53:09.411 INFO:teuthology.orchestra.run.smithi044.stderr: 91.4% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.9.log.gz 2023-09-10T01:53:09.414 INFO:teuthology.orchestra.run.smithi044.stderr: 2023-09-10T01:53:09.414 INFO:teuthology.orchestra.run.smithi044.stderr:real 12m43.393s 2023-09-10T01:53:09.414 INFO:teuthology.orchestra.run.smithi044.stderr:user 17m8.995s 2023-09-10T01:53:09.415 INFO:teuthology.orchestra.run.smithi044.stderr:sys 0m36.246s 2023-09-10T01:54:55.037 INFO:teuthology.orchestra.run.smithi131.stderr: 98.3% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.i.log.gz 2023-09-10T01:55:12.504 INFO:teuthology.orchestra.run.smithi131.stderr: 97.8% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-mds.l.log.gz 2023-09-10T01:56:32.204 INFO:teuthology.orchestra.run.smithi131.stderr: 91.3% -- replaced with /var/log/ceph/3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da/ceph-osd.5.log.gz 2023-09-10T01:56:32.206 INFO:teuthology.orchestra.run.smithi131.stderr: 2023-09-10T01:56:32.206 INFO:teuthology.orchestra.run.smithi131.stderr:real 16m6.183s 2023-09-10T01:56:32.207 INFO:teuthology.orchestra.run.smithi131.stderr:user 17m15.492s 2023-09-10T01:56:32.207 INFO:teuthology.orchestra.run.smithi131.stderr:sys 0m44.290s 2023-09-10T01:56:32.207 INFO:tasks.cephadm:Archiving logs... 2023-09-10T01:56:32.209 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526/remote/smithi044/log 2023-09-10T01:56:32.210 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-10T01:59:10.243 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526/remote/smithi099/log 2023-09-10T01:59:10.244 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-10T02:01:20.136 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526/remote/smithi131/log 2023-09-10T02:01:20.138 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-10T02:03:50.305 INFO:tasks.cephadm:Removing cluster... 2023-09-10T02:03:50.306 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da --force 2023-09-10T02:03:50.747 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T02:03:52.097 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da --force 2023-09-10T02:03:52.724 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T02:03:54.066 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da --force 2023-09-10T02:03:54.635 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: 3d8afcc6-4f71-11ee-9ab7-7b867c8bd7da 2023-09-10T02:03:55.956 INFO:tasks.cephadm:Removing cephadm ... 2023-09-10T02:03:55.956 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-10T02:03:55.979 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-10T02:03:56.071 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-10T02:03:56.094 INFO:tasks.cephadm:Teardown complete 2023-09-10T02:03:56.095 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-10T02:03:56.106 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-09-10T02:03:56.107 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-10T02:03:56.107 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-10T02:03:56.111 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-10T02:03:56.113 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-10T02:03:56.253 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-10T02:03:56.265 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-10T02:03:56.265 DEBUG:teuthology.orchestra.run.smithi044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-10T02:03:56.268 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-10T02:03:56.271 DEBUG:teuthology.orchestra.run.smithi131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-10T02:03:56.287 INFO:teuthology.orchestra.run.smithi131.stderr:bash: ntpq: command not found 2023-09-10T02:03:56.289 INFO:teuthology.orchestra.run.smithi099.stderr:bash: ntpq: command not found 2023-09-10T02:03:56.296 INFO:teuthology.orchestra.run.smithi044.stderr:bash: ntpq: command not found 2023-09-10T02:03:56.309 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-10T02:03:56.310 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2023-09-10T02:03:56.310 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 13 +101us[ +140us] +/- 96ms 2023-09-10T02:03:56.310 INFO:teuthology.orchestra.run.smithi131.stdout:^* hv02.front.sepia.ceph.com 3 6 377 9 -1091us[-1052us] +/- 60ms 2023-09-10T02:03:56.310 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +1230us[+1269us] +/- 68ms 2023-09-10T02:03:56.310 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-09-10T02:03:56.312 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-10T02:03:56.312 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2023-09-10T02:03:56.312 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 8 +80us[ +80us] +/- 96ms 2023-09-10T02:03:56.312 INFO:teuthology.orchestra.run.smithi044.stdout:^* hv02.front.sepia.ceph.com 3 6 377 9 -1111us[-1108us] +/- 60ms 2023-09-10T02:03:56.313 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +1217us[+1221us] +/- 68ms 2023-09-10T02:03:56.313 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-09-10T02:03:56.316 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-10T02:03:56.317 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2023-09-10T02:03:56.317 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 9 +76us[ +83us] +/- 96ms 2023-09-10T02:03:56.317 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv02.front.sepia.ceph.com 3 6 377 8 -1093us[-1086us] +/- 60ms 2023-09-10T02:03:56.317 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +1226us[+1233us] +/- 68ms 2023-09-10T02:03:56.317 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-09-10T02:03:56.319 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-10T02:03:56.332 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-10T02:03:56.333 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-10T02:03:56.347 DEBUG:teuthology.orchestra.run.smithi044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-10T02:03:56.553 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-10T02:03:56.773 DEBUG:teuthology.orchestra.run.smithi131:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-10T02:03:56.998 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-10T02:03:57.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-10T02:03:57.037 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-10T02:03:57.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-10T02:03:57.075 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-10T02:03:57.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-10T02:03:57.108 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-10T02:03:57.120 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-10T02:03:57.146 INFO:teuthology.task.internal:Duration was 6328.515195 seconds 2023-09-10T02:03:57.147 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-10T02:03:57.187 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-10T02:03:57.188 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-10T02:03:57.191 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-10T02:03:57.194 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-10T02:03:57.249 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-10T02:03:57.256 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-10T02:03:57.290 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-10T02:03:57.570 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-10T02:03:57.571 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi044.front.sepia.ceph.com 2023-09-10T02:03:57.571 DEBUG:teuthology.orchestra.run.smithi044:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-10T02:03:57.601 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2023-09-10T02:03:57.602 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-10T02:03:57.632 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi131.front.sepia.ceph.com 2023-09-10T02:03:57.633 DEBUG:teuthology.orchestra.run.smithi131:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-10T02:03:57.655 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-10T02:03:57.656 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-10T02:03:57.659 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-10T02:03:57.676 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-10T02:03:57.797 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-10T02:03:57.797 DEBUG:teuthology.orchestra.run.smithi044:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-10T02:03:57.800 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-10T02:03:57.803 DEBUG:teuthology.orchestra.run.smithi131:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-10T02:03:57.937 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-10T02:03:57.949 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-10T02:03:57.950 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-10T02:03:57.983 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-10T02:03:58.023 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-10T02:03:58.059 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-10T02:03:58.071 DEBUG:teuthology.orchestra.run.smithi044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-10T02:03:58.073 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-10T02:03:58.076 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-10T02:03:58.101 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2023-09-10T02:03:58.103 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = core 2023-09-10T02:03:58.133 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2023-09-10T02:03:58.158 DEBUG:teuthology.orchestra.run.smithi044:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-10T02:03:58.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-10T02:03:58.189 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-10T02:03:58.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-10T02:03:58.206 DEBUG:teuthology.orchestra.run.smithi131:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-10T02:03:58.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-10T02:03:58.223 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-10T02:03:58.236 INFO:teuthology.task.internal:Transferring archived files... 2023-09-10T02:03:58.237 DEBUG:teuthology.misc:Transferring archived files from smithi044:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526/remote/smithi044 2023-09-10T02:03:58.238 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-10T02:03:58.282 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526/remote/smithi099 2023-09-10T02:03:58.282 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-10T02:03:58.325 DEBUG:teuthology.misc:Transferring archived files from smithi131:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526/remote/smithi131 2023-09-10T02:03:58.325 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-10T02:03:58.368 INFO:teuthology.task.internal:Removing archive directory... 2023-09-10T02:03:58.369 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-10T02:03:58.372 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-10T02:03:58.376 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-10T02:03:58.417 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-10T02:03:58.438 INFO:teuthology.task.internal:Not uploading archives. 2023-09-10T02:03:58.438 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-10T02:03:58.452 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-10T02:03:58.452 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-10T02:03:58.455 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-10T02:03:58.457 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-10T02:03:58.470 INFO:teuthology.orchestra.run.smithi044.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 02:03 /home/ubuntu/cephtest 2023-09-10T02:03:58.471 INFO:teuthology.orchestra.run.smithi099.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 02:03 /home/ubuntu/cephtest 2023-09-10T02:03:58.473 INFO:teuthology.orchestra.run.smithi131.stdout: 262170 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 02:03 /home/ubuntu/cephtest 2023-09-10T02:03:58.481 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-10T02:03:58.493 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-10T02:03:58.642 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-09-10T02:03:58.642 INFO:teuthology.nuke:Checking targets against current locks 2023-09-10T02:03:58.666 DEBUG:teuthology.nuke:shortname: smithi044 2023-09-10T02:03:58.666 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-10T02:03:58.689 DEBUG:teuthology.nuke:shortname: smithi099 2023-09-10T02:03:58.689 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-10T02:03:58.694 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-10 00:13:07.006153', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-10T02:03:58.722 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-10 00:13:07.008015', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-10T02:03:58.723 DEBUG:teuthology.nuke:shortname: smithi131 2023-09-10T02:03:58.724 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-10T02:03:58.755 INFO:teuthology.orchestra.console.smithi044:Power off 2023-09-10T02:03:58.756 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-10T02:03:58.766 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392526', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-10 00:13:07.009325', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-10T02:03:58.779 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off 2023-09-10T02:03:58.780 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-10T02:03:58.797 INFO:teuthology.orchestra.console.smithi099:Power off 2023-09-10T02:03:58.797 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-10T02:03:58.810 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2023-09-10T02:03:58.820 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off 2023-09-10T02:03:58.820 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-10T02:03:58.836 INFO:teuthology.orchestra.console.smithi131:Power off 2023-09-10T02:03:58.836 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-10T02:03:58.847 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-09-10T02:03:58.860 DEBUG:teuthology.orchestra.console.smithi131:power off output: Chassis Power Control: Down/Off 2023-09-10T02:03:58.861 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-10T02:03:58.888 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2023-09-10T02:04:02.811 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-10T02:04:02.849 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-10T02:04:02.889 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-10T02:04:02.921 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2023-09-10T02:04:02.959 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-09-10T02:04:03.001 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2023-09-10T02:04:06.926 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-10T02:04:06.960 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-10T02:04:07.002 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-10T02:04:07.038 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2023-09-10T02:04:07.070 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-09-10T02:04:07.118 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off 2023-09-10T02:04:07.139 INFO:teuthology.orchestra.console.smithi044:Power off completed 2023-09-10T02:04:07.219 INFO:teuthology.orchestra.console.smithi131:Power off completed 2023-09-10T02:04:11.073 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-10T02:04:11.183 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off 2023-09-10T02:04:11.283 INFO:teuthology.orchestra.console.smithi099:Power off completed 2023-09-10T02:04:11.388 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} duration: 6328.5151953697205 failure_reason: 'error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2023-09-10T02:04:11.389 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-10T02:04:11.475 INFO:teuthology.run:FAIL